[INFO|2026-08-30 04:18:53] llamafactory.launcher:144 >> Initializing 2 distributed tasks at: 127.0.0.1:29651

*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/jieba/_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
[INFO|2026-08-30 04:19:01] llamafactory.hparams.parser:523 >> Process rank: 0, world size: 2, device: cuda:0, distributed training: True, compute dtype: torch.float16
[INFO|2026-08-30 04:19:01] llamafactory.hparams.parser:523 >> Process rank: 1, world size: 2, device: cuda:1, distributed training: True, compute dtype: torch.float16
[INFO|configuration_utils.py:765] 2026-08-30 04:19:01,759 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
[INFO|configuration_utils.py:841] 2026-08-30 04:19:01,765 >> Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

[INFO|configuration_utils.py:765] 2026-08-30 04:19:03,521 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
[INFO|configuration_utils.py:841] 2026-08-30 04:19:03,524 >> Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

[INFO|configuration_utils.py:765] 2026-08-30 04:19:03,670 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
[INFO|configuration_utils.py:841] 2026-08-30 04:19:03,673 >> Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

[INFO|2026-08-30 04:19:04] llamafactory.data.loader:144 >> Loading dataset CohereLabs/aya_collection_language_split...
[rank1]:[W830 04:19:04.262110171 ProcessGroupNCCL.cpp:4561] [PG ID 0 PG GUID 0 Rank 1]  using GPU 1 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. Specify device_ids in barrier() to force use of a particular device, or call init_process_group() with a device_id.
[rank0]:[W830 04:19:09.235154451 ProcessGroupNCCL.cpp:4561] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. Specify device_ids in barrier() to force use of a particular device, or call init_process_group() with a device_id.
training example:
input_ids:
[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28965, 28714, 2892, 493, 28747, 550, 28724, 537, 424, 2144, 363, 28873, 299, 4845, 16439, 2594, 28797, 6149, 28862, 884, 264, 14444, 28862, 28827, 1100, 28768, 424, 28725, 9940, 29826, 4798, 5972, 28708, 28862, 485, 3734, 12210, 28717, 380, 6670, 6294, 731, 28725, 1627, 13866, 412, 28725, 10531, 16592, 28718, 28725, 9275, 20524, 28718, 438, 29365, 28723, 10650, 300, 363, 1416, 28709, 28862, 28827, 1100, 28768, 424, 363, 28873, 299, 4845, 20133, 311, 2099, 724, 28827, 28768, 18057, 28723, 3779, 28827, 424, 2144, 446, 28863, 1857, 29114, 2853, 363, 408, 10422, 1189, 268, 27172, 2124, 9335, 2345, 28862, 17512, 28723, 367, 280, 28879, 570, 3270, 446, 1093, 3712, 28862, 3907, 28725, 534, 28724, 2710, 4321, 16449, 1144, 5972, 28708, 28862, 28711, 28863, 8154, 4566, 3907, 8178, 322, 28797, 4845, 28723, 11629, 28712, 28827, 4784, 316, 284, 28906, 28879, 13096, 2117, 1776, 15173, 28873, 405, 28705, 28740, 28784, 28734, 28705, 28734, 28734, 28734, 13824, 28820, 11620, 268, 6149, 311, 9902, 28705, 28781, 28823, 957, 450, 341, 28820, 29114, 18782, 28879, 28715, 28889, 5972, 14480, 284, 758, 1144, 28879, 485, 28705, 28787, 28784, 28781, 13824, 28820, 11620, 28723, 18786, 28873, 28708, 8178, 322, 28797, 2117, 486, 10766, 4575, 7871, 303, 494, 5393, 29114, 3074, 323, 264, 4798, 28705, 28770, 28740, 28823, 14558, 28873, 2124, 5972, 28708, 28862, 8269, 2124, 724, 28827, 28768, 2390, 264, 13296, 263, 20242, 9538, 446, 724, 28827, 28768, 2390, 486, 10766, 282, 1258, 14475, 3556, 29114, 28705, 28781, 28770, 13210, 1054, 3711, 6701, 28724, 290, 374, 1252, 8739, 2853, 626, 23407, 28215, 3811, 1879, 4686, 626, 8739, 1117, 2892, 28729, 595, 264, 4798, 1319, 353, 515, 28725, 1054, 425, 336, 264, 1298, 2304, 271, 28723, 675, 28723, 550, 28724, 28716, 29826, 316, 28820, 7675, 412, 3531, 29826, 28708, 5192, 9221, 264, 277, 20552, 28723, 16121, 28879, 28715, 28889, 686, 3711, 6701, 486, 6125, 4128, 29114, 818, 28712, 8905, 4845, 28725, 10644, 18765, 290, 28906, 28879, 3811, 686, 28827, 9883, 29114, 25699, 491, 28889, 21688, 279, 710, 435, 10721, 1723, 29826, 21775, 29114, 28723, 351, 28906, 28879, 3811, 14628, 28879, 18904, 17101, 29114, 582, 3711, 1334, 269, 515, 1879, 7988, 22605, 8739, 686, 261, 8739, 532, 4686, 626, 8739, 1117, 2892, 493, 28723, 550, 731, 28879, 424, 268, 3144, 2099, 446, 872, 8909, 28820, 264, 2513, 300, 281, 504, 276, 3811, 317, 28733, 6645, 28725, 446, 7264, 4798, 29826, 333, 28729, 637, 1879, 467, 28716, 281, 504, 1564, 2853, 28725, 446, 6909, 28889, 668, 199, 189, 29026, 28708, 261, 1296, 28709, 446, 872, 8909, 28820, 28723, 524, 1052, 19758, 13704, 28889, 1353, 279, 24160, 290, 491, 28714, 3160, 1627, 7745, 28820, 467, 28716, 686, 28728, 28711, 28863, 1947, 613, 686, 28728, 266, 3907, 28723, 351, 28906, 28879, 28718, 14628, 28879, 16986, 412, 29114, 289, 290, 374, 1252, 8739, 2853, 4042, 1879, 7871, 1150, 28725, 446, 6909, 28797, 317, 28873, 424, 435, 2329, 28710, 332, 1800, 269, 28797, 28723, 351, 28906, 28879, 3811, 2144, 446, 28863, 1857, 29114, 290, 374, 25811, 2853, 18037, 8073, 28708, 28725, 12121, 290, 4827, 486, 2710, 2144, 7463, 307, 12209, 29114, 28725, 13067, 10766, 28906, 28879, 3811, 23407, 2777, 29114, 3061, 28768, 380, 29826, 8269, 435, 10721, 1723, 29826, 21805, 12121, 1798, 13067, 363, 2507, 10268, 28709, 22234, 11621, 436, 28827, 28723, 23393, 290, 28906, 28879, 3811, 23407, 2777, 29114, 363, 6584, 28722, 3203, 11991, 686, 3711, 861, 12121, 1798, 1879, 7865, 28706, 28723, 1670, 28889, 1100, 28768, 424, 637, 14628, 28879, 3061, 380, 29826, 626, 12121, 1798, 16439, 4279, 28725, 6612, 28710, 486, 4321, 2006, 28718, 28862, 2689, 13067, 28889, 4798, 29826, 333, 28729, 16306, 1638, 493, 28725, 446, 6909, 28889, 363, 1187, 353, 22160, 8631, 25991, 1052, 28710, 28723, 11149, 28862, 570, 2144, 5369, 3811, 484, 9041, 29114, 2853, 363, 290, 7939, 28725, 12121, 390, 2345, 29026, 686, 15504, 28879, 424, 347, 28879, 8269, 435, 28728, 28889, 28716, 1174, 28725, 534, 28724, 2710, 637, 9335, 28716, 350, 1144, 5227, 626, 1564, 28723, 1054, 314, 846, 28714, 570, 637, 12778, 12472, 1006, 5872, 28863, 28717, 321, 28747, 6711, 28724, 363, 290, 7939, 9433, 28863, 6294, 3556, 12121, 1798, 3074, 323, 341, 20552, 579, 3121, 286, 3589, 28723, 9532, 268, 28863, 261, 28827, 532, 3121, 5184, 295, 9072, 26166, 28725, 2513, 300, 486, 2710, 290, 1371, 1144, 4128, 29114, 363, 4147, 300, 281, 525, 435, 783, 28827, 21608, 24160, 9492, 4475, 28723, 351, 28711, 1371, 28797, 290, 374, 20079, 2853, 28724, 3191, 316, 515, 20133, 551, 27488, 515, 9433, 570, 29826, 626, 2853, 626, 325, 4104, 28741, 557, 446, 6909, 28797, 290, 28906, 28879, 28718, 17920, 17694, 29114, 818, 1591, 19716, 363, 6679, 28725, 6612, 28709, 290, 28906, 28879, 3811, 712, 6309, 29114, 28723, 11629, 28712, 28827, 4784, 316, 12203, 28741, 363, 10422, 290, 28906, 18765, 24473, 28820, 2166, 29114, 4128, 29114, 2853, 28820, 358, 2124, 290, 941, 20946, 28862, 6012, 12121, 1798, 15374, 1750, 29114, 14558, 28873, 2853, 28723, 12203, 28741, 14628, 28879, 6149, 28862, 28707, 5872, 28863, 712, 28862, 8269, 1852, 7387, 4845, 28723, 351, 28906, 28879, 3811, 486, 29114, 686, 350, 28720, 2947, 24160, 6670, 6149, 2922, 28879, 7330, 3497, 598, 28711, 8739, 28725, 435, 9902, 263, 28862, 28711, 8739, 1808, 28729, 1150, 12490, 338, 16032, 271, 626, 28723, 9532, 435, 338, 28717, 3811, 9884, 28710, 316, 485, 24270, 2594, 22160, 28725, 290, 4827, 486, 2710, 686, 15504, 28879, 28710, 29114, 486, 28707, 28723, 5464, 2562, 19758, 28820, 1189, 28710, 435, 10721, 1723, 29826, 21805, 28754, 3711, 28716, 350, 28711, 570, 637, 28725, 9940, 29826, 4798, 2853, 28718, 2144, 290, 28906, 28879, 3811, 511, 3871, 28710, 29114, 28723, 25392, 373, 637, 1879, 7865, 20208, 686, 3711, 6701, 28724, 28723, 26293, 3589, 1353, 279, 8269, 2124, 290, 491, 28714, 18924, 28725, 13067, 2513, 267, 7330, 2099, 28873, 13296, 402, 28723, 1054, 314, 846, 28714, 570, 637, 12778, 435, 28728, 28889, 28716, 350, 6449, 363, 4081, 4588, 515, 290, 374, 1252, 12689, 2853, 28718, 28723, 9363, 14131, 28827, 5219, 29114, 435, 10721, 1723, 29826, 21805, 28723, 1670, 28889, 1100, 28768, 424, 637, 9433, 570, 29826, 28708, 28723, 2, 13, 13]
inputs:
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Článok: Vyberte si všetky rodinné účty a spočítajte, koľko mesačne utrácate za jedlo, poistenie, školu, dopravu atď. Potom vypočítajte všetky zdroje príjmov. Musíte si kúpiť dom v rámci svojho rozpočtu. Použite online kalkulačku, aby ste odhadli mesačnú splátku hypotéky. Napríklad pôžička vo výške 160 000 dolárov s úrokom 4% bude stáť každý mesiac približne 764 dolárov. Vaša hypotéka by nemala predstavovať viac ako 31% vášho mesačného príjmu a pomer dlhu k príjmu by nemal presiahnuť 43%. Zoznamy mestských domov nájdete na webových stránkach ako Trulia, Zillow a Realtor.com. Vyhľadávanie podľa miesta a ceny. Každý zoznam by mal mať obrázky, takže môžete získať nejaký pocit pre nehnuteľnosť. Môžete tiež nastaviť upozornenia na niektorých z týchto webových stránok. Vložte svoje kritériá a potom dostanete e-mail, kedykoľvek sa na trh dostane dom, ktorý spĺňa tieto kritériá. Kvalifikovaný realitný maklér pozná trh zvnútra i zvonku. Môžu tiež vedieť o mestských domoch na predaj, ktoré ešte neboli uvedené. Môžete si kúpiť mestský dom bez agenta, ale mali by ste si ho najať, ak nemôžete nájsť prijateľné nehnuteľnosti alebo ak vás tento proces ohromí. Agent môžete nájsť v telefónnom zozname alebo na internete. Spýtajte sa tiež priateľov alebo rodiny, či by odporučili akýkoľvek prostriedok, ktorý v minulosti používali. Určite si budete chcieť dom v meste, ale aspoň zvážte bežné nevýhody, aby ste sa rozhodli informovane. Zamyslite sa nad nasledujúcim: Domy v meste majú jednu alebo viac steny so susedmi. Ak sú títo susedia hluční, potom by ste mohli mať v novom dome nepríjemný pobyt. Mnohé mestské domy riadia združenia majiteľov domov (HOA), ktoré môžu stanoviť obmedzenia v tom, čo môžete robiť. Napríklad HOA vám môže zakázať mať domáceho maznáčika alebo prestavať váš dom. HOA tiež účtujú ročné poplatky. Môžete byť zodpovedný za údržbu osobných, nekomerčných vonkajších priestorov. Ak nechcete žiadne povinnosti, mali by ste zvážiť byt. Po identifikácii nehnuteľnostiRozhodnite sa, koľko domu si môžete dovoliť. Pozri sa na internetové zoznamy. Najmi realitného makléra, ak potrebuješ pomoc. Zamyslite sa nad nevýhodami vlastnenia mestského domu. Navštíviť nehnuteľnosti. Spýtajte sa majiteľa.</s>


label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 28754, 3711, 28716, 350, 28711, 570, 637, 28725, 9940, 29826, 4798, 2853, 28718, 2144, 290, 28906, 28879, 3811, 511, 3871, 28710, 29114, 28723, 25392, 373, 637, 1879, 7865, 20208, 686, 3711, 6701, 28724, 28723, 26293, 3589, 1353, 279, 8269, 2124, 290, 491, 28714, 18924, 28725, 13067, 2513, 267, 7330, 2099, 28873, 13296, 402, 28723, 1054, 314, 846, 28714, 570, 637, 12778, 435, 28728, 28889, 28716, 350, 6449, 363, 4081, 4588, 515, 290, 374, 1252, 12689, 2853, 28718, 28723, 9363, 14131, 28827, 5219, 29114, 435, 10721, 1723, 29826, 21805, 28723, 1670, 28889, 1100, 28768, 424, 637, 9433, 570, 29826, 28708, 28723, 2, 13, 13]
labels:
Rozhodnite sa, koľko domu si môžete dovoliť. Pozri sa na internetové zoznamy. Najmi realitného makléra, ak potrebuješ pomoc. Zamyslite sa nad nevýhodami vlastnenia mestského domu. Navštíviť nehnuteľnosti. Spýtajte sa majiteľa.</s>


eval example:
input_ids:
[20548, 336, 349, 396, 13126, 369, 13966, 264, 3638, 28723, 12018, 264, 2899, 369, 6582, 1999, 2691, 274, 272, 2159, 28723, 13, 13, 27332, 3133, 3112, 28747, 13, 28762, 8805, 2947, 14798, 1879, 6957, 8905, 4845, 1879, 5765, 1189, 1879, 19269, 4784, 770, 2245, 28718, 28723, 259, 28828, 9001, 266, 289, 11621, 311, 3039, 8739, 4415, 2124, 338, 686, 6739, 28705, 28740, 28774, 28787, 28770, 325, 2255, 28741, 28745, 28705, 28740, 28784, 500, 28723, 28735, 28723, 28743, 28723, 25585, 28705, 28740, 28782, 28770, 28740, 911, 17407, 2974, 2218, 6294, 28711, 15690, 686, 7988, 4798, 29826, 28729, 8739, 634, 9348, 493, 4466, 4304, 10317, 686, 9001, 266, 626, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 28725, 446, 6909, 28797, 287, 7320, 3061, 28768, 270, 28797, 363, 9178, 366, 2279, 9348, 7702, 712, 28729, 4042, 1187, 353, 12689, 341, 8811, 28862, 515, 28725, 264, 1135, 28863, 28879, 28710, 264, 4798, 14864, 7387, 28827, 3026, 1879, 15612, 14483, 28820, 7675, 412, 332, 13511, 8722, 28827, 332, 1800, 269, 8739, 363, 384, 1371, 626, 431, 289, 1342, 26404, 28820, 9413, 11991, 818, 338, 573, 268, 11621, 311, 3039, 28889, 3589, 4415, 28716, 3589, 1776, 29826, 485, 9884, 1230, 28863, 28717, 539, 9884, 5462, 28862, 28827, 338, 626, 264, 408, 529, 28714, 5905, 325, 28743, 1153, 2255, 609, 413, 5530, 16361, 3531, 28720, 8341, 276, 28889, 710, 28764, 1129, 300, 6359, 300, 418, 833, 6196, 28705, 28750, 28783, 28723, 23009, 1757, 520, 28705, 28740, 28774, 28787, 28770, 28725, 16361, 5697, 28712, 10721, 329, 28889, 1879, 2634, 8377, 28718, 446, 872, 19722, 11621, 311, 3039, 8739, 4415, 28716, 626, 7871, 15612, 28716, 1035, 329, 9619, 264, 4798, 345, 28715, 28906, 28713, 1006, 493, 17688, 350, 28820, 4586, 405, 2124, 408, 529, 28718, 264, 9335, 3144, 2831, 28725, 446, 6909, 28889, 7988, 2218, 3531, 28719, 20597, 28889, 13537, 276, 280, 4580, 504, 28714, 19975, 29114, 280, 264, 2634, 8377, 280, 2586, 26293, 12246, 28873, 12490, 268, 28863, 28715, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 686, 392, 309, 28725, 23374, 345, 20490, 28711, 15690, 19269, 794, 300, 12544, 411, 28718, 3061, 3061, 28768, 28827, 1294, 28827, 28739, 413, 5530, 287, 5053, 345, 28764, 529, 17101, 29114, 264, 686, 11277, 4566, 28710, 29114, 9156, 268, 794, 5872, 28863, 1189, 446, 15612, 28716, 1035, 329, 7509, 4415, 28716, 626, 28725, 6670, 13067, 28863, 4798, 29826, 333, 28729, 277, 4265, 2586, 1054, 9001, 266, 7378, 494, 5872, 28863, 281, 333, 27006, 4304, 485, 8073, 28863, 643, 28725, 4764, 28718, 28879, 3175, 710, 24780, 1403, 264, 1776, 29826, 485, 9884, 1230, 28863, 358, 9884, 5462, 28862, 28827, 338, 28724, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 325, 28765, 7965, 28731, 264, 418, 28820, 9413, 20946, 289, 358, 2892, 9883, 264, 12644, 28722, 3160, 681, 28820, 7378, 28820, 3026, 325, 4032, 3598, 609, 259, 28798, 11245, 4415, 28716, 818, 28764, 12275, 28820, 28873, 29114, 686, 28827, 1252, 282, 9335, 28713, 515, 7529, 28718, 22571, 1334, 385, 29114, 387, 9884, 21344, 494, 28723, 10236, 681, 28889, 9335, 28713, 912, 298, 407, 28709, 4415, 9538, 637, 9335, 12490, 9085, 4321, 341, 893, 10719, 14748, 4425, 1879, 5666, 28716, 264, 28879, 1627, 6874, 13370, 264, 4321, 23672, 28718, 264, 28879, 1627, 9492, 267, 28879, 412, 7751, 440, 23550, 28723, 418, 27740, 353, 13704, 28889, 5718, 264, 1117, 563, 4240, 322, 410, 28718, 724, 10391, 20949, 446, 435, 469, 28820, 291, 2390, 16004, 867, 28718, 21751, 28820, 9041, 9884, 21344, 494, 626, 28725, 264, 28879, 446, 15690, 363, 6739, 28705, 28740, 28783, 28774, 28734, 435, 22432, 463, 1144, 686, 15364, 2413, 5856, 485, 2124, 12487, 28729, 28862, 8269, 2124, 284, 2507, 705, 363, 2425, 11991, 2811, 450, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 28723, 1319, 1052, 28709, 28705, 29365, 282, 12490, 338, 289, 5367, 712, 12577, 28725, 446, 15690, 16361, 3531, 28720, 8341, 276, 28889, 24949, 28889, 23407, 9413, 24160, 686, 9001, 266, 582, 20524, 5872, 28863, 1189, 818, 22219, 268, 1776, 29826, 485, 9884, 1230, 28863, 28717, 13840, 9884, 5462, 28862, 28827, 338, 3589, 28725, 264, 28705, 29365, 282, 28873, 412, 281, 3026, 712, 4845, 28725, 446, 15690, 287, 4362, 3061, 28768, 270, 28820, 724, 15504, 1429, 28764, 515, 686, 9001, 3748, 289, 11621, 311, 3039, 8739, 4415, 2124, 338, 28723, 11424, 16449, 12214, 637, 28725, 23374, 363, 6739, 28705, 28740, 28774, 28781, 28740, 287, 5053, 363, 2901, 28709, 28862, 473, 317, 28873, 424, 3919, 390, 28710, 28705, 28740, 28784, 363, 28707, 9001, 626, 28723, 259, 3393, 15504, 28723, 550, 264, 9902, 6739, 16361, 3061, 28768, 270, 28889, 686, 9001, 266, 289, 11621, 311, 3039, 8739, 4415, 2124, 338, 28804, 17799, 3393, 15504, 28723, 399, 493, 28705, 28740, 28774, 28787, 28770, 17799, 28716, 28889, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 28705, 28770, 28723, 9884, 21344, 494, 28705, 28781, 28723, 418, 27740, 353, 13704, 28889, 5718, 28705, 28782, 28723, 1117, 563, 4240, 322, 410, 28718, 13688, 274, 29114, 28723, 28705, 28740, 28783, 28774, 28734, 325, 28728, 3720, 1144, 28862, 28707, 473, 28731, 28705, 28787, 28723, 2425, 20946, 341, 893, 20946, 28705, 28783, 28723, 28705, 29365, 282, 12490, 338, 289, 5367, 712, 12577, 28705, 28774, 28723, 28705, 29365, 282, 28873, 412, 281, 3026, 712, 4845, 28705, 28740, 28734, 28723, 390, 28710, 3919, 28705, 28740, 28784, 363, 28707, 9001, 626, 475, 11245, 2507, 29114, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 384, 7675, 2507, 29114, 28723, 1054, 9001, 266, 289, 11621, 311, 3039, 8739, 4415, 2124, 338, 1319, 262, 1150, 303, 2117, 28723, 399, 493, 28705, 28740, 28774, 28787, 28770, 13525, 434, 28711, 2507, 29114, 28723, 4925, 387, 367, 5291, 28711, 2507, 29114, 28723, 28705, 28750, 28783, 28723, 387, 13688, 374, 28711, 2507, 29114, 28723, 26293, 12246, 28873, 12490, 268, 28863, 28715, 7035, 23945, 366, 28711, 2507, 29114, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 28705, 28740, 28783, 28723, 4764, 28718, 28879, 3175, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 710, 24780, 1403, 264, 1776, 29826, 485, 9884, 1230, 28863, 358, 9884, 5462, 28862, 28827, 338, 28724, 28705, 28740, 28774, 28723, 418, 28820, 9413, 20946, 289, 358, 2892, 9883, 264, 12644, 28722, 3160, 681, 28820, 7378, 28820, 3026, 387, 384, 28728, 316, 3820, 29114, 28723, 7929, 3598, 2, 13, 13]
inputs:
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Odpovedzte na otázky na konci na základe textu.  Zákon o ohrozených druhoch z roku 1973 (ESA; 16 U.S.C. § 1531 et seq.) je jedným z niekoľkých desiatok environmentálnych zákonov Spojených štátov, ktoré boli prijaté v sedemdesiatych rokoch minulého storočia, a slúži ako legislatíva na vykonávanie ustanovení uvedených v Dohovore o medzinárodnom obchode s ohrozenými druhmi voľne žijúcich živočíchov a rastlín (CITES). ESA bol podpísaný prezidentom Richardom Nixonom 28. decembra 1973, bol navrhnutý na ochranu kriticky ohrozených druhov pred vyhynutím ako "dôsledok hospodárskeho rastu a rozvoja, ktorý nie je podmienený primeranou starostlivosťou a ochranou". Najvyšší súd Spojených štátov zistil, že "jasným zámerom Kongresu pri prijímaní" ESA bolo "zastaviť a zvrátiť trend smerujúci k vyhynutiu druhov, za akúkoľvek cenu". Zákon spravujú dve federálne agentúry, Služba pre ryby a voľne žijúce živočíchy Spojených štátov (FWS) a Národná oceánska a atmosférická správa (NOAA).  Jeden druh obzvlášť získal rozsiahlu pozornosť - žeriav. Historický rozsah tohto druhu sa rozšíril od strednej Kanady na juh až po Mexiko a od Utahu až po pobrežie Atlantiku. Neregulovaný lov a strata biotopu prispeli k neustálemu poklesu populácie žeriavov, až kým v roku 1890 nezmizli z ich primárneho reprodukčného pásma v severnom strede Spojených štátov. Trvalo ďalších osem rokov, kým bol podpísaný prvý národný zákon upravujúci obchod s voľne žijúcimi živočíchmi, a ďalšie dva roky, kým bola prijatá prvá verzia zákona o ohrozených druhoch. Odhaduje sa, že v roku 1941 bolo v divočine ešte len asi 16 vtákov.  Prvá. V akom roku bol prijatý zákon o ohrozených druhoch? DruPrvá. Rok 1973 Druhý. Áno, áno. 3. žeriav 4. Neregulovaný lov 5. strata biotopu Šesť. 1890 (v angličtine) 7. severná stredná 8. ďalších osem rokov 9. ďalšie dva roky 10. asi len 16 vtákov Jedenásť. Áno, áno. Dvanásť. Zákon o ohrozených druhoch Trinajstka. Rok 1973 Čtrnásť. December - Pätnásť. 28. - Šestnásť. Najvyšší súd USA Sedemnásť. Áno, áno. 18. Služba Spojených štátov pre ryby a voľne žijúce živočíchy 19. Národná oceánska a atmosférická správa - Dvadsať. NOAA</s>


label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 3393, 15504, 28723, 399, 493, 28705, 28740, 28774, 28787, 28770, 17799, 28716, 28889, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 28705, 28770, 28723, 9884, 21344, 494, 28705, 28781, 28723, 418, 27740, 353, 13704, 28889, 5718, 28705, 28782, 28723, 1117, 563, 4240, 322, 410, 28718, 13688, 274, 29114, 28723, 28705, 28740, 28783, 28774, 28734, 325, 28728, 3720, 1144, 28862, 28707, 473, 28731, 28705, 28787, 28723, 2425, 20946, 341, 893, 20946, 28705, 28783, 28723, 28705, 29365, 282, 12490, 338, 289, 5367, 712, 12577, 28705, 28774, 28723, 28705, 29365, 282, 28873, 412, 281, 3026, 712, 4845, 28705, 28740, 28734, 28723, 390, 28710, 3919, 28705, 28740, 28784, 363, 28707, 9001, 626, 475, 11245, 2507, 29114, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 384, 7675, 2507, 29114, 28723, 1054, 9001, 266, 289, 11621, 311, 3039, 8739, 4415, 2124, 338, 1319, 262, 1150, 303, 2117, 28723, 399, 493, 28705, 28740, 28774, 28787, 28770, 13525, 434, 28711, 2507, 29114, 28723, 4925, 387, 367, 5291, 28711, 2507, 29114, 28723, 28705, 28750, 28783, 28723, 387, 13688, 374, 28711, 2507, 29114, 28723, 26293, 12246, 28873, 12490, 268, 28863, 28715, 7035, 23945, 366, 28711, 2507, 29114, 28723, 10415, 1510, 28725, 8533, 1510, 28723, 28705, 28740, 28783, 28723, 4764, 28718, 28879, 3175, 1670, 6054, 269, 8739, 10531, 28707, 28820, 532, 28728, 710, 24780, 1403, 264, 1776, 29826, 485, 9884, 1230, 28863, 358, 9884, 5462, 28862, 28827, 338, 28724, 28705, 28740, 28774, 28723, 418, 28820, 9413, 20946, 289, 358, 2892, 9883, 264, 12644, 28722, 3160, 681, 28820, 7378, 28820, 3026, 387, 384, 28728, 316, 3820, 29114, 28723, 7929, 3598, 2, 13, 13]
labels:
Prvá. Rok 1973 Druhý. Áno, áno. 3. žeriav 4. Neregulovaný lov 5. strata biotopu Šesť. 1890 (v angličtine) 7. severná stredná 8. ďalších osem rokov 9. ďalšie dva roky 10. asi len 16 vtákov Jedenásť. Áno, áno. Dvanásť. Zákon o ohrozených druhoch Trinajstka. Rok 1973 Čtrnásť. December - Pätnásť. 28. - Šestnásť. Najvyšší súd USA Sedemnásť. Áno, áno. 18. Služba Spojených štátov pre ryby a voľne žijúce živočíchy 19. Národná oceánska a atmosférická správa - Dvadsať. NOAA</s>


[INFO|configuration_utils.py:765] 2026-08-30 04:19:29,725 >> loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
[INFO|configuration_utils.py:841] 2026-08-30 04:19:29,728 >> Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

[INFO|2026-08-30 04:19:29] llamafactory.model.model_utils.kv_cache:144 >> KV cache is disabled during training.
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/unsloth/__init__.py:153: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.

Please restructure your imports with 'import unsloth' at the top of your file.
  from ._gpu_init import *
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/unsloth/__init__.py:153: UserWarning: WARNING: Unsloth should be imported before [trl, transformers, peft] to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.

Please restructure your imports with 'import unsloth' at the top of your file.
  from ._gpu_init import *
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

Unsloth: WARNING `trust_remote_code` is True.
Are you certain you want to do remote code execution?
==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
   \\   /|    NVIDIA TITAN RTX. Num GPUs = 2. Max memory: 23.46 GB. Platform: Linux.
O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 7.5. CUDA Toolkit: 12.4. Triton: 3.2.0
\        /    Bfloat16 = FALSE. FA [Xformers = None. FA2 = False]
 "-____-"     Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Unsloth: WARNING `trust_remote_code` is True.
Are you certain you want to do remote code execution?
==((====))==  Unsloth 2026.6.1: Fast Mistral patching. Transformers: 5.5.0.
   \\   /|    NVIDIA TITAN RTX. Num GPUs = 2. Max memory: 23.46 GB. Platform: Linux.
O^O/ \_/ \    Torch: 2.6.0+cu124. CUDA: 7.5. CUDA Toolkit: 12.4. Triton: 3.2.0
\        /    Bfloat16 = FALSE. FA [Xformers = None. FA2 = False]
 "-____-"     Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "float16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]loading weights file model.safetensors from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/model.safetensors.index.json
Will use dtype=torch.float16 as defined in model's config object
Generate config GenerationConfig {
  "bos_token_id": 1,
  "eos_token_id": 2,
  "output_attentions": false,
  "output_hidden_states": false,
  "pad_token_id": 2,
  "use_cache": true
}


Loading weights:   0%|          | 0/291 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/291 [00:00<01:33,  3.09it/s]
Loading weights:   0%|          | 1/291 [00:00<01:02,  4.65it/s]
Loading weights:   1%|          | 2/291 [00:13<39:04,  8.11s/it]
Loading weights:   1%|          | 2/291 [00:14<39:52,  8.28s/it]
Loading weights:  14%|█▎        | 40/291 [00:19<01:35,  2.64it/s]
Loading weights:  14%|█▎        | 40/291 [00:18<01:33,  2.68it/s]
Loading weights:  14%|█▍        | 41/291 [00:22<01:59,  2.09it/s]
Loading weights:  14%|█▍        | 41/291 [00:22<01:58,  2.11it/s]
Loading weights:  17%|█▋        | 49/291 [00:34<03:13,  1.25it/s]
Loading weights:  17%|█▋        | 49/291 [00:35<03:14,  1.25it/s]
Loading weights:  26%|██▌       | 76/291 [00:57<02:56,  1.22it/s]
Loading weights:  26%|██▌       | 76/291 [00:57<02:57,  1.21it/s]
Loading weights:  36%|███▌      | 105/291 [01:04<01:39,  1.86it/s]
Loading weights:  36%|███▌      | 105/291 [01:03<01:39,  1.87it/s]
Loading weights:  45%|████▌     | 132/291 [01:07<01:01,  2.59it/s]
Loading weights:  45%|████▌     | 132/291 [01:08<01:01,  2.59it/s]
Loading weights:  48%|████▊     | 139/291 [01:08<00:53,  2.84it/s]
Loading weights:  48%|████▊     | 139/291 [01:08<00:53,  2.84it/s]
Loading weights:  51%|█████     | 148/291 [01:08<00:41,  3.45it/s]
Loading weights:  51%|█████     | 148/291 [01:09<00:41,  3.45it/s]
Loading weights:  52%|█████▏    | 151/291 [01:09<00:40,  3.43it/s]
Loading weights:  52%|█████▏    | 150/291 [01:09<00:42,  3.33it/s]
Loading weights:  54%|█████▍    | 157/291 [01:10<00:35,  3.81it/s]
Loading weights:  54%|█████▍    | 157/291 [01:10<00:34,  3.84it/s]
Loading weights:  56%|█████▌    | 163/291 [01:11<00:28,  4.54it/s]
Loading weights:  56%|█████▌    | 163/291 [01:10<00:28,  4.51it/s]
Loading weights:  57%|█████▋    | 166/291 [01:12<00:31,  4.00it/s]
Loading weights:  57%|█████▋    | 166/291 [01:12<00:31,  3.98it/s]
Loading weights:  57%|█████▋    | 167/291 [01:13<00:34,  3.59it/s]
Loading weights:  57%|█████▋    | 167/291 [01:12<00:34,  3.57it/s]
Loading weights:  60%|██████    | 175/291 [01:15<00:31,  3.69it/s]
Loading weights:  60%|██████    | 175/291 [01:15<00:31,  3.70it/s]
Loading weights:  60%|██████    | 176/291 [01:20<01:12,  1.59it/s]
Loading weights:  60%|██████    | 176/291 [01:20<01:12,  1.60it/s]
Loading weights:  66%|██████▋   | 193/291 [01:21<00:25,  3.84it/s]
Loading weights:  66%|██████▋   | 193/291 [01:21<00:25,  3.85it/s]
Loading weights:  67%|██████▋   | 195/291 [01:22<00:25,  3.73it/s]
Loading weights:  67%|██████▋   | 195/291 [01:21<00:25,  3.73it/s]
Loading weights:  69%|██████▉   | 202/291 [01:26<00:36,  2.47it/s]
Loading weights:  69%|██████▉   | 202/291 [01:27<00:36,  2.47it/s]
Loading weights:  70%|██████▉   | 203/291 [01:32<01:01,  1.42it/s]
Loading weights:  70%|██████▉   | 203/291 [01:32<01:01,  1.42it/s]
Loading weights:  82%|████████▏ | 238/291 [01:32<00:09,  5.53it/s]
Loading weights:  82%|████████▏ | 238/291 [01:32<00:09,  5.53it/s]
Loading weights:  83%|████████▎ | 242/291 [01:32<00:08,  5.85it/s]
Loading weights:  83%|████████▎ | 242/291 [01:33<00:08,  5.85it/s]
Loading weights:  85%|████████▍ | 247/291 [01:34<00:08,  5.09it/s]
Loading weights:  85%|████████▍ | 247/291 [01:34<00:08,  5.08it/s]
Loading weights:  86%|████████▌ | 249/291 [01:35<00:08,  4.94it/s]
Loading weights:  86%|████████▌ | 249/291 [01:34<00:08,  4.94it/s]
Loading weights:  88%|████████▊ | 257/291 [01:35<00:04,  6.82it/s]
Loading weights:  88%|████████▊ | 257/291 [01:35<00:04,  6.82it/s]
Loading weights:  91%|█████████ | 265/291 [01:35<00:03,  8.65it/s]
Loading weights:  91%|█████████ | 265/291 [01:35<00:03,  8.65it/s]
Loading weights:  94%|█████████▍| 274/291 [01:35<00:01, 12.45it/s]
Loading weights:  94%|█████████▍| 274/291 [01:36<00:01, 12.44it/s]
Loading weights:  97%|█████████▋| 283/291 [01:35<00:00, 16.17it/s]
Loading weights:  97%|█████████▋| 283/291 [01:36<00:00, 15.93it/s]
Loading weights: 100%|██████████| 291/291 [01:35<00:00,  3.03it/s]
Loading weights: 100%|██████████| 291/291 [01:36<00:00,  3.02it/s]

loading configuration file generation_config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/generation_config.json
Generate config GenerationConfig {
  "bos_token_id": 1,
  "eos_token_id": 2
}

Could not locate the custom_generate/generate.py inside slovak-nlp/mistral-sk-7b.
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

slovak-nlp/mistral-sk-7b does not have a padding token! Will use pad_token = <unk>.
slovak-nlp/mistral-sk-7b does not have a padding token! Will use pad_token = <unk>.
/home/schwarc/LlamaFactory/src/llamafactory/model/model_utils/checkpointing.py:48: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd
/home/schwarc/LlamaFactory/src/llamafactory/model/model_utils/checkpointing.py:65: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_bwd
[INFO|2026-08-30 04:21:32] llamafactory.model.model_utils.checkpointing:144 >> Gradient checkpointing enabled.
[INFO|2026-08-30 04:21:32] llamafactory.model.adapter:144 >> Upcasting trainable params to float32.
[INFO|2026-08-30 04:21:32] llamafactory.model.adapter:144 >> Fine-tuning method: LoRA
/home/schwarc/LlamaFactory/src/llamafactory/model/model_utils/checkpointing.py:48: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd
/home/schwarc/LlamaFactory/src/llamafactory/model/model_utils/checkpointing.py:65: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_bwd
Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
Unsloth 2026.6.1 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
[INFO|2026-08-30 04:21:36] llamafactory.model.loader:144 >> trainable params: 41,943,040 || all params: 7,283,675,136 || trainable%: 0.5758
max_steps is given, it will override any value given in num_train_epochs
The `train_sampling_strategy='random'` option is ignored when using an `IterableDataset`. Samplers cannot be used with IterableDataset as they require indexed access to the dataset.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'pad_token_id': 2}.
Unsloth is running with multi GPUs - the effective batch size is multiplied by 2
Unsloth is running with multi GPUs - the effective batch size is multiplied by 2
==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
   \\   /|    Num examples = 3,600,000 | Num Epochs = 9,223,372,036,854,775,807 | Total steps = 225,000
O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 8
\        /    Data Parallel GPUs = 2 | Total batch size (1 x 8 x 2) = 16
 "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)
==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1
   \\   /|    Num examples = 3,600,000 | Num Epochs = 9,223,372,036,854,775,807 | Total steps = 225,000
O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 8
\        /    Data Parallel GPUs = 2 | Total batch size (1 x 8 x 2) = 16
 "-____-"     Trainable parameters = 41,943,040 of 7,283,675,136 (0.58% trained)

  0%|          | 0/225000 [00:00<?, ?it/s]`use_return_dict` is deprecated! Use `return_dict` instead!
`use_return_dict` is deprecated! Use `return_dict` instead!
Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.

  0%|          | 1/225000 [00:15<981:20:23, 15.70s/it]
  0%|          | 2/225000 [00:24<725:40:30, 11.61s/it]
  0%|          | 3/225000 [00:32<638:15:43, 10.21s/it]
  0%|          | 4/225000 [00:41<588:12:01,  9.41s/it]
  0%|          | 5/225000 [00:48<547:01:32,  8.75s/it]
  0%|          | 6/225000 [00:57<555:35:32,  8.89s/it]
  0%|          | 7/225000 [01:07<565:02:10,  9.04s/it]
  0%|          | 8/225000 [01:21<666:43:51, 10.67s/it]
  0%|          | 9/225000 [01:42<860:23:14, 13.77s/it]
  0%|          | 10/225000 [01:58<913:11:59, 14.61s/it]
  0%|          | 11/225000 [02:15<964:19:55, 15.43s/it]
  0%|          | 12/225000 [02:32<989:30:37, 15.83s/it]
  0%|          | 13/225000 [02:49<1015:29:39, 16.25s/it]
  0%|          | 14/225000 [03:16<1214:06:42, 19.43s/it]
  0%|          | 15/225000 [03:34<1179:21:11, 18.87s/it]
  0%|          | 16/225000 [03:47<1082:55:42, 17.33s/it]
  0%|          | 17/225000 [04:16<1293:27:29, 20.70s/it]
  0%|          | 18/225000 [04:40<1352:48:13, 21.65s/it]
  0%|          | 19/225000 [05:07<1451:24:35, 23.22s/it]
  0%|          | 20/225000 [05:25<1369:03:16, 21.91s/it]
  0%|          | 21/225000 [05:46<1334:52:04, 21.36s/it]
  0%|          | 22/225000 [06:08<1353:43:46, 21.66s/it]
  0%|          | 23/225000 [06:29<1339:42:14, 21.44s/it]
  0%|          | 24/225000 [06:45<1246:26:45, 19.95s/it]
  0%|          | 25/225000 [07:10<1333:37:31, 21.34s/it]
  0%|          | 26/225000 [07:32<1341:31:09, 21.47s/it]
  0%|          | 27/225000 [07:44<1163:51:54, 18.62s/it]
  0%|          | 28/225000 [08:04<1201:13:11, 19.22s/it]
  0%|          | 29/225000 [08:26<1256:09:44, 20.10s/it]
  0%|          | 30/225000 [08:51<1336:36:32, 21.39s/it]
  0%|          | 31/225000 [09:14<1367:57:34, 21.89s/it]
  0%|          | 32/225000 [09:32<1301:06:13, 20.82s/it]
  0%|          | 33/225000 [09:50<1237:00:44, 19.80s/it]
  0%|          | 34/225000 [10:06<1172:08:57, 18.76s/it]
  0%|          | 35/225000 [10:33<1325:40:48, 21.21s/it]
  0%|          | 36/225000 [10:54<1315:45:18, 21.06s/it]
  0%|          | 37/225000 [11:14<1310:45:30, 20.98s/it]
  0%|          | 38/225000 [11:34<1288:03:04, 20.61s/it]
  0%|          | 39/225000 [11:54<1272:02:49, 20.36s/it]
  0%|          | 40/225000 [12:12<1225:37:08, 19.61s/it]
  0%|          | 41/225000 [12:38<1356:04:09, 21.70s/it]
  0%|          | 42/225000 [13:05<1450:23:10, 23.21s/it]
  0%|          | 43/225000 [13:28<1448:48:32, 23.19s/it]
  0%|          | 44/225000 [13:42<1265:35:35, 20.25s/it]
  0%|          | 45/225000 [13:56<1161:26:00, 18.59s/it]
  0%|          | 46/225000 [14:24<1334:44:55, 21.36s/it]
  0%|          | 47/225000 [14:41<1259:43:31, 20.16s/it]
  0%|          | 48/225000 [15:04<1311:37:49, 20.99s/it]
  0%|          | 49/225000 [15:20<1211:07:25, 19.38s/it]
  0%|          | 50/225000 [15:35<1121:03:13, 17.94s/it]
                                                        
{'loss': '1.624', 'grad_norm': '0.8831', 'learning_rate': '2.35e-07', 'epoch': '0.0002222', 'num_input_tokens_seen': 537960, 'train_runtime': '935.1', 'train_tokens_per_second': '575.3'}

  0%|          | 50/225000 [15:35<1121:03:13, 17.94s/it]
  0%|          | 51/225000 [16:01<1275:57:07, 20.42s/it]
  0%|          | 52/225000 [16:10<1057:53:59, 16.93s/it]
  0%|          | 53/225000 [16:37<1262:37:49, 20.21s/it]
  0%|          | 54/225000 [16:55<1219:56:40, 19.52s/it]
  0%|          | 55/225000 [17:22<1358:28:10, 21.74s/it]
  0%|          | 56/225000 [17:41<1292:56:22, 20.69s/it]
  0%|          | 57/225000 [17:49<1056:32:43, 16.91s/it]
  0%|          | 58/225000 [18:16<1253:01:21, 20.05s/it]
  0%|          | 59/225000 [18:31<1163:48:50, 18.63s/it]
  0%|          | 60/225000 [18:46<1090:31:39, 17.45s/it]
  0%|          | 61/225000 [19:12<1256:08:08, 20.10s/it]
  0%|          | 62/225000 [19:28<1167:26:50, 18.68s/it]
  0%|          | 63/225000 [19:52<1270:57:51, 20.34s/it]
  0%|          | 64/225000 [20:13<1287:50:10, 20.61s/it]
  0%|          | 65/225000 [20:29<1189:33:23, 19.04s/it]
  0%|          | 66/225000 [20:41<1063:53:21, 17.03s/it]
  0%|          | 67/225000 [21:06<1208:10:59, 19.34s/it]
  0%|          | 68/225000 [21:27<1252:39:14, 20.05s/it]
  0%|          | 69/225000 [21:42<1145:31:33, 18.33s/it]
  0%|          | 70/225000 [21:58<1100:56:45, 17.62s/it]
  0%|          | 71/225000 [22:20<1189:06:27, 19.03s/it]
  0%|          | 72/225000 [22:40<1215:23:49, 19.45s/it]
  0%|          | 73/225000 [22:58<1187:28:56, 19.01s/it]
  0%|          | 74/225000 [23:17<1183:46:06, 18.95s/it]
  0%|          | 75/225000 [23:34<1146:20:38, 18.35s/it]
  0%|          | 76/225000 [23:54<1176:40:16, 18.83s/it]
  0%|          | 77/225000 [24:12<1158:46:12, 18.55s/it]
  0%|          | 78/225000 [24:28<1115:45:57, 17.86s/it]
  0%|          | 79/225000 [24:51<1216:14:52, 19.47s/it]
  0%|          | 80/225000 [25:08<1161:36:55, 18.59s/it]
  0%|          | 81/225000 [25:30<1229:43:45, 19.68s/it]
  0%|          | 82/225000 [25:49<1209:33:04, 19.36s/it]
  0%|          | 83/225000 [26:14<1323:42:17, 21.19s/it]
  0%|          | 84/225000 [26:37<1360:45:16, 21.78s/it]
  0%|          | 85/225000 [26:53<1240:39:46, 19.86s/it]
  0%|          | 86/225000 [27:11<1203:16:58, 19.26s/it]
  0%|          | 87/225000 [27:35<1306:58:14, 20.92s/it]
  0%|          | 88/225000 [27:49<1175:01:50, 18.81s/it]
  0%|          | 89/225000 [28:10<1206:29:20, 19.31s/it]
  0%|          | 90/225000 [28:40<1411:15:00, 22.59s/it]
  0%|          | 91/225000 [28:55<1263:06:53, 20.22s/it]
  0%|          | 92/225000 [29:16<1280:09:13, 20.49s/it]
  0%|          | 93/225000 [29:38<1308:32:26, 20.95s/it]
  0%|          | 94/225000 [29:51<1171:11:01, 18.75s/it]
  0%|          | 95/225000 [30:21<1376:21:03, 22.03s/it]
  0%|          | 96/225000 [30:37<1263:40:46, 20.23s/it]
  0%|          | 97/225000 [30:55<1227:02:51, 19.64s/it]
  0%|          | 98/225000 [31:24<1397:15:02, 22.37s/it]
  0%|          | 99/225000 [31:42<1306:40:03, 20.92s/it]
  0%|          | 100/225000 [32:03<1311:22:39, 20.99s/it]
                                                         
{'loss': '1.628', 'grad_norm': '0.8823', 'learning_rate': '4.85e-07', 'epoch': '0.0004444', 'num_input_tokens_seen': 1065640, 'train_runtime': '1923', 'train_tokens_per_second': '554'}

  0%|          | 100/225000 [32:03<1311:22:39, 20.99s/it]
  0%|          | 101/225000 [32:26<1357:54:24, 21.74s/it]
  0%|          | 102/225000 [32:46<1318:52:03, 21.11s/it]
  0%|          | 103/225000 [33:08<1331:12:47, 21.31s/it]
  0%|          | 104/225000 [33:28<1318:25:09, 21.10s/it]
  0%|          | 105/225000 [33:45<1227:43:44, 19.65s/it]
  0%|          | 106/225000 [34:02<1178:35:37, 18.87s/it]
  0%|          | 107/225000 [34:21<1187:32:30, 19.01s/it]
  0%|          | 108/225000 [34:50<1368:15:10, 21.90s/it]
  0%|          | 109/225000 [35:08<1301:37:56, 20.84s/it]
  0%|          | 110/225000 [35:33<1381:48:39, 22.12s/it]
  0%|          | 111/225000 [35:51<1307:05:45, 20.92s/it]
  0%|          | 112/225000 [36:14<1337:44:15, 21.41s/it]
  0%|          | 113/225000 [36:31<1262:20:16, 20.21s/it]
  0%|          | 114/225000 [36:49<1221:34:16, 19.56s/it]
  0%|          | 115/225000 [37:09<1225:50:58, 19.62s/it]
  0%|          | 116/225000 [37:38<1404:43:19, 22.49s/it]
  0%|          | 117/225000 [37:54<1285:38:29, 20.58s/it]
  0%|          | 118/225000 [38:07<1143:12:18, 18.30s/it]
  0%|          | 119/225000 [38:32<1260:12:02, 20.17s/it]
  0%|          | 120/225000 [38:48<1192:24:59, 19.09s/it]
  0%|          | 121/225000 [39:11<1252:25:34, 20.05s/it]
  0%|          | 122/225000 [39:32<1267:21:44, 20.29s/it]
  0%|          | 123/225000 [39:49<1207:13:25, 19.33s/it]
  0%|          | 124/225000 [40:13<1296:23:41, 20.75s/it]
  0%|          | 125/225000 [40:39<1397:09:26, 22.37s/it]
  0%|          | 126/225000 [41:04<1443:05:10, 23.10s/it]
  0%|          | 127/225000 [41:17<1267:49:26, 20.30s/it]
  0%|          | 128/225000 [41:37<1258:05:32, 20.14s/it]
  0%|          | 129/225000 [41:53<1170:56:47, 18.75s/it]
  0%|          | 130/225000 [42:15<1233:12:29, 19.74s/it]
  0%|          | 131/225000 [42:32<1185:16:29, 18.98s/it]
  0%|          | 132/225000 [42:50<1162:40:06, 18.61s/it]
  0%|          | 133/225000 [43:12<1228:29:54, 19.67s/it]
  0%|          | 134/225000 [43:40<1380:31:14, 22.10s/it]
  0%|          | 135/225000 [44:03<1398:00:12, 22.38s/it]
  0%|          | 136/225000 [44:24<1379:37:53, 22.09s/it]
  0%|          | 137/225000 [44:39<1250:11:08, 20.02s/it]
  0%|          | 138/225000 [44:57<1203:19:35, 19.27s/it]
  0%|          | 139/225000 [45:12<1121:39:30, 17.96s/it]
  0%|          | 140/225000 [45:35<1225:07:49, 19.61s/it]
  0%|          | 141/225000 [45:58<1282:31:26, 20.53s/it]
  0%|          | 142/225000 [46:11<1148:30:29, 18.39s/it]
  0%|          | 143/225000 [46:34<1234:05:51, 19.76s/it]
  0%|          | 144/225000 [46:50<1168:54:05, 18.71s/it]
  0%|          | 145/225000 [47:10<1180:35:21, 18.90s/it]
  0%|          | 146/225000 [47:26<1131:17:29, 18.11s/it]
  0%|          | 147/225000 [47:50<1235:07:15, 19.77s/it]
  0%|          | 148/225000 [48:09<1216:58:58, 19.48s/it]
  0%|          | 149/225000 [48:34<1323:43:36, 21.19s/it]
  0%|          | 150/225000 [48:51<1251:47:50, 20.04s/it]
                                                         
{'loss': '1.58', 'grad_norm': '1.066', 'learning_rate': '7.35e-07', 'epoch': '0.0006667', 'num_input_tokens_seen': 1590840, 'train_runtime': '2932', 'train_tokens_per_second': '542.7'}

  0%|          | 150/225000 [48:51<1251:47:50, 20.04s/it]
  0%|          | 151/225000 [49:08<1193:58:37, 19.12s/it]
  0%|          | 152/225000 [49:27<1197:06:13, 19.17s/it]
  0%|          | 153/225000 [49:51<1287:43:51, 20.62s/it]
  0%|          | 154/225000 [50:17<1376:15:42, 22.04s/it]
  0%|          | 155/225000 [50:34<1283:27:29, 20.55s/it]
  0%|          | 156/225000 [50:50<1194:12:25, 19.12s/it]
  0%|          | 157/225000 [51:13<1284:44:57, 20.57s/it]
  0%|          | 158/225000 [51:23<1082:52:08, 17.34s/it]
  0%|          | 159/225000 [51:48<1212:21:13, 19.41s/it]
  0%|          | 160/225000 [52:07<1212:33:00, 19.41s/it]
  0%|          | 161/225000 [52:25<1184:57:46, 18.97s/it]
  0%|          | 162/225000 [52:49<1272:35:33, 20.38s/it]
  0%|          | 163/225000 [53:04<1180:35:12, 18.90s/it]
  0%|          | 164/225000 [53:32<1347:09:59, 21.57s/it]
  0%|          | 165/225000 [53:56<1391:30:22, 22.28s/it]
  0%|          | 166/225000 [54:17<1382:10:01, 22.13s/it]
  0%|          | 167/225000 [54:31<1221:39:29, 19.56s/it]
  0%|          | 168/225000 [54:47<1150:15:58, 18.42s/it]
  0%|          | 169/225000 [54:58<1014:54:24, 16.25s/it]
  0%|          | 170/225000 [55:20<1129:02:39, 18.08s/it]
  0%|          | 171/225000 [55:43<1207:22:29, 19.33s/it]
  0%|          | 172/225000 [56:01<1180:22:50, 18.90s/it]
  0%|          | 173/225000 [56:21<1217:52:21, 19.50s/it]
  0%|          | 174/225000 [56:44<1271:17:47, 20.36s/it]
  0%|          | 175/225000 [56:59<1169:55:40, 18.73s/it]
  0%|          | 176/225000 [57:31<1426:00:10, 22.83s/it]
  0%|          | 177/225000 [57:45<1258:44:01, 20.16s/it]
  0%|          | 178/225000 [58:03<1215:22:37, 19.46s/it]
  0%|          | 179/225000 [58:28<1312:41:01, 21.02s/it]
  0%|          | 180/225000 [58:52<1386:17:27, 22.20s/it]
  0%|          | 181/225000 [59:05<1196:36:54, 19.16s/it]
  0%|          | 182/225000 [59:19<1109:57:26, 17.77s/it]
  0%|          | 183/225000 [59:43<1227:01:54, 19.65s/it]
  0%|          | 184/225000 [1:00:09<1352:38:05, 21.66s/it]
  0%|          | 185/225000 [1:00:25<1239:55:17, 19.86s/it]
  0%|          | 186/225000 [1:00:48<1300:28:47, 20.82s/it]
  0%|          | 187/225000 [1:01:07<1258:28:38, 20.15s/it]
  0%|          | 188/225000 [1:01:20<1137:23:15, 18.21s/it]
  0%|          | 189/225000 [1:01:45<1257:51:00, 20.14s/it]
  0%|          | 190/225000 [1:02:04<1236:45:44, 19.80s/it]
  0%|          | 191/225000 [1:02:30<1355:52:59, 21.71s/it]
  0%|          | 192/225000 [1:02:59<1485:26:46, 23.79s/it]
  0%|          | 193/225000 [1:03:16<1363:06:59, 21.83s/it]
  0%|          | 194/225000 [1:03:32<1252:15:08, 20.05s/it]
  0%|          | 195/225000 [1:03:49<1200:23:03, 19.22s/it]
  0%|          | 196/225000 [1:04:08<1186:00:06, 18.99s/it]
  0%|          | 197/225000 [1:04:28<1206:02:09, 19.31s/it]
  0%|          | 198/225000 [1:04:41<1085:01:18, 17.38s/it]
  0%|          | 199/225000 [1:05:07<1244:11:41, 19.92s/it]
  0%|          | 200/225000 [1:05:25<1216:05:27, 19.47s/it]
                                                           
{'loss': '1.497', 'grad_norm': '0.528', 'learning_rate': '9.85e-07', 'epoch': '0.0008889', 'num_input_tokens_seen': 2121272, 'train_runtime': '3926', 'train_tokens_per_second': '540.4'}

  0%|          | 200/225000 [1:05:25<1216:05:27, 19.47s/it]
  0%|          | 201/225000 [1:05:47<1263:53:21, 20.24s/it]
  0%|          | 202/225000 [1:06:06<1230:36:29, 19.71s/it]
  0%|          | 203/225000 [1:06:27<1259:23:09, 20.17s/it]
  0%|          | 204/225000 [1:06:50<1309:18:55, 20.97s/it]
  0%|          | 205/225000 [1:07:05<1205:45:38, 19.31s/it]
  0%|          | 206/225000 [1:07:25<1218:56:12, 19.52s/it]
  0%|          | 207/225000 [1:07:46<1252:08:39, 20.05s/it]
  0%|          | 208/225000 [1:08:00<1123:35:20, 17.99s/it]
  0%|          | 209/225000 [1:08:17<1110:52:53, 17.79s/it]
  0%|          | 210/225000 [1:08:33<1079:15:37, 17.28s/it]
  0%|          | 211/225000 [1:08:54<1150:20:28, 18.42s/it]
  0%|          | 212/225000 [1:09:20<1282:33:10, 20.54s/it]
  0%|          | 213/225000 [1:09:39<1270:18:24, 20.34s/it]
  0%|          | 214/225000 [1:09:53<1137:27:45, 18.22s/it]
  0%|          | 215/225000 [1:10:14<1187:39:57, 19.02s/it]
  0%|          | 216/225000 [1:10:33<1189:45:43, 19.05s/it]
  0%|          | 217/225000 [1:10:49<1135:50:33, 18.19s/it]
  0%|          | 218/225000 [1:11:04<1083:12:54, 17.35s/it]
  0%|          | 219/225000 [1:11:27<1187:09:21, 19.01s/it]
  0%|          | 220/225000 [1:11:50<1266:41:39, 20.29s/it]
  0%|          | 221/225000 [1:12:08<1216:40:13, 19.49s/it]
  0%|          | 222/225000 [1:12:33<1329:01:46, 21.29s/it]
  0%|          | 223/225000 [1:12:47<1192:03:12, 19.09s/it]
  0%|          | 224/225000 [1:13:06<1178:30:54, 18.88s/it]
  0%|          | 225/225000 [1:13:30<1283:35:42, 20.56s/it]
  0%|          | 226/225000 [1:13:52<1303:40:12, 20.88s/it]
  0%|          | 227/225000 [1:14:08<1215:53:53, 19.47s/it]
  0%|          | 228/225000 [1:14:30<1256:45:31, 20.13s/it]
  0%|          | 229/225000 [1:14:47<1205:51:41, 19.31s/it]
  0%|          | 230/225000 [1:15:11<1287:29:32, 20.62s/it]
  0%|          | 231/225000 [1:15:31<1286:48:48, 20.61s/it]
  0%|          | 232/225000 [1:15:51<1258:37:55, 20.16s/it]
  0%|          | 233/225000 [1:16:03<1112:35:10, 17.82s/it]
  0%|          | 234/225000 [1:16:26<1204:33:29, 19.29s/it]
  0%|          | 235/225000 [1:16:43<1171:25:50, 18.76s/it]
  0%|          | 236/225000 [1:17:04<1214:38:22, 19.45s/it]
  0%|          | 237/225000 [1:17:33<1392:56:52, 22.31s/it]
  0%|          | 238/225000 [1:17:43<1157:15:01, 18.54s/it]
  0%|          | 239/225000 [1:17:59<1114:08:33, 17.85s/it]
  0%|          | 240/225000 [1:18:20<1175:43:35, 18.83s/it]
  0%|          | 241/225000 [1:18:41<1212:24:38, 19.42s/it]
  0%|          | 242/225000 [1:19:04<1284:06:13, 20.57s/it]
  0%|          | 243/225000 [1:19:26<1303:10:50, 20.87s/it]
  0%|          | 244/225000 [1:19:52<1405:22:17, 22.51s/it]
  0%|          | 245/225000 [1:20:06<1236:38:05, 19.81s/it]
  0%|          | 246/225000 [1:20:38<1464:57:15, 23.46s/it]
  0%|          | 247/225000 [1:20:57<1393:50:10, 22.33s/it]
  0%|          | 248/225000 [1:21:11<1222:35:01, 19.58s/it]
  0%|          | 249/225000 [1:21:38<1371:35:35, 21.97s/it]
  0%|          | 250/225000 [1:21:50<1184:17:38, 18.97s/it]
                                                           
{'loss': '1.475', 'grad_norm': '0.8209', 'learning_rate': '1.235e-06', 'epoch': '0.001111', 'num_input_tokens_seen': 2636088, 'train_runtime': '4911', 'train_tokens_per_second': '536.8'}

  0%|          | 250/225000 [1:21:50<1184:17:38, 18.97s/it]
  0%|          | 251/225000 [1:22:05<1105:38:40, 17.71s/it]
  0%|          | 252/225000 [1:22:24<1129:16:24, 18.09s/it]
  0%|          | 253/225000 [1:22:49<1256:23:34, 20.12s/it]
  0%|          | 254/225000 [1:23:03<1145:40:26, 18.35s/it]
  0%|          | 255/225000 [1:23:24<1196:30:42, 19.17s/it]
  0%|          | 256/225000 [1:23:43<1195:47:36, 19.15s/it]
  0%|          | 257/225000 [1:23:58<1118:11:49, 17.91s/it]
  0%|          | 258/225000 [1:24:17<1125:58:58, 18.04s/it]
  0%|          | 259/225000 [1:24:37<1171:17:13, 18.76s/it]
  0%|          | 260/225000 [1:24:58<1218:40:58, 19.52s/it]
  0%|          | 261/225000 [1:25:17<1203:56:50, 19.29s/it]
  0%|          | 262/225000 [1:25:37<1223:33:19, 19.60s/it]
  0%|          | 263/225000 [1:25:59<1265:15:07, 20.27s/it]
  0%|          | 264/225000 [1:26:18<1235:25:53, 19.79s/it]
  0%|          | 265/225000 [1:26:38<1241:10:35, 19.88s/it]
  0%|          | 266/225000 [1:26:57<1223:39:06, 19.60s/it]
  0%|          | 267/225000 [1:27:16<1214:43:54, 19.46s/it]
  0%|          | 268/225000 [1:27:36<1223:38:38, 19.60s/it]
  0%|          | 269/225000 [1:27:54<1190:57:10, 19.08s/it]
  0%|          | 270/225000 [1:28:16<1247:52:56, 19.99s/it]
  0%|          | 271/225000 [1:28:34<1206:26:00, 19.33s/it]
  0%|          | 272/225000 [1:28:56<1259:35:02, 20.18s/it]
  0%|          | 273/225000 [1:29:22<1379:11:34, 22.09s/it]
  0%|          | 274/225000 [1:29:39<1275:35:08, 20.43s/it]
  0%|          | 275/225000 [1:29:56<1219:30:07, 19.54s/it]
  0%|          | 276/225000 [1:30:20<1294:27:03, 20.74s/it]
  0%|          | 277/225000 [1:30:30<1097:29:08, 17.58s/it]
  0%|          | 278/225000 [1:30:52<1170:43:13, 18.75s/it]
  0%|          | 279/225000 [1:31:08<1117:54:57, 17.91s/it]
  0%|          | 280/225000 [1:31:26<1125:58:04, 18.04s/it]
  0%|          | 281/225000 [1:31:44<1119:03:03, 17.93s/it]
  0%|          | 282/225000 [1:31:58<1061:01:47, 17.00s/it]
  0%|          | 283/225000 [1:32:16<1074:36:37, 17.22s/it]
  0%|          | 284/225000 [1:32:33<1066:50:11, 17.09s/it]
  0%|          | 285/225000 [1:32:57<1188:25:30, 19.04s/it]
  0%|          | 286/225000 [1:33:24<1346:25:37, 21.57s/it]
  0%|          | 287/225000 [1:33:45<1334:26:08, 21.38s/it]
  0%|          | 288/225000 [1:34:07<1340:31:24, 21.48s/it]
  0%|          | 289/225000 [1:34:29<1348:22:01, 21.60s/it]
  0%|          | 290/225000 [1:34:46<1275:58:20, 20.44s/it]
  0%|          | 291/225000 [1:35:06<1261:39:54, 20.21s/it]
  0%|          | 292/225000 [1:35:26<1262:49:47, 20.23s/it]
  0%|          | 293/225000 [1:35:50<1337:54:05, 21.43s/it]
  0%|          | 294/225000 [1:36:03<1172:06:57, 18.78s/it]
  0%|          | 295/225000 [1:36:26<1243:31:27, 19.92s/it]
  0%|          | 296/225000 [1:36:47<1267:57:33, 20.31s/it]
  0%|          | 297/225000 [1:37:06<1243:15:45, 19.92s/it]
  0%|          | 298/225000 [1:37:28<1290:55:07, 20.68s/it]
  0%|          | 299/225000 [1:37:43<1175:53:12, 18.84s/it]
  0%|          | 300/225000 [1:38:01<1160:40:42, 18.60s/it]
                                                           
{'loss': '1.474', 'grad_norm': '0.6545', 'learning_rate': '1.485e-06', 'epoch': '0.001333', 'num_input_tokens_seen': 3147984, 'train_runtime': '5881', 'train_tokens_per_second': '535.2'}

  0%|          | 300/225000 [1:38:01<1160:40:42, 18.60s/it]
  0%|          | 301/225000 [1:38:13<1040:04:10, 16.66s/it]
  0%|          | 302/225000 [1:38:32<1080:28:16, 17.31s/it]
  0%|          | 303/225000 [1:38:45<1009:36:15, 16.18s/it]
  0%|          | 304/225000 [1:39:15<1258:27:51, 20.16s/it]
  0%|          | 305/225000 [1:39:35<1250:24:36, 20.03s/it]
  0%|          | 306/225000 [1:39:54<1240:54:36, 19.88s/it]
  0%|          | 307/225000 [1:40:07<1101:02:17, 17.64s/it]
  0%|          | 308/225000 [1:40:22<1057:38:49, 16.95s/it]
  0%|          | 309/225000 [1:40:43<1140:56:56, 18.28s/it]
  0%|          | 310/225000 [1:41:02<1152:25:50, 18.46s/it]
  0%|          | 311/225000 [1:41:27<1279:59:00, 20.51s/it]
  0%|          | 312/225000 [1:41:43<1178:42:45, 18.89s/it]
  0%|          | 313/225000 [1:42:03<1215:32:34, 19.48s/it]
  0%|          | 314/225000 [1:42:17<1111:29:02, 17.81s/it]
  0%|          | 315/225000 [1:42:35<1102:55:54, 17.67s/it]
  0%|          | 316/225000 [1:42:48<1020:17:46, 16.35s/it]
  0%|          | 317/225000 [1:43:12<1170:09:09, 18.75s/it]
  0%|          | 318/225000 [1:43:26<1083:39:38, 17.36s/it]
  0%|          | 319/225000 [1:43:41<1026:24:14, 16.45s/it]
  0%|          | 320/225000 [1:43:52<934:39:30, 14.98s/it] 
  0%|          | 321/225000 [1:44:15<1073:27:09, 17.20s/it]
  0%|          | 322/225000 [1:44:33<1103:38:10, 17.68s/it]
  0%|          | 323/225000 [1:44:59<1254:49:05, 20.11s/it]
  0%|          | 324/225000 [1:45:08<1050:34:31, 16.83s/it]
  0%|          | 325/225000 [1:45:26<1064:09:17, 17.05s/it]
  0%|          | 326/225000 [1:45:40<1003:06:21, 16.07s/it]
  0%|          | 327/225000 [1:46:03<1134:32:30, 18.18s/it]
  0%|          | 328/225000 [1:46:14<1010:32:11, 16.19s/it]
  0%|          | 329/225000 [1:46:33<1053:18:47, 16.88s/it]
  0%|          | 330/225000 [1:46:57<1183:56:59, 18.97s/it]
  0%|          | 331/225000 [1:47:21<1291:39:47, 20.70s/it]
  0%|          | 332/225000 [1:47:41<1268:39:59, 20.33s/it]
  0%|          | 333/225000 [1:48:00<1248:19:47, 20.00s/it]
  0%|          | 334/225000 [1:48:17<1181:54:45, 18.94s/it]
  0%|          | 335/225000 [1:48:38<1220:37:37, 19.56s/it]
  0%|          | 336/225000 [1:48:56<1193:05:12, 19.12s/it]
  0%|          | 337/225000 [1:49:16<1220:29:38, 19.56s/it]
  0%|          | 338/225000 [1:49:34<1190:40:16, 19.08s/it]
  0%|          | 339/225000 [1:49:56<1244:31:22, 19.94s/it]
  0%|          | 340/225000 [1:50:16<1244:24:24, 19.94s/it]
  0%|          | 341/225000 [1:50:34<1201:53:28, 19.26s/it]
  0%|          | 342/225000 [1:51:03<1394:59:12, 22.35s/it]
  0%|          | 343/225000 [1:51:15<1201:32:16, 19.25s/it]
  0%|          | 344/225000 [1:51:37<1253:59:22, 20.09s/it]
  0%|          | 345/225000 [1:52:00<1297:34:16, 20.79s/it]
  0%|          | 346/225000 [1:52:21<1310:36:34, 21.00s/it]
  0%|          | 347/225000 [1:52:36<1192:44:54, 19.11s/it]
  0%|          | 348/225000 [1:52:52<1130:42:06, 18.12s/it]
  0%|          | 349/225000 [1:53:13<1192:21:47, 19.11s/it]
  0%|          | 350/225000 [1:53:30<1154:36:53, 18.50s/it]
                                                           
{'loss': '1.423', 'grad_norm': '0.4801', 'learning_rate': '1.735e-06', 'epoch': '0.001556', 'num_input_tokens_seen': 3642480, 'train_runtime': '6811', 'train_tokens_per_second': '534.8'}

  0%|          | 350/225000 [1:53:30<1154:36:53, 18.50s/it]
  0%|          | 351/225000 [1:53:49<1164:20:06, 18.66s/it]
  0%|          | 352/225000 [1:54:10<1203:01:49, 19.28s/it]
  0%|          | 353/225000 [1:54:28<1177:28:01, 18.87s/it]
  0%|          | 354/225000 [1:54:45<1146:04:13, 18.37s/it]
  0%|          | 355/225000 [1:55:10<1256:59:08, 20.14s/it]
  0%|          | 356/225000 [1:55:31<1281:15:09, 20.53s/it]
  0%|          | 357/225000 [1:55:52<1289:50:55, 20.67s/it]
  0%|          | 358/225000 [1:56:13<1293:03:45, 20.72s/it]
  0%|          | 359/225000 [1:56:29<1200:15:19, 19.23s/it]
  0%|          | 360/225000 [1:56:49<1226:29:11, 19.66s/it]
  0%|          | 361/225000 [1:57:08<1217:47:17, 19.52s/it]
  0%|          | 362/225000 [1:57:30<1263:18:32, 20.25s/it]
  0%|          | 363/225000 [1:57:55<1351:46:58, 21.66s/it]
  0%|          | 364/225000 [1:58:13<1283:56:08, 20.58s/it]
  0%|          | 365/225000 [1:58:33<1268:16:03, 20.33s/it]
  0%|          | 366/225000 [1:58:56<1320:08:28, 21.16s/it]
  0%|          | 367/225000 [1:59:09<1156:48:08, 18.54s/it]
  0%|          | 368/225000 [1:59:29<1183:50:00, 18.97s/it]
  0%|          | 369/225000 [1:59:50<1231:35:08, 19.74s/it]
  0%|          | 370/225000 [2:00:07<1171:16:16, 18.77s/it]
  0%|          | 371/225000 [2:00:26<1190:31:00, 19.08s/it]
  0%|          | 372/225000 [2:00:53<1321:38:09, 21.18s/it]
  0%|          | 373/225000 [2:01:16<1364:42:48, 21.87s/it]
  0%|          | 374/225000 [2:01:37<1343:23:08, 21.53s/it]
  0%|          | 375/225000 [2:01:53<1239:43:35, 19.87s/it]
  0%|          | 376/225000 [2:02:17<1318:15:41, 21.13s/it]
  0%|          | 377/225000 [2:02:36<1284:11:42, 20.58s/it]
  0%|          | 378/225000 [2:02:57<1294:04:57, 20.74s/it]
  0%|          | 379/225000 [2:03:17<1281:07:29, 20.53s/it]
  0%|          | 380/225000 [2:03:36<1240:19:04, 19.88s/it]
  0%|          | 381/225000 [2:03:52<1168:37:53, 18.73s/it]
  0%|          | 382/225000 [2:04:05<1069:45:20, 17.15s/it]
  0%|          | 383/225000 [2:04:23<1078:05:15, 17.28s/it]
  0%|          | 384/225000 [2:04:45<1173:44:07, 18.81s/it]
  0%|          | 385/225000 [2:05:01<1122:37:27, 17.99s/it]
  0%|          | 386/225000 [2:05:23<1200:06:44, 19.23s/it]
  0%|          | 387/225000 [2:05:43<1203:07:20, 19.28s/it]
  0%|          | 388/225000 [2:06:03<1228:02:37, 19.68s/it]
  0%|          | 389/225000 [2:06:14<1060:25:59, 17.00s/it]
  0%|          | 390/225000 [2:06:35<1130:48:28, 18.12s/it]
  0%|          | 391/225000 [2:06:51<1102:57:19, 17.68s/it]
  0%|          | 392/225000 [2:07:10<1109:52:07, 17.79s/it]
  0%|          | 393/225000 [2:07:29<1133:04:37, 18.16s/it]
  0%|          | 394/225000 [2:07:47<1138:18:59, 18.25s/it]
  0%|          | 395/225000 [2:08:05<1134:19:45, 18.18s/it]
  0%|          | 396/225000 [2:08:20<1080:10:40, 17.31s/it]
  0%|          | 397/225000 [2:08:35<1027:49:56, 16.47s/it]
  0%|          | 398/225000 [2:08:57<1131:03:18, 18.13s/it]
  0%|          | 399/225000 [2:09:19<1205:02:22, 19.31s/it]
  0%|          | 400/225000 [2:09:41<1265:22:12, 20.28s/it]
                                                           
{'loss': '1.401', 'grad_norm': '0.5505', 'learning_rate': '1.985e-06', 'epoch': '0.001778', 'num_input_tokens_seen': 4153584, 'train_runtime': '7782', 'train_tokens_per_second': '533.7'}

  0%|          | 400/225000 [2:09:41<1265:22:12, 20.28s/it]
  0%|          | 401/225000 [2:10:02<1261:10:33, 20.21s/it]
  0%|          | 402/225000 [2:10:25<1316:18:58, 21.10s/it]
  0%|          | 403/225000 [2:10:46<1323:40:56, 21.22s/it]
  0%|          | 404/225000 [2:11:04<1255:14:11, 20.12s/it]
  0%|          | 405/225000 [2:11:21<1194:38:23, 19.15s/it]
  0%|          | 406/225000 [2:11:39<1188:28:41, 19.05s/it]
  0%|          | 407/225000 [2:12:02<1250:52:47, 20.05s/it]
  0%|          | 408/225000 [2:12:18<1173:28:24, 18.81s/it]
  0%|          | 409/225000 [2:12:36<1156:16:45, 18.53s/it]
  0%|          | 410/225000 [2:12:55<1181:30:22, 18.94s/it]
  0%|          | 411/225000 [2:13:11<1123:16:45, 18.01s/it]
  0%|          | 412/225000 [2:13:24<1028:07:04, 16.48s/it]
  0%|          | 413/225000 [2:13:46<1133:39:56, 18.17s/it]
  0%|          | 414/225000 [2:14:05<1135:01:41, 18.19s/it]
  0%|          | 415/225000 [2:14:23<1131:50:57, 18.14s/it]
  0%|          | 416/225000 [2:14:45<1206:33:25, 19.34s/it]
  0%|          | 417/225000 [2:15:07<1258:02:19, 20.17s/it]
  0%|          | 418/225000 [2:15:24<1196:25:50, 19.18s/it]
  0%|          | 419/225000 [2:15:38<1096:17:12, 17.57s/it]
  0%|          | 420/225000 [2:16:06<1298:49:09, 20.82s/it]
  0%|          | 421/225000 [2:16:20<1171:37:14, 18.78s/it]
  0%|          | 422/225000 [2:16:37<1131:06:36, 18.13s/it]
  0%|          | 423/225000 [2:17:01<1250:21:17, 20.04s/it]
  0%|          | 424/225000 [2:17:14<1116:34:53, 17.90s/it]
  0%|          | 425/225000 [2:17:25<992:57:24, 15.92s/it] 
  0%|          | 426/225000 [2:17:54<1234:37:52, 19.79s/it]
  0%|          | 427/225000 [2:18:23<1407:54:56, 22.57s/it]
  0%|          | 428/225000 [2:18:42<1333:04:17, 21.37s/it]
  0%|          | 429/225000 [2:19:03<1321:49:52, 21.19s/it]
  0%|          | 430/225000 [2:19:26<1356:23:28, 21.74s/it]
  0%|          | 431/225000 [2:19:40<1212:46:19, 19.44s/it]
  0%|          | 432/225000 [2:20:01<1252:27:14, 20.08s/it]
  0%|          | 433/225000 [2:20:15<1142:43:51, 18.32s/it]
  0%|          | 434/225000 [2:20:29<1049:53:06, 16.83s/it]
  0%|          | 435/225000 [2:20:50<1139:34:22, 18.27s/it]
  0%|          | 436/225000 [2:21:15<1258:04:15, 20.17s/it]
  0%|          | 437/225000 [2:21:28<1122:46:03, 18.00s/it]
  0%|          | 438/225000 [2:21:49<1175:56:33, 18.85s/it]
  0%|          | 439/225000 [2:22:03<1089:41:32, 17.47s/it]
  0%|          | 440/225000 [2:22:22<1113:25:40, 17.85s/it]
  0%|          | 441/225000 [2:22:36<1042:50:46, 16.72s/it]
  0%|          | 442/225000 [2:22:56<1101:46:01, 17.66s/it]
  0%|          | 443/225000 [2:23:11<1065:45:55, 17.09s/it]
  0%|          | 444/225000 [2:23:28<1058:06:02, 16.96s/it]
  0%|          | 445/225000 [2:23:50<1153:00:36, 18.48s/it]
  0%|          | 446/225000 [2:24:15<1275:11:48, 20.44s/it]
  0%|          | 447/225000 [2:24:35<1259:23:56, 20.19s/it]
  0%|          | 448/225000 [2:24:57<1294:57:33, 20.76s/it]
  0%|          | 449/225000 [2:25:12<1196:39:54, 19.18s/it]
  0%|          | 450/225000 [2:25:32<1203:00:41, 19.29s/it]
                                                           
{'loss': '1.379', 'grad_norm': '0.4228', 'learning_rate': '2.23e-06', 'epoch': '0.002', 'num_input_tokens_seen': 4654400, 'train_runtime': '8732', 'train_tokens_per_second': '533'}

  0%|          | 450/225000 [2:25:32<1203:00:41, 19.29s/it]
  0%|          | 451/225000 [2:25:47<1122:11:02, 17.99s/it]
  0%|          | 452/225000 [2:26:05<1124:06:09, 18.02s/it]
  0%|          | 453/225000 [2:26:23<1116:21:18, 17.90s/it]
  0%|          | 454/225000 [2:26:37<1050:16:07, 16.84s/it]
  0%|          | 455/225000 [2:27:02<1202:28:19, 19.28s/it]
  0%|          | 456/225000 [2:27:30<1367:34:55, 21.93s/it]
  0%|          | 457/225000 [2:27:48<1288:46:10, 20.66s/it]
  0%|          | 458/225000 [2:28:08<1274:24:27, 20.43s/it]
  0%|          | 459/225000 [2:28:23<1171:24:47, 18.78s/it]
  0%|          | 460/225000 [2:28:39<1124:54:31, 18.04s/it]
  0%|          | 461/225000 [2:29:00<1183:55:39, 18.98s/it]
  0%|          | 462/225000 [2:29:17<1144:54:32, 18.36s/it]
  0%|          | 463/225000 [2:29:37<1178:35:29, 18.90s/it]
  0%|          | 464/225000 [2:29:56<1171:39:18, 18.79s/it]
  0%|          | 465/225000 [2:30:25<1371:08:25, 21.98s/it]
  0%|          | 466/225000 [2:30:39<1218:25:44, 19.54s/it]
  0%|          | 467/225000 [2:30:58<1206:06:52, 19.34s/it]
  0%|          | 468/225000 [2:31:24<1332:57:49, 21.37s/it]
  0%|          | 469/225000 [2:31:45<1335:08:48, 21.41s/it]
  0%|          | 470/225000 [2:32:06<1317:40:36, 21.13s/it]
  0%|          | 471/225000 [2:32:25<1278:27:09, 20.50s/it]
  0%|          | 472/225000 [2:32:49<1339:59:26, 21.48s/it]
  0%|          | 473/225000 [2:33:08<1302:50:44, 20.89s/it]
  0%|          | 474/225000 [2:33:24<1216:33:35, 19.51s/it]
  0%|          | 475/225000 [2:33:42<1179:01:53, 18.90s/it]
  0%|          | 476/225000 [2:33:59<1151:07:15, 18.46s/it]
  0%|          | 477/225000 [2:34:14<1086:20:40, 17.42s/it]
  0%|          | 478/225000 [2:34:35<1139:01:22, 18.26s/it]
  0%|          | 479/225000 [2:34:50<1086:56:03, 17.43s/it]
  0%|          | 480/225000 [2:35:22<1356:58:52, 21.76s/it]
  0%|          | 481/225000 [2:35:45<1387:55:04, 22.25s/it]
  0%|          | 482/225000 [2:35:58<1203:28:33, 19.30s/it]
  0%|          | 483/225000 [2:36:14<1146:15:07, 18.38s/it]
  0%|          | 484/225000 [2:36:36<1211:16:13, 19.42s/it]
  0%|          | 485/225000 [2:36:47<1051:33:05, 16.86s/it]
  0%|          | 486/225000 [2:36:57<925:54:28, 14.85s/it] 
  0%|          | 487/225000 [2:37:21<1101:40:26, 17.67s/it]
  0%|          | 488/225000 [2:37:37<1060:38:51, 17.01s/it]
  0%|          | 489/225000 [2:38:00<1173:22:20, 18.81s/it]
  0%|          | 490/225000 [2:38:24<1273:42:10, 20.42s/it]
  0%|          | 491/225000 [2:38:40<1186:50:26, 19.03s/it]
  0%|          | 492/225000 [2:38:56<1136:13:41, 18.22s/it]
  0%|          | 493/225000 [2:39:13<1111:11:34, 17.82s/it]
  0%|          | 494/225000 [2:39:31<1124:47:07, 18.04s/it]
  0%|          | 495/225000 [2:39:51<1154:49:02, 18.52s/it]
  0%|          | 496/225000 [2:40:00<979:42:32, 15.71s/it] 
  0%|          | 497/225000 [2:40:13<927:09:38, 14.87s/it]
  0%|          | 498/225000 [2:40:36<1087:29:51, 17.44s/it]
  0%|          | 499/225000 [2:40:56<1136:27:15, 18.22s/it]
  0%|          | 500/225000 [2:41:18<1205:29:13, 19.33s/it]
                                                           
{'loss': '1.334', 'grad_norm': '0.8552', 'learning_rate': '2.48e-06', 'epoch': '0.002222', 'num_input_tokens_seen': 5147176, 'train_runtime': '9679', 'train_tokens_per_second': '531.8'}

  0%|          | 500/225000 [2:41:18<1205:29:13, 19.33s/it]
  0%|          | 501/225000 [2:41:40<1240:04:32, 19.89s/it]
  0%|          | 502/225000 [2:41:58<1221:58:59, 19.60s/it]
  0%|          | 503/225000 [2:42:24<1337:02:12, 21.44s/it]
  0%|          | 504/225000 [2:42:40<1227:13:02, 19.68s/it]
  0%|          | 505/225000 [2:42:59<1219:25:51, 19.55s/it]
  0%|          | 506/225000 [2:43:20<1239:11:46, 19.87s/it]
  0%|          | 507/225000 [2:43:40<1242:46:05, 19.93s/it]
  0%|          | 508/225000 [2:44:01<1267:23:00, 20.32s/it]
  0%|          | 509/225000 [2:44:18<1211:47:43, 19.43s/it]
  0%|          | 510/225000 [2:44:43<1311:25:24, 21.03s/it]
  0%|          | 511/225000 [2:45:03<1293:23:53, 20.74s/it]
  0%|          | 512/225000 [2:45:09<1017:43:22, 16.32s/it]
  0%|          | 513/225000 [2:45:21<932:27:11, 14.95s/it] 
  0%|          | 514/225000 [2:45:33<870:06:18, 13.95s/it]
  0%|          | 515/225000 [2:45:56<1047:59:00, 16.81s/it]
  0%|          | 516/225000 [2:46:16<1100:55:56, 17.66s/it]
  0%|          | 517/225000 [2:46:25<941:21:02, 15.10s/it] 
  0%|          | 518/225000 [2:46:49<1121:11:00, 17.98s/it]
  0%|          | 519/225000 [2:47:01<998:59:12, 16.02s/it] 
  0%|          | 520/225000 [2:47:27<1188:48:14, 19.06s/it]
  0%|          | 521/225000 [2:47:46<1177:14:56, 18.88s/it]
  0%|          | 522/225000 [2:48:03<1151:49:52, 18.47s/it]
  0%|          | 523/225000 [2:48:22<1157:29:28, 18.56s/it]
  0%|          | 524/225000 [2:48:48<1305:10:00, 20.93s/it]
  0%|          | 525/225000 [2:49:04<1203:07:28, 19.30s/it]
  0%|          | 526/225000 [2:49:25<1246:23:14, 19.99s/it]
  0%|          | 527/225000 [2:49:44<1220:52:28, 19.58s/it]
  0%|          | 528/225000 [2:50:05<1252:41:07, 20.09s/it]
  0%|          | 529/225000 [2:50:21<1170:29:43, 18.77s/it]
  0%|          | 530/225000 [2:50:47<1313:08:36, 21.06s/it]
  0%|          | 531/225000 [2:51:03<1208:30:08, 19.38s/it]
  0%|          | 532/225000 [2:51:28<1322:31:37, 21.21s/it]
  0%|          | 533/225000 [2:51:42<1185:43:44, 19.02s/it]
  0%|          | 534/225000 [2:52:06<1273:45:43, 20.43s/it]
  0%|          | 535/225000 [2:52:24<1222:14:57, 19.60s/it]
  0%|          | 536/225000 [2:52:45<1246:57:31, 20.00s/it]
  0%|          | 537/225000 [2:52:58<1128:33:06, 18.10s/it]
  0%|          | 538/225000 [2:53:13<1070:48:05, 17.17s/it]
  0%|          | 539/225000 [2:53:31<1073:06:09, 17.21s/it]
  0%|          | 540/225000 [2:53:52<1153:29:47, 18.50s/it]
  0%|          | 541/225000 [2:54:10<1150:57:21, 18.46s/it]
  0%|          | 542/225000 [2:54:23<1038:10:09, 16.65s/it]
  0%|          | 543/225000 [2:54:44<1116:05:33, 17.90s/it]
  0%|          | 544/225000 [2:55:01<1102:06:20, 17.68s/it]
  0%|          | 545/225000 [2:55:25<1224:21:27, 19.64s/it]
  0%|          | 546/225000 [2:55:46<1250:02:45, 20.05s/it]
  0%|          | 547/225000 [2:56:08<1285:19:03, 20.62s/it]
  0%|          | 548/225000 [2:56:27<1259:28:49, 20.20s/it]
  0%|          | 549/225000 [2:56:45<1216:07:57, 19.51s/it]
  0%|          | 550/225000 [2:57:01<1139:49:30, 18.28s/it]
                                                           
{'loss': '1.309', 'grad_norm': '0.7418', 'learning_rate': '2.73e-06', 'epoch': '0.002444', 'num_input_tokens_seen': 5638120, 'train_runtime': '1.062e+04', 'train_tokens_per_second': '530.8'}

  0%|          | 550/225000 [2:57:01<1139:49:30, 18.28s/it]
  0%|          | 551/225000 [2:57:19<1143:36:12, 18.34s/it]
  0%|          | 552/225000 [2:57:33<1058:35:56, 16.98s/it]
  0%|          | 553/225000 [2:57:51<1087:04:15, 17.44s/it]
  0%|          | 554/225000 [2:58:08<1071:51:54, 17.19s/it]
  0%|          | 555/225000 [2:58:27<1108:43:09, 17.78s/it]
  0%|          | 556/225000 [2:58:45<1109:31:10, 17.80s/it]
  0%|          | 557/225000 [2:59:00<1064:56:19, 17.08s/it]
  0%|          | 558/225000 [2:59:20<1113:00:31, 17.85s/it]
  0%|          | 559/225000 [2:59:44<1221:18:29, 19.59s/it]
  0%|          | 560/225000 [3:00:02<1189:41:46, 19.08s/it]
  0%|          | 561/225000 [3:00:17<1119:45:53, 17.96s/it]
  0%|          | 562/225000 [3:00:33<1083:45:31, 17.38s/it]
  0%|          | 563/225000 [3:00:47<1020:31:55, 16.37s/it]
  0%|          | 564/225000 [3:01:07<1098:17:48, 17.62s/it]
  0%|          | 565/225000 [3:01:28<1160:23:23, 18.61s/it]
  0%|          | 566/225000 [3:01:46<1147:09:34, 18.40s/it]
  0%|          | 567/225000 [3:02:10<1251:08:11, 20.07s/it]
  0%|          | 568/225000 [3:02:25<1153:35:35, 18.50s/it]
  0%|          | 569/225000 [3:02:37<1024:52:55, 16.44s/it]
  0%|          | 570/225000 [3:03:07<1280:40:34, 20.54s/it]
  0%|          | 571/225000 [3:03:22<1188:21:15, 19.06s/it]
  0%|          | 572/225000 [3:03:37<1104:21:59, 17.71s/it]
  0%|          | 573/225000 [3:03:57<1143:35:45, 18.34s/it]
  0%|          | 574/225000 [3:04:12<1081:57:54, 17.36s/it]
  0%|          | 575/225000 [3:04:32<1131:09:07, 18.14s/it]
  0%|          | 576/225000 [3:04:48<1090:32:34, 17.49s/it]
  0%|          | 577/225000 [3:05:14<1252:02:40, 20.08s/it]
  0%|          | 578/225000 [3:05:34<1257:58:20, 20.18s/it]
  0%|          | 579/225000 [3:05:54<1250:31:28, 20.06s/it]
  0%|          | 580/225000 [3:06:15<1273:50:58, 20.43s/it]
  0%|          | 581/225000 [3:06:29<1151:37:59, 18.47s/it]
  0%|          | 582/225000 [3:06:48<1154:05:10, 18.51s/it]
  0%|          | 583/225000 [3:07:07<1157:46:45, 18.57s/it]
  0%|          | 584/225000 [3:07:31<1270:39:37, 20.38s/it]
  0%|          | 585/225000 [3:07:49<1224:30:24, 19.64s/it]
  0%|          | 586/225000 [3:08:14<1330:04:27, 21.34s/it]
  0%|          | 587/225000 [3:08:33<1268:39:12, 20.35s/it]
  0%|          | 588/225000 [3:08:49<1190:52:40, 19.10s/it]
  0%|          | 589/225000 [3:09:06<1160:18:32, 18.61s/it]
  0%|          | 590/225000 [3:09:21<1091:07:10, 17.50s/it]
  0%|          | 591/225000 [3:09:44<1198:35:59, 19.23s/it]
  0%|          | 592/225000 [3:10:03<1183:00:52, 18.98s/it]
  0%|          | 593/225000 [3:10:17<1101:01:27, 17.66s/it]
  0%|          | 594/225000 [3:10:34<1090:50:28, 17.50s/it]
  0%|          | 595/225000 [3:10:58<1197:36:00, 19.21s/it]
  0%|          | 596/225000 [3:11:16<1184:55:25, 19.01s/it]
  0%|          | 597/225000 [3:11:41<1292:33:14, 20.74s/it]
  0%|          | 598/225000 [3:11:54<1146:42:51, 18.40s/it]
  0%|          | 599/225000 [3:12:11<1121:48:56, 18.00s/it]
  0%|          | 600/225000 [3:12:29<1118:26:15, 17.94s/it]
                                                           
{'loss': '1.287', 'grad_norm': '0.5821', 'learning_rate': '2.98e-06', 'epoch': '0.002667', 'num_input_tokens_seen': 6117648, 'train_runtime': '1.155e+04', 'train_tokens_per_second': '529.7'}

  0%|          | 600/225000 [3:12:29<1118:26:15, 17.94s/it]
  0%|          | 601/225000 [3:12:45<1083:02:51, 17.38s/it]
  0%|          | 602/225000 [3:13:06<1150:23:08, 18.46s/it]
  0%|          | 603/225000 [3:13:27<1208:15:31, 19.38s/it]
  0%|          | 604/225000 [3:13:35<986:36:25, 15.83s/it] 
  0%|          | 605/225000 [3:13:49<956:26:44, 15.34s/it]
  0%|          | 606/225000 [3:14:09<1043:41:24, 16.74s/it]
  0%|          | 607/225000 [3:14:21<950:01:24, 15.24s/it] 
  0%|          | 608/225000 [3:14:42<1057:04:46, 16.96s/it]
  0%|          | 609/225000 [3:14:59<1063:05:46, 17.06s/it]
  0%|          | 610/225000 [3:15:20<1126:59:32, 18.08s/it]
  0%|          | 611/225000 [3:15:34<1056:46:28, 16.95s/it]
  0%|          | 612/225000 [3:15:50<1042:24:11, 16.72s/it]
  0%|          | 613/225000 [3:16:06<1032:57:49, 16.57s/it]
  0%|          | 614/225000 [3:16:25<1065:16:29, 17.09s/it]
  0%|          | 615/225000 [3:16:49<1204:48:42, 19.33s/it]
  0%|          | 616/225000 [3:17:11<1258:29:40, 20.19s/it]
  0%|          | 617/225000 [3:17:29<1209:48:30, 19.41s/it]
  0%|          | 618/225000 [3:17:49<1227:56:34, 19.70s/it]
  0%|          | 619/225000 [3:18:11<1256:27:25, 20.16s/it]
  0%|          | 620/225000 [3:18:31<1254:47:11, 20.13s/it]
  0%|          | 621/225000 [3:18:45<1146:22:15, 18.39s/it]
  0%|          | 622/225000 [3:18:59<1055:38:31, 16.94s/it]
  0%|          | 623/225000 [3:19:25<1242:18:22, 19.93s/it]
  0%|          | 624/225000 [3:19:40<1150:56:32, 18.47s/it]
  0%|          | 625/225000 [3:19:54<1065:41:02, 17.10s/it]
  0%|          | 626/225000 [3:20:08<1007:12:42, 16.16s/it]
  0%|          | 627/225000 [3:20:26<1044:11:11, 16.75s/it]
  0%|          | 628/225000 [3:20:44<1061:12:47, 17.03s/it]
  0%|          | 629/225000 [3:20:59<1025:29:53, 16.45s/it]
  0%|          | 630/225000 [3:21:28<1262:23:55, 20.26s/it]
  0%|          | 631/225000 [3:21:46<1204:49:19, 19.33s/it]
  0%|          | 632/225000 [3:22:07<1235:48:33, 19.83s/it]
  0%|          | 633/225000 [3:22:18<1088:10:28, 17.46s/it]
  0%|          | 634/225000 [3:22:35<1066:09:54, 17.11s/it]
  0%|          | 635/225000 [3:22:59<1193:24:13, 19.15s/it]
  0%|          | 636/225000 [3:23:14<1114:50:42, 17.89s/it]
  0%|          | 637/225000 [3:23:38<1230:03:08, 19.74s/it]
  0%|          | 638/225000 [3:23:53<1143:36:47, 18.35s/it]
  0%|          | 639/225000 [3:24:18<1280:17:43, 20.54s/it]
  0%|          | 640/225000 [3:24:35<1208:09:27, 19.39s/it]
  0%|          | 641/225000 [3:24:51<1145:55:47, 18.39s/it]
  0%|          | 642/225000 [3:25:15<1247:30:47, 20.02s/it]
  0%|          | 643/225000 [3:25:33<1204:25:35, 19.33s/it]
  0%|          | 644/225000 [3:25:49<1154:54:50, 18.53s/it]
  0%|          | 645/225000 [3:26:07<1128:15:43, 18.10s/it]
  0%|          | 646/225000 [3:26:30<1223:45:58, 19.64s/it]
  0%|          | 647/225000 [3:26:42<1087:12:45, 17.45s/it]
  0%|          | 648/225000 [3:27:03<1150:10:33, 18.46s/it]
  0%|          | 649/225000 [3:27:19<1105:28:38, 17.74s/it]
  0%|          | 650/225000 [3:27:44<1233:54:24, 19.80s/it]
                                                           
{'loss': '1.265', 'grad_norm': '1.261', 'learning_rate': '3.23e-06', 'epoch': '0.002889', 'num_input_tokens_seen': 6591960, 'train_runtime': '1.246e+04', 'train_tokens_per_second': '528.9'}

  0%|          | 650/225000 [3:27:44<1233:54:24, 19.80s/it]
  0%|          | 651/225000 [3:28:03<1233:34:58, 19.79s/it]
  0%|          | 652/225000 [3:28:23<1233:38:42, 19.80s/it]
  0%|          | 653/225000 [3:28:39<1162:54:23, 18.66s/it]
  0%|          | 654/225000 [3:28:57<1152:28:36, 18.49s/it]
  0%|          | 655/225000 [3:29:11<1061:48:55, 17.04s/it]
  0%|          | 656/225000 [3:29:23<971:32:04, 15.59s/it] 
  0%|          | 657/225000 [3:29:37<941:31:34, 15.11s/it]
  0%|          | 658/225000 [3:30:01<1097:06:28, 17.61s/it]
  0%|          | 659/225000 [3:30:26<1241:14:58, 19.92s/it]
  0%|          | 660/225000 [3:30:41<1156:31:26, 18.56s/it]
  0%|          | 661/225000 [3:31:06<1281:59:47, 20.57s/it]
  0%|          | 662/225000 [3:31:26<1259:02:19, 20.20s/it]
  0%|          | 663/225000 [3:31:51<1348:02:39, 21.63s/it]
  0%|          | 664/225000 [3:32:15<1405:02:08, 22.55s/it]
  0%|          | 665/225000 [3:32:38<1404:05:28, 22.53s/it]
  0%|          | 666/225000 [3:32:58<1356:28:45, 21.77s/it]
  0%|          | 667/225000 [3:33:16<1288:17:24, 20.67s/it]
  0%|          | 668/225000 [3:33:28<1131:10:08, 18.15s/it]
  0%|          | 669/225000 [3:33:47<1138:44:34, 18.27s/it]
  0%|          | 670/225000 [3:34:09<1209:49:28, 19.42s/it]
  0%|          | 671/225000 [3:34:30<1234:22:04, 19.81s/it]
  0%|          | 672/225000 [3:34:56<1360:09:26, 21.83s/it]
  0%|          | 673/225000 [3:35:09<1181:41:18, 18.96s/it]
  0%|          | 674/225000 [3:35:22<1083:51:47, 17.39s/it]
  0%|          | 675/225000 [3:35:48<1234:55:08, 19.82s/it]
  0%|          | 676/225000 [3:36:06<1204:14:15, 19.33s/it]
  0%|          | 677/225000 [3:36:25<1194:05:29, 19.16s/it]
  0%|          | 678/225000 [3:36:43<1181:51:36, 18.97s/it]
  0%|          | 679/225000 [3:37:01<1152:56:13, 18.50s/it]
  0%|          | 680/225000 [3:37:18<1140:36:47, 18.31s/it]
  0%|          | 681/225000 [3:37:38<1157:12:26, 18.57s/it]
  0%|          | 682/225000 [3:38:00<1229:00:51, 19.72s/it]
  0%|          | 683/225000 [3:38:20<1227:05:34, 19.69s/it]
  0%|          | 684/225000 [3:38:43<1287:28:39, 20.66s/it]
  0%|          | 685/225000 [3:38:54<1113:56:35, 17.88s/it]
  0%|          | 686/225000 [3:39:16<1199:50:21, 19.26s/it]
  0%|          | 687/225000 [3:39:31<1107:44:59, 17.78s/it]
  0%|          | 688/225000 [3:39:58<1282:00:53, 20.58s/it]
  0%|          | 689/225000 [3:40:14<1192:27:17, 19.14s/it]
  0%|          | 690/225000 [3:40:35<1223:33:07, 19.64s/it]
  0%|          | 691/225000 [3:40:51<1165:46:20, 18.71s/it]
  0%|          | 692/225000 [3:41:05<1069:35:55, 17.17s/it]
  0%|          | 693/225000 [3:41:28<1180:42:27, 18.95s/it]
  0%|          | 694/225000 [3:41:43<1114:35:49, 17.89s/it]
  0%|          | 695/225000 [3:42:00<1090:23:05, 17.50s/it]
  0%|          | 696/225000 [3:42:16<1072:35:55, 17.21s/it]
  0%|          | 697/225000 [3:42:34<1088:57:45, 17.48s/it]
  0%|          | 698/225000 [3:42:59<1220:46:53, 19.59s/it]
  0%|          | 699/225000 [3:43:20<1250:52:21, 20.08s/it]
  0%|          | 700/225000 [3:43:40<1239:25:06, 19.89s/it]
                                                           
{'loss': '1.255', 'grad_norm': '0.8851', 'learning_rate': '3.48e-06', 'epoch': '0.003111', 'num_input_tokens_seen': 7081400, 'train_runtime': '1.342e+04', 'train_tokens_per_second': '527.7'}

  0%|          | 700/225000 [3:43:40<1239:25:06, 19.89s/it]
  0%|          | 701/225000 [3:43:53<1116:02:19, 17.91s/it]
  0%|          | 702/225000 [3:44:11<1127:56:24, 18.10s/it]
  0%|          | 703/225000 [3:44:36<1242:55:15, 19.95s/it]
  0%|          | 704/225000 [3:44:56<1241:22:48, 19.92s/it]
  0%|          | 705/225000 [3:45:05<1040:27:19, 16.70s/it]
  0%|          | 706/225000 [3:45:20<1018:01:41, 16.34s/it]
  0%|          | 707/225000 [3:45:42<1116:52:25, 17.93s/it]
  0%|          | 708/225000 [3:45:58<1086:03:42, 17.43s/it]
  0%|          | 709/225000 [3:46:20<1177:12:02, 18.89s/it]
  0%|          | 710/225000 [3:46:46<1306:54:16, 20.98s/it]
  0%|          | 711/225000 [3:47:04<1251:21:45, 20.09s/it]
  0%|          | 712/225000 [3:47:20<1165:00:36, 18.70s/it]
  0%|          | 713/225000 [3:47:41<1206:18:24, 19.36s/it]
  0%|          | 714/225000 [3:48:01<1223:48:20, 19.64s/it]
  0%|          | 715/225000 [3:48:18<1173:19:55, 18.83s/it]
  0%|          | 716/225000 [3:48:35<1136:43:41, 18.25s/it]
  0%|          | 717/225000 [3:48:57<1202:33:08, 19.30s/it]
  0%|          | 718/225000 [3:49:18<1239:00:14, 19.89s/it]
  0%|          | 719/225000 [3:49:33<1145:22:58, 18.38s/it]
  0%|          | 720/225000 [3:49:45<1035:31:38, 16.62s/it]
  0%|          | 721/225000 [3:50:08<1158:03:52, 18.59s/it]
  0%|          | 722/225000 [3:50:30<1218:40:44, 19.56s/it]
  0%|          | 723/225000 [3:50:54<1291:24:00, 20.73s/it]
  0%|          | 724/225000 [3:51:03<1087:04:54, 17.45s/it]
  0%|          | 725/225000 [3:51:20<1068:54:06, 17.16s/it]
  0%|          | 726/225000 [3:51:40<1126:39:30, 18.08s/it]
  0%|          | 727/225000 [3:52:03<1214:29:41, 19.49s/it]
  0%|          | 728/225000 [3:52:23<1219:25:52, 19.57s/it]
  0%|          | 729/225000 [3:52:41<1188:22:56, 19.08s/it]
  0%|          | 730/225000 [3:52:56<1124:50:57, 18.06s/it]
  0%|          | 731/225000 [3:53:12<1089:10:50, 17.48s/it]
  0%|          | 732/225000 [3:53:33<1143:46:41, 18.36s/it]
  0%|          | 733/225000 [3:53:53<1185:11:17, 19.02s/it]
  0%|          | 734/225000 [3:54:16<1253:55:22, 20.13s/it]
  0%|          | 735/225000 [3:54:32<1169:38:10, 18.78s/it]
  0%|          | 736/225000 [3:54:56<1264:53:02, 20.30s/it]
  0%|          | 737/225000 [3:55:09<1133:05:57, 18.19s/it]
  0%|          | 738/225000 [3:55:32<1232:49:05, 19.79s/it]
  0%|          | 739/225000 [3:55:50<1196:29:10, 19.21s/it]
  0%|          | 740/225000 [3:56:08<1178:00:33, 18.91s/it]
  0%|          | 741/225000 [3:56:32<1265:31:53, 20.32s/it]
  0%|          | 742/225000 [3:56:53<1280:06:40, 20.55s/it]
  0%|          | 743/225000 [3:57:16<1327:19:02, 21.31s/it]
  0%|          | 744/225000 [3:57:32<1231:07:38, 19.76s/it]
  0%|          | 745/225000 [3:57:53<1245:31:35, 19.99s/it]
  0%|          | 746/225000 [3:58:15<1276:10:30, 20.49s/it]
  0%|          | 747/225000 [3:58:34<1249:05:26, 20.05s/it]
  0%|          | 748/225000 [3:58:54<1247:46:20, 20.03s/it]
  0%|          | 749/225000 [3:59:03<1058:38:43, 16.99s/it]
  0%|          | 750/225000 [3:59:19<1022:16:21, 16.41s/it]
                                                           
{'loss': '1.249', 'grad_norm': '0.7964', 'learning_rate': '3.73e-06', 'epoch': '0.003333', 'num_input_tokens_seen': 7564312, 'train_runtime': '1.436e+04', 'train_tokens_per_second': '526.8'}

  0%|          | 750/225000 [3:59:19<1022:16:21, 16.41s/it]
  0%|          | 751/225000 [3:59:35<1032:27:21, 16.57s/it]
  0%|          | 752/225000 [3:59:52<1030:46:49, 16.55s/it]
  0%|          | 753/225000 [4:00:10<1067:15:28, 17.13s/it]
  0%|          | 754/225000 [4:00:24<1007:00:22, 16.17s/it]
  0%|          | 755/225000 [4:00:47<1129:47:09, 18.14s/it]
  0%|          | 756/225000 [4:01:05<1124:32:46, 18.05s/it]
  0%|          | 757/225000 [4:01:27<1201:50:59, 19.29s/it]
  0%|          | 758/225000 [4:01:36<1006:41:36, 16.16s/it]
  0%|          | 759/225000 [4:01:54<1033:48:19, 16.60s/it]
  0%|          | 760/225000 [4:02:08<999:27:42, 16.05s/it] 
  0%|          | 761/225000 [4:02:28<1067:07:44, 17.13s/it]
  0%|          | 762/225000 [4:02:53<1220:46:20, 19.60s/it]
  0%|          | 763/225000 [4:03:12<1201:00:28, 19.28s/it]
  0%|          | 764/225000 [4:03:27<1122:27:03, 18.02s/it]
  0%|          | 765/225000 [4:03:39<1009:25:42, 16.21s/it]
  0%|          | 766/225000 [4:03:57<1036:28:20, 16.64s/it]
  0%|          | 767/225000 [4:04:16<1079:43:01, 17.33s/it]
  0%|          | 768/225000 [4:04:32<1069:01:10, 17.16s/it]
  0%|          | 769/225000 [4:04:50<1073:17:09, 17.23s/it]
  0%|          | 770/225000 [4:05:11<1139:07:09, 18.29s/it]
  0%|          | 771/225000 [4:05:29<1137:44:26, 18.27s/it]
  0%|          | 772/225000 [4:05:37<960:11:20, 15.42s/it] 
  0%|          | 773/225000 [4:06:03<1143:24:10, 18.36s/it]
  0%|          | 774/225000 [4:06:19<1096:41:22, 17.61s/it]
  0%|          | 775/225000 [4:06:37<1112:58:37, 17.87s/it]
  0%|          | 776/225000 [4:06:57<1150:38:02, 18.47s/it]
  0%|          | 777/225000 [4:07:15<1140:54:58, 18.32s/it]
  0%|          | 778/225000 [4:07:31<1091:08:24, 17.52s/it]
  0%|          | 779/225000 [4:07:56<1242:05:28, 19.94s/it]
  0%|          | 780/225000 [4:08:07<1070:18:29, 17.18s/it]
  0%|          | 781/225000 [4:08:31<1202:22:50, 19.31s/it]
  0%|          | 782/225000 [4:08:49<1166:52:43, 18.74s/it]
  0%|          | 783/225000 [4:09:04<1107:48:34, 17.79s/it]
  0%|          | 784/225000 [4:09:26<1177:07:10, 18.90s/it]
  0%|          | 785/225000 [4:09:45<1194:13:08, 19.17s/it]
  0%|          | 786/225000 [4:10:11<1319:55:32, 21.19s/it]
  0%|          | 787/225000 [4:10:32<1316:21:11, 21.14s/it]
  0%|          | 788/225000 [4:10:57<1386:50:12, 22.27s/it]
  0%|          | 789/225000 [4:11:12<1245:31:41, 20.00s/it]
  0%|          | 790/225000 [4:11:25<1109:54:16, 17.82s/it]
  0%|          | 791/225000 [4:11:54<1325:10:14, 21.28s/it]
  0%|          | 792/225000 [4:12:12<1255:29:58, 20.16s/it]
  0%|          | 793/225000 [4:12:26<1145:34:31, 18.39s/it]
  0%|          | 794/225000 [4:12:42<1096:50:26, 17.61s/it]
  0%|          | 795/225000 [4:12:52<958:50:13, 15.40s/it] 
  0%|          | 796/225000 [4:13:14<1082:37:10, 17.38s/it]
  0%|          | 797/225000 [4:13:32<1092:16:38, 17.54s/it]
  0%|          | 798/225000 [4:13:57<1238:41:39, 19.89s/it]
  0%|          | 799/225000 [4:14:16<1210:26:11, 19.44s/it]
  0%|          | 800/225000 [4:14:31<1143:52:13, 18.37s/it]
                                                           
{'loss': '1.216', 'grad_norm': '0.9676', 'learning_rate': '3.98e-06', 'epoch': '0.003556', 'num_input_tokens_seen': 8027968, 'train_runtime': '1.527e+04', 'train_tokens_per_second': '525.7'}

  0%|          | 800/225000 [4:14:31<1143:52:13, 18.37s/it]
  0%|          | 801/225000 [4:14:55<1241:44:42, 19.94s/it]
  0%|          | 802/225000 [4:15:06<1072:12:17, 17.22s/it]
  0%|          | 803/225000 [4:15:29<1176:52:52, 18.90s/it]
  0%|          | 804/225000 [4:15:42<1076:02:09, 17.28s/it]
  0%|          | 805/225000 [4:15:56<1016:29:18, 16.32s/it]
  0%|          | 806/225000 [4:16:21<1164:04:35, 18.69s/it]
  0%|          | 807/225000 [4:16:36<1102:49:47, 17.71s/it]
  0%|          | 808/225000 [4:16:55<1129:44:11, 18.14s/it]
  0%|          | 809/225000 [4:17:15<1170:43:06, 18.80s/it]
  0%|          | 810/225000 [4:17:29<1074:22:17, 17.25s/it]
  0%|          | 811/225000 [4:17:46<1074:20:02, 17.25s/it]
  0%|          | 812/225000 [4:18:12<1239:52:50, 19.91s/it]
  0%|          | 813/225000 [4:18:33<1245:55:51, 20.01s/it]
  0%|          | 814/225000 [4:18:49<1170:07:42, 18.79s/it]
  0%|          | 815/225000 [4:19:10<1223:45:22, 19.65s/it]
  0%|          | 816/225000 [4:19:26<1157:11:14, 18.58s/it]
  0%|          | 817/225000 [4:19:39<1038:16:07, 16.67s/it]
  0%|          | 818/225000 [4:19:57<1062:08:18, 17.06s/it]
  0%|          | 819/225000 [4:20:04<882:55:10, 14.18s/it] 
  0%|          | 820/225000 [4:20:24<999:53:15, 16.06s/it]
  0%|          | 821/225000 [4:20:45<1092:44:35, 17.55s/it]
  0%|          | 822/225000 [4:21:02<1072:54:21, 17.23s/it]
  0%|          | 823/225000 [4:21:14<968:46:22, 15.56s/it] 
  0%|          | 824/225000 [4:21:33<1049:03:55, 16.85s/it]
  0%|          | 825/225000 [4:21:53<1097:19:13, 17.62s/it]
  0%|          | 826/225000 [4:22:16<1192:53:12, 19.16s/it]
  0%|          | 827/225000 [4:22:30<1095:39:53, 17.60s/it]
  0%|          | 828/225000 [4:22:49<1133:09:23, 18.20s/it]
  0%|          | 829/225000 [4:23:02<1034:55:12, 16.62s/it]
  0%|          | 830/225000 [4:23:21<1081:07:43, 17.36s/it]
  0%|          | 831/225000 [4:23:34<1000:05:36, 16.06s/it]
  0%|          | 832/225000 [4:23:59<1153:30:22, 18.52s/it]
  0%|          | 833/225000 [4:24:13<1070:33:58, 17.19s/it]
  0%|          | 834/225000 [4:24:29<1063:58:36, 17.09s/it]
  0%|          | 835/225000 [4:24:39<920:34:29, 14.78s/it] 
  0%|          | 836/225000 [4:25:03<1091:54:56, 17.54s/it]
  0%|          | 837/225000 [4:25:17<1035:47:41, 16.63s/it]
  0%|          | 838/225000 [4:25:40<1146:56:38, 18.42s/it]
  0%|          | 839/225000 [4:26:03<1239:39:20, 19.91s/it]
  0%|          | 840/225000 [4:26:19<1165:52:20, 18.72s/it]
  0%|          | 841/225000 [4:26:37<1156:41:12, 18.58s/it]
  0%|          | 842/225000 [4:26:51<1052:51:57, 16.91s/it]
  0%|          | 843/225000 [4:27:13<1155:03:22, 18.55s/it]
  0%|          | 844/225000 [4:27:30<1134:06:50, 18.21s/it]
  0%|          | 845/225000 [4:27:45<1062:34:50, 17.07s/it]
  0%|          | 846/225000 [4:28:06<1147:36:50, 18.43s/it]
  0%|          | 847/225000 [4:28:29<1222:29:23, 19.63s/it]
  0%|          | 848/225000 [4:28:48<1209:37:46, 19.43s/it]
  0%|          | 849/225000 [4:29:03<1138:08:36, 18.28s/it]
  0%|          | 850/225000 [4:29:23<1172:19:29, 18.83s/it]
                                                           
{'loss': '1.219', 'grad_norm': '1.112', 'learning_rate': '4.23e-06', 'epoch': '0.003778', 'num_input_tokens_seen': 8487312, 'train_runtime': '1.616e+04', 'train_tokens_per_second': '525.1'}

  0%|          | 850/225000 [4:29:23<1172:19:29, 18.83s/it]
  0%|          | 851/225000 [4:29:40<1124:02:23, 18.05s/it]
  0%|          | 852/225000 [4:29:59<1157:00:00, 18.58s/it]
  0%|          | 853/225000 [4:30:18<1154:33:05, 18.54s/it]
  0%|          | 854/225000 [4:30:38<1191:23:00, 19.13s/it]
  0%|          | 855/225000 [4:30:49<1029:17:45, 16.53s/it]
  0%|          | 856/225000 [4:31:11<1126:29:31, 18.09s/it]
  0%|          | 857/225000 [4:31:30<1152:44:09, 18.51s/it]
  0%|          | 858/225000 [4:31:50<1174:17:07, 18.86s/it]
  0%|          | 859/225000 [4:32:05<1106:20:23, 17.77s/it]
  0%|          | 860/225000 [4:32:28<1211:29:00, 19.46s/it]
  0%|          | 861/225000 [4:32:49<1226:31:17, 19.70s/it]
  0%|          | 862/225000 [4:32:55<976:09:37, 15.68s/it] 
  0%|          | 863/225000 [4:33:12<1002:09:07, 16.10s/it]
  0%|          | 864/225000 [4:33:35<1129:56:56, 18.15s/it]
  0%|          | 865/225000 [4:33:51<1083:37:04, 17.40s/it]
  0%|          | 866/225000 [4:34:14<1194:21:03, 19.18s/it]
  0%|          | 867/225000 [4:34:29<1121:05:51, 18.01s/it]
  0%|          | 868/225000 [4:34:55<1272:03:39, 20.43s/it]
  0%|          | 869/225000 [4:35:15<1253:23:06, 20.13s/it]
  0%|          | 870/225000 [4:35:32<1198:23:01, 19.25s/it]
  0%|          | 871/225000 [4:35:50<1180:44:35, 18.97s/it]
  0%|          | 872/225000 [4:36:07<1142:10:25, 18.35s/it]
  0%|          | 873/225000 [4:36:27<1178:32:29, 18.93s/it]
  0%|          | 874/225000 [4:36:47<1192:13:03, 19.15s/it]
  0%|          | 875/225000 [4:37:07<1204:23:56, 19.35s/it]
  0%|          | 876/225000 [4:37:23<1151:13:37, 18.49s/it]
  0%|          | 877/225000 [4:37:36<1049:11:27, 16.85s/it]
  0%|          | 878/225000 [4:37:59<1150:48:05, 18.48s/it]
  0%|          | 879/225000 [4:38:16<1128:19:12, 18.12s/it]
  0%|          | 880/225000 [4:38:34<1118:39:13, 17.97s/it]
  0%|          | 881/225000 [4:38:49<1061:58:32, 17.06s/it]
  0%|          | 882/225000 [4:39:10<1142:44:23, 18.36s/it]
  0%|          | 883/225000 [4:39:29<1159:26:19, 18.62s/it]
  0%|          | 884/225000 [4:39:46<1117:42:25, 17.95s/it]
  0%|          | 885/225000 [4:39:59<1031:20:43, 16.57s/it]
  0%|          | 886/225000 [4:40:17<1065:16:52, 17.11s/it]
  0%|          | 887/225000 [4:40:34<1053:16:20, 16.92s/it]
  0%|          | 888/225000 [4:40:50<1046:44:22, 16.81s/it]
  0%|          | 889/225000 [4:41:18<1252:14:18, 20.12s/it]
  0%|          | 890/225000 [4:41:34<1171:00:53, 18.81s/it]
  0%|          | 891/225000 [4:41:45<1018:02:08, 16.35s/it]
  0%|          | 892/225000 [4:42:01<1025:59:48, 16.48s/it]
  0%|          | 893/225000 [4:42:25<1166:42:36, 18.74s/it]
  0%|          | 894/225000 [4:42:37<1043:09:03, 16.76s/it]
  0%|          | 895/225000 [4:42:52<1008:51:27, 16.21s/it]
  0%|          | 896/225000 [4:43:10<1027:48:04, 16.51s/it]
  0%|          | 897/225000 [4:43:28<1053:43:01, 16.93s/it]
  0%|          | 898/225000 [4:43:49<1132:48:12, 18.20s/it]
  0%|          | 899/225000 [4:44:12<1235:24:33, 19.85s/it]
  0%|          | 900/225000 [4:44:28<1152:03:52, 18.51s/it]
                                                           
{'loss': '1.192', 'grad_norm': '0.9092', 'learning_rate': '4.48e-06', 'epoch': '0.004', 'num_input_tokens_seen': 8949536, 'train_runtime': '1.707e+04', 'train_tokens_per_second': '524.3'}

  0%|          | 900/225000 [4:44:28<1152:03:52, 18.51s/it]
  0%|          | 901/225000 [4:44:42<1079:41:00, 17.34s/it]
  0%|          | 902/225000 [4:44:57<1036:05:42, 16.64s/it]
  0%|          | 903/225000 [4:45:13<1008:35:28, 16.20s/it]
  0%|          | 904/225000 [4:45:25<933:20:57, 14.99s/it] 
  0%|          | 905/225000 [4:45:51<1139:52:23, 18.31s/it]
  0%|          | 906/225000 [4:46:04<1040:55:06, 16.72s/it]
  0%|          | 907/225000 [4:46:17<969:07:43, 15.57s/it] 
  0%|          | 908/225000 [4:46:44<1180:33:37, 18.97s/it]
  0%|          | 909/225000 [4:47:04<1209:52:33, 19.44s/it]
  0%|          | 910/225000 [4:47:24<1214:43:25, 19.51s/it]
  0%|          | 911/225000 [4:47:39<1133:24:19, 18.21s/it]
  0%|          | 912/225000 [4:47:52<1029:49:50, 16.54s/it]
  0%|          | 913/225000 [4:48:14<1129:22:36, 18.14s/it]
  0%|          | 914/225000 [4:48:31<1113:30:03, 17.89s/it]
  0%|          | 915/225000 [4:48:49<1112:24:57, 17.87s/it]
  0%|          | 916/225000 [4:49:09<1158:24:41, 18.61s/it]
  0%|          | 917/225000 [4:49:23<1077:33:15, 17.31s/it]
  0%|          | 918/225000 [4:49:32<921:07:21, 14.80s/it] 
  0%|          | 919/225000 [4:49:53<1041:40:08, 16.74s/it]
  0%|          | 920/225000 [4:50:12<1071:25:24, 17.21s/it]
  0%|          | 921/225000 [4:50:29<1073:21:37, 17.24s/it]
  0%|          | 922/225000 [4:50:41<980:23:56, 15.75s/it] 
  0%|          | 923/225000 [4:50:53<911:47:02, 14.65s/it]
  0%|          | 924/225000 [4:51:04<834:54:54, 13.41s/it]
  0%|          | 925/225000 [4:51:17<836:35:59, 13.44s/it]
  0%|          | 926/225000 [4:51:39<989:05:09, 15.89s/it]
  0%|          | 927/225000 [4:51:54<963:22:43, 15.48s/it]
  0%|          | 928/225000 [4:52:18<1137:20:58, 18.27s/it]
  0%|          | 929/225000 [4:52:31<1028:33:36, 16.53s/it]
  0%|          | 930/225000 [4:52:54<1147:09:47, 18.43s/it]
  0%|          | 931/225000 [4:53:03<970:44:01, 15.60s/it] 
  0%|          | 932/225000 [4:53:15<915:56:40, 14.72s/it]
  0%|          | 933/225000 [4:53:30<911:16:54, 14.64s/it]
  0%|          | 934/225000 [4:53:53<1072:29:56, 17.23s/it]
  0%|          | 935/225000 [4:54:09<1044:51:02, 16.79s/it]
  0%|          | 936/225000 [4:54:28<1082:21:19, 17.39s/it]
  0%|          | 937/225000 [4:54:41<1000:40:25, 16.08s/it]
  0%|          | 938/225000 [4:55:03<1110:50:42, 17.85s/it]
  0%|          | 939/225000 [4:55:15<1004:49:13, 16.14s/it]
  0%|          | 940/225000 [4:55:39<1156:40:38, 18.58s/it]
  0%|          | 941/225000 [4:55:57<1140:31:31, 18.33s/it]
  0%|          | 942/225000 [4:56:17<1171:19:27, 18.82s/it]
  0%|          | 943/225000 [4:56:27<1014:45:45, 16.30s/it]
  0%|          | 944/225000 [4:56:38<918:54:16, 14.76s/it] 
  0%|          | 945/225000 [4:57:00<1056:02:14, 16.97s/it]
  0%|          | 946/225000 [4:57:16<1024:32:45, 16.46s/it]
  0%|          | 947/225000 [4:57:30<983:54:30, 15.81s/it] 
  0%|          | 948/225000 [4:57:53<1109:43:06, 17.83s/it]
  0%|          | 949/225000 [4:58:08<1072:50:13, 17.24s/it]
  0%|          | 950/225000 [4:58:30<1158:52:26, 18.62s/it]
                                                           
{'loss': '1.208', 'grad_norm': '0.6871', 'learning_rate': '4.73e-06', 'epoch': '0.004222', 'num_input_tokens_seen': 9388720, 'train_runtime': '1.791e+04', 'train_tokens_per_second': '524.2'}

  0%|          | 950/225000 [4:58:30<1158:52:26, 18.62s/it]
  0%|          | 951/225000 [4:58:46<1113:14:44, 17.89s/it]
  0%|          | 952/225000 [4:59:00<1028:22:12, 16.52s/it]
  0%|          | 953/225000 [4:59:08<876:18:55, 14.08s/it] 
  0%|          | 954/225000 [4:59:17<774:22:08, 12.44s/it]
  0%|          | 955/225000 [4:59:37<912:54:48, 14.67s/it]
  0%|          | 956/225000 [4:59:50<891:37:32, 14.33s/it]
  0%|          | 957/225000 [5:00:04<889:04:52, 14.29s/it]
  0%|          | 958/225000 [5:00:22<945:24:08, 15.19s/it]
  0%|          | 959/225000 [5:00:39<980:19:45, 15.75s/it]
  0%|          | 960/225000 [5:00:55<992:41:38, 15.95s/it]
  0%|          | 961/225000 [5:01:13<1027:40:18, 16.51s/it]
  0%|          | 962/225000 [5:01:30<1037:48:44, 16.68s/it]
  0%|          | 963/225000 [5:01:46<1031:24:55, 16.57s/it]
  0%|          | 964/225000 [5:01:57<923:14:14, 14.84s/it] 
  0%|          | 965/225000 [5:02:12<930:28:43, 14.95s/it]
  0%|          | 966/225000 [5:02:35<1079:31:07, 17.35s/it]
  0%|          | 967/225000 [5:02:48<988:03:28, 15.88s/it] 
  0%|          | 968/225000 [5:03:06<1032:01:58, 16.58s/it]
  0%|          | 969/225000 [5:03:27<1107:40:04, 17.80s/it]
  0%|          | 970/225000 [5:03:52<1243:40:29, 19.98s/it]
  0%|          | 971/225000 [5:04:12<1255:39:50, 20.18s/it]
  0%|          | 972/225000 [5:04:23<1078:06:18, 17.32s/it]
  0%|          | 973/225000 [5:04:46<1175:10:56, 18.88s/it]
  0%|          | 974/225000 [5:04:59<1069:48:47, 17.19s/it]
  0%|          | 975/225000 [5:05:19<1121:21:34, 18.02s/it]
  0%|          | 976/225000 [5:05:36<1115:18:27, 17.92s/it]
  0%|          | 977/225000 [5:05:52<1072:26:15, 17.23s/it]
  0%|          | 978/225000 [5:06:14<1155:54:34, 18.58s/it]
  0%|          | 979/225000 [5:06:30<1111:37:20, 17.86s/it]
  0%|          | 980/225000 [5:06:49<1126:03:30, 18.10s/it]
  0%|          | 981/225000 [5:07:03<1048:53:33, 16.86s/it]
  0%|          | 982/225000 [5:07:17<1006:58:43, 16.18s/it]
  0%|          | 983/225000 [5:07:37<1071:53:48, 17.23s/it]
  0%|          | 984/225000 [5:07:50<1001:45:06, 16.10s/it]
  0%|          | 985/225000 [5:08:06<1000:20:16, 16.08s/it]
  0%|          | 986/225000 [5:08:19<937:36:27, 15.07s/it] 
  0%|          | 987/225000 [5:08:42<1078:06:15, 17.33s/it]
  0%|          | 988/225000 [5:08:59<1074:02:55, 17.26s/it]
  0%|          | 989/225000 [5:09:09<946:43:13, 15.21s/it] 
  0%|          | 990/225000 [5:09:20<867:14:39, 13.94s/it]
  0%|          | 991/225000 [5:09:37<926:13:04, 14.89s/it]
  0%|          | 992/225000 [5:09:58<1027:16:41, 16.51s/it]
  0%|          | 993/225000 [5:10:15<1039:48:38, 16.71s/it]
  0%|          | 994/225000 [5:10:33<1060:37:52, 17.05s/it]
  0%|          | 995/225000 [5:10:48<1024:23:13, 16.46s/it]
  0%|          | 996/225000 [5:11:07<1071:54:41, 17.23s/it]
  0%|          | 997/225000 [5:11:28<1156:13:40, 18.58s/it]
  0%|          | 998/225000 [5:11:46<1142:53:11, 18.37s/it]
  0%|          | 999/225000 [5:12:05<1155:09:23, 18.56s/it]
  0%|          | 1000/225000 [5:12:28<1234:23:55, 19.84s/it]
                                                            
{'loss': '1.179', 'grad_norm': '1.283', 'learning_rate': '4.98e-06', 'epoch': '0.004444', 'num_input_tokens_seen': 9821976, 'train_runtime': '1.875e+04', 'train_tokens_per_second': '523.9'}

  0%|          | 1000/225000 [5:12:28<1234:23:55, 19.84s/it]
  0%|          | 1001/225000 [5:12:44<1151:38:46, 18.51s/it]
  0%|          | 1002/225000 [5:13:00<1106:07:57, 17.78s/it]
  0%|          | 1003/225000 [5:13:14<1045:27:56, 16.80s/it]
  0%|          | 1004/225000 [5:13:28<999:01:23, 16.06s/it] 
  0%|          | 1005/225000 [5:13:47<1039:55:16, 16.71s/it]
  0%|          | 1006/225000 [5:14:01<993:31:24, 15.97s/it] 
  0%|          | 1007/225000 [5:14:16<968:26:40, 15.56s/it]
  0%|          | 1008/225000 [5:14:34<1018:15:50, 16.37s/it]
  0%|          | 1009/225000 [5:14:49<995:24:54, 16.00s/it] 
  0%|          | 1010/225000 [5:15:05<1002:08:19, 16.11s/it]
  0%|          | 1011/225000 [5:15:23<1031:40:40, 16.58s/it]
  0%|          | 1012/225000 [5:15:37<984:23:28, 15.82s/it] 
  0%|          | 1013/225000 [5:15:58<1085:28:21, 17.45s/it]
  0%|          | 1014/225000 [5:16:13<1028:58:18, 16.54s/it]
  0%|          | 1015/225000 [5:16:28<1002:35:42, 16.11s/it]
  0%|          | 1016/225000 [5:16:44<1003:03:04, 16.12s/it]
  0%|          | 1017/225000 [5:17:02<1048:32:49, 16.85s/it]
  0%|          | 1018/225000 [5:17:11<900:26:17, 14.47s/it] 
  0%|          | 1019/225000 [5:17:32<1009:15:31, 16.22s/it]
  0%|          | 1020/225000 [5:17:47<994:40:03, 15.99s/it] 
  0%|          | 1021/225000 [5:18:06<1043:07:20, 16.77s/it]
  0%|          | 1022/225000 [5:18:24<1062:40:42, 17.08s/it]
  0%|          | 1023/225000 [5:18:44<1129:49:33, 18.16s/it]
  0%|          | 1024/225000 [5:19:05<1182:03:29, 19.00s/it]
  0%|          | 1025/225000 [5:19:20<1110:11:03, 17.84s/it]
  0%|          | 1026/225000 [5:19:42<1187:28:30, 19.09s/it]
  0%|          | 1027/225000 [5:20:03<1211:10:32, 19.47s/it]
  0%|          | 1028/225000 [5:20:24<1237:40:16, 19.89s/it]
  0%|          | 1029/225000 [5:20:38<1134:14:33, 18.23s/it]
  0%|          | 1030/225000 [5:21:01<1231:20:42, 19.79s/it]
  0%|          | 1031/225000 [5:21:18<1179:02:16, 18.95s/it]
  0%|          | 1032/225000 [5:21:40<1238:26:48, 19.91s/it]
  0%|          | 1033/225000 [5:21:57<1182:03:43, 19.00s/it]
  0%|          | 1034/225000 [5:22:14<1128:58:40, 18.15s/it]
  0%|          | 1035/225000 [5:22:28<1069:31:53, 17.19s/it]
  0%|          | 1036/225000 [5:22:42<1000:52:20, 16.09s/it]
  0%|          | 1037/225000 [5:22:57<976:52:02, 15.70s/it] 
  0%|          | 1038/225000 [5:23:03<795:53:11, 12.79s/it]
  0%|          | 1039/225000 [5:23:18<839:37:39, 13.50s/it]
  0%|          | 1040/225000 [5:23:39<971:52:39, 15.62s/it]
  0%|          | 1041/225000 [5:23:57<1022:15:21, 16.43s/it]
  0%|          | 1042/225000 [5:24:17<1087:46:29, 17.49s/it]
  0%|          | 1043/225000 [5:24:32<1050:27:27, 16.89s/it]
  0%|          | 1044/225000 [5:24:45<979:00:27, 15.74s/it] 
  0%|          | 1045/225000 [5:25:01<972:09:46, 15.63s/it]
  0%|          | 1046/225000 [5:25:18<1011:02:17, 16.25s/it]
  0%|          | 1047/225000 [5:25:41<1127:23:58, 18.12s/it]
  0%|          | 1048/225000 [5:26:02<1177:09:46, 18.92s/it]
  0%|          | 1049/225000 [5:26:27<1289:46:40, 20.73s/it]
  0%|          | 1050/225000 [5:26:40<1157:11:04, 18.60s/it]
                                                            
{'loss': '1.174', 'grad_norm': '0.6847', 'learning_rate': '5.23e-06', 'epoch': '0.004667', 'num_input_tokens_seen': 10261256, 'train_runtime': '1.96e+04', 'train_tokens_per_second': '523.5'}

  0%|          | 1050/225000 [5:26:40<1157:11:04, 18.60s/it]
  0%|          | 1051/225000 [5:26:55<1089:03:09, 17.51s/it]
  0%|          | 1052/225000 [5:27:09<1024:26:15, 16.47s/it]
  0%|          | 1053/225000 [5:27:22<950:14:27, 15.28s/it] 
  0%|          | 1054/225000 [5:27:44<1070:48:32, 17.21s/it]
  0%|          | 1055/225000 [5:27:59<1033:13:05, 16.61s/it]
  0%|          | 1056/225000 [5:28:17<1070:17:14, 17.21s/it]
  0%|          | 1057/225000 [5:28:32<1021:34:11, 16.42s/it]
  0%|          | 1058/225000 [5:28:54<1129:29:32, 18.16s/it]
  0%|          | 1059/225000 [5:29:06<1021:13:49, 16.42s/it]
  0%|          | 1060/225000 [5:29:26<1070:23:30, 17.21s/it]
  0%|          | 1061/225000 [5:29:35<924:39:07, 14.86s/it] 
  0%|          | 1062/225000 [5:29:51<940:41:18, 15.12s/it]
  0%|          | 1063/225000 [5:30:10<1013:42:32, 16.30s/it]
  0%|          | 1064/225000 [5:30:35<1188:27:41, 19.11s/it]
  0%|          | 1065/225000 [5:30:52<1144:17:03, 18.40s/it]
  0%|          | 1066/225000 [5:31:05<1035:41:51, 16.65s/it]
  0%|          | 1067/225000 [5:31:30<1206:14:12, 19.39s/it]
  0%|          | 1068/225000 [5:31:50<1204:49:57, 19.37s/it]
  0%|          | 1069/225000 [5:32:09<1205:26:07, 19.38s/it]
  0%|          | 1070/225000 [5:32:23<1101:15:50, 17.70s/it]
  0%|          | 1071/225000 [5:32:42<1127:36:00, 18.13s/it]
  0%|          | 1072/225000 [5:32:58<1080:15:26, 17.37s/it]
  0%|          | 1073/225000 [5:33:14<1064:15:47, 17.11s/it]
  0%|          | 1074/225000 [5:33:26<960:22:23, 15.44s/it] 
  0%|          | 1075/225000 [5:33:40<947:13:25, 15.23s/it]
  0%|          | 1076/225000 [5:33:53<895:04:07, 14.39s/it]
  0%|          | 1077/225000 [5:34:13<1009:01:14, 16.22s/it]
  0%|          | 1078/225000 [5:34:30<1010:16:23, 16.24s/it]
  0%|          | 1079/225000 [5:34:45<997:14:34, 16.03s/it] 
  0%|          | 1080/225000 [5:35:08<1125:55:32, 18.10s/it]
  0%|          | 1081/225000 [5:35:35<1286:45:46, 20.69s/it]
  0%|          | 1082/225000 [5:35:54<1248:47:39, 20.08s/it]
  0%|          | 1083/225000 [5:36:12<1222:30:15, 19.65s/it]
  0%|          | 1084/225000 [5:36:24<1072:21:07, 17.24s/it]
  0%|          | 1085/225000 [5:36:43<1114:05:14, 17.91s/it]
  0%|          | 1086/225000 [5:37:04<1170:43:57, 18.82s/it]
  0%|          | 1087/225000 [5:37:19<1099:40:03, 17.68s/it]
  0%|          | 1088/225000 [5:37:32<1014:49:23, 16.32s/it]
  0%|          | 1089/225000 [5:37:52<1069:32:20, 17.20s/it]
  0%|          | 1090/225000 [5:38:07<1028:54:13, 16.54s/it]
  0%|          | 1091/225000 [5:38:18<939:12:47, 15.10s/it] 
  0%|          | 1092/225000 [5:38:28<836:14:08, 13.45s/it]
  0%|          | 1093/225000 [5:38:47<941:49:27, 15.14s/it]
  0%|          | 1094/225000 [5:39:01<910:24:36, 14.64s/it]
  0%|          | 1095/225000 [5:39:24<1082:49:28, 17.41s/it]
  0%|          | 1096/225000 [5:39:43<1113:19:21, 17.90s/it]
  0%|          | 1097/225000 [5:39:59<1076:49:27, 17.31s/it]
  0%|          | 1098/225000 [5:40:16<1057:39:24, 17.01s/it]
  0%|          | 1099/225000 [5:40:28<971:46:53, 15.62s/it] 
  0%|          | 1100/225000 [5:40:55<1177:26:32, 18.93s/it]
                                                            
{'loss': '1.179', 'grad_norm': '1.277', 'learning_rate': '5.48e-06', 'epoch': '0.004889', 'num_input_tokens_seen': 10709464, 'train_runtime': '2.046e+04', 'train_tokens_per_second': '523.6'}

  0%|          | 1100/225000 [5:40:55<1177:26:32, 18.93s/it]
  0%|          | 1101/225000 [5:41:19<1283:00:52, 20.63s/it]
  0%|          | 1102/225000 [5:41:30<1096:53:01, 17.64s/it]
  0%|          | 1103/225000 [5:41:48<1097:56:27, 17.65s/it]
  0%|          | 1104/225000 [5:42:02<1039:53:18, 16.72s/it]
  0%|          | 1105/225000 [5:42:26<1163:47:59, 18.71s/it]
  0%|          | 1106/225000 [5:42:40<1089:33:16, 17.52s/it]
  0%|          | 1107/225000 [5:42:57<1075:53:22, 17.30s/it]
  0%|          | 1108/225000 [5:43:14<1063:25:04, 17.10s/it]
  0%|          | 1109/225000 [5:43:21<882:17:35, 14.19s/it] 
  0%|          | 1110/225000 [5:43:38<935:18:56, 15.04s/it]
  0%|          | 1111/225000 [5:43:55<965:04:17, 15.52s/it]
  0%|          | 1112/225000 [5:44:11<981:40:39, 15.78s/it]
  0%|          | 1113/225000 [5:44:25<944:53:36, 15.19s/it]
  0%|          | 1114/225000 [5:44:47<1064:44:59, 17.12s/it]
  0%|          | 1115/225000 [5:45:10<1184:25:02, 19.05s/it]
  0%|          | 1116/225000 [5:45:28<1159:00:57, 18.64s/it]
  0%|          | 1117/225000 [5:45:36<968:09:40, 15.57s/it] 
  0%|          | 1118/225000 [5:45:54<1010:13:49, 16.24s/it]
  0%|          | 1119/225000 [5:46:05<912:42:35, 14.68s/it] 
  0%|          | 1120/225000 [5:46:26<1026:00:46, 16.50s/it]
  0%|          | 1121/225000 [5:46:39<957:49:14, 15.40s/it] 
  0%|          | 1122/225000 [5:46:59<1042:17:39, 16.76s/it]
  0%|          | 1123/225000 [5:47:10<939:30:11, 15.11s/it] 
  0%|          | 1124/225000 [5:47:21<871:21:32, 14.01s/it]
  0%|          | 1125/225000 [5:47:35<865:48:52, 13.92s/it]
  1%|          | 1126/225000 [5:47:48<849:54:07, 13.67s/it]
  1%|          | 1127/225000 [5:48:09<991:37:41, 15.95s/it]
  1%|          | 1128/225000 [5:48:24<960:07:31, 15.44s/it]
  1%|          | 1129/225000 [5:48:50<1161:23:52, 18.68s/it]
  1%|          | 1130/225000 [5:48:56<924:40:36, 14.87s/it] 
  1%|          | 1131/225000 [5:49:13<973:24:45, 15.65s/it]
  1%|          | 1132/225000 [5:49:25<903:16:29, 14.53s/it]
  1%|          | 1133/225000 [5:49:49<1085:24:37, 17.45s/it]
  1%|          | 1134/225000 [5:50:10<1143:59:15, 18.40s/it]
  1%|          | 1135/225000 [5:50:21<1012:44:40, 16.29s/it]
  1%|          | 1136/225000 [5:50:31<880:22:16, 14.16s/it] 
  1%|          | 1137/225000 [5:50:50<981:33:33, 15.78s/it]
  1%|          | 1138/225000 [5:51:04<938:47:42, 15.10s/it]
  1%|          | 1139/225000 [5:51:24<1035:39:40, 16.65s/it]
  1%|          | 1140/225000 [5:51:43<1077:14:53, 17.32s/it]
  1%|          | 1141/225000 [5:51:58<1028:27:26, 16.54s/it]
  1%|          | 1142/225000 [5:52:12<982:01:05, 15.79s/it] 
  1%|          | 1143/225000 [5:52:32<1069:23:08, 17.20s/it]
  1%|          | 1144/225000 [5:52:51<1091:39:16, 17.56s/it]
  1%|          | 1145/225000 [5:53:10<1127:11:38, 18.13s/it]
  1%|          | 1146/225000 [5:53:33<1226:17:08, 19.72s/it]
  1%|          | 1147/225000 [5:53:49<1152:34:39, 18.54s/it]
  1%|          | 1148/225000 [5:54:02<1040:18:20, 16.73s/it]
  1%|          | 1149/225000 [5:54:18<1025:03:13, 16.49s/it]
  1%|          | 1150/225000 [5:54:32<990:31:56, 15.93s/it] 
                                                           
{'loss': '1.151', 'grad_norm': '1.543', 'learning_rate': '5.73e-06', 'epoch': '0.005111', 'num_input_tokens_seen': 11137664, 'train_runtime': '2.127e+04', 'train_tokens_per_second': '523.6'}

  1%|          | 1150/225000 [5:54:32<990:31:56, 15.93s/it]
  1%|          | 1151/225000 [5:54:50<1031:52:29, 16.59s/it]
  1%|          | 1152/225000 [5:55:07<1039:10:48, 16.71s/it]
  1%|          | 1153/225000 [5:55:27<1092:16:09, 17.57s/it]
  1%|          | 1154/225000 [5:55:49<1176:13:22, 18.92s/it]
  1%|          | 1155/225000 [5:56:03<1086:22:30, 17.47s/it]
  1%|          | 1156/225000 [5:56:17<1020:57:24, 16.42s/it]
  1%|          | 1157/225000 [5:56:29<938:29:54, 15.09s/it] 
  1%|          | 1158/225000 [5:56:48<1016:03:23, 16.34s/it]
  1%|          | 1159/225000 [5:56:59<908:39:01, 14.61s/it] 
  1%|          | 1160/225000 [5:57:20<1031:06:42, 16.58s/it]
  1%|          | 1161/225000 [5:57:46<1196:31:36, 19.24s/it]
  1%|          | 1162/225000 [5:57:56<1025:22:00, 16.49s/it]
  1%|          | 1163/225000 [5:58:13<1044:28:48, 16.80s/it]
  1%|          | 1164/225000 [5:58:31<1059:26:25, 17.04s/it]
  1%|          | 1165/225000 [5:58:49<1086:11:30, 17.47s/it]
  1%|          | 1166/225000 [5:59:07<1099:44:23, 17.69s/it]
  1%|          | 1167/225000 [5:59:24<1088:35:26, 17.51s/it]
  1%|          | 1168/225000 [5:59:48<1195:47:11, 19.23s/it]
  1%|          | 1169/225000 [6:00:08<1208:28:51, 19.44s/it]
  1%|          | 1170/225000 [6:00:20<1077:06:31, 17.32s/it]
  1%|          | 1171/225000 [6:00:32<983:31:27, 15.82s/it] 
  1%|          | 1172/225000 [6:00:51<1036:45:48, 16.68s/it]
  1%|          | 1173/225000 [6:01:03<942:31:18, 15.16s/it] 
  1%|          | 1174/225000 [6:01:25<1082:03:43, 17.40s/it]
  1%|          | 1175/225000 [6:01:38<995:12:36, 16.01s/it] 
  1%|          | 1176/225000 [6:01:47<871:12:06, 14.01s/it]
  1%|          | 1177/225000 [6:02:05<929:27:28, 14.95s/it]
  1%|          | 1178/225000 [6:02:24<1009:50:01, 16.24s/it]
  1%|          | 1179/225000 [6:02:49<1181:32:59, 19.00s/it]
  1%|          | 1180/225000 [6:03:01<1037:48:12, 16.69s/it]
  1%|          | 1181/225000 [6:03:26<1204:07:47, 19.37s/it]
  1%|          | 1182/225000 [6:03:38<1069:41:20, 17.21s/it]
  1%|          | 1183/225000 [6:03:54<1047:07:37, 16.84s/it]
  1%|          | 1184/225000 [6:04:15<1113:12:06, 17.91s/it]
  1%|          | 1185/225000 [6:04:32<1104:42:41, 17.77s/it]
  1%|          | 1186/225000 [6:04:50<1111:02:49, 17.87s/it]
  1%|          | 1187/225000 [6:05:16<1258:12:39, 20.24s/it]
  1%|          | 1188/225000 [6:05:40<1337:08:53, 21.51s/it]
  1%|          | 1189/225000 [6:05:58<1263:30:02, 20.32s/it]
  1%|          | 1190/225000 [6:06:13<1167:06:19, 18.77s/it]
  1%|          | 1191/225000 [6:06:29<1121:18:27, 18.04s/it]
  1%|          | 1192/225000 [6:06:43<1045:02:08, 16.81s/it]
  1%|          | 1193/225000 [6:06:56<956:49:06, 15.39s/it] 
  1%|          | 1194/225000 [6:07:15<1039:11:21, 16.72s/it]
  1%|          | 1195/225000 [6:07:37<1129:51:37, 18.17s/it]
  1%|          | 1196/225000 [6:07:53<1087:05:41, 17.49s/it]
  1%|          | 1197/225000 [6:08:11<1098:00:26, 17.66s/it]
  1%|          | 1198/225000 [6:08:27<1074:00:11, 17.28s/it]
  1%|          | 1199/225000 [6:08:38<958:29:05, 15.42s/it] 
  1%|          | 1200/225000 [6:08:57<1023:59:40, 16.47s/it]
                                                            
{'loss': '1.132', 'grad_norm': '1.853', 'learning_rate': '5.98e-06', 'epoch': '0.005333', 'num_input_tokens_seen': 11580080, 'train_runtime': '2.214e+04', 'train_tokens_per_second': '523.1'}

  1%|          | 1200/225000 [6:08:57<1023:59:40, 16.47s/it]
  1%|          | 1201/225000 [6:09:16<1067:22:19, 17.17s/it]
  1%|          | 1202/225000 [6:09:31<1019:47:47, 16.40s/it]
  1%|          | 1203/225000 [6:09:50<1070:30:39, 17.22s/it]
  1%|          | 1204/225000 [6:10:04<1019:22:11, 16.40s/it]
  1%|          | 1205/225000 [6:10:18<963:47:47, 15.50s/it] 
  1%|          | 1206/225000 [6:10:35<988:50:18, 15.91s/it]
  1%|          | 1207/225000 [6:10:53<1029:58:15, 16.57s/it]
  1%|          | 1208/225000 [6:11:07<990:47:51, 15.94s/it] 
  1%|          | 1209/225000 [6:11:24<1000:11:01, 16.09s/it]
  1%|          | 1210/225000 [6:11:40<1010:07:07, 16.25s/it]
  1%|          | 1211/225000 [6:11:58<1048:17:55, 16.86s/it]
  1%|          | 1212/225000 [6:12:08<917:11:26, 14.75s/it] 
  1%|          | 1213/225000 [6:12:22<895:39:33, 14.41s/it]
  1%|          | 1214/225000 [6:12:45<1063:22:20, 17.11s/it]
  1%|          | 1215/225000 [6:13:01<1030:32:36, 16.58s/it]
  1%|          | 1216/225000 [6:13:14<979:14:13, 15.75s/it] 
  1%|          | 1217/225000 [6:13:22<820:35:32, 13.20s/it]
  1%|          | 1218/225000 [6:13:34<802:04:20, 12.90s/it]
  1%|          | 1219/225000 [6:13:50<861:18:06, 13.86s/it]
  1%|          | 1220/225000 [6:14:09<950:06:50, 15.28s/it]
  1%|          | 1221/225000 [6:14:16<804:39:38, 12.94s/it]
  1%|          | 1222/225000 [6:14:38<977:02:21, 15.72s/it]
  1%|          | 1223/225000 [6:14:53<950:08:03, 15.29s/it]
  1%|          | 1224/225000 [6:15:06<917:22:09, 14.76s/it]
  1%|          | 1225/225000 [6:15:27<1034:18:30, 16.64s/it]
  1%|          | 1226/225000 [6:15:38<926:32:22, 14.91s/it] 
  1%|          | 1227/225000 [6:15:56<975:46:22, 15.70s/it]
  1%|          | 1228/225000 [6:16:14<1032:47:45, 16.62s/it]
  1%|          | 1229/225000 [6:16:33<1069:31:17, 17.21s/it]
  1%|          | 1230/225000 [6:16:47<1008:22:48, 16.22s/it]
  1%|          | 1231/225000 [6:17:09<1121:49:47, 18.05s/it]
  1%|          | 1232/225000 [6:17:28<1135:41:16, 18.27s/it]
  1%|          | 1233/225000 [6:17:48<1168:22:17, 18.80s/it]
  1%|          | 1234/225000 [6:18:07<1180:58:56, 19.00s/it]
  1%|          | 1235/225000 [6:18:15<963:07:14, 15.49s/it] 
  1%|          | 1236/225000 [6:18:35<1053:07:25, 16.94s/it]
  1%|          | 1237/225000 [6:18:50<1016:00:08, 16.35s/it]
  1%|          | 1238/225000 [6:19:08<1049:13:30, 16.88s/it]
  1%|          | 1239/225000 [6:19:18<915:16:10, 14.73s/it] 
  1%|          | 1240/225000 [6:19:35<962:20:33, 15.48s/it]
  1%|          | 1241/225000 [6:19:47<897:10:20, 14.43s/it]
  1%|          | 1242/225000 [6:20:06<986:21:01, 15.87s/it]
  1%|          | 1243/225000 [6:20:22<980:30:49, 15.78s/it]
  1%|          | 1244/225000 [6:20:31<859:32:38, 13.83s/it]
  1%|          | 1245/225000 [6:20:47<903:11:09, 14.53s/it]
  1%|          | 1246/225000 [6:21:02<910:34:58, 14.65s/it]
  1%|          | 1247/225000 [6:21:15<874:26:13, 14.07s/it]
  1%|          | 1248/225000 [6:21:34<972:15:00, 15.64s/it]
  1%|          | 1249/225000 [6:21:45<881:44:00, 14.19s/it]
  1%|          | 1250/225000 [6:22:05<987:02:46, 15.88s/it]
                                                           
{'loss': '1.14', 'grad_norm': '1.123', 'learning_rate': '6.23e-06', 'epoch': '0.005556', 'num_input_tokens_seen': 11983744, 'train_runtime': '2.293e+04', 'train_tokens_per_second': '522.7'}

  1%|          | 1250/225000 [6:22:05<987:02:46, 15.88s/it]
  1%|          | 1251/225000 [6:22:25<1064:44:23, 17.13s/it]
  1%|          | 1252/225000 [6:22:40<1026:32:35, 16.52s/it]
  1%|          | 1253/225000 [6:22:56<1024:53:03, 16.49s/it]
  1%|          | 1254/225000 [6:23:04<865:21:54, 13.92s/it] 
  1%|          | 1255/225000 [6:23:16<824:00:38, 13.26s/it]
  1%|          | 1256/225000 [6:23:35<935:42:36, 15.06s/it]
  1%|          | 1257/225000 [6:24:02<1144:21:45, 18.41s/it]
  1%|          | 1258/225000 [6:24:14<1040:18:29, 16.74s/it]
  1%|          | 1259/225000 [6:24:31<1038:29:04, 16.71s/it]
  1%|          | 1260/225000 [6:24:55<1166:44:04, 18.77s/it]
  1%|          | 1261/225000 [6:25:05<1005:11:45, 16.17s/it]
  1%|          | 1262/225000 [6:25:27<1110:06:36, 17.86s/it]
  1%|          | 1263/225000 [6:25:44<1107:05:13, 17.81s/it]
  1%|          | 1264/225000 [6:25:55<980:54:21, 15.78s/it] 
  1%|          | 1265/225000 [6:26:15<1054:38:01, 16.97s/it]
  1%|          | 1266/225000 [6:26:27<954:32:02, 15.36s/it] 
  1%|          | 1267/225000 [6:26:36<850:37:55, 13.69s/it]
  1%|          | 1268/225000 [6:26:56<952:11:23, 15.32s/it]
  1%|          | 1269/225000 [6:27:05<847:31:09, 13.64s/it]
  1%|          | 1270/225000 [6:27:30<1061:50:06, 17.09s/it]
  1%|          | 1271/225000 [6:27:44<990:29:24, 15.94s/it] 
  1%|          | 1272/225000 [6:28:13<1245:46:25, 20.05s/it]
  1%|          | 1273/225000 [6:28:29<1165:54:32, 18.76s/it]
  1%|          | 1274/225000 [6:28:40<1021:06:09, 16.43s/it]
  1%|          | 1275/225000 [6:29:01<1108:14:50, 17.83s/it]
  1%|          | 1276/225000 [6:29:21<1142:01:54, 18.38s/it]
  1%|          | 1277/225000 [6:29:45<1258:38:17, 20.25s/it]
  1%|          | 1278/225000 [6:30:10<1330:55:34, 21.42s/it]
  1%|          | 1279/225000 [6:30:25<1219:56:32, 19.63s/it]
  1%|          | 1280/225000 [6:30:41<1158:11:29, 18.64s/it]
  1%|          | 1281/225000 [6:30:58<1129:59:10, 18.18s/it]
  1%|          | 1282/225000 [6:31:12<1038:46:06, 16.72s/it]
  1%|          | 1283/225000 [6:31:22<917:29:21, 14.76s/it] 
  1%|          | 1284/225000 [6:31:38<944:41:33, 15.20s/it]
  1%|          | 1285/225000 [6:31:53<945:16:26, 15.21s/it]
  1%|          | 1286/225000 [6:32:09<958:07:13, 15.42s/it]
  1%|          | 1287/225000 [6:32:27<999:15:42, 16.08s/it]
  1%|          | 1288/225000 [6:32:49<1119:19:42, 18.01s/it]
  1%|          | 1289/225000 [6:33:08<1134:14:44, 18.25s/it]
  1%|          | 1290/225000 [6:33:23<1067:50:31, 17.18s/it]
  1%|          | 1291/225000 [6:33:38<1027:00:24, 16.53s/it]
  1%|          | 1292/225000 [6:33:54<1010:46:54, 16.27s/it]
  1%|          | 1293/225000 [6:34:08<980:14:03, 15.77s/it] 
  1%|          | 1294/225000 [6:34:31<1106:54:41, 17.81s/it]
  1%|          | 1295/225000 [6:34:49<1109:41:23, 17.86s/it]
  1%|          | 1296/225000 [6:35:14<1251:19:03, 20.14s/it]
  1%|          | 1297/225000 [6:35:26<1100:10:12, 17.70s/it]
  1%|          | 1298/225000 [6:35:44<1109:15:55, 17.85s/it]
  1%|          | 1299/225000 [6:36:00<1057:54:21, 17.02s/it]
  1%|          | 1300/225000 [6:36:12<972:00:33, 15.64s/it] 
                                                           
{'loss': '1.119', 'grad_norm': '1.397', 'learning_rate': '6.48e-06', 'epoch': '0.005778', 'num_input_tokens_seen': 12412920, 'train_runtime': '2.377e+04', 'train_tokens_per_second': '522.2'}

  1%|          | 1300/225000 [6:36:12<972:00:33, 15.64s/it]
  1%|          | 1301/225000 [6:36:32<1055:52:51, 16.99s/it]
  1%|          | 1302/225000 [6:36:41<906:10:10, 14.58s/it] 
  1%|          | 1303/225000 [6:37:01<1013:53:39, 16.32s/it]
  1%|          | 1304/225000 [6:37:19<1028:25:15, 16.55s/it]
  1%|          | 1305/225000 [6:37:34<1016:56:59, 16.37s/it]
  1%|          | 1306/225000 [6:37:48<958:50:45, 15.43s/it] 
  1%|          | 1307/225000 [6:38:09<1064:21:50, 17.13s/it]
  1%|          | 1308/225000 [6:38:22<984:36:48, 15.85s/it] 
  1%|          | 1309/225000 [6:38:42<1059:23:06, 17.05s/it]
  1%|          | 1310/225000 [6:38:54<977:25:07, 15.73s/it] 
  1%|          | 1311/225000 [6:39:10<978:51:30, 15.75s/it]
  1%|          | 1312/225000 [6:39:23<926:41:35, 14.91s/it]
  1%|          | 1313/225000 [6:39:44<1037:36:25, 16.70s/it]
  1%|          | 1314/225000 [6:40:03<1077:06:57, 17.34s/it]
  1%|          | 1315/225000 [6:40:16<1002:45:14, 16.14s/it]
  1%|          | 1316/225000 [6:40:28<922:15:56, 14.84s/it] 
  1%|          | 1317/225000 [6:40:53<1120:08:06, 18.03s/it]
  1%|          | 1318/225000 [6:41:01<931:54:09, 15.00s/it] 
  1%|          | 1319/225000 [6:41:11<840:17:59, 13.52s/it]
  1%|          | 1320/225000 [6:41:26<860:40:46, 13.85s/it]
  1%|          | 1321/225000 [6:41:43<930:23:10, 14.97s/it]
  1%|          | 1322/225000 [6:42:04<1027:00:53, 16.53s/it]
  1%|          | 1323/225000 [6:42:23<1074:20:43, 17.29s/it]
  1%|          | 1324/225000 [6:42:42<1104:30:11, 17.78s/it]
  1%|          | 1325/225000 [6:42:53<992:38:39, 15.98s/it] 
  1%|          | 1326/225000 [6:43:09<982:57:28, 15.82s/it]
  1%|          | 1327/225000 [6:43:29<1060:52:59, 17.07s/it]
  1%|          | 1328/225000 [6:43:48<1102:35:22, 17.75s/it]
  1%|          | 1329/225000 [6:44:02<1028:38:41, 16.56s/it]
  1%|          | 1330/225000 [6:44:29<1215:56:36, 19.57s/it]
  1%|          | 1331/225000 [6:44:43<1113:41:15, 17.93s/it]
  1%|          | 1332/225000 [6:45:03<1154:46:52, 18.59s/it]
  1%|          | 1333/225000 [6:45:17<1069:26:19, 17.21s/it]
  1%|          | 1334/225000 [6:45:35<1083:38:41, 17.44s/it]
  1%|          | 1335/225000 [6:45:44<927:58:47, 14.94s/it] 
  1%|          | 1336/225000 [6:45:57<902:14:57, 14.52s/it]
  1%|          | 1337/225000 [6:46:16<977:51:18, 15.74s/it]
  1%|          | 1338/225000 [6:46:31<968:18:51, 15.59s/it]
  1%|          | 1339/225000 [6:46:49<1005:12:48, 16.18s/it]
  1%|          | 1340/225000 [6:47:00<908:49:57, 14.63s/it] 
  1%|          | 1341/225000 [6:47:11<851:21:13, 13.70s/it]
  1%|          | 1342/225000 [6:47:29<926:48:23, 14.92s/it]
  1%|          | 1343/225000 [6:47:42<881:47:32, 14.19s/it]
  1%|          | 1344/225000 [6:48:02<990:20:25, 15.94s/it]
  1%|          | 1345/225000 [6:48:23<1095:12:04, 17.63s/it]
  1%|          | 1346/225000 [6:48:43<1137:33:45, 18.31s/it]
  1%|          | 1347/225000 [6:48:50<924:47:59, 14.89s/it] 
  1%|          | 1348/225000 [6:49:12<1054:42:40, 16.98s/it]
  1%|          | 1349/225000 [6:49:25<984:20:41, 15.84s/it] 
  1%|          | 1350/225000 [6:49:41<984:02:06, 15.84s/it]
                                                           
{'loss': '1.128', 'grad_norm': '1.266', 'learning_rate': '6.73e-06', 'epoch': '0.006', 'num_input_tokens_seen': 12824000, 'train_runtime': '2.458e+04', 'train_tokens_per_second': '521.7'}

  1%|          | 1350/225000 [6:49:41<984:02:06, 15.84s/it]
  1%|          | 1351/225000 [6:49:58<1016:53:38, 16.37s/it]
  1%|          | 1352/225000 [6:50:13<978:25:52, 15.75s/it] 
  1%|          | 1353/225000 [6:50:30<1004:22:56, 16.17s/it]
  1%|          | 1354/225000 [6:50:45<988:32:11, 15.91s/it] 
  1%|          | 1355/225000 [6:50:56<887:39:05, 14.29s/it]
  1%|          | 1356/225000 [6:51:08<858:16:05, 13.82s/it]
  1%|          | 1357/225000 [6:51:23<867:51:39, 13.97s/it]
  1%|          | 1358/225000 [6:51:44<1009:08:29, 16.24s/it]
  1%|          | 1359/225000 [6:52:00<1004:17:32, 16.17s/it]
  1%|          | 1360/225000 [6:52:11<902:22:53, 14.53s/it] 
  1%|          | 1361/225000 [6:52:26<902:37:23, 14.53s/it]
  1%|          | 1362/225000 [6:52:46<1014:42:41, 16.33s/it]
  1%|          | 1363/225000 [6:52:57<905:21:55, 14.57s/it] 
  1%|          | 1364/225000 [6:53:21<1098:52:01, 17.69s/it]
  1%|          | 1365/225000 [6:53:40<1114:48:35, 17.95s/it]
  1%|          | 1366/225000 [6:53:56<1073:05:14, 17.27s/it]
  1%|          | 1367/225000 [6:54:07<969:47:22, 15.61s/it] 
  1%|          | 1368/225000 [6:54:18<877:41:08, 14.13s/it]
  1%|          | 1369/225000 [6:54:39<1011:41:52, 16.29s/it]
  1%|          | 1370/225000 [6:55:00<1098:26:10, 17.68s/it]
  1%|          | 1371/225000 [6:55:15<1038:39:44, 16.72s/it]
  1%|          | 1372/225000 [6:55:30<1012:22:10, 16.30s/it]
  1%|          | 1373/225000 [6:55:50<1086:57:32, 17.50s/it]
  1%|          | 1374/225000 [6:56:15<1226:30:53, 19.74s/it]
  1%|          | 1375/225000 [6:56:38<1270:22:10, 20.45s/it]
  1%|          | 1376/225000 [6:56:50<1130:01:14, 18.19s/it]
  1%|          | 1377/225000 [6:57:02<1001:40:31, 16.13s/it]
  1%|          | 1378/225000 [6:57:17<979:19:51, 15.77s/it] 
  1%|          | 1379/225000 [6:57:26<857:53:02, 13.81s/it]
  1%|          | 1380/225000 [6:57:46<976:11:01, 15.72s/it]
  1%|          | 1381/225000 [6:57:56<871:27:17, 14.03s/it]
  1%|          | 1382/225000 [6:58:18<1011:50:26, 16.29s/it]
  1%|          | 1383/225000 [6:58:40<1122:14:50, 18.07s/it]
  1%|          | 1384/225000 [6:58:57<1097:04:10, 17.66s/it]
  1%|          | 1385/225000 [6:59:16<1125:17:00, 18.12s/it]
  1%|          | 1386/225000 [6:59:21<881:13:35, 14.19s/it] 
  1%|          | 1387/225000 [6:59:41<982:39:49, 15.82s/it]
  1%|          | 1388/225000 [6:59:51<880:54:11, 14.18s/it]
  1%|          | 1389/225000 [7:00:10<978:33:23, 15.75s/it]
  1%|          | 1390/225000 [7:00:18<820:45:30, 13.21s/it]
  1%|          | 1391/225000 [7:00:33<869:37:46, 14.00s/it]
  1%|          | 1392/225000 [7:00:48<883:09:51, 14.22s/it]
  1%|          | 1393/225000 [7:01:09<1011:49:25, 16.29s/it]
  1%|          | 1394/225000 [7:01:27<1041:53:31, 16.77s/it]
  1%|          | 1395/225000 [7:01:43<1016:51:32, 16.37s/it]
  1%|          | 1396/225000 [7:02:02<1067:36:51, 17.19s/it]
  1%|          | 1397/225000 [7:02:17<1040:55:47, 16.76s/it]
  1%|          | 1398/225000 [7:02:34<1027:09:11, 16.54s/it]
  1%|          | 1399/225000 [7:02:53<1073:35:42, 17.28s/it]
  1%|          | 1400/225000 [7:03:01<909:42:34, 14.65s/it] 
                                                           
{'loss': '1.115', 'grad_norm': '1.091', 'learning_rate': '6.98e-06', 'epoch': '0.006222', 'num_input_tokens_seen': 13234736, 'train_runtime': '2.538e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1400/225000 [7:03:01<909:42:34, 14.65s/it]
  1%|          | 1401/225000 [7:03:20<998:48:51, 16.08s/it]
  1%|          | 1402/225000 [7:03:34<954:00:26, 15.36s/it]
  1%|          | 1403/225000 [7:03:49<937:16:59, 15.09s/it]
  1%|          | 1404/225000 [7:04:01<888:32:14, 14.31s/it]
  1%|          | 1405/225000 [7:04:22<1016:28:18, 16.37s/it]
  1%|          | 1406/225000 [7:04:28<812:46:15, 13.09s/it] 
  1%|          | 1407/225000 [7:04:45<892:09:26, 14.36s/it]
  1%|          | 1408/225000 [7:04:55<816:07:40, 13.14s/it]
  1%|          | 1409/225000 [7:05:14<916:34:13, 14.76s/it]
  1%|          | 1410/225000 [7:05:32<975:25:42, 15.71s/it]
  1%|          | 1411/225000 [7:05:51<1041:05:03, 16.76s/it]
  1%|          | 1412/225000 [7:06:05<996:43:43, 16.05s/it] 
  1%|          | 1413/225000 [7:06:21<981:15:16, 15.80s/it]
  1%|          | 1414/225000 [7:06:37<993:08:17, 15.99s/it]
  1%|          | 1415/225000 [7:06:50<932:05:56, 15.01s/it]
  1%|          | 1416/225000 [7:07:02<884:19:20, 14.24s/it]
  1%|          | 1417/225000 [7:07:21<969:05:59, 15.60s/it]
  1%|          | 1418/225000 [7:07:32<887:02:33, 14.28s/it]
  1%|          | 1419/225000 [7:07:47<899:05:50, 14.48s/it]
  1%|          | 1420/225000 [7:07:59<854:14:48, 13.75s/it]
  1%|          | 1421/225000 [7:08:10<808:00:22, 13.01s/it]
  1%|          | 1422/225000 [7:08:22<785:54:34, 12.65s/it]
  1%|          | 1423/225000 [7:08:35<787:20:04, 12.68s/it]
  1%|          | 1424/225000 [7:08:49<805:54:34, 12.98s/it]
  1%|          | 1425/225000 [7:08:58<733:40:23, 11.81s/it]
  1%|          | 1426/225000 [7:09:09<716:11:31, 11.53s/it]
  1%|          | 1427/225000 [7:09:32<931:01:00, 14.99s/it]
  1%|          | 1428/225000 [7:09:44<884:49:49, 14.25s/it]
  1%|          | 1429/225000 [7:10:09<1080:16:47, 17.39s/it]
  1%|          | 1430/225000 [7:10:25<1047:02:38, 16.86s/it]
  1%|          | 1431/225000 [7:10:37<972:29:58, 15.66s/it] 
  1%|          | 1432/225000 [7:10:58<1064:30:36, 17.14s/it]
  1%|          | 1433/225000 [7:11:11<996:12:43, 16.04s/it] 
  1%|          | 1434/225000 [7:11:30<1032:46:40, 16.63s/it]
  1%|          | 1435/225000 [7:11:47<1051:34:28, 16.93s/it]
  1%|          | 1436/225000 [7:12:05<1073:19:19, 17.28s/it]
  1%|          | 1437/225000 [7:12:15<940:04:52, 15.14s/it] 
  1%|          | 1438/225000 [7:12:28<900:49:03, 14.51s/it]
  1%|          | 1439/225000 [7:12:43<907:21:32, 14.61s/it]
  1%|          | 1440/225000 [7:13:00<943:52:26, 15.20s/it]
  1%|          | 1441/225000 [7:13:14<929:53:33, 14.97s/it]
  1%|          | 1442/225000 [7:13:27<885:02:41, 14.25s/it]
  1%|          | 1443/225000 [7:13:41<878:21:11, 14.14s/it]
  1%|          | 1444/225000 [7:14:03<1022:11:40, 16.46s/it]
  1%|          | 1445/225000 [7:14:18<1007:22:38, 16.22s/it]
  1%|          | 1446/225000 [7:14:34<1006:52:03, 16.21s/it]
  1%|          | 1447/225000 [7:14:48<947:43:41, 15.26s/it] 
  1%|          | 1448/225000 [7:14:55<805:09:59, 12.97s/it]
  1%|          | 1449/225000 [7:15:10<834:11:30, 13.43s/it]
  1%|          | 1450/225000 [7:15:27<908:44:21, 14.63s/it]
                                                           
{'loss': '1.096', 'grad_norm': '1.25', 'learning_rate': '7.23e-06', 'epoch': '0.006444', 'num_input_tokens_seen': 13622600, 'train_runtime': '2.613e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1450/225000 [7:15:27<908:44:21, 14.63s/it]
  1%|          | 1451/225000 [7:15:34<769:26:36, 12.39s/it]
  1%|          | 1452/225000 [7:15:50<830:17:34, 13.37s/it]
  1%|          | 1453/225000 [7:16:06<886:12:29, 14.27s/it]
  1%|          | 1454/225000 [7:16:20<870:36:06, 14.02s/it]
  1%|          | 1455/225000 [7:16:43<1046:15:01, 16.85s/it]
  1%|          | 1456/225000 [7:16:59<1032:32:59, 16.63s/it]
  1%|          | 1457/225000 [7:17:14<993:49:15, 16.00s/it] 
  1%|          | 1458/225000 [7:17:28<968:45:31, 15.60s/it]
  1%|          | 1459/225000 [7:17:43<941:32:30, 15.16s/it]
  1%|          | 1460/225000 [7:17:56<901:01:48, 14.51s/it]
  1%|          | 1461/225000 [7:18:13<963:45:23, 15.52s/it]
  1%|          | 1462/225000 [7:18:23<843:36:34, 13.59s/it]
  1%|          | 1463/225000 [7:18:40<914:50:55, 14.73s/it]
  1%|          | 1464/225000 [7:18:59<986:34:55, 15.89s/it]
  1%|          | 1465/225000 [7:19:12<945:07:28, 15.22s/it]
  1%|          | 1466/225000 [7:19:29<982:10:20, 15.82s/it]
  1%|          | 1467/225000 [7:19:45<983:38:59, 15.84s/it]
  1%|          | 1468/225000 [7:20:00<969:34:58, 15.62s/it]
  1%|          | 1469/225000 [7:20:21<1056:53:01, 17.02s/it]
  1%|          | 1470/225000 [7:20:35<1009:22:37, 16.26s/it]
  1%|          | 1471/225000 [7:20:53<1031:50:31, 16.62s/it]
  1%|          | 1472/225000 [7:21:09<1027:14:09, 16.54s/it]
  1%|          | 1473/225000 [7:21:25<1008:24:55, 16.24s/it]
  1%|          | 1474/225000 [7:21:41<1018:51:58, 16.41s/it]
  1%|          | 1475/225000 [7:21:55<973:43:03, 15.68s/it] 
  1%|          | 1476/225000 [7:22:16<1066:38:27, 17.18s/it]
  1%|          | 1477/225000 [7:22:34<1088:08:09, 17.53s/it]
  1%|          | 1478/225000 [7:22:55<1139:29:13, 18.35s/it]
  1%|          | 1479/225000 [7:23:09<1063:41:51, 17.13s/it]
  1%|          | 1480/225000 [7:23:20<958:27:56, 15.44s/it] 
  1%|          | 1481/225000 [7:23:33<914:49:20, 14.73s/it]
  1%|          | 1482/225000 [7:23:47<896:59:04, 14.45s/it]
  1%|          | 1483/225000 [7:24:02<908:07:47, 14.63s/it]
  1%|          | 1484/225000 [7:24:14<856:45:16, 13.80s/it]
  1%|          | 1485/225000 [7:24:25<808:44:12, 13.03s/it]
  1%|          | 1486/225000 [7:24:42<881:09:08, 14.19s/it]
  1%|          | 1487/225000 [7:25:02<991:43:57, 15.97s/it]
  1%|          | 1488/225000 [7:25:16<940:39:43, 15.15s/it]
  1%|          | 1489/225000 [7:25:32<965:09:59, 15.55s/it]
  1%|          | 1490/225000 [7:25:40<826:30:48, 13.31s/it]
  1%|          | 1491/225000 [7:25:51<778:55:02, 12.55s/it]
  1%|          | 1492/225000 [7:26:11<917:36:19, 14.78s/it]
  1%|          | 1493/225000 [7:26:29<973:56:16, 15.69s/it]
  1%|          | 1494/225000 [7:26:41<918:25:34, 14.79s/it]
  1%|          | 1495/225000 [7:26:59<972:47:59, 15.67s/it]
  1%|          | 1496/225000 [7:27:16<989:35:09, 15.94s/it]
  1%|          | 1497/225000 [7:27:33<1005:35:38, 16.20s/it]
  1%|          | 1498/225000 [7:27:48<999:01:51, 16.09s/it] 
  1%|          | 1499/225000 [7:28:13<1163:47:31, 18.75s/it]
  1%|          | 1500/225000 [7:28:26<1044:42:35, 16.83s/it]
                                                            
{'loss': '1.074', 'grad_norm': '1.407', 'learning_rate': '7.48e-06', 'epoch': '0.006667', 'num_input_tokens_seen': 14034264, 'train_runtime': '2.691e+04', 'train_tokens_per_second': '521.6'}

  1%|          | 1500/225000 [7:28:26<1044:42:35, 16.83s/it]
  1%|          | 1501/225000 [7:28:42<1025:28:08, 16.52s/it]
  1%|          | 1502/225000 [7:28:53<933:27:09, 15.04s/it] 
  1%|          | 1503/225000 [7:29:03<837:24:10, 13.49s/it]
  1%|          | 1504/225000 [7:29:22<934:41:07, 15.06s/it]
  1%|          | 1505/225000 [7:29:41<1023:02:40, 16.48s/it]
  1%|          | 1506/225000 [7:29:57<1007:17:24, 16.23s/it]
  1%|          | 1507/225000 [7:30:06<865:33:00, 13.94s/it] 
  1%|          | 1508/225000 [7:30:22<906:57:58, 14.61s/it]
  1%|          | 1509/225000 [7:30:33<848:16:13, 13.66s/it]
  1%|          | 1510/225000 [7:30:45<816:25:03, 13.15s/it]
  1%|          | 1511/225000 [7:31:04<919:44:58, 14.82s/it]
  1%|          | 1512/225000 [7:31:17<878:20:08, 14.15s/it]
  1%|          | 1513/225000 [7:31:35<949:11:45, 15.29s/it]
  1%|          | 1514/225000 [7:31:40<768:58:29, 12.39s/it]
  1%|          | 1515/225000 [7:31:52<749:37:22, 12.08s/it]
  1%|          | 1516/225000 [7:32:09<847:20:30, 13.65s/it]
  1%|          | 1517/225000 [7:32:19<780:32:24, 12.57s/it]
  1%|          | 1518/225000 [7:32:37<882:19:53, 14.21s/it]
  1%|          | 1519/225000 [7:32:55<955:07:49, 15.39s/it]
  1%|          | 1520/225000 [7:33:14<1023:23:32, 16.49s/it]
  1%|          | 1521/225000 [7:33:32<1042:40:15, 16.80s/it]
  1%|          | 1522/225000 [7:33:41<904:32:03, 14.57s/it] 
  1%|          | 1523/225000 [7:34:01<998:52:45, 16.09s/it]
  1%|          | 1524/225000 [7:34:11<899:57:15, 14.50s/it]
  1%|          | 1525/225000 [7:34:22<835:30:16, 13.46s/it]
  1%|          | 1526/225000 [7:34:37<848:06:35, 13.66s/it]
  1%|          | 1527/225000 [7:34:58<989:36:27, 15.94s/it]
  1%|          | 1528/225000 [7:35:16<1031:11:14, 16.61s/it]
  1%|          | 1529/225000 [7:35:33<1045:34:41, 16.84s/it]
  1%|          | 1530/225000 [7:35:49<1022:54:56, 16.48s/it]
  1%|          | 1531/225000 [7:36:02<949:17:58, 15.29s/it] 
  1%|          | 1532/225000 [7:36:16<939:47:39, 15.14s/it]
  1%|          | 1533/225000 [7:36:30<920:39:59, 14.83s/it]
  1%|          | 1534/225000 [7:36:48<967:07:17, 15.58s/it]
  1%|          | 1535/225000 [7:37:00<896:35:52, 14.44s/it]
  1%|          | 1536/225000 [7:37:20<1007:20:03, 16.23s/it]
  1%|          | 1537/225000 [7:37:27<831:02:07, 13.39s/it] 
  1%|          | 1538/225000 [7:37:40<822:47:30, 13.26s/it]
  1%|          | 1539/225000 [7:37:59<934:33:57, 15.06s/it]
  1%|          | 1540/225000 [7:38:11<874:48:48, 14.09s/it]
  1%|          | 1541/225000 [7:38:28<939:56:59, 15.14s/it]
  1%|          | 1542/225000 [7:38:44<944:22:34, 15.21s/it]
  1%|          | 1543/225000 [7:39:04<1040:59:55, 16.77s/it]
  1%|          | 1544/225000 [7:39:28<1181:05:25, 19.03s/it]
  1%|          | 1545/225000 [7:39:35<953:33:05, 15.36s/it] 
  1%|          | 1546/225000 [7:39:51<954:07:43, 15.37s/it]
  1%|          | 1547/225000 [7:40:04<921:29:22, 14.85s/it]
  1%|          | 1548/225000 [7:40:22<966:01:50, 15.56s/it]
  1%|          | 1549/225000 [7:40:43<1071:57:00, 17.27s/it]
  1%|          | 1550/225000 [7:40:55<978:03:27, 15.76s/it] 
                                                           
{'loss': '1.098', 'grad_norm': '1.195', 'learning_rate': '7.73e-06', 'epoch': '0.006889', 'num_input_tokens_seen': 14428944, 'train_runtime': '2.766e+04', 'train_tokens_per_second': '521.7'}

  1%|          | 1550/225000 [7:40:55<978:03:27, 15.76s/it]
  1%|          | 1551/225000 [7:41:08<934:14:26, 15.05s/it]
  1%|          | 1552/225000 [7:41:21<896:32:12, 14.44s/it]
  1%|          | 1553/225000 [7:41:40<970:53:47, 15.64s/it]
  1%|          | 1554/225000 [7:41:51<880:32:12, 14.19s/it]
  1%|          | 1555/225000 [7:42:01<811:00:40, 13.07s/it]
  1%|          | 1556/225000 [7:42:23<978:36:33, 15.77s/it]
  1%|          | 1557/225000 [7:42:38<959:15:49, 15.46s/it]
  1%|          | 1558/225000 [7:42:51<907:09:03, 14.62s/it]
  1%|          | 1559/225000 [7:43:09<986:51:35, 15.90s/it]
  1%|          | 1560/225000 [7:43:20<887:30:16, 14.30s/it]
  1%|          | 1561/225000 [7:43:31<831:22:37, 13.39s/it]
  1%|          | 1562/225000 [7:43:49<903:16:41, 14.55s/it]
  1%|          | 1563/225000 [7:44:05<946:24:03, 15.25s/it]
  1%|          | 1564/225000 [7:44:21<959:01:23, 15.45s/it]
  1%|          | 1565/225000 [7:44:29<811:14:00, 13.07s/it]
  1%|          | 1566/225000 [7:44:50<957:46:56, 15.43s/it]
  1%|          | 1567/225000 [7:45:01<871:28:56, 14.04s/it]
  1%|          | 1568/225000 [7:45:18<931:45:22, 15.01s/it]
  1%|          | 1569/225000 [7:45:28<844:59:28, 13.61s/it]
  1%|          | 1570/225000 [7:45:44<883:34:04, 14.24s/it]
  1%|          | 1571/225000 [7:45:59<902:16:41, 14.54s/it]
  1%|          | 1572/225000 [7:46:09<806:00:01, 12.99s/it]
  1%|          | 1573/225000 [7:46:23<834:59:57, 13.45s/it]
  1%|          | 1574/225000 [7:46:40<898:02:38, 14.47s/it]
  1%|          | 1575/225000 [7:46:57<951:33:58, 15.33s/it]
  1%|          | 1576/225000 [7:47:10<907:45:45, 14.63s/it]
  1%|          | 1577/225000 [7:47:25<900:53:36, 14.52s/it]
  1%|          | 1578/225000 [7:47:41<929:01:38, 14.97s/it]
  1%|          | 1579/225000 [7:48:06<1130:57:29, 18.22s/it]
  1%|          | 1580/225000 [7:48:16<962:47:41, 15.51s/it] 
  1%|          | 1581/225000 [7:48:33<997:34:52, 16.07s/it]
  1%|          | 1582/225000 [7:48:45<914:57:57, 14.74s/it]
  1%|          | 1583/225000 [7:49:07<1053:27:06, 16.97s/it]
  1%|          | 1584/225000 [7:49:18<944:56:27, 15.23s/it] 
  1%|          | 1585/225000 [7:49:32<926:02:41, 14.92s/it]
  1%|          | 1586/225000 [7:49:41<812:17:32, 13.09s/it]
  1%|          | 1587/225000 [7:49:53<792:57:08, 12.78s/it]
  1%|          | 1588/225000 [7:50:05<777:04:01, 12.52s/it]
  1%|          | 1589/225000 [7:50:26<942:47:48, 15.19s/it]
  1%|          | 1590/225000 [7:50:43<972:48:59, 15.68s/it]
  1%|          | 1591/225000 [7:51:07<1122:33:40, 18.09s/it]
  1%|          | 1592/225000 [7:51:18<984:52:19, 15.87s/it] 
  1%|          | 1593/225000 [7:51:36<1025:10:12, 16.52s/it]
  1%|          | 1594/225000 [7:51:59<1148:26:23, 18.51s/it]
  1%|          | 1595/225000 [7:52:18<1165:23:21, 18.78s/it]
  1%|          | 1596/225000 [7:52:36<1154:44:50, 18.61s/it]
  1%|          | 1597/225000 [7:52:55<1153:10:03, 18.58s/it]
  1%|          | 1598/225000 [7:53:08<1056:37:17, 17.03s/it]
  1%|          | 1599/225000 [7:53:24<1028:50:49, 16.58s/it]
  1%|          | 1600/225000 [7:53:37<959:23:43, 15.46s/it] 
                                                           
{'loss': '1.082', 'grad_norm': '0.9679', 'learning_rate': '7.98e-06', 'epoch': '0.007111', 'num_input_tokens_seen': 14820768, 'train_runtime': '2.842e+04', 'train_tokens_per_second': '521.5'}

  1%|          | 1600/225000 [7:53:37<959:23:43, 15.46s/it]
  1%|          | 1601/225000 [7:53:49<903:34:20, 14.56s/it]
  1%|          | 1602/225000 [7:54:08<984:08:36, 15.86s/it]
  1%|          | 1603/225000 [7:54:18<874:18:07, 14.09s/it]
  1%|          | 1604/225000 [7:54:32<879:38:00, 14.18s/it]
  1%|          | 1605/225000 [7:54:47<882:32:46, 14.22s/it]
  1%|          | 1606/225000 [7:54:57<815:22:55, 13.14s/it]
  1%|          | 1607/225000 [7:55:10<800:37:13, 12.90s/it]
  1%|          | 1608/225000 [7:55:25<846:52:27, 13.65s/it]
  1%|          | 1609/225000 [7:55:38<829:49:18, 13.37s/it]
  1%|          | 1610/225000 [7:56:02<1032:44:31, 16.64s/it]
  1%|          | 1611/225000 [7:56:17<1008:04:11, 16.25s/it]
  1%|          | 1612/225000 [7:56:31<965:36:21, 15.56s/it] 
  1%|          | 1613/225000 [7:56:47<966:19:42, 15.57s/it]
  1%|          | 1614/225000 [7:57:02<959:28:07, 15.46s/it]
  1%|          | 1615/225000 [7:57:15<916:48:23, 14.77s/it]
  1%|          | 1616/225000 [7:57:31<928:32:05, 14.96s/it]
  1%|          | 1617/225000 [7:57:47<948:22:03, 15.28s/it]
  1%|          | 1618/225000 [7:58:01<921:15:02, 14.85s/it]
  1%|          | 1619/225000 [7:58:12<863:37:01, 13.92s/it]
  1%|          | 1620/225000 [7:58:31<944:29:48, 15.22s/it]
  1%|          | 1621/225000 [7:58:51<1041:30:58, 16.79s/it]
  1%|          | 1622/225000 [7:59:09<1055:31:54, 17.01s/it]
  1%|          | 1623/225000 [7:59:25<1049:58:32, 16.92s/it]
  1%|          | 1624/225000 [7:59:43<1056:38:55, 17.03s/it]
  1%|          | 1625/225000 [7:59:53<926:34:28, 14.93s/it] 
  1%|          | 1626/225000 [8:00:01<811:04:40, 13.07s/it]
  1%|          | 1627/225000 [8:00:19<892:25:49, 14.38s/it]
  1%|          | 1628/225000 [8:00:34<910:52:48, 14.68s/it]
  1%|          | 1629/225000 [8:00:50<935:57:37, 15.08s/it]
  1%|          | 1630/225000 [8:01:08<995:53:54, 16.05s/it]
  1%|          | 1631/225000 [8:01:28<1056:24:34, 17.03s/it]
  1%|          | 1632/225000 [8:01:42<1013:32:11, 16.34s/it]
  1%|          | 1633/225000 [8:01:56<963:42:34, 15.53s/it] 
  1%|          | 1634/225000 [8:02:10<937:15:07, 15.11s/it]
  1%|          | 1635/225000 [8:02:21<849:04:12, 13.68s/it]
  1%|          | 1636/225000 [8:02:31<790:17:01, 12.74s/it]
  1%|          | 1637/225000 [8:02:45<816:39:15, 13.16s/it]
  1%|          | 1638/225000 [8:02:59<829:48:43, 13.37s/it]
  1%|          | 1639/225000 [8:03:15<868:09:04, 13.99s/it]
  1%|          | 1640/225000 [8:03:25<803:51:15, 12.96s/it]
  1%|          | 1641/225000 [8:03:41<851:36:30, 13.73s/it]
  1%|          | 1642/225000 [8:03:53<821:41:15, 13.24s/it]
  1%|          | 1643/225000 [8:04:11<918:06:42, 14.80s/it]
  1%|          | 1644/225000 [8:04:30<992:43:55, 16.00s/it]
  1%|          | 1645/225000 [8:04:48<1033:32:52, 16.66s/it]
  1%|          | 1646/225000 [8:05:04<1025:05:52, 16.52s/it]
  1%|          | 1647/225000 [8:05:26<1125:00:36, 18.13s/it]
  1%|          | 1648/225000 [8:05:42<1081:32:27, 17.43s/it]
  1%|          | 1649/225000 [8:05:50<907:18:14, 14.62s/it] 
  1%|          | 1650/225000 [8:06:00<816:40:17, 13.16s/it]
                                                           
{'loss': '1.091', 'grad_norm': '1.703', 'learning_rate': '8.23e-06', 'epoch': '0.007333', 'num_input_tokens_seen': 15209264, 'train_runtime': '2.916e+04', 'train_tokens_per_second': '521.6'}

  1%|          | 1650/225000 [8:06:00<816:40:17, 13.16s/it]
  1%|          | 1651/225000 [8:06:18<905:16:53, 14.59s/it]
  1%|          | 1652/225000 [8:06:25<772:07:17, 12.45s/it]
  1%|          | 1653/225000 [8:06:45<908:51:15, 14.65s/it]
  1%|          | 1654/225000 [8:07:01<933:54:54, 15.05s/it]
  1%|          | 1655/225000 [8:07:13<876:29:45, 14.13s/it]
  1%|          | 1656/225000 [8:07:30<927:02:59, 14.94s/it]
  1%|          | 1657/225000 [8:07:49<1000:43:46, 16.13s/it]
  1%|          | 1658/225000 [8:07:58<872:29:52, 14.06s/it] 
  1%|          | 1659/225000 [8:08:10<839:02:26, 13.52s/it]
  1%|          | 1660/225000 [8:08:28<912:16:14, 14.70s/it]
  1%|          | 1661/225000 [8:08:51<1077:57:17, 17.38s/it]
  1%|          | 1662/225000 [8:09:04<989:59:15, 15.96s/it] 
  1%|          | 1663/225000 [8:09:19<972:25:54, 15.67s/it]
  1%|          | 1664/225000 [8:09:37<1016:23:05, 16.38s/it]
  1%|          | 1665/225000 [8:09:52<990:34:48, 15.97s/it] 
  1%|          | 1666/225000 [8:10:02<876:01:59, 14.12s/it]
  1%|          | 1667/225000 [8:10:10<766:45:07, 12.36s/it]
  1%|          | 1668/225000 [8:10:23<768:13:50, 12.38s/it]
  1%|          | 1669/225000 [8:10:38<831:16:19, 13.40s/it]
  1%|          | 1670/225000 [8:10:57<930:47:35, 15.00s/it]
  1%|          | 1671/225000 [8:11:07<835:04:08, 13.46s/it]
  1%|          | 1672/225000 [8:11:24<901:53:55, 14.54s/it]
  1%|          | 1673/225000 [8:11:49<1090:43:19, 17.58s/it]
  1%|          | 1674/225000 [8:12:05<1062:50:55, 17.13s/it]
  1%|          | 1675/225000 [8:12:17<976:34:14, 15.74s/it] 
  1%|          | 1676/225000 [8:12:26<855:33:12, 13.79s/it]
  1%|          | 1677/225000 [8:12:41<864:15:55, 13.93s/it]
  1%|          | 1678/225000 [8:13:01<975:02:45, 15.72s/it]
  1%|          | 1679/225000 [8:13:12<894:49:34, 14.42s/it]
  1%|          | 1680/225000 [8:13:27<911:11:44, 14.69s/it]
  1%|          | 1681/225000 [8:13:47<1011:01:25, 16.30s/it]
  1%|          | 1682/225000 [8:14:06<1062:46:43, 17.13s/it]
  1%|          | 1683/225000 [8:14:21<1006:11:17, 16.22s/it]
  1%|          | 1684/225000 [8:14:35<967:45:30, 15.60s/it] 
  1%|          | 1685/225000 [8:14:43<825:02:39, 13.30s/it]
  1%|          | 1686/225000 [8:14:52<749:20:37, 12.08s/it]
  1%|          | 1687/225000 [8:15:18<1007:30:49, 16.24s/it]
  1%|          | 1688/225000 [8:15:31<942:52:17, 15.20s/it] 
  1%|          | 1689/225000 [8:15:39<807:55:30, 13.02s/it]
  1%|          | 1690/225000 [8:15:58<929:44:32, 14.99s/it]
  1%|          | 1691/225000 [8:16:09<859:22:45, 13.85s/it]
  1%|          | 1692/225000 [8:16:24<876:53:23, 14.14s/it]
  1%|          | 1693/225000 [8:16:41<922:54:11, 14.88s/it]
  1%|          | 1694/225000 [8:16:58<972:14:43, 15.67s/it]
  1%|          | 1695/225000 [8:17:06<829:39:31, 13.38s/it]
  1%|          | 1696/225000 [8:17:27<957:44:09, 15.44s/it]
  1%|          | 1697/225000 [8:17:43<974:40:30, 15.71s/it]
  1%|          | 1698/225000 [8:17:56<920:32:46, 14.84s/it]
  1%|          | 1699/225000 [8:18:10<913:43:41, 14.73s/it]
  1%|          | 1700/225000 [8:18:23<882:21:37, 14.23s/it]
                                                           
{'loss': '1.09', 'grad_norm': '1.691', 'learning_rate': '8.48e-06', 'epoch': '0.007556', 'num_input_tokens_seen': 15592704, 'train_runtime': '2.99e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1700/225000 [8:18:23<882:21:37, 14.23s/it]
  1%|          | 1701/225000 [8:18:38<888:44:51, 14.33s/it]
  1%|          | 1702/225000 [8:18:58<991:27:40, 15.98s/it]
  1%|          | 1703/225000 [8:19:11<942:43:50, 15.20s/it]
  1%|          | 1704/225000 [8:19:23<892:13:51, 14.38s/it]
  1%|          | 1705/225000 [8:19:40<931:25:23, 15.02s/it]
  1%|          | 1706/225000 [8:19:47<788:09:46, 12.71s/it]
  1%|          | 1707/225000 [8:19:55<700:11:08, 11.29s/it]
  1%|          | 1708/225000 [8:20:11<790:57:29, 12.75s/it]
  1%|          | 1709/225000 [8:20:21<732:21:41, 11.81s/it]
  1%|          | 1710/225000 [8:20:37<812:56:47, 13.11s/it]
  1%|          | 1711/225000 [8:20:51<825:45:43, 13.31s/it]
  1%|          | 1712/225000 [8:21:00<751:05:01, 12.11s/it]
  1%|          | 1713/225000 [8:21:09<694:59:06, 11.21s/it]
  1%|          | 1714/225000 [8:21:20<681:03:54, 10.98s/it]
  1%|          | 1715/225000 [8:21:36<783:57:11, 12.64s/it]
  1%|          | 1716/225000 [8:21:57<932:29:32, 15.03s/it]
  1%|          | 1717/225000 [8:22:15<985:45:18, 15.89s/it]
  1%|          | 1718/225000 [8:22:37<1095:42:52, 17.67s/it]
  1%|          | 1719/225000 [8:22:53<1068:55:17, 17.23s/it]
  1%|          | 1720/225000 [8:23:05<966:32:54, 15.58s/it] 
  1%|          | 1721/225000 [8:23:20<960:47:26, 15.49s/it]
  1%|          | 1722/225000 [8:23:29<844:09:41, 13.61s/it]
  1%|          | 1723/225000 [8:23:45<881:22:38, 14.21s/it]
  1%|          | 1724/225000 [8:23:58<871:30:51, 14.05s/it]
  1%|          | 1725/225000 [8:24:14<891:39:51, 14.38s/it]
  1%|          | 1726/225000 [8:24:24<827:32:17, 13.34s/it]
  1%|          | 1727/225000 [8:24:41<879:03:00, 14.17s/it]
  1%|          | 1728/225000 [8:24:50<789:27:27, 12.73s/it]
  1%|          | 1729/225000 [8:25:12<971:05:13, 15.66s/it]
  1%|          | 1730/225000 [8:25:31<1026:42:04, 16.55s/it]
  1%|          | 1731/225000 [8:25:46<1000:59:15, 16.14s/it]
  1%|          | 1732/225000 [8:25:55<865:50:39, 13.96s/it] 
  1%|          | 1733/225000 [8:26:16<995:41:27, 16.05s/it]
  1%|          | 1734/225000 [8:26:33<1019:52:41, 16.44s/it]
  1%|          | 1735/225000 [8:26:47<964:48:11, 15.56s/it] 
  1%|          | 1736/225000 [8:27:06<1033:41:56, 16.67s/it]
  1%|          | 1737/225000 [8:27:17<931:47:55, 15.02s/it] 
  1%|          | 1738/225000 [8:27:36<990:02:42, 15.96s/it]
  1%|          | 1739/225000 [8:27:50<964:28:52, 15.55s/it]
  1%|          | 1740/225000 [8:28:05<943:34:46, 15.21s/it]
  1%|          | 1741/225000 [8:28:19<936:39:11, 15.10s/it]
  1%|          | 1742/225000 [8:28:33<911:39:15, 14.70s/it]
  1%|          | 1743/225000 [8:28:47<887:51:53, 14.32s/it]
  1%|          | 1744/225000 [8:29:01<891:05:38, 14.37s/it]
  1%|          | 1745/225000 [8:29:12<826:34:25, 13.33s/it]
  1%|          | 1746/225000 [8:29:32<949:31:47, 15.31s/it]
  1%|          | 1747/225000 [8:29:46<921:12:07, 14.85s/it]
  1%|          | 1748/225000 [8:29:54<801:06:48, 12.92s/it]
  1%|          | 1749/225000 [8:30:18<998:04:55, 16.09s/it]
  1%|          | 1750/225000 [8:30:27<870:47:41, 14.04s/it]
                                                           
{'loss': '1.067', 'grad_norm': '1.089', 'learning_rate': '8.73e-06', 'epoch': '0.007778', 'num_input_tokens_seen': 15978072, 'train_runtime': '3.063e+04', 'train_tokens_per_second': '521.7'}

  1%|          | 1750/225000 [8:30:27<870:47:41, 14.04s/it]
  1%|          | 1751/225000 [8:30:49<1025:29:41, 16.54s/it]
  1%|          | 1752/225000 [8:30:56<846:54:11, 13.66s/it] 
  1%|          | 1753/225000 [8:31:14<923:37:02, 14.89s/it]
  1%|          | 1754/225000 [8:31:29<930:01:42, 15.00s/it]
  1%|          | 1755/225000 [8:31:44<921:05:05, 14.85s/it]
  1%|          | 1756/225000 [8:31:55<855:26:17, 13.79s/it]
  1%|          | 1757/225000 [8:32:12<906:39:14, 14.62s/it]
  1%|          | 1758/225000 [8:32:30<982:32:48, 15.84s/it]
  1%|          | 1759/225000 [8:32:48<1024:17:56, 16.52s/it]
  1%|          | 1760/225000 [8:32:57<871:49:00, 14.06s/it] 
  1%|          | 1761/225000 [8:33:14<930:42:50, 15.01s/it]
  1%|          | 1762/225000 [8:33:26<885:26:21, 14.28s/it]
  1%|          | 1763/225000 [8:33:33<735:56:09, 11.87s/it]
  1%|          | 1764/225000 [8:33:55<937:43:07, 15.12s/it]
  1%|          | 1765/225000 [8:34:15<1021:47:45, 16.48s/it]
  1%|          | 1766/225000 [8:34:32<1034:52:18, 16.69s/it]
  1%|          | 1767/225000 [8:34:40<862:03:13, 13.90s/it] 
  1%|          | 1768/225000 [8:34:56<906:14:23, 14.61s/it]
  1%|          | 1769/225000 [8:35:05<805:21:22, 12.99s/it]
  1%|          | 1770/225000 [8:35:21<860:40:01, 13.88s/it]
  1%|          | 1771/225000 [8:35:40<954:38:40, 15.40s/it]
  1%|          | 1772/225000 [8:35:53<904:21:47, 14.58s/it]
  1%|          | 1773/225000 [8:36:17<1081:46:07, 17.45s/it]
  1%|          | 1774/225000 [8:36:35<1087:53:11, 17.54s/it]
  1%|          | 1775/225000 [8:36:48<1004:47:02, 16.20s/it]
  1%|          | 1776/225000 [8:37:03<982:37:47, 15.85s/it] 
  1%|          | 1777/225000 [8:37:20<1003:23:19, 16.18s/it]
  1%|          | 1778/225000 [8:37:33<945:27:18, 15.25s/it] 
  1%|          | 1779/225000 [8:37:53<1037:57:03, 16.74s/it]
  1%|          | 1780/225000 [8:38:12<1089:59:38, 17.58s/it]
  1%|          | 1781/225000 [8:38:28<1042:49:04, 16.82s/it]
  1%|          | 1782/225000 [8:38:37<907:39:54, 14.64s/it] 
  1%|          | 1783/225000 [8:38:53<939:56:19, 15.16s/it]
  1%|          | 1784/225000 [8:39:04<854:58:12, 13.79s/it]
  1%|          | 1785/225000 [8:39:17<845:28:09, 13.64s/it]
  1%|          | 1786/225000 [8:39:32<862:40:35, 13.91s/it]
  1%|          | 1787/225000 [8:39:50<945:57:04, 15.26s/it]
  1%|          | 1788/225000 [8:40:02<887:51:08, 14.32s/it]
  1%|          | 1789/225000 [8:40:21<961:53:36, 15.51s/it]
  1%|          | 1790/225000 [8:40:35<945:06:29, 15.24s/it]
  1%|          | 1791/225000 [8:40:51<955:37:08, 15.41s/it]
  1%|          | 1792/225000 [8:41:03<892:04:33, 14.39s/it]
  1%|          | 1793/225000 [8:41:20<930:53:49, 15.01s/it]
  1%|          | 1794/225000 [8:41:32<888:30:54, 14.33s/it]
  1%|          | 1795/225000 [8:41:47<889:12:02, 14.34s/it]
  1%|          | 1796/225000 [8:42:04<950:05:49, 15.32s/it]
  1%|          | 1797/225000 [8:42:18<913:39:06, 14.74s/it]
  1%|          | 1798/225000 [8:42:36<973:31:27, 15.70s/it]
  1%|          | 1799/225000 [8:42:45<859:04:46, 13.86s/it]
  1%|          | 1800/225000 [8:42:59<855:28:24, 13.80s/it]
                                                           
{'loss': '1.063', 'grad_norm': '2.823', 'learning_rate': '8.98e-06', 'epoch': '0.008', 'num_input_tokens_seen': 16367064, 'train_runtime': '3.138e+04', 'train_tokens_per_second': '521.6'}

  1%|          | 1800/225000 [8:42:59<855:28:24, 13.80s/it]
  1%|          | 1801/225000 [8:43:09<790:46:33, 12.75s/it]
  1%|          | 1802/225000 [8:43:29<929:27:07, 14.99s/it]
  1%|          | 1803/225000 [8:43:42<878:15:12, 14.17s/it]
  1%|          | 1804/225000 [8:43:58<913:58:01, 14.74s/it]
  1%|          | 1805/225000 [8:44:15<957:11:21, 15.44s/it]
  1%|          | 1806/225000 [8:44:30<957:13:10, 15.44s/it]
  1%|          | 1807/225000 [8:44:52<1081:21:14, 17.44s/it]
  1%|          | 1808/225000 [8:45:07<1033:55:30, 16.68s/it]
  1%|          | 1809/225000 [8:45:18<933:40:09, 15.06s/it] 
  1%|          | 1810/225000 [8:45:36<979:03:30, 15.79s/it]
  1%|          | 1811/225000 [8:45:52<987:28:49, 15.93s/it]
  1%|          | 1812/225000 [8:46:11<1034:48:31, 16.69s/it]
  1%|          | 1813/225000 [8:46:21<916:42:31, 14.79s/it] 
  1%|          | 1814/225000 [8:46:34<884:45:22, 14.27s/it]
  1%|          | 1815/225000 [8:46:52<943:37:31, 15.22s/it]
  1%|          | 1816/225000 [8:47:05<903:21:07, 14.57s/it]
  1%|          | 1817/225000 [8:47:18<873:41:20, 14.09s/it]
  1%|          | 1818/225000 [8:47:39<1014:08:02, 16.36s/it]
  1%|          | 1819/225000 [8:47:51<920:55:21, 14.85s/it] 
  1%|          | 1820/225000 [8:48:03<870:22:35, 14.04s/it]
  1%|          | 1821/225000 [8:48:15<831:30:07, 13.41s/it]
  1%|          | 1822/225000 [8:48:28<830:37:55, 13.40s/it]
  1%|          | 1823/225000 [8:48:43<865:53:59, 13.97s/it]
  1%|          | 1824/225000 [8:48:54<803:47:17, 12.97s/it]
  1%|          | 1825/225000 [8:49:08<824:29:37, 13.30s/it]
  1%|          | 1826/225000 [8:49:23<860:03:01, 13.87s/it]
  1%|          | 1827/225000 [8:49:40<905:57:33, 14.61s/it]
  1%|          | 1828/225000 [8:49:51<838:29:36, 13.53s/it]
  1%|          | 1829/225000 [8:50:14<1015:46:58, 16.39s/it]
  1%|          | 1830/225000 [8:50:21<853:49:37, 13.77s/it] 
  1%|          | 1831/225000 [8:50:42<984:22:19, 15.88s/it]
  1%|          | 1832/225000 [8:50:56<955:51:23, 15.42s/it]
  1%|          | 1833/225000 [8:51:10<916:28:41, 14.78s/it]
  1%|          | 1834/225000 [8:51:23<896:27:36, 14.46s/it]
  1%|          | 1835/225000 [8:51:40<943:48:36, 15.23s/it]
  1%|          | 1836/225000 [8:51:48<805:34:38, 13.00s/it]
  1%|          | 1837/225000 [8:52:00<780:24:46, 12.59s/it]
  1%|          | 1838/225000 [8:52:13<788:51:11, 12.73s/it]
  1%|          | 1839/225000 [8:52:29<845:21:06, 13.64s/it]
  1%|          | 1840/225000 [8:52:39<790:51:13, 12.76s/it]
  1%|          | 1841/225000 [8:52:59<912:23:51, 14.72s/it]
  1%|          | 1842/225000 [8:53:12<893:10:13, 14.41s/it]
  1%|          | 1843/225000 [8:53:27<901:44:55, 14.55s/it]
  1%|          | 1844/225000 [8:53:52<1098:01:01, 17.71s/it]
  1%|          | 1845/225000 [8:54:08<1056:13:26, 17.04s/it]
  1%|          | 1846/225000 [8:54:23<1030:52:41, 16.63s/it]
  1%|          | 1847/225000 [8:54:35<938:09:14, 15.13s/it] 
  1%|          | 1848/225000 [8:54:49<912:06:46, 14.71s/it]
  1%|          | 1849/225000 [8:55:00<853:50:20, 13.77s/it]
  1%|          | 1850/225000 [8:55:15<876:35:37, 14.14s/it]
                                                           
{'loss': '1.077', 'grad_norm': '1.617', 'learning_rate': '9.23e-06', 'epoch': '0.008222', 'num_input_tokens_seen': 16746824, 'train_runtime': '3.212e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1850/225000 [8:55:15<876:35:37, 14.14s/it]
  1%|          | 1851/225000 [8:55:30<883:08:12, 14.25s/it]
  1%|          | 1852/225000 [8:55:48<946:03:46, 15.26s/it]
  1%|          | 1853/225000 [8:55:57<831:31:10, 13.41s/it]
  1%|          | 1854/225000 [8:56:14<906:23:04, 14.62s/it]
  1%|          | 1855/225000 [8:56:33<993:29:13, 16.03s/it]
  1%|          | 1856/225000 [8:56:47<952:29:23, 15.37s/it]
  1%|          | 1857/225000 [8:57:00<909:28:47, 14.67s/it]
  1%|          | 1858/225000 [8:57:19<985:50:32, 15.90s/it]
  1%|          | 1859/225000 [8:57:31<908:54:37, 14.66s/it]
  1%|          | 1860/225000 [8:57:51<1006:57:09, 16.25s/it]
  1%|          | 1861/225000 [8:58:11<1089:39:06, 17.58s/it]
  1%|          | 1862/225000 [8:58:25<1018:53:15, 16.44s/it]
  1%|          | 1863/225000 [8:58:36<907:41:30, 14.64s/it] 
  1%|          | 1864/225000 [8:58:52<936:52:05, 15.12s/it]
  1%|          | 1865/225000 [8:59:08<955:24:39, 15.41s/it]
  1%|          | 1866/225000 [8:59:22<933:40:48, 15.06s/it]
  1%|          | 1867/225000 [8:59:36<901:02:12, 14.54s/it]
  1%|          | 1868/225000 [8:59:47<838:57:48, 13.54s/it]
  1%|          | 1869/225000 [8:59:58<799:26:24, 12.90s/it]
  1%|          | 1870/225000 [9:00:15<865:53:35, 13.97s/it]
  1%|          | 1871/225000 [9:00:27<831:20:47, 13.41s/it]
  1%|          | 1872/225000 [9:00:39<802:16:16, 12.94s/it]
  1%|          | 1873/225000 [9:00:51<792:24:59, 12.79s/it]
  1%|          | 1874/225000 [9:01:06<826:32:52, 13.34s/it]
  1%|          | 1875/225000 [9:01:19<828:14:46, 13.36s/it]
  1%|          | 1876/225000 [9:01:37<906:26:42, 14.63s/it]
  1%|          | 1877/225000 [9:01:54<961:46:04, 15.52s/it]
  1%|          | 1878/225000 [9:02:16<1069:28:34, 17.26s/it]
  1%|          | 1879/225000 [9:02:34<1088:12:26, 17.56s/it]
  1%|          | 1880/225000 [9:02:54<1132:46:10, 18.28s/it]
  1%|          | 1881/225000 [9:03:13<1150:01:23, 18.56s/it]
  1%|          | 1882/225000 [9:03:20<942:20:44, 15.20s/it] 
  1%|          | 1883/225000 [9:03:30<831:43:20, 13.42s/it]
  1%|          | 1884/225000 [9:03:52<997:08:53, 16.09s/it]
  1%|          | 1885/225000 [9:04:02<883:44:55, 14.26s/it]
  1%|          | 1886/225000 [9:04:09<742:58:19, 11.99s/it]
  1%|          | 1887/225000 [9:04:20<722:48:53, 11.66s/it]
  1%|          | 1888/225000 [9:04:39<863:03:30, 13.93s/it]
  1%|          | 1889/225000 [9:04:49<798:13:54, 12.88s/it]
  1%|          | 1890/225000 [9:05:02<800:54:35, 12.92s/it]
  1%|          | 1891/225000 [9:05:15<793:00:12, 12.80s/it]
  1%|          | 1892/225000 [9:05:29<817:37:46, 13.19s/it]
  1%|          | 1893/225000 [9:05:47<903:15:30, 14.57s/it]
  1%|          | 1894/225000 [9:05:58<837:02:14, 13.51s/it]
  1%|          | 1895/225000 [9:06:14<890:42:03, 14.37s/it]
  1%|          | 1896/225000 [9:06:24<812:24:04, 13.11s/it]
  1%|          | 1897/225000 [9:06:33<738:17:49, 11.91s/it]
  1%|          | 1898/225000 [9:06:44<714:01:00, 11.52s/it]
  1%|          | 1899/225000 [9:07:01<823:08:34, 13.28s/it]
  1%|          | 1900/225000 [9:07:21<933:29:32, 15.06s/it]
                                                           
{'loss': '1.025', 'grad_norm': '2.106', 'learning_rate': '9.48e-06', 'epoch': '0.008444', 'num_input_tokens_seen': 17122840, 'train_runtime': '3.284e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1900/225000 [9:07:21<933:29:32, 15.06s/it]
  1%|          | 1901/225000 [9:07:33<889:28:17, 14.35s/it]
  1%|          | 1902/225000 [9:07:47<884:03:07, 14.27s/it]
  1%|          | 1903/225000 [9:07:59<829:39:38, 13.39s/it]
  1%|          | 1904/225000 [9:08:06<714:20:49, 11.53s/it]
  1%|          | 1905/225000 [9:08:17<711:31:45, 11.48s/it]
  1%|          | 1906/225000 [9:08:30<739:18:33, 11.93s/it]
  1%|          | 1907/225000 [9:08:38<668:02:29, 10.78s/it]
  1%|          | 1908/225000 [9:08:54<750:46:22, 12.12s/it]
  1%|          | 1909/225000 [9:09:04<721:33:17, 11.64s/it]
  1%|          | 1910/225000 [9:09:19<791:12:45, 12.77s/it]
  1%|          | 1911/225000 [9:09:31<763:51:07, 12.33s/it]
  1%|          | 1912/225000 [9:09:52<925:40:35, 14.94s/it]
  1%|          | 1913/225000 [9:10:05<892:50:38, 14.41s/it]
  1%|          | 1914/225000 [9:10:27<1043:30:16, 16.84s/it]
  1%|          | 1915/225000 [9:10:39<941:51:07, 15.20s/it] 
  1%|          | 1916/225000 [9:10:58<1022:48:22, 16.51s/it]
  1%|          | 1917/225000 [9:11:12<972:27:39, 15.69s/it] 
  1%|          | 1918/225000 [9:11:21<838:16:03, 13.53s/it]
  1%|          | 1919/225000 [9:11:44<1014:28:08, 16.37s/it]
  1%|          | 1920/225000 [9:11:59<1001:15:09, 16.16s/it]
  1%|          | 1921/225000 [9:12:10<904:06:25, 14.59s/it] 
  1%|          | 1922/225000 [9:12:26<929:26:06, 15.00s/it]
  1%|          | 1923/225000 [9:12:35<808:40:49, 13.05s/it]
  1%|          | 1924/225000 [9:12:46<781:39:57, 12.61s/it]
  1%|          | 1925/225000 [9:13:00<799:24:57, 12.90s/it]
  1%|          | 1926/225000 [9:13:13<803:47:50, 12.97s/it]
  1%|          | 1927/225000 [9:13:27<815:14:53, 13.16s/it]
  1%|          | 1928/225000 [9:13:47<945:42:51, 15.26s/it]
  1%|          | 1929/225000 [9:14:00<913:46:19, 14.75s/it]
  1%|          | 1930/225000 [9:14:13<879:19:19, 14.19s/it]
  1%|          | 1931/225000 [9:14:26<850:55:46, 13.73s/it]
  1%|          | 1932/225000 [9:14:43<904:53:17, 14.60s/it]
  1%|          | 1933/225000 [9:14:52<804:11:23, 12.98s/it]
  1%|          | 1934/225000 [9:15:12<942:19:10, 15.21s/it]
  1%|          | 1935/225000 [9:15:20<813:19:24, 13.13s/it]
  1%|          | 1936/225000 [9:15:31<766:36:50, 12.37s/it]
  1%|          | 1937/225000 [9:15:46<809:01:19, 13.06s/it]
  1%|          | 1938/225000 [9:15:56<755:10:22, 12.19s/it]
  1%|          | 1939/225000 [9:16:10<787:35:58, 12.71s/it]
  1%|          | 1940/225000 [9:16:31<952:41:49, 15.38s/it]
  1%|          | 1941/225000 [9:16:39<812:51:09, 13.12s/it]
  1%|          | 1942/225000 [9:16:55<859:32:06, 13.87s/it]
  1%|          | 1943/225000 [9:17:08<843:27:14, 13.61s/it]
  1%|          | 1944/225000 [9:17:15<726:24:04, 11.72s/it]
  1%|          | 1945/225000 [9:17:35<884:59:33, 14.28s/it]
  1%|          | 1946/225000 [9:17:51<901:34:58, 14.55s/it]
  1%|          | 1947/225000 [9:18:10<989:15:28, 15.97s/it]
  1%|          | 1948/225000 [9:18:21<906:30:12, 14.63s/it]
  1%|          | 1949/225000 [9:18:32<834:24:50, 13.47s/it]
  1%|          | 1950/225000 [9:18:49<900:06:46, 14.53s/it]
                                                           
{'loss': '1.015', 'grad_norm': '24.93', 'learning_rate': '9.725e-06', 'epoch': '0.008667', 'num_input_tokens_seen': 17482288, 'train_runtime': '3.353e+04', 'train_tokens_per_second': '521.4'}

  1%|          | 1950/225000 [9:18:49<900:06:46, 14.53s/it]
  1%|          | 1951/225000 [9:19:11<1043:11:08, 16.84s/it]
  1%|          | 1952/225000 [9:19:23<948:26:56, 15.31s/it] 
  1%|          | 1953/225000 [9:19:37<927:50:34, 14.98s/it]
  1%|          | 1954/225000 [9:19:49<875:47:32, 14.14s/it]
  1%|          | 1955/225000 [9:20:06<915:25:56, 14.78s/it]
  1%|          | 1956/225000 [9:20:18<877:11:13, 14.16s/it]
  1%|          | 1957/225000 [9:20:34<906:14:49, 14.63s/it]
  1%|          | 1958/225000 [9:20:46<845:21:43, 13.64s/it]
  1%|          | 1959/225000 [9:21:02<900:43:19, 14.54s/it]
  1%|          | 1960/225000 [9:21:20<967:25:10, 15.61s/it]
  1%|          | 1961/225000 [9:21:38<1016:09:17, 16.40s/it]
  1%|          | 1962/225000 [9:21:48<893:11:50, 14.42s/it] 
  1%|          | 1963/225000 [9:21:59<829:13:34, 13.38s/it]
  1%|          | 1964/225000 [9:22:12<809:56:42, 13.07s/it]
  1%|          | 1965/225000 [9:22:26<841:06:55, 13.58s/it]
  1%|          | 1966/225000 [9:22:38<801:00:38, 12.93s/it]
  1%|          | 1967/225000 [9:22:52<824:33:56, 13.31s/it]
  1%|          | 1968/225000 [9:23:06<830:41:06, 13.41s/it]
  1%|          | 1969/225000 [9:23:22<883:54:53, 14.27s/it]
  1%|          | 1970/225000 [9:23:34<841:05:56, 13.58s/it]
  1%|          | 1971/225000 [9:23:45<791:20:58, 12.77s/it]
  1%|          | 1972/225000 [9:24:06<955:26:32, 15.42s/it]
  1%|          | 1973/225000 [9:24:21<935:22:44, 15.10s/it]
  1%|          | 1974/225000 [9:24:40<1010:06:24, 16.30s/it]
  1%|          | 1975/225000 [9:24:52<932:10:07, 15.05s/it] 
  1%|          | 1976/225000 [9:25:03<863:34:58, 13.94s/it]
  1%|          | 1977/225000 [9:25:18<880:56:49, 14.22s/it]
  1%|          | 1978/225000 [9:25:32<870:53:10, 14.06s/it]
  1%|          | 1979/225000 [9:25:48<902:27:17, 14.57s/it]
  1%|          | 1980/225000 [9:26:03<911:08:43, 14.71s/it]
  1%|          | 1981/225000 [9:26:13<831:23:45, 13.42s/it]
  1%|          | 1982/225000 [9:26:31<917:37:16, 14.81s/it]
  1%|          | 1983/225000 [9:26:40<799:03:30, 12.90s/it]
  1%|          | 1984/225000 [9:27:01<949:38:26, 15.33s/it]
  1%|          | 1985/225000 [9:27:11<867:38:58, 14.01s/it]
  1%|          | 1986/225000 [9:27:23<828:06:26, 13.37s/it]
  1%|          | 1987/225000 [9:27:35<789:45:51, 12.75s/it]
  1%|          | 1988/225000 [9:27:42<692:01:09, 11.17s/it]
  1%|          | 1989/225000 [9:27:59<795:07:05, 12.84s/it]
  1%|          | 1990/225000 [9:28:16<877:27:13, 14.16s/it]
  1%|          | 1991/225000 [9:28:32<910:20:02, 14.70s/it]
  1%|          | 1992/225000 [9:28:54<1041:36:45, 16.81s/it]
  1%|          | 1993/225000 [9:29:03<901:31:52, 14.55s/it] 
  1%|          | 1994/225000 [9:29:21<959:40:39, 15.49s/it]
  1%|          | 1995/225000 [9:29:38<992:18:56, 16.02s/it]
  1%|          | 1996/225000 [9:29:46<846:08:54, 13.66s/it]
  1%|          | 1997/225000 [9:30:02<890:03:26, 14.37s/it]
  1%|          | 1998/225000 [9:30:21<974:42:27, 15.74s/it]
  1%|          | 1999/225000 [9:30:36<967:47:25, 15.62s/it]
  1%|          | 2000/225000 [9:30:49<912:36:39, 14.73s/it]
                                                           
{'loss': '1.049', 'grad_norm': '2.003', 'learning_rate': '9.975e-06', 'epoch': '0.008889', 'num_input_tokens_seen': 17850744, 'train_runtime': '3.425e+04', 'train_tokens_per_second': '521.2'}

  1%|          | 2000/225000 [9:30:49<912:36:39, 14.73s/it]
  1%|          | 2001/225000 [9:31:06<954:07:50, 15.40s/it]
  1%|          | 2002/225000 [9:31:17<876:47:18, 14.15s/it]
  1%|          | 2003/225000 [9:31:33<901:57:39, 14.56s/it]
  1%|          | 2004/225000 [9:31:44<839:11:40, 13.55s/it]
  1%|          | 2005/225000 [9:32:07<1015:40:40, 16.40s/it]
  1%|          | 2006/225000 [9:32:20<944:56:28, 15.26s/it] 
  1%|          | 2007/225000 [9:32:30<856:41:53, 13.83s/it]
  1%|          | 2008/225000 [9:32:39<761:33:58, 12.29s/it]
  1%|          | 2009/225000 [9:32:55<841:42:47, 13.59s/it]
  1%|          | 2010/225000 [9:33:04<747:45:02, 12.07s/it]
  1%|          | 2011/225000 [9:33:22<860:58:30, 13.90s/it]
  1%|          | 2012/225000 [9:33:40<936:23:07, 15.12s/it]
  1%|          | 2013/225000 [9:33:57<963:16:22, 15.55s/it]
  1%|          | 2014/225000 [9:34:14<989:45:04, 15.98s/it]
  1%|          | 2015/225000 [9:34:20<815:29:31, 13.17s/it]
  1%|          | 2016/225000 [9:34:33<802:34:10, 12.96s/it]
  1%|          | 2017/225000 [9:34:47<833:12:52, 13.45s/it]
  1%|          | 2018/225000 [9:34:57<755:45:11, 12.20s/it]
  1%|          | 2019/225000 [9:35:09<760:52:04, 12.28s/it]
  1%|          | 2020/225000 [9:35:27<861:09:04, 13.90s/it]
  1%|          | 2021/225000 [9:35:43<906:10:14, 14.63s/it]
  1%|          | 2022/225000 [9:35:53<811:27:08, 13.10s/it]
  1%|          | 2023/225000 [9:36:05<801:02:04, 12.93s/it]
  1%|          | 2024/225000 [9:36:22<870:45:14, 14.06s/it]
  1%|          | 2025/225000 [9:36:43<1001:29:07, 16.17s/it]
  1%|          | 2026/225000 [9:36:53<889:45:54, 14.37s/it] 
  1%|          | 2027/225000 [9:37:08<907:42:24, 14.66s/it]
  1%|          | 2028/225000 [9:37:20<842:15:07, 13.60s/it]
  1%|          | 2029/225000 [9:37:31<794:34:50, 12.83s/it]
  1%|          | 2030/225000 [9:37:47<868:10:27, 14.02s/it]
  1%|          | 2031/225000 [9:37:55<756:34:55, 12.22s/it]
  1%|          | 2032/225000 [9:38:15<901:32:18, 14.56s/it]
  1%|          | 2033/225000 [9:38:34<976:29:05, 15.77s/it]
  1%|          | 2034/225000 [9:38:48<949:20:23, 15.33s/it]
  1%|          | 2035/225000 [9:39:15<1161:35:50, 18.76s/it]
  1%|          | 2036/225000 [9:39:29<1068:47:14, 17.26s/it]
  1%|          | 2037/225000 [9:39:43<1013:02:24, 16.36s/it]
  1%|          | 2038/225000 [9:39:50<838:56:51, 13.55s/it] 
  1%|          | 2039/225000 [9:40:01<795:49:09, 12.85s/it]
  1%|          | 2040/225000 [9:40:27<1025:36:44, 16.56s/it]
  1%|          | 2041/225000 [9:40:41<984:34:11, 15.90s/it] 
  1%|          | 2042/225000 [9:40:57<979:31:49, 15.82s/it]
  1%|          | 2043/225000 [9:41:12<967:31:18, 15.62s/it]
  1%|          | 2044/225000 [9:41:25<915:44:23, 14.79s/it]
  1%|          | 2045/225000 [9:41:41<943:57:24, 15.24s/it]
  1%|          | 2046/225000 [9:41:53<893:49:19, 14.43s/it]
  1%|          | 2047/225000 [9:42:14<1013:50:19, 16.37s/it]
  1%|          | 2048/225000 [9:42:24<890:30:41, 14.38s/it] 
  1%|          | 2049/225000 [9:42:38<886:37:59, 14.32s/it]
  1%|          | 2050/225000 [9:42:44<722:15:24, 11.66s/it]
                                                           
{'loss': '1.022', 'grad_norm': '1.845', 'learning_rate': '1.022e-05', 'epoch': '0.009111', 'num_input_tokens_seen': 18216448, 'train_runtime': '3.496e+04', 'train_tokens_per_second': '521'}

  1%|          | 2050/225000 [9:42:44<722:15:24, 11.66s/it]
  1%|          | 2051/225000 [9:42:56<730:50:05, 11.80s/it]
  1%|          | 2052/225000 [9:43:10<779:18:40, 12.58s/it]
  1%|          | 2053/225000 [9:43:31<928:50:17, 15.00s/it]
  1%|          | 2054/225000 [9:43:43<870:17:51, 14.05s/it]
  1%|          | 2055/225000 [9:44:03<989:49:10, 15.98s/it]
  1%|          | 2056/225000 [9:44:20<1001:39:31, 16.17s/it]
  1%|          | 2057/225000 [9:44:27<839:53:50, 13.56s/it] 
  1%|          | 2058/225000 [9:44:44<902:38:05, 14.58s/it]
  1%|          | 2059/225000 [9:44:59<916:26:45, 14.80s/it]
  1%|          | 2060/225000 [9:45:17<970:15:30, 15.67s/it]
  1%|          | 2061/225000 [9:45:32<952:08:55, 15.38s/it]
  1%|          | 2062/225000 [9:45:47<941:06:12, 15.20s/it]
  1%|          | 2063/225000 [9:46:04<974:25:55, 15.74s/it]
  1%|          | 2064/225000 [9:46:16<915:30:49, 14.78s/it]
  1%|          | 2065/225000 [9:46:26<831:47:34, 13.43s/it]
  1%|          | 2066/225000 [9:46:46<944:46:59, 15.26s/it]
  1%|          | 2067/225000 [9:47:01<933:30:32, 15.07s/it]
  1%|          | 2068/225000 [9:47:12<872:19:26, 14.09s/it]
  1%|          | 2069/225000 [9:47:24<832:11:54, 13.44s/it]
  1%|          | 2070/225000 [9:47:41<890:04:54, 14.37s/it]
  1%|          | 2071/225000 [9:47:58<932:37:49, 15.06s/it]
  1%|          | 2072/225000 [9:48:15<984:18:41, 15.90s/it]
  1%|          | 2073/225000 [9:48:32<998:17:55, 16.12s/it]
  1%|          | 2074/225000 [9:48:53<1078:53:16, 17.42s/it]
  1%|          | 2075/225000 [9:49:03<946:14:38, 15.28s/it] 
  1%|          | 2076/225000 [9:49:17<930:23:53, 15.03s/it]
  1%|          | 2077/225000 [9:49:35<986:27:05, 15.93s/it]
  1%|          | 2078/225000 [9:49:41<804:57:54, 13.00s/it]
  1%|          | 2079/225000 [9:49:50<717:49:48, 11.59s/it]
  1%|          | 2080/225000 [9:50:08<845:00:19, 13.65s/it]
  1%|          | 2081/225000 [9:50:20<813:08:59, 13.13s/it]
  1%|          | 2082/225000 [9:50:41<957:32:05, 15.46s/it]
  1%|          | 2083/225000 [9:50:58<980:55:20, 15.84s/it]
  1%|          | 2084/225000 [9:51:15<1003:04:09, 16.20s/it]
  1%|          | 2085/225000 [9:51:23<857:10:06, 13.84s/it] 
  1%|          | 2086/225000 [9:51:31<751:20:40, 12.13s/it]
  1%|          | 2087/225000 [9:51:39<673:05:59, 10.87s/it]
  1%|          | 2088/225000 [9:52:01<884:30:00, 14.28s/it]
  1%|          | 2089/225000 [9:52:13<840:05:41, 13.57s/it]
  1%|          | 2090/225000 [9:52:24<780:12:05, 12.60s/it]
  1%|          | 2091/225000 [9:52:48<989:52:30, 15.99s/it]
  1%|          | 2092/225000 [9:53:10<1113:23:48, 17.98s/it]
  1%|          | 2093/225000 [9:53:27<1085:48:22, 17.54s/it]
  1%|          | 2094/225000 [9:53:41<1026:02:22, 16.57s/it]
  1%|          | 2095/225000 [9:53:52<929:24:16, 15.01s/it] 
  1%|          | 2096/225000 [9:54:07<914:26:53, 14.77s/it]
  1%|          | 2097/225000 [9:54:22<930:20:10, 15.03s/it]
  1%|          | 2098/225000 [9:54:39<961:21:40, 15.53s/it]
  1%|          | 2099/225000 [9:54:50<878:54:09, 14.19s/it]
  1%|          | 2100/225000 [9:55:02<830:18:00, 13.41s/it]
                                                           
{'loss': '1.041', 'grad_norm': '2.142', 'learning_rate': '1.048e-05', 'epoch': '0.009333', 'num_input_tokens_seen': 18590016, 'train_runtime': '3.57e+04', 'train_tokens_per_second': '520.7'}

  1%|          | 2100/225000 [9:55:02<830:18:00, 13.41s/it]
  1%|          | 2101/225000 [9:55:21<939:25:05, 15.17s/it]
  1%|          | 2102/225000 [9:55:31<852:49:49, 13.77s/it]
  1%|          | 2103/225000 [9:55:44<832:47:17, 13.45s/it]
  1%|          | 2104/225000 [9:55:59<859:03:03, 13.87s/it]
  1%|          | 2105/225000 [9:56:10<813:30:07, 13.14s/it]
  1%|          | 2106/225000 [9:56:28<904:54:59, 14.62s/it]
  1%|          | 2107/225000 [9:56:40<848:10:39, 13.70s/it]
  1%|          | 2108/225000 [9:56:56<882:50:09, 14.26s/it]
  1%|          | 2109/225000 [9:57:07<825:31:06, 13.33s/it]
  1%|          | 2110/225000 [9:57:24<907:05:09, 14.65s/it]
  1%|          | 2111/225000 [9:57:41<934:00:36, 15.09s/it]
  1%|          | 2112/225000 [9:57:48<797:14:15, 12.88s/it]
  1%|          | 2113/225000 [9:58:03<838:03:23, 13.54s/it]
  1%|          | 2114/225000 [9:58:20<887:24:55, 14.33s/it]
  1%|          | 2115/225000 [9:58:28<775:11:09, 12.52s/it]
  1%|          | 2116/225000 [9:58:39<751:06:04, 12.13s/it]
  1%|          | 2117/225000 [9:58:59<888:57:25, 14.36s/it]
  1%|          | 2118/225000 [9:59:15<932:48:51, 15.07s/it]
  1%|          | 2119/225000 [9:59:32<965:03:58, 15.59s/it]
  1%|          | 2120/225000 [9:59:44<902:41:43, 14.58s/it]
  1%|          | 2121/225000 [9:59:50<743:35:52, 12.01s/it]
  1%|          | 2122/225000 [10:00:05<797:21:31, 12.88s/it]
  1%|          | 2123/225000 [10:00:16<752:21:44, 12.15s/it]
  1%|          | 2124/225000 [10:00:22<645:51:51, 10.43s/it]
  1%|          | 2125/225000 [10:00:36<713:05:52, 11.52s/it]
  1%|          | 2126/225000 [10:00:49<740:08:41, 11.96s/it]
  1%|          | 2127/225000 [10:01:03<770:59:02, 12.45s/it]
  1%|          | 2128/225000 [10:01:19<842:22:37, 13.61s/it]
  1%|          | 2129/225000 [10:01:37<920:27:49, 14.87s/it]
  1%|          | 2130/225000 [10:01:45<800:16:00, 12.93s/it]
  1%|          | 2131/225000 [10:01:57<768:36:48, 12.42s/it]
  1%|          | 2132/225000 [10:02:21<998:41:19, 16.13s/it]
  1%|          | 2133/225000 [10:02:40<1036:22:43, 16.74s/it]
  1%|          | 2134/225000 [10:02:47<871:33:01, 14.08s/it] 
  1%|          | 2135/225000 [10:03:04<924:18:27, 14.93s/it]
  1%|          | 2136/225000 [10:03:22<969:42:54, 15.66s/it]
  1%|          | 2137/225000 [10:03:37<963:17:06, 15.56s/it]
  1%|          | 2138/225000 [10:03:49<888:30:19, 14.35s/it]
  1%|          | 2139/225000 [10:04:01<857:01:05, 13.84s/it]
  1%|          | 2140/225000 [10:04:15<858:08:16, 13.86s/it]
  1%|          | 2141/225000 [10:04:32<911:11:16, 14.72s/it]
  1%|          | 2142/225000 [10:04:41<817:23:11, 13.20s/it]
  1%|          | 2143/225000 [10:04:54<812:52:47, 13.13s/it]
  1%|          | 2144/225000 [10:05:08<822:08:51, 13.28s/it]
  1%|          | 2145/225000 [10:05:21<820:23:18, 13.25s/it]
  1%|          | 2146/225000 [10:05:34<814:04:57, 13.15s/it]
  1%|          | 2147/225000 [10:05:53<923:12:52, 14.91s/it]
  1%|          | 2148/225000 [10:06:02<816:57:06, 13.20s/it]
  1%|          | 2149/225000 [10:06:22<939:16:09, 15.17s/it]
  1%|          | 2150/225000 [10:06:38<953:33:25, 15.40s/it]
                                                            
{'loss': '1.015', 'grad_norm': '1.654', 'learning_rate': '1.072e-05', 'epoch': '0.009556', 'num_input_tokens_seen': 18947576, 'train_runtime': '3.64e+04', 'train_tokens_per_second': '520.6'}

  1%|          | 2150/225000 [10:06:38<953:33:25, 15.40s/it]
  1%|          | 2151/225000 [10:06:49<873:20:34, 14.11s/it]
  1%|          | 2152/225000 [10:06:59<797:51:15, 12.89s/it]
  1%|          | 2153/225000 [10:07:09<735:12:14, 11.88s/it]
  1%|          | 2154/225000 [10:07:24<798:52:13, 12.91s/it]
  1%|          | 2155/225000 [10:07:35<756:12:10, 12.22s/it]
  1%|          | 2156/225000 [10:07:42<661:05:26, 10.68s/it]
  1%|          | 2157/225000 [10:07:56<730:46:25, 11.81s/it]
  1%|          | 2158/225000 [10:08:03<629:55:25, 10.18s/it]
  1%|          | 2159/225000 [10:08:18<719:14:05, 11.62s/it]
  1%|          | 2160/225000 [10:08:31<749:13:05, 12.10s/it]
  1%|          | 2161/225000 [10:08:51<903:46:51, 14.60s/it]
  1%|          | 2162/225000 [10:09:07<929:42:10, 15.02s/it]
  1%|          | 2163/225000 [10:09:29<1053:59:27, 17.03s/it]
  1%|          | 2164/225000 [10:09:40<952:09:37, 15.38s/it] 
  1%|          | 2165/225000 [10:09:57<966:33:29, 15.62s/it]
  1%|          | 2166/225000 [10:10:10<927:34:39, 14.99s/it]
  1%|          | 2167/225000 [10:10:20<823:43:09, 13.31s/it]
  1%|          | 2168/225000 [10:10:38<918:29:25, 14.84s/it]
  1%|          | 2169/225000 [10:10:55<957:43:57, 15.47s/it]
  1%|          | 2170/225000 [10:11:09<939:25:18, 15.18s/it]
  1%|          | 2171/225000 [10:11:26<967:46:20, 15.64s/it]
  1%|          | 2172/225000 [10:11:30<755:52:24, 12.21s/it]
  1%|          | 2173/225000 [10:11:46<825:06:11, 13.33s/it]
  1%|          | 2174/225000 [10:12:06<942:02:58, 15.22s/it]
  1%|          | 2175/225000 [10:12:22<953:08:53, 15.40s/it]
  1%|          | 2176/225000 [10:12:36<939:27:20, 15.18s/it]
  1%|          | 2177/225000 [10:12:56<1019:36:14, 16.47s/it]
  1%|          | 2178/225000 [10:13:10<979:48:39, 15.83s/it] 
  1%|          | 2179/225000 [10:13:26<971:56:21, 15.70s/it]
  1%|          | 2180/225000 [10:13:38<918:42:58, 14.84s/it]
  1%|          | 2181/225000 [10:13:49<830:51:35, 13.42s/it]
  1%|          | 2182/225000 [10:14:04<863:49:51, 13.96s/it]
  1%|          | 2183/225000 [10:14:13<785:44:54, 12.70s/it]
  1%|          | 2184/225000 [10:14:24<743:41:20, 12.02s/it]
  1%|          | 2185/225000 [10:14:35<731:44:08, 11.82s/it]
  1%|          | 2186/225000 [10:14:52<827:37:19, 13.37s/it]
  1%|          | 2187/225000 [10:15:01<733:02:40, 11.84s/it]
  1%|          | 2188/225000 [10:15:09<660:41:09, 10.67s/it]
  1%|          | 2189/225000 [10:15:30<859:17:45, 13.88s/it]
  1%|          | 2190/225000 [10:15:51<1000:48:27, 16.17s/it]
  1%|          | 2191/225000 [10:16:08<1006:46:37, 16.27s/it]
  1%|          | 2192/225000 [10:16:20<926:50:02, 14.98s/it] 
  1%|          | 2193/225000 [10:16:30<840:43:04, 13.58s/it]
  1%|          | 2194/225000 [10:16:48<917:43:09, 14.83s/it]
  1%|          | 2195/225000 [10:16:59<848:42:52, 13.71s/it]
  1%|          | 2196/225000 [10:17:10<801:05:44, 12.94s/it]
  1%|          | 2197/225000 [10:17:23<802:19:48, 12.96s/it]
  1%|          | 2198/225000 [10:17:44<949:42:04, 15.35s/it]
  1%|          | 2199/225000 [10:17:56<877:14:33, 14.17s/it]
  1%|          | 2200/225000 [10:18:15<978:58:10, 15.82s/it]
                                                            
{'loss': '1.029', 'grad_norm': '7.197', 'learning_rate': '1.098e-05', 'epoch': '0.009778', 'num_input_tokens_seen': 19294560, 'train_runtime': '3.71e+04', 'train_tokens_per_second': '520.1'}

  1%|          | 2200/225000 [10:18:15<978:58:10, 15.82s/it]
  1%|          | 2201/225000 [10:18:28<917:03:11, 14.82s/it]
  1%|          | 2202/225000 [10:18:34<758:52:21, 12.26s/it]
  1%|          | 2203/225000 [10:18:44<724:46:47, 11.71s/it]
  1%|          | 2204/225000 [10:19:02<828:33:44, 13.39s/it]
  1%|          | 2205/225000 [10:19:16<853:44:01, 13.79s/it]
  1%|          | 2206/225000 [10:19:24<737:48:02, 11.92s/it]
  1%|          | 2207/225000 [10:19:37<754:23:11, 12.19s/it]
  1%|          | 2208/225000 [10:20:01<973:02:21, 15.72s/it]
  1%|          | 2209/225000 [10:20:15<939:25:41, 15.18s/it]
  1%|          | 2210/225000 [10:20:33<1001:05:32, 16.18s/it]
  1%|          | 2211/225000 [10:20:44<904:17:36, 14.61s/it] 
  1%|          | 2212/225000 [10:20:58<881:15:56, 14.24s/it]
  1%|          | 2213/225000 [10:21:14<914:55:05, 14.78s/it]
  1%|          | 2214/225000 [10:21:30<945:09:13, 15.27s/it]
  1%|          | 2215/225000 [10:21:35<749:43:17, 12.11s/it]
  1%|          | 2216/225000 [10:21:51<831:22:36, 13.43s/it]
  1%|          | 2217/225000 [10:22:02<777:46:10, 12.57s/it]
  1%|          | 2218/225000 [10:22:23<946:48:45, 15.30s/it]
  1%|          | 2219/225000 [10:22:38<938:22:28, 15.16s/it]
  1%|          | 2220/225000 [10:22:55<958:12:11, 15.48s/it]
  1%|          | 2221/225000 [10:23:09<938:46:46, 15.17s/it]
  1%|          | 2222/225000 [10:23:23<917:24:06, 14.82s/it]
  1%|          | 2223/225000 [10:23:38<923:39:01, 14.93s/it]
  1%|          | 2224/225000 [10:23:51<883:45:25, 14.28s/it]
  1%|          | 2225/225000 [10:24:01<800:51:54, 12.94s/it]
  1%|          | 2226/225000 [10:24:09<706:02:22, 11.41s/it]
  1%|          | 2227/225000 [10:24:19<691:30:17, 11.17s/it]
  1%|          | 2228/225000 [10:24:33<749:10:18, 12.11s/it]
  1%|          | 2229/225000 [10:24:43<691:47:19, 11.18s/it]
  1%|          | 2230/225000 [10:24:56<729:04:10, 11.78s/it]
  1%|          | 2231/225000 [10:25:17<901:24:44, 14.57s/it]
  1%|          | 2232/225000 [10:25:32<918:40:27, 14.85s/it]
  1%|          | 2233/225000 [10:25:47<922:44:38, 14.91s/it]
  1%|          | 2234/225000 [10:25:59<864:53:21, 13.98s/it]
  1%|          | 2235/225000 [10:26:16<920:57:01, 14.88s/it]
  1%|          | 2236/225000 [10:26:29<877:02:54, 14.17s/it]
  1%|          | 2237/225000 [10:26:39<806:21:32, 13.03s/it]
  1%|          | 2238/225000 [10:26:49<757:35:14, 12.24s/it]
  1%|          | 2239/225000 [10:27:03<790:41:37, 12.78s/it]
  1%|          | 2240/225000 [10:27:19<836:27:37, 13.52s/it]
  1%|          | 2241/225000 [10:27:26<728:08:16, 11.77s/it]
  1%|          | 2242/225000 [10:27:42<791:50:49, 12.80s/it]
  1%|          | 2243/225000 [10:27:58<853:15:05, 13.79s/it]
  1%|          | 2244/225000 [10:28:08<794:13:21, 12.84s/it]
  1%|          | 2245/225000 [10:28:20<775:26:20, 12.53s/it]
  1%|          | 2246/225000 [10:28:39<885:44:26, 14.31s/it]
  1%|          | 2247/225000 [10:28:47<769:51:19, 12.44s/it]
  1%|          | 2248/225000 [10:28:56<718:02:55, 11.60s/it]
  1%|          | 2249/225000 [10:29:18<904:17:25, 14.61s/it]
  1%|          | 2250/225000 [10:29:35<950:14:08, 15.36s/it]
                                                            
{'loss': '1.034', 'grad_norm': '1.359', 'learning_rate': '1.123e-05', 'epoch': '0.01', 'num_input_tokens_seen': 19655448, 'train_runtime': '3.778e+04', 'train_tokens_per_second': '520.3'}

  1%|          | 2250/225000 [10:29:35<950:14:08, 15.36s/it]
  1%|          | 2251/225000 [10:29:52<986:22:34, 15.94s/it]
  1%|          | 2252/225000 [10:30:03<893:45:43, 14.44s/it]
  1%|          | 2253/225000 [10:30:17<884:26:22, 14.29s/it]
  1%|          | 2254/225000 [10:30:31<873:50:36, 14.12s/it]
  1%|          | 2255/225000 [10:30:44<848:31:13, 13.71s/it]
  1%|          | 2256/225000 [10:30:56<821:54:46, 13.28s/it]
  1%|          | 2257/225000 [10:31:12<878:20:35, 14.20s/it]
  1%|          | 2258/225000 [10:31:22<785:27:51, 12.69s/it]
  1%|          | 2259/225000 [10:31:33<756:05:58, 12.22s/it]
  1%|          | 2260/225000 [10:31:41<678:57:39, 10.97s/it]
  1%|          | 2261/225000 [10:31:52<687:55:49, 11.12s/it]
  1%|          | 2262/225000 [10:32:10<817:04:19, 13.21s/it]
  1%|          | 2263/225000 [10:32:20<751:08:40, 12.14s/it]
  1%|          | 2264/225000 [10:32:33<761:12:55, 12.30s/it]
  1%|          | 2265/225000 [10:32:45<772:00:40, 12.48s/it]
  1%|          | 2266/225000 [10:32:58<777:15:13, 12.56s/it]
  1%|          | 2267/225000 [10:33:13<809:44:52, 13.09s/it]
  1%|          | 2268/225000 [10:33:26<814:08:41, 13.16s/it]
  1%|          | 2269/225000 [10:33:45<934:42:47, 15.11s/it]
  1%|          | 2270/225000 [10:33:53<800:17:33, 12.94s/it]
  1%|          | 2271/225000 [10:34:06<792:54:53, 12.82s/it]
  1%|          | 2272/225000 [10:34:15<725:56:20, 11.73s/it]
  1%|          | 2273/225000 [10:34:37<906:35:25, 14.65s/it]
  1%|          | 2274/225000 [10:34:56<993:42:06, 16.06s/it]
  1%|          | 2275/225000 [10:35:04<839:49:12, 13.57s/it]
  1%|          | 2276/225000 [10:35:19<878:47:59, 14.20s/it]
  1%|          | 2277/225000 [10:35:30<813:29:24, 13.15s/it]
  1%|          | 2278/225000 [10:35:35<655:39:01, 10.60s/it]
  1%|          | 2279/225000 [10:35:45<644:19:17, 10.41s/it]
  1%|          | 2280/225000 [10:36:04<809:03:01, 13.08s/it]
  1%|          | 2281/225000 [10:36:16<793:33:40, 12.83s/it]
  1%|          | 2282/225000 [10:36:30<811:56:47, 13.12s/it]
  1%|          | 2283/225000 [10:36:43<817:11:28, 13.21s/it]
  1%|          | 2284/225000 [10:36:59<860:03:14, 13.90s/it]
  1%|          | 2285/225000 [10:37:12<837:51:22, 13.54s/it]
  1%|          | 2286/225000 [10:37:27<878:40:40, 14.20s/it]
  1%|          | 2287/225000 [10:37:49<1009:55:47, 16.32s/it]
  1%|          | 2288/225000 [10:37:59<894:03:58, 14.45s/it] 
  1%|          | 2289/225000 [10:38:18<976:00:54, 15.78s/it]
  1%|          | 2290/225000 [10:38:40<1101:53:43, 17.81s/it]
  1%|          | 2291/225000 [10:38:52<988:38:58, 15.98s/it] 
  1%|          | 2292/225000 [10:39:08<990:34:31, 16.01s/it]
  1%|          | 2293/225000 [10:39:18<881:29:56, 14.25s/it]
  1%|          | 2294/225000 [10:39:34<903:08:00, 14.60s/it]
  1%|          | 2295/225000 [10:39:46<871:40:38, 14.09s/it]
  1%|          | 2296/225000 [10:40:01<884:42:16, 14.30s/it]
  1%|          | 2297/225000 [10:40:08<752:30:12, 12.16s/it]
  1%|          | 2298/225000 [10:40:20<733:52:12, 11.86s/it]
  1%|          | 2299/225000 [10:40:39<873:06:04, 14.11s/it]
  1%|          | 2300/225000 [10:40:55<908:30:28, 14.69s/it]
                                                            
{'loss': '1.045', 'grad_norm': '1.923', 'learning_rate': '1.148e-05', 'epoch': '0.01022', 'num_input_tokens_seen': 20004048, 'train_runtime': '3.846e+04', 'train_tokens_per_second': '520.2'}

  1%|          | 2300/225000 [10:40:55<908:30:28, 14.69s/it]
  1%|          | 2301/225000 [10:41:06<842:59:45, 13.63s/it]
  1%|          | 2302/225000 [10:41:24<912:57:25, 14.76s/it]
  1%|          | 2303/225000 [10:41:35<858:08:15, 13.87s/it]
  1%|          | 2304/225000 [10:41:52<906:30:50, 14.65s/it]
  1%|          | 2305/225000 [10:42:04<862:07:49, 13.94s/it]
  1%|          | 2306/225000 [10:42:20<903:02:24, 14.60s/it]
  1%|          | 2307/225000 [10:42:37<940:04:30, 15.20s/it]
  1%|          | 2308/225000 [10:42:48<874:30:41, 14.14s/it]
  1%|          | 2309/225000 [10:43:05<925:24:31, 14.96s/it]
  1%|          | 2310/225000 [10:43:16<853:24:47, 13.80s/it]
  1%|          | 2311/225000 [10:43:33<906:30:56, 14.65s/it]
  1%|          | 2312/225000 [10:43:49<926:45:56, 14.98s/it]
  1%|          | 2313/225000 [10:44:03<920:37:08, 14.88s/it]
  1%|          | 2314/225000 [10:44:15<865:24:07, 13.99s/it]
  1%|          | 2315/225000 [10:44:30<873:57:52, 14.13s/it]
  1%|          | 2316/225000 [10:44:47<922:40:30, 14.92s/it]
  1%|          | 2317/225000 [10:44:58<856:01:00, 13.84s/it]
  1%|          | 2318/225000 [10:45:12<853:52:28, 13.80s/it]
  1%|          | 2319/225000 [10:45:29<910:44:03, 14.72s/it]
  1%|          | 2320/225000 [10:45:39<825:16:34, 13.34s/it]
  1%|          | 2321/225000 [10:45:50<780:18:06, 12.61s/it]
  1%|          | 2322/225000 [10:46:05<828:54:45, 13.40s/it]
  1%|          | 2323/225000 [10:46:27<988:02:56, 15.97s/it]
  1%|          | 2324/225000 [10:46:48<1082:13:17, 17.50s/it]
  1%|          | 2325/225000 [10:47:06<1102:53:20, 17.83s/it]
  1%|          | 2326/225000 [10:47:18<995:29:13, 16.09s/it] 
  1%|          | 2327/225000 [10:47:28<883:04:51, 14.28s/it]
  1%|          | 2328/225000 [10:47:42<872:32:23, 14.11s/it]
  1%|          | 2329/225000 [10:47:53<815:44:04, 13.19s/it]
  1%|          | 2330/225000 [10:48:12<920:28:47, 14.88s/it]
  1%|          | 2331/225000 [10:48:20<785:45:33, 12.70s/it]
  1%|          | 2332/225000 [10:48:38<898:15:12, 14.52s/it]
  1%|          | 2333/225000 [10:48:52<882:58:52, 14.28s/it]
  1%|          | 2334/225000 [10:49:06<866:43:57, 14.01s/it]
  1%|          | 2335/225000 [10:49:25<959:22:00, 15.51s/it]
  1%|          | 2336/225000 [10:49:47<1089:32:28, 17.62s/it]
  1%|          | 2337/225000 [10:50:05<1093:40:55, 17.68s/it]
  1%|          | 2338/225000 [10:50:12<897:14:33, 14.51s/it] 
  1%|          | 2339/225000 [10:50:28<916:15:00, 14.81s/it]
  1%|          | 2340/225000 [10:50:45<961:33:30, 15.55s/it]
  1%|          | 2341/225000 [10:51:07<1080:56:11, 17.48s/it]
  1%|          | 2342/225000 [10:51:21<1012:40:15, 16.37s/it]
  1%|          | 2343/225000 [10:51:34<957:02:56, 15.47s/it] 
  1%|          | 2344/225000 [10:51:47<902:54:06, 14.60s/it]
  1%|          | 2345/225000 [10:52:10<1067:22:44, 17.26s/it]
  1%|          | 2346/225000 [10:52:24<1006:03:11, 16.27s/it]
  1%|          | 2347/225000 [10:52:34<899:15:41, 14.54s/it] 
  1%|          | 2348/225000 [10:52:56<1027:14:43, 16.61s/it]
  1%|          | 2349/225000 [10:53:09<956:47:46, 15.47s/it] 
  1%|          | 2350/225000 [10:53:25<979:19:19, 15.83s/it]
                                                            
{'loss': '1.049', 'grad_norm': '1.957', 'learning_rate': '1.173e-05', 'epoch': '0.01044', 'num_input_tokens_seen': 20377048, 'train_runtime': '3.921e+04', 'train_tokens_per_second': '519.7'}

  1%|          | 2350/225000 [10:53:25<979:19:19, 15.83s/it]
  1%|          | 2351/225000 [10:53:33<833:57:04, 13.48s/it]
  1%|          | 2352/225000 [10:53:55<987:24:35, 15.97s/it]
  1%|          | 2353/225000 [10:54:08<921:54:49, 14.91s/it]
  1%|          | 2354/225000 [10:54:22<916:14:36, 14.81s/it]
  1%|          | 2355/225000 [10:54:30<777:25:06, 12.57s/it]
  1%|          | 2356/225000 [10:54:48<891:54:04, 14.42s/it]
  1%|          | 2357/225000 [10:55:06<955:52:50, 15.46s/it]
  1%|          | 2358/225000 [10:55:22<967:42:08, 15.65s/it]
  1%|          | 2359/225000 [10:55:35<912:01:03, 14.75s/it]
  1%|          | 2360/225000 [10:55:51<941:58:50, 15.23s/it]
  1%|          | 2361/225000 [10:56:03<879:13:21, 14.22s/it]
  1%|          | 2362/225000 [10:56:19<909:44:41, 14.71s/it]
  1%|          | 2363/225000 [10:56:37<963:31:36, 15.58s/it]
  1%|          | 2364/225000 [10:56:52<958:33:37, 15.50s/it]
  1%|          | 2365/225000 [10:57:07<945:30:10, 15.29s/it]
  1%|          | 2366/225000 [10:57:21<922:39:05, 14.92s/it]
  1%|          | 2367/225000 [10:57:42<1044:33:58, 16.89s/it]
  1%|          | 2368/225000 [10:57:49<854:01:06, 13.81s/it] 
  1%|          | 2369/225000 [10:58:01<825:07:54, 13.34s/it]
  1%|          | 2370/225000 [10:58:25<1027:20:14, 16.61s/it]
  1%|          | 2371/225000 [10:58:38<946:19:45, 15.30s/it] 
  1%|          | 2372/225000 [10:58:58<1046:13:21, 16.92s/it]
  1%|          | 2373/225000 [10:59:09<928:04:32, 15.01s/it] 
  1%|          | 2374/225000 [10:59:20<853:25:20, 13.80s/it]
  1%|          | 2375/225000 [10:59:32<818:36:20, 13.24s/it]
  1%|          | 2376/225000 [10:59:54<991:02:30, 16.03s/it]
  1%|          | 2377/225000 [11:00:11<1007:02:23, 16.28s/it]
  1%|          | 2378/225000 [11:00:23<919:01:40, 14.86s/it] 
  1%|          | 2379/225000 [11:00:42<997:19:21, 16.13s/it]
  1%|          | 2380/225000 [11:01:03<1085:50:36, 17.56s/it]
  1%|          | 2381/225000 [11:01:23<1131:30:50, 18.30s/it]
  1%|          | 2382/225000 [11:01:38<1079:06:14, 17.45s/it]
  1%|          | 2383/225000 [11:01:44<858:19:19, 13.88s/it] 
  1%|          | 2384/225000 [11:01:55<811:20:31, 13.12s/it]
  1%|          | 2385/225000 [11:02:02<693:01:40, 11.21s/it]
  1%|          | 2386/225000 [11:02:20<824:59:19, 13.34s/it]
  1%|          | 2387/225000 [11:02:33<809:10:18, 13.09s/it]
  1%|          | 2388/225000 [11:02:44<769:25:02, 12.44s/it]
  1%|          | 2389/225000 [11:03:03<902:11:36, 14.59s/it]
  1%|          | 2390/225000 [11:03:16<873:56:50, 14.13s/it]
  1%|          | 2391/225000 [11:03:24<763:00:31, 12.34s/it]
  1%|          | 2392/225000 [11:03:43<882:25:42, 14.27s/it]
  1%|          | 2393/225000 [11:03:56<862:34:19, 13.95s/it]
  1%|          | 2394/225000 [11:04:05<759:34:19, 12.28s/it]
  1%|          | 2395/225000 [11:04:14<697:24:21, 11.28s/it]
  1%|          | 2396/225000 [11:04:31<806:27:44, 13.04s/it]
  1%|          | 2397/225000 [11:04:38<704:01:46, 11.39s/it]
  1%|          | 2398/225000 [11:04:51<724:27:49, 11.72s/it]
  1%|          | 2399/225000 [11:05:02<721:16:10, 11.66s/it]
  1%|          | 2400/225000 [11:05:24<898:02:06, 14.52s/it]
                                                            
{'loss': '1.002', 'grad_norm': '1.792', 'learning_rate': '1.197e-05', 'epoch': '0.01067', 'num_input_tokens_seen': 20725112, 'train_runtime': '3.992e+04', 'train_tokens_per_second': '519.1'}

  1%|          | 2400/225000 [11:05:24<898:02:06, 14.52s/it]
  1%|          | 2401/225000 [11:05:37<873:38:35, 14.13s/it]
  1%|          | 2402/225000 [11:05:46<776:50:17, 12.56s/it]
  1%|          | 2403/225000 [11:06:00<816:16:26, 13.20s/it]
  1%|          | 2404/225000 [11:06:12<788:35:19, 12.75s/it]
  1%|          | 2405/225000 [11:06:30<877:26:21, 14.19s/it]
  1%|          | 2406/225000 [11:06:47<938:39:13, 15.18s/it]
  1%|          | 2407/225000 [11:06:58<850:40:31, 13.76s/it]
  1%|          | 2408/225000 [11:07:23<1070:07:19, 17.31s/it]
  1%|          | 2409/225000 [11:07:35<972:15:34, 15.72s/it] 
  1%|          | 2410/225000 [11:07:48<922:21:16, 14.92s/it]
  1%|          | 2411/225000 [11:08:02<902:58:17, 14.60s/it]
  1%|          | 2412/225000 [11:08:12<817:34:32, 13.22s/it]
  1%|          | 2413/225000 [11:08:18<676:54:52, 10.95s/it]
  1%|          | 2414/225000 [11:08:38<847:38:49, 13.71s/it]
  1%|          | 2415/225000 [11:09:01<1029:33:54, 16.65s/it]
  1%|          | 2416/225000 [11:09:10<883:38:31, 14.29s/it] 
  1%|          | 2417/225000 [11:09:29<970:49:52, 15.70s/it]
  1%|          | 2418/225000 [11:09:42<921:08:53, 14.90s/it]
  1%|          | 2419/225000 [11:10:00<966:12:36, 15.63s/it]
  1%|          | 2420/225000 [11:10:08<839:42:40, 13.58s/it]
  1%|          | 2421/225000 [11:10:18<766:40:19, 12.40s/it]
  1%|          | 2422/225000 [11:10:31<772:07:03, 12.49s/it]
  1%|          | 2423/225000 [11:10:41<724:24:19, 11.72s/it]
  1%|          | 2424/225000 [11:10:55<776:44:21, 12.56s/it]
  1%|          | 2425/225000 [11:11:02<674:36:56, 10.91s/it]
  1%|          | 2426/225000 [11:11:11<637:01:07, 10.30s/it]
  1%|          | 2427/225000 [11:11:27<733:04:30, 11.86s/it]
  1%|          | 2428/225000 [11:11:34<657:03:20, 10.63s/it]
  1%|          | 2429/225000 [11:11:59<913:28:20, 14.78s/it]
  1%|          | 2430/225000 [11:12:12<889:53:35, 14.39s/it]
  1%|          | 2431/225000 [11:12:27<893:25:02, 14.45s/it]
  1%|          | 2432/225000 [11:12:40<877:14:03, 14.19s/it]
  1%|          | 2433/225000 [11:12:47<740:08:48, 11.97s/it]
  1%|          | 2434/225000 [11:12:56<689:08:06, 11.15s/it]
  1%|          | 2435/225000 [11:13:17<869:43:53, 14.07s/it]
  1%|          | 2436/225000 [11:13:28<804:38:17, 13.02s/it]
  1%|          | 2437/225000 [11:13:41<812:03:12, 13.14s/it]
  1%|          | 2438/225000 [11:13:48<694:43:49, 11.24s/it]
  1%|          | 2439/225000 [11:13:57<651:40:54, 10.54s/it]
  1%|          | 2440/225000 [11:14:15<784:03:23, 12.68s/it]
  1%|          | 2441/225000 [11:14:25<736:42:35, 11.92s/it]
  1%|          | 2442/225000 [11:14:40<793:45:53, 12.84s/it]
  1%|          | 2443/225000 [11:14:51<755:40:11, 12.22s/it]
  1%|          | 2444/225000 [11:15:05<789:50:57, 12.78s/it]
  1%|          | 2445/225000 [11:15:21<850:55:19, 13.76s/it]
  1%|          | 2446/225000 [11:15:41<976:26:10, 15.79s/it]
  1%|          | 2447/225000 [11:15:56<947:48:20, 15.33s/it]
  1%|          | 2448/225000 [11:16:10<931:19:07, 15.07s/it]
  1%|          | 2449/225000 [11:16:25<937:10:08, 15.16s/it]
  1%|          | 2450/225000 [11:16:38<890:32:59, 14.41s/it]
                                                            
{'loss': '1.006', 'grad_norm': '1.932', 'learning_rate': '1.223e-05', 'epoch': '0.01089', 'num_input_tokens_seen': 21072288, 'train_runtime': '4.06e+04', 'train_tokens_per_second': '519'}

  1%|          | 2450/225000 [11:16:38<890:32:59, 14.41s/it]
  1%|          | 2451/225000 [11:16:56<949:19:27, 15.36s/it]
  1%|          | 2452/225000 [11:17:05<842:47:40, 13.63s/it]
  1%|          | 2453/225000 [11:17:19<844:26:49, 13.66s/it]
  1%|          | 2454/225000 [11:17:33<859:10:09, 13.90s/it]
  1%|          | 2455/225000 [11:17:52<940:05:45, 15.21s/it]
  1%|          | 2456/225000 [11:18:04<894:04:17, 14.46s/it]
  1%|          | 2457/225000 [11:18:17<868:35:46, 14.05s/it]
  1%|          | 2458/225000 [11:18:41<1049:26:50, 16.98s/it]
  1%|          | 2459/225000 [11:19:01<1101:29:52, 17.82s/it]
  1%|          | 2460/225000 [11:19:14<1017:08:06, 16.45s/it]
  1%|          | 2461/225000 [11:19:25<907:33:48, 14.68s/it] 
  1%|          | 2462/225000 [11:19:38<877:34:50, 14.20s/it]
  1%|          | 2463/225000 [11:20:00<1021:03:59, 16.52s/it]
  1%|          | 2464/225000 [11:20:10<902:29:43, 14.60s/it] 
  1%|          | 2465/225000 [11:20:24<886:18:21, 14.34s/it]
  1%|          | 2466/225000 [11:20:41<937:37:39, 15.17s/it]
  1%|          | 2467/225000 [11:20:52<863:07:09, 13.96s/it]
  1%|          | 2468/225000 [11:21:09<924:18:53, 14.95s/it]
  1%|          | 2469/225000 [11:21:23<898:04:06, 14.53s/it]
  1%|          | 2470/225000 [11:21:39<931:38:47, 15.07s/it]
  1%|          | 2471/225000 [11:21:49<835:53:16, 13.52s/it]
  1%|          | 2472/225000 [11:21:57<731:16:14, 11.83s/it]
  1%|          | 2473/225000 [11:22:20<945:55:46, 15.30s/it]
  1%|          | 2474/225000 [11:22:38<998:48:37, 16.16s/it]
  1%|          | 2475/225000 [11:22:53<963:12:31, 15.58s/it]
  1%|          | 2476/225000 [11:23:03<866:55:23, 14.03s/it]
  1%|          | 2477/225000 [11:23:16<848:49:18, 13.73s/it]
  1%|          | 2478/225000 [11:23:30<858:42:51, 13.89s/it]
  1%|          | 2479/225000 [11:23:46<882:28:53, 14.28s/it]
  1%|          | 2480/225000 [11:23:59<870:42:38, 14.09s/it]
  1%|          | 2481/225000 [11:24:10<804:03:58, 13.01s/it]
  1%|          | 2482/225000 [11:24:21<775:39:42, 12.55s/it]
  1%|          | 2483/225000 [11:24:40<887:19:57, 14.36s/it]
  1%|          | 2484/225000 [11:24:52<847:09:32, 13.71s/it]
  1%|          | 2485/225000 [11:25:12<973:32:09, 15.75s/it]
  1%|          | 2486/225000 [11:25:21<845:38:13, 13.68s/it]
  1%|          | 2487/225000 [11:25:31<763:43:43, 12.36s/it]
  1%|          | 2488/225000 [11:25:54<960:42:00, 15.54s/it]
  1%|          | 2489/225000 [11:26:01<815:14:58, 13.19s/it]
  1%|          | 2490/225000 [11:26:08<700:14:30, 11.33s/it]
  1%|          | 2491/225000 [11:26:26<826:54:49, 13.38s/it]
  1%|          | 2492/225000 [11:26:42<868:34:19, 14.05s/it]
  1%|          | 2493/225000 [11:26:56<864:51:55, 13.99s/it]
  1%|          | 2494/225000 [11:27:11<877:21:46, 14.20s/it]
  1%|          | 2495/225000 [11:27:27<919:22:34, 14.87s/it]
  1%|          | 2496/225000 [11:27:44<965:52:38, 15.63s/it]
  1%|          | 2497/225000 [11:27:58<930:29:02, 15.05s/it]
  1%|          | 2498/225000 [11:28:12<901:01:50, 14.58s/it]
  1%|          | 2499/225000 [11:28:25<874:12:01, 14.14s/it]
  1%|          | 2500/225000 [11:28:44<962:48:27, 15.58s/it]
                                                            
{'loss': '0.9727', 'grad_norm': '1.803', 'learning_rate': '1.247e-05', 'epoch': '0.01111', 'num_input_tokens_seen': 21420104, 'train_runtime': '4.132e+04', 'train_tokens_per_second': '518.3'}

  1%|          | 2500/225000 [11:28:44<962:48:27, 15.58s/it]
  1%|          | 2501/225000 [11:28:50<795:22:59, 12.87s/it]
  1%|          | 2502/225000 [11:29:03<801:09:50, 12.96s/it]
  1%|          | 2503/225000 [11:29:15<784:00:44, 12.69s/it]
  1%|          | 2504/225000 [11:29:25<726:16:30, 11.75s/it]
  1%|          | 2505/225000 [11:29:42<830:10:49, 13.43s/it]
  1%|          | 2506/225000 [11:29:54<791:27:59, 12.81s/it]
  1%|          | 2507/225000 [11:30:06<779:02:55, 12.61s/it]
  1%|          | 2508/225000 [11:30:22<853:06:07, 13.80s/it]
  1%|          | 2509/225000 [11:30:34<819:29:32, 13.26s/it]
  1%|          | 2510/225000 [11:30:50<860:59:17, 13.93s/it]
  1%|          | 2511/225000 [11:31:03<845:54:29, 13.69s/it]
  1%|          | 2512/225000 [11:31:19<894:41:01, 14.48s/it]
  1%|          | 2513/225000 [11:31:35<909:37:51, 14.72s/it]
  1%|          | 2514/225000 [11:31:47<857:14:08, 13.87s/it]
  1%|          | 2515/225000 [11:32:00<856:01:17, 13.85s/it]
  1%|          | 2516/225000 [11:32:14<858:14:07, 13.89s/it]
  1%|          | 2517/225000 [11:32:28<850:07:49, 13.76s/it]
  1%|          | 2518/225000 [11:32:42<858:19:20, 13.89s/it]
  1%|          | 2519/225000 [11:32:50<749:20:28, 12.13s/it]
  1%|          | 2520/225000 [11:32:56<629:22:08, 10.18s/it]
  1%|          | 2521/225000 [11:33:05<623:21:53, 10.09s/it]
  1%|          | 2522/225000 [11:33:21<729:35:53, 11.81s/it]
  1%|          | 2523/225000 [11:33:27<618:38:53, 10.01s/it]
  1%|          | 2524/225000 [11:33:46<780:01:26, 12.62s/it]
  1%|          | 2525/225000 [11:33:59<798:31:26, 12.92s/it]
  1%|          | 2526/225000 [11:34:04<651:03:21, 10.54s/it]
  1%|          | 2527/225000 [11:34:30<925:58:52, 14.98s/it]
  1%|          | 2528/225000 [11:34:42<879:55:16, 14.24s/it]
  1%|          | 2529/225000 [11:34:56<865:00:26, 14.00s/it]
  1%|          | 2530/225000 [11:35:11<883:42:29, 14.30s/it]
  1%|          | 2531/225000 [11:35:20<793:49:19, 12.85s/it]
  1%|          | 2532/225000 [11:35:32<777:01:17, 12.57s/it]
  1%|          | 2533/225000 [11:35:44<762:12:32, 12.33s/it]
  1%|          | 2534/225000 [11:35:55<742:04:15, 12.01s/it]
  1%|          | 2535/225000 [11:35:59<600:12:52,  9.71s/it]
  1%|          | 2536/225000 [11:36:21<818:15:27, 13.24s/it]
  1%|          | 2537/225000 [11:36:32<780:41:27, 12.63s/it]
  1%|          | 2538/225000 [11:36:43<754:23:42, 12.21s/it]
  1%|          | 2539/225000 [11:36:54<730:53:54, 11.83s/it]
  1%|          | 2540/225000 [11:37:03<673:11:45, 10.89s/it]
  1%|          | 2541/225000 [11:37:23<841:39:17, 13.62s/it]
  1%|          | 2542/225000 [11:37:38<871:48:10, 14.11s/it]
  1%|          | 2543/225000 [11:37:56<938:10:23, 15.18s/it]
  1%|          | 2544/225000 [11:38:03<785:14:10, 12.71s/it]
  1%|          | 2545/225000 [11:38:14<762:57:11, 12.35s/it]
  1%|          | 2546/225000 [11:38:23<695:36:56, 11.26s/it]
  1%|          | 2547/225000 [11:38:39<784:42:09, 12.70s/it]
  1%|          | 2548/225000 [11:38:57<870:42:44, 14.09s/it]
  1%|          | 2549/225000 [11:39:08<823:13:03, 13.32s/it]
  1%|          | 2550/225000 [11:39:22<839:17:03, 13.58s/it]
                                                            
{'loss': '0.965', 'grad_norm': '1.112', 'learning_rate': '1.273e-05', 'epoch': '0.01133', 'num_input_tokens_seen': 21738600, 'train_runtime': '4.196e+04', 'train_tokens_per_second': '518'}

  1%|          | 2550/225000 [11:39:22<839:17:03, 13.58s/it]
  1%|          | 2551/225000 [11:39:37<869:17:29, 14.07s/it]
  1%|          | 2552/225000 [11:39:53<890:55:43, 14.42s/it]
  1%|          | 2553/225000 [11:40:16<1050:54:03, 17.01s/it]
  1%|          | 2554/225000 [11:40:25<909:10:20, 14.71s/it] 
  1%|          | 2555/225000 [11:40:39<902:22:45, 14.60s/it]
  1%|          | 2556/225000 [11:40:54<907:18:10, 14.68s/it]
  1%|          | 2557/225000 [11:41:07<874:00:35, 14.14s/it]
  1%|          | 2558/225000 [11:41:23<897:55:47, 14.53s/it]
  1%|          | 2559/225000 [11:41:40<945:25:22, 15.30s/it]
  1%|          | 2560/225000 [11:41:51<879:01:20, 14.23s/it]
  1%|          | 2561/225000 [11:42:10<950:47:27, 15.39s/it]
  1%|          | 2562/225000 [11:42:24<930:36:34, 15.06s/it]
  1%|          | 2563/225000 [11:42:33<825:00:59, 13.35s/it]
  1%|          | 2564/225000 [11:42:47<830:53:27, 13.45s/it]
  1%|          | 2565/225000 [11:42:55<729:04:02, 11.80s/it]
  1%|          | 2566/225000 [11:43:16<901:41:45, 14.59s/it]
  1%|          | 2567/225000 [11:43:23<758:16:15, 12.27s/it]
  1%|          | 2568/225000 [11:43:45<933:43:34, 15.11s/it]
  1%|          | 2569/225000 [11:44:04<1009:54:44, 16.35s/it]
  1%|          | 2570/225000 [11:44:13<872:10:49, 14.12s/it] 
  1%|          | 2571/225000 [11:44:19<736:28:18, 11.92s/it]
  1%|          | 2572/225000 [11:44:32<738:40:50, 11.96s/it]
  1%|          | 2573/225000 [11:44:38<640:51:32, 10.37s/it]
  1%|          | 2574/225000 [11:44:53<724:55:01, 11.73s/it]
  1%|          | 2575/225000 [11:45:04<709:15:44, 11.48s/it]
  1%|          | 2576/225000 [11:45:17<732:58:20, 11.86s/it]
  1%|          | 2577/225000 [11:45:28<716:10:28, 11.59s/it]
  1%|          | 2578/225000 [11:45:38<687:16:56, 11.12s/it]
  1%|          | 2579/225000 [11:45:51<733:44:31, 11.88s/it]
  1%|          | 2580/225000 [11:46:00<679:19:44, 11.00s/it]
  1%|          | 2581/225000 [11:46:15<739:12:37, 11.96s/it]
  1%|          | 2582/225000 [11:46:26<725:05:41, 11.74s/it]
  1%|          | 2583/225000 [11:46:34<661:15:12, 10.70s/it]
  1%|          | 2584/225000 [11:46:50<756:14:31, 12.24s/it]
  1%|          | 2585/225000 [11:47:01<732:35:14, 11.86s/it]
  1%|          | 2586/225000 [11:47:12<715:46:28, 11.59s/it]
  1%|          | 2587/225000 [11:47:28<800:05:50, 12.95s/it]
  1%|          | 2588/225000 [11:47:34<678:03:19, 10.98s/it]
  1%|          | 2589/225000 [11:47:52<801:47:15, 12.98s/it]
  1%|          | 2590/225000 [11:48:11<921:14:00, 14.91s/it]
  1%|          | 2591/225000 [11:48:23<868:00:29, 14.05s/it]
  1%|          | 2592/225000 [11:48:45<1015:16:57, 16.43s/it]
  1%|          | 2593/225000 [11:48:54<870:52:03, 14.10s/it] 
  1%|          | 2594/225000 [11:49:05<822:11:44, 13.31s/it]
  1%|          | 2595/225000 [11:49:15<760:42:52, 12.31s/it]
  1%|          | 2596/225000 [11:49:24<693:26:18, 11.22s/it]
  1%|          | 2597/225000 [11:49:40<776:32:08, 12.57s/it]
  1%|          | 2598/225000 [11:49:52<776:54:58, 12.58s/it]
  1%|          | 2599/225000 [11:50:01<696:19:07, 11.27s/it]
  1%|          | 2600/225000 [11:50:20<838:39:05, 13.58s/it]
                                                            
{'loss': '1.015', 'grad_norm': '1.54', 'learning_rate': '1.298e-05', 'epoch': '0.01156', 'num_input_tokens_seen': 22071984, 'train_runtime': '4.262e+04', 'train_tokens_per_second': '517.9'}

  1%|          | 2600/225000 [11:50:20<838:39:05, 13.58s/it]
  1%|          | 2601/225000 [11:50:39<940:15:03, 15.22s/it]
  1%|          | 2602/225000 [11:50:50<871:00:18, 14.10s/it]
  1%|          | 2603/225000 [11:50:59<778:36:34, 12.60s/it]
  1%|          | 2604/225000 [11:51:09<727:50:04, 11.78s/it]
  1%|          | 2605/225000 [11:51:24<784:34:57, 12.70s/it]
  1%|          | 2606/225000 [11:51:38<806:40:10, 13.06s/it]
  1%|          | 2607/225000 [11:51:54<853:55:38, 13.82s/it]
  1%|          | 2608/225000 [11:52:01<743:29:07, 12.04s/it]
  1%|          | 2609/225000 [11:52:17<807:39:07, 13.07s/it]
  1%|          | 2610/225000 [11:52:25<719:41:58, 11.65s/it]
  1%|          | 2611/225000 [11:52:42<812:02:39, 13.15s/it]
  1%|          | 2612/225000 [11:52:59<880:17:15, 14.25s/it]
  1%|          | 2613/225000 [11:53:06<757:25:51, 12.26s/it]
  1%|          | 2614/225000 [11:53:23<833:23:16, 13.49s/it]
  1%|          | 2615/225000 [11:53:38<870:15:16, 14.09s/it]
  1%|          | 2616/225000 [11:53:51<842:56:54, 13.65s/it]
  1%|          | 2617/225000 [11:54:04<827:06:22, 13.39s/it]
  1%|          | 2618/225000 [11:54:13<759:07:30, 12.29s/it]
  1%|          | 2619/225000 [11:54:34<923:45:06, 14.95s/it]
  1%|          | 2620/225000 [11:54:46<857:54:28, 13.89s/it]
  1%|          | 2621/225000 [11:55:00<862:01:23, 13.95s/it]
  1%|          | 2622/225000 [11:55:09<769:04:31, 12.45s/it]
  1%|          | 2623/225000 [11:55:25<835:40:00, 13.53s/it]
  1%|          | 2624/225000 [11:55:41<887:48:37, 14.37s/it]
  1%|          | 2625/225000 [11:56:01<985:46:25, 15.96s/it]
  1%|          | 2626/225000 [11:56:18<1008:50:38, 16.33s/it]
  1%|          | 2627/225000 [11:56:35<1013:46:52, 16.41s/it]
  1%|          | 2628/225000 [11:56:42<853:08:23, 13.81s/it] 
  1%|          | 2629/225000 [11:57:01<934:36:35, 15.13s/it]
  1%|          | 2630/225000 [11:57:11<841:04:39, 13.62s/it]
  1%|          | 2631/225000 [11:57:29<919:32:18, 14.89s/it]
  1%|          | 2632/225000 [11:57:41<868:09:02, 14.05s/it]
  1%|          | 2633/225000 [11:57:49<764:52:52, 12.38s/it]
  1%|          | 2634/225000 [11:58:00<743:43:07, 12.04s/it]
  1%|          | 2635/225000 [11:58:06<628:37:13, 10.18s/it]
  1%|          | 2636/225000 [11:58:22<723:40:54, 11.72s/it]
  1%|          | 2637/225000 [11:58:37<798:27:51, 12.93s/it]
  1%|          | 2638/225000 [11:58:44<689:16:57, 11.16s/it]
  1%|          | 2639/225000 [11:58:56<689:03:04, 11.16s/it]
  1%|          | 2640/225000 [11:59:07<690:12:44, 11.17s/it]
  1%|          | 2641/225000 [11:59:33<975:32:41, 15.79s/it]
  1%|          | 2642/225000 [11:59:47<937:05:27, 15.17s/it]
  1%|          | 2643/225000 [12:00:06<1003:46:49, 16.25s/it]
  1%|          | 2644/225000 [12:00:26<1073:49:45, 17.39s/it]
  1%|          | 2645/225000 [12:00:44<1089:05:39, 17.63s/it]
  1%|          | 2646/225000 [12:01:05<1158:56:33, 18.76s/it]
  1%|          | 2647/225000 [12:01:20<1079:43:12, 17.48s/it]
  1%|          | 2648/225000 [12:01:32<974:44:09, 15.78s/it] 
  1%|          | 2649/225000 [12:01:48<983:08:09, 15.92s/it]
  1%|          | 2650/225000 [12:01:58<874:36:55, 14.16s/it]
                                                            
{'loss': '0.9642', 'grad_norm': '1.176', 'learning_rate': '1.323e-05', 'epoch': '0.01178', 'num_input_tokens_seen': 22414856, 'train_runtime': '4.332e+04', 'train_tokens_per_second': '517.4'}

  1%|          | 2650/225000 [12:01:58<874:36:55, 14.16s/it]
  1%|          | 2651/225000 [12:02:20<1026:18:36, 16.62s/it]
  1%|          | 2652/225000 [12:02:32<940:05:56, 15.22s/it] 
  1%|          | 2653/225000 [12:02:45<899:43:19, 14.57s/it]
  1%|          | 2654/225000 [12:02:59<875:52:28, 14.18s/it]
  1%|          | 2655/225000 [12:03:13<874:09:31, 14.15s/it]
  1%|          | 2656/225000 [12:03:24<824:29:09, 13.35s/it]
  1%|          | 2657/225000 [12:03:41<886:03:19, 14.35s/it]
  1%|          | 2658/225000 [12:03:51<802:48:59, 13.00s/it]
  1%|          | 2659/225000 [12:04:03<792:35:42, 12.83s/it]
  1%|          | 2660/225000 [12:04:10<684:32:58, 11.08s/it]
  1%|          | 2661/225000 [12:04:25<744:10:18, 12.05s/it]
  1%|          | 2662/225000 [12:04:38<764:02:39, 12.37s/it]
  1%|          | 2663/225000 [12:04:50<764:31:26, 12.38s/it]
  1%|          | 2664/225000 [12:05:00<728:45:54, 11.80s/it]
  1%|          | 2665/225000 [12:05:11<708:22:01, 11.47s/it]
  1%|          | 2666/225000 [12:05:27<797:50:35, 12.92s/it]
  1%|          | 2667/225000 [12:05:45<880:54:43, 14.26s/it]
  1%|          | 2668/225000 [12:06:00<892:22:28, 14.45s/it]
  1%|          | 2669/225000 [12:06:11<824:16:41, 13.35s/it]
  1%|          | 2670/225000 [12:06:19<739:01:46, 11.97s/it]
  1%|          | 2671/225000 [12:06:39<891:49:33, 14.44s/it]
  1%|          | 2672/225000 [12:07:03<1058:27:19, 17.14s/it]
  1%|          | 2673/225000 [12:07:13<925:50:45, 14.99s/it] 
  1%|          | 2674/225000 [12:07:29<954:37:29, 15.46s/it]
  1%|          | 2675/225000 [12:07:40<854:21:37, 13.83s/it]
  1%|          | 2676/225000 [12:07:53<846:27:49, 13.71s/it]
  1%|          | 2677/225000 [12:08:11<920:08:09, 14.90s/it]
  1%|          | 2678/225000 [12:08:19<795:17:06, 12.88s/it]
  1%|          | 2679/225000 [12:08:32<800:18:05, 12.96s/it]
  1%|          | 2680/225000 [12:08:41<731:47:42, 11.85s/it]
  1%|          | 2681/225000 [12:08:58<824:48:33, 13.36s/it]
  1%|          | 2682/225000 [12:09:09<776:12:57, 12.57s/it]
  1%|          | 2683/225000 [12:09:22<790:03:20, 12.79s/it]
  1%|          | 2684/225000 [12:09:40<892:41:26, 14.46s/it]
  1%|          | 2685/225000 [12:09:54<867:36:59, 14.05s/it]
  1%|          | 2686/225000 [12:10:04<794:08:13, 12.86s/it]
  1%|          | 2687/225000 [12:10:17<810:31:12, 13.13s/it]
  1%|          | 2688/225000 [12:10:33<861:32:00, 13.95s/it]
  1%|          | 2689/225000 [12:10:49<901:53:55, 14.60s/it]
  1%|          | 2690/225000 [12:10:55<744:05:04, 12.05s/it]
  1%|          | 2691/225000 [12:11:12<820:15:40, 13.28s/it]
  1%|          | 2692/225000 [12:11:17<666:39:46, 10.80s/it]
  1%|          | 2693/225000 [12:11:29<693:24:54, 11.23s/it]
  1%|          | 2694/225000 [12:11:44<769:23:43, 12.46s/it]
  1%|          | 2695/225000 [12:11:57<770:38:06, 12.48s/it]
  1%|          | 2696/225000 [12:12:04<669:59:58, 10.85s/it]
  1%|          | 2697/225000 [12:12:16<702:12:55, 11.37s/it]
  1%|          | 2698/225000 [12:12:39<909:35:22, 14.73s/it]
  1%|          | 2699/225000 [12:12:54<911:48:43, 14.77s/it]
  1%|          | 2700/225000 [12:13:06<874:06:18, 14.16s/it]
                                                            
{'loss': '0.9243', 'grad_norm': '1.841', 'learning_rate': '1.348e-05', 'epoch': '0.012', 'num_input_tokens_seen': 22746312, 'train_runtime': '4.399e+04', 'train_tokens_per_second': '517.1'}

  1%|          | 2700/225000 [12:13:06<874:06:18, 14.16s/it]
  1%|          | 2701/225000 [12:13:17<811:57:36, 13.15s/it]
  1%|          | 2702/225000 [12:13:35<904:18:21, 14.64s/it]
  1%|          | 2703/225000 [12:13:50<906:08:50, 14.67s/it]
  1%|          | 2704/225000 [12:13:59<789:12:28, 12.78s/it]
  1%|          | 2705/225000 [12:14:13<817:25:39, 13.24s/it]
  1%|          | 2706/225000 [12:14:24<781:04:31, 12.65s/it]
  1%|          | 2707/225000 [12:14:37<785:46:22, 12.73s/it]
  1%|          | 2708/225000 [12:14:50<795:06:48, 12.88s/it]
  1%|          | 2709/225000 [12:15:02<769:26:56, 12.46s/it]
  1%|          | 2710/225000 [12:15:13<756:06:51, 12.25s/it]
  1%|          | 2711/225000 [12:15:22<695:22:28, 11.26s/it]
  1%|          | 2712/225000 [12:15:33<682:44:35, 11.06s/it]
  1%|          | 2713/225000 [12:15:46<727:27:25, 11.78s/it]
  1%|          | 2714/225000 [12:16:05<844:16:06, 13.67s/it]
  1%|          | 2715/225000 [12:16:21<896:47:43, 14.52s/it]
  1%|          | 2716/225000 [12:16:33<847:26:38, 13.72s/it]
  1%|          | 2717/225000 [12:16:46<837:21:57, 13.56s/it]
  1%|          | 2718/225000 [12:17:02<881:54:29, 14.28s/it]
  1%|          | 2719/225000 [12:17:17<894:27:26, 14.49s/it]
  1%|          | 2720/225000 [12:17:33<927:27:09, 15.02s/it]
  1%|          | 2721/225000 [12:17:47<895:19:00, 14.50s/it]
  1%|          | 2722/225000 [12:17:58<830:44:34, 13.45s/it]
  1%|          | 2723/225000 [12:18:18<955:29:04, 15.48s/it]
  1%|          | 2724/225000 [12:18:29<875:05:05, 14.17s/it]
  1%|          | 2725/225000 [12:18:43<864:30:41, 14.00s/it]
  1%|          | 2726/225000 [12:18:52<781:12:34, 12.65s/it]
  1%|          | 2727/225000 [12:19:06<810:04:48, 13.12s/it]
  1%|          | 2728/225000 [12:19:22<859:46:29, 13.93s/it]
  1%|          | 2729/225000 [12:19:32<776:33:06, 12.58s/it]
  1%|          | 2730/225000 [12:19:50<879:29:59, 14.24s/it]
  1%|          | 2731/225000 [12:19:59<782:21:02, 12.67s/it]
  1%|          | 2732/225000 [12:20:15<846:18:59, 13.71s/it]
  1%|          | 2733/225000 [12:20:29<858:50:48, 13.91s/it]
  1%|          | 2734/225000 [12:20:46<919:31:24, 14.89s/it]
  1%|          | 2735/225000 [12:21:03<944:34:53, 15.30s/it]
  1%|          | 2736/225000 [12:21:14<866:32:38, 14.04s/it]
  1%|          | 2737/225000 [12:21:22<752:16:00, 12.18s/it]
  1%|          | 2738/225000 [12:21:38<823:55:40, 13.35s/it]
  1%|          | 2739/225000 [12:21:50<797:30:15, 12.92s/it]
  1%|          | 2740/225000 [12:22:00<760:04:50, 12.31s/it]
  1%|          | 2741/225000 [12:22:07<650:48:11, 10.54s/it]
  1%|          | 2742/225000 [12:22:26<805:45:19, 13.05s/it]
  1%|          | 2743/225000 [12:22:37<772:00:11, 12.50s/it]
  1%|          | 2744/225000 [12:22:49<761:11:21, 12.33s/it]
  1%|          | 2745/225000 [12:22:57<682:04:30, 11.05s/it]
  1%|          | 2746/225000 [12:23:14<795:49:55, 12.89s/it]
  1%|          | 2747/225000 [12:23:28<820:52:58, 13.30s/it]
  1%|          | 2748/225000 [12:23:40<786:48:24, 12.74s/it]
  1%|          | 2749/225000 [12:23:57<862:21:03, 13.97s/it]
  1%|          | 2750/225000 [12:24:04<742:22:30, 12.02s/it]
                                                            
{'loss': '0.9918', 'grad_norm': '2.714', 'learning_rate': '1.373e-05', 'epoch': '0.01222', 'num_input_tokens_seen': 23075384, 'train_runtime': '4.464e+04', 'train_tokens_per_second': '516.9'}

  1%|          | 2750/225000 [12:24:04<742:22:30, 12.02s/it]
  1%|          | 2751/225000 [12:24:18<770:43:25, 12.48s/it]
  1%|          | 2752/225000 [12:24:29<745:52:23, 12.08s/it]
  1%|          | 2753/225000 [12:24:42<757:48:45, 12.28s/it]
  1%|          | 2754/225000 [12:25:01<898:44:02, 14.56s/it]
  1%|          | 2755/225000 [12:25:13<834:16:03, 13.51s/it]
  1%|          | 2756/225000 [12:25:23<785:04:07, 12.72s/it]
  1%|          | 2757/225000 [12:25:45<958:27:54, 15.53s/it]
  1%|          | 2758/225000 [12:25:56<866:45:51, 14.04s/it]
  1%|          | 2759/225000 [12:26:04<746:26:40, 12.09s/it]
  1%|          | 2760/225000 [12:26:15<742:51:04, 12.03s/it]
  1%|          | 2761/225000 [12:26:39<954:43:04, 15.47s/it]
  1%|          | 2762/225000 [12:26:53<928:32:12, 15.04s/it]
  1%|          | 2763/225000 [12:27:09<937:35:02, 15.19s/it]
  1%|          | 2764/225000 [12:27:28<1020:56:27, 16.54s/it]
  1%|          | 2765/225000 [12:27:46<1036:17:54, 16.79s/it]
  1%|          | 2766/225000 [12:28:04<1057:57:33, 17.14s/it]
  1%|          | 2767/225000 [12:28:18<1010:46:44, 16.37s/it]
  1%|          | 2768/225000 [12:28:34<1001:58:28, 16.23s/it]
  1%|          | 2769/225000 [12:28:45<904:30:42, 14.65s/it] 
  1%|          | 2770/225000 [12:28:58<882:24:42, 14.29s/it]
  1%|          | 2771/225000 [12:29:08<802:17:16, 13.00s/it]
  1%|          | 2772/225000 [12:29:16<709:59:14, 11.50s/it]
  1%|          | 2773/225000 [12:29:34<823:06:22, 13.33s/it]
  1%|          | 2774/225000 [12:29:46<793:15:34, 12.85s/it]
  1%|          | 2775/225000 [12:30:02<854:44:45, 13.85s/it]
  1%|          | 2776/225000 [12:30:22<974:10:36, 15.78s/it]
  1%|          | 2777/225000 [12:30:37<950:56:26, 15.41s/it]
  1%|          | 2778/225000 [12:30:47<847:53:01, 13.74s/it]
  1%|          | 2779/225000 [12:31:02<875:12:22, 14.18s/it]
  1%|          | 2780/225000 [12:31:16<869:33:25, 14.09s/it]
  1%|          | 2781/225000 [12:31:22<717:55:05, 11.63s/it]
  1%|          | 2782/225000 [12:31:37<793:51:36, 12.86s/it]
  1%|          | 2783/225000 [12:31:49<779:08:08, 12.62s/it]
  1%|          | 2784/225000 [12:32:03<802:23:21, 13.00s/it]
  1%|          | 2785/225000 [12:32:19<844:21:51, 13.68s/it]
  1%|          | 2786/225000 [12:32:32<832:24:21, 13.49s/it]
  1%|          | 2787/225000 [12:32:45<826:21:17, 13.39s/it]
  1%|          | 2788/225000 [12:32:55<773:35:38, 12.53s/it]
  1%|          | 2789/225000 [12:33:07<751:43:34, 12.18s/it]
  1%|          | 2790/225000 [12:33:15<682:05:19, 11.05s/it]
  1%|          | 2791/225000 [12:33:28<710:19:36, 11.51s/it]
  1%|          | 2792/225000 [12:33:52<955:53:29, 15.49s/it]
  1%|          | 2793/225000 [12:34:02<854:50:04, 13.85s/it]
  1%|          | 2794/225000 [12:34:13<798:24:42, 12.94s/it]
  1%|          | 2795/225000 [12:34:26<801:50:46, 12.99s/it]
  1%|          | 2796/225000 [12:34:42<858:34:26, 13.91s/it]
  1%|          | 2797/225000 [12:34:55<830:11:14, 13.45s/it]
  1%|          | 2798/225000 [12:35:08<822:04:49, 13.32s/it]
  1%|          | 2799/225000 [12:35:18<772:53:57, 12.52s/it]
  1%|          | 2800/225000 [12:35:31<780:31:40, 12.65s/it]
                                                            
{'loss': '0.9607', 'grad_norm': '2.122', 'learning_rate': '1.398e-05', 'epoch': '0.01244', 'num_input_tokens_seen': 23418224, 'train_runtime': '4.533e+04', 'train_tokens_per_second': '516.6'}

  1%|          | 2800/225000 [12:35:31<780:31:40, 12.65s/it]
  1%|          | 2801/225000 [12:35:42<747:49:33, 12.12s/it]
  1%|          | 2802/225000 [12:35:57<789:30:32, 12.79s/it]
  1%|          | 2803/225000 [12:36:04<689:02:37, 11.16s/it]
  1%|          | 2804/225000 [12:36:18<745:18:33, 12.08s/it]
  1%|          | 2805/225000 [12:36:26<673:11:13, 10.91s/it]
  1%|          | 2806/225000 [12:36:49<892:39:34, 14.46s/it]
  1%|          | 2807/225000 [12:37:09<996:30:31, 16.15s/it]
  1%|          | 2808/225000 [12:37:24<979:00:41, 15.86s/it]
  1%|          | 2809/225000 [12:37:38<932:03:32, 15.10s/it]
  1%|          | 2810/225000 [12:37:53<939:40:43, 15.23s/it]
  1%|          | 2811/225000 [12:37:59<762:53:18, 12.36s/it]
  1%|          | 2812/225000 [12:38:13<802:14:59, 13.00s/it]
  1%|▏         | 2813/225000 [12:38:24<760:31:58, 12.32s/it]
  1%|▏         | 2814/225000 [12:38:40<833:58:18, 13.51s/it]
  1%|▏         | 2815/225000 [12:38:45<676:43:30, 10.96s/it]
  1%|▏         | 2816/225000 [12:38:59<717:19:18, 11.62s/it]
  1%|▏         | 2817/225000 [12:39:12<747:53:14, 12.12s/it]
  1%|▏         | 2818/225000 [12:39:24<750:30:04, 12.16s/it]
  1%|▏         | 2819/225000 [12:39:40<820:29:13, 13.29s/it]
  1%|▏         | 2820/225000 [12:39:54<830:16:58, 13.45s/it]
  1%|▏         | 2821/225000 [12:40:09<851:57:32, 13.80s/it]
  1%|▏         | 2822/225000 [12:40:26<922:51:59, 14.95s/it]
  1%|▏         | 2823/225000 [12:40:36<822:45:48, 13.33s/it]
  1%|▏         | 2824/225000 [12:40:50<837:03:42, 13.56s/it]
  1%|▏         | 2825/225000 [12:41:04<851:59:49, 13.81s/it]
  1%|▏         | 2826/225000 [12:41:13<762:22:36, 12.35s/it]
  1%|▏         | 2827/225000 [12:41:23<711:41:49, 11.53s/it]
  1%|▏         | 2828/225000 [12:41:37<770:07:29, 12.48s/it]
  1%|▏         | 2829/225000 [12:41:46<704:32:05, 11.42s/it]
  1%|▏         | 2830/225000 [12:42:01<761:58:50, 12.35s/it]
  1%|▏         | 2831/225000 [12:42:11<721:49:49, 11.70s/it]
  1%|▏         | 2832/225000 [12:42:35<951:19:37, 15.42s/it]
  1%|▏         | 2833/225000 [12:42:42<794:55:54, 12.88s/it]
  1%|▏         | 2834/225000 [12:43:01<911:30:18, 14.77s/it]
  1%|▏         | 2835/225000 [12:43:15<888:39:20, 14.40s/it]
  1%|▏         | 2836/225000 [12:43:25<816:44:47, 13.23s/it]
  1%|▏         | 2837/225000 [12:43:32<697:37:48, 11.30s/it]
  1%|▏         | 2838/225000 [12:43:46<752:59:51, 12.20s/it]
  1%|▏         | 2839/225000 [12:43:53<639:13:37, 10.36s/it]
  1%|▏         | 2840/225000 [12:44:11<781:48:42, 12.67s/it]
  1%|▏         | 2841/225000 [12:44:16<643:10:09, 10.42s/it]
  1%|▏         | 2842/225000 [12:44:26<644:43:46, 10.45s/it]
  1%|▏         | 2843/225000 [12:44:36<624:46:52, 10.12s/it]
  1%|▏         | 2844/225000 [12:44:43<574:16:25,  9.31s/it]
  1%|▏         | 2845/225000 [12:44:59<700:33:20, 11.35s/it]
  1%|▏         | 2846/225000 [12:45:10<695:43:03, 11.27s/it]
  1%|▏         | 2847/225000 [12:45:19<649:45:32, 10.53s/it]
  1%|▏         | 2848/225000 [12:45:37<793:19:43, 12.86s/it]
  1%|▏         | 2849/225000 [12:45:51<813:08:14, 13.18s/it]
  1%|▏         | 2850/225000 [12:46:05<830:28:05, 13.46s/it]
                                                            
{'loss': '0.9372', 'grad_norm': '1.806', 'learning_rate': '1.423e-05', 'epoch': '0.01267', 'num_input_tokens_seen': 23743032, 'train_runtime': '4.597e+04', 'train_tokens_per_second': '516.5'}

  1%|▏         | 2850/225000 [12:46:05<830:28:05, 13.46s/it]
  1%|▏         | 2851/225000 [12:46:15<768:03:54, 12.45s/it]
  1%|▏         | 2852/225000 [12:46:30<806:03:49, 13.06s/it]
  1%|▏         | 2853/225000 [12:46:43<801:58:16, 13.00s/it]
  1%|▏         | 2854/225000 [12:46:52<738:52:28, 11.97s/it]
  1%|▏         | 2855/225000 [12:47:12<873:48:19, 14.16s/it]
  1%|▏         | 2856/225000 [12:47:31<969:44:28, 15.72s/it]
  1%|▏         | 2857/225000 [12:47:44<926:59:31, 15.02s/it]
  1%|▏         | 2858/225000 [12:47:59<910:46:40, 14.76s/it]
  1%|▏         | 2859/225000 [12:48:15<932:41:42, 15.12s/it]
  1%|▏         | 2860/225000 [12:48:19<739:31:43, 11.98s/it]
  1%|▏         | 2861/225000 [12:48:32<759:01:10, 12.30s/it]
  1%|▏         | 2862/225000 [12:48:46<779:59:14, 12.64s/it]
  1%|▏         | 2863/225000 [12:49:10<995:15:30, 16.13s/it]
  1%|▏         | 2864/225000 [12:49:18<842:25:17, 13.65s/it]
  1%|▏         | 2865/225000 [12:49:26<745:33:58, 12.08s/it]
  1%|▏         | 2866/225000 [12:49:38<731:49:37, 11.86s/it]
  1%|▏         | 2867/225000 [12:49:52<773:46:16, 12.54s/it]
  1%|▏         | 2868/225000 [12:50:05<782:07:56, 12.68s/it]
  1%|▏         | 2869/225000 [12:50:14<720:58:53, 11.68s/it]
  1%|▏         | 2870/225000 [12:50:20<609:56:24,  9.89s/it]
  1%|▏         | 2871/225000 [12:50:30<622:39:54, 10.09s/it]
  1%|▏         | 2872/225000 [12:50:49<783:13:26, 12.69s/it]
  1%|▏         | 2873/225000 [12:51:02<794:21:51, 12.87s/it]
  1%|▏         | 2874/225000 [12:51:12<731:16:37, 11.85s/it]
  1%|▏         | 2875/225000 [12:51:31<859:38:49, 13.93s/it]
  1%|▏         | 2876/225000 [12:51:44<856:39:19, 13.88s/it]
  1%|▏         | 2877/225000 [12:51:57<835:29:24, 13.54s/it]
  1%|▏         | 2878/225000 [12:52:12<857:05:57, 13.89s/it]
  1%|▏         | 2879/225000 [12:52:25<852:13:34, 13.81s/it]
  1%|▏         | 2880/225000 [12:52:40<871:33:56, 14.13s/it]
  1%|▏         | 2881/225000 [12:52:57<910:08:57, 14.75s/it]
  1%|▏         | 2882/225000 [12:53:09<865:41:28, 14.03s/it]
  1%|▏         | 2883/225000 [12:53:25<907:36:34, 14.71s/it]
  1%|▏         | 2884/225000 [12:53:36<840:06:18, 13.62s/it]
  1%|▏         | 2885/225000 [12:53:52<878:49:49, 14.24s/it]
  1%|▏         | 2886/225000 [12:54:01<778:52:02, 12.62s/it]
  1%|▏         | 2887/225000 [12:54:13<776:07:51, 12.58s/it]
  1%|▏         | 2888/225000 [12:54:23<725:53:24, 11.77s/it]
  1%|▏         | 2889/225000 [12:54:31<649:09:02, 10.52s/it]
  1%|▏         | 2890/225000 [12:54:44<690:13:25, 11.19s/it]
  1%|▏         | 2891/225000 [12:54:54<683:08:07, 11.07s/it]
  1%|▏         | 2892/225000 [12:55:09<746:11:26, 12.09s/it]
  1%|▏         | 2893/225000 [12:55:15<646:03:34, 10.47s/it]
  1%|▏         | 2894/225000 [12:55:27<659:54:54, 10.70s/it]
  1%|▏         | 2895/225000 [12:55:34<604:32:38,  9.80s/it]
  1%|▏         | 2896/225000 [12:55:51<729:25:02, 11.82s/it]
  1%|▏         | 2897/225000 [12:56:04<760:28:51, 12.33s/it]
  1%|▏         | 2898/225000 [12:56:21<836:02:00, 13.55s/it]
  1%|▏         | 2899/225000 [12:56:36<864:51:25, 14.02s/it]
  1%|▏         | 2900/225000 [12:56:49<845:06:16, 13.70s/it]
                                                            
{'loss': '0.944', 'grad_norm': '1.557', 'learning_rate': '1.447e-05', 'epoch': '0.01289', 'num_input_tokens_seen': 24066736, 'train_runtime': '4.661e+04', 'train_tokens_per_second': '516.3'}

  1%|▏         | 2900/225000 [12:56:49<845:06:16, 13.70s/it]
  1%|▏         | 2901/225000 [12:57:05<889:01:02, 14.41s/it]
  1%|▏         | 2902/225000 [12:57:13<764:55:21, 12.40s/it]
  1%|▏         | 2903/225000 [12:57:19<651:54:12, 10.57s/it]
  1%|▏         | 2904/225000 [12:57:25<566:46:20,  9.19s/it]
  1%|▏         | 2905/225000 [12:57:45<776:07:01, 12.58s/it]
  1%|▏         | 2906/225000 [12:57:58<782:37:12, 12.69s/it]
  1%|▏         | 2907/225000 [12:58:12<808:01:46, 13.10s/it]
  1%|▏         | 2908/225000 [12:58:28<850:42:15, 13.79s/it]
  1%|▏         | 2909/225000 [12:58:35<722:54:59, 11.72s/it]
  1%|▏         | 2910/225000 [12:58:49<774:53:20, 12.56s/it]
  1%|▏         | 2911/225000 [12:58:59<724:44:14, 11.75s/it]
  1%|▏         | 2912/225000 [12:59:16<815:12:44, 13.21s/it]
  1%|▏         | 2913/225000 [12:59:28<806:03:25, 13.07s/it]
  1%|▏         | 2914/225000 [12:59:36<703:46:12, 11.41s/it]
  1%|▏         | 2915/225000 [12:59:43<619:56:03, 10.05s/it]
  1%|▏         | 2916/225000 [13:00:00<746:03:46, 12.09s/it]
  1%|▏         | 2917/225000 [13:00:10<708:38:46, 11.49s/it]
  1%|▏         | 2918/225000 [13:00:19<665:02:51, 10.78s/it]
  1%|▏         | 2919/225000 [13:00:38<818:16:29, 13.26s/it]
  1%|▏         | 2920/225000 [13:00:46<716:07:35, 11.61s/it]
  1%|▏         | 2921/225000 [13:01:00<765:24:22, 12.41s/it]
  1%|▏         | 2922/225000 [13:01:12<750:34:58, 12.17s/it]
  1%|▏         | 2923/225000 [13:01:25<774:10:57, 12.55s/it]
  1%|▏         | 2924/225000 [13:01:35<725:37:35, 11.76s/it]
  1%|▏         | 2925/225000 [13:01:47<732:24:57, 11.87s/it]
  1%|▏         | 2926/225000 [13:01:59<739:11:31, 11.98s/it]
  1%|▏         | 2927/225000 [13:02:12<744:19:53, 12.07s/it]
  1%|▏         | 2928/225000 [13:02:24<748:50:41, 12.14s/it]
  1%|▏         | 2929/225000 [13:02:34<705:45:20, 11.44s/it]
  1%|▏         | 2930/225000 [13:02:45<697:59:58, 11.32s/it]
  1%|▏         | 2931/225000 [13:03:00<762:10:09, 12.36s/it]
  1%|▏         | 2932/225000 [13:03:16<836:55:46, 13.57s/it]
  1%|▏         | 2933/225000 [13:03:31<872:41:54, 14.15s/it]
  1%|▏         | 2934/225000 [13:03:50<956:42:33, 15.51s/it]
  1%|▏         | 2935/225000 [13:03:57<792:07:56, 12.84s/it]
  1%|▏         | 2936/225000 [13:04:08<760:57:47, 12.34s/it]
  1%|▏         | 2937/225000 [13:04:22<794:20:55, 12.88s/it]
  1%|▏         | 2938/225000 [13:04:33<754:53:19, 12.24s/it]
  1%|▏         | 2939/225000 [13:04:39<646:03:52, 10.47s/it]
  1%|▏         | 2940/225000 [13:04:45<569:02:27,  9.23s/it]
  1%|▏         | 2941/225000 [13:05:01<677:39:32, 10.99s/it]
  1%|▏         | 2942/225000 [13:05:20<837:41:10, 13.58s/it]
  1%|▏         | 2943/225000 [13:05:33<829:54:24, 13.45s/it]
  1%|▏         | 2944/225000 [13:05:40<708:59:41, 11.49s/it]
  1%|▏         | 2945/225000 [13:05:53<732:06:57, 11.87s/it]
  1%|▏         | 2946/225000 [13:06:02<683:46:10, 11.09s/it]
  1%|▏         | 2947/225000 [13:06:17<745:58:04, 12.09s/it]
  1%|▏         | 2948/225000 [13:06:26<687:18:39, 11.14s/it]
  1%|▏         | 2949/225000 [13:06:33<613:22:50,  9.94s/it]
  1%|▏         | 2950/225000 [13:06:47<685:25:31, 11.11s/it]
                                                            
{'loss': '0.9361', 'grad_norm': '1.635', 'learning_rate': '1.473e-05', 'epoch': '0.01311', 'num_input_tokens_seen': 24360568, 'train_runtime': '4.721e+04', 'train_tokens_per_second': '516'}

  1%|▏         | 2950/225000 [13:06:47<685:25:31, 11.11s/it]
  1%|▏         | 2951/225000 [13:06:54<612:43:39,  9.93s/it]
  1%|▏         | 2952/225000 [13:07:12<771:53:24, 12.51s/it]
  1%|▏         | 2953/225000 [13:07:21<701:22:27, 11.37s/it]
  1%|▏         | 2954/225000 [13:07:34<731:37:00, 11.86s/it]
  1%|▏         | 2955/225000 [13:07:45<713:51:15, 11.57s/it]
  1%|▏         | 2956/225000 [13:08:01<791:19:01, 12.83s/it]
  1%|▏         | 2957/225000 [13:08:16<845:22:50, 13.71s/it]
  1%|▏         | 2958/225000 [13:08:25<741:01:09, 12.01s/it]
  1%|▏         | 2959/225000 [13:08:48<951:00:15, 15.42s/it]
  1%|▏         | 2960/225000 [13:09:01<912:33:17, 14.80s/it]
  1%|▏         | 2961/225000 [13:09:29<1158:53:16, 18.79s/it]
  1%|▏         | 2962/225000 [13:09:43<1058:23:13, 17.16s/it]
  1%|▏         | 2963/225000 [13:09:52<905:05:38, 14.67s/it] 
  1%|▏         | 2964/225000 [13:10:05<873:19:42, 14.16s/it]
  1%|▏         | 2965/225000 [13:10:16<830:58:38, 13.47s/it]
  1%|▏         | 2966/225000 [13:10:28<788:51:10, 12.79s/it]
  1%|▏         | 2967/225000 [13:10:45<868:50:01, 14.09s/it]
  1%|▏         | 2968/225000 [13:10:58<847:51:10, 13.75s/it]
  1%|▏         | 2969/225000 [13:11:08<788:24:36, 12.78s/it]
  1%|▏         | 2970/225000 [13:11:18<738:00:48, 11.97s/it]
  1%|▏         | 2971/225000 [13:11:34<799:25:00, 12.96s/it]
  1%|▏         | 2972/225000 [13:11:43<733:02:49, 11.89s/it]
  1%|▏         | 2973/225000 [13:11:53<702:16:16, 11.39s/it]
  1%|▏         | 2974/225000 [13:12:10<801:34:39, 13.00s/it]
  1%|▏         | 2975/225000 [13:12:17<695:40:25, 11.28s/it]
  1%|▏         | 2976/225000 [13:12:29<701:48:34, 11.38s/it]
  1%|▏         | 2977/225000 [13:12:39<684:17:07, 11.10s/it]
  1%|▏         | 2978/225000 [13:12:49<652:52:14, 10.59s/it]
  1%|▏         | 2979/225000 [13:13:00<660:11:29, 10.70s/it]
  1%|▏         | 2980/225000 [13:13:14<724:38:24, 11.75s/it]
  1%|▏         | 2981/225000 [13:13:38<947:44:58, 15.37s/it]
  1%|▏         | 2982/225000 [13:13:53<943:44:34, 15.30s/it]
  1%|▏         | 2983/225000 [13:14:01<815:20:58, 13.22s/it]
  1%|▏         | 2984/225000 [13:14:18<876:15:39, 14.21s/it]
  1%|▏         | 2985/225000 [13:14:24<724:10:10, 11.74s/it]
  1%|▏         | 2986/225000 [13:14:38<771:52:53, 12.52s/it]
  1%|▏         | 2987/225000 [13:14:45<672:07:32, 10.90s/it]
  1%|▏         | 2988/225000 [13:15:00<745:43:08, 12.09s/it]
  1%|▏         | 2989/225000 [13:15:15<805:42:25, 13.06s/it]
  1%|▏         | 2990/225000 [13:15:26<766:35:36, 12.43s/it]
  1%|▏         | 2991/225000 [13:15:34<681:09:18, 11.05s/it]
  1%|▏         | 2992/225000 [13:15:48<741:09:04, 12.02s/it]
  1%|▏         | 2993/225000 [13:15:56<669:06:05, 10.85s/it]
  1%|▏         | 2994/225000 [13:16:09<705:31:03, 11.44s/it]
  1%|▏         | 2995/225000 [13:16:21<708:37:14, 11.49s/it]
  1%|▏         | 2996/225000 [13:16:36<768:28:51, 12.46s/it]
  1%|▏         | 2997/225000 [13:16:52<845:20:48, 13.71s/it]
  1%|▏         | 2998/225000 [13:17:06<840:27:43, 13.63s/it]
  1%|▏         | 2999/225000 [13:17:23<903:43:26, 14.65s/it]
  1%|▏         | 3000/225000 [13:17:35<855:20:12, 13.87s/it]
                                                            
{'loss': '0.9036', 'grad_norm': '2.042', 'learning_rate': '1.497e-05', 'epoch': '0.01333', 'num_input_tokens_seen': 24675896, 'train_runtime': '4.786e+04', 'train_tokens_per_second': '515.6'}

  1%|▏         | 3000/225000 [13:17:35<855:20:12, 13.87s/it]
  1%|▏         | 3001/225000 [13:17:42<735:47:51, 11.93s/it]
  1%|▏         | 3002/225000 [13:18:05<929:25:52, 15.07s/it]
  1%|▏         | 3003/225000 [13:18:10<752:35:31, 12.20s/it]
  1%|▏         | 3004/225000 [13:18:20<713:06:13, 11.56s/it]
  1%|▏         | 3005/225000 [13:18:36<796:34:07, 12.92s/it]
  1%|▏         | 3006/225000 [13:18:48<771:26:47, 12.51s/it]
  1%|▏         | 3007/225000 [13:18:58<729:10:29, 11.82s/it]
  1%|▏         | 3008/225000 [13:19:14<812:16:07, 13.17s/it]
  1%|▏         | 3009/225000 [13:19:30<859:53:05, 13.94s/it]
  1%|▏         | 3010/225000 [13:19:47<916:53:46, 14.87s/it]
  1%|▏         | 3011/225000 [13:20:01<903:50:56, 14.66s/it]
  1%|▏         | 3012/225000 [13:20:17<922:18:22, 14.96s/it]
  1%|▏         | 3013/225000 [13:20:27<829:08:10, 13.45s/it]
  1%|▏         | 3014/225000 [13:20:37<765:54:38, 12.42s/it]
  1%|▏         | 3015/225000 [13:20:50<774:18:26, 12.56s/it]
  1%|▏         | 3016/225000 [13:20:57<668:51:04, 10.85s/it]
  1%|▏         | 3017/225000 [13:21:09<693:39:09, 11.25s/it]
  1%|▏         | 3018/225000 [13:21:22<737:24:56, 11.96s/it]
  1%|▏         | 3019/225000 [13:21:40<850:13:53, 13.79s/it]
  1%|▏         | 3020/225000 [13:21:49<746:30:57, 12.11s/it]
  1%|▏         | 3021/225000 [13:21:57<670:18:27, 10.87s/it]
  1%|▏         | 3022/225000 [13:22:07<659:57:54, 10.70s/it]
  1%|▏         | 3023/225000 [13:22:16<621:43:29, 10.08s/it]
  1%|▏         | 3024/225000 [13:22:26<630:19:21, 10.22s/it]
  1%|▏         | 3025/225000 [13:22:34<589:30:31,  9.56s/it]
  1%|▏         | 3026/225000 [13:22:47<650:19:38, 10.55s/it]
  1%|▏         | 3027/225000 [13:23:02<726:06:28, 11.78s/it]
  1%|▏         | 3028/225000 [13:23:24<912:58:04, 14.81s/it]
  1%|▏         | 3029/225000 [13:23:33<810:42:54, 13.15s/it]
  1%|▏         | 3030/225000 [13:23:38<665:42:03, 10.80s/it]
  1%|▏         | 3031/225000 [13:23:55<773:25:57, 12.54s/it]
  1%|▏         | 3032/225000 [13:24:03<696:26:52, 11.30s/it]
  1%|▏         | 3033/225000 [13:24:16<734:59:51, 11.92s/it]
  1%|▏         | 3034/225000 [13:24:38<919:46:50, 14.92s/it]
  1%|▏         | 3035/225000 [13:24:44<743:38:15, 12.06s/it]
  1%|▏         | 3036/225000 [13:24:55<730:18:16, 11.84s/it]
  1%|▏         | 3037/225000 [13:25:06<712:02:50, 11.55s/it]
  1%|▏         | 3038/225000 [13:25:17<700:38:45, 11.36s/it]
  1%|▏         | 3039/225000 [13:25:29<716:02:43, 11.61s/it]
  1%|▏         | 3040/225000 [13:25:43<761:08:51, 12.35s/it]
  1%|▏         | 3041/225000 [13:25:55<743:00:09, 12.05s/it]
  1%|▏         | 3042/225000 [13:26:08<766:58:26, 12.44s/it]
  1%|▏         | 3043/225000 [13:26:21<785:58:36, 12.75s/it]
  1%|▏         | 3044/225000 [13:26:32<746:21:24, 12.11s/it]
  1%|▏         | 3045/225000 [13:26:45<763:19:46, 12.38s/it]
  1%|▏         | 3046/225000 [13:26:54<699:31:15, 11.35s/it]
  1%|▏         | 3047/225000 [13:27:04<679:03:37, 11.01s/it]
  1%|▏         | 3048/225000 [13:27:14<665:42:29, 10.80s/it]
  1%|▏         | 3049/225000 [13:27:27<691:15:50, 11.21s/it]
  1%|▏         | 3050/225000 [13:27:41<750:38:15, 12.18s/it]
                                                            
{'loss': '0.9455', 'grad_norm': '1.275', 'learning_rate': '1.523e-05', 'epoch': '0.01356', 'num_input_tokens_seen': 24978896, 'train_runtime': '4.846e+04', 'train_tokens_per_second': '515.4'}

  1%|▏         | 3050/225000 [13:27:41<750:38:15, 12.18s/it]
  1%|▏         | 3051/225000 [13:27:53<740:40:59, 12.01s/it]
  1%|▏         | 3052/225000 [13:28:05<745:17:23, 12.09s/it]
  1%|▏         | 3053/225000 [13:28:26<906:28:03, 14.70s/it]
  1%|▏         | 3054/225000 [13:28:30<720:42:18, 11.69s/it]
  1%|▏         | 3055/225000 [13:28:40<681:14:10, 11.05s/it]
  1%|▏         | 3056/225000 [13:28:53<709:29:49, 11.51s/it]
  1%|▏         | 3057/225000 [13:29:07<758:35:30, 12.30s/it]
  1%|▏         | 3058/225000 [13:29:19<765:23:54, 12.42s/it]
  1%|▏         | 3059/225000 [13:29:35<819:51:19, 13.30s/it]
  1%|▏         | 3060/225000 [13:29:47<805:05:35, 13.06s/it]
  1%|▏         | 3061/225000 [13:29:55<707:47:20, 11.48s/it]
  1%|▏         | 3062/225000 [13:30:04<657:58:24, 10.67s/it]
  1%|▏         | 3063/225000 [13:30:17<702:43:46, 11.40s/it]
  1%|▏         | 3064/225000 [13:30:26<655:33:11, 10.63s/it]
  1%|▏         | 3065/225000 [13:30:34<612:50:57,  9.94s/it]
  1%|▏         | 3066/225000 [13:30:43<599:28:29,  9.72s/it]
  1%|▏         | 3067/225000 [13:30:59<709:32:45, 11.51s/it]
  1%|▏         | 3068/225000 [13:31:14<766:49:23, 12.44s/it]
  1%|▏         | 3069/225000 [13:31:27<789:11:19, 12.80s/it]
  1%|▏         | 3070/225000 [13:31:39<767:20:15, 12.45s/it]
  1%|▏         | 3071/225000 [13:31:53<802:49:39, 13.02s/it]
  1%|▏         | 3072/225000 [13:32:01<703:24:26, 11.41s/it]
  1%|▏         | 3073/225000 [13:32:22<886:15:34, 14.38s/it]
  1%|▏         | 3074/225000 [13:32:32<810:56:20, 13.15s/it]
  1%|▏         | 3075/225000 [13:32:45<807:46:51, 13.10s/it]
  1%|▏         | 3076/225000 [13:32:55<750:33:44, 12.18s/it]
  1%|▏         | 3077/225000 [13:33:07<743:26:50, 12.06s/it]
  1%|▏         | 3078/225000 [13:33:12<608:33:15,  9.87s/it]
  1%|▏         | 3079/225000 [13:33:23<634:46:18, 10.30s/it]
  1%|▏         | 3080/225000 [13:33:37<692:00:10, 11.23s/it]
  1%|▏         | 3081/225000 [13:33:43<597:48:36,  9.70s/it]
  1%|▏         | 3082/225000 [13:33:58<707:10:00, 11.47s/it]
  1%|▏         | 3083/225000 [13:34:08<667:44:35, 10.83s/it]
  1%|▏         | 3084/225000 [13:34:18<659:24:32, 10.70s/it]
  1%|▏         | 3085/225000 [13:34:29<658:55:42, 10.69s/it]
  1%|▏         | 3086/225000 [13:34:42<706:22:11, 11.46s/it]
  1%|▏         | 3087/225000 [13:34:52<669:37:39, 10.86s/it]
  1%|▏         | 3088/225000 [13:35:09<793:51:54, 12.88s/it]
  1%|▏         | 3089/225000 [13:35:22<798:05:18, 12.95s/it]
  1%|▏         | 3090/225000 [13:35:31<715:02:44, 11.60s/it]
  1%|▏         | 3091/225000 [13:35:48<822:14:22, 13.34s/it]
  1%|▏         | 3092/225000 [13:36:05<885:17:13, 14.36s/it]
  1%|▏         | 3093/225000 [13:36:19<875:21:20, 14.20s/it]
  1%|▏         | 3094/225000 [13:36:30<827:56:45, 13.43s/it]
  1%|▏         | 3095/225000 [13:36:39<739:14:40, 11.99s/it]
  1%|▏         | 3096/225000 [13:36:44<605:13:41,  9.82s/it]
  1%|▏         | 3097/225000 [13:36:56<655:14:36, 10.63s/it]
  1%|▏         | 3098/225000 [13:37:14<789:49:01, 12.81s/it]
  1%|▏         | 3099/225000 [13:37:23<721:57:16, 11.71s/it]
  1%|▏         | 3100/225000 [13:37:35<717:03:09, 11.63s/it]
                                                            
{'loss': '0.9247', 'grad_norm': '1.928', 'learning_rate': '1.548e-05', 'epoch': '0.01378', 'num_input_tokens_seen': 25288048, 'train_runtime': '4.906e+04', 'train_tokens_per_second': '515.5'}

  1%|▏         | 3100/225000 [13:37:35<717:03:09, 11.63s/it]
  1%|▏         | 3101/225000 [13:37:44<671:34:57, 10.90s/it]
  1%|▏         | 3102/225000 [13:37:51<606:13:37,  9.84s/it]
  1%|▏         | 3103/225000 [13:38:01<609:03:15,  9.88s/it]
  1%|▏         | 3104/225000 [13:38:26<879:21:02, 14.27s/it]
  1%|▏         | 3105/225000 [13:38:35<783:07:09, 12.71s/it]
  1%|▏         | 3106/225000 [13:38:40<651:04:35, 10.56s/it]
  1%|▏         | 3107/225000 [13:38:54<704:56:20, 11.44s/it]
  1%|▏         | 3108/225000 [13:39:08<753:41:01, 12.23s/it]
  1%|▏         | 3109/225000 [13:39:20<742:35:55, 12.05s/it]
  1%|▏         | 3110/225000 [13:39:28<673:13:09, 10.92s/it]
  1%|▏         | 3111/225000 [13:39:37<643:25:32, 10.44s/it]
  1%|▏         | 3112/225000 [13:39:50<682:33:03, 11.07s/it]
  1%|▏         | 3113/225000 [13:39:58<637:27:49, 10.34s/it]
  1%|▏         | 3114/225000 [13:40:11<677:30:01, 10.99s/it]
  1%|▏         | 3115/225000 [13:40:19<633:23:53, 10.28s/it]
  1%|▏         | 3116/225000 [13:40:41<844:11:06, 13.70s/it]
  1%|▏         | 3117/225000 [13:40:51<772:17:00, 12.53s/it]
  1%|▏         | 3118/225000 [13:41:07<828:12:32, 13.44s/it]
  1%|▏         | 3119/225000 [13:41:18<789:21:34, 12.81s/it]
  1%|▏         | 3120/225000 [13:41:29<754:32:28, 12.24s/it]
  1%|▏         | 3121/225000 [13:41:36<657:54:51, 10.67s/it]
  1%|▏         | 3122/225000 [13:41:50<716:37:56, 11.63s/it]
  1%|▏         | 3123/225000 [13:41:58<656:11:36, 10.65s/it]
  1%|▏         | 3124/225000 [13:42:11<698:39:12, 11.34s/it]
  1%|▏         | 3125/225000 [13:42:17<604:52:11,  9.81s/it]
  1%|▏         | 3126/225000 [13:42:23<525:26:45,  8.53s/it]
  1%|▏         | 3127/225000 [13:42:31<522:03:20,  8.47s/it]
  1%|▏         | 3128/225000 [13:42:47<653:05:19, 10.60s/it]
  1%|▏         | 3129/225000 [13:42:52<555:10:35,  9.01s/it]
  1%|▏         | 3130/225000 [13:43:08<682:26:33, 11.07s/it]
  1%|▏         | 3131/225000 [13:43:17<645:21:19, 10.47s/it]
  1%|▏         | 3132/225000 [13:43:29<678:20:33, 11.01s/it]
  1%|▏         | 3133/225000 [13:43:35<591:20:39,  9.60s/it]
  1%|▏         | 3134/225000 [13:43:47<633:44:28, 10.28s/it]
  1%|▏         | 3135/225000 [13:44:03<724:48:44, 11.76s/it]
  1%|▏         | 3136/225000 [13:44:24<896:40:28, 14.55s/it]
  1%|▏         | 3137/225000 [13:44:37<873:10:12, 14.17s/it]
  1%|▏         | 3138/225000 [13:44:42<707:40:37, 11.48s/it]
  1%|▏         | 3139/225000 [13:44:53<702:19:51, 11.40s/it]
  1%|▏         | 3140/225000 [13:44:59<594:12:11,  9.64s/it]
  1%|▏         | 3141/225000 [13:45:13<683:14:39, 11.09s/it]
  1%|▏         | 3142/225000 [13:45:23<657:23:55, 10.67s/it]
  1%|▏         | 3143/225000 [13:45:36<702:33:05, 11.40s/it]
  1%|▏         | 3144/225000 [13:45:42<599:31:58,  9.73s/it]
  1%|▏         | 3145/225000 [13:45:48<535:50:54,  8.70s/it]
  1%|▏         | 3146/225000 [13:46:02<638:51:16, 10.37s/it]
  1%|▏         | 3147/225000 [13:46:12<622:00:09, 10.09s/it]
  1%|▏         | 3148/225000 [13:46:29<746:07:32, 12.11s/it]
  1%|▏         | 3149/225000 [13:46:38<688:06:06, 11.17s/it]
  1%|▏         | 3150/225000 [13:46:54<777:56:47, 12.62s/it]
                                                            
{'loss': '0.9305', 'grad_norm': '1.838', 'learning_rate': '1.572e-05', 'epoch': '0.014', 'num_input_tokens_seen': 25585576, 'train_runtime': '4.961e+04', 'train_tokens_per_second': '515.7'}

  1%|▏         | 3150/225000 [13:46:54<777:56:47, 12.62s/it]
  1%|▏         | 3151/225000 [13:47:10<846:41:34, 13.74s/it]
  1%|▏         | 3152/225000 [13:47:18<741:07:32, 12.03s/it]
  1%|▏         | 3153/225000 [13:47:28<704:03:37, 11.43s/it]
  1%|▏         | 3154/225000 [13:47:41<721:58:17, 11.72s/it]
  1%|▏         | 3155/225000 [13:48:02<898:26:27, 14.58s/it]
  1%|▏         | 3156/225000 [13:48:16<900:51:44, 14.62s/it]
  1%|▏         | 3157/225000 [13:48:23<758:14:16, 12.30s/it]
  1%|▏         | 3158/225000 [13:48:39<823:21:06, 13.36s/it]
  1%|▏         | 3159/225000 [13:48:50<776:33:39, 12.60s/it]
  1%|▏         | 3160/225000 [13:49:02<758:29:59, 12.31s/it]
  1%|▏         | 3161/225000 [13:49:20<873:18:44, 14.17s/it]
  1%|▏         | 3162/225000 [13:49:33<841:53:55, 13.66s/it]
  1%|▏         | 3163/225000 [13:49:40<722:26:32, 11.72s/it]
  1%|▏         | 3164/225000 [13:49:48<660:23:00, 10.72s/it]
  1%|▏         | 3165/225000 [13:50:07<801:40:55, 13.01s/it]
  1%|▏         | 3166/225000 [13:50:18<765:17:49, 12.42s/it]
  1%|▏         | 3167/225000 [13:50:28<718:42:32, 11.66s/it]
  1%|▏         | 3168/225000 [13:50:41<749:40:30, 12.17s/it]
  1%|▏         | 3169/225000 [13:50:54<761:12:25, 12.35s/it]
  1%|▏         | 3170/225000 [13:51:14<905:37:56, 14.70s/it]
  1%|▏         | 3171/225000 [13:51:24<819:40:27, 13.30s/it]
  1%|▏         | 3172/225000 [13:51:31<696:27:49, 11.30s/it]
  1%|▏         | 3173/225000 [13:51:41<672:15:29, 10.91s/it]
  1%|▏         | 3174/225000 [13:51:52<682:52:55, 11.08s/it]
  1%|▏         | 3175/225000 [13:52:03<685:55:45, 11.13s/it]
  1%|▏         | 3176/225000 [13:52:22<824:08:41, 13.38s/it]
  1%|▏         | 3177/225000 [13:52:33<776:05:33, 12.60s/it]
  1%|▏         | 3178/225000 [13:52:38<634:19:36, 10.29s/it]
  1%|▏         | 3179/225000 [13:52:56<789:56:44, 12.82s/it]
  1%|▏         | 3180/225000 [13:53:09<796:58:46, 12.93s/it]
  1%|▏         | 3181/225000 [13:53:19<743:01:49, 12.06s/it]
  1%|▏         | 3182/225000 [13:53:27<657:22:16, 10.67s/it]
  1%|▏         | 3183/225000 [13:53:42<733:15:10, 11.90s/it]
  1%|▏         | 3184/225000 [13:53:47<617:02:00, 10.01s/it]
  1%|▏         | 3185/225000 [13:54:05<761:54:23, 12.37s/it]
  1%|▏         | 3186/225000 [13:54:23<868:34:50, 14.10s/it]
  1%|▏         | 3187/225000 [13:54:37<858:17:00, 13.93s/it]
  1%|▏         | 3188/225000 [13:54:47<789:00:39, 12.81s/it]
  1%|▏         | 3189/225000 [13:54:57<741:40:20, 12.04s/it]
  1%|▏         | 3190/225000 [13:55:13<810:57:20, 13.16s/it]
  1%|▏         | 3191/225000 [13:55:28<838:38:36, 13.61s/it]
  1%|▏         | 3192/225000 [13:55:36<735:55:35, 11.94s/it]
  1%|▏         | 3193/225000 [13:55:51<800:42:20, 13.00s/it]
  1%|▏         | 3194/225000 [13:56:03<773:10:24, 12.55s/it]
  1%|▏         | 3195/225000 [13:56:10<674:30:07, 10.95s/it]
  1%|▏         | 3196/225000 [13:56:22<702:31:45, 11.40s/it]
  1%|▏         | 3197/225000 [13:56:35<730:31:49, 11.86s/it]
  1%|▏         | 3198/225000 [13:56:50<776:42:19, 12.61s/it]
  1%|▏         | 3199/225000 [13:57:02<779:44:36, 12.66s/it]
  1%|▏         | 3200/225000 [13:57:16<796:11:41, 12.92s/it]
                                                            
{'loss': '0.927', 'grad_norm': '2.368', 'learning_rate': '1.598e-05', 'epoch': '0.01422', 'num_input_tokens_seen': 25896344, 'train_runtime': '5.024e+04', 'train_tokens_per_second': '515.5'}

  1%|▏         | 3200/225000 [13:57:16<796:11:41, 12.92s/it]
  1%|▏         | 3201/225000 [13:57:32<850:56:09, 13.81s/it]
  1%|▏         | 3202/225000 [13:57:40<739:33:04, 12.00s/it]
  1%|▏         | 3203/225000 [13:57:52<739:52:12, 12.01s/it]
  1%|▏         | 3204/225000 [13:58:02<717:49:04, 11.65s/it]
  1%|▏         | 3205/225000 [13:58:17<768:25:50, 12.47s/it]
  1%|▏         | 3206/225000 [13:58:26<700:42:15, 11.37s/it]
  1%|▏         | 3207/225000 [13:58:37<692:34:00, 11.24s/it]
  1%|▏         | 3208/225000 [13:58:51<757:54:08, 12.30s/it]
  1%|▏         | 3209/225000 [13:59:12<913:00:53, 14.82s/it]
  1%|▏         | 3210/225000 [13:59:17<730:22:40, 11.86s/it]
  1%|▏         | 3211/225000 [13:59:27<700:36:27, 11.37s/it]
  1%|▏         | 3212/225000 [13:59:35<640:02:21, 10.39s/it]
  1%|▏         | 3213/225000 [13:59:43<589:36:38,  9.57s/it]
  1%|▏         | 3214/225000 [13:59:53<606:06:14,  9.84s/it]
  1%|▏         | 3215/225000 [14:00:07<676:51:12, 10.99s/it]
  1%|▏         | 3216/225000 [14:00:13<582:59:09,  9.46s/it]
  1%|▏         | 3217/225000 [14:00:28<678:53:01, 11.02s/it]
  1%|▏         | 3218/225000 [14:00:47<823:14:32, 13.36s/it]
  1%|▏         | 3219/225000 [14:00:55<739:41:03, 12.01s/it]
  1%|▏         | 3220/225000 [14:01:10<795:07:49, 12.91s/it]
  1%|▏         | 3221/225000 [14:01:29<909:29:43, 14.76s/it]
  1%|▏         | 3222/225000 [14:01:35<731:41:26, 11.88s/it]
  1%|▏         | 3223/225000 [14:01:52<829:00:46, 13.46s/it]
  1%|▏         | 3224/225000 [14:02:02<775:17:49, 12.59s/it]
  1%|▏         | 3225/225000 [14:02:14<756:34:31, 12.28s/it]
  1%|▏         | 3226/225000 [14:02:25<739:57:38, 12.01s/it]
  1%|▏         | 3227/225000 [14:02:34<678:12:00, 11.01s/it]
  1%|▏         | 3228/225000 [14:02:44<665:06:17, 10.80s/it]
  1%|▏         | 3229/225000 [14:03:02<797:00:17, 12.94s/it]
  1%|▏         | 3230/225000 [14:03:10<695:13:12, 11.29s/it]
  1%|▏         | 3231/225000 [14:03:21<704:40:50, 11.44s/it]
  1%|▏         | 3232/225000 [14:03:38<806:04:24, 13.09s/it]
  1%|▏         | 3233/225000 [14:03:46<714:49:26, 11.60s/it]
  1%|▏         | 3234/225000 [14:03:59<735:35:34, 11.94s/it]
  1%|▏         | 3235/225000 [14:04:17<838:20:45, 13.61s/it]
  1%|▏         | 3236/225000 [14:04:33<884:05:26, 14.35s/it]
  1%|▏         | 3237/225000 [14:04:38<713:02:38, 11.58s/it]
  1%|▏         | 3238/225000 [14:04:53<769:11:10, 12.49s/it]
  1%|▏         | 3239/225000 [14:05:03<733:00:47, 11.90s/it]
  1%|▏         | 3240/225000 [14:05:10<645:14:17, 10.47s/it]
  1%|▏         | 3241/225000 [14:05:24<706:15:03, 11.47s/it]
  1%|▏         | 3242/225000 [14:05:30<612:14:01,  9.94s/it]
  1%|▏         | 3243/225000 [14:05:42<652:52:26, 10.60s/it]
  1%|▏         | 3244/225000 [14:05:55<689:50:19, 11.20s/it]
  1%|▏         | 3245/225000 [14:06:01<591:11:26,  9.60s/it]
  1%|▏         | 3246/225000 [14:06:10<583:25:06,  9.47s/it]
  1%|▏         | 3247/225000 [14:06:19<566:35:34,  9.20s/it]
  1%|▏         | 3248/225000 [14:06:30<600:17:57,  9.75s/it]
  1%|▏         | 3249/225000 [14:06:39<583:06:15,  9.47s/it]
  1%|▏         | 3250/225000 [14:06:56<734:15:15, 11.92s/it]
                                                            
{'loss': '0.9319', 'grad_norm': '2.07', 'learning_rate': '1.623e-05', 'epoch': '0.01444', 'num_input_tokens_seen': 26195168, 'train_runtime': '5.082e+04', 'train_tokens_per_second': '515.5'}

  1%|▏         | 3250/225000 [14:06:56<734:15:15, 11.92s/it]
  1%|▏         | 3251/225000 [14:07:08<736:12:26, 11.95s/it]
  1%|▏         | 3252/225000 [14:07:27<855:23:59, 13.89s/it]
  1%|▏         | 3253/225000 [14:07:40<848:10:11, 13.77s/it]
  1%|▏         | 3254/225000 [14:07:50<785:32:26, 12.75s/it]
  1%|▏         | 3255/225000 [14:08:06<828:12:55, 13.45s/it]
  1%|▏         | 3256/225000 [14:08:18<813:11:24, 13.20s/it]
  1%|▏         | 3257/225000 [14:08:34<869:57:14, 14.12s/it]
  1%|▏         | 3258/225000 [14:08:41<724:31:50, 11.76s/it]
  1%|▏         | 3259/225000 [14:08:58<832:39:01, 13.52s/it]
  1%|▏         | 3260/225000 [14:09:06<733:50:53, 11.91s/it]
  1%|▏         | 3261/225000 [14:09:20<756:26:08, 12.28s/it]
  1%|▏         | 3262/225000 [14:09:27<662:16:13, 10.75s/it]
  1%|▏         | 3263/225000 [14:09:39<687:29:45, 11.16s/it]
  1%|▏         | 3264/225000 [14:09:52<716:44:08, 11.64s/it]
  1%|▏         | 3265/225000 [14:10:11<852:22:52, 13.84s/it]
  1%|▏         | 3266/225000 [14:10:24<836:39:28, 13.58s/it]
  1%|▏         | 3267/225000 [14:10:37<832:43:54, 13.52s/it]
  1%|▏         | 3268/225000 [14:10:50<825:34:10, 13.40s/it]
  1%|▏         | 3269/225000 [14:11:03<806:23:31, 13.09s/it]
  1%|▏         | 3270/225000 [14:11:10<705:33:05, 11.46s/it]
  1%|▏         | 3271/225000 [14:11:19<648:44:50, 10.53s/it]
  1%|▏         | 3272/225000 [14:11:29<647:27:30, 10.51s/it]
  1%|▏         | 3273/225000 [14:11:47<776:58:10, 12.62s/it]
  1%|▏         | 3274/225000 [14:12:02<831:12:09, 13.50s/it]
  1%|▏         | 3275/225000 [14:12:18<876:15:45, 14.23s/it]
  1%|▏         | 3276/225000 [14:12:25<738:11:05, 11.99s/it]
  1%|▏         | 3277/225000 [14:12:43<854:04:13, 13.87s/it]
  1%|▏         | 3278/225000 [14:12:51<740:14:45, 12.02s/it]
  1%|▏         | 3279/225000 [14:13:15<973:37:14, 15.81s/it]
  1%|▏         | 3280/225000 [14:13:23<815:42:36, 13.24s/it]
  1%|▏         | 3281/225000 [14:13:31<728:45:23, 11.83s/it]
  1%|▏         | 3282/225000 [14:13:42<716:22:09, 11.63s/it]
  1%|▏         | 3283/225000 [14:13:57<781:15:14, 12.69s/it]
  1%|▏         | 3284/225000 [14:14:07<715:14:02, 11.61s/it]
  1%|▏         | 3285/225000 [14:14:14<635:21:02, 10.32s/it]
  1%|▏         | 3286/225000 [14:14:27<688:56:59, 11.19s/it]
  1%|▏         | 3287/225000 [14:14:44<802:19:23, 13.03s/it]
  1%|▏         | 3288/225000 [14:14:57<786:39:36, 12.77s/it]
  1%|▏         | 3289/225000 [14:15:09<775:58:49, 12.60s/it]
  1%|▏         | 3290/225000 [14:15:25<845:01:44, 13.72s/it]
  1%|▏         | 3291/225000 [14:15:34<756:00:39, 12.28s/it]
  1%|▏         | 3292/225000 [14:15:47<760:04:16, 12.34s/it]
  1%|▏         | 3293/225000 [14:15:51<617:42:50, 10.03s/it]
  1%|▏         | 3294/225000 [14:16:04<669:22:55, 10.87s/it]
  1%|▏         | 3295/225000 [14:16:18<735:52:07, 11.95s/it]
  1%|▏         | 3296/225000 [14:16:25<642:57:04, 10.44s/it]
  1%|▏         | 3297/225000 [14:16:39<697:33:48, 11.33s/it]
  1%|▏         | 3298/225000 [14:16:56<807:58:50, 13.12s/it]
  1%|▏         | 3299/225000 [14:17:04<713:15:42, 11.58s/it]
  1%|▏         | 3300/225000 [14:17:13<663:34:08, 10.78s/it]
                                                            
{'loss': '0.9375', 'grad_norm': '1.943', 'learning_rate': '1.648e-05', 'epoch': '0.01467', 'num_input_tokens_seen': 26499248, 'train_runtime': '5.143e+04', 'train_tokens_per_second': '515.2'}

  1%|▏         | 3300/225000 [14:17:13<663:34:08, 10.78s/it]
  1%|▏         | 3301/225000 [14:17:26<697:22:09, 11.32s/it]
  1%|▏         | 3302/225000 [14:17:38<727:08:44, 11.81s/it]
  1%|▏         | 3303/225000 [14:17:54<802:57:03, 13.04s/it]
  1%|▏         | 3304/225000 [14:18:06<772:42:58, 12.55s/it]
  1%|▏         | 3305/225000 [14:18:18<768:43:01, 12.48s/it]
  1%|▏         | 3306/225000 [14:18:36<872:48:36, 14.17s/it]
  1%|▏         | 3307/225000 [14:18:46<782:37:47, 12.71s/it]
  1%|▏         | 3308/225000 [14:19:04<882:53:42, 14.34s/it]
  1%|▏         | 3309/225000 [14:19:23<971:55:16, 15.78s/it]
  1%|▏         | 3310/225000 [14:19:38<966:19:35, 15.69s/it]
  1%|▏         | 3311/225000 [14:19:50<894:57:46, 14.53s/it]
  1%|▏         | 3312/225000 [14:20:01<821:17:05, 13.34s/it]
  1%|▏         | 3313/225000 [14:20:10<753:38:44, 12.24s/it]
  1%|▏         | 3314/225000 [14:20:21<717:14:28, 11.65s/it]
  1%|▏         | 3315/225000 [14:20:36<793:00:01, 12.88s/it]
  1%|▏         | 3316/225000 [14:20:42<663:11:29, 10.77s/it]
  1%|▏         | 3317/225000 [14:20:52<643:37:14, 10.45s/it]
  1%|▏         | 3318/225000 [14:21:07<732:24:38, 11.89s/it]
  1%|▏         | 3319/225000 [14:21:13<627:33:41, 10.19s/it]
  1%|▏         | 3320/225000 [14:21:25<645:01:59, 10.48s/it]
  1%|▏         | 3321/225000 [14:21:35<651:14:20, 10.58s/it]
  1%|▏         | 3322/225000 [14:21:44<607:29:18,  9.87s/it]
  1%|▏         | 3323/225000 [14:21:58<689:15:45, 11.19s/it]
  1%|▏         | 3324/225000 [14:22:03<582:32:41,  9.46s/it]
  1%|▏         | 3325/225000 [14:22:23<768:10:27, 12.48s/it]
  1%|▏         | 3326/225000 [14:22:37<799:04:03, 12.98s/it]
  1%|▏         | 3327/225000 [14:22:47<743:16:58, 12.07s/it]
  1%|▏         | 3328/225000 [14:22:56<680:30:50, 11.05s/it]
  1%|▏         | 3329/225000 [14:23:11<768:59:12, 12.49s/it]
  1%|▏         | 3330/225000 [14:23:28<851:38:05, 13.83s/it]
  1%|▏         | 3331/225000 [14:23:44<891:16:32, 14.47s/it]
  1%|▏         | 3332/225000 [14:23:58<872:22:56, 14.17s/it]
  1%|▏         | 3333/225000 [14:24:11<856:08:22, 13.90s/it]
  1%|▏         | 3334/225000 [14:24:27<897:06:05, 14.57s/it]
  1%|▏         | 3335/225000 [14:24:37<810:06:50, 13.16s/it]
  1%|▏         | 3336/225000 [14:24:47<750:39:28, 12.19s/it]
  1%|▏         | 3337/225000 [14:25:00<758:48:58, 12.32s/it]
  1%|▏         | 3338/225000 [14:25:06<650:44:50, 10.57s/it]
  1%|▏         | 3339/225000 [14:25:15<610:30:46,  9.92s/it]
  1%|▏         | 3340/225000 [14:25:29<700:43:28, 11.38s/it]
  1%|▏         | 3341/225000 [14:25:36<609:19:29,  9.90s/it]
  1%|▏         | 3342/225000 [14:25:50<692:15:24, 11.24s/it]
  1%|▏         | 3343/225000 [14:26:11<871:37:27, 14.16s/it]
  1%|▏         | 3344/225000 [14:26:27<907:46:54, 14.74s/it]
  1%|▏         | 3345/225000 [14:26:35<777:24:45, 12.63s/it]
  1%|▏         | 3346/225000 [14:26:48<778:05:28, 12.64s/it]
  1%|▏         | 3347/225000 [14:26:55<690:46:21, 11.22s/it]
  1%|▏         | 3348/225000 [14:27:05<667:48:13, 10.85s/it]
  1%|▏         | 3349/225000 [14:27:13<598:58:44,  9.73s/it]
  1%|▏         | 3350/225000 [14:27:29<721:05:06, 11.71s/it]
                                                            
{'loss': '0.9175', 'grad_norm': '2.083', 'learning_rate': '1.673e-05', 'epoch': '0.01489', 'num_input_tokens_seen': 26804104, 'train_runtime': '5.205e+04', 'train_tokens_per_second': '515'}

  1%|▏         | 3350/225000 [14:27:29<721:05:06, 11.71s/it]
  1%|▏         | 3351/225000 [14:27:43<758:45:35, 12.32s/it]
  1%|▏         | 3352/225000 [14:27:53<721:44:54, 11.72s/it]
  1%|▏         | 3353/225000 [14:28:02<678:10:02, 11.01s/it]
  1%|▏         | 3354/225000 [14:28:20<792:55:03, 12.88s/it]
  1%|▏         | 3355/225000 [14:28:29<727:58:22, 11.82s/it]
  1%|▏         | 3356/225000 [14:28:44<794:28:42, 12.90s/it]
  1%|▏         | 3357/225000 [14:28:56<765:34:10, 12.43s/it]
  1%|▏         | 3358/225000 [14:29:08<755:26:04, 12.27s/it]
  1%|▏         | 3359/225000 [14:29:21<776:08:19, 12.61s/it]
  1%|▏         | 3360/225000 [14:29:40<898:53:01, 14.60s/it]
  1%|▏         | 3361/225000 [14:29:52<847:58:11, 13.77s/it]
  1%|▏         | 3362/225000 [14:30:05<824:31:46, 13.39s/it]
  1%|▏         | 3363/225000 [14:30:16<784:50:05, 12.75s/it]
  1%|▏         | 3364/225000 [14:30:28<772:57:38, 12.56s/it]
  1%|▏         | 3365/225000 [14:30:34<648:07:29, 10.53s/it]
  1%|▏         | 3366/225000 [14:30:46<687:01:37, 11.16s/it]
  1%|▏         | 3367/225000 [14:30:52<585:50:15,  9.52s/it]
  1%|▏         | 3368/225000 [14:31:05<650:08:26, 10.56s/it]
  1%|▏         | 3369/225000 [14:31:25<817:37:13, 13.28s/it]
  1%|▏         | 3370/225000 [14:31:35<768:59:46, 12.49s/it]
  1%|▏         | 3371/225000 [14:31:46<727:46:51, 11.82s/it]
  1%|▏         | 3372/225000 [14:31:58<732:51:12, 11.90s/it]
  1%|▏         | 3373/225000 [14:32:06<663:25:31, 10.78s/it]
  1%|▏         | 3374/225000 [14:32:19<711:30:18, 11.56s/it]
  2%|▏         | 3375/225000 [14:32:25<610:11:50,  9.91s/it]
  2%|▏         | 3376/225000 [14:32:40<705:24:25, 11.46s/it]
  2%|▏         | 3377/225000 [14:32:54<739:00:55, 12.00s/it]
  2%|▏         | 3378/225000 [14:33:00<641:15:37, 10.42s/it]
  2%|▏         | 3379/225000 [14:33:22<845:38:15, 13.74s/it]
  2%|▏         | 3380/225000 [14:33:38<887:58:18, 14.42s/it]
  2%|▏         | 3381/225000 [14:33:45<755:45:17, 12.28s/it]
  2%|▏         | 3382/225000 [14:34:01<821:55:09, 13.35s/it]
  2%|▏         | 3383/225000 [14:34:10<733:15:23, 11.91s/it]
  2%|▏         | 3384/225000 [14:34:15<609:35:15,  9.90s/it]
  2%|▏         | 3385/225000 [14:34:27<649:02:04, 10.54s/it]
  2%|▏         | 3386/225000 [14:34:43<753:22:33, 12.24s/it]
  2%|▏         | 3387/225000 [14:34:56<767:32:05, 12.47s/it]
  2%|▏         | 3388/225000 [14:35:05<712:15:56, 11.57s/it]
  2%|▏         | 3389/225000 [14:35:18<736:16:14, 11.96s/it]
  2%|▏         | 3390/225000 [14:35:28<703:21:50, 11.43s/it]
  2%|▏         | 3391/225000 [14:35:44<778:53:25, 12.65s/it]
  2%|▏         | 3392/225000 [14:35:55<739:39:18, 12.02s/it]
  2%|▏         | 3393/225000 [14:36:09<780:04:33, 12.67s/it]
  2%|▏         | 3394/225000 [14:36:18<714:07:04, 11.60s/it]
  2%|▏         | 3395/225000 [14:36:25<635:50:10, 10.33s/it]
  2%|▏         | 3396/225000 [14:36:37<669:57:25, 10.88s/it]
  2%|▏         | 3397/225000 [14:36:48<661:43:39, 10.75s/it]
  2%|▏         | 3398/225000 [14:36:53<555:37:06,  9.03s/it]
  2%|▏         | 3399/225000 [14:37:02<559:58:46,  9.10s/it]
  2%|▏         | 3400/225000 [14:37:19<713:49:22, 11.60s/it]
                                                            
{'loss': '0.8878', 'grad_norm': '1.689', 'learning_rate': '1.698e-05', 'epoch': '0.01511', 'num_input_tokens_seen': 27097192, 'train_runtime': '5.264e+04', 'train_tokens_per_second': '514.8'}

  2%|▏         | 3400/225000 [14:37:20<713:49:22, 11.60s/it]
  2%|▏         | 3401/225000 [14:37:32<734:14:58, 11.93s/it]
  2%|▏         | 3402/225000 [14:37:42<697:59:19, 11.34s/it]
  2%|▏         | 3403/225000 [14:38:00<813:14:48, 13.21s/it]
  2%|▏         | 3404/225000 [14:38:10<749:52:34, 12.18s/it]
  2%|▏         | 3405/225000 [14:38:26<831:17:58, 13.51s/it]
  2%|▏         | 3406/225000 [14:38:36<761:02:54, 12.36s/it]
  2%|▏         | 3407/225000 [14:38:48<760:35:48, 12.36s/it]
  2%|▏         | 3408/225000 [14:39:01<769:13:54, 12.50s/it]
  2%|▏         | 3409/225000 [14:39:09<689:21:03, 11.20s/it]
  2%|▏         | 3410/225000 [14:39:17<631:34:17, 10.26s/it]
  2%|▏         | 3411/225000 [14:39:28<639:25:45, 10.39s/it]
  2%|▏         | 3412/225000 [14:39:36<603:18:07,  9.80s/it]
  2%|▏         | 3413/225000 [14:40:00<860:05:46, 13.97s/it]
  2%|▏         | 3414/225000 [14:40:09<765:28:54, 12.44s/it]
  2%|▏         | 3415/225000 [14:40:28<892:15:15, 14.50s/it]
  2%|▏         | 3416/225000 [14:40:37<788:22:02, 12.81s/it]
  2%|▏         | 3417/225000 [14:40:48<753:44:30, 12.25s/it]
  2%|▏         | 3418/225000 [14:40:56<682:36:10, 11.09s/it]
  2%|▏         | 3419/225000 [14:41:06<656:03:55, 10.66s/it]
  2%|▏         | 3420/225000 [14:41:17<654:47:49, 10.64s/it]
  2%|▏         | 3421/225000 [14:41:25<618:59:35, 10.06s/it]
  2%|▏         | 3422/225000 [14:41:39<677:54:54, 11.01s/it]
  2%|▏         | 3423/225000 [14:41:54<756:34:24, 12.29s/it]
  2%|▏         | 3424/225000 [14:41:59<623:27:37, 10.13s/it]
  2%|▏         | 3425/225000 [14:42:14<722:12:11, 11.73s/it]
  2%|▏         | 3426/225000 [14:42:22<652:43:00, 10.60s/it]
  2%|▏         | 3427/225000 [14:42:29<577:51:29,  9.39s/it]
  2%|▏         | 3428/225000 [14:42:38<568:24:19,  9.24s/it]
  2%|▏         | 3429/225000 [14:42:45<533:48:25,  8.67s/it]
  2%|▏         | 3430/225000 [14:43:00<645:21:58, 10.49s/it]
  2%|▏         | 3431/225000 [14:43:10<644:58:09, 10.48s/it]
  2%|▏         | 3432/225000 [14:43:26<733:41:41, 11.92s/it]
  2%|▏         | 3433/225000 [14:43:38<740:34:24, 12.03s/it]
  2%|▏         | 3434/225000 [14:43:50<734:13:14, 11.93s/it]
  2%|▏         | 3435/225000 [14:44:05<806:32:36, 13.10s/it]
  2%|▏         | 3436/225000 [14:44:14<729:20:51, 11.85s/it]
  2%|▏         | 3437/225000 [14:44:23<674:54:43, 10.97s/it]
  2%|▏         | 3438/225000 [14:44:32<634:18:37, 10.31s/it]
  2%|▏         | 3439/225000 [14:44:46<699:46:54, 11.37s/it]
  2%|▏         | 3440/225000 [14:45:03<800:08:35, 13.00s/it]
  2%|▏         | 3441/225000 [14:45:15<783:17:14, 12.73s/it]
  2%|▏         | 3442/225000 [14:45:28<786:37:22, 12.78s/it]
  2%|▏         | 3443/225000 [14:45:36<699:44:52, 11.37s/it]
  2%|▏         | 3444/225000 [14:45:44<641:42:56, 10.43s/it]
  2%|▏         | 3445/225000 [14:46:02<772:23:12, 12.55s/it]
  2%|▏         | 3446/225000 [14:46:06<630:20:47, 10.24s/it]
  2%|▏         | 3447/225000 [14:46:22<725:33:36, 11.79s/it]
  2%|▏         | 3448/225000 [14:46:28<631:12:20, 10.26s/it]
  2%|▏         | 3449/225000 [14:46:40<653:01:21, 10.61s/it]
  2%|▏         | 3450/225000 [14:46:48<613:34:49,  9.97s/it]
                                                            
{'loss': '0.9111', 'grad_norm': '1.52', 'learning_rate': '1.722e-05', 'epoch': '0.01533', 'num_input_tokens_seen': 27387760, 'train_runtime': '5.321e+04', 'train_tokens_per_second': '514.7'}

  2%|▏         | 3450/225000 [14:46:48<613:34:49,  9.97s/it]
  2%|▏         | 3451/225000 [14:47:01<661:10:04, 10.74s/it]
  2%|▏         | 3452/225000 [14:47:15<723:36:11, 11.76s/it]
  2%|▏         | 3453/225000 [14:47:28<753:17:52, 12.24s/it]
  2%|▏         | 3454/225000 [14:47:40<732:25:39, 11.90s/it]
  2%|▏         | 3455/225000 [14:47:52<745:59:23, 12.12s/it]
  2%|▏         | 3456/225000 [14:48:09<824:39:33, 13.40s/it]
  2%|▏         | 3457/225000 [14:48:14<681:00:00, 11.07s/it]
  2%|▏         | 3458/225000 [14:48:28<736:09:27, 11.96s/it]
  2%|▏         | 3459/225000 [14:48:39<708:10:56, 11.51s/it]
  2%|▏         | 3460/225000 [14:48:47<658:03:41, 10.69s/it]
  2%|▏         | 3461/225000 [14:48:58<653:39:40, 10.62s/it]
  2%|▏         | 3462/225000 [14:49:03<548:05:43,  8.91s/it]
  2%|▏         | 3463/225000 [14:49:16<633:20:50, 10.29s/it]
  2%|▏         | 3464/225000 [14:49:27<639:33:42, 10.39s/it]
  2%|▏         | 3465/225000 [14:49:40<686:08:36, 11.15s/it]
  2%|▏         | 3466/225000 [14:49:49<656:58:10, 10.68s/it]
  2%|▏         | 3467/225000 [14:50:03<706:12:32, 11.48s/it]
  2%|▏         | 3468/225000 [14:50:08<595:04:04,  9.67s/it]
  2%|▏         | 3469/225000 [14:50:23<680:53:50, 11.06s/it]
  2%|▏         | 3470/225000 [14:50:32<641:55:47, 10.43s/it]
  2%|▏         | 3471/225000 [14:50:44<685:20:25, 11.14s/it]
  2%|▏         | 3472/225000 [14:50:54<665:00:12, 10.81s/it]
  2%|▏         | 3473/225000 [14:51:07<692:15:35, 11.25s/it]
  2%|▏         | 3474/225000 [14:51:23<778:11:53, 12.65s/it]
  2%|▏         | 3475/225000 [14:51:40<859:29:22, 13.97s/it]
  2%|▏         | 3476/225000 [14:51:49<778:10:01, 12.65s/it]
  2%|▏         | 3477/225000 [14:51:59<733:01:06, 11.91s/it]
  2%|▏         | 3478/225000 [14:52:09<693:47:58, 11.28s/it]
  2%|▏         | 3479/225000 [14:52:24<767:40:23, 12.48s/it]
  2%|▏         | 3480/225000 [14:52:36<752:12:07, 12.22s/it]
  2%|▏         | 3481/225000 [14:52:46<716:43:26, 11.65s/it]
  2%|▏         | 3482/225000 [14:53:01<765:35:02, 12.44s/it]
  2%|▏         | 3483/225000 [14:53:19<865:54:35, 14.07s/it]
  2%|▏         | 3484/225000 [14:53:31<827:55:25, 13.46s/it]
  2%|▏         | 3485/225000 [14:53:42<799:05:16, 12.99s/it]
  2%|▏         | 3486/225000 [14:53:53<750:33:13, 12.20s/it]
  2%|▏         | 3487/225000 [14:54:01<680:21:29, 11.06s/it]
  2%|▏         | 3488/225000 [14:54:14<717:35:14, 11.66s/it]
  2%|▏         | 3489/225000 [14:54:29<773:53:00, 12.58s/it]
  2%|▏         | 3490/225000 [14:54:37<698:04:26, 11.35s/it]
  2%|▏         | 3491/225000 [14:54:54<787:27:15, 12.80s/it]
  2%|▏         | 3492/225000 [14:55:04<739:22:08, 12.02s/it]
  2%|▏         | 3493/225000 [14:55:17<761:25:40, 12.37s/it]
  2%|▏         | 3494/225000 [14:55:22<624:15:26, 10.15s/it]
  2%|▏         | 3495/225000 [14:55:28<544:20:55,  8.85s/it]
  2%|▏         | 3496/225000 [14:55:34<493:34:46,  8.02s/it]
  2%|▏         | 3497/225000 [14:55:51<657:42:39, 10.69s/it]
  2%|▏         | 3498/225000 [14:56:09<792:38:43, 12.88s/it]
  2%|▏         | 3499/225000 [14:56:18<727:28:13, 11.82s/it]
  2%|▏         | 3500/225000 [14:56:24<618:53:26, 10.06s/it]
                                                            
{'loss': '0.9315', 'grad_norm': '1.361', 'learning_rate': '1.748e-05', 'epoch': '0.01556', 'num_input_tokens_seen': 27680112, 'train_runtime': '5.378e+04', 'train_tokens_per_second': '514.6'}

  2%|▏         | 3500/225000 [14:56:24<618:53:26, 10.06s/it]
  2%|▏         | 3501/225000 [14:56:39<703:51:31, 11.44s/it]
  2%|▏         | 3502/225000 [14:56:54<765:58:54, 12.45s/it]
  2%|▏         | 3503/225000 [14:57:08<806:57:58, 13.12s/it]
  2%|▏         | 3504/225000 [14:57:18<736:18:56, 11.97s/it]
  2%|▏         | 3505/225000 [14:57:37<878:13:13, 14.27s/it]
  2%|▏         | 3506/225000 [14:57:48<810:15:32, 13.17s/it]
  2%|▏         | 3507/225000 [14:57:56<721:19:07, 11.72s/it]
  2%|▏         | 3508/225000 [14:58:16<864:38:16, 14.05s/it]
  2%|▏         | 3509/225000 [14:58:27<817:07:33, 13.28s/it]
  2%|▏         | 3510/225000 [14:58:36<737:34:32, 11.99s/it]
  2%|▏         | 3511/225000 [14:58:44<661:29:34, 10.75s/it]
  2%|▏         | 3512/225000 [14:58:51<601:34:27,  9.78s/it]
  2%|▏         | 3513/225000 [14:59:06<690:15:49, 11.22s/it]
  2%|▏         | 3514/225000 [14:59:20<748:19:05, 12.16s/it]
  2%|▏         | 3515/225000 [14:59:30<704:24:20, 11.45s/it]
  2%|▏         | 3516/225000 [14:59:42<707:41:06, 11.50s/it]
  2%|▏         | 3517/225000 [14:59:50<639:49:16, 10.40s/it]
  2%|▏         | 3518/225000 [15:00:02<683:11:05, 11.10s/it]
  2%|▏         | 3519/225000 [15:00:18<774:18:35, 12.59s/it]
  2%|▏         | 3520/225000 [15:00:29<732:31:04, 11.91s/it]
  2%|▏         | 3521/225000 [15:00:39<697:34:43, 11.34s/it]
  2%|▏         | 3522/225000 [15:00:55<793:59:36, 12.91s/it]
  2%|▏         | 3523/225000 [15:01:02<674:16:45, 10.96s/it]
  2%|▏         | 3524/225000 [15:01:19<781:20:56, 12.70s/it]
  2%|▏         | 3525/225000 [15:01:29<745:14:12, 12.11s/it]
  2%|▏         | 3526/225000 [15:01:37<669:05:15, 10.88s/it]
  2%|▏         | 3527/225000 [15:01:46<634:25:44, 10.31s/it]
  2%|▏         | 3528/225000 [15:01:57<639:08:50, 10.39s/it]
  2%|▏         | 3529/225000 [15:02:18<835:48:33, 13.59s/it]
  2%|▏         | 3530/225000 [15:02:32<853:48:50, 13.88s/it]
  2%|▏         | 3531/225000 [15:02:47<864:39:14, 14.06s/it]
  2%|▏         | 3532/225000 [15:02:56<775:33:32, 12.61s/it]
  2%|▏         | 3533/225000 [15:03:14<871:50:31, 14.17s/it]
  2%|▏         | 3534/225000 [15:03:21<736:51:42, 11.98s/it]
  2%|▏         | 3535/225000 [15:03:37<809:50:15, 13.16s/it]
  2%|▏         | 3536/225000 [15:03:48<782:59:20, 12.73s/it]
  2%|▏         | 3537/225000 [15:03:57<714:00:58, 11.61s/it]
  2%|▏         | 3538/225000 [15:04:14<801:25:54, 13.03s/it]
  2%|▏         | 3539/225000 [15:04:25<774:16:45, 12.59s/it]
  2%|▏         | 3540/225000 [15:04:35<725:04:26, 11.79s/it]
  2%|▏         | 3541/225000 [15:04:41<616:37:21, 10.02s/it]
  2%|▏         | 3542/225000 [15:04:47<543:20:12,  8.83s/it]
  2%|▏         | 3543/225000 [15:04:52<477:29:00,  7.76s/it]
  2%|▏         | 3544/225000 [15:05:14<734:57:20, 11.95s/it]
  2%|▏         | 3545/225000 [15:05:31<820:33:49, 13.34s/it]
  2%|▏         | 3546/225000 [15:05:38<716:06:55, 11.64s/it]
  2%|▏         | 3547/225000 [15:05:52<752:24:50, 12.23s/it]
  2%|▏         | 3548/225000 [15:06:08<816:25:18, 13.27s/it]
  2%|▏         | 3549/225000 [15:06:20<788:22:33, 12.82s/it]
  2%|▏         | 3550/225000 [15:06:26<673:19:27, 10.95s/it]
                                                            
{'loss': '0.8916', 'grad_norm': '2.108', 'learning_rate': '1.773e-05', 'epoch': '0.01578', 'num_input_tokens_seen': 27981008, 'train_runtime': '5.439e+04', 'train_tokens_per_second': '514.5'}

  2%|▏         | 3550/225000 [15:06:26<673:19:27, 10.95s/it]
  2%|▏         | 3551/225000 [15:06:38<692:57:41, 11.27s/it]
  2%|▏         | 3552/225000 [15:06:50<708:17:53, 11.51s/it]
  2%|▏         | 3553/225000 [15:07:02<721:19:11, 11.73s/it]
  2%|▏         | 3554/225000 [15:07:16<757:39:31, 12.32s/it]
  2%|▏         | 3555/225000 [15:07:22<641:59:59, 10.44s/it]
  2%|▏         | 3556/225000 [15:07:36<709:32:19, 11.53s/it]
  2%|▏         | 3557/225000 [15:07:45<662:25:31, 10.77s/it]
  2%|▏         | 3558/225000 [15:08:03<786:59:39, 12.79s/it]
  2%|▏         | 3559/225000 [15:08:12<719:15:02, 11.69s/it]
  2%|▏         | 3560/225000 [15:08:24<733:39:40, 11.93s/it]
  2%|▏         | 3561/225000 [15:08:31<641:13:18, 10.42s/it]
  2%|▏         | 3562/225000 [15:08:46<729:13:50, 11.86s/it]
  2%|▏         | 3563/225000 [15:08:54<646:07:10, 10.50s/it]
  2%|▏         | 3564/225000 [15:09:06<675:35:18, 10.98s/it]
  2%|▏         | 3565/225000 [15:09:18<700:18:29, 11.39s/it]
  2%|▏         | 3566/225000 [15:09:26<625:24:14, 10.17s/it]
  2%|▏         | 3567/225000 [15:09:33<576:44:15,  9.38s/it]
  2%|▏         | 3568/225000 [15:09:44<605:13:50,  9.84s/it]
  2%|▏         | 3569/225000 [15:10:04<788:49:01, 12.82s/it]
  2%|▏         | 3570/225000 [15:10:08<634:27:38, 10.32s/it]
  2%|▏         | 3571/225000 [15:10:20<668:26:23, 10.87s/it]
  2%|▏         | 3572/225000 [15:10:32<684:22:04, 11.13s/it]
  2%|▏         | 3573/225000 [15:10:47<747:30:28, 12.15s/it]
  2%|▏         | 3574/225000 [15:10:54<666:09:50, 10.83s/it]
  2%|▏         | 3575/225000 [15:11:03<616:21:34, 10.02s/it]
  2%|▏         | 3576/225000 [15:11:13<616:50:16, 10.03s/it]
  2%|▏         | 3577/225000 [15:11:32<796:19:33, 12.95s/it]
  2%|▏         | 3578/225000 [15:11:41<714:14:43, 11.61s/it]
  2%|▏         | 3579/225000 [15:11:56<773:04:07, 12.57s/it]
  2%|▏         | 3580/225000 [15:12:07<742:06:10, 12.07s/it]
  2%|▏         | 3581/225000 [15:12:21<786:19:19, 12.78s/it]
  2%|▏         | 3582/225000 [15:12:30<718:01:21, 11.67s/it]
  2%|▏         | 3583/225000 [15:12:39<669:29:26, 10.89s/it]
  2%|▏         | 3584/225000 [15:12:49<648:21:28, 10.54s/it]
  2%|▏         | 3585/225000 [15:13:12<871:23:08, 14.17s/it]
  2%|▏         | 3586/225000 [15:13:19<739:44:54, 12.03s/it]
  2%|▏         | 3587/225000 [15:13:30<732:52:02, 11.92s/it]
  2%|▏         | 3588/225000 [15:13:43<752:35:30, 12.24s/it]
  2%|▏         | 3589/225000 [15:13:49<637:04:11, 10.36s/it]
  2%|▏         | 3590/225000 [15:14:05<743:31:27, 12.09s/it]
  2%|▏         | 3591/225000 [15:14:13<653:57:44, 10.63s/it]
  2%|▏         | 3592/225000 [15:14:26<699:33:50, 11.37s/it]
  2%|▏         | 3593/225000 [15:14:34<637:13:06, 10.36s/it]
  2%|▏         | 3594/225000 [15:14:47<693:12:00, 11.27s/it]
  2%|▏         | 3595/225000 [15:14:57<661:24:42, 10.75s/it]
  2%|▏         | 3596/225000 [15:15:06<640:30:45, 10.41s/it]
  2%|▏         | 3597/225000 [15:15:18<674:16:29, 10.96s/it]
  2%|▏         | 3598/225000 [15:15:30<685:04:56, 11.14s/it]
  2%|▏         | 3599/225000 [15:15:41<678:19:21, 11.03s/it]
  2%|▏         | 3600/225000 [15:15:52<673:37:38, 10.95s/it]
                                                            
{'loss': '0.8655', 'grad_norm': '1.938', 'learning_rate': '1.797e-05', 'epoch': '0.016', 'num_input_tokens_seen': 28267616, 'train_runtime': '5.495e+04', 'train_tokens_per_second': '514.4'}

  2%|▏         | 3600/225000 [15:15:52<673:37:38, 10.95s/it]
  2%|▏         | 3601/225000 [15:16:03<684:29:50, 11.13s/it]
  2%|▏         | 3602/225000 [15:16:17<731:20:52, 11.89s/it]
  2%|▏         | 3603/225000 [15:16:32<788:23:08, 12.82s/it]
  2%|▏         | 3604/225000 [15:16:42<743:46:35, 12.09s/it]
  2%|▏         | 3605/225000 [15:16:55<756:52:20, 12.31s/it]
  2%|▏         | 3606/225000 [15:17:06<727:53:26, 11.84s/it]
  2%|▏         | 3607/225000 [15:17:13<648:26:09, 10.54s/it]
  2%|▏         | 3608/225000 [15:17:27<701:00:57, 11.40s/it]
  2%|▏         | 3609/225000 [15:17:32<595:42:22,  9.69s/it]
  2%|▏         | 3610/225000 [15:17:46<660:34:36, 10.74s/it]
  2%|▏         | 3611/225000 [15:17:58<686:33:46, 11.16s/it]
  2%|▏         | 3612/225000 [15:18:08<675:13:25, 10.98s/it]
  2%|▏         | 3613/225000 [15:18:28<836:13:31, 13.60s/it]
  2%|▏         | 3614/225000 [15:18:39<789:29:03, 12.84s/it]
  2%|▏         | 3615/225000 [15:18:45<662:53:25, 10.78s/it]
  2%|▏         | 3616/225000 [15:18:51<579:53:27,  9.43s/it]
  2%|▏         | 3617/225000 [15:19:09<733:46:18, 11.93s/it]
  2%|▏         | 3618/225000 [15:19:19<694:14:48, 11.29s/it]
  2%|▏         | 3619/225000 [15:19:27<629:37:08, 10.24s/it]
  2%|▏         | 3620/225000 [15:19:35<590:41:53,  9.61s/it]
  2%|▏         | 3621/225000 [15:19:50<689:16:26, 11.21s/it]
  2%|▏         | 3622/225000 [15:20:04<752:38:55, 12.24s/it]
  2%|▏         | 3623/225000 [15:20:13<680:45:07, 11.07s/it]
  2%|▏         | 3624/225000 [15:20:23<671:30:49, 10.92s/it]
  2%|▏         | 3625/225000 [15:20:43<834:36:27, 13.57s/it]
  2%|▏         | 3626/225000 [15:20:52<749:22:51, 12.19s/it]
  2%|▏         | 3627/225000 [15:20:57<616:24:29, 10.02s/it]
  2%|▏         | 3628/225000 [15:21:08<633:54:21, 10.31s/it]
  2%|▏         | 3629/225000 [15:21:25<767:52:03, 12.49s/it]
  2%|▏         | 3630/225000 [15:21:35<722:12:36, 11.74s/it]
  2%|▏         | 3631/225000 [15:21:47<711:52:27, 11.58s/it]
  2%|▏         | 3632/225000 [15:21:59<733:59:16, 11.94s/it]
  2%|▏         | 3633/225000 [15:22:07<657:57:41, 10.70s/it]
  2%|▏         | 3634/225000 [15:22:13<571:45:24,  9.30s/it]
  2%|▏         | 3635/225000 [15:22:28<676:10:04, 11.00s/it]
  2%|▏         | 3636/225000 [15:22:49<848:15:36, 13.80s/it]
  2%|▏         | 3637/225000 [15:22:56<732:31:40, 11.91s/it]
  2%|▏         | 3638/225000 [15:23:10<775:54:08, 12.62s/it]
  2%|▏         | 3639/225000 [15:23:21<748:18:45, 12.17s/it]
  2%|▏         | 3640/225000 [15:23:32<726:23:39, 11.81s/it]
  2%|▏         | 3641/225000 [15:23:41<667:24:40, 10.85s/it]
  2%|▏         | 3642/225000 [15:23:58<781:07:07, 12.70s/it]
  2%|▏         | 3643/225000 [15:24:10<771:19:00, 12.54s/it]
  2%|▏         | 3644/225000 [15:24:18<677:15:43, 11.01s/it]
  2%|▏         | 3645/225000 [15:24:33<747:11:42, 12.15s/it]
  2%|▏         | 3646/225000 [15:24:42<693:45:43, 11.28s/it]
  2%|▏         | 3647/225000 [15:24:54<710:40:56, 11.56s/it]
  2%|▏         | 3648/225000 [15:25:07<739:09:49, 12.02s/it]
  2%|▏         | 3649/225000 [15:25:17<706:46:28, 11.49s/it]
  2%|▏         | 3650/225000 [15:25:29<713:24:54, 11.60s/it]
                                                            
{'loss': '0.919', 'grad_norm': '2.223', 'learning_rate': '1.822e-05', 'epoch': '0.01622', 'num_input_tokens_seen': 28556040, 'train_runtime': '5.553e+04', 'train_tokens_per_second': '514.2'}

  2%|▏         | 3650/225000 [15:25:29<713:24:54, 11.60s/it]
  2%|▏         | 3651/225000 [15:25:37<651:53:41, 10.60s/it]
  2%|▏         | 3652/225000 [15:25:47<635:53:51, 10.34s/it]
  2%|▏         | 3653/225000 [15:25:59<657:49:36, 10.70s/it]
  2%|▏         | 3654/225000 [15:26:11<686:38:50, 11.17s/it]
  2%|▏         | 3655/225000 [15:26:24<725:30:11, 11.80s/it]
  2%|▏         | 3656/225000 [15:26:37<741:21:27, 12.06s/it]
  2%|▏         | 3657/225000 [15:26:44<646:58:52, 10.52s/it]
  2%|▏         | 3658/225000 [15:26:59<728:44:42, 11.85s/it]
  2%|▏         | 3659/225000 [15:27:12<747:18:40, 12.15s/it]
  2%|▏         | 3660/225000 [15:27:18<647:47:00, 10.54s/it]
  2%|▏         | 3661/225000 [15:27:35<761:41:51, 12.39s/it]
  2%|▏         | 3662/225000 [15:27:47<759:06:33, 12.35s/it]
  2%|▏         | 3663/225000 [15:27:58<730:33:00, 11.88s/it]
  2%|▏         | 3664/225000 [15:28:09<711:28:48, 11.57s/it]
  2%|▏         | 3665/225000 [15:28:23<749:01:49, 12.18s/it]
  2%|▏         | 3666/225000 [15:28:35<758:50:43, 12.34s/it]
  2%|▏         | 3667/225000 [15:28:44<686:17:31, 11.16s/it]
  2%|▏         | 3668/225000 [15:28:56<696:48:31, 11.33s/it]
  2%|▏         | 3669/225000 [15:29:06<675:25:47, 10.99s/it]
  2%|▏         | 3670/225000 [15:29:15<646:47:14, 10.52s/it]
  2%|▏         | 3671/225000 [15:29:23<596:30:52,  9.70s/it]
  2%|▏         | 3672/225000 [15:29:38<700:39:19, 11.40s/it]
  2%|▏         | 3673/225000 [15:29:50<699:07:11, 11.37s/it]
  2%|▏         | 3674/225000 [15:30:03<728:17:35, 11.85s/it]
  2%|▏         | 3675/225000 [15:30:16<749:14:18, 12.19s/it]
  2%|▏         | 3676/225000 [15:30:22<650:26:02, 10.58s/it]
  2%|▏         | 3677/225000 [15:30:28<559:18:03,  9.10s/it]
  2%|▏         | 3678/225000 [15:30:44<678:46:02, 11.04s/it]
  2%|▏         | 3679/225000 [15:31:02<809:34:50, 13.17s/it]
  2%|▏         | 3680/225000 [15:31:20<905:38:46, 14.73s/it]
  2%|▏         | 3681/225000 [15:31:33<871:55:20, 14.18s/it]
  2%|▏         | 3682/225000 [15:31:45<835:40:24, 13.59s/it]
  2%|▏         | 3683/225000 [15:31:52<702:03:15, 11.42s/it]
  2%|▏         | 3684/225000 [15:32:01<675:00:04, 10.98s/it]
  2%|▏         | 3685/225000 [15:32:14<697:16:13, 11.34s/it]
  2%|▏         | 3686/225000 [15:32:22<637:09:31, 10.36s/it]
  2%|▏         | 3687/225000 [15:32:34<676:42:22, 11.01s/it]
  2%|▏         | 3688/225000 [15:32:41<598:10:11,  9.73s/it]
  2%|▏         | 3689/225000 [15:32:54<655:25:23, 10.66s/it]
  2%|▏         | 3690/225000 [15:33:03<622:26:10, 10.13s/it]
  2%|▏         | 3691/225000 [15:33:10<565:07:07,  9.19s/it]
  2%|▏         | 3692/225000 [15:33:18<544:33:08,  8.86s/it]
  2%|▏         | 3693/225000 [15:33:26<537:45:33,  8.75s/it]
  2%|▏         | 3694/225000 [15:33:44<696:38:21, 11.33s/it]
  2%|▏         | 3695/225000 [15:33:49<580:55:20,  9.45s/it]
  2%|▏         | 3696/225000 [15:33:56<549:00:21,  8.93s/it]
  2%|▏         | 3697/225000 [15:34:06<553:26:26,  9.00s/it]
  2%|▏         | 3698/225000 [15:34:16<585:28:07,  9.52s/it]
  2%|▏         | 3699/225000 [15:34:33<719:02:00, 11.70s/it]
  2%|▏         | 3700/225000 [15:34:42<676:05:26, 11.00s/it]
                                                            
{'loss': '0.8655', 'grad_norm': '2.882', 'learning_rate': '1.848e-05', 'epoch': '0.01644', 'num_input_tokens_seen': 28847416, 'train_runtime': '5.608e+04', 'train_tokens_per_second': '514.4'}

  2%|▏         | 3700/225000 [15:34:43<676:05:26, 11.00s/it]
  2%|▏         | 3701/225000 [15:34:53<670:46:11, 10.91s/it]
  2%|▏         | 3702/225000 [15:35:07<724:17:51, 11.78s/it]
  2%|▏         | 3703/225000 [15:35:20<744:04:38, 12.10s/it]
  2%|▏         | 3704/225000 [15:35:31<732:57:50, 11.92s/it]
  2%|▏         | 3705/225000 [15:35:40<665:35:17, 10.83s/it]
  2%|▏         | 3706/225000 [15:35:45<572:32:18,  9.31s/it]
  2%|▏         | 3707/225000 [15:35:57<621:48:25, 10.12s/it]
  2%|▏         | 3708/225000 [15:36:12<702:00:47, 11.42s/it]
  2%|▏         | 3709/225000 [15:36:16<571:36:30,  9.30s/it]
  2%|▏         | 3710/225000 [15:36:34<735:19:12, 11.96s/it]
  2%|▏         | 3711/225000 [15:36:47<754:58:09, 12.28s/it]
  2%|▏         | 3712/225000 [15:36:58<716:38:21, 11.66s/it]
  2%|▏         | 3713/225000 [15:37:08<692:49:40, 11.27s/it]
  2%|▏         | 3714/225000 [15:37:18<661:53:12, 10.77s/it]
  2%|▏         | 3715/225000 [15:37:25<600:38:01,  9.77s/it]
  2%|▏         | 3716/225000 [15:37:44<767:17:30, 12.48s/it]
  2%|▏         | 3717/225000 [15:37:57<784:20:49, 12.76s/it]
  2%|▏         | 3718/225000 [15:38:06<714:37:11, 11.63s/it]
  2%|▏         | 3719/225000 [15:38:20<748:16:07, 12.17s/it]
  2%|▏         | 3720/225000 [15:38:35<802:22:42, 13.05s/it]
  2%|▏         | 3721/225000 [15:38:43<721:26:00, 11.74s/it]
  2%|▏         | 3722/225000 [15:38:57<747:06:14, 12.15s/it]
  2%|▏         | 3723/225000 [15:39:17<891:23:05, 14.50s/it]
  2%|▏         | 3724/225000 [15:39:30<863:12:31, 14.04s/it]
  2%|▏         | 3725/225000 [15:39:45<880:29:14, 14.32s/it]
  2%|▏         | 3726/225000 [15:40:03<963:29:03, 15.68s/it]
  2%|▏         | 3727/225000 [15:40:16<916:45:09, 14.92s/it]
  2%|▏         | 3728/225000 [15:40:27<840:15:29, 13.67s/it]
  2%|▏         | 3729/225000 [15:40:40<822:24:58, 13.38s/it]
  2%|▏         | 3730/225000 [15:40:47<705:57:23, 11.49s/it]
  2%|▏         | 3731/225000 [15:41:02<767:04:08, 12.48s/it]
  2%|▏         | 3732/225000 [15:41:13<748:25:37, 12.18s/it]
  2%|▏         | 3733/225000 [15:41:24<729:29:36, 11.87s/it]
  2%|▏         | 3734/225000 [15:41:35<702:57:47, 11.44s/it]
  2%|▏         | 3735/225000 [15:41:42<626:16:09, 10.19s/it]
  2%|▏         | 3736/225000 [15:41:57<714:31:33, 11.63s/it]
  2%|▏         | 3737/225000 [15:42:13<801:41:13, 13.04s/it]
  2%|▏         | 3738/225000 [15:42:22<725:39:42, 11.81s/it]
  2%|▏         | 3739/225000 [15:42:31<668:51:22, 10.88s/it]
  2%|▏         | 3740/225000 [15:42:44<702:41:21, 11.43s/it]
  2%|▏         | 3741/225000 [15:42:56<722:06:38, 11.75s/it]
  2%|▏         | 3742/225000 [15:43:03<627:32:39, 10.21s/it]
  2%|▏         | 3743/225000 [15:43:14<642:12:22, 10.45s/it]
  2%|▏         | 3744/225000 [15:43:21<585:01:47,  9.52s/it]
  2%|▏         | 3745/225000 [15:43:35<658:42:13, 10.72s/it]
  2%|▏         | 3746/225000 [15:43:48<701:16:34, 11.41s/it]
  2%|▏         | 3747/225000 [15:43:54<597:52:52,  9.73s/it]
  2%|▏         | 3748/225000 [15:44:01<562:50:33,  9.16s/it]
  2%|▏         | 3749/225000 [15:44:13<603:41:02,  9.82s/it]
  2%|▏         | 3750/225000 [15:44:22<582:31:31,  9.48s/it]
                                                            
{'loss': '0.8923', 'grad_norm': '1.463', 'learning_rate': '1.873e-05', 'epoch': '0.01667', 'num_input_tokens_seen': 29134192, 'train_runtime': '5.666e+04', 'train_tokens_per_second': '514.2'}

  2%|▏         | 3750/225000 [15:44:22<582:31:31,  9.48s/it]
  2%|▏         | 3751/225000 [15:44:29<544:53:30,  8.87s/it]
  2%|▏         | 3752/225000 [15:44:43<633:15:47, 10.30s/it]
  2%|▏         | 3753/225000 [15:44:54<643:47:56, 10.48s/it]
  2%|▏         | 3754/225000 [15:44:59<543:31:42,  8.84s/it]
  2%|▏         | 3755/225000 [15:45:08<555:10:40,  9.03s/it]
  2%|▏         | 3756/225000 [15:45:14<498:32:33,  8.11s/it]
  2%|▏         | 3757/225000 [15:45:22<492:34:37,  8.02s/it]
  2%|▏         | 3758/225000 [15:45:32<527:28:25,  8.58s/it]
  2%|▏         | 3759/225000 [15:45:46<633:00:02, 10.30s/it]
  2%|▏         | 3760/225000 [15:46:04<778:57:23, 12.68s/it]
  2%|▏         | 3761/225000 [15:46:22<873:35:02, 14.21s/it]
  2%|▏         | 3762/225000 [15:46:34<838:12:20, 13.64s/it]
  2%|▏         | 3763/225000 [15:46:42<734:21:47, 11.95s/it]
  2%|▏         | 3764/225000 [15:46:52<689:27:55, 11.22s/it]
  2%|▏         | 3765/225000 [15:47:03<685:41:57, 11.16s/it]
  2%|▏         | 3766/225000 [15:47:26<907:24:42, 14.77s/it]
  2%|▏         | 3767/225000 [15:47:34<785:31:02, 12.78s/it]
  2%|▏         | 3768/225000 [15:47:42<685:47:26, 11.16s/it]
  2%|▏         | 3769/225000 [15:47:57<759:16:09, 12.36s/it]
  2%|▏         | 3770/225000 [15:48:09<762:57:37, 12.42s/it]
  2%|▏         | 3771/225000 [15:48:17<681:51:33, 11.10s/it]
  2%|▏         | 3772/225000 [15:48:29<699:00:51, 11.37s/it]
  2%|▏         | 3773/225000 [15:48:43<737:53:13, 12.01s/it]
  2%|▏         | 3774/225000 [15:48:57<771:34:02, 12.56s/it]
  2%|▏         | 3775/225000 [15:49:06<706:38:21, 11.50s/it]
  2%|▏         | 3776/225000 [15:49:13<633:03:38, 10.30s/it]
  2%|▏         | 3777/225000 [15:49:27<705:09:21, 11.48s/it]
  2%|▏         | 3778/225000 [15:49:39<709:40:58, 11.55s/it]
  2%|▏         | 3779/225000 [15:49:45<607:58:39,  9.89s/it]
  2%|▏         | 3780/225000 [15:49:56<634:41:33, 10.33s/it]
  2%|▏         | 3781/225000 [15:50:15<778:27:38, 12.67s/it]
  2%|▏         | 3782/225000 [15:50:28<792:56:23, 12.90s/it]
  2%|▏         | 3783/225000 [15:50:40<766:46:10, 12.48s/it]
  2%|▏         | 3784/225000 [15:50:54<796:01:02, 12.95s/it]
  2%|▏         | 3785/225000 [15:51:01<692:50:59, 11.28s/it]
  2%|▏         | 3786/225000 [15:51:19<818:20:42, 13.32s/it]
  2%|▏         | 3787/225000 [15:51:33<823:48:01, 13.41s/it]
  2%|▏         | 3788/225000 [15:51:49<873:45:08, 14.22s/it]
  2%|▏         | 3789/225000 [15:52:01<831:56:13, 13.54s/it]
  2%|▏         | 3790/225000 [15:52:12<788:22:06, 12.83s/it]
  2%|▏         | 3791/225000 [15:52:22<746:35:57, 12.15s/it]
  2%|▏         | 3792/225000 [15:52:31<685:03:31, 11.15s/it]
  2%|▏         | 3793/225000 [15:52:40<649:07:55, 10.56s/it]
  2%|▏         | 3794/225000 [15:52:53<687:23:41, 11.19s/it]
  2%|▏         | 3795/225000 [15:53:05<701:56:25, 11.42s/it]
  2%|▏         | 3796/225000 [15:53:14<650:57:33, 10.59s/it]
  2%|▏         | 3797/225000 [15:53:27<697:57:21, 11.36s/it]
  2%|▏         | 3798/225000 [15:53:33<609:33:55,  9.92s/it]
  2%|▏         | 3799/225000 [15:53:44<621:02:06, 10.11s/it]
  2%|▏         | 3800/225000 [15:54:01<749:41:54, 12.20s/it]
                                                            
{'loss': '0.8898', 'grad_norm': '0.937', 'learning_rate': '1.898e-05', 'epoch': '0.01689', 'num_input_tokens_seen': 29428104, 'train_runtime': '5.724e+04', 'train_tokens_per_second': '514.1'}

  2%|▏         | 3800/225000 [15:54:01<749:41:54, 12.20s/it]
  2%|▏         | 3801/225000 [15:54:10<696:38:44, 11.34s/it]
  2%|▏         | 3802/225000 [15:54:18<624:00:28, 10.16s/it]
  2%|▏         | 3803/225000 [15:54:33<714:52:00, 11.63s/it]
  2%|▏         | 3804/225000 [15:54:41<653:23:40, 10.63s/it]
  2%|▏         | 3805/225000 [15:54:55<706:46:42, 11.50s/it]
  2%|▏         | 3806/225000 [15:55:02<629:55:43, 10.25s/it]
  2%|▏         | 3807/225000 [15:55:10<586:59:02,  9.55s/it]
  2%|▏         | 3808/225000 [15:55:28<738:17:51, 12.02s/it]
  2%|▏         | 3809/225000 [15:55:38<702:45:24, 11.44s/it]
  2%|▏         | 3810/225000 [15:55:55<805:12:49, 13.11s/it]
  2%|▏         | 3811/225000 [15:56:05<747:02:26, 12.16s/it]
  2%|▏         | 3812/225000 [15:56:18<772:55:18, 12.58s/it]
  2%|▏         | 3813/225000 [15:56:26<676:16:23, 11.01s/it]
  2%|▏         | 3814/225000 [15:56:34<635:44:33, 10.35s/it]
  2%|▏         | 3815/225000 [15:56:50<736:24:36, 11.99s/it]
  2%|▏         | 3816/225000 [15:57:01<714:33:21, 11.63s/it]
  2%|▏         | 3817/225000 [15:57:14<737:51:24, 12.01s/it]
  2%|▏         | 3818/225000 [15:57:29<799:56:12, 13.02s/it]
  2%|▏         | 3819/225000 [15:57:41<767:28:23, 12.49s/it]
  2%|▏         | 3820/225000 [15:57:48<678:39:56, 11.05s/it]
  2%|▏         | 3821/225000 [15:58:02<734:00:16, 11.95s/it]
  2%|▏         | 3822/225000 [15:58:17<787:42:46, 12.82s/it]
  2%|▏         | 3823/225000 [15:58:32<817:13:36, 13.30s/it]
  2%|▏         | 3824/225000 [15:58:38<686:29:37, 11.17s/it]
  2%|▏         | 3825/225000 [15:58:52<747:11:24, 12.16s/it]
  2%|▏         | 3826/225000 [15:59:02<705:39:30, 11.49s/it]
  2%|▏         | 3827/225000 [15:59:17<766:31:27, 12.48s/it]
  2%|▏         | 3828/225000 [15:59:30<772:27:43, 12.57s/it]
  2%|▏         | 3829/225000 [15:59:45<814:42:48, 13.26s/it]
  2%|▏         | 3830/225000 [15:59:51<679:08:41, 11.05s/it]
  2%|▏         | 3831/225000 [16:00:04<729:17:36, 11.87s/it]
  2%|▏         | 3832/225000 [16:00:10<619:47:53, 10.09s/it]
  2%|▏         | 3833/225000 [16:00:30<790:32:50, 12.87s/it]
  2%|▏         | 3834/225000 [16:00:51<941:32:58, 15.33s/it]
  2%|▏         | 3835/225000 [16:01:00<834:30:45, 13.58s/it]
  2%|▏         | 3836/225000 [16:01:07<704:25:20, 11.47s/it]
  2%|▏         | 3837/225000 [16:01:18<705:25:40, 11.48s/it]
  2%|▏         | 3838/225000 [16:01:31<727:37:53, 11.84s/it]
  2%|▏         | 3839/225000 [16:01:44<741:25:16, 12.07s/it]
  2%|▏         | 3840/225000 [16:02:00<828:32:13, 13.49s/it]
  2%|▏         | 3841/225000 [16:02:13<806:17:34, 13.12s/it]
  2%|▏         | 3842/225000 [16:02:30<877:49:11, 14.29s/it]
  2%|▏         | 3843/225000 [16:02:43<860:17:43, 14.00s/it]
  2%|▏         | 3844/225000 [16:03:01<941:05:38, 15.32s/it]
  2%|▏         | 3845/225000 [16:03:12<850:29:06, 13.84s/it]
  2%|▏         | 3846/225000 [16:03:20<751:27:02, 12.23s/it]
  2%|▏         | 3847/225000 [16:03:29<693:50:58, 11.29s/it]
  2%|▏         | 3848/225000 [16:03:42<711:32:15, 11.58s/it]
  2%|▏         | 3849/225000 [16:03:54<729:54:19, 11.88s/it]
  2%|▏         | 3850/225000 [16:04:06<726:04:27, 11.82s/it]
                                                            
{'loss': '0.9176', 'grad_norm': '1.386', 'learning_rate': '1.922e-05', 'epoch': '0.01711', 'num_input_tokens_seen': 29721120, 'train_runtime': '5.785e+04', 'train_tokens_per_second': '513.8'}

  2%|▏         | 3850/225000 [16:04:06<726:04:27, 11.82s/it]
  2%|▏         | 3851/225000 [16:04:24<845:04:29, 13.76s/it]
  2%|▏         | 3852/225000 [16:04:33<754:39:59, 12.28s/it]
  2%|▏         | 3853/225000 [16:04:42<688:55:02, 11.21s/it]
  2%|▏         | 3854/225000 [16:04:53<696:08:29, 11.33s/it]
  2%|▏         | 3855/225000 [16:05:03<668:49:51, 10.89s/it]
  2%|▏         | 3856/225000 [16:05:13<641:21:59, 10.44s/it]
  2%|▏         | 3857/225000 [16:05:29<745:04:07, 12.13s/it]
  2%|▏         | 3858/225000 [16:05:37<683:00:09, 11.12s/it]
  2%|▏         | 3859/225000 [16:05:48<671:53:35, 10.94s/it]
  2%|▏         | 3860/225000 [16:05:57<639:03:12, 10.40s/it]
  2%|▏         | 3861/225000 [16:06:17<809:18:33, 13.18s/it]
  2%|▏         | 3862/225000 [16:06:28<782:39:26, 12.74s/it]
  2%|▏         | 3863/225000 [16:06:34<658:20:39, 10.72s/it]
  2%|▏         | 3864/225000 [16:06:45<655:27:36, 10.67s/it]
  2%|▏         | 3865/225000 [16:06:51<569:18:47,  9.27s/it]
  2%|▏         | 3866/225000 [16:07:03<612:46:39,  9.98s/it]
  2%|▏         | 3867/225000 [16:07:11<587:04:35,  9.56s/it]
  2%|▏         | 3868/225000 [16:07:20<568:18:34,  9.25s/it]
  2%|▏         | 3869/225000 [16:07:31<601:20:18,  9.79s/it]
  2%|▏         | 3870/225000 [16:07:46<698:46:29, 11.38s/it]
  2%|▏         | 3871/225000 [16:08:00<742:21:17, 12.09s/it]
  2%|▏         | 3872/225000 [16:08:08<672:42:56, 10.95s/it]
  2%|▏         | 3873/225000 [16:08:18<648:57:24, 10.57s/it]
  2%|▏         | 3874/225000 [16:08:25<584:31:30,  9.52s/it]
  2%|▏         | 3875/225000 [16:08:37<633:34:02, 10.31s/it]
  2%|▏         | 3876/225000 [16:08:49<661:45:04, 10.77s/it]
  2%|▏         | 3877/225000 [16:09:07<807:11:40, 13.14s/it]
  2%|▏         | 3878/225000 [16:09:15<704:51:35, 11.48s/it]
  2%|▏         | 3879/225000 [16:09:23<633:54:22, 10.32s/it]
  2%|▏         | 3880/225000 [16:09:40<757:50:24, 12.34s/it]
  2%|▏         | 3881/225000 [16:09:59<885:23:09, 14.41s/it]
  2%|▏         | 3882/225000 [16:10:13<876:25:25, 14.27s/it]
  2%|▏         | 3883/225000 [16:10:21<765:07:02, 12.46s/it]
  2%|▏         | 3884/225000 [16:10:39<859:55:04, 14.00s/it]
  2%|▏         | 3885/225000 [16:10:45<723:03:54, 11.77s/it]
  2%|▏         | 3886/225000 [16:10:57<727:53:05, 11.85s/it]
  2%|▏         | 3887/225000 [16:11:15<842:18:33, 13.71s/it]
  2%|▏         | 3888/225000 [16:11:26<787:54:38, 12.83s/it]
  2%|▏         | 3889/225000 [16:11:43<858:19:12, 13.97s/it]
  2%|▏         | 3890/225000 [16:11:51<746:02:28, 12.15s/it]
  2%|▏         | 3891/225000 [16:11:56<618:45:04, 10.07s/it]
  2%|▏         | 3892/225000 [16:12:02<543:33:06,  8.85s/it]
  2%|▏         | 3893/225000 [16:12:07<476:59:57,  7.77s/it]
  2%|▏         | 3894/225000 [16:12:19<555:37:13,  9.05s/it]
  2%|▏         | 3895/225000 [16:12:28<563:09:05,  9.17s/it]
  2%|▏         | 3896/225000 [16:12:40<609:35:32,  9.93s/it]
  2%|▏         | 3897/225000 [16:12:56<719:29:23, 11.71s/it]
  2%|▏         | 3898/225000 [16:13:13<816:31:37, 13.29s/it]
  2%|▏         | 3899/225000 [16:13:31<907:14:36, 14.77s/it]
  2%|▏         | 3900/225000 [16:13:39<778:18:51, 12.67s/it]
                                                            
{'loss': '0.8682', 'grad_norm': '1.593', 'learning_rate': '1.948e-05', 'epoch': '0.01733', 'num_input_tokens_seen': 29992848, 'train_runtime': '5.842e+04', 'train_tokens_per_second': '513.4'}

  2%|▏         | 3900/225000 [16:13:39<778:18:51, 12.67s/it]
  2%|▏         | 3901/225000 [16:13:51<770:45:19, 12.55s/it]
  2%|▏         | 3902/225000 [16:14:08<852:01:05, 13.87s/it]
  2%|▏         | 3903/225000 [16:14:25<911:00:59, 14.83s/it]
  2%|▏         | 3904/225000 [16:14:36<839:45:40, 13.67s/it]
  2%|▏         | 3905/225000 [16:14:48<799:03:46, 13.01s/it]
  2%|▏         | 3906/225000 [16:14:54<672:26:00, 10.95s/it]
  2%|▏         | 3907/225000 [16:15:08<731:12:22, 11.91s/it]
  2%|▏         | 3908/225000 [16:15:20<724:09:12, 11.79s/it]
  2%|▏         | 3909/225000 [16:15:30<690:23:59, 11.24s/it]
  2%|▏         | 3910/225000 [16:15:46<778:45:39, 12.68s/it]
  2%|▏         | 3911/225000 [16:15:51<644:16:47, 10.49s/it]
  2%|▏         | 3912/225000 [16:16:12<830:19:12, 13.52s/it]
  2%|▏         | 3913/225000 [16:16:18<692:58:24, 11.28s/it]
  2%|▏         | 3914/225000 [16:16:28<673:17:14, 10.96s/it]
  2%|▏         | 3915/225000 [16:16:35<594:50:56,  9.69s/it]
  2%|▏         | 3916/225000 [16:16:44<598:16:43,  9.74s/it]
  2%|▏         | 3917/225000 [16:17:00<710:43:30, 11.57s/it]
  2%|▏         | 3918/225000 [16:17:15<766:22:07, 12.48s/it]
  2%|▏         | 3919/225000 [16:17:23<678:05:57, 11.04s/it]
  2%|▏         | 3920/225000 [16:17:30<612:25:06,  9.97s/it]
  2%|▏         | 3921/225000 [16:17:38<568:55:26,  9.26s/it]
  2%|▏         | 3922/225000 [16:17:53<677:59:16, 11.04s/it]
  2%|▏         | 3923/225000 [16:17:59<592:22:20,  9.65s/it]
  2%|▏         | 3924/225000 [16:18:10<612:14:33,  9.97s/it]
  2%|▏         | 3925/225000 [16:18:19<594:23:56,  9.68s/it]
  2%|▏         | 3926/225000 [16:18:31<638:31:00, 10.40s/it]
  2%|▏         | 3927/225000 [16:18:53<858:07:00, 13.97s/it]
  2%|▏         | 3928/225000 [16:19:00<728:27:00, 11.86s/it]
  2%|▏         | 3929/225000 [16:19:06<608:36:46,  9.91s/it]
  2%|▏         | 3930/225000 [16:19:17<636:41:31, 10.37s/it]
  2%|▏         | 3931/225000 [16:19:28<647:50:56, 10.55s/it]
  2%|▏         | 3932/225000 [16:19:34<569:42:58,  9.28s/it]
  2%|▏         | 3933/225000 [16:19:51<709:35:31, 11.56s/it]
  2%|▏         | 3934/225000 [16:20:05<747:13:46, 12.17s/it]
  2%|▏         | 3935/225000 [16:20:19<790:15:43, 12.87s/it]
  2%|▏         | 3936/225000 [16:20:29<728:16:21, 11.86s/it]
  2%|▏         | 3937/225000 [16:20:35<622:08:59, 10.13s/it]
  2%|▏         | 3938/225000 [16:20:47<662:36:56, 10.79s/it]
  2%|▏         | 3939/225000 [16:20:58<663:02:24, 10.80s/it]
  2%|▏         | 3940/225000 [16:21:10<682:23:38, 11.11s/it]
  2%|▏         | 3941/225000 [16:21:22<700:33:02, 11.41s/it]
  2%|▏         | 3942/225000 [16:21:32<672:55:55, 10.96s/it]
  2%|▏         | 3943/225000 [16:21:41<642:01:25, 10.46s/it]
  2%|▏         | 3944/225000 [16:21:50<603:42:36,  9.83s/it]
  2%|▏         | 3945/225000 [16:21:58<585:55:39,  9.54s/it]
  2%|▏         | 3946/225000 [16:22:07<574:33:14,  9.36s/it]
  2%|▏         | 3947/225000 [16:22:19<624:06:05, 10.16s/it]
  2%|▏         | 3948/225000 [16:22:32<668:47:26, 10.89s/it]
  2%|▏         | 3949/225000 [16:22:45<701:01:29, 11.42s/it]
  2%|▏         | 3950/225000 [16:22:54<656:59:27, 10.70s/it]
                                                            
{'loss': '0.868', 'grad_norm': '1.73', 'learning_rate': '1.973e-05', 'epoch': '0.01756', 'num_input_tokens_seen': 30257312, 'train_runtime': '5.897e+04', 'train_tokens_per_second': '513.1'}

  2%|▏         | 3950/225000 [16:22:54<656:59:27, 10.70s/it]
  2%|▏         | 3951/225000 [16:23:03<629:35:16, 10.25s/it]
  2%|▏         | 3952/225000 [16:23:13<628:44:59, 10.24s/it]
  2%|▏         | 3953/225000 [16:23:20<564:06:54,  9.19s/it]
  2%|▏         | 3954/225000 [16:23:32<621:29:29, 10.12s/it]
  2%|▏         | 3955/225000 [16:23:43<642:07:15, 10.46s/it]
  2%|▏         | 3956/225000 [16:23:54<641:23:58, 10.45s/it]
  2%|▏         | 3957/225000 [16:24:03<618:09:08, 10.07s/it]
  2%|▏         | 3958/225000 [16:24:13<620:05:08, 10.10s/it]
  2%|▏         | 3959/225000 [16:24:24<627:34:55, 10.22s/it]
  2%|▏         | 3960/225000 [16:24:33<607:57:49,  9.90s/it]
  2%|▏         | 3961/225000 [16:24:56<849:56:16, 13.84s/it]
  2%|▏         | 3962/225000 [16:25:07<796:25:38, 12.97s/it]
  2%|▏         | 3963/225000 [16:25:13<677:06:33, 11.03s/it]
  2%|▏         | 3964/225000 [16:25:26<700:00:10, 11.40s/it]
  2%|▏         | 3965/225000 [16:25:38<715:22:09, 11.65s/it]
  2%|▏         | 3966/225000 [16:26:01<921:25:35, 15.01s/it]
  2%|▏         | 3967/225000 [16:26:04<715:38:45, 11.66s/it]
  2%|▏         | 3968/225000 [16:26:17<731:02:50, 11.91s/it]
  2%|▏         | 3969/225000 [16:26:28<707:27:04, 11.52s/it]
  2%|▏         | 3970/225000 [16:26:33<596:40:02,  9.72s/it]
  2%|▏         | 3971/225000 [16:26:53<785:21:24, 12.79s/it]
  2%|▏         | 3972/225000 [16:27:09<843:59:43, 13.75s/it]
  2%|▏         | 3973/225000 [16:27:19<776:56:21, 12.65s/it]
  2%|▏         | 3974/225000 [16:27:33<792:23:33, 12.91s/it]
  2%|▏         | 3975/225000 [16:27:41<704:17:21, 11.47s/it]
  2%|▏         | 3976/225000 [16:27:47<609:54:47,  9.93s/it]
  2%|▏         | 3977/225000 [16:27:53<534:38:58,  8.71s/it]
  2%|▏         | 3978/225000 [16:28:05<594:34:52,  9.68s/it]
  2%|▏         | 3979/225000 [16:28:30<870:27:20, 14.18s/it]
  2%|▏         | 3980/225000 [16:28:38<755:53:26, 12.31s/it]
  2%|▏         | 3981/225000 [16:28:55<849:51:02, 13.84s/it]
  2%|▏         | 3982/225000 [16:29:01<711:23:07, 11.59s/it]
  2%|▏         | 3983/225000 [16:29:12<693:52:46, 11.30s/it]
  2%|▏         | 3984/225000 [16:29:23<694:59:55, 11.32s/it]
  2%|▏         | 3985/225000 [16:29:34<678:11:57, 11.05s/it]
  2%|▏         | 3986/225000 [16:29:48<739:36:54, 12.05s/it]
  2%|▏         | 3987/225000 [16:30:01<763:27:31, 12.44s/it]
  2%|▏         | 3988/225000 [16:30:08<662:41:52, 10.79s/it]
  2%|▏         | 3989/225000 [16:30:19<655:40:56, 10.68s/it]
  2%|▏         | 3990/225000 [16:30:29<639:05:31, 10.41s/it]
  2%|▏         | 3991/225000 [16:30:38<628:32:50, 10.24s/it]
  2%|▏         | 3992/225000 [16:30:44<538:24:03,  8.77s/it]
  2%|▏         | 3993/225000 [16:30:59<650:21:24, 10.59s/it]
  2%|▏         | 3994/225000 [16:31:11<685:45:57, 11.17s/it]
  2%|▏         | 3995/225000 [16:31:26<754:19:39, 12.29s/it]
  2%|▏         | 3996/225000 [16:31:40<792:12:41, 12.90s/it]
  2%|▏         | 3997/225000 [16:31:48<703:28:20, 11.46s/it]
  2%|▏         | 3998/225000 [16:31:54<602:06:29,  9.81s/it]
  2%|▏         | 3999/225000 [16:32:07<659:36:50, 10.74s/it]
  2%|▏         | 4000/225000 [16:32:26<811:52:50, 13.23s/it]
                                                            
{'loss': '0.886', 'grad_norm': '2.182', 'learning_rate': '1.998e-05', 'epoch': '0.01778', 'num_input_tokens_seen': 30533944, 'train_runtime': '5.955e+04', 'train_tokens_per_second': '512.8'}

  2%|▏         | 4000/225000 [16:32:26<811:52:50, 13.23s/it]
  2%|▏         | 4001/225000 [16:32:39<793:28:35, 12.93s/it]
  2%|▏         | 4002/225000 [16:32:44<650:04:54, 10.59s/it]
  2%|▏         | 4003/225000 [16:32:51<597:30:11,  9.73s/it]
  2%|▏         | 4004/225000 [16:33:01<588:09:29,  9.58s/it]
  2%|▏         | 4005/225000 [16:33:08<553:40:23,  9.02s/it]
  2%|▏         | 4006/225000 [16:33:20<596:44:17,  9.72s/it]
  2%|▏         | 4007/225000 [16:33:30<604:29:16,  9.85s/it]
  2%|▏         | 4008/225000 [16:33:47<743:08:24, 12.11s/it]
  2%|▏         | 4009/225000 [16:34:00<759:49:50, 12.38s/it]
  2%|▏         | 4010/225000 [16:34:14<786:11:57, 12.81s/it]
  2%|▏         | 4011/225000 [16:34:23<723:24:34, 11.78s/it]
  2%|▏         | 4012/225000 [16:34:34<692:33:42, 11.28s/it]
  2%|▏         | 4013/225000 [16:34:43<667:10:38, 10.87s/it]
  2%|▏         | 4014/225000 [16:34:52<632:29:09, 10.30s/it]
  2%|▏         | 4015/225000 [16:35:03<632:53:42, 10.31s/it]
  2%|▏         | 4016/225000 [16:35:20<756:48:05, 12.33s/it]
  2%|▏         | 4017/225000 [16:35:29<695:59:22, 11.34s/it]
  2%|▏         | 4018/225000 [16:35:42<728:40:37, 11.87s/it]
  2%|▏         | 4019/225000 [16:35:55<756:32:53, 12.32s/it]
  2%|▏         | 4020/225000 [16:36:02<647:37:02, 10.55s/it]
  2%|▏         | 4021/225000 [16:36:15<701:47:10, 11.43s/it]
  2%|▏         | 4022/225000 [16:36:20<585:19:41,  9.54s/it]
  2%|▏         | 4023/225000 [16:36:29<563:59:48,  9.19s/it]
  2%|▏         | 4024/225000 [16:36:42<641:01:51, 10.44s/it]
  2%|▏         | 4025/225000 [16:36:49<574:53:28,  9.37s/it]
  2%|▏         | 4026/225000 [16:37:02<650:41:02, 10.60s/it]
  2%|▏         | 4027/225000 [16:37:07<547:34:41,  8.92s/it]
  2%|▏         | 4028/225000 [16:37:25<713:41:41, 11.63s/it]
  2%|▏         | 4029/225000 [16:37:39<750:02:06, 12.22s/it]
  2%|▏         | 4030/225000 [16:37:44<619:37:27, 10.09s/it]
  2%|▏         | 4031/225000 [16:37:55<639:51:14, 10.42s/it]
  2%|▏         | 4032/225000 [16:38:09<699:28:07, 11.40s/it]
  2%|▏         | 4033/225000 [16:38:21<705:30:53, 11.49s/it]
  2%|▏         | 4034/225000 [16:38:31<676:15:55, 11.02s/it]
  2%|▏         | 4035/225000 [16:38:44<724:50:25, 11.81s/it]
  2%|▏         | 4036/225000 [16:38:56<718:47:32, 11.71s/it]
  2%|▏         | 4037/225000 [16:39:07<715:17:56, 11.65s/it]
  2%|▏         | 4038/225000 [16:39:18<700:06:33, 11.41s/it]
  2%|▏         | 4039/225000 [16:39:32<742:51:23, 12.10s/it]
  2%|▏         | 4040/225000 [16:39:36<594:32:50,  9.69s/it]
  2%|▏         | 4041/225000 [16:39:43<552:30:20,  9.00s/it]
  2%|▏         | 4042/225000 [16:40:09<858:00:09, 13.98s/it]
  2%|▏         | 4043/225000 [16:40:16<728:14:37, 11.87s/it]
  2%|▏         | 4044/225000 [16:40:28<732:02:10, 11.93s/it]
  2%|▏         | 4045/225000 [16:40:42<766:02:29, 12.48s/it]
  2%|▏         | 4046/225000 [16:40:49<673:57:33, 10.98s/it]
  2%|▏         | 4047/225000 [16:41:07<793:16:31, 12.92s/it]
  2%|▏         | 4048/225000 [16:41:13<675:22:40, 11.00s/it]
  2%|▏         | 4049/225000 [16:41:24<664:57:31, 10.83s/it]
  2%|▏         | 4050/225000 [16:41:29<572:16:11,  9.32s/it]
                                                            
{'loss': '0.8541', 'grad_norm': '2', 'learning_rate': '2.023e-05', 'epoch': '0.018', 'num_input_tokens_seen': 30800816, 'train_runtime': '6.009e+04', 'train_tokens_per_second': '512.6'}

  2%|▏         | 4050/225000 [16:41:29<572:16:11,  9.32s/it]
  2%|▏         | 4051/225000 [16:41:44<664:32:35, 10.83s/it]
  2%|▏         | 4052/225000 [16:41:48<553:06:38,  9.01s/it]
  2%|▏         | 4053/225000 [16:42:02<641:30:07, 10.45s/it]
  2%|▏         | 4054/225000 [16:42:16<705:19:51, 11.49s/it]
  2%|▏         | 4055/225000 [16:42:23<622:09:37, 10.14s/it]
  2%|▏         | 4056/225000 [16:42:41<767:01:08, 12.50s/it]
  2%|▏         | 4057/225000 [16:42:53<750:41:29, 12.23s/it]
  2%|▏         | 4058/225000 [16:42:59<648:31:05, 10.57s/it]
  2%|▏         | 4059/225000 [16:43:15<749:10:05, 12.21s/it]
  2%|▏         | 4060/225000 [16:43:26<717:25:50, 11.69s/it]
  2%|▏         | 4061/225000 [16:43:43<811:07:57, 13.22s/it]
  2%|▏         | 4062/225000 [16:44:00<885:11:59, 14.42s/it]
  2%|▏         | 4063/225000 [16:44:05<716:01:56, 11.67s/it]
  2%|▏         | 4064/225000 [16:44:16<691:15:00, 11.26s/it]
  2%|▏         | 4065/225000 [16:44:25<663:56:21, 10.82s/it]
  2%|▏         | 4066/225000 [16:44:33<598:24:03,  9.75s/it]
  2%|▏         | 4067/225000 [16:44:38<519:33:46,  8.47s/it]
  2%|▏         | 4068/225000 [16:44:51<596:02:09,  9.71s/it]
  2%|▏         | 4069/225000 [16:45:02<625:09:22, 10.19s/it]
  2%|▏         | 4070/225000 [16:45:13<631:52:00, 10.30s/it]
  2%|▏         | 4071/225000 [16:45:27<703:53:50, 11.47s/it]
  2%|▏         | 4072/225000 [16:45:34<623:42:17, 10.16s/it]
  2%|▏         | 4073/225000 [16:45:42<577:57:31,  9.42s/it]
  2%|▏         | 4074/225000 [16:45:58<706:46:20, 11.52s/it]
  2%|▏         | 4075/225000 [16:46:05<632:28:29, 10.31s/it]
  2%|▏         | 4076/225000 [16:46:15<613:57:19, 10.00s/it]
  2%|▏         | 4077/225000 [16:46:32<751:57:07, 12.25s/it]
  2%|▏         | 4078/225000 [16:46:46<771:32:36, 12.57s/it]
  2%|▏         | 4079/225000 [16:46:55<706:19:06, 11.51s/it]
  2%|▏         | 4080/225000 [16:47:02<640:18:42, 10.43s/it]
  2%|▏         | 4081/225000 [16:47:15<685:06:15, 11.16s/it]
  2%|▏         | 4082/225000 [16:47:27<691:52:45, 11.27s/it]
  2%|▏         | 4083/225000 [16:47:36<649:06:22, 10.58s/it]
  2%|▏         | 4084/225000 [16:47:42<572:34:17,  9.33s/it]
  2%|▏         | 4085/225000 [16:47:53<603:33:47,  9.84s/it]
  2%|▏         | 4086/225000 [16:48:06<652:04:52, 10.63s/it]
  2%|▏         | 4087/225000 [16:48:21<740:21:55, 12.06s/it]
  2%|▏         | 4088/225000 [16:48:34<757:41:44, 12.35s/it]
  2%|▏         | 4089/225000 [16:48:48<777:42:36, 12.67s/it]
  2%|▏         | 4090/225000 [16:48:57<724:40:10, 11.81s/it]
  2%|▏         | 4091/225000 [16:49:09<713:13:10, 11.62s/it]
  2%|▏         | 4092/225000 [16:49:30<887:05:17, 14.46s/it]
  2%|▏         | 4093/225000 [16:49:45<895:19:36, 14.59s/it]
  2%|▏         | 4094/225000 [16:49:55<810:23:13, 13.21s/it]
  2%|▏         | 4095/225000 [16:50:10<843:06:23, 13.74s/it]
  2%|▏         | 4096/225000 [16:50:19<766:49:10, 12.50s/it]
  2%|▏         | 4097/225000 [16:50:28<692:06:39, 11.28s/it]
  2%|▏         | 4098/225000 [16:50:34<607:19:56,  9.90s/it]
  2%|▏         | 4099/225000 [16:50:40<526:50:56,  8.59s/it]
  2%|▏         | 4100/225000 [16:50:53<617:31:39, 10.06s/it]
                                                            
{'loss': '0.8531', 'grad_norm': '1.464', 'learning_rate': '2.047e-05', 'epoch': '0.01822', 'num_input_tokens_seen': 31075896, 'train_runtime': '6.065e+04', 'train_tokens_per_second': '512.3'}

  2%|▏         | 4100/225000 [16:50:53<617:31:39, 10.06s/it]
  2%|▏         | 4101/225000 [16:50:58<519:10:31,  8.46s/it]
  2%|▏         | 4102/225000 [16:51:06<508:58:07,  8.29s/it]
  2%|▏         | 4103/225000 [16:51:14<505:51:47,  8.24s/it]
  2%|▏         | 4104/225000 [16:51:21<473:45:57,  7.72s/it]
  2%|▏         | 4105/225000 [16:51:37<629:49:02, 10.26s/it]
  2%|▏         | 4106/225000 [16:51:55<775:22:50, 12.64s/it]
  2%|▏         | 4107/225000 [16:52:08<784:26:12, 12.78s/it]
  2%|▏         | 4108/225000 [16:52:27<892:18:51, 14.54s/it]
  2%|▏         | 4109/225000 [16:52:41<882:26:16, 14.38s/it]
  2%|▏         | 4110/225000 [16:52:54<864:31:02, 14.09s/it]
  2%|▏         | 4111/225000 [16:53:08<856:00:50, 13.95s/it]
  2%|▏         | 4112/225000 [16:53:17<766:59:47, 12.50s/it]
  2%|▏         | 4113/225000 [16:53:27<718:25:56, 11.71s/it]
  2%|▏         | 4114/225000 [16:53:36<672:46:24, 10.96s/it]
  2%|▏         | 4115/225000 [16:53:46<662:06:22, 10.79s/it]
  2%|▏         | 4116/225000 [16:54:02<756:40:36, 12.33s/it]
  2%|▏         | 4117/225000 [16:54:10<665:06:53, 10.84s/it]
  2%|▏         | 4118/225000 [16:54:20<653:11:07, 10.65s/it]
  2%|▏         | 4119/225000 [16:54:34<716:38:09, 11.68s/it]
  2%|▏         | 4120/225000 [16:54:41<634:52:22, 10.35s/it]
  2%|▏         | 4121/225000 [16:54:54<685:36:05, 11.17s/it]
  2%|▏         | 4122/225000 [16:55:04<665:08:06, 10.84s/it]
  2%|▏         | 4123/225000 [16:55:18<717:31:24, 11.69s/it]
  2%|▏         | 4124/225000 [16:55:24<613:50:44, 10.00s/it]
  2%|▏         | 4125/225000 [16:55:30<530:25:18,  8.65s/it]
  2%|▏         | 4126/225000 [16:55:38<531:02:24,  8.66s/it]
  2%|▏         | 4127/225000 [16:55:47<531:00:37,  8.65s/it]
  2%|▏         | 4128/225000 [16:56:06<726:40:28, 11.84s/it]
  2%|▏         | 4129/225000 [16:56:15<669:26:22, 10.91s/it]
  2%|▏         | 4130/225000 [16:56:25<661:46:17, 10.79s/it]
  2%|▏         | 4131/225000 [16:56:34<614:25:54, 10.01s/it]
  2%|▏         | 4132/225000 [16:56:46<665:42:17, 10.85s/it]
  2%|▏         | 4133/225000 [16:56:57<669:16:16, 10.91s/it]
  2%|▏         | 4134/225000 [16:57:12<732:23:06, 11.94s/it]
  2%|▏         | 4135/225000 [16:57:22<697:54:24, 11.38s/it]
  2%|▏         | 4136/225000 [16:57:27<579:25:33,  9.44s/it]
  2%|▏         | 4137/225000 [16:57:33<515:40:26,  8.41s/it]
  2%|▏         | 4138/225000 [16:57:41<511:20:58,  8.33s/it]
  2%|▏         | 4139/225000 [16:57:49<513:36:18,  8.37s/it]
  2%|▏         | 4140/225000 [16:58:00<552:45:16,  9.01s/it]
  2%|▏         | 4141/225000 [16:58:16<692:59:18, 11.30s/it]
  2%|▏         | 4142/225000 [16:58:23<602:03:58,  9.81s/it]
  2%|▏         | 4143/225000 [16:58:29<530:25:16,  8.65s/it]
  2%|▏         | 4144/225000 [16:58:36<495:38:11,  8.08s/it]
  2%|▏         | 4145/225000 [16:58:54<693:02:42, 11.30s/it]
  2%|▏         | 4146/225000 [16:59:02<625:34:21, 10.20s/it]
  2%|▏         | 4147/225000 [16:59:11<600:51:25,  9.79s/it]
  2%|▏         | 4148/225000 [16:59:16<510:46:18,  8.33s/it]
  2%|▏         | 4149/225000 [16:59:25<520:31:20,  8.48s/it]
  2%|▏         | 4150/225000 [16:59:33<511:34:54,  8.34s/it]
                                                            
{'loss': '0.8633', 'grad_norm': '1.892', 'learning_rate': '2.072e-05', 'epoch': '0.01844', 'num_input_tokens_seen': 31337000, 'train_runtime': '6.117e+04', 'train_tokens_per_second': '512.3'}

  2%|▏         | 4150/225000 [16:59:33<511:34:54,  8.34s/it]
  2%|▏         | 4151/225000 [16:59:44<561:29:49,  9.15s/it]
  2%|▏         | 4152/225000 [16:59:49<500:55:57,  8.17s/it]
  2%|▏         | 4153/225000 [17:00:03<601:16:06,  9.80s/it]
  2%|▏         | 4154/225000 [17:00:10<551:18:58,  8.99s/it]
  2%|▏         | 4155/225000 [17:00:28<709:16:55, 11.56s/it]
  2%|▏         | 4156/225000 [17:00:38<678:41:19, 11.06s/it]
  2%|▏         | 4157/225000 [17:00:44<599:24:10,  9.77s/it]
  2%|▏         | 4158/225000 [17:00:54<591:53:00,  9.65s/it]
  2%|▏         | 4159/225000 [17:01:06<638:30:03, 10.41s/it]
  2%|▏         | 4160/225000 [17:01:24<782:12:07, 12.75s/it]
  2%|▏         | 4161/225000 [17:01:36<773:52:42, 12.62s/it]
  2%|▏         | 4162/225000 [17:01:49<775:57:10, 12.65s/it]
  2%|▏         | 4163/225000 [17:02:00<749:43:50, 12.22s/it]
  2%|▏         | 4164/225000 [17:02:14<769:44:11, 12.55s/it]
  2%|▏         | 4165/225000 [17:02:22<682:08:36, 11.12s/it]
  2%|▏         | 4166/225000 [17:02:33<685:18:43, 11.17s/it]
  2%|▏         | 4167/225000 [17:02:48<759:53:00, 12.39s/it]
  2%|▏         | 4168/225000 [17:02:58<712:20:07, 11.61s/it]
  2%|▏         | 4169/225000 [17:03:04<612:13:11,  9.98s/it]
  2%|▏         | 4170/225000 [17:03:13<596:37:56,  9.73s/it]
  2%|▏         | 4171/225000 [17:03:25<644:05:38, 10.50s/it]
  2%|▏         | 4172/225000 [17:03:36<646:09:22, 10.53s/it]
  2%|▏         | 4173/225000 [17:03:46<628:06:16, 10.24s/it]
  2%|▏         | 4174/225000 [17:04:00<700:20:28, 11.42s/it]
  2%|▏         | 4175/225000 [17:04:08<638:51:08, 10.41s/it]
  2%|▏         | 4176/225000 [17:04:16<602:22:30,  9.82s/it]
  2%|▏         | 4177/225000 [17:04:37<800:00:37, 13.04s/it]
  2%|▏         | 4178/225000 [17:04:49<785:49:30, 12.81s/it]
  2%|▏         | 4179/225000 [17:04:54<643:23:22, 10.49s/it]
  2%|▏         | 4180/225000 [17:05:08<712:38:47, 11.62s/it]
  2%|▏         | 4181/225000 [17:05:25<795:58:22, 12.98s/it]
  2%|▏         | 4182/225000 [17:05:31<669:51:35, 10.92s/it]
  2%|▏         | 4183/225000 [17:05:39<612:50:02,  9.99s/it]
  2%|▏         | 4184/225000 [17:05:52<677:22:47, 11.04s/it]
  2%|▏         | 4185/225000 [17:05:59<593:13:35,  9.67s/it]
  2%|▏         | 4186/225000 [17:06:06<562:08:16,  9.16s/it]
  2%|▏         | 4187/225000 [17:06:25<727:35:58, 11.86s/it]
  2%|▏         | 4188/225000 [17:06:37<740:35:57, 12.07s/it]
  2%|▏         | 4189/225000 [17:06:46<673:37:53, 10.98s/it]
  2%|▏         | 4190/225000 [17:06:56<659:09:05, 10.75s/it]
  2%|▏         | 4191/225000 [17:07:07<674:22:46, 10.99s/it]
  2%|▏         | 4192/225000 [17:07:21<727:57:40, 11.87s/it]
  2%|▏         | 4193/225000 [17:07:37<790:42:24, 12.89s/it]
  2%|▏         | 4194/225000 [17:07:43<663:58:46, 10.83s/it]
  2%|▏         | 4195/225000 [17:07:58<757:01:51, 12.34s/it]
  2%|▏         | 4196/225000 [17:08:08<698:12:31, 11.38s/it]
  2%|▏         | 4197/225000 [17:08:15<630:45:00, 10.28s/it]
  2%|▏         | 4198/225000 [17:08:25<621:40:59, 10.14s/it]
  2%|▏         | 4199/225000 [17:08:37<648:09:04, 10.57s/it]
  2%|▏         | 4200/225000 [17:08:45<613:16:57, 10.00s/it]
                                                            
{'loss': '0.8666', 'grad_norm': '1.44', 'learning_rate': '2.098e-05', 'epoch': '0.01867', 'num_input_tokens_seen': 31614352, 'train_runtime': '6.173e+04', 'train_tokens_per_second': '512.2'}

  2%|▏         | 4200/225000 [17:08:45<613:16:57, 10.00s/it]
  2%|▏         | 4201/225000 [17:08:59<671:47:34, 10.95s/it]
  2%|▏         | 4202/225000 [17:09:12<718:30:33, 11.71s/it]
  2%|▏         | 4203/225000 [17:09:18<618:42:46, 10.09s/it]
  2%|▏         | 4204/225000 [17:09:32<690:38:33, 11.26s/it]
  2%|▏         | 4205/225000 [17:09:43<681:45:33, 11.12s/it]
  2%|▏         | 4206/225000 [17:09:56<720:15:11, 11.74s/it]
  2%|▏         | 4207/225000 [17:10:04<644:54:49, 10.52s/it]
  2%|▏         | 4208/225000 [17:10:14<634:55:57, 10.35s/it]
  2%|▏         | 4209/225000 [17:10:24<624:07:30, 10.18s/it]
  2%|▏         | 4210/225000 [17:10:46<844:13:15, 13.77s/it]
  2%|▏         | 4211/225000 [17:10:58<823:11:58, 13.42s/it]
  2%|▏         | 4212/225000 [17:11:06<711:39:06, 11.60s/it]
  2%|▏         | 4213/225000 [17:11:15<661:12:24, 10.78s/it]
  2%|▏         | 4214/225000 [17:11:21<574:23:24,  9.37s/it]
  2%|▏         | 4215/225000 [17:11:33<635:21:54, 10.36s/it]
  2%|▏         | 4216/225000 [17:11:46<677:46:03, 11.05s/it]
  2%|▏         | 4217/225000 [17:11:54<624:42:05, 10.19s/it]
  2%|▏         | 4218/225000 [17:12:00<549:48:02,  8.96s/it]
  2%|▏         | 4219/225000 [17:12:13<624:15:09, 10.18s/it]
  2%|▏         | 4220/225000 [17:12:25<642:55:04, 10.48s/it]
  2%|▏         | 4221/225000 [17:12:40<729:55:11, 11.90s/it]
  2%|▏         | 4222/225000 [17:12:54<763:34:02, 12.45s/it]
  2%|▏         | 4223/225000 [17:13:04<718:45:13, 11.72s/it]
  2%|▏         | 4224/225000 [17:13:19<783:48:00, 12.78s/it]
  2%|▏         | 4225/225000 [17:13:33<815:29:06, 13.30s/it]
  2%|▏         | 4226/225000 [17:13:43<756:01:27, 12.33s/it]
  2%|▏         | 4227/225000 [17:13:56<769:14:23, 12.54s/it]
  2%|▏         | 4228/225000 [17:14:02<637:33:42, 10.40s/it]
  2%|▏         | 4229/225000 [17:14:09<581:13:09,  9.48s/it]
  2%|▏         | 4230/225000 [17:14:20<615:09:24, 10.03s/it]
  2%|▏         | 4231/225000 [17:14:28<576:14:57,  9.40s/it]
  2%|▏         | 4232/225000 [17:14:43<666:08:59, 10.86s/it]
  2%|▏         | 4233/225000 [17:15:04<865:39:50, 14.12s/it]
  2%|▏         | 4234/225000 [17:15:15<802:42:48, 13.09s/it]
  2%|▏         | 4235/225000 [17:15:23<702:50:29, 11.46s/it]
  2%|▏         | 4236/225000 [17:15:37<749:52:17, 12.23s/it]
  2%|▏         | 4237/225000 [17:15:44<649:42:17, 10.59s/it]
  2%|▏         | 4238/225000 [17:15:59<738:07:09, 12.04s/it]
  2%|▏         | 4239/225000 [17:16:06<648:18:29, 10.57s/it]
  2%|▏         | 4240/225000 [17:16:13<576:49:22,  9.41s/it]
  2%|▏         | 4241/225000 [17:16:21<549:59:37,  8.97s/it]
  2%|▏         | 4242/225000 [17:16:31<573:51:52,  9.36s/it]
  2%|▏         | 4243/225000 [17:16:40<564:17:27,  9.20s/it]
  2%|▏         | 4244/225000 [17:16:57<718:26:05, 11.72s/it]
  2%|▏         | 4245/225000 [17:17:09<719:24:59, 11.73s/it]
  2%|▏         | 4246/225000 [17:17:14<590:08:29,  9.62s/it]
  2%|▏         | 4247/225000 [17:17:24<599:54:54,  9.78s/it]
  2%|▏         | 4248/225000 [17:17:29<508:19:30,  8.29s/it]
  2%|▏         | 4249/225000 [17:17:39<539:24:28,  8.80s/it]
  2%|▏         | 4250/225000 [17:17:57<708:01:50, 11.55s/it]
                                                            
{'loss': '0.8749', 'grad_norm': '1.489', 'learning_rate': '2.123e-05', 'epoch': '0.01889', 'num_input_tokens_seen': 31881200, 'train_runtime': '6.228e+04', 'train_tokens_per_second': '511.9'}

  2%|▏         | 4250/225000 [17:17:57<708:01:50, 11.55s/it]
  2%|▏         | 4251/225000 [17:18:11<751:23:57, 12.25s/it]
  2%|▏         | 4252/225000 [17:18:17<650:32:54, 10.61s/it]
  2%|▏         | 4253/225000 [17:18:27<630:36:41, 10.28s/it]
  2%|▏         | 4254/225000 [17:18:35<595:27:32,  9.71s/it]
  2%|▏         | 4255/225000 [17:18:41<513:10:31,  8.37s/it]
  2%|▏         | 4256/225000 [17:18:59<694:15:57, 11.32s/it]
  2%|▏         | 4257/225000 [17:19:05<593:44:09,  9.68s/it]
  2%|▏         | 4258/225000 [17:19:20<704:37:57, 11.49s/it]
  2%|▏         | 4259/225000 [17:19:35<754:01:15, 12.30s/it]
  2%|▏         | 4260/225000 [17:19:48<766:24:54, 12.50s/it]
  2%|▏         | 4261/225000 [17:19:56<691:57:34, 11.29s/it]
  2%|▏         | 4262/225000 [17:20:06<661:39:14, 10.79s/it]
  2%|▏         | 4263/225000 [17:20:13<602:09:01,  9.82s/it]
  2%|▏         | 4264/225000 [17:20:26<665:23:17, 10.85s/it]
  2%|▏         | 4265/225000 [17:20:34<610:38:42,  9.96s/it]
  2%|▏         | 4266/225000 [17:20:46<643:04:25, 10.49s/it]
  2%|▏         | 4267/225000 [17:20:58<676:38:21, 11.04s/it]
  2%|▏         | 4268/225000 [17:21:10<694:25:24, 11.33s/it]
  2%|▏         | 4269/225000 [17:21:23<721:12:05, 11.76s/it]
  2%|▏         | 4270/225000 [17:21:40<811:34:59, 13.24s/it]
  2%|▏         | 4271/225000 [17:21:50<746:30:14, 12.18s/it]
  2%|▏         | 4272/225000 [17:22:01<732:33:53, 11.95s/it]
  2%|▏         | 4273/225000 [17:22:14<745:55:08, 12.17s/it]
  2%|▏         | 4274/225000 [17:22:22<672:28:13, 10.97s/it]
  2%|▏         | 4275/225000 [17:22:32<666:37:58, 10.87s/it]
  2%|▏         | 4276/225000 [17:22:46<722:33:49, 11.78s/it]
  2%|▏         | 4277/225000 [17:23:00<760:23:56, 12.40s/it]
  2%|▏         | 4278/225000 [17:23:13<759:25:34, 12.39s/it]
  2%|▏         | 4279/225000 [17:23:22<709:00:34, 11.56s/it]
  2%|▏         | 4280/225000 [17:23:41<849:43:53, 13.86s/it]
  2%|▏         | 4281/225000 [17:23:57<876:07:51, 14.29s/it]
  2%|▏         | 4282/225000 [17:24:06<788:13:19, 12.86s/it]
  2%|▏         | 4283/225000 [17:24:14<690:00:44, 11.25s/it]
  2%|▏         | 4284/225000 [17:24:23<650:15:50, 10.61s/it]
  2%|▏         | 4285/225000 [17:24:29<562:36:11,  9.18s/it]
  2%|▏         | 4286/225000 [17:24:36<526:35:24,  8.59s/it]
  2%|▏         | 4287/225000 [17:24:51<638:59:47, 10.42s/it]
  2%|▏         | 4288/225000 [17:25:11<823:42:10, 13.44s/it]
  2%|▏         | 4289/225000 [17:25:20<736:59:53, 12.02s/it]
  2%|▏         | 4290/225000 [17:25:30<696:43:51, 11.36s/it]
  2%|▏         | 4291/225000 [17:25:37<628:33:09, 10.25s/it]
  2%|▏         | 4292/225000 [17:25:47<610:10:35,  9.95s/it]
  2%|▏         | 4293/225000 [17:26:03<733:14:53, 11.96s/it]
  2%|▏         | 4294/225000 [17:26:15<731:38:00, 11.93s/it]
  2%|▏         | 4295/225000 [17:26:24<673:53:42, 10.99s/it]
  2%|▏         | 4296/225000 [17:26:31<599:26:08,  9.78s/it]
  2%|▏         | 4297/225000 [17:26:38<546:55:12,  8.92s/it]
  2%|▏         | 4298/225000 [17:26:51<626:25:22, 10.22s/it]
  2%|▏         | 4299/225000 [17:27:04<671:59:50, 10.96s/it]
  2%|▏         | 4300/225000 [17:27:15<682:18:38, 11.13s/it]
                                                            
{'loss': '0.8354', 'grad_norm': '2.095', 'learning_rate': '2.148e-05', 'epoch': '0.01911', 'num_input_tokens_seen': 32156040, 'train_runtime': '6.284e+04', 'train_tokens_per_second': '511.7'}

  2%|▏         | 4300/225000 [17:27:15<682:18:38, 11.13s/it]
  2%|▏         | 4301/225000 [17:27:26<679:24:08, 11.08s/it]
  2%|▏         | 4302/225000 [17:27:36<650:20:16, 10.61s/it]
  2%|▏         | 4303/225000 [17:27:46<653:37:48, 10.66s/it]
  2%|▏         | 4304/225000 [17:28:09<881:26:57, 14.38s/it]
  2%|▏         | 4305/225000 [17:28:19<784:53:56, 12.80s/it]
  2%|▏         | 4306/225000 [17:28:31<770:03:37, 12.56s/it]
  2%|▏         | 4307/225000 [17:28:43<764:45:05, 12.47s/it]
  2%|▏         | 4308/225000 [17:28:51<678:04:59, 11.06s/it]
  2%|▏         | 4309/225000 [17:29:00<648:15:48, 10.57s/it]
  2%|▏         | 4310/225000 [17:29:12<680:24:10, 11.10s/it]
  2%|▏         | 4311/225000 [17:29:24<698:27:29, 11.39s/it]
  2%|▏         | 4312/225000 [17:29:39<753:19:05, 12.29s/it]
  2%|▏         | 4313/225000 [17:29:50<732:04:04, 11.94s/it]
  2%|▏         | 4314/225000 [17:29:59<669:29:41, 10.92s/it]
  2%|▏         | 4315/225000 [17:30:12<712:51:48, 11.63s/it]
  2%|▏         | 4316/225000 [17:30:17<586:47:17,  9.57s/it]
  2%|▏         | 4317/225000 [17:30:26<584:24:23,  9.53s/it]
  2%|▏         | 4318/225000 [17:30:38<634:41:13, 10.35s/it]
  2%|▏         | 4319/225000 [17:30:52<691:43:40, 11.28s/it]
  2%|▏         | 4320/225000 [17:31:02<670:38:35, 10.94s/it]
  2%|▏         | 4321/225000 [17:31:09<593:20:25,  9.68s/it]
  2%|▏         | 4322/225000 [17:31:18<581:14:28,  9.48s/it]
  2%|▏         | 4323/225000 [17:31:31<648:57:05, 10.59s/it]
  2%|▏         | 4324/225000 [17:31:39<598:47:59,  9.77s/it]
  2%|▏         | 4325/225000 [17:31:46<555:45:19,  9.07s/it]
  2%|▏         | 4326/225000 [17:32:04<727:28:02, 11.87s/it]
  2%|▏         | 4327/225000 [17:32:17<745:04:57, 12.16s/it]
  2%|▏         | 4328/225000 [17:32:27<704:33:08, 11.49s/it]
  2%|▏         | 4329/225000 [17:32:37<663:56:47, 10.83s/it]
  2%|▏         | 4330/225000 [17:32:51<723:51:28, 11.81s/it]
  2%|▏         | 4331/225000 [17:32:59<656:06:52, 10.70s/it]
  2%|▏         | 4332/225000 [17:33:13<717:18:18, 11.70s/it]
  2%|▏         | 4333/225000 [17:33:28<789:47:46, 12.88s/it]
  2%|▏         | 4334/225000 [17:33:38<732:09:52, 11.94s/it]
  2%|▏         | 4335/225000 [17:33:45<637:54:32, 10.41s/it]
  2%|▏         | 4336/225000 [17:33:59<698:09:30, 11.39s/it]
  2%|▏         | 4337/225000 [17:34:05<613:22:53, 10.01s/it]
  2%|▏         | 4338/225000 [17:34:17<640:32:54, 10.45s/it]
  2%|▏         | 4339/225000 [17:34:25<596:50:59,  9.74s/it]
  2%|▏         | 4340/225000 [17:34:40<697:05:01, 11.37s/it]
  2%|▏         | 4341/225000 [17:34:44<564:32:58,  9.21s/it]
  2%|▏         | 4342/225000 [17:34:57<621:33:43, 10.14s/it]
  2%|▏         | 4343/225000 [17:35:06<598:29:25,  9.76s/it]
  2%|▏         | 4344/225000 [17:35:14<580:24:03,  9.47s/it]
  2%|▏         | 4345/225000 [17:35:25<604:59:31,  9.87s/it]
  2%|▏         | 4346/225000 [17:35:30<510:51:10,  8.33s/it]
  2%|▏         | 4347/225000 [17:35:45<626:58:58, 10.23s/it]
  2%|▏         | 4348/225000 [17:35:57<663:30:44, 10.83s/it]
  2%|▏         | 4349/225000 [17:36:11<733:58:27, 11.98s/it]
  2%|▏         | 4350/225000 [17:36:21<694:18:17, 11.33s/it]
                                                            
{'loss': '0.8694', 'grad_norm': '2.152', 'learning_rate': '2.172e-05', 'epoch': '0.01933', 'num_input_tokens_seen': 32427944, 'train_runtime': '6.338e+04', 'train_tokens_per_second': '511.6'}

  2%|▏         | 4350/225000 [17:36:21<694:18:17, 11.33s/it]
  2%|▏         | 4351/225000 [17:36:37<772:06:29, 12.60s/it]
  2%|▏         | 4352/225000 [17:36:48<753:49:40, 12.30s/it]
  2%|▏         | 4353/225000 [17:37:04<811:51:44, 13.25s/it]
  2%|▏         | 4354/225000 [17:37:13<728:42:11, 11.89s/it]
  2%|▏         | 4355/225000 [17:37:32<859:29:41, 14.02s/it]
  2%|▏         | 4356/225000 [17:37:45<843:47:08, 13.77s/it]
  2%|▏         | 4357/225000 [17:37:50<680:42:33, 11.11s/it]
  2%|▏         | 4358/225000 [17:38:00<670:31:14, 10.94s/it]
  2%|▏         | 4359/225000 [17:38:11<662:23:01, 10.81s/it]
  2%|▏         | 4360/225000 [17:38:25<725:06:12, 11.83s/it]
  2%|▏         | 4361/225000 [17:38:32<636:49:59, 10.39s/it]
  2%|▏         | 4362/225000 [17:38:41<605:43:52,  9.88s/it]
  2%|▏         | 4363/225000 [17:38:48<566:11:45,  9.24s/it]
  2%|▏         | 4364/225000 [17:39:01<622:27:49, 10.16s/it]
  2%|▏         | 4365/225000 [17:39:17<738:05:45, 12.04s/it]
  2%|▏         | 4366/225000 [17:39:31<768:00:12, 12.53s/it]
  2%|▏         | 4367/225000 [17:39:41<730:08:02, 11.91s/it]
  2%|▏         | 4368/225000 [17:39:51<681:24:57, 11.12s/it]
  2%|▏         | 4369/225000 [17:39:56<586:19:01,  9.57s/it]
  2%|▏         | 4370/225000 [17:40:09<641:02:01, 10.46s/it]
  2%|▏         | 4371/225000 [17:40:15<565:45:09,  9.23s/it]
  2%|▏         | 4372/225000 [17:40:40<845:57:15, 13.80s/it]
  2%|▏         | 4373/225000 [17:40:50<779:39:31, 12.72s/it]
  2%|▏         | 4374/225000 [17:40:57<670:36:38, 10.94s/it]
  2%|▏         | 4375/225000 [17:41:06<639:01:03, 10.43s/it]
  2%|▏         | 4376/225000 [17:41:20<710:52:23, 11.60s/it]
  2%|▏         | 4377/225000 [17:41:33<721:51:59, 11.78s/it]
  2%|▏         | 4378/225000 [17:41:41<653:51:09, 10.67s/it]
  2%|▏         | 4379/225000 [17:41:52<670:29:45, 10.94s/it]
  2%|▏         | 4380/225000 [17:42:02<649:46:26, 10.60s/it]
  2%|▏         | 4381/225000 [17:42:15<699:48:16, 11.42s/it]
  2%|▏         | 4382/225000 [17:42:30<760:42:37, 12.41s/it]
  2%|▏         | 4383/225000 [17:42:35<614:53:28, 10.03s/it]
  2%|▏         | 4384/225000 [17:42:42<572:34:35,  9.34s/it]
  2%|▏         | 4385/225000 [17:42:51<561:40:51,  9.17s/it]
  2%|▏         | 4386/225000 [17:43:06<658:50:30, 10.75s/it]
  2%|▏         | 4387/225000 [17:43:11<555:55:11,  9.07s/it]
  2%|▏         | 4388/225000 [17:43:23<624:02:30, 10.18s/it]
  2%|▏         | 4389/225000 [17:43:32<596:53:07,  9.74s/it]
  2%|▏         | 4390/225000 [17:43:40<568:42:39,  9.28s/it]
  2%|▏         | 4391/225000 [17:43:51<585:16:35,  9.55s/it]
  2%|▏         | 4392/225000 [17:44:02<622:42:33, 10.16s/it]
  2%|▏         | 4393/225000 [17:44:08<542:25:48,  8.85s/it]
  2%|▏         | 4394/225000 [17:44:16<526:58:53,  8.60s/it]
  2%|▏         | 4395/225000 [17:44:33<679:23:01, 11.09s/it]
  2%|▏         | 4396/225000 [17:44:53<837:40:24, 13.67s/it]
  2%|▏         | 4397/225000 [17:44:59<698:08:16, 11.39s/it]
  2%|▏         | 4398/225000 [17:45:17<828:36:37, 13.52s/it]
  2%|▏         | 4399/225000 [17:45:23<683:41:41, 11.16s/it]
  2%|▏         | 4400/225000 [17:45:33<670:46:14, 10.95s/it]
                                                            
{'loss': '0.8726', 'grad_norm': '1.861', 'learning_rate': '2.198e-05', 'epoch': '0.01956', 'num_input_tokens_seen': 32696248, 'train_runtime': '6.393e+04', 'train_tokens_per_second': '511.4'}

  2%|▏         | 4400/225000 [17:45:33<670:46:14, 10.95s/it]
  2%|▏         | 4401/225000 [17:45:50<784:34:18, 12.80s/it]
  2%|▏         | 4402/225000 [17:46:03<784:54:08, 12.81s/it]
  2%|▏         | 4403/225000 [17:46:12<715:46:59, 11.68s/it]
  2%|▏         | 4404/225000 [17:46:24<714:19:14, 11.66s/it]
  2%|▏         | 4405/225000 [17:46:35<707:05:54, 11.54s/it]
  2%|▏         | 4406/225000 [17:46:48<740:04:51, 12.08s/it]
  2%|▏         | 4407/225000 [17:46:54<623:39:31, 10.18s/it]
  2%|▏         | 4408/225000 [17:47:03<591:59:48,  9.66s/it]
  2%|▏         | 4409/225000 [17:47:10<557:56:51,  9.11s/it]
  2%|▏         | 4410/225000 [17:47:22<609:30:23,  9.95s/it]
  2%|▏         | 4411/225000 [17:47:38<705:25:14, 11.51s/it]
  2%|▏         | 4412/225000 [17:47:57<851:00:55, 13.89s/it]
  2%|▏         | 4413/225000 [17:48:11<858:33:15, 14.01s/it]
  2%|▏         | 4414/225000 [17:48:21<780:48:34, 12.74s/it]
  2%|▏         | 4415/225000 [17:48:31<720:59:42, 11.77s/it]
  2%|▏         | 4416/225000 [17:48:40<671:43:20, 10.96s/it]
  2%|▏         | 4417/225000 [17:48:52<696:03:20, 11.36s/it]
  2%|▏         | 4418/225000 [17:49:06<743:53:29, 12.14s/it]
  2%|▏         | 4419/225000 [17:49:21<800:14:44, 13.06s/it]
  2%|▏         | 4420/225000 [17:49:29<700:23:40, 11.43s/it]
  2%|▏         | 4421/225000 [17:49:34<591:42:59,  9.66s/it]
  2%|▏         | 4422/225000 [17:49:42<550:26:27,  8.98s/it]
  2%|▏         | 4423/225000 [17:49:48<497:22:46,  8.12s/it]
  2%|▏         | 4424/225000 [17:50:02<608:13:46,  9.93s/it]
  2%|▏         | 4425/225000 [17:50:08<533:20:22,  8.70s/it]
  2%|▏         | 4426/225000 [17:50:13<465:28:17,  7.60s/it]
  2%|▏         | 4427/225000 [17:50:24<529:15:35,  8.64s/it]
  2%|▏         | 4428/225000 [17:50:33<546:49:31,  8.92s/it]
  2%|▏         | 4429/225000 [17:50:39<489:30:25,  7.99s/it]
  2%|▏         | 4430/225000 [17:50:45<455:42:00,  7.44s/it]
  2%|▏         | 4431/225000 [17:51:04<662:15:51, 10.81s/it]
  2%|▏         | 4432/225000 [17:51:13<631:26:47, 10.31s/it]
  2%|▏         | 4433/225000 [17:51:25<666:49:08, 10.88s/it]
  2%|▏         | 4434/225000 [17:51:35<651:30:02, 10.63s/it]
  2%|▏         | 4435/225000 [17:51:44<615:41:08, 10.05s/it]
  2%|▏         | 4436/225000 [17:51:55<636:33:39, 10.39s/it]
  2%|▏         | 4437/225000 [17:52:02<563:27:49,  9.20s/it]
  2%|▏         | 4438/225000 [17:52:14<614:04:24, 10.02s/it]
  2%|▏         | 4439/225000 [17:52:22<588:50:31,  9.61s/it]
  2%|▏         | 4440/225000 [17:52:29<543:14:01,  8.87s/it]
  2%|▏         | 4441/225000 [17:52:35<489:42:16,  7.99s/it]
  2%|▏         | 4442/225000 [17:52:41<451:35:11,  7.37s/it]
  2%|▏         | 4443/225000 [17:52:55<574:38:55,  9.38s/it]
  2%|▏         | 4444/225000 [17:53:01<503:08:40,  8.21s/it]
  2%|▏         | 4445/225000 [17:53:17<650:33:04, 10.62s/it]
  2%|▏         | 4446/225000 [17:53:26<617:49:01, 10.08s/it]
  2%|▏         | 4447/225000 [17:53:41<714:36:00, 11.66s/it]
  2%|▏         | 4448/225000 [17:53:45<576:34:19,  9.41s/it]
  2%|▏         | 4449/225000 [17:53:57<619:41:04, 10.11s/it]
  2%|▏         | 4450/225000 [17:54:12<700:49:42, 11.44s/it]
                                                            
{'loss': '0.8048', 'grad_norm': '1.332', 'learning_rate': '2.223e-05', 'epoch': '0.01978', 'num_input_tokens_seen': 32955736, 'train_runtime': '6.445e+04', 'train_tokens_per_second': '511.3'}

  2%|▏         | 4450/225000 [17:54:12<700:49:42, 11.44s/it]
  2%|▏         | 4451/225000 [17:54:21<665:47:21, 10.87s/it]
  2%|▏         | 4452/225000 [17:54:30<629:22:15, 10.27s/it]
  2%|▏         | 4453/225000 [17:54:38<577:36:10,  9.43s/it]
  2%|▏         | 4454/225000 [17:54:44<513:53:23,  8.39s/it]
  2%|▏         | 4455/225000 [17:54:49<454:43:34,  7.42s/it]
  2%|▏         | 4456/225000 [17:55:01<535:45:30,  8.75s/it]
  2%|▏         | 4457/225000 [17:55:11<560:28:35,  9.15s/it]
  2%|▏         | 4458/225000 [17:55:18<532:41:31,  8.70s/it]
  2%|▏         | 4459/225000 [17:55:30<579:05:00,  9.45s/it]
  2%|▏         | 4460/225000 [17:55:37<542:21:27,  8.85s/it]
  2%|▏         | 4461/225000 [17:55:48<587:53:59,  9.60s/it]
  2%|▏         | 4462/225000 [17:56:03<675:47:28, 11.03s/it]
  2%|▏         | 4463/225000 [17:56:13<655:17:31, 10.70s/it]
  2%|▏         | 4464/225000 [17:56:33<835:01:43, 13.63s/it]
  2%|▏         | 4465/225000 [17:56:43<758:49:27, 12.39s/it]
  2%|▏         | 4466/225000 [17:56:48<638:43:18, 10.43s/it]
  2%|▏         | 4467/225000 [17:56:54<554:13:45,  9.05s/it]
  2%|▏         | 4468/225000 [17:57:04<561:41:07,  9.17s/it]
  2%|▏         | 4469/225000 [17:57:19<673:36:26, 11.00s/it]
  2%|▏         | 4470/225000 [17:57:32<718:35:15, 11.73s/it]
  2%|▏         | 4471/225000 [17:57:38<604:43:55,  9.87s/it]
  2%|▏         | 4472/225000 [17:57:50<637:22:48, 10.40s/it]
  2%|▏         | 4473/225000 [17:58:09<801:02:25, 13.08s/it]
  2%|▏         | 4474/225000 [17:58:19<743:48:00, 12.14s/it]
  2%|▏         | 4475/225000 [17:58:26<657:33:58, 10.73s/it]
  2%|▏         | 4476/225000 [17:58:40<706:33:16, 11.53s/it]
  2%|▏         | 4477/225000 [17:58:50<680:53:19, 11.12s/it]
  2%|▏         | 4478/225000 [17:59:00<666:25:00, 10.88s/it]
  2%|▏         | 4479/225000 [17:59:09<621:54:27, 10.15s/it]
  2%|▏         | 4480/225000 [17:59:22<684:04:26, 11.17s/it]
  2%|▏         | 4481/225000 [17:59:33<676:11:13, 11.04s/it]
  2%|▏         | 4482/225000 [17:59:49<775:31:43, 12.66s/it]
  2%|▏         | 4483/225000 [17:59:58<705:16:51, 11.51s/it]
  2%|▏         | 4484/225000 [18:00:12<754:05:08, 12.31s/it]
  2%|▏         | 4485/225000 [18:00:22<710:09:41, 11.59s/it]
  2%|▏         | 4486/225000 [18:00:39<810:09:53, 13.23s/it]
  2%|▏         | 4487/225000 [18:00:46<682:41:34, 11.15s/it]
  2%|▏         | 4488/225000 [18:00:51<580:39:20,  9.48s/it]
  2%|▏         | 4489/225000 [18:00:57<508:13:02,  8.30s/it]
  2%|▏         | 4490/225000 [18:01:16<716:53:26, 11.70s/it]
  2%|▏         | 4491/225000 [18:01:29<734:02:40, 11.98s/it]
  2%|▏         | 4492/225000 [18:01:41<724:20:36, 11.83s/it]
  2%|▏         | 4493/225000 [18:01:48<646:48:12, 10.56s/it]
  2%|▏         | 4494/225000 [18:01:55<579:47:49,  9.47s/it]
  2%|▏         | 4495/225000 [18:02:05<592:24:54,  9.67s/it]
  2%|▏         | 4496/225000 [18:02:14<580:47:02,  9.48s/it]
  2%|▏         | 4497/225000 [18:02:20<519:40:26,  8.48s/it]
  2%|▏         | 4498/225000 [18:02:34<615:27:29, 10.05s/it]
  2%|▏         | 4499/225000 [18:02:42<570:07:51,  9.31s/it]
  2%|▏         | 4500/225000 [18:02:49<533:21:07,  8.71s/it]
                                                            
{'loss': '0.8279', 'grad_norm': '1.883', 'learning_rate': '2.248e-05', 'epoch': '0.02', 'num_input_tokens_seen': 33209984, 'train_runtime': '6.497e+04', 'train_tokens_per_second': '511.2'}

  2%|▏         | 4500/225000 [18:02:49<533:21:07,  8.71s/it]
  2%|▏         | 4501/225000 [18:02:58<546:24:29,  8.92s/it]
  2%|▏         | 4502/225000 [18:03:11<609:09:21,  9.95s/it]
  2%|▏         | 4503/225000 [18:03:19<573:16:27,  9.36s/it]
  2%|▏         | 4504/225000 [18:03:41<803:52:04, 13.12s/it]
  2%|▏         | 4505/225000 [18:03:47<681:52:37, 11.13s/it]
  2%|▏         | 4506/225000 [18:04:02<756:33:22, 12.35s/it]
  2%|▏         | 4507/225000 [18:04:16<780:25:54, 12.74s/it]
  2%|▏         | 4508/225000 [18:04:26<739:11:41, 12.07s/it]
  2%|▏         | 4509/225000 [18:04:32<613:59:08, 10.02s/it]
  2%|▏         | 4510/225000 [18:04:40<575:05:15,  9.39s/it]
  2%|▏         | 4511/225000 [18:04:53<650:26:30, 10.62s/it]
  2%|▏         | 4512/225000 [18:05:04<654:27:31, 10.69s/it]
  2%|▏         | 4513/225000 [18:05:16<672:56:55, 10.99s/it]
  2%|▏         | 4514/225000 [18:05:25<637:45:51, 10.41s/it]
  2%|▏         | 4515/225000 [18:05:30<544:47:58,  8.90s/it]
  2%|▏         | 4516/225000 [18:05:36<495:10:22,  8.09s/it]
  2%|▏         | 4517/225000 [18:05:42<445:16:19,  7.27s/it]
  2%|▏         | 4518/225000 [18:05:57<597:49:11,  9.76s/it]
  2%|▏         | 4519/225000 [18:06:08<622:46:55, 10.17s/it]
  2%|▏         | 4520/225000 [18:06:15<561:43:47,  9.17s/it]
  2%|▏         | 4521/225000 [18:06:21<491:51:22,  8.03s/it]
  2%|▏         | 4522/225000 [18:06:40<706:31:01, 11.54s/it]
  2%|▏         | 4523/225000 [18:06:46<608:07:32,  9.93s/it]
  2%|▏         | 4524/225000 [18:07:03<726:38:58, 11.86s/it]
  2%|▏         | 4525/225000 [18:07:15<723:50:44, 11.82s/it]
  2%|▏         | 4526/225000 [18:07:31<807:38:15, 13.19s/it]
  2%|▏         | 4527/225000 [18:07:44<797:55:08, 13.03s/it]
  2%|▏         | 4528/225000 [18:07:51<702:28:53, 11.47s/it]
  2%|▏         | 4529/225000 [18:08:07<773:50:42, 12.64s/it]
  2%|▏         | 4530/225000 [18:08:17<724:27:45, 11.83s/it]
  2%|▏         | 4531/225000 [18:08:28<723:56:39, 11.82s/it]
  2%|▏         | 4532/225000 [18:08:33<595:32:05,  9.72s/it]
  2%|▏         | 4533/225000 [18:08:45<624:07:13, 10.19s/it]
  2%|▏         | 4534/225000 [18:08:56<650:40:12, 10.62s/it]
  2%|▏         | 4535/225000 [18:09:06<639:43:19, 10.45s/it]
  2%|▏         | 4536/225000 [18:09:13<579:30:09,  9.46s/it]
  2%|▏         | 4537/225000 [18:09:27<648:50:25, 10.60s/it]
  2%|▏         | 4538/225000 [18:09:35<605:41:43,  9.89s/it]
  2%|▏         | 4539/225000 [18:09:41<537:29:49,  8.78s/it]
  2%|▏         | 4540/225000 [18:09:56<649:57:20, 10.61s/it]
  2%|▏         | 4541/225000 [18:10:06<646:19:42, 10.55s/it]
  2%|▏         | 4542/225000 [18:10:20<693:42:49, 11.33s/it]
  2%|▏         | 4543/225000 [18:10:31<687:00:16, 11.22s/it]
  2%|▏         | 4544/225000 [18:10:42<698:59:55, 11.41s/it]
  2%|▏         | 4545/225000 [18:10:52<662:48:00, 10.82s/it]
  2%|▏         | 4546/225000 [18:11:02<643:58:54, 10.52s/it]
  2%|▏         | 4547/225000 [18:11:08<566:24:46,  9.25s/it]
  2%|▏         | 4548/225000 [18:11:19<605:12:07,  9.88s/it]
  2%|▏         | 4549/225000 [18:11:31<630:36:31, 10.30s/it]
  2%|▏         | 4550/225000 [18:11:35<523:27:54,  8.55s/it]
                                                            
{'loss': '0.8384', 'grad_norm': '2.337', 'learning_rate': '2.273e-05', 'epoch': '0.02022', 'num_input_tokens_seen': 33477608, 'train_runtime': '6.55e+04', 'train_tokens_per_second': '511.1'}

  2%|▏         | 4550/225000 [18:11:35<523:27:54,  8.55s/it]
  2%|▏         | 4551/225000 [18:11:52<678:32:43, 11.08s/it]
  2%|▏         | 4552/225000 [18:12:05<717:34:45, 11.72s/it]
  2%|▏         | 4553/225000 [18:12:11<616:23:53, 10.07s/it]
  2%|▏         | 4554/225000 [18:12:21<616:22:25, 10.07s/it]
  2%|▏         | 4555/225000 [18:12:35<683:06:03, 11.16s/it]
  2%|▏         | 4556/225000 [18:12:46<678:49:06, 11.09s/it]
  2%|▏         | 4557/225000 [18:12:58<695:55:57, 11.37s/it]
  2%|▏         | 4558/225000 [18:13:05<604:17:09,  9.87s/it]
  2%|▏         | 4559/225000 [18:13:18<674:43:34, 11.02s/it]
  2%|▏         | 4560/225000 [18:13:27<631:26:09, 10.31s/it]
  2%|▏         | 4561/225000 [18:13:39<659:30:25, 10.77s/it]
  2%|▏         | 4562/225000 [18:13:46<602:49:37,  9.84s/it]
  2%|▏         | 4563/225000 [18:13:56<590:18:47,  9.64s/it]
  2%|▏         | 4564/225000 [18:14:16<796:23:49, 13.01s/it]
  2%|▏         | 4565/225000 [18:14:24<704:43:50, 11.51s/it]
  2%|▏         | 4566/225000 [18:14:33<644:07:47, 10.52s/it]
  2%|▏         | 4567/225000 [18:14:40<578:57:52,  9.46s/it]
  2%|▏         | 4568/225000 [18:14:47<534:17:52,  8.73s/it]
  2%|▏         | 4569/225000 [18:14:52<478:59:02,  7.82s/it]
  2%|▏         | 4570/225000 [18:15:11<684:03:30, 11.17s/it]
  2%|▏         | 4571/225000 [18:15:20<646:52:09, 10.56s/it]
  2%|▏         | 4572/225000 [18:15:35<726:41:50, 11.87s/it]
  2%|▏         | 4573/225000 [18:15:45<689:54:38, 11.27s/it]
  2%|▏         | 4574/225000 [18:15:56<675:03:21, 11.03s/it]
  2%|▏         | 4575/225000 [18:16:03<600:10:19,  9.80s/it]
  2%|▏         | 4576/225000 [18:16:11<576:58:10,  9.42s/it]
  2%|▏         | 4577/225000 [18:16:20<568:54:22,  9.29s/it]
  2%|▏         | 4578/225000 [18:16:29<556:03:57,  9.08s/it]
  2%|▏         | 4579/225000 [18:16:37<535:28:41,  8.75s/it]
  2%|▏         | 4580/225000 [18:16:48<584:53:29,  9.55s/it]
  2%|▏         | 4581/225000 [18:17:00<620:00:37, 10.13s/it]
  2%|▏         | 4582/225000 [18:17:10<625:53:13, 10.22s/it]
  2%|▏         | 4583/225000 [18:17:26<723:00:36, 11.81s/it]
  2%|▏         | 4584/225000 [18:17:40<768:12:29, 12.55s/it]
  2%|▏         | 4585/225000 [18:17:53<784:07:26, 12.81s/it]
  2%|▏         | 4586/225000 [18:18:02<705:27:36, 11.52s/it]
  2%|▏         | 4587/225000 [18:18:11<654:35:45, 10.69s/it]
  2%|▏         | 4588/225000 [18:18:25<718:14:06, 11.73s/it]
  2%|▏         | 4589/225000 [18:18:33<655:11:50, 10.70s/it]
  2%|▏         | 4590/225000 [18:18:39<565:12:49,  9.23s/it]
  2%|▏         | 4591/225000 [18:18:55<699:54:59, 11.43s/it]
  2%|▏         | 4592/225000 [18:19:05<674:38:24, 11.02s/it]
  2%|▏         | 4593/225000 [18:19:13<614:06:50, 10.03s/it]
  2%|▏         | 4594/225000 [18:19:23<615:24:34, 10.05s/it]
  2%|▏         | 4595/225000 [18:19:31<577:21:27,  9.43s/it]
  2%|▏         | 4596/225000 [18:19:43<619:14:45, 10.11s/it]
  2%|▏         | 4597/225000 [18:19:59<726:25:13, 11.87s/it]
  2%|▏         | 4598/225000 [18:20:09<693:52:09, 11.33s/it]
  2%|▏         | 4599/225000 [18:20:17<623:58:26, 10.19s/it]
  2%|▏         | 4600/225000 [18:20:26<604:57:29,  9.88s/it]
                                                            
{'loss': '0.807', 'grad_norm': '1.727', 'learning_rate': '2.298e-05', 'epoch': '0.02044', 'num_input_tokens_seen': 33733304, 'train_runtime': '6.603e+04', 'train_tokens_per_second': '510.9'}

  2%|▏         | 4600/225000 [18:20:26<604:57:29,  9.88s/it]
  2%|▏         | 4601/225000 [18:20:36<615:51:24, 10.06s/it]
  2%|▏         | 4602/225000 [18:20:49<660:10:53, 10.78s/it]
  2%|▏         | 4603/225000 [18:20:57<617:19:59, 10.08s/it]
  2%|▏         | 4604/225000 [18:21:10<662:22:09, 10.82s/it]
  2%|▏         | 4605/225000 [18:21:21<670:15:24, 10.95s/it]
  2%|▏         | 4606/225000 [18:21:28<598:33:27,  9.78s/it]
  2%|▏         | 4607/225000 [18:21:42<680:19:40, 11.11s/it]
  2%|▏         | 4608/225000 [18:21:47<562:22:55,  9.19s/it]
  2%|▏         | 4609/225000 [18:21:57<580:46:25,  9.49s/it]
  2%|▏         | 4610/225000 [18:22:09<620:07:27, 10.13s/it]
  2%|▏         | 4611/225000 [18:22:14<538:44:30,  8.80s/it]
  2%|▏         | 4612/225000 [18:22:20<484:41:13,  7.92s/it]
  2%|▏         | 4613/225000 [18:22:37<639:28:36, 10.45s/it]
  2%|▏         | 4614/225000 [18:22:47<645:12:39, 10.54s/it]
  2%|▏         | 4615/225000 [18:23:00<686:37:41, 11.22s/it]
  2%|▏         | 4616/225000 [18:23:07<602:59:44,  9.85s/it]
  2%|▏         | 4617/225000 [18:23:19<640:48:22, 10.47s/it]
  2%|▏         | 4618/225000 [18:23:33<715:50:19, 11.69s/it]
  2%|▏         | 4619/225000 [18:23:41<639:05:56, 10.44s/it]
  2%|▏         | 4620/225000 [18:23:54<681:30:25, 11.13s/it]
  2%|▏         | 4621/225000 [18:24:09<756:45:39, 12.36s/it]
  2%|▏         | 4622/225000 [18:24:17<681:09:49, 11.13s/it]
  2%|▏         | 4623/225000 [18:24:24<610:14:42,  9.97s/it]
  2%|▏         | 4624/225000 [18:24:38<671:27:53, 10.97s/it]
  2%|▏         | 4625/225000 [18:24:45<603:05:54,  9.85s/it]
  2%|▏         | 4626/225000 [18:24:58<665:06:39, 10.87s/it]
  2%|▏         | 4627/225000 [18:25:08<653:46:59, 10.68s/it]
  2%|▏         | 4628/225000 [18:25:20<670:52:24, 10.96s/it]
  2%|▏         | 4629/225000 [18:25:34<725:35:49, 11.85s/it]
  2%|▏         | 4630/225000 [18:25:41<635:42:47, 10.39s/it]
  2%|▏         | 4631/225000 [18:25:54<683:05:38, 11.16s/it]
  2%|▏         | 4632/225000 [18:25:58<550:46:39,  9.00s/it]
  2%|▏         | 4633/225000 [18:26:11<629:27:00, 10.28s/it]
  2%|▏         | 4634/225000 [18:26:15<519:15:53,  8.48s/it]
  2%|▏         | 4635/225000 [18:26:27<581:07:02,  9.49s/it]
  2%|▏         | 4636/225000 [18:26:33<509:14:04,  8.32s/it]
  2%|▏         | 4637/225000 [18:26:37<439:06:00,  7.17s/it]
  2%|▏         | 4638/225000 [18:26:51<567:06:54,  9.26s/it]
  2%|▏         | 4639/225000 [18:27:01<579:43:29,  9.47s/it]
  2%|▏         | 4640/225000 [18:27:10<564:08:46,  9.22s/it]
  2%|▏         | 4641/225000 [18:27:17<532:12:21,  8.69s/it]
  2%|▏         | 4642/225000 [18:27:29<589:12:00,  9.63s/it]
  2%|▏         | 4643/225000 [18:27:35<511:31:47,  8.36s/it]
  2%|▏         | 4644/225000 [18:27:46<567:44:36,  9.28s/it]
  2%|▏         | 4645/225000 [18:27:56<589:18:21,  9.63s/it]
  2%|▏         | 4646/225000 [18:28:10<656:39:09, 10.73s/it]
  2%|▏         | 4647/225000 [18:28:30<826:50:51, 13.51s/it]
  2%|▏         | 4648/225000 [18:28:38<737:08:31, 12.04s/it]
  2%|▏         | 4649/225000 [18:28:53<776:34:03, 12.69s/it]
  2%|▏         | 4650/225000 [18:29:01<707:24:03, 11.56s/it]
                                                            
{'loss': '0.8255', 'grad_norm': '1.581', 'learning_rate': '2.323e-05', 'epoch': '0.02067', 'num_input_tokens_seen': 33983992, 'train_runtime': '6.654e+04', 'train_tokens_per_second': '510.7'}

  2%|▏         | 4650/225000 [18:29:01<707:24:03, 11.56s/it]
  2%|▏         | 4651/225000 [18:29:14<716:07:07, 11.70s/it]
  2%|▏         | 4652/225000 [18:29:24<696:55:17, 11.39s/it]
  2%|▏         | 4653/225000 [18:29:33<643:24:29, 10.51s/it]
  2%|▏         | 4654/225000 [18:29:47<714:19:25, 11.67s/it]
  2%|▏         | 4655/225000 [18:29:58<696:56:04, 11.39s/it]
  2%|▏         | 4656/225000 [18:30:09<698:24:02, 11.41s/it]
  2%|▏         | 4657/225000 [18:30:15<601:16:52,  9.82s/it]
  2%|▏         | 4658/225000 [18:30:26<622:08:54, 10.16s/it]
  2%|▏         | 4659/225000 [18:30:35<603:51:25,  9.87s/it]
  2%|▏         | 4660/225000 [18:30:45<592:04:03,  9.67s/it]
  2%|▏         | 4661/225000 [18:30:52<552:39:39,  9.03s/it]
  2%|▏         | 4662/225000 [18:31:07<650:52:04, 10.63s/it]
  2%|▏         | 4663/225000 [18:31:19<686:21:41, 11.21s/it]
  2%|▏         | 4664/225000 [18:31:29<665:26:49, 10.87s/it]
  2%|▏         | 4665/225000 [18:31:43<726:11:26, 11.87s/it]
  2%|▏         | 4666/225000 [18:31:49<616:43:20, 10.08s/it]
  2%|▏         | 4667/225000 [18:31:59<618:29:49, 10.11s/it]
  2%|▏         | 4668/225000 [18:32:10<631:35:21, 10.32s/it]
  2%|▏         | 4669/225000 [18:32:20<617:03:53, 10.08s/it]
  2%|▏         | 4670/225000 [18:32:32<654:57:16, 10.70s/it]
  2%|▏         | 4671/225000 [18:32:37<558:16:54,  9.12s/it]
  2%|▏         | 4672/225000 [18:32:51<648:46:59, 10.60s/it]
  2%|▏         | 4673/225000 [18:33:00<611:30:17,  9.99s/it]
  2%|▏         | 4674/225000 [18:33:08<580:27:08,  9.48s/it]
  2%|▏         | 4675/225000 [18:33:14<510:21:06,  8.34s/it]
  2%|▏         | 4676/225000 [18:33:19<445:42:52,  7.28s/it]
  2%|▏         | 4677/225000 [18:33:25<422:29:53,  6.90s/it]
  2%|▏         | 4678/225000 [18:33:41<586:30:34,  9.58s/it]
  2%|▏         | 4679/225000 [18:33:54<652:20:20, 10.66s/it]
  2%|▏         | 4680/225000 [18:34:06<676:30:49, 11.05s/it]
  2%|▏         | 4681/225000 [18:34:13<609:56:25,  9.97s/it]
  2%|▏         | 4682/225000 [18:34:27<682:50:57, 11.16s/it]
  2%|▏         | 4683/225000 [18:34:39<698:22:08, 11.41s/it]
  2%|▏         | 4684/225000 [18:34:45<603:22:40,  9.86s/it]
  2%|▏         | 4685/225000 [18:34:50<508:53:52,  8.32s/it]
  2%|▏         | 4686/225000 [18:34:58<498:42:53,  8.15s/it]
  2%|▏         | 4687/225000 [18:35:08<536:56:42,  8.77s/it]
  2%|▏         | 4688/225000 [18:35:21<618:44:25, 10.11s/it]
  2%|▏         | 4689/225000 [18:35:36<701:38:40, 11.47s/it]
  2%|▏         | 4690/225000 [18:35:47<697:40:57, 11.40s/it]
  2%|▏         | 4691/225000 [18:35:57<676:26:55, 11.05s/it]
  2%|▏         | 4692/225000 [18:36:12<747:12:05, 12.21s/it]
  2%|▏         | 4693/225000 [18:36:20<667:20:03, 10.90s/it]
  2%|▏         | 4694/225000 [18:36:29<630:54:08, 10.31s/it]
  2%|▏         | 4695/225000 [18:36:39<615:45:30, 10.06s/it]
  2%|▏         | 4696/225000 [18:36:49<628:17:27, 10.27s/it]
  2%|▏         | 4697/225000 [18:37:07<757:36:19, 12.38s/it]
  2%|▏         | 4698/225000 [18:37:23<821:25:55, 13.42s/it]
  2%|▏         | 4699/225000 [18:37:32<747:58:46, 12.22s/it]
  2%|▏         | 4700/225000 [18:37:39<645:01:05, 10.54s/it]
                                                            
{'loss': '0.8441', 'grad_norm': '1.118', 'learning_rate': '2.348e-05', 'epoch': '0.02089', 'num_input_tokens_seen': 34239888, 'train_runtime': '6.706e+04', 'train_tokens_per_second': '510.6'}

  2%|▏         | 4700/225000 [18:37:39<645:01:05, 10.54s/it]
  2%|▏         | 4701/225000 [18:37:49<645:28:22, 10.55s/it]
  2%|▏         | 4702/225000 [18:37:56<572:47:45,  9.36s/it]
  2%|▏         | 4703/225000 [18:38:06<595:41:15,  9.73s/it]
  2%|▏         | 4704/225000 [18:38:12<529:20:26,  8.65s/it]
  2%|▏         | 4705/225000 [18:38:23<558:42:52,  9.13s/it]
  2%|▏         | 4706/225000 [18:38:28<493:37:34,  8.07s/it]
  2%|▏         | 4707/225000 [18:38:39<543:49:48,  8.89s/it]
  2%|▏         | 4708/225000 [18:38:51<595:18:34,  9.73s/it]
  2%|▏         | 4709/225000 [18:39:07<707:40:02, 11.56s/it]
  2%|▏         | 4710/225000 [18:39:17<676:05:29, 11.05s/it]
  2%|▏         | 4711/225000 [18:39:26<648:53:38, 10.60s/it]
  2%|▏         | 4712/225000 [18:39:42<741:30:33, 12.12s/it]
  2%|▏         | 4713/225000 [18:39:52<701:52:58, 11.47s/it]
  2%|▏         | 4714/225000 [18:40:05<730:11:30, 11.93s/it]
  2%|▏         | 4715/225000 [18:40:12<636:07:33, 10.40s/it]
  2%|▏         | 4716/225000 [18:40:22<633:51:57, 10.36s/it]
  2%|▏         | 4717/225000 [18:40:36<706:07:55, 11.54s/it]
  2%|▏         | 4718/225000 [18:40:49<728:46:10, 11.91s/it]
  2%|▏         | 4719/225000 [18:40:55<617:01:02, 10.08s/it]
  2%|▏         | 4720/225000 [18:41:04<604:39:53,  9.88s/it]
  2%|▏         | 4721/225000 [18:41:12<569:34:26,  9.31s/it]
  2%|▏         | 4722/225000 [18:41:26<661:51:39, 10.82s/it]
  2%|▏         | 4723/225000 [18:41:41<727:35:02, 11.89s/it]
  2%|▏         | 4724/225000 [18:41:45<590:51:21,  9.66s/it]
  2%|▏         | 4725/225000 [18:41:51<523:43:32,  8.56s/it]
  2%|▏         | 4726/225000 [18:42:02<564:10:55,  9.22s/it]
  2%|▏         | 4727/225000 [18:42:14<623:34:34, 10.19s/it]
  2%|▏         | 4728/225000 [18:42:21<563:26:42,  9.21s/it]
  2%|▏         | 4729/225000 [18:42:33<613:36:56, 10.03s/it]
  2%|▏         | 4730/225000 [18:42:44<625:54:14, 10.23s/it]
  2%|▏         | 4731/225000 [18:42:54<623:14:32, 10.19s/it]
  2%|▏         | 4732/225000 [18:43:00<545:14:34,  8.91s/it]
  2%|▏         | 4733/225000 [18:43:11<591:39:07,  9.67s/it]
  2%|▏         | 4734/225000 [18:43:19<549:17:14,  8.98s/it]
  2%|▏         | 4735/225000 [18:43:35<677:05:46, 11.07s/it]
  2%|▏         | 4736/225000 [18:43:40<570:05:46,  9.32s/it]
  2%|▏         | 4737/225000 [18:43:54<658:37:32, 10.76s/it]
  2%|▏         | 4738/225000 [18:44:08<709:35:40, 11.60s/it]
  2%|▏         | 4739/225000 [18:44:16<643:01:28, 10.51s/it]
  2%|▏         | 4740/225000 [18:44:21<555:59:52,  9.09s/it]
  2%|▏         | 4741/225000 [18:44:31<559:24:03,  9.14s/it]
  2%|▏         | 4742/225000 [18:44:42<606:16:47,  9.91s/it]
  2%|▏         | 4743/225000 [18:44:51<583:28:03,  9.54s/it]
  2%|▏         | 4744/225000 [18:45:02<618:06:23, 10.10s/it]
  2%|▏         | 4745/225000 [18:45:09<548:59:48,  8.97s/it]
  2%|▏         | 4746/225000 [18:45:20<594:18:34,  9.71s/it]
  2%|▏         | 4747/225000 [18:45:30<597:06:54,  9.76s/it]
  2%|▏         | 4748/225000 [18:45:47<733:12:53, 11.98s/it]
  2%|▏         | 4749/225000 [18:45:53<610:36:31,  9.98s/it]
  2%|▏         | 4750/225000 [18:46:03<620:03:37, 10.13s/it]
                                                            
{'loss': '0.7809', 'grad_norm': '1.602', 'learning_rate': '2.373e-05', 'epoch': '0.02111', 'num_input_tokens_seen': 34495208, 'train_runtime': '6.756e+04', 'train_tokens_per_second': '510.6'}

  2%|▏         | 4750/225000 [18:46:03<620:03:37, 10.13s/it]
  2%|▏         | 4751/225000 [18:46:13<621:56:56, 10.17s/it]
  2%|▏         | 4752/225000 [18:46:34<819:20:43, 13.39s/it]
  2%|▏         | 4753/225000 [18:46:47<813:53:40, 13.30s/it]
  2%|▏         | 4754/225000 [18:46:58<759:55:56, 12.42s/it]
  2%|▏         | 4755/225000 [18:47:07<693:47:04, 11.34s/it]
  2%|▏         | 4756/225000 [18:47:16<654:56:01, 10.71s/it]
  2%|▏         | 4757/225000 [18:47:21<559:09:56,  9.14s/it]
  2%|▏         | 4758/225000 [18:47:30<556:12:58,  9.09s/it]
  2%|▏         | 4759/225000 [18:47:34<462:17:40,  7.56s/it]
  2%|▏         | 4760/225000 [18:48:00<794:21:39, 12.98s/it]
  2%|▏         | 4761/225000 [18:48:13<792:05:26, 12.95s/it]
  2%|▏         | 4762/225000 [18:48:20<683:05:18, 11.17s/it]
  2%|▏         | 4763/225000 [18:48:33<724:45:11, 11.85s/it]
  2%|▏         | 4764/225000 [18:48:48<777:52:59, 12.72s/it]
  2%|▏         | 4765/225000 [18:48:55<667:03:51, 10.90s/it]
  2%|▏         | 4766/225000 [18:49:06<684:21:09, 11.19s/it]
  2%|▏         | 4767/225000 [18:49:23<779:56:36, 12.75s/it]
  2%|▏         | 4768/225000 [18:49:30<673:48:09, 11.01s/it]
  2%|▏         | 4769/225000 [18:49:38<621:41:35, 10.16s/it]
  2%|▏         | 4770/225000 [18:49:53<715:46:37, 11.70s/it]
  2%|▏         | 4771/225000 [18:49:58<595:15:04,  9.73s/it]
  2%|▏         | 4772/225000 [18:50:09<609:17:42,  9.96s/it]
  2%|▏         | 4773/225000 [18:50:14<525:37:20,  8.59s/it]
  2%|▏         | 4774/225000 [18:50:20<477:19:12,  7.80s/it]
  2%|▏         | 4775/225000 [18:50:38<650:52:30, 10.64s/it]
  2%|▏         | 4776/225000 [18:50:45<593:07:32,  9.70s/it]
  2%|▏         | 4777/225000 [18:50:58<662:37:57, 10.83s/it]
  2%|▏         | 4778/225000 [18:51:16<780:48:17, 12.76s/it]
  2%|▏         | 4779/225000 [18:51:23<686:16:57, 11.22s/it]
  2%|▏         | 4780/225000 [18:51:32<647:29:10, 10.58s/it]
  2%|▏         | 4781/225000 [18:51:39<575:37:26,  9.41s/it]
  2%|▏         | 4782/225000 [18:51:55<686:04:54, 11.22s/it]
  2%|▏         | 4783/225000 [18:52:09<737:42:19, 12.06s/it]
  2%|▏         | 4784/225000 [18:52:15<626:47:06, 10.25s/it]
  2%|▏         | 4785/225000 [18:52:24<619:20:32, 10.12s/it]
  2%|▏         | 4786/225000 [18:52:30<533:06:08,  8.72s/it]
  2%|▏         | 4787/225000 [18:52:40<565:22:27,  9.24s/it]
  2%|▏         | 4788/225000 [18:52:47<511:36:09,  8.36s/it]
  2%|▏         | 4789/225000 [18:52:53<467:06:21,  7.64s/it]
  2%|▏         | 4790/225000 [18:53:12<682:37:33, 11.16s/it]
  2%|▏         | 4791/225000 [18:53:19<612:47:18, 10.02s/it]
  2%|▏         | 4792/225000 [18:53:30<619:01:04, 10.12s/it]
  2%|▏         | 4793/225000 [18:53:42<663:27:47, 10.85s/it]
  2%|▏         | 4794/225000 [18:53:50<598:17:24,  9.78s/it]
  2%|▏         | 4795/225000 [18:53:58<580:29:10,  9.49s/it]
  2%|▏         | 4796/225000 [18:54:10<628:16:48, 10.27s/it]
  2%|▏         | 4797/225000 [18:54:15<522:05:36,  8.54s/it]
  2%|▏         | 4798/225000 [18:54:23<510:08:29,  8.34s/it]
  2%|▏         | 4799/225000 [18:54:37<622:26:04, 10.18s/it]
  2%|▏         | 4800/225000 [18:54:49<655:58:34, 10.72s/it]
                                                            
{'loss': '0.8519', 'grad_norm': '6.777', 'learning_rate': '2.398e-05', 'epoch': '0.02133', 'num_input_tokens_seen': 34754784, 'train_runtime': '6.809e+04', 'train_tokens_per_second': '510.4'}

  2%|▏         | 4800/225000 [18:54:49<655:58:34, 10.72s/it]
  2%|▏         | 4801/225000 [18:54:55<557:01:38,  9.11s/it]
  2%|▏         | 4802/225000 [18:55:07<608:30:30,  9.95s/it]
  2%|▏         | 4803/225000 [18:55:16<604:00:56,  9.88s/it]
  2%|▏         | 4804/225000 [18:55:30<682:42:29, 11.16s/it]
  2%|▏         | 4805/225000 [18:55:43<716:21:35, 11.71s/it]
  2%|▏         | 4806/225000 [18:55:54<698:16:35, 11.42s/it]
  2%|▏         | 4807/225000 [18:56:08<744:04:16, 12.17s/it]
  2%|▏         | 4808/225000 [18:56:16<671:50:33, 10.98s/it]
  2%|▏         | 4809/225000 [18:56:23<596:45:57,  9.76s/it]
  2%|▏         | 4810/225000 [18:56:36<653:53:01, 10.69s/it]
  2%|▏         | 4811/225000 [18:56:44<601:03:06,  9.83s/it]
  2%|▏         | 4812/225000 [18:56:53<582:29:18,  9.52s/it]
  2%|▏         | 4813/225000 [18:57:04<613:18:50, 10.03s/it]
  2%|▏         | 4814/225000 [18:57:12<570:11:45,  9.32s/it]
  2%|▏         | 4815/225000 [18:57:29<718:35:43, 11.75s/it]
  2%|▏         | 4816/225000 [18:57:39<681:35:04, 11.14s/it]
  2%|▏         | 4817/225000 [18:57:49<665:13:15, 10.88s/it]
  2%|▏         | 4818/225000 [18:57:58<636:22:27, 10.40s/it]
  2%|▏         | 4819/225000 [18:58:12<703:08:42, 11.50s/it]
  2%|▏         | 4820/225000 [18:58:19<618:19:52, 10.11s/it]
  2%|▏         | 4821/225000 [18:58:28<600:30:32,  9.82s/it]
  2%|▏         | 4822/225000 [18:58:38<590:58:13,  9.66s/it]
  2%|▏         | 4823/225000 [18:58:47<588:52:19,  9.63s/it]
  2%|▏         | 4824/225000 [18:58:54<536:07:14,  8.77s/it]
  2%|▏         | 4825/225000 [18:59:10<675:06:55, 11.04s/it]
  2%|▏         | 4826/225000 [18:59:26<770:06:14, 12.59s/it]
  2%|▏         | 4827/225000 [18:59:36<713:51:03, 11.67s/it]
  2%|▏         | 4828/225000 [18:59:48<712:31:14, 11.65s/it]
  2%|▏         | 4829/225000 [18:59:58<692:06:10, 11.32s/it]
  2%|▏         | 4830/225000 [19:00:12<734:33:42, 12.01s/it]
  2%|▏         | 4831/225000 [19:00:21<691:07:34, 11.30s/it]
  2%|▏         | 4832/225000 [19:00:27<588:32:01,  9.62s/it]
  2%|▏         | 4833/225000 [19:00:35<564:20:51,  9.23s/it]
  2%|▏         | 4834/225000 [19:00:44<553:44:45,  9.05s/it]
  2%|▏         | 4835/225000 [19:00:55<597:33:06,  9.77s/it]
  2%|▏         | 4836/225000 [19:01:03<561:19:28,  9.18s/it]
  2%|▏         | 4837/225000 [19:01:19<688:27:37, 11.26s/it]
  2%|▏         | 4838/225000 [19:01:29<654:47:19, 10.71s/it]
  2%|▏         | 4839/225000 [19:01:40<664:05:37, 10.86s/it]
  2%|▏         | 4840/225000 [19:01:47<592:20:16,  9.69s/it]
  2%|▏         | 4841/225000 [19:01:59<639:44:27, 10.46s/it]
  2%|▏         | 4842/225000 [19:02:05<544:43:55,  8.91s/it]
  2%|▏         | 4843/225000 [19:02:20<670:11:47, 10.96s/it]
  2%|▏         | 4844/225000 [19:02:28<610:53:31,  9.99s/it]
  2%|▏         | 4845/225000 [19:02:35<557:35:59,  9.12s/it]
  2%|▏         | 4846/225000 [19:02:50<662:40:49, 10.84s/it]
  2%|▏         | 4847/225000 [19:02:57<593:28:31,  9.70s/it]
  2%|▏         | 4848/225000 [19:03:03<524:49:06,  8.58s/it]
  2%|▏         | 4849/225000 [19:03:16<613:08:04, 10.03s/it]
  2%|▏         | 4850/225000 [19:03:31<700:21:15, 11.45s/it]
                                                            
{'loss': '0.7763', 'grad_norm': '1.781', 'learning_rate': '2.422e-05', 'epoch': '0.02156', 'num_input_tokens_seen': 35012176, 'train_runtime': '6.861e+04', 'train_tokens_per_second': '510.3'}

  2%|▏         | 4850/225000 [19:03:31<700:21:15, 11.45s/it]
  2%|▏         | 4851/225000 [19:03:37<595:26:38,  9.74s/it]
  2%|▏         | 4852/225000 [19:03:51<671:33:50, 10.98s/it]
  2%|▏         | 4853/225000 [19:03:58<610:37:52,  9.99s/it]
  2%|▏         | 4854/225000 [19:04:10<643:57:59, 10.53s/it]
  2%|▏         | 4855/225000 [19:04:22<666:01:51, 10.89s/it]
  2%|▏         | 4856/225000 [19:04:31<636:49:02, 10.41s/it]
  2%|▏         | 4857/225000 [19:04:43<663:36:43, 10.85s/it]
  2%|▏         | 4858/225000 [19:04:51<604:12:14,  9.88s/it]
  2%|▏         | 4859/225000 [19:04:55<507:59:48,  8.31s/it]
  2%|▏         | 4860/225000 [19:05:05<523:34:25,  8.56s/it]
  2%|▏         | 4861/225000 [19:05:16<585:31:24,  9.58s/it]
  2%|▏         | 4862/225000 [19:05:26<581:52:42,  9.52s/it]
  2%|▏         | 4863/225000 [19:05:43<719:42:07, 11.77s/it]
  2%|▏         | 4864/225000 [19:05:59<800:10:05, 13.09s/it]
  2%|▏         | 4865/225000 [19:06:06<693:27:57, 11.34s/it]
  2%|▏         | 4866/225000 [19:06:15<652:51:56, 10.68s/it]
  2%|▏         | 4867/225000 [19:06:23<591:32:36,  9.67s/it]
  2%|▏         | 4868/225000 [19:06:41<743:49:11, 12.16s/it]
  2%|▏         | 4869/225000 [19:06:46<621:20:45, 10.16s/it]
  2%|▏         | 4870/225000 [19:06:52<546:35:19,  8.94s/it]
  2%|▏         | 4871/225000 [19:07:10<705:34:14, 11.54s/it]
  2%|▏         | 4872/225000 [19:07:22<713:02:11, 11.66s/it]
  2%|▏         | 4873/225000 [19:07:38<801:35:13, 13.11s/it]
  2%|▏         | 4874/225000 [19:07:49<760:50:55, 12.44s/it]
  2%|▏         | 4875/225000 [19:07:58<696:38:27, 11.39s/it]
  2%|▏         | 4876/225000 [19:08:13<754:38:36, 12.34s/it]
  2%|▏         | 4877/225000 [19:08:23<715:38:17, 11.70s/it]
  2%|▏         | 4878/225000 [19:08:28<598:19:35,  9.79s/it]
  2%|▏         | 4879/225000 [19:08:44<707:11:42, 11.57s/it]
  2%|▏         | 4880/225000 [19:08:50<611:14:32, 10.00s/it]
  2%|▏         | 4881/225000 [19:09:00<604:05:05,  9.88s/it]
  2%|▏         | 4882/225000 [19:09:07<554:45:19,  9.07s/it]
  2%|▏         | 4883/225000 [19:09:16<558:14:41,  9.13s/it]
  2%|▏         | 4884/225000 [19:09:29<627:07:20, 10.26s/it]
  2%|▏         | 4885/225000 [19:09:42<669:16:45, 10.95s/it]
  2%|▏         | 4886/225000 [19:09:53<673:41:04, 11.02s/it]
  2%|▏         | 4887/225000 [19:10:03<650:52:25, 10.65s/it]
  2%|▏         | 4888/225000 [19:10:17<713:52:20, 11.68s/it]
  2%|▏         | 4889/225000 [19:10:25<653:45:01, 10.69s/it]
  2%|▏         | 4890/225000 [19:10:30<549:45:42,  8.99s/it]
  2%|▏         | 4891/225000 [19:10:41<582:44:39,  9.53s/it]
  2%|▏         | 4892/225000 [19:10:46<495:06:16,  8.10s/it]
  2%|▏         | 4893/225000 [19:10:52<463:44:40,  7.58s/it]
  2%|▏         | 4894/225000 [19:11:02<512:11:39,  8.38s/it]
  2%|▏         | 4895/225000 [19:11:17<623:47:39, 10.20s/it]
  2%|▏         | 4896/225000 [19:11:31<699:59:35, 11.45s/it]
  2%|▏         | 4897/225000 [19:11:39<640:08:44, 10.47s/it]
  2%|▏         | 4898/225000 [19:11:49<615:18:08, 10.06s/it]
  2%|▏         | 4899/225000 [19:11:54<526:59:47,  8.62s/it]
  2%|▏         | 4900/225000 [19:12:07<619:30:26, 10.13s/it]
                                                            
{'loss': '0.8425', 'grad_norm': '1.688', 'learning_rate': '2.448e-05', 'epoch': '0.02178', 'num_input_tokens_seen': 35267144, 'train_runtime': '6.913e+04', 'train_tokens_per_second': '510.2'}

  2%|▏         | 4900/225000 [19:12:07<619:30:26, 10.13s/it]
  2%|▏         | 4901/225000 [19:12:14<547:15:10,  8.95s/it]
  2%|▏         | 4902/225000 [19:12:20<496:43:50,  8.12s/it]
  2%|▏         | 4903/225000 [19:12:30<530:25:59,  8.68s/it]
  2%|▏         | 4904/225000 [19:12:37<494:47:45,  8.09s/it]
  2%|▏         | 4905/225000 [19:12:50<597:43:34,  9.78s/it]
  2%|▏         | 4906/225000 [19:13:00<588:23:35,  9.62s/it]
  2%|▏         | 4907/225000 [19:13:16<718:44:39, 11.76s/it]
  2%|▏         | 4908/225000 [19:13:24<636:02:23, 10.40s/it]
  2%|▏         | 4909/225000 [19:13:32<600:33:36,  9.82s/it]
  2%|▏         | 4910/225000 [19:13:40<561:07:23,  9.18s/it]
  2%|▏         | 4911/225000 [19:13:53<643:14:38, 10.52s/it]
  2%|▏         | 4912/225000 [19:14:03<631:00:40, 10.32s/it]
  2%|▏         | 4913/225000 [19:14:14<645:05:04, 10.55s/it]
  2%|▏         | 4914/225000 [19:14:20<553:35:12,  9.06s/it]
  2%|▏         | 4915/225000 [19:14:38<716:18:40, 11.72s/it]
  2%|▏         | 4916/225000 [19:14:46<656:53:27, 10.75s/it]
  2%|▏         | 4917/225000 [19:14:57<664:31:20, 10.87s/it]
  2%|▏         | 4918/225000 [19:15:15<786:15:17, 12.86s/it]
  2%|▏         | 4919/225000 [19:15:22<685:19:02, 11.21s/it]
  2%|▏         | 4920/225000 [19:15:36<726:26:39, 11.88s/it]
  2%|▏         | 4921/225000 [19:15:46<693:46:07, 11.35s/it]
  2%|▏         | 4922/225000 [19:15:52<595:27:08,  9.74s/it]
  2%|▏         | 4923/225000 [19:16:03<623:31:47, 10.20s/it]
  2%|▏         | 4924/225000 [19:16:12<596:03:35,  9.75s/it]
  2%|▏         | 4925/225000 [19:16:31<772:11:42, 12.63s/it]
  2%|▏         | 4926/225000 [19:16:40<711:01:07, 11.63s/it]
  2%|▏         | 4927/225000 [19:16:45<573:07:34,  9.38s/it]
  2%|▏         | 4928/225000 [19:16:55<586:25:15,  9.59s/it]
  2%|▏         | 4929/225000 [19:17:02<548:26:37,  8.97s/it]
  2%|▏         | 4930/225000 [19:17:11<544:40:42,  8.91s/it]
  2%|▏         | 4931/225000 [19:17:22<589:37:49,  9.65s/it]
  2%|▏         | 4932/225000 [19:17:30<559:31:04,  9.15s/it]
  2%|▏         | 4933/225000 [19:17:36<503:24:45,  8.24s/it]
  2%|▏         | 4934/225000 [19:17:43<467:43:03,  7.65s/it]
  2%|▏         | 4935/225000 [19:17:49<444:01:13,  7.26s/it]
  2%|▏         | 4936/225000 [19:18:02<552:39:20,  9.04s/it]
  2%|▏         | 4937/225000 [19:18:12<565:21:14,  9.25s/it]
  2%|▏         | 4938/225000 [19:18:26<645:12:27, 10.55s/it]
  2%|▏         | 4939/225000 [19:18:37<659:44:25, 10.79s/it]
  2%|▏         | 4940/225000 [19:18:45<609:47:59,  9.98s/it]
  2%|▏         | 4941/225000 [19:18:57<656:42:00, 10.74s/it]
  2%|▏         | 4942/225000 [19:19:05<605:16:10,  9.90s/it]
  2%|▏         | 4943/225000 [19:19:19<669:31:44, 10.95s/it]
  2%|▏         | 4944/225000 [19:19:25<581:59:48,  9.52s/it]
  2%|▏         | 4945/225000 [19:19:41<702:29:46, 11.49s/it]
  2%|▏         | 4946/225000 [19:19:47<598:35:02,  9.79s/it]
  2%|▏         | 4947/225000 [19:19:54<544:20:27,  8.91s/it]
  2%|▏         | 4948/225000 [19:20:08<636:35:16, 10.41s/it]
  2%|▏         | 4949/225000 [19:20:19<656:01:18, 10.73s/it]
  2%|▏         | 4950/225000 [19:20:37<786:43:42, 12.87s/it]
                                                            
{'loss': '0.7987', 'grad_norm': '1.516', 'learning_rate': '2.473e-05', 'epoch': '0.022', 'num_input_tokens_seen': 35522064, 'train_runtime': '6.964e+04', 'train_tokens_per_second': '510.1'}

  2%|▏         | 4950/225000 [19:20:37<786:43:42, 12.87s/it]
  2%|▏         | 4951/225000 [19:20:44<672:24:56, 11.00s/it]
  2%|▏         | 4952/225000 [19:20:53<646:01:16, 10.57s/it]
  2%|▏         | 4953/225000 [19:21:00<583:40:59,  9.55s/it]
  2%|▏         | 4954/225000 [19:21:10<582:08:51,  9.52s/it]
  2%|▏         | 4955/225000 [19:21:25<691:53:06, 11.32s/it]
  2%|▏         | 4956/225000 [19:21:31<583:35:14,  9.55s/it]
  2%|▏         | 4957/225000 [19:21:45<672:03:17, 11.00s/it]
  2%|▏         | 4958/225000 [19:21:55<658:38:37, 10.78s/it]
  2%|▏         | 4959/225000 [19:22:05<632:28:28, 10.35s/it]
  2%|▏         | 4960/225000 [19:22:20<727:05:18, 11.90s/it]
  2%|▏         | 4961/225000 [19:22:28<656:47:20, 10.75s/it]
  2%|▏         | 4962/225000 [19:22:34<556:04:43,  9.10s/it]
  2%|▏         | 4963/225000 [19:22:53<737:02:38, 12.06s/it]
  2%|▏         | 4964/225000 [19:23:02<684:56:44, 11.21s/it]
  2%|▏         | 4965/225000 [19:23:13<680:08:39, 11.13s/it]
  2%|▏         | 4966/225000 [19:23:19<595:06:52,  9.74s/it]
  2%|▏         | 4967/225000 [19:23:27<567:35:51,  9.29s/it]
  2%|▏         | 4968/225000 [19:23:36<556:10:46,  9.10s/it]
  2%|▏         | 4969/225000 [19:23:48<601:20:17,  9.84s/it]
  2%|▏         | 4970/225000 [19:23:57<600:03:57,  9.82s/it]
  2%|▏         | 4971/225000 [19:24:05<563:54:26,  9.23s/it]
  2%|▏         | 4972/225000 [19:24:21<684:35:46, 11.20s/it]
  2%|▏         | 4973/225000 [19:24:31<658:09:45, 10.77s/it]
  2%|▏         | 4974/225000 [19:24:41<645:50:13, 10.57s/it]
  2%|▏         | 4975/225000 [19:24:54<687:29:25, 11.25s/it]
  2%|▏         | 4976/225000 [19:24:58<561:58:43,  9.20s/it]
  2%|▏         | 4977/225000 [19:25:06<544:08:53,  8.90s/it]
  2%|▏         | 4978/225000 [19:25:13<510:15:20,  8.35s/it]
  2%|▏         | 4979/225000 [19:25:23<522:22:53,  8.55s/it]
  2%|▏         | 4980/225000 [19:25:35<594:10:35,  9.72s/it]
  2%|▏         | 4981/225000 [19:25:42<546:17:27,  8.94s/it]
  2%|▏         | 4982/225000 [19:25:56<635:21:10, 10.40s/it]
  2%|▏         | 4983/225000 [19:26:08<666:24:43, 10.90s/it]
  2%|▏         | 4984/225000 [19:26:17<632:08:00, 10.34s/it]
  2%|▏         | 4985/225000 [19:26:22<539:27:58,  8.83s/it]
  2%|▏         | 4986/225000 [19:26:36<635:48:07, 10.40s/it]
  2%|▏         | 4987/225000 [19:26:48<656:59:32, 10.75s/it]
  2%|▏         | 4988/225000 [19:27:03<739:44:25, 12.10s/it]
  2%|▏         | 4989/225000 [19:27:14<710:20:30, 11.62s/it]
  2%|▏         | 4990/225000 [19:27:22<642:22:57, 10.51s/it]
  2%|▏         | 4991/225000 [19:27:31<626:31:13, 10.25s/it]
  2%|▏         | 4992/225000 [19:27:41<610:55:56, 10.00s/it]
  2%|▏         | 4993/225000 [19:28:01<797:41:42, 13.05s/it]
  2%|▏         | 4994/225000 [19:28:07<662:51:41, 10.85s/it]
  2%|▏         | 4995/225000 [19:28:16<646:31:05, 10.58s/it]
  2%|▏         | 4996/225000 [19:28:31<724:53:26, 11.86s/it]
  2%|▏         | 4997/225000 [19:28:36<588:48:39,  9.63s/it]
  2%|▏         | 4998/225000 [19:28:45<588:26:44,  9.63s/it]
  2%|▏         | 4999/225000 [19:28:52<527:00:49,  8.62s/it]
  2%|▏         | 5000/225000 [19:28:57<469:47:04,  7.69s/it]
                                                            
{'loss': '0.8177', 'grad_norm': '1.975', 'learning_rate': '2.498e-05', 'epoch': '0.02222', 'num_input_tokens_seen': 35769056, 'train_runtime': '7.014e+04', 'train_tokens_per_second': '510'}

  2%|▏         | 5000/225000 [19:28:57<469:47:04,  7.69s/it]
  2%|▏         | 5001/225000 [19:29:05<469:44:05,  7.69s/it]
  2%|▏         | 5002/225000 [19:29:11<445:21:48,  7.29s/it]
  2%|▏         | 5003/225000 [19:29:31<682:27:57, 11.17s/it]
  2%|▏         | 5004/225000 [19:29:38<589:24:43,  9.65s/it]
  2%|▏         | 5005/225000 [19:29:49<627:41:35, 10.27s/it]
  2%|▏         | 5006/225000 [19:29:57<589:56:26,  9.65s/it]
  2%|▏         | 5007/225000 [19:30:11<654:06:19, 10.70s/it]
  2%|▏         | 5008/225000 [19:30:29<796:36:03, 13.04s/it]
  2%|▏         | 5009/225000 [19:30:33<622:40:44, 10.19s/it]
  2%|▏         | 5010/225000 [19:30:43<622:37:25, 10.19s/it]
  2%|▏         | 5011/225000 [19:30:50<574:30:56,  9.40s/it]
  2%|▏         | 5012/225000 [19:30:58<544:37:05,  8.91s/it]
  2%|▏         | 5013/225000 [19:31:14<676:12:13, 11.07s/it]
  2%|▏         | 5014/225000 [19:31:28<721:39:00, 11.81s/it]
  2%|▏         | 5015/225000 [19:31:33<595:10:09,  9.74s/it]
  2%|▏         | 5016/225000 [19:31:40<556:07:25,  9.10s/it]
  2%|▏         | 5017/225000 [19:31:47<517:19:55,  8.47s/it]
  2%|▏         | 5018/225000 [19:31:59<579:10:48,  9.48s/it]
  2%|▏         | 5019/225000 [19:32:09<587:45:58,  9.62s/it]
  2%|▏         | 5020/225000 [19:32:16<541:16:54,  8.86s/it]
  2%|▏         | 5021/225000 [19:32:28<602:16:17,  9.86s/it]
  2%|▏         | 5022/225000 [19:32:37<588:02:23,  9.62s/it]
  2%|▏         | 5023/225000 [19:32:44<538:56:35,  8.82s/it]
  2%|▏         | 5024/225000 [19:32:52<521:24:46,  8.53s/it]
  2%|▏         | 5025/225000 [19:32:59<492:45:46,  8.06s/it]
  2%|▏         | 5026/225000 [19:33:08<513:54:05,  8.41s/it]
  2%|▏         | 5027/225000 [19:33:17<524:48:29,  8.59s/it]
  2%|▏         | 5028/225000 [19:33:31<620:46:42, 10.16s/it]
  2%|▏         | 5029/225000 [19:33:40<586:24:24,  9.60s/it]
  2%|▏         | 5030/225000 [19:33:52<645:46:58, 10.57s/it]
  2%|▏         | 5031/225000 [19:34:03<645:33:01, 10.57s/it]
  2%|▏         | 5032/225000 [19:34:12<618:35:12, 10.12s/it]
  2%|▏         | 5033/225000 [19:34:19<554:26:19,  9.07s/it]
  2%|▏         | 5034/225000 [19:34:33<653:41:52, 10.70s/it]
  2%|▏         | 5035/225000 [19:34:42<624:22:45, 10.22s/it]
  2%|▏         | 5036/225000 [19:34:48<541:27:41,  8.86s/it]
  2%|▏         | 5037/225000 [19:34:54<481:53:53,  7.89s/it]
  2%|▏         | 5038/225000 [19:35:08<602:07:54,  9.85s/it]
  2%|▏         | 5039/225000 [19:35:14<524:51:15,  8.59s/it]
  2%|▏         | 5040/225000 [19:35:19<468:22:38,  7.67s/it]
  2%|▏         | 5041/225000 [19:35:35<609:15:18,  9.97s/it]
  2%|▏         | 5042/225000 [19:35:41<539:44:56,  8.83s/it]
  2%|▏         | 5043/225000 [19:35:47<484:55:34,  7.94s/it]
  2%|▏         | 5044/225000 [19:35:58<555:44:34,  9.10s/it]
  2%|▏         | 5045/225000 [19:36:03<468:28:30,  7.67s/it]
  2%|▏         | 5046/225000 [19:36:15<547:34:30,  8.96s/it]
  2%|▏         | 5047/225000 [19:36:26<596:02:42,  9.76s/it]
  2%|▏         | 5048/225000 [19:36:39<644:53:21, 10.56s/it]
  2%|▏         | 5049/225000 [19:36:51<671:43:59, 10.99s/it]
  2%|▏         | 5050/225000 [19:36:58<597:24:22,  9.78s/it]
                                                            
{'loss': '0.7952', 'grad_norm': '2.035', 'learning_rate': '2.522e-05', 'epoch': '0.02244', 'num_input_tokens_seen': 36018344, 'train_runtime': '7.062e+04', 'train_tokens_per_second': '510'}

  2%|▏         | 5050/225000 [19:36:58<597:24:22,  9.78s/it]
  2%|▏         | 5051/225000 [19:37:05<548:27:18,  8.98s/it]
  2%|▏         | 5052/225000 [19:37:17<605:51:20,  9.92s/it]
  2%|▏         | 5053/225000 [19:37:23<531:25:00,  8.70s/it]
  2%|▏         | 5054/225000 [19:37:29<491:22:26,  8.04s/it]
  2%|▏         | 5055/225000 [19:37:38<508:47:22,  8.33s/it]
  2%|▏         | 5056/225000 [19:37:45<483:40:02,  7.92s/it]
  2%|▏         | 5057/225000 [19:37:54<498:19:07,  8.16s/it]
  2%|▏         | 5058/225000 [19:38:07<593:06:09,  9.71s/it]
  2%|▏         | 5059/225000 [19:38:16<567:07:03,  9.28s/it]
  2%|▏         | 5060/225000 [19:38:24<547:45:46,  8.97s/it]
  2%|▏         | 5061/225000 [19:38:34<577:50:12,  9.46s/it]
  2%|▏         | 5062/225000 [19:38:41<534:52:25,  8.75s/it]
  2%|▏         | 5063/225000 [19:38:50<537:50:54,  8.80s/it]
  2%|▏         | 5064/225000 [19:38:59<538:05:01,  8.81s/it]
  2%|▏         | 5065/225000 [19:39:04<471:59:54,  7.73s/it]
  2%|▏         | 5066/225000 [19:39:15<525:17:53,  8.60s/it]
  2%|▏         | 5067/225000 [19:39:28<606:50:51,  9.93s/it]
  2%|▏         | 5068/225000 [19:39:37<589:05:00,  9.64s/it]
  2%|▏         | 5069/225000 [19:39:44<541:52:40,  8.87s/it]
  2%|▏         | 5070/225000 [19:39:50<480:01:30,  7.86s/it]
  2%|▏         | 5071/225000 [19:39:58<489:20:05,  8.01s/it]
  2%|▏         | 5072/225000 [19:40:13<609:58:59,  9.98s/it]
  2%|▏         | 5073/225000 [19:40:17<503:15:45,  8.24s/it]
  2%|▏         | 5074/225000 [19:40:30<593:36:16,  9.72s/it]
  2%|▏         | 5075/225000 [19:40:44<682:42:23, 11.18s/it]
  2%|▏         | 5076/225000 [19:40:52<607:43:42,  9.95s/it]
  2%|▏         | 5077/225000 [19:41:04<657:46:46, 10.77s/it]
  2%|▏         | 5078/225000 [19:41:17<689:36:31, 11.29s/it]
  2%|▏         | 5079/225000 [19:41:30<722:22:31, 11.82s/it]
  2%|▏         | 5080/225000 [19:41:41<701:55:28, 11.49s/it]
  2%|▏         | 5081/225000 [19:41:47<617:15:07, 10.10s/it]
  2%|▏         | 5082/225000 [19:42:00<659:37:56, 10.80s/it]
  2%|▏         | 5083/225000 [19:42:06<571:26:08,  9.35s/it]
  2%|▏         | 5084/225000 [19:42:12<522:35:38,  8.55s/it]
  2%|▏         | 5085/225000 [19:42:22<546:26:23,  8.95s/it]
  2%|▏         | 5086/225000 [19:42:30<514:09:34,  8.42s/it]
  2%|▏         | 5087/225000 [19:42:35<463:13:22,  7.58s/it]
  2%|▏         | 5088/225000 [19:42:41<432:13:03,  7.08s/it]
  2%|▏         | 5089/225000 [19:42:53<530:08:45,  8.68s/it]
  2%|▏         | 5090/225000 [19:42:58<461:17:11,  7.55s/it]
  2%|▏         | 5091/225000 [19:43:09<521:19:40,  8.53s/it]
  2%|▏         | 5092/225000 [19:43:17<507:55:22,  8.31s/it]
  2%|▏         | 5093/225000 [19:43:31<606:26:07,  9.93s/it]
  2%|▏         | 5094/225000 [19:43:39<578:55:33,  9.48s/it]
  2%|▏         | 5095/225000 [19:43:51<626:57:33, 10.26s/it]
  2%|▏         | 5096/225000 [19:44:01<620:07:49, 10.15s/it]
  2%|▏         | 5097/225000 [19:44:13<657:16:02, 10.76s/it]
  2%|▏         | 5098/225000 [19:44:20<575:50:58,  9.43s/it]
  2%|▏         | 5099/225000 [19:44:30<600:44:49,  9.83s/it]
  2%|▏         | 5100/225000 [19:44:41<610:12:06,  9.99s/it]
                                                            
{'loss': '0.8101', 'grad_norm': '1.345', 'learning_rate': '2.547e-05', 'epoch': '0.02267', 'num_input_tokens_seen': 36249728, 'train_runtime': '7.108e+04', 'train_tokens_per_second': '510'}

  2%|▏         | 5100/225000 [19:44:41<610:12:06,  9.99s/it]
  2%|▏         | 5101/225000 [19:44:53<651:41:07, 10.67s/it]
  2%|▏         | 5102/225000 [19:45:03<637:21:04, 10.43s/it]
  2%|▏         | 5103/225000 [19:45:12<603:39:30,  9.88s/it]
  2%|▏         | 5104/225000 [19:45:22<609:29:25,  9.98s/it]
  2%|▏         | 5105/225000 [19:45:38<728:02:12, 11.92s/it]
  2%|▏         | 5106/225000 [19:45:52<754:31:53, 12.35s/it]
  2%|▏         | 5107/225000 [19:45:57<624:50:25, 10.23s/it]
  2%|▏         | 5108/225000 [19:46:05<586:47:58,  9.61s/it]
  2%|▏         | 5109/225000 [19:46:18<647:00:15, 10.59s/it]
  2%|▏         | 5110/225000 [19:46:28<646:59:18, 10.59s/it]
  2%|▏         | 5111/225000 [19:46:37<608:48:50,  9.97s/it]
  2%|▏         | 5112/225000 [19:46:43<528:05:39,  8.65s/it]
  2%|▏         | 5113/225000 [19:46:54<582:34:13,  9.54s/it]
  2%|▏         | 5114/225000 [19:47:07<645:48:40, 10.57s/it]
  2%|▏         | 5115/225000 [19:47:16<612:20:17, 10.03s/it]
  2%|▏         | 5116/225000 [19:47:21<531:02:11,  8.69s/it]
  2%|▏         | 5117/225000 [19:47:31<540:18:44,  8.85s/it]
  2%|▏         | 5118/225000 [19:47:39<539:38:58,  8.84s/it]
  2%|▏         | 5119/225000 [19:47:45<482:08:27,  7.89s/it]
  2%|▏         | 5120/225000 [19:47:55<517:56:09,  8.48s/it]
  2%|▏         | 5121/225000 [19:48:04<527:53:48,  8.64s/it]
  2%|▏         | 5122/225000 [19:48:13<538:45:15,  8.82s/it]
  2%|▏         | 5123/225000 [19:48:19<483:17:35,  7.91s/it]
  2%|▏         | 5124/225000 [19:48:30<540:12:36,  8.84s/it]
  2%|▏         | 5125/225000 [19:48:38<528:44:18,  8.66s/it]
  2%|▏         | 5126/225000 [19:48:53<634:13:24, 10.38s/it]
  2%|▏         | 5127/225000 [19:49:08<725:59:49, 11.89s/it]
  2%|▏         | 5128/225000 [19:49:13<588:43:25,  9.64s/it]
  2%|▏         | 5129/225000 [19:49:26<657:07:15, 10.76s/it]
  2%|▏         | 5130/225000 [19:49:33<589:54:30,  9.66s/it]
  2%|▏         | 5131/225000 [19:49:43<596:18:16,  9.76s/it]
  2%|▏         | 5132/225000 [19:49:49<522:44:27,  8.56s/it]
  2%|▏         | 5133/225000 [19:50:00<580:44:28,  9.51s/it]
  2%|▏         | 5134/225000 [19:50:10<577:35:17,  9.46s/it]
  2%|▏         | 5135/225000 [19:50:18<549:30:12,  9.00s/it]
  2%|▏         | 5136/225000 [19:50:32<644:36:45, 10.55s/it]
  2%|▏         | 5137/225000 [19:50:39<575:39:07,  9.43s/it]
  2%|▏         | 5138/225000 [19:50:54<680:14:44, 11.14s/it]
  2%|▏         | 5139/225000 [19:51:00<594:14:39,  9.73s/it]
  2%|▏         | 5140/225000 [19:51:07<533:26:35,  8.73s/it]
  2%|▏         | 5141/225000 [19:51:20<614:07:19, 10.06s/it]
  2%|▏         | 5142/225000 [19:51:27<561:34:00,  9.20s/it]
  2%|▏         | 5143/225000 [19:51:41<654:06:12, 10.71s/it]
  2%|▏         | 5144/225000 [19:51:49<591:02:36,  9.68s/it]
  2%|▏         | 5145/225000 [19:51:59<612:56:23, 10.04s/it]
  2%|▏         | 5146/225000 [19:52:07<571:08:30,  9.35s/it]
  2%|▏         | 5147/225000 [19:52:13<498:32:21,  8.16s/it]
  2%|▏         | 5148/225000 [19:52:23<531:40:09,  8.71s/it]
  2%|▏         | 5149/225000 [19:52:29<484:15:00,  7.93s/it]
  2%|▏         | 5150/225000 [19:52:43<603:12:14,  9.88s/it]
                                                            
{'loss': '0.7857', 'grad_norm': '1.495', 'learning_rate': '2.572e-05', 'epoch': '0.02289', 'num_input_tokens_seen': 36487648, 'train_runtime': '7.156e+04', 'train_tokens_per_second': '509.9'}

  2%|▏         | 5150/225000 [19:52:43<603:12:14,  9.88s/it]
  2%|▏         | 5151/225000 [19:52:51<573:06:09,  9.38s/it]
  2%|▏         | 5152/225000 [19:53:06<666:57:34, 10.92s/it]
  2%|▏         | 5153/225000 [19:53:18<699:18:05, 11.45s/it]
  2%|▏         | 5154/225000 [19:53:31<714:27:43, 11.70s/it]
  2%|▏         | 5155/225000 [19:53:37<605:42:42,  9.92s/it]
  2%|▏         | 5156/225000 [19:53:52<709:13:31, 11.61s/it]
  2%|▏         | 5157/225000 [19:54:05<726:49:06, 11.90s/it]
  2%|▏         | 5158/225000 [19:54:14<681:08:39, 11.15s/it]
  2%|▏         | 5159/225000 [19:54:24<666:15:07, 10.91s/it]
  2%|▏         | 5160/225000 [19:54:34<650:53:55, 10.66s/it]
  2%|▏         | 5161/225000 [19:54:43<610:56:08, 10.00s/it]
  2%|▏         | 5162/225000 [19:54:48<525:34:25,  8.61s/it]
  2%|▏         | 5163/225000 [19:55:07<710:06:49, 11.63s/it]
  2%|▏         | 5164/225000 [19:55:11<579:13:02,  9.49s/it]
  2%|▏         | 5165/225000 [19:55:26<663:59:28, 10.87s/it]
  2%|▏         | 5166/225000 [19:55:34<614:41:17, 10.07s/it]
  2%|▏         | 5167/225000 [19:55:47<675:48:59, 11.07s/it]
  2%|▏         | 5168/225000 [19:56:01<721:05:48, 11.81s/it]
  2%|▏         | 5169/225000 [19:56:09<648:03:32, 10.61s/it]
  2%|▏         | 5170/225000 [19:56:16<582:26:24,  9.54s/it]
  2%|▏         | 5171/225000 [19:56:32<705:52:39, 11.56s/it]
  2%|▏         | 5172/225000 [19:56:46<761:00:04, 12.46s/it]
  2%|▏         | 5173/225000 [19:56:54<668:29:24, 10.95s/it]
  2%|▏         | 5174/225000 [19:57:07<708:27:05, 11.60s/it]
  2%|▏         | 5175/225000 [19:57:16<657:01:02, 10.76s/it]
  2%|▏         | 5176/225000 [19:57:24<615:24:16, 10.08s/it]
  2%|▏         | 5177/225000 [19:57:33<585:15:30,  9.58s/it]
  2%|▏         | 5178/225000 [19:57:41<566:20:51,  9.28s/it]
  2%|▏         | 5179/225000 [19:57:54<634:44:26, 10.40s/it]
  2%|▏         | 5180/225000 [19:58:04<616:40:30, 10.10s/it]
  2%|▏         | 5181/225000 [19:58:17<667:57:43, 10.94s/it]
  2%|▏         | 5182/225000 [19:58:24<608:37:02,  9.97s/it]
  2%|▏         | 5183/225000 [19:58:34<600:31:26,  9.83s/it]
  2%|▏         | 5184/225000 [19:58:47<656:51:34, 10.76s/it]
  2%|▏         | 5185/225000 [19:58:54<592:20:21,  9.70s/it]
  2%|▏         | 5186/225000 [19:59:03<585:35:07,  9.59s/it]
  2%|▏         | 5187/225000 [19:59:18<685:15:53, 11.22s/it]
  2%|▏         | 5188/225000 [19:59:33<745:25:45, 12.21s/it]
  2%|▏         | 5189/225000 [19:59:44<726:14:40, 11.89s/it]
  2%|▏         | 5190/225000 [19:59:54<694:06:22, 11.37s/it]
  2%|▏         | 5191/225000 [20:00:01<609:42:28,  9.99s/it]
  2%|▏         | 5192/225000 [20:00:14<662:34:22, 10.85s/it]
  2%|▏         | 5193/225000 [20:00:23<637:23:29, 10.44s/it]
  2%|▏         | 5194/225000 [20:00:34<644:05:18, 10.55s/it]
  2%|▏         | 5195/225000 [20:00:39<536:51:01,  8.79s/it]
  2%|▏         | 5196/225000 [20:00:44<473:16:30,  7.75s/it]
  2%|▏         | 5197/225000 [20:00:54<521:51:48,  8.55s/it]
  2%|▏         | 5198/225000 [20:01:00<471:33:53,  7.72s/it]
  2%|▏         | 5199/225000 [20:01:15<593:32:09,  9.72s/it]
  2%|▏         | 5200/225000 [20:01:25<611:44:10, 10.02s/it]
                                                            
{'loss': '0.8117', 'grad_norm': '1.675', 'learning_rate': '2.597e-05', 'epoch': '0.02311', 'num_input_tokens_seen': 36739672, 'train_runtime': '7.209e+04', 'train_tokens_per_second': '509.7'}

  2%|▏         | 5200/225000 [20:01:25<611:44:10, 10.02s/it]
  2%|▏         | 5201/225000 [20:01:29<490:31:39,  8.03s/it]
  2%|▏         | 5202/225000 [20:01:39<534:16:57,  8.75s/it]
  2%|▏         | 5203/225000 [20:01:49<558:36:47,  9.15s/it]
  2%|▏         | 5204/225000 [20:02:03<635:33:22, 10.41s/it]
  2%|▏         | 5205/225000 [20:02:13<631:04:34, 10.34s/it]
  2%|▏         | 5206/225000 [20:02:24<643:35:55, 10.54s/it]
  2%|▏         | 5207/225000 [20:02:34<629:35:50, 10.31s/it]
  2%|▏         | 5208/225000 [20:02:49<731:13:52, 11.98s/it]
  2%|▏         | 5209/225000 [20:02:58<662:05:56, 10.84s/it]
  2%|▏         | 5210/225000 [20:03:10<698:20:55, 11.44s/it]
  2%|▏         | 5211/225000 [20:03:15<578:05:41,  9.47s/it]
  2%|▏         | 5212/225000 [20:03:20<495:07:05,  8.11s/it]
  2%|▏         | 5213/225000 [20:03:30<523:28:38,  8.57s/it]
  2%|▏         | 5214/225000 [20:03:45<638:50:53, 10.46s/it]
  2%|▏         | 5215/225000 [20:03:54<620:39:03, 10.17s/it]
  2%|▏         | 5216/225000 [20:04:12<760:33:54, 12.46s/it]
  2%|▏         | 5217/225000 [20:04:31<875:27:19, 14.34s/it]
  2%|▏         | 5218/225000 [20:04:35<692:12:45, 11.34s/it]
  2%|▏         | 5219/225000 [20:04:41<598:42:05,  9.81s/it]
  2%|▏         | 5220/225000 [20:04:49<559:07:22,  9.16s/it]
  2%|▏         | 5221/225000 [20:05:05<678:35:34, 11.12s/it]
  2%|▏         | 5222/225000 [20:05:14<643:08:50, 10.53s/it]
  2%|▏         | 5223/225000 [20:05:19<550:34:12,  9.02s/it]
  2%|▏         | 5224/225000 [20:05:25<494:13:23,  8.10s/it]
  2%|▏         | 5225/225000 [20:05:36<546:09:45,  8.95s/it]
  2%|▏         | 5226/225000 [20:05:48<600:25:10,  9.84s/it]
  2%|▏         | 5227/225000 [20:05:58<600:03:52,  9.83s/it]
  2%|▏         | 5228/225000 [20:06:09<623:43:01, 10.22s/it]
  2%|▏         | 5229/225000 [20:06:18<596:51:03,  9.78s/it]
  2%|▏         | 5230/225000 [20:06:32<673:52:27, 11.04s/it]
  2%|▏         | 5231/225000 [20:06:40<628:21:05, 10.29s/it]
  2%|▏         | 5232/225000 [20:06:48<577:05:25,  9.45s/it]
  2%|▏         | 5233/225000 [20:07:05<723:33:12, 11.85s/it]
  2%|▏         | 5234/225000 [20:07:14<658:24:49, 10.79s/it]
  2%|▏         | 5235/225000 [20:07:22<621:42:48, 10.18s/it]
  2%|▏         | 5236/225000 [20:07:30<574:26:33,  9.41s/it]
  2%|▏         | 5237/225000 [20:07:36<505:21:00,  8.28s/it]
  2%|▏         | 5238/225000 [20:07:53<675:52:45, 11.07s/it]
  2%|▏         | 5239/225000 [20:08:00<604:02:25,  9.90s/it]
  2%|▏         | 5240/225000 [20:08:12<636:33:00, 10.43s/it]
  2%|▏         | 5241/225000 [20:08:18<561:28:18,  9.20s/it]
  2%|▏         | 5242/225000 [20:08:28<568:06:38,  9.31s/it]
  2%|▏         | 5243/225000 [20:08:33<484:07:45,  7.93s/it]
  2%|▏         | 5244/225000 [20:08:39<454:19:10,  7.44s/it]
  2%|▏         | 5245/225000 [20:08:52<552:28:43,  9.05s/it]
  2%|▏         | 5246/225000 [20:08:58<499:19:13,  8.18s/it]
  2%|▏         | 5247/225000 [20:09:13<623:15:03, 10.21s/it]
  2%|▏         | 5248/225000 [20:09:22<601:12:12,  9.85s/it]
  2%|▏         | 5249/225000 [20:09:28<539:14:14,  8.83s/it]
  2%|▏         | 5250/225000 [20:09:42<636:58:52, 10.44s/it]
                                                            
{'loss': '0.8032', 'grad_norm': '1.276', 'learning_rate': '2.623e-05', 'epoch': '0.02333', 'num_input_tokens_seen': 36987624, 'train_runtime': '7.258e+04', 'train_tokens_per_second': '509.6'}

  2%|▏         | 5250/225000 [20:09:42<636:58:52, 10.44s/it]
  2%|▏         | 5251/225000 [20:09:53<638:02:04, 10.45s/it]
  2%|▏         | 5252/225000 [20:09:58<531:22:11,  8.71s/it]
  2%|▏         | 5253/225000 [20:10:03<472:39:39,  7.74s/it]
  2%|▏         | 5254/225000 [20:10:10<457:51:06,  7.50s/it]
  2%|▏         | 5255/225000 [20:10:27<637:33:24, 10.44s/it]
  2%|▏         | 5256/225000 [20:10:43<735:19:50, 12.05s/it]
  2%|▏         | 5257/225000 [20:10:51<662:36:43, 10.86s/it]
  2%|▏         | 5258/225000 [20:11:02<657:34:31, 10.77s/it]
  2%|▏         | 5259/225000 [20:11:07<563:24:51,  9.23s/it]
  2%|▏         | 5260/225000 [20:11:13<504:08:26,  8.26s/it]
  2%|▏         | 5261/225000 [20:11:28<616:06:29, 10.09s/it]
  2%|▏         | 5262/225000 [20:11:36<581:10:52,  9.52s/it]
  2%|▏         | 5263/225000 [20:11:45<567:48:57,  9.30s/it]
  2%|▏         | 5264/225000 [20:11:54<571:16:00,  9.36s/it]
  2%|▏         | 5265/225000 [20:12:10<684:24:48, 11.21s/it]
  2%|▏         | 5266/225000 [20:12:17<603:30:03,  9.89s/it]
  2%|▏         | 5267/225000 [20:12:30<677:19:35, 11.10s/it]
  2%|▏         | 5268/225000 [20:12:35<558:12:47,  9.15s/it]
  2%|▏         | 5269/225000 [20:12:40<486:40:00,  7.97s/it]
  2%|▏         | 5270/225000 [20:12:54<596:34:05,  9.77s/it]
  2%|▏         | 5271/225000 [20:13:02<567:33:38,  9.30s/it]
  2%|▏         | 5272/225000 [20:13:09<514:06:29,  8.42s/it]
  2%|▏         | 5273/225000 [20:13:22<608:01:31,  9.96s/it]
  2%|▏         | 5274/225000 [20:13:35<664:35:36, 10.89s/it]
  2%|▏         | 5275/225000 [20:13:45<632:03:08, 10.36s/it]
  2%|▏         | 5276/225000 [20:13:50<547:34:14,  8.97s/it]
  2%|▏         | 5277/225000 [20:14:03<624:26:37, 10.23s/it]
  2%|▏         | 5278/225000 [20:14:11<583:14:13,  9.56s/it]
  2%|▏         | 5279/225000 [20:14:20<565:28:54,  9.27s/it]
  2%|▏         | 5280/225000 [20:14:27<524:30:12,  8.59s/it]
  2%|▏         | 5281/225000 [20:14:39<591:53:30,  9.70s/it]
  2%|▏         | 5282/225000 [20:14:54<673:49:25, 11.04s/it]
  2%|▏         | 5283/225000 [20:15:01<611:04:38, 10.01s/it]
  2%|▏         | 5284/225000 [20:15:08<560:11:24,  9.18s/it]
  2%|▏         | 5285/225000 [20:15:18<576:09:55,  9.44s/it]
  2%|▏         | 5286/225000 [20:15:27<560:08:51,  9.18s/it]
  2%|▏         | 5287/225000 [20:15:35<538:37:57,  8.83s/it]
  2%|▏         | 5288/225000 [20:15:50<647:02:22, 10.60s/it]
  2%|▏         | 5289/225000 [20:16:00<649:29:51, 10.64s/it]
  2%|▏         | 5290/225000 [20:16:11<640:27:11, 10.49s/it]
  2%|▏         | 5291/225000 [20:16:26<732:42:15, 12.01s/it]
  2%|▏         | 5292/225000 [20:16:38<722:14:35, 11.83s/it]
  2%|▏         | 5293/225000 [20:16:45<631:59:06, 10.36s/it]
  2%|▏         | 5294/225000 [20:16:55<632:59:31, 10.37s/it]
  2%|▏         | 5295/225000 [20:17:10<724:06:35, 11.86s/it]
  2%|▏         | 5296/225000 [20:17:16<603:50:56,  9.89s/it]
  2%|▏         | 5297/225000 [20:17:20<511:29:25,  8.38s/it]
  2%|▏         | 5298/225000 [20:17:29<521:34:25,  8.55s/it]
  2%|▏         | 5299/225000 [20:17:43<606:06:08,  9.93s/it]
  2%|▏         | 5300/225000 [20:17:54<643:19:21, 10.54s/it]
                                                            
{'loss': '0.7704', 'grad_norm': '1.246', 'learning_rate': '2.647e-05', 'epoch': '0.02356', 'num_input_tokens_seen': 37224184, 'train_runtime': '7.307e+04', 'train_tokens_per_second': '509.4'}

  2%|▏         | 5300/225000 [20:17:54<643:19:21, 10.54s/it]
  2%|▏         | 5301/225000 [20:18:04<618:11:51, 10.13s/it]
  2%|▏         | 5302/225000 [20:18:18<693:21:03, 11.36s/it]
  2%|▏         | 5303/225000 [20:18:24<602:49:00,  9.88s/it]
  2%|▏         | 5304/225000 [20:18:34<595:08:13,  9.75s/it]
  2%|▏         | 5305/225000 [20:18:40<533:44:04,  8.75s/it]
  2%|▏         | 5306/225000 [20:18:47<506:45:36,  8.30s/it]
  2%|▏         | 5307/225000 [20:19:03<644:06:48, 10.55s/it]
  2%|▏         | 5308/225000 [20:19:08<540:08:03,  8.85s/it]
  2%|▏         | 5309/225000 [20:19:22<639:19:32, 10.48s/it]
  2%|▏         | 5310/225000 [20:19:39<744:12:18, 12.20s/it]
  2%|▏         | 5311/225000 [20:19:51<742:45:20, 12.17s/it]
  2%|▏         | 5312/225000 [20:19:55<607:42:50,  9.96s/it]
  2%|▏         | 5313/225000 [20:20:04<578:47:56,  9.48s/it]
  2%|▏         | 5314/225000 [20:20:14<590:54:29,  9.68s/it]
  2%|▏         | 5315/225000 [20:20:19<505:17:20,  8.28s/it]
  2%|▏         | 5316/225000 [20:20:30<556:31:34,  9.12s/it]
  2%|▏         | 5317/225000 [20:20:37<508:40:34,  8.34s/it]
  2%|▏         | 5318/225000 [20:20:42<460:54:24,  7.55s/it]
  2%|▏         | 5319/225000 [20:20:51<473:14:57,  7.76s/it]
  2%|▏         | 5320/225000 [20:20:57<457:35:17,  7.50s/it]
  2%|▏         | 5321/225000 [20:21:09<530:53:34,  8.70s/it]
  2%|▏         | 5322/225000 [20:21:18<537:19:04,  8.81s/it]
  2%|▏         | 5323/225000 [20:21:23<461:21:50,  7.56s/it]
  2%|▏         | 5324/225000 [20:21:37<584:10:39,  9.57s/it]
  2%|▏         | 5325/225000 [20:21:56<748:44:39, 12.27s/it]
  2%|▏         | 5326/225000 [20:22:02<641:08:36, 10.51s/it]
  2%|▏         | 5327/225000 [20:22:13<654:39:31, 10.73s/it]
  2%|▏         | 5328/225000 [20:22:22<627:33:57, 10.28s/it]
  2%|▏         | 5329/225000 [20:22:32<608:54:54,  9.98s/it]
  2%|▏         | 5330/225000 [20:22:43<632:00:11, 10.36s/it]
  2%|▏         | 5331/225000 [20:22:48<543:43:19,  8.91s/it]
  2%|▏         | 5332/225000 [20:23:03<645:20:30, 10.58s/it]
  2%|▏         | 5333/225000 [20:23:13<631:34:20, 10.35s/it]
  2%|▏         | 5334/225000 [20:23:16<510:31:50,  8.37s/it]
  2%|▏         | 5335/225000 [20:23:31<623:53:01, 10.22s/it]
  2%|▏         | 5336/225000 [20:23:39<575:18:25,  9.43s/it]
  2%|▏         | 5337/225000 [20:23:46<545:48:39,  8.95s/it]
  2%|▏         | 5338/225000 [20:23:57<575:10:37,  9.43s/it]
  2%|▏         | 5339/225000 [20:24:09<617:23:27, 10.12s/it]
  2%|▏         | 5340/225000 [20:24:20<645:06:31, 10.57s/it]
  2%|▏         | 5341/225000 [20:24:29<603:02:55,  9.88s/it]
  2%|▏         | 5342/225000 [20:24:33<501:55:26,  8.23s/it]
  2%|▏         | 5343/225000 [20:24:38<450:43:25,  7.39s/it]
  2%|▏         | 5344/225000 [20:24:43<400:03:05,  6.56s/it]
  2%|▏         | 5345/225000 [20:24:53<457:42:03,  7.50s/it]
  2%|▏         | 5346/225000 [20:25:06<570:02:51,  9.34s/it]
  2%|▏         | 5347/225000 [20:25:20<650:01:58, 10.65s/it]
  2%|▏         | 5348/225000 [20:25:30<639:29:44, 10.48s/it]
  2%|▏         | 5349/225000 [20:25:36<564:01:46,  9.24s/it]
  2%|▏         | 5350/225000 [20:25:44<524:32:26,  8.60s/it]
                                                            
{'loss': '0.7566', 'grad_norm': '1.741', 'learning_rate': '2.672e-05', 'epoch': '0.02378', 'num_input_tokens_seen': 37454920, 'train_runtime': '7.354e+04', 'train_tokens_per_second': '509.3'}

  2%|▏         | 5350/225000 [20:25:44<524:32:26,  8.60s/it]
  2%|▏         | 5351/225000 [20:25:53<538:59:47,  8.83s/it]
  2%|▏         | 5352/225000 [20:26:02<547:29:19,  8.97s/it]
  2%|▏         | 5353/225000 [20:26:10<523:30:23,  8.58s/it]
  2%|▏         | 5354/225000 [20:26:20<547:09:22,  8.97s/it]
  2%|▏         | 5355/225000 [20:26:31<582:58:33,  9.56s/it]
  2%|▏         | 5356/225000 [20:26:51<774:38:19, 12.70s/it]
  2%|▏         | 5357/225000 [20:26:54<607:42:37,  9.96s/it]
  2%|▏         | 5358/225000 [20:27:04<593:19:25,  9.72s/it]
  2%|▏         | 5359/225000 [20:27:12<576:16:25,  9.45s/it]
  2%|▏         | 5360/225000 [20:27:23<599:45:57,  9.83s/it]
  2%|▏         | 5361/225000 [20:27:31<557:08:19,  9.13s/it]
  2%|▏         | 5362/225000 [20:27:36<495:19:07,  8.12s/it]
  2%|▏         | 5363/225000 [20:27:50<589:49:09,  9.67s/it]
  2%|▏         | 5364/225000 [20:27:54<492:06:19,  8.07s/it]
  2%|▏         | 5365/225000 [20:28:00<461:48:23,  7.57s/it]
  2%|▏         | 5366/225000 [20:28:08<466:11:41,  7.64s/it]
  2%|▏         | 5367/225000 [20:28:17<481:02:58,  7.88s/it]
  2%|▏         | 5368/225000 [20:28:22<431:32:12,  7.07s/it]
  2%|▏         | 5369/225000 [20:28:36<557:03:16,  9.13s/it]
  2%|▏         | 5370/225000 [20:28:43<532:47:41,  8.73s/it]
  2%|▏         | 5371/225000 [20:28:51<517:39:19,  8.49s/it]
  2%|▏         | 5372/225000 [20:29:01<533:20:27,  8.74s/it]
  2%|▏         | 5373/225000 [20:29:15<633:40:57, 10.39s/it]
  2%|▏         | 5374/225000 [20:29:26<639:35:02, 10.48s/it]
  2%|▏         | 5375/225000 [20:29:37<646:25:10, 10.60s/it]
  2%|▏         | 5376/225000 [20:29:49<680:11:44, 11.15s/it]
  2%|▏         | 5377/225000 [20:30:03<724:07:21, 11.87s/it]
  2%|▏         | 5378/225000 [20:30:08<605:22:00,  9.92s/it]
  2%|▏         | 5379/225000 [20:30:20<650:45:01, 10.67s/it]
  2%|▏         | 5380/225000 [20:30:30<641:33:24, 10.52s/it]
  2%|▏         | 5381/225000 [20:30:42<668:17:24, 10.95s/it]
  2%|▏         | 5382/225000 [20:30:53<655:19:32, 10.74s/it]
  2%|▏         | 5383/225000 [20:31:05<681:57:17, 11.18s/it]
  2%|▏         | 5384/225000 [20:31:15<670:10:04, 10.99s/it]
  2%|▏         | 5385/225000 [20:31:27<684:44:01, 11.22s/it]
  2%|▏         | 5386/225000 [20:31:36<636:17:07, 10.43s/it]
  2%|▏         | 5387/225000 [20:31:46<629:28:55, 10.32s/it]
  2%|▏         | 5388/225000 [20:31:57<645:22:23, 10.58s/it]
  2%|▏         | 5389/225000 [20:32:08<660:19:31, 10.82s/it]
  2%|▏         | 5390/225000 [20:32:19<654:33:00, 10.73s/it]
  2%|▏         | 5391/225000 [20:32:29<650:57:14, 10.67s/it]
  2%|▏         | 5392/225000 [20:32:36<580:40:53,  9.52s/it]
  2%|▏         | 5393/225000 [20:32:44<545:02:41,  8.93s/it]
  2%|▏         | 5394/225000 [20:32:52<529:32:47,  8.68s/it]
  2%|▏         | 5395/225000 [20:33:04<595:39:12,  9.76s/it]
  2%|▏         | 5396/225000 [20:33:16<626:27:04, 10.27s/it]
  2%|▏         | 5397/225000 [20:33:21<530:01:14,  8.69s/it]
  2%|▏         | 5398/225000 [20:33:32<569:35:42,  9.34s/it]
  2%|▏         | 5399/225000 [20:33:40<551:39:03,  9.04s/it]
  2%|▏         | 5400/225000 [20:33:50<566:11:27,  9.28s/it]
                                                            
{'loss': '0.794', 'grad_norm': '1.556', 'learning_rate': '2.698e-05', 'epoch': '0.024', 'num_input_tokens_seen': 37696400, 'train_runtime': '7.403e+04', 'train_tokens_per_second': '509.2'}

  2%|▏         | 5400/225000 [20:33:50<566:11:27,  9.28s/it]
  2%|▏         | 5401/225000 [20:33:58<555:17:38,  9.10s/it]
  2%|▏         | 5402/225000 [20:34:07<548:44:17,  9.00s/it]
  2%|▏         | 5403/225000 [20:34:19<606:35:48,  9.94s/it]
  2%|▏         | 5404/225000 [20:34:28<586:45:37,  9.62s/it]
  2%|▏         | 5405/225000 [20:34:37<570:05:29,  9.35s/it]
  2%|▏         | 5406/225000 [20:34:48<599:15:13,  9.82s/it]
  2%|▏         | 5407/225000 [20:35:00<640:56:23, 10.51s/it]
  2%|▏         | 5408/225000 [20:35:12<660:57:24, 10.84s/it]
  2%|▏         | 5409/225000 [20:35:18<572:28:18,  9.39s/it]
  2%|▏         | 5410/225000 [20:35:24<510:54:44,  8.38s/it]
  2%|▏         | 5411/225000 [20:35:30<473:41:34,  7.77s/it]
  2%|▏         | 5412/225000 [20:35:44<591:27:15,  9.70s/it]
  2%|▏         | 5413/225000 [20:35:52<556:17:43,  9.12s/it]
  2%|▏         | 5414/225000 [20:36:06<645:06:25, 10.58s/it]
  2%|▏         | 5415/225000 [20:36:12<554:58:31,  9.10s/it]
  2%|▏         | 5416/225000 [20:36:20<543:16:49,  8.91s/it]
  2%|▏         | 5417/225000 [20:36:27<505:11:07,  8.28s/it]
  2%|▏         | 5418/225000 [20:36:38<560:31:40,  9.19s/it]
  2%|▏         | 5419/225000 [20:36:44<494:35:12,  8.11s/it]
  2%|▏         | 5420/225000 [20:36:52<500:32:36,  8.21s/it]
  2%|▏         | 5421/225000 [20:37:02<529:34:38,  8.68s/it]
  2%|▏         | 5422/225000 [20:37:11<536:30:29,  8.80s/it]
  2%|▏         | 5423/225000 [20:37:18<513:03:09,  8.41s/it]
  2%|▏         | 5424/225000 [20:37:33<630:47:17, 10.34s/it]
  2%|▏         | 5425/225000 [20:37:38<527:33:11,  8.65s/it]
  2%|▏         | 5426/225000 [20:37:46<518:58:13,  8.51s/it]
  2%|▏         | 5427/225000 [20:37:57<554:16:10,  9.09s/it]
  2%|▏         | 5428/225000 [20:38:05<548:01:13,  8.99s/it]
  2%|▏         | 5429/225000 [20:38:19<632:42:34, 10.37s/it]
  2%|▏         | 5430/225000 [20:38:24<542:49:08,  8.90s/it]
  2%|▏         | 5431/225000 [20:38:34<555:13:44,  9.10s/it]
  2%|▏         | 5432/225000 [20:38:39<486:01:45,  7.97s/it]
  2%|▏         | 5433/225000 [20:38:49<510:30:37,  8.37s/it]
  2%|▏         | 5434/225000 [20:38:56<488:24:35,  8.01s/it]
  2%|▏         | 5435/225000 [20:39:05<501:02:35,  8.22s/it]
  2%|▏         | 5436/225000 [20:39:17<571:53:28,  9.38s/it]
  2%|▏         | 5437/225000 [20:39:28<613:29:58, 10.06s/it]
  2%|▏         | 5438/225000 [20:39:35<549:52:49,  9.02s/it]
  2%|▏         | 5439/225000 [20:39:46<593:23:47,  9.73s/it]
  2%|▏         | 5440/225000 [20:39:51<494:18:41,  8.10s/it]
  2%|▏         | 5441/225000 [20:40:03<565:53:32,  9.28s/it]
  2%|▏         | 5442/225000 [20:40:07<481:35:51,  7.90s/it]
  2%|▏         | 5443/225000 [20:40:15<483:09:30,  7.92s/it]
  2%|▏         | 5444/225000 [20:40:27<546:00:50,  8.95s/it]
  2%|▏         | 5445/225000 [20:40:33<498:24:07,  8.17s/it]
  2%|▏         | 5446/225000 [20:40:45<570:31:18,  9.35s/it]
  2%|▏         | 5447/225000 [20:40:58<641:48:26, 10.52s/it]
  2%|▏         | 5448/225000 [20:41:09<649:08:18, 10.64s/it]
  2%|▏         | 5449/225000 [20:41:20<656:45:21, 10.77s/it]
  2%|▏         | 5450/225000 [20:41:29<625:55:44, 10.26s/it]
                                                            
{'loss': '0.8119', 'grad_norm': '1.669', 'learning_rate': '2.723e-05', 'epoch': '0.02422', 'num_input_tokens_seen': 37925920, 'train_runtime': '7.449e+04', 'train_tokens_per_second': '509.1'}

  2%|▏         | 5450/225000 [20:41:29<625:55:44, 10.26s/it]
  2%|▏         | 5451/225000 [20:41:35<536:50:23,  8.80s/it]
  2%|▏         | 5452/225000 [20:41:50<657:30:02, 10.78s/it]
  2%|▏         | 5453/225000 [20:41:58<601:08:19,  9.86s/it]
  2%|▏         | 5454/225000 [20:42:05<555:43:41,  9.11s/it]
  2%|▏         | 5455/225000 [20:42:18<621:41:02, 10.19s/it]
  2%|▏         | 5456/225000 [20:42:25<567:12:02,  9.30s/it]
  2%|▏         | 5457/225000 [20:42:31<504:20:50,  8.27s/it]
  2%|▏         | 5458/225000 [20:42:37<464:40:17,  7.62s/it]
  2%|▏         | 5459/225000 [20:42:50<553:14:26,  9.07s/it]
  2%|▏         | 5460/225000 [20:42:54<475:23:11,  7.80s/it]
  2%|▏         | 5461/225000 [20:43:05<528:34:34,  8.67s/it]
  2%|▏         | 5462/225000 [20:43:15<559:00:29,  9.17s/it]
  2%|▏         | 5463/225000 [20:43:25<564:33:06,  9.26s/it]
  2%|▏         | 5464/225000 [20:43:35<586:40:56,  9.62s/it]
  2%|▏         | 5465/225000 [20:43:44<564:42:14,  9.26s/it]
  2%|▏         | 5466/225000 [20:43:53<570:35:21,  9.36s/it]
  2%|▏         | 5467/225000 [20:44:10<695:48:46, 11.41s/it]
  2%|▏         | 5468/225000 [20:44:19<650:31:54, 10.67s/it]
  2%|▏         | 5469/225000 [20:44:29<651:15:28, 10.68s/it]
  2%|▏         | 5470/225000 [20:44:39<636:04:28, 10.43s/it]
  2%|▏         | 5471/225000 [20:44:43<515:00:23,  8.45s/it]
  2%|▏         | 5472/225000 [20:44:53<551:43:56,  9.05s/it]
  2%|▏         | 5473/225000 [20:45:02<535:27:17,  8.78s/it]
  2%|▏         | 5474/225000 [20:45:15<619:06:58, 10.15s/it]
  2%|▏         | 5475/225000 [20:45:20<534:14:56,  8.76s/it]
  2%|▏         | 5476/225000 [20:45:27<493:10:34,  8.09s/it]
  2%|▏         | 5477/225000 [20:45:39<573:57:06,  9.41s/it]
  2%|▏         | 5478/225000 [20:45:46<527:44:59,  8.65s/it]
  2%|▏         | 5479/225000 [20:46:01<641:43:56, 10.52s/it]
  2%|▏         | 5480/225000 [20:46:08<568:40:17,  9.33s/it]
  2%|▏         | 5481/225000 [20:46:15<528:20:03,  8.66s/it]
  2%|▏         | 5482/225000 [20:46:20<463:36:50,  7.60s/it]
  2%|▏         | 5483/225000 [20:46:36<626:49:30, 10.28s/it]
  2%|▏         | 5484/225000 [20:46:40<506:50:52,  8.31s/it]
  2%|▏         | 5485/225000 [20:46:50<535:51:20,  8.79s/it]
  2%|▏         | 5486/225000 [20:47:07<683:24:21, 11.21s/it]
  2%|▏         | 5487/225000 [20:47:18<674:04:29, 11.05s/it]
  2%|▏         | 5488/225000 [20:47:30<702:32:27, 11.52s/it]
  2%|▏         | 5489/225000 [20:47:41<690:43:14, 11.33s/it]
  2%|▏         | 5490/225000 [20:47:55<735:12:04, 12.06s/it]
  2%|▏         | 5491/225000 [20:48:09<771:22:50, 12.65s/it]
  2%|▏         | 5492/225000 [20:48:16<669:34:42, 10.98s/it]
  2%|▏         | 5493/225000 [20:48:25<634:07:40, 10.40s/it]
  2%|▏         | 5494/225000 [20:48:30<532:29:14,  8.73s/it]
  2%|▏         | 5495/225000 [20:48:39<531:26:11,  8.72s/it]
  2%|▏         | 5496/225000 [20:48:44<470:09:20,  7.71s/it]
  2%|▏         | 5497/225000 [20:48:53<494:23:53,  8.11s/it]
  2%|▏         | 5498/225000 [20:48:59<462:25:55,  7.58s/it]
  2%|▏         | 5499/225000 [20:49:06<452:05:05,  7.41s/it]
  2%|▏         | 5500/225000 [20:49:24<640:25:27, 10.50s/it]
                                                            
{'loss': '0.7859', 'grad_norm': '1.129', 'learning_rate': '2.748e-05', 'epoch': '0.02444', 'num_input_tokens_seen': 38166880, 'train_runtime': '7.496e+04', 'train_tokens_per_second': '509.1'}

  2%|▏         | 5500/225000 [20:49:24<640:25:27, 10.50s/it]
  2%|▏         | 5501/225000 [20:49:34<624:02:01, 10.23s/it]
  2%|▏         | 5502/225000 [20:49:49<717:26:12, 11.77s/it]
  2%|▏         | 5503/225000 [20:49:55<604:34:51,  9.92s/it]
  2%|▏         | 5504/225000 [20:50:13<762:31:15, 12.51s/it]
  2%|▏         | 5505/225000 [20:50:26<764:42:07, 12.54s/it]
  2%|▏         | 5506/225000 [20:50:30<617:11:44, 10.12s/it]
  2%|▏         | 5507/225000 [20:50:39<593:01:54,  9.73s/it]
  2%|▏         | 5508/225000 [20:50:47<560:13:02,  9.19s/it]
  2%|▏         | 5509/225000 [20:51:04<697:30:36, 11.44s/it]
  2%|▏         | 5510/225000 [20:51:10<599:50:00,  9.84s/it]
  2%|▏         | 5511/225000 [20:51:15<512:27:32,  8.41s/it]
  2%|▏         | 5512/225000 [20:51:26<567:17:41,  9.30s/it]
  2%|▏         | 5513/225000 [20:51:38<602:54:48,  9.89s/it]
  2%|▏         | 5514/225000 [20:51:50<650:13:01, 10.66s/it]
  2%|▏         | 5515/225000 [20:52:04<714:28:23, 11.72s/it]
  2%|▏         | 5516/225000 [20:52:16<722:09:11, 11.84s/it]
  2%|▏         | 5517/225000 [20:52:21<598:39:41,  9.82s/it]
  2%|▏         | 5518/225000 [20:52:38<724:34:38, 11.88s/it]
  2%|▏         | 5519/225000 [20:52:50<726:50:38, 11.92s/it]
  2%|▏         | 5520/225000 [20:53:01<706:18:53, 11.59s/it]
  2%|▏         | 5521/225000 [20:53:08<626:36:31, 10.28s/it]
  2%|▏         | 5522/225000 [20:53:14<544:11:40,  8.93s/it]
  2%|▏         | 5523/225000 [20:53:20<484:56:09,  7.95s/it]
  2%|▏         | 5524/225000 [20:53:25<429:37:23,  7.05s/it]
  2%|▏         | 5525/225000 [20:53:36<502:35:24,  8.24s/it]
  2%|▏         | 5526/225000 [20:53:45<526:48:43,  8.64s/it]
  2%|▏         | 5527/225000 [20:53:54<531:44:41,  8.72s/it]
  2%|▏         | 5528/225000 [20:54:05<564:24:38,  9.26s/it]
  2%|▏         | 5529/225000 [20:54:10<495:56:38,  8.14s/it]
  2%|▏         | 5530/225000 [20:54:18<486:39:17,  7.98s/it]
  2%|▏         | 5531/225000 [20:54:29<553:48:24,  9.08s/it]
  2%|▏         | 5532/225000 [20:54:35<491:25:49,  8.06s/it]
  2%|▏         | 5533/225000 [20:54:45<528:39:58,  8.67s/it]
  2%|▏         | 5534/225000 [20:54:54<536:06:20,  8.79s/it]
  2%|▏         | 5535/225000 [20:55:03<538:53:08,  8.84s/it]
  2%|▏         | 5536/225000 [20:55:12<545:44:30,  8.95s/it]
  2%|▏         | 5537/225000 [20:55:20<523:05:05,  8.58s/it]
  2%|▏         | 5538/225000 [20:55:27<487:14:15,  7.99s/it]
  2%|▏         | 5539/225000 [20:55:36<517:03:26,  8.48s/it]
  2%|▏         | 5540/225000 [20:55:44<496:52:49,  8.15s/it]
  2%|▏         | 5541/225000 [20:55:53<524:49:30,  8.61s/it]
  2%|▏         | 5542/225000 [20:56:13<729:49:33, 11.97s/it]
  2%|▏         | 5543/225000 [20:56:24<704:25:42, 11.56s/it]
  2%|▏         | 5544/225000 [20:56:37<734:07:47, 12.04s/it]
  2%|▏         | 5545/225000 [20:56:46<684:02:07, 11.22s/it]
  2%|▏         | 5546/225000 [20:56:53<600:11:53,  9.85s/it]
  2%|▏         | 5547/225000 [20:57:03<604:05:25,  9.91s/it]
  2%|▏         | 5548/225000 [20:57:09<541:08:29,  8.88s/it]
  2%|▏         | 5549/225000 [20:57:17<515:32:40,  8.46s/it]
  2%|▏         | 5550/225000 [20:57:31<612:17:39, 10.04s/it]
                                                            
{'loss': '0.7985', 'grad_norm': '1.585', 'learning_rate': '2.772e-05', 'epoch': '0.02467', 'num_input_tokens_seen': 38406232, 'train_runtime': '7.545e+04', 'train_tokens_per_second': '509'}

  2%|▏         | 5550/225000 [20:57:31<612:17:39, 10.04s/it]
  2%|▏         | 5551/225000 [20:57:38<568:24:03,  9.32s/it]
  2%|▏         | 5552/225000 [20:57:44<509:36:40,  8.36s/it]
  2%|▏         | 5553/225000 [20:57:57<588:24:42,  9.65s/it]
  2%|▏         | 5554/225000 [20:58:10<655:46:35, 10.76s/it]
  2%|▏         | 5555/225000 [20:58:15<548:01:03,  8.99s/it]
  2%|▏         | 5556/225000 [20:58:25<558:34:50,  9.16s/it]
  2%|▏         | 5557/225000 [20:58:32<512:28:23,  8.41s/it]
  2%|▏         | 5558/225000 [20:58:37<458:08:19,  7.52s/it]
  2%|▏         | 5559/225000 [20:58:42<406:14:21,  6.66s/it]
  2%|▏         | 5560/225000 [20:58:58<586:12:30,  9.62s/it]
  2%|▏         | 5561/225000 [20:59:10<619:18:23, 10.16s/it]
  2%|▏         | 5562/225000 [20:59:19<608:22:34,  9.98s/it]
  2%|▏         | 5563/225000 [20:59:26<551:00:21,  9.04s/it]
  2%|▏         | 5564/225000 [20:59:42<682:38:38, 11.20s/it]
  2%|▏         | 5565/225000 [20:59:54<694:25:54, 11.39s/it]
  2%|▏         | 5566/225000 [21:00:02<636:34:24, 10.44s/it]
  2%|▏         | 5567/225000 [21:00:09<563:14:12,  9.24s/it]
  2%|▏         | 5568/225000 [21:00:20<608:47:40,  9.99s/it]
  2%|▏         | 5569/225000 [21:00:27<543:05:32,  8.91s/it]
  2%|▏         | 5570/225000 [21:00:42<657:32:18, 10.79s/it]
  2%|▏         | 5571/225000 [21:00:49<582:56:57,  9.56s/it]
  2%|▏         | 5572/225000 [21:00:56<548:20:32,  9.00s/it]
  2%|▏         | 5573/225000 [21:01:11<646:18:53, 10.60s/it]
  2%|▏         | 5574/225000 [21:01:18<576:23:23,  9.46s/it]
  2%|▏         | 5575/225000 [21:01:26<560:48:59,  9.20s/it]
  2%|▏         | 5576/225000 [21:01:31<477:14:01,  7.83s/it]
  2%|▏         | 5577/225000 [21:01:46<613:00:28, 10.06s/it]
  2%|▏         | 5578/225000 [21:01:50<500:25:16,  8.21s/it]
  2%|▏         | 5579/225000 [21:02:00<527:21:48,  8.65s/it]
  2%|▏         | 5580/225000 [21:02:13<618:36:27, 10.15s/it]
  2%|▏         | 5581/225000 [21:02:22<585:43:10,  9.61s/it]
  2%|▏         | 5582/225000 [21:02:27<517:53:38,  8.50s/it]
  2%|▏         | 5583/225000 [21:02:36<525:22:00,  8.62s/it]
  2%|▏         | 5584/225000 [21:02:47<553:14:23,  9.08s/it]
  2%|▏         | 5585/225000 [21:02:59<617:50:07, 10.14s/it]
  2%|▏         | 5586/225000 [21:03:10<627:21:45, 10.29s/it]
  2%|▏         | 5587/225000 [21:03:20<626:17:15, 10.28s/it]
  2%|▏         | 5588/225000 [21:03:31<634:17:41, 10.41s/it]
  2%|▏         | 5589/225000 [21:03:49<784:55:39, 12.88s/it]
  2%|▏         | 5590/225000 [21:03:53<624:28:15, 10.25s/it]
  2%|▏         | 5591/225000 [21:03:59<540:03:20,  8.86s/it]
  2%|▏         | 5592/225000 [21:04:04<474:32:14,  7.79s/it]
  2%|▏         | 5593/225000 [21:04:20<613:46:25, 10.07s/it]
  2%|▏         | 5594/225000 [21:04:31<629:51:21, 10.33s/it]
  2%|▏         | 5595/225000 [21:04:39<591:48:04,  9.71s/it]
  2%|▏         | 5596/225000 [21:04:45<527:01:13,  8.65s/it]
  2%|▏         | 5597/225000 [21:04:51<480:57:29,  7.89s/it]
  2%|▏         | 5598/225000 [21:05:03<541:53:28,  8.89s/it]
  2%|▏         | 5599/225000 [21:05:15<611:20:46, 10.03s/it]
  2%|▏         | 5600/225000 [21:05:21<532:14:50,  8.73s/it]
                                                            
{'loss': '0.7866', 'grad_norm': '0.9488', 'learning_rate': '2.798e-05', 'epoch': '0.02489', 'num_input_tokens_seen': 38644808, 'train_runtime': '7.592e+04', 'train_tokens_per_second': '509'}

  2%|▏         | 5600/225000 [21:05:21<532:14:50,  8.73s/it]
  2%|▏         | 5601/225000 [21:05:29<512:52:18,  8.42s/it]
  2%|▏         | 5602/225000 [21:05:43<624:18:08, 10.24s/it]
  2%|▏         | 5603/225000 [21:05:59<724:38:16, 11.89s/it]
  2%|▏         | 5604/225000 [21:06:02<568:50:28,  9.33s/it]
  2%|▏         | 5605/225000 [21:06:14<617:34:06, 10.13s/it]
  2%|▏         | 5606/225000 [21:06:25<627:55:03, 10.30s/it]
  2%|▏         | 5607/225000 [21:06:37<652:01:52, 10.70s/it]
  2%|▏         | 5608/225000 [21:06:43<579:47:42,  9.51s/it]
  2%|▏         | 5609/225000 [21:06:58<668:15:24, 10.97s/it]
  2%|▏         | 5610/225000 [21:07:09<676:54:08, 11.11s/it]
  2%|▏         | 5611/225000 [21:07:15<581:56:22,  9.55s/it]
  2%|▏         | 5612/225000 [21:07:26<615:56:26, 10.11s/it]
  2%|▏         | 5613/225000 [21:07:35<584:39:37,  9.59s/it]
  2%|▏         | 5614/225000 [21:07:41<531:09:01,  8.72s/it]
  2%|▏         | 5615/225000 [21:07:47<466:20:41,  7.65s/it]
  2%|▏         | 5616/225000 [21:07:59<547:32:22,  8.98s/it]
  2%|▏         | 5617/225000 [21:08:09<568:16:39,  9.33s/it]
  2%|▏         | 5618/225000 [21:08:18<563:51:23,  9.25s/it]
  2%|▏         | 5619/225000 [21:08:28<575:11:19,  9.44s/it]
  2%|▏         | 5620/225000 [21:08:33<501:47:04,  8.23s/it]
  2%|▏         | 5621/225000 [21:08:45<559:30:29,  9.18s/it]
  2%|▏         | 5622/225000 [21:08:52<522:44:05,  8.58s/it]
  2%|▏         | 5623/225000 [21:09:01<530:58:30,  8.71s/it]
  2%|▏         | 5624/225000 [21:09:12<581:40:38,  9.55s/it]
  2%|▎         | 5625/225000 [21:09:23<606:11:02,  9.95s/it]
  3%|▎         | 5626/225000 [21:09:41<747:00:44, 12.26s/it]
  3%|▎         | 5627/225000 [21:09:48<656:12:47, 10.77s/it]
  3%|▎         | 5628/225000 [21:09:53<547:53:20,  8.99s/it]
  3%|▎         | 5629/225000 [21:09:58<476:56:29,  7.83s/it]
  3%|▎         | 5630/225000 [21:10:14<618:04:30, 10.14s/it]
  3%|▎         | 5631/225000 [21:10:22<588:02:42,  9.65s/it]
  3%|▎         | 5632/225000 [21:10:35<645:54:18, 10.60s/it]
  3%|▎         | 5633/225000 [21:10:46<648:07:57, 10.64s/it]
  3%|▎         | 5634/225000 [21:10:57<665:39:36, 10.92s/it]
  3%|▎         | 5635/225000 [21:11:02<547:06:42,  8.98s/it]
  3%|▎         | 5636/225000 [21:11:11<548:37:05,  9.00s/it]
  3%|▎         | 5637/225000 [21:11:17<493:12:37,  8.09s/it]
  3%|▎         | 5638/225000 [21:11:29<572:35:36,  9.40s/it]
  3%|▎         | 5639/225000 [21:11:40<600:22:30,  9.85s/it]
  3%|▎         | 5640/225000 [21:11:47<549:42:03,  9.02s/it]
  3%|▎         | 5641/225000 [21:11:53<493:31:29,  8.10s/it]
  3%|▎         | 5642/225000 [21:11:59<456:36:19,  7.49s/it]
  3%|▎         | 5643/225000 [21:12:04<414:44:11,  6.81s/it]
  3%|▎         | 5644/225000 [21:12:11<419:57:02,  6.89s/it]
  3%|▎         | 5645/225000 [21:12:27<571:39:17,  9.38s/it]
  3%|▎         | 5646/225000 [21:12:39<617:36:49, 10.14s/it]
  3%|▎         | 5647/225000 [21:12:49<621:19:32, 10.20s/it]
  3%|▎         | 5648/225000 [21:12:54<522:12:38,  8.57s/it]
  3%|▎         | 5649/225000 [21:13:02<509:24:06,  8.36s/it]
  3%|▎         | 5650/225000 [21:13:15<605:50:07,  9.94s/it]
                                                            
{'loss': '0.7746', 'grad_norm': '1.64', 'learning_rate': '2.823e-05', 'epoch': '0.02511', 'num_input_tokens_seen': 38878960, 'train_runtime': '7.64e+04', 'train_tokens_per_second': '508.9'}

  3%|▎         | 5650/225000 [21:13:15<605:50:07,  9.94s/it]
  3%|▎         | 5651/225000 [21:13:24<581:12:35,  9.54s/it]
  3%|▎         | 5652/225000 [21:13:33<574:55:50,  9.44s/it]
  3%|▎         | 5653/225000 [21:13:40<527:14:45,  8.65s/it]
  3%|▎         | 5654/225000 [21:13:50<551:30:09,  9.05s/it]
  3%|▎         | 5655/225000 [21:13:59<552:08:42,  9.06s/it]
  3%|▎         | 5656/225000 [21:14:11<609:44:50, 10.01s/it]
  3%|▎         | 5657/225000 [21:14:20<588:13:52,  9.65s/it]
  3%|▎         | 5658/225000 [21:14:32<624:51:51, 10.26s/it]
  3%|▎         | 5659/225000 [21:14:36<514:54:24,  8.45s/it]
  3%|▎         | 5660/225000 [21:14:51<644:30:48, 10.58s/it]
  3%|▎         | 5661/225000 [21:15:04<685:13:58, 11.25s/it]
  3%|▎         | 5662/225000 [21:15:15<669:42:31, 10.99s/it]
  3%|▎         | 5663/225000 [21:15:21<592:05:13,  9.72s/it]
  3%|▎         | 5664/225000 [21:15:30<571:21:38,  9.38s/it]
  3%|▎         | 5665/225000 [21:15:41<611:31:17, 10.04s/it]
  3%|▎         | 5666/225000 [21:15:47<536:06:28,  8.80s/it]
  3%|▎         | 5667/225000 [21:15:55<519:30:53,  8.53s/it]
  3%|▎         | 5668/225000 [21:16:03<496:57:44,  8.16s/it]
  3%|▎         | 5669/225000 [21:16:19<647:36:50, 10.63s/it]
  3%|▎         | 5670/225000 [21:16:26<575:40:34,  9.45s/it]
  3%|▎         | 5671/225000 [21:16:31<496:40:37,  8.15s/it]
  3%|▎         | 5672/225000 [21:16:48<659:31:45, 10.83s/it]
  3%|▎         | 5673/225000 [21:17:03<738:36:29, 12.12s/it]
  3%|▎         | 5674/225000 [21:17:16<750:39:52, 12.32s/it]
  3%|▎         | 5675/225000 [21:17:22<635:24:23, 10.43s/it]
  3%|▎         | 5676/225000 [21:17:29<572:18:18,  9.39s/it]
  3%|▎         | 5677/225000 [21:17:42<644:50:35, 10.58s/it]
  3%|▎         | 5678/225000 [21:17:51<621:34:21, 10.20s/it]
  3%|▎         | 5679/225000 [21:18:02<634:08:38, 10.41s/it]
  3%|▎         | 5680/225000 [21:18:16<686:10:14, 11.26s/it]
  3%|▎         | 5681/225000 [21:18:27<689:22:23, 11.32s/it]
  3%|▎         | 5682/225000 [21:18:42<763:31:23, 12.53s/it]
  3%|▎         | 5683/225000 [21:18:48<642:33:29, 10.55s/it]
  3%|▎         | 5684/225000 [21:18:54<554:01:10,  9.09s/it]
  3%|▎         | 5685/225000 [21:19:03<559:38:09,  9.19s/it]
  3%|▎         | 5686/225000 [21:19:10<508:36:57,  8.35s/it]
  3%|▎         | 5687/225000 [21:19:22<579:54:09,  9.52s/it]
  3%|▎         | 5688/225000 [21:19:36<666:44:57, 10.94s/it]
  3%|▎         | 5689/225000 [21:19:44<606:29:48,  9.96s/it]
  3%|▎         | 5690/225000 [21:19:52<565:36:49,  9.28s/it]
  3%|▎         | 5691/225000 [21:19:57<493:24:21,  8.10s/it]
  3%|▎         | 5692/225000 [21:20:06<509:54:55,  8.37s/it]
  3%|▎         | 5693/225000 [21:20:21<628:50:12, 10.32s/it]
  3%|▎         | 5694/225000 [21:20:30<607:49:51,  9.98s/it]
  3%|▎         | 5695/225000 [21:20:43<654:39:10, 10.75s/it]
  3%|▎         | 5696/225000 [21:20:49<571:44:57,  9.39s/it]
  3%|▎         | 5697/225000 [21:20:55<515:06:58,  8.46s/it]
  3%|▎         | 5698/225000 [21:21:11<648:39:00, 10.65s/it]
  3%|▎         | 5699/225000 [21:21:15<528:16:55,  8.67s/it]
  3%|▎         | 5700/225000 [21:21:24<540:32:46,  8.87s/it]
                                                            
{'loss': '0.7792', 'grad_norm': '1.328', 'learning_rate': '2.848e-05', 'epoch': '0.02533', 'num_input_tokens_seen': 39126832, 'train_runtime': '7.688e+04', 'train_tokens_per_second': '508.9'}

  3%|▎         | 5700/225000 [21:21:24<540:32:46,  8.87s/it]
  3%|▎         | 5701/225000 [21:21:31<507:32:32,  8.33s/it]
  3%|▎         | 5702/225000 [21:21:41<528:16:14,  8.67s/it]
  3%|▎         | 5703/225000 [21:21:45<436:57:47,  7.17s/it]
  3%|▎         | 5704/225000 [21:21:57<537:06:32,  8.82s/it]
  3%|▎         | 5705/225000 [21:22:04<503:33:36,  8.27s/it]
  3%|▎         | 5706/225000 [21:22:10<462:02:48,  7.59s/it]
  3%|▎         | 5707/225000 [21:22:29<663:04:02, 10.89s/it]
  3%|▎         | 5708/225000 [21:22:39<650:55:14, 10.69s/it]
  3%|▎         | 5709/225000 [21:22:47<608:05:19,  9.98s/it]
  3%|▎         | 5710/225000 [21:22:52<513:36:08,  8.43s/it]
  3%|▎         | 5711/225000 [21:23:09<666:23:11, 10.94s/it]
  3%|▎         | 5712/225000 [21:23:20<671:50:31, 11.03s/it]
  3%|▎         | 5713/225000 [21:23:32<694:15:43, 11.40s/it]
  3%|▎         | 5714/225000 [21:23:36<557:42:52,  9.16s/it]
  3%|▎         | 5715/225000 [21:23:46<565:54:57,  9.29s/it]
  3%|▎         | 5716/225000 [21:23:54<543:55:42,  8.93s/it]
  3%|▎         | 5717/225000 [21:24:04<566:57:18,  9.31s/it]
  3%|▎         | 5718/225000 [21:24:12<546:21:24,  8.97s/it]
  3%|▎         | 5719/225000 [21:24:23<580:01:02,  9.52s/it]
  3%|▎         | 5720/225000 [21:24:36<633:00:06, 10.39s/it]
  3%|▎         | 5721/225000 [21:24:42<564:03:52,  9.26s/it]
  3%|▎         | 5722/225000 [21:24:53<592:58:44,  9.74s/it]
  3%|▎         | 5723/225000 [21:25:00<548:09:21,  9.00s/it]
  3%|▎         | 5724/225000 [21:25:14<631:28:52, 10.37s/it]
  3%|▎         | 5725/225000 [21:25:19<542:03:41,  8.90s/it]
  3%|▎         | 5726/225000 [21:25:28<537:47:36,  8.83s/it]
  3%|▎         | 5727/225000 [21:25:40<594:52:24,  9.77s/it]
  3%|▎         | 5728/225000 [21:25:46<534:41:29,  8.78s/it]
  3%|▎         | 5729/225000 [21:26:01<635:48:09, 10.44s/it]
  3%|▎         | 5730/225000 [21:26:05<527:33:05,  8.66s/it]
  3%|▎         | 5731/225000 [21:26:16<573:21:00,  9.41s/it]
  3%|▎         | 5732/225000 [21:26:26<573:48:17,  9.42s/it]
  3%|▎         | 5733/225000 [21:26:34<555:10:56,  9.12s/it]
  3%|▎         | 5734/225000 [21:26:46<600:29:38,  9.86s/it]
  3%|▎         | 5735/225000 [21:26:55<585:07:55,  9.61s/it]
  3%|▎         | 5736/225000 [21:27:02<538:13:50,  8.84s/it]
  3%|▎         | 5737/225000 [21:27:09<499:13:46,  8.20s/it]
  3%|▎         | 5738/225000 [21:27:25<653:25:51, 10.73s/it]
  3%|▎         | 5739/225000 [21:27:35<627:36:54, 10.30s/it]
  3%|▎         | 5740/225000 [21:27:47<664:27:06, 10.91s/it]
  3%|▎         | 5741/225000 [21:27:54<601:01:22,  9.87s/it]
  3%|▎         | 5742/225000 [21:28:09<682:48:36, 11.21s/it]
  3%|▎         | 5743/225000 [21:28:20<674:48:57, 11.08s/it]
  3%|▎         | 5744/225000 [21:28:32<701:12:58, 11.51s/it]
  3%|▎         | 5745/225000 [21:28:37<586:03:44,  9.62s/it]
  3%|▎         | 5746/225000 [21:28:49<625:29:30, 10.27s/it]
  3%|▎         | 5747/225000 [21:28:57<588:43:34,  9.67s/it]
  3%|▎         | 5748/225000 [21:29:09<627:32:31, 10.30s/it]
  3%|▎         | 5749/225000 [21:29:23<698:14:42, 11.46s/it]
  3%|▎         | 5750/225000 [21:29:36<721:22:02, 11.84s/it]
                                                            
{'loss': '0.7854', 'grad_norm': '1.123', 'learning_rate': '2.873e-05', 'epoch': '0.02556', 'num_input_tokens_seen': 39364560, 'train_runtime': '7.738e+04', 'train_tokens_per_second': '508.7'}

  3%|▎         | 5750/225000 [21:29:36<721:22:02, 11.84s/it]
  3%|▎         | 5751/225000 [21:29:44<644:39:00, 10.58s/it]
  3%|▎         | 5752/225000 [21:29:50<567:29:26,  9.32s/it]
  3%|▎         | 5753/225000 [21:30:03<628:03:24, 10.31s/it]
  3%|▎         | 5754/225000 [21:30:11<586:11:37,  9.63s/it]
  3%|▎         | 5755/225000 [21:30:20<586:20:57,  9.63s/it]
  3%|▎         | 5756/225000 [21:30:27<535:43:58,  8.80s/it]
  3%|▎         | 5757/225000 [21:30:38<570:18:18,  9.36s/it]
  3%|▎         | 5758/225000 [21:30:42<482:06:07,  7.92s/it]
  3%|▎         | 5759/225000 [21:30:52<519:06:12,  8.52s/it]
  3%|▎         | 5760/225000 [21:30:58<466:45:38,  7.66s/it]
  3%|▎         | 5761/225000 [21:31:07<486:09:57,  7.98s/it]
  3%|▎         | 5762/225000 [21:31:19<564:28:04,  9.27s/it]
  3%|▎         | 5763/225000 [21:31:27<541:41:29,  8.89s/it]
  3%|▎         | 5764/225000 [21:31:38<585:31:24,  9.61s/it]
  3%|▎         | 5765/225000 [21:31:47<562:31:19,  9.24s/it]
  3%|▎         | 5766/225000 [21:31:58<594:21:37,  9.76s/it]
  3%|▎         | 5767/225000 [21:32:03<518:19:03,  8.51s/it]
  3%|▎         | 5768/225000 [21:32:16<587:33:45,  9.65s/it]
  3%|▎         | 5769/225000 [21:32:22<530:36:25,  8.71s/it]
  3%|▎         | 5770/225000 [21:32:34<587:18:39,  9.64s/it]
  3%|▎         | 5771/225000 [21:32:42<558:52:30,  9.18s/it]
  3%|▎         | 5772/225000 [21:32:55<633:20:36, 10.40s/it]
  3%|▎         | 5773/225000 [21:33:00<536:42:15,  8.81s/it]
  3%|▎         | 5774/225000 [21:33:05<462:15:32,  7.59s/it]
  3%|▎         | 5775/225000 [21:33:17<543:49:00,  8.93s/it]
  3%|▎         | 5776/225000 [21:33:30<621:12:04, 10.20s/it]
  3%|▎         | 5777/225000 [21:33:38<584:25:44,  9.60s/it]
  3%|▎         | 5778/225000 [21:33:50<619:20:56, 10.17s/it]
  3%|▎         | 5779/225000 [21:33:59<599:11:37,  9.84s/it]
  3%|▎         | 5780/225000 [21:34:10<624:39:16, 10.26s/it]
  3%|▎         | 5781/225000 [21:34:21<641:36:42, 10.54s/it]
  3%|▎         | 5782/225000 [21:34:30<606:40:24,  9.96s/it]
  3%|▎         | 5783/225000 [21:34:42<651:08:37, 10.69s/it]
  3%|▎         | 5784/225000 [21:34:50<599:52:48,  9.85s/it]
  3%|▎         | 5785/225000 [21:34:57<547:30:34,  8.99s/it]
  3%|▎         | 5786/225000 [21:35:12<650:53:37, 10.69s/it]
  3%|▎         | 5787/225000 [21:35:21<628:54:36, 10.33s/it]
  3%|▎         | 5788/225000 [21:35:35<689:47:04, 11.33s/it]
  3%|▎         | 5789/225000 [21:35:49<738:41:33, 12.13s/it]
  3%|▎         | 5790/225000 [21:35:53<585:59:23,  9.62s/it]
  3%|▎         | 5791/225000 [21:36:05<624:25:24, 10.25s/it]
  3%|▎         | 5792/225000 [21:36:18<674:47:36, 11.08s/it]
  3%|▎         | 5793/225000 [21:36:27<647:36:03, 10.64s/it]
  3%|▎         | 5794/225000 [21:36:40<685:11:10, 11.25s/it]
  3%|▎         | 5795/225000 [21:36:50<660:23:18, 10.85s/it]
  3%|▎         | 5796/225000 [21:36:57<588:50:49,  9.67s/it]
  3%|▎         | 5797/225000 [21:37:04<550:52:03,  9.05s/it]
  3%|▎         | 5798/225000 [21:37:12<515:51:48,  8.47s/it]
  3%|▎         | 5799/225000 [21:37:19<499:17:00,  8.20s/it]
  3%|▎         | 5800/225000 [21:37:26<478:33:02,  7.86s/it]
                                                            
{'loss': '0.7662', 'grad_norm': '1.342', 'learning_rate': '2.898e-05', 'epoch': '0.02578', 'num_input_tokens_seen': 39605912, 'train_runtime': '7.785e+04', 'train_tokens_per_second': '508.8'}

  3%|▎         | 5800/225000 [21:37:26<478:33:02,  7.86s/it]
  3%|▎         | 5801/225000 [21:37:33<458:28:35,  7.53s/it]
  3%|▎         | 5802/225000 [21:37:41<471:50:41,  7.75s/it]
  3%|▎         | 5803/225000 [21:37:51<505:45:03,  8.31s/it]
  3%|▎         | 5804/225000 [21:38:00<521:41:33,  8.57s/it]
  3%|▎         | 5805/225000 [21:38:05<458:49:57,  7.54s/it]
  3%|▎         | 5806/225000 [21:38:19<568:23:51,  9.34s/it]
  3%|▎         | 5807/225000 [21:38:33<660:56:09, 10.86s/it]
  3%|▎         | 5808/225000 [21:38:40<589:06:29,  9.68s/it]
  3%|▎         | 5809/225000 [21:38:46<519:08:03,  8.53s/it]
  3%|▎         | 5810/225000 [21:39:01<639:01:03, 10.50s/it]
  3%|▎         | 5811/225000 [21:39:14<689:51:36, 11.33s/it]
  3%|▎         | 5812/225000 [21:39:26<694:37:57, 11.41s/it]
  3%|▎         | 5813/225000 [21:39:32<603:56:03,  9.92s/it]
  3%|▎         | 5814/225000 [21:39:44<630:53:07, 10.36s/it]
  3%|▎         | 5815/225000 [21:39:53<618:21:05, 10.16s/it]
  3%|▎         | 5816/225000 [21:40:05<644:50:56, 10.59s/it]
  3%|▎         | 5817/225000 [21:40:10<553:04:17,  9.08s/it]
  3%|▎         | 5818/225000 [21:40:22<595:34:11,  9.78s/it]
  3%|▎         | 5819/225000 [21:40:30<570:47:22,  9.38s/it]
  3%|▎         | 5820/225000 [21:40:36<509:45:02,  8.37s/it]
  3%|▎         | 5821/225000 [21:40:45<512:31:40,  8.42s/it]
  3%|▎         | 5822/225000 [21:40:52<498:32:20,  8.19s/it]
  3%|▎         | 5823/225000 [21:41:04<555:31:21,  9.12s/it]
  3%|▎         | 5824/225000 [21:41:12<536:58:14,  8.82s/it]
  3%|▎         | 5825/225000 [21:41:24<591:54:38,  9.72s/it]
  3%|▎         | 5826/225000 [21:41:33<589:14:41,  9.68s/it]
  3%|▎         | 5827/225000 [21:41:39<517:59:42,  8.51s/it]
  3%|▎         | 5828/225000 [21:41:45<472:16:23,  7.76s/it]
  3%|▎         | 5829/225000 [21:41:58<557:59:59,  9.17s/it]
  3%|▎         | 5830/225000 [21:42:03<496:24:24,  8.15s/it]
  3%|▎         | 5831/225000 [21:42:10<467:50:33,  7.68s/it]
  3%|▎         | 5832/225000 [21:42:15<415:08:24,  6.82s/it]
  3%|▎         | 5833/225000 [21:42:30<564:45:30,  9.28s/it]
  3%|▎         | 5834/225000 [21:42:39<567:47:00,  9.33s/it]
  3%|▎         | 5835/225000 [21:42:45<495:24:04,  8.14s/it]
  3%|▎         | 5836/225000 [21:42:59<613:51:28, 10.08s/it]
  3%|▎         | 5837/225000 [21:43:05<534:56:53,  8.79s/it]
  3%|▎         | 5838/225000 [21:43:13<529:19:11,  8.69s/it]
  3%|▎         | 5839/225000 [21:43:20<499:29:33,  8.20s/it]
  3%|▎         | 5840/225000 [21:43:30<520:01:04,  8.54s/it]
  3%|▎         | 5841/225000 [21:43:42<595:07:56,  9.78s/it]
  3%|▎         | 5842/225000 [21:43:48<524:30:23,  8.62s/it]
  3%|▎         | 5843/225000 [21:43:59<561:17:57,  9.22s/it]
  3%|▎         | 5844/225000 [21:44:05<504:40:16,  8.29s/it]
  3%|▎         | 5845/225000 [21:44:21<637:10:51, 10.47s/it]
  3%|▎         | 5846/225000 [21:44:32<662:37:47, 10.88s/it]
  3%|▎         | 5847/225000 [21:44:42<645:28:35, 10.60s/it]
  3%|▎         | 5848/225000 [21:44:54<671:36:09, 11.03s/it]
  3%|▎         | 5849/225000 [21:45:03<634:27:46, 10.42s/it]
  3%|▎         | 5850/225000 [21:45:10<560:37:45,  9.21s/it]
                                                            
{'loss': '0.7408', 'grad_norm': '1.429', 'learning_rate': '2.923e-05', 'epoch': '0.026', 'num_input_tokens_seen': 39835808, 'train_runtime': '7.831e+04', 'train_tokens_per_second': '508.7'}

  3%|▎         | 5850/225000 [21:45:10<560:37:45,  9.21s/it]
  3%|▎         | 5851/225000 [21:45:24<650:03:14, 10.68s/it]
  3%|▎         | 5852/225000 [21:45:31<587:52:04,  9.66s/it]
  3%|▎         | 5853/225000 [21:45:40<572:24:04,  9.40s/it]
  3%|▎         | 5854/225000 [21:45:46<515:30:55,  8.47s/it]
  3%|▎         | 5855/225000 [21:45:52<455:21:58,  7.48s/it]
  3%|▎         | 5856/225000 [21:45:58<428:14:22,  7.03s/it]
  3%|▎         | 5857/225000 [21:46:10<533:20:40,  8.76s/it]
  3%|▎         | 5858/225000 [21:46:25<633:13:58, 10.40s/it]
  3%|▎         | 5859/225000 [21:46:31<557:13:48,  9.15s/it]
  3%|▎         | 5860/225000 [21:46:39<533:30:06,  8.76s/it]
  3%|▎         | 5861/225000 [21:46:44<480:36:23,  7.90s/it]
  3%|▎         | 5862/225000 [21:46:51<451:21:59,  7.42s/it]
  3%|▎         | 5863/225000 [21:47:01<511:01:37,  8.40s/it]
  3%|▎         | 5864/225000 [21:47:11<537:18:13,  8.83s/it]
  3%|▎         | 5865/225000 [21:47:21<552:31:25,  9.08s/it]
  3%|▎         | 5866/225000 [21:47:25<464:18:47,  7.63s/it]
  3%|▎         | 5867/225000 [21:47:37<532:41:39,  8.75s/it]
  3%|▎         | 5868/225000 [21:47:48<586:43:08,  9.64s/it]
  3%|▎         | 5869/225000 [21:47:57<564:17:34,  9.27s/it]
  3%|▎         | 5870/225000 [21:48:03<513:51:31,  8.44s/it]
  3%|▎         | 5871/225000 [21:48:11<505:51:12,  8.31s/it]
  3%|▎         | 5872/225000 [21:48:23<568:00:11,  9.33s/it]
  3%|▎         | 5873/225000 [21:48:28<497:05:27,  8.17s/it]
  3%|▎         | 5874/225000 [21:48:40<561:15:14,  9.22s/it]
  3%|▎         | 5875/225000 [21:48:45<490:46:44,  8.06s/it]
  3%|▎         | 5876/225000 [21:48:54<503:36:44,  8.27s/it]
  3%|▎         | 5877/225000 [21:49:00<457:36:06,  7.52s/it]
  3%|▎         | 5878/225000 [21:49:11<516:26:03,  8.48s/it]
  3%|▎         | 5879/225000 [21:49:23<591:49:40,  9.72s/it]
  3%|▎         | 5880/225000 [21:49:29<517:45:17,  8.51s/it]
  3%|▎         | 5881/225000 [21:49:39<538:12:16,  8.84s/it]
  3%|▎         | 5882/225000 [21:49:48<542:26:28,  8.91s/it]
  3%|▎         | 5883/225000 [21:49:57<558:54:33,  9.18s/it]
  3%|▎         | 5884/225000 [21:50:04<519:15:59,  8.53s/it]
  3%|▎         | 5885/225000 [21:50:20<645:41:15, 10.61s/it]
  3%|▎         | 5886/225000 [21:50:28<603:10:36,  9.91s/it]
  3%|▎         | 5887/225000 [21:50:33<509:10:58,  8.37s/it]
  3%|▎         | 5888/225000 [21:50:50<670:55:01, 11.02s/it]
  3%|▎         | 5889/225000 [21:50:58<613:46:48, 10.08s/it]
  3%|▎         | 5890/225000 [21:51:15<746:39:39, 12.27s/it]
  3%|▎         | 5891/225000 [21:51:26<706:41:43, 11.61s/it]
  3%|▎         | 5892/225000 [21:51:34<643:15:00, 10.57s/it]
  3%|▎         | 5893/225000 [21:51:42<608:33:36, 10.00s/it]
  3%|▎         | 5894/225000 [21:51:54<632:37:02, 10.39s/it]
  3%|▎         | 5895/225000 [21:52:01<571:32:46,  9.39s/it]
  3%|▎         | 5896/225000 [21:52:14<640:49:39, 10.53s/it]
  3%|▎         | 5897/225000 [21:52:26<672:23:42, 11.05s/it]
  3%|▎         | 5898/225000 [21:52:33<591:09:44,  9.71s/it]
  3%|▎         | 5899/225000 [21:52:52<770:46:36, 12.66s/it]
  3%|▎         | 5900/225000 [21:52:59<657:25:16, 10.80s/it]
                                                            
{'loss': '0.7595', 'grad_norm': '1.436', 'learning_rate': '2.948e-05', 'epoch': '0.02622', 'num_input_tokens_seen': 40062624, 'train_runtime': '7.878e+04', 'train_tokens_per_second': '508.5'}

  3%|▎         | 5900/225000 [21:52:59<657:25:16, 10.80s/it]
  3%|▎         | 5901/225000 [21:53:09<643:41:26, 10.58s/it]
  3%|▎         | 5902/225000 [21:53:16<586:33:28,  9.64s/it]
  3%|▎         | 5903/225000 [21:53:32<699:58:04, 11.50s/it]
  3%|▎         | 5904/225000 [21:53:44<698:48:59, 11.48s/it]
  3%|▎         | 5905/225000 [21:53:47<559:16:04,  9.19s/it]
  3%|▎         | 5906/225000 [21:53:54<518:56:46,  8.53s/it]
  3%|▎         | 5907/225000 [21:54:00<474:41:45,  7.80s/it]
  3%|▎         | 5908/225000 [21:54:15<600:41:02,  9.87s/it]
  3%|▎         | 5909/225000 [21:54:19<496:47:41,  8.16s/it]
  3%|▎         | 5910/225000 [21:54:28<511:36:56,  8.41s/it]
  3%|▎         | 5911/225000 [21:54:37<520:05:05,  8.55s/it]
  3%|▎         | 5912/225000 [21:54:49<572:27:35,  9.41s/it]
  3%|▎         | 5913/225000 [21:54:56<528:33:03,  8.69s/it]
  3%|▎         | 5914/225000 [21:55:09<609:33:34, 10.02s/it]
  3%|▎         | 5915/225000 [21:55:20<626:43:05, 10.30s/it]
  3%|▎         | 5916/225000 [21:55:29<609:05:14, 10.01s/it]
  3%|▎         | 5917/225000 [21:55:34<513:55:06,  8.44s/it]
  3%|▎         | 5918/225000 [21:55:50<653:25:36, 10.74s/it]
  3%|▎         | 5919/225000 [21:56:07<777:34:21, 12.78s/it]
  3%|▎         | 5920/225000 [21:56:15<682:28:45, 11.21s/it]
  3%|▎         | 5921/225000 [21:56:22<608:54:55, 10.01s/it]
  3%|▎         | 5922/225000 [21:56:33<623:50:10, 10.25s/it]
  3%|▎         | 5923/225000 [21:56:41<574:01:43,  9.43s/it]
  3%|▎         | 5924/225000 [21:56:49<558:03:23,  9.17s/it]
  3%|▎         | 5925/225000 [21:57:00<585:35:39,  9.62s/it]
  3%|▎         | 5926/225000 [21:57:05<503:07:06,  8.27s/it]
  3%|▎         | 5927/225000 [21:57:13<504:19:09,  8.29s/it]
  3%|▎         | 5928/225000 [21:57:25<575:42:26,  9.46s/it]
  3%|▎         | 5929/225000 [21:57:30<485:14:03,  7.97s/it]
  3%|▎         | 5930/225000 [21:57:38<490:50:34,  8.07s/it]
  3%|▎         | 5931/225000 [21:57:42<420:11:14,  6.91s/it]
  3%|▎         | 5932/225000 [21:57:48<393:13:09,  6.46s/it]
  3%|▎         | 5933/225000 [21:57:56<424:20:19,  6.97s/it]
  3%|▎         | 5934/225000 [21:58:10<551:57:06,  9.07s/it]
  3%|▎         | 5935/225000 [21:58:16<505:44:48,  8.31s/it]
  3%|▎         | 5936/225000 [21:58:28<570:28:30,  9.37s/it]
  3%|▎         | 5937/225000 [21:58:40<608:30:21, 10.00s/it]
  3%|▎         | 5938/225000 [21:58:44<508:22:23,  8.35s/it]
  3%|▎         | 5939/225000 [21:58:55<545:37:37,  8.97s/it]
  3%|▎         | 5940/225000 [21:59:05<571:03:01,  9.38s/it]
  3%|▎         | 5941/225000 [21:59:16<593:04:10,  9.75s/it]
  3%|▎         | 5942/225000 [21:59:29<650:42:38, 10.69s/it]
  3%|▎         | 5943/225000 [21:59:35<567:44:46,  9.33s/it]
  3%|▎         | 5944/225000 [21:59:40<488:53:20,  8.03s/it]
  3%|▎         | 5945/225000 [21:59:49<519:18:36,  8.53s/it]
  3%|▎         | 5946/225000 [21:59:58<514:58:21,  8.46s/it]
  3%|▎         | 5947/225000 [22:00:07<537:15:22,  8.83s/it]
  3%|▎         | 5948/225000 [22:00:14<498:26:21,  8.19s/it]
  3%|▎         | 5949/225000 [22:00:19<439:20:17,  7.22s/it]
  3%|▎         | 5950/225000 [22:00:39<670:15:37, 11.02s/it]
                                                            
{'loss': '0.7109', 'grad_norm': '1.128', 'learning_rate': '2.973e-05', 'epoch': '0.02644', 'num_input_tokens_seen': 40287368, 'train_runtime': '7.924e+04', 'train_tokens_per_second': '508.4'}

  3%|▎         | 5950/225000 [22:00:39<670:15:37, 11.02s/it]
  3%|▎         | 5951/225000 [22:00:49<645:54:34, 10.62s/it]
  3%|▎         | 5952/225000 [22:00:55<560:46:47,  9.22s/it]
  3%|▎         | 5953/225000 [22:01:09<651:59:16, 10.72s/it]
  3%|▎         | 5954/225000 [22:01:17<601:51:09,  9.89s/it]
  3%|▎         | 5955/225000 [22:01:23<529:13:52,  8.70s/it]
  3%|▎         | 5956/225000 [22:01:35<604:25:56,  9.93s/it]
  3%|▎         | 5957/225000 [22:01:47<639:10:44, 10.50s/it]
  3%|▎         | 5958/225000 [22:01:56<604:35:58,  9.94s/it]
  3%|▎         | 5959/225000 [22:02:02<539:15:40,  8.86s/it]
  3%|▎         | 5960/225000 [22:02:15<600:10:58,  9.86s/it]
  3%|▎         | 5961/225000 [22:02:19<494:02:56,  8.12s/it]
  3%|▎         | 5962/225000 [22:02:26<478:16:14,  7.86s/it]
  3%|▎         | 5963/225000 [22:02:32<456:46:09,  7.51s/it]
  3%|▎         | 5964/225000 [22:02:41<472:42:50,  7.77s/it]
  3%|▎         | 5965/225000 [22:02:51<513:54:21,  8.45s/it]
  3%|▎         | 5966/225000 [22:02:59<513:13:44,  8.44s/it]
  3%|▎         | 5967/225000 [22:03:16<655:23:02, 10.77s/it]
  3%|▎         | 5968/225000 [22:03:26<642:48:07, 10.57s/it]
  3%|▎         | 5969/225000 [22:03:41<730:40:49, 12.01s/it]
  3%|▎         | 5970/225000 [22:03:51<691:56:23, 11.37s/it]
  3%|▎         | 5971/225000 [22:03:58<613:32:55, 10.08s/it]
  3%|▎         | 5972/225000 [22:04:06<568:04:07,  9.34s/it]
  3%|▎         | 5973/225000 [22:04:13<531:02:53,  8.73s/it]
  3%|▎         | 5974/225000 [22:04:19<478:30:34,  7.86s/it]
  3%|▎         | 5975/225000 [22:04:23<420:33:14,  6.91s/it]
  3%|▎         | 5976/225000 [22:04:32<448:35:41,  7.37s/it]
  3%|▎         | 5977/225000 [22:04:43<525:21:29,  8.64s/it]
  3%|▎         | 5978/225000 [22:04:48<451:31:55,  7.42s/it]
  3%|▎         | 5979/225000 [22:04:55<448:18:24,  7.37s/it]
  3%|▎         | 5980/225000 [22:05:05<491:01:34,  8.07s/it]
  3%|▎         | 5981/225000 [22:05:19<594:39:09,  9.77s/it]
  3%|▎         | 5982/225000 [22:05:25<531:03:39,  8.73s/it]
  3%|▎         | 5983/225000 [22:05:35<556:29:08,  9.15s/it]
  3%|▎         | 5984/225000 [22:05:40<477:02:45,  7.84s/it]
  3%|▎         | 5985/225000 [22:05:50<522:38:48,  8.59s/it]
  3%|▎         | 5986/225000 [22:05:57<490:36:03,  8.06s/it]
  3%|▎         | 5987/225000 [22:06:04<468:42:10,  7.70s/it]
  3%|▎         | 5988/225000 [22:06:18<584:34:09,  9.61s/it]
  3%|▎         | 5989/225000 [22:06:24<526:49:17,  8.66s/it]
  3%|▎         | 5990/225000 [22:06:35<559:00:16,  9.19s/it]
  3%|▎         | 5991/225000 [22:06:43<539:14:22,  8.86s/it]
  3%|▎         | 5992/225000 [22:06:56<622:25:31, 10.23s/it]
  3%|▎         | 5993/225000 [22:07:10<692:14:49, 11.38s/it]
  3%|▎         | 5994/225000 [22:07:15<563:09:07,  9.26s/it]
  3%|▎         | 5995/225000 [22:07:31<694:05:11, 11.41s/it]
  3%|▎         | 5996/225000 [22:07:35<560:55:36,  9.22s/it]
  3%|▎         | 5997/225000 [22:07:50<667:42:49, 10.98s/it]
  3%|▎         | 5998/225000 [22:07:59<628:39:28, 10.33s/it]
  3%|▎         | 5999/225000 [22:08:08<594:06:20,  9.77s/it]
  3%|▎         | 6000/225000 [22:08:13<514:09:58,  8.45s/it]
                                                            
{'loss': '0.6873', 'grad_norm': '1.696', 'learning_rate': '2.998e-05', 'epoch': '0.02667', 'num_input_tokens_seen': 40508488, 'train_runtime': '7.969e+04', 'train_tokens_per_second': '508.3'}

  3%|▎         | 6000/225000 [22:08:13<514:09:58,  8.45s/it]
  3%|▎         | 6001/225000 [22:08:23<532:17:37,  8.75s/it]
  3%|▎         | 6002/225000 [22:08:35<599:11:35,  9.85s/it]
  3%|▎         | 6003/225000 [22:08:43<564:03:29,  9.27s/it]
  3%|▎         | 6004/225000 [22:08:48<487:13:22,  8.01s/it]
  3%|▎         | 6005/225000 [22:09:00<567:00:45,  9.32s/it]
  3%|▎         | 6006/225000 [22:09:11<598:00:34,  9.83s/it]
  3%|▎         | 6007/225000 [22:09:17<513:22:24,  8.44s/it]
  3%|▎         | 6008/225000 [22:09:27<557:56:04,  9.17s/it]
  3%|▎         | 6009/225000 [22:09:42<658:47:26, 10.83s/it]
  3%|▎         | 6010/225000 [22:09:47<553:50:33,  9.10s/it]
  3%|▎         | 6011/225000 [22:09:59<611:58:29, 10.06s/it]
  3%|▎         | 6012/225000 [22:10:06<554:32:22,  9.12s/it]
  3%|▎         | 6013/225000 [22:10:13<508:40:35,  8.36s/it]
  3%|▎         | 6014/225000 [22:10:20<477:05:05,  7.84s/it]
  3%|▎         | 6015/225000 [22:10:25<438:23:32,  7.21s/it]
  3%|▎         | 6016/225000 [22:10:40<569:50:42,  9.37s/it]
  3%|▎         | 6017/225000 [22:10:46<514:16:01,  8.45s/it]
  3%|▎         | 6018/225000 [22:10:53<484:49:13,  7.97s/it]
  3%|▎         | 6019/225000 [22:11:07<603:13:20,  9.92s/it]
  3%|▎         | 6020/225000 [22:11:12<512:24:28,  8.42s/it]
  3%|▎         | 6021/225000 [22:11:26<602:48:57,  9.91s/it]
  3%|▎         | 6022/225000 [22:11:30<506:58:28,  8.33s/it]
  3%|▎         | 6023/225000 [22:11:44<600:27:39,  9.87s/it]
  3%|▎         | 6024/225000 [22:11:56<639:39:59, 10.52s/it]
  3%|▎         | 6025/225000 [22:12:06<640:55:11, 10.54s/it]
  3%|▎         | 6026/225000 [22:12:16<622:57:27, 10.24s/it]
  3%|▎         | 6027/225000 [22:12:22<541:40:53,  8.91s/it]
  3%|▎         | 6028/225000 [22:12:30<538:19:07,  8.85s/it]
  3%|▎         | 6029/225000 [22:12:37<498:15:32,  8.19s/it]
  3%|▎         | 6030/225000 [22:12:49<564:46:38,  9.29s/it]
  3%|▎         | 6031/225000 [22:12:55<503:21:16,  8.28s/it]
  3%|▎         | 6032/225000 [22:13:10<621:59:31, 10.23s/it]
  3%|▎         | 6033/225000 [22:13:19<598:16:59,  9.84s/it]
  3%|▎         | 6034/225000 [22:13:26<548:27:17,  9.02s/it]
  3%|▎         | 6035/225000 [22:13:34<531:47:14,  8.74s/it]
  3%|▎         | 6036/225000 [22:13:44<557:26:10,  9.16s/it]
  3%|▎         | 6037/225000 [22:13:54<565:33:16,  9.30s/it]
  3%|▎         | 6038/225000 [22:14:04<584:58:59,  9.62s/it]
  3%|▎         | 6039/225000 [22:14:19<679:59:19, 11.18s/it]
  3%|▎         | 6040/225000 [22:14:24<570:20:34,  9.38s/it]
  3%|▎         | 6041/225000 [22:14:38<647:56:37, 10.65s/it]
  3%|▎         | 6042/225000 [22:14:52<722:42:32, 11.88s/it]
  3%|▎         | 6043/225000 [22:14:56<574:05:39,  9.44s/it]
  3%|▎         | 6044/225000 [22:15:05<566:42:05,  9.32s/it]
  3%|▎         | 6045/225000 [22:15:11<496:36:43,  8.17s/it]
  3%|▎         | 6046/225000 [22:15:16<448:29:06,  7.37s/it]
  3%|▎         | 6047/225000 [22:15:25<477:37:19,  7.85s/it]
  3%|▎         | 6048/225000 [22:15:42<650:36:12, 10.70s/it]
  3%|▎         | 6049/225000 [22:15:50<589:30:00,  9.69s/it]
  3%|▎         | 6050/225000 [22:15:56<520:34:02,  8.56s/it]
                                                            
{'loss': '0.6996', 'grad_norm': '1.3', 'learning_rate': '3.023e-05', 'epoch': '0.02689', 'num_input_tokens_seen': 40740688, 'train_runtime': '8.016e+04', 'train_tokens_per_second': '508.3'}

  3%|▎         | 6050/225000 [22:15:56<520:34:02,  8.56s/it]
  3%|▎         | 6051/225000 [22:16:03<491:43:38,  8.09s/it]
  3%|▎         | 6052/225000 [22:16:09<469:59:05,  7.73s/it]
  3%|▎         | 6053/225000 [22:16:28<660:38:12, 10.86s/it]
  3%|▎         | 6054/225000 [22:16:38<646:06:38, 10.62s/it]
  3%|▎         | 6055/225000 [22:16:43<542:50:22,  8.93s/it]
  3%|▎         | 6056/225000 [22:16:55<604:32:54,  9.94s/it]
  3%|▎         | 6057/225000 [22:17:02<541:43:09,  8.91s/it]
  3%|▎         | 6058/225000 [22:17:07<481:44:24,  7.92s/it]
  3%|▎         | 6059/225000 [22:17:17<524:47:40,  8.63s/it]
  3%|▎         | 6060/225000 [22:17:26<523:48:10,  8.61s/it]
  3%|▎         | 6061/225000 [22:17:32<467:40:19,  7.69s/it]
  3%|▎         | 6062/225000 [22:17:43<538:04:50,  8.85s/it]
  3%|▎         | 6063/225000 [22:17:52<542:19:10,  8.92s/it]
  3%|▎         | 6064/225000 [22:17:59<501:16:57,  8.24s/it]
  3%|▎         | 6065/225000 [22:18:13<614:02:09, 10.10s/it]
  3%|▎         | 6066/225000 [22:18:20<552:20:10,  9.08s/it]
  3%|▎         | 6067/225000 [22:18:29<546:06:18,  8.98s/it]
  3%|▎         | 6068/225000 [22:18:35<488:30:54,  8.03s/it]
  3%|▎         | 6069/225000 [22:18:46<550:00:32,  9.04s/it]
  3%|▎         | 6070/225000 [22:18:51<474:38:56,  7.80s/it]
  3%|▎         | 6071/225000 [22:19:02<535:43:18,  8.81s/it]
  3%|▎         | 6072/225000 [22:19:09<498:10:11,  8.19s/it]
  3%|▎         | 6073/225000 [22:19:15<467:15:38,  7.68s/it]
  3%|▎         | 6074/225000 [22:19:33<652:34:28, 10.73s/it]
  3%|▎         | 6075/225000 [22:19:40<586:40:19,  9.65s/it]
  3%|▎         | 6076/225000 [22:19:51<613:40:09, 10.09s/it]
  3%|▎         | 6077/225000 [22:20:04<662:04:10, 10.89s/it]
  3%|▎         | 6078/225000 [22:20:09<561:05:49,  9.23s/it]
  3%|▎         | 6079/225000 [22:20:18<544:42:08,  8.96s/it]
  3%|▎         | 6080/225000 [22:20:32<647:17:18, 10.64s/it]
  3%|▎         | 6081/225000 [22:20:40<594:09:38,  9.77s/it]
  3%|▎         | 6082/225000 [22:20:53<652:26:06, 10.73s/it]
  3%|▎         | 6083/225000 [22:20:58<550:07:05,  9.05s/it]
  3%|▎         | 6084/225000 [22:21:09<584:55:51,  9.62s/it]
  3%|▎         | 6085/225000 [22:21:27<726:53:20, 11.95s/it]
  3%|▎         | 6086/225000 [22:21:35<656:58:49, 10.80s/it]
  3%|▎         | 6087/225000 [22:21:44<638:03:34, 10.49s/it]
  3%|▎         | 6088/225000 [22:21:54<621:22:02, 10.22s/it]
  3%|▎         | 6089/225000 [22:22:12<768:36:31, 12.64s/it]
  3%|▎         | 6090/225000 [22:22:20<672:36:27, 11.06s/it]
  3%|▎         | 6091/225000 [22:22:26<583:57:30,  9.60s/it]
  3%|▎         | 6092/225000 [22:22:35<578:11:48,  9.51s/it]
  3%|▎         | 6093/225000 [22:22:41<508:42:40,  8.37s/it]
  3%|▎         | 6094/225000 [22:22:55<611:57:26, 10.06s/it]
  3%|▎         | 6095/225000 [22:23:01<544:58:27,  8.96s/it]
  3%|▎         | 6096/225000 [22:23:13<588:00:35,  9.67s/it]
  3%|▎         | 6097/225000 [22:23:20<540:14:40,  8.88s/it]
  3%|▎         | 6098/225000 [22:23:25<479:40:12,  7.89s/it]
  3%|▎         | 6099/225000 [22:23:35<519:34:38,  8.54s/it]
  3%|▎         | 6100/225000 [22:23:44<514:12:19,  8.46s/it]
                                                            
{'loss': '0.7753', 'grad_norm': '1.521', 'learning_rate': '3.048e-05', 'epoch': '0.02711', 'num_input_tokens_seen': 40971768, 'train_runtime': '8.062e+04', 'train_tokens_per_second': '508.2'}

  3%|▎         | 6100/225000 [22:23:44<514:12:19,  8.46s/it]
  3%|▎         | 6101/225000 [22:23:56<584:24:42,  9.61s/it]
  3%|▎         | 6102/225000 [22:24:02<514:51:10,  8.47s/it]
  3%|▎         | 6103/225000 [22:24:07<459:41:44,  7.56s/it]
  3%|▎         | 6104/225000 [22:24:20<552:33:29,  9.09s/it]
  3%|▎         | 6105/225000 [22:24:29<552:55:11,  9.09s/it]
  3%|▎         | 6106/225000 [22:24:37<532:05:46,  8.75s/it]
  3%|▎         | 6107/225000 [22:24:52<651:27:20, 10.71s/it]
  3%|▎         | 6108/225000 [22:25:04<676:22:05, 11.12s/it]
  3%|▎         | 6109/225000 [22:25:11<605:02:05,  9.95s/it]
  3%|▎         | 6110/225000 [22:25:19<560:27:14,  9.22s/it]
  3%|▎         | 6111/225000 [22:25:29<572:59:39,  9.42s/it]
  3%|▎         | 6112/225000 [22:25:34<497:42:09,  8.19s/it]
  3%|▎         | 6113/225000 [22:25:44<532:01:05,  8.75s/it]
  3%|▎         | 6114/225000 [22:25:57<604:25:16,  9.94s/it]
  3%|▎         | 6115/225000 [22:26:02<524:23:32,  8.62s/it]
  3%|▎         | 6116/225000 [22:26:11<516:27:28,  8.49s/it]
  3%|▎         | 6117/225000 [22:26:17<472:25:29,  7.77s/it]
  3%|▎         | 6118/225000 [22:26:26<508:45:43,  8.37s/it]
  3%|▎         | 6119/225000 [22:26:31<432:20:19,  7.11s/it]
  3%|▎         | 6120/225000 [22:26:42<514:21:33,  8.46s/it]
  3%|▎         | 6121/225000 [22:26:47<447:24:25,  7.36s/it]
  3%|▎         | 6122/225000 [22:26:53<420:07:42,  6.91s/it]
  3%|▎         | 6123/225000 [22:27:02<459:05:39,  7.55s/it]
  3%|▎         | 6124/225000 [22:27:16<574:59:58,  9.46s/it]
  3%|▎         | 6125/225000 [22:27:31<673:03:42, 11.07s/it]
  3%|▎         | 6126/225000 [22:27:41<652:22:26, 10.73s/it]
  3%|▎         | 6127/225000 [22:27:46<554:56:31,  9.13s/it]
  3%|▎         | 6128/225000 [22:27:56<578:13:49,  9.51s/it]
  3%|▎         | 6129/225000 [22:28:09<640:33:31, 10.54s/it]
  3%|▎         | 6130/225000 [22:28:24<715:50:33, 11.77s/it]
  3%|▎         | 6131/225000 [22:28:29<598:39:41,  9.85s/it]
  3%|▎         | 6132/225000 [22:28:41<626:15:16, 10.30s/it]
  3%|▎         | 6133/225000 [22:28:50<616:05:05, 10.13s/it]
  3%|▎         | 6134/225000 [22:29:04<675:40:41, 11.11s/it]
  3%|▎         | 6135/225000 [22:29:16<699:08:18, 11.50s/it]
  3%|▎         | 6136/225000 [22:29:23<616:02:29, 10.13s/it]
  3%|▎         | 6137/225000 [22:29:29<537:21:45,  8.84s/it]
  3%|▎         | 6138/225000 [22:29:40<570:09:47,  9.38s/it]
  3%|▎         | 6139/225000 [22:29:50<591:15:52,  9.73s/it]
  3%|▎         | 6140/225000 [22:30:00<585:13:54,  9.63s/it]
  3%|▎         | 6141/225000 [22:30:11<620:02:11, 10.20s/it]
  3%|▎         | 6142/225000 [22:30:19<579:49:46,  9.54s/it]
  3%|▎         | 6143/225000 [22:30:24<489:03:33,  8.04s/it]
  3%|▎         | 6144/225000 [22:30:32<494:19:12,  8.13s/it]
  3%|▎         | 6145/225000 [22:30:37<433:16:03,  7.13s/it]
  3%|▎         | 6146/225000 [22:30:45<453:33:09,  7.46s/it]
  3%|▎         | 6147/225000 [22:30:51<424:23:06,  6.98s/it]
  3%|▎         | 6148/225000 [22:30:59<447:39:48,  7.36s/it]
  3%|▎         | 6149/225000 [22:31:06<439:50:14,  7.24s/it]
  3%|▎         | 6150/225000 [22:31:11<402:44:51,  6.63s/it]
                                                            
{'loss': '0.6995', 'grad_norm': '1.831', 'learning_rate': '3.073e-05', 'epoch': '0.02733', 'num_input_tokens_seen': 41195424, 'train_runtime': '8.107e+04', 'train_tokens_per_second': '508.1'}

  3%|▎         | 6150/225000 [22:31:11<402:44:51,  6.63s/it]
  3%|▎         | 6151/225000 [22:31:25<525:54:40,  8.65s/it]
  3%|▎         | 6152/225000 [22:31:36<566:23:42,  9.32s/it]
  3%|▎         | 6153/225000 [22:31:44<551:27:52,  9.07s/it]
  3%|▎         | 6154/225000 [22:31:53<540:58:15,  8.90s/it]
  3%|▎         | 6155/225000 [22:32:04<593:05:02,  9.76s/it]
  3%|▎         | 6156/225000 [22:32:14<600:39:05,  9.88s/it]
  3%|▎         | 6157/225000 [22:32:23<571:28:32,  9.40s/it]
  3%|▎         | 6158/225000 [22:32:43<778:35:25, 12.81s/it]
  3%|▎         | 6159/225000 [22:32:54<729:16:02, 12.00s/it]
  3%|▎         | 6160/225000 [22:33:06<731:12:35, 12.03s/it]
  3%|▎         | 6161/225000 [22:33:10<596:40:00,  9.82s/it]
  3%|▎         | 6162/225000 [22:33:18<554:43:30,  9.13s/it]
  3%|▎         | 6163/225000 [22:33:23<487:12:38,  8.01s/it]
  3%|▎         | 6164/225000 [22:33:31<490:31:28,  8.07s/it]
  3%|▎         | 6165/225000 [22:33:43<562:05:59,  9.25s/it]
  3%|▎         | 6166/225000 [22:33:49<494:39:37,  8.14s/it]
  3%|▎         | 6167/225000 [22:33:55<460:40:22,  7.58s/it]
  3%|▎         | 6168/225000 [22:34:10<591:55:29,  9.74s/it]
  3%|▎         | 6169/225000 [22:34:19<580:30:07,  9.55s/it]
  3%|▎         | 6170/225000 [22:34:26<538:59:47,  8.87s/it]
  3%|▎         | 6171/225000 [22:34:37<572:20:53,  9.42s/it]
  3%|▎         | 6172/225000 [22:34:52<664:40:16, 10.93s/it]
  3%|▎         | 6173/225000 [22:35:01<645:11:38, 10.61s/it]
  3%|▎         | 6174/225000 [22:35:11<632:03:26, 10.40s/it]
  3%|▎         | 6175/225000 [22:35:24<677:54:52, 11.15s/it]
  3%|▎         | 6176/225000 [22:35:34<656:33:12, 10.80s/it]
  3%|▎         | 6177/225000 [22:35:44<632:55:23, 10.41s/it]
  3%|▎         | 6178/225000 [22:35:52<600:13:02,  9.87s/it]
  3%|▎         | 6179/225000 [22:36:07<687:43:03, 11.31s/it]
  3%|▎         | 6180/225000 [22:36:12<568:27:00,  9.35s/it]
  3%|▎         | 6181/225000 [22:36:19<521:02:00,  8.57s/it]
  3%|▎         | 6182/225000 [22:36:33<629:01:07, 10.35s/it]
  3%|▎         | 6183/225000 [22:36:43<619:07:18, 10.19s/it]
  3%|▎         | 6184/225000 [22:36:48<527:59:35,  8.69s/it]
  3%|▎         | 6185/225000 [22:36:55<486:27:34,  8.00s/it]
  3%|▎         | 6186/225000 [22:37:14<687:37:11, 11.31s/it]
  3%|▎         | 6187/225000 [22:37:19<581:50:48,  9.57s/it]
  3%|▎         | 6188/225000 [22:37:27<547:13:49,  9.00s/it]
  3%|▎         | 6189/225000 [22:37:41<647:02:41, 10.65s/it]
  3%|▎         | 6190/225000 [22:37:47<567:15:45,  9.33s/it]
  3%|▎         | 6191/225000 [22:38:00<631:22:19, 10.39s/it]
  3%|▎         | 6192/225000 [22:38:07<564:19:31,  9.28s/it]
  3%|▎         | 6193/225000 [22:38:17<570:09:09,  9.38s/it]
  3%|▎         | 6194/225000 [22:38:24<536:34:57,  8.83s/it]
  3%|▎         | 6195/225000 [22:38:31<504:13:21,  8.30s/it]
  3%|▎         | 6196/225000 [22:38:42<551:36:51,  9.08s/it]
  3%|▎         | 6197/225000 [22:38:55<612:54:25, 10.08s/it]
  3%|▎         | 6198/225000 [22:39:07<663:55:38, 10.92s/it]
  3%|▎         | 6199/225000 [22:39:14<579:53:20,  9.54s/it]
  3%|▎         | 6200/225000 [22:39:21<535:13:37,  8.81s/it]
                                                            
{'loss': '0.7459', 'grad_norm': '1.243', 'learning_rate': '3.098e-05', 'epoch': '0.02756', 'num_input_tokens_seen': 41435656, 'train_runtime': '8.156e+04', 'train_tokens_per_second': '508'}

  3%|▎         | 6200/225000 [22:39:21<535:13:37,  8.81s/it]
  3%|▎         | 6201/225000 [22:39:29<523:53:09,  8.62s/it]
  3%|▎         | 6202/225000 [22:39:44<647:26:51, 10.65s/it]
  3%|▎         | 6203/225000 [22:39:51<581:29:12,  9.57s/it]
  3%|▎         | 6204/225000 [22:39:58<520:47:01,  8.57s/it]
  3%|▎         | 6205/225000 [22:40:06<523:36:28,  8.62s/it]
  3%|▎         | 6206/225000 [22:40:20<612:52:47, 10.08s/it]
  3%|▎         | 6207/225000 [22:40:27<564:59:50,  9.30s/it]
  3%|▎         | 6208/225000 [22:40:36<551:06:07,  9.07s/it]
  3%|▎         | 6209/225000 [22:40:42<491:25:39,  8.09s/it]
  3%|▎         | 6210/225000 [22:40:58<639:40:35, 10.53s/it]
  3%|▎         | 6211/225000 [22:41:07<609:28:44, 10.03s/it]
  3%|▎         | 6212/225000 [22:41:12<525:09:36,  8.64s/it]
  3%|▎         | 6213/225000 [22:41:18<473:43:20,  7.79s/it]
  3%|▎         | 6214/225000 [22:41:28<517:38:29,  8.52s/it]
  3%|▎         | 6215/225000 [22:41:45<673:25:40, 11.08s/it]
  3%|▎         | 6216/225000 [22:41:59<727:52:21, 11.98s/it]
  3%|▎         | 6217/225000 [22:42:06<621:44:35, 10.23s/it]
  3%|▎         | 6218/225000 [22:42:23<746:20:30, 12.28s/it]
  3%|▎         | 6219/225000 [22:42:31<669:26:26, 11.02s/it]
  3%|▎         | 6220/225000 [22:42:39<619:14:37, 10.19s/it]
  3%|▎         | 6221/225000 [22:42:47<585:41:46,  9.64s/it]
  3%|▎         | 6222/225000 [22:43:01<667:30:33, 10.98s/it]
  3%|▎         | 6223/225000 [22:43:07<568:30:30,  9.35s/it]
  3%|▎         | 6224/225000 [22:43:19<621:15:01, 10.22s/it]
  3%|▎         | 6225/225000 [22:43:28<595:59:19,  9.81s/it]
  3%|▎         | 6226/225000 [22:43:34<520:47:00,  8.57s/it]
  3%|▎         | 6227/225000 [22:43:41<493:46:26,  8.13s/it]
  3%|▎         | 6228/225000 [22:43:50<509:28:50,  8.38s/it]
  3%|▎         | 6229/225000 [22:43:55<452:44:48,  7.45s/it]
  3%|▎         | 6230/225000 [22:44:04<483:34:19,  7.96s/it]
  3%|▎         | 6231/225000 [22:44:13<507:34:26,  8.35s/it]
  3%|▎         | 6232/225000 [22:44:24<552:30:39,  9.09s/it]
  3%|▎         | 6233/225000 [22:44:29<473:37:58,  7.79s/it]
  3%|▎         | 6234/225000 [22:44:40<534:31:57,  8.80s/it]
  3%|▎         | 6235/225000 [22:44:54<633:12:09, 10.42s/it]
  3%|▎         | 6236/225000 [22:45:04<625:56:50, 10.30s/it]
  3%|▎         | 6237/225000 [22:45:11<561:07:06,  9.23s/it]
  3%|▎         | 6238/225000 [22:45:24<618:18:21, 10.17s/it]
  3%|▎         | 6239/225000 [22:45:31<576:15:12,  9.48s/it]
  3%|▎         | 6240/225000 [22:45:36<486:19:52,  8.00s/it]
  3%|▎         | 6241/225000 [22:45:43<459:59:15,  7.57s/it]
  3%|▎         | 6242/225000 [22:45:56<565:04:59,  9.30s/it]
  3%|▎         | 6243/225000 [22:46:08<622:54:23, 10.25s/it]
  3%|▎         | 6244/225000 [22:46:16<577:40:47,  9.51s/it]
  3%|▎         | 6245/225000 [22:46:22<506:14:43,  8.33s/it]
  3%|▎         | 6246/225000 [22:46:33<567:33:33,  9.34s/it]
  3%|▎         | 6247/225000 [22:46:39<499:36:08,  8.22s/it]
  3%|▎         | 6248/225000 [22:46:49<524:49:22,  8.64s/it]
  3%|▎         | 6249/225000 [22:46:59<552:25:19,  9.09s/it]
  3%|▎         | 6250/225000 [22:47:06<522:39:25,  8.60s/it]
                                                            
{'loss': '0.6878', 'grad_norm': '1.425', 'learning_rate': '3.123e-05', 'epoch': '0.02778', 'num_input_tokens_seen': 41664720, 'train_runtime': '8.203e+04', 'train_tokens_per_second': '507.9'}

  3%|▎         | 6250/225000 [22:47:06<522:39:25,  8.60s/it]
  3%|▎         | 6251/225000 [22:47:20<617:47:11, 10.17s/it]
  3%|▎         | 6252/225000 [22:47:30<617:28:11, 10.16s/it]
  3%|▎         | 6253/225000 [22:47:38<572:12:09,  9.42s/it]
  3%|▎         | 6254/225000 [22:47:46<550:56:42,  9.07s/it]
  3%|▎         | 6255/225000 [22:48:00<632:25:44, 10.41s/it]
  3%|▎         | 6256/225000 [22:48:06<559:24:01,  9.21s/it]
  3%|▎         | 6257/225000 [22:48:18<605:15:16,  9.96s/it]
  3%|▎         | 6258/225000 [22:48:33<695:47:48, 11.45s/it]
  3%|▎         | 6259/225000 [22:48:45<704:46:56, 11.60s/it]
  3%|▎         | 6260/225000 [22:48:52<621:31:31, 10.23s/it]
  3%|▎         | 6261/225000 [22:49:02<622:57:50, 10.25s/it]
  3%|▎         | 6262/225000 [22:49:08<553:56:47,  9.12s/it]
  3%|▎         | 6263/225000 [22:49:17<537:43:19,  8.85s/it]
  3%|▎         | 6264/225000 [22:49:26<542:31:42,  8.93s/it]
  3%|▎         | 6265/225000 [22:49:39<623:37:32, 10.26s/it]
  3%|▎         | 6266/225000 [22:49:51<652:56:33, 10.75s/it]
  3%|▎         | 6267/225000 [22:49:57<565:49:31,  9.31s/it]
  3%|▎         | 6268/225000 [22:50:10<631:10:47, 10.39s/it]
  3%|▎         | 6269/225000 [22:50:14<522:47:36,  8.60s/it]
  3%|▎         | 6270/225000 [22:50:27<594:15:35,  9.78s/it]
  3%|▎         | 6271/225000 [22:50:36<585:33:32,  9.64s/it]
  3%|▎         | 6272/225000 [22:50:45<568:57:18,  9.36s/it]
  3%|▎         | 6273/225000 [22:50:52<529:26:21,  8.71s/it]
  3%|▎         | 6274/225000 [22:50:58<486:11:08,  8.00s/it]
  3%|▎         | 6275/225000 [22:51:13<606:40:38,  9.99s/it]
  3%|▎         | 6276/225000 [22:51:17<494:42:51,  8.14s/it]
  3%|▎         | 6277/225000 [22:51:29<571:33:25,  9.41s/it]
  3%|▎         | 6278/225000 [22:51:35<509:09:36,  8.38s/it]
  3%|▎         | 6279/225000 [22:51:48<587:13:50,  9.67s/it]
  3%|▎         | 6280/225000 [22:51:56<566:00:41,  9.32s/it]
  3%|▎         | 6281/225000 [22:52:04<537:56:12,  8.85s/it]
  3%|▎         | 6282/225000 [22:52:12<521:01:31,  8.58s/it]
  3%|▎         | 6283/225000 [22:52:21<527:31:21,  8.68s/it]
  3%|▎         | 6284/225000 [22:52:31<558:41:18,  9.20s/it]
  3%|▎         | 6285/225000 [22:52:41<574:04:24,  9.45s/it]
  3%|▎         | 6286/225000 [22:52:50<560:17:55,  9.22s/it]
  3%|▎         | 6287/225000 [22:52:58<541:04:52,  8.91s/it]
  3%|▎         | 6288/225000 [22:53:10<589:59:56,  9.71s/it]
  3%|▎         | 6289/225000 [22:53:20<593:28:11,  9.77s/it]
  3%|▎         | 6290/225000 [22:53:25<503:19:00,  8.28s/it]
  3%|▎         | 6291/225000 [22:53:32<483:27:18,  7.96s/it]
  3%|▎         | 6292/225000 [22:53:49<650:51:02, 10.71s/it]
  3%|▎         | 6293/225000 [22:53:59<645:06:46, 10.62s/it]
  3%|▎         | 6294/225000 [22:54:12<678:50:18, 11.17s/it]
  3%|▎         | 6295/225000 [22:54:30<800:20:32, 13.17s/it]
  3%|▎         | 6296/225000 [22:54:37<686:54:23, 11.31s/it]
  3%|▎         | 6297/225000 [22:54:54<806:04:47, 13.27s/it]
  3%|▎         | 6298/225000 [22:55:00<668:18:00, 11.00s/it]
  3%|▎         | 6299/225000 [22:55:06<581:35:25,  9.57s/it]
  3%|▎         | 6300/225000 [22:55:19<632:37:56, 10.41s/it]
                                                            
{'loss': '0.6879', 'grad_norm': '1.33', 'learning_rate': '3.147e-05', 'epoch': '0.028', 'num_input_tokens_seen': 41901424, 'train_runtime': '8.252e+04', 'train_tokens_per_second': '507.8'}

  3%|▎         | 6300/225000 [22:55:19<632:37:56, 10.41s/it]
  3%|▎         | 6301/225000 [22:55:24<532:34:48,  8.77s/it]
  3%|▎         | 6302/225000 [22:55:29<471:53:12,  7.77s/it]
  3%|▎         | 6303/225000 [22:55:35<439:38:42,  7.24s/it]
  3%|▎         | 6304/225000 [22:55:54<653:22:26, 10.76s/it]
  3%|▎         | 6305/225000 [22:55:59<544:41:56,  8.97s/it]
  3%|▎         | 6306/225000 [22:56:06<504:37:27,  8.31s/it]
  3%|▎         | 6307/225000 [22:56:17<564:35:46,  9.29s/it]
  3%|▎         | 6308/225000 [22:56:25<530:49:39,  8.74s/it]
  3%|▎         | 6309/225000 [22:56:35<556:59:04,  9.17s/it]
  3%|▎         | 6310/225000 [22:56:40<477:00:13,  7.85s/it]
  3%|▎         | 6311/225000 [22:56:46<448:54:22,  7.39s/it]
  3%|▎         | 6312/225000 [22:56:59<554:36:44,  9.13s/it]
  3%|▎         | 6313/225000 [22:57:08<542:25:23,  8.93s/it]
  3%|▎         | 6314/225000 [22:57:17<557:40:32,  9.18s/it]
  3%|▎         | 6315/225000 [22:57:23<494:55:09,  8.15s/it]
  3%|▎         | 6316/225000 [22:57:33<519:35:34,  8.55s/it]
  3%|▎         | 6317/225000 [22:57:41<521:16:47,  8.58s/it]
  3%|▎         | 6318/225000 [22:57:52<558:56:17,  9.20s/it]
  3%|▎         | 6319/225000 [22:57:59<524:09:20,  8.63s/it]
  3%|▎         | 6320/225000 [22:58:09<536:36:40,  8.83s/it]
  3%|▎         | 6321/225000 [22:58:20<577:06:53,  9.50s/it]
  3%|▎         | 6322/225000 [22:58:40<779:39:28, 12.84s/it]
  3%|▎         | 6323/225000 [22:58:45<624:52:25, 10.29s/it]
  3%|▎         | 6324/225000 [22:58:53<592:43:00,  9.76s/it]
  3%|▎         | 6325/225000 [22:59:01<566:02:45,  9.32s/it]
  3%|▎         | 6326/225000 [22:59:11<575:17:16,  9.47s/it]
  3%|▎         | 6327/225000 [22:59:21<574:00:09,  9.45s/it]
  3%|▎         | 6328/225000 [22:59:27<525:27:56,  8.65s/it]
  3%|▎         | 6329/225000 [22:59:43<647:16:24, 10.66s/it]
  3%|▎         | 6330/225000 [22:59:53<647:24:16, 10.66s/it]
  3%|▎         | 6331/225000 [23:00:05<661:59:51, 10.90s/it]
  3%|▎         | 6332/225000 [23:00:14<633:25:54, 10.43s/it]
  3%|▎         | 6333/225000 [23:00:25<647:53:38, 10.67s/it]
  3%|▎         | 6334/225000 [23:00:34<616:33:08, 10.15s/it]
  3%|▎         | 6335/225000 [23:00:45<629:48:53, 10.37s/it]
  3%|▎         | 6336/225000 [23:00:50<524:52:24,  8.64s/it]
  3%|▎         | 6337/225000 [23:00:54<444:12:51,  7.31s/it]
  3%|▎         | 6338/225000 [23:01:06<528:35:46,  8.70s/it]
  3%|▎         | 6339/225000 [23:01:17<571:58:06,  9.42s/it]
  3%|▎         | 6340/225000 [23:01:25<536:02:10,  8.83s/it]
  3%|▎         | 6341/225000 [23:01:28<446:13:30,  7.35s/it]
  3%|▎         | 6342/225000 [23:01:35<428:23:14,  7.05s/it]
  3%|▎         | 6343/225000 [23:01:47<525:43:29,  8.66s/it]
  3%|▎         | 6344/225000 [23:01:58<570:14:29,  9.39s/it]
  3%|▎         | 6345/225000 [23:02:04<504:41:28,  8.31s/it]
  3%|▎         | 6346/225000 [23:02:18<599:10:38,  9.87s/it]
  3%|▎         | 6347/225000 [23:02:26<573:14:47,  9.44s/it]
  3%|▎         | 6348/225000 [23:02:42<694:22:04, 11.43s/it]
  3%|▎         | 6349/225000 [23:02:46<557:55:02,  9.19s/it]
  3%|▎         | 6350/225000 [23:02:53<523:20:12,  8.62s/it]
                                                            
{'loss': '0.7196', 'grad_norm': '2.058', 'learning_rate': '3.172e-05', 'epoch': '0.02822', 'num_input_tokens_seen': 42126264, 'train_runtime': '8.297e+04', 'train_tokens_per_second': '507.7'}

  3%|▎         | 6350/225000 [23:02:53<523:20:12,  8.62s/it]
  3%|▎         | 6351/225000 [23:03:01<510:56:28,  8.41s/it]
  3%|▎         | 6352/225000 [23:03:12<560:05:45,  9.22s/it]
  3%|▎         | 6353/225000 [23:03:17<484:29:39,  7.98s/it]
  3%|▎         | 6354/225000 [23:03:24<458:45:27,  7.55s/it]
  3%|▎         | 6355/225000 [23:03:30<420:54:55,  6.93s/it]
  3%|▎         | 6356/225000 [23:03:42<524:55:21,  8.64s/it]
  3%|▎         | 6357/225000 [23:03:49<488:56:49,  8.05s/it]
  3%|▎         | 6358/225000 [23:04:05<637:36:23, 10.50s/it]
  3%|▎         | 6359/225000 [23:04:12<564:44:25,  9.30s/it]
  3%|▎         | 6360/225000 [23:04:21<560:05:01,  9.22s/it]
  3%|▎         | 6361/225000 [23:04:25<473:16:32,  7.79s/it]
  3%|▎         | 6362/225000 [23:04:42<632:15:16, 10.41s/it]
  3%|▎         | 6363/225000 [23:04:52<630:24:33, 10.38s/it]
  3%|▎         | 6364/225000 [23:04:58<551:15:14,  9.08s/it]
  3%|▎         | 6365/225000 [23:05:06<540:19:08,  8.90s/it]
  3%|▎         | 6366/225000 [23:05:12<481:08:47,  7.92s/it]
  3%|▎         | 6367/225000 [23:05:21<498:52:45,  8.21s/it]
  3%|▎         | 6368/225000 [23:05:35<601:37:00,  9.91s/it]
  3%|▎         | 6369/225000 [23:05:40<523:41:00,  8.62s/it]
  3%|▎         | 6370/225000 [23:05:46<463:10:12,  7.63s/it]
  3%|▎         | 6371/225000 [23:05:58<557:24:15,  9.18s/it]
  3%|▎         | 6372/225000 [23:06:06<529:26:57,  8.72s/it]
  3%|▎         | 6373/225000 [23:06:15<539:55:45,  8.89s/it]
  3%|▎         | 6374/225000 [23:06:29<626:48:06, 10.32s/it]
  3%|▎         | 6375/225000 [23:06:41<652:24:55, 10.74s/it]
  3%|▎         | 6376/225000 [23:06:50<628:47:31, 10.35s/it]
  3%|▎         | 6377/225000 [23:06:56<538:43:55,  8.87s/it]
  3%|▎         | 6378/225000 [23:07:01<478:44:09,  7.88s/it]
  3%|▎         | 6379/225000 [23:07:06<429:27:12,  7.07s/it]
  3%|▎         | 6380/225000 [23:07:16<481:42:05,  7.93s/it]
  3%|▎         | 6381/225000 [23:07:26<519:28:27,  8.55s/it]
  3%|▎         | 6382/225000 [23:07:32<469:14:48,  7.73s/it]
  3%|▎         | 6383/225000 [23:07:40<478:53:11,  7.89s/it]
  3%|▎         | 6384/225000 [23:07:49<491:00:13,  8.09s/it]
  3%|▎         | 6385/225000 [23:08:00<546:38:04,  9.00s/it]
  3%|▎         | 6386/225000 [23:08:11<584:59:50,  9.63s/it]
  3%|▎         | 6387/225000 [23:08:17<510:37:45,  8.41s/it]
  3%|▎         | 6388/225000 [23:08:29<588:20:42,  9.69s/it]
  3%|▎         | 6389/225000 [23:08:40<606:57:42, 10.00s/it]
  3%|▎         | 6390/225000 [23:08:52<648:00:46, 10.67s/it]
  3%|▎         | 6391/225000 [23:08:58<557:56:10,  9.19s/it]
  3%|▎         | 6392/225000 [23:09:11<625:47:24, 10.31s/it]
  3%|▎         | 6393/225000 [23:09:19<584:27:17,  9.62s/it]
  3%|▎         | 6394/225000 [23:09:28<572:17:45,  9.42s/it]
  3%|▎         | 6395/225000 [23:09:33<498:36:38,  8.21s/it]
  3%|▎         | 6396/225000 [23:09:42<502:00:03,  8.27s/it]
  3%|▎         | 6397/225000 [23:09:53<549:27:41,  9.05s/it]
  3%|▎         | 6398/225000 [23:10:00<525:59:10,  8.66s/it]
  3%|▎         | 6399/225000 [23:10:05<455:33:23,  7.50s/it]
  3%|▎         | 6400/225000 [23:10:16<508:21:23,  8.37s/it]
                                                            
{'loss': '0.7252', 'grad_norm': '1.194', 'learning_rate': '3.197e-05', 'epoch': '0.02844', 'num_input_tokens_seen': 42345288, 'train_runtime': '8.342e+04', 'train_tokens_per_second': '507.6'}

  3%|▎         | 6400/225000 [23:10:16<508:21:23,  8.37s/it]
  3%|▎         | 6401/225000 [23:10:23<489:15:45,  8.06s/it]
  3%|▎         | 6402/225000 [23:10:28<428:24:17,  7.06s/it]
  3%|▎         | 6403/225000 [23:10:33<402:22:31,  6.63s/it]
  3%|▎         | 6404/225000 [23:10:45<487:45:59,  8.03s/it]
  3%|▎         | 6405/225000 [23:10:52<478:18:41,  7.88s/it]
  3%|▎         | 6406/225000 [23:11:03<539:49:41,  8.89s/it]
  3%|▎         | 6407/225000 [23:11:14<572:57:56,  9.44s/it]
  3%|▎         | 6408/225000 [23:11:21<517:51:35,  8.53s/it]
  3%|▎         | 6409/225000 [23:11:26<462:39:01,  7.62s/it]
  3%|▎         | 6410/225000 [23:11:35<490:17:50,  8.07s/it]
  3%|▎         | 6411/225000 [23:11:48<579:37:01,  9.55s/it]
  3%|▎         | 6412/225000 [23:11:55<527:55:58,  8.69s/it]
  3%|▎         | 6413/225000 [23:12:08<615:28:29, 10.14s/it]
  3%|▎         | 6414/225000 [23:12:21<655:34:33, 10.80s/it]
  3%|▎         | 6415/225000 [23:12:27<576:24:52,  9.49s/it]
  3%|▎         | 6416/225000 [23:12:38<607:36:22, 10.01s/it]
  3%|▎         | 6417/225000 [23:12:44<522:42:53,  8.61s/it]
  3%|▎         | 6418/225000 [23:12:58<631:39:30, 10.40s/it]
  3%|▎         | 6419/225000 [23:13:03<532:13:51,  8.77s/it]
  3%|▎         | 6420/225000 [23:13:11<521:11:03,  8.58s/it]
  3%|▎         | 6421/225000 [23:13:25<604:46:29,  9.96s/it]
  3%|▎         | 6422/225000 [23:13:32<566:20:44,  9.33s/it]
  3%|▎         | 6423/225000 [23:13:45<632:18:59, 10.41s/it]
  3%|▎         | 6424/225000 [23:13:54<596:52:09,  9.83s/it]
  3%|▎         | 6425/225000 [23:14:08<679:06:10, 11.19s/it]
  3%|▎         | 6426/225000 [23:14:18<660:14:51, 10.87s/it]
  3%|▎         | 6427/225000 [23:14:26<600:52:16,  9.90s/it]
  3%|▎         | 6428/225000 [23:14:38<632:27:56, 10.42s/it]
  3%|▎         | 6429/225000 [23:14:43<532:42:45,  8.77s/it]
  3%|▎         | 6430/225000 [23:14:53<559:19:42,  9.21s/it]
  3%|▎         | 6431/225000 [23:14:58<490:31:04,  8.08s/it]
  3%|▎         | 6432/225000 [23:15:04<446:54:59,  7.36s/it]
  3%|▎         | 6433/225000 [23:15:17<556:13:16,  9.16s/it]
  3%|▎         | 6434/225000 [23:15:28<583:21:46,  9.61s/it]
  3%|▎         | 6435/225000 [23:15:35<539:57:16,  8.89s/it]
  3%|▎         | 6436/225000 [23:15:41<488:28:34,  8.05s/it]
  3%|▎         | 6437/225000 [23:15:46<437:57:57,  7.21s/it]
  3%|▎         | 6438/225000 [23:15:55<462:43:12,  7.62s/it]
  3%|▎         | 6439/225000 [23:16:02<450:08:28,  7.41s/it]
  3%|▎         | 6440/225000 [23:16:19<626:30:37, 10.32s/it]
  3%|▎         | 6441/225000 [23:16:24<532:23:41,  8.77s/it]
  3%|▎         | 6442/225000 [23:16:31<499:40:22,  8.23s/it]
  3%|▎         | 6443/225000 [23:16:49<683:03:55, 11.25s/it]
  3%|▎         | 6444/225000 [23:16:54<556:06:36,  9.16s/it]
  3%|▎         | 6445/225000 [23:17:04<578:59:24,  9.54s/it]
  3%|▎         | 6446/225000 [23:17:15<598:20:09,  9.86s/it]
  3%|▎         | 6447/225000 [23:17:19<495:57:57,  8.17s/it]
  3%|▎         | 6448/225000 [23:17:30<543:43:11,  8.96s/it]
  3%|▎         | 6449/225000 [23:17:37<509:54:21,  8.40s/it]
  3%|▎         | 6450/225000 [23:17:43<461:43:43,  7.61s/it]
                                                            
{'loss': '0.6564', 'grad_norm': '1.222', 'learning_rate': '3.222e-05', 'epoch': '0.02867', 'num_input_tokens_seen': 42566512, 'train_runtime': '8.386e+04', 'train_tokens_per_second': '507.6'}

  3%|▎         | 6450/225000 [23:17:43<461:43:43,  7.61s/it]
  3%|▎         | 6451/225000 [23:17:53<504:33:31,  8.31s/it]
  3%|▎         | 6452/225000 [23:18:02<519:37:55,  8.56s/it]
  3%|▎         | 6453/225000 [23:18:13<570:56:47,  9.40s/it]
  3%|▎         | 6454/225000 [23:18:21<537:39:52,  8.86s/it]
  3%|▎         | 6455/225000 [23:18:28<510:25:57,  8.41s/it]
  3%|▎         | 6456/225000 [23:18:47<696:42:51, 11.48s/it]
  3%|▎         | 6457/225000 [23:18:53<596:52:10,  9.83s/it]
  3%|▎         | 6458/225000 [23:19:08<689:29:44, 11.36s/it]
  3%|▎         | 6459/225000 [23:19:14<600:29:48,  9.89s/it]
  3%|▎         | 6460/225000 [23:19:24<602:35:36,  9.93s/it]
  3%|▎         | 6461/225000 [23:19:32<571:37:58,  9.42s/it]
  3%|▎         | 6462/225000 [23:19:45<629:18:41, 10.37s/it]
  3%|▎         | 6463/225000 [23:19:49<513:58:57,  8.47s/it]
  3%|▎         | 6464/225000 [23:19:57<511:05:42,  8.42s/it]
  3%|▎         | 6465/225000 [23:20:03<461:03:11,  7.60s/it]
  3%|▎         | 6466/225000 [23:20:18<601:32:44,  9.91s/it]
  3%|▎         | 6467/225000 [23:20:33<684:06:33, 11.27s/it]
  3%|▎         | 6468/225000 [23:20:39<596:11:39,  9.82s/it]
  3%|▎         | 6469/225000 [23:20:50<617:11:53, 10.17s/it]
  3%|▎         | 6470/225000 [23:21:00<616:47:45, 10.16s/it]
  3%|▎         | 6471/225000 [23:21:05<527:05:49,  8.68s/it]
  3%|▎         | 6472/225000 [23:21:20<633:17:41, 10.43s/it]
  3%|▎         | 6473/225000 [23:21:30<628:45:03, 10.36s/it]
  3%|▎         | 6474/225000 [23:21:36<548:50:06,  9.04s/it]
  3%|▎         | 6475/225000 [23:21:43<518:21:00,  8.54s/it]
  3%|▎         | 6476/225000 [23:21:54<555:31:58,  9.15s/it]
  3%|▎         | 6477/225000 [23:22:07<618:39:25, 10.19s/it]
  3%|▎         | 6478/225000 [23:22:14<566:47:06,  9.34s/it]
  3%|▎         | 6479/225000 [23:22:26<606:56:50, 10.00s/it]
  3%|▎         | 6480/225000 [23:22:34<585:09:28,  9.64s/it]
  3%|▎         | 6481/225000 [23:22:40<506:29:31,  8.34s/it]
  3%|▎         | 6482/225000 [23:22:52<576:46:19,  9.50s/it]
  3%|▎         | 6483/225000 [23:23:04<630:14:26, 10.38s/it]
  3%|▎         | 6484/225000 [23:23:08<509:15:00,  8.39s/it]
  3%|▎         | 6485/225000 [23:23:17<511:18:54,  8.42s/it]
  3%|▎         | 6486/225000 [23:23:27<541:27:08,  8.92s/it]
  3%|▎         | 6487/225000 [23:23:36<553:24:49,  9.12s/it]
  3%|▎         | 6488/225000 [23:23:50<629:43:52, 10.37s/it]
  3%|▎         | 6489/225000 [23:23:55<543:18:47,  8.95s/it]
  3%|▎         | 6490/225000 [23:24:03<522:08:55,  8.60s/it]
  3%|▎         | 6491/225000 [23:24:14<570:38:17,  9.40s/it]
  3%|▎         | 6492/225000 [23:24:19<492:23:02,  8.11s/it]
  3%|▎         | 6493/225000 [23:24:25<447:14:11,  7.37s/it]
  3%|▎         | 6494/225000 [23:24:38<541:14:09,  8.92s/it]
  3%|▎         | 6495/225000 [23:24:46<526:55:13,  8.68s/it]
  3%|▎         | 6496/225000 [23:25:01<645:17:06, 10.63s/it]
  3%|▎         | 6497/225000 [23:25:08<584:02:27,  9.62s/it]
  3%|▎         | 6498/225000 [23:25:16<549:09:30,  9.05s/it]
  3%|▎         | 6499/225000 [23:25:25<545:46:56,  8.99s/it]
  3%|▎         | 6500/225000 [23:25:34<549:22:44,  9.05s/it]
                                                            
{'loss': '0.7114', 'grad_norm': '1.289', 'learning_rate': '3.247e-05', 'epoch': '0.02889', 'num_input_tokens_seen': 42790480, 'train_runtime': '8.433e+04', 'train_tokens_per_second': '507.4'}

  3%|▎         | 6500/225000 [23:25:34<549:22:44,  9.05s/it]
  3%|▎         | 6501/225000 [23:25:41<506:57:15,  8.35s/it]
  3%|▎         | 6502/225000 [23:25:48<489:27:04,  8.06s/it]
  3%|▎         | 6503/225000 [23:26:00<561:46:47,  9.26s/it]
  3%|▎         | 6504/225000 [23:26:09<559:01:23,  9.21s/it]
  3%|▎         | 6505/225000 [23:26:15<497:42:13,  8.20s/it]
  3%|▎         | 6506/225000 [23:26:22<478:10:06,  7.88s/it]
  3%|▎         | 6507/225000 [23:26:27<432:26:30,  7.13s/it]
  3%|▎         | 6508/225000 [23:26:36<458:34:07,  7.56s/it]
  3%|▎         | 6509/225000 [23:26:49<548:51:49,  9.04s/it]
  3%|▎         | 6510/225000 [23:26:59<568:16:37,  9.36s/it]
  3%|▎         | 6511/225000 [23:27:12<647:48:06, 10.67s/it]
  3%|▎         | 6512/225000 [23:27:17<534:59:11,  8.81s/it]
  3%|▎         | 6513/225000 [23:27:24<509:21:11,  8.39s/it]
  3%|▎         | 6514/225000 [23:27:31<485:28:21,  8.00s/it]
  3%|▎         | 6515/225000 [23:27:36<430:29:47,  7.09s/it]
  3%|▎         | 6516/225000 [23:27:49<535:08:18,  8.82s/it]
  3%|▎         | 6517/225000 [23:27:53<438:46:44,  7.23s/it]
  3%|▎         | 6518/225000 [23:28:05<536:17:40,  8.84s/it]
  3%|▎         | 6519/225000 [23:28:16<570:07:56,  9.39s/it]
  3%|▎         | 6520/225000 [23:28:22<516:32:17,  8.51s/it]
  3%|▎         | 6521/225000 [23:28:27<439:14:36,  7.24s/it]
  3%|▎         | 6522/225000 [23:28:33<419:34:20,  6.91s/it]
  3%|▎         | 6523/225000 [23:28:49<588:37:04,  9.70s/it]
  3%|▎         | 6524/225000 [23:28:58<582:58:02,  9.61s/it]
  3%|▎         | 6525/225000 [23:29:12<652:39:48, 10.75s/it]
  3%|▎         | 6526/225000 [23:29:19<592:25:45,  9.76s/it]
  3%|▎         | 6527/225000 [23:29:29<598:03:29,  9.85s/it]
  3%|▎         | 6528/225000 [23:29:38<567:55:41,  9.36s/it]
  3%|▎         | 6529/225000 [23:29:45<531:09:00,  8.75s/it]
  3%|▎         | 6530/225000 [23:29:51<482:50:09,  7.96s/it]
  3%|▎         | 6531/225000 [23:29:56<428:56:34,  7.07s/it]
  3%|▎         | 6532/225000 [23:30:01<393:35:48,  6.49s/it]
  3%|▎         | 6533/225000 [23:30:10<441:09:43,  7.27s/it]
  3%|▎         | 6534/225000 [23:30:24<556:10:51,  9.17s/it]
  3%|▎         | 6535/225000 [23:30:32<533:08:57,  8.79s/it]
  3%|▎         | 6536/225000 [23:30:37<461:28:29,  7.60s/it]
  3%|▎         | 6537/225000 [23:30:50<565:07:08,  9.31s/it]
  3%|▎         | 6538/225000 [23:30:59<569:17:15,  9.38s/it]
  3%|▎         | 6539/225000 [23:31:07<531:54:57,  8.77s/it]
  3%|▎         | 6540/225000 [23:31:19<604:14:40,  9.96s/it]
  3%|▎         | 6541/225000 [23:31:28<577:57:08,  9.52s/it]
  3%|▎         | 6542/225000 [23:31:36<543:54:42,  8.96s/it]
  3%|▎         | 6543/225000 [23:31:42<494:26:24,  8.15s/it]
  3%|▎         | 6544/225000 [23:31:51<520:31:52,  8.58s/it]
  3%|▎         | 6545/225000 [23:32:01<530:51:32,  8.75s/it]
  3%|▎         | 6546/225000 [23:32:10<543:19:57,  8.95s/it]
  3%|▎         | 6547/225000 [23:32:14<453:48:48,  7.48s/it]
  3%|▎         | 6548/225000 [23:32:23<472:09:31,  7.78s/it]
  3%|▎         | 6549/225000 [23:32:32<503:17:42,  8.29s/it]
  3%|▎         | 6550/225000 [23:32:47<619:24:56, 10.21s/it]
                                                            
{'loss': '0.71', 'grad_norm': '1.695', 'learning_rate': '3.272e-05', 'epoch': '0.02911', 'num_input_tokens_seen': 43002072, 'train_runtime': '8.477e+04', 'train_tokens_per_second': '507.3'}

  3%|▎         | 6550/225000 [23:32:47<619:24:56, 10.21s/it]
  3%|▎         | 6551/225000 [23:32:55<591:37:53,  9.75s/it]
  3%|▎         | 6552/225000 [23:33:02<530:22:20,  8.74s/it]
  3%|▎         | 6553/225000 [23:33:13<569:29:44,  9.39s/it]
  3%|▎         | 6554/225000 [23:33:26<632:39:39, 10.43s/it]
  3%|▎         | 6555/225000 [23:33:31<539:32:34,  8.89s/it]
  3%|▎         | 6556/225000 [23:33:42<582:15:43,  9.60s/it]
  3%|▎         | 6557/225000 [23:33:56<657:14:29, 10.83s/it]
  3%|▎         | 6558/225000 [23:34:04<614:27:54, 10.13s/it]
  3%|▎         | 6559/225000 [23:34:14<606:28:48, 10.00s/it]
  3%|▎         | 6560/225000 [23:34:24<604:45:24,  9.97s/it]
  3%|▎         | 6561/225000 [23:34:31<555:37:23,  9.16s/it]
  3%|▎         | 6562/225000 [23:34:45<632:25:13, 10.42s/it]
  3%|▎         | 6563/225000 [23:34:55<639:47:50, 10.54s/it]
  3%|▎         | 6564/225000 [23:35:00<539:03:49,  8.88s/it]
  3%|▎         | 6565/225000 [23:35:09<536:06:59,  8.84s/it]
  3%|▎         | 6566/225000 [23:35:22<604:46:12,  9.97s/it]
  3%|▎         | 6567/225000 [23:35:30<575:25:35,  9.48s/it]
  3%|▎         | 6568/225000 [23:35:41<611:05:05, 10.07s/it]
  3%|▎         | 6569/225000 [23:35:50<580:42:11,  9.57s/it]
  3%|▎         | 6570/225000 [23:35:55<498:20:17,  8.21s/it]
  3%|▎         | 6571/225000 [23:36:05<534:41:07,  8.81s/it]
  3%|▎         | 6572/225000 [23:36:21<659:13:16, 10.86s/it]
  3%|▎         | 6573/225000 [23:36:26<552:03:08,  9.10s/it]
  3%|▎         | 6574/225000 [23:36:34<538:25:12,  8.87s/it]
  3%|▎         | 6575/225000 [23:36:41<506:05:16,  8.34s/it]
  3%|▎         | 6576/225000 [23:36:47<464:55:14,  7.66s/it]
  3%|▎         | 6577/225000 [23:36:56<479:25:24,  7.90s/it]
  3%|▎         | 6578/225000 [23:37:08<562:26:25,  9.27s/it]
  3%|▎         | 6579/225000 [23:37:15<511:31:56,  8.43s/it]
  3%|▎         | 6580/225000 [23:37:22<497:45:19,  8.20s/it]
  3%|▎         | 6581/225000 [23:37:38<631:13:28, 10.40s/it]
  3%|▎         | 6582/225000 [23:37:49<645:36:32, 10.64s/it]
  3%|▎         | 6583/225000 [23:37:57<597:51:14,  9.85s/it]
  3%|▎         | 6584/225000 [23:38:09<637:55:14, 10.51s/it]
  3%|▎         | 6585/225000 [23:38:16<562:52:59,  9.28s/it]
  3%|▎         | 6586/225000 [23:38:25<569:18:03,  9.38s/it]
  3%|▎         | 6587/225000 [23:38:31<495:28:34,  8.17s/it]
  3%|▎         | 6588/225000 [23:38:40<515:20:56,  8.49s/it]
  3%|▎         | 6589/225000 [23:38:49<521:53:26,  8.60s/it]
  3%|▎         | 6590/225000 [23:38:57<514:32:32,  8.48s/it]
  3%|▎         | 6591/225000 [23:39:03<471:40:56,  7.77s/it]
  3%|▎         | 6592/225000 [23:39:10<451:04:47,  7.44s/it]
  3%|▎         | 6593/225000 [23:39:16<426:25:38,  7.03s/it]
  3%|▎         | 6594/225000 [23:39:23<422:41:44,  6.97s/it]
  3%|▎         | 6595/225000 [23:39:29<420:25:11,  6.93s/it]
  3%|▎         | 6596/225000 [23:39:44<566:21:15,  9.34s/it]
  3%|▎         | 6597/225000 [23:39:53<554:10:23,  9.13s/it]
  3%|▎         | 6598/225000 [23:39:59<503:50:55,  8.31s/it]
  3%|▎         | 6599/225000 [23:40:04<444:46:11,  7.33s/it]
  3%|▎         | 6600/225000 [23:40:19<573:07:56,  9.45s/it]
                                                            
{'loss': '0.6905', 'grad_norm': '1.083', 'learning_rate': '3.297e-05', 'epoch': '0.02933', 'num_input_tokens_seen': 43222544, 'train_runtime': '8.522e+04', 'train_tokens_per_second': '507.2'}

  3%|▎         | 6600/225000 [23:40:19<573:07:56,  9.45s/it]
  3%|▎         | 6601/225000 [23:40:28<572:54:33,  9.44s/it]
  3%|▎         | 6602/225000 [23:40:37<553:36:08,  9.13s/it]
  3%|▎         | 6603/225000 [23:40:47<582:14:38,  9.60s/it]
  3%|▎         | 6604/225000 [23:40:55<542:36:16,  8.94s/it]
  3%|▎         | 6605/225000 [23:41:01<500:03:24,  8.24s/it]
  3%|▎         | 6606/225000 [23:41:07<456:37:21,  7.53s/it]
  3%|▎         | 6607/225000 [23:41:12<406:23:18,  6.70s/it]
  3%|▎         | 6608/225000 [23:41:27<561:02:42,  9.25s/it]
  3%|▎         | 6609/225000 [23:41:32<480:26:58,  7.92s/it]
  3%|▎         | 6610/225000 [23:41:41<503:55:43,  8.31s/it]
  3%|▎         | 6611/225000 [23:41:52<558:16:24,  9.20s/it]
  3%|▎         | 6612/225000 [23:42:04<592:17:32,  9.76s/it]
  3%|▎         | 6613/225000 [23:42:15<619:14:45, 10.21s/it]
  3%|▎         | 6614/225000 [23:42:25<628:24:51, 10.36s/it]
  3%|▎         | 6615/225000 [23:42:33<577:45:40,  9.52s/it]
  3%|▎         | 6616/225000 [23:42:42<574:39:39,  9.47s/it]
  3%|▎         | 6617/225000 [23:42:52<574:28:19,  9.47s/it]
  3%|▎         | 6618/225000 [23:42:58<520:04:41,  8.57s/it]
  3%|▎         | 6619/225000 [23:43:10<581:47:16,  9.59s/it]
  3%|▎         | 6620/225000 [23:43:17<521:25:24,  8.60s/it]
  3%|▎         | 6621/225000 [23:43:25<515:51:13,  8.50s/it]
  3%|▎         | 6622/225000 [23:43:35<551:55:28,  9.10s/it]
  3%|▎         | 6623/225000 [23:43:41<487:59:44,  8.04s/it]
  3%|▎         | 6624/225000 [23:43:47<455:22:42,  7.51s/it]
  3%|▎         | 6625/225000 [23:43:55<467:53:43,  7.71s/it]
  3%|▎         | 6626/225000 [23:44:02<440:30:07,  7.26s/it]
  3%|▎         | 6627/225000 [23:44:14<525:52:45,  8.67s/it]
  3%|▎         | 6628/225000 [23:44:19<468:52:52,  7.73s/it]
  3%|▎         | 6629/225000 [23:44:28<496:00:33,  8.18s/it]
  3%|▎         | 6630/225000 [23:44:34<448:57:28,  7.40s/it]
  3%|▎         | 6631/225000 [23:44:44<491:49:01,  8.11s/it]
  3%|▎         | 6632/225000 [23:44:52<492:21:32,  8.12s/it]
  3%|▎         | 6633/225000 [23:45:03<550:58:48,  9.08s/it]
  3%|▎         | 6634/225000 [23:45:09<486:43:58,  8.02s/it]
  3%|▎         | 6635/225000 [23:45:15<462:49:31,  7.63s/it]
  3%|▎         | 6636/225000 [23:45:30<586:44:34,  9.67s/it]
  3%|▎         | 6637/225000 [23:45:39<577:40:49,  9.52s/it]
  3%|▎         | 6638/225000 [23:45:45<505:33:10,  8.33s/it]
  3%|▎         | 6639/225000 [23:45:53<501:57:19,  8.28s/it]
  3%|▎         | 6640/225000 [23:45:58<444:38:45,  7.33s/it]
  3%|▎         | 6641/225000 [23:46:10<524:45:43,  8.65s/it]
  3%|▎         | 6642/225000 [23:46:21<576:55:41,  9.51s/it]
  3%|▎         | 6643/225000 [23:46:25<479:42:57,  7.91s/it]
  3%|▎         | 6644/225000 [23:46:35<505:50:43,  8.34s/it]
  3%|▎         | 6645/225000 [23:46:45<535:41:08,  8.83s/it]
  3%|▎         | 6646/225000 [23:46:51<491:42:27,  8.11s/it]
  3%|▎         | 6647/225000 [23:47:03<568:57:13,  9.38s/it]
  3%|▎         | 6648/225000 [23:47:10<522:22:50,  8.61s/it]
  3%|▎         | 6649/225000 [23:47:17<486:58:07,  8.03s/it]
  3%|▎         | 6650/225000 [23:47:27<520:00:16,  8.57s/it]
                                                            
{'loss': '0.7068', 'grad_norm': '0.8552', 'learning_rate': '3.322e-05', 'epoch': '0.02956', 'num_input_tokens_seen': 43427592, 'train_runtime': '8.565e+04', 'train_tokens_per_second': '507.1'}

  3%|▎         | 6650/225000 [23:47:27<520:00:16,  8.57s/it]
  3%|▎         | 6651/225000 [23:47:37<559:16:48,  9.22s/it]
  3%|▎         | 6652/225000 [23:47:51<641:30:55, 10.58s/it]
  3%|▎         | 6653/225000 [23:48:00<616:35:45, 10.17s/it]
  3%|▎         | 6654/225000 [23:48:07<559:18:20,  9.22s/it]
  3%|▎         | 6655/225000 [23:48:22<656:39:52, 10.83s/it]
  3%|▎         | 6656/225000 [23:48:28<568:54:22,  9.38s/it]
  3%|▎         | 6657/225000 [23:48:33<497:55:37,  8.21s/it]
  3%|▎         | 6658/225000 [23:48:50<641:29:23, 10.58s/it]
  3%|▎         | 6659/225000 [23:48:59<621:13:10, 10.24s/it]
  3%|▎         | 6660/225000 [23:49:06<566:32:38,  9.34s/it]
  3%|▎         | 6661/225000 [23:49:19<626:36:46, 10.33s/it]
  3%|▎         | 6662/225000 [23:49:36<753:40:45, 12.43s/it]
  3%|▎         | 6663/225000 [23:49:47<715:15:58, 11.79s/it]
  3%|▎         | 6664/225000 [23:49:57<690:15:11, 11.38s/it]
  3%|▎         | 6665/225000 [23:50:07<665:53:02, 10.98s/it]
  3%|▎         | 6666/225000 [23:50:16<630:05:38, 10.39s/it]
  3%|▎         | 6667/225000 [23:50:24<590:15:43,  9.73s/it]
  3%|▎         | 6668/225000 [23:50:33<566:03:03,  9.33s/it]
  3%|▎         | 6669/225000 [23:50:39<509:24:59,  8.40s/it]
  3%|▎         | 6670/225000 [23:50:46<485:25:43,  8.00s/it]
  3%|▎         | 6671/225000 [23:50:51<440:35:38,  7.26s/it]
  3%|▎         | 6672/225000 [23:50:58<423:12:28,  6.98s/it]
  3%|▎         | 6673/225000 [23:51:05<427:47:58,  7.05s/it]
  3%|▎         | 6674/225000 [23:51:17<515:55:53,  8.51s/it]
  3%|▎         | 6675/225000 [23:51:24<489:59:30,  8.08s/it]
  3%|▎         | 6676/225000 [23:51:36<566:03:55,  9.33s/it]
  3%|▎         | 6677/225000 [23:51:41<476:55:26,  7.86s/it]
  3%|▎         | 6678/225000 [23:51:51<518:56:07,  8.56s/it]
  3%|▎         | 6679/225000 [23:51:56<465:38:39,  7.68s/it]
  3%|▎         | 6680/225000 [23:52:10<563:57:49,  9.30s/it]
  3%|▎         | 6681/225000 [23:52:15<491:49:52,  8.11s/it]
  3%|▎         | 6682/225000 [23:52:21<457:25:35,  7.54s/it]
  3%|▎         | 6683/225000 [23:52:30<490:40:39,  8.09s/it]
  3%|▎         | 6684/225000 [23:52:44<589:26:45,  9.72s/it]
  3%|▎         | 6685/225000 [23:52:54<599:13:15,  9.88s/it]
  3%|▎         | 6686/225000 [23:53:03<576:14:35,  9.50s/it]
  3%|▎         | 6687/225000 [23:53:11<555:19:45,  9.16s/it]
  3%|▎         | 6688/225000 [23:53:17<500:26:56,  8.25s/it]
  3%|▎         | 6689/225000 [23:53:35<671:47:07, 11.08s/it]
  3%|▎         | 6690/225000 [23:53:42<588:29:30,  9.70s/it]
  3%|▎         | 6691/225000 [23:53:48<525:57:07,  8.67s/it]
  3%|▎         | 6692/225000 [23:54:00<593:33:42,  9.79s/it]
  3%|▎         | 6693/225000 [23:54:13<648:07:53, 10.69s/it]
  3%|▎         | 6694/225000 [23:54:20<574:20:47,  9.47s/it]
  3%|▎         | 6695/225000 [23:54:28<559:17:12,  9.22s/it]
  3%|▎         | 6696/225000 [23:54:37<555:28:58,  9.16s/it]
  3%|▎         | 6697/225000 [23:54:44<506:48:02,  8.36s/it]
  3%|▎         | 6698/225000 [23:54:53<519:56:13,  8.57s/it]
  3%|▎         | 6699/225000 [23:55:00<491:00:47,  8.10s/it]
  3%|▎         | 6700/225000 [23:55:12<557:56:50,  9.20s/it]
                                                            
{'loss': '0.6525', 'grad_norm': '1.295', 'learning_rate': '3.347e-05', 'epoch': '0.02978', 'num_input_tokens_seen': 43650648, 'train_runtime': '8.611e+04', 'train_tokens_per_second': '506.9'}

  3%|▎         | 6700/225000 [23:55:12<557:56:50,  9.20s/it]
  3%|▎         | 6701/225000 [23:55:24<624:59:56, 10.31s/it]
  3%|▎         | 6702/225000 [23:55:36<652:57:06, 10.77s/it]
  3%|▎         | 6703/225000 [23:55:44<587:59:31,  9.70s/it]
  3%|▎         | 6704/225000 [23:55:52<563:21:40,  9.29s/it]
  3%|▎         | 6705/225000 [23:56:00<538:34:55,  8.88s/it]
  3%|▎         | 6706/225000 [23:56:07<514:49:58,  8.49s/it]
  3%|▎         | 6707/225000 [23:56:13<463:05:03,  7.64s/it]
  3%|▎         | 6708/225000 [23:56:18<415:29:44,  6.85s/it]
  3%|▎         | 6709/225000 [23:56:30<514:59:23,  8.49s/it]
  3%|▎         | 6710/225000 [23:56:39<523:54:19,  8.64s/it]
  3%|▎         | 6711/225000 [23:56:49<546:06:16,  9.01s/it]
  3%|▎         | 6712/225000 [23:56:55<480:44:48,  7.93s/it]
  3%|▎         | 6713/225000 [23:57:06<536:22:31,  8.85s/it]
  3%|▎         | 6714/225000 [23:57:10<448:57:21,  7.40s/it]
  3%|▎         | 6715/225000 [23:57:15<410:47:19,  6.77s/it]
  3%|▎         | 6716/225000 [23:57:26<487:41:15,  8.04s/it]
  3%|▎         | 6717/225000 [23:57:38<567:54:32,  9.37s/it]
  3%|▎         | 6718/225000 [23:57:48<569:11:54,  9.39s/it]
  3%|▎         | 6719/225000 [23:57:52<480:18:01,  7.92s/it]
  3%|▎         | 6720/225000 [23:57:58<435:08:30,  7.18s/it]
  3%|▎         | 6721/225000 [23:58:03<400:53:01,  6.61s/it]
  3%|▎         | 6722/225000 [23:58:14<483:04:53,  7.97s/it]
  3%|▎         | 6723/225000 [23:58:22<473:51:53,  7.82s/it]
  3%|▎         | 6724/225000 [23:58:36<591:18:04,  9.75s/it]
  3%|▎         | 6725/225000 [23:58:44<565:18:01,  9.32s/it]
  3%|▎         | 6726/225000 [23:58:53<557:03:58,  9.19s/it]
  3%|▎         | 6727/225000 [23:59:02<548:20:32,  9.04s/it]
  3%|▎         | 6728/225000 [23:59:06<464:49:42,  7.67s/it]
  3%|▎         | 6729/225000 [23:59:12<433:29:08,  7.15s/it]
  3%|▎         | 6730/225000 [23:59:22<489:47:00,  8.08s/it]
  3%|▎         | 6731/225000 [23:59:33<535:00:32,  8.82s/it]
  3%|▎         | 6732/225000 [23:59:38<464:36:22,  7.66s/it]
  3%|▎         | 6733/225000 [23:59:46<461:44:56,  7.62s/it]
  3%|▎         | 6734/225000 [23:59:56<514:34:59,  8.49s/it]
  3%|▎         | 6735/225000 [24:00:00<439:22:00,  7.25s/it]
  3%|▎         | 6736/225000 [24:00:09<465:20:53,  7.68s/it]
  3%|▎         | 6737/225000 [24:00:13<404:06:23,  6.67s/it]
  3%|▎         | 6738/225000 [24:00:20<402:13:26,  6.63s/it]
  3%|▎         | 6739/225000 [24:00:32<504:51:12,  8.33s/it]
  3%|▎         | 6740/225000 [24:00:39<481:58:55,  7.95s/it]
  3%|▎         | 6741/225000 [24:00:44<429:44:52,  7.09s/it]
  3%|▎         | 6742/225000 [24:00:50<408:34:30,  6.74s/it]
  3%|▎         | 6743/225000 [24:01:02<505:11:57,  8.33s/it]
  3%|▎         | 6744/225000 [24:01:08<450:21:46,  7.43s/it]
  3%|▎         | 6745/225000 [24:01:23<593:54:50,  9.80s/it]
  3%|▎         | 6746/225000 [24:01:33<600:21:51,  9.90s/it]
  3%|▎         | 6747/225000 [24:01:40<551:41:48,  9.10s/it]
  3%|▎         | 6748/225000 [24:01:54<637:21:48, 10.51s/it]
  3%|▎         | 6749/225000 [24:01:59<541:37:33,  8.93s/it]
  3%|▎         | 6750/225000 [24:02:10<579:37:40,  9.56s/it]
                                                            
{'loss': '0.6691', 'grad_norm': '1.286', 'learning_rate': '3.372e-05', 'epoch': '0.03', 'num_input_tokens_seen': 43852368, 'train_runtime': '8.653e+04', 'train_tokens_per_second': '506.8'}

  3%|▎         | 6750/225000 [24:02:10<579:37:40,  9.56s/it]
  3%|▎         | 6751/225000 [24:02:23<633:02:10, 10.44s/it]
  3%|▎         | 6752/225000 [24:02:28<531:21:58,  8.76s/it]
  3%|▎         | 6753/225000 [24:02:35<502:33:39,  8.29s/it]
  3%|▎         | 6754/225000 [24:02:46<548:20:13,  9.04s/it]
  3%|▎         | 6755/225000 [24:02:56<576:44:48,  9.51s/it]
  3%|▎         | 6756/225000 [24:03:04<534:04:17,  8.81s/it]
  3%|▎         | 6757/225000 [24:03:08<456:30:09,  7.53s/it]
  3%|▎         | 6758/225000 [24:03:20<528:58:54,  8.73s/it]
  3%|▎         | 6759/225000 [24:03:31<574:50:21,  9.48s/it]
  3%|▎         | 6760/225000 [24:03:37<515:28:40,  8.50s/it]
  3%|▎         | 6761/225000 [24:03:42<442:18:42,  7.30s/it]
  3%|▎         | 6762/225000 [24:03:50<468:08:16,  7.72s/it]
  3%|▎         | 6763/225000 [24:03:58<467:29:26,  7.71s/it]
  3%|▎         | 6764/225000 [24:04:10<553:31:44,  9.13s/it]
  3%|▎         | 6765/225000 [24:04:21<585:32:48,  9.66s/it]
  3%|▎         | 6766/225000 [24:04:29<549:51:26,  9.07s/it]
  3%|▎         | 6767/225000 [24:04:36<504:41:49,  8.33s/it]
  3%|▎         | 6768/225000 [24:04:41<449:51:55,  7.42s/it]
  3%|▎         | 6769/225000 [24:04:50<473:47:28,  7.82s/it]
  3%|▎         | 6770/225000 [24:05:03<569:09:09,  9.39s/it]
  3%|▎         | 6771/225000 [24:05:07<483:24:02,  7.97s/it]
  3%|▎         | 6772/225000 [24:05:15<472:14:16,  7.79s/it]
  3%|▎         | 6773/225000 [24:05:26<529:54:57,  8.74s/it]
  3%|▎         | 6774/225000 [24:05:35<537:36:11,  8.87s/it]
  3%|▎         | 6775/225000 [24:05:43<528:33:06,  8.72s/it]
  3%|▎         | 6776/225000 [24:05:49<466:43:42,  7.70s/it]
  3%|▎         | 6777/225000 [24:06:02<578:47:58,  9.55s/it]
  3%|▎         | 6778/225000 [24:06:13<591:10:19,  9.75s/it]
  3%|▎         | 6779/225000 [24:06:26<649:41:49, 10.72s/it]
  3%|▎         | 6780/225000 [24:06:40<717:11:21, 11.83s/it]
  3%|▎         | 6781/225000 [24:06:46<608:38:37, 10.04s/it]
  3%|▎         | 6782/225000 [24:06:55<590:20:25,  9.74s/it]
  3%|▎         | 6783/225000 [24:07:10<695:09:02, 11.47s/it]
  3%|▎         | 6784/225000 [24:07:14<559:36:58,  9.23s/it]
  3%|▎         | 6785/225000 [24:07:22<535:21:16,  8.83s/it]
  3%|▎         | 6786/225000 [24:07:31<523:37:30,  8.64s/it]
  3%|▎         | 6787/225000 [24:07:40<539:13:50,  8.90s/it]
  3%|▎         | 6788/225000 [24:07:45<467:25:29,  7.71s/it]
  3%|▎         | 6789/225000 [24:07:50<425:59:12,  7.03s/it]
  3%|▎         | 6790/225000 [24:08:04<544:40:33,  8.99s/it]
  3%|▎         | 6791/225000 [24:08:15<574:28:54,  9.48s/it]
  3%|▎         | 6792/225000 [24:08:20<491:39:56,  8.11s/it]
  3%|▎         | 6793/225000 [24:08:25<435:15:02,  7.18s/it]
  3%|▎         | 6794/225000 [24:08:35<499:26:23,  8.24s/it]
  3%|▎         | 6795/225000 [24:08:46<549:08:54,  9.06s/it]
  3%|▎         | 6796/225000 [24:08:51<467:25:53,  7.71s/it]
  3%|▎         | 6797/225000 [24:09:01<517:17:25,  8.53s/it]
  3%|▎         | 6798/225000 [24:09:13<567:34:52,  9.36s/it]
  3%|▎         | 6799/225000 [24:09:18<491:21:18,  8.11s/it]
  3%|▎         | 6800/225000 [24:09:23<432:35:47,  7.14s/it]
                                                            
{'loss': '0.7671', 'grad_norm': '1.326', 'learning_rate': '3.397e-05', 'epoch': '0.03022', 'num_input_tokens_seen': 44066208, 'train_runtime': '8.696e+04', 'train_tokens_per_second': '506.7'}

  3%|▎         | 6800/225000 [24:09:23<432:35:47,  7.14s/it]
  3%|▎         | 6801/225000 [24:09:33<495:57:39,  8.18s/it]
  3%|▎         | 6802/225000 [24:09:42<500:52:42,  8.26s/it]
  3%|▎         | 6803/225000 [24:09:54<581:14:36,  9.59s/it]
  3%|▎         | 6804/225000 [24:10:03<565:32:44,  9.33s/it]
  3%|▎         | 6805/225000 [24:10:14<593:06:01,  9.79s/it]
  3%|▎         | 6806/225000 [24:10:26<628:19:48, 10.37s/it]
  3%|▎         | 6807/225000 [24:10:34<600:28:48,  9.91s/it]
  3%|▎         | 6808/225000 [24:10:44<588:55:19,  9.72s/it]
  3%|▎         | 6809/225000 [24:10:49<508:42:12,  8.39s/it]
  3%|▎         | 6810/225000 [24:11:02<583:34:35,  9.63s/it]
  3%|▎         | 6811/225000 [24:11:08<532:03:17,  8.78s/it]
  3%|▎         | 6812/225000 [24:11:18<548:11:44,  9.04s/it]
  3%|▎         | 6813/225000 [24:11:29<582:50:55,  9.62s/it]
  3%|▎         | 6814/225000 [24:11:34<505:25:18,  8.34s/it]
  3%|▎         | 6815/225000 [24:11:43<519:22:01,  8.57s/it]
  3%|▎         | 6816/225000 [24:11:55<582:58:04,  9.62s/it]
  3%|▎         | 6817/225000 [24:12:05<573:37:01,  9.46s/it]
  3%|▎         | 6818/225000 [24:12:12<538:52:27,  8.89s/it]
  3%|▎         | 6819/225000 [24:12:20<527:17:21,  8.70s/it]
  3%|▎         | 6820/225000 [24:12:34<614:55:14, 10.15s/it]
  3%|▎         | 6821/225000 [24:12:47<673:51:26, 11.12s/it]
  3%|▎         | 6822/225000 [24:13:03<755:50:31, 12.47s/it]
  3%|▎         | 6823/225000 [24:13:11<682:24:33, 11.26s/it]
  3%|▎         | 6824/225000 [24:13:19<615:58:20, 10.16s/it]
  3%|▎         | 6825/225000 [24:13:27<578:27:32,  9.54s/it]
  3%|▎         | 6826/225000 [24:13:32<501:25:09,  8.27s/it]
  3%|▎         | 6827/225000 [24:13:40<492:54:10,  8.13s/it]
  3%|▎         | 6828/225000 [24:13:47<466:46:49,  7.70s/it]
  3%|▎         | 6829/225000 [24:13:52<425:17:45,  7.02s/it]
  3%|▎         | 6830/225000 [24:13:57<379:15:39,  6.26s/it]
  3%|▎         | 6831/225000 [24:14:14<573:07:19,  9.46s/it]
  3%|▎         | 6832/225000 [24:14:19<503:55:09,  8.32s/it]
  3%|▎         | 6833/225000 [24:14:30<538:51:46,  8.89s/it]
  3%|▎         | 6834/225000 [24:14:36<489:07:25,  8.07s/it]
  3%|▎         | 6835/225000 [24:14:49<581:25:34,  9.59s/it]
  3%|▎         | 6836/225000 [24:15:00<611:39:34, 10.09s/it]
  3%|▎         | 6837/225000 [24:15:10<606:00:03, 10.00s/it]
  3%|▎         | 6838/225000 [24:15:16<530:41:39,  8.76s/it]
  3%|▎         | 6839/225000 [24:15:27<575:15:15,  9.49s/it]
  3%|▎         | 6840/225000 [24:15:37<576:46:23,  9.52s/it]
  3%|▎         | 6841/225000 [24:15:43<519:27:43,  8.57s/it]
  3%|▎         | 6842/225000 [24:15:54<567:10:52,  9.36s/it]
  3%|▎         | 6843/225000 [24:15:59<477:36:07,  7.88s/it]
  3%|▎         | 6844/225000 [24:16:04<435:40:56,  7.19s/it]
  3%|▎         | 6845/225000 [24:16:16<521:43:24,  8.61s/it]
  3%|▎         | 6846/225000 [24:16:25<529:15:02,  8.73s/it]
  3%|▎         | 6847/225000 [24:16:37<588:08:39,  9.71s/it]
  3%|▎         | 6848/225000 [24:16:42<494:37:05,  8.16s/it]
  3%|▎         | 6849/225000 [24:16:52<537:53:41,  8.88s/it]
  3%|▎         | 6850/225000 [24:17:05<607:13:12, 10.02s/it]
                                                            
{'loss': '0.6913', 'grad_norm': '0.9642', 'learning_rate': '3.422e-05', 'epoch': '0.03044', 'num_input_tokens_seen': 44284168, 'train_runtime': '8.743e+04', 'train_tokens_per_second': '506.5'}

  3%|▎         | 6850/225000 [24:17:05<607:13:12, 10.02s/it]
  3%|▎         | 6851/225000 [24:17:15<606:29:58, 10.01s/it]
  3%|▎         | 6852/225000 [24:17:30<696:04:32, 11.49s/it]
  3%|▎         | 6853/225000 [24:17:36<595:40:52,  9.83s/it]
  3%|▎         | 6854/225000 [24:17:45<587:03:40,  9.69s/it]
  3%|▎         | 6855/225000 [24:17:51<518:14:42,  8.55s/it]
  3%|▎         | 6856/225000 [24:18:03<579:16:47,  9.56s/it]
  3%|▎         | 6857/225000 [24:18:09<516:13:14,  8.52s/it]
  3%|▎         | 6858/225000 [24:18:16<480:47:35,  7.93s/it]
  3%|▎         | 6859/225000 [24:18:28<555:51:32,  9.17s/it]
  3%|▎         | 6860/225000 [24:18:35<521:15:13,  8.60s/it]
  3%|▎         | 6861/225000 [24:18:40<456:52:16,  7.54s/it]
  3%|▎         | 6862/225000 [24:18:55<590:32:48,  9.75s/it]
  3%|▎         | 6863/225000 [24:19:00<512:52:16,  8.46s/it]
  3%|▎         | 6864/225000 [24:19:04<432:30:52,  7.14s/it]
  3%|▎         | 6865/225000 [24:19:10<397:26:05,  6.56s/it]
  3%|▎         | 6866/225000 [24:19:14<364:31:57,  6.02s/it]
  3%|▎         | 6867/225000 [24:19:23<408:41:39,  6.74s/it]
  3%|▎         | 6868/225000 [24:19:32<452:14:49,  7.46s/it]
  3%|▎         | 6869/225000 [24:19:37<414:25:29,  6.84s/it]
  3%|▎         | 6870/225000 [24:19:50<515:14:51,  8.50s/it]
  3%|▎         | 6871/225000 [24:19:57<486:08:45,  8.02s/it]
  3%|▎         | 6872/225000 [24:20:08<541:34:24,  8.94s/it]
  3%|▎         | 6873/225000 [24:20:13<481:25:06,  7.95s/it]
  3%|▎         | 6874/225000 [24:20:29<622:19:29, 10.27s/it]
  3%|▎         | 6875/225000 [24:20:35<540:07:34,  8.91s/it]
  3%|▎         | 6876/225000 [24:20:40<466:59:43,  7.71s/it]
  3%|▎         | 6877/225000 [24:20:49<505:09:43,  8.34s/it]
  3%|▎         | 6878/225000 [24:20:57<493:52:12,  8.15s/it]
  3%|▎         | 6879/225000 [24:21:07<517:36:02,  8.54s/it]
  3%|▎         | 6880/225000 [24:21:12<456:21:37,  7.53s/it]
  3%|▎         | 6881/225000 [24:21:21<482:01:52,  7.96s/it]
  3%|▎         | 6882/225000 [24:21:28<463:20:53,  7.65s/it]
  3%|▎         | 6883/225000 [24:21:34<434:56:53,  7.18s/it]
  3%|▎         | 6884/225000 [24:21:43<467:31:30,  7.72s/it]
  3%|▎         | 6885/225000 [24:21:48<421:58:19,  6.96s/it]
  3%|▎         | 6886/225000 [24:22:00<508:41:56,  8.40s/it]
  3%|▎         | 6887/225000 [24:22:05<455:41:33,  7.52s/it]
  3%|▎         | 6888/225000 [24:22:14<488:25:48,  8.06s/it]
  3%|▎         | 6889/225000 [24:22:19<429:56:50,  7.10s/it]
  3%|▎         | 6890/225000 [24:22:30<492:48:27,  8.13s/it]
  3%|▎         | 6891/225000 [24:22:35<441:33:05,  7.29s/it]
  3%|▎         | 6892/225000 [24:22:46<511:27:30,  8.44s/it]
  3%|▎         | 6893/225000 [24:22:54<496:37:09,  8.20s/it]
  3%|▎         | 6894/225000 [24:23:02<497:14:54,  8.21s/it]
  3%|▎         | 6895/225000 [24:23:08<453:55:44,  7.49s/it]
  3%|▎         | 6896/225000 [24:23:16<466:46:39,  7.70s/it]
  3%|▎         | 6897/225000 [24:23:28<536:49:15,  8.86s/it]
  3%|▎         | 6898/225000 [24:23:33<469:06:08,  7.74s/it]
  3%|▎         | 6899/225000 [24:23:40<456:12:07,  7.53s/it]
  3%|▎         | 6900/225000 [24:23:50<508:18:14,  8.39s/it]
                                                            
{'loss': '0.6853', 'grad_norm': '1.087', 'learning_rate': '3.447e-05', 'epoch': '0.03067', 'num_input_tokens_seen': 44482128, 'train_runtime': '8.783e+04', 'train_tokens_per_second': '506.5'}

  3%|▎         | 6900/225000 [24:23:50<508:18:14,  8.39s/it]
  3%|▎         | 6901/225000 [24:24:02<563:02:51,  9.29s/it]
  3%|▎         | 6902/225000 [24:24:09<522:09:38,  8.62s/it]
  3%|▎         | 6903/225000 [24:24:14<458:20:18,  7.57s/it]
  3%|▎         | 6904/225000 [24:24:20<425:32:36,  7.02s/it]
  3%|▎         | 6905/225000 [24:24:31<505:29:04,  8.34s/it]
  3%|▎         | 6906/225000 [24:24:39<490:41:21,  8.10s/it]
  3%|▎         | 6907/225000 [24:24:44<445:35:04,  7.36s/it]
  3%|▎         | 6908/225000 [24:24:56<529:24:50,  8.74s/it]
  3%|▎         | 6909/225000 [24:25:06<546:13:04,  9.02s/it]
  3%|▎         | 6910/225000 [24:25:11<474:05:12,  7.83s/it]
  3%|▎         | 6911/225000 [24:25:21<523:13:53,  8.64s/it]
  3%|▎         | 6912/225000 [24:25:29<511:25:19,  8.44s/it]
  3%|▎         | 6913/225000 [24:25:43<602:14:53,  9.94s/it]
  3%|▎         | 6914/225000 [24:25:53<612:47:44, 10.12s/it]
  3%|▎         | 6915/225000 [24:26:02<585:54:35,  9.67s/it]
  3%|▎         | 6916/225000 [24:26:12<588:41:07,  9.72s/it]
  3%|▎         | 6917/225000 [24:26:15<470:22:13,  7.76s/it]
  3%|▎         | 6918/225000 [24:26:33<656:13:36, 10.83s/it]
  3%|▎         | 6919/225000 [24:26:37<533:02:54,  8.80s/it]
  3%|▎         | 6920/225000 [24:26:49<581:23:50,  9.60s/it]
  3%|▎         | 6921/225000 [24:26:55<527:57:23,  8.72s/it]
  3%|▎         | 6922/225000 [24:27:04<534:09:09,  8.82s/it]
  3%|▎         | 6923/225000 [24:27:09<453:13:40,  7.48s/it]
  3%|▎         | 6924/225000 [24:27:17<474:53:58,  7.84s/it]
  3%|▎         | 6925/225000 [24:27:23<435:16:44,  7.19s/it]
  3%|▎         | 6926/225000 [24:27:36<539:58:24,  8.91s/it]
  3%|▎         | 6927/225000 [24:27:51<649:32:58, 10.72s/it]
  3%|▎         | 6928/225000 [24:27:57<570:08:19,  9.41s/it]
  3%|▎         | 6929/225000 [24:28:03<509:41:42,  8.41s/it]
  3%|▎         | 6930/225000 [24:28:10<481:15:28,  7.94s/it]
  3%|▎         | 6931/225000 [24:28:21<538:13:06,  8.89s/it]
  3%|▎         | 6932/225000 [24:28:27<486:06:15,  8.02s/it]
  3%|▎         | 6933/225000 [24:28:33<437:04:38,  7.22s/it]
  3%|▎         | 6934/225000 [24:28:40<449:07:42,  7.41s/it]
  3%|▎         | 6935/225000 [24:28:55<587:10:06,  9.69s/it]
  3%|▎         | 6936/225000 [24:29:06<594:42:46,  9.82s/it]
  3%|▎         | 6937/225000 [24:29:12<535:51:31,  8.85s/it]
  3%|▎         | 6938/225000 [24:29:21<544:59:28,  9.00s/it]
  3%|▎         | 6939/225000 [24:29:36<651:42:38, 10.76s/it]
  3%|▎         | 6940/225000 [24:29:44<595:06:41,  9.82s/it]
  3%|▎         | 6941/225000 [24:29:49<500:17:16,  8.26s/it]
  3%|▎         | 6942/225000 [24:29:55<465:29:45,  7.69s/it]
  3%|▎         | 6943/225000 [24:30:06<518:18:27,  8.56s/it]
  3%|▎         | 6944/225000 [24:30:14<524:53:09,  8.67s/it]
  3%|▎         | 6945/225000 [24:30:19<453:47:04,  7.49s/it]
  3%|▎         | 6946/225000 [24:30:30<513:07:57,  8.47s/it]
  3%|▎         | 6947/225000 [24:30:34<441:00:49,  7.28s/it]
  3%|▎         | 6948/225000 [24:30:42<452:21:31,  7.47s/it]
  3%|▎         | 6949/225000 [24:30:55<538:29:27,  8.89s/it]
  3%|▎         | 6950/225000 [24:31:05<560:33:31,  9.25s/it]
                                                            
{'loss': '0.7303', 'grad_norm': '0.9414', 'learning_rate': '3.472e-05', 'epoch': '0.03089', 'num_input_tokens_seen': 44692816, 'train_runtime': '8.827e+04', 'train_tokens_per_second': '506.3'}

  3%|▎         | 6950/225000 [24:31:05<560:33:31,  9.25s/it]
  3%|▎         | 6951/225000 [24:31:11<498:28:52,  8.23s/it]
  3%|▎         | 6952/225000 [24:31:17<461:30:40,  7.62s/it]
  3%|▎         | 6953/225000 [24:31:30<559:24:06,  9.24s/it]
  3%|▎         | 6954/225000 [24:31:35<491:52:39,  8.12s/it]
  3%|▎         | 6955/225000 [24:31:47<559:39:56,  9.24s/it]
  3%|▎         | 6956/225000 [24:31:51<463:37:17,  7.65s/it]
  3%|▎         | 6957/225000 [24:32:03<544:45:59,  8.99s/it]
  3%|▎         | 6958/225000 [24:32:08<460:14:29,  7.60s/it]
  3%|▎         | 6959/225000 [24:32:13<422:51:53,  6.98s/it]
  3%|▎         | 6960/225000 [24:32:19<402:56:54,  6.65s/it]
  3%|▎         | 6961/225000 [24:32:32<510:47:04,  8.43s/it]
  3%|▎         | 6962/225000 [24:32:38<467:54:33,  7.73s/it]
  3%|▎         | 6963/225000 [24:32:43<421:44:53,  6.96s/it]
  3%|▎         | 6964/225000 [24:32:56<528:49:22,  8.73s/it]
  3%|▎         | 6965/225000 [24:33:02<489:58:22,  8.09s/it]
  3%|▎         | 6966/225000 [24:33:14<558:00:07,  9.21s/it]
  3%|▎         | 6967/225000 [24:33:21<523:40:22,  8.65s/it]
  3%|▎         | 6968/225000 [24:33:28<494:50:01,  8.17s/it]
  3%|▎         | 6969/225000 [24:33:40<547:23:35,  9.04s/it]
  3%|▎         | 6970/225000 [24:33:48<542:28:50,  8.96s/it]
  3%|▎         | 6971/225000 [24:34:00<584:29:52,  9.65s/it]
  3%|▎         | 6972/225000 [24:34:07<539:50:54,  8.91s/it]
  3%|▎         | 6973/225000 [24:34:19<602:05:28,  9.94s/it]
  3%|▎         | 6974/225000 [24:34:28<587:44:15,  9.70s/it]
  3%|▎         | 6975/225000 [24:34:38<579:43:45,  9.57s/it]
  3%|▎         | 6976/225000 [24:34:44<520:06:06,  8.59s/it]
  3%|▎         | 6977/225000 [24:34:54<555:10:10,  9.17s/it]
  3%|▎         | 6978/225000 [24:34:59<482:21:24,  7.96s/it]
  3%|▎         | 6979/225000 [24:35:11<548:30:08,  9.06s/it]
  3%|▎         | 6980/225000 [24:35:16<471:39:26,  7.79s/it]
  3%|▎         | 6981/225000 [24:35:21<424:13:26,  7.00s/it]
  3%|▎         | 6982/225000 [24:35:26<390:53:06,  6.45s/it]
  3%|▎         | 6983/225000 [24:35:43<572:26:48,  9.45s/it]
  3%|▎         | 6984/225000 [24:35:50<538:32:19,  8.89s/it]
  3%|▎         | 6985/225000 [24:36:01<578:34:43,  9.55s/it]
  3%|▎         | 6986/225000 [24:36:15<654:43:32, 10.81s/it]
  3%|▎         | 6987/225000 [24:36:31<744:05:48, 12.29s/it]
  3%|▎         | 6988/225000 [24:36:37<626:41:28, 10.35s/it]
  3%|▎         | 6989/225000 [24:36:47<629:15:53, 10.39s/it]
  3%|▎         | 6990/225000 [24:36:55<576:28:52,  9.52s/it]
  3%|▎         | 6991/225000 [24:37:06<603:50:12,  9.97s/it]
  3%|▎         | 6992/225000 [24:37:10<508:44:08,  8.40s/it]
  3%|▎         | 6993/225000 [24:37:19<514:29:54,  8.50s/it]
  3%|▎         | 6994/225000 [24:37:30<558:49:16,  9.23s/it]
  3%|▎         | 6995/225000 [24:37:41<582:09:12,  9.61s/it]
  3%|▎         | 6996/225000 [24:37:55<676:25:30, 11.17s/it]
  3%|▎         | 6997/225000 [24:38:05<652:02:22, 10.77s/it]
  3%|▎         | 6998/225000 [24:38:18<683:53:14, 11.29s/it]
  3%|▎         | 6999/225000 [24:38:22<549:55:52,  9.08s/it]
  3%|▎         | 7000/225000 [24:38:37<665:12:42, 10.99s/it]
                                                            
{'loss': '0.7007', 'grad_norm': '1.27', 'learning_rate': '3.497e-05', 'epoch': '0.03111', 'num_input_tokens_seen': 44904720, 'train_runtime': '8.872e+04', 'train_tokens_per_second': '506.2'}

  3%|▎         | 7000/225000 [24:38:37<665:12:42, 10.99s/it]
  3%|▎         | 7001/225000 [24:38:46<627:59:38, 10.37s/it]
  3%|▎         | 7002/225000 [24:38:50<518:16:07,  8.56s/it]
  3%|▎         | 7003/225000 [24:38:59<516:07:38,  8.52s/it]
  3%|▎         | 7004/225000 [24:39:10<566:53:13,  9.36s/it]
  3%|▎         | 7005/225000 [24:39:18<537:26:30,  8.88s/it]
  3%|▎         | 7006/225000 [24:39:29<575:20:39,  9.50s/it]
  3%|▎         | 7007/225000 [24:39:36<532:38:39,  8.80s/it]
  3%|▎         | 7008/225000 [24:39:44<513:39:30,  8.48s/it]
  3%|▎         | 7009/225000 [24:39:52<504:09:05,  8.33s/it]
  3%|▎         | 7010/225000 [24:40:00<499:54:20,  8.26s/it]
  3%|▎         | 7011/225000 [24:40:09<515:37:56,  8.52s/it]
  3%|▎         | 7012/225000 [24:40:15<473:04:52,  7.81s/it]
  3%|▎         | 7013/225000 [24:40:21<442:56:54,  7.32s/it]
  3%|▎         | 7014/225000 [24:40:34<548:44:27,  9.06s/it]
  3%|▎         | 7015/225000 [24:40:43<542:42:23,  8.96s/it]
  3%|▎         | 7016/225000 [24:40:54<573:24:12,  9.47s/it]
  3%|▎         | 7017/225000 [24:41:06<614:51:48, 10.15s/it]
  3%|▎         | 7018/225000 [24:41:10<503:59:08,  8.32s/it]
  3%|▎         | 7019/225000 [24:41:19<527:45:34,  8.72s/it]
  3%|▎         | 7020/225000 [24:41:27<507:02:36,  8.37s/it]
  3%|▎         | 7021/225000 [24:41:34<493:56:35,  8.16s/it]
  3%|▎         | 7022/225000 [24:41:46<564:41:11,  9.33s/it]
  3%|▎         | 7023/225000 [24:41:58<598:37:30,  9.89s/it]
  3%|▎         | 7024/225000 [24:42:08<610:14:46, 10.08s/it]
  3%|▎         | 7025/225000 [24:42:13<518:34:50,  8.56s/it]
  3%|▎         | 7026/225000 [24:42:22<517:39:12,  8.55s/it]
  3%|▎         | 7027/225000 [24:42:27<453:12:57,  7.49s/it]
  3%|▎         | 7028/225000 [24:42:42<597:04:18,  9.86s/it]
  3%|▎         | 7029/225000 [24:42:53<616:23:55, 10.18s/it]
  3%|▎         | 7030/225000 [24:42:57<511:17:16,  8.44s/it]
  3%|▎         | 7031/225000 [24:43:11<612:20:41, 10.11s/it]
  3%|▎         | 7032/225000 [24:43:18<544:34:00,  8.99s/it]
  3%|▎         | 7033/225000 [24:43:25<507:37:10,  8.38s/it]
  3%|▎         | 7034/225000 [24:43:32<480:39:14,  7.94s/it]
  3%|▎         | 7035/225000 [24:43:42<527:52:58,  8.72s/it]
  3%|▎         | 7036/225000 [24:43:53<570:37:44,  9.42s/it]
  3%|▎         | 7037/225000 [24:43:59<506:26:18,  8.36s/it]
  3%|▎         | 7038/225000 [24:44:05<459:06:41,  7.58s/it]
  3%|▎         | 7039/225000 [24:44:16<524:53:14,  8.67s/it]
  3%|▎         | 7040/225000 [24:44:23<492:35:31,  8.14s/it]
  3%|▎         | 7041/225000 [24:44:28<432:39:22,  7.15s/it]
  3%|▎         | 7042/225000 [24:44:33<399:44:02,  6.60s/it]
  3%|▎         | 7043/225000 [24:44:39<381:48:00,  6.31s/it]
  3%|▎         | 7044/225000 [24:44:51<490:43:55,  8.11s/it]
  3%|▎         | 7045/225000 [24:45:02<536:15:55,  8.86s/it]
  3%|▎         | 7046/225000 [24:45:08<482:24:06,  7.97s/it]
  3%|▎         | 7047/225000 [24:45:19<541:50:35,  8.95s/it]
  3%|▎         | 7048/225000 [24:45:24<472:24:31,  7.80s/it]
  3%|▎         | 7049/225000 [24:45:43<670:03:48, 11.07s/it]
  3%|▎         | 7050/225000 [24:45:48<573:42:06,  9.48s/it]
                                                            
{'loss': '0.6915', 'grad_norm': '1.126', 'learning_rate': '3.522e-05', 'epoch': '0.03133', 'num_input_tokens_seen': 45114104, 'train_runtime': '8.915e+04', 'train_tokens_per_second': '506.1'}

  3%|▎         | 7050/225000 [24:45:48<573:42:06,  9.48s/it]
  3%|▎         | 7051/225000 [24:45:54<503:34:37,  8.32s/it]
  3%|▎         | 7052/225000 [24:46:05<543:55:49,  8.98s/it]
  3%|▎         | 7053/225000 [24:46:14<551:59:36,  9.12s/it]
  3%|▎         | 7054/225000 [24:46:21<512:21:36,  8.46s/it]
  3%|▎         | 7055/225000 [24:46:25<424:03:03,  7.00s/it]
  3%|▎         | 7056/225000 [24:46:37<521:02:40,  8.61s/it]
  3%|▎         | 7057/225000 [24:46:45<512:25:57,  8.46s/it]
  3%|▎         | 7058/225000 [24:46:55<535:28:30,  8.85s/it]
  3%|▎         | 7059/225000 [24:47:03<520:48:21,  8.60s/it]
  3%|▎         | 7060/225000 [24:47:07<438:59:00,  7.25s/it]
  3%|▎         | 7061/225000 [24:47:15<457:39:47,  7.56s/it]
  3%|▎         | 7062/225000 [24:47:26<511:43:11,  8.45s/it]
  3%|▎         | 7063/225000 [24:47:31<447:42:28,  7.40s/it]
  3%|▎         | 7064/225000 [24:47:38<446:37:13,  7.38s/it]
  3%|▎         | 7065/225000 [24:47:47<480:07:01,  7.93s/it]
  3%|▎         | 7066/225000 [24:47:56<486:52:11,  8.04s/it]
  3%|▎         | 7067/225000 [24:48:00<420:06:55,  6.94s/it]
  3%|▎         | 7068/225000 [24:48:07<423:45:15,  7.00s/it]
  3%|▎         | 7069/225000 [24:48:24<609:57:52, 10.08s/it]
  3%|▎         | 7070/225000 [24:48:34<611:01:07, 10.09s/it]
  3%|▎         | 7071/225000 [24:48:44<605:10:37, 10.00s/it]
  3%|▎         | 7072/225000 [24:48:53<575:29:07,  9.51s/it]
  3%|▎         | 7073/225000 [24:49:00<537:54:07,  8.89s/it]
  3%|▎         | 7074/225000 [24:49:07<505:22:49,  8.35s/it]
  3%|▎         | 7075/225000 [24:49:21<602:27:22,  9.95s/it]
  3%|▎         | 7076/225000 [24:49:29<562:07:47,  9.29s/it]
  3%|▎         | 7077/225000 [24:49:37<551:34:54,  9.11s/it]
  3%|▎         | 7078/225000 [24:49:47<555:47:20,  9.18s/it]
  3%|▎         | 7079/225000 [24:49:57<576:27:09,  9.52s/it]
  3%|▎         | 7080/225000 [24:50:07<586:05:08,  9.68s/it]
  3%|▎         | 7081/225000 [24:50:13<523:03:45,  8.64s/it]
  3%|▎         | 7082/225000 [24:50:21<513:47:03,  8.49s/it]
  3%|▎         | 7083/225000 [24:50:26<447:40:30,  7.40s/it]
  3%|▎         | 7084/225000 [24:50:39<553:05:31,  9.14s/it]
  3%|▎         | 7085/225000 [24:50:43<461:16:16,  7.62s/it]
  3%|▎         | 7086/225000 [24:50:53<498:41:36,  8.24s/it]
  3%|▎         | 7087/225000 [24:50:58<434:53:40,  7.18s/it]
  3%|▎         | 7088/225000 [24:51:04<418:31:38,  6.91s/it]
  3%|▎         | 7089/225000 [24:51:10<400:36:11,  6.62s/it]
  3%|▎         | 7090/225000 [24:51:15<377:08:01,  6.23s/it]
  3%|▎         | 7091/225000 [24:51:22<385:23:22,  6.37s/it]
  3%|▎         | 7092/225000 [24:51:31<440:03:53,  7.27s/it]
  3%|▎         | 7093/225000 [24:51:39<439:48:31,  7.27s/it]
  3%|▎         | 7094/225000 [24:51:49<494:25:58,  8.17s/it]
  3%|▎         | 7095/225000 [24:52:03<592:34:42,  9.79s/it]
  3%|▎         | 7096/225000 [24:52:13<603:18:28,  9.97s/it]
  3%|▎         | 7097/225000 [24:52:26<656:10:27, 10.84s/it]
  3%|▎         | 7098/225000 [24:52:37<657:10:36, 10.86s/it]
  3%|▎         | 7099/225000 [24:52:44<588:17:15,  9.72s/it]
  3%|▎         | 7100/225000 [24:52:51<542:31:51,  8.96s/it]
                                                            
{'loss': '0.705', 'grad_norm': '1.299', 'learning_rate': '3.547e-05', 'epoch': '0.03156', 'num_input_tokens_seen': 45323952, 'train_runtime': '8.957e+04', 'train_tokens_per_second': '506'}

  3%|▎         | 7100/225000 [24:52:51<542:31:51,  8.96s/it]
  3%|▎         | 7101/225000 [24:52:56<471:54:01,  7.80s/it]
  3%|▎         | 7102/225000 [24:53:07<527:47:06,  8.72s/it]
  3%|▎         | 7103/225000 [24:53:16<529:59:52,  8.76s/it]
  3%|▎         | 7104/225000 [24:53:21<462:27:34,  7.64s/it]
  3%|▎         | 7105/225000 [24:53:30<489:12:35,  8.08s/it]
  3%|▎         | 7106/225000 [24:53:39<512:51:52,  8.47s/it]
  3%|▎         | 7107/225000 [24:53:47<498:25:29,  8.23s/it]
  3%|▎         | 7108/225000 [24:54:00<588:28:07,  9.72s/it]
  3%|▎         | 7109/225000 [24:54:06<512:56:14,  8.47s/it]
  3%|▎         | 7110/225000 [24:54:11<460:13:47,  7.60s/it]
  3%|▎         | 7111/225000 [24:54:20<473:01:40,  7.82s/it]
  3%|▎         | 7112/225000 [24:54:27<458:48:02,  7.58s/it]
  3%|▎         | 7113/225000 [24:54:31<397:45:30,  6.57s/it]
  3%|▎         | 7114/225000 [24:54:40<447:23:45,  7.39s/it]
  3%|▎         | 7115/225000 [24:54:45<397:19:48,  6.56s/it]
  3%|▎         | 7116/225000 [24:55:04<629:02:18, 10.39s/it]
  3%|▎         | 7117/225000 [24:55:11<566:43:40,  9.36s/it]
  3%|▎         | 7118/225000 [24:55:18<522:56:41,  8.64s/it]
  3%|▎         | 7119/225000 [24:55:23<461:33:10,  7.63s/it]
  3%|▎         | 7120/225000 [24:55:34<510:05:28,  8.43s/it]
  3%|▎         | 7121/225000 [24:55:42<517:16:57,  8.55s/it]
  3%|▎         | 7122/225000 [24:55:47<448:35:10,  7.41s/it]
  3%|▎         | 7123/225000 [24:55:55<450:48:33,  7.45s/it]
  3%|▎         | 7124/225000 [24:56:08<553:46:03,  9.15s/it]
  3%|▎         | 7125/225000 [24:56:16<544:38:24,  9.00s/it]
  3%|▎         | 7126/225000 [24:56:22<480:55:28,  7.95s/it]
  3%|▎         | 7127/225000 [24:56:39<644:04:34, 10.64s/it]
  3%|▎         | 7128/225000 [24:56:55<735:19:59, 12.15s/it]
  3%|▎         | 7129/225000 [24:56:59<601:51:31,  9.94s/it]
  3%|▎         | 7130/225000 [24:57:09<604:19:55,  9.99s/it]
  3%|▎         | 7131/225000 [24:57:17<559:57:12,  9.25s/it]
  3%|▎         | 7132/225000 [24:57:22<486:16:52,  8.04s/it]
  3%|▎         | 7133/225000 [24:57:29<461:55:30,  7.63s/it]
  3%|▎         | 7134/225000 [24:57:40<522:41:14,  8.64s/it]
  3%|▎         | 7135/225000 [24:57:50<557:56:36,  9.22s/it]
  3%|▎         | 7136/225000 [24:58:02<595:58:15,  9.85s/it]
  3%|▎         | 7137/225000 [24:58:07<513:14:23,  8.48s/it]
  3%|▎         | 7138/225000 [24:58:14<488:18:22,  8.07s/it]
  3%|▎         | 7139/225000 [24:58:23<496:14:19,  8.20s/it]
  3%|▎         | 7140/225000 [24:58:31<507:20:16,  8.38s/it]
  3%|▎         | 7141/225000 [24:58:36<444:15:44,  7.34s/it]
  3%|▎         | 7142/225000 [24:58:46<485:57:22,  8.03s/it]
  3%|▎         | 7143/225000 [24:58:51<433:41:11,  7.17s/it]
  3%|▎         | 7144/225000 [24:59:00<471:44:02,  7.80s/it]
  3%|▎         | 7145/225000 [24:59:16<611:52:45, 10.11s/it]
  3%|▎         | 7146/225000 [24:59:21<525:44:11,  8.69s/it]
  3%|▎         | 7147/225000 [24:59:40<710:49:29, 11.75s/it]
  3%|▎         | 7148/225000 [24:59:53<734:52:50, 12.14s/it]
  3%|▎         | 7149/225000 [24:59:58<605:01:39, 10.00s/it]
  3%|▎         | 7150/225000 [25:00:08<595:46:57,  9.85s/it]
                                                            
{'loss': '0.701', 'grad_norm': '1.304', 'learning_rate': '3.572e-05', 'epoch': '0.03178', 'num_input_tokens_seen': 45537904, 'train_runtime': '9.001e+04', 'train_tokens_per_second': '505.9'}

  3%|▎         | 7150/225000 [25:00:08<595:46:57,  9.85s/it]
  3%|▎         | 7151/225000 [25:00:13<503:15:23,  8.32s/it]
  3%|▎         | 7152/225000 [25:00:25<582:27:44,  9.63s/it]
  3%|▎         | 7153/225000 [25:00:30<495:25:37,  8.19s/it]
  3%|▎         | 7154/225000 [25:00:42<556:38:08,  9.20s/it]
  3%|▎         | 7155/225000 [25:00:46<474:54:59,  7.85s/it]
  3%|▎         | 7156/225000 [25:00:55<492:29:47,  8.14s/it]
  3%|▎         | 7157/225000 [25:01:02<471:37:35,  7.79s/it]
  3%|▎         | 7158/225000 [25:01:07<415:25:46,  6.87s/it]
  3%|▎         | 7159/225000 [25:01:12<387:26:04,  6.40s/it]
  3%|▎         | 7160/225000 [25:01:19<392:03:22,  6.48s/it]
  3%|▎         | 7161/225000 [25:01:33<536:25:53,  8.87s/it]
  3%|▎         | 7162/225000 [25:01:37<438:28:33,  7.25s/it]
  3%|▎         | 7163/225000 [25:01:47<499:49:04,  8.26s/it]
  3%|▎         | 7164/225000 [25:01:59<568:23:24,  9.39s/it]
  3%|▎         | 7165/225000 [25:02:04<480:10:42,  7.94s/it]
  3%|▎         | 7166/225000 [25:02:16<552:56:22,  9.14s/it]
  3%|▎         | 7167/225000 [25:02:22<497:16:04,  8.22s/it]
  3%|▎         | 7168/225000 [25:02:27<440:56:06,  7.29s/it]
  3%|▎         | 7169/225000 [25:02:35<451:34:31,  7.46s/it]
  3%|▎         | 7170/225000 [25:02:48<562:55:22,  9.30s/it]
  3%|▎         | 7171/225000 [25:03:00<605:45:55, 10.01s/it]
  3%|▎         | 7172/225000 [25:03:04<492:06:58,  8.13s/it]
  3%|▎         | 7173/225000 [25:03:11<472:02:03,  7.80s/it]
  3%|▎         | 7174/225000 [25:03:16<419:55:05,  6.94s/it]
  3%|▎         | 7175/225000 [25:03:26<471:47:33,  7.80s/it]
  3%|▎         | 7176/225000 [25:03:33<462:44:29,  7.65s/it]
  3%|▎         | 7177/225000 [25:03:45<544:27:39,  9.00s/it]
  3%|▎         | 7178/225000 [25:03:52<501:53:42,  8.29s/it]
  3%|▎         | 7179/225000 [25:03:58<461:48:16,  7.63s/it]
  3%|▎         | 7180/225000 [25:04:05<461:29:10,  7.63s/it]
  3%|▎         | 7181/225000 [25:04:19<564:29:24,  9.33s/it]
  3%|▎         | 7182/225000 [25:04:23<465:57:48,  7.70s/it]
  3%|▎         | 7183/225000 [25:04:35<545:36:40,  9.02s/it]
  3%|▎         | 7184/225000 [25:04:42<518:16:21,  8.57s/it]
  3%|▎         | 7185/225000 [25:04:48<467:56:51,  7.73s/it]
  3%|▎         | 7186/225000 [25:04:54<439:00:24,  7.26s/it]
  3%|▎         | 7187/225000 [25:05:14<665:00:51, 10.99s/it]
  3%|▎         | 7188/225000 [25:05:23<630:19:29, 10.42s/it]
  3%|▎         | 7189/225000 [25:05:28<532:50:04,  8.81s/it]
  3%|▎         | 7190/225000 [25:05:34<479:20:58,  7.92s/it]
  3%|▎         | 7191/225000 [25:05:41<461:34:04,  7.63s/it]
  3%|▎         | 7192/225000 [25:05:50<483:25:03,  7.99s/it]
  3%|▎         | 7193/225000 [25:05:57<472:55:26,  7.82s/it]
  3%|▎         | 7194/225000 [25:06:08<522:59:06,  8.64s/it]
  3%|▎         | 7195/225000 [25:06:21<604:34:44,  9.99s/it]
  3%|▎         | 7196/225000 [25:06:25<506:24:01,  8.37s/it]
  3%|▎         | 7197/225000 [25:06:41<635:40:36, 10.51s/it]
  3%|▎         | 7198/225000 [25:06:49<583:59:57,  9.65s/it]
  3%|▎         | 7199/225000 [25:06:57<562:22:58,  9.30s/it]
  3%|▎         | 7200/225000 [25:07:07<572:15:29,  9.46s/it]
                                                            
{'loss': '0.7176', 'grad_norm': '1.061', 'learning_rate': '3.597e-05', 'epoch': '0.032', 'num_input_tokens_seen': 45743728, 'train_runtime': '9.043e+04', 'train_tokens_per_second': '505.9'}

  3%|▎         | 7200/225000 [25:07:07<572:15:29,  9.46s/it]
  3%|▎         | 7201/225000 [25:07:13<511:02:01,  8.45s/it]
  3%|▎         | 7202/225000 [25:07:18<442:50:39,  7.32s/it]
  3%|▎         | 7203/225000 [25:07:22<396:40:10,  6.56s/it]
  3%|▎         | 7204/225000 [25:07:32<444:14:15,  7.34s/it]
  3%|▎         | 7205/225000 [25:07:43<512:07:33,  8.47s/it]
  3%|▎         | 7206/225000 [25:07:56<593:03:48,  9.80s/it]
  3%|▎         | 7207/225000 [25:08:04<570:44:36,  9.43s/it]
  3%|▎         | 7208/225000 [25:08:14<585:04:04,  9.67s/it]
  3%|▎         | 7209/225000 [25:08:26<613:13:10, 10.14s/it]
  3%|▎         | 7210/225000 [25:08:29<496:25:40,  8.21s/it]
  3%|▎         | 7211/225000 [25:08:39<515:21:51,  8.52s/it]
  3%|▎         | 7212/225000 [25:08:46<504:05:29,  8.33s/it]
  3%|▎         | 7213/225000 [25:08:54<487:54:02,  8.06s/it]
  3%|▎         | 7214/225000 [25:09:04<521:47:29,  8.63s/it]
  3%|▎         | 7215/225000 [25:09:16<577:50:40,  9.55s/it]
  3%|▎         | 7216/225000 [25:09:25<584:25:59,  9.66s/it]
  3%|▎         | 7217/225000 [25:09:29<477:14:19,  7.89s/it]
  3%|▎         | 7218/225000 [25:09:34<415:16:26,  6.86s/it]
  3%|▎         | 7219/225000 [25:09:39<392:17:38,  6.48s/it]
  3%|▎         | 7220/225000 [25:09:51<479:47:32,  7.93s/it]
  3%|▎         | 7221/225000 [25:09:57<450:04:46,  7.44s/it]
  3%|▎         | 7222/225000 [25:10:04<443:20:44,  7.33s/it]
  3%|▎         | 7223/225000 [25:10:11<437:23:53,  7.23s/it]
  3%|▎         | 7224/225000 [25:10:16<391:53:34,  6.48s/it]
  3%|▎         | 7225/225000 [25:10:31<551:55:08,  9.12s/it]
  3%|▎         | 7226/225000 [25:10:36<477:29:57,  7.89s/it]
  3%|▎         | 7227/225000 [25:10:45<502:48:02,  8.31s/it]
  3%|▎         | 7228/225000 [25:10:54<511:16:59,  8.45s/it]
  3%|▎         | 7229/225000 [25:10:59<449:25:19,  7.43s/it]
  3%|▎         | 7230/225000 [25:11:09<494:04:37,  8.17s/it]
  3%|▎         | 7231/225000 [25:11:15<454:08:00,  7.51s/it]
  3%|▎         | 7232/225000 [25:11:23<470:06:42,  7.77s/it]
  3%|▎         | 7233/225000 [25:11:33<497:45:58,  8.23s/it]
  3%|▎         | 7234/225000 [25:11:45<574:36:35,  9.50s/it]
  3%|▎         | 7235/225000 [25:11:53<550:26:02,  9.10s/it]
  3%|▎         | 7236/225000 [25:12:00<501:04:24,  8.28s/it]
  3%|▎         | 7237/225000 [25:12:16<652:44:09, 10.79s/it]
  3%|▎         | 7238/225000 [25:12:22<567:41:57,  9.39s/it]
  3%|▎         | 7239/225000 [25:12:30<527:28:42,  8.72s/it]
  3%|▎         | 7240/225000 [25:12:40<555:43:03,  9.19s/it]
  3%|▎         | 7241/225000 [25:12:47<511:00:35,  8.45s/it]
  3%|▎         | 7242/225000 [25:12:59<591:23:59,  9.78s/it]
  3%|▎         | 7243/225000 [25:13:05<523:56:26,  8.66s/it]
  3%|▎         | 7244/225000 [25:13:20<630:44:07, 10.43s/it]
  3%|▎         | 7245/225000 [25:13:31<648:44:17, 10.73s/it]
  3%|▎         | 7246/225000 [25:13:42<641:47:46, 10.61s/it]
  3%|▎         | 7247/225000 [25:13:51<619:32:27, 10.24s/it]
  3%|▎         | 7248/225000 [25:13:55<506:41:51,  8.38s/it]
  3%|▎         | 7249/225000 [25:14:04<513:41:15,  8.49s/it]
  3%|▎         | 7250/225000 [25:14:09<448:59:15,  7.42s/it]
                                                            
{'loss': '0.7087', 'grad_norm': '0.9161', 'learning_rate': '3.622e-05', 'epoch': '0.03222', 'num_input_tokens_seen': 45947984, 'train_runtime': '9.085e+04', 'train_tokens_per_second': '505.8'}

  3%|▎         | 7250/225000 [25:14:09<448:59:15,  7.42s/it]
  3%|▎         | 7251/225000 [25:14:21<540:15:16,  8.93s/it]
  3%|▎         | 7252/225000 [25:14:29<520:23:58,  8.60s/it]
  3%|▎         | 7253/225000 [25:14:41<585:51:38,  9.69s/it]
  3%|▎         | 7254/225000 [25:14:50<570:39:30,  9.43s/it]
  3%|▎         | 7255/225000 [25:14:58<543:59:45,  8.99s/it]
  3%|▎         | 7256/225000 [25:15:07<549:28:26,  9.08s/it]
  3%|▎         | 7257/225000 [25:15:19<591:56:20,  9.79s/it]
  3%|▎         | 7258/225000 [25:15:29<592:16:05,  9.79s/it]
  3%|▎         | 7259/225000 [25:15:35<535:47:06,  8.86s/it]
  3%|▎         | 7260/225000 [25:15:42<495:58:07,  8.20s/it]
  3%|▎         | 7261/225000 [25:15:52<522:58:58,  8.65s/it]
  3%|▎         | 7262/225000 [25:15:57<464:26:52,  7.68s/it]
  3%|▎         | 7263/225000 [25:16:13<615:38:17, 10.18s/it]
  3%|▎         | 7264/225000 [25:16:18<517:01:30,  8.55s/it]
  3%|▎         | 7265/225000 [25:16:26<514:19:39,  8.50s/it]
  3%|▎         | 7266/225000 [25:16:35<510:58:37,  8.45s/it]
  3%|▎         | 7267/225000 [25:16:40<448:22:00,  7.41s/it]
  3%|▎         | 7268/225000 [25:16:48<457:20:44,  7.56s/it]
  3%|▎         | 7269/225000 [25:17:02<581:06:43,  9.61s/it]
  3%|▎         | 7270/225000 [25:17:15<645:40:15, 10.68s/it]
  3%|▎         | 7271/225000 [25:17:22<574:33:10,  9.50s/it]
  3%|▎         | 7272/225000 [25:17:26<468:57:48,  7.75s/it]
  3%|▎         | 7273/225000 [25:17:34<490:41:37,  8.11s/it]
  3%|▎         | 7274/225000 [25:17:41<461:33:41,  7.63s/it]
  3%|▎         | 7275/225000 [25:17:47<432:42:11,  7.15s/it]
  3%|▎         | 7276/225000 [25:17:54<431:15:32,  7.13s/it]
  3%|▎         | 7277/225000 [25:18:02<453:45:10,  7.50s/it]
  3%|▎         | 7278/225000 [25:18:09<442:37:23,  7.32s/it]
  3%|▎         | 7279/225000 [25:18:23<549:21:58,  9.08s/it]
  3%|▎         | 7280/225000 [25:18:31<529:13:19,  8.75s/it]
  3%|▎         | 7281/225000 [25:18:38<507:33:45,  8.39s/it]
  3%|▎         | 7282/225000 [25:18:43<446:31:29,  7.38s/it]
  3%|▎         | 7283/225000 [25:18:55<524:37:10,  8.67s/it]
  3%|▎         | 7284/225000 [25:18:59<447:06:39,  7.39s/it]
  3%|▎         | 7285/225000 [25:19:05<410:18:05,  6.78s/it]
  3%|▎         | 7286/225000 [25:19:15<481:17:34,  7.96s/it]
  3%|▎         | 7287/225000 [25:19:28<570:01:03,  9.43s/it]
  3%|▎         | 7288/225000 [25:19:37<562:09:25,  9.30s/it]
  3%|▎         | 7289/225000 [25:19:43<492:23:04,  8.14s/it]
  3%|▎         | 7290/225000 [25:19:52<508:44:04,  8.41s/it]
  3%|▎         | 7291/225000 [25:20:02<544:58:47,  9.01s/it]
  3%|▎         | 7292/225000 [25:20:06<451:34:35,  7.47s/it]
  3%|▎         | 7293/225000 [25:20:14<471:22:40,  7.79s/it]
  3%|▎         | 7294/225000 [25:20:31<628:25:20, 10.39s/it]
  3%|▎         | 7295/225000 [25:20:36<537:18:25,  8.88s/it]
  3%|▎         | 7296/225000 [25:20:43<495:14:08,  8.19s/it]
  3%|▎         | 7297/225000 [25:20:56<587:07:41,  9.71s/it]
  3%|▎         | 7298/225000 [25:21:02<524:13:11,  8.67s/it]
  3%|▎         | 7299/225000 [25:21:11<524:46:32,  8.68s/it]
  3%|▎         | 7300/225000 [25:21:16<461:47:14,  7.64s/it]
                                                            
{'loss': '0.7207', 'grad_norm': '1.007', 'learning_rate': '3.647e-05', 'epoch': '0.03244', 'num_input_tokens_seen': 46153904, 'train_runtime': '9.128e+04', 'train_tokens_per_second': '505.6'}

  3%|▎         | 7300/225000 [25:21:16<461:47:14,  7.64s/it]
  3%|▎         | 7301/225000 [25:21:23<447:31:25,  7.40s/it]
  3%|▎         | 7302/225000 [25:21:41<628:57:48, 10.40s/it]
  3%|▎         | 7303/225000 [25:21:49<595:02:49,  9.84s/it]
  3%|▎         | 7304/225000 [25:21:57<559:09:20,  9.25s/it]
  3%|▎         | 7305/225000 [25:22:06<564:51:18,  9.34s/it]
  3%|▎         | 7306/225000 [25:22:11<480:17:58,  7.94s/it]
  3%|▎         | 7307/225000 [25:22:21<522:39:27,  8.64s/it]
  3%|▎         | 7308/225000 [25:22:28<483:44:05,  8.00s/it]
  3%|▎         | 7309/225000 [25:22:37<511:54:02,  8.47s/it]
  3%|▎         | 7310/225000 [25:22:45<488:24:40,  8.08s/it]
  3%|▎         | 7311/225000 [25:22:52<471:17:04,  7.79s/it]
  3%|▎         | 7312/225000 [25:22:59<468:37:47,  7.75s/it]
  3%|▎         | 7313/225000 [25:23:07<457:08:12,  7.56s/it]
  3%|▎         | 7314/225000 [25:23:11<398:22:24,  6.59s/it]
  3%|▎         | 7315/225000 [25:23:16<365:46:07,  6.05s/it]
  3%|▎         | 7316/225000 [25:23:26<435:16:59,  7.20s/it]
  3%|▎         | 7317/225000 [25:23:35<473:10:00,  7.83s/it]
  3%|▎         | 7318/225000 [25:23:39<414:56:32,  6.86s/it]
  3%|▎         | 7319/225000 [25:23:51<507:15:00,  8.39s/it]
  3%|▎         | 7320/225000 [25:23:56<431:04:35,  7.13s/it]
  3%|▎         | 7321/225000 [25:24:06<491:11:13,  8.12s/it]
  3%|▎         | 7322/225000 [25:24:12<450:17:17,  7.45s/it]
  3%|▎         | 7323/225000 [25:24:18<434:03:24,  7.18s/it]
  3%|▎         | 7324/225000 [25:24:26<434:37:45,  7.19s/it]
  3%|▎         | 7325/225000 [25:24:34<450:53:49,  7.46s/it]
  3%|▎         | 7326/225000 [25:24:47<556:03:31,  9.20s/it]
  3%|▎         | 7327/225000 [25:24:53<491:02:39,  8.12s/it]
  3%|▎         | 7328/225000 [25:24:59<456:43:27,  7.55s/it]
  3%|▎         | 7329/225000 [25:25:13<571:24:16,  9.45s/it]
  3%|▎         | 7330/225000 [25:25:19<508:10:47,  8.40s/it]
  3%|▎         | 7331/225000 [25:25:27<504:33:21,  8.34s/it]
  3%|▎         | 7332/225000 [25:25:33<469:56:37,  7.77s/it]
  3%|▎         | 7333/225000 [25:25:42<488:53:19,  8.09s/it]
  3%|▎         | 7334/225000 [25:25:53<545:51:01,  9.03s/it]
  3%|▎         | 7335/225000 [25:25:57<449:39:44,  7.44s/it]
  3%|▎         | 7336/225000 [25:26:02<412:16:45,  6.82s/it]
  3%|▎         | 7337/225000 [25:26:17<558:11:11,  9.23s/it]
  3%|▎         | 7338/225000 [25:26:26<551:45:22,  9.13s/it]
  3%|▎         | 7339/225000 [25:26:31<478:51:35,  7.92s/it]
  3%|▎         | 7340/225000 [25:26:36<417:25:47,  6.90s/it]
  3%|▎         | 7341/225000 [25:26:46<484:53:04,  8.02s/it]
  3%|▎         | 7342/225000 [25:26:52<433:45:05,  7.17s/it]
  3%|▎         | 7343/225000 [25:27:04<529:04:37,  8.75s/it]
  3%|▎         | 7344/225000 [25:27:13<524:31:33,  8.68s/it]
  3%|▎         | 7345/225000 [25:27:21<512:30:00,  8.48s/it]
  3%|▎         | 7346/225000 [25:27:26<459:52:31,  7.61s/it]
  3%|▎         | 7347/225000 [25:27:44<644:03:44, 10.65s/it]
  3%|▎         | 7348/225000 [25:27:49<540:14:53,  8.94s/it]
  3%|▎         | 7349/225000 [25:27:53<459:57:48,  7.61s/it]
  3%|▎         | 7350/225000 [25:28:00<437:13:42,  7.23s/it]
                                                            
{'loss': '0.7335', 'grad_norm': '1.184', 'learning_rate': '3.672e-05', 'epoch': '0.03267', 'num_input_tokens_seen': 46351672, 'train_runtime': '9.168e+04', 'train_tokens_per_second': '505.6'}

  3%|▎         | 7350/225000 [25:28:00<437:13:42,  7.23s/it]
  3%|▎         | 7351/225000 [25:28:10<491:43:42,  8.13s/it]
  3%|▎         | 7352/225000 [25:28:14<419:54:30,  6.95s/it]
  3%|▎         | 7353/225000 [25:28:22<432:09:31,  7.15s/it]
  3%|▎         | 7354/225000 [25:28:32<490:45:54,  8.12s/it]
  3%|▎         | 7355/225000 [25:28:37<431:12:03,  7.13s/it]
  3%|▎         | 7356/225000 [25:28:42<388:13:44,  6.42s/it]
  3%|▎         | 7357/225000 [25:28:49<402:17:31,  6.65s/it]
  3%|▎         | 7358/225000 [25:29:00<473:21:57,  7.83s/it]
  3%|▎         | 7359/225000 [25:29:04<411:26:02,  6.81s/it]
  3%|▎         | 7360/225000 [25:29:16<498:01:59,  8.24s/it]
  3%|▎         | 7361/225000 [25:29:24<504:50:38,  8.35s/it]
  3%|▎         | 7362/225000 [25:29:38<611:51:01, 10.12s/it]
  3%|▎         | 7363/225000 [25:29:43<504:52:15,  8.35s/it]
  3%|▎         | 7364/225000 [25:29:53<533:14:44,  8.82s/it]
  3%|▎         | 7365/225000 [25:29:57<458:33:32,  7.59s/it]
  3%|▎         | 7366/225000 [25:30:03<435:02:28,  7.20s/it]
  3%|▎         | 7367/225000 [25:30:08<385:59:30,  6.38s/it]
  3%|▎         | 7368/225000 [25:30:13<364:56:56,  6.04s/it]
  3%|▎         | 7369/225000 [25:30:18<349:26:37,  5.78s/it]
  3%|▎         | 7370/225000 [25:30:32<482:20:34,  7.98s/it]
  3%|▎         | 7371/225000 [25:30:43<537:14:58,  8.89s/it]
  3%|▎         | 7372/225000 [25:30:47<461:35:16,  7.64s/it]
  3%|▎         | 7373/225000 [25:31:01<568:26:05,  9.40s/it]
  3%|▎         | 7374/225000 [25:31:05<479:33:42,  7.93s/it]
  3%|▎         | 7375/225000 [25:31:16<535:57:11,  8.87s/it]
  3%|▎         | 7376/225000 [25:31:23<494:40:52,  8.18s/it]
  3%|▎         | 7377/225000 [25:31:32<507:37:05,  8.40s/it]
  3%|▎         | 7378/225000 [25:31:38<476:01:34,  7.87s/it]
  3%|▎         | 7379/225000 [25:31:51<554:11:44,  9.17s/it]
  3%|▎         | 7380/225000 [25:32:02<601:38:46,  9.95s/it]
  3%|▎         | 7381/225000 [25:32:09<535:52:43,  8.86s/it]
  3%|▎         | 7382/225000 [25:32:15<480:59:48,  7.96s/it]
  3%|▎         | 7383/225000 [25:32:20<436:39:17,  7.22s/it]
  3%|▎         | 7384/225000 [25:32:25<401:53:04,  6.65s/it]
  3%|▎         | 7385/225000 [25:32:42<580:52:13,  9.61s/it]
  3%|▎         | 7386/225000 [25:32:46<483:12:59,  7.99s/it]
  3%|▎         | 7387/225000 [25:32:56<515:18:31,  8.52s/it]
  3%|▎         | 7388/225000 [25:33:01<453:05:05,  7.50s/it]
  3%|▎         | 7389/225000 [25:33:11<498:46:57,  8.25s/it]
  3%|▎         | 7390/225000 [25:33:16<430:56:33,  7.13s/it]
  3%|▎         | 7391/225000 [25:33:23<445:12:28,  7.37s/it]
  3%|▎         | 7392/225000 [25:33:31<448:12:45,  7.42s/it]
  3%|▎         | 7393/225000 [25:33:37<417:06:19,  6.90s/it]
  3%|▎         | 7394/225000 [25:33:46<455:08:21,  7.53s/it]
  3%|▎         | 7395/225000 [25:33:57<525:01:35,  8.69s/it]
  3%|▎         | 7396/225000 [25:34:02<463:25:17,  7.67s/it]
  3%|▎         | 7397/225000 [25:34:08<421:05:31,  6.97s/it]
  3%|▎         | 7398/225000 [25:34:19<503:49:51,  8.34s/it]
  3%|▎         | 7399/225000 [25:34:32<591:04:50,  9.78s/it]
  3%|▎         | 7400/225000 [25:34:39<528:37:09,  8.75s/it]
                                                            
{'loss': '0.7176', 'grad_norm': '1.176', 'learning_rate': '3.697e-05', 'epoch': '0.03289', 'num_input_tokens_seen': 46543552, 'train_runtime': '9.208e+04', 'train_tokens_per_second': '505.5'}

  3%|▎         | 7400/225000 [25:34:39<528:37:09,  8.75s/it]
  3%|▎         | 7401/225000 [25:34:48<533:43:26,  8.83s/it]
  3%|▎         | 7402/225000 [25:34:52<448:19:01,  7.42s/it]
  3%|▎         | 7403/225000 [25:34:57<407:07:19,  6.74s/it]
  3%|▎         | 7404/225000 [25:35:07<460:06:22,  7.61s/it]
  3%|▎         | 7405/225000 [25:35:12<422:30:34,  6.99s/it]
  3%|▎         | 7406/225000 [25:35:20<435:56:36,  7.21s/it]
  3%|▎         | 7407/225000 [25:35:32<525:29:47,  8.69s/it]
  3%|▎         | 7408/225000 [25:35:36<443:30:34,  7.34s/it]
  3%|▎         | 7409/225000 [25:35:46<487:34:47,  8.07s/it]
  3%|▎         | 7410/225000 [25:35:51<423:16:56,  7.00s/it]
  3%|▎         | 7411/225000 [25:35:58<432:52:24,  7.16s/it]
  3%|▎         | 7412/225000 [25:36:09<506:00:22,  8.37s/it]
  3%|▎         | 7413/225000 [25:36:17<486:17:58,  8.05s/it]
  3%|▎         | 7414/225000 [25:36:28<547:06:47,  9.05s/it]
  3%|▎         | 7415/225000 [25:36:33<466:00:29,  7.71s/it]
  3%|▎         | 7416/225000 [25:36:41<474:31:28,  7.85s/it]
  3%|▎         | 7417/225000 [25:36:50<505:42:18,  8.37s/it]
  3%|▎         | 7418/225000 [25:36:59<514:03:16,  8.51s/it]
  3%|▎         | 7419/225000 [25:37:06<484:19:37,  8.01s/it]
  3%|▎         | 7420/225000 [25:37:09<403:01:02,  6.67s/it]
  3%|▎         | 7421/225000 [25:37:15<390:08:58,  6.46s/it]
  3%|▎         | 7422/225000 [25:37:29<519:33:10,  8.60s/it]
  3%|▎         | 7423/225000 [25:37:37<499:42:05,  8.27s/it]
  3%|▎         | 7424/225000 [25:37:42<445:24:34,  7.37s/it]
  3%|▎         | 7425/225000 [25:37:50<459:18:15,  7.60s/it]
  3%|▎         | 7426/225000 [25:38:01<526:38:43,  8.71s/it]
  3%|▎         | 7427/225000 [25:38:11<546:40:22,  9.05s/it]
  3%|▎         | 7428/225000 [25:38:18<512:23:32,  8.48s/it]
  3%|▎         | 7429/225000 [25:38:29<555:01:25,  9.18s/it]
  3%|▎         | 7430/225000 [25:38:36<512:05:53,  8.47s/it]
  3%|▎         | 7431/225000 [25:38:45<528:51:07,  8.75s/it]
  3%|▎         | 7432/225000 [25:38:51<471:11:13,  7.80s/it]
  3%|▎         | 7433/225000 [25:39:01<510:21:00,  8.44s/it]
  3%|▎         | 7434/225000 [25:39:06<455:46:48,  7.54s/it]
  3%|▎         | 7435/225000 [25:39:16<489:21:16,  8.10s/it]
  3%|▎         | 7436/225000 [25:39:21<437:42:54,  7.24s/it]
  3%|▎         | 7437/225000 [25:39:25<387:45:44,  6.42s/it]
  3%|▎         | 7438/225000 [25:39:34<435:38:15,  7.21s/it]
  3%|▎         | 7439/225000 [25:39:44<474:56:05,  7.86s/it]
  3%|▎         | 7440/225000 [25:39:51<460:59:15,  7.63s/it]
  3%|▎         | 7441/225000 [25:40:03<550:32:38,  9.11s/it]
  3%|▎         | 7442/225000 [25:40:08<475:39:20,  7.87s/it]
  3%|▎         | 7443/225000 [25:40:18<513:47:19,  8.50s/it]
  3%|▎         | 7444/225000 [25:40:22<431:29:18,  7.14s/it]
  3%|▎         | 7445/225000 [25:40:34<514:06:53,  8.51s/it]
  3%|▎         | 7446/225000 [25:40:41<477:03:17,  7.89s/it]
  3%|▎         | 7447/225000 [25:40:47<445:05:50,  7.37s/it]
  3%|▎         | 7448/225000 [25:40:52<400:29:46,  6.63s/it]
  3%|▎         | 7449/225000 [25:41:04<513:18:06,  8.49s/it]
  3%|▎         | 7450/225000 [25:41:12<488:33:03,  8.08s/it]
                                                            
{'loss': '0.7037', 'grad_norm': '1.018', 'learning_rate': '3.722e-05', 'epoch': '0.03311', 'num_input_tokens_seen': 46738296, 'train_runtime': '9.247e+04', 'train_tokens_per_second': '505.4'}

  3%|▎         | 7450/225000 [25:41:12<488:33:03,  8.08s/it]
  3%|▎         | 7451/225000 [25:41:22<523:18:18,  8.66s/it]
  3%|▎         | 7452/225000 [25:41:33<570:09:43,  9.44s/it]
  3%|▎         | 7453/225000 [25:41:41<542:27:01,  8.98s/it]
  3%|▎         | 7454/225000 [25:41:47<498:21:07,  8.25s/it]
  3%|▎         | 7455/225000 [25:41:54<467:39:54,  7.74s/it]
  3%|▎         | 7456/225000 [25:42:00<431:36:08,  7.14s/it]
  3%|▎         | 7457/225000 [25:42:13<537:00:38,  8.89s/it]
  3%|▎         | 7458/225000 [25:42:22<549:56:00,  9.10s/it]
  3%|▎         | 7459/225000 [25:42:27<476:43:49,  7.89s/it]
  3%|▎         | 7460/225000 [25:42:33<433:08:34,  7.17s/it]
  3%|▎         | 7461/225000 [25:42:44<516:47:28,  8.55s/it]
  3%|▎         | 7462/225000 [25:42:54<541:51:58,  8.97s/it]
  3%|▎         | 7463/225000 [25:43:01<506:28:09,  8.38s/it]
  3%|▎         | 7464/225000 [25:43:12<546:35:44,  9.05s/it]
  3%|▎         | 7465/225000 [25:43:18<482:58:53,  7.99s/it]
  3%|▎         | 7466/225000 [25:43:23<440:58:35,  7.30s/it]
  3%|▎         | 7467/225000 [25:43:32<477:08:32,  7.90s/it]
  3%|▎         | 7468/225000 [25:43:40<478:39:23,  7.92s/it]
  3%|▎         | 7469/225000 [25:43:51<527:23:37,  8.73s/it]
  3%|▎         | 7470/225000 [25:43:57<485:27:32,  8.03s/it]
  3%|▎         | 7471/225000 [25:44:03<440:43:05,  7.29s/it]
  3%|▎         | 7472/225000 [25:44:14<511:36:30,  8.47s/it]
  3%|▎         | 7473/225000 [25:44:20<463:55:38,  7.68s/it]
  3%|▎         | 7474/225000 [25:44:30<509:00:04,  8.42s/it]
  3%|▎         | 7475/225000 [25:44:35<448:42:36,  7.43s/it]
  3%|▎         | 7476/225000 [25:44:49<563:21:54,  9.32s/it]
  3%|▎         | 7477/225000 [25:44:57<537:53:03,  8.90s/it]
  3%|▎         | 7478/225000 [25:45:01<444:07:45,  7.35s/it]
  3%|▎         | 7479/225000 [25:45:13<533:20:44,  8.83s/it]
  3%|▎         | 7480/225000 [25:45:18<455:50:54,  7.54s/it]
  3%|▎         | 7481/225000 [25:45:22<390:22:33,  6.46s/it]
  3%|▎         | 7482/225000 [25:45:30<418:16:44,  6.92s/it]
  3%|▎         | 7483/225000 [25:45:43<534:13:30,  8.84s/it]
  3%|▎         | 7484/225000 [25:45:55<597:33:15,  9.89s/it]
  3%|▎         | 7485/225000 [25:46:00<497:28:16,  8.23s/it]
  3%|▎         | 7486/225000 [25:46:10<528:56:38,  8.75s/it]
  3%|▎         | 7487/225000 [25:46:16<494:59:29,  8.19s/it]
  3%|▎         | 7488/225000 [25:46:25<498:36:01,  8.25s/it]
  3%|▎         | 7489/225000 [25:46:34<517:38:32,  8.57s/it]
  3%|▎         | 7490/225000 [25:46:43<524:23:25,  8.68s/it]
  3%|▎         | 7491/225000 [25:46:54<574:26:01,  9.51s/it]
  3%|▎         | 7492/225000 [25:47:03<555:33:44,  9.20s/it]
  3%|▎         | 7493/225000 [25:47:09<502:22:11,  8.31s/it]
  3%|▎         | 7494/225000 [25:47:15<449:33:59,  7.44s/it]
  3%|▎         | 7495/225000 [25:47:28<564:31:18,  9.34s/it]
  3%|▎         | 7496/225000 [25:47:34<505:58:13,  8.37s/it]
  3%|▎         | 7497/225000 [25:47:39<444:27:43,  7.36s/it]
  3%|▎         | 7498/225000 [25:47:46<433:16:33,  7.17s/it]
  3%|▎         | 7499/225000 [25:47:58<518:31:54,  8.58s/it]
  3%|▎         | 7500/225000 [25:48:04<463:50:15,  7.68s/it]
                                                            
{'loss': '0.7233', 'grad_norm': '1.035', 'learning_rate': '3.747e-05', 'epoch': '0.03333', 'num_input_tokens_seen': 46932688, 'train_runtime': '9.288e+04', 'train_tokens_per_second': '505.3'}

  3%|▎         | 7500/225000 [25:48:04<463:50:15,  7.68s/it]
  3%|▎         | 7501/225000 [25:48:16<551:23:10,  9.13s/it]
  3%|▎         | 7502/225000 [25:48:20<460:28:32,  7.62s/it]
  3%|▎         | 7503/225000 [25:48:25<409:22:11,  6.78s/it]
  3%|▎         | 7504/225000 [25:48:35<471:09:38,  7.80s/it]
  3%|▎         | 7505/225000 [25:48:41<436:02:07,  7.22s/it]
  3%|▎         | 7506/225000 [25:48:50<469:22:15,  7.77s/it]
  3%|▎         | 7507/225000 [25:48:59<482:19:15,  7.98s/it]
  3%|▎         | 7508/225000 [25:49:06<465:25:20,  7.70s/it]
  3%|▎         | 7509/225000 [25:49:13<460:24:04,  7.62s/it]
  3%|▎         | 7510/225000 [25:49:22<485:15:46,  8.03s/it]
  3%|▎         | 7511/225000 [25:49:27<429:38:34,  7.11s/it]
  3%|▎         | 7512/225000 [25:49:35<448:04:58,  7.42s/it]
  3%|▎         | 7513/225000 [25:49:43<460:54:08,  7.63s/it]
  3%|▎         | 7514/225000 [25:49:48<407:20:55,  6.74s/it]
  3%|▎         | 7515/225000 [25:49:57<452:29:18,  7.49s/it]
  3%|▎         | 7516/225000 [25:50:08<509:20:33,  8.43s/it]
  3%|▎         | 7517/225000 [25:50:18<542:30:05,  8.98s/it]
  3%|▎         | 7518/225000 [25:50:31<603:44:39,  9.99s/it]
  3%|▎         | 7519/225000 [25:50:35<503:05:37,  8.33s/it]
  3%|▎         | 7520/225000 [25:50:43<502:17:16,  8.31s/it]
  3%|▎         | 7521/225000 [25:50:48<437:36:27,  7.24s/it]
  3%|▎         | 7522/225000 [25:51:02<552:09:34,  9.14s/it]
  3%|▎         | 7523/225000 [25:51:11<553:23:15,  9.16s/it]
  3%|▎         | 7524/225000 [25:51:19<544:42:36,  9.02s/it]
  3%|▎         | 7525/225000 [25:51:32<611:26:51, 10.12s/it]
  3%|▎         | 7526/225000 [25:51:38<541:26:42,  8.96s/it]
  3%|▎         | 7527/225000 [25:51:43<457:53:47,  7.58s/it]
  3%|▎         | 7528/225000 [25:51:54<532:35:48,  8.82s/it]
  3%|▎         | 7529/225000 [25:51:59<448:57:23,  7.43s/it]
  3%|▎         | 7530/225000 [25:52:06<449:20:07,  7.44s/it]
  3%|▎         | 7531/225000 [25:52:17<509:13:23,  8.43s/it]
  3%|▎         | 7532/225000 [25:52:23<462:51:07,  7.66s/it]
  3%|▎         | 7533/225000 [25:52:32<485:22:00,  8.03s/it]
  3%|▎         | 7534/225000 [25:52:39<479:53:26,  7.94s/it]
  3%|▎         | 7535/225000 [25:52:47<482:41:10,  7.99s/it]
  3%|▎         | 7536/225000 [25:52:54<459:45:50,  7.61s/it]
  3%|▎         | 7537/225000 [25:53:04<491:40:37,  8.14s/it]
  3%|▎         | 7538/225000 [25:53:14<539:49:38,  8.94s/it]
  3%|▎         | 7539/225000 [25:53:31<685:50:57, 11.35s/it]
  3%|▎         | 7540/225000 [25:53:39<624:27:04, 10.34s/it]
  3%|▎         | 7541/225000 [25:53:44<518:40:51,  8.59s/it]
  3%|▎         | 7542/225000 [25:53:54<545:06:53,  9.02s/it]
  3%|▎         | 7543/225000 [25:53:58<463:26:35,  7.67s/it]
  3%|▎         | 7544/225000 [25:54:08<493:53:36,  8.18s/it]
  3%|▎         | 7545/225000 [25:54:16<491:36:06,  8.14s/it]
  3%|▎         | 7546/225000 [25:54:20<421:24:28,  6.98s/it]
  3%|▎         | 7547/225000 [25:54:26<411:14:58,  6.81s/it]
  3%|▎         | 7548/225000 [25:54:34<421:24:50,  6.98s/it]
  3%|▎         | 7549/225000 [25:54:41<422:23:42,  6.99s/it]
  3%|▎         | 7550/225000 [25:54:51<487:57:22,  8.08s/it]
                                                            
{'loss': '0.7599', 'grad_norm': '0.9657', 'learning_rate': '3.772e-05', 'epoch': '0.03356', 'num_input_tokens_seen': 47130240, 'train_runtime': '9.329e+04', 'train_tokens_per_second': '505.2'}

  3%|▎         | 7550/225000 [25:54:51<487:57:22,  8.08s/it]
  3%|▎         | 7551/225000 [25:55:00<497:51:13,  8.24s/it]
  3%|▎         | 7552/225000 [25:55:09<507:34:40,  8.40s/it]
  3%|▎         | 7553/225000 [25:55:14<441:26:56,  7.31s/it]
  3%|▎         | 7554/225000 [25:55:18<395:39:27,  6.55s/it]
  3%|▎         | 7555/225000 [25:55:30<482:49:08,  7.99s/it]
  3%|▎         | 7556/225000 [25:55:40<516:43:23,  8.55s/it]
  3%|▎         | 7557/225000 [25:55:55<633:36:02, 10.49s/it]
  3%|▎         | 7558/225000 [25:56:09<702:53:42, 11.64s/it]
  3%|▎         | 7559/225000 [25:56:14<579:30:06,  9.59s/it]
  3%|▎         | 7560/225000 [25:56:21<540:25:07,  8.95s/it]
  3%|▎         | 7561/225000 [25:56:28<508:54:51,  8.43s/it]
  3%|▎         | 7562/225000 [25:56:47<695:25:16, 11.51s/it]
  3%|▎         | 7563/225000 [25:56:53<595:19:16,  9.86s/it]
  3%|▎         | 7564/225000 [25:57:05<627:51:18, 10.40s/it]
  3%|▎         | 7565/225000 [25:57:18<680:16:28, 11.26s/it]
  3%|▎         | 7566/225000 [25:57:26<618:06:24, 10.23s/it]
  3%|▎         | 7567/225000 [25:57:34<579:25:54,  9.59s/it]
  3%|▎         | 7568/225000 [25:57:40<510:34:18,  8.45s/it]
  3%|▎         | 7569/225000 [25:57:44<430:35:06,  7.13s/it]
  3%|▎         | 7570/225000 [25:57:52<442:10:02,  7.32s/it]
  3%|▎         | 7571/225000 [25:57:56<395:38:07,  6.55s/it]
  3%|▎         | 7572/225000 [25:58:04<415:45:20,  6.88s/it]
  3%|▎         | 7573/225000 [25:58:22<614:08:05, 10.17s/it]
  3%|▎         | 7574/225000 [25:58:25<493:26:10,  8.17s/it]
  3%|▎         | 7575/225000 [25:58:35<513:45:54,  8.51s/it]
  3%|▎         | 7576/225000 [25:58:41<465:56:44,  7.71s/it]
  3%|▎         | 7577/225000 [25:58:49<473:06:12,  7.83s/it]
  3%|▎         | 7578/225000 [25:58:53<409:40:25,  6.78s/it]
  3%|▎         | 7579/225000 [25:59:05<498:10:01,  8.25s/it]
  3%|▎         | 7580/225000 [25:59:11<459:07:45,  7.60s/it]
  3%|▎         | 7581/225000 [25:59:16<408:50:08,  6.77s/it]
  3%|▎         | 7582/225000 [25:59:22<398:54:24,  6.61s/it]
  3%|▎         | 7583/225000 [25:59:30<420:59:55,  6.97s/it]
  3%|▎         | 7584/225000 [25:59:42<513:09:07,  8.50s/it]
  3%|▎         | 7585/225000 [25:59:47<458:23:48,  7.59s/it]
  3%|▎         | 7586/225000 [25:59:54<452:43:24,  7.50s/it]
  3%|▎         | 7587/225000 [26:00:02<458:43:03,  7.60s/it]
  3%|▎         | 7588/225000 [26:00:08<434:36:41,  7.20s/it]
  3%|▎         | 7589/225000 [26:00:13<394:01:30,  6.52s/it]
  3%|▎         | 7590/225000 [26:00:29<562:23:20,  9.31s/it]
  3%|▎         | 7591/225000 [26:00:39<571:24:44,  9.46s/it]
  3%|▎         | 7592/225000 [26:00:47<545:39:33,  9.04s/it]
  3%|▎         | 7593/225000 [26:01:00<613:38:07, 10.16s/it]
  3%|▎         | 7594/225000 [26:01:09<590:34:24,  9.78s/it]
  3%|▎         | 7595/225000 [26:01:19<606:11:39, 10.04s/it]
  3%|▎         | 7596/225000 [26:01:27<558:18:05,  9.24s/it]
  3%|▎         | 7597/225000 [26:01:31<462:51:38,  7.66s/it]
  3%|▎         | 7598/225000 [26:01:39<468:37:03,  7.76s/it]
  3%|▎         | 7599/225000 [26:01:53<581:07:59,  9.62s/it]
  3%|▎         | 7600/225000 [26:02:00<531:28:54,  8.80s/it]
                                                            
{'loss': '0.7056', 'grad_norm': '0.9358', 'learning_rate': '3.797e-05', 'epoch': '0.03378', 'num_input_tokens_seen': 47326296, 'train_runtime': '9.372e+04', 'train_tokens_per_second': '505'}

  3%|▎         | 7600/225000 [26:02:00<531:28:54,  8.80s/it]
  3%|▎         | 7601/225000 [26:02:07<504:08:51,  8.35s/it]
  3%|▎         | 7602/225000 [26:02:18<561:27:42,  9.30s/it]
  3%|▎         | 7603/225000 [26:02:23<483:20:59,  8.00s/it]
  3%|▎         | 7604/225000 [26:02:27<410:31:07,  6.80s/it]
  3%|▎         | 7605/225000 [26:02:38<475:20:54,  7.87s/it]
  3%|▎         | 7606/225000 [26:02:46<487:16:21,  8.07s/it]
  3%|▎         | 7607/225000 [26:03:00<580:54:17,  9.62s/it]
  3%|▎         | 7608/225000 [26:03:10<590:40:54,  9.78s/it]
  3%|▎         | 7609/225000 [26:03:14<492:05:58,  8.15s/it]
  3%|▎         | 7610/225000 [26:03:21<477:53:47,  7.91s/it]
  3%|▎         | 7611/225000 [26:03:30<486:36:51,  8.06s/it]
  3%|▎         | 7612/225000 [26:03:37<464:00:58,  7.68s/it]
  3%|▎         | 7613/225000 [26:03:46<496:30:45,  8.22s/it]
  3%|▎         | 7614/225000 [26:03:54<493:56:44,  8.18s/it]
  3%|▎         | 7615/225000 [26:04:06<556:58:59,  9.22s/it]
  3%|▎         | 7616/225000 [26:04:11<478:36:39,  7.93s/it]
  3%|▎         | 7617/225000 [26:04:15<407:49:35,  6.75s/it]
  3%|▎         | 7618/225000 [26:04:26<489:37:48,  8.11s/it]
  3%|▎         | 7619/225000 [26:04:32<459:00:37,  7.60s/it]
  3%|▎         | 7620/225000 [26:04:43<517:35:25,  8.57s/it]
  3%|▎         | 7621/225000 [26:04:55<570:35:25,  9.45s/it]
  3%|▎         | 7622/225000 [26:05:03<543:26:50,  9.00s/it]
  3%|▎         | 7623/225000 [26:05:14<589:46:34,  9.77s/it]
  3%|▎         | 7624/225000 [26:05:20<510:13:15,  8.45s/it]
  3%|▎         | 7625/225000 [26:05:29<527:07:03,  8.73s/it]
  3%|▎         | 7626/225000 [26:05:42<611:49:01, 10.13s/it]
  3%|▎         | 7627/225000 [26:05:46<490:24:23,  8.12s/it]
  3%|▎         | 7628/225000 [26:05:53<472:14:20,  7.82s/it]
  3%|▎         | 7629/225000 [26:05:58<416:17:47,  6.89s/it]
  3%|▎         | 7630/225000 [26:06:06<442:51:54,  7.33s/it]
  3%|▎         | 7631/225000 [26:06:15<470:04:17,  7.79s/it]
  3%|▎         | 7632/225000 [26:06:20<427:21:45,  7.08s/it]
  3%|▎         | 7633/225000 [26:06:26<394:46:39,  6.54s/it]
  3%|▎         | 7634/225000 [26:06:37<484:43:43,  8.03s/it]
  3%|▎         | 7635/225000 [26:06:43<447:01:52,  7.40s/it]
  3%|▎         | 7636/225000 [26:06:54<518:09:21,  8.58s/it]
  3%|▎         | 7637/225000 [26:06:59<439:25:30,  7.28s/it]
  3%|▎         | 7638/225000 [26:07:04<407:52:04,  6.76s/it]
  3%|▎         | 7639/225000 [26:07:20<569:31:51,  9.43s/it]
  3%|▎         | 7640/225000 [26:07:25<489:18:32,  8.10s/it]
  3%|▎         | 7641/225000 [26:07:34<500:04:52,  8.28s/it]
  3%|▎         | 7642/225000 [26:07:39<442:07:57,  7.32s/it]
  3%|▎         | 7643/225000 [26:07:43<391:01:08,  6.48s/it]
  3%|▎         | 7644/225000 [26:07:57<524:02:47,  8.68s/it]
  3%|▎         | 7645/225000 [26:08:05<515:55:32,  8.55s/it]
  3%|▎         | 7646/225000 [26:08:14<516:03:57,  8.55s/it]
  3%|▎         | 7647/225000 [26:08:22<505:10:39,  8.37s/it]
  3%|▎         | 7648/225000 [26:08:31<522:24:37,  8.65s/it]
  3%|▎         | 7649/225000 [26:08:45<609:47:05, 10.10s/it]
  3%|▎         | 7650/225000 [26:08:52<555:21:35,  9.20s/it]
                                                            
{'loss': '0.7713', 'grad_norm': '0.9741', 'learning_rate': '3.822e-05', 'epoch': '0.034', 'num_input_tokens_seen': 47524136, 'train_runtime': '9.413e+04', 'train_tokens_per_second': '504.9'}

  3%|▎         | 7650/225000 [26:08:52<555:21:35,  9.20s/it]
  3%|▎         | 7651/225000 [26:08:59<513:55:35,  8.51s/it]
  3%|▎         | 7652/225000 [26:09:06<494:03:28,  8.18s/it]
  3%|▎         | 7653/225000 [26:09:15<518:28:21,  8.59s/it]
  3%|▎         | 7654/225000 [26:09:21<471:47:28,  7.81s/it]
  3%|▎         | 7655/225000 [26:09:32<519:51:50,  8.61s/it]
  3%|▎         | 7656/225000 [26:09:40<510:43:55,  8.46s/it]
  3%|▎         | 7657/225000 [26:09:45<452:12:52,  7.49s/it]
  3%|▎         | 7658/225000 [26:09:57<537:30:27,  8.90s/it]
  3%|▎         | 7659/225000 [26:10:03<475:43:47,  7.88s/it]
  3%|▎         | 7660/225000 [26:10:11<481:39:18,  7.98s/it]
  3%|▎         | 7661/225000 [26:10:19<470:32:34,  7.79s/it]
  3%|▎         | 7662/225000 [26:10:28<493:24:55,  8.17s/it]
  3%|▎         | 7663/225000 [26:10:35<483:51:25,  8.01s/it]
  3%|▎         | 7664/225000 [26:10:41<448:25:30,  7.43s/it]
  3%|▎         | 7665/225000 [26:10:46<391:50:30,  6.49s/it]
  3%|▎         | 7666/225000 [26:10:50<357:46:24,  5.93s/it]
  3%|▎         | 7667/225000 [26:11:01<452:51:04,  7.50s/it]
  3%|▎         | 7668/225000 [26:11:13<524:05:49,  8.68s/it]
  3%|▎         | 7669/225000 [26:11:19<471:39:27,  7.81s/it]
  3%|▎         | 7670/225000 [26:11:29<526:38:23,  8.72s/it]
  3%|▎         | 7671/225000 [26:11:35<473:25:14,  7.84s/it]
  3%|▎         | 7672/225000 [26:11:43<479:51:05,  7.95s/it]
  3%|▎         | 7673/225000 [26:11:53<500:29:29,  8.29s/it]
  3%|▎         | 7674/225000 [26:12:02<523:19:29,  8.67s/it]
  3%|▎         | 7675/225000 [26:12:08<481:54:30,  7.98s/it]
  3%|▎         | 7676/225000 [26:12:19<530:45:44,  8.79s/it]
  3%|▎         | 7677/225000 [26:12:24<461:15:09,  7.64s/it]
  3%|▎         | 7678/225000 [26:12:35<529:03:38,  8.76s/it]
  3%|▎         | 7679/225000 [26:12:42<481:22:28,  7.97s/it]
  3%|▎         | 7680/225000 [26:12:46<422:16:50,  7.00s/it]
  3%|▎         | 7681/225000 [26:13:02<588:15:37,  9.74s/it]
  3%|▎         | 7682/225000 [26:13:06<470:16:58,  7.79s/it]
  3%|▎         | 7683/225000 [26:13:16<521:35:55,  8.64s/it]
  3%|▎         | 7684/225000 [26:13:30<615:44:44, 10.20s/it]
  3%|▎         | 7685/225000 [26:13:37<561:15:45,  9.30s/it]
  3%|▎         | 7686/225000 [26:13:45<539:18:37,  8.93s/it]
  3%|▎         | 7687/225000 [26:13:57<594:33:33,  9.85s/it]
  3%|▎         | 7688/225000 [26:14:03<508:37:58,  8.43s/it]
  3%|▎         | 7689/225000 [26:14:07<441:35:32,  7.32s/it]
  3%|▎         | 7690/225000 [26:14:15<449:12:42,  7.44s/it]
  3%|▎         | 7691/225000 [26:14:21<423:26:28,  7.01s/it]
  3%|▎         | 7692/225000 [26:14:32<493:24:52,  8.17s/it]
  3%|▎         | 7693/225000 [26:14:44<555:33:15,  9.20s/it]
  3%|▎         | 7694/225000 [26:14:48<475:12:55,  7.87s/it]
  3%|▎         | 7695/225000 [26:14:54<438:20:11,  7.26s/it]
  3%|▎         | 7696/225000 [26:15:11<605:00:04, 10.02s/it]
  3%|▎         | 7697/225000 [26:15:15<502:19:27,  8.32s/it]
  3%|▎         | 7698/225000 [26:15:26<547:51:37,  9.08s/it]
  3%|▎         | 7699/225000 [26:15:32<496:08:55,  8.22s/it]
  3%|▎         | 7700/225000 [26:15:38<447:45:37,  7.42s/it]
                                                            
{'loss': '0.745', 'grad_norm': '1.028', 'learning_rate': '3.847e-05', 'epoch': '0.03422', 'num_input_tokens_seen': 47718400, 'train_runtime': '9.454e+04', 'train_tokens_per_second': '504.8'}

  3%|▎         | 7700/225000 [26:15:38<447:45:37,  7.42s/it]
  3%|▎         | 7701/225000 [26:15:49<527:38:04,  8.74s/it]
  3%|▎         | 7702/225000 [26:15:54<446:41:27,  7.40s/it]
  3%|▎         | 7703/225000 [26:16:00<422:26:44,  7.00s/it]
  3%|▎         | 7704/225000 [26:16:12<515:25:53,  8.54s/it]
  3%|▎         | 7705/225000 [26:16:20<507:27:20,  8.41s/it]
  3%|▎         | 7706/225000 [26:16:25<446:21:40,  7.40s/it]
  3%|▎         | 7707/225000 [26:16:37<538:16:20,  8.92s/it]
  3%|▎         | 7708/225000 [26:16:46<523:24:45,  8.67s/it]
  3%|▎         | 7709/225000 [26:16:57<579:58:25,  9.61s/it]
  3%|▎         | 7710/225000 [26:17:06<560:36:32,  9.29s/it]
  3%|▎         | 7711/225000 [26:17:14<547:42:51,  9.07s/it]
  3%|▎         | 7712/225000 [26:17:27<616:36:13, 10.22s/it]
  3%|▎         | 7713/225000 [26:17:39<647:18:30, 10.72s/it]
  3%|▎         | 7714/225000 [26:17:45<564:38:51,  9.36s/it]
  3%|▎         | 7715/225000 [26:17:54<551:34:07,  9.14s/it]
  3%|▎         | 7716/225000 [26:18:03<541:02:45,  8.96s/it]
  3%|▎         | 7717/225000 [26:18:07<467:09:16,  7.74s/it]
  3%|▎         | 7718/225000 [26:18:18<510:01:39,  8.45s/it]
  3%|▎         | 7719/225000 [26:18:23<455:35:17,  7.55s/it]
  3%|▎         | 7720/225000 [26:18:33<507:15:10,  8.40s/it]
  3%|▎         | 7721/225000 [26:18:41<486:33:53,  8.06s/it]
  3%|▎         | 7722/225000 [26:18:48<464:29:42,  7.70s/it]
  3%|▎         | 7723/225000 [26:18:57<497:17:11,  8.24s/it]
  3%|▎         | 7724/225000 [26:19:06<507:51:13,  8.41s/it]
  3%|▎         | 7725/225000 [26:19:12<468:58:15,  7.77s/it]
  3%|▎         | 7726/225000 [26:19:19<454:34:13,  7.53s/it]
  3%|▎         | 7727/225000 [26:19:34<590:08:43,  9.78s/it]
  3%|▎         | 7728/225000 [26:19:40<522:45:10,  8.66s/it]
  3%|▎         | 7729/225000 [26:19:45<461:28:08,  7.65s/it]
  3%|▎         | 7730/225000 [26:20:04<654:57:31, 10.85s/it]
  3%|▎         | 7731/225000 [26:20:11<579:50:23,  9.61s/it]
  3%|▎         | 7732/225000 [26:20:24<647:56:57, 10.74s/it]
  3%|▎         | 7733/225000 [26:20:31<590:41:44,  9.79s/it]
  3%|▎         | 7734/225000 [26:20:45<650:00:04, 10.77s/it]
  3%|▎         | 7735/225000 [26:20:51<579:58:17,  9.61s/it]
  3%|▎         | 7736/225000 [26:20:58<523:38:05,  8.68s/it]
  3%|▎         | 7737/225000 [26:21:04<468:41:34,  7.77s/it]
  3%|▎         | 7738/225000 [26:21:13<495:41:28,  8.21s/it]
  3%|▎         | 7739/225000 [26:21:21<502:52:16,  8.33s/it]
  3%|▎         | 7740/225000 [26:21:28<470:10:09,  7.79s/it]
  3%|▎         | 7741/225000 [26:21:32<409:58:35,  6.79s/it]
  3%|▎         | 7742/225000 [26:21:42<466:35:59,  7.73s/it]
  3%|▎         | 7743/225000 [26:21:47<416:14:16,  6.90s/it]
  3%|▎         | 7744/225000 [26:21:54<415:33:17,  6.89s/it]
  3%|▎         | 7745/225000 [26:22:00<391:03:48,  6.48s/it]
  3%|▎         | 7746/225000 [26:22:12<498:28:35,  8.26s/it]
  3%|▎         | 7747/225000 [26:22:23<543:07:03,  9.00s/it]
  3%|▎         | 7748/225000 [26:22:27<456:01:17,  7.56s/it]
  3%|▎         | 7749/225000 [26:22:38<513:34:05,  8.51s/it]
  3%|▎         | 7750/225000 [26:22:44<474:05:19,  7.86s/it]
                                                            
{'loss': '0.7343', 'grad_norm': '1.027', 'learning_rate': '3.872e-05', 'epoch': '0.03444', 'num_input_tokens_seen': 47917696, 'train_runtime': '9.496e+04', 'train_tokens_per_second': '504.6'}

  3%|▎         | 7750/225000 [26:22:44<474:05:19,  7.86s/it]
  3%|▎         | 7751/225000 [26:22:56<542:52:39,  9.00s/it]
  3%|▎         | 7752/225000 [26:23:06<567:41:27,  9.41s/it]
  3%|▎         | 7753/225000 [26:23:13<526:33:29,  8.73s/it]
  3%|▎         | 7754/225000 [26:23:21<507:54:27,  8.42s/it]
  3%|▎         | 7755/225000 [26:23:26<444:32:35,  7.37s/it]
  3%|▎         | 7756/225000 [26:23:33<447:36:35,  7.42s/it]
  3%|▎         | 7757/225000 [26:23:42<471:08:43,  7.81s/it]
  3%|▎         | 7758/225000 [26:23:52<501:23:10,  8.31s/it]
  3%|▎         | 7759/225000 [26:24:00<509:42:46,  8.45s/it]
  3%|▎         | 7760/225000 [26:24:09<510:20:56,  8.46s/it]
  3%|▎         | 7761/225000 [26:24:17<514:01:39,  8.52s/it]
  3%|▎         | 7762/225000 [26:24:26<521:15:15,  8.64s/it]
  3%|▎         | 7763/225000 [26:24:37<548:39:44,  9.09s/it]
  3%|▎         | 7764/225000 [26:24:47<570:02:19,  9.45s/it]
  3%|▎         | 7765/225000 [26:24:51<482:02:21,  7.99s/it]
  3%|▎         | 7766/225000 [26:25:04<562:52:52,  9.33s/it]
  3%|▎         | 7767/225000 [26:25:08<472:54:40,  7.84s/it]
  3%|▎         | 7768/225000 [26:25:19<531:10:14,  8.80s/it]
  3%|▎         | 7769/225000 [26:25:27<514:15:25,  8.52s/it]
  3%|▎         | 7770/225000 [26:25:38<553:28:02,  9.17s/it]
  3%|▎         | 7771/225000 [26:25:46<530:52:40,  8.80s/it]
  3%|▎         | 7772/225000 [26:25:55<543:11:18,  9.00s/it]
  3%|▎         | 7773/225000 [26:26:01<476:00:19,  7.89s/it]
  3%|▎         | 7774/225000 [26:26:07<456:36:51,  7.57s/it]
  3%|▎         | 7775/225000 [26:26:12<408:09:43,  6.76s/it]
  3%|▎         | 7776/225000 [26:26:19<412:58:46,  6.84s/it]
  3%|▎         | 7777/225000 [26:26:30<488:13:58,  8.09s/it]
  3%|▎         | 7778/225000 [26:26:37<466:29:06,  7.73s/it]
  3%|▎         | 7779/225000 [26:26:42<420:00:39,  6.96s/it]
  3%|▎         | 7780/225000 [26:26:48<390:40:01,  6.47s/it]
  3%|▎         | 7781/225000 [26:27:01<520:35:34,  8.63s/it]
  3%|▎         | 7782/225000 [26:27:05<427:27:48,  7.08s/it]
  3%|▎         | 7783/225000 [26:27:24<651:40:02, 10.80s/it]
  3%|▎         | 7784/225000 [26:27:31<582:27:55,  9.65s/it]
  3%|▎         | 7785/225000 [26:27:45<653:15:51, 10.83s/it]
  3%|▎         | 7786/225000 [26:27:59<712:17:41, 11.81s/it]
  3%|▎         | 7787/225000 [26:28:10<696:54:26, 11.55s/it]
  3%|▎         | 7788/225000 [26:28:14<567:21:54,  9.40s/it]
  3%|▎         | 7789/225000 [26:28:20<508:27:01,  8.43s/it]
  3%|▎         | 7790/225000 [26:28:26<453:56:55,  7.52s/it]
  3%|▎         | 7791/225000 [26:28:36<506:12:17,  8.39s/it]
  3%|▎         | 7792/225000 [26:28:42<450:56:15,  7.47s/it]
  3%|▎         | 7793/225000 [26:28:52<511:24:44,  8.48s/it]
  3%|▎         | 7794/225000 [26:28:57<449:29:13,  7.45s/it]
  3%|▎         | 7795/225000 [26:29:09<532:46:50,  8.83s/it]
  3%|▎         | 7796/225000 [26:29:19<549:57:01,  9.12s/it]
  3%|▎         | 7797/225000 [26:29:25<483:36:19,  8.02s/it]
  3%|▎         | 7798/225000 [26:29:39<594:58:47,  9.86s/it]
  3%|▎         | 7799/225000 [26:29:44<509:08:56,  8.44s/it]
  3%|▎         | 7800/225000 [26:29:55<555:08:27,  9.20s/it]
                                                            
{'loss': '0.7722', 'grad_norm': '1.054', 'learning_rate': '3.897e-05', 'epoch': '0.03467', 'num_input_tokens_seen': 48117648, 'train_runtime': '9.54e+04', 'train_tokens_per_second': '504.4'}

  3%|▎         | 7800/225000 [26:29:55<555:08:27,  9.20s/it]
  3%|▎         | 7801/225000 [26:29:58<448:49:49,  7.44s/it]
  3%|▎         | 7802/225000 [26:30:07<466:13:15,  7.73s/it]
  3%|▎         | 7803/225000 [26:30:13<436:12:29,  7.23s/it]
  3%|▎         | 7804/225000 [26:30:22<478:22:59,  7.93s/it]
  3%|▎         | 7805/225000 [26:30:33<522:32:47,  8.66s/it]
  3%|▎         | 7806/225000 [26:30:40<497:45:42,  8.25s/it]
  3%|▎         | 7807/225000 [26:30:45<437:18:05,  7.25s/it]
  3%|▎         | 7808/225000 [26:30:56<509:13:18,  8.44s/it]
  3%|▎         | 7809/225000 [26:31:02<467:05:14,  7.74s/it]
  3%|▎         | 7810/225000 [26:31:10<463:26:10,  7.68s/it]
  3%|▎         | 7811/225000 [26:31:22<549:37:55,  9.11s/it]
  3%|▎         | 7812/225000 [26:31:27<463:05:15,  7.68s/it]
  3%|▎         | 7813/225000 [26:31:43<619:19:17, 10.27s/it]
  3%|▎         | 7814/225000 [26:31:49<541:15:36,  8.97s/it]
  3%|▎         | 7815/225000 [26:31:54<479:45:23,  7.95s/it]
  3%|▎         | 7816/225000 [26:32:01<454:19:44,  7.53s/it]
  3%|▎         | 7817/225000 [26:32:13<544:25:37,  9.02s/it]
  3%|▎         | 7818/225000 [26:32:19<486:52:36,  8.07s/it]
  3%|▎         | 7819/225000 [26:32:24<435:02:46,  7.21s/it]
  3%|▎         | 7820/225000 [26:32:41<596:16:41,  9.88s/it]
  3%|▎         | 7821/225000 [26:32:45<488:44:06,  8.10s/it]
  3%|▎         | 7822/225000 [26:32:54<507:15:07,  8.41s/it]
  3%|▎         | 7823/225000 [26:33:01<482:10:02,  7.99s/it]
  3%|▎         | 7824/225000 [26:33:10<501:45:01,  8.32s/it]
  3%|▎         | 7825/225000 [26:33:19<515:48:31,  8.55s/it]
  3%|▎         | 7826/225000 [26:33:28<532:30:29,  8.83s/it]
  3%|▎         | 7827/225000 [26:33:36<518:08:53,  8.59s/it]
  3%|▎         | 7828/225000 [26:33:44<492:32:01,  8.16s/it]
  3%|▎         | 7829/225000 [26:33:58<599:49:22,  9.94s/it]
  3%|▎         | 7830/225000 [26:34:03<510:43:11,  8.47s/it]
  3%|▎         | 7831/225000 [26:34:23<725:39:39, 12.03s/it]
  3%|▎         | 7832/225000 [26:34:31<658:49:12, 10.92s/it]
  3%|▎         | 7833/225000 [26:34:38<572:56:39,  9.50s/it]
  3%|▎         | 7834/225000 [26:34:54<695:37:52, 11.53s/it]
  3%|▎         | 7835/225000 [26:34:59<573:15:15,  9.50s/it]
  3%|▎         | 7836/225000 [26:35:10<610:49:07, 10.13s/it]
  3%|▎         | 7837/225000 [26:35:17<544:52:31,  9.03s/it]
  3%|▎         | 7838/225000 [26:35:23<502:51:31,  8.34s/it]
  3%|▎         | 7839/225000 [26:35:30<481:16:13,  7.98s/it]
  3%|▎         | 7840/225000 [26:35:41<520:51:19,  8.63s/it]
  3%|▎         | 7841/225000 [26:35:47<477:02:22,  7.91s/it]
  3%|▎         | 7842/225000 [26:35:51<415:10:45,  6.88s/it]
  3%|▎         | 7843/225000 [26:36:03<493:56:14,  8.19s/it]
  3%|▎         | 7844/225000 [26:36:14<547:46:00,  9.08s/it]
  3%|▎         | 7845/225000 [26:36:21<517:31:03,  8.58s/it]
  3%|▎         | 7846/225000 [26:36:26<445:53:28,  7.39s/it]
  3%|▎         | 7847/225000 [26:36:36<488:44:34,  8.10s/it]
  3%|▎         | 7848/225000 [26:36:42<452:04:38,  7.49s/it]
  3%|▎         | 7849/225000 [26:36:47<406:36:47,  6.74s/it]
  3%|▎         | 7850/225000 [26:36:51<370:08:04,  6.14s/it]
                                                            
{'loss': '0.7039', 'grad_norm': '0.9958', 'learning_rate': '3.922e-05', 'epoch': '0.03489', 'num_input_tokens_seen': 48318296, 'train_runtime': '9.581e+04', 'train_tokens_per_second': '504.3'}

  3%|▎         | 7850/225000 [26:36:51<370:08:04,  6.14s/it]
  3%|▎         | 7851/225000 [26:37:10<596:39:55,  9.89s/it]
  3%|▎         | 7852/225000 [26:37:14<499:01:53,  8.27s/it]
  3%|▎         | 7853/225000 [26:37:25<536:38:23,  8.90s/it]
  3%|▎         | 7854/225000 [26:37:31<482:20:44,  8.00s/it]
  3%|▎         | 7855/225000 [26:37:44<572:35:50,  9.49s/it]
  3%|▎         | 7856/225000 [26:37:51<538:06:37,  8.92s/it]
  3%|▎         | 7857/225000 [26:37:58<495:39:51,  8.22s/it]
  3%|▎         | 7858/225000 [26:38:02<426:31:55,  7.07s/it]
  3%|▎         | 7859/225000 [26:38:13<493:55:32,  8.19s/it]
  3%|▎         | 7860/225000 [26:38:20<472:28:28,  7.83s/it]
  3%|▎         | 7861/225000 [26:38:25<428:34:46,  7.11s/it]
  3%|▎         | 7862/225000 [26:38:34<460:28:59,  7.63s/it]
  3%|▎         | 7863/225000 [26:38:42<452:51:35,  7.51s/it]
  3%|▎         | 7864/225000 [26:38:51<483:09:47,  8.01s/it]
  3%|▎         | 7865/225000 [26:38:59<489:47:19,  8.12s/it]
  3%|▎         | 7866/225000 [26:39:06<470:07:52,  7.79s/it]
  3%|▎         | 7867/225000 [26:39:13<460:22:29,  7.63s/it]
  3%|▎         | 7868/225000 [26:39:22<478:00:18,  7.93s/it]
  3%|▎         | 7869/225000 [26:39:30<484:54:52,  8.04s/it]
  3%|▎         | 7870/225000 [26:39:40<522:24:20,  8.66s/it]
  3%|▎         | 7871/225000 [26:39:45<439:49:02,  7.29s/it]
  3%|▎         | 7872/225000 [26:40:05<677:51:39, 11.24s/it]
  3%|▎         | 7873/225000 [26:40:10<570:17:58,  9.46s/it]
  3%|▎         | 7874/225000 [26:40:17<514:48:46,  8.54s/it]
  4%|▎         | 7875/225000 [26:40:22<465:34:49,  7.72s/it]
  4%|▎         | 7876/225000 [26:40:28<424:31:18,  7.04s/it]
  4%|▎         | 7877/225000 [26:40:33<388:01:53,  6.43s/it]
  4%|▎         | 7878/225000 [26:40:39<380:25:47,  6.31s/it]
  4%|▎         | 7879/225000 [26:40:44<354:37:49,  5.88s/it]
  4%|▎         | 7880/225000 [26:40:59<520:21:16,  8.63s/it]
  4%|▎         | 7881/225000 [26:41:04<462:30:53,  7.67s/it]
  4%|▎         | 7882/225000 [26:41:09<408:49:34,  6.78s/it]
  4%|▎         | 7883/225000 [26:41:21<496:41:06,  8.24s/it]
  4%|▎         | 7884/225000 [26:41:30<522:55:12,  8.67s/it]
  4%|▎         | 7885/225000 [26:41:39<515:43:15,  8.55s/it]
  4%|▎         | 7886/225000 [26:41:55<664:51:10, 11.02s/it]
  4%|▎         | 7887/225000 [26:42:03<594:22:51,  9.86s/it]
  4%|▎         | 7888/225000 [26:42:10<559:18:21,  9.27s/it]
  4%|▎         | 7889/225000 [26:42:22<595:12:27,  9.87s/it]
  4%|▎         | 7890/225000 [26:42:31<591:25:48,  9.81s/it]
  4%|▎         | 7891/225000 [26:42:38<526:26:25,  8.73s/it]
  4%|▎         | 7892/225000 [26:42:42<453:39:08,  7.52s/it]
  4%|▎         | 7893/225000 [26:42:53<503:20:55,  8.35s/it]
  4%|▎         | 7894/225000 [26:43:00<480:49:18,  7.97s/it]
  4%|▎         | 7895/225000 [26:43:10<524:05:58,  8.69s/it]
  4%|▎         | 7896/225000 [26:43:20<544:57:52,  9.04s/it]
  4%|▎         | 7897/225000 [26:43:26<493:07:44,  8.18s/it]
  4%|▎         | 7898/225000 [26:43:31<428:21:45,  7.10s/it]
  4%|▎         | 7899/225000 [26:43:41<481:45:40,  7.99s/it]
  4%|▎         | 7900/225000 [26:43:52<535:20:58,  8.88s/it]
                                                            
{'loss': '0.7282', 'grad_norm': '1.064', 'learning_rate': '3.947e-05', 'epoch': '0.03511', 'num_input_tokens_seen': 48513440, 'train_runtime': '9.623e+04', 'train_tokens_per_second': '504.1'}

  4%|▎         | 7900/225000 [26:43:52<535:20:58,  8.88s/it]
  4%|▎         | 7901/225000 [26:43:58<490:15:43,  8.13s/it]
  4%|▎         | 7902/225000 [26:44:05<467:28:43,  7.75s/it]
  4%|▎         | 7903/225000 [26:44:20<601:34:59,  9.98s/it]
  4%|▎         | 7904/225000 [26:44:32<633:08:30, 10.50s/it]
  4%|▎         | 7905/225000 [26:44:36<516:14:02,  8.56s/it]
  4%|▎         | 7906/225000 [26:44:45<523:13:24,  8.68s/it]
  4%|▎         | 7907/225000 [26:44:49<449:09:25,  7.45s/it]
  4%|▎         | 7908/225000 [26:45:05<602:39:38,  9.99s/it]
  4%|▎         | 7909/225000 [26:45:16<606:42:35, 10.06s/it]
  4%|▎         | 7910/225000 [26:45:28<644:18:53, 10.68s/it]
  4%|▎         | 7911/225000 [26:45:38<630:01:57, 10.45s/it]
  4%|▎         | 7912/225000 [26:45:48<625:14:30, 10.37s/it]
  4%|▎         | 7913/225000 [26:45:54<548:20:52,  9.09s/it]
  4%|▎         | 7914/225000 [26:45:59<475:41:37,  7.89s/it]
  4%|▎         | 7915/225000 [26:46:07<475:55:04,  7.89s/it]
  4%|▎         | 7916/225000 [26:46:11<400:59:52,  6.65s/it]
  4%|▎         | 7917/225000 [26:46:22<488:54:28,  8.11s/it]
  4%|▎         | 7918/225000 [26:46:33<539:42:09,  8.95s/it]
  4%|▎         | 7919/225000 [26:46:44<576:05:36,  9.55s/it]
  4%|▎         | 7920/225000 [26:46:54<586:07:25,  9.72s/it]
  4%|▎         | 7921/225000 [26:47:02<559:37:09,  9.28s/it]
  4%|▎         | 7922/225000 [26:47:06<464:31:30,  7.70s/it]
  4%|▎         | 7923/225000 [26:47:11<409:20:40,  6.79s/it]
  4%|▎         | 7924/225000 [26:47:20<453:31:39,  7.52s/it]
  4%|▎         | 7925/225000 [26:47:31<514:26:36,  8.53s/it]
  4%|▎         | 7926/225000 [26:47:41<531:01:19,  8.81s/it]
  4%|▎         | 7927/225000 [26:47:46<465:39:55,  7.72s/it]
  4%|▎         | 7928/225000 [26:47:51<420:49:32,  6.98s/it]
  4%|▎         | 7929/225000 [26:48:00<462:15:43,  7.67s/it]
  4%|▎         | 7930/225000 [26:48:15<582:59:24,  9.67s/it]
  4%|▎         | 7931/225000 [26:48:20<497:05:41,  8.24s/it]
  4%|▎         | 7932/225000 [26:48:30<534:17:50,  8.86s/it]
  4%|▎         | 7933/225000 [26:48:36<484:53:45,  8.04s/it]
  4%|▎         | 7934/225000 [26:48:49<573:11:47,  9.51s/it]
  4%|▎         | 7935/225000 [26:48:54<490:45:27,  8.14s/it]
  4%|▎         | 7936/225000 [26:48:58<426:19:46,  7.07s/it]
  4%|▎         | 7937/225000 [26:49:14<571:44:43,  9.48s/it]
  4%|▎         | 7938/225000 [26:49:19<494:28:01,  8.20s/it]
  4%|▎         | 7939/225000 [26:49:24<434:21:09,  7.20s/it]
  4%|▎         | 7940/225000 [26:49:30<416:52:10,  6.91s/it]
  4%|▎         | 7941/225000 [26:49:43<520:47:11,  8.64s/it]
  4%|▎         | 7942/225000 [26:49:49<478:59:47,  7.94s/it]
  4%|▎         | 7943/225000 [26:49:54<423:59:35,  7.03s/it]
  4%|▎         | 7944/225000 [26:50:06<512:27:20,  8.50s/it]
  4%|▎         | 7945/225000 [26:50:10<445:54:50,  7.40s/it]
  4%|▎         | 7946/225000 [26:50:22<512:51:27,  8.51s/it]
  4%|▎         | 7947/225000 [26:50:28<474:53:45,  7.88s/it]
  4%|▎         | 7948/225000 [26:50:32<399:19:59,  6.62s/it]
  4%|▎         | 7949/225000 [26:50:38<385:45:11,  6.40s/it]
  4%|▎         | 7950/225000 [26:50:55<582:34:12,  9.66s/it]
                                                            
{'loss': '0.7508', 'grad_norm': '0.9895', 'learning_rate': '3.972e-05', 'epoch': '0.03533', 'num_input_tokens_seen': 48709984, 'train_runtime': '9.666e+04', 'train_tokens_per_second': '504'}

  4%|▎         | 7950/225000 [26:50:55<582:34:12,  9.66s/it]
  4%|▎         | 7951/225000 [26:51:00<497:13:40,  8.25s/it]
  4%|▎         | 7952/225000 [26:51:05<437:19:28,  7.25s/it]
  4%|▎         | 7953/225000 [26:51:15<493:47:23,  8.19s/it]
  4%|▎         | 7954/225000 [26:51:21<445:17:09,  7.39s/it]
  4%|▎         | 7955/225000 [26:51:27<423:37:06,  7.03s/it]
  4%|▎         | 7956/225000 [26:51:40<537:47:01,  8.92s/it]
  4%|▎         | 7957/225000 [26:51:45<458:41:20,  7.61s/it]
  4%|▎         | 7958/225000 [26:51:53<463:43:04,  7.69s/it]
  4%|▎         | 7959/225000 [26:51:58<431:06:38,  7.15s/it]
  4%|▎         | 7960/225000 [26:52:11<525:19:11,  8.71s/it]
  4%|▎         | 7961/225000 [26:52:15<435:31:40,  7.22s/it]
  4%|▎         | 7962/225000 [26:52:23<465:43:31,  7.72s/it]
  4%|▎         | 7963/225000 [26:52:29<422:59:45,  7.02s/it]
  4%|▎         | 7964/225000 [26:52:38<463:37:04,  7.69s/it]
  4%|▎         | 7965/225000 [26:52:45<458:02:38,  7.60s/it]
  4%|▎         | 7966/225000 [26:52:56<511:33:15,  8.49s/it]
  4%|▎         | 7967/225000 [26:53:03<488:55:19,  8.11s/it]
  4%|▎         | 7968/225000 [26:53:09<445:04:30,  7.38s/it]
  4%|▎         | 7969/225000 [26:53:19<484:37:12,  8.04s/it]
  4%|▎         | 7970/225000 [26:53:26<481:38:45,  7.99s/it]
  4%|▎         | 7971/225000 [26:53:34<477:12:06,  7.92s/it]
  4%|▎         | 7972/225000 [26:53:40<445:27:13,  7.39s/it]
  4%|▎         | 7973/225000 [26:53:45<393:30:25,  6.53s/it]
  4%|▎         | 7974/225000 [26:53:54<446:39:42,  7.41s/it]
  4%|▎         | 7975/225000 [26:53:58<385:53:13,  6.40s/it]
  4%|▎         | 7976/225000 [26:54:03<348:39:35,  5.78s/it]
  4%|▎         | 7977/225000 [26:54:15<463:27:32,  7.69s/it]
  4%|▎         | 7978/225000 [26:54:19<405:28:54,  6.73s/it]
  4%|▎         | 7979/225000 [26:54:25<388:43:31,  6.45s/it]
  4%|▎         | 7980/225000 [26:54:36<473:38:05,  7.86s/it]
  4%|▎         | 7981/225000 [26:54:46<510:54:40,  8.48s/it]
  4%|▎         | 7982/225000 [26:54:51<451:07:46,  7.48s/it]
  4%|▎         | 7983/225000 [26:54:57<425:24:49,  7.06s/it]
  4%|▎         | 7984/225000 [26:55:04<425:47:53,  7.06s/it]
  4%|▎         | 7985/225000 [26:55:17<528:57:25,  8.77s/it]
  4%|▎         | 7986/225000 [26:55:22<447:46:56,  7.43s/it]
  4%|▎         | 7987/225000 [26:55:27<413:08:30,  6.85s/it]
  4%|▎         | 7988/225000 [26:55:34<421:37:27,  6.99s/it]
  4%|▎         | 7989/225000 [26:55:43<446:34:55,  7.41s/it]
  4%|▎         | 7990/225000 [26:55:51<468:15:59,  7.77s/it]
  4%|▎         | 7991/225000 [26:55:56<416:58:05,  6.92s/it]
  4%|▎         | 7992/225000 [26:56:01<382:39:48,  6.35s/it]
  4%|▎         | 7993/225000 [26:56:06<359:10:24,  5.96s/it]
  4%|▎         | 7994/225000 [26:56:22<533:18:00,  8.85s/it]
  4%|▎         | 7995/225000 [26:56:29<504:07:39,  8.36s/it]
  4%|▎         | 7996/225000 [26:56:38<515:57:58,  8.56s/it]
  4%|▎         | 7997/225000 [26:56:45<481:01:35,  7.98s/it]
  4%|▎         | 7998/225000 [26:56:50<435:13:44,  7.22s/it]
  4%|▎         | 7999/225000 [26:56:56<411:44:10,  6.83s/it]
  4%|▎         | 8000/225000 [26:57:13<598:03:03,  9.92s/it]
                                                            
{'loss': '0.7377', 'grad_norm': '0.7007', 'learning_rate': '3.997e-05', 'epoch': '0.03556', 'num_input_tokens_seen': 48894944, 'train_runtime': '9.703e+04', 'train_tokens_per_second': '503.9'}

  4%|▎         | 8000/225000 [26:57:13<598:03:03,  9.92s/it]
  4%|▎         | 8001/225000 [26:57:24<620:55:26, 10.30s/it]
  4%|▎         | 8002/225000 [26:57:29<507:26:39,  8.42s/it]
  4%|▎         | 8003/225000 [26:57:36<491:11:29,  8.15s/it]
  4%|▎         | 8004/225000 [26:57:52<638:41:24, 10.60s/it]
  4%|▎         | 8005/225000 [26:57:58<545:22:08,  9.05s/it]
  4%|▎         | 8006/225000 [26:58:03<468:21:53,  7.77s/it]
  4%|▎         | 8007/225000 [26:58:07<417:08:46,  6.92s/it]
  4%|▎         | 8008/225000 [26:58:19<500:24:41,  8.30s/it]
  4%|▎         | 8009/225000 [26:58:25<454:47:25,  7.55s/it]
  4%|▎         | 8010/225000 [26:58:30<419:13:48,  6.96s/it]
  4%|▎         | 8011/225000 [26:58:39<454:39:48,  7.54s/it]
  4%|▎         | 8012/225000 [26:58:51<521:40:07,  8.65s/it]
  4%|▎         | 8013/225000 [26:59:05<631:49:34, 10.48s/it]
  4%|▎         | 8014/225000 [26:59:12<571:39:56,  9.48s/it]
  4%|▎         | 8015/225000 [26:59:17<481:24:59,  7.99s/it]
  4%|▎         | 8016/225000 [26:59:24<471:54:32,  7.83s/it]
  4%|▎         | 8017/225000 [26:59:35<515:39:01,  8.56s/it]
  4%|▎         | 8018/225000 [26:59:48<597:03:28,  9.91s/it]
  4%|▎         | 8019/225000 [26:59:58<598:53:00,  9.94s/it]
  4%|▎         | 8020/225000 [27:00:06<575:45:13,  9.55s/it]
  4%|▎         | 8021/225000 [27:00:13<528:41:47,  8.77s/it]
  4%|▎         | 8022/225000 [27:00:21<517:18:18,  8.58s/it]
  4%|▎         | 8023/225000 [27:00:27<462:57:29,  7.68s/it]
  4%|▎         | 8024/225000 [27:00:32<410:51:37,  6.82s/it]
  4%|▎         | 8025/225000 [27:00:40<437:45:18,  7.26s/it]
  4%|▎         | 8026/225000 [27:00:52<527:28:00,  8.75s/it]
  4%|▎         | 8027/225000 [27:00:59<483:57:52,  8.03s/it]
  4%|▎         | 8028/225000 [27:01:10<544:36:13,  9.04s/it]
  4%|▎         | 8029/225000 [27:01:18<517:17:48,  8.58s/it]
  4%|▎         | 8030/225000 [27:01:22<445:10:15,  7.39s/it]
  4%|▎         | 8031/225000 [27:01:28<421:49:09,  7.00s/it]
  4%|▎         | 8032/225000 [27:01:33<381:09:42,  6.32s/it]
  4%|▎         | 8033/225000 [27:01:46<493:23:42,  8.19s/it]
  4%|▎         | 8034/225000 [27:01:52<457:44:08,  7.59s/it]
  4%|▎         | 8035/225000 [27:01:57<420:06:31,  6.97s/it]
  4%|▎         | 8036/225000 [27:02:07<465:19:16,  7.72s/it]
  4%|▎         | 8037/225000 [27:02:12<414:23:06,  6.88s/it]
  4%|▎         | 8038/225000 [27:02:17<394:48:53,  6.55s/it]
  4%|▎         | 8039/225000 [27:02:34<566:23:45,  9.40s/it]
  4%|▎         | 8040/225000 [27:02:45<602:02:58,  9.99s/it]
  4%|▎         | 8041/225000 [27:02:49<499:17:18,  8.28s/it]
  4%|▎         | 8042/225000 [27:03:00<537:17:28,  8.92s/it]
  4%|▎         | 8043/225000 [27:03:05<475:12:00,  7.89s/it]
  4%|▎         | 8044/225000 [27:03:11<431:37:32,  7.16s/it]
  4%|▎         | 8045/225000 [27:03:16<403:22:07,  6.69s/it]
  4%|▎         | 8046/225000 [27:03:28<502:22:38,  8.34s/it]
  4%|▎         | 8047/225000 [27:03:35<474:16:17,  7.87s/it]
  4%|▎         | 8048/225000 [27:03:46<530:29:10,  8.80s/it]
  4%|▎         | 8049/225000 [27:03:50<433:57:50,  7.20s/it]
  4%|▎         | 8050/225000 [27:03:56<427:42:09,  7.10s/it]
                                                            
{'loss': '0.7426', 'grad_norm': '0.9331', 'learning_rate': '4.022e-05', 'epoch': '0.03578', 'num_input_tokens_seen': 49085624, 'train_runtime': '9.744e+04', 'train_tokens_per_second': '503.8'}

  4%|▎         | 8050/225000 [27:03:56<427:42:09,  7.10s/it]
  4%|▎         | 8051/225000 [27:04:05<448:44:55,  7.45s/it]
  4%|▎         | 8052/225000 [27:04:09<383:31:38,  6.36s/it]
  4%|▎         | 8053/225000 [27:04:21<497:02:22,  8.25s/it]
  4%|▎         | 8054/225000 [27:04:25<416:14:43,  6.91s/it]
  4%|▎         | 8055/225000 [27:04:38<531:09:38,  8.81s/it]
  4%|▎         | 8056/225000 [27:04:42<449:19:16,  7.46s/it]
  4%|▎         | 8057/225000 [27:04:53<505:03:19,  8.38s/it]
  4%|▎         | 8058/225000 [27:04:57<423:01:58,  7.02s/it]
  4%|▎         | 8059/225000 [27:05:04<433:12:52,  7.19s/it]
  4%|▎         | 8060/225000 [27:05:16<503:08:29,  8.35s/it]
  4%|▎         | 8061/225000 [27:05:25<519:12:14,  8.62s/it]
  4%|▎         | 8062/225000 [27:05:37<586:06:40,  9.73s/it]
  4%|▎         | 8063/225000 [27:05:49<619:55:57, 10.29s/it]
  4%|▎         | 8064/225000 [27:05:53<514:56:41,  8.55s/it]
  4%|▎         | 8065/225000 [27:05:57<434:29:19,  7.21s/it]
  4%|▎         | 8066/225000 [27:06:12<568:58:48,  9.44s/it]
  4%|▎         | 8067/225000 [27:06:24<617:21:29, 10.25s/it]
  4%|▎         | 8068/225000 [27:06:33<600:09:39,  9.96s/it]
  4%|▎         | 8069/225000 [27:06:43<589:16:56,  9.78s/it]
  4%|▎         | 8070/225000 [27:06:45<462:54:37,  7.68s/it]
  4%|▎         | 8071/225000 [27:06:58<550:15:10,  9.13s/it]
  4%|▎         | 8072/225000 [27:07:04<487:13:06,  8.09s/it]
  4%|▎         | 8073/225000 [27:07:11<470:58:26,  7.82s/it]
  4%|▎         | 8074/225000 [27:07:16<425:47:05,  7.07s/it]
  4%|▎         | 8075/225000 [27:07:22<403:29:15,  6.70s/it]
  4%|▎         | 8076/225000 [27:07:28<384:24:41,  6.38s/it]
  4%|▎         | 8077/225000 [27:07:39<470:57:24,  7.82s/it]
  4%|▎         | 8078/225000 [27:07:50<533:42:05,  8.86s/it]
  4%|▎         | 8079/225000 [27:07:57<502:20:05,  8.34s/it]
  4%|▎         | 8080/225000 [27:08:05<490:10:51,  8.14s/it]
  4%|▎         | 8081/225000 [27:08:09<424:49:17,  7.05s/it]
  4%|▎         | 8082/225000 [27:08:20<492:38:24,  8.18s/it]
  4%|▎         | 8083/225000 [27:08:31<545:05:59,  9.05s/it]
  4%|▎         | 8084/225000 [27:08:37<488:36:49,  8.11s/it]
  4%|▎         | 8085/225000 [27:08:42<436:01:33,  7.24s/it]
  4%|▎         | 8086/225000 [27:08:48<403:14:47,  6.69s/it]
  4%|▎         | 8087/225000 [27:08:59<491:08:41,  8.15s/it]
  4%|▎         | 8088/225000 [27:09:03<409:21:52,  6.79s/it]
  4%|▎         | 8089/225000 [27:09:14<479:45:33,  7.96s/it]
  4%|▎         | 8090/225000 [27:09:22<478:41:27,  7.94s/it]
  4%|▎         | 8091/225000 [27:09:29<477:41:43,  7.93s/it]
  4%|▎         | 8092/225000 [27:09:38<497:27:46,  8.26s/it]
  4%|▎         | 8093/225000 [27:09:48<516:48:13,  8.58s/it]
  4%|▎         | 8094/225000 [27:09:52<436:28:27,  7.24s/it]
  4%|▎         | 8095/225000 [27:10:04<518:44:33,  8.61s/it]
  4%|▎         | 8096/225000 [27:10:11<492:49:15,  8.18s/it]
  4%|▎         | 8097/225000 [27:10:20<509:43:37,  8.46s/it]
  4%|▎         | 8098/225000 [27:10:29<528:29:12,  8.77s/it]
  4%|▎         | 8099/225000 [27:10:37<497:13:44,  8.25s/it]
  4%|▎         | 8100/225000 [27:10:48<562:30:51,  9.34s/it]
                                                            
{'loss': '0.72', 'grad_norm': '0.8949', 'learning_rate': '4.047e-05', 'epoch': '0.036', 'num_input_tokens_seen': 49275640, 'train_runtime': '9.785e+04', 'train_tokens_per_second': '503.6'}

  4%|▎         | 8100/225000 [27:10:48<562:30:51,  9.34s/it]
  4%|▎         | 8101/225000 [27:10:55<509:36:55,  8.46s/it]
  4%|▎         | 8102/225000 [27:11:01<474:26:48,  7.87s/it]
  4%|▎         | 8103/225000 [27:11:14<553:44:37,  9.19s/it]
  4%|▎         | 8104/225000 [27:11:22<539:35:45,  8.96s/it]
  4%|▎         | 8105/225000 [27:11:26<446:36:10,  7.41s/it]
  4%|▎         | 8106/225000 [27:11:33<447:02:25,  7.42s/it]
  4%|▎         | 8107/225000 [27:11:40<437:34:32,  7.26s/it]
  4%|▎         | 8108/225000 [27:11:48<444:02:03,  7.37s/it]
  4%|▎         | 8109/225000 [27:11:57<485:18:38,  8.06s/it]
  4%|▎         | 8110/225000 [27:12:01<413:02:19,  6.86s/it]
  4%|▎         | 8111/225000 [27:12:12<479:37:13,  7.96s/it]
  4%|▎         | 8112/225000 [27:12:16<401:35:27,  6.67s/it]
  4%|▎         | 8113/225000 [27:12:20<365:38:07,  6.07s/it]
  4%|▎         | 8114/225000 [27:12:25<344:47:44,  5.72s/it]
  4%|▎         | 8115/225000 [27:12:42<550:38:28,  9.14s/it]
  4%|▎         | 8116/225000 [27:12:46<452:16:10,  7.51s/it]
  4%|▎         | 8117/225000 [27:12:54<457:45:31,  7.60s/it]
  4%|▎         | 8118/225000 [27:13:01<453:47:32,  7.53s/it]
  4%|▎         | 8119/225000 [27:13:09<453:01:13,  7.52s/it]
  4%|▎         | 8120/225000 [27:13:13<394:41:25,  6.55s/it]
  4%|▎         | 8121/225000 [27:13:26<507:35:24,  8.43s/it]
  4%|▎         | 8122/225000 [27:13:34<503:59:27,  8.37s/it]
  4%|▎         | 8123/225000 [27:13:39<450:30:56,  7.48s/it]
  4%|▎         | 8124/225000 [27:13:45<409:25:33,  6.80s/it]
  4%|▎         | 8125/225000 [27:13:52<413:08:21,  6.86s/it]
  4%|▎         | 8126/225000 [27:14:03<490:49:02,  8.15s/it]
  4%|▎         | 8127/225000 [27:14:08<440:22:30,  7.31s/it]
  4%|▎         | 8128/225000 [27:14:15<436:03:38,  7.24s/it]
  4%|▎         | 8129/225000 [27:14:21<405:16:06,  6.73s/it]
  4%|▎         | 8130/225000 [27:14:27<391:35:10,  6.50s/it]
  4%|▎         | 8131/225000 [27:14:41<533:36:09,  8.86s/it]
  4%|▎         | 8132/225000 [27:14:52<576:42:03,  9.57s/it]
  4%|▎         | 8133/225000 [27:14:57<479:49:42,  7.97s/it]
  4%|▎         | 8134/225000 [27:15:06<512:45:25,  8.51s/it]
  4%|▎         | 8135/225000 [27:15:14<492:58:33,  8.18s/it]
  4%|▎         | 8136/225000 [27:15:24<529:41:38,  8.79s/it]
  4%|▎         | 8137/225000 [27:15:32<518:57:50,  8.61s/it]
  4%|▎         | 8138/225000 [27:15:47<631:54:31, 10.49s/it]
  4%|▎         | 8139/225000 [27:15:52<526:20:38,  8.74s/it]
  4%|▎         | 8140/225000 [27:16:03<573:34:49,  9.52s/it]
  4%|▎         | 8141/225000 [27:16:08<490:43:08,  8.15s/it]
  4%|▎         | 8142/225000 [27:16:15<469:09:55,  7.79s/it]
  4%|▎         | 8143/225000 [27:16:29<574:30:13,  9.54s/it]
  4%|▎         | 8144/225000 [27:16:34<502:54:20,  8.35s/it]
  4%|▎         | 8145/225000 [27:16:39<442:16:06,  7.34s/it]
  4%|▎         | 8146/225000 [27:16:45<407:11:39,  6.76s/it]
  4%|▎         | 8147/225000 [27:16:51<397:52:00,  6.61s/it]
  4%|▎         | 8148/225000 [27:16:58<415:41:40,  6.90s/it]
  4%|▎         | 8149/225000 [27:17:10<497:32:03,  8.26s/it]
  4%|▎         | 8150/225000 [27:17:15<433:28:06,  7.20s/it]
                                                            
{'loss': '0.7505', 'grad_norm': '0.9045', 'learning_rate': '4.072e-05', 'epoch': '0.03622', 'num_input_tokens_seen': 49462384, 'train_runtime': '9.824e+04', 'train_tokens_per_second': '503.5'}

  4%|▎         | 8150/225000 [27:17:15<433:28:06,  7.20s/it]
  4%|▎         | 8151/225000 [27:17:20<396:40:09,  6.59s/it]
  4%|▎         | 8152/225000 [27:17:25<372:30:45,  6.18s/it]
  4%|▎         | 8153/225000 [27:17:37<483:06:45,  8.02s/it]
  4%|▎         | 8154/225000 [27:17:48<529:43:17,  8.79s/it]
  4%|▎         | 8155/225000 [27:17:53<471:08:17,  7.82s/it]
  4%|▎         | 8156/225000 [27:18:01<466:10:28,  7.74s/it]
  4%|▎         | 8157/225000 [27:18:12<532:50:26,  8.85s/it]
  4%|▎         | 8158/225000 [27:18:17<454:27:10,  7.54s/it]
  4%|▎         | 8159/225000 [27:18:22<405:02:03,  6.72s/it]
  4%|▎         | 8160/225000 [27:18:33<484:25:46,  8.04s/it]
  4%|▎         | 8161/225000 [27:18:39<454:23:52,  7.54s/it]
  4%|▎         | 8162/225000 [27:18:50<508:54:13,  8.45s/it]
  4%|▎         | 8163/225000 [27:18:54<432:41:34,  7.18s/it]
  4%|▎         | 8164/225000 [27:19:02<443:36:09,  7.36s/it]
  4%|▎         | 8165/225000 [27:19:10<462:08:12,  7.67s/it]
  4%|▎         | 8166/225000 [27:19:18<466:26:03,  7.74s/it]
  4%|▎         | 8167/225000 [27:19:28<505:43:30,  8.40s/it]
  4%|▎         | 8168/225000 [27:19:34<461:19:48,  7.66s/it]
  4%|▎         | 8169/225000 [27:19:40<434:02:26,  7.21s/it]
  4%|▎         | 8170/225000 [27:19:45<401:56:36,  6.67s/it]
  4%|▎         | 8171/225000 [27:19:57<494:37:51,  8.21s/it]
  4%|▎         | 8172/225000 [27:20:01<417:44:35,  6.94s/it]
  4%|▎         | 8173/225000 [27:20:06<382:29:23,  6.35s/it]
  4%|▎         | 8174/225000 [27:20:11<356:05:52,  5.91s/it]
  4%|▎         | 8175/225000 [27:20:22<453:49:22,  7.53s/it]
  4%|▎         | 8176/225000 [27:20:27<395:10:20,  6.56s/it]
  4%|▎         | 8177/225000 [27:20:34<403:45:56,  6.70s/it]
  4%|▎         | 8178/225000 [27:20:40<388:11:51,  6.45s/it]
  4%|▎         | 8179/225000 [27:20:45<372:12:24,  6.18s/it]
  4%|▎         | 8180/225000 [27:20:55<445:14:47,  7.39s/it]
  4%|▎         | 8181/225000 [27:21:05<483:08:28,  8.02s/it]
  4%|▎         | 8182/225000 [27:21:11<452:20:02,  7.51s/it]
  4%|▎         | 8183/225000 [27:21:21<487:12:09,  8.09s/it]
  4%|▎         | 8184/225000 [27:21:38<645:49:38, 10.72s/it]
  4%|▎         | 8185/225000 [27:21:42<538:15:13,  8.94s/it]
  4%|▎         | 8186/225000 [27:21:52<554:58:27,  9.21s/it]
  4%|▎         | 8187/225000 [27:22:01<545:03:20,  9.05s/it]
  4%|▎         | 8188/225000 [27:22:09<531:45:05,  8.83s/it]
  4%|▎         | 8189/225000 [27:22:19<544:21:23,  9.04s/it]
  4%|▎         | 8190/225000 [27:22:24<477:07:56,  7.92s/it]
  4%|▎         | 8191/225000 [27:22:31<466:51:39,  7.75s/it]
  4%|▎         | 8192/225000 [27:22:42<528:28:43,  8.78s/it]
  4%|▎         | 8193/225000 [27:22:52<541:29:25,  8.99s/it]
  4%|▎         | 8194/225000 [27:23:04<599:21:10,  9.95s/it]
  4%|▎         | 8195/225000 [27:23:12<563:25:18,  9.36s/it]
  4%|▎         | 8196/225000 [27:23:18<509:15:57,  8.46s/it]
  4%|▎         | 8197/225000 [27:23:26<487:48:55,  8.10s/it]
  4%|▎         | 8198/225000 [27:23:41<609:35:17, 10.12s/it]
  4%|▎         | 8199/225000 [27:23:46<517:58:48,  8.60s/it]
  4%|▎         | 8200/225000 [27:23:55<537:11:41,  8.92s/it]
                                                            
{'loss': '0.7776', 'grad_norm': '0.9011', 'learning_rate': '4.097e-05', 'epoch': '0.03644', 'num_input_tokens_seen': 49651520, 'train_runtime': '9.864e+04', 'train_tokens_per_second': '503.4'}

  4%|▎         | 8200/225000 [27:23:55<537:11:41,  8.92s/it]
  4%|▎         | 8201/225000 [27:24:04<531:10:44,  8.82s/it]
  4%|▎         | 8202/225000 [27:24:08<446:26:36,  7.41s/it]
  4%|▎         | 8203/225000 [27:24:16<448:01:19,  7.44s/it]
  4%|▎         | 8204/225000 [27:24:22<430:50:25,  7.15s/it]
  4%|▎         | 8205/225000 [27:24:26<372:34:38,  6.19s/it]
  4%|▎         | 8206/225000 [27:24:39<490:12:00,  8.14s/it]
  4%|▎         | 8207/225000 [27:24:43<417:27:42,  6.93s/it]
  4%|▎         | 8208/225000 [27:24:53<481:53:01,  8.00s/it]
  4%|▎         | 8209/225000 [27:24:59<444:44:42,  7.39s/it]
  4%|▎         | 8210/225000 [27:25:06<433:37:33,  7.20s/it]
  4%|▎         | 8211/225000 [27:25:14<449:27:58,  7.46s/it]
  4%|▎         | 8212/225000 [27:25:21<438:50:33,  7.29s/it]
  4%|▎         | 8213/225000 [27:25:29<457:28:58,  7.60s/it]
  4%|▎         | 8214/225000 [27:25:34<400:22:40,  6.65s/it]
  4%|▎         | 8215/225000 [27:25:38<350:40:27,  5.82s/it]
  4%|▎         | 8216/225000 [27:25:50<466:14:14,  7.74s/it]
  4%|▎         | 8217/225000 [27:25:54<408:02:22,  6.78s/it]
  4%|▎         | 8218/225000 [27:26:00<396:44:25,  6.59s/it]
  4%|▎         | 8219/225000 [27:26:10<444:43:33,  7.39s/it]
  4%|▎         | 8220/225000 [27:26:17<448:58:19,  7.46s/it]
  4%|▎         | 8221/225000 [27:26:22<405:34:47,  6.74s/it]
  4%|▎         | 8222/225000 [27:26:32<464:18:35,  7.71s/it]
  4%|▎         | 8223/225000 [27:26:37<408:34:54,  6.79s/it]
  4%|▎         | 8224/225000 [27:26:50<526:50:56,  8.75s/it]
  4%|▎         | 8225/225000 [27:26:56<468:33:19,  7.78s/it]
  4%|▎         | 8226/225000 [27:27:08<550:27:30,  9.14s/it]
  4%|▎         | 8227/225000 [27:27:15<507:54:34,  8.43s/it]
  4%|▎         | 8228/225000 [27:27:23<499:07:27,  8.29s/it]
  4%|▎         | 8229/225000 [27:27:30<474:18:52,  7.88s/it]
  4%|▎         | 8230/225000 [27:27:35<428:07:10,  7.11s/it]
  4%|▎         | 8231/225000 [27:27:47<509:16:18,  8.46s/it]
  4%|▎         | 8232/225000 [27:27:52<455:12:32,  7.56s/it]
  4%|▎         | 8233/225000 [27:27:58<414:04:00,  6.88s/it]
  4%|▎         | 8234/225000 [27:28:09<497:04:54,  8.26s/it]
  4%|▎         | 8235/225000 [27:28:22<586:57:44,  9.75s/it]
  4%|▎         | 8236/225000 [27:28:31<575:17:22,  9.55s/it]
  4%|▎         | 8237/225000 [27:28:45<643:35:27, 10.69s/it]
  4%|▎         | 8238/225000 [27:28:53<603:54:26, 10.03s/it]
  4%|▎         | 8239/225000 [27:29:06<654:07:59, 10.86s/it]
  4%|▎         | 8240/225000 [27:29:10<534:54:03,  8.88s/it]
  4%|▎         | 8241/225000 [27:29:16<483:08:06,  8.02s/it]
  4%|▎         | 8242/225000 [27:29:21<429:38:21,  7.14s/it]
  4%|▎         | 8243/225000 [27:29:26<394:01:40,  6.54s/it]
  4%|▎         | 8244/225000 [27:29:33<389:42:04,  6.47s/it]
  4%|▎         | 8245/225000 [27:29:39<379:29:35,  6.30s/it]
  4%|▎         | 8246/225000 [27:29:50<467:41:42,  7.77s/it]
  4%|▎         | 8247/225000 [27:29:56<440:41:11,  7.32s/it]
  4%|▎         | 8248/225000 [27:30:04<452:48:32,  7.52s/it]
  4%|▎         | 8249/225000 [27:30:14<498:07:50,  8.27s/it]
  4%|▎         | 8250/225000 [27:30:24<518:20:45,  8.61s/it]
                                                            
{'loss': '0.7362', 'grad_norm': '1.126', 'learning_rate': '4.122e-05', 'epoch': '0.03667', 'num_input_tokens_seen': 49835208, 'train_runtime': '9.902e+04', 'train_tokens_per_second': '503.3'}

  4%|▎         | 8250/225000 [27:30:24<518:20:45,  8.61s/it]
  4%|▎         | 8251/225000 [27:30:28<452:05:41,  7.51s/it]
  4%|▎         | 8252/225000 [27:30:35<429:29:29,  7.13s/it]
  4%|▎         | 8253/225000 [27:30:48<538:11:26,  8.94s/it]
  4%|▎         | 8254/225000 [27:30:53<476:55:51,  7.92s/it]
  4%|▎         | 8255/225000 [27:30:58<412:02:27,  6.84s/it]
  4%|▎         | 8256/225000 [27:31:04<400:07:35,  6.65s/it]
  4%|▎         | 8257/225000 [27:31:15<470:39:13,  7.82s/it]
  4%|▎         | 8258/225000 [27:31:21<444:42:54,  7.39s/it]
  4%|▎         | 8259/225000 [27:31:28<438:32:44,  7.28s/it]
  4%|▎         | 8260/225000 [27:31:37<465:52:00,  7.74s/it]
  4%|▎         | 8261/225000 [27:31:42<415:18:27,  6.90s/it]
  4%|▎         | 8262/225000 [27:31:48<411:17:45,  6.83s/it]
  4%|▎         | 8263/225000 [27:31:54<390:40:36,  6.49s/it]
  4%|▎         | 8264/225000 [27:32:05<477:28:57,  7.93s/it]
  4%|▎         | 8265/225000 [27:32:15<502:54:29,  8.35s/it]
  4%|▎         | 8266/225000 [27:32:27<570:19:44,  9.47s/it]
  4%|▎         | 8267/225000 [27:32:33<511:14:03,  8.49s/it]
  4%|▎         | 8268/225000 [27:32:45<581:50:43,  9.66s/it]
  4%|▎         | 8269/225000 [27:32:50<497:36:58,  8.27s/it]
  4%|▎         | 8270/225000 [27:32:58<488:03:40,  8.11s/it]
  4%|▎         | 8271/225000 [27:33:06<481:57:56,  8.01s/it]
  4%|▎         | 8272/225000 [27:33:15<498:12:23,  8.28s/it]
  4%|▎         | 8273/225000 [27:33:21<458:26:51,  7.62s/it]
  4%|▎         | 8274/225000 [27:33:29<470:19:17,  7.81s/it]
  4%|▎         | 8275/225000 [27:33:37<466:51:53,  7.76s/it]
  4%|▎         | 8276/225000 [27:33:45<469:17:50,  7.80s/it]
  4%|▎         | 8277/225000 [27:33:50<426:27:08,  7.08s/it]
  4%|▎         | 8278/225000 [27:33:57<429:49:52,  7.14s/it]
  4%|▎         | 8279/225000 [27:34:07<475:53:14,  7.91s/it]
  4%|▎         | 8280/225000 [27:34:17<513:46:13,  8.53s/it]
  4%|▎         | 8281/225000 [27:34:21<433:58:20,  7.21s/it]
  4%|▎         | 8282/225000 [27:34:32<504:19:29,  8.38s/it]
  4%|▎         | 8283/225000 [27:34:36<422:43:10,  7.02s/it]
  4%|▎         | 8284/225000 [27:34:44<441:59:56,  7.34s/it]
  4%|▎         | 8285/225000 [27:34:49<389:20:34,  6.47s/it]
  4%|▎         | 8286/225000 [27:34:53<356:46:10,  5.93s/it]
  4%|▎         | 8287/225000 [27:34:59<350:53:11,  5.83s/it]
  4%|▎         | 8288/225000 [27:35:09<437:13:20,  7.26s/it]
  4%|▎         | 8289/225000 [27:35:16<415:00:27,  6.89s/it]
  4%|▎         | 8290/225000 [27:35:27<489:18:13,  8.13s/it]
  4%|▎         | 8291/225000 [27:35:32<442:43:00,  7.35s/it]
  4%|▎         | 8292/225000 [27:35:46<560:01:11,  9.30s/it]
  4%|▎         | 8293/225000 [27:35:51<490:18:17,  8.15s/it]
  4%|▎         | 8294/225000 [27:35:56<435:06:45,  7.23s/it]
  4%|▎         | 8295/225000 [27:36:10<545:51:48,  9.07s/it]
  4%|▎         | 8296/225000 [27:36:16<495:00:35,  8.22s/it]
  4%|▎         | 8297/225000 [27:36:22<444:45:04,  7.39s/it]
  4%|▎         | 8298/225000 [27:36:32<503:19:11,  8.36s/it]
  4%|▎         | 8299/225000 [27:36:42<523:40:29,  8.70s/it]
  4%|▎         | 8300/225000 [27:36:47<469:00:41,  7.79s/it]
                                                            
{'loss': '0.7227', 'grad_norm': '0.9048', 'learning_rate': '4.147e-05', 'epoch': '0.03689', 'num_input_tokens_seen': 50016160, 'train_runtime': '9.941e+04', 'train_tokens_per_second': '503.1'}

  4%|▎         | 8300/225000 [27:36:47<469:00:41,  7.79s/it]
  4%|▎         | 8301/225000 [27:36:56<480:43:25,  7.99s/it]
  4%|▎         | 8302/225000 [27:37:00<421:11:42,  7.00s/it]
  4%|▎         | 8303/225000 [27:37:17<591:28:44,  9.83s/it]
  4%|▎         | 8304/225000 [27:37:23<527:12:31,  8.76s/it]
  4%|▎         | 8305/225000 [27:37:35<580:21:38,  9.64s/it]
  4%|▎         | 8306/225000 [27:37:40<500:15:58,  8.31s/it]
  4%|▎         | 8307/225000 [27:37:49<515:38:29,  8.57s/it]
  4%|▎         | 8308/225000 [27:37:54<443:13:30,  7.36s/it]
  4%|▎         | 8309/225000 [27:37:59<410:51:29,  6.83s/it]
  4%|▎         | 8310/225000 [27:38:05<396:02:56,  6.58s/it]
  4%|▎         | 8311/225000 [27:38:19<528:18:54,  8.78s/it]
  4%|▎         | 8312/225000 [27:38:25<475:20:39,  7.90s/it]
  4%|▎         | 8313/225000 [27:38:36<537:07:38,  8.92s/it]
  4%|▎         | 8314/225000 [27:38:46<543:12:38,  9.02s/it]
  4%|▎         | 8315/225000 [27:38:50<457:51:30,  7.61s/it]
  4%|▎         | 8316/225000 [27:38:56<430:51:58,  7.16s/it]
  4%|▎         | 8317/225000 [27:39:09<538:51:03,  8.95s/it]
  4%|▎         | 8318/225000 [27:39:14<465:54:04,  7.74s/it]
  4%|▎         | 8319/225000 [27:39:22<473:12:18,  7.86s/it]
  4%|▎         | 8320/225000 [27:39:27<425:39:45,  7.07s/it]
  4%|▎         | 8321/225000 [27:39:33<405:38:17,  6.74s/it]
  4%|▎         | 8322/225000 [27:39:38<372:56:03,  6.20s/it]
  4%|▎         | 8323/225000 [27:39:44<357:14:56,  5.94s/it]
  4%|▎         | 8324/225000 [27:39:50<371:00:47,  6.16s/it]
  4%|▎         | 8325/225000 [27:39:56<358:02:49,  5.95s/it]
  4%|▎         | 8326/225000 [27:40:06<432:01:29,  7.18s/it]
  4%|▎         | 8327/225000 [27:40:15<474:25:45,  7.88s/it]
  4%|▎         | 8328/225000 [27:40:27<549:38:09,  9.13s/it]
  4%|▎         | 8329/225000 [27:40:32<463:46:41,  7.71s/it]
  4%|▎         | 8330/225000 [27:40:39<445:50:07,  7.41s/it]
  4%|▎         | 8331/225000 [27:40:44<408:15:48,  6.78s/it]
  4%|▎         | 8332/225000 [27:40:57<526:15:27,  8.74s/it]
  4%|▎         | 8333/225000 [27:41:03<479:51:53,  7.97s/it]
  4%|▎         | 8334/225000 [27:41:14<523:49:04,  8.70s/it]
  4%|▎         | 8335/225000 [27:41:20<478:08:09,  7.94s/it]
  4%|▎         | 8336/225000 [27:41:26<438:07:40,  7.28s/it]
  4%|▎         | 8337/225000 [27:41:38<525:24:38,  8.73s/it]
  4%|▎         | 8338/225000 [27:41:48<544:33:50,  9.05s/it]
  4%|▎         | 8339/225000 [27:42:01<629:48:27, 10.46s/it]
  4%|▎         | 8340/225000 [27:42:08<554:02:49,  9.21s/it]
  4%|▎         | 8341/225000 [27:42:12<470:57:41,  7.83s/it]
  4%|▎         | 8342/225000 [27:42:18<434:41:40,  7.22s/it]
  4%|▎         | 8343/225000 [27:42:26<441:52:36,  7.34s/it]
  4%|▎         | 8344/225000 [27:42:37<517:42:41,  8.60s/it]
  4%|▎         | 8345/225000 [27:42:44<481:31:20,  8.00s/it]
  4%|▎         | 8346/225000 [27:42:54<516:59:00,  8.59s/it]
  4%|▎         | 8347/225000 [27:42:58<441:04:04,  7.33s/it]
  4%|▎         | 8348/225000 [27:43:11<539:27:22,  8.96s/it]
  4%|▎         | 8349/225000 [27:43:18<500:39:43,  8.32s/it]
  4%|▎         | 8350/225000 [27:43:25<474:02:24,  7.88s/it]
                                                            
{'loss': '0.7681', 'grad_norm': '0.9323', 'learning_rate': '4.172e-05', 'epoch': '0.03711', 'num_input_tokens_seen': 50206488, 'train_runtime': '9.981e+04', 'train_tokens_per_second': '503'}

  4%|▎         | 8350/225000 [27:43:25<474:02:24,  7.88s/it]
  4%|▎         | 8351/225000 [27:43:31<443:10:05,  7.36s/it]
  4%|▎         | 8352/225000 [27:43:36<403:52:51,  6.71s/it]
  4%|▎         | 8353/225000 [27:43:48<494:49:56,  8.22s/it]
  4%|▎         | 8354/225000 [27:43:54<454:25:39,  7.55s/it]
  4%|▎         | 8355/225000 [27:44:04<499:03:53,  8.29s/it]
  4%|▎         | 8356/225000 [27:44:09<439:23:15,  7.30s/it]
  4%|▎         | 8357/225000 [27:44:18<478:41:24,  7.95s/it]
  4%|▎         | 8358/225000 [27:44:27<499:00:48,  8.29s/it]
  4%|▎         | 8359/225000 [27:44:33<453:03:27,  7.53s/it]
  4%|▎         | 8360/225000 [27:44:39<418:20:16,  6.95s/it]
  4%|▎         | 8361/225000 [27:44:46<428:51:09,  7.13s/it]
  4%|▎         | 8362/225000 [27:45:04<621:36:51, 10.33s/it]
  4%|▎         | 8363/225000 [27:45:16<648:21:01, 10.77s/it]
  4%|▎         | 8364/225000 [27:45:25<623:37:38, 10.36s/it]
  4%|▎         | 8365/225000 [27:45:34<592:34:55,  9.85s/it]
  4%|▎         | 8366/225000 [27:45:40<534:36:12,  8.88s/it]
  4%|▎         | 8367/225000 [27:45:52<582:57:06,  9.69s/it]
  4%|▎         | 8368/225000 [27:45:59<533:02:33,  8.86s/it]
  4%|▎         | 8369/225000 [27:46:05<485:37:13,  8.07s/it]
  4%|▎         | 8370/225000 [27:46:12<467:17:35,  7.77s/it]
  4%|▎         | 8371/225000 [27:46:19<440:50:03,  7.33s/it]
  4%|▎         | 8372/225000 [27:46:29<500:48:32,  8.32s/it]
  4%|▎         | 8373/225000 [27:46:39<521:42:20,  8.67s/it]
  4%|▎         | 8374/225000 [27:46:49<549:55:55,  9.14s/it]
  4%|▎         | 8375/225000 [27:46:57<522:46:15,  8.69s/it]
  4%|▎         | 8376/225000 [27:47:05<522:07:08,  8.68s/it]
  4%|▎         | 8377/225000 [27:47:10<448:05:20,  7.45s/it]
  4%|▎         | 8378/225000 [27:47:20<490:06:45,  8.15s/it]
  4%|▎         | 8379/225000 [27:47:24<420:27:47,  6.99s/it]
  4%|▎         | 8380/225000 [27:47:30<397:04:48,  6.60s/it]
  4%|▎         | 8381/225000 [27:47:37<407:08:42,  6.77s/it]
  4%|▎         | 8382/225000 [27:47:47<476:00:09,  7.91s/it]
  4%|▎         | 8383/225000 [27:47:55<471:11:34,  7.83s/it]
  4%|▎         | 8384/225000 [27:48:02<463:12:48,  7.70s/it]
  4%|▎         | 8385/225000 [27:48:09<441:23:25,  7.34s/it]
  4%|▎         | 8386/225000 [27:48:19<499:52:01,  8.31s/it]
  4%|▎         | 8387/225000 [27:48:25<457:24:43,  7.60s/it]
  4%|▎         | 8388/225000 [27:48:30<398:43:35,  6.63s/it]
  4%|▎         | 8389/225000 [27:48:35<381:12:00,  6.34s/it]
  4%|▎         | 8390/225000 [27:48:44<415:41:16,  6.91s/it]
  4%|▎         | 8391/225000 [27:48:57<535:25:57,  8.90s/it]
  4%|▎         | 8392/225000 [27:49:04<506:47:23,  8.42s/it]
  4%|▎         | 8393/225000 [27:49:13<506:56:00,  8.43s/it]
  4%|▎         | 8394/225000 [27:49:21<506:18:34,  8.41s/it]
  4%|▎         | 8395/225000 [27:49:29<486:20:25,  8.08s/it]
  4%|▎         | 8396/225000 [27:49:37<497:54:57,  8.28s/it]
  4%|▎         | 8397/225000 [27:49:43<452:42:27,  7.52s/it]
  4%|▎         | 8398/225000 [27:49:52<476:47:50,  7.92s/it]
  4%|▎         | 8399/225000 [27:50:00<478:58:41,  7.96s/it]
  4%|▎         | 8400/225000 [27:50:11<536:18:42,  8.91s/it]
                                                            
{'loss': '0.746', 'grad_norm': '0.7868', 'learning_rate': '4.197e-05', 'epoch': '0.03733', 'num_input_tokens_seen': 50395840, 'train_runtime': '1.002e+05', 'train_tokens_per_second': '502.9'}

  4%|▎         | 8400/225000 [27:50:11<536:18:42,  8.91s/it]
  4%|▎         | 8401/225000 [27:50:16<464:25:54,  7.72s/it]
  4%|▎         | 8402/225000 [27:50:26<498:44:44,  8.29s/it]
  4%|▎         | 8403/225000 [27:50:31<451:43:38,  7.51s/it]
  4%|▎         | 8404/225000 [27:50:36<402:00:01,  6.68s/it]
  4%|▎         | 8405/225000 [27:50:41<368:33:36,  6.13s/it]
  4%|▎         | 8406/225000 [27:50:55<517:05:29,  8.59s/it]
  4%|▎         | 8407/225000 [27:51:03<498:08:20,  8.28s/it]
  4%|▎         | 8408/225000 [27:51:08<443:32:08,  7.37s/it]
  4%|▎         | 8409/225000 [27:51:19<502:21:15,  8.35s/it]
  4%|▎         | 8410/225000 [27:51:33<608:42:01, 10.12s/it]
  4%|▎         | 8411/225000 [27:51:41<572:18:30,  9.51s/it]
  4%|▎         | 8412/225000 [27:51:47<503:56:40,  8.38s/it]
  4%|▎         | 8413/225000 [27:51:59<570:57:25,  9.49s/it]
  4%|▎         | 8414/225000 [27:52:03<483:28:37,  8.04s/it]
  4%|▎         | 8415/225000 [27:52:14<533:55:56,  8.87s/it]
  4%|▎         | 8416/225000 [27:52:20<478:26:43,  7.95s/it]
  4%|▎         | 8417/225000 [27:52:26<433:56:12,  7.21s/it]
  4%|▎         | 8418/225000 [27:52:30<391:19:40,  6.50s/it]
  4%|▎         | 8419/225000 [27:52:45<540:40:31,  8.99s/it]
  4%|▎         | 8420/225000 [27:52:50<457:36:34,  7.61s/it]
  4%|▎         | 8421/225000 [27:52:55<422:49:43,  7.03s/it]
  4%|▎         | 8422/225000 [27:53:01<397:04:01,  6.60s/it]
  4%|▎         | 8423/225000 [27:53:11<460:15:26,  7.65s/it]
  4%|▎         | 8424/225000 [27:53:17<431:41:58,  7.18s/it]
  4%|▎         | 8425/225000 [27:53:28<505:48:17,  8.41s/it]
  4%|▎         | 8426/225000 [27:53:33<432:21:14,  7.19s/it]
  4%|▎         | 8427/225000 [27:53:39<420:40:53,  6.99s/it]
  4%|▎         | 8428/225000 [27:53:52<531:34:18,  8.84s/it]
  4%|▎         | 8429/225000 [27:53:56<444:49:41,  7.39s/it]
  4%|▎         | 8430/225000 [27:54:04<443:40:51,  7.38s/it]
  4%|▎         | 8431/225000 [27:54:07<375:18:52,  6.24s/it]
  4%|▎         | 8432/225000 [27:54:20<491:54:01,  8.18s/it]
  4%|▎         | 8433/225000 [27:54:25<435:31:39,  7.24s/it]
  4%|▎         | 8434/225000 [27:54:38<535:49:12,  8.91s/it]
  4%|▎         | 8435/225000 [27:54:44<479:45:34,  7.98s/it]
  4%|▎         | 8436/225000 [27:54:53<495:57:43,  8.24s/it]
  4%|▎         | 8437/225000 [27:54:57<431:07:23,  7.17s/it]
  4%|▍         | 8438/225000 [27:55:05<440:40:25,  7.33s/it]
  4%|▍         | 8439/225000 [27:55:14<476:18:57,  7.92s/it]
  4%|▍         | 8440/225000 [27:55:20<435:41:00,  7.24s/it]
  4%|▍         | 8441/225000 [27:55:28<443:47:04,  7.38s/it]
  4%|▍         | 8442/225000 [27:55:35<452:03:21,  7.51s/it]
  4%|▍         | 8443/225000 [27:55:46<516:12:58,  8.58s/it]
  4%|▍         | 8444/225000 [27:56:04<686:26:59, 11.41s/it]
  4%|▍         | 8445/225000 [27:56:08<550:57:38,  9.16s/it]
  4%|▍         | 8446/225000 [27:56:13<469:38:06,  7.81s/it]
  4%|▍         | 8447/225000 [27:56:21<464:48:06,  7.73s/it]
  4%|▍         | 8448/225000 [27:56:28<451:50:14,  7.51s/it]
  4%|▍         | 8449/225000 [27:56:32<400:59:02,  6.67s/it]
  4%|▍         | 8450/225000 [27:56:41<437:33:24,  7.27s/it]
                                                            
{'loss': '0.7175', 'grad_norm': '0.673', 'learning_rate': '4.222e-05', 'epoch': '0.03756', 'num_input_tokens_seen': 50584160, 'train_runtime': '1.006e+05', 'train_tokens_per_second': '502.8'}

  4%|▍         | 8450/225000 [27:56:41<437:33:24,  7.27s/it]
  4%|▍         | 8451/225000 [27:56:49<459:20:57,  7.64s/it]
  4%|▍         | 8452/225000 [27:56:58<476:07:32,  7.92s/it]
  4%|▍         | 8453/225000 [27:57:07<487:05:11,  8.10s/it]
  4%|▍         | 8454/225000 [27:57:14<477:53:28,  7.94s/it]
  4%|▍         | 8455/225000 [27:57:22<483:44:37,  8.04s/it]
  4%|▍         | 8456/225000 [27:57:34<542:49:48,  9.02s/it]
  4%|▍         | 8457/225000 [27:57:41<518:23:31,  8.62s/it]
  4%|▍         | 8458/225000 [27:57:52<556:49:49,  9.26s/it]
  4%|▍         | 8459/225000 [27:58:01<547:41:38,  9.11s/it]
  4%|▍         | 8460/225000 [27:58:13<597:14:20,  9.93s/it]
  4%|▍         | 8461/225000 [27:58:17<493:23:19,  8.20s/it]
  4%|▍         | 8462/225000 [27:58:29<572:06:29,  9.51s/it]
  4%|▍         | 8463/225000 [27:58:36<510:37:18,  8.49s/it]
  4%|▍         | 8464/225000 [27:58:41<454:18:23,  7.55s/it]
  4%|▍         | 8465/225000 [27:58:55<577:45:57,  9.61s/it]
  4%|▍         | 8466/225000 [27:59:00<484:24:11,  8.05s/it]
  4%|▍         | 8467/225000 [27:59:09<506:43:37,  8.42s/it]
  4%|▍         | 8468/225000 [27:59:20<553:17:46,  9.20s/it]
  4%|▍         | 8469/225000 [27:59:25<475:39:38,  7.91s/it]
  4%|▍         | 8470/225000 [27:59:38<561:53:44,  9.34s/it]
  4%|▍         | 8471/225000 [27:59:46<548:10:55,  9.11s/it]
  4%|▍         | 8472/225000 [27:59:53<498:05:12,  8.28s/it]
  4%|▍         | 8473/225000 [27:59:58<443:44:44,  7.38s/it]
  4%|▍         | 8474/225000 [28:00:05<434:15:09,  7.22s/it]
  4%|▍         | 8475/225000 [28:00:10<391:09:46,  6.50s/it]
  4%|▍         | 8476/225000 [28:00:14<351:53:53,  5.85s/it]
  4%|▍         | 8477/225000 [28:00:28<504:25:36,  8.39s/it]
  4%|▍         | 8478/225000 [28:00:33<446:46:45,  7.43s/it]
  4%|▍         | 8479/225000 [28:00:41<444:04:52,  7.38s/it]
  4%|▍         | 8480/225000 [28:00:49<466:30:01,  7.76s/it]
  4%|▍         | 8481/225000 [28:00:54<407:22:52,  6.77s/it]
  4%|▍         | 8482/225000 [28:01:04<474:09:27,  7.88s/it]
  4%|▍         | 8483/225000 [28:01:12<467:12:34,  7.77s/it]
  4%|▍         | 8484/225000 [28:01:17<431:20:59,  7.17s/it]
  4%|▍         | 8485/225000 [28:01:23<395:17:59,  6.57s/it]
  4%|▍         | 8486/225000 [28:01:28<371:58:15,  6.18s/it]
  4%|▍         | 8487/225000 [28:01:43<535:03:59,  8.90s/it]
  4%|▍         | 8488/225000 [28:01:49<480:11:37,  7.98s/it]
  4%|▍         | 8489/225000 [28:01:55<438:59:53,  7.30s/it]
  4%|▍         | 8490/225000 [28:02:00<398:57:47,  6.63s/it]
  4%|▍         | 8491/225000 [28:02:13<518:37:01,  8.62s/it]
  4%|▍         | 8492/225000 [28:02:18<449:43:28,  7.48s/it]
  4%|▍         | 8493/225000 [28:02:27<486:42:08,  8.09s/it]
  4%|▍         | 8494/225000 [28:02:32<431:22:27,  7.17s/it]
  4%|▍         | 8495/225000 [28:02:37<391:37:25,  6.51s/it]
  4%|▍         | 8496/225000 [28:02:42<364:01:54,  6.05s/it]
  4%|▍         | 8497/225000 [28:02:56<495:52:23,  8.25s/it]
  4%|▍         | 8498/225000 [28:03:00<429:29:56,  7.14s/it]
  4%|▍         | 8499/225000 [28:03:05<377:39:47,  6.28s/it]
  4%|▍         | 8500/225000 [28:03:09<344:37:21,  5.73s/it]
                                                            
{'loss': '0.7374', 'grad_norm': '0.8606', 'learning_rate': '4.247e-05', 'epoch': '0.03778', 'num_input_tokens_seen': 50769648, 'train_runtime': '1.01e+05', 'train_tokens_per_second': '502.7'}

  4%|▍         | 8500/225000 [28:03:09<344:37:21,  5.73s/it]
  4%|▍         | 8501/225000 [28:03:23<490:35:54,  8.16s/it]
  4%|▍         | 8502/225000 [28:03:27<426:02:37,  7.08s/it]
  4%|▍         | 8503/225000 [28:03:37<470:56:38,  7.83s/it]
  4%|▍         | 8504/225000 [28:03:42<424:23:11,  7.06s/it]
  4%|▍         | 8505/225000 [28:03:52<475:16:36,  7.90s/it]
  4%|▍         | 8506/225000 [28:03:59<459:26:49,  7.64s/it]
  4%|▍         | 8507/225000 [28:04:10<524:49:01,  8.73s/it]
  4%|▍         | 8508/225000 [28:04:21<552:20:30,  9.18s/it]
  4%|▍         | 8509/225000 [28:04:28<515:05:44,  8.57s/it]
  4%|▍         | 8510/225000 [28:04:34<476:24:32,  7.92s/it]
  4%|▍         | 8511/225000 [28:04:40<438:55:26,  7.30s/it]
  4%|▍         | 8512/225000 [28:04:46<408:57:29,  6.80s/it]
  4%|▍         | 8513/225000 [28:04:50<364:44:01,  6.07s/it]
  4%|▍         | 8514/225000 [28:04:58<395:44:09,  6.58s/it]
  4%|▍         | 8515/225000 [28:05:13<547:41:27,  9.11s/it]
  4%|▍         | 8516/225000 [28:05:18<473:16:19,  7.87s/it]
  4%|▍         | 8517/225000 [28:05:23<421:48:08,  7.01s/it]
  4%|▍         | 8518/225000 [28:05:37<548:13:36,  9.12s/it]
  4%|▍         | 8519/225000 [28:05:47<575:48:07,  9.58s/it]
  4%|▍         | 8520/225000 [28:05:53<494:25:43,  8.22s/it]
  4%|▍         | 8521/225000 [28:06:01<495:55:21,  8.25s/it]
  4%|▍         | 8522/225000 [28:06:08<473:07:10,  7.87s/it]
  4%|▍         | 8523/225000 [28:06:11<396:03:16,  6.59s/it]
  4%|▍         | 8524/225000 [28:06:17<378:11:33,  6.29s/it]
  4%|▍         | 8525/225000 [28:06:24<384:20:36,  6.39s/it]
  4%|▍         | 8526/225000 [28:06:34<457:55:33,  7.62s/it]
  4%|▍         | 8527/225000 [28:06:45<509:11:11,  8.47s/it]
  4%|▍         | 8528/225000 [28:06:49<443:09:57,  7.37s/it]
  4%|▍         | 8529/225000 [28:06:54<396:55:29,  6.60s/it]
  4%|▍         | 8530/225000 [28:07:07<511:23:28,  8.50s/it]
  4%|▍         | 8531/225000 [28:07:13<469:58:24,  7.82s/it]
  4%|▍         | 8532/225000 [28:07:19<423:49:00,  7.05s/it]
  4%|▍         | 8533/225000 [28:07:35<584:39:40,  9.72s/it]
  4%|▍         | 8534/225000 [28:07:38<473:46:10,  7.88s/it]
  4%|▍         | 8535/225000 [28:07:43<417:08:45,  6.94s/it]
  4%|▍         | 8536/225000 [28:07:48<391:32:41,  6.51s/it]
  4%|▍         | 8537/225000 [28:08:01<508:55:20,  8.46s/it]
  4%|▍         | 8538/225000 [28:08:07<452:00:00,  7.52s/it]
  4%|▍         | 8539/225000 [28:08:13<423:27:37,  7.04s/it]
  4%|▍         | 8540/225000 [28:08:25<521:59:24,  8.68s/it]
  4%|▍         | 8541/225000 [28:08:34<522:40:35,  8.69s/it]
  4%|▍         | 8542/225000 [28:08:42<511:58:29,  8.51s/it]
  4%|▍         | 8543/225000 [28:08:47<456:31:11,  7.59s/it]
  4%|▍         | 8544/225000 [28:08:53<416:55:43,  6.93s/it]
  4%|▍         | 8545/225000 [28:08:58<393:27:01,  6.54s/it]
  4%|▍         | 8546/225000 [28:09:08<446:07:07,  7.42s/it]
  4%|▍         | 8547/225000 [28:09:12<389:07:11,  6.47s/it]
  4%|▍         | 8548/225000 [28:09:17<362:46:31,  6.03s/it]
  4%|▍         | 8549/225000 [28:09:29<461:02:25,  7.67s/it]
  4%|▍         | 8550/225000 [28:09:35<444:27:56,  7.39s/it]
                                                            
{'loss': '0.7261', 'grad_norm': '0.961', 'learning_rate': '4.272e-05', 'epoch': '0.038', 'num_input_tokens_seen': 50951304, 'train_runtime': '1.014e+05', 'train_tokens_per_second': '502.6'}

  4%|▍         | 8550/225000 [28:09:35<444:27:56,  7.39s/it]
  4%|▍         | 8551/225000 [28:09:43<450:33:12,  7.49s/it]
  4%|▍         | 8552/225000 [28:09:56<555:39:31,  9.24s/it]
  4%|▍         | 8553/225000 [28:10:02<493:38:00,  8.21s/it]
  4%|▍         | 8554/225000 [28:10:10<479:48:19,  7.98s/it]
  4%|▍         | 8555/225000 [28:10:21<534:01:31,  8.88s/it]
  4%|▍         | 8556/225000 [28:10:25<456:28:07,  7.59s/it]
  4%|▍         | 8557/225000 [28:10:35<490:02:10,  8.15s/it]
  4%|▍         | 8558/225000 [28:10:40<433:59:07,  7.22s/it]
  4%|▍         | 8559/225000 [28:10:51<510:13:08,  8.49s/it]
  4%|▍         | 8560/225000 [28:10:59<499:53:32,  8.31s/it]
  4%|▍         | 8561/225000 [28:11:04<432:47:31,  7.20s/it]
  4%|▍         | 8562/225000 [28:11:21<610:15:38, 10.15s/it]
  4%|▍         | 8563/225000 [28:11:25<507:44:40,  8.45s/it]
  4%|▍         | 8564/225000 [28:11:34<511:59:46,  8.52s/it]
  4%|▍         | 8565/225000 [28:11:39<448:27:03,  7.46s/it]
  4%|▍         | 8566/225000 [28:11:50<510:29:10,  8.49s/it]
  4%|▍         | 8567/225000 [28:11:54<430:40:36,  7.16s/it]
  4%|▍         | 8568/225000 [28:12:05<502:48:28,  8.36s/it]
  4%|▍         | 8569/225000 [28:12:11<467:10:43,  7.77s/it]
  4%|▍         | 8570/225000 [28:12:23<531:59:17,  8.85s/it]
  4%|▍         | 8571/225000 [28:12:31<524:27:14,  8.72s/it]
  4%|▍         | 8572/225000 [28:12:36<450:48:28,  7.50s/it]
  4%|▍         | 8573/225000 [28:12:42<424:48:27,  7.07s/it]
  4%|▍         | 8574/225000 [28:12:54<512:18:52,  8.52s/it]
  4%|▍         | 8575/225000 [28:13:02<504:04:25,  8.38s/it]
  4%|▍         | 8576/225000 [28:13:09<472:40:23,  7.86s/it]
  4%|▍         | 8577/225000 [28:13:14<420:56:58,  7.00s/it]
  4%|▍         | 8578/225000 [28:13:20<406:54:55,  6.77s/it]
  4%|▍         | 8579/225000 [28:13:26<389:38:44,  6.48s/it]
  4%|▍         | 8580/225000 [28:13:44<595:59:17,  9.91s/it]
  4%|▍         | 8581/225000 [28:13:47<486:58:00,  8.10s/it]
  4%|▍         | 8582/225000 [28:13:55<484:34:54,  8.06s/it]
  4%|▍         | 8583/225000 [28:14:04<490:26:43,  8.16s/it]
  4%|▍         | 8584/225000 [28:14:10<453:28:10,  7.54s/it]
  4%|▍         | 8585/225000 [28:14:17<454:46:36,  7.57s/it]
  4%|▍         | 8586/225000 [28:14:27<496:46:47,  8.26s/it]
  4%|▍         | 8587/225000 [28:14:42<606:23:57, 10.09s/it]
  4%|▍         | 8588/225000 [28:14:55<657:49:03, 10.94s/it]
  4%|▍         | 8589/225000 [28:15:00<558:49:36,  9.30s/it]
  4%|▍         | 8590/225000 [28:15:06<493:23:15,  8.21s/it]
  4%|▍         | 8591/225000 [28:15:10<427:57:32,  7.12s/it]
  4%|▍         | 8592/225000 [28:15:23<522:23:13,  8.69s/it]
  4%|▍         | 8593/225000 [28:15:28<469:37:28,  7.81s/it]
  4%|▍         | 8594/225000 [28:15:33<412:17:06,  6.86s/it]
  4%|▍         | 8595/225000 [28:15:44<491:21:13,  8.17s/it]
  4%|▍         | 8596/225000 [28:15:49<426:30:58,  7.10s/it]
  4%|▍         | 8597/225000 [28:15:53<380:33:33,  6.33s/it]
  4%|▍         | 8598/225000 [28:16:06<487:14:17,  8.11s/it]
  4%|▍         | 8599/225000 [28:16:17<538:19:27,  8.96s/it]
  4%|▍         | 8600/225000 [28:16:26<540:34:45,  8.99s/it]
                                                            
{'loss': '0.7237', 'grad_norm': '0.8452', 'learning_rate': '4.297e-05', 'epoch': '0.03822', 'num_input_tokens_seen': 51146136, 'train_runtime': '1.018e+05', 'train_tokens_per_second': '502.5'}

  4%|▍         | 8600/225000 [28:16:26<540:34:45,  8.99s/it]
  4%|▍         | 8601/225000 [28:16:34<521:03:09,  8.67s/it]
  4%|▍         | 8602/225000 [28:16:39<458:10:56,  7.62s/it]
  4%|▍         | 8603/225000 [28:16:44<409:39:06,  6.82s/it]
  4%|▍         | 8604/225000 [28:16:50<401:13:39,  6.67s/it]
  4%|▍         | 8605/225000 [28:17:00<454:42:34,  7.56s/it]
  4%|▍         | 8606/225000 [28:17:11<516:30:12,  8.59s/it]
  4%|▍         | 8607/225000 [28:17:15<445:16:01,  7.41s/it]
  4%|▍         | 8608/225000 [28:17:20<402:45:17,  6.70s/it]
  4%|▍         | 8609/225000 [28:17:32<495:26:08,  8.24s/it]
  4%|▍         | 8610/225000 [28:17:37<440:52:35,  7.33s/it]
  4%|▍         | 8611/225000 [28:17:50<525:37:25,  8.74s/it]
  4%|▍         | 8612/225000 [28:17:54<456:16:56,  7.59s/it]
  4%|▍         | 8613/225000 [28:18:05<515:53:23,  8.58s/it]
  4%|▍         | 8614/225000 [28:18:14<513:39:58,  8.55s/it]
  4%|▍         | 8615/225000 [28:18:18<439:45:26,  7.32s/it]
  4%|▍         | 8616/225000 [28:18:28<478:22:00,  7.96s/it]
  4%|▍         | 8617/225000 [28:18:35<466:15:09,  7.76s/it]
  4%|▍         | 8618/225000 [28:18:46<531:46:35,  8.85s/it]
  4%|▍         | 8619/225000 [28:18:53<493:28:42,  8.21s/it]
  4%|▍         | 8620/225000 [28:19:00<464:21:03,  7.73s/it]
  4%|▍         | 8621/225000 [28:19:15<607:37:56, 10.11s/it]
  4%|▍         | 8622/225000 [28:19:19<489:35:31,  8.15s/it]
  4%|▍         | 8623/225000 [28:19:24<435:58:28,  7.25s/it]
  4%|▍         | 8624/225000 [28:19:30<405:58:28,  6.75s/it]
  4%|▍         | 8625/225000 [28:19:38<436:59:27,  7.27s/it]
  4%|▍         | 8626/225000 [28:19:45<436:03:44,  7.26s/it]
  4%|▍         | 8627/225000 [28:19:56<495:21:29,  8.24s/it]
  4%|▍         | 8628/225000 [28:20:04<498:04:29,  8.29s/it]
  4%|▍         | 8629/225000 [28:20:09<427:47:20,  7.12s/it]
  4%|▍         | 8630/225000 [28:20:13<380:44:45,  6.33s/it]
  4%|▍         | 8631/225000 [28:20:26<495:17:49,  8.24s/it]
  4%|▍         | 8632/225000 [28:20:32<449:47:42,  7.48s/it]
  4%|▍         | 8633/225000 [28:20:37<403:05:26,  6.71s/it]
  4%|▍         | 8634/225000 [28:20:51<544:17:40,  9.06s/it]
  4%|▍         | 8635/225000 [28:20:55<454:05:02,  7.56s/it]
  4%|▍         | 8636/225000 [28:21:00<405:10:30,  6.74s/it]
  4%|▍         | 8637/225000 [28:21:14<535:11:03,  8.90s/it]
  4%|▍         | 8638/225000 [28:21:18<449:25:14,  7.48s/it]
  4%|▍         | 8639/225000 [28:21:27<481:48:13,  8.02s/it]
  4%|▍         | 8640/225000 [28:21:36<501:01:23,  8.34s/it]
  4%|▍         | 8641/225000 [28:21:43<469:04:10,  7.80s/it]
  4%|▍         | 8642/225000 [28:21:51<468:10:49,  7.79s/it]
  4%|▍         | 8643/225000 [28:22:02<525:18:17,  8.74s/it]
  4%|▍         | 8644/225000 [28:22:13<567:17:26,  9.44s/it]
  4%|▍         | 8645/225000 [28:22:20<527:40:55,  8.78s/it]
  4%|▍         | 8646/225000 [28:22:25<468:03:48,  7.79s/it]
  4%|▍         | 8647/225000 [28:22:36<513:07:37,  8.54s/it]
  4%|▍         | 8648/225000 [28:22:40<437:56:30,  7.29s/it]
  4%|▍         | 8649/225000 [28:22:44<383:03:33,  6.37s/it]
  4%|▍         | 8650/225000 [28:22:49<358:58:06,  5.97s/it]
                                                            
{'loss': '0.7046', 'grad_norm': '0.7132', 'learning_rate': '4.322e-05', 'epoch': '0.03844', 'num_input_tokens_seen': 51335664, 'train_runtime': '1.022e+05', 'train_tokens_per_second': '502.5'}

  4%|▍         | 8650/225000 [28:22:49<358:58:06,  5.97s/it]
  4%|▍         | 8651/225000 [28:23:00<449:52:15,  7.49s/it]
  4%|▍         | 8652/225000 [28:23:15<577:31:11,  9.61s/it]
  4%|▍         | 8653/225000 [28:23:27<620:32:55, 10.33s/it]
  4%|▍         | 8654/225000 [28:23:38<627:08:14, 10.44s/it]
  4%|▍         | 8655/225000 [28:23:44<558:23:30,  9.29s/it]
  4%|▍         | 8656/225000 [28:23:54<572:53:15,  9.53s/it]
  4%|▍         | 8657/225000 [28:23:59<492:16:50,  8.19s/it]
  4%|▍         | 8658/225000 [28:24:06<470:34:04,  7.83s/it]
  4%|▍         | 8659/225000 [28:24:12<425:18:26,  7.08s/it]
  4%|▍         | 8660/225000 [28:24:18<402:07:22,  6.69s/it]
  4%|▍         | 8661/225000 [28:24:24<400:53:02,  6.67s/it]
  4%|▍         | 8662/225000 [28:24:33<438:09:32,  7.29s/it]
  4%|▍         | 8663/225000 [28:24:42<471:11:23,  7.84s/it]
  4%|▍         | 8664/225000 [28:24:50<474:45:24,  7.90s/it]
  4%|▍         | 8665/225000 [28:24:57<455:46:14,  7.58s/it]
  4%|▍         | 8666/225000 [28:25:09<543:11:34,  9.04s/it]
  4%|▍         | 8667/225000 [28:25:14<466:01:00,  7.75s/it]
  4%|▍         | 8668/225000 [28:25:23<481:38:18,  8.01s/it]
  4%|▍         | 8669/225000 [28:25:31<492:07:16,  8.19s/it]
  4%|▍         | 8670/225000 [28:25:35<418:32:01,  6.96s/it]
  4%|▍         | 8671/225000 [28:25:44<441:58:45,  7.36s/it]
  4%|▍         | 8672/225000 [28:25:51<448:53:39,  7.47s/it]
  4%|▍         | 8673/225000 [28:26:03<528:29:42,  8.79s/it]
  4%|▍         | 8674/225000 [28:26:08<451:14:27,  7.51s/it]
  4%|▍         | 8675/225000 [28:26:14<422:18:43,  7.03s/it]
  4%|▍         | 8676/225000 [28:26:27<525:38:54,  8.75s/it]
  4%|▍         | 8677/225000 [28:26:33<477:35:49,  7.95s/it]
  4%|▍         | 8678/225000 [28:26:41<481:36:46,  8.01s/it]
  4%|▍         | 8679/225000 [28:26:48<473:00:23,  7.87s/it]
  4%|▍         | 8680/225000 [28:26:57<480:34:21,  8.00s/it]
  4%|▍         | 8681/225000 [28:27:02<433:54:25,  7.22s/it]
  4%|▍         | 8682/225000 [28:27:10<453:29:33,  7.55s/it]
  4%|▍         | 8683/225000 [28:27:17<437:35:44,  7.28s/it]
  4%|▍         | 8684/225000 [28:27:24<434:27:58,  7.23s/it]
  4%|▍         | 8685/225000 [28:27:29<393:33:53,  6.55s/it]
  4%|▍         | 8686/225000 [28:27:35<380:41:21,  6.34s/it]
  4%|▍         | 8687/225000 [28:27:41<377:25:13,  6.28s/it]
  4%|▍         | 8688/225000 [28:27:54<491:04:43,  8.17s/it]
  4%|▍         | 8689/225000 [28:28:00<464:01:02,  7.72s/it]
  4%|▍         | 8690/225000 [28:28:12<533:21:44,  8.88s/it]
  4%|▍         | 8691/225000 [28:28:19<494:58:35,  8.24s/it]
  4%|▍         | 8692/225000 [28:28:33<609:14:13, 10.14s/it]
  4%|▍         | 8693/225000 [28:28:38<516:09:50,  8.59s/it]
  4%|▍         | 8694/225000 [28:28:48<535:40:47,  8.92s/it]
  4%|▍         | 8695/225000 [28:28:57<538:26:59,  8.96s/it]
  4%|▍         | 8696/225000 [28:29:02<470:32:10,  7.83s/it]
  4%|▍         | 8697/225000 [28:29:15<563:31:48,  9.38s/it]
  4%|▍         | 8698/225000 [28:29:20<476:41:00,  7.93s/it]
  4%|▍         | 8699/225000 [28:29:28<489:09:30,  8.14s/it]
  4%|▍         | 8700/225000 [28:29:33<436:16:34,  7.26s/it]
                                                            
{'loss': '0.7317', 'grad_norm': '0.8314', 'learning_rate': '4.347e-05', 'epoch': '0.03867', 'num_input_tokens_seen': 51525424, 'train_runtime': '1.026e+05', 'train_tokens_per_second': '502.3'}

  4%|▍         | 8700/225000 [28:29:33<436:16:34,  7.26s/it]
  4%|▍         | 8701/225000 [28:29:43<485:45:43,  8.08s/it]
  4%|▍         | 8702/225000 [28:29:50<462:47:38,  7.70s/it]
  4%|▍         | 8703/225000 [28:29:57<447:34:35,  7.45s/it]
  4%|▍         | 8704/225000 [28:30:07<493:21:54,  8.21s/it]
  4%|▍         | 8705/225000 [28:30:11<417:56:10,  6.96s/it]
  4%|▍         | 8706/225000 [28:30:21<468:41:19,  7.80s/it]
  4%|▍         | 8707/225000 [28:30:26<426:17:58,  7.10s/it]
  4%|▍         | 8708/225000 [28:30:32<391:12:59,  6.51s/it]
  4%|▍         | 8709/225000 [28:30:39<402:32:29,  6.70s/it]
  4%|▍         | 8710/225000 [28:30:44<372:24:51,  6.20s/it]
  4%|▍         | 8711/225000 [28:30:59<540:00:11,  8.99s/it]
  4%|▍         | 8712/225000 [28:31:05<477:30:33,  7.95s/it]
  4%|▍         | 8713/225000 [28:31:15<527:59:29,  8.79s/it]
  4%|▍         | 8714/225000 [28:31:21<473:10:00,  7.88s/it]
  4%|▍         | 8715/225000 [28:31:29<467:03:49,  7.77s/it]
  4%|▍         | 8716/225000 [28:31:38<500:04:49,  8.32s/it]
  4%|▍         | 8717/225000 [28:31:50<555:07:56,  9.24s/it]
  4%|▍         | 8718/225000 [28:31:54<466:24:01,  7.76s/it]
  4%|▍         | 8719/225000 [28:31:59<412:46:21,  6.87s/it]
  4%|▍         | 8720/225000 [28:32:09<472:43:18,  7.87s/it]
  4%|▍         | 8721/225000 [28:32:15<439:24:08,  7.31s/it]
  4%|▍         | 8722/225000 [28:32:22<431:40:01,  7.19s/it]
  4%|▍         | 8723/225000 [28:32:31<465:44:48,  7.75s/it]
  4%|▍         | 8724/225000 [28:32:36<419:43:38,  6.99s/it]
  4%|▍         | 8725/225000 [28:32:41<380:21:10,  6.33s/it]
  4%|▍         | 8726/225000 [28:32:46<361:07:45,  6.01s/it]
  4%|▍         | 8727/225000 [28:33:00<507:20:45,  8.45s/it]
  4%|▍         | 8728/225000 [28:33:06<452:54:21,  7.54s/it]
  4%|▍         | 8729/225000 [28:33:11<401:42:09,  6.69s/it]
  4%|▍         | 8730/225000 [28:33:18<408:20:21,  6.80s/it]
  4%|▍         | 8731/225000 [28:33:30<515:08:06,  8.57s/it]
  4%|▍         | 8732/225000 [28:33:40<530:05:21,  8.82s/it]
  4%|▍         | 8733/225000 [28:33:43<427:49:43,  7.12s/it]
  4%|▍         | 8734/225000 [28:33:47<375:23:38,  6.25s/it]
  4%|▍         | 8735/225000 [28:33:52<354:58:36,  5.91s/it]
  4%|▍         | 8736/225000 [28:34:07<508:02:10,  8.46s/it]
  4%|▍         | 8737/225000 [28:34:11<439:33:15,  7.32s/it]
  4%|▍         | 8738/225000 [28:34:24<536:53:56,  8.94s/it]
  4%|▍         | 8739/225000 [28:34:28<455:46:46,  7.59s/it]
  4%|▍         | 8740/225000 [28:34:40<522:47:09,  8.70s/it]
  4%|▍         | 8741/225000 [28:34:46<473:06:49,  7.88s/it]
  4%|▍         | 8742/225000 [28:34:52<440:38:42,  7.34s/it]
  4%|▍         | 8743/225000 [28:35:01<475:49:13,  7.92s/it]
  4%|▍         | 8744/225000 [28:35:11<520:33:55,  8.67s/it]
  4%|▍         | 8745/225000 [28:35:17<466:35:44,  7.77s/it]
  4%|▍         | 8746/225000 [28:35:23<427:59:57,  7.12s/it]
  4%|▍         | 8747/225000 [28:35:32<464:08:11,  7.73s/it]
  4%|▍         | 8748/225000 [28:35:41<481:55:40,  8.02s/it]
  4%|▍         | 8749/225000 [28:35:51<523:27:15,  8.71s/it]
  4%|▍         | 8750/225000 [28:35:57<472:40:37,  7.87s/it]
                                                            
{'loss': '0.7678', 'grad_norm': '1.027', 'learning_rate': '4.372e-05', 'epoch': '0.03889', 'num_input_tokens_seen': 51707352, 'train_runtime': '1.03e+05', 'train_tokens_per_second': '502.2'}

  4%|▍         | 8750/225000 [28:35:57<472:40:37,  7.87s/it]
  4%|▍         | 8751/225000 [28:36:02<424:41:51,  7.07s/it]
  4%|▍         | 8752/225000 [28:36:08<398:44:02,  6.64s/it]
  4%|▍         | 8753/225000 [28:36:16<428:51:06,  7.14s/it]
  4%|▍         | 8754/225000 [28:36:22<401:53:20,  6.69s/it]
  4%|▍         | 8755/225000 [28:36:27<380:23:29,  6.33s/it]
  4%|▍         | 8756/225000 [28:36:32<356:37:55,  5.94s/it]
  4%|▍         | 8757/225000 [28:36:37<340:06:33,  5.66s/it]
  4%|▍         | 8758/225000 [28:36:43<342:01:20,  5.69s/it]
  4%|▍         | 8759/225000 [28:36:50<368:12:30,  6.13s/it]
  4%|▍         | 8760/225000 [28:37:03<493:28:22,  8.22s/it]
  4%|▍         | 8761/225000 [28:37:08<441:24:40,  7.35s/it]
  4%|▍         | 8762/225000 [28:37:21<541:57:39,  9.02s/it]
  4%|▍         | 8763/225000 [28:37:27<473:16:47,  7.88s/it]
  4%|▍         | 8764/225000 [28:37:31<419:50:22,  6.99s/it]
  4%|▍         | 8765/225000 [28:37:43<499:38:11,  8.32s/it]
  4%|▍         | 8766/225000 [28:37:52<506:30:10,  8.43s/it]
  4%|▍         | 8767/225000 [28:38:02<546:52:30,  9.10s/it]
  4%|▍         | 8768/225000 [28:38:05<438:50:00,  7.31s/it]
  4%|▍         | 8769/225000 [28:38:15<475:19:01,  7.91s/it]
  4%|▍         | 8770/225000 [28:38:24<499:42:41,  8.32s/it]
  4%|▍         | 8771/225000 [28:38:33<517:37:57,  8.62s/it]
  4%|▍         | 8772/225000 [28:38:38<450:17:15,  7.50s/it]
  4%|▍         | 8773/225000 [28:38:48<485:02:21,  8.08s/it]
  4%|▍         | 8774/225000 [28:38:57<502:40:50,  8.37s/it]
  4%|▍         | 8775/225000 [28:39:03<457:12:34,  7.61s/it]
  4%|▍         | 8776/225000 [28:39:07<399:55:28,  6.66s/it]
  4%|▍         | 8777/225000 [28:39:13<387:45:16,  6.46s/it]
  4%|▍         | 8778/225000 [28:39:23<451:54:31,  7.52s/it]
  4%|▍         | 8779/225000 [28:39:28<402:35:17,  6.70s/it]
  4%|▍         | 8780/225000 [28:39:32<360:17:36,  6.00s/it]
  4%|▍         | 8781/225000 [28:39:37<346:22:12,  5.77s/it]
  4%|▍         | 8782/225000 [28:39:56<575:52:44,  9.59s/it]
  4%|▍         | 8783/225000 [28:40:04<545:43:38,  9.09s/it]
  4%|▍         | 8784/225000 [28:40:13<544:24:23,  9.06s/it]
  4%|▍         | 8785/225000 [28:40:18<483:47:47,  8.06s/it]
  4%|▍         | 8786/225000 [28:40:26<483:29:31,  8.05s/it]
  4%|▍         | 8787/225000 [28:40:34<470:21:39,  7.83s/it]
  4%|▍         | 8788/225000 [28:40:44<509:07:08,  8.48s/it]
  4%|▍         | 8789/225000 [28:40:49<448:55:45,  7.47s/it]
  4%|▍         | 8790/225000 [28:40:55<426:06:34,  7.09s/it]
  4%|▍         | 8791/225000 [28:41:15<660:02:58, 10.99s/it]
  4%|▍         | 8792/225000 [28:41:22<580:52:13,  9.67s/it]
  4%|▍         | 8793/225000 [28:41:29<544:09:37,  9.06s/it]
  4%|▍         | 8794/225000 [28:41:38<528:59:15,  8.81s/it]
  4%|▍         | 8795/225000 [28:41:48<554:30:44,  9.23s/it]
  4%|▍         | 8796/225000 [28:41:53<477:02:11,  7.94s/it]
  4%|▍         | 8797/225000 [28:42:01<488:57:17,  8.14s/it]
  4%|▍         | 8798/225000 [28:42:09<486:29:18,  8.10s/it]
  4%|▍         | 8799/225000 [28:42:14<430:44:51,  7.17s/it]
  4%|▍         | 8800/225000 [28:42:22<432:13:20,  7.20s/it]
                                                            
{'loss': '0.724', 'grad_norm': '1.057', 'learning_rate': '4.397e-05', 'epoch': '0.03911', 'num_input_tokens_seen': 51894824, 'train_runtime': '1.033e+05', 'train_tokens_per_second': '502.2'}

  4%|▍         | 8800/225000 [28:42:22<432:13:20,  7.20s/it]
  4%|▍         | 8801/225000 [28:42:39<605:20:00, 10.08s/it]
  4%|▍         | 8802/225000 [28:42:47<585:24:02,  9.75s/it]
  4%|▍         | 8803/225000 [28:42:53<500:25:22,  8.33s/it]
  4%|▍         | 8804/225000 [28:43:00<490:31:57,  8.17s/it]
  4%|▍         | 8805/225000 [28:43:11<527:44:46,  8.79s/it]
  4%|▍         | 8806/225000 [28:43:21<566:25:34,  9.43s/it]
  4%|▍         | 8807/225000 [28:43:33<606:53:12, 10.11s/it]
  4%|▍         | 8808/225000 [28:43:41<558:20:38,  9.30s/it]
  4%|▍         | 8809/225000 [28:43:51<578:34:45,  9.63s/it]
  4%|▍         | 8810/225000 [28:43:57<518:31:14,  8.63s/it]
  4%|▍         | 8811/225000 [28:44:07<530:18:56,  8.83s/it]
  4%|▍         | 8812/225000 [28:44:11<457:22:53,  7.62s/it]
  4%|▍         | 8813/225000 [28:44:16<410:52:10,  6.84s/it]
  4%|▍         | 8814/225000 [28:44:21<371:52:17,  6.19s/it]
  4%|▍         | 8815/225000 [28:44:36<522:34:02,  8.70s/it]
  4%|▍         | 8816/225000 [28:44:40<437:28:18,  7.28s/it]
  4%|▍         | 8817/225000 [28:44:50<496:33:40,  8.27s/it]
  4%|▍         | 8818/225000 [28:44:55<443:11:50,  7.38s/it]
  4%|▍         | 8819/225000 [28:45:01<418:26:02,  6.97s/it]
  4%|▍         | 8820/225000 [28:45:06<381:42:33,  6.36s/it]
  4%|▍         | 8821/225000 [28:45:16<448:27:02,  7.47s/it]
  4%|▍         | 8822/225000 [28:45:23<426:33:49,  7.10s/it]
  4%|▍         | 8823/225000 [28:45:31<442:53:39,  7.38s/it]
  4%|▍         | 8824/225000 [28:45:38<446:30:16,  7.44s/it]
  4%|▍         | 8825/225000 [28:45:52<559:05:27,  9.31s/it]
  4%|▍         | 8826/225000 [28:46:01<546:01:53,  9.09s/it]
  4%|▍         | 8827/225000 [28:46:08<512:46:11,  8.54s/it]
  4%|▍         | 8828/225000 [28:46:12<437:01:02,  7.28s/it]
  4%|▍         | 8829/225000 [28:46:21<467:18:33,  7.78s/it]
  4%|▍         | 8830/225000 [28:46:27<431:22:40,  7.18s/it]
  4%|▍         | 8831/225000 [28:46:33<403:40:43,  6.72s/it]
  4%|▍         | 8832/225000 [28:46:39<391:38:51,  6.52s/it]
  4%|▍         | 8833/225000 [28:46:48<448:13:12,  7.46s/it]
  4%|▍         | 8834/225000 [28:46:57<464:57:37,  7.74s/it]
  4%|▍         | 8835/225000 [28:47:08<532:18:58,  8.87s/it]
  4%|▍         | 8836/225000 [28:47:16<520:03:25,  8.66s/it]
  4%|▍         | 8837/225000 [28:47:25<516:31:47,  8.60s/it]
  4%|▍         | 8838/225000 [28:47:32<496:28:33,  8.27s/it]
  4%|▍         | 8839/225000 [28:47:47<613:51:08, 10.22s/it]
  4%|▍         | 8840/225000 [28:47:54<560:09:59,  9.33s/it]
  4%|▍         | 8841/225000 [28:47:59<473:05:30,  7.88s/it]
  4%|▍         | 8842/225000 [28:48:07<484:50:27,  8.07s/it]
  4%|▍         | 8843/225000 [28:48:18<524:42:23,  8.74s/it]
  4%|▍         | 8844/225000 [28:48:28<558:12:19,  9.30s/it]
  4%|▍         | 8845/225000 [28:48:40<601:55:07, 10.02s/it]
  4%|▍         | 8846/225000 [28:48:50<606:38:17, 10.10s/it]
  4%|▍         | 8847/225000 [28:48:58<571:08:39,  9.51s/it]
  4%|▍         | 8848/225000 [28:49:08<572:32:03,  9.54s/it]
  4%|▍         | 8849/225000 [28:49:12<477:51:02,  7.96s/it]
  4%|▍         | 8850/225000 [28:49:26<577:09:23,  9.61s/it]
                                                            
{'loss': '0.7588', 'grad_norm': '0.7446', 'learning_rate': '4.422e-05', 'epoch': '0.03933', 'num_input_tokens_seen': 52088536, 'train_runtime': '1.038e+05', 'train_tokens_per_second': '502'}

  4%|▍         | 8850/225000 [28:49:26<577:09:23,  9.61s/it]
  4%|▍         | 8851/225000 [28:49:29<466:34:36,  7.77s/it]
  4%|▍         | 8852/225000 [28:49:35<423:43:49,  7.06s/it]
  4%|▍         | 8853/225000 [28:49:41<411:43:24,  6.86s/it]
  4%|▍         | 8854/225000 [28:49:52<479:20:57,  7.98s/it]
  4%|▍         | 8855/225000 [28:49:57<439:21:16,  7.32s/it]
  4%|▍         | 8856/225000 [28:50:02<398:39:42,  6.64s/it]
  4%|▍         | 8857/225000 [28:50:15<505:34:01,  8.42s/it]
  4%|▍         | 8858/225000 [28:50:24<523:20:51,  8.72s/it]
  4%|▍         | 8859/225000 [28:50:38<618:24:31, 10.30s/it]
  4%|▍         | 8860/225000 [28:50:45<550:04:03,  9.16s/it]
  4%|▍         | 8861/225000 [28:50:52<508:58:53,  8.48s/it]
  4%|▍         | 8862/225000 [28:50:56<436:23:35,  7.27s/it]
  4%|▍         | 8863/225000 [28:51:05<467:40:30,  7.79s/it]
  4%|▍         | 8864/225000 [28:51:19<572:17:15,  9.53s/it]
  4%|▍         | 8865/225000 [28:51:26<536:36:10,  8.94s/it]
  4%|▍         | 8866/225000 [28:51:32<478:32:21,  7.97s/it]
  4%|▍         | 8867/225000 [28:51:41<494:27:57,  8.24s/it]
  4%|▍         | 8868/225000 [28:51:45<414:29:33,  6.90s/it]
  4%|▍         | 8869/225000 [28:51:51<406:00:09,  6.76s/it]
  4%|▍         | 8870/225000 [28:52:06<547:47:02,  9.12s/it]
  4%|▍         | 8871/225000 [28:52:11<471:12:54,  7.85s/it]
  4%|▍         | 8872/225000 [28:52:18<470:45:50,  7.84s/it]
  4%|▍         | 8873/225000 [28:52:23<407:58:33,  6.80s/it]
  4%|▍         | 8874/225000 [28:52:29<403:06:05,  6.71s/it]
  4%|▍         | 8875/225000 [28:52:42<509:22:38,  8.48s/it]
  4%|▍         | 8876/225000 [28:52:49<486:32:57,  8.10s/it]
  4%|▍         | 8877/225000 [28:52:59<508:24:19,  8.47s/it]
  4%|▍         | 8878/225000 [28:53:06<486:12:36,  8.10s/it]
  4%|▍         | 8879/225000 [28:53:13<464:13:54,  7.73s/it]
  4%|▍         | 8880/225000 [28:53:25<542:39:11,  9.04s/it]
  4%|▍         | 8881/225000 [28:53:35<573:43:12,  9.56s/it]
  4%|▍         | 8882/225000 [28:53:41<500:53:25,  8.34s/it]
  4%|▍         | 8883/225000 [28:53:46<434:01:16,  7.23s/it]
  4%|▍         | 8884/225000 [28:53:52<418:16:53,  6.97s/it]
  4%|▍         | 8885/225000 [28:54:01<462:06:45,  7.70s/it]
  4%|▍         | 8886/225000 [28:54:10<471:22:21,  7.85s/it]
  4%|▍         | 8887/225000 [28:54:14<402:18:18,  6.70s/it]
  4%|▍         | 8888/225000 [28:54:19<385:54:47,  6.43s/it]
  4%|▍         | 8889/225000 [28:54:33<511:04:57,  8.51s/it]
  4%|▍         | 8890/225000 [28:54:40<495:11:46,  8.25s/it]
  4%|▍         | 8891/225000 [28:54:50<522:55:26,  8.71s/it]
  4%|▍         | 8892/225000 [28:54:59<520:35:26,  8.67s/it]
  4%|▍         | 8893/225000 [28:55:07<514:28:18,  8.57s/it]
  4%|▍         | 8894/225000 [28:55:16<514:53:15,  8.58s/it]
  4%|▍         | 8895/225000 [28:55:23<483:55:50,  8.06s/it]
  4%|▍         | 8896/225000 [28:55:33<526:01:01,  8.76s/it]
  4%|▍         | 8897/225000 [28:55:40<487:14:51,  8.12s/it]
  4%|▍         | 8898/225000 [28:55:45<437:12:39,  7.28s/it]
  4%|▍         | 8899/225000 [28:55:49<388:22:42,  6.47s/it]
  4%|▍         | 8900/225000 [28:55:55<371:38:56,  6.19s/it]
                                                            
{'loss': '0.7615', 'grad_norm': '0.7851', 'learning_rate': '4.447e-05', 'epoch': '0.03956', 'num_input_tokens_seen': 52275256, 'train_runtime': '1.042e+05', 'train_tokens_per_second': '501.9'}

  4%|▍         | 8900/225000 [28:55:55<371:38:56,  6.19s/it]
  4%|▍         | 8901/225000 [28:56:04<423:36:43,  7.06s/it]
  4%|▍         | 8902/225000 [28:56:10<400:28:58,  6.67s/it]
  4%|▍         | 8903/225000 [28:56:22<489:59:04,  8.16s/it]
  4%|▍         | 8904/225000 [28:56:30<502:12:57,  8.37s/it]
  4%|▍         | 8905/225000 [28:56:35<436:44:54,  7.28s/it]
  4%|▍         | 8906/225000 [28:56:41<408:52:08,  6.81s/it]
  4%|▍         | 8907/225000 [28:56:49<441:03:09,  7.35s/it]
  4%|▍         | 8908/225000 [28:56:57<438:23:37,  7.30s/it]
  4%|▍         | 8909/225000 [28:57:01<388:42:12,  6.48s/it]
  4%|▍         | 8910/225000 [28:57:06<363:27:47,  6.06s/it]
  4%|▍         | 8911/225000 [28:57:18<464:23:36,  7.74s/it]
  4%|▍         | 8912/225000 [28:57:24<438:53:41,  7.31s/it]
  4%|▍         | 8913/225000 [28:57:28<374:04:47,  6.23s/it]
  4%|▍         | 8914/225000 [28:57:38<444:15:15,  7.40s/it]
  4%|▍         | 8915/225000 [28:57:43<396:13:07,  6.60s/it]
  4%|▍         | 8916/225000 [28:57:49<383:19:08,  6.39s/it]
  4%|▍         | 8917/225000 [28:58:02<500:05:08,  8.33s/it]
  4%|▍         | 8918/225000 [28:58:06<421:29:56,  7.02s/it]
  4%|▍         | 8919/225000 [28:58:15<462:21:28,  7.70s/it]
  4%|▍         | 8920/225000 [28:58:22<445:40:23,  7.43s/it]
  4%|▍         | 8921/225000 [28:58:28<433:48:37,  7.23s/it]
  4%|▍         | 8922/225000 [28:58:40<512:39:44,  8.54s/it]
  4%|▍         | 8923/225000 [28:58:45<450:08:19,  7.50s/it]
  4%|▍         | 8924/225000 [28:58:55<487:08:46,  8.12s/it]
  4%|▍         | 8925/225000 [28:59:01<457:20:12,  7.62s/it]
  4%|▍         | 8926/225000 [28:59:10<489:46:33,  8.16s/it]
  4%|▍         | 8927/225000 [28:59:17<459:07:58,  7.65s/it]
  4%|▍         | 8928/225000 [28:59:27<499:50:03,  8.33s/it]
  4%|▍         | 8929/225000 [28:59:34<470:01:35,  7.83s/it]
  4%|▍         | 8930/225000 [28:59:45<535:48:23,  8.93s/it]
  4%|▍         | 8931/225000 [28:59:52<503:49:38,  8.39s/it]
  4%|▍         | 8932/225000 [28:59:59<484:43:13,  8.08s/it]
  4%|▍         | 8933/225000 [29:00:07<466:47:54,  7.78s/it]
  4%|▍         | 8934/225000 [29:00:15<470:27:54,  7.84s/it]
  4%|▍         | 8935/225000 [29:00:30<606:03:33, 10.10s/it]
  4%|▍         | 8936/225000 [29:00:37<545:47:59,  9.09s/it]
  4%|▍         | 8937/225000 [29:00:44<506:51:09,  8.45s/it]
  4%|▍         | 8938/225000 [29:00:49<455:52:21,  7.60s/it]
  4%|▍         | 8939/225000 [29:00:54<410:04:59,  6.83s/it]
  4%|▍         | 8940/225000 [29:01:00<385:14:34,  6.42s/it]
  4%|▍         | 8941/225000 [29:01:11<479:51:05,  8.00s/it]
  4%|▍         | 8942/225000 [29:01:15<404:27:08,  6.74s/it]
  4%|▍         | 8943/225000 [29:01:25<450:47:33,  7.51s/it]
  4%|▍         | 8944/225000 [29:01:28<386:13:40,  6.44s/it]
  4%|▍         | 8945/225000 [29:01:41<493:02:54,  8.22s/it]
  4%|▍         | 8946/225000 [29:01:48<473:25:43,  7.89s/it]
  4%|▍         | 8947/225000 [29:01:54<439:34:54,  7.32s/it]
  4%|▍         | 8948/225000 [29:01:59<398:26:00,  6.64s/it]
  4%|▍         | 8949/225000 [29:02:12<509:14:58,  8.49s/it]
  4%|▍         | 8950/225000 [29:02:17<447:26:25,  7.46s/it]
                                                            
{'loss': '0.7717', 'grad_norm': '0.8837', 'learning_rate': '4.472e-05', 'epoch': '0.03978', 'num_input_tokens_seen': 52453848, 'train_runtime': '1.045e+05', 'train_tokens_per_second': '501.8'}

  4%|▍         | 8950/225000 [29:02:17<447:26:25,  7.46s/it]
  4%|▍         | 8951/225000 [29:02:22<404:06:03,  6.73s/it]
  4%|▍         | 8952/225000 [29:02:30<422:27:09,  7.04s/it]
  4%|▍         | 8953/225000 [29:02:41<496:16:12,  8.27s/it]
  4%|▍         | 8954/225000 [29:02:47<457:35:46,  7.62s/it]
  4%|▍         | 8955/225000 [29:02:56<483:44:06,  8.06s/it]
  4%|▍         | 8956/225000 [29:03:06<522:05:25,  8.70s/it]
  4%|▍         | 8957/225000 [29:03:10<436:00:25,  7.27s/it]
  4%|▍         | 8958/225000 [29:03:23<541:35:10,  9.02s/it]
  4%|▍         | 8959/225000 [29:03:28<460:58:15,  7.68s/it]
  4%|▍         | 8960/225000 [29:03:32<404:03:42,  6.73s/it]
  4%|▍         | 8961/225000 [29:03:38<383:38:34,  6.39s/it]
  4%|▍         | 8962/225000 [29:03:47<433:56:49,  7.23s/it]
  4%|▍         | 8963/225000 [29:03:51<381:02:51,  6.35s/it]
  4%|▍         | 8964/225000 [29:03:57<370:00:03,  6.17s/it]
  4%|▍         | 8965/225000 [29:04:05<400:32:41,  6.67s/it]
  4%|▍         | 8966/225000 [29:04:14<450:52:45,  7.51s/it]
  4%|▍         | 8967/225000 [29:04:19<402:00:27,  6.70s/it]
  4%|▍         | 8968/225000 [29:04:36<580:47:54,  9.68s/it]
  4%|▍         | 8969/225000 [29:04:48<625:58:52, 10.43s/it]
  4%|▍         | 8970/225000 [29:04:54<553:52:43,  9.23s/it]
  4%|▍         | 8971/225000 [29:05:01<502:56:42,  8.38s/it]
  4%|▍         | 8972/225000 [29:05:11<536:31:18,  8.94s/it]
  4%|▍         | 8973/225000 [29:05:16<469:43:02,  7.83s/it]
  4%|▍         | 8974/225000 [29:05:22<424:07:02,  7.07s/it]
  4%|▍         | 8975/225000 [29:05:27<389:16:06,  6.49s/it]
  4%|▍         | 8976/225000 [29:05:31<353:15:41,  5.89s/it]
  4%|▍         | 8977/225000 [29:05:41<419:39:52,  6.99s/it]
  4%|▍         | 8978/225000 [29:05:45<374:51:19,  6.25s/it]
  4%|▍         | 8979/225000 [29:06:00<532:42:45,  8.88s/it]
  4%|▍         | 8980/225000 [29:06:05<461:43:40,  7.69s/it]
  4%|▍         | 8981/225000 [29:06:18<544:45:24,  9.08s/it]
  4%|▍         | 8982/225000 [29:06:21<451:03:20,  7.52s/it]
  4%|▍         | 8983/225000 [29:06:31<480:12:12,  8.00s/it]
  4%|▍         | 8984/225000 [29:06:35<421:53:35,  7.03s/it]
  4%|▍         | 8985/225000 [29:06:45<459:59:33,  7.67s/it]
  4%|▍         | 8986/225000 [29:06:49<403:23:48,  6.72s/it]
  4%|▍         | 8987/225000 [29:06:55<395:14:51,  6.59s/it]
  4%|▍         | 8988/225000 [29:07:01<372:25:06,  6.21s/it]
  4%|▍         | 8989/225000 [29:07:05<346:15:16,  5.77s/it]
  4%|▍         | 8990/225000 [29:07:15<415:00:24,  6.92s/it]
  4%|▍         | 8991/225000 [29:07:21<402:52:28,  6.71s/it]
  4%|▍         | 8992/225000 [29:07:32<481:38:37,  8.03s/it]
  4%|▍         | 8993/225000 [29:07:39<465:27:45,  7.76s/it]
  4%|▍         | 8994/225000 [29:07:49<492:44:35,  8.21s/it]
  4%|▍         | 8995/225000 [29:07:54<434:34:22,  7.24s/it]
  4%|▍         | 8996/225000 [29:08:04<482:40:32,  8.04s/it]
  4%|▍         | 8997/225000 [29:08:12<494:00:18,  8.23s/it]
  4%|▍         | 8998/225000 [29:08:20<492:14:53,  8.20s/it]
  4%|▍         | 8999/225000 [29:08:27<460:20:26,  7.67s/it]
  4%|▍         | 9000/225000 [29:08:35<461:24:28,  7.69s/it]
                                                            
{'loss': '0.7337', 'grad_norm': '0.7745', 'learning_rate': '4.497e-05', 'epoch': '0.04', 'num_input_tokens_seen': 52630760, 'train_runtime': '1.049e+05', 'train_tokens_per_second': '501.7'}

  4%|▍         | 9000/225000 [29:08:35<461:24:28,  7.69s/it]
  4%|▍         | 9001/225000 [29:08:39<401:58:32,  6.70s/it]
  4%|▍         | 9002/225000 [29:08:51<499:15:56,  8.32s/it]
  4%|▍         | 9003/225000 [29:08:58<474:02:34,  7.90s/it]
  4%|▍         | 9004/225000 [29:09:06<470:47:57,  7.85s/it]
  4%|▍         | 9005/225000 [29:09:09<394:31:40,  6.58s/it]
  4%|▍         | 9006/225000 [29:09:17<415:33:07,  6.93s/it]
  4%|▍         | 9007/225000 [29:09:23<398:18:59,  6.64s/it]
  4%|▍         | 9008/225000 [29:09:29<388:53:57,  6.48s/it]
  4%|▍         | 9009/225000 [29:09:37<405:04:57,  6.75s/it]
  4%|▍         | 9010/225000 [29:09:44<423:25:54,  7.06s/it]
  4%|▍         | 9011/225000 [29:09:51<417:57:29,  6.97s/it]
  4%|▍         | 9012/225000 [29:09:58<411:03:28,  6.85s/it]
  4%|▍         | 9013/225000 [29:10:04<402:49:11,  6.71s/it]
  4%|▍         | 9014/225000 [29:10:13<440:39:06,  7.34s/it]
  4%|▍         | 9015/225000 [29:10:19<413:10:13,  6.89s/it]
  4%|▍         | 9016/225000 [29:10:33<556:12:01,  9.27s/it]
  4%|▍         | 9017/225000 [29:10:39<494:06:57,  8.24s/it]
  4%|▍         | 9018/225000 [29:10:44<436:33:27,  7.28s/it]
  4%|▍         | 9019/225000 [29:10:54<483:27:08,  8.06s/it]
  4%|▍         | 9020/225000 [29:10:59<429:22:35,  7.16s/it]
  4%|▍         | 9021/225000 [29:11:14<570:16:18,  9.51s/it]
  4%|▍         | 9022/225000 [29:11:24<573:18:57,  9.56s/it]
  4%|▍         | 9023/225000 [29:11:31<529:57:50,  8.83s/it]
  4%|▍         | 9024/225000 [29:11:37<478:30:22,  7.98s/it]
  4%|▍         | 9025/225000 [29:11:42<425:58:21,  7.10s/it]
  4%|▍         | 9026/225000 [29:11:53<491:13:30,  8.19s/it]
  4%|▍         | 9027/225000 [29:12:02<502:59:00,  8.38s/it]
  4%|▍         | 9028/225000 [29:12:11<514:30:40,  8.58s/it]
  4%|▍         | 9029/225000 [29:12:21<543:11:07,  9.05s/it]
  4%|▍         | 9030/225000 [29:12:27<495:09:49,  8.25s/it]
  4%|▍         | 9031/225000 [29:12:32<439:16:16,  7.32s/it]
  4%|▍         | 9032/225000 [29:12:37<397:05:40,  6.62s/it]
  4%|▍         | 9033/225000 [29:12:43<372:53:34,  6.22s/it]
  4%|▍         | 9034/225000 [29:12:55<479:38:40,  8.00s/it]
  4%|▍         | 9035/225000 [29:13:01<439:21:20,  7.32s/it]
  4%|▍         | 9036/225000 [29:13:07<426:21:39,  7.11s/it]
  4%|▍         | 9037/225000 [29:13:13<399:56:52,  6.67s/it]
  4%|▍         | 9038/225000 [29:13:18<372:50:56,  6.22s/it]
  4%|▍         | 9039/225000 [29:13:26<407:02:57,  6.79s/it]
  4%|▍         | 9040/225000 [29:13:37<486:15:25,  8.11s/it]
  4%|▍         | 9041/225000 [29:13:42<433:05:22,  7.22s/it]
  4%|▍         | 9042/225000 [29:13:55<534:03:16,  8.90s/it]
  4%|▍         | 9043/225000 [29:14:01<473:06:08,  7.89s/it]
  4%|▍         | 9044/225000 [29:14:06<420:49:40,  7.02s/it]
  4%|▍         | 9045/225000 [29:14:10<369:21:30,  6.16s/it]
  4%|▍         | 9046/225000 [29:14:24<516:03:29,  8.60s/it]
  4%|▍         | 9047/225000 [29:14:28<428:45:22,  7.15s/it]
  4%|▍         | 9048/225000 [29:14:36<443:58:34,  7.40s/it]
  4%|▍         | 9049/225000 [29:14:41<407:13:24,  6.79s/it]
  4%|▍         | 9050/225000 [29:14:47<378:32:44,  6.31s/it]
                                                            
{'loss': '0.7244', 'grad_norm': '0.6426', 'learning_rate': '4.522e-05', 'epoch': '0.04022', 'num_input_tokens_seen': 52806680, 'train_runtime': '1.053e+05', 'train_tokens_per_second': '501.6'}

  4%|▍         | 9050/225000 [29:14:47<378:32:44,  6.31s/it]
  4%|▍         | 9051/225000 [29:14:53<384:11:41,  6.40s/it]
  4%|▍         | 9052/225000 [29:15:06<493:16:07,  8.22s/it]
  4%|▍         | 9053/225000 [29:15:09<409:54:34,  6.83s/it]
  4%|▍         | 9054/225000 [29:15:18<441:41:10,  7.36s/it]
  4%|▍         | 9055/225000 [29:15:22<376:03:24,  6.27s/it]
  4%|▍         | 9056/225000 [29:15:31<435:16:09,  7.26s/it]
  4%|▍         | 9057/225000 [29:15:43<518:16:32,  8.64s/it]
  4%|▍         | 9058/225000 [29:15:52<523:28:25,  8.73s/it]
  4%|▍         | 9059/225000 [29:15:56<446:04:01,  7.44s/it]
  4%|▍         | 9060/225000 [29:16:01<396:44:01,  6.61s/it]
  4%|▍         | 9061/225000 [29:16:11<462:44:17,  7.71s/it]
  4%|▍         | 9062/225000 [29:16:18<451:27:45,  7.53s/it]
  4%|▍         | 9063/225000 [29:16:30<517:14:09,  8.62s/it]
  4%|▍         | 9064/225000 [29:16:35<455:54:11,  7.60s/it]
  4%|▍         | 9065/225000 [29:16:43<469:26:45,  7.83s/it]
  4%|▍         | 9066/225000 [29:16:53<507:29:07,  8.46s/it]
  4%|▍         | 9067/225000 [29:17:00<481:38:12,  8.03s/it]
  4%|▍         | 9068/225000 [29:17:12<549:48:18,  9.17s/it]
  4%|▍         | 9069/225000 [29:17:18<486:23:55,  8.11s/it]
  4%|▍         | 9070/225000 [29:17:23<439:05:21,  7.32s/it]
  4%|▍         | 9071/225000 [29:17:28<396:23:49,  6.61s/it]
  4%|▍         | 9072/225000 [29:17:37<444:12:26,  7.41s/it]
  4%|▍         | 9073/225000 [29:17:47<482:31:46,  8.04s/it]
  4%|▍         | 9074/225000 [29:18:00<567:15:59,  9.46s/it]
  4%|▍         | 9075/225000 [29:18:11<602:04:13, 10.04s/it]
  4%|▍         | 9076/225000 [29:18:20<581:39:34,  9.70s/it]
  4%|▍         | 9077/225000 [29:18:28<560:14:05,  9.34s/it]
  4%|▍         | 9078/225000 [29:18:35<516:30:31,  8.61s/it]
  4%|▍         | 9079/225000 [29:18:41<457:29:59,  7.63s/it]
  4%|▍         | 9080/225000 [29:18:46<425:54:22,  7.10s/it]
  4%|▍         | 9081/225000 [29:18:54<442:17:05,  7.37s/it]
  4%|▍         | 9082/225000 [29:19:06<511:45:03,  8.53s/it]
  4%|▍         | 9083/225000 [29:19:12<472:25:18,  7.88s/it]
  4%|▍         | 9084/225000 [29:19:22<516:25:18,  8.61s/it]
  4%|▍         | 9085/225000 [29:19:28<461:47:12,  7.70s/it]
  4%|▍         | 9086/225000 [29:19:34<436:50:29,  7.28s/it]
  4%|▍         | 9087/225000 [29:19:39<396:12:54,  6.61s/it]
  4%|▍         | 9088/225000 [29:19:46<391:29:32,  6.53s/it]
  4%|▍         | 9089/225000 [29:19:58<501:29:24,  8.36s/it]
  4%|▍         | 9090/225000 [29:20:05<467:53:19,  7.80s/it]
  4%|▍         | 9091/225000 [29:20:09<412:41:31,  6.88s/it]
  4%|▍         | 9092/225000 [29:20:19<454:09:29,  7.57s/it]
  4%|▍         | 9093/225000 [29:20:24<412:12:28,  6.87s/it]
  4%|▍         | 9094/225000 [29:20:33<443:30:59,  7.40s/it]
  4%|▍         | 9095/225000 [29:20:41<458:08:12,  7.64s/it]
  4%|▍         | 9096/225000 [29:20:46<406:22:52,  6.78s/it]
  4%|▍         | 9097/225000 [29:20:50<368:53:28,  6.15s/it]
  4%|▍         | 9098/225000 [29:21:07<556:34:08,  9.28s/it]
  4%|▍         | 9099/225000 [29:21:16<560:39:18,  9.35s/it]
  4%|▍         | 9100/225000 [29:21:20<459:34:28,  7.66s/it]
                                                            
{'loss': '0.7214', 'grad_norm': '0.8671', 'learning_rate': '4.547e-05', 'epoch': '0.04044', 'num_input_tokens_seen': 52990048, 'train_runtime': '1.057e+05', 'train_tokens_per_second': '501.4'}

  4%|▍         | 9100/225000 [29:21:20<459:34:28,  7.66s/it]
  4%|▍         | 9101/225000 [29:21:28<468:52:47,  7.82s/it]
  4%|▍         | 9102/225000 [29:21:34<425:04:36,  7.09s/it]
  4%|▍         | 9103/225000 [29:21:38<374:03:11,  6.24s/it]
  4%|▍         | 9104/225000 [29:21:51<490:01:02,  8.17s/it]
  4%|▍         | 9105/225000 [29:21:57<461:47:18,  7.70s/it]
  4%|▍         | 9106/225000 [29:22:09<528:07:10,  8.81s/it]
  4%|▍         | 9107/225000 [29:22:16<505:11:14,  8.42s/it]
  4%|▍         | 9108/225000 [29:22:21<446:36:41,  7.45s/it]
  4%|▍         | 9109/225000 [29:22:34<543:26:34,  9.06s/it]
  4%|▍         | 9110/225000 [29:22:39<478:18:44,  7.98s/it]
  4%|▍         | 9111/225000 [29:22:51<540:01:28,  9.01s/it]
  4%|▍         | 9112/225000 [29:22:57<485:07:50,  8.09s/it]
  4%|▍         | 9113/225000 [29:23:01<418:53:40,  6.99s/it]
  4%|▍         | 9114/225000 [29:23:12<479:55:29,  8.00s/it]
  4%|▍         | 9115/225000 [29:23:21<498:46:10,  8.32s/it]
  4%|▍         | 9116/225000 [29:23:28<481:13:37,  8.02s/it]
  4%|▍         | 9117/225000 [29:23:32<413:27:54,  6.89s/it]
  4%|▍         | 9118/225000 [29:23:38<385:44:23,  6.43s/it]
  4%|▍         | 9119/225000 [29:23:43<360:03:54,  6.00s/it]
  4%|▍         | 9120/225000 [29:23:48<341:54:39,  5.70s/it]
  4%|▍         | 9121/225000 [29:23:58<426:52:03,  7.12s/it]
  4%|▍         | 9122/225000 [29:24:05<432:21:23,  7.21s/it]
  4%|▍         | 9123/225000 [29:24:15<471:00:28,  7.85s/it]
  4%|▍         | 9124/225000 [29:24:19<410:47:31,  6.85s/it]
  4%|▍         | 9125/225000 [29:24:25<398:10:51,  6.64s/it]
  4%|▍         | 9126/225000 [29:24:33<410:15:04,  6.84s/it]
  4%|▍         | 9127/225000 [29:24:37<368:04:35,  6.14s/it]
  4%|▍         | 9128/225000 [29:24:43<351:26:01,  5.86s/it]
  4%|▍         | 9129/225000 [29:24:48<342:43:01,  5.72s/it]
  4%|▍         | 9130/225000 [29:24:54<345:49:56,  5.77s/it]
  4%|▍         | 9131/225000 [29:25:00<350:31:26,  5.85s/it]
  4%|▍         | 9132/225000 [29:25:14<495:42:04,  8.27s/it]
  4%|▍         | 9133/225000 [29:25:19<441:18:54,  7.36s/it]
  4%|▍         | 9134/225000 [29:25:24<398:52:05,  6.65s/it]
  4%|▍         | 9135/225000 [29:25:30<384:26:35,  6.41s/it]
  4%|▍         | 9136/225000 [29:25:44<526:44:57,  8.78s/it]
  4%|▍         | 9137/225000 [29:25:55<567:19:41,  9.46s/it]
  4%|▍         | 9138/225000 [29:26:00<486:11:36,  8.11s/it]
  4%|▍         | 9139/225000 [29:26:05<419:45:56,  7.00s/it]
  4%|▍         | 9140/225000 [29:26:10<399:35:21,  6.66s/it]
  4%|▍         | 9141/225000 [29:26:30<624:04:56, 10.41s/it]
  4%|▍         | 9142/225000 [29:26:34<513:03:53,  8.56s/it]
  4%|▍         | 9143/225000 [29:26:45<565:45:11,  9.44s/it]
  4%|▍         | 9144/225000 [29:26:50<488:54:57,  8.15s/it]
  4%|▍         | 9145/225000 [29:27:03<569:45:14,  9.50s/it]
  4%|▍         | 9146/225000 [29:27:07<460:55:46,  7.69s/it]
  4%|▍         | 9147/225000 [29:27:16<495:03:58,  8.26s/it]
  4%|▍         | 9148/225000 [29:27:21<432:37:19,  7.22s/it]
  4%|▍         | 9149/225000 [29:27:31<479:30:30,  8.00s/it]
  4%|▍         | 9150/225000 [29:27:38<457:04:41,  7.62s/it]
                                                            
{'loss': '0.7239', 'grad_norm': '0.7572', 'learning_rate': '4.572e-05', 'epoch': '0.04067', 'num_input_tokens_seen': 53164888, 'train_runtime': '1.061e+05', 'train_tokens_per_second': '501.3'}

  4%|▍         | 9150/225000 [29:27:38<457:04:41,  7.62s/it]
  4%|▍         | 9151/225000 [29:27:41<389:49:15,  6.50s/it]
  4%|▍         | 9152/225000 [29:27:52<460:10:32,  7.67s/it]
  4%|▍         | 9153/225000 [29:28:01<481:08:08,  8.02s/it]
  4%|▍         | 9154/225000 [29:28:10<513:10:24,  8.56s/it]
  4%|▍         | 9155/225000 [29:28:19<517:47:19,  8.64s/it]
  4%|▍         | 9156/225000 [29:28:29<537:45:55,  8.97s/it]
  4%|▍         | 9157/225000 [29:28:35<477:23:50,  7.96s/it]
  4%|▍         | 9158/225000 [29:28:40<422:06:28,  7.04s/it]
  4%|▍         | 9159/225000 [29:28:51<508:06:30,  8.47s/it]
  4%|▍         | 9160/225000 [29:28:56<448:24:30,  7.48s/it]
  4%|▍         | 9161/225000 [29:29:12<589:21:37,  9.83s/it]
  4%|▍         | 9162/225000 [29:29:17<502:26:38,  8.38s/it]
  4%|▍         | 9163/225000 [29:29:27<543:42:01,  9.07s/it]
  4%|▍         | 9164/225000 [29:29:32<463:18:32,  7.73s/it]
  4%|▍         | 9165/225000 [29:29:38<437:41:02,  7.30s/it]
  4%|▍         | 9166/225000 [29:29:45<429:41:09,  7.17s/it]
  4%|▍         | 9167/225000 [29:29:58<526:46:45,  8.79s/it]
  4%|▍         | 9168/225000 [29:30:04<473:01:56,  7.89s/it]
  4%|▍         | 9169/225000 [29:30:08<416:09:43,  6.94s/it]
  4%|▍         | 9170/225000 [29:30:13<371:21:51,  6.19s/it]
  4%|▍         | 9171/225000 [29:30:18<345:22:10,  5.76s/it]
  4%|▍         | 9172/225000 [29:30:27<416:49:09,  6.95s/it]
  4%|▍         | 9173/225000 [29:30:36<447:32:10,  7.46s/it]
  4%|▍         | 9174/225000 [29:30:40<379:21:34,  6.33s/it]
  4%|▍         | 9175/225000 [29:30:48<413:47:33,  6.90s/it]
  4%|▍         | 9176/225000 [29:30:53<390:18:25,  6.51s/it]
  4%|▍         | 9177/225000 [29:31:09<553:51:35,  9.24s/it]
  4%|▍         | 9178/225000 [29:31:13<465:33:00,  7.77s/it]
  4%|▍         | 9179/225000 [29:31:27<569:22:34,  9.50s/it]
  4%|▍         | 9180/225000 [29:31:31<474:30:50,  7.92s/it]
  4%|▍         | 9181/225000 [29:31:39<466:51:07,  7.79s/it]
  4%|▍         | 9182/225000 [29:31:44<430:39:06,  7.18s/it]
  4%|▍         | 9183/225000 [29:31:57<535:41:18,  8.94s/it]
  4%|▍         | 9184/225000 [29:32:07<547:15:28,  9.13s/it]
  4%|▍         | 9185/225000 [29:32:13<489:59:32,  8.17s/it]
  4%|▍         | 9186/225000 [29:32:19<447:19:22,  7.46s/it]
  4%|▍         | 9187/225000 [29:32:30<515:03:04,  8.59s/it]
  4%|▍         | 9188/225000 [29:32:35<452:23:13,  7.55s/it]
  4%|▍         | 9189/225000 [29:32:40<406:13:58,  6.78s/it]
  4%|▍         | 9190/225000 [29:32:49<440:38:47,  7.35s/it]
  4%|▍         | 9191/225000 [29:32:54<395:10:01,  6.59s/it]
  4%|▍         | 9192/225000 [29:33:04<465:36:24,  7.77s/it]
  4%|▍         | 9193/225000 [29:33:10<432:18:11,  7.21s/it]
  4%|▍         | 9194/225000 [29:33:21<494:58:40,  8.26s/it]
  4%|▍         | 9195/225000 [29:33:26<449:36:00,  7.50s/it]
  4%|▍         | 9196/225000 [29:33:34<451:44:33,  7.54s/it]
  4%|▍         | 9197/225000 [29:33:45<509:09:09,  8.49s/it]
  4%|▍         | 9198/225000 [29:33:51<471:11:02,  7.86s/it]
  4%|▍         | 9199/225000 [29:34:02<526:21:48,  8.78s/it]
  4%|▍         | 9200/225000 [29:34:08<472:40:14,  7.89s/it]
                                                            
{'loss': '0.7637', 'grad_norm': '0.8623', 'learning_rate': '4.597e-05', 'epoch': '0.04089', 'num_input_tokens_seen': 53344904, 'train_runtime': '1.064e+05', 'train_tokens_per_second': '501.1'}

  4%|▍         | 9200/225000 [29:34:08<472:40:14,  7.89s/it]
  4%|▍         | 9201/225000 [29:34:16<482:08:33,  8.04s/it]
  4%|▍         | 9202/225000 [29:34:24<479:52:53,  8.01s/it]
  4%|▍         | 9203/225000 [29:34:30<435:46:40,  7.27s/it]
  4%|▍         | 9204/225000 [29:34:37<442:59:31,  7.39s/it]
  4%|▍         | 9205/225000 [29:34:49<524:52:20,  8.76s/it]
  4%|▍         | 9206/225000 [29:34:55<460:38:16,  7.68s/it]
  4%|▍         | 9207/225000 [29:35:04<486:22:04,  8.11s/it]
  4%|▍         | 9208/225000 [29:35:12<481:09:42,  8.03s/it]
  4%|▍         | 9209/225000 [29:35:17<433:23:05,  7.23s/it]
  4%|▍         | 9210/225000 [29:35:26<469:07:37,  7.83s/it]
  4%|▍         | 9211/225000 [29:35:33<448:24:44,  7.48s/it]
  4%|▍         | 9212/225000 [29:35:37<396:01:40,  6.61s/it]
  4%|▍         | 9213/225000 [29:35:43<381:03:13,  6.36s/it]
  4%|▍         | 9214/225000 [29:35:48<350:50:04,  5.85s/it]
  4%|▍         | 9215/225000 [29:36:01<490:13:38,  8.18s/it]
  4%|▍         | 9216/225000 [29:36:08<466:43:23,  7.79s/it]
  4%|▍         | 9217/225000 [29:36:14<427:04:13,  7.12s/it]
  4%|▍         | 9218/225000 [29:36:23<471:28:52,  7.87s/it]
  4%|▍         | 9219/225000 [29:36:31<461:14:49,  7.70s/it]
  4%|▍         | 9220/225000 [29:36:43<545:20:22,  9.10s/it]
  4%|▍         | 9221/225000 [29:36:51<515:10:20,  8.59s/it]
  4%|▍         | 9222/225000 [29:36:57<468:29:26,  7.82s/it]
  4%|▍         | 9223/225000 [29:37:08<542:57:55,  9.06s/it]
  4%|▍         | 9224/225000 [29:37:13<461:05:19,  7.69s/it]
  4%|▍         | 9225/225000 [29:37:19<422:28:19,  7.05s/it]
  4%|▍         | 9226/225000 [29:37:23<376:37:04,  6.28s/it]
  4%|▍         | 9227/225000 [29:37:29<366:26:07,  6.11s/it]
  4%|▍         | 9228/225000 [29:37:42<491:55:38,  8.21s/it]
  4%|▍         | 9229/225000 [29:37:53<553:14:01,  9.23s/it]
  4%|▍         | 9230/225000 [29:38:03<552:48:33,  9.22s/it]
  4%|▍         | 9231/225000 [29:38:08<482:24:10,  8.05s/it]
  4%|▍         | 9232/225000 [29:38:21<568:17:55,  9.48s/it]
  4%|▍         | 9233/225000 [29:38:26<484:37:55,  8.09s/it]
  4%|▍         | 9234/225000 [29:38:34<482:38:51,  8.05s/it]
  4%|▍         | 9235/225000 [29:38:39<441:32:28,  7.37s/it]
  4%|▍         | 9236/225000 [29:38:52<534:36:21,  8.92s/it]
  4%|▍         | 9237/225000 [29:38:58<479:24:07,  8.00s/it]
  4%|▍         | 9238/225000 [29:39:06<483:13:35,  8.06s/it]
  4%|▍         | 9239/225000 [29:39:17<543:36:32,  9.07s/it]
  4%|▍         | 9240/225000 [29:39:21<445:32:53,  7.43s/it]
  4%|▍         | 9241/225000 [29:39:31<495:46:11,  8.27s/it]
  4%|▍         | 9242/225000 [29:39:36<427:49:45,  7.14s/it]
  4%|▍         | 9243/225000 [29:39:47<493:28:03,  8.23s/it]
  4%|▍         | 9244/225000 [29:39:53<468:12:27,  7.81s/it]
  4%|▍         | 9245/225000 [29:39:58<410:05:47,  6.84s/it]
  4%|▍         | 9246/225000 [29:40:06<434:22:08,  7.25s/it]
  4%|▍         | 9247/225000 [29:40:11<383:09:58,  6.39s/it]
  4%|▍         | 9248/225000 [29:40:15<351:04:05,  5.86s/it]
  4%|▍         | 9249/225000 [29:40:20<333:53:49,  5.57s/it]
  4%|▍         | 9250/225000 [29:40:39<577:46:21,  9.64s/it]
                                                            
{'loss': '0.7499', 'grad_norm': '0.6283', 'learning_rate': '4.622e-05', 'epoch': '0.04111', 'num_input_tokens_seen': 53522856, 'train_runtime': '1.068e+05', 'train_tokens_per_second': '501'}

  4%|▍         | 9250/225000 [29:40:39<577:46:21,  9.64s/it]
  4%|▍         | 9251/225000 [29:40:48<562:52:39,  9.39s/it]
  4%|▍         | 9252/225000 [29:40:52<468:50:26,  7.82s/it]
  4%|▍         | 9253/225000 [29:40:58<432:08:37,  7.21s/it]
  4%|▍         | 9254/225000 [29:41:09<508:36:09,  8.49s/it]
  4%|▍         | 9255/225000 [29:41:16<479:37:22,  8.00s/it]
  4%|▍         | 9256/225000 [29:41:25<495:37:17,  8.27s/it]
  4%|▍         | 9257/225000 [29:41:33<492:49:07,  8.22s/it]
  4%|▍         | 9258/225000 [29:41:41<482:27:36,  8.05s/it]
  4%|▍         | 9259/225000 [29:41:56<614:57:35, 10.26s/it]
  4%|▍         | 9260/225000 [29:42:02<528:19:09,  8.82s/it]
  4%|▍         | 9261/225000 [29:42:07<461:28:19,  7.70s/it]
  4%|▍         | 9262/225000 [29:42:13<436:17:12,  7.28s/it]
  4%|▍         | 9263/225000 [29:42:18<392:04:27,  6.54s/it]
  4%|▍         | 9264/225000 [29:42:23<358:37:26,  5.98s/it]
  4%|▍         | 9265/225000 [29:42:40<565:29:05,  9.44s/it]
  4%|▍         | 9266/225000 [29:42:44<468:12:34,  7.81s/it]
  4%|▍         | 9267/225000 [29:42:57<559:39:55,  9.34s/it]
  4%|▍         | 9268/225000 [29:43:02<472:47:34,  7.89s/it]
  4%|▍         | 9269/225000 [29:43:08<444:19:33,  7.41s/it]
  4%|▍         | 9270/225000 [29:43:13<410:23:14,  6.85s/it]
  4%|▍         | 9271/225000 [29:43:19<379:00:06,  6.32s/it]
  4%|▍         | 9272/225000 [29:43:32<508:58:15,  8.49s/it]
  4%|▍         | 9273/225000 [29:43:47<619:29:57, 10.34s/it]
  4%|▍         | 9274/225000 [29:43:53<551:11:53,  9.20s/it]
  4%|▍         | 9275/225000 [29:43:57<449:01:11,  7.49s/it]
  4%|▍         | 9276/225000 [29:44:08<507:42:42,  8.47s/it]
  4%|▍         | 9277/225000 [29:44:15<485:12:42,  8.10s/it]
  4%|▍         | 9278/225000 [29:44:22<474:20:19,  7.92s/it]
  4%|▍         | 9279/225000 [29:44:26<404:00:27,  6.74s/it]
  4%|▍         | 9280/225000 [29:44:31<361:44:52,  6.04s/it]
  4%|▍         | 9281/225000 [29:44:36<356:25:59,  5.95s/it]
  4%|▍         | 9282/225000 [29:44:44<377:43:52,  6.30s/it]
  4%|▍         | 9283/225000 [29:44:55<477:56:11,  7.98s/it]
  4%|▍         | 9284/225000 [29:45:02<451:33:01,  7.54s/it]
  4%|▍         | 9285/225000 [29:45:08<432:23:51,  7.22s/it]
  4%|▍         | 9286/225000 [29:45:22<544:34:33,  9.09s/it]
  4%|▍         | 9287/225000 [29:45:28<495:28:50,  8.27s/it]
  4%|▍         | 9288/225000 [29:45:34<448:40:25,  7.49s/it]
  4%|▍         | 9289/225000 [29:45:42<467:22:37,  7.80s/it]
  4%|▍         | 9290/225000 [29:45:52<499:05:00,  8.33s/it]
  4%|▍         | 9291/225000 [29:46:03<554:15:47,  9.25s/it]
  4%|▍         | 9292/225000 [29:46:09<494:51:12,  8.26s/it]
  4%|▍         | 9293/225000 [29:46:20<544:12:17,  9.08s/it]
  4%|▍         | 9294/225000 [29:46:25<462:48:49,  7.72s/it]
  4%|▍         | 9295/225000 [29:46:32<449:44:06,  7.51s/it]
  4%|▍         | 9296/225000 [29:46:43<507:28:44,  8.47s/it]
  4%|▍         | 9297/225000 [29:46:50<491:24:30,  8.20s/it]
  4%|▍         | 9298/225000 [29:46:55<431:05:47,  7.19s/it]
  4%|▍         | 9299/225000 [29:47:08<533:27:20,  8.90s/it]
  4%|▍         | 9300/225000 [29:47:14<475:15:07,  7.93s/it]
                                                            
{'loss': '0.7786', 'grad_norm': '0.7723', 'learning_rate': '4.647e-05', 'epoch': '0.04133', 'num_input_tokens_seen': 53707688, 'train_runtime': '1.072e+05', 'train_tokens_per_second': '500.8'}

  4%|▍         | 9300/225000 [29:47:14<475:15:07,  7.93s/it]
  4%|▍         | 9301/225000 [29:47:19<426:32:58,  7.12s/it]
  4%|▍         | 9302/225000 [29:47:33<557:49:08,  9.31s/it]
  4%|▍         | 9303/225000 [29:47:37<458:49:31,  7.66s/it]
  4%|▍         | 9304/225000 [29:47:41<396:33:34,  6.62s/it]
  4%|▍         | 9305/225000 [29:47:46<364:28:09,  6.08s/it]
  4%|▍         | 9306/225000 [29:47:54<397:27:12,  6.63s/it]
  4%|▍         | 9307/225000 [29:48:04<466:28:18,  7.79s/it]
  4%|▍         | 9308/225000 [29:48:12<467:13:50,  7.80s/it]
  4%|▍         | 9309/225000 [29:48:17<420:37:30,  7.02s/it]
  4%|▍         | 9310/225000 [29:48:23<393:25:56,  6.57s/it]
  4%|▍         | 9311/225000 [29:48:29<375:32:09,  6.27s/it]
  4%|▍         | 9312/225000 [29:48:35<377:39:27,  6.30s/it]
  4%|▍         | 9313/225000 [29:48:46<471:08:12,  7.86s/it]
  4%|▍         | 9314/225000 [29:48:50<399:43:38,  6.67s/it]
  4%|▍         | 9315/225000 [29:48:56<375:55:25,  6.27s/it]
  4%|▍         | 9316/225000 [29:49:08<489:01:42,  8.16s/it]
  4%|▍         | 9317/225000 [29:49:13<427:23:23,  7.13s/it]
  4%|▍         | 9318/225000 [29:49:18<396:00:38,  6.61s/it]
  4%|▍         | 9319/225000 [29:49:29<468:28:32,  7.82s/it]
  4%|▍         | 9320/225000 [29:49:35<441:20:14,  7.37s/it]
  4%|▍         | 9321/225000 [29:49:42<425:17:48,  7.10s/it]
  4%|▍         | 9322/225000 [29:49:49<427:19:30,  7.13s/it]
  4%|▍         | 9323/225000 [29:50:01<522:41:43,  8.72s/it]
  4%|▍         | 9324/225000 [29:50:05<430:41:57,  7.19s/it]
  4%|▍         | 9325/225000 [29:50:13<444:39:45,  7.42s/it]
  4%|▍         | 9326/225000 [29:50:18<409:27:02,  6.83s/it]
  4%|▍         | 9327/225000 [29:50:29<479:33:30,  8.00s/it]
  4%|▍         | 9328/225000 [29:50:38<499:26:54,  8.34s/it]
  4%|▍         | 9329/225000 [29:50:46<489:03:20,  8.16s/it]
  4%|▍         | 9330/225000 [29:50:53<472:37:48,  7.89s/it]
  4%|▍         | 9331/225000 [29:51:01<464:06:21,  7.75s/it]
  4%|▍         | 9332/225000 [29:51:08<453:04:37,  7.56s/it]
  4%|▍         | 9333/225000 [29:51:23<593:12:56,  9.90s/it]
  4%|▍         | 9334/225000 [29:51:33<595:07:14,  9.93s/it]
  4%|▍         | 9335/225000 [29:51:43<586:35:13,  9.79s/it]
  4%|▍         | 9336/225000 [29:51:48<512:25:28,  8.55s/it]
  4%|▍         | 9337/225000 [29:51:55<472:15:56,  7.88s/it]
  4%|▍         | 9338/225000 [29:52:01<438:14:33,  7.32s/it]
  4%|▍         | 9339/225000 [29:52:07<414:57:14,  6.93s/it]
  4%|▍         | 9340/225000 [29:52:12<382:03:26,  6.38s/it]
  4%|▍         | 9341/225000 [29:52:17<367:26:13,  6.13s/it]
  4%|▍         | 9342/225000 [29:52:24<367:36:17,  6.14s/it]
  4%|▍         | 9343/225000 [29:52:43<610:16:57, 10.19s/it]
  4%|▍         | 9344/225000 [29:52:51<567:39:41,  9.48s/it]
  4%|▍         | 9345/225000 [29:52:55<469:37:46,  7.84s/it]
  4%|▍         | 9346/225000 [29:52:59<404:57:52,  6.76s/it]
  4%|▍         | 9347/225000 [29:53:08<442:21:46,  7.38s/it]
  4%|▍         | 9348/225000 [29:53:18<492:42:38,  8.23s/it]
  4%|▍         | 9349/225000 [29:53:25<473:25:32,  7.90s/it]
  4%|▍         | 9350/225000 [29:53:31<428:19:20,  7.15s/it]
                                                            
{'loss': '0.7388', 'grad_norm': '0.8731', 'learning_rate': '4.672e-05', 'epoch': '0.04156', 'num_input_tokens_seen': 53883296, 'train_runtime': '1.076e+05', 'train_tokens_per_second': '500.7'}

  4%|▍         | 9350/225000 [29:53:31<428:19:20,  7.15s/it]
  4%|▍         | 9351/225000 [29:53:38<434:22:15,  7.25s/it]
  4%|▍         | 9352/225000 [29:53:52<549:39:25,  9.18s/it]
  4%|▍         | 9353/225000 [29:53:59<507:42:11,  8.48s/it]
  4%|▍         | 9354/225000 [29:54:04<449:40:03,  7.51s/it]
  4%|▍         | 9355/225000 [29:54:09<411:44:10,  6.87s/it]
  4%|▍         | 9356/225000 [29:54:22<506:13:46,  8.45s/it]
  4%|▍         | 9357/225000 [29:54:30<499:25:37,  8.34s/it]
  4%|▍         | 9358/225000 [29:54:35<439:08:35,  7.33s/it]
  4%|▍         | 9359/225000 [29:54:50<589:21:46,  9.84s/it]
  4%|▍         | 9360/225000 [29:54:56<507:22:32,  8.47s/it]
  4%|▍         | 9361/225000 [29:55:06<544:41:03,  9.09s/it]
  4%|▍         | 9362/225000 [29:55:13<505:04:36,  8.43s/it]
  4%|▍         | 9363/225000 [29:55:19<460:41:23,  7.69s/it]
  4%|▍         | 9364/225000 [29:55:29<497:16:13,  8.30s/it]
  4%|▍         | 9365/225000 [29:55:35<464:48:36,  7.76s/it]
  4%|▍         | 9366/225000 [29:55:43<469:06:21,  7.83s/it]
  4%|▍         | 9367/225000 [29:55:48<417:34:09,  6.97s/it]
  4%|▍         | 9368/225000 [29:55:53<380:56:13,  6.36s/it]
  4%|▍         | 9369/225000 [29:55:58<351:53:13,  5.87s/it]
  4%|▍         | 9370/225000 [29:56:10<457:18:23,  7.63s/it]
  4%|▍         | 9371/225000 [29:56:15<418:10:28,  6.98s/it]
  4%|▍         | 9372/225000 [29:56:20<387:53:01,  6.48s/it]
  4%|▍         | 9373/225000 [29:56:26<372:38:47,  6.22s/it]
  4%|▍         | 9374/225000 [29:56:44<583:38:15,  9.74s/it]
  4%|▍         | 9375/225000 [29:56:48<483:22:47,  8.07s/it]
  4%|▍         | 9376/225000 [29:56:56<485:23:14,  8.10s/it]
  4%|▍         | 9377/225000 [29:57:01<429:53:31,  7.18s/it]
  4%|▍         | 9378/225000 [29:57:07<395:48:48,  6.61s/it]
  4%|▍         | 9379/225000 [29:57:12<377:11:04,  6.30s/it]
  4%|▍         | 9380/225000 [29:57:19<379:08:46,  6.33s/it]
  4%|▍         | 9381/225000 [29:57:36<583:42:56,  9.75s/it]
  4%|▍         | 9382/225000 [29:57:41<492:03:45,  8.22s/it]
  4%|▍         | 9383/225000 [29:57:45<425:44:16,  7.11s/it]
  4%|▍         | 9384/225000 [29:57:50<384:25:40,  6.42s/it]
  4%|▍         | 9385/225000 [29:58:00<446:10:20,  7.45s/it]
  4%|▍         | 9386/225000 [29:58:08<457:17:28,  7.64s/it]
  4%|▍         | 9387/225000 [29:58:16<458:57:07,  7.66s/it]
  4%|▍         | 9388/225000 [29:58:23<445:03:37,  7.43s/it]
  4%|▍         | 9389/225000 [29:58:28<413:04:39,  6.90s/it]
  4%|▍         | 9390/225000 [29:58:41<506:43:56,  8.46s/it]
  4%|▍         | 9391/225000 [29:58:50<523:04:51,  8.73s/it]
  4%|▍         | 9392/225000 [29:58:59<535:27:36,  8.94s/it]
  4%|▍         | 9393/225000 [29:59:07<510:59:32,  8.53s/it]
  4%|▍         | 9394/225000 [29:59:12<451:19:07,  7.54s/it]
  4%|▍         | 9395/225000 [29:59:19<438:29:46,  7.32s/it]
  4%|▍         | 9396/225000 [29:59:27<459:50:16,  7.68s/it]
  4%|▍         | 9397/225000 [29:59:39<521:12:49,  8.70s/it]
  4%|▍         | 9398/225000 [29:59:43<446:03:10,  7.45s/it]
  4%|▍         | 9399/225000 [29:59:52<479:51:33,  8.01s/it]
  4%|▍         | 9400/225000 [29:59:57<415:15:09,  6.93s/it]
                                                            
{'loss': '0.7432', 'grad_norm': '0.7488', 'learning_rate': '4.696e-05', 'epoch': '0.04178', 'num_input_tokens_seen': 54064808, 'train_runtime': '1.08e+05', 'train_tokens_per_second': '500.6'}

  4%|▍         | 9400/225000 [29:59:57<415:15:09,  6.93s/it]
  4%|▍         | 9401/225000 [30:00:08<500:20:15,  8.35s/it]
  4%|▍         | 9402/225000 [30:00:18<513:48:39,  8.58s/it]
  4%|▍         | 9403/225000 [30:00:22<442:11:28,  7.38s/it]
  4%|▍         | 9404/225000 [30:00:30<450:58:49,  7.53s/it]
  4%|▍         | 9405/225000 [30:00:41<520:44:24,  8.70s/it]
  4%|▍         | 9406/225000 [30:00:46<445:44:08,  7.44s/it]
  4%|▍         | 9407/225000 [30:00:54<455:05:09,  7.60s/it]
  4%|▍         | 9408/225000 [30:00:59<411:44:12,  6.88s/it]
  4%|▍         | 9409/225000 [30:01:04<379:49:57,  6.34s/it]
  4%|▍         | 9410/225000 [30:01:11<380:18:01,  6.35s/it]
  4%|▍         | 9411/225000 [30:01:26<544:37:41,  9.09s/it]
  4%|▍         | 9412/225000 [30:01:33<498:24:15,  8.32s/it]
  4%|▍         | 9413/225000 [30:01:42<509:26:08,  8.51s/it]
  4%|▍         | 9414/225000 [30:01:46<428:28:33,  7.15s/it]
  4%|▍         | 9415/225000 [30:01:54<449:43:07,  7.51s/it]
  4%|▍         | 9416/225000 [30:01:59<401:15:26,  6.70s/it]
  4%|▍         | 9417/225000 [30:02:03<366:19:32,  6.12s/it]
  4%|▍         | 9418/225000 [30:02:19<536:50:46,  8.96s/it]
  4%|▍         | 9419/225000 [30:02:26<495:34:58,  8.28s/it]
  4%|▍         | 9420/225000 [30:02:31<440:31:04,  7.36s/it]
  4%|▍         | 9421/225000 [30:02:36<395:34:55,  6.61s/it]
  4%|▍         | 9422/225000 [30:02:42<380:06:37,  6.35s/it]
  4%|▍         | 9423/225000 [30:02:49<404:54:54,  6.76s/it]
  4%|▍         | 9424/225000 [30:03:00<477:54:20,  7.98s/it]
  4%|▍         | 9425/225000 [30:03:11<526:27:09,  8.79s/it]
  4%|▍         | 9426/225000 [30:03:15<442:37:12,  7.39s/it]
  4%|▍         | 9427/225000 [30:03:23<457:18:41,  7.64s/it]
  4%|▍         | 9428/225000 [30:03:31<464:37:25,  7.76s/it]
  4%|▍         | 9429/225000 [30:03:38<446:38:19,  7.46s/it]
  4%|▍         | 9430/225000 [30:03:45<443:56:12,  7.41s/it]
  4%|▍         | 9431/225000 [30:03:50<393:04:53,  6.56s/it]
  4%|▍         | 9432/225000 [30:03:55<373:41:02,  6.24s/it]
  4%|▍         | 9433/225000 [30:04:01<358:54:12,  5.99s/it]
  4%|▍         | 9434/225000 [30:04:08<387:38:35,  6.47s/it]
  4%|▍         | 9435/225000 [30:04:18<452:39:14,  7.56s/it]
  4%|▍         | 9436/225000 [30:04:23<406:36:09,  6.79s/it]
  4%|▍         | 9437/225000 [30:04:29<378:00:42,  6.31s/it]
  4%|▍         | 9438/225000 [30:04:35<381:59:54,  6.38s/it]
  4%|▍         | 9439/225000 [30:04:50<538:20:34,  8.99s/it]
  4%|▍         | 9440/225000 [30:04:57<500:21:33,  8.36s/it]
  4%|▍         | 9441/225000 [30:05:07<535:08:33,  8.94s/it]
  4%|▍         | 9442/225000 [30:05:15<509:40:41,  8.51s/it]
  4%|▍         | 9443/225000 [30:05:27<578:13:19,  9.66s/it]
  4%|▍         | 9444/225000 [30:05:39<621:24:12, 10.38s/it]
  4%|▍         | 9445/225000 [30:05:44<526:52:57,  8.80s/it]
  4%|▍         | 9446/225000 [30:05:52<500:32:56,  8.36s/it]
  4%|▍         | 9447/225000 [30:06:00<498:07:12,  8.32s/it]
  4%|▍         | 9448/225000 [30:06:06<461:55:06,  7.71s/it]
  4%|▍         | 9449/225000 [30:06:11<412:13:54,  6.88s/it]
  4%|▍         | 9450/225000 [30:06:19<421:06:05,  7.03s/it]
                                                            
{'loss': '0.7686', 'grad_norm': '0.678', 'learning_rate': '4.721e-05', 'epoch': '0.042', 'num_input_tokens_seen': 54243752, 'train_runtime': '1.084e+05', 'train_tokens_per_second': '500.5'}

  4%|▍         | 9450/225000 [30:06:19<421:06:05,  7.03s/it]
  4%|▍         | 9451/225000 [30:06:27<446:03:41,  7.45s/it]
  4%|▍         | 9452/225000 [30:06:32<408:58:37,  6.83s/it]
  4%|▍         | 9453/225000 [30:06:45<519:30:22,  8.68s/it]
  4%|▍         | 9454/225000 [30:06:53<495:21:13,  8.27s/it]
  4%|▍         | 9455/225000 [30:06:57<429:33:20,  7.17s/it]
  4%|▍         | 9456/225000 [30:07:08<500:04:49,  8.35s/it]
  4%|▍         | 9457/225000 [30:07:16<494:16:09,  8.26s/it]
  4%|▍         | 9458/225000 [30:07:20<415:46:18,  6.94s/it]
  4%|▍         | 9459/225000 [30:07:30<459:41:44,  7.68s/it]
  4%|▍         | 9460/225000 [30:07:34<404:15:17,  6.75s/it]
  4%|▍         | 9461/225000 [30:07:39<374:55:16,  6.26s/it]
  4%|▍         | 9462/225000 [30:07:45<356:46:10,  5.96s/it]
  4%|▍         | 9463/225000 [30:07:58<494:21:27,  8.26s/it]
  4%|▍         | 9464/225000 [30:08:03<421:31:51,  7.04s/it]
  4%|▍         | 9465/225000 [30:08:12<472:16:36,  7.89s/it]
  4%|▍         | 9466/225000 [30:08:21<490:48:39,  8.20s/it]
  4%|▍         | 9467/225000 [30:08:31<521:55:34,  8.72s/it]
  4%|▍         | 9468/225000 [30:08:38<488:31:52,  8.16s/it]
  4%|▍         | 9469/225000 [30:08:48<520:42:17,  8.70s/it]
  4%|▍         | 9470/225000 [30:09:02<622:21:44, 10.40s/it]
  4%|▍         | 9471/225000 [30:09:07<516:22:23,  8.63s/it]
  4%|▍         | 9472/225000 [30:09:11<443:10:47,  7.40s/it]
  4%|▍         | 9473/225000 [30:09:17<411:58:52,  6.88s/it]
  4%|▍         | 9474/225000 [30:09:29<496:01:51,  8.29s/it]
  4%|▍         | 9475/225000 [30:09:34<437:41:02,  7.31s/it]
  4%|▍         | 9476/225000 [30:09:39<405:29:51,  6.77s/it]
  4%|▍         | 9477/225000 [30:09:45<383:39:52,  6.41s/it]
  4%|▍         | 9478/225000 [30:09:56<469:13:49,  7.84s/it]
  4%|▍         | 9479/225000 [30:10:04<471:15:06,  7.87s/it]
  4%|▍         | 9480/225000 [30:10:18<585:51:42,  9.79s/it]
  4%|▍         | 9481/225000 [30:10:22<478:50:08,  8.00s/it]
  4%|▍         | 9482/225000 [30:10:32<512:24:41,  8.56s/it]
  4%|▍         | 9483/225000 [30:10:38<460:51:37,  7.70s/it]
  4%|▍         | 9484/225000 [30:10:42<400:03:19,  6.68s/it]
  4%|▍         | 9485/225000 [30:10:57<543:27:11,  9.08s/it]
  4%|▍         | 9486/225000 [30:11:02<475:12:14,  7.94s/it]
  4%|▍         | 9487/225000 [30:11:07<434:13:29,  7.25s/it]
  4%|▍         | 9488/225000 [30:11:18<488:04:39,  8.15s/it]
  4%|▍         | 9489/225000 [30:11:22<422:21:33,  7.06s/it]
  4%|▍         | 9490/225000 [30:11:27<383:23:37,  6.40s/it]
  4%|▍         | 9491/225000 [30:11:33<371:12:27,  6.20s/it]
  4%|▍         | 9492/225000 [30:11:43<443:09:58,  7.40s/it]
  4%|▍         | 9493/225000 [30:11:53<480:13:11,  8.02s/it]
  4%|▍         | 9494/225000 [30:12:01<484:03:44,  8.09s/it]
  4%|▍         | 9495/225000 [30:12:07<448:33:57,  7.49s/it]
  4%|▍         | 9496/225000 [30:12:12<410:17:04,  6.85s/it]
  4%|▍         | 9497/225000 [30:12:17<381:00:41,  6.36s/it]
  4%|▍         | 9498/225000 [30:12:22<355:54:41,  5.95s/it]
  4%|▍         | 9499/225000 [30:12:39<540:37:47,  9.03s/it]
  4%|▍         | 9500/225000 [30:12:43<451:28:31,  7.54s/it]
                                                            
{'loss': '0.777', 'grad_norm': '0.7559', 'learning_rate': '4.746e-05', 'epoch': '0.04222', 'num_input_tokens_seen': 54419616, 'train_runtime': '1.088e+05', 'train_tokens_per_second': '500.3'}

  4%|▍         | 9500/225000 [30:12:43<451:28:31,  7.54s/it]
  4%|▍         | 9501/225000 [30:12:47<399:51:34,  6.68s/it]
  4%|▍         | 9502/225000 [30:12:58<474:57:33,  7.93s/it]
  4%|▍         | 9503/225000 [30:13:13<605:54:02, 10.12s/it]
  4%|▍         | 9504/225000 [30:13:18<507:12:23,  8.47s/it]
  4%|▍         | 9505/225000 [30:13:22<428:56:11,  7.17s/it]
  4%|▍         | 9506/225000 [30:13:32<482:21:52,  8.06s/it]
  4%|▍         | 9507/225000 [30:13:39<459:48:13,  7.68s/it]
  4%|▍         | 9508/225000 [30:13:44<401:01:52,  6.70s/it]
  4%|▍         | 9509/225000 [30:13:50<391:29:47,  6.54s/it]
  4%|▍         | 9510/225000 [30:13:56<378:15:13,  6.32s/it]
  4%|▍         | 9511/225000 [30:14:09<510:17:14,  8.52s/it]
  4%|▍         | 9512/225000 [30:14:15<459:06:58,  7.67s/it]
  4%|▍         | 9513/225000 [30:14:20<412:54:54,  6.90s/it]
  4%|▍         | 9514/225000 [30:14:26<399:25:39,  6.67s/it]
  4%|▍         | 9515/225000 [30:14:32<384:42:08,  6.43s/it]
  4%|▍         | 9516/225000 [30:14:46<528:17:04,  8.83s/it]
  4%|▍         | 9517/225000 [30:14:57<561:01:55,  9.37s/it]
  4%|▍         | 9518/225000 [30:15:05<541:03:14,  9.04s/it]
  4%|▍         | 9519/225000 [30:15:10<469:08:26,  7.84s/it]
  4%|▍         | 9520/225000 [30:15:21<523:55:17,  8.75s/it]
  4%|▍         | 9521/225000 [30:15:26<455:33:38,  7.61s/it]
  4%|▍         | 9522/225000 [30:15:33<432:26:27,  7.22s/it]
  4%|▍         | 9523/225000 [30:15:48<588:48:43,  9.84s/it]
  4%|▍         | 9524/225000 [30:15:52<476:35:00,  7.96s/it]
  4%|▍         | 9525/225000 [30:16:01<501:15:41,  8.37s/it]
  4%|▍         | 9526/225000 [30:16:11<520:34:48,  8.70s/it]
  4%|▍         | 9527/225000 [30:16:18<491:37:07,  8.21s/it]
  4%|▍         | 9528/225000 [30:16:25<470:28:17,  7.86s/it]
  4%|▍         | 9529/225000 [30:16:34<485:18:44,  8.11s/it]
  4%|▍         | 9530/225000 [30:16:40<447:32:02,  7.48s/it]
  4%|▍         | 9531/225000 [30:16:44<385:12:46,  6.44s/it]
  4%|▍         | 9532/225000 [30:16:53<442:38:34,  7.40s/it]
  4%|▍         | 9533/225000 [30:16:59<406:14:47,  6.79s/it]
  4%|▍         | 9534/225000 [30:17:03<370:02:38,  6.18s/it]
  4%|▍         | 9535/225000 [30:17:13<432:00:00,  7.22s/it]
  4%|▍         | 9536/225000 [30:17:18<398:24:23,  6.66s/it]
  4%|▍         | 9537/225000 [30:17:24<376:15:03,  6.29s/it]
  4%|▍         | 9538/225000 [30:17:30<371:22:02,  6.20s/it]
  4%|▍         | 9539/225000 [30:17:35<347:24:30,  5.80s/it]
  4%|▍         | 9540/225000 [30:17:42<372:21:16,  6.22s/it]
  4%|▍         | 9541/225000 [30:17:57<524:20:01,  8.76s/it]
  4%|▍         | 9542/225000 [30:18:04<503:13:31,  8.41s/it]
  4%|▍         | 9543/225000 [30:18:10<448:34:17,  7.50s/it]
  4%|▍         | 9544/225000 [30:18:19<487:11:24,  8.14s/it]
  4%|▍         | 9545/225000 [30:18:28<493:21:51,  8.24s/it]
  4%|▍         | 9546/225000 [30:18:31<414:04:18,  6.92s/it]
  4%|▍         | 9547/225000 [30:18:44<506:17:35,  8.46s/it]
  4%|▍         | 9548/225000 [30:18:48<435:40:10,  7.28s/it]
  4%|▍         | 9549/225000 [30:18:58<475:57:24,  7.95s/it]
  4%|▍         | 9550/225000 [30:19:02<417:05:48,  6.97s/it]
                                                            
{'loss': '0.7379', 'grad_norm': '0.7135', 'learning_rate': '4.771e-05', 'epoch': '0.04244', 'num_input_tokens_seen': 54597072, 'train_runtime': '1.091e+05', 'train_tokens_per_second': '500.2'}

  4%|▍         | 9550/225000 [30:19:02<417:05:48,  6.97s/it]
  4%|▍         | 9551/225000 [30:19:07<380:57:15,  6.37s/it]
  4%|▍         | 9552/225000 [30:19:13<377:31:18,  6.31s/it]
  4%|▍         | 9553/225000 [30:19:26<482:41:08,  8.07s/it]
  4%|▍         | 9554/225000 [30:19:39<574:41:45,  9.60s/it]
  4%|▍         | 9555/225000 [30:19:45<523:20:08,  8.74s/it]
  4%|▍         | 9556/225000 [30:19:51<461:09:40,  7.71s/it]
  4%|▍         | 9557/225000 [30:19:56<418:31:51,  6.99s/it]
  4%|▍         | 9558/225000 [30:20:05<452:33:56,  7.56s/it]
  4%|▍         | 9559/225000 [30:20:12<440:17:48,  7.36s/it]
  4%|▍         | 9560/225000 [30:20:22<495:10:58,  8.27s/it]
  4%|▍         | 9561/225000 [30:20:26<417:26:39,  6.98s/it]
  4%|▍         | 9562/225000 [30:20:39<527:48:37,  8.82s/it]
  4%|▍         | 9563/225000 [30:20:45<477:51:32,  7.99s/it]
  4%|▍         | 9564/225000 [30:20:57<538:45:16,  9.00s/it]
  4%|▍         | 9565/225000 [30:21:06<537:26:34,  8.98s/it]
  4%|▍         | 9566/225000 [30:21:11<473:17:41,  7.91s/it]
  4%|▍         | 9567/225000 [30:21:19<466:37:59,  7.80s/it]
  4%|▍         | 9568/225000 [30:21:28<490:48:22,  8.20s/it]
  4%|▍         | 9569/225000 [30:21:37<516:00:06,  8.62s/it]
  4%|▍         | 9570/225000 [30:21:42<435:16:58,  7.27s/it]
  4%|▍         | 9571/225000 [30:21:49<435:22:47,  7.28s/it]
  4%|▍         | 9572/225000 [30:21:57<459:43:52,  7.68s/it]
  4%|▍         | 9573/225000 [30:22:03<429:01:41,  7.17s/it]
  4%|▍         | 9574/225000 [30:22:08<380:41:20,  6.36s/it]
  4%|▍         | 9575/225000 [30:22:13<358:04:33,  5.98s/it]
  4%|▍         | 9576/225000 [30:22:24<455:47:20,  7.62s/it]
  4%|▍         | 9577/225000 [30:22:32<458:07:58,  7.66s/it]
  4%|▍         | 9578/225000 [30:22:44<531:34:02,  8.88s/it]
  4%|▍         | 9579/225000 [30:22:49<470:49:31,  7.87s/it]
  4%|▍         | 9580/225000 [30:22:55<426:33:49,  7.13s/it]
  4%|▍         | 9581/225000 [30:23:02<424:38:54,  7.10s/it]
  4%|▍         | 9582/225000 [30:23:07<396:43:41,  6.63s/it]
  4%|▍         | 9583/225000 [30:23:14<389:57:09,  6.52s/it]
  4%|▍         | 9584/225000 [30:23:19<364:24:11,  6.09s/it]
  4%|▍         | 9585/225000 [30:23:28<421:15:38,  7.04s/it]
  4%|▍         | 9586/225000 [30:23:36<435:14:17,  7.27s/it]
  4%|▍         | 9587/225000 [30:23:43<428:16:25,  7.16s/it]
  4%|▍         | 9588/225000 [30:23:48<402:58:47,  6.73s/it]
  4%|▍         | 9589/225000 [30:23:54<378:05:59,  6.32s/it]
  4%|▍         | 9590/225000 [30:23:59<361:29:39,  6.04s/it]
  4%|▍         | 9591/225000 [30:24:07<391:53:21,  6.55s/it]
  4%|▍         | 9592/225000 [30:24:16<434:37:29,  7.26s/it]
  4%|▍         | 9593/225000 [30:24:28<523:41:45,  8.75s/it]
  4%|▍         | 9594/225000 [30:24:32<442:56:07,  7.40s/it]
  4%|▍         | 9595/225000 [30:24:40<450:43:00,  7.53s/it]
  4%|▍         | 9596/225000 [30:24:47<439:30:14,  7.35s/it]
  4%|▍         | 9597/225000 [30:24:51<383:44:18,  6.41s/it]
  4%|▍         | 9598/225000 [30:25:00<428:49:08,  7.17s/it]
  4%|▍         | 9599/225000 [30:25:05<378:26:04,  6.32s/it]
  4%|▍         | 9600/225000 [30:25:09<349:52:34,  5.85s/it]
                                                            
{'loss': '0.7473', 'grad_norm': '0.8187', 'learning_rate': '4.796e-05', 'epoch': '0.04267', 'num_input_tokens_seen': 54771288, 'train_runtime': '1.095e+05', 'train_tokens_per_second': '500.1'}

  4%|▍         | 9600/225000 [30:25:09<349:52:34,  5.85s/it]
  4%|▍         | 9601/225000 [30:25:15<339:41:43,  5.68s/it]
  4%|▍         | 9602/225000 [30:25:26<438:31:18,  7.33s/it]
  4%|▍         | 9603/225000 [30:25:40<569:15:04,  9.51s/it]
  4%|▍         | 9604/225000 [30:25:45<483:28:29,  8.08s/it]
  4%|▍         | 9605/225000 [30:25:51<451:59:52,  7.55s/it]
  4%|▍         | 9606/225000 [30:26:02<496:46:15,  8.30s/it]
  4%|▍         | 9607/225000 [30:26:07<452:15:04,  7.56s/it]
  4%|▍         | 9608/225000 [30:26:18<506:56:16,  8.47s/it]
  4%|▍         | 9609/225000 [30:26:23<439:32:31,  7.35s/it]
  4%|▍         | 9610/225000 [30:26:30<435:58:59,  7.29s/it]
  4%|▍         | 9611/225000 [30:26:34<388:39:02,  6.50s/it]
  4%|▍         | 9612/225000 [30:26:44<440:58:50,  7.37s/it]
  4%|▍         | 9613/225000 [30:26:50<418:43:11,  7.00s/it]
  4%|▍         | 9614/225000 [30:26:54<370:24:10,  6.19s/it]
  4%|▍         | 9615/225000 [30:27:05<453:44:34,  7.58s/it]
  4%|▍         | 9616/225000 [30:27:21<608:30:23, 10.17s/it]
  4%|▍         | 9617/225000 [30:27:25<496:32:04,  8.30s/it]
  4%|▍         | 9618/225000 [30:27:32<468:17:58,  7.83s/it]
  4%|▍         | 9619/225000 [30:27:37<423:58:17,  7.09s/it]
  4%|▍         | 9620/225000 [30:27:50<527:50:53,  8.82s/it]
  4%|▍         | 9621/225000 [30:27:59<524:02:29,  8.76s/it]
  4%|▍         | 9622/225000 [30:28:03<445:27:15,  7.45s/it]
  4%|▍         | 9623/225000 [30:28:13<492:34:58,  8.23s/it]
  4%|▍         | 9624/225000 [30:28:18<425:16:04,  7.11s/it]
  4%|▍         | 9625/225000 [30:28:30<523:03:45,  8.74s/it]
  4%|▍         | 9626/225000 [30:28:37<478:15:49,  7.99s/it]
  4%|▍         | 9627/225000 [30:28:41<405:53:30,  6.78s/it]
  4%|▍         | 9628/225000 [30:28:45<364:51:36,  6.10s/it]
  4%|▍         | 9629/225000 [30:28:55<436:20:14,  7.29s/it]
  4%|▍         | 9630/225000 [30:28:59<372:26:31,  6.23s/it]
  4%|▍         | 9631/225000 [30:29:05<367:25:22,  6.14s/it]
  4%|▍         | 9632/225000 [30:29:13<409:43:10,  6.85s/it]
  4%|▍         | 9633/225000 [30:29:18<369:33:26,  6.18s/it]
  4%|▍         | 9634/225000 [30:29:23<351:08:25,  5.87s/it]
  4%|▍         | 9635/225000 [30:29:31<390:15:24,  6.52s/it]
  4%|▍         | 9636/225000 [30:29:40<436:09:37,  7.29s/it]
  4%|▍         | 9637/225000 [30:29:48<443:28:50,  7.41s/it]
  4%|▍         | 9638/225000 [30:29:52<389:30:58,  6.51s/it]
  4%|▍         | 9639/225000 [30:30:01<429:18:22,  7.18s/it]
  4%|▍         | 9640/225000 [30:30:09<447:00:21,  7.47s/it]
  4%|▍         | 9641/225000 [30:30:14<407:11:19,  6.81s/it]
  4%|▍         | 9642/225000 [30:30:20<377:45:45,  6.31s/it]
  4%|▍         | 9643/225000 [30:30:30<445:05:33,  7.44s/it]
  4%|▍         | 9644/225000 [30:30:37<446:08:03,  7.46s/it]
  4%|▍         | 9645/225000 [30:30:41<388:06:34,  6.49s/it]
  4%|▍         | 9646/225000 [30:30:50<423:13:44,  7.07s/it]
  4%|▍         | 9647/225000 [30:31:04<543:49:07,  9.09s/it]
  4%|▍         | 9648/225000 [30:31:09<474:21:40,  7.93s/it]
  4%|▍         | 9649/225000 [30:31:14<427:51:45,  7.15s/it]
  4%|▍         | 9650/225000 [30:31:19<387:33:56,  6.48s/it]
                                                            
{'loss': '0.7837', 'grad_norm': '0.7236', 'learning_rate': '4.821e-05', 'epoch': '0.04289', 'num_input_tokens_seen': 54947848, 'train_runtime': '1.099e+05', 'train_tokens_per_second': '500.1'}

  4%|▍         | 9650/225000 [30:31:19<387:33:56,  6.48s/it]
  4%|▍         | 9651/225000 [30:31:25<378:10:20,  6.32s/it]
  4%|▍         | 9652/225000 [30:31:30<358:41:52,  6.00s/it]
  4%|▍         | 9653/225000 [30:31:37<365:57:55,  6.12s/it]
  4%|▍         | 9654/225000 [30:31:42<356:16:36,  5.96s/it]
  4%|▍         | 9655/225000 [30:31:48<356:16:18,  5.96s/it]
  4%|▍         | 9656/225000 [30:31:56<387:56:41,  6.49s/it]
  4%|▍         | 9657/225000 [30:32:09<501:23:42,  8.38s/it]
  4%|▍         | 9658/225000 [30:32:16<477:25:26,  7.98s/it]
  4%|▍         | 9659/225000 [30:32:24<482:34:26,  8.07s/it]
  4%|▍         | 9660/225000 [30:32:36<546:10:49,  9.13s/it]
  4%|▍         | 9661/225000 [30:32:44<523:04:18,  8.74s/it]
  4%|▍         | 9662/225000 [30:32:51<505:29:40,  8.45s/it]
  4%|▍         | 9663/225000 [30:32:57<451:02:31,  7.54s/it]
  4%|▍         | 9664/225000 [30:33:03<419:35:24,  7.01s/it]
  4%|▍         | 9665/225000 [30:33:14<500:32:06,  8.37s/it]
  4%|▍         | 9666/225000 [30:33:18<425:18:58,  7.11s/it]
  4%|▍         | 9667/225000 [30:33:30<514:47:34,  8.61s/it]
  4%|▍         | 9668/225000 [30:33:37<475:37:12,  7.95s/it]
  4%|▍         | 9669/225000 [30:33:43<436:54:11,  7.30s/it]
  4%|▍         | 9670/225000 [30:33:48<405:59:35,  6.79s/it]
  4%|▍         | 9671/225000 [30:33:58<461:48:46,  7.72s/it]
  4%|▍         | 9672/225000 [30:34:05<448:36:50,  7.50s/it]
  4%|▍         | 9673/225000 [30:34:08<375:43:14,  6.28s/it]
  4%|▍         | 9674/225000 [30:34:16<400:15:58,  6.69s/it]
  4%|▍         | 9675/225000 [30:34:21<372:51:49,  6.23s/it]
  4%|▍         | 9676/225000 [30:34:29<404:40:35,  6.77s/it]
  4%|▍         | 9677/225000 [30:34:34<367:04:18,  6.14s/it]
  4%|▍         | 9678/225000 [30:34:39<347:41:39,  5.81s/it]
  4%|▍         | 9679/225000 [30:34:45<344:25:47,  5.76s/it]
  4%|▍         | 9680/225000 [30:34:50<338:45:10,  5.66s/it]
  4%|▍         | 9681/225000 [30:34:56<340:25:12,  5.69s/it]
  4%|▍         | 9682/225000 [30:35:11<510:51:21,  8.54s/it]
  4%|▍         | 9683/225000 [30:35:18<479:09:39,  8.01s/it]
  4%|▍         | 9684/225000 [30:35:22<414:21:58,  6.93s/it]
  4%|▍         | 9685/225000 [30:35:31<442:32:42,  7.40s/it]
  4%|▍         | 9686/225000 [30:35:42<510:43:20,  8.54s/it]
  4%|▍         | 9687/225000 [30:35:52<534:11:37,  8.93s/it]
  4%|▍         | 9688/225000 [30:35:57<460:36:02,  7.70s/it]
  4%|▍         | 9689/225000 [30:36:02<416:32:03,  6.96s/it]
  4%|▍         | 9690/225000 [30:36:07<386:22:49,  6.46s/it]
  4%|▍         | 9691/225000 [30:36:12<367:39:19,  6.15s/it]
  4%|▍         | 9692/225000 [30:36:19<378:58:57,  6.34s/it]
  4%|▍         | 9693/225000 [30:36:32<488:26:26,  8.17s/it]
  4%|▍         | 9694/225000 [30:36:36<412:01:19,  6.89s/it]
  4%|▍         | 9695/225000 [30:36:41<377:31:29,  6.31s/it]
  4%|▍         | 9696/225000 [30:36:50<427:27:37,  7.15s/it]
  4%|▍         | 9697/225000 [30:37:01<496:20:42,  8.30s/it]
  4%|▍         | 9698/225000 [30:37:11<535:53:57,  8.96s/it]
  4%|▍         | 9699/225000 [30:37:16<467:05:11,  7.81s/it]
  4%|▍         | 9700/225000 [30:37:27<520:49:05,  8.71s/it]
                                                            
{'loss': '0.7521', 'grad_norm': '0.9559', 'learning_rate': '4.846e-05', 'epoch': '0.04311', 'num_input_tokens_seen': 55120864, 'train_runtime': '1.102e+05', 'train_tokens_per_second': '500'}

  4%|▍         | 9700/225000 [30:37:27<520:49:05,  8.71s/it]
  4%|▍         | 9701/225000 [30:37:34<482:04:22,  8.06s/it]
  4%|▍         | 9702/225000 [30:37:42<492:03:19,  8.23s/it]
  4%|▍         | 9703/225000 [30:37:46<420:03:26,  7.02s/it]
  4%|▍         | 9704/225000 [30:37:56<470:42:01,  7.87s/it]
  4%|▍         | 9705/225000 [30:38:03<449:03:59,  7.51s/it]
  4%|▍         | 9706/225000 [30:38:09<420:00:25,  7.02s/it]
  4%|▍         | 9707/225000 [30:38:15<408:01:17,  6.82s/it]
  4%|▍         | 9708/225000 [30:38:22<399:37:48,  6.68s/it]
  4%|▍         | 9709/225000 [30:38:32<470:04:21,  7.86s/it]
  4%|▍         | 9710/225000 [30:38:37<411:31:38,  6.88s/it]
  4%|▍         | 9711/225000 [30:38:44<408:51:25,  6.84s/it]
  4%|▍         | 9712/225000 [30:38:48<370:58:34,  6.20s/it]
  4%|▍         | 9713/225000 [30:38:59<458:26:31,  7.67s/it]
  4%|▍         | 9714/225000 [30:39:04<395:44:04,  6.62s/it]
  4%|▍         | 9715/225000 [30:39:16<502:04:19,  8.40s/it]
  4%|▍         | 9716/225000 [30:39:23<474:52:14,  7.94s/it]
  4%|▍         | 9717/225000 [30:39:28<420:54:22,  7.04s/it]
  4%|▍         | 9718/225000 [30:39:32<372:50:56,  6.23s/it]
  4%|▍         | 9719/225000 [30:39:37<354:27:16,  5.93s/it]
  4%|▍         | 9720/225000 [30:39:48<438:09:04,  7.33s/it]
  4%|▍         | 9721/225000 [30:39:51<366:54:00,  6.14s/it]
  4%|▍         | 9722/225000 [30:40:01<431:53:40,  7.22s/it]
  4%|▍         | 9723/225000 [30:40:08<418:18:44,  7.00s/it]
  4%|▍         | 9724/225000 [30:40:19<490:29:35,  8.20s/it]
  4%|▍         | 9725/225000 [30:40:22<407:18:35,  6.81s/it]
  4%|▍         | 9726/225000 [30:40:33<471:11:06,  7.88s/it]
  4%|▍         | 9727/225000 [30:40:37<401:57:41,  6.72s/it]
  4%|▍         | 9728/225000 [30:40:41<357:38:48,  5.98s/it]
  4%|▍         | 9729/225000 [30:40:51<425:40:16,  7.12s/it]
  4%|▍         | 9730/225000 [30:40:55<372:43:24,  6.23s/it]
  4%|▍         | 9731/225000 [30:41:00<362:01:21,  6.05s/it]
  4%|▍         | 9732/225000 [30:41:08<396:46:29,  6.64s/it]
  4%|▍         | 9733/225000 [30:41:17<433:49:33,  7.26s/it]
  4%|▍         | 9734/225000 [30:41:22<396:19:20,  6.63s/it]
  4%|▍         | 9735/225000 [30:41:32<454:40:34,  7.60s/it]
  4%|▍         | 9736/225000 [30:41:43<505:02:10,  8.45s/it]
  4%|▍         | 9737/225000 [30:41:47<433:31:16,  7.25s/it]
  4%|▍         | 9738/225000 [30:41:53<412:20:07,  6.90s/it]
  4%|▍         | 9739/225000 [30:41:59<390:43:22,  6.53s/it]
  4%|▍         | 9740/225000 [30:42:03<344:14:49,  5.76s/it]
  4%|▍         | 9741/225000 [30:42:11<381:23:40,  6.38s/it]
  4%|▍         | 9742/225000 [30:42:16<360:34:38,  6.03s/it]
  4%|▍         | 9743/225000 [30:42:22<357:46:15,  5.98s/it]
  4%|▍         | 9744/225000 [30:42:31<418:37:11,  7.00s/it]
  4%|▍         | 9745/225000 [30:42:36<375:39:40,  6.28s/it]
  4%|▍         | 9746/225000 [30:42:41<366:59:02,  6.14s/it]
  4%|▍         | 9747/225000 [30:42:48<367:16:01,  6.14s/it]
  4%|▍         | 9748/225000 [30:42:58<445:34:53,  7.45s/it]
  4%|▍         | 9749/225000 [30:43:05<427:35:06,  7.15s/it]
  4%|▍         | 9750/225000 [30:43:16<502:07:58,  8.40s/it]
                                                            
{'loss': '0.7422', 'grad_norm': '0.7006', 'learning_rate': '4.871e-05', 'epoch': '0.04333', 'num_input_tokens_seen': 55285624, 'train_runtime': '1.106e+05', 'train_tokens_per_second': '499.9'}

  4%|▍         | 9750/225000 [30:43:16<502:07:58,  8.40s/it]
  4%|▍         | 9751/225000 [30:43:22<453:02:00,  7.58s/it]
  4%|▍         | 9752/225000 [30:43:28<433:19:03,  7.25s/it]
  4%|▍         | 9753/225000 [30:43:33<389:25:54,  6.51s/it]
  4%|▍         | 9754/225000 [30:43:44<469:32:37,  7.85s/it]
  4%|▍         | 9755/225000 [30:43:55<529:44:31,  8.86s/it]
  4%|▍         | 9756/225000 [30:44:05<543:15:18,  9.09s/it]
  4%|▍         | 9757/225000 [30:44:10<484:00:33,  8.10s/it]
  4%|▍         | 9758/225000 [30:44:15<420:00:14,  7.02s/it]
  4%|▍         | 9759/225000 [30:44:22<422:45:06,  7.07s/it]
  4%|▍         | 9760/225000 [30:44:30<437:49:27,  7.32s/it]
  4%|▍         | 9761/225000 [30:44:43<533:59:50,  8.93s/it]
  4%|▍         | 9762/225000 [30:44:52<545:46:56,  9.13s/it]
  4%|▍         | 9763/225000 [30:44:59<497:22:46,  8.32s/it]
  4%|▍         | 9764/225000 [30:45:08<523:41:30,  8.76s/it]
  4%|▍         | 9765/225000 [30:45:18<539:02:39,  9.02s/it]
  4%|▍         | 9766/225000 [30:45:22<443:32:45,  7.42s/it]
  4%|▍         | 9767/225000 [30:45:29<447:27:08,  7.48s/it]
  4%|▍         | 9768/225000 [30:45:36<426:48:24,  7.14s/it]
  4%|▍         | 9769/225000 [30:45:42<415:32:41,  6.95s/it]
  4%|▍         | 9770/225000 [30:45:47<380:55:13,  6.37s/it]
  4%|▍         | 9771/225000 [30:45:53<375:36:51,  6.28s/it]
  4%|▍         | 9772/225000 [30:46:04<448:49:34,  7.51s/it]
  4%|▍         | 9773/225000 [30:46:09<416:36:39,  6.97s/it]
  4%|▍         | 9774/225000 [30:46:15<383:38:04,  6.42s/it]
  4%|▍         | 9775/225000 [30:46:25<462:43:59,  7.74s/it]
  4%|▍         | 9776/225000 [30:46:30<401:59:21,  6.72s/it]
  4%|▍         | 9777/225000 [30:46:41<476:32:24,  7.97s/it]
  4%|▍         | 9778/225000 [30:46:49<487:26:43,  8.15s/it]
  4%|▍         | 9779/225000 [30:46:57<472:44:12,  7.91s/it]
  4%|▍         | 9780/225000 [30:47:02<434:22:51,  7.27s/it]
  4%|▍         | 9781/225000 [30:47:07<395:38:12,  6.62s/it]
  4%|▍         | 9782/225000 [30:47:13<375:06:52,  6.27s/it]
  4%|▍         | 9783/225000 [30:47:19<367:56:02,  6.15s/it]
  4%|▍         | 9784/225000 [30:47:32<502:28:20,  8.41s/it]
  4%|▍         | 9785/225000 [30:47:39<466:07:24,  7.80s/it]
  4%|▍         | 9786/225000 [30:47:46<448:48:54,  7.51s/it]
  4%|▍         | 9787/225000 [30:47:50<399:36:29,  6.68s/it]
  4%|▍         | 9788/225000 [30:47:59<433:38:27,  7.25s/it]
  4%|▍         | 9789/225000 [30:48:07<449:43:38,  7.52s/it]
  4%|▍         | 9790/225000 [30:48:12<406:26:52,  6.80s/it]
  4%|▍         | 9791/225000 [30:48:20<424:22:33,  7.10s/it]
  4%|▍         | 9792/225000 [30:48:33<527:14:51,  8.82s/it]
  4%|▍         | 9793/225000 [30:48:38<462:40:29,  7.74s/it]
  4%|▍         | 9794/225000 [30:48:43<404:21:56,  6.76s/it]
  4%|▍         | 9795/225000 [30:48:47<370:51:38,  6.20s/it]
  4%|▍         | 9796/225000 [30:48:59<462:39:12,  7.74s/it]
  4%|▍         | 9797/225000 [30:49:04<414:01:44,  6.93s/it]
  4%|▍         | 9798/225000 [30:49:10<399:10:50,  6.68s/it]
  4%|▍         | 9799/225000 [30:49:19<435:06:59,  7.28s/it]
  4%|▍         | 9800/225000 [30:49:24<400:17:20,  6.70s/it]
                                                            
{'loss': '0.7386', 'grad_norm': '0.7377', 'learning_rate': '4.896e-05', 'epoch': '0.04356', 'num_input_tokens_seen': 55459400, 'train_runtime': '1.11e+05', 'train_tokens_per_second': '499.8'}

  4%|▍         | 9800/225000 [30:49:24<400:17:20,  6.70s/it]
  4%|▍         | 9801/225000 [30:49:29<378:03:43,  6.32s/it]
  4%|▍         | 9802/225000 [30:49:39<432:41:53,  7.24s/it]
  4%|▍         | 9803/225000 [30:49:45<414:09:38,  6.93s/it]
  4%|▍         | 9804/225000 [30:49:51<390:38:24,  6.53s/it]
  4%|▍         | 9805/225000 [30:49:57<394:17:17,  6.60s/it]
  4%|▍         | 9806/225000 [30:50:08<464:34:26,  7.77s/it]
  4%|▍         | 9807/225000 [30:50:13<426:19:46,  7.13s/it]
  4%|▍         | 9808/225000 [30:50:18<375:09:27,  6.28s/it]
  4%|▍         | 9809/225000 [30:50:25<387:54:48,  6.49s/it]
  4%|▍         | 9810/225000 [30:50:32<405:20:46,  6.78s/it]
  4%|▍         | 9811/225000 [30:50:39<404:10:31,  6.76s/it]
  4%|▍         | 9812/225000 [30:50:45<391:21:50,  6.55s/it]
  4%|▍         | 9813/225000 [30:50:49<354:40:32,  5.93s/it]
  4%|▍         | 9814/225000 [30:51:00<439:06:15,  7.35s/it]
  4%|▍         | 9815/225000 [30:51:08<452:34:44,  7.57s/it]
  4%|▍         | 9816/225000 [30:51:15<438:45:40,  7.34s/it]
  4%|▍         | 9817/225000 [30:51:20<392:00:50,  6.56s/it]
  4%|▍         | 9818/225000 [30:51:24<351:36:59,  5.88s/it]
  4%|▍         | 9819/225000 [30:51:28<316:38:58,  5.30s/it]
  4%|▍         | 9820/225000 [30:51:39<416:16:00,  6.96s/it]
  4%|▍         | 9821/225000 [30:51:48<446:49:51,  7.48s/it]
  4%|▍         | 9822/225000 [30:51:52<395:23:40,  6.62s/it]
  4%|▍         | 9823/225000 [30:52:00<419:29:38,  7.02s/it]
  4%|▍         | 9824/225000 [30:52:11<485:23:17,  8.12s/it]
  4%|▍         | 9825/225000 [30:52:21<517:40:40,  8.66s/it]
  4%|▍         | 9826/225000 [30:52:25<438:26:22,  7.34s/it]
  4%|▍         | 9827/225000 [30:52:29<380:46:00,  6.37s/it]
  4%|▍         | 9828/225000 [30:52:42<497:45:33,  8.33s/it]
  4%|▍         | 9829/225000 [30:52:48<463:47:38,  7.76s/it]
  4%|▍         | 9830/225000 [30:52:58<504:44:04,  8.44s/it]
  4%|▍         | 9831/225000 [30:53:03<442:52:55,  7.41s/it]
  4%|▍         | 9832/225000 [30:53:09<402:09:50,  6.73s/it]
  4%|▍         | 9833/225000 [30:53:22<514:01:20,  8.60s/it]
  4%|▍         | 9834/225000 [30:53:26<432:01:17,  7.23s/it]
  4%|▍         | 9835/225000 [30:53:33<436:16:47,  7.30s/it]
  4%|▍         | 9836/225000 [30:53:38<398:33:32,  6.67s/it]
  4%|▍         | 9837/225000 [30:53:42<355:39:33,  5.95s/it]
  4%|▍         | 9838/225000 [30:53:50<383:25:07,  6.42s/it]
  4%|▍         | 9839/225000 [30:53:59<426:18:07,  7.13s/it]
  4%|▍         | 9840/225000 [30:54:06<431:59:16,  7.23s/it]
  4%|▍         | 9841/225000 [30:54:10<378:12:16,  6.33s/it]
  4%|▍         | 9842/225000 [30:54:20<439:30:27,  7.35s/it]
  4%|▍         | 9843/225000 [30:54:27<422:29:35,  7.07s/it]
  4%|▍         | 9844/225000 [30:54:37<476:55:34,  7.98s/it]
  4%|▍         | 9845/225000 [30:54:42<426:46:49,  7.14s/it]
  4%|▍         | 9846/225000 [30:54:46<377:42:04,  6.32s/it]
  4%|▍         | 9847/225000 [30:54:53<375:55:53,  6.29s/it]
  4%|▍         | 9848/225000 [30:55:03<454:01:04,  7.60s/it]
  4%|▍         | 9849/225000 [30:55:14<510:26:10,  8.54s/it]
  4%|▍         | 9850/225000 [30:55:26<577:49:20,  9.67s/it]
                                                            
{'loss': '0.7585', 'grad_norm': '0.8117', 'learning_rate': '4.921e-05', 'epoch': '0.04378', 'num_input_tokens_seen': 55634360, 'train_runtime': '1.113e+05', 'train_tokens_per_second': '499.7'}

  4%|▍         | 9850/225000 [30:55:26<577:49:20,  9.67s/it]
  4%|▍         | 9851/225000 [30:55:34<546:51:30,  9.15s/it]
  4%|▍         | 9852/225000 [30:55:39<477:20:15,  7.99s/it]
  4%|▍         | 9853/225000 [30:55:44<414:36:40,  6.94s/it]
  4%|▍         | 9854/225000 [30:55:49<381:01:24,  6.38s/it]
  4%|▍         | 9855/225000 [30:55:55<365:37:30,  6.12s/it]
  4%|▍         | 9856/225000 [30:56:00<351:29:52,  5.88s/it]
  4%|▍         | 9857/225000 [30:56:13<478:21:35,  8.00s/it]
  4%|▍         | 9858/225000 [30:56:21<484:09:07,  8.10s/it]
  4%|▍         | 9859/225000 [30:56:25<403:39:34,  6.75s/it]
  4%|▍         | 9860/225000 [30:56:31<396:20:42,  6.63s/it]
  4%|▍         | 9861/225000 [30:56:38<401:31:10,  6.72s/it]
  4%|▍         | 9862/225000 [30:56:42<360:15:41,  6.03s/it]
  4%|▍         | 9863/225000 [30:56:52<428:12:20,  7.17s/it]
  4%|▍         | 9864/225000 [30:57:00<446:25:07,  7.47s/it]
  4%|▍         | 9865/225000 [30:57:05<392:05:47,  6.56s/it]
  4%|▍         | 9866/225000 [30:57:12<401:43:15,  6.72s/it]
  4%|▍         | 9867/225000 [30:57:16<360:11:08,  6.03s/it]
  4%|▍         | 9868/225000 [30:57:25<411:17:17,  6.88s/it]
  4%|▍         | 9869/225000 [30:57:33<420:46:21,  7.04s/it]
  4%|▍         | 9870/225000 [30:57:36<358:45:28,  6.00s/it]
  4%|▍         | 9871/225000 [30:57:47<449:44:54,  7.53s/it]
  4%|▍         | 9872/225000 [30:57:52<402:34:56,  6.74s/it]
  4%|▍         | 9873/225000 [30:58:00<423:14:37,  7.08s/it]
  4%|▍         | 9874/225000 [30:58:04<372:01:07,  6.23s/it]
  4%|▍         | 9875/225000 [30:58:13<423:57:45,  7.09s/it]
  4%|▍         | 9876/225000 [30:58:19<403:41:08,  6.76s/it]
  4%|▍         | 9877/225000 [30:58:25<377:06:28,  6.31s/it]
  4%|▍         | 9878/225000 [30:58:32<398:27:44,  6.67s/it]
  4%|▍         | 9879/225000 [30:58:39<398:22:37,  6.67s/it]
  4%|▍         | 9880/225000 [30:58:49<452:51:06,  7.58s/it]
  4%|▍         | 9881/225000 [30:58:53<403:22:13,  6.75s/it]
  4%|▍         | 9882/225000 [30:59:04<466:11:35,  7.80s/it]
  4%|▍         | 9883/225000 [30:59:09<417:09:44,  6.98s/it]
  4%|▍         | 9884/225000 [30:59:19<469:47:35,  7.86s/it]
  4%|▍         | 9885/225000 [30:59:26<453:05:10,  7.58s/it]
  4%|▍         | 9886/225000 [30:59:31<413:35:32,  6.92s/it]
  4%|▍         | 9887/225000 [30:59:39<429:30:16,  7.19s/it]
  4%|▍         | 9888/225000 [30:59:49<476:28:38,  7.97s/it]
  4%|▍         | 9889/225000 [30:59:56<464:18:13,  7.77s/it]
  4%|▍         | 9890/225000 [31:00:01<425:40:42,  7.12s/it]
  4%|▍         | 9891/225000 [31:00:06<379:05:00,  6.34s/it]
  4%|▍         | 9892/225000 [31:00:19<503:16:19,  8.42s/it]
  4%|▍         | 9893/225000 [31:00:25<449:54:07,  7.53s/it]
  4%|▍         | 9894/225000 [31:00:34<480:42:09,  8.05s/it]
  4%|▍         | 9895/225000 [31:00:40<437:45:00,  7.33s/it]
  4%|▍         | 9896/225000 [31:00:44<391:22:10,  6.55s/it]
  4%|▍         | 9897/225000 [31:00:52<406:54:22,  6.81s/it]
  4%|▍         | 9898/225000 [31:01:05<525:16:13,  8.79s/it]
  4%|▍         | 9899/225000 [31:01:14<532:58:45,  8.92s/it]
  4%|▍         | 9900/225000 [31:01:18<439:18:50,  7.35s/it]
                                                            
{'loss': '0.762', 'grad_norm': '0.8182', 'learning_rate': '4.946e-05', 'epoch': '0.044', 'num_input_tokens_seen': 55805776, 'train_runtime': '1.117e+05', 'train_tokens_per_second': '499.7'}

  4%|▍         | 9900/225000 [31:01:18<439:18:50,  7.35s/it]
  4%|▍         | 9901/225000 [31:01:23<401:04:41,  6.71s/it]
  4%|▍         | 9902/225000 [31:01:33<448:30:34,  7.51s/it]
  4%|▍         | 9903/225000 [31:01:42<484:31:13,  8.11s/it]
  4%|▍         | 9904/225000 [31:01:48<435:05:21,  7.28s/it]
  4%|▍         | 9905/225000 [31:01:53<402:10:55,  6.73s/it]
  4%|▍         | 9906/225000 [31:01:58<378:10:51,  6.33s/it]
  4%|▍         | 9907/225000 [31:02:04<358:34:29,  6.00s/it]
  4%|▍         | 9908/225000 [31:02:16<473:29:25,  7.92s/it]
  4%|▍         | 9909/225000 [31:02:25<490:06:20,  8.20s/it]
  4%|▍         | 9910/225000 [31:02:36<541:58:56,  9.07s/it]
  4%|▍         | 9911/225000 [31:02:40<444:59:09,  7.45s/it]
  4%|▍         | 9912/225000 [31:02:49<478:36:59,  8.01s/it]
  4%|▍         | 9913/225000 [31:02:54<420:13:45,  7.03s/it]
  4%|▍         | 9914/225000 [31:02:59<388:05:32,  6.50s/it]
  4%|▍         | 9915/225000 [31:03:04<363:19:15,  6.08s/it]
  4%|▍         | 9916/225000 [31:03:18<501:16:34,  8.39s/it]
  4%|▍         | 9917/225000 [31:03:22<425:55:12,  7.13s/it]
  4%|▍         | 9918/225000 [31:03:27<392:29:37,  6.57s/it]
  4%|▍         | 9919/225000 [31:03:38<463:37:03,  7.76s/it]
  4%|▍         | 9920/225000 [31:03:43<424:16:11,  7.10s/it]
  4%|▍         | 9921/225000 [31:03:52<448:28:11,  7.51s/it]
  4%|▍         | 9922/225000 [31:03:58<420:39:58,  7.04s/it]
  4%|▍         | 9923/225000 [31:04:06<443:16:43,  7.42s/it]
  4%|▍         | 9924/225000 [31:04:11<392:22:59,  6.57s/it]
  4%|▍         | 9925/225000 [31:04:15<359:33:46,  6.02s/it]
  4%|▍         | 9926/225000 [31:04:20<334:17:06,  5.60s/it]
  4%|▍         | 9927/225000 [31:04:29<393:49:50,  6.59s/it]
  4%|▍         | 9928/225000 [31:04:37<427:08:56,  7.15s/it]
  4%|▍         | 9929/225000 [31:04:46<446:50:05,  7.48s/it]
  4%|▍         | 9930/225000 [31:04:49<381:39:58,  6.39s/it]
  4%|▍         | 9931/225000 [31:05:04<521:09:19,  8.72s/it]
  4%|▍         | 9932/225000 [31:05:08<437:29:16,  7.32s/it]
  4%|▍         | 9933/225000 [31:05:14<412:38:22,  6.91s/it]
  4%|▍         | 9934/225000 [31:05:19<389:17:35,  6.52s/it]
  4%|▍         | 9935/225000 [31:05:30<459:45:49,  7.70s/it]
  4%|▍         | 9936/225000 [31:05:35<418:01:01,  7.00s/it]
  4%|▍         | 9937/225000 [31:05:41<393:22:49,  6.58s/it]
  4%|▍         | 9938/225000 [31:05:46<370:20:48,  6.20s/it]
  4%|▍         | 9939/225000 [31:05:51<348:29:35,  5.83s/it]
  4%|▍         | 9940/225000 [31:06:00<412:45:48,  6.91s/it]
  4%|▍         | 9941/225000 [31:06:11<480:15:41,  8.04s/it]
  4%|▍         | 9942/225000 [31:06:15<414:52:18,  6.94s/it]
  4%|▍         | 9943/225000 [31:06:20<363:05:50,  6.08s/it]
  4%|▍         | 9944/225000 [31:06:27<385:43:16,  6.46s/it]
  4%|▍         | 9945/225000 [31:06:34<398:24:52,  6.67s/it]
  4%|▍         | 9946/225000 [31:06:41<410:45:51,  6.88s/it]
  4%|▍         | 9947/225000 [31:06:46<372:22:49,  6.23s/it]
  4%|▍         | 9948/225000 [31:06:54<396:27:30,  6.64s/it]
  4%|▍         | 9949/225000 [31:07:01<413:47:04,  6.93s/it]
  4%|▍         | 9950/225000 [31:07:07<385:09:53,  6.45s/it]
                                                            
{'loss': '0.7269', 'grad_norm': '0.6891', 'learning_rate': '4.971e-05', 'epoch': '0.04422', 'num_input_tokens_seen': 55973448, 'train_runtime': '1.12e+05', 'train_tokens_per_second': '499.6'}

  4%|▍         | 9950/225000 [31:07:07<385:09:53,  6.45s/it]
  4%|▍         | 9951/225000 [31:07:11<354:30:02,  5.93s/it]
  4%|▍         | 9952/225000 [31:07:23<451:58:17,  7.57s/it]
  4%|▍         | 9953/225000 [31:07:30<454:59:36,  7.62s/it]
  4%|▍         | 9954/225000 [31:07:40<491:23:50,  8.23s/it]
  4%|▍         | 9955/225000 [31:07:47<467:38:05,  7.83s/it]
  4%|▍         | 9956/225000 [31:07:51<407:30:00,  6.82s/it]
  4%|▍         | 9957/225000 [31:07:56<367:18:08,  6.15s/it]
  4%|▍         | 9958/225000 [31:08:07<457:26:05,  7.66s/it]
  4%|▍         | 9959/225000 [31:08:13<429:10:38,  7.18s/it]
  4%|▍         | 9960/225000 [31:08:23<476:55:22,  7.98s/it]
  4%|▍         | 9961/225000 [31:08:29<437:22:41,  7.32s/it]
  4%|▍         | 9962/225000 [31:08:34<397:47:35,  6.66s/it]
  4%|▍         | 9963/225000 [31:08:46<491:13:59,  8.22s/it]
  4%|▍         | 9964/225000 [31:08:50<413:51:26,  6.93s/it]
  4%|▍         | 9965/225000 [31:08:54<371:30:19,  6.22s/it]
  4%|▍         | 9966/225000 [31:09:03<414:12:52,  6.93s/it]
  4%|▍         | 9967/225000 [31:09:09<392:50:36,  6.58s/it]
  4%|▍         | 9968/225000 [31:09:19<457:36:57,  7.66s/it]
  4%|▍         | 9969/225000 [31:09:27<461:45:30,  7.73s/it]
  4%|▍         | 9970/225000 [31:09:32<410:24:45,  6.87s/it]
  4%|▍         | 9971/225000 [31:09:39<419:52:15,  7.03s/it]
  4%|▍         | 9972/225000 [31:09:46<418:55:40,  7.01s/it]
  4%|▍         | 9973/225000 [31:09:54<440:04:03,  7.37s/it]
  4%|▍         | 9974/225000 [31:09:58<382:57:20,  6.41s/it]
  4%|▍         | 9975/225000 [31:10:03<347:40:32,  5.82s/it]
  4%|▍         | 9976/225000 [31:10:16<472:28:54,  7.91s/it]
  4%|▍         | 9977/225000 [31:10:25<496:58:23,  8.32s/it]
  4%|▍         | 9978/225000 [31:10:33<495:28:14,  8.30s/it]
  4%|▍         | 9979/225000 [31:10:39<444:22:42,  7.44s/it]
  4%|▍         | 9980/225000 [31:10:46<442:58:39,  7.42s/it]
  4%|▍         | 9981/225000 [31:10:50<386:59:31,  6.48s/it]
  4%|▍         | 9982/225000 [31:10:55<361:14:33,  6.05s/it]
  4%|▍         | 9983/225000 [31:11:04<409:34:09,  6.86s/it]
  4%|▍         | 9984/225000 [31:11:08<361:19:52,  6.05s/it]
  4%|▍         | 9985/225000 [31:11:13<339:09:58,  5.68s/it]
  4%|▍         | 9986/225000 [31:11:18<322:24:52,  5.40s/it]
  4%|▍         | 9987/225000 [31:11:29<421:54:25,  7.06s/it]
  4%|▍         | 9988/225000 [31:11:38<460:29:47,  7.71s/it]
  4%|▍         | 9989/225000 [31:11:46<461:32:26,  7.73s/it]
  4%|▍         | 9990/225000 [31:11:50<395:33:42,  6.62s/it]
  4%|▍         | 9991/225000 [31:11:57<397:19:18,  6.65s/it]
  4%|▍         | 9992/225000 [31:12:04<410:15:02,  6.87s/it]
  4%|▍         | 9993/225000 [31:12:09<382:10:59,  6.40s/it]
  4%|▍         | 9994/225000 [31:12:16<392:08:21,  6.57s/it]
  4%|▍         | 9995/225000 [31:12:23<404:05:38,  6.77s/it]
  4%|▍         | 9996/225000 [31:12:31<422:36:29,  7.08s/it]
  4%|▍         | 9997/225000 [31:12:45<538:13:16,  9.01s/it]
  4%|▍         | 9998/225000 [31:12:54<550:46:14,  9.22s/it]
  4%|▍         | 9999/225000 [31:12:59<470:22:53,  7.88s/it]
  4%|▍         | 10000/225000 [31:13:04<415:55:18,  6.96s/it]
                                                             
{'loss': '0.7663', 'grad_norm': '0.7033', 'learning_rate': '4.996e-05', 'epoch': '0.04444', 'num_input_tokens_seen': 56145104, 'train_runtime': '1.124e+05', 'train_tokens_per_second': '499.6'}

  4%|▍         | 10000/225000 [31:13:04<415:55:18,  6.96s/it]
  4%|▍         | 10001/225000 [31:13:12<429:52:53,  7.20s/it]
  4%|▍         | 10002/225000 [31:13:23<509:25:06,  8.53s/it]
  4%|▍         | 10003/225000 [31:13:28<442:07:59,  7.40s/it]
  4%|▍         | 10004/225000 [31:13:33<403:44:14,  6.76s/it]
  4%|▍         | 10005/225000 [31:13:39<375:31:56,  6.29s/it]
  4%|▍         | 10006/225000 [31:13:47<410:49:41,  6.88s/it]
  4%|▍         | 10007/225000 [31:13:54<420:54:18,  7.05s/it]
  4%|▍         | 10008/225000 [31:14:02<435:16:55,  7.29s/it]
  4%|▍         | 10009/225000 [31:14:11<465:57:25,  7.80s/it]
  4%|▍         | 10010/225000 [31:14:16<415:33:28,  6.96s/it]
  4%|▍         | 10011/225000 [31:14:29<526:17:20,  8.81s/it]
  4%|▍         | 10012/225000 [31:14:35<462:18:33,  7.74s/it]
  4%|▍         | 10013/225000 [31:14:48<566:28:15,  9.49s/it]
  4%|▍         | 10014/225000 [31:14:53<488:43:28,  8.18s/it]
  4%|▍         | 10015/225000 [31:15:02<501:38:19,  8.40s/it]
  4%|▍         | 10016/225000 [31:15:09<472:49:34,  7.92s/it]
  4%|▍         | 10017/225000 [31:15:14<425:52:13,  7.13s/it]
  4%|▍         | 10018/225000 [31:15:22<437:06:54,  7.32s/it]
  4%|▍         | 10019/225000 [31:15:27<400:14:34,  6.70s/it]
  4%|▍         | 10020/225000 [31:15:37<453:18:02,  7.59s/it]
  4%|▍         | 10021/225000 [31:15:45<469:50:07,  7.87s/it]
  4%|▍         | 10022/225000 [31:15:53<464:43:48,  7.78s/it]
  4%|▍         | 10023/225000 [31:15:58<419:50:56,  7.03s/it]
  4%|▍         | 10024/225000 [31:16:05<406:58:29,  6.82s/it]
  4%|▍         | 10025/225000 [31:16:10<376:51:32,  6.31s/it]
  4%|▍         | 10026/225000 [31:16:17<387:37:12,  6.49s/it]
  4%|▍         | 10027/225000 [31:16:22<374:28:09,  6.27s/it]
  4%|▍         | 10028/225000 [31:16:35<481:25:12,  8.06s/it]
  4%|▍         | 10029/225000 [31:16:44<508:24:59,  8.51s/it]
  4%|▍         | 10030/225000 [31:16:48<428:34:43,  7.18s/it]
  4%|▍         | 10031/225000 [31:16:57<464:22:32,  7.78s/it]
  4%|▍         | 10032/225000 [31:17:02<412:22:36,  6.91s/it]
  4%|▍         | 10033/225000 [31:17:12<461:41:39,  7.73s/it]
  4%|▍         | 10034/225000 [31:17:16<399:10:29,  6.68s/it]
  4%|▍         | 10035/225000 [31:17:22<388:41:04,  6.51s/it]
  4%|▍         | 10036/225000 [31:17:33<465:05:53,  7.79s/it]
  4%|▍         | 10037/225000 [31:17:38<420:22:46,  7.04s/it]
  4%|▍         | 10038/225000 [31:17:44<388:36:52,  6.51s/it]
  4%|▍         | 10039/225000 [31:17:56<495:21:37,  8.30s/it]
  4%|▍         | 10040/225000 [31:18:01<427:00:27,  7.15s/it]
  4%|▍         | 10041/225000 [31:18:13<520:42:40,  8.72s/it]
  4%|▍         | 10042/225000 [31:18:17<436:13:39,  7.31s/it]
  4%|▍         | 10043/225000 [31:18:22<395:49:15,  6.63s/it]
  4%|▍         | 10044/225000 [31:18:35<517:52:27,  8.67s/it]
  4%|▍         | 10045/225000 [31:18:42<475:54:22,  7.97s/it]
  4%|▍         | 10046/225000 [31:18:51<494:14:18,  8.28s/it]
  4%|▍         | 10047/225000 [31:18:58<474:31:50,  7.95s/it]
  4%|▍         | 10048/225000 [31:19:03<416:40:20,  6.98s/it]
  4%|▍         | 10049/225000 [31:19:07<372:04:58,  6.23s/it]
  4%|▍         | 10050/225000 [31:19:12<339:24:11,  5.68s/it]
                                                             
{'loss': '0.7498', 'grad_norm': '0.6542', 'learning_rate': '5e-05', 'epoch': '0.04467', 'num_input_tokens_seen': 56324768, 'train_runtime': '1.128e+05', 'train_tokens_per_second': '499.5'}

  4%|▍         | 10050/225000 [31:19:12<339:24:11,  5.68s/it]
  4%|▍         | 10051/225000 [31:19:21<399:16:06,  6.69s/it]
  4%|▍         | 10052/225000 [31:19:27<391:56:31,  6.56s/it]
  4%|▍         | 10053/225000 [31:19:32<374:34:38,  6.27s/it]
  4%|▍         | 10054/225000 [31:19:37<335:07:36,  5.61s/it]
  4%|▍         | 10055/225000 [31:19:42<331:46:08,  5.56s/it]
  4%|▍         | 10056/225000 [31:19:52<418:31:52,  7.01s/it]
  4%|▍         | 10057/225000 [31:19:56<363:39:37,  6.09s/it]
  4%|▍         | 10058/225000 [31:20:04<387:33:24,  6.49s/it]
  4%|▍         | 10059/225000 [31:20:12<418:31:00,  7.01s/it]
  4%|▍         | 10060/225000 [31:20:18<400:26:53,  6.71s/it]
  4%|▍         | 10061/225000 [31:20:23<364:03:11,  6.10s/it]
  4%|▍         | 10062/225000 [31:20:29<375:35:54,  6.29s/it]
  4%|▍         | 10063/225000 [31:20:35<367:19:12,  6.15s/it]
  4%|▍         | 10064/225000 [31:20:43<397:37:43,  6.66s/it]
  4%|▍         | 10065/225000 [31:20:49<388:56:35,  6.51s/it]
  4%|▍         | 10066/225000 [31:20:55<369:20:55,  6.19s/it]
  4%|▍         | 10067/225000 [31:21:00<358:06:16,  6.00s/it]
  4%|▍         | 10068/225000 [31:21:06<353:17:03,  5.92s/it]
  4%|▍         | 10069/225000 [31:21:20<497:25:25,  8.33s/it]
  4%|▍         | 10070/225000 [31:21:27<467:15:23,  7.83s/it]
  4%|▍         | 10071/225000 [31:21:33<450:46:55,  7.55s/it]
  4%|▍         | 10072/225000 [31:21:41<444:37:46,  7.45s/it]
  4%|▍         | 10073/225000 [31:21:45<395:45:00,  6.63s/it]
  4%|▍         | 10074/225000 [31:21:56<459:49:52,  7.70s/it]
  4%|▍         | 10075/225000 [31:22:01<421:40:47,  7.06s/it]
  4%|▍         | 10076/225000 [31:22:07<397:07:13,  6.65s/it]
  4%|▍         | 10077/225000 [31:22:12<378:44:53,  6.34s/it]
  4%|▍         | 10078/225000 [31:22:23<460:11:42,  7.71s/it]
  4%|▍         | 10079/225000 [31:22:31<453:45:40,  7.60s/it]
  4%|▍         | 10080/225000 [31:22:36<413:50:04,  6.93s/it]
  4%|▍         | 10081/225000 [31:22:44<430:15:27,  7.21s/it]
  4%|▍         | 10082/225000 [31:22:49<390:57:43,  6.55s/it]
  4%|▍         | 10083/225000 [31:22:53<352:55:38,  5.91s/it]
  4%|▍         | 10084/225000 [31:22:59<339:42:55,  5.69s/it]
  4%|▍         | 10085/225000 [31:23:05<350:34:48,  5.87s/it]
  4%|▍         | 10086/225000 [31:23:22<544:48:34,  9.13s/it]
  4%|▍         | 10087/225000 [31:23:25<446:51:35,  7.49s/it]
  4%|▍         | 10088/225000 [31:23:33<457:28:57,  7.66s/it]
  4%|▍         | 10089/225000 [31:23:37<384:00:28,  6.43s/it]
  4%|▍         | 10090/225000 [31:23:47<449:43:49,  7.53s/it]
  4%|▍         | 10091/225000 [31:23:54<439:09:53,  7.36s/it]
  4%|▍         | 10092/225000 [31:23:59<403:35:09,  6.76s/it]
  4%|▍         | 10093/225000 [31:24:06<398:12:22,  6.67s/it]
  4%|▍         | 10094/225000 [31:24:17<480:10:26,  8.04s/it]
  4%|▍         | 10095/225000 [31:24:22<430:00:48,  7.20s/it]
  4%|▍         | 10096/225000 [31:24:32<484:09:05,  8.11s/it]
  4%|▍         | 10097/225000 [31:24:37<427:42:16,  7.16s/it]
  4%|▍         | 10098/225000 [31:24:43<402:41:17,  6.75s/it]
  4%|▍         | 10099/225000 [31:24:51<420:42:47,  7.05s/it]
  4%|▍         | 10100/225000 [31:24:57<394:13:57,  6.60s/it]
                                                             
{'loss': '0.7379', 'grad_norm': '0.8274', 'learning_rate': '5e-05', 'epoch': '0.04489', 'num_input_tokens_seen': 56491296, 'train_runtime': '1.131e+05', 'train_tokens_per_second': '499.5'}

  4%|▍         | 10100/225000 [31:24:57<394:13:57,  6.60s/it]
  4%|▍         | 10101/225000 [31:25:01<349:36:44,  5.86s/it]
  4%|▍         | 10102/225000 [31:25:08<370:22:50,  6.20s/it]
  4%|▍         | 10103/225000 [31:25:17<428:02:46,  7.17s/it]
  4%|▍         | 10104/225000 [31:25:25<442:24:22,  7.41s/it]
  4%|▍         | 10105/225000 [31:25:33<454:10:10,  7.61s/it]
  4%|▍         | 10106/225000 [31:25:40<440:00:37,  7.37s/it]
  4%|▍         | 10107/225000 [31:25:45<392:33:27,  6.58s/it]
  4%|▍         | 10108/225000 [31:25:57<492:17:27,  8.25s/it]
  4%|▍         | 10109/225000 [31:26:04<474:10:15,  7.94s/it]
  4%|▍         | 10110/225000 [31:26:10<433:06:53,  7.26s/it]
  4%|▍         | 10111/225000 [31:26:16<409:20:28,  6.86s/it]
  4%|▍         | 10112/225000 [31:26:21<380:14:46,  6.37s/it]
  4%|▍         | 10113/225000 [31:26:31<452:16:12,  7.58s/it]
  4%|▍         | 10114/225000 [31:26:40<470:58:28,  7.89s/it]
  4%|▍         | 10115/225000 [31:26:47<452:21:08,  7.58s/it]
  4%|▍         | 10116/225000 [31:26:57<494:44:10,  8.29s/it]
  4%|▍         | 10117/225000 [31:27:02<435:20:00,  7.29s/it]
  4%|▍         | 10118/225000 [31:27:09<433:45:14,  7.27s/it]
  4%|▍         | 10119/225000 [31:27:19<481:55:16,  8.07s/it]
  4%|▍         | 10120/225000 [31:27:23<411:35:57,  6.90s/it]
  4%|▍         | 10121/225000 [31:27:30<423:22:36,  7.09s/it]
  4%|▍         | 10122/225000 [31:27:36<392:24:16,  6.57s/it]
  4%|▍         | 10123/225000 [31:27:45<440:48:37,  7.39s/it]
  4%|▍         | 10124/225000 [31:27:54<462:40:08,  7.75s/it]
  4%|▍         | 10125/225000 [31:28:03<484:02:17,  8.11s/it]
  5%|▍         | 10126/225000 [31:28:11<484:36:24,  8.12s/it]
  5%|▍         | 10127/225000 [31:28:16<433:39:57,  7.27s/it]
  5%|▍         | 10128/225000 [31:28:23<426:35:44,  7.15s/it]
  5%|▍         | 10129/225000 [31:28:34<501:09:26,  8.40s/it]
  5%|▍         | 10130/225000 [31:28:41<468:02:09,  7.84s/it]
  5%|▍         | 10131/225000 [31:28:50<488:00:26,  8.18s/it]
  5%|▍         | 10132/225000 [31:28:54<409:52:48,  6.87s/it]
  5%|▍         | 10133/225000 [31:29:02<431:33:37,  7.23s/it]
  5%|▍         | 10134/225000 [31:29:06<373:42:31,  6.26s/it]
  5%|▍         | 10135/225000 [31:29:15<421:26:10,  7.06s/it]
  5%|▍         | 10136/225000 [31:29:23<453:10:10,  7.59s/it]
  5%|▍         | 10137/225000 [31:29:29<408:19:44,  6.84s/it]
  5%|▍         | 10138/225000 [31:29:34<384:48:40,  6.45s/it]
  5%|▍         | 10139/225000 [31:29:41<397:09:08,  6.65s/it]
  5%|▍         | 10140/225000 [31:29:49<420:34:23,  7.05s/it]
  5%|▍         | 10141/225000 [31:30:00<492:34:55,  8.25s/it]
  5%|▍         | 10142/225000 [31:30:04<417:45:45,  7.00s/it]
  5%|▍         | 10143/225000 [31:30:12<432:12:40,  7.24s/it]
  5%|▍         | 10144/225000 [31:30:18<400:56:48,  6.72s/it]
  5%|▍         | 10145/225000 [31:30:25<421:44:14,  7.07s/it]
  5%|▍         | 10146/225000 [31:30:34<439:11:36,  7.36s/it]
  5%|▍         | 10147/225000 [31:30:41<439:12:14,  7.36s/it]
  5%|▍         | 10148/225000 [31:30:52<499:48:55,  8.37s/it]
  5%|▍         | 10149/225000 [31:30:57<448:20:20,  7.51s/it]
  5%|▍         | 10150/225000 [31:31:03<411:55:32,  6.90s/it]
                                                             
{'loss': '0.7579', 'grad_norm': '0.7342', 'learning_rate': '5e-05', 'epoch': '0.04511', 'num_input_tokens_seen': 56667776, 'train_runtime': '1.135e+05', 'train_tokens_per_second': '499.4'}

  5%|▍         | 10150/225000 [31:31:03<411:55:32,  6.90s/it]
  5%|▍         | 10151/225000 [31:31:11<437:54:38,  7.34s/it]
  5%|▍         | 10152/225000 [31:31:20<467:26:53,  7.83s/it]
  5%|▍         | 10153/225000 [31:31:25<424:35:46,  7.11s/it]
  5%|▍         | 10154/225000 [31:31:32<408:13:03,  6.84s/it]
  5%|▍         | 10155/225000 [31:31:39<414:02:36,  6.94s/it]
  5%|▍         | 10156/225000 [31:31:47<430:35:24,  7.22s/it]
  5%|▍         | 10157/225000 [31:31:50<366:48:23,  6.15s/it]
  5%|▍         | 10158/225000 [31:31:59<411:16:34,  6.89s/it]
  5%|▍         | 10159/225000 [31:32:05<392:37:24,  6.58s/it]
  5%|▍         | 10160/225000 [31:32:09<359:22:14,  6.02s/it]
  5%|▍         | 10161/225000 [31:32:15<345:30:15,  5.79s/it]
  5%|▍         | 10162/225000 [31:32:22<377:19:47,  6.32s/it]
  5%|▍         | 10163/225000 [31:32:27<356:56:22,  5.98s/it]
  5%|▍         | 10164/225000 [31:32:33<347:19:34,  5.82s/it]
  5%|▍         | 10165/225000 [31:32:44<445:31:50,  7.47s/it]
  5%|▍         | 10166/225000 [31:32:49<403:11:33,  6.76s/it]
  5%|▍         | 10167/225000 [31:32:56<395:46:02,  6.63s/it]
  5%|▍         | 10168/225000 [31:33:07<484:09:16,  8.11s/it]
  5%|▍         | 10169/225000 [31:33:14<456:23:40,  7.65s/it]
  5%|▍         | 10170/225000 [31:33:18<397:54:22,  6.67s/it]
  5%|▍         | 10171/225000 [31:33:22<351:25:06,  5.89s/it]
  5%|▍         | 10172/225000 [31:33:37<513:04:28,  8.60s/it]
  5%|▍         | 10173/225000 [31:33:42<441:10:40,  7.39s/it]
  5%|▍         | 10174/225000 [31:33:51<473:39:06,  7.94s/it]
  5%|▍         | 10175/225000 [31:33:55<405:20:18,  6.79s/it]
  5%|▍         | 10176/225000 [31:34:00<370:01:48,  6.20s/it]
  5%|▍         | 10177/225000 [31:34:09<426:39:16,  7.15s/it]
  5%|▍         | 10178/225000 [31:34:14<380:29:01,  6.38s/it]
  5%|▍         | 10179/225000 [31:34:22<405:02:49,  6.79s/it]
  5%|▍         | 10180/225000 [31:34:30<436:45:21,  7.32s/it]
  5%|▍         | 10181/225000 [31:34:42<514:01:54,  8.61s/it]
  5%|▍         | 10182/225000 [31:34:46<430:45:01,  7.22s/it]
  5%|▍         | 10183/225000 [31:34:53<439:30:07,  7.37s/it]
  5%|▍         | 10184/225000 [31:34:58<388:21:47,  6.51s/it]
  5%|▍         | 10185/225000 [31:35:03<370:02:29,  6.20s/it]
  5%|▍         | 10186/225000 [31:35:12<411:11:56,  6.89s/it]
  5%|▍         | 10187/225000 [31:35:16<367:05:15,  6.15s/it]
  5%|▍         | 10188/225000 [31:35:21<342:06:26,  5.73s/it]
  5%|▍         | 10189/225000 [31:35:27<345:32:38,  5.79s/it]
  5%|▍         | 10190/225000 [31:35:40<468:20:45,  7.85s/it]
  5%|▍         | 10191/225000 [31:35:44<411:12:42,  6.89s/it]
  5%|▍         | 10192/225000 [31:35:51<406:02:29,  6.80s/it]
  5%|▍         | 10193/225000 [31:35:56<372:57:37,  6.25s/it]
  5%|▍         | 10194/225000 [31:36:00<342:07:59,  5.73s/it]
  5%|▍         | 10195/225000 [31:36:11<431:08:26,  7.23s/it]
  5%|▍         | 10196/225000 [31:36:19<441:20:18,  7.40s/it]
  5%|▍         | 10197/225000 [31:36:28<469:55:04,  7.88s/it]
  5%|▍         | 10198/225000 [31:36:32<400:46:13,  6.72s/it]
  5%|▍         | 10199/225000 [31:36:45<507:23:02,  8.50s/it]
  5%|▍         | 10200/225000 [31:36:52<492:08:18,  8.25s/it]
                                                             
{'loss': '0.76', 'grad_norm': '0.5683', 'learning_rate': '5e-05', 'epoch': '0.04533', 'num_input_tokens_seen': 56837512, 'train_runtime': '1.138e+05', 'train_tokens_per_second': '499.4'}

  5%|▍         | 10200/225000 [31:36:52<492:08:18,  8.25s/it]
  5%|▍         | 10201/225000 [31:36:57<436:44:30,  7.32s/it]
  5%|▍         | 10202/225000 [31:37:09<514:26:47,  8.62s/it]
  5%|▍         | 10203/225000 [31:37:18<526:58:56,  8.83s/it]
  5%|▍         | 10204/225000 [31:37:26<498:09:11,  8.35s/it]
  5%|▍         | 10205/225000 [31:37:30<430:28:34,  7.21s/it]
  5%|▍         | 10206/225000 [31:37:34<370:06:58,  6.20s/it]
  5%|▍         | 10207/225000 [31:37:48<513:29:27,  8.61s/it]
  5%|▍         | 10208/225000 [31:37:54<457:31:05,  7.67s/it]
  5%|▍         | 10209/225000 [31:37:58<392:48:13,  6.58s/it]
  5%|▍         | 10210/225000 [31:38:03<364:42:56,  6.11s/it]
  5%|▍         | 10211/225000 [31:38:08<341:01:02,  5.72s/it]
  5%|▍         | 10212/225000 [31:38:13<331:14:27,  5.55s/it]
  5%|▍         | 10213/225000 [31:38:18<323:05:14,  5.42s/it]
  5%|▍         | 10214/225000 [31:38:23<326:34:58,  5.47s/it]
  5%|▍         | 10215/225000 [31:38:37<465:26:15,  7.80s/it]
  5%|▍         | 10216/225000 [31:38:41<411:21:24,  6.89s/it]
  5%|▍         | 10217/225000 [31:38:47<378:20:42,  6.34s/it]
  5%|▍         | 10218/225000 [31:38:52<361:12:06,  6.05s/it]
  5%|▍         | 10219/225000 [31:38:58<366:42:52,  6.15s/it]
  5%|▍         | 10220/225000 [31:39:12<500:05:30,  8.38s/it]
  5%|▍         | 10221/225000 [31:39:17<433:32:22,  7.27s/it]
  5%|▍         | 10222/225000 [31:39:22<404:27:41,  6.78s/it]
  5%|▍         | 10223/225000 [31:39:35<507:40:14,  8.51s/it]
  5%|▍         | 10224/225000 [31:39:40<448:11:33,  7.51s/it]
  5%|▍         | 10225/225000 [31:39:46<430:06:52,  7.21s/it]
  5%|▍         | 10226/225000 [31:39:56<468:03:01,  7.85s/it]
  5%|▍         | 10227/225000 [31:40:02<431:47:15,  7.24s/it]
  5%|▍         | 10228/225000 [31:40:06<390:42:25,  6.55s/it]
  5%|▍         | 10229/225000 [31:40:13<383:40:26,  6.43s/it]
  5%|▍         | 10230/225000 [31:40:18<355:44:46,  5.96s/it]
  5%|▍         | 10231/225000 [31:40:23<347:35:50,  5.83s/it]
  5%|▍         | 10232/225000 [31:40:31<393:37:48,  6.60s/it]
  5%|▍         | 10233/225000 [31:40:40<427:48:08,  7.17s/it]
  5%|▍         | 10234/225000 [31:40:50<487:01:30,  8.16s/it]
  5%|▍         | 10235/225000 [31:40:54<404:33:24,  6.78s/it]
  5%|▍         | 10236/225000 [31:41:06<490:01:06,  8.21s/it]
  5%|▍         | 10237/225000 [31:41:11<445:40:06,  7.47s/it]
  5%|▍         | 10238/225000 [31:41:16<400:24:14,  6.71s/it]
  5%|▍         | 10239/225000 [31:41:25<440:53:30,  7.39s/it]
  5%|▍         | 10240/225000 [31:41:33<449:20:57,  7.53s/it]
  5%|▍         | 10241/225000 [31:41:39<426:26:06,  7.15s/it]
  5%|▍         | 10242/225000 [31:41:44<378:37:53,  6.35s/it]
  5%|▍         | 10243/225000 [31:41:49<366:37:52,  6.15s/it]
  5%|▍         | 10244/225000 [31:41:58<404:24:37,  6.78s/it]
  5%|▍         | 10245/225000 [31:42:05<408:40:44,  6.85s/it]
  5%|▍         | 10246/225000 [31:42:09<356:38:49,  5.98s/it]
  5%|▍         | 10247/225000 [31:42:16<384:42:08,  6.45s/it]
  5%|▍         | 10248/225000 [31:42:25<427:20:33,  7.16s/it]
  5%|▍         | 10249/225000 [31:42:30<393:38:05,  6.60s/it]
  5%|▍         | 10250/225000 [31:42:38<419:13:37,  7.03s/it]
                                                             
{'loss': '0.7419', 'grad_norm': '0.7133', 'learning_rate': '5e-05', 'epoch': '0.04556', 'num_input_tokens_seen': 57005576, 'train_runtime': '1.142e+05', 'train_tokens_per_second': '499.4'}

  5%|▍         | 10250/225000 [31:42:38<419:13:37,  7.03s/it]
  5%|▍         | 10251/225000 [31:42:43<370:51:53,  6.22s/it]
  5%|▍         | 10252/225000 [31:42:54<467:44:38,  7.84s/it]
  5%|▍         | 10253/225000 [31:43:02<464:04:27,  7.78s/it]
  5%|▍         | 10254/225000 [31:43:10<464:38:39,  7.79s/it]
  5%|▍         | 10255/225000 [31:43:18<480:16:26,  8.05s/it]
  5%|▍         | 10256/225000 [31:43:26<469:30:35,  7.87s/it]
  5%|▍         | 10257/225000 [31:43:37<534:20:48,  8.96s/it]
  5%|▍         | 10258/225000 [31:43:41<440:42:35,  7.39s/it]
  5%|▍         | 10259/225000 [31:43:45<379:21:42,  6.36s/it]
  5%|▍         | 10260/225000 [31:43:54<417:58:40,  7.01s/it]
  5%|▍         | 10261/225000 [31:44:01<420:24:26,  7.05s/it]
  5%|▍         | 10262/225000 [31:44:08<428:24:49,  7.18s/it]
  5%|▍         | 10263/225000 [31:44:15<415:39:27,  6.97s/it]
  5%|▍         | 10264/225000 [31:44:18<354:38:49,  5.95s/it]
  5%|▍         | 10265/225000 [31:44:26<393:11:12,  6.59s/it]
  5%|▍         | 10266/225000 [31:44:36<448:29:37,  7.52s/it]
  5%|▍         | 10267/225000 [31:44:43<437:06:45,  7.33s/it]
  5%|▍         | 10268/225000 [31:44:49<420:16:10,  7.05s/it]
  5%|▍         | 10269/225000 [31:44:54<374:06:55,  6.27s/it]
  5%|▍         | 10270/225000 [31:45:02<403:32:07,  6.77s/it]
  5%|▍         | 10271/225000 [31:45:13<489:49:27,  8.21s/it]
  5%|▍         | 10272/225000 [31:45:18<429:55:39,  7.21s/it]
  5%|▍         | 10273/225000 [31:45:22<374:36:29,  6.28s/it]
  5%|▍         | 10274/225000 [31:45:32<434:43:21,  7.29s/it]
  5%|▍         | 10275/225000 [31:45:38<406:42:59,  6.82s/it]
  5%|▍         | 10276/225000 [31:45:42<369:34:01,  6.20s/it]
  5%|▍         | 10277/225000 [31:45:55<490:12:38,  8.22s/it]
  5%|▍         | 10278/225000 [31:46:00<431:01:56,  7.23s/it]
  5%|▍         | 10279/225000 [31:46:12<520:23:18,  8.72s/it]
  5%|▍         | 10280/225000 [31:46:16<431:25:32,  7.23s/it]
  5%|▍         | 10281/225000 [31:46:21<381:57:41,  6.40s/it]
  5%|▍         | 10282/225000 [31:46:26<361:51:32,  6.07s/it]
  5%|▍         | 10283/225000 [31:46:32<360:30:27,  6.04s/it]
  5%|▍         | 10284/225000 [31:46:42<438:14:22,  7.35s/it]
  5%|▍         | 10285/225000 [31:46:48<401:13:12,  6.73s/it]
  5%|▍         | 10286/225000 [31:46:54<393:49:37,  6.60s/it]
  5%|▍         | 10287/225000 [31:47:05<475:41:34,  7.98s/it]
  5%|▍         | 10288/225000 [31:47:09<401:58:37,  6.74s/it]
  5%|▍         | 10289/225000 [31:47:18<436:52:51,  7.33s/it]
  5%|▍         | 10290/225000 [31:47:22<379:28:23,  6.36s/it]
  5%|▍         | 10291/225000 [31:47:26<342:48:37,  5.75s/it]
  5%|▍         | 10292/225000 [31:47:31<325:17:53,  5.45s/it]
  5%|▍         | 10293/225000 [31:47:43<441:55:47,  7.41s/it]
  5%|▍         | 10294/225000 [31:47:47<382:45:20,  6.42s/it]
  5%|▍         | 10295/225000 [31:47:57<451:38:57,  7.57s/it]
  5%|▍         | 10296/225000 [31:48:05<461:59:08,  7.75s/it]
  5%|▍         | 10297/225000 [31:48:12<437:47:33,  7.34s/it]
  5%|▍         | 10298/225000 [31:48:16<383:59:39,  6.44s/it]
  5%|▍         | 10299/225000 [31:48:21<352:03:43,  5.90s/it]
  5%|▍         | 10300/225000 [31:48:33<461:19:08,  7.74s/it]
                                                             
{'loss': '0.7445', 'grad_norm': '0.7401', 'learning_rate': '5e-05', 'epoch': '0.04578', 'num_input_tokens_seen': 57172288, 'train_runtime': '1.145e+05', 'train_tokens_per_second': '499.3'}

  5%|▍         | 10300/225000 [31:48:33<461:19:08,  7.74s/it]
  5%|▍         | 10301/225000 [31:48:41<469:47:24,  7.88s/it]
  5%|▍         | 10302/225000 [31:48:51<504:16:12,  8.46s/it]
  5%|▍         | 10303/225000 [31:48:57<467:19:33,  7.84s/it]
  5%|▍         | 10304/225000 [31:49:08<519:33:52,  8.71s/it]
  5%|▍         | 10305/225000 [31:49:12<430:06:15,  7.21s/it]
  5%|▍         | 10306/225000 [31:49:16<378:48:17,  6.35s/it]
  5%|▍         | 10307/225000 [31:49:21<355:10:13,  5.96s/it]
  5%|▍         | 10308/225000 [31:49:33<459:19:58,  7.70s/it]
  5%|▍         | 10309/225000 [31:49:39<428:16:26,  7.18s/it]
  5%|▍         | 10310/225000 [31:49:45<404:33:06,  6.78s/it]
  5%|▍         | 10311/225000 [31:49:49<367:23:45,  6.16s/it]
  5%|▍         | 10312/225000 [31:49:54<350:51:27,  5.88s/it]
  5%|▍         | 10313/225000 [31:50:04<417:01:10,  6.99s/it]
  5%|▍         | 10314/225000 [31:50:11<408:52:44,  6.86s/it]
  5%|▍         | 10315/225000 [31:50:19<431:37:55,  7.24s/it]
  5%|▍         | 10316/225000 [31:50:24<393:32:11,  6.60s/it]
  5%|▍         | 10317/225000 [31:50:29<370:52:14,  6.22s/it]
  5%|▍         | 10318/225000 [31:50:41<462:27:02,  7.75s/it]
  5%|▍         | 10319/225000 [31:50:47<444:18:33,  7.45s/it]
  5%|▍         | 10320/225000 [31:50:54<426:30:37,  7.15s/it]
  5%|▍         | 10321/225000 [31:51:01<423:29:30,  7.10s/it]
  5%|▍         | 10322/225000 [31:51:08<429:19:22,  7.20s/it]
  5%|▍         | 10323/225000 [31:51:12<367:32:41,  6.16s/it]
  5%|▍         | 10324/225000 [31:51:20<398:41:59,  6.69s/it]
  5%|▍         | 10325/225000 [31:51:23<341:55:20,  5.73s/it]
  5%|▍         | 10326/225000 [31:51:33<412:12:20,  6.91s/it]
  5%|▍         | 10327/225000 [31:51:39<390:20:41,  6.55s/it]
  5%|▍         | 10328/225000 [31:51:51<487:30:57,  8.18s/it]
  5%|▍         | 10329/225000 [31:51:55<425:09:46,  7.13s/it]
  5%|▍         | 10330/225000 [31:52:00<375:34:07,  6.30s/it]
  5%|▍         | 10331/225000 [31:52:05<354:31:33,  5.95s/it]
  5%|▍         | 10332/225000 [31:52:11<352:19:25,  5.91s/it]
  5%|▍         | 10333/225000 [31:52:20<413:34:58,  6.94s/it]
  5%|▍         | 10334/225000 [31:52:28<434:08:23,  7.28s/it]
  5%|▍         | 10335/225000 [31:52:33<399:06:41,  6.69s/it]
  5%|▍         | 10336/225000 [31:52:38<365:37:51,  6.13s/it]
  5%|▍         | 10337/225000 [31:52:43<349:45:54,  5.87s/it]
  5%|▍         | 10338/225000 [31:52:49<347:36:38,  5.83s/it]
  5%|▍         | 10339/225000 [31:53:02<476:05:35,  7.98s/it]
  5%|▍         | 10340/225000 [31:53:06<398:50:11,  6.69s/it]
  5%|▍         | 10341/225000 [31:53:14<425:12:39,  7.13s/it]
  5%|▍         | 10342/225000 [31:53:20<408:42:53,  6.85s/it]
  5%|▍         | 10343/225000 [31:53:26<386:03:24,  6.47s/it]
  5%|▍         | 10344/225000 [31:53:31<361:34:21,  6.06s/it]
  5%|▍         | 10345/225000 [31:53:36<341:42:29,  5.73s/it]
  5%|▍         | 10346/225000 [31:53:42<344:54:36,  5.78s/it]
  5%|▍         | 10347/225000 [31:53:54<456:34:55,  7.66s/it]
  5%|▍         | 10348/225000 [31:54:01<439:55:53,  7.38s/it]
  5%|▍         | 10349/225000 [31:54:07<432:06:36,  7.25s/it]
  5%|▍         | 10350/225000 [31:54:12<375:21:57,  6.30s/it]
                                                             
{'loss': '0.7589', 'grad_norm': '0.8079', 'learning_rate': '5e-05', 'epoch': '0.046', 'num_input_tokens_seen': 57340992, 'train_runtime': '1.149e+05', 'train_tokens_per_second': '499.3'}

  5%|▍         | 10350/225000 [31:54:12<375:21:57,  6.30s/it]
  5%|▍         | 10351/225000 [31:54:22<447:16:07,  7.50s/it]
  5%|▍         | 10352/225000 [31:54:28<423:27:56,  7.10s/it]
  5%|▍         | 10353/225000 [31:54:34<395:29:24,  6.63s/it]
  5%|▍         | 10354/225000 [31:54:40<383:20:59,  6.43s/it]
  5%|▍         | 10355/225000 [31:54:46<389:30:04,  6.53s/it]
  5%|▍         | 10356/225000 [31:54:53<399:42:18,  6.70s/it]
  5%|▍         | 10357/225000 [31:54:58<358:35:02,  6.01s/it]
  5%|▍         | 10358/225000 [31:55:10<465:17:12,  7.80s/it]
  5%|▍         | 10359/225000 [31:55:13<390:05:35,  6.54s/it]
  5%|▍         | 10360/225000 [31:55:19<367:49:30,  6.17s/it]
  5%|▍         | 10361/225000 [31:55:28<423:27:41,  7.10s/it]
  5%|▍         | 10362/225000 [31:55:35<429:35:25,  7.21s/it]
  5%|▍         | 10363/225000 [31:55:41<395:24:48,  6.63s/it]
  5%|▍         | 10364/225000 [31:55:46<371:02:13,  6.22s/it]
  5%|▍         | 10365/225000 [31:55:51<346:29:29,  5.81s/it]
  5%|▍         | 10366/225000 [31:55:57<349:08:43,  5.86s/it]
  5%|▍         | 10367/225000 [31:56:10<481:14:57,  8.07s/it]
  5%|▍         | 10368/225000 [31:56:15<423:09:33,  7.10s/it]
  5%|▍         | 10369/225000 [31:56:26<504:03:10,  8.45s/it]
  5%|▍         | 10370/225000 [31:56:34<489:13:54,  8.21s/it]
  5%|▍         | 10371/225000 [31:56:41<461:16:53,  7.74s/it]
  5%|▍         | 10372/225000 [31:56:46<413:19:11,  6.93s/it]
  5%|▍         | 10373/225000 [31:56:52<408:39:55,  6.85s/it]
  5%|▍         | 10374/225000 [31:56:58<380:41:14,  6.39s/it]
  5%|▍         | 10375/225000 [31:57:04<374:51:32,  6.29s/it]
  5%|▍         | 10376/225000 [31:57:15<470:04:56,  7.88s/it]
  5%|▍         | 10377/225000 [31:57:23<459:16:50,  7.70s/it]
  5%|▍         | 10378/225000 [31:57:28<419:19:23,  7.03s/it]
  5%|▍         | 10379/225000 [31:57:34<390:12:08,  6.55s/it]
  5%|▍         | 10380/225000 [31:57:39<361:47:53,  6.07s/it]
  5%|▍         | 10381/225000 [31:57:51<472:07:00,  7.92s/it]
  5%|▍         | 10382/225000 [31:57:57<447:22:30,  7.50s/it]
  5%|▍         | 10383/225000 [31:58:02<405:01:11,  6.79s/it]
  5%|▍         | 10384/225000 [31:58:14<482:27:08,  8.09s/it]
  5%|▍         | 10385/225000 [31:58:20<448:09:59,  7.52s/it]
  5%|▍         | 10386/225000 [31:58:24<382:07:49,  6.41s/it]
  5%|▍         | 10387/225000 [31:58:32<419:51:11,  7.04s/it]
  5%|▍         | 10388/225000 [31:58:41<449:02:14,  7.53s/it]
  5%|▍         | 10389/225000 [31:58:50<473:33:54,  7.94s/it]
  5%|▍         | 10390/225000 [31:58:55<426:13:06,  7.15s/it]
  5%|▍         | 10391/225000 [31:59:00<385:58:39,  6.47s/it]
  5%|▍         | 10392/225000 [31:59:05<366:53:46,  6.15s/it]
  5%|▍         | 10393/225000 [31:59:10<347:04:21,  5.82s/it]
  5%|▍         | 10394/225000 [31:59:16<342:54:37,  5.75s/it]
  5%|▍         | 10395/225000 [31:59:22<340:57:59,  5.72s/it]
  5%|▍         | 10396/225000 [31:59:35<479:53:48,  8.05s/it]
  5%|▍         | 10397/225000 [31:59:41<449:03:42,  7.53s/it]
  5%|▍         | 10398/225000 [31:59:46<396:11:23,  6.65s/it]
  5%|▍         | 10399/225000 [31:59:56<453:27:22,  7.61s/it]
  5%|▍         | 10400/225000 [32:00:00<389:56:42,  6.54s/it]
                                                             
{'loss': '0.7364', 'grad_norm': '0.7042', 'learning_rate': '5e-05', 'epoch': '0.04622', 'num_input_tokens_seen': 57507920, 'train_runtime': '1.152e+05', 'train_tokens_per_second': '499.2'}

  5%|▍         | 10400/225000 [32:00:00<389:56:42,  6.54s/it]
  5%|▍         | 10401/225000 [32:00:09<436:22:13,  7.32s/it]
  5%|▍         | 10402/225000 [32:00:19<479:10:25,  8.04s/it]
  5%|▍         | 10403/225000 [32:00:28<494:43:38,  8.30s/it]
  5%|▍         | 10404/225000 [32:00:37<513:59:20,  8.62s/it]
  5%|▍         | 10405/225000 [32:00:44<490:08:14,  8.22s/it]
  5%|▍         | 10406/225000 [32:00:54<524:21:03,  8.80s/it]
  5%|▍         | 10407/225000 [32:00:59<445:22:58,  7.47s/it]
  5%|▍         | 10408/225000 [32:01:04<411:17:03,  6.90s/it]
  5%|▍         | 10409/225000 [32:01:15<483:36:36,  8.11s/it]
  5%|▍         | 10410/225000 [32:01:20<415:09:50,  6.96s/it]
  5%|▍         | 10411/225000 [32:01:25<378:34:44,  6.35s/it]
  5%|▍         | 10412/225000 [32:01:32<402:15:22,  6.75s/it]
  5%|▍         | 10413/225000 [32:01:37<365:11:26,  6.13s/it]
  5%|▍         | 10414/225000 [32:01:45<409:14:57,  6.87s/it]
  5%|▍         | 10415/225000 [32:01:53<425:53:00,  7.14s/it]
  5%|▍         | 10416/225000 [32:01:57<373:04:42,  6.26s/it]
  5%|▍         | 10417/225000 [32:02:04<382:57:02,  6.42s/it]
  5%|▍         | 10418/225000 [32:02:09<344:15:26,  5.78s/it]
  5%|▍         | 10419/225000 [32:02:14<345:51:35,  5.80s/it]
  5%|▍         | 10420/225000 [32:02:22<369:48:55,  6.20s/it]
  5%|▍         | 10421/225000 [32:02:31<434:26:18,  7.29s/it]
  5%|▍         | 10422/225000 [32:02:36<390:34:03,  6.55s/it]
  5%|▍         | 10423/225000 [32:02:41<357:31:37,  6.00s/it]
  5%|▍         | 10424/225000 [32:02:46<349:12:11,  5.86s/it]
  5%|▍         | 10425/225000 [32:02:58<457:38:57,  7.68s/it]
  5%|▍         | 10426/225000 [32:03:02<384:47:27,  6.46s/it]
  5%|▍         | 10427/225000 [32:03:06<345:13:49,  5.79s/it]
  5%|▍         | 10428/225000 [32:03:11<325:14:02,  5.46s/it]
  5%|▍         | 10429/225000 [32:03:17<341:51:34,  5.74s/it]
  5%|▍         | 10430/225000 [32:03:25<386:51:07,  6.49s/it]
  5%|▍         | 10431/225000 [32:03:30<348:14:22,  5.84s/it]
  5%|▍         | 10432/225000 [32:03:40<433:16:59,  7.27s/it]
  5%|▍         | 10433/225000 [32:03:47<424:14:19,  7.12s/it]
  5%|▍         | 10434/225000 [32:03:52<376:15:52,  6.31s/it]
  5%|▍         | 10435/225000 [32:04:01<429:43:17,  7.21s/it]
  5%|▍         | 10436/225000 [32:04:09<440:55:39,  7.40s/it]
  5%|▍         | 10437/225000 [32:04:17<453:35:50,  7.61s/it]
  5%|▍         | 10438/225000 [32:04:26<484:19:40,  8.13s/it]
  5%|▍         | 10439/225000 [32:04:30<412:05:29,  6.91s/it]
  5%|▍         | 10440/225000 [32:04:42<504:35:40,  8.47s/it]
  5%|▍         | 10441/225000 [32:04:49<468:42:56,  7.86s/it]
  5%|▍         | 10442/225000 [32:04:53<406:47:00,  6.83s/it]
  5%|▍         | 10443/225000 [32:04:57<360:15:09,  6.04s/it]
  5%|▍         | 10444/225000 [32:05:06<413:26:05,  6.94s/it]
  5%|▍         | 10445/225000 [32:05:17<483:13:50,  8.11s/it]
  5%|▍         | 10446/225000 [32:05:22<425:45:57,  7.14s/it]
  5%|▍         | 10447/225000 [32:05:29<423:23:28,  7.10s/it]
  5%|▍         | 10448/225000 [32:05:36<425:27:38,  7.14s/it]
  5%|▍         | 10449/225000 [32:05:49<518:09:55,  8.69s/it]
  5%|▍         | 10450/225000 [32:05:53<436:31:15,  7.32s/it]
                                                             
{'loss': '0.7228', 'grad_norm': '0.75', 'learning_rate': '5e-05', 'epoch': '0.04644', 'num_input_tokens_seen': 57677264, 'train_runtime': '1.156e+05', 'train_tokens_per_second': '499.1'}

  5%|▍         | 10450/225000 [32:05:53<436:31:15,  7.32s/it]
  5%|▍         | 10451/225000 [32:06:02<469:11:54,  7.87s/it]
  5%|▍         | 10452/225000 [32:06:06<404:19:08,  6.78s/it]
  5%|▍         | 10453/225000 [32:06:11<372:28:03,  6.25s/it]
  5%|▍         | 10454/225000 [32:06:18<388:46:30,  6.52s/it]
  5%|▍         | 10455/225000 [32:06:26<411:59:46,  6.91s/it]
  5%|▍         | 10456/225000 [32:06:34<421:12:45,  7.07s/it]
  5%|▍         | 10457/225000 [32:06:38<373:15:25,  6.26s/it]
  5%|▍         | 10458/225000 [32:06:43<354:44:36,  5.95s/it]
  5%|▍         | 10459/225000 [32:06:51<386:23:16,  6.48s/it]
  5%|▍         | 10460/225000 [32:06:59<409:42:03,  6.87s/it]
  5%|▍         | 10461/225000 [32:07:08<454:44:28,  7.63s/it]
  5%|▍         | 10462/225000 [32:07:12<379:55:40,  6.38s/it]
  5%|▍         | 10463/225000 [32:07:15<332:08:15,  5.57s/it]
  5%|▍         | 10464/225000 [32:07:24<377:46:17,  6.34s/it]
  5%|▍         | 10465/225000 [32:07:31<391:53:19,  6.58s/it]
  5%|▍         | 10466/225000 [32:07:39<421:53:14,  7.08s/it]
  5%|▍         | 10467/225000 [32:07:44<387:58:50,  6.51s/it]
  5%|▍         | 10468/225000 [32:07:49<366:07:46,  6.14s/it]
  5%|▍         | 10469/225000 [32:07:55<359:16:37,  6.03s/it]
  5%|▍         | 10470/225000 [32:08:00<345:16:11,  5.79s/it]
  5%|▍         | 10471/225000 [32:08:06<334:25:37,  5.61s/it]
  5%|▍         | 10472/225000 [32:08:11<330:34:19,  5.55s/it]
  5%|▍         | 10473/225000 [32:08:23<443:30:19,  7.44s/it]
  5%|▍         | 10474/225000 [32:08:28<409:54:19,  6.88s/it]
  5%|▍         | 10475/225000 [32:08:34<378:35:17,  6.35s/it]
  5%|▍         | 10476/225000 [32:08:39<365:24:21,  6.13s/it]
  5%|▍         | 10477/225000 [32:08:52<486:52:24,  8.17s/it]
  5%|▍         | 10478/225000 [32:08:59<468:40:29,  7.87s/it]
  5%|▍         | 10479/225000 [32:09:06<445:18:10,  7.47s/it]
  5%|▍         | 10480/225000 [32:09:13<435:48:04,  7.31s/it]
  5%|▍         | 10481/225000 [32:09:22<471:28:28,  7.91s/it]
  5%|▍         | 10482/225000 [32:09:31<483:02:40,  8.11s/it]
  5%|▍         | 10483/225000 [32:09:38<463:54:56,  7.79s/it]
  5%|▍         | 10484/225000 [32:09:42<412:14:18,  6.92s/it]
  5%|▍         | 10485/225000 [32:09:47<363:55:31,  6.11s/it]
  5%|▍         | 10486/225000 [32:09:59<480:26:55,  8.06s/it]
  5%|▍         | 10487/225000 [32:10:04<426:22:43,  7.16s/it]
  5%|▍         | 10488/225000 [32:10:09<387:23:30,  6.50s/it]
  5%|▍         | 10489/225000 [32:10:19<441:27:30,  7.41s/it]
  5%|▍         | 10490/225000 [32:10:24<408:32:21,  6.86s/it]
  5%|▍         | 10491/225000 [32:10:29<366:12:28,  6.15s/it]
  5%|▍         | 10492/225000 [32:10:34<346:07:35,  5.81s/it]
  5%|▍         | 10493/225000 [32:10:49<504:40:34,  8.47s/it]
  5%|▍         | 10494/225000 [32:10:55<475:09:21,  7.97s/it]
  5%|▍         | 10495/225000 [32:11:00<405:59:46,  6.81s/it]
  5%|▍         | 10496/225000 [32:11:06<400:33:44,  6.72s/it]
  5%|▍         | 10497/225000 [32:11:11<364:58:01,  6.13s/it]
  5%|▍         | 10498/225000 [32:11:16<342:08:43,  5.74s/it]
  5%|▍         | 10499/225000 [32:11:27<439:10:20,  7.37s/it]
  5%|▍         | 10500/225000 [32:11:34<430:08:11,  7.22s/it]
                                                             
{'loss': '0.7307', 'grad_norm': '0.6986', 'learning_rate': '5e-05', 'epoch': '0.04667', 'num_input_tokens_seen': 57840544, 'train_runtime': '1.159e+05', 'train_tokens_per_second': '499.1'}

  5%|▍         | 10500/225000 [32:11:34<430:08:11,  7.22s/it]
  5%|▍         | 10501/225000 [32:11:40<411:06:37,  6.90s/it]
  5%|▍         | 10502/225000 [32:11:53<518:29:07,  8.70s/it]
  5%|▍         | 10503/225000 [32:11:58<457:40:34,  7.68s/it]
  5%|▍         | 10504/225000 [32:12:09<523:40:55,  8.79s/it]
  5%|▍         | 10505/225000 [32:12:14<447:57:36,  7.52s/it]
  5%|▍         | 10506/225000 [32:12:25<507:45:49,  8.52s/it]
  5%|▍         | 10507/225000 [32:12:30<454:38:42,  7.63s/it]
  5%|▍         | 10508/225000 [32:12:37<429:30:03,  7.21s/it]
  5%|▍         | 10509/225000 [32:12:46<472:36:18,  7.93s/it]
  5%|▍         | 10510/225000 [32:12:51<407:36:32,  6.84s/it]
  5%|▍         | 10511/225000 [32:12:59<445:41:48,  7.48s/it]
  5%|▍         | 10512/225000 [32:13:04<395:10:50,  6.63s/it]
  5%|▍         | 10513/225000 [32:13:09<358:23:34,  6.02s/it]
  5%|▍         | 10514/225000 [32:13:14<338:40:47,  5.68s/it]
  5%|▍         | 10515/225000 [32:13:26<459:16:28,  7.71s/it]
  5%|▍         | 10516/225000 [32:13:38<537:31:38,  9.02s/it]
  5%|▍         | 10517/225000 [32:13:43<458:31:09,  7.70s/it]
  5%|▍         | 10518/225000 [32:13:48<407:37:54,  6.84s/it]
  5%|▍         | 10519/225000 [32:13:53<380:00:36,  6.38s/it]
  5%|▍         | 10520/225000 [32:13:58<358:49:22,  6.02s/it]
  5%|▍         | 10521/225000 [32:14:04<362:01:28,  6.08s/it]
  5%|▍         | 10522/225000 [32:14:10<347:53:24,  5.84s/it]
  5%|▍         | 10523/225000 [32:14:17<382:07:38,  6.41s/it]
  5%|▍         | 10524/225000 [32:14:24<393:20:49,  6.60s/it]
  5%|▍         | 10525/225000 [32:14:38<513:23:10,  8.62s/it]
  5%|▍         | 10526/225000 [32:14:48<534:44:52,  8.98s/it]
  5%|▍         | 10527/225000 [32:14:52<462:53:02,  7.77s/it]
  5%|▍         | 10528/225000 [32:15:00<457:57:15,  7.69s/it]
  5%|▍         | 10529/225000 [32:15:05<401:45:54,  6.74s/it]
  5%|▍         | 10530/225000 [32:15:09<368:00:25,  6.18s/it]
  5%|▍         | 10531/225000 [32:15:20<442:48:26,  7.43s/it]
  5%|▍         | 10532/225000 [32:15:26<423:55:26,  7.12s/it]
  5%|▍         | 10533/225000 [32:15:33<419:03:59,  7.03s/it]
  5%|▍         | 10534/225000 [32:15:38<376:15:11,  6.32s/it]
  5%|▍         | 10535/225000 [32:15:42<348:47:28,  5.85s/it]
  5%|▍         | 10536/225000 [32:15:47<327:40:22,  5.50s/it]
  5%|▍         | 10537/225000 [32:15:58<424:30:10,  7.13s/it]
  5%|▍         | 10538/225000 [32:16:04<411:44:46,  6.91s/it]
  5%|▍         | 10539/225000 [32:16:14<456:11:34,  7.66s/it]
  5%|▍         | 10540/225000 [32:16:20<433:32:18,  7.28s/it]
  5%|▍         | 10541/225000 [32:16:25<395:10:59,  6.63s/it]
  5%|▍         | 10542/225000 [32:16:39<519:08:10,  8.71s/it]
  5%|▍         | 10543/225000 [32:16:44<446:22:33,  7.49s/it]
  5%|▍         | 10544/225000 [32:16:53<490:12:51,  8.23s/it]
  5%|▍         | 10545/225000 [32:17:09<629:47:19, 10.57s/it]
  5%|▍         | 10546/225000 [32:17:15<532:16:29,  8.94s/it]
  5%|▍         | 10547/225000 [32:17:20<467:22:09,  7.85s/it]
  5%|▍         | 10548/225000 [32:17:32<537:08:29,  9.02s/it]
  5%|▍         | 10549/225000 [32:17:36<453:08:42,  7.61s/it]
  5%|▍         | 10550/225000 [32:17:40<393:37:38,  6.61s/it]
                                                             
{'loss': '0.7505', 'grad_norm': '0.6809', 'learning_rate': '5e-05', 'epoch': '0.04689', 'num_input_tokens_seen': 58015144, 'train_runtime': '1.163e+05', 'train_tokens_per_second': '499'}

  5%|▍         | 10550/225000 [32:17:40<393:37:38,  6.61s/it]
  5%|▍         | 10551/225000 [32:17:47<389:36:34,  6.54s/it]
  5%|▍         | 10552/225000 [32:17:53<391:24:17,  6.57s/it]
  5%|▍         | 10553/225000 [32:17:58<366:50:21,  6.16s/it]
  5%|▍         | 10554/225000 [32:18:10<467:03:42,  7.84s/it]
  5%|▍         | 10555/225000 [32:18:16<422:18:28,  7.09s/it]
  5%|▍         | 10556/225000 [32:18:23<428:52:32,  7.20s/it]
  5%|▍         | 10557/225000 [32:18:28<383:05:15,  6.43s/it]
  5%|▍         | 10558/225000 [32:18:35<392:57:55,  6.60s/it]
  5%|▍         | 10559/225000 [32:18:46<474:04:30,  7.96s/it]
  5%|▍         | 10560/225000 [32:18:51<421:34:49,  7.08s/it]
  5%|▍         | 10561/225000 [32:18:59<434:18:54,  7.29s/it]
  5%|▍         | 10562/225000 [32:19:03<386:21:47,  6.49s/it]
  5%|▍         | 10563/225000 [32:19:08<348:33:18,  5.85s/it]
  5%|▍         | 10564/225000 [32:19:17<415:08:11,  6.97s/it]
  5%|▍         | 10565/225000 [32:19:24<420:04:49,  7.05s/it]
  5%|▍         | 10566/225000 [32:19:30<386:31:34,  6.49s/it]
  5%|▍         | 10567/225000 [32:19:35<371:15:14,  6.23s/it]
  5%|▍         | 10568/225000 [32:19:40<340:15:46,  5.71s/it]
  5%|▍         | 10569/225000 [32:19:45<338:24:36,  5.68s/it]
  5%|▍         | 10570/225000 [32:19:58<466:27:58,  7.83s/it]
  5%|▍         | 10571/225000 [32:20:06<473:37:58,  7.95s/it]
  5%|▍         | 10572/225000 [32:20:10<401:31:34,  6.74s/it]
  5%|▍         | 10573/225000 [32:20:16<374:25:55,  6.29s/it]
  5%|▍         | 10574/225000 [32:20:22<369:57:21,  6.21s/it]
  5%|▍         | 10575/225000 [32:20:28<369:28:10,  6.20s/it]
  5%|▍         | 10576/225000 [32:20:35<396:48:13,  6.66s/it]
  5%|▍         | 10577/225000 [32:20:51<546:18:58,  9.17s/it]
  5%|▍         | 10578/225000 [32:20:59<529:39:56,  8.89s/it]
  5%|▍         | 10579/225000 [32:21:05<476:19:31,  8.00s/it]
  5%|▍         | 10580/225000 [32:21:16<532:35:56,  8.94s/it]
  5%|▍         | 10581/225000 [32:21:22<487:19:20,  8.18s/it]
  5%|▍         | 10582/225000 [32:21:27<435:00:05,  7.30s/it]
  5%|▍         | 10583/225000 [32:21:33<398:00:05,  6.68s/it]
  5%|▍         | 10584/225000 [32:21:39<389:21:07,  6.54s/it]
  5%|▍         | 10585/225000 [32:21:45<375:02:08,  6.30s/it]
  5%|▍         | 10586/225000 [32:22:00<530:01:40,  8.90s/it]
  5%|▍         | 10587/225000 [32:22:04<448:27:28,  7.53s/it]
  5%|▍         | 10588/225000 [32:22:10<421:15:38,  7.07s/it]
  5%|▍         | 10589/225000 [32:22:15<382:24:37,  6.42s/it]
  5%|▍         | 10590/225000 [32:22:20<353:30:13,  5.94s/it]
  5%|▍         | 10591/225000 [32:22:26<352:58:04,  5.93s/it]
  5%|▍         | 10592/225000 [32:22:31<341:30:56,  5.73s/it]
  5%|▍         | 10593/225000 [32:22:48<544:04:12,  9.14s/it]
  5%|▍         | 10594/225000 [32:22:58<552:12:46,  9.27s/it]
  5%|▍         | 10595/225000 [32:23:07<557:24:09,  9.36s/it]
  5%|▍         | 10596/225000 [32:23:12<477:16:01,  8.01s/it]
  5%|▍         | 10597/225000 [32:23:18<437:29:57,  7.35s/it]
  5%|▍         | 10598/225000 [32:23:23<394:33:41,  6.63s/it]
  5%|▍         | 10599/225000 [32:23:28<370:24:10,  6.22s/it]
  5%|▍         | 10600/225000 [32:23:33<351:11:48,  5.90s/it]
                                                             
{'loss': '0.7423', 'grad_norm': '0.6997', 'learning_rate': '5e-05', 'epoch': '0.04711', 'num_input_tokens_seen': 58187224, 'train_runtime': '1.166e+05', 'train_tokens_per_second': '499'}

  5%|▍         | 10600/225000 [32:23:33<351:11:48,  5.90s/it]
  5%|▍         | 10601/225000 [32:23:43<424:27:23,  7.13s/it]
  5%|▍         | 10602/225000 [32:23:55<502:27:03,  8.44s/it]
  5%|▍         | 10603/225000 [32:24:06<548:17:08,  9.21s/it]
  5%|▍         | 10604/225000 [32:24:10<464:35:57,  7.80s/it]
  5%|▍         | 10605/225000 [32:24:16<426:46:58,  7.17s/it]
  5%|▍         | 10606/225000 [32:24:22<407:09:11,  6.84s/it]
  5%|▍         | 10607/225000 [32:24:28<389:03:02,  6.53s/it]
  5%|▍         | 10608/225000 [32:24:33<359:42:20,  6.04s/it]
  5%|▍         | 10609/225000 [32:24:46<496:38:34,  8.34s/it]
  5%|▍         | 10610/225000 [32:24:51<426:59:45,  7.17s/it]
  5%|▍         | 10611/225000 [32:24:55<377:36:46,  6.34s/it]
  5%|▍         | 10612/225000 [32:25:02<379:59:19,  6.38s/it]
  5%|▍         | 10613/225000 [32:25:07<366:02:31,  6.15s/it]
  5%|▍         | 10614/225000 [32:25:16<416:21:17,  6.99s/it]
  5%|▍         | 10615/225000 [32:25:26<467:18:31,  7.85s/it]
  5%|▍         | 10616/225000 [32:25:33<444:24:04,  7.46s/it]
  5%|▍         | 10617/225000 [32:25:37<382:51:53,  6.43s/it]
  5%|▍         | 10618/225000 [32:25:42<358:08:20,  6.01s/it]
  5%|▍         | 10619/225000 [32:25:47<342:19:10,  5.75s/it]
  5%|▍         | 10620/225000 [32:25:59<463:31:33,  7.78s/it]
  5%|▍         | 10621/225000 [32:26:04<407:04:01,  6.84s/it]
  5%|▍         | 10622/225000 [32:26:08<359:29:41,  6.04s/it]
  5%|▍         | 10623/225000 [32:26:20<469:19:59,  7.88s/it]
  5%|▍         | 10624/225000 [32:26:25<406:29:41,  6.83s/it]
  5%|▍         | 10625/225000 [32:26:32<418:47:53,  7.03s/it]
  5%|▍         | 10626/225000 [32:26:41<459:51:26,  7.72s/it]
  5%|▍         | 10627/225000 [32:26:46<400:35:57,  6.73s/it]
  5%|▍         | 10628/225000 [32:26:56<460:57:45,  7.74s/it]
  5%|▍         | 10629/225000 [32:27:02<423:24:27,  7.11s/it]
  5%|▍         | 10630/225000 [32:27:06<370:15:13,  6.22s/it]
  5%|▍         | 10631/225000 [32:27:17<466:32:07,  7.83s/it]
  5%|▍         | 10632/225000 [32:27:22<405:16:25,  6.81s/it]
  5%|▍         | 10633/225000 [32:27:27<370:08:25,  6.22s/it]
  5%|▍         | 10634/225000 [32:27:34<395:29:33,  6.64s/it]
  5%|▍         | 10635/225000 [32:27:43<427:59:09,  7.19s/it]
  5%|▍         | 10636/225000 [32:27:47<375:47:42,  6.31s/it]
  5%|▍         | 10637/225000 [32:27:51<339:31:48,  5.70s/it]
  5%|▍         | 10638/225000 [32:28:04<460:49:56,  7.74s/it]
  5%|▍         | 10639/225000 [32:28:15<531:50:14,  8.93s/it]
  5%|▍         | 10640/225000 [32:28:21<469:42:51,  7.89s/it]
  5%|▍         | 10641/225000 [32:28:30<483:14:54,  8.12s/it]
  5%|▍         | 10642/225000 [32:28:38<487:25:46,  8.19s/it]
  5%|▍         | 10643/225000 [32:28:41<401:17:29,  6.74s/it]
  5%|▍         | 10644/225000 [32:28:46<368:06:38,  6.18s/it]
  5%|▍         | 10645/225000 [32:28:51<342:43:49,  5.76s/it]
  5%|▍         | 10646/225000 [32:29:00<398:04:57,  6.69s/it]
  5%|▍         | 10647/225000 [32:29:08<419:27:51,  7.04s/it]
  5%|▍         | 10648/225000 [32:29:12<376:41:48,  6.33s/it]
  5%|▍         | 10649/225000 [32:29:23<448:06:02,  7.53s/it]
  5%|▍         | 10650/225000 [32:29:30<445:53:56,  7.49s/it]
                                                             
{'loss': '0.7509', 'grad_norm': '0.6802', 'learning_rate': '5e-05', 'epoch': '0.04733', 'num_input_tokens_seen': 58355640, 'train_runtime': '1.17e+05', 'train_tokens_per_second': '498.9'}

  5%|▍         | 10650/225000 [32:29:30<445:53:56,  7.49s/it]
  5%|▍         | 10651/225000 [32:29:44<558:00:18,  9.37s/it]
  5%|▍         | 10652/225000 [32:29:55<583:30:31,  9.80s/it]
  5%|▍         | 10653/225000 [32:29:58<472:33:27,  7.94s/it]
  5%|▍         | 10654/225000 [32:30:06<461:41:25,  7.75s/it]
  5%|▍         | 10655/225000 [32:30:10<397:29:27,  6.68s/it]
  5%|▍         | 10656/225000 [32:30:19<447:39:50,  7.52s/it]
  5%|▍         | 10657/225000 [32:30:28<472:02:28,  7.93s/it]
  5%|▍         | 10658/225000 [32:30:36<464:35:24,  7.80s/it]
  5%|▍         | 10659/225000 [32:30:41<416:52:31,  7.00s/it]
  5%|▍         | 10660/225000 [32:30:46<387:48:20,  6.51s/it]
  5%|▍         | 10661/225000 [32:30:59<503:49:34,  8.46s/it]
  5%|▍         | 10662/225000 [32:31:07<500:05:22,  8.40s/it]
  5%|▍         | 10663/225000 [32:31:12<433:29:58,  7.28s/it]
  5%|▍         | 10664/225000 [32:31:19<434:29:52,  7.30s/it]
  5%|▍         | 10665/225000 [32:31:29<470:07:43,  7.90s/it]
  5%|▍         | 10666/225000 [32:31:36<452:02:49,  7.59s/it]
  5%|▍         | 10667/225000 [32:31:41<409:33:09,  6.88s/it]
  5%|▍         | 10668/225000 [32:31:46<375:01:08,  6.30s/it]
  5%|▍         | 10669/225000 [32:31:55<430:57:00,  7.24s/it]
  5%|▍         | 10670/225000 [32:32:00<387:34:21,  6.51s/it]
  5%|▍         | 10671/225000 [32:32:08<422:33:00,  7.10s/it]
  5%|▍         | 10672/225000 [32:32:12<360:06:53,  6.05s/it]
  5%|▍         | 10673/225000 [32:32:18<351:58:52,  5.91s/it]
  5%|▍         | 10674/225000 [32:32:31<477:40:26,  8.02s/it]
  5%|▍         | 10675/225000 [32:32:37<448:25:11,  7.53s/it]
  5%|▍         | 10676/225000 [32:32:42<406:32:52,  6.83s/it]
  5%|▍         | 10677/225000 [32:32:48<386:20:19,  6.49s/it]
  5%|▍         | 10678/225000 [32:32:56<408:14:57,  6.86s/it]
  5%|▍         | 10679/225000 [32:33:03<419:37:39,  7.05s/it]
  5%|▍         | 10680/225000 [32:33:12<446:05:00,  7.49s/it]
  5%|▍         | 10681/225000 [32:33:19<438:59:53,  7.37s/it]
  5%|▍         | 10682/225000 [32:33:24<408:31:37,  6.86s/it]
  5%|▍         | 10683/225000 [32:33:35<482:05:37,  8.10s/it]
  5%|▍         | 10684/225000 [32:33:42<451:13:53,  7.58s/it]
  5%|▍         | 10685/225000 [32:33:48<423:59:14,  7.12s/it]
  5%|▍         | 10686/225000 [32:33:59<492:52:18,  8.28s/it]
  5%|▍         | 10687/225000 [32:34:03<421:42:27,  7.08s/it]
  5%|▍         | 10688/225000 [32:34:08<383:47:12,  6.45s/it]
  5%|▍         | 10689/225000 [32:34:13<355:22:13,  5.97s/it]
  5%|▍         | 10690/225000 [32:34:19<367:53:26,  6.18s/it]
  5%|▍         | 10691/225000 [32:34:24<346:28:53,  5.82s/it]
  5%|▍         | 10692/225000 [32:34:37<457:58:09,  7.69s/it]
  5%|▍         | 10693/225000 [32:34:43<436:40:09,  7.34s/it]
  5%|▍         | 10694/225000 [32:34:48<393:42:13,  6.61s/it]
  5%|▍         | 10695/225000 [32:34:54<388:40:35,  6.53s/it]
  5%|▍         | 10696/225000 [32:35:04<437:38:16,  7.35s/it]
  5%|▍         | 10697/225000 [32:35:08<393:42:43,  6.61s/it]
  5%|▍         | 10698/225000 [32:35:13<364:59:02,  6.13s/it]
  5%|▍         | 10699/225000 [32:35:25<469:39:26,  7.89s/it]
  5%|▍         | 10700/225000 [32:35:32<441:57:37,  7.42s/it]
                                                             
{'loss': '0.7365', 'grad_norm': '0.7632', 'learning_rate': '5e-05', 'epoch': '0.04756', 'num_input_tokens_seen': 58524712, 'train_runtime': '1.173e+05', 'train_tokens_per_second': '498.8'}

  5%|▍         | 10700/225000 [32:35:32<441:57:37,  7.42s/it]
  5%|▍         | 10701/225000 [32:35:40<448:15:48,  7.53s/it]
  5%|▍         | 10702/225000 [32:35:46<431:29:16,  7.25s/it]
  5%|▍         | 10703/225000 [32:35:53<430:08:37,  7.23s/it]
  5%|▍         | 10704/225000 [32:35:57<373:32:12,  6.28s/it]
  5%|▍         | 10705/225000 [32:36:02<348:13:14,  5.85s/it]
  5%|▍         | 10706/225000 [32:36:07<326:17:15,  5.48s/it]
  5%|▍         | 10707/225000 [32:36:20<466:25:25,  7.84s/it]
  5%|▍         | 10708/225000 [32:36:25<419:18:41,  7.04s/it]
  5%|▍         | 10709/225000 [32:36:31<392:45:35,  6.60s/it]
  5%|▍         | 10710/225000 [32:36:36<368:57:28,  6.20s/it]
  5%|▍         | 10711/225000 [32:36:42<358:29:19,  6.02s/it]
  5%|▍         | 10712/225000 [32:36:53<457:23:03,  7.68s/it]
  5%|▍         | 10713/225000 [32:37:04<506:03:32,  8.50s/it]
  5%|▍         | 10714/225000 [32:37:12<493:32:03,  8.29s/it]
  5%|▍         | 10715/225000 [32:37:22<522:27:03,  8.78s/it]
  5%|▍         | 10716/225000 [32:37:28<480:22:41,  8.07s/it]
  5%|▍         | 10717/225000 [32:37:33<420:56:31,  7.07s/it]
  5%|▍         | 10718/225000 [32:37:40<433:35:37,  7.28s/it]
  5%|▍         | 10719/225000 [32:37:48<437:42:33,  7.35s/it]
  5%|▍         | 10720/225000 [32:37:54<416:18:40,  6.99s/it]
  5%|▍         | 10721/225000 [32:38:05<486:58:27,  8.18s/it]
  5%|▍         | 10722/225000 [32:38:12<472:58:34,  7.95s/it]
  5%|▍         | 10723/225000 [32:38:16<401:21:57,  6.74s/it]
  5%|▍         | 10724/225000 [32:38:20<349:02:57,  5.86s/it]
  5%|▍         | 10725/225000 [32:38:28<377:58:27,  6.35s/it]
  5%|▍         | 10726/225000 [32:38:38<444:15:59,  7.46s/it]
  5%|▍         | 10727/225000 [32:38:42<388:30:15,  6.53s/it]
  5%|▍         | 10728/225000 [32:38:52<442:48:56,  7.44s/it]
  5%|▍         | 10729/225000 [32:38:57<400:59:06,  6.74s/it]
  5%|▍         | 10730/225000 [32:39:03<389:38:07,  6.55s/it]
  5%|▍         | 10731/225000 [32:39:08<368:12:11,  6.19s/it]
  5%|▍         | 10732/225000 [32:39:18<435:41:46,  7.32s/it]
  5%|▍         | 10733/225000 [32:39:23<396:58:55,  6.67s/it]
  5%|▍         | 10734/225000 [32:39:28<363:16:59,  6.10s/it]
  5%|▍         | 10735/225000 [32:39:33<346:40:44,  5.82s/it]
  5%|▍         | 10736/225000 [32:39:47<490:40:24,  8.24s/it]
  5%|▍         | 10737/225000 [32:39:53<449:26:28,  7.55s/it]
  5%|▍         | 10738/225000 [32:39:57<392:28:29,  6.59s/it]
  5%|▍         | 10739/225000 [32:40:02<354:52:48,  5.96s/it]
  5%|▍         | 10740/225000 [32:40:13<436:40:48,  7.34s/it]
  5%|▍         | 10741/225000 [32:40:20<432:18:25,  7.26s/it]
  5%|▍         | 10742/225000 [32:40:26<421:23:58,  7.08s/it]
  5%|▍         | 10743/225000 [32:40:33<416:47:23,  7.00s/it]
  5%|▍         | 10744/225000 [32:40:42<458:45:46,  7.71s/it]
  5%|▍         | 10745/225000 [32:40:46<385:04:40,  6.47s/it]
  5%|▍         | 10746/225000 [32:40:50<347:24:26,  5.84s/it]
  5%|▍         | 10747/225000 [32:41:02<451:23:46,  7.58s/it]
  5%|▍         | 10748/225000 [32:41:11<477:33:40,  8.02s/it]
  5%|▍         | 10749/225000 [32:41:20<497:04:42,  8.35s/it]
  5%|▍         | 10750/225000 [32:41:27<473:26:50,  7.96s/it]
                                                             
{'loss': '0.757', 'grad_norm': '0.6506', 'learning_rate': '5e-05', 'epoch': '0.04778', 'num_input_tokens_seen': 58690232, 'train_runtime': '1.177e+05', 'train_tokens_per_second': '498.7'}

  5%|▍         | 10750/225000 [32:41:27<473:26:50,  7.96s/it]
  5%|▍         | 10751/225000 [32:41:33<439:47:21,  7.39s/it]
  5%|▍         | 10752/225000 [32:41:42<469:29:56,  7.89s/it]
  5%|▍         | 10753/225000 [32:41:47<414:04:48,  6.96s/it]
  5%|▍         | 10754/225000 [32:41:51<359:52:34,  6.05s/it]
  5%|▍         | 10755/225000 [32:42:04<475:55:58,  8.00s/it]
  5%|▍         | 10756/225000 [32:42:10<439:57:06,  7.39s/it]
  5%|▍         | 10757/225000 [32:42:14<380:10:08,  6.39s/it]
  5%|▍         | 10758/225000 [32:42:19<358:26:28,  6.02s/it]
  5%|▍         | 10759/225000 [32:42:29<434:21:39,  7.30s/it]
  5%|▍         | 10760/225000 [32:42:37<452:14:22,  7.60s/it]
  5%|▍         | 10761/225000 [32:42:45<453:44:50,  7.62s/it]
  5%|▍         | 10762/225000 [32:42:56<512:12:16,  8.61s/it]
  5%|▍         | 10763/225000 [32:43:01<443:33:31,  7.45s/it]
  5%|▍         | 10764/225000 [32:43:06<399:54:02,  6.72s/it]
  5%|▍         | 10765/225000 [32:43:11<372:44:42,  6.26s/it]
  5%|▍         | 10766/225000 [32:43:25<505:30:18,  8.49s/it]
  5%|▍         | 10767/225000 [32:43:33<508:57:04,  8.55s/it]
  5%|▍         | 10768/225000 [32:43:39<451:48:37,  7.59s/it]
  5%|▍         | 10769/225000 [32:43:44<405:13:31,  6.81s/it]
  5%|▍         | 10770/225000 [32:43:53<446:50:46,  7.51s/it]
  5%|▍         | 10771/225000 [32:44:02<483:16:40,  8.12s/it]
  5%|▍         | 10772/225000 [32:44:07<415:43:00,  6.99s/it]
  5%|▍         | 10773/225000 [32:44:13<402:51:55,  6.77s/it]
  5%|▍         | 10774/225000 [32:44:23<462:39:31,  7.77s/it]
  5%|▍         | 10775/225000 [32:44:33<496:37:12,  8.35s/it]
  5%|▍         | 10776/225000 [32:44:41<487:56:01,  8.20s/it]
  5%|▍         | 10777/225000 [32:44:44<408:24:22,  6.86s/it]
  5%|▍         | 10778/225000 [32:44:52<422:49:24,  7.11s/it]
  5%|▍         | 10779/225000 [32:44:56<373:45:46,  6.28s/it]
  5%|▍         | 10780/225000 [32:45:08<468:40:16,  7.88s/it]
  5%|▍         | 10781/225000 [32:45:14<433:30:48,  7.29s/it]
  5%|▍         | 10782/225000 [32:45:23<472:21:52,  7.94s/it]
  5%|▍         | 10783/225000 [32:45:27<397:14:45,  6.68s/it]
  5%|▍         | 10784/225000 [32:45:35<427:42:35,  7.19s/it]
  5%|▍         | 10785/225000 [32:45:40<380:54:30,  6.40s/it]
  5%|▍         | 10786/225000 [32:45:48<403:12:51,  6.78s/it]
  5%|▍         | 10787/225000 [32:45:55<416:46:19,  7.00s/it]
  5%|▍         | 10788/225000 [32:46:02<418:20:54,  7.03s/it]
  5%|▍         | 10789/225000 [32:46:10<421:58:09,  7.09s/it]
  5%|▍         | 10790/225000 [32:46:15<388:27:45,  6.53s/it]
  5%|▍         | 10791/225000 [32:46:22<395:12:18,  6.64s/it]
  5%|▍         | 10792/225000 [32:46:27<367:40:11,  6.18s/it]
  5%|▍         | 10793/225000 [32:46:31<339:57:34,  5.71s/it]
  5%|▍         | 10794/225000 [32:46:36<325:44:52,  5.47s/it]
  5%|▍         | 10795/225000 [32:46:42<320:56:05,  5.39s/it]
  5%|▍         | 10796/225000 [32:46:51<401:16:11,  6.74s/it]
  5%|▍         | 10797/225000 [32:46:59<419:47:35,  7.06s/it]
  5%|▍         | 10798/225000 [32:47:09<459:58:27,  7.73s/it]
  5%|▍         | 10799/225000 [32:47:16<462:17:14,  7.77s/it]
  5%|▍         | 10800/225000 [32:47:27<519:07:54,  8.72s/it]
                                                             
{'loss': '0.7382', 'grad_norm': '0.7703', 'learning_rate': '5e-05', 'epoch': '0.048', 'num_input_tokens_seen': 58858544, 'train_runtime': '1.18e+05', 'train_tokens_per_second': '498.6'}

  5%|▍         | 10800/225000 [32:47:27<519:07:54,  8.72s/it]
  5%|▍         | 10801/225000 [32:47:32<445:13:59,  7.48s/it]
  5%|▍         | 10802/225000 [32:47:37<404:39:31,  6.80s/it]
  5%|▍         | 10803/225000 [32:47:48<483:58:59,  8.13s/it]
  5%|▍         | 10804/225000 [32:47:52<411:10:27,  6.91s/it]
  5%|▍         | 10805/225000 [32:48:00<416:27:57,  7.00s/it]
  5%|▍         | 10806/225000 [32:48:05<391:17:25,  6.58s/it]
  5%|▍         | 10807/225000 [32:48:11<373:24:59,  6.28s/it]
  5%|▍         | 10808/225000 [32:48:24<496:10:29,  8.34s/it]
  5%|▍         | 10809/225000 [32:48:28<423:32:24,  7.12s/it]
  5%|▍         | 10810/225000 [32:48:37<448:01:06,  7.53s/it]
  5%|▍         | 10811/225000 [32:48:42<408:45:44,  6.87s/it]
  5%|▍         | 10812/225000 [32:48:47<377:56:30,  6.35s/it]
  5%|▍         | 10813/225000 [32:48:53<362:30:41,  6.09s/it]
  5%|▍         | 10814/225000 [32:48:58<340:20:02,  5.72s/it]
  5%|▍         | 10815/225000 [32:49:07<415:51:22,  6.99s/it]
  5%|▍         | 10816/225000 [32:49:17<453:24:53,  7.62s/it]
  5%|▍         | 10817/225000 [32:49:23<430:47:42,  7.24s/it]
  5%|▍         | 10818/225000 [32:49:30<436:03:51,  7.33s/it]
  5%|▍         | 10819/225000 [32:49:37<413:19:27,  6.95s/it]
  5%|▍         | 10820/225000 [32:49:45<435:41:46,  7.32s/it]
  5%|▍         | 10821/225000 [32:49:54<471:34:47,  7.93s/it]
  5%|▍         | 10822/225000 [32:49:58<409:23:17,  6.88s/it]
  5%|▍         | 10823/225000 [32:50:06<428:45:40,  7.21s/it]
  5%|▍         | 10824/225000 [32:50:12<394:30:02,  6.63s/it]
  5%|▍         | 10825/225000 [32:50:20<429:44:01,  7.22s/it]
  5%|▍         | 10826/225000 [32:50:27<425:00:08,  7.14s/it]
  5%|▍         | 10827/225000 [32:50:38<485:44:48,  8.16s/it]
  5%|▍         | 10828/225000 [32:50:46<486:27:12,  8.18s/it]
  5%|▍         | 10829/225000 [32:50:54<487:49:56,  8.20s/it]
  5%|▍         | 10830/225000 [32:50:59<431:59:11,  7.26s/it]
  5%|▍         | 10831/225000 [32:51:07<442:12:20,  7.43s/it]
  5%|▍         | 10832/225000 [32:51:15<445:54:25,  7.50s/it]
  5%|▍         | 10833/225000 [32:51:25<498:44:42,  8.38s/it]
  5%|▍         | 10834/225000 [32:51:31<457:24:14,  7.69s/it]
  5%|▍         | 10835/225000 [32:51:36<406:46:47,  6.84s/it]
  5%|▍         | 10836/225000 [32:51:48<496:48:53,  8.35s/it]
  5%|▍         | 10837/225000 [32:51:54<453:03:29,  7.62s/it]
  5%|▍         | 10838/225000 [32:51:59<412:29:05,  6.93s/it]
  5%|▍         | 10839/225000 [32:52:07<423:21:09,  7.12s/it]
  5%|▍         | 10840/225000 [32:52:14<431:07:40,  7.25s/it]
  5%|▍         | 10841/225000 [32:52:19<386:54:18,  6.50s/it]
  5%|▍         | 10842/225000 [32:52:26<390:54:27,  6.57s/it]
  5%|▍         | 10843/225000 [32:52:34<412:49:32,  6.94s/it]
  5%|▍         | 10844/225000 [32:52:41<426:44:06,  7.17s/it]
  5%|▍         | 10845/225000 [32:52:48<422:59:16,  7.11s/it]
  5%|▍         | 10846/225000 [32:52:53<377:19:31,  6.34s/it]
  5%|▍         | 10847/225000 [32:53:03<438:52:21,  7.38s/it]
  5%|▍         | 10848/225000 [32:53:10<438:27:20,  7.37s/it]
  5%|▍         | 10849/225000 [32:53:15<385:45:26,  6.48s/it]
  5%|▍         | 10850/225000 [32:53:22<406:01:33,  6.83s/it]
                                                             
{'loss': '0.7565', 'grad_norm': '0.743', 'learning_rate': '5e-05', 'epoch': '0.04822', 'num_input_tokens_seen': 59026896, 'train_runtime': '1.184e+05', 'train_tokens_per_second': '498.5'}

  5%|▍         | 10850/225000 [32:53:22<406:01:33,  6.83s/it]
  5%|▍         | 10851/225000 [32:53:27<370:05:25,  6.22s/it]
  5%|▍         | 10852/225000 [32:53:34<385:06:24,  6.47s/it]
  5%|▍         | 10853/225000 [32:53:45<456:25:40,  7.67s/it]
  5%|▍         | 10854/225000 [32:53:49<402:26:30,  6.77s/it]
  5%|▍         | 10855/225000 [32:53:54<367:42:15,  6.18s/it]
  5%|▍         | 10856/225000 [32:53:59<341:13:57,  5.74s/it]
  5%|▍         | 10857/225000 [32:54:06<376:22:47,  6.33s/it]
  5%|▍         | 10858/225000 [32:54:23<556:37:19,  9.36s/it]
  5%|▍         | 10859/225000 [32:54:27<462:21:53,  7.77s/it]
  5%|▍         | 10860/225000 [32:54:32<418:54:51,  7.04s/it]
  5%|▍         | 10861/225000 [32:54:38<398:33:30,  6.70s/it]
  5%|▍         | 10862/225000 [32:54:52<534:15:25,  8.98s/it]
  5%|▍         | 10863/225000 [32:54:57<449:45:20,  7.56s/it]
  5%|▍         | 10864/225000 [32:55:01<394:23:38,  6.63s/it]
  5%|▍         | 10865/225000 [32:55:05<351:15:42,  5.91s/it]
  5%|▍         | 10866/225000 [32:55:15<422:21:51,  7.10s/it]
  5%|▍         | 10867/225000 [32:55:23<435:22:55,  7.32s/it]
  5%|▍         | 10868/225000 [32:55:29<419:06:45,  7.05s/it]
  5%|▍         | 10869/225000 [32:55:40<477:42:58,  8.03s/it]
  5%|▍         | 10870/225000 [32:55:48<476:33:20,  8.01s/it]
  5%|▍         | 10871/225000 [32:55:53<432:04:35,  7.26s/it]
  5%|▍         | 10872/225000 [32:56:00<416:32:47,  7.00s/it]
  5%|▍         | 10873/225000 [32:56:06<404:59:14,  6.81s/it]
  5%|▍         | 10874/225000 [32:56:20<526:54:43,  8.86s/it]
  5%|▍         | 10875/225000 [32:56:25<455:27:17,  7.66s/it]
  5%|▍         | 10876/225000 [32:56:29<397:38:59,  6.69s/it]
  5%|▍         | 10877/225000 [32:56:34<368:53:30,  6.20s/it]
  5%|▍         | 10878/225000 [32:56:39<352:46:29,  5.93s/it]
  5%|▍         | 10879/225000 [32:56:44<324:36:42,  5.46s/it]
  5%|▍         | 10880/225000 [32:56:53<397:38:02,  6.69s/it]
  5%|▍         | 10881/225000 [32:57:01<424:22:05,  7.13s/it]
  5%|▍         | 10882/225000 [32:57:12<479:18:11,  8.06s/it]
  5%|▍         | 10883/225000 [32:57:20<484:08:24,  8.14s/it]
  5%|▍         | 10884/225000 [32:57:27<467:51:44,  7.87s/it]
  5%|▍         | 10885/225000 [32:57:34<450:03:29,  7.57s/it]
  5%|▍         | 10886/225000 [32:57:39<407:25:04,  6.85s/it]
  5%|▍         | 10887/225000 [32:57:49<466:13:51,  7.84s/it]
  5%|▍         | 10888/225000 [32:57:57<460:47:20,  7.75s/it]
  5%|▍         | 10889/225000 [32:58:02<406:29:09,  6.83s/it]
  5%|▍         | 10890/225000 [32:58:06<368:11:18,  6.19s/it]
  5%|▍         | 10891/225000 [32:58:17<456:56:01,  7.68s/it]
  5%|▍         | 10892/225000 [32:58:28<501:40:46,  8.44s/it]
  5%|▍         | 10893/225000 [32:58:34<464:49:45,  7.82s/it]
  5%|▍         | 10894/225000 [32:58:41<447:47:44,  7.53s/it]
  5%|▍         | 10895/225000 [32:58:45<390:05:30,  6.56s/it]
  5%|▍         | 10896/225000 [32:58:50<356:36:57,  6.00s/it]
  5%|▍         | 10897/225000 [32:58:55<344:33:02,  5.79s/it]
  5%|▍         | 10898/225000 [32:59:06<427:35:09,  7.19s/it]
  5%|▍         | 10899/225000 [32:59:13<437:17:33,  7.35s/it]
  5%|▍         | 10900/225000 [32:59:18<385:14:24,  6.48s/it]
                                                             
{'loss': '0.7514', 'grad_norm': '0.7185', 'learning_rate': '5e-05', 'epoch': '0.04844', 'num_input_tokens_seen': 59194640, 'train_runtime': '1.188e+05', 'train_tokens_per_second': '498.4'}

  5%|▍         | 10900/225000 [32:59:18<385:14:24,  6.48s/it]
  5%|▍         | 10901/225000 [32:59:23<354:51:56,  5.97s/it]
  5%|▍         | 10902/225000 [32:59:34<453:03:26,  7.62s/it]
  5%|▍         | 10903/225000 [32:59:41<446:39:26,  7.51s/it]
  5%|▍         | 10904/225000 [32:59:50<474:35:16,  7.98s/it]
  5%|▍         | 10905/225000 [32:59:58<460:53:05,  7.75s/it]
  5%|▍         | 10906/225000 [33:00:04<438:01:47,  7.37s/it]
  5%|▍         | 10907/225000 [33:00:08<377:04:11,  6.34s/it]
  5%|▍         | 10908/225000 [33:00:15<381:07:05,  6.41s/it]
  5%|▍         | 10909/225000 [33:00:25<452:46:58,  7.61s/it]
  5%|▍         | 10910/225000 [33:00:29<390:10:24,  6.56s/it]
  5%|▍         | 10911/225000 [33:00:39<444:55:55,  7.48s/it]
  5%|▍         | 10912/225000 [33:00:45<425:07:43,  7.15s/it]
  5%|▍         | 10913/225000 [33:00:50<378:31:13,  6.37s/it]
  5%|▍         | 10914/225000 [33:00:57<404:12:07,  6.80s/it]
  5%|▍         | 10915/225000 [33:01:07<453:57:19,  7.63s/it]
  5%|▍         | 10916/225000 [33:01:13<418:55:52,  7.04s/it]
  5%|▍         | 10917/225000 [33:01:21<432:15:17,  7.27s/it]
  5%|▍         | 10918/225000 [33:01:26<406:13:46,  6.83s/it]
  5%|▍         | 10919/225000 [33:01:36<460:04:52,  7.74s/it]
  5%|▍         | 10920/225000 [33:01:43<444:19:52,  7.47s/it]
  5%|▍         | 10921/225000 [33:01:50<426:38:27,  7.17s/it]
  5%|▍         | 10922/225000 [33:01:54<379:57:03,  6.39s/it]
  5%|▍         | 10923/225000 [33:02:01<394:02:55,  6.63s/it]
  5%|▍         | 10924/225000 [33:02:12<462:37:41,  7.78s/it]
  5%|▍         | 10925/225000 [33:02:19<453:54:14,  7.63s/it]
  5%|▍         | 10926/225000 [33:02:27<462:15:59,  7.77s/it]
  5%|▍         | 10927/225000 [33:02:37<503:43:11,  8.47s/it]
  5%|▍         | 10928/225000 [33:02:42<442:03:14,  7.43s/it]
  5%|▍         | 10929/225000 [33:02:47<393:25:52,  6.62s/it]
  5%|▍         | 10930/225000 [33:02:52<370:26:46,  6.23s/it]
  5%|▍         | 10931/225000 [33:03:05<492:04:03,  8.28s/it]
  5%|▍         | 10932/225000 [33:03:10<424:13:41,  7.13s/it]
  5%|▍         | 10933/225000 [33:03:17<431:02:44,  7.25s/it]
  5%|▍         | 10934/225000 [33:03:24<413:30:13,  6.95s/it]
  5%|▍         | 10935/225000 [33:03:28<371:02:41,  6.24s/it]
  5%|▍         | 10936/225000 [33:03:33<339:50:50,  5.72s/it]
  5%|▍         | 10937/225000 [33:03:37<320:54:50,  5.40s/it]
  5%|▍         | 10938/225000 [33:03:44<348:22:58,  5.86s/it]
  5%|▍         | 10939/225000 [33:04:00<533:39:45,  8.97s/it]
  5%|▍         | 10940/225000 [33:04:04<431:18:45,  7.25s/it]
  5%|▍         | 10941/225000 [33:04:16<525:10:16,  8.83s/it]
  5%|▍         | 10942/225000 [33:04:25<522:55:21,  8.79s/it]
  5%|▍         | 10943/225000 [33:04:30<450:46:44,  7.58s/it]
  5%|▍         | 10944/225000 [33:04:34<400:31:18,  6.74s/it]
  5%|▍         | 10945/225000 [33:04:39<366:00:26,  6.16s/it]
  5%|▍         | 10946/225000 [33:04:44<348:17:38,  5.86s/it]
  5%|▍         | 10947/225000 [33:04:56<455:01:42,  7.65s/it]
  5%|▍         | 10948/225000 [33:05:04<459:08:14,  7.72s/it]
  5%|▍         | 10949/225000 [33:05:10<431:57:26,  7.26s/it]
  5%|▍         | 10950/225000 [33:05:16<399:28:39,  6.72s/it]
                                                             
{'loss': '0.7473', 'grad_norm': '0.7205', 'learning_rate': '5e-05', 'epoch': '0.04867', 'num_input_tokens_seen': 59363024, 'train_runtime': '1.191e+05', 'train_tokens_per_second': '498.4'}

  5%|▍         | 10950/225000 [33:05:16<399:28:39,  6.72s/it]
  5%|▍         | 10951/225000 [33:05:21<375:36:39,  6.32s/it]
  5%|▍         | 10952/225000 [33:05:34<499:21:34,  8.40s/it]
  5%|▍         | 10953/225000 [33:05:44<513:04:55,  8.63s/it]
  5%|▍         | 10954/225000 [33:05:49<463:51:37,  7.80s/it]
  5%|▍         | 10955/225000 [33:05:55<420:12:28,  7.07s/it]
  5%|▍         | 10956/225000 [33:06:07<514:15:28,  8.65s/it]
  5%|▍         | 10957/225000 [33:06:11<437:04:25,  7.35s/it]
  5%|▍         | 10958/225000 [33:06:17<400:54:50,  6.74s/it]
  5%|▍         | 10959/225000 [33:06:25<423:48:51,  7.13s/it]
  5%|▍         | 10960/225000 [33:06:30<391:29:08,  6.58s/it]
  5%|▍         | 10961/225000 [33:06:38<422:35:50,  7.11s/it]
  5%|▍         | 10962/225000 [33:06:43<380:04:19,  6.39s/it]
  5%|▍         | 10963/225000 [33:06:51<405:37:01,  6.82s/it]
  5%|▍         | 10964/225000 [33:06:58<416:06:52,  7.00s/it]
  5%|▍         | 10965/225000 [33:07:07<439:16:06,  7.39s/it]
  5%|▍         | 10966/225000 [33:07:16<464:20:33,  7.81s/it]
  5%|▍         | 10967/225000 [33:07:19<388:40:24,  6.54s/it]
  5%|▍         | 10968/225000 [33:07:27<411:01:59,  6.91s/it]
  5%|▍         | 10969/225000 [33:07:31<364:49:34,  6.14s/it]
  5%|▍         | 10970/225000 [33:07:35<331:16:54,  5.57s/it]
  5%|▍         | 10971/225000 [33:07:43<374:45:39,  6.30s/it]
  5%|▍         | 10972/225000 [33:07:54<448:34:04,  7.55s/it]
  5%|▍         | 10973/225000 [33:08:01<441:37:54,  7.43s/it]
  5%|▍         | 10974/225000 [33:08:07<409:40:41,  6.89s/it]
  5%|▍         | 10975/225000 [33:08:16<448:26:56,  7.54s/it]
  5%|▍         | 10976/225000 [33:08:23<449:13:32,  7.56s/it]
  5%|▍         | 10977/225000 [33:08:36<537:39:28,  9.04s/it]
  5%|▍         | 10978/225000 [33:08:44<517:39:59,  8.71s/it]
  5%|▍         | 10979/225000 [33:08:48<438:16:17,  7.37s/it]
  5%|▍         | 10980/225000 [33:08:57<472:54:16,  7.95s/it]
  5%|▍         | 10981/225000 [33:09:03<423:14:27,  7.12s/it]
  5%|▍         | 10982/225000 [33:09:09<418:13:05,  7.03s/it]
  5%|▍         | 10983/225000 [33:09:19<456:00:28,  7.67s/it]
  5%|▍         | 10984/225000 [33:09:28<491:39:11,  8.27s/it]
  5%|▍         | 10985/225000 [33:09:33<428:21:51,  7.21s/it]
  5%|▍         | 10986/225000 [33:09:38<389:47:21,  6.56s/it]
  5%|▍         | 10987/225000 [33:09:44<385:41:14,  6.49s/it]
  5%|▍         | 10988/225000 [33:09:58<507:48:08,  8.54s/it]
  5%|▍         | 10989/225000 [33:10:01<421:15:01,  7.09s/it]
  5%|▍         | 10990/225000 [33:10:05<367:10:54,  6.18s/it]
  5%|▍         | 10991/225000 [33:10:15<432:56:45,  7.28s/it]
  5%|▍         | 10992/225000 [33:10:18<360:22:47,  6.06s/it]
  5%|▍         | 10993/225000 [33:10:27<402:24:26,  6.77s/it]
  5%|▍         | 10994/225000 [33:10:34<414:09:33,  6.97s/it]
  5%|▍         | 10995/225000 [33:10:39<379:31:11,  6.38s/it]
  5%|▍         | 10996/225000 [33:10:53<517:07:07,  8.70s/it]
  5%|▍         | 10997/225000 [33:10:57<433:05:46,  7.29s/it]
  5%|▍         | 10998/225000 [33:11:07<474:10:32,  7.98s/it]
  5%|▍         | 10999/225000 [33:11:13<440:08:13,  7.40s/it]
  5%|▍         | 11000/225000 [33:11:19<409:30:01,  6.89s/it]
                                                             
{'loss': '0.7649', 'grad_norm': '0.6865', 'learning_rate': '5e-05', 'epoch': '0.04889', 'num_input_tokens_seen': 59534368, 'train_runtime': '1.195e+05', 'train_tokens_per_second': '498.3'}

  5%|▍         | 11000/225000 [33:11:19<409:30:01,  6.89s/it]
  5%|▍         | 11001/225000 [33:11:24<380:41:25,  6.40s/it]
  5%|▍         | 11002/225000 [33:11:29<362:14:57,  6.09s/it]
  5%|▍         | 11003/225000 [33:11:35<350:59:23,  5.90s/it]
  5%|▍         | 11004/225000 [33:11:41<348:04:41,  5.86s/it]
  5%|▍         | 11005/225000 [33:11:53<470:34:36,  7.92s/it]
  5%|▍         | 11006/225000 [33:11:58<406:16:25,  6.83s/it]
  5%|▍         | 11007/225000 [33:12:10<496:39:36,  8.36s/it]
  5%|▍         | 11008/225000 [33:12:17<478:05:12,  8.04s/it]
  5%|▍         | 11009/225000 [33:12:22<418:53:24,  7.05s/it]
  5%|▍         | 11010/225000 [33:12:26<375:43:07,  6.32s/it]
  5%|▍         | 11011/225000 [33:12:31<350:38:34,  5.90s/it]
  5%|▍         | 11012/225000 [33:12:42<445:44:04,  7.50s/it]
  5%|▍         | 11013/225000 [33:12:51<462:17:05,  7.78s/it]
  5%|▍         | 11014/225000 [33:12:57<433:52:05,  7.30s/it]
  5%|▍         | 11015/225000 [33:13:02<400:15:01,  6.73s/it]
  5%|▍         | 11016/225000 [33:13:10<411:53:18,  6.93s/it]
  5%|▍         | 11017/225000 [33:13:15<385:20:53,  6.48s/it]
  5%|▍         | 11018/225000 [33:13:26<460:30:18,  7.75s/it]
  5%|▍         | 11019/225000 [33:13:30<388:06:01,  6.53s/it]
  5%|▍         | 11020/225000 [33:13:37<407:35:12,  6.86s/it]
  5%|▍         | 11021/225000 [33:13:45<422:18:54,  7.11s/it]
  5%|▍         | 11022/225000 [33:13:50<391:53:42,  6.59s/it]
  5%|▍         | 11023/225000 [33:14:01<458:00:50,  7.71s/it]
  5%|▍         | 11024/225000 [33:14:05<407:34:50,  6.86s/it]
  5%|▍         | 11025/225000 [33:14:11<378:26:12,  6.37s/it]
  5%|▍         | 11026/225000 [33:14:15<348:33:27,  5.86s/it]
  5%|▍         | 11027/225000 [33:14:21<339:22:11,  5.71s/it]
  5%|▍         | 11028/225000 [33:14:26<336:21:12,  5.66s/it]
  5%|▍         | 11029/225000 [33:14:32<333:12:06,  5.61s/it]
  5%|▍         | 11030/225000 [33:14:45<465:47:24,  7.84s/it]
  5%|▍         | 11031/225000 [33:14:50<418:06:59,  7.03s/it]
  5%|▍         | 11032/225000 [33:14:54<361:23:33,  6.08s/it]
  5%|▍         | 11033/225000 [33:14:58<329:57:52,  5.55s/it]
  5%|▍         | 11034/225000 [33:15:10<436:56:07,  7.35s/it]
  5%|▍         | 11035/225000 [33:15:16<417:23:30,  7.02s/it]
  5%|▍         | 11036/225000 [33:15:31<565:54:38,  9.52s/it]
  5%|▍         | 11037/225000 [33:15:35<465:24:25,  7.83s/it]
  5%|▍         | 11038/225000 [33:15:42<442:30:53,  7.45s/it]
  5%|▍         | 11039/225000 [33:15:46<387:07:32,  6.51s/it]
  5%|▍         | 11040/225000 [33:15:51<365:10:41,  6.14s/it]
  5%|▍         | 11041/225000 [33:15:56<338:53:14,  5.70s/it]
  5%|▍         | 11042/225000 [33:16:02<336:53:13,  5.67s/it]
  5%|▍         | 11043/225000 [33:16:14<458:48:42,  7.72s/it]
  5%|▍         | 11044/225000 [33:16:18<398:06:21,  6.70s/it]
  5%|▍         | 11045/225000 [33:16:25<403:40:54,  6.79s/it]
  5%|▍         | 11046/225000 [33:16:30<356:34:36,  6.00s/it]
  5%|▍         | 11047/225000 [33:16:35<344:19:55,  5.79s/it]
  5%|▍         | 11048/225000 [33:16:40<333:05:02,  5.60s/it]
  5%|▍         | 11049/225000 [33:16:53<461:09:09,  7.76s/it]
  5%|▍         | 11050/225000 [33:16:56<387:24:28,  6.52s/it]
                                                             
{'loss': '0.7483', 'grad_norm': '0.7423', 'learning_rate': '5e-05', 'epoch': '0.04911', 'num_input_tokens_seen': 59699216, 'train_runtime': '1.198e+05', 'train_tokens_per_second': '498.3'}

  5%|▍         | 11050/225000 [33:16:56<387:24:28,  6.52s/it]
  5%|▍         | 11051/225000 [33:17:05<428:46:53,  7.21s/it]
  5%|▍         | 11052/225000 [33:17:09<374:51:03,  6.31s/it]
  5%|▍         | 11053/225000 [33:17:18<418:36:43,  7.04s/it]
  5%|▍         | 11054/225000 [33:17:22<368:37:15,  6.20s/it]
  5%|▍         | 11055/225000 [33:17:32<424:25:11,  7.14s/it]
  5%|▍         | 11056/225000 [33:17:37<384:36:39,  6.47s/it]
  5%|▍         | 11057/225000 [33:17:47<443:33:37,  7.46s/it]
  5%|▍         | 11058/225000 [33:17:50<375:16:17,  6.31s/it]
  5%|▍         | 11059/225000 [33:18:02<465:35:33,  7.83s/it]
  5%|▍         | 11060/225000 [33:18:06<401:30:53,  6.76s/it]
  5%|▍         | 11061/225000 [33:18:11<377:45:09,  6.36s/it]
  5%|▍         | 11062/225000 [33:18:17<361:47:28,  6.09s/it]
  5%|▍         | 11063/225000 [33:18:27<440:09:42,  7.41s/it]
  5%|▍         | 11064/225000 [33:18:32<397:01:22,  6.68s/it]
  5%|▍         | 11065/225000 [33:18:36<349:16:50,  5.88s/it]
  5%|▍         | 11066/225000 [33:18:41<333:21:29,  5.61s/it]
  5%|▍         | 11067/225000 [33:18:55<484:19:18,  8.15s/it]
  5%|▍         | 11068/225000 [33:19:00<422:32:22,  7.11s/it]
  5%|▍         | 11069/225000 [33:19:04<373:41:48,  6.29s/it]
  5%|▍         | 11070/225000 [33:19:10<370:39:51,  6.24s/it]
  5%|▍         | 11071/225000 [33:19:20<432:13:26,  7.27s/it]
  5%|▍         | 11072/225000 [33:19:24<371:53:05,  6.26s/it]
  5%|▍         | 11073/225000 [33:19:31<378:15:28,  6.37s/it]
  5%|▍         | 11074/225000 [33:19:35<342:55:12,  5.77s/it]
  5%|▍         | 11075/225000 [33:19:45<413:34:22,  6.96s/it]
  5%|▍         | 11076/225000 [33:19:49<364:08:43,  6.13s/it]
  5%|▍         | 11077/225000 [33:19:53<334:10:57,  5.62s/it]
  5%|▍         | 11078/225000 [33:19:59<327:04:39,  5.50s/it]
  5%|▍         | 11079/225000 [33:20:15<515:15:23,  8.67s/it]
  5%|▍         | 11080/225000 [33:20:20<448:36:43,  7.55s/it]
  5%|▍         | 11081/225000 [33:20:25<415:21:10,  6.99s/it]
  5%|▍         | 11082/225000 [33:20:31<392:13:10,  6.60s/it]
  5%|▍         | 11083/225000 [33:20:40<439:18:08,  7.39s/it]
  5%|▍         | 11084/225000 [33:20:54<546:32:54,  9.20s/it]
  5%|▍         | 11085/225000 [33:21:00<496:52:56,  8.36s/it]
  5%|▍         | 11086/225000 [33:21:04<412:44:04,  6.95s/it]
  5%|▍         | 11087/225000 [33:21:13<458:40:01,  7.72s/it]
  5%|▍         | 11088/225000 [33:21:18<400:01:59,  6.73s/it]
  5%|▍         | 11089/225000 [33:21:24<389:40:39,  6.56s/it]
  5%|▍         | 11090/225000 [33:21:34<452:04:13,  7.61s/it]
  5%|▍         | 11091/225000 [33:21:38<397:33:50,  6.69s/it]
  5%|▍         | 11092/225000 [33:21:44<386:26:47,  6.50s/it]
  5%|▍         | 11093/225000 [33:21:50<368:43:33,  6.21s/it]
  5%|▍         | 11094/225000 [33:21:55<353:50:55,  5.96s/it]
  5%|▍         | 11095/225000 [33:22:01<355:02:18,  5.98s/it]
  5%|▍         | 11096/225000 [33:22:07<342:26:24,  5.76s/it]
  5%|▍         | 11097/225000 [33:22:21<493:25:59,  8.30s/it]
  5%|▍         | 11098/225000 [33:22:25<416:03:32,  7.00s/it]
  5%|▍         | 11099/225000 [33:22:32<429:23:55,  7.23s/it]
  5%|▍         | 11100/225000 [33:22:38<393:38:46,  6.63s/it]
                                                             
{'loss': '0.7612', 'grad_norm': '0.766', 'learning_rate': '5e-05', 'epoch': '0.04933', 'num_input_tokens_seen': 59861832, 'train_runtime': '1.202e+05', 'train_tokens_per_second': '498.2'}

  5%|▍         | 11100/225000 [33:22:38<393:38:46,  6.63s/it]
  5%|▍         | 11101/225000 [33:22:45<399:44:20,  6.73s/it]
  5%|▍         | 11102/225000 [33:22:49<356:15:22,  6.00s/it]
  5%|▍         | 11103/225000 [33:22:54<332:00:24,  5.59s/it]
  5%|▍         | 11104/225000 [33:23:05<436:26:03,  7.35s/it]
  5%|▍         | 11105/225000 [33:23:10<385:41:54,  6.49s/it]
  5%|▍         | 11106/225000 [33:23:14<357:02:05,  6.01s/it]
  5%|▍         | 11107/225000 [33:23:19<333:29:59,  5.61s/it]
  5%|▍         | 11108/225000 [33:23:27<366:17:08,  6.16s/it]
  5%|▍         | 11109/225000 [33:23:39<482:00:04,  8.11s/it]
  5%|▍         | 11110/225000 [33:23:43<404:53:09,  6.81s/it]
  5%|▍         | 11111/225000 [33:23:50<416:01:59,  7.00s/it]
  5%|▍         | 11112/225000 [33:23:54<362:10:32,  6.10s/it]
  5%|▍         | 11113/225000 [33:24:02<396:46:34,  6.68s/it]
  5%|▍         | 11114/225000 [33:24:10<404:07:20,  6.80s/it]
  5%|▍         | 11115/225000 [33:24:16<393:19:11,  6.62s/it]
  5%|▍         | 11116/225000 [33:24:23<409:34:41,  6.89s/it]
  5%|▍         | 11117/225000 [33:24:30<410:39:30,  6.91s/it]
  5%|▍         | 11118/225000 [33:24:36<387:41:25,  6.53s/it]
  5%|▍         | 11119/225000 [33:24:48<485:01:03,  8.16s/it]
  5%|▍         | 11120/225000 [33:24:59<533:24:40,  8.98s/it]
  5%|▍         | 11121/225000 [33:25:04<461:20:36,  7.77s/it]
  5%|▍         | 11122/225000 [33:25:09<416:23:07,  7.01s/it]
  5%|▍         | 11123/225000 [33:25:14<388:20:21,  6.54s/it]
  5%|▍         | 11124/225000 [33:25:29<537:23:36,  9.05s/it]
  5%|▍         | 11125/225000 [33:25:33<437:13:01,  7.36s/it]
  5%|▍         | 11126/225000 [33:25:39<423:34:09,  7.13s/it]
  5%|▍         | 11127/225000 [33:25:45<389:39:43,  6.56s/it]
  5%|▍         | 11128/225000 [33:25:49<348:24:41,  5.86s/it]
  5%|▍         | 11129/225000 [33:26:01<464:41:25,  7.82s/it]
  5%|▍         | 11130/225000 [33:26:06<407:46:00,  6.86s/it]
  5%|▍         | 11131/225000 [33:26:13<417:40:35,  7.03s/it]
  5%|▍         | 11132/225000 [33:26:17<366:59:14,  6.18s/it]
  5%|▍         | 11133/225000 [33:26:22<345:22:38,  5.81s/it]
  5%|▍         | 11134/225000 [33:26:27<332:33:39,  5.60s/it]
  5%|▍         | 11135/225000 [33:26:36<381:23:17,  6.42s/it]
  5%|▍         | 11136/225000 [33:26:47<459:46:00,  7.74s/it]
  5%|▍         | 11137/225000 [33:26:53<431:02:54,  7.26s/it]
  5%|▍         | 11138/225000 [33:26:59<422:30:00,  7.11s/it]
  5%|▍         | 11139/225000 [33:27:04<379:29:39,  6.39s/it]
  5%|▍         | 11140/225000 [33:27:17<497:07:59,  8.37s/it]
  5%|▍         | 11141/225000 [33:27:24<463:32:07,  7.80s/it]
  5%|▍         | 11142/225000 [33:27:31<455:20:36,  7.67s/it]
  5%|▍         | 11143/225000 [33:27:37<416:39:44,  7.01s/it]
  5%|▍         | 11144/225000 [33:27:42<392:50:17,  6.61s/it]
  5%|▍         | 11145/225000 [33:27:48<373:17:34,  6.28s/it]
  5%|▍         | 11146/225000 [33:27:53<355:12:43,  5.98s/it]
  5%|▍         | 11147/225000 [33:28:04<438:45:06,  7.39s/it]
  5%|▍         | 11148/225000 [33:28:09<397:51:06,  6.70s/it]
  5%|▍         | 11149/225000 [33:28:14<365:27:13,  6.15s/it]
  5%|▍         | 11150/225000 [33:28:19<351:18:23,  5.91s/it]
                                                             
{'loss': '0.7471', 'grad_norm': '0.6515', 'learning_rate': '5e-05', 'epoch': '0.04956', 'num_input_tokens_seen': 60026976, 'train_runtime': '1.205e+05', 'train_tokens_per_second': '498.2'}

  5%|▍         | 11150/225000 [33:28:19<351:18:23,  5.91s/it]
  5%|▍         | 11151/225000 [33:28:26<371:21:04,  6.25s/it]
  5%|▍         | 11152/225000 [33:28:40<510:08:39,  8.59s/it]
  5%|▍         | 11153/225000 [33:28:46<471:21:49,  7.94s/it]
  5%|▍         | 11154/225000 [33:28:59<546:42:47,  9.20s/it]
  5%|▍         | 11155/225000 [33:29:03<457:54:51,  7.71s/it]
  5%|▍         | 11156/225000 [33:29:08<406:53:19,  6.85s/it]
  5%|▍         | 11157/225000 [33:29:17<451:26:34,  7.60s/it]
  5%|▍         | 11158/225000 [33:29:21<394:23:56,  6.64s/it]
  5%|▍         | 11159/225000 [33:29:28<386:36:48,  6.51s/it]
  5%|▍         | 11160/225000 [33:29:34<392:15:31,  6.60s/it]
  5%|▍         | 11161/225000 [33:29:45<464:02:19,  7.81s/it]
  5%|▍         | 11162/225000 [33:29:52<439:26:06,  7.40s/it]
  5%|▍         | 11163/225000 [33:29:56<390:24:44,  6.57s/it]
  5%|▍         | 11164/225000 [33:30:01<362:18:39,  6.10s/it]
  5%|▍         | 11165/225000 [33:30:12<448:22:58,  7.55s/it]
  5%|▍         | 11166/225000 [33:30:21<466:48:37,  7.86s/it]
  5%|▍         | 11167/225000 [33:30:28<448:40:35,  7.55s/it]
  5%|▍         | 11168/225000 [33:30:34<430:33:52,  7.25s/it]
  5%|▍         | 11169/225000 [33:30:40<416:11:03,  7.01s/it]
  5%|▍         | 11170/225000 [33:30:48<422:38:35,  7.12s/it]
  5%|▍         | 11171/225000 [33:30:52<371:48:45,  6.26s/it]
  5%|▍         | 11172/225000 [33:31:02<428:08:53,  7.21s/it]
  5%|▍         | 11173/225000 [33:31:08<408:15:09,  6.87s/it]
  5%|▍         | 11174/225000 [33:31:12<366:49:33,  6.18s/it]
  5%|▍         | 11175/225000 [33:31:17<350:06:10,  5.89s/it]
  5%|▍         | 11176/225000 [33:31:29<445:08:32,  7.49s/it]
  5%|▍         | 11177/225000 [33:31:34<405:06:19,  6.82s/it]
  5%|▍         | 11178/225000 [33:31:39<382:31:44,  6.44s/it]
  5%|▍         | 11179/225000 [33:31:50<452:31:55,  7.62s/it]
  5%|▍         | 11180/225000 [33:31:54<395:08:02,  6.65s/it]
  5%|▍         | 11181/225000 [33:32:01<396:56:21,  6.68s/it]
  5%|▍         | 11182/225000 [33:32:07<390:33:55,  6.58s/it]
  5%|▍         | 11183/225000 [33:32:12<363:51:57,  6.13s/it]
  5%|▍         | 11184/225000 [33:32:25<482:04:26,  8.12s/it]
  5%|▍         | 11185/225000 [33:32:29<412:20:09,  6.94s/it]
  5%|▍         | 11186/225000 [33:32:37<422:58:40,  7.12s/it]
  5%|▍         | 11187/225000 [33:32:43<401:12:11,  6.76s/it]
  5%|▍         | 11188/225000 [33:32:50<403:54:34,  6.80s/it]
  5%|▍         | 11189/225000 [33:32:59<444:48:18,  7.49s/it]
  5%|▍         | 11190/225000 [33:33:05<424:29:21,  7.15s/it]
  5%|▍         | 11191/225000 [33:33:11<394:51:38,  6.65s/it]
  5%|▍         | 11192/225000 [33:33:22<471:34:20,  7.94s/it]
  5%|▍         | 11193/225000 [33:33:27<427:03:33,  7.19s/it]
  5%|▍         | 11194/225000 [33:33:38<486:09:01,  8.19s/it]
  5%|▍         | 11195/225000 [33:33:45<466:57:34,  7.86s/it]
  5%|▍         | 11196/225000 [33:33:49<405:06:57,  6.82s/it]
  5%|▍         | 11197/225000 [33:33:53<358:48:53,  6.04s/it]
  5%|▍         | 11198/225000 [33:33:58<329:29:27,  5.55s/it]
  5%|▍         | 11199/225000 [33:34:11<470:26:32,  7.92s/it]
  5%|▍         | 11200/225000 [33:34:16<408:59:44,  6.89s/it]
                                                             
{'loss': '0.7579', 'grad_norm': '0.6189', 'learning_rate': '5e-05', 'epoch': '0.04978', 'num_input_tokens_seen': 60194704, 'train_runtime': '1.209e+05', 'train_tokens_per_second': '498.1'}

  5%|▍         | 11200/225000 [33:34:16<408:59:44,  6.89s/it]
  5%|▍         | 11201/225000 [33:34:25<462:00:19,  7.78s/it]
  5%|▍         | 11202/225000 [33:34:30<402:49:59,  6.78s/it]
  5%|▍         | 11203/225000 [33:34:35<379:53:36,  6.40s/it]
  5%|▍         | 11204/225000 [33:34:45<436:37:35,  7.35s/it]
  5%|▍         | 11205/225000 [33:34:52<424:18:32,  7.14s/it]
  5%|▍         | 11206/225000 [33:34:59<432:46:21,  7.29s/it]
  5%|▍         | 11207/225000 [33:35:04<385:41:41,  6.49s/it]
  5%|▍         | 11208/225000 [33:35:11<398:54:56,  6.72s/it]
  5%|▍         | 11209/225000 [33:35:18<397:32:16,  6.69s/it]
  5%|▍         | 11210/225000 [33:35:24<396:06:13,  6.67s/it]
  5%|▍         | 11211/225000 [33:35:35<457:37:33,  7.71s/it]
  5%|▍         | 11212/225000 [33:35:43<473:36:39,  7.98s/it]
  5%|▍         | 11213/225000 [33:35:55<550:48:00,  9.28s/it]
  5%|▍         | 11214/225000 [33:36:01<484:07:44,  8.15s/it]
  5%|▍         | 11215/225000 [33:36:09<475:21:37,  8.00s/it]
  5%|▍         | 11216/225000 [33:36:16<460:00:23,  7.75s/it]
  5%|▍         | 11217/225000 [33:36:22<439:39:50,  7.40s/it]
  5%|▍         | 11218/225000 [33:36:30<451:32:15,  7.60s/it]
  5%|▍         | 11219/225000 [33:36:38<453:11:11,  7.63s/it]
  5%|▍         | 11220/225000 [33:36:47<472:22:21,  7.95s/it]
  5%|▍         | 11221/225000 [33:36:55<473:54:26,  7.98s/it]
  5%|▍         | 11222/225000 [33:37:05<503:32:25,  8.48s/it]
  5%|▍         | 11223/225000 [33:37:09<438:05:41,  7.38s/it]
  5%|▍         | 11224/225000 [33:37:17<439:45:21,  7.41s/it]
  5%|▍         | 11225/225000 [33:37:22<393:11:50,  6.62s/it]
  5%|▍         | 11226/225000 [33:37:31<450:01:10,  7.58s/it]
  5%|▍         | 11227/225000 [33:37:38<430:48:16,  7.25s/it]
  5%|▍         | 11228/225000 [33:37:46<444:33:10,  7.49s/it]
  5%|▍         | 11229/225000 [33:37:52<414:24:17,  6.98s/it]
  5%|▍         | 11230/225000 [33:37:57<391:03:25,  6.59s/it]
  5%|▍         | 11231/225000 [33:38:03<372:18:44,  6.27s/it]
  5%|▍         | 11232/225000 [33:38:14<451:56:43,  7.61s/it]
  5%|▍         | 11233/225000 [33:38:23<482:20:33,  8.12s/it]
  5%|▍         | 11234/225000 [33:38:27<413:42:54,  6.97s/it]
  5%|▍         | 11235/225000 [33:38:35<423:20:27,  7.13s/it]
  5%|▍         | 11236/225000 [33:38:39<374:19:13,  6.30s/it]
  5%|▍         | 11237/225000 [33:38:50<455:18:12,  7.67s/it]
  5%|▍         | 11238/225000 [33:38:55<403:13:37,  6.79s/it]
  5%|▍         | 11239/225000 [33:39:01<388:26:53,  6.54s/it]
  5%|▍         | 11240/225000 [33:39:06<368:31:55,  6.21s/it]
  5%|▍         | 11241/225000 [33:39:11<343:47:35,  5.79s/it]
  5%|▍         | 11242/225000 [33:39:17<343:34:19,  5.79s/it]
  5%|▍         | 11243/225000 [33:39:23<353:56:20,  5.96s/it]
  5%|▍         | 11244/225000 [33:39:37<503:23:33,  8.48s/it]
  5%|▍         | 11245/225000 [33:39:43<452:40:43,  7.62s/it]
  5%|▍         | 11246/225000 [33:39:49<418:12:12,  7.04s/it]
  5%|▍         | 11247/225000 [33:39:54<380:36:54,  6.41s/it]
  5%|▍         | 11248/225000 [33:39:59<360:00:48,  6.06s/it]
  5%|▍         | 11249/225000 [33:40:12<484:10:39,  8.15s/it]
  5%|▌         | 11250/225000 [33:40:17<420:38:22,  7.08s/it]
                                                             
{'loss': '0.7397', 'grad_norm': '0.6474', 'learning_rate': '5e-05', 'epoch': '0.05', 'num_input_tokens_seen': 60364056, 'train_runtime': '1.212e+05', 'train_tokens_per_second': '498'}

  5%|▌         | 11250/225000 [33:40:17<420:38:22,  7.08s/it]
  5%|▌         | 11251/225000 [33:40:25<436:10:06,  7.35s/it]
  5%|▌         | 11252/225000 [33:40:38<550:38:42,  9.27s/it]
  5%|▌         | 11253/225000 [33:40:43<462:22:38,  7.79s/it]
  5%|▌         | 11254/225000 [33:40:48<423:11:33,  7.13s/it]
  5%|▌         | 11255/225000 [33:40:57<461:07:54,  7.77s/it]
  5%|▌         | 11256/225000 [33:41:05<455:36:56,  7.67s/it]
  5%|▌         | 11257/225000 [33:41:09<391:10:40,  6.59s/it]
  5%|▌         | 11258/225000 [33:41:17<422:22:48,  7.11s/it]
  5%|▌         | 11259/225000 [33:41:21<358:49:07,  6.04s/it]
  5%|▌         | 11260/225000 [33:41:32<454:46:52,  7.66s/it]
  5%|▌         | 11261/225000 [33:41:37<405:14:28,  6.83s/it]
  5%|▌         | 11262/225000 [33:41:46<440:05:47,  7.41s/it]
  5%|▌         | 11263/225000 [33:41:50<381:00:12,  6.42s/it]
  5%|▌         | 11264/225000 [33:41:58<406:50:04,  6.85s/it]
  5%|▌         | 11265/225000 [33:42:11<509:17:25,  8.58s/it]
  5%|▌         | 11266/225000 [33:42:17<472:12:04,  7.95s/it]
  5%|▌         | 11267/225000 [33:42:25<468:10:10,  7.89s/it]
  5%|▌         | 11268/225000 [33:42:35<502:23:45,  8.46s/it]
  5%|▌         | 11269/225000 [33:42:39<436:18:05,  7.35s/it]
  5%|▌         | 11270/225000 [33:42:44<391:24:00,  6.59s/it]
  5%|▌         | 11271/225000 [33:42:51<394:53:18,  6.65s/it]
  5%|▌         | 11272/225000 [33:42:57<392:31:36,  6.61s/it]
  5%|▌         | 11273/225000 [33:43:14<569:35:41,  9.59s/it]
  5%|▌         | 11274/225000 [33:43:20<501:49:43,  8.45s/it]
  5%|▌         | 11275/225000 [33:43:25<442:51:23,  7.46s/it]
  5%|▌         | 11276/225000 [33:43:30<395:33:46,  6.66s/it]
  5%|▌         | 11277/225000 [33:43:36<393:55:01,  6.64s/it]
  5%|▌         | 11278/225000 [33:43:41<362:19:46,  6.10s/it]
  5%|▌         | 11279/225000 [33:43:48<377:38:10,  6.36s/it]
  5%|▌         | 11280/225000 [33:43:57<416:05:01,  7.01s/it]
  5%|▌         | 11281/225000 [33:44:04<431:05:05,  7.26s/it]
  5%|▌         | 11282/225000 [33:44:10<405:36:23,  6.83s/it]
  5%|▌         | 11283/225000 [33:44:16<376:43:48,  6.35s/it]
  5%|▌         | 11284/225000 [33:44:21<361:22:55,  6.09s/it]
  5%|▌         | 11285/225000 [33:44:26<342:06:33,  5.76s/it]
  5%|▌         | 11286/225000 [33:44:36<409:46:58,  6.90s/it]
  5%|▌         | 11287/225000 [33:44:45<458:48:12,  7.73s/it]
  5%|▌         | 11288/225000 [33:44:50<398:04:31,  6.71s/it]
  5%|▌         | 11289/225000 [33:44:58<435:52:40,  7.34s/it]
  5%|▌         | 11290/225000 [33:45:03<389:36:25,  6.56s/it]
  5%|▌         | 11291/225000 [33:45:09<382:01:29,  6.44s/it]
  5%|▌         | 11292/225000 [33:45:17<409:13:15,  6.89s/it]
  5%|▌         | 11293/225000 [33:45:22<375:53:28,  6.33s/it]
  5%|▌         | 11294/225000 [33:45:27<346:15:23,  5.83s/it]
  5%|▌         | 11295/225000 [33:45:33<355:42:27,  5.99s/it]
  5%|▌         | 11296/225000 [33:45:38<338:24:15,  5.70s/it]
  5%|▌         | 11297/225000 [33:45:44<338:49:24,  5.71s/it]
  5%|▌         | 11298/225000 [33:45:55<433:39:34,  7.31s/it]
  5%|▌         | 11299/225000 [33:46:00<384:34:39,  6.48s/it]
  5%|▌         | 11300/225000 [33:46:05<365:17:50,  6.15s/it]
                                                             
{'loss': '0.7534', 'grad_norm': '0.7135', 'learning_rate': '5e-05', 'epoch': '0.05022', 'num_input_tokens_seen': 60529504, 'train_runtime': '1.216e+05', 'train_tokens_per_second': '497.9'}

  5%|▌         | 11300/225000 [33:46:05<365:17:50,  6.15s/it]
  5%|▌         | 11301/225000 [33:46:18<484:54:36,  8.17s/it]
  5%|▌         | 11302/225000 [33:46:22<412:08:30,  6.94s/it]
  5%|▌         | 11303/225000 [33:46:30<438:16:18,  7.38s/it]
  5%|▌         | 11304/225000 [33:46:35<380:48:28,  6.42s/it]
  5%|▌         | 11305/225000 [33:46:44<428:44:44,  7.22s/it]
  5%|▌         | 11306/225000 [33:46:48<373:06:27,  6.29s/it]
  5%|▌         | 11307/225000 [33:46:56<415:11:08,  6.99s/it]
  5%|▌         | 11308/225000 [33:47:03<417:03:22,  7.03s/it]
  5%|▌         | 11309/225000 [33:47:14<487:36:14,  8.21s/it]
  5%|▌         | 11310/225000 [33:47:20<444:36:37,  7.49s/it]
  5%|▌         | 11311/225000 [33:47:26<411:01:00,  6.92s/it]
  5%|▌         | 11312/225000 [33:47:33<406:35:47,  6.85s/it]
  5%|▌         | 11313/225000 [33:47:38<388:16:21,  6.54s/it]
  5%|▌         | 11314/225000 [33:47:44<376:00:25,  6.33s/it]
  5%|▌         | 11315/225000 [33:47:54<444:43:11,  7.49s/it]
  5%|▌         | 11316/225000 [33:47:58<381:58:24,  6.44s/it]
  5%|▌         | 11317/225000 [33:48:06<401:42:48,  6.77s/it]
  5%|▌         | 11318/225000 [33:48:13<410:51:45,  6.92s/it]
  5%|▌         | 11319/225000 [33:48:20<408:18:56,  6.88s/it]
  5%|▌         | 11320/225000 [33:48:24<359:05:34,  6.05s/it]
  5%|▌         | 11321/225000 [33:48:33<400:53:19,  6.75s/it]
  5%|▌         | 11322/225000 [33:48:37<355:54:09,  6.00s/it]
  5%|▌         | 11323/225000 [33:48:45<391:30:21,  6.60s/it]
  5%|▌         | 11324/225000 [33:48:52<406:56:27,  6.86s/it]
  5%|▌         | 11325/225000 [33:49:03<473:02:31,  7.97s/it]
  5%|▌         | 11326/225000 [33:49:12<494:48:07,  8.34s/it]
  5%|▌         | 11327/225000 [33:49:16<417:46:37,  7.04s/it]
  5%|▌         | 11328/225000 [33:49:27<483:20:19,  8.14s/it]
  5%|▌         | 11329/225000 [33:49:33<459:04:16,  7.73s/it]
  5%|▌         | 11330/225000 [33:49:38<405:22:17,  6.83s/it]
  5%|▌         | 11331/225000 [33:49:45<409:10:12,  6.89s/it]
  5%|▌         | 11332/225000 [33:49:56<484:20:22,  8.16s/it]
  5%|▌         | 11333/225000 [33:50:00<402:01:13,  6.77s/it]
  5%|▌         | 11334/225000 [33:50:09<437:53:00,  7.38s/it]
  5%|▌         | 11335/225000 [33:50:13<378:57:21,  6.38s/it]
  5%|▌         | 11336/225000 [33:50:24<468:09:17,  7.89s/it]
  5%|▌         | 11337/225000 [33:50:29<416:29:37,  7.02s/it]
  5%|▌         | 11338/225000 [33:50:36<407:01:02,  6.86s/it]
  5%|▌         | 11339/225000 [33:50:42<397:02:51,  6.69s/it]
  5%|▌         | 11340/225000 [33:50:47<363:09:52,  6.12s/it]
  5%|▌         | 11341/225000 [33:51:00<483:36:48,  8.15s/it]
  5%|▌         | 11342/225000 [33:51:04<410:24:22,  6.92s/it]
  5%|▌         | 11343/225000 [33:51:12<430:56:08,  7.26s/it]
  5%|▌         | 11344/225000 [33:51:20<441:41:00,  7.44s/it]
  5%|▌         | 11345/225000 [33:51:25<400:41:27,  6.75s/it]
  5%|▌         | 11346/225000 [33:51:30<368:35:14,  6.21s/it]
  5%|▌         | 11347/225000 [33:51:40<446:20:11,  7.52s/it]
  5%|▌         | 11348/225000 [33:51:52<527:43:53,  8.89s/it]
  5%|▌         | 11349/225000 [33:51:56<439:53:19,  7.41s/it]
  5%|▌         | 11350/225000 [33:52:03<434:43:03,  7.32s/it]
                                                             
{'loss': '0.7729', 'grad_norm': '0.6948', 'learning_rate': '5e-05', 'epoch': '0.05044', 'num_input_tokens_seen': 60699144, 'train_runtime': '1.219e+05', 'train_tokens_per_second': '497.8'}

  5%|▌         | 11350/225000 [33:52:03<434:43:03,  7.32s/it]
  5%|▌         | 11351/225000 [33:52:09<410:26:11,  6.92s/it]
  5%|▌         | 11352/225000 [33:52:17<423:08:32,  7.13s/it]
  5%|▌         | 11353/225000 [33:52:23<405:43:01,  6.84s/it]
  5%|▌         | 11354/225000 [33:52:37<532:26:10,  8.97s/it]
  5%|▌         | 11355/225000 [33:52:41<442:45:37,  7.46s/it]
  5%|▌         | 11356/225000 [33:52:50<466:43:15,  7.86s/it]
  5%|▌         | 11357/225000 [33:52:54<403:13:19,  6.79s/it]
  5%|▌         | 11358/225000 [33:53:02<425:17:30,  7.17s/it]
  5%|▌         | 11359/225000 [33:53:06<366:43:59,  6.18s/it]
  5%|▌         | 11360/225000 [33:53:15<423:03:51,  7.13s/it]
  5%|▌         | 11361/225000 [33:53:21<401:07:06,  6.76s/it]
  5%|▌         | 11362/225000 [33:53:31<461:38:21,  7.78s/it]
  5%|▌         | 11363/225000 [33:53:40<478:18:19,  8.06s/it]
  5%|▌         | 11364/225000 [33:53:46<442:29:49,  7.46s/it]
  5%|▌         | 11365/225000 [33:53:51<399:50:05,  6.74s/it]
  5%|▌         | 11366/225000 [33:54:01<452:07:36,  7.62s/it]
  5%|▌         | 11367/225000 [33:54:08<448:43:49,  7.56s/it]
  5%|▌         | 11368/225000 [33:54:15<441:00:41,  7.43s/it]
  5%|▌         | 11369/225000 [33:54:21<399:48:26,  6.74s/it]
  5%|▌         | 11370/225000 [33:54:26<373:49:11,  6.30s/it]
  5%|▌         | 11371/225000 [33:54:31<351:29:48,  5.92s/it]
  5%|▌         | 11372/225000 [33:54:36<338:39:02,  5.71s/it]
  5%|▌         | 11373/225000 [33:54:42<342:06:09,  5.77s/it]
  5%|▌         | 11374/225000 [33:54:47<332:05:25,  5.60s/it]
  5%|▌         | 11375/225000 [33:55:03<516:30:49,  8.70s/it]
  5%|▌         | 11376/225000 [33:55:07<426:26:50,  7.19s/it]
  5%|▌         | 11377/225000 [33:55:16<469:06:55,  7.91s/it]
  5%|▌         | 11378/225000 [33:55:21<417:51:19,  7.04s/it]
  5%|▌         | 11379/225000 [33:55:27<388:53:46,  6.55s/it]
  5%|▌         | 11380/225000 [33:55:31<354:48:42,  5.98s/it]
  5%|▌         | 11381/225000 [33:55:36<334:17:06,  5.63s/it]
  5%|▌         | 11382/225000 [33:55:48<446:02:10,  7.52s/it]
  5%|▌         | 11383/225000 [33:55:53<394:39:12,  6.65s/it]
  5%|▌         | 11384/225000 [33:55:58<364:35:36,  6.14s/it]
  5%|▌         | 11385/225000 [33:56:13<518:01:03,  8.73s/it]
  5%|▌         | 11386/225000 [33:56:18<467:15:10,  7.87s/it]
  5%|▌         | 11387/225000 [33:56:23<404:06:12,  6.81s/it]
  5%|▌         | 11388/225000 [33:56:31<435:05:48,  7.33s/it]
  5%|▌         | 11389/225000 [33:56:37<404:52:25,  6.82s/it]
  5%|▌         | 11390/225000 [33:56:44<402:31:24,  6.78s/it]
  5%|▌         | 11391/225000 [33:56:50<401:40:08,  6.77s/it]
  5%|▌         | 11392/225000 [33:56:58<422:22:36,  7.12s/it]
  5%|▌         | 11393/225000 [33:57:05<417:09:31,  7.03s/it]
  5%|▌         | 11394/225000 [33:57:16<490:52:08,  8.27s/it]
  5%|▌         | 11395/225000 [33:57:22<450:33:25,  7.59s/it]
  5%|▌         | 11396/225000 [33:57:27<404:58:32,  6.83s/it]
  5%|▌         | 11397/225000 [33:57:39<481:43:44,  8.12s/it]
  5%|▌         | 11398/225000 [33:57:43<424:30:56,  7.15s/it]
  5%|▌         | 11399/225000 [33:57:48<377:04:23,  6.36s/it]
  5%|▌         | 11400/225000 [33:57:53<362:44:21,  6.11s/it]
                                                             
{'loss': '0.7542', 'grad_norm': '0.5622', 'learning_rate': '4.999e-05', 'epoch': '0.05067', 'num_input_tokens_seen': 60866552, 'train_runtime': '1.223e+05', 'train_tokens_per_second': '497.8'}

  5%|▌         | 11400/225000 [33:57:53<362:44:21,  6.11s/it]
  5%|▌         | 11401/225000 [33:58:03<428:29:47,  7.22s/it]
  5%|▌         | 11402/225000 [33:58:12<455:10:07,  7.67s/it]
  5%|▌         | 11403/225000 [33:58:22<489:42:02,  8.25s/it]
  5%|▌         | 11404/225000 [33:58:26<427:31:02,  7.21s/it]
  5%|▌         | 11405/225000 [33:58:31<375:26:58,  6.33s/it]
  5%|▌         | 11406/225000 [33:58:37<376:42:52,  6.35s/it]
  5%|▌         | 11407/225000 [33:58:50<491:41:52,  8.29s/it]
  5%|▌         | 11408/225000 [33:58:57<475:16:48,  8.01s/it]
  5%|▌         | 11409/225000 [33:59:05<466:18:06,  7.86s/it]
  5%|▌         | 11410/225000 [33:59:10<426:26:00,  7.19s/it]
  5%|▌         | 11411/225000 [33:59:21<483:34:29,  8.15s/it]
  5%|▌         | 11412/225000 [33:59:25<418:37:45,  7.06s/it]
  5%|▌         | 11413/225000 [33:59:31<386:47:40,  6.52s/it]
  5%|▌         | 11414/225000 [33:59:36<361:22:38,  6.09s/it]
  5%|▌         | 11415/225000 [33:59:47<457:29:28,  7.71s/it]
  5%|▌         | 11416/225000 [33:59:52<411:00:07,  6.93s/it]
  5%|▌         | 11417/225000 [33:59:57<374:24:23,  6.31s/it]
  5%|▌         | 11418/225000 [34:00:10<487:12:56,  8.21s/it]
  5%|▌         | 11419/225000 [34:00:15<443:14:29,  7.47s/it]
  5%|▌         | 11420/225000 [34:00:27<509:09:33,  8.58s/it]
  5%|▌         | 11421/225000 [34:00:30<424:21:53,  7.15s/it]
  5%|▌         | 11422/225000 [34:00:39<441:05:43,  7.43s/it]
  5%|▌         | 11423/225000 [34:00:45<429:26:19,  7.24s/it]
  5%|▌         | 11424/225000 [34:00:49<373:00:47,  6.29s/it]
  5%|▌         | 11425/225000 [34:00:54<343:19:52,  5.79s/it]
  5%|▌         | 11426/225000 [34:00:59<325:41:47,  5.49s/it]
  5%|▌         | 11427/225000 [34:01:04<326:38:57,  5.51s/it]
  5%|▌         | 11428/225000 [34:01:18<468:19:05,  7.89s/it]
  5%|▌         | 11429/225000 [34:01:21<391:41:49,  6.60s/it]
  5%|▌         | 11430/225000 [34:01:28<390:44:03,  6.59s/it]
  5%|▌         | 11431/225000 [34:01:33<372:16:03,  6.28s/it]
  5%|▌         | 11432/225000 [34:01:38<345:40:10,  5.83s/it]
  5%|▌         | 11433/225000 [34:01:43<329:52:25,  5.56s/it]
  5%|▌         | 11434/225000 [34:01:49<328:31:22,  5.54s/it]
  5%|▌         | 11435/225000 [34:01:54<327:09:21,  5.51s/it]
  5%|▌         | 11436/225000 [34:02:00<331:10:42,  5.58s/it]
  5%|▌         | 11437/225000 [34:02:11<434:37:58,  7.33s/it]
  5%|▌         | 11438/225000 [34:02:19<440:11:29,  7.42s/it]
  5%|▌         | 11439/225000 [34:02:24<401:33:37,  6.77s/it]
  5%|▌         | 11440/225000 [34:02:33<430:50:23,  7.26s/it]
  5%|▌         | 11441/225000 [34:02:37<386:34:03,  6.52s/it]
  5%|▌         | 11442/225000 [34:02:43<362:17:13,  6.11s/it]
  5%|▌         | 11443/225000 [34:02:47<336:11:14,  5.67s/it]
  5%|▌         | 11444/225000 [34:03:01<476:28:21,  8.03s/it]
  5%|▌         | 11445/225000 [34:03:05<405:34:18,  6.84s/it]
  5%|▌         | 11446/225000 [34:03:09<365:30:18,  6.16s/it]
  5%|▌         | 11447/225000 [34:03:21<465:38:50,  7.85s/it]
  5%|▌         | 11448/225000 [34:03:29<472:33:09,  7.97s/it]
  5%|▌         | 11449/225000 [34:03:33<402:48:56,  6.79s/it]
  5%|▌         | 11450/225000 [34:03:39<384:56:54,  6.49s/it]
                                                             
{'loss': '0.7398', 'grad_norm': '0.5613', 'learning_rate': '4.999e-05', 'epoch': '0.05089', 'num_input_tokens_seen': 61027032, 'train_runtime': '1.226e+05', 'train_tokens_per_second': '497.7'}

  5%|▌         | 11450/225000 [34:03:39<384:56:54,  6.49s/it]
  5%|▌         | 11451/225000 [34:03:48<421:11:57,  7.10s/it]
  5%|▌         | 11452/225000 [34:03:53<390:46:00,  6.59s/it]
  5%|▌         | 11453/225000 [34:03:58<368:25:17,  6.21s/it]
  5%|▌         | 11454/225000 [34:04:03<344:14:11,  5.80s/it]
  5%|▌         | 11455/225000 [34:04:08<327:55:45,  5.53s/it]
  5%|▌         | 11456/225000 [34:04:14<325:22:20,  5.49s/it]
  5%|▌         | 11457/225000 [34:04:19<322:45:01,  5.44s/it]
  5%|▌         | 11458/225000 [34:04:31<439:24:30,  7.41s/it]
  5%|▌         | 11459/225000 [34:04:38<436:43:47,  7.36s/it]
  5%|▌         | 11460/225000 [34:04:46<448:04:32,  7.55s/it]
  5%|▌         | 11461/225000 [34:04:51<394:18:02,  6.65s/it]
  5%|▌         | 11462/225000 [34:04:55<351:28:12,  5.93s/it]
  5%|▌         | 11463/225000 [34:05:05<419:45:45,  7.08s/it]
  5%|▌         | 11464/225000 [34:05:13<441:44:51,  7.45s/it]
  5%|▌         | 11465/225000 [34:05:18<405:57:40,  6.84s/it]
  5%|▌         | 11466/225000 [34:05:27<438:29:13,  7.39s/it]
  5%|▌         | 11467/225000 [34:05:31<377:40:35,  6.37s/it]
  5%|▌         | 11468/225000 [34:05:42<459:05:03,  7.74s/it]
  5%|▌         | 11469/225000 [34:05:49<453:03:52,  7.64s/it]
  5%|▌         | 11470/225000 [34:05:55<417:11:54,  7.03s/it]
  5%|▌         | 11471/225000 [34:06:02<423:47:42,  7.14s/it]
  5%|▌         | 11472/225000 [34:06:07<380:42:22,  6.42s/it]
  5%|▌         | 11473/225000 [34:06:12<357:07:03,  6.02s/it]
  5%|▌         | 11474/225000 [34:06:18<347:37:01,  5.86s/it]
  5%|▌         | 11475/225000 [34:06:27<413:38:04,  6.97s/it]
  5%|▌         | 11476/225000 [34:06:32<372:33:18,  6.28s/it]
  5%|▌         | 11477/225000 [34:06:37<345:16:40,  5.82s/it]
  5%|▌         | 11478/225000 [34:06:44<365:24:21,  6.16s/it]
  5%|▌         | 11479/225000 [34:06:54<442:11:32,  7.46s/it]
  5%|▌         | 11480/225000 [34:07:01<435:19:59,  7.34s/it]
  5%|▌         | 11481/225000 [34:07:07<407:06:57,  6.86s/it]
  5%|▌         | 11482/225000 [34:07:12<366:49:26,  6.18s/it]
  5%|▌         | 11483/225000 [34:07:17<351:06:12,  5.92s/it]
  5%|▌         | 11484/225000 [34:07:25<384:18:34,  6.48s/it]
  5%|▌         | 11485/225000 [34:07:39<524:59:31,  8.85s/it]
  5%|▌         | 11486/225000 [34:07:46<487:29:39,  8.22s/it]
  5%|▌         | 11487/225000 [34:07:53<468:42:11,  7.90s/it]
  5%|▌         | 11488/225000 [34:07:58<421:35:16,  7.11s/it]
  5%|▌         | 11489/225000 [34:08:03<380:47:31,  6.42s/it]
  5%|▌         | 11490/225000 [34:08:09<370:49:34,  6.25s/it]
  5%|▌         | 11491/225000 [34:08:20<458:24:43,  7.73s/it]
  5%|▌         | 11492/225000 [34:08:24<390:49:03,  6.59s/it]
  5%|▌         | 11493/225000 [34:08:29<360:57:22,  6.09s/it]
  5%|▌         | 11494/225000 [34:08:34<342:57:53,  5.78s/it]
  5%|▌         | 11495/225000 [34:08:45<439:15:14,  7.41s/it]
  5%|▌         | 11496/225000 [34:08:49<374:06:11,  6.31s/it]
  5%|▌         | 11497/225000 [34:08:54<345:41:38,  5.83s/it]
  5%|▌         | 11498/225000 [34:08:58<327:01:22,  5.51s/it]
  5%|▌         | 11499/225000 [34:09:04<319:41:45,  5.39s/it]
  5%|▌         | 11500/225000 [34:09:17<465:03:59,  7.84s/it]
                                                             
{'loss': '0.7256', 'grad_norm': '0.6559', 'learning_rate': '4.999e-05', 'epoch': '0.05111', 'num_input_tokens_seen': 61188424, 'train_runtime': '1.23e+05', 'train_tokens_per_second': '497.6'}

  5%|▌         | 11500/225000 [34:09:17<465:03:59,  7.84s/it]
  5%|▌         | 11501/225000 [34:09:21<398:13:20,  6.71s/it]
  5%|▌         | 11502/225000 [34:09:31<451:59:50,  7.62s/it]
  5%|▌         | 11503/225000 [34:09:35<383:35:42,  6.47s/it]
  5%|▌         | 11504/225000 [34:09:43<419:38:47,  7.08s/it]
  5%|▌         | 11505/225000 [34:09:53<461:39:03,  7.78s/it]
  5%|▌         | 11506/225000 [34:09:59<433:33:58,  7.31s/it]
  5%|▌         | 11507/225000 [34:10:08<468:39:24,  7.90s/it]
  5%|▌         | 11508/225000 [34:10:12<399:10:24,  6.73s/it]
  5%|▌         | 11509/225000 [34:10:18<385:37:08,  6.50s/it]
  5%|▌         | 11510/225000 [34:10:23<358:00:36,  6.04s/it]
  5%|▌         | 11511/225000 [34:10:28<331:17:57,  5.59s/it]
  5%|▌         | 11512/225000 [34:10:33<320:45:26,  5.41s/it]
  5%|▌         | 11513/225000 [34:10:45<451:39:06,  7.62s/it]
  5%|▌         | 11514/225000 [34:10:51<421:36:47,  7.11s/it]
  5%|▌         | 11515/225000 [34:10:58<409:30:50,  6.91s/it]
  5%|▌         | 11516/225000 [34:11:02<356:00:06,  6.00s/it]
  5%|▌         | 11517/225000 [34:11:07<336:33:36,  5.68s/it]
  5%|▌         | 11518/225000 [34:11:16<401:07:16,  6.76s/it]
  5%|▌         | 11519/225000 [34:11:23<403:11:20,  6.80s/it]
  5%|▌         | 11520/225000 [34:11:27<349:55:09,  5.90s/it]
  5%|▌         | 11521/225000 [34:11:32<334:04:59,  5.63s/it]
  5%|▌         | 11522/225000 [34:11:42<413:25:46,  6.97s/it]
  5%|▌         | 11523/225000 [34:11:53<485:06:00,  8.18s/it]
  5%|▌         | 11524/225000 [34:12:03<526:13:04,  8.87s/it]
  5%|▌         | 11525/225000 [34:12:09<466:24:38,  7.87s/it]
  5%|▌         | 11526/225000 [34:12:14<421:01:55,  7.10s/it]
  5%|▌         | 11527/225000 [34:12:19<389:02:46,  6.56s/it]
  5%|▌         | 11528/225000 [34:12:29<447:42:26,  7.55s/it]
  5%|▌         | 11529/225000 [34:12:36<435:46:03,  7.35s/it]
  5%|▌         | 11530/225000 [34:12:41<394:53:30,  6.66s/it]
  5%|▌         | 11531/225000 [34:12:50<434:27:28,  7.33s/it]
  5%|▌         | 11532/225000 [34:12:54<382:28:50,  6.45s/it]
  5%|▌         | 11533/225000 [34:13:02<400:30:15,  6.75s/it]
  5%|▌         | 11534/225000 [34:13:09<405:26:46,  6.84s/it]
  5%|▌         | 11535/225000 [34:13:14<375:21:59,  6.33s/it]
  5%|▌         | 11536/225000 [34:13:23<415:05:36,  7.00s/it]
  5%|▌         | 11537/225000 [34:13:30<421:47:32,  7.11s/it]
  5%|▌         | 11538/225000 [34:13:36<409:20:35,  6.90s/it]
  5%|▌         | 11539/225000 [34:13:41<373:14:46,  6.29s/it]
  5%|▌         | 11540/225000 [34:13:46<344:28:53,  5.81s/it]
  5%|▌         | 11541/225000 [34:13:51<339:11:29,  5.72s/it]
  5%|▌         | 11542/225000 [34:13:56<327:36:02,  5.53s/it]
  5%|▌         | 11543/225000 [34:14:02<333:25:11,  5.62s/it]
  5%|▌         | 11544/225000 [34:14:08<329:02:46,  5.55s/it]
  5%|▌         | 11545/225000 [34:14:13<324:14:54,  5.47s/it]
  5%|▌         | 11546/225000 [34:14:21<369:14:32,  6.23s/it]
  5%|▌         | 11547/225000 [34:14:33<476:51:15,  8.04s/it]
  5%|▌         | 11548/225000 [34:14:39<431:21:44,  7.28s/it]
  5%|▌         | 11549/225000 [34:14:44<400:24:38,  6.75s/it]
  5%|▌         | 11550/225000 [34:14:52<423:06:53,  7.14s/it]
                                                             
{'loss': '0.7534', 'grad_norm': '0.7823', 'learning_rate': '4.999e-05', 'epoch': '0.05133', 'num_input_tokens_seen': 61350088, 'train_runtime': '1.233e+05', 'train_tokens_per_second': '497.6'}

  5%|▌         | 11550/225000 [34:14:52<423:06:53,  7.14s/it]
  5%|▌         | 11551/225000 [34:14:59<415:38:45,  7.01s/it]
  5%|▌         | 11552/225000 [34:15:05<404:03:00,  6.81s/it]
  5%|▌         | 11553/225000 [34:15:10<360:24:54,  6.08s/it]
  5%|▌         | 11554/225000 [34:15:14<331:42:17,  5.59s/it]
  5%|▌         | 11555/225000 [34:15:25<427:45:29,  7.21s/it]
  5%|▌         | 11556/225000 [34:15:31<394:14:28,  6.65s/it]
  5%|▌         | 11557/225000 [34:15:35<360:42:32,  6.08s/it]
  5%|▌         | 11558/225000 [34:15:47<462:22:52,  7.80s/it]
  5%|▌         | 11559/225000 [34:15:52<408:06:45,  6.88s/it]
  5%|▌         | 11560/225000 [34:16:02<474:24:44,  8.00s/it]
  5%|▌         | 11561/225000 [34:16:08<436:24:29,  7.36s/it]
  5%|▌         | 11562/225000 [34:16:14<400:31:33,  6.76s/it]
  5%|▌         | 11563/225000 [34:16:19<380:49:51,  6.42s/it]
  5%|▌         | 11564/225000 [34:16:29<430:21:01,  7.26s/it]
  5%|▌         | 11565/225000 [34:16:34<391:59:01,  6.61s/it]
  5%|▌         | 11566/225000 [34:16:39<373:32:14,  6.30s/it]
  5%|▌         | 11567/225000 [34:16:45<367:44:42,  6.20s/it]
  5%|▌         | 11568/225000 [34:16:55<429:52:56,  7.25s/it]
  5%|▌         | 11569/225000 [34:17:06<490:55:26,  8.28s/it]
  5%|▌         | 11570/225000 [34:17:10<420:03:06,  7.09s/it]
  5%|▌         | 11571/225000 [34:17:16<395:49:43,  6.68s/it]
  5%|▌         | 11572/225000 [34:17:20<360:28:58,  6.08s/it]
  5%|▌         | 11573/225000 [34:17:26<348:51:19,  5.88s/it]
  5%|▌         | 11574/225000 [34:17:38<467:53:34,  7.89s/it]
  5%|▌         | 11575/225000 [34:17:45<449:06:00,  7.58s/it]
  5%|▌         | 11576/225000 [34:17:50<399:29:20,  6.74s/it]
  5%|▌         | 11577/225000 [34:18:00<459:51:04,  7.76s/it]
  5%|▌         | 11578/225000 [34:18:11<522:05:05,  8.81s/it]
  5%|▌         | 11579/225000 [34:18:16<444:58:19,  7.51s/it]
  5%|▌         | 11580/225000 [34:18:24<455:29:22,  7.68s/it]
  5%|▌         | 11581/225000 [34:18:31<447:24:02,  7.55s/it]
  5%|▌         | 11582/225000 [34:18:36<401:25:19,  6.77s/it]
  5%|▌         | 11583/225000 [34:18:45<436:07:24,  7.36s/it]
  5%|▌         | 11584/225000 [34:18:52<436:57:06,  7.37s/it]
  5%|▌         | 11585/225000 [34:18:57<389:10:24,  6.56s/it]
  5%|▌         | 11586/225000 [34:19:05<411:54:21,  6.95s/it]
  5%|▌         | 11587/225000 [34:19:09<359:16:06,  6.06s/it]
  5%|▌         | 11588/225000 [34:19:19<432:54:07,  7.30s/it]
  5%|▌         | 11589/225000 [34:19:23<382:24:11,  6.45s/it]
  5%|▌         | 11590/225000 [34:19:28<358:53:24,  6.05s/it]
  5%|▌         | 11591/225000 [34:19:37<402:13:07,  6.79s/it]
  5%|▌         | 11592/225000 [34:19:45<418:29:29,  7.06s/it]
  5%|▌         | 11593/225000 [34:19:49<368:56:11,  6.22s/it]
  5%|▌         | 11594/225000 [34:19:53<336:52:33,  5.68s/it]
  5%|▌         | 11595/225000 [34:20:03<401:13:31,  6.77s/it]
  5%|▌         | 11596/225000 [34:20:12<448:21:15,  7.56s/it]
  5%|▌         | 11597/225000 [34:20:16<388:34:38,  6.56s/it]
  5%|▌         | 11598/225000 [34:20:21<349:35:51,  5.90s/it]
  5%|▌         | 11599/225000 [34:20:32<454:26:49,  7.67s/it]
  5%|▌         | 11600/225000 [34:20:39<440:22:30,  7.43s/it]
                                                             
{'loss': '0.7374', 'grad_norm': '0.7542', 'learning_rate': '4.999e-05', 'epoch': '0.05156', 'num_input_tokens_seen': 61516952, 'train_runtime': '1.236e+05', 'train_tokens_per_second': '497.6'}

  5%|▌         | 11600/225000 [34:20:39<440:22:30,  7.43s/it]
  5%|▌         | 11601/225000 [34:20:44<384:30:34,  6.49s/it]
  5%|▌         | 11602/225000 [34:20:48<351:28:24,  5.93s/it]
  5%|▌         | 11603/225000 [34:20:54<341:03:14,  5.75s/it]
  5%|▌         | 11604/225000 [34:21:00<360:24:16,  6.08s/it]
  5%|▌         | 11605/225000 [34:21:07<365:44:07,  6.17s/it]
  5%|▌         | 11606/225000 [34:21:20<493:45:02,  8.33s/it]
  5%|▌         | 11607/225000 [34:21:25<433:04:24,  7.31s/it]
  5%|▌         | 11608/225000 [34:21:33<451:59:59,  7.63s/it]
  5%|▌         | 11609/225000 [34:21:42<477:02:56,  8.05s/it]
  5%|▌         | 11610/225000 [34:21:51<488:43:03,  8.24s/it]
  5%|▌         | 11611/225000 [34:21:56<428:59:09,  7.24s/it]
  5%|▌         | 11612/225000 [34:22:02<402:33:14,  6.79s/it]
  5%|▌         | 11613/225000 [34:22:10<434:37:43,  7.33s/it]
  5%|▌         | 11614/225000 [34:22:16<396:23:12,  6.69s/it]
  5%|▌         | 11615/225000 [34:22:21<375:54:09,  6.34s/it]
  5%|▌         | 11616/225000 [34:22:26<352:42:14,  5.95s/it]
  5%|▌         | 11617/225000 [34:22:31<331:20:43,  5.59s/it]
  5%|▌         | 11618/225000 [34:22:36<329:45:18,  5.56s/it]
  5%|▌         | 11619/225000 [34:22:48<432:27:55,  7.30s/it]
  5%|▌         | 11620/225000 [34:22:54<416:37:26,  7.03s/it]
  5%|▌         | 11621/225000 [34:23:00<399:39:51,  6.74s/it]
  5%|▌         | 11622/225000 [34:23:05<365:07:25,  6.16s/it]
  5%|▌         | 11623/225000 [34:23:11<361:56:44,  6.11s/it]
  5%|▌         | 11624/225000 [34:23:23<460:05:41,  7.76s/it]
  5%|▌         | 11625/225000 [34:23:28<417:51:43,  7.05s/it]
  5%|▌         | 11626/225000 [34:23:34<393:09:47,  6.63s/it]
  5%|▌         | 11627/225000 [34:23:39<368:56:35,  6.22s/it]
  5%|▌         | 11628/225000 [34:23:44<354:29:35,  5.98s/it]
  5%|▌         | 11629/225000 [34:23:50<343:54:03,  5.80s/it]
  5%|▌         | 11630/225000 [34:23:55<327:45:45,  5.53s/it]
  5%|▌         | 11631/225000 [34:24:07<452:55:59,  7.64s/it]
  5%|▌         | 11632/225000 [34:24:14<431:17:03,  7.28s/it]
  5%|▌         | 11633/225000 [34:24:21<424:52:11,  7.17s/it]
  5%|▌         | 11634/225000 [34:24:26<399:36:37,  6.74s/it]
  5%|▌         | 11635/225000 [34:24:37<471:22:19,  7.95s/it]
  5%|▌         | 11636/225000 [34:24:46<479:47:25,  8.10s/it]
  5%|▌         | 11637/225000 [34:24:56<514:12:38,  8.68s/it]
  5%|▌         | 11638/225000 [34:25:00<442:01:48,  7.46s/it]
  5%|▌         | 11639/225000 [34:25:07<436:01:06,  7.36s/it]
  5%|▌         | 11640/225000 [34:25:12<392:03:43,  6.62s/it]
  5%|▌         | 11641/225000 [34:25:17<360:16:54,  6.08s/it]
  5%|▌         | 11642/225000 [34:25:29<470:20:36,  7.94s/it]
  5%|▌         | 11643/225000 [34:25:35<436:21:06,  7.36s/it]
  5%|▌         | 11644/225000 [34:25:42<416:37:49,  7.03s/it]
  5%|▌         | 11645/225000 [34:25:47<390:07:37,  6.58s/it]
  5%|▌         | 11646/225000 [34:25:53<372:36:31,  6.29s/it]
  5%|▌         | 11647/225000 [34:25:58<357:46:56,  6.04s/it]
  5%|▌         | 11648/225000 [34:26:07<401:33:27,  6.78s/it]
  5%|▌         | 11649/225000 [34:26:16<456:37:26,  7.70s/it]
  5%|▌         | 11650/225000 [34:26:21<395:03:26,  6.67s/it]
                                                             
{'loss': '0.7546', 'grad_norm': '0.691', 'learning_rate': '4.999e-05', 'epoch': '0.05178', 'num_input_tokens_seen': 61679040, 'train_runtime': '1.24e+05', 'train_tokens_per_second': '497.5'}

  5%|▌         | 11650/225000 [34:26:21<395:03:26,  6.67s/it]
  5%|▌         | 11651/225000 [34:26:27<390:06:15,  6.58s/it]
  5%|▌         | 11652/225000 [34:26:39<475:44:20,  8.03s/it]
  5%|▌         | 11653/225000 [34:26:44<438:07:47,  7.39s/it]
  5%|▌         | 11654/225000 [34:26:49<385:25:04,  6.50s/it]
  5%|▌         | 11655/225000 [34:26:58<429:34:51,  7.25s/it]
  5%|▌         | 11656/225000 [34:27:02<371:36:43,  6.27s/it]
  5%|▌         | 11657/225000 [34:27:07<350:32:59,  5.92s/it]
  5%|▌         | 11658/225000 [34:27:12<340:58:57,  5.75s/it]
  5%|▌         | 11659/225000 [34:27:18<339:02:52,  5.72s/it]
  5%|▌         | 11660/225000 [34:27:24<338:25:51,  5.71s/it]
  5%|▌         | 11661/225000 [34:27:29<339:28:54,  5.73s/it]
  5%|▌         | 11662/225000 [34:27:35<339:37:23,  5.73s/it]
  5%|▌         | 11663/225000 [34:27:45<417:16:14,  7.04s/it]
  5%|▌         | 11664/225000 [34:27:57<493:40:17,  8.33s/it]
  5%|▌         | 11665/225000 [34:28:02<435:29:44,  7.35s/it]
  5%|▌         | 11666/225000 [34:28:09<426:58:41,  7.21s/it]
  5%|▌         | 11667/225000 [34:28:14<391:54:03,  6.61s/it]
  5%|▌         | 11668/225000 [34:28:20<376:58:52,  6.36s/it]
  5%|▌         | 11669/225000 [34:28:31<463:13:26,  7.82s/it]
  5%|▌         | 11670/225000 [34:28:36<421:37:53,  7.12s/it]
  5%|▌         | 11671/225000 [34:28:42<404:45:53,  6.83s/it]
  5%|▌         | 11672/225000 [34:28:49<403:47:19,  6.81s/it]
  5%|▌         | 11673/225000 [34:28:53<356:45:07,  6.02s/it]
  5%|▌         | 11674/225000 [34:29:05<459:45:50,  7.76s/it]
  5%|▌         | 11675/225000 [34:29:13<456:19:30,  7.70s/it]
  5%|▌         | 11676/225000 [34:29:18<409:49:27,  6.92s/it]
  5%|▌         | 11677/225000 [34:29:23<375:10:41,  6.33s/it]
  5%|▌         | 11678/225000 [34:29:28<359:12:16,  6.06s/it]
  5%|▌         | 11679/225000 [34:29:42<505:08:42,  8.52s/it]
  5%|▌         | 11680/225000 [34:29:48<448:45:31,  7.57s/it]
  5%|▌         | 11681/225000 [34:29:53<412:38:27,  6.96s/it]
  5%|▌         | 11682/225000 [34:29:58<364:26:56,  6.15s/it]
  5%|▌         | 11683/225000 [34:30:06<398:37:46,  6.73s/it]
  5%|▌         | 11684/225000 [34:30:15<448:57:15,  7.58s/it]
  5%|▌         | 11685/225000 [34:30:19<379:51:38,  6.41s/it]
  5%|▌         | 11686/225000 [34:30:27<417:51:56,  7.05s/it]
  5%|▌         | 11687/225000 [34:30:31<360:19:50,  6.08s/it]
  5%|▌         | 11688/225000 [34:30:40<413:27:46,  6.98s/it]
  5%|▌         | 11689/225000 [34:30:47<410:29:59,  6.93s/it]
  5%|▌         | 11690/225000 [34:30:55<431:38:20,  7.28s/it]
  5%|▌         | 11691/225000 [34:31:01<405:07:49,  6.84s/it]
  5%|▌         | 11692/225000 [34:31:14<517:17:02,  8.73s/it]
  5%|▌         | 11693/225000 [34:31:21<484:23:27,  8.18s/it]
  5%|▌         | 11694/225000 [34:31:25<408:29:20,  6.89s/it]
  5%|▌         | 11695/225000 [34:31:34<440:32:16,  7.44s/it]
  5%|▌         | 11696/225000 [34:31:41<437:02:40,  7.38s/it]
  5%|▌         | 11697/225000 [34:31:46<392:17:50,  6.62s/it]
  5%|▌         | 11698/225000 [34:31:50<354:33:04,  5.98s/it]
  5%|▌         | 11699/225000 [34:31:57<366:34:11,  6.19s/it]
  5%|▌         | 11700/225000 [34:32:02<339:43:51,  5.73s/it]
                                                             
{'loss': '0.759', 'grad_norm': '0.5881', 'learning_rate': '4.999e-05', 'epoch': '0.052', 'num_input_tokens_seen': 61844512, 'train_runtime': '1.243e+05', 'train_tokens_per_second': '497.5'}

  5%|▌         | 11700/225000 [34:32:02<339:43:51,  5.73s/it]
  5%|▌         | 11701/225000 [34:32:14<452:26:30,  7.64s/it]
  5%|▌         | 11702/225000 [34:32:19<402:43:45,  6.80s/it]
  5%|▌         | 11703/225000 [34:32:26<410:50:41,  6.93s/it]
  5%|▌         | 11704/225000 [34:32:33<415:48:04,  7.02s/it]
  5%|▌         | 11705/225000 [34:32:43<466:57:27,  7.88s/it]
  5%|▌         | 11706/225000 [34:32:49<440:46:29,  7.44s/it]
  5%|▌         | 11707/225000 [34:32:54<391:44:57,  6.61s/it]
  5%|▌         | 11708/225000 [34:33:05<473:30:03,  7.99s/it]
  5%|▌         | 11709/225000 [34:33:09<401:10:11,  6.77s/it]
  5%|▌         | 11710/225000 [34:33:14<358:17:36,  6.05s/it]
  5%|▌         | 11711/225000 [34:33:25<455:16:18,  7.68s/it]
  5%|▌         | 11712/225000 [34:33:30<405:35:31,  6.85s/it]
  5%|▌         | 11713/225000 [34:33:35<382:09:11,  6.45s/it]
  5%|▌         | 11714/225000 [34:33:43<397:53:58,  6.72s/it]
  5%|▌         | 11715/225000 [34:33:54<481:48:24,  8.13s/it]
  5%|▌         | 11716/225000 [34:34:00<438:00:27,  7.39s/it]
  5%|▌         | 11717/225000 [34:34:08<455:03:34,  7.68s/it]
  5%|▌         | 11718/225000 [34:34:15<442:49:58,  7.47s/it]
  5%|▌         | 11719/225000 [34:34:23<448:57:56,  7.58s/it]
  5%|▌         | 11720/225000 [34:34:27<387:26:04,  6.54s/it]
  5%|▌         | 11721/225000 [34:34:35<406:40:38,  6.86s/it]
  5%|▌         | 11722/225000 [34:34:40<379:00:15,  6.40s/it]
  5%|▌         | 11723/225000 [34:34:46<363:19:14,  6.13s/it]
  5%|▌         | 11724/225000 [34:34:51<356:07:15,  6.01s/it]
  5%|▌         | 11725/225000 [34:34:57<346:24:46,  5.85s/it]
  5%|▌         | 11726/225000 [34:35:06<413:24:49,  6.98s/it]
  5%|▌         | 11727/225000 [34:35:10<353:50:53,  5.97s/it]
  5%|▌         | 11728/225000 [34:35:19<413:31:33,  6.98s/it]
  5%|▌         | 11729/225000 [34:35:24<373:14:17,  6.30s/it]
  5%|▌         | 11730/225000 [34:35:32<410:41:26,  6.93s/it]
  5%|▌         | 11731/225000 [34:35:39<396:56:57,  6.70s/it]
  5%|▌         | 11732/225000 [34:35:47<425:04:26,  7.18s/it]
  5%|▌         | 11733/225000 [34:35:53<414:13:48,  6.99s/it]
  5%|▌         | 11734/225000 [34:35:59<385:36:15,  6.51s/it]
  5%|▌         | 11735/225000 [34:36:04<368:54:55,  6.23s/it]
  5%|▌         | 11736/225000 [34:36:14<428:26:21,  7.23s/it]
  5%|▌         | 11737/225000 [34:36:18<371:34:58,  6.27s/it]
  5%|▌         | 11738/225000 [34:36:27<421:05:40,  7.11s/it]
  5%|▌         | 11739/225000 [34:36:32<386:39:12,  6.53s/it]
  5%|▌         | 11740/225000 [34:36:36<345:37:47,  5.83s/it]
  5%|▌         | 11741/225000 [34:36:46<416:08:23,  7.02s/it]
  5%|▌         | 11742/225000 [34:36:53<413:03:30,  6.97s/it]
  5%|▌         | 11743/225000 [34:37:01<424:24:34,  7.16s/it]
  5%|▌         | 11744/225000 [34:37:11<476:57:46,  8.05s/it]
  5%|▌         | 11745/225000 [34:37:15<415:31:38,  7.01s/it]
  5%|▌         | 11746/225000 [34:37:29<527:27:35,  8.90s/it]
  5%|▌         | 11747/225000 [34:37:39<544:30:22,  9.19s/it]
  5%|▌         | 11748/225000 [34:37:44<480:33:13,  8.11s/it]
  5%|▌         | 11749/225000 [34:37:48<409:50:27,  6.92s/it]
  5%|▌         | 11750/225000 [34:37:58<458:40:29,  7.74s/it]
                                                             
{'loss': '0.7601', 'grad_norm': '0.5776', 'learning_rate': '4.999e-05', 'epoch': '0.05222', 'num_input_tokens_seen': 62008592, 'train_runtime': '1.247e+05', 'train_tokens_per_second': '497.3'}

  5%|▌         | 11750/225000 [34:37:58<458:40:29,  7.74s/it]
  5%|▌         | 11751/225000 [34:38:06<458:54:59,  7.75s/it]
  5%|▌         | 11752/225000 [34:38:11<417:43:26,  7.05s/it]
  5%|▌         | 11753/225000 [34:38:16<378:12:16,  6.38s/it]
  5%|▌         | 11754/225000 [34:38:21<348:44:15,  5.89s/it]
  5%|▌         | 11755/225000 [34:38:34<472:29:01,  7.98s/it]
  5%|▌         | 11756/225000 [34:38:40<445:45:39,  7.53s/it]
  5%|▌         | 11757/225000 [34:38:46<408:22:05,  6.89s/it]
  5%|▌         | 11758/225000 [34:38:49<350:19:23,  5.91s/it]
  5%|▌         | 11759/225000 [34:38:58<396:37:09,  6.70s/it]
  5%|▌         | 11760/225000 [34:39:05<404:51:26,  6.83s/it]
  5%|▌         | 11761/225000 [34:39:14<445:17:23,  7.52s/it]
  5%|▌         | 11762/225000 [34:39:21<437:11:58,  7.38s/it]
  5%|▌         | 11763/225000 [34:39:27<411:34:18,  6.95s/it]
  5%|▌         | 11764/225000 [34:39:32<379:38:29,  6.41s/it]
  5%|▌         | 11765/225000 [34:39:44<474:25:25,  8.01s/it]
  5%|▌         | 11766/225000 [34:39:50<442:01:09,  7.46s/it]
  5%|▌         | 11767/225000 [34:39:55<401:06:13,  6.77s/it]
  5%|▌         | 11768/225000 [34:40:00<370:22:27,  6.25s/it]
  5%|▌         | 11769/225000 [34:40:06<357:20:51,  6.03s/it]
  5%|▌         | 11770/225000 [34:40:11<337:55:42,  5.71s/it]
  5%|▌         | 11771/225000 [34:40:25<493:28:26,  8.33s/it]
  5%|▌         | 11772/225000 [34:40:32<461:31:31,  7.79s/it]
  5%|▌         | 11773/225000 [34:40:41<483:25:01,  8.16s/it]
  5%|▌         | 11774/225000 [34:40:45<421:59:37,  7.12s/it]
  5%|▌         | 11775/225000 [34:40:54<445:42:21,  7.53s/it]
  5%|▌         | 11776/225000 [34:40:59<395:08:58,  6.67s/it]
  5%|▌         | 11777/225000 [34:41:08<441:47:25,  7.46s/it]
  5%|▌         | 11778/225000 [34:41:13<409:08:05,  6.91s/it]
  5%|▌         | 11779/225000 [34:41:21<419:00:12,  7.07s/it]
  5%|▌         | 11780/225000 [34:41:29<437:56:23,  7.39s/it]
  5%|▌         | 11781/225000 [34:41:36<422:32:29,  7.13s/it]
  5%|▌         | 11782/225000 [34:41:43<435:47:48,  7.36s/it]
  5%|▌         | 11783/225000 [34:41:48<382:56:08,  6.47s/it]
  5%|▌         | 11784/225000 [34:41:53<356:25:01,  6.02s/it]
  5%|▌         | 11785/225000 [34:42:02<413:29:02,  6.98s/it]
  5%|▌         | 11786/225000 [34:42:07<378:44:44,  6.39s/it]
  5%|▌         | 11787/225000 [34:42:13<364:12:22,  6.15s/it]
  5%|▌         | 11788/225000 [34:42:18<355:21:47,  6.00s/it]
  5%|▌         | 11789/225000 [34:42:24<341:18:39,  5.76s/it]
  5%|▌         | 11790/225000 [34:42:29<336:50:02,  5.69s/it]
  5%|▌         | 11791/225000 [34:42:34<325:59:52,  5.50s/it]
  5%|▌         | 11792/225000 [34:42:49<490:39:16,  8.28s/it]
  5%|▌         | 11793/225000 [34:42:54<427:05:09,  7.21s/it]
  5%|▌         | 11794/225000 [34:42:59<387:24:47,  6.54s/it]
  5%|▌         | 11795/225000 [34:43:03<345:16:30,  5.83s/it]
  5%|▌         | 11796/225000 [34:43:14<450:18:25,  7.60s/it]
  5%|▌         | 11797/225000 [34:43:20<413:26:25,  6.98s/it]
  5%|▌         | 11798/225000 [34:43:26<399:06:21,  6.74s/it]
  5%|▌         | 11799/225000 [34:43:31<369:28:48,  6.24s/it]
  5%|▌         | 11800/225000 [34:43:37<352:54:04,  5.96s/it]
                                                             
{'loss': '0.7173', 'grad_norm': '0.6821', 'learning_rate': '4.999e-05', 'epoch': '0.05244', 'num_input_tokens_seen': 62174504, 'train_runtime': '1.25e+05', 'train_tokens_per_second': '497.3'}

  5%|▌         | 11800/225000 [34:43:37<352:54:04,  5.96s/it]
  5%|▌         | 11801/225000 [34:43:41<326:42:24,  5.52s/it]
  5%|▌         | 11802/225000 [34:43:48<356:00:48,  6.01s/it]
  5%|▌         | 11803/225000 [34:44:00<463:50:35,  7.83s/it]
  5%|▌         | 11804/225000 [34:44:05<401:14:20,  6.78s/it]
  5%|▌         | 11805/225000 [34:44:10<375:16:09,  6.34s/it]
  5%|▌         | 11806/225000 [34:44:17<387:37:00,  6.55s/it]
  5%|▌         | 11807/225000 [34:44:28<476:10:16,  8.04s/it]
  5%|▌         | 11808/225000 [34:44:39<527:21:13,  8.90s/it]
  5%|▌         | 11809/225000 [34:44:45<461:39:32,  7.80s/it]
  5%|▌         | 11810/225000 [34:44:51<435:10:53,  7.35s/it]
  5%|▌         | 11811/225000 [34:45:00<457:01:14,  7.72s/it]
  5%|▌         | 11812/225000 [34:45:04<392:50:11,  6.63s/it]
  5%|▌         | 11813/225000 [34:45:13<446:18:09,  7.54s/it]
  5%|▌         | 11814/225000 [34:45:17<383:32:31,  6.48s/it]
  5%|▌         | 11815/225000 [34:45:25<402:29:19,  6.80s/it]
  5%|▌         | 11816/225000 [34:45:31<395:28:06,  6.68s/it]
  5%|▌         | 11817/225000 [34:45:41<458:15:22,  7.74s/it]
  5%|▌         | 11818/225000 [34:45:46<396:11:31,  6.69s/it]
  5%|▌         | 11819/225000 [34:45:50<361:59:02,  6.11s/it]
  5%|▌         | 11820/225000 [34:45:57<366:34:18,  6.19s/it]
  5%|▌         | 11821/225000 [34:46:03<365:46:36,  6.18s/it]
  5%|▌         | 11822/225000 [34:46:11<396:56:41,  6.70s/it]
  5%|▌         | 11823/225000 [34:46:23<491:47:10,  8.30s/it]
  5%|▌         | 11824/225000 [34:46:26<403:58:20,  6.82s/it]
  5%|▌         | 11825/225000 [34:46:32<380:49:14,  6.43s/it]
  5%|▌         | 11826/225000 [34:46:37<353:35:24,  5.97s/it]
  5%|▌         | 11827/225000 [34:46:42<341:52:08,  5.77s/it]
  5%|▌         | 11828/225000 [34:46:48<348:25:59,  5.88s/it]
  5%|▌         | 11829/225000 [34:47:00<458:38:16,  7.75s/it]
  5%|▌         | 11830/225000 [34:47:05<405:09:44,  6.84s/it]
  5%|▌         | 11831/225000 [34:47:10<367:21:02,  6.20s/it]
  5%|▌         | 11832/225000 [34:47:16<375:53:48,  6.35s/it]
  5%|▌         | 11833/225000 [34:47:26<433:10:11,  7.32s/it]
  5%|▌         | 11834/225000 [34:47:31<395:50:33,  6.69s/it]
  5%|▌         | 11835/225000 [34:47:37<376:16:02,  6.35s/it]
  5%|▌         | 11836/225000 [34:47:43<373:43:07,  6.31s/it]
  5%|▌         | 11837/225000 [34:47:49<363:26:31,  6.14s/it]
  5%|▌         | 11838/225000 [34:47:55<358:03:56,  6.05s/it]
  5%|▌         | 11839/225000 [34:48:00<351:17:47,  5.93s/it]
  5%|▌         | 11840/225000 [34:48:08<389:46:24,  6.58s/it]
  5%|▌         | 11841/225000 [34:48:21<507:21:41,  8.57s/it]
  5%|▌         | 11842/225000 [34:48:26<443:41:10,  7.49s/it]
  5%|▌         | 11843/225000 [34:48:31<382:15:35,  6.46s/it]
  5%|▌         | 11844/225000 [34:48:41<449:14:10,  7.59s/it]
  5%|▌         | 11845/225000 [34:48:46<410:46:55,  6.94s/it]
  5%|▌         | 11846/225000 [34:48:54<421:07:52,  7.11s/it]
  5%|▌         | 11847/225000 [34:49:00<400:07:30,  6.76s/it]
  5%|▌         | 11848/225000 [34:49:08<431:01:22,  7.28s/it]
  5%|▌         | 11849/225000 [34:49:13<381:02:14,  6.44s/it]
  5%|▌         | 11850/225000 [34:49:17<344:34:15,  5.82s/it]
                                                             
{'loss': '0.7566', 'grad_norm': '0.6094', 'learning_rate': '4.999e-05', 'epoch': '0.05267', 'num_input_tokens_seen': 62337080, 'train_runtime': '1.254e+05', 'train_tokens_per_second': '497.3'}

  5%|▌         | 11850/225000 [34:49:17<344:34:15,  5.82s/it]
  5%|▌         | 11851/225000 [34:49:22<332:31:47,  5.62s/it]
  5%|▌         | 11852/225000 [34:49:34<442:46:34,  7.48s/it]
  5%|▌         | 11853/225000 [34:49:40<422:16:56,  7.13s/it]
  5%|▌         | 11854/225000 [34:49:45<378:27:51,  6.39s/it]
  5%|▌         | 11855/225000 [34:49:50<360:55:10,  6.10s/it]
  5%|▌         | 11856/225000 [34:50:01<437:06:27,  7.38s/it]
  5%|▌         | 11857/225000 [34:50:05<383:35:48,  6.48s/it]
  5%|▌         | 11858/225000 [34:50:15<445:37:23,  7.53s/it]
  5%|▌         | 11859/225000 [34:50:20<398:16:59,  6.73s/it]
  5%|▌         | 11860/225000 [34:50:25<362:40:32,  6.13s/it]
  5%|▌         | 11861/225000 [34:50:35<432:50:07,  7.31s/it]
  5%|▌         | 11862/225000 [34:50:42<436:18:44,  7.37s/it]
  5%|▌         | 11863/225000 [34:50:48<404:33:50,  6.83s/it]
  5%|▌         | 11864/225000 [34:50:56<420:53:01,  7.11s/it]
  5%|▌         | 11865/225000 [34:51:02<401:26:46,  6.78s/it]
  5%|▌         | 11866/225000 [34:51:06<357:57:30,  6.05s/it]
  5%|▌         | 11867/225000 [34:51:17<451:04:29,  7.62s/it]
  5%|▌         | 11868/225000 [34:51:21<390:27:29,  6.60s/it]
  5%|▌         | 11869/225000 [34:51:28<388:32:18,  6.56s/it]
  5%|▌         | 11870/225000 [34:51:38<459:02:06,  7.75s/it]
  5%|▌         | 11871/225000 [34:51:45<439:51:50,  7.43s/it]
  5%|▌         | 11872/225000 [34:51:49<371:47:45,  6.28s/it]
  5%|▌         | 11873/225000 [34:51:54<352:25:12,  5.95s/it]
  5%|▌         | 11874/225000 [34:52:00<347:10:28,  5.86s/it]
  5%|▌         | 11875/225000 [34:52:05<346:27:08,  5.85s/it]
  5%|▌         | 11876/225000 [34:52:11<346:04:55,  5.85s/it]
  5%|▌         | 11877/225000 [34:52:25<488:04:05,  8.24s/it]
  5%|▌         | 11878/225000 [34:52:30<428:18:29,  7.23s/it]
  5%|▌         | 11879/225000 [34:52:34<372:37:42,  6.29s/it]
  5%|▌         | 11880/225000 [34:52:45<448:04:29,  7.57s/it]
  5%|▌         | 11881/225000 [34:52:54<488:29:51,  8.25s/it]
  5%|▌         | 11882/225000 [34:52:58<407:36:17,  6.89s/it]
  5%|▌         | 11883/225000 [34:53:03<377:37:30,  6.38s/it]
  5%|▌         | 11884/225000 [34:53:09<368:07:30,  6.22s/it]
  5%|▌         | 11885/225000 [34:53:17<389:01:35,  6.57s/it]
  5%|▌         | 11886/225000 [34:53:25<418:38:11,  7.07s/it]
  5%|▌         | 11887/225000 [34:53:29<373:09:00,  6.30s/it]
  5%|▌         | 11888/225000 [34:53:36<372:28:58,  6.29s/it]
  5%|▌         | 11889/225000 [34:53:41<364:37:52,  6.16s/it]
  5%|▌         | 11890/225000 [34:53:53<463:47:59,  7.83s/it]
  5%|▌         | 11891/225000 [34:54:01<460:12:55,  7.77s/it]
  5%|▌         | 11892/225000 [34:54:05<400:48:43,  6.77s/it]
  5%|▌         | 11893/225000 [34:54:09<345:25:03,  5.84s/it]
  5%|▌         | 11894/225000 [34:54:19<425:35:32,  7.19s/it]
  5%|▌         | 11895/225000 [34:54:24<381:46:56,  6.45s/it]
  5%|▌         | 11896/225000 [34:54:33<429:32:07,  7.26s/it]
  5%|▌         | 11897/225000 [34:54:43<481:36:43,  8.14s/it]
  5%|▌         | 11898/225000 [34:54:50<451:42:00,  7.63s/it]
  5%|▌         | 11899/225000 [34:54:54<386:11:20,  6.52s/it]
  5%|▌         | 11900/225000 [34:55:04<447:48:36,  7.57s/it]
                                                             
{'loss': '0.7501', 'grad_norm': '0.6346', 'learning_rate': '4.999e-05', 'epoch': '0.05289', 'num_input_tokens_seen': 62499568, 'train_runtime': '1.257e+05', 'train_tokens_per_second': '497.2'}

  5%|▌         | 11900/225000 [34:55:04<447:48:36,  7.57s/it]
  5%|▌         | 11901/225000 [34:55:09<410:25:16,  6.93s/it]
  5%|▌         | 11902/225000 [34:55:14<367:39:41,  6.21s/it]
  5%|▌         | 11903/225000 [34:55:18<337:38:15,  5.70s/it]
  5%|▌         | 11904/225000 [34:55:26<375:16:15,  6.34s/it]
  5%|▌         | 11905/225000 [34:55:36<437:43:11,  7.39s/it]
  5%|▌         | 11906/225000 [34:55:44<458:01:42,  7.74s/it]
  5%|▌         | 11907/225000 [34:55:49<406:29:14,  6.87s/it]
  5%|▌         | 11908/225000 [34:55:54<374:32:41,  6.33s/it]
  5%|▌         | 11909/225000 [34:55:59<354:59:11,  6.00s/it]
  5%|▌         | 11910/225000 [34:56:09<411:55:39,  6.96s/it]
  5%|▌         | 11911/225000 [34:56:17<432:11:35,  7.30s/it]
  5%|▌         | 11912/225000 [34:56:26<466:49:55,  7.89s/it]
  5%|▌         | 11913/225000 [34:56:30<404:54:02,  6.84s/it]
  5%|▌         | 11914/225000 [34:56:35<369:55:01,  6.25s/it]
  5%|▌         | 11915/225000 [34:56:43<403:50:09,  6.82s/it]
  5%|▌         | 11916/225000 [34:56:51<409:46:14,  6.92s/it]
  5%|▌         | 11917/225000 [34:56:57<391:47:23,  6.62s/it]
  5%|▌         | 11918/225000 [34:57:01<360:45:43,  6.10s/it]
  5%|▌         | 11919/225000 [34:57:08<376:13:50,  6.36s/it]
  5%|▌         | 11920/225000 [34:57:15<381:02:25,  6.44s/it]
  5%|▌         | 11921/225000 [34:57:24<428:17:33,  7.24s/it]
  5%|▌         | 11922/225000 [34:57:32<441:20:08,  7.46s/it]
  5%|▌         | 11923/225000 [34:57:39<423:53:39,  7.16s/it]
  5%|▌         | 11924/225000 [34:57:44<394:34:57,  6.67s/it]
  5%|▌         | 11925/225000 [34:57:50<375:19:01,  6.34s/it]
  5%|▌         | 11926/225000 [34:58:02<485:14:30,  8.20s/it]
  5%|▌         | 11927/225000 [34:58:07<426:50:30,  7.21s/it]
  5%|▌         | 11928/225000 [34:58:12<389:26:26,  6.58s/it]
  5%|▌         | 11929/225000 [34:58:18<370:34:19,  6.26s/it]
  5%|▌         | 11930/225000 [34:58:23<352:18:03,  5.95s/it]
  5%|▌         | 11931/225000 [34:58:29<348:39:59,  5.89s/it]
  5%|▌         | 11932/225000 [34:58:35<347:31:39,  5.87s/it]
  5%|▌         | 11933/225000 [34:58:45<434:12:08,  7.34s/it]
  5%|▌         | 11934/225000 [34:59:00<556:46:47,  9.41s/it]
  5%|▌         | 11935/225000 [34:59:03<452:20:45,  7.64s/it]
  5%|▌         | 11936/225000 [34:59:12<471:33:50,  7.97s/it]
  5%|▌         | 11937/225000 [34:59:16<399:28:43,  6.75s/it]
  5%|▌         | 11938/225000 [34:59:21<368:14:11,  6.22s/it]
  5%|▌         | 11939/225000 [34:59:30<422:55:41,  7.15s/it]
  5%|▌         | 11940/225000 [34:59:38<432:23:59,  7.31s/it]
  5%|▌         | 11941/225000 [34:59:44<408:41:16,  6.91s/it]
  5%|▌         | 11942/225000 [34:59:52<437:50:46,  7.40s/it]
  5%|▌         | 11943/225000 [34:59:57<389:51:12,  6.59s/it]
  5%|▌         | 11944/225000 [35:00:02<364:27:31,  6.16s/it]
  5%|▌         | 11945/225000 [35:00:07<349:30:33,  5.91s/it]
  5%|▌         | 11946/225000 [35:00:24<542:54:19,  9.17s/it]
  5%|▌         | 11947/225000 [35:00:28<454:56:24,  7.69s/it]
  5%|▌         | 11948/225000 [35:00:37<479:32:46,  8.10s/it]
  5%|▌         | 11949/225000 [35:00:42<415:45:54,  7.03s/it]
  5%|▌         | 11950/225000 [35:00:49<419:04:41,  7.08s/it]
                                                             
{'loss': '0.7415', 'grad_norm': '0.7275', 'learning_rate': '4.999e-05', 'epoch': '0.05311', 'num_input_tokens_seen': 62666960, 'train_runtime': '1.26e+05', 'train_tokens_per_second': '497.2'}

  5%|▌         | 11950/225000 [35:00:49<419:04:41,  7.08s/it]
  5%|▌         | 11951/225000 [35:01:00<479:30:19,  8.10s/it]
  5%|▌         | 11952/225000 [35:01:05<427:21:30,  7.22s/it]
  5%|▌         | 11953/225000 [35:01:09<380:14:02,  6.43s/it]
  5%|▌         | 11954/225000 [35:01:14<354:40:57,  5.99s/it]
  5%|▌         | 11955/225000 [35:01:31<551:32:17,  9.32s/it]
  5%|▌         | 11956/225000 [35:01:38<496:32:22,  8.39s/it]
  5%|▌         | 11957/225000 [35:01:43<448:20:04,  7.58s/it]
  5%|▌         | 11958/225000 [35:01:48<397:28:05,  6.72s/it]
  5%|▌         | 11959/225000 [35:01:52<356:24:39,  6.02s/it]
  5%|▌         | 11960/225000 [35:02:01<401:41:44,  6.79s/it]
  5%|▌         | 11961/225000 [35:02:09<415:33:52,  7.02s/it]
  5%|▌         | 11962/225000 [35:02:14<390:29:53,  6.60s/it]
  5%|▌         | 11963/225000 [35:02:20<371:57:09,  6.29s/it]
  5%|▌         | 11964/225000 [35:02:30<443:43:10,  7.50s/it]
  5%|▌         | 11965/225000 [35:02:37<427:11:43,  7.22s/it]
  5%|▌         | 11966/225000 [35:02:44<435:06:24,  7.35s/it]
  5%|▌         | 11967/225000 [35:02:50<400:30:18,  6.77s/it]
  5%|▌         | 11968/225000 [35:02:55<377:35:44,  6.38s/it]
  5%|▌         | 11969/225000 [35:03:00<347:45:25,  5.88s/it]
  5%|▌         | 11970/225000 [35:03:05<340:38:58,  5.76s/it]
  5%|▌         | 11971/225000 [35:03:10<329:29:42,  5.57s/it]
  5%|▌         | 11972/225000 [35:03:16<333:23:07,  5.63s/it]
  5%|▌         | 11973/225000 [35:03:24<370:09:03,  6.26s/it]
  5%|▌         | 11974/225000 [35:03:35<462:52:23,  7.82s/it]
  5%|▌         | 11975/225000 [35:03:43<466:17:36,  7.88s/it]
  5%|▌         | 11976/225000 [35:03:50<449:52:01,  7.60s/it]
  5%|▌         | 11977/225000 [35:03:56<422:22:34,  7.14s/it]
  5%|▌         | 11978/225000 [35:04:02<392:06:40,  6.63s/it]
  5%|▌         | 11979/225000 [35:04:07<372:12:37,  6.29s/it]
  5%|▌         | 11980/225000 [35:04:12<350:24:19,  5.92s/it]
  5%|▌         | 11981/225000 [35:04:18<344:20:17,  5.82s/it]
  5%|▌         | 11982/225000 [35:04:24<345:51:26,  5.84s/it]
  5%|▌         | 11983/225000 [35:04:39<507:40:43,  8.58s/it]
  5%|▌         | 11984/225000 [35:04:44<441:36:26,  7.46s/it]
  5%|▌         | 11985/225000 [35:04:49<396:46:04,  6.71s/it]
  5%|▌         | 11986/225000 [35:04:53<359:43:00,  6.08s/it]
  5%|▌         | 11987/225000 [35:04:58<334:01:21,  5.65s/it]
  5%|▌         | 11988/225000 [35:05:10<438:57:59,  7.42s/it]
  5%|▌         | 11989/225000 [35:05:17<432:58:12,  7.32s/it]
  5%|▌         | 11990/225000 [35:05:22<392:53:43,  6.64s/it]
  5%|▌         | 11991/225000 [35:05:27<367:36:28,  6.21s/it]
  5%|▌         | 11992/225000 [35:05:32<351:58:07,  5.95s/it]
  5%|▌         | 11993/225000 [35:05:38<343:00:50,  5.80s/it]
  5%|▌         | 11994/225000 [35:05:51<484:22:56,  8.19s/it]
  5%|▌         | 11995/225000 [35:05:58<462:52:16,  7.82s/it]
  5%|▌         | 11996/225000 [35:06:04<425:47:16,  7.20s/it]
  5%|▌         | 11997/225000 [35:06:13<450:33:03,  7.61s/it]
  5%|▌         | 11998/225000 [35:06:17<395:25:23,  6.68s/it]
  5%|▌         | 11999/225000 [35:06:22<368:47:56,  6.23s/it]
  5%|▌         | 12000/225000 [35:06:28<355:40:13,  6.01s/it]
                                                             
{'loss': '0.7544', 'grad_norm': '0.6675', 'learning_rate': '4.999e-05', 'epoch': '0.05333', 'num_input_tokens_seen': 62826248, 'train_runtime': '1.264e+05', 'train_tokens_per_second': '497.1'}

  5%|▌         | 12000/225000 [35:06:28<355:40:13,  6.01s/it]
  5%|▌         | 12001/225000 [35:06:37<404:47:55,  6.84s/it]
  5%|▌         | 12002/225000 [35:06:44<409:26:57,  6.92s/it]
  5%|▌         | 12003/225000 [35:06:50<401:46:15,  6.79s/it]
  5%|▌         | 12004/225000 [35:06:55<373:22:24,  6.31s/it]
  5%|▌         | 12005/225000 [35:07:03<401:45:54,  6.79s/it]
  5%|▌         | 12006/225000 [35:07:11<411:25:53,  6.95s/it]
  5%|▌         | 12007/225000 [35:07:15<365:36:32,  6.18s/it]
  5%|▌         | 12008/225000 [35:07:20<339:38:54,  5.74s/it]
  5%|▌         | 12009/225000 [35:07:33<466:28:33,  7.88s/it]
  5%|▌         | 12010/225000 [35:07:37<403:37:30,  6.82s/it]
  5%|▌         | 12011/225000 [35:07:44<411:25:47,  6.95s/it]
  5%|▌         | 12012/225000 [35:07:53<449:35:45,  7.60s/it]
  5%|▌         | 12013/225000 [35:08:01<455:29:30,  7.70s/it]
  5%|▌         | 12014/225000 [35:08:05<388:11:49,  6.56s/it]
  5%|▌         | 12015/225000 [35:08:14<423:12:28,  7.15s/it]
  5%|▌         | 12016/225000 [35:08:19<382:22:20,  6.46s/it]
  5%|▌         | 12017/225000 [35:08:24<365:33:53,  6.18s/it]
  5%|▌         | 12018/225000 [35:08:31<381:34:11,  6.45s/it]
  5%|▌         | 12019/225000 [35:08:44<497:24:52,  8.41s/it]
  5%|▌         | 12020/225000 [35:08:51<474:42:59,  8.02s/it]
  5%|▌         | 12021/225000 [35:08:57<427:48:50,  7.23s/it]
  5%|▌         | 12022/225000 [35:09:05<439:16:47,  7.43s/it]
  5%|▌         | 12023/225000 [35:09:09<383:10:30,  6.48s/it]
  5%|▌         | 12024/225000 [35:09:19<440:18:57,  7.44s/it]
  5%|▌         | 12025/225000 [35:09:23<378:55:23,  6.41s/it]
  5%|▌         | 12026/225000 [35:09:31<409:34:33,  6.92s/it]
  5%|▌         | 12027/225000 [35:09:35<366:53:45,  6.20s/it]
  5%|▌         | 12028/225000 [35:09:44<407:16:04,  6.88s/it]
  5%|▌         | 12029/225000 [35:09:49<382:27:34,  6.46s/it]
  5%|▌         | 12030/225000 [35:09:59<445:11:39,  7.53s/it]
  5%|▌         | 12031/225000 [35:10:08<469:25:08,  7.93s/it]
  5%|▌         | 12032/225000 [35:10:13<408:10:05,  6.90s/it]
  5%|▌         | 12033/225000 [35:10:20<416:50:15,  7.05s/it]
  5%|▌         | 12034/225000 [35:10:25<374:30:18,  6.33s/it]
  5%|▌         | 12035/225000 [35:10:36<469:16:11,  7.93s/it]
  5%|▌         | 12036/225000 [35:10:40<390:06:14,  6.59s/it]
  5%|▌         | 12037/225000 [35:10:51<472:28:04,  7.99s/it]
  5%|▌         | 12038/225000 [35:10:55<400:50:12,  6.78s/it]
  5%|▌         | 12039/225000 [35:10:59<351:58:19,  5.95s/it]
  5%|▌         | 12040/225000 [35:11:10<450:49:21,  7.62s/it]
  5%|▌         | 12041/225000 [35:11:14<381:39:15,  6.45s/it]
  5%|▌         | 12042/225000 [35:11:20<362:19:38,  6.13s/it]
  5%|▌         | 12043/225000 [35:11:29<427:45:41,  7.23s/it]
  5%|▌         | 12044/225000 [35:11:36<413:01:46,  6.98s/it]
  5%|▌         | 12045/225000 [35:11:40<370:16:09,  6.26s/it]
  5%|▌         | 12046/225000 [35:11:51<455:28:55,  7.70s/it]
  5%|▌         | 12047/225000 [35:12:01<481:51:38,  8.15s/it]
  5%|▌         | 12048/225000 [35:12:06<426:11:18,  7.20s/it]
  5%|▌         | 12049/225000 [35:12:10<373:45:39,  6.32s/it]
  5%|▌         | 12050/225000 [35:12:15<351:06:13,  5.94s/it]
                                                             
{'loss': '0.7255', 'grad_norm': '0.5561', 'learning_rate': '4.999e-05', 'epoch': '0.05356', 'num_input_tokens_seen': 62989432, 'train_runtime': '1.267e+05', 'train_tokens_per_second': '497'}

  5%|▌         | 12050/225000 [35:12:15<351:06:13,  5.94s/it]
  5%|▌         | 12051/225000 [35:12:21<357:20:22,  6.04s/it]
  5%|▌         | 12052/225000 [35:12:32<436:07:09,  7.37s/it]
  5%|▌         | 12053/225000 [35:12:36<383:31:20,  6.48s/it]
  5%|▌         | 12054/225000 [35:12:43<399:44:09,  6.76s/it]
  5%|▌         | 12055/225000 [35:12:51<406:25:48,  6.87s/it]
  5%|▌         | 12056/225000 [35:13:01<472:06:36,  7.98s/it]
  5%|▌         | 12057/225000 [35:13:06<410:42:46,  6.94s/it]
  5%|▌         | 12058/225000 [35:13:11<379:44:11,  6.42s/it]
  5%|▌         | 12059/225000 [35:13:23<474:04:43,  8.01s/it]
  5%|▌         | 12060/225000 [35:13:31<482:32:49,  8.16s/it]
  5%|▌         | 12061/225000 [35:13:36<423:52:06,  7.17s/it]
  5%|▌         | 12062/225000 [35:13:43<426:51:26,  7.22s/it]
  5%|▌         | 12063/225000 [35:13:48<389:17:22,  6.58s/it]
  5%|▌         | 12064/225000 [35:13:52<341:12:46,  5.77s/it]
  5%|▌         | 12065/225000 [35:13:59<357:17:21,  6.04s/it]
  5%|▌         | 12066/225000 [35:14:12<476:39:48,  8.06s/it]
  5%|▌         | 12067/225000 [35:14:17<424:35:42,  7.18s/it]
  5%|▌         | 12068/225000 [35:14:22<390:39:21,  6.60s/it]
  5%|▌         | 12069/225000 [35:14:26<349:29:27,  5.91s/it]
  5%|▌         | 12070/225000 [35:14:31<320:55:22,  5.43s/it]
  5%|▌         | 12071/225000 [35:14:44<463:15:05,  7.83s/it]
  5%|▌         | 12072/225000 [35:14:51<453:43:00,  7.67s/it]
  5%|▌         | 12073/225000 [35:14:56<404:34:34,  6.84s/it]
  5%|▌         | 12074/225000 [35:15:00<356:08:11,  6.02s/it]
  5%|▌         | 12075/225000 [35:15:14<491:25:21,  8.31s/it]
  5%|▌         | 12076/225000 [35:15:22<485:36:00,  8.21s/it]
  5%|▌         | 12077/225000 [35:15:27<418:53:37,  7.08s/it]
  5%|▌         | 12078/225000 [35:15:35<438:34:42,  7.42s/it]
  5%|▌         | 12079/225000 [35:15:39<389:55:01,  6.59s/it]
  5%|▌         | 12080/225000 [35:15:44<350:07:37,  5.92s/it]
  5%|▌         | 12081/225000 [35:15:55<440:47:12,  7.45s/it]
  5%|▌         | 12082/225000 [35:15:59<385:23:15,  6.52s/it]
  5%|▌         | 12083/225000 [35:16:03<342:19:27,  5.79s/it]
  5%|▌         | 12084/225000 [35:16:08<327:54:31,  5.54s/it]
  5%|▌         | 12085/225000 [35:16:22<467:41:10,  7.91s/it]
  5%|▌         | 12086/225000 [35:16:25<389:26:18,  6.58s/it]
  5%|▌         | 12087/225000 [35:16:34<432:00:17,  7.30s/it]
  5%|▌         | 12088/225000 [35:16:37<358:32:15,  6.06s/it]
  5%|▌         | 12089/225000 [35:16:44<379:17:14,  6.41s/it]
  5%|▌         | 12090/225000 [35:16:49<347:04:42,  5.87s/it]
  5%|▌         | 12091/225000 [35:16:59<418:31:03,  7.08s/it]
  5%|▌         | 12092/225000 [35:17:04<381:30:31,  6.45s/it]
  5%|▌         | 12093/225000 [35:17:09<364:57:22,  6.17s/it]
  5%|▌         | 12094/225000 [35:17:14<333:54:35,  5.65s/it]
  5%|▌         | 12095/225000 [35:17:27<466:31:30,  7.89s/it]
  5%|▌         | 12096/225000 [35:17:32<407:02:22,  6.88s/it]
  5%|▌         | 12097/225000 [35:17:38<407:23:57,  6.89s/it]
  5%|▌         | 12098/225000 [35:17:47<435:47:20,  7.37s/it]
  5%|▌         | 12099/225000 [35:17:51<381:07:19,  6.44s/it]
  5%|▌         | 12100/225000 [35:17:56<347:34:21,  5.88s/it]
                                                             
{'loss': '0.737', 'grad_norm': '0.7421', 'learning_rate': '4.999e-05', 'epoch': '0.05378', 'num_input_tokens_seen': 63151176, 'train_runtime': '1.271e+05', 'train_tokens_per_second': '497'}

  5%|▌         | 12100/225000 [35:17:56<347:34:21,  5.88s/it]
  5%|▌         | 12101/225000 [35:18:01<330:03:42,  5.58s/it]
  5%|▌         | 12102/225000 [35:18:11<407:01:46,  6.88s/it]
  5%|▌         | 12103/225000 [35:18:17<396:59:04,  6.71s/it]
  5%|▌         | 12104/225000 [35:18:25<420:52:08,  7.12s/it]
  5%|▌         | 12105/225000 [35:18:35<471:30:50,  7.97s/it]
  5%|▌         | 12106/225000 [35:18:40<420:21:59,  7.11s/it]
  5%|▌         | 12107/225000 [35:18:46<394:57:25,  6.68s/it]
  5%|▌         | 12108/225000 [35:18:51<371:23:39,  6.28s/it]
  5%|▌         | 12109/225000 [35:18:57<367:06:07,  6.21s/it]
  5%|▌         | 12110/225000 [35:19:06<406:48:07,  6.88s/it]
  5%|▌         | 12111/225000 [35:19:12<408:07:16,  6.90s/it]
  5%|▌         | 12112/225000 [35:19:24<486:30:08,  8.23s/it]
  5%|▌         | 12113/225000 [35:19:28<420:58:39,  7.12s/it]
  5%|▌         | 12114/225000 [35:19:33<376:58:15,  6.37s/it]
  5%|▌         | 12115/225000 [35:19:43<437:57:50,  7.41s/it]
  5%|▌         | 12116/225000 [35:19:50<439:22:13,  7.43s/it]
  5%|▌         | 12117/225000 [35:19:55<391:31:53,  6.62s/it]
  5%|▌         | 12118/225000 [35:20:00<360:44:55,  6.10s/it]
  5%|▌         | 12119/225000 [35:20:09<408:38:04,  6.91s/it]
  5%|▌         | 12120/225000 [35:20:13<369:43:48,  6.25s/it]
  5%|▌         | 12121/225000 [35:20:18<342:53:29,  5.80s/it]
  5%|▌         | 12122/225000 [35:20:23<334:01:00,  5.65s/it]
  5%|▌         | 12123/225000 [35:20:34<415:56:54,  7.03s/it]
  5%|▌         | 12124/225000 [35:20:44<468:55:23,  7.93s/it]
  5%|▌         | 12125/225000 [35:20:49<413:30:01,  6.99s/it]
  5%|▌         | 12126/225000 [35:20:54<378:45:56,  6.41s/it]
  5%|▌         | 12127/225000 [35:21:02<412:57:31,  6.98s/it]
  5%|▌         | 12128/225000 [35:21:12<461:32:28,  7.81s/it]
  5%|▌         | 12129/225000 [35:21:17<415:05:41,  7.02s/it]
  5%|▌         | 12130/225000 [35:21:25<433:45:09,  7.34s/it]
  5%|▌         | 12131/225000 [35:21:30<387:32:31,  6.55s/it]
  5%|▌         | 12132/225000 [35:21:37<405:01:44,  6.85s/it]
  5%|▌         | 12133/225000 [35:21:42<371:30:48,  6.28s/it]
  5%|▌         | 12134/225000 [35:21:47<352:55:24,  5.97s/it]
  5%|▌         | 12135/225000 [35:21:55<387:14:43,  6.55s/it]
  5%|▌         | 12136/225000 [35:22:06<466:44:38,  7.89s/it]
  5%|▌         | 12137/225000 [35:22:11<403:18:39,  6.82s/it]
  5%|▌         | 12138/225000 [35:22:21<467:00:03,  7.90s/it]
  5%|▌         | 12139/225000 [35:22:30<487:05:50,  8.24s/it]
  5%|▌         | 12140/225000 [35:22:34<418:10:41,  7.07s/it]
  5%|▌         | 12141/225000 [35:22:39<376:31:51,  6.37s/it]
  5%|▌         | 12142/225000 [35:22:49<430:50:02,  7.29s/it]
  5%|▌         | 12143/225000 [35:22:54<399:32:22,  6.76s/it]
  5%|▌         | 12144/225000 [35:23:03<437:24:14,  7.40s/it]
  5%|▌         | 12145/225000 [35:23:07<377:20:23,  6.38s/it]
  5%|▌         | 12146/225000 [35:23:12<349:26:10,  5.91s/it]
  5%|▌         | 12147/225000 [35:23:17<337:24:11,  5.71s/it]
  5%|▌         | 12148/225000 [35:23:23<335:35:18,  5.68s/it]
  5%|▌         | 12149/225000 [35:23:28<324:25:54,  5.49s/it]
  5%|▌         | 12150/225000 [35:23:33<322:27:20,  5.45s/it]
                                                             
{'loss': '0.7463', 'grad_norm': '0.6423', 'learning_rate': '4.999e-05', 'epoch': '0.054', 'num_input_tokens_seen': 63313888, 'train_runtime': '1.274e+05', 'train_tokens_per_second': '496.9'}

  5%|▌         | 12150/225000 [35:23:33<322:27:20,  5.45s/it]
  5%|▌         | 12151/225000 [35:23:38<321:39:49,  5.44s/it]
  5%|▌         | 12152/225000 [35:23:51<441:04:32,  7.46s/it]
  5%|▌         | 12153/225000 [35:23:56<406:38:42,  6.88s/it]
  5%|▌         | 12154/225000 [35:24:02<379:40:20,  6.42s/it]
  5%|▌         | 12155/225000 [35:24:07<370:04:37,  6.26s/it]
  5%|▌         | 12156/225000 [35:24:17<422:39:54,  7.15s/it]
  5%|▌         | 12157/225000 [35:24:26<464:44:25,  7.86s/it]
  5%|▌         | 12158/225000 [35:24:31<403:56:49,  6.83s/it]
  5%|▌         | 12159/225000 [35:24:42<490:40:35,  8.30s/it]
  5%|▌         | 12160/225000 [35:24:47<425:17:11,  7.19s/it]
  5%|▌         | 12161/225000 [35:24:52<386:25:05,  6.54s/it]
  5%|▌         | 12162/225000 [35:25:03<470:55:18,  7.97s/it]
  5%|▌         | 12163/225000 [35:25:09<434:39:06,  7.35s/it]
  5%|▌         | 12164/225000 [35:25:14<398:30:46,  6.74s/it]
  5%|▌         | 12165/225000 [35:25:20<384:49:36,  6.51s/it]
  5%|▌         | 12166/225000 [35:25:26<367:21:56,  6.21s/it]
  5%|▌         | 12167/225000 [35:25:35<417:46:19,  7.07s/it]
  5%|▌         | 12168/225000 [35:25:39<370:22:45,  6.26s/it]
  5%|▌         | 12169/225000 [35:25:44<340:15:11,  5.76s/it]
  5%|▌         | 12170/225000 [35:25:56<447:56:37,  7.58s/it]
  5%|▌         | 12171/225000 [35:25:59<376:09:20,  6.36s/it]
  5%|▌         | 12172/225000 [35:26:08<411:24:20,  6.96s/it]
  5%|▌         | 12173/225000 [35:26:13<378:30:31,  6.40s/it]
  5%|▌         | 12174/225000 [35:26:18<357:37:05,  6.05s/it]
  5%|▌         | 12175/225000 [35:26:23<341:59:14,  5.78s/it]
  5%|▌         | 12176/225000 [35:26:28<331:23:15,  5.61s/it]
  5%|▌         | 12177/225000 [35:26:34<335:44:23,  5.68s/it]
  5%|▌         | 12178/225000 [35:26:39<325:46:52,  5.51s/it]
  5%|▌         | 12179/225000 [35:26:51<434:35:36,  7.35s/it]
  5%|▌         | 12180/225000 [35:27:00<468:02:38,  7.92s/it]
  5%|▌         | 12181/225000 [35:27:05<410:02:47,  6.94s/it]
  5%|▌         | 12182/225000 [35:27:14<444:16:45,  7.52s/it]
  5%|▌         | 12183/225000 [35:27:18<386:05:39,  6.53s/it]
  5%|▌         | 12184/225000 [35:27:27<439:05:41,  7.43s/it]
  5%|▌         | 12185/225000 [35:27:37<482:47:15,  8.17s/it]
  5%|▌         | 12186/225000 [35:27:43<429:28:53,  7.27s/it]
  5%|▌         | 12187/225000 [35:27:48<394:20:28,  6.67s/it]
  5%|▌         | 12188/225000 [35:27:53<372:22:42,  6.30s/it]
  5%|▌         | 12189/225000 [35:27:58<349:04:38,  5.91s/it]
  5%|▌         | 12190/225000 [35:28:05<359:22:54,  6.08s/it]
  5%|▌         | 12191/225000 [35:28:18<478:40:41,  8.10s/it]
  5%|▌         | 12192/225000 [35:28:21<400:05:02,  6.77s/it]
  5%|▌         | 12193/225000 [35:28:29<418:56:23,  7.09s/it]
  5%|▌         | 12194/225000 [35:28:35<406:41:48,  6.88s/it]
  5%|▌         | 12195/225000 [35:28:43<423:46:37,  7.17s/it]
  5%|▌         | 12196/225000 [35:28:48<386:30:39,  6.54s/it]
  5%|▌         | 12197/225000 [35:28:54<375:54:28,  6.36s/it]
  5%|▌         | 12198/225000 [35:29:01<373:58:46,  6.33s/it]
  5%|▌         | 12199/225000 [35:29:08<389:08:55,  6.58s/it]
  5%|▌         | 12200/225000 [35:29:17<433:00:49,  7.33s/it]
                                                             
{'loss': '0.7373', 'grad_norm': '0.6962', 'learning_rate': '4.999e-05', 'epoch': '0.05422', 'num_input_tokens_seen': 63474320, 'train_runtime': '1.278e+05', 'train_tokens_per_second': '496.8'}

  5%|▌         | 12200/225000 [35:29:17<433:00:49,  7.33s/it]
  5%|▌         | 12201/225000 [35:29:21<374:59:00,  6.34s/it]
  5%|▌         | 12202/225000 [35:29:30<424:16:48,  7.18s/it]
  5%|▌         | 12203/225000 [35:29:38<433:53:03,  7.34s/it]
  5%|▌         | 12204/225000 [35:29:42<381:34:40,  6.46s/it]
  5%|▌         | 12205/225000 [35:29:47<357:40:11,  6.05s/it]
  5%|▌         | 12206/225000 [35:29:52<340:41:11,  5.76s/it]
  5%|▌         | 12207/225000 [35:29:57<330:16:05,  5.59s/it]
  5%|▌         | 12208/225000 [35:30:14<527:50:07,  8.93s/it]
  5%|▌         | 12209/225000 [35:30:18<440:34:57,  7.45s/it]
  5%|▌         | 12210/225000 [35:30:25<435:50:02,  7.37s/it]
  5%|▌         | 12211/225000 [35:30:33<446:24:13,  7.55s/it]
  5%|▌         | 12212/225000 [35:30:39<405:43:50,  6.86s/it]
  5%|▌         | 12213/225000 [35:30:44<378:08:23,  6.40s/it]
  5%|▌         | 12214/225000 [35:30:52<412:55:51,  6.99s/it]
  5%|▌         | 12215/225000 [35:30:59<406:19:30,  6.87s/it]
  5%|▌         | 12216/225000 [35:31:06<403:08:17,  6.82s/it]
  5%|▌         | 12217/225000 [35:31:11<384:24:07,  6.50s/it]
  5%|▌         | 12218/225000 [35:31:15<341:49:25,  5.78s/it]
  5%|▌         | 12219/225000 [35:31:27<439:12:46,  7.43s/it]
  5%|▌         | 12220/225000 [35:31:38<500:12:09,  8.46s/it]
  5%|▌         | 12221/225000 [35:31:42<423:48:26,  7.17s/it]
  5%|▌         | 12222/225000 [35:31:50<451:05:42,  7.63s/it]
  5%|▌         | 12223/225000 [35:31:55<401:46:16,  6.80s/it]
  5%|▌         | 12224/225000 [35:32:00<359:55:14,  6.09s/it]
  5%|▌         | 12225/225000 [35:32:04<336:46:15,  5.70s/it]
  5%|▌         | 12226/225000 [35:32:18<474:34:07,  8.03s/it]
  5%|▌         | 12227/225000 [35:32:23<413:39:23,  7.00s/it]
  5%|▌         | 12228/225000 [35:32:28<382:41:47,  6.48s/it]
  5%|▌         | 12229/225000 [35:32:33<356:39:26,  6.03s/it]
  5%|▌         | 12230/225000 [35:32:38<343:50:49,  5.82s/it]
  5%|▌         | 12231/225000 [35:32:43<327:46:49,  5.55s/it]
  5%|▌         | 12232/225000 [35:32:57<483:08:52,  8.17s/it]
  5%|▌         | 12233/225000 [35:33:02<416:57:14,  7.05s/it]
  5%|▌         | 12234/225000 [35:33:09<426:48:04,  7.22s/it]
  5%|▌         | 12235/225000 [35:33:16<413:04:56,  6.99s/it]
  5%|▌         | 12236/225000 [35:33:26<477:43:40,  8.08s/it]
  5%|▌         | 12237/225000 [35:33:31<422:29:43,  7.15s/it]
  5%|▌         | 12238/225000 [35:33:37<391:24:19,  6.62s/it]
  5%|▌         | 12239/225000 [35:33:41<356:09:06,  6.03s/it]
  5%|▌         | 12240/225000 [35:33:50<400:28:22,  6.78s/it]
  5%|▌         | 12241/225000 [35:33:55<375:12:02,  6.35s/it]
  5%|▌         | 12242/225000 [35:34:01<357:12:57,  6.04s/it]
  5%|▌         | 12243/225000 [35:34:06<348:06:02,  5.89s/it]
  5%|▌         | 12244/225000 [35:34:12<353:21:24,  5.98s/it]
  5%|▌         | 12245/225000 [35:34:26<485:09:58,  8.21s/it]
  5%|▌         | 12246/225000 [35:34:32<445:42:14,  7.54s/it]
  5%|▌         | 12247/225000 [35:34:37<395:59:46,  6.70s/it]
  5%|▌         | 12248/225000 [35:34:41<364:53:22,  6.17s/it]
  5%|▌         | 12249/225000 [35:34:46<340:20:08,  5.76s/it]
  5%|▌         | 12250/225000 [35:34:51<325:08:56,  5.50s/it]
                                                             
{'loss': '0.7574', 'grad_norm': '0.7521', 'learning_rate': '4.999e-05', 'epoch': '0.05444', 'num_input_tokens_seen': 63634136, 'train_runtime': '1.281e+05', 'train_tokens_per_second': '496.8'}

  5%|▌         | 12250/225000 [35:34:51<325:08:56,  5.50s/it]
  5%|▌         | 12251/225000 [35:35:03<439:15:00,  7.43s/it]
  5%|▌         | 12252/225000 [35:35:07<381:02:36,  6.45s/it]
  5%|▌         | 12253/225000 [35:35:12<357:59:26,  6.06s/it]
  5%|▌         | 12254/225000 [35:35:19<372:20:52,  6.30s/it]
  5%|▌         | 12255/225000 [35:35:29<437:42:08,  7.41s/it]
  5%|▌         | 12256/225000 [35:35:34<382:49:01,  6.48s/it]
  5%|▌         | 12257/225000 [35:35:39<370:19:23,  6.27s/it]
  5%|▌         | 12258/225000 [35:35:45<359:22:44,  6.08s/it]
  5%|▌         | 12259/225000 [35:35:51<358:04:32,  6.06s/it]
  5%|▌         | 12260/225000 [35:36:01<435:33:00,  7.37s/it]
  5%|▌         | 12261/225000 [35:36:06<385:20:17,  6.52s/it]
  5%|▌         | 12262/225000 [35:36:12<368:14:12,  6.23s/it]
  5%|▌         | 12263/225000 [35:36:21<433:19:09,  7.33s/it]
  5%|▌         | 12264/225000 [35:36:27<401:11:00,  6.79s/it]
  5%|▌         | 12265/225000 [35:36:32<374:23:55,  6.34s/it]
  5%|▌         | 12266/225000 [35:36:37<342:09:30,  5.79s/it]
  5%|▌         | 12267/225000 [35:36:48<445:41:22,  7.54s/it]
  5%|▌         | 12268/225000 [35:36:54<411:54:53,  6.97s/it]
  5%|▌         | 12269/225000 [35:37:01<411:02:28,  6.96s/it]
  5%|▌         | 12270/225000 [35:37:06<372:02:19,  6.30s/it]
  5%|▌         | 12271/225000 [35:37:10<343:43:44,  5.82s/it]
  5%|▌         | 12272/225000 [35:37:17<356:23:17,  6.03s/it]
  5%|▌         | 12273/225000 [35:37:29<463:11:36,  7.84s/it]
  5%|▌         | 12274/225000 [35:37:33<396:48:22,  6.72s/it]
  5%|▌         | 12275/225000 [35:37:43<460:53:34,  7.80s/it]
  5%|▌         | 12276/225000 [35:37:48<398:52:51,  6.75s/it]
  5%|▌         | 12277/225000 [35:37:54<386:41:17,  6.54s/it]
  5%|▌         | 12278/225000 [35:38:07<511:30:20,  8.66s/it]
  5%|▌         | 12279/225000 [35:38:11<425:20:04,  7.20s/it]
  5%|▌         | 12280/225000 [35:38:19<428:34:23,  7.25s/it]
  5%|▌         | 12281/225000 [35:38:23<370:24:08,  6.27s/it]
  5%|▌         | 12282/225000 [35:38:32<424:57:31,  7.19s/it]
  5%|▌         | 12283/225000 [35:38:36<374:24:46,  6.34s/it]
  5%|▌         | 12284/225000 [35:38:42<358:27:01,  6.07s/it]
  5%|▌         | 12285/225000 [35:38:47<339:17:17,  5.74s/it]
  5%|▌         | 12286/225000 [35:38:59<457:47:23,  7.75s/it]
  5%|▌         | 12287/225000 [35:39:05<417:18:35,  7.06s/it]
  5%|▌         | 12288/225000 [35:39:10<388:20:06,  6.57s/it]
  5%|▌         | 12289/225000 [35:39:20<443:23:07,  7.50s/it]
  5%|▌         | 12290/225000 [35:39:26<422:22:33,  7.15s/it]
  5%|▌         | 12291/225000 [35:39:37<500:35:36,  8.47s/it]
  5%|▌         | 12292/225000 [35:39:43<448:40:02,  7.59s/it]
  5%|▌         | 12293/225000 [35:39:48<403:19:08,  6.83s/it]
  5%|▌         | 12294/225000 [35:39:53<372:34:20,  6.31s/it]
  5%|▌         | 12295/225000 [35:39:58<341:55:57,  5.79s/it]
  5%|▌         | 12296/225000 [35:40:10<452:35:28,  7.66s/it]
  5%|▌         | 12297/225000 [35:40:21<509:39:10,  8.63s/it]
  5%|▌         | 12298/225000 [35:40:27<464:22:49,  7.86s/it]
  5%|▌         | 12299/225000 [35:40:31<408:08:55,  6.91s/it]
  5%|▌         | 12300/225000 [35:40:35<356:54:18,  6.04s/it]
                                                             
{'loss': '0.7513', 'grad_norm': '0.6456', 'learning_rate': '4.999e-05', 'epoch': '0.05467', 'num_input_tokens_seen': 63793464, 'train_runtime': '1.284e+05', 'train_tokens_per_second': '496.7'}

  5%|▌         | 12300/225000 [35:40:35<356:54:18,  6.04s/it]
  5%|▌         | 12301/225000 [35:40:44<404:53:47,  6.85s/it]
  5%|▌         | 12302/225000 [35:40:52<416:30:02,  7.05s/it]
  5%|▌         | 12303/225000 [35:40:59<418:43:21,  7.09s/it]
  5%|▌         | 12304/225000 [35:41:04<375:32:42,  6.36s/it]
  5%|▌         | 12305/225000 [35:41:12<419:10:51,  7.09s/it]
  5%|▌         | 12306/225000 [35:41:21<441:24:46,  7.47s/it]
  5%|▌         | 12307/225000 [35:41:31<488:54:54,  8.28s/it]
  5%|▌         | 12308/225000 [35:41:37<449:59:27,  7.62s/it]
  5%|▌         | 12309/225000 [35:41:43<421:51:48,  7.14s/it]
  5%|▌         | 12310/225000 [35:41:49<405:35:30,  6.87s/it]
  5%|▌         | 12311/225000 [35:41:55<382:30:01,  6.47s/it]
  5%|▌         | 12312/225000 [35:42:00<365:45:08,  6.19s/it]
  5%|▌         | 12313/225000 [35:42:14<491:07:38,  8.31s/it]
  5%|▌         | 12314/225000 [35:42:20<455:55:05,  7.72s/it]
  5%|▌         | 12315/225000 [35:42:25<405:11:42,  6.86s/it]
  5%|▌         | 12316/225000 [35:42:33<425:34:43,  7.20s/it]
  5%|▌         | 12317/225000 [35:42:38<383:41:29,  6.49s/it]
  5%|▌         | 12318/225000 [35:42:42<352:11:10,  5.96s/it]
  5%|▌         | 12319/225000 [35:42:52<423:52:17,  7.17s/it]
  5%|▌         | 12320/225000 [35:43:01<447:09:37,  7.57s/it]
  5%|▌         | 12321/225000 [35:43:05<388:00:26,  6.57s/it]
  5%|▌         | 12322/225000 [35:43:15<445:46:21,  7.55s/it]
  5%|▌         | 12323/225000 [35:43:19<394:27:38,  6.68s/it]
  5%|▌         | 12324/225000 [35:43:24<357:33:28,  6.05s/it]
  5%|▌         | 12325/225000 [35:43:29<344:15:16,  5.83s/it]
  5%|▌         | 12326/225000 [35:43:40<423:09:04,  7.16s/it]
  5%|▌         | 12327/225000 [35:43:46<413:35:17,  7.00s/it]
  5%|▌         | 12328/225000 [35:43:53<402:48:50,  6.82s/it]
  5%|▌         | 12329/225000 [35:43:57<362:27:01,  6.14s/it]
  5%|▌         | 12330/225000 [35:44:03<348:52:49,  5.91s/it]
  5%|▌         | 12331/225000 [35:44:07<328:58:06,  5.57s/it]
  5%|▌         | 12332/225000 [35:44:12<313:05:12,  5.30s/it]
  5%|▌         | 12333/225000 [35:44:28<509:44:21,  8.63s/it]
  5%|▌         | 12334/225000 [35:44:32<425:31:24,  7.20s/it]
  5%|▌         | 12335/225000 [35:44:39<419:49:02,  7.11s/it]
  5%|▌         | 12336/225000 [35:44:45<393:37:44,  6.66s/it]
  5%|▌         | 12337/225000 [35:44:50<368:40:13,  6.24s/it]
  5%|▌         | 12338/225000 [35:44:55<343:51:34,  5.82s/it]
  5%|▌         | 12339/225000 [35:45:00<337:17:03,  5.71s/it]
  5%|▌         | 12340/225000 [35:45:09<381:05:17,  6.45s/it]
  5%|▌         | 12341/225000 [35:45:19<457:25:16,  7.74s/it]
  5%|▌         | 12342/225000 [35:45:26<442:22:10,  7.49s/it]
  5%|▌         | 12343/225000 [35:45:32<405:51:50,  6.87s/it]
  5%|▌         | 12344/225000 [35:45:39<408:52:32,  6.92s/it]
  5%|▌         | 12345/225000 [35:45:45<400:22:42,  6.78s/it]
  5%|▌         | 12346/225000 [35:45:51<392:49:08,  6.65s/it]
  5%|▌         | 12347/225000 [35:46:00<418:29:10,  7.08s/it]
  5%|▌         | 12348/225000 [35:46:06<415:12:47,  7.03s/it]
  5%|▌         | 12349/225000 [35:46:14<432:58:46,  7.33s/it]
  5%|▌         | 12350/225000 [35:46:20<406:08:12,  6.88s/it]
                                                             
{'loss': '0.7415', 'grad_norm': '0.6607', 'learning_rate': '4.999e-05', 'epoch': '0.05489', 'num_input_tokens_seen': 63957280, 'train_runtime': '1.288e+05', 'train_tokens_per_second': '496.6'}

  5%|▌         | 12350/225000 [35:46:20<406:08:12,  6.88s/it]
  5%|▌         | 12351/225000 [35:46:24<351:22:17,  5.95s/it]
  5%|▌         | 12352/225000 [35:46:36<454:29:52,  7.69s/it]
  5%|▌         | 12353/225000 [35:46:41<415:32:53,  7.04s/it]
  5%|▌         | 12354/225000 [35:46:47<386:14:04,  6.54s/it]
  5%|▌         | 12355/225000 [35:46:52<355:53:59,  6.03s/it]
  5%|▌         | 12356/225000 [35:46:57<337:50:33,  5.72s/it]
  5%|▌         | 12357/225000 [35:47:02<338:35:27,  5.73s/it]
  5%|▌         | 12358/225000 [35:47:08<332:50:56,  5.64s/it]
  5%|▌         | 12359/225000 [35:47:13<322:52:17,  5.47s/it]
  5%|▌         | 12360/225000 [35:47:21<376:31:53,  6.37s/it]
  5%|▌         | 12361/225000 [35:47:35<509:14:38,  8.62s/it]
  5%|▌         | 12362/225000 [35:47:40<449:07:26,  7.60s/it]
  5%|▌         | 12363/225000 [35:47:45<397:38:39,  6.73s/it]
  5%|▌         | 12364/225000 [35:47:51<378:48:17,  6.41s/it]
  5%|▌         | 12365/225000 [35:48:00<433:34:37,  7.34s/it]
  5%|▌         | 12366/225000 [35:48:05<390:49:45,  6.62s/it]
  5%|▌         | 12367/225000 [35:48:10<365:55:47,  6.20s/it]
  5%|▌         | 12368/225000 [35:48:15<340:58:00,  5.77s/it]
  5%|▌         | 12369/225000 [35:48:30<494:08:07,  8.37s/it]
  5%|▌         | 12370/225000 [35:48:34<428:37:48,  7.26s/it]
  5%|▌         | 12371/225000 [35:48:39<382:14:24,  6.47s/it]
  5%|▌         | 12372/225000 [35:48:55<549:55:06,  9.31s/it]
  5%|▌         | 12373/225000 [35:48:59<456:39:58,  7.73s/it]
  5%|▌         | 12374/225000 [35:49:04<413:33:17,  7.00s/it]
  6%|▌         | 12375/225000 [35:49:10<385:50:13,  6.53s/it]
  6%|▌         | 12376/225000 [35:49:15<359:32:25,  6.09s/it]
  6%|▌         | 12377/225000 [35:49:27<470:53:14,  7.97s/it]
  6%|▌         | 12378/225000 [35:49:32<420:32:57,  7.12s/it]
  6%|▌         | 12379/225000 [35:49:37<376:13:08,  6.37s/it]
  6%|▌         | 12380/225000 [35:49:42<353:30:16,  5.99s/it]
  6%|▌         | 12381/225000 [35:49:53<441:49:41,  7.48s/it]
  6%|▌         | 12382/225000 [35:49:58<407:18:59,  6.90s/it]
  6%|▌         | 12383/225000 [35:50:04<377:15:27,  6.39s/it]
  6%|▌         | 12384/225000 [35:50:14<452:38:25,  7.66s/it]
  6%|▌         | 12385/225000 [35:50:21<443:08:24,  7.50s/it]
  6%|▌         | 12386/225000 [35:50:26<392:41:51,  6.65s/it]
  6%|▌         | 12387/225000 [35:50:36<442:49:40,  7.50s/it]
  6%|▌         | 12388/225000 [35:50:41<398:54:28,  6.75s/it]
  6%|▌         | 12389/225000 [35:50:47<392:25:11,  6.64s/it]
  6%|▌         | 12390/225000 [35:50:55<411:37:29,  6.97s/it]
  6%|▌         | 12391/225000 [35:50:59<370:45:10,  6.28s/it]
  6%|▌         | 12392/225000 [35:51:04<346:11:03,  5.86s/it]
  6%|▌         | 12393/225000 [35:51:11<359:03:53,  6.08s/it]
  6%|▌         | 12394/225000 [35:51:16<345:45:44,  5.85s/it]
  6%|▌         | 12395/225000 [35:51:23<367:40:40,  6.23s/it]
  6%|▌         | 12396/225000 [35:51:33<433:58:55,  7.35s/it]
  6%|▌         | 12397/225000 [35:51:38<381:47:41,  6.46s/it]
  6%|▌         | 12398/225000 [35:51:47<435:36:27,  7.38s/it]
  6%|▌         | 12399/225000 [35:51:52<395:58:44,  6.71s/it]
  6%|▌         | 12400/225000 [35:51:59<402:29:07,  6.82s/it]
                                                             
{'loss': '0.7516', 'grad_norm': '0.735', 'learning_rate': '4.998e-05', 'epoch': '0.05511', 'num_input_tokens_seen': 64116136, 'train_runtime': '1.291e+05', 'train_tokens_per_second': '496.6'}

  6%|▌         | 12400/225000 [35:51:59<402:29:07,  6.82s/it]
  6%|▌         | 12401/225000 [35:52:06<391:52:17,  6.64s/it]
  6%|▌         | 12402/225000 [35:52:13<412:04:27,  6.98s/it]
  6%|▌         | 12403/225000 [35:52:17<360:13:12,  6.10s/it]
  6%|▌         | 12404/225000 [35:52:23<359:15:42,  6.08s/it]
  6%|▌         | 12405/225000 [35:52:29<359:58:12,  6.10s/it]
  6%|▌         | 12406/225000 [35:52:35<353:03:00,  5.98s/it]
  6%|▌         | 12407/225000 [35:52:41<345:48:18,  5.86s/it]
  6%|▌         | 12408/225000 [35:52:47<345:37:15,  5.85s/it]
  6%|▌         | 12409/225000 [35:52:52<341:10:42,  5.78s/it]
  6%|▌         | 12410/225000 [35:53:04<448:41:14,  7.60s/it]
  6%|▌         | 12411/225000 [35:53:10<415:48:20,  7.04s/it]
  6%|▌         | 12412/225000 [35:53:16<408:23:12,  6.92s/it]
  6%|▌         | 12413/225000 [35:53:24<425:32:14,  7.21s/it]
  6%|▌         | 12414/225000 [35:53:28<368:21:12,  6.24s/it]
  6%|▌         | 12415/225000 [35:53:41<474:06:22,  8.03s/it]
  6%|▌         | 12416/225000 [35:53:45<412:58:13,  6.99s/it]
  6%|▌         | 12417/225000 [35:53:51<392:19:22,  6.64s/it]
  6%|▌         | 12418/225000 [35:54:01<460:46:45,  7.80s/it]
  6%|▌         | 12419/225000 [35:54:07<413:20:55,  7.00s/it]
  6%|▌         | 12420/225000 [35:54:14<415:31:17,  7.04s/it]
  6%|▌         | 12421/225000 [35:54:18<372:16:46,  6.30s/it]
  6%|▌         | 12422/225000 [35:54:24<353:39:51,  5.99s/it]
  6%|▌         | 12423/225000 [35:54:32<406:23:53,  6.88s/it]
  6%|▌         | 12424/225000 [35:54:40<413:36:00,  7.00s/it]
  6%|▌         | 12425/225000 [35:54:46<395:00:21,  6.69s/it]
  6%|▌         | 12426/225000 [35:54:54<415:20:58,  7.03s/it]
  6%|▌         | 12427/225000 [35:55:03<454:24:52,  7.70s/it]
  6%|▌         | 12428/225000 [35:55:07<383:41:58,  6.50s/it]
  6%|▌         | 12429/225000 [35:55:16<436:52:13,  7.40s/it]
  6%|▌         | 12430/225000 [35:55:20<379:12:27,  6.42s/it]
  6%|▌         | 12431/225000 [35:55:29<430:36:45,  7.29s/it]
  6%|▌         | 12432/225000 [35:55:37<428:32:41,  7.26s/it]
  6%|▌         | 12433/225000 [35:55:40<366:58:48,  6.22s/it]
  6%|▌         | 12434/225000 [35:55:48<397:45:51,  6.74s/it]
  6%|▌         | 12435/225000 [35:55:53<361:08:06,  6.12s/it]
  6%|▌         | 12436/225000 [35:56:02<408:13:51,  6.91s/it]
  6%|▌         | 12437/225000 [35:56:12<468:51:35,  7.94s/it]
  6%|▌         | 12438/225000 [35:56:16<395:30:20,  6.70s/it]
  6%|▌         | 12439/225000 [35:56:20<344:28:35,  5.83s/it]
  6%|▌         | 12440/225000 [35:56:28<395:24:07,  6.70s/it]
  6%|▌         | 12441/225000 [35:56:36<402:24:40,  6.82s/it]
  6%|▌         | 12442/225000 [35:56:40<364:03:07,  6.17s/it]
  6%|▌         | 12443/225000 [35:56:46<354:38:13,  6.01s/it]
  6%|▌         | 12444/225000 [35:56:51<344:02:13,  5.83s/it]
  6%|▌         | 12445/225000 [35:57:02<438:59:57,  7.44s/it]
  6%|▌         | 12446/225000 [35:57:08<397:48:37,  6.74s/it]
  6%|▌         | 12447/225000 [35:57:14<389:28:47,  6.60s/it]
  6%|▌         | 12448/225000 [35:57:23<443:30:47,  7.51s/it]
  6%|▌         | 12449/225000 [35:57:31<435:33:59,  7.38s/it]
  6%|▌         | 12450/225000 [35:57:34<373:12:18,  6.32s/it]
                                                             
{'loss': '0.7364', 'grad_norm': '0.6126', 'learning_rate': '4.998e-05', 'epoch': '0.05533', 'num_input_tokens_seen': 64276816, 'train_runtime': '1.295e+05', 'train_tokens_per_second': '496.5'}

  6%|▌         | 12450/225000 [35:57:34<373:12:18,  6.32s/it]
  6%|▌         | 12451/225000 [35:57:44<426:37:33,  7.23s/it]
  6%|▌         | 12452/225000 [35:57:51<424:25:46,  7.19s/it]
  6%|▌         | 12453/225000 [35:57:56<387:16:29,  6.56s/it]
  6%|▌         | 12454/225000 [35:58:00<349:27:48,  5.92s/it]
  6%|▌         | 12455/225000 [35:58:14<481:28:17,  8.15s/it]
  6%|▌         | 12456/225000 [35:58:19<438:53:28,  7.43s/it]
  6%|▌         | 12457/225000 [35:58:26<419:19:54,  7.10s/it]
  6%|▌         | 12458/225000 [35:58:30<374:06:04,  6.34s/it]
  6%|▌         | 12459/225000 [35:58:40<440:09:43,  7.46s/it]
  6%|▌         | 12460/225000 [35:58:46<414:31:17,  7.02s/it]
  6%|▌         | 12461/225000 [35:58:52<396:56:51,  6.72s/it]
  6%|▌         | 12462/225000 [35:58:58<379:22:37,  6.43s/it]
  6%|▌         | 12463/225000 [35:59:04<372:19:15,  6.31s/it]
  6%|▌         | 12464/225000 [35:59:09<352:22:50,  5.97s/it]
  6%|▌         | 12465/225000 [35:59:15<352:45:30,  5.98s/it]
  6%|▌         | 12466/225000 [35:59:21<345:16:17,  5.85s/it]
  6%|▌         | 12467/225000 [35:59:27<346:20:44,  5.87s/it]
  6%|▌         | 12468/225000 [35:59:33<344:39:27,  5.84s/it]
  6%|▌         | 12469/225000 [35:59:38<341:40:46,  5.79s/it]
  6%|▌         | 12470/225000 [35:59:53<498:07:43,  8.44s/it]
  6%|▌         | 12471/225000 [35:59:59<456:57:12,  7.74s/it]
  6%|▌         | 12472/225000 [36:00:05<423:11:37,  7.17s/it]
  6%|▌         | 12473/225000 [36:00:10<392:42:15,  6.65s/it]
  6%|▌         | 12474/225000 [36:00:22<489:02:52,  8.28s/it]
  6%|▌         | 12475/225000 [36:00:27<418:36:24,  7.09s/it]
  6%|▌         | 12476/225000 [36:00:32<379:55:50,  6.44s/it]
  6%|▌         | 12477/225000 [36:00:39<395:54:09,  6.71s/it]
  6%|▌         | 12478/225000 [36:00:47<412:14:03,  6.98s/it]
  6%|▌         | 12479/225000 [36:00:52<379:33:46,  6.43s/it]
  6%|▌         | 12480/225000 [36:00:57<353:01:40,  5.98s/it]
  6%|▌         | 12481/225000 [36:01:06<417:37:12,  7.07s/it]
  6%|▌         | 12482/225000 [36:01:17<484:17:45,  8.20s/it]
  6%|▌         | 12483/225000 [36:01:21<410:53:04,  6.96s/it]
  6%|▌         | 12484/225000 [36:01:29<425:00:58,  7.20s/it]
  6%|▌         | 12485/225000 [36:01:35<403:46:57,  6.84s/it]
  6%|▌         | 12486/225000 [36:01:40<377:04:54,  6.39s/it]
  6%|▌         | 12487/225000 [36:01:47<377:06:11,  6.39s/it]
  6%|▌         | 12488/225000 [36:01:59<481:28:31,  8.16s/it]
  6%|▌         | 12489/225000 [36:02:03<417:06:36,  7.07s/it]
  6%|▌         | 12490/225000 [36:02:13<467:10:42,  7.91s/it]
  6%|▌         | 12491/225000 [36:02:17<389:44:01,  6.60s/it]
  6%|▌         | 12492/225000 [36:02:22<359:54:17,  6.10s/it]
  6%|▌         | 12493/225000 [36:02:28<356:39:59,  6.04s/it]
  6%|▌         | 12494/225000 [36:02:33<343:13:53,  5.81s/it]
  6%|▌         | 12495/225000 [36:02:39<341:53:09,  5.79s/it]
  6%|▌         | 12496/225000 [36:02:52<472:30:31,  8.00s/it]
  6%|▌         | 12497/225000 [36:02:57<421:11:53,  7.14s/it]
  6%|▌         | 12498/225000 [36:03:11<544:12:38,  9.22s/it]
  6%|▌         | 12499/225000 [36:03:17<489:01:04,  8.28s/it]
  6%|▌         | 12500/225000 [36:03:23<451:41:59,  7.65s/it]
                                                             
{'loss': '0.7516', 'grad_norm': '0.6278', 'learning_rate': '4.998e-05', 'epoch': '0.05556', 'num_input_tokens_seen': 64442456, 'train_runtime': '1.298e+05', 'train_tokens_per_second': '496.5'}

  6%|▌         | 12500/225000 [36:03:23<451:41:59,  7.65s/it]
  6%|▌         | 12501/225000 [36:03:27<376:24:05,  6.38s/it]
  6%|▌         | 12502/225000 [36:03:36<432:46:18,  7.33s/it]
  6%|▌         | 12503/225000 [36:03:45<454:02:43,  7.69s/it]
  6%|▌         | 12504/225000 [36:03:49<395:11:49,  6.70s/it]
  6%|▌         | 12505/225000 [36:03:54<362:48:01,  6.15s/it]
  6%|▌         | 12506/225000 [36:04:03<411:42:06,  6.97s/it]
  6%|▌         | 12507/225000 [36:04:11<430:57:56,  7.30s/it]
  6%|▌         | 12508/225000 [36:04:16<389:37:56,  6.60s/it]
  6%|▌         | 12509/225000 [36:04:22<379:29:48,  6.43s/it]
  6%|▌         | 12510/225000 [36:04:28<366:58:15,  6.22s/it]
  6%|▌         | 12511/225000 [36:04:32<335:59:45,  5.69s/it]
  6%|▌         | 12512/225000 [36:04:45<455:14:38,  7.71s/it]
  6%|▌         | 12513/225000 [36:04:49<393:47:39,  6.67s/it]
  6%|▌         | 12514/225000 [36:04:55<376:25:33,  6.38s/it]
  6%|▌         | 12515/225000 [36:05:00<366:58:19,  6.22s/it]
  6%|▌         | 12516/225000 [36:05:12<461:09:20,  7.81s/it]
  6%|▌         | 12517/225000 [36:05:20<465:43:01,  7.89s/it]
  6%|▌         | 12518/225000 [36:05:25<413:32:18,  7.01s/it]
  6%|▌         | 12519/225000 [36:05:36<489:49:14,  8.30s/it]
  6%|▌         | 12520/225000 [36:05:43<460:53:09,  7.81s/it]
  6%|▌         | 12521/225000 [36:05:51<461:12:34,  7.81s/it]
  6%|▌         | 12522/225000 [36:05:55<394:15:22,  6.68s/it]
  6%|▌         | 12523/225000 [36:06:06<467:18:48,  7.92s/it]
  6%|▌         | 12524/225000 [36:06:13<452:35:36,  7.67s/it]
  6%|▌         | 12525/225000 [36:06:21<464:30:01,  7.87s/it]
  6%|▌         | 12526/225000 [36:06:26<411:07:04,  6.97s/it]
  6%|▌         | 12527/225000 [36:06:34<425:09:30,  7.20s/it]
  6%|▌         | 12528/225000 [36:06:39<388:54:44,  6.59s/it]
  6%|▌         | 12529/225000 [36:06:44<364:44:33,  6.18s/it]
  6%|▌         | 12530/225000 [36:06:50<355:54:26,  6.03s/it]
  6%|▌         | 12531/225000 [36:06:55<346:33:59,  5.87s/it]
  6%|▌         | 12532/225000 [36:07:10<505:18:01,  8.56s/it]
  6%|▌         | 12533/225000 [36:07:16<458:49:36,  7.77s/it]
  6%|▌         | 12534/225000 [36:07:21<414:18:42,  7.02s/it]
  6%|▌         | 12535/225000 [36:07:27<387:32:30,  6.57s/it]
  6%|▌         | 12536/225000 [36:07:32<364:18:20,  6.17s/it]
  6%|▌         | 12537/225000 [36:07:37<349:59:21,  5.93s/it]
  6%|▌         | 12538/225000 [36:07:42<333:27:13,  5.65s/it]
  6%|▌         | 12539/225000 [36:07:48<337:57:42,  5.73s/it]
  6%|▌         | 12540/225000 [36:07:55<357:35:22,  6.06s/it]
  6%|▌         | 12541/225000 [36:08:07<468:16:01,  7.93s/it]
  6%|▌         | 12542/225000 [36:08:20<556:46:01,  9.43s/it]
  6%|▌         | 12543/225000 [36:08:24<459:51:46,  7.79s/it]
  6%|▌         | 12544/225000 [36:08:33<472:52:51,  8.01s/it]
  6%|▌         | 12545/225000 [36:08:37<403:24:07,  6.84s/it]
  6%|▌         | 12546/225000 [36:08:44<402:20:49,  6.82s/it]
  6%|▌         | 12547/225000 [36:08:52<432:55:27,  7.34s/it]
  6%|▌         | 12548/225000 [36:08:57<380:34:05,  6.45s/it]
  6%|▌         | 12549/225000 [36:09:05<415:07:51,  7.03s/it]
  6%|▌         | 12550/225000 [36:09:10<381:11:26,  6.46s/it]
                                                             
{'loss': '0.7717', 'grad_norm': '0.5627', 'learning_rate': '4.998e-05', 'epoch': '0.05578', 'num_input_tokens_seen': 64608248, 'train_runtime': '1.302e+05', 'train_tokens_per_second': '496.4'}

  6%|▌         | 12550/225000 [36:09:10<381:11:26,  6.46s/it]
  6%|▌         | 12551/225000 [36:09:16<365:02:45,  6.19s/it]
  6%|▌         | 12552/225000 [36:09:29<483:31:46,  8.19s/it]
  6%|▌         | 12553/225000 [36:09:34<432:51:24,  7.33s/it]
  6%|▌         | 12554/225000 [36:09:39<397:09:09,  6.73s/it]
  6%|▌         | 12555/225000 [36:09:45<374:53:37,  6.35s/it]
  6%|▌         | 12556/225000 [36:09:50<354:27:54,  6.01s/it]
  6%|▌         | 12557/225000 [36:09:55<344:22:47,  5.84s/it]
  6%|▌         | 12558/225000 [36:10:01<335:25:49,  5.68s/it]
  6%|▌         | 12559/225000 [36:10:08<361:01:18,  6.12s/it]
  6%|▌         | 12560/225000 [36:10:21<492:31:46,  8.35s/it]
  6%|▌         | 12561/225000 [36:10:26<418:25:44,  7.09s/it]
  6%|▌         | 12562/225000 [36:10:34<436:15:58,  7.39s/it]
  6%|▌         | 12563/225000 [36:10:44<489:35:37,  8.30s/it]
  6%|▌         | 12564/225000 [36:10:50<451:15:57,  7.65s/it]
  6%|▌         | 12565/225000 [36:10:58<461:27:12,  7.82s/it]
  6%|▌         | 12566/225000 [36:11:03<397:51:10,  6.74s/it]
  6%|▌         | 12567/225000 [36:11:08<368:04:49,  6.24s/it]
  6%|▌         | 12568/225000 [36:11:12<337:31:51,  5.72s/it]
  6%|▌         | 12569/225000 [36:11:23<432:17:32,  7.33s/it]
  6%|▌         | 12570/225000 [36:11:29<406:05:55,  6.88s/it]
  6%|▌         | 12571/225000 [36:11:40<469:53:57,  7.96s/it]
  6%|▌         | 12572/225000 [36:11:44<412:45:54,  7.00s/it]
  6%|▌         | 12573/225000 [36:11:48<361:25:17,  6.13s/it]
  6%|▌         | 12574/225000 [36:11:59<446:13:31,  7.56s/it]
  6%|▌         | 12575/225000 [36:12:04<385:20:01,  6.53s/it]
  6%|▌         | 12576/225000 [36:12:09<362:41:27,  6.15s/it]
  6%|▌         | 12577/225000 [36:12:17<407:23:20,  6.90s/it]
  6%|▌         | 12578/225000 [36:12:22<360:29:10,  6.11s/it]
  6%|▌         | 12579/225000 [36:12:31<421:52:43,  7.15s/it]
  6%|▌         | 12580/225000 [36:12:40<450:35:50,  7.64s/it]
  6%|▌         | 12581/225000 [36:12:47<446:20:52,  7.56s/it]
  6%|▌         | 12582/225000 [36:12:52<388:09:40,  6.58s/it]
  6%|▌         | 12583/225000 [36:13:00<423:22:09,  7.18s/it]
  6%|▌         | 12584/225000 [36:13:09<455:30:46,  7.72s/it]
  6%|▌         | 12585/225000 [36:13:14<401:47:50,  6.81s/it]
  6%|▌         | 12586/225000 [36:13:19<370:42:36,  6.28s/it]
  6%|▌         | 12587/225000 [36:13:24<339:20:07,  5.75s/it]
  6%|▌         | 12588/225000 [36:13:37<481:32:48,  8.16s/it]
  6%|▌         | 12589/225000 [36:13:41<399:37:50,  6.77s/it]
  6%|▌         | 12590/225000 [36:13:46<376:06:57,  6.37s/it]
  6%|▌         | 12591/225000 [36:13:51<349:52:05,  5.93s/it]
  6%|▌         | 12592/225000 [36:13:56<330:46:23,  5.61s/it]
  6%|▌         | 12593/225000 [36:14:01<321:10:30,  5.44s/it]
  6%|▌         | 12594/225000 [36:14:14<451:27:41,  7.65s/it]
  6%|▌         | 12595/225000 [36:14:20<429:08:24,  7.27s/it]
  6%|▌         | 12596/225000 [36:14:30<465:38:04,  7.89s/it]
  6%|▌         | 12597/225000 [36:14:36<442:37:00,  7.50s/it]
  6%|▌         | 12598/225000 [36:14:42<413:32:01,  7.01s/it]
  6%|▌         | 12599/225000 [36:14:50<430:12:46,  7.29s/it]
  6%|▌         | 12600/225000 [36:14:56<413:42:02,  7.01s/it]
                                                             
{'loss': '0.7658', 'grad_norm': '0.7499', 'learning_rate': '4.998e-05', 'epoch': '0.056', 'num_input_tokens_seen': 64770856, 'train_runtime': '1.305e+05', 'train_tokens_per_second': '496.3'}

  6%|▌         | 12600/225000 [36:14:56<413:42:02,  7.01s/it]
  6%|▌         | 12601/225000 [36:15:04<426:48:51,  7.23s/it]
  6%|▌         | 12602/225000 [36:15:09<389:45:01,  6.61s/it]
  6%|▌         | 12603/225000 [36:15:14<363:01:47,  6.15s/it]
  6%|▌         | 12604/225000 [36:15:25<446:04:42,  7.56s/it]
  6%|▌         | 12605/225000 [36:15:33<445:33:09,  7.55s/it]
  6%|▌         | 12606/225000 [36:15:38<402:26:36,  6.82s/it]
  6%|▌         | 12607/225000 [36:15:45<409:20:22,  6.94s/it]
  6%|▌         | 12608/225000 [36:15:50<371:11:29,  6.29s/it]
  6%|▌         | 12609/225000 [36:15:56<369:40:21,  6.27s/it]
  6%|▌         | 12610/225000 [36:16:03<385:51:27,  6.54s/it]
  6%|▌         | 12611/225000 [36:16:08<352:16:26,  5.97s/it]
  6%|▌         | 12612/225000 [36:16:13<342:05:16,  5.80s/it]
  6%|▌         | 12613/225000 [36:16:19<331:57:39,  5.63s/it]
  6%|▌         | 12614/225000 [36:16:24<331:55:25,  5.63s/it]
  6%|▌         | 12615/225000 [36:16:34<410:36:00,  6.96s/it]
  6%|▌         | 12616/225000 [36:16:42<418:17:29,  7.09s/it]
  6%|▌         | 12617/225000 [36:16:46<368:31:11,  6.25s/it]
  6%|▌         | 12618/225000 [36:16:52<359:32:41,  6.09s/it]
  6%|▌         | 12619/225000 [36:17:00<402:55:34,  6.83s/it]
  6%|▌         | 12620/225000 [36:17:04<358:22:46,  6.07s/it]
  6%|▌         | 12621/225000 [36:17:13<401:53:21,  6.81s/it]
  6%|▌         | 12622/225000 [36:17:20<400:27:25,  6.79s/it]
  6%|▌         | 12623/225000 [36:17:24<354:59:14,  6.02s/it]
  6%|▌         | 12624/225000 [36:17:29<342:00:10,  5.80s/it]
  6%|▌         | 12625/225000 [36:17:39<418:35:19,  7.10s/it]
  6%|▌         | 12626/225000 [36:17:43<365:50:23,  6.20s/it]
  6%|▌         | 12627/225000 [36:17:52<399:30:28,  6.77s/it]
  6%|▌         | 12628/225000 [36:17:58<395:17:59,  6.70s/it]
  6%|▌         | 12629/225000 [36:18:06<414:42:27,  7.03s/it]
  6%|▌         | 12630/225000 [36:18:12<396:17:16,  6.72s/it]
  6%|▌         | 12631/225000 [36:18:20<414:09:37,  7.02s/it]
  6%|▌         | 12632/225000 [36:18:25<390:01:57,  6.61s/it]
  6%|▌         | 12633/225000 [36:18:29<344:55:15,  5.85s/it]
  6%|▌         | 12634/225000 [36:18:39<416:57:54,  7.07s/it]
  6%|▌         | 12635/225000 [36:18:46<406:14:30,  6.89s/it]
  6%|▌         | 12636/225000 [36:18:55<445:54:47,  7.56s/it]
  6%|▌         | 12637/225000 [36:18:59<379:25:14,  6.43s/it]
  6%|▌         | 12638/225000 [36:19:05<385:02:54,  6.53s/it]
  6%|▌         | 12639/225000 [36:19:10<351:54:43,  5.97s/it]
  6%|▌         | 12640/225000 [36:19:18<391:29:35,  6.64s/it]
  6%|▌         | 12641/225000 [36:19:22<345:33:44,  5.86s/it]
  6%|▌         | 12642/225000 [36:19:26<311:57:19,  5.29s/it]
  6%|▌         | 12643/225000 [36:19:39<440:56:52,  7.48s/it]
  6%|▌         | 12644/225000 [36:19:44<392:09:40,  6.65s/it]
  6%|▌         | 12645/225000 [36:19:55<468:13:34,  7.94s/it]
  6%|▌         | 12646/225000 [36:20:02<466:06:30,  7.90s/it]
  6%|▌         | 12647/225000 [36:20:07<409:04:31,  6.94s/it]
  6%|▌         | 12648/225000 [36:20:12<373:55:53,  6.34s/it]
  6%|▌         | 12649/225000 [36:20:20<408:30:05,  6.93s/it]
  6%|▌         | 12650/225000 [36:20:26<389:23:53,  6.60s/it]
                                                             
{'loss': '0.7362', 'grad_norm': '0.7259', 'learning_rate': '4.998e-05', 'epoch': '0.05622', 'num_input_tokens_seen': 64927752, 'train_runtime': '1.308e+05', 'train_tokens_per_second': '496.3'}

  6%|▌         | 12650/225000 [36:20:26<389:23:53,  6.60s/it]
  6%|▌         | 12651/225000 [36:20:31<356:44:58,  6.05s/it]
  6%|▌         | 12652/225000 [36:20:36<332:05:14,  5.63s/it]
  6%|▌         | 12653/225000 [36:20:49<463:37:30,  7.86s/it]
  6%|▌         | 12654/225000 [36:20:55<444:50:11,  7.54s/it]
  6%|▌         | 12655/225000 [36:21:01<406:51:45,  6.90s/it]
  6%|▌         | 12656/225000 [36:21:11<471:18:19,  7.99s/it]
  6%|▌         | 12657/225000 [36:21:16<413:06:36,  7.00s/it]
  6%|▌         | 12658/225000 [36:21:21<377:53:24,  6.41s/it]
  6%|▌         | 12659/225000 [36:21:27<366:30:02,  6.21s/it]
  6%|▌         | 12660/225000 [36:21:33<372:03:45,  6.31s/it]
  6%|▌         | 12661/225000 [36:21:46<481:29:05,  8.16s/it]
  6%|▌         | 12662/225000 [36:21:54<473:31:42,  8.03s/it]
  6%|▌         | 12663/225000 [36:21:57<401:42:07,  6.81s/it]
  6%|▌         | 12664/225000 [36:22:03<375:33:00,  6.37s/it]
  6%|▌         | 12665/225000 [36:22:08<354:04:23,  6.00s/it]
  6%|▌         | 12666/225000 [36:22:13<342:32:05,  5.81s/it]
  6%|▌         | 12667/225000 [36:22:24<430:40:52,  7.30s/it]
  6%|▌         | 12668/225000 [36:22:31<427:59:05,  7.26s/it]
  6%|▌         | 12669/225000 [36:22:37<399:02:31,  6.77s/it]
  6%|▌         | 12670/225000 [36:22:43<379:16:29,  6.43s/it]
  6%|▌         | 12671/225000 [36:22:52<434:47:29,  7.37s/it]
  6%|▌         | 12672/225000 [36:22:56<372:16:10,  6.31s/it]
  6%|▌         | 12673/225000 [36:23:04<410:13:43,  6.96s/it]
  6%|▌         | 12674/225000 [36:23:11<401:27:42,  6.81s/it]
  6%|▌         | 12675/225000 [36:23:17<386:50:13,  6.56s/it]
  6%|▌         | 12676/225000 [36:23:22<364:49:39,  6.19s/it]
  6%|▌         | 12677/225000 [36:23:27<348:07:07,  5.90s/it]
  6%|▌         | 12678/225000 [36:23:42<496:58:49,  8.43s/it]
  6%|▌         | 12679/225000 [36:23:50<502:06:58,  8.51s/it]
  6%|▌         | 12680/225000 [36:23:55<433:46:49,  7.35s/it]
  6%|▌         | 12681/225000 [36:24:03<441:12:59,  7.48s/it]
  6%|▌         | 12682/225000 [36:24:11<460:20:26,  7.81s/it]
  6%|▌         | 12683/225000 [36:24:16<402:05:39,  6.82s/it]
  6%|▌         | 12684/225000 [36:24:23<412:11:56,  6.99s/it]
  6%|▌         | 12685/225000 [36:24:30<397:54:13,  6.75s/it]
  6%|▌         | 12686/225000 [36:24:35<382:23:03,  6.48s/it]
  6%|▌         | 12687/225000 [36:24:45<430:41:22,  7.30s/it]
  6%|▌         | 12688/225000 [36:24:49<372:03:21,  6.31s/it]
  6%|▌         | 12689/225000 [36:24:54<354:30:12,  6.01s/it]
  6%|▌         | 12690/225000 [36:25:00<349:13:50,  5.92s/it]
  6%|▌         | 12691/225000 [36:25:05<341:45:23,  5.79s/it]
  6%|▌         | 12692/225000 [36:25:11<342:53:27,  5.81s/it]
  6%|▌         | 12693/225000 [36:25:16<330:27:38,  5.60s/it]
  6%|▌         | 12694/225000 [36:25:22<330:46:59,  5.61s/it]
  6%|▌         | 12695/225000 [36:25:27<332:37:10,  5.64s/it]
  6%|▌         | 12696/225000 [36:25:33<339:38:34,  5.76s/it]
  6%|▌         | 12697/225000 [36:25:39<339:53:21,  5.76s/it]
  6%|▌         | 12698/225000 [36:25:46<365:11:02,  6.19s/it]
  6%|▌         | 12699/225000 [36:26:02<534:31:53,  9.06s/it]
  6%|▌         | 12700/225000 [36:26:06<441:41:06,  7.49s/it]
                                                             
{'loss': '0.7515', 'grad_norm': '0.6935', 'learning_rate': '4.998e-05', 'epoch': '0.05644', 'num_input_tokens_seen': 65089168, 'train_runtime': '1.312e+05', 'train_tokens_per_second': '496.2'}

  6%|▌         | 12700/225000 [36:26:06<441:41:06,  7.49s/it]
  6%|▌         | 12701/225000 [36:26:15<463:21:18,  7.86s/it]
  6%|▌         | 12702/225000 [36:26:20<423:29:48,  7.18s/it]
  6%|▌         | 12703/225000 [36:26:29<443:07:40,  7.51s/it]
  6%|▌         | 12704/225000 [36:26:33<381:36:35,  6.47s/it]
  6%|▌         | 12705/225000 [36:26:41<413:51:40,  7.02s/it]
  6%|▌         | 12706/225000 [36:26:51<462:10:36,  7.84s/it]
  6%|▌         | 12707/225000 [36:26:58<449:47:05,  7.63s/it]
  6%|▌         | 12708/225000 [36:27:08<490:08:53,  8.31s/it]
  6%|▌         | 12709/225000 [36:27:13<434:28:09,  7.37s/it]
  6%|▌         | 12710/225000 [36:27:18<389:34:14,  6.61s/it]
  6%|▌         | 12711/225000 [36:27:23<369:26:03,  6.26s/it]
  6%|▌         | 12712/225000 [36:27:29<354:46:49,  6.02s/it]
  6%|▌         | 12713/225000 [36:27:33<334:19:55,  5.67s/it]
  6%|▌         | 12714/225000 [36:27:43<408:37:45,  6.93s/it]
  6%|▌         | 12715/225000 [36:27:48<375:40:41,  6.37s/it]
  6%|▌         | 12716/225000 [36:27:54<357:49:10,  6.07s/it]
  6%|▌         | 12717/225000 [36:27:59<343:57:20,  5.83s/it]
  6%|▌         | 12718/225000 [36:28:05<342:05:12,  5.80s/it]
  6%|▌         | 12719/225000 [36:28:11<341:23:33,  5.79s/it]
  6%|▌         | 12720/225000 [36:28:18<364:00:29,  6.17s/it]
  6%|▌         | 12721/225000 [36:28:28<432:52:47,  7.34s/it]
  6%|▌         | 12722/225000 [36:28:38<486:47:39,  8.26s/it]
  6%|▌         | 12723/225000 [36:28:42<408:48:34,  6.93s/it]
  6%|▌         | 12724/225000 [36:28:46<364:24:16,  6.18s/it]
  6%|▌         | 12725/225000 [36:28:57<445:09:18,  7.55s/it]
  6%|▌         | 12726/225000 [36:29:04<431:31:43,  7.32s/it]
  6%|▌         | 12727/225000 [36:29:13<457:26:09,  7.76s/it]
  6%|▌         | 12728/225000 [36:29:18<414:35:53,  7.03s/it]
  6%|▌         | 12729/225000 [36:29:26<425:51:35,  7.22s/it]
  6%|▌         | 12730/225000 [36:29:30<375:56:50,  6.38s/it]
  6%|▌         | 12731/225000 [36:29:38<403:18:53,  6.84s/it]
  6%|▌         | 12732/225000 [36:29:46<418:37:15,  7.10s/it]
  6%|▌         | 12733/225000 [36:29:50<370:02:09,  6.28s/it]
  6%|▌         | 12734/225000 [36:29:58<394:21:34,  6.69s/it]
  6%|▌         | 12735/225000 [36:30:05<402:22:17,  6.82s/it]
  6%|▌         | 12736/225000 [36:30:14<442:29:51,  7.50s/it]
  6%|▌         | 12737/225000 [36:30:18<381:37:08,  6.47s/it]
  6%|▌         | 12738/225000 [36:30:23<347:28:35,  5.89s/it]
  6%|▌         | 12739/225000 [36:30:33<431:45:21,  7.32s/it]
  6%|▌         | 12740/225000 [36:30:37<375:04:32,  6.36s/it]
  6%|▌         | 12741/225000 [36:30:46<415:55:04,  7.05s/it]
  6%|▌         | 12742/225000 [36:30:51<373:23:01,  6.33s/it]
  6%|▌         | 12743/225000 [36:30:59<408:04:18,  6.92s/it]
  6%|▌         | 12744/225000 [36:31:03<349:38:56,  5.93s/it]
  6%|▌         | 12745/225000 [36:31:12<414:30:24,  7.03s/it]
  6%|▌         | 12746/225000 [36:31:17<377:39:42,  6.41s/it]
  6%|▌         | 12747/225000 [36:31:30<486:22:32,  8.25s/it]
  6%|▌         | 12748/225000 [36:31:35<429:46:26,  7.29s/it]
  6%|▌         | 12749/225000 [36:31:40<397:11:20,  6.74s/it]
  6%|▌         | 12750/225000 [36:31:46<383:10:53,  6.50s/it]
                                                             
{'loss': '0.7594', 'grad_norm': '0.715', 'learning_rate': '4.998e-05', 'epoch': '0.05667', 'num_input_tokens_seen': 65251264, 'train_runtime': '1.315e+05', 'train_tokens_per_second': '496.2'}

  6%|▌         | 12750/225000 [36:31:46<383:10:53,  6.50s/it]
  6%|▌         | 12751/225000 [36:31:52<365:13:51,  6.19s/it]
  6%|▌         | 12752/225000 [36:31:57<350:25:43,  5.94s/it]
  6%|▌         | 12753/225000 [36:32:02<340:27:21,  5.77s/it]
  6%|▌         | 12754/225000 [36:32:10<370:28:34,  6.28s/it]
  6%|▌         | 12755/225000 [36:32:22<482:11:14,  8.18s/it]
  6%|▌         | 12756/225000 [36:32:27<413:20:03,  7.01s/it]
  6%|▌         | 12757/225000 [36:32:32<392:06:21,  6.65s/it]
  6%|▌         | 12758/225000 [36:32:38<377:41:50,  6.41s/it]
  6%|▌         | 12759/225000 [36:32:44<364:56:21,  6.19s/it]
  6%|▌         | 12760/225000 [36:32:53<409:28:03,  6.95s/it]
  6%|▌         | 12761/225000 [36:32:57<369:13:01,  6.26s/it]
  6%|▌         | 12762/225000 [36:33:02<345:24:07,  5.86s/it]
  6%|▌         | 12763/225000 [36:33:17<508:38:48,  8.63s/it]
  6%|▌         | 12764/225000 [36:33:24<475:00:21,  8.06s/it]
  6%|▌         | 12765/225000 [36:33:29<421:18:16,  7.15s/it]
  6%|▌         | 12766/225000 [36:33:35<398:11:43,  6.75s/it]
  6%|▌         | 12767/225000 [36:33:46<479:27:12,  8.13s/it]
  6%|▌         | 12768/225000 [36:33:51<416:59:32,  7.07s/it]
  6%|▌         | 12769/225000 [36:33:56<380:15:20,  6.45s/it]
  6%|▌         | 12770/225000 [36:34:01<363:47:35,  6.17s/it]
  6%|▌         | 12771/225000 [36:34:07<346:37:15,  5.88s/it]
  6%|▌         | 12772/225000 [36:34:14<369:51:56,  6.27s/it]
  6%|▌         | 12773/225000 [36:34:27<500:27:06,  8.49s/it]
  6%|▌         | 12774/225000 [36:34:37<519:38:50,  8.81s/it]
  6%|▌         | 12775/225000 [36:34:43<461:01:29,  7.82s/it]
  6%|▌         | 12776/225000 [36:34:48<413:58:23,  7.02s/it]
  6%|▌         | 12777/225000 [36:34:54<394:50:56,  6.70s/it]
  6%|▌         | 12778/225000 [36:34:59<368:51:31,  6.26s/it]
  6%|▌         | 12779/225000 [36:35:04<353:14:08,  5.99s/it]
  6%|▌         | 12780/225000 [36:35:15<438:56:34,  7.45s/it]
  6%|▌         | 12781/225000 [36:35:22<430:18:41,  7.30s/it]
  6%|▌         | 12782/225000 [36:35:28<400:56:23,  6.80s/it]
  6%|▌         | 12783/225000 [36:35:34<400:34:48,  6.80s/it]
  6%|▌         | 12784/225000 [36:35:42<419:39:49,  7.12s/it]
  6%|▌         | 12785/225000 [36:35:48<395:13:49,  6.70s/it]
  6%|▌         | 12786/225000 [36:35:55<405:01:01,  6.87s/it]
  6%|▌         | 12787/225000 [36:36:01<389:01:38,  6.60s/it]
  6%|▌         | 12788/225000 [36:36:07<366:58:29,  6.23s/it]
  6%|▌         | 12789/225000 [36:36:14<377:58:46,  6.41s/it]
  6%|▌         | 12790/225000 [36:36:18<342:46:46,  5.82s/it]
  6%|▌         | 12791/225000 [36:36:23<327:51:44,  5.56s/it]
  6%|▌         | 12792/225000 [36:36:31<372:56:07,  6.33s/it]
  6%|▌         | 12793/225000 [36:36:45<500:15:40,  8.49s/it]
  6%|▌         | 12794/225000 [36:36:48<412:25:23,  7.00s/it]
  6%|▌         | 12795/225000 [36:36:53<382:11:50,  6.48s/it]
  6%|▌         | 12796/225000 [36:36:59<363:55:45,  6.17s/it]
  6%|▌         | 12797/225000 [36:37:04<341:50:40,  5.80s/it]
  6%|▌         | 12798/225000 [36:37:09<330:34:36,  5.61s/it]
  6%|▌         | 12799/225000 [36:37:14<316:52:16,  5.38s/it]
  6%|▌         | 12800/225000 [36:37:27<459:16:21,  7.79s/it]
                                                             
{'loss': '0.7563', 'grad_norm': '0.5688', 'learning_rate': '4.998e-05', 'epoch': '0.05689', 'num_input_tokens_seen': 65409952, 'train_runtime': '1.318e+05', 'train_tokens_per_second': '496.1'}

  6%|▌         | 12800/225000 [36:37:27<459:16:21,  7.79s/it]
  6%|▌         | 12801/225000 [36:37:34<437:38:11,  7.42s/it]
  6%|▌         | 12802/225000 [36:37:38<375:42:04,  6.37s/it]
  6%|▌         | 12803/225000 [36:37:47<432:57:16,  7.35s/it]
  6%|▌         | 12804/225000 [36:37:59<510:34:48,  8.66s/it]
  6%|▌         | 12805/225000 [36:38:03<434:30:01,  7.37s/it]
  6%|▌         | 12806/225000 [36:38:08<380:03:42,  6.45s/it]
  6%|▌         | 12807/225000 [36:38:16<415:28:21,  7.05s/it]
  6%|▌         | 12808/225000 [36:38:27<475:35:28,  8.07s/it]
  6%|▌         | 12809/225000 [36:38:33<438:46:56,  7.44s/it]
  6%|▌         | 12810/225000 [36:38:44<503:09:39,  8.54s/it]
  6%|▌         | 12811/225000 [36:38:48<425:12:13,  7.21s/it]
  6%|▌         | 12812/225000 [36:38:55<427:43:42,  7.26s/it]
  6%|▌         | 12813/225000 [36:39:01<402:16:01,  6.82s/it]
  6%|▌         | 12814/225000 [36:39:07<384:06:11,  6.52s/it]
  6%|▌         | 12815/225000 [36:39:12<357:49:21,  6.07s/it]
  6%|▌         | 12816/225000 [36:39:20<403:07:17,  6.84s/it]
  6%|▌         | 12817/225000 [36:39:28<417:56:00,  7.09s/it]
  6%|▌         | 12818/225000 [36:39:33<377:31:07,  6.41s/it]
  6%|▌         | 12819/225000 [36:39:37<341:09:25,  5.79s/it]
  6%|▌         | 12820/225000 [36:39:47<411:31:44,  6.98s/it]
  6%|▌         | 12821/225000 [36:40:03<571:28:17,  9.70s/it]
  6%|▌         | 12822/225000 [36:40:07<467:45:32,  7.94s/it]
  6%|▌         | 12823/225000 [36:40:12<419:39:03,  7.12s/it]
  6%|▌         | 12824/225000 [36:40:17<377:47:23,  6.41s/it]
  6%|▌         | 12825/225000 [36:40:28<464:39:47,  7.88s/it]
  6%|▌         | 12826/225000 [36:40:33<409:50:02,  6.95s/it]
  6%|▌         | 12827/225000 [36:40:37<361:51:47,  6.14s/it]
  6%|▌         | 12828/225000 [36:40:46<402:46:44,  6.83s/it]
  6%|▌         | 12829/225000 [36:40:55<442:43:02,  7.51s/it]
  6%|▌         | 12830/225000 [36:41:02<445:01:56,  7.55s/it]
  6%|▌         | 12831/225000 [36:41:07<398:50:01,  6.77s/it]
  6%|▌         | 12832/225000 [36:41:12<357:47:18,  6.07s/it]
  6%|▌         | 12833/225000 [36:41:16<322:57:04,  5.48s/it]
  6%|▌         | 12834/225000 [36:41:28<445:20:20,  7.56s/it]
  6%|▌         | 12835/225000 [36:41:33<387:40:20,  6.58s/it]
  6%|▌         | 12836/225000 [36:41:38<362:14:56,  6.15s/it]
  6%|▌         | 12837/225000 [36:41:46<399:09:35,  6.77s/it]
  6%|▌         | 12838/225000 [36:41:55<432:42:14,  7.34s/it]
  6%|▌         | 12839/225000 [36:42:01<416:09:44,  7.06s/it]
  6%|▌         | 12840/225000 [36:42:05<357:24:18,  6.06s/it]
  6%|▌         | 12841/225000 [36:42:12<373:34:47,  6.34s/it]
  6%|▌         | 12842/225000 [36:42:18<376:28:06,  6.39s/it]
  6%|▌         | 12843/225000 [36:42:25<386:10:42,  6.55s/it]
  6%|▌         | 12844/225000 [36:42:34<428:25:43,  7.27s/it]
  6%|▌         | 12845/225000 [36:42:39<383:29:36,  6.51s/it]
  6%|▌         | 12846/225000 [36:42:44<351:29:19,  5.96s/it]
  6%|▌         | 12847/225000 [36:42:51<382:54:21,  6.50s/it]
  6%|▌         | 12848/225000 [36:43:01<435:27:44,  7.39s/it]
  6%|▌         | 12849/225000 [36:43:06<396:27:11,  6.73s/it]
  6%|▌         | 12850/225000 [36:43:14<413:24:13,  7.02s/it]
                                                             
{'loss': '0.7422', 'grad_norm': '0.6124', 'learning_rate': '4.998e-05', 'epoch': '0.05711', 'num_input_tokens_seen': 65572856, 'train_runtime': '1.322e+05', 'train_tokens_per_second': '496'}

  6%|▌         | 12850/225000 [36:43:14<413:24:13,  7.02s/it]
  6%|▌         | 12851/225000 [36:43:19<381:17:43,  6.47s/it]
  6%|▌         | 12852/225000 [36:43:24<367:10:38,  6.23s/it]
  6%|▌         | 12853/225000 [36:43:31<369:44:54,  6.27s/it]
  6%|▌         | 12854/225000 [36:43:41<435:20:15,  7.39s/it]
  6%|▌         | 12855/225000 [36:43:46<397:42:59,  6.75s/it]
  6%|▌         | 12856/225000 [36:43:50<355:06:12,  6.03s/it]
  6%|▌         | 12857/225000 [36:43:55<324:00:15,  5.50s/it]
  6%|▌         | 12858/225000 [36:44:10<495:10:43,  8.40s/it]
  6%|▌         | 12859/225000 [36:44:15<440:04:43,  7.47s/it]
  6%|▌         | 12860/225000 [36:44:20<401:02:11,  6.81s/it]
  6%|▌         | 12861/225000 [36:44:26<372:25:16,  6.32s/it]
  6%|▌         | 12862/225000 [36:44:30<342:43:50,  5.82s/it]
  6%|▌         | 12863/225000 [36:44:36<336:03:51,  5.70s/it]
  6%|▌         | 12864/225000 [36:44:48<444:36:18,  7.55s/it]
  6%|▌         | 12865/225000 [36:44:53<411:46:26,  6.99s/it]
  6%|▌         | 12866/225000 [36:44:58<367:28:32,  6.24s/it]
  6%|▌         | 12867/225000 [36:45:03<359:11:37,  6.10s/it]
  6%|▌         | 12868/225000 [36:45:09<347:46:29,  5.90s/it]
  6%|▌         | 12869/225000 [36:45:23<487:26:07,  8.27s/it]
  6%|▌         | 12870/225000 [36:45:27<416:58:29,  7.08s/it]
  6%|▌         | 12871/225000 [36:45:31<364:18:20,  6.18s/it]
  6%|▌         | 12872/225000 [36:45:41<431:10:28,  7.32s/it]
  6%|▌         | 12873/225000 [36:45:48<423:05:36,  7.18s/it]
  6%|▌         | 12874/225000 [36:45:54<411:49:42,  6.99s/it]
  6%|▌         | 12875/225000 [36:46:02<418:32:45,  7.10s/it]
  6%|▌         | 12876/225000 [36:46:11<457:18:23,  7.76s/it]
  6%|▌         | 12877/225000 [36:46:19<454:20:27,  7.71s/it]
  6%|▌         | 12878/225000 [36:46:24<415:11:00,  7.05s/it]
  6%|▌         | 12879/225000 [36:46:36<491:55:54,  8.35s/it]
  6%|▌         | 12880/225000 [36:46:41<447:33:39,  7.60s/it]
  6%|▌         | 12881/225000 [36:46:45<380:49:57,  6.46s/it]
  6%|▌         | 12882/225000 [36:46:55<437:28:48,  7.42s/it]
  6%|▌         | 12883/225000 [36:47:02<425:05:55,  7.21s/it]
  6%|▌         | 12884/225000 [36:47:06<381:54:32,  6.48s/it]
  6%|▌         | 12885/225000 [36:47:11<349:54:39,  5.94s/it]
  6%|▌         | 12886/225000 [36:47:16<323:01:34,  5.48s/it]
  6%|▌         | 12887/225000 [36:47:26<413:38:00,  7.02s/it]
  6%|▌         | 12888/225000 [36:47:33<404:28:22,  6.86s/it]
  6%|▌         | 12889/225000 [36:47:39<392:14:22,  6.66s/it]
  6%|▌         | 12890/225000 [36:47:48<442:42:18,  7.51s/it]
  6%|▌         | 12891/225000 [36:47:53<400:42:25,  6.80s/it]
  6%|▌         | 12892/225000 [36:48:07<525:05:14,  8.91s/it]
  6%|▌         | 12893/225000 [36:48:12<456:29:00,  7.75s/it]
  6%|▌         | 12894/225000 [36:48:19<442:31:02,  7.51s/it]
  6%|▌         | 12895/225000 [36:48:23<383:00:15,  6.50s/it]
  6%|▌         | 12896/225000 [36:48:33<436:26:14,  7.41s/it]
  6%|▌         | 12897/225000 [36:48:39<410:42:20,  6.97s/it]
  6%|▌         | 12898/225000 [36:48:47<424:47:14,  7.21s/it]
  6%|▌         | 12899/225000 [36:48:51<378:14:29,  6.42s/it]
  6%|▌         | 12900/225000 [36:48:56<355:41:48,  6.04s/it]
                                                             
{'loss': '0.7421', 'grad_norm': '0.6486', 'learning_rate': '4.998e-05', 'epoch': '0.05733', 'num_input_tokens_seen': 65736160, 'train_runtime': '1.325e+05', 'train_tokens_per_second': '496'}

  6%|▌         | 12900/225000 [36:48:56<355:41:48,  6.04s/it]
  6%|▌         | 12901/225000 [36:49:01<323:07:16,  5.48s/it]
  6%|▌         | 12902/225000 [36:49:14<461:58:27,  7.84s/it]
  6%|▌         | 12903/225000 [36:49:19<419:54:50,  7.13s/it]
  6%|▌         | 12904/225000 [36:49:24<370:40:02,  6.29s/it]
  6%|▌         | 12905/225000 [36:49:36<467:41:13,  7.94s/it]
  6%|▌         | 12906/225000 [36:49:39<387:27:31,  6.58s/it]
  6%|▌         | 12907/225000 [36:49:46<398:25:28,  6.76s/it]
  6%|▌         | 12908/225000 [36:49:53<404:29:23,  6.87s/it]
  6%|▌         | 12909/225000 [36:50:01<415:54:45,  7.06s/it]
  6%|▌         | 12910/225000 [36:50:06<387:53:50,  6.58s/it]
  6%|▌         | 12911/225000 [36:50:11<356:17:36,  6.05s/it]
  6%|▌         | 12912/225000 [36:50:23<465:20:17,  7.90s/it]
  6%|▌         | 12913/225000 [36:50:29<423:32:31,  7.19s/it]
  6%|▌         | 12914/225000 [36:50:34<382:01:45,  6.48s/it]
  6%|▌         | 12915/225000 [36:50:41<400:58:10,  6.81s/it]
  6%|▌         | 12916/225000 [36:50:48<406:26:04,  6.90s/it]
  6%|▌         | 12917/225000 [36:50:56<428:45:46,  7.28s/it]
  6%|▌         | 12918/225000 [36:51:04<440:47:17,  7.48s/it]
  6%|▌         | 12919/225000 [36:51:10<414:05:18,  7.03s/it]
  6%|▌         | 12920/225000 [36:51:17<413:52:40,  7.03s/it]
  6%|▌         | 12921/225000 [36:51:25<429:45:17,  7.30s/it]
  6%|▌         | 12922/225000 [36:51:31<394:55:43,  6.70s/it]
  6%|▌         | 12923/225000 [36:51:35<360:06:34,  6.11s/it]
  6%|▌         | 12924/225000 [36:51:40<328:41:26,  5.58s/it]
  6%|▌         | 12925/225000 [36:51:47<352:45:24,  5.99s/it]
  6%|▌         | 12926/225000 [36:51:56<408:00:55,  6.93s/it]
  6%|▌         | 12927/225000 [36:52:00<359:10:40,  6.10s/it]
  6%|▌         | 12928/225000 [36:52:13<479:05:53,  8.13s/it]
  6%|▌         | 12929/225000 [36:52:18<432:28:34,  7.34s/it]
  6%|▌         | 12930/225000 [36:52:22<366:22:22,  6.22s/it]
  6%|▌         | 12931/225000 [36:52:30<396:35:18,  6.73s/it]
  6%|▌         | 12932/225000 [36:52:34<350:25:42,  5.95s/it]
  6%|▌         | 12933/225000 [36:52:44<429:01:12,  7.28s/it]
  6%|▌         | 12934/225000 [36:52:48<363:55:09,  6.18s/it]
  6%|▌         | 12935/225000 [36:52:55<373:56:51,  6.35s/it]
  6%|▌         | 12936/225000 [36:53:03<410:43:10,  6.97s/it]
  6%|▌         | 12937/225000 [36:53:07<361:10:33,  6.13s/it]
  6%|▌         | 12938/225000 [36:53:12<335:44:55,  5.70s/it]
  6%|▌         | 12939/225000 [36:53:22<404:25:25,  6.87s/it]
  6%|▌         | 12940/225000 [36:53:27<383:20:34,  6.51s/it]
  6%|▌         | 12941/225000 [36:53:33<373:46:56,  6.35s/it]
  6%|▌         | 12942/225000 [36:53:38<340:27:53,  5.78s/it]
  6%|▌         | 12943/225000 [36:53:43<329:08:54,  5.59s/it]
  6%|▌         | 12944/225000 [36:53:48<319:51:48,  5.43s/it]
  6%|▌         | 12945/225000 [36:54:04<500:28:44,  8.50s/it]
  6%|▌         | 12946/225000 [36:54:08<438:27:51,  7.44s/it]
  6%|▌         | 12947/225000 [36:54:14<398:22:39,  6.76s/it]
  6%|▌         | 12948/225000 [36:54:19<369:57:51,  6.28s/it]
  6%|▌         | 12949/225000 [36:54:27<402:29:16,  6.83s/it]
  6%|▌         | 12950/225000 [36:54:33<380:12:00,  6.45s/it]
                                                             
{'loss': '0.735', 'grad_norm': '0.7583', 'learning_rate': '4.998e-05', 'epoch': '0.05756', 'num_input_tokens_seen': 65893184, 'train_runtime': '1.329e+05', 'train_tokens_per_second': '495.9'}

  6%|▌         | 12950/225000 [36:54:33<380:12:00,  6.45s/it]
  6%|▌         | 12951/225000 [36:54:41<422:38:45,  7.18s/it]
  6%|▌         | 12952/225000 [36:54:46<369:29:37,  6.27s/it]
  6%|▌         | 12953/225000 [36:54:51<352:58:19,  5.99s/it]
  6%|▌         | 12954/225000 [36:54:56<342:35:39,  5.82s/it]
  6%|▌         | 12955/225000 [36:55:02<333:29:06,  5.66s/it]
  6%|▌         | 12956/225000 [36:55:07<329:28:32,  5.59s/it]
  6%|▌         | 12957/225000 [36:55:13<331:57:06,  5.64s/it]
  6%|▌         | 12958/225000 [36:55:26<475:10:19,  8.07s/it]
  6%|▌         | 12959/225000 [36:55:34<457:17:43,  7.76s/it]
  6%|▌         | 12960/225000 [36:55:42<460:55:04,  7.83s/it]
  6%|▌         | 12961/225000 [36:55:47<410:47:41,  6.97s/it]
  6%|▌         | 12962/225000 [36:55:54<412:27:22,  7.00s/it]
  6%|▌         | 12963/225000 [36:56:00<398:44:19,  6.77s/it]
  6%|▌         | 12964/225000 [36:56:09<439:09:46,  7.46s/it]
  6%|▌         | 12965/225000 [36:56:12<371:28:38,  6.31s/it]
  6%|▌         | 12966/225000 [36:56:19<370:17:13,  6.29s/it]
  6%|▌         | 12967/225000 [36:56:23<339:11:59,  5.76s/it]
  6%|▌         | 12968/225000 [36:56:35<444:42:09,  7.55s/it]
  6%|▌         | 12969/225000 [36:56:40<393:40:56,  6.68s/it]
  6%|▌         | 12970/225000 [36:56:46<379:22:06,  6.44s/it]
  6%|▌         | 12971/225000 [36:56:51<357:41:26,  6.07s/it]
  6%|▌         | 12972/225000 [36:56:56<342:48:18,  5.82s/it]
  6%|▌         | 12973/225000 [36:57:04<380:44:08,  6.46s/it]
  6%|▌         | 12974/225000 [36:57:10<370:20:08,  6.29s/it]
  6%|▌         | 12975/225000 [36:57:17<390:32:52,  6.63s/it]
  6%|▌         | 12976/225000 [36:57:31<509:43:52,  8.65s/it]
  6%|▌         | 12977/225000 [36:57:34<421:21:10,  7.15s/it]
  6%|▌         | 12978/225000 [36:57:43<442:15:09,  7.51s/it]
  6%|▌         | 12979/225000 [36:57:47<393:17:25,  6.68s/it]
  6%|▌         | 12980/225000 [36:57:54<399:08:01,  6.78s/it]
  6%|▌         | 12981/225000 [36:58:02<420:46:27,  7.14s/it]
  6%|▌         | 12982/225000 [36:58:07<383:46:47,  6.52s/it]
  6%|▌         | 12983/225000 [36:58:20<486:36:46,  8.26s/it]
  6%|▌         | 12984/225000 [36:58:25<430:22:30,  7.31s/it]
  6%|▌         | 12985/225000 [36:58:30<389:45:41,  6.62s/it]
  6%|▌         | 12986/225000 [36:58:36<387:21:50,  6.58s/it]
  6%|▌         | 12987/225000 [36:58:41<358:33:29,  6.09s/it]
  6%|▌         | 12988/225000 [36:58:47<350:08:51,  5.95s/it]
  6%|▌         | 12989/225000 [36:59:00<484:20:20,  8.22s/it]
  6%|▌         | 12990/225000 [36:59:05<419:03:43,  7.12s/it]
  6%|▌         | 12991/225000 [36:59:09<372:12:16,  6.32s/it]
  6%|▌         | 12992/225000 [36:59:14<344:20:51,  5.85s/it]
  6%|▌         | 12993/225000 [36:59:21<353:18:43,  6.00s/it]
  6%|▌         | 12994/225000 [36:59:33<470:17:46,  7.99s/it]
  6%|▌         | 12995/225000 [36:59:39<435:11:23,  7.39s/it]
  6%|▌         | 12996/225000 [36:59:47<436:27:18,  7.41s/it]
  6%|▌         | 12997/225000 [36:59:51<386:12:15,  6.56s/it]
  6%|▌         | 12998/225000 [37:00:00<418:14:46,  7.10s/it]
  6%|▌         | 12999/225000 [37:00:04<376:46:38,  6.40s/it]
  6%|▌         | 13000/225000 [37:00:10<360:12:19,  6.12s/it]
                                                             
{'loss': '0.7576', 'grad_norm': '0.5956', 'learning_rate': '4.998e-05', 'epoch': '0.05778', 'num_input_tokens_seen': 66055176, 'train_runtime': '1.332e+05', 'train_tokens_per_second': '495.9'}

  6%|▌         | 13000/225000 [37:00:10<360:12:19,  6.12s/it]
  6%|▌         | 13001/225000 [37:00:14<331:07:54,  5.62s/it]
  6%|▌         | 13002/225000 [37:00:25<429:08:05,  7.29s/it]
  6%|▌         | 13003/225000 [37:00:31<406:31:31,  6.90s/it]
  6%|▌         | 13004/225000 [37:00:36<362:36:10,  6.16s/it]
  6%|▌         | 13005/225000 [37:00:44<406:20:45,  6.90s/it]
  6%|▌         | 13006/225000 [37:00:52<416:05:43,  7.07s/it]
  6%|▌         | 13007/225000 [37:00:56<361:24:20,  6.14s/it]
  6%|▌         | 13008/225000 [37:01:05<416:01:19,  7.06s/it]
  6%|▌         | 13009/225000 [37:01:11<387:34:43,  6.58s/it]
  6%|▌         | 13010/225000 [37:01:15<349:23:09,  5.93s/it]
  6%|▌         | 13011/225000 [37:01:28<471:01:10,  8.00s/it]
  6%|▌         | 13012/225000 [37:01:32<407:25:38,  6.92s/it]
  6%|▌         | 13013/225000 [37:01:40<418:17:59,  7.10s/it]
  6%|▌         | 13014/225000 [37:01:51<483:13:01,  8.21s/it]
  6%|▌         | 13015/225000 [37:01:56<436:38:49,  7.42s/it]
  6%|▌         | 13016/225000 [37:02:02<408:48:00,  6.94s/it]
  6%|▌         | 13017/225000 [37:02:07<377:44:50,  6.42s/it]
  6%|▌         | 13018/225000 [37:02:13<369:03:00,  6.27s/it]
  6%|▌         | 13019/225000 [37:02:18<353:00:03,  5.99s/it]
  6%|▌         | 13020/225000 [37:02:27<406:27:37,  6.90s/it]
  6%|▌         | 13021/225000 [37:02:38<475:11:00,  8.07s/it]
  6%|▌         | 13022/225000 [37:02:42<405:20:21,  6.88s/it]
  6%|▌         | 13023/225000 [37:02:50<427:30:28,  7.26s/it]
  6%|▌         | 13024/225000 [37:02:56<391:54:48,  6.66s/it]
  6%|▌         | 13025/225000 [37:03:01<364:14:38,  6.19s/it]
  6%|▌         | 13026/225000 [37:03:06<354:13:13,  6.02s/it]
  6%|▌         | 13027/225000 [37:03:11<337:26:19,  5.73s/it]
  6%|▌         | 13028/225000 [37:03:17<336:23:10,  5.71s/it]
  6%|▌         | 13029/225000 [37:03:22<325:24:53,  5.53s/it]
  6%|▌         | 13030/225000 [37:03:28<323:06:42,  5.49s/it]
  6%|▌         | 13031/225000 [37:03:33<325:55:38,  5.54s/it]
  6%|▌         | 13032/225000 [37:03:39<326:14:34,  5.54s/it]
  6%|▌         | 13033/225000 [37:03:44<327:01:56,  5.55s/it]
  6%|▌         | 13034/225000 [37:03:53<374:47:41,  6.37s/it]
  6%|▌         | 13035/225000 [37:04:02<432:56:01,  7.35s/it]
  6%|▌         | 13036/225000 [37:04:08<399:02:57,  6.78s/it]
  6%|▌         | 13037/225000 [37:04:16<425:15:15,  7.22s/it]
  6%|▌         | 13038/225000 [37:04:21<389:26:07,  6.61s/it]
  6%|▌         | 13039/225000 [37:04:34<497:34:52,  8.45s/it]
  6%|▌         | 13040/225000 [37:04:39<429:07:12,  7.29s/it]
  6%|▌         | 13041/225000 [37:04:44<393:31:36,  6.68s/it]
  6%|▌         | 13042/225000 [37:04:51<399:30:39,  6.79s/it]
  6%|▌         | 13043/225000 [37:05:00<433:46:39,  7.37s/it]
  6%|▌         | 13044/225000 [37:05:04<379:41:10,  6.45s/it]
  6%|▌         | 13045/225000 [37:05:15<469:39:33,  7.98s/it]
  6%|▌         | 13046/225000 [37:05:21<428:00:17,  7.27s/it]
  6%|▌         | 13047/225000 [37:05:25<373:23:25,  6.34s/it]
  6%|▌         | 13048/225000 [37:05:36<453:09:36,  7.70s/it]
  6%|▌         | 13049/225000 [37:05:44<452:58:42,  7.69s/it]
  6%|▌         | 13050/225000 [37:05:48<390:51:04,  6.64s/it]
                                                             
{'loss': '0.7476', 'grad_norm': '0.6675', 'learning_rate': '4.998e-05', 'epoch': '0.058', 'num_input_tokens_seen': 66211512, 'train_runtime': '1.335e+05', 'train_tokens_per_second': '495.8'}

  6%|▌         | 13050/225000 [37:05:48<390:51:04,  6.64s/it]
  6%|▌         | 13051/225000 [37:05:55<406:09:51,  6.90s/it]
  6%|▌         | 13052/225000 [37:06:00<368:44:37,  6.26s/it]
  6%|▌         | 13053/225000 [37:06:05<343:24:36,  5.83s/it]
  6%|▌         | 13054/225000 [37:06:15<420:03:40,  7.13s/it]
  6%|▌         | 13055/225000 [37:06:26<477:19:37,  8.11s/it]
  6%|▌         | 13056/225000 [37:06:30<409:11:22,  6.95s/it]
  6%|▌         | 13057/225000 [37:06:36<389:53:53,  6.62s/it]
  6%|▌         | 13058/225000 [37:06:47<466:39:09,  7.93s/it]
  6%|▌         | 13059/225000 [37:06:51<400:34:48,  6.80s/it]
  6%|▌         | 13060/225000 [37:06:57<386:46:28,  6.57s/it]
  6%|▌         | 13061/225000 [37:07:03<372:29:51,  6.33s/it]
  6%|▌         | 13062/225000 [37:07:13<448:38:19,  7.62s/it]
  6%|▌         | 13063/225000 [37:07:20<426:19:10,  7.24s/it]
  6%|▌         | 13064/225000 [37:07:24<375:17:29,  6.37s/it]
  6%|▌         | 13065/225000 [37:07:29<346:51:59,  5.89s/it]
  6%|▌         | 13066/225000 [37:07:34<331:44:33,  5.64s/it]
  6%|▌         | 13067/225000 [37:07:39<324:57:52,  5.52s/it]
  6%|▌         | 13068/225000 [37:07:51<433:45:15,  7.37s/it]
  6%|▌         | 13069/225000 [37:07:57<415:30:04,  7.06s/it]
  6%|▌         | 13070/225000 [37:08:03<393:24:30,  6.68s/it]
  6%|▌         | 13071/225000 [37:08:11<417:22:27,  7.09s/it]
  6%|▌         | 13072/225000 [37:08:16<376:39:08,  6.40s/it]
  6%|▌         | 13073/225000 [37:08:30<508:12:41,  8.63s/it]
  6%|▌         | 13074/225000 [37:08:37<488:30:32,  8.30s/it]
  6%|▌         | 13075/225000 [37:08:43<450:14:45,  7.65s/it]
  6%|▌         | 13076/225000 [37:08:52<466:30:37,  7.92s/it]
  6%|▌         | 13077/225000 [37:08:56<406:16:18,  6.90s/it]
  6%|▌         | 13078/225000 [37:09:01<367:39:46,  6.25s/it]
  6%|▌         | 13079/225000 [37:09:06<341:15:47,  5.80s/it]
  6%|▌         | 13080/225000 [37:09:17<429:13:20,  7.29s/it]
  6%|▌         | 13081/225000 [37:09:25<446:27:07,  7.58s/it]
  6%|▌         | 13082/225000 [37:09:28<373:13:15,  6.34s/it]
  6%|▌         | 13083/225000 [37:09:36<405:50:35,  6.89s/it]
  6%|▌         | 13084/225000 [37:09:41<364:18:49,  6.19s/it]
  6%|▌         | 13085/225000 [37:09:45<330:22:52,  5.61s/it]
  6%|▌         | 13086/225000 [37:09:55<410:42:58,  6.98s/it]
  6%|▌         | 13087/225000 [37:10:00<376:15:35,  6.39s/it]
  6%|▌         | 13088/225000 [37:10:11<450:23:52,  7.65s/it]
  6%|▌         | 13089/225000 [37:10:21<493:47:14,  8.39s/it]
  6%|▌         | 13090/225000 [37:10:26<423:45:15,  7.20s/it]
  6%|▌         | 13091/225000 [37:10:36<474:39:15,  8.06s/it]
  6%|▌         | 13092/225000 [37:10:39<398:47:56,  6.78s/it]
  6%|▌         | 13093/225000 [37:10:49<454:37:09,  7.72s/it]
  6%|▌         | 13094/225000 [37:10:54<405:42:29,  6.89s/it]
  6%|▌         | 13095/225000 [37:11:03<445:56:16,  7.58s/it]
  6%|▌         | 13096/225000 [37:11:10<435:56:38,  7.41s/it]
  6%|▌         | 13097/225000 [37:11:15<384:41:53,  6.54s/it]
  6%|▌         | 13098/225000 [37:11:26<459:18:50,  7.80s/it]
  6%|▌         | 13099/225000 [37:11:31<411:50:46,  7.00s/it]
  6%|▌         | 13100/225000 [37:11:36<373:01:34,  6.34s/it]
                                                             
{'loss': '0.7545', 'grad_norm': '0.7132', 'learning_rate': '4.997e-05', 'epoch': '0.05822', 'num_input_tokens_seen': 66374400, 'train_runtime': '1.339e+05', 'train_tokens_per_second': '495.7'}

  6%|▌         | 13100/225000 [37:11:36<373:01:34,  6.34s/it]
  6%|▌         | 13101/225000 [37:11:40<346:33:52,  5.89s/it]
  6%|▌         | 13102/225000 [37:11:45<327:29:05,  5.56s/it]
  6%|▌         | 13103/225000 [37:11:50<308:07:05,  5.23s/it]
  6%|▌         | 13104/225000 [37:11:59<385:47:23,  6.55s/it]
  6%|▌         | 13105/225000 [37:12:07<398:20:52,  6.77s/it]
  6%|▌         | 13106/225000 [37:12:11<350:59:46,  5.96s/it]
  6%|▌         | 13107/225000 [37:12:21<421:46:19,  7.17s/it]
  6%|▌         | 13108/225000 [37:12:27<413:24:56,  7.02s/it]
  6%|▌         | 13109/225000 [37:12:32<368:32:22,  6.26s/it]
  6%|▌         | 13110/225000 [37:12:37<346:20:23,  5.88s/it]
  6%|▌         | 13111/225000 [37:12:41<321:54:44,  5.47s/it]
  6%|▌         | 13112/225000 [37:12:51<389:28:38,  6.62s/it]
  6%|▌         | 13113/225000 [37:12:57<391:15:18,  6.65s/it]
  6%|▌         | 13114/225000 [37:13:04<399:09:18,  6.78s/it]
  6%|▌         | 13115/225000 [37:13:11<396:39:59,  6.74s/it]
  6%|▌         | 13116/225000 [37:13:19<425:25:39,  7.23s/it]
  6%|▌         | 13117/225000 [37:13:24<369:31:36,  6.28s/it]
  6%|▌         | 13118/225000 [37:13:32<403:37:47,  6.86s/it]
  6%|▌         | 13119/225000 [37:13:38<395:39:44,  6.72s/it]
  6%|▌         | 13120/225000 [37:13:43<360:02:20,  6.12s/it]
  6%|▌         | 13121/225000 [37:13:47<330:18:52,  5.61s/it]
  6%|▌         | 13122/225000 [37:13:52<317:04:32,  5.39s/it]
  6%|▌         | 13123/225000 [37:14:00<365:38:56,  6.21s/it]
  6%|▌         | 13124/225000 [37:14:11<452:54:05,  7.70s/it]
  6%|▌         | 13125/225000 [37:14:22<504:42:55,  8.58s/it]
  6%|▌         | 13126/225000 [37:14:27<448:13:32,  7.62s/it]
  6%|▌         | 13127/225000 [37:14:33<404:41:48,  6.88s/it]
  6%|▌         | 13128/225000 [37:14:38<385:47:47,  6.56s/it]
  6%|▌         | 13129/225000 [37:14:45<385:09:56,  6.54s/it]
  6%|▌         | 13130/225000 [37:14:50<350:37:59,  5.96s/it]
  6%|▌         | 13131/225000 [37:14:59<419:15:31,  7.12s/it]
  6%|▌         | 13132/225000 [37:15:06<403:33:15,  6.86s/it]
  6%|▌         | 13133/225000 [37:15:12<391:31:00,  6.65s/it]
  6%|▌         | 13134/225000 [37:15:15<335:00:13,  5.69s/it]
  6%|▌         | 13135/225000 [37:15:29<473:34:06,  8.05s/it]
  6%|▌         | 13136/225000 [37:15:36<451:01:38,  7.66s/it]
  6%|▌         | 13137/225000 [37:15:39<378:41:41,  6.43s/it]
  6%|▌         | 13138/225000 [37:15:48<422:06:03,  7.17s/it]
  6%|▌         | 13139/225000 [37:15:52<364:30:15,  6.19s/it]
  6%|▌         | 13140/225000 [37:16:00<404:28:33,  6.87s/it]
  6%|▌         | 13141/225000 [37:16:04<352:16:09,  5.99s/it]
  6%|▌         | 13142/225000 [37:16:13<399:15:51,  6.78s/it]
  6%|▌         | 13143/225000 [37:16:18<361:46:44,  6.15s/it]
  6%|▌         | 13144/225000 [37:16:22<335:10:52,  5.70s/it]
  6%|▌         | 13145/225000 [37:16:27<322:43:08,  5.48s/it]
  6%|▌         | 13146/225000 [37:16:32<318:45:29,  5.42s/it]
  6%|▌         | 13147/225000 [37:16:38<316:39:16,  5.38s/it]
  6%|▌         | 13148/225000 [37:16:43<310:04:49,  5.27s/it]
  6%|▌         | 13149/225000 [37:16:56<447:14:54,  7.60s/it]
  6%|▌         | 13150/225000 [37:17:02<426:04:24,  7.24s/it]
                                                             
{'loss': '0.7495', 'grad_norm': '0.7059', 'learning_rate': '4.997e-05', 'epoch': '0.05844', 'num_input_tokens_seen': 66530448, 'train_runtime': '1.342e+05', 'train_tokens_per_second': '495.7'}

  6%|▌         | 13150/225000 [37:17:02<426:04:24,  7.24s/it]
  6%|▌         | 13151/225000 [37:17:10<437:35:47,  7.44s/it]
  6%|▌         | 13152/225000 [37:17:14<377:23:25,  6.41s/it]
  6%|▌         | 13153/225000 [37:17:23<428:58:55,  7.29s/it]
  6%|▌         | 13154/225000 [37:17:28<387:00:04,  6.58s/it]
  6%|▌         | 13155/225000 [37:17:37<423:33:35,  7.20s/it]
  6%|▌         | 13156/225000 [37:17:41<360:23:19,  6.12s/it]
  6%|▌         | 13157/225000 [37:17:52<458:04:48,  7.78s/it]
  6%|▌         | 13158/225000 [37:17:57<404:32:55,  6.87s/it]
  6%|▌         | 13159/225000 [37:18:02<368:56:33,  6.27s/it]
  6%|▌         | 13160/225000 [37:18:07<342:35:48,  5.82s/it]
  6%|▌         | 13161/225000 [37:18:17<412:48:02,  7.02s/it]
  6%|▌         | 13162/225000 [37:18:22<380:49:55,  6.47s/it]
  6%|▌         | 13163/225000 [37:18:28<376:47:15,  6.40s/it]
  6%|▌         | 13164/225000 [37:18:33<343:53:14,  5.84s/it]
  6%|▌         | 13165/225000 [37:18:37<323:24:07,  5.50s/it]
  6%|▌         | 13166/225000 [37:18:45<357:52:20,  6.08s/it]
  6%|▌         | 13167/225000 [37:18:57<469:04:09,  7.97s/it]
  6%|▌         | 13168/225000 [37:19:04<455:44:30,  7.75s/it]
  6%|▌         | 13169/225000 [37:19:09<402:23:31,  6.84s/it]
  6%|▌         | 13170/225000 [37:19:18<436:48:03,  7.42s/it]
  6%|▌         | 13171/225000 [37:19:21<369:08:25,  6.27s/it]
  6%|▌         | 13172/225000 [37:19:30<402:59:53,  6.85s/it]
  6%|▌         | 13173/225000 [37:19:35<379:34:23,  6.45s/it]
  6%|▌         | 13174/225000 [37:19:44<431:12:59,  7.33s/it]
  6%|▌         | 13175/225000 [37:19:53<451:51:05,  7.68s/it]
  6%|▌         | 13176/225000 [37:19:59<429:27:00,  7.30s/it]
  6%|▌         | 13177/225000 [37:20:07<435:44:07,  7.41s/it]
  6%|▌         | 13178/225000 [37:20:14<422:18:00,  7.18s/it]
  6%|▌         | 13179/225000 [37:20:20<407:06:30,  6.92s/it]
  6%|▌         | 13180/225000 [37:20:24<360:49:40,  6.13s/it]
  6%|▌         | 13181/225000 [37:20:29<338:43:36,  5.76s/it]
  6%|▌         | 13182/225000 [37:20:39<413:53:21,  7.03s/it]
  6%|▌         | 13183/225000 [37:20:46<414:35:28,  7.05s/it]
  6%|▌         | 13184/225000 [37:20:51<366:57:34,  6.24s/it]
  6%|▌         | 13185/225000 [37:20:59<413:39:41,  7.03s/it]
  6%|▌         | 13186/225000 [37:21:05<388:51:42,  6.61s/it]
  6%|▌         | 13187/225000 [37:21:14<427:41:29,  7.27s/it]
  6%|▌         | 13188/225000 [37:21:21<430:49:34,  7.32s/it]
  6%|▌         | 13189/225000 [37:21:28<411:32:48,  6.99s/it]
  6%|▌         | 13190/225000 [37:21:31<353:09:25,  6.00s/it]
  6%|▌         | 13191/225000 [37:21:41<414:34:01,  7.05s/it]
  6%|▌         | 13192/225000 [37:21:48<417:15:55,  7.09s/it]
  6%|▌         | 13193/225000 [37:21:54<392:28:05,  6.67s/it]
  6%|▌         | 13194/225000 [37:22:01<405:33:22,  6.89s/it]
  6%|▌         | 13195/225000 [37:22:06<366:44:53,  6.23s/it]
  6%|▌         | 13196/225000 [37:22:11<342:20:08,  5.82s/it]
  6%|▌         | 13197/225000 [37:22:15<324:28:32,  5.52s/it]
  6%|▌         | 13198/225000 [37:22:27<430:22:32,  7.32s/it]
  6%|▌         | 13199/225000 [37:22:31<370:05:39,  6.29s/it]
  6%|▌         | 13200/225000 [37:22:36<345:12:30,  5.87s/it]
                                                             
{'loss': '0.7407', 'grad_norm': '0.7277', 'learning_rate': '4.997e-05', 'epoch': '0.05867', 'num_input_tokens_seen': 66687216, 'train_runtime': '1.346e+05', 'train_tokens_per_second': '495.6'}

  6%|▌         | 13200/225000 [37:22:36<345:12:30,  5.87s/it]
  6%|▌         | 13201/225000 [37:22:45<407:58:35,  6.93s/it]
  6%|▌         | 13202/225000 [37:22:49<352:08:48,  5.99s/it]
  6%|▌         | 13203/225000 [37:22:54<335:04:51,  5.70s/it]
  6%|▌         | 13204/225000 [37:22:59<322:47:50,  5.49s/it]
  6%|▌         | 13205/225000 [37:23:09<397:30:32,  6.76s/it]
  6%|▌         | 13206/225000 [37:23:17<428:25:02,  7.28s/it]
  6%|▌         | 13207/225000 [37:23:22<391:09:02,  6.65s/it]
  6%|▌         | 13208/225000 [37:23:31<429:16:37,  7.30s/it]
  6%|▌         | 13209/225000 [37:23:41<474:02:58,  8.06s/it]
  6%|▌         | 13210/225000 [37:23:46<419:20:40,  7.13s/it]
  6%|▌         | 13211/225000 [37:23:50<362:42:39,  6.17s/it]
  6%|▌         | 13212/225000 [37:23:56<364:13:07,  6.19s/it]
  6%|▌         | 13213/225000 [37:24:08<457:56:18,  7.78s/it]
  6%|▌         | 13214/225000 [37:24:12<391:38:50,  6.66s/it]
  6%|▌         | 13215/225000 [37:24:21<432:48:30,  7.36s/it]
  6%|▌         | 13216/225000 [37:24:25<382:08:42,  6.50s/it]
  6%|▌         | 13217/225000 [37:24:29<343:18:39,  5.84s/it]
  6%|▌         | 13218/225000 [37:24:40<422:26:21,  7.18s/it]
  6%|▌         | 13219/225000 [37:24:46<409:45:08,  6.97s/it]
  6%|▌         | 13220/225000 [37:24:51<381:16:50,  6.48s/it]
  6%|▌         | 13221/225000 [37:24:58<384:39:42,  6.54s/it]
  6%|▌         | 13222/225000 [37:25:04<367:38:29,  6.25s/it]
  6%|▌         | 13223/225000 [37:25:09<353:45:48,  6.01s/it]
  6%|▌         | 13224/225000 [37:25:15<344:11:32,  5.85s/it]
  6%|▌         | 13225/225000 [37:25:31<537:22:27,  9.13s/it]
  6%|▌         | 13226/225000 [37:25:36<456:30:17,  7.76s/it]
  6%|▌         | 13227/225000 [37:25:40<389:36:17,  6.62s/it]
  6%|▌         | 13228/225000 [37:25:51<469:07:01,  7.97s/it]
  6%|▌         | 13229/225000 [37:25:58<447:37:47,  7.61s/it]
  6%|▌         | 13230/225000 [37:26:02<390:18:13,  6.63s/it]
  6%|▌         | 13231/225000 [37:26:11<419:56:58,  7.14s/it]
  6%|▌         | 13232/225000 [37:26:16<385:02:20,  6.55s/it]
  6%|▌         | 13233/225000 [37:26:21<361:45:01,  6.15s/it]
  6%|▌         | 13234/225000 [37:26:26<336:25:45,  5.72s/it]
  6%|▌         | 13235/225000 [37:26:30<317:15:39,  5.39s/it]
  6%|▌         | 13236/225000 [37:26:41<419:39:35,  7.13s/it]
  6%|▌         | 13237/225000 [37:26:52<471:47:30,  8.02s/it]
  6%|▌         | 13238/225000 [37:26:55<397:41:22,  6.76s/it]
  6%|▌         | 13239/225000 [37:27:00<353:26:22,  6.01s/it]
  6%|▌         | 13240/225000 [37:27:11<454:09:54,  7.72s/it]
  6%|▌         | 13241/225000 [37:27:17<409:22:35,  6.96s/it]
  6%|▌         | 13242/225000 [37:27:21<368:55:53,  6.27s/it]
  6%|▌         | 13243/225000 [37:27:25<328:19:46,  5.58s/it]
  6%|▌         | 13244/225000 [37:27:30<319:09:08,  5.43s/it]
  6%|▌         | 13245/225000 [37:27:41<410:07:56,  6.97s/it]
  6%|▌         | 13246/225000 [37:27:45<365:06:46,  6.21s/it]
  6%|▌         | 13247/225000 [37:27:52<375:01:45,  6.38s/it]
  6%|▌         | 13248/225000 [37:27:57<352:56:47,  6.00s/it]
  6%|▌         | 13249/225000 [37:28:02<328:54:53,  5.59s/it]
  6%|▌         | 13250/225000 [37:28:13<436:25:24,  7.42s/it]
                                                             
{'loss': '0.7553', 'grad_norm': '0.6059', 'learning_rate': '4.997e-05', 'epoch': '0.05889', 'num_input_tokens_seen': 66842896, 'train_runtime': '1.349e+05', 'train_tokens_per_second': '495.5'}

  6%|▌         | 13250/225000 [37:28:13<436:25:24,  7.42s/it]
  6%|▌         | 13251/225000 [37:28:18<388:24:19,  6.60s/it]
  6%|▌         | 13252/225000 [37:28:26<407:15:03,  6.92s/it]
  6%|▌         | 13253/225000 [37:28:35<442:01:51,  7.52s/it]
  6%|▌         | 13254/225000 [37:28:44<471:59:36,  8.02s/it]
  6%|▌         | 13255/225000 [37:28:49<421:58:42,  7.17s/it]
  6%|▌         | 13256/225000 [37:28:54<388:07:35,  6.60s/it]
  6%|▌         | 13257/225000 [37:29:00<362:47:23,  6.17s/it]
  6%|▌         | 13258/225000 [37:29:05<351:45:43,  5.98s/it]
  6%|▌         | 13259/225000 [37:29:11<343:45:48,  5.84s/it]
  6%|▌         | 13260/225000 [37:29:25<497:58:05,  8.47s/it]
  6%|▌         | 13261/225000 [37:29:31<450:32:52,  7.66s/it]
  6%|▌         | 13262/225000 [37:29:37<412:52:32,  7.02s/it]
  6%|▌         | 13263/225000 [37:29:42<393:57:22,  6.70s/it]
  6%|▌         | 13264/225000 [37:29:47<363:01:09,  6.17s/it]
  6%|▌         | 13265/225000 [37:29:54<370:54:34,  6.31s/it]
  6%|▌         | 13266/225000 [37:29:59<351:26:29,  5.98s/it]
  6%|▌         | 13267/225000 [37:30:05<350:43:29,  5.96s/it]
  6%|▌         | 13268/225000 [37:30:10<331:38:01,  5.64s/it]
  6%|▌         | 13269/225000 [37:30:22<450:29:26,  7.66s/it]
  6%|▌         | 13270/225000 [37:30:28<411:34:09,  7.00s/it]
  6%|▌         | 13271/225000 [37:30:37<455:03:31,  7.74s/it]
  6%|▌         | 13272/225000 [37:30:41<389:56:12,  6.63s/it]
  6%|▌         | 13273/225000 [37:30:47<365:58:51,  6.22s/it]
  6%|▌         | 13274/225000 [37:30:56<425:38:04,  7.24s/it]
  6%|▌         | 13275/225000 [37:31:01<384:28:17,  6.54s/it]
  6%|▌         | 13276/225000 [37:31:06<353:30:50,  6.01s/it]
  6%|▌         | 13277/225000 [37:31:16<425:11:31,  7.23s/it]
  6%|▌         | 13278/225000 [37:31:20<371:39:37,  6.32s/it]
  6%|▌         | 13279/225000 [37:31:32<459:58:10,  7.82s/it]
  6%|▌         | 13280/225000 [37:31:42<502:08:26,  8.54s/it]
  6%|▌         | 13281/225000 [37:31:46<427:04:25,  7.26s/it]
  6%|▌         | 13282/225000 [37:31:50<366:37:42,  6.23s/it]
  6%|▌         | 13283/225000 [37:32:00<439:08:32,  7.47s/it]
  6%|▌         | 13284/225000 [37:32:06<416:51:03,  7.09s/it]
  6%|▌         | 13285/225000 [37:32:11<375:39:57,  6.39s/it]
  6%|▌         | 13286/225000 [37:32:17<357:21:05,  6.08s/it]
  6%|▌         | 13287/225000 [37:32:22<339:03:01,  5.77s/it]
  6%|▌         | 13288/225000 [37:32:27<341:28:58,  5.81s/it]
  6%|▌         | 13289/225000 [37:32:33<339:06:25,  5.77s/it]
  6%|▌         | 13290/225000 [37:32:46<455:51:59,  7.75s/it]
  6%|▌         | 13291/225000 [37:32:51<423:41:19,  7.20s/it]
  6%|▌         | 13292/225000 [37:32:58<410:51:55,  6.99s/it]
  6%|▌         | 13293/225000 [37:33:03<380:52:43,  6.48s/it]
  6%|▌         | 13294/225000 [37:33:08<353:54:09,  6.02s/it]
  6%|▌         | 13295/225000 [37:33:14<343:18:03,  5.84s/it]
  6%|▌         | 13296/225000 [37:33:19<334:01:29,  5.68s/it]
  6%|▌         | 13297/225000 [37:33:31<442:41:59,  7.53s/it]
  6%|▌         | 13298/225000 [37:33:37<419:49:50,  7.14s/it]
  6%|▌         | 13299/225000 [37:33:45<433:39:27,  7.37s/it]
  6%|▌         | 13300/225000 [37:33:51<403:55:24,  6.87s/it]
                                                             
{'loss': '0.7586', 'grad_norm': '0.6245', 'learning_rate': '4.997e-05', 'epoch': '0.05911', 'num_input_tokens_seen': 67003336, 'train_runtime': '1.352e+05', 'train_tokens_per_second': '495.5'}

  6%|▌         | 13300/225000 [37:33:51<403:55:24,  6.87s/it]
  6%|▌         | 13301/225000 [37:33:55<358:37:56,  6.10s/it]
  6%|▌         | 13302/225000 [37:34:00<333:45:46,  5.68s/it]
  6%|▌         | 13303/225000 [37:34:10<413:34:24,  7.03s/it]
  6%|▌         | 13304/225000 [37:34:14<369:26:26,  6.28s/it]
  6%|▌         | 13305/225000 [37:34:19<341:53:18,  5.81s/it]
  6%|▌         | 13306/225000 [37:34:25<339:39:21,  5.78s/it]
  6%|▌         | 13307/225000 [37:34:34<399:22:39,  6.79s/it]
  6%|▌         | 13308/225000 [37:34:40<383:44:52,  6.53s/it]
  6%|▌         | 13309/225000 [37:34:46<369:58:12,  6.29s/it]
  6%|▌         | 13310/225000 [37:34:51<356:21:12,  6.06s/it]
  6%|▌         | 13311/225000 [37:34:58<378:55:50,  6.44s/it]
  6%|▌         | 13312/225000 [37:35:10<471:38:17,  8.02s/it]
  6%|▌         | 13313/225000 [37:35:15<411:39:36,  7.00s/it]
  6%|▌         | 13314/225000 [37:35:24<443:46:42,  7.55s/it]
  6%|▌         | 13315/225000 [37:35:29<410:53:00,  6.99s/it]
  6%|▌         | 13316/225000 [37:35:42<517:54:59,  8.81s/it]
  6%|▌         | 13317/225000 [37:35:46<422:09:52,  7.18s/it]
  6%|▌         | 13318/225000 [37:35:53<429:55:02,  7.31s/it]
  6%|▌         | 13319/225000 [37:36:00<413:24:45,  7.03s/it]
  6%|▌         | 13320/225000 [37:36:04<361:52:51,  6.15s/it]
  6%|▌         | 13321/225000 [37:36:08<329:15:53,  5.60s/it]
  6%|▌         | 13322/225000 [37:36:18<398:05:26,  6.77s/it]
  6%|▌         | 13323/225000 [37:36:22<352:20:32,  5.99s/it]
  6%|▌         | 13324/225000 [37:36:31<414:58:38,  7.06s/it]
  6%|▌         | 13325/225000 [37:36:37<391:34:25,  6.66s/it]
  6%|▌         | 13326/225000 [37:36:43<374:06:12,  6.36s/it]
  6%|▌         | 13327/225000 [37:36:52<431:29:33,  7.34s/it]
  6%|▌         | 13328/225000 [37:36:59<422:31:10,  7.19s/it]
  6%|▌         | 13329/225000 [37:37:04<377:04:34,  6.41s/it]
  6%|▌         | 13330/225000 [37:37:08<335:56:30,  5.71s/it]
  6%|▌         | 13331/225000 [37:37:12<307:08:50,  5.22s/it]
  6%|▌         | 13332/225000 [37:37:17<302:25:33,  5.14s/it]
  6%|▌         | 13333/225000 [37:37:29<421:27:57,  7.17s/it]
  6%|▌         | 13334/225000 [37:37:33<373:28:26,  6.35s/it]
  6%|▌         | 13335/225000 [37:37:43<430:56:10,  7.33s/it]
  6%|▌         | 13336/225000 [37:37:47<374:27:08,  6.37s/it]
  6%|▌         | 13337/225000 [37:37:57<434:54:26,  7.40s/it]
  6%|▌         | 13338/225000 [37:38:01<376:41:14,  6.41s/it]
  6%|▌         | 13339/225000 [37:38:06<357:27:33,  6.08s/it]
  6%|▌         | 13340/225000 [37:38:16<427:39:15,  7.27s/it]
  6%|▌         | 13341/225000 [37:38:24<443:29:09,  7.54s/it]
  6%|▌         | 13342/225000 [37:38:32<436:34:52,  7.43s/it]
  6%|▌         | 13343/225000 [37:38:38<416:45:21,  7.09s/it]
  6%|▌         | 13344/225000 [37:38:42<369:39:31,  6.29s/it]
  6%|▌         | 13345/225000 [37:38:52<436:19:57,  7.42s/it]
  6%|▌         | 13346/225000 [37:38:58<404:00:10,  6.87s/it]
  6%|▌         | 13347/225000 [37:39:03<371:12:35,  6.31s/it]
  6%|▌         | 13348/225000 [37:39:11<394:15:16,  6.71s/it]
  6%|▌         | 13349/225000 [37:39:14<340:27:56,  5.79s/it]
  6%|▌         | 13350/225000 [37:39:23<400:04:14,  6.80s/it]
                                                             
{'loss': '0.7513', 'grad_norm': '0.6202', 'learning_rate': '4.997e-05', 'epoch': '0.05933', 'num_input_tokens_seen': 67161544, 'train_runtime': '1.356e+05', 'train_tokens_per_second': '495.4'}

  6%|▌         | 13350/225000 [37:39:23<400:04:14,  6.80s/it]
  6%|▌         | 13351/225000 [37:39:30<389:33:55,  6.63s/it]
  6%|▌         | 13352/225000 [37:39:35<361:53:23,  6.16s/it]
  6%|▌         | 13353/225000 [37:39:40<343:40:20,  5.85s/it]
  6%|▌         | 13354/225000 [37:39:45<329:31:00,  5.60s/it]
  6%|▌         | 13355/225000 [37:39:50<321:57:08,  5.48s/it]
  6%|▌         | 13356/225000 [37:39:56<336:13:52,  5.72s/it]
  6%|▌         | 13357/225000 [37:40:09<461:12:19,  7.84s/it]
  6%|▌         | 13358/225000 [37:40:13<391:03:37,  6.65s/it]
  6%|▌         | 13359/225000 [37:40:21<421:21:16,  7.17s/it]
  6%|▌         | 13360/225000 [37:40:31<469:23:10,  7.98s/it]
  6%|▌         | 13361/225000 [37:40:37<425:29:35,  7.24s/it]
  6%|▌         | 13362/225000 [37:40:47<475:06:46,  8.08s/it]
  6%|▌         | 13363/225000 [37:40:53<441:38:12,  7.51s/it]
  6%|▌         | 13364/225000 [37:40:59<414:22:54,  7.05s/it]
  6%|▌         | 13365/225000 [37:41:07<431:06:13,  7.33s/it]
  6%|▌         | 13366/225000 [37:41:11<367:30:33,  6.25s/it]
  6%|▌         | 13367/225000 [37:41:19<409:07:48,  6.96s/it]
  6%|▌         | 13368/225000 [37:41:26<405:19:07,  6.89s/it]
  6%|▌         | 13369/225000 [37:41:31<376:27:14,  6.40s/it]
  6%|▌         | 13370/225000 [37:41:36<354:20:32,  6.03s/it]
  6%|▌         | 13371/225000 [37:41:42<340:17:12,  5.79s/it]
  6%|▌         | 13372/225000 [37:41:46<320:54:25,  5.46s/it]
  6%|▌         | 13373/225000 [37:41:52<333:28:45,  5.67s/it]
  6%|▌         | 13374/225000 [37:42:03<418:35:15,  7.12s/it]
  6%|▌         | 13375/225000 [37:42:07<370:43:05,  6.31s/it]
  6%|▌         | 13376/225000 [37:42:13<353:36:59,  6.02s/it]
  6%|▌         | 13377/225000 [37:42:18<342:27:20,  5.83s/it]
  6%|▌         | 13378/225000 [37:42:23<334:37:55,  5.69s/it]
  6%|▌         | 13379/225000 [37:42:29<330:12:48,  5.62s/it]
  6%|▌         | 13380/225000 [37:42:43<487:04:06,  8.29s/it]
  6%|▌         | 13381/225000 [37:42:50<461:57:03,  7.86s/it]
  6%|▌         | 13382/225000 [37:42:55<402:39:20,  6.85s/it]
  6%|▌         | 13383/225000 [37:42:59<363:56:12,  6.19s/it]
  6%|▌         | 13384/225000 [37:43:03<323:50:26,  5.51s/it]
  6%|▌         | 13385/225000 [37:43:15<424:25:52,  7.22s/it]
  6%|▌         | 13386/225000 [37:43:19<370:08:25,  6.30s/it]
  6%|▌         | 13387/225000 [37:43:24<359:52:17,  6.12s/it]
  6%|▌         | 13388/225000 [37:43:37<465:11:29,  7.91s/it]
  6%|▌         | 13389/225000 [37:43:42<421:04:38,  7.16s/it]
  6%|▌         | 13390/225000 [37:43:47<392:22:08,  6.68s/it]
  6%|▌         | 13391/225000 [37:43:53<375:00:58,  6.38s/it]
  6%|▌         | 13392/225000 [37:43:59<360:40:09,  6.14s/it]
  6%|▌         | 13393/225000 [37:44:04<351:06:55,  5.97s/it]
  6%|▌         | 13394/225000 [37:44:10<343:32:15,  5.84s/it]
  6%|▌         | 13395/225000 [37:44:15<324:14:46,  5.52s/it]
  6%|▌         | 13396/225000 [37:44:20<318:07:50,  5.41s/it]
  6%|▌         | 13397/225000 [37:44:26<326:26:23,  5.55s/it]
  6%|▌         | 13398/225000 [37:44:41<506:00:24,  8.61s/it]
  6%|▌         | 13399/225000 [37:44:46<429:14:09,  7.30s/it]
  6%|▌         | 13400/225000 [37:44:51<395:42:58,  6.73s/it]
                                                             
{'loss': '0.7385', 'grad_norm': '0.6992', 'learning_rate': '4.997e-05', 'epoch': '0.05956', 'num_input_tokens_seen': 67318936, 'train_runtime': '1.359e+05', 'train_tokens_per_second': '495.4'}

  6%|▌         | 13400/225000 [37:44:51<395:42:58,  6.73s/it]
  6%|▌         | 13401/225000 [37:45:01<457:39:08,  7.79s/it]
  6%|▌         | 13402/225000 [37:45:09<459:42:58,  7.82s/it]
  6%|▌         | 13403/225000 [37:45:13<385:52:36,  6.57s/it]
  6%|▌         | 13404/225000 [37:45:21<412:46:14,  7.02s/it]
  6%|▌         | 13405/225000 [37:45:29<425:15:10,  7.24s/it]
  6%|▌         | 13406/225000 [37:45:34<391:53:35,  6.67s/it]
  6%|▌         | 13407/225000 [37:45:43<435:20:29,  7.41s/it]
  6%|▌         | 13408/225000 [37:45:49<403:09:43,  6.86s/it]
  6%|▌         | 13409/225000 [37:45:56<410:41:13,  6.99s/it]
  6%|▌         | 13410/225000 [37:46:03<403:36:15,  6.87s/it]
  6%|▌         | 13411/225000 [37:46:11<429:03:21,  7.30s/it]
  6%|▌         | 13412/225000 [37:46:21<475:37:24,  8.09s/it]
  6%|▌         | 13413/225000 [37:46:26<416:14:24,  7.08s/it]
  6%|▌         | 13414/225000 [37:46:31<378:48:51,  6.45s/it]
  6%|▌         | 13415/225000 [37:46:36<357:43:26,  6.09s/it]
  6%|▌         | 13416/225000 [37:46:41<334:57:25,  5.70s/it]
  6%|▌         | 13417/225000 [37:46:46<337:37:54,  5.74s/it]
  6%|▌         | 13418/225000 [37:46:52<327:05:22,  5.57s/it]
  6%|▌         | 13419/225000 [37:46:57<324:36:15,  5.52s/it]
  6%|▌         | 13420/225000 [37:47:02<316:19:28,  5.38s/it]
  6%|▌         | 13421/225000 [37:47:08<319:08:12,  5.43s/it]
  6%|▌         | 13422/225000 [37:47:22<474:11:51,  8.07s/it]
  6%|▌         | 13423/225000 [37:47:31<496:46:15,  8.45s/it]
  6%|▌         | 13424/225000 [37:47:37<442:42:04,  7.53s/it]
  6%|▌         | 13425/225000 [37:47:42<412:17:12,  7.02s/it]
  6%|▌         | 13426/225000 [37:47:47<376:36:39,  6.41s/it]
  6%|▌         | 13427/225000 [37:47:57<429:49:23,  7.31s/it]
  6%|▌         | 13428/225000 [37:48:02<401:00:47,  6.82s/it]
  6%|▌         | 13429/225000 [37:48:09<400:04:57,  6.81s/it]
  6%|▌         | 13430/225000 [37:48:17<417:39:25,  7.11s/it]
  6%|▌         | 13431/225000 [37:48:21<360:28:16,  6.13s/it]
  6%|▌         | 13432/225000 [37:48:30<416:00:07,  7.08s/it]
  6%|▌         | 13433/225000 [37:48:35<379:26:33,  6.46s/it]
  6%|▌         | 13434/225000 [37:48:40<357:39:32,  6.09s/it]
  6%|▌         | 13435/225000 [37:48:49<403:17:38,  6.86s/it]
  6%|▌         | 13436/225000 [37:48:53<355:11:29,  6.04s/it]
  6%|▌         | 13437/225000 [37:49:03<425:15:32,  7.24s/it]
  6%|▌         | 13438/225000 [37:49:07<369:25:58,  6.29s/it]
  6%|▌         | 13439/225000 [37:49:12<343:48:25,  5.85s/it]
  6%|▌         | 13440/225000 [37:49:19<367:39:46,  6.26s/it]
  6%|▌         | 13441/225000 [37:49:32<475:37:17,  8.09s/it]
  6%|▌         | 13442/225000 [37:49:36<405:13:03,  6.90s/it]
  6%|▌         | 13443/225000 [37:49:40<365:58:49,  6.23s/it]
  6%|▌         | 13444/225000 [37:49:45<341:02:21,  5.80s/it]
  6%|▌         | 13445/225000 [37:49:56<421:25:41,  7.17s/it]
  6%|▌         | 13446/225000 [37:50:01<395:09:34,  6.72s/it]
  6%|▌         | 13447/225000 [37:50:05<347:22:20,  5.91s/it]
  6%|▌         | 13448/225000 [37:50:15<411:30:29,  7.00s/it]
  6%|▌         | 13449/225000 [37:50:22<416:24:29,  7.09s/it]
  6%|▌         | 13450/225000 [37:50:28<397:54:35,  6.77s/it]
                                                             
{'loss': '0.761', 'grad_norm': '0.6055', 'learning_rate': '4.997e-05', 'epoch': '0.05978', 'num_input_tokens_seen': 67479632, 'train_runtime': '1.362e+05', 'train_tokens_per_second': '495.3'}

  6%|▌         | 13450/225000 [37:50:28<397:54:35,  6.77s/it]
  6%|▌         | 13451/225000 [37:50:33<354:31:26,  6.03s/it]
  6%|▌         | 13452/225000 [37:50:42<421:14:53,  7.17s/it]
  6%|▌         | 13453/225000 [37:50:48<389:17:33,  6.62s/it]
  6%|▌         | 13454/225000 [37:50:55<401:43:39,  6.84s/it]
  6%|▌         | 13455/225000 [37:51:01<394:53:24,  6.72s/it]
  6%|▌         | 13456/225000 [37:51:07<379:49:40,  6.46s/it]
  6%|▌         | 13457/225000 [37:51:12<351:40:09,  5.98s/it]
  6%|▌         | 13458/225000 [37:51:18<344:01:05,  5.85s/it]
  6%|▌         | 13459/225000 [37:51:24<348:22:41,  5.93s/it]
  6%|▌         | 13460/225000 [37:51:36<456:25:00,  7.77s/it]
  6%|▌         | 13461/225000 [37:51:39<377:00:21,  6.42s/it]
  6%|▌         | 13462/225000 [37:51:47<404:58:06,  6.89s/it]
  6%|▌         | 13463/225000 [37:51:56<434:54:24,  7.40s/it]
  6%|▌         | 13464/225000 [37:52:02<416:30:08,  7.09s/it]
  6%|▌         | 13465/225000 [37:52:14<492:29:52,  8.38s/it]
  6%|▌         | 13466/225000 [37:52:18<417:23:26,  7.10s/it]
  6%|▌         | 13467/225000 [37:52:23<379:03:38,  6.45s/it]
  6%|▌         | 13468/225000 [37:52:27<335:53:32,  5.72s/it]
  6%|▌         | 13469/225000 [37:52:38<429:41:04,  7.31s/it]
  6%|▌         | 13470/225000 [37:52:49<505:04:41,  8.60s/it]
  6%|▌         | 13471/225000 [37:52:53<419:25:18,  7.14s/it]
  6%|▌         | 13472/225000 [37:53:01<442:39:11,  7.53s/it]
  6%|▌         | 13473/225000 [37:53:05<380:44:37,  6.48s/it]
  6%|▌         | 13474/225000 [37:53:14<417:15:22,  7.10s/it]
  6%|▌         | 13475/225000 [37:53:19<379:30:09,  6.46s/it]
  6%|▌         | 13476/225000 [37:53:25<369:20:15,  6.29s/it]
  6%|▌         | 13477/225000 [37:53:30<356:54:59,  6.07s/it]
  6%|▌         | 13478/225000 [37:53:42<456:01:27,  7.76s/it]
  6%|▌         | 13479/225000 [37:53:51<473:53:53,  8.07s/it]
  6%|▌         | 13480/225000 [37:54:01<517:47:39,  8.81s/it]
  6%|▌         | 13481/225000 [37:54:06<444:43:03,  7.57s/it]
  6%|▌         | 13482/225000 [37:54:16<493:09:09,  8.39s/it]
  6%|▌         | 13483/225000 [37:54:23<464:23:15,  7.90s/it]
  6%|▌         | 13484/225000 [37:54:35<525:10:25,  8.94s/it]
  6%|▌         | 13485/225000 [37:54:39<446:38:22,  7.60s/it]
  6%|▌         | 13486/225000 [37:54:43<391:13:46,  6.66s/it]
  6%|▌         | 13487/225000 [37:54:53<436:42:01,  7.43s/it]
  6%|▌         | 13488/225000 [37:54:59<409:05:07,  6.96s/it]
  6%|▌         | 13489/225000 [37:55:04<382:25:45,  6.51s/it]
  6%|▌         | 13490/225000 [37:55:12<402:20:59,  6.85s/it]
  6%|▌         | 13491/225000 [37:55:15<347:36:01,  5.92s/it]
  6%|▌         | 13492/225000 [37:55:23<372:30:37,  6.34s/it]
  6%|▌         | 13493/225000 [37:55:27<341:03:43,  5.81s/it]
  6%|▌         | 13494/225000 [37:55:37<410:47:02,  6.99s/it]
  6%|▌         | 13495/225000 [37:55:46<437:27:48,  7.45s/it]
  6%|▌         | 13496/225000 [37:55:53<440:43:27,  7.50s/it]
  6%|▌         | 13497/225000 [37:55:57<375:54:37,  6.40s/it]
  6%|▌         | 13498/225000 [37:56:06<429:41:41,  7.31s/it]
  6%|▌         | 13499/225000 [37:56:11<388:06:18,  6.61s/it]
  6%|▌         | 13500/225000 [37:56:17<361:42:46,  6.16s/it]
                                                             
{'loss': '0.7549', 'grad_norm': '0.5395', 'learning_rate': '4.997e-05', 'epoch': '0.06', 'num_input_tokens_seen': 67644792, 'train_runtime': '1.366e+05', 'train_tokens_per_second': '495.3'}

  6%|▌         | 13500/225000 [37:56:17<361:42:46,  6.16s/it]
  6%|▌         | 13501/225000 [37:56:22<346:58:16,  5.91s/it]
  6%|▌         | 13502/225000 [37:56:27<330:04:47,  5.62s/it]
  6%|▌         | 13503/225000 [37:56:33<332:21:01,  5.66s/it]
  6%|▌         | 13504/225000 [37:56:44<433:19:02,  7.38s/it]
  6%|▌         | 13505/225000 [37:56:48<383:40:00,  6.53s/it]
  6%|▌         | 13506/225000 [37:56:57<426:49:39,  7.27s/it]
  6%|▌         | 13507/225000 [37:57:04<411:00:51,  7.00s/it]
  6%|▌         | 13508/225000 [37:57:10<397:49:45,  6.77s/it]
  6%|▌         | 13509/225000 [37:57:15<363:32:01,  6.19s/it]
  6%|▌         | 13510/225000 [37:57:19<335:06:28,  5.70s/it]
  6%|▌         | 13511/225000 [37:57:30<420:36:27,  7.16s/it]
  6%|▌         | 13512/225000 [37:57:34<364:31:08,  6.20s/it]
  6%|▌         | 13513/225000 [37:57:43<418:09:15,  7.12s/it]
  6%|▌         | 13514/225000 [37:57:47<360:08:28,  6.13s/it]
  6%|▌         | 13515/225000 [37:57:52<337:11:07,  5.74s/it]
  6%|▌         | 13516/225000 [37:58:00<379:17:05,  6.46s/it]
  6%|▌         | 13517/225000 [37:58:08<409:03:14,  6.96s/it]
  6%|▌         | 13518/225000 [37:58:13<367:24:29,  6.25s/it]
  6%|▌         | 13519/225000 [37:58:18<352:28:09,  6.00s/it]
  6%|▌         | 13520/225000 [37:58:24<342:09:21,  5.82s/it]
  6%|▌         | 13521/225000 [37:58:29<333:50:21,  5.68s/it]
  6%|▌         | 13522/225000 [37:58:44<500:05:34,  8.51s/it]
  6%|▌         | 13523/225000 [37:58:48<418:22:36,  7.12s/it]
  6%|▌         | 13524/225000 [37:58:54<404:52:09,  6.89s/it]
  6%|▌         | 13525/225000 [37:59:03<442:35:07,  7.53s/it]
  6%|▌         | 13526/225000 [37:59:11<450:51:42,  7.68s/it]
  6%|▌         | 13527/225000 [37:59:15<386:52:17,  6.59s/it]
  6%|▌         | 13528/225000 [37:59:23<405:26:25,  6.90s/it]
  6%|▌         | 13529/225000 [37:59:29<393:48:09,  6.70s/it]
  6%|▌         | 13530/225000 [37:59:35<369:35:22,  6.29s/it]
  6%|▌         | 13531/225000 [37:59:43<398:11:52,  6.78s/it]
  6%|▌         | 13532/225000 [37:59:49<393:50:55,  6.70s/it]
  6%|▌         | 13533/225000 [37:59:54<360:41:48,  6.14s/it]
  6%|▌         | 13534/225000 [38:00:01<377:23:42,  6.42s/it]
  6%|▌         | 13535/225000 [38:00:06<358:10:20,  6.10s/it]
  6%|▌         | 13536/225000 [38:00:16<421:41:43,  7.18s/it]
  6%|▌         | 13537/225000 [38:00:20<370:22:10,  6.31s/it]
  6%|▌         | 13538/225000 [38:00:25<341:39:47,  5.82s/it]
  6%|▌         | 13539/225000 [38:00:30<325:38:25,  5.54s/it]
  6%|▌         | 13540/225000 [38:00:34<305:37:20,  5.20s/it]
  6%|▌         | 13541/225000 [38:00:47<436:45:59,  7.44s/it]
  6%|▌         | 13542/225000 [38:00:52<399:18:18,  6.80s/it]
  6%|▌         | 13543/225000 [38:00:57<359:06:26,  6.11s/it]
  6%|▌         | 13544/225000 [38:01:06<422:53:33,  7.20s/it]
  6%|▌         | 13545/225000 [38:01:15<453:01:16,  7.71s/it]
  6%|▌         | 13546/225000 [38:01:20<389:41:47,  6.63s/it]
  6%|▌         | 13547/225000 [38:01:28<428:26:22,  7.29s/it]
  6%|▌         | 13548/225000 [38:01:35<424:57:17,  7.23s/it]
  6%|▌         | 13549/225000 [38:01:41<391:29:07,  6.67s/it]
  6%|▌         | 13550/225000 [38:01:46<357:36:53,  6.09s/it]
                                                             
{'loss': '0.748', 'grad_norm': '0.6093', 'learning_rate': '4.997e-05', 'epoch': '0.06022', 'num_input_tokens_seen': 67800904, 'train_runtime': '1.369e+05', 'train_tokens_per_second': '495.2'}

  6%|▌         | 13550/225000 [38:01:46<357:36:53,  6.09s/it]
  6%|▌         | 13551/225000 [38:01:50<334:39:19,  5.70s/it]
  6%|▌         | 13552/225000 [38:02:01<429:51:21,  7.32s/it]
  6%|▌         | 13553/225000 [38:02:10<451:46:21,  7.69s/it]
  6%|▌         | 13554/225000 [38:02:15<396:22:17,  6.75s/it]
  6%|▌         | 13555/225000 [38:02:20<365:30:25,  6.22s/it]
  6%|▌         | 13556/225000 [38:02:27<395:07:15,  6.73s/it]
  6%|▌         | 13557/225000 [38:02:36<428:03:28,  7.29s/it]
  6%|▌         | 13558/225000 [38:02:40<366:43:36,  6.24s/it]
  6%|▌         | 13559/225000 [38:02:49<410:41:15,  6.99s/it]
  6%|▌         | 13560/225000 [38:02:54<384:53:22,  6.55s/it]
  6%|▌         | 13561/225000 [38:02:58<347:01:40,  5.91s/it]
  6%|▌         | 13562/225000 [38:03:03<324:59:47,  5.53s/it]
  6%|▌         | 13563/225000 [38:03:09<338:41:56,  5.77s/it]
  6%|▌         | 13564/225000 [38:03:19<397:18:21,  6.76s/it]
  6%|▌         | 13565/225000 [38:03:24<379:10:23,  6.46s/it]
  6%|▌         | 13566/225000 [38:03:31<375:23:41,  6.39s/it]
  6%|▌         | 13567/225000 [38:03:36<357:57:18,  6.09s/it]
  6%|▌         | 13568/225000 [38:03:41<342:38:03,  5.83s/it]
  6%|▌         | 13569/225000 [38:03:48<369:05:20,  6.28s/it]
  6%|▌         | 13570/225000 [38:03:54<353:09:18,  6.01s/it]
  6%|▌         | 13571/225000 [38:03:59<340:35:58,  5.80s/it]
  6%|▌         | 13572/225000 [38:04:04<329:40:18,  5.61s/it]
  6%|▌         | 13573/225000 [38:04:10<325:27:38,  5.54s/it]
  6%|▌         | 13574/225000 [38:04:16<330:33:20,  5.63s/it]
  6%|▌         | 13575/225000 [38:04:27<435:17:17,  7.41s/it]
  6%|▌         | 13576/225000 [38:04:33<405:38:41,  6.91s/it]
  6%|▌         | 13577/225000 [38:04:38<366:02:09,  6.23s/it]
  6%|▌         | 13578/225000 [38:04:49<453:32:20,  7.72s/it]
  6%|▌         | 13579/225000 [38:04:54<410:02:09,  6.98s/it]
  6%|▌         | 13580/225000 [38:04:59<371:14:42,  6.32s/it]
  6%|▌         | 13581/225000 [38:05:04<352:05:44,  6.00s/it]
  6%|▌         | 13582/225000 [38:05:10<347:56:55,  5.92s/it]
  6%|▌         | 13583/225000 [38:05:15<335:39:38,  5.72s/it]
  6%|▌         | 13584/225000 [38:05:21<337:17:12,  5.74s/it]
  6%|▌         | 13585/225000 [38:05:34<470:01:09,  8.00s/it]
  6%|▌         | 13586/225000 [38:05:40<441:23:26,  7.52s/it]
  6%|▌         | 13587/225000 [38:05:48<446:28:05,  7.60s/it]
  6%|▌         | 13588/225000 [38:05:54<422:37:40,  7.20s/it]
  6%|▌         | 13589/225000 [38:06:02<425:13:19,  7.24s/it]
  6%|▌         | 13590/225000 [38:06:07<380:12:48,  6.47s/it]
  6%|▌         | 13591/225000 [38:06:16<435:17:45,  7.41s/it]
  6%|▌         | 13592/225000 [38:06:21<394:27:01,  6.72s/it]
  6%|▌         | 13593/225000 [38:06:29<419:08:28,  7.14s/it]
  6%|▌         | 13594/225000 [38:06:34<367:21:42,  6.26s/it]
  6%|▌         | 13595/225000 [38:06:45<453:55:12,  7.73s/it]
  6%|▌         | 13596/225000 [38:06:49<391:54:47,  6.67s/it]
  6%|▌         | 13597/225000 [38:06:54<360:07:06,  6.13s/it]
  6%|▌         | 13598/225000 [38:07:03<416:55:03,  7.10s/it]
  6%|▌         | 13599/225000 [38:07:12<442:28:59,  7.54s/it]
  6%|▌         | 13600/225000 [38:07:15<373:38:23,  6.36s/it]
                                                             
{'loss': '0.7396', 'grad_norm': '0.5733', 'learning_rate': '4.997e-05', 'epoch': '0.06044', 'num_input_tokens_seen': 67956256, 'train_runtime': '1.372e+05', 'train_tokens_per_second': '495.2'}

  6%|▌         | 13600/225000 [38:07:15<373:38:23,  6.36s/it]
  6%|▌         | 13601/225000 [38:07:20<344:36:57,  5.87s/it]
  6%|▌         | 13602/225000 [38:07:25<331:07:40,  5.64s/it]
  6%|▌         | 13603/225000 [38:07:30<318:05:51,  5.42s/it]
  6%|▌         | 13604/225000 [38:07:41<419:03:12,  7.14s/it]
  6%|▌         | 13605/225000 [38:07:49<438:06:23,  7.46s/it]
  6%|▌         | 13606/225000 [38:07:53<375:54:42,  6.40s/it]
  6%|▌         | 13607/225000 [38:07:58<352:42:30,  6.01s/it]
  6%|▌         | 13608/225000 [38:08:03<331:42:48,  5.65s/it]
  6%|▌         | 13609/225000 [38:08:15<432:57:02,  7.37s/it]
  6%|▌         | 13610/225000 [38:08:20<392:39:54,  6.69s/it]
  6%|▌         | 13611/225000 [38:08:25<373:05:24,  6.35s/it]
  6%|▌         | 13612/225000 [38:08:31<357:08:12,  6.08s/it]
  6%|▌         | 13613/225000 [38:08:36<341:27:51,  5.82s/it]
  6%|▌         | 13614/225000 [38:08:42<337:45:56,  5.75s/it]
  6%|▌         | 13615/225000 [38:08:47<328:19:46,  5.59s/it]
  6%|▌         | 13616/225000 [38:08:53<333:25:35,  5.68s/it]
  6%|▌         | 13617/225000 [38:09:05<458:29:00,  7.81s/it]
  6%|▌         | 13618/225000 [38:09:11<424:36:35,  7.23s/it]
  6%|▌         | 13619/225000 [38:09:18<423:21:02,  7.21s/it]
  6%|▌         | 13620/225000 [38:09:28<471:18:30,  8.03s/it]
  6%|▌         | 13621/225000 [38:09:34<426:15:43,  7.26s/it]
  6%|▌         | 13622/225000 [38:09:41<432:51:49,  7.37s/it]
  6%|▌         | 13623/225000 [38:09:48<416:56:21,  7.10s/it]
  6%|▌         | 13624/225000 [38:09:52<369:45:32,  6.30s/it]
  6%|▌         | 13625/225000 [38:10:02<423:33:40,  7.21s/it]
  6%|▌         | 13626/225000 [38:10:09<418:12:42,  7.12s/it]
  6%|▌         | 13627/225000 [38:10:13<365:12:59,  6.22s/it]
  6%|▌         | 13628/225000 [38:10:22<427:08:32,  7.27s/it]
  6%|▌         | 13629/225000 [38:10:27<375:14:16,  6.39s/it]
  6%|▌         | 13630/225000 [38:10:33<368:01:38,  6.27s/it]
  6%|▌         | 13631/225000 [38:10:42<413:10:19,  7.04s/it]
  6%|▌         | 13632/225000 [38:10:48<400:52:28,  6.83s/it]
  6%|▌         | 13633/225000 [38:10:53<374:18:53,  6.38s/it]
  6%|▌         | 13634/225000 [38:11:04<450:00:19,  7.66s/it]
  6%|▌         | 13635/225000 [38:11:08<385:40:28,  6.57s/it]
  6%|▌         | 13636/225000 [38:11:16<405:43:32,  6.91s/it]
  6%|▌         | 13637/225000 [38:11:22<394:49:17,  6.72s/it]
  6%|▌         | 13638/225000 [38:11:26<349:13:01,  5.95s/it]
  6%|▌         | 13639/225000 [38:11:32<356:20:43,  6.07s/it]
  6%|▌         | 13640/225000 [38:11:41<398:40:48,  6.79s/it]
  6%|▌         | 13641/225000 [38:11:48<396:19:14,  6.75s/it]
  6%|▌         | 13642/225000 [38:11:53<377:29:37,  6.43s/it]
  6%|▌         | 13643/225000 [38:12:05<472:30:31,  8.05s/it]
  6%|▌         | 13644/225000 [38:12:10<421:33:24,  7.18s/it]
  6%|▌         | 13645/225000 [38:12:15<385:27:39,  6.57s/it]
  6%|▌         | 13646/225000 [38:12:21<375:37:53,  6.40s/it]
  6%|▌         | 13647/225000 [38:12:27<359:52:46,  6.13s/it]
  6%|▌         | 13648/225000 [38:12:33<352:08:08,  6.00s/it]
  6%|▌         | 13649/225000 [38:12:38<339:02:19,  5.77s/it]
  6%|▌         | 13650/225000 [38:12:44<339:02:56,  5.78s/it]
                                                             
{'loss': '0.7442', 'grad_norm': '0.5806', 'learning_rate': '4.996e-05', 'epoch': '0.06067', 'num_input_tokens_seen': 68114368, 'train_runtime': '1.376e+05', 'train_tokens_per_second': '495.1'}

  6%|▌         | 13650/225000 [38:12:44<339:02:56,  5.78s/it]
  6%|▌         | 13651/225000 [38:12:49<336:31:03,  5.73s/it]
  6%|▌         | 13652/225000 [38:12:55<337:47:53,  5.75s/it]
  6%|▌         | 13653/225000 [38:13:00<331:03:31,  5.64s/it]
  6%|▌         | 13654/225000 [38:13:13<459:36:13,  7.83s/it]
  6%|▌         | 13655/225000 [38:13:18<406:07:47,  6.92s/it]
  6%|▌         | 13656/225000 [38:13:26<418:57:50,  7.14s/it]
  6%|▌         | 13657/225000 [38:13:34<430:03:09,  7.33s/it]
  6%|▌         | 13658/225000 [38:13:37<366:10:55,  6.24s/it]
  6%|▌         | 13659/225000 [38:13:45<399:30:36,  6.81s/it]
  6%|▌         | 13660/225000 [38:13:49<348:34:15,  5.94s/it]
  6%|▌         | 13661/225000 [38:13:57<377:38:23,  6.43s/it]
  6%|▌         | 13662/225000 [38:14:01<343:44:43,  5.86s/it]
  6%|▌         | 13663/225000 [38:14:12<431:12:40,  7.35s/it]
  6%|▌         | 13664/225000 [38:14:16<373:50:54,  6.37s/it]
  6%|▌         | 13665/225000 [38:14:23<382:41:28,  6.52s/it]
  6%|▌         | 13666/225000 [38:14:29<373:30:11,  6.36s/it]
  6%|▌         | 13667/225000 [38:14:34<342:32:18,  5.84s/it]
  6%|▌         | 13668/225000 [38:14:41<364:39:51,  6.21s/it]
  6%|▌         | 13669/225000 [38:14:46<341:06:54,  5.81s/it]
  6%|▌         | 13670/225000 [38:14:58<457:33:08,  7.79s/it]
  6%|▌         | 13671/225000 [38:15:03<406:17:26,  6.92s/it]
  6%|▌         | 13672/225000 [38:15:08<376:21:16,  6.41s/it]
  6%|▌         | 13673/225000 [38:15:16<399:16:39,  6.80s/it]
  6%|▌         | 13674/225000 [38:15:26<452:28:45,  7.71s/it]
  6%|▌         | 13675/225000 [38:15:33<444:22:15,  7.57s/it]
  6%|▌         | 13676/225000 [38:15:37<382:09:52,  6.51s/it]
  6%|▌         | 13677/225000 [38:15:46<423:12:38,  7.21s/it]
  6%|▌         | 13678/225000 [38:15:55<448:52:32,  7.65s/it]
  6%|▌         | 13679/225000 [38:15:59<387:57:09,  6.61s/it]
  6%|▌         | 13680/225000 [38:16:07<411:00:36,  7.00s/it]
  6%|▌         | 13681/225000 [38:16:10<353:55:30,  6.03s/it]
  6%|▌         | 13682/225000 [38:16:18<384:24:35,  6.55s/it]
  6%|▌         | 13683/225000 [38:16:23<359:49:59,  6.13s/it]
  6%|▌         | 13684/225000 [38:16:31<385:19:40,  6.56s/it]
  6%|▌         | 13685/225000 [38:16:35<338:09:15,  5.76s/it]
  6%|▌         | 13686/225000 [38:16:44<398:56:18,  6.80s/it]
  6%|▌         | 13687/225000 [38:16:48<347:00:26,  5.91s/it]
  6%|▌         | 13688/225000 [38:16:54<344:45:02,  5.87s/it]
  6%|▌         | 13689/225000 [38:17:00<349:58:43,  5.96s/it]
  6%|▌         | 13690/225000 [38:17:05<341:29:52,  5.82s/it]
  6%|▌         | 13691/225000 [38:17:11<336:20:16,  5.73s/it]
  6%|▌         | 13692/225000 [38:17:17<334:34:06,  5.70s/it]
  6%|▌         | 13693/225000 [38:17:27<416:33:09,  7.10s/it]
  6%|▌         | 13694/225000 [38:17:34<421:31:06,  7.18s/it]
  6%|▌         | 13695/225000 [38:17:40<392:09:41,  6.68s/it]
  6%|▌         | 13696/225000 [38:17:48<415:28:54,  7.08s/it]
  6%|▌         | 13697/225000 [38:17:55<412:07:06,  7.02s/it]
  6%|▌         | 13698/225000 [38:18:05<461:49:42,  7.87s/it]
  6%|▌         | 13699/225000 [38:18:11<430:25:50,  7.33s/it]
  6%|▌         | 13700/225000 [38:18:16<390:15:11,  6.65s/it]
                                                             
{'loss': '0.7511', 'grad_norm': '0.6951', 'learning_rate': '4.996e-05', 'epoch': '0.06089', 'num_input_tokens_seen': 68271416, 'train_runtime': '1.379e+05', 'train_tokens_per_second': '495.1'}

  6%|▌         | 13700/225000 [38:18:16<390:15:11,  6.65s/it]
  6%|▌         | 13701/225000 [38:18:24<415:59:49,  7.09s/it]
  6%|▌         | 13702/225000 [38:18:34<477:37:17,  8.14s/it]
  6%|▌         | 13703/225000 [38:18:39<414:24:03,  7.06s/it]
  6%|▌         | 13704/225000 [38:18:43<364:20:08,  6.21s/it]
  6%|▌         | 13705/225000 [38:18:52<405:40:00,  6.91s/it]
  6%|▌         | 13706/225000 [38:18:59<411:09:24,  7.01s/it]
  6%|▌         | 13707/225000 [38:19:05<389:45:01,  6.64s/it]
  6%|▌         | 13708/225000 [38:19:10<375:11:41,  6.39s/it]
  6%|▌         | 13709/225000 [38:19:17<374:21:21,  6.38s/it]
  6%|▌         | 13710/225000 [38:19:21<340:50:44,  5.81s/it]
  6%|▌         | 13711/225000 [38:19:26<319:14:35,  5.44s/it]
  6%|▌         | 13712/225000 [38:19:37<420:55:43,  7.17s/it]
  6%|▌         | 13713/225000 [38:19:42<375:13:57,  6.39s/it]
  6%|▌         | 13714/225000 [38:19:48<369:49:03,  6.30s/it]
  6%|▌         | 13715/225000 [38:19:53<353:56:24,  6.03s/it]
  6%|▌         | 13716/225000 [38:20:03<417:27:56,  7.11s/it]
  6%|▌         | 13717/225000 [38:20:07<374:12:03,  6.38s/it]
  6%|▌         | 13718/225000 [38:20:12<342:51:58,  5.84s/it]
  6%|▌         | 13719/225000 [38:20:17<329:03:40,  5.61s/it]
  6%|▌         | 13720/225000 [38:20:24<360:14:26,  6.14s/it]
  6%|▌         | 13721/225000 [38:20:33<403:51:00,  6.88s/it]
  6%|▌         | 13722/225000 [38:20:37<356:31:07,  6.07s/it]
  6%|▌         | 13723/225000 [38:20:44<372:14:57,  6.34s/it]
  6%|▌         | 13724/225000 [38:20:52<400:32:56,  6.83s/it]
  6%|▌         | 13725/225000 [38:21:02<444:37:18,  7.58s/it]
  6%|▌         | 13726/225000 [38:21:06<381:36:47,  6.50s/it]
  6%|▌         | 13727/225000 [38:21:14<422:36:24,  7.20s/it]
  6%|▌         | 13728/225000 [38:21:19<380:18:10,  6.48s/it]
  6%|▌         | 13729/225000 [38:21:27<401:31:51,  6.84s/it]
  6%|▌         | 13730/225000 [38:21:34<405:49:15,  6.92s/it]
  6%|▌         | 13731/225000 [38:21:38<360:25:50,  6.14s/it]
  6%|▌         | 13732/225000 [38:21:43<333:08:45,  5.68s/it]
  6%|▌         | 13733/225000 [38:21:58<505:41:21,  8.62s/it]
  6%|▌         | 13734/225000 [38:22:03<431:07:41,  7.35s/it]
  6%|▌         | 13735/225000 [38:22:09<419:01:40,  7.14s/it]
  6%|▌         | 13736/225000 [38:22:15<392:50:34,  6.69s/it]
  6%|▌         | 13737/225000 [38:22:22<398:10:02,  6.78s/it]
  6%|▌         | 13738/225000 [38:22:28<388:01:51,  6.61s/it]
  6%|▌         | 13739/225000 [38:22:35<388:14:36,  6.62s/it]
  6%|▌         | 13740/225000 [38:22:47<478:33:47,  8.16s/it]
  6%|▌         | 13741/225000 [38:22:50<400:23:12,  6.82s/it]
  6%|▌         | 13742/225000 [38:22:56<382:33:53,  6.52s/it]
  6%|▌         | 13743/225000 [38:23:02<365:00:18,  6.22s/it]
  6%|▌         | 13744/225000 [38:23:07<341:14:49,  5.82s/it]
  6%|▌         | 13745/225000 [38:23:12<332:43:00,  5.67s/it]
  6%|▌         | 13746/225000 [38:23:29<526:52:12,  8.98s/it]
  6%|▌         | 13747/225000 [38:23:33<445:40:24,  7.59s/it]
  6%|▌         | 13748/225000 [38:23:38<395:49:19,  6.75s/it]
  6%|▌         | 13749/225000 [38:23:43<365:07:32,  6.22s/it]
  6%|▌         | 13750/225000 [38:23:48<348:54:54,  5.95s/it]
                                                             
{'loss': '0.735', 'grad_norm': '0.5687', 'learning_rate': '4.996e-05', 'epoch': '0.06111', 'num_input_tokens_seen': 68429744, 'train_runtime': '1.382e+05', 'train_tokens_per_second': '495'}

  6%|▌         | 13750/225000 [38:23:48<348:54:54,  5.95s/it]
  6%|▌         | 13751/225000 [38:23:58<414:57:25,  7.07s/it]
  6%|▌         | 13752/225000 [38:24:04<404:44:29,  6.90s/it]
  6%|▌         | 13753/225000 [38:24:09<369:32:47,  6.30s/it]
  6%|▌         | 13754/225000 [38:24:18<408:46:58,  6.97s/it]
  6%|▌         | 13755/225000 [38:24:22<367:38:16,  6.27s/it]
  6%|▌         | 13756/225000 [38:24:31<408:45:32,  6.97s/it]
  6%|▌         | 13757/225000 [38:24:37<395:42:25,  6.74s/it]
  6%|▌         | 13758/225000 [38:24:46<430:25:10,  7.34s/it]
  6%|▌         | 13759/225000 [38:24:49<366:35:51,  6.25s/it]
  6%|▌         | 13760/225000 [38:24:58<403:52:17,  6.88s/it]
  6%|▌         | 13761/225000 [38:25:03<375:32:06,  6.40s/it]
  6%|▌         | 13762/225000 [38:25:09<362:44:14,  6.18s/it]
  6%|▌         | 13763/225000 [38:25:15<362:24:59,  6.18s/it]
  6%|▌         | 13764/225000 [38:25:20<336:52:58,  5.74s/it]
  6%|▌         | 13765/225000 [38:25:28<385:57:32,  6.58s/it]
  6%|▌         | 13766/225000 [38:25:33<359:02:28,  6.12s/it]
  6%|▌         | 13767/225000 [38:25:38<327:04:35,  5.57s/it]
  6%|▌         | 13768/225000 [38:25:46<385:00:04,  6.56s/it]
  6%|▌         | 13769/225000 [38:25:54<393:45:25,  6.71s/it]
  6%|▌         | 13770/225000 [38:26:01<401:09:10,  6.84s/it]
  6%|▌         | 13771/225000 [38:26:05<357:11:19,  6.09s/it]
  6%|▌         | 13772/225000 [38:26:09<323:12:40,  5.51s/it]
  6%|▌         | 13773/225000 [38:26:18<390:29:25,  6.66s/it]
  6%|▌         | 13774/225000 [38:26:27<418:57:28,  7.14s/it]
  6%|▌         | 13775/225000 [38:26:31<361:04:38,  6.15s/it]
  6%|▌         | 13776/225000 [38:26:35<331:33:59,  5.65s/it]
  6%|▌         | 13777/225000 [38:26:40<312:55:18,  5.33s/it]
  6%|▌         | 13778/225000 [38:26:54<475:37:07,  8.11s/it]
  6%|▌         | 13779/225000 [38:27:00<429:31:05,  7.32s/it]
  6%|▌         | 13780/225000 [38:27:05<385:57:36,  6.58s/it]
  6%|▌         | 13781/225000 [38:27:10<363:58:31,  6.20s/it]
  6%|▌         | 13782/225000 [38:27:17<381:39:24,  6.50s/it]
  6%|▌         | 13783/225000 [38:27:30<488:22:30,  8.32s/it]
  6%|▌         | 13784/225000 [38:27:34<417:38:23,  7.12s/it]
  6%|▌         | 13785/225000 [38:27:47<520:33:16,  8.87s/it]
  6%|▌         | 13786/225000 [38:27:53<478:12:25,  8.15s/it]
  6%|▌         | 13787/225000 [38:27:58<416:52:05,  7.11s/it]
  6%|▌         | 13788/225000 [38:28:03<374:42:16,  6.39s/it]
  6%|▌         | 13789/225000 [38:28:08<352:50:12,  6.01s/it]
  6%|▌         | 13790/225000 [38:28:13<329:37:49,  5.62s/it]
  6%|▌         | 13791/225000 [38:28:19<338:43:14,  5.77s/it]
  6%|▌         | 13792/225000 [38:28:31<456:04:40,  7.77s/it]
  6%|▌         | 13793/225000 [38:28:38<443:04:20,  7.55s/it]
  6%|▌         | 13794/225000 [38:28:42<370:21:05,  6.31s/it]
  6%|▌         | 13795/225000 [38:28:49<392:11:06,  6.68s/it]
  6%|▌         | 13796/225000 [38:28:57<406:38:38,  6.93s/it]
  6%|▌         | 13797/225000 [38:29:00<348:44:40,  5.94s/it]
  6%|▌         | 13798/225000 [38:29:07<360:26:44,  6.14s/it]
  6%|▌         | 13799/225000 [38:29:12<344:09:16,  5.87s/it]
  6%|▌         | 13800/225000 [38:29:19<361:11:10,  6.16s/it]
                                                             
{'loss': '0.7474', 'grad_norm': '0.5921', 'learning_rate': '4.996e-05', 'epoch': '0.06133', 'num_input_tokens_seen': 68584584, 'train_runtime': '1.386e+05', 'train_tokens_per_second': '495'}

  6%|▌         | 13800/225000 [38:29:19<361:11:10,  6.16s/it]
  6%|▌         | 13801/225000 [38:29:26<383:34:49,  6.54s/it]
  6%|▌         | 13802/225000 [38:29:31<347:20:22,  5.92s/it]
  6%|▌         | 13803/225000 [38:29:39<379:23:25,  6.47s/it]
  6%|▌         | 13804/225000 [38:29:44<365:56:40,  6.24s/it]
  6%|▌         | 13805/225000 [38:29:52<382:19:34,  6.52s/it]
  6%|▌         | 13806/225000 [38:29:56<350:25:18,  5.97s/it]
  6%|▌         | 13807/225000 [38:30:04<382:57:09,  6.53s/it]
  6%|▌         | 13808/225000 [38:30:10<372:28:57,  6.35s/it]
  6%|▌         | 13809/225000 [38:30:16<367:07:16,  6.26s/it]
  6%|▌         | 13810/225000 [38:30:21<349:23:37,  5.96s/it]
  6%|▌         | 13811/225000 [38:30:27<337:05:45,  5.75s/it]
  6%|▌         | 13812/225000 [38:30:32<329:32:24,  5.62s/it]
  6%|▌         | 13813/225000 [38:30:38<339:26:50,  5.79s/it]
  6%|▌         | 13814/225000 [38:30:54<521:58:37,  8.90s/it]
  6%|▌         | 13815/225000 [38:30:58<429:34:44,  7.32s/it]
  6%|▌         | 13816/225000 [38:31:06<442:48:47,  7.55s/it]
  6%|▌         | 13817/225000 [38:31:11<399:36:50,  6.81s/it]
  6%|▌         | 13818/225000 [38:31:16<374:36:32,  6.39s/it]
  6%|▌         | 13819/225000 [38:31:22<365:53:15,  6.24s/it]
  6%|▌         | 13820/225000 [38:31:32<430:18:49,  7.34s/it]
  6%|▌         | 13821/225000 [38:31:36<372:36:17,  6.35s/it]
  6%|▌         | 13822/225000 [38:31:42<364:06:29,  6.21s/it]
  6%|▌         | 13823/225000 [38:31:51<416:31:47,  7.10s/it]
  6%|▌         | 13824/225000 [38:31:57<383:41:10,  6.54s/it]
  6%|▌         | 13825/225000 [38:32:02<359:49:53,  6.13s/it]
  6%|▌         | 13826/225000 [38:32:06<335:23:03,  5.72s/it]
  6%|▌         | 13827/225000 [38:32:19<458:50:03,  7.82s/it]
  6%|▌         | 13828/225000 [38:32:23<386:02:02,  6.58s/it]
  6%|▌         | 13829/225000 [38:32:31<408:38:12,  6.97s/it]
  6%|▌         | 13830/225000 [38:32:35<362:33:12,  6.18s/it]
  6%|▌         | 13831/225000 [38:32:41<357:43:31,  6.10s/it]
  6%|▌         | 13832/225000 [38:32:46<332:46:04,  5.67s/it]
  6%|▌         | 13833/225000 [38:32:57<437:59:07,  7.47s/it]
  6%|▌         | 13834/225000 [38:33:05<438:19:14,  7.47s/it]
  6%|▌         | 13835/225000 [38:33:12<431:32:53,  7.36s/it]
  6%|▌         | 13836/225000 [38:33:17<384:35:22,  6.56s/it]
  6%|▌         | 13837/225000 [38:33:22<361:12:21,  6.16s/it]
  6%|▌         | 13838/225000 [38:33:27<347:55:25,  5.93s/it]
  6%|▌         | 13839/225000 [38:33:36<389:47:41,  6.65s/it]
  6%|▌         | 13840/225000 [38:33:46<458:05:57,  7.81s/it]
  6%|▌         | 13841/225000 [38:33:52<419:33:04,  7.15s/it]
  6%|▌         | 13842/225000 [38:33:59<414:12:01,  7.06s/it]
  6%|▌         | 13843/225000 [38:34:04<379:17:53,  6.47s/it]
  6%|▌         | 13844/225000 [38:34:08<343:35:24,  5.86s/it]
  6%|▌         | 13845/225000 [38:34:22<494:24:03,  8.43s/it]
  6%|▌         | 13846/225000 [38:34:27<421:05:45,  7.18s/it]
  6%|▌         | 13847/225000 [38:34:34<419:07:22,  7.15s/it]
  6%|▌         | 13848/225000 [38:34:38<364:37:55,  6.22s/it]
  6%|▌         | 13849/225000 [38:34:42<334:31:25,  5.70s/it]
  6%|▌         | 13850/225000 [38:34:53<414:16:17,  7.06s/it]
                                                             
{'loss': '0.7742', 'grad_norm': '0.6312', 'learning_rate': '4.996e-05', 'epoch': '0.06156', 'num_input_tokens_seen': 68742000, 'train_runtime': '1.389e+05', 'train_tokens_per_second': '494.9'}

  6%|▌         | 13850/225000 [38:34:53<414:16:17,  7.06s/it]
  6%|▌         | 13851/225000 [38:34:59<396:51:16,  6.77s/it]
  6%|▌         | 13852/225000 [38:35:05<393:04:28,  6.70s/it]
  6%|▌         | 13853/225000 [38:35:12<393:21:12,  6.71s/it]
  6%|▌         | 13854/225000 [38:35:18<389:41:38,  6.64s/it]
  6%|▌         | 13855/225000 [38:35:23<351:03:16,  5.99s/it]
  6%|▌         | 13856/225000 [38:35:28<328:43:09,  5.60s/it]
  6%|▌         | 13857/225000 [38:35:38<415:04:45,  7.08s/it]
  6%|▌         | 13858/225000 [38:35:46<430:00:52,  7.33s/it]
  6%|▌         | 13859/225000 [38:35:51<380:36:01,  6.49s/it]
  6%|▌         | 13860/225000 [38:35:56<367:03:50,  6.26s/it]
  6%|▌         | 13861/225000 [38:36:05<403:01:29,  6.87s/it]
  6%|▌         | 13862/225000 [38:36:10<380:35:09,  6.49s/it]
  6%|▌         | 13863/225000 [38:36:15<354:02:04,  6.04s/it]
  6%|▌         | 13864/225000 [38:36:25<419:24:55,  7.15s/it]
  6%|▌         | 13865/225000 [38:36:29<371:44:11,  6.34s/it]
  6%|▌         | 13866/225000 [38:36:38<410:37:51,  7.00s/it]
  6%|▌         | 13867/225000 [38:36:42<364:43:20,  6.22s/it]
  6%|▌         | 13868/225000 [38:36:47<341:14:44,  5.82s/it]
  6%|▌         | 13869/225000 [38:36:52<330:05:10,  5.63s/it]
  6%|▌         | 13870/225000 [38:36:58<326:11:52,  5.56s/it]
  6%|▌         | 13871/225000 [38:37:06<366:24:15,  6.25s/it]
  6%|▌         | 13872/225000 [38:37:15<428:05:37,  7.30s/it]
  6%|▌         | 13873/225000 [38:37:20<382:47:52,  6.53s/it]
  6%|▌         | 13874/225000 [38:37:31<457:02:36,  7.79s/it]
  6%|▌         | 13875/225000 [38:37:36<407:14:26,  6.94s/it]
  6%|▌         | 13876/225000 [38:37:41<374:52:20,  6.39s/it]
  6%|▌         | 13877/225000 [38:37:46<359:02:10,  6.12s/it]
  6%|▌         | 13878/225000 [38:37:59<474:28:53,  8.09s/it]
  6%|▌         | 13879/225000 [38:38:05<438:51:01,  7.48s/it]
  6%|▌         | 13880/225000 [38:38:09<372:45:51,  6.36s/it]
  6%|▌         | 13881/225000 [38:38:16<391:39:22,  6.68s/it]
  6%|▌         | 13882/225000 [38:38:21<351:58:26,  6.00s/it]
  6%|▌         | 13883/225000 [38:38:28<375:45:26,  6.41s/it]
  6%|▌         | 13884/225000 [38:38:36<404:50:20,  6.90s/it]
  6%|▌         | 13885/225000 [38:38:40<358:57:02,  6.12s/it]
  6%|▌         | 13886/225000 [38:38:46<341:11:15,  5.82s/it]
  6%|▌         | 13887/225000 [38:38:50<318:54:40,  5.44s/it]
  6%|▌         | 13888/225000 [38:38:57<351:36:31,  6.00s/it]
  6%|▌         | 13889/225000 [38:39:06<395:47:33,  6.75s/it]
  6%|▌         | 13890/225000 [38:39:13<395:43:23,  6.75s/it]
  6%|▌         | 13891/225000 [38:39:18<369:34:37,  6.30s/it]
  6%|▌         | 13892/225000 [38:39:23<342:18:10,  5.84s/it]
  6%|▌         | 13893/225000 [38:39:33<428:09:14,  7.30s/it]
  6%|▌         | 13894/225000 [38:39:39<391:42:23,  6.68s/it]
  6%|▌         | 13895/225000 [38:39:44<361:09:20,  6.16s/it]
  6%|▌         | 13896/225000 [38:39:49<342:03:30,  5.83s/it]
  6%|▌         | 13897/225000 [38:39:53<311:59:11,  5.32s/it]
  6%|▌         | 13898/225000 [38:40:04<420:46:40,  7.18s/it]
  6%|▌         | 13899/225000 [38:40:09<378:57:05,  6.46s/it]
  6%|▌         | 13900/225000 [38:40:18<425:12:47,  7.25s/it]
                                                             
{'loss': '0.7548', 'grad_norm': '0.6394', 'learning_rate': '4.996e-05', 'epoch': '0.06178', 'num_input_tokens_seen': 68896896, 'train_runtime': '1.392e+05', 'train_tokens_per_second': '494.9'}

  6%|▌         | 13900/225000 [38:40:18<425:12:47,  7.25s/it]
  6%|▌         | 13901/225000 [38:40:26<433:28:30,  7.39s/it]
  6%|▌         | 13902/225000 [38:40:29<365:46:53,  6.24s/it]
  6%|▌         | 13903/225000 [38:40:37<384:38:18,  6.56s/it]
  6%|▌         | 13904/225000 [38:40:40<334:31:02,  5.70s/it]
  6%|▌         | 13905/225000 [38:40:51<415:37:49,  7.09s/it]
  6%|▌         | 13906/225000 [38:40:55<367:03:29,  6.26s/it]
  6%|▌         | 13907/225000 [38:41:04<407:15:17,  6.95s/it]
  6%|▌         | 13908/225000 [38:41:10<396:56:39,  6.77s/it]
  6%|▌         | 13909/225000 [38:41:21<473:41:30,  8.08s/it]
  6%|▌         | 13910/225000 [38:41:25<390:52:00,  6.67s/it]
  6%|▌         | 13911/225000 [38:41:34<441:21:19,  7.53s/it]
  6%|▌         | 13912/225000 [38:41:38<381:14:26,  6.50s/it]
  6%|▌         | 13913/225000 [38:41:44<367:05:21,  6.26s/it]
  6%|▌         | 13914/225000 [38:41:50<357:24:16,  6.10s/it]
  6%|▌         | 13915/225000 [38:42:00<426:32:29,  7.27s/it]
  6%|▌         | 13916/225000 [38:42:05<399:13:08,  6.81s/it]
  6%|▌         | 13917/225000 [38:42:17<490:00:52,  8.36s/it]
  6%|▌         | 13918/225000 [38:42:23<447:18:15,  7.63s/it]
  6%|▌         | 13919/225000 [38:42:29<423:14:36,  7.22s/it]
  6%|▌         | 13920/225000 [38:42:35<390:26:06,  6.66s/it]
  6%|▌         | 13921/225000 [38:42:41<381:22:31,  6.50s/it]
  6%|▌         | 13922/225000 [38:42:47<374:25:25,  6.39s/it]
  6%|▌         | 13923/225000 [38:42:53<367:17:22,  6.26s/it]
  6%|▌         | 13924/225000 [38:42:59<363:02:22,  6.19s/it]
  6%|▌         | 13925/225000 [38:43:05<361:31:02,  6.17s/it]
  6%|▌         | 13926/225000 [38:43:15<430:23:51,  7.34s/it]
  6%|▌         | 13927/225000 [38:43:21<397:59:40,  6.79s/it]
  6%|▌         | 13928/225000 [38:43:25<344:14:40,  5.87s/it]
  6%|▌         | 13929/225000 [38:43:32<367:43:04,  6.27s/it]
  6%|▌         | 13930/225000 [38:43:37<352:53:17,  6.02s/it]
  6%|▌         | 13931/225000 [38:43:46<394:44:03,  6.73s/it]
  6%|▌         | 13932/225000 [38:43:55<448:08:07,  7.64s/it]
  6%|▌         | 13933/225000 [38:44:05<480:39:31,  8.20s/it]
  6%|▌         | 13934/225000 [38:44:10<430:41:51,  7.35s/it]
  6%|▌         | 13935/225000 [38:44:22<517:47:23,  8.83s/it]
  6%|▌         | 13936/225000 [38:44:27<433:41:56,  7.40s/it]
  6%|▌         | 13937/225000 [38:44:31<380:50:35,  6.50s/it]
  6%|▌         | 13938/225000 [38:44:35<340:53:35,  5.81s/it]
  6%|▌         | 13939/225000 [38:44:44<402:59:39,  6.87s/it]
  6%|▌         | 13940/225000 [38:44:51<393:01:35,  6.70s/it]
  6%|▌         | 13941/225000 [38:44:57<383:49:35,  6.55s/it]
  6%|▌         | 13942/225000 [38:45:04<399:53:07,  6.82s/it]
  6%|▌         | 13943/225000 [38:45:09<358:59:10,  6.12s/it]
  6%|▌         | 13944/225000 [38:45:14<341:42:26,  5.83s/it]
  6%|▌         | 13945/225000 [38:45:26<441:21:44,  7.53s/it]
  6%|▌         | 13946/225000 [38:45:29<372:56:26,  6.36s/it]
  6%|▌         | 13947/225000 [38:45:37<397:22:58,  6.78s/it]
  6%|▌         | 13948/225000 [38:45:42<371:44:24,  6.34s/it]
  6%|▌         | 13949/225000 [38:45:48<360:08:52,  6.14s/it]
  6%|▌         | 13950/225000 [38:45:54<350:07:02,  5.97s/it]
                                                             
{'loss': '0.7543', 'grad_norm': '0.7364', 'learning_rate': '4.996e-05', 'epoch': '0.062', 'num_input_tokens_seen': 69054664, 'train_runtime': '1.396e+05', 'train_tokens_per_second': '494.8'}

  6%|▌         | 13950/225000 [38:45:54<350:07:02,  5.97s/it]
  6%|▌         | 13951/225000 [38:45:59<348:26:57,  5.94s/it]
  6%|▌         | 13952/225000 [38:46:05<340:59:07,  5.82s/it]
  6%|▌         | 13953/225000 [38:46:10<335:29:31,  5.72s/it]
  6%|▌         | 13954/225000 [38:46:16<340:27:59,  5.81s/it]
  6%|▌         | 13955/225000 [38:46:22<335:55:57,  5.73s/it]
  6%|▌         | 13956/225000 [38:46:28<334:29:09,  5.71s/it]
  6%|▌         | 13957/225000 [38:46:33<337:26:58,  5.76s/it]
  6%|▌         | 13958/225000 [38:46:39<327:57:59,  5.59s/it]
  6%|▌         | 13959/225000 [38:46:51<440:52:20,  7.52s/it]
  6%|▌         | 13960/225000 [38:47:00<477:13:00,  8.14s/it]
  6%|▌         | 13961/225000 [38:47:09<481:35:45,  8.22s/it]
  6%|▌         | 13962/225000 [38:47:13<409:09:46,  6.98s/it]
  6%|▌         | 13963/225000 [38:47:17<367:10:49,  6.26s/it]
  6%|▌         | 13964/225000 [38:47:24<374:54:14,  6.40s/it]
  6%|▌         | 13965/225000 [38:47:34<430:09:28,  7.34s/it]
  6%|▌         | 13966/225000 [38:47:38<373:03:35,  6.36s/it]
  6%|▌         | 13967/225000 [38:47:42<336:04:28,  5.73s/it]
  6%|▌         | 13968/225000 [38:47:53<422:47:38,  7.21s/it]
  6%|▌         | 13969/225000 [38:47:56<361:58:47,  6.18s/it]
  6%|▌         | 13970/225000 [38:48:02<344:30:26,  5.88s/it]
  6%|▌         | 13971/225000 [38:48:06<326:10:59,  5.56s/it]
  6%|▌         | 13972/225000 [38:48:17<413:05:29,  7.05s/it]
  6%|▌         | 13973/225000 [38:48:22<384:42:05,  6.56s/it]
  6%|▌         | 13974/225000 [38:48:29<377:43:19,  6.44s/it]
  6%|▌         | 13975/225000 [38:48:35<373:14:52,  6.37s/it]
  6%|▌         | 13976/225000 [38:48:40<358:48:34,  6.12s/it]
  6%|▌         | 13977/225000 [38:48:45<342:22:07,  5.84s/it]
  6%|▌         | 13978/225000 [38:48:57<442:47:44,  7.55s/it]
  6%|▌         | 13979/225000 [38:49:03<419:38:32,  7.16s/it]
  6%|▌         | 13980/225000 [38:49:09<396:42:41,  6.77s/it]
  6%|▌         | 13981/225000 [38:49:15<384:40:48,  6.56s/it]
  6%|▌         | 13982/225000 [38:49:22<381:14:54,  6.50s/it]
  6%|▌         | 13983/225000 [38:49:26<340:08:41,  5.80s/it]
  6%|▌         | 13984/225000 [38:49:31<331:04:50,  5.65s/it]
  6%|▌         | 13985/225000 [38:49:43<443:52:59,  7.57s/it]
  6%|▌         | 13986/225000 [38:49:47<380:02:02,  6.48s/it]
  6%|▌         | 13987/225000 [38:49:56<432:03:22,  7.37s/it]
  6%|▌         | 13988/225000 [38:50:04<435:08:23,  7.42s/it]
  6%|▌         | 13989/225000 [38:50:08<376:57:09,  6.43s/it]
  6%|▌         | 13990/225000 [38:50:12<337:11:56,  5.75s/it]
  6%|▌         | 13991/225000 [38:50:23<421:19:20,  7.19s/it]
  6%|▌         | 13992/225000 [38:50:28<394:07:08,  6.72s/it]
  6%|▌         | 13993/225000 [38:50:34<379:01:12,  6.47s/it]
  6%|▌         | 13994/225000 [38:50:42<404:57:37,  6.91s/it]
  6%|▌         | 13995/225000 [38:50:46<355:57:26,  6.07s/it]
  6%|▌         | 13996/225000 [38:50:56<415:05:23,  7.08s/it]
  6%|▌         | 13997/225000 [38:51:02<396:59:35,  6.77s/it]
  6%|▌         | 13998/225000 [38:51:07<374:25:29,  6.39s/it]
  6%|▌         | 13999/225000 [38:51:20<479:30:59,  8.18s/it]
  6%|▌         | 14000/225000 [38:51:24<402:09:00,  6.86s/it]
                                                             
{'loss': '0.7436', 'grad_norm': '0.7491', 'learning_rate': '4.996e-05', 'epoch': '0.06222', 'num_input_tokens_seen': 69210480, 'train_runtime': '1.399e+05', 'train_tokens_per_second': '494.8'}

  6%|▌         | 14000/225000 [38:51:24<402:09:00,  6.86s/it]
  6%|▌         | 14001/225000 [38:51:28<365:54:06,  6.24s/it]
  6%|▌         | 14002/225000 [38:51:34<349:42:32,  5.97s/it]
  6%|▌         | 14003/225000 [38:51:40<362:26:01,  6.18s/it]
  6%|▌         | 14004/225000 [38:51:50<426:35:20,  7.28s/it]
  6%|▌         | 14005/225000 [38:51:54<372:44:06,  6.36s/it]
  6%|▌         | 14006/225000 [38:52:00<363:04:27,  6.19s/it]
  6%|▌         | 14007/225000 [38:52:04<329:58:28,  5.63s/it]
  6%|▌         | 14008/225000 [38:52:16<435:02:53,  7.42s/it]
  6%|▌         | 14009/225000 [38:52:22<410:37:22,  7.01s/it]
  6%|▌         | 14010/225000 [38:52:26<361:00:43,  6.16s/it]
  6%|▌         | 14011/225000 [38:52:36<431:11:05,  7.36s/it]
  6%|▌         | 14012/225000 [38:52:41<373:14:36,  6.37s/it]
  6%|▌         | 14013/225000 [38:52:49<410:56:59,  7.01s/it]
  6%|▌         | 14014/225000 [38:52:53<358:43:15,  6.12s/it]
  6%|▌         | 14015/225000 [38:53:01<392:43:41,  6.70s/it]
  6%|▌         | 14016/225000 [38:53:05<347:31:19,  5.93s/it]
  6%|▌         | 14017/225000 [38:53:11<342:07:33,  5.84s/it]
  6%|▌         | 14018/225000 [38:53:19<373:45:47,  6.38s/it]
  6%|▌         | 14019/225000 [38:53:22<329:49:22,  5.63s/it]
  6%|▌         | 14020/225000 [38:53:32<402:38:48,  6.87s/it]
  6%|▌         | 14021/225000 [38:53:36<354:33:28,  6.05s/it]
  6%|▌         | 14022/225000 [38:53:46<411:53:10,  7.03s/it]
  6%|▌         | 14023/225000 [38:53:51<386:11:44,  6.59s/it]
  6%|▌         | 14024/225000 [38:53:57<365:58:11,  6.24s/it]
  6%|▌         | 14025/225000 [38:54:03<363:39:57,  6.21s/it]
  6%|▌         | 14026/225000 [38:54:08<353:46:55,  6.04s/it]
  6%|▌         | 14027/225000 [38:54:14<337:33:44,  5.76s/it]
  6%|▌         | 14028/225000 [38:54:19<338:48:22,  5.78s/it]
  6%|▌         | 14029/225000 [38:54:25<333:50:28,  5.70s/it]
  6%|▌         | 14030/225000 [38:54:30<331:23:23,  5.65s/it]
  6%|▌         | 14031/225000 [38:54:36<322:26:23,  5.50s/it]
  6%|▌         | 14032/225000 [38:54:46<406:51:11,  6.94s/it]
  6%|▌         | 14033/225000 [38:54:55<446:16:28,  7.62s/it]
  6%|▌         | 14034/225000 [38:55:03<445:28:24,  7.60s/it]
  6%|▌         | 14035/225000 [38:55:07<389:43:30,  6.65s/it]
  6%|▌         | 14036/225000 [38:55:17<456:24:34,  7.79s/it]
  6%|▌         | 14037/225000 [38:55:21<380:46:27,  6.50s/it]
  6%|▌         | 14038/225000 [38:55:25<338:42:11,  5.78s/it]
  6%|▌         | 14039/225000 [38:55:29<310:27:04,  5.30s/it]
  6%|▌         | 14040/225000 [38:55:42<433:13:39,  7.39s/it]
  6%|▌         | 14041/225000 [38:55:46<379:18:09,  6.47s/it]
  6%|▌         | 14042/225000 [38:55:50<347:00:22,  5.92s/it]
  6%|▌         | 14043/225000 [38:55:55<318:57:41,  5.44s/it]
  6%|▌         | 14044/225000 [38:56:08<453:38:20,  7.74s/it]
  6%|▌         | 14045/225000 [38:56:12<396:22:27,  6.76s/it]
  6%|▌         | 14046/225000 [38:56:17<360:04:56,  6.14s/it]
  6%|▌         | 14047/225000 [38:56:28<448:48:13,  7.66s/it]
  6%|▌         | 14048/225000 [38:56:33<395:16:40,  6.75s/it]
  6%|▌         | 14049/225000 [38:56:38<369:18:58,  6.30s/it]
  6%|▌         | 14050/225000 [38:56:44<352:21:11,  6.01s/it]
                                                             
{'loss': '0.7229', 'grad_norm': '0.667', 'learning_rate': '4.996e-05', 'epoch': '0.06244', 'num_input_tokens_seen': 69366688, 'train_runtime': '1.402e+05', 'train_tokens_per_second': '494.8'}

  6%|▌         | 14050/225000 [38:56:44<352:21:11,  6.01s/it]
  6%|▌         | 14051/225000 [38:56:49<337:26:38,  5.76s/it]
  6%|▌         | 14052/225000 [38:57:01<445:42:28,  7.61s/it]
  6%|▌         | 14053/225000 [38:57:07<417:10:34,  7.12s/it]
  6%|▌         | 14054/225000 [38:57:12<383:38:45,  6.55s/it]
  6%|▌         | 14055/225000 [38:57:17<354:23:48,  6.05s/it]
  6%|▌         | 14056/225000 [38:57:22<340:14:47,  5.81s/it]
  6%|▌         | 14057/225000 [38:57:32<414:12:53,  7.07s/it]
  6%|▌         | 14058/225000 [38:57:40<440:31:59,  7.52s/it]
  6%|▌         | 14059/225000 [38:57:47<423:12:44,  7.22s/it]
  6%|▌         | 14060/225000 [38:57:54<423:36:11,  7.23s/it]
  6%|▌         | 14061/225000 [38:58:00<388:38:12,  6.63s/it]
  6%|▌         | 14062/225000 [38:58:04<359:49:46,  6.14s/it]
  6%|▋         | 14063/225000 [38:58:11<367:55:55,  6.28s/it]
  6%|▋         | 14064/225000 [38:58:20<408:03:25,  6.96s/it]
  6%|▋         | 14065/225000 [38:58:23<349:08:14,  5.96s/it]
  6%|▋         | 14066/225000 [38:58:30<370:56:48,  6.33s/it]
  6%|▋         | 14067/225000 [38:58:35<344:40:08,  5.88s/it]
  6%|▋         | 14068/225000 [38:58:40<330:49:54,  5.65s/it]
  6%|▋         | 14069/225000 [38:58:45<319:11:04,  5.45s/it]
  6%|▋         | 14070/225000 [38:58:58<447:32:13,  7.64s/it]
  6%|▋         | 14071/225000 [38:59:02<378:14:38,  6.46s/it]
  6%|▋         | 14072/225000 [38:59:10<413:31:33,  7.06s/it]
  6%|▋         | 14073/225000 [38:59:14<353:29:45,  6.03s/it]
  6%|▋         | 14074/225000 [38:59:21<377:55:10,  6.45s/it]
  6%|▋         | 14075/225000 [38:59:26<340:09:19,  5.81s/it]
  6%|▋         | 14076/225000 [38:59:29<302:19:13,  5.16s/it]
  6%|▋         | 14077/225000 [38:59:40<393:59:23,  6.72s/it]
  6%|▋         | 14078/225000 [38:59:44<343:01:10,  5.85s/it]
  6%|▋         | 14079/225000 [38:59:48<313:42:22,  5.35s/it]
  6%|▋         | 14080/225000 [38:59:55<341:49:15,  5.83s/it]
  6%|▋         | 14081/225000 [39:00:04<405:59:24,  6.93s/it]
  6%|▋         | 14082/225000 [39:00:09<372:17:08,  6.35s/it]
  6%|▋         | 14083/225000 [39:00:20<447:20:11,  7.64s/it]
  6%|▋         | 14084/225000 [39:00:24<382:34:49,  6.53s/it]
  6%|▋         | 14085/225000 [39:00:28<347:02:05,  5.92s/it]
  6%|▋         | 14086/225000 [39:00:33<334:23:20,  5.71s/it]
  6%|▋         | 14087/225000 [39:00:39<331:52:21,  5.66s/it]
  6%|▋         | 14088/225000 [39:00:45<336:08:13,  5.74s/it]
  6%|▋         | 14089/225000 [39:00:55<410:11:21,  7.00s/it]
  6%|▋         | 14090/225000 [39:01:06<474:20:45,  8.10s/it]
  6%|▋         | 14091/225000 [39:01:10<408:04:33,  6.97s/it]
  6%|▋         | 14092/225000 [39:01:22<499:14:24,  8.52s/it]
  6%|▋         | 14093/225000 [39:01:29<478:59:22,  8.18s/it]
  6%|▋         | 14094/225000 [39:01:35<431:09:53,  7.36s/it]
  6%|▋         | 14095/225000 [39:01:40<401:24:56,  6.85s/it]
  6%|▋         | 14096/225000 [39:01:46<377:33:52,  6.44s/it]
  6%|▋         | 14097/225000 [39:01:52<365:41:39,  6.24s/it]
  6%|▋         | 14098/225000 [39:02:04<473:17:17,  8.08s/it]
  6%|▋         | 14099/225000 [39:02:10<436:08:55,  7.44s/it]
  6%|▋         | 14100/225000 [39:02:16<402:48:06,  6.88s/it]
                                                             
{'loss': '0.7221', 'grad_norm': '0.6728', 'learning_rate': '4.996e-05', 'epoch': '0.06267', 'num_input_tokens_seen': 69523632, 'train_runtime': '1.405e+05', 'train_tokens_per_second': '494.7'}

  6%|▋         | 14100/225000 [39:02:16<402:48:06,  6.88s/it]
  6%|▋         | 14101/225000 [39:02:21<380:48:06,  6.50s/it]
  6%|▋         | 14102/225000 [39:02:27<363:10:09,  6.20s/it]
  6%|▋         | 14103/225000 [39:02:32<349:24:03,  5.96s/it]
  6%|▋         | 14104/225000 [39:02:37<337:41:46,  5.76s/it]
  6%|▋         | 14105/225000 [39:02:43<333:46:10,  5.70s/it]
  6%|▋         | 14106/225000 [39:02:49<337:30:15,  5.76s/it]
  6%|▋         | 14107/225000 [39:02:55<340:40:24,  5.82s/it]
  6%|▋         | 14108/225000 [39:03:01<341:20:02,  5.83s/it]
  6%|▋         | 14109/225000 [39:03:15<491:30:34,  8.39s/it]
  6%|▋         | 14110/225000 [39:03:19<404:08:48,  6.90s/it]
  6%|▋         | 14111/225000 [39:03:27<432:30:15,  7.38s/it]
  6%|▋         | 14112/225000 [39:03:32<388:28:14,  6.63s/it]
  6%|▋         | 14113/225000 [39:03:37<355:40:46,  6.07s/it]
  6%|▋         | 14114/225000 [39:03:41<332:28:15,  5.68s/it]
  6%|▋         | 14115/225000 [39:03:50<391:30:36,  6.68s/it]
  6%|▋         | 14116/225000 [39:03:57<393:43:15,  6.72s/it]
  6%|▋         | 14117/225000 [39:04:04<391:23:27,  6.68s/it]
  6%|▋         | 14118/225000 [39:04:10<380:25:20,  6.49s/it]
  6%|▋         | 14119/225000 [39:04:14<343:05:08,  5.86s/it]
  6%|▋         | 14120/225000 [39:04:23<393:09:30,  6.71s/it]
  6%|▋         | 14121/225000 [39:04:27<342:31:55,  5.85s/it]
  6%|▋         | 14122/225000 [39:04:40<477:19:24,  8.15s/it]
  6%|▋         | 14123/225000 [39:04:45<408:27:01,  6.97s/it]
  6%|▋         | 14124/225000 [39:04:50<380:22:27,  6.49s/it]
  6%|▋         | 14125/225000 [39:05:00<443:11:26,  7.57s/it]
  6%|▋         | 14126/225000 [39:05:07<440:06:06,  7.51s/it]
  6%|▋         | 14127/225000 [39:05:12<387:20:32,  6.61s/it]
  6%|▋         | 14128/225000 [39:05:19<402:10:26,  6.87s/it]
  6%|▋         | 14129/225000 [39:05:28<440:26:10,  7.52s/it]
  6%|▋         | 14130/225000 [39:05:33<389:33:07,  6.65s/it]
  6%|▋         | 14131/225000 [39:05:38<351:29:33,  6.00s/it]
  6%|▋         | 14132/225000 [39:05:42<325:38:08,  5.56s/it]
  6%|▋         | 14133/225000 [39:05:48<335:36:52,  5.73s/it]
  6%|▋         | 14134/225000 [39:06:00<438:57:50,  7.49s/it]
  6%|▋         | 14135/225000 [39:06:04<378:57:47,  6.47s/it]
  6%|▋         | 14136/225000 [39:06:10<370:00:48,  6.32s/it]
  6%|▋         | 14137/225000 [39:06:15<358:35:05,  6.12s/it]
  6%|▋         | 14138/225000 [39:06:21<348:00:01,  5.94s/it]
  6%|▋         | 14139/225000 [39:06:34<480:23:13,  8.20s/it]
  6%|▋         | 14140/225000 [39:06:41<450:56:23,  7.70s/it]
  6%|▋         | 14141/225000 [39:06:46<398:55:09,  6.81s/it]
  6%|▋         | 14142/225000 [39:06:54<428:52:00,  7.32s/it]
  6%|▋         | 14143/225000 [39:06:59<385:30:24,  6.58s/it]
  6%|▋         | 14144/225000 [39:07:04<355:53:54,  6.08s/it]
  6%|▋         | 14145/225000 [39:07:16<460:13:17,  7.86s/it]
  6%|▋         | 14146/225000 [39:07:20<393:12:23,  6.71s/it]
  6%|▋         | 14147/225000 [39:07:24<352:11:04,  6.01s/it]
  6%|▋         | 14148/225000 [39:07:30<335:23:05,  5.73s/it]
  6%|▋         | 14149/225000 [39:07:38<385:05:40,  6.57s/it]
  6%|▋         | 14150/225000 [39:07:48<444:13:00,  7.58s/it]
                                                             
{'loss': '0.7511', 'grad_norm': '0.6293', 'learning_rate': '4.995e-05', 'epoch': '0.06289', 'num_input_tokens_seen': 69682088, 'train_runtime': '1.409e+05', 'train_tokens_per_second': '494.7'}

  6%|▋         | 14150/225000 [39:07:48<444:13:00,  7.58s/it]
  6%|▋         | 14151/225000 [39:07:53<400:34:20,  6.84s/it]
  6%|▋         | 14152/225000 [39:07:58<358:25:23,  6.12s/it]
  6%|▋         | 14153/225000 [39:08:02<333:44:23,  5.70s/it]
  6%|▋         | 14154/225000 [39:08:12<396:45:12,  6.77s/it]
  6%|▋         | 14155/225000 [39:08:18<399:34:57,  6.82s/it]
  6%|▋         | 14156/225000 [39:08:24<372:02:34,  6.35s/it]
  6%|▋         | 14157/225000 [39:08:31<396:34:53,  6.77s/it]
  6%|▋         | 14158/225000 [39:08:36<359:18:41,  6.14s/it]
  6%|▋         | 14159/225000 [39:08:47<444:17:04,  7.59s/it]
  6%|▋         | 14160/225000 [39:08:56<466:44:18,  7.97s/it]
  6%|▋         | 14161/225000 [39:09:00<399:21:48,  6.82s/it]
  6%|▋         | 14162/225000 [39:09:09<440:02:42,  7.51s/it]
  6%|▋         | 14163/225000 [39:09:14<393:15:23,  6.71s/it]
  6%|▋         | 14164/225000 [39:09:19<368:18:54,  6.29s/it]
  6%|▋         | 14165/225000 [39:09:25<349:22:57,  5.97s/it]
  6%|▋         | 14166/225000 [39:09:30<335:50:17,  5.73s/it]
  6%|▋         | 14167/225000 [39:09:42<453:41:49,  7.75s/it]
  6%|▋         | 14168/225000 [39:09:46<388:36:17,  6.64s/it]
  6%|▋         | 14169/225000 [39:09:53<394:46:46,  6.74s/it]
  6%|▋         | 14170/225000 [39:09:59<385:42:58,  6.59s/it]
  6%|▋         | 14171/225000 [39:10:05<361:45:37,  6.18s/it]
  6%|▋         | 14172/225000 [39:10:14<423:41:50,  7.23s/it]
  6%|▋         | 14173/225000 [39:10:22<433:29:54,  7.40s/it]
  6%|▋         | 14174/225000 [39:10:31<458:27:58,  7.83s/it]
  6%|▋         | 14175/225000 [39:10:39<466:53:57,  7.97s/it]
  6%|▋         | 14176/225000 [39:10:47<460:19:14,  7.86s/it]
  6%|▋         | 14177/225000 [39:10:52<407:35:16,  6.96s/it]
  6%|▋         | 14178/225000 [39:10:56<366:46:54,  6.26s/it]
  6%|▋         | 14179/225000 [39:11:07<445:06:33,  7.60s/it]
  6%|▋         | 14180/225000 [39:11:12<401:35:59,  6.86s/it]
  6%|▋         | 14181/225000 [39:11:17<366:44:28,  6.26s/it]
  6%|▋         | 14182/225000 [39:11:22<345:08:43,  5.89s/it]
  6%|▋         | 14183/225000 [39:11:33<431:05:42,  7.36s/it]
  6%|▋         | 14184/225000 [39:11:38<395:17:03,  6.75s/it]
  6%|▋         | 14185/225000 [39:11:44<372:16:23,  6.36s/it]
  6%|▋         | 14186/225000 [39:11:49<347:29:27,  5.93s/it]
  6%|▋         | 14187/225000 [39:11:53<327:47:46,  5.60s/it]
  6%|▋         | 14188/225000 [39:12:04<407:07:49,  6.95s/it]
  6%|▋         | 14189/225000 [39:12:13<448:19:16,  7.66s/it]
  6%|▋         | 14190/225000 [39:12:18<408:04:50,  6.97s/it]
  6%|▋         | 14191/225000 [39:12:25<409:59:22,  7.00s/it]
  6%|▋         | 14192/225000 [39:12:30<365:24:47,  6.24s/it]
  6%|▋         | 14193/225000 [39:12:39<419:37:10,  7.17s/it]
  6%|▋         | 14194/225000 [39:12:47<432:45:03,  7.39s/it]
  6%|▋         | 14195/225000 [39:12:51<373:19:52,  6.38s/it]
  6%|▋         | 14196/225000 [39:12:59<392:55:30,  6.71s/it]
  6%|▋         | 14197/225000 [39:13:04<364:49:45,  6.23s/it]
  6%|▋         | 14198/225000 [39:13:09<346:33:43,  5.92s/it]
  6%|▋         | 14199/225000 [39:13:15<345:31:37,  5.90s/it]
  6%|▋         | 14200/225000 [39:13:20<331:43:23,  5.67s/it]
                                                             
{'loss': '0.7415', 'grad_norm': '0.6025', 'learning_rate': '4.995e-05', 'epoch': '0.06311', 'num_input_tokens_seen': 69842248, 'train_runtime': '1.412e+05', 'train_tokens_per_second': '494.6'}

  6%|▋         | 14200/225000 [39:13:20<331:43:23,  5.67s/it]
  6%|▋         | 14201/225000 [39:13:25<325:09:19,  5.55s/it]
  6%|▋         | 14202/225000 [39:13:32<354:31:19,  6.05s/it]
  6%|▋         | 14203/225000 [39:13:44<446:36:09,  7.63s/it]
  6%|▋         | 14204/225000 [39:13:48<392:53:38,  6.71s/it]
  6%|▋         | 14205/225000 [39:13:54<369:22:45,  6.31s/it]
  6%|▋         | 14206/225000 [39:13:59<352:09:36,  6.01s/it]
  6%|▋         | 14207/225000 [39:14:04<339:22:07,  5.80s/it]
  6%|▋         | 14208/225000 [39:14:09<330:44:18,  5.65s/it]
  6%|▋         | 14209/225000 [39:14:17<362:38:30,  6.19s/it]
  6%|▋         | 14210/225000 [39:14:29<458:01:14,  7.82s/it]
  6%|▋         | 14211/225000 [39:14:32<388:43:39,  6.64s/it]
  6%|▋         | 14212/225000 [39:14:41<425:08:28,  7.26s/it]
  6%|▋         | 14213/225000 [39:14:45<371:35:28,  6.35s/it]
  6%|▋         | 14214/225000 [39:14:50<335:21:06,  5.73s/it]
  6%|▋         | 14215/225000 [39:15:01<433:59:21,  7.41s/it]
  6%|▋         | 14216/225000 [39:15:05<371:13:29,  6.34s/it]
  6%|▋         | 14217/225000 [39:15:14<414:43:57,  7.08s/it]
  6%|▋         | 14218/225000 [39:15:18<357:38:26,  6.11s/it]
  6%|▋         | 14219/225000 [39:15:26<403:12:17,  6.89s/it]
  6%|▋         | 14220/225000 [39:15:30<350:18:39,  5.98s/it]
  6%|▋         | 14221/225000 [39:15:35<336:50:27,  5.75s/it]
  6%|▋         | 14222/225000 [39:15:41<328:25:07,  5.61s/it]
  6%|▋         | 14223/225000 [39:15:46<324:28:34,  5.54s/it]
  6%|▋         | 14224/225000 [39:15:51<321:11:57,  5.49s/it]
  6%|▋         | 14225/225000 [39:16:03<436:42:43,  7.46s/it]
  6%|▋         | 14226/225000 [39:16:09<403:34:09,  6.89s/it]
  6%|▋         | 14227/225000 [39:16:17<427:38:46,  7.30s/it]
  6%|▋         | 14228/225000 [39:16:21<361:04:55,  6.17s/it]
  6%|▋         | 14229/225000 [39:16:26<350:49:34,  5.99s/it]
  6%|▋         | 14230/225000 [39:16:32<347:59:00,  5.94s/it]
  6%|▋         | 14231/225000 [39:16:38<338:12:45,  5.78s/it]
  6%|▋         | 14232/225000 [39:16:52<495:11:50,  8.46s/it]
  6%|▋         | 14233/225000 [39:16:56<407:45:08,  6.96s/it]
  6%|▋         | 14234/225000 [39:17:04<425:51:26,  7.27s/it]
  6%|▋         | 14235/225000 [39:17:10<400:41:18,  6.84s/it]
  6%|▋         | 14236/225000 [39:17:15<383:05:31,  6.54s/it]
  6%|▋         | 14237/225000 [39:17:21<359:05:30,  6.13s/it]
  6%|▋         | 14238/225000 [39:17:27<365:12:16,  6.24s/it]
  6%|▋         | 14239/225000 [39:17:33<351:12:27,  6.00s/it]
  6%|▋         | 14240/225000 [39:17:38<335:05:48,  5.72s/it]
  6%|▋         | 14241/225000 [39:17:43<332:06:18,  5.67s/it]
  6%|▋         | 14242/225000 [39:17:49<328:02:39,  5.60s/it]
  6%|▋         | 14243/225000 [39:18:06<533:34:16,  9.11s/it]
  6%|▋         | 14244/225000 [39:18:10<443:38:02,  7.58s/it]
  6%|▋         | 14245/225000 [39:18:17<439:00:56,  7.50s/it]
  6%|▋         | 14246/225000 [39:18:22<388:26:43,  6.64s/it]
  6%|▋         | 14247/225000 [39:18:27<356:19:59,  6.09s/it]
  6%|▋         | 14248/225000 [39:18:37<425:39:12,  7.27s/it]
  6%|▋         | 14249/225000 [39:18:41<378:54:39,  6.47s/it]
  6%|▋         | 14250/225000 [39:18:52<461:12:23,  7.88s/it]
                                                             
{'loss': '0.7423', 'grad_norm': '0.619', 'learning_rate': '4.995e-05', 'epoch': '0.06333', 'num_input_tokens_seen': 69997832, 'train_runtime': '1.415e+05', 'train_tokens_per_second': '494.6'}

  6%|▋         | 14250/225000 [39:18:52<461:12:23,  7.88s/it]
  6%|▋         | 14251/225000 [39:18:58<421:13:44,  7.20s/it]
  6%|▋         | 14252/225000 [39:19:02<362:41:56,  6.20s/it]
  6%|▋         | 14253/225000 [39:19:12<428:33:56,  7.32s/it]
  6%|▋         | 14254/225000 [39:19:16<379:47:45,  6.49s/it]
  6%|▋         | 14255/225000 [39:19:21<346:00:26,  5.91s/it]
  6%|▋         | 14256/225000 [39:19:25<320:06:35,  5.47s/it]
  6%|▋         | 14257/225000 [39:19:31<317:23:34,  5.42s/it]
  6%|▋         | 14258/225000 [39:19:36<312:38:32,  5.34s/it]
  6%|▋         | 14259/225000 [39:19:52<502:20:06,  8.58s/it]
  6%|▋         | 14260/225000 [39:19:56<417:37:57,  7.13s/it]
  6%|▋         | 14261/225000 [39:20:00<364:29:05,  6.23s/it]
  6%|▋         | 14262/225000 [39:20:07<380:46:43,  6.50s/it]
  6%|▋         | 14263/225000 [39:20:13<378:20:50,  6.46s/it]
  6%|▋         | 14264/225000 [39:20:19<362:54:49,  6.20s/it]
  6%|▋         | 14265/225000 [39:20:25<361:08:48,  6.17s/it]
  6%|▋         | 14266/225000 [39:20:33<399:48:29,  6.83s/it]
  6%|▋         | 14267/225000 [39:20:37<341:45:05,  5.84s/it]
  6%|▋         | 14268/225000 [39:20:43<337:39:28,  5.77s/it]
  6%|▋         | 14269/225000 [39:20:48<338:22:32,  5.78s/it]
  6%|▋         | 14270/225000 [39:20:59<432:21:19,  7.39s/it]
  6%|▋         | 14271/225000 [39:21:04<385:04:45,  6.58s/it]
  6%|▋         | 14272/225000 [39:21:09<353:30:23,  6.04s/it]
  6%|▋         | 14273/225000 [39:21:14<339:00:30,  5.79s/it]
  6%|▋         | 14274/225000 [39:21:21<351:25:28,  6.00s/it]
  6%|▋         | 14275/225000 [39:21:31<431:44:11,  7.38s/it]
  6%|▋         | 14276/225000 [39:21:40<451:24:00,  7.71s/it]
  6%|▋         | 14277/225000 [39:21:45<414:51:04,  7.09s/it]
  6%|▋         | 14278/225000 [39:21:50<380:07:17,  6.49s/it]
  6%|▋         | 14279/225000 [39:21:56<364:55:31,  6.23s/it]
  6%|▋         | 14280/225000 [39:22:01<344:51:07,  5.89s/it]
  6%|▋         | 14281/225000 [39:22:07<340:11:07,  5.81s/it]
  6%|▋         | 14282/225000 [39:22:13<340:43:22,  5.82s/it]
  6%|▋         | 14283/225000 [39:22:18<329:21:28,  5.63s/it]
  6%|▋         | 14284/225000 [39:22:23<320:55:24,  5.48s/it]
  6%|▋         | 14285/225000 [39:22:29<328:04:00,  5.60s/it]
  6%|▋         | 14286/225000 [39:22:42<457:15:32,  7.81s/it]
  6%|▋         | 14287/225000 [39:22:49<444:52:56,  7.60s/it]
  6%|▋         | 14288/225000 [39:22:53<376:38:17,  6.43s/it]
  6%|▋         | 14289/225000 [39:22:57<332:08:18,  5.67s/it]
  6%|▋         | 14290/225000 [39:23:07<408:56:30,  6.99s/it]
  6%|▋         | 14291/225000 [39:23:10<353:20:12,  6.04s/it]
  6%|▋         | 14292/225000 [39:23:19<398:26:24,  6.81s/it]
  6%|▋         | 14293/225000 [39:23:26<400:22:46,  6.84s/it]
  6%|▋         | 14294/225000 [39:23:37<478:15:41,  8.17s/it]
  6%|▋         | 14295/225000 [39:23:41<403:34:21,  6.90s/it]
  6%|▋         | 14296/225000 [39:23:46<364:37:06,  6.23s/it]
  6%|▋         | 14297/225000 [39:23:52<363:29:14,  6.21s/it]
  6%|▋         | 14298/225000 [39:23:57<348:07:38,  5.95s/it]
  6%|▋         | 14299/225000 [39:24:03<339:01:42,  5.79s/it]
  6%|▋         | 14300/225000 [39:24:08<333:44:01,  5.70s/it]
                                                             
{'loss': '0.7476', 'grad_norm': '0.6196', 'learning_rate': '4.995e-05', 'epoch': '0.06356', 'num_input_tokens_seen': 70152528, 'train_runtime': '1.418e+05', 'train_tokens_per_second': '494.6'}

  6%|▋         | 14300/225000 [39:24:08<333:44:01,  5.70s/it]
  6%|▋         | 14301/225000 [39:24:14<329:40:20,  5.63s/it]
  6%|▋         | 14302/225000 [39:24:24<416:43:53,  7.12s/it]
  6%|▋         | 14303/225000 [39:24:31<411:29:25,  7.03s/it]
  6%|▋         | 14304/225000 [39:24:40<443:52:15,  7.58s/it]
  6%|▋         | 14305/225000 [39:24:48<449:13:22,  7.68s/it]
  6%|▋         | 14306/225000 [39:24:51<369:30:57,  6.31s/it]
  6%|▋         | 14307/225000 [39:24:58<389:26:17,  6.65s/it]
  6%|▋         | 14308/225000 [39:25:06<398:56:24,  6.82s/it]
  6%|▋         | 14309/225000 [39:25:11<375:59:40,  6.42s/it]
  6%|▋         | 14310/225000 [39:25:19<392:54:24,  6.71s/it]
  6%|▋         | 14311/225000 [39:25:24<362:56:20,  6.20s/it]
  6%|▋         | 14312/225000 [39:25:28<337:54:11,  5.77s/it]
  6%|▋         | 14313/225000 [39:25:34<328:46:53,  5.62s/it]
  6%|▋         | 14314/225000 [39:25:39<319:16:13,  5.46s/it]
  6%|▋         | 14315/225000 [39:25:47<366:51:17,  6.27s/it]
  6%|▋         | 14316/225000 [39:25:55<394:24:23,  6.74s/it]
  6%|▋         | 14317/225000 [39:26:02<410:17:20,  7.01s/it]
  6%|▋         | 14318/225000 [39:26:07<362:06:48,  6.19s/it]
  6%|▋         | 14319/225000 [39:26:12<347:09:49,  5.93s/it]
  6%|▋         | 14320/225000 [39:26:18<342:11:44,  5.85s/it]
  6%|▋         | 14321/225000 [39:26:23<331:01:56,  5.66s/it]
  6%|▋         | 14322/225000 [39:26:28<326:09:46,  5.57s/it]
  6%|▋         | 14323/225000 [39:26:42<466:25:53,  7.97s/it]
  6%|▋         | 14324/225000 [39:26:49<457:22:55,  7.82s/it]
  6%|▋         | 14325/225000 [39:26:54<396:31:04,  6.78s/it]
  6%|▋         | 14326/225000 [39:27:04<456:02:49,  7.79s/it]
  6%|▋         | 14327/225000 [39:27:08<400:30:57,  6.84s/it]
  6%|▋         | 14328/225000 [39:27:13<360:56:26,  6.17s/it]
  6%|▋         | 14329/225000 [39:27:21<400:34:03,  6.85s/it]
  6%|▋         | 14330/225000 [39:27:29<419:02:51,  7.16s/it]
  6%|▋         | 14331/225000 [39:27:33<357:42:28,  6.11s/it]
  6%|▋         | 14332/225000 [39:27:41<399:14:29,  6.82s/it]
  6%|▋         | 14333/225000 [39:27:49<407:15:14,  6.96s/it]
  6%|▋         | 14334/225000 [39:27:55<395:08:51,  6.75s/it]
  6%|▋         | 14335/225000 [39:28:00<361:57:24,  6.19s/it]
  6%|▋         | 14336/225000 [39:28:05<340:45:08,  5.82s/it]
  6%|▋         | 14337/225000 [39:28:10<332:01:09,  5.67s/it]
  6%|▋         | 14338/225000 [39:28:15<319:02:15,  5.45s/it]
  6%|▋         | 14339/225000 [39:28:20<315:02:05,  5.38s/it]
  6%|▋         | 14340/225000 [39:28:33<448:37:02,  7.67s/it]
  6%|▋         | 14341/225000 [39:28:40<423:56:15,  7.24s/it]
  6%|▋         | 14342/225000 [39:28:45<386:53:14,  6.61s/it]
  6%|▋         | 14343/225000 [39:28:52<391:15:13,  6.69s/it]
  6%|▋         | 14344/225000 [39:28:56<344:32:37,  5.89s/it]
  6%|▋         | 14345/225000 [39:29:08<454:38:32,  7.77s/it]
  6%|▋         | 14346/225000 [39:29:17<476:25:17,  8.14s/it]
  6%|▋         | 14347/225000 [39:29:24<460:05:54,  7.86s/it]
  6%|▋         | 14348/225000 [39:29:30<430:34:46,  7.36s/it]
  6%|▋         | 14349/225000 [39:29:34<371:29:20,  6.35s/it]
  6%|▋         | 14350/225000 [39:29:45<448:31:25,  7.67s/it]
                                                             
{'loss': '0.7437', 'grad_norm': '0.8327', 'learning_rate': '4.995e-05', 'epoch': '0.06378', 'num_input_tokens_seen': 70309352, 'train_runtime': '1.422e+05', 'train_tokens_per_second': '494.5'}

  6%|▋         | 14350/225000 [39:29:45<448:31:25,  7.67s/it]
  6%|▋         | 14351/225000 [39:29:50<398:13:21,  6.81s/it]
  6%|▋         | 14352/225000 [39:29:54<349:28:30,  5.97s/it]
  6%|▋         | 14353/225000 [39:30:02<382:06:26,  6.53s/it]
  6%|▋         | 14354/225000 [39:30:08<386:57:08,  6.61s/it]
  6%|▋         | 14355/225000 [39:30:13<358:48:48,  6.13s/it]
  6%|▋         | 14356/225000 [39:30:18<329:17:51,  5.63s/it]
  6%|▋         | 14357/225000 [39:30:26<380:22:16,  6.50s/it]
  6%|▋         | 14358/225000 [39:30:32<358:34:17,  6.13s/it]
  6%|▋         | 14359/225000 [39:30:44<462:24:22,  7.90s/it]
  6%|▋         | 14360/225000 [39:30:48<396:22:00,  6.77s/it]
  6%|▋         | 14361/225000 [39:30:52<350:55:51,  6.00s/it]
  6%|▋         | 14362/225000 [39:31:02<427:50:58,  7.31s/it]
  6%|▋         | 14363/225000 [39:31:07<378:06:09,  6.46s/it]
  6%|▋         | 14364/225000 [39:31:12<355:58:21,  6.08s/it]
  6%|▋         | 14365/225000 [39:31:17<341:47:37,  5.84s/it]
  6%|▋         | 14366/225000 [39:31:23<336:58:44,  5.76s/it]
  6%|▋         | 14367/225000 [39:31:28<333:28:26,  5.70s/it]
  6%|▋         | 14368/225000 [39:31:34<330:05:19,  5.64s/it]
  6%|▋         | 14369/225000 [39:31:39<325:25:25,  5.56s/it]
  6%|▋         | 14370/225000 [39:31:53<472:37:50,  8.08s/it]
  6%|▋         | 14371/225000 [39:31:59<439:32:01,  7.51s/it]
  6%|▋         | 14372/225000 [39:32:06<422:25:35,  7.22s/it]
  6%|▋         | 14373/225000 [39:32:14<433:17:18,  7.41s/it]
  6%|▋         | 14374/225000 [39:32:21<432:33:40,  7.39s/it]
  6%|▋         | 14375/225000 [39:32:29<441:26:36,  7.55s/it]
  6%|▋         | 14376/225000 [39:32:34<394:38:12,  6.75s/it]
  6%|▋         | 14377/225000 [39:32:46<483:25:19,  8.26s/it]
  6%|▋         | 14378/225000 [39:32:52<443:20:34,  7.58s/it]
  6%|▋         | 14379/225000 [39:32:57<400:33:28,  6.85s/it]
  6%|▋         | 14380/225000 [39:33:03<379:08:09,  6.48s/it]
  6%|▋         | 14381/225000 [39:33:08<365:55:51,  6.25s/it]
  6%|▋         | 14382/225000 [39:33:18<423:08:46,  7.23s/it]
  6%|▋         | 14383/225000 [39:33:22<364:33:53,  6.23s/it]
  6%|▋         | 14384/225000 [39:33:27<348:05:47,  5.95s/it]
  6%|▋         | 14385/225000 [39:33:34<361:13:56,  6.17s/it]
  6%|▋         | 14386/225000 [39:33:39<338:59:15,  5.79s/it]
  6%|▋         | 14387/225000 [39:33:48<404:04:22,  6.91s/it]
  6%|▋         | 14388/225000 [39:33:54<381:55:10,  6.53s/it]
  6%|▋         | 14389/225000 [39:33:59<362:57:42,  6.20s/it]
  6%|▋         | 14390/225000 [39:34:07<384:04:41,  6.57s/it]
  6%|▋         | 14391/225000 [39:34:11<344:00:43,  5.88s/it]
  6%|▋         | 14392/225000 [39:34:23<448:57:38,  7.67s/it]
  6%|▋         | 14393/225000 [39:34:30<451:09:43,  7.71s/it]
  6%|▋         | 14394/225000 [39:34:35<401:53:48,  6.87s/it]
  6%|▋         | 14395/225000 [39:34:40<357:24:42,  6.11s/it]
  6%|▋         | 14396/225000 [39:34:45<336:15:05,  5.75s/it]
  6%|▋         | 14397/225000 [39:34:52<365:13:15,  6.24s/it]
  6%|▋         | 14398/225000 [39:35:01<420:39:47,  7.19s/it]
  6%|▋         | 14399/225000 [39:35:10<440:49:15,  7.54s/it]
  6%|▋         | 14400/225000 [39:35:16<412:02:38,  7.04s/it]
                                                             
{'loss': '0.7598', 'grad_norm': '0.6305', 'learning_rate': '4.995e-05', 'epoch': '0.064', 'num_input_tokens_seen': 70468096, 'train_runtime': '1.425e+05', 'train_tokens_per_second': '494.5'}

  6%|▋         | 14400/225000 [39:35:16<412:02:38,  7.04s/it]
  6%|▋         | 14401/225000 [39:35:21<380:51:04,  6.51s/it]
  6%|▋         | 14402/225000 [39:35:27<369:44:36,  6.32s/it]
  6%|▋         | 14403/225000 [39:35:32<349:17:41,  5.97s/it]
  6%|▋         | 14404/225000 [39:35:37<337:35:28,  5.77s/it]
  6%|▋         | 14405/225000 [39:35:43<337:33:51,  5.77s/it]
  6%|▋         | 14406/225000 [39:35:49<332:30:40,  5.68s/it]
  6%|▋         | 14407/225000 [39:35:54<329:36:06,  5.63s/it]
  6%|▋         | 14408/225000 [39:36:05<426:23:11,  7.29s/it]
  6%|▋         | 14409/225000 [39:36:13<439:08:03,  7.51s/it]
  6%|▋         | 14410/225000 [39:36:18<393:09:41,  6.72s/it]
  6%|▋         | 14411/225000 [39:36:23<361:12:29,  6.17s/it]
  6%|▋         | 14412/225000 [39:36:28<342:53:01,  5.86s/it]
  6%|▋         | 14413/225000 [39:36:33<320:22:53,  5.48s/it]
  6%|▋         | 14414/225000 [39:36:41<366:08:20,  6.26s/it]
  6%|▋         | 14415/225000 [39:36:51<436:31:01,  7.46s/it]
  6%|▋         | 14416/225000 [39:37:00<459:15:45,  7.85s/it]
  6%|▋         | 14417/225000 [39:37:05<406:30:50,  6.95s/it]
  6%|▋         | 14418/225000 [39:37:11<401:40:33,  6.87s/it]
  6%|▋         | 14419/225000 [39:37:18<402:06:40,  6.87s/it]
  6%|▋         | 14420/225000 [39:37:23<364:05:03,  6.22s/it]
  6%|▋         | 14421/225000 [39:37:28<340:26:28,  5.82s/it]
  6%|▋         | 14422/225000 [39:37:39<438:38:32,  7.50s/it]
  6%|▋         | 14423/225000 [39:37:48<454:43:03,  7.77s/it]
  6%|▋         | 14424/225000 [39:37:51<381:16:03,  6.52s/it]
  6%|▋         | 14425/225000 [39:38:00<412:52:44,  7.06s/it]
  6%|▋         | 14426/225000 [39:38:07<420:35:01,  7.19s/it]
  6%|▋         | 14427/225000 [39:38:12<374:15:03,  6.40s/it]
  6%|▋         | 14428/225000 [39:38:15<327:36:06,  5.60s/it]
  6%|▋         | 14429/225000 [39:38:23<356:53:04,  6.10s/it]
  6%|▋         | 14430/225000 [39:38:30<381:08:09,  6.52s/it]
  6%|▋         | 14431/225000 [39:38:34<338:56:24,  5.79s/it]
  6%|▋         | 14432/225000 [39:38:44<417:41:52,  7.14s/it]
  6%|▋         | 14433/225000 [39:38:52<415:47:51,  7.11s/it]
  6%|▋         | 14434/225000 [39:38:55<360:51:24,  6.17s/it]
  6%|▋         | 14435/225000 [39:38:59<318:42:33,  5.45s/it]
  6%|▋         | 14436/225000 [39:39:10<409:20:40,  7.00s/it]
  6%|▋         | 14437/225000 [39:39:15<381:25:38,  6.52s/it]
  6%|▋         | 14438/225000 [39:39:19<337:56:18,  5.78s/it]
  6%|▋         | 14439/225000 [39:39:29<413:57:24,  7.08s/it]
  6%|▋         | 14440/225000 [39:39:36<401:43:16,  6.87s/it]
  6%|▋         | 14441/225000 [39:39:44<416:48:18,  7.13s/it]
  6%|▋         | 14442/225000 [39:39:53<450:46:46,  7.71s/it]
  6%|▋         | 14443/225000 [39:40:01<457:03:33,  7.81s/it]
  6%|▋         | 14444/225000 [39:40:05<393:54:12,  6.73s/it]
  6%|▋         | 14445/225000 [39:40:10<359:12:48,  6.14s/it]
  6%|▋         | 14446/225000 [39:40:18<397:54:21,  6.80s/it]
  6%|▋         | 14447/225000 [39:40:24<390:14:32,  6.67s/it]
  6%|▋         | 14448/225000 [39:40:29<359:41:12,  6.15s/it]
  6%|▋         | 14449/225000 [39:40:36<361:28:06,  6.18s/it]
  6%|▋         | 14450/225000 [39:40:42<369:40:31,  6.32s/it]
                                                             
{'loss': '0.7462', 'grad_norm': '0.6279', 'learning_rate': '4.995e-05', 'epoch': '0.06422', 'num_input_tokens_seen': 70623656, 'train_runtime': '1.428e+05', 'train_tokens_per_second': '494.4'}

  6%|▋         | 14450/225000 [39:40:42<369:40:31,  6.32s/it]
  6%|▋         | 14451/225000 [39:40:48<362:02:29,  6.19s/it]
  6%|▋         | 14452/225000 [39:40:54<354:32:06,  6.06s/it]
  6%|▋         | 14453/225000 [39:40:59<339:12:23,  5.80s/it]
  6%|▋         | 14454/225000 [39:41:05<338:55:01,  5.79s/it]
  6%|▋         | 14455/225000 [39:41:15<412:02:50,  7.05s/it]
  6%|▋         | 14456/225000 [39:41:19<369:23:52,  6.32s/it]
  6%|▋         | 14457/225000 [39:41:29<425:11:09,  7.27s/it]
  6%|▋         | 14458/225000 [39:41:34<383:47:22,  6.56s/it]
  6%|▋         | 14459/225000 [39:41:41<388:48:35,  6.65s/it]
  6%|▋         | 14460/225000 [39:41:47<387:33:23,  6.63s/it]
  6%|▋         | 14461/225000 [39:41:53<371:10:24,  6.35s/it]
  6%|▋         | 14462/225000 [39:42:00<389:14:54,  6.66s/it]
  6%|▋         | 14463/225000 [39:42:08<408:15:34,  6.98s/it]
  6%|▋         | 14464/225000 [39:42:12<349:26:54,  5.98s/it]
  6%|▋         | 14465/225000 [39:42:16<327:15:31,  5.60s/it]
  6%|▋         | 14466/225000 [39:42:21<311:28:11,  5.33s/it]
  6%|▋         | 14467/225000 [39:42:27<318:27:09,  5.45s/it]
  6%|▋         | 14468/225000 [39:42:40<450:05:31,  7.70s/it]
  6%|▋         | 14469/225000 [39:42:43<380:42:20,  6.51s/it]
  6%|▋         | 14470/225000 [39:42:52<409:21:32,  7.00s/it]
  6%|▋         | 14471/225000 [39:42:55<353:34:30,  6.05s/it]
  6%|▋         | 14472/225000 [39:43:01<352:06:43,  6.02s/it]
  6%|▋         | 14473/225000 [39:43:07<336:24:59,  5.75s/it]
  6%|▋         | 14474/225000 [39:43:12<330:01:07,  5.64s/it]
  6%|▋         | 14475/225000 [39:43:18<329:24:36,  5.63s/it]
  6%|▋         | 14476/225000 [39:43:23<331:52:19,  5.68s/it]
  6%|▋         | 14477/225000 [39:43:29<334:23:13,  5.72s/it]
  6%|▋         | 14478/225000 [39:43:35<337:22:49,  5.77s/it]
  6%|▋         | 14479/225000 [39:43:41<338:13:18,  5.78s/it]
  6%|▋         | 14480/225000 [39:43:53<457:21:25,  7.82s/it]
  6%|▋         | 14481/225000 [39:44:00<429:04:29,  7.34s/it]
  6%|▋         | 14482/225000 [39:44:08<451:56:42,  7.73s/it]
  6%|▋         | 14483/225000 [39:44:14<423:07:14,  7.24s/it]
  6%|▋         | 14484/225000 [39:44:23<445:11:09,  7.61s/it]
  6%|▋         | 14485/225000 [39:44:28<404:20:44,  6.91s/it]
  6%|▋         | 14486/225000 [39:44:34<381:15:41,  6.52s/it]
  6%|▋         | 14487/225000 [39:44:39<363:10:20,  6.21s/it]
  6%|▋         | 14488/225000 [39:44:45<360:50:52,  6.17s/it]
  6%|▋         | 14489/225000 [39:44:51<351:15:29,  6.01s/it]
  6%|▋         | 14490/225000 [39:44:57<346:30:44,  5.93s/it]
  6%|▋         | 14491/225000 [39:45:02<340:49:40,  5.83s/it]
  6%|▋         | 14492/225000 [39:45:08<331:50:04,  5.67s/it]
  6%|▋         | 14493/225000 [39:45:13<324:58:44,  5.56s/it]
  6%|▋         | 14494/225000 [39:45:28<485:56:15,  8.31s/it]
  6%|▋         | 14495/225000 [39:45:32<419:59:40,  7.18s/it]
  6%|▋         | 14496/225000 [39:45:37<374:07:09,  6.40s/it]
  6%|▋         | 14497/225000 [39:45:42<356:00:38,  6.09s/it]
  6%|▋         | 14498/225000 [39:45:52<417:18:49,  7.14s/it]
  6%|▋         | 14499/225000 [39:46:01<464:20:49,  7.94s/it]
  6%|▋         | 14500/225000 [39:46:06<410:40:33,  7.02s/it]
                                                             
{'loss': '0.7382', 'grad_norm': '0.6378', 'learning_rate': '4.995e-05', 'epoch': '0.06444', 'num_input_tokens_seen': 70780576, 'train_runtime': '1.432e+05', 'train_tokens_per_second': '494.4'}

  6%|▋         | 14500/225000 [39:46:06<410:40:33,  7.02s/it]
  6%|▋         | 14501/225000 [39:46:12<378:45:19,  6.48s/it]
  6%|▋         | 14502/225000 [39:46:17<354:40:54,  6.07s/it]
  6%|▋         | 14503/225000 [39:46:22<341:53:11,  5.85s/it]
  6%|▋         | 14504/225000 [39:46:27<334:35:55,  5.72s/it]
  6%|▋         | 14505/225000 [39:46:39<437:54:12,  7.49s/it]
  6%|▋         | 14506/225000 [39:46:47<443:04:40,  7.58s/it]
  6%|▋         | 14507/225000 [39:46:56<470:06:24,  8.04s/it]
  6%|▋         | 14508/225000 [39:47:01<410:35:44,  7.02s/it]
  6%|▋         | 14509/225000 [39:47:05<371:13:05,  6.35s/it]
  6%|▋         | 14510/225000 [39:47:15<426:11:47,  7.29s/it]
  6%|▋         | 14511/225000 [39:47:21<410:16:21,  7.02s/it]
  6%|▋         | 14512/225000 [39:47:26<369:24:01,  6.32s/it]
  6%|▋         | 14513/225000 [39:47:30<335:04:30,  5.73s/it]
  6%|▋         | 14514/225000 [39:47:35<319:11:47,  5.46s/it]
  6%|▋         | 14515/225000 [39:47:40<314:16:28,  5.38s/it]
  6%|▋         | 14516/225000 [39:47:47<341:45:02,  5.85s/it]
  6%|▋         | 14517/225000 [39:47:56<400:10:33,  6.84s/it]
  6%|▋         | 14518/225000 [39:48:03<403:16:12,  6.90s/it]
  6%|▋         | 14519/225000 [39:48:09<378:22:16,  6.47s/it]
  6%|▋         | 14520/225000 [39:48:16<389:52:28,  6.67s/it]
  6%|▋         | 14521/225000 [39:48:26<446:54:36,  7.64s/it]
  6%|▋         | 14522/225000 [39:48:34<449:32:21,  7.69s/it]
  6%|▋         | 14523/225000 [39:48:40<419:38:35,  7.18s/it]
  6%|▋         | 14524/225000 [39:48:44<364:23:16,  6.23s/it]
  6%|▋         | 14525/225000 [39:48:51<389:13:18,  6.66s/it]
  6%|▋         | 14526/225000 [39:48:59<409:18:19,  7.00s/it]
  6%|▋         | 14527/225000 [39:49:03<357:03:13,  6.11s/it]
  6%|▋         | 14528/225000 [39:49:09<343:40:50,  5.88s/it]
  6%|▋         | 14529/225000 [39:49:16<376:12:32,  6.43s/it]
  6%|▋         | 14530/225000 [39:49:21<344:09:50,  5.89s/it]
  6%|▋         | 14531/225000 [39:49:29<382:58:21,  6.55s/it]
  6%|▋         | 14532/225000 [39:49:33<342:01:30,  5.85s/it]
  6%|▋         | 14533/225000 [39:49:38<320:37:39,  5.48s/it]
  6%|▋         | 14534/225000 [39:49:50<433:59:49,  7.42s/it]
  6%|▋         | 14535/225000 [39:49:55<401:13:31,  6.86s/it]
  6%|▋         | 14536/225000 [39:50:02<403:03:39,  6.89s/it]
  6%|▋         | 14537/225000 [39:50:10<420:38:46,  7.20s/it]
  6%|▋         | 14538/225000 [39:50:16<396:59:44,  6.79s/it]
  6%|▋         | 14539/225000 [39:50:21<365:49:17,  6.26s/it]
  6%|▋         | 14540/225000 [39:50:26<345:48:40,  5.92s/it]
  6%|▋         | 14541/225000 [39:50:39<474:02:51,  8.11s/it]
  6%|▋         | 14542/225000 [39:50:45<428:37:22,  7.33s/it]
  6%|▋         | 14543/225000 [39:50:50<386:22:43,  6.61s/it]
  6%|▋         | 14544/225000 [39:50:55<365:18:33,  6.25s/it]
  6%|▋         | 14545/225000 [39:51:00<343:03:03,  5.87s/it]
  6%|▋         | 14546/225000 [39:51:05<328:18:24,  5.62s/it]
  6%|▋         | 14547/225000 [39:51:11<325:04:24,  5.56s/it]
  6%|▋         | 14548/225000 [39:51:24<461:38:00,  7.90s/it]
  6%|▋         | 14549/225000 [39:51:29<409:36:16,  7.01s/it]
  6%|▋         | 14550/225000 [39:51:37<429:31:55,  7.35s/it]
                                                             
{'loss': '0.73', 'grad_norm': '0.642', 'learning_rate': '4.994e-05', 'epoch': '0.06467', 'num_input_tokens_seen': 70938864, 'train_runtime': '1.435e+05', 'train_tokens_per_second': '494.4'}

  6%|▋         | 14550/225000 [39:51:37<429:31:55,  7.35s/it]
  6%|▋         | 14551/225000 [39:51:42<393:08:34,  6.73s/it]
  6%|▋         | 14552/225000 [39:51:48<373:40:24,  6.39s/it]
  6%|▋         | 14553/225000 [39:51:56<404:24:10,  6.92s/it]
  6%|▋         | 14554/225000 [39:52:00<356:19:57,  6.10s/it]
  6%|▋         | 14555/225000 [39:52:06<353:32:52,  6.05s/it]
  6%|▋         | 14556/225000 [39:52:15<401:29:39,  6.87s/it]
  6%|▋         | 14557/225000 [39:52:19<348:22:30,  5.96s/it]
  6%|▋         | 14558/225000 [39:52:24<330:47:01,  5.66s/it]
  6%|▋         | 14559/225000 [39:52:34<405:35:24,  6.94s/it]
  6%|▋         | 14560/225000 [39:52:39<382:04:21,  6.54s/it]
  6%|▋         | 14561/225000 [39:52:45<371:12:04,  6.35s/it]
  6%|▋         | 14562/225000 [39:52:51<358:26:23,  6.13s/it]
  6%|▋         | 14563/225000 [39:52:57<357:02:36,  6.11s/it]
  6%|▋         | 14564/225000 [39:53:06<406:58:11,  6.96s/it]
  6%|▋         | 14565/225000 [39:53:13<412:25:25,  7.06s/it]
  6%|▋         | 14566/225000 [39:53:19<385:59:38,  6.60s/it]
  6%|▋         | 14567/225000 [39:53:24<368:32:13,  6.30s/it]
  6%|▋         | 14568/225000 [39:53:30<351:08:42,  6.01s/it]
  6%|▋         | 14569/225000 [39:53:36<349:47:55,  5.98s/it]
  6%|▋         | 14570/225000 [39:53:41<344:04:32,  5.89s/it]
  6%|▋         | 14571/225000 [39:53:54<469:07:28,  8.03s/it]
  6%|▋         | 14572/225000 [39:54:00<430:35:49,  7.37s/it]
  6%|▋         | 14573/225000 [39:54:08<446:29:03,  7.64s/it]
  6%|▋         | 14574/225000 [39:54:12<379:21:33,  6.49s/it]
  6%|▋         | 14575/225000 [39:54:19<378:58:01,  6.48s/it]
  6%|▋         | 14576/225000 [39:54:25<371:23:51,  6.35s/it]
  6%|▋         | 14577/225000 [39:54:34<424:42:27,  7.27s/it]
  6%|▋         | 14578/225000 [39:54:39<374:59:28,  6.42s/it]
  6%|▋         | 14579/225000 [39:54:43<347:18:07,  5.94s/it]
  6%|▋         | 14580/225000 [39:54:48<320:49:48,  5.49s/it]
  6%|▋         | 14581/225000 [39:54:57<386:17:38,  6.61s/it]
  6%|▋         | 14582/225000 [39:55:04<395:51:03,  6.77s/it]
  6%|▋         | 14583/225000 [39:55:11<399:19:50,  6.83s/it]
  6%|▋         | 14584/225000 [39:55:16<360:54:04,  6.17s/it]
  6%|▋         | 14585/225000 [39:55:23<385:54:27,  6.60s/it]
  6%|▋         | 14586/225000 [39:55:29<370:13:22,  6.33s/it]
  6%|▋         | 14587/225000 [39:55:34<352:27:43,  6.03s/it]
  6%|▋         | 14588/225000 [39:55:39<320:37:42,  5.49s/it]
  6%|▋         | 14589/225000 [39:55:44<310:49:14,  5.32s/it]
  6%|▋         | 14590/225000 [39:55:53<380:08:29,  6.50s/it]
  6%|▋         | 14591/225000 [39:56:02<422:29:37,  7.23s/it]
  6%|▋         | 14592/225000 [39:56:06<371:56:49,  6.36s/it]
  6%|▋         | 14593/225000 [39:56:11<347:48:22,  5.95s/it]
  6%|▋         | 14594/225000 [39:56:16<331:53:46,  5.68s/it]
  6%|▋         | 14595/225000 [39:56:26<406:15:31,  6.95s/it]
  6%|▋         | 14596/225000 [39:56:30<356:09:35,  6.09s/it]
  6%|▋         | 14597/225000 [39:56:34<323:44:21,  5.54s/it]
  6%|▋         | 14598/225000 [39:56:41<348:12:19,  5.96s/it]
  6%|▋         | 14599/225000 [39:56:52<433:30:57,  7.42s/it]
  6%|▋         | 14600/225000 [39:56:57<391:36:44,  6.70s/it]
                                                             
{'loss': '0.7336', 'grad_norm': '0.6525', 'learning_rate': '4.994e-05', 'epoch': '0.06489', 'num_input_tokens_seen': 71093448, 'train_runtime': '1.438e+05', 'train_tokens_per_second': '494.3'}

  6%|▋         | 14600/225000 [39:56:57<391:36:44,  6.70s/it]
  6%|▋         | 14601/225000 [39:57:02<360:22:28,  6.17s/it]
  6%|▋         | 14602/225000 [39:57:07<335:41:19,  5.74s/it]
  6%|▋         | 14603/225000 [39:57:13<341:16:13,  5.84s/it]
  6%|▋         | 14604/225000 [39:57:24<429:45:14,  7.35s/it]
  6%|▋         | 14605/225000 [39:57:28<375:02:29,  6.42s/it]
  6%|▋         | 14606/225000 [39:57:32<332:04:31,  5.68s/it]
  6%|▋         | 14607/225000 [39:57:43<427:08:34,  7.31s/it]
  6%|▋         | 14608/225000 [39:57:47<368:30:49,  6.31s/it]
  6%|▋         | 14609/225000 [39:57:51<331:26:44,  5.67s/it]
  6%|▋         | 14610/225000 [39:58:02<425:52:23,  7.29s/it]
  6%|▋         | 14611/225000 [39:58:08<396:43:34,  6.79s/it]
  6%|▋         | 14612/225000 [39:58:14<386:12:29,  6.61s/it]
  6%|▋         | 14613/225000 [39:58:23<419:15:13,  7.17s/it]
  6%|▋         | 14614/225000 [39:58:28<388:55:01,  6.65s/it]
  6%|▋         | 14615/225000 [39:58:39<471:10:24,  8.06s/it]
  6%|▋         | 14616/225000 [39:58:43<395:11:35,  6.76s/it]
  6%|▋         | 14617/225000 [39:58:52<435:49:52,  7.46s/it]
  6%|▋         | 14618/225000 [39:58:57<391:23:16,  6.70s/it]
  6%|▋         | 14619/225000 [39:59:03<378:28:43,  6.48s/it]
  6%|▋         | 14620/225000 [39:59:09<375:07:53,  6.42s/it]
  6%|▋         | 14621/225000 [39:59:16<385:50:37,  6.60s/it]
  6%|▋         | 14622/225000 [39:59:21<350:33:13,  6.00s/it]
  6%|▋         | 14623/225000 [39:59:25<323:31:00,  5.54s/it]
  6%|▋         | 14624/225000 [39:59:30<310:13:49,  5.31s/it]
  6%|▋         | 14625/225000 [39:59:40<389:54:07,  6.67s/it]
  7%|▋         | 14626/225000 [39:59:46<374:40:07,  6.41s/it]
  7%|▋         | 14627/225000 [39:59:53<380:28:14,  6.51s/it]
  7%|▋         | 14628/225000 [39:59:57<349:51:35,  5.99s/it]
  7%|▋         | 14629/225000 [40:00:03<343:29:13,  5.88s/it]
  7%|▋         | 14630/225000 [40:00:08<332:41:12,  5.69s/it]
  7%|▋         | 14631/225000 [40:00:13<322:20:55,  5.52s/it]
  7%|▋         | 14632/225000 [40:00:21<364:16:16,  6.23s/it]
  7%|▋         | 14633/225000 [40:00:32<440:27:43,  7.54s/it]
  7%|▋         | 14634/225000 [40:00:36<386:53:41,  6.62s/it]
  7%|▋         | 14635/225000 [40:00:41<347:05:25,  5.94s/it]
  7%|▋         | 14636/225000 [40:00:50<397:00:06,  6.79s/it]
  7%|▋         | 14637/225000 [40:00:57<405:08:02,  6.93s/it]
  7%|▋         | 14638/225000 [40:01:01<356:29:20,  6.10s/it]
  7%|▋         | 14639/225000 [40:01:05<329:20:10,  5.64s/it]
  7%|▋         | 14640/225000 [40:01:13<360:46:15,  6.17s/it]
  7%|▋         | 14641/225000 [40:01:22<419:24:57,  7.18s/it]
  7%|▋         | 14642/225000 [40:01:30<419:58:05,  7.19s/it]
  7%|▋         | 14643/225000 [40:01:35<386:17:35,  6.61s/it]
  7%|▋         | 14644/225000 [40:01:40<357:01:37,  6.11s/it]
  7%|▋         | 14645/225000 [40:01:44<328:44:33,  5.63s/it]
  7%|▋         | 14646/225000 [40:01:49<310:47:26,  5.32s/it]
  7%|▋         | 14647/225000 [40:02:02<440:29:27,  7.54s/it]
  7%|▋         | 14648/225000 [40:02:06<378:50:16,  6.48s/it]
  7%|▋         | 14649/225000 [40:02:14<404:32:24,  6.92s/it]
  7%|▋         | 14650/225000 [40:02:18<367:41:07,  6.29s/it]
                                                             
{'loss': '0.722', 'grad_norm': '0.7', 'learning_rate': '4.994e-05', 'epoch': '0.06511', 'num_input_tokens_seen': 71249024, 'train_runtime': '1.441e+05', 'train_tokens_per_second': '494.3'}

  7%|▋         | 14650/225000 [40:02:18<367:41:07,  6.29s/it]
  7%|▋         | 14651/225000 [40:02:24<351:25:32,  6.01s/it]
  7%|▋         | 14652/225000 [40:02:29<331:49:50,  5.68s/it]
  7%|▋         | 14653/225000 [40:02:38<401:38:20,  6.87s/it]
  7%|▋         | 14654/225000 [40:02:47<438:16:46,  7.50s/it]
  7%|▋         | 14655/225000 [40:02:51<371:46:45,  6.36s/it]
  7%|▋         | 14656/225000 [40:03:01<433:55:43,  7.43s/it]
  7%|▋         | 14657/225000 [40:03:06<389:33:52,  6.67s/it]
  7%|▋         | 14658/225000 [40:03:11<369:23:07,  6.32s/it]
  7%|▋         | 14659/225000 [40:03:16<346:29:27,  5.93s/it]
  7%|▋         | 14660/225000 [40:03:22<338:51:45,  5.80s/it]
  7%|▋         | 14661/225000 [40:03:27<323:51:06,  5.54s/it]
  7%|▋         | 14662/225000 [40:03:35<377:47:25,  6.47s/it]
  7%|▋         | 14663/225000 [40:03:45<434:37:19,  7.44s/it]
  7%|▋         | 14664/225000 [40:03:52<418:45:40,  7.17s/it]
  7%|▋         | 14665/225000 [40:03:56<370:31:13,  6.34s/it]
  7%|▋         | 14666/225000 [40:04:06<436:41:13,  7.47s/it]
  7%|▋         | 14667/225000 [40:04:11<391:27:28,  6.70s/it]
  7%|▋         | 14668/225000 [40:04:16<352:46:07,  6.04s/it]
  7%|▋         | 14669/225000 [40:04:20<330:59:34,  5.67s/it]
  7%|▋         | 14670/225000 [40:04:26<331:08:18,  5.67s/it]
  7%|▋         | 14671/225000 [40:04:32<331:18:42,  5.67s/it]
  7%|▋         | 14672/225000 [40:04:38<335:09:15,  5.74s/it]
  7%|▋         | 14673/225000 [40:04:44<342:34:55,  5.86s/it]
  7%|▋         | 14674/225000 [40:04:49<338:23:11,  5.79s/it]
  7%|▋         | 14675/225000 [40:05:01<446:44:15,  7.65s/it]
  7%|▋         | 14676/225000 [40:05:08<423:59:21,  7.26s/it]
  7%|▋         | 14677/225000 [40:05:16<441:11:46,  7.55s/it]
  7%|▋         | 14678/225000 [40:05:20<379:04:01,  6.49s/it]
  7%|▋         | 14679/225000 [40:05:28<413:29:26,  7.08s/it]
  7%|▋         | 14680/225000 [40:05:34<384:08:53,  6.58s/it]
  7%|▋         | 14681/225000 [40:05:43<430:57:29,  7.38s/it]
  7%|▋         | 14682/225000 [40:05:48<382:25:26,  6.55s/it]
  7%|▋         | 14683/225000 [40:05:53<355:06:14,  6.08s/it]
  7%|▋         | 14684/225000 [40:06:03<427:54:58,  7.32s/it]
  7%|▋         | 14685/225000 [40:06:11<445:06:34,  7.62s/it]
  7%|▋         | 14686/225000 [40:06:16<393:42:05,  6.74s/it]
  7%|▋         | 14687/225000 [40:06:24<422:06:51,  7.23s/it]
  7%|▋         | 14688/225000 [40:06:29<371:47:33,  6.36s/it]
  7%|▋         | 14689/225000 [40:06:33<338:53:42,  5.80s/it]
  7%|▋         | 14690/225000 [40:06:45<439:06:15,  7.52s/it]
  7%|▋         | 14691/225000 [40:06:50<398:51:32,  6.83s/it]
  7%|▋         | 14692/225000 [40:06:55<364:21:29,  6.24s/it]
  7%|▋         | 14693/225000 [40:07:00<351:14:07,  6.01s/it]
  7%|▋         | 14694/225000 [40:07:05<333:04:43,  5.70s/it]
  7%|▋         | 14695/225000 [40:07:11<330:49:47,  5.66s/it]
  7%|▋         | 14696/225000 [40:07:16<321:08:20,  5.50s/it]
  7%|▋         | 14697/225000 [40:07:22<327:07:16,  5.60s/it]
  7%|▋         | 14698/225000 [40:07:31<383:14:41,  6.56s/it]
  7%|▋         | 14699/225000 [40:07:40<427:42:10,  7.32s/it]
  7%|▋         | 14700/225000 [40:07:45<390:07:26,  6.68s/it]
                                                             
{'loss': '0.7312', 'grad_norm': '0.6798', 'learning_rate': '4.994e-05', 'epoch': '0.06533', 'num_input_tokens_seen': 71405608, 'train_runtime': '1.445e+05', 'train_tokens_per_second': '494.3'}

  7%|▋         | 14700/225000 [40:07:45<390:07:26,  6.68s/it]
  7%|▋         | 14701/225000 [40:07:50<370:54:30,  6.35s/it]
  7%|▋         | 14702/225000 [40:07:57<369:05:34,  6.32s/it]
  7%|▋         | 14703/225000 [40:08:09<474:48:52,  8.13s/it]
  7%|▋         | 14704/225000 [40:08:13<403:57:37,  6.92s/it]
  7%|▋         | 14705/225000 [40:08:21<420:28:56,  7.20s/it]
  7%|▋         | 14706/225000 [40:08:28<411:51:05,  7.05s/it]
  7%|▋         | 14707/225000 [40:08:32<368:41:04,  6.31s/it]
  7%|▋         | 14708/225000 [40:08:37<337:55:30,  5.78s/it]
  7%|▋         | 14709/225000 [40:08:45<379:38:32,  6.50s/it]
  7%|▋         | 14710/225000 [40:08:53<408:05:00,  6.99s/it]
  7%|▋         | 14711/225000 [40:09:02<444:36:49,  7.61s/it]
  7%|▋         | 14712/225000 [40:09:12<476:48:26,  8.16s/it]
  7%|▋         | 14713/225000 [40:09:17<429:55:59,  7.36s/it]
  7%|▋         | 14714/225000 [40:09:24<421:56:29,  7.22s/it]
  7%|▋         | 14715/225000 [40:09:30<392:24:23,  6.72s/it]
  7%|▋         | 14716/225000 [40:09:38<422:20:56,  7.23s/it]
  7%|▋         | 14717/225000 [40:09:44<401:32:34,  6.87s/it]
  7%|▋         | 14718/225000 [40:09:50<378:10:57,  6.47s/it]
  7%|▋         | 14719/225000 [40:09:55<362:01:19,  6.20s/it]
  7%|▋         | 14720/225000 [40:10:00<346:56:02,  5.94s/it]
  7%|▋         | 14721/225000 [40:10:12<441:08:00,  7.55s/it]
  7%|▋         | 14722/225000 [40:10:16<388:13:41,  6.65s/it]
  7%|▋         | 14723/225000 [40:10:25<424:01:39,  7.26s/it]
  7%|▋         | 14724/225000 [40:10:29<365:03:12,  6.25s/it]
  7%|▋         | 14725/225000 [40:10:36<382:40:10,  6.55s/it]
  7%|▋         | 14726/225000 [40:10:41<346:32:56,  5.93s/it]
  7%|▋         | 14727/225000 [40:10:48<380:43:33,  6.52s/it]
  7%|▋         | 14728/225000 [40:11:01<482:44:58,  8.27s/it]
  7%|▋         | 14729/225000 [40:11:05<414:22:06,  7.09s/it]
  7%|▋         | 14730/225000 [40:11:12<405:53:00,  6.95s/it]
  7%|▋         | 14731/225000 [40:11:16<361:53:42,  6.20s/it]
  7%|▋         | 14732/225000 [40:11:21<331:19:50,  5.67s/it]
  7%|▋         | 14733/225000 [40:11:34<463:51:34,  7.94s/it]
  7%|▋         | 14734/225000 [40:11:38<395:31:13,  6.77s/it]
  7%|▋         | 14735/225000 [40:11:43<359:06:37,  6.15s/it]
  7%|▋         | 14736/225000 [40:11:49<355:06:49,  6.08s/it]
  7%|▋         | 14737/225000 [40:11:54<340:52:56,  5.84s/it]
  7%|▋         | 14738/225000 [40:12:01<372:48:37,  6.38s/it]
  7%|▋         | 14739/225000 [40:12:07<360:12:03,  6.17s/it]
  7%|▋         | 14740/225000 [40:12:13<349:57:31,  5.99s/it]
  7%|▋         | 14741/225000 [40:12:18<344:15:47,  5.89s/it]
  7%|▋         | 14742/225000 [40:12:27<400:00:46,  6.85s/it]
  7%|▋         | 14743/225000 [40:12:36<421:55:08,  7.22s/it]
  7%|▋         | 14744/225000 [40:12:40<375:43:15,  6.43s/it]
  7%|▋         | 14745/225000 [40:12:45<342:41:58,  5.87s/it]
  7%|▋         | 14746/225000 [40:12:50<336:37:27,  5.76s/it]
  7%|▋         | 14747/225000 [40:12:56<330:06:26,  5.65s/it]
  7%|▋         | 14748/225000 [40:13:02<338:14:20,  5.79s/it]
  7%|▋         | 14749/225000 [40:13:09<360:18:59,  6.17s/it]
  7%|▋         | 14750/225000 [40:13:21<471:58:34,  8.08s/it]
                                                             
{'loss': '0.7535', 'grad_norm': '0.7965', 'learning_rate': '4.994e-05', 'epoch': '0.06556', 'num_input_tokens_seen': 71563680, 'train_runtime': '1.448e+05', 'train_tokens_per_second': '494.2'}

  7%|▋         | 14750/225000 [40:13:21<471:58:34,  8.08s/it]
  7%|▋         | 14751/225000 [40:13:25<402:15:56,  6.89s/it]
  7%|▋         | 14752/225000 [40:13:31<375:45:30,  6.43s/it]
  7%|▋         | 14753/225000 [40:13:41<435:22:49,  7.45s/it]
  7%|▋         | 14754/225000 [40:13:50<463:32:42,  7.94s/it]
  7%|▋         | 14755/225000 [40:13:54<402:13:20,  6.89s/it]
  7%|▋         | 14756/225000 [40:13:59<362:14:49,  6.20s/it]
  7%|▋         | 14757/225000 [40:14:10<457:06:34,  7.83s/it]
  7%|▋         | 14758/225000 [40:14:14<382:03:41,  6.54s/it]
  7%|▋         | 14759/225000 [40:14:26<481:36:17,  8.25s/it]
  7%|▋         | 14760/225000 [40:14:30<410:48:09,  7.03s/it]
  7%|▋         | 14761/225000 [40:14:39<432:15:32,  7.40s/it]
  7%|▋         | 14762/225000 [40:14:44<394:37:23,  6.76s/it]
  7%|▋         | 14763/225000 [40:14:49<372:50:11,  6.38s/it]
  7%|▋         | 14764/225000 [40:14:58<404:20:46,  6.92s/it]
  7%|▋         | 14765/225000 [40:15:03<381:21:57,  6.53s/it]
  7%|▋         | 14766/225000 [40:15:11<410:23:08,  7.03s/it]
  7%|▋         | 14767/225000 [40:15:15<357:13:13,  6.12s/it]
  7%|▋         | 14768/225000 [40:15:23<385:08:37,  6.60s/it]
  7%|▋         | 14769/225000 [40:15:27<345:24:51,  5.91s/it]
  7%|▋         | 14770/225000 [40:15:32<326:25:42,  5.59s/it]
  7%|▋         | 14771/225000 [40:15:37<311:43:38,  5.34s/it]
  7%|▋         | 14772/225000 [40:15:49<424:50:12,  7.28s/it]
  7%|▋         | 14773/225000 [40:15:54<388:54:56,  6.66s/it]
  7%|▋         | 14774/225000 [40:15:59<359:00:02,  6.15s/it]
  7%|▋         | 14775/225000 [40:16:04<343:21:11,  5.88s/it]
  7%|▋         | 14776/225000 [40:16:09<331:14:18,  5.67s/it]
  7%|▋         | 14777/225000 [40:16:15<323:58:06,  5.55s/it]
  7%|▋         | 14778/225000 [40:16:20<318:09:04,  5.45s/it]
  7%|▋         | 14779/225000 [40:16:27<351:25:01,  6.02s/it]
  7%|▋         | 14780/225000 [40:16:39<453:22:38,  7.76s/it]
  7%|▋         | 14781/225000 [40:16:43<378:55:57,  6.49s/it]
  7%|▋         | 14782/225000 [40:16:47<337:08:43,  5.77s/it]
  7%|▋         | 14783/225000 [40:16:51<315:09:10,  5.40s/it]
  7%|▋         | 14784/225000 [40:17:03<426:56:04,  7.31s/it]
  7%|▋         | 14785/225000 [40:17:07<371:57:17,  6.37s/it]
  7%|▋         | 14786/225000 [40:17:11<335:37:59,  5.75s/it]
  7%|▋         | 14787/225000 [40:17:20<378:12:02,  6.48s/it]
  7%|▋         | 14788/225000 [40:17:28<411:05:02,  7.04s/it]
  7%|▋         | 14789/225000 [40:17:34<401:21:11,  6.87s/it]
  7%|▋         | 14790/225000 [40:17:40<373:14:29,  6.39s/it]
  7%|▋         | 14791/225000 [40:17:51<452:58:25,  7.76s/it]
  7%|▋         | 14792/225000 [40:17:58<453:10:56,  7.76s/it]
  7%|▋         | 14793/225000 [40:18:07<473:31:20,  8.11s/it]
  7%|▋         | 14794/225000 [40:18:12<416:04:38,  7.13s/it]
  7%|▋         | 14795/225000 [40:18:17<369:10:27,  6.32s/it]
  7%|▋         | 14796/225000 [40:18:22<344:35:58,  5.90s/it]
  7%|▋         | 14797/225000 [40:18:32<418:27:35,  7.17s/it]
  7%|▋         | 14798/225000 [40:18:36<369:49:00,  6.33s/it]
  7%|▋         | 14799/225000 [40:18:42<365:35:19,  6.26s/it]
  7%|▋         | 14800/225000 [40:18:48<351:33:32,  6.02s/it]
                                                             
{'loss': '0.7326', 'grad_norm': '0.6495', 'learning_rate': '4.994e-05', 'epoch': '0.06578', 'num_input_tokens_seen': 71720016, 'train_runtime': '1.451e+05', 'train_tokens_per_second': '494.2'}

  7%|▋         | 14800/225000 [40:18:48<351:33:32,  6.02s/it]
  7%|▋         | 14801/225000 [40:18:53<332:30:03,  5.69s/it]
  7%|▋         | 14802/225000 [40:18:57<317:12:06,  5.43s/it]
  7%|▋         | 14803/225000 [40:19:08<407:59:54,  6.99s/it]
  7%|▋         | 14804/225000 [40:19:17<440:47:46,  7.55s/it]
  7%|▋         | 14805/225000 [40:19:28<510:14:43,  8.74s/it]
  7%|▋         | 14806/225000 [40:19:32<421:56:25,  7.23s/it]
  7%|▋         | 14807/225000 [40:19:41<451:51:59,  7.74s/it]
  7%|▋         | 14808/225000 [40:19:46<410:16:24,  7.03s/it]
  7%|▋         | 14809/225000 [40:19:51<366:53:09,  6.28s/it]
  7%|▋         | 14810/225000 [40:19:56<338:58:49,  5.81s/it]
  7%|▋         | 14811/225000 [40:20:09<467:16:23,  8.00s/it]
  7%|▋         | 14812/225000 [40:20:13<399:39:07,  6.85s/it]
  7%|▋         | 14813/225000 [40:20:22<440:55:38,  7.55s/it]
  7%|▋         | 14814/225000 [40:20:28<409:11:19,  7.01s/it]
  7%|▋         | 14815/225000 [40:20:31<344:23:11,  5.90s/it]
  7%|▋         | 14816/225000 [40:20:39<378:16:17,  6.48s/it]
  7%|▋         | 14817/225000 [40:20:44<357:23:14,  6.12s/it]
  7%|▋         | 14818/225000 [40:20:50<345:37:49,  5.92s/it]
  7%|▋         | 14819/225000 [40:20:58<389:35:30,  6.67s/it]
  7%|▋         | 14820/225000 [40:21:04<374:03:19,  6.41s/it]
  7%|▋         | 14821/225000 [40:21:10<367:32:04,  6.30s/it]
  7%|▋         | 14822/225000 [40:21:15<342:21:05,  5.86s/it]
  7%|▋         | 14823/225000 [40:21:20<328:17:34,  5.62s/it]
  7%|▋         | 14824/225000 [40:21:25<316:29:47,  5.42s/it]
  7%|▋         | 14825/225000 [40:21:37<443:47:00,  7.60s/it]
  7%|▋         | 14826/225000 [40:21:41<373:09:13,  6.39s/it]
  7%|▋         | 14827/225000 [40:21:48<378:02:52,  6.48s/it]
  7%|▋         | 14828/225000 [40:21:55<384:32:37,  6.59s/it]
  7%|▋         | 14829/225000 [40:22:02<401:52:59,  6.88s/it]
  7%|▋         | 14830/225000 [40:22:08<387:46:53,  6.64s/it]
  7%|▋         | 14831/225000 [40:22:13<357:52:40,  6.13s/it]
  7%|▋         | 14832/225000 [40:22:18<333:56:14,  5.72s/it]
  7%|▋         | 14833/225000 [40:22:23<327:07:28,  5.60s/it]
  7%|▋         | 14834/225000 [40:22:29<327:52:56,  5.62s/it]
  7%|▋         | 14835/225000 [40:22:41<436:05:30,  7.47s/it]
  7%|▋         | 14836/225000 [40:22:45<388:19:32,  6.65s/it]
  7%|▋         | 14837/225000 [40:22:54<423:36:33,  7.26s/it]
  7%|▋         | 14838/225000 [40:23:02<434:34:59,  7.44s/it]
  7%|▋         | 14839/225000 [40:23:13<491:15:12,  8.42s/it]
  7%|▋         | 14840/225000 [40:23:17<419:08:26,  7.18s/it]
  7%|▋         | 14841/225000 [40:23:30<524:33:06,  8.99s/it]
  7%|▋         | 14842/225000 [40:23:35<445:36:40,  7.63s/it]
  7%|▋         | 14843/225000 [40:23:39<384:27:37,  6.59s/it]
  7%|▋         | 14844/225000 [40:23:49<442:53:26,  7.59s/it]
  7%|▋         | 14845/225000 [40:23:54<396:04:16,  6.78s/it]
  7%|▋         | 14846/225000 [40:23:59<363:40:42,  6.23s/it]
  7%|▋         | 14847/225000 [40:24:04<356:45:22,  6.11s/it]
  7%|▋         | 14848/225000 [40:24:15<432:24:56,  7.41s/it]
  7%|▋         | 14849/225000 [40:24:19<384:27:27,  6.59s/it]
  7%|▋         | 14850/225000 [40:24:29<440:58:24,  7.55s/it]
                                                             
{'loss': '0.7334', 'grad_norm': '0.7492', 'learning_rate': '4.994e-05', 'epoch': '0.066', 'num_input_tokens_seen': 71876920, 'train_runtime': '1.455e+05', 'train_tokens_per_second': '494.1'}

  7%|▋         | 14850/225000 [40:24:29<440:58:24,  7.55s/it]
  7%|▋         | 14851/225000 [40:24:33<375:16:53,  6.43s/it]
  7%|▋         | 14852/225000 [40:24:44<444:35:47,  7.62s/it]
  7%|▋         | 14853/225000 [40:24:47<376:59:17,  6.46s/it]
  7%|▋         | 14854/225000 [40:24:55<400:02:15,  6.85s/it]
  7%|▋         | 14855/225000 [40:24:59<350:10:13,  6.00s/it]
  7%|▋         | 14856/225000 [40:25:07<388:56:53,  6.66s/it]
  7%|▋         | 14857/225000 [40:25:15<405:45:13,  6.95s/it]
  7%|▋         | 14858/225000 [40:25:20<371:14:19,  6.36s/it]
  7%|▋         | 14859/225000 [40:25:25<355:26:41,  6.09s/it]
  7%|▋         | 14860/225000 [40:25:30<338:10:38,  5.79s/it]
  7%|▋         | 14861/225000 [40:25:39<384:47:57,  6.59s/it]
  7%|▋         | 14862/225000 [40:25:47<408:08:27,  6.99s/it]
  7%|▋         | 14863/225000 [40:25:52<372:21:01,  6.38s/it]
  7%|▋         | 14864/225000 [40:25:57<352:30:23,  6.04s/it]
  7%|▋         | 14865/225000 [40:26:02<336:46:57,  5.77s/it]
  7%|▋         | 14866/225000 [40:26:07<329:32:01,  5.65s/it]
  7%|▋         | 14867/225000 [40:26:15<368:53:29,  6.32s/it]
  7%|▋         | 14868/225000 [40:26:26<445:03:50,  7.62s/it]
  7%|▋         | 14869/225000 [40:26:30<377:49:40,  6.47s/it]
  7%|▋         | 14870/225000 [40:26:40<433:59:05,  7.44s/it]
  7%|▋         | 14871/225000 [40:26:46<420:06:26,  7.20s/it]
  7%|▋         | 14872/225000 [40:26:50<363:29:58,  6.23s/it]
  7%|▋         | 14873/225000 [40:27:00<430:06:22,  7.37s/it]
  7%|▋         | 14874/225000 [40:27:04<363:12:12,  6.22s/it]
  7%|▋         | 14875/225000 [40:27:12<407:44:06,  6.99s/it]
  7%|▋         | 14876/225000 [40:27:17<356:26:30,  6.11s/it]
  7%|▋         | 14877/225000 [40:27:23<366:55:40,  6.29s/it]
  7%|▋         | 14878/225000 [40:27:32<402:18:56,  6.89s/it]
  7%|▋         | 14879/225000 [40:27:38<397:03:42,  6.80s/it]
  7%|▋         | 14880/225000 [40:27:44<374:40:31,  6.42s/it]
  7%|▋         | 14881/225000 [40:27:49<355:01:00,  6.08s/it]
  7%|▋         | 14882/225000 [40:27:54<342:39:14,  5.87s/it]
  7%|▋         | 14883/225000 [40:28:05<434:36:31,  7.45s/it]
  7%|▋         | 14884/225000 [40:28:10<388:40:07,  6.66s/it]
  7%|▋         | 14885/225000 [40:28:15<350:23:36,  6.00s/it]
  7%|▋         | 14886/225000 [40:28:21<350:49:26,  6.01s/it]
  7%|▋         | 14887/225000 [40:28:31<425:21:02,  7.29s/it]
  7%|▋         | 14888/225000 [40:28:36<382:11:53,  6.55s/it]
  7%|▋         | 14889/225000 [40:28:45<430:12:40,  7.37s/it]
  7%|▋         | 14890/225000 [40:28:51<394:57:30,  6.77s/it]
  7%|▋         | 14891/225000 [40:28:58<415:20:11,  7.12s/it]
  7%|▋         | 14892/225000 [40:29:03<368:14:12,  6.31s/it]
  7%|▋         | 14893/225000 [40:29:08<353:05:06,  6.05s/it]
  7%|▋         | 14894/225000 [40:29:18<421:41:12,  7.23s/it]
  7%|▋         | 14895/225000 [40:29:24<387:42:22,  6.64s/it]
  7%|▋         | 14896/225000 [40:29:28<353:48:54,  6.06s/it]
  7%|▋         | 14897/225000 [40:29:37<406:36:52,  6.97s/it]
  7%|▋         | 14898/225000 [40:29:46<427:41:58,  7.33s/it]
  7%|▋         | 14899/225000 [40:29:50<376:43:31,  6.46s/it]
  7%|▋         | 14900/225000 [40:29:59<422:54:02,  7.25s/it]
                                                             
{'loss': '0.7249', 'grad_norm': '0.6199', 'learning_rate': '4.994e-05', 'epoch': '0.06622', 'num_input_tokens_seen': 72030280, 'train_runtime': '1.458e+05', 'train_tokens_per_second': '494'}

  7%|▋         | 14900/225000 [40:29:59<422:54:02,  7.25s/it]
  7%|▋         | 14901/225000 [40:30:09<465:51:02,  7.98s/it]
  7%|▋         | 14902/225000 [40:30:13<396:11:07,  6.79s/it]
  7%|▋         | 14903/225000 [40:30:22<442:21:00,  7.58s/it]
  7%|▋         | 14904/225000 [40:30:29<424:44:14,  7.28s/it]
  7%|▋         | 14905/225000 [40:30:36<421:47:03,  7.23s/it]
  7%|▋         | 14906/225000 [40:30:42<394:31:40,  6.76s/it]
  7%|▋         | 14907/225000 [40:30:46<354:45:55,  6.08s/it]
  7%|▋         | 14908/225000 [40:30:53<369:12:35,  6.33s/it]
  7%|▋         | 14909/225000 [40:31:05<474:07:12,  8.12s/it]
  7%|▋         | 14910/225000 [40:31:09<403:29:24,  6.91s/it]
  7%|▋         | 14911/225000 [40:31:17<420:58:07,  7.21s/it]
  7%|▋         | 14912/225000 [40:31:24<408:31:52,  7.00s/it]
  7%|▋         | 14913/225000 [40:31:31<411:33:50,  7.05s/it]
  7%|▋         | 14914/225000 [40:31:35<363:30:58,  6.23s/it]
  7%|▋         | 14915/225000 [40:31:44<401:13:03,  6.88s/it]
  7%|▋         | 14916/225000 [40:31:49<382:02:32,  6.55s/it]
  7%|▋         | 14917/225000 [40:31:57<399:23:13,  6.84s/it]
  7%|▋         | 14918/225000 [40:32:05<415:44:34,  7.12s/it]
  7%|▋         | 14919/225000 [40:32:12<410:52:10,  7.04s/it]
  7%|▋         | 14920/225000 [40:32:17<390:34:21,  6.69s/it]
  7%|▋         | 14921/225000 [40:32:23<368:46:47,  6.32s/it]
  7%|▋         | 14922/225000 [40:32:27<328:23:52,  5.63s/it]
  7%|▋         | 14923/225000 [40:32:32<311:48:34,  5.34s/it]
  7%|▋         | 14924/225000 [40:32:37<320:27:07,  5.49s/it]
  7%|▋         | 14925/225000 [40:32:48<411:48:28,  7.06s/it]
  7%|▋         | 14926/225000 [40:32:55<408:51:44,  7.01s/it]
  7%|▋         | 14927/225000 [40:33:01<383:16:14,  6.57s/it]
  7%|▋         | 14928/225000 [40:33:09<421:25:09,  7.22s/it]
  7%|▋         | 14929/225000 [40:33:19<462:02:50,  7.92s/it]
  7%|▋         | 14930/225000 [40:33:26<444:42:30,  7.62s/it]
  7%|▋         | 14931/225000 [40:33:34<447:06:22,  7.66s/it]
  7%|▋         | 14932/225000 [40:33:43<478:35:30,  8.20s/it]
  7%|▋         | 14933/225000 [40:33:49<440:50:53,  7.55s/it]
  7%|▋         | 14934/225000 [40:33:57<454:30:49,  7.79s/it]
  7%|▋         | 14935/225000 [40:34:03<411:43:35,  7.06s/it]
  7%|▋         | 14936/225000 [40:34:08<380:18:46,  6.52s/it]
  7%|▋         | 14937/225000 [40:34:13<352:30:51,  6.04s/it]
  7%|▋         | 14938/225000 [40:34:24<448:55:43,  7.69s/it]
  7%|▋         | 14939/225000 [40:34:29<397:04:18,  6.80s/it]
  7%|▋         | 14940/225000 [40:34:35<372:23:33,  6.38s/it]
  7%|▋         | 14941/225000 [40:34:46<469:16:07,  8.04s/it]
  7%|▋         | 14942/225000 [40:34:54<462:09:51,  7.92s/it]
  7%|▋         | 14943/225000 [40:34:58<395:39:36,  6.78s/it]
  7%|▋         | 14944/225000 [40:35:06<408:36:54,  7.00s/it]
  7%|▋         | 14945/225000 [40:35:10<367:26:17,  6.30s/it]
  7%|▋         | 14946/225000 [40:35:20<421:53:59,  7.23s/it]
  7%|▋         | 14947/225000 [40:35:25<387:50:50,  6.65s/it]
  7%|▋         | 14948/225000 [40:35:30<360:04:05,  6.17s/it]
  7%|▋         | 14949/225000 [40:35:35<341:41:41,  5.86s/it]
  7%|▋         | 14950/225000 [40:35:40<323:58:47,  5.55s/it]
                                                             
{'loss': '0.7438', 'grad_norm': '0.5873', 'learning_rate': '4.993e-05', 'epoch': '0.06644', 'num_input_tokens_seen': 72187608, 'train_runtime': '1.461e+05', 'train_tokens_per_second': '494'}

  7%|▋         | 14950/225000 [40:35:40<323:58:47,  5.55s/it]
  7%|▋         | 14951/225000 [40:35:48<363:11:42,  6.22s/it]
  7%|▋         | 14952/225000 [40:35:59<443:32:43,  7.60s/it]
  7%|▋         | 14953/225000 [40:36:05<419:14:43,  7.19s/it]
  7%|▋         | 14954/225000 [40:36:11<400:02:31,  6.86s/it]
  7%|▋         | 14955/225000 [40:36:20<433:49:04,  7.44s/it]
  7%|▋         | 14956/225000 [40:36:25<393:40:31,  6.75s/it]
  7%|▋         | 14957/225000 [40:36:30<362:20:15,  6.21s/it]
  7%|▋         | 14958/225000 [40:36:35<343:29:50,  5.89s/it]
  7%|▋         | 14959/225000 [40:36:40<324:59:48,  5.57s/it]
  7%|▋         | 14960/225000 [40:36:45<311:11:57,  5.33s/it]
  7%|▋         | 14961/225000 [40:36:58<456:13:09,  7.82s/it]
  7%|▋         | 14962/225000 [40:37:04<426:54:26,  7.32s/it]
  7%|▋         | 14963/225000 [40:37:11<407:41:19,  6.99s/it]
  7%|▋         | 14964/225000 [40:37:15<368:21:54,  6.31s/it]
  7%|▋         | 14965/225000 [40:37:20<341:23:01,  5.85s/it]
  7%|▋         | 14966/225000 [40:37:26<336:45:59,  5.77s/it]
  7%|▋         | 14967/225000 [40:37:31<322:24:17,  5.53s/it]
  7%|▋         | 14968/225000 [40:37:44<451:03:31,  7.73s/it]
  7%|▋         | 14969/225000 [40:37:50<422:30:22,  7.24s/it]
  7%|▋         | 14970/225000 [40:37:56<404:48:03,  6.94s/it]
  7%|▋         | 14971/225000 [40:38:06<453:45:12,  7.78s/it]
  7%|▋         | 14972/225000 [40:38:10<392:29:38,  6.73s/it]
  7%|▋         | 14973/225000 [40:38:15<363:37:15,  6.23s/it]
  7%|▋         | 14974/225000 [40:38:23<396:22:07,  6.79s/it]
  7%|▋         | 14975/225000 [40:38:33<450:17:41,  7.72s/it]
  7%|▋         | 14976/225000 [40:38:37<387:27:50,  6.64s/it]
  7%|▋         | 14977/225000 [40:38:46<422:58:38,  7.25s/it]
  7%|▋         | 14978/225000 [40:38:50<363:13:23,  6.23s/it]
  7%|▋         | 14979/225000 [40:38:59<424:20:19,  7.27s/it]
  7%|▋         | 14980/225000 [40:39:03<363:42:08,  6.23s/it]
  7%|▋         | 14981/225000 [40:39:11<389:16:36,  6.67s/it]
  7%|▋         | 14982/225000 [40:39:19<412:20:25,  7.07s/it]
  7%|▋         | 14983/225000 [40:39:23<355:02:01,  6.09s/it]
  7%|▋         | 14984/225000 [40:39:29<354:11:41,  6.07s/it]
  7%|▋         | 14985/225000 [40:39:36<379:22:33,  6.50s/it]
  7%|▋         | 14986/225000 [40:39:43<382:31:41,  6.56s/it]
  7%|▋         | 14987/225000 [40:39:54<465:17:20,  7.98s/it]
  7%|▋         | 14988/225000 [40:40:00<432:30:44,  7.41s/it]
  7%|▋         | 14989/225000 [40:40:06<401:24:41,  6.88s/it]
  7%|▋         | 14990/225000 [40:40:11<369:18:29,  6.33s/it]
  7%|▋         | 14991/225000 [40:40:20<420:20:53,  7.21s/it]
  7%|▋         | 14992/225000 [40:40:26<394:13:57,  6.76s/it]
  7%|▋         | 14993/225000 [40:40:32<390:38:12,  6.70s/it]
  7%|▋         | 14994/225000 [40:40:38<367:57:51,  6.31s/it]
  7%|▋         | 14995/225000 [40:40:43<355:31:43,  6.09s/it]
  7%|▋         | 14996/225000 [40:40:49<351:19:44,  6.02s/it]
  7%|▋         | 14997/225000 [40:40:55<348:19:51,  5.97s/it]
  7%|▋         | 14998/225000 [40:41:01<340:37:32,  5.84s/it]
  7%|▋         | 14999/225000 [40:41:06<336:28:20,  5.77s/it]
  7%|▋         | 15000/225000 [40:41:20<475:45:49,  8.16s/it]
                                                             
{'loss': '0.7512', 'grad_norm': '0.6915', 'learning_rate': '4.993e-05', 'epoch': '0.06667', 'num_input_tokens_seen': 72341872, 'train_runtime': '1.465e+05', 'train_tokens_per_second': '493.9'}

  7%|▋         | 15000/225000 [40:41:20<475:45:49,  8.16s/it]
  7%|▋         | 15001/225000 [40:41:27<453:27:36,  7.77s/it]
  7%|▋         | 15002/225000 [40:41:34<434:15:57,  7.44s/it]
  7%|▋         | 15003/225000 [40:41:38<387:50:26,  6.65s/it]
  7%|▋         | 15004/225000 [40:41:43<349:52:02,  6.00s/it]
  7%|▋         | 15005/225000 [40:41:53<417:29:41,  7.16s/it]
  7%|▋         | 15006/225000 [40:42:00<411:25:46,  7.05s/it]
  7%|▋         | 15007/225000 [40:42:03<354:29:22,  6.08s/it]
  7%|▋         | 15008/225000 [40:42:14<431:35:04,  7.40s/it]
  7%|▋         | 15009/225000 [40:42:18<381:41:03,  6.54s/it]
  7%|▋         | 15010/225000 [40:42:23<340:38:28,  5.84s/it]
  7%|▋         | 15011/225000 [40:42:27<314:15:20,  5.39s/it]
  7%|▋         | 15012/225000 [40:42:39<431:11:34,  7.39s/it]
  7%|▋         | 15013/225000 [40:42:43<379:32:39,  6.51s/it]
  7%|▋         | 15014/225000 [40:42:50<389:50:55,  6.68s/it]
  7%|▋         | 15015/225000 [40:42:55<343:28:35,  5.89s/it]
  7%|▋         | 15016/225000 [40:43:05<425:50:57,  7.30s/it]
  7%|▋         | 15017/225000 [40:43:09<365:31:57,  6.27s/it]
  7%|▋         | 15018/225000 [40:43:14<342:27:02,  5.87s/it]
  7%|▋         | 15019/225000 [40:43:24<410:51:18,  7.04s/it]
  7%|▋         | 15020/225000 [40:43:31<417:53:52,  7.16s/it]
  7%|▋         | 15021/225000 [40:43:36<376:03:50,  6.45s/it]
  7%|▋         | 15022/225000 [40:43:47<450:48:28,  7.73s/it]
  7%|▋         | 15023/225000 [40:43:51<386:42:33,  6.63s/it]
  7%|▋         | 15024/225000 [40:43:55<348:07:27,  5.97s/it]
  7%|▋         | 15025/225000 [40:44:05<424:24:16,  7.28s/it]
  7%|▋         | 15026/225000 [40:44:12<412:39:09,  7.07s/it]
  7%|▋         | 15027/225000 [40:44:17<379:35:57,  6.51s/it]
  7%|▋         | 15028/225000 [40:44:22<354:17:56,  6.07s/it]
  7%|▋         | 15029/225000 [40:44:27<334:39:57,  5.74s/it]
  7%|▋         | 15030/225000 [40:44:36<390:26:53,  6.69s/it]
  7%|▋         | 15031/225000 [40:44:45<430:44:59,  7.39s/it]
  7%|▋         | 15032/225000 [40:44:53<439:45:13,  7.54s/it]
  7%|▋         | 15033/225000 [40:44:59<412:26:59,  7.07s/it]
  7%|▋         | 15034/225000 [40:45:05<392:13:20,  6.72s/it]
  7%|▋         | 15035/225000 [40:45:15<442:00:12,  7.58s/it]
  7%|▋         | 15036/225000 [40:45:18<377:19:13,  6.47s/it]
  7%|▋         | 15037/225000 [40:45:29<454:49:54,  7.80s/it]
  7%|▋         | 15038/225000 [40:45:34<394:51:24,  6.77s/it]
  7%|▋         | 15039/225000 [40:45:39<366:20:45,  6.28s/it]
  7%|▋         | 15040/225000 [40:45:50<443:18:39,  7.60s/it]
  7%|▋         | 15041/225000 [40:45:53<377:30:47,  6.47s/it]
  7%|▋         | 15042/225000 [40:46:03<425:18:06,  7.29s/it]
  7%|▋         | 15043/225000 [40:46:09<410:21:20,  7.04s/it]
  7%|▋         | 15044/225000 [40:46:15<394:23:41,  6.76s/it]
  7%|▋         | 15045/225000 [40:46:22<392:34:02,  6.73s/it]
  7%|▋         | 15046/225000 [40:46:26<352:16:56,  6.04s/it]
  7%|▋         | 15047/225000 [40:46:30<320:42:53,  5.50s/it]
  7%|▋         | 15048/225000 [40:46:42<421:12:57,  7.22s/it]
  7%|▋         | 15049/225000 [40:46:46<373:24:07,  6.40s/it]
  7%|▋         | 15050/225000 [40:46:52<356:22:18,  6.11s/it]
                                                             
{'loss': '0.7548', 'grad_norm': '0.706', 'learning_rate': '4.993e-05', 'epoch': '0.06689', 'num_input_tokens_seen': 72494928, 'train_runtime': '1.468e+05', 'train_tokens_per_second': '493.8'}

  7%|▋         | 15050/225000 [40:46:52<356:22:18,  6.11s/it]
  7%|▋         | 15051/225000 [40:46:57<347:36:27,  5.96s/it]
  7%|▋         | 15052/225000 [40:47:02<334:12:13,  5.73s/it]
  7%|▋         | 15053/225000 [40:47:11<381:02:32,  6.53s/it]
  7%|▋         | 15054/225000 [40:47:20<432:48:39,  7.42s/it]
  7%|▋         | 15055/225000 [40:47:25<391:00:18,  6.70s/it]
  7%|▋         | 15056/225000 [40:47:33<413:16:52,  7.09s/it]
  7%|▋         | 15057/225000 [40:47:40<402:36:19,  6.90s/it]
  7%|▋         | 15058/225000 [40:47:45<364:43:59,  6.25s/it]
  7%|▋         | 15059/225000 [40:47:49<339:43:07,  5.83s/it]
  7%|▋         | 15060/225000 [40:48:02<454:19:34,  7.79s/it]
  7%|▋         | 15061/225000 [40:48:11<480:48:52,  8.24s/it]
  7%|▋         | 15062/225000 [40:48:18<454:00:02,  7.79s/it]
  7%|▋         | 15063/225000 [40:48:22<390:49:32,  6.70s/it]
  7%|▋         | 15064/225000 [40:48:32<456:55:06,  7.84s/it]
  7%|▋         | 15065/225000 [40:48:38<408:39:07,  7.01s/it]
  7%|▋         | 15066/225000 [40:48:42<359:54:58,  6.17s/it]
  7%|▋         | 15067/225000 [40:48:46<335:02:55,  5.75s/it]
  7%|▋         | 15068/225000 [40:48:55<386:51:45,  6.63s/it]
  7%|▋         | 15069/225000 [40:49:04<433:13:26,  7.43s/it]
  7%|▋         | 15070/225000 [40:49:10<402:43:12,  6.91s/it]
  7%|▋         | 15071/225000 [40:49:17<398:12:50,  6.83s/it]
  7%|▋         | 15072/225000 [40:49:21<359:35:56,  6.17s/it]
  7%|▋         | 15073/225000 [40:49:26<339:39:03,  5.82s/it]
  7%|▋         | 15074/225000 [40:49:31<321:50:37,  5.52s/it]
  7%|▋         | 15075/225000 [40:49:42<415:05:42,  7.12s/it]
  7%|▋         | 15076/225000 [40:49:50<432:56:03,  7.42s/it]
  7%|▋         | 15077/225000 [40:49:56<400:36:45,  6.87s/it]
  7%|▋         | 15078/225000 [40:50:04<421:49:50,  7.23s/it]
  7%|▋         | 15079/225000 [40:50:09<385:57:02,  6.62s/it]
  7%|▋         | 15080/225000 [40:50:15<368:35:22,  6.32s/it]
  7%|▋         | 15081/225000 [40:50:23<394:16:37,  6.76s/it]
  7%|▋         | 15082/225000 [40:50:27<355:07:59,  6.09s/it]
  7%|▋         | 15083/225000 [40:50:31<325:14:12,  5.58s/it]
  7%|▋         | 15084/225000 [40:50:42<414:54:43,  7.12s/it]
  7%|▋         | 15085/225000 [40:50:49<414:11:02,  7.10s/it]
  7%|▋         | 15086/225000 [40:50:57<420:13:53,  7.21s/it]
  7%|▋         | 15087/225000 [40:51:03<405:28:14,  6.95s/it]
  7%|▋         | 15088/225000 [40:51:08<363:05:47,  6.23s/it]
  7%|▋         | 15089/225000 [40:51:12<338:28:13,  5.80s/it]
  7%|▋         | 15090/225000 [40:51:17<322:43:43,  5.53s/it]
  7%|▋         | 15091/225000 [40:51:22<312:50:08,  5.37s/it]
  7%|▋         | 15092/225000 [40:51:33<399:57:16,  6.86s/it]
  7%|▋         | 15093/225000 [40:51:41<425:41:37,  7.30s/it]
  7%|▋         | 15094/225000 [40:51:47<408:27:15,  7.01s/it]
  7%|▋         | 15095/225000 [40:51:52<375:36:02,  6.44s/it]
  7%|▋         | 15096/225000 [40:51:57<343:43:49,  5.90s/it]
  7%|▋         | 15097/225000 [40:52:03<339:52:43,  5.83s/it]
  7%|▋         | 15098/225000 [40:52:16<465:25:38,  7.98s/it]
  7%|▋         | 15099/225000 [40:52:20<405:25:31,  6.95s/it]
  7%|▋         | 15100/225000 [40:52:25<375:57:52,  6.45s/it]
                                                             
{'loss': '0.7265', 'grad_norm': '0.7097', 'learning_rate': '4.993e-05', 'epoch': '0.06711', 'num_input_tokens_seen': 72649848, 'train_runtime': '1.471e+05', 'train_tokens_per_second': '493.7'}

  7%|▋         | 15100/225000 [40:52:25<375:57:52,  6.45s/it]
  7%|▋         | 15101/225000 [40:52:31<360:56:25,  6.19s/it]
  7%|▋         | 15102/225000 [40:52:36<343:53:24,  5.90s/it]
  7%|▋         | 15103/225000 [40:52:48<454:16:04,  7.79s/it]
  7%|▋         | 15104/225000 [40:52:53<394:21:39,  6.76s/it]
  7%|▋         | 15105/225000 [40:52:57<354:25:19,  6.08s/it]
  7%|▋         | 15106/225000 [40:53:09<446:42:49,  7.66s/it]
  7%|▋         | 15107/225000 [40:53:16<435:21:44,  7.47s/it]
  7%|▋         | 15108/225000 [40:53:23<427:41:13,  7.34s/it]
  7%|▋         | 15109/225000 [40:53:27<367:05:32,  6.30s/it]
  7%|▋         | 15110/225000 [40:53:36<424:04:23,  7.27s/it]
  7%|▋         | 15111/225000 [40:53:41<388:46:26,  6.67s/it]
  7%|▋         | 15112/225000 [40:53:47<366:03:47,  6.28s/it]
  7%|▋         | 15113/225000 [40:53:51<324:26:16,  5.56s/it]
  7%|▋         | 15114/225000 [40:53:55<306:51:49,  5.26s/it]
  7%|▋         | 15115/225000 [40:54:04<369:39:20,  6.34s/it]
  7%|▋         | 15116/225000 [40:54:12<405:01:21,  6.95s/it]
  7%|▋         | 15117/225000 [40:54:16<351:21:57,  6.03s/it]
  7%|▋         | 15118/225000 [40:54:27<426:26:14,  7.31s/it]
  7%|▋         | 15119/225000 [40:54:31<380:18:54,  6.52s/it]
  7%|▋         | 15120/225000 [40:54:36<344:24:20,  5.91s/it]
  7%|▋         | 15121/225000 [40:54:48<456:58:46,  7.84s/it]
  7%|▋         | 15122/225000 [40:54:53<403:45:30,  6.93s/it]
  7%|▋         | 15123/225000 [40:55:00<401:43:35,  6.89s/it]
  7%|▋         | 15124/225000 [40:55:04<361:21:27,  6.20s/it]
  7%|▋         | 15125/225000 [40:55:12<385:04:00,  6.61s/it]
  7%|▋         | 15126/225000 [40:55:20<411:17:25,  7.05s/it]
  7%|▋         | 15127/225000 [40:55:29<453:28:14,  7.78s/it]
  7%|▋         | 15128/225000 [40:55:34<396:56:37,  6.81s/it]
  7%|▋         | 15129/225000 [40:55:39<359:06:37,  6.16s/it]
  7%|▋         | 15130/225000 [40:55:43<335:33:14,  5.76s/it]
  7%|▋         | 15131/225000 [40:55:48<316:11:01,  5.42s/it]
  7%|▋         | 15132/225000 [40:56:01<439:59:43,  7.55s/it]
  7%|▋         | 15133/225000 [40:56:05<377:43:38,  6.48s/it]
  7%|▋         | 15134/225000 [40:56:12<397:05:09,  6.81s/it]
  7%|▋         | 15135/225000 [40:56:18<386:10:32,  6.62s/it]
  7%|▋         | 15136/225000 [40:56:23<355:36:00,  6.10s/it]
  7%|▋         | 15137/225000 [40:56:28<338:40:39,  5.81s/it]
  7%|▋         | 15138/225000 [40:56:33<324:45:55,  5.57s/it]
  7%|▋         | 15139/225000 [40:56:46<443:27:50,  7.61s/it]
  7%|▋         | 15140/225000 [40:56:50<378:32:31,  6.49s/it]
  7%|▋         | 15141/225000 [40:56:57<391:30:02,  6.72s/it]
  7%|▋         | 15142/225000 [40:57:04<401:13:01,  6.88s/it]
  7%|▋         | 15143/225000 [40:57:09<364:17:36,  6.25s/it]
  7%|▋         | 15144/225000 [40:57:18<409:25:31,  7.02s/it]
  7%|▋         | 15145/225000 [40:57:22<355:18:47,  6.10s/it]
  7%|▋         | 15146/225000 [40:57:32<422:43:13,  7.25s/it]
  7%|▋         | 15147/225000 [40:57:36<370:46:04,  6.36s/it]
  7%|▋         | 15148/225000 [40:57:41<347:02:05,  5.95s/it]
  7%|▋         | 15149/225000 [40:57:50<400:40:44,  6.87s/it]
  7%|▋         | 15150/225000 [40:57:56<383:02:02,  6.57s/it]
                                                             
{'loss': '0.7356', 'grad_norm': '0.6769', 'learning_rate': '4.993e-05', 'epoch': '0.06733', 'num_input_tokens_seen': 72802432, 'train_runtime': '1.475e+05', 'train_tokens_per_second': '493.7'}

  7%|▋         | 15150/225000 [40:57:56<383:02:02,  6.57s/it]
  7%|▋         | 15151/225000 [40:58:02<378:26:55,  6.49s/it]
  7%|▋         | 15152/225000 [40:58:11<414:56:02,  7.12s/it]
  7%|▋         | 15153/225000 [40:58:16<386:33:28,  6.63s/it]
  7%|▋         | 15154/225000 [40:58:23<386:10:59,  6.63s/it]
  7%|▋         | 15155/225000 [40:58:34<472:43:33,  8.11s/it]
  7%|▋         | 15156/225000 [40:58:40<431:46:27,  7.41s/it]
  7%|▋         | 15157/225000 [40:58:47<429:37:16,  7.37s/it]
  7%|▋         | 15158/225000 [40:58:54<415:48:59,  7.13s/it]
  7%|▋         | 15159/225000 [40:59:00<396:03:40,  6.79s/it]
  7%|▋         | 15160/225000 [40:59:07<406:53:47,  6.98s/it]
  7%|▋         | 15161/225000 [40:59:12<370:06:44,  6.35s/it]
  7%|▋         | 15162/225000 [40:59:17<344:24:44,  5.91s/it]
  7%|▋         | 15163/225000 [40:59:22<322:13:45,  5.53s/it]
  7%|▋         | 15164/225000 [40:59:28<340:12:07,  5.84s/it]
  7%|▋         | 15165/225000 [40:59:39<430:41:56,  7.39s/it]
  7%|▋         | 15166/225000 [40:59:43<368:08:18,  6.32s/it]
  7%|▋         | 15167/225000 [40:59:52<413:20:46,  7.09s/it]
  7%|▋         | 15168/225000 [40:59:56<363:03:48,  6.23s/it]
  7%|▋         | 15169/225000 [41:00:01<338:02:20,  5.80s/it]
  7%|▋         | 15170/225000 [41:00:14<461:44:32,  7.92s/it]
  7%|▋         | 15171/225000 [41:00:18<389:31:50,  6.68s/it]
  7%|▋         | 15172/225000 [41:00:27<431:32:19,  7.40s/it]
  7%|▋         | 15173/225000 [41:00:32<395:03:39,  6.78s/it]
  7%|▋         | 15174/225000 [41:00:37<361:07:21,  6.20s/it]
  7%|▋         | 15175/225000 [41:00:42<336:53:03,  5.78s/it]
  7%|▋         | 15176/225000 [41:00:47<319:01:23,  5.47s/it]
  7%|▋         | 15177/225000 [41:00:55<368:58:12,  6.33s/it]
  7%|▋         | 15178/225000 [41:01:06<447:56:47,  7.69s/it]
  7%|▋         | 15179/225000 [41:01:12<415:26:32,  7.13s/it]
  7%|▋         | 15180/225000 [41:01:17<381:50:44,  6.55s/it]
  7%|▋         | 15181/225000 [41:01:22<359:51:58,  6.17s/it]
  7%|▋         | 15182/225000 [41:01:27<344:01:52,  5.90s/it]
  7%|▋         | 15183/225000 [41:01:33<334:24:22,  5.74s/it]
  7%|▋         | 15184/225000 [41:01:38<325:22:00,  5.58s/it]
  7%|▋         | 15185/225000 [41:01:43<323:13:59,  5.55s/it]
  7%|▋         | 15186/225000 [41:01:56<445:13:15,  7.64s/it]
  7%|▋         | 15187/225000 [41:02:02<423:32:48,  7.27s/it]
  7%|▋         | 15188/225000 [41:02:07<382:03:08,  6.56s/it]
  7%|▋         | 15189/225000 [41:02:12<347:52:25,  5.97s/it]
  7%|▋         | 15190/225000 [41:02:17<333:48:32,  5.73s/it]
  7%|▋         | 15191/225000 [41:02:22<323:44:00,  5.55s/it]
  7%|▋         | 15192/225000 [41:02:34<442:21:40,  7.59s/it]
  7%|▋         | 15193/225000 [41:02:42<432:20:32,  7.42s/it]
  7%|▋         | 15194/225000 [41:02:47<395:40:48,  6.79s/it]
  7%|▋         | 15195/225000 [41:02:51<347:39:23,  5.97s/it]
  7%|▋         | 15196/225000 [41:03:02<443:14:34,  7.61s/it]
  7%|▋         | 15197/225000 [41:03:07<392:01:22,  6.73s/it]
  7%|▋         | 15198/225000 [41:03:13<373:20:01,  6.41s/it]
  7%|▋         | 15199/225000 [41:03:17<336:51:10,  5.78s/it]
  7%|▋         | 15200/225000 [41:03:21<310:20:03,  5.33s/it]
                                                             
{'loss': '0.734', 'grad_norm': '0.6513', 'learning_rate': '4.993e-05', 'epoch': '0.06756', 'num_input_tokens_seen': 72953424, 'train_runtime': '1.478e+05', 'train_tokens_per_second': '493.6'}

  7%|▋         | 15200/225000 [41:03:21<310:20:03,  5.33s/it]
  7%|▋         | 15201/225000 [41:03:34<449:21:02,  7.71s/it]
  7%|▋         | 15202/225000 [41:03:39<384:53:30,  6.60s/it]
  7%|▋         | 15203/225000 [41:03:45<385:01:15,  6.61s/it]
  7%|▋         | 15204/225000 [41:03:50<348:21:08,  5.98s/it]
  7%|▋         | 15205/225000 [41:03:54<321:32:13,  5.52s/it]
  7%|▋         | 15206/225000 [41:04:07<445:19:25,  7.64s/it]
  7%|▋         | 15207/225000 [41:04:11<382:32:40,  6.56s/it]
  7%|▋         | 15208/225000 [41:04:15<339:04:48,  5.82s/it]
  7%|▋         | 15209/225000 [41:04:25<411:58:54,  7.07s/it]
  7%|▋         | 15210/225000 [41:04:29<368:07:30,  6.32s/it]
  7%|▋         | 15211/225000 [41:04:36<366:55:40,  6.30s/it]
  7%|▋         | 15212/225000 [41:04:44<400:46:58,  6.88s/it]
  7%|▋         | 15213/225000 [41:04:48<351:41:30,  6.04s/it]
  7%|▋         | 15214/225000 [41:04:55<373:16:28,  6.41s/it]
  7%|▋         | 15215/225000 [41:05:02<386:42:37,  6.64s/it]
  7%|▋         | 15216/225000 [41:05:10<408:19:10,  7.01s/it]
  7%|▋         | 15217/225000 [41:05:17<396:38:29,  6.81s/it]
  7%|▋         | 15218/225000 [41:05:23<395:10:25,  6.78s/it]
  7%|▋         | 15219/225000 [41:05:27<337:37:38,  5.79s/it]
  7%|▋         | 15220/225000 [41:05:37<411:37:15,  7.06s/it]
  7%|▋         | 15221/225000 [41:05:40<349:53:32,  6.00s/it]
  7%|▋         | 15222/225000 [41:05:48<381:33:32,  6.55s/it]
  7%|▋         | 15223/225000 [41:05:55<393:18:06,  6.75s/it]
  7%|▋         | 15224/225000 [41:06:03<408:25:54,  7.01s/it]
  7%|▋         | 15225/225000 [41:06:08<372:47:13,  6.40s/it]
  7%|▋         | 15226/225000 [41:06:13<341:27:36,  5.86s/it]
  7%|▋         | 15227/225000 [41:06:17<323:20:50,  5.55s/it]
  7%|▋         | 15228/225000 [41:06:22<306:36:17,  5.26s/it]
  7%|▋         | 15229/225000 [41:06:27<298:05:15,  5.12s/it]
  7%|▋         | 15230/225000 [41:06:34<338:33:03,  5.81s/it]
  7%|▋         | 15231/225000 [41:06:46<440:19:03,  7.56s/it]
  7%|▋         | 15232/225000 [41:06:54<446:16:46,  7.66s/it]
  7%|▋         | 15233/225000 [41:07:00<419:29:13,  7.20s/it]
  7%|▋         | 15234/225000 [41:07:05<382:00:09,  6.56s/it]
  7%|▋         | 15235/225000 [41:07:15<435:49:31,  7.48s/it]
  7%|▋         | 15236/225000 [41:07:19<379:16:17,  6.51s/it]
  7%|▋         | 15237/225000 [41:07:26<400:07:16,  6.87s/it]
  7%|▋         | 15238/225000 [41:07:34<407:00:43,  6.99s/it]
  7%|▋         | 15239/225000 [41:07:38<367:28:35,  6.31s/it]
  7%|▋         | 15240/225000 [41:07:49<438:04:03,  7.52s/it]
  7%|▋         | 15241/225000 [41:07:53<379:42:38,  6.52s/it]
  7%|▋         | 15242/225000 [41:07:57<338:21:57,  5.81s/it]
  7%|▋         | 15243/225000 [41:08:02<317:11:21,  5.44s/it]
  7%|▋         | 15244/225000 [41:08:07<312:47:30,  5.37s/it]
  7%|▋         | 15245/225000 [41:08:13<319:51:20,  5.49s/it]
  7%|▋         | 15246/225000 [41:08:18<320:50:26,  5.51s/it]
  7%|▋         | 15247/225000 [41:08:24<318:05:08,  5.46s/it]
  7%|▋         | 15248/225000 [41:08:34<404:35:54,  6.94s/it]
  7%|▋         | 15249/225000 [41:08:45<474:40:24,  8.15s/it]
  7%|▋         | 15250/225000 [41:08:50<418:32:18,  7.18s/it]
                                                             
{'loss': '0.7208', 'grad_norm': '0.5462', 'learning_rate': '4.993e-05', 'epoch': '0.06778', 'num_input_tokens_seen': 73106592, 'train_runtime': '1.481e+05', 'train_tokens_per_second': '493.5'}

  7%|▋         | 15250/225000 [41:08:50<418:32:18,  7.18s/it]
  7%|▋         | 15251/225000 [41:08:58<429:55:30,  7.38s/it]
  7%|▋         | 15252/225000 [41:09:03<396:18:15,  6.80s/it]
  7%|▋         | 15253/225000 [41:09:09<375:19:02,  6.44s/it]
  7%|▋         | 15254/225000 [41:09:14<349:08:18,  5.99s/it]
  7%|▋         | 15255/225000 [41:09:20<348:47:22,  5.99s/it]
  7%|▋         | 15256/225000 [41:09:25<340:35:53,  5.85s/it]
  7%|▋         | 15257/225000 [41:09:32<350:12:18,  6.01s/it]
  7%|▋         | 15258/225000 [41:09:43<452:05:37,  7.76s/it]
  7%|▋         | 15259/225000 [41:09:49<412:36:54,  7.08s/it]
  7%|▋         | 15260/225000 [41:09:56<414:23:59,  7.11s/it]
  7%|▋         | 15261/225000 [41:10:05<437:29:37,  7.51s/it]
  7%|▋         | 15262/225000 [41:10:12<429:18:29,  7.37s/it]
  7%|▋         | 15263/225000 [41:10:18<411:42:53,  7.07s/it]
  7%|▋         | 15264/225000 [41:10:22<363:40:59,  6.24s/it]
  7%|▋         | 15265/225000 [41:10:29<370:45:21,  6.36s/it]
  7%|▋         | 15266/225000 [41:10:39<428:11:56,  7.35s/it]
  7%|▋         | 15267/225000 [41:10:45<414:40:23,  7.12s/it]
  7%|▋         | 15268/225000 [41:10:50<377:07:10,  6.47s/it]
  7%|▋         | 15269/225000 [41:10:55<348:09:36,  5.98s/it]
  7%|▋         | 15270/225000 [41:11:00<331:59:07,  5.70s/it]
  7%|▋         | 15271/225000 [41:11:05<320:15:08,  5.50s/it]
  7%|▋         | 15272/225000 [41:11:18<450:40:37,  7.74s/it]
  7%|▋         | 15273/225000 [41:11:22<387:28:22,  6.65s/it]
  7%|▋         | 15274/225000 [41:11:30<410:58:41,  7.05s/it]
  7%|▋         | 15275/225000 [41:11:36<392:42:46,  6.74s/it]
  7%|▋         | 15276/225000 [41:11:41<364:32:27,  6.26s/it]
  7%|▋         | 15277/225000 [41:11:47<347:59:39,  5.97s/it]
  7%|▋         | 15278/225000 [41:11:52<331:32:23,  5.69s/it]
  7%|▋         | 15279/225000 [41:11:57<321:45:06,  5.52s/it]
  7%|▋         | 15280/225000 [41:12:16<555:02:32,  9.53s/it]
  7%|▋         | 15281/225000 [41:12:23<510:03:50,  8.76s/it]
  7%|▋         | 15282/225000 [41:12:27<430:51:59,  7.40s/it]
  7%|▋         | 15283/225000 [41:12:33<403:05:56,  6.92s/it]
  7%|▋         | 15284/225000 [41:12:42<447:02:57,  7.67s/it]
  7%|▋         | 15285/225000 [41:12:48<420:25:27,  7.22s/it]
  7%|▋         | 15286/225000 [41:12:53<371:36:32,  6.38s/it]
  7%|▋         | 15287/225000 [41:13:02<428:45:38,  7.36s/it]
  7%|▋         | 15288/225000 [41:13:07<386:16:26,  6.63s/it]
  7%|▋         | 15289/225000 [41:13:12<353:25:08,  6.07s/it]
  7%|▋         | 15290/225000 [41:13:17<338:12:00,  5.81s/it]
  7%|▋         | 15291/225000 [41:13:22<324:47:04,  5.58s/it]
  7%|▋         | 15292/225000 [41:13:27<310:07:54,  5.32s/it]
  7%|▋         | 15293/225000 [41:13:32<314:41:56,  5.40s/it]
  7%|▋         | 15294/225000 [41:13:38<317:18:15,  5.45s/it]
  7%|▋         | 15295/225000 [41:13:51<448:32:38,  7.70s/it]
  7%|▋         | 15296/225000 [41:13:56<409:00:24,  7.02s/it]
  7%|▋         | 15297/225000 [41:14:01<362:20:50,  6.22s/it]
  7%|▋         | 15298/225000 [41:14:11<440:24:13,  7.56s/it]
  7%|▋         | 15299/225000 [41:14:16<383:14:33,  6.58s/it]
  7%|▋         | 15300/225000 [41:14:25<433:16:55,  7.44s/it]
                                                             
{'loss': '0.7356', 'grad_norm': '0.7114', 'learning_rate': '4.993e-05', 'epoch': '0.068', 'num_input_tokens_seen': 73260896, 'train_runtime': '1.485e+05', 'train_tokens_per_second': '493.5'}

  7%|▋         | 15300/225000 [41:14:25<433:16:55,  7.44s/it]
  7%|▋         | 15301/225000 [41:14:31<401:37:36,  6.89s/it]
  7%|▋         | 15302/225000 [41:14:37<390:27:53,  6.70s/it]
  7%|▋         | 15303/225000 [41:14:42<367:52:18,  6.32s/it]
  7%|▋         | 15304/225000 [41:14:50<392:52:38,  6.74s/it]
  7%|▋         | 15305/225000 [41:14:55<363:09:40,  6.23s/it]
  7%|▋         | 15306/225000 [41:15:03<390:19:34,  6.70s/it]
  7%|▋         | 15307/225000 [41:15:10<389:41:07,  6.69s/it]
  7%|▋         | 15308/225000 [41:15:15<361:43:01,  6.21s/it]
  7%|▋         | 15309/225000 [41:15:25<426:26:21,  7.32s/it]
  7%|▋         | 15310/225000 [41:15:31<407:22:30,  6.99s/it]
  7%|▋         | 15311/225000 [41:15:40<435:27:04,  7.48s/it]
  7%|▋         | 15312/225000 [41:15:44<379:05:29,  6.51s/it]
  7%|▋         | 15313/225000 [41:15:50<378:37:22,  6.50s/it]
  7%|▋         | 15314/225000 [41:16:02<474:31:46,  8.15s/it]
  7%|▋         | 15315/225000 [41:16:08<434:18:29,  7.46s/it]
  7%|▋         | 15316/225000 [41:16:14<398:05:15,  6.83s/it]
  7%|▋         | 15317/225000 [41:16:18<361:36:21,  6.21s/it]
  7%|▋         | 15318/225000 [41:16:24<349:01:03,  5.99s/it]
  7%|▋         | 15319/225000 [41:16:29<335:38:39,  5.76s/it]
  7%|▋         | 15320/225000 [41:16:34<315:45:04,  5.42s/it]
  7%|▋         | 15321/225000 [41:16:39<317:13:09,  5.45s/it]
  7%|▋         | 15322/225000 [41:16:45<322:00:25,  5.53s/it]
  7%|▋         | 15323/225000 [41:16:50<314:38:04,  5.40s/it]
  7%|▋         | 15324/225000 [41:17:03<447:55:05,  7.69s/it]
  7%|▋         | 15325/225000 [41:17:09<418:49:33,  7.19s/it]
  7%|▋         | 15326/225000 [41:17:13<368:27:47,  6.33s/it]
  7%|▋         | 15327/225000 [41:17:18<339:12:35,  5.82s/it]
  7%|▋         | 15328/225000 [41:17:23<322:39:40,  5.54s/it]
  7%|▋         | 15329/225000 [41:17:33<398:00:27,  6.83s/it]
  7%|▋         | 15330/225000 [41:17:42<437:19:33,  7.51s/it]
  7%|▋         | 15331/225000 [41:17:49<436:27:09,  7.49s/it]
  7%|▋         | 15332/225000 [41:17:53<369:57:46,  6.35s/it]
  7%|▋         | 15333/225000 [41:18:02<418:53:43,  7.19s/it]
  7%|▋         | 15334/225000 [41:18:07<371:35:32,  6.38s/it]
  7%|▋         | 15335/225000 [41:18:15<407:10:35,  6.99s/it]
  7%|▋         | 15336/225000 [41:18:20<373:32:17,  6.41s/it]
  7%|▋         | 15337/225000 [41:18:25<351:15:08,  6.03s/it]
  7%|▋         | 15338/225000 [41:18:31<342:28:47,  5.88s/it]
  7%|▋         | 15339/225000 [41:18:36<333:40:19,  5.73s/it]
  7%|▋         | 15340/225000 [41:18:48<438:39:25,  7.53s/it]
  7%|▋         | 15341/225000 [41:18:54<422:48:47,  7.26s/it]
  7%|▋         | 15342/225000 [41:19:00<388:56:54,  6.68s/it]
  7%|▋         | 15343/225000 [41:19:05<366:08:18,  6.29s/it]
  7%|▋         | 15344/225000 [41:19:11<352:14:48,  6.05s/it]
  7%|▋         | 15345/225000 [41:19:17<349:17:56,  6.00s/it]
  7%|▋         | 15346/225000 [41:19:28<440:56:46,  7.57s/it]
  7%|▋         | 15347/225000 [41:19:34<412:11:11,  7.08s/it]
  7%|▋         | 15348/225000 [41:19:42<438:17:52,  7.53s/it]
  7%|▋         | 15349/225000 [41:19:49<422:00:52,  7.25s/it]
  7%|▋         | 15350/225000 [41:19:55<406:10:57,  6.97s/it]
                                                             
{'loss': '0.739', 'grad_norm': '0.634', 'learning_rate': '4.992e-05', 'epoch': '0.06822', 'num_input_tokens_seen': 73413688, 'train_runtime': '1.488e+05', 'train_tokens_per_second': '493.4'}

  7%|▋         | 15350/225000 [41:19:55<406:10:57,  6.97s/it]
  7%|▋         | 15351/225000 [41:20:00<368:37:46,  6.33s/it]
  7%|▋         | 15352/225000 [41:20:11<448:35:13,  7.70s/it]
  7%|▋         | 15353/225000 [41:20:16<406:43:31,  6.98s/it]
  7%|▋         | 15354/225000 [41:20:23<400:54:37,  6.88s/it]
  7%|▋         | 15355/225000 [41:20:28<378:41:27,  6.50s/it]
  7%|▋         | 15356/225000 [41:20:33<347:54:42,  5.97s/it]
  7%|▋         | 15357/225000 [41:20:38<328:39:12,  5.64s/it]
  7%|▋         | 15358/225000 [41:20:43<311:41:52,  5.35s/it]
  7%|▋         | 15359/225000 [41:20:53<402:57:29,  6.92s/it]
  7%|▋         | 15360/225000 [41:21:00<406:18:30,  6.98s/it]
  7%|▋         | 15361/225000 [41:21:06<388:03:48,  6.66s/it]
  7%|▋         | 15362/225000 [41:21:13<385:33:25,  6.62s/it]
  7%|▋         | 15363/225000 [41:21:21<416:16:50,  7.15s/it]
  7%|▋         | 15364/225000 [41:21:27<382:29:54,  6.57s/it]
  7%|▋         | 15365/225000 [41:21:31<351:31:12,  6.04s/it]
  7%|▋         | 15366/225000 [41:21:36<328:22:34,  5.64s/it]
  7%|▋         | 15367/225000 [41:21:48<441:34:36,  7.58s/it]
  7%|▋         | 15368/225000 [41:21:53<393:16:40,  6.75s/it]
  7%|▋         | 15369/225000 [41:22:00<394:46:07,  6.78s/it]
  7%|▋         | 15370/225000 [41:22:07<401:01:23,  6.89s/it]
  7%|▋         | 15371/225000 [41:22:11<345:56:17,  5.94s/it]
  7%|▋         | 15372/225000 [41:22:18<375:03:53,  6.44s/it]
  7%|▋         | 15373/225000 [41:22:27<418:36:01,  7.19s/it]
  7%|▋         | 15374/225000 [41:22:32<381:03:59,  6.54s/it]
  7%|▋         | 15375/225000 [41:22:41<422:51:44,  7.26s/it]
  7%|▋         | 15376/225000 [41:22:50<448:49:53,  7.71s/it]
  7%|▋         | 15377/225000 [41:22:55<394:31:21,  6.78s/it]
  7%|▋         | 15378/225000 [41:23:01<383:58:44,  6.59s/it]
  7%|▋         | 15379/225000 [41:23:11<448:29:50,  7.70s/it]
  7%|▋         | 15380/225000 [41:23:17<415:29:01,  7.14s/it]
  7%|▋         | 15381/225000 [41:23:22<373:55:14,  6.42s/it]
  7%|▋         | 15382/225000 [41:23:27<349:50:40,  6.01s/it]
  7%|▋         | 15383/225000 [41:23:32<341:48:36,  5.87s/it]
  7%|▋         | 15384/225000 [41:23:37<325:56:45,  5.60s/it]
  7%|▋         | 15385/225000 [41:23:50<457:48:14,  7.86s/it]
  7%|▋         | 15386/225000 [41:23:56<426:54:43,  7.33s/it]
  7%|▋         | 15387/225000 [41:24:01<379:05:28,  6.51s/it]
  7%|▋         | 15388/225000 [41:24:13<470:15:22,  8.08s/it]
  7%|▋         | 15389/225000 [41:24:19<447:42:04,  7.69s/it]
  7%|▋         | 15390/225000 [41:24:25<403:51:14,  6.94s/it]
  7%|▋         | 15391/225000 [41:24:29<365:42:30,  6.28s/it]
  7%|▋         | 15392/225000 [41:24:35<349:42:14,  6.01s/it]
  7%|▋         | 15393/225000 [41:24:40<339:29:42,  5.83s/it]
  7%|▋         | 15394/225000 [41:24:45<325:03:51,  5.58s/it]
  7%|▋         | 15395/225000 [41:24:54<389:20:44,  6.69s/it]
  7%|▋         | 15396/225000 [41:25:04<434:24:17,  7.46s/it]
  7%|▋         | 15397/225000 [41:25:07<366:17:22,  6.29s/it]
  7%|▋         | 15398/225000 [41:25:16<404:45:17,  6.95s/it]
  7%|▋         | 15399/225000 [41:25:24<429:32:11,  7.38s/it]
  7%|▋         | 15400/225000 [41:25:29<381:18:47,  6.55s/it]
                                                             
{'loss': '0.7364', 'grad_norm': '0.5957', 'learning_rate': '4.992e-05', 'epoch': '0.06844', 'num_input_tokens_seen': 73566096, 'train_runtime': '1.491e+05', 'train_tokens_per_second': '493.3'}

  7%|▋         | 15400/225000 [41:25:29<381:18:47,  6.55s/it]
  7%|▋         | 15401/225000 [41:25:34<360:29:18,  6.19s/it]
  7%|▋         | 15402/225000 [41:25:43<415:41:45,  7.14s/it]
  7%|▋         | 15403/225000 [41:25:50<398:22:55,  6.84s/it]
  7%|▋         | 15404/225000 [41:25:55<380:59:43,  6.54s/it]
  7%|▋         | 15405/225000 [41:26:01<371:06:17,  6.37s/it]
  7%|▋         | 15406/225000 [41:26:06<342:21:20,  5.88s/it]
  7%|▋         | 15407/225000 [41:26:11<321:17:03,  5.52s/it]
  7%|▋         | 15408/225000 [41:26:22<426:17:57,  7.32s/it]
  7%|▋         | 15409/225000 [41:26:28<390:21:13,  6.70s/it]
  7%|▋         | 15410/225000 [41:26:34<384:49:44,  6.61s/it]
  7%|▋         | 15411/225000 [41:26:40<379:00:48,  6.51s/it]
  7%|▋         | 15412/225000 [41:26:49<411:54:52,  7.08s/it]
  7%|▋         | 15413/225000 [41:26:53<357:52:36,  6.15s/it]
  7%|▋         | 15414/225000 [41:26:57<333:55:34,  5.74s/it]
  7%|▋         | 15415/225000 [41:27:07<400:17:02,  6.88s/it]
  7%|▋         | 15416/225000 [41:27:11<346:07:00,  5.95s/it]
  7%|▋         | 15417/225000 [41:27:15<316:45:44,  5.44s/it]
  7%|▋         | 15418/225000 [41:27:24<386:13:33,  6.63s/it]
  7%|▋         | 15419/225000 [41:27:32<402:51:38,  6.92s/it]
  7%|▋         | 15420/225000 [41:27:39<404:37:30,  6.95s/it]
  7%|▋         | 15421/225000 [41:27:46<396:44:48,  6.82s/it]
  7%|▋         | 15422/225000 [41:27:50<363:41:14,  6.25s/it]
  7%|▋         | 15423/225000 [41:27:58<379:48:20,  6.52s/it]
  7%|▋         | 15424/225000 [41:28:06<420:10:18,  7.22s/it]
  7%|▋         | 15425/225000 [41:28:11<365:28:51,  6.28s/it]
  7%|▋         | 15426/225000 [41:28:20<413:38:36,  7.11s/it]
  7%|▋         | 15427/225000 [41:28:29<450:20:41,  7.74s/it]
  7%|▋         | 15428/225000 [41:28:35<415:25:11,  7.14s/it]
  7%|▋         | 15429/225000 [41:28:42<424:37:33,  7.29s/it]
  7%|▋         | 15430/225000 [41:28:47<374:28:31,  6.43s/it]
  7%|▋         | 15431/225000 [41:28:52<361:13:40,  6.21s/it]
  7%|▋         | 15432/225000 [41:29:02<427:18:36,  7.34s/it]
  7%|▋         | 15433/225000 [41:29:07<373:23:09,  6.41s/it]
  7%|▋         | 15434/225000 [41:29:11<336:45:43,  5.79s/it]
  7%|▋         | 15435/225000 [41:29:16<324:45:23,  5.58s/it]
  7%|▋         | 15436/225000 [41:29:30<475:46:03,  8.17s/it]
  7%|▋         | 15437/225000 [41:29:34<393:40:58,  6.76s/it]
  7%|▋         | 15438/225000 [41:29:38<347:32:48,  5.97s/it]
  7%|▋         | 15439/225000 [41:29:49<447:03:21,  7.68s/it]
  7%|▋         | 15440/225000 [41:29:55<402:00:49,  6.91s/it]
  7%|▋         | 15441/225000 [41:30:00<379:47:33,  6.52s/it]
  7%|▋         | 15442/225000 [41:30:05<349:06:00,  6.00s/it]
  7%|▋         | 15443/225000 [41:30:10<328:35:10,  5.64s/it]
  7%|▋         | 15444/225000 [41:30:21<432:34:39,  7.43s/it]
  7%|▋         | 15445/225000 [41:30:25<370:05:54,  6.36s/it]
  7%|▋         | 15446/225000 [41:30:31<352:01:08,  6.05s/it]
  7%|▋         | 15447/225000 [41:30:36<340:36:03,  5.85s/it]
  7%|▋         | 15448/225000 [41:30:42<343:24:27,  5.90s/it]
  7%|▋         | 15449/225000 [41:30:47<329:56:32,  5.67s/it]
  7%|▋         | 15450/225000 [41:30:53<327:59:21,  5.63s/it]
                                                             
{'loss': '0.7293', 'grad_norm': '0.6245', 'learning_rate': '4.992e-05', 'epoch': '0.06867', 'num_input_tokens_seen': 73721016, 'train_runtime': '1.495e+05', 'train_tokens_per_second': '493.3'}

  7%|▋         | 15450/225000 [41:30:53<327:59:21,  5.63s/it]
  7%|▋         | 15451/225000 [41:31:03<409:42:13,  7.04s/it]
  7%|▋         | 15452/225000 [41:31:12<453:09:50,  7.79s/it]
  7%|▋         | 15453/225000 [41:31:17<393:05:19,  6.75s/it]
  7%|▋         | 15454/225000 [41:31:22<367:42:45,  6.32s/it]
  7%|▋         | 15455/225000 [41:31:28<357:19:28,  6.14s/it]
  7%|▋         | 15456/225000 [41:31:33<343:10:44,  5.90s/it]
  7%|▋         | 15457/225000 [41:31:47<475:21:27,  8.17s/it]
  7%|▋         | 15458/225000 [41:31:52<424:41:31,  7.30s/it]
  7%|▋         | 15459/225000 [41:31:57<391:10:56,  6.72s/it]
  7%|▋         | 15460/225000 [41:32:03<373:12:38,  6.41s/it]
  7%|▋         | 15461/225000 [41:32:08<355:25:00,  6.11s/it]
  7%|▋         | 15462/225000 [41:32:13<337:09:53,  5.79s/it]
  7%|▋         | 15463/225000 [41:32:19<330:47:06,  5.68s/it]
  7%|▋         | 15464/225000 [41:32:24<326:16:38,  5.61s/it]
  7%|▋         | 15465/225000 [41:32:30<324:36:52,  5.58s/it]
  7%|▋         | 15466/225000 [41:32:36<328:27:36,  5.64s/it]
  7%|▋         | 15467/225000 [41:32:49<464:37:09,  7.98s/it]
  7%|▋         | 15468/225000 [41:32:55<430:37:35,  7.40s/it]
  7%|▋         | 15469/225000 [41:33:04<455:43:50,  7.83s/it]
  7%|▋         | 15470/225000 [41:33:10<420:35:15,  7.23s/it]
  7%|▋         | 15471/225000 [41:33:18<438:46:34,  7.54s/it]
  7%|▋         | 15472/225000 [41:33:24<416:17:56,  7.15s/it]
  7%|▋         | 15473/225000 [41:33:30<398:33:14,  6.85s/it]
  7%|▋         | 15474/225000 [41:33:34<346:27:03,  5.95s/it]
  7%|▋         | 15475/225000 [41:33:45<438:45:41,  7.54s/it]
  7%|▋         | 15476/225000 [41:33:52<423:27:26,  7.28s/it]
  7%|▋         | 15477/225000 [41:33:57<386:45:45,  6.65s/it]
  7%|▋         | 15478/225000 [41:34:03<368:16:22,  6.33s/it]
  7%|▋         | 15479/225000 [41:34:09<355:38:41,  6.11s/it]
  7%|▋         | 15480/225000 [41:34:13<332:16:52,  5.71s/it]
  7%|▋         | 15481/225000 [41:34:20<354:44:38,  6.10s/it]
  7%|▋         | 15482/225000 [41:34:32<454:47:35,  7.81s/it]
  7%|▋         | 15483/225000 [41:34:36<384:28:43,  6.61s/it]
  7%|▋         | 15484/225000 [41:34:45<430:33:22,  7.40s/it]
  7%|▋         | 15485/225000 [41:34:50<382:05:44,  6.57s/it]
  7%|▋         | 15486/225000 [41:34:55<363:31:29,  6.25s/it]
  7%|▋         | 15487/225000 [41:35:04<404:16:13,  6.95s/it]
  7%|▋         | 15488/225000 [41:35:08<354:23:22,  6.09s/it]
  7%|▋         | 15489/225000 [41:35:14<355:21:45,  6.11s/it]
  7%|▋         | 15490/225000 [41:35:23<401:22:46,  6.90s/it]
  7%|▋         | 15491/225000 [41:35:28<374:45:30,  6.44s/it]
  7%|▋         | 15492/225000 [41:35:34<362:01:52,  6.22s/it]
  7%|▋         | 15493/225000 [41:35:40<353:27:38,  6.07s/it]
  7%|▋         | 15494/225000 [41:35:52<459:55:07,  7.90s/it]
  7%|▋         | 15495/225000 [41:35:57<414:22:57,  7.12s/it]
  7%|▋         | 15496/225000 [41:36:03<396:22:58,  6.81s/it]
  7%|▋         | 15497/225000 [41:36:07<350:20:23,  6.02s/it]
  7%|▋         | 15498/225000 [41:36:15<381:19:36,  6.55s/it]
  7%|▋         | 15499/225000 [41:36:22<388:09:53,  6.67s/it]
  7%|▋         | 15500/225000 [41:36:31<430:45:20,  7.40s/it]
                                                             
{'loss': '0.7299', 'grad_norm': '0.61', 'learning_rate': '4.992e-05', 'epoch': '0.06889', 'num_input_tokens_seen': 73874912, 'train_runtime': '1.498e+05', 'train_tokens_per_second': '493.2'}

  7%|▋         | 15500/225000 [41:36:31<430:45:20,  7.40s/it]
  7%|▋         | 15501/225000 [41:36:37<395:50:30,  6.80s/it]
  7%|▋         | 15502/225000 [41:36:41<358:18:19,  6.16s/it]
  7%|▋         | 15503/225000 [41:36:45<324:23:18,  5.57s/it]
  7%|▋         | 15504/225000 [41:36:49<296:09:03,  5.09s/it]
  7%|▋         | 15505/225000 [41:37:01<403:48:46,  6.94s/it]
  7%|▋         | 15506/225000 [41:37:06<380:33:01,  6.54s/it]
  7%|▋         | 15507/225000 [41:37:11<348:11:13,  5.98s/it]
  7%|▋         | 15508/225000 [41:37:16<337:48:20,  5.80s/it]
  7%|▋         | 15509/225000 [41:37:22<338:20:48,  5.81s/it]
  7%|▋         | 15510/225000 [41:37:33<428:02:29,  7.36s/it]
  7%|▋         | 15511/225000 [41:37:40<412:41:49,  7.09s/it]
  7%|▋         | 15512/225000 [41:37:49<448:01:40,  7.70s/it]
  7%|▋         | 15513/225000 [41:37:54<411:20:10,  7.07s/it]
  7%|▋         | 15514/225000 [41:37:59<373:47:24,  6.42s/it]
  7%|▋         | 15515/225000 [41:38:04<340:35:24,  5.85s/it]
  7%|▋         | 15516/225000 [41:38:09<333:02:38,  5.72s/it]
  7%|▋         | 15517/225000 [41:38:15<334:12:07,  5.74s/it]
  7%|▋         | 15518/225000 [41:38:20<327:55:45,  5.64s/it]
  7%|▋         | 15519/225000 [41:38:26<327:39:48,  5.63s/it]
  7%|▋         | 15520/225000 [41:38:34<376:42:06,  6.47s/it]
  7%|▋         | 15521/225000 [41:38:47<490:40:01,  8.43s/it]
  7%|▋         | 15522/225000 [41:38:53<435:15:39,  7.48s/it]
  7%|▋         | 15523/225000 [41:38:58<389:46:13,  6.70s/it]
  7%|▋         | 15524/225000 [41:39:03<360:12:08,  6.19s/it]
  7%|▋         | 15525/225000 [41:39:08<344:31:35,  5.92s/it]
  7%|▋         | 15526/225000 [41:39:13<332:14:48,  5.71s/it]
  7%|▋         | 15527/225000 [41:39:23<396:41:56,  6.82s/it]
  7%|▋         | 15528/225000 [41:39:32<445:00:50,  7.65s/it]
  7%|▋         | 15529/225000 [41:39:38<415:31:06,  7.14s/it]
  7%|▋         | 15530/225000 [41:39:45<406:17:01,  6.98s/it]
  7%|▋         | 15531/225000 [41:39:49<364:41:38,  6.27s/it]
  7%|▋         | 15532/225000 [41:39:54<344:50:01,  5.93s/it]
  7%|▋         | 15533/225000 [41:40:05<432:24:52,  7.43s/it]
  7%|▋         | 15534/225000 [41:40:12<415:09:26,  7.14s/it]
  7%|▋         | 15535/225000 [41:40:20<428:30:02,  7.36s/it]
  7%|▋         | 15536/225000 [41:40:23<365:18:57,  6.28s/it]
  7%|▋         | 15537/225000 [41:40:29<350:04:59,  6.02s/it]
  7%|▋         | 15538/225000 [41:40:35<348:28:52,  5.99s/it]
  7%|▋         | 15539/225000 [41:40:41<345:12:02,  5.93s/it]
  7%|▋         | 15540/225000 [41:40:47<347:41:35,  5.98s/it]
  7%|▋         | 15541/225000 [41:40:53<347:47:20,  5.98s/it]
  7%|▋         | 15542/225000 [41:41:00<368:33:59,  6.33s/it]
  7%|▋         | 15543/225000 [41:41:13<496:15:38,  8.53s/it]
  7%|▋         | 15544/225000 [41:41:22<501:58:07,  8.63s/it]
  7%|▋         | 15545/225000 [41:41:27<434:01:38,  7.46s/it]
  7%|▋         | 15546/225000 [41:41:37<483:31:18,  8.31s/it]
  7%|▋         | 15547/225000 [41:41:46<483:55:35,  8.32s/it]
  7%|▋         | 15548/225000 [41:41:54<482:17:40,  8.29s/it]
  7%|▋         | 15549/225000 [41:41:58<402:41:27,  6.92s/it]
  7%|▋         | 15550/225000 [41:42:06<423:14:18,  7.27s/it]
                                                             
{'loss': '0.7143', 'grad_norm': '0.5688', 'learning_rate': '4.992e-05', 'epoch': '0.06911', 'num_input_tokens_seen': 74030008, 'train_runtime': '1.501e+05', 'train_tokens_per_second': '493.1'}

  7%|▋         | 15550/225000 [41:42:06<423:14:18,  7.27s/it]
  7%|▋         | 15551/225000 [41:42:10<371:09:16,  6.38s/it]
  7%|▋         | 15552/225000 [41:42:16<368:17:59,  6.33s/it]
  7%|▋         | 15553/225000 [41:42:24<394:11:17,  6.78s/it]
  7%|▋         | 15554/225000 [41:42:28<349:38:56,  6.01s/it]
  7%|▋         | 15555/225000 [41:42:33<329:52:02,  5.67s/it]
  7%|▋         | 15556/225000 [41:42:44<421:41:38,  7.25s/it]
  7%|▋         | 15557/225000 [41:42:47<355:09:28,  6.10s/it]
  7%|▋         | 15558/225000 [41:42:54<365:55:12,  6.29s/it]
  7%|▋         | 15559/225000 [41:43:03<405:49:38,  6.98s/it]
  7%|▋         | 15560/225000 [41:43:06<347:08:25,  5.97s/it]
  7%|▋         | 15561/225000 [41:43:14<375:25:50,  6.45s/it]
  7%|▋         | 15562/225000 [41:43:18<334:47:39,  5.75s/it]
  7%|▋         | 15563/225000 [41:43:28<406:47:49,  6.99s/it]
  7%|▋         | 15564/225000 [41:43:32<357:21:46,  6.14s/it]
  7%|▋         | 15565/225000 [41:43:41<397:54:58,  6.84s/it]
  7%|▋         | 15566/225000 [41:43:46<368:48:47,  6.34s/it]
  7%|▋         | 15567/225000 [41:43:51<347:44:35,  5.98s/it]
  7%|▋         | 15568/225000 [41:43:56<336:18:39,  5.78s/it]
  7%|▋         | 15569/225000 [41:44:02<329:26:45,  5.66s/it]
  7%|▋         | 15570/225000 [41:44:08<341:27:00,  5.87s/it]
  7%|▋         | 15571/225000 [41:44:17<399:22:55,  6.87s/it]
  7%|▋         | 15572/225000 [41:44:27<453:21:06,  7.79s/it]
  7%|▋         | 15573/225000 [41:44:35<456:53:08,  7.85s/it]
  7%|▋         | 15574/225000 [41:44:39<393:24:41,  6.76s/it]
  7%|▋         | 15575/225000 [41:44:46<390:01:56,  6.70s/it]
  7%|▋         | 15576/225000 [41:44:53<397:26:02,  6.83s/it]
  7%|▋         | 15577/225000 [41:45:00<404:34:21,  6.95s/it]
  7%|▋         | 15578/225000 [41:45:06<387:05:57,  6.65s/it]
  7%|▋         | 15579/225000 [41:45:15<420:16:19,  7.22s/it]
  7%|▋         | 15580/225000 [41:45:22<415:36:22,  7.14s/it]
  7%|▋         | 15581/225000 [41:45:29<410:29:49,  7.06s/it]
  7%|▋         | 15582/225000 [41:45:32<355:09:00,  6.11s/it]
  7%|▋         | 15583/225000 [41:45:37<323:11:59,  5.56s/it]
  7%|▋         | 15584/225000 [41:45:47<401:52:44,  6.91s/it]
  7%|▋         | 15585/225000 [41:45:52<372:31:39,  6.40s/it]
  7%|▋         | 15586/225000 [41:46:00<399:30:34,  6.87s/it]
  7%|▋         | 15587/225000 [41:46:06<378:57:27,  6.51s/it]
  7%|▋         | 15588/225000 [41:46:13<396:20:19,  6.81s/it]
  7%|▋         | 15589/225000 [41:46:19<385:08:23,  6.62s/it]
  7%|▋         | 15590/225000 [41:46:24<352:14:51,  6.06s/it]
  7%|▋         | 15591/225000 [41:46:29<335:41:44,  5.77s/it]
  7%|▋         | 15592/225000 [41:46:35<344:25:45,  5.92s/it]
  7%|▋         | 15593/225000 [41:46:47<436:29:22,  7.50s/it]
  7%|▋         | 15594/225000 [41:46:55<451:24:56,  7.76s/it]
  7%|▋         | 15595/225000 [41:46:59<386:23:27,  6.64s/it]
  7%|▋         | 15596/225000 [41:47:03<341:57:13,  5.88s/it]
  7%|▋         | 15597/225000 [41:47:13<415:39:56,  7.15s/it]
  7%|▋         | 15598/225000 [41:47:18<367:58:44,  6.33s/it]
  7%|▋         | 15599/225000 [41:47:23<348:51:54,  6.00s/it]
  7%|▋         | 15600/225000 [41:47:28<326:03:43,  5.61s/it]
                                                             
{'loss': '0.7358', 'grad_norm': '0.7193', 'learning_rate': '4.992e-05', 'epoch': '0.06933', 'num_input_tokens_seen': 74184968, 'train_runtime': '1.504e+05', 'train_tokens_per_second': '493.1'}

  7%|▋         | 15600/225000 [41:47:28<326:03:43,  5.61s/it]
  7%|▋         | 15601/225000 [41:47:40<437:31:16,  7.52s/it]
  7%|▋         | 15602/225000 [41:47:45<391:47:16,  6.74s/it]
  7%|▋         | 15603/225000 [41:47:49<356:22:03,  6.13s/it]
  7%|▋         | 15604/225000 [41:47:54<338:29:13,  5.82s/it]
  7%|▋         | 15605/225000 [41:48:03<386:19:13,  6.64s/it]
  7%|▋         | 15606/225000 [41:48:13<455:32:39,  7.83s/it]
  7%|▋         | 15607/225000 [41:48:22<459:51:25,  7.91s/it]
  7%|▋         | 15608/225000 [41:48:25<383:49:10,  6.60s/it]
  7%|▋         | 15609/225000 [41:48:30<359:06:16,  6.17s/it]
  7%|▋         | 15610/225000 [41:48:36<349:46:33,  6.01s/it]
  7%|▋         | 15611/225000 [41:48:42<348:56:11,  6.00s/it]
  7%|▋         | 15612/225000 [41:48:47<339:46:27,  5.84s/it]
  7%|▋         | 15613/225000 [41:48:59<442:27:53,  7.61s/it]
  7%|▋         | 15614/225000 [41:49:07<444:58:48,  7.65s/it]
  7%|▋         | 15615/225000 [41:49:11<387:52:44,  6.67s/it]
  7%|▋         | 15616/225000 [41:49:20<432:35:38,  7.44s/it]
  7%|▋         | 15617/225000 [41:49:27<414:36:14,  7.13s/it]
  7%|▋         | 15618/225000 [41:49:34<406:36:03,  6.99s/it]
  7%|▋         | 15619/225000 [41:49:39<385:36:48,  6.63s/it]
  7%|▋         | 15620/225000 [41:49:45<365:49:42,  6.29s/it]
  7%|▋         | 15621/225000 [41:49:54<407:57:37,  7.01s/it]
  7%|▋         | 15622/225000 [41:49:58<369:44:38,  6.36s/it]
  7%|▋         | 15623/225000 [41:50:06<387:59:11,  6.67s/it]
  7%|▋         | 15624/225000 [41:50:14<419:11:54,  7.21s/it]
  7%|▋         | 15625/225000 [41:50:18<367:36:20,  6.32s/it]
  7%|▋         | 15626/225000 [41:50:27<397:40:36,  6.84s/it]
  7%|▋         | 15627/225000 [41:50:34<405:26:55,  6.97s/it]
  7%|▋         | 15628/225000 [41:50:39<373:20:28,  6.42s/it]
  7%|▋         | 15629/225000 [41:50:44<345:49:45,  5.95s/it]
  7%|▋         | 15630/225000 [41:50:49<328:03:40,  5.64s/it]
  7%|▋         | 15631/225000 [41:50:53<306:31:25,  5.27s/it]
  7%|▋         | 15632/225000 [41:50:58<295:58:12,  5.09s/it]
  7%|▋         | 15633/225000 [41:51:07<359:52:48,  6.19s/it]
  7%|▋         | 15634/225000 [41:51:15<407:13:40,  7.00s/it]
  7%|▋         | 15635/225000 [41:51:20<363:38:20,  6.25s/it]
  7%|▋         | 15636/225000 [41:51:24<330:36:55,  5.68s/it]
  7%|▋         | 15637/225000 [41:51:34<397:14:10,  6.83s/it]
  7%|▋         | 15638/225000 [41:51:41<399:45:20,  6.87s/it]
  7%|▋         | 15639/225000 [41:51:45<357:31:39,  6.15s/it]
  7%|▋         | 15640/225000 [41:51:55<428:48:23,  7.37s/it]
  7%|▋         | 15641/225000 [41:52:04<443:36:17,  7.63s/it]
  7%|▋         | 15642/225000 [41:52:08<384:52:30,  6.62s/it]
  7%|▋         | 15643/225000 [41:52:14<366:31:18,  6.30s/it]
  7%|▋         | 15644/225000 [41:52:19<357:32:51,  6.15s/it]
  7%|▋         | 15645/225000 [41:52:25<347:34:58,  5.98s/it]
  7%|▋         | 15646/225000 [41:52:32<376:13:46,  6.47s/it]
  7%|▋         | 15647/225000 [41:52:42<423:18:29,  7.28s/it]
  7%|▋         | 15648/225000 [41:52:46<376:15:06,  6.47s/it]
  7%|▋         | 15649/225000 [41:52:56<429:15:23,  7.38s/it]
  7%|▋         | 15650/225000 [41:53:02<412:49:56,  7.10s/it]
                                                             
{'loss': '0.749', 'grad_norm': '0.6627', 'learning_rate': '4.992e-05', 'epoch': '0.06956', 'num_input_tokens_seen': 74337984, 'train_runtime': '1.508e+05', 'train_tokens_per_second': '493'}

  7%|▋         | 15650/225000 [41:53:02<412:49:56,  7.10s/it]
  7%|▋         | 15651/225000 [41:53:08<397:41:00,  6.84s/it]
  7%|▋         | 15652/225000 [41:53:12<347:09:14,  5.97s/it]
  7%|▋         | 15653/225000 [41:53:22<404:45:08,  6.96s/it]
  7%|▋         | 15654/225000 [41:53:26<354:28:52,  6.10s/it]
  7%|▋         | 15655/225000 [41:53:38<456:34:12,  7.85s/it]
  7%|▋         | 15656/225000 [41:53:44<428:49:16,  7.37s/it]
  7%|▋         | 15657/225000 [41:53:49<392:44:20,  6.75s/it]
  7%|▋         | 15658/225000 [41:53:56<385:57:40,  6.64s/it]
  7%|▋         | 15659/225000 [41:54:03<392:24:35,  6.75s/it]
  7%|▋         | 15660/225000 [41:54:09<383:20:16,  6.59s/it]
  7%|▋         | 15661/225000 [41:54:17<406:19:04,  6.99s/it]
  7%|▋         | 15662/225000 [41:54:23<392:01:45,  6.74s/it]
  7%|▋         | 15663/225000 [41:54:27<338:52:18,  5.83s/it]
  7%|▋         | 15664/225000 [41:54:31<317:21:37,  5.46s/it]
  7%|▋         | 15665/225000 [41:54:37<322:49:42,  5.55s/it]
  7%|▋         | 15666/225000 [41:54:48<424:28:43,  7.30s/it]
  7%|▋         | 15667/225000 [41:54:53<381:19:42,  6.56s/it]
  7%|▋         | 15668/225000 [41:55:01<398:47:38,  6.86s/it]
  7%|▋         | 15669/225000 [41:55:08<412:41:25,  7.10s/it]
  7%|▋         | 15670/225000 [41:55:14<379:22:51,  6.52s/it]
  7%|▋         | 15671/225000 [41:55:19<356:48:09,  6.14s/it]
  7%|▋         | 15672/225000 [41:55:24<348:19:16,  5.99s/it]
  7%|▋         | 15673/225000 [41:55:30<335:40:33,  5.77s/it]
  7%|▋         | 15674/225000 [41:55:41<431:06:12,  7.41s/it]
  7%|▋         | 15675/225000 [41:55:48<422:20:48,  7.26s/it]
  7%|▋         | 15676/225000 [41:55:53<378:35:53,  6.51s/it]
  7%|▋         | 15677/225000 [41:56:02<434:05:29,  7.47s/it]
  7%|▋         | 15678/225000 [41:56:07<384:13:15,  6.61s/it]
  7%|▋         | 15679/225000 [41:56:12<353:04:00,  6.07s/it]
  7%|▋         | 15680/225000 [41:56:17<330:41:04,  5.69s/it]
  7%|▋         | 15681/225000 [41:56:30<457:41:47,  7.87s/it]
  7%|▋         | 15682/225000 [41:56:37<454:13:23,  7.81s/it]
  7%|▋         | 15683/225000 [41:56:42<401:52:20,  6.91s/it]
  7%|▋         | 15684/225000 [41:56:48<384:22:59,  6.61s/it]
  7%|▋         | 15685/225000 [41:56:58<453:07:26,  7.79s/it]
  7%|▋         | 15686/225000 [41:57:04<416:26:34,  7.16s/it]
  7%|▋         | 15687/225000 [41:57:09<373:58:54,  6.43s/it]
  7%|▋         | 15688/225000 [41:57:21<477:45:26,  8.22s/it]
  7%|▋         | 15689/225000 [41:57:27<430:53:07,  7.41s/it]
  7%|▋         | 15690/225000 [41:57:38<500:22:49,  8.61s/it]
  7%|▋         | 15691/225000 [41:57:44<456:36:38,  7.85s/it]
  7%|▋         | 15692/225000 [41:57:51<435:56:51,  7.50s/it]
  7%|▋         | 15693/225000 [41:57:56<385:02:28,  6.62s/it]
  7%|▋         | 15694/225000 [41:58:04<425:25:27,  7.32s/it]
  7%|▋         | 15695/225000 [41:58:08<366:38:17,  6.31s/it]
  7%|▋         | 15696/225000 [41:58:16<392:33:25,  6.75s/it]
  7%|▋         | 15697/225000 [41:58:22<382:34:58,  6.58s/it]
  7%|▋         | 15698/225000 [41:58:27<341:49:02,  5.88s/it]
  7%|▋         | 15699/225000 [41:58:35<380:27:10,  6.54s/it]
  7%|▋         | 15700/225000 [41:58:40<357:52:50,  6.16s/it]
                                                             
{'loss': '0.7268', 'grad_norm': '0.602', 'learning_rate': '4.991e-05', 'epoch': '0.06978', 'num_input_tokens_seen': 74493288, 'train_runtime': '1.511e+05', 'train_tokens_per_second': '492.9'}

  7%|▋         | 15700/225000 [41:58:40<357:52:50,  6.16s/it]
  7%|▋         | 15701/225000 [41:58:45<337:20:38,  5.80s/it]
  7%|▋         | 15702/225000 [41:58:50<323:20:36,  5.56s/it]
  7%|▋         | 15703/225000 [41:58:55<310:05:10,  5.33s/it]
  7%|▋         | 15704/225000 [41:59:00<307:43:20,  5.29s/it]
  7%|▋         | 15705/225000 [41:59:09<373:53:31,  6.43s/it]
  7%|▋         | 15706/225000 [41:59:18<421:52:32,  7.26s/it]
  7%|▋         | 15707/225000 [41:59:24<395:35:03,  6.80s/it]
  7%|▋         | 15708/225000 [41:59:28<354:36:09,  6.10s/it]
  7%|▋         | 15709/225000 [41:59:39<425:22:16,  7.32s/it]
  7%|▋         | 15710/225000 [41:59:42<362:02:42,  6.23s/it]
  7%|▋         | 15711/225000 [41:59:52<429:44:53,  7.39s/it]
  7%|▋         | 15712/225000 [41:59:56<368:35:46,  6.34s/it]
  7%|▋         | 15713/225000 [42:00:08<468:50:25,  8.06s/it]
  7%|▋         | 15714/225000 [42:00:16<453:54:58,  7.81s/it]
  7%|▋         | 15715/225000 [42:00:22<428:36:14,  7.37s/it]
  7%|▋         | 15716/225000 [42:00:28<402:33:47,  6.92s/it]
  7%|▋         | 15717/225000 [42:00:33<367:13:10,  6.32s/it]
  7%|▋         | 15718/225000 [42:00:44<456:31:30,  7.85s/it]
  7%|▋         | 15719/225000 [42:00:53<473:28:27,  8.14s/it]
  7%|▋         | 15720/225000 [42:00:58<413:34:31,  7.11s/it]
  7%|▋         | 15721/225000 [42:01:06<436:47:56,  7.51s/it]
  7%|▋         | 15722/225000 [42:01:10<371:58:33,  6.40s/it]
  7%|▋         | 15723/225000 [42:01:20<440:46:16,  7.58s/it]
  7%|▋         | 15724/225000 [42:01:25<388:02:14,  6.68s/it]
  7%|▋         | 15725/225000 [42:01:29<352:31:38,  6.06s/it]
  7%|▋         | 15726/225000 [42:01:34<329:30:02,  5.67s/it]
  7%|▋         | 15727/225000 [42:01:39<309:42:44,  5.33s/it]
  7%|▋         | 15728/225000 [42:01:50<408:11:16,  7.02s/it]
  7%|▋         | 15729/225000 [42:01:56<387:39:12,  6.67s/it]
  7%|▋         | 15730/225000 [42:02:05<428:54:40,  7.38s/it]
  7%|▋         | 15731/225000 [42:02:08<359:15:26,  6.18s/it]
  7%|▋         | 15732/225000 [42:02:16<386:31:15,  6.65s/it]
  7%|▋         | 15733/225000 [42:02:21<355:23:50,  6.11s/it]
  7%|▋         | 15734/225000 [42:02:31<424:28:30,  7.30s/it]
  7%|▋         | 15735/225000 [42:02:35<369:14:35,  6.35s/it]
  7%|▋         | 15736/225000 [42:02:39<340:22:03,  5.86s/it]
  7%|▋         | 15737/225000 [42:02:44<316:55:24,  5.45s/it]
  7%|▋         | 15738/225000 [42:02:55<411:25:37,  7.08s/it]
  7%|▋         | 15739/225000 [42:02:59<365:11:42,  6.28s/it]
  7%|▋         | 15740/225000 [42:03:09<428:48:56,  7.38s/it]
  7%|▋         | 15741/225000 [42:03:17<427:38:28,  7.36s/it]
  7%|▋         | 15742/225000 [42:03:21<379:19:24,  6.53s/it]
  7%|▋         | 15743/225000 [42:03:30<423:27:04,  7.28s/it]
  7%|▋         | 15744/225000 [42:03:35<383:25:35,  6.60s/it]
  7%|▋         | 15745/225000 [42:03:43<404:07:29,  6.95s/it]
  7%|▋         | 15746/225000 [42:03:47<353:07:40,  6.08s/it]
  7%|▋         | 15747/225000 [42:03:52<340:58:33,  5.87s/it]
  7%|▋         | 15748/225000 [42:04:02<414:45:31,  7.14s/it]
  7%|▋         | 15749/225000 [42:04:10<419:46:58,  7.22s/it]
  7%|▋         | 15750/225000 [42:04:15<386:05:20,  6.64s/it]
                                                             
{'loss': '0.7349', 'grad_norm': '0.6076', 'learning_rate': '4.991e-05', 'epoch': '0.07', 'num_input_tokens_seen': 74648896, 'train_runtime': '1.515e+05', 'train_tokens_per_second': '492.9'}

  7%|▋         | 15750/225000 [42:04:15<386:05:20,  6.64s/it]
  7%|▋         | 15751/225000 [42:04:23<402:51:59,  6.93s/it]
  7%|▋         | 15752/225000 [42:04:31<421:01:03,  7.24s/it]
  7%|▋         | 15753/225000 [42:04:35<360:14:59,  6.20s/it]
  7%|▋         | 15754/225000 [42:04:45<426:38:31,  7.34s/it]
  7%|▋         | 15755/225000 [42:04:49<384:21:35,  6.61s/it]
  7%|▋         | 15756/225000 [42:04:58<420:19:18,  7.23s/it]
  7%|▋         | 15757/225000 [42:05:02<366:24:24,  6.30s/it]
  7%|▋         | 15758/225000 [42:05:11<409:40:56,  7.05s/it]
  7%|▋         | 15759/225000 [42:05:15<363:01:55,  6.25s/it]
  7%|▋         | 15760/225000 [42:05:20<329:25:33,  5.67s/it]
  7%|▋         | 15761/225000 [42:05:31<422:08:15,  7.26s/it]
  7%|▋         | 15762/225000 [42:05:35<377:22:50,  6.49s/it]
  7%|▋         | 15763/225000 [42:05:40<341:53:36,  5.88s/it]
  7%|▋         | 15764/225000 [42:05:50<424:38:10,  7.31s/it]
  7%|▋         | 15765/225000 [42:05:57<403:19:46,  6.94s/it]
  7%|▋         | 15766/225000 [42:06:01<355:40:00,  6.12s/it]
  7%|▋         | 15767/225000 [42:06:08<376:16:19,  6.47s/it]
  7%|▋         | 15768/225000 [42:06:13<341:58:11,  5.88s/it]
  7%|▋         | 15769/225000 [42:06:24<434:05:46,  7.47s/it]
  7%|▋         | 15770/225000 [42:06:27<365:51:56,  6.30s/it]
  7%|▋         | 15771/225000 [42:06:34<375:22:27,  6.46s/it]
  7%|▋         | 15772/225000 [42:06:39<341:06:07,  5.87s/it]
  7%|▋         | 15773/225000 [42:06:43<310:52:02,  5.35s/it]
  7%|▋         | 15774/225000 [42:06:54<413:23:17,  7.11s/it]
  7%|▋         | 15775/225000 [42:06:58<360:27:22,  6.20s/it]
  7%|▋         | 15776/225000 [42:07:08<418:32:51,  7.20s/it]
  7%|▋         | 15777/225000 [42:07:11<355:47:03,  6.12s/it]
  7%|▋         | 15778/225000 [42:07:21<420:56:39,  7.24s/it]
  7%|▋         | 15779/225000 [42:07:25<363:40:37,  6.26s/it]
  7%|▋         | 15780/225000 [42:07:31<353:28:24,  6.08s/it]
  7%|▋         | 15781/225000 [42:07:36<338:31:00,  5.82s/it]
  7%|▋         | 15782/225000 [42:07:49<459:58:48,  7.91s/it]
  7%|▋         | 15783/225000 [42:07:54<419:27:29,  7.22s/it]
  7%|▋         | 15784/225000 [42:08:00<393:56:09,  6.78s/it]
  7%|▋         | 15785/225000 [42:08:09<426:38:53,  7.34s/it]
  7%|▋         | 15786/225000 [42:08:15<404:05:17,  6.95s/it]
  7%|▋         | 15787/225000 [42:08:22<402:10:50,  6.92s/it]
  7%|▋         | 15788/225000 [42:08:26<351:16:01,  6.04s/it]
  7%|▋         | 15789/225000 [42:08:34<399:59:17,  6.88s/it]
  7%|▋         | 15790/225000 [42:08:40<374:34:18,  6.45s/it]
  7%|▋         | 15791/225000 [42:08:45<343:49:13,  5.92s/it]
  7%|▋         | 15792/225000 [42:08:49<317:27:16,  5.46s/it]
  7%|▋         | 15793/225000 [42:09:01<436:22:10,  7.51s/it]
  7%|▋         | 15794/225000 [42:09:05<373:35:03,  6.43s/it]
  7%|▋         | 15795/225000 [42:09:14<407:27:10,  7.01s/it]
  7%|▋         | 15796/225000 [42:09:19<380:07:36,  6.54s/it]
  7%|▋         | 15797/225000 [42:09:23<339:17:47,  5.84s/it]
  7%|▋         | 15798/225000 [42:09:28<326:59:25,  5.63s/it]
  7%|▋         | 15799/225000 [42:09:34<325:02:04,  5.59s/it]
  7%|▋         | 15800/225000 [42:09:39<320:48:57,  5.52s/it]
                                                             
{'loss': '0.726', 'grad_norm': '0.6591', 'learning_rate': '4.991e-05', 'epoch': '0.07022', 'num_input_tokens_seen': 74799944, 'train_runtime': '1.518e+05', 'train_tokens_per_second': '492.8'}

  7%|▋         | 15800/225000 [42:09:39<320:48:57,  5.52s/it]
  7%|▋         | 15801/225000 [42:09:53<460:16:17,  7.92s/it]
  7%|▋         | 15802/225000 [42:09:59<433:59:51,  7.47s/it]
  7%|▋         | 15803/225000 [42:10:03<367:30:34,  6.32s/it]
  7%|▋         | 15804/225000 [42:10:07<337:19:27,  5.80s/it]
  7%|▋         | 15805/225000 [42:10:15<374:10:01,  6.44s/it]
  7%|▋         | 15806/225000 [42:10:25<433:54:51,  7.47s/it]
  7%|▋         | 15807/225000 [42:10:30<380:46:47,  6.55s/it]
  7%|▋         | 15808/225000 [42:10:37<392:29:21,  6.75s/it]
  7%|▋         | 15809/225000 [42:10:44<397:53:33,  6.85s/it]
  7%|▋         | 15810/225000 [42:10:48<358:39:26,  6.17s/it]
  7%|▋         | 15811/225000 [42:10:54<340:23:13,  5.86s/it]
  7%|▋         | 15812/225000 [42:11:00<346:18:59,  5.96s/it]
  7%|▋         | 15813/225000 [42:11:09<396:39:47,  6.83s/it]
  7%|▋         | 15814/225000 [42:11:16<403:37:12,  6.95s/it]
  7%|▋         | 15815/225000 [42:11:23<411:05:37,  7.07s/it]
  7%|▋         | 15816/225000 [42:11:28<378:20:52,  6.51s/it]
  7%|▋         | 15817/225000 [42:11:35<386:08:51,  6.65s/it]
  7%|▋         | 15818/225000 [42:11:43<403:16:25,  6.94s/it]
  7%|▋         | 15819/225000 [42:11:47<358:26:30,  6.17s/it]
  7%|▋         | 15820/225000 [42:11:52<335:05:55,  5.77s/it]
  7%|▋         | 15821/225000 [42:12:00<367:44:56,  6.33s/it]
  7%|▋         | 15822/225000 [42:12:08<401:04:53,  6.90s/it]
  7%|▋         | 15823/225000 [42:12:14<380:02:54,  6.54s/it]
  7%|▋         | 15824/225000 [42:12:21<383:55:21,  6.61s/it]
  7%|▋         | 15825/225000 [42:12:25<353:48:27,  6.09s/it]
  7%|▋         | 15826/225000 [42:12:31<339:15:24,  5.84s/it]
  7%|▋         | 15827/225000 [42:12:39<381:45:41,  6.57s/it]
  7%|▋         | 15828/225000 [42:12:47<399:50:04,  6.88s/it]
  7%|▋         | 15829/225000 [42:12:53<393:58:39,  6.78s/it]
  7%|▋         | 15830/225000 [42:13:00<403:57:03,  6.95s/it]
  7%|▋         | 15831/225000 [42:13:05<367:38:24,  6.33s/it]
  7%|▋         | 15832/225000 [42:13:10<338:15:02,  5.82s/it]
  7%|▋         | 15833/225000 [42:13:21<428:20:40,  7.37s/it]
  7%|▋         | 15834/225000 [42:13:25<376:45:14,  6.48s/it]
  7%|▋         | 15835/225000 [42:13:33<400:52:36,  6.90s/it]
  7%|▋         | 15836/225000 [42:13:38<355:28:22,  6.12s/it]
  7%|▋         | 15837/225000 [42:13:43<342:23:38,  5.89s/it]
  7%|▋         | 15838/225000 [42:13:48<334:36:34,  5.76s/it]
  7%|▋         | 15839/225000 [42:13:54<326:10:58,  5.61s/it]
  7%|▋         | 15840/225000 [42:13:59<326:42:13,  5.62s/it]
  7%|▋         | 15841/225000 [42:14:12<452:41:23,  7.79s/it]
  7%|▋         | 15842/225000 [42:14:16<389:44:48,  6.71s/it]
  7%|▋         | 15843/225000 [42:14:21<348:44:49,  6.00s/it]
  7%|▋         | 15844/225000 [42:14:29<397:34:00,  6.84s/it]
  7%|▋         | 15845/225000 [42:14:37<417:25:14,  7.18s/it]
  7%|▋         | 15846/225000 [42:14:44<410:43:12,  7.07s/it]
  7%|▋         | 15847/225000 [42:14:49<368:17:12,  6.34s/it]
  7%|▋         | 15848/225000 [42:14:56<376:13:43,  6.48s/it]
  7%|▋         | 15849/225000 [42:15:01<353:38:46,  6.09s/it]
  7%|▋         | 15850/225000 [42:15:06<333:08:47,  5.73s/it]
                                                             
{'loss': '0.7244', 'grad_norm': '0.7153', 'learning_rate': '4.991e-05', 'epoch': '0.07044', 'num_input_tokens_seen': 74951992, 'train_runtime': '1.521e+05', 'train_tokens_per_second': '492.8'}

  7%|▋         | 15850/225000 [42:15:06<333:08:47,  5.73s/it]
  7%|▋         | 15851/225000 [42:15:11<323:25:25,  5.57s/it]
  7%|▋         | 15852/225000 [42:15:16<309:34:36,  5.33s/it]
  7%|▋         | 15853/225000 [42:15:22<321:00:50,  5.53s/it]
  7%|▋         | 15854/225000 [42:15:34<440:03:57,  7.57s/it]
  7%|▋         | 15855/225000 [42:15:41<423:29:09,  7.29s/it]
  7%|▋         | 15856/225000 [42:15:45<370:35:26,  6.38s/it]
  7%|▋         | 15857/225000 [42:15:51<362:10:28,  6.23s/it]
  7%|▋         | 15858/225000 [42:16:01<433:24:02,  7.46s/it]
  7%|▋         | 15859/225000 [42:16:06<392:36:57,  6.76s/it]
  7%|▋         | 15860/225000 [42:16:14<413:31:37,  7.12s/it]
  7%|▋         | 15861/225000 [42:16:20<388:42:00,  6.69s/it]
  7%|▋         | 15862/225000 [42:16:26<373:25:22,  6.43s/it]
  7%|▋         | 15863/225000 [42:16:35<423:56:41,  7.30s/it]
  7%|▋         | 15864/225000 [42:16:41<394:30:41,  6.79s/it]
  7%|▋         | 15865/225000 [42:16:47<388:22:46,  6.69s/it]
  7%|▋         | 15866/225000 [42:16:56<422:01:23,  7.26s/it]
  7%|▋         | 15867/225000 [42:17:02<405:14:33,  6.98s/it]
  7%|▋         | 15868/225000 [42:17:07<366:53:25,  6.32s/it]
  7%|▋         | 15869/225000 [42:17:12<352:10:53,  6.06s/it]
  7%|▋         | 15870/225000 [42:17:17<330:19:24,  5.69s/it]
  7%|▋         | 15871/225000 [42:17:30<455:18:32,  7.84s/it]
  7%|▋         | 15872/225000 [42:17:34<382:17:11,  6.58s/it]
  7%|▋         | 15873/225000 [42:17:43<423:55:39,  7.30s/it]
  7%|▋         | 15874/225000 [42:17:47<376:57:39,  6.49s/it]
  7%|▋         | 15875/225000 [42:17:52<346:32:25,  5.97s/it]
  7%|▋         | 15876/225000 [42:18:02<413:48:49,  7.12s/it]
  7%|▋         | 15877/225000 [42:18:08<397:49:08,  6.85s/it]
  7%|▋         | 15878/225000 [42:18:17<438:05:16,  7.54s/it]
  7%|▋         | 15879/225000 [42:18:21<382:50:46,  6.59s/it]
  7%|▋         | 15880/225000 [42:18:26<354:55:16,  6.11s/it]
  7%|▋         | 15881/225000 [42:18:31<331:48:18,  5.71s/it]
  7%|▋         | 15882/225000 [42:18:43<432:12:11,  7.44s/it]
  7%|▋         | 15883/225000 [42:18:48<394:12:48,  6.79s/it]
  7%|▋         | 15884/225000 [42:18:55<406:35:49,  7.00s/it]
  7%|▋         | 15885/225000 [42:19:02<402:17:59,  6.93s/it]
  7%|▋         | 15886/225000 [42:19:09<395:10:30,  6.80s/it]
  7%|▋         | 15887/225000 [42:19:13<348:47:07,  6.00s/it]
  7%|▋         | 15888/225000 [42:19:23<413:28:59,  7.12s/it]
  7%|▋         | 15889/225000 [42:19:28<380:08:39,  6.54s/it]
  7%|▋         | 15890/225000 [42:19:33<362:18:30,  6.24s/it]
  7%|▋         | 15891/225000 [42:19:38<338:27:50,  5.83s/it]
  7%|▋         | 15892/225000 [42:19:44<331:17:55,  5.70s/it]
  7%|▋         | 15893/225000 [42:19:50<335:00:33,  5.77s/it]
  7%|▋         | 15894/225000 [42:19:55<333:17:18,  5.74s/it]
  7%|▋         | 15895/225000 [42:20:09<481:07:58,  8.28s/it]
  7%|▋         | 15896/225000 [42:20:15<425:47:07,  7.33s/it]
  7%|▋         | 15897/225000 [42:20:21<411:22:04,  7.08s/it]
  7%|▋         | 15898/225000 [42:20:29<422:54:55,  7.28s/it]
  7%|▋         | 15899/225000 [42:20:33<372:40:21,  6.42s/it]
  7%|▋         | 15900/225000 [42:20:37<335:45:22,  5.78s/it]
                                                             
{'loss': '0.7342', 'grad_norm': '0.631', 'learning_rate': '4.991e-05', 'epoch': '0.07067', 'num_input_tokens_seen': 75106808, 'train_runtime': '1.524e+05', 'train_tokens_per_second': '492.7'}

  7%|▋         | 15900/225000 [42:20:37<335:45:22,  5.78s/it]
  7%|▋         | 15901/225000 [42:20:46<385:01:19,  6.63s/it]
  7%|▋         | 15902/225000 [42:20:56<439:50:42,  7.57s/it]
  7%|▋         | 15903/225000 [42:21:00<376:33:42,  6.48s/it]
  7%|▋         | 15904/225000 [42:21:05<350:27:57,  6.03s/it]
  7%|▋         | 15905/225000 [42:21:10<334:43:52,  5.76s/it]
  7%|▋         | 15906/225000 [42:21:15<319:49:29,  5.51s/it]
  7%|▋         | 15907/225000 [42:21:19<304:04:11,  5.24s/it]
  7%|▋         | 15908/225000 [42:21:31<419:16:26,  7.22s/it]
  7%|▋         | 15909/225000 [42:21:38<414:26:47,  7.14s/it]
  7%|▋         | 15910/225000 [42:21:46<417:48:52,  7.19s/it]
  7%|▋         | 15911/225000 [42:21:49<356:52:13,  6.14s/it]
  7%|▋         | 15912/225000 [42:21:56<359:02:31,  6.18s/it]
  7%|▋         | 15913/225000 [42:21:59<319:59:52,  5.51s/it]
  7%|▋         | 15914/225000 [42:22:14<468:59:26,  8.07s/it]
  7%|▋         | 15915/225000 [42:22:20<438:41:16,  7.55s/it]
  7%|▋         | 15916/225000 [42:22:29<472:20:58,  8.13s/it]
  7%|▋         | 15917/225000 [42:22:33<396:56:49,  6.83s/it]
  7%|▋         | 15918/225000 [42:22:42<425:56:51,  7.33s/it]
  7%|▋         | 15919/225000 [42:22:45<364:48:56,  6.28s/it]
  7%|▋         | 15920/225000 [42:22:50<329:53:57,  5.68s/it]
  7%|▋         | 15921/225000 [42:23:02<448:59:52,  7.73s/it]
  7%|▋         | 15922/225000 [42:23:08<411:44:43,  7.09s/it]
  7%|▋         | 15923/225000 [42:23:13<379:01:59,  6.53s/it]
  7%|▋         | 15924/225000 [42:23:17<334:04:56,  5.75s/it]
  7%|▋         | 15925/225000 [42:23:28<419:41:54,  7.23s/it]
  7%|▋         | 15926/225000 [42:23:33<383:33:18,  6.60s/it]
  7%|▋         | 15927/225000 [42:23:39<370:29:09,  6.38s/it]
  7%|▋         | 15928/225000 [42:23:44<355:05:47,  6.11s/it]
  7%|▋         | 15929/225000 [42:23:49<340:12:47,  5.86s/it]
  7%|▋         | 15930/225000 [42:23:55<328:12:19,  5.65s/it]
  7%|▋         | 15931/225000 [42:24:08<455:47:23,  7.85s/it]
  7%|▋         | 15932/225000 [42:24:14<431:36:07,  7.43s/it]
  7%|▋         | 15933/225000 [42:24:19<386:40:06,  6.66s/it]
  7%|▋         | 15934/225000 [42:24:28<427:32:25,  7.36s/it]
  7%|▋         | 15935/225000 [42:24:36<434:59:27,  7.49s/it]
  7%|▋         | 15936/225000 [42:24:41<404:31:19,  6.97s/it]
  7%|▋         | 15937/225000 [42:24:47<382:34:27,  6.59s/it]
  7%|▋         | 15938/225000 [42:24:55<404:03:37,  6.96s/it]
  7%|▋         | 15939/225000 [42:25:01<379:32:48,  6.54s/it]
  7%|▋         | 15940/225000 [42:25:07<385:02:46,  6.63s/it]
  7%|▋         | 15941/225000 [42:25:13<370:47:37,  6.39s/it]
  7%|▋         | 15942/225000 [42:25:22<416:46:48,  7.18s/it]
  7%|▋         | 15943/225000 [42:25:31<436:37:45,  7.52s/it]
  7%|▋         | 15944/225000 [42:25:34<370:16:27,  6.38s/it]
  7%|▋         | 15945/225000 [42:25:38<332:16:32,  5.72s/it]
  7%|▋         | 15946/225000 [42:25:49<420:30:59,  7.24s/it]
  7%|▋         | 15947/225000 [42:25:53<359:20:45,  6.19s/it]
  7%|▋         | 15948/225000 [42:25:58<342:42:26,  5.90s/it]
  7%|▋         | 15949/225000 [42:26:06<379:05:52,  6.53s/it]
  7%|▋         | 15950/225000 [42:26:14<406:51:30,  7.01s/it]
                                                             
{'loss': '0.7352', 'grad_norm': '0.6055', 'learning_rate': '4.991e-05', 'epoch': '0.07089', 'num_input_tokens_seen': 75261296, 'train_runtime': '1.528e+05', 'train_tokens_per_second': '492.6'}

  7%|▋         | 15950/225000 [42:26:14<406:51:30,  7.01s/it]
  7%|▋         | 15951/225000 [42:26:19<361:32:12,  6.23s/it]
  7%|▋         | 15952/225000 [42:26:27<397:09:40,  6.84s/it]
  7%|▋         | 15953/225000 [42:26:36<436:23:38,  7.52s/it]
  7%|▋         | 15954/225000 [42:26:45<457:50:50,  7.88s/it]
  7%|▋         | 15955/225000 [42:26:50<413:38:30,  7.12s/it]
  7%|▋         | 15956/225000 [42:26:55<377:35:28,  6.50s/it]
  7%|▋         | 15957/225000 [42:27:00<352:41:22,  6.07s/it]
  7%|▋         | 15958/225000 [42:27:05<335:44:42,  5.78s/it]
  7%|▋         | 15959/225000 [42:27:10<319:06:04,  5.50s/it]
  7%|▋         | 15960/225000 [42:27:22<429:50:23,  7.40s/it]
  7%|▋         | 15961/225000 [42:27:27<380:13:00,  6.55s/it]
  7%|▋         | 15962/225000 [42:27:32<356:45:14,  6.14s/it]
  7%|▋         | 15963/225000 [42:27:46<488:21:51,  8.41s/it]
  7%|▋         | 15964/225000 [42:27:49<405:52:52,  6.99s/it]
  7%|▋         | 15965/225000 [42:27:56<402:13:18,  6.93s/it]
  7%|▋         | 15966/225000 [42:28:02<390:00:27,  6.72s/it]
  7%|▋         | 15967/225000 [42:28:11<432:40:31,  7.45s/it]
  7%|▋         | 15968/225000 [42:28:18<415:07:42,  7.15s/it]
  7%|▋         | 15969/225000 [42:28:22<358:07:46,  6.17s/it]
  7%|▋         | 15970/225000 [42:28:29<375:41:42,  6.47s/it]
  7%|▋         | 15971/225000 [42:28:34<359:17:15,  6.19s/it]
  7%|▋         | 15972/225000 [42:28:40<342:04:43,  5.89s/it]
  7%|▋         | 15973/225000 [42:28:45<325:43:37,  5.61s/it]
  7%|▋         | 15974/225000 [42:28:49<313:38:33,  5.40s/it]
  7%|▋         | 15975/225000 [42:28:59<392:21:00,  6.76s/it]
  7%|▋         | 15976/225000 [42:29:09<440:25:49,  7.59s/it]
  7%|▋         | 15977/225000 [42:29:15<418:40:14,  7.21s/it]
  7%|▋         | 15978/225000 [42:29:20<377:23:01,  6.50s/it]
  7%|▋         | 15979/225000 [42:29:26<362:24:47,  6.24s/it]
  7%|▋         | 15980/225000 [42:29:31<346:43:54,  5.97s/it]
  7%|▋         | 15981/225000 [42:29:37<342:22:10,  5.90s/it]
  7%|▋         | 15982/225000 [42:29:42<329:39:34,  5.68s/it]
  7%|▋         | 15983/225000 [42:29:48<330:20:50,  5.69s/it]
  7%|▋         | 15984/225000 [42:29:53<322:52:48,  5.56s/it]
  7%|▋         | 15985/225000 [42:29:58<322:42:29,  5.56s/it]
  7%|▋         | 15986/225000 [42:30:14<498:38:17,  8.59s/it]
  7%|▋         | 15987/225000 [42:30:19<426:33:47,  7.35s/it]
  7%|▋         | 15988/225000 [42:30:24<392:44:32,  6.76s/it]
  7%|▋         | 15989/225000 [42:30:35<465:45:51,  8.02s/it]
  7%|▋         | 15990/225000 [42:30:40<408:09:46,  7.03s/it]
  7%|▋         | 15991/225000 [42:30:45<377:25:59,  6.50s/it]
  7%|▋         | 15992/225000 [42:30:53<399:12:19,  6.88s/it]
  7%|▋         | 15993/225000 [42:31:01<423:27:18,  7.29s/it]
  7%|▋         | 15994/225000 [42:31:05<362:17:59,  6.24s/it]
  7%|▋         | 15995/225000 [42:31:13<395:09:14,  6.81s/it]
  7%|▋         | 15996/225000 [42:31:18<360:12:02,  6.20s/it]
  7%|▋         | 15997/225000 [42:31:23<338:29:02,  5.83s/it]
  7%|▋         | 15998/225000 [42:31:28<323:57:08,  5.58s/it]
  7%|▋         | 15999/225000 [42:31:33<329:23:34,  5.67s/it]
  7%|▋         | 16000/225000 [42:31:39<325:48:38,  5.61s/it]
                                                             
{'loss': '0.7518', 'grad_norm': '0.6773', 'learning_rate': '4.99e-05', 'epoch': '0.07111', 'num_input_tokens_seen': 75413976, 'train_runtime': '1.531e+05', 'train_tokens_per_second': '492.6'}

  7%|▋         | 16000/225000 [42:31:39<325:48:38,  5.61s/it]
  7%|▋         | 16001/225000 [42:31:45<329:18:59,  5.67s/it]
  7%|▋         | 16002/225000 [42:31:59<473:06:03,  8.15s/it]
  7%|▋         | 16003/225000 [42:32:05<448:31:54,  7.73s/it]
  7%|▋         | 16004/225000 [42:32:10<397:29:51,  6.85s/it]
  7%|▋         | 16005/225000 [42:32:18<413:17:27,  7.12s/it]
  7%|▋         | 16006/225000 [42:32:26<434:05:47,  7.48s/it]
  7%|▋         | 16007/225000 [42:32:34<436:05:59,  7.51s/it]
  7%|▋         | 16008/225000 [42:32:40<416:05:23,  7.17s/it]
  7%|▋         | 16009/225000 [42:32:45<371:45:31,  6.40s/it]
  7%|▋         | 16010/225000 [42:32:53<408:42:36,  7.04s/it]
  7%|▋         | 16011/225000 [42:32:59<388:47:52,  6.70s/it]
  7%|▋         | 16012/225000 [42:33:07<398:09:27,  6.86s/it]
  7%|▋         | 16013/225000 [42:33:11<353:58:08,  6.10s/it]
  7%|▋         | 16014/225000 [42:33:15<325:18:26,  5.60s/it]
  7%|▋         | 16015/225000 [42:33:24<376:26:45,  6.48s/it]
  7%|▋         | 16016/225000 [42:33:33<419:53:49,  7.23s/it]
  7%|▋         | 16017/225000 [42:33:37<367:16:44,  6.33s/it]
  7%|▋         | 16018/225000 [42:33:46<414:58:19,  7.15s/it]
  7%|▋         | 16019/225000 [42:33:53<406:44:24,  7.01s/it]
  7%|▋         | 16020/225000 [42:34:00<406:03:23,  6.99s/it]
  7%|▋         | 16021/225000 [42:34:07<416:50:16,  7.18s/it]
  7%|▋         | 16022/225000 [42:34:11<363:46:24,  6.27s/it]
  7%|▋         | 16023/225000 [42:34:22<430:09:23,  7.41s/it]
  7%|▋         | 16024/225000 [42:34:27<397:38:21,  6.85s/it]
  7%|▋         | 16025/225000 [42:34:32<361:07:19,  6.22s/it]
  7%|▋         | 16026/225000 [42:34:40<393:46:59,  6.78s/it]
  7%|▋         | 16027/225000 [42:34:44<349:35:54,  6.02s/it]
  7%|▋         | 16028/225000 [42:34:53<400:53:06,  6.91s/it]
  7%|▋         | 16029/225000 [42:34:58<369:53:34,  6.37s/it]
  7%|▋         | 16030/225000 [42:35:04<349:42:58,  6.02s/it]
  7%|▋         | 16031/225000 [42:35:09<338:07:14,  5.82s/it]
  7%|▋         | 16032/225000 [42:35:14<325:43:09,  5.61s/it]
  7%|▋         | 16033/225000 [42:35:19<317:20:29,  5.47s/it]
  7%|▋         | 16034/225000 [42:35:26<349:02:00,  6.01s/it]
  7%|▋         | 16035/225000 [42:35:38<452:20:11,  7.79s/it]
  7%|▋         | 16036/225000 [42:35:44<410:05:53,  7.07s/it]
  7%|▋         | 16037/225000 [42:35:49<382:56:10,  6.60s/it]
  7%|▋         | 16038/225000 [42:35:54<356:53:09,  6.15s/it]
  7%|▋         | 16039/225000 [42:36:00<342:47:19,  5.91s/it]
  7%|▋         | 16040/225000 [42:36:05<324:43:39,  5.59s/it]
  7%|▋         | 16041/225000 [42:36:16<432:13:42,  7.45s/it]
  7%|▋         | 16042/225000 [42:36:23<410:29:35,  7.07s/it]
  7%|▋         | 16043/225000 [42:36:28<380:57:17,  6.56s/it]
  7%|▋         | 16044/225000 [42:36:33<350:12:24,  6.03s/it]
  7%|▋         | 16045/225000 [42:36:41<388:15:31,  6.69s/it]
  7%|▋         | 16046/225000 [42:36:46<361:29:52,  6.23s/it]
  7%|▋         | 16047/225000 [42:36:53<373:38:05,  6.44s/it]
  7%|▋         | 16048/225000 [42:37:03<443:11:25,  7.64s/it]
  7%|▋         | 16049/225000 [42:37:08<384:03:51,  6.62s/it]
  7%|▋         | 16050/225000 [42:37:15<396:03:14,  6.82s/it]
                                                             
{'loss': '0.7311', 'grad_norm': '0.6572', 'learning_rate': '4.99e-05', 'epoch': '0.07133', 'num_input_tokens_seen': 75568576, 'train_runtime': '1.534e+05', 'train_tokens_per_second': '492.5'}

  7%|▋         | 16050/225000 [42:37:15<396:03:14,  6.82s/it]
  7%|▋         | 16051/225000 [42:37:23<410:49:07,  7.08s/it]
  7%|▋         | 16052/225000 [42:37:29<396:50:26,  6.84s/it]
  7%|▋         | 16053/225000 [42:37:36<394:09:08,  6.79s/it]
  7%|▋         | 16054/225000 [42:37:43<404:21:01,  6.97s/it]
  7%|▋         | 16055/225000 [42:37:48<364:58:43,  6.29s/it]
  7%|▋         | 16056/225000 [42:37:58<432:38:36,  7.45s/it]
  7%|▋         | 16057/225000 [42:38:05<427:22:52,  7.36s/it]
  7%|▋         | 16058/225000 [42:38:11<398:29:40,  6.87s/it]
  7%|▋         | 16059/225000 [42:38:15<357:20:37,  6.16s/it]
  7%|▋         | 16060/225000 [42:38:20<328:31:04,  5.66s/it]
  7%|▋         | 16061/225000 [42:38:25<320:19:36,  5.52s/it]
  7%|▋         | 16062/225000 [42:38:34<391:22:18,  6.74s/it]
  7%|▋         | 16063/225000 [42:38:40<361:14:57,  6.22s/it]
  7%|▋         | 16064/225000 [42:38:46<374:11:06,  6.45s/it]
  7%|▋         | 16065/225000 [42:38:55<410:01:42,  7.06s/it]
  7%|▋         | 16066/225000 [42:38:59<353:00:09,  6.08s/it]
  7%|▋         | 16067/225000 [42:39:07<387:58:57,  6.69s/it]
  7%|▋         | 16068/225000 [42:39:13<381:17:20,  6.57s/it]
  7%|▋         | 16069/225000 [42:39:19<369:14:17,  6.36s/it]
  7%|▋         | 16070/225000 [42:39:24<341:25:25,  5.88s/it]
  7%|▋         | 16071/225000 [42:39:28<316:19:22,  5.45s/it]
  7%|▋         | 16072/225000 [42:39:39<412:41:10,  7.11s/it]
  7%|▋         | 16073/225000 [42:39:44<372:27:27,  6.42s/it]
  7%|▋         | 16074/225000 [42:39:54<431:11:59,  7.43s/it]
  7%|▋         | 16075/225000 [42:39:59<385:09:56,  6.64s/it]
  7%|▋         | 16076/225000 [42:40:03<347:18:52,  5.98s/it]
  7%|▋         | 16077/225000 [42:40:08<333:32:48,  5.75s/it]
  7%|▋         | 16078/225000 [42:40:13<322:17:18,  5.55s/it]
  7%|▋         | 16079/225000 [42:40:24<415:48:36,  7.16s/it]
  7%|▋         | 16080/225000 [42:40:29<379:37:28,  6.54s/it]
  7%|▋         | 16081/225000 [42:40:37<401:14:42,  6.91s/it]
  7%|▋         | 16082/225000 [42:40:41<350:11:48,  6.03s/it]
  7%|▋         | 16083/225000 [42:40:51<419:58:03,  7.24s/it]
  7%|▋         | 16084/225000 [42:40:58<419:43:01,  7.23s/it]
  7%|▋         | 16085/225000 [42:41:02<353:12:34,  6.09s/it]
  7%|▋         | 16086/225000 [42:41:07<334:24:23,  5.76s/it]
  7%|▋         | 16087/225000 [42:41:12<329:36:52,  5.68s/it]
  7%|▋         | 16088/225000 [42:41:18<334:54:42,  5.77s/it]
  7%|▋         | 16089/225000 [42:41:24<325:48:13,  5.61s/it]
  7%|▋         | 16090/225000 [42:41:29<325:03:43,  5.60s/it]
  7%|▋         | 16091/225000 [42:41:43<466:20:42,  8.04s/it]
  7%|▋         | 16092/225000 [42:41:48<410:41:13,  7.08s/it]
  7%|▋         | 16093/225000 [42:41:55<415:53:08,  7.17s/it]
  7%|▋         | 16094/225000 [42:42:00<369:24:33,  6.37s/it]
  7%|▋         | 16095/225000 [42:42:04<337:19:48,  5.81s/it]
  7%|▋         | 16096/225000 [42:42:09<323:18:10,  5.57s/it]
  7%|▋         | 16097/225000 [42:42:21<435:50:07,  7.51s/it]
  7%|▋         | 16098/225000 [42:42:25<366:37:58,  6.32s/it]
  7%|▋         | 16099/225000 [42:42:33<395:19:31,  6.81s/it]
  7%|▋         | 16100/225000 [42:42:37<358:33:50,  6.18s/it]
                                                             
{'loss': '0.7136', 'grad_norm': '0.6704', 'learning_rate': '4.99e-05', 'epoch': '0.07156', 'num_input_tokens_seen': 75718720, 'train_runtime': '1.538e+05', 'train_tokens_per_second': '492.5'}

  7%|▋         | 16100/225000 [42:42:37<358:33:50,  6.18s/it]
  7%|▋         | 16101/225000 [42:42:43<347:32:10,  5.99s/it]
  7%|▋         | 16102/225000 [42:42:48<328:24:41,  5.66s/it]
  7%|▋         | 16103/225000 [42:42:53<319:23:14,  5.50s/it]
  7%|▋         | 16104/225000 [42:43:09<509:38:33,  8.78s/it]
  7%|▋         | 16105/225000 [42:43:13<425:51:58,  7.34s/it]
  7%|▋         | 16106/225000 [42:43:18<373:09:40,  6.43s/it]
  7%|▋         | 16107/225000 [42:43:25<394:43:50,  6.80s/it]
  7%|▋         | 16108/225000 [42:43:34<432:49:53,  7.46s/it]
  7%|▋         | 16109/225000 [42:43:39<378:42:40,  6.53s/it]
  7%|▋         | 16110/225000 [42:43:46<389:46:39,  6.72s/it]
  7%|▋         | 16111/225000 [42:43:51<360:02:41,  6.21s/it]
  7%|▋         | 16112/225000 [42:44:01<427:10:23,  7.36s/it]
  7%|▋         | 16113/225000 [42:44:05<376:23:27,  6.49s/it]
  7%|▋         | 16114/225000 [42:44:10<353:49:46,  6.10s/it]
  7%|▋         | 16115/225000 [42:44:20<419:38:22,  7.23s/it]
  7%|▋         | 16116/225000 [42:44:24<358:09:42,  6.17s/it]
  7%|▋         | 16117/225000 [42:44:34<421:05:42,  7.26s/it]
  7%|▋         | 16118/225000 [42:44:38<366:13:15,  6.31s/it]
  7%|▋         | 16119/225000 [42:44:45<382:02:04,  6.58s/it]
  7%|▋         | 16120/225000 [42:44:52<387:10:52,  6.67s/it]
  7%|▋         | 16121/225000 [42:44:59<400:22:38,  6.90s/it]
  7%|▋         | 16122/225000 [42:45:04<353:24:47,  6.09s/it]
  7%|▋         | 16123/225000 [42:45:08<319:03:00,  5.50s/it]
  7%|▋         | 16124/225000 [42:45:13<310:43:09,  5.36s/it]
  7%|▋         | 16125/225000 [42:45:20<337:39:22,  5.82s/it]
  7%|▋         | 16126/225000 [42:45:30<415:04:36,  7.15s/it]
  7%|▋         | 16127/225000 [42:45:34<365:14:13,  6.29s/it]
  7%|▋         | 16128/225000 [42:45:39<333:49:31,  5.75s/it]
  7%|▋         | 16129/225000 [42:45:45<339:03:49,  5.84s/it]
  7%|▋         | 16130/225000 [42:45:54<391:53:22,  6.75s/it]
  7%|▋         | 16131/225000 [42:45:58<351:59:01,  6.07s/it]
  7%|▋         | 16132/225000 [42:46:03<330:07:29,  5.69s/it]
  7%|▋         | 16133/225000 [42:46:11<372:16:37,  6.42s/it]
  7%|▋         | 16134/225000 [42:46:21<435:47:28,  7.51s/it]
  7%|▋         | 16135/225000 [42:46:25<374:01:15,  6.45s/it]
  7%|▋         | 16136/225000 [42:46:36<445:59:59,  7.69s/it]
  7%|▋         | 16137/225000 [42:46:40<390:07:20,  6.72s/it]
  7%|▋         | 16138/225000 [42:46:45<357:54:03,  6.17s/it]
  7%|▋         | 16139/225000 [42:46:50<334:04:58,  5.76s/it]
  7%|▋         | 16140/225000 [42:47:02<451:41:21,  7.79s/it]
  7%|▋         | 16141/225000 [42:47:07<393:44:30,  6.79s/it]
  7%|▋         | 16142/225000 [42:47:15<409:41:27,  7.06s/it]
  7%|▋         | 16143/225000 [42:47:20<373:12:39,  6.43s/it]
  7%|▋         | 16144/225000 [42:47:25<355:58:08,  6.14s/it]
  7%|▋         | 16145/225000 [42:47:33<394:32:34,  6.80s/it]
  7%|▋         | 16146/225000 [42:47:38<357:26:21,  6.16s/it]
  7%|▋         | 16147/225000 [42:47:43<329:15:48,  5.68s/it]
  7%|▋         | 16148/225000 [42:47:53<405:39:48,  6.99s/it]
  7%|▋         | 16149/225000 [42:47:56<342:01:43,  5.90s/it]
  7%|▋         | 16150/225000 [42:48:05<389:46:49,  6.72s/it]
                                                             
{'loss': '0.7072', 'grad_norm': '0.6854', 'learning_rate': '4.99e-05', 'epoch': '0.07178', 'num_input_tokens_seen': 75871224, 'train_runtime': '1.541e+05', 'train_tokens_per_second': '492.4'}

  7%|▋         | 16150/225000 [42:48:05<389:46:49,  6.72s/it]
  7%|▋         | 16151/225000 [42:48:12<403:14:01,  6.95s/it]
  7%|▋         | 16152/225000 [42:48:17<366:56:21,  6.33s/it]
  7%|▋         | 16153/225000 [42:48:22<350:08:32,  6.04s/it]
  7%|▋         | 16154/225000 [42:48:28<339:04:20,  5.84s/it]
  7%|▋         | 16155/225000 [42:48:33<327:41:48,  5.65s/it]
  7%|▋         | 16156/225000 [42:48:38<314:33:34,  5.42s/it]
  7%|▋         | 16157/225000 [42:48:47<385:07:54,  6.64s/it]
  7%|▋         | 16158/225000 [42:49:01<506:44:36,  8.74s/it]
  7%|▋         | 16159/225000 [42:49:05<419:18:02,  7.23s/it]
  7%|▋         | 16160/225000 [42:49:11<402:20:09,  6.94s/it]
  7%|▋         | 16161/225000 [42:49:18<408:39:24,  7.04s/it]
  7%|▋         | 16162/225000 [42:49:28<453:03:22,  7.81s/it]
  7%|▋         | 16163/225000 [42:49:32<383:14:37,  6.61s/it]
  7%|▋         | 16164/225000 [42:49:40<421:18:33,  7.26s/it]
  7%|▋         | 16165/225000 [42:49:50<465:40:07,  8.03s/it]
  7%|▋         | 16166/225000 [42:49:56<421:36:55,  7.27s/it]
  7%|▋         | 16167/225000 [42:50:01<396:26:56,  6.83s/it]
  7%|▋         | 16168/225000 [42:50:08<385:41:29,  6.65s/it]
  7%|▋         | 16169/225000 [42:50:17<426:16:02,  7.35s/it]
  7%|▋         | 16170/225000 [42:50:21<373:05:12,  6.43s/it]
  7%|▋         | 16171/225000 [42:50:26<347:21:53,  5.99s/it]
  7%|▋         | 16172/225000 [42:50:31<325:20:02,  5.61s/it]
  7%|▋         | 16173/225000 [42:50:42<428:06:48,  7.38s/it]
  7%|▋         | 16174/225000 [42:50:49<427:10:10,  7.36s/it]
  7%|▋         | 16175/225000 [42:50:54<376:08:48,  6.48s/it]
  7%|▋         | 16176/225000 [42:50:58<342:06:11,  5.90s/it]
  7%|▋         | 16177/225000 [42:51:03<322:55:56,  5.57s/it]
  7%|▋         | 16178/225000 [42:51:10<348:03:31,  6.00s/it]
  7%|▋         | 16179/225000 [42:51:20<421:43:11,  7.27s/it]
  7%|▋         | 16180/225000 [42:51:26<393:13:26,  6.78s/it]
  7%|▋         | 16181/225000 [42:51:31<365:50:54,  6.31s/it]
  7%|▋         | 16182/225000 [42:51:36<344:58:45,  5.95s/it]
  7%|▋         | 16183/225000 [42:51:50<470:05:42,  8.10s/it]
  7%|▋         | 16184/225000 [42:51:57<465:34:31,  8.03s/it]
  7%|▋         | 16185/225000 [42:52:02<405:25:56,  6.99s/it]
  7%|▋         | 16186/225000 [42:52:07<365:55:38,  6.31s/it]
  7%|▋         | 16187/225000 [42:52:14<380:09:51,  6.55s/it]
  7%|▋         | 16188/225000 [42:52:23<421:00:23,  7.26s/it]
  7%|▋         | 16189/225000 [42:52:26<357:31:07,  6.16s/it]
  7%|▋         | 16190/225000 [42:52:32<353:07:48,  6.09s/it]
  7%|▋         | 16191/225000 [42:52:37<338:41:17,  5.84s/it]
  7%|▋         | 16192/225000 [42:52:43<328:16:45,  5.66s/it]
  7%|▋         | 16193/225000 [42:52:52<393:34:29,  6.79s/it]
  7%|▋         | 16194/225000 [42:53:03<467:22:42,  8.06s/it]
  7%|▋         | 16195/225000 [42:53:08<419:09:11,  7.23s/it]
  7%|▋         | 16196/225000 [42:53:14<395:38:22,  6.82s/it]
  7%|▋         | 16197/225000 [42:53:22<406:59:14,  7.02s/it]
  7%|▋         | 16198/225000 [42:53:26<354:20:27,  6.11s/it]
  7%|▋         | 16199/225000 [42:53:34<395:58:26,  6.83s/it]
  7%|▋         | 16200/225000 [42:53:40<375:13:26,  6.47s/it]
                                                             
{'loss': '0.7359', 'grad_norm': '0.6987', 'learning_rate': '4.99e-05', 'epoch': '0.072', 'num_input_tokens_seen': 76028776, 'train_runtime': '1.544e+05', 'train_tokens_per_second': '492.3'}

  7%|▋         | 16200/225000 [42:53:40<375:13:26,  6.47s/it]
  7%|▋         | 16201/225000 [42:53:47<391:13:22,  6.75s/it]
  7%|▋         | 16202/225000 [42:53:52<357:34:31,  6.17s/it]
  7%|▋         | 16203/225000 [42:53:59<378:33:04,  6.53s/it]
  7%|▋         | 16204/225000 [42:54:06<372:07:07,  6.42s/it]
  7%|▋         | 16205/225000 [42:54:10<333:51:48,  5.76s/it]
  7%|▋         | 16206/225000 [42:54:19<384:58:51,  6.64s/it]
  7%|▋         | 16207/225000 [42:54:25<383:02:18,  6.60s/it]
  7%|▋         | 16208/225000 [42:54:30<347:04:33,  5.98s/it]
  7%|▋         | 16209/225000 [42:54:37<376:19:07,  6.49s/it]
  7%|▋         | 16210/225000 [42:54:45<392:19:46,  6.76s/it]
  7%|▋         | 16211/225000 [42:54:50<371:15:47,  6.40s/it]
  7%|▋         | 16212/225000 [42:55:00<422:14:07,  7.28s/it]
  7%|▋         | 16213/225000 [42:55:03<359:41:09,  6.20s/it]
  7%|▋         | 16214/225000 [42:55:13<416:37:46,  7.18s/it]
  7%|▋         | 16215/225000 [42:55:19<407:42:13,  7.03s/it]
  7%|▋         | 16216/225000 [42:55:25<377:44:19,  6.51s/it]
  7%|▋         | 16217/225000 [42:55:30<354:46:35,  6.12s/it]
  7%|▋         | 16218/225000 [42:55:38<391:50:46,  6.76s/it]
  7%|▋         | 16219/225000 [42:55:47<422:19:20,  7.28s/it]
  7%|▋         | 16220/225000 [42:55:51<378:14:57,  6.52s/it]
  7%|▋         | 16221/225000 [42:56:00<419:52:30,  7.24s/it]
  7%|▋         | 16222/225000 [42:56:04<364:12:28,  6.28s/it]
  7%|▋         | 16223/225000 [42:56:09<333:19:07,  5.75s/it]
  7%|▋         | 16224/225000 [42:56:18<386:18:42,  6.66s/it]
  7%|▋         | 16225/225000 [42:56:24<373:38:50,  6.44s/it]
  7%|▋         | 16226/225000 [42:56:31<384:16:22,  6.63s/it]
  7%|▋         | 16227/225000 [42:56:35<351:45:18,  6.07s/it]
  7%|▋         | 16228/225000 [42:56:42<368:31:07,  6.35s/it]
  7%|▋         | 16229/225000 [42:56:52<426:15:55,  7.35s/it]
  7%|▋         | 16230/225000 [42:56:58<395:52:27,  6.83s/it]
  7%|▋         | 16231/225000 [42:57:03<375:57:29,  6.48s/it]
  7%|▋         | 16232/225000 [42:57:09<356:37:15,  6.15s/it]
  7%|▋         | 16233/225000 [42:57:14<346:23:26,  5.97s/it]
  7%|▋         | 16234/225000 [42:57:20<337:03:56,  5.81s/it]
  7%|▋         | 16235/225000 [42:57:25<333:59:05,  5.76s/it]
  7%|▋         | 16236/225000 [42:57:31<329:04:39,  5.67s/it]
  7%|▋         | 16237/225000 [42:57:45<476:52:09,  8.22s/it]
  7%|▋         | 16238/225000 [42:57:50<412:01:29,  7.11s/it]
  7%|▋         | 16239/225000 [42:58:00<468:34:10,  8.08s/it]
  7%|▋         | 16240/225000 [42:58:04<406:26:00,  7.01s/it]
  7%|▋         | 16241/225000 [42:58:12<423:42:30,  7.31s/it]
  7%|▋         | 16242/225000 [42:58:19<413:27:36,  7.13s/it]
  7%|▋         | 16243/225000 [42:58:24<380:36:40,  6.56s/it]
  7%|▋         | 16244/225000 [42:58:30<364:11:54,  6.28s/it]
  7%|▋         | 16245/225000 [42:58:35<347:39:21,  6.00s/it]
  7%|▋         | 16246/225000 [42:58:41<334:24:37,  5.77s/it]
  7%|▋         | 16247/225000 [42:58:46<321:22:13,  5.54s/it]
  7%|▋         | 16248/225000 [42:58:51<310:59:50,  5.36s/it]
  7%|▋         | 16249/225000 [42:58:59<361:52:34,  6.24s/it]
  7%|▋         | 16250/225000 [42:59:10<454:49:54,  7.84s/it]
                                                             
{'loss': '0.7211', 'grad_norm': '0.7478', 'learning_rate': '4.99e-05', 'epoch': '0.07222', 'num_input_tokens_seen': 76184008, 'train_runtime': '1.548e+05', 'train_tokens_per_second': '492.3'}

  7%|▋         | 16250/225000 [42:59:10<454:49:54,  7.84s/it]
  7%|▋         | 16251/225000 [42:59:16<411:52:24,  7.10s/it]
  7%|▋         | 16252/225000 [42:59:21<373:28:08,  6.44s/it]
  7%|▋         | 16253/225000 [42:59:26<348:13:36,  6.01s/it]
  7%|▋         | 16254/225000 [42:59:30<322:58:22,  5.57s/it]
  7%|▋         | 16255/225000 [42:59:35<315:05:36,  5.43s/it]
  7%|▋         | 16256/225000 [42:59:50<477:49:20,  8.24s/it]
  7%|▋         | 16257/225000 [42:59:56<444:26:32,  7.66s/it]
  7%|▋         | 16258/225000 [43:00:00<381:18:15,  6.58s/it]
  7%|▋         | 16259/225000 [43:00:08<389:38:37,  6.72s/it]
  7%|▋         | 16260/225000 [43:00:16<423:01:16,  7.30s/it]
  7%|▋         | 16261/225000 [43:00:20<360:19:44,  6.21s/it]
  7%|▋         | 16262/225000 [43:00:24<317:16:16,  5.47s/it]
  7%|▋         | 16263/225000 [43:00:33<393:12:40,  6.78s/it]
  7%|▋         | 16264/225000 [43:00:37<340:56:59,  5.88s/it]
  7%|▋         | 16265/225000 [43:00:48<425:47:28,  7.34s/it]
  7%|▋         | 16266/225000 [43:00:56<434:39:37,  7.50s/it]
  7%|▋         | 16267/225000 [43:01:03<424:57:04,  7.33s/it]
  7%|▋         | 16268/225000 [43:01:08<381:12:29,  6.57s/it]
  7%|▋         | 16269/225000 [43:01:13<354:34:37,  6.12s/it]
  7%|▋         | 16270/225000 [43:01:17<330:22:14,  5.70s/it]
  7%|▋         | 16271/225000 [43:01:22<310:15:28,  5.35s/it]
  7%|▋         | 16272/225000 [43:01:34<427:01:38,  7.37s/it]
  7%|▋         | 16273/225000 [43:01:38<372:45:40,  6.43s/it]
  7%|▋         | 16274/225000 [43:01:47<416:08:16,  7.18s/it]
  7%|▋         | 16275/225000 [43:01:54<406:14:35,  7.01s/it]
  7%|▋         | 16276/225000 [43:01:58<364:40:34,  6.29s/it]
  7%|▋         | 16277/225000 [43:02:03<328:43:36,  5.67s/it]
  7%|▋         | 16278/225000 [43:02:07<311:56:28,  5.38s/it]
  7%|▋         | 16279/225000 [43:02:16<370:06:06,  6.38s/it]
  7%|▋         | 16280/225000 [43:02:24<400:59:35,  6.92s/it]
  7%|▋         | 16281/225000 [43:02:30<373:49:03,  6.45s/it]
  7%|▋         | 16282/225000 [43:02:34<346:50:38,  5.98s/it]
  7%|▋         | 16283/225000 [43:02:40<337:53:29,  5.83s/it]
  7%|▋         | 16284/225000 [43:02:45<324:36:14,  5.60s/it]
  7%|▋         | 16285/225000 [43:02:58<456:40:11,  7.88s/it]
  7%|▋         | 16286/225000 [43:03:03<396:20:20,  6.84s/it]
  7%|▋         | 16287/225000 [43:03:07<360:56:03,  6.23s/it]
  7%|▋         | 16288/225000 [43:03:12<336:23:14,  5.80s/it]
  7%|▋         | 16289/225000 [43:03:23<419:08:00,  7.23s/it]
  7%|▋         | 16290/225000 [43:03:29<394:23:25,  6.80s/it]
  7%|▋         | 16291/225000 [43:03:33<359:14:02,  6.20s/it]
  7%|▋         | 16292/225000 [43:03:43<416:45:42,  7.19s/it]
  7%|▋         | 16293/225000 [43:03:49<406:48:05,  7.02s/it]
  7%|▋         | 16294/225000 [43:03:55<384:31:02,  6.63s/it]
  7%|▋         | 16295/225000 [43:03:59<338:46:50,  5.84s/it]
  7%|▋         | 16296/225000 [43:04:05<334:59:58,  5.78s/it]
  7%|▋         | 16297/225000 [43:04:18<456:04:14,  7.87s/it]
  7%|▋         | 16298/225000 [43:04:22<401:36:11,  6.93s/it]
  7%|▋         | 16299/225000 [43:04:27<359:14:51,  6.20s/it]
  7%|▋         | 16300/225000 [43:04:32<339:11:57,  5.85s/it]
                                                             
{'loss': '0.7272', 'grad_norm': '0.6957', 'learning_rate': '4.989e-05', 'epoch': '0.07244', 'num_input_tokens_seen': 76334848, 'train_runtime': '1.551e+05', 'train_tokens_per_second': '492.3'}

  7%|▋         | 16300/225000 [43:04:32<339:11:57,  5.85s/it]
  7%|▋         | 16301/225000 [43:04:37<334:25:47,  5.77s/it]
  7%|▋         | 16302/225000 [43:04:50<457:55:12,  7.90s/it]
  7%|▋         | 16303/225000 [43:04:54<391:56:43,  6.76s/it]
  7%|▋         | 16304/225000 [43:05:01<382:56:27,  6.61s/it]
  7%|▋         | 16305/225000 [43:05:06<368:49:01,  6.36s/it]
  7%|▋         | 16306/225000 [43:05:15<415:55:28,  7.17s/it]
  7%|▋         | 16307/225000 [43:05:21<386:22:11,  6.66s/it]
  7%|▋         | 16308/225000 [43:05:29<415:05:51,  7.16s/it]
  7%|▋         | 16309/225000 [43:05:36<415:42:15,  7.17s/it]
  7%|▋         | 16310/225000 [43:05:43<410:26:47,  7.08s/it]
  7%|▋         | 16311/225000 [43:05:48<365:58:20,  6.31s/it]
  7%|▋         | 16312/225000 [43:05:56<401:54:39,  6.93s/it]
  7%|▋         | 16313/225000 [43:06:04<409:23:25,  7.06s/it]
  7%|▋         | 16314/225000 [43:06:07<353:04:34,  6.09s/it]
  7%|▋         | 16315/225000 [43:06:15<383:40:39,  6.62s/it]
  7%|▋         | 16316/225000 [43:06:21<367:07:26,  6.33s/it]
  7%|▋         | 16317/225000 [43:06:29<396:21:28,  6.84s/it]
  7%|▋         | 16318/225000 [43:06:33<345:57:11,  5.97s/it]
  7%|▋         | 16319/225000 [43:06:37<315:03:54,  5.44s/it]
  7%|▋         | 16320/225000 [43:06:48<416:19:52,  7.18s/it]
  7%|▋         | 16321/225000 [43:06:52<360:37:31,  6.22s/it]
  7%|▋         | 16322/225000 [43:06:58<342:42:55,  5.91s/it]
  7%|▋         | 16323/225000 [43:07:08<427:07:22,  7.37s/it]
  7%|▋         | 16324/225000 [43:07:12<361:11:46,  6.23s/it]
  7%|▋         | 16325/225000 [43:07:20<396:32:48,  6.84s/it]
  7%|▋         | 16326/225000 [43:07:24<350:48:52,  6.05s/it]
  7%|▋         | 16327/225000 [43:07:35<433:23:09,  7.48s/it]
  7%|▋         | 16328/225000 [43:07:39<378:10:03,  6.52s/it]
  7%|▋         | 16329/225000 [43:07:48<416:02:33,  7.18s/it]
  7%|▋         | 16330/225000 [43:07:53<382:12:16,  6.59s/it]
  7%|▋         | 16331/225000 [43:07:58<354:50:42,  6.12s/it]
  7%|▋         | 16332/225000 [43:08:04<339:17:09,  5.85s/it]
  7%|▋         | 16333/225000 [43:08:15<440:46:03,  7.60s/it]
  7%|▋         | 16334/225000 [43:08:20<392:29:17,  6.77s/it]
  7%|▋         | 16335/225000 [43:08:25<366:29:02,  6.32s/it]
  7%|▋         | 16336/225000 [43:08:31<346:35:47,  5.98s/it]
  7%|▋         | 16337/225000 [43:08:36<333:35:19,  5.76s/it]
  7%|▋         | 16338/225000 [43:08:41<323:36:32,  5.58s/it]
  7%|▋         | 16339/225000 [43:08:47<322:26:23,  5.56s/it]
  7%|▋         | 16340/225000 [43:08:52<314:53:54,  5.43s/it]
  7%|▋         | 16341/225000 [43:09:01<375:51:12,  6.48s/it]
  7%|▋         | 16342/225000 [43:09:12<458:20:18,  7.91s/it]
  7%|▋         | 16343/225000 [43:09:16<386:10:55,  6.66s/it]
  7%|▋         | 16344/225000 [43:09:24<420:35:49,  7.26s/it]
  7%|▋         | 16345/225000 [43:09:29<373:26:13,  6.44s/it]
  7%|▋         | 16346/225000 [43:09:36<387:29:50,  6.69s/it]
  7%|▋         | 16347/225000 [43:09:42<379:50:09,  6.55s/it]
  7%|▋         | 16348/225000 [43:09:50<395:27:19,  6.82s/it]
  7%|▋         | 16349/225000 [43:09:56<394:48:47,  6.81s/it]
  7%|▋         | 16350/225000 [43:10:04<398:35:12,  6.88s/it]
                                                             
{'loss': '0.7251', 'grad_norm': '0.6487', 'learning_rate': '4.989e-05', 'epoch': '0.07267', 'num_input_tokens_seen': 76485624, 'train_runtime': '1.554e+05', 'train_tokens_per_second': '492.2'}

  7%|▋         | 16350/225000 [43:10:04<398:35:12,  6.88s/it]
  7%|▋         | 16351/225000 [43:10:08<358:49:24,  6.19s/it]
  7%|▋         | 16352/225000 [43:10:16<387:25:28,  6.68s/it]
  7%|▋         | 16353/225000 [43:10:23<398:15:46,  6.87s/it]
  7%|▋         | 16354/225000 [43:10:30<398:12:28,  6.87s/it]
  7%|▋         | 16355/225000 [43:10:35<358:21:50,  6.18s/it]
  7%|▋         | 16356/225000 [43:10:40<341:26:26,  5.89s/it]
  7%|▋         | 16357/225000 [43:10:46<340:36:58,  5.88s/it]
  7%|▋         | 16358/225000 [43:10:51<323:54:40,  5.59s/it]
  7%|▋         | 16359/225000 [43:11:00<389:19:39,  6.72s/it]
  7%|▋         | 16360/225000 [43:11:09<426:14:35,  7.35s/it]
  7%|▋         | 16361/225000 [43:11:13<366:04:36,  6.32s/it]
  7%|▋         | 16362/225000 [43:11:20<390:32:39,  6.74s/it]
  7%|▋         | 16363/225000 [43:11:27<379:59:14,  6.56s/it]
  7%|▋         | 16364/225000 [43:11:32<363:20:47,  6.27s/it]
  7%|▋         | 16365/225000 [43:11:38<348:47:03,  6.02s/it]
  7%|▋         | 16366/225000 [43:11:43<343:50:21,  5.93s/it]
  7%|▋         | 16367/225000 [43:11:50<349:54:02,  6.04s/it]
  7%|▋         | 16368/225000 [43:11:57<364:55:18,  6.30s/it]
  7%|▋         | 16369/225000 [43:12:04<389:29:53,  6.72s/it]
  7%|▋         | 16370/225000 [43:12:10<371:02:39,  6.40s/it]
  7%|▋         | 16371/225000 [43:12:16<358:32:02,  6.19s/it]
  7%|▋         | 16372/225000 [43:12:21<345:40:52,  5.96s/it]
  7%|▋         | 16373/225000 [43:12:34<472:17:49,  8.15s/it]
  7%|▋         | 16374/225000 [43:12:39<410:52:25,  7.09s/it]
  7%|▋         | 16375/225000 [43:12:44<367:47:23,  6.35s/it]
  7%|▋         | 16376/225000 [43:12:53<424:59:37,  7.33s/it]
  7%|▋         | 16377/225000 [43:13:01<427:21:09,  7.37s/it]
  7%|▋         | 16378/225000 [43:13:06<387:50:30,  6.69s/it]
  7%|▋         | 16379/225000 [43:13:11<363:54:41,  6.28s/it]
  7%|▋         | 16380/225000 [43:13:16<343:47:08,  5.93s/it]
  7%|▋         | 16381/225000 [43:13:21<328:43:08,  5.67s/it]
  7%|▋         | 16382/225000 [43:13:30<386:15:10,  6.67s/it]
  7%|▋         | 16383/225000 [43:13:41<465:48:16,  8.04s/it]
  7%|▋         | 16384/225000 [43:13:45<387:29:06,  6.69s/it]
  7%|▋         | 16385/225000 [43:13:52<397:24:35,  6.86s/it]
  7%|▋         | 16386/225000 [43:13:57<369:09:16,  6.37s/it]
  7%|▋         | 16387/225000 [43:14:02<332:25:24,  5.74s/it]
  7%|▋         | 16388/225000 [43:14:10<382:14:18,  6.60s/it]
  7%|▋         | 16389/225000 [43:14:17<389:21:38,  6.72s/it]
  7%|▋         | 16390/225000 [43:14:22<350:57:45,  6.06s/it]
  7%|▋         | 16391/225000 [43:14:27<336:00:21,  5.80s/it]
  7%|▋         | 16392/225000 [43:14:32<317:57:45,  5.49s/it]
  7%|▋         | 16393/225000 [43:14:43<423:05:26,  7.30s/it]
  7%|▋         | 16394/225000 [43:14:51<432:16:19,  7.46s/it]
  7%|▋         | 16395/225000 [43:14:56<383:30:48,  6.62s/it]
  7%|▋         | 16396/225000 [43:15:02<372:04:12,  6.42s/it]
  7%|▋         | 16397/225000 [43:15:06<341:22:57,  5.89s/it]
  7%|▋         | 16398/225000 [43:15:11<325:40:51,  5.62s/it]
  7%|▋         | 16399/225000 [43:15:24<446:04:00,  7.70s/it]
  7%|▋         | 16400/225000 [43:15:29<397:35:32,  6.86s/it]
                                                             
{'loss': '0.7323', 'grad_norm': '0.5738', 'learning_rate': '4.989e-05', 'epoch': '0.07289', 'num_input_tokens_seen': 76638144, 'train_runtime': '1.557e+05', 'train_tokens_per_second': '492.1'}

  7%|▋         | 16400/225000 [43:15:29<397:35:32,  6.86s/it]
  7%|▋         | 16401/225000 [43:15:34<370:25:45,  6.39s/it]
  7%|▋         | 16402/225000 [43:15:40<354:11:19,  6.11s/it]
  7%|▋         | 16403/225000 [43:15:45<345:31:22,  5.96s/it]
  7%|▋         | 16404/225000 [43:15:50<331:39:46,  5.72s/it]
  7%|▋         | 16405/225000 [43:15:58<360:54:33,  6.23s/it]
  7%|▋         | 16406/225000 [43:16:09<438:55:32,  7.58s/it]
  7%|▋         | 16407/225000 [43:16:12<372:26:56,  6.43s/it]
  7%|▋         | 16408/225000 [43:16:18<358:07:24,  6.18s/it]
  7%|▋         | 16409/225000 [43:16:23<337:35:20,  5.83s/it]
  7%|▋         | 16410/225000 [43:16:28<326:34:19,  5.64s/it]
  7%|▋         | 16411/225000 [43:16:33<316:55:11,  5.47s/it]
  7%|▋         | 16412/225000 [43:16:47<463:47:56,  8.00s/it]
  7%|▋         | 16413/225000 [43:16:52<418:06:11,  7.22s/it]
  7%|▋         | 16414/225000 [43:16:58<386:50:07,  6.68s/it]
  7%|▋         | 16415/225000 [43:17:03<359:02:55,  6.20s/it]
  7%|▋         | 16416/225000 [43:17:09<358:52:17,  6.19s/it]
  7%|▋         | 16417/225000 [43:17:14<342:32:34,  5.91s/it]
  7%|▋         | 16418/225000 [43:17:20<337:27:30,  5.82s/it]
  7%|▋         | 16419/225000 [43:17:33<459:44:05,  7.93s/it]
  7%|▋         | 16420/225000 [43:17:37<398:40:18,  6.88s/it]
  7%|▋         | 16421/225000 [43:17:42<368:13:29,  6.36s/it]
  7%|▋         | 16422/225000 [43:17:54<459:18:13,  7.93s/it]
  7%|▋         | 16423/225000 [43:18:00<420:13:13,  7.25s/it]
  7%|▋         | 16424/225000 [43:18:05<380:30:05,  6.57s/it]
  7%|▋         | 16425/225000 [43:18:10<356:59:41,  6.16s/it]
  7%|▋         | 16426/225000 [43:18:15<341:11:28,  5.89s/it]
  7%|▋         | 16427/225000 [43:18:20<330:56:38,  5.71s/it]
  7%|▋         | 16428/225000 [43:18:26<321:48:14,  5.55s/it]
  7%|▋         | 16429/225000 [43:18:39<455:31:43,  7.86s/it]
  7%|▋         | 16430/225000 [43:18:44<408:19:13,  7.05s/it]
  7%|▋         | 16431/225000 [43:18:52<419:37:52,  7.24s/it]
  7%|▋         | 16432/225000 [43:18:56<360:58:44,  6.23s/it]
  7%|▋         | 16433/225000 [43:19:07<452:05:44,  7.80s/it]
  7%|▋         | 16434/225000 [43:19:13<420:09:24,  7.25s/it]
  7%|▋         | 16435/225000 [43:19:19<394:42:17,  6.81s/it]
  7%|▋         | 16436/225000 [43:19:25<385:06:49,  6.65s/it]
  7%|▋         | 16437/225000 [43:19:32<387:23:42,  6.69s/it]
  7%|▋         | 16438/225000 [43:19:37<360:11:22,  6.22s/it]
  7%|▋         | 16439/225000 [43:19:42<344:11:36,  5.94s/it]
  7%|▋         | 16440/225000 [43:19:50<371:14:20,  6.41s/it]
  7%|▋         | 16441/225000 [43:19:58<396:44:31,  6.85s/it]
  7%|▋         | 16442/225000 [43:20:05<403:41:30,  6.97s/it]
  7%|▋         | 16443/225000 [43:20:11<386:27:07,  6.67s/it]
  7%|▋         | 16444/225000 [43:20:19<404:24:04,  6.98s/it]
  7%|▋         | 16445/225000 [43:20:23<363:15:15,  6.27s/it]
  7%|▋         | 16446/225000 [43:20:28<335:32:29,  5.79s/it]
  7%|▋         | 16447/225000 [43:20:33<326:04:11,  5.63s/it]
  7%|▋         | 16448/225000 [43:20:39<322:49:13,  5.57s/it]
  7%|▋         | 16449/225000 [43:20:51<441:45:16,  7.63s/it]
  7%|▋         | 16450/225000 [43:20:56<402:48:29,  6.95s/it]
                                                             
{'loss': '0.7141', 'grad_norm': '0.6515', 'learning_rate': '4.989e-05', 'epoch': '0.07311', 'num_input_tokens_seen': 76791208, 'train_runtime': '1.561e+05', 'train_tokens_per_second': '492.1'}

  7%|▋         | 16450/225000 [43:20:56<402:48:29,  6.95s/it]
  7%|▋         | 16451/225000 [43:21:02<381:46:22,  6.59s/it]
  7%|▋         | 16452/225000 [43:21:10<398:35:05,  6.88s/it]
  7%|▋         | 16453/225000 [43:21:18<431:43:22,  7.45s/it]
  7%|▋         | 16454/225000 [43:21:23<372:27:47,  6.43s/it]
  7%|▋         | 16455/225000 [43:21:35<477:38:15,  8.25s/it]
  7%|▋         | 16456/225000 [43:21:41<435:05:32,  7.51s/it]
  7%|▋         | 16457/225000 [43:21:45<371:08:23,  6.41s/it]
  7%|▋         | 16458/225000 [43:21:53<412:54:56,  7.13s/it]
  7%|▋         | 16459/225000 [43:21:58<365:03:08,  6.30s/it]
  7%|▋         | 16460/225000 [43:22:03<338:34:40,  5.84s/it]
  7%|▋         | 16461/225000 [43:22:08<325:30:51,  5.62s/it]
  7%|▋         | 16462/225000 [43:22:16<380:43:24,  6.57s/it]
  7%|▋         | 16463/225000 [43:22:26<438:01:04,  7.56s/it]
  7%|▋         | 16464/225000 [43:22:33<413:36:38,  7.14s/it]
  7%|▋         | 16465/225000 [43:22:41<437:30:18,  7.55s/it]
  7%|▋         | 16466/225000 [43:22:45<377:22:54,  6.51s/it]
  7%|▋         | 16467/225000 [43:22:52<392:28:29,  6.78s/it]
  7%|▋         | 16468/225000 [43:22:58<374:59:32,  6.47s/it]
  7%|▋         | 16469/225000 [43:23:03<347:19:36,  6.00s/it]
  7%|▋         | 16470/225000 [43:23:14<424:55:03,  7.34s/it]
  7%|▋         | 16471/225000 [43:23:19<382:35:26,  6.60s/it]
  7%|▋         | 16472/225000 [43:23:23<350:54:12,  6.06s/it]
  7%|▋         | 16473/225000 [43:23:29<340:21:13,  5.88s/it]
  7%|▋         | 16474/225000 [43:23:33<314:11:18,  5.42s/it]
  7%|▋         | 16475/225000 [43:23:46<437:13:24,  7.55s/it]
  7%|▋         | 16476/225000 [43:23:52<409:25:08,  7.07s/it]
  7%|▋         | 16477/225000 [43:24:00<431:21:00,  7.45s/it]
  7%|▋         | 16478/225000 [43:24:05<391:19:50,  6.76s/it]
  7%|▋         | 16479/225000 [43:24:10<353:28:13,  6.10s/it]
  7%|▋         | 16480/225000 [43:24:14<324:24:30,  5.60s/it]
  7%|▋         | 16481/225000 [43:24:23<377:24:43,  6.52s/it]
  7%|▋         | 16482/225000 [43:24:30<384:06:22,  6.63s/it]
  7%|▋         | 16483/225000 [43:24:35<364:03:22,  6.29s/it]
  7%|▋         | 16484/225000 [43:24:40<346:50:20,  5.99s/it]
  7%|▋         | 16485/225000 [43:24:45<328:18:43,  5.67s/it]
  7%|▋         | 16486/225000 [43:24:51<324:13:43,  5.60s/it]
  7%|▋         | 16487/225000 [43:25:03<447:20:57,  7.72s/it]
  7%|▋         | 16488/225000 [43:25:09<402:41:29,  6.95s/it]
  7%|▋         | 16489/225000 [43:25:13<350:51:11,  6.06s/it]
  7%|▋         | 16490/225000 [43:25:23<428:26:47,  7.40s/it]
  7%|▋         | 16491/225000 [43:25:30<421:11:28,  7.27s/it]
  7%|▋         | 16492/225000 [43:25:40<464:49:49,  8.03s/it]
  7%|▋         | 16493/225000 [43:25:46<438:40:21,  7.57s/it]
  7%|▋         | 16494/225000 [43:25:51<389:15:04,  6.72s/it]
  7%|▋         | 16495/225000 [43:26:01<450:10:58,  7.77s/it]
  7%|▋         | 16496/225000 [43:26:07<406:55:40,  7.03s/it]
  7%|▋         | 16497/225000 [43:26:11<362:52:35,  6.27s/it]
  7%|▋         | 16498/225000 [43:26:16<335:39:24,  5.80s/it]
  7%|▋         | 16499/225000 [43:26:25<402:08:00,  6.94s/it]
  7%|▋         | 16500/225000 [43:26:36<459:25:27,  7.93s/it]
                                                             
{'loss': '0.7169', 'grad_norm': '0.6409', 'learning_rate': '4.989e-05', 'epoch': '0.07333', 'num_input_tokens_seen': 76948416, 'train_runtime': '1.564e+05', 'train_tokens_per_second': '492'}

  7%|▋         | 16500/225000 [43:26:36<459:25:27,  7.93s/it]
  7%|▋         | 16501/225000 [43:26:41<421:42:16,  7.28s/it]
  7%|▋         | 16502/225000 [43:26:48<412:11:16,  7.12s/it]
  7%|▋         | 16503/225000 [43:26:53<373:38:43,  6.45s/it]
  7%|▋         | 16504/225000 [43:26:58<341:31:02,  5.90s/it]
  7%|▋         | 16505/225000 [43:27:06<379:21:37,  6.55s/it]
  7%|▋         | 16506/225000 [43:27:14<409:15:18,  7.07s/it]
  7%|▋         | 16507/225000 [43:27:19<371:24:25,  6.41s/it]
  7%|▋         | 16508/225000 [43:27:24<347:17:21,  6.00s/it]
  7%|▋         | 16509/225000 [43:27:29<329:13:27,  5.68s/it]
  7%|▋         | 16510/225000 [43:27:34<318:59:40,  5.51s/it]
  7%|▋         | 16511/225000 [43:27:39<310:16:38,  5.36s/it]
  7%|▋         | 16512/225000 [43:27:48<369:55:52,  6.39s/it]
  7%|▋         | 16513/225000 [43:28:00<469:07:49,  8.10s/it]
  7%|▋         | 16514/225000 [43:28:03<390:35:57,  6.74s/it]
  7%|▋         | 16515/225000 [43:28:08<345:54:31,  5.97s/it]
  7%|▋         | 16516/225000 [43:28:12<326:10:58,  5.63s/it]
  7%|▋         | 16517/225000 [43:28:21<381:00:17,  6.58s/it]
  7%|▋         | 16518/225000 [43:28:29<409:01:47,  7.06s/it]
  7%|▋         | 16519/225000 [43:28:37<413:28:10,  7.14s/it]
  7%|▋         | 16520/225000 [43:28:41<366:44:22,  6.33s/it]
  7%|▋         | 16521/225000 [43:28:47<361:21:36,  6.24s/it]
  7%|▋         | 16522/225000 [43:28:52<341:20:16,  5.89s/it]
  7%|▋         | 16523/225000 [43:28:58<340:13:56,  5.88s/it]
  7%|▋         | 16524/225000 [43:29:03<331:03:39,  5.72s/it]
  7%|▋         | 16525/225000 [43:29:11<368:22:26,  6.36s/it]
  7%|▋         | 16526/225000 [43:29:23<468:24:28,  8.09s/it]
  7%|▋         | 16527/225000 [43:29:27<395:55:12,  6.84s/it]
  7%|▋         | 16528/225000 [43:29:35<413:52:33,  7.15s/it]
  7%|▋         | 16529/225000 [43:29:40<367:00:11,  6.34s/it]
  7%|▋         | 16530/225000 [43:29:48<396:01:41,  6.84s/it]
  7%|▋         | 16531/225000 [43:29:53<367:46:30,  6.35s/it]
  7%|▋         | 16532/225000 [43:30:01<392:08:10,  6.77s/it]
  7%|▋         | 16533/225000 [43:30:06<361:52:59,  6.25s/it]
  7%|▋         | 16534/225000 [43:30:18<475:42:17,  8.21s/it]
  7%|▋         | 16535/225000 [43:30:23<413:12:13,  7.14s/it]
  7%|▋         | 16536/225000 [43:30:31<418:36:17,  7.23s/it]
  7%|▋         | 16537/225000 [43:30:35<371:58:08,  6.42s/it]
  7%|▋         | 16538/225000 [43:30:44<409:18:15,  7.07s/it]
  7%|▋         | 16539/225000 [43:30:51<417:20:10,  7.21s/it]
  7%|▋         | 16540/225000 [43:30:58<405:57:22,  7.01s/it]
  7%|▋         | 16541/225000 [43:31:05<415:42:51,  7.18s/it]
  7%|▋         | 16542/225000 [43:31:12<408:41:22,  7.06s/it]
  7%|▋         | 16543/225000 [43:31:22<465:28:13,  8.04s/it]
  7%|▋         | 16544/225000 [43:31:29<432:05:05,  7.46s/it]
  7%|▋         | 16545/225000 [43:31:34<399:26:25,  6.90s/it]
  7%|▋         | 16546/225000 [43:31:39<369:30:36,  6.38s/it]
  7%|▋         | 16547/225000 [43:31:44<337:54:23,  5.84s/it]
  7%|▋         | 16548/225000 [43:31:49<332:00:50,  5.73s/it]
  7%|▋         | 16549/225000 [43:31:55<326:18:17,  5.64s/it]
  7%|▋         | 16550/225000 [43:32:03<366:03:39,  6.32s/it]
                                                             
{'loss': '0.7314', 'grad_norm': '0.6925', 'learning_rate': '4.989e-05', 'epoch': '0.07356', 'num_input_tokens_seen': 77102728, 'train_runtime': '1.567e+05', 'train_tokens_per_second': '492'}

  7%|▋         | 16550/225000 [43:32:03<366:03:39,  6.32s/it]
  7%|▋         | 16551/225000 [43:32:12<416:12:55,  7.19s/it]
  7%|▋         | 16552/225000 [43:32:18<403:42:57,  6.97s/it]
  7%|▋         | 16553/225000 [43:32:23<360:38:32,  6.23s/it]
  7%|▋         | 16554/225000 [43:32:32<406:48:46,  7.03s/it]
  7%|▋         | 16555/225000 [43:32:37<373:30:01,  6.45s/it]
  7%|▋         | 16556/225000 [43:32:42<342:49:09,  5.92s/it]
  7%|▋         | 16557/225000 [43:32:47<328:21:38,  5.67s/it]
  7%|▋         | 16558/225000 [43:32:59<437:14:50,  7.55s/it]
  7%|▋         | 16559/225000 [43:33:04<395:36:55,  6.83s/it]
  7%|▋         | 16560/225000 [43:33:12<420:46:02,  7.27s/it]
  7%|▋         | 16561/225000 [43:33:19<424:00:38,  7.32s/it]
  7%|▋         | 16562/225000 [43:33:24<371:19:35,  6.41s/it]
  7%|▋         | 16563/225000 [43:33:32<410:50:50,  7.10s/it]
  7%|▋         | 16564/225000 [43:33:37<366:22:25,  6.33s/it]
  7%|▋         | 16565/225000 [43:33:45<398:14:38,  6.88s/it]
  7%|▋         | 16566/225000 [43:33:52<400:05:35,  6.91s/it]
  7%|▋         | 16567/225000 [43:34:04<485:53:39,  8.39s/it]
  7%|▋         | 16568/225000 [43:34:07<398:34:59,  6.88s/it]
  7%|▋         | 16569/225000 [43:34:14<396:53:43,  6.86s/it]
  7%|▋         | 16570/225000 [43:34:18<345:19:54,  5.96s/it]
  7%|▋         | 16571/225000 [43:34:28<410:10:26,  7.08s/it]
  7%|▋         | 16572/225000 [43:34:32<363:38:41,  6.28s/it]
  7%|▋         | 16573/225000 [43:34:41<402:58:09,  6.96s/it]
  7%|▋         | 16574/225000 [43:34:46<370:53:35,  6.41s/it]
  7%|▋         | 16575/225000 [43:34:50<341:20:46,  5.90s/it]
  7%|▋         | 16576/225000 [43:34:56<328:44:34,  5.68s/it]
  7%|▋         | 16577/225000 [43:35:08<443:56:08,  7.67s/it]
  7%|▋         | 16578/225000 [43:35:13<393:05:51,  6.79s/it]
  7%|▋         | 16579/225000 [43:35:17<349:08:05,  6.03s/it]
  7%|▋         | 16580/225000 [43:35:22<327:20:50,  5.65s/it]
  7%|▋         | 16581/225000 [43:35:28<345:50:14,  5.97s/it]
  7%|▋         | 16582/225000 [43:35:39<429:49:26,  7.42s/it]
  7%|▋         | 16583/225000 [43:35:43<361:43:59,  6.25s/it]
  7%|▋         | 16584/225000 [43:35:52<405:46:52,  7.01s/it]
  7%|▋         | 16585/225000 [43:35:58<396:57:39,  6.86s/it]
  7%|▋         | 16586/225000 [43:36:05<406:18:47,  7.02s/it]
  7%|▋         | 16587/225000 [43:36:11<376:36:43,  6.51s/it]
  7%|▋         | 16588/225000 [43:36:16<354:04:13,  6.12s/it]
  7%|▋         | 16589/225000 [43:36:21<332:38:57,  5.75s/it]
  7%|▋         | 16590/225000 [43:36:26<324:47:28,  5.61s/it]
  7%|▋         | 16591/225000 [43:36:37<422:34:33,  7.30s/it]
  7%|▋         | 16592/225000 [43:36:42<371:49:13,  6.42s/it]
  7%|▋         | 16593/225000 [43:36:46<336:14:11,  5.81s/it]
  7%|▋         | 16594/225000 [43:36:59<460:33:52,  7.96s/it]
  7%|▋         | 16595/225000 [43:37:03<388:00:54,  6.70s/it]
  7%|▋         | 16596/225000 [43:37:12<425:56:12,  7.36s/it]
  7%|▋         | 16597/225000 [43:37:15<361:11:38,  6.24s/it]
  7%|▋         | 16598/225000 [43:37:23<383:05:45,  6.62s/it]
  7%|▋         | 16599/225000 [43:37:29<371:55:21,  6.42s/it]
  7%|▋         | 16600/225000 [43:37:34<343:24:16,  5.93s/it]
                                                             
{'loss': '0.7044', 'grad_norm': '0.5943', 'learning_rate': '4.988e-05', 'epoch': '0.07378', 'num_input_tokens_seen': 77256656, 'train_runtime': '1.571e+05', 'train_tokens_per_second': '491.9'}

  7%|▋         | 16600/225000 [43:37:34<343:24:16,  5.93s/it]
  7%|▋         | 16601/225000 [43:37:38<321:29:50,  5.55s/it]
  7%|▋         | 16602/225000 [43:37:44<331:52:38,  5.73s/it]
  7%|▋         | 16603/225000 [43:37:56<440:09:20,  7.60s/it]
  7%|▋         | 16604/225000 [43:38:04<443:41:01,  7.66s/it]
  7%|▋         | 16605/225000 [43:38:09<393:49:20,  6.80s/it]
  7%|▋         | 16606/225000 [43:38:14<364:04:23,  6.29s/it]
  7%|▋         | 16607/225000 [43:38:24<418:02:40,  7.22s/it]
  7%|▋         | 16608/225000 [43:38:30<396:47:40,  6.85s/it]
  7%|▋         | 16609/225000 [43:38:36<391:15:31,  6.76s/it]
  7%|▋         | 16610/225000 [43:38:43<386:40:50,  6.68s/it]
  7%|▋         | 16611/225000 [43:38:48<359:55:28,  6.22s/it]
  7%|▋         | 16612/225000 [43:38:53<344:06:51,  5.94s/it]
  7%|▋         | 16613/225000 [43:38:59<339:17:15,  5.86s/it]
  7%|▋         | 16614/225000 [43:39:04<326:35:01,  5.64s/it]
  7%|▋         | 16615/225000 [43:39:09<318:03:28,  5.49s/it]
  7%|▋         | 16616/225000 [43:39:21<438:13:07,  7.57s/it]
  7%|▋         | 16617/225000 [43:39:26<387:09:01,  6.69s/it]
  7%|▋         | 16618/225000 [43:39:37<463:51:14,  8.01s/it]
  7%|▋         | 16619/225000 [43:39:43<419:47:35,  7.25s/it]
  7%|▋         | 16620/225000 [43:39:49<401:47:37,  6.94s/it]
  7%|▋         | 16621/225000 [43:39:57<429:20:00,  7.42s/it]
  7%|▋         | 16622/225000 [43:40:03<405:43:29,  7.01s/it]
  7%|▋         | 16623/225000 [43:40:09<383:37:55,  6.63s/it]
  7%|▋         | 16624/225000 [43:40:14<347:06:20,  6.00s/it]
  7%|▋         | 16625/225000 [43:40:23<400:28:48,  6.92s/it]
  7%|▋         | 16626/225000 [43:40:28<365:07:48,  6.31s/it]
  7%|▋         | 16627/225000 [43:40:39<461:54:45,  7.98s/it]
  7%|▋         | 16628/225000 [43:40:44<407:41:54,  7.04s/it]
  7%|▋         | 16629/225000 [43:40:49<366:02:07,  6.32s/it]
  7%|▋         | 16630/225000 [43:40:54<345:56:05,  5.98s/it]
  7%|▋         | 16631/225000 [43:41:05<430:33:25,  7.44s/it]
  7%|▋         | 16632/225000 [43:41:10<389:57:00,  6.74s/it]
  7%|▋         | 16633/225000 [43:41:15<361:48:31,  6.25s/it]
  7%|▋         | 16634/225000 [43:41:20<342:50:40,  5.92s/it]
  7%|▋         | 16635/225000 [43:41:26<329:03:31,  5.69s/it]
  7%|▋         | 16636/225000 [43:41:30<312:05:20,  5.39s/it]
  7%|▋         | 16637/225000 [43:41:43<440:02:48,  7.60s/it]
  7%|▋         | 16638/225000 [43:41:50<421:36:34,  7.28s/it]
  7%|▋         | 16639/225000 [43:41:57<421:56:34,  7.29s/it]
  7%|▋         | 16640/225000 [43:42:03<403:38:21,  6.97s/it]
  7%|▋         | 16641/225000 [43:42:08<368:48:13,  6.37s/it]
  7%|▋         | 16642/225000 [43:42:17<409:49:53,  7.08s/it]
  7%|▋         | 16643/225000 [43:42:24<413:07:50,  7.14s/it]
  7%|▋         | 16644/225000 [43:42:31<408:53:48,  7.06s/it]
  7%|▋         | 16645/225000 [43:42:38<413:21:23,  7.14s/it]
  7%|▋         | 16646/225000 [43:42:43<374:02:49,  6.46s/it]
  7%|▋         | 16647/225000 [43:42:51<403:22:42,  6.97s/it]
  7%|▋         | 16648/225000 [43:42:56<365:43:44,  6.32s/it]
  7%|▋         | 16649/225000 [43:43:03<374:21:33,  6.47s/it]
  7%|▋         | 16650/225000 [43:43:09<371:33:59,  6.42s/it]
                                                             
{'loss': '0.7225', 'grad_norm': '0.694', 'learning_rate': '4.988e-05', 'epoch': '0.074', 'num_input_tokens_seen': 77413416, 'train_runtime': '1.574e+05', 'train_tokens_per_second': '491.9'}

  7%|▋         | 16650/225000 [43:43:09<371:33:59,  6.42s/it]
  7%|▋         | 16651/225000 [43:43:20<445:07:40,  7.69s/it]
  7%|▋         | 16652/225000 [43:43:24<378:15:10,  6.54s/it]
  7%|▋         | 16653/225000 [43:43:32<414:40:54,  7.17s/it]
  7%|▋         | 16654/225000 [43:43:39<399:30:49,  6.90s/it]
  7%|▋         | 16655/225000 [43:43:46<404:18:07,  6.99s/it]
  7%|▋         | 16656/225000 [43:43:51<372:24:40,  6.43s/it]
  7%|▋         | 16657/225000 [43:43:58<380:02:04,  6.57s/it]
  7%|▋         | 16658/225000 [43:44:06<399:52:31,  6.91s/it]
  7%|▋         | 16659/225000 [43:44:10<357:21:09,  6.17s/it]
  7%|▋         | 16660/225000 [43:44:20<428:01:33,  7.40s/it]
  7%|▋         | 16661/225000 [43:44:26<403:45:29,  6.98s/it]
  7%|▋         | 16662/225000 [43:44:32<382:27:20,  6.61s/it]
  7%|▋         | 16663/225000 [43:44:38<371:39:06,  6.42s/it]
  7%|▋         | 16664/225000 [43:44:42<331:40:26,  5.73s/it]
  7%|▋         | 16665/225000 [43:44:53<426:48:40,  7.38s/it]
  7%|▋         | 16666/225000 [43:44:57<364:36:22,  6.30s/it]
  7%|▋         | 16667/225000 [43:45:03<352:11:11,  6.09s/it]
  7%|▋         | 16668/225000 [43:45:08<344:13:55,  5.95s/it]
  7%|▋         | 16669/225000 [43:45:14<344:26:21,  5.95s/it]
  7%|▋         | 16670/225000 [43:45:20<336:00:22,  5.81s/it]
  7%|▋         | 16671/225000 [43:45:25<332:40:10,  5.75s/it]
  7%|▋         | 16672/225000 [43:45:31<337:00:25,  5.82s/it]
  7%|▋         | 16673/225000 [43:45:39<367:30:29,  6.35s/it]
  7%|▋         | 16674/225000 [43:45:52<487:46:35,  8.43s/it]
  7%|▋         | 16675/225000 [43:45:57<427:41:02,  7.39s/it]
  7%|▋         | 16676/225000 [43:46:03<393:40:59,  6.80s/it]
  7%|▋         | 16677/225000 [43:46:08<370:33:00,  6.40s/it]
  7%|▋         | 16678/225000 [43:46:13<351:50:16,  6.08s/it]
  7%|▋         | 16679/225000 [43:46:19<342:05:14,  5.91s/it]
  7%|▋         | 16680/225000 [43:46:29<416:54:30,  7.20s/it]
  7%|▋         | 16681/225000 [43:46:37<431:35:16,  7.46s/it]
  7%|▋         | 16682/225000 [43:46:44<423:10:28,  7.31s/it]
  7%|▋         | 16683/225000 [43:46:52<440:24:53,  7.61s/it]
  7%|▋         | 16684/225000 [43:46:59<417:07:12,  7.21s/it]
  7%|▋         | 16685/225000 [43:47:03<371:01:17,  6.41s/it]
  7%|▋         | 16686/225000 [43:47:08<345:03:16,  5.96s/it]
  7%|▋         | 16687/225000 [43:47:18<413:52:25,  7.15s/it]
  7%|▋         | 16688/225000 [43:47:24<387:18:43,  6.69s/it]
  7%|▋         | 16689/225000 [43:47:32<410:11:12,  7.09s/it]
  7%|▋         | 16690/225000 [43:47:37<373:50:02,  6.46s/it]
  7%|▋         | 16691/225000 [43:47:44<394:31:48,  6.82s/it]
  7%|▋         | 16692/225000 [43:47:52<404:34:38,  6.99s/it]
  7%|▋         | 16693/225000 [43:47:56<361:46:10,  6.25s/it]
  7%|▋         | 16694/225000 [43:48:02<345:19:27,  5.97s/it]
  7%|▋         | 16695/225000 [43:48:07<326:14:52,  5.64s/it]
  7%|▋         | 16696/225000 [43:48:19<449:08:59,  7.76s/it]
  7%|▋         | 16697/225000 [43:48:24<396:22:05,  6.85s/it]
  7%|▋         | 16698/225000 [43:48:28<346:39:31,  5.99s/it]
  7%|▋         | 16699/225000 [43:48:34<345:52:16,  5.98s/it]
  7%|▋         | 16700/225000 [43:48:39<338:36:14,  5.85s/it]
                                                             
{'loss': '0.716', 'grad_norm': '0.7199', 'learning_rate': '4.988e-05', 'epoch': '0.07422', 'num_input_tokens_seen': 77569376, 'train_runtime': '1.577e+05', 'train_tokens_per_second': '491.8'}

  7%|▋         | 16700/225000 [43:48:39<338:36:14,  5.85s/it]
  7%|▋         | 16701/225000 [43:48:45<334:05:32,  5.77s/it]
  7%|▋         | 16702/225000 [43:48:59<470:34:19,  8.13s/it]
  7%|▋         | 16703/225000 [43:49:03<400:37:11,  6.92s/it]
  7%|▋         | 16704/225000 [43:49:08<375:01:19,  6.48s/it]
  7%|▋         | 16705/225000 [43:49:17<412:05:26,  7.12s/it]
  7%|▋         | 16706/225000 [43:49:21<367:28:53,  6.35s/it]
  7%|▋         | 16707/225000 [43:49:26<341:54:47,  5.91s/it]
  7%|▋         | 16708/225000 [43:49:37<425:37:42,  7.36s/it]
  7%|▋         | 16709/225000 [43:49:45<434:45:24,  7.51s/it]
  7%|▋         | 16710/225000 [43:49:52<423:20:56,  7.32s/it]
  7%|▋         | 16711/225000 [43:49:55<360:47:14,  6.24s/it]
  7%|▋         | 16712/225000 [43:50:03<376:26:40,  6.51s/it]
  7%|▋         | 16713/225000 [43:50:07<346:11:54,  5.98s/it]
  7%|▋         | 16714/225000 [43:50:13<336:23:18,  5.81s/it]
  7%|▋         | 16715/225000 [43:50:18<331:25:52,  5.73s/it]
  7%|▋         | 16716/225000 [43:50:24<326:24:26,  5.64s/it]
  7%|▋         | 16717/225000 [43:50:29<324:43:43,  5.61s/it]
  7%|▋         | 16718/225000 [43:50:41<431:59:41,  7.47s/it]
  7%|▋         | 16719/225000 [43:50:48<418:58:26,  7.24s/it]
  7%|▋         | 16720/225000 [43:50:56<430:01:12,  7.43s/it]
  7%|▋         | 16721/225000 [43:51:03<429:13:03,  7.42s/it]
  7%|▋         | 16722/225000 [43:51:07<364:10:52,  6.29s/it]
  7%|▋         | 16723/225000 [43:51:14<384:27:03,  6.65s/it]
  7%|▋         | 16724/225000 [43:51:21<390:54:58,  6.76s/it]
  7%|▋         | 16725/225000 [43:51:26<352:47:15,  6.10s/it]
  7%|▋         | 16726/225000 [43:51:30<319:58:37,  5.53s/it]
  7%|▋         | 16727/225000 [43:51:37<338:27:56,  5.85s/it]
  7%|▋         | 16728/225000 [43:51:47<410:31:21,  7.10s/it]
  7%|▋         | 16729/225000 [43:51:52<387:55:36,  6.71s/it]
  7%|▋         | 16730/225000 [43:51:59<385:27:41,  6.66s/it]
  7%|▋         | 16731/225000 [43:52:09<441:01:17,  7.62s/it]
  7%|▋         | 16732/225000 [43:52:13<374:18:24,  6.47s/it]
  7%|▋         | 16733/225000 [43:52:21<400:59:11,  6.93s/it]
  7%|▋         | 16734/225000 [43:52:29<419:12:21,  7.25s/it]
  7%|▋         | 16735/225000 [43:52:34<391:20:20,  6.76s/it]
  7%|▋         | 16736/225000 [43:52:40<367:28:11,  6.35s/it]
  7%|▋         | 16737/225000 [43:52:47<377:36:14,  6.53s/it]
  7%|▋         | 16738/225000 [43:52:53<378:15:32,  6.54s/it]
  7%|▋         | 16739/225000 [43:53:02<415:25:00,  7.18s/it]
  7%|▋         | 16740/225000 [43:53:07<375:24:17,  6.49s/it]
  7%|▋         | 16741/225000 [43:53:11<345:50:09,  5.98s/it]
  7%|▋         | 16742/225000 [43:53:16<319:01:58,  5.51s/it]
  7%|▋         | 16743/225000 [43:53:28<430:57:02,  7.45s/it]
  7%|▋         | 16744/225000 [43:53:32<372:31:30,  6.44s/it]
  7%|▋         | 16745/225000 [43:53:36<333:38:10,  5.77s/it]
  7%|▋         | 16746/225000 [43:53:42<336:26:10,  5.82s/it]
  7%|▋         | 16747/225000 [43:53:55<461:08:27,  7.97s/it]
  7%|▋         | 16748/225000 [43:54:00<401:58:26,  6.95s/it]
  7%|▋         | 16749/225000 [43:54:04<359:54:55,  6.22s/it]
  7%|▋         | 16750/225000 [43:54:09<336:17:59,  5.81s/it]
                                                             
{'loss': '0.7173', 'grad_norm': '0.6489', 'learning_rate': '4.988e-05', 'epoch': '0.07444', 'num_input_tokens_seen': 77721864, 'train_runtime': '1.58e+05', 'train_tokens_per_second': '491.8'}

  7%|▋         | 16750/225000 [43:54:09<336:17:59,  5.81s/it]
  7%|▋         | 16751/225000 [43:54:14<327:43:09,  5.67s/it]
  7%|▋         | 16752/225000 [43:54:28<459:42:04,  7.95s/it]
  7%|▋         | 16753/225000 [43:54:32<402:18:30,  6.95s/it]
  7%|▋         | 16754/225000 [43:54:38<387:21:51,  6.70s/it]
  7%|▋         | 16755/225000 [43:54:47<415:24:56,  7.18s/it]
  7%|▋         | 16756/225000 [43:54:54<410:05:50,  7.09s/it]
  7%|▋         | 16757/225000 [43:55:02<435:21:18,  7.53s/it]
  7%|▋         | 16758/225000 [43:55:09<421:21:59,  7.28s/it]
  7%|▋         | 16759/225000 [43:55:13<372:29:04,  6.44s/it]
  7%|▋         | 16760/225000 [43:55:19<367:50:49,  6.36s/it]
  7%|▋         | 16761/225000 [43:55:27<384:27:19,  6.65s/it]
  7%|▋         | 16762/225000 [43:55:35<403:34:09,  6.98s/it]
  7%|▋         | 16763/225000 [43:55:38<348:54:32,  6.03s/it]
  7%|▋         | 16764/225000 [43:55:47<393:38:17,  6.81s/it]
  7%|▋         | 16765/225000 [43:55:51<343:29:08,  5.94s/it]
  7%|▋         | 16766/225000 [43:55:56<333:21:33,  5.76s/it]
  7%|▋         | 16767/225000 [43:56:01<323:08:21,  5.59s/it]
  7%|▋         | 16768/225000 [43:56:12<412:27:12,  7.13s/it]
  7%|▋         | 16769/225000 [43:56:18<389:43:10,  6.74s/it]
  7%|▋         | 16770/225000 [43:56:24<373:11:02,  6.45s/it]
  7%|▋         | 16771/225000 [43:56:31<393:10:00,  6.80s/it]
  7%|▋         | 16772/225000 [43:56:39<405:06:58,  7.00s/it]
  7%|▋         | 16773/225000 [43:56:43<356:01:55,  6.16s/it]
  7%|▋         | 16774/225000 [43:56:53<420:04:55,  7.26s/it]
  7%|▋         | 16775/225000 [43:56:57<367:19:15,  6.35s/it]
  7%|▋         | 16776/225000 [43:57:01<330:26:47,  5.71s/it]
  7%|▋         | 16777/225000 [43:57:10<385:39:38,  6.67s/it]
  7%|▋         | 16778/225000 [43:57:15<355:55:58,  6.15s/it]
  7%|▋         | 16779/225000 [43:57:20<334:17:28,  5.78s/it]
  7%|▋         | 16780/225000 [43:57:29<387:43:38,  6.70s/it]
  7%|▋         | 16781/225000 [43:57:36<392:44:26,  6.79s/it]
  7%|▋         | 16782/225000 [43:57:42<376:21:47,  6.51s/it]
  7%|▋         | 16783/225000 [43:57:47<361:11:06,  6.24s/it]
  7%|▋         | 16784/225000 [43:57:53<353:39:44,  6.11s/it]
  7%|▋         | 16785/225000 [43:57:58<337:03:52,  5.83s/it]
  7%|▋         | 16786/225000 [43:58:04<330:17:44,  5.71s/it]
  7%|▋         | 16787/225000 [43:58:10<333:57:45,  5.77s/it]
  7%|▋         | 16788/225000 [43:58:20<416:39:15,  7.20s/it]
  7%|▋         | 16789/225000 [43:58:30<467:05:24,  8.08s/it]
  7%|▋         | 16790/225000 [43:58:35<411:56:35,  7.12s/it]
  7%|▋         | 16791/225000 [43:58:42<403:33:12,  6.98s/it]
  7%|▋         | 16792/225000 [43:58:47<367:29:51,  6.35s/it]
  7%|▋         | 16793/225000 [43:58:51<337:30:02,  5.84s/it]
  7%|▋         | 16794/225000 [43:58:56<321:39:07,  5.56s/it]
  7%|▋         | 16795/225000 [43:59:09<444:11:57,  7.68s/it]
  7%|▋         | 16796/225000 [43:59:13<386:45:23,  6.69s/it]
  7%|▋         | 16797/225000 [43:59:17<333:01:11,  5.76s/it]
  7%|▋         | 16798/225000 [43:59:28<425:42:26,  7.36s/it]
  7%|▋         | 16799/225000 [43:59:32<374:49:24,  6.48s/it]
  7%|▋         | 16800/225000 [43:59:40<392:22:22,  6.78s/it]
                                                             
{'loss': '0.7175', 'grad_norm': '0.783', 'learning_rate': '4.988e-05', 'epoch': '0.07467', 'num_input_tokens_seen': 77873296, 'train_runtime': '1.584e+05', 'train_tokens_per_second': '491.7'}

  7%|▋         | 16800/225000 [43:59:40<392:22:22,  6.78s/it]
  7%|▋         | 16801/225000 [43:59:44<349:20:36,  6.04s/it]
  7%|▋         | 16802/225000 [43:59:49<318:29:31,  5.51s/it]
  7%|▋         | 16803/225000 [44:00:01<443:25:14,  7.67s/it]
  7%|▋         | 16804/225000 [44:00:06<389:54:28,  6.74s/it]
  7%|▋         | 16805/225000 [44:00:11<371:15:23,  6.42s/it]
  7%|▋         | 16806/225000 [44:00:17<353:10:04,  6.11s/it]
  7%|▋         | 16807/225000 [44:00:22<331:26:14,  5.73s/it]
  7%|▋         | 16808/225000 [44:00:27<327:09:34,  5.66s/it]
  7%|▋         | 16809/225000 [44:00:32<320:47:04,  5.55s/it]
  7%|▋         | 16810/225000 [44:00:38<327:05:59,  5.66s/it]
  7%|▋         | 16811/225000 [44:00:44<321:04:34,  5.55s/it]
  7%|▋         | 16812/225000 [44:00:49<319:34:19,  5.53s/it]
  7%|▋         | 16813/225000 [44:01:03<456:07:07,  7.89s/it]
  7%|▋         | 16814/225000 [44:01:08<420:30:09,  7.27s/it]
  7%|▋         | 16815/225000 [44:01:17<439:00:20,  7.59s/it]
  7%|▋         | 16816/225000 [44:01:23<416:32:26,  7.20s/it]
  7%|▋         | 16817/225000 [44:01:28<379:41:00,  6.57s/it]
  7%|▋         | 16818/225000 [44:01:33<353:06:11,  6.11s/it]
  7%|▋         | 16819/225000 [44:01:38<334:23:11,  5.78s/it]
  7%|▋         | 16820/225000 [44:01:43<324:27:55,  5.61s/it]
  7%|▋         | 16821/225000 [44:01:50<343:25:43,  5.94s/it]
  7%|▋         | 16822/225000 [44:02:02<440:09:38,  7.61s/it]
  7%|▋         | 16823/225000 [44:02:06<380:02:08,  6.57s/it]
  7%|▋         | 16824/225000 [44:02:11<352:59:27,  6.10s/it]
  7%|▋         | 16825/225000 [44:02:22<436:49:01,  7.55s/it]
  7%|▋         | 16826/225000 [44:02:26<378:04:49,  6.54s/it]
  7%|▋         | 16827/225000 [44:02:34<412:56:51,  7.14s/it]
  7%|▋         | 16828/225000 [44:02:40<386:40:02,  6.69s/it]
  7%|▋         | 16829/225000 [44:02:48<416:29:19,  7.20s/it]
  7%|▋         | 16830/225000 [44:02:55<405:13:26,  7.01s/it]
  7%|▋         | 16831/225000 [44:03:01<389:03:06,  6.73s/it]
  7%|▋         | 16832/225000 [44:03:07<378:55:18,  6.55s/it]
  7%|▋         | 16833/225000 [44:03:12<352:27:09,  6.10s/it]
  7%|▋         | 16834/225000 [44:03:17<328:54:35,  5.69s/it]
  7%|▋         | 16835/225000 [44:03:22<315:13:03,  5.45s/it]
  7%|▋         | 16836/225000 [44:03:34<426:57:16,  7.38s/it]
  7%|▋         | 16837/225000 [44:03:38<379:01:22,  6.55s/it]
  7%|▋         | 16838/225000 [44:03:47<418:52:27,  7.24s/it]
  7%|▋         | 16839/225000 [44:03:52<384:18:08,  6.65s/it]
  7%|▋         | 16840/225000 [44:04:01<419:34:37,  7.26s/it]
  7%|▋         | 16841/225000 [44:04:09<422:55:33,  7.31s/it]
  7%|▋         | 16842/225000 [44:04:13<365:06:09,  6.31s/it]
  7%|▋         | 16843/225000 [44:04:17<325:45:20,  5.63s/it]
  7%|▋         | 16844/225000 [44:04:23<342:04:55,  5.92s/it]
  7%|▋         | 16845/225000 [44:04:33<408:40:15,  7.07s/it]
  7%|▋         | 16846/225000 [44:04:38<369:04:04,  6.38s/it]
  7%|▋         | 16847/225000 [44:04:43<349:44:52,  6.05s/it]
  7%|▋         | 16848/225000 [44:04:50<364:56:44,  6.31s/it]
  7%|▋         | 16849/225000 [44:04:59<408:02:15,  7.06s/it]
  7%|▋         | 16850/225000 [44:05:03<362:36:58,  6.27s/it]
                                                             
{'loss': '0.7268', 'grad_norm': '0.7402', 'learning_rate': '4.988e-05', 'epoch': '0.07489', 'num_input_tokens_seen': 78024496, 'train_runtime': '1.587e+05', 'train_tokens_per_second': '491.6'}

  7%|▋         | 16850/225000 [44:05:03<362:36:58,  6.27s/it]
  7%|▋         | 16851/225000 [44:05:09<354:58:33,  6.14s/it]
  7%|▋         | 16852/225000 [44:05:20<441:01:51,  7.63s/it]
  7%|▋         | 16853/225000 [44:05:23<366:31:52,  6.34s/it]
  7%|▋         | 16854/225000 [44:05:32<404:19:39,  6.99s/it]
  7%|▋         | 16855/225000 [44:05:38<389:34:02,  6.74s/it]
  7%|▋         | 16856/225000 [44:05:43<364:35:12,  6.31s/it]
  7%|▋         | 16857/225000 [44:05:52<400:37:35,  6.93s/it]
  7%|▋         | 16858/225000 [44:05:56<354:26:40,  6.13s/it]
  7%|▋         | 16859/225000 [44:06:07<439:58:21,  7.61s/it]
  7%|▋         | 16860/225000 [44:06:11<370:32:15,  6.41s/it]
  7%|▋         | 16861/225000 [44:06:15<328:12:00,  5.68s/it]
  7%|▋         | 16862/225000 [44:06:25<408:59:58,  7.07s/it]
  7%|▋         | 16863/225000 [44:06:29<351:59:50,  6.09s/it]
  7%|▋         | 16864/225000 [44:06:33<319:03:52,  5.52s/it]
  7%|▋         | 16865/225000 [44:06:38<311:02:40,  5.38s/it]
  7%|▋         | 16866/225000 [44:06:51<434:18:02,  7.51s/it]
  7%|▋         | 16867/225000 [44:06:55<374:31:40,  6.48s/it]
  7%|▋         | 16868/225000 [44:06:59<329:36:54,  5.70s/it]
  7%|▋         | 16869/225000 [44:07:11<451:43:09,  7.81s/it]
  7%|▋         | 16870/225000 [44:07:18<430:11:29,  7.44s/it]
  7%|▋         | 16871/225000 [44:07:25<420:10:43,  7.27s/it]
  7%|▋         | 16872/225000 [44:07:28<358:17:51,  6.20s/it]
  7%|▋         | 16873/225000 [44:07:36<377:38:15,  6.53s/it]
  7%|▋         | 16874/225000 [44:07:44<410:27:14,  7.10s/it]
  8%|▊         | 16875/225000 [44:07:50<389:31:15,  6.74s/it]
  8%|▊         | 16876/225000 [44:07:59<436:55:49,  7.56s/it]
  8%|▊         | 16877/225000 [44:08:08<446:28:21,  7.72s/it]
  8%|▊         | 16878/225000 [44:08:16<454:08:23,  7.86s/it]
  8%|▊         | 16879/225000 [44:08:20<396:33:39,  6.86s/it]
  8%|▊         | 16880/225000 [44:08:30<442:22:47,  7.65s/it]
  8%|▊         | 16881/225000 [44:08:35<404:02:19,  6.99s/it]
  8%|▊         | 16882/225000 [44:08:39<349:48:04,  6.05s/it]
  8%|▊         | 16883/225000 [44:08:47<382:13:11,  6.61s/it]
  8%|▊         | 16884/225000 [44:08:53<372:49:56,  6.45s/it]
  8%|▊         | 16885/225000 [44:09:01<395:53:11,  6.85s/it]
  8%|▊         | 16886/225000 [44:09:04<339:16:34,  5.87s/it]
  8%|▊         | 16887/225000 [44:09:09<309:58:20,  5.36s/it]
  8%|▊         | 16888/225000 [44:09:18<383:07:43,  6.63s/it]
  8%|▊         | 16889/225000 [44:09:23<345:03:23,  5.97s/it]
  8%|▊         | 16890/225000 [44:09:28<333:41:38,  5.77s/it]
  8%|▊         | 16891/225000 [44:09:33<325:55:26,  5.64s/it]
  8%|▊         | 16892/225000 [44:09:39<320:06:38,  5.54s/it]
  8%|▊         | 16893/225000 [44:09:44<314:24:35,  5.44s/it]
  8%|▊         | 16894/225000 [44:09:49<317:10:36,  5.49s/it]
  8%|▊         | 16895/225000 [44:09:58<378:11:07,  6.54s/it]
  8%|▊         | 16896/225000 [44:10:07<421:07:26,  7.29s/it]
  8%|▊         | 16897/225000 [44:10:16<450:10:43,  7.79s/it]
  8%|▊         | 16898/225000 [44:10:22<408:31:29,  7.07s/it]
  8%|▊         | 16899/225000 [44:10:29<405:26:32,  7.01s/it]
  8%|▊         | 16900/225000 [44:10:35<393:25:19,  6.81s/it]
                                                             
{'loss': '0.7524', 'grad_norm': '0.6051', 'learning_rate': '4.987e-05', 'epoch': '0.07511', 'num_input_tokens_seen': 78179600, 'train_runtime': '1.59e+05', 'train_tokens_per_second': '491.6'}

  8%|▊         | 16900/225000 [44:10:35<393:25:19,  6.81s/it]
  8%|▊         | 16901/225000 [44:10:41<381:46:33,  6.60s/it]
  8%|▊         | 16902/225000 [44:10:50<423:53:26,  7.33s/it]
  8%|▊         | 16903/225000 [44:10:54<364:18:21,  6.30s/it]
  8%|▊         | 16904/225000 [44:11:04<423:13:13,  7.32s/it]
  8%|▊         | 16905/225000 [44:11:10<400:28:48,  6.93s/it]
  8%|▊         | 16906/225000 [44:11:15<377:29:43,  6.53s/it]
  8%|▊         | 16907/225000 [44:11:20<337:44:22,  5.84s/it]
  8%|▊         | 16908/225000 [44:11:27<372:57:55,  6.45s/it]
  8%|▊         | 16909/225000 [44:11:32<346:29:25,  5.99s/it]
  8%|▊         | 16910/225000 [44:11:38<332:57:50,  5.76s/it]
  8%|▊         | 16911/225000 [44:11:42<311:54:46,  5.40s/it]
  8%|▊         | 16912/225000 [44:11:48<312:07:13,  5.40s/it]
  8%|▊         | 16913/225000 [44:11:59<411:34:04,  7.12s/it]
  8%|▊         | 16914/225000 [44:12:05<395:01:54,  6.83s/it]
  8%|▊         | 16915/225000 [44:12:09<352:32:33,  6.10s/it]
  8%|▊         | 16916/225000 [44:12:19<408:40:04,  7.07s/it]
  8%|▊         | 16917/225000 [44:12:24<371:56:16,  6.43s/it]
  8%|▊         | 16918/225000 [44:12:30<372:18:39,  6.44s/it]
  8%|▊         | 16919/225000 [44:12:37<380:31:58,  6.58s/it]
  8%|▊         | 16920/225000 [44:12:42<354:53:30,  6.14s/it]
  8%|▊         | 16921/225000 [44:12:48<344:36:07,  5.96s/it]
  8%|▊         | 16922/225000 [44:12:53<338:18:49,  5.85s/it]
  8%|▊         | 16923/225000 [44:12:59<332:24:37,  5.75s/it]
  8%|▊         | 16924/225000 [44:13:04<323:47:03,  5.60s/it]
  8%|▊         | 16925/225000 [44:13:09<322:36:30,  5.58s/it]
  8%|▊         | 16926/225000 [44:13:19<392:26:14,  6.79s/it]
  8%|▊         | 16927/225000 [44:13:27<406:33:45,  7.03s/it]
  8%|▊         | 16928/225000 [44:13:37<469:38:03,  8.13s/it]
  8%|▊         | 16929/225000 [44:13:41<397:58:59,  6.89s/it]
  8%|▊         | 16930/225000 [44:13:46<366:35:05,  6.34s/it]
  8%|▊         | 16931/225000 [44:13:56<426:17:25,  7.38s/it]
  8%|▊         | 16932/225000 [44:14:00<365:48:45,  6.33s/it]
  8%|▊         | 16933/225000 [44:14:08<399:08:42,  6.91s/it]
  8%|▊         | 16934/225000 [44:14:12<344:56:31,  5.97s/it]
  8%|▊         | 16935/225000 [44:14:22<405:37:31,  7.02s/it]
  8%|▊         | 16936/225000 [44:14:26<361:48:40,  6.26s/it]
  8%|▊         | 16937/225000 [44:14:31<346:58:33,  6.00s/it]
  8%|▊         | 16938/225000 [44:14:42<433:47:57,  7.51s/it]
  8%|▊         | 16939/225000 [44:14:51<446:51:07,  7.73s/it]
  8%|▊         | 16940/225000 [44:14:55<387:48:22,  6.71s/it]
  8%|▊         | 16941/225000 [44:15:00<353:41:39,  6.12s/it]
  8%|▊         | 16942/225000 [44:15:05<339:47:17,  5.88s/it]
  8%|▊         | 16943/225000 [44:15:12<362:48:03,  6.28s/it]
  8%|▊         | 16944/225000 [44:15:21<411:16:38,  7.12s/it]
  8%|▊         | 16945/225000 [44:15:29<417:14:32,  7.22s/it]
  8%|▊         | 16946/225000 [44:15:34<380:06:20,  6.58s/it]
  8%|▊         | 16947/225000 [44:15:41<385:34:00,  6.67s/it]
  8%|▊         | 16948/225000 [44:15:49<418:56:31,  7.25s/it]
  8%|▊         | 16949/225000 [44:15:54<372:07:56,  6.44s/it]
  8%|▊         | 16950/225000 [44:15:59<341:22:03,  5.91s/it]
                                                             
{'loss': '0.7123', 'grad_norm': '0.6229', 'learning_rate': '4.987e-05', 'epoch': '0.07533', 'num_input_tokens_seen': 78330784, 'train_runtime': '1.594e+05', 'train_tokens_per_second': '491.5'}

  8%|▊         | 16950/225000 [44:15:59<341:22:03,  5.91s/it]
  8%|▊         | 16951/225000 [44:16:06<373:37:26,  6.47s/it]
  8%|▊         | 16952/225000 [44:16:15<415:11:09,  7.18s/it]
  8%|▊         | 16953/225000 [44:16:19<362:05:39,  6.27s/it]
  8%|▊         | 16954/225000 [44:16:29<413:04:43,  7.15s/it]
  8%|▊         | 16955/225000 [44:16:33<359:35:11,  6.22s/it]
  8%|▊         | 16956/225000 [44:16:39<355:45:47,  6.16s/it]
  8%|▊         | 16957/225000 [44:16:48<416:40:38,  7.21s/it]
  8%|▊         | 16958/225000 [44:16:58<458:23:45,  7.93s/it]
  8%|▊         | 16959/225000 [44:17:02<383:24:19,  6.63s/it]
  8%|▊         | 16960/225000 [44:17:06<340:31:11,  5.89s/it]
  8%|▊         | 16961/225000 [44:17:17<432:56:46,  7.49s/it]
  8%|▊         | 16962/225000 [44:17:22<388:40:36,  6.73s/it]
  8%|▊         | 16963/225000 [44:17:27<358:39:08,  6.21s/it]
  8%|▊         | 16964/225000 [44:17:32<343:01:13,  5.94s/it]
  8%|▊         | 16965/225000 [44:17:37<331:56:43,  5.74s/it]
  8%|▊         | 16966/225000 [44:17:42<317:54:26,  5.50s/it]
  8%|▊         | 16967/225000 [44:17:51<370:49:39,  6.42s/it]
  8%|▊         | 16968/225000 [44:18:02<442:45:05,  7.66s/it]
  8%|▊         | 16969/225000 [44:18:06<390:03:05,  6.75s/it]
  8%|▊         | 16970/225000 [44:18:12<367:13:33,  6.35s/it]
  8%|▊         | 16971/225000 [44:18:16<330:12:31,  5.71s/it]
  8%|▊         | 16972/225000 [44:18:21<321:46:36,  5.57s/it]
  8%|▊         | 16973/225000 [44:18:29<359:52:29,  6.23s/it]
  8%|▊         | 16974/225000 [44:18:41<459:36:44,  7.95s/it]
  8%|▊         | 16975/225000 [44:18:44<384:57:03,  6.66s/it]
  8%|▊         | 16976/225000 [44:18:50<360:38:48,  6.24s/it]
  8%|▊         | 16977/225000 [44:18:55<348:24:37,  6.03s/it]
  8%|▊         | 16978/225000 [44:19:01<345:32:15,  5.98s/it]
  8%|▊         | 16979/225000 [44:19:07<337:08:00,  5.83s/it]
  8%|▊         | 16980/225000 [44:19:12<329:24:12,  5.70s/it]
  8%|▊         | 16981/225000 [44:19:18<330:30:51,  5.72s/it]
  8%|▊         | 16982/225000 [44:19:27<382:57:52,  6.63s/it]
  8%|▊         | 16983/225000 [44:19:35<418:32:35,  7.24s/it]
  8%|▊         | 16984/225000 [44:19:42<418:33:03,  7.24s/it]
  8%|▊         | 16985/225000 [44:19:48<386:07:30,  6.68s/it]
  8%|▊         | 16986/225000 [44:19:53<352:25:00,  6.10s/it]
  8%|▊         | 16987/225000 [44:19:57<330:53:16,  5.73s/it]
  8%|▊         | 16988/225000 [44:20:02<317:58:59,  5.50s/it]
  8%|▊         | 16989/225000 [44:20:11<379:48:14,  6.57s/it]
  8%|▊         | 16990/225000 [44:20:21<424:43:29,  7.35s/it]
  8%|▊         | 16991/225000 [44:20:27<415:57:52,  7.20s/it]
  8%|▊         | 16992/225000 [44:20:32<370:19:41,  6.41s/it]
  8%|▊         | 16993/225000 [44:20:36<331:23:32,  5.74s/it]
  8%|▊         | 16994/225000 [44:20:47<423:06:20,  7.32s/it]
  8%|▊         | 16995/225000 [44:20:53<398:27:14,  6.90s/it]
  8%|▊         | 16996/225000 [44:21:01<420:22:22,  7.28s/it]
  8%|▊         | 16997/225000 [44:21:06<367:29:18,  6.36s/it]
  8%|▊         | 16998/225000 [44:21:14<406:41:42,  7.04s/it]
  8%|▊         | 16999/225000 [44:21:18<356:18:48,  6.17s/it]
  8%|▊         | 17000/225000 [44:21:29<433:07:25,  7.50s/it]
                                                             
{'loss': '0.7123', 'grad_norm': '0.5726', 'learning_rate': '4.987e-05', 'epoch': '0.07556', 'num_input_tokens_seen': 78483392, 'train_runtime': '1.597e+05', 'train_tokens_per_second': '491.5'}

  8%|▊         | 17000/225000 [44:21:29<433:07:25,  7.50s/it]
  8%|▊         | 17001/225000 [44:21:34<400:04:02,  6.92s/it]
  8%|▊         | 17002/225000 [44:21:40<376:44:55,  6.52s/it]
  8%|▊         | 17003/225000 [44:21:45<358:06:13,  6.20s/it]
  8%|▊         | 17004/225000 [44:21:51<347:30:59,  6.01s/it]
  8%|▊         | 17005/225000 [44:21:57<338:25:23,  5.86s/it]
  8%|▊         | 17006/225000 [44:22:02<328:13:09,  5.68s/it]
  8%|▊         | 17007/225000 [44:22:07<325:27:42,  5.63s/it]
  8%|▊         | 17008/225000 [44:22:13<328:33:38,  5.69s/it]
  8%|▊         | 17009/225000 [44:22:19<333:02:23,  5.76s/it]
  8%|▊         | 17010/225000 [44:22:33<468:16:47,  8.11s/it]
  8%|▊         | 17011/225000 [44:22:38<417:17:50,  7.22s/it]
  8%|▊         | 17012/225000 [44:22:43<376:50:35,  6.52s/it]
  8%|▊         | 17013/225000 [44:22:48<348:04:22,  6.02s/it]
  8%|▊         | 17014/225000 [44:22:55<366:15:57,  6.34s/it]
  8%|▊         | 17015/225000 [44:23:05<427:37:51,  7.40s/it]
  8%|▊         | 17016/225000 [44:23:09<376:07:53,  6.51s/it]
  8%|▊         | 17017/225000 [44:23:16<378:33:48,  6.55s/it]
  8%|▊         | 17018/225000 [44:23:26<438:15:24,  7.59s/it]
  8%|▊         | 17019/225000 [44:23:32<409:21:23,  7.09s/it]
  8%|▊         | 17020/225000 [44:23:36<372:42:02,  6.45s/it]
  8%|▊         | 17021/225000 [44:23:42<360:09:20,  6.23s/it]
  8%|▊         | 17022/225000 [44:23:48<357:17:47,  6.18s/it]
  8%|▊         | 17023/225000 [44:23:57<409:31:05,  7.09s/it]
  8%|▊         | 17024/225000 [44:24:03<377:23:48,  6.53s/it]
  8%|▊         | 17025/225000 [44:24:08<356:46:13,  6.18s/it]
  8%|▊         | 17026/225000 [44:24:20<460:05:25,  7.96s/it]
  8%|▊         | 17027/225000 [44:24:27<446:12:56,  7.72s/it]
  8%|▊         | 17028/225000 [44:24:33<416:19:18,  7.21s/it]
  8%|▊         | 17029/225000 [44:24:38<374:17:56,  6.48s/it]
  8%|▊         | 17030/225000 [44:24:43<339:37:22,  5.88s/it]
  8%|▊         | 17031/225000 [44:24:48<327:37:54,  5.67s/it]
  8%|▊         | 17032/225000 [44:24:59<426:36:20,  7.38s/it]
  8%|▊         | 17033/225000 [44:25:06<416:09:07,  7.20s/it]
  8%|▊         | 17034/225000 [44:25:10<359:33:55,  6.22s/it]
  8%|▊         | 17035/225000 [44:25:15<334:43:08,  5.79s/it]
  8%|▊         | 17036/225000 [44:25:25<420:07:30,  7.27s/it]
  8%|▊         | 17037/225000 [44:25:30<376:30:48,  6.52s/it]
  8%|▊         | 17038/225000 [44:25:35<345:12:57,  5.98s/it]
  8%|▊         | 17039/225000 [44:25:40<332:05:25,  5.75s/it]
  8%|▊         | 17040/225000 [44:25:45<319:15:18,  5.53s/it]
  8%|▊         | 17041/225000 [44:25:56<414:10:46,  7.17s/it]
  8%|▊         | 17042/225000 [44:26:04<423:55:17,  7.34s/it]
  8%|▊         | 17043/225000 [44:26:09<380:16:25,  6.58s/it]
  8%|▊         | 17044/225000 [44:26:18<426:12:12,  7.38s/it]
  8%|▊         | 17045/225000 [44:26:22<374:26:48,  6.48s/it]
  8%|▊         | 17046/225000 [44:26:29<382:10:39,  6.62s/it]
  8%|▊         | 17047/225000 [44:26:37<395:34:41,  6.85s/it]
  8%|▊         | 17048/225000 [44:26:41<356:43:58,  6.18s/it]
  8%|▊         | 17049/225000 [44:26:47<342:02:09,  5.92s/it]
  8%|▊         | 17050/225000 [44:26:52<332:23:44,  5.75s/it]
                                                             
{'loss': '0.7242', 'grad_norm': '0.6513', 'learning_rate': '4.987e-05', 'epoch': '0.07578', 'num_input_tokens_seen': 78635792, 'train_runtime': '1.6e+05', 'train_tokens_per_second': '491.4'}

  8%|▊         | 17050/225000 [44:26:52<332:23:44,  5.75s/it]
  8%|▊         | 17051/225000 [44:26:57<327:50:20,  5.68s/it]
  8%|▊         | 17052/225000 [44:27:09<431:40:25,  7.47s/it]
  8%|▊         | 17053/225000 [44:27:15<398:32:18,  6.90s/it]
  8%|▊         | 17054/225000 [44:27:23<419:41:05,  7.27s/it]
  8%|▊         | 17055/225000 [44:27:27<372:07:57,  6.44s/it]
  8%|▊         | 17056/225000 [44:27:36<413:38:12,  7.16s/it]
  8%|▊         | 17057/225000 [44:27:41<373:17:20,  6.46s/it]
  8%|▊         | 17058/225000 [44:27:46<345:21:54,  5.98s/it]
  8%|▊         | 17059/225000 [44:27:51<325:01:45,  5.63s/it]
  8%|▊         | 17060/225000 [44:27:56<312:56:28,  5.42s/it]
  8%|▊         | 17061/225000 [44:28:09<450:09:27,  7.79s/it]
  8%|▊         | 17062/225000 [44:28:14<396:59:30,  6.87s/it]
  8%|▊         | 17063/225000 [44:28:23<437:31:28,  7.57s/it]
  8%|▊         | 17064/225000 [44:28:27<381:42:29,  6.61s/it]
  8%|▊         | 17065/225000 [44:28:35<408:08:31,  7.07s/it]
  8%|▊         | 17066/225000 [44:28:39<349:35:15,  6.05s/it]
  8%|▊         | 17067/225000 [44:28:44<330:37:53,  5.72s/it]
  8%|▊         | 17068/225000 [44:28:49<312:51:08,  5.42s/it]
  8%|▊         | 17069/225000 [44:29:02<444:03:44,  7.69s/it]
  8%|▊         | 17070/225000 [44:29:06<390:17:47,  6.76s/it]
  8%|▊         | 17071/225000 [44:29:14<402:12:23,  6.96s/it]
  8%|▊         | 17072/225000 [44:29:20<393:13:10,  6.81s/it]
  8%|▊         | 17073/225000 [44:29:25<361:03:52,  6.25s/it]
  8%|▊         | 17074/225000 [44:29:30<333:17:44,  5.77s/it]
  8%|▊         | 17075/225000 [44:29:35<329:30:25,  5.71s/it]
  8%|▊         | 17076/225000 [44:29:41<334:55:21,  5.80s/it]
  8%|▊         | 17077/225000 [44:29:49<359:53:30,  6.23s/it]
  8%|▊         | 17078/225000 [44:30:02<482:43:08,  8.36s/it]
  8%|▊         | 17079/225000 [44:30:06<409:39:11,  7.09s/it]
  8%|▊         | 17080/225000 [44:30:12<397:17:42,  6.88s/it]
  8%|▊         | 17081/225000 [44:30:16<347:19:24,  6.01s/it]
  8%|▊         | 17082/225000 [44:30:26<408:31:41,  7.07s/it]
  8%|▊         | 17083/225000 [44:30:31<373:12:13,  6.46s/it]
  8%|▊         | 17084/225000 [44:30:36<342:59:52,  5.94s/it]
  8%|▊         | 17085/225000 [44:30:40<315:53:49,  5.47s/it]
  8%|▊         | 17086/225000 [44:30:47<334:22:07,  5.79s/it]
  8%|▊         | 17087/225000 [44:30:59<455:12:10,  7.88s/it]
  8%|▊         | 17088/225000 [44:31:06<430:25:46,  7.45s/it]
  8%|▊         | 17089/225000 [44:31:14<446:37:31,  7.73s/it]
  8%|▊         | 17090/225000 [44:31:19<387:49:06,  6.72s/it]
  8%|▊         | 17091/225000 [44:31:28<443:39:03,  7.68s/it]
  8%|▊         | 17092/225000 [44:31:34<403:11:35,  6.98s/it]
  8%|▊         | 17093/225000 [44:31:41<413:31:52,  7.16s/it]
  8%|▊         | 17094/225000 [44:31:48<401:12:11,  6.95s/it]
  8%|▊         | 17095/225000 [44:31:54<395:58:11,  6.86s/it]
  8%|▊         | 17096/225000 [44:32:03<421:51:54,  7.30s/it]
  8%|▊         | 17097/225000 [44:32:10<420:09:51,  7.28s/it]
  8%|▊         | 17098/225000 [44:32:13<354:04:16,  6.13s/it]
  8%|▊         | 17099/225000 [44:32:19<351:23:25,  6.08s/it]
  8%|▊         | 17100/225000 [44:32:28<385:08:41,  6.67s/it]
                                                             
{'loss': '0.7445', 'grad_norm': '0.6745', 'learning_rate': '4.987e-05', 'epoch': '0.076', 'num_input_tokens_seen': 78788192, 'train_runtime': '1.603e+05', 'train_tokens_per_second': '491.4'}

  8%|▊         | 17100/225000 [44:32:28<385:08:41,  6.67s/it]
  8%|▊         | 17101/225000 [44:32:33<373:24:20,  6.47s/it]
  8%|▊         | 17102/225000 [44:32:41<398:20:17,  6.90s/it]
  8%|▊         | 17103/225000 [44:32:47<370:54:44,  6.42s/it]
  8%|▊         | 17104/225000 [44:32:52<350:06:27,  6.06s/it]
  8%|▊         | 17105/225000 [44:32:57<334:58:04,  5.80s/it]
  8%|▊         | 17106/225000 [44:33:02<319:39:16,  5.54s/it]
  8%|▊         | 17107/225000 [44:33:07<309:51:07,  5.37s/it]
  8%|▊         | 17108/225000 [44:33:20<443:01:41,  7.67s/it]
  8%|▊         | 17109/225000 [44:33:25<402:26:35,  6.97s/it]
  8%|▊         | 17110/225000 [44:33:34<423:16:59,  7.33s/it]
  8%|▊         | 17111/225000 [44:33:38<367:20:57,  6.36s/it]
  8%|▊         | 17112/225000 [44:33:46<398:44:02,  6.90s/it]
  8%|▊         | 17113/225000 [44:33:51<376:25:40,  6.52s/it]
  8%|▊         | 17114/225000 [44:33:59<394:43:46,  6.84s/it]
  8%|▊         | 17115/225000 [44:34:07<416:03:11,  7.20s/it]
  8%|▊         | 17116/225000 [44:34:13<393:26:49,  6.81s/it]
  8%|▊         | 17117/225000 [44:34:22<437:13:51,  7.57s/it]
  8%|▊         | 17118/225000 [44:34:27<378:33:51,  6.56s/it]
  8%|▊         | 17119/225000 [44:34:34<400:12:53,  6.93s/it]
  8%|▊         | 17120/225000 [44:34:40<372:36:26,  6.45s/it]
  8%|▊         | 17121/225000 [44:34:48<410:26:12,  7.11s/it]
  8%|▊         | 17122/225000 [44:34:55<403:01:33,  6.98s/it]
  8%|▊         | 17123/225000 [44:35:01<378:44:17,  6.56s/it]
  8%|▊         | 17124/225000 [44:35:06<352:54:20,  6.11s/it]
  8%|▊         | 17125/225000 [44:35:12<349:50:49,  6.06s/it]
  8%|▊         | 17126/225000 [44:35:17<340:22:42,  5.89s/it]
  8%|▊         | 17127/225000 [44:35:22<327:16:17,  5.67s/it]
  8%|▊         | 17128/225000 [44:35:28<322:30:47,  5.59s/it]
  8%|▊         | 17129/225000 [44:35:33<312:58:12,  5.42s/it]
  8%|▊         | 17130/225000 [44:35:47<461:29:47,  7.99s/it]
  8%|▊         | 17131/225000 [44:35:51<397:54:07,  6.89s/it]
  8%|▊         | 17132/225000 [44:36:00<430:24:17,  7.45s/it]
  8%|▊         | 17133/225000 [44:36:05<386:18:36,  6.69s/it]
  8%|▊         | 17134/225000 [44:36:10<369:48:13,  6.40s/it]
  8%|▊         | 17135/225000 [44:36:15<341:46:24,  5.92s/it]
  8%|▊         | 17136/225000 [44:36:24<399:31:34,  6.92s/it]
  8%|▊         | 17137/225000 [44:36:32<415:53:15,  7.20s/it]
  8%|▊         | 17138/225000 [44:36:37<374:50:00,  6.49s/it]
  8%|▊         | 17139/225000 [44:36:45<400:53:45,  6.94s/it]
  8%|▊         | 17140/225000 [44:36:53<417:04:35,  7.22s/it]
  8%|▊         | 17141/225000 [44:36:58<379:34:56,  6.57s/it]
  8%|▊         | 17142/225000 [44:37:03<350:34:15,  6.07s/it]
  8%|▊         | 17143/225000 [44:37:08<334:07:57,  5.79s/it]
  8%|▊         | 17144/225000 [44:37:14<328:05:01,  5.68s/it]
  8%|▊         | 17145/225000 [44:37:25<434:37:06,  7.53s/it]
  8%|▊         | 17146/225000 [44:37:32<414:50:07,  7.18s/it]
  8%|▊         | 17147/225000 [44:37:38<407:42:58,  7.06s/it]
  8%|▊         | 17148/225000 [44:37:43<364:23:13,  6.31s/it]
  8%|▊         | 17149/225000 [44:37:48<339:20:06,  5.88s/it]
  8%|▊         | 17150/225000 [44:37:57<389:01:17,  6.74s/it]
                                                             
{'loss': '0.7321', 'grad_norm': '0.6593', 'learning_rate': '4.986e-05', 'epoch': '0.07622', 'num_input_tokens_seen': 78940632, 'train_runtime': '1.607e+05', 'train_tokens_per_second': '491.3'}

  8%|▊         | 17150/225000 [44:37:57<389:01:17,  6.74s/it]
  8%|▊         | 17151/225000 [44:38:05<419:53:38,  7.27s/it]
  8%|▊         | 17152/225000 [44:38:10<378:12:57,  6.55s/it]
  8%|▊         | 17153/225000 [44:38:15<352:31:31,  6.11s/it]
  8%|▊         | 17154/225000 [44:38:26<434:43:08,  7.53s/it]
  8%|▊         | 17155/225000 [44:38:30<373:55:05,  6.48s/it]
  8%|▊         | 17156/225000 [44:38:36<369:22:51,  6.40s/it]
  8%|▊         | 17157/225000 [44:38:47<446:08:24,  7.73s/it]
  8%|▊         | 17158/225000 [44:38:51<380:38:35,  6.59s/it]
  8%|▊         | 17159/225000 [44:39:00<417:05:14,  7.22s/it]
  8%|▊         | 17160/225000 [44:39:07<421:41:16,  7.30s/it]
  8%|▊         | 17161/225000 [44:39:12<377:36:30,  6.54s/it]
  8%|▊         | 17162/225000 [44:39:17<353:05:58,  6.12s/it]
  8%|▊         | 17163/225000 [44:39:22<337:09:44,  5.84s/it]
  8%|▊         | 17164/225000 [44:39:35<455:15:34,  7.89s/it]
  8%|▊         | 17165/225000 [44:39:40<409:00:18,  7.08s/it]
  8%|▊         | 17166/225000 [44:39:45<372:00:25,  6.44s/it]
  8%|▊         | 17167/225000 [44:39:50<338:34:48,  5.86s/it]
  8%|▊         | 17168/225000 [44:40:02<458:19:44,  7.94s/it]
  8%|▊         | 17169/225000 [44:40:07<395:56:43,  6.86s/it]
  8%|▊         | 17170/225000 [44:40:15<415:00:00,  7.19s/it]
  8%|▊         | 17171/225000 [44:40:22<411:43:27,  7.13s/it]
  8%|▊         | 17172/225000 [44:40:29<415:22:11,  7.20s/it]
  8%|▊         | 17173/225000 [44:40:34<372:24:43,  6.45s/it]
  8%|▊         | 17174/225000 [44:40:41<395:02:46,  6.84s/it]
  8%|▊         | 17175/225000 [44:40:55<510:15:10,  8.84s/it]
  8%|▊         | 17176/225000 [44:41:02<470:31:03,  8.15s/it]
  8%|▊         | 17177/225000 [44:41:08<445:11:16,  7.71s/it]
  8%|▊         | 17178/225000 [44:41:14<420:29:55,  7.28s/it]
  8%|▊         | 17179/225000 [44:41:21<409:12:42,  7.09s/it]
  8%|▊         | 17180/225000 [44:41:29<421:14:02,  7.30s/it]
  8%|▊         | 17181/225000 [44:41:32<353:24:02,  6.12s/it]
  8%|▊         | 17182/225000 [44:41:39<364:37:53,  6.32s/it]
  8%|▊         | 17183/225000 [44:41:46<368:19:05,  6.38s/it]
  8%|▊         | 17184/225000 [44:41:55<413:02:33,  7.16s/it]
  8%|▊         | 17185/225000 [44:41:59<365:41:30,  6.33s/it]
  8%|▊         | 17186/225000 [44:42:04<338:30:38,  5.86s/it]
  8%|▊         | 17187/225000 [44:42:12<376:27:29,  6.52s/it]
  8%|▊         | 17188/225000 [44:42:25<496:18:29,  8.60s/it]
  8%|▊         | 17189/225000 [44:42:29<407:18:44,  7.06s/it]
  8%|▊         | 17190/225000 [44:42:35<393:30:13,  6.82s/it]
  8%|▊         | 17191/225000 [44:42:43<420:39:14,  7.29s/it]
  8%|▊         | 17192/225000 [44:42:48<380:27:14,  6.59s/it]
  8%|▊         | 17193/225000 [44:42:53<353:06:35,  6.12s/it]
  8%|▊         | 17194/225000 [44:42:58<333:29:36,  5.78s/it]
  8%|▊         | 17195/225000 [44:43:04<326:41:40,  5.66s/it]
  8%|▊         | 17196/225000 [44:43:13<390:37:40,  6.77s/it]
  8%|▊         | 17197/225000 [44:43:21<416:15:37,  7.21s/it]
  8%|▊         | 17198/225000 [44:43:28<408:38:45,  7.08s/it]
  8%|▊         | 17199/225000 [44:43:35<410:06:25,  7.10s/it]
  8%|▊         | 17200/225000 [44:43:40<374:04:58,  6.48s/it]
                                                             
{'loss': '0.7185', 'grad_norm': '0.5979', 'learning_rate': '4.986e-05', 'epoch': '0.07644', 'num_input_tokens_seen': 79095560, 'train_runtime': '1.61e+05', 'train_tokens_per_second': '491.2'}

  8%|▊         | 17200/225000 [44:43:40<374:04:58,  6.48s/it]
  8%|▊         | 17201/225000 [44:43:45<342:07:22,  5.93s/it]
  8%|▊         | 17202/225000 [44:43:52<366:07:17,  6.34s/it]
  8%|▊         | 17203/225000 [44:44:02<428:56:11,  7.43s/it]
  8%|▊         | 17204/225000 [44:44:09<413:13:13,  7.16s/it]
  8%|▊         | 17205/225000 [44:44:16<419:56:43,  7.28s/it]
  8%|▊         | 17206/225000 [44:44:20<363:33:29,  6.30s/it]
  8%|▊         | 17207/225000 [44:44:31<444:08:33,  7.69s/it]
  8%|▊         | 17208/225000 [44:44:35<375:55:19,  6.51s/it]
  8%|▊         | 17209/225000 [44:44:40<342:34:11,  5.94s/it]
  8%|▊         | 17210/225000 [44:44:45<338:37:07,  5.87s/it]
  8%|▊         | 17211/225000 [44:44:51<331:38:43,  5.75s/it]
  8%|▊         | 17212/225000 [44:44:57<333:52:27,  5.78s/it]
  8%|▊         | 17213/225000 [44:45:02<329:45:57,  5.71s/it]
  8%|▊         | 17214/225000 [44:45:08<331:45:20,  5.75s/it]
  8%|▊         | 17215/225000 [44:45:13<326:12:04,  5.65s/it]
  8%|▊         | 17216/225000 [44:45:26<443:31:13,  7.68s/it]
  8%|▊         | 17217/225000 [44:45:31<405:41:06,  7.03s/it]
  8%|▊         | 17218/225000 [44:45:44<496:18:02,  8.60s/it]
  8%|▊         | 17219/225000 [44:45:50<466:38:35,  8.09s/it]
  8%|▊         | 17220/225000 [44:45:55<399:06:18,  6.91s/it]
  8%|▊         | 17221/225000 [44:46:00<377:43:29,  6.54s/it]
  8%|▊         | 17222/225000 [44:46:10<427:27:47,  7.41s/it]
  8%|▊         | 17223/225000 [44:46:15<391:31:47,  6.78s/it]
  8%|▊         | 17224/225000 [44:46:19<346:06:18,  6.00s/it]
  8%|▊         | 17225/225000 [44:46:29<415:13:33,  7.19s/it]
  8%|▊         | 17226/225000 [44:46:35<396:24:18,  6.87s/it]
  8%|▊         | 17227/225000 [44:46:43<403:52:45,  7.00s/it]
  8%|▊         | 17228/225000 [44:46:49<399:31:47,  6.92s/it]
  8%|▊         | 17229/225000 [44:46:53<344:51:09,  5.98s/it]
  8%|▊         | 17230/225000 [44:47:02<400:23:07,  6.94s/it]
  8%|▊         | 17231/225000 [44:47:11<422:23:03,  7.32s/it]
  8%|▊         | 17232/225000 [44:47:14<363:40:48,  6.30s/it]
  8%|▊         | 17233/225000 [44:47:22<388:26:25,  6.73s/it]
  8%|▊         | 17234/225000 [44:47:28<380:27:42,  6.59s/it]
  8%|▊         | 17235/225000 [44:47:34<356:08:37,  6.17s/it]
  8%|▊         | 17236/225000 [44:47:40<364:16:43,  6.31s/it]
  8%|▊         | 17237/225000 [44:47:45<330:38:31,  5.73s/it]
  8%|▊         | 17238/225000 [44:47:54<396:45:01,  6.87s/it]
  8%|▊         | 17239/225000 [44:48:03<424:07:30,  7.35s/it]
  8%|▊         | 17240/225000 [44:48:08<388:09:25,  6.73s/it]
  8%|▊         | 17241/225000 [44:48:16<405:32:38,  7.03s/it]
  8%|▊         | 17242/225000 [44:48:20<365:08:48,  6.33s/it]
  8%|▊         | 17243/225000 [44:48:25<338:50:08,  5.87s/it]
  8%|▊         | 17244/225000 [44:48:31<343:00:54,  5.94s/it]
  8%|▊         | 17245/225000 [44:48:41<410:10:50,  7.11s/it]
  8%|▊         | 17246/225000 [44:48:46<366:34:42,  6.35s/it]
  8%|▊         | 17247/225000 [44:48:55<424:05:01,  7.35s/it]
  8%|▊         | 17248/225000 [44:49:00<384:35:57,  6.66s/it]
  8%|▊         | 17249/225000 [44:49:05<349:31:49,  6.06s/it]
  8%|▊         | 17250/225000 [44:49:11<339:23:14,  5.88s/it]
                                                             
{'loss': '0.7381', 'grad_norm': '0.6384', 'learning_rate': '4.986e-05', 'epoch': '0.07667', 'num_input_tokens_seen': 79248344, 'train_runtime': '1.614e+05', 'train_tokens_per_second': '491.2'}

  8%|▊         | 17250/225000 [44:49:11<339:23:14,  5.88s/it]
  8%|▊         | 17251/225000 [44:49:16<338:44:37,  5.87s/it]
  8%|▊         | 17252/225000 [44:49:22<333:20:51,  5.78s/it]
  8%|▊         | 17253/225000 [44:49:29<349:10:58,  6.05s/it]
  8%|▊         | 17254/225000 [44:49:42<474:26:20,  8.22s/it]
  8%|▊         | 17255/225000 [44:49:49<459:44:44,  7.97s/it]
  8%|▊         | 17256/225000 [44:49:54<398:12:01,  6.90s/it]
  8%|▊         | 17257/225000 [44:50:02<423:18:11,  7.34s/it]
  8%|▊         | 17258/225000 [44:50:08<403:06:44,  6.99s/it]
  8%|▊         | 17259/225000 [44:50:12<355:01:32,  6.15s/it]
  8%|▊         | 17260/225000 [44:50:18<338:33:09,  5.87s/it]
  8%|▊         | 17261/225000 [44:50:28<409:22:27,  7.09s/it]
  8%|▊         | 17262/225000 [44:50:36<427:29:35,  7.41s/it]
  8%|▊         | 17263/225000 [44:50:40<367:31:35,  6.37s/it]
  8%|▊         | 17264/225000 [44:50:49<423:41:06,  7.34s/it]
  8%|▊         | 17265/225000 [44:50:54<370:35:32,  6.42s/it]
  8%|▊         | 17266/225000 [44:51:02<402:22:06,  6.97s/it]
  8%|▊         | 17267/225000 [44:51:07<376:42:59,  6.53s/it]
  8%|▊         | 17268/225000 [44:51:19<465:30:59,  8.07s/it]
  8%|▊         | 17269/225000 [44:51:26<441:16:14,  7.65s/it]
  8%|▊         | 17270/225000 [44:51:34<453:58:03,  7.87s/it]
  8%|▊         | 17271/225000 [44:51:41<443:36:11,  7.69s/it]
  8%|▊         | 17272/225000 [44:51:46<400:07:28,  6.93s/it]
  8%|▊         | 17273/225000 [44:51:53<389:41:40,  6.75s/it]
  8%|▊         | 17274/225000 [44:51:57<349:10:13,  6.05s/it]
  8%|▊         | 17275/225000 [44:52:02<327:35:44,  5.68s/it]
  8%|▊         | 17276/225000 [44:52:14<445:00:46,  7.71s/it]
  8%|▊         | 17277/225000 [44:52:26<507:13:44,  8.79s/it]
  8%|▊         | 17278/225000 [44:52:30<435:23:01,  7.55s/it]
  8%|▊         | 17279/225000 [44:52:37<410:30:38,  7.11s/it]
  8%|▊         | 17280/225000 [44:52:44<411:51:36,  7.14s/it]
  8%|▊         | 17281/225000 [44:52:50<397:14:27,  6.88s/it]
  8%|▊         | 17282/225000 [44:52:54<344:51:27,  5.98s/it]
  8%|▊         | 17283/225000 [44:53:02<389:29:26,  6.75s/it]
  8%|▊         | 17284/225000 [44:53:09<390:28:37,  6.77s/it]
  8%|▊         | 17285/225000 [44:53:17<405:25:06,  7.03s/it]
  8%|▊         | 17286/225000 [44:53:22<374:23:44,  6.49s/it]
  8%|▊         | 17287/225000 [44:53:28<359:37:08,  6.23s/it]
  8%|▊         | 17288/225000 [44:53:33<338:51:27,  5.87s/it]
  8%|▊         | 17289/225000 [44:53:48<505:01:36,  8.75s/it]
  8%|▊         | 17290/225000 [44:53:52<418:58:27,  7.26s/it]
  8%|▊         | 17291/225000 [44:54:00<433:41:13,  7.52s/it]
  8%|▊         | 17292/225000 [44:54:06<405:19:26,  7.03s/it]
  8%|▊         | 17293/225000 [44:54:14<428:33:26,  7.43s/it]
  8%|▊         | 17294/225000 [44:54:20<391:17:35,  6.78s/it]
  8%|▊         | 17295/225000 [44:54:25<366:30:02,  6.35s/it]
  8%|▊         | 17296/225000 [44:54:31<351:54:35,  6.10s/it]
  8%|▊         | 17297/225000 [44:54:36<335:18:58,  5.81s/it]
  8%|▊         | 17298/225000 [44:54:41<328:40:20,  5.70s/it]
  8%|▊         | 17299/225000 [44:54:50<381:53:02,  6.62s/it]
  8%|▊         | 17300/225000 [44:55:02<470:13:37,  8.15s/it]
                                                             
{'loss': '0.7109', 'grad_norm': '0.6582', 'learning_rate': '4.986e-05', 'epoch': '0.07689', 'num_input_tokens_seen': 79404472, 'train_runtime': '1.617e+05', 'train_tokens_per_second': '491.1'}

  8%|▊         | 17300/225000 [44:55:02<470:13:37,  8.15s/it]
  8%|▊         | 17301/225000 [44:55:05<390:13:03,  6.76s/it]
  8%|▊         | 17302/225000 [44:55:11<367:20:15,  6.37s/it]
  8%|▊         | 17303/225000 [44:55:16<350:14:41,  6.07s/it]
  8%|▊         | 17304/225000 [44:55:21<330:50:43,  5.73s/it]
  8%|▊         | 17305/225000 [44:55:26<315:24:18,  5.47s/it]
  8%|▊         | 17306/225000 [44:55:37<423:52:30,  7.35s/it]
  8%|▊         | 17307/225000 [44:55:43<391:35:52,  6.79s/it]
  8%|▊         | 17308/225000 [44:55:48<367:39:57,  6.37s/it]
  8%|▊         | 17309/225000 [44:55:55<377:22:06,  6.54s/it]
  8%|▊         | 17310/225000 [44:56:09<494:03:32,  8.56s/it]
  8%|▊         | 17311/225000 [44:56:12<404:22:38,  7.01s/it]
  8%|▊         | 17312/225000 [44:56:19<409:30:43,  7.10s/it]
  8%|▊         | 17313/225000 [44:56:24<363:52:07,  6.31s/it]
  8%|▊         | 17314/225000 [44:56:31<376:56:18,  6.53s/it]
  8%|▊         | 17315/225000 [44:56:41<437:08:06,  7.58s/it]
  8%|▊         | 17316/225000 [44:56:47<416:24:35,  7.22s/it]
  8%|▊         | 17317/225000 [44:56:52<380:33:40,  6.60s/it]
  8%|▊         | 17318/225000 [44:57:03<458:40:10,  7.95s/it]
  8%|▊         | 17319/225000 [44:57:10<437:08:26,  7.58s/it]
  8%|▊         | 17320/225000 [44:57:15<390:22:11,  6.77s/it]
  8%|▊         | 17321/225000 [44:57:24<430:38:00,  7.46s/it]
  8%|▊         | 17322/225000 [44:57:28<364:14:18,  6.31s/it]
  8%|▊         | 17323/225000 [44:57:38<428:35:14,  7.43s/it]
  8%|▊         | 17324/225000 [44:57:42<369:33:27,  6.41s/it]
  8%|▊         | 17325/225000 [44:57:50<407:08:08,  7.06s/it]
  8%|▊         | 17326/225000 [44:57:56<388:07:47,  6.73s/it]
  8%|▊         | 17327/225000 [44:58:01<346:25:29,  6.01s/it]
  8%|▊         | 17328/225000 [44:58:05<317:12:12,  5.50s/it]
  8%|▊         | 17329/225000 [44:58:16<420:04:14,  7.28s/it]
  8%|▊         | 17330/225000 [44:58:22<389:12:09,  6.75s/it]
  8%|▊         | 17331/225000 [44:58:27<359:53:38,  6.24s/it]
  8%|▊         | 17332/225000 [44:58:32<346:35:25,  6.01s/it]
  8%|▊         | 17333/225000 [44:58:38<340:51:59,  5.91s/it]
  8%|▊         | 17334/225000 [44:58:44<334:50:56,  5.80s/it]
  8%|▊         | 17335/225000 [44:58:48<317:13:59,  5.50s/it]
  8%|▊         | 17336/225000 [44:58:54<317:25:33,  5.50s/it]
  8%|▊         | 17337/225000 [44:58:59<316:37:09,  5.49s/it]
  8%|▊         | 17338/225000 [44:59:12<437:46:19,  7.59s/it]
  8%|▊         | 17339/225000 [44:59:17<402:37:59,  6.98s/it]
  8%|▊         | 17340/225000 [44:59:23<368:50:39,  6.39s/it]
  8%|▊         | 17341/225000 [44:59:34<453:59:56,  7.87s/it]
  8%|▊         | 17342/225000 [44:59:42<453:02:11,  7.85s/it]
  8%|▊         | 17343/225000 [44:59:46<395:03:56,  6.85s/it]
  8%|▊         | 17344/225000 [44:59:51<366:51:51,  6.36s/it]
  8%|▊         | 17345/225000 [45:00:02<434:51:09,  7.54s/it]
  8%|▊         | 17346/225000 [45:00:06<380:55:11,  6.60s/it]
  8%|▊         | 17347/225000 [45:00:16<444:57:36,  7.71s/it]
  8%|▊         | 17348/225000 [45:00:22<402:18:16,  6.97s/it]
  8%|▊         | 17349/225000 [45:00:28<393:22:51,  6.82s/it]
  8%|▊         | 17350/225000 [45:00:34<370:39:43,  6.43s/it]
                                                             
{'loss': '0.7072', 'grad_norm': '0.5766', 'learning_rate': '4.986e-05', 'epoch': '0.07711', 'num_input_tokens_seen': 79556696, 'train_runtime': '1.62e+05', 'train_tokens_per_second': '491'}

  8%|▊         | 17350/225000 [45:00:34<370:39:43,  6.43s/it]
  8%|▊         | 17351/225000 [45:00:39<347:48:45,  6.03s/it]
  8%|▊         | 17352/225000 [45:00:44<330:56:01,  5.74s/it]
  8%|▊         | 17353/225000 [45:00:50<331:38:41,  5.75s/it]
  8%|▊         | 17354/225000 [45:00:55<325:43:31,  5.65s/it]
  8%|▊         | 17355/225000 [45:01:01<325:33:52,  5.64s/it]
  8%|▊         | 17356/225000 [45:01:06<324:16:08,  5.62s/it]
  8%|▊         | 17357/225000 [45:01:19<451:03:29,  7.82s/it]
  8%|▊         | 17358/225000 [45:01:26<434:54:15,  7.54s/it]
  8%|▊         | 17359/225000 [45:01:30<381:10:33,  6.61s/it]
  8%|▊         | 17360/225000 [45:01:41<447:39:47,  7.76s/it]
  8%|▊         | 17361/225000 [45:01:45<379:31:33,  6.58s/it]
  8%|▊         | 17362/225000 [45:01:53<415:26:27,  7.20s/it]
  8%|▊         | 17363/225000 [45:01:58<367:55:44,  6.38s/it]
  8%|▊         | 17364/225000 [45:02:02<330:32:51,  5.73s/it]
  8%|▊         | 17365/225000 [45:02:15<455:08:52,  7.89s/it]
  8%|▊         | 17366/225000 [45:02:20<405:59:39,  7.04s/it]
  8%|▊         | 17367/225000 [45:02:27<403:00:28,  6.99s/it]
  8%|▊         | 17368/225000 [45:02:32<375:20:54,  6.51s/it]
  8%|▊         | 17369/225000 [45:02:37<340:30:56,  5.90s/it]
  8%|▊         | 17370/225000 [45:02:47<423:35:29,  7.34s/it]
  8%|▊         | 17371/225000 [45:02:52<382:08:25,  6.63s/it]
  8%|▊         | 17372/225000 [45:02:57<346:43:35,  6.01s/it]
  8%|▊         | 17373/225000 [45:03:02<330:54:30,  5.74s/it]
  8%|▊         | 17374/225000 [45:03:08<326:22:46,  5.66s/it]
  8%|▊         | 17375/225000 [45:03:15<350:08:42,  6.07s/it]
  8%|▊         | 17376/225000 [45:03:27<460:51:37,  7.99s/it]
  8%|▊         | 17377/225000 [45:03:31<394:41:03,  6.84s/it]
  8%|▊         | 17378/225000 [45:03:36<352:51:07,  6.12s/it]
  8%|▊         | 17379/225000 [45:03:47<448:40:17,  7.78s/it]
  8%|▊         | 17380/225000 [45:03:52<390:26:19,  6.77s/it]
  8%|▊         | 17381/225000 [45:04:01<438:31:59,  7.60s/it]
  8%|▊         | 17382/225000 [45:04:07<408:09:26,  7.08s/it]
  8%|▊         | 17383/225000 [45:04:14<410:33:50,  7.12s/it]
  8%|▊         | 17384/225000 [45:04:20<383:46:30,  6.65s/it]
  8%|▊         | 17385/225000 [45:04:29<423:53:37,  7.35s/it]
  8%|▊         | 17386/225000 [45:04:33<367:15:58,  6.37s/it]
  8%|▊         | 17387/225000 [45:04:37<327:11:24,  5.67s/it]
  8%|▊         | 17388/225000 [45:04:49<437:49:43,  7.59s/it]
  8%|▊         | 17389/225000 [45:04:54<393:41:12,  6.83s/it]
  8%|▊         | 17390/225000 [45:04:59<359:22:16,  6.23s/it]
  8%|▊         | 17391/225000 [45:05:04<335:29:24,  5.82s/it]
  8%|▊         | 17392/225000 [45:05:14<414:34:48,  7.19s/it]
  8%|▊         | 17393/225000 [45:05:20<393:17:17,  6.82s/it]
  8%|▊         | 17394/225000 [45:05:25<356:39:21,  6.18s/it]
  8%|▊         | 17395/225000 [45:05:30<346:41:03,  6.01s/it]
  8%|▊         | 17396/225000 [45:05:41<424:52:15,  7.37s/it]
  8%|▊         | 17397/225000 [45:05:48<422:09:33,  7.32s/it]
  8%|▊         | 17398/225000 [45:05:56<421:53:36,  7.32s/it]
  8%|▊         | 17399/225000 [45:06:05<454:10:56,  7.88s/it]
  8%|▊         | 17400/225000 [45:06:15<497:56:50,  8.63s/it]
                                                             
{'loss': '0.7152', 'grad_norm': '0.7542', 'learning_rate': '4.985e-05', 'epoch': '0.07733', 'num_input_tokens_seen': 79709776, 'train_runtime': '1.624e+05', 'train_tokens_per_second': '490.9'}

  8%|▊         | 17400/225000 [45:06:15<497:56:50,  8.63s/it]
  8%|▊         | 17401/225000 [45:06:22<462:18:34,  8.02s/it]
  8%|▊         | 17402/225000 [45:06:26<399:55:41,  6.94s/it]
  8%|▊         | 17403/225000 [45:06:36<451:24:45,  7.83s/it]
  8%|▊         | 17404/225000 [45:06:40<387:26:09,  6.72s/it]
  8%|▊         | 17405/225000 [45:06:46<367:34:18,  6.37s/it]
  8%|▊         | 17406/225000 [45:06:52<359:11:56,  6.23s/it]
  8%|▊         | 17407/225000 [45:06:58<366:16:47,  6.35s/it]
  8%|▊         | 17408/225000 [45:07:11<472:38:13,  8.20s/it]
  8%|▊         | 17409/225000 [45:07:15<403:01:20,  6.99s/it]
  8%|▊         | 17410/225000 [45:07:20<362:29:19,  6.29s/it]
  8%|▊         | 17411/225000 [45:07:31<456:22:08,  7.91s/it]
  8%|▊         | 17412/225000 [45:07:35<391:24:26,  6.79s/it]
  8%|▊         | 17413/225000 [45:07:42<385:24:25,  6.68s/it]
  8%|▊         | 17414/225000 [45:07:50<410:38:42,  7.12s/it]
  8%|▊         | 17415/225000 [45:07:56<397:12:34,  6.89s/it]
  8%|▊         | 17416/225000 [45:08:01<354:08:15,  6.14s/it]
  8%|▊         | 17417/225000 [45:08:05<324:32:37,  5.63s/it]
  8%|▊         | 17418/225000 [45:08:16<417:43:24,  7.24s/it]
  8%|▊         | 17419/225000 [45:08:22<384:57:43,  6.68s/it]
  8%|▊         | 17420/225000 [45:08:27<356:28:17,  6.18s/it]
  8%|▊         | 17421/225000 [45:08:32<342:22:36,  5.94s/it]
  8%|▊         | 17422/225000 [45:08:45<459:44:30,  7.97s/it]
  8%|▊         | 17423/225000 [45:08:51<434:23:29,  7.53s/it]
  8%|▊         | 17424/225000 [45:08:57<412:53:24,  7.16s/it]
  8%|▊         | 17425/225000 [45:09:04<401:09:17,  6.96s/it]
  8%|▊         | 17426/225000 [45:09:08<350:00:51,  6.07s/it]
  8%|▊         | 17427/225000 [45:09:18<412:45:34,  7.16s/it]
  8%|▊         | 17428/225000 [45:09:22<371:56:17,  6.45s/it]
  8%|▊         | 17429/225000 [45:09:27<339:04:09,  5.88s/it]
  8%|▊         | 17430/225000 [45:09:34<361:25:29,  6.27s/it]
  8%|▊         | 17431/225000 [45:09:42<394:55:12,  6.85s/it]
  8%|▊         | 17432/225000 [45:09:47<357:26:47,  6.20s/it]
  8%|▊         | 17433/225000 [45:09:57<415:37:24,  7.21s/it]
  8%|▊         | 17434/225000 [45:10:02<381:17:27,  6.61s/it]
  8%|▊         | 17435/225000 [45:10:07<352:43:57,  6.12s/it]
  8%|▊         | 17436/225000 [45:10:13<351:14:52,  6.09s/it]
  8%|▊         | 17437/225000 [45:10:24<434:09:01,  7.53s/it]
  8%|▊         | 17438/225000 [45:10:28<377:22:18,  6.55s/it]
  8%|▊         | 17439/225000 [45:10:33<357:37:16,  6.20s/it]
  8%|▊         | 17440/225000 [45:10:45<442:53:13,  7.68s/it]
  8%|▊         | 17441/225000 [45:10:49<382:34:53,  6.64s/it]
  8%|▊         | 17442/225000 [45:10:54<351:41:43,  6.10s/it]
  8%|▊         | 17443/225000 [45:10:59<345:00:36,  5.98s/it]
  8%|▊         | 17444/225000 [45:11:13<473:58:28,  8.22s/it]
  8%|▊         | 17445/225000 [45:11:17<410:15:01,  7.12s/it]
  8%|▊         | 17446/225000 [45:11:22<365:21:05,  6.34s/it]
  8%|▊         | 17447/225000 [45:11:26<336:03:08,  5.83s/it]
  8%|▊         | 17448/225000 [45:11:37<420:56:02,  7.30s/it]
  8%|▊         | 17449/225000 [45:11:46<453:20:47,  7.86s/it]
  8%|▊         | 17450/225000 [45:11:50<382:35:01,  6.64s/it]
                                                             
{'loss': '0.7152', 'grad_norm': '0.6449', 'learning_rate': '4.985e-05', 'epoch': '0.07756', 'num_input_tokens_seen': 79862496, 'train_runtime': '1.627e+05', 'train_tokens_per_second': '490.8'}

  8%|▊         | 17450/225000 [45:11:50<382:35:01,  6.64s/it]
  8%|▊         | 17451/225000 [45:12:01<448:07:51,  7.77s/it]
  8%|▊         | 17452/225000 [45:12:05<393:40:55,  6.83s/it]
  8%|▊         | 17453/225000 [45:12:12<400:40:41,  6.95s/it]
  8%|▊         | 17454/225000 [45:12:18<375:06:50,  6.51s/it]
  8%|▊         | 17455/225000 [45:12:23<356:32:58,  6.18s/it]
  8%|▊         | 17456/225000 [45:12:29<348:17:11,  6.04s/it]
  8%|▊         | 17457/225000 [45:12:35<340:42:01,  5.91s/it]
  8%|▊         | 17458/225000 [45:12:40<338:54:55,  5.88s/it]
  8%|▊         | 17459/225000 [45:12:46<335:53:45,  5.83s/it]
  8%|▊         | 17460/225000 [45:12:56<401:08:40,  6.96s/it]
  8%|▊         | 17461/225000 [45:13:06<451:12:08,  7.83s/it]
  8%|▊         | 17462/225000 [45:13:10<393:22:08,  6.82s/it]
  8%|▊         | 17463/225000 [45:13:15<355:50:19,  6.17s/it]
  8%|▊         | 17464/225000 [45:13:27<456:46:46,  7.92s/it]
  8%|▊         | 17465/225000 [45:13:32<404:19:04,  7.01s/it]
  8%|▊         | 17466/225000 [45:13:40<433:18:30,  7.52s/it]
  8%|▊         | 17467/225000 [45:13:48<444:18:37,  7.71s/it]
  8%|▊         | 17468/225000 [45:13:55<416:59:25,  7.23s/it]
  8%|▊         | 17469/225000 [45:14:04<449:27:52,  7.80s/it]
  8%|▊         | 17470/225000 [45:14:08<394:07:49,  6.84s/it]
  8%|▊         | 17471/225000 [45:14:19<462:47:03,  8.03s/it]
  8%|▊         | 17472/225000 [45:14:25<420:45:59,  7.30s/it]
  8%|▊         | 17473/225000 [45:14:32<420:50:15,  7.30s/it]
  8%|▊         | 17474/225000 [45:14:36<372:20:23,  6.46s/it]
  8%|▊         | 17475/225000 [45:14:44<398:25:57,  6.91s/it]
  8%|▊         | 17476/225000 [45:14:53<422:50:10,  7.34s/it]
  8%|▊         | 17477/225000 [45:14:59<409:00:05,  7.10s/it]
  8%|▊         | 17478/225000 [45:15:04<367:55:25,  6.38s/it]
  8%|▊         | 17479/225000 [45:15:09<341:56:24,  5.93s/it]
  8%|▊         | 17480/225000 [45:15:14<331:29:12,  5.75s/it]
  8%|▊         | 17481/225000 [45:15:26<430:07:09,  7.46s/it]
  8%|▊         | 17482/225000 [45:15:30<377:23:03,  6.55s/it]
  8%|▊         | 17483/225000 [45:15:36<365:32:59,  6.34s/it]
  8%|▊         | 17484/225000 [45:15:41<346:23:09,  6.01s/it]
  8%|▊         | 17485/225000 [45:15:55<488:38:13,  8.48s/it]
  8%|▊         | 17486/225000 [45:16:01<437:08:18,  7.58s/it]
  8%|▊         | 17487/225000 [45:16:06<388:09:01,  6.73s/it]
  8%|▊         | 17488/225000 [45:16:10<354:02:09,  6.14s/it]
  8%|▊         | 17489/225000 [45:16:16<343:31:06,  5.96s/it]
  8%|▊         | 17490/225000 [45:16:21<331:29:46,  5.75s/it]
  8%|▊         | 17491/225000 [45:16:30<375:43:45,  6.52s/it]
  8%|▊         | 17492/225000 [45:16:45<536:55:06,  9.31s/it]
  8%|▊         | 17493/225000 [45:16:49<446:41:19,  7.75s/it]
  8%|▊         | 17494/225000 [45:16:54<388:15:19,  6.74s/it]
  8%|▊         | 17495/225000 [45:17:03<423:29:49,  7.35s/it]
  8%|▊         | 17496/225000 [45:17:10<422:23:33,  7.33s/it]
  8%|▊         | 17497/225000 [45:17:16<401:26:54,  6.96s/it]
  8%|▊         | 17498/225000 [45:17:22<380:57:00,  6.61s/it]
  8%|▊         | 17499/225000 [45:17:30<404:01:16,  7.01s/it]
  8%|▊         | 17500/225000 [45:17:35<373:25:15,  6.48s/it]
                                                             
{'loss': '0.7265', 'grad_norm': '0.7302', 'learning_rate': '4.985e-05', 'epoch': '0.07778', 'num_input_tokens_seen': 80017744, 'train_runtime': '1.631e+05', 'train_tokens_per_second': '490.7'}

  8%|▊         | 17500/225000 [45:17:35<373:25:15,  6.48s/it]
  8%|▊         | 17501/225000 [45:17:40<352:26:49,  6.11s/it]
  8%|▊         | 17502/225000 [45:17:45<335:00:07,  5.81s/it]
  8%|▊         | 17503/225000 [45:17:58<446:51:43,  7.75s/it]
  8%|▊         | 17504/225000 [45:18:01<373:01:40,  6.47s/it]
  8%|▊         | 17505/225000 [45:18:10<422:08:56,  7.32s/it]
  8%|▊         | 17506/225000 [45:18:16<398:51:23,  6.92s/it]
  8%|▊         | 17507/225000 [45:18:23<387:34:51,  6.72s/it]
  8%|▊         | 17508/225000 [45:18:26<332:08:20,  5.76s/it]
  8%|▊         | 17509/225000 [45:18:37<415:39:41,  7.21s/it]
  8%|▊         | 17510/225000 [45:18:43<404:23:17,  7.02s/it]
  8%|▊         | 17511/225000 [45:18:48<366:50:04,  6.36s/it]
  8%|▊         | 17512/225000 [45:18:53<339:31:52,  5.89s/it]
  8%|▊         | 17513/225000 [45:18:59<337:47:40,  5.86s/it]
  8%|▊         | 17514/225000 [45:19:11<446:03:09,  7.74s/it]
  8%|▊         | 17515/225000 [45:19:15<382:29:32,  6.64s/it]
  8%|▊         | 17516/225000 [45:19:25<446:42:54,  7.75s/it]
  8%|▊         | 17517/225000 [45:19:31<410:53:29,  7.13s/it]
  8%|▊         | 17518/225000 [45:19:38<401:19:43,  6.96s/it]
  8%|▊         | 17519/225000 [45:19:43<382:20:06,  6.63s/it]
  8%|▊         | 17520/225000 [45:19:49<362:35:39,  6.29s/it]
  8%|▊         | 17521/225000 [45:19:55<364:05:01,  6.32s/it]
  8%|▊         | 17522/225000 [45:20:05<426:40:51,  7.40s/it]
  8%|▊         | 17523/225000 [45:20:09<369:38:11,  6.41s/it]
  8%|▊         | 17524/225000 [45:20:15<355:40:41,  6.17s/it]
  8%|▊         | 17525/225000 [45:20:25<414:43:20,  7.20s/it]
  8%|▊         | 17526/225000 [45:20:29<370:18:15,  6.43s/it]
  8%|▊         | 17527/225000 [45:20:35<355:51:07,  6.17s/it]
  8%|▊         | 17528/225000 [45:20:40<338:10:54,  5.87s/it]
  8%|▊         | 17529/225000 [45:20:45<326:25:24,  5.66s/it]
  8%|▊         | 17530/225000 [45:20:52<355:37:16,  6.17s/it]
  8%|▊         | 17531/225000 [45:21:05<466:18:59,  8.09s/it]
  8%|▊         | 17532/225000 [45:21:10<409:14:18,  7.10s/it]
  8%|▊         | 17533/225000 [45:21:15<378:51:36,  6.57s/it]
  8%|▊         | 17534/225000 [45:21:21<359:05:28,  6.23s/it]
  8%|▊         | 17535/225000 [45:21:26<348:39:14,  6.05s/it]
  8%|▊         | 17536/225000 [45:21:32<335:45:14,  5.83s/it]
  8%|▊         | 17537/225000 [45:21:44<455:37:01,  7.91s/it]
  8%|▊         | 17538/225000 [45:21:52<444:32:08,  7.71s/it]
  8%|▊         | 17539/225000 [45:21:58<414:44:31,  7.20s/it]
  8%|▊         | 17540/225000 [45:22:03<386:50:36,  6.71s/it]
  8%|▊         | 17541/225000 [45:22:08<356:37:56,  6.19s/it]
  8%|▊         | 17542/225000 [45:22:13<340:32:16,  5.91s/it]
  8%|▊         | 17543/225000 [45:22:28<492:06:58,  8.54s/it]
  8%|▊         | 17544/225000 [45:22:35<460:18:26,  7.99s/it]
  8%|▊         | 17545/225000 [45:22:39<396:55:33,  6.89s/it]
  8%|▊         | 17546/225000 [45:22:48<430:34:24,  7.47s/it]
  8%|▊         | 17547/225000 [45:22:57<451:30:52,  7.84s/it]
  8%|▊         | 17548/225000 [45:23:02<413:22:33,  7.17s/it]
  8%|▊         | 17549/225000 [45:23:08<392:48:43,  6.82s/it]
  8%|▊         | 17550/225000 [45:23:18<440:52:27,  7.65s/it]
                                                             
{'loss': '0.7295', 'grad_norm': '0.7232', 'learning_rate': '4.985e-05', 'epoch': '0.078', 'num_input_tokens_seen': 80169696, 'train_runtime': '1.634e+05', 'train_tokens_per_second': '490.6'}

  8%|▊         | 17550/225000 [45:23:18<440:52:27,  7.65s/it]
  8%|▊         | 17551/225000 [45:23:24<415:11:08,  7.20s/it]
  8%|▊         | 17552/225000 [45:23:28<360:41:12,  6.26s/it]
  8%|▊         | 17553/225000 [45:23:32<327:01:34,  5.68s/it]
  8%|▊         | 17554/225000 [45:23:37<311:38:38,  5.41s/it]
  8%|▊         | 17555/225000 [45:23:42<294:31:34,  5.11s/it]
  8%|▊         | 17556/225000 [45:23:54<423:05:45,  7.34s/it]
  8%|▊         | 17557/225000 [45:24:00<400:30:33,  6.95s/it]
  8%|▊         | 17558/225000 [45:24:09<429:53:56,  7.46s/it]
  8%|▊         | 17559/225000 [45:24:12<363:21:41,  6.31s/it]
  8%|▊         | 17560/225000 [45:24:21<400:21:58,  6.95s/it]
  8%|▊         | 17561/225000 [45:24:26<366:24:43,  6.36s/it]
  8%|▊         | 17562/225000 [45:24:31<345:46:35,  6.00s/it]
  8%|▊         | 17563/225000 [45:24:36<332:12:13,  5.77s/it]
  8%|▊         | 17564/225000 [45:24:48<431:59:04,  7.50s/it]
  8%|▊         | 17565/225000 [45:24:54<409:10:50,  7.10s/it]
  8%|▊         | 17566/225000 [45:25:00<389:40:29,  6.76s/it]
  8%|▊         | 17567/225000 [45:25:05<370:13:03,  6.43s/it]
  8%|▊         | 17568/225000 [45:25:14<404:19:43,  7.02s/it]
  8%|▊         | 17569/225000 [45:25:20<385:02:08,  6.68s/it]
  8%|▊         | 17570/225000 [45:25:25<363:56:36,  6.32s/it]
  8%|▊         | 17571/225000 [45:25:30<336:27:54,  5.84s/it]
  8%|▊         | 17572/225000 [45:25:39<392:04:02,  6.80s/it]
  8%|▊         | 17573/225000 [45:25:50<469:44:39,  8.15s/it]
  8%|▊         | 17574/225000 [45:25:55<412:52:28,  7.17s/it]
  8%|▊         | 17575/225000 [45:26:00<367:05:39,  6.37s/it]
  8%|▊         | 17576/225000 [45:26:05<349:36:26,  6.07s/it]
  8%|▊         | 17577/225000 [45:26:18<465:53:05,  8.09s/it]
  8%|▊         | 17578/225000 [45:26:22<392:17:05,  6.81s/it]
  8%|▊         | 17579/225000 [45:26:27<367:29:41,  6.38s/it]
  8%|▊         | 17580/225000 [45:26:35<398:32:20,  6.92s/it]
  8%|▊         | 17581/225000 [45:26:42<397:39:06,  6.90s/it]
  8%|▊         | 17582/225000 [45:26:49<397:26:49,  6.90s/it]
  8%|▊         | 17583/225000 [45:26:57<415:59:10,  7.22s/it]
  8%|▊         | 17584/225000 [45:27:01<365:28:42,  6.34s/it]
  8%|▊         | 17585/225000 [45:27:10<403:21:59,  7.00s/it]
  8%|▊         | 17586/225000 [45:27:15<380:14:53,  6.60s/it]
  8%|▊         | 17587/225000 [45:27:20<348:04:00,  6.04s/it]
  8%|▊         | 17588/225000 [45:27:29<396:33:58,  6.88s/it]
  8%|▊         | 17589/225000 [45:27:36<393:47:11,  6.83s/it]
  8%|▊         | 17590/225000 [45:27:42<379:40:40,  6.59s/it]
  8%|▊         | 17591/225000 [45:27:47<349:04:03,  6.06s/it]
  8%|▊         | 17592/225000 [45:27:51<322:21:39,  5.60s/it]
  8%|▊         | 17593/225000 [45:28:03<425:57:21,  7.39s/it]
  8%|▊         | 17594/225000 [45:28:06<359:19:22,  6.24s/it]
  8%|▊         | 17595/225000 [45:28:16<417:48:17,  7.25s/it]
  8%|▊         | 17596/225000 [45:28:24<428:33:57,  7.44s/it]
  8%|▊         | 17597/225000 [45:28:29<392:05:11,  6.81s/it]
  8%|▊         | 17598/225000 [45:28:36<398:26:54,  6.92s/it]
  8%|▊         | 17599/225000 [45:28:40<346:55:36,  6.02s/it]
  8%|▊         | 17600/225000 [45:28:46<334:57:02,  5.81s/it]
                                                             
{'loss': '0.7081', 'grad_norm': '0.715', 'learning_rate': '4.985e-05', 'epoch': '0.07822', 'num_input_tokens_seen': 80323248, 'train_runtime': '1.637e+05', 'train_tokens_per_second': '490.6'}

  8%|▊         | 17600/225000 [45:28:46<334:57:02,  5.81s/it]
  8%|▊         | 17601/225000 [45:28:51<332:30:10,  5.77s/it]
  8%|▊         | 17602/225000 [45:28:57<334:45:45,  5.81s/it]
  8%|▊         | 17603/225000 [45:29:06<392:51:34,  6.82s/it]
  8%|▊         | 17604/225000 [45:29:15<426:47:11,  7.41s/it]
  8%|▊         | 17605/225000 [45:29:19<374:12:20,  6.50s/it]
  8%|▊         | 17606/225000 [45:29:24<342:45:03,  5.95s/it]
  8%|▊         | 17607/225000 [45:29:31<354:16:49,  6.15s/it]
  8%|▊         | 17608/225000 [45:29:43<463:10:11,  8.04s/it]
  8%|▊         | 17609/225000 [45:29:47<390:09:22,  6.77s/it]
  8%|▊         | 17610/225000 [45:29:56<428:43:01,  7.44s/it]
  8%|▊         | 17611/225000 [45:30:00<370:14:35,  6.43s/it]
  8%|▊         | 17612/225000 [45:30:08<401:15:24,  6.97s/it]
  8%|▊         | 17613/225000 [45:30:14<376:14:12,  6.53s/it]
  8%|▊         | 17614/225000 [45:30:20<363:00:03,  6.30s/it]
  8%|▊         | 17615/225000 [45:30:25<348:00:49,  6.04s/it]
  8%|▊         | 17616/225000 [45:30:31<349:06:04,  6.06s/it]
  8%|▊         | 17617/225000 [45:30:36<329:02:25,  5.71s/it]
  8%|▊         | 17618/225000 [45:30:41<315:04:27,  5.47s/it]
  8%|▊         | 17619/225000 [45:30:53<427:34:33,  7.42s/it]
  8%|▊         | 17620/225000 [45:30:58<387:39:40,  6.73s/it]
  8%|▊         | 17621/225000 [45:31:04<372:15:00,  6.46s/it]
  8%|▊         | 17622/225000 [45:31:09<352:41:08,  6.12s/it]
  8%|▊         | 17623/225000 [45:31:15<353:05:35,  6.13s/it]
  8%|▊         | 17624/225000 [45:31:21<346:18:26,  6.01s/it]
  8%|▊         | 17625/225000 [45:31:33<448:31:08,  7.79s/it]
  8%|▊         | 17626/225000 [45:31:42<470:28:27,  8.17s/it]
  8%|▊         | 17627/225000 [45:31:48<429:18:51,  7.45s/it]
  8%|▊         | 17628/225000 [45:31:57<457:53:15,  7.95s/it]
  8%|▊         | 17629/225000 [45:32:02<415:48:58,  7.22s/it]
  8%|▊         | 17630/225000 [45:32:07<363:12:21,  6.31s/it]
  8%|▊         | 17631/225000 [45:32:12<342:46:22,  5.95s/it]
  8%|▊         | 17632/225000 [45:32:23<436:42:08,  7.58s/it]
  8%|▊         | 17633/225000 [45:32:27<375:09:39,  6.51s/it]
  8%|▊         | 17634/225000 [45:32:37<436:12:26,  7.57s/it]
  8%|▊         | 17635/225000 [45:32:41<374:08:00,  6.50s/it]
  8%|▊         | 17636/225000 [45:32:49<391:08:55,  6.79s/it]
  8%|▊         | 17637/225000 [45:32:53<349:54:39,  6.07s/it]
  8%|▊         | 17638/225000 [45:33:01<374:48:04,  6.51s/it]
  8%|▊         | 17639/225000 [45:33:08<391:13:08,  6.79s/it]
  8%|▊         | 17640/225000 [45:33:15<399:03:57,  6.93s/it]
  8%|▊         | 17641/225000 [45:33:21<374:51:24,  6.51s/it]
  8%|▊         | 17642/225000 [45:33:28<389:43:20,  6.77s/it]
  8%|▊         | 17643/225000 [45:33:33<361:31:16,  6.28s/it]
  8%|▊         | 17644/225000 [45:33:39<351:38:12,  6.10s/it]
  8%|▊         | 17645/225000 [45:33:49<423:55:55,  7.36s/it]
  8%|▊         | 17646/225000 [45:33:55<399:04:23,  6.93s/it]
  8%|▊         | 17647/225000 [45:34:01<371:48:34,  6.46s/it]
  8%|▊         | 17648/225000 [45:34:05<336:32:17,  5.84s/it]
  8%|▊         | 17649/225000 [45:34:12<348:45:49,  6.06s/it]
  8%|▊         | 17650/225000 [45:34:22<419:33:59,  7.28s/it]
                                                             
{'loss': '0.7214', 'grad_norm': '0.6685', 'learning_rate': '4.984e-05', 'epoch': '0.07844', 'num_input_tokens_seen': 80475480, 'train_runtime': '1.641e+05', 'train_tokens_per_second': '490.5'}

  8%|▊         | 17650/225000 [45:34:22<419:33:59,  7.28s/it]
  8%|▊         | 17651/225000 [45:34:30<437:26:06,  7.59s/it]
  8%|▊         | 17652/225000 [45:34:35<387:00:40,  6.72s/it]
  8%|▊         | 17653/225000 [45:34:40<358:38:39,  6.23s/it]
  8%|▊         | 17654/225000 [45:34:45<342:35:46,  5.95s/it]
  8%|▊         | 17655/225000 [45:34:50<324:20:40,  5.63s/it]
  8%|▊         | 17656/225000 [45:34:55<315:52:31,  5.48s/it]
  8%|▊         | 17657/225000 [45:35:05<388:03:35,  6.74s/it]
  8%|▊         | 17658/225000 [45:35:16<473:07:05,  8.21s/it]
  8%|▊         | 17659/225000 [45:35:21<408:55:23,  7.10s/it]
  8%|▊         | 17660/225000 [45:35:29<424:19:04,  7.37s/it]
  8%|▊         | 17661/225000 [45:35:34<381:52:35,  6.63s/it]
  8%|▊         | 17662/225000 [45:35:39<351:52:50,  6.11s/it]
  8%|▊         | 17663/225000 [45:35:44<331:50:54,  5.76s/it]
  8%|▊         | 17664/225000 [45:35:58<476:31:42,  8.27s/it]
  8%|▊         | 17665/225000 [45:36:03<415:42:54,  7.22s/it]
  8%|▊         | 17666/225000 [45:36:10<428:09:37,  7.43s/it]
  8%|▊         | 17667/225000 [45:36:15<381:04:33,  6.62s/it]
  8%|▊         | 17668/225000 [45:36:19<339:34:12,  5.90s/it]
  8%|▊         | 17669/225000 [45:36:30<416:43:08,  7.24s/it]
  8%|▊         | 17670/225000 [45:36:36<404:47:54,  7.03s/it]
  8%|▊         | 17671/225000 [45:36:44<411:32:10,  7.15s/it]
  8%|▊         | 17672/225000 [45:36:51<414:41:37,  7.20s/it]
  8%|▊         | 17673/225000 [45:36:56<373:15:43,  6.48s/it]
  8%|▊         | 17674/225000 [45:37:00<336:13:57,  5.84s/it]
  8%|▊         | 17675/225000 [45:37:05<322:49:49,  5.61s/it]
  8%|▊         | 17676/225000 [45:37:19<466:16:17,  8.10s/it]
  8%|▊         | 17677/225000 [45:37:24<404:26:26,  7.02s/it]
  8%|▊         | 17678/225000 [45:37:32<423:22:22,  7.35s/it]
  8%|▊         | 17679/225000 [45:37:39<420:08:13,  7.30s/it]
  8%|▊         | 17680/225000 [45:37:44<375:24:28,  6.52s/it]
  8%|▊         | 17681/225000 [45:37:51<385:51:57,  6.70s/it]
  8%|▊         | 17682/225000 [45:37:57<382:25:35,  6.64s/it]
  8%|▊         | 17683/225000 [45:38:06<413:49:22,  7.19s/it]
  8%|▊         | 17684/225000 [45:38:11<373:51:59,  6.49s/it]
  8%|▊         | 17685/225000 [45:38:17<363:57:07,  6.32s/it]
  8%|▊         | 17686/225000 [45:38:22<351:51:27,  6.11s/it]
  8%|▊         | 17687/225000 [45:38:28<345:29:06,  6.00s/it]
  8%|▊         | 17688/225000 [45:38:34<341:50:27,  5.94s/it]
  8%|▊         | 17689/225000 [45:38:39<333:17:29,  5.79s/it]
  8%|▊         | 17690/225000 [45:38:44<309:45:59,  5.38s/it]
  8%|▊         | 17691/225000 [45:38:49<312:26:37,  5.43s/it]
  8%|▊         | 17692/225000 [45:38:56<341:42:41,  5.93s/it]
  8%|▊         | 17693/225000 [45:39:09<465:01:15,  8.08s/it]
  8%|▊         | 17694/225000 [45:39:19<487:19:49,  8.46s/it]
  8%|▊         | 17695/225000 [45:39:23<408:39:18,  7.10s/it]
  8%|▊         | 17696/225000 [45:39:32<449:13:53,  7.80s/it]
  8%|▊         | 17697/225000 [45:39:36<389:25:44,  6.76s/it]
  8%|▊         | 17698/225000 [45:39:45<417:59:33,  7.26s/it]
  8%|▊         | 17699/225000 [45:39:51<394:46:26,  6.86s/it]
  8%|▊         | 17700/225000 [45:39:55<359:31:41,  6.24s/it]
                                                             
{'loss': '0.7139', 'grad_norm': '0.6558', 'learning_rate': '4.984e-05', 'epoch': '0.07867', 'num_input_tokens_seen': 80628048, 'train_runtime': '1.644e+05', 'train_tokens_per_second': '490.4'}

  8%|▊         | 17700/225000 [45:39:55<359:31:41,  6.24s/it]
  8%|▊         | 17701/225000 [45:40:00<337:40:56,  5.86s/it]
  8%|▊         | 17702/225000 [45:40:05<320:18:36,  5.56s/it]
  8%|▊         | 17703/225000 [45:40:12<339:41:49,  5.90s/it]
  8%|▊         | 17704/225000 [45:40:24<436:47:19,  7.59s/it]
  8%|▊         | 17705/225000 [45:40:30<425:05:16,  7.38s/it]
  8%|▊         | 17706/225000 [45:40:38<426:21:04,  7.40s/it]
  8%|▊         | 17707/225000 [45:40:44<395:54:33,  6.88s/it]
  8%|▊         | 17708/225000 [45:40:49<365:01:48,  6.34s/it]
  8%|▊         | 17709/225000 [45:40:54<347:35:12,  6.04s/it]
  8%|▊         | 17710/225000 [45:41:00<345:48:17,  6.01s/it]
  8%|▊         | 17711/225000 [45:41:06<340:08:25,  5.91s/it]
  8%|▊         | 17712/225000 [45:41:17<428:17:48,  7.44s/it]
  8%|▊         | 17713/225000 [45:41:22<395:12:05,  6.86s/it]
  8%|▊         | 17714/225000 [45:41:28<373:34:47,  6.49s/it]
  8%|▊         | 17715/225000 [45:41:39<450:14:47,  7.82s/it]
  8%|▊         | 17716/225000 [45:41:45<427:30:06,  7.42s/it]
  8%|▊         | 17717/225000 [45:41:52<418:36:58,  7.27s/it]
  8%|▊         | 17718/225000 [45:41:57<375:05:33,  6.51s/it]
  8%|▊         | 17719/225000 [45:42:06<414:12:13,  7.19s/it]
  8%|▊         | 17720/225000 [45:42:14<432:50:06,  7.52s/it]
  8%|▊         | 17721/225000 [45:42:18<382:55:07,  6.65s/it]
  8%|▊         | 17722/225000 [45:42:24<356:22:54,  6.19s/it]
  8%|▊         | 17723/225000 [45:42:29<347:02:19,  6.03s/it]
  8%|▊         | 17724/225000 [45:42:35<335:48:51,  5.83s/it]
  8%|▊         | 17725/225000 [45:42:40<325:49:37,  5.66s/it]
  8%|▊         | 17726/225000 [45:42:51<416:17:18,  7.23s/it]
  8%|▊         | 17727/225000 [45:43:00<449:23:07,  7.81s/it]
  8%|▊         | 17728/225000 [45:43:06<421:26:39,  7.32s/it]
  8%|▊         | 17729/225000 [45:43:12<388:18:25,  6.74s/it]
  8%|▊         | 17730/225000 [45:43:19<401:29:33,  6.97s/it]
  8%|▊         | 17731/225000 [45:43:25<381:55:25,  6.63s/it]
  8%|▊         | 17732/225000 [45:43:32<393:45:47,  6.84s/it]
  8%|▊         | 17733/225000 [45:43:42<446:03:34,  7.75s/it]
  8%|▊         | 17734/225000 [45:43:46<386:27:29,  6.71s/it]
  8%|▊         | 17735/225000 [45:43:58<464:32:54,  8.07s/it]
  8%|▊         | 17736/225000 [45:44:01<386:45:43,  6.72s/it]
  8%|▊         | 17737/225000 [45:44:07<375:43:01,  6.53s/it]
  8%|▊         | 17738/225000 [45:44:13<358:03:30,  6.22s/it]
  8%|▊         | 17739/225000 [45:44:19<353:51:10,  6.15s/it]
  8%|▊         | 17740/225000 [45:44:24<345:34:43,  6.00s/it]
  8%|▊         | 17741/225000 [45:44:34<402:01:37,  6.98s/it]
  8%|▊         | 17742/225000 [45:44:43<442:02:58,  7.68s/it]
  8%|▊         | 17743/225000 [45:44:47<374:41:46,  6.51s/it]
  8%|▊         | 17744/225000 [45:44:58<450:11:59,  7.82s/it]
  8%|▊         | 17745/225000 [45:45:05<451:46:44,  7.85s/it]
  8%|▊         | 17746/225000 [45:45:10<392:38:25,  6.82s/it]
  8%|▊         | 17747/225000 [45:45:14<352:41:10,  6.13s/it]
  8%|▊         | 17748/225000 [45:45:22<383:37:58,  6.66s/it]
  8%|▊         | 17749/225000 [45:45:28<365:49:27,  6.35s/it]
  8%|▊         | 17750/225000 [45:45:34<351:56:41,  6.11s/it]
                                                             
{'loss': '0.7177', 'grad_norm': '0.6437', 'learning_rate': '4.984e-05', 'epoch': '0.07889', 'num_input_tokens_seen': 80780584, 'train_runtime': '1.647e+05', 'train_tokens_per_second': '490.4'}

  8%|▊         | 17750/225000 [45:45:34<351:56:41,  6.11s/it]
  8%|▊         | 17751/225000 [45:45:39<340:32:30,  5.92s/it]
  8%|▊         | 17752/225000 [45:45:44<325:28:30,  5.65s/it]
  8%|▊         | 17753/225000 [45:45:49<311:14:31,  5.41s/it]
  8%|▊         | 17754/225000 [45:46:01<425:56:54,  7.40s/it]
  8%|▊         | 17755/225000 [45:46:11<478:26:53,  8.31s/it]
  8%|▊         | 17756/225000 [45:46:17<427:52:17,  7.43s/it]
  8%|▊         | 17757/225000 [45:46:24<424:10:15,  7.37s/it]
  8%|▊         | 17758/225000 [45:46:30<407:43:33,  7.08s/it]
  8%|▊         | 17759/225000 [45:46:37<396:35:04,  6.89s/it]
  8%|▊         | 17760/225000 [45:46:42<359:52:37,  6.25s/it]
  8%|▊         | 17761/225000 [45:46:47<340:47:51,  5.92s/it]
  8%|▊         | 17762/225000 [45:46:52<328:58:13,  5.71s/it]
  8%|▊         | 17763/225000 [45:46:57<321:31:32,  5.59s/it]
  8%|▊         | 17764/225000 [45:47:11<466:27:35,  8.10s/it]
  8%|▊         | 17765/225000 [45:47:17<430:16:00,  7.47s/it]
  8%|▊         | 17766/225000 [45:47:23<398:28:21,  6.92s/it]
  8%|▊         | 17767/225000 [45:47:28<371:37:28,  6.46s/it]
  8%|▊         | 17768/225000 [45:47:35<380:13:51,  6.61s/it]
  8%|▊         | 17769/225000 [45:47:41<365:00:50,  6.34s/it]
  8%|▊         | 17770/225000 [45:47:47<364:09:20,  6.33s/it]
  8%|▊         | 17771/225000 [45:48:00<482:36:57,  8.38s/it]
  8%|▊         | 17772/225000 [45:48:04<397:04:59,  6.90s/it]
  8%|▊         | 17773/225000 [45:48:13<444:48:31,  7.73s/it]
  8%|▊         | 17774/225000 [45:48:18<383:00:25,  6.65s/it]
  8%|▊         | 17775/225000 [45:48:29<457:02:50,  7.94s/it]
  8%|▊         | 17776/225000 [45:48:32<387:32:54,  6.73s/it]
  8%|▊         | 17777/225000 [45:48:40<404:45:13,  7.03s/it]
  8%|▊         | 17778/225000 [45:48:44<357:13:04,  6.21s/it]
  8%|▊         | 17779/225000 [45:48:49<324:23:00,  5.64s/it]
  8%|▊         | 17780/225000 [45:49:02<452:57:47,  7.87s/it]
  8%|▊         | 17781/225000 [45:49:06<387:03:48,  6.72s/it]
  8%|▊         | 17782/225000 [45:49:12<384:08:48,  6.67s/it]
  8%|▊         | 17783/225000 [45:49:22<425:16:18,  7.39s/it]
  8%|▊         | 17784/225000 [45:49:26<367:18:11,  6.38s/it]
  8%|▊         | 17785/225000 [45:49:31<343:13:47,  5.96s/it]
  8%|▊         | 17786/225000 [45:49:36<332:35:35,  5.78s/it]
  8%|▊         | 17787/225000 [45:49:42<330:45:36,  5.75s/it]
  8%|▊         | 17788/225000 [45:49:47<317:37:00,  5.52s/it]
  8%|▊         | 17789/225000 [45:49:58<414:52:36,  7.21s/it]
  8%|▊         | 17790/225000 [45:50:10<505:01:18,  8.77s/it]
  8%|▊         | 17791/225000 [45:50:17<473:15:22,  8.22s/it]
  8%|▊         | 17792/225000 [45:50:21<398:18:38,  6.92s/it]
  8%|▊         | 17793/225000 [45:50:31<454:51:19,  7.90s/it]
  8%|▊         | 17794/225000 [45:50:35<389:11:59,  6.76s/it]
  8%|▊         | 17795/225000 [45:50:40<358:33:53,  6.23s/it]
  8%|▊         | 17796/225000 [45:50:51<432:21:56,  7.51s/it]
  8%|▊         | 17797/225000 [45:50:55<372:33:57,  6.47s/it]
  8%|▊         | 17798/225000 [45:51:00<346:44:47,  6.02s/it]
  8%|▊         | 17799/225000 [45:51:05<328:10:31,  5.70s/it]
  8%|▊         | 17800/225000 [45:51:10<324:22:08,  5.64s/it]
                                                             
{'loss': '0.7248', 'grad_norm': '0.678', 'learning_rate': '4.984e-05', 'epoch': '0.07911', 'num_input_tokens_seen': 80933232, 'train_runtime': '1.651e+05', 'train_tokens_per_second': '490.3'}

  8%|▊         | 17800/225000 [45:51:10<324:22:08,  5.64s/it]
  8%|▊         | 17801/225000 [45:51:22<434:53:10,  7.56s/it]
  8%|▊         | 17802/225000 [45:51:26<376:33:40,  6.54s/it]
  8%|▊         | 17803/225000 [45:51:35<408:20:55,  7.09s/it]
  8%|▊         | 17804/225000 [45:51:39<353:17:51,  6.14s/it]
  8%|▊         | 17805/225000 [45:51:48<404:37:21,  7.03s/it]
  8%|▊         | 17806/225000 [45:51:53<371:01:33,  6.45s/it]
  8%|▊         | 17807/225000 [45:51:57<334:00:15,  5.80s/it]
  8%|▊         | 17808/225000 [45:52:03<333:35:13,  5.80s/it]
  8%|▊         | 17809/225000 [45:52:15<440:03:12,  7.65s/it]
  8%|▊         | 17810/225000 [45:52:19<385:56:14,  6.71s/it]
  8%|▊         | 17811/225000 [45:52:27<399:30:00,  6.94s/it]
  8%|▊         | 17812/225000 [45:52:32<361:52:08,  6.29s/it]
  8%|▊         | 17813/225000 [45:52:37<345:15:32,  6.00s/it]
  8%|▊         | 17814/225000 [45:52:46<400:50:00,  6.96s/it]
  8%|▊         | 17815/225000 [45:52:52<388:08:54,  6.74s/it]
  8%|▊         | 17816/225000 [45:52:57<348:46:18,  6.06s/it]
  8%|▊         | 17817/225000 [45:53:02<326:18:49,  5.67s/it]
  8%|▊         | 17818/225000 [45:53:08<332:28:52,  5.78s/it]
  8%|▊         | 17819/225000 [45:53:21<465:53:58,  8.10s/it]
  8%|▊         | 17820/225000 [45:53:26<408:06:53,  7.09s/it]
  8%|▊         | 17821/225000 [45:53:30<357:29:33,  6.21s/it]
  8%|▊         | 17822/225000 [45:53:41<438:56:18,  7.63s/it]
  8%|▊         | 17823/225000 [45:53:50<456:53:32,  7.94s/it]
  8%|▊         | 17824/225000 [45:53:54<388:37:45,  6.75s/it]
  8%|▊         | 17825/225000 [45:54:04<442:34:10,  7.69s/it]
  8%|▊         | 17826/225000 [45:54:08<393:23:30,  6.84s/it]
  8%|▊         | 17827/225000 [45:54:13<357:00:51,  6.20s/it]
  8%|▊         | 17828/225000 [45:54:18<327:06:10,  5.68s/it]
  8%|▊         | 17829/225000 [45:54:31<453:32:58,  7.88s/it]
  8%|▊         | 17830/225000 [45:54:38<438:53:39,  7.63s/it]
  8%|▊         | 17831/225000 [45:54:42<388:51:56,  6.76s/it]
  8%|▊         | 17832/225000 [45:54:51<415:09:54,  7.21s/it]
  8%|▊         | 17833/225000 [45:54:56<385:32:25,  6.70s/it]
  8%|▊         | 17834/225000 [45:55:02<368:51:07,  6.41s/it]
  8%|▊         | 17835/225000 [45:55:10<389:49:56,  6.77s/it]
  8%|▊         | 17836/225000 [45:55:15<374:31:50,  6.51s/it]
  8%|▊         | 17837/225000 [45:55:25<424:17:27,  7.37s/it]
  8%|▊         | 17838/225000 [45:55:30<384:48:43,  6.69s/it]
  8%|▊         | 17839/225000 [45:55:35<364:47:11,  6.34s/it]
  8%|▊         | 17840/225000 [45:55:40<329:19:37,  5.72s/it]
  8%|▊         | 17841/225000 [45:55:54<472:22:02,  8.21s/it]
  8%|▊         | 17842/225000 [45:56:00<439:17:51,  7.63s/it]
  8%|▊         | 17843/225000 [45:56:06<409:19:56,  7.11s/it]
  8%|▊         | 17844/225000 [45:56:11<366:53:01,  6.38s/it]
  8%|▊         | 17845/225000 [45:56:16<346:34:21,  6.02s/it]
  8%|▊         | 17846/225000 [45:56:21<329:07:23,  5.72s/it]
  8%|▊         | 17847/225000 [45:56:26<324:17:18,  5.64s/it]
  8%|▊         | 17848/225000 [45:56:32<321:01:01,  5.58s/it]
  8%|▊         | 17849/225000 [45:56:45<456:19:11,  7.93s/it]
  8%|▊         | 17850/225000 [45:56:52<446:19:54,  7.76s/it]
                                                             
{'loss': '0.72', 'grad_norm': '0.5888', 'learning_rate': '4.984e-05', 'epoch': '0.07933', 'num_input_tokens_seen': 81086184, 'train_runtime': '1.654e+05', 'train_tokens_per_second': '490.2'}

  8%|▊         | 17850/225000 [45:56:52<446:19:54,  7.76s/it]
  8%|▊         | 17851/225000 [45:57:00<435:09:57,  7.56s/it]
  8%|▊         | 17852/225000 [45:57:06<415:13:13,  7.22s/it]
  8%|▊         | 17853/225000 [45:57:13<414:27:03,  7.20s/it]
  8%|▊         | 17854/225000 [45:57:19<399:20:40,  6.94s/it]
  8%|▊         | 17855/225000 [45:57:24<359:26:12,  6.25s/it]
  8%|▊         | 17856/225000 [45:57:28<324:50:34,  5.65s/it]
  8%|▊         | 17857/225000 [45:57:33<304:14:08,  5.29s/it]
  8%|▊         | 17858/225000 [45:57:43<397:35:47,  6.91s/it]
  8%|▊         | 17859/225000 [45:57:50<397:18:22,  6.90s/it]
  8%|▊         | 17860/225000 [45:57:55<355:17:08,  6.17s/it]
  8%|▊         | 17861/225000 [45:58:04<410:32:42,  7.14s/it]
  8%|▊         | 17862/225000 [45:58:13<431:37:21,  7.50s/it]
  8%|▊         | 17863/225000 [45:58:17<385:16:53,  6.70s/it]
  8%|▊         | 17864/225000 [45:58:23<366:19:05,  6.37s/it]
  8%|▊         | 17865/225000 [45:58:27<332:51:42,  5.79s/it]
  8%|▊         | 17866/225000 [45:58:36<377:51:44,  6.57s/it]
  8%|▊         | 17867/225000 [45:58:46<442:55:40,  7.70s/it]
  8%|▊         | 17868/225000 [45:58:52<416:41:50,  7.24s/it]
  8%|▊         | 17869/225000 [45:58:58<395:33:50,  6.88s/it]
  8%|▊         | 17870/225000 [45:59:03<361:58:15,  6.29s/it]
  8%|▊         | 17871/225000 [45:59:08<336:06:57,  5.84s/it]
  8%|▊         | 17872/225000 [45:59:18<401:33:43,  6.98s/it]
  8%|▊         | 17873/225000 [45:59:27<438:20:59,  7.62s/it]
  8%|▊         | 17874/225000 [45:59:31<373:03:08,  6.48s/it]
  8%|▊         | 17875/225000 [45:59:35<343:16:47,  5.97s/it]
  8%|▊         | 17876/225000 [45:59:46<430:42:06,  7.49s/it]
  8%|▊         | 17877/225000 [45:59:50<369:56:26,  6.43s/it]
  8%|▊         | 17878/225000 [45:59:56<357:08:52,  6.21s/it]
  8%|▊         | 17879/225000 [46:00:05<412:05:31,  7.16s/it]
  8%|▊         | 17880/225000 [46:00:11<387:12:43,  6.73s/it]
  8%|▊         | 17881/225000 [46:00:17<367:03:19,  6.38s/it]
  8%|▊         | 17882/225000 [46:00:22<353:28:52,  6.14s/it]
  8%|▊         | 17883/225000 [46:00:28<340:54:17,  5.93s/it]
  8%|▊         | 17884/225000 [46:00:34<338:52:07,  5.89s/it]
  8%|▊         | 17885/225000 [46:00:40<339:49:08,  5.91s/it]
  8%|▊         | 17886/225000 [46:00:51<442:47:27,  7.70s/it]
  8%|▊         | 17887/225000 [46:00:56<385:18:16,  6.70s/it]
  8%|▊         | 17888/225000 [46:01:00<350:04:25,  6.08s/it]
  8%|▊         | 17889/225000 [46:01:13<469:17:36,  8.16s/it]
  8%|▊         | 17890/225000 [46:01:18<404:13:11,  7.03s/it]
  8%|▊         | 17891/225000 [46:01:23<380:06:29,  6.61s/it]
  8%|▊         | 17892/225000 [46:01:29<362:22:30,  6.30s/it]
  8%|▊         | 17893/225000 [46:01:35<350:45:40,  6.10s/it]
  8%|▊         | 17894/225000 [46:01:40<343:35:05,  5.97s/it]
  8%|▊         | 17895/225000 [46:01:46<337:20:25,  5.86s/it]
  8%|▊         | 17896/225000 [46:01:57<426:03:03,  7.41s/it]
  8%|▊         | 17897/225000 [46:02:03<397:24:31,  6.91s/it]
  8%|▊         | 17898/225000 [46:02:08<372:49:49,  6.48s/it]
  8%|▊         | 17899/225000 [46:02:14<358:19:09,  6.23s/it]
  8%|▊         | 17900/225000 [46:02:20<357:40:58,  6.22s/it]
                                                             
{'loss': '0.7189', 'grad_norm': '0.6909', 'learning_rate': '4.983e-05', 'epoch': '0.07956', 'num_input_tokens_seen': 81238544, 'train_runtime': '1.657e+05', 'train_tokens_per_second': '490.2'}

  8%|▊         | 17900/225000 [46:02:20<357:40:58,  6.22s/it]
  8%|▊         | 17901/225000 [46:02:26<357:48:59,  6.22s/it]
  8%|▊         | 17902/225000 [46:02:33<363:11:07,  6.31s/it]
  8%|▊         | 17903/225000 [46:02:46<480:27:49,  8.35s/it]
  8%|▊         | 17904/225000 [46:02:50<410:35:54,  7.14s/it]
  8%|▊         | 17905/225000 [46:02:59<444:06:34,  7.72s/it]
  8%|▊         | 17906/225000 [46:03:05<409:46:50,  7.12s/it]
  8%|▊         | 17907/225000 [46:03:10<381:12:06,  6.63s/it]
  8%|▊         | 17908/225000 [46:03:16<358:14:44,  6.23s/it]
  8%|▊         | 17909/225000 [46:03:21<340:38:47,  5.92s/it]
  8%|▊         | 17910/225000 [46:03:26<331:10:43,  5.76s/it]
  8%|▊         | 17911/225000 [46:03:32<328:11:45,  5.71s/it]
  8%|▊         | 17912/225000 [46:03:44<442:38:42,  7.69s/it]
  8%|▊         | 17913/225000 [46:03:49<393:27:57,  6.84s/it]
  8%|▊         | 17914/225000 [46:03:58<424:51:36,  7.39s/it]
  8%|▊         | 17915/225000 [46:04:06<443:18:40,  7.71s/it]
  8%|▊         | 17916/225000 [46:04:12<404:52:29,  7.04s/it]
  8%|▊         | 17917/225000 [46:04:17<371:17:13,  6.45s/it]
  8%|▊         | 17918/225000 [46:04:21<340:20:45,  5.92s/it]
  8%|▊         | 17919/225000 [46:04:26<324:48:54,  5.65s/it]
  8%|▊         | 17920/225000 [46:04:31<306:43:28,  5.33s/it]
  8%|▊         | 17921/225000 [46:04:44<436:13:11,  7.58s/it]
  8%|▊         | 17922/225000 [46:04:50<409:20:39,  7.12s/it]
  8%|▊         | 17923/225000 [46:04:57<400:27:39,  6.96s/it]
  8%|▊         | 17924/225000 [46:05:06<450:35:21,  7.83s/it]
  8%|▊         | 17925/225000 [46:05:12<404:39:42,  7.04s/it]
  8%|▊         | 17926/225000 [46:05:17<370:10:04,  6.44s/it]
  8%|▊         | 17927/225000 [46:05:22<346:22:32,  6.02s/it]
  8%|▊         | 17928/225000 [46:05:27<328:13:46,  5.71s/it]
  8%|▊         | 17929/225000 [46:05:32<322:36:53,  5.61s/it]
  8%|▊         | 17930/225000 [46:05:37<318:32:04,  5.54s/it]
  8%|▊         | 17931/225000 [46:05:47<393:35:45,  6.84s/it]
  8%|▊         | 17932/225000 [46:05:59<472:24:20,  8.21s/it]
  8%|▊         | 17933/225000 [46:06:04<424:27:42,  7.38s/it]
  8%|▊         | 17934/225000 [46:06:11<410:16:37,  7.13s/it]
  8%|▊         | 17935/225000 [46:06:15<361:42:09,  6.29s/it]
  8%|▊         | 17936/225000 [46:06:20<340:17:45,  5.92s/it]
  8%|▊         | 17937/225000 [46:06:28<370:25:44,  6.44s/it]
  8%|▊         | 17938/225000 [46:06:38<435:45:36,  7.58s/it]
  8%|▊         | 17939/225000 [46:06:42<374:51:20,  6.52s/it]
  8%|▊         | 17940/225000 [46:06:47<346:46:08,  6.03s/it]
  8%|▊         | 17941/225000 [46:06:58<428:43:39,  7.45s/it]
  8%|▊         | 17942/225000 [46:07:03<391:25:53,  6.81s/it]
  8%|▊         | 17943/225000 [46:07:12<430:03:01,  7.48s/it]
  8%|▊         | 17944/225000 [46:07:19<420:24:40,  7.31s/it]
  8%|▊         | 17945/225000 [46:07:23<368:14:44,  6.40s/it]
  8%|▊         | 17946/225000 [46:07:28<346:54:59,  6.03s/it]
  8%|▊         | 17947/225000 [46:07:38<405:34:22,  7.05s/it]
  8%|▊         | 17948/225000 [46:07:46<419:20:34,  7.29s/it]
  8%|▊         | 17949/225000 [46:07:51<378:55:39,  6.59s/it]
  8%|▊         | 17950/225000 [46:07:55<345:13:03,  6.00s/it]
                                                             
{'loss': '0.7231', 'grad_norm': '0.5905', 'learning_rate': '4.983e-05', 'epoch': '0.07978', 'num_input_tokens_seen': 81392944, 'train_runtime': '1.661e+05', 'train_tokens_per_second': '490.1'}

  8%|▊         | 17950/225000 [46:07:55<345:13:03,  6.00s/it]
  8%|▊         | 17951/225000 [46:08:02<359:24:45,  6.25s/it]
  8%|▊         | 17952/225000 [46:08:12<419:47:07,  7.30s/it]
  8%|▊         | 17953/225000 [46:08:16<371:45:07,  6.46s/it]
  8%|▊         | 17954/225000 [46:08:23<368:18:35,  6.40s/it]
  8%|▊         | 17955/225000 [46:08:33<438:46:30,  7.63s/it]
  8%|▊         | 17956/225000 [46:08:38<386:17:39,  6.72s/it]
  8%|▊         | 17957/225000 [46:08:46<420:04:45,  7.30s/it]
  8%|▊         | 17958/225000 [46:08:52<385:12:48,  6.70s/it]
  8%|▊         | 17959/225000 [46:09:00<422:14:52,  7.34s/it]
  8%|▊         | 17960/225000 [46:09:04<359:23:19,  6.25s/it]
  8%|▊         | 17961/225000 [46:09:14<420:37:53,  7.31s/it]
  8%|▊         | 17962/225000 [46:09:18<359:51:52,  6.26s/it]
  8%|▊         | 17963/225000 [46:09:27<407:56:13,  7.09s/it]
  8%|▊         | 17964/225000 [46:09:32<374:37:50,  6.51s/it]
  8%|▊         | 17965/225000 [46:09:37<353:31:32,  6.15s/it]
  8%|▊         | 17966/225000 [46:09:42<328:04:27,  5.70s/it]
  8%|▊         | 17967/225000 [46:09:47<313:56:33,  5.46s/it]
  8%|▊         | 17968/225000 [46:09:52<305:13:28,  5.31s/it]
  8%|▊         | 17969/225000 [46:10:03<410:37:23,  7.14s/it]
  8%|▊         | 17970/225000 [46:10:11<426:56:44,  7.42s/it]
  8%|▊         | 17971/225000 [46:10:18<407:15:14,  7.08s/it]
  8%|▊         | 17972/225000 [46:10:22<354:18:48,  6.16s/it]
  8%|▊         | 17973/225000 [46:10:29<368:42:58,  6.41s/it]
  8%|▊         | 17974/225000 [46:10:33<341:47:28,  5.94s/it]
  8%|▊         | 17975/225000 [46:10:38<324:49:15,  5.65s/it]
  8%|▊         | 17976/225000 [46:10:43<315:24:18,  5.48s/it]
  8%|▊         | 17977/225000 [46:10:56<442:20:39,  7.69s/it]
  8%|▊         | 17978/225000 [46:11:00<377:51:40,  6.57s/it]
  8%|▊         | 17979/225000 [46:11:08<390:48:25,  6.80s/it]
  8%|▊         | 17980/225000 [46:11:16<420:12:36,  7.31s/it]
  8%|▊         | 17981/225000 [46:11:21<382:19:07,  6.65s/it]
  8%|▊         | 17982/225000 [46:11:26<343:48:47,  5.98s/it]
  8%|▊         | 17983/225000 [46:11:30<321:18:49,  5.59s/it]
  8%|▊         | 17984/225000 [46:11:42<428:49:33,  7.46s/it]
  8%|▊         | 17985/225000 [46:11:48<406:35:57,  7.07s/it]
  8%|▊         | 17986/225000 [46:11:56<417:43:35,  7.26s/it]
  8%|▊         | 17987/225000 [46:11:59<351:31:59,  6.11s/it]
  8%|▊         | 17988/225000 [46:12:07<378:06:00,  6.58s/it]
  8%|▊         | 17989/225000 [46:12:11<331:11:32,  5.76s/it]
  8%|▊         | 17990/225000 [46:12:19<376:10:58,  6.54s/it]
  8%|▊         | 17991/225000 [46:12:27<391:39:46,  6.81s/it]
  8%|▊         | 17992/225000 [46:12:32<358:59:40,  6.24s/it]
  8%|▊         | 17993/225000 [46:12:37<337:35:21,  5.87s/it]
  8%|▊         | 17994/225000 [46:12:42<328:04:35,  5.71s/it]
  8%|▊         | 17995/225000 [46:12:48<329:33:13,  5.73s/it]
  8%|▊         | 17996/225000 [46:12:53<317:43:42,  5.53s/it]
  8%|▊         | 17997/225000 [46:13:04<422:12:44,  7.34s/it]
  8%|▊         | 17998/225000 [46:13:13<447:24:29,  7.78s/it]
  8%|▊         | 17999/225000 [46:13:22<465:49:13,  8.10s/it]
  8%|▊         | 18000/225000 [46:13:26<402:59:53,  7.01s/it]
                                                             
{'loss': '0.7075', 'grad_norm': '0.5756', 'learning_rate': '4.983e-05', 'epoch': '0.08', 'num_input_tokens_seen': 81544216, 'train_runtime': '1.664e+05', 'train_tokens_per_second': '490'}

  8%|▊         | 18000/225000 [46:13:26<402:59:53,  7.01s/it]
  8%|▊         | 18001/225000 [46:13:38<476:12:42,  8.28s/it]
  8%|▊         | 18002/225000 [46:13:43<429:01:46,  7.46s/it]
  8%|▊         | 18003/225000 [46:13:49<396:45:42,  6.90s/it]
  8%|▊         | 18004/225000 [46:13:54<365:56:33,  6.36s/it]
  8%|▊         | 18005/225000 [46:13:59<341:56:45,  5.95s/it]
  8%|▊         | 18006/225000 [46:14:09<415:48:56,  7.23s/it]
  8%|▊         | 18007/225000 [46:14:19<466:00:06,  8.10s/it]
  8%|▊         | 18008/225000 [46:14:23<390:38:30,  6.79s/it]
  8%|▊         | 18009/225000 [46:14:31<410:36:48,  7.14s/it]
  8%|▊         | 18010/225000 [46:14:37<385:37:22,  6.71s/it]
  8%|▊         | 18011/225000 [46:14:42<357:34:19,  6.22s/it]
  8%|▊         | 18012/225000 [46:14:47<341:41:28,  5.94s/it]
  8%|▊         | 18013/225000 [46:14:52<322:32:54,  5.61s/it]
  8%|▊         | 18014/225000 [46:14:58<323:48:27,  5.63s/it]
  8%|▊         | 18015/225000 [46:15:03<316:08:34,  5.50s/it]
  8%|▊         | 18016/225000 [46:15:16<441:00:16,  7.67s/it]
  8%|▊         | 18017/225000 [46:15:21<394:56:29,  6.87s/it]
  8%|▊         | 18018/225000 [46:15:26<371:03:51,  6.45s/it]
  8%|▊         | 18019/225000 [46:15:32<360:47:42,  6.28s/it]
  8%|▊         | 18020/225000 [46:15:38<359:02:38,  6.24s/it]
  8%|▊         | 18021/225000 [46:15:44<349:12:23,  6.07s/it]
  8%|▊         | 18022/225000 [46:15:52<390:01:18,  6.78s/it]
  8%|▊         | 18023/225000 [46:16:05<487:28:19,  8.48s/it]
  8%|▊         | 18024/225000 [46:16:10<432:31:51,  7.52s/it]
  8%|▊         | 18025/225000 [46:16:15<391:41:59,  6.81s/it]
  8%|▊         | 18026/225000 [46:16:20<364:13:37,  6.34s/it]
  8%|▊         | 18027/225000 [46:16:25<342:34:30,  5.96s/it]
  8%|▊         | 18028/225000 [46:16:38<455:21:08,  7.92s/it]
  8%|▊         | 18029/225000 [46:16:42<397:08:22,  6.91s/it]
  8%|▊         | 18030/225000 [46:16:51<428:59:26,  7.46s/it]
  8%|▊         | 18031/225000 [46:16:56<389:06:52,  6.77s/it]
  8%|▊         | 18032/225000 [46:17:06<444:39:35,  7.73s/it]
  8%|▊         | 18033/225000 [46:17:12<418:15:05,  7.28s/it]
  8%|▊         | 18034/225000 [46:17:21<439:42:44,  7.65s/it]
  8%|▊         | 18035/225000 [46:17:25<380:38:43,  6.62s/it]
  8%|▊         | 18036/225000 [46:17:30<352:50:15,  6.14s/it]
  8%|▊         | 18037/225000 [46:17:35<334:33:18,  5.82s/it]
  8%|▊         | 18038/225000 [46:17:48<456:03:19,  7.93s/it]
  8%|▊         | 18039/225000 [46:17:52<384:09:17,  6.68s/it]
  8%|▊         | 18040/225000 [46:18:00<404:53:21,  7.04s/it]
  8%|▊         | 18041/225000 [46:18:04<356:42:05,  6.20s/it]
  8%|▊         | 18042/225000 [46:18:11<361:10:43,  6.28s/it]
  8%|▊         | 18043/225000 [46:18:19<405:55:52,  7.06s/it]
  8%|▊         | 18044/225000 [46:18:28<439:27:34,  7.64s/it]
  8%|▊         | 18045/225000 [46:18:33<386:58:27,  6.73s/it]
  8%|▊         | 18046/225000 [46:18:38<350:49:41,  6.10s/it]
  8%|▊         | 18047/225000 [46:18:48<426:07:12,  7.41s/it]
  8%|▊         | 18048/225000 [46:18:54<394:41:32,  6.87s/it]
  8%|▊         | 18049/225000 [46:18:59<358:45:26,  6.24s/it]
  8%|▊         | 18050/225000 [46:19:07<390:18:53,  6.79s/it]
                                                             
{'loss': '0.7072', 'grad_norm': '0.5882', 'learning_rate': '4.983e-05', 'epoch': '0.08022', 'num_input_tokens_seen': 81696672, 'train_runtime': '1.667e+05', 'train_tokens_per_second': '489.9'}

  8%|▊         | 18050/225000 [46:19:07<390:18:53,  6.79s/it]
  8%|▊         | 18051/225000 [46:19:11<352:46:51,  6.14s/it]
  8%|▊         | 18052/225000 [46:19:20<406:59:24,  7.08s/it]
  8%|▊         | 18053/225000 [46:19:25<361:29:20,  6.29s/it]
  8%|▊         | 18054/225000 [46:19:29<329:18:23,  5.73s/it]
  8%|▊         | 18055/225000 [46:19:34<312:24:09,  5.43s/it]
  8%|▊         | 18056/225000 [46:19:41<344:22:03,  5.99s/it]
  8%|▊         | 18057/225000 [46:19:52<417:09:15,  7.26s/it]
  8%|▊         | 18058/225000 [46:20:00<439:36:28,  7.65s/it]
  8%|▊         | 18059/225000 [46:20:05<386:07:55,  6.72s/it]
  8%|▊         | 18060/225000 [46:20:09<348:12:04,  6.06s/it]
  8%|▊         | 18061/225000 [46:20:16<363:44:24,  6.33s/it]
  8%|▊         | 18062/225000 [46:20:23<378:15:00,  6.58s/it]
  8%|▊         | 18063/225000 [46:20:32<410:08:50,  7.14s/it]
  8%|▊         | 18064/225000 [46:20:36<366:03:47,  6.37s/it]
  8%|▊         | 18065/225000 [46:20:41<339:19:30,  5.90s/it]
  8%|▊         | 18066/225000 [46:20:54<459:18:49,  7.99s/it]
  8%|▊         | 18067/225000 [46:21:01<442:02:14,  7.69s/it]
  8%|▊         | 18068/225000 [46:21:09<443:32:15,  7.72s/it]
  8%|▊         | 18069/225000 [46:21:16<435:34:10,  7.58s/it]
  8%|▊         | 18070/225000 [46:21:22<410:56:34,  7.15s/it]
  8%|▊         | 18071/225000 [46:21:26<357:37:43,  6.22s/it]
  8%|▊         | 18072/225000 [46:21:37<439:01:47,  7.64s/it]
  8%|▊         | 18073/225000 [46:21:42<386:25:34,  6.72s/it]
  8%|▊         | 18074/225000 [46:21:46<343:09:44,  5.97s/it]
  8%|▊         | 18075/225000 [46:21:56<416:35:18,  7.25s/it]
  8%|▊         | 18076/225000 [46:22:03<400:04:23,  6.96s/it]
  8%|▊         | 18077/225000 [46:22:07<355:45:40,  6.19s/it]
  8%|▊         | 18078/225000 [46:22:18<432:20:55,  7.52s/it]
  8%|▊         | 18079/225000 [46:22:22<381:25:32,  6.64s/it]
  8%|▊         | 18080/225000 [46:22:27<357:04:04,  6.21s/it]
  8%|▊         | 18081/225000 [46:22:33<347:37:16,  6.05s/it]
  8%|▊         | 18082/225000 [46:22:44<430:12:36,  7.48s/it]
  8%|▊         | 18083/225000 [46:22:50<414:04:30,  7.20s/it]
  8%|▊         | 18084/225000 [46:22:55<363:06:14,  6.32s/it]
  8%|▊         | 18085/225000 [46:23:01<359:24:37,  6.25s/it]
  8%|▊         | 18086/225000 [46:23:12<445:03:02,  7.74s/it]
  8%|▊         | 18087/225000 [46:23:17<402:45:59,  7.01s/it]
  8%|▊         | 18088/225000 [46:23:23<384:35:54,  6.69s/it]
  8%|▊         | 18089/225000 [46:23:28<351:23:43,  6.11s/it]
  8%|▊         | 18090/225000 [46:23:34<344:16:58,  5.99s/it]
  8%|▊         | 18091/225000 [46:23:41<367:54:19,  6.40s/it]
  8%|▊         | 18092/225000 [46:23:51<434:14:02,  7.56s/it]
  8%|▊         | 18093/225000 [46:23:56<381:13:52,  6.63s/it]
  8%|▊         | 18094/225000 [46:24:03<394:11:16,  6.86s/it]
  8%|▊         | 18095/225000 [46:24:14<461:45:03,  8.03s/it]
  8%|▊         | 18096/225000 [46:24:17<383:29:06,  6.67s/it]
  8%|▊         | 18097/225000 [46:24:26<419:24:18,  7.30s/it]
  8%|▊         | 18098/225000 [46:24:30<363:53:54,  6.33s/it]
  8%|▊         | 18099/225000 [46:24:39<398:22:40,  6.93s/it]
  8%|▊         | 18100/225000 [46:24:42<342:49:51,  5.97s/it]
                                                             
{'loss': '0.7075', 'grad_norm': '0.6637', 'learning_rate': '4.983e-05', 'epoch': '0.08044', 'num_input_tokens_seen': 81849304, 'train_runtime': '1.671e+05', 'train_tokens_per_second': '489.9'}

  8%|▊         | 18100/225000 [46:24:42<342:49:51,  5.97s/it]
  8%|▊         | 18101/225000 [46:24:52<406:33:40,  7.07s/it]
  8%|▊         | 18102/225000 [46:24:58<390:09:19,  6.79s/it]
  8%|▊         | 18103/225000 [46:25:03<358:54:09,  6.24s/it]
  8%|▊         | 18104/225000 [46:25:08<341:21:00,  5.94s/it]
  8%|▊         | 18105/225000 [46:25:13<317:38:57,  5.53s/it]
  8%|▊         | 18106/225000 [46:25:26<443:17:45,  7.71s/it]
  8%|▊         | 18107/225000 [46:25:32<418:40:36,  7.29s/it]
  8%|▊         | 18108/225000 [46:25:39<418:50:05,  7.29s/it]
  8%|▊         | 18109/225000 [46:25:45<392:39:30,  6.83s/it]
  8%|▊         | 18110/225000 [46:25:50<363:33:15,  6.33s/it]
  8%|▊         | 18111/225000 [46:25:56<356:57:47,  6.21s/it]
  8%|▊         | 18112/225000 [46:26:02<349:52:33,  6.09s/it]
  8%|▊         | 18113/225000 [46:26:07<339:21:30,  5.91s/it]
  8%|▊         | 18114/225000 [46:26:13<333:14:10,  5.80s/it]
  8%|▊         | 18115/225000 [46:26:18<326:37:07,  5.68s/it]
  8%|▊         | 18116/225000 [46:26:32<459:23:16,  7.99s/it]
  8%|▊         | 18117/225000 [46:26:39<445:29:19,  7.75s/it]
  8%|▊         | 18118/225000 [46:26:44<400:03:16,  6.96s/it]
  8%|▊         | 18119/225000 [46:26:49<364:14:42,  6.34s/it]
  8%|▊         | 18120/225000 [46:26:57<388:09:52,  6.75s/it]
  8%|▊         | 18121/225000 [46:27:05<422:00:20,  7.34s/it]
  8%|▊         | 18122/225000 [46:27:10<371:29:53,  6.46s/it]
  8%|▊         | 18123/225000 [46:27:21<448:13:19,  7.80s/it]
  8%|▊         | 18124/225000 [46:27:24<370:57:34,  6.46s/it]
  8%|▊         | 18125/225000 [46:27:33<422:48:05,  7.36s/it]
  8%|▊         | 18126/225000 [46:27:37<363:49:33,  6.33s/it]
  8%|▊         | 18127/225000 [46:27:46<410:58:57,  7.15s/it]
  8%|▊         | 18128/225000 [46:27:51<365:05:07,  6.35s/it]
  8%|▊         | 18129/225000 [46:27:59<390:34:54,  6.80s/it]
  8%|▊         | 18130/225000 [46:28:05<372:44:17,  6.49s/it]
  8%|▊         | 18131/225000 [46:28:13<414:47:13,  7.22s/it]
  8%|▊         | 18132/225000 [46:28:18<372:32:17,  6.48s/it]
  8%|▊         | 18133/225000 [46:28:23<348:43:32,  6.07s/it]
  8%|▊         | 18134/225000 [46:28:28<326:55:05,  5.69s/it]
  8%|▊         | 18135/225000 [46:28:33<317:47:03,  5.53s/it]
  8%|▊         | 18136/225000 [46:28:47<457:32:45,  7.96s/it]
  8%|▊         | 18137/225000 [46:28:52<400:22:54,  6.97s/it]
  8%|▊         | 18138/225000 [46:29:01<443:06:33,  7.71s/it]
  8%|▊         | 18139/225000 [46:29:06<399:11:59,  6.95s/it]
  8%|▊         | 18140/225000 [46:29:11<364:44:05,  6.35s/it]
  8%|▊         | 18141/225000 [46:29:18<378:23:28,  6.59s/it]
  8%|▊         | 18142/225000 [46:29:24<365:54:57,  6.37s/it]
  8%|▊         | 18143/225000 [46:29:31<380:24:47,  6.62s/it]
  8%|▊         | 18144/225000 [46:29:41<425:55:16,  7.41s/it]
  8%|▊         | 18145/225000 [46:29:48<433:52:19,  7.55s/it]
  8%|▊         | 18146/225000 [46:29:53<383:22:50,  6.67s/it]
  8%|▊         | 18147/225000 [46:29:58<353:36:35,  6.15s/it]
  8%|▊         | 18148/225000 [46:30:03<324:25:12,  5.65s/it]
  8%|▊         | 18149/225000 [46:30:07<304:57:24,  5.31s/it]
  8%|▊         | 18150/225000 [46:30:13<309:50:51,  5.39s/it]
                                                             
{'loss': '0.723', 'grad_norm': '0.6068', 'learning_rate': '4.982e-05', 'epoch': '0.08067', 'num_input_tokens_seen': 82001592, 'train_runtime': '1.674e+05', 'train_tokens_per_second': '489.8'}

  8%|▊         | 18150/225000 [46:30:13<309:50:51,  5.39s/it]
  8%|▊         | 18151/225000 [46:30:18<317:58:47,  5.53s/it]
  8%|▊         | 18152/225000 [46:30:32<452:30:56,  7.88s/it]
  8%|▊         | 18153/225000 [46:30:38<429:29:46,  7.48s/it]
  8%|▊         | 18154/225000 [46:30:45<413:54:55,  7.20s/it]
  8%|▊         | 18155/225000 [46:30:50<370:04:40,  6.44s/it]
  8%|▊         | 18156/225000 [46:30:54<341:31:05,  5.94s/it]
  8%|▊         | 18157/225000 [46:31:02<374:08:52,  6.51s/it]
  8%|▊         | 18158/225000 [46:31:15<473:31:56,  8.24s/it]
  8%|▊         | 18159/225000 [46:31:20<423:42:07,  7.37s/it]
  8%|▊         | 18160/225000 [46:31:27<426:53:10,  7.43s/it]
  8%|▊         | 18161/225000 [46:31:33<388:50:22,  6.77s/it]
  8%|▊         | 18162/225000 [46:31:40<397:40:31,  6.92s/it]
  8%|▊         | 18163/225000 [46:31:48<411:28:17,  7.16s/it]
  8%|▊         | 18164/225000 [46:31:55<414:04:08,  7.21s/it]
  8%|▊         | 18165/225000 [46:32:01<390:37:49,  6.80s/it]
  8%|▊         | 18166/225000 [46:32:09<406:39:00,  7.08s/it]
  8%|▊         | 18167/225000 [46:32:16<412:35:04,  7.18s/it]
  8%|▊         | 18168/225000 [46:32:20<356:15:32,  6.20s/it]
  8%|▊         | 18169/225000 [46:32:29<399:18:48,  6.95s/it]
  8%|▊         | 18170/225000 [46:32:32<343:46:46,  5.98s/it]
  8%|▊         | 18171/225000 [46:32:41<389:25:45,  6.78s/it]
  8%|▊         | 18172/225000 [46:32:45<334:23:32,  5.82s/it]
  8%|▊         | 18173/225000 [46:32:54<390:49:57,  6.80s/it]
  8%|▊         | 18174/225000 [46:33:00<379:06:05,  6.60s/it]
  8%|▊         | 18175/225000 [46:33:10<434:04:23,  7.56s/it]
  8%|▊         | 18176/225000 [46:33:16<407:27:36,  7.09s/it]
  8%|▊         | 18177/225000 [46:33:21<380:02:39,  6.62s/it]
  8%|▊         | 18178/225000 [46:33:27<361:32:18,  6.29s/it]
  8%|▊         | 18179/225000 [46:33:40<482:18:37,  8.40s/it]
  8%|▊         | 18180/225000 [46:33:44<402:38:02,  7.01s/it]
  8%|▊         | 18181/225000 [46:33:48<356:38:42,  6.21s/it]
  8%|▊         | 18182/225000 [46:33:54<348:03:54,  6.06s/it]
  8%|▊         | 18183/225000 [46:34:03<403:08:57,  7.02s/it]
  8%|▊         | 18184/225000 [46:34:13<450:07:57,  7.84s/it]
  8%|▊         | 18185/225000 [46:34:17<394:27:31,  6.87s/it]
  8%|▊         | 18186/225000 [46:34:23<367:33:46,  6.40s/it]
  8%|▊         | 18187/225000 [46:34:28<351:30:09,  6.12s/it]
  8%|▊         | 18188/225000 [46:34:34<343:35:52,  5.98s/it]
  8%|▊         | 18189/225000 [46:34:40<343:04:37,  5.97s/it]
  8%|▊         | 18190/225000 [46:34:45<326:19:19,  5.68s/it]
  8%|▊         | 18191/225000 [46:34:54<391:55:03,  6.82s/it]
  8%|▊         | 18192/225000 [46:35:06<472:18:45,  8.22s/it]
  8%|▊         | 18193/225000 [46:35:10<402:21:50,  7.00s/it]
  8%|▊         | 18194/225000 [46:35:16<393:39:09,  6.85s/it]
  8%|▊         | 18195/225000 [46:35:25<418:46:29,  7.29s/it]
  8%|▊         | 18196/225000 [46:35:30<384:20:09,  6.69s/it]
  8%|▊         | 18197/225000 [46:35:38<399:44:33,  6.96s/it]
  8%|▊         | 18198/225000 [46:35:46<421:38:37,  7.34s/it]
  8%|▊         | 18199/225000 [46:35:51<380:19:29,  6.62s/it]
  8%|▊         | 18200/225000 [46:35:56<360:21:14,  6.27s/it]
                                                             
{'loss': '0.7092', 'grad_norm': '0.6052', 'learning_rate': '4.982e-05', 'epoch': '0.08089', 'num_input_tokens_seen': 82154904, 'train_runtime': '1.678e+05', 'train_tokens_per_second': '489.7'}

  8%|▊         | 18200/225000 [46:35:56<360:21:14,  6.27s/it]
  8%|▊         | 18201/225000 [46:36:02<355:39:13,  6.19s/it]
  8%|▊         | 18202/225000 [46:36:08<343:00:37,  5.97s/it]
  8%|▊         | 18203/225000 [46:36:19<443:18:18,  7.72s/it]
  8%|▊         | 18204/225000 [46:36:24<382:40:52,  6.66s/it]
  8%|▊         | 18205/225000 [46:36:32<420:33:17,  7.32s/it]
  8%|▊         | 18206/225000 [46:36:40<420:04:29,  7.31s/it]
  8%|▊         | 18207/225000 [46:36:45<384:24:23,  6.69s/it]
  8%|▊         | 18208/225000 [46:36:49<346:06:35,  6.03s/it]
  8%|▊         | 18209/225000 [46:37:01<444:20:51,  7.74s/it]
  8%|▊         | 18210/225000 [46:37:06<388:53:57,  6.77s/it]
  8%|▊         | 18211/225000 [46:37:13<404:25:05,  7.04s/it]
  8%|▊         | 18212/225000 [46:37:19<374:31:55,  6.52s/it]
  8%|▊         | 18213/225000 [46:37:24<352:17:47,  6.13s/it]
  8%|▊         | 18214/225000 [46:37:29<340:57:48,  5.94s/it]
  8%|▊         | 18215/225000 [46:37:35<327:33:14,  5.70s/it]
  8%|▊         | 18216/225000 [46:37:43<377:38:05,  6.57s/it]
  8%|▊         | 18217/225000 [46:37:55<465:23:31,  8.10s/it]
  8%|▊         | 18218/225000 [46:38:02<445:41:55,  7.76s/it]
  8%|▊         | 18219/225000 [46:38:06<386:17:11,  6.73s/it]
  8%|▊         | 18220/225000 [46:38:10<343:00:41,  5.97s/it]
  8%|▊         | 18221/225000 [46:38:21<427:01:49,  7.43s/it]
  8%|▊         | 18222/225000 [46:38:25<362:03:35,  6.30s/it]
  8%|▊         | 18223/225000 [46:38:29<326:06:24,  5.68s/it]
  8%|▊         | 18224/225000 [46:38:40<415:52:39,  7.24s/it]
  8%|▊         | 18225/225000 [46:38:46<403:51:18,  7.03s/it]
  8%|▊         | 18226/225000 [46:38:51<357:31:27,  6.22s/it]
  8%|▊         | 18227/225000 [46:38:55<324:43:04,  5.65s/it]
  8%|▊         | 18228/225000 [46:39:10<483:28:24,  8.42s/it]
  8%|▊         | 18229/225000 [46:39:13<398:49:49,  6.94s/it]
  8%|▊         | 18230/225000 [46:39:25<472:50:19,  8.23s/it]
  8%|▊         | 18231/225000 [46:39:29<405:55:27,  7.07s/it]
  8%|▊         | 18232/225000 [46:39:34<367:46:19,  6.40s/it]
  8%|▊         | 18233/225000 [46:39:43<422:10:24,  7.35s/it]
  8%|▊         | 18234/225000 [46:39:50<415:40:40,  7.24s/it]
  8%|▊         | 18235/225000 [46:39:57<403:18:59,  7.02s/it]
  8%|▊         | 18236/225000 [46:40:05<415:37:11,  7.24s/it]
  8%|▊         | 18237/225000 [46:40:10<383:36:26,  6.68s/it]
  8%|▊         | 18238/225000 [46:40:17<394:04:37,  6.86s/it]
  8%|▊         | 18239/225000 [46:40:22<349:25:55,  6.08s/it]
  8%|▊         | 18240/225000 [46:40:27<332:32:13,  5.79s/it]
  8%|▊         | 18241/225000 [46:40:32<320:37:13,  5.58s/it]
  8%|▊         | 18242/225000 [46:40:37<316:09:00,  5.50s/it]
  8%|▊         | 18243/225000 [46:40:42<311:52:10,  5.43s/it]
  8%|▊         | 18244/225000 [46:40:50<341:55:12,  5.95s/it]
  8%|▊         | 18245/225000 [46:41:03<472:20:33,  8.22s/it]
  8%|▊         | 18246/225000 [46:41:07<402:27:48,  7.01s/it]
  8%|▊         | 18247/225000 [46:41:11<350:55:48,  6.11s/it]
  8%|▊         | 18248/225000 [46:41:20<395:53:52,  6.89s/it]
  8%|▊         | 18249/225000 [46:41:30<455:37:58,  7.93s/it]
  8%|▊         | 18250/225000 [46:41:35<401:02:53,  6.98s/it]
                                                             
{'loss': '0.7087', 'grad_norm': '0.5475', 'learning_rate': '4.982e-05', 'epoch': '0.08111', 'num_input_tokens_seen': 82307920, 'train_runtime': '1.681e+05', 'train_tokens_per_second': '489.6'}

  8%|▊         | 18250/225000 [46:41:35<401:02:53,  6.98s/it]
  8%|▊         | 18251/225000 [46:41:40<369:58:41,  6.44s/it]
  8%|▊         | 18252/225000 [46:41:46<348:08:58,  6.06s/it]
  8%|▊         | 18253/225000 [46:41:56<418:02:12,  7.28s/it]
  8%|▊         | 18254/225000 [46:42:05<451:59:54,  7.87s/it]
  8%|▊         | 18255/225000 [46:42:12<439:16:38,  7.65s/it]
  8%|▊         | 18256/225000 [46:42:17<397:56:43,  6.93s/it]
  8%|▊         | 18257/225000 [46:42:25<413:48:23,  7.21s/it]
  8%|▊         | 18258/225000 [46:42:29<364:41:49,  6.35s/it]
  8%|▊         | 18259/225000 [46:42:34<338:31:31,  5.89s/it]
  8%|▊         | 18260/225000 [46:42:45<412:48:58,  7.19s/it]
  8%|▊         | 18261/225000 [46:42:49<370:53:19,  6.46s/it]
  8%|▊         | 18262/225000 [46:42:57<392:15:31,  6.83s/it]
  8%|▊         | 18263/225000 [46:43:01<349:42:51,  6.09s/it]
  8%|▊         | 18264/225000 [46:43:12<428:38:22,  7.46s/it]
  8%|▊         | 18265/225000 [46:43:17<384:36:43,  6.70s/it]
  8%|▊         | 18266/225000 [46:43:25<404:01:20,  7.04s/it]
  8%|▊         | 18267/225000 [46:43:32<401:34:20,  6.99s/it]
  8%|▊         | 18268/225000 [46:43:36<351:48:15,  6.13s/it]
  8%|▊         | 18269/225000 [46:43:44<396:06:48,  6.90s/it]
  8%|▊         | 18270/225000 [46:43:51<396:28:23,  6.90s/it]
  8%|▊         | 18271/225000 [46:43:56<364:26:16,  6.35s/it]
  8%|▊         | 18272/225000 [46:44:01<342:21:34,  5.96s/it]
  8%|▊         | 18273/225000 [46:44:06<321:58:06,  5.61s/it]
  8%|▊         | 18274/225000 [46:44:11<308:12:30,  5.37s/it]
  8%|▊         | 18275/225000 [46:44:22<411:44:11,  7.17s/it]
  8%|▊         | 18276/225000 [46:44:29<406:27:28,  7.08s/it]
  8%|▊         | 18277/225000 [46:44:35<383:06:24,  6.67s/it]
  8%|▊         | 18278/225000 [46:44:40<351:46:17,  6.13s/it]
  8%|▊         | 18279/225000 [46:44:46<344:06:20,  5.99s/it]
  8%|▊         | 18280/225000 [46:44:51<340:52:26,  5.94s/it]
  8%|▊         | 18281/225000 [46:44:57<341:00:13,  5.94s/it]
  8%|▊         | 18282/225000 [46:45:03<329:51:20,  5.74s/it]
  8%|▊         | 18283/225000 [46:45:15<450:52:54,  7.85s/it]
  8%|▊         | 18284/225000 [46:45:22<428:23:45,  7.46s/it]
  8%|▊         | 18285/225000 [46:45:30<433:23:53,  7.55s/it]
  8%|▊         | 18286/225000 [46:45:35<387:20:40,  6.75s/it]
  8%|▊         | 18287/225000 [46:45:42<405:17:21,  7.06s/it]
  8%|▊         | 18288/225000 [46:45:46<351:41:16,  6.12s/it]
  8%|▊         | 18289/225000 [46:45:56<414:30:56,  7.22s/it]
  8%|▊         | 18290/225000 [46:46:02<384:33:40,  6.70s/it]
  8%|▊         | 18291/225000 [46:46:07<357:25:38,  6.22s/it]
  8%|▊         | 18292/225000 [46:46:12<347:00:12,  6.04s/it]
  8%|▊         | 18293/225000 [46:46:18<337:03:23,  5.87s/it]
  8%|▊         | 18294/225000 [46:46:23<319:51:03,  5.57s/it]
  8%|▊         | 18295/225000 [46:46:35<439:06:44,  7.65s/it]
  8%|▊         | 18296/225000 [46:46:40<394:11:53,  6.87s/it]
  8%|▊         | 18297/225000 [46:46:45<362:51:28,  6.32s/it]
  8%|▊         | 18298/225000 [46:46:50<341:32:24,  5.95s/it]
  8%|▊         | 18299/225000 [46:46:56<331:37:49,  5.78s/it]
  8%|▊         | 18300/225000 [46:47:06<409:56:39,  7.14s/it]
                                                             
{'loss': '0.7085', 'grad_norm': '0.6888', 'learning_rate': '4.982e-05', 'epoch': '0.08133', 'num_input_tokens_seen': 82458160, 'train_runtime': '1.684e+05', 'train_tokens_per_second': '489.6'}

  8%|▊         | 18300/225000 [46:47:06<409:56:39,  7.14s/it]
  8%|▊         | 18301/225000 [46:47:17<481:04:13,  8.38s/it]
  8%|▊         | 18302/225000 [46:47:22<419:55:46,  7.31s/it]
  8%|▊         | 18303/225000 [46:47:28<397:53:35,  6.93s/it]
  8%|▊         | 18304/225000 [46:47:38<442:48:27,  7.71s/it]
  8%|▊         | 18305/225000 [46:47:43<400:33:15,  6.98s/it]
  8%|▊         | 18306/225000 [46:47:48<360:02:06,  6.27s/it]
  8%|▊         | 18307/225000 [46:47:52<336:43:25,  5.86s/it]
  8%|▊         | 18308/225000 [46:47:58<336:05:14,  5.85s/it]
  8%|▊         | 18309/225000 [46:48:11<452:20:22,  7.88s/it]
  8%|▊         | 18310/225000 [46:48:15<386:43:20,  6.74s/it]
  8%|▊         | 18311/225000 [46:48:20<360:21:27,  6.28s/it]
  8%|▊         | 18312/225000 [46:48:31<439:17:43,  7.65s/it]
  8%|▊         | 18313/225000 [46:48:36<399:44:27,  6.96s/it]
  8%|▊         | 18314/225000 [46:48:43<395:40:24,  6.89s/it]
  8%|▊         | 18315/225000 [46:48:47<345:43:59,  6.02s/it]
  8%|▊         | 18316/225000 [46:48:59<448:20:24,  7.81s/it]
  8%|▊         | 18317/225000 [46:49:03<380:17:42,  6.62s/it]
  8%|▊         | 18318/225000 [46:49:09<376:12:00,  6.55s/it]
  8%|▊         | 18319/225000 [46:49:18<406:02:18,  7.07s/it]
  8%|▊         | 18320/225000 [46:49:21<350:29:45,  6.11s/it]
  8%|▊         | 18321/225000 [46:49:31<415:05:59,  7.23s/it]
  8%|▊         | 18322/225000 [46:49:35<351:14:15,  6.12s/it]
  8%|▊         | 18323/225000 [46:49:45<414:03:34,  7.21s/it]
  8%|▊         | 18324/225000 [46:49:49<357:57:48,  6.24s/it]
  8%|▊         | 18325/225000 [46:49:53<332:06:07,  5.78s/it]
  8%|▊         | 18326/225000 [46:50:02<381:22:07,  6.64s/it]
  8%|▊         | 18327/225000 [46:50:10<407:34:32,  7.10s/it]
  8%|▊         | 18328/225000 [46:50:19<436:49:37,  7.61s/it]
  8%|▊         | 18329/225000 [46:50:23<379:24:22,  6.61s/it]
  8%|▊         | 18330/225000 [46:50:27<339:20:19,  5.91s/it]
  8%|▊         | 18331/225000 [46:50:39<428:48:00,  7.47s/it]
  8%|▊         | 18332/225000 [46:50:43<379:00:35,  6.60s/it]
  8%|▊         | 18333/225000 [46:50:49<358:51:05,  6.25s/it]
  8%|▊         | 18334/225000 [46:50:54<348:55:26,  6.08s/it]
  8%|▊         | 18335/225000 [46:51:00<338:24:18,  5.89s/it]
  8%|▊         | 18336/225000 [46:51:05<331:31:39,  5.78s/it]
  8%|▊         | 18337/225000 [46:51:11<331:48:18,  5.78s/it]
  8%|▊         | 18338/225000 [46:51:16<320:23:59,  5.58s/it]
  8%|▊         | 18339/225000 [46:51:28<434:43:39,  7.57s/it]
  8%|▊         | 18340/225000 [46:51:38<478:41:41,  8.34s/it]
  8%|▊         | 18341/225000 [46:51:43<419:30:22,  7.31s/it]
  8%|▊         | 18342/225000 [46:51:51<432:51:29,  7.54s/it]
  8%|▊         | 18343/225000 [46:51:57<403:34:54,  7.03s/it]
  8%|▊         | 18344/225000 [46:52:05<410:10:12,  7.15s/it]
  8%|▊         | 18345/225000 [46:52:09<367:21:46,  6.40s/it]
  8%|▊         | 18346/225000 [46:52:15<350:48:26,  6.11s/it]
  8%|▊         | 18347/225000 [46:52:20<343:22:24,  5.98s/it]
  8%|▊         | 18348/225000 [46:52:26<337:36:09,  5.88s/it]
  8%|▊         | 18349/225000 [46:52:32<334:31:53,  5.83s/it]
  8%|▊         | 18350/225000 [46:52:37<329:24:43,  5.74s/it]
                                                             
{'loss': '0.734', 'grad_norm': '0.6402', 'learning_rate': '4.981e-05', 'epoch': '0.08156', 'num_input_tokens_seen': 82609608, 'train_runtime': '1.688e+05', 'train_tokens_per_second': '489.5'}

  8%|▊         | 18350/225000 [46:52:37<329:24:43,  5.74s/it]
  8%|▊         | 18351/225000 [46:52:50<456:18:20,  7.95s/it]
  8%|▊         | 18352/225000 [46:52:58<448:52:31,  7.82s/it]
  8%|▊         | 18353/225000 [46:53:06<449:44:14,  7.83s/it]
  8%|▊         | 18354/225000 [46:53:12<415:34:34,  7.24s/it]
  8%|▊         | 18355/225000 [46:53:15<354:11:28,  6.17s/it]
  8%|▊         | 18356/225000 [46:53:25<415:59:44,  7.25s/it]
  8%|▊         | 18357/225000 [46:53:30<380:16:00,  6.62s/it]
  8%|▊         | 18358/225000 [46:53:36<360:15:19,  6.28s/it]
  8%|▊         | 18359/225000 [46:53:41<347:16:12,  6.05s/it]
  8%|▊         | 18360/225000 [46:53:46<329:32:10,  5.74s/it]
  8%|▊         | 18361/225000 [46:53:52<324:55:23,  5.66s/it]
  8%|▊         | 18362/225000 [46:54:03<413:52:35,  7.21s/it]
  8%|▊         | 18363/225000 [46:54:13<464:03:01,  8.08s/it]
  8%|▊         | 18364/225000 [46:54:17<397:08:07,  6.92s/it]
  8%|▊         | 18365/225000 [46:54:25<414:10:07,  7.22s/it]
  8%|▊         | 18366/225000 [46:54:31<393:33:52,  6.86s/it]
  8%|▊         | 18367/225000 [46:54:40<429:55:31,  7.49s/it]
  8%|▊         | 18368/225000 [46:54:46<408:25:26,  7.12s/it]
  8%|▊         | 18369/225000 [46:54:52<387:05:02,  6.74s/it]
  8%|▊         | 18370/225000 [46:55:00<401:31:30,  7.00s/it]
  8%|▊         | 18371/225000 [46:55:09<441:53:06,  7.70s/it]
  8%|▊         | 18372/225000 [46:55:14<396:24:50,  6.91s/it]
  8%|▊         | 18373/225000 [46:55:19<360:27:12,  6.28s/it]
  8%|▊         | 18374/225000 [46:55:24<342:18:11,  5.96s/it]
  8%|▊         | 18375/225000 [46:55:36<445:44:36,  7.77s/it]
  8%|▊         | 18376/225000 [46:55:40<378:44:31,  6.60s/it]
  8%|▊         | 18377/225000 [46:55:49<417:22:44,  7.27s/it]
  8%|▊         | 18378/225000 [46:55:53<370:20:42,  6.45s/it]
  8%|▊         | 18379/225000 [46:55:57<328:30:54,  5.72s/it]
  8%|▊         | 18380/225000 [46:56:08<419:20:10,  7.31s/it]
  8%|▊         | 18381/225000 [46:56:19<487:06:49,  8.49s/it]
  8%|▊         | 18382/225000 [46:56:23<400:38:05,  6.98s/it]
  8%|▊         | 18383/225000 [46:56:31<424:30:02,  7.40s/it]
  8%|▊         | 18384/225000 [46:56:39<429:46:38,  7.49s/it]
  8%|▊         | 18385/225000 [46:56:43<369:28:51,  6.44s/it]
  8%|▊         | 18386/225000 [46:56:51<401:32:59,  7.00s/it]
  8%|▊         | 18387/225000 [46:56:55<347:48:12,  6.06s/it]
  8%|▊         | 18388/225000 [46:57:04<396:31:54,  6.91s/it]
  8%|▊         | 18389/225000 [46:57:09<358:23:37,  6.24s/it]
  8%|▊         | 18390/225000 [46:57:17<390:08:54,  6.80s/it]
  8%|▊         | 18391/225000 [46:57:22<364:35:15,  6.35s/it]
  8%|▊         | 18392/225000 [46:57:33<433:53:54,  7.56s/it]
  8%|▊         | 18393/225000 [46:57:38<394:27:46,  6.87s/it]
  8%|▊         | 18394/225000 [46:57:43<371:59:19,  6.48s/it]
  8%|▊         | 18395/225000 [46:57:52<406:37:28,  7.09s/it]
  8%|▊         | 18396/225000 [46:57:59<413:24:48,  7.20s/it]
  8%|▊         | 18397/225000 [46:58:03<355:19:32,  6.19s/it]
  8%|▊         | 18398/225000 [46:58:12<399:27:43,  6.96s/it]
  8%|▊         | 18399/225000 [46:58:16<358:04:21,  6.24s/it]
  8%|▊         | 18400/225000 [46:58:21<333:30:56,  5.81s/it]
                                                             
{'loss': '0.7315', 'grad_norm': '0.6604', 'learning_rate': '4.981e-05', 'epoch': '0.08178', 'num_input_tokens_seen': 82762352, 'train_runtime': '1.691e+05', 'train_tokens_per_second': '489.4'}

  8%|▊         | 18400/225000 [46:58:21<333:30:56,  5.81s/it]
  8%|▊         | 18401/225000 [46:58:26<308:55:05,  5.38s/it]
  8%|▊         | 18402/225000 [46:58:36<402:38:45,  7.02s/it]
  8%|▊         | 18403/225000 [46:58:44<417:44:04,  7.28s/it]
  8%|▊         | 18404/225000 [46:58:49<376:01:23,  6.55s/it]
  8%|▊         | 18405/225000 [46:58:54<347:15:51,  6.05s/it]
  8%|▊         | 18406/225000 [46:58:59<327:28:12,  5.71s/it]
  8%|▊         | 18407/225000 [46:59:06<346:01:20,  6.03s/it]
  8%|▊         | 18408/225000 [46:59:18<454:36:29,  7.92s/it]
  8%|▊         | 18409/225000 [46:59:23<400:35:50,  6.98s/it]
  8%|▊         | 18410/225000 [46:59:27<357:08:11,  6.22s/it]
  8%|▊         | 18411/225000 [46:59:37<416:44:08,  7.26s/it]
  8%|▊         | 18412/225000 [46:59:42<381:15:12,  6.64s/it]
  8%|▊         | 18413/225000 [46:59:47<351:26:23,  6.12s/it]
  8%|▊         | 18414/225000 [46:59:54<363:20:09,  6.33s/it]
  8%|▊         | 18415/225000 [47:00:08<486:50:32,  8.48s/it]
  8%|▊         | 18416/225000 [47:00:13<433:15:32,  7.55s/it]
  8%|▊         | 18417/225000 [47:00:18<392:24:52,  6.84s/it]
  8%|▊         | 18418/225000 [47:00:23<359:08:50,  6.26s/it]
  8%|▊         | 18419/225000 [47:00:33<416:45:07,  7.26s/it]
  8%|▊         | 18420/225000 [47:00:38<381:51:41,  6.65s/it]
  8%|▊         | 18421/225000 [47:00:44<367:19:04,  6.40s/it]
  8%|▊         | 18422/225000 [47:00:56<464:12:39,  8.09s/it]
  8%|▊         | 18423/225000 [47:01:00<395:24:42,  6.89s/it]
  8%|▊         | 18424/225000 [47:01:05<374:38:21,  6.53s/it]
  8%|▊         | 18425/225000 [47:01:11<353:35:14,  6.16s/it]
  8%|▊         | 18426/225000 [47:01:22<438:53:53,  7.65s/it]
  8%|▊         | 18427/225000 [47:01:28<412:05:22,  7.18s/it]
  8%|▊         | 18428/225000 [47:01:35<412:11:16,  7.18s/it]
  8%|▊         | 18429/225000 [47:01:39<363:09:20,  6.33s/it]
  8%|▊         | 18430/225000 [47:01:46<371:51:06,  6.48s/it]
  8%|▊         | 18431/225000 [47:01:58<468:54:55,  8.17s/it]
  8%|▊         | 18432/225000 [47:02:04<424:23:06,  7.40s/it]
  8%|▊         | 18433/225000 [47:02:09<376:43:56,  6.57s/it]
  8%|▊         | 18434/225000 [47:02:14<357:56:42,  6.24s/it]
  8%|▊         | 18435/225000 [47:02:19<336:54:42,  5.87s/it]
  8%|▊         | 18436/225000 [47:02:24<328:13:56,  5.72s/it]
  8%|▊         | 18437/225000 [47:02:38<464:38:08,  8.10s/it]
  8%|▊         | 18438/225000 [47:02:43<415:55:04,  7.25s/it]
  8%|▊         | 18439/225000 [47:02:48<368:39:47,  6.43s/it]
  8%|▊         | 18440/225000 [47:02:58<438:50:52,  7.65s/it]
  8%|▊         | 18441/225000 [47:03:03<379:04:21,  6.61s/it]
  8%|▊         | 18442/225000 [47:03:08<352:55:04,  6.15s/it]
  8%|▊         | 18443/225000 [47:03:13<341:55:46,  5.96s/it]
  8%|▊         | 18444/225000 [47:03:25<444:43:52,  7.75s/it]
  8%|▊         | 18445/225000 [47:03:30<387:17:06,  6.75s/it]
  8%|▊         | 18446/225000 [47:03:39<430:44:33,  7.51s/it]
  8%|▊         | 18447/225000 [47:03:44<396:57:38,  6.92s/it]
  8%|▊         | 18448/225000 [47:03:50<372:03:18,  6.48s/it]
  8%|▊         | 18449/225000 [47:03:59<420:24:04,  7.33s/it]
  8%|▊         | 18450/225000 [47:04:05<392:45:32,  6.85s/it]
                                                             
{'loss': '0.7122', 'grad_norm': '0.6426', 'learning_rate': '4.981e-05', 'epoch': '0.082', 'num_input_tokens_seen': 82915424, 'train_runtime': '1.694e+05', 'train_tokens_per_second': '489.3'}

  8%|▊         | 18450/225000 [47:04:05<392:45:32,  6.85s/it]
  8%|▊         | 18451/225000 [47:04:11<375:46:07,  6.55s/it]
  8%|▊         | 18452/225000 [47:04:16<358:18:12,  6.24s/it]
  8%|▊         | 18453/225000 [47:04:22<353:31:17,  6.16s/it]
  8%|▊         | 18454/225000 [47:04:28<344:34:44,  6.01s/it]
  8%|▊         | 18455/225000 [47:04:34<339:35:15,  5.92s/it]
  8%|▊         | 18456/225000 [47:04:45<441:48:10,  7.70s/it]
  8%|▊         | 18457/225000 [47:04:50<384:06:10,  6.69s/it]
  8%|▊         | 18458/225000 [47:04:54<347:20:23,  6.05s/it]
  8%|▊         | 18459/225000 [47:05:03<387:25:58,  6.75s/it]
  8%|▊         | 18460/225000 [47:05:13<440:03:28,  7.67s/it]
  8%|▊         | 18461/225000 [47:05:17<387:49:57,  6.76s/it]
  8%|▊         | 18462/225000 [47:05:25<403:27:19,  7.03s/it]
  8%|▊         | 18463/225000 [47:05:33<416:59:06,  7.27s/it]
  8%|▊         | 18464/225000 [47:05:43<464:16:11,  8.09s/it]
  8%|▊         | 18465/225000 [47:05:49<436:56:31,  7.62s/it]
  8%|▊         | 18466/225000 [47:05:53<377:49:51,  6.59s/it]
  8%|▊         | 18467/225000 [47:06:01<403:45:23,  7.04s/it]
  8%|▊         | 18468/225000 [47:06:05<351:55:53,  6.13s/it]
  8%|▊         | 18469/225000 [47:06:15<418:40:05,  7.30s/it]
  8%|▊         | 18470/225000 [47:06:20<368:05:34,  6.42s/it]
  8%|▊         | 18471/225000 [47:06:25<351:01:10,  6.12s/it]
  8%|▊         | 18472/225000 [47:06:36<425:21:33,  7.41s/it]
  8%|▊         | 18473/225000 [47:06:40<371:03:04,  6.47s/it]
  8%|▊         | 18474/225000 [47:06:48<404:26:41,  7.05s/it]
  8%|▊         | 18475/225000 [47:06:53<359:15:13,  6.26s/it]
  8%|▊         | 18476/225000 [47:07:02<405:07:29,  7.06s/it]
  8%|▊         | 18477/225000 [47:07:07<382:19:24,  6.66s/it]
  8%|▊         | 18478/225000 [47:07:12<349:05:58,  6.09s/it]
  8%|▊         | 18479/225000 [47:07:17<330:52:01,  5.77s/it]
  8%|▊         | 18480/225000 [47:07:29<432:36:26,  7.54s/it]
  8%|▊         | 18481/225000 [47:07:33<377:25:45,  6.58s/it]
  8%|▊         | 18482/225000 [47:07:37<336:43:20,  5.87s/it]
  8%|▊         | 18483/225000 [47:07:47<399:47:06,  6.97s/it]
  8%|▊         | 18484/225000 [47:07:55<413:14:43,  7.20s/it]
  8%|▊         | 18485/225000 [47:08:00<385:36:38,  6.72s/it]
  8%|▊         | 18486/225000 [47:08:05<357:53:01,  6.24s/it]
  8%|▊         | 18487/225000 [47:08:11<343:30:28,  5.99s/it]
  8%|▊         | 18488/225000 [47:08:16<333:06:29,  5.81s/it]
  8%|▊         | 18489/225000 [47:08:22<324:55:42,  5.66s/it]
  8%|▊         | 18490/225000 [47:08:27<323:36:43,  5.64s/it]
  8%|▊         | 18491/225000 [47:08:37<393:55:09,  6.87s/it]
  8%|▊         | 18492/225000 [47:08:47<452:50:15,  7.89s/it]
  8%|▊         | 18493/225000 [47:08:54<432:59:24,  7.55s/it]
  8%|▊         | 18494/225000 [47:08:59<386:25:59,  6.74s/it]
  8%|▊         | 18495/225000 [47:09:07<413:10:05,  7.20s/it]
  8%|▊         | 18496/225000 [47:09:15<423:16:51,  7.38s/it]
  8%|▊         | 18497/225000 [47:09:23<433:48:03,  7.56s/it]
  8%|▊         | 18498/225000 [47:09:27<384:15:04,  6.70s/it]
  8%|▊         | 18499/225000 [47:09:34<384:28:38,  6.70s/it]
  8%|▊         | 18500/225000 [47:09:43<418:46:57,  7.30s/it]
                                                             
{'loss': '0.716', 'grad_norm': '0.6252', 'learning_rate': '4.981e-05', 'epoch': '0.08222', 'num_input_tokens_seen': 83068064, 'train_runtime': '1.698e+05', 'train_tokens_per_second': '489.3'}

  8%|▊         | 18500/225000 [47:09:43<418:46:57,  7.30s/it]
  8%|▊         | 18501/225000 [47:09:48<384:38:23,  6.71s/it]
  8%|▊         | 18502/225000 [47:09:53<358:53:13,  6.26s/it]
  8%|▊         | 18503/225000 [47:09:58<333:27:51,  5.81s/it]
  8%|▊         | 18504/225000 [47:10:08<400:26:36,  6.98s/it]
  8%|▊         | 18505/225000 [47:10:17<433:48:03,  7.56s/it]
  8%|▊         | 18506/225000 [47:10:21<373:24:51,  6.51s/it]
  8%|▊         | 18507/225000 [47:10:30<413:53:42,  7.22s/it]
  8%|▊         | 18508/225000 [47:10:34<361:59:05,  6.31s/it]
  8%|▊         | 18509/225000 [47:10:41<376:26:55,  6.56s/it]
  8%|▊         | 18510/225000 [47:10:49<407:00:30,  7.10s/it]
  8%|▊         | 18511/225000 [47:10:55<378:27:40,  6.60s/it]
  8%|▊         | 18512/225000 [47:11:01<363:43:42,  6.34s/it]
  8%|▊         | 18513/225000 [47:11:07<361:36:53,  6.30s/it]
  8%|▊         | 18514/225000 [47:11:15<400:51:50,  6.99s/it]
  8%|▊         | 18515/225000 [47:11:24<435:07:07,  7.59s/it]
  8%|▊         | 18516/225000 [47:11:29<388:13:48,  6.77s/it]
  8%|▊         | 18517/225000 [47:11:34<349:57:40,  6.10s/it]
  8%|▊         | 18518/225000 [47:11:42<377:44:23,  6.59s/it]
  8%|▊         | 18519/225000 [47:11:51<425:35:20,  7.42s/it]
  8%|▊         | 18520/225000 [47:11:55<373:49:15,  6.52s/it]
  8%|▊         | 18521/225000 [47:12:01<351:30:19,  6.13s/it]
  8%|▊         | 18522/225000 [47:12:06<341:45:12,  5.96s/it]
  8%|▊         | 18523/225000 [47:12:11<322:56:23,  5.63s/it]
  8%|▊         | 18524/225000 [47:12:22<414:48:10,  7.23s/it]
  8%|▊         | 18525/225000 [47:12:31<439:35:46,  7.66s/it]
  8%|▊         | 18526/225000 [47:12:39<456:14:24,  7.95s/it]
  8%|▊         | 18527/225000 [47:12:44<393:12:52,  6.86s/it]
  8%|▊         | 18528/225000 [47:12:52<416:12:57,  7.26s/it]
  8%|▊         | 18529/225000 [47:12:56<365:01:01,  6.36s/it]
  8%|▊         | 18530/225000 [47:13:04<396:02:18,  6.91s/it]
  8%|▊         | 18531/225000 [47:13:12<405:39:52,  7.07s/it]
  8%|▊         | 18532/225000 [47:13:20<424:25:10,  7.40s/it]
  8%|▊         | 18533/225000 [47:13:25<386:43:31,  6.74s/it]
  8%|▊         | 18534/225000 [47:13:31<367:11:08,  6.40s/it]
  8%|▊         | 18535/225000 [47:13:36<347:00:38,  6.05s/it]
  8%|▊         | 18536/225000 [47:13:49<466:53:52,  8.14s/it]
  8%|▊         | 18537/225000 [47:13:55<440:49:26,  7.69s/it]
  8%|▊         | 18538/225000 [47:14:03<431:17:53,  7.52s/it]
  8%|▊         | 18539/225000 [47:14:06<366:46:30,  6.40s/it]
  8%|▊         | 18540/225000 [47:14:14<389:14:28,  6.79s/it]
  8%|▊         | 18541/225000 [47:14:19<353:31:07,  6.16s/it]
  8%|▊         | 18542/225000 [47:14:24<332:12:06,  5.79s/it]
  8%|▊         | 18543/225000 [47:14:28<304:00:04,  5.30s/it]
  8%|▊         | 18544/225000 [47:14:36<348:31:22,  6.08s/it]
  8%|▊         | 18545/225000 [47:14:47<438:59:53,  7.65s/it]
  8%|▊         | 18546/225000 [47:14:51<371:47:30,  6.48s/it]
  8%|▊         | 18547/225000 [47:14:56<350:35:05,  6.11s/it]
  8%|▊         | 18548/225000 [47:15:08<453:14:29,  7.90s/it]
  8%|▊         | 18549/225000 [47:15:15<437:25:07,  7.63s/it]
  8%|▊         | 18550/225000 [47:15:21<407:03:40,  7.10s/it]
                                                             
{'loss': '0.7198', 'grad_norm': '0.6049', 'learning_rate': '4.981e-05', 'epoch': '0.08244', 'num_input_tokens_seen': 83220456, 'train_runtime': '1.701e+05', 'train_tokens_per_second': '489.2'}

  8%|▊         | 18550/225000 [47:15:21<407:03:40,  7.10s/it]
  8%|▊         | 18551/225000 [47:15:26<378:09:20,  6.59s/it]
  8%|▊         | 18552/225000 [47:15:32<362:13:01,  6.32s/it]
  8%|▊         | 18553/225000 [47:15:38<352:47:38,  6.15s/it]
  8%|▊         | 18554/225000 [47:15:44<346:26:27,  6.04s/it]
  8%|▊         | 18555/225000 [47:15:49<340:29:40,  5.94s/it]
  8%|▊         | 18556/225000 [47:15:55<330:36:45,  5.77s/it]
  8%|▊         | 18557/225000 [47:16:00<328:30:02,  5.73s/it]
  8%|▊         | 18558/225000 [47:16:06<329:07:19,  5.74s/it]
  8%|▊         | 18559/225000 [47:16:12<326:20:19,  5.69s/it]
  8%|▊         | 18560/225000 [47:16:19<361:38:29,  6.31s/it]
  8%|▊         | 18561/225000 [47:16:33<488:17:14,  8.52s/it]
  8%|▊         | 18562/225000 [47:16:38<424:05:34,  7.40s/it]
  8%|▊         | 18563/225000 [47:16:47<445:59:52,  7.78s/it]
  8%|▊         | 18564/225000 [47:16:51<393:29:03,  6.86s/it]
  8%|▊         | 18565/225000 [47:16:57<375:39:20,  6.55s/it]
  8%|▊         | 18566/225000 [47:17:06<408:45:40,  7.13s/it]
  8%|▊         | 18567/225000 [47:17:15<444:40:03,  7.75s/it]
  8%|▊         | 18568/225000 [47:17:19<377:39:36,  6.59s/it]
  8%|▊         | 18569/225000 [47:17:24<356:50:46,  6.22s/it]
  8%|▊         | 18570/225000 [47:17:35<436:56:08,  7.62s/it]
  8%|▊         | 18571/225000 [47:17:38<367:07:17,  6.40s/it]
  8%|▊         | 18572/225000 [47:17:48<426:09:43,  7.43s/it]
  8%|▊         | 18573/225000 [47:17:53<378:12:06,  6.60s/it]
  8%|▊         | 18574/225000 [47:17:57<338:00:09,  5.89s/it]
  8%|▊         | 18575/225000 [47:18:08<425:04:25,  7.41s/it]
  8%|▊         | 18576/225000 [47:18:12<362:24:14,  6.32s/it]
  8%|▊         | 18577/225000 [47:18:22<431:47:08,  7.53s/it]
  8%|▊         | 18578/225000 [47:18:27<386:54:48,  6.75s/it]
  8%|▊         | 18579/225000 [47:18:35<397:22:54,  6.93s/it]
  8%|▊         | 18580/225000 [47:18:39<360:35:05,  6.29s/it]
  8%|▊         | 18581/225000 [47:18:44<333:27:13,  5.82s/it]
  8%|▊         | 18582/225000 [47:18:49<326:04:28,  5.69s/it]
  8%|▊         | 18583/225000 [47:18:56<344:59:15,  6.02s/it]
  8%|▊         | 18584/225000 [47:19:09<463:33:14,  8.08s/it]
  8%|▊         | 18585/225000 [47:19:14<410:29:00,  7.16s/it]
  8%|▊         | 18586/225000 [47:19:21<403:56:10,  7.04s/it]
  8%|▊         | 18587/225000 [47:19:26<365:44:11,  6.38s/it]
  8%|▊         | 18588/225000 [47:19:34<402:00:52,  7.01s/it]
  8%|▊         | 18589/225000 [47:19:40<388:52:04,  6.78s/it]
  8%|▊         | 18590/225000 [47:19:49<417:37:58,  7.28s/it]
  8%|▊         | 18591/225000 [47:19:56<410:21:15,  7.16s/it]
  8%|▊         | 18592/225000 [47:20:01<376:17:37,  6.56s/it]
  8%|▊         | 18593/225000 [47:20:13<468:25:39,  8.17s/it]
  8%|▊         | 18594/225000 [47:20:17<393:57:13,  6.87s/it]
  8%|▊         | 18595/225000 [47:20:25<422:53:16,  7.38s/it]
  8%|▊         | 18596/225000 [47:20:29<364:03:06,  6.35s/it]
  8%|▊         | 18597/225000 [47:20:40<437:34:08,  7.63s/it]
  8%|▊         | 18598/225000 [47:20:44<377:42:53,  6.59s/it]
  8%|▊         | 18599/225000 [47:20:52<400:44:40,  6.99s/it]
  8%|▊         | 18600/225000 [47:21:00<418:30:40,  7.30s/it]
                                                             
{'loss': '0.7044', 'grad_norm': '0.7017', 'learning_rate': '4.98e-05', 'epoch': '0.08267', 'num_input_tokens_seen': 83372576, 'train_runtime': '1.705e+05', 'train_tokens_per_second': '489.1'}

  8%|▊         | 18600/225000 [47:21:00<418:30:40,  7.30s/it]
  8%|▊         | 18601/225000 [47:21:04<370:27:45,  6.46s/it]
  8%|▊         | 18602/225000 [47:21:11<370:59:03,  6.47s/it]
  8%|▊         | 18603/225000 [47:21:16<349:55:27,  6.10s/it]
  8%|▊         | 18604/225000 [47:21:21<334:58:21,  5.84s/it]
  8%|▊         | 18605/225000 [47:21:27<326:42:23,  5.70s/it]
  8%|▊         | 18606/225000 [47:21:32<321:38:01,  5.61s/it]
  8%|▊         | 18607/225000 [47:21:46<456:39:41,  7.97s/it]
  8%|▊         | 18608/225000 [47:21:50<400:08:04,  6.98s/it]
  8%|▊         | 18609/225000 [47:21:55<367:02:54,  6.40s/it]
  8%|▊         | 18610/225000 [47:22:00<338:32:29,  5.91s/it]
  8%|▊         | 18611/225000 [47:22:11<422:39:53,  7.37s/it]
  8%|▊         | 18612/225000 [47:22:16<375:07:04,  6.54s/it]
  8%|▊         | 18613/225000 [47:22:21<348:06:00,  6.07s/it]
  8%|▊         | 18614/225000 [47:22:32<442:16:36,  7.71s/it]
  8%|▊         | 18615/225000 [47:22:40<445:16:27,  7.77s/it]
  8%|▊         | 18616/225000 [47:22:47<430:49:12,  7.51s/it]
  8%|▊         | 18617/225000 [47:22:51<373:50:17,  6.52s/it]
  8%|▊         | 18618/225000 [47:22:56<348:22:13,  6.08s/it]
  8%|▊         | 18619/225000 [47:23:02<340:31:24,  5.94s/it]
  8%|▊         | 18620/225000 [47:23:15<465:44:36,  8.12s/it]
  8%|▊         | 18621/225000 [47:23:20<414:40:26,  7.23s/it]
  8%|▊         | 18622/225000 [47:23:25<370:14:06,  6.46s/it]
  8%|▊         | 18623/225000 [47:23:29<337:28:16,  5.89s/it]
  8%|▊         | 18624/225000 [47:23:43<468:48:20,  8.18s/it]
  8%|▊         | 18625/225000 [47:23:47<399:31:16,  6.97s/it]
  8%|▊         | 18626/225000 [47:23:52<361:04:35,  6.30s/it]
  8%|▊         | 18627/225000 [47:24:01<406:47:41,  7.10s/it]
  8%|▊         | 18628/225000 [47:24:06<371:09:57,  6.47s/it]
  8%|▊         | 18629/225000 [47:24:10<339:06:09,  5.92s/it]
  8%|▊         | 18630/225000 [47:24:16<333:48:52,  5.82s/it]
  8%|▊         | 18631/225000 [47:24:25<385:42:46,  6.73s/it]
  8%|▊         | 18632/225000 [47:24:36<455:41:18,  7.95s/it]
  8%|▊         | 18633/225000 [47:24:44<461:56:46,  8.06s/it]
  8%|▊         | 18634/225000 [47:24:53<471:54:40,  8.23s/it]
  8%|▊         | 18635/225000 [47:24:57<399:29:51,  6.97s/it]
  8%|▊         | 18636/225000 [47:25:05<431:35:37,  7.53s/it]
  8%|▊         | 18637/225000 [47:25:12<418:16:31,  7.30s/it]
  8%|▊         | 18638/225000 [47:25:16<358:57:15,  6.26s/it]
  8%|▊         | 18639/225000 [47:25:26<420:28:17,  7.34s/it]
  8%|▊         | 18640/225000 [47:25:31<376:27:27,  6.57s/it]
  8%|▊         | 18641/225000 [47:25:38<387:42:35,  6.76s/it]
  8%|▊         | 18642/225000 [47:25:46<417:02:55,  7.28s/it]
  8%|▊         | 18643/225000 [47:25:50<361:11:20,  6.30s/it]
  8%|▊         | 18644/225000 [47:25:59<393:52:46,  6.87s/it]
  8%|▊         | 18645/225000 [47:26:05<389:26:25,  6.79s/it]
  8%|▊         | 18646/225000 [47:26:12<394:47:38,  6.89s/it]
  8%|▊         | 18647/225000 [47:26:18<382:01:27,  6.66s/it]
  8%|▊         | 18648/225000 [47:26:25<384:57:11,  6.72s/it]
  8%|▊         | 18649/225000 [47:26:32<393:18:47,  6.86s/it]
  8%|▊         | 18650/225000 [47:26:40<400:25:38,  6.99s/it]
                                                             
{'loss': '0.7322', 'grad_norm': '0.6002', 'learning_rate': '4.98e-05', 'epoch': '0.08289', 'num_input_tokens_seen': 83524664, 'train_runtime': '1.708e+05', 'train_tokens_per_second': '489'}

  8%|▊         | 18650/225000 [47:26:40<400:25:38,  6.99s/it]
  8%|▊         | 18651/225000 [47:26:45<373:04:42,  6.51s/it]
  8%|▊         | 18652/225000 [47:26:52<385:10:43,  6.72s/it]
  8%|▊         | 18653/225000 [47:26:58<361:20:08,  6.30s/it]
  8%|▊         | 18654/225000 [47:27:03<339:49:29,  5.93s/it]
  8%|▊         | 18655/225000 [47:27:08<331:09:51,  5.78s/it]
  8%|▊         | 18656/225000 [47:27:16<366:49:21,  6.40s/it]
  8%|▊         | 18657/225000 [47:27:29<487:22:32,  8.50s/it]
  8%|▊         | 18658/225000 [47:27:33<399:21:17,  6.97s/it]
  8%|▊         | 18659/225000 [47:27:40<405:10:04,  7.07s/it]
  8%|▊         | 18660/225000 [47:27:44<342:59:00,  5.98s/it]
  8%|▊         | 18661/225000 [47:27:54<414:07:33,  7.23s/it]
  8%|▊         | 18662/225000 [47:27:59<381:45:08,  6.66s/it]
  8%|▊         | 18663/225000 [47:28:04<354:54:16,  6.19s/it]
  8%|▊         | 18664/225000 [47:28:09<335:36:25,  5.86s/it]
  8%|▊         | 18665/225000 [47:28:14<319:52:20,  5.58s/it]
  8%|▊         | 18666/225000 [47:28:23<378:08:53,  6.60s/it]
  8%|▊         | 18667/225000 [47:28:34<449:17:26,  7.84s/it]
  8%|▊         | 18668/225000 [47:28:41<433:18:41,  7.56s/it]
  8%|▊         | 18669/225000 [47:28:47<412:48:13,  7.20s/it]
  8%|▊         | 18670/225000 [47:28:52<380:09:40,  6.63s/it]
  8%|▊         | 18671/225000 [47:29:02<431:17:24,  7.53s/it]
  8%|▊         | 18672/225000 [47:29:10<436:04:57,  7.61s/it]
  8%|▊         | 18673/225000 [47:29:16<407:17:49,  7.11s/it]
  8%|▊         | 18674/225000 [47:29:22<388:44:09,  6.78s/it]
  8%|▊         | 18675/225000 [47:29:30<406:30:15,  7.09s/it]
  8%|▊         | 18676/225000 [47:29:38<422:41:57,  7.38s/it]
  8%|▊         | 18677/225000 [47:29:44<412:36:47,  7.20s/it]
  8%|▊         | 18678/225000 [47:29:51<393:57:11,  6.87s/it]
  8%|▊         | 18679/225000 [47:29:57<389:34:56,  6.80s/it]
  8%|▊         | 18680/225000 [47:30:06<424:23:04,  7.40s/it]
  8%|▊         | 18681/225000 [47:30:09<353:22:13,  6.17s/it]
  8%|▊         | 18682/225000 [47:30:16<361:14:32,  6.30s/it]
  8%|▊         | 18683/225000 [47:30:25<413:40:23,  7.22s/it]
  8%|▊         | 18684/225000 [47:30:31<388:43:21,  6.78s/it]
  8%|▊         | 18685/225000 [47:30:38<389:04:57,  6.79s/it]
  8%|▊         | 18686/225000 [47:30:43<364:07:22,  6.35s/it]
  8%|▊         | 18687/225000 [47:30:48<338:33:01,  5.91s/it]
  8%|▊         | 18688/225000 [47:30:53<329:51:24,  5.76s/it]
  8%|▊         | 18689/225000 [47:30:58<313:48:46,  5.48s/it]
  8%|▊         | 18690/225000 [47:31:11<447:17:51,  7.81s/it]
  8%|▊         | 18691/225000 [47:31:19<439:31:02,  7.67s/it]
  8%|▊         | 18692/225000 [47:31:27<451:14:36,  7.87s/it]
  8%|▊         | 18693/225000 [47:31:31<387:36:34,  6.76s/it]
  8%|▊         | 18694/225000 [47:31:36<357:49:18,  6.24s/it]
  8%|▊         | 18695/225000 [47:31:42<342:38:11,  5.98s/it]
  8%|▊         | 18696/225000 [47:31:47<332:58:21,  5.81s/it]
  8%|▊         | 18697/225000 [47:31:53<326:04:11,  5.69s/it]
  8%|▊         | 18698/225000 [47:32:07<476:28:07,  8.31s/it]
  8%|▊         | 18699/225000 [47:32:11<408:19:46,  7.13s/it]
  8%|▊         | 18700/225000 [47:32:16<369:51:41,  6.45s/it]
                                                             
{'loss': '0.7065', 'grad_norm': '0.702', 'learning_rate': '4.98e-05', 'epoch': '0.08311', 'num_input_tokens_seen': 83676392, 'train_runtime': '1.711e+05', 'train_tokens_per_second': '488.9'}

  8%|▊         | 18700/225000 [47:32:16<369:51:41,  6.45s/it]
  8%|▊         | 18701/225000 [47:32:21<341:32:00,  5.96s/it]
  8%|▊         | 18702/225000 [47:32:37<514:30:13,  8.98s/it]
  8%|▊         | 18703/225000 [47:32:43<466:15:51,  8.14s/it]
  8%|▊         | 18704/225000 [47:32:48<410:57:40,  7.17s/it]
  8%|▊         | 18705/225000 [47:32:53<367:19:59,  6.41s/it]
  8%|▊         | 18706/225000 [47:32:58<346:35:03,  6.05s/it]
  8%|▊         | 18707/225000 [47:33:10<444:01:16,  7.75s/it]
  8%|▊         | 18708/225000 [47:33:15<405:13:27,  7.07s/it]
  8%|▊         | 18709/225000 [47:33:21<379:09:19,  6.62s/it]
  8%|▊         | 18710/225000 [47:33:27<365:38:37,  6.38s/it]
  8%|▊         | 18711/225000 [47:33:32<349:50:02,  6.11s/it]
  8%|▊         | 18712/225000 [47:33:38<340:37:09,  5.94s/it]
  8%|▊         | 18713/225000 [47:33:43<334:40:55,  5.84s/it]
  8%|▊         | 18714/225000 [47:33:49<328:47:06,  5.74s/it]
  8%|▊         | 18715/225000 [47:33:54<325:43:32,  5.68s/it]
  8%|▊         | 18716/225000 [47:34:00<330:02:37,  5.76s/it]
  8%|▊         | 18717/225000 [47:34:14<465:07:51,  8.12s/it]
  8%|▊         | 18718/225000 [47:34:21<442:39:20,  7.73s/it]
  8%|▊         | 18719/225000 [47:34:26<401:26:51,  7.01s/it]
  8%|▊         | 18720/225000 [47:34:31<365:37:28,  6.38s/it]
  8%|▊         | 18721/225000 [47:34:41<425:29:43,  7.43s/it]
  8%|▊         | 18722/225000 [47:34:50<465:09:07,  8.12s/it]
  8%|▊         | 18723/225000 [47:34:56<423:07:57,  7.38s/it]
  8%|▊         | 18724/225000 [47:35:04<436:20:39,  7.62s/it]
  8%|▊         | 18725/225000 [47:35:10<398:41:58,  6.96s/it]
  8%|▊         | 18726/225000 [47:35:17<400:16:51,  6.99s/it]
  8%|▊         | 18727/225000 [47:35:24<410:08:22,  7.16s/it]
  8%|▊         | 18728/225000 [47:35:29<359:25:55,  6.27s/it]
  8%|▊         | 18729/225000 [47:35:38<420:47:14,  7.34s/it]
  8%|▊         | 18730/225000 [47:35:44<392:12:36,  6.85s/it]
  8%|▊         | 18731/225000 [47:35:49<352:20:22,  6.15s/it]
  8%|▊         | 18732/225000 [47:35:54<341:33:54,  5.96s/it]
  8%|▊         | 18733/225000 [47:36:00<340:32:21,  5.94s/it]
  8%|▊         | 18734/225000 [47:36:06<339:55:18,  5.93s/it]
  8%|▊         | 18735/225000 [47:36:11<331:23:47,  5.78s/it]
  8%|▊         | 18736/225000 [47:36:17<334:03:24,  5.83s/it]
  8%|▊         | 18737/225000 [47:36:31<471:31:44,  8.23s/it]
  8%|▊         | 18738/225000 [47:36:39<464:55:57,  8.11s/it]
  8%|▊         | 18739/225000 [47:36:46<441:54:21,  7.71s/it]
  8%|▊         | 18740/225000 [47:36:53<431:56:49,  7.54s/it]
  8%|▊         | 18741/225000 [47:36:58<387:19:25,  6.76s/it]
  8%|▊         | 18742/225000 [47:37:07<421:57:37,  7.36s/it]
  8%|▊         | 18743/225000 [47:37:13<412:18:15,  7.20s/it]
  8%|▊         | 18744/225000 [47:37:18<368:57:54,  6.44s/it]
  8%|▊         | 18745/225000 [47:37:27<404:55:27,  7.07s/it]
  8%|▊         | 18746/225000 [47:37:32<374:04:27,  6.53s/it]
  8%|▊         | 18747/225000 [47:37:40<394:24:40,  6.88s/it]
  8%|▊         | 18748/225000 [47:37:46<389:26:02,  6.80s/it]
  8%|▊         | 18749/225000 [47:37:53<388:26:33,  6.78s/it]
  8%|▊         | 18750/225000 [47:38:03<439:26:15,  7.67s/it]
                                                             
{'loss': '0.7366', 'grad_norm': '0.5764', 'learning_rate': '4.98e-05', 'epoch': '0.08333', 'num_input_tokens_seen': 83829176, 'train_runtime': '1.715e+05', 'train_tokens_per_second': '488.8'}

  8%|▊         | 18750/225000 [47:38:03<439:26:15,  7.67s/it]
  8%|▊         | 18751/225000 [47:38:07<384:07:50,  6.70s/it]
  8%|▊         | 18752/225000 [47:38:12<350:41:36,  6.12s/it]
  8%|▊         | 18753/225000 [47:38:16<323:54:42,  5.65s/it]
  8%|▊         | 18754/225000 [47:38:29<439:54:26,  7.68s/it]
  8%|▊         | 18755/225000 [47:38:33<374:38:09,  6.54s/it]
  8%|▊         | 18756/225000 [47:38:42<416:54:08,  7.28s/it]
  8%|▊         | 18757/225000 [47:38:50<428:18:30,  7.48s/it]
  8%|▊         | 18758/225000 [47:38:54<373:04:51,  6.51s/it]
  8%|▊         | 18759/225000 [47:38:58<335:12:39,  5.85s/it]
  8%|▊         | 18760/225000 [47:39:06<365:35:09,  6.38s/it]
  8%|▊         | 18761/225000 [47:39:16<421:49:09,  7.36s/it]
  8%|▊         | 18762/225000 [47:39:20<373:42:48,  6.52s/it]
  8%|▊         | 18763/225000 [47:39:25<347:01:34,  6.06s/it]
  8%|▊         | 18764/225000 [47:39:30<324:44:57,  5.67s/it]
  8%|▊         | 18765/225000 [47:39:42<434:48:38,  7.59s/it]
  8%|▊         | 18766/225000 [47:39:46<379:48:34,  6.63s/it]
  8%|▊         | 18767/225000 [47:39:51<347:36:35,  6.07s/it]
  8%|▊         | 18768/225000 [47:40:03<451:24:08,  7.88s/it]
  8%|▊         | 18769/225000 [47:40:07<382:23:08,  6.67s/it]
  8%|▊         | 18770/225000 [47:40:13<371:39:39,  6.49s/it]
  8%|▊         | 18771/225000 [47:40:23<426:53:27,  7.45s/it]
  8%|▊         | 18772/225000 [47:40:27<369:39:22,  6.45s/it]
  8%|▊         | 18773/225000 [47:40:33<369:23:44,  6.45s/it]
  8%|▊         | 18774/225000 [47:40:43<420:18:00,  7.34s/it]
  8%|▊         | 18775/225000 [47:40:47<361:18:36,  6.31s/it]
  8%|▊         | 18776/225000 [47:40:57<422:38:33,  7.38s/it]
  8%|▊         | 18777/225000 [47:41:03<399:18:59,  6.97s/it]
  8%|▊         | 18778/225000 [47:41:10<408:08:02,  7.12s/it]
  8%|▊         | 18779/225000 [47:41:14<350:20:47,  6.12s/it]
  8%|▊         | 18780/225000 [47:41:23<405:02:33,  7.07s/it]
  8%|▊         | 18781/225000 [47:41:27<346:00:26,  6.04s/it]
  8%|▊         | 18782/225000 [47:41:35<385:17:33,  6.73s/it]
  8%|▊         | 18783/225000 [47:41:39<339:15:33,  5.92s/it]
  8%|▊         | 18784/225000 [47:41:43<309:42:04,  5.41s/it]
  8%|▊         | 18785/225000 [47:41:55<409:49:07,  7.15s/it]
  8%|▊         | 18786/225000 [47:41:59<365:13:29,  6.38s/it]
  8%|▊         | 18787/225000 [47:42:04<342:09:32,  5.97s/it]
  8%|▊         | 18788/225000 [47:42:09<319:49:32,  5.58s/it]
  8%|▊         | 18789/225000 [47:42:20<421:54:51,  7.37s/it]
  8%|▊         | 18790/225000 [47:42:24<365:09:28,  6.37s/it]
  8%|▊         | 18791/225000 [47:42:33<397:33:45,  6.94s/it]
  8%|▊         | 18792/225000 [47:42:36<341:48:57,  5.97s/it]
  8%|▊         | 18793/225000 [47:42:46<411:18:38,  7.18s/it]
  8%|▊         | 18794/225000 [47:42:51<375:33:26,  6.56s/it]
  8%|▊         | 18795/225000 [47:42:57<352:04:06,  6.15s/it]
  8%|▊         | 18796/225000 [47:43:04<375:13:49,  6.55s/it]
  8%|▊         | 18797/225000 [47:43:10<370:03:33,  6.46s/it]
  8%|▊         | 18798/225000 [47:43:18<383:26:11,  6.69s/it]
  8%|▊         | 18799/225000 [47:43:25<396:17:31,  6.92s/it]
  8%|▊         | 18800/225000 [47:43:33<415:31:34,  7.25s/it]
                                                             
{'loss': '0.7005', 'grad_norm': '0.7696', 'learning_rate': '4.979e-05', 'epoch': '0.08356', 'num_input_tokens_seen': 83980840, 'train_runtime': '1.718e+05', 'train_tokens_per_second': '488.8'}

  8%|▊         | 18800/225000 [47:43:33<415:31:34,  7.25s/it]
  8%|▊         | 18801/225000 [47:43:38<370:34:40,  6.47s/it]
  8%|▊         | 18802/225000 [47:43:45<376:18:21,  6.57s/it]
  8%|▊         | 18803/225000 [47:43:52<393:01:44,  6.86s/it]
  8%|▊         | 18804/225000 [47:44:00<413:21:53,  7.22s/it]
  8%|▊         | 18805/225000 [47:44:04<347:32:15,  6.07s/it]
  8%|▊         | 18806/225000 [47:44:12<391:58:03,  6.84s/it]
  8%|▊         | 18807/225000 [47:44:19<390:08:15,  6.81s/it]
  8%|▊         | 18808/225000 [47:44:27<405:10:24,  7.07s/it]
  8%|▊         | 18809/225000 [47:44:32<369:23:29,  6.45s/it]
  8%|▊         | 18810/225000 [47:44:39<384:57:49,  6.72s/it]
  8%|▊         | 18811/225000 [47:44:44<358:39:29,  6.26s/it]
  8%|▊         | 18812/225000 [47:44:50<344:02:57,  6.01s/it]
  8%|▊         | 18813/225000 [47:44:55<328:44:39,  5.74s/it]
  8%|▊         | 18814/225000 [47:45:00<323:29:54,  5.65s/it]
  8%|▊         | 18815/225000 [47:45:12<423:33:41,  7.40s/it]
  8%|▊         | 18816/225000 [47:45:16<380:48:55,  6.65s/it]
  8%|▊         | 18817/225000 [47:45:21<340:21:53,  5.94s/it]
  8%|▊         | 18818/225000 [47:45:31<406:01:56,  7.09s/it]
  8%|▊         | 18819/225000 [47:45:36<385:30:18,  6.73s/it]
  8%|▊         | 18820/225000 [47:45:41<343:01:55,  5.99s/it]
  8%|▊         | 18821/225000 [47:45:46<333:25:41,  5.82s/it]
  8%|▊         | 18822/225000 [47:45:51<319:40:53,  5.58s/it]
  8%|▊         | 18823/225000 [47:46:05<465:13:20,  8.12s/it]
  8%|▊         | 18824/225000 [47:46:16<503:40:19,  8.79s/it]
  8%|▊         | 18825/225000 [47:46:23<473:16:06,  8.26s/it]
  8%|▊         | 18826/225000 [47:46:26<393:53:07,  6.88s/it]
  8%|▊         | 18827/225000 [47:46:35<427:41:53,  7.47s/it]
  8%|▊         | 18828/225000 [47:46:40<387:24:33,  6.76s/it]
  8%|▊         | 18829/225000 [47:46:45<357:09:15,  6.24s/it]
  8%|▊         | 18830/225000 [47:46:50<338:37:10,  5.91s/it]
  8%|▊         | 18831/225000 [47:46:56<328:22:58,  5.73s/it]
  8%|▊         | 18832/225000 [47:47:01<320:12:51,  5.59s/it]
  8%|▊         | 18833/225000 [47:47:06<312:37:45,  5.46s/it]
  8%|▊         | 18834/225000 [47:47:20<450:04:50,  7.86s/it]
  8%|▊         | 18835/225000 [47:47:27<435:21:22,  7.60s/it]
  8%|▊         | 18836/225000 [47:47:33<420:47:40,  7.35s/it]
  8%|▊         | 18837/225000 [47:47:38<374:47:12,  6.54s/it]
  8%|▊         | 18838/225000 [47:47:44<368:03:27,  6.43s/it]
  8%|▊         | 18839/225000 [47:47:52<400:00:46,  6.99s/it]
  8%|▊         | 18840/225000 [47:48:01<424:34:04,  7.41s/it]
  8%|▊         | 18841/225000 [47:48:04<360:02:23,  6.29s/it]
  8%|▊         | 18842/225000 [47:48:09<323:10:02,  5.64s/it]
  8%|▊         | 18843/225000 [47:48:18<391:29:48,  6.84s/it]
  8%|▊         | 18844/225000 [47:48:26<406:27:47,  7.10s/it]
  8%|▊         | 18845/225000 [47:48:30<347:37:53,  6.07s/it]
  8%|▊         | 18846/225000 [47:48:39<403:12:47,  7.04s/it]
  8%|▊         | 18847/225000 [47:48:43<355:03:54,  6.20s/it]
  8%|▊         | 18848/225000 [47:48:47<319:34:41,  5.58s/it]
  8%|▊         | 18849/225000 [47:49:00<440:43:17,  7.70s/it]
  8%|▊         | 18850/225000 [47:49:04<370:18:26,  6.47s/it]
                                                             
{'loss': '0.7068', 'grad_norm': '0.6635', 'learning_rate': '4.979e-05', 'epoch': '0.08378', 'num_input_tokens_seen': 84134848, 'train_runtime': '1.721e+05', 'train_tokens_per_second': '488.7'}

  8%|▊         | 18850/225000 [47:49:04<370:18:26,  6.47s/it]
  8%|▊         | 18851/225000 [47:49:13<416:00:26,  7.26s/it]
  8%|▊         | 18852/225000 [47:49:19<399:01:41,  6.97s/it]
  8%|▊         | 18853/225000 [47:49:23<342:27:39,  5.98s/it]
  8%|▊         | 18854/225000 [47:49:32<399:18:04,  6.97s/it]
  8%|▊         | 18855/225000 [47:49:37<364:01:56,  6.36s/it]
  8%|▊         | 18856/225000 [47:49:42<341:33:19,  5.96s/it]
  8%|▊         | 18857/225000 [47:49:47<333:09:36,  5.82s/it]
  8%|▊         | 18858/225000 [47:49:52<319:24:55,  5.58s/it]
  8%|▊         | 18859/225000 [47:49:58<324:16:52,  5.66s/it]
  8%|▊         | 18860/225000 [47:50:09<410:35:24,  7.17s/it]
  8%|▊         | 18861/225000 [47:50:15<393:35:34,  6.87s/it]
  8%|▊         | 18862/225000 [47:50:21<379:59:54,  6.64s/it]
  8%|▊         | 18863/225000 [47:50:31<438:24:31,  7.66s/it]
  8%|▊         | 18864/225000 [47:50:36<384:32:20,  6.72s/it]
  8%|▊         | 18865/225000 [47:50:40<344:33:09,  6.02s/it]
  8%|▊         | 18866/225000 [47:50:51<430:10:29,  7.51s/it]
  8%|▊         | 18867/225000 [47:50:55<366:40:51,  6.40s/it]
  8%|▊         | 18868/225000 [47:51:03<391:03:11,  6.83s/it]
  8%|▊         | 18869/225000 [47:51:10<392:01:39,  6.85s/it]
  8%|▊         | 18870/225000 [47:51:15<361:36:36,  6.32s/it]
  8%|▊         | 18871/225000 [47:51:20<336:53:47,  5.88s/it]
  8%|▊         | 18872/225000 [47:51:24<318:13:31,  5.56s/it]
  8%|▊         | 18873/225000 [47:51:30<314:35:44,  5.49s/it]
  8%|▊         | 18874/225000 [47:51:37<343:39:23,  6.00s/it]
  8%|▊         | 18875/225000 [47:51:50<464:42:00,  8.12s/it]
  8%|▊         | 18876/225000 [47:51:54<401:34:57,  7.01s/it]
  8%|▊         | 18877/225000 [47:51:59<362:50:51,  6.34s/it]
  8%|▊         | 18878/225000 [47:52:04<343:37:06,  6.00s/it]
  8%|▊         | 18879/225000 [47:52:09<327:46:46,  5.72s/it]
  8%|▊         | 18880/225000 [47:52:16<345:22:38,  6.03s/it]
  8%|▊         | 18881/225000 [47:52:28<440:55:50,  7.70s/it]
  8%|▊         | 18882/225000 [47:52:32<378:10:12,  6.61s/it]
  8%|▊         | 18883/225000 [47:52:44<466:19:17,  8.14s/it]
  8%|▊         | 18884/225000 [47:52:47<387:28:27,  6.77s/it]
  8%|▊         | 18885/225000 [47:52:51<343:25:03,  6.00s/it]
  8%|▊         | 18886/225000 [47:53:02<424:16:47,  7.41s/it]
  8%|▊         | 18887/225000 [47:53:06<365:43:28,  6.39s/it]
  8%|▊         | 18888/225000 [47:53:15<413:08:50,  7.22s/it]
  8%|▊         | 18889/225000 [47:53:20<371:03:22,  6.48s/it]
  8%|▊         | 18890/225000 [47:53:24<324:17:29,  5.66s/it]
  8%|▊         | 18891/225000 [47:53:34<404:31:14,  7.07s/it]
  8%|▊         | 18892/225000 [47:53:38<358:09:16,  6.26s/it]
  8%|▊         | 18893/225000 [47:53:43<335:22:33,  5.86s/it]
  8%|▊         | 18894/225000 [47:53:54<415:33:41,  7.26s/it]
  8%|▊         | 18895/225000 [47:53:59<385:40:59,  6.74s/it]
  8%|▊         | 18896/225000 [47:54:05<359:15:10,  6.28s/it]
  8%|▊         | 18897/225000 [47:54:10<341:04:37,  5.96s/it]
  8%|▊         | 18898/225000 [47:54:15<335:15:12,  5.86s/it]
  8%|▊         | 18899/225000 [47:54:21<328:26:19,  5.74s/it]
  8%|▊         | 18900/225000 [47:54:33<432:11:10,  7.55s/it]
                                                             
{'loss': '0.7323', 'grad_norm': '0.6584', 'learning_rate': '4.979e-05', 'epoch': '0.084', 'num_input_tokens_seen': 84286256, 'train_runtime': '1.725e+05', 'train_tokens_per_second': '488.7'}

  8%|▊         | 18900/225000 [47:54:33<432:11:10,  7.55s/it]
  8%|▊         | 18901/225000 [47:54:39<411:39:32,  7.19s/it]
  8%|▊         | 18902/225000 [47:54:43<361:34:48,  6.32s/it]
  8%|▊         | 18903/225000 [47:54:53<420:27:59,  7.34s/it]
  8%|▊         | 18904/225000 [47:54:59<395:41:36,  6.91s/it]
  8%|▊         | 18905/225000 [47:55:07<421:49:31,  7.37s/it]
  8%|▊         | 18906/225000 [47:55:14<405:35:06,  7.08s/it]
  8%|▊         | 18907/225000 [47:55:19<367:53:24,  6.43s/it]
  8%|▊         | 18908/225000 [47:55:24<339:54:22,  5.94s/it]
  8%|▊         | 18909/225000 [47:55:34<409:53:26,  7.16s/it]
  8%|▊         | 18910/225000 [47:55:40<401:03:14,  7.01s/it]
  8%|▊         | 18911/225000 [47:55:46<372:46:22,  6.51s/it]
  8%|▊         | 18912/225000 [47:55:51<354:47:26,  6.20s/it]
  8%|▊         | 18913/225000 [47:55:57<348:15:59,  6.08s/it]
  8%|▊         | 18914/225000 [47:56:02<338:28:24,  5.91s/it]
  8%|▊         | 18915/225000 [47:56:08<333:51:24,  5.83s/it]
  8%|▊         | 18916/225000 [47:56:19<415:45:33,  7.26s/it]
  8%|▊         | 18917/225000 [47:56:27<435:54:19,  7.61s/it]
  8%|▊         | 18918/225000 [47:56:34<426:27:11,  7.45s/it]
  8%|▊         | 18919/225000 [47:56:39<377:09:03,  6.59s/it]
  8%|▊         | 18920/225000 [47:56:49<440:10:37,  7.69s/it]
  8%|▊         | 18921/225000 [47:56:56<436:36:39,  7.63s/it]
  8%|▊         | 18922/225000 [47:57:00<371:54:53,  6.50s/it]
  8%|▊         | 18923/225000 [47:57:05<342:43:42,  5.99s/it]
  8%|▊         | 18924/225000 [47:57:16<425:02:12,  7.43s/it]
  8%|▊         | 18925/225000 [47:57:21<380:07:58,  6.64s/it]
  8%|▊         | 18926/225000 [47:57:30<433:19:32,  7.57s/it]
  8%|▊         | 18927/225000 [47:57:40<461:11:17,  8.06s/it]
  8%|▊         | 18928/225000 [47:57:47<442:12:42,  7.73s/it]
  8%|▊         | 18929/225000 [47:57:50<375:00:52,  6.55s/it]
  8%|▊         | 18930/225000 [47:58:00<423:59:31,  7.41s/it]
  8%|▊         | 18931/225000 [47:58:04<369:40:31,  6.46s/it]
  8%|▊         | 18932/225000 [47:58:09<337:21:08,  5.89s/it]
  8%|▊         | 18933/225000 [47:58:19<419:34:05,  7.33s/it]
  8%|▊         | 18934/225000 [47:58:24<368:23:17,  6.44s/it]
  8%|▊         | 18935/225000 [47:58:33<420:41:32,  7.35s/it]
  8%|▊         | 18936/225000 [47:58:38<380:25:05,  6.65s/it]
  8%|▊         | 18937/225000 [47:58:45<392:52:02,  6.86s/it]
  8%|▊         | 18938/225000 [47:58:55<431:23:29,  7.54s/it]
  8%|▊         | 18939/225000 [47:58:59<384:35:50,  6.72s/it]
  8%|▊         | 18940/225000 [47:59:07<400:59:40,  7.01s/it]
  8%|▊         | 18941/225000 [47:59:12<358:50:19,  6.27s/it]
  8%|▊         | 18942/225000 [47:59:19<383:27:01,  6.70s/it]
  8%|▊         | 18943/225000 [47:59:28<410:01:30,  7.16s/it]
  8%|▊         | 18944/225000 [47:59:32<355:19:47,  6.21s/it]
  8%|▊         | 18945/225000 [47:59:36<328:34:45,  5.74s/it]
  8%|▊         | 18946/225000 [47:59:48<434:32:52,  7.59s/it]
  8%|▊         | 18947/225000 [47:59:54<407:41:07,  7.12s/it]
  8%|▊         | 18948/225000 [48:00:01<407:21:32,  7.12s/it]
  8%|▊         | 18949/225000 [48:00:06<365:50:11,  6.39s/it]
  8%|▊         | 18950/225000 [48:00:12<354:53:33,  6.20s/it]
                                                             
{'loss': '0.7221', 'grad_norm': '0.6445', 'learning_rate': '4.979e-05', 'epoch': '0.08422', 'num_input_tokens_seen': 84439824, 'train_runtime': '1.728e+05', 'train_tokens_per_second': '488.6'}

  8%|▊         | 18950/225000 [48:00:12<354:53:33,  6.20s/it]
  8%|▊         | 18951/225000 [48:00:17<335:24:35,  5.86s/it]
  8%|▊         | 18952/225000 [48:00:25<368:29:03,  6.44s/it]
  8%|▊         | 18953/225000 [48:00:36<457:46:19,  8.00s/it]
  8%|▊         | 18954/225000 [48:00:41<400:41:38,  7.00s/it]
  8%|▊         | 18955/225000 [48:00:45<352:22:06,  6.16s/it]
  8%|▊         | 18956/225000 [48:00:50<325:57:40,  5.70s/it]
  8%|▊         | 18957/225000 [48:01:01<419:05:58,  7.32s/it]
  8%|▊         | 18958/225000 [48:01:05<372:29:49,  6.51s/it]
  8%|▊         | 18959/225000 [48:01:11<353:20:01,  6.17s/it]
  8%|▊         | 18960/225000 [48:01:16<340:55:25,  5.96s/it]
  8%|▊         | 18961/225000 [48:01:21<327:06:30,  5.72s/it]
  8%|▊         | 18962/225000 [48:01:35<456:22:37,  7.97s/it]
  8%|▊         | 18963/225000 [48:01:41<434:34:50,  7.59s/it]
  8%|▊         | 18964/225000 [48:01:47<408:55:59,  7.15s/it]
  8%|▊         | 18965/225000 [48:01:54<394:28:29,  6.89s/it]
  8%|▊         | 18966/225000 [48:02:02<419:08:46,  7.32s/it]
  8%|▊         | 18967/225000 [48:02:09<404:07:24,  7.06s/it]
  8%|▊         | 18968/225000 [48:02:16<417:33:32,  7.30s/it]
  8%|▊         | 18969/225000 [48:02:21<368:16:49,  6.43s/it]
  8%|▊         | 18970/225000 [48:02:26<352:36:07,  6.16s/it]
  8%|▊         | 18971/225000 [48:02:38<445:26:10,  7.78s/it]
  8%|▊         | 18972/225000 [48:02:44<411:08:08,  7.18s/it]
  8%|▊         | 18973/225000 [48:02:51<406:22:15,  7.10s/it]
  8%|▊         | 18974/225000 [48:02:55<358:36:22,  6.27s/it]
  8%|▊         | 18975/225000 [48:03:01<354:12:29,  6.19s/it]
  8%|▊         | 18976/225000 [48:03:08<374:03:56,  6.54s/it]
  8%|▊         | 18977/225000 [48:03:20<471:20:22,  8.24s/it]
  8%|▊         | 18978/225000 [48:03:26<433:13:07,  7.57s/it]
  8%|▊         | 18979/225000 [48:03:32<392:41:32,  6.86s/it]
  8%|▊         | 18980/225000 [48:03:36<355:44:36,  6.22s/it]
  8%|▊         | 18981/225000 [48:03:47<435:24:38,  7.61s/it]
  8%|▊         | 18982/225000 [48:03:55<441:30:36,  7.72s/it]
  8%|▊         | 18983/225000 [48:03:59<379:18:14,  6.63s/it]
  8%|▊         | 18984/225000 [48:04:04<339:18:44,  5.93s/it]
  8%|▊         | 18985/225000 [48:04:08<311:57:45,  5.45s/it]
  8%|▊         | 18986/225000 [48:04:17<367:19:00,  6.42s/it]
  8%|▊         | 18987/225000 [48:04:26<417:37:58,  7.30s/it]
  8%|▊         | 18988/225000 [48:04:32<397:00:21,  6.94s/it]
  8%|▊         | 18989/225000 [48:04:37<354:48:25,  6.20s/it]
  8%|▊         | 18990/225000 [48:04:45<386:54:03,  6.76s/it]
  8%|▊         | 18991/225000 [48:04:52<394:09:57,  6.89s/it]
  8%|▊         | 18992/225000 [48:04:57<358:11:18,  6.26s/it]
  8%|▊         | 18993/225000 [48:05:02<345:45:15,  6.04s/it]
  8%|▊         | 18994/225000 [48:05:08<341:23:43,  5.97s/it]
  8%|▊         | 18995/225000 [48:05:14<339:20:56,  5.93s/it]
  8%|▊         | 18996/225000 [48:05:24<420:25:25,  7.35s/it]
  8%|▊         | 18997/225000 [48:05:29<378:08:32,  6.61s/it]
  8%|▊         | 18998/225000 [48:05:35<362:18:37,  6.33s/it]
  8%|▊         | 18999/225000 [48:05:40<342:15:33,  5.98s/it]
  8%|▊         | 19000/225000 [48:05:53<463:04:01,  8.09s/it]
                                                             
{'loss': '0.7161', 'grad_norm': '0.7061', 'learning_rate': '4.978e-05', 'epoch': '0.08444', 'num_input_tokens_seen': 84592560, 'train_runtime': '1.732e+05', 'train_tokens_per_second': '488.5'}

  8%|▊         | 19000/225000 [48:05:53<463:04:01,  8.09s/it]
  8%|▊         | 19001/225000 [48:05:57<387:06:07,  6.76s/it]
  8%|▊         | 19002/225000 [48:06:05<418:25:05,  7.31s/it]
  8%|▊         | 19003/225000 [48:06:10<364:37:41,  6.37s/it]
  8%|▊         | 19004/225000 [48:06:19<414:11:19,  7.24s/it]
  8%|▊         | 19005/225000 [48:06:25<389:26:12,  6.81s/it]
  8%|▊         | 19006/225000 [48:06:29<353:42:11,  6.18s/it]
  8%|▊         | 19007/225000 [48:06:34<331:43:04,  5.80s/it]
  8%|▊         | 19008/225000 [48:06:47<448:46:09,  7.84s/it]
  8%|▊         | 19009/225000 [48:06:55<457:45:29,  8.00s/it]
  8%|▊         | 19010/225000 [48:07:00<400:31:15,  7.00s/it]
  8%|▊         | 19011/225000 [48:07:09<434:45:56,  7.60s/it]
  8%|▊         | 19012/225000 [48:07:14<383:39:18,  6.71s/it]
  8%|▊         | 19013/225000 [48:07:18<348:40:10,  6.09s/it]
  8%|▊         | 19014/225000 [48:07:28<416:22:54,  7.28s/it]
  8%|▊         | 19015/225000 [48:07:34<398:23:10,  6.96s/it]
  8%|▊         | 19016/225000 [48:07:38<347:43:18,  6.08s/it]
  8%|▊         | 19017/225000 [48:07:51<453:34:21,  7.93s/it]
  8%|▊         | 19018/225000 [48:07:56<406:03:08,  7.10s/it]
  8%|▊         | 19019/225000 [48:08:00<363:04:39,  6.35s/it]
  8%|▊         | 19020/225000 [48:08:06<353:19:23,  6.18s/it]
  8%|▊         | 19021/225000 [48:08:18<454:44:56,  7.95s/it]
  8%|▊         | 19022/225000 [48:08:23<395:01:22,  6.90s/it]
  8%|▊         | 19023/225000 [48:08:31<425:41:11,  7.44s/it]
  8%|▊         | 19024/225000 [48:08:36<374:04:58,  6.54s/it]
  8%|▊         | 19025/225000 [48:08:41<355:35:37,  6.22s/it]
  8%|▊         | 19026/225000 [48:08:48<366:52:58,  6.41s/it]
  8%|▊         | 19027/225000 [48:08:59<436:41:09,  7.63s/it]
  8%|▊         | 19028/225000 [48:09:07<446:17:41,  7.80s/it]
  8%|▊         | 19029/225000 [48:09:14<435:00:34,  7.60s/it]
  8%|▊         | 19030/225000 [48:09:18<368:24:29,  6.44s/it]
  8%|▊         | 19031/225000 [48:09:27<417:17:37,  7.29s/it]
  8%|▊         | 19032/225000 [48:09:32<378:16:42,  6.61s/it]
  8%|▊         | 19033/225000 [48:09:41<415:56:45,  7.27s/it]
  8%|▊         | 19034/225000 [48:09:45<356:25:04,  6.23s/it]
  8%|▊         | 19035/225000 [48:09:53<397:31:22,  6.95s/it]
  8%|▊         | 19036/225000 [48:09:57<346:59:53,  6.07s/it]
  8%|▊         | 19037/225000 [48:10:08<431:38:57,  7.54s/it]
  8%|▊         | 19038/225000 [48:10:12<369:05:04,  6.45s/it]
  8%|▊         | 19039/225000 [48:10:21<413:55:16,  7.23s/it]
  8%|▊         | 19040/225000 [48:10:26<362:09:42,  6.33s/it]
  8%|▊         | 19041/225000 [48:10:34<393:17:44,  6.87s/it]
  8%|▊         | 19042/225000 [48:10:42<411:36:41,  7.19s/it]
  8%|▊         | 19043/225000 [48:10:46<355:19:31,  6.21s/it]
  8%|▊         | 19044/225000 [48:10:55<407:02:19,  7.11s/it]
  8%|▊         | 19045/225000 [48:11:00<368:07:18,  6.43s/it]
  8%|▊         | 19046/225000 [48:11:04<338:37:58,  5.92s/it]
  8%|▊         | 19047/225000 [48:11:11<350:58:46,  6.14s/it]
  8%|▊         | 19048/225000 [48:11:23<452:14:14,  7.91s/it]
  8%|▊         | 19049/225000 [48:11:30<435:32:50,  7.61s/it]
  8%|▊         | 19050/225000 [48:11:34<381:50:19,  6.67s/it]
                                                             
{'loss': '0.7248', 'grad_norm': '0.7149', 'learning_rate': '4.978e-05', 'epoch': '0.08467', 'num_input_tokens_seen': 84746744, 'train_runtime': '1.735e+05', 'train_tokens_per_second': '488.5'}

  8%|▊         | 19050/225000 [48:11:34<381:50:19,  6.67s/it]
  8%|▊         | 19051/225000 [48:11:39<346:34:04,  6.06s/it]
  8%|▊         | 19052/225000 [48:11:44<334:41:28,  5.85s/it]
  8%|▊         | 19053/225000 [48:11:59<484:13:06,  8.46s/it]
  8%|▊         | 19054/225000 [48:12:03<411:42:37,  7.20s/it]
  8%|▊         | 19055/225000 [48:12:11<425:42:40,  7.44s/it]
  8%|▊         | 19056/225000 [48:12:15<362:30:30,  6.34s/it]
  8%|▊         | 19057/225000 [48:12:25<418:33:35,  7.32s/it]
  8%|▊         | 19058/225000 [48:12:32<423:16:41,  7.40s/it]
  8%|▊         | 19059/225000 [48:12:37<374:07:34,  6.54s/it]
  8%|▊         | 19060/225000 [48:12:45<407:39:20,  7.13s/it]
  8%|▊         | 19061/225000 [48:12:51<378:05:39,  6.61s/it]
  8%|▊         | 19062/225000 [48:12:55<343:37:55,  6.01s/it]
  8%|▊         | 19063/225000 [48:13:00<324:05:35,  5.67s/it]
  8%|▊         | 19064/225000 [48:13:11<413:11:35,  7.22s/it]
  8%|▊         | 19065/225000 [48:13:17<388:31:56,  6.79s/it]
  8%|▊         | 19066/225000 [48:13:22<369:22:35,  6.46s/it]
  8%|▊         | 19067/225000 [48:13:28<350:40:47,  6.13s/it]
  8%|▊         | 19068/225000 [48:13:36<391:33:12,  6.84s/it]
  8%|▊         | 19069/225000 [48:13:47<455:34:20,  7.96s/it]
  8%|▊         | 19070/225000 [48:13:54<437:45:22,  7.65s/it]
  8%|▊         | 19071/225000 [48:14:01<431:05:02,  7.54s/it]
  8%|▊         | 19072/225000 [48:14:07<408:21:17,  7.14s/it]
  8%|▊         | 19073/225000 [48:14:12<363:20:03,  6.35s/it]
  8%|▊         | 19074/225000 [48:14:16<330:11:32,  5.77s/it]
  8%|▊         | 19075/225000 [48:14:21<320:06:43,  5.60s/it]
  8%|▊         | 19076/225000 [48:14:27<315:34:12,  5.52s/it]
  8%|▊         | 19077/225000 [48:14:32<316:18:18,  5.53s/it]
  8%|▊         | 19078/225000 [48:14:46<452:55:22,  7.92s/it]
  8%|▊         | 19079/225000 [48:14:52<433:07:35,  7.57s/it]
  8%|▊         | 19080/225000 [48:14:57<385:24:03,  6.74s/it]
  8%|▊         | 19081/225000 [48:15:02<355:12:07,  6.21s/it]
  8%|▊         | 19082/225000 [48:15:15<469:44:12,  8.21s/it]
  8%|▊         | 19083/225000 [48:15:23<469:32:44,  8.21s/it]
  8%|▊         | 19084/225000 [48:15:28<410:39:43,  7.18s/it]
  8%|▊         | 19085/225000 [48:15:34<390:30:20,  6.83s/it]
  8%|▊         | 19086/225000 [48:15:42<415:55:18,  7.27s/it]
  8%|▊         | 19087/225000 [48:15:48<384:53:56,  6.73s/it]
  8%|▊         | 19088/225000 [48:15:53<351:08:59,  6.14s/it]
  8%|▊         | 19089/225000 [48:16:03<420:41:12,  7.35s/it]
  8%|▊         | 19090/225000 [48:16:09<404:15:56,  7.07s/it]
  8%|▊         | 19091/225000 [48:16:13<349:16:36,  6.11s/it]
  8%|▊         | 19092/225000 [48:16:18<334:57:45,  5.86s/it]
  8%|▊         | 19093/225000 [48:16:24<328:05:25,  5.74s/it]
  8%|▊         | 19094/225000 [48:16:29<325:04:29,  5.68s/it]
  8%|▊         | 19095/225000 [48:16:35<325:47:57,  5.70s/it]
  8%|▊         | 19096/225000 [48:16:48<453:36:06,  7.93s/it]
  8%|▊         | 19097/225000 [48:16:57<469:29:27,  8.21s/it]
  8%|▊         | 19098/225000 [48:17:03<427:23:46,  7.47s/it]
  8%|▊         | 19099/225000 [48:17:10<422:21:07,  7.38s/it]
  8%|▊         | 19100/225000 [48:17:18<438:19:02,  7.66s/it]
                                                             
{'loss': '0.7267', 'grad_norm': '0.6643', 'learning_rate': '4.978e-05', 'epoch': '0.08489', 'num_input_tokens_seen': 84898232, 'train_runtime': '1.738e+05', 'train_tokens_per_second': '488.4'}

  8%|▊         | 19100/225000 [48:17:18<438:19:02,  7.66s/it]
  8%|▊         | 19101/225000 [48:17:23<389:14:44,  6.81s/it]
  8%|▊         | 19102/225000 [48:17:29<366:56:55,  6.42s/it]
  8%|▊         | 19103/225000 [48:17:33<338:40:54,  5.92s/it]
  8%|▊         | 19104/225000 [48:17:43<394:30:05,  6.90s/it]
  8%|▊         | 19105/225000 [48:17:53<456:18:50,  7.98s/it]
  8%|▊         | 19106/225000 [48:17:59<416:11:07,  7.28s/it]
  8%|▊         | 19107/225000 [48:18:04<388:14:29,  6.79s/it]
  8%|▊         | 19108/225000 [48:18:10<374:31:00,  6.55s/it]
  8%|▊         | 19109/225000 [48:18:16<351:35:58,  6.15s/it]
  8%|▊         | 19110/225000 [48:18:21<344:22:12,  6.02s/it]
  8%|▊         | 19111/225000 [48:18:27<334:08:21,  5.84s/it]
  8%|▊         | 19112/225000 [48:18:33<340:14:20,  5.95s/it]
  8%|▊         | 19113/225000 [48:18:41<379:59:20,  6.64s/it]
  8%|▊         | 19114/225000 [48:18:54<481:05:17,  8.41s/it]
  8%|▊         | 19115/225000 [48:18:59<430:13:39,  7.52s/it]
  8%|▊         | 19116/225000 [48:19:06<412:23:36,  7.21s/it]
  8%|▊         | 19117/225000 [48:19:11<382:08:56,  6.68s/it]
  8%|▊         | 19118/225000 [48:19:17<360:54:56,  6.31s/it]
  8%|▊         | 19119/225000 [48:19:22<344:28:24,  6.02s/it]
  8%|▊         | 19120/225000 [48:19:27<330:24:22,  5.78s/it]
  8%|▊         | 19121/225000 [48:19:40<451:38:08,  7.90s/it]
  8%|▊         | 19122/225000 [48:19:47<440:30:24,  7.70s/it]
  8%|▊         | 19123/225000 [48:19:55<442:54:11,  7.74s/it]
  8%|▊         | 19124/225000 [48:20:00<388:58:30,  6.80s/it]
  8%|▊         | 19125/225000 [48:20:09<423:18:58,  7.40s/it]
  9%|▊         | 19126/225000 [48:20:13<370:50:42,  6.48s/it]
  9%|▊         | 19127/225000 [48:20:18<347:16:54,  6.07s/it]
  9%|▊         | 19128/225000 [48:20:23<336:15:24,  5.88s/it]
  9%|▊         | 19129/225000 [48:20:34<419:57:09,  7.34s/it]
  9%|▊         | 19130/225000 [48:20:42<431:19:42,  7.54s/it]
  9%|▊         | 19131/225000 [48:20:50<429:28:13,  7.51s/it]
  9%|▊         | 19132/225000 [48:20:56<409:01:43,  7.15s/it]
  9%|▊         | 19133/225000 [48:21:01<376:56:21,  6.59s/it]
  9%|▊         | 19134/225000 [48:21:06<353:11:05,  6.18s/it]
  9%|▊         | 19135/225000 [48:21:12<342:05:30,  5.98s/it]
  9%|▊         | 19136/225000 [48:21:17<332:39:23,  5.82s/it]
  9%|▊         | 19137/225000 [48:21:27<404:51:35,  7.08s/it]
  9%|▊         | 19138/225000 [48:21:36<437:17:21,  7.65s/it]
  9%|▊         | 19139/225000 [48:21:41<389:47:09,  6.82s/it]
  9%|▊         | 19140/225000 [48:21:46<351:34:12,  6.15s/it]
  9%|▊         | 19141/225000 [48:21:57<436:01:17,  7.63s/it]
  9%|▊         | 19142/225000 [48:22:05<448:26:54,  7.84s/it]
  9%|▊         | 19143/225000 [48:22:09<375:53:22,  6.57s/it]
  9%|▊         | 19144/225000 [48:22:19<439:29:09,  7.69s/it]
  9%|▊         | 19145/225000 [48:22:23<372:16:32,  6.51s/it]
  9%|▊         | 19146/225000 [48:22:33<425:28:00,  7.44s/it]
  9%|▊         | 19147/225000 [48:22:37<366:48:42,  6.41s/it]
  9%|▊         | 19148/225000 [48:22:47<441:33:54,  7.72s/it]
  9%|▊         | 19149/225000 [48:22:52<389:57:01,  6.82s/it]
  9%|▊         | 19150/225000 [48:22:56<349:27:31,  6.11s/it]
                                                             
{'loss': '0.7069', 'grad_norm': '0.6261', 'learning_rate': '4.978e-05', 'epoch': '0.08511', 'num_input_tokens_seen': 85049648, 'train_runtime': '1.742e+05', 'train_tokens_per_second': '488.3'}

  9%|▊         | 19150/225000 [48:22:56<349:27:31,  6.11s/it]
  9%|▊         | 19151/225000 [48:23:07<427:50:33,  7.48s/it]
  9%|▊         | 19152/225000 [48:23:13<396:56:21,  6.94s/it]
  9%|▊         | 19153/225000 [48:23:19<376:06:25,  6.58s/it]
  9%|▊         | 19154/225000 [48:23:24<357:49:45,  6.26s/it]
  9%|▊         | 19155/225000 [48:23:30<344:56:20,  6.03s/it]
  9%|▊         | 19156/225000 [48:23:35<337:51:23,  5.91s/it]
  9%|▊         | 19157/225000 [48:23:41<330:56:02,  5.79s/it]
  9%|▊         | 19158/225000 [48:23:47<333:04:00,  5.83s/it]
  9%|▊         | 19159/225000 [48:23:53<334:46:22,  5.85s/it]
  9%|▊         | 19160/225000 [48:24:06<465:26:53,  8.14s/it]
  9%|▊         | 19161/225000 [48:24:13<445:54:15,  7.80s/it]
  9%|▊         | 19162/225000 [48:24:19<414:56:52,  7.26s/it]
  9%|▊         | 19163/225000 [48:24:27<427:49:30,  7.48s/it]
  9%|▊         | 19164/225000 [48:24:32<390:53:50,  6.84s/it]
  9%|▊         | 19165/225000 [48:24:41<413:23:42,  7.23s/it]
  9%|▊         | 19166/225000 [48:24:45<371:10:16,  6.49s/it]
  9%|▊         | 19167/225000 [48:24:50<348:50:02,  6.10s/it]
  9%|▊         | 19168/225000 [48:25:01<432:48:03,  7.57s/it]
  9%|▊         | 19169/225000 [48:25:06<384:12:19,  6.72s/it]
  9%|▊         | 19170/225000 [48:25:11<350:22:57,  6.13s/it]
  9%|▊         | 19171/225000 [48:25:15<318:01:03,  5.56s/it]
  9%|▊         | 19172/225000 [48:25:24<380:09:52,  6.65s/it]
  9%|▊         | 19173/225000 [48:25:32<400:09:37,  7.00s/it]
  9%|▊         | 19174/225000 [48:25:37<368:03:22,  6.44s/it]
  9%|▊         | 19175/225000 [48:25:49<464:03:54,  8.12s/it]
  9%|▊         | 19176/225000 [48:25:53<395:22:22,  6.92s/it]
  9%|▊         | 19177/225000 [48:25:59<376:27:00,  6.58s/it]
  9%|▊         | 19178/225000 [48:26:04<353:07:35,  6.18s/it]
  9%|▊         | 19179/225000 [48:26:10<345:09:07,  6.04s/it]
  9%|▊         | 19180/225000 [48:26:15<327:01:00,  5.72s/it]
  9%|▊         | 19181/225000 [48:26:24<379:35:41,  6.64s/it]
  9%|▊         | 19182/225000 [48:26:32<410:41:20,  7.18s/it]
  9%|▊         | 19183/225000 [48:26:38<384:13:01,  6.72s/it]
  9%|▊         | 19184/225000 [48:26:44<364:39:38,  6.38s/it]
  9%|▊         | 19185/225000 [48:26:49<347:57:43,  6.09s/it]
  9%|▊         | 19186/225000 [48:26:54<332:01:23,  5.81s/it]
  9%|▊         | 19187/225000 [48:26:59<320:26:05,  5.60s/it]
  9%|▊         | 19188/225000 [48:27:08<368:18:07,  6.44s/it]
  9%|▊         | 19189/225000 [48:27:16<406:19:12,  7.11s/it]
  9%|▊         | 19190/225000 [48:27:25<428:00:57,  7.49s/it]
  9%|▊         | 19191/225000 [48:27:29<370:35:36,  6.48s/it]
  9%|▊         | 19192/225000 [48:27:33<334:13:13,  5.85s/it]
  9%|▊         | 19193/225000 [48:27:38<321:07:33,  5.62s/it]
  9%|▊         | 19194/225000 [48:27:44<314:03:17,  5.49s/it]
  9%|▊         | 19195/225000 [48:27:53<383:53:22,  6.72s/it]
  9%|▊         | 19196/225000 [48:28:04<462:03:09,  8.08s/it]
  9%|▊         | 19197/225000 [48:28:09<396:43:47,  6.94s/it]
  9%|▊         | 19198/225000 [48:28:13<356:38:55,  6.24s/it]
  9%|▊         | 19199/225000 [48:28:25<450:26:13,  7.88s/it]
  9%|▊         | 19200/225000 [48:28:31<423:41:18,  7.41s/it]
                                                             
{'loss': '0.7145', 'grad_norm': '0.584', 'learning_rate': '4.977e-05', 'epoch': '0.08533', 'num_input_tokens_seen': 85201560, 'train_runtime': '1.745e+05', 'train_tokens_per_second': '488.2'}

  9%|▊         | 19200/225000 [48:28:31<423:41:18,  7.41s/it]
  9%|▊         | 19201/225000 [48:28:36<379:43:19,  6.64s/it]
  9%|▊         | 19202/225000 [48:28:45<420:07:28,  7.35s/it]
  9%|▊         | 19203/225000 [48:28:50<377:30:56,  6.60s/it]
  9%|▊         | 19204/225000 [48:28:57<381:01:03,  6.67s/it]
  9%|▊         | 19205/225000 [48:29:02<353:03:29,  6.18s/it]
  9%|▊         | 19206/225000 [48:29:11<407:45:14,  7.13s/it]
  9%|▊         | 19207/225000 [48:29:17<390:41:50,  6.83s/it]
  9%|▊         | 19208/225000 [48:29:28<462:45:13,  8.10s/it]
  9%|▊         | 19209/225000 [48:29:36<449:37:41,  7.87s/it]
  9%|▊         | 19210/225000 [48:29:40<385:22:50,  6.74s/it]
  9%|▊         | 19211/225000 [48:29:45<362:25:06,  6.34s/it]
  9%|▊         | 19212/225000 [48:29:50<332:47:41,  5.82s/it]
  9%|▊         | 19213/225000 [48:29:55<321:47:57,  5.63s/it]
  9%|▊         | 19214/225000 [48:30:01<323:20:05,  5.66s/it]
  9%|▊         | 19215/225000 [48:30:06<315:40:12,  5.52s/it]
  9%|▊         | 19216/225000 [48:30:18<425:43:39,  7.45s/it]
  9%|▊         | 19217/225000 [48:30:27<446:32:05,  7.81s/it]
  9%|▊         | 19218/225000 [48:30:30<376:02:21,  6.58s/it]
  9%|▊         | 19219/225000 [48:30:38<400:21:05,  7.00s/it]
  9%|▊         | 19220/225000 [48:30:44<370:26:42,  6.48s/it]
  9%|▊         | 19221/225000 [48:30:51<387:34:20,  6.78s/it]
  9%|▊         | 19222/225000 [48:30:56<351:36:56,  6.15s/it]
  9%|▊         | 19223/225000 [48:31:04<391:20:38,  6.85s/it]
  9%|▊         | 19224/225000 [48:31:13<423:06:15,  7.40s/it]
  9%|▊         | 19225/225000 [48:31:17<371:46:20,  6.50s/it]
  9%|▊         | 19226/225000 [48:31:22<338:01:12,  5.91s/it]
  9%|▊         | 19227/225000 [48:31:33<422:02:19,  7.38s/it]
  9%|▊         | 19228/225000 [48:31:37<365:41:22,  6.40s/it]
  9%|▊         | 19229/225000 [48:31:41<326:21:26,  5.71s/it]
  9%|▊         | 19230/225000 [48:31:53<429:12:40,  7.51s/it]
  9%|▊         | 19231/225000 [48:31:58<387:33:30,  6.78s/it]
  9%|▊         | 19232/225000 [48:32:03<369:04:03,  6.46s/it]
  9%|▊         | 19233/225000 [48:32:09<352:23:54,  6.17s/it]
  9%|▊         | 19234/225000 [48:32:14<342:29:55,  5.99s/it]
  9%|▊         | 19235/225000 [48:32:20<340:11:00,  5.95s/it]
  9%|▊         | 19236/225000 [48:32:26<340:31:22,  5.96s/it]
  9%|▊         | 19237/225000 [48:32:34<371:18:36,  6.50s/it]
  9%|▊         | 19238/225000 [48:32:46<461:33:10,  8.08s/it]
  9%|▊         | 19239/225000 [48:32:52<426:53:31,  7.47s/it]
  9%|▊         | 19240/225000 [48:32:58<403:55:33,  7.07s/it]
  9%|▊         | 19241/225000 [48:33:05<405:08:23,  7.09s/it]
  9%|▊         | 19242/225000 [48:33:10<375:01:00,  6.56s/it]
  9%|▊         | 19243/225000 [48:33:16<350:40:54,  6.14s/it]
  9%|▊         | 19244/225000 [48:33:21<338:28:02,  5.92s/it]
  9%|▊         | 19245/225000 [48:33:29<372:49:06,  6.52s/it]
  9%|▊         | 19246/225000 [48:33:40<456:22:41,  7.99s/it]
  9%|▊         | 19247/225000 [48:33:45<402:41:48,  7.05s/it]
  9%|▊         | 19248/225000 [48:33:49<356:52:48,  6.24s/it]
  9%|▊         | 19249/225000 [48:33:57<373:38:59,  6.54s/it]
  9%|▊         | 19250/225000 [48:34:06<414:03:41,  7.24s/it]
                                                             
{'loss': '0.7067', 'grad_norm': '0.5608', 'learning_rate': '4.977e-05', 'epoch': '0.08556', 'num_input_tokens_seen': 85354136, 'train_runtime': '1.748e+05', 'train_tokens_per_second': '488.2'}

  9%|▊         | 19250/225000 [48:34:06<414:03:41,  7.24s/it]
  9%|▊         | 19251/225000 [48:34:11<385:30:16,  6.75s/it]
  9%|▊         | 19252/225000 [48:34:19<399:44:18,  6.99s/it]
  9%|▊         | 19253/225000 [48:34:24<374:02:20,  6.54s/it]
  9%|▊         | 19254/225000 [48:34:30<358:35:49,  6.27s/it]
  9%|▊         | 19255/225000 [48:34:35<340:00:02,  5.95s/it]
  9%|▊         | 19256/225000 [48:34:41<336:50:09,  5.89s/it]
  9%|▊         | 19257/225000 [48:34:52<420:46:37,  7.36s/it]
  9%|▊         | 19258/225000 [48:34:57<387:10:19,  6.77s/it]
  9%|▊         | 19259/225000 [48:35:03<377:21:46,  6.60s/it]
  9%|▊         | 19260/225000 [48:35:09<355:30:23,  6.22s/it]
  9%|▊         | 19261/225000 [48:35:14<343:46:15,  6.02s/it]
  9%|▊         | 19262/225000 [48:35:20<338:58:09,  5.93s/it]
  9%|▊         | 19263/225000 [48:35:26<348:25:01,  6.10s/it]
  9%|▊         | 19264/225000 [48:35:40<470:25:06,  8.23s/it]
  9%|▊         | 19265/225000 [48:35:45<423:58:14,  7.42s/it]
  9%|▊         | 19266/225000 [48:35:51<398:49:59,  6.98s/it]
  9%|▊         | 19267/225000 [48:35:58<404:53:59,  7.09s/it]
  9%|▊         | 19268/225000 [48:36:03<355:54:01,  6.23s/it]
  9%|▊         | 19269/225000 [48:36:13<428:52:44,  7.50s/it]
  9%|▊         | 19270/225000 [48:36:17<363:44:43,  6.37s/it]
  9%|▊         | 19271/225000 [48:36:25<393:43:04,  6.89s/it]
  9%|▊         | 19272/225000 [48:36:30<360:20:28,  6.31s/it]
  9%|▊         | 19273/225000 [48:36:39<405:24:21,  7.09s/it]
  9%|▊         | 19274/225000 [48:36:47<426:25:44,  7.46s/it]
  9%|▊         | 19275/225000 [48:36:55<429:58:55,  7.52s/it]
  9%|▊         | 19276/225000 [48:36:58<364:45:28,  6.38s/it]
  9%|▊         | 19277/225000 [48:37:04<342:12:34,  5.99s/it]
  9%|▊         | 19278/225000 [48:37:09<336:24:18,  5.89s/it]
  9%|▊         | 19279/225000 [48:37:15<326:47:00,  5.72s/it]
  9%|▊         | 19280/225000 [48:37:20<320:50:30,  5.61s/it]
  9%|▊         | 19281/225000 [48:37:26<335:08:59,  5.86s/it]
  9%|▊         | 19282/225000 [48:37:39<452:00:21,  7.91s/it]
  9%|▊         | 19283/225000 [48:37:43<393:09:25,  6.88s/it]
  9%|▊         | 19284/225000 [48:37:49<361:14:26,  6.32s/it]
  9%|▊         | 19285/225000 [48:37:54<344:56:55,  6.04s/it]
  9%|▊         | 19286/225000 [48:38:01<360:00:35,  6.30s/it]
  9%|▊         | 19287/225000 [48:38:13<461:25:19,  8.07s/it]
  9%|▊         | 19288/225000 [48:38:18<411:44:04,  7.21s/it]
  9%|▊         | 19289/225000 [48:38:24<381:11:59,  6.67s/it]
  9%|▊         | 19290/225000 [48:38:31<387:34:34,  6.78s/it]
  9%|▊         | 19291/225000 [48:38:37<378:02:01,  6.62s/it]
  9%|▊         | 19292/225000 [48:38:45<399:41:29,  6.99s/it]
  9%|▊         | 19293/225000 [48:38:50<375:34:23,  6.57s/it]
  9%|▊         | 19294/225000 [48:38:58<386:36:39,  6.77s/it]
  9%|▊         | 19295/225000 [48:39:06<411:20:19,  7.20s/it]
  9%|▊         | 19296/225000 [48:39:11<385:53:00,  6.75s/it]
  9%|▊         | 19297/225000 [48:39:16<341:32:27,  5.98s/it]
  9%|▊         | 19298/225000 [48:39:21<322:49:15,  5.65s/it]
  9%|▊         | 19299/225000 [48:39:28<350:53:40,  6.14s/it]
  9%|▊         | 19300/225000 [48:39:39<435:05:15,  7.61s/it]
                                                             
{'loss': '0.7268', 'grad_norm': '0.6548', 'learning_rate': '4.977e-05', 'epoch': '0.08578', 'num_input_tokens_seen': 85505208, 'train_runtime': '1.752e+05', 'train_tokens_per_second': '488.1'}

  9%|▊         | 19300/225000 [48:39:39<435:05:15,  7.61s/it]
  9%|▊         | 19301/225000 [48:39:43<368:50:10,  6.46s/it]
  9%|▊         | 19302/225000 [48:39:48<352:50:19,  6.18s/it]
  9%|▊         | 19303/225000 [48:39:58<416:28:58,  7.29s/it]
  9%|▊         | 19304/225000 [48:40:02<361:19:41,  6.32s/it]
  9%|▊         | 19305/225000 [48:40:07<331:49:54,  5.81s/it]
  9%|▊         | 19306/225000 [48:40:12<325:32:21,  5.70s/it]
  9%|▊         | 19307/225000 [48:40:17<317:12:33,  5.55s/it]
  9%|▊         | 19308/225000 [48:40:30<445:23:55,  7.80s/it]
  9%|▊         | 19309/225000 [48:40:36<404:03:04,  7.07s/it]
  9%|▊         | 19310/225000 [48:40:44<415:05:45,  7.27s/it]
  9%|▊         | 19311/225000 [48:40:47<358:50:19,  6.28s/it]
  9%|▊         | 19312/225000 [48:40:58<425:35:38,  7.45s/it]
  9%|▊         | 19313/225000 [48:41:03<385:47:59,  6.75s/it]
  9%|▊         | 19314/225000 [48:41:08<363:12:35,  6.36s/it]
  9%|▊         | 19315/225000 [48:41:14<352:11:07,  6.16s/it]
  9%|▊         | 19316/225000 [48:41:20<342:01:24,  5.99s/it]
  9%|▊         | 19317/225000 [48:41:25<334:21:17,  5.85s/it]
  9%|▊         | 19318/225000 [48:41:34<388:49:38,  6.81s/it]
  9%|▊         | 19319/225000 [48:41:48<517:26:22,  9.06s/it]
  9%|▊         | 19320/225000 [48:41:52<430:09:57,  7.53s/it]
  9%|▊         | 19321/225000 [48:41:57<383:25:26,  6.71s/it]
  9%|▊         | 19322/225000 [48:42:02<350:06:23,  6.13s/it]
  9%|▊         | 19323/225000 [48:42:16<480:04:22,  8.40s/it]
  9%|▊         | 19324/225000 [48:42:20<418:56:22,  7.33s/it]
  9%|▊         | 19325/225000 [48:42:26<384:21:20,  6.73s/it]
  9%|▊         | 19326/225000 [48:42:31<353:09:02,  6.18s/it]
  9%|▊         | 19327/225000 [48:42:36<343:43:43,  6.02s/it]
  9%|▊         | 19328/225000 [48:42:41<322:25:48,  5.64s/it]
  9%|▊         | 19329/225000 [48:42:52<411:36:42,  7.20s/it]
  9%|▊         | 19330/225000 [48:42:57<378:14:56,  6.62s/it]
  9%|▊         | 19331/225000 [48:43:07<436:05:30,  7.63s/it]
  9%|▊         | 19332/225000 [48:43:11<377:45:15,  6.61s/it]
  9%|▊         | 19333/225000 [48:43:16<343:25:26,  6.01s/it]
  9%|▊         | 19334/225000 [48:43:21<330:33:36,  5.79s/it]
  9%|▊         | 19335/225000 [48:43:27<331:02:57,  5.79s/it]
  9%|▊         | 19336/225000 [48:43:33<339:49:55,  5.95s/it]
  9%|▊         | 19337/225000 [48:43:48<484:38:36,  8.48s/it]
  9%|▊         | 19338/225000 [48:43:52<416:55:59,  7.30s/it]
  9%|▊         | 19339/225000 [48:43:57<377:20:59,  6.61s/it]
  9%|▊         | 19340/225000 [48:44:02<345:15:10,  6.04s/it]
  9%|▊         | 19341/225000 [48:44:14<440:51:21,  7.72s/it]
  9%|▊         | 19342/225000 [48:44:20<423:59:41,  7.42s/it]
  9%|▊         | 19343/225000 [48:44:25<374:12:27,  6.55s/it]
  9%|▊         | 19344/225000 [48:44:29<332:56:24,  5.83s/it]
  9%|▊         | 19345/225000 [48:44:42<447:10:31,  7.83s/it]
  9%|▊         | 19346/225000 [48:44:46<393:42:08,  6.89s/it]
  9%|▊         | 19347/225000 [48:44:55<419:54:33,  7.35s/it]
  9%|▊         | 19348/225000 [48:45:00<381:17:09,  6.67s/it]
  9%|▊         | 19349/225000 [48:45:08<406:10:00,  7.11s/it]
  9%|▊         | 19350/225000 [48:45:13<368:53:13,  6.46s/it]
                                                             
{'loss': '0.7293', 'grad_norm': '0.6173', 'learning_rate': '4.977e-05', 'epoch': '0.086', 'num_input_tokens_seen': 85659272, 'train_runtime': '1.755e+05', 'train_tokens_per_second': '488.1'}

  9%|▊         | 19350/225000 [48:45:13<368:53:13,  6.46s/it]
  9%|▊         | 19351/225000 [48:45:17<337:36:28,  5.91s/it]
  9%|▊         | 19352/225000 [48:45:24<341:02:24,  5.97s/it]
  9%|▊         | 19353/225000 [48:45:29<335:43:50,  5.88s/it]
  9%|▊         | 19354/225000 [48:45:43<466:27:08,  8.17s/it]
  9%|▊         | 19355/225000 [48:45:46<389:37:57,  6.82s/it]
  9%|▊         | 19356/225000 [48:45:55<419:29:50,  7.34s/it]
  9%|▊         | 19357/225000 [48:46:02<419:59:43,  7.35s/it]
  9%|▊         | 19358/225000 [48:46:07<375:29:13,  6.57s/it]
  9%|▊         | 19359/225000 [48:46:19<471:20:36,  8.25s/it]
  9%|▊         | 19360/225000 [48:46:24<413:38:52,  7.24s/it]
  9%|▊         | 19361/225000 [48:46:31<397:48:30,  6.96s/it]
  9%|▊         | 19362/225000 [48:46:35<362:41:05,  6.35s/it]
  9%|▊         | 19363/225000 [48:46:44<403:14:09,  7.06s/it]
  9%|▊         | 19364/225000 [48:46:51<402:50:36,  7.05s/it]
  9%|▊         | 19365/225000 [48:46:58<394:59:50,  6.92s/it]
  9%|▊         | 19366/225000 [48:47:02<343:06:36,  6.01s/it]
  9%|▊         | 19367/225000 [48:47:12<424:18:18,  7.43s/it]
  9%|▊         | 19368/225000 [48:47:18<388:38:23,  6.80s/it]
  9%|▊         | 19369/225000 [48:47:22<345:49:32,  6.05s/it]
  9%|▊         | 19370/225000 [48:47:33<426:42:47,  7.47s/it]
  9%|▊         | 19371/225000 [48:47:37<364:07:53,  6.37s/it]
  9%|▊         | 19372/225000 [48:47:46<414:34:31,  7.26s/it]
  9%|▊         | 19373/225000 [48:47:50<365:36:20,  6.40s/it]
  9%|▊         | 19374/225000 [48:47:58<393:17:15,  6.89s/it]
  9%|▊         | 19375/225000 [48:48:03<351:12:35,  6.15s/it]
  9%|▊         | 19376/225000 [48:48:12<407:29:23,  7.13s/it]
  9%|▊         | 19377/225000 [48:48:17<373:11:21,  6.53s/it]
  9%|▊         | 19378/225000 [48:48:24<366:11:43,  6.41s/it]
  9%|▊         | 19379/225000 [48:48:29<347:41:25,  6.09s/it]
  9%|▊         | 19380/225000 [48:48:34<328:18:17,  5.75s/it]
  9%|▊         | 19381/225000 [48:48:40<328:57:17,  5.76s/it]
  9%|▊         | 19382/225000 [48:48:45<322:46:58,  5.65s/it]
  9%|▊         | 19383/225000 [48:48:51<320:56:22,  5.62s/it]
  9%|▊         | 19384/225000 [48:48:56<321:18:07,  5.63s/it]
  9%|▊         | 19385/225000 [48:49:08<424:07:44,  7.43s/it]
  9%|▊         | 19386/225000 [48:49:15<413:28:52,  7.24s/it]
  9%|▊         | 19387/225000 [48:49:19<367:29:11,  6.43s/it]
  9%|▊         | 19388/225000 [48:49:24<335:49:51,  5.88s/it]
  9%|▊         | 19389/225000 [48:49:31<362:45:28,  6.35s/it]
  9%|▊         | 19390/225000 [48:49:42<442:02:27,  7.74s/it]
  9%|▊         | 19391/225000 [48:49:46<378:09:55,  6.62s/it]
  9%|▊         | 19392/225000 [48:49:51<339:00:12,  5.94s/it]
  9%|▊         | 19393/225000 [48:50:02<434:28:23,  7.61s/it]
  9%|▊         | 19394/225000 [48:50:07<382:37:07,  6.70s/it]
  9%|▊         | 19395/225000 [48:50:12<355:36:59,  6.23s/it]
  9%|▊         | 19396/225000 [48:50:21<409:56:35,  7.18s/it]
  9%|▊         | 19397/225000 [48:50:32<465:58:59,  8.16s/it]
  9%|▊         | 19398/225000 [48:50:36<397:58:16,  6.97s/it]
  9%|▊         | 19399/225000 [48:50:40<358:33:49,  6.28s/it]
  9%|▊         | 19400/225000 [48:50:51<432:11:35,  7.57s/it]
                                                             
{'loss': '0.712', 'grad_norm': '0.6807', 'learning_rate': '4.977e-05', 'epoch': '0.08622', 'num_input_tokens_seen': 85812872, 'train_runtime': '1.759e+05', 'train_tokens_per_second': '488'}

  9%|▊         | 19400/225000 [48:50:51<432:11:35,  7.57s/it]
  9%|▊         | 19401/225000 [48:50:56<381:35:52,  6.68s/it]
  9%|▊         | 19402/225000 [48:51:01<350:50:44,  6.14s/it]
  9%|▊         | 19403/225000 [48:51:06<332:06:51,  5.82s/it]
  9%|▊         | 19404/225000 [48:51:16<418:14:26,  7.32s/it]
  9%|▊         | 19405/225000 [48:51:23<401:14:19,  7.03s/it]
  9%|▊         | 19406/225000 [48:51:27<356:01:49,  6.23s/it]
  9%|▊         | 19407/225000 [48:51:38<431:28:32,  7.56s/it]
  9%|▊         | 19408/225000 [48:51:44<403:15:32,  7.06s/it]
  9%|▊         | 19409/225000 [48:51:53<436:17:24,  7.64s/it]
  9%|▊         | 19410/225000 [48:52:01<442:09:10,  7.74s/it]
  9%|▊         | 19411/225000 [48:52:07<418:06:51,  7.32s/it]
  9%|▊         | 19412/225000 [48:52:12<377:16:36,  6.61s/it]
  9%|▊         | 19413/225000 [48:52:19<393:22:49,  6.89s/it]
  9%|▊         | 19414/225000 [48:52:24<344:43:44,  6.04s/it]
  9%|▊         | 19415/225000 [48:52:33<400:33:54,  7.01s/it]
  9%|▊         | 19416/225000 [48:52:37<347:50:14,  6.09s/it]
  9%|▊         | 19417/225000 [48:52:46<401:05:18,  7.02s/it]
  9%|▊         | 19418/225000 [48:52:50<348:12:05,  6.10s/it]
  9%|▊         | 19419/225000 [48:52:59<405:01:26,  7.09s/it]
  9%|▊         | 19420/225000 [48:53:03<348:46:42,  6.11s/it]
  9%|▊         | 19421/225000 [48:53:11<383:59:16,  6.72s/it]
  9%|▊         | 19422/225000 [48:53:15<339:08:56,  5.94s/it]
  9%|▊         | 19423/225000 [48:53:21<331:31:09,  5.81s/it]
  9%|▊         | 19424/225000 [48:53:33<439:36:42,  7.70s/it]
  9%|▊         | 19425/225000 [48:53:37<372:17:51,  6.52s/it]
  9%|▊         | 19426/225000 [48:53:48<456:28:31,  7.99s/it]
  9%|▊         | 19427/225000 [48:53:52<378:35:21,  6.63s/it]
  9%|▊         | 19428/225000 [48:53:56<346:47:27,  6.07s/it]
  9%|▊         | 19429/225000 [48:54:02<346:29:53,  6.07s/it]
  9%|▊         | 19430/225000 [48:54:08<335:05:17,  5.87s/it]
  9%|▊         | 19431/225000 [48:54:14<331:32:55,  5.81s/it]
  9%|▊         | 19432/225000 [48:54:26<440:48:17,  7.72s/it]
  9%|▊         | 19433/225000 [48:54:31<400:32:50,  7.01s/it]
  9%|▊         | 19434/225000 [48:54:38<395:40:55,  6.93s/it]
  9%|▊         | 19435/225000 [48:54:45<391:56:01,  6.86s/it]
  9%|▊         | 19436/225000 [48:54:52<399:47:46,  7.00s/it]
  9%|▊         | 19437/225000 [48:54:57<371:49:38,  6.51s/it]
  9%|▊         | 19438/225000 [48:55:04<373:44:02,  6.55s/it]
  9%|▊         | 19439/225000 [48:55:10<375:24:15,  6.57s/it]
  9%|▊         | 19440/225000 [48:55:15<339:29:40,  5.95s/it]
  9%|▊         | 19441/225000 [48:55:20<316:26:20,  5.54s/it]
  9%|▊         | 19442/225000 [48:55:25<307:37:12,  5.39s/it]
  9%|▊         | 19443/225000 [48:55:38<450:12:37,  7.88s/it]
  9%|▊         | 19444/225000 [48:55:45<437:01:29,  7.65s/it]
  9%|▊         | 19445/225000 [48:55:54<456:56:30,  8.00s/it]
  9%|▊         | 19446/225000 [48:55:59<400:42:20,  7.02s/it]
  9%|▊         | 19447/225000 [48:56:07<413:29:56,  7.24s/it]
  9%|▊         | 19448/225000 [48:56:13<402:39:38,  7.05s/it]
  9%|▊         | 19449/225000 [48:56:18<355:09:24,  6.22s/it]
  9%|▊         | 19450/225000 [48:56:23<332:36:53,  5.83s/it]
                                                             
{'loss': '0.7047', 'grad_norm': '0.692', 'learning_rate': '4.976e-05', 'epoch': '0.08644', 'num_input_tokens_seen': 85965024, 'train_runtime': '1.762e+05', 'train_tokens_per_second': '487.9'}

  9%|▊         | 19450/225000 [48:56:23<332:36:53,  5.83s/it]
  9%|▊         | 19451/225000 [48:56:31<379:25:44,  6.65s/it]
  9%|▊         | 19452/225000 [48:56:39<403:04:56,  7.06s/it]
  9%|▊         | 19453/225000 [48:56:45<388:47:51,  6.81s/it]
  9%|▊         | 19454/225000 [48:56:50<355:51:02,  6.23s/it]
  9%|▊         | 19455/225000 [48:56:56<341:43:07,  5.99s/it]
  9%|▊         | 19456/225000 [48:57:07<428:05:05,  7.50s/it]
  9%|▊         | 19457/225000 [48:57:12<389:12:54,  6.82s/it]
  9%|▊         | 19458/225000 [48:57:17<357:15:50,  6.26s/it]
  9%|▊         | 19459/225000 [48:57:27<417:57:01,  7.32s/it]
  9%|▊         | 19460/225000 [48:57:31<369:56:54,  6.48s/it]
  9%|▊         | 19461/225000 [48:57:41<420:43:45,  7.37s/it]
  9%|▊         | 19462/225000 [48:57:44<360:10:15,  6.31s/it]
  9%|▊         | 19463/225000 [48:57:53<406:34:21,  7.12s/it]
  9%|▊         | 19464/225000 [48:57:59<372:43:45,  6.53s/it]
  9%|▊         | 19465/225000 [48:58:03<342:42:49,  6.00s/it]
  9%|▊         | 19466/225000 [48:58:09<334:16:47,  5.86s/it]
  9%|▊         | 19467/225000 [48:58:14<323:49:25,  5.67s/it]
  9%|▊         | 19468/225000 [48:58:27<454:23:34,  7.96s/it]
  9%|▊         | 19469/225000 [48:58:33<415:00:44,  7.27s/it]
  9%|▊         | 19470/225000 [48:58:40<400:47:48,  7.02s/it]
  9%|▊         | 19471/225000 [48:58:48<423:00:12,  7.41s/it]
  9%|▊         | 19472/225000 [48:58:52<366:06:23,  6.41s/it]
  9%|▊         | 19473/225000 [48:59:00<395:50:29,  6.93s/it]
  9%|▊         | 19474/225000 [48:59:05<354:02:14,  6.20s/it]
  9%|▊         | 19475/225000 [48:59:14<403:23:14,  7.07s/it]
  9%|▊         | 19476/225000 [48:59:19<380:02:01,  6.66s/it]
  9%|▊         | 19477/225000 [48:59:26<380:44:41,  6.67s/it]
  9%|▊         | 19478/225000 [48:59:31<358:26:58,  6.28s/it]
  9%|▊         | 19479/225000 [48:59:36<337:47:55,  5.92s/it]
  9%|▊         | 19480/225000 [48:59:43<345:39:12,  6.05s/it]
  9%|▊         | 19481/225000 [48:59:57<488:24:17,  8.56s/it]
  9%|▊         | 19482/225000 [49:00:02<431:08:32,  7.55s/it]
  9%|▊         | 19483/225000 [49:00:08<394:11:17,  6.90s/it]
  9%|▊         | 19484/225000 [49:00:12<338:25:37,  5.93s/it]
  9%|▊         | 19485/225000 [49:00:20<388:01:25,  6.80s/it]
  9%|▊         | 19486/225000 [49:00:26<363:27:56,  6.37s/it]
  9%|▊         | 19487/225000 [49:00:34<404:04:09,  7.08s/it]
  9%|▊         | 19488/225000 [49:00:40<385:31:01,  6.75s/it]
  9%|▊         | 19489/225000 [49:00:50<425:46:35,  7.46s/it]
  9%|▊         | 19490/225000 [49:00:56<406:54:13,  7.13s/it]
  9%|▊         | 19491/225000 [49:01:01<377:48:48,  6.62s/it]
  9%|▊         | 19492/225000 [49:01:06<346:04:45,  6.06s/it]
  9%|▊         | 19493/225000 [49:01:11<324:32:06,  5.69s/it]
  9%|▊         | 19494/225000 [49:01:21<397:36:25,  6.97s/it]
  9%|▊         | 19495/225000 [49:01:30<434:07:51,  7.61s/it]
  9%|▊         | 19496/225000 [49:01:34<376:28:53,  6.60s/it]
  9%|▊         | 19497/225000 [49:01:44<431:06:45,  7.55s/it]
  9%|▊         | 19498/225000 [49:01:48<373:19:34,  6.54s/it]
  9%|▊         | 19499/225000 [49:01:54<354:59:02,  6.22s/it]
  9%|▊         | 19500/225000 [49:02:05<438:47:40,  7.69s/it]
                                                             
{'loss': '0.7111', 'grad_norm': '0.6249', 'learning_rate': '4.976e-05', 'epoch': '0.08667', 'num_input_tokens_seen': 86117320, 'train_runtime': '1.765e+05', 'train_tokens_per_second': '487.8'}

  9%|▊         | 19500/225000 [49:02:05<438:47:40,  7.69s/it]
  9%|▊         | 19501/225000 [49:02:10<395:10:38,  6.92s/it]
  9%|▊         | 19502/225000 [49:02:15<356:45:40,  6.25s/it]
  9%|▊         | 19503/225000 [49:02:20<335:55:21,  5.88s/it]
  9%|▊         | 19504/225000 [49:02:25<321:14:43,  5.63s/it]
  9%|▊         | 19505/225000 [49:02:33<366:45:50,  6.43s/it]
  9%|▊         | 19506/225000 [49:02:45<467:57:21,  8.20s/it]
  9%|▊         | 19507/225000 [49:02:52<446:49:19,  7.83s/it]
  9%|▊         | 19508/225000 [49:02:58<417:49:15,  7.32s/it]
  9%|▊         | 19509/225000 [49:03:04<396:26:27,  6.95s/it]
  9%|▊         | 19510/225000 [49:03:11<384:27:34,  6.74s/it]
  9%|▊         | 19511/225000 [49:03:18<387:37:39,  6.79s/it]
  9%|▊         | 19512/225000 [49:03:26<412:05:44,  7.22s/it]
  9%|▊         | 19513/225000 [49:03:33<410:57:21,  7.20s/it]
  9%|▊         | 19514/225000 [49:03:39<386:51:57,  6.78s/it]
  9%|▊         | 19515/225000 [49:03:44<361:42:48,  6.34s/it]
  9%|▊         | 19516/225000 [49:03:52<383:41:25,  6.72s/it]
  9%|▊         | 19517/225000 [49:03:56<351:02:45,  6.15s/it]
  9%|▊         | 19518/225000 [49:04:01<326:01:48,  5.71s/it]
  9%|▊         | 19519/225000 [49:04:12<420:16:03,  7.36s/it]
  9%|▊         | 19520/225000 [49:04:17<370:20:12,  6.49s/it]
  9%|▊         | 19521/225000 [49:04:21<335:11:35,  5.87s/it]
  9%|▊         | 19522/225000 [49:04:29<361:48:37,  6.34s/it]
  9%|▊         | 19523/225000 [49:04:40<449:09:58,  7.87s/it]
  9%|▊         | 19524/225000 [49:04:44<379:57:09,  6.66s/it]
  9%|▊         | 19525/225000 [49:04:50<365:03:43,  6.40s/it]
  9%|▊         | 19526/225000 [49:04:57<379:18:54,  6.65s/it]
  9%|▊         | 19527/225000 [49:05:03<363:16:35,  6.36s/it]
  9%|▊         | 19528/225000 [49:05:11<396:56:07,  6.95s/it]
  9%|▊         | 19529/225000 [49:05:15<351:50:59,  6.16s/it]
  9%|▊         | 19530/225000 [49:05:20<323:19:07,  5.66s/it]
  9%|▊         | 19531/225000 [49:05:29<385:51:41,  6.76s/it]
  9%|▊         | 19532/225000 [49:05:37<405:08:42,  7.10s/it]
  9%|▊         | 19533/225000 [49:05:44<402:15:00,  7.05s/it]
  9%|▊         | 19534/225000 [49:05:48<350:03:13,  6.13s/it]
  9%|▊         | 19535/225000 [49:05:58<411:27:29,  7.21s/it]
  9%|▊         | 19536/225000 [49:06:04<396:35:30,  6.95s/it]
  9%|▊         | 19537/225000 [49:06:09<359:37:18,  6.30s/it]
  9%|▊         | 19538/225000 [49:06:14<334:07:14,  5.85s/it]
  9%|▊         | 19539/225000 [49:06:19<323:13:54,  5.66s/it]
  9%|▊         | 19540/225000 [49:06:31<426:46:14,  7.48s/it]
  9%|▊         | 19541/225000 [49:06:37<410:28:45,  7.19s/it]
  9%|▊         | 19542/225000 [49:06:43<389:57:10,  6.83s/it]
  9%|▊         | 19543/225000 [49:06:47<344:56:11,  6.04s/it]
  9%|▊         | 19544/225000 [49:06:54<353:07:20,  6.19s/it]
  9%|▊         | 19545/225000 [49:07:04<426:14:19,  7.47s/it]
  9%|▊         | 19546/225000 [49:07:08<366:27:44,  6.42s/it]
  9%|▊         | 19547/225000 [49:07:17<400:22:01,  7.02s/it]
  9%|▊         | 19548/225000 [49:07:24<401:50:33,  7.04s/it]
  9%|▊         | 19549/225000 [49:07:30<382:14:05,  6.70s/it]
  9%|▊         | 19550/225000 [49:07:35<362:34:03,  6.35s/it]
                                                             
{'loss': '0.7001', 'grad_norm': '0.6867', 'learning_rate': '4.976e-05', 'epoch': '0.08689', 'num_input_tokens_seen': 86269752, 'train_runtime': '1.769e+05', 'train_tokens_per_second': '487.8'}

  9%|▊         | 19550/225000 [49:07:35<362:34:03,  6.35s/it]
  9%|▊         | 19551/225000 [49:07:43<389:04:58,  6.82s/it]
  9%|▊         | 19552/225000 [49:07:47<345:10:58,  6.05s/it]
  9%|▊         | 19553/225000 [49:07:58<426:38:43,  7.48s/it]
  9%|▊         | 19554/225000 [49:08:04<390:46:58,  6.85s/it]
  9%|▊         | 19555/225000 [49:08:07<340:51:08,  5.97s/it]
  9%|▊         | 19556/225000 [49:08:20<455:34:56,  7.98s/it]
  9%|▊         | 19557/225000 [49:08:24<389:14:57,  6.82s/it]
  9%|▊         | 19558/225000 [49:08:33<414:07:22,  7.26s/it]
  9%|▊         | 19559/225000 [49:08:38<388:03:36,  6.80s/it]
  9%|▊         | 19560/225000 [49:08:44<366:03:27,  6.41s/it]
  9%|▊         | 19561/225000 [49:08:50<355:55:55,  6.24s/it]
  9%|▊         | 19562/225000 [49:08:55<345:18:00,  6.05s/it]
  9%|▊         | 19563/225000 [49:09:01<336:04:18,  5.89s/it]
  9%|▊         | 19564/225000 [49:09:06<325:36:37,  5.71s/it]
  9%|▊         | 19565/225000 [49:09:14<357:07:31,  6.26s/it]
  9%|▊         | 19566/225000 [49:09:27<479:08:57,  8.40s/it]
  9%|▊         | 19567/225000 [49:09:32<421:02:09,  7.38s/it]
  9%|▊         | 19568/225000 [49:09:41<445:22:06,  7.80s/it]
  9%|▊         | 19569/225000 [49:09:48<429:15:05,  7.52s/it]
  9%|▊         | 19570/225000 [49:09:52<371:12:08,  6.51s/it]
  9%|▊         | 19571/225000 [49:09:56<335:35:20,  5.88s/it]
  9%|▊         | 19572/225000 [49:10:02<328:29:20,  5.76s/it]
  9%|▊         | 19573/225000 [49:10:12<408:33:52,  7.16s/it]
  9%|▊         | 19574/225000 [49:10:20<423:08:31,  7.42s/it]
  9%|▊         | 19575/225000 [49:10:24<364:58:27,  6.40s/it]
  9%|▊         | 19576/225000 [49:10:30<361:49:29,  6.34s/it]
  9%|▊         | 19577/225000 [49:10:39<408:32:09,  7.16s/it]
  9%|▊         | 19578/225000 [49:10:45<384:50:10,  6.74s/it]
  9%|▊         | 19579/225000 [49:10:52<392:48:27,  6.88s/it]
  9%|▊         | 19580/225000 [49:10:57<358:31:58,  6.28s/it]
  9%|▊         | 19581/225000 [49:11:08<439:39:29,  7.71s/it]
  9%|▊         | 19582/225000 [49:11:13<382:05:48,  6.70s/it]
  9%|▊         | 19583/225000 [49:11:18<365:37:25,  6.41s/it]
  9%|▊         | 19584/225000 [49:11:24<350:10:30,  6.14s/it]
  9%|▊         | 19585/225000 [49:11:29<341:18:13,  5.98s/it]
  9%|▊         | 19586/225000 [49:11:35<328:02:32,  5.75s/it]
  9%|▊         | 19587/225000 [49:11:43<380:30:43,  6.67s/it]
  9%|▊         | 19588/225000 [49:11:55<461:02:15,  8.08s/it]
  9%|▊         | 19589/225000 [49:12:02<443:29:24,  7.77s/it]
  9%|▊         | 19590/225000 [49:12:09<433:57:29,  7.61s/it]
  9%|▊         | 19591/225000 [49:12:13<370:15:45,  6.49s/it]
  9%|▊         | 19592/225000 [49:12:22<421:03:53,  7.38s/it]
  9%|▊         | 19593/225000 [49:12:26<363:59:44,  6.38s/it]
  9%|▊         | 19594/225000 [49:12:35<401:56:05,  7.04s/it]
  9%|▊         | 19595/225000 [49:12:41<388:22:49,  6.81s/it]
  9%|▊         | 19596/225000 [49:12:47<372:44:07,  6.53s/it]
  9%|▊         | 19597/225000 [49:12:55<397:58:55,  6.98s/it]
  9%|▊         | 19598/225000 [49:13:00<359:22:54,  6.30s/it]
  9%|▊         | 19599/225000 [49:13:07<373:05:58,  6.54s/it]
  9%|▊         | 19600/225000 [49:13:12<350:14:16,  6.14s/it]
                                                             
{'loss': '0.7119', 'grad_norm': '0.595', 'learning_rate': '4.975e-05', 'epoch': '0.08711', 'num_input_tokens_seen': 86420040, 'train_runtime': '1.772e+05', 'train_tokens_per_second': '487.7'}

  9%|▊         | 19600/225000 [49:13:12<350:14:16,  6.14s/it]
  9%|▊         | 19601/225000 [49:13:21<400:06:28,  7.01s/it]
  9%|▊         | 19602/225000 [49:13:28<395:23:22,  6.93s/it]
  9%|▊         | 19603/225000 [49:13:36<410:11:41,  7.19s/it]
  9%|▊         | 19604/225000 [49:13:42<389:04:01,  6.82s/it]
  9%|▊         | 19605/225000 [49:13:46<342:51:28,  6.01s/it]
  9%|▊         | 19606/225000 [49:13:54<385:50:36,  6.76s/it]
  9%|▊         | 19607/225000 [49:14:01<380:41:56,  6.67s/it]
  9%|▊         | 19608/225000 [49:14:05<336:16:34,  5.89s/it]
  9%|▊         | 19609/225000 [49:14:10<320:17:17,  5.61s/it]
  9%|▊         | 19610/225000 [49:14:15<316:58:53,  5.56s/it]
  9%|▊         | 19611/225000 [49:14:25<389:17:38,  6.82s/it]
  9%|▊         | 19612/225000 [49:14:35<433:34:52,  7.60s/it]
  9%|▊         | 19613/225000 [49:14:39<376:31:38,  6.60s/it]
  9%|▊         | 19614/225000 [49:14:43<337:55:31,  5.92s/it]
  9%|▊         | 19615/225000 [49:14:48<327:26:40,  5.74s/it]
  9%|▊         | 19616/225000 [49:14:59<415:42:38,  7.29s/it]
  9%|▊         | 19617/225000 [49:15:03<360:33:34,  6.32s/it]
  9%|▊         | 19618/225000 [49:15:13<421:05:05,  7.38s/it]
  9%|▊         | 19619/225000 [49:15:18<370:54:02,  6.50s/it]
  9%|▊         | 19620/225000 [49:15:22<338:51:51,  5.94s/it]
  9%|▊         | 19621/225000 [49:15:32<393:39:36,  6.90s/it]
  9%|▊         | 19622/225000 [49:15:38<383:14:25,  6.72s/it]
  9%|▊         | 19623/225000 [49:15:42<343:20:15,  6.02s/it]
  9%|▊         | 19624/225000 [49:15:47<314:32:29,  5.51s/it]
  9%|▊         | 19625/225000 [49:15:58<412:36:34,  7.23s/it]
  9%|▊         | 19626/225000 [49:16:03<369:55:55,  6.48s/it]
  9%|▊         | 19627/225000 [49:16:11<400:56:20,  7.03s/it]
  9%|▊         | 19628/225000 [49:16:15<356:03:11,  6.24s/it]
  9%|▊         | 19629/225000 [49:16:20<332:12:01,  5.82s/it]
  9%|▊         | 19630/225000 [49:16:29<380:29:27,  6.67s/it]
  9%|▊         | 19631/225000 [49:16:37<414:15:23,  7.26s/it]
  9%|▊         | 19632/225000 [49:16:42<364:41:09,  6.39s/it]
  9%|▊         | 19633/225000 [49:16:50<403:55:46,  7.08s/it]
  9%|▊         | 19634/225000 [49:16:59<434:41:39,  7.62s/it]
  9%|▊         | 19635/225000 [49:17:05<400:38:27,  7.02s/it]
  9%|▊         | 19636/225000 [49:17:16<468:08:19,  8.21s/it]
  9%|▊         | 19637/225000 [49:17:25<490:37:26,  8.60s/it]
  9%|▊         | 19638/225000 [49:17:30<423:33:37,  7.43s/it]
  9%|▊         | 19639/225000 [49:17:35<388:04:26,  6.80s/it]
  9%|▊         | 19640/225000 [49:17:48<495:04:03,  8.68s/it]
  9%|▊         | 19641/225000 [49:17:55<465:52:50,  8.17s/it]
  9%|▊         | 19642/225000 [49:18:00<395:09:47,  6.93s/it]
  9%|▊         | 19643/225000 [49:18:09<434:06:42,  7.61s/it]
  9%|▊         | 19644/225000 [49:18:16<424:12:34,  7.44s/it]
  9%|▊         | 19645/225000 [49:18:20<362:55:36,  6.36s/it]
  9%|▊         | 19646/225000 [49:18:24<329:30:12,  5.78s/it]
  9%|▊         | 19647/225000 [49:18:29<315:26:51,  5.53s/it]
  9%|▊         | 19648/225000 [49:18:38<381:45:33,  6.69s/it]
  9%|▊         | 19649/225000 [49:18:46<400:19:53,  7.02s/it]
  9%|▊         | 19650/225000 [49:18:50<346:24:22,  6.07s/it]
                                                             
{'loss': '0.6921', 'grad_norm': '0.6516', 'learning_rate': '4.975e-05', 'epoch': '0.08733', 'num_input_tokens_seen': 86573800, 'train_runtime': '1.775e+05', 'train_tokens_per_second': '487.7'}

  9%|▊         | 19650/225000 [49:18:50<346:24:22,  6.07s/it]
  9%|▊         | 19651/225000 [49:18:59<403:35:48,  7.08s/it]
  9%|▊         | 19652/225000 [49:19:06<391:14:28,  6.86s/it]
  9%|▊         | 19653/225000 [49:19:11<355:55:53,  6.24s/it]
  9%|▊         | 19654/225000 [49:19:16<337:03:46,  5.91s/it]
  9%|▊         | 19655/225000 [49:19:20<316:59:33,  5.56s/it]
  9%|▊         | 19656/225000 [49:19:26<308:22:42,  5.41s/it]
  9%|▊         | 19657/225000 [49:19:31<314:42:51,  5.52s/it]
  9%|▊         | 19658/225000 [49:19:44<435:27:49,  7.63s/it]
  9%|▊         | 19659/225000 [49:19:49<399:25:17,  7.00s/it]
  9%|▊         | 19660/225000 [49:19:58<423:45:12,  7.43s/it]
  9%|▊         | 19661/225000 [49:20:04<400:54:05,  7.03s/it]
  9%|▊         | 19662/225000 [49:20:09<360:02:56,  6.31s/it]
  9%|▊         | 19663/225000 [49:20:13<335:34:16,  5.88s/it]
  9%|▊         | 19664/225000 [49:20:25<434:33:25,  7.62s/it]
  9%|▊         | 19665/225000 [49:20:30<380:40:40,  6.67s/it]
  9%|▊         | 19666/225000 [49:20:36<369:34:35,  6.48s/it]
  9%|▊         | 19667/225000 [49:20:44<405:59:34,  7.12s/it]
  9%|▊         | 19668/225000 [49:20:49<359:39:15,  6.31s/it]
  9%|▊         | 19669/225000 [49:20:53<335:32:23,  5.88s/it]
  9%|▊         | 19670/225000 [49:20:58<314:32:53,  5.51s/it]
  9%|▊         | 19671/225000 [49:21:07<363:31:15,  6.37s/it]
  9%|▊         | 19672/225000 [49:21:16<417:58:47,  7.33s/it]
  9%|▊         | 19673/225000 [49:21:20<367:02:12,  6.44s/it]
  9%|▊         | 19674/225000 [49:21:30<412:33:28,  7.23s/it]
  9%|▊         | 19675/225000 [49:21:34<369:49:01,  6.48s/it]
  9%|▊         | 19676/225000 [49:21:39<334:32:24,  5.87s/it]
  9%|▊         | 19677/225000 [49:21:49<406:24:23,  7.13s/it]
  9%|▊         | 19678/225000 [49:21:55<399:17:49,  7.00s/it]
  9%|▊         | 19679/225000 [49:22:01<368:37:33,  6.46s/it]
  9%|▊         | 19680/225000 [49:22:07<363:04:48,  6.37s/it]
  9%|▊         | 19681/225000 [49:22:16<413:54:09,  7.26s/it]
  9%|▊         | 19682/225000 [49:22:23<413:01:23,  7.24s/it]
  9%|▊         | 19683/225000 [49:22:31<420:14:31,  7.37s/it]
  9%|▊         | 19684/225000 [49:22:37<388:18:25,  6.81s/it]
  9%|▊         | 19685/225000 [49:22:49<487:20:04,  8.54s/it]
  9%|▊         | 19686/225000 [49:22:55<446:22:44,  7.83s/it]
  9%|▊         | 19687/225000 [49:22:59<378:46:40,  6.64s/it]
  9%|▉         | 19688/225000 [49:23:04<344:12:23,  6.04s/it]
  9%|▉         | 19689/225000 [49:23:11<365:26:49,  6.41s/it]
  9%|▉         | 19690/225000 [49:23:20<407:57:12,  7.15s/it]
  9%|▉         | 19691/225000 [49:23:25<375:29:31,  6.58s/it]
  9%|▉         | 19692/225000 [49:23:30<339:22:55,  5.95s/it]
  9%|▉         | 19693/225000 [49:23:41<428:28:04,  7.51s/it]
  9%|▉         | 19694/225000 [49:23:46<387:11:14,  6.79s/it]
  9%|▉         | 19695/225000 [49:23:54<406:19:13,  7.12s/it]
  9%|▉         | 19696/225000 [49:23:58<359:42:38,  6.31s/it]
  9%|▉         | 19697/225000 [49:24:03<339:57:11,  5.96s/it]
  9%|▉         | 19698/225000 [49:24:15<435:39:38,  7.64s/it]
  9%|▉         | 19699/225000 [49:24:20<396:42:25,  6.96s/it]
  9%|▉         | 19700/225000 [49:24:30<435:27:25,  7.64s/it]
                                                             
{'loss': '0.6966', 'grad_norm': '0.6194', 'learning_rate': '4.975e-05', 'epoch': '0.08756', 'num_input_tokens_seen': 86726752, 'train_runtime': '1.779e+05', 'train_tokens_per_second': '487.6'}

  9%|▉         | 19700/225000 [49:24:30<435:27:25,  7.64s/it]
  9%|▉         | 19701/225000 [49:24:35<394:51:38,  6.92s/it]
  9%|▉         | 19702/225000 [49:24:41<374:19:28,  6.56s/it]
  9%|▉         | 19703/225000 [49:24:50<419:08:40,  7.35s/it]
  9%|▉         | 19704/225000 [49:24:57<415:38:03,  7.29s/it]
  9%|▉         | 19705/225000 [49:25:03<401:56:10,  7.05s/it]
  9%|▉         | 19706/225000 [49:25:08<369:42:49,  6.48s/it]
  9%|▉         | 19707/225000 [49:25:14<348:02:34,  6.10s/it]
  9%|▉         | 19708/225000 [49:25:19<339:20:44,  5.95s/it]
  9%|▉         | 19709/225000 [49:25:24<322:17:50,  5.65s/it]
  9%|▉         | 19710/225000 [49:25:29<311:32:52,  5.46s/it]
  9%|▉         | 19711/225000 [49:25:42<433:42:58,  7.61s/it]
  9%|▉         | 19712/225000 [49:25:48<402:47:42,  7.06s/it]
  9%|▉         | 19713/225000 [49:25:53<377:35:21,  6.62s/it]
  9%|▉         | 19714/225000 [49:25:59<360:38:38,  6.32s/it]
  9%|▉         | 19715/225000 [49:26:04<341:42:06,  5.99s/it]
  9%|▉         | 19716/225000 [49:26:09<330:37:01,  5.80s/it]
  9%|▉         | 19717/225000 [49:26:20<417:08:40,  7.32s/it]
  9%|▉         | 19718/225000 [49:26:29<440:20:05,  7.72s/it]
  9%|▉         | 19719/225000 [49:26:36<435:46:43,  7.64s/it]
  9%|▉         | 19720/225000 [49:26:43<410:32:53,  7.20s/it]
  9%|▉         | 19721/225000 [49:26:51<429:15:42,  7.53s/it]
  9%|▉         | 19722/225000 [49:26:56<379:24:54,  6.65s/it]
  9%|▉         | 19723/225000 [49:27:00<342:11:03,  6.00s/it]
  9%|▉         | 19724/225000 [49:27:11<431:18:07,  7.56s/it]
  9%|▉         | 19725/225000 [49:27:15<371:02:32,  6.51s/it]
  9%|▉         | 19726/225000 [49:27:23<393:58:22,  6.91s/it]
  9%|▉         | 19727/225000 [49:27:30<394:03:31,  6.91s/it]
  9%|▉         | 19728/225000 [49:27:37<395:23:48,  6.93s/it]
  9%|▉         | 19729/225000 [49:27:43<377:54:14,  6.63s/it]
  9%|▉         | 19730/225000 [49:27:52<420:59:37,  7.38s/it]
  9%|▉         | 19731/225000 [49:27:59<412:35:21,  7.24s/it]
  9%|▉         | 19732/225000 [49:28:03<366:01:33,  6.42s/it]
  9%|▉         | 19733/225000 [49:28:07<324:13:37,  5.69s/it]
  9%|▉         | 19734/225000 [49:28:12<308:14:50,  5.41s/it]
  9%|▉         | 19735/225000 [49:28:21<360:59:31,  6.33s/it]
  9%|▉         | 19736/225000 [49:28:28<373:19:40,  6.55s/it]
  9%|▉         | 19737/225000 [49:28:36<410:13:24,  7.19s/it]
  9%|▉         | 19738/225000 [49:28:42<376:24:10,  6.60s/it]
  9%|▉         | 19739/225000 [49:28:47<355:56:40,  6.24s/it]
  9%|▉         | 19740/225000 [49:28:53<345:37:23,  6.06s/it]
  9%|▉         | 19741/225000 [49:28:58<330:56:30,  5.80s/it]
  9%|▉         | 19742/225000 [49:29:03<322:44:12,  5.66s/it]
  9%|▉         | 19743/225000 [49:29:09<322:15:56,  5.65s/it]
  9%|▉         | 19744/225000 [49:29:15<323:04:39,  5.67s/it]
  9%|▉         | 19745/225000 [49:29:24<386:06:00,  6.77s/it]
  9%|▉         | 19746/225000 [49:29:36<475:49:11,  8.35s/it]
  9%|▉         | 19747/225000 [49:29:40<403:43:34,  7.08s/it]
  9%|▉         | 19748/225000 [49:29:46<379:29:36,  6.66s/it]
  9%|▉         | 19749/225000 [49:29:53<384:48:47,  6.75s/it]
  9%|▉         | 19750/225000 [49:29:59<373:49:02,  6.56s/it]
                                                             
{'loss': '0.7084', 'grad_norm': '0.6721', 'learning_rate': '4.975e-05', 'epoch': '0.08778', 'num_input_tokens_seen': 86879104, 'train_runtime': '1.782e+05', 'train_tokens_per_second': '487.5'}

  9%|▉         | 19750/225000 [49:29:59<373:49:02,  6.56s/it]
  9%|▉         | 19751/225000 [49:30:04<351:26:01,  6.16s/it]
  9%|▉         | 19752/225000 [49:30:10<343:20:48,  6.02s/it]
  9%|▉         | 19753/225000 [49:30:16<339:58:34,  5.96s/it]
  9%|▉         | 19754/225000 [49:30:25<392:03:50,  6.88s/it]
  9%|▉         | 19755/225000 [49:30:34<430:04:33,  7.54s/it]
  9%|▉         | 19756/225000 [49:30:41<431:47:19,  7.57s/it]
  9%|▉         | 19757/225000 [49:30:47<391:44:25,  6.87s/it]
  9%|▉         | 19758/225000 [49:30:52<359:27:10,  6.30s/it]
  9%|▉         | 19759/225000 [49:30:56<336:17:14,  5.90s/it]
  9%|▉         | 19760/225000 [49:31:09<457:06:38,  8.02s/it]
  9%|▉         | 19761/225000 [49:31:15<415:22:03,  7.29s/it]
  9%|▉         | 19762/225000 [49:31:23<431:20:28,  7.57s/it]
  9%|▉         | 19763/225000 [49:31:27<362:12:13,  6.35s/it]
  9%|▉         | 19764/225000 [49:31:36<416:27:45,  7.31s/it]
  9%|▉         | 19765/225000 [49:31:40<351:54:14,  6.17s/it]
  9%|▉         | 19766/225000 [49:31:50<422:45:18,  7.42s/it]
  9%|▉         | 19767/225000 [49:31:56<399:44:38,  7.01s/it]
  9%|▉         | 19768/225000 [49:32:03<402:26:58,  7.06s/it]
  9%|▉         | 19769/225000 [49:32:10<396:33:24,  6.96s/it]
  9%|▉         | 19770/225000 [49:32:16<371:30:19,  6.52s/it]
  9%|▉         | 19771/225000 [49:32:22<363:48:50,  6.38s/it]
  9%|▉         | 19772/225000 [49:32:28<360:31:30,  6.32s/it]
  9%|▉         | 19773/225000 [49:32:33<344:19:53,  6.04s/it]
  9%|▉         | 19774/225000 [49:32:38<330:22:32,  5.80s/it]
  9%|▉         | 19775/225000 [49:32:44<323:59:58,  5.68s/it]
  9%|▉         | 19776/225000 [49:32:50<323:44:11,  5.68s/it]
  9%|▉         | 19777/225000 [49:33:04<468:45:25,  8.22s/it]
  9%|▉         | 19778/225000 [49:33:10<427:16:37,  7.50s/it]
  9%|▉         | 19779/225000 [49:33:14<378:54:52,  6.65s/it]
  9%|▉         | 19780/225000 [49:33:25<442:45:37,  7.77s/it]
  9%|▉         | 19781/225000 [49:33:31<427:09:08,  7.49s/it]
  9%|▉         | 19782/225000 [49:33:36<376:30:13,  6.60s/it]
  9%|▉         | 19783/225000 [49:33:40<334:55:59,  5.88s/it]
  9%|▉         | 19784/225000 [49:33:52<438:23:00,  7.69s/it]
  9%|▉         | 19785/225000 [49:33:56<367:20:53,  6.44s/it]
  9%|▉         | 19786/225000 [49:34:05<418:14:56,  7.34s/it]
  9%|▉         | 19787/225000 [49:34:10<377:13:31,  6.62s/it]
  9%|▉         | 19788/225000 [49:34:16<361:36:06,  6.34s/it]
  9%|▉         | 19789/225000 [49:34:21<348:59:56,  6.12s/it]
  9%|▉         | 19790/225000 [49:34:26<327:42:17,  5.75s/it]
  9%|▉         | 19791/225000 [49:34:32<325:41:39,  5.71s/it]
  9%|▉         | 19792/225000 [49:34:46<475:56:05,  8.35s/it]
  9%|▉         | 19793/225000 [49:34:53<456:35:45,  8.01s/it]
  9%|▉         | 19794/225000 [49:35:00<427:42:47,  7.50s/it]
  9%|▉         | 19795/225000 [49:35:05<383:34:29,  6.73s/it]
  9%|▉         | 19796/225000 [49:35:10<353:19:43,  6.20s/it]
  9%|▉         | 19797/225000 [49:35:20<428:10:54,  7.51s/it]
  9%|▉         | 19798/225000 [49:35:26<397:54:18,  6.98s/it]
  9%|▉         | 19799/225000 [49:35:34<415:05:43,  7.28s/it]
  9%|▉         | 19800/225000 [49:35:39<374:03:49,  6.56s/it]
                                                             
{'loss': '0.7168', 'grad_norm': '0.6559', 'learning_rate': '4.974e-05', 'epoch': '0.088', 'num_input_tokens_seen': 87031544, 'train_runtime': '1.785e+05', 'train_tokens_per_second': '487.5'}

  9%|▉         | 19800/225000 [49:35:39<374:03:49,  6.56s/it]
  9%|▉         | 19801/225000 [49:35:44<349:43:14,  6.14s/it]
  9%|▉         | 19802/225000 [49:35:49<328:53:33,  5.77s/it]
  9%|▉         | 19803/225000 [49:35:54<321:00:39,  5.63s/it]
  9%|▉         | 19804/225000 [49:35:59<312:42:06,  5.49s/it]
  9%|▉         | 19805/225000 [49:36:05<309:54:02,  5.44s/it]
  9%|▉         | 19806/225000 [49:36:18<439:04:18,  7.70s/it]
  9%|▉         | 19807/225000 [49:36:23<394:48:13,  6.93s/it]
  9%|▉         | 19808/225000 [49:36:27<352:38:09,  6.19s/it]
  9%|▉         | 19809/225000 [49:36:36<397:54:58,  6.98s/it]
  9%|▉         | 19810/225000 [49:36:41<363:24:44,  6.38s/it]
  9%|▉         | 19811/225000 [49:36:46<339:12:50,  5.95s/it]
  9%|▉         | 19812/225000 [49:36:59<465:10:37,  8.16s/it]
  9%|▉         | 19813/225000 [49:37:04<402:28:14,  7.06s/it]
  9%|▉         | 19814/225000 [49:37:08<354:12:08,  6.21s/it]
  9%|▉         | 19815/225000 [49:37:13<327:12:49,  5.74s/it]
  9%|▉         | 19816/225000 [49:37:26<449:58:09,  7.89s/it]
  9%|▉         | 19817/225000 [49:37:30<381:47:11,  6.70s/it]
  9%|▉         | 19818/225000 [49:37:35<352:35:32,  6.19s/it]
  9%|▉         | 19819/225000 [49:37:39<326:25:08,  5.73s/it]
  9%|▉         | 19820/225000 [49:37:50<412:54:25,  7.24s/it]
  9%|▉         | 19821/225000 [49:37:59<448:24:39,  7.87s/it]
  9%|▉         | 19822/225000 [49:38:05<402:58:38,  7.07s/it]
  9%|▉         | 19823/225000 [49:38:10<369:24:35,  6.48s/it]
  9%|▉         | 19824/225000 [49:38:15<357:18:59,  6.27s/it]
  9%|▉         | 19825/225000 [49:38:21<337:44:16,  5.93s/it]
  9%|▉         | 19826/225000 [49:38:26<322:23:47,  5.66s/it]
  9%|▉         | 19827/225000 [49:38:31<315:08:26,  5.53s/it]
  9%|▉         | 19828/225000 [49:38:42<415:27:36,  7.29s/it]
  9%|▉         | 19829/225000 [49:38:50<423:48:46,  7.44s/it]
  9%|▉         | 19830/225000 [49:38:59<455:53:26,  8.00s/it]
  9%|▉         | 19831/225000 [49:39:05<418:43:47,  7.35s/it]
  9%|▉         | 19832/225000 [49:39:13<422:46:23,  7.42s/it]
  9%|▉         | 19833/225000 [49:39:18<383:40:17,  6.73s/it]
  9%|▉         | 19834/225000 [49:39:23<351:42:46,  6.17s/it]
  9%|▉         | 19835/225000 [49:39:32<401:12:26,  7.04s/it]
  9%|▉         | 19836/225000 [49:39:41<441:02:59,  7.74s/it]
  9%|▉         | 19837/225000 [49:39:46<393:41:07,  6.91s/it]
  9%|▉         | 19838/225000 [49:39:51<357:15:33,  6.27s/it]
  9%|▉         | 19839/225000 [49:39:57<347:40:24,  6.10s/it]
  9%|▉         | 19840/225000 [49:40:02<328:55:53,  5.77s/it]
  9%|▉         | 19841/225000 [49:40:09<359:37:14,  6.31s/it]
  9%|▉         | 19842/225000 [49:40:20<436:38:44,  7.66s/it]
  9%|▉         | 19843/225000 [49:40:24<369:20:19,  6.48s/it]
  9%|▉         | 19844/225000 [49:40:34<434:19:10,  7.62s/it]
  9%|▉         | 19845/225000 [49:40:41<416:13:00,  7.30s/it]
  9%|▉         | 19846/225000 [49:40:45<361:34:54,  6.34s/it]
  9%|▉         | 19847/225000 [49:40:53<388:20:52,  6.81s/it]
  9%|▉         | 19848/225000 [49:40:57<350:16:12,  6.15s/it]
  9%|▉         | 19849/225000 [49:41:04<368:56:03,  6.47s/it]
  9%|▉         | 19850/225000 [49:41:13<409:48:15,  7.19s/it]
                                                             
{'loss': '0.6974', 'grad_norm': '0.4672', 'learning_rate': '4.974e-05', 'epoch': '0.08822', 'num_input_tokens_seen': 87182728, 'train_runtime': '1.789e+05', 'train_tokens_per_second': '487.4'}

  9%|▉         | 19850/225000 [49:41:13<409:48:15,  7.19s/it]
  9%|▉         | 19851/225000 [49:41:18<370:09:01,  6.50s/it]
  9%|▉         | 19852/225000 [49:41:23<344:02:03,  6.04s/it]
  9%|▉         | 19853/225000 [49:41:28<326:57:37,  5.74s/it]
  9%|▉         | 19854/225000 [49:41:34<325:08:06,  5.71s/it]
  9%|▉         | 19855/225000 [49:41:47<454:24:16,  7.97s/it]
  9%|▉         | 19856/225000 [49:41:54<429:16:11,  7.53s/it]
  9%|▉         | 19857/225000 [49:41:59<389:30:13,  6.84s/it]
  9%|▉         | 19858/225000 [49:42:04<365:28:26,  6.41s/it]
  9%|▉         | 19859/225000 [49:42:10<350:41:31,  6.15s/it]
  9%|▉         | 19860/225000 [49:42:15<335:11:15,  5.88s/it]
  9%|▉         | 19861/225000 [49:42:20<327:03:31,  5.74s/it]
  9%|▉         | 19862/225000 [49:42:33<448:22:19,  7.87s/it]
  9%|▉         | 19863/225000 [49:42:38<398:41:20,  7.00s/it]
  9%|▉         | 19864/225000 [49:42:47<433:22:30,  7.61s/it]
  9%|▉         | 19865/225000 [49:42:51<366:23:30,  6.43s/it]
  9%|▉         | 19866/225000 [49:42:56<349:58:43,  6.14s/it]
  9%|▉         | 19867/225000 [49:43:01<330:29:46,  5.80s/it]
  9%|▉         | 19868/225000 [49:43:07<328:12:25,  5.76s/it]
  9%|▉         | 19869/225000 [49:43:15<374:48:05,  6.58s/it]
  9%|▉         | 19870/225000 [49:43:25<423:58:04,  7.44s/it]
  9%|▉         | 19871/225000 [49:43:33<436:25:30,  7.66s/it]
  9%|▉         | 19872/225000 [49:43:39<404:05:48,  7.09s/it]
  9%|▉         | 19873/225000 [49:43:48<432:42:54,  7.59s/it]
  9%|▉         | 19874/225000 [49:43:54<413:48:55,  7.26s/it]
  9%|▉         | 19875/225000 [49:44:00<383:45:20,  6.74s/it]
  9%|▉         | 19876/225000 [49:44:05<357:32:54,  6.28s/it]
  9%|▉         | 19877/225000 [49:44:10<334:45:27,  5.88s/it]
  9%|▉         | 19878/225000 [49:44:15<318:43:48,  5.59s/it]
  9%|▉         | 19879/225000 [49:44:28<446:08:35,  7.83s/it]
  9%|▉         | 19880/225000 [49:44:33<402:13:24,  7.06s/it]
  9%|▉         | 19881/225000 [49:44:41<413:51:51,  7.26s/it]
  9%|▉         | 19882/225000 [49:44:45<361:46:16,  6.35s/it]
  9%|▉         | 19883/225000 [49:44:51<362:23:29,  6.36s/it]
  9%|▉         | 19884/225000 [49:44:59<376:08:45,  6.60s/it]
  9%|▉         | 19885/225000 [49:45:07<403:39:02,  7.08s/it]
  9%|▉         | 19886/225000 [49:45:11<363:27:36,  6.38s/it]
  9%|▉         | 19887/225000 [49:45:17<345:37:03,  6.07s/it]
  9%|▉         | 19888/225000 [49:45:22<335:09:11,  5.88s/it]
  9%|▉         | 19889/225000 [49:45:28<337:52:08,  5.93s/it]
  9%|▉         | 19890/225000 [49:45:35<356:09:53,  6.25s/it]
  9%|▉         | 19891/225000 [49:45:48<466:04:14,  8.18s/it]
  9%|▉         | 19892/225000 [49:45:51<380:48:42,  6.68s/it]
  9%|▉         | 19893/225000 [49:46:02<456:18:07,  8.01s/it]
  9%|▉         | 19894/225000 [49:46:08<418:54:10,  7.35s/it]
  9%|▉         | 19895/225000 [49:46:13<375:33:39,  6.59s/it]
  9%|▉         | 19896/225000 [49:46:22<421:37:37,  7.40s/it]
  9%|▉         | 19897/225000 [49:46:26<359:21:49,  6.31s/it]
  9%|▉         | 19898/225000 [49:46:36<427:57:15,  7.51s/it]
  9%|▉         | 19899/225000 [49:46:42<398:36:05,  7.00s/it]
  9%|▉         | 19900/225000 [49:46:46<344:08:44,  6.04s/it]
                                                             
{'loss': '0.7322', 'grad_norm': '0.6285', 'learning_rate': '4.974e-05', 'epoch': '0.08844', 'num_input_tokens_seen': 87336024, 'train_runtime': '1.792e+05', 'train_tokens_per_second': '487.3'}

  9%|▉         | 19900/225000 [49:46:46<344:08:44,  6.04s/it]
  9%|▉         | 19901/225000 [49:46:55<403:02:18,  7.07s/it]
  9%|▉         | 19902/225000 [49:46:59<349:33:31,  6.14s/it]
  9%|▉         | 19903/225000 [49:47:09<407:56:19,  7.16s/it]
  9%|▉         | 19904/225000 [49:47:16<407:43:02,  7.16s/it]
  9%|▉         | 19905/225000 [49:47:19<344:10:51,  6.04s/it]
  9%|▉         | 19906/225000 [49:47:28<392:07:19,  6.88s/it]
  9%|▉         | 19907/225000 [49:47:32<345:49:14,  6.07s/it]
  9%|▉         | 19908/225000 [49:47:43<417:28:48,  7.33s/it]
  9%|▉         | 19909/225000 [49:47:47<373:33:08,  6.56s/it]
  9%|▉         | 19910/225000 [49:47:52<332:29:28,  5.84s/it]
  9%|▉         | 19911/225000 [49:47:56<313:55:18,  5.51s/it]
  9%|▉         | 19912/225000 [49:48:06<389:02:45,  6.83s/it]
  9%|▉         | 19913/225000 [49:48:15<418:33:36,  7.35s/it]
  9%|▉         | 19914/225000 [49:48:19<364:04:55,  6.39s/it]
  9%|▉         | 19915/225000 [49:48:23<326:03:21,  5.72s/it]
  9%|▉         | 19916/225000 [49:48:28<311:35:12,  5.47s/it]
  9%|▉         | 19917/225000 [49:48:41<439:00:03,  7.71s/it]
  9%|▉         | 19918/225000 [49:48:47<403:50:55,  7.09s/it]
  9%|▉         | 19919/225000 [49:48:52<368:34:48,  6.47s/it]
  9%|▉         | 19920/225000 [49:48:58<374:49:57,  6.58s/it]
  9%|▉         | 19921/225000 [49:49:09<449:28:52,  7.89s/it]
  9%|▉         | 19922/225000 [49:49:13<375:23:54,  6.59s/it]
  9%|▉         | 19923/225000 [49:49:20<377:48:10,  6.63s/it]
  9%|▉         | 19924/225000 [49:49:28<404:35:38,  7.10s/it]
  9%|▉         | 19925/225000 [49:49:33<372:21:59,  6.54s/it]
  9%|▉         | 19926/225000 [49:49:38<346:50:11,  6.09s/it]
  9%|▉         | 19927/225000 [49:49:43<328:35:57,  5.77s/it]
  9%|▉         | 19928/225000 [49:49:48<317:24:53,  5.57s/it]
  9%|▉         | 19929/225000 [49:49:53<309:32:58,  5.43s/it]
  9%|▉         | 19930/225000 [49:50:00<322:27:13,  5.66s/it]
  9%|▉         | 19931/225000 [49:50:13<453:43:27,  7.97s/it]
  9%|▉         | 19932/225000 [49:50:20<437:55:49,  7.69s/it]
  9%|▉         | 19933/225000 [49:50:27<425:16:18,  7.47s/it]
  9%|▉         | 19934/225000 [49:50:32<391:20:52,  6.87s/it]
  9%|▉         | 19935/225000 [49:50:38<364:06:06,  6.39s/it]
  9%|▉         | 19936/225000 [49:50:43<347:27:56,  6.10s/it]
  9%|▉         | 19937/225000 [49:50:48<326:05:35,  5.72s/it]
  9%|▉         | 19938/225000 [49:50:54<329:55:42,  5.79s/it]
  9%|▉         | 19939/225000 [49:51:07<453:47:25,  7.97s/it]
  9%|▉         | 19940/225000 [49:51:14<438:53:12,  7.71s/it]
  9%|▉         | 19941/225000 [49:51:19<392:58:31,  6.90s/it]
  9%|▉         | 19942/225000 [49:51:24<364:44:51,  6.40s/it]
  9%|▉         | 19943/225000 [49:51:30<349:47:47,  6.14s/it]
  9%|▉         | 19944/225000 [49:51:39<401:21:29,  7.05s/it]
  9%|▉         | 19945/225000 [49:51:48<432:00:01,  7.58s/it]
  9%|▉         | 19946/225000 [49:51:55<430:27:22,  7.56s/it]
  9%|▉         | 19947/225000 [49:52:00<379:04:08,  6.66s/it]
  9%|▉         | 19948/225000 [49:52:09<419:18:54,  7.36s/it]
  9%|▉         | 19949/225000 [49:52:14<374:10:00,  6.57s/it]
  9%|▉         | 19950/225000 [49:52:22<412:32:21,  7.24s/it]
                                                             
{'loss': '0.7186', 'grad_norm': '0.5872', 'learning_rate': '4.974e-05', 'epoch': '0.08867', 'num_input_tokens_seen': 87489136, 'train_runtime': '1.795e+05', 'train_tokens_per_second': '487.3'}

  9%|▉         | 19950/225000 [49:52:22<412:32:21,  7.24s/it]
  9%|▉         | 19951/225000 [49:52:29<395:16:05,  6.94s/it]
  9%|▉         | 19952/225000 [49:52:35<392:14:36,  6.89s/it]
  9%|▉         | 19953/225000 [49:52:40<352:23:11,  6.19s/it]
  9%|▉         | 19954/225000 [49:52:52<448:43:11,  7.88s/it]
  9%|▉         | 19955/225000 [49:53:01<462:43:39,  8.12s/it]
  9%|▉         | 19956/225000 [49:53:07<428:46:42,  7.53s/it]
  9%|▉         | 19957/225000 [49:53:13<406:19:20,  7.13s/it]
  9%|▉         | 19958/225000 [49:53:21<422:12:18,  7.41s/it]
  9%|▉         | 19959/225000 [49:53:27<399:12:39,  7.01s/it]
  9%|▉         | 19960/225000 [49:53:32<359:12:56,  6.31s/it]
  9%|▉         | 19961/225000 [49:53:37<336:48:36,  5.91s/it]
  9%|▉         | 19962/225000 [49:53:42<329:27:57,  5.78s/it]
  9%|▉         | 19963/225000 [49:53:48<326:20:58,  5.73s/it]
  9%|▉         | 19964/225000 [49:53:53<322:08:35,  5.66s/it]
  9%|▉         | 19965/225000 [49:54:06<438:23:06,  7.70s/it]
  9%|▉         | 19966/225000 [49:54:13<432:39:42,  7.60s/it]
  9%|▉         | 19967/225000 [49:54:21<442:22:45,  7.77s/it]
  9%|▉         | 19968/225000 [49:54:26<393:01:22,  6.90s/it]
  9%|▉         | 19969/225000 [49:54:31<364:25:14,  6.40s/it]
  9%|▉         | 19970/225000 [49:54:36<340:57:12,  5.99s/it]
  9%|▉         | 19971/225000 [49:54:41<321:08:15,  5.64s/it]
  9%|▉         | 19972/225000 [49:54:50<377:00:21,  6.62s/it]
  9%|▉         | 19973/225000 [49:54:59<422:51:23,  7.42s/it]
  9%|▉         | 19974/225000 [49:55:09<458:57:31,  8.06s/it]
  9%|▉         | 19975/225000 [49:55:13<391:20:11,  6.87s/it]
  9%|▉         | 19976/225000 [49:55:20<396:43:52,  6.97s/it]
  9%|▉         | 19977/225000 [49:55:27<384:59:01,  6.76s/it]
  9%|▉         | 19978/225000 [49:55:33<380:30:33,  6.68s/it]
  9%|▉         | 19979/225000 [49:55:38<356:44:41,  6.26s/it]
  9%|▉         | 19980/225000 [49:55:48<415:45:41,  7.30s/it]
  9%|▉         | 19981/225000 [49:55:53<374:11:55,  6.57s/it]
  9%|▉         | 19982/225000 [49:55:57<340:04:02,  5.97s/it]
  9%|▉         | 19983/225000 [49:56:02<318:10:58,  5.59s/it]
  9%|▉         | 19984/225000 [49:56:12<398:52:24,  7.00s/it]
  9%|▉         | 19985/225000 [49:56:20<407:09:56,  7.15s/it]
  9%|▉         | 19986/225000 [49:56:25<376:44:20,  6.62s/it]
  9%|▉         | 19987/225000 [49:56:31<358:49:43,  6.30s/it]
  9%|▉         | 19988/225000 [49:56:36<341:13:52,  5.99s/it]
  9%|▉         | 19989/225000 [49:56:42<330:57:16,  5.81s/it]
  9%|▉         | 19990/225000 [49:56:47<324:26:08,  5.70s/it]
  9%|▉         | 19991/225000 [49:56:52<320:45:02,  5.63s/it]
  9%|▉         | 19992/225000 [49:57:03<411:16:14,  7.22s/it]
  9%|▉         | 19993/225000 [49:57:12<438:45:11,  7.70s/it]
  9%|▉         | 19994/225000 [49:57:17<381:22:33,  6.70s/it]
  9%|▉         | 19995/225000 [49:57:23<374:38:26,  6.58s/it]
  9%|▉         | 19996/225000 [49:57:32<421:50:06,  7.41s/it]
  9%|▉         | 19997/225000 [49:57:39<417:39:51,  7.33s/it]
  9%|▉         | 19998/225000 [49:57:47<422:00:08,  7.41s/it]
  9%|▉         | 19999/225000 [49:57:53<393:02:22,  6.90s/it]
  9%|▉         | 20000/225000 [49:58:00<400:20:19,  7.03s/it]
                                                             
{'loss': '0.7249', 'grad_norm': '0.7437', 'learning_rate': '4.973e-05', 'epoch': '0.08889', 'num_input_tokens_seen': 87642592, 'train_runtime': '1.799e+05', 'train_tokens_per_second': '487.2'}

  9%|▉         | 20000/225000 [49:58:00<400:20:19,  7.03s/it]
  9%|▉         | 20001/225000 [49:58:04<345:26:52,  6.07s/it]
  9%|▉         | 20002/225000 [49:58:14<420:20:15,  7.38s/it]
  9%|▉         | 20003/225000 [49:58:21<403:06:04,  7.08s/it]
  9%|▉         | 20004/225000 [49:58:25<361:52:59,  6.36s/it]
  9%|▉         | 20005/225000 [49:58:37<445:05:44,  7.82s/it]
  9%|▉         | 20006/225000 [49:58:41<382:44:32,  6.72s/it]
  9%|▉         | 20007/225000 [49:58:50<425:17:06,  7.47s/it]
  9%|▉         | 20008/225000 [49:58:55<381:31:45,  6.70s/it]
  9%|▉         | 20009/225000 [49:59:00<361:49:02,  6.35s/it]
  9%|▉         | 20010/225000 [49:59:05<338:50:13,  5.95s/it]
  9%|▉         | 20011/225000 [49:59:11<331:54:18,  5.83s/it]
  9%|▉         | 20012/225000 [49:59:16<322:01:37,  5.66s/it]
  9%|▉         | 20013/225000 [49:59:31<486:37:50,  8.55s/it]
  9%|▉         | 20014/225000 [49:59:37<430:39:40,  7.56s/it]
  9%|▉         | 20015/225000 [49:59:41<376:01:46,  6.60s/it]
  9%|▉         | 20016/225000 [49:59:52<446:40:54,  7.84s/it]
  9%|▉         | 20017/225000 [49:59:55<370:48:16,  6.51s/it]
  9%|▉         | 20018/225000 [50:00:05<427:18:53,  7.50s/it]
  9%|▉         | 20019/225000 [50:00:10<384:57:00,  6.76s/it]
  9%|▉         | 20020/225000 [50:00:15<355:04:59,  6.24s/it]
  9%|▉         | 20021/225000 [50:00:20<326:25:53,  5.73s/it]
  9%|▉         | 20022/225000 [50:00:30<399:55:39,  7.02s/it]
  9%|▉         | 20023/225000 [50:00:37<407:22:22,  7.15s/it]
  9%|▉         | 20024/225000 [50:00:44<409:10:58,  7.19s/it]
  9%|▉         | 20025/225000 [50:00:51<396:30:19,  6.96s/it]
  9%|▉         | 20026/225000 [50:00:55<351:54:29,  6.18s/it]
  9%|▉         | 20027/225000 [50:01:05<414:55:30,  7.29s/it]
  9%|▉         | 20028/225000 [50:01:09<349:48:10,  6.14s/it]
  9%|▉         | 20029/225000 [50:01:17<382:46:04,  6.72s/it]
  9%|▉         | 20030/225000 [50:01:25<417:38:34,  7.34s/it]
  9%|▉         | 20031/225000 [50:01:29<359:47:41,  6.32s/it]
  9%|▉         | 20032/225000 [50:01:38<396:01:41,  6.96s/it]
  9%|▉         | 20033/225000 [50:01:44<380:58:21,  6.69s/it]
  9%|▉         | 20034/225000 [50:01:52<402:10:00,  7.06s/it]
  9%|▉         | 20035/225000 [50:01:58<383:36:15,  6.74s/it]
  9%|▉         | 20036/225000 [50:02:02<343:02:29,  6.03s/it]
  9%|▉         | 20037/225000 [50:02:08<331:56:12,  5.83s/it]
  9%|▉         | 20038/225000 [50:02:13<322:57:41,  5.67s/it]
  9%|▉         | 20039/225000 [50:02:17<303:11:28,  5.33s/it]
  9%|▉         | 20040/225000 [50:02:23<311:19:48,  5.47s/it]
  9%|▉         | 20041/225000 [50:02:36<445:35:01,  7.83s/it]
  9%|▉         | 20042/225000 [50:02:43<420:53:39,  7.39s/it]
  9%|▉         | 20043/225000 [50:02:51<426:46:50,  7.50s/it]
  9%|▉         | 20044/225000 [50:02:54<356:13:46,  6.26s/it]
  9%|▉         | 20045/225000 [50:03:04<419:16:42,  7.36s/it]
  9%|▉         | 20046/225000 [50:03:12<425:56:05,  7.48s/it]
  9%|▉         | 20047/225000 [50:03:16<377:53:05,  6.64s/it]
  9%|▉         | 20048/225000 [50:03:25<414:53:20,  7.29s/it]
  9%|▉         | 20049/225000 [50:03:31<395:39:22,  6.95s/it]
  9%|▉         | 20050/225000 [50:03:40<433:04:56,  7.61s/it]
                                                             
{'loss': '0.712', 'grad_norm': '0.7161', 'learning_rate': '4.973e-05', 'epoch': '0.08911', 'num_input_tokens_seen': 87795376, 'train_runtime': '1.802e+05', 'train_tokens_per_second': '487.2'}

  9%|▉         | 20050/225000 [50:03:40<433:04:56,  7.61s/it]
  9%|▉         | 20051/225000 [50:03:47<414:48:19,  7.29s/it]
  9%|▉         | 20052/225000 [50:03:52<377:27:57,  6.63s/it]
  9%|▉         | 20053/225000 [50:03:58<358:42:19,  6.30s/it]
  9%|▉         | 20054/225000 [50:04:03<346:17:25,  6.08s/it]
  9%|▉         | 20055/225000 [50:04:13<408:59:55,  7.18s/it]
  9%|▉         | 20056/225000 [50:04:22<443:18:20,  7.79s/it]
  9%|▉         | 20057/225000 [50:04:27<400:55:48,  7.04s/it]
  9%|▉         | 20058/225000 [50:04:33<367:56:10,  6.46s/it]
  9%|▉         | 20059/225000 [50:04:38<343:47:14,  6.04s/it]
  9%|▉         | 20060/225000 [50:04:44<358:49:41,  6.30s/it]
  9%|▉         | 20061/225000 [50:04:55<424:42:41,  7.46s/it]
  9%|▉         | 20062/225000 [50:05:00<382:31:07,  6.72s/it]
  9%|▉         | 20063/225000 [50:05:06<370:28:34,  6.51s/it]
  9%|▉         | 20064/225000 [50:05:15<415:12:23,  7.29s/it]
  9%|▉         | 20065/225000 [50:05:21<391:18:40,  6.87s/it]
  9%|▉         | 20066/225000 [50:05:26<369:39:56,  6.49s/it]
  9%|▉         | 20067/225000 [50:05:31<338:05:34,  5.94s/it]
  9%|▉         | 20068/225000 [50:05:37<334:19:08,  5.87s/it]
  9%|▉         | 20069/225000 [50:05:44<355:52:38,  6.25s/it]
  9%|▉         | 20070/225000 [50:05:57<476:14:11,  8.37s/it]
  9%|▉         | 20071/225000 [50:06:01<405:27:27,  7.12s/it]
  9%|▉         | 20072/225000 [50:06:06<361:40:30,  6.35s/it]
  9%|▉         | 20073/225000 [50:06:16<425:53:34,  7.48s/it]
  9%|▉         | 20074/225000 [50:06:21<387:44:24,  6.81s/it]
  9%|▉         | 20075/225000 [50:06:31<440:24:58,  7.74s/it]
  9%|▉         | 20076/225000 [50:06:34<364:45:47,  6.41s/it]
  9%|▉         | 20077/225000 [50:06:41<364:42:39,  6.41s/it]
  9%|▉         | 20078/225000 [50:06:50<410:49:47,  7.22s/it]
  9%|▉         | 20079/225000 [50:06:54<364:39:02,  6.41s/it]
  9%|▉         | 20080/225000 [50:07:00<342:23:17,  6.02s/it]
  9%|▉         | 20081/225000 [50:07:09<400:15:03,  7.03s/it]
  9%|▉         | 20082/225000 [50:07:16<408:05:13,  7.17s/it]
  9%|▉         | 20083/225000 [50:07:21<360:02:59,  6.33s/it]
  9%|▉         | 20084/225000 [50:07:32<437:33:37,  7.69s/it]
  9%|▉         | 20085/225000 [50:07:36<383:27:35,  6.74s/it]
  9%|▉         | 20086/225000 [50:07:41<352:49:20,  6.20s/it]
  9%|▉         | 20087/225000 [50:07:46<337:42:34,  5.93s/it]
  9%|▉         | 20088/225000 [50:07:53<345:08:36,  6.06s/it]
  9%|▉         | 20089/225000 [50:08:05<441:40:44,  7.76s/it]
  9%|▉         | 20090/225000 [50:08:09<388:57:53,  6.83s/it]
  9%|▉         | 20091/225000 [50:08:15<376:43:28,  6.62s/it]
  9%|▉         | 20092/225000 [50:08:20<338:10:26,  5.94s/it]
  9%|▉         | 20093/225000 [50:08:24<310:30:09,  5.46s/it]
  9%|▉         | 20094/225000 [50:08:33<369:28:38,  6.49s/it]
  9%|▉         | 20095/225000 [50:08:40<377:36:04,  6.63s/it]
  9%|▉         | 20096/225000 [50:08:45<351:10:33,  6.17s/it]
  9%|▉         | 20097/225000 [50:08:51<351:42:43,  6.18s/it]
  9%|▉         | 20098/225000 [50:09:02<438:45:40,  7.71s/it]
  9%|▉         | 20099/225000 [50:09:06<372:57:32,  6.55s/it]
  9%|▉         | 20100/225000 [50:09:15<415:40:43,  7.30s/it]
                                                             
{'loss': '0.7137', 'grad_norm': '0.8188', 'learning_rate': '4.973e-05', 'epoch': '0.08933', 'num_input_tokens_seen': 87945424, 'train_runtime': '1.806e+05', 'train_tokens_per_second': '487.1'}

  9%|▉         | 20100/225000 [50:09:15<415:40:43,  7.30s/it]
  9%|▉         | 20101/225000 [50:09:20<372:15:27,  6.54s/it]
  9%|▉         | 20102/225000 [50:09:28<395:32:20,  6.95s/it]
  9%|▉         | 20103/225000 [50:09:36<407:53:09,  7.17s/it]
  9%|▉         | 20104/225000 [50:09:40<359:16:37,  6.31s/it]
  9%|▉         | 20105/225000 [50:09:52<450:00:33,  7.91s/it]
  9%|▉         | 20106/225000 [50:09:56<389:09:00,  6.84s/it]
  9%|▉         | 20107/225000 [50:10:04<400:48:15,  7.04s/it]
  9%|▉         | 20108/225000 [50:10:11<403:38:21,  7.09s/it]
  9%|▉         | 20109/225000 [50:10:15<364:00:13,  6.40s/it]
  9%|▉         | 20110/225000 [50:10:20<330:50:28,  5.81s/it]
  9%|▉         | 20111/225000 [50:10:32<435:43:34,  7.66s/it]
  9%|▉         | 20112/225000 [50:10:37<397:08:49,  6.98s/it]
  9%|▉         | 20113/225000 [50:10:43<372:06:32,  6.54s/it]
  9%|▉         | 20114/225000 [50:10:49<361:45:30,  6.36s/it]
  9%|▉         | 20115/225000 [50:10:54<342:03:06,  6.01s/it]
  9%|▉         | 20116/225000 [50:10:59<324:36:10,  5.70s/it]
  9%|▉         | 20117/225000 [50:11:04<317:58:56,  5.59s/it]
  9%|▉         | 20118/225000 [50:11:10<320:03:58,  5.62s/it]
  9%|▉         | 20119/225000 [50:11:18<365:22:04,  6.42s/it]
  9%|▉         | 20120/225000 [50:11:32<486:13:51,  8.54s/it]
  9%|▉         | 20121/225000 [50:11:37<433:41:14,  7.62s/it]
  9%|▉         | 20122/225000 [50:11:41<375:43:55,  6.60s/it]
  9%|▉         | 20123/225000 [50:11:50<401:21:30,  7.05s/it]
  9%|▉         | 20124/225000 [50:11:57<412:03:44,  7.24s/it]
  9%|▉         | 20125/225000 [50:12:05<427:50:11,  7.52s/it]
  9%|▉         | 20126/225000 [50:12:12<415:30:56,  7.30s/it]
  9%|▉         | 20127/225000 [50:12:18<387:50:01,  6.81s/it]
  9%|▉         | 20128/225000 [50:12:23<361:25:36,  6.35s/it]
  9%|▉         | 20129/225000 [50:12:29<348:41:25,  6.13s/it]
  9%|▉         | 20130/225000 [50:12:34<337:15:07,  5.93s/it]
  9%|▉         | 20131/225000 [50:12:40<330:37:34,  5.81s/it]
  9%|▉         | 20132/225000 [50:12:49<387:48:13,  6.81s/it]
  9%|▉         | 20133/225000 [50:13:00<457:11:18,  8.03s/it]
  9%|▉         | 20134/225000 [50:13:07<449:08:32,  7.89s/it]
  9%|▉         | 20135/225000 [50:13:13<415:23:32,  7.30s/it]
  9%|▉         | 20136/225000 [50:13:19<385:03:03,  6.77s/it]
  9%|▉         | 20137/225000 [50:13:24<359:05:10,  6.31s/it]
  9%|▉         | 20138/225000 [50:13:30<347:42:37,  6.11s/it]
  9%|▉         | 20139/225000 [50:13:35<339:14:34,  5.96s/it]
  9%|▉         | 20140/225000 [50:13:41<339:15:27,  5.96s/it]
  9%|▉         | 20141/225000 [50:13:47<336:22:23,  5.91s/it]
  9%|▉         | 20142/225000 [50:13:53<345:21:12,  6.07s/it]
  9%|▉         | 20143/225000 [50:14:08<485:57:45,  8.54s/it]
  9%|▉         | 20144/225000 [50:14:13<429:02:15,  7.54s/it]
  9%|▉         | 20145/225000 [50:14:18<382:45:32,  6.73s/it]
  9%|▉         | 20146/225000 [50:14:30<470:09:00,  8.26s/it]
  9%|▉         | 20147/225000 [50:14:34<406:33:53,  7.14s/it]
  9%|▉         | 20148/225000 [50:14:41<394:47:17,  6.94s/it]
  9%|▉         | 20149/225000 [50:14:48<403:06:05,  7.08s/it]
  9%|▉         | 20150/225000 [50:14:55<408:26:45,  7.18s/it]
                                                             
{'loss': '0.7261', 'grad_norm': '0.692', 'learning_rate': '4.973e-05', 'epoch': '0.08956', 'num_input_tokens_seen': 88099184, 'train_runtime': '1.809e+05', 'train_tokens_per_second': '487'}

  9%|▉         | 20150/225000 [50:14:55<408:26:45,  7.18s/it]
  9%|▉         | 20151/225000 [50:15:01<373:04:32,  6.56s/it]
  9%|▉         | 20152/225000 [50:15:06<345:55:50,  6.08s/it]
  9%|▉         | 20153/225000 [50:15:11<338:10:23,  5.94s/it]
  9%|▉         | 20154/225000 [50:15:16<323:28:04,  5.68s/it]
  9%|▉         | 20155/225000 [50:15:25<374:29:30,  6.58s/it]
  9%|▉         | 20156/225000 [50:15:37<460:32:23,  8.09s/it]
  9%|▉         | 20157/225000 [50:15:42<418:19:42,  7.35s/it]
  9%|▉         | 20158/225000 [50:15:48<396:45:45,  6.97s/it]
  9%|▉         | 20159/225000 [50:15:54<376:46:13,  6.62s/it]
  9%|▉         | 20160/225000 [50:16:02<392:50:51,  6.90s/it]
  9%|▉         | 20161/225000 [50:16:10<412:00:21,  7.24s/it]
  9%|▉         | 20162/225000 [50:16:18<424:19:48,  7.46s/it]
  9%|▉         | 20163/225000 [50:16:21<362:09:15,  6.36s/it]
  9%|▉         | 20164/225000 [50:16:31<416:27:51,  7.32s/it]
  9%|▉         | 20165/225000 [50:16:35<363:39:47,  6.39s/it]
  9%|▉         | 20166/225000 [50:16:43<386:14:29,  6.79s/it]
  9%|▉         | 20167/225000 [50:16:48<361:21:06,  6.35s/it]
  9%|▉         | 20168/225000 [50:16:53<332:37:19,  5.85s/it]
  9%|▉         | 20169/225000 [50:16:58<325:34:27,  5.72s/it]
  9%|▉         | 20170/225000 [50:17:05<335:31:02,  5.90s/it]
  9%|▉         | 20171/225000 [50:17:17<447:33:01,  7.87s/it]
  9%|▉         | 20172/225000 [50:17:22<392:18:45,  6.90s/it]
  9%|▉         | 20173/225000 [50:17:28<375:19:04,  6.60s/it]
  9%|▉         | 20174/225000 [50:17:35<388:44:16,  6.83s/it]
  9%|▉         | 20175/225000 [50:17:42<390:40:27,  6.87s/it]
  9%|▉         | 20176/225000 [50:17:49<397:21:42,  6.98s/it]
  9%|▉         | 20177/225000 [50:17:56<387:17:22,  6.81s/it]
  9%|▉         | 20178/225000 [50:18:00<351:36:59,  6.18s/it]
  9%|▉         | 20179/225000 [50:18:05<328:00:32,  5.77s/it]
  9%|▉         | 20180/225000 [50:18:10<313:52:01,  5.52s/it]
  9%|▉         | 20181/225000 [50:18:21<411:28:54,  7.23s/it]
  9%|▉         | 20182/225000 [50:18:28<398:12:57,  7.00s/it]
  9%|▉         | 20183/225000 [50:18:35<401:13:30,  7.05s/it]
  9%|▉         | 20184/225000 [50:18:40<363:43:26,  6.39s/it]
  9%|▉         | 20185/225000 [50:18:44<335:00:06,  5.89s/it]
  9%|▉         | 20186/225000 [50:18:57<439:44:56,  7.73s/it]
  9%|▉         | 20187/225000 [50:19:01<379:26:26,  6.67s/it]
  9%|▉         | 20188/225000 [50:19:05<343:20:15,  6.03s/it]
  9%|▉         | 20189/225000 [50:19:13<373:27:31,  6.56s/it]
  9%|▉         | 20190/225000 [50:19:23<438:43:49,  7.71s/it]
  9%|▉         | 20191/225000 [50:19:27<374:26:46,  6.58s/it]
  9%|▉         | 20192/225000 [50:19:38<437:15:17,  7.69s/it]
  9%|▉         | 20193/225000 [50:19:43<394:10:40,  6.93s/it]
  9%|▉         | 20194/225000 [50:19:50<401:33:03,  7.06s/it]
  9%|▉         | 20195/225000 [50:19:58<409:53:16,  7.20s/it]
  9%|▉         | 20196/225000 [50:20:01<347:46:07,  6.11s/it]
  9%|▉         | 20197/225000 [50:20:12<425:15:49,  7.48s/it]
  9%|▉         | 20198/225000 [50:20:16<370:07:59,  6.51s/it]
  9%|▉         | 20199/225000 [50:20:20<331:16:23,  5.82s/it]
  9%|▉         | 20200/225000 [50:20:25<307:25:28,  5.40s/it]
                                                             
{'loss': '0.724', 'grad_norm': '0.7183', 'learning_rate': '4.972e-05', 'epoch': '0.08978', 'num_input_tokens_seen': 88250528, 'train_runtime': '1.812e+05', 'train_tokens_per_second': '487'}

  9%|▉         | 20200/225000 [50:20:25<307:25:28,  5.40s/it]
  9%|▉         | 20201/225000 [50:20:30<309:38:03,  5.44s/it]
  9%|▉         | 20202/225000 [50:20:36<304:59:17,  5.36s/it]
  9%|▉         | 20203/225000 [50:20:49<436:03:00,  7.67s/it]
  9%|▉         | 20204/225000 [50:20:55<418:48:13,  7.36s/it]
  9%|▉         | 20205/225000 [50:21:00<366:06:10,  6.44s/it]
  9%|▉         | 20206/225000 [50:21:04<334:39:27,  5.88s/it]
  9%|▉         | 20207/225000 [50:21:12<362:23:24,  6.37s/it]
  9%|▉         | 20208/225000 [50:21:22<428:26:29,  7.53s/it]
  9%|▉         | 20209/225000 [50:21:29<422:05:04,  7.42s/it]
  9%|▉         | 20210/225000 [50:21:34<374:48:02,  6.59s/it]
  9%|▉         | 20211/225000 [50:21:39<354:08:42,  6.23s/it]
  9%|▉         | 20212/225000 [50:21:44<337:37:22,  5.94s/it]
  9%|▉         | 20213/225000 [50:21:50<330:07:18,  5.80s/it]
  9%|▉         | 20214/225000 [50:22:02<439:39:33,  7.73s/it]
  9%|▉         | 20215/225000 [50:22:08<415:41:04,  7.31s/it]
  9%|▉         | 20216/225000 [50:22:14<391:52:08,  6.89s/it]
  9%|▉         | 20217/225000 [50:22:26<469:40:52,  8.26s/it]
  9%|▉         | 20218/225000 [50:22:30<400:07:14,  7.03s/it]
  9%|▉         | 20219/225000 [50:22:40<447:11:14,  7.86s/it]
  9%|▉         | 20220/225000 [50:22:45<405:54:28,  7.14s/it]
  9%|▉         | 20221/225000 [50:22:52<409:53:51,  7.21s/it]
  9%|▉         | 20222/225000 [50:22:58<375:35:09,  6.60s/it]
  9%|▉         | 20223/225000 [50:23:03<352:13:08,  6.19s/it]
  9%|▉         | 20224/225000 [50:23:08<340:26:04,  5.98s/it]
  9%|▉         | 20225/225000 [50:23:13<324:46:07,  5.71s/it]
  9%|▉         | 20226/225000 [50:23:19<326:14:25,  5.74s/it]
  9%|▉         | 20227/225000 [50:23:25<320:07:39,  5.63s/it]
  9%|▉         | 20228/225000 [50:23:33<374:11:44,  6.58s/it]
  9%|▉         | 20229/225000 [50:23:45<464:52:11,  8.17s/it]
  9%|▉         | 20230/225000 [50:23:52<441:36:16,  7.76s/it]
  9%|▉         | 20231/225000 [50:23:57<400:12:50,  7.04s/it]
  9%|▉         | 20232/225000 [50:24:02<365:19:33,  6.42s/it]
  9%|▉         | 20233/225000 [50:24:07<340:41:05,  5.99s/it]
  9%|▉         | 20234/225000 [50:24:14<354:29:48,  6.23s/it]
  9%|▉         | 20235/225000 [50:24:27<461:22:27,  8.11s/it]
  9%|▉         | 20236/225000 [50:24:32<413:09:54,  7.26s/it]
  9%|▉         | 20237/225000 [50:24:37<366:30:59,  6.44s/it]
  9%|▉         | 20238/225000 [50:24:41<330:57:10,  5.82s/it]
  9%|▉         | 20239/225000 [50:24:53<439:30:05,  7.73s/it]
  9%|▉         | 20240/225000 [50:24:59<403:09:16,  7.09s/it]
  9%|▉         | 20241/225000 [50:25:04<368:21:56,  6.48s/it]
  9%|▉         | 20242/225000 [50:25:09<340:35:36,  5.99s/it]
  9%|▉         | 20243/225000 [50:25:18<390:16:27,  6.86s/it]
  9%|▉         | 20244/225000 [50:25:27<442:12:12,  7.77s/it]
  9%|▉         | 20245/225000 [50:25:34<417:49:59,  7.35s/it]
  9%|▉         | 20246/225000 [50:25:42<427:17:16,  7.51s/it]
  9%|▉         | 20247/225000 [50:25:46<381:17:51,  6.70s/it]
  9%|▉         | 20248/225000 [50:25:51<340:20:32,  5.98s/it]
  9%|▉         | 20249/225000 [50:26:02<427:35:59,  7.52s/it]
  9%|▉         | 20250/225000 [50:26:07<394:49:21,  6.94s/it]
                                                             
{'loss': '0.7189', 'grad_norm': '0.6397', 'learning_rate': '4.972e-05', 'epoch': '0.09', 'num_input_tokens_seen': 88404760, 'train_runtime': '1.816e+05', 'train_tokens_per_second': '486.9'}

  9%|▉         | 20250/225000 [50:26:07<394:49:21,  6.94s/it]
  9%|▉         | 20251/225000 [50:26:13<365:11:36,  6.42s/it]
  9%|▉         | 20252/225000 [50:26:18<347:37:58,  6.11s/it]
  9%|▉         | 20253/225000 [50:26:26<382:03:35,  6.72s/it]
  9%|▉         | 20254/225000 [50:26:35<415:31:29,  7.31s/it]
  9%|▉         | 20255/225000 [50:26:40<385:17:50,  6.77s/it]
  9%|▉         | 20256/225000 [50:26:47<386:14:36,  6.79s/it]
  9%|▉         | 20257/225000 [50:26:53<360:30:15,  6.34s/it]
  9%|▉         | 20258/225000 [50:26:59<357:22:20,  6.28s/it]
  9%|▉         | 20259/225000 [50:27:03<327:53:47,  5.77s/it]
  9%|▉         | 20260/225000 [50:27:09<328:56:01,  5.78s/it]
  9%|▉         | 20261/225000 [50:27:15<329:53:47,  5.80s/it]
  9%|▉         | 20262/225000 [50:27:28<448:04:07,  7.88s/it]
  9%|▉         | 20263/225000 [50:27:34<419:13:13,  7.37s/it]
  9%|▉         | 20264/225000 [50:27:39<375:18:49,  6.60s/it]
  9%|▉         | 20265/225000 [50:27:46<390:44:47,  6.87s/it]
  9%|▉         | 20266/225000 [50:27:56<447:27:47,  7.87s/it]
  9%|▉         | 20267/225000 [50:28:01<391:06:28,  6.88s/it]
  9%|▉         | 20268/225000 [50:28:05<349:48:59,  6.15s/it]
  9%|▉         | 20269/225000 [50:28:10<328:04:57,  5.77s/it]
  9%|▉         | 20270/225000 [50:28:18<353:58:39,  6.22s/it]
  9%|▉         | 20271/225000 [50:28:28<428:19:19,  7.53s/it]
  9%|▉         | 20272/225000 [50:28:32<362:01:23,  6.37s/it]
  9%|▉         | 20273/225000 [50:28:40<401:48:24,  7.07s/it]
  9%|▉         | 20274/225000 [50:28:46<379:14:01,  6.67s/it]
  9%|▉         | 20275/225000 [50:28:54<402:27:41,  7.08s/it]
  9%|▉         | 20276/225000 [50:28:59<368:59:27,  6.49s/it]
  9%|▉         | 20277/225000 [50:29:04<333:54:07,  5.87s/it]
  9%|▉         | 20278/225000 [50:29:16<438:23:01,  7.71s/it]
  9%|▉         | 20279/225000 [50:29:22<419:31:19,  7.38s/it]
  9%|▉         | 20280/225000 [50:29:27<372:39:11,  6.55s/it]
  9%|▉         | 20281/225000 [50:29:31<330:28:37,  5.81s/it]
  9%|▉         | 20282/225000 [50:29:40<389:44:27,  6.85s/it]
  9%|▉         | 20283/225000 [50:29:46<371:31:16,  6.53s/it]
  9%|▉         | 20284/225000 [50:29:54<394:44:58,  6.94s/it]
  9%|▉         | 20285/225000 [50:29:59<358:24:19,  6.30s/it]
  9%|▉         | 20286/225000 [50:30:03<327:36:35,  5.76s/it]
  9%|▉         | 20287/225000 [50:30:09<317:40:34,  5.59s/it]
  9%|▉         | 20288/225000 [50:30:21<442:44:28,  7.79s/it]
  9%|▉         | 20289/225000 [50:30:26<391:56:32,  6.89s/it]
  9%|▉         | 20290/225000 [50:30:32<370:29:31,  6.52s/it]
  9%|▉         | 20291/225000 [50:30:37<348:18:15,  6.13s/it]
  9%|▉         | 20292/225000 [50:30:43<345:09:42,  6.07s/it]
  9%|▉         | 20293/225000 [50:30:48<333:38:59,  5.87s/it]
  9%|▉         | 20294/225000 [50:30:54<328:09:08,  5.77s/it]
  9%|▉         | 20295/225000 [50:30:59<322:40:45,  5.67s/it]
  9%|▉         | 20296/225000 [50:31:07<363:15:29,  6.39s/it]
  9%|▉         | 20297/225000 [50:31:21<480:02:31,  8.44s/it]
  9%|▉         | 20298/225000 [50:31:25<416:42:08,  7.33s/it]
  9%|▉         | 20299/225000 [50:31:30<370:18:27,  6.51s/it]
  9%|▉         | 20300/225000 [50:31:41<444:08:28,  7.81s/it]
                                                             
{'loss': '0.7147', 'grad_norm': '0.6284', 'learning_rate': '4.972e-05', 'epoch': '0.09022', 'num_input_tokens_seen': 88555880, 'train_runtime': '1.819e+05', 'train_tokens_per_second': '486.8'}

  9%|▉         | 20300/225000 [50:31:41<444:08:28,  7.81s/it]
  9%|▉         | 20301/225000 [50:31:45<387:06:41,  6.81s/it]
  9%|▉         | 20302/225000 [50:31:53<407:22:26,  7.16s/it]
  9%|▉         | 20303/225000 [50:32:00<400:00:06,  7.03s/it]
  9%|▉         | 20304/225000 [50:32:05<363:03:57,  6.39s/it]
  9%|▉         | 20305/225000 [50:32:11<358:58:51,  6.31s/it]
  9%|▉         | 20306/225000 [50:32:21<419:17:51,  7.37s/it]
  9%|▉         | 20307/225000 [50:32:29<430:21:21,  7.57s/it]
  9%|▉         | 20308/225000 [50:32:34<388:26:30,  6.83s/it]
  9%|▉         | 20309/225000 [50:32:40<374:40:09,  6.59s/it]
  9%|▉         | 20310/225000 [50:32:45<353:29:42,  6.22s/it]
  9%|▉         | 20311/225000 [50:32:56<431:26:31,  7.59s/it]
  9%|▉         | 20312/225000 [50:33:04<431:29:49,  7.59s/it]
  9%|▉         | 20313/225000 [50:33:08<380:40:14,  6.70s/it]
  9%|▉         | 20314/225000 [50:33:14<364:33:39,  6.41s/it]
  9%|▉         | 20315/225000 [50:33:25<441:52:14,  7.77s/it]
  9%|▉         | 20316/225000 [50:33:29<378:07:10,  6.65s/it]
  9%|▉         | 20317/225000 [50:33:36<388:32:11,  6.83s/it]
  9%|▉         | 20318/225000 [50:33:45<413:20:46,  7.27s/it]
  9%|▉         | 20319/225000 [50:33:49<368:52:16,  6.49s/it]
  9%|▉         | 20320/225000 [50:33:54<339:18:46,  5.97s/it]
  9%|▉         | 20321/225000 [50:33:59<321:46:05,  5.66s/it]
  9%|▉         | 20322/225000 [50:34:09<387:53:07,  6.82s/it]
  9%|▉         | 20323/225000 [50:34:18<436:32:07,  7.68s/it]
  9%|▉         | 20324/225000 [50:34:24<399:09:54,  7.02s/it]
  9%|▉         | 20325/225000 [50:34:29<375:54:10,  6.61s/it]
  9%|▉         | 20326/225000 [50:34:35<356:34:56,  6.27s/it]
  9%|▉         | 20327/225000 [50:34:40<341:51:19,  6.01s/it]
  9%|▉         | 20328/225000 [50:34:46<328:27:15,  5.78s/it]
  9%|▉         | 20329/225000 [50:34:51<323:52:11,  5.70s/it]
  9%|▉         | 20330/225000 [50:34:57<319:23:17,  5.62s/it]
  9%|▉         | 20331/225000 [50:35:09<430:28:30,  7.57s/it]
  9%|▉         | 20332/225000 [50:35:15<417:06:33,  7.34s/it]
  9%|▉         | 20333/225000 [50:35:21<394:51:55,  6.95s/it]
  9%|▉         | 20334/225000 [50:35:29<398:48:54,  7.02s/it]
  9%|▉         | 20335/225000 [50:35:33<359:39:08,  6.33s/it]
  9%|▉         | 20336/225000 [50:35:38<335:17:19,  5.90s/it]
  9%|▉         | 20337/225000 [50:35:44<337:12:49,  5.93s/it]
  9%|▉         | 20338/225000 [50:35:57<458:02:57,  8.06s/it]
  9%|▉         | 20339/225000 [50:36:02<404:48:50,  7.12s/it]
  9%|▉         | 20340/225000 [50:36:07<363:27:29,  6.39s/it]
  9%|▉         | 20341/225000 [50:36:18<441:33:39,  7.77s/it]
  9%|▉         | 20342/225000 [50:36:23<398:53:24,  7.02s/it]
  9%|▉         | 20343/225000 [50:36:28<353:50:59,  6.22s/it]
  9%|▉         | 20344/225000 [50:36:40<457:47:58,  8.05s/it]
  9%|▉         | 20345/225000 [50:36:45<409:49:32,  7.21s/it]
  9%|▉         | 20346/225000 [50:36:51<384:31:41,  6.76s/it]
  9%|▉         | 20347/225000 [50:36:56<365:26:28,  6.43s/it]
  9%|▉         | 20348/225000 [50:37:02<349:19:33,  6.14s/it]
  9%|▉         | 20349/225000 [50:37:08<342:23:06,  6.02s/it]
  9%|▉         | 20350/225000 [50:37:18<414:54:18,  7.30s/it]
                                                             
{'loss': '0.7252', 'grad_norm': '0.6203', 'learning_rate': '4.972e-05', 'epoch': '0.09044', 'num_input_tokens_seen': 88709712, 'train_runtime': '1.822e+05', 'train_tokens_per_second': '486.8'}

  9%|▉         | 20350/225000 [50:37:18<414:54:18,  7.30s/it]
  9%|▉         | 20351/225000 [50:37:26<426:53:13,  7.51s/it]
  9%|▉         | 20352/225000 [50:37:31<391:16:46,  6.88s/it]
  9%|▉         | 20353/225000 [50:37:36<359:56:26,  6.33s/it]
  9%|▉         | 20354/225000 [50:37:42<340:40:35,  5.99s/it]
  9%|▉         | 20355/225000 [50:37:47<331:44:49,  5.84s/it]
  9%|▉         | 20356/225000 [50:37:52<320:47:47,  5.64s/it]
  9%|▉         | 20357/225000 [50:38:04<424:47:34,  7.47s/it]
  9%|▉         | 20358/225000 [50:38:12<430:19:57,  7.57s/it]
  9%|▉         | 20359/225000 [50:38:17<386:30:32,  6.80s/it]
  9%|▉         | 20360/225000 [50:38:22<354:36:42,  6.24s/it]
  9%|▉         | 20361/225000 [50:38:27<335:18:15,  5.90s/it]
  9%|▉         | 20362/225000 [50:38:35<365:31:54,  6.43s/it]
  9%|▉         | 20363/225000 [50:38:47<473:53:22,  8.34s/it]
  9%|▉         | 20364/225000 [50:38:51<391:37:29,  6.89s/it]
  9%|▉         | 20365/225000 [50:38:55<353:21:06,  6.22s/it]
  9%|▉         | 20366/225000 [50:39:01<333:42:50,  5.87s/it]
  9%|▉         | 20367/225000 [50:39:06<320:27:38,  5.64s/it]
  9%|▉         | 20368/225000 [50:39:11<310:34:03,  5.46s/it]
  9%|▉         | 20369/225000 [50:39:19<355:51:11,  6.26s/it]
  9%|▉         | 20370/225000 [50:39:29<423:11:43,  7.45s/it]
  9%|▉         | 20371/225000 [50:39:33<361:22:58,  6.36s/it]
  9%|▉         | 20372/225000 [50:39:44<441:20:23,  7.76s/it]
  9%|▉         | 20373/225000 [50:39:48<384:58:12,  6.77s/it]
  9%|▉         | 20374/225000 [50:39:58<434:26:38,  7.64s/it]
  9%|▉         | 20375/225000 [50:40:02<377:34:28,  6.64s/it]
  9%|▉         | 20376/225000 [50:40:12<433:56:09,  7.63s/it]
  9%|▉         | 20377/225000 [50:40:18<404:14:38,  7.11s/it]
  9%|▉         | 20378/225000 [50:40:26<411:54:10,  7.25s/it]
  9%|▉         | 20379/225000 [50:40:30<360:12:12,  6.34s/it]
  9%|▉         | 20380/225000 [50:40:35<343:27:15,  6.04s/it]
  9%|▉         | 20381/225000 [50:40:41<339:02:28,  5.96s/it]
  9%|▉         | 20382/225000 [50:40:47<331:17:40,  5.83s/it]
  9%|▉         | 20383/225000 [50:40:52<331:30:05,  5.83s/it]
  9%|▉         | 20384/225000 [50:40:58<329:29:17,  5.80s/it]
  9%|▉         | 20385/225000 [50:41:11<450:13:04,  7.92s/it]
  9%|▉         | 20386/225000 [50:41:19<447:14:42,  7.87s/it]
  9%|▉         | 20387/225000 [50:41:24<396:36:34,  6.98s/it]
  9%|▉         | 20388/225000 [50:41:29<359:56:11,  6.33s/it]
  9%|▉         | 20389/225000 [50:41:34<339:22:36,  5.97s/it]
  9%|▉         | 20390/225000 [50:41:48<489:38:45,  8.62s/it]
  9%|▉         | 20391/225000 [50:41:53<417:38:29,  7.35s/it]
  9%|▉         | 20392/225000 [50:42:00<414:17:20,  7.29s/it]
  9%|▉         | 20393/225000 [50:42:07<409:16:38,  7.20s/it]
  9%|▉         | 20394/225000 [50:42:15<429:02:30,  7.55s/it]
  9%|▉         | 20395/225000 [50:42:21<393:35:21,  6.93s/it]
  9%|▉         | 20396/225000 [50:42:30<426:07:17,  7.50s/it]
  9%|▉         | 20397/225000 [50:42:36<404:05:09,  7.11s/it]
  9%|▉         | 20398/225000 [50:42:42<386:59:30,  6.81s/it]
  9%|▉         | 20399/225000 [50:42:48<381:06:48,  6.71s/it]
  9%|▉         | 20400/225000 [50:42:57<407:53:54,  7.18s/it]
                                                             
{'loss': '0.7178', 'grad_norm': '0.5724', 'learning_rate': '4.971e-05', 'epoch': '0.09067', 'num_input_tokens_seen': 88861736, 'train_runtime': '1.826e+05', 'train_tokens_per_second': '486.7'}

  9%|▉         | 20400/225000 [50:42:57<407:53:54,  7.18s/it]
  9%|▉         | 20401/225000 [50:43:02<380:12:09,  6.69s/it]
  9%|▉         | 20402/225000 [50:43:11<411:21:38,  7.24s/it]
  9%|▉         | 20403/225000 [50:43:17<388:17:49,  6.83s/it]
  9%|▉         | 20404/225000 [50:43:20<336:27:23,  5.92s/it]
  9%|▉         | 20405/225000 [50:43:30<394:36:29,  6.94s/it]
  9%|▉         | 20406/225000 [50:43:37<398:37:37,  7.01s/it]
  9%|▉         | 20407/225000 [50:43:45<423:34:41,  7.45s/it]
  9%|▉         | 20408/225000 [50:43:49<364:54:49,  6.42s/it]
  9%|▉         | 20409/225000 [50:43:59<414:23:01,  7.29s/it]
  9%|▉         | 20410/225000 [50:44:03<368:16:43,  6.48s/it]
  9%|▉         | 20411/225000 [50:44:08<337:57:22,  5.95s/it]
  9%|▉         | 20412/225000 [50:44:19<429:15:06,  7.55s/it]
  9%|▉         | 20413/225000 [50:44:23<358:12:05,  6.30s/it]
  9%|▉         | 20414/225000 [50:44:32<400:46:40,  7.05s/it]
  9%|▉         | 20415/225000 [50:44:35<348:13:16,  6.13s/it]
  9%|▉         | 20416/225000 [50:44:45<398:21:09,  7.01s/it]
  9%|▉         | 20417/225000 [50:44:49<357:45:26,  6.30s/it]
  9%|▉         | 20418/225000 [50:44:54<333:59:36,  5.88s/it]
  9%|▉         | 20419/225000 [50:44:59<316:57:49,  5.58s/it]
  9%|▉         | 20420/225000 [50:45:11<434:48:45,  7.65s/it]
  9%|▉         | 20421/225000 [50:45:16<382:19:31,  6.73s/it]
  9%|▉         | 20422/225000 [50:45:21<359:57:33,  6.33s/it]
  9%|▉         | 20423/225000 [50:45:27<354:24:54,  6.24s/it]
  9%|▉         | 20424/225000 [50:45:32<329:29:25,  5.80s/it]
  9%|▉         | 20425/225000 [50:45:38<322:19:08,  5.67s/it]
  9%|▉         | 20426/225000 [50:45:43<316:09:56,  5.56s/it]
  9%|▉         | 20427/225000 [50:45:53<393:10:40,  6.92s/it]
  9%|▉         | 20428/225000 [50:46:02<437:02:17,  7.69s/it]
  9%|▉         | 20429/225000 [50:46:11<448:04:31,  7.89s/it]
  9%|▉         | 20430/225000 [50:46:18<437:01:53,  7.69s/it]
  9%|▉         | 20431/225000 [50:46:24<409:30:10,  7.21s/it]
  9%|▉         | 20432/225000 [50:46:31<397:13:51,  6.99s/it]
  9%|▉         | 20433/225000 [50:46:35<360:30:28,  6.34s/it]
  9%|▉         | 20434/225000 [50:46:40<331:07:46,  5.83s/it]
  9%|▉         | 20435/225000 [50:46:45<315:45:46,  5.56s/it]
  9%|▉         | 20436/225000 [50:46:57<426:04:44,  7.50s/it]
  9%|▉         | 20437/225000 [50:47:03<406:59:46,  7.16s/it]
  9%|▉         | 20438/225000 [50:47:09<372:28:43,  6.56s/it]
  9%|▉         | 20439/225000 [50:47:13<339:14:00,  5.97s/it]
  9%|▉         | 20440/225000 [50:47:26<449:35:48,  7.91s/it]
  9%|▉         | 20441/225000 [50:47:32<417:11:07,  7.34s/it]
  9%|▉         | 20442/225000 [50:47:36<371:08:56,  6.53s/it]
  9%|▉         | 20443/225000 [50:47:41<344:40:03,  6.07s/it]
  9%|▉         | 20444/225000 [50:47:46<323:28:02,  5.69s/it]
  9%|▉         | 20445/225000 [50:47:51<313:08:03,  5.51s/it]
  9%|▉         | 20446/225000 [50:47:56<306:07:28,  5.39s/it]
  9%|▉         | 20447/225000 [50:48:09<432:24:17,  7.61s/it]
  9%|▉         | 20448/225000 [50:48:15<400:28:05,  7.05s/it]
  9%|▉         | 20449/225000 [50:48:23<411:58:11,  7.25s/it]
  9%|▉         | 20450/225000 [50:48:28<375:22:49,  6.61s/it]
                                                             
{'loss': '0.7173', 'grad_norm': '0.6364', 'learning_rate': '4.971e-05', 'epoch': '0.09089', 'num_input_tokens_seen': 89013040, 'train_runtime': '1.829e+05', 'train_tokens_per_second': '486.7'}

  9%|▉         | 20450/225000 [50:48:28<375:22:49,  6.61s/it]
  9%|▉         | 20451/225000 [50:48:38<433:18:17,  7.63s/it]
  9%|▉         | 20452/225000 [50:48:41<362:25:05,  6.38s/it]
  9%|▉         | 20453/225000 [50:48:50<412:36:24,  7.26s/it]
  9%|▉         | 20454/225000 [50:48:54<355:35:37,  6.26s/it]
  9%|▉         | 20455/225000 [50:48:59<328:03:07,  5.77s/it]
  9%|▉         | 20456/225000 [50:49:14<487:57:31,  8.59s/it]
  9%|▉         | 20457/225000 [50:49:18<412:14:59,  7.26s/it]
  9%|▉         | 20458/225000 [50:49:23<361:23:06,  6.36s/it]
  9%|▉         | 20459/225000 [50:49:31<402:21:50,  7.08s/it]
  9%|▉         | 20460/225000 [50:49:35<352:14:28,  6.20s/it]
  9%|▉         | 20461/225000 [50:49:45<407:38:53,  7.17s/it]
  9%|▉         | 20462/225000 [50:49:49<356:16:43,  6.27s/it]
  9%|▉         | 20463/225000 [50:49:58<397:04:53,  6.99s/it]
  9%|▉         | 20464/225000 [50:50:02<344:01:21,  6.06s/it]
  9%|▉         | 20465/225000 [50:50:12<413:03:23,  7.27s/it]
  9%|▉         | 20466/225000 [50:50:17<371:57:46,  6.55s/it]
  9%|▉         | 20467/225000 [50:50:25<397:18:33,  6.99s/it]
  9%|▉         | 20468/225000 [50:50:31<391:13:06,  6.89s/it]
  9%|▉         | 20469/225000 [50:50:40<428:20:28,  7.54s/it]
  9%|▉         | 20470/225000 [50:50:44<363:35:52,  6.40s/it]
  9%|▉         | 20471/225000 [50:50:51<378:14:41,  6.66s/it]
  9%|▉         | 20472/225000 [50:50:57<356:01:53,  6.27s/it]
  9%|▉         | 20473/225000 [50:51:02<332:39:06,  5.86s/it]
  9%|▉         | 20474/225000 [50:51:07<320:24:19,  5.64s/it]
  9%|▉         | 20475/225000 [50:51:12<311:35:04,  5.48s/it]
  9%|▉         | 20476/225000 [50:51:17<310:06:51,  5.46s/it]
  9%|▉         | 20477/225000 [50:51:26<368:46:49,  6.49s/it]
  9%|▉         | 20478/225000 [50:51:39<471:58:06,  8.31s/it]
  9%|▉         | 20479/225000 [50:51:43<411:14:43,  7.24s/it]
  9%|▉         | 20480/225000 [50:51:48<371:11:05,  6.53s/it]
  9%|▉         | 20481/225000 [50:51:53<342:49:01,  6.03s/it]
  9%|▉         | 20482/225000 [50:52:00<350:23:53,  6.17s/it]
  9%|▉         | 20483/225000 [50:52:09<412:28:13,  7.26s/it]
  9%|▉         | 20484/225000 [50:52:15<383:17:47,  6.75s/it]
  9%|▉         | 20485/225000 [50:52:20<350:43:08,  6.17s/it]
  9%|▉         | 20486/225000 [50:52:29<399:22:51,  7.03s/it]
  9%|▉         | 20487/225000 [50:52:35<379:30:41,  6.68s/it]
  9%|▉         | 20488/225000 [50:52:39<343:40:12,  6.05s/it]
  9%|▉         | 20489/225000 [50:52:45<335:53:46,  5.91s/it]
  9%|▉         | 20490/225000 [50:52:52<355:29:01,  6.26s/it]
  9%|▉         | 20491/225000 [50:53:05<466:23:52,  8.21s/it]
  9%|▉         | 20492/225000 [50:53:10<408:39:30,  7.19s/it]
  9%|▉         | 20493/225000 [50:53:18<426:50:52,  7.51s/it]
  9%|▉         | 20494/225000 [50:53:23<391:26:45,  6.89s/it]
  9%|▉         | 20495/225000 [50:53:31<405:16:10,  7.13s/it]
  9%|▉         | 20496/225000 [50:53:36<368:22:26,  6.48s/it]
  9%|▉         | 20497/225000 [50:53:40<336:00:45,  5.92s/it]
  9%|▉         | 20498/225000 [50:53:46<322:03:08,  5.67s/it]
  9%|▉         | 20499/225000 [50:54:00<467:06:44,  8.22s/it]
  9%|▉         | 20500/225000 [50:54:05<412:29:25,  7.26s/it]
                                                             
{'loss': '0.6817', 'grad_norm': '0.5999', 'learning_rate': '4.971e-05', 'epoch': '0.09111', 'num_input_tokens_seen': 89163728, 'train_runtime': '1.832e+05', 'train_tokens_per_second': '486.6'}

  9%|▉         | 20500/225000 [50:54:05<412:29:25,  7.26s/it]
  9%|▉         | 20501/225000 [50:54:12<407:54:44,  7.18s/it]
  9%|▉         | 20502/225000 [50:54:17<380:20:24,  6.70s/it]
  9%|▉         | 20503/225000 [50:54:22<350:13:07,  6.17s/it]
  9%|▉         | 20504/225000 [50:54:28<338:53:10,  5.97s/it]
  9%|▉         | 20505/225000 [50:54:33<333:54:22,  5.88s/it]
  9%|▉         | 20506/225000 [50:54:39<323:38:31,  5.70s/it]
  9%|▉         | 20507/225000 [50:54:52<444:38:23,  7.83s/it]
  9%|▉         | 20508/225000 [50:55:00<456:49:56,  8.04s/it]
  9%|▉         | 20509/225000 [50:55:05<398:55:43,  7.02s/it]
  9%|▉         | 20510/225000 [50:55:15<453:09:55,  7.98s/it]
  9%|▉         | 20511/225000 [50:55:19<378:20:42,  6.66s/it]
  9%|▉         | 20512/225000 [50:55:26<386:18:16,  6.80s/it]
  9%|▉         | 20513/225000 [50:55:30<340:27:20,  5.99s/it]
  9%|▉         | 20514/225000 [50:55:39<399:03:58,  7.03s/it]
  9%|▉         | 20515/225000 [50:55:46<394:00:13,  6.94s/it]
  9%|▉         | 20516/225000 [50:55:51<358:42:36,  6.32s/it]
  9%|▉         | 20517/225000 [50:56:01<428:13:06,  7.54s/it]
  9%|▉         | 20518/225000 [50:56:07<407:24:18,  7.17s/it]
  9%|▉         | 20519/225000 [50:56:12<359:23:07,  6.33s/it]
  9%|▉         | 20520/225000 [50:56:22<416:49:06,  7.34s/it]
  9%|▉         | 20521/225000 [50:56:27<381:19:47,  6.71s/it]
  9%|▉         | 20522/225000 [50:56:31<344:12:02,  6.06s/it]
  9%|▉         | 20523/225000 [50:56:37<330:16:03,  5.81s/it]
  9%|▉         | 20524/225000 [50:56:42<316:02:11,  5.56s/it]
  9%|▉         | 20525/225000 [50:56:51<374:16:50,  6.59s/it]
  9%|▉         | 20526/225000 [50:56:59<407:48:50,  7.18s/it]
  9%|▉         | 20527/225000 [50:57:03<357:04:26,  6.29s/it]
  9%|▉         | 20528/225000 [50:57:08<326:43:09,  5.75s/it]
  9%|▉         | 20529/225000 [50:57:13<321:21:25,  5.66s/it]
  9%|▉         | 20530/225000 [50:57:19<316:22:37,  5.57s/it]
  9%|▉         | 20531/225000 [50:57:24<316:44:35,  5.58s/it]
  9%|▉         | 20532/225000 [50:57:36<426:34:52,  7.51s/it]
  9%|▉         | 20533/225000 [50:57:46<461:52:22,  8.13s/it]
  9%|▉         | 20534/225000 [50:57:50<397:45:22,  7.00s/it]
  9%|▉         | 20535/225000 [50:57:57<393:42:54,  6.93s/it]
  9%|▉         | 20536/225000 [50:58:07<446:05:59,  7.85s/it]
  9%|▉         | 20537/225000 [50:58:14<435:28:54,  7.67s/it]
  9%|▉         | 20538/225000 [50:58:18<378:29:37,  6.66s/it]
  9%|▉         | 20539/225000 [50:58:28<426:49:32,  7.52s/it]
  9%|▉         | 20540/225000 [50:58:33<389:10:14,  6.85s/it]
  9%|▉         | 20541/225000 [50:58:39<371:04:08,  6.53s/it]
  9%|▉         | 20542/225000 [50:58:44<349:39:45,  6.16s/it]
  9%|▉         | 20543/225000 [50:58:50<332:59:20,  5.86s/it]
  9%|▉         | 20544/225000 [50:58:55<321:44:01,  5.66s/it]
  9%|▉         | 20545/225000 [50:59:07<436:25:06,  7.68s/it]
  9%|▉         | 20546/225000 [50:59:14<414:40:59,  7.30s/it]
  9%|▉         | 20547/225000 [50:59:18<360:01:35,  6.34s/it]
  9%|▉         | 20548/225000 [50:59:29<448:20:55,  7.89s/it]
  9%|▉         | 20549/225000 [50:59:33<381:24:04,  6.72s/it]
  9%|▉         | 20550/225000 [50:59:39<360:00:02,  6.34s/it]
                                                             
{'loss': '0.7188', 'grad_norm': '0.6058', 'learning_rate': '4.97e-05', 'epoch': '0.09133', 'num_input_tokens_seen': 89316128, 'train_runtime': '1.836e+05', 'train_tokens_per_second': '486.5'}

  9%|▉         | 20550/225000 [50:59:39<360:00:02,  6.34s/it]
  9%|▉         | 20551/225000 [50:59:46<384:31:56,  6.77s/it]
  9%|▉         | 20552/225000 [50:59:51<348:26:03,  6.14s/it]
  9%|▉         | 20553/225000 [51:00:02<430:34:24,  7.58s/it]
  9%|▉         | 20554/225000 [51:00:06<363:29:22,  6.40s/it]
  9%|▉         | 20555/225000 [51:00:15<411:08:03,  7.24s/it]
  9%|▉         | 20556/225000 [51:00:20<383:39:45,  6.76s/it]
  9%|▉         | 20557/225000 [51:00:25<353:55:51,  6.23s/it]
  9%|▉         | 20558/225000 [51:00:31<339:31:14,  5.98s/it]
  9%|▉         | 20559/225000 [51:00:36<332:17:27,  5.85s/it]
  9%|▉         | 20560/225000 [51:00:42<322:50:37,  5.68s/it]
  9%|▉         | 20561/225000 [51:00:47<316:12:35,  5.57s/it]
  9%|▉         | 20562/225000 [51:01:00<436:10:17,  7.68s/it]
  9%|▉         | 20563/225000 [51:01:07<425:53:09,  7.50s/it]
  9%|▉         | 20564/225000 [51:01:12<380:36:27,  6.70s/it]
  9%|▉         | 20565/225000 [51:01:17<355:26:50,  6.26s/it]
  9%|▉         | 20566/225000 [51:01:22<338:10:55,  5.96s/it]
  9%|▉         | 20567/225000 [51:01:28<333:20:44,  5.87s/it]
  9%|▉         | 20568/225000 [51:01:34<333:17:59,  5.87s/it]
  9%|▉         | 20569/225000 [51:01:40<339:42:56,  5.98s/it]
  9%|▉         | 20570/225000 [51:01:51<425:44:40,  7.50s/it]
  9%|▉         | 20571/225000 [51:02:02<484:48:21,  8.54s/it]
  9%|▉         | 20572/225000 [51:02:07<429:42:54,  7.57s/it]
  9%|▉         | 20573/225000 [51:02:12<383:46:49,  6.76s/it]
  9%|▉         | 20574/225000 [51:02:17<357:33:44,  6.30s/it]
  9%|▉         | 20575/225000 [51:02:22<333:14:12,  5.87s/it]
  9%|▉         | 20576/225000 [51:02:35<456:25:19,  8.04s/it]
  9%|▉         | 20577/225000 [51:02:40<401:57:10,  7.08s/it]
  9%|▉         | 20578/225000 [51:02:48<420:12:32,  7.40s/it]
  9%|▉         | 20579/225000 [51:02:56<421:41:17,  7.43s/it]
  9%|▉         | 20580/225000 [51:03:01<394:59:21,  6.96s/it]
  9%|▉         | 20581/225000 [51:03:07<367:35:01,  6.47s/it]
  9%|▉         | 20582/225000 [51:03:12<352:56:43,  6.22s/it]
  9%|▉         | 20583/225000 [51:03:18<337:43:08,  5.95s/it]
  9%|▉         | 20584/225000 [51:03:24<341:09:22,  6.01s/it]
  9%|▉         | 20585/225000 [51:03:29<332:16:37,  5.85s/it]
  9%|▉         | 20586/225000 [51:03:42<451:13:29,  7.95s/it]
  9%|▉         | 20587/225000 [51:03:50<452:15:58,  7.97s/it]
  9%|▉         | 20588/225000 [51:03:55<397:13:10,  7.00s/it]
  9%|▉         | 20589/225000 [51:03:59<353:13:42,  6.22s/it]
  9%|▉         | 20590/225000 [51:04:11<445:16:16,  7.84s/it]
  9%|▉         | 20591/225000 [51:04:15<379:31:07,  6.68s/it]
  9%|▉         | 20592/225000 [51:04:24<413:05:06,  7.28s/it]
  9%|▉         | 20593/225000 [51:04:31<409:56:40,  7.22s/it]
  9%|▉         | 20594/225000 [51:04:37<395:29:28,  6.97s/it]
  9%|▉         | 20595/225000 [51:04:41<348:23:23,  6.14s/it]
  9%|▉         | 20596/225000 [51:04:46<322:49:58,  5.69s/it]
  9%|▉         | 20597/225000 [51:04:59<441:19:03,  7.77s/it]
  9%|▉         | 20598/225000 [51:05:06<429:16:57,  7.56s/it]
  9%|▉         | 20599/225000 [51:05:11<392:03:07,  6.90s/it]
  9%|▉         | 20600/225000 [51:05:17<368:31:07,  6.49s/it]
                                                             
{'loss': '0.7141', 'grad_norm': '0.5858', 'learning_rate': '4.97e-05', 'epoch': '0.09156', 'num_input_tokens_seen': 89468040, 'train_runtime': '1.839e+05', 'train_tokens_per_second': '486.5'}

  9%|▉         | 20600/225000 [51:05:17<368:31:07,  6.49s/it]
  9%|▉         | 20601/225000 [51:05:26<413:27:11,  7.28s/it]
  9%|▉         | 20602/225000 [51:05:30<357:37:57,  6.30s/it]
  9%|▉         | 20603/225000 [51:05:34<323:05:56,  5.69s/it]
  9%|▉         | 20604/225000 [51:05:40<324:21:23,  5.71s/it]
  9%|▉         | 20605/225000 [51:05:53<446:08:30,  7.86s/it]
  9%|▉         | 20606/225000 [51:05:56<375:01:15,  6.61s/it]
  9%|▉         | 20607/225000 [51:06:02<356:07:22,  6.27s/it]
  9%|▉         | 20608/225000 [51:06:07<338:05:22,  5.95s/it]
  9%|▉         | 20609/225000 [51:06:13<343:06:49,  6.04s/it]
  9%|▉         | 20610/225000 [51:06:19<335:03:59,  5.90s/it]
  9%|▉         | 20611/225000 [51:06:25<334:21:56,  5.89s/it]
  9%|▉         | 20612/225000 [51:06:30<328:06:25,  5.78s/it]
  9%|▉         | 20613/225000 [51:06:43<456:16:04,  8.04s/it]
  9%|▉         | 20614/225000 [51:06:49<414:48:36,  7.31s/it]
  9%|▉         | 20615/225000 [51:06:54<381:53:52,  6.73s/it]
  9%|▉         | 20616/225000 [51:07:00<358:13:14,  6.31s/it]
  9%|▉         | 20617/225000 [51:07:06<350:47:27,  6.18s/it]
  9%|▉         | 20618/225000 [51:07:11<344:35:12,  6.07s/it]
  9%|▉         | 20619/225000 [51:07:17<328:03:17,  5.78s/it]
  9%|▉         | 20620/225000 [51:07:22<325:18:12,  5.73s/it]
  9%|▉         | 20621/225000 [51:07:34<435:49:22,  7.68s/it]
  9%|▉         | 20622/225000 [51:07:40<403:18:41,  7.10s/it]
  9%|▉         | 20623/225000 [51:07:51<458:26:11,  8.08s/it]
  9%|▉         | 20624/225000 [51:07:54<380:46:03,  6.71s/it]
  9%|▉         | 20625/225000 [51:08:02<408:41:06,  7.20s/it]
  9%|▉         | 20626/225000 [51:08:07<366:04:05,  6.45s/it]
  9%|▉         | 20627/225000 [51:08:12<334:25:09,  5.89s/it]
  9%|▉         | 20628/225000 [51:08:17<317:46:29,  5.60s/it]
  9%|▉         | 20629/225000 [51:08:29<440:40:24,  7.76s/it]
  9%|▉         | 20630/225000 [51:08:34<384:01:29,  6.76s/it]
  9%|▉         | 20631/225000 [51:08:38<340:16:53,  5.99s/it]
  9%|▉         | 20632/225000 [51:08:48<408:37:40,  7.20s/it]
  9%|▉         | 20633/225000 [51:08:54<382:10:55,  6.73s/it]
  9%|▉         | 20634/225000 [51:09:03<434:23:45,  7.65s/it]
  9%|▉         | 20635/225000 [51:09:07<370:01:53,  6.52s/it]
  9%|▉         | 20636/225000 [51:09:12<344:36:18,  6.07s/it]
  9%|▉         | 20637/225000 [51:09:17<319:04:58,  5.62s/it]
  9%|▉         | 20638/225000 [51:09:27<400:57:00,  7.06s/it]
  9%|▉         | 20639/225000 [51:09:35<414:56:09,  7.31s/it]
  9%|▉         | 20640/225000 [51:09:39<356:16:21,  6.28s/it]
  9%|▉         | 20641/225000 [51:09:49<418:33:52,  7.37s/it]
  9%|▉         | 20642/225000 [51:09:54<384:43:32,  6.78s/it]
  9%|▉         | 20643/225000 [51:10:02<405:24:13,  7.14s/it]
  9%|▉         | 20644/225000 [51:10:11<427:13:10,  7.53s/it]
  9%|▉         | 20645/225000 [51:10:14<352:08:37,  6.20s/it]
  9%|▉         | 20646/225000 [51:10:19<331:18:34,  5.84s/it]
  9%|▉         | 20647/225000 [51:10:25<330:24:10,  5.82s/it]
  9%|▉         | 20648/225000 [51:10:34<388:52:25,  6.85s/it]
  9%|▉         | 20649/225000 [51:10:40<375:35:01,  6.62s/it]
  9%|▉         | 20650/225000 [51:10:45<342:03:43,  6.03s/it]
                                                             
{'loss': '0.7226', 'grad_norm': '0.5807', 'learning_rate': '4.97e-05', 'epoch': '0.09178', 'num_input_tokens_seen': 89620240, 'train_runtime': '1.842e+05', 'train_tokens_per_second': '486.4'}

  9%|▉         | 20650/225000 [51:10:45<342:03:43,  6.03s/it]
  9%|▉         | 20651/225000 [51:10:51<351:52:35,  6.20s/it]
  9%|▉         | 20652/225000 [51:11:03<444:53:22,  7.84s/it]
  9%|▉         | 20653/225000 [51:11:07<388:07:43,  6.84s/it]
  9%|▉         | 20654/225000 [51:11:15<394:47:23,  6.96s/it]
  9%|▉         | 20655/225000 [51:11:20<359:00:45,  6.32s/it]
  9%|▉         | 20656/225000 [51:11:24<334:51:39,  5.90s/it]
  9%|▉         | 20657/225000 [51:11:30<328:01:48,  5.78s/it]
  9%|▉         | 20658/225000 [51:11:35<323:00:10,  5.69s/it]
  9%|▉         | 20659/225000 [51:11:41<318:13:51,  5.61s/it]
  9%|▉         | 20660/225000 [51:11:46<316:34:46,  5.58s/it]
  9%|▉         | 20661/225000 [51:11:59<435:32:27,  7.67s/it]
  9%|▉         | 20662/225000 [51:12:06<425:59:40,  7.51s/it]
  9%|▉         | 20663/225000 [51:12:13<411:26:36,  7.25s/it]
  9%|▉         | 20664/225000 [51:12:18<379:43:49,  6.69s/it]
  9%|▉         | 20665/225000 [51:12:23<351:25:05,  6.19s/it]
  9%|▉         | 20666/225000 [51:12:29<338:05:37,  5.96s/it]
  9%|▉         | 20667/225000 [51:12:34<321:55:24,  5.67s/it]
  9%|▉         | 20668/225000 [51:12:41<355:04:04,  6.26s/it]
  9%|▉         | 20669/225000 [51:12:55<481:37:57,  8.49s/it]
  9%|▉         | 20670/225000 [51:13:00<420:09:43,  7.40s/it]
  9%|▉         | 20671/225000 [51:13:08<440:10:19,  7.76s/it]
  9%|▉         | 20672/225000 [51:13:13<392:04:22,  6.91s/it]
  9%|▉         | 20673/225000 [51:13:17<346:32:47,  6.11s/it]
  9%|▉         | 20674/225000 [51:13:30<450:56:04,  7.94s/it]
  9%|▉         | 20675/225000 [51:13:35<400:26:19,  7.06s/it]
  9%|▉         | 20676/225000 [51:13:39<348:25:57,  6.14s/it]
  9%|▉         | 20677/225000 [51:13:51<453:13:27,  7.99s/it]
  9%|▉         | 20678/225000 [51:13:55<384:43:20,  6.78s/it]
  9%|▉         | 20679/225000 [51:14:04<416:59:48,  7.35s/it]
  9%|▉         | 20680/225000 [51:14:11<423:24:31,  7.46s/it]
  9%|▉         | 20681/225000 [51:14:18<415:10:44,  7.32s/it]
  9%|▉         | 20682/225000 [51:14:23<364:47:43,  6.43s/it]
  9%|▉         | 20683/225000 [51:14:33<423:54:41,  7.47s/it]
  9%|▉         | 20684/225000 [51:14:37<376:34:14,  6.64s/it]
  9%|▉         | 20685/225000 [51:14:42<348:58:56,  6.15s/it]
  9%|▉         | 20686/225000 [51:14:48<337:09:36,  5.94s/it]
  9%|▉         | 20687/225000 [51:14:54<336:25:03,  5.93s/it]
  9%|▉         | 20688/225000 [51:15:09<504:37:17,  8.89s/it]
  9%|▉         | 20689/225000 [51:15:16<473:03:38,  8.34s/it]
  9%|▉         | 20690/225000 [51:15:21<400:43:36,  7.06s/it]
  9%|▉         | 20691/225000 [51:15:29<416:15:39,  7.33s/it]
  9%|▉         | 20692/225000 [51:15:34<387:40:48,  6.83s/it]
  9%|▉         | 20693/225000 [51:15:40<362:50:16,  6.39s/it]
  9%|▉         | 20694/225000 [51:15:45<347:56:17,  6.13s/it]
  9%|▉         | 20695/225000 [51:15:51<337:59:13,  5.96s/it]
  9%|▉         | 20696/225000 [51:15:58<358:39:37,  6.32s/it]
  9%|▉         | 20697/225000 [51:16:11<481:48:05,  8.49s/it]
  9%|▉         | 20698/225000 [51:16:17<437:28:01,  7.71s/it]
  9%|▉         | 20699/225000 [51:16:24<424:35:04,  7.48s/it]
  9%|▉         | 20700/225000 [51:16:32<438:04:23,  7.72s/it]
                                                             
{'loss': '0.7064', 'grad_norm': '0.6322', 'learning_rate': '4.97e-05', 'epoch': '0.092', 'num_input_tokens_seen': 89774720, 'train_runtime': '1.846e+05', 'train_tokens_per_second': '486.3'}

  9%|▉         | 20700/225000 [51:16:32<438:04:23,  7.72s/it]
  9%|▉         | 20701/225000 [51:16:37<377:50:40,  6.66s/it]
  9%|▉         | 20702/225000 [51:16:46<431:51:25,  7.61s/it]
  9%|▉         | 20703/225000 [51:16:53<414:26:18,  7.30s/it]
  9%|▉         | 20704/225000 [51:16:57<363:07:10,  6.40s/it]
  9%|▉         | 20705/225000 [51:17:02<328:59:07,  5.80s/it]
  9%|▉         | 20706/225000 [51:17:11<395:07:04,  6.96s/it]
  9%|▉         | 20707/225000 [51:17:16<355:31:11,  6.26s/it]
  9%|▉         | 20708/225000 [51:17:20<322:03:14,  5.68s/it]
  9%|▉         | 20709/225000 [51:17:26<314:52:52,  5.55s/it]
  9%|▉         | 20710/225000 [51:17:36<402:19:24,  7.09s/it]
  9%|▉         | 20711/225000 [51:17:41<364:00:30,  6.41s/it]
  9%|▉         | 20712/225000 [51:17:50<397:35:23,  7.01s/it]
  9%|▉         | 20713/225000 [51:17:59<432:34:47,  7.62s/it]
  9%|▉         | 20714/225000 [51:18:02<362:07:47,  6.38s/it]
  9%|▉         | 20715/225000 [51:18:11<407:57:55,  7.19s/it]
  9%|▉         | 20716/225000 [51:18:16<373:30:08,  6.58s/it]
  9%|▉         | 20717/225000 [51:18:21<347:10:18,  6.12s/it]
  9%|▉         | 20718/225000 [51:18:26<326:07:48,  5.75s/it]
  9%|▉         | 20719/225000 [51:18:31<314:17:05,  5.54s/it]
  9%|▉         | 20720/225000 [51:18:44<437:50:01,  7.72s/it]
  9%|▉         | 20721/225000 [51:18:50<409:15:17,  7.21s/it]
  9%|▉         | 20722/225000 [51:18:58<417:36:04,  7.36s/it]
  9%|▉         | 20723/225000 [51:19:05<419:05:19,  7.39s/it]
  9%|▉         | 20724/225000 [51:19:10<367:38:34,  6.48s/it]
  9%|▉         | 20725/225000 [51:19:14<337:56:59,  5.96s/it]
  9%|▉         | 20726/225000 [51:19:19<323:38:13,  5.70s/it]
  9%|▉         | 20727/225000 [51:19:25<318:57:58,  5.62s/it]
  9%|▉         | 20728/225000 [51:19:34<378:06:18,  6.66s/it]
  9%|▉         | 20729/225000 [51:19:44<428:44:00,  7.56s/it]
  9%|▉         | 20730/225000 [51:19:53<455:29:52,  8.03s/it]
  9%|▉         | 20731/225000 [51:19:57<386:14:59,  6.81s/it]
  9%|▉         | 20732/225000 [51:20:03<381:03:53,  6.72s/it]
  9%|▉         | 20733/225000 [51:20:12<413:01:59,  7.28s/it]
  9%|▉         | 20734/225000 [51:20:18<389:48:53,  6.87s/it]
  9%|▉         | 20735/225000 [51:20:25<396:40:50,  6.99s/it]
  9%|▉         | 20736/225000 [51:20:31<371:36:04,  6.55s/it]
  9%|▉         | 20737/225000 [51:20:36<351:27:03,  6.19s/it]
  9%|▉         | 20738/225000 [51:20:41<331:50:35,  5.85s/it]
  9%|▉         | 20739/225000 [51:20:46<326:00:32,  5.75s/it]
  9%|▉         | 20740/225000 [51:20:52<324:19:38,  5.72s/it]
  9%|▉         | 20741/225000 [51:20:58<319:29:10,  5.63s/it]
  9%|▉         | 20742/225000 [51:21:11<445:29:05,  7.85s/it]
  9%|▉         | 20743/225000 [51:21:18<444:19:51,  7.83s/it]
  9%|▉         | 20744/225000 [51:21:27<462:51:20,  8.16s/it]
  9%|▉         | 20745/225000 [51:21:31<388:00:09,  6.84s/it]
  9%|▉         | 20746/225000 [51:21:39<402:17:56,  7.09s/it]
  9%|▉         | 20747/225000 [51:21:44<364:02:57,  6.42s/it]
  9%|▉         | 20748/225000 [51:21:48<332:18:08,  5.86s/it]
  9%|▉         | 20749/225000 [51:21:55<344:31:28,  6.07s/it]
  9%|▉         | 20750/225000 [51:22:06<441:04:46,  7.77s/it]
                                                             
{'loss': '0.7351', 'grad_norm': '0.6415', 'learning_rate': '4.969e-05', 'epoch': '0.09222', 'num_input_tokens_seen': 89925944, 'train_runtime': '1.849e+05', 'train_tokens_per_second': '486.3'}

  9%|▉         | 20750/225000 [51:22:06<441:04:46,  7.77s/it]
  9%|▉         | 20751/225000 [51:22:11<382:48:25,  6.75s/it]
  9%|▉         | 20752/225000 [51:22:15<336:02:35,  5.92s/it]
  9%|▉         | 20753/225000 [51:22:24<393:11:17,  6.93s/it]
  9%|▉         | 20754/225000 [51:22:32<402:55:17,  7.10s/it]
  9%|▉         | 20755/225000 [51:22:38<397:03:29,  7.00s/it]
  9%|▉         | 20756/225000 [51:22:46<402:14:23,  7.09s/it]
  9%|▉         | 20757/225000 [51:22:51<378:37:34,  6.67s/it]
  9%|▉         | 20758/225000 [51:22:57<365:10:40,  6.44s/it]
  9%|▉         | 20759/225000 [51:23:03<355:27:14,  6.27s/it]
  9%|▉         | 20760/225000 [51:23:12<400:58:36,  7.07s/it]
  9%|▉         | 20761/225000 [51:23:19<408:19:20,  7.20s/it]
  9%|▉         | 20762/225000 [51:23:27<420:01:24,  7.40s/it]
  9%|▉         | 20763/225000 [51:23:31<360:25:37,  6.35s/it]
  9%|▉         | 20764/225000 [51:23:40<396:44:53,  6.99s/it]
  9%|▉         | 20765/225000 [51:23:47<399:52:08,  7.05s/it]
  9%|▉         | 20766/225000 [51:23:53<375:28:16,  6.62s/it]
  9%|▉         | 20767/225000 [51:24:00<394:02:59,  6.95s/it]
  9%|▉         | 20768/225000 [51:24:04<345:33:40,  6.09s/it]
  9%|▉         | 20769/225000 [51:24:15<417:58:31,  7.37s/it]
  9%|▉         | 20770/225000 [51:24:26<482:22:05,  8.50s/it]
  9%|▉         | 20771/225000 [51:24:29<393:28:30,  6.94s/it]
  9%|▉         | 20772/225000 [51:24:34<350:32:04,  6.18s/it]
  9%|▉         | 20773/225000 [51:24:39<344:45:58,  6.08s/it]
  9%|▉         | 20774/225000 [51:24:52<447:54:25,  7.90s/it]
  9%|▉         | 20775/225000 [51:24:55<375:12:38,  6.61s/it]
  9%|▉         | 20776/225000 [51:25:03<390:38:24,  6.89s/it]
  9%|▉         | 20777/225000 [51:25:07<352:59:07,  6.22s/it]
  9%|▉         | 20778/225000 [51:25:12<330:11:42,  5.82s/it]
  9%|▉         | 20779/225000 [51:25:16<301:12:18,  5.31s/it]
  9%|▉         | 20780/225000 [51:25:24<343:11:23,  6.05s/it]
  9%|▉         | 20781/225000 [51:25:36<447:17:19,  7.88s/it]
  9%|▉         | 20782/225000 [51:25:41<388:13:36,  6.84s/it]
  9%|▉         | 20783/225000 [51:25:51<438:27:22,  7.73s/it]
  9%|▉         | 20784/225000 [51:25:58<428:40:17,  7.56s/it]
  9%|▉         | 20785/225000 [51:26:02<381:35:09,  6.73s/it]
  9%|▉         | 20786/225000 [51:26:11<413:20:54,  7.29s/it]
  9%|▉         | 20787/225000 [51:26:16<367:56:21,  6.49s/it]
  9%|▉         | 20788/225000 [51:26:23<383:13:14,  6.76s/it]
  9%|▉         | 20789/225000 [51:26:31<396:14:11,  6.99s/it]
  9%|▉         | 20790/225000 [51:26:34<343:22:43,  6.05s/it]
  9%|▉         | 20791/225000 [51:26:44<400:23:07,  7.06s/it]
  9%|▉         | 20792/225000 [51:26:50<390:53:03,  6.89s/it]
  9%|▉         | 20793/225000 [51:26:54<335:17:58,  5.91s/it]
  9%|▉         | 20794/225000 [51:27:04<408:06:07,  7.19s/it]
  9%|▉         | 20795/225000 [51:27:09<365:01:57,  6.44s/it]
  9%|▉         | 20796/225000 [51:27:14<347:45:20,  6.13s/it]
  9%|▉         | 20797/225000 [51:27:19<328:40:24,  5.79s/it]
  9%|▉         | 20798/225000 [51:27:32<443:35:31,  7.82s/it]
  9%|▉         | 20799/225000 [51:27:38<421:14:18,  7.43s/it]
  9%|▉         | 20800/225000 [51:27:48<451:56:40,  7.97s/it]
                                                             
{'loss': '0.6969', 'grad_norm': '0.6262', 'learning_rate': '4.969e-05', 'epoch': '0.09244', 'num_input_tokens_seen': 90078952, 'train_runtime': '1.853e+05', 'train_tokens_per_second': '486.2'}

  9%|▉         | 20800/225000 [51:27:48<451:56:40,  7.97s/it]
  9%|▉         | 20801/225000 [51:27:51<381:32:15,  6.73s/it]
  9%|▉         | 20802/225000 [51:28:01<434:05:00,  7.65s/it]
  9%|▉         | 20803/225000 [51:28:06<383:56:27,  6.77s/it]
  9%|▉         | 20804/225000 [51:28:14<405:26:19,  7.15s/it]
  9%|▉         | 20805/225000 [51:28:17<341:55:06,  6.03s/it]
  9%|▉         | 20806/225000 [51:28:25<370:35:54,  6.53s/it]
  9%|▉         | 20807/225000 [51:28:30<338:37:24,  5.97s/it]
  9%|▉         | 20808/225000 [51:28:38<384:54:34,  6.79s/it]
  9%|▉         | 20809/225000 [51:28:44<364:50:48,  6.43s/it]
  9%|▉         | 20810/225000 [51:28:50<354:39:31,  6.25s/it]
  9%|▉         | 20811/225000 [51:28:56<344:29:31,  6.07s/it]
  9%|▉         | 20812/225000 [51:29:01<337:33:09,  5.95s/it]
  9%|▉         | 20813/225000 [51:29:08<345:03:31,  6.08s/it]
  9%|▉         | 20814/225000 [51:29:19<430:52:24,  7.60s/it]
  9%|▉         | 20815/225000 [51:29:26<434:18:31,  7.66s/it]
  9%|▉         | 20816/225000 [51:29:31<383:03:33,  6.75s/it]
  9%|▉         | 20817/225000 [51:29:40<414:10:53,  7.30s/it]
  9%|▉         | 20818/225000 [51:29:44<363:01:28,  6.40s/it]
  9%|▉         | 20819/225000 [51:29:52<396:16:48,  6.99s/it]
  9%|▉         | 20820/225000 [51:29:59<397:14:59,  7.00s/it]
  9%|▉         | 20821/225000 [51:30:05<379:18:06,  6.69s/it]
  9%|▉         | 20822/225000 [51:30:12<379:38:00,  6.69s/it]
  9%|▉         | 20823/225000 [51:30:19<383:44:35,  6.77s/it]
  9%|▉         | 20824/225000 [51:30:27<402:30:27,  7.10s/it]
  9%|▉         | 20825/225000 [51:30:31<352:36:32,  6.22s/it]
  9%|▉         | 20826/225000 [51:30:35<319:24:59,  5.63s/it]
  9%|▉         | 20827/225000 [51:30:48<447:51:14,  7.90s/it]
  9%|▉         | 20828/225000 [51:30:52<378:37:30,  6.68s/it]
  9%|▉         | 20829/225000 [51:31:02<423:13:51,  7.46s/it]
  9%|▉         | 20830/225000 [51:31:06<377:30:12,  6.66s/it]
  9%|▉         | 20831/225000 [51:31:14<387:32:20,  6.83s/it]
  9%|▉         | 20832/225000 [51:31:18<353:51:43,  6.24s/it]
  9%|▉         | 20833/225000 [51:31:23<325:39:25,  5.74s/it]
  9%|▉         | 20834/225000 [51:31:28<315:06:16,  5.56s/it]
  9%|▉         | 20835/225000 [51:31:41<442:32:27,  7.80s/it]
  9%|▉         | 20836/225000 [51:31:47<410:37:42,  7.24s/it]
  9%|▉         | 20837/225000 [51:31:51<354:00:36,  6.24s/it]
  9%|▉         | 20838/225000 [51:32:01<423:14:34,  7.46s/it]
  9%|▉         | 20839/225000 [51:32:06<380:54:21,  6.72s/it]
  9%|▉         | 20840/225000 [51:32:11<349:06:53,  6.16s/it]
  9%|▉         | 20841/225000 [51:32:17<338:18:42,  5.97s/it]
  9%|▉         | 20842/225000 [51:32:21<314:57:02,  5.55s/it]
  9%|▉         | 20843/225000 [51:32:34<429:58:07,  7.58s/it]
  9%|▉         | 20844/225000 [51:32:41<419:59:31,  7.41s/it]
  9%|▉         | 20845/225000 [51:32:45<370:16:23,  6.53s/it]
  9%|▉         | 20846/225000 [51:32:50<344:34:28,  6.08s/it]
  9%|▉         | 20847/225000 [51:32:59<392:28:37,  6.92s/it]
  9%|▉         | 20848/225000 [51:33:07<408:21:33,  7.20s/it]
  9%|▉         | 20849/225000 [51:33:11<362:28:28,  6.39s/it]
  9%|▉         | 20850/225000 [51:33:23<452:43:54,  7.98s/it]
                                                             
{'loss': '0.7354', 'grad_norm': '0.691', 'learning_rate': '4.969e-05', 'epoch': '0.09267', 'num_input_tokens_seen': 90229112, 'train_runtime': '1.856e+05', 'train_tokens_per_second': '486.1'}

  9%|▉         | 20850/225000 [51:33:23<452:43:54,  7.98s/it]
  9%|▉         | 20851/225000 [51:33:27<378:50:14,  6.68s/it]
  9%|▉         | 20852/225000 [51:33:32<348:16:53,  6.14s/it]
  9%|▉         | 20853/225000 [51:33:36<325:13:28,  5.74s/it]
  9%|▉         | 20854/225000 [51:33:44<359:24:11,  6.34s/it]
  9%|▉         | 20855/225000 [51:33:55<428:14:33,  7.55s/it]
  9%|▉         | 20856/225000 [51:34:00<386:38:05,  6.82s/it]
  9%|▉         | 20857/225000 [51:34:04<350:33:25,  6.18s/it]
  9%|▉         | 20858/225000 [51:34:10<343:50:38,  6.06s/it]
  9%|▉         | 20859/225000 [51:34:15<323:17:26,  5.70s/it]
  9%|▉         | 20860/225000 [51:34:20<320:16:40,  5.65s/it]
  9%|▉         | 20861/225000 [51:34:35<478:46:16,  8.44s/it]
  9%|▉         | 20862/225000 [51:34:42<440:09:24,  7.76s/it]
  9%|▉         | 20863/225000 [51:34:47<405:28:45,  7.15s/it]
  9%|▉         | 20864/225000 [51:34:53<380:31:45,  6.71s/it]
  9%|▉         | 20865/225000 [51:34:58<356:13:29,  6.28s/it]
  9%|▉         | 20866/225000 [51:35:04<347:03:31,  6.12s/it]
  9%|▉         | 20867/225000 [51:35:09<334:19:33,  5.90s/it]
  9%|▉         | 20868/225000 [51:35:16<339:54:37,  5.99s/it]
  9%|▉         | 20869/225000 [51:35:29<464:06:52,  8.19s/it]
  9%|▉         | 20870/225000 [51:35:35<421:57:04,  7.44s/it]
  9%|▉         | 20871/225000 [51:35:40<380:15:08,  6.71s/it]
  9%|▉         | 20872/225000 [51:35:45<357:21:33,  6.30s/it]
  9%|▉         | 20873/225000 [51:35:50<338:05:17,  5.96s/it]
  9%|▉         | 20874/225000 [51:35:58<373:22:43,  6.58s/it]
  9%|▉         | 20875/225000 [51:36:12<488:16:42,  8.61s/it]
  9%|▉         | 20876/225000 [51:36:17<435:42:14,  7.68s/it]
  9%|▉         | 20877/225000 [51:36:22<382:12:29,  6.74s/it]
  9%|▉         | 20878/225000 [51:36:30<409:31:54,  7.22s/it]
  9%|▉         | 20879/225000 [51:36:34<360:14:38,  6.35s/it]
  9%|▉         | 20880/225000 [51:36:39<339:18:03,  5.98s/it]
  9%|▉         | 20881/225000 [51:36:45<340:13:07,  6.00s/it]
  9%|▉         | 20882/225000 [51:36:51<333:40:21,  5.88s/it]
  9%|▉         | 20883/225000 [51:36:57<330:30:14,  5.83s/it]
  9%|▉         | 20884/225000 [51:37:10<451:15:41,  7.96s/it]
  9%|▉         | 20885/225000 [51:37:17<433:59:19,  7.65s/it]
  9%|▉         | 20886/225000 [51:37:22<387:01:22,  6.83s/it]
  9%|▉         | 20887/225000 [51:37:27<355:55:00,  6.28s/it]
  9%|▉         | 20888/225000 [51:37:40<480:13:47,  8.47s/it]
  9%|▉         | 20889/225000 [51:37:44<405:40:00,  7.15s/it]
  9%|▉         | 20890/225000 [51:37:48<356:58:21,  6.30s/it]
  9%|▉         | 20891/225000 [51:37:53<333:24:27,  5.88s/it]
  9%|▉         | 20892/225000 [51:38:05<433:02:28,  7.64s/it]
  9%|▉         | 20893/225000 [51:38:12<415:44:51,  7.33s/it]
  9%|▉         | 20894/225000 [51:38:16<359:30:18,  6.34s/it]
  9%|▉         | 20895/225000 [51:38:24<391:34:07,  6.91s/it]
  9%|▉         | 20896/225000 [51:38:30<369:38:32,  6.52s/it]
  9%|▉         | 20897/225000 [51:38:37<380:49:56,  6.72s/it]
  9%|▉         | 20898/225000 [51:38:43<364:54:50,  6.44s/it]
  9%|▉         | 20899/225000 [51:38:50<376:23:04,  6.64s/it]
  9%|▉         | 20900/225000 [51:38:55<349:30:31,  6.16s/it]
                                                             
{'loss': '0.7131', 'grad_norm': '0.7909', 'learning_rate': '4.968e-05', 'epoch': '0.09289', 'num_input_tokens_seen': 90380704, 'train_runtime': '1.859e+05', 'train_tokens_per_second': '486.1'}

  9%|▉         | 20900/225000 [51:38:55<349:30:31,  6.16s/it]
  9%|▉         | 20901/225000 [51:39:00<333:08:41,  5.88s/it]
  9%|▉         | 20902/225000 [51:39:11<414:17:54,  7.31s/it]
  9%|▉         | 20903/225000 [51:39:17<405:56:09,  7.16s/it]
  9%|▉         | 20904/225000 [51:39:22<354:27:21,  6.25s/it]
  9%|▉         | 20905/225000 [51:39:28<364:55:49,  6.44s/it]
  9%|▉         | 20906/225000 [51:39:38<413:52:27,  7.30s/it]
  9%|▉         | 20907/225000 [51:39:44<402:58:06,  7.11s/it]
  9%|▉         | 20908/225000 [51:39:53<430:30:43,  7.59s/it]
  9%|▉         | 20909/225000 [51:39:58<385:58:03,  6.81s/it]
  9%|▉         | 20910/225000 [51:40:03<346:58:57,  6.12s/it]
  9%|▉         | 20911/225000 [51:40:09<344:33:05,  6.08s/it]
  9%|▉         | 20912/225000 [51:40:21<444:37:52,  7.84s/it]
  9%|▉         | 20913/225000 [51:40:27<419:02:43,  7.39s/it]
  9%|▉         | 20914/225000 [51:40:34<407:31:29,  7.19s/it]
  9%|▉         | 20915/225000 [51:40:42<421:27:07,  7.43s/it]
  9%|▉         | 20916/225000 [51:40:48<401:39:31,  7.09s/it]
  9%|▉         | 20917/225000 [51:40:55<399:00:28,  7.04s/it]
  9%|▉         | 20918/225000 [51:40:59<352:33:58,  6.22s/it]
  9%|▉         | 20919/225000 [51:41:10<436:23:06,  7.70s/it]
  9%|▉         | 20920/225000 [51:41:14<371:55:42,  6.56s/it]
  9%|▉         | 20921/225000 [51:41:23<402:53:50,  7.11s/it]
  9%|▉         | 20922/225000 [51:41:27<359:12:33,  6.34s/it]
  9%|▉         | 20923/225000 [51:41:32<335:39:09,  5.92s/it]
  9%|▉         | 20924/225000 [51:41:45<450:14:05,  7.94s/it]
  9%|▉         | 20925/225000 [51:41:51<423:02:49,  7.46s/it]
  9%|▉         | 20926/225000 [51:41:55<360:46:48,  6.36s/it]
  9%|▉         | 20927/225000 [51:42:01<351:03:56,  6.19s/it]
  9%|▉         | 20928/225000 [51:42:06<344:18:23,  6.07s/it]
  9%|▉         | 20929/225000 [51:42:11<318:11:16,  5.61s/it]
  9%|▉         | 20930/225000 [51:42:16<311:32:11,  5.50s/it]
  9%|▉         | 20931/225000 [51:42:23<334:02:35,  5.89s/it]
  9%|▉         | 20932/225000 [51:42:37<473:16:02,  8.35s/it]
  9%|▉         | 20933/225000 [51:42:42<407:56:41,  7.20s/it]
  9%|▉         | 20934/225000 [51:42:46<357:58:30,  6.32s/it]
  9%|▉         | 20935/225000 [51:42:58<457:43:53,  8.08s/it]
  9%|▉         | 20936/225000 [51:43:04<428:52:38,  7.57s/it]
  9%|▉         | 20937/225000 [51:43:12<425:02:33,  7.50s/it]
  9%|▉         | 20938/225000 [51:43:20<429:05:00,  7.57s/it]
  9%|▉         | 20939/225000 [51:43:31<498:04:43,  8.79s/it]
  9%|▉         | 20940/225000 [51:43:36<423:13:34,  7.47s/it]
  9%|▉         | 20941/225000 [51:43:45<450:17:49,  7.94s/it]
  9%|▉         | 20942/225000 [51:43:49<387:08:41,  6.83s/it]
  9%|▉         | 20943/225000 [51:43:57<417:16:02,  7.36s/it]
  9%|▉         | 20944/225000 [51:44:04<406:27:55,  7.17s/it]
  9%|▉         | 20945/225000 [51:44:13<431:03:09,  7.60s/it]
  9%|▉         | 20946/225000 [51:44:17<367:09:55,  6.48s/it]
  9%|▉         | 20947/225000 [51:44:21<335:04:54,  5.91s/it]
  9%|▉         | 20948/225000 [51:44:34<453:10:47,  8.00s/it]
  9%|▉         | 20949/225000 [51:44:38<388:34:16,  6.86s/it]
  9%|▉         | 20950/225000 [51:44:43<347:12:22,  6.13s/it]
                                                             
{'loss': '0.7238', 'grad_norm': '0.5823', 'learning_rate': '4.968e-05', 'epoch': '0.09311', 'num_input_tokens_seen': 90535192, 'train_runtime': '1.863e+05', 'train_tokens_per_second': '486'}

  9%|▉         | 20950/225000 [51:44:43<347:12:22,  6.13s/it]
  9%|▉         | 20951/225000 [51:44:53<425:08:34,  7.50s/it]
  9%|▉         | 20952/225000 [51:44:58<377:02:04,  6.65s/it]
  9%|▉         | 20953/225000 [51:45:04<369:06:29,  6.51s/it]
  9%|▉         | 20954/225000 [51:45:14<421:51:36,  7.44s/it]
  9%|▉         | 20955/225000 [51:45:22<428:46:07,  7.56s/it]
  9%|▉         | 20956/225000 [51:45:28<411:54:23,  7.27s/it]
  9%|▉         | 20957/225000 [51:45:34<387:49:39,  6.84s/it]
  9%|▉         | 20958/225000 [51:45:42<407:19:13,  7.19s/it]
  9%|▉         | 20959/225000 [51:45:47<371:24:41,  6.55s/it]
  9%|▉         | 20960/225000 [51:45:57<419:18:13,  7.40s/it]
  9%|▉         | 20961/225000 [51:46:01<370:28:56,  6.54s/it]
  9%|▉         | 20962/225000 [51:46:10<408:13:09,  7.20s/it]
  9%|▉         | 20963/225000 [51:46:15<368:44:39,  6.51s/it]
  9%|▉         | 20964/225000 [51:46:19<329:52:35,  5.82s/it]
  9%|▉         | 20965/225000 [51:46:25<341:25:19,  6.02s/it]
  9%|▉         | 20966/225000 [51:46:36<420:03:19,  7.41s/it]
  9%|▉         | 20967/225000 [51:46:40<364:16:20,  6.43s/it]
  9%|▉         | 20968/225000 [51:46:48<384:58:37,  6.79s/it]
  9%|▉         | 20969/225000 [51:46:55<396:56:34,  7.00s/it]
  9%|▉         | 20970/225000 [51:47:00<352:52:23,  6.23s/it]
  9%|▉         | 20971/225000 [51:47:04<322:42:25,  5.69s/it]
  9%|▉         | 20972/225000 [51:47:18<459:14:36,  8.10s/it]
  9%|▉         | 20973/225000 [51:47:25<439:16:15,  7.75s/it]
  9%|▉         | 20974/225000 [51:47:30<396:47:45,  7.00s/it]
  9%|▉         | 20975/225000 [51:47:35<364:44:34,  6.44s/it]
  9%|▉         | 20976/225000 [51:47:40<339:02:58,  5.98s/it]
  9%|▉         | 20977/225000 [51:47:45<322:58:41,  5.70s/it]
  9%|▉         | 20978/225000 [51:47:50<312:44:30,  5.52s/it]
  9%|▉         | 20979/225000 [51:47:58<350:00:36,  6.18s/it]
  9%|▉         | 20980/225000 [51:48:12<476:25:19,  8.41s/it]
  9%|▉         | 20981/225000 [51:48:18<438:58:00,  7.75s/it]
  9%|▉         | 20982/225000 [51:48:22<375:44:57,  6.63s/it]
  9%|▉         | 20983/225000 [51:48:31<413:44:32,  7.30s/it]
  9%|▉         | 20984/225000 [51:48:38<421:30:57,  7.44s/it]
  9%|▉         | 20985/225000 [51:48:42<360:04:18,  6.35s/it]
  9%|▉         | 20986/225000 [51:48:52<417:14:46,  7.36s/it]
  9%|▉         | 20987/225000 [51:48:57<373:07:27,  6.58s/it]
  9%|▉         | 20988/225000 [51:49:02<345:39:49,  6.10s/it]
  9%|▉         | 20989/225000 [51:49:06<321:51:20,  5.68s/it]
  9%|▉         | 20990/225000 [51:49:12<320:19:34,  5.65s/it]
  9%|▉         | 20991/225000 [51:49:18<318:45:40,  5.62s/it]
  9%|▉         | 20992/225000 [51:49:23<312:43:39,  5.52s/it]
  9%|▉         | 20993/225000 [51:49:29<319:56:37,  5.65s/it]
  9%|▉         | 20994/225000 [51:49:42<451:14:27,  7.96s/it]
  9%|▉         | 20995/225000 [51:49:49<437:04:46,  7.71s/it]
  9%|▉         | 20996/225000 [51:49:57<436:04:38,  7.70s/it]
  9%|▉         | 20997/225000 [51:50:01<381:51:54,  6.74s/it]
  9%|▉         | 20998/225000 [51:50:13<470:12:51,  8.30s/it]
  9%|▉         | 20999/225000 [51:50:18<400:52:13,  7.07s/it]
  9%|▉         | 21000/225000 [51:50:22<361:29:55,  6.38s/it]
                                                             
{'loss': '0.7135', 'grad_norm': '0.5967', 'learning_rate': '4.968e-05', 'epoch': '0.09333', 'num_input_tokens_seen': 90687200, 'train_runtime': '1.866e+05', 'train_tokens_per_second': '485.9'}

  9%|▉         | 21000/225000 [51:50:22<361:29:55,  6.38s/it]
  9%|▉         | 21001/225000 [51:50:31<396:44:54,  7.00s/it]
  9%|▉         | 21002/225000 [51:50:40<427:53:15,  7.55s/it]
  9%|▉         | 21003/225000 [51:50:48<436:46:02,  7.71s/it]
  9%|▉         | 21004/225000 [51:50:53<400:16:06,  7.06s/it]
  9%|▉         | 21005/225000 [51:50:59<370:37:55,  6.54s/it]
  9%|▉         | 21006/225000 [51:51:04<346:00:33,  6.11s/it]
  9%|▉         | 21007/225000 [51:51:10<355:50:42,  6.28s/it]
  9%|▉         | 21008/225000 [51:51:24<481:26:13,  8.50s/it]
  9%|▉         | 21009/225000 [51:51:28<397:52:58,  7.02s/it]
  9%|▉         | 21010/225000 [51:51:33<369:43:21,  6.52s/it]
  9%|▉         | 21011/225000 [51:51:38<348:24:57,  6.15s/it]
  9%|▉         | 21012/225000 [51:51:43<330:46:59,  5.84s/it]
  9%|▉         | 21013/225000 [51:51:55<434:30:07,  7.67s/it]
  9%|▉         | 21014/225000 [51:52:00<374:29:20,  6.61s/it]
  9%|▉         | 21015/225000 [51:52:09<424:21:24,  7.49s/it]
  9%|▉         | 21016/225000 [51:52:14<388:45:47,  6.86s/it]
  9%|▉         | 21017/225000 [51:52:20<373:08:51,  6.59s/it]
  9%|▉         | 21018/225000 [51:52:26<355:25:38,  6.27s/it]
  9%|▉         | 21019/225000 [51:52:32<349:02:56,  6.16s/it]
  9%|▉         | 21020/225000 [51:52:38<349:33:34,  6.17s/it]
  9%|▉         | 21021/225000 [51:52:44<339:27:19,  5.99s/it]
  9%|▉         | 21022/225000 [51:52:57<458:33:15,  8.09s/it]
  9%|▉         | 21023/225000 [51:53:01<402:07:46,  7.10s/it]
  9%|▉         | 21024/225000 [51:53:06<364:39:00,  6.44s/it]
  9%|▉         | 21025/225000 [51:53:11<333:44:19,  5.89s/it]
  9%|▉         | 21026/225000 [51:53:22<426:58:45,  7.54s/it]
  9%|▉         | 21027/225000 [51:53:27<385:57:51,  6.81s/it]
  9%|▉         | 21028/225000 [51:53:32<341:23:48,  6.03s/it]
  9%|▉         | 21029/225000 [51:53:44<446:27:33,  7.88s/it]
  9%|▉         | 21030/225000 [51:53:50<422:47:00,  7.46s/it]
  9%|▉         | 21031/225000 [51:53:57<412:12:00,  7.28s/it]
  9%|▉         | 21032/225000 [51:54:05<424:35:44,  7.49s/it]
  9%|▉         | 21033/225000 [51:54:10<375:42:15,  6.63s/it]
  9%|▉         | 21034/225000 [51:54:15<347:58:16,  6.14s/it]
  9%|▉         | 21035/225000 [51:54:19<323:21:53,  5.71s/it]
  9%|▉         | 21036/225000 [51:54:24<307:33:52,  5.43s/it]
  9%|▉         | 21037/225000 [51:54:37<439:45:33,  7.76s/it]
  9%|▉         | 21038/225000 [51:54:42<386:47:50,  6.83s/it]
  9%|▉         | 21039/225000 [51:54:49<382:10:58,  6.75s/it]
  9%|▉         | 21040/225000 [51:54:53<340:00:07,  6.00s/it]
  9%|▉         | 21041/225000 [51:55:03<417:45:56,  7.37s/it]
  9%|▉         | 21042/225000 [51:55:08<367:07:19,  6.48s/it]
  9%|▉         | 21043/225000 [51:55:12<329:54:24,  5.82s/it]
  9%|▉         | 21044/225000 [51:55:20<372:37:33,  6.58s/it]
  9%|▉         | 21045/225000 [51:55:27<378:06:58,  6.67s/it]
  9%|▉         | 21046/225000 [51:55:36<405:31:50,  7.16s/it]
  9%|▉         | 21047/225000 [51:55:42<390:41:24,  6.90s/it]
  9%|▉         | 21048/225000 [51:55:49<387:34:03,  6.84s/it]
  9%|▉         | 21049/225000 [51:55:52<335:43:55,  5.93s/it]
  9%|▉         | 21050/225000 [51:56:01<387:43:24,  6.84s/it]
                                                             
{'loss': '0.6959', 'grad_norm': '0.6455', 'learning_rate': '4.968e-05', 'epoch': '0.09356', 'num_input_tokens_seen': 90838944, 'train_runtime': '1.87e+05', 'train_tokens_per_second': '485.9'}

  9%|▉         | 21050/225000 [51:56:01<387:43:24,  6.84s/it]
  9%|▉         | 21051/225000 [51:56:10<412:47:19,  7.29s/it]
  9%|▉         | 21052/225000 [51:56:14<357:21:53,  6.31s/it]
  9%|▉         | 21053/225000 [51:56:23<402:47:16,  7.11s/it]
  9%|▉         | 21054/225000 [51:56:27<360:49:18,  6.37s/it]
  9%|▉         | 21055/225000 [51:56:32<331:33:16,  5.85s/it]
  9%|▉         | 21056/225000 [51:56:38<341:33:06,  6.03s/it]
  9%|▉         | 21057/225000 [51:56:50<431:51:06,  7.62s/it]
  9%|▉         | 21058/225000 [51:56:58<437:37:13,  7.72s/it]
  9%|▉         | 21059/225000 [51:57:02<382:42:46,  6.76s/it]
  9%|▉         | 21060/225000 [51:57:09<376:34:05,  6.65s/it]
  9%|▉         | 21061/225000 [51:57:16<383:37:47,  6.77s/it]
  9%|▉         | 21062/225000 [51:57:20<349:24:45,  6.17s/it]
  9%|▉         | 21063/225000 [51:57:28<378:31:27,  6.68s/it]
  9%|▉         | 21064/225000 [51:57:37<418:53:02,  7.39s/it]
  9%|▉         | 21065/225000 [51:57:42<375:24:09,  6.63s/it]
  9%|▉         | 21066/225000 [51:57:50<390:07:52,  6.89s/it]
  9%|▉         | 21067/225000 [51:57:57<400:06:12,  7.06s/it]
  9%|▉         | 21068/225000 [51:58:02<355:04:20,  6.27s/it]
  9%|▉         | 21069/225000 [51:58:12<425:57:38,  7.52s/it]
  9%|▉         | 21070/225000 [51:58:16<362:38:31,  6.40s/it]
  9%|▉         | 21071/225000 [51:58:23<378:51:58,  6.69s/it]
  9%|▉         | 21072/225000 [51:58:31<397:07:14,  7.01s/it]
  9%|▉         | 21073/225000 [51:58:38<398:39:45,  7.04s/it]
  9%|▉         | 21074/225000 [51:58:42<353:02:51,  6.23s/it]
  9%|▉         | 21075/225000 [51:58:48<338:14:31,  5.97s/it]
  9%|▉         | 21076/225000 [51:58:53<322:34:07,  5.69s/it]
  9%|▉         | 21077/225000 [51:58:59<334:02:27,  5.90s/it]
  9%|▉         | 21078/225000 [51:59:13<467:59:46,  8.26s/it]
  9%|▉         | 21079/225000 [51:59:20<448:45:14,  7.92s/it]
  9%|▉         | 21080/225000 [51:59:25<398:50:02,  7.04s/it]
  9%|▉         | 21081/225000 [51:59:31<379:50:57,  6.71s/it]
  9%|▉         | 21082/225000 [51:59:37<365:09:59,  6.45s/it]
  9%|▉         | 21083/225000 [51:59:43<353:06:48,  6.23s/it]
  9%|▉         | 21084/225000 [51:59:49<349:57:46,  6.18s/it]
  9%|▉         | 21085/225000 [52:00:00<436:11:36,  7.70s/it]
  9%|▉         | 21086/225000 [52:00:08<440:48:16,  7.78s/it]
  9%|▉         | 21087/225000 [52:00:13<388:47:11,  6.86s/it]
  9%|▉         | 21088/225000 [52:00:18<356:13:50,  6.29s/it]
  9%|▉         | 21089/225000 [52:00:22<330:48:51,  5.84s/it]
  9%|▉         | 21090/225000 [52:00:35<451:52:56,  7.98s/it]
  9%|▉         | 21091/225000 [52:00:40<389:06:22,  6.87s/it]
  9%|▉         | 21092/225000 [52:00:44<342:40:09,  6.05s/it]
  9%|▉         | 21093/225000 [52:00:48<319:49:35,  5.65s/it]
  9%|▉         | 21094/225000 [52:00:53<307:00:45,  5.42s/it]
  9%|▉         | 21095/225000 [52:01:04<397:38:04,  7.02s/it]
  9%|▉         | 21096/225000 [52:01:12<419:43:57,  7.41s/it]
  9%|▉         | 21097/225000 [52:01:19<411:23:26,  7.26s/it]
  9%|▉         | 21098/225000 [52:01:26<400:12:26,  7.07s/it]
  9%|▉         | 21099/225000 [52:01:31<360:05:44,  6.36s/it]
  9%|▉         | 21100/225000 [52:01:38<380:58:36,  6.73s/it]
                                                             
{'loss': '0.7002', 'grad_norm': '0.8623', 'learning_rate': '4.967e-05', 'epoch': '0.09378', 'num_input_tokens_seen': 90991808, 'train_runtime': '1.873e+05', 'train_tokens_per_second': '485.8'}

  9%|▉         | 21100/225000 [52:01:38<380:58:36,  6.73s/it]
  9%|▉         | 21101/225000 [52:01:47<407:20:44,  7.19s/it]
  9%|▉         | 21102/225000 [52:01:54<405:06:27,  7.15s/it]
  9%|▉         | 21103/225000 [52:01:59<379:47:02,  6.71s/it]
  9%|▉         | 21104/225000 [52:02:04<351:22:07,  6.20s/it]
  9%|▉         | 21105/225000 [52:02:10<335:02:38,  5.92s/it]
  9%|▉         | 21106/225000 [52:02:14<315:55:38,  5.58s/it]
  9%|▉         | 21107/225000 [52:02:23<377:16:43,  6.66s/it]
  9%|▉         | 21108/225000 [52:02:33<428:50:49,  7.57s/it]
  9%|▉         | 21109/225000 [52:02:40<410:29:36,  7.25s/it]
  9%|▉         | 21110/225000 [52:02:45<372:02:58,  6.57s/it]
  9%|▉         | 21111/225000 [52:02:55<431:47:36,  7.62s/it]
  9%|▉         | 21112/225000 [52:03:01<402:16:20,  7.10s/it]
  9%|▉         | 21113/225000 [52:03:08<414:43:26,  7.32s/it]
  9%|▉         | 21114/225000 [52:03:13<374:34:42,  6.61s/it]
  9%|▉         | 21115/225000 [52:03:19<349:15:30,  6.17s/it]
  9%|▉         | 21116/225000 [52:03:24<335:37:42,  5.93s/it]
  9%|▉         | 21117/225000 [52:03:29<315:21:26,  5.57s/it]
  9%|▉         | 21118/225000 [52:03:42<444:22:10,  7.85s/it]
  9%|▉         | 21119/225000 [52:03:47<390:53:21,  6.90s/it]
  9%|▉         | 21120/225000 [52:03:55<410:57:08,  7.26s/it]
  9%|▉         | 21121/225000 [52:04:00<380:15:11,  6.71s/it]
  9%|▉         | 21122/225000 [52:04:05<356:33:21,  6.30s/it]
  9%|▉         | 21123/225000 [52:04:11<342:08:42,  6.04s/it]
  9%|▉         | 21124/225000 [52:04:16<324:41:49,  5.73s/it]
  9%|▉         | 21125/225000 [52:04:25<386:44:25,  6.83s/it]
  9%|▉         | 21126/225000 [52:04:36<449:29:32,  7.94s/it]
  9%|▉         | 21127/225000 [52:04:41<398:12:37,  7.03s/it]
  9%|▉         | 21128/225000 [52:04:46<367:03:30,  6.48s/it]
  9%|▉         | 21129/225000 [52:04:54<395:19:44,  6.98s/it]
  9%|▉         | 21130/225000 [52:05:04<451:56:46,  7.98s/it]
  9%|▉         | 21131/225000 [52:05:09<396:42:33,  7.01s/it]
  9%|▉         | 21132/225000 [52:05:14<358:56:58,  6.34s/it]
  9%|▉         | 21133/225000 [52:05:19<338:10:14,  5.97s/it]
  9%|▉         | 21134/225000 [52:05:30<427:16:46,  7.55s/it]
  9%|▉         | 21135/225000 [52:05:34<372:06:53,  6.57s/it]
  9%|▉         | 21136/225000 [52:05:39<329:19:01,  5.82s/it]
  9%|▉         | 21137/225000 [52:05:50<429:09:16,  7.58s/it]
  9%|▉         | 21138/225000 [52:05:55<378:55:40,  6.69s/it]
  9%|▉         | 21139/225000 [52:05:59<344:23:37,  6.08s/it]
  9%|▉         | 21140/225000 [52:06:11<432:19:10,  7.63s/it]
  9%|▉         | 21141/225000 [52:06:15<377:07:07,  6.66s/it]
  9%|▉         | 21142/225000 [52:06:26<441:07:28,  7.79s/it]
  9%|▉         | 21143/225000 [52:06:29<374:20:11,  6.61s/it]
  9%|▉         | 21144/225000 [52:06:35<354:28:57,  6.26s/it]
  9%|▉         | 21145/225000 [52:06:40<335:53:39,  5.93s/it]
  9%|▉         | 21146/225000 [52:06:46<330:57:36,  5.84s/it]
  9%|▉         | 21147/225000 [52:06:54<365:39:26,  6.46s/it]
  9%|▉         | 21148/225000 [52:07:05<451:49:31,  7.98s/it]
  9%|▉         | 21149/225000 [52:07:11<416:43:53,  7.36s/it]
  9%|▉         | 21150/225000 [52:07:17<401:06:51,  7.08s/it]
                                                             
{'loss': '0.7083', 'grad_norm': '0.6681', 'learning_rate': '4.967e-05', 'epoch': '0.094', 'num_input_tokens_seen': 91143800, 'train_runtime': '1.876e+05', 'train_tokens_per_second': '485.7'}

  9%|▉         | 21150/225000 [52:07:17<401:06:51,  7.08s/it]
  9%|▉         | 21151/225000 [52:07:22<357:12:24,  6.31s/it]
  9%|▉         | 21152/225000 [52:07:33<441:23:30,  7.80s/it]
  9%|▉         | 21153/225000 [52:07:37<371:02:24,  6.55s/it]
  9%|▉         | 21154/225000 [52:07:42<344:08:43,  6.08s/it]
  9%|▉         | 21155/225000 [52:07:47<329:34:09,  5.82s/it]
  9%|▉         | 21156/225000 [52:07:59<436:43:04,  7.71s/it]
  9%|▉         | 21157/225000 [52:08:03<371:13:42,  6.56s/it]
  9%|▉         | 21158/225000 [52:08:11<401:46:29,  7.10s/it]
  9%|▉         | 21159/225000 [52:08:18<401:58:27,  7.10s/it]
  9%|▉         | 21160/225000 [52:08:24<372:45:59,  6.58s/it]
  9%|▉         | 21161/225000 [52:08:30<358:32:35,  6.33s/it]
  9%|▉         | 21162/225000 [52:08:36<351:37:14,  6.21s/it]
  9%|▉         | 21163/225000 [52:08:43<369:33:59,  6.53s/it]
  9%|▉         | 21164/225000 [52:08:48<351:02:38,  6.20s/it]
  9%|▉         | 21165/225000 [52:08:54<344:53:01,  6.09s/it]
  9%|▉         | 21166/225000 [52:09:00<339:21:34,  5.99s/it]
  9%|▉         | 21167/225000 [52:09:12<442:06:21,  7.81s/it]
  9%|▉         | 21168/225000 [52:09:17<400:56:48,  7.08s/it]
  9%|▉         | 21169/225000 [52:09:24<388:48:31,  6.87s/it]
  9%|▉         | 21170/225000 [52:09:29<367:32:16,  6.49s/it]
  9%|▉         | 21171/225000 [52:09:35<350:43:55,  6.19s/it]
  9%|▉         | 21172/225000 [52:09:45<419:56:09,  7.42s/it]
  9%|▉         | 21173/225000 [52:09:51<392:43:14,  6.94s/it]
  9%|▉         | 21174/225000 [52:09:56<370:36:33,  6.55s/it]
  9%|▉         | 21175/225000 [52:10:02<358:46:02,  6.34s/it]
  9%|▉         | 21176/225000 [52:10:08<351:28:19,  6.21s/it]
  9%|▉         | 21177/225000 [52:10:17<390:51:02,  6.90s/it]
  9%|▉         | 21178/225000 [52:10:29<486:38:02,  8.60s/it]
  9%|▉         | 21179/225000 [52:10:35<431:11:09,  7.62s/it]
  9%|▉         | 21180/225000 [52:10:41<402:52:17,  7.12s/it]
  9%|▉         | 21181/225000 [52:10:46<380:22:24,  6.72s/it]
  9%|▉         | 21182/225000 [52:10:52<359:08:46,  6.34s/it]
  9%|▉         | 21183/225000 [52:10:57<344:47:59,  6.09s/it]
  9%|▉         | 21184/225000 [52:11:03<333:17:12,  5.89s/it]
  9%|▉         | 21185/225000 [52:11:09<335:49:06,  5.93s/it]
  9%|▉         | 21186/225000 [52:11:18<384:08:16,  6.79s/it]
  9%|▉         | 21187/225000 [52:11:29<459:50:18,  8.12s/it]
  9%|▉         | 21188/225000 [52:11:36<437:51:01,  7.73s/it]
  9%|▉         | 21189/225000 [52:11:45<466:30:37,  8.24s/it]
  9%|▉         | 21190/225000 [52:11:50<409:31:11,  7.23s/it]
  9%|▉         | 21191/225000 [52:11:55<372:24:19,  6.58s/it]
  9%|▉         | 21192/225000 [52:12:00<343:18:08,  6.06s/it]
  9%|▉         | 21193/225000 [52:12:12<444:53:07,  7.86s/it]
  9%|▉         | 21194/225000 [52:12:17<391:51:36,  6.92s/it]
  9%|▉         | 21195/225000 [52:12:25<424:00:57,  7.49s/it]
  9%|▉         | 21196/225000 [52:12:30<378:50:29,  6.69s/it]
  9%|▉         | 21197/225000 [52:12:36<355:00:20,  6.27s/it]
  9%|▉         | 21198/225000 [52:12:40<330:30:59,  5.84s/it]
  9%|▉         | 21199/225000 [52:12:53<452:56:34,  8.00s/it]
  9%|▉         | 21200/225000 [52:12:59<409:19:32,  7.23s/it]
                                                             
{'loss': '0.7066', 'grad_norm': '0.7303', 'learning_rate': '4.967e-05', 'epoch': '0.09422', 'num_input_tokens_seen': 91298648, 'train_runtime': '1.88e+05', 'train_tokens_per_second': '485.7'}

  9%|▉         | 21200/225000 [52:12:59<409:19:32,  7.23s/it]
  9%|▉         | 21201/225000 [52:13:07<417:52:56,  7.38s/it]
  9%|▉         | 21202/225000 [52:13:11<361:53:51,  6.39s/it]
  9%|▉         | 21203/225000 [52:13:16<341:06:20,  6.03s/it]
  9%|▉         | 21204/225000 [52:13:25<395:32:59,  6.99s/it]
  9%|▉         | 21205/225000 [52:13:32<394:11:14,  6.96s/it]
  9%|▉         | 21206/225000 [52:13:43<460:59:14,  8.14s/it]
  9%|▉         | 21207/225000 [52:13:47<396:46:21,  7.01s/it]
  9%|▉         | 21208/225000 [52:13:53<373:49:09,  6.60s/it]
  9%|▉         | 21209/225000 [52:13:58<355:24:05,  6.28s/it]
  9%|▉         | 21210/225000 [52:14:05<362:33:48,  6.40s/it]
  9%|▉         | 21211/225000 [52:14:15<415:03:21,  7.33s/it]
  9%|▉         | 21212/225000 [52:14:18<355:26:42,  6.28s/it]
  9%|▉         | 21213/225000 [52:14:30<442:46:54,  7.82s/it]
  9%|▉         | 21214/225000 [52:14:35<396:16:46,  7.00s/it]
  9%|▉         | 21215/225000 [52:14:44<437:58:42,  7.74s/it]
  9%|▉         | 21216/225000 [52:14:49<381:56:58,  6.75s/it]
  9%|▉         | 21217/225000 [52:14:54<348:40:34,  6.16s/it]
  9%|▉         | 21218/225000 [52:15:04<417:56:18,  7.38s/it]
  9%|▉         | 21219/225000 [52:15:09<387:30:18,  6.85s/it]
  9%|▉         | 21220/225000 [52:15:14<353:03:32,  6.24s/it]
  9%|▉         | 21221/225000 [52:15:19<324:23:32,  5.73s/it]
  9%|▉         | 21222/225000 [52:15:29<396:37:37,  7.01s/it]
  9%|▉         | 21223/225000 [52:15:35<386:51:23,  6.83s/it]
  9%|▉         | 21224/225000 [52:15:40<346:52:15,  6.13s/it]
  9%|▉         | 21225/225000 [52:15:50<421:43:20,  7.45s/it]
  9%|▉         | 21226/225000 [52:15:57<405:47:22,  7.17s/it]
  9%|▉         | 21227/225000 [52:16:02<365:24:50,  6.46s/it]
  9%|▉         | 21228/225000 [52:16:06<334:05:31,  5.90s/it]
  9%|▉         | 21229/225000 [52:16:15<388:48:27,  6.87s/it]
  9%|▉         | 21230/225000 [52:16:23<395:28:25,  6.99s/it]
  9%|▉         | 21231/225000 [52:16:31<414:21:22,  7.32s/it]
  9%|▉         | 21232/225000 [52:16:35<365:27:08,  6.46s/it]
  9%|▉         | 21233/225000 [52:16:40<343:47:52,  6.07s/it]
  9%|▉         | 21234/225000 [52:16:51<421:27:05,  7.45s/it]
  9%|▉         | 21235/225000 [52:16:56<376:57:25,  6.66s/it]
  9%|▉         | 21236/225000 [52:17:02<376:35:19,  6.65s/it]
  9%|▉         | 21237/225000 [52:17:07<340:23:20,  6.01s/it]
  9%|▉         | 21238/225000 [52:17:12<316:38:52,  5.59s/it]
  9%|▉         | 21239/225000 [52:17:21<380:20:58,  6.72s/it]
  9%|▉         | 21240/225000 [52:17:31<438:50:57,  7.75s/it]
  9%|▉         | 21241/225000 [52:17:38<427:36:31,  7.55s/it]
  9%|▉         | 21242/225000 [52:17:43<389:26:49,  6.88s/it]
  9%|▉         | 21243/225000 [52:17:54<447:36:55,  7.91s/it]
  9%|▉         | 21244/225000 [52:18:01<431:52:11,  7.63s/it]
  9%|▉         | 21245/225000 [52:18:05<372:33:23,  6.58s/it]
  9%|▉         | 21246/225000 [52:18:09<336:19:29,  5.94s/it]
  9%|▉         | 21247/225000 [52:18:21<425:27:45,  7.52s/it]
  9%|▉         | 21248/225000 [52:18:27<412:16:19,  7.28s/it]
  9%|▉         | 21249/225000 [52:18:33<390:59:41,  6.91s/it]
  9%|▉         | 21250/225000 [52:18:40<380:22:14,  6.72s/it]
                                                             
{'loss': '0.7089', 'grad_norm': '0.6539', 'learning_rate': '4.966e-05', 'epoch': '0.09444', 'num_input_tokens_seen': 91452968, 'train_runtime': '1.883e+05', 'train_tokens_per_second': '485.6'}

  9%|▉         | 21250/225000 [52:18:40<380:22:14,  6.72s/it]
  9%|▉         | 21251/225000 [52:18:45<353:40:21,  6.25s/it]
  9%|▉         | 21252/225000 [52:18:50<331:39:18,  5.86s/it]
  9%|▉         | 21253/225000 [52:18:55<330:02:10,  5.83s/it]
  9%|▉         | 21254/225000 [52:19:01<320:19:47,  5.66s/it]
  9%|▉         | 21255/225000 [52:19:06<308:05:26,  5.44s/it]
  9%|▉         | 21256/225000 [52:19:18<418:55:20,  7.40s/it]
  9%|▉         | 21257/225000 [52:19:26<429:50:02,  7.59s/it]
  9%|▉         | 21258/225000 [52:19:33<431:30:14,  7.62s/it]
  9%|▉         | 21259/225000 [52:19:42<457:01:55,  8.08s/it]
  9%|▉         | 21260/225000 [52:19:46<386:19:49,  6.83s/it]
  9%|▉         | 21261/225000 [52:19:55<417:08:43,  7.37s/it]
  9%|▉         | 21262/225000 [52:20:01<390:10:13,  6.89s/it]
  9%|▉         | 21263/225000 [52:20:07<371:01:32,  6.56s/it]
  9%|▉         | 21264/225000 [52:20:15<399:54:39,  7.07s/it]
  9%|▉         | 21265/225000 [52:20:22<407:27:40,  7.20s/it]
  9%|▉         | 21266/225000 [52:20:29<393:00:13,  6.94s/it]
  9%|▉         | 21267/225000 [52:20:34<360:30:17,  6.37s/it]
  9%|▉         | 21268/225000 [52:20:42<398:43:24,  7.05s/it]
  9%|▉         | 21269/225000 [52:20:46<344:00:32,  6.08s/it]
  9%|▉         | 21270/225000 [52:20:53<362:39:51,  6.41s/it]
  9%|▉         | 21271/225000 [52:20:58<330:23:38,  5.84s/it]
  9%|▉         | 21272/225000 [52:21:03<314:40:31,  5.56s/it]
  9%|▉         | 21273/225000 [52:21:16<438:00:01,  7.74s/it]
  9%|▉         | 21274/225000 [52:21:21<401:56:37,  7.10s/it]
  9%|▉         | 21275/225000 [52:21:29<419:09:37,  7.41s/it]
  9%|▉         | 21276/225000 [52:21:34<365:45:25,  6.46s/it]
  9%|▉         | 21277/225000 [52:21:42<405:48:39,  7.17s/it]
  9%|▉         | 21278/225000 [52:21:47<355:14:36,  6.28s/it]
  9%|▉         | 21279/225000 [52:21:57<424:51:57,  7.51s/it]
  9%|▉         | 21280/225000 [52:22:01<361:50:24,  6.39s/it]
  9%|▉         | 21281/225000 [52:22:10<409:52:54,  7.24s/it]
  9%|▉         | 21282/225000 [52:22:17<400:37:59,  7.08s/it]
  9%|▉         | 21283/225000 [52:22:22<372:31:49,  6.58s/it]
  9%|▉         | 21284/225000 [52:22:26<325:30:38,  5.75s/it]
  9%|▉         | 21285/225000 [52:22:36<403:29:16,  7.13s/it]
  9%|▉         | 21286/225000 [52:22:41<370:31:30,  6.55s/it]
  9%|▉         | 21287/225000 [52:22:47<353:52:36,  6.25s/it]
  9%|▉         | 21288/225000 [52:22:58<429:02:29,  7.58s/it]
  9%|▉         | 21289/225000 [52:23:02<371:33:19,  6.57s/it]
  9%|▉         | 21290/225000 [52:23:07<338:02:26,  5.97s/it]
  9%|▉         | 21291/225000 [52:23:15<385:37:04,  6.81s/it]
  9%|▉         | 21292/225000 [52:23:22<383:23:54,  6.78s/it]
  9%|▉         | 21293/225000 [52:23:30<407:33:50,  7.20s/it]
  9%|▉         | 21294/225000 [52:23:35<363:00:18,  6.42s/it]
  9%|▉         | 21295/225000 [52:23:42<377:13:38,  6.67s/it]
  9%|▉         | 21296/225000 [52:23:50<394:20:59,  6.97s/it]
  9%|▉         | 21297/225000 [52:23:54<345:43:06,  6.11s/it]
  9%|▉         | 21298/225000 [52:24:03<403:14:24,  7.13s/it]
  9%|▉         | 21299/225000 [52:24:08<355:19:47,  6.28s/it]
  9%|▉         | 21300/225000 [52:24:12<328:55:47,  5.81s/it]
                                                             
{'loss': '0.717', 'grad_norm': '0.6288', 'learning_rate': '4.966e-05', 'epoch': '0.09467', 'num_input_tokens_seen': 91604384, 'train_runtime': '1.887e+05', 'train_tokens_per_second': '485.6'}

  9%|▉         | 21300/225000 [52:24:12<328:55:47,  5.81s/it]
  9%|▉         | 21301/225000 [52:24:19<350:42:12,  6.20s/it]
  9%|▉         | 21302/225000 [52:24:32<453:49:11,  8.02s/it]
  9%|▉         | 21303/225000 [52:24:35<379:37:27,  6.71s/it]
  9%|▉         | 21304/225000 [52:24:44<414:43:47,  7.33s/it]
  9%|▉         | 21305/225000 [52:24:49<367:04:38,  6.49s/it]
  9%|▉         | 21306/225000 [52:25:00<443:34:55,  7.84s/it]
  9%|▉         | 21307/225000 [52:25:04<378:10:43,  6.68s/it]
  9%|▉         | 21308/225000 [52:25:13<417:43:45,  7.38s/it]
  9%|▉         | 21309/225000 [52:25:17<365:57:51,  6.47s/it]
  9%|▉         | 21310/225000 [52:25:21<332:01:46,  5.87s/it]
  9%|▉         | 21311/225000 [52:25:32<416:36:35,  7.36s/it]
  9%|▉         | 21312/225000 [52:25:37<378:39:06,  6.69s/it]
  9%|▉         | 21313/225000 [52:25:42<350:43:30,  6.20s/it]
  9%|▉         | 21314/225000 [52:25:47<325:55:50,  5.76s/it]
  9%|▉         | 21315/225000 [52:25:51<300:57:27,  5.32s/it]
  9%|▉         | 21316/225000 [52:26:05<438:41:31,  7.75s/it]
  9%|▉         | 21317/225000 [52:26:11<409:16:01,  7.23s/it]
  9%|▉         | 21318/225000 [52:26:22<477:04:47,  8.43s/it]
  9%|▉         | 21319/225000 [52:26:26<407:07:32,  7.20s/it]
  9%|▉         | 21320/225000 [52:26:31<360:03:26,  6.36s/it]
  9%|▉         | 21321/225000 [52:26:36<339:22:05,  6.00s/it]
  9%|▉         | 21322/225000 [52:26:48<439:23:10,  7.77s/it]
  9%|▉         | 21323/225000 [52:26:52<373:49:10,  6.61s/it]
  9%|▉         | 21324/225000 [52:26:57<348:18:37,  6.16s/it]
  9%|▉         | 21325/225000 [52:27:03<341:02:15,  6.03s/it]
  9%|▉         | 21326/225000 [52:27:08<336:59:01,  5.96s/it]
  9%|▉         | 21327/225000 [52:27:21<445:57:57,  7.88s/it]
  9%|▉         | 21328/225000 [52:27:27<410:22:30,  7.25s/it]
  9%|▉         | 21329/225000 [52:27:32<370:40:46,  6.55s/it]
  9%|▉         | 21330/225000 [52:27:37<347:09:47,  6.14s/it]
  9%|▉         | 21331/225000 [52:27:48<435:32:37,  7.70s/it]
  9%|▉         | 21332/225000 [52:27:53<385:49:53,  6.82s/it]
  9%|▉         | 21333/225000 [52:28:00<383:44:06,  6.78s/it]
  9%|▉         | 21334/225000 [52:28:08<408:28:08,  7.22s/it]
  9%|▉         | 21335/225000 [52:28:13<376:17:45,  6.65s/it]
  9%|▉         | 21336/225000 [52:28:21<390:52:51,  6.91s/it]
  9%|▉         | 21337/225000 [52:28:28<402:45:49,  7.12s/it]
  9%|▉         | 21338/225000 [52:28:36<416:04:31,  7.35s/it]
  9%|▉         | 21339/225000 [52:28:42<397:14:58,  7.02s/it]
  9%|▉         | 21340/225000 [52:28:48<376:56:14,  6.66s/it]
  9%|▉         | 21341/225000 [52:28:54<359:34:43,  6.36s/it]
  9%|▉         | 21342/225000 [52:29:01<371:48:35,  6.57s/it]
  9%|▉         | 21343/225000 [52:29:06<343:30:36,  6.07s/it]
  9%|▉         | 21344/225000 [52:29:11<322:29:18,  5.70s/it]
  9%|▉         | 21345/225000 [52:29:18<355:16:40,  6.28s/it]
  9%|▉         | 21346/225000 [52:29:28<406:54:16,  7.19s/it]
  9%|▉         | 21347/225000 [52:29:34<386:18:53,  6.83s/it]
  9%|▉         | 21348/225000 [52:29:42<409:20:48,  7.24s/it]
  9%|▉         | 21349/225000 [52:29:47<369:58:42,  6.54s/it]
  9%|▉         | 21350/225000 [52:29:52<344:45:48,  6.09s/it]
                                                             
{'loss': '0.7071', 'grad_norm': '0.5978', 'learning_rate': '4.966e-05', 'epoch': '0.09489', 'num_input_tokens_seen': 91757440, 'train_runtime': '1.89e+05', 'train_tokens_per_second': '485.5'}

  9%|▉         | 21350/225000 [52:29:52<344:45:48,  6.09s/it]
  9%|▉         | 21351/225000 [52:29:57<331:01:48,  5.85s/it]
  9%|▉         | 21352/225000 [52:30:06<385:47:11,  6.82s/it]
  9%|▉         | 21353/225000 [52:30:17<449:36:26,  7.95s/it]
  9%|▉         | 21354/225000 [52:30:20<377:53:18,  6.68s/it]
  9%|▉         | 21355/225000 [52:30:26<365:38:31,  6.46s/it]
  9%|▉         | 21356/225000 [52:30:38<460:07:46,  8.13s/it]
  9%|▉         | 21357/225000 [52:30:43<402:50:52,  7.12s/it]
  9%|▉         | 21358/225000 [52:30:48<359:00:22,  6.35s/it]
  9%|▉         | 21359/225000 [52:30:59<448:16:09,  7.92s/it]
  9%|▉         | 21360/225000 [52:31:04<399:08:13,  7.06s/it]
  9%|▉         | 21361/225000 [52:31:10<370:48:08,  6.56s/it]
  9%|▉         | 21362/225000 [52:31:15<341:52:07,  6.04s/it]
  9%|▉         | 21363/225000 [52:31:20<327:10:29,  5.78s/it]
  9%|▉         | 21364/225000 [52:31:32<438:50:30,  7.76s/it]
  9%|▉         | 21365/225000 [52:31:36<374:39:12,  6.62s/it]
  9%|▉         | 21366/225000 [52:31:42<357:22:19,  6.32s/it]
  9%|▉         | 21367/225000 [52:31:47<340:07:40,  6.01s/it]
  9%|▉         | 21368/225000 [52:32:00<465:47:39,  8.23s/it]
  9%|▉         | 21369/225000 [52:32:05<409:55:37,  7.25s/it]
  9%|▉         | 21370/225000 [52:32:14<429:56:23,  7.60s/it]
  9%|▉         | 21371/225000 [52:32:21<416:18:00,  7.36s/it]
  9%|▉         | 21372/225000 [52:32:24<352:25:14,  6.23s/it]
  9%|▉         | 21373/225000 [52:32:34<416:56:03,  7.37s/it]
  9%|▉         | 21374/225000 [52:32:38<352:19:08,  6.23s/it]
 10%|▉         | 21375/225000 [52:32:47<396:59:11,  7.02s/it]
 10%|▉         | 21376/225000 [52:32:53<393:01:51,  6.95s/it]
 10%|▉         | 21377/225000 [52:32:59<363:07:29,  6.42s/it]
 10%|▉         | 21378/225000 [52:33:09<439:19:51,  7.77s/it]
 10%|▉         | 21379/225000 [52:33:13<371:02:49,  6.56s/it]
 10%|▉         | 21380/225000 [52:33:19<358:50:32,  6.34s/it]
 10%|▉         | 21381/225000 [52:33:25<345:27:37,  6.11s/it]
 10%|▉         | 21382/225000 [52:33:31<342:00:28,  6.05s/it]
 10%|▉         | 21383/225000 [52:33:42<435:40:41,  7.70s/it]
 10%|▉         | 21384/225000 [52:33:49<428:04:01,  7.57s/it]
 10%|▉         | 21385/225000 [52:33:54<377:03:03,  6.67s/it]
 10%|▉         | 21386/225000 [52:34:02<408:33:37,  7.22s/it]
 10%|▉         | 21387/225000 [52:34:10<414:58:50,  7.34s/it]
 10%|▉         | 21388/225000 [52:34:15<379:28:32,  6.71s/it]
 10%|▉         | 21389/225000 [52:34:20<346:02:13,  6.12s/it]
 10%|▉         | 21390/225000 [52:34:25<331:05:41,  5.85s/it]
 10%|▉         | 21391/225000 [52:34:30<316:05:19,  5.59s/it]
 10%|▉         | 21392/225000 [52:34:41<401:53:04,  7.11s/it]
 10%|▉         | 21393/225000 [52:34:52<470:42:50,  8.32s/it]
 10%|▉         | 21394/225000 [52:34:56<390:43:06,  6.91s/it]
 10%|▉         | 21395/225000 [52:35:05<425:14:16,  7.52s/it]
 10%|▉         | 21396/225000 [52:35:08<360:27:35,  6.37s/it]
 10%|▉         | 21397/225000 [52:35:19<426:46:06,  7.55s/it]
 10%|▉         | 21398/225000 [52:35:25<399:57:48,  7.07s/it]
 10%|▉         | 21399/225000 [52:35:31<395:51:21,  7.00s/it]
 10%|▉         | 21400/225000 [52:35:35<346:42:05,  6.13s/it]
                                                             
{'loss': '0.7287', 'grad_norm': '0.6413', 'learning_rate': '4.965e-05', 'epoch': '0.09511', 'num_input_tokens_seen': 91909216, 'train_runtime': '1.893e+05', 'train_tokens_per_second': '485.4'}

 10%|▉         | 21400/225000 [52:35:35<346:42:05,  6.13s/it]
 10%|▉         | 21401/225000 [52:35:40<318:33:52,  5.63s/it]
 10%|▉         | 21402/225000 [52:35:51<408:45:04,  7.23s/it]
 10%|▉         | 21403/225000 [52:35:57<383:07:40,  6.77s/it]
 10%|▉         | 21404/225000 [52:36:02<353:41:31,  6.25s/it]
 10%|▉         | 21405/225000 [52:36:07<338:21:28,  5.98s/it]
 10%|▉         | 21406/225000 [52:36:12<327:44:52,  5.80s/it]
 10%|▉         | 21407/225000 [52:36:18<333:00:59,  5.89s/it]
 10%|▉         | 21408/225000 [52:36:25<337:25:52,  5.97s/it]
 10%|▉         | 21409/225000 [52:36:37<451:19:22,  7.98s/it]
 10%|▉         | 21410/225000 [52:36:44<433:05:48,  7.66s/it]
 10%|▉         | 21411/225000 [52:36:52<438:37:02,  7.76s/it]
 10%|▉         | 21412/225000 [52:36:56<371:45:49,  6.57s/it]
 10%|▉         | 21413/225000 [52:37:07<440:12:53,  7.78s/it]
 10%|▉         | 21414/225000 [52:37:11<383:04:34,  6.77s/it]
 10%|▉         | 21415/225000 [52:37:20<413:57:30,  7.32s/it]
 10%|▉         | 21416/225000 [52:37:27<412:50:37,  7.30s/it]
 10%|▉         | 21417/225000 [52:37:34<404:32:24,  7.15s/it]
 10%|▉         | 21418/225000 [52:37:38<362:49:55,  6.42s/it]
 10%|▉         | 21419/225000 [52:37:46<377:12:16,  6.67s/it]
 10%|▉         | 21420/225000 [52:37:57<457:59:31,  8.10s/it]
 10%|▉         | 21421/225000 [52:38:03<412:56:17,  7.30s/it]
 10%|▉         | 21422/225000 [52:38:08<382:31:21,  6.76s/it]
 10%|▉         | 21423/225000 [52:38:14<362:32:59,  6.41s/it]
 10%|▉         | 21424/225000 [52:38:19<344:19:37,  6.09s/it]
 10%|▉         | 21425/225000 [52:38:31<453:26:01,  8.02s/it]
 10%|▉         | 21426/225000 [52:38:37<415:53:45,  7.35s/it]
 10%|▉         | 21427/225000 [52:38:47<448:53:13,  7.94s/it]
 10%|▉         | 21428/225000 [52:38:53<426:13:01,  7.54s/it]
 10%|▉         | 21429/225000 [52:39:00<413:42:10,  7.32s/it]
 10%|▉         | 21430/225000 [52:39:05<377:54:17,  6.68s/it]
 10%|▉         | 21431/225000 [52:39:09<334:39:45,  5.92s/it]
 10%|▉         | 21432/225000 [52:39:14<318:50:16,  5.64s/it]
 10%|▉         | 21433/225000 [52:39:26<425:28:21,  7.52s/it]
 10%|▉         | 21434/225000 [52:39:31<375:34:56,  6.64s/it]
 10%|▉         | 21435/225000 [52:39:36<348:21:40,  6.16s/it]
 10%|▉         | 21436/225000 [52:39:47<436:15:01,  7.72s/it]
 10%|▉         | 21437/225000 [52:39:56<461:14:13,  8.16s/it]
 10%|▉         | 21438/225000 [52:40:00<389:04:15,  6.88s/it]
 10%|▉         | 21439/225000 [52:40:11<449:13:46,  7.94s/it]
 10%|▉         | 21440/225000 [52:40:15<382:43:49,  6.77s/it]
 10%|▉         | 21441/225000 [52:40:18<331:10:21,  5.86s/it]
 10%|▉         | 21442/225000 [52:40:29<406:28:09,  7.19s/it]
 10%|▉         | 21443/225000 [52:40:33<350:58:19,  6.21s/it]
 10%|▉         | 21444/225000 [52:40:38<339:03:06,  6.00s/it]
 10%|▉         | 21445/225000 [52:40:50<433:37:40,  7.67s/it]
 10%|▉         | 21446/225000 [52:40:54<372:08:25,  6.58s/it]
 10%|▉         | 21447/225000 [52:41:03<412:31:38,  7.30s/it]
 10%|▉         | 21448/225000 [52:41:10<416:49:34,  7.37s/it]
 10%|▉         | 21449/225000 [52:41:15<366:10:18,  6.48s/it]
 10%|▉         | 21450/225000 [52:41:22<372:58:19,  6.60s/it]
                                                             
{'loss': '0.7239', 'grad_norm': '0.637', 'learning_rate': '4.965e-05', 'epoch': '0.09533', 'num_input_tokens_seen': 92062576, 'train_runtime': '1.897e+05', 'train_tokens_per_second': '485.4'}

 10%|▉         | 21450/225000 [52:41:22<372:58:19,  6.60s/it]
 10%|▉         | 21451/225000 [52:41:28<367:56:34,  6.51s/it]
 10%|▉         | 21452/225000 [52:41:34<359:23:18,  6.36s/it]
 10%|▉         | 21453/225000 [52:41:41<365:23:42,  6.46s/it]
 10%|▉         | 21454/225000 [52:41:46<350:31:06,  6.20s/it]
 10%|▉         | 21455/225000 [52:41:58<444:09:48,  7.86s/it]
 10%|▉         | 21456/225000 [52:42:03<389:34:16,  6.89s/it]
 10%|▉         | 21457/225000 [52:42:07<351:31:57,  6.22s/it]
 10%|▉         | 21458/225000 [52:42:15<377:35:34,  6.68s/it]
 10%|▉         | 21459/225000 [52:42:27<469:25:17,  8.30s/it]
 10%|▉         | 21460/225000 [52:42:30<386:36:59,  6.84s/it]
 10%|▉         | 21461/225000 [52:42:39<410:59:39,  7.27s/it]
 10%|▉         | 21462/225000 [52:42:45<392:52:14,  6.95s/it]
 10%|▉         | 21463/225000 [52:42:53<414:44:30,  7.34s/it]
 10%|▉         | 21464/225000 [52:42:58<365:09:18,  6.46s/it]
 10%|▉         | 21465/225000 [52:43:06<390:10:22,  6.90s/it]
 10%|▉         | 21466/225000 [52:43:11<365:07:48,  6.46s/it]
 10%|▉         | 21467/225000 [52:43:15<323:59:18,  5.73s/it]
 10%|▉         | 21468/225000 [52:43:26<417:05:23,  7.38s/it]
 10%|▉         | 21469/225000 [52:43:31<367:17:29,  6.50s/it]
 10%|▉         | 21470/225000 [52:43:36<353:43:13,  6.26s/it]
 10%|▉         | 21471/225000 [52:43:42<340:40:53,  6.03s/it]
 10%|▉         | 21472/225000 [52:43:48<340:37:43,  6.03s/it]
 10%|▉         | 21473/225000 [52:44:00<438:15:20,  7.75s/it]
 10%|▉         | 21474/225000 [52:44:04<375:40:14,  6.64s/it]
 10%|▉         | 21475/225000 [52:44:12<410:40:01,  7.26s/it]
 10%|▉         | 21476/225000 [52:44:18<387:58:56,  6.86s/it]
 10%|▉         | 21477/225000 [52:44:27<413:17:02,  7.31s/it]
 10%|▉         | 21478/225000 [52:44:34<408:42:03,  7.23s/it]
 10%|▉         | 21479/225000 [52:44:38<361:10:25,  6.39s/it]
 10%|▉         | 21480/225000 [52:44:43<329:40:02,  5.83s/it]
 10%|▉         | 21481/225000 [52:44:55<439:33:11,  7.78s/it]
 10%|▉         | 21482/225000 [52:45:00<397:01:48,  7.02s/it]
 10%|▉         | 21483/225000 [52:45:05<362:42:53,  6.42s/it]
 10%|▉         | 21484/225000 [52:45:10<341:52:21,  6.05s/it]
 10%|▉         | 21485/225000 [52:45:16<328:33:51,  5.81s/it]
 10%|▉         | 21486/225000 [52:45:22<333:37:15,  5.90s/it]
 10%|▉         | 21487/225000 [52:45:37<485:19:12,  8.58s/it]
 10%|▉         | 21488/225000 [52:45:42<423:11:09,  7.49s/it]
 10%|▉         | 21489/225000 [52:45:50<434:05:47,  7.68s/it]
 10%|▉         | 21490/225000 [52:45:54<379:13:45,  6.71s/it]
 10%|▉         | 21491/225000 [52:46:03<408:15:45,  7.22s/it]
 10%|▉         | 21492/225000 [52:46:08<384:59:18,  6.81s/it]
 10%|▉         | 21493/225000 [52:46:17<420:43:57,  7.44s/it]
 10%|▉         | 21494/225000 [52:46:24<410:03:22,  7.25s/it]
 10%|▉         | 21495/225000 [52:46:30<392:19:00,  6.94s/it]
 10%|▉         | 21496/225000 [52:46:35<349:02:24,  6.17s/it]
 10%|▉         | 21497/225000 [52:46:45<425:26:15,  7.53s/it]
 10%|▉         | 21498/225000 [52:46:51<400:18:32,  7.08s/it]
 10%|▉         | 21499/225000 [52:46:57<378:15:18,  6.69s/it]
 10%|▉         | 21500/225000 [52:47:02<353:01:59,  6.25s/it]
                                                             
{'loss': '0.7252', 'grad_norm': '0.7537', 'learning_rate': '4.965e-05', 'epoch': '0.09556', 'num_input_tokens_seen': 92215640, 'train_runtime': '1.9e+05', 'train_tokens_per_second': '485.3'}

 10%|▉         | 21500/225000 [52:47:02<353:01:59,  6.25s/it]
 10%|▉         | 21501/225000 [52:47:14<439:15:08,  7.77s/it]
 10%|▉         | 21502/225000 [52:47:17<362:53:01,  6.42s/it]
 10%|▉         | 21503/225000 [52:47:26<402:33:08,  7.12s/it]
 10%|▉         | 21504/225000 [52:47:30<350:54:47,  6.21s/it]
 10%|▉         | 21505/225000 [52:47:39<395:19:51,  6.99s/it]
 10%|▉         | 21506/225000 [52:47:43<353:30:31,  6.25s/it]
 10%|▉         | 21507/225000 [52:47:51<375:45:15,  6.65s/it]
 10%|▉         | 21508/225000 [52:48:00<421:04:59,  7.45s/it]
 10%|▉         | 21509/225000 [52:48:05<375:59:34,  6.65s/it]
 10%|▉         | 21510/225000 [52:48:10<341:03:58,  6.03s/it]
 10%|▉         | 21511/225000 [52:48:18<388:07:37,  6.87s/it]
 10%|▉         | 21512/225000 [52:48:27<411:57:19,  7.29s/it]
 10%|▉         | 21513/225000 [52:48:35<433:25:43,  7.67s/it]
 10%|▉         | 21514/225000 [52:48:39<370:07:00,  6.55s/it]
 10%|▉         | 21515/225000 [52:48:43<329:34:19,  5.83s/it]
 10%|▉         | 21516/225000 [52:48:50<337:03:46,  5.96s/it]
 10%|▉         | 21517/225000 [52:49:01<424:32:18,  7.51s/it]
 10%|▉         | 21518/225000 [52:49:07<412:10:25,  7.29s/it]
 10%|▉         | 21519/225000 [52:49:13<375:46:41,  6.65s/it]
 10%|▉         | 21520/225000 [52:49:18<359:43:44,  6.36s/it]
 10%|▉         | 21521/225000 [52:49:23<339:50:27,  6.01s/it]
 10%|▉         | 21522/225000 [52:49:28<321:51:46,  5.69s/it]
 10%|▉         | 21523/225000 [52:49:38<380:32:44,  6.73s/it]
 10%|▉         | 21524/225000 [52:49:51<497:00:31,  8.79s/it]
 10%|▉         | 21525/225000 [52:49:55<419:42:37,  7.43s/it]
 10%|▉         | 21526/225000 [52:50:00<365:25:59,  6.47s/it]
 10%|▉         | 21527/225000 [52:50:09<421:37:39,  7.46s/it]
 10%|▉         | 21528/225000 [52:50:18<447:19:58,  7.91s/it]
 10%|▉         | 21529/225000 [52:50:23<383:56:35,  6.79s/it]
 10%|▉         | 21530/225000 [52:50:31<406:37:56,  7.19s/it]
 10%|▉         | 21531/225000 [52:50:38<407:50:37,  7.22s/it]
 10%|▉         | 21532/225000 [52:50:47<435:28:27,  7.70s/it]
 10%|▉         | 21533/225000 [52:50:54<424:27:12,  7.51s/it]
 10%|▉         | 21534/225000 [52:50:58<375:20:36,  6.64s/it]
 10%|▉         | 21535/225000 [52:51:03<335:48:08,  5.94s/it]
 10%|▉         | 21536/225000 [52:51:08<318:13:48,  5.63s/it]
 10%|▉         | 21537/225000 [52:51:20<438:45:15,  7.76s/it]
 10%|▉         | 21538/225000 [52:51:25<385:17:22,  6.82s/it]
 10%|▉         | 21539/225000 [52:51:30<360:30:54,  6.38s/it]
 10%|▉         | 21540/225000 [52:51:36<345:01:51,  6.10s/it]
 10%|▉         | 21541/225000 [52:51:41<323:50:16,  5.73s/it]
 10%|▉         | 21542/225000 [52:51:54<448:11:39,  7.93s/it]
 10%|▉         | 21543/225000 [52:51:59<403:49:34,  7.15s/it]
 10%|▉         | 21544/225000 [52:52:04<365:55:46,  6.47s/it]
 10%|▉         | 21545/225000 [52:52:10<355:14:12,  6.29s/it]
 10%|▉         | 21546/225000 [52:52:15<342:04:22,  6.05s/it]
 10%|▉         | 21547/225000 [52:52:21<336:53:00,  5.96s/it]
 10%|▉         | 21548/225000 [52:52:27<335:12:36,  5.93s/it]
 10%|▉         | 21549/225000 [52:52:33<331:21:22,  5.86s/it]
 10%|▉         | 21550/225000 [52:52:43<399:23:27,  7.07s/it]
                                                             
{'loss': '0.7126', 'grad_norm': '0.7176', 'learning_rate': '4.965e-05', 'epoch': '0.09578', 'num_input_tokens_seen': 92368952, 'train_runtime': '1.904e+05', 'train_tokens_per_second': '485.2'}

 10%|▉         | 21550/225000 [52:52:43<399:23:27,  7.07s/it]
 10%|▉         | 21551/225000 [52:52:54<465:34:16,  8.24s/it]
 10%|▉         | 21552/225000 [52:53:02<463:05:50,  8.19s/it]
 10%|▉         | 21553/225000 [52:53:06<392:08:00,  6.94s/it]
 10%|▉         | 21554/225000 [52:53:14<413:07:39,  7.31s/it]
 10%|▉         | 21555/225000 [52:53:22<425:57:20,  7.54s/it]
 10%|▉         | 21556/225000 [52:53:27<383:42:05,  6.79s/it]
 10%|▉         | 21557/225000 [52:53:32<350:53:23,  6.21s/it]
 10%|▉         | 21558/225000 [52:53:37<331:08:41,  5.86s/it]
 10%|▉         | 21559/225000 [52:53:43<338:52:27,  6.00s/it]
 10%|▉         | 21560/225000 [52:53:56<455:53:53,  8.07s/it]
 10%|▉         | 21561/225000 [52:53:59<375:07:01,  6.64s/it]
 10%|▉         | 21562/225000 [52:54:09<421:41:15,  7.46s/it]
 10%|▉         | 21563/225000 [52:54:16<416:28:33,  7.37s/it]
 10%|▉         | 21564/225000 [52:54:21<383:59:56,  6.80s/it]
 10%|▉         | 21565/225000 [52:54:30<412:01:31,  7.29s/it]
 10%|▉         | 21566/225000 [52:54:33<349:55:31,  6.19s/it]
 10%|▉         | 21567/225000 [52:54:42<387:13:28,  6.85s/it]
 10%|▉         | 21568/225000 [52:54:47<360:50:34,  6.39s/it]
 10%|▉         | 21569/225000 [52:54:52<333:52:30,  5.91s/it]
 10%|▉         | 21570/225000 [52:54:57<313:30:59,  5.55s/it]
 10%|▉         | 21571/225000 [52:55:12<485:36:31,  8.59s/it]
 10%|▉         | 21572/225000 [52:55:17<420:47:21,  7.45s/it]
 10%|▉         | 21573/225000 [52:55:24<410:51:36,  7.27s/it]
 10%|▉         | 21574/225000 [52:55:32<426:36:20,  7.55s/it]
 10%|▉         | 21575/225000 [52:55:36<364:07:34,  6.44s/it]
 10%|▉         | 21576/225000 [52:55:46<422:46:59,  7.48s/it]
 10%|▉         | 21577/225000 [52:55:50<371:18:41,  6.57s/it]
 10%|▉         | 21578/225000 [52:55:55<334:24:28,  5.92s/it]
 10%|▉         | 21579/225000 [52:56:02<352:43:07,  6.24s/it]
 10%|▉         | 21580/225000 [52:56:12<423:24:28,  7.49s/it]
 10%|▉         | 21581/225000 [52:56:16<367:12:27,  6.50s/it]
 10%|▉         | 21582/225000 [52:56:27<431:34:50,  7.64s/it]
 10%|▉         | 21583/225000 [52:56:31<368:42:51,  6.53s/it]
 10%|▉         | 21584/225000 [52:56:39<403:01:31,  7.13s/it]
 10%|▉         | 21585/225000 [52:56:43<351:29:07,  6.22s/it]
 10%|▉         | 21586/225000 [52:56:51<373:18:55,  6.61s/it]
 10%|▉         | 21587/225000 [52:56:59<408:16:47,  7.23s/it]
 10%|▉         | 21588/225000 [52:57:06<400:37:05,  7.09s/it]
 10%|▉         | 21589/225000 [52:57:11<361:49:33,  6.40s/it]
 10%|▉         | 21590/225000 [52:57:15<330:35:56,  5.85s/it]
 10%|▉         | 21591/225000 [52:57:26<415:55:38,  7.36s/it]
 10%|▉         | 21592/225000 [52:57:30<356:17:46,  6.31s/it]
 10%|▉         | 21593/225000 [52:57:35<325:51:46,  5.77s/it]
 10%|▉         | 21594/225000 [52:57:40<325:41:05,  5.76s/it]
 10%|▉         | 21595/225000 [52:57:52<414:55:50,  7.34s/it]
 10%|▉         | 21596/225000 [52:58:02<470:25:51,  8.33s/it]
 10%|▉         | 21597/225000 [52:58:08<436:05:10,  7.72s/it]
 10%|▉         | 21598/225000 [52:58:13<380:37:05,  6.74s/it]
 10%|▉         | 21599/225000 [52:58:23<437:36:33,  7.75s/it]
 10%|▉         | 21600/225000 [52:58:27<376:26:06,  6.66s/it]
                                                             
{'loss': '0.717', 'grad_norm': '0.6187', 'learning_rate': '4.964e-05', 'epoch': '0.096', 'num_input_tokens_seen': 92521608, 'train_runtime': '1.907e+05', 'train_tokens_per_second': '485.1'}

 10%|▉         | 21600/225000 [52:58:27<376:26:06,  6.66s/it]
 10%|▉         | 21601/225000 [52:58:32<348:57:01,  6.18s/it]
 10%|▉         | 21602/225000 [52:58:43<420:13:38,  7.44s/it]
 10%|▉         | 21603/225000 [52:58:47<376:07:21,  6.66s/it]
 10%|▉         | 21604/225000 [52:58:52<348:12:36,  6.16s/it]
 10%|▉         | 21605/225000 [52:58:57<325:18:19,  5.76s/it]
 10%|▉         | 21606/225000 [52:59:03<325:17:50,  5.76s/it]
 10%|▉         | 21607/225000 [52:59:14<423:24:04,  7.49s/it]
 10%|▉         | 21608/225000 [52:59:21<413:04:48,  7.31s/it]
 10%|▉         | 21609/225000 [52:59:29<410:39:57,  7.27s/it]
 10%|▉         | 21610/225000 [52:59:38<442:41:30,  7.84s/it]
 10%|▉         | 21611/225000 [52:59:42<386:10:36,  6.84s/it]
 10%|▉         | 21612/225000 [52:59:47<348:11:55,  6.16s/it]
 10%|▉         | 21613/225000 [53:00:00<465:57:35,  8.25s/it]
 10%|▉         | 21614/225000 [53:00:04<399:09:51,  7.07s/it]
 10%|▉         | 21615/225000 [53:00:09<355:59:07,  6.30s/it]
 10%|▉         | 21616/225000 [53:00:13<329:33:09,  5.83s/it]
 10%|▉         | 21617/225000 [53:00:26<443:07:39,  7.84s/it]
 10%|▉         | 21618/225000 [53:00:31<401:26:45,  7.11s/it]
 10%|▉         | 21619/225000 [53:00:37<380:55:43,  6.74s/it]
 10%|▉         | 21620/225000 [53:00:46<410:07:41,  7.26s/it]
 10%|▉         | 21621/225000 [53:00:54<424:14:24,  7.51s/it]
 10%|▉         | 21622/225000 [53:00:58<365:03:38,  6.46s/it]
 10%|▉         | 21623/225000 [53:01:03<347:27:38,  6.15s/it]
 10%|▉         | 21624/225000 [53:01:08<324:58:34,  5.75s/it]
 10%|▉         | 21625/225000 [53:01:20<436:12:07,  7.72s/it]
 10%|▉         | 21626/225000 [53:01:28<432:33:00,  7.66s/it]
 10%|▉         | 21627/225000 [53:01:32<374:46:47,  6.63s/it]
 10%|▉         | 21628/225000 [53:01:36<330:56:57,  5.86s/it]
 10%|▉         | 21629/225000 [53:01:45<383:44:13,  6.79s/it]
 10%|▉         | 21630/225000 [53:01:55<432:05:24,  7.65s/it]
 10%|▉         | 21631/225000 [53:02:00<394:19:03,  6.98s/it]
 10%|▉         | 21632/225000 [53:02:13<487:37:24,  8.63s/it]
 10%|▉         | 21633/225000 [53:02:17<420:56:11,  7.45s/it]
 10%|▉         | 21634/225000 [53:02:23<385:31:13,  6.82s/it]
 10%|▉         | 21635/225000 [53:02:28<353:09:36,  6.25s/it]
 10%|▉         | 21636/225000 [53:02:33<334:24:19,  5.92s/it]
 10%|▉         | 21637/225000 [53:02:40<356:31:00,  6.31s/it]
 10%|▉         | 21638/225000 [53:02:53<463:34:35,  8.21s/it]
 10%|▉         | 21639/225000 [53:02:58<407:44:19,  7.22s/it]
 10%|▉         | 21640/225000 [53:03:05<417:28:39,  7.39s/it]
 10%|▉         | 21641/225000 [53:03:12<402:39:25,  7.13s/it]
 10%|▉         | 21642/225000 [53:03:20<416:23:14,  7.37s/it]
 10%|▉         | 21643/225000 [53:03:28<423:09:02,  7.49s/it]
 10%|▉         | 21644/225000 [53:03:34<396:45:49,  7.02s/it]
 10%|▉         | 21645/225000 [53:03:39<370:47:24,  6.56s/it]
 10%|▉         | 21646/225000 [53:03:45<363:26:36,  6.43s/it]
 10%|▉         | 21647/225000 [53:03:51<347:28:35,  6.15s/it]
 10%|▉         | 21648/225000 [53:03:57<346:25:12,  6.13s/it]
 10%|▉         | 21649/225000 [53:04:03<340:56:59,  6.04s/it]
 10%|▉         | 21650/225000 [53:04:16<458:58:26,  8.13s/it]
                                                             
{'loss': '0.7182', 'grad_norm': '0.6671', 'learning_rate': '4.964e-05', 'epoch': '0.09622', 'num_input_tokens_seen': 92675880, 'train_runtime': '1.911e+05', 'train_tokens_per_second': '485.1'}

 10%|▉         | 21650/225000 [53:04:16<458:58:26,  8.13s/it]
 10%|▉         | 21651/225000 [53:04:22<436:55:51,  7.74s/it]
 10%|▉         | 21652/225000 [53:04:27<383:48:40,  6.79s/it]
 10%|▉         | 21653/225000 [53:04:31<343:45:27,  6.09s/it]
 10%|▉         | 21654/225000 [53:04:40<388:55:16,  6.89s/it]
 10%|▉         | 21655/225000 [53:04:51<457:41:11,  8.10s/it]
 10%|▉         | 21656/225000 [53:04:55<389:59:58,  6.90s/it]
 10%|▉         | 21657/225000 [53:05:01<362:18:42,  6.41s/it]
 10%|▉         | 21658/225000 [53:05:06<351:48:36,  6.23s/it]
 10%|▉         | 21659/225000 [53:05:12<347:57:04,  6.16s/it]
 10%|▉         | 21660/225000 [53:05:18<342:15:23,  6.06s/it]
 10%|▉         | 21661/225000 [53:05:24<337:07:24,  5.97s/it]
 10%|▉         | 21662/225000 [53:05:30<338:55:17,  6.00s/it]
 10%|▉         | 21663/225000 [53:05:43<459:05:11,  8.13s/it]
 10%|▉         | 21664/225000 [53:05:50<446:50:46,  7.91s/it]
 10%|▉         | 21665/225000 [53:05:56<398:06:54,  7.05s/it]
 10%|▉         | 21666/225000 [53:06:01<368:57:38,  6.53s/it]
 10%|▉         | 21667/225000 [53:06:14<484:14:38,  8.57s/it]
 10%|▉         | 21668/225000 [53:06:19<418:30:13,  7.41s/it]
 10%|▉         | 21669/225000 [53:06:24<380:06:23,  6.73s/it]
 10%|▉         | 21670/225000 [53:06:29<349:59:20,  6.20s/it]
 10%|▉         | 21671/225000 [53:06:42<457:30:27,  8.10s/it]
 10%|▉         | 21672/225000 [53:06:46<391:37:23,  6.93s/it]
 10%|▉         | 21673/225000 [53:06:50<342:18:51,  6.06s/it]
 10%|▉         | 21674/225000 [53:07:01<427:11:49,  7.56s/it]
 10%|▉         | 21675/225000 [53:07:06<383:46:22,  6.79s/it]
 10%|▉         | 21676/225000 [53:07:12<370:23:41,  6.56s/it]
 10%|▉         | 21677/225000 [53:07:21<420:03:27,  7.44s/it]
 10%|▉         | 21678/225000 [53:07:30<439:23:14,  7.78s/it]
 10%|▉         | 21679/225000 [53:07:36<412:37:50,  7.31s/it]
 10%|▉         | 21680/225000 [53:07:43<408:41:02,  7.24s/it]
 10%|▉         | 21681/225000 [53:07:48<363:49:48,  6.44s/it]
 10%|▉         | 21682/225000 [53:07:57<414:08:10,  7.33s/it]
 10%|▉         | 21683/225000 [53:08:02<373:54:31,  6.62s/it]
 10%|▉         | 21684/225000 [53:08:07<348:52:00,  6.18s/it]
 10%|▉         | 21685/225000 [53:08:12<330:53:56,  5.86s/it]
 10%|▉         | 21686/225000 [53:08:18<322:01:26,  5.70s/it]
 10%|▉         | 21687/225000 [53:08:27<390:22:51,  6.91s/it]
 10%|▉         | 21688/225000 [53:08:36<414:26:27,  7.34s/it]
 10%|▉         | 21689/225000 [53:08:41<369:43:28,  6.55s/it]
 10%|▉         | 21690/225000 [53:08:46<360:14:29,  6.38s/it]
 10%|▉         | 21691/225000 [53:08:59<461:41:50,  8.18s/it]
 10%|▉         | 21692/225000 [53:09:03<385:52:28,  6.83s/it]
 10%|▉         | 21693/225000 [53:09:11<415:47:57,  7.36s/it]
 10%|▉         | 21694/225000 [53:09:17<391:04:22,  6.92s/it]
 10%|▉         | 21695/225000 [53:09:23<371:27:11,  6.58s/it]
 10%|▉         | 21696/225000 [53:09:27<334:01:18,  5.91s/it]
 10%|▉         | 21697/225000 [53:09:32<308:06:29,  5.46s/it]
 10%|▉         | 21698/225000 [53:09:45<450:35:33,  7.98s/it]
 10%|▉         | 21699/225000 [53:09:49<373:55:05,  6.62s/it]
 10%|▉         | 21700/225000 [53:09:57<402:29:47,  7.13s/it]
                                                             
{'loss': '0.7206', 'grad_norm': '0.6451', 'learning_rate': '4.964e-05', 'epoch': '0.09644', 'num_input_tokens_seen': 92829416, 'train_runtime': '1.914e+05', 'train_tokens_per_second': '485'}

 10%|▉         | 21700/225000 [53:09:57<402:29:47,  7.13s/it]
 10%|▉         | 21701/225000 [53:10:01<346:10:23,  6.13s/it]
 10%|▉         | 21702/225000 [53:10:10<399:51:21,  7.08s/it]
 10%|▉         | 21703/225000 [53:10:16<368:33:45,  6.53s/it]
 10%|▉         | 21704/225000 [53:10:25<416:21:45,  7.37s/it]
 10%|▉         | 21705/225000 [53:10:29<362:55:21,  6.43s/it]
 10%|▉         | 21706/225000 [53:10:35<351:06:19,  6.22s/it]
 10%|▉         | 21707/225000 [53:10:45<419:46:16,  7.43s/it]
 10%|▉         | 21708/225000 [53:10:51<386:30:24,  6.84s/it]
 10%|▉         | 21709/225000 [53:10:56<367:35:42,  6.51s/it]
 10%|▉         | 21710/225000 [53:11:04<390:16:34,  6.91s/it]
 10%|▉         | 21711/225000 [53:11:13<422:49:15,  7.49s/it]
 10%|▉         | 21712/225000 [53:11:17<363:28:43,  6.44s/it]
 10%|▉         | 21713/225000 [53:11:25<390:39:04,  6.92s/it]
 10%|▉         | 21714/225000 [53:11:33<416:45:19,  7.38s/it]
 10%|▉         | 21715/225000 [53:11:38<367:09:49,  6.50s/it]
 10%|▉         | 21716/225000 [53:11:43<337:58:53,  5.99s/it]
 10%|▉         | 21717/225000 [53:11:49<334:53:32,  5.93s/it]
 10%|▉         | 21718/225000 [53:12:00<425:27:00,  7.53s/it]
 10%|▉         | 21719/225000 [53:12:07<422:26:12,  7.48s/it]
 10%|▉         | 21720/225000 [53:12:12<382:35:22,  6.78s/it]
 10%|▉         | 21721/225000 [53:12:17<353:59:53,  6.27s/it]
 10%|▉         | 21722/225000 [53:12:23<338:03:32,  5.99s/it]
 10%|▉         | 21723/225000 [53:12:28<334:47:10,  5.93s/it]
 10%|▉         | 21724/225000 [53:12:34<321:09:24,  5.69s/it]
 10%|▉         | 21725/225000 [53:12:47<448:23:44,  7.94s/it]
 10%|▉         | 21726/225000 [53:12:55<460:29:42,  8.16s/it]
 10%|▉         | 21727/225000 [53:13:03<451:54:43,  8.00s/it]
 10%|▉         | 21728/225000 [53:13:10<428:06:40,  7.58s/it]
 10%|▉         | 21729/225000 [53:13:15<396:29:40,  7.02s/it]
 10%|▉         | 21730/225000 [53:13:20<351:26:11,  6.22s/it]
 10%|▉         | 21731/225000 [53:13:31<428:32:18,  7.59s/it]
 10%|▉         | 21732/225000 [53:13:37<406:08:15,  7.19s/it]
 10%|▉         | 21733/225000 [53:13:42<366:47:48,  6.50s/it]
 10%|▉         | 21734/225000 [53:13:46<334:49:02,  5.93s/it]
 10%|▉         | 21735/225000 [53:13:57<411:14:12,  7.28s/it]
 10%|▉         | 21736/225000 [53:14:06<437:11:19,  7.74s/it]
 10%|▉         | 21737/225000 [53:14:12<409:06:00,  7.25s/it]
 10%|▉         | 21738/225000 [53:14:19<407:41:44,  7.22s/it]
 10%|▉         | 21739/225000 [53:14:23<354:47:54,  6.28s/it]
 10%|▉         | 21740/225000 [53:14:28<331:57:52,  5.88s/it]
 10%|▉         | 21741/225000 [53:14:33<323:41:21,  5.73s/it]
 10%|▉         | 21742/225000 [53:14:41<362:32:48,  6.42s/it]
 10%|▉         | 21743/225000 [53:14:53<457:39:05,  8.11s/it]
 10%|▉         | 21744/225000 [53:14:59<423:19:15,  7.50s/it]
 10%|▉         | 21745/225000 [53:15:04<377:52:26,  6.69s/it]
 10%|▉         | 21746/225000 [53:15:11<375:25:17,  6.65s/it]
 10%|▉         | 21747/225000 [53:15:16<344:41:23,  6.11s/it]
 10%|▉         | 21748/225000 [53:15:23<359:39:24,  6.37s/it]
 10%|▉         | 21749/225000 [53:15:33<425:24:36,  7.53s/it]
 10%|▉         | 21750/225000 [53:15:42<449:21:19,  7.96s/it]
                                                             
{'loss': '0.7288', 'grad_norm': '0.7148', 'learning_rate': '4.963e-05', 'epoch': '0.09667', 'num_input_tokens_seen': 92982792, 'train_runtime': '1.917e+05', 'train_tokens_per_second': '484.9'}

 10%|▉         | 21750/225000 [53:15:42<449:21:19,  7.96s/it]
 10%|▉         | 21751/225000 [53:15:47<396:18:59,  7.02s/it]
 10%|▉         | 21752/225000 [53:15:52<361:43:07,  6.41s/it]
 10%|▉         | 21753/225000 [53:15:57<344:28:53,  6.10s/it]
 10%|▉         | 21754/225000 [53:16:02<329:25:46,  5.84s/it]
 10%|▉         | 21755/225000 [53:16:14<435:56:30,  7.72s/it]
 10%|▉         | 21756/225000 [53:16:23<456:13:24,  8.08s/it]
 10%|▉         | 21757/225000 [53:16:29<416:08:49,  7.37s/it]
 10%|▉         | 21758/225000 [53:16:38<438:19:12,  7.76s/it]
 10%|▉         | 21759/225000 [53:16:46<444:20:55,  7.87s/it]
 10%|▉         | 21760/225000 [53:16:50<384:57:46,  6.82s/it]
 10%|▉         | 21761/225000 [53:16:55<353:09:36,  6.26s/it]
 10%|▉         | 21762/225000 [53:17:00<324:18:37,  5.74s/it]
 10%|▉         | 21763/225000 [53:17:08<369:11:46,  6.54s/it]
 10%|▉         | 21764/225000 [53:17:17<410:19:28,  7.27s/it]
 10%|▉         | 21765/225000 [53:17:22<380:53:22,  6.75s/it]
 10%|▉         | 21766/225000 [53:17:30<396:40:41,  7.03s/it]
 10%|▉         | 21767/225000 [53:17:37<395:45:23,  7.01s/it]
 10%|▉         | 21768/225000 [53:17:42<358:38:36,  6.35s/it]
 10%|▉         | 21769/225000 [53:17:52<426:42:10,  7.56s/it]
 10%|▉         | 21770/225000 [53:18:02<455:31:30,  8.07s/it]
 10%|▉         | 21771/225000 [53:18:06<397:49:05,  7.05s/it]
 10%|▉         | 21772/225000 [53:18:13<389:10:29,  6.89s/it]
 10%|▉         | 21773/225000 [53:18:20<402:22:11,  7.13s/it]
 10%|▉         | 21774/225000 [53:18:27<392:31:11,  6.95s/it]
 10%|▉         | 21775/225000 [53:18:32<365:56:21,  6.48s/it]
 10%|▉         | 21776/225000 [53:18:41<405:25:11,  7.18s/it]
 10%|▉         | 21777/225000 [53:18:46<369:02:32,  6.54s/it]
 10%|▉         | 21778/225000 [53:18:51<344:09:58,  6.10s/it]
 10%|▉         | 21779/225000 [53:18:56<322:45:30,  5.72s/it]
 10%|▉         | 21780/225000 [53:19:01<310:03:51,  5.49s/it]
 10%|▉         | 21781/225000 [53:19:14<441:36:39,  7.82s/it]
 10%|▉         | 21782/225000 [53:19:19<380:24:14,  6.74s/it]
 10%|▉         | 21783/225000 [53:19:23<336:02:24,  5.95s/it]
 10%|▉         | 21784/225000 [53:19:32<397:23:10,  7.04s/it]
 10%|▉         | 21785/225000 [53:19:37<362:14:15,  6.42s/it]
 10%|▉         | 21786/225000 [53:19:42<334:46:28,  5.93s/it]
 10%|▉         | 21787/225000 [53:19:47<322:30:36,  5.71s/it]
 10%|▉         | 21788/225000 [53:19:55<350:03:59,  6.20s/it]
 10%|▉         | 21789/225000 [53:20:06<445:04:28,  7.88s/it]
 10%|▉         | 21790/225000 [53:20:11<386:41:08,  6.85s/it]
 10%|▉         | 21791/225000 [53:20:19<415:34:30,  7.36s/it]
 10%|▉         | 21792/225000 [53:20:26<403:50:54,  7.15s/it]
 10%|▉         | 21793/225000 [53:20:31<370:02:50,  6.56s/it]
 10%|▉         | 21794/225000 [53:20:37<349:55:02,  6.20s/it]
 10%|▉         | 21795/225000 [53:20:42<331:16:37,  5.87s/it]
 10%|▉         | 21796/225000 [53:20:48<340:33:51,  6.03s/it]
 10%|▉         | 21797/225000 [53:21:01<461:46:13,  8.18s/it]
 10%|▉         | 21798/225000 [53:21:08<437:41:33,  7.75s/it]
 10%|▉         | 21799/225000 [53:21:13<395:43:51,  7.01s/it]
 10%|▉         | 21800/225000 [53:21:19<370:28:35,  6.56s/it]
                                                             
{'loss': '0.702', 'grad_norm': '0.6481', 'learning_rate': '4.963e-05', 'epoch': '0.09689', 'num_input_tokens_seen': 93135944, 'train_runtime': '1.921e+05', 'train_tokens_per_second': '484.9'}

 10%|▉         | 21800/225000 [53:21:19<370:28:35,  6.56s/it]
 10%|▉         | 21801/225000 [53:21:25<356:20:14,  6.31s/it]
 10%|▉         | 21802/225000 [53:21:30<342:30:47,  6.07s/it]
 10%|▉         | 21803/225000 [53:21:37<353:40:51,  6.27s/it]
 10%|▉         | 21804/225000 [53:21:50<468:28:16,  8.30s/it]
 10%|▉         | 21805/225000 [53:21:56<432:09:17,  7.66s/it]
 10%|▉         | 21806/225000 [53:22:04<431:45:56,  7.65s/it]
 10%|▉         | 21807/225000 [53:22:08<375:55:15,  6.66s/it]
 10%|▉         | 21808/225000 [53:22:15<374:33:15,  6.64s/it]
 10%|▉         | 21809/225000 [53:22:24<416:42:03,  7.38s/it]
 10%|▉         | 21810/225000 [53:22:27<355:05:06,  6.29s/it]
 10%|▉         | 21811/225000 [53:22:32<322:10:23,  5.71s/it]
 10%|▉         | 21812/225000 [53:22:45<445:39:48,  7.90s/it]
 10%|▉         | 21813/225000 [53:22:50<404:11:33,  7.16s/it]
 10%|▉         | 21814/225000 [53:22:57<400:43:16,  7.10s/it]
 10%|▉         | 21815/225000 [53:23:02<362:20:20,  6.42s/it]
 10%|▉         | 21816/225000 [53:23:06<323:22:36,  5.73s/it]
 10%|▉         | 21817/225000 [53:23:18<428:11:51,  7.59s/it]
 10%|▉         | 21818/225000 [53:23:24<404:56:09,  7.17s/it]
 10%|▉         | 21819/225000 [53:23:33<426:09:13,  7.55s/it]
 10%|▉         | 21820/225000 [53:23:38<391:51:57,  6.94s/it]
 10%|▉         | 21821/225000 [53:23:46<406:36:48,  7.20s/it]
 10%|▉         | 21822/225000 [53:23:54<422:17:35,  7.48s/it]
 10%|▉         | 21823/225000 [53:23:59<379:56:51,  6.73s/it]
 10%|▉         | 21824/225000 [53:24:04<353:29:37,  6.26s/it]
 10%|▉         | 21825/225000 [53:24:09<331:25:08,  5.87s/it]
 10%|▉         | 21826/225000 [53:24:15<324:16:41,  5.75s/it]
 10%|▉         | 21827/225000 [53:24:20<318:47:16,  5.65s/it]
 10%|▉         | 21828/225000 [53:24:33<439:04:49,  7.78s/it]
 10%|▉         | 21829/225000 [53:24:37<379:27:06,  6.72s/it]
 10%|▉         | 21830/225000 [53:24:44<375:20:54,  6.65s/it]
 10%|▉         | 21831/225000 [53:24:53<426:05:23,  7.55s/it]
 10%|▉         | 21832/225000 [53:24:57<369:15:43,  6.54s/it]
 10%|▉         | 21833/225000 [53:25:02<328:51:09,  5.83s/it]
 10%|▉         | 21834/225000 [53:25:11<394:26:19,  6.99s/it]
 10%|▉         | 21835/225000 [53:25:19<410:22:06,  7.27s/it]
 10%|▉         | 21836/225000 [53:25:23<353:57:07,  6.27s/it]
 10%|▉         | 21837/225000 [53:25:32<395:51:40,  7.01s/it]
 10%|▉         | 21838/225000 [53:25:37<365:34:44,  6.48s/it]
 10%|▉         | 21839/225000 [53:25:42<341:20:40,  6.05s/it]
 10%|▉         | 21840/225000 [53:25:47<321:08:55,  5.69s/it]
 10%|▉         | 21841/225000 [53:25:53<320:59:58,  5.69s/it]
 10%|▉         | 21842/225000 [53:26:05<427:18:35,  7.57s/it]
 10%|▉         | 21843/225000 [53:26:13<438:07:39,  7.76s/it]
 10%|▉         | 21844/225000 [53:26:21<444:52:14,  7.88s/it]
 10%|▉         | 21845/225000 [53:26:25<379:31:10,  6.73s/it]
 10%|▉         | 21846/225000 [53:26:34<423:35:56,  7.51s/it]
 10%|▉         | 21847/225000 [53:26:39<365:32:17,  6.48s/it]
 10%|▉         | 21848/225000 [53:26:48<415:45:18,  7.37s/it]
 10%|▉         | 21849/225000 [53:26:53<368:04:49,  6.52s/it]
 10%|▉         | 21850/225000 [53:26:57<340:15:19,  6.03s/it]
                                                             
{'loss': '0.7146', 'grad_norm': '0.6976', 'learning_rate': '4.963e-05', 'epoch': '0.09711', 'num_input_tokens_seen': 93288496, 'train_runtime': '1.924e+05', 'train_tokens_per_second': '484.8'}

 10%|▉         | 21850/225000 [53:26:57<340:15:19,  6.03s/it]
 10%|▉         | 21851/225000 [53:27:02<318:06:24,  5.64s/it]
 10%|▉         | 21852/225000 [53:27:11<373:38:25,  6.62s/it]
 10%|▉         | 21853/225000 [53:27:22<441:48:10,  7.83s/it]
 10%|▉         | 21854/225000 [53:27:29<437:55:03,  7.76s/it]
 10%|▉         | 21855/225000 [53:27:35<402:12:28,  7.13s/it]
 10%|▉         | 21856/225000 [53:27:40<362:56:07,  6.43s/it]
 10%|▉         | 21857/225000 [53:27:45<343:53:56,  6.09s/it]
 10%|▉         | 21858/225000 [53:27:53<380:08:24,  6.74s/it]
 10%|▉         | 21859/225000 [53:28:03<431:23:07,  7.64s/it]
 10%|▉         | 21860/225000 [53:28:07<376:20:55,  6.67s/it]
 10%|▉         | 21861/225000 [53:28:12<345:16:14,  6.12s/it]
 10%|▉         | 21862/225000 [53:28:18<331:39:19,  5.88s/it]
 10%|▉         | 21863/225000 [53:28:28<416:22:13,  7.38s/it]
 10%|▉         | 21864/225000 [53:28:40<488:20:42,  8.65s/it]
 10%|▉         | 21865/225000 [53:28:44<404:36:11,  7.17s/it]
 10%|▉         | 21866/225000 [53:28:54<455:21:11,  8.07s/it]
 10%|▉         | 21867/225000 [53:28:58<389:42:33,  6.91s/it]
 10%|▉         | 21868/225000 [53:29:03<347:51:41,  6.16s/it]
 10%|▉         | 21869/225000 [53:29:16<467:16:44,  8.28s/it]
 10%|▉         | 21870/225000 [53:29:23<443:38:31,  7.86s/it]
 10%|▉         | 21871/225000 [53:29:27<376:22:23,  6.67s/it]
 10%|▉         | 21872/225000 [53:29:36<422:18:23,  7.48s/it]
 10%|▉         | 21873/225000 [53:29:43<409:33:06,  7.26s/it]
 10%|▉         | 21874/225000 [53:29:47<359:44:27,  6.38s/it]
 10%|▉         | 21875/225000 [53:29:57<413:23:41,  7.33s/it]
 10%|▉         | 21876/225000 [53:30:03<400:16:36,  7.09s/it]
 10%|▉         | 21877/225000 [53:30:08<360:10:10,  6.38s/it]
 10%|▉         | 21878/225000 [53:30:13<332:45:18,  5.90s/it]
 10%|▉         | 21879/225000 [53:30:23<410:39:30,  7.28s/it]
 10%|▉         | 21880/225000 [53:30:30<397:47:49,  7.05s/it]
 10%|▉         | 21881/225000 [53:30:37<406:54:25,  7.21s/it]
 10%|▉         | 21882/225000 [53:30:44<407:23:47,  7.22s/it]
 10%|▉         | 21883/225000 [53:30:51<401:54:30,  7.12s/it]
 10%|▉         | 21884/225000 [53:30:56<358:32:44,  6.35s/it]
 10%|▉         | 21885/225000 [53:31:05<398:02:05,  7.05s/it]
 10%|▉         | 21886/225000 [53:31:10<377:28:34,  6.69s/it]
 10%|▉         | 21887/225000 [53:31:14<329:37:25,  5.84s/it]
 10%|▉         | 21888/225000 [53:31:23<385:27:06,  6.83s/it]
 10%|▉         | 21889/225000 [53:31:28<345:41:25,  6.13s/it]
 10%|▉         | 21890/225000 [53:31:32<318:43:22,  5.65s/it]
 10%|▉         | 21891/225000 [53:31:43<406:37:50,  7.21s/it]
 10%|▉         | 21892/225000 [53:31:50<404:27:44,  7.17s/it]
 10%|▉         | 21893/225000 [53:31:59<433:46:27,  7.69s/it]
 10%|▉         | 21894/225000 [53:32:05<405:33:53,  7.19s/it]
 10%|▉         | 21895/225000 [53:32:10<359:42:30,  6.38s/it]
 10%|▉         | 21896/225000 [53:32:15<343:42:52,  6.09s/it]
 10%|▉         | 21897/225000 [53:32:21<342:45:02,  6.08s/it]
 10%|▉         | 21898/225000 [53:32:34<460:51:36,  8.17s/it]
 10%|▉         | 21899/225000 [53:32:41<427:14:18,  7.57s/it]
 10%|▉         | 21900/225000 [53:32:46<396:26:51,  7.03s/it]
                                                             
{'loss': '0.7054', 'grad_norm': '0.7188', 'learning_rate': '4.962e-05', 'epoch': '0.09733', 'num_input_tokens_seen': 93441176, 'train_runtime': '1.928e+05', 'train_tokens_per_second': '484.7'}

 10%|▉         | 21900/225000 [53:32:46<396:26:51,  7.03s/it]
 10%|▉         | 21901/225000 [53:32:51<354:26:14,  6.28s/it]
 10%|▉         | 21902/225000 [53:32:57<345:34:55,  6.13s/it]
 10%|▉         | 21903/225000 [53:33:02<340:30:54,  6.04s/it]
 10%|▉         | 21904/225000 [53:33:08<329:05:36,  5.83s/it]
 10%|▉         | 21905/225000 [53:33:21<452:42:46,  8.02s/it]
 10%|▉         | 21906/225000 [53:33:25<391:37:20,  6.94s/it]
 10%|▉         | 21907/225000 [53:33:34<417:39:21,  7.40s/it]
 10%|▉         | 21908/225000 [53:33:40<405:18:41,  7.18s/it]
 10%|▉         | 21909/225000 [53:33:45<364:44:53,  6.47s/it]
 10%|▉         | 21910/225000 [53:33:51<352:31:13,  6.25s/it]
 10%|▉         | 21911/225000 [53:34:00<393:37:07,  6.98s/it]
 10%|▉         | 21912/225000 [53:34:12<485:09:59,  8.60s/it]
 10%|▉         | 21913/225000 [53:34:16<405:02:41,  7.18s/it]
 10%|▉         | 21914/225000 [53:34:22<386:12:42,  6.85s/it]
 10%|▉         | 21915/225000 [53:34:32<437:24:41,  7.75s/it]
 10%|▉         | 21916/225000 [53:34:37<396:06:19,  7.02s/it]
 10%|▉         | 21917/225000 [53:34:43<367:36:31,  6.52s/it]
 10%|▉         | 21918/225000 [53:34:48<347:10:29,  6.15s/it]
 10%|▉         | 21919/225000 [53:34:53<332:32:34,  5.89s/it]
 10%|▉         | 21920/225000 [53:34:58<317:40:49,  5.63s/it]
 10%|▉         | 21921/225000 [53:35:04<322:42:53,  5.72s/it]
 10%|▉         | 21922/225000 [53:35:18<465:44:01,  8.26s/it]
 10%|▉         | 21923/225000 [53:35:27<480:17:01,  8.51s/it]
 10%|▉         | 21924/225000 [53:35:32<417:23:38,  7.40s/it]
 10%|▉         | 21925/225000 [53:35:40<425:13:43,  7.54s/it]
 10%|▉         | 21926/225000 [53:35:48<436:58:34,  7.75s/it]
 10%|▉         | 21927/225000 [53:35:52<367:41:15,  6.52s/it]
 10%|▉         | 21928/225000 [53:36:01<417:09:20,  7.40s/it]
 10%|▉         | 21929/225000 [53:36:05<357:11:01,  6.33s/it]
 10%|▉         | 21930/225000 [53:36:13<373:59:05,  6.63s/it]
 10%|▉         | 21931/225000 [53:36:20<382:19:30,  6.78s/it]
 10%|▉         | 21932/225000 [53:36:26<375:28:53,  6.66s/it]
 10%|▉         | 21933/225000 [53:36:33<377:35:34,  6.69s/it]
 10%|▉         | 21934/225000 [53:36:39<370:57:59,  6.58s/it]
 10%|▉         | 21935/225000 [53:36:45<360:16:22,  6.39s/it]
 10%|▉         | 21936/225000 [53:36:53<389:18:18,  6.90s/it]
 10%|▉         | 21937/225000 [53:36:59<366:08:57,  6.49s/it]
 10%|▉         | 21938/225000 [53:37:04<350:31:12,  6.21s/it]
 10%|▉         | 21939/225000 [53:37:09<330:37:47,  5.86s/it]
 10%|▉         | 21940/225000 [53:37:14<318:30:19,  5.65s/it]
 10%|▉         | 21941/225000 [53:37:20<320:15:01,  5.68s/it]
 10%|▉         | 21942/225000 [53:37:25<312:39:32,  5.54s/it]
 10%|▉         | 21943/225000 [53:37:34<358:02:41,  6.35s/it]
 10%|▉         | 21944/225000 [53:37:42<392:26:29,  6.96s/it]
 10%|▉         | 21945/225000 [53:37:52<435:20:37,  7.72s/it]
 10%|▉         | 21946/225000 [53:37:56<376:10:16,  6.67s/it]
 10%|▉         | 21947/225000 [53:38:05<418:13:00,  7.41s/it]
 10%|▉         | 21948/225000 [53:38:09<364:47:08,  6.47s/it]
 10%|▉         | 21949/225000 [53:38:19<423:16:46,  7.50s/it]
 10%|▉         | 21950/225000 [53:38:24<385:32:26,  6.84s/it]
                                                             
{'loss': '0.7009', 'grad_norm': '0.6486', 'learning_rate': '4.962e-05', 'epoch': '0.09756', 'num_input_tokens_seen': 93592432, 'train_runtime': '1.931e+05', 'train_tokens_per_second': '484.7'}

 10%|▉         | 21950/225000 [53:38:24<385:32:26,  6.84s/it]
 10%|▉         | 21951/225000 [53:38:30<361:06:11,  6.40s/it]
 10%|▉         | 21952/225000 [53:38:35<347:24:01,  6.16s/it]
 10%|▉         | 21953/225000 [53:38:41<342:10:57,  6.07s/it]
 10%|▉         | 21954/225000 [53:38:47<332:15:28,  5.89s/it]
 10%|▉         | 21955/225000 [53:38:53<335:40:41,  5.95s/it]
 10%|▉         | 21956/225000 [53:38:58<326:48:36,  5.79s/it]
 10%|▉         | 21957/225000 [53:39:07<381:20:27,  6.76s/it]
 10%|▉         | 21958/225000 [53:39:20<479:11:00,  8.50s/it]
 10%|▉         | 21959/225000 [53:39:23<398:43:16,  7.07s/it]
 10%|▉         | 21960/225000 [53:39:28<358:38:48,  6.36s/it]
 10%|▉         | 21961/225000 [53:39:40<455:01:11,  8.07s/it]
 10%|▉         | 21962/225000 [53:39:46<413:42:00,  7.34s/it]
 10%|▉         | 21963/225000 [53:39:57<480:24:33,  8.52s/it]
 10%|▉         | 21964/225000 [53:40:01<403:50:02,  7.16s/it]
 10%|▉         | 21965/225000 [53:40:06<368:03:45,  6.53s/it]
 10%|▉         | 21966/225000 [53:40:12<355:19:08,  6.30s/it]
 10%|▉         | 21967/225000 [53:40:20<392:52:33,  6.97s/it]
 10%|▉         | 21968/225000 [53:40:26<363:56:25,  6.45s/it]
 10%|▉         | 21969/225000 [53:40:31<348:06:34,  6.17s/it]
 10%|▉         | 21970/225000 [53:40:42<421:35:16,  7.48s/it]
 10%|▉         | 21971/225000 [53:40:49<410:30:15,  7.28s/it]
 10%|▉         | 21972/225000 [53:40:54<380:53:57,  6.75s/it]
 10%|▉         | 21973/225000 [53:41:00<364:24:03,  6.46s/it]
 10%|▉         | 21974/225000 [53:41:10<428:42:24,  7.60s/it]
 10%|▉         | 21975/225000 [53:41:15<378:24:55,  6.71s/it]
 10%|▉         | 21976/225000 [53:41:20<347:14:08,  6.16s/it]
 10%|▉         | 21977/225000 [53:41:27<362:17:39,  6.42s/it]
 10%|▉         | 21978/225000 [53:41:39<463:18:12,  8.22s/it]
 10%|▉         | 21979/225000 [53:41:43<393:18:34,  6.97s/it]
 10%|▉         | 21980/225000 [53:41:51<415:35:30,  7.37s/it]
 10%|▉         | 21981/225000 [53:41:56<369:31:23,  6.55s/it]
 10%|▉         | 21982/225000 [53:42:01<341:41:54,  6.06s/it]
 10%|▉         | 21983/225000 [53:42:10<398:12:31,  7.06s/it]
 10%|▉         | 21984/225000 [53:42:19<423:03:39,  7.50s/it]
 10%|▉         | 21985/225000 [53:42:25<403:42:17,  7.16s/it]
 10%|▉         | 21986/225000 [53:42:29<348:05:11,  6.17s/it]
 10%|▉         | 21987/225000 [53:42:34<319:55:06,  5.67s/it]
 10%|▉         | 21988/225000 [53:42:45<408:37:48,  7.25s/it]
 10%|▉         | 21989/225000 [53:42:52<413:27:02,  7.33s/it]
 10%|▉         | 21990/225000 [53:42:57<367:33:30,  6.52s/it]
 10%|▉         | 21991/225000 [53:43:04<373:54:29,  6.63s/it]
 10%|▉         | 21992/225000 [53:43:10<367:02:28,  6.51s/it]
 10%|▉         | 21993/225000 [53:43:16<361:28:54,  6.41s/it]
 10%|▉         | 21994/225000 [53:43:22<354:18:11,  6.28s/it]
 10%|▉         | 21995/225000 [53:43:27<336:43:00,  5.97s/it]
 10%|▉         | 21996/225000 [53:43:33<331:31:59,  5.88s/it]
 10%|▉         | 21997/225000 [53:43:39<330:33:54,  5.86s/it]
 10%|▉         | 21998/225000 [53:43:45<329:57:22,  5.85s/it]
 10%|▉         | 21999/225000 [53:43:50<328:49:55,  5.83s/it]
 10%|▉         | 22000/225000 [53:44:02<427:41:16,  7.58s/it]
                                                             
{'loss': '0.7071', 'grad_norm': '0.6228', 'learning_rate': '4.962e-05', 'epoch': '0.09778', 'num_input_tokens_seen': 93745672, 'train_runtime': '1.934e+05', 'train_tokens_per_second': '484.6'}

 10%|▉         | 22000/225000 [53:44:02<427:41:16,  7.58s/it]
 10%|▉         | 22001/225000 [53:44:11<455:49:51,  8.08s/it]
 10%|▉         | 22002/225000 [53:44:18<426:50:14,  7.57s/it]
 10%|▉         | 22003/225000 [53:44:21<362:58:44,  6.44s/it]
 10%|▉         | 22004/225000 [53:44:32<434:33:46,  7.71s/it]
 10%|▉         | 22005/225000 [53:44:36<369:30:59,  6.55s/it]
 10%|▉         | 22006/225000 [53:44:45<404:10:42,  7.17s/it]
 10%|▉         | 22007/225000 [53:44:49<352:33:41,  6.25s/it]
 10%|▉         | 22008/225000 [53:44:57<382:36:22,  6.79s/it]
 10%|▉         | 22009/225000 [53:45:01<348:07:40,  6.17s/it]
 10%|▉         | 22010/225000 [53:45:10<389:46:19,  6.91s/it]
 10%|▉         | 22011/225000 [53:45:20<441:53:12,  7.84s/it]
 10%|▉         | 22012/225000 [53:45:24<382:10:33,  6.78s/it]
 10%|▉         | 22013/225000 [53:45:32<387:21:08,  6.87s/it]
 10%|▉         | 22014/225000 [53:45:38<374:31:56,  6.64s/it]
 10%|▉         | 22015/225000 [53:45:44<363:30:08,  6.45s/it]
 10%|▉         | 22016/225000 [53:45:49<352:37:36,  6.25s/it]
 10%|▉         | 22017/225000 [53:45:58<388:46:56,  6.90s/it]
 10%|▉         | 22018/225000 [53:46:04<384:27:03,  6.82s/it]
 10%|▉         | 22019/225000 [53:46:10<362:12:48,  6.42s/it]
 10%|▉         | 22020/225000 [53:46:15<345:07:53,  6.12s/it]
 10%|▉         | 22021/225000 [53:46:27<444:38:27,  7.89s/it]
 10%|▉         | 22022/225000 [53:46:33<407:29:37,  7.23s/it]
 10%|▉         | 22023/225000 [53:46:38<363:03:19,  6.44s/it]
 10%|▉         | 22024/225000 [53:46:42<334:29:13,  5.93s/it]
 10%|▉         | 22025/225000 [53:46:48<331:06:59,  5.87s/it]
 10%|▉         | 22026/225000 [53:47:01<454:59:12,  8.07s/it]
 10%|▉         | 22027/225000 [53:47:06<404:17:03,  7.17s/it]
 10%|▉         | 22028/225000 [53:47:13<399:36:46,  7.09s/it]
 10%|▉         | 22029/225000 [53:47:18<365:40:54,  6.49s/it]
 10%|▉         | 22030/225000 [53:47:23<341:56:57,  6.07s/it]
 10%|▉         | 22031/225000 [53:47:29<328:29:05,  5.83s/it]
 10%|▉         | 22032/225000 [53:47:42<452:55:31,  8.03s/it]
 10%|▉         | 22033/225000 [53:47:48<427:10:36,  7.58s/it]
 10%|▉         | 22034/225000 [53:47:54<392:48:53,  6.97s/it]
 10%|▉         | 22035/225000 [53:47:59<353:19:55,  6.27s/it]
 10%|▉         | 22036/225000 [53:48:03<324:21:35,  5.75s/it]
 10%|▉         | 22037/225000 [53:48:14<416:04:36,  7.38s/it]
 10%|▉         | 22038/225000 [53:48:19<366:37:05,  6.50s/it]
 10%|▉         | 22039/225000 [53:48:28<418:25:59,  7.42s/it]
 10%|▉         | 22040/225000 [53:48:32<353:25:48,  6.27s/it]
 10%|▉         | 22041/225000 [53:48:41<402:19:55,  7.14s/it]
 10%|▉         | 22042/225000 [53:48:47<389:09:43,  6.90s/it]
 10%|▉         | 22043/225000 [53:48:52<355:15:46,  6.30s/it]
 10%|▉         | 22044/225000 [53:49:00<384:35:50,  6.82s/it]
 10%|▉         | 22045/225000 [53:49:09<408:26:31,  7.24s/it]
 10%|▉         | 22046/225000 [53:49:13<361:13:43,  6.41s/it]
 10%|▉         | 22047/225000 [53:49:22<403:15:19,  7.15s/it]
 10%|▉         | 22048/225000 [53:49:26<354:05:07,  6.28s/it]
 10%|▉         | 22049/225000 [53:49:34<386:14:36,  6.85s/it]
 10%|▉         | 22050/225000 [53:49:43<409:47:26,  7.27s/it]
                                                             
{'loss': '0.6893', 'grad_norm': '0.7964', 'learning_rate': '4.961e-05', 'epoch': '0.098', 'num_input_tokens_seen': 93897200, 'train_runtime': '1.938e+05', 'train_tokens_per_second': '484.5'}

 10%|▉         | 22050/225000 [53:49:43<409:47:26,  7.27s/it]
 10%|▉         | 22051/225000 [53:49:47<357:30:16,  6.34s/it]
 10%|▉         | 22052/225000 [53:49:53<350:19:12,  6.21s/it]
 10%|▉         | 22053/225000 [53:50:01<379:42:50,  6.74s/it]
 10%|▉         | 22054/225000 [53:50:08<389:44:16,  6.91s/it]
 10%|▉         | 22055/225000 [53:50:16<403:27:26,  7.16s/it]
 10%|▉         | 22056/225000 [53:50:23<402:43:26,  7.14s/it]
 10%|▉         | 22057/225000 [53:50:27<355:49:36,  6.31s/it]
 10%|▉         | 22058/225000 [53:50:33<338:23:42,  6.00s/it]
 10%|▉         | 22059/225000 [53:50:38<329:37:59,  5.85s/it]
 10%|▉         | 22060/225000 [53:50:43<312:31:26,  5.54s/it]
 10%|▉         | 22061/225000 [53:50:55<419:25:52,  7.44s/it]
 10%|▉         | 22062/225000 [53:51:01<408:11:53,  7.24s/it]
 10%|▉         | 22063/225000 [53:51:07<372:42:20,  6.61s/it]
 10%|▉         | 22064/225000 [53:51:17<435:05:44,  7.72s/it]
 10%|▉         | 22065/225000 [53:51:21<375:11:01,  6.66s/it]
 10%|▉         | 22066/225000 [53:51:31<434:41:22,  7.71s/it]
 10%|▉         | 22067/225000 [53:51:36<384:14:31,  6.82s/it]
 10%|▉         | 22068/225000 [53:51:43<389:40:36,  6.91s/it]
 10%|▉         | 22069/225000 [53:51:51<407:47:49,  7.23s/it]
 10%|▉         | 22070/225000 [53:51:56<367:50:22,  6.53s/it]
 10%|▉         | 22071/225000 [53:52:01<345:03:46,  6.12s/it]
 10%|▉         | 22072/225000 [53:52:07<336:51:02,  5.98s/it]
 10%|▉         | 22073/225000 [53:52:13<335:13:42,  5.95s/it]
 10%|▉         | 22074/225000 [53:52:18<325:49:50,  5.78s/it]
 10%|▉         | 22075/225000 [53:52:31<446:07:37,  7.91s/it]
 10%|▉         | 22076/225000 [53:52:37<412:53:18,  7.32s/it]
 10%|▉         | 22077/225000 [53:52:44<413:23:51,  7.33s/it]
 10%|▉         | 22078/225000 [53:52:51<400:52:06,  7.11s/it]
 10%|▉         | 22079/225000 [53:52:55<355:09:56,  6.30s/it]
 10%|▉         | 22080/225000 [53:53:01<342:04:51,  6.07s/it]
 10%|▉         | 22081/225000 [53:53:14<458:51:05,  8.14s/it]
 10%|▉         | 22082/225000 [53:53:18<392:28:48,  6.96s/it]
 10%|▉         | 22083/225000 [53:53:27<426:19:27,  7.56s/it]
 10%|▉         | 22084/225000 [53:53:32<387:34:07,  6.88s/it]
 10%|▉         | 22085/225000 [53:53:37<352:46:24,  6.26s/it]
 10%|▉         | 22086/225000 [53:53:43<347:44:54,  6.17s/it]
 10%|▉         | 22087/225000 [53:53:49<337:45:06,  5.99s/it]
 10%|▉         | 22088/225000 [53:54:00<426:19:20,  7.56s/it]
 10%|▉         | 22089/225000 [53:54:07<424:22:02,  7.53s/it]
 10%|▉         | 22090/225000 [53:54:16<447:48:09,  7.94s/it]
 10%|▉         | 22091/225000 [53:54:21<402:46:35,  7.15s/it]
 10%|▉         | 22092/225000 [53:54:29<407:13:29,  7.22s/it]
 10%|▉         | 22093/225000 [53:54:36<404:21:09,  7.17s/it]
 10%|▉         | 22094/225000 [53:54:44<427:33:51,  7.59s/it]
 10%|▉         | 22095/225000 [53:54:49<374:01:19,  6.64s/it]
 10%|▉         | 22096/225000 [53:54:53<329:31:06,  5.85s/it]
 10%|▉         | 22097/225000 [53:55:05<434:18:40,  7.71s/it]
 10%|▉         | 22098/225000 [53:55:10<391:09:04,  6.94s/it]
 10%|▉         | 22099/225000 [53:55:15<362:48:22,  6.44s/it]
 10%|▉         | 22100/225000 [53:55:21<341:24:10,  6.06s/it]
                                                             
{'loss': '0.7119', 'grad_norm': '0.6333', 'learning_rate': '4.961e-05', 'epoch': '0.09822', 'num_input_tokens_seen': 94050744, 'train_runtime': '1.941e+05', 'train_tokens_per_second': '484.5'}

 10%|▉         | 22100/225000 [53:55:21<341:24:10,  6.06s/it]
 10%|▉         | 22101/225000 [53:55:26<327:56:31,  5.82s/it]
 10%|▉         | 22102/225000 [53:55:31<325:34:25,  5.78s/it]
 10%|▉         | 22103/225000 [53:55:40<369:13:35,  6.55s/it]
 10%|▉         | 22104/225000 [53:55:52<459:16:39,  8.15s/it]
 10%|▉         | 22105/225000 [53:55:59<439:23:05,  7.80s/it]
 10%|▉         | 22106/225000 [53:56:04<392:37:55,  6.97s/it]
 10%|▉         | 22107/225000 [53:56:08<349:27:20,  6.20s/it]
 10%|▉         | 22108/225000 [53:56:13<323:16:00,  5.74s/it]
 10%|▉         | 22109/225000 [53:56:20<355:59:52,  6.32s/it]
 10%|▉         | 22110/225000 [53:56:31<425:44:40,  7.55s/it]
 10%|▉         | 22111/225000 [53:56:35<371:59:52,  6.60s/it]
 10%|▉         | 22112/225000 [53:56:41<364:48:07,  6.47s/it]
 10%|▉         | 22113/225000 [53:56:49<389:13:39,  6.91s/it]
 10%|▉         | 22114/225000 [53:56:58<413:49:20,  7.34s/it]
 10%|▉         | 22115/225000 [53:57:02<369:07:51,  6.55s/it]
 10%|▉         | 22116/225000 [53:57:07<338:06:24,  6.00s/it]
 10%|▉         | 22117/225000 [53:57:12<324:38:35,  5.76s/it]
 10%|▉         | 22118/225000 [53:57:21<367:56:07,  6.53s/it]
 10%|▉         | 22119/225000 [53:57:31<432:53:44,  7.68s/it]
 10%|▉         | 22120/225000 [53:57:35<368:27:06,  6.54s/it]
 10%|▉         | 22121/225000 [53:57:45<432:16:40,  7.67s/it]
 10%|▉         | 22122/225000 [53:57:50<375:49:27,  6.67s/it]
 10%|▉         | 22123/225000 [53:57:58<408:16:29,  7.24s/it]
 10%|▉         | 22124/225000 [53:58:10<478:22:19,  8.49s/it]
 10%|▉         | 22125/225000 [53:58:14<406:59:23,  7.22s/it]
 10%|▉         | 22126/225000 [53:58:18<349:15:21,  6.20s/it]
 10%|▉         | 22127/225000 [53:58:27<408:24:37,  7.25s/it]
 10%|▉         | 22128/225000 [53:58:32<370:01:36,  6.57s/it]
 10%|▉         | 22129/225000 [53:58:37<334:31:27,  5.94s/it]
 10%|▉         | 22130/225000 [53:58:46<387:46:00,  6.88s/it]
 10%|▉         | 22131/225000 [53:58:55<429:01:22,  7.61s/it]
 10%|▉         | 22132/225000 [53:59:03<437:52:52,  7.77s/it]
 10%|▉         | 22133/225000 [53:59:12<451:16:38,  8.01s/it]
 10%|▉         | 22134/225000 [53:59:16<387:03:56,  6.87s/it]
 10%|▉         | 22135/225000 [53:59:20<343:21:12,  6.09s/it]
 10%|▉         | 22136/225000 [53:59:32<430:22:36,  7.64s/it]
 10%|▉         | 22137/225000 [53:59:37<392:14:56,  6.96s/it]
 10%|▉         | 22138/225000 [53:59:43<381:52:10,  6.78s/it]
 10%|▉         | 22139/225000 [53:59:50<375:16:15,  6.66s/it]
 10%|▉         | 22140/225000 [53:59:58<406:42:56,  7.22s/it]
 10%|▉         | 22141/225000 [54:00:03<368:32:14,  6.54s/it]
 10%|▉         | 22142/225000 [54:00:08<341:02:41,  6.05s/it]
 10%|▉         | 22143/225000 [54:00:13<321:00:00,  5.70s/it]
 10%|▉         | 22144/225000 [54:00:18<315:30:58,  5.60s/it]
 10%|▉         | 22145/225000 [54:00:32<457:54:38,  8.13s/it]
 10%|▉         | 22146/225000 [54:00:38<420:31:32,  7.46s/it]
 10%|▉         | 22147/225000 [54:00:43<377:46:56,  6.70s/it]
 10%|▉         | 22148/225000 [54:00:49<360:16:13,  6.39s/it]
 10%|▉         | 22149/225000 [54:00:54<340:00:54,  6.03s/it]
 10%|▉         | 22150/225000 [54:01:00<342:59:44,  6.09s/it]
                                                             
{'loss': '0.7085', 'grad_norm': '0.6853', 'learning_rate': '4.961e-05', 'epoch': '0.09844', 'num_input_tokens_seen': 94202904, 'train_runtime': '1.945e+05', 'train_tokens_per_second': '484.4'}

 10%|▉         | 22150/225000 [54:01:00<342:59:44,  6.09s/it]
 10%|▉         | 22151/225000 [54:01:06<338:52:38,  6.01s/it]
 10%|▉         | 22152/225000 [54:01:12<336:32:20,  5.97s/it]
 10%|▉         | 22153/225000 [54:01:26<469:06:33,  8.33s/it]
 10%|▉         | 22154/225000 [54:01:34<459:53:41,  8.16s/it]
 10%|▉         | 22155/225000 [54:01:41<451:52:50,  8.02s/it]
 10%|▉         | 22156/225000 [54:01:46<394:57:09,  7.01s/it]
 10%|▉         | 22157/225000 [54:01:51<353:50:23,  6.28s/it]
 10%|▉         | 22158/225000 [54:02:02<441:23:51,  7.83s/it]
 10%|▉         | 22159/225000 [54:02:07<392:03:50,  6.96s/it]
 10%|▉         | 22160/225000 [54:02:16<436:26:38,  7.75s/it]
 10%|▉         | 22161/225000 [54:02:20<373:24:15,  6.63s/it]
 10%|▉         | 22162/225000 [54:02:31<440:31:07,  7.82s/it]
 10%|▉         | 22163/225000 [54:02:36<386:01:20,  6.85s/it]
 10%|▉         | 22164/225000 [54:02:41<357:51:48,  6.35s/it]
 10%|▉         | 22165/225000 [54:02:46<344:23:24,  6.11s/it]
 10%|▉         | 22166/225000 [54:02:57<413:08:53,  7.33s/it]
 10%|▉         | 22167/225000 [54:03:02<386:37:14,  6.86s/it]
 10%|▉         | 22168/225000 [54:03:07<352:19:15,  6.25s/it]
 10%|▉         | 22169/225000 [54:03:12<332:17:27,  5.90s/it]
 10%|▉         | 22170/225000 [54:03:27<483:16:27,  8.58s/it]
 10%|▉         | 22171/225000 [54:03:33<433:21:07,  7.69s/it]
 10%|▉         | 22172/225000 [54:03:39<416:16:25,  7.39s/it]
 10%|▉         | 22173/225000 [54:03:46<398:32:01,  7.07s/it]
 10%|▉         | 22174/225000 [54:03:54<421:06:14,  7.47s/it]
 10%|▉         | 22175/225000 [54:03:59<369:13:04,  6.55s/it]
 10%|▉         | 22176/225000 [54:04:08<415:00:03,  7.37s/it]
 10%|▉         | 22177/225000 [54:04:13<375:36:26,  6.67s/it]
 10%|▉         | 22178/225000 [54:04:18<344:19:43,  6.11s/it]
 10%|▉         | 22179/225000 [54:04:23<326:40:06,  5.80s/it]
 10%|▉         | 22180/225000 [54:04:33<406:12:27,  7.21s/it]
 10%|▉         | 22181/225000 [54:04:43<444:15:25,  7.89s/it]
 10%|▉         | 22182/225000 [54:04:48<393:25:32,  6.98s/it]
 10%|▉         | 22183/225000 [54:04:56<419:33:18,  7.45s/it]
 10%|▉         | 22184/225000 [54:05:01<374:38:05,  6.65s/it]
 10%|▉         | 22185/225000 [54:05:09<390:59:55,  6.94s/it]
 10%|▉         | 22186/225000 [54:05:20<470:31:01,  8.35s/it]
 10%|▉         | 22187/225000 [54:05:24<397:29:27,  7.06s/it]
 10%|▉         | 22188/225000 [54:05:29<364:20:11,  6.47s/it]
 10%|▉         | 22189/225000 [54:05:34<341:00:25,  6.05s/it]
 10%|▉         | 22190/225000 [54:05:46<442:59:45,  7.86s/it]
 10%|▉         | 22191/225000 [54:05:53<422:51:13,  7.51s/it]
 10%|▉         | 22192/225000 [54:05:58<385:02:32,  6.83s/it]
 10%|▉         | 22193/225000 [54:06:08<431:06:44,  7.65s/it]
 10%|▉         | 22194/225000 [54:06:14<399:46:24,  7.10s/it]
 10%|▉         | 22195/225000 [54:06:19<364:32:49,  6.47s/it]
 10%|▉         | 22196/225000 [54:06:25<354:32:09,  6.29s/it]
 10%|▉         | 22197/225000 [54:06:30<345:12:34,  6.13s/it]
 10%|▉         | 22198/225000 [54:06:36<338:57:34,  6.02s/it]
 10%|▉         | 22199/225000 [54:06:42<337:06:34,  5.98s/it]
 10%|▉         | 22200/225000 [54:06:48<328:26:15,  5.83s/it]
                                                             
{'loss': '0.6921', 'grad_norm': '0.7407', 'learning_rate': '4.96e-05', 'epoch': '0.09867', 'num_input_tokens_seen': 94355624, 'train_runtime': '1.948e+05', 'train_tokens_per_second': '484.4'}

 10%|▉         | 22200/225000 [54:06:48<328:26:15,  5.83s/it]
 10%|▉         | 22201/225000 [54:07:00<447:00:33,  7.94s/it]
 10%|▉         | 22202/225000 [54:07:08<450:05:27,  7.99s/it]
 10%|▉         | 22203/225000 [54:07:13<398:07:17,  7.07s/it]
 10%|▉         | 22204/225000 [54:07:19<367:45:18,  6.53s/it]
 10%|▉         | 22205/225000 [54:07:26<387:50:23,  6.88s/it]
 10%|▉         | 22206/225000 [54:07:37<447:20:45,  7.94s/it]
 10%|▉         | 22207/225000 [54:07:43<415:01:53,  7.37s/it]
 10%|▉         | 22208/225000 [54:07:49<400:13:45,  7.10s/it]
 10%|▉         | 22209/225000 [54:07:59<450:54:24,  8.00s/it]
 10%|▉         | 22210/225000 [54:08:03<382:59:04,  6.80s/it]
 10%|▉         | 22211/225000 [54:08:09<368:12:12,  6.54s/it]
 10%|▉         | 22212/225000 [54:08:15<351:07:39,  6.23s/it]
 10%|▉         | 22213/225000 [54:08:21<349:11:14,  6.20s/it]
 10%|▉         | 22214/225000 [54:08:29<387:51:37,  6.89s/it]
 10%|▉         | 22215/225000 [54:08:35<366:30:05,  6.51s/it]
 10%|▉         | 22216/225000 [54:08:41<358:06:07,  6.36s/it]
 10%|▉         | 22217/225000 [54:08:47<349:38:14,  6.21s/it]
 10%|▉         | 22218/225000 [54:08:53<339:36:41,  6.03s/it]
 10%|▉         | 22219/225000 [54:08:58<326:29:42,  5.80s/it]
 10%|▉         | 22220/225000 [54:09:06<370:17:03,  6.57s/it]
 10%|▉         | 22221/225000 [54:09:15<403:56:00,  7.17s/it]
 10%|▉         | 22222/225000 [54:09:24<437:14:25,  7.76s/it]
 10%|▉         | 22223/225000 [54:09:29<388:31:15,  6.90s/it]
 10%|▉         | 22224/225000 [54:09:34<355:49:39,  6.32s/it]
 10%|▉         | 22225/225000 [54:09:43<407:22:24,  7.23s/it]
 10%|▉         | 22226/225000 [54:09:55<481:18:38,  8.55s/it]
 10%|▉         | 22227/225000 [54:10:00<425:09:54,  7.55s/it]
 10%|▉         | 22228/225000 [54:10:04<370:51:02,  6.58s/it]
 10%|▉         | 22229/225000 [54:10:15<432:12:32,  7.67s/it]
 10%|▉         | 22230/225000 [54:10:19<378:49:25,  6.73s/it]
 10%|▉         | 22231/225000 [54:10:29<432:48:33,  7.68s/it]
 10%|▉         | 22232/225000 [54:10:37<443:15:56,  7.87s/it]
 10%|▉         | 22233/225000 [54:10:43<408:20:43,  7.25s/it]
 10%|▉         | 22234/225000 [54:10:47<356:58:57,  6.34s/it]
 10%|▉         | 22235/225000 [54:10:57<409:18:57,  7.27s/it]
 10%|▉         | 22236/225000 [54:11:04<413:40:23,  7.34s/it]
 10%|▉         | 22237/225000 [54:11:09<365:37:28,  6.49s/it]
 10%|▉         | 22238/225000 [54:11:14<346:27:59,  6.15s/it]
 10%|▉         | 22239/225000 [54:11:19<332:41:17,  5.91s/it]
 10%|▉         | 22240/225000 [54:11:25<324:26:51,  5.76s/it]
 10%|▉         | 22241/225000 [54:11:31<327:22:45,  5.81s/it]
 10%|▉         | 22242/225000 [54:11:36<310:35:18,  5.51s/it]
 10%|▉         | 22243/225000 [54:11:41<314:42:17,  5.59s/it]
 10%|▉         | 22244/225000 [54:11:54<432:00:29,  7.67s/it]
 10%|▉         | 22245/225000 [54:12:02<432:13:51,  7.67s/it]
 10%|▉         | 22246/225000 [54:12:08<407:00:10,  7.23s/it]
 10%|▉         | 22247/225000 [54:12:16<416:52:17,  7.40s/it]
 10%|▉         | 22248/225000 [54:12:22<403:36:47,  7.17s/it]
 10%|▉         | 22249/225000 [54:12:26<355:43:48,  6.32s/it]
 10%|▉         | 22250/225000 [54:12:32<345:47:46,  6.14s/it]
                                                             
{'loss': '0.6906', 'grad_norm': '0.609', 'learning_rate': '4.96e-05', 'epoch': '0.09889', 'num_input_tokens_seen': 94506560, 'train_runtime': '1.952e+05', 'train_tokens_per_second': '484.3'}

 10%|▉         | 22250/225000 [54:12:32<345:47:46,  6.14s/it]
 10%|▉         | 22251/225000 [54:12:38<331:29:16,  5.89s/it]
 10%|▉         | 22252/225000 [54:12:45<357:53:14,  6.35s/it]
 10%|▉         | 22253/225000 [54:12:58<471:05:58,  8.36s/it]
 10%|▉         | 22254/225000 [54:13:03<409:34:59,  7.27s/it]
 10%|▉         | 22255/225000 [54:13:11<422:46:36,  7.51s/it]
 10%|▉         | 22256/225000 [54:13:19<436:15:07,  7.75s/it]
 10%|▉         | 22257/225000 [54:13:23<376:24:10,  6.68s/it]
 10%|▉         | 22258/225000 [54:13:32<410:33:42,  7.29s/it]
 10%|▉         | 22259/225000 [54:13:39<400:55:34,  7.12s/it]
 10%|▉         | 22260/225000 [54:13:45<387:29:58,  6.88s/it]
 10%|▉         | 22261/225000 [54:13:51<364:15:31,  6.47s/it]
 10%|▉         | 22262/225000 [54:13:57<356:41:25,  6.33s/it]
 10%|▉         | 22263/225000 [54:14:02<345:28:06,  6.13s/it]
 10%|▉         | 22264/225000 [54:14:08<343:11:31,  6.09s/it]
 10%|▉         | 22265/225000 [54:14:21<449:33:46,  7.98s/it]
 10%|▉         | 22266/225000 [54:14:25<394:56:02,  7.01s/it]
 10%|▉         | 22267/225000 [54:14:30<351:58:55,  6.25s/it]
 10%|▉         | 22268/225000 [54:14:35<329:09:17,  5.84s/it]
 10%|▉         | 22269/225000 [54:14:40<311:47:45,  5.54s/it]
 10%|▉         | 22270/225000 [54:14:50<391:36:40,  6.95s/it]
 10%|▉         | 22271/225000 [54:14:58<404:28:36,  7.18s/it]
 10%|▉         | 22272/225000 [54:15:05<404:03:49,  7.18s/it]
 10%|▉         | 22273/225000 [54:15:13<415:29:55,  7.38s/it]
 10%|▉         | 22274/225000 [54:15:17<371:16:56,  6.59s/it]
 10%|▉         | 22275/225000 [54:15:26<401:54:25,  7.14s/it]
 10%|▉         | 22276/225000 [54:15:30<349:33:52,  6.21s/it]
 10%|▉         | 22277/225000 [54:15:37<372:07:40,  6.61s/it]
 10%|▉         | 22278/225000 [54:15:41<328:33:53,  5.83s/it]
 10%|▉         | 22279/225000 [54:15:51<385:47:51,  6.85s/it]
 10%|▉         | 22280/225000 [54:15:55<352:02:10,  6.25s/it]
 10%|▉         | 22281/225000 [54:16:00<330:23:07,  5.87s/it]
 10%|▉         | 22282/225000 [54:16:05<315:54:54,  5.61s/it]
 10%|▉         | 22283/225000 [54:16:10<302:18:05,  5.37s/it]
 10%|▉         | 22284/225000 [54:16:22<409:52:43,  7.28s/it]
 10%|▉         | 22285/225000 [54:16:28<387:43:29,  6.89s/it]
 10%|▉         | 22286/225000 [54:16:33<358:07:26,  6.36s/it]
 10%|▉         | 22287/225000 [54:16:42<408:48:02,  7.26s/it]
 10%|▉         | 22288/225000 [54:16:50<422:01:02,  7.49s/it]
 10%|▉         | 22289/225000 [54:16:55<380:26:11,  6.76s/it]
 10%|▉         | 22290/225000 [54:17:00<347:00:44,  6.16s/it]
 10%|▉         | 22291/225000 [54:17:04<311:47:21,  5.54s/it]
 10%|▉         | 22292/225000 [54:17:18<448:16:55,  7.96s/it]
 10%|▉         | 22293/225000 [54:17:22<377:15:06,  6.70s/it]
 10%|▉         | 22294/225000 [54:17:31<420:01:47,  7.46s/it]
 10%|▉         | 22295/225000 [54:17:36<373:50:26,  6.64s/it]
 10%|▉         | 22296/225000 [54:17:40<337:36:08,  6.00s/it]
 10%|▉         | 22297/225000 [54:17:45<321:02:14,  5.70s/it]
 10%|▉         | 22298/225000 [54:17:53<355:50:42,  6.32s/it]
 10%|▉         | 22299/225000 [54:18:04<432:42:04,  7.68s/it]
 10%|▉         | 22300/225000 [54:18:08<375:40:41,  6.67s/it]
                                                             
{'loss': '0.7111', 'grad_norm': '0.6256', 'learning_rate': '4.96e-05', 'epoch': '0.09911', 'num_input_tokens_seen': 94657936, 'train_runtime': '1.955e+05', 'train_tokens_per_second': '484.2'}

 10%|▉         | 22300/225000 [54:18:08<375:40:41,  6.67s/it]
 10%|▉         | 22301/225000 [54:18:19<449:40:53,  7.99s/it]
 10%|▉         | 22302/225000 [54:18:23<382:20:34,  6.79s/it]
 10%|▉         | 22303/225000 [54:18:32<418:56:56,  7.44s/it]
 10%|▉         | 22304/225000 [54:18:39<406:19:04,  7.22s/it]
 10%|▉         | 22305/225000 [54:18:46<398:24:04,  7.08s/it]
 10%|▉         | 22306/225000 [54:18:51<373:08:05,  6.63s/it]
 10%|▉         | 22307/225000 [54:18:58<372:31:50,  6.62s/it]
 10%|▉         | 22308/225000 [54:19:07<410:46:46,  7.30s/it]
 10%|▉         | 22309/225000 [54:19:10<350:43:18,  6.23s/it]
 10%|▉         | 22310/225000 [54:19:19<386:20:24,  6.86s/it]
 10%|▉         | 22311/225000 [54:19:26<392:54:18,  6.98s/it]
 10%|▉         | 22312/225000 [54:19:29<334:00:39,  5.93s/it]
 10%|▉         | 22313/225000 [54:19:40<410:17:20,  7.29s/it]
 10%|▉         | 22314/225000 [54:19:47<401:36:24,  7.13s/it]
 10%|▉         | 22315/225000 [54:19:51<351:15:57,  6.24s/it]
 10%|▉         | 22316/225000 [54:19:59<382:57:39,  6.80s/it]
 10%|▉         | 22317/225000 [54:20:04<353:41:26,  6.28s/it]
 10%|▉         | 22318/225000 [54:20:12<388:11:38,  6.90s/it]
 10%|▉         | 22319/225000 [54:20:20<399:32:41,  7.10s/it]
 10%|▉         | 22320/225000 [54:20:24<354:08:41,  6.29s/it]
 10%|▉         | 22321/225000 [54:20:29<323:37:09,  5.75s/it]
 10%|▉         | 22322/225000 [54:20:41<436:34:24,  7.75s/it]
 10%|▉         | 22323/225000 [54:20:45<365:34:11,  6.49s/it]
 10%|▉         | 22324/225000 [54:20:55<420:28:14,  7.47s/it]
 10%|▉         | 22325/225000 [54:21:01<400:46:18,  7.12s/it]
 10%|▉         | 22326/225000 [54:21:07<377:06:31,  6.70s/it]
 10%|▉         | 22327/225000 [54:21:13<378:54:02,  6.73s/it]
 10%|▉         | 22328/225000 [54:21:17<329:23:28,  5.85s/it]
 10%|▉         | 22329/225000 [54:21:27<400:01:41,  7.11s/it]
 10%|▉         | 22330/225000 [54:21:32<364:21:26,  6.47s/it]
 10%|▉         | 22331/225000 [54:21:37<341:47:35,  6.07s/it]
 10%|▉         | 22332/225000 [54:21:42<322:12:44,  5.72s/it]
 10%|▉         | 22333/225000 [54:21:47<313:52:18,  5.58s/it]
 10%|▉         | 22334/225000 [54:21:57<377:56:16,  6.71s/it]
 10%|▉         | 22335/225000 [54:22:05<403:21:51,  7.17s/it]
 10%|▉         | 22336/225000 [54:22:11<386:51:20,  6.87s/it]
 10%|▉         | 22337/225000 [54:22:17<361:35:24,  6.42s/it]
 10%|▉         | 22338/225000 [54:22:22<347:05:42,  6.17s/it]
 10%|▉         | 22339/225000 [54:22:28<333:51:58,  5.93s/it]
 10%|▉         | 22340/225000 [54:22:33<328:10:30,  5.83s/it]
 10%|▉         | 22341/225000 [54:22:39<323:25:25,  5.75s/it]
 10%|▉         | 22342/225000 [54:22:46<342:36:19,  6.09s/it]
 10%|▉         | 22343/225000 [54:22:59<462:25:46,  8.21s/it]
 10%|▉         | 22344/225000 [54:23:05<434:51:14,  7.72s/it]
 10%|▉         | 22345/225000 [54:23:10<390:53:10,  6.94s/it]
 10%|▉         | 22346/225000 [54:23:16<364:55:56,  6.48s/it]
 10%|▉         | 22347/225000 [54:23:21<349:02:54,  6.20s/it]
 10%|▉         | 22348/225000 [54:23:27<335:26:04,  5.96s/it]
 10%|▉         | 22349/225000 [54:23:32<329:30:54,  5.85s/it]
 10%|▉         | 22350/225000 [54:23:38<322:22:53,  5.73s/it]
                                                             
{'loss': '0.7099', 'grad_norm': '0.6658', 'learning_rate': '4.959e-05', 'epoch': '0.09933', 'num_input_tokens_seen': 94809248, 'train_runtime': '1.958e+05', 'train_tokens_per_second': '484.2'}

 10%|▉         | 22350/225000 [54:23:38<322:22:53,  5.73s/it]
 10%|▉         | 22351/225000 [54:23:47<375:41:26,  6.67s/it]
 10%|▉         | 22352/225000 [54:24:00<486:45:31,  8.65s/it]
 10%|▉         | 22353/225000 [54:24:04<414:04:35,  7.36s/it]
 10%|▉         | 22354/225000 [54:24:11<402:35:47,  7.15s/it]
 10%|▉         | 22355/225000 [54:24:18<393:04:39,  6.98s/it]
 10%|▉         | 22356/225000 [54:24:22<349:34:38,  6.21s/it]
 10%|▉         | 22357/225000 [54:24:28<351:22:05,  6.24s/it]
 10%|▉         | 22358/225000 [54:24:38<416:07:29,  7.39s/it]
 10%|▉         | 22359/225000 [54:24:45<398:18:28,  7.08s/it]
 10%|▉         | 22360/225000 [54:24:53<417:28:19,  7.42s/it]
 10%|▉         | 22361/225000 [54:24:57<367:47:28,  6.53s/it]
 10%|▉         | 22362/225000 [54:25:05<385:17:06,  6.84s/it]
 10%|▉         | 22363/225000 [54:25:10<351:27:34,  6.24s/it]
 10%|▉         | 22364/225000 [54:25:15<325:06:54,  5.78s/it]
 10%|▉         | 22365/225000 [54:25:26<416:58:01,  7.41s/it]
 10%|▉         | 22366/225000 [54:25:32<392:09:00,  6.97s/it]
 10%|▉         | 22367/225000 [54:25:40<413:35:12,  7.35s/it]
 10%|▉         | 22368/225000 [54:25:48<431:32:16,  7.67s/it]
 10%|▉         | 22369/225000 [54:25:53<383:12:37,  6.81s/it]
 10%|▉         | 22370/225000 [54:25:58<351:03:24,  6.24s/it]
 10%|▉         | 22371/225000 [54:26:04<341:39:59,  6.07s/it]
 10%|▉         | 22372/225000 [54:26:16<453:02:46,  8.05s/it]
 10%|▉         | 22373/225000 [54:26:22<410:46:23,  7.30s/it]
 10%|▉         | 22374/225000 [54:26:27<374:26:41,  6.65s/it]
 10%|▉         | 22375/225000 [54:26:33<360:18:01,  6.40s/it]
 10%|▉         | 22376/225000 [54:26:38<341:31:27,  6.07s/it]
 10%|▉         | 22377/225000 [54:26:44<332:29:07,  5.91s/it]
 10%|▉         | 22378/225000 [54:26:50<337:52:27,  6.00s/it]
 10%|▉         | 22379/225000 [54:27:03<464:44:48,  8.26s/it]
 10%|▉         | 22380/225000 [54:27:09<422:45:08,  7.51s/it]
 10%|▉         | 22381/225000 [54:27:18<436:37:45,  7.76s/it]
 10%|▉         | 22382/225000 [54:27:22<386:15:24,  6.86s/it]
 10%|▉         | 22383/225000 [54:27:32<426:12:47,  7.57s/it]
 10%|▉         | 22384/225000 [54:27:37<387:28:22,  6.88s/it]
 10%|▉         | 22385/225000 [54:27:45<410:06:56,  7.29s/it]
 10%|▉         | 22386/225000 [54:27:51<389:52:54,  6.93s/it]
 10%|▉         | 22387/225000 [54:27:57<380:08:57,  6.75s/it]
 10%|▉         | 22388/225000 [54:28:02<336:17:19,  5.98s/it]
 10%|▉         | 22389/225000 [54:28:10<382:56:37,  6.80s/it]
 10%|▉         | 22390/225000 [54:28:16<357:52:52,  6.36s/it]
 10%|▉         | 22391/225000 [54:28:23<373:31:29,  6.64s/it]
 10%|▉         | 22392/225000 [54:28:32<417:53:11,  7.43s/it]
 10%|▉         | 22393/225000 [54:28:36<359:43:14,  6.39s/it]
 10%|▉         | 22394/225000 [54:28:41<338:29:55,  6.01s/it]
 10%|▉         | 22395/225000 [54:28:52<415:35:56,  7.38s/it]
 10%|▉         | 22396/225000 [54:28:56<358:31:18,  6.37s/it]
 10%|▉         | 22397/225000 [54:29:06<423:52:46,  7.53s/it]
 10%|▉         | 22398/225000 [54:29:10<360:59:24,  6.41s/it]
 10%|▉         | 22399/225000 [54:29:19<396:16:58,  7.04s/it]
 10%|▉         | 22400/225000 [54:29:23<359:41:18,  6.39s/it]
                                                             
{'loss': '0.7134', 'grad_norm': '0.7355', 'learning_rate': '4.959e-05', 'epoch': '0.09956', 'num_input_tokens_seen': 94961056, 'train_runtime': '1.962e+05', 'train_tokens_per_second': '484.1'}

 10%|▉         | 22400/225000 [54:29:23<359:41:18,  6.39s/it]
 10%|▉         | 22401/225000 [54:29:28<334:38:20,  5.95s/it]
 10%|▉         | 22402/225000 [54:29:33<320:33:01,  5.70s/it]
 10%|▉         | 22403/225000 [54:29:42<374:41:31,  6.66s/it]
 10%|▉         | 22404/225000 [54:29:54<460:24:30,  8.18s/it]
 10%|▉         | 22405/225000 [54:29:57<379:48:19,  6.75s/it]
 10%|▉         | 22406/225000 [54:30:05<399:17:59,  7.10s/it]
 10%|▉         | 22407/225000 [54:30:13<408:00:03,  7.25s/it]
 10%|▉         | 22408/225000 [54:30:18<366:40:26,  6.52s/it]
 10%|▉         | 22409/225000 [54:30:22<334:26:15,  5.94s/it]
 10%|▉         | 22410/225000 [54:30:28<323:59:03,  5.76s/it]
 10%|▉         | 22411/225000 [54:30:40<433:31:01,  7.70s/it]
 10%|▉         | 22412/225000 [54:30:46<401:22:10,  7.13s/it]
 10%|▉         | 22413/225000 [54:30:50<358:28:48,  6.37s/it]
 10%|▉         | 22414/225000 [54:31:02<440:36:01,  7.83s/it]
 10%|▉         | 22415/225000 [54:31:08<421:15:11,  7.49s/it]
 10%|▉         | 22416/225000 [54:31:14<394:40:12,  7.01s/it]
 10%|▉         | 22417/225000 [54:31:20<382:18:02,  6.79s/it]
 10%|▉         | 22418/225000 [54:31:27<375:51:37,  6.68s/it]
 10%|▉         | 22419/225000 [54:31:34<379:38:59,  6.75s/it]
 10%|▉         | 22420/225000 [54:31:41<395:55:43,  7.04s/it]
 10%|▉         | 22421/225000 [54:31:46<347:55:18,  6.18s/it]
 10%|▉         | 22422/225000 [54:31:55<398:41:51,  7.09s/it]
 10%|▉         | 22423/225000 [54:32:00<363:50:18,  6.47s/it]
 10%|▉         | 22424/225000 [54:32:05<338:10:20,  6.01s/it]
 10%|▉         | 22425/225000 [54:32:10<319:34:49,  5.68s/it]
 10%|▉         | 22426/225000 [54:32:15<308:56:40,  5.49s/it]
 10%|▉         | 22427/225000 [54:32:28<433:12:15,  7.70s/it]
 10%|▉         | 22428/225000 [54:32:33<395:50:24,  7.03s/it]
 10%|▉         | 22429/225000 [54:32:38<357:25:16,  6.35s/it]
 10%|▉         | 22430/225000 [54:32:46<382:09:08,  6.79s/it]
 10%|▉         | 22431/225000 [54:32:56<439:32:01,  7.81s/it]
 10%|▉         | 22432/225000 [54:33:05<456:47:28,  8.12s/it]
 10%|▉         | 22433/225000 [54:33:10<414:53:13,  7.37s/it]
 10%|▉         | 22434/225000 [54:33:19<435:48:35,  7.75s/it]
 10%|▉         | 22435/225000 [54:33:23<380:01:15,  6.75s/it]
 10%|▉         | 22436/225000 [54:33:33<428:42:12,  7.62s/it]
 10%|▉         | 22437/225000 [54:33:39<399:53:51,  7.11s/it]
 10%|▉         | 22438/225000 [54:33:46<400:53:25,  7.12s/it]
 10%|▉         | 22439/225000 [54:33:51<362:13:59,  6.44s/it]
 10%|▉         | 22440/225000 [54:33:56<338:43:29,  6.02s/it]
 10%|▉         | 22441/225000 [54:34:01<320:43:58,  5.70s/it]
 10%|▉         | 22442/225000 [54:34:06<308:40:32,  5.49s/it]
 10%|▉         | 22443/225000 [54:34:17<401:00:40,  7.13s/it]
 10%|▉         | 22444/225000 [54:34:23<389:19:33,  6.92s/it]
 10%|▉         | 22445/225000 [54:34:32<418:34:03,  7.44s/it]
 10%|▉         | 22446/225000 [54:34:36<360:21:50,  6.40s/it]
 10%|▉         | 22447/225000 [54:34:45<410:43:12,  7.30s/it]
 10%|▉         | 22448/225000 [54:34:49<356:10:39,  6.33s/it]
 10%|▉         | 22449/225000 [54:34:59<412:31:39,  7.33s/it]
 10%|▉         | 22450/225000 [54:35:03<360:10:40,  6.40s/it]
                                                             
{'loss': '0.7266', 'grad_norm': '0.65', 'learning_rate': '4.959e-05', 'epoch': '0.09978', 'num_input_tokens_seen': 95114160, 'train_runtime': '1.965e+05', 'train_tokens_per_second': '484'}

 10%|▉         | 22450/225000 [54:35:03<360:10:40,  6.40s/it]
 10%|▉         | 22451/225000 [54:35:12<402:29:09,  7.15s/it]
 10%|▉         | 22452/225000 [54:35:19<392:36:41,  6.98s/it]
 10%|▉         | 22453/225000 [54:35:24<366:09:28,  6.51s/it]
 10%|▉         | 22454/225000 [54:35:30<350:30:18,  6.23s/it]
 10%|▉         | 22455/225000 [54:35:35<338:35:16,  6.02s/it]
 10%|▉         | 22456/225000 [54:35:41<333:34:53,  5.93s/it]
 10%|▉         | 22457/225000 [54:35:47<326:17:55,  5.80s/it]
 10%|▉         | 22458/225000 [54:35:52<326:20:54,  5.80s/it]
 10%|▉         | 22459/225000 [54:36:04<430:01:35,  7.64s/it]
 10%|▉         | 22460/225000 [54:36:13<445:22:39,  7.92s/it]
 10%|▉         | 22461/225000 [54:36:22<464:43:11,  8.26s/it]
 10%|▉         | 22462/225000 [54:36:26<391:31:11,  6.96s/it]
 10%|▉         | 22463/225000 [54:36:30<342:09:56,  6.08s/it]
 10%|▉         | 22464/225000 [54:36:42<442:32:41,  7.87s/it]
 10%|▉         | 22465/225000 [54:36:46<382:53:20,  6.81s/it]
 10%|▉         | 22466/225000 [54:36:56<434:12:21,  7.72s/it]
 10%|▉         | 22467/225000 [54:37:00<377:24:38,  6.71s/it]
 10%|▉         | 22468/225000 [54:37:07<381:28:00,  6.78s/it]
 10%|▉         | 22469/225000 [54:37:14<379:25:11,  6.74s/it]
 10%|▉         | 22470/225000 [54:37:23<413:20:40,  7.35s/it]
 10%|▉         | 22471/225000 [54:37:27<357:51:36,  6.36s/it]
 10%|▉         | 22472/225000 [54:37:35<395:10:58,  7.02s/it]
 10%|▉         | 22473/225000 [54:37:42<395:05:57,  7.02s/it]
 10%|▉         | 22474/225000 [54:37:48<376:36:31,  6.69s/it]
 10%|▉         | 22475/225000 [54:37:56<395:47:44,  7.04s/it]
 10%|▉         | 22476/225000 [54:38:00<344:16:52,  6.12s/it]
 10%|▉         | 22477/225000 [54:38:10<413:04:22,  7.34s/it]
 10%|▉         | 22478/225000 [54:38:15<366:03:28,  6.51s/it]
 10%|▉         | 22479/225000 [54:38:24<405:01:07,  7.20s/it]
 10%|▉         | 22480/225000 [54:38:29<365:06:16,  6.49s/it]
 10%|▉         | 22481/225000 [54:38:33<336:38:10,  5.98s/it]
 10%|▉         | 22482/225000 [54:38:38<320:08:39,  5.69s/it]
 10%|▉         | 22483/225000 [54:38:46<352:04:45,  6.26s/it]
 10%|▉         | 22484/225000 [54:38:57<439:09:21,  7.81s/it]
 10%|▉         | 22485/225000 [54:39:02<379:42:48,  6.75s/it]
 10%|▉         | 22486/225000 [54:39:07<353:26:39,  6.28s/it]
 10%|▉         | 22487/225000 [54:39:12<337:02:02,  5.99s/it]
 10%|▉         | 22488/225000 [54:39:19<357:29:38,  6.36s/it]
 10%|▉         | 22489/225000 [54:39:29<415:35:52,  7.39s/it]
 10%|▉         | 22490/225000 [54:39:34<364:29:44,  6.48s/it]
 10%|▉         | 22491/225000 [54:39:40<357:54:48,  6.36s/it]
 10%|▉         | 22492/225000 [54:39:50<429:14:17,  7.63s/it]
 10%|▉         | 22493/225000 [54:39:57<415:45:47,  7.39s/it]
 10%|▉         | 22494/225000 [54:40:01<365:43:28,  6.50s/it]
 10%|▉         | 22495/225000 [54:40:11<421:38:40,  7.50s/it]
 10%|▉         | 22496/225000 [54:40:15<357:56:48,  6.36s/it]
 10%|▉         | 22497/225000 [54:40:22<375:17:38,  6.67s/it]
 10%|▉         | 22498/225000 [54:40:30<384:46:54,  6.84s/it]
 10%|▉         | 22499/225000 [54:40:36<384:24:21,  6.83s/it]
 10%|█         | 22500/225000 [54:40:41<345:20:12,  6.14s/it]
                                                             
{'loss': '0.6972', 'grad_norm': '0.6904', 'learning_rate': '4.958e-05', 'epoch': '0.1', 'num_input_tokens_seen': 95266168, 'train_runtime': '1.968e+05', 'train_tokens_per_second': '484'}

 10%|█         | 22500/225000 [54:40:41<345:20:12,  6.14s/it]
 10%|█         | 22501/225000 [54:40:51<409:43:47,  7.28s/it]
 10%|█         | 22502/225000 [54:40:56<372:06:52,  6.62s/it]
 10%|█         | 22503/225000 [54:41:02<363:59:01,  6.47s/it]
 10%|█         | 22504/225000 [54:41:10<382:35:22,  6.80s/it]
 10%|█         | 22505/225000 [54:41:14<344:55:41,  6.13s/it]
 10%|█         | 22506/225000 [54:41:19<326:20:55,  5.80s/it]
 10%|█         | 22507/225000 [54:41:29<391:34:20,  6.96s/it]
 10%|█         | 22508/225000 [54:41:36<386:45:05,  6.88s/it]
 10%|█         | 22509/225000 [54:41:41<354:43:08,  6.31s/it]
 10%|█         | 22510/225000 [54:41:49<388:14:29,  6.90s/it]
 10%|█         | 22511/225000 [54:41:55<367:08:05,  6.53s/it]
 10%|█         | 22512/225000 [54:42:00<357:07:59,  6.35s/it]
 10%|█         | 22513/225000 [54:42:09<389:09:09,  6.92s/it]
 10%|█         | 22514/225000 [54:42:12<329:44:48,  5.86s/it]
 10%|█         | 22515/225000 [54:42:22<405:25:01,  7.21s/it]
 10%|█         | 22516/225000 [54:42:28<377:42:37,  6.72s/it]
 10%|█         | 22517/225000 [54:42:34<371:28:00,  6.60s/it]
 10%|█         | 22518/225000 [54:42:43<396:59:31,  7.06s/it]
 10%|█         | 22519/225000 [54:42:47<349:58:09,  6.22s/it]
 10%|█         | 22520/225000 [54:42:52<331:44:11,  5.90s/it]
 10%|█         | 22521/225000 [54:42:57<319:11:00,  5.67s/it]
 10%|█         | 22522/225000 [54:43:06<373:59:16,  6.65s/it]
 10%|█         | 22523/225000 [54:43:15<410:29:22,  7.30s/it]
 10%|█         | 22524/225000 [54:43:20<366:35:03,  6.52s/it]
 10%|█         | 22525/225000 [54:43:24<334:26:37,  5.95s/it]
 10%|█         | 22526/225000 [54:43:35<412:35:47,  7.34s/it]
 10%|█         | 22527/225000 [54:43:41<398:08:17,  7.08s/it]
 10%|█         | 22528/225000 [54:43:47<373:57:04,  6.65s/it]
 10%|█         | 22529/225000 [54:43:56<409:26:34,  7.28s/it]
 10%|█         | 22530/225000 [54:44:00<357:03:55,  6.35s/it]
 10%|█         | 22531/225000 [54:44:09<414:10:17,  7.36s/it]
 10%|█         | 22532/225000 [54:44:16<407:34:32,  7.25s/it]
 10%|█         | 22533/225000 [54:44:20<349:11:02,  6.21s/it]
 10%|█         | 22534/225000 [54:44:29<396:00:54,  7.04s/it]
 10%|█         | 22535/225000 [54:44:36<393:29:42,  7.00s/it]
 10%|█         | 22536/225000 [54:44:40<344:47:17,  6.13s/it]
 10%|█         | 22537/225000 [54:44:51<418:16:43,  7.44s/it]
 10%|█         | 22538/225000 [54:44:56<384:29:48,  6.84s/it]
 10%|█         | 22539/225000 [54:45:02<375:08:50,  6.67s/it]
 10%|█         | 22540/225000 [54:45:08<357:39:01,  6.36s/it]
 10%|█         | 22541/225000 [54:45:18<421:01:21,  7.49s/it]
 10%|█         | 22542/225000 [54:45:23<373:47:28,  6.65s/it]
 10%|█         | 22543/225000 [54:45:27<336:15:31,  5.98s/it]
 10%|█         | 22544/225000 [54:45:33<331:22:59,  5.89s/it]
 10%|█         | 22545/225000 [54:45:44<414:58:18,  7.38s/it]
 10%|█         | 22546/225000 [54:45:48<363:05:12,  6.46s/it]
 10%|█         | 22547/225000 [54:45:53<340:01:05,  6.05s/it]
 10%|█         | 22548/225000 [54:45:59<334:43:35,  5.95s/it]
 10%|█         | 22549/225000 [54:46:05<328:57:45,  5.85s/it]
 10%|█         | 22550/225000 [54:46:11<330:28:50,  5.88s/it]
                                                             
{'loss': '0.7227', 'grad_norm': '0.6427', 'learning_rate': '4.958e-05', 'epoch': '0.1002', 'num_input_tokens_seen': 95418040, 'train_runtime': '1.972e+05', 'train_tokens_per_second': '483.9'}

 10%|█         | 22550/225000 [54:46:11<330:28:50,  5.88s/it]
 10%|█         | 22551/225000 [54:46:16<324:34:51,  5.77s/it]
 10%|█         | 22552/225000 [54:46:22<321:49:56,  5.72s/it]
 10%|█         | 22553/225000 [54:46:31<378:31:06,  6.73s/it]
 10%|█         | 22554/225000 [54:46:44<496:14:12,  8.82s/it]
 10%|█         | 22555/225000 [54:46:49<426:39:38,  7.59s/it]
 10%|█         | 22556/225000 [54:46:54<384:36:42,  6.84s/it]
 10%|█         | 22557/225000 [54:47:00<360:40:37,  6.41s/it]
 10%|█         | 22558/225000 [54:47:13<478:32:31,  8.51s/it]
 10%|█         | 22559/225000 [54:47:17<396:41:21,  7.05s/it]
 10%|█         | 22560/225000 [54:47:27<446:32:59,  7.94s/it]
 10%|█         | 22561/225000 [54:47:31<388:46:30,  6.91s/it]
 10%|█         | 22562/225000 [54:47:35<341:39:10,  6.08s/it]
 10%|█         | 22563/225000 [54:47:47<428:30:57,  7.62s/it]
 10%|█         | 22564/225000 [54:47:52<387:38:00,  6.89s/it]
 10%|█         | 22565/225000 [54:48:00<410:18:07,  7.30s/it]
 10%|█         | 22566/225000 [54:48:04<362:39:53,  6.45s/it]
 10%|█         | 22567/225000 [54:48:09<328:57:02,  5.85s/it]
 10%|█         | 22568/225000 [54:48:13<305:31:34,  5.43s/it]
 10%|█         | 22569/225000 [54:48:26<418:14:40,  7.44s/it]
 10%|█         | 22570/225000 [54:48:30<370:43:33,  6.59s/it]
 10%|█         | 22571/225000 [54:48:38<400:19:12,  7.12s/it]
 10%|█         | 22572/225000 [54:48:44<379:00:32,  6.74s/it]
 10%|█         | 22573/225000 [54:48:49<341:41:57,  6.08s/it]
 10%|█         | 22574/225000 [54:48:54<318:53:19,  5.67s/it]
 10%|█         | 22575/225000 [54:49:07<445:06:24,  7.92s/it]
 10%|█         | 22576/225000 [54:49:11<377:15:25,  6.71s/it]
 10%|█         | 22577/225000 [54:49:20<414:49:19,  7.38s/it]
 10%|█         | 22578/225000 [54:49:25<375:28:55,  6.68s/it]
 10%|█         | 22579/225000 [54:49:31<368:30:56,  6.55s/it]
 10%|█         | 22580/225000 [54:49:39<399:24:31,  7.10s/it]
 10%|█         | 22581/225000 [54:49:43<349:25:12,  6.21s/it]
 10%|█         | 22582/225000 [54:49:53<404:25:19,  7.19s/it]
 10%|█         | 22583/225000 [54:49:58<371:18:53,  6.60s/it]
 10%|█         | 22584/225000 [54:50:03<344:01:37,  6.12s/it]
 10%|█         | 22585/225000 [54:50:08<323:36:16,  5.76s/it]
 10%|█         | 22586/225000 [54:50:13<318:00:01,  5.66s/it]
 10%|█         | 22587/225000 [54:50:19<313:40:07,  5.58s/it]
 10%|█         | 22588/225000 [54:50:30<406:23:06,  7.23s/it]
 10%|█         | 22589/225000 [54:50:39<435:48:51,  7.75s/it]
 10%|█         | 22590/225000 [54:50:43<371:54:13,  6.61s/it]
 10%|█         | 22591/225000 [54:50:48<340:13:56,  6.05s/it]
 10%|█         | 22592/225000 [54:50:53<322:28:31,  5.74s/it]
 10%|█         | 22593/225000 [54:50:58<319:45:09,  5.69s/it]
 10%|█         | 22594/225000 [54:51:06<356:08:37,  6.33s/it]
 10%|█         | 22595/225000 [54:51:18<450:24:47,  8.01s/it]
 10%|█         | 22596/225000 [54:51:26<454:32:26,  8.08s/it]
 10%|█         | 22597/225000 [54:51:30<374:43:11,  6.66s/it]
 10%|█         | 22598/225000 [54:51:39<423:10:37,  7.53s/it]
 10%|█         | 22599/225000 [54:51:44<372:43:24,  6.63s/it]
 10%|█         | 22600/225000 [54:51:49<355:07:34,  6.32s/it]
                                                             
{'loss': '0.7082', 'grad_norm': '0.5854', 'learning_rate': '4.958e-05', 'epoch': '0.1004', 'num_input_tokens_seen': 95570008, 'train_runtime': '1.975e+05', 'train_tokens_per_second': '483.9'}

 10%|█         | 22600/225000 [54:51:49<355:07:34,  6.32s/it]
 10%|█         | 22601/225000 [54:51:56<367:03:26,  6.53s/it]
 10%|█         | 22602/225000 [54:52:01<337:03:36,  6.00s/it]
 10%|█         | 22603/225000 [54:52:06<315:02:43,  5.60s/it]
 10%|█         | 22604/225000 [54:52:17<419:48:23,  7.47s/it]
 10%|█         | 22605/225000 [54:52:24<410:25:42,  7.30s/it]
 10%|█         | 22606/225000 [54:52:32<410:25:51,  7.30s/it]
 10%|█         | 22607/225000 [54:52:38<396:47:42,  7.06s/it]
 10%|█         | 22608/225000 [54:52:42<348:45:55,  6.20s/it]
 10%|█         | 22609/225000 [54:52:52<414:20:45,  7.37s/it]
 10%|█         | 22610/225000 [54:52:59<393:52:46,  7.01s/it]
 10%|█         | 22611/225000 [54:53:06<403:51:49,  7.18s/it]
 10%|█         | 22612/225000 [54:53:11<358:32:50,  6.38s/it]
 10%|█         | 22613/225000 [54:53:15<327:46:54,  5.83s/it]
 10%|█         | 22614/225000 [54:53:20<313:57:51,  5.58s/it]
 10%|█         | 22615/225000 [54:53:25<302:35:09,  5.38s/it]
 10%|█         | 22616/225000 [54:53:38<423:46:58,  7.54s/it]
 10%|█         | 22617/225000 [54:53:44<404:48:52,  7.20s/it]
 10%|█         | 22618/225000 [54:53:50<375:00:10,  6.67s/it]
 10%|█         | 22619/225000 [54:53:55<349:36:04,  6.22s/it]
 10%|█         | 22620/225000 [54:54:00<335:47:12,  5.97s/it]
 10%|█         | 22621/225000 [54:54:05<324:05:42,  5.77s/it]
 10%|█         | 22622/225000 [54:54:15<381:09:35,  6.78s/it]
 10%|█         | 22623/225000 [54:54:27<476:32:23,  8.48s/it]
 10%|█         | 22624/225000 [54:54:31<403:34:53,  7.18s/it]
 10%|█         | 22625/225000 [54:54:37<384:01:41,  6.83s/it]
 10%|█         | 22626/225000 [54:54:45<399:38:42,  7.11s/it]
 10%|█         | 22627/225000 [54:54:50<366:00:07,  6.51s/it]
 10%|█         | 22628/225000 [54:54:56<351:49:29,  6.26s/it]
 10%|█         | 22629/225000 [54:55:01<340:11:33,  6.05s/it]
 10%|█         | 22630/225000 [54:55:11<409:10:51,  7.28s/it]
 10%|█         | 22631/225000 [54:55:19<413:13:10,  7.35s/it]
 10%|█         | 22632/225000 [54:55:23<358:23:53,  6.38s/it]
 10%|█         | 22633/225000 [54:55:33<416:15:16,  7.40s/it]
 10%|█         | 22634/225000 [54:55:39<387:11:59,  6.89s/it]
 10%|█         | 22635/225000 [54:55:45<386:42:48,  6.88s/it]
 10%|█         | 22636/225000 [54:55:50<348:22:54,  6.20s/it]
 10%|█         | 22637/225000 [54:55:56<344:06:00,  6.12s/it]
 10%|█         | 22638/225000 [54:56:07<420:03:40,  7.47s/it]
 10%|█         | 22639/225000 [54:56:14<410:24:44,  7.30s/it]
 10%|█         | 22640/225000 [54:56:20<391:41:49,  6.97s/it]
 10%|█         | 22641/225000 [54:56:25<365:30:01,  6.50s/it]
 10%|█         | 22642/225000 [54:56:30<344:24:58,  6.13s/it]
 10%|█         | 22643/225000 [54:56:36<339:56:52,  6.05s/it]
 10%|█         | 22644/225000 [54:56:42<329:48:43,  5.87s/it]
 10%|█         | 22645/225000 [54:56:49<349:56:35,  6.23s/it]
 10%|█         | 22646/225000 [54:57:00<438:52:19,  7.81s/it]
 10%|█         | 22647/225000 [54:57:05<390:05:20,  6.94s/it]
 10%|█         | 22648/225000 [54:57:10<353:53:36,  6.30s/it]
 10%|█         | 22649/225000 [54:57:15<333:10:46,  5.93s/it]
 10%|█         | 22650/225000 [54:57:25<396:34:58,  7.06s/it]
                                                             
{'loss': '0.691', 'grad_norm': '0.6099', 'learning_rate': '4.957e-05', 'epoch': '0.1007', 'num_input_tokens_seen': 95721696, 'train_runtime': '1.978e+05', 'train_tokens_per_second': '483.8'}

 10%|█         | 22650/225000 [54:57:25<396:34:58,  7.06s/it]
 10%|█         | 22651/225000 [54:57:33<409:47:39,  7.29s/it]
 10%|█         | 22652/225000 [54:57:41<436:18:53,  7.76s/it]
 10%|█         | 22653/225000 [54:57:45<373:35:02,  6.65s/it]
 10%|█         | 22654/225000 [54:57:54<402:01:43,  7.15s/it]
 10%|█         | 22655/225000 [54:58:00<378:52:45,  6.74s/it]
 10%|█         | 22656/225000 [54:58:07<392:26:08,  6.98s/it]
 10%|█         | 22657/225000 [54:58:11<347:28:21,  6.18s/it]
 10%|█         | 22658/225000 [54:58:22<428:51:13,  7.63s/it]
 10%|█         | 22659/225000 [54:58:29<419:04:17,  7.46s/it]
 10%|█         | 22660/225000 [54:58:34<365:11:16,  6.50s/it]
 10%|█         | 22661/225000 [54:58:40<364:35:21,  6.49s/it]
 10%|█         | 22662/225000 [54:58:49<395:19:40,  7.03s/it]
 10%|█         | 22663/225000 [54:58:56<401:31:46,  7.14s/it]
 10%|█         | 22664/225000 [54:59:02<378:29:13,  6.73s/it]
 10%|█         | 22665/225000 [54:59:07<353:44:20,  6.29s/it]
 10%|█         | 22666/225000 [54:59:17<418:04:53,  7.44s/it]
 10%|█         | 22667/225000 [54:59:22<372:52:18,  6.63s/it]
 10%|█         | 22668/225000 [54:59:29<385:59:02,  6.87s/it]
 10%|█         | 22669/225000 [54:59:33<335:08:49,  5.96s/it]
 10%|█         | 22670/225000 [54:59:42<390:45:00,  6.95s/it]
 10%|█         | 22671/225000 [54:59:46<340:02:39,  6.05s/it]
 10%|█         | 22672/225000 [54:59:56<399:03:04,  7.10s/it]
 10%|█         | 22673/225000 [55:00:04<416:44:57,  7.42s/it]
 10%|█         | 22674/225000 [55:00:10<392:19:50,  6.98s/it]
 10%|█         | 22675/225000 [55:00:16<369:23:04,  6.57s/it]
 10%|█         | 22676/225000 [55:00:21<357:36:46,  6.36s/it]
 10%|█         | 22677/225000 [55:00:26<331:01:27,  5.89s/it]
 10%|█         | 22678/225000 [55:00:31<313:22:31,  5.58s/it]
 10%|█         | 22679/225000 [55:00:36<297:59:53,  5.30s/it]
 10%|█         | 22680/225000 [55:00:48<419:01:56,  7.46s/it]
 10%|█         | 22681/225000 [55:01:00<485:36:46,  8.64s/it]
 10%|█         | 22682/225000 [55:01:06<440:30:49,  7.84s/it]
 10%|█         | 22683/225000 [55:01:12<415:48:37,  7.40s/it]
 10%|█         | 22684/225000 [55:01:16<355:20:10,  6.32s/it]
 10%|█         | 22685/225000 [55:01:25<403:03:48,  7.17s/it]
 10%|█         | 22686/225000 [55:01:29<342:51:00,  6.10s/it]
 10%|█         | 22687/225000 [55:01:39<418:20:12,  7.44s/it]
 10%|█         | 22688/225000 [55:01:44<375:49:43,  6.69s/it]
 10%|█         | 22689/225000 [55:01:50<367:34:26,  6.54s/it]
 10%|█         | 22690/225000 [55:01:54<318:10:50,  5.66s/it]
 10%|█         | 22691/225000 [55:02:04<398:22:12,  7.09s/it]
 10%|█         | 22692/225000 [55:02:08<343:08:22,  6.11s/it]
 10%|█         | 22693/225000 [55:02:13<318:23:55,  5.67s/it]
 10%|█         | 22694/225000 [55:02:21<360:37:03,  6.42s/it]
 10%|█         | 22695/225000 [55:02:32<439:26:30,  7.82s/it]
 10%|█         | 22696/225000 [55:02:40<439:16:54,  7.82s/it]
 10%|█         | 22697/225000 [55:02:44<379:53:46,  6.76s/it]
 10%|█         | 22698/225000 [55:02:54<430:14:56,  7.66s/it]
 10%|█         | 22699/225000 [55:02:58<371:03:52,  6.60s/it]
 10%|█         | 22700/225000 [55:03:02<327:29:41,  5.83s/it]
                                                             
{'loss': '0.7348', 'grad_norm': '0.7417', 'learning_rate': '4.957e-05', 'epoch': '0.1009', 'num_input_tokens_seen': 95872584, 'train_runtime': '1.982e+05', 'train_tokens_per_second': '483.8'}

 10%|█         | 22700/225000 [55:03:02<327:29:41,  5.83s/it]
 10%|█         | 22701/225000 [55:03:13<416:45:22,  7.42s/it]
 10%|█         | 22702/225000 [55:03:18<368:06:21,  6.55s/it]
 10%|█         | 22703/225000 [55:03:26<390:49:00,  6.95s/it]
 10%|█         | 22704/225000 [55:03:33<394:52:53,  7.03s/it]
 10%|█         | 22705/225000 [55:03:37<353:52:08,  6.30s/it]
 10%|█         | 22706/225000 [55:03:45<372:01:08,  6.62s/it]
 10%|█         | 22707/225000 [55:03:51<362:34:03,  6.45s/it]
 10%|█         | 22708/225000 [55:03:57<365:20:20,  6.50s/it]
 10%|█         | 22709/225000 [55:04:01<321:59:16,  5.73s/it]
 10%|█         | 22710/225000 [55:04:11<382:12:05,  6.80s/it]
 10%|█         | 22711/225000 [55:04:19<407:52:37,  7.26s/it]
 10%|█         | 22712/225000 [55:04:23<360:12:13,  6.41s/it]
 10%|█         | 22713/225000 [55:04:28<325:50:10,  5.80s/it]
 10%|█         | 22714/225000 [55:04:38<399:31:48,  7.11s/it]
 10%|█         | 22715/225000 [55:04:43<370:21:24,  6.59s/it]
 10%|█         | 22716/225000 [55:04:49<347:58:40,  6.19s/it]
 10%|█         | 22717/225000 [55:04:54<334:37:46,  5.96s/it]
 10%|█         | 22718/225000 [55:05:00<334:39:39,  5.96s/it]
 10%|█         | 22719/225000 [55:05:05<327:49:20,  5.83s/it]
 10%|█         | 22720/225000 [55:05:11<321:49:53,  5.73s/it]
 10%|█         | 22721/225000 [55:05:24<446:42:05,  7.95s/it]
 10%|█         | 22722/225000 [55:05:32<441:43:47,  7.86s/it]
 10%|█         | 22723/225000 [55:05:39<435:06:33,  7.74s/it]
 10%|█         | 22724/225000 [55:05:43<366:40:22,  6.53s/it]
 10%|█         | 22725/225000 [55:05:51<390:16:50,  6.95s/it]
 10%|█         | 22726/225000 [55:05:56<352:53:20,  6.28s/it]
 10%|█         | 22727/225000 [55:06:04<396:12:32,  7.05s/it]
 10%|█         | 22728/225000 [55:06:12<406:36:43,  7.24s/it]
 10%|█         | 22729/225000 [55:06:21<429:11:07,  7.64s/it]
 10%|█         | 22730/225000 [55:06:24<364:34:14,  6.49s/it]
 10%|█         | 22731/225000 [55:06:35<433:07:19,  7.71s/it]
 10%|█         | 22732/225000 [55:06:42<414:04:27,  7.37s/it]
 10%|█         | 22733/225000 [55:06:46<372:00:25,  6.62s/it]
 10%|█         | 22734/225000 [55:06:51<342:41:34,  6.10s/it]
 10%|█         | 22735/225000 [55:06:56<323:21:46,  5.76s/it]
 10%|█         | 22736/225000 [55:07:01<310:47:29,  5.53s/it]
 10%|█         | 22737/225000 [55:07:13<412:32:12,  7.34s/it]
 10%|█         | 22738/225000 [55:07:17<364:04:30,  6.48s/it]
 10%|█         | 22739/225000 [55:07:23<349:48:56,  6.23s/it]
 10%|█         | 22740/225000 [55:07:30<369:56:53,  6.58s/it]
 10%|█         | 22741/225000 [55:07:36<349:29:56,  6.22s/it]
 10%|█         | 22742/225000 [55:07:43<371:17:47,  6.61s/it]
 10%|█         | 22743/225000 [55:07:50<366:53:05,  6.53s/it]
 10%|█         | 22744/225000 [55:07:55<351:21:02,  6.25s/it]
 10%|█         | 22745/225000 [55:08:01<340:59:40,  6.07s/it]
 10%|█         | 22746/225000 [55:08:08<363:48:38,  6.48s/it]
 10%|█         | 22747/225000 [55:08:18<409:49:15,  7.29s/it]
 10%|█         | 22748/225000 [55:08:25<415:36:31,  7.40s/it]
 10%|█         | 22749/225000 [55:08:30<365:53:24,  6.51s/it]
 10%|█         | 22750/225000 [55:08:35<339:15:28,  6.04s/it]
                                                             
{'loss': '0.7206', 'grad_norm': '0.7434', 'learning_rate': '4.957e-05', 'epoch': '0.1011', 'num_input_tokens_seen': 96021976, 'train_runtime': '1.985e+05', 'train_tokens_per_second': '483.7'}

 10%|█         | 22750/225000 [55:08:35<339:15:28,  6.04s/it]
 10%|█         | 22751/225000 [55:08:40<327:27:48,  5.83s/it]
 10%|█         | 22752/225000 [55:08:53<451:23:43,  8.03s/it]
 10%|█         | 22753/225000 [55:08:57<373:58:46,  6.66s/it]
 10%|█         | 22754/225000 [55:09:04<384:35:23,  6.85s/it]
 10%|█         | 22755/225000 [55:09:08<343:37:47,  6.12s/it]
 10%|█         | 22756/225000 [55:09:19<424:12:41,  7.55s/it]
 10%|█         | 22757/225000 [55:09:24<375:31:20,  6.68s/it]
 10%|█         | 22758/225000 [55:09:28<342:28:30,  6.10s/it]
 10%|█         | 22759/225000 [55:09:38<402:29:51,  7.16s/it]
 10%|█         | 22760/225000 [55:09:43<370:36:06,  6.60s/it]
 10%|█         | 22761/225000 [55:09:47<325:04:36,  5.79s/it]
 10%|█         | 22762/225000 [55:09:58<410:42:43,  7.31s/it]
 10%|█         | 22763/225000 [55:10:03<375:36:40,  6.69s/it]
 10%|█         | 22764/225000 [55:10:08<345:47:22,  6.16s/it]
 10%|█         | 22765/225000 [55:10:13<319:17:50,  5.68s/it]
 10%|█         | 22766/225000 [55:10:26<445:43:24,  7.93s/it]
 10%|█         | 22767/225000 [55:10:33<422:16:27,  7.52s/it]
 10%|█         | 22768/225000 [55:10:36<359:32:09,  6.40s/it]
 10%|█         | 22769/225000 [55:10:44<379:22:22,  6.75s/it]
 10%|█         | 22770/225000 [55:10:52<395:49:55,  7.05s/it]
 10%|█         | 22771/225000 [55:10:57<370:52:17,  6.60s/it]
 10%|█         | 22772/225000 [55:11:07<425:30:52,  7.57s/it]
 10%|█         | 22773/225000 [55:11:12<371:40:43,  6.62s/it]
 10%|█         | 22774/225000 [55:11:16<330:28:58,  5.88s/it]
 10%|█         | 22775/225000 [55:11:28<437:34:12,  7.79s/it]
 10%|█         | 22776/225000 [55:11:33<383:42:43,  6.83s/it]
 10%|█         | 22777/225000 [55:11:37<348:01:17,  6.20s/it]
 10%|█         | 22778/225000 [55:11:46<395:50:00,  7.05s/it]
 10%|█         | 22779/225000 [55:11:52<367:44:10,  6.55s/it]
 10%|█         | 22780/225000 [55:11:57<348:30:29,  6.20s/it]
 10%|█         | 22781/225000 [55:12:02<333:27:02,  5.94s/it]
 10%|█         | 22782/225000 [55:12:08<329:50:12,  5.87s/it]
 10%|█         | 22783/225000 [55:12:14<325:32:08,  5.80s/it]
 10%|█         | 22784/225000 [55:12:20<325:49:21,  5.80s/it]
 10%|█         | 22785/225000 [55:12:25<324:04:29,  5.77s/it]
 10%|█         | 22786/225000 [55:12:32<348:41:27,  6.21s/it]
 10%|█         | 22787/225000 [55:12:47<485:05:05,  8.64s/it]
 10%|█         | 22788/225000 [55:12:51<410:19:52,  7.31s/it]
 10%|█         | 22789/225000 [55:12:56<380:10:47,  6.77s/it]
 10%|█         | 22790/225000 [55:13:02<357:21:45,  6.36s/it]
 10%|█         | 22791/225000 [55:13:08<347:04:38,  6.18s/it]
 10%|█         | 22792/225000 [55:13:14<341:44:52,  6.08s/it]
 10%|█         | 22793/225000 [55:13:19<331:40:27,  5.90s/it]
 10%|█         | 22794/225000 [55:13:33<466:07:18,  8.30s/it]
 10%|█         | 22795/225000 [55:13:39<429:46:30,  7.65s/it]
 10%|█         | 22796/225000 [55:13:47<443:22:30,  7.89s/it]
 10%|█         | 22797/225000 [55:13:52<384:24:31,  6.84s/it]
 10%|█         | 22798/225000 [55:13:57<363:24:07,  6.47s/it]
 10%|█         | 22799/225000 [55:14:02<336:31:27,  5.99s/it]
 10%|█         | 22800/225000 [55:14:07<319:06:30,  5.68s/it]
                                                             
{'loss': '0.7091', 'grad_norm': '0.5752', 'learning_rate': '4.956e-05', 'epoch': '0.1013', 'num_input_tokens_seen': 96174240, 'train_runtime': '1.988e+05', 'train_tokens_per_second': '483.7'}

 10%|█         | 22800/225000 [55:14:07<319:06:30,  5.68s/it]
 10%|█         | 22801/225000 [55:14:13<318:09:35,  5.66s/it]
 10%|█         | 22802/225000 [55:14:27<454:32:42,  8.09s/it]
 10%|█         | 22803/225000 [55:14:31<397:47:44,  7.08s/it]
 10%|█         | 22804/225000 [55:14:39<409:54:46,  7.30s/it]
 10%|█         | 22805/225000 [55:14:46<407:32:12,  7.26s/it]
 10%|█         | 22806/225000 [55:14:55<423:05:47,  7.53s/it]
 10%|█         | 22807/225000 [55:14:59<367:20:48,  6.54s/it]
 10%|█         | 22808/225000 [55:15:03<326:35:57,  5.82s/it]
 10%|█         | 22809/225000 [55:15:08<317:09:27,  5.65s/it]
 10%|█         | 22810/225000 [55:15:14<322:12:24,  5.74s/it]
 10%|█         | 22811/225000 [55:15:21<343:00:07,  6.11s/it]
 10%|█         | 22812/225000 [55:15:34<453:20:39,  8.07s/it]
 10%|█         | 22813/225000 [55:15:38<393:46:13,  7.01s/it]
 10%|█         | 22814/225000 [55:15:48<443:48:02,  7.90s/it]
 10%|█         | 22815/225000 [55:15:54<399:31:18,  7.11s/it]
 10%|█         | 22816/225000 [55:15:57<345:44:15,  6.16s/it]
 10%|█         | 22817/225000 [55:16:07<402:19:50,  7.16s/it]
 10%|█         | 22818/225000 [55:16:12<369:00:04,  6.57s/it]
 10%|█         | 22819/225000 [55:16:17<347:16:49,  6.18s/it]
 10%|█         | 22820/225000 [55:16:23<328:36:56,  5.85s/it]
 10%|█         | 22821/225000 [55:16:28<315:40:45,  5.62s/it]
 10%|█         | 22822/225000 [55:16:37<380:00:01,  6.77s/it]
 10%|█         | 22823/225000 [55:16:47<432:30:05,  7.70s/it]
 10%|█         | 22824/225000 [55:16:51<365:52:18,  6.51s/it]
 10%|█         | 22825/225000 [55:17:00<419:31:45,  7.47s/it]
 10%|█         | 22826/225000 [55:17:07<400:31:42,  7.13s/it]
 10%|█         | 22827/225000 [55:17:11<353:30:02,  6.29s/it]
 10%|█         | 22828/225000 [55:17:19<376:12:32,  6.70s/it]
 10%|█         | 22829/225000 [55:17:27<405:47:25,  7.23s/it]
 10%|█         | 22830/225000 [55:17:32<373:29:32,  6.65s/it]
 10%|█         | 22831/225000 [55:17:38<350:47:19,  6.25s/it]
 10%|█         | 22832/225000 [55:17:43<340:00:06,  6.05s/it]
 10%|█         | 22833/225000 [55:17:51<371:03:06,  6.61s/it]
 10%|█         | 22834/225000 [55:17:57<364:38:22,  6.49s/it]
 10%|█         | 22835/225000 [55:18:03<343:18:23,  6.11s/it]
 10%|█         | 22836/225000 [55:18:10<360:13:29,  6.41s/it]
 10%|█         | 22837/225000 [55:18:21<443:27:40,  7.90s/it]
 10%|█         | 22838/225000 [55:18:25<378:14:19,  6.74s/it]
 10%|█         | 22839/225000 [55:18:32<383:41:49,  6.83s/it]
 10%|█         | 22840/225000 [55:18:41<408:27:51,  7.27s/it]
 10%|█         | 22841/225000 [55:18:48<405:20:12,  7.22s/it]
 10%|█         | 22842/225000 [55:18:52<351:20:30,  6.26s/it]
 10%|█         | 22843/225000 [55:18:57<342:53:13,  6.11s/it]
 10%|█         | 22844/225000 [55:19:06<390:39:24,  6.96s/it]
 10%|█         | 22845/225000 [55:19:12<368:35:48,  6.56s/it]
 10%|█         | 22846/225000 [55:19:22<419:44:58,  7.47s/it]
 10%|█         | 22847/225000 [55:19:25<354:30:54,  6.31s/it]
 10%|█         | 22848/225000 [55:19:33<384:27:11,  6.85s/it]
 10%|█         | 22849/225000 [55:19:40<385:01:58,  6.86s/it]
 10%|█         | 22850/225000 [55:19:46<364:50:23,  6.50s/it]
                                                             
{'loss': '0.709', 'grad_norm': '0.6381', 'learning_rate': '4.956e-05', 'epoch': '0.1016', 'num_input_tokens_seen': 96324488, 'train_runtime': '1.992e+05', 'train_tokens_per_second': '483.6'}

 10%|█         | 22850/225000 [55:19:46<364:50:23,  6.50s/it]
 10%|█         | 22851/225000 [55:19:54<393:57:44,  7.02s/it]
 10%|█         | 22852/225000 [55:19:59<354:32:51,  6.31s/it]
 10%|█         | 22853/225000 [55:20:13<484:03:48,  8.62s/it]
 10%|█         | 22854/225000 [55:20:17<406:37:32,  7.24s/it]
 10%|█         | 22855/225000 [55:20:25<420:18:23,  7.49s/it]
 10%|█         | 22856/225000 [55:20:30<374:01:26,  6.66s/it]
 10%|█         | 22857/225000 [55:20:35<347:02:21,  6.18s/it]
 10%|█         | 22858/225000 [55:20:40<327:52:45,  5.84s/it]
 10%|█         | 22859/225000 [55:20:45<319:56:23,  5.70s/it]
 10%|█         | 22860/225000 [55:20:50<311:04:44,  5.54s/it]
 10%|█         | 22861/225000 [55:21:04<451:00:10,  8.03s/it]
 10%|█         | 22862/225000 [55:21:10<412:05:11,  7.34s/it]
 10%|█         | 22863/225000 [55:21:15<378:38:12,  6.74s/it]
 10%|█         | 22864/225000 [55:21:21<357:02:12,  6.36s/it]
 10%|█         | 22865/225000 [55:21:26<338:00:25,  6.02s/it]
 10%|█         | 22866/225000 [55:21:31<323:58:47,  5.77s/it]
 10%|█         | 22867/225000 [55:21:37<324:57:11,  5.79s/it]
 10%|█         | 22868/225000 [55:21:50<450:41:18,  8.03s/it]
 10%|█         | 22869/225000 [55:21:57<431:50:27,  7.69s/it]
 10%|█         | 22870/225000 [55:22:06<452:04:38,  8.05s/it]
 10%|█         | 22871/225000 [55:22:11<403:26:03,  7.19s/it]
 10%|█         | 22872/225000 [55:22:18<405:51:29,  7.23s/it]
 10%|█         | 22873/225000 [55:22:23<359:10:52,  6.40s/it]
 10%|█         | 22874/225000 [55:22:27<327:30:29,  5.83s/it]
 10%|█         | 22875/225000 [55:22:38<414:43:53,  7.39s/it]
 10%|█         | 22876/225000 [55:22:42<357:34:05,  6.37s/it]
 10%|█         | 22877/225000 [55:22:47<328:31:32,  5.85s/it]
 10%|█         | 22878/225000 [55:22:52<313:05:01,  5.58s/it]
 10%|█         | 22879/225000 [55:23:05<440:18:07,  7.84s/it]
 10%|█         | 22880/225000 [55:23:10<396:05:07,  7.05s/it]
 10%|█         | 22881/225000 [55:23:15<360:19:23,  6.42s/it]
 10%|█         | 22882/225000 [55:23:20<332:39:57,  5.93s/it]
 10%|█         | 22883/225000 [55:23:29<392:40:25,  6.99s/it]
 10%|█         | 22884/225000 [55:23:36<389:41:35,  6.94s/it]
 10%|█         | 22885/225000 [55:23:44<400:45:34,  7.14s/it]
 10%|█         | 22886/225000 [55:23:52<409:40:30,  7.30s/it]
 10%|█         | 22887/225000 [55:23:58<402:48:18,  7.17s/it]
 10%|█         | 22888/225000 [55:24:05<396:10:14,  7.06s/it]
 10%|█         | 22889/225000 [55:24:15<436:07:42,  7.77s/it]
 10%|█         | 22890/225000 [55:24:21<409:02:54,  7.29s/it]
 10%|█         | 22891/225000 [55:24:25<350:20:58,  6.24s/it]
 10%|█         | 22892/225000 [55:24:34<405:19:45,  7.22s/it]
 10%|█         | 22893/225000 [55:24:38<354:13:35,  6.31s/it]
 10%|█         | 22894/225000 [55:24:47<393:33:35,  7.01s/it]
 10%|█         | 22895/225000 [55:24:53<375:09:00,  6.68s/it]
 10%|█         | 22896/225000 [55:24:58<350:58:24,  6.25s/it]
 10%|█         | 22897/225000 [55:25:04<342:53:23,  6.11s/it]
 10%|█         | 22898/225000 [55:25:09<327:24:20,  5.83s/it]
 10%|█         | 22899/225000 [55:25:14<318:37:08,  5.68s/it]
 10%|█         | 22900/225000 [55:25:22<349:55:50,  6.23s/it]
                                                             
{'loss': '0.7325', 'grad_norm': '0.5987', 'learning_rate': '4.956e-05', 'epoch': '0.1018', 'num_input_tokens_seen': 96475808, 'train_runtime': '1.995e+05', 'train_tokens_per_second': '483.5'}

 10%|█         | 22900/225000 [55:25:22<349:55:50,  6.23s/it]
 10%|█         | 22901/225000 [55:25:33<438:26:51,  7.81s/it]
 10%|█         | 22902/225000 [55:25:39<393:47:31,  7.01s/it]
 10%|█         | 22903/225000 [55:25:45<385:05:49,  6.86s/it]
 10%|█         | 22904/225000 [55:25:53<399:32:58,  7.12s/it]
 10%|█         | 22905/225000 [55:25:58<366:34:38,  6.53s/it]
 10%|█         | 22906/225000 [55:26:02<331:42:25,  5.91s/it]
 10%|█         | 22907/225000 [55:26:11<371:56:00,  6.63s/it]
 10%|█         | 22908/225000 [55:26:20<415:17:23,  7.40s/it]
 10%|█         | 22909/225000 [55:26:27<414:17:47,  7.38s/it]
 10%|█         | 22910/225000 [55:26:33<391:23:56,  6.97s/it]
 10%|█         | 22911/225000 [55:26:38<357:50:25,  6.37s/it]
 10%|█         | 22912/225000 [55:26:44<339:57:51,  6.06s/it]
 10%|█         | 22913/225000 [55:26:49<326:37:09,  5.82s/it]
 10%|█         | 22914/225000 [55:27:01<435:46:09,  7.76s/it]
 10%|█         | 22915/225000 [55:27:05<370:40:49,  6.60s/it]
 10%|█         | 22916/225000 [55:27:14<413:44:04,  7.37s/it]
 10%|█         | 22917/225000 [55:27:18<359:07:23,  6.40s/it]
 10%|█         | 22918/225000 [55:27:24<342:18:07,  6.10s/it]
 10%|█         | 22919/225000 [55:27:29<323:56:10,  5.77s/it]
 10%|█         | 22920/225000 [55:27:40<414:17:31,  7.38s/it]
 10%|█         | 22921/225000 [55:27:45<374:55:20,  6.68s/it]
 10%|█         | 22922/225000 [55:27:53<407:10:17,  7.25s/it]
 10%|█         | 22923/225000 [55:27:59<373:26:32,  6.65s/it]
 10%|█         | 22924/225000 [55:28:04<349:23:14,  6.22s/it]
 10%|█         | 22925/225000 [55:28:10<339:07:10,  6.04s/it]
 10%|█         | 22926/225000 [55:28:15<329:21:16,  5.87s/it]
 10%|█         | 22927/225000 [55:28:20<319:20:30,  5.69s/it]
 10%|█         | 22928/225000 [55:28:33<441:00:41,  7.86s/it]
 10%|█         | 22929/225000 [55:28:39<397:40:21,  7.08s/it]
 10%|█         | 22930/225000 [55:28:43<357:56:19,  6.38s/it]
 10%|█         | 22931/225000 [55:28:54<428:18:09,  7.63s/it]
 10%|█         | 22932/225000 [55:28:58<370:17:04,  6.60s/it]
 10%|█         | 22933/225000 [55:29:02<329:50:49,  5.88s/it]
 10%|█         | 22934/225000 [55:29:11<380:45:55,  6.78s/it]
 10%|█         | 22935/225000 [55:29:18<380:21:28,  6.78s/it]
 10%|█         | 22936/225000 [55:29:27<415:37:40,  7.40s/it]
 10%|█         | 22937/225000 [55:29:31<356:20:46,  6.35s/it]
 10%|█         | 22938/225000 [55:29:36<336:40:13,  6.00s/it]
 10%|█         | 22939/225000 [55:29:46<411:39:33,  7.33s/it]
 10%|█         | 22940/225000 [55:29:51<363:22:47,  6.47s/it]
 10%|█         | 22941/225000 [55:29:55<324:28:17,  5.78s/it]
 10%|█         | 22942/225000 [55:30:05<389:48:09,  6.94s/it]
 10%|█         | 22943/225000 [55:30:09<353:06:42,  6.29s/it]
 10%|█         | 22944/225000 [55:30:17<380:29:29,  6.78s/it]
 10%|█         | 22945/225000 [55:30:22<343:46:28,  6.13s/it]
 10%|█         | 22946/225000 [55:30:27<330:25:59,  5.89s/it]
 10%|█         | 22947/225000 [55:30:33<329:35:09,  5.87s/it]
 10%|█         | 22948/225000 [55:30:39<330:14:50,  5.88s/it]
 10%|█         | 22949/225000 [55:30:44<323:05:45,  5.76s/it]
 10%|█         | 22950/225000 [55:30:50<320:36:28,  5.71s/it]
                                                             
{'loss': '0.7251', 'grad_norm': '0.6769', 'learning_rate': '4.955e-05', 'epoch': '0.102', 'num_input_tokens_seen': 96625880, 'train_runtime': '1.999e+05', 'train_tokens_per_second': '483.5'}

 10%|█         | 22950/225000 [55:30:50<320:36:28,  5.71s/it]
 10%|█         | 22951/225000 [55:31:03<451:56:08,  8.05s/it]
 10%|█         | 22952/225000 [55:31:11<441:08:34,  7.86s/it]
 10%|█         | 22953/225000 [55:31:19<448:05:47,  7.98s/it]
 10%|█         | 22954/225000 [55:31:25<415:53:57,  7.41s/it]
 10%|█         | 22955/225000 [55:31:32<410:56:39,  7.32s/it]
 10%|█         | 22956/225000 [55:31:39<404:29:08,  7.21s/it]
 10%|█         | 22957/225000 [55:31:45<386:50:25,  6.89s/it]
 10%|█         | 22958/225000 [55:31:53<394:08:20,  7.02s/it]
 10%|█         | 22959/225000 [55:31:56<335:33:29,  5.98s/it]
 10%|█         | 22960/225000 [55:32:05<378:35:39,  6.75s/it]
 10%|█         | 22961/225000 [55:32:11<375:35:21,  6.69s/it]
 10%|█         | 22962/225000 [55:32:16<341:13:27,  6.08s/it]
 10%|█         | 22963/225000 [55:32:21<315:43:17,  5.63s/it]
 10%|█         | 22964/225000 [55:32:31<393:01:20,  7.00s/it]
 10%|█         | 22965/225000 [55:32:39<411:41:59,  7.34s/it]
 10%|█         | 22966/225000 [55:32:45<393:29:40,  7.01s/it]
 10%|█         | 22967/225000 [55:32:49<345:18:51,  6.15s/it]
 10%|█         | 22968/225000 [55:32:54<323:20:11,  5.76s/it]
 10%|█         | 22969/225000 [55:33:04<393:13:12,  7.01s/it]
 10%|█         | 22970/225000 [55:33:11<387:52:32,  6.91s/it]
 10%|█         | 22971/225000 [55:33:18<396:41:59,  7.07s/it]
 10%|█         | 22972/225000 [55:33:23<364:19:08,  6.49s/it]
 10%|█         | 22973/225000 [55:33:31<387:30:27,  6.91s/it]
 10%|█         | 22974/225000 [55:33:36<358:04:17,  6.38s/it]
 10%|█         | 22975/225000 [55:33:45<390:49:45,  6.96s/it]
 10%|█         | 22976/225000 [55:33:50<369:40:15,  6.59s/it]
 10%|█         | 22977/225000 [55:33:58<379:43:19,  6.77s/it]
 10%|█         | 22978/225000 [55:34:02<345:11:07,  6.15s/it]
 10%|█         | 22979/225000 [55:34:07<324:27:14,  5.78s/it]
 10%|█         | 22980/225000 [55:34:13<315:49:01,  5.63s/it]
 10%|█         | 22981/225000 [55:34:25<436:29:33,  7.78s/it]
 10%|█         | 22982/225000 [55:34:31<397:15:15,  7.08s/it]
 10%|█         | 22983/225000 [55:34:36<364:08:10,  6.49s/it]
 10%|█         | 22984/225000 [55:34:41<347:19:11,  6.19s/it]
 10%|█         | 22985/225000 [55:34:47<337:54:25,  6.02s/it]
 10%|█         | 22986/225000 [55:34:53<333:07:52,  5.94s/it]
 10%|█         | 22987/225000 [55:34:58<318:44:34,  5.68s/it]
 10%|█         | 22988/225000 [55:35:04<322:58:07,  5.76s/it]
 10%|█         | 22989/225000 [55:35:17<443:25:21,  7.90s/it]
 10%|█         | 22990/225000 [55:35:24<433:38:29,  7.73s/it]
 10%|█         | 22991/225000 [55:35:32<445:13:09,  7.93s/it]
 10%|█         | 22992/225000 [55:35:37<388:31:26,  6.92s/it]
 10%|█         | 22993/225000 [55:35:45<414:59:29,  7.40s/it]
 10%|█         | 22994/225000 [55:35:51<391:33:51,  6.98s/it]
 10%|█         | 22995/225000 [55:36:00<412:40:57,  7.35s/it]
 10%|█         | 22996/225000 [55:36:04<362:09:32,  6.45s/it]
 10%|█         | 22997/225000 [55:36:09<329:40:16,  5.88s/it]
 10%|█         | 22998/225000 [55:36:17<376:07:49,  6.70s/it]
 10%|█         | 22999/225000 [55:36:27<428:15:30,  7.63s/it]
 10%|█         | 23000/225000 [55:36:32<386:59:00,  6.90s/it]
                                                             
{'loss': '0.7186', 'grad_norm': '0.6055', 'learning_rate': '4.955e-05', 'epoch': '0.1022', 'num_input_tokens_seen': 96776112, 'train_runtime': '2.002e+05', 'train_tokens_per_second': '483.4'}

 10%|█         | 23000/225000 [55:36:32<386:59:00,  6.90s/it]
 10%|█         | 23001/225000 [55:36:40<401:49:41,  7.16s/it]
 10%|█         | 23002/225000 [55:36:44<355:35:21,  6.34s/it]
 10%|█         | 23003/225000 [55:36:50<338:57:05,  6.04s/it]
 10%|█         | 23004/225000 [55:36:55<322:13:22,  5.74s/it]
 10%|█         | 23005/225000 [55:37:00<316:58:55,  5.65s/it]
 10%|█         | 23006/225000 [55:37:10<383:04:07,  6.83s/it]
 10%|█         | 23007/225000 [55:37:22<475:27:51,  8.47s/it]
 10%|█         | 23008/225000 [55:37:26<402:51:29,  7.18s/it]
 10%|█         | 23009/225000 [55:37:31<353:09:18,  6.29s/it]
 10%|█         | 23010/225000 [55:37:42<436:28:45,  7.78s/it]
 10%|█         | 23011/225000 [55:37:47<389:05:56,  6.93s/it]
 10%|█         | 23012/225000 [55:37:52<358:39:50,  6.39s/it]
 10%|█         | 23013/225000 [55:37:57<331:41:27,  5.91s/it]
 10%|█         | 23014/225000 [55:38:01<309:47:08,  5.52s/it]
 10%|█         | 23015/225000 [55:38:12<406:13:22,  7.24s/it]
 10%|█         | 23016/225000 [55:38:20<415:01:02,  7.40s/it]
 10%|█         | 23017/225000 [55:38:25<369:55:27,  6.59s/it]
 10%|█         | 23018/225000 [55:38:32<384:48:07,  6.86s/it]
 10%|█         | 23019/225000 [55:38:42<428:15:17,  7.63s/it]
 10%|█         | 23020/225000 [55:38:46<369:57:45,  6.59s/it]
 10%|█         | 23021/225000 [55:38:56<427:24:48,  7.62s/it]
 10%|█         | 23022/225000 [55:39:03<408:14:57,  7.28s/it]
 10%|█         | 23023/225000 [55:39:07<362:30:09,  6.46s/it]
 10%|█         | 23024/225000 [55:39:12<336:28:56,  6.00s/it]
 10%|█         | 23025/225000 [55:39:19<348:04:58,  6.20s/it]
 10%|█         | 23026/225000 [55:39:29<419:10:50,  7.47s/it]
 10%|█         | 23027/225000 [55:39:33<356:34:40,  6.36s/it]
 10%|█         | 23028/225000 [55:39:43<422:06:37,  7.52s/it]
 10%|█         | 23029/225000 [55:39:47<353:46:38,  6.31s/it]
 10%|█         | 23030/225000 [55:39:55<382:35:54,  6.82s/it]
 10%|█         | 23031/225000 [55:39:59<345:42:39,  6.16s/it]
 10%|█         | 23032/225000 [55:40:04<316:29:35,  5.64s/it]
 10%|█         | 23033/225000 [55:40:09<306:32:06,  5.46s/it]
 10%|█         | 23034/225000 [55:40:20<400:08:18,  7.13s/it]
 10%|█         | 23035/225000 [55:40:27<407:44:18,  7.27s/it]
 10%|█         | 23036/225000 [55:40:35<407:03:20,  7.26s/it]
 10%|█         | 23037/225000 [55:40:40<374:15:36,  6.67s/it]
 10%|█         | 23038/225000 [55:40:45<355:10:55,  6.33s/it]
 10%|█         | 23039/225000 [55:40:51<345:22:19,  6.16s/it]
 10%|█         | 23040/225000 [55:40:57<332:44:48,  5.93s/it]
 10%|█         | 23041/225000 [55:41:02<326:10:26,  5.81s/it]
 10%|█         | 23042/225000 [55:41:08<335:56:19,  5.99s/it]
 10%|█         | 23043/225000 [55:41:15<343:20:41,  6.12s/it]
 10%|█         | 23044/225000 [55:41:26<430:50:50,  7.68s/it]
 10%|█         | 23045/225000 [55:41:32<397:15:35,  7.08s/it]
 10%|█         | 23046/225000 [55:41:38<377:46:30,  6.73s/it]
 10%|█         | 23047/225000 [55:41:44<366:40:52,  6.54s/it]
 10%|█         | 23048/225000 [55:41:55<449:00:19,  8.00s/it]
 10%|█         | 23049/225000 [55:42:00<399:50:10,  7.13s/it]
 10%|█         | 23050/225000 [55:42:06<374:52:40,  6.68s/it]
                                                             
{'loss': '0.6981', 'grad_norm': '0.7584', 'learning_rate': '4.955e-05', 'epoch': '0.1024', 'num_input_tokens_seen': 96925816, 'train_runtime': '2.005e+05', 'train_tokens_per_second': '483.4'}

 10%|█         | 23050/225000 [55:42:06<374:52:40,  6.68s/it]
 10%|█         | 23051/225000 [55:42:11<353:28:58,  6.30s/it]
 10%|█         | 23052/225000 [55:42:17<340:30:46,  6.07s/it]
 10%|█         | 23053/225000 [55:42:23<336:03:42,  5.99s/it]
 10%|█         | 23054/225000 [55:42:32<386:05:13,  6.88s/it]
 10%|█         | 23055/225000 [55:42:37<362:39:27,  6.46s/it]
 10%|█         | 23056/225000 [55:42:43<343:58:21,  6.13s/it]
 10%|█         | 23057/225000 [55:42:48<333:43:48,  5.95s/it]
 10%|█         | 23058/225000 [55:43:01<449:46:20,  8.02s/it]
 10%|█         | 23059/225000 [55:43:05<375:31:23,  6.69s/it]
 10%|█         | 23060/225000 [55:43:14<414:25:32,  7.39s/it]
 10%|█         | 23061/225000 [55:43:21<416:41:30,  7.43s/it]
 10%|█         | 23062/225000 [55:43:25<361:10:53,  6.44s/it]
 10%|█         | 23063/225000 [55:43:35<412:34:14,  7.36s/it]
 10%|█         | 23064/225000 [55:43:41<400:05:17,  7.13s/it]
 10%|█         | 23065/225000 [55:43:47<381:35:19,  6.80s/it]
 10%|█         | 23066/225000 [55:43:54<371:41:32,  6.63s/it]
 10%|█         | 23067/225000 [55:44:00<364:02:16,  6.49s/it]
 10%|█         | 23068/225000 [55:44:07<376:12:35,  6.71s/it]
 10%|█         | 23069/225000 [55:44:11<336:21:12,  6.00s/it]
 10%|█         | 23070/225000 [55:44:16<317:58:32,  5.67s/it]
 10%|█         | 23071/225000 [55:44:21<309:38:59,  5.52s/it]
 10%|█         | 23072/225000 [55:44:34<429:54:10,  7.66s/it]
 10%|█         | 23073/225000 [55:44:39<381:10:51,  6.80s/it]
 10%|█         | 23074/225000 [55:44:45<371:30:11,  6.62s/it]
 10%|█         | 23075/225000 [55:44:53<393:58:13,  7.02s/it]
 10%|█         | 23076/225000 [55:45:00<387:01:19,  6.90s/it]
 10%|█         | 23077/225000 [55:45:04<344:30:01,  6.14s/it]
 10%|█         | 23078/225000 [55:45:12<367:22:26,  6.55s/it]
 10%|█         | 23079/225000 [55:45:19<389:50:22,  6.95s/it]
 10%|█         | 23080/225000 [55:45:26<388:48:50,  6.93s/it]
 10%|█         | 23081/225000 [55:45:30<340:38:49,  6.07s/it]
 10%|█         | 23082/225000 [55:45:35<310:12:07,  5.53s/it]
 10%|█         | 23083/225000 [55:45:47<418:55:21,  7.47s/it]
 10%|█         | 23084/225000 [55:45:51<363:23:08,  6.48s/it]
 10%|█         | 23085/225000 [55:45:56<342:19:36,  6.10s/it]
 10%|█         | 23086/225000 [55:46:01<326:19:38,  5.82s/it]
 10%|█         | 23087/225000 [55:46:12<416:46:30,  7.43s/it]
 10%|█         | 23088/225000 [55:46:16<352:25:07,  6.28s/it]
 10%|█         | 23089/225000 [55:46:26<408:41:01,  7.29s/it]
 10%|█         | 23090/225000 [55:46:29<349:22:20,  6.23s/it]
 10%|█         | 23091/225000 [55:46:39<411:36:13,  7.34s/it]
 10%|█         | 23092/225000 [55:46:46<406:39:41,  7.25s/it]
 10%|█         | 23093/225000 [55:46:53<398:02:25,  7.10s/it]
 10%|█         | 23094/225000 [55:46:57<347:38:32,  6.20s/it]
 10%|█         | 23095/225000 [55:47:01<310:11:02,  5.53s/it]
 10%|█         | 23096/225000 [55:47:10<358:02:55,  6.38s/it]
 10%|█         | 23097/225000 [55:47:18<392:32:08,  7.00s/it]
 10%|█         | 23098/225000 [55:47:23<361:22:57,  6.44s/it]
 10%|█         | 23099/225000 [55:47:29<344:39:00,  6.15s/it]
 10%|█         | 23100/225000 [55:47:34<330:32:27,  5.89s/it]
                                                             
{'loss': '0.7095', 'grad_norm': '0.7866', 'learning_rate': '4.954e-05', 'epoch': '0.1027', 'num_input_tokens_seen': 97075936, 'train_runtime': '2.009e+05', 'train_tokens_per_second': '483.3'}

 10%|█         | 23100/225000 [55:47:34<330:32:27,  5.89s/it]
 10%|█         | 23101/225000 [55:47:40<327:50:41,  5.85s/it]
 10%|█         | 23102/225000 [55:47:45<318:33:54,  5.68s/it]
 10%|█         | 23103/225000 [55:47:50<311:32:14,  5.55s/it]
 10%|█         | 23104/225000 [55:47:59<370:30:34,  6.61s/it]
 10%|█         | 23105/225000 [55:48:05<356:46:14,  6.36s/it]
 10%|█         | 23106/225000 [55:48:11<342:12:24,  6.10s/it]
 10%|█         | 23107/225000 [55:48:16<332:55:04,  5.94s/it]
 10%|█         | 23108/225000 [55:48:23<343:58:39,  6.13s/it]
 10%|█         | 23109/225000 [55:48:36<466:18:29,  8.31s/it]
 10%|█         | 23110/225000 [55:48:43<438:53:09,  7.83s/it]
 10%|█         | 23111/225000 [55:48:47<382:16:58,  6.82s/it]
 10%|█         | 23112/225000 [55:48:55<395:58:25,  7.06s/it]
 10%|█         | 23113/225000 [55:49:07<476:53:17,  8.50s/it]
 10%|█         | 23114/225000 [55:49:12<414:34:13,  7.39s/it]
 10%|█         | 23115/225000 [55:49:19<410:20:51,  7.32s/it]
 10%|█         | 23116/225000 [55:49:24<371:38:23,  6.63s/it]
 10%|█         | 23117/225000 [55:49:29<344:15:08,  6.14s/it]
 10%|█         | 23118/225000 [55:49:34<330:01:01,  5.88s/it]
 10%|█         | 23119/225000 [55:49:47<446:46:17,  7.97s/it]
 10%|█         | 23120/225000 [55:49:52<406:42:09,  7.25s/it]
 10%|█         | 23121/225000 [55:49:57<362:52:35,  6.47s/it]
 10%|█         | 23122/225000 [55:50:03<350:49:35,  6.26s/it]
 10%|█         | 23123/225000 [55:50:17<483:32:49,  8.62s/it]
 10%|█         | 23124/225000 [55:50:21<401:54:58,  7.17s/it]
 10%|█         | 23125/225000 [55:50:28<410:35:38,  7.32s/it]
 10%|█         | 23126/225000 [55:50:33<359:46:58,  6.42s/it]
 10%|█         | 23127/225000 [55:50:37<324:36:18,  5.79s/it]
 10%|█         | 23128/225000 [55:50:44<351:51:07,  6.27s/it]
 10%|█         | 23129/225000 [55:50:53<386:08:57,  6.89s/it]
 10%|█         | 23130/225000 [55:51:01<414:49:08,  7.40s/it]
 10%|█         | 23131/225000 [55:51:05<355:41:42,  6.34s/it]
 10%|█         | 23132/225000 [55:51:14<396:45:55,  7.08s/it]
 10%|█         | 23133/225000 [55:51:19<366:46:09,  6.54s/it]
 10%|█         | 23134/225000 [55:51:25<356:40:05,  6.36s/it]
 10%|█         | 23135/225000 [55:51:31<344:39:53,  6.15s/it]
 10%|█         | 23136/225000 [55:51:38<368:01:11,  6.56s/it]
 10%|█         | 23137/225000 [55:51:48<423:48:20,  7.56s/it]
 10%|█         | 23138/225000 [55:51:53<376:28:33,  6.71s/it]
 10%|█         | 23139/225000 [55:52:02<406:49:37,  7.26s/it]
 10%|█         | 23140/225000 [55:52:06<362:26:01,  6.46s/it]
 10%|█         | 23141/225000 [55:52:12<344:33:27,  6.14s/it]
 10%|█         | 23142/225000 [55:52:23<440:50:35,  7.86s/it]
 10%|█         | 23143/225000 [55:52:31<438:53:57,  7.83s/it]
 10%|█         | 23144/225000 [55:52:40<461:36:48,  8.23s/it]
 10%|█         | 23145/225000 [55:52:45<408:33:09,  7.29s/it]
 10%|█         | 23146/225000 [55:52:51<386:17:22,  6.89s/it]
 10%|█         | 23147/225000 [55:52:58<387:19:58,  6.91s/it]
 10%|█         | 23148/225000 [55:53:03<343:08:37,  6.12s/it]
 10%|█         | 23149/225000 [55:53:11<376:22:03,  6.71s/it]
 10%|█         | 23150/225000 [55:53:15<338:22:13,  6.03s/it]
                                                             
{'loss': '0.7074', 'grad_norm': '0.7136', 'learning_rate': '4.954e-05', 'epoch': '0.1029', 'num_input_tokens_seen': 97227888, 'train_runtime': '2.012e+05', 'train_tokens_per_second': '483.2'}

 10%|█         | 23150/225000 [55:53:15<338:22:13,  6.03s/it]
 10%|█         | 23151/225000 [55:53:19<306:53:29,  5.47s/it]
 10%|█         | 23152/225000 [55:53:30<393:21:18,  7.02s/it]
 10%|█         | 23153/225000 [55:53:34<340:34:20,  6.07s/it]
 10%|█         | 23154/225000 [55:53:43<393:49:39,  7.02s/it]
 10%|█         | 23155/225000 [55:53:48<351:33:50,  6.27s/it]
 10%|█         | 23156/225000 [55:53:57<406:50:03,  7.26s/it]
 10%|█         | 23157/225000 [55:54:02<360:00:56,  6.42s/it]
 10%|█         | 23158/225000 [55:54:11<406:19:59,  7.25s/it]
 10%|█         | 23159/225000 [55:54:15<348:42:43,  6.22s/it]
 10%|█         | 23160/225000 [55:54:20<336:04:36,  5.99s/it]
 10%|█         | 23161/225000 [55:54:26<329:31:39,  5.88s/it]
 10%|█         | 23162/225000 [55:54:31<322:04:32,  5.74s/it]
 10%|█         | 23163/225000 [55:54:41<394:09:23,  7.03s/it]
 10%|█         | 23164/225000 [55:54:50<429:11:37,  7.66s/it]
 10%|█         | 23165/225000 [55:54:58<433:32:51,  7.73s/it]
 10%|█         | 23166/225000 [55:55:03<379:35:28,  6.77s/it]
 10%|█         | 23167/225000 [55:55:07<345:32:14,  6.16s/it]
 10%|█         | 23168/225000 [55:55:17<404:54:26,  7.22s/it]
 10%|█         | 23169/225000 [55:55:23<386:35:20,  6.90s/it]
 10%|█         | 23170/225000 [55:55:28<355:29:54,  6.34s/it]
 10%|█         | 23171/225000 [55:55:33<332:47:36,  5.94s/it]
 10%|█         | 23172/225000 [55:55:39<336:43:36,  6.01s/it]
 10%|█         | 23173/225000 [55:55:53<461:52:35,  8.24s/it]
 10%|█         | 23174/225000 [55:55:59<428:49:21,  7.65s/it]
 10%|█         | 23175/225000 [55:56:03<363:28:06,  6.48s/it]
 10%|█         | 23176/225000 [55:56:11<397:47:25,  7.10s/it]
 10%|█         | 23177/225000 [55:56:17<367:00:20,  6.55s/it]
 10%|█         | 23178/225000 [55:56:21<326:47:35,  5.83s/it]
 10%|█         | 23179/225000 [55:56:26<309:20:42,  5.52s/it]
 10%|█         | 23180/225000 [55:56:35<366:38:07,  6.54s/it]
 10%|█         | 23181/225000 [55:56:43<394:44:55,  7.04s/it]
 10%|█         | 23182/225000 [55:56:48<366:02:06,  6.53s/it]
 10%|█         | 23183/225000 [55:56:54<353:30:49,  6.31s/it]
 10%|█         | 23184/225000 [55:57:00<351:04:06,  6.26s/it]
 10%|█         | 23185/225000 [55:57:11<435:31:15,  7.77s/it]
 10%|█         | 23186/225000 [55:57:17<403:08:40,  7.19s/it]
 10%|█         | 23187/225000 [55:57:22<354:02:04,  6.32s/it]
 10%|█         | 23188/225000 [55:57:26<325:49:47,  5.81s/it]
 10%|█         | 23189/225000 [55:57:38<420:56:57,  7.51s/it]
 10%|█         | 23190/225000 [55:57:44<407:47:49,  7.27s/it]
 10%|█         | 23191/225000 [55:57:50<378:44:50,  6.76s/it]
 10%|█         | 23192/225000 [55:57:55<349:28:24,  6.23s/it]
 10%|█         | 23193/225000 [55:58:00<337:22:29,  6.02s/it]
 10%|█         | 23194/225000 [55:58:05<318:50:43,  5.69s/it]
 10%|█         | 23195/225000 [55:58:19<445:05:55,  7.94s/it]
 10%|█         | 23196/225000 [55:58:24<397:13:08,  7.09s/it]
 10%|█         | 23197/225000 [55:58:33<428:34:21,  7.65s/it]
 10%|█         | 23198/225000 [55:58:37<379:30:16,  6.77s/it]
 10%|█         | 23199/225000 [55:58:42<340:22:35,  6.07s/it]
 10%|█         | 23200/225000 [55:58:54<441:57:31,  7.88s/it]
                                                             
{'loss': '0.7218', 'grad_norm': '0.6346', 'learning_rate': '4.954e-05', 'epoch': '0.1031', 'num_input_tokens_seen': 97379360, 'train_runtime': '2.015e+05', 'train_tokens_per_second': '483.2'}

 10%|█         | 23200/225000 [55:58:54<441:57:31,  7.88s/it]
 10%|█         | 23201/225000 [55:58:58<376:02:04,  6.71s/it]
 10%|█         | 23202/225000 [55:59:04<360:20:53,  6.43s/it]
 10%|█         | 23203/225000 [55:59:12<397:01:38,  7.08s/it]
 10%|█         | 23204/225000 [55:59:20<402:22:38,  7.18s/it]
 10%|█         | 23205/225000 [55:59:25<372:20:46,  6.64s/it]
 10%|█         | 23206/225000 [55:59:31<363:28:33,  6.48s/it]
 10%|█         | 23207/225000 [55:59:39<382:10:55,  6.82s/it]
 10%|█         | 23208/225000 [55:59:47<406:57:29,  7.26s/it]
 10%|█         | 23209/225000 [55:59:51<345:51:26,  6.17s/it]
 10%|█         | 23210/225000 [55:59:59<379:36:50,  6.77s/it]
 10%|█         | 23211/225000 [56:00:05<373:08:16,  6.66s/it]
 10%|█         | 23212/225000 [56:00:10<340:42:17,  6.08s/it]
 10%|█         | 23213/225000 [56:00:15<331:16:55,  5.91s/it]
 10%|█         | 23214/225000 [56:00:21<326:54:40,  5.83s/it]
 10%|█         | 23215/225000 [56:00:27<321:22:35,  5.73s/it]
 10%|█         | 23216/225000 [56:00:32<316:37:59,  5.65s/it]
 10%|█         | 23217/225000 [56:00:37<306:47:22,  5.47s/it]
 10%|█         | 23218/225000 [56:00:43<311:04:49,  5.55s/it]
 10%|█         | 23219/225000 [56:00:49<317:13:11,  5.66s/it]
 10%|█         | 23220/225000 [56:01:02<443:06:54,  7.91s/it]
 10%|█         | 23221/225000 [56:01:07<400:48:28,  7.15s/it]
 10%|█         | 23222/225000 [56:01:11<345:14:30,  6.16s/it]
 10%|█         | 23223/225000 [56:01:21<405:38:35,  7.24s/it]
 10%|█         | 23224/225000 [56:01:25<348:29:11,  6.22s/it]
 10%|█         | 23225/225000 [56:01:36<432:04:13,  7.71s/it]
 10%|█         | 23226/225000 [56:01:41<385:08:10,  6.87s/it]
 10%|█         | 23227/225000 [56:01:45<339:54:46,  6.06s/it]
 10%|█         | 23228/225000 [56:01:50<319:24:53,  5.70s/it]
 10%|█         | 23229/225000 [56:01:55<312:39:48,  5.58s/it]
 10%|█         | 23230/225000 [56:02:02<333:41:04,  5.95s/it]
 10%|█         | 23231/225000 [56:02:13<425:05:43,  7.58s/it]
 10%|█         | 23232/225000 [56:02:19<385:20:25,  6.88s/it]
 10%|█         | 23233/225000 [56:02:26<387:13:18,  6.91s/it]
 10%|█         | 23234/225000 [56:02:30<352:03:27,  6.28s/it]
 10%|█         | 23235/225000 [56:02:36<334:01:05,  5.96s/it]
 10%|█         | 23236/225000 [56:02:41<327:36:51,  5.85s/it]
 10%|█         | 23237/225000 [56:02:46<311:17:26,  5.55s/it]
 10%|█         | 23238/225000 [56:02:56<389:31:51,  6.95s/it]
 10%|█         | 23239/225000 [56:03:07<447:46:15,  7.99s/it]
 10%|█         | 23240/225000 [56:03:11<388:43:10,  6.94s/it]
 10%|█         | 23241/225000 [56:03:16<354:38:19,  6.33s/it]
 10%|█         | 23242/225000 [56:03:29<462:08:21,  8.25s/it]
 10%|█         | 23243/225000 [56:03:34<404:30:17,  7.22s/it]
 10%|█         | 23244/225000 [56:03:39<373:06:22,  6.66s/it]
 10%|█         | 23245/225000 [56:03:44<352:46:13,  6.29s/it]
 10%|█         | 23246/225000 [56:03:51<363:30:13,  6.49s/it]
 10%|█         | 23247/225000 [56:04:03<450:27:37,  8.04s/it]
 10%|█         | 23248/225000 [56:04:10<433:37:41,  7.74s/it]
 10%|█         | 23249/225000 [56:04:15<383:14:19,  6.84s/it]
 10%|█         | 23250/225000 [56:04:20<362:28:31,  6.47s/it]
                                                             
{'loss': '0.7093', 'grad_norm': '0.5886', 'learning_rate': '4.953e-05', 'epoch': '0.1033', 'num_input_tokens_seen': 97529648, 'train_runtime': '2.019e+05', 'train_tokens_per_second': '483.2'}

 10%|█         | 23250/225000 [56:04:20<362:28:31,  6.47s/it]
 10%|█         | 23251/225000 [56:04:25<324:33:44,  5.79s/it]
 10%|█         | 23252/225000 [56:04:32<359:14:05,  6.41s/it]
 10%|█         | 23253/225000 [56:04:43<421:49:53,  7.53s/it]
 10%|█         | 23254/225000 [56:04:47<373:12:02,  6.66s/it]
 10%|█         | 23255/225000 [56:04:53<350:14:59,  6.25s/it]
 10%|█         | 23256/225000 [56:05:02<404:08:56,  7.21s/it]
 10%|█         | 23257/225000 [56:05:10<421:20:31,  7.52s/it]
 10%|█         | 23258/225000 [56:05:14<356:34:16,  6.36s/it]
 10%|█         | 23259/225000 [56:05:20<358:10:30,  6.39s/it]
 10%|█         | 23260/225000 [56:05:29<401:21:34,  7.16s/it]
 10%|█         | 23261/225000 [56:05:34<365:25:37,  6.52s/it]
 10%|█         | 23262/225000 [56:05:39<340:43:23,  6.08s/it]
 10%|█         | 23263/225000 [56:05:44<313:00:15,  5.59s/it]
 10%|█         | 23264/225000 [56:05:56<420:19:09,  7.50s/it]
 10%|█         | 23265/225000 [56:06:01<387:27:32,  6.91s/it]
 10%|█         | 23266/225000 [56:06:06<352:23:43,  6.29s/it]
 10%|█         | 23267/225000 [56:06:12<349:43:17,  6.24s/it]
 10%|█         | 23268/225000 [56:06:26<467:51:36,  8.35s/it]
 10%|█         | 23269/225000 [56:06:29<388:38:16,  6.94s/it]
 10%|█         | 23270/225000 [56:06:34<354:35:15,  6.33s/it]
 10%|█         | 23271/225000 [56:06:39<331:16:43,  5.91s/it]
 10%|█         | 23272/225000 [56:06:45<327:02:53,  5.84s/it]
 10%|█         | 23273/225000 [56:06:50<318:17:29,  5.68s/it]
 10%|█         | 23274/225000 [56:06:55<312:07:18,  5.57s/it]
 10%|█         | 23275/225000 [56:07:09<440:58:51,  7.87s/it]
 10%|█         | 23276/225000 [56:07:16<441:31:47,  7.88s/it]
 10%|█         | 23277/225000 [56:07:26<477:23:06,  8.52s/it]
 10%|█         | 23278/225000 [56:07:33<442:13:36,  7.89s/it]
 10%|█         | 23279/225000 [56:07:40<427:15:23,  7.63s/it]
 10%|█         | 23280/225000 [56:07:47<426:01:49,  7.60s/it]
 10%|█         | 23281/225000 [56:07:52<366:43:05,  6.54s/it]
 10%|█         | 23282/225000 [56:07:57<349:40:32,  6.24s/it]
 10%|█         | 23283/225000 [56:08:08<433:55:38,  7.74s/it]
 10%|█         | 23284/225000 [56:08:12<368:54:25,  6.58s/it]
 10%|█         | 23285/225000 [56:08:18<350:45:05,  6.26s/it]
 10%|█         | 23286/225000 [56:08:23<342:34:24,  6.11s/it]
 10%|█         | 23287/225000 [56:08:29<331:53:56,  5.92s/it]
 10%|█         | 23288/225000 [56:08:35<329:29:49,  5.88s/it]
 10%|█         | 23289/225000 [56:08:47<430:52:46,  7.69s/it]
 10%|█         | 23290/225000 [56:08:53<401:48:01,  7.17s/it]
 10%|█         | 23291/225000 [56:08:59<397:11:50,  7.09s/it]
 10%|█         | 23292/225000 [56:09:07<407:53:05,  7.28s/it]
 10%|█         | 23293/225000 [56:09:15<412:05:17,  7.35s/it]
 10%|█         | 23294/225000 [56:09:24<437:29:25,  7.81s/it]
 10%|█         | 23295/225000 [56:09:28<373:44:24,  6.67s/it]
 10%|█         | 23296/225000 [56:09:35<390:31:35,  6.97s/it]
 10%|█         | 23297/225000 [56:09:41<371:41:31,  6.63s/it]
 10%|█         | 23298/225000 [56:09:50<404:45:53,  7.22s/it]
 10%|█         | 23299/225000 [56:09:54<356:43:17,  6.37s/it]
 10%|█         | 23300/225000 [56:10:00<341:39:30,  6.10s/it]
                                                             
{'loss': '0.733', 'grad_norm': '0.6784', 'learning_rate': '4.953e-05', 'epoch': '0.1036', 'num_input_tokens_seen': 97682712, 'train_runtime': '2.022e+05', 'train_tokens_per_second': '483.1'}

 10%|█         | 23300/225000 [56:10:00<341:39:30,  6.10s/it]
 10%|█         | 23301/225000 [56:10:05<329:42:39,  5.88s/it]
 10%|█         | 23302/225000 [56:10:10<322:24:55,  5.75s/it]
 10%|█         | 23303/225000 [56:10:23<435:23:53,  7.77s/it]
 10%|█         | 23304/225000 [56:10:27<368:40:20,  6.58s/it]
 10%|█         | 23305/225000 [56:10:37<428:28:44,  7.65s/it]
 10%|█         | 23306/225000 [56:10:41<370:28:31,  6.61s/it]
 10%|█         | 23307/225000 [56:10:45<332:13:29,  5.93s/it]
 10%|█         | 23308/225000 [56:10:58<444:01:14,  7.93s/it]
 10%|█         | 23309/225000 [56:11:02<378:07:01,  6.75s/it]
 10%|█         | 23310/225000 [56:11:08<363:49:53,  6.49s/it]
 10%|█         | 23311/225000 [56:11:14<349:57:34,  6.25s/it]
 10%|█         | 23312/225000 [56:11:19<344:40:35,  6.15s/it]
 10%|█         | 23313/225000 [56:11:33<461:29:56,  8.24s/it]
 10%|█         | 23314/225000 [56:11:37<393:18:45,  7.02s/it]
 10%|█         | 23315/225000 [56:11:44<393:58:22,  7.03s/it]
 10%|█         | 23316/225000 [56:11:52<410:58:25,  7.34s/it]
 10%|█         | 23317/225000 [56:11:58<388:59:47,  6.94s/it]
 10%|█         | 23318/225000 [56:12:02<337:23:36,  6.02s/it]
 10%|█         | 23319/225000 [56:12:06<311:58:15,  5.57s/it]
 10%|█         | 23320/225000 [56:12:12<310:29:49,  5.54s/it]
 10%|█         | 23321/225000 [56:12:24<416:26:13,  7.43s/it]
 10%|█         | 23322/225000 [56:12:29<390:12:11,  6.97s/it]
 10%|█         | 23323/225000 [56:12:35<373:56:04,  6.67s/it]
 10%|█         | 23324/225000 [56:12:44<408:39:01,  7.29s/it]
 10%|█         | 23325/225000 [56:12:49<359:27:59,  6.42s/it]
 10%|█         | 23326/225000 [56:12:53<332:35:22,  5.94s/it]
 10%|█         | 23327/225000 [56:12:58<318:44:39,  5.69s/it]
 10%|█         | 23328/225000 [56:13:04<316:55:31,  5.66s/it]
 10%|█         | 23329/225000 [56:13:12<353:00:40,  6.30s/it]
 10%|█         | 23330/225000 [56:13:23<437:08:38,  7.80s/it]
 10%|█         | 23331/225000 [56:13:29<407:52:42,  7.28s/it]
 10%|█         | 23332/225000 [56:13:34<368:54:24,  6.59s/it]
 10%|█         | 23333/225000 [56:13:40<352:07:48,  6.29s/it]
 10%|█         | 23334/225000 [56:13:45<336:36:02,  6.01s/it]
 10%|█         | 23335/225000 [56:13:51<327:16:52,  5.84s/it]
 10%|█         | 23336/225000 [56:14:03<432:05:22,  7.71s/it]
 10%|█         | 23337/225000 [56:14:07<368:50:00,  6.58s/it]
 10%|█         | 23338/225000 [56:14:15<397:13:15,  7.09s/it]
 10%|█         | 23339/225000 [56:14:22<395:04:38,  7.05s/it]
 10%|█         | 23340/225000 [56:14:26<352:10:01,  6.29s/it]
 10%|█         | 23341/225000 [56:14:31<327:44:29,  5.85s/it]
 10%|█         | 23342/225000 [56:14:37<324:27:04,  5.79s/it]
 10%|█         | 23343/225000 [56:14:42<316:40:29,  5.65s/it]
 10%|█         | 23344/225000 [56:14:48<317:17:58,  5.66s/it]
 10%|█         | 23345/225000 [56:14:53<313:14:00,  5.59s/it]
 10%|█         | 23346/225000 [56:15:04<392:42:39,  7.01s/it]
 10%|█         | 23347/225000 [56:15:10<387:41:14,  6.92s/it]
 10%|█         | 23348/225000 [56:15:16<360:22:48,  6.43s/it]
 10%|█         | 23349/225000 [56:15:22<358:52:21,  6.41s/it]
 10%|█         | 23350/225000 [56:15:27<332:02:09,  5.93s/it]
                                                             
{'loss': '0.7172', 'grad_norm': '0.6098', 'learning_rate': '4.953e-05', 'epoch': '0.1038', 'num_input_tokens_seen': 97831976, 'train_runtime': '2.025e+05', 'train_tokens_per_second': '483.1'}

 10%|█         | 23350/225000 [56:15:27<332:02:09,  5.93s/it]
 10%|█         | 23351/225000 [56:15:32<326:16:54,  5.83s/it]
 10%|█         | 23352/225000 [56:15:38<319:07:40,  5.70s/it]
 10%|█         | 23353/225000 [56:15:51<443:17:17,  7.91s/it]
 10%|█         | 23354/225000 [56:15:57<413:54:11,  7.39s/it]
 10%|█         | 23355/225000 [56:16:01<362:56:58,  6.48s/it]
 10%|█         | 23356/225000 [56:16:07<344:47:56,  6.16s/it]
 10%|█         | 23357/225000 [56:16:15<387:18:45,  6.91s/it]
 10%|█         | 23358/225000 [56:16:25<437:19:10,  7.81s/it]
 10%|█         | 23359/225000 [56:16:29<367:30:50,  6.56s/it]
 10%|█         | 23360/225000 [56:16:37<389:08:53,  6.95s/it]
 10%|█         | 23361/225000 [56:16:42<360:31:28,  6.44s/it]
 10%|█         | 23362/225000 [56:16:49<369:36:50,  6.60s/it]
 10%|█         | 23363/225000 [56:16:54<345:12:53,  6.16s/it]
 10%|█         | 23364/225000 [56:16:59<326:56:28,  5.84s/it]
 10%|█         | 23365/225000 [56:17:11<424:42:47,  7.58s/it]
 10%|█         | 23366/225000 [56:17:17<396:59:57,  7.09s/it]
 10%|█         | 23367/225000 [56:17:22<365:41:03,  6.53s/it]
 10%|█         | 23368/225000 [56:17:27<339:38:03,  6.06s/it]
 10%|█         | 23369/225000 [56:17:35<374:54:00,  6.69s/it]
 10%|█         | 23370/225000 [56:17:44<411:03:48,  7.34s/it]
 10%|█         | 23371/225000 [56:17:48<360:58:31,  6.45s/it]
 10%|█         | 23372/225000 [56:17:55<366:08:59,  6.54s/it]
 10%|█         | 23373/225000 [56:18:03<393:14:54,  7.02s/it]
 10%|█         | 23374/225000 [56:18:10<394:21:44,  7.04s/it]
 10%|█         | 23375/225000 [56:18:14<343:31:17,  6.13s/it]
 10%|█         | 23376/225000 [56:18:24<403:03:41,  7.20s/it]
 10%|█         | 23377/225000 [56:18:33<431:31:38,  7.70s/it]
 10%|█         | 23378/225000 [56:18:37<375:43:34,  6.71s/it]
 10%|█         | 23379/225000 [56:18:48<448:13:13,  8.00s/it]
 10%|█         | 23380/225000 [56:18:55<417:56:55,  7.46s/it]
 10%|█         | 23381/225000 [56:19:01<405:40:01,  7.24s/it]
 10%|█         | 23382/225000 [56:19:09<409:11:06,  7.31s/it]
 10%|█         | 23383/225000 [56:19:13<360:19:32,  6.43s/it]
 10%|█         | 23384/225000 [56:19:22<402:54:35,  7.19s/it]
 10%|█         | 23385/225000 [56:19:28<380:00:06,  6.79s/it]
 10%|█         | 23386/225000 [56:19:34<359:33:14,  6.42s/it]
 10%|█         | 23387/225000 [56:19:42<395:54:07,  7.07s/it]
 10%|█         | 23388/225000 [56:19:48<374:52:09,  6.69s/it]
 10%|█         | 23389/225000 [56:19:55<381:59:37,  6.82s/it]
 10%|█         | 23390/225000 [56:19:59<332:28:01,  5.94s/it]
 10%|█         | 23391/225000 [56:20:09<406:43:24,  7.26s/it]
 10%|█         | 23392/225000 [56:20:13<348:56:40,  6.23s/it]
 10%|█         | 23393/225000 [56:20:20<360:01:37,  6.43s/it]
 10%|█         | 23394/225000 [56:20:27<365:44:50,  6.53s/it]
 10%|█         | 23395/225000 [56:20:32<334:20:32,  5.97s/it]
 10%|█         | 23396/225000 [56:20:40<372:04:02,  6.64s/it]
 10%|█         | 23397/225000 [56:20:47<380:28:22,  6.79s/it]
 10%|█         | 23398/225000 [56:20:56<416:32:13,  7.44s/it]
 10%|█         | 23399/225000 [56:21:00<362:11:12,  6.47s/it]
 10%|█         | 23400/225000 [56:21:04<323:40:30,  5.78s/it]
                                                             
{'loss': '0.7189', 'grad_norm': '0.6974', 'learning_rate': '4.952e-05', 'epoch': '0.104', 'num_input_tokens_seen': 97980456, 'train_runtime': '2.029e+05', 'train_tokens_per_second': '483'}

 10%|█         | 23400/225000 [56:21:04<323:40:30,  5.78s/it]
 10%|█         | 23401/225000 [56:21:08<292:34:46,  5.22s/it]
 10%|█         | 23402/225000 [56:21:21<425:02:13,  7.59s/it]
 10%|█         | 23403/225000 [56:21:26<375:41:47,  6.71s/it]
 10%|█         | 23404/225000 [56:21:36<426:10:54,  7.61s/it]
 10%|█         | 23405/225000 [56:21:39<356:57:50,  6.37s/it]
 10%|█         | 23406/225000 [56:21:45<341:44:07,  6.10s/it]
 10%|█         | 23407/225000 [56:21:50<329:04:12,  5.88s/it]
 10%|█         | 23408/225000 [56:21:55<323:27:52,  5.78s/it]
 10%|█         | 23409/225000 [56:22:01<313:05:37,  5.59s/it]
 10%|█         | 23410/225000 [56:22:06<314:39:17,  5.62s/it]
 10%|█         | 23411/225000 [56:22:19<431:48:31,  7.71s/it]
 10%|█         | 23412/225000 [56:22:26<417:13:10,  7.45s/it]
 10%|█         | 23413/225000 [56:22:31<383:41:37,  6.85s/it]
 10%|█         | 23414/225000 [56:22:36<355:25:57,  6.35s/it]
 10%|█         | 23415/225000 [56:22:42<340:04:59,  6.07s/it]
 10%|█         | 23416/225000 [56:22:47<321:06:50,  5.73s/it]
 10%|█         | 23417/225000 [56:22:52<316:08:25,  5.65s/it]
 10%|█         | 23418/225000 [56:23:05<436:50:29,  7.80s/it]
 10%|█         | 23419/225000 [56:23:11<406:57:25,  7.27s/it]
 10%|█         | 23420/225000 [56:23:17<384:11:24,  6.86s/it]
 10%|█         | 23421/225000 [56:23:26<413:37:50,  7.39s/it]
 10%|█         | 23422/225000 [56:23:31<373:49:17,  6.68s/it]
 10%|█         | 23423/225000 [56:23:38<391:33:38,  6.99s/it]
 10%|█         | 23424/225000 [56:23:42<343:18:55,  6.13s/it]
 10%|█         | 23425/225000 [56:23:47<319:06:07,  5.70s/it]
 10%|█         | 23426/225000 [56:24:01<453:33:58,  8.10s/it]
 10%|█         | 23427/225000 [56:24:05<389:58:38,  6.96s/it]
 10%|█         | 23428/225000 [56:24:10<359:15:10,  6.42s/it]
 10%|█         | 23429/225000 [56:24:16<340:00:07,  6.07s/it]
 10%|█         | 23430/225000 [56:24:21<333:41:34,  5.96s/it]
 10%|█         | 23431/225000 [56:24:27<324:58:18,  5.80s/it]
 10%|█         | 23432/225000 [56:24:33<328:12:00,  5.86s/it]
 10%|█         | 23433/225000 [56:24:39<337:09:27,  6.02s/it]
 10%|█         | 23434/225000 [56:24:52<448:05:14,  8.00s/it]
 10%|█         | 23435/225000 [56:24:59<429:30:05,  7.67s/it]
 10%|█         | 23436/225000 [56:25:02<362:30:37,  6.47s/it]
 10%|█         | 23437/225000 [56:25:07<332:08:07,  5.93s/it]
 10%|█         | 23438/225000 [56:25:12<319:20:03,  5.70s/it]
 10%|█         | 23439/225000 [56:25:25<434:39:02,  7.76s/it]
 10%|█         | 23440/225000 [56:25:29<385:24:06,  6.88s/it]
 10%|█         | 23441/225000 [56:25:35<355:16:53,  6.35s/it]
 10%|█         | 23442/225000 [56:25:40<338:08:41,  6.04s/it]
 10%|█         | 23443/225000 [56:25:45<327:08:29,  5.84s/it]
 10%|█         | 23444/225000 [56:25:51<318:30:49,  5.69s/it]
 10%|█         | 23445/225000 [56:25:57<324:21:46,  5.79s/it]
 10%|█         | 23446/225000 [56:26:06<376:55:51,  6.73s/it]
 10%|█         | 23447/225000 [56:26:16<443:05:40,  7.91s/it]
 10%|█         | 23448/225000 [56:26:22<411:07:50,  7.34s/it]
 10%|█         | 23449/225000 [56:26:29<399:16:21,  7.13s/it]
 10%|█         | 23450/225000 [56:26:34<362:06:45,  6.47s/it]
                                                             
{'loss': '0.7056', 'grad_norm': '0.5959', 'learning_rate': '4.952e-05', 'epoch': '0.1042', 'num_input_tokens_seen': 98129400, 'train_runtime': '2.032e+05', 'train_tokens_per_second': '482.9'}

 10%|█         | 23450/225000 [56:26:34<362:06:45,  6.47s/it]
 10%|█         | 23451/225000 [56:26:39<337:38:27,  6.03s/it]
 10%|█         | 23452/225000 [56:26:43<314:34:40,  5.62s/it]
 10%|█         | 23453/225000 [56:26:51<341:45:02,  6.10s/it]
 10%|█         | 23454/225000 [56:27:03<444:54:01,  7.95s/it]
 10%|█         | 23455/225000 [56:27:08<391:47:07,  7.00s/it]
 10%|█         | 23456/225000 [56:27:12<353:09:44,  6.31s/it]
 10%|█         | 23457/225000 [56:27:17<318:41:25,  5.69s/it]
 10%|█         | 23458/225000 [56:27:27<403:51:35,  7.21s/it]
 10%|█         | 23459/225000 [56:27:36<432:31:32,  7.73s/it]
 10%|█         | 23460/225000 [56:27:42<392:51:07,  7.02s/it]
 10%|█         | 23461/225000 [56:27:50<414:30:09,  7.40s/it]
 10%|█         | 23462/225000 [56:27:56<387:10:38,  6.92s/it]
 10%|█         | 23463/225000 [56:28:05<419:40:09,  7.50s/it]
 10%|█         | 23464/225000 [56:28:08<356:13:39,  6.36s/it]
 10%|█         | 23465/225000 [56:28:14<342:42:34,  6.12s/it]
 10%|█         | 23466/225000 [56:28:20<336:10:25,  6.01s/it]
 10%|█         | 23467/225000 [56:28:26<335:25:25,  5.99s/it]
 10%|█         | 23468/225000 [56:28:32<338:52:38,  6.05s/it]
 10%|█         | 23469/225000 [56:28:37<331:42:09,  5.93s/it]
 10%|█         | 23470/225000 [56:28:44<340:00:46,  6.07s/it]
 10%|█         | 23471/225000 [56:28:56<435:49:14,  7.79s/it]
 10%|█         | 23472/225000 [56:29:03<421:49:43,  7.54s/it]
 10%|█         | 23473/225000 [56:29:09<407:11:20,  7.27s/it]
 10%|█         | 23474/225000 [56:29:16<403:35:42,  7.21s/it]
 10%|█         | 23475/225000 [56:29:21<366:31:43,  6.55s/it]
 10%|█         | 23476/225000 [56:29:29<384:16:08,  6.86s/it]
 10%|█         | 23477/225000 [56:29:35<373:12:20,  6.67s/it]
 10%|█         | 23478/225000 [56:29:41<363:10:17,  6.49s/it]
 10%|█         | 23479/225000 [56:29:49<376:14:44,  6.72s/it]
 10%|█         | 23480/225000 [56:29:56<385:45:43,  6.89s/it]
 10%|█         | 23481/225000 [56:30:03<383:37:52,  6.85s/it]
 10%|█         | 23482/225000 [56:30:08<360:13:57,  6.44s/it]
 10%|█         | 23483/225000 [56:30:16<383:15:21,  6.85s/it]
 10%|█         | 23484/225000 [56:30:20<340:57:48,  6.09s/it]
 10%|█         | 23485/225000 [56:30:29<385:49:43,  6.89s/it]
 10%|█         | 23486/225000 [56:30:36<389:35:57,  6.96s/it]
 10%|█         | 23487/225000 [56:30:42<371:18:45,  6.63s/it]
 10%|█         | 23488/225000 [56:30:50<393:33:24,  7.03s/it]
 10%|█         | 23489/225000 [56:30:54<341:55:57,  6.11s/it]
 10%|█         | 23490/225000 [56:31:02<383:42:03,  6.85s/it]
 10%|█         | 23491/225000 [56:31:08<354:11:45,  6.33s/it]
 10%|█         | 23492/225000 [56:31:17<402:21:31,  7.19s/it]
 10%|█         | 23493/225000 [56:31:21<356:00:02,  6.36s/it]
 10%|█         | 23494/225000 [56:31:25<318:40:15,  5.69s/it]
 10%|█         | 23495/225000 [56:31:30<305:39:09,  5.46s/it]
 10%|█         | 23496/225000 [56:31:42<415:20:33,  7.42s/it]
 10%|█         | 23497/225000 [56:31:46<361:56:13,  6.47s/it]
 10%|█         | 23498/225000 [56:31:56<414:06:19,  7.40s/it]
 10%|█         | 23499/225000 [56:32:00<359:39:06,  6.43s/it]
 10%|█         | 23500/225000 [56:32:05<327:59:06,  5.86s/it]
                                                             
{'loss': '0.73', 'grad_norm': '0.7425', 'learning_rate': '4.952e-05', 'epoch': '0.1044', 'num_input_tokens_seen': 98278760, 'train_runtime': '2.035e+05', 'train_tokens_per_second': '482.9'}

 10%|█         | 23500/225000 [56:32:05<327:59:06,  5.86s/it]
 10%|█         | 23501/225000 [56:32:10<316:21:29,  5.65s/it]
 10%|█         | 23502/225000 [56:32:15<309:49:08,  5.54s/it]
 10%|█         | 23503/225000 [56:32:28<437:10:37,  7.81s/it]
 10%|█         | 23504/225000 [56:32:35<411:23:36,  7.35s/it]
 10%|█         | 23505/225000 [56:32:42<407:13:35,  7.28s/it]
 10%|█         | 23506/225000 [56:32:49<410:51:59,  7.34s/it]
 10%|█         | 23507/225000 [56:32:53<355:52:47,  6.36s/it]
 10%|█         | 23508/225000 [56:32:59<352:45:11,  6.30s/it]
 10%|█         | 23509/225000 [56:33:05<334:06:51,  5.97s/it]
 10%|█         | 23510/225000 [56:33:10<324:16:54,  5.79s/it]
 10%|█         | 23511/225000 [56:33:16<323:09:52,  5.77s/it]
 10%|█         | 23512/225000 [56:33:21<320:46:10,  5.73s/it]
 10%|█         | 23513/225000 [56:33:27<316:34:36,  5.66s/it]
 10%|█         | 23514/225000 [56:33:32<316:27:50,  5.65s/it]
 10%|█         | 23515/225000 [56:33:43<403:27:35,  7.21s/it]
 10%|█         | 23516/225000 [56:33:51<414:46:55,  7.41s/it]
 10%|█         | 23517/225000 [56:33:56<374:01:17,  6.68s/it]
 10%|█         | 23518/225000 [56:34:01<337:16:56,  6.03s/it]
 10%|█         | 23519/225000 [56:34:13<441:51:06,  7.89s/it]
 10%|█         | 23520/225000 [56:34:19<418:50:38,  7.48s/it]
 10%|█         | 23521/225000 [56:34:26<405:39:16,  7.25s/it]
 10%|█         | 23522/225000 [56:34:30<352:39:34,  6.30s/it]
 10%|█         | 23523/225000 [56:34:34<315:27:20,  5.64s/it]
 10%|█         | 23524/225000 [56:34:46<420:18:05,  7.51s/it]
 10%|█         | 23525/225000 [56:34:51<370:22:04,  6.62s/it]
 10%|█         | 23526/225000 [56:35:00<408:54:24,  7.31s/it]
 10%|█         | 23527/225000 [56:35:03<350:28:17,  6.26s/it]
 10%|█         | 23528/225000 [56:35:13<399:59:00,  7.15s/it]
 10%|█         | 23529/225000 [56:35:17<347:35:14,  6.21s/it]
 10%|█         | 23530/225000 [56:35:22<328:19:31,  5.87s/it]
 10%|█         | 23531/225000 [56:35:27<318:25:11,  5.69s/it]
 10%|█         | 23532/225000 [56:35:33<319:43:05,  5.71s/it]
 10%|█         | 23533/225000 [56:35:38<308:22:44,  5.51s/it]
 10%|█         | 23534/225000 [56:35:43<309:14:53,  5.53s/it]
 10%|█         | 23535/225000 [56:35:55<416:00:43,  7.43s/it]
 10%|█         | 23536/225000 [56:36:03<422:30:59,  7.55s/it]
 10%|█         | 23537/225000 [56:36:07<367:23:33,  6.57s/it]
 10%|█         | 23538/225000 [56:36:13<347:13:57,  6.20s/it]
 10%|█         | 23539/225000 [56:36:20<361:52:06,  6.47s/it]
 10%|█         | 23540/225000 [56:36:33<473:25:23,  8.46s/it]
 10%|█         | 23541/225000 [56:36:37<398:47:29,  7.13s/it]
 10%|█         | 23542/225000 [56:36:41<354:30:20,  6.33s/it]
 10%|█         | 23543/225000 [56:36:53<450:28:36,  8.05s/it]
 10%|█         | 23544/225000 [56:36:58<392:12:39,  7.01s/it]
 10%|█         | 23545/225000 [56:37:04<374:56:39,  6.70s/it]
 10%|█         | 23546/225000 [56:37:09<349:02:37,  6.24s/it]
 10%|█         | 23547/225000 [56:37:14<321:12:08,  5.74s/it]
 10%|█         | 23548/225000 [56:37:26<435:27:41,  7.78s/it]
 10%|█         | 23549/225000 [56:37:32<393:52:38,  7.04s/it]
 10%|█         | 23550/225000 [56:37:36<346:30:31,  6.19s/it]
                                                             
{'loss': '0.7334', 'grad_norm': '0.6728', 'learning_rate': '4.951e-05', 'epoch': '0.1047', 'num_input_tokens_seen': 98426832, 'train_runtime': '2.039e+05', 'train_tokens_per_second': '482.8'}

 10%|█         | 23550/225000 [56:37:36<346:30:31,  6.19s/it]
 10%|█         | 23551/225000 [56:37:42<344:23:59,  6.15s/it]
 10%|█         | 23552/225000 [56:37:51<397:40:07,  7.11s/it]
 10%|█         | 23553/225000 [56:38:00<417:35:04,  7.46s/it]
 10%|█         | 23554/225000 [56:38:04<370:18:05,  6.62s/it]
 10%|█         | 23555/225000 [56:38:09<337:44:54,  6.04s/it]
 10%|█         | 23556/225000 [56:38:14<319:24:15,  5.71s/it]
 10%|█         | 23557/225000 [56:38:20<323:16:48,  5.78s/it]
 10%|█         | 23558/225000 [56:38:33<450:44:13,  8.06s/it]
 10%|█         | 23559/225000 [56:38:37<383:42:43,  6.86s/it]
 10%|█         | 23560/225000 [56:38:47<433:31:34,  7.75s/it]
 10%|█         | 23561/225000 [56:38:53<405:40:31,  7.25s/it]
 10%|█         | 23562/225000 [56:38:57<357:06:53,  6.38s/it]
 10%|█         | 23563/225000 [56:39:02<328:14:58,  5.87s/it]
 10%|█         | 23564/225000 [56:39:08<327:41:04,  5.86s/it]
 10%|█         | 23565/225000 [56:39:14<328:36:21,  5.87s/it]
 10%|█         | 23566/225000 [56:39:24<405:34:06,  7.25s/it]
 10%|█         | 23567/225000 [56:39:31<394:04:31,  7.04s/it]
 10%|█         | 23568/225000 [56:39:35<343:47:10,  6.14s/it]
 10%|█         | 23569/225000 [56:39:45<416:49:36,  7.45s/it]
 10%|█         | 23570/225000 [56:39:51<381:14:32,  6.81s/it]
 10%|█         | 23571/225000 [56:39:55<336:48:48,  6.02s/it]
 10%|█         | 23572/225000 [56:39:59<312:36:42,  5.59s/it]
 10%|█         | 23573/225000 [56:40:05<307:05:40,  5.49s/it]
 10%|█         | 23574/225000 [56:40:11<321:42:54,  5.75s/it]
 10%|█         | 23575/225000 [56:40:25<450:12:09,  8.05s/it]
 10%|█         | 23576/225000 [56:40:30<401:45:52,  7.18s/it]
 10%|█         | 23577/225000 [56:40:39<433:10:54,  7.74s/it]
 10%|█         | 23578/225000 [56:40:43<377:43:01,  6.75s/it]
 10%|█         | 23579/225000 [56:40:48<350:40:15,  6.27s/it]
 10%|█         | 23580/225000 [56:40:53<323:00:16,  5.77s/it]
 10%|█         | 23581/225000 [56:41:02<375:37:26,  6.71s/it]
 10%|█         | 23582/225000 [56:41:11<416:51:09,  7.45s/it]
 10%|█         | 23583/225000 [56:41:15<366:26:45,  6.55s/it]
 10%|█         | 23584/225000 [56:41:21<352:00:24,  6.29s/it]
 10%|█         | 23585/225000 [56:41:27<338:28:54,  6.05s/it]
 10%|█         | 23586/225000 [56:41:32<329:36:02,  5.89s/it]
 10%|█         | 23587/225000 [56:41:37<318:28:11,  5.69s/it]
 10%|█         | 23588/225000 [56:41:50<426:59:48,  7.63s/it]
 10%|█         | 23589/225000 [56:41:56<410:44:32,  7.34s/it]
 10%|█         | 23590/225000 [56:42:01<369:01:52,  6.60s/it]
 10%|█         | 23591/225000 [56:42:07<354:37:19,  6.34s/it]
 10%|█         | 23592/225000 [56:42:12<342:03:00,  6.11s/it]
 10%|█         | 23593/225000 [56:42:24<439:37:17,  7.86s/it]
 10%|█         | 23594/225000 [56:42:29<384:08:26,  6.87s/it]
 10%|█         | 23595/225000 [56:42:34<349:53:56,  6.25s/it]
 10%|█         | 23596/225000 [56:42:39<333:37:25,  5.96s/it]
 10%|█         | 23597/225000 [56:42:48<392:21:27,  7.01s/it]
 10%|█         | 23598/225000 [56:43:00<471:19:44,  8.42s/it]
 10%|█         | 23599/225000 [56:43:05<414:47:56,  7.41s/it]
 10%|█         | 23600/225000 [56:43:10<375:23:42,  6.71s/it]
                                                             
{'loss': '0.7206', 'grad_norm': '0.733', 'learning_rate': '4.951e-05', 'epoch': '0.1049', 'num_input_tokens_seen': 98576488, 'train_runtime': '2.042e+05', 'train_tokens_per_second': '482.8'}

 10%|█         | 23600/225000 [56:43:10<375:23:42,  6.71s/it]
 10%|█         | 23601/225000 [56:43:15<342:37:36,  6.12s/it]
 10%|█         | 23602/225000 [56:43:21<335:09:59,  5.99s/it]
 10%|█         | 23603/225000 [56:43:31<402:51:39,  7.20s/it]
 10%|█         | 23604/225000 [56:43:38<403:21:20,  7.21s/it]
 10%|█         | 23605/225000 [56:43:42<345:38:33,  6.18s/it]
 10%|█         | 23606/225000 [56:43:50<376:53:57,  6.74s/it]
 10%|█         | 23607/225000 [56:43:54<334:09:35,  5.97s/it]
 10%|█         | 23608/225000 [56:44:00<330:45:39,  5.91s/it]
 10%|█         | 23609/225000 [56:44:11<418:46:47,  7.49s/it]
 10%|█         | 23610/225000 [56:44:17<398:47:22,  7.13s/it]
 10%|█         | 23611/225000 [56:44:22<362:02:17,  6.47s/it]
 10%|█         | 23612/225000 [56:44:30<380:48:24,  6.81s/it]
 10%|█         | 23613/225000 [56:44:37<386:42:28,  6.91s/it]
 10%|█         | 23614/225000 [56:44:45<413:14:36,  7.39s/it]
 10%|█         | 23615/225000 [56:44:49<348:57:20,  6.24s/it]
 10%|█         | 23616/225000 [56:44:58<397:39:21,  7.11s/it]
 10%|█         | 23617/225000 [56:45:02<343:08:43,  6.13s/it]
 10%|█         | 23618/225000 [56:45:11<398:18:46,  7.12s/it]
 10%|█         | 23619/225000 [56:45:16<363:21:19,  6.50s/it]
 10%|█         | 23620/225000 [56:45:21<337:38:05,  6.04s/it]
 10%|█         | 23621/225000 [56:45:26<314:30:32,  5.62s/it]
 10%|█         | 23622/225000 [56:45:38<419:38:31,  7.50s/it]
 10%|█         | 23623/225000 [56:45:43<378:42:51,  6.77s/it]
 10%|█         | 23624/225000 [56:45:51<395:12:37,  7.07s/it]
 10%|█         | 23625/225000 [56:45:56<366:44:54,  6.56s/it]
 11%|█         | 23626/225000 [56:46:01<340:57:53,  6.10s/it]
 11%|█         | 23627/225000 [56:46:10<382:30:18,  6.84s/it]
 11%|█         | 23628/225000 [56:46:17<389:56:45,  6.97s/it]
 11%|█         | 23629/225000 [56:46:21<344:12:42,  6.15s/it]
 11%|█         | 23630/225000 [56:46:29<378:36:06,  6.77s/it]
 11%|█         | 23631/225000 [56:46:36<378:07:14,  6.76s/it]
 11%|█         | 23632/225000 [56:46:40<337:09:38,  6.03s/it]
 11%|█         | 23633/225000 [56:46:50<400:39:13,  7.16s/it]
 11%|█         | 23634/225000 [56:46:57<391:51:55,  7.01s/it]
 11%|█         | 23635/225000 [56:47:05<407:26:01,  7.28s/it]
 11%|█         | 23636/225000 [56:47:10<364:12:48,  6.51s/it]
 11%|█         | 23637/225000 [56:47:14<337:42:45,  6.04s/it]
 11%|█         | 23638/225000 [56:47:19<318:52:42,  5.70s/it]
 11%|█         | 23639/225000 [56:47:24<305:17:02,  5.46s/it]
 11%|█         | 23640/225000 [56:47:37<425:01:35,  7.60s/it]
 11%|█         | 23641/225000 [56:47:41<374:11:30,  6.69s/it]
 11%|█         | 23642/225000 [56:47:48<372:26:46,  6.66s/it]
 11%|█         | 23643/225000 [56:47:52<329:19:17,  5.89s/it]
 11%|█         | 23644/225000 [56:48:03<413:07:03,  7.39s/it]
 11%|█         | 23645/225000 [56:48:08<372:09:09,  6.65s/it]
 11%|█         | 23646/225000 [56:48:16<391:07:10,  6.99s/it]
 11%|█         | 23647/225000 [56:48:22<386:21:16,  6.91s/it]
 11%|█         | 23648/225000 [56:48:28<362:39:32,  6.48s/it]
 11%|█         | 23649/225000 [56:48:33<339:57:48,  6.08s/it]
 11%|█         | 23650/225000 [56:48:39<331:43:07,  5.93s/it]
                                                             
{'loss': '0.7034', 'grad_norm': '0.6782', 'learning_rate': '4.951e-05', 'epoch': '0.1051', 'num_input_tokens_seen': 98725200, 'train_runtime': '2.045e+05', 'train_tokens_per_second': '482.7'}

 11%|█         | 23650/225000 [56:48:39<331:43:07,  5.93s/it]
 11%|█         | 23651/225000 [56:48:45<332:49:41,  5.95s/it]
 11%|█         | 23652/225000 [56:48:50<325:57:39,  5.83s/it]
 11%|█         | 23653/225000 [56:48:55<315:11:46,  5.64s/it]
 11%|█         | 23654/225000 [56:49:03<340:15:26,  6.08s/it]
 11%|█         | 23655/225000 [56:49:15<454:45:44,  8.13s/it]
 11%|█         | 23656/225000 [56:49:21<406:49:56,  7.27s/it]
 11%|█         | 23657/225000 [56:49:25<354:07:26,  6.33s/it]
 11%|█         | 23658/225000 [56:49:35<424:16:57,  7.59s/it]
 11%|█         | 23659/225000 [56:49:39<356:24:27,  6.37s/it]
 11%|█         | 23660/225000 [56:49:48<401:19:27,  7.18s/it]
 11%|█         | 23661/225000 [56:49:57<439:32:53,  7.86s/it]
 11%|█         | 23662/225000 [56:50:01<361:39:45,  6.47s/it]
 11%|█         | 23663/225000 [56:50:07<366:26:43,  6.55s/it]
 11%|█         | 23664/225000 [56:50:15<377:27:14,  6.75s/it]
 11%|█         | 23665/225000 [56:50:19<331:13:59,  5.92s/it]
 11%|█         | 23666/225000 [56:50:28<397:46:48,  7.11s/it]
 11%|█         | 23667/225000 [56:50:33<354:22:28,  6.34s/it]
 11%|█         | 23668/225000 [56:50:41<383:33:24,  6.86s/it]
 11%|█         | 23669/225000 [56:50:46<344:35:14,  6.16s/it]
 11%|█         | 23670/225000 [56:50:50<322:19:22,  5.76s/it]
 11%|█         | 23671/225000 [56:51:01<395:09:25,  7.07s/it]
 11%|█         | 23672/225000 [56:51:07<377:37:24,  6.75s/it]
 11%|█         | 23673/225000 [56:51:16<421:33:05,  7.54s/it]
 11%|█         | 23674/225000 [56:51:20<362:51:31,  6.49s/it]
 11%|█         | 23675/225000 [56:51:29<403:37:39,  7.22s/it]
 11%|█         | 23676/225000 [56:51:33<352:18:08,  6.30s/it]
 11%|█         | 23677/225000 [56:51:43<405:37:19,  7.25s/it]
 11%|█         | 23678/225000 [56:51:46<349:30:58,  6.25s/it]
 11%|█         | 23679/225000 [56:51:52<339:51:35,  6.08s/it]
 11%|█         | 23680/225000 [56:51:58<333:30:47,  5.96s/it]
 11%|█         | 23681/225000 [56:52:04<335:07:31,  5.99s/it]
 11%|█         | 23682/225000 [56:52:09<327:51:56,  5.86s/it]
 11%|█         | 23683/225000 [56:52:15<327:22:43,  5.85s/it]
 11%|█         | 23684/225000 [56:52:21<320:24:47,  5.73s/it]
 11%|█         | 23685/225000 [56:52:30<383:20:56,  6.86s/it]
 11%|█         | 23686/225000 [56:52:42<459:18:08,  8.21s/it]
 11%|█         | 23687/225000 [56:52:46<404:02:02,  7.23s/it]
 11%|█         | 23688/225000 [56:52:53<386:33:17,  6.91s/it]
 11%|█         | 23689/225000 [56:52:58<368:05:21,  6.58s/it]
 11%|█         | 23690/225000 [56:53:04<343:10:23,  6.14s/it]
 11%|█         | 23691/225000 [56:53:08<322:16:46,  5.76s/it]
 11%|█         | 23692/225000 [56:53:14<316:03:08,  5.65s/it]
 11%|█         | 23693/225000 [56:53:20<318:51:59,  5.70s/it]
 11%|█         | 23694/225000 [56:53:25<316:25:06,  5.66s/it]
 11%|█         | 23695/225000 [56:53:33<346:02:30,  6.19s/it]
 11%|█         | 23696/225000 [56:53:46<468:31:20,  8.38s/it]
 11%|█         | 23697/225000 [56:53:54<467:24:58,  8.36s/it]
 11%|█         | 23698/225000 [56:53:58<384:54:57,  6.88s/it]
 11%|█         | 23699/225000 [56:54:06<411:09:31,  7.35s/it]
 11%|█         | 23700/225000 [56:54:14<410:07:00,  7.33s/it]
                                                             
{'loss': '0.7062', 'grad_norm': '0.6883', 'learning_rate': '4.95e-05', 'epoch': '0.1053', 'num_input_tokens_seen': 98873192, 'train_runtime': '2.049e+05', 'train_tokens_per_second': '482.7'}

 11%|█         | 23700/225000 [56:54:14<410:07:00,  7.33s/it]
 11%|█         | 23701/225000 [56:54:18<357:56:47,  6.40s/it]
 11%|█         | 23702/225000 [56:54:27<402:28:37,  7.20s/it]
 11%|█         | 23703/225000 [56:54:33<385:59:50,  6.90s/it]
 11%|█         | 23704/225000 [56:54:38<356:17:02,  6.37s/it]
 11%|█         | 23705/225000 [56:54:44<344:40:37,  6.16s/it]
 11%|█         | 23706/225000 [56:54:50<334:52:46,  5.99s/it]
 11%|█         | 23707/225000 [56:54:55<332:05:09,  5.94s/it]
 11%|█         | 23708/225000 [56:55:01<325:35:38,  5.82s/it]
 11%|█         | 23709/225000 [56:55:06<311:58:04,  5.58s/it]
 11%|█         | 23710/225000 [56:55:16<386:11:03,  6.91s/it]
 11%|█         | 23711/225000 [56:55:26<432:17:28,  7.73s/it]
 11%|█         | 23712/225000 [56:55:30<384:54:26,  6.88s/it]
 11%|█         | 23713/225000 [56:55:35<344:40:57,  6.16s/it]
 11%|█         | 23714/225000 [56:55:40<325:43:50,  5.83s/it]
 11%|█         | 23715/225000 [56:55:47<343:06:21,  6.14s/it]
 11%|█         | 23716/225000 [56:55:59<441:43:09,  7.90s/it]
 11%|█         | 23717/225000 [56:56:02<368:57:18,  6.60s/it]
 11%|█         | 23718/225000 [56:56:11<399:24:03,  7.14s/it]
 11%|█         | 23719/225000 [56:56:16<364:22:32,  6.52s/it]
 11%|█         | 23720/225000 [56:56:24<387:45:57,  6.94s/it]
 11%|█         | 23721/225000 [56:56:28<342:38:09,  6.13s/it]
 11%|█         | 23722/225000 [56:56:37<392:33:25,  7.02s/it]
 11%|█         | 23723/225000 [56:56:42<353:51:55,  6.33s/it]
 11%|█         | 23724/225000 [56:56:46<320:57:21,  5.74s/it]
 11%|█         | 23725/225000 [56:56:57<403:23:03,  7.21s/it]
 11%|█         | 23726/225000 [56:57:02<372:36:52,  6.66s/it]
 11%|█         | 23727/225000 [56:57:08<352:08:22,  6.30s/it]
 11%|█         | 23728/225000 [56:57:13<339:16:12,  6.07s/it]
 11%|█         | 23729/225000 [56:57:19<329:34:59,  5.90s/it]
 11%|█         | 23730/225000 [56:57:24<320:39:03,  5.74s/it]
 11%|█         | 23731/225000 [56:57:37<446:47:22,  7.99s/it]
 11%|█         | 23732/225000 [56:57:46<460:47:33,  8.24s/it]
 11%|█         | 23733/225000 [56:57:53<436:10:48,  7.80s/it]
 11%|█         | 23734/225000 [56:58:01<448:14:50,  8.02s/it]
 11%|█         | 23735/225000 [56:58:05<373:36:47,  6.68s/it]
 11%|█         | 23736/225000 [56:58:13<392:20:48,  7.02s/it]
 11%|█         | 23737/225000 [56:58:19<382:18:39,  6.84s/it]
 11%|█         | 23738/225000 [56:58:24<339:36:04,  6.07s/it]
 11%|█         | 23739/225000 [56:58:33<401:29:51,  7.18s/it]
 11%|█         | 23740/225000 [56:58:38<367:42:36,  6.58s/it]
 11%|█         | 23741/225000 [56:58:47<392:08:02,  7.01s/it]
 11%|█         | 23742/225000 [56:58:51<357:08:45,  6.39s/it]
 11%|█         | 23743/225000 [56:58:57<334:36:26,  5.99s/it]
 11%|█         | 23744/225000 [56:59:04<358:44:17,  6.42s/it]
 11%|█         | 23745/225000 [56:59:15<438:14:55,  7.84s/it]
 11%|█         | 23746/225000 [56:59:19<372:47:44,  6.67s/it]
 11%|█         | 23747/225000 [56:59:28<408:58:44,  7.32s/it]
 11%|█         | 23748/225000 [56:59:32<363:10:17,  6.50s/it]
 11%|█         | 23749/225000 [56:59:37<332:35:27,  5.95s/it]
 11%|█         | 23750/225000 [56:59:43<323:20:36,  5.78s/it]
                                                             
{'loss': '0.7241', 'grad_norm': '0.5742', 'learning_rate': '4.95e-05', 'epoch': '0.1056', 'num_input_tokens_seen': 99020712, 'train_runtime': '2.052e+05', 'train_tokens_per_second': '482.6'}

 11%|█         | 23750/225000 [56:59:43<323:20:36,  5.78s/it]
 11%|█         | 23751/225000 [56:59:47<305:39:47,  5.47s/it]
 11%|█         | 23752/225000 [56:59:57<384:44:57,  6.88s/it]
 11%|█         | 23753/225000 [57:00:09<464:02:07,  8.30s/it]
 11%|█         | 23754/225000 [57:00:14<404:38:53,  7.24s/it]
 11%|█         | 23755/225000 [57:00:18<359:00:34,  6.42s/it]
 11%|█         | 23756/225000 [57:00:23<329:46:13,  5.90s/it]
 11%|█         | 23757/225000 [57:00:32<375:01:46,  6.71s/it]
 11%|█         | 23758/225000 [57:00:38<375:14:35,  6.71s/it]
 11%|█         | 23759/225000 [57:00:47<406:26:53,  7.27s/it]
 11%|█         | 23760/225000 [57:00:53<389:43:40,  6.97s/it]
 11%|█         | 23761/225000 [57:00:59<367:17:08,  6.57s/it]
 11%|█         | 23762/225000 [57:01:04<341:26:44,  6.11s/it]
 11%|█         | 23763/225000 [57:01:09<317:47:11,  5.68s/it]
 11%|█         | 23764/225000 [57:01:14<307:16:26,  5.50s/it]
 11%|█         | 23765/225000 [57:01:21<341:09:41,  6.10s/it]
 11%|█         | 23766/225000 [57:01:34<448:52:26,  8.03s/it]
 11%|█         | 23767/225000 [57:01:38<388:57:36,  6.96s/it]
 11%|█         | 23768/225000 [57:01:42<344:19:04,  6.16s/it]
 11%|█         | 23769/225000 [57:01:54<428:04:13,  7.66s/it]
 11%|█         | 23770/225000 [57:01:58<380:19:17,  6.80s/it]
 11%|█         | 23771/225000 [57:02:05<371:46:30,  6.65s/it]
 11%|█         | 23772/225000 [57:02:11<373:38:20,  6.68s/it]
 11%|█         | 23773/225000 [57:02:18<373:20:23,  6.68s/it]
 11%|█         | 23774/225000 [57:02:23<342:55:45,  6.14s/it]
 11%|█         | 23775/225000 [57:02:29<336:07:43,  6.01s/it]
 11%|█         | 23776/225000 [57:02:36<361:09:17,  6.46s/it]
 11%|█         | 23777/225000 [57:02:44<384:50:10,  6.88s/it]
 11%|█         | 23778/225000 [57:02:49<350:33:31,  6.27s/it]
 11%|█         | 23779/225000 [57:02:53<315:38:12,  5.65s/it]
 11%|█         | 23780/225000 [57:03:05<421:38:52,  7.54s/it]
 11%|█         | 23781/225000 [57:03:09<366:59:31,  6.57s/it]
 11%|█         | 23782/225000 [57:03:18<400:03:08,  7.16s/it]
 11%|█         | 23783/225000 [57:03:22<351:38:02,  6.29s/it]
 11%|█         | 23784/225000 [57:03:33<422:08:00,  7.55s/it]
 11%|█         | 23785/225000 [57:03:37<366:30:09,  6.56s/it]
 11%|█         | 23786/225000 [57:03:41<322:07:15,  5.76s/it]
 11%|█         | 23787/225000 [57:03:51<404:27:18,  7.24s/it]
 11%|█         | 23788/225000 [57:03:56<351:19:20,  6.29s/it]
 11%|█         | 23789/225000 [57:04:01<329:49:21,  5.90s/it]
 11%|█         | 23790/225000 [57:04:10<386:32:41,  6.92s/it]
 11%|█         | 23791/225000 [57:04:17<384:27:46,  6.88s/it]
 11%|█         | 23792/225000 [57:04:24<386:09:04,  6.91s/it]
 11%|█         | 23793/225000 [57:04:31<388:01:32,  6.94s/it]
 11%|█         | 23794/225000 [57:04:37<373:11:51,  6.68s/it]
 11%|█         | 23795/225000 [57:04:41<329:41:54,  5.90s/it]
 11%|█         | 23796/225000 [57:04:45<301:51:22,  5.40s/it]
 11%|█         | 23797/225000 [57:04:51<320:34:36,  5.74s/it]
 11%|█         | 23798/225000 [57:05:05<444:33:26,  7.95s/it]
 11%|█         | 23799/225000 [57:05:09<383:50:54,  6.87s/it]
 11%|█         | 23800/225000 [57:05:16<384:30:32,  6.88s/it]
                                                             
{'loss': '0.7088', 'grad_norm': '0.6366', 'learning_rate': '4.949e-05', 'epoch': '0.1058', 'num_input_tokens_seen': 99168600, 'train_runtime': '2.055e+05', 'train_tokens_per_second': '482.5'}

 11%|█         | 23800/225000 [57:05:16<384:30:32,  6.88s/it]
 11%|█         | 23801/225000 [57:05:21<352:45:57,  6.31s/it]
 11%|█         | 23802/225000 [57:05:26<331:34:22,  5.93s/it]
 11%|█         | 23803/225000 [57:05:31<319:15:19,  5.71s/it]
 11%|█         | 23804/225000 [57:05:36<309:38:43,  5.54s/it]
 11%|█         | 23805/225000 [57:05:42<307:26:30,  5.50s/it]
 11%|█         | 23806/225000 [57:05:56<447:50:59,  8.01s/it]
 11%|█         | 23807/225000 [57:06:01<410:57:20,  7.35s/it]
 11%|█         | 23808/225000 [57:06:07<376:06:21,  6.73s/it]
 11%|█         | 23809/225000 [57:06:11<339:47:30,  6.08s/it]
 11%|█         | 23810/225000 [57:06:16<325:11:06,  5.82s/it]
 11%|█         | 23811/225000 [57:06:23<346:04:20,  6.19s/it]
 11%|█         | 23812/225000 [57:06:36<456:19:20,  8.17s/it]
 11%|█         | 23813/225000 [57:06:42<420:33:59,  7.53s/it]
 11%|█         | 23814/225000 [57:06:48<389:46:48,  6.97s/it]
 11%|█         | 23815/225000 [57:06:56<403:04:27,  7.21s/it]
 11%|█         | 23816/225000 [57:07:02<382:32:20,  6.85s/it]
 11%|█         | 23817/225000 [57:07:07<348:32:20,  6.24s/it]
 11%|█         | 23818/225000 [57:07:11<321:35:17,  5.75s/it]
 11%|█         | 23819/225000 [57:07:23<427:25:07,  7.65s/it]
 11%|█         | 23820/225000 [57:07:28<387:34:03,  6.94s/it]
 11%|█         | 23821/225000 [57:07:33<346:51:47,  6.21s/it]
 11%|█         | 23822/225000 [57:07:37<310:40:54,  5.56s/it]
 11%|█         | 23823/225000 [57:07:49<419:24:11,  7.51s/it]
 11%|█         | 23824/225000 [57:07:53<357:57:14,  6.41s/it]
 11%|█         | 23825/225000 [57:08:03<411:43:48,  7.37s/it]
 11%|█         | 23826/225000 [57:08:07<355:12:40,  6.36s/it]
 11%|█         | 23827/225000 [57:08:16<404:17:14,  7.23s/it]
 11%|█         | 23828/225000 [57:08:22<380:53:39,  6.82s/it]
 11%|█         | 23829/225000 [57:08:26<340:14:19,  6.09s/it]
 11%|█         | 23830/225000 [57:08:31<318:36:01,  5.70s/it]
 11%|█         | 23831/225000 [57:08:43<427:11:22,  7.64s/it]
 11%|█         | 23832/225000 [57:08:48<374:38:17,  6.70s/it]
 11%|█         | 23833/225000 [57:08:52<336:40:03,  6.02s/it]
 11%|█         | 23834/225000 [57:08:58<336:39:26,  6.02s/it]
 11%|█         | 23835/225000 [57:09:08<406:04:35,  7.27s/it]
 11%|█         | 23836/225000 [57:09:16<417:55:31,  7.48s/it]
 11%|█         | 23837/225000 [57:09:21<371:56:14,  6.66s/it]
 11%|█         | 23838/225000 [57:09:25<336:41:54,  6.03s/it]
 11%|█         | 23839/225000 [57:09:29<301:57:45,  5.40s/it]
 11%|█         | 23840/225000 [57:09:35<298:29:02,  5.34s/it]
 11%|█         | 23841/225000 [57:09:40<300:44:53,  5.38s/it]
 11%|█         | 23842/225000 [57:09:46<316:59:07,  5.67s/it]
 11%|█         | 23843/225000 [57:10:00<445:47:58,  7.98s/it]
 11%|█         | 23844/225000 [57:10:06<411:05:35,  7.36s/it]
 11%|█         | 23845/225000 [57:10:10<367:54:33,  6.58s/it]
 11%|█         | 23846/225000 [57:10:16<346:30:43,  6.20s/it]
 11%|█         | 23847/225000 [57:10:22<342:44:01,  6.13s/it]
 11%|█         | 23848/225000 [57:10:35<462:42:10,  8.28s/it]
 11%|█         | 23849/225000 [57:10:40<399:33:29,  7.15s/it]
 11%|█         | 23850/225000 [57:10:44<352:52:07,  6.32s/it]
                                                             
{'loss': '0.7163', 'grad_norm': '0.7017', 'learning_rate': '4.949e-05', 'epoch': '0.106', 'num_input_tokens_seen': 99315552, 'train_runtime': '2.058e+05', 'train_tokens_per_second': '482.5'}

 11%|█         | 23850/225000 [57:10:44<352:52:07,  6.32s/it]
 11%|█         | 23851/225000 [57:10:49<328:13:47,  5.87s/it]
 11%|█         | 23852/225000 [57:11:00<416:22:09,  7.45s/it]
 11%|█         | 23853/225000 [57:11:06<388:45:31,  6.96s/it]
 11%|█         | 23854/225000 [57:11:13<398:03:36,  7.12s/it]
 11%|█         | 23855/225000 [57:11:17<349:56:18,  6.26s/it]
 11%|█         | 23856/225000 [57:11:26<383:29:21,  6.86s/it]
 11%|█         | 23857/225000 [57:11:31<352:13:28,  6.30s/it]
 11%|█         | 23858/225000 [57:11:35<324:55:44,  5.82s/it]
 11%|█         | 23859/225000 [57:11:41<328:32:05,  5.88s/it]
 11%|█         | 23860/225000 [57:11:54<433:51:55,  7.77s/it]
 11%|█         | 23861/225000 [57:11:57<366:02:19,  6.55s/it]
 11%|█         | 23862/225000 [57:12:07<424:36:18,  7.60s/it]
 11%|█         | 23863/225000 [57:12:12<378:01:31,  6.77s/it]
 11%|█         | 23864/225000 [57:12:17<346:47:45,  6.21s/it]
 11%|█         | 23865/225000 [57:12:23<341:06:58,  6.11s/it]
 11%|█         | 23866/225000 [57:12:29<333:00:43,  5.96s/it]
 11%|█         | 23867/225000 [57:12:34<327:26:03,  5.86s/it]
 11%|█         | 23868/225000 [57:12:40<330:21:06,  5.91s/it]
 11%|█         | 23869/225000 [57:12:46<322:41:25,  5.78s/it]
 11%|█         | 23870/225000 [57:12:57<410:07:28,  7.34s/it]
 11%|█         | 23871/225000 [57:13:07<457:44:24,  8.19s/it]
 11%|█         | 23872/225000 [57:13:11<395:02:30,  7.07s/it]
 11%|█         | 23873/225000 [57:13:16<354:18:05,  6.34s/it]
 11%|█         | 23874/225000 [57:13:20<319:20:01,  5.72s/it]
 11%|█         | 23875/225000 [57:13:33<436:15:08,  7.81s/it]
 11%|█         | 23876/225000 [57:13:40<422:55:49,  7.57s/it]
 11%|█         | 23877/225000 [57:13:44<370:26:47,  6.63s/it]
 11%|█         | 23878/225000 [57:13:48<324:22:04,  5.81s/it]
 11%|█         | 23879/225000 [57:13:53<300:11:34,  5.37s/it]
 11%|█         | 23880/225000 [57:14:04<395:38:47,  7.08s/it]
 11%|█         | 23881/225000 [57:14:10<378:52:34,  6.78s/it]
 11%|█         | 23882/225000 [57:14:15<349:23:50,  6.25s/it]
 11%|█         | 23883/225000 [57:14:24<398:29:57,  7.13s/it]
 11%|█         | 23884/225000 [57:14:29<369:56:37,  6.62s/it]
 11%|█         | 23885/225000 [57:14:34<330:34:23,  5.92s/it]
 11%|█         | 23886/225000 [57:14:38<309:02:02,  5.53s/it]
 11%|█         | 23887/225000 [57:14:50<405:02:09,  7.25s/it]
 11%|█         | 23888/225000 [57:14:56<399:24:54,  7.15s/it]
 11%|█         | 23889/225000 [57:15:02<364:43:09,  6.53s/it]
 11%|█         | 23890/225000 [57:15:06<329:06:20,  5.89s/it]
 11%|█         | 23891/225000 [57:15:18<429:02:07,  7.68s/it]
 11%|█         | 23892/225000 [57:15:22<378:37:48,  6.78s/it]
 11%|█         | 23893/225000 [57:15:27<344:42:50,  6.17s/it]
 11%|█         | 23894/225000 [57:15:32<320:19:51,  5.73s/it]
 11%|█         | 23895/225000 [57:15:42<400:40:55,  7.17s/it]
 11%|█         | 23896/225000 [57:15:50<410:43:06,  7.35s/it]
 11%|█         | 23897/225000 [57:15:55<366:32:19,  6.56s/it]
 11%|█         | 23898/225000 [57:16:00<337:57:26,  6.05s/it]
 11%|█         | 23899/225000 [57:16:10<403:49:25,  7.23s/it]
 11%|█         | 23900/225000 [57:16:14<353:37:09,  6.33s/it]
                                                             
{'loss': '0.7343', 'grad_norm': '0.6825', 'learning_rate': '4.949e-05', 'epoch': '0.1062', 'num_input_tokens_seen': 99463416, 'train_runtime': '2.062e+05', 'train_tokens_per_second': '482.4'}

 11%|█         | 23900/225000 [57:16:14<353:37:09,  6.33s/it]
 11%|█         | 23901/225000 [57:16:18<315:09:11,  5.64s/it]
 11%|█         | 23902/225000 [57:16:26<358:22:59,  6.42s/it]
 11%|█         | 23903/225000 [57:16:36<414:42:50,  7.42s/it]
 11%|█         | 23904/225000 [57:16:40<350:37:42,  6.28s/it]
 11%|█         | 23905/225000 [57:16:50<426:38:45,  7.64s/it]
 11%|█         | 23906/225000 [57:16:54<363:41:29,  6.51s/it]
 11%|█         | 23907/225000 [57:17:00<351:02:18,  6.28s/it]
 11%|█         | 23908/225000 [57:17:09<389:57:06,  6.98s/it]
 11%|█         | 23909/225000 [57:17:13<346:32:26,  6.20s/it]
 11%|█         | 23910/225000 [57:17:23<415:56:28,  7.45s/it]
 11%|█         | 23911/225000 [57:17:28<362:15:58,  6.49s/it]
 11%|█         | 23912/225000 [57:17:32<321:07:50,  5.75s/it]
 11%|█         | 23913/225000 [57:17:39<355:00:25,  6.36s/it]
 11%|█         | 23914/225000 [57:17:48<384:09:38,  6.88s/it]
 11%|█         | 23915/225000 [57:17:56<409:29:02,  7.33s/it]
 11%|█         | 23916/225000 [57:18:01<372:44:24,  6.67s/it]
 11%|█         | 23917/225000 [57:18:06<344:03:17,  6.16s/it]
 11%|█         | 23918/225000 [57:18:11<323:06:57,  5.78s/it]
 11%|█         | 23919/225000 [57:18:18<344:59:58,  6.18s/it]
 11%|█         | 23920/225000 [57:18:27<394:20:48,  7.06s/it]
 11%|█         | 23921/225000 [57:18:32<352:33:20,  6.31s/it]
 11%|█         | 23922/225000 [57:18:39<368:17:19,  6.59s/it]
 11%|█         | 23923/225000 [57:18:49<425:18:48,  7.61s/it]
 11%|█         | 23924/225000 [57:18:53<365:52:35,  6.55s/it]
 11%|█         | 23925/225000 [57:18:58<335:34:37,  6.01s/it]
 11%|█         | 23926/225000 [57:19:03<323:26:07,  5.79s/it]
 11%|█         | 23927/225000 [57:19:08<304:34:07,  5.45s/it]
 11%|█         | 23928/225000 [57:19:16<353:50:09,  6.34s/it]
 11%|█         | 23929/225000 [57:19:29<457:31:56,  8.19s/it]
 11%|█         | 23930/225000 [57:19:33<389:19:58,  6.97s/it]
 11%|█         | 23931/225000 [57:19:37<340:55:35,  6.10s/it]
 11%|█         | 23932/225000 [57:19:45<377:42:17,  6.76s/it]
 11%|█         | 23933/225000 [57:19:50<352:59:14,  6.32s/it]
 11%|█         | 23934/225000 [57:20:00<399:46:36,  7.16s/it]
 11%|█         | 23935/225000 [57:20:05<370:49:29,  6.64s/it]
 11%|█         | 23936/225000 [57:20:10<343:21:25,  6.15s/it]
 11%|█         | 23937/225000 [57:20:15<325:22:07,  5.83s/it]
 11%|█         | 23938/225000 [57:20:20<317:28:21,  5.68s/it]
 11%|█         | 23939/225000 [57:20:31<395:46:44,  7.09s/it]
 11%|█         | 23940/225000 [57:20:42<459:43:03,  8.23s/it]
 11%|█         | 23941/225000 [57:20:48<426:07:16,  7.63s/it]
 11%|█         | 23942/225000 [57:20:53<375:11:58,  6.72s/it]
 11%|█         | 23943/225000 [57:21:01<409:10:16,  7.33s/it]
 11%|█         | 23944/225000 [57:21:08<399:14:57,  7.15s/it]
 11%|█         | 23945/225000 [57:21:16<406:16:20,  7.27s/it]
 11%|█         | 23946/225000 [57:21:20<363:00:23,  6.50s/it]
 11%|█         | 23947/225000 [57:21:25<330:36:35,  5.92s/it]
 11%|█         | 23948/225000 [57:21:30<314:09:45,  5.63s/it]
 11%|█         | 23949/225000 [57:21:35<309:00:29,  5.53s/it]
 11%|█         | 23950/225000 [57:21:48<436:58:28,  7.82s/it]
                                                             
{'loss': '0.737', 'grad_norm': '0.5802', 'learning_rate': '4.948e-05', 'epoch': '0.1064', 'num_input_tokens_seen': 99611568, 'train_runtime': '2.065e+05', 'train_tokens_per_second': '482.4'}

 11%|█         | 23950/225000 [57:21:48<436:58:28,  7.82s/it]
 11%|█         | 23951/225000 [57:21:53<379:25:29,  6.79s/it]
 11%|█         | 23952/225000 [57:21:59<371:31:11,  6.65s/it]
 11%|█         | 23953/225000 [57:22:06<377:21:22,  6.76s/it]
 11%|█         | 23954/225000 [57:22:13<385:37:51,  6.91s/it]
 11%|█         | 23955/225000 [57:22:18<350:28:18,  6.28s/it]
 11%|█         | 23956/225000 [57:22:23<322:19:58,  5.77s/it]
 11%|█         | 23957/225000 [57:22:29<336:37:49,  6.03s/it]
 11%|█         | 23958/225000 [57:22:41<438:00:01,  7.84s/it]
 11%|█         | 23959/225000 [57:22:45<372:40:40,  6.67s/it]
 11%|█         | 23960/225000 [57:22:53<394:21:15,  7.06s/it]
 11%|█         | 23961/225000 [57:22:57<347:00:57,  6.21s/it]
 11%|█         | 23962/225000 [57:23:07<403:25:13,  7.22s/it]
 11%|█         | 23963/225000 [57:23:13<379:50:19,  6.80s/it]
 11%|█         | 23964/225000 [57:23:21<394:50:51,  7.07s/it]
 11%|█         | 23965/225000 [57:23:27<386:50:59,  6.93s/it]
 11%|█         | 23966/225000 [57:23:31<335:42:50,  6.01s/it]
 11%|█         | 23967/225000 [57:23:40<386:05:16,  6.91s/it]
 11%|█         | 23968/225000 [57:23:44<331:26:49,  5.94s/it]
 11%|█         | 23969/225000 [57:23:52<371:51:09,  6.66s/it]
 11%|█         | 23970/225000 [57:23:59<379:46:10,  6.80s/it]
 11%|█         | 23971/225000 [57:24:07<400:43:26,  7.18s/it]
 11%|█         | 23972/225000 [57:24:12<362:20:58,  6.49s/it]
 11%|█         | 23973/225000 [57:24:20<388:38:12,  6.96s/it]
 11%|█         | 23974/225000 [57:24:25<350:56:35,  6.28s/it]
 11%|█         | 23975/225000 [57:24:34<394:35:56,  7.07s/it]
 11%|█         | 23976/225000 [57:24:38<353:42:57,  6.33s/it]
 11%|█         | 23977/225000 [57:24:43<324:53:38,  5.82s/it]
 11%|█         | 23978/225000 [57:24:48<306:54:27,  5.50s/it]
 11%|█         | 23979/225000 [57:24:53<294:14:54,  5.27s/it]
 11%|█         | 23980/225000 [57:25:01<352:44:34,  6.32s/it]
 11%|█         | 23981/225000 [57:25:12<418:32:56,  7.50s/it]
 11%|█         | 23982/225000 [57:25:16<361:30:54,  6.47s/it]
 11%|█         | 23983/225000 [57:25:20<326:37:46,  5.85s/it]
 11%|█         | 23984/225000 [57:25:32<434:08:47,  7.78s/it]
 11%|█         | 23985/225000 [57:25:37<374:39:43,  6.71s/it]
 11%|█         | 23986/225000 [57:25:42<347:00:17,  6.21s/it]
 11%|█         | 23987/225000 [57:25:47<326:35:51,  5.85s/it]
 11%|█         | 23988/225000 [57:25:54<359:00:45,  6.43s/it]
 11%|█         | 23989/225000 [57:26:07<461:40:34,  8.27s/it]
 11%|█         | 23990/225000 [57:26:11<384:41:38,  6.89s/it]
 11%|█         | 23991/225000 [57:26:18<401:03:51,  7.18s/it]
 11%|█         | 23992/225000 [57:26:22<345:18:40,  6.18s/it]
 11%|█         | 23993/225000 [57:26:31<391:26:50,  7.01s/it]
 11%|█         | 23994/225000 [57:26:37<365:22:36,  6.54s/it]
 11%|█         | 23995/225000 [57:26:42<337:40:42,  6.05s/it]
 11%|█         | 23996/225000 [57:26:46<316:20:24,  5.67s/it]
 11%|█         | 23997/225000 [57:26:52<317:02:46,  5.68s/it]
 11%|█         | 23998/225000 [57:27:04<419:33:37,  7.51s/it]
 11%|█         | 23999/225000 [57:27:08<360:05:36,  6.45s/it]
 11%|█         | 24000/225000 [57:27:15<379:36:30,  6.80s/it]
                                                             
{'loss': '0.7237', 'grad_norm': '0.8976', 'learning_rate': '4.948e-05', 'epoch': '0.1067', 'num_input_tokens_seen': 99758656, 'train_runtime': '2.068e+05', 'train_tokens_per_second': '482.3'}

 11%|█         | 24000/225000 [57:27:15<379:36:30,  6.80s/it]
 11%|█         | 24001/225000 [57:27:21<354:14:02,  6.34s/it]
 11%|█         | 24002/225000 [57:27:29<380:36:01,  6.82s/it]
 11%|█         | 24003/225000 [57:27:33<339:28:54,  6.08s/it]
 11%|█         | 24004/225000 [57:27:38<314:22:11,  5.63s/it]
 11%|█         | 24005/225000 [57:27:42<296:20:26,  5.31s/it]
 11%|█         | 24006/225000 [57:27:50<335:37:05,  6.01s/it]
 11%|█         | 24007/225000 [57:28:01<422:52:53,  7.57s/it]
 11%|█         | 24008/225000 [57:28:08<409:54:16,  7.34s/it]
 11%|█         | 24009/225000 [57:28:16<427:54:16,  7.66s/it]
 11%|█         | 24010/225000 [57:28:19<353:11:42,  6.33s/it]
 11%|█         | 24011/225000 [57:28:24<319:01:37,  5.71s/it]
 11%|█         | 24012/225000 [57:28:30<329:22:29,  5.90s/it]
 11%|█         | 24013/225000 [57:28:40<404:14:08,  7.24s/it]
 11%|█         | 24014/225000 [57:28:44<347:49:19,  6.23s/it]
 11%|█         | 24015/225000 [57:28:49<326:30:52,  5.85s/it]
 11%|█         | 24016/225000 [57:28:54<307:45:00,  5.51s/it]
 11%|█         | 24017/225000 [57:28:59<302:32:53,  5.42s/it]
 11%|█         | 24018/225000 [57:29:11<417:56:24,  7.49s/it]
 11%|█         | 24019/225000 [57:29:18<396:24:48,  7.10s/it]
 11%|█         | 24020/225000 [57:29:27<440:39:47,  7.89s/it]
 11%|█         | 24021/225000 [57:29:31<374:19:35,  6.71s/it]
 11%|█         | 24022/225000 [57:29:40<399:52:33,  7.16s/it]
 11%|█         | 24023/225000 [57:29:44<354:48:33,  6.36s/it]
 11%|█         | 24024/225000 [57:29:49<332:49:22,  5.96s/it]
 11%|█         | 24025/225000 [57:29:54<315:03:02,  5.64s/it]
 11%|█         | 24026/225000 [57:30:00<319:52:52,  5.73s/it]
 11%|█         | 24027/225000 [57:30:14<459:32:35,  8.23s/it]
 11%|█         | 24028/225000 [57:30:20<422:45:14,  7.57s/it]
 11%|█         | 24029/225000 [57:30:25<384:25:45,  6.89s/it]
 11%|█         | 24030/225000 [57:30:30<349:13:05,  6.26s/it]
 11%|█         | 24031/225000 [57:30:36<341:32:30,  6.12s/it]
 11%|█         | 24032/225000 [57:30:41<328:13:52,  5.88s/it]
 11%|█         | 24033/225000 [57:30:47<326:54:35,  5.86s/it]
 11%|█         | 24034/225000 [57:30:53<322:17:50,  5.77s/it]
 11%|█         | 24035/225000 [57:30:58<312:59:40,  5.61s/it]
 11%|█         | 24036/225000 [57:31:05<344:47:39,  6.18s/it]
 11%|█         | 24037/225000 [57:31:18<461:18:59,  8.26s/it]
 11%|█         | 24038/225000 [57:31:24<418:38:52,  7.50s/it]
 11%|█         | 24039/225000 [57:31:34<460:14:49,  8.24s/it]
 11%|█         | 24040/225000 [57:31:39<398:08:00,  7.13s/it]
 11%|█         | 24041/225000 [57:31:43<347:42:19,  6.23s/it]
 11%|█         | 24042/225000 [57:31:48<324:58:10,  5.82s/it]
 11%|█         | 24043/225000 [57:32:01<445:46:23,  7.99s/it]
 11%|█         | 24044/225000 [57:32:06<394:13:00,  7.06s/it]
 11%|█         | 24045/225000 [57:32:10<354:05:46,  6.34s/it]
 11%|█         | 24046/225000 [57:32:21<419:13:54,  7.51s/it]
 11%|█         | 24047/225000 [57:32:24<356:43:46,  6.39s/it]
 11%|█         | 24048/225000 [57:32:28<311:04:20,  5.57s/it]
 11%|█         | 24049/225000 [57:32:40<411:23:19,  7.37s/it]
 11%|█         | 24050/225000 [57:32:44<358:37:25,  6.42s/it]
                                                             
{'loss': '0.7163', 'grad_norm': '0.6877', 'learning_rate': '4.948e-05', 'epoch': '0.1069', 'num_input_tokens_seen': 99905088, 'train_runtime': '2.072e+05', 'train_tokens_per_second': '482.3'}

 11%|█         | 24050/225000 [57:32:44<358:37:25,  6.42s/it]
 11%|█         | 24051/225000 [57:32:51<368:45:52,  6.61s/it]
 11%|█         | 24052/225000 [57:32:59<403:08:49,  7.22s/it]
 11%|█         | 24053/225000 [57:33:04<362:09:51,  6.49s/it]
 11%|█         | 24054/225000 [57:33:09<332:35:28,  5.96s/it]
 11%|█         | 24055/225000 [57:33:13<308:47:28,  5.53s/it]
 11%|█         | 24056/225000 [57:33:19<302:14:05,  5.41s/it]
 11%|█         | 24057/225000 [57:33:29<385:04:34,  6.90s/it]
 11%|█         | 24058/225000 [57:33:39<440:16:25,  7.89s/it]
 11%|█         | 24059/225000 [57:33:43<373:39:43,  6.69s/it]
 11%|█         | 24060/225000 [57:33:48<343:11:12,  6.15s/it]
 11%|█         | 24061/225000 [57:33:53<324:50:20,  5.82s/it]
 11%|█         | 24062/225000 [57:33:59<331:48:47,  5.94s/it]
 11%|█         | 24063/225000 [57:34:10<408:18:21,  7.32s/it]
 11%|█         | 24064/225000 [57:34:20<453:30:16,  8.13s/it]
 11%|█         | 24065/225000 [57:34:24<387:30:37,  6.94s/it]
 11%|█         | 24066/225000 [57:34:29<358:50:10,  6.43s/it]
 11%|█         | 24067/225000 [57:34:35<342:05:01,  6.13s/it]
 11%|█         | 24068/225000 [57:34:40<321:15:21,  5.76s/it]
 11%|█         | 24069/225000 [57:34:48<372:26:23,  6.67s/it]
 11%|█         | 24070/225000 [57:35:00<462:24:36,  8.28s/it]
 11%|█         | 24071/225000 [57:35:05<393:37:16,  7.05s/it]
 11%|█         | 24072/225000 [57:35:09<345:49:52,  6.20s/it]
 11%|█         | 24073/225000 [57:35:20<434:17:59,  7.78s/it]
 11%|█         | 24074/225000 [57:35:24<365:11:34,  6.54s/it]
 11%|█         | 24075/225000 [57:35:33<408:09:38,  7.31s/it]
 11%|█         | 24076/225000 [57:35:37<356:13:25,  6.38s/it]
 11%|█         | 24077/225000 [57:35:42<324:58:01,  5.82s/it]
 11%|█         | 24078/225000 [57:35:47<322:08:48,  5.77s/it]
 11%|█         | 24079/225000 [57:36:00<441:34:18,  7.91s/it]
 11%|█         | 24080/225000 [57:36:04<373:10:02,  6.69s/it]
 11%|█         | 24081/225000 [57:36:14<422:50:55,  7.58s/it]
 11%|█         | 24082/225000 [57:36:17<357:39:34,  6.41s/it]
 11%|█         | 24083/225000 [57:36:23<344:55:37,  6.18s/it]
 11%|█         | 24084/225000 [57:36:33<399:01:05,  7.15s/it]
 11%|█         | 24085/225000 [57:36:37<359:21:45,  6.44s/it]
 11%|█         | 24086/225000 [57:36:47<414:02:43,  7.42s/it]
 11%|█         | 24087/225000 [57:36:58<477:36:13,  8.56s/it]
 11%|█         | 24088/225000 [57:37:02<393:45:28,  7.06s/it]
 11%|█         | 24089/225000 [57:37:09<389:21:10,  6.98s/it]
 11%|█         | 24090/225000 [57:37:15<384:19:08,  6.89s/it]
 11%|█         | 24091/225000 [57:37:23<392:32:23,  7.03s/it]
 11%|█         | 24092/225000 [57:37:29<387:32:23,  6.94s/it]
 11%|█         | 24093/225000 [57:37:34<344:18:59,  6.17s/it]
 11%|█         | 24094/225000 [57:37:38<319:12:53,  5.72s/it]
 11%|█         | 24095/225000 [57:37:46<357:09:26,  6.40s/it]
 11%|█         | 24096/225000 [57:37:55<393:35:13,  7.05s/it]
 11%|█         | 24097/225000 [57:38:03<405:34:26,  7.27s/it]
 11%|█         | 24098/225000 [57:38:07<351:57:53,  6.31s/it]
 11%|█         | 24099/225000 [57:38:11<315:16:29,  5.65s/it]
 11%|█         | 24100/225000 [57:38:22<405:27:45,  7.27s/it]
                                                             
{'loss': '0.7034', 'grad_norm': '0.7585', 'learning_rate': '4.947e-05', 'epoch': '0.1071', 'num_input_tokens_seen': 100052144, 'train_runtime': '2.075e+05', 'train_tokens_per_second': '482.2'}

 11%|█         | 24100/225000 [57:38:22<405:27:45,  7.27s/it]
 11%|█         | 24101/225000 [57:38:30<411:06:26,  7.37s/it]
 11%|█         | 24102/225000 [57:38:34<369:44:35,  6.63s/it]
 11%|█         | 24103/225000 [57:38:43<406:59:16,  7.29s/it]
 11%|█         | 24104/225000 [57:38:48<364:36:59,  6.53s/it]
 11%|█         | 24105/225000 [57:38:55<371:55:17,  6.66s/it]
 11%|█         | 24106/225000 [57:39:03<389:34:29,  6.98s/it]
 11%|█         | 24107/225000 [57:39:07<338:01:04,  6.06s/it]
 11%|█         | 24108/225000 [57:39:16<386:34:41,  6.93s/it]
 11%|█         | 24109/225000 [57:39:20<350:37:36,  6.28s/it]
 11%|█         | 24110/225000 [57:39:25<330:57:53,  5.93s/it]
 11%|█         | 24111/225000 [57:39:31<317:06:20,  5.68s/it]
 11%|█         | 24112/225000 [57:39:35<303:20:48,  5.44s/it]
 11%|█         | 24113/225000 [57:39:40<294:14:40,  5.27s/it]
 11%|█         | 24114/225000 [57:39:50<361:20:50,  6.48s/it]
 11%|█         | 24115/225000 [57:40:02<459:25:59,  8.23s/it]
 11%|█         | 24116/225000 [57:40:06<384:51:06,  6.90s/it]
 11%|█         | 24117/225000 [57:40:15<422:26:54,  7.57s/it]
 11%|█         | 24118/225000 [57:40:20<377:45:49,  6.77s/it]
 11%|█         | 24119/225000 [57:40:25<350:44:03,  6.29s/it]
 11%|█         | 24120/225000 [57:40:30<327:13:32,  5.86s/it]
 11%|█         | 24121/225000 [57:40:34<297:56:35,  5.34s/it]
 11%|█         | 24122/225000 [57:40:40<309:52:32,  5.55s/it]
 11%|█         | 24123/225000 [57:40:52<422:10:58,  7.57s/it]
 11%|█         | 24124/225000 [57:40:58<395:41:50,  7.09s/it]
 11%|█         | 24125/225000 [57:41:03<362:05:18,  6.49s/it]
 11%|█         | 24126/225000 [57:41:08<328:56:03,  5.90s/it]
 11%|█         | 24127/225000 [57:41:12<307:19:06,  5.51s/it]
 11%|█         | 24128/225000 [57:41:25<432:17:04,  7.75s/it]
 11%|█         | 24129/225000 [57:41:30<383:16:10,  6.87s/it]
 11%|█         | 24130/225000 [57:41:34<332:05:03,  5.95s/it]
 11%|█         | 24131/225000 [57:41:46<430:11:07,  7.71s/it]
 11%|█         | 24132/225000 [57:41:49<355:48:27,  6.38s/it]
 11%|█         | 24133/225000 [57:41:59<414:13:24,  7.42s/it]
 11%|█         | 24134/225000 [57:42:03<361:47:12,  6.48s/it]
 11%|█         | 24135/225000 [57:42:09<357:06:21,  6.40s/it]
 11%|█         | 24136/225000 [57:42:17<369:59:38,  6.63s/it]
 11%|█         | 24137/225000 [57:42:22<345:11:45,  6.19s/it]
 11%|█         | 24138/225000 [57:42:27<331:41:16,  5.94s/it]
 11%|█         | 24139/225000 [57:42:32<319:49:58,  5.73s/it]
 11%|█         | 24140/225000 [57:42:38<323:54:10,  5.81s/it]
 11%|█         | 24141/225000 [57:42:44<315:26:23,  5.65s/it]
 11%|█         | 24142/225000 [57:42:56<429:36:39,  7.70s/it]
 11%|█         | 24143/225000 [57:43:03<407:34:46,  7.31s/it]
 11%|█         | 24144/225000 [57:43:10<408:23:39,  7.32s/it]
 11%|█         | 24145/225000 [57:43:16<382:58:07,  6.86s/it]
 11%|█         | 24146/225000 [57:43:24<410:50:16,  7.36s/it]
 11%|█         | 24147/225000 [57:43:30<381:16:18,  6.83s/it]
 11%|█         | 24148/225000 [57:43:35<355:54:16,  6.38s/it]
 11%|█         | 24149/225000 [57:43:46<423:22:29,  7.59s/it]
 11%|█         | 24150/225000 [57:43:49<361:02:23,  6.47s/it]
                                                             
{'loss': '0.7167', 'grad_norm': '0.6608', 'learning_rate': '4.947e-05', 'epoch': '0.1073', 'num_input_tokens_seen': 100197616, 'train_runtime': '2.078e+05', 'train_tokens_per_second': '482.1'}

 11%|█         | 24150/225000 [57:43:49<361:02:23,  6.47s/it]
 11%|█         | 24151/225000 [57:43:55<339:28:24,  6.08s/it]
 11%|█         | 24152/225000 [57:44:03<374:11:15,  6.71s/it]
 11%|█         | 24153/225000 [57:44:07<336:40:30,  6.03s/it]
 11%|█         | 24154/225000 [57:44:12<317:19:32,  5.69s/it]
 11%|█         | 24155/225000 [57:44:23<403:47:01,  7.24s/it]
 11%|█         | 24156/225000 [57:44:29<386:32:19,  6.93s/it]
 11%|█         | 24157/225000 [57:44:35<361:15:26,  6.48s/it]
 11%|█         | 24158/225000 [57:44:43<398:14:45,  7.14s/it]
 11%|█         | 24159/225000 [57:44:48<356:35:19,  6.39s/it]
 11%|█         | 24160/225000 [57:44:53<330:12:22,  5.92s/it]
 11%|█         | 24161/225000 [57:45:04<425:56:48,  7.64s/it]
 11%|█         | 24162/225000 [57:45:09<372:48:04,  6.68s/it]
 11%|█         | 24163/225000 [57:45:15<367:09:49,  6.58s/it]
 11%|█         | 24164/225000 [57:45:20<341:50:07,  6.13s/it]
 11%|█         | 24165/225000 [57:45:25<312:40:09,  5.60s/it]
 11%|█         | 24166/225000 [57:45:29<292:43:32,  5.25s/it]
 11%|█         | 24167/225000 [57:45:34<283:50:33,  5.09s/it]
 11%|█         | 24168/225000 [57:45:46<411:27:33,  7.38s/it]
 11%|█         | 24169/225000 [57:45:52<386:46:22,  6.93s/it]
 11%|█         | 24170/225000 [57:45:59<376:49:06,  6.75s/it]
 11%|█         | 24171/225000 [57:46:02<325:56:26,  5.84s/it]
 11%|█         | 24172/225000 [57:46:12<391:43:43,  7.02s/it]
 11%|█         | 24173/225000 [57:46:17<358:50:48,  6.43s/it]
 11%|█         | 24174/225000 [57:46:22<335:33:58,  6.02s/it]
 11%|█         | 24175/225000 [57:46:27<320:38:50,  5.75s/it]
 11%|█         | 24176/225000 [57:46:32<309:07:45,  5.54s/it]
 11%|█         | 24177/225000 [57:46:38<308:43:19,  5.53s/it]
 11%|█         | 24178/225000 [57:46:43<302:59:32,  5.43s/it]
 11%|█         | 24179/225000 [57:46:49<301:08:19,  5.40s/it]
 11%|█         | 24180/225000 [57:46:58<364:02:01,  6.53s/it]
 11%|█         | 24181/225000 [57:47:10<465:48:49,  8.35s/it]
 11%|█         | 24182/225000 [57:47:14<385:55:37,  6.92s/it]
 11%|█         | 24183/225000 [57:47:24<431:41:38,  7.74s/it]
 11%|█         | 24184/225000 [57:47:27<364:00:06,  6.53s/it]
 11%|█         | 24185/225000 [57:47:38<432:02:52,  7.75s/it]
 11%|█         | 24186/225000 [57:47:42<376:26:41,  6.75s/it]
 11%|█         | 24187/225000 [57:47:47<335:24:35,  6.01s/it]
 11%|█         | 24188/225000 [57:47:58<428:20:52,  7.68s/it]
 11%|█         | 24189/225000 [57:48:04<406:04:44,  7.28s/it]
 11%|█         | 24190/225000 [57:48:09<361:47:13,  6.49s/it]
 11%|█         | 24191/225000 [57:48:14<340:17:50,  6.10s/it]
 11%|█         | 24192/225000 [57:48:20<326:08:45,  5.85s/it]
 11%|█         | 24193/225000 [57:48:25<313:13:27,  5.62s/it]
 11%|█         | 24194/225000 [57:48:30<304:39:52,  5.46s/it]
 11%|█         | 24195/225000 [57:48:35<297:36:42,  5.34s/it]
 11%|█         | 24196/225000 [57:48:46<400:16:14,  7.18s/it]
 11%|█         | 24197/225000 [57:48:55<434:12:32,  7.78s/it]
 11%|█         | 24198/225000 [57:49:01<390:30:27,  7.00s/it]
 11%|█         | 24199/225000 [57:49:06<361:26:08,  6.48s/it]
 11%|█         | 24200/225000 [57:49:11<346:11:49,  6.21s/it]
                                                             
{'loss': '0.7077', 'grad_norm': '0.7281', 'learning_rate': '4.946e-05', 'epoch': '0.1076', 'num_input_tokens_seen': 100343048, 'train_runtime': '2.082e+05', 'train_tokens_per_second': '482.1'}

 11%|█         | 24200/225000 [57:49:11<346:11:49,  6.21s/it]
 11%|█         | 24201/225000 [57:49:17<328:22:07,  5.89s/it]
 11%|█         | 24202/225000 [57:49:21<310:33:07,  5.57s/it]
 11%|█         | 24203/225000 [57:49:27<308:21:08,  5.53s/it]
 11%|█         | 24204/225000 [57:49:32<307:50:00,  5.52s/it]
 11%|█         | 24205/225000 [57:49:39<326:57:32,  5.86s/it]
 11%|█         | 24206/225000 [57:49:52<450:25:53,  8.08s/it]
 11%|█         | 24207/225000 [57:49:58<419:35:49,  7.52s/it]
 11%|█         | 24208/225000 [57:50:06<412:44:40,  7.40s/it]
 11%|█         | 24209/225000 [57:50:10<370:19:58,  6.64s/it]
 11%|█         | 24210/225000 [57:50:16<349:17:43,  6.26s/it]
 11%|█         | 24211/225000 [57:50:21<329:59:49,  5.92s/it]
 11%|█         | 24212/225000 [57:50:33<437:32:14,  7.84s/it]
 11%|█         | 24213/225000 [57:50:39<405:52:15,  7.28s/it]
 11%|█         | 24214/225000 [57:50:45<378:48:08,  6.79s/it]
 11%|█         | 24215/225000 [57:50:50<356:36:23,  6.39s/it]
 11%|█         | 24216/225000 [57:50:56<341:30:46,  6.12s/it]
 11%|█         | 24217/225000 [57:51:02<337:51:50,  6.06s/it]
 11%|█         | 24218/225000 [57:51:10<375:42:13,  6.74s/it]
 11%|█         | 24219/225000 [57:51:18<388:26:18,  6.96s/it]
 11%|█         | 24220/225000 [57:51:22<347:39:46,  6.23s/it]
 11%|█         | 24221/225000 [57:51:26<313:30:47,  5.62s/it]
 11%|█         | 24222/225000 [57:51:39<424:30:52,  7.61s/it]
 11%|█         | 24223/225000 [57:51:43<373:37:09,  6.70s/it]
 11%|█         | 24224/225000 [57:51:48<342:18:51,  6.14s/it]
 11%|█         | 24225/225000 [57:51:53<326:47:28,  5.86s/it]
 11%|█         | 24226/225000 [57:51:58<313:47:25,  5.63s/it]
 11%|█         | 24227/225000 [57:52:03<305:25:24,  5.48s/it]
 11%|█         | 24228/225000 [57:52:09<308:24:36,  5.53s/it]
 11%|█         | 24229/225000 [57:52:18<358:56:57,  6.44s/it]
 11%|█         | 24230/225000 [57:52:30<452:03:02,  8.11s/it]
 11%|█         | 24231/225000 [57:52:34<393:41:58,  7.06s/it]
 11%|█         | 24232/225000 [57:52:39<354:45:39,  6.36s/it]
 11%|█         | 24233/225000 [57:52:44<325:43:03,  5.84s/it]
 11%|█         | 24234/225000 [57:52:51<345:10:10,  6.19s/it]
 11%|█         | 24235/225000 [57:53:01<424:14:01,  7.61s/it]
 11%|█         | 24236/225000 [57:53:05<354:49:24,  6.36s/it]
 11%|█         | 24237/225000 [57:53:10<339:42:12,  6.09s/it]
 11%|█         | 24238/225000 [57:53:15<321:37:46,  5.77s/it]
 11%|█         | 24239/225000 [57:53:21<312:03:33,  5.60s/it]
 11%|█         | 24240/225000 [57:53:25<300:19:02,  5.39s/it]
 11%|█         | 24241/225000 [57:53:31<305:41:27,  5.48s/it]
 11%|█         | 24242/225000 [57:53:44<431:06:38,  7.73s/it]
 11%|█         | 24243/225000 [57:53:48<373:38:50,  6.70s/it]
 11%|█         | 24244/225000 [57:53:54<353:40:25,  6.34s/it]
 11%|█         | 24245/225000 [57:53:59<337:50:44,  6.06s/it]
 11%|█         | 24246/225000 [57:54:05<332:42:08,  5.97s/it]
 11%|█         | 24247/225000 [57:54:10<317:22:39,  5.69s/it]
 11%|█         | 24248/225000 [57:54:22<423:28:25,  7.59s/it]
 11%|█         | 24249/225000 [57:54:27<373:55:24,  6.71s/it]
 11%|█         | 24250/225000 [57:54:32<341:21:04,  6.12s/it]
                                                             
{'loss': '0.7337', 'grad_norm': '0.653', 'learning_rate': '4.946e-05', 'epoch': '0.1078', 'num_input_tokens_seen': 100487400, 'train_runtime': '2.085e+05', 'train_tokens_per_second': '482'}

 11%|█         | 24250/225000 [57:54:32<341:21:04,  6.12s/it]
 11%|█         | 24251/225000 [57:54:37<325:47:21,  5.84s/it]
 11%|█         | 24252/225000 [57:54:41<303:47:52,  5.45s/it]
 11%|█         | 24253/225000 [57:54:54<416:57:32,  7.48s/it]
 11%|█         | 24254/225000 [57:55:00<401:15:55,  7.20s/it]
 11%|█         | 24255/225000 [57:55:04<347:51:04,  6.24s/it]
 11%|█         | 24256/225000 [57:55:10<347:49:03,  6.24s/it]
 11%|█         | 24257/225000 [57:55:20<407:03:58,  7.30s/it]
 11%|█         | 24258/225000 [57:55:26<379:16:12,  6.80s/it]
 11%|█         | 24259/225000 [57:55:29<328:43:01,  5.90s/it]
 11%|█         | 24260/225000 [57:55:39<389:07:08,  6.98s/it]
 11%|█         | 24261/225000 [57:55:43<345:46:15,  6.20s/it]
 11%|█         | 24262/225000 [57:55:50<351:42:28,  6.31s/it]
 11%|█         | 24263/225000 [57:55:58<373:00:45,  6.69s/it]
 11%|█         | 24264/225000 [57:56:02<328:20:18,  5.89s/it]
 11%|█         | 24265/225000 [57:56:11<383:31:22,  6.88s/it]
 11%|█         | 24266/225000 [57:56:14<330:22:56,  5.93s/it]
 11%|█         | 24267/225000 [57:56:25<400:50:00,  7.19s/it]
 11%|█         | 24268/225000 [57:56:28<343:27:46,  6.16s/it]
 11%|█         | 24269/225000 [57:56:36<373:12:25,  6.69s/it]
 11%|█         | 24270/225000 [57:56:45<406:37:57,  7.29s/it]
 11%|█         | 24271/225000 [57:56:49<346:24:35,  6.21s/it]
 11%|█         | 24272/225000 [57:56:52<303:09:19,  5.44s/it]
 11%|█         | 24273/225000 [57:57:04<401:41:25,  7.20s/it]
 11%|█         | 24274/225000 [57:57:08<347:50:23,  6.24s/it]
 11%|█         | 24275/225000 [57:57:12<319:49:56,  5.74s/it]
 11%|█         | 24276/225000 [57:57:17<300:24:49,  5.39s/it]
 11%|█         | 24277/225000 [57:57:30<429:35:46,  7.70s/it]
 11%|█         | 24278/225000 [57:57:37<420:03:57,  7.53s/it]
 11%|█         | 24279/225000 [57:57:43<401:45:46,  7.21s/it]
 11%|█         | 24280/225000 [57:57:48<360:02:32,  6.46s/it]
 11%|█         | 24281/225000 [57:57:53<328:26:20,  5.89s/it]
 11%|█         | 24282/225000 [57:58:00<347:56:07,  6.24s/it]
 11%|█         | 24283/225000 [57:58:08<386:18:28,  6.93s/it]
 11%|█         | 24284/225000 [57:58:13<343:50:51,  6.17s/it]
 11%|█         | 24285/225000 [57:58:18<324:21:13,  5.82s/it]
 11%|█         | 24286/225000 [57:58:23<308:24:50,  5.53s/it]
 11%|█         | 24287/225000 [57:58:28<310:02:32,  5.56s/it]
 11%|█         | 24288/225000 [57:58:40<414:45:29,  7.44s/it]
 11%|█         | 24289/225000 [57:58:47<403:32:04,  7.24s/it]
 11%|█         | 24290/225000 [57:58:51<355:44:40,  6.38s/it]
 11%|█         | 24291/225000 [57:58:56<335:16:53,  6.01s/it]
 11%|█         | 24292/225000 [57:59:07<407:18:42,  7.31s/it]
 11%|█         | 24293/225000 [57:59:11<356:55:00,  6.40s/it]
 11%|█         | 24294/225000 [57:59:15<325:42:18,  5.84s/it]
 11%|█         | 24295/225000 [57:59:22<337:06:17,  6.05s/it]
 11%|█         | 24296/225000 [57:59:33<426:27:37,  7.65s/it]
 11%|█         | 24297/225000 [57:59:40<414:05:31,  7.43s/it]
 11%|█         | 24298/225000 [57:59:47<397:14:44,  7.13s/it]
 11%|█         | 24299/225000 [57:59:55<414:37:16,  7.44s/it]
 11%|█         | 24300/225000 [58:00:00<369:31:52,  6.63s/it]
                                                             
{'loss': '0.6964', 'grad_norm': '0.7554', 'learning_rate': '4.946e-05', 'epoch': '0.108', 'num_input_tokens_seen': 100633272, 'train_runtime': '2.088e+05', 'train_tokens_per_second': '482'}

 11%|█         | 24300/225000 [58:00:00<369:31:52,  6.63s/it]
 11%|█         | 24301/225000 [58:00:07<389:39:54,  6.99s/it]
 11%|█         | 24302/225000 [58:00:17<431:50:16,  7.75s/it]
 11%|█         | 24303/225000 [58:00:20<361:20:01,  6.48s/it]
 11%|█         | 24304/225000 [58:00:29<390:15:35,  7.00s/it]
 11%|█         | 24305/225000 [58:00:33<345:25:38,  6.20s/it]
 11%|█         | 24306/225000 [58:00:38<323:00:08,  5.79s/it]
 11%|█         | 24307/225000 [58:00:45<341:36:26,  6.13s/it]
 11%|█         | 24308/225000 [58:00:54<389:27:44,  6.99s/it]
 11%|█         | 24309/225000 [58:00:58<349:07:59,  6.26s/it]
 11%|█         | 24310/225000 [58:01:03<314:38:15,  5.64s/it]
 11%|█         | 24311/225000 [58:01:12<373:05:03,  6.69s/it]
 11%|█         | 24312/225000 [58:01:22<431:12:10,  7.74s/it]
 11%|█         | 24313/225000 [58:01:26<367:37:36,  6.59s/it]
 11%|█         | 24314/225000 [58:01:31<348:30:40,  6.25s/it]
 11%|█         | 24315/225000 [58:01:39<377:19:54,  6.77s/it]
 11%|█         | 24316/225000 [58:01:44<351:51:45,  6.31s/it]
 11%|█         | 24317/225000 [58:01:53<386:18:01,  6.93s/it]
 11%|█         | 24318/225000 [58:02:00<389:32:16,  6.99s/it]
 11%|█         | 24319/225000 [58:02:04<348:23:11,  6.25s/it]
 11%|█         | 24320/225000 [58:02:13<385:27:00,  6.91s/it]
 11%|█         | 24321/225000 [58:02:20<393:45:07,  7.06s/it]
 11%|█         | 24322/225000 [58:02:23<328:32:52,  5.89s/it]
 11%|█         | 24323/225000 [58:02:29<319:37:49,  5.73s/it]
 11%|█         | 24324/225000 [58:02:35<318:41:15,  5.72s/it]
 11%|█         | 24325/225000 [58:02:40<311:34:47,  5.59s/it]
 11%|█         | 24326/225000 [58:02:45<305:35:39,  5.48s/it]
 11%|█         | 24327/225000 [58:02:54<363:49:08,  6.53s/it]
 11%|█         | 24328/225000 [58:03:05<442:53:07,  7.95s/it]
 11%|█         | 24329/225000 [58:03:11<411:58:18,  7.39s/it]
 11%|█         | 24330/225000 [58:03:16<371:49:26,  6.67s/it]
 11%|█         | 24331/225000 [58:03:22<347:08:01,  6.23s/it]
 11%|█         | 24332/225000 [58:03:26<322:39:33,  5.79s/it]
 11%|█         | 24333/225000 [58:03:31<299:13:27,  5.37s/it]
 11%|█         | 24334/225000 [58:03:36<298:22:17,  5.35s/it]
 11%|█         | 24335/225000 [58:03:41<296:59:51,  5.33s/it]
 11%|█         | 24336/225000 [58:03:46<292:45:09,  5.25s/it]
 11%|█         | 24337/225000 [58:03:58<397:58:17,  7.14s/it]
 11%|█         | 24338/225000 [58:04:06<410:53:47,  7.37s/it]
 11%|█         | 24339/225000 [58:04:13<403:20:38,  7.24s/it]
 11%|█         | 24340/225000 [58:04:18<376:25:00,  6.75s/it]
 11%|█         | 24341/225000 [58:04:24<354:25:16,  6.36s/it]
 11%|█         | 24342/225000 [58:04:30<347:55:27,  6.24s/it]
 11%|█         | 24343/225000 [58:04:35<329:16:04,  5.91s/it]
 11%|█         | 24344/225000 [58:04:43<367:21:37,  6.59s/it]
 11%|█         | 24345/225000 [58:04:54<443:10:35,  7.95s/it]
 11%|█         | 24346/225000 [58:05:01<424:11:59,  7.61s/it]
 11%|█         | 24347/225000 [58:05:06<372:36:30,  6.69s/it]
 11%|█         | 24348/225000 [58:05:12<374:59:46,  6.73s/it]
 11%|█         | 24349/225000 [58:05:19<374:28:29,  6.72s/it]
 11%|█         | 24350/225000 [58:05:23<329:25:39,  5.91s/it]
                                                             
{'loss': '0.7093', 'grad_norm': '0.7309', 'learning_rate': '4.945e-05', 'epoch': '0.1082', 'num_input_tokens_seen': 100778640, 'train_runtime': '2.091e+05', 'train_tokens_per_second': '481.9'}

 11%|█         | 24350/225000 [58:05:23<329:25:39,  5.91s/it]
 11%|█         | 24351/225000 [58:05:31<368:19:10,  6.61s/it]
 11%|█         | 24352/225000 [58:05:38<361:52:02,  6.49s/it]
 11%|█         | 24353/225000 [58:05:43<347:53:41,  6.24s/it]
 11%|█         | 24354/225000 [58:05:51<366:37:54,  6.58s/it]
 11%|█         | 24355/225000 [58:05:59<395:30:34,  7.10s/it]
 11%|█         | 24356/225000 [58:06:03<347:37:40,  6.24s/it]
 11%|█         | 24357/225000 [58:06:08<323:15:57,  5.80s/it]
 11%|█         | 24358/225000 [58:06:13<303:11:18,  5.44s/it]
 11%|█         | 24359/225000 [58:06:18<298:59:37,  5.36s/it]
 11%|█         | 24360/225000 [58:06:23<291:48:30,  5.24s/it]
 11%|█         | 24361/225000 [58:06:35<414:47:22,  7.44s/it]
 11%|█         | 24362/225000 [58:06:39<359:39:55,  6.45s/it]
 11%|█         | 24363/225000 [58:06:50<430:56:00,  7.73s/it]
 11%|█         | 24364/225000 [58:06:55<385:34:03,  6.92s/it]
 11%|█         | 24365/225000 [58:07:01<360:10:18,  6.46s/it]
 11%|█         | 24366/225000 [58:07:06<341:12:03,  6.12s/it]
 11%|█         | 24367/225000 [58:07:11<322:44:29,  5.79s/it]
 11%|█         | 24368/225000 [58:07:16<311:05:22,  5.58s/it]
 11%|█         | 24369/225000 [58:07:21<304:20:08,  5.46s/it]
 11%|█         | 24370/225000 [58:07:33<404:37:46,  7.26s/it]
 11%|█         | 24371/225000 [58:07:40<408:25:51,  7.33s/it]
 11%|█         | 24372/225000 [58:07:48<418:36:23,  7.51s/it]
 11%|█         | 24373/225000 [58:07:53<369:24:31,  6.63s/it]
 11%|█         | 24374/225000 [58:07:57<329:40:53,  5.92s/it]
 11%|█         | 24375/225000 [58:08:01<304:34:26,  5.47s/it]
 11%|█         | 24376/225000 [58:08:13<414:27:58,  7.44s/it]
 11%|█         | 24377/225000 [58:08:18<371:48:42,  6.67s/it]
 11%|█         | 24378/225000 [58:08:26<385:38:48,  6.92s/it]
 11%|█         | 24379/225000 [58:08:35<420:38:29,  7.55s/it]
 11%|█         | 24380/225000 [58:08:40<375:40:04,  6.74s/it]
 11%|█         | 24381/225000 [58:08:44<333:08:46,  5.98s/it]
 11%|█         | 24382/225000 [58:08:48<308:17:31,  5.53s/it]
 11%|█         | 24383/225000 [58:08:58<377:27:05,  6.77s/it]
 11%|█         | 24384/225000 [58:09:06<405:34:02,  7.28s/it]
 11%|█         | 24385/225000 [58:09:10<347:18:42,  6.23s/it]
 11%|█         | 24386/225000 [58:09:20<407:44:20,  7.32s/it]
 11%|█         | 24387/225000 [58:09:28<411:17:54,  7.38s/it]
 11%|█         | 24388/225000 [58:09:32<364:59:36,  6.55s/it]
 11%|█         | 24389/225000 [58:09:38<361:27:45,  6.49s/it]
 11%|█         | 24390/225000 [58:09:46<382:14:24,  6.86s/it]
 11%|█         | 24391/225000 [58:09:50<326:19:07,  5.86s/it]
 11%|█         | 24392/225000 [58:09:59<389:53:36,  7.00s/it]
 11%|█         | 24393/225000 [58:10:04<357:24:29,  6.41s/it]
 11%|█         | 24394/225000 [58:10:09<325:54:20,  5.85s/it]
 11%|█         | 24395/225000 [58:10:14<309:42:12,  5.56s/it]
 11%|█         | 24396/225000 [58:10:26<418:34:59,  7.51s/it]
 11%|█         | 24397/225000 [58:10:30<362:00:09,  6.50s/it]
 11%|█         | 24398/225000 [58:10:39<401:59:54,  7.21s/it]
 11%|█         | 24399/225000 [58:10:43<343:48:18,  6.17s/it]
 11%|█         | 24400/225000 [58:10:51<380:44:03,  6.83s/it]
                                                             
{'loss': '0.7158', 'grad_norm': '0.658', 'learning_rate': '4.945e-05', 'epoch': '0.1084', 'num_input_tokens_seen': 100922880, 'train_runtime': '2.095e+05', 'train_tokens_per_second': '481.8'}

 11%|█         | 24400/225000 [58:10:51<380:44:03,  6.83s/it]
 11%|█         | 24401/225000 [58:10:57<359:14:47,  6.45s/it]
 11%|█         | 24402/225000 [58:11:05<395:34:19,  7.10s/it]
 11%|█         | 24403/225000 [58:11:09<342:29:59,  6.15s/it]
 11%|█         | 24404/225000 [58:11:13<302:28:15,  5.43s/it]
 11%|█         | 24405/225000 [58:11:17<288:47:09,  5.18s/it]
 11%|█         | 24406/225000 [58:11:28<370:42:41,  6.65s/it]
 11%|█         | 24407/225000 [58:11:37<418:46:52,  7.52s/it]
 11%|█         | 24408/225000 [58:11:43<398:52:23,  7.16s/it]
 11%|█         | 24409/225000 [58:11:48<363:39:15,  6.53s/it]
 11%|█         | 24410/225000 [58:11:58<407:50:23,  7.32s/it]
 11%|█         | 24411/225000 [58:12:02<366:00:05,  6.57s/it]
 11%|█         | 24412/225000 [58:12:07<325:10:17,  5.84s/it]
 11%|█         | 24413/225000 [58:12:10<288:00:28,  5.17s/it]
 11%|█         | 24414/225000 [58:12:15<289:43:37,  5.20s/it]
 11%|█         | 24415/225000 [58:12:28<411:11:20,  7.38s/it]
 11%|█         | 24416/225000 [58:12:34<388:12:56,  6.97s/it]
 11%|█         | 24417/225000 [58:12:43<422:19:47,  7.58s/it]
 11%|█         | 24418/225000 [58:12:47<359:11:48,  6.45s/it]
 11%|█         | 24419/225000 [58:12:52<333:09:43,  5.98s/it]
 11%|█         | 24420/225000 [58:13:01<389:16:42,  6.99s/it]
 11%|█         | 24421/225000 [58:13:05<331:41:13,  5.95s/it]
 11%|█         | 24422/225000 [58:13:09<314:08:53,  5.64s/it]
 11%|█         | 24423/225000 [58:13:14<304:19:15,  5.46s/it]
 11%|█         | 24424/225000 [58:13:21<325:36:35,  5.84s/it]
 11%|█         | 24425/225000 [58:13:34<440:53:04,  7.91s/it]
 11%|█         | 24426/225000 [58:13:39<399:59:41,  7.18s/it]
 11%|█         | 24427/225000 [58:13:45<366:40:36,  6.58s/it]
 11%|█         | 24428/225000 [58:13:50<340:00:24,  6.10s/it]
 11%|█         | 24429/225000 [58:13:55<329:41:59,  5.92s/it]
 11%|█         | 24430/225000 [58:14:01<323:31:39,  5.81s/it]
 11%|█         | 24431/225000 [58:14:06<318:31:56,  5.72s/it]
 11%|█         | 24432/225000 [58:14:19<441:07:03,  7.92s/it]
 11%|█         | 24433/225000 [58:14:24<381:08:01,  6.84s/it]
 11%|█         | 24434/225000 [58:14:28<340:30:02,  6.11s/it]
 11%|█         | 24435/225000 [58:14:37<397:45:45,  7.14s/it]
 11%|█         | 24436/225000 [58:14:46<424:43:38,  7.62s/it]
 11%|█         | 24437/225000 [58:14:53<407:56:52,  7.32s/it]
 11%|█         | 24438/225000 [58:14:58<374:49:36,  6.73s/it]
 11%|█         | 24439/225000 [58:15:03<335:48:52,  6.03s/it]
 11%|█         | 24440/225000 [58:15:08<318:59:06,  5.73s/it]
 11%|█         | 24441/225000 [58:15:21<441:01:37,  7.92s/it]
 11%|█         | 24442/225000 [58:15:25<386:01:24,  6.93s/it]
 11%|█         | 24443/225000 [58:15:30<353:34:54,  6.35s/it]
 11%|█         | 24444/225000 [58:15:35<319:50:41,  5.74s/it]
 11%|█         | 24445/225000 [58:15:40<306:57:59,  5.51s/it]
 11%|█         | 24446/225000 [58:15:52<422:04:40,  7.58s/it]
 11%|█         | 24447/225000 [58:15:56<361:13:29,  6.48s/it]
 11%|█         | 24448/225000 [58:16:06<422:09:16,  7.58s/it]
 11%|█         | 24449/225000 [58:16:10<358:11:36,  6.43s/it]
 11%|█         | 24450/225000 [58:16:15<334:07:00,  6.00s/it]
                                                             
{'loss': '0.7174', 'grad_norm': '0.6296', 'learning_rate': '4.945e-05', 'epoch': '0.1087', 'num_input_tokens_seen': 101067296, 'train_runtime': '2.098e+05', 'train_tokens_per_second': '481.8'}

 11%|█         | 24450/225000 [58:16:15<334:07:00,  6.00s/it]
 11%|█         | 24451/225000 [58:16:20<323:34:23,  5.81s/it]
 11%|█         | 24452/225000 [58:16:33<434:14:05,  7.79s/it]
 11%|█         | 24453/225000 [58:16:40<426:11:13,  7.65s/it]
 11%|█         | 24454/225000 [58:16:43<353:14:57,  6.34s/it]
 11%|█         | 24455/225000 [58:16:51<384:28:21,  6.90s/it]
 11%|█         | 24456/225000 [58:16:56<342:51:21,  6.15s/it]
 11%|█         | 24457/225000 [58:17:01<332:28:53,  5.97s/it]
 11%|█         | 24458/225000 [58:17:06<314:45:05,  5.65s/it]
 11%|█         | 24459/225000 [58:17:11<302:31:29,  5.43s/it]
 11%|█         | 24460/225000 [58:17:20<361:07:36,  6.48s/it]
 11%|█         | 24461/225000 [58:17:29<406:28:10,  7.30s/it]
 11%|█         | 24462/225000 [58:17:37<413:38:57,  7.43s/it]
 11%|█         | 24463/225000 [58:17:41<358:52:08,  6.44s/it]
 11%|█         | 24464/225000 [58:17:50<404:32:26,  7.26s/it]
 11%|█         | 24465/225000 [58:17:57<389:39:01,  6.99s/it]
 11%|█         | 24466/225000 [58:18:01<337:09:34,  6.05s/it]
 11%|█         | 24467/225000 [58:18:06<328:08:33,  5.89s/it]
 11%|█         | 24468/225000 [58:18:15<379:34:24,  6.81s/it]
 11%|█         | 24469/225000 [58:18:20<355:22:38,  6.38s/it]
 11%|█         | 24470/225000 [58:18:29<389:46:42,  7.00s/it]
 11%|█         | 24471/225000 [58:18:36<395:20:16,  7.10s/it]
 11%|█         | 24472/225000 [58:18:43<396:01:27,  7.11s/it]
 11%|█         | 24473/225000 [58:18:47<346:46:39,  6.23s/it]
 11%|█         | 24474/225000 [58:18:54<359:51:28,  6.46s/it]
 11%|█         | 24475/225000 [58:19:01<355:40:32,  6.39s/it]
 11%|█         | 24476/225000 [58:19:04<312:33:59,  5.61s/it]
 11%|█         | 24477/225000 [58:19:13<364:17:55,  6.54s/it]
 11%|█         | 24478/225000 [58:19:18<336:22:12,  6.04s/it]
 11%|█         | 24479/225000 [58:19:23<317:59:13,  5.71s/it]
 11%|█         | 24480/225000 [58:19:28<298:46:04,  5.36s/it]
 11%|█         | 24481/225000 [58:19:40<411:07:11,  7.38s/it]
 11%|█         | 24482/225000 [58:19:45<380:06:50,  6.82s/it]
 11%|█         | 24483/225000 [58:19:53<392:31:42,  7.05s/it]
 11%|█         | 24484/225000 [58:19:58<360:27:46,  6.47s/it]
 11%|█         | 24485/225000 [58:20:06<390:32:35,  7.01s/it]
 11%|█         | 24486/225000 [58:20:12<372:16:28,  6.68s/it]
 11%|█         | 24487/225000 [58:20:19<378:53:33,  6.80s/it]
 11%|█         | 24488/225000 [58:20:26<387:25:13,  6.96s/it]
 11%|█         | 24489/225000 [58:20:30<332:07:29,  5.96s/it]
 11%|█         | 24490/225000 [58:20:36<334:46:33,  6.01s/it]
 11%|█         | 24491/225000 [58:20:46<397:30:55,  7.14s/it]
 11%|█         | 24492/225000 [58:20:51<355:04:26,  6.38s/it]
 11%|█         | 24493/225000 [58:20:55<328:09:13,  5.89s/it]
 11%|█         | 24494/225000 [58:21:01<318:25:42,  5.72s/it]
 11%|█         | 24495/225000 [58:21:05<303:22:32,  5.45s/it]
 11%|█         | 24496/225000 [58:21:17<405:21:56,  7.28s/it]
 11%|█         | 24497/225000 [58:21:24<394:28:38,  7.08s/it]
 11%|█         | 24498/225000 [58:21:29<364:02:58,  6.54s/it]
 11%|█         | 24499/225000 [58:21:34<335:02:53,  6.02s/it]
 11%|█         | 24500/225000 [58:21:39<319:20:53,  5.73s/it]
                                                             
{'loss': '0.7188', 'grad_norm': '0.6343', 'learning_rate': '4.944e-05', 'epoch': '0.1089', 'num_input_tokens_seen': 101213008, 'train_runtime': '2.101e+05', 'train_tokens_per_second': '481.7'}

 11%|█         | 24500/225000 [58:21:39<319:20:53,  5.73s/it]
 11%|█         | 24501/225000 [58:21:52<443:05:56,  7.96s/it]
 11%|█         | 24502/225000 [58:21:56<372:57:25,  6.70s/it]
 11%|█         | 24503/225000 [58:22:04<401:11:43,  7.20s/it]
 11%|█         | 24504/225000 [58:22:09<371:31:59,  6.67s/it]
 11%|█         | 24505/225000 [58:22:15<343:58:16,  6.18s/it]
 11%|█         | 24506/225000 [58:22:20<326:32:26,  5.86s/it]
 11%|█         | 24507/225000 [58:22:25<315:14:39,  5.66s/it]
 11%|█         | 24508/225000 [58:22:31<319:15:20,  5.73s/it]
 11%|█         | 24509/225000 [58:22:44<445:42:29,  8.00s/it]
 11%|█         | 24510/225000 [58:22:48<382:10:28,  6.86s/it]
 11%|█         | 24511/225000 [58:22:52<333:02:16,  5.98s/it]
 11%|█         | 24512/225000 [58:22:59<354:46:56,  6.37s/it]
 11%|█         | 24513/225000 [58:23:07<381:05:19,  6.84s/it]
 11%|█         | 24514/225000 [58:23:12<340:22:20,  6.11s/it]
 11%|█         | 24515/225000 [58:23:17<317:13:07,  5.70s/it]
 11%|█         | 24516/225000 [58:23:21<303:58:29,  5.46s/it]
 11%|█         | 24517/225000 [58:23:32<386:25:41,  6.94s/it]
 11%|█         | 24518/225000 [58:23:39<396:30:02,  7.12s/it]
 11%|█         | 24519/225000 [58:23:43<342:45:21,  6.15s/it]
 11%|█         | 24520/225000 [58:23:53<409:31:02,  7.35s/it]
 11%|█         | 24521/225000 [58:23:59<374:40:58,  6.73s/it]
 11%|█         | 24522/225000 [58:24:04<350:24:26,  6.29s/it]
 11%|█         | 24523/225000 [58:24:10<339:34:50,  6.10s/it]
 11%|█         | 24524/225000 [58:24:15<333:42:10,  5.99s/it]
 11%|█         | 24525/225000 [58:24:21<329:38:19,  5.92s/it]
 11%|█         | 24526/225000 [58:24:27<323:21:20,  5.81s/it]
 11%|█         | 24527/225000 [58:24:32<322:04:39,  5.78s/it]
 11%|█         | 24528/225000 [58:24:42<378:38:00,  6.80s/it]
 11%|█         | 24529/225000 [58:24:53<458:39:58,  8.24s/it]
 11%|█         | 24530/225000 [58:24:57<392:43:41,  7.05s/it]
 11%|█         | 24531/225000 [58:25:05<401:21:02,  7.21s/it]
 11%|█         | 24532/225000 [58:25:12<391:20:07,  7.03s/it]
 11%|█         | 24533/225000 [58:25:17<358:19:18,  6.43s/it]
 11%|█         | 24534/225000 [58:25:24<376:49:05,  6.77s/it]
 11%|█         | 24535/225000 [58:25:30<360:19:01,  6.47s/it]
 11%|█         | 24536/225000 [58:25:36<351:16:33,  6.31s/it]
 11%|█         | 24537/225000 [58:25:42<349:49:54,  6.28s/it]
 11%|█         | 24538/225000 [58:25:49<363:30:37,  6.53s/it]
 11%|█         | 24539/225000 [58:25:54<329:41:27,  5.92s/it]
 11%|█         | 24540/225000 [58:25:59<319:44:52,  5.74s/it]
 11%|█         | 24541/225000 [58:26:07<357:24:53,  6.42s/it]
 11%|█         | 24542/225000 [58:26:18<425:22:03,  7.64s/it]
 11%|█         | 24543/225000 [58:26:22<365:22:48,  6.56s/it]
 11%|█         | 24544/225000 [58:26:28<357:48:01,  6.43s/it]
 11%|█         | 24545/225000 [58:26:36<391:32:01,  7.03s/it]
 11%|█         | 24546/225000 [58:26:41<358:27:45,  6.44s/it]
 11%|█         | 24547/225000 [58:26:47<349:09:52,  6.27s/it]
 11%|█         | 24548/225000 [58:26:52<329:48:36,  5.92s/it]
 11%|█         | 24549/225000 [58:26:58<326:00:58,  5.86s/it]
 11%|█         | 24550/225000 [58:27:03<318:45:27,  5.72s/it]
                                                             
{'loss': '0.724', 'grad_norm': '0.7213', 'learning_rate': '4.944e-05', 'epoch': '0.1091', 'num_input_tokens_seen': 101358840, 'train_runtime': '2.104e+05', 'train_tokens_per_second': '481.7'}

 11%|█         | 24550/225000 [58:27:03<318:45:27,  5.72s/it]
 11%|█         | 24551/225000 [58:27:08<307:21:23,  5.52s/it]
 11%|█         | 24552/225000 [58:27:18<373:31:50,  6.71s/it]
 11%|█         | 24553/225000 [58:27:28<439:02:21,  7.89s/it]
 11%|█         | 24554/225000 [58:27:35<416:44:20,  7.48s/it]
 11%|█         | 24555/225000 [58:27:40<377:12:57,  6.77s/it]
 11%|█         | 24556/225000 [58:27:45<343:49:55,  6.18s/it]
 11%|█         | 24557/225000 [58:27:51<334:29:58,  6.01s/it]
 11%|█         | 24558/225000 [58:27:59<379:03:32,  6.81s/it]
 11%|█         | 24559/225000 [58:28:06<381:19:26,  6.85s/it]
 11%|█         | 24560/225000 [58:28:11<346:07:12,  6.22s/it]
 11%|█         | 24561/225000 [58:28:16<332:02:50,  5.96s/it]
 11%|█         | 24562/225000 [58:28:21<313:27:51,  5.63s/it]
 11%|█         | 24563/225000 [58:28:34<433:09:49,  7.78s/it]
 11%|█         | 24564/225000 [58:28:41<423:36:10,  7.61s/it]
 11%|█         | 24565/225000 [58:28:47<390:28:18,  7.01s/it]
 11%|█         | 24566/225000 [58:28:51<337:16:20,  6.06s/it]
 11%|█         | 24567/225000 [58:29:03<449:10:22,  8.07s/it]
 11%|█         | 24568/225000 [58:29:08<394:18:50,  7.08s/it]
 11%|█         | 24569/225000 [58:29:15<388:31:41,  6.98s/it]
 11%|█         | 24570/225000 [58:29:22<388:24:21,  6.98s/it]
 11%|█         | 24571/225000 [58:29:30<407:06:28,  7.31s/it]
 11%|█         | 24572/225000 [58:29:35<367:04:04,  6.59s/it]
 11%|█         | 24573/225000 [58:29:39<330:07:36,  5.93s/it]
 11%|█         | 24574/225000 [58:29:43<302:43:58,  5.44s/it]
 11%|█         | 24575/225000 [58:29:54<382:17:19,  6.87s/it]
 11%|█         | 24576/225000 [58:30:00<374:01:48,  6.72s/it]
 11%|█         | 24577/225000 [58:30:04<319:50:07,  5.74s/it]
 11%|█         | 24578/225000 [58:30:15<409:01:03,  7.35s/it]
 11%|█         | 24579/225000 [58:30:18<348:19:19,  6.26s/it]
 11%|█         | 24580/225000 [58:30:26<378:50:05,  6.80s/it]
 11%|█         | 24581/225000 [58:30:32<357:07:16,  6.41s/it]
 11%|█         | 24582/225000 [58:30:39<374:57:11,  6.74s/it]
 11%|█         | 24583/225000 [58:30:44<331:10:59,  5.95s/it]
 11%|█         | 24584/225000 [58:30:49<331:33:28,  5.96s/it]
 11%|█         | 24585/225000 [58:30:59<386:12:08,  6.94s/it]
 11%|█         | 24586/225000 [58:31:06<392:03:22,  7.04s/it]
 11%|█         | 24587/225000 [58:31:10<347:56:55,  6.25s/it]
 11%|█         | 24588/225000 [58:31:15<318:11:38,  5.72s/it]
 11%|█         | 24589/225000 [58:31:20<306:45:34,  5.51s/it]
 11%|█         | 24590/225000 [58:31:25<295:58:52,  5.32s/it]
 11%|█         | 24591/225000 [58:31:34<367:54:15,  6.61s/it]
 11%|█         | 24592/225000 [58:31:43<399:04:36,  7.17s/it]
 11%|█         | 24593/225000 [58:31:48<364:01:37,  6.54s/it]
 11%|█         | 24594/225000 [58:31:53<332:44:23,  5.98s/it]
 11%|█         | 24595/225000 [58:32:00<360:11:02,  6.47s/it]
 11%|█         | 24596/225000 [58:32:13<471:47:18,  8.48s/it]
 11%|█         | 24597/225000 [58:32:17<391:31:39,  7.03s/it]
 11%|█         | 24598/225000 [58:32:21<339:09:11,  6.09s/it]
 11%|█         | 24599/225000 [58:32:31<410:17:35,  7.37s/it]
 11%|█         | 24600/225000 [58:32:35<353:48:41,  6.36s/it]
                                                             
{'loss': '0.7232', 'grad_norm': '0.6443', 'learning_rate': '4.943e-05', 'epoch': '0.1093', 'num_input_tokens_seen': 101504440, 'train_runtime': '2.108e+05', 'train_tokens_per_second': '481.6'}

 11%|█         | 24600/225000 [58:32:35<353:48:41,  6.36s/it]
 11%|█         | 24601/225000 [58:32:45<410:56:50,  7.38s/it]
 11%|█         | 24602/225000 [58:32:49<348:10:29,  6.25s/it]
 11%|█         | 24603/225000 [58:32:54<333:59:49,  6.00s/it]
 11%|█         | 24604/225000 [58:33:00<327:49:15,  5.89s/it]
 11%|█         | 24605/225000 [58:33:05<319:12:18,  5.73s/it]
 11%|█         | 24606/225000 [58:33:10<311:43:27,  5.60s/it]
 11%|█         | 24607/225000 [58:33:21<388:45:30,  6.98s/it]
 11%|█         | 24608/225000 [58:33:30<422:38:08,  7.59s/it]
 11%|█         | 24609/225000 [58:33:37<422:22:44,  7.59s/it]
 11%|█         | 24610/225000 [58:33:42<378:05:11,  6.79s/it]
 11%|█         | 24611/225000 [58:33:50<403:22:40,  7.25s/it]
 11%|█         | 24612/225000 [58:34:00<436:38:10,  7.84s/it]
 11%|█         | 24613/225000 [58:34:08<448:40:01,  8.06s/it]
 11%|█         | 24614/225000 [58:34:13<388:35:25,  6.98s/it]
 11%|█         | 24615/225000 [58:34:20<402:18:25,  7.23s/it]
 11%|█         | 24616/225000 [58:34:25<364:51:28,  6.55s/it]
 11%|█         | 24617/225000 [58:34:34<391:31:33,  7.03s/it]
 11%|█         | 24618/225000 [58:34:38<351:43:40,  6.32s/it]
 11%|█         | 24619/225000 [58:34:43<329:16:09,  5.92s/it]
 11%|█         | 24620/225000 [58:34:48<316:28:59,  5.69s/it]
 11%|█         | 24621/225000 [58:34:53<301:26:43,  5.42s/it]
 11%|█         | 24622/225000 [58:34:58<289:52:14,  5.21s/it]
 11%|█         | 24623/225000 [58:35:09<382:33:06,  6.87s/it]
 11%|█         | 24624/225000 [58:35:17<410:04:02,  7.37s/it]
 11%|█         | 24625/225000 [58:35:24<403:16:57,  7.25s/it]
 11%|█         | 24626/225000 [58:35:28<343:36:35,  6.17s/it]
 11%|█         | 24627/225000 [58:35:33<330:18:14,  5.93s/it]
 11%|█         | 24628/225000 [58:35:39<321:19:16,  5.77s/it]
 11%|█         | 24629/225000 [58:35:45<327:22:15,  5.88s/it]
 11%|█         | 24630/225000 [58:35:57<440:37:09,  7.92s/it]
 11%|█         | 24631/225000 [58:36:03<398:38:50,  7.16s/it]
 11%|█         | 24632/225000 [58:36:07<344:36:01,  6.19s/it]
 11%|█         | 24633/225000 [58:36:16<389:01:08,  6.99s/it]
 11%|█         | 24634/225000 [58:36:20<343:25:12,  6.17s/it]
 11%|█         | 24635/225000 [58:36:27<364:56:06,  6.56s/it]
 11%|█         | 24636/225000 [58:36:36<401:47:59,  7.22s/it]
 11%|█         | 24637/225000 [58:36:40<351:27:09,  6.31s/it]
 11%|█         | 24638/225000 [58:36:44<314:19:14,  5.65s/it]
 11%|█         | 24639/225000 [58:36:51<337:30:15,  6.06s/it]
 11%|█         | 24640/225000 [58:37:02<407:30:31,  7.32s/it]
 11%|█         | 24641/225000 [58:37:06<361:50:48,  6.50s/it]
 11%|█         | 24642/225000 [58:37:11<334:36:04,  6.01s/it]
 11%|█         | 24643/225000 [58:37:16<319:49:46,  5.75s/it]
 11%|█         | 24644/225000 [58:37:21<301:49:44,  5.42s/it]
 11%|█         | 24645/225000 [58:37:28<321:26:04,  5.78s/it]
 11%|█         | 24646/225000 [58:37:40<439:06:05,  7.89s/it]
 11%|█         | 24647/225000 [58:37:47<411:07:46,  7.39s/it]
 11%|█         | 24648/225000 [58:37:50<351:30:28,  6.32s/it]
 11%|█         | 24649/225000 [58:37:58<370:16:59,  6.65s/it]
 11%|█         | 24650/225000 [58:38:01<315:16:21,  5.66s/it]
                                                             
{'loss': '0.7129', 'grad_norm': '0.7707', 'learning_rate': '4.943e-05', 'epoch': '0.1096', 'num_input_tokens_seen': 101649808, 'train_runtime': '2.111e+05', 'train_tokens_per_second': '481.6'}

 11%|█         | 24650/225000 [58:38:01<315:16:21,  5.66s/it]
 11%|█         | 24651/225000 [58:38:10<366:15:15,  6.58s/it]
 11%|█         | 24652/225000 [58:38:16<359:00:54,  6.45s/it]
 11%|█         | 24653/225000 [58:38:24<381:48:30,  6.86s/it]
 11%|█         | 24654/225000 [58:38:28<340:33:15,  6.12s/it]
 11%|█         | 24655/225000 [58:38:39<414:22:34,  7.45s/it]
 11%|█         | 24656/225000 [58:38:44<378:47:45,  6.81s/it]
 11%|█         | 24657/225000 [58:38:50<370:04:07,  6.65s/it]
 11%|█         | 24658/225000 [58:38:55<330:51:14,  5.95s/it]
 11%|█         | 24659/225000 [58:39:02<350:33:02,  6.30s/it]
 11%|█         | 24660/225000 [58:39:07<334:39:01,  6.01s/it]
 11%|█         | 24661/225000 [58:39:13<330:08:54,  5.93s/it]
 11%|█         | 24662/225000 [58:39:22<377:10:26,  6.78s/it]
 11%|█         | 24663/225000 [58:39:27<351:30:11,  6.32s/it]
 11%|█         | 24664/225000 [58:39:32<338:07:14,  6.08s/it]
 11%|█         | 24665/225000 [58:39:42<399:29:27,  7.18s/it]
 11%|█         | 24666/225000 [58:39:46<351:38:04,  6.32s/it]
 11%|█         | 24667/225000 [58:39:51<321:25:40,  5.78s/it]
 11%|█         | 24668/225000 [58:40:03<418:46:12,  7.53s/it]
 11%|█         | 24669/225000 [58:40:09<393:15:07,  7.07s/it]
 11%|█         | 24670/225000 [58:40:13<354:43:57,  6.37s/it]
 11%|█         | 24671/225000 [58:40:18<332:44:58,  5.98s/it]
 11%|█         | 24672/225000 [58:40:23<314:49:51,  5.66s/it]
 11%|█         | 24673/225000 [58:40:29<319:36:18,  5.74s/it]
 11%|█         | 24674/225000 [58:40:42<432:57:24,  7.78s/it]
 11%|█         | 24675/225000 [58:40:47<389:55:31,  7.01s/it]
 11%|█         | 24676/225000 [58:40:54<382:54:52,  6.88s/it]
 11%|█         | 24677/225000 [58:40:57<331:07:08,  5.95s/it]
 11%|█         | 24678/225000 [58:41:08<408:12:46,  7.34s/it]
 11%|█         | 24679/225000 [58:41:11<340:29:15,  6.12s/it]
 11%|█         | 24680/225000 [58:41:24<452:09:48,  8.13s/it]
 11%|█         | 24681/225000 [58:41:29<391:51:43,  7.04s/it]
 11%|█         | 24682/225000 [58:41:34<374:00:24,  6.72s/it]
 11%|█         | 24683/225000 [58:41:39<342:25:01,  6.15s/it]
 11%|█         | 24684/225000 [58:41:44<326:14:30,  5.86s/it]
 11%|█         | 24685/225000 [58:41:50<322:29:44,  5.80s/it]
 11%|█         | 24686/225000 [58:41:55<311:13:16,  5.59s/it]
 11%|█         | 24687/225000 [58:42:00<303:36:41,  5.46s/it]
 11%|█         | 24688/225000 [58:42:06<303:15:03,  5.45s/it]
 11%|█         | 24689/225000 [58:42:15<364:14:49,  6.55s/it]
 11%|█         | 24690/225000 [58:42:27<456:56:21,  8.21s/it]
 11%|█         | 24691/225000 [58:42:30<372:36:40,  6.70s/it]
 11%|█         | 24692/225000 [58:42:39<402:28:38,  7.23s/it]
 11%|█         | 24693/225000 [58:42:42<341:01:09,  6.13s/it]
 11%|█         | 24694/225000 [58:42:52<398:13:27,  7.16s/it]
 11%|█         | 24695/225000 [58:42:57<359:37:48,  6.46s/it]
 11%|█         | 24696/225000 [58:43:01<324:23:15,  5.83s/it]
 11%|█         | 24697/225000 [58:43:06<308:55:07,  5.55s/it]
 11%|█         | 24698/225000 [58:43:17<398:28:03,  7.16s/it]
 11%|█         | 24699/225000 [58:43:21<350:29:34,  6.30s/it]
 11%|█         | 24700/225000 [58:43:25<312:54:00,  5.62s/it]
                                                             
{'loss': '0.7236', 'grad_norm': '0.6641', 'learning_rate': '4.943e-05', 'epoch': '0.1098', 'num_input_tokens_seen': 101794696, 'train_runtime': '2.114e+05', 'train_tokens_per_second': '481.5'}

 11%|█         | 24700/225000 [58:43:25<312:54:00,  5.62s/it]
 11%|█         | 24701/225000 [58:43:37<414:24:59,  7.45s/it]
 11%|█         | 24702/225000 [58:43:41<360:10:20,  6.47s/it]
 11%|█         | 24703/225000 [58:43:45<316:01:40,  5.68s/it]
 11%|█         | 24704/225000 [58:43:56<399:53:16,  7.19s/it]
 11%|█         | 24705/225000 [58:44:01<372:39:53,  6.70s/it]
 11%|█         | 24706/225000 [58:44:07<356:12:25,  6.40s/it]
 11%|█         | 24707/225000 [58:44:15<387:16:50,  6.96s/it]
 11%|█         | 24708/225000 [58:44:20<360:45:54,  6.48s/it]
 11%|█         | 24709/225000 [58:44:27<361:50:59,  6.50s/it]
 11%|█         | 24710/225000 [58:44:34<375:49:57,  6.76s/it]
 11%|█         | 24711/225000 [58:44:40<363:13:11,  6.53s/it]
 11%|█         | 24712/225000 [58:44:48<386:38:27,  6.95s/it]
 11%|█         | 24713/225000 [58:44:55<374:45:43,  6.74s/it]
 11%|█         | 24714/225000 [58:45:01<361:49:34,  6.50s/it]
 11%|█         | 24715/225000 [58:45:06<345:21:52,  6.21s/it]
 11%|█         | 24716/225000 [58:45:09<296:03:31,  5.32s/it]
 11%|█         | 24717/225000 [58:45:15<294:56:06,  5.30s/it]
 11%|█         | 24718/225000 [58:45:24<372:17:45,  6.69s/it]
 11%|█         | 24719/225000 [58:45:29<332:05:05,  5.97s/it]
 11%|█         | 24720/225000 [58:45:33<306:46:25,  5.51s/it]
 11%|█         | 24721/225000 [58:45:42<366:54:30,  6.60s/it]
 11%|█         | 24722/225000 [58:45:47<335:59:15,  6.04s/it]
 11%|█         | 24723/225000 [58:45:56<383:05:08,  6.89s/it]
 11%|█         | 24724/225000 [58:46:01<356:24:20,  6.41s/it]
 11%|█         | 24725/225000 [58:46:06<336:28:58,  6.05s/it]
 11%|█         | 24726/225000 [58:46:12<332:04:00,  5.97s/it]
 11%|█         | 24727/225000 [58:46:18<325:28:57,  5.85s/it]
 11%|█         | 24728/225000 [58:46:23<320:41:24,  5.76s/it]
 11%|█         | 24729/225000 [58:46:32<375:20:13,  6.75s/it]
 11%|█         | 24730/225000 [58:46:43<442:18:12,  7.95s/it]
 11%|█         | 24731/225000 [58:46:52<450:59:08,  8.11s/it]
 11%|█         | 24732/225000 [58:46:58<423:00:13,  7.60s/it]
 11%|█         | 24733/225000 [58:47:04<390:25:47,  7.02s/it]
 11%|█         | 24734/225000 [58:47:11<387:46:09,  6.97s/it]
 11%|█         | 24735/225000 [58:47:17<376:40:56,  6.77s/it]
 11%|█         | 24736/225000 [58:47:22<351:49:17,  6.32s/it]
 11%|█         | 24737/225000 [58:47:27<327:41:20,  5.89s/it]
 11%|█         | 24738/225000 [58:47:32<314:03:08,  5.65s/it]
 11%|█         | 24739/225000 [58:47:42<387:11:52,  6.96s/it]
 11%|█         | 24740/225000 [58:47:50<396:42:34,  7.13s/it]
 11%|█         | 24741/225000 [58:47:56<379:11:11,  6.82s/it]
 11%|█         | 24742/225000 [58:48:02<372:42:54,  6.70s/it]
 11%|█         | 24743/225000 [58:48:06<329:46:25,  5.93s/it]
 11%|█         | 24744/225000 [58:48:16<392:32:59,  7.06s/it]
 11%|█         | 24745/225000 [58:48:22<383:04:06,  6.89s/it]
 11%|█         | 24746/225000 [58:48:26<328:13:44,  5.90s/it]
 11%|█         | 24747/225000 [58:48:35<384:38:26,  6.91s/it]
 11%|█         | 24748/225000 [58:48:42<383:39:50,  6.90s/it]
 11%|█         | 24749/225000 [58:48:47<341:31:38,  6.14s/it]
 11%|█         | 24750/225000 [58:48:51<318:46:03,  5.73s/it]
                                                             
{'loss': '0.7255', 'grad_norm': '0.6822', 'learning_rate': '4.942e-05', 'epoch': '0.11', 'num_input_tokens_seen': 101938104, 'train_runtime': '2.117e+05', 'train_tokens_per_second': '481.4'}

 11%|█         | 24750/225000 [58:48:51<318:46:03,  5.73s/it]
 11%|█         | 24751/225000 [58:48:56<308:39:53,  5.55s/it]
 11%|█         | 24752/225000 [58:49:09<428:27:02,  7.70s/it]
 11%|█         | 24753/225000 [58:49:13<362:05:57,  6.51s/it]
 11%|█         | 24754/225000 [58:49:21<386:28:19,  6.95s/it]
 11%|█         | 24755/225000 [58:49:26<353:40:32,  6.36s/it]
 11%|█         | 24756/225000 [58:49:31<338:48:18,  6.09s/it]
 11%|█         | 24757/225000 [58:49:37<333:10:56,  5.99s/it]
 11%|█         | 24758/225000 [58:49:43<329:04:52,  5.92s/it]
 11%|█         | 24759/225000 [58:49:48<314:34:44,  5.66s/it]
 11%|█         | 24760/225000 [58:49:55<338:29:04,  6.09s/it]
 11%|█         | 24761/225000 [58:50:06<417:35:54,  7.51s/it]
 11%|█         | 24762/225000 [58:50:11<371:30:20,  6.68s/it]
 11%|█         | 24763/225000 [58:50:18<377:42:25,  6.79s/it]
 11%|█         | 24764/225000 [58:50:23<354:29:22,  6.37s/it]
 11%|█         | 24765/225000 [58:50:29<343:38:55,  6.18s/it]
 11%|█         | 24766/225000 [58:50:34<326:49:46,  5.88s/it]
 11%|█         | 24767/225000 [58:50:39<313:38:04,  5.64s/it]
 11%|█         | 24768/225000 [58:50:44<298:09:18,  5.36s/it]
 11%|█         | 24769/225000 [58:50:49<299:52:59,  5.39s/it]
 11%|█         | 24770/225000 [58:50:59<367:53:06,  6.61s/it]
 11%|█         | 24771/225000 [58:51:11<459:14:10,  8.26s/it]
 11%|█         | 24772/225000 [58:51:15<385:22:46,  6.93s/it]
 11%|█         | 24773/225000 [58:51:24<428:24:06,  7.70s/it]
 11%|█         | 24774/225000 [58:51:29<376:55:13,  6.78s/it]
 11%|█         | 24775/225000 [58:51:35<366:13:53,  6.58s/it]
 11%|█         | 24776/225000 [58:51:41<358:10:14,  6.44s/it]
 11%|█         | 24777/225000 [58:51:47<346:37:48,  6.23s/it]
 11%|█         | 24778/225000 [58:51:52<333:26:51,  6.00s/it]
 11%|█         | 24779/225000 [58:51:57<321:36:54,  5.78s/it]
 11%|█         | 24780/225000 [58:52:05<347:07:00,  6.24s/it]
 11%|█         | 24781/225000 [58:52:17<441:32:57,  7.94s/it]
 11%|█         | 24782/225000 [58:52:21<380:44:36,  6.85s/it]
 11%|█         | 24783/225000 [58:52:26<345:59:02,  6.22s/it]
 11%|█         | 24784/225000 [58:52:32<354:04:42,  6.37s/it]
 11%|█         | 24785/225000 [58:52:41<399:16:35,  7.18s/it]
 11%|█         | 24786/225000 [58:52:45<334:08:26,  6.01s/it]
 11%|█         | 24787/225000 [58:52:51<330:58:15,  5.95s/it]
 11%|█         | 24788/225000 [58:52:59<379:15:47,  6.82s/it]
 11%|█         | 24789/225000 [58:53:03<331:54:27,  5.97s/it]
 11%|█         | 24790/225000 [58:53:08<307:54:47,  5.54s/it]
 11%|█         | 24791/225000 [58:53:18<385:52:55,  6.94s/it]
 11%|█         | 24792/225000 [58:53:25<377:55:31,  6.80s/it]
 11%|█         | 24793/225000 [58:53:29<336:30:36,  6.05s/it]
 11%|█         | 24794/225000 [58:53:39<405:15:49,  7.29s/it]
 11%|█         | 24795/225000 [58:53:44<371:39:49,  6.68s/it]
 11%|█         | 24796/225000 [58:53:52<382:00:24,  6.87s/it]
 11%|█         | 24797/225000 [58:53:57<363:14:33,  6.53s/it]
 11%|█         | 24798/225000 [58:54:05<377:16:49,  6.78s/it]
 11%|█         | 24799/225000 [58:54:10<356:39:18,  6.41s/it]
 11%|█         | 24800/225000 [58:54:16<342:24:40,  6.16s/it]
                                                             
{'loss': '0.7272', 'grad_norm': '0.7269', 'learning_rate': '4.942e-05', 'epoch': '0.1102', 'num_input_tokens_seen': 102081472, 'train_runtime': '2.121e+05', 'train_tokens_per_second': '481.4'}

 11%|█         | 24800/225000 [58:54:16<342:24:40,  6.16s/it]
 11%|█         | 24801/225000 [58:54:21<327:47:07,  5.89s/it]
 11%|█         | 24802/225000 [58:54:30<380:48:24,  6.85s/it]
 11%|█         | 24803/225000 [58:54:36<364:37:33,  6.56s/it]
 11%|█         | 24804/225000 [58:54:40<325:28:21,  5.85s/it]
 11%|█         | 24805/225000 [58:54:45<308:52:53,  5.55s/it]
 11%|█         | 24806/225000 [58:54:50<301:08:09,  5.42s/it]
 11%|█         | 24807/225000 [58:55:04<439:56:31,  7.91s/it]
 11%|█         | 24808/225000 [58:55:07<366:06:07,  6.58s/it]
 11%|█         | 24809/225000 [58:55:13<347:54:42,  6.26s/it]
 11%|█         | 24810/225000 [58:55:19<341:27:11,  6.14s/it]
 11%|█         | 24811/225000 [58:55:25<338:44:12,  6.09s/it]
 11%|█         | 24812/225000 [58:55:32<349:11:19,  6.28s/it]
 11%|█         | 24813/225000 [58:55:37<336:28:11,  6.05s/it]
 11%|█         | 24814/225000 [58:55:42<323:28:53,  5.82s/it]
 11%|█         | 24815/225000 [58:55:48<320:41:42,  5.77s/it]
 11%|█         | 24816/225000 [58:55:53<304:13:04,  5.47s/it]
 11%|█         | 24817/225000 [58:55:58<300:14:55,  5.40s/it]
 11%|█         | 24818/225000 [58:56:07<364:23:38,  6.55s/it]
 11%|█         | 24819/225000 [58:56:15<384:26:56,  6.91s/it]
 11%|█         | 24820/225000 [58:56:20<357:54:31,  6.44s/it]
 11%|█         | 24821/225000 [58:56:25<336:01:02,  6.04s/it]
 11%|█         | 24822/225000 [58:56:31<330:46:53,  5.95s/it]
 11%|█         | 24823/225000 [58:56:40<376:03:32,  6.76s/it]
 11%|█         | 24824/225000 [58:56:52<470:41:37,  8.47s/it]
 11%|█         | 24825/225000 [58:56:56<387:59:34,  6.98s/it]
 11%|█         | 24826/225000 [58:57:03<394:00:20,  7.09s/it]
 11%|█         | 24827/225000 [58:57:07<346:59:22,  6.24s/it]
 11%|█         | 24828/225000 [58:57:11<311:05:44,  5.59s/it]
 11%|█         | 24829/225000 [58:57:20<357:30:28,  6.43s/it]
 11%|█         | 24830/225000 [58:57:29<396:21:29,  7.13s/it]
 11%|█         | 24831/225000 [58:57:34<372:28:40,  6.70s/it]
 11%|█         | 24832/225000 [58:57:40<362:04:02,  6.51s/it]
 11%|█         | 24833/225000 [58:57:44<321:10:16,  5.78s/it]
 11%|█         | 24834/225000 [58:57:49<307:32:12,  5.53s/it]
 11%|█         | 24835/225000 [58:58:01<408:32:52,  7.35s/it]
 11%|█         | 24836/225000 [58:58:04<344:07:17,  6.19s/it]
 11%|█         | 24837/225000 [58:58:13<381:00:06,  6.85s/it]
 11%|█         | 24838/225000 [58:58:17<331:20:12,  5.96s/it]
 11%|█         | 24839/225000 [58:58:27<406:18:56,  7.31s/it]
 11%|█         | 24840/225000 [58:58:32<365:34:42,  6.58s/it]
 11%|█         | 24841/225000 [58:58:37<338:13:17,  6.08s/it]
 11%|█         | 24842/225000 [58:58:46<386:33:53,  6.95s/it]
 11%|█         | 24843/225000 [58:58:51<358:13:22,  6.44s/it]
 11%|█         | 24844/225000 [58:58:57<347:45:15,  6.25s/it]
 11%|█         | 24845/225000 [58:59:01<310:04:09,  5.58s/it]
 11%|█         | 24846/225000 [58:59:06<296:36:14,  5.33s/it]
 11%|█         | 24847/225000 [58:59:18<405:40:52,  7.30s/it]
 11%|█         | 24848/225000 [58:59:24<387:18:54,  6.97s/it]
 11%|█         | 24849/225000 [58:59:29<349:49:39,  6.29s/it]
 11%|█         | 24850/225000 [58:59:33<324:42:53,  5.84s/it]
                                                             
{'loss': '0.725', 'grad_norm': '0.7174', 'learning_rate': '4.941e-05', 'epoch': '0.1104', 'num_input_tokens_seen': 102223896, 'train_runtime': '2.124e+05', 'train_tokens_per_second': '481.3'}

 11%|█         | 24850/225000 [58:59:33<324:42:53,  5.84s/it]
 11%|█         | 24851/225000 [58:59:39<314:50:10,  5.66s/it]
 11%|█         | 24852/225000 [58:59:44<304:41:28,  5.48s/it]
 11%|█         | 24853/225000 [58:59:56<419:53:19,  7.55s/it]
 11%|█         | 24854/225000 [59:00:00<366:14:35,  6.59s/it]
 11%|█         | 24855/225000 [59:00:05<337:18:10,  6.07s/it]
 11%|█         | 24856/225000 [59:00:10<313:40:41,  5.64s/it]
 11%|█         | 24857/225000 [59:00:19<376:32:03,  6.77s/it]
 11%|█         | 24858/225000 [59:00:28<412:31:30,  7.42s/it]
 11%|█         | 24859/225000 [59:00:35<400:35:59,  7.21s/it]
 11%|█         | 24860/225000 [59:00:39<348:36:29,  6.27s/it]
 11%|█         | 24861/225000 [59:00:44<331:08:26,  5.96s/it]
 11%|█         | 24862/225000 [59:00:53<385:03:30,  6.93s/it]
 11%|█         | 24863/225000 [59:00:59<363:49:33,  6.54s/it]
 11%|█         | 24864/225000 [59:01:07<378:48:24,  6.81s/it]
 11%|█         | 24865/225000 [59:01:13<374:14:10,  6.73s/it]
 11%|█         | 24866/225000 [59:01:17<323:39:34,  5.82s/it]
 11%|█         | 24867/225000 [59:01:27<391:29:21,  7.04s/it]
 11%|█         | 24868/225000 [59:01:31<344:44:06,  6.20s/it]
 11%|█         | 24869/225000 [59:01:37<334:57:46,  6.03s/it]
 11%|█         | 24870/225000 [59:01:43<338:09:43,  6.08s/it]
 11%|█         | 24871/225000 [59:01:54<417:31:27,  7.51s/it]
 11%|█         | 24872/225000 [59:02:00<395:38:14,  7.12s/it]
 11%|█         | 24873/225000 [59:02:08<414:14:19,  7.45s/it]
 11%|█         | 24874/225000 [59:02:13<366:22:43,  6.59s/it]
 11%|█         | 24875/225000 [59:02:18<352:04:26,  6.33s/it]
 11%|█         | 24876/225000 [59:02:24<334:53:48,  6.02s/it]
 11%|█         | 24877/225000 [59:02:30<335:43:03,  6.04s/it]
 11%|█         | 24878/225000 [59:02:35<318:28:12,  5.73s/it]
 11%|█         | 24879/225000 [59:02:44<378:19:02,  6.81s/it]
 11%|█         | 24880/225000 [59:02:54<437:10:24,  7.86s/it]
 11%|█         | 24881/225000 [59:02:59<376:28:12,  6.77s/it]
 11%|█         | 24882/225000 [59:03:03<343:57:13,  6.19s/it]
 11%|█         | 24883/225000 [59:03:09<339:21:23,  6.10s/it]
 11%|█         | 24884/225000 [59:03:15<327:46:37,  5.90s/it]
 11%|█         | 24885/225000 [59:03:20<317:38:05,  5.71s/it]
 11%|█         | 24886/225000 [59:03:26<316:49:55,  5.70s/it]
 11%|█         | 24887/225000 [59:03:31<314:21:10,  5.66s/it]
 11%|█         | 24888/225000 [59:03:39<342:54:36,  6.17s/it]
 11%|█         | 24889/225000 [59:03:44<328:54:06,  5.92s/it]
 11%|█         | 24890/225000 [59:03:56<425:54:47,  7.66s/it]
 11%|█         | 24891/225000 [59:04:04<434:52:50,  7.82s/it]
 11%|█         | 24892/225000 [59:04:09<391:21:33,  7.04s/it]
 11%|█         | 24893/225000 [59:04:14<348:52:32,  6.28s/it]
 11%|█         | 24894/225000 [59:04:19<331:28:50,  5.96s/it]
 11%|█         | 24895/225000 [59:04:28<388:05:20,  6.98s/it]
 11%|█         | 24896/225000 [59:04:37<412:28:56,  7.42s/it]
 11%|█         | 24897/225000 [59:04:44<416:54:20,  7.50s/it]
 11%|█         | 24898/225000 [59:04:49<375:05:24,  6.75s/it]
 11%|█         | 24899/225000 [59:04:59<418:35:41,  7.53s/it]
 11%|█         | 24900/225000 [59:05:05<396:49:49,  7.14s/it]
                                                             
{'loss': '0.7126', 'grad_norm': '0.6767', 'learning_rate': '4.941e-05', 'epoch': '0.1107', 'num_input_tokens_seen': 102370288, 'train_runtime': '2.127e+05', 'train_tokens_per_second': '481.3'}

 11%|█         | 24900/225000 [59:05:05<396:49:49,  7.14s/it]
 11%|█         | 24901/225000 [59:05:10<359:54:27,  6.48s/it]
 11%|█         | 24902/225000 [59:05:15<336:27:06,  6.05s/it]
 11%|█         | 24903/225000 [59:05:20<319:53:57,  5.76s/it]
 11%|█         | 24904/225000 [59:05:25<304:50:49,  5.48s/it]
 11%|█         | 24905/225000 [59:05:30<307:13:00,  5.53s/it]
 11%|█         | 24906/225000 [59:05:36<314:16:20,  5.65s/it]
 11%|█         | 24907/225000 [59:05:48<418:14:45,  7.52s/it]
 11%|█         | 24908/225000 [59:05:53<379:58:55,  6.84s/it]
 11%|█         | 24909/225000 [59:05:58<349:24:49,  6.29s/it]
 11%|█         | 24910/225000 [59:06:04<334:16:05,  6.01s/it]
 11%|█         | 24911/225000 [59:06:09<316:50:33,  5.70s/it]
 11%|█         | 24912/225000 [59:06:15<316:41:26,  5.70s/it]
 11%|█         | 24913/225000 [59:06:25<392:28:29,  7.06s/it]
 11%|█         | 24914/225000 [59:06:33<412:09:29,  7.42s/it]
 11%|█         | 24915/225000 [59:06:39<393:48:59,  7.09s/it]
 11%|█         | 24916/225000 [59:06:46<386:08:20,  6.95s/it]
 11%|█         | 24917/225000 [59:06:51<346:03:41,  6.23s/it]
 11%|█         | 24918/225000 [59:06:55<317:18:56,  5.71s/it]
 11%|█         | 24919/225000 [59:07:00<303:28:45,  5.46s/it]
 11%|█         | 24920/225000 [59:07:05<299:27:25,  5.39s/it]
 11%|█         | 24921/225000 [59:07:15<381:01:56,  6.86s/it]
 11%|█         | 24922/225000 [59:07:23<396:23:21,  7.13s/it]
 11%|█         | 24923/225000 [59:07:28<353:27:33,  6.36s/it]
 11%|█         | 24924/225000 [59:07:38<410:54:18,  7.39s/it]
 11%|█         | 24925/225000 [59:07:41<352:40:29,  6.35s/it]
 11%|█         | 24926/225000 [59:07:46<318:21:09,  5.73s/it]
 11%|█         | 24927/225000 [59:07:53<347:21:39,  6.25s/it]
 11%|█         | 24928/225000 [59:08:04<421:45:16,  7.59s/it]
 11%|█         | 24929/225000 [59:08:11<412:31:37,  7.42s/it]
 11%|█         | 24930/225000 [59:08:15<364:36:26,  6.56s/it]
 11%|█         | 24931/225000 [59:08:20<330:57:23,  5.96s/it]
 11%|█         | 24932/225000 [59:08:25<312:40:45,  5.63s/it]
 11%|█         | 24933/225000 [59:08:36<401:36:10,  7.23s/it]
 11%|█         | 24934/225000 [59:08:40<357:29:39,  6.43s/it]
 11%|█         | 24935/225000 [59:08:46<350:14:22,  6.30s/it]
 11%|█         | 24936/225000 [59:08:55<391:00:55,  7.04s/it]
 11%|█         | 24937/225000 [59:08:59<332:19:09,  5.98s/it]
 11%|█         | 24938/225000 [59:09:09<411:21:14,  7.40s/it]
 11%|█         | 24939/225000 [59:09:13<352:23:23,  6.34s/it]
 11%|█         | 24940/225000 [59:09:19<336:37:09,  6.06s/it]
 11%|█         | 24941/225000 [59:09:27<371:06:08,  6.68s/it]
 11%|█         | 24942/225000 [59:09:33<367:38:23,  6.62s/it]
 11%|█         | 24943/225000 [59:09:38<338:09:55,  6.09s/it]
 11%|█         | 24944/225000 [59:09:43<310:57:56,  5.60s/it]
 11%|█         | 24945/225000 [59:09:48<302:50:14,  5.45s/it]
 11%|█         | 24946/225000 [59:09:59<399:13:48,  7.18s/it]
 11%|█         | 24947/225000 [59:10:03<341:47:08,  6.15s/it]
 11%|█         | 24948/225000 [59:10:08<328:05:23,  5.90s/it]
 11%|█         | 24949/225000 [59:10:13<320:36:02,  5.77s/it]
 11%|█         | 24950/225000 [59:10:19<311:12:59,  5.60s/it]
                                                             
{'loss': '0.7142', 'grad_norm': '0.7531', 'learning_rate': '4.941e-05', 'epoch': '0.1109', 'num_input_tokens_seen': 102514872, 'train_runtime': '2.13e+05', 'train_tokens_per_second': '481.2'}

 11%|█         | 24950/225000 [59:10:19<311:12:59,  5.60s/it]
 11%|█         | 24951/225000 [59:10:24<314:15:55,  5.66s/it]
 11%|█         | 24952/225000 [59:10:37<435:38:23,  7.84s/it]
 11%|█         | 24953/225000 [59:10:43<400:41:38,  7.21s/it]
 11%|█         | 24954/225000 [59:10:47<346:40:14,  6.24s/it]
 11%|█         | 24955/225000 [59:10:55<367:15:16,  6.61s/it]
 11%|█         | 24956/225000 [59:11:03<391:20:11,  7.04s/it]
 11%|█         | 24957/225000 [59:11:06<334:26:52,  6.02s/it]
 11%|█         | 24958/225000 [59:11:12<336:43:37,  6.06s/it]
 11%|█         | 24959/225000 [59:11:23<410:56:32,  7.40s/it]
 11%|█         | 24960/225000 [59:11:31<419:29:00,  7.55s/it]
 11%|█         | 24961/225000 [59:11:34<352:47:27,  6.35s/it]
 11%|█         | 24962/225000 [59:11:40<335:51:15,  6.04s/it]
 11%|█         | 24963/225000 [59:11:45<328:33:08,  5.91s/it]
 11%|█         | 24964/225000 [59:11:51<317:59:14,  5.72s/it]
 11%|█         | 24965/225000 [59:11:56<315:39:39,  5.68s/it]
 11%|█         | 24966/225000 [59:12:02<310:22:44,  5.59s/it]
 11%|█         | 24967/225000 [59:12:07<308:42:55,  5.56s/it]
 11%|█         | 24968/225000 [59:12:13<308:41:37,  5.56s/it]
 11%|█         | 24969/225000 [59:12:18<305:11:49,  5.49s/it]
 11%|█         | 24970/225000 [59:12:28<381:27:09,  6.87s/it]
 11%|█         | 24971/225000 [59:12:34<362:48:27,  6.53s/it]
 11%|█         | 24972/225000 [59:12:39<342:55:05,  6.17s/it]
 11%|█         | 24973/225000 [59:12:44<326:27:21,  5.88s/it]
 11%|█         | 24974/225000 [59:12:49<309:12:18,  5.56s/it]
 11%|█         | 24975/225000 [59:12:55<308:30:54,  5.55s/it]
 11%|█         | 24976/225000 [59:13:08<433:04:47,  7.79s/it]
 11%|█         | 24977/225000 [59:13:15<422:12:55,  7.60s/it]
 11%|█         | 24978/225000 [59:13:20<387:46:54,  6.98s/it]
 11%|█         | 24979/225000 [59:13:28<404:15:16,  7.28s/it]
 11%|█         | 24980/225000 [59:13:36<404:15:57,  7.28s/it]
 11%|█         | 24981/225000 [59:13:42<394:36:27,  7.10s/it]
 11%|█         | 24982/225000 [59:13:47<361:30:27,  6.51s/it]
 11%|█         | 24983/225000 [59:13:52<327:00:50,  5.89s/it]
 11%|█         | 24984/225000 [59:14:04<424:34:57,  7.64s/it]
 11%|█         | 24985/225000 [59:14:10<399:23:18,  7.19s/it]
 11%|█         | 24986/225000 [59:14:14<355:56:49,  6.41s/it]
 11%|█         | 24987/225000 [59:14:19<331:57:19,  5.97s/it]
 11%|█         | 24988/225000 [59:14:25<324:44:03,  5.84s/it]
 11%|█         | 24989/225000 [59:14:37<424:02:44,  7.63s/it]
 11%|█         | 24990/225000 [59:14:43<397:58:36,  7.16s/it]
 11%|█         | 24991/225000 [59:14:47<353:51:49,  6.37s/it]
 11%|█         | 24992/225000 [59:14:52<320:56:30,  5.78s/it]
 11%|█         | 24993/225000 [59:14:56<307:02:58,  5.53s/it]
 11%|█         | 24994/225000 [59:15:09<426:40:25,  7.68s/it]
 11%|█         | 24995/225000 [59:15:13<365:12:38,  6.57s/it]
 11%|█         | 24996/225000 [59:15:17<326:59:40,  5.89s/it]
 11%|█         | 24997/225000 [59:15:27<382:29:39,  6.88s/it]
 11%|█         | 24998/225000 [59:15:35<405:14:02,  7.29s/it]
 11%|█         | 24999/225000 [59:15:39<347:50:58,  6.26s/it]
 11%|█         | 25000/225000 [59:15:48<402:18:05,  7.24s/it]
                                                             
{'loss': '0.7204', 'grad_norm': '0.744', 'learning_rate': '4.94e-05', 'epoch': '0.1111', 'num_input_tokens_seen': 102660112, 'train_runtime': '2.133e+05', 'train_tokens_per_second': '481.2'}

 11%|█         | 25000/225000 [59:15:48<402:18:05,  7.24s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                             
{'eval_loss': '1.06', 'eval_runtime': '456.5', 'eval_samples_per_second': '1.095', 'eval_steps_per_second': '0.548', 'epoch': '0.1111', 'num_input_tokens_seen': 102660112}

 11%|█         | 25000/225000 [59:23:25<402:18:05,  7.24s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-25000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 11%|█         | 25001/225000 [59:23:31<7987:49:17, 143.78s/it]
 11%|█         | 25002/225000 [59:23:35<5670:10:54, 102.06s/it]
 11%|█         | 25003/225000 [59:23:40<4038:40:38, 72.70s/it] 
 11%|█         | 25004/225000 [59:23:44<2904:34:26, 52.28s/it]
 11%|█         | 25005/225000 [59:23:50<2134:58:56, 38.43s/it]
 11%|█         | 25006/225000 [59:24:03<1705:21:15, 30.70s/it]
 11%|█         | 25007/225000 [59:24:07<1259:22:35, 22.67s/it]
 11%|█         | 25008/225000 [59:24:15<1011:40:55, 18.21s/it]
 11%|█         | 25009/225000 [59:24:21<807:46:58, 14.54s/it] 
 11%|█         | 25010/225000 [59:24:25<641:11:28, 11.54s/it]
 11%|█         | 25011/225000 [59:24:32<563:28:46, 10.14s/it]
 11%|█         | 25012/225000 [59:24:37<482:51:48,  8.69s/it]
 11%|█         | 25013/225000 [59:24:43<427:21:30,  7.69s/it]
 11%|█         | 25014/225000 [59:24:48<381:13:43,  6.86s/it]
 11%|█         | 25015/225000 [59:24:53<360:08:53,  6.48s/it]
 11%|█         | 25016/225000 [59:24:58<333:45:14,  6.01s/it]
 11%|█         | 25017/225000 [59:25:10<429:43:18,  7.74s/it]
 11%|█         | 25018/225000 [59:25:15<386:06:36,  6.95s/it]
 11%|█         | 25019/225000 [59:25:20<348:14:06,  6.27s/it]
 11%|█         | 25020/225000 [59:25:30<417:40:11,  7.52s/it]
 11%|█         | 25021/225000 [59:25:35<372:09:35,  6.70s/it]
 11%|█         | 25022/225000 [59:25:43<385:54:11,  6.95s/it]
 11%|█         | 25023/225000 [59:25:47<341:39:25,  6.15s/it]
 11%|█         | 25024/225000 [59:25:51<310:42:26,  5.59s/it]
 11%|█         | 25025/225000 [59:26:01<379:46:09,  6.84s/it]
 11%|█         | 25026/225000 [59:26:07<371:06:13,  6.68s/it]
 11%|█         | 25027/225000 [59:26:15<394:06:48,  7.10s/it]
 11%|█         | 25028/225000 [59:26:19<339:42:31,  6.12s/it]
 11%|█         | 25029/225000 [59:26:28<390:59:56,  7.04s/it]
 11%|█         | 25030/225000 [59:26:33<352:10:22,  6.34s/it]
 11%|█         | 25031/225000 [59:26:37<320:36:47,  5.77s/it]
 11%|█         | 25032/225000 [59:26:43<311:20:25,  5.61s/it]
 11%|█         | 25033/225000 [59:26:56<437:28:22,  7.88s/it]
 11%|█         | 25034/225000 [59:27:00<379:19:03,  6.83s/it]
 11%|█         | 25035/225000 [59:27:04<337:02:02,  6.07s/it]
 11%|█         | 25036/225000 [59:27:09<314:57:18,  5.67s/it]
 11%|█         | 25037/225000 [59:27:22<430:23:07,  7.75s/it]
 11%|█         | 25038/225000 [59:27:27<395:22:30,  7.12s/it]
 11%|█         | 25039/225000 [59:27:32<345:25:14,  6.22s/it]
 11%|█         | 25040/225000 [59:27:42<414:10:53,  7.46s/it]
 11%|█         | 25041/225000 [59:27:47<374:10:37,  6.74s/it]
 11%|█         | 25042/225000 [59:27:51<335:02:54,  6.03s/it]
 11%|█         | 25043/225000 [59:27:57<329:55:03,  5.94s/it]
 11%|█         | 25044/225000 [59:28:07<397:22:40,  7.15s/it]
 11%|█         | 25045/225000 [59:28:11<351:23:49,  6.33s/it]
 11%|█         | 25046/225000 [59:28:16<324:02:25,  5.83s/it]
 11%|█         | 25047/225000 [59:28:21<307:39:35,  5.54s/it]
 11%|█         | 25048/225000 [59:28:26<293:07:24,  5.28s/it]
 11%|█         | 25049/225000 [59:28:37<394:26:37,  7.10s/it]
 11%|█         | 25050/225000 [59:28:44<392:05:05,  7.06s/it]
                                                             
{'loss': '0.7118', 'grad_norm': '0.7177', 'learning_rate': '4.94e-05', 'epoch': '0.1113', 'num_input_tokens_seen': 102804288, 'train_runtime': '2.141e+05', 'train_tokens_per_second': '480.1'}

 11%|█         | 25050/225000 [59:28:44<392:05:05,  7.06s/it]
 11%|█         | 25051/225000 [59:28:49<350:03:04,  6.30s/it]
 11%|█         | 25052/225000 [59:28:58<409:42:07,  7.38s/it]
 11%|█         | 25053/225000 [59:29:04<384:33:58,  6.92s/it]
 11%|█         | 25054/225000 [59:29:11<380:02:14,  6.84s/it]
 11%|█         | 25055/225000 [59:29:18<375:15:39,  6.76s/it]
 11%|█         | 25056/225000 [59:29:24<363:41:32,  6.55s/it]
 11%|█         | 25057/225000 [59:29:28<328:56:51,  5.92s/it]
 11%|█         | 25058/225000 [59:29:33<305:45:44,  5.51s/it]
 11%|█         | 25059/225000 [59:29:44<398:35:30,  7.18s/it]
 11%|█         | 25060/225000 [59:29:47<342:16:21,  6.16s/it]
 11%|█         | 25061/225000 [59:29:57<401:47:06,  7.23s/it]
 11%|█         | 25062/225000 [59:30:01<345:34:31,  6.22s/it]
 11%|█         | 25063/225000 [59:30:10<386:11:51,  6.95s/it]
 11%|█         | 25064/225000 [59:30:14<345:34:20,  6.22s/it]
 11%|█         | 25065/225000 [59:30:21<361:56:59,  6.52s/it]
 11%|█         | 25066/225000 [59:30:26<327:30:25,  5.90s/it]
 11%|█         | 25067/225000 [59:30:30<306:05:44,  5.51s/it]
 11%|█         | 25068/225000 [59:30:35<290:06:34,  5.22s/it]
 11%|█         | 25069/225000 [59:30:39<277:22:49,  4.99s/it]
 11%|█         | 25070/225000 [59:30:48<334:46:40,  6.03s/it]
 11%|█         | 25071/225000 [59:30:58<398:23:23,  7.17s/it]
 11%|█         | 25072/225000 [59:31:02<355:13:57,  6.40s/it]
 11%|█         | 25073/225000 [59:31:07<318:16:08,  5.73s/it]
 11%|█         | 25074/225000 [59:31:17<399:12:29,  7.19s/it]
 11%|█         | 25075/225000 [59:31:23<378:45:10,  6.82s/it]
 11%|█         | 25076/225000 [59:31:29<356:51:41,  6.43s/it]
 11%|█         | 25077/225000 [59:31:36<365:56:53,  6.59s/it]
 11%|█         | 25078/225000 [59:31:40<325:51:56,  5.87s/it]
 11%|█         | 25079/225000 [59:31:44<306:06:38,  5.51s/it]
 11%|█         | 25080/225000 [59:31:56<403:40:33,  7.27s/it]
 11%|█         | 25081/225000 [59:32:00<352:19:13,  6.34s/it]
 11%|█         | 25082/225000 [59:32:09<392:05:15,  7.06s/it]
 11%|█         | 25083/225000 [59:32:15<371:19:08,  6.69s/it]
 11%|█         | 25084/225000 [59:32:22<377:45:36,  6.80s/it]
 11%|█         | 25085/225000 [59:32:26<345:55:22,  6.23s/it]
 11%|█         | 25086/225000 [59:32:31<324:18:18,  5.84s/it]
 11%|█         | 25087/225000 [59:32:36<311:17:39,  5.61s/it]
 11%|█         | 25088/225000 [59:32:41<298:21:40,  5.37s/it]
 11%|█         | 25089/225000 [59:32:54<425:13:33,  7.66s/it]
 11%|█         | 25090/225000 [59:33:01<402:45:28,  7.25s/it]
 11%|█         | 25091/225000 [59:33:08<410:08:51,  7.39s/it]
 11%|█         | 25092/225000 [59:33:14<385:26:29,  6.94s/it]
 11%|█         | 25093/225000 [59:33:22<396:25:14,  7.14s/it]
 11%|█         | 25094/225000 [59:33:26<347:23:16,  6.26s/it]
 11%|█         | 25095/225000 [59:33:30<312:30:37,  5.63s/it]
 11%|█         | 25096/225000 [59:33:41<398:19:59,  7.17s/it]
 11%|█         | 25097/225000 [59:33:45<354:09:14,  6.38s/it]
 11%|█         | 25098/225000 [59:33:54<391:22:00,  7.05s/it]
 11%|█         | 25099/225000 [59:33:58<336:43:39,  6.06s/it]
 11%|█         | 25100/225000 [59:34:04<334:47:59,  6.03s/it]
                                                             
{'loss': '0.7189', 'grad_norm': '0.622', 'learning_rate': '4.939e-05', 'epoch': '0.1116', 'num_input_tokens_seen': 102948952, 'train_runtime': '2.144e+05', 'train_tokens_per_second': '480.1'}

 11%|█         | 25100/225000 [59:34:04<334:47:59,  6.03s/it]
 11%|█         | 25101/225000 [59:34:13<384:31:27,  6.92s/it]
 11%|█         | 25102/225000 [59:34:18<350:18:29,  6.31s/it]
 11%|█         | 25103/225000 [59:34:25<366:53:28,  6.61s/it]
 11%|█         | 25104/225000 [59:34:28<313:07:05,  5.64s/it]
 11%|█         | 25105/225000 [59:34:38<376:56:22,  6.79s/it]
 11%|█         | 25106/225000 [59:34:43<342:26:22,  6.17s/it]
 11%|█         | 25107/225000 [59:34:47<313:03:56,  5.64s/it]
 11%|█         | 25108/225000 [59:34:52<295:52:08,  5.33s/it]
 11%|█         | 25109/225000 [59:35:01<364:09:47,  6.56s/it]
 11%|█         | 25110/225000 [59:35:08<379:34:48,  6.84s/it]
 11%|█         | 25111/225000 [59:35:16<397:42:35,  7.16s/it]
 11%|█         | 25112/225000 [59:35:24<403:44:56,  7.27s/it]
 11%|█         | 25113/225000 [59:35:30<378:24:17,  6.82s/it]
 11%|█         | 25114/225000 [59:35:39<419:46:15,  7.56s/it]
 11%|█         | 25115/225000 [59:35:45<396:46:14,  7.15s/it]
 11%|█         | 25116/225000 [59:35:49<341:16:25,  6.15s/it]
 11%|█         | 25117/225000 [59:36:00<417:52:26,  7.53s/it]
 11%|█         | 25118/225000 [59:36:03<348:46:45,  6.28s/it]
 11%|█         | 25119/225000 [59:36:08<325:14:25,  5.86s/it]
 11%|█         | 25120/225000 [59:36:13<312:37:01,  5.63s/it]
 11%|█         | 25121/225000 [59:36:19<311:04:20,  5.60s/it]
 11%|█         | 25122/225000 [59:36:23<293:48:13,  5.29s/it]
 11%|█         | 25123/225000 [59:36:28<290:36:39,  5.23s/it]
 11%|█         | 25124/225000 [59:36:34<295:24:20,  5.32s/it]
 11%|█         | 25125/225000 [59:36:47<425:09:25,  7.66s/it]
 11%|█         | 25126/225000 [59:36:53<399:12:38,  7.19s/it]
 11%|█         | 25127/225000 [59:36:57<342:45:02,  6.17s/it]
 11%|█         | 25128/225000 [59:37:07<413:03:02,  7.44s/it]
 11%|█         | 25129/225000 [59:37:12<366:54:05,  6.61s/it]
 11%|█         | 25130/225000 [59:37:17<338:30:37,  6.10s/it]
 11%|█         | 25131/225000 [59:37:25<380:15:33,  6.85s/it]
 11%|█         | 25132/225000 [59:37:29<332:35:17,  5.99s/it]
 11%|█         | 25133/225000 [59:37:34<306:01:06,  5.51s/it]
 11%|█         | 25134/225000 [59:37:43<367:31:58,  6.62s/it]
 11%|█         | 25135/225000 [59:37:49<362:40:11,  6.53s/it]
 11%|█         | 25136/225000 [59:37:57<387:39:44,  6.98s/it]
 11%|█         | 25137/225000 [59:38:02<351:54:08,  6.34s/it]
 11%|█         | 25138/225000 [59:38:12<414:17:32,  7.46s/it]
 11%|█         | 25139/225000 [59:38:16<350:28:44,  6.31s/it]
 11%|█         | 25140/225000 [59:38:22<354:16:10,  6.38s/it]
 11%|█         | 25141/225000 [59:38:28<346:25:51,  6.24s/it]
 11%|█         | 25142/225000 [59:38:33<318:09:02,  5.73s/it]
 11%|█         | 25143/225000 [59:38:37<297:23:26,  5.36s/it]
 11%|█         | 25144/225000 [59:38:42<292:55:24,  5.28s/it]
 11%|█         | 25145/225000 [59:38:56<425:01:13,  7.66s/it]
 11%|█         | 25146/225000 [59:39:01<381:44:22,  6.88s/it]
 11%|█         | 25147/225000 [59:39:08<382:38:39,  6.89s/it]
 11%|█         | 25148/225000 [59:39:12<332:10:57,  5.98s/it]
 11%|█         | 25149/225000 [59:39:23<418:22:14,  7.54s/it]
 11%|█         | 25150/225000 [59:39:28<375:26:21,  6.76s/it]
                                                             
{'loss': '0.7215', 'grad_norm': '0.6774', 'learning_rate': '4.939e-05', 'epoch': '0.1118', 'num_input_tokens_seen': 103092152, 'train_runtime': '2.148e+05', 'train_tokens_per_second': '480'}

 11%|█         | 25150/225000 [59:39:28<375:26:21,  6.76s/it]
 11%|█         | 25151/225000 [59:39:32<342:41:45,  6.17s/it]
 11%|█         | 25152/225000 [59:39:38<328:50:19,  5.92s/it]
 11%|█         | 25153/225000 [59:39:43<323:14:52,  5.82s/it]
 11%|█         | 25154/225000 [59:39:49<316:16:32,  5.70s/it]
 11%|█         | 25155/225000 [59:39:56<334:07:46,  6.02s/it]
 11%|█         | 25156/225000 [59:40:10<470:26:57,  8.47s/it]
 11%|█         | 25157/225000 [59:40:14<393:02:47,  7.08s/it]
 11%|█         | 25158/225000 [59:40:18<346:53:23,  6.25s/it]
 11%|█         | 25159/225000 [59:40:25<358:24:59,  6.46s/it]
 11%|█         | 25160/225000 [59:40:33<392:28:08,  7.07s/it]
 11%|█         | 25161/225000 [59:40:38<350:26:20,  6.31s/it]
 11%|█         | 25162/225000 [59:40:42<321:02:38,  5.78s/it]
 11%|█         | 25163/225000 [59:40:50<357:35:20,  6.44s/it]
 11%|█         | 25164/225000 [59:40:58<379:46:17,  6.84s/it]
 11%|█         | 25165/225000 [59:41:02<331:50:50,  5.98s/it]
 11%|█         | 25166/225000 [59:41:13<416:37:40,  7.51s/it]
 11%|█         | 25167/225000 [59:41:17<351:53:29,  6.34s/it]
 11%|█         | 25168/225000 [59:41:21<319:55:39,  5.76s/it]
 11%|█         | 25169/225000 [59:41:26<296:20:04,  5.34s/it]
 11%|█         | 25170/225000 [59:41:38<409:08:58,  7.37s/it]
 11%|█         | 25171/225000 [59:41:44<389:58:46,  7.03s/it]
 11%|█         | 25172/225000 [59:41:51<397:49:05,  7.17s/it]
 11%|█         | 25173/225000 [59:41:55<337:16:29,  6.08s/it]
 11%|█         | 25174/225000 [59:42:00<323:32:15,  5.83s/it]
 11%|█         | 25175/225000 [59:42:05<308:10:05,  5.55s/it]
 11%|█         | 25176/225000 [59:42:11<309:24:16,  5.57s/it]
 11%|█         | 25177/225000 [59:42:16<299:27:55,  5.40s/it]
 11%|█         | 25178/225000 [59:42:21<299:48:00,  5.40s/it]
 11%|█         | 25179/225000 [59:42:32<391:21:45,  7.05s/it]
 11%|█         | 25180/225000 [59:42:42<437:03:21,  7.87s/it]
 11%|█         | 25181/225000 [59:42:46<381:40:09,  6.88s/it]
 11%|█         | 25182/225000 [59:42:51<347:22:13,  6.26s/it]
 11%|█         | 25183/225000 [59:43:03<439:45:46,  7.92s/it]
 11%|█         | 25184/225000 [59:43:09<405:46:04,  7.31s/it]
 11%|█         | 25185/225000 [59:43:16<411:09:11,  7.41s/it]
 11%|█         | 25186/225000 [59:43:22<382:47:19,  6.90s/it]
 11%|█         | 25187/225000 [59:43:27<346:51:40,  6.25s/it]
 11%|█         | 25188/225000 [59:43:32<328:46:50,  5.92s/it]
 11%|█         | 25189/225000 [59:43:37<318:52:01,  5.75s/it]
 11%|█         | 25190/225000 [59:43:45<342:14:25,  6.17s/it]
 11%|█         | 25191/225000 [59:43:56<424:28:13,  7.65s/it]
 11%|█         | 25192/225000 [59:44:02<399:38:57,  7.20s/it]
 11%|█         | 25193/225000 [59:44:06<356:29:06,  6.42s/it]
 11%|█         | 25194/225000 [59:44:11<323:26:24,  5.83s/it]
 11%|█         | 25195/225000 [59:44:22<404:36:48,  7.29s/it]
 11%|█         | 25196/225000 [59:44:25<343:04:51,  6.18s/it]
 11%|█         | 25197/225000 [59:44:35<411:23:49,  7.41s/it]
 11%|█         | 25198/225000 [59:44:39<352:13:50,  6.35s/it]
 11%|█         | 25199/225000 [59:44:48<385:00:26,  6.94s/it]
 11%|█         | 25200/225000 [59:44:54<378:49:50,  6.83s/it]
                                                             
{'loss': '0.7129', 'grad_norm': '0.6718', 'learning_rate': '4.939e-05', 'epoch': '0.112', 'num_input_tokens_seen': 103235240, 'train_runtime': '2.151e+05', 'train_tokens_per_second': '480'}

 11%|█         | 25200/225000 [59:44:54<378:49:50,  6.83s/it]
 11%|█         | 25201/225000 [59:44:58<326:28:20,  5.88s/it]
 11%|█         | 25202/225000 [59:45:08<390:29:08,  7.04s/it]
 11%|█         | 25203/225000 [59:45:12<341:38:37,  6.16s/it]
 11%|█         | 25204/225000 [59:45:23<421:48:07,  7.60s/it]
 11%|█         | 25205/225000 [59:45:26<352:33:15,  6.35s/it]
 11%|█         | 25206/225000 [59:45:32<341:14:42,  6.15s/it]
 11%|█         | 25207/225000 [59:45:38<344:50:30,  6.21s/it]
 11%|█         | 25208/225000 [59:45:44<338:22:30,  6.10s/it]
 11%|█         | 25209/225000 [59:45:49<318:51:46,  5.75s/it]
 11%|█         | 25210/225000 [59:45:55<317:16:23,  5.72s/it]
 11%|█         | 25211/225000 [59:46:04<380:18:03,  6.85s/it]
 11%|█         | 25212/225000 [59:46:09<349:40:35,  6.30s/it]
 11%|█         | 25213/225000 [59:46:14<329:49:15,  5.94s/it]
 11%|█         | 25214/225000 [59:46:19<316:55:35,  5.71s/it]
 11%|█         | 25215/225000 [59:46:24<301:31:48,  5.43s/it]
 11%|█         | 25216/225000 [59:46:30<302:59:44,  5.46s/it]
 11%|█         | 25217/225000 [59:46:42<417:51:21,  7.53s/it]
 11%|█         | 25218/225000 [59:46:46<352:46:00,  6.36s/it]
 11%|█         | 25219/225000 [59:46:53<376:29:20,  6.78s/it]
 11%|█         | 25220/225000 [59:46:58<332:39:01,  5.99s/it]
 11%|█         | 25221/225000 [59:47:03<328:08:41,  5.91s/it]
 11%|█         | 25222/225000 [59:47:14<406:05:14,  7.32s/it]
 11%|█         | 25223/225000 [59:47:18<346:55:41,  6.25s/it]
 11%|█         | 25224/225000 [59:47:22<320:17:18,  5.77s/it]
 11%|█         | 25225/225000 [59:47:27<300:02:07,  5.41s/it]
 11%|█         | 25226/225000 [59:47:39<410:40:18,  7.40s/it]
 11%|█         | 25227/225000 [59:47:43<350:24:42,  6.31s/it]
 11%|█         | 25228/225000 [59:47:52<408:04:22,  7.35s/it]
 11%|█         | 25229/225000 [59:47:56<348:42:15,  6.28s/it]
 11%|█         | 25230/225000 [59:48:03<360:00:29,  6.49s/it]
 11%|█         | 25231/225000 [59:48:12<389:57:09,  7.03s/it]
 11%|█         | 25232/225000 [59:48:21<437:17:58,  7.88s/it]
 11%|█         | 25233/225000 [59:48:26<374:55:45,  6.76s/it]
 11%|█         | 25234/225000 [59:48:36<433:03:32,  7.80s/it]
 11%|█         | 25235/225000 [59:48:40<380:56:25,  6.86s/it]
 11%|█         | 25236/225000 [59:48:48<394:28:22,  7.11s/it]
 11%|█         | 25237/225000 [59:48:52<344:49:40,  6.21s/it]
 11%|█         | 25238/225000 [59:48:57<321:10:14,  5.79s/it]
 11%|█         | 25239/225000 [59:49:02<312:56:29,  5.64s/it]
 11%|█         | 25240/225000 [59:49:08<307:14:29,  5.54s/it]
 11%|█         | 25241/225000 [59:49:18<394:02:07,  7.10s/it]
 11%|█         | 25242/225000 [59:49:24<370:23:54,  6.68s/it]
 11%|█         | 25243/225000 [59:49:33<407:19:04,  7.34s/it]
 11%|█         | 25244/225000 [59:49:38<375:07:57,  6.76s/it]
 11%|█         | 25245/225000 [59:49:45<378:43:19,  6.83s/it]
 11%|█         | 25246/225000 [59:49:50<338:16:07,  6.10s/it]
 11%|█         | 25247/225000 [59:50:01<417:32:10,  7.52s/it]
 11%|█         | 25248/225000 [59:50:04<353:31:13,  6.37s/it]
 11%|█         | 25249/225000 [59:50:10<339:16:35,  6.11s/it]
 11%|█         | 25250/225000 [59:50:16<336:10:30,  6.06s/it]
                                                             
{'loss': '0.735', 'grad_norm': '0.5799', 'learning_rate': '4.938e-05', 'epoch': '0.1122', 'num_input_tokens_seen': 103380176, 'train_runtime': '2.154e+05', 'train_tokens_per_second': '479.9'}

 11%|█         | 25250/225000 [59:50:16<336:10:30,  6.06s/it]
 11%|█         | 25251/225000 [59:50:21<328:14:11,  5.92s/it]
 11%|█         | 25252/225000 [59:50:27<323:43:25,  5.83s/it]
 11%|█         | 25253/225000 [59:50:33<319:42:11,  5.76s/it]
 11%|█         | 25254/225000 [59:50:45<433:20:38,  7.81s/it]
 11%|█         | 25255/225000 [59:50:52<409:50:43,  7.39s/it]
 11%|█         | 25256/225000 [59:50:58<389:54:58,  7.03s/it]
 11%|█         | 25257/225000 [59:51:03<359:44:19,  6.48s/it]
 11%|█         | 25258/225000 [59:51:08<343:09:33,  6.18s/it]
 11%|█         | 25259/225000 [59:51:18<404:24:14,  7.29s/it]
 11%|█         | 25260/225000 [59:51:23<366:46:00,  6.61s/it]
 11%|█         | 25261/225000 [59:51:28<341:12:19,  6.15s/it]
 11%|█         | 25262/225000 [59:51:34<324:35:54,  5.85s/it]
 11%|█         | 25263/225000 [59:51:39<313:47:37,  5.66s/it]
 11%|█         | 25264/225000 [59:51:47<352:35:32,  6.36s/it]
 11%|█         | 25265/225000 [59:52:00<459:47:40,  8.29s/it]
 11%|█         | 25266/225000 [59:52:05<414:47:44,  7.48s/it]
 11%|█         | 25267/225000 [59:52:12<405:20:04,  7.31s/it]
 11%|█         | 25268/225000 [59:52:17<363:35:13,  6.55s/it]
 11%|█         | 25269/225000 [59:52:22<333:53:27,  6.02s/it]
 11%|█         | 25270/225000 [59:52:26<309:28:22,  5.58s/it]
 11%|█         | 25271/225000 [59:52:36<374:06:05,  6.74s/it]
 11%|█         | 25272/225000 [59:52:45<424:25:08,  7.65s/it]
 11%|█         | 25273/225000 [59:52:49<361:07:02,  6.51s/it]
 11%|█         | 25274/225000 [59:52:59<410:34:45,  7.40s/it]
 11%|█         | 25275/225000 [59:53:03<354:25:11,  6.39s/it]
 11%|█         | 25276/225000 [59:53:09<355:42:29,  6.41s/it]
 11%|█         | 25277/225000 [59:53:18<396:33:35,  7.15s/it]
 11%|█         | 25278/225000 [59:53:22<345:34:01,  6.23s/it]
 11%|█         | 25279/225000 [59:53:26<309:05:36,  5.57s/it]
 11%|█         | 25280/225000 [59:53:35<367:10:13,  6.62s/it]
 11%|█         | 25281/225000 [59:53:42<372:33:02,  6.72s/it]
 11%|█         | 25282/225000 [59:53:46<317:16:37,  5.72s/it]
 11%|█         | 25283/225000 [59:53:55<379:47:51,  6.85s/it]
 11%|█         | 25284/225000 [59:53:59<334:17:10,  6.03s/it]
 11%|█         | 25285/225000 [59:54:03<302:57:42,  5.46s/it]
 11%|█         | 25286/225000 [59:54:16<419:03:30,  7.55s/it]
 11%|█         | 25287/225000 [59:54:20<362:08:03,  6.53s/it]
 11%|█         | 25288/225000 [59:54:27<376:43:14,  6.79s/it]
 11%|█         | 25289/225000 [59:54:34<369:22:23,  6.66s/it]
 11%|█         | 25290/225000 [59:54:38<324:45:00,  5.85s/it]
 11%|█         | 25291/225000 [59:54:42<299:31:05,  5.40s/it]
 11%|█         | 25292/225000 [59:54:48<302:45:30,  5.46s/it]
 11%|█         | 25293/225000 [59:55:00<424:37:45,  7.65s/it]
 11%|█         | 25294/225000 [59:55:07<406:53:21,  7.33s/it]
 11%|█         | 25295/225000 [59:55:11<351:35:27,  6.34s/it]
 11%|█         | 25296/225000 [59:55:19<383:19:09,  6.91s/it]
 11%|█         | 25297/225000 [59:55:28<409:26:08,  7.38s/it]
 11%|█         | 25298/225000 [59:55:34<394:38:21,  7.11s/it]
 11%|█         | 25299/225000 [59:55:39<357:01:25,  6.44s/it]
 11%|█         | 25300/225000 [59:55:44<328:13:40,  5.92s/it]
                                                             
{'loss': '0.7309', 'grad_norm': '0.6749', 'learning_rate': '4.938e-05', 'epoch': '0.1124', 'num_input_tokens_seen': 103524688, 'train_runtime': '2.157e+05', 'train_tokens_per_second': '479.8'}

 11%|█         | 25300/225000 [59:55:44<328:13:40,  5.92s/it]
 11%|█         | 25301/225000 [59:55:48<307:37:44,  5.55s/it]
 11%|█         | 25302/225000 [59:55:59<385:23:11,  6.95s/it]
 11%|█         | 25303/225000 [59:56:03<351:02:29,  6.33s/it]
 11%|█         | 25304/225000 [59:56:12<394:04:09,  7.10s/it]
 11%|█         | 25305/225000 [59:56:17<349:49:03,  6.31s/it]
 11%|█         | 25306/225000 [59:56:26<402:42:53,  7.26s/it]
 11%|█         | 25307/225000 [59:56:30<347:45:48,  6.27s/it]
 11%|█         | 25308/225000 [59:56:41<421:33:01,  7.60s/it]
 11%|█         | 25309/225000 [59:56:45<362:20:07,  6.53s/it]
 11%|█         | 25310/225000 [59:56:52<363:53:41,  6.56s/it]
 11%|█         | 25311/225000 [59:56:56<330:49:55,  5.96s/it]
 11%|█         | 25312/225000 [59:57:01<303:45:15,  5.48s/it]
 11%|█▏        | 25313/225000 [59:57:14<429:53:27,  7.75s/it]
 11%|█▏        | 25314/225000 [59:57:19<386:07:06,  6.96s/it]
 11%|█▏        | 25315/225000 [59:57:23<344:45:25,  6.22s/it]
 11%|█▏        | 25316/225000 [59:57:28<321:52:35,  5.80s/it]
 11%|█▏        | 25317/225000 [59:57:34<318:32:47,  5.74s/it]
 11%|█▏        | 25318/225000 [59:57:39<310:14:33,  5.59s/it]
 11%|█▏        | 25319/225000 [59:57:44<307:51:01,  5.55s/it]
 11%|█▏        | 25320/225000 [59:57:55<389:09:16,  7.02s/it]
 11%|█▏        | 25321/225000 [59:58:03<402:49:45,  7.26s/it]
 11%|█▏        | 25322/225000 [59:58:09<392:10:08,  7.07s/it]
 11%|█▏        | 25323/225000 [59:58:15<378:31:39,  6.82s/it]
 11%|█▏        | 25324/225000 [59:58:21<352:13:20,  6.35s/it]
 11%|█▏        | 25325/225000 [59:58:25<322:10:28,  5.81s/it]
 11%|█▏        | 25326/225000 [59:58:33<347:34:19,  6.27s/it]
 11%|█▏        | 25327/225000 [59:58:46<466:37:54,  8.41s/it]
 11%|█▏        | 25328/225000 [59:58:52<422:55:21,  7.63s/it]
 11%|█▏        | 25329/225000 [59:58:57<377:29:06,  6.81s/it]
 11%|█▏        | 25330/225000 [59:59:01<340:00:01,  6.13s/it]
 11%|█▏        | 25331/225000 [59:59:06<313:29:36,  5.65s/it]
 11%|█▏        | 25332/225000 [59:59:11<304:07:08,  5.48s/it]
 11%|█▏        | 25333/225000 [59:59:24<424:13:27,  7.65s/it]
 11%|█▏        | 25334/225000 [59:59:28<377:49:37,  6.81s/it]
 11%|█▏        | 25335/225000 [59:59:35<378:59:40,  6.83s/it]
 11%|█▏        | 25336/225000 [59:59:40<339:05:24,  6.11s/it]
 11%|█▏        | 25337/225000 [59:59:44<313:48:15,  5.66s/it]
 11%|█▏        | 25338/225000 [59:59:49<298:44:24,  5.39s/it]
 11%|█▏        | 25339/225000 [59:59:56<319:04:01,  5.75s/it]
 11%|█▏        | 25340/225000 [60:00:09<445:08:53,  8.03s/it]
 11%|█▏        | 25341/225000 [60:00:13<376:42:27,  6.79s/it]
 11%|█▏        | 25342/225000 [60:00:17<327:38:07,  5.91s/it]
 11%|█▏        | 25343/225000 [60:00:28<410:48:20,  7.41s/it]
 11%|█▏        | 25344/225000 [60:00:31<347:34:02,  6.27s/it]
 11%|█▏        | 25345/225000 [60:00:38<353:14:23,  6.37s/it]
 11%|█▏        | 25346/225000 [60:00:43<335:01:21,  6.04s/it]
 11%|█▏        | 25347/225000 [60:00:52<380:05:47,  6.85s/it]
 11%|█▏        | 25348/225000 [60:00:56<339:46:37,  6.13s/it]
 11%|█▏        | 25349/225000 [60:01:01<320:15:06,  5.77s/it]
 11%|█▏        | 25350/225000 [60:01:06<307:12:56,  5.54s/it]
                                                             
{'loss': '0.7292', 'grad_norm': '0.7808', 'learning_rate': '4.937e-05', 'epoch': '0.1127', 'num_input_tokens_seen': 103667160, 'train_runtime': '2.161e+05', 'train_tokens_per_second': '479.8'}

 11%|█▏        | 25350/225000 [60:01:06<307:12:56,  5.54s/it]
 11%|█▏        | 25351/225000 [60:01:11<295:36:08,  5.33s/it]
 11%|█▏        | 25352/225000 [60:01:20<348:50:11,  6.29s/it]
 11%|█▏        | 25353/225000 [60:01:31<438:58:11,  7.92s/it]
 11%|█▏        | 25354/225000 [60:01:36<387:24:49,  6.99s/it]
 11%|█▏        | 25355/225000 [60:01:44<400:49:46,  7.23s/it]
 11%|█▏        | 25356/225000 [60:01:49<368:36:15,  6.65s/it]
 11%|█▏        | 25357/225000 [60:01:55<350:06:53,  6.31s/it]
 11%|█▏        | 25358/225000 [60:02:04<396:52:42,  7.16s/it]
 11%|█▏        | 25359/225000 [60:02:08<346:35:01,  6.25s/it]
 11%|█▏        | 25360/225000 [60:02:14<345:21:19,  6.23s/it]
 11%|█▏        | 25361/225000 [60:02:22<362:03:19,  6.53s/it]
 11%|█▏        | 25362/225000 [60:02:27<345:21:48,  6.23s/it]
 11%|█▏        | 25363/225000 [60:02:36<382:49:18,  6.90s/it]
 11%|█▏        | 25364/225000 [60:02:42<373:45:10,  6.74s/it]
 11%|█▏        | 25365/225000 [60:02:48<371:31:13,  6.70s/it]
 11%|█▏        | 25366/225000 [60:02:52<324:35:56,  5.85s/it]
 11%|█▏        | 25367/225000 [60:02:58<322:16:25,  5.81s/it]
 11%|█▏        | 25368/225000 [60:03:08<387:04:45,  6.98s/it]
 11%|█▏        | 25369/225000 [60:03:11<328:28:06,  5.92s/it]
 11%|█▏        | 25370/225000 [60:03:20<381:21:57,  6.88s/it]
 11%|█▏        | 25371/225000 [60:03:29<407:45:20,  7.35s/it]
 11%|█▏        | 25372/225000 [60:03:33<358:51:02,  6.47s/it]
 11%|█▏        | 25373/225000 [60:03:37<313:26:32,  5.65s/it]
 11%|█▏        | 25374/225000 [60:03:48<406:56:57,  7.34s/it]
 11%|█▏        | 25375/225000 [60:03:53<365:20:26,  6.59s/it]
 11%|█▏        | 25376/225000 [60:04:02<407:52:05,  7.36s/it]
 11%|█▏        | 25377/225000 [60:04:06<346:57:47,  6.26s/it]
 11%|█▏        | 25378/225000 [60:04:12<351:19:18,  6.34s/it]
 11%|█▏        | 25379/225000 [60:04:21<384:14:55,  6.93s/it]
 11%|█▏        | 25380/225000 [60:04:25<340:38:53,  6.14s/it]
 11%|█▏        | 25381/225000 [60:04:29<302:43:35,  5.46s/it]
 11%|█▏        | 25382/225000 [60:04:33<283:14:10,  5.11s/it]
 11%|█▏        | 25383/225000 [60:04:45<391:06:01,  7.05s/it]
 11%|█▏        | 25384/225000 [60:04:51<370:32:58,  6.68s/it]
 11%|█▏        | 25385/225000 [60:04:56<349:32:00,  6.30s/it]
 11%|█▏        | 25386/225000 [60:05:01<331:58:51,  5.99s/it]
 11%|█▏        | 25387/225000 [60:05:07<325:41:52,  5.87s/it]
 11%|█▏        | 25388/225000 [60:05:13<323:25:12,  5.83s/it]
 11%|█▏        | 25389/225000 [60:05:19<327:19:39,  5.90s/it]
 11%|█▏        | 25390/225000 [60:05:26<345:27:09,  6.23s/it]
 11%|█▏        | 25391/225000 [60:05:38<445:44:35,  8.04s/it]
 11%|█▏        | 25392/225000 [60:05:42<382:45:46,  6.90s/it]
 11%|█▏        | 25393/225000 [60:05:52<431:58:02,  7.79s/it]
 11%|█▏        | 25394/225000 [60:05:56<375:12:21,  6.77s/it]
 11%|█▏        | 25395/225000 [60:06:04<392:30:12,  7.08s/it]
 11%|█▏        | 25396/225000 [60:06:11<393:49:45,  7.10s/it]
 11%|█▏        | 25397/225000 [60:06:16<357:26:44,  6.45s/it]
 11%|█▏        | 25398/225000 [60:06:22<338:06:33,  6.10s/it]
 11%|█▏        | 25399/225000 [60:06:27<322:55:26,  5.82s/it]
 11%|█▏        | 25400/225000 [60:06:33<321:55:16,  5.81s/it]
                                                             
{'loss': '0.7255', 'grad_norm': '0.6543', 'learning_rate': '4.937e-05', 'epoch': '0.1129', 'num_input_tokens_seen': 103811568, 'train_runtime': '2.164e+05', 'train_tokens_per_second': '479.7'}

 11%|█▏        | 25400/225000 [60:06:33<321:55:16,  5.81s/it]
 11%|█▏        | 25401/225000 [60:06:39<336:52:51,  6.08s/it]
 11%|█▏        | 25402/225000 [60:06:50<412:56:39,  7.45s/it]
 11%|█▏        | 25403/225000 [60:06:56<388:25:43,  7.01s/it]
 11%|█▏        | 25404/225000 [60:07:00<341:14:06,  6.15s/it]
 11%|█▏        | 25405/225000 [60:07:09<394:55:13,  7.12s/it]
 11%|█▏        | 25406/225000 [60:07:13<341:50:43,  6.17s/it]
 11%|█▏        | 25407/225000 [60:07:22<377:07:41,  6.80s/it]
 11%|█▏        | 25408/225000 [60:07:26<328:44:05,  5.93s/it]
 11%|█▏        | 25409/225000 [60:07:30<300:30:12,  5.42s/it]
 11%|█▏        | 25410/225000 [60:07:35<299:01:58,  5.39s/it]
 11%|█▏        | 25411/225000 [60:07:40<291:53:45,  5.26s/it]
 11%|█▏        | 25412/225000 [60:07:45<292:32:44,  5.28s/it]
 11%|█▏        | 25413/225000 [60:07:57<389:38:46,  7.03s/it]
 11%|█▏        | 25414/225000 [60:08:06<432:27:32,  7.80s/it]
 11%|█▏        | 25415/225000 [60:08:10<363:38:50,  6.56s/it]
 11%|█▏        | 25416/225000 [60:08:19<401:34:27,  7.24s/it]
 11%|█▏        | 25417/225000 [60:08:26<401:08:30,  7.24s/it]
 11%|█▏        | 25418/225000 [60:08:30<343:39:41,  6.20s/it]
 11%|█▏        | 25419/225000 [60:08:40<407:54:06,  7.36s/it]
 11%|█▏        | 25420/225000 [60:08:47<400:12:39,  7.22s/it]
 11%|█▏        | 25421/225000 [60:08:51<351:48:09,  6.35s/it]
 11%|█▏        | 25422/225000 [60:08:55<315:43:17,  5.70s/it]
 11%|█▏        | 25423/225000 [60:09:05<390:33:50,  7.05s/it]
 11%|█▏        | 25424/225000 [60:09:11<374:08:12,  6.75s/it]
 11%|█▏        | 25425/225000 [60:09:15<324:38:49,  5.86s/it]
 11%|█▏        | 25426/225000 [60:09:24<369:58:23,  6.67s/it]
 11%|█▏        | 25427/225000 [60:09:28<338:08:31,  6.10s/it]
 11%|█▏        | 25428/225000 [60:09:34<328:11:00,  5.92s/it]
 11%|█▏        | 25429/225000 [60:09:39<313:15:13,  5.65s/it]
 11%|█▏        | 25430/225000 [60:09:44<301:14:44,  5.43s/it]
 11%|█▏        | 25431/225000 [60:09:50<309:04:17,  5.58s/it]
 11%|█▏        | 25432/225000 [60:09:54<289:32:48,  5.22s/it]
 11%|█▏        | 25433/225000 [60:10:00<291:56:45,  5.27s/it]
 11%|█▏        | 25434/225000 [60:10:09<357:07:00,  6.44s/it]
 11%|█▏        | 25435/225000 [60:10:20<437:12:04,  7.89s/it]
 11%|█▏        | 25436/225000 [60:10:25<383:12:39,  6.91s/it]
 11%|█▏        | 25437/225000 [60:10:33<415:12:38,  7.49s/it]
 11%|█▏        | 25438/225000 [60:10:38<358:03:21,  6.46s/it]
 11%|█▏        | 25439/225000 [60:10:48<421:49:46,  7.61s/it]
 11%|█▏        | 25440/225000 [60:10:53<387:20:28,  6.99s/it]
 11%|█▏        | 25441/225000 [60:10:58<351:05:51,  6.33s/it]
 11%|█▏        | 25442/225000 [60:11:04<335:43:50,  6.06s/it]
 11%|█▏        | 25443/225000 [60:11:13<395:03:28,  7.13s/it]
 11%|█▏        | 25444/225000 [60:11:19<377:43:02,  6.81s/it]
 11%|█▏        | 25445/225000 [60:11:29<418:18:33,  7.55s/it]
 11%|█▏        | 25446/225000 [60:11:33<370:57:08,  6.69s/it]
 11%|█▏        | 25447/225000 [60:11:38<340:06:20,  6.14s/it]
 11%|█▏        | 25448/225000 [60:11:43<313:50:11,  5.66s/it]
 11%|█▏        | 25449/225000 [60:11:50<341:48:54,  6.17s/it]
 11%|█▏        | 25450/225000 [60:12:01<424:51:47,  7.66s/it]
                                                             
{'loss': '0.7176', 'grad_norm': '0.7245', 'learning_rate': '4.937e-05', 'epoch': '0.1131', 'num_input_tokens_seen': 103955288, 'train_runtime': '2.167e+05', 'train_tokens_per_second': '479.7'}

 11%|█▏        | 25450/225000 [60:12:01<424:51:47,  7.66s/it]
 11%|█▏        | 25451/225000 [60:12:04<353:34:05,  6.38s/it]
 11%|█▏        | 25452/225000 [60:12:09<321:53:27,  5.81s/it]
 11%|█▏        | 25453/225000 [60:12:18<377:10:59,  6.80s/it]
 11%|█▏        | 25454/225000 [60:12:24<363:16:40,  6.55s/it]
 11%|█▏        | 25455/225000 [60:12:32<392:07:52,  7.07s/it]
 11%|█▏        | 25456/225000 [60:12:37<356:48:21,  6.44s/it]
 11%|█▏        | 25457/225000 [60:12:42<334:10:16,  6.03s/it]
 11%|█▏        | 25458/225000 [60:12:51<368:47:52,  6.65s/it]
 11%|█▏        | 25459/225000 [60:12:58<391:10:30,  7.06s/it]
 11%|█▏        | 25460/225000 [60:13:02<339:12:28,  6.12s/it]
 11%|█▏        | 25461/225000 [60:13:14<430:30:11,  7.77s/it]
 11%|█▏        | 25462/225000 [60:13:18<367:54:24,  6.64s/it]
 11%|█▏        | 25463/225000 [60:13:27<413:22:08,  7.46s/it]
 11%|█▏        | 25464/225000 [60:13:32<364:31:59,  6.58s/it]
 11%|█▏        | 25465/225000 [60:13:36<330:36:56,  5.96s/it]
 11%|█▏        | 25466/225000 [60:13:42<321:21:44,  5.80s/it]
 11%|█▏        | 25467/225000 [60:13:47<302:19:36,  5.45s/it]
 11%|█▏        | 25468/225000 [60:13:59<418:31:23,  7.55s/it]
 11%|█▏        | 25469/225000 [60:14:02<345:14:48,  6.23s/it]
 11%|█▏        | 25470/225000 [60:14:09<353:37:09,  6.38s/it]
 11%|█▏        | 25471/225000 [60:14:17<378:26:17,  6.83s/it]
 11%|█▏        | 25472/225000 [60:14:23<371:05:06,  6.70s/it]
 11%|█▏        | 25473/225000 [60:14:27<319:38:07,  5.77s/it]
 11%|█▏        | 25474/225000 [60:14:35<364:33:52,  6.58s/it]
 11%|█▏        | 25475/225000 [60:14:40<332:34:15,  6.00s/it]
 11%|█▏        | 25476/225000 [60:14:48<365:32:02,  6.60s/it]
 11%|█▏        | 25477/225000 [60:14:54<363:12:11,  6.55s/it]
 11%|█▏        | 25478/225000 [60:15:02<375:28:47,  6.77s/it]
 11%|█▏        | 25479/225000 [60:15:05<323:58:42,  5.85s/it]
 11%|█▏        | 25480/225000 [60:15:17<422:03:04,  7.62s/it]
 11%|█▏        | 25481/225000 [60:15:24<404:02:25,  7.29s/it]
 11%|█▏        | 25482/225000 [60:15:29<371:07:04,  6.70s/it]
 11%|█▏        | 25483/225000 [60:15:33<334:02:28,  6.03s/it]
 11%|█▏        | 25484/225000 [60:15:38<308:28:54,  5.57s/it]
 11%|█▏        | 25485/225000 [60:15:42<292:29:24,  5.28s/it]
 11%|█▏        | 25486/225000 [60:15:47<283:14:36,  5.11s/it]
 11%|█▏        | 25487/225000 [60:15:55<327:25:24,  5.91s/it]
 11%|█▏        | 25488/225000 [60:16:06<412:29:46,  7.44s/it]
 11%|█▏        | 25489/225000 [60:16:10<356:00:55,  6.42s/it]
 11%|█▏        | 25490/225000 [60:16:14<324:20:22,  5.85s/it]
 11%|█▏        | 25491/225000 [60:16:23<367:48:04,  6.64s/it]
 11%|█▏        | 25492/225000 [60:16:30<374:48:58,  6.76s/it]
 11%|█▏        | 25493/225000 [60:16:38<397:48:28,  7.18s/it]
 11%|█▏        | 25494/225000 [60:16:43<357:28:41,  6.45s/it]
 11%|█▏        | 25495/225000 [60:16:47<310:32:59,  5.60s/it]
 11%|█▏        | 25496/225000 [60:16:56<366:53:50,  6.62s/it]
 11%|█▏        | 25497/225000 [60:17:01<350:48:58,  6.33s/it]
 11%|█▏        | 25498/225000 [60:17:09<369:42:35,  6.67s/it]
 11%|█▏        | 25499/225000 [60:17:16<377:31:11,  6.81s/it]
 11%|█▏        | 25500/225000 [60:17:21<354:37:34,  6.40s/it]
                                                             
{'loss': '0.7266', 'grad_norm': '0.6576', 'learning_rate': '4.936e-05', 'epoch': '0.1133', 'num_input_tokens_seen': 104097712, 'train_runtime': '2.17e+05', 'train_tokens_per_second': '479.6'}

 11%|█▏        | 25500/225000 [60:17:21<354:37:34,  6.40s/it]
 11%|█▏        | 25501/225000 [60:17:26<327:01:30,  5.90s/it]
 11%|█▏        | 25502/225000 [60:17:31<309:40:26,  5.59s/it]
 11%|█▏        | 25503/225000 [60:17:36<305:46:02,  5.52s/it]
 11%|█▏        | 25504/225000 [60:17:41<292:08:22,  5.27s/it]
 11%|█▏        | 25505/225000 [60:17:47<304:34:38,  5.50s/it]
 11%|█▏        | 25506/225000 [60:17:52<299:35:39,  5.41s/it]
 11%|█▏        | 25507/225000 [60:18:04<409:00:21,  7.38s/it]
 11%|█▏        | 25508/225000 [60:18:11<402:51:40,  7.27s/it]
 11%|█▏        | 25509/225000 [60:18:20<433:31:19,  7.82s/it]
 11%|█▏        | 25510/225000 [60:18:24<372:50:11,  6.73s/it]
 11%|█▏        | 25511/225000 [60:18:31<374:28:36,  6.76s/it]
 11%|█▏        | 25512/225000 [60:18:37<355:08:51,  6.41s/it]
 11%|█▏        | 25513/225000 [60:18:42<341:57:10,  6.17s/it]
 11%|█▏        | 25514/225000 [60:18:52<402:55:26,  7.27s/it]
 11%|█▏        | 25515/225000 [60:18:56<342:00:26,  6.17s/it]
 11%|█▏        | 25516/225000 [60:19:01<318:17:23,  5.74s/it]
 11%|█▏        | 25517/225000 [60:19:05<299:33:12,  5.41s/it]
 11%|█▏        | 25518/225000 [60:19:14<356:26:57,  6.43s/it]
 11%|█▏        | 25519/225000 [60:19:24<412:47:32,  7.45s/it]
 11%|█▏        | 25520/225000 [60:19:30<396:40:08,  7.16s/it]
 11%|█▏        | 25521/225000 [60:19:35<352:11:49,  6.36s/it]
 11%|█▏        | 25522/225000 [60:19:47<445:27:51,  8.04s/it]
 11%|█▏        | 25523/225000 [60:19:51<378:50:07,  6.84s/it]
 11%|█▏        | 25524/225000 [60:19:55<332:54:48,  6.01s/it]
 11%|█▏        | 25525/225000 [60:20:03<361:38:34,  6.53s/it]
 11%|█▏        | 25526/225000 [60:20:12<413:46:01,  7.47s/it]
 11%|█▏        | 25527/225000 [60:20:16<358:39:59,  6.47s/it]
 11%|█▏        | 25528/225000 [60:20:25<395:39:08,  7.14s/it]
 11%|█▏        | 25529/225000 [60:20:30<359:47:13,  6.49s/it]
 11%|█▏        | 25530/225000 [60:20:35<340:00:53,  6.14s/it]
 11%|█▏        | 25531/225000 [60:20:41<323:19:15,  5.84s/it]
 11%|█▏        | 25532/225000 [60:20:46<310:54:08,  5.61s/it]
 11%|█▏        | 25533/225000 [60:20:50<296:49:25,  5.36s/it]
 11%|█▏        | 25534/225000 [60:21:04<425:36:18,  7.68s/it]
 11%|█▏        | 25535/225000 [60:21:09<392:26:36,  7.08s/it]
 11%|█▏        | 25536/225000 [60:21:15<363:02:06,  6.55s/it]
 11%|█▏        | 25537/225000 [60:21:19<333:52:33,  6.03s/it]
 11%|█▏        | 25538/225000 [60:21:24<317:32:50,  5.73s/it]
 11%|█▏        | 25539/225000 [60:21:35<396:15:00,  7.15s/it]
 11%|█▏        | 25540/225000 [60:21:42<389:27:52,  7.03s/it]
 11%|█▏        | 25541/225000 [60:21:46<352:45:39,  6.37s/it]
 11%|█▏        | 25542/225000 [60:21:55<395:51:21,  7.14s/it]
 11%|█▏        | 25543/225000 [60:22:04<415:37:33,  7.50s/it]
 11%|█▏        | 25544/225000 [60:22:10<403:05:20,  7.28s/it]
 11%|█▏        | 25545/225000 [60:22:15<364:03:15,  6.57s/it]
 11%|█▏        | 25546/225000 [60:22:20<324:17:39,  5.85s/it]
 11%|█▏        | 25547/225000 [60:22:24<304:16:10,  5.49s/it]
 11%|█▏        | 25548/225000 [60:22:29<295:56:00,  5.34s/it]
 11%|█▏        | 25549/225000 [60:22:42<413:45:57,  7.47s/it]
 11%|█▏        | 25550/225000 [60:22:48<390:43:14,  7.05s/it]
                                                             
{'loss': '0.72', 'grad_norm': '0.6602', 'learning_rate': '4.936e-05', 'epoch': '0.1136', 'num_input_tokens_seen': 104243040, 'train_runtime': '2.174e+05', 'train_tokens_per_second': '479.6'}

 11%|█▏        | 25550/225000 [60:22:48<390:43:14,  7.05s/it]
 11%|█▏        | 25551/225000 [60:22:52<341:58:49,  6.17s/it]
 11%|█▏        | 25552/225000 [60:23:02<415:06:36,  7.49s/it]
 11%|█▏        | 25553/225000 [60:23:08<377:16:46,  6.81s/it]
 11%|█▏        | 25554/225000 [60:23:13<349:28:52,  6.31s/it]
 11%|█▏        | 25555/225000 [60:23:21<387:35:27,  7.00s/it]
 11%|█▏        | 25556/225000 [60:23:27<366:10:32,  6.61s/it]
 11%|█▏        | 25557/225000 [60:23:32<331:55:46,  5.99s/it]
 11%|█▏        | 25558/225000 [60:23:36<308:23:37,  5.57s/it]
 11%|█▏        | 25559/225000 [60:23:48<420:12:06,  7.58s/it]
 11%|█▏        | 25560/225000 [60:23:54<380:45:29,  6.87s/it]
 11%|█▏        | 25561/225000 [60:23:58<331:57:22,  5.99s/it]
 11%|█▏        | 25562/225000 [60:24:05<355:11:58,  6.41s/it]
 11%|█▏        | 25563/225000 [60:24:11<350:10:25,  6.32s/it]
 11%|█▏        | 25564/225000 [60:24:20<390:15:56,  7.04s/it]
 11%|█▏        | 25565/225000 [60:24:25<360:58:23,  6.52s/it]
 11%|█▏        | 25566/225000 [60:24:30<340:30:22,  6.15s/it]
 11%|█▏        | 25567/225000 [60:24:36<324:00:39,  5.85s/it]
 11%|█▏        | 25568/225000 [60:24:44<363:16:41,  6.56s/it]
 11%|█▏        | 25569/225000 [60:24:53<404:06:53,  7.29s/it]
 11%|█▏        | 25570/225000 [60:24:57<350:23:01,  6.32s/it]
 11%|█▏        | 25571/225000 [60:25:02<332:18:25,  6.00s/it]
 11%|█▏        | 25572/225000 [60:25:07<318:01:12,  5.74s/it]
 11%|█▏        | 25573/225000 [60:25:12<307:32:42,  5.55s/it]
 11%|█▏        | 25574/225000 [60:25:17<299:56:47,  5.41s/it]
 11%|█▏        | 25575/225000 [60:25:30<414:03:07,  7.47s/it]
 11%|█▏        | 25576/225000 [60:25:35<370:58:37,  6.70s/it]
 11%|█▏        | 25577/225000 [60:25:39<330:47:13,  5.97s/it]
 11%|█▏        | 25578/225000 [60:25:43<305:05:22,  5.51s/it]
 11%|█▏        | 25579/225000 [60:25:51<348:37:18,  6.29s/it]
 11%|█▏        | 25580/225000 [60:26:05<467:26:34,  8.44s/it]
 11%|█▏        | 25581/225000 [60:26:10<415:32:19,  7.50s/it]
 11%|█▏        | 25582/225000 [60:26:19<428:56:55,  7.74s/it]
 11%|█▏        | 25583/225000 [60:26:22<363:20:08,  6.56s/it]
 11%|█▏        | 25584/225000 [60:26:32<407:55:14,  7.36s/it]
 11%|█▏        | 25585/225000 [60:26:36<354:04:57,  6.39s/it]
 11%|█▏        | 25586/225000 [60:26:40<324:29:31,  5.86s/it]
 11%|█▏        | 25587/225000 [60:26:47<340:54:19,  6.15s/it]
 11%|█▏        | 25588/225000 [60:26:57<400:02:20,  7.22s/it]
 11%|█▏        | 25589/225000 [60:27:02<370:52:22,  6.70s/it]
 11%|█▏        | 25590/225000 [60:27:08<354:16:28,  6.40s/it]
 11%|█▏        | 25591/225000 [60:27:16<382:14:28,  6.90s/it]
 11%|█▏        | 25592/225000 [60:27:22<362:07:10,  6.54s/it]
 11%|█▏        | 25593/225000 [60:27:27<335:24:57,  6.06s/it]
 11%|█▏        | 25594/225000 [60:27:32<328:33:55,  5.93s/it]
 11%|█▏        | 25595/225000 [60:27:38<323:12:29,  5.84s/it]
 11%|█▏        | 25596/225000 [60:27:44<329:14:19,  5.94s/it]
 11%|█▏        | 25597/225000 [60:27:56<431:35:07,  7.79s/it]
 11%|█▏        | 25598/225000 [60:28:01<377:40:59,  6.82s/it]
 11%|█▏        | 25599/225000 [60:28:08<378:04:46,  6.83s/it]
 11%|█▏        | 25600/225000 [60:28:19<455:23:47,  8.22s/it]
                                                             
{'loss': '0.6977', 'grad_norm': '0.6571', 'learning_rate': '4.935e-05', 'epoch': '0.1138', 'num_input_tokens_seen': 104388000, 'train_runtime': '2.177e+05', 'train_tokens_per_second': '479.5'}

 11%|█▏        | 25600/225000 [60:28:19<455:23:47,  8.22s/it]
 11%|█▏        | 25601/225000 [60:28:24<393:11:10,  7.10s/it]
 11%|█▏        | 25602/225000 [60:28:28<341:53:37,  6.17s/it]
 11%|█▏        | 25603/225000 [60:28:38<414:22:13,  7.48s/it]
 11%|█▏        | 25604/225000 [60:28:43<363:38:56,  6.57s/it]
 11%|█▏        | 25605/225000 [60:28:47<326:08:11,  5.89s/it]
 11%|█▏        | 25606/225000 [60:28:51<300:17:21,  5.42s/it]
 11%|█▏        | 25607/225000 [60:29:03<401:15:56,  7.24s/it]
 11%|█▏        | 25608/225000 [60:29:07<351:12:07,  6.34s/it]
 11%|█▏        | 25609/225000 [60:29:11<320:38:16,  5.79s/it]
 11%|█▏        | 25610/225000 [60:29:16<304:49:01,  5.50s/it]
 11%|█▏        | 25611/225000 [60:29:21<290:19:18,  5.24s/it]
 11%|█▏        | 25612/225000 [60:29:35<431:26:21,  7.79s/it]
 11%|█▏        | 25613/225000 [60:29:41<402:12:56,  7.26s/it]
 11%|█▏        | 25614/225000 [60:29:49<412:26:08,  7.45s/it]
 11%|█▏        | 25615/225000 [60:29:53<358:28:11,  6.47s/it]
 11%|█▏        | 25616/225000 [60:29:59<351:06:03,  6.34s/it]
 11%|█▏        | 25617/225000 [60:30:07<377:43:22,  6.82s/it]
 11%|█▏        | 25618/225000 [60:30:10<324:17:45,  5.86s/it]
 11%|█▏        | 25619/225000 [60:30:19<369:57:38,  6.68s/it]
 11%|█▏        | 25620/225000 [60:30:23<321:12:21,  5.80s/it]
 11%|█▏        | 25621/225000 [60:30:27<303:49:31,  5.49s/it]
 11%|█▏        | 25622/225000 [60:30:39<404:34:17,  7.31s/it]
 11%|█▏        | 25623/225000 [60:30:43<347:06:12,  6.27s/it]
 11%|█▏        | 25624/225000 [60:30:52<387:36:11,  7.00s/it]
 11%|█▏        | 25625/225000 [60:30:56<350:10:40,  6.32s/it]
 11%|█▏        | 25626/225000 [60:31:02<338:36:56,  6.11s/it]
 11%|█▏        | 25627/225000 [60:31:07<320:35:24,  5.79s/it]
 11%|█▏        | 25628/225000 [60:31:13<318:09:50,  5.74s/it]
 11%|█▏        | 25629/225000 [60:31:18<306:28:31,  5.53s/it]
 11%|█▏        | 25630/225000 [60:31:30<420:54:49,  7.60s/it]
 11%|█▏        | 25631/225000 [60:31:34<363:47:57,  6.57s/it]
 11%|█▏        | 25632/225000 [60:31:39<331:57:51,  5.99s/it]
 11%|█▏        | 25633/225000 [60:31:43<306:31:46,  5.54s/it]
 11%|█▏        | 25634/225000 [60:31:56<417:30:05,  7.54s/it]
 11%|█▏        | 25635/225000 [60:32:00<372:07:39,  6.72s/it]
 11%|█▏        | 25636/225000 [60:32:07<377:15:44,  6.81s/it]
 11%|█▏        | 25637/225000 [60:32:14<370:09:10,  6.68s/it]
 11%|█▏        | 25638/225000 [60:32:18<333:07:58,  6.02s/it]
 11%|█▏        | 25639/225000 [60:32:23<305:48:23,  5.52s/it]
 11%|█▏        | 25640/225000 [60:32:28<302:45:03,  5.47s/it]
 11%|█▏        | 25641/225000 [60:32:37<354:39:18,  6.40s/it]
 11%|█▏        | 25642/225000 [60:32:44<375:36:48,  6.78s/it]
 11%|█▏        | 25643/225000 [60:32:51<379:07:49,  6.85s/it]
 11%|█▏        | 25644/225000 [60:32:56<345:17:50,  6.24s/it]
 11%|█▏        | 25645/225000 [60:33:01<324:00:59,  5.85s/it]
 11%|█▏        | 25646/225000 [60:33:10<374:42:08,  6.77s/it]
 11%|█▏        | 25647/225000 [60:33:13<320:43:15,  5.79s/it]
 11%|█▏        | 25648/225000 [60:33:24<402:24:42,  7.27s/it]
 11%|█▏        | 25649/225000 [60:33:28<347:34:31,  6.28s/it]
 11%|█▏        | 25650/225000 [60:33:33<327:13:57,  5.91s/it]
                                                             
{'loss': '0.7366', 'grad_norm': '0.7904', 'learning_rate': '4.935e-05', 'epoch': '0.114', 'num_input_tokens_seen': 104530320, 'train_runtime': '2.18e+05', 'train_tokens_per_second': '479.5'}

 11%|█▏        | 25650/225000 [60:33:33<327:13:57,  5.91s/it]
 11%|█▏        | 25651/225000 [60:33:41<366:00:50,  6.61s/it]
 11%|█▏        | 25652/225000 [60:33:48<369:36:54,  6.67s/it]
 11%|█▏        | 25653/225000 [60:33:54<353:13:24,  6.38s/it]
 11%|█▏        | 25654/225000 [60:34:00<351:03:46,  6.34s/it]
 11%|█▏        | 25655/225000 [60:34:05<323:23:07,  5.84s/it]
 11%|█▏        | 25656/225000 [60:34:10<317:49:04,  5.74s/it]
 11%|█▏        | 25657/225000 [60:34:15<307:53:52,  5.56s/it]
 11%|█▏        | 25658/225000 [60:34:20<293:48:44,  5.31s/it]
 11%|█▏        | 25659/225000 [60:34:27<317:15:57,  5.73s/it]
 11%|█▏        | 25660/225000 [60:34:40<447:44:07,  8.09s/it]
 11%|█▏        | 25661/225000 [60:34:45<387:24:12,  7.00s/it]
 11%|█▏        | 25662/225000 [60:34:52<387:12:46,  6.99s/it]
 11%|█▏        | 25663/225000 [60:34:58<367:45:32,  6.64s/it]
 11%|█▏        | 25664/225000 [60:35:05<382:27:56,  6.91s/it]
 11%|█▏        | 25665/225000 [60:35:09<330:50:55,  5.98s/it]
 11%|█▏        | 25666/225000 [60:35:19<403:21:58,  7.28s/it]
 11%|█▏        | 25667/225000 [60:35:23<342:30:48,  6.19s/it]
 11%|█▏        | 25668/225000 [60:35:31<368:15:26,  6.65s/it]
 11%|█▏        | 25669/225000 [60:35:36<343:38:57,  6.21s/it]
 11%|█▏        | 25670/225000 [60:35:43<361:12:58,  6.52s/it]
 11%|█▏        | 25671/225000 [60:35:47<321:18:23,  5.80s/it]
 11%|█▏        | 25672/225000 [60:35:51<294:23:59,  5.32s/it]
 11%|█▏        | 25673/225000 [60:36:04<406:31:57,  7.34s/it]
 11%|█▏        | 25674/225000 [60:36:07<345:33:58,  6.24s/it]
 11%|█▏        | 25675/225000 [60:36:14<352:33:34,  6.37s/it]
 11%|█▏        | 25676/225000 [60:36:21<365:15:04,  6.60s/it]
 11%|█▏        | 25677/225000 [60:36:25<324:38:43,  5.86s/it]
 11%|█▏        | 25678/225000 [60:36:31<316:12:13,  5.71s/it]
 11%|█▏        | 25679/225000 [60:36:36<307:02:29,  5.55s/it]
 11%|█▏        | 25680/225000 [60:36:42<311:54:51,  5.63s/it]
 11%|█▏        | 25681/225000 [60:36:46<300:54:38,  5.43s/it]
 11%|█▏        | 25682/225000 [60:36:52<297:29:42,  5.37s/it]
 11%|█▏        | 25683/225000 [60:36:57<297:14:15,  5.37s/it]
 11%|█▏        | 25684/225000 [60:37:02<295:37:00,  5.34s/it]
 11%|█▏        | 25685/225000 [60:37:10<338:09:23,  6.11s/it]
 11%|█▏        | 25686/225000 [60:37:23<440:33:24,  7.96s/it]
 11%|█▏        | 25687/225000 [60:37:26<366:10:55,  6.61s/it]
 11%|█▏        | 25688/225000 [60:37:35<402:30:55,  7.27s/it]
 11%|█▏        | 25689/225000 [60:37:39<359:30:58,  6.49s/it]
 11%|█▏        | 25690/225000 [60:37:44<325:49:46,  5.89s/it]
 11%|█▏        | 25691/225000 [60:37:49<306:38:02,  5.54s/it]
 11%|█▏        | 25692/225000 [60:37:54<296:04:00,  5.35s/it]
 11%|█▏        | 25693/225000 [60:37:58<283:15:04,  5.12s/it]
 11%|█▏        | 25694/225000 [60:38:11<410:55:01,  7.42s/it]
 11%|█▏        | 25695/225000 [60:38:15<361:56:52,  6.54s/it]
 11%|█▏        | 25696/225000 [60:38:25<412:13:58,  7.45s/it]
 11%|█▏        | 25697/225000 [60:38:32<406:38:02,  7.35s/it]
 11%|█▏        | 25698/225000 [60:38:36<343:57:44,  6.21s/it]
 11%|█▏        | 25699/225000 [60:38:43<369:04:34,  6.67s/it]
 11%|█▏        | 25700/225000 [60:38:48<330:43:15,  5.97s/it]
                                                             
{'loss': '0.7214', 'grad_norm': '0.7789', 'learning_rate': '4.935e-05', 'epoch': '0.1142', 'num_input_tokens_seen': 104672240, 'train_runtime': '2.183e+05', 'train_tokens_per_second': '479.4'}

 11%|█▏        | 25700/225000 [60:38:48<330:43:15,  5.97s/it]
 11%|█▏        | 25701/225000 [60:38:56<362:47:30,  6.55s/it]
 11%|█▏        | 25702/225000 [60:38:59<313:48:30,  5.67s/it]
 11%|█▏        | 25703/225000 [60:39:06<337:31:17,  6.10s/it]
 11%|█▏        | 25704/225000 [60:39:14<355:30:20,  6.42s/it]
 11%|█▏        | 25705/225000 [60:39:18<330:15:45,  5.97s/it]
 11%|█▏        | 25706/225000 [60:39:24<319:49:14,  5.78s/it]
 11%|█▏        | 25707/225000 [60:39:29<303:14:52,  5.48s/it]
 11%|█▏        | 25708/225000 [60:39:34<300:54:16,  5.44s/it]
 11%|█▏        | 25709/225000 [60:39:39<301:12:10,  5.44s/it]
 11%|█▏        | 25710/225000 [60:39:45<302:04:46,  5.46s/it]
 11%|█▏        | 25711/225000 [60:39:57<418:15:52,  7.56s/it]
 11%|█▏        | 25712/225000 [60:40:03<383:51:20,  6.93s/it]
 11%|█▏        | 25713/225000 [60:40:12<425:22:46,  7.68s/it]
 11%|█▏        | 25714/225000 [60:40:16<364:15:59,  6.58s/it]
 11%|█▏        | 25715/225000 [60:40:20<326:02:13,  5.89s/it]
 11%|█▏        | 25716/225000 [60:40:25<295:12:11,  5.33s/it]
 11%|█▏        | 25717/225000 [60:40:33<347:14:46,  6.27s/it]
 11%|█▏        | 25718/225000 [60:40:41<379:02:33,  6.85s/it]
 11%|█▏        | 25719/225000 [60:40:45<334:08:04,  6.04s/it]
 11%|█▏        | 25720/225000 [60:40:56<411:22:32,  7.43s/it]
 11%|█▏        | 25721/225000 [60:41:00<352:56:17,  6.38s/it]
 11%|█▏        | 25722/225000 [60:41:04<310:21:04,  5.61s/it]
 11%|█▏        | 25723/225000 [60:41:09<301:06:40,  5.44s/it]
 11%|█▏        | 25724/225000 [60:41:21<416:00:50,  7.52s/it]
 11%|█▏        | 25725/225000 [60:41:25<356:58:19,  6.45s/it]
 11%|█▏        | 25726/225000 [60:41:29<318:09:45,  5.75s/it]
 11%|█▏        | 25727/225000 [60:41:33<289:52:41,  5.24s/it]
 11%|█▏        | 25728/225000 [60:41:40<318:24:56,  5.75s/it]
 11%|█▏        | 25729/225000 [60:41:51<402:26:21,  7.27s/it]
 11%|█▏        | 25730/225000 [60:41:55<351:46:53,  6.36s/it]
 11%|█▏        | 25731/225000 [60:42:00<322:41:24,  5.83s/it]
 11%|█▏        | 25732/225000 [60:42:06<333:48:33,  6.03s/it]
 11%|█▏        | 25733/225000 [60:42:17<406:33:06,  7.34s/it]
 11%|█▏        | 25734/225000 [60:42:25<424:10:33,  7.66s/it]
 11%|█▏        | 25735/225000 [60:42:29<358:11:22,  6.47s/it]
 11%|█▏        | 25736/225000 [60:42:37<384:58:44,  6.96s/it]
 11%|█▏        | 25737/225000 [60:42:41<337:52:10,  6.10s/it]
 11%|█▏        | 25738/225000 [60:42:46<321:45:49,  5.81s/it]
 11%|█▏        | 25739/225000 [60:42:52<317:55:08,  5.74s/it]
 11%|█▏        | 25740/225000 [60:42:57<316:27:52,  5.72s/it]
 11%|█▏        | 25741/225000 [60:43:03<312:19:29,  5.64s/it]
 11%|█▏        | 25742/225000 [60:43:08<308:59:08,  5.58s/it]
 11%|█▏        | 25743/225000 [60:43:19<391:46:00,  7.08s/it]
 11%|█▏        | 25744/225000 [60:43:27<414:16:45,  7.48s/it]
 11%|█▏        | 25745/225000 [60:43:32<359:20:54,  6.49s/it]
 11%|█▏        | 25746/225000 [60:43:41<404:19:06,  7.30s/it]
 11%|█▏        | 25747/225000 [60:43:46<365:12:28,  6.60s/it]
 11%|█▏        | 25748/225000 [60:43:51<341:05:29,  6.16s/it]
 11%|█▏        | 25749/225000 [60:43:56<327:49:08,  5.92s/it]
 11%|█▏        | 25750/225000 [60:44:02<321:20:31,  5.81s/it]
                                                             
{'loss': '0.7087', 'grad_norm': '0.8041', 'learning_rate': '4.934e-05', 'epoch': '0.1144', 'num_input_tokens_seen': 104814520, 'train_runtime': '2.186e+05', 'train_tokens_per_second': '479.4'}

 11%|█▏        | 25750/225000 [60:44:02<321:20:31,  5.81s/it]
 11%|█▏        | 25751/225000 [60:44:07<310:05:42,  5.60s/it]
 11%|█▏        | 25752/225000 [60:44:12<308:49:49,  5.58s/it]
 11%|█▏        | 25753/225000 [60:44:26<437:17:37,  7.90s/it]
 11%|█▏        | 25754/225000 [60:44:30<384:49:28,  6.95s/it]
 11%|█▏        | 25755/225000 [60:44:37<373:10:25,  6.74s/it]
 11%|█▏        | 25756/225000 [60:44:45<395:27:11,  7.15s/it]
 11%|█▏        | 25757/225000 [60:44:48<331:33:33,  5.99s/it]
 11%|█▏        | 25758/225000 [60:44:53<314:39:44,  5.69s/it]
 11%|█▏        | 25759/225000 [60:45:04<394:59:17,  7.14s/it]
 11%|█▏        | 25760/225000 [60:45:10<388:48:03,  7.03s/it]
 11%|█▏        | 25761/225000 [60:45:15<350:14:11,  6.33s/it]
 11%|█▏        | 25762/225000 [60:45:22<355:35:04,  6.42s/it]
 11%|█▏        | 25763/225000 [60:45:28<355:09:04,  6.42s/it]
 11%|█▏        | 25764/225000 [60:45:34<347:37:30,  6.28s/it]
 11%|█▏        | 25765/225000 [60:45:39<330:20:27,  5.97s/it]
 11%|█▏        | 25766/225000 [60:45:45<332:15:41,  6.00s/it]
 11%|█▏        | 25767/225000 [60:45:53<351:50:21,  6.36s/it]
 11%|█▏        | 25768/225000 [60:45:59<345:16:05,  6.24s/it]
 11%|█▏        | 25769/225000 [60:46:06<368:04:03,  6.65s/it]
 11%|█▏        | 25770/225000 [60:46:10<327:50:57,  5.92s/it]
 11%|█▏        | 25771/225000 [60:46:15<299:19:33,  5.41s/it]
 11%|█▏        | 25772/225000 [60:46:23<344:44:13,  6.23s/it]
 11%|█▏        | 25773/225000 [60:46:32<402:31:44,  7.27s/it]
 11%|█▏        | 25774/225000 [60:46:37<352:52:01,  6.38s/it]
 11%|█▏        | 25775/225000 [60:46:41<316:33:58,  5.72s/it]
 11%|█▏        | 25776/225000 [60:46:52<402:15:55,  7.27s/it]
 11%|█▏        | 25777/225000 [60:46:58<390:57:06,  7.06s/it]
 11%|█▏        | 25778/225000 [60:47:04<370:14:25,  6.69s/it]
 11%|█▏        | 25779/225000 [60:47:10<354:07:45,  6.40s/it]
 11%|█▏        | 25780/225000 [60:47:19<395:43:54,  7.15s/it]
 11%|█▏        | 25781/225000 [60:47:23<347:45:06,  6.28s/it]
 11%|█▏        | 25782/225000 [60:47:28<318:44:19,  5.76s/it]
 11%|█▏        | 25783/225000 [60:47:36<354:33:02,  6.41s/it]
 11%|█▏        | 25784/225000 [60:47:45<401:56:14,  7.26s/it]
 11%|█▏        | 25785/225000 [60:47:49<349:45:59,  6.32s/it]
 11%|█▏        | 25786/225000 [60:47:59<410:55:54,  7.43s/it]
 11%|█▏        | 25787/225000 [60:48:03<352:18:36,  6.37s/it]
 11%|█▏        | 25788/225000 [60:48:08<327:49:14,  5.92s/it]
 11%|█▏        | 25789/225000 [60:48:12<307:07:15,  5.55s/it]
 11%|█▏        | 25790/225000 [60:48:19<332:48:26,  6.01s/it]
 11%|█▏        | 25791/225000 [60:48:29<395:43:22,  7.15s/it]
 11%|█▏        | 25792/225000 [60:48:33<332:12:10,  6.00s/it]
 11%|█▏        | 25793/225000 [60:48:40<352:52:26,  6.38s/it]
 11%|█▏        | 25794/225000 [60:48:43<307:14:19,  5.55s/it]
 11%|█▏        | 25795/225000 [60:48:54<394:23:12,  7.13s/it]
 11%|█▏        | 25796/225000 [60:48:58<333:10:49,  6.02s/it]
 11%|█▏        | 25797/225000 [60:49:05<360:10:24,  6.51s/it]
 11%|█▏        | 25798/225000 [60:49:11<348:42:02,  6.30s/it]
 11%|█▏        | 25799/225000 [60:49:19<369:58:35,  6.69s/it]
 11%|█▏        | 25800/225000 [60:49:24<344:18:59,  6.22s/it]
                                                             
{'loss': '0.7152', 'grad_norm': '0.7052', 'learning_rate': '4.934e-05', 'epoch': '0.1147', 'num_input_tokens_seen': 104958360, 'train_runtime': '2.19e+05', 'train_tokens_per_second': '479.3'}

 11%|█▏        | 25800/225000 [60:49:24<344:18:59,  6.22s/it]
 11%|█▏        | 25801/225000 [60:49:28<312:53:09,  5.65s/it]
 11%|█▏        | 25802/225000 [60:49:33<296:57:28,  5.37s/it]
 11%|█▏        | 25803/225000 [60:49:38<286:29:52,  5.18s/it]
 11%|█▏        | 25804/225000 [60:49:49<389:51:56,  7.05s/it]
 11%|█▏        | 25805/225000 [60:49:55<370:54:30,  6.70s/it]
 11%|█▏        | 25806/225000 [60:50:03<391:41:29,  7.08s/it]
 11%|█▏        | 25807/225000 [60:50:10<385:48:18,  6.97s/it]
 11%|█▏        | 25808/225000 [60:50:16<383:27:24,  6.93s/it]
 11%|█▏        | 25809/225000 [60:50:21<335:42:44,  6.07s/it]
 11%|█▏        | 25810/225000 [60:50:25<302:23:28,  5.47s/it]
 11%|█▏        | 25811/225000 [60:50:33<348:20:46,  6.30s/it]
 11%|█▏        | 25812/225000 [60:50:41<376:04:10,  6.80s/it]
 11%|█▏        | 25813/225000 [60:50:50<408:20:02,  7.38s/it]
 11%|█▏        | 25814/225000 [60:50:54<365:35:06,  6.61s/it]
 11%|█▏        | 25815/225000 [60:50:59<330:05:45,  5.97s/it]
 11%|█▏        | 25816/225000 [60:51:04<311:22:17,  5.63s/it]
 11%|█▏        | 25817/225000 [60:51:13<374:47:26,  6.77s/it]
 11%|█▏        | 25818/225000 [60:51:20<372:58:05,  6.74s/it]
 11%|█▏        | 25819/225000 [60:51:27<379:46:24,  6.86s/it]
 11%|█▏        | 25820/225000 [60:51:31<328:02:24,  5.93s/it]
 11%|█▏        | 25821/225000 [60:51:35<296:36:26,  5.36s/it]
 11%|█▏        | 25822/225000 [60:51:44<367:43:02,  6.65s/it]
 11%|█▏        | 25823/225000 [60:51:49<336:20:41,  6.08s/it]
 11%|█▏        | 25824/225000 [60:51:54<318:54:38,  5.76s/it]
 11%|█▏        | 25825/225000 [60:51:59<296:17:23,  5.36s/it]
 11%|█▏        | 25826/225000 [60:52:05<314:20:36,  5.68s/it]
 11%|█▏        | 25827/225000 [60:52:16<397:01:34,  7.18s/it]
 11%|█▏        | 25828/225000 [60:52:19<340:18:50,  6.15s/it]
 11%|█▏        | 25829/225000 [60:52:28<378:58:53,  6.85s/it]
 11%|█▏        | 25830/225000 [60:52:31<324:21:20,  5.86s/it]
 11%|█▏        | 25831/225000 [60:52:41<384:53:59,  6.96s/it]
 11%|█▏        | 25832/225000 [60:52:45<335:24:06,  6.06s/it]
 11%|█▏        | 25833/225000 [60:52:49<298:36:34,  5.40s/it]
 11%|█▏        | 25834/225000 [60:52:55<311:39:24,  5.63s/it]
 11%|█▏        | 25835/225000 [60:53:04<363:12:30,  6.57s/it]
 11%|█▏        | 25836/225000 [60:53:11<373:53:17,  6.76s/it]
 11%|█▏        | 25837/225000 [60:53:15<327:11:35,  5.91s/it]
 11%|█▏        | 25838/225000 [60:53:19<301:52:55,  5.46s/it]
 11%|█▏        | 25839/225000 [60:53:24<284:33:16,  5.14s/it]
 11%|█▏        | 25840/225000 [60:53:35<394:25:46,  7.13s/it]
 11%|█▏        | 25841/225000 [60:53:40<347:34:04,  6.28s/it]
 11%|█▏        | 25842/225000 [60:53:43<305:38:59,  5.52s/it]
 11%|█▏        | 25843/225000 [60:53:48<294:27:36,  5.32s/it]
 11%|█▏        | 25844/225000 [60:53:53<286:57:15,  5.19s/it]
 11%|█▏        | 25845/225000 [60:54:04<374:56:44,  6.78s/it]
 11%|█▏        | 25846/225000 [60:54:13<416:17:33,  7.53s/it]
 11%|█▏        | 25847/225000 [60:54:16<349:02:25,  6.31s/it]
 11%|█▏        | 25848/225000 [60:54:25<384:02:08,  6.94s/it]
 11%|█▏        | 25849/225000 [60:54:29<336:56:17,  6.09s/it]
 11%|█▏        | 25850/225000 [60:54:39<399:10:37,  7.22s/it]
                                                             
{'loss': '0.7393', 'grad_norm': '0.7521', 'learning_rate': '4.933e-05', 'epoch': '0.1149', 'num_input_tokens_seen': 105099104, 'train_runtime': '2.193e+05', 'train_tokens_per_second': '479.3'}

 11%|█▏        | 25850/225000 [60:54:39<399:10:37,  7.22s/it]
 11%|█▏        | 25851/225000 [60:54:43<350:02:28,  6.33s/it]
 11%|█▏        | 25852/225000 [60:54:48<324:07:51,  5.86s/it]
 11%|█▏        | 25853/225000 [60:54:52<300:47:25,  5.44s/it]
 11%|█▏        | 25854/225000 [60:54:57<281:01:16,  5.08s/it]
 11%|█▏        | 25855/225000 [60:55:02<286:05:35,  5.17s/it]
 11%|█▏        | 25856/225000 [60:55:12<366:38:59,  6.63s/it]
 11%|█▏        | 25857/225000 [60:55:22<426:19:30,  7.71s/it]
 11%|█▏        | 25858/225000 [60:55:26<361:33:01,  6.54s/it]
 11%|█▏        | 25859/225000 [60:55:30<325:21:12,  5.88s/it]
 11%|█▏        | 25860/225000 [60:55:40<384:24:27,  6.95s/it]
 11%|█▏        | 25861/225000 [60:55:45<361:59:02,  6.54s/it]
 11%|█▏        | 25862/225000 [60:55:49<314:27:25,  5.68s/it]
 11%|█▏        | 25863/225000 [60:56:00<394:17:50,  7.13s/it]
 11%|█▏        | 25864/225000 [60:56:04<356:58:49,  6.45s/it]
 11%|█▏        | 25865/225000 [60:56:09<324:19:07,  5.86s/it]
 11%|█▏        | 25866/225000 [60:56:13<303:20:55,  5.48s/it]
 11%|█▏        | 25867/225000 [60:56:24<388:11:12,  7.02s/it]
 11%|█▏        | 25868/225000 [60:56:28<334:47:41,  6.05s/it]
 11%|█▏        | 25869/225000 [60:56:37<382:19:33,  6.91s/it]
 11%|█▏        | 25870/225000 [60:56:41<336:38:28,  6.09s/it]
 11%|█▏        | 25871/225000 [60:56:44<294:30:25,  5.32s/it]
 11%|█▏        | 25872/225000 [60:56:54<362:17:38,  6.55s/it]
 11%|█▏        | 25873/225000 [60:56:57<311:48:00,  5.64s/it]
 11%|█▏        | 25874/225000 [60:57:06<368:27:41,  6.66s/it]
 12%|█▏        | 25875/225000 [60:57:14<382:28:15,  6.91s/it]
 12%|█▏        | 25876/225000 [60:57:19<354:45:22,  6.41s/it]
 12%|█▏        | 25877/225000 [60:57:23<317:08:02,  5.73s/it]
 12%|█▏        | 25878/225000 [60:57:33<376:57:53,  6.82s/it]
 12%|█▏        | 25879/225000 [60:57:38<351:34:26,  6.36s/it]
 12%|█▏        | 25880/225000 [60:57:44<339:10:06,  6.13s/it]
 12%|█▏        | 25881/225000 [60:57:54<402:23:50,  7.28s/it]
 12%|█▏        | 25882/225000 [60:57:59<365:56:29,  6.62s/it]
 12%|█▏        | 25883/225000 [60:58:04<337:38:26,  6.10s/it]
 12%|█▏        | 25884/225000 [60:58:08<309:27:14,  5.59s/it]
 12%|█▏        | 25885/225000 [60:58:12<289:39:57,  5.24s/it]
 12%|█▏        | 25886/225000 [60:58:18<289:46:31,  5.24s/it]
 12%|█▏        | 25887/225000 [60:58:27<356:55:56,  6.45s/it]
 12%|█▏        | 25888/225000 [60:58:34<373:53:22,  6.76s/it]
 12%|█▏        | 25889/225000 [60:58:43<401:57:32,  7.27s/it]
 12%|█▏        | 25890/225000 [60:58:49<379:37:38,  6.86s/it]
 12%|█▏        | 25891/225000 [60:58:54<359:31:23,  6.50s/it]
 12%|█▏        | 25892/225000 [60:59:00<340:09:58,  6.15s/it]
 12%|█▏        | 25893/225000 [60:59:05<328:45:50,  5.94s/it]
 12%|█▏        | 25894/225000 [60:59:10<309:09:20,  5.59s/it]
 12%|█▏        | 25895/225000 [60:59:15<295:15:40,  5.34s/it]
 12%|█▏        | 25896/225000 [60:59:27<407:07:26,  7.36s/it]
 12%|█▏        | 25897/225000 [60:59:32<372:01:19,  6.73s/it]
 12%|█▏        | 25898/225000 [60:59:38<367:26:20,  6.64s/it]
 12%|█▏        | 25899/225000 [60:59:43<326:48:26,  5.91s/it]
 12%|█▏        | 25900/225000 [60:59:52<384:55:39,  6.96s/it]
                                                             
{'loss': '0.7013', 'grad_norm': '0.7058', 'learning_rate': '4.933e-05', 'epoch': '0.1151', 'num_input_tokens_seen': 105240976, 'train_runtime': '2.196e+05', 'train_tokens_per_second': '479.3'}

 12%|█▏        | 25900/225000 [60:59:52<384:55:39,  6.96s/it]
 12%|█▏        | 25901/225000 [60:59:57<358:06:26,  6.48s/it]
 12%|█▏        | 25902/225000 [61:00:05<379:04:36,  6.85s/it]
 12%|█▏        | 25903/225000 [61:00:10<343:09:54,  6.20s/it]
 12%|█▏        | 25904/225000 [61:00:14<312:26:06,  5.65s/it]
 12%|█▏        | 25905/225000 [61:00:19<290:41:26,  5.26s/it]
 12%|█▏        | 25906/225000 [61:00:23<281:31:15,  5.09s/it]
 12%|█▏        | 25907/225000 [61:00:33<351:26:44,  6.35s/it]
 12%|█▏        | 25908/225000 [61:00:42<402:41:55,  7.28s/it]
 12%|█▏        | 25909/225000 [61:00:48<387:06:46,  7.00s/it]
 12%|█▏        | 25910/225000 [61:00:53<343:58:50,  6.22s/it]
 12%|█▏        | 25911/225000 [61:00:57<318:54:08,  5.77s/it]
 12%|█▏        | 25912/225000 [61:01:02<299:06:22,  5.41s/it]
 12%|█▏        | 25913/225000 [61:01:15<420:34:54,  7.61s/it]
 12%|█▏        | 25914/225000 [61:01:19<367:20:12,  6.64s/it]
 12%|█▏        | 25915/225000 [61:01:25<360:07:09,  6.51s/it]
 12%|█▏        | 25916/225000 [61:01:32<355:21:35,  6.43s/it]
 12%|█▏        | 25917/225000 [61:01:37<331:16:55,  5.99s/it]
 12%|█▏        | 25918/225000 [61:01:46<383:09:15,  6.93s/it]
 12%|█▏        | 25919/225000 [61:01:49<327:20:36,  5.92s/it]
 12%|█▏        | 25920/225000 [61:01:58<373:30:25,  6.75s/it]
 12%|█▏        | 25921/225000 [61:02:02<332:11:39,  6.01s/it]
 12%|█▏        | 25922/225000 [61:02:06<300:55:20,  5.44s/it]
 12%|█▏        | 25923/225000 [61:02:14<332:22:28,  6.01s/it]
 12%|█▏        | 25924/225000 [61:02:23<383:45:05,  6.94s/it]
 12%|█▏        | 25925/225000 [61:02:27<338:00:24,  6.11s/it]
 12%|█▏        | 25926/225000 [61:02:31<309:15:58,  5.59s/it]
 12%|█▏        | 25927/225000 [61:02:36<289:41:43,  5.24s/it]
 12%|█▏        | 25928/225000 [61:02:41<281:59:08,  5.10s/it]
 12%|█▏        | 25929/225000 [61:02:53<408:38:42,  7.39s/it]
 12%|█▏        | 25930/225000 [61:02:58<365:15:01,  6.61s/it]
 12%|█▏        | 25931/225000 [61:03:03<334:55:47,  6.06s/it]
 12%|█▏        | 25932/225000 [61:03:11<363:24:23,  6.57s/it]
 12%|█▏        | 25933/225000 [61:03:15<332:58:20,  6.02s/it]
 12%|█▏        | 25934/225000 [61:03:21<319:59:53,  5.79s/it]
 12%|█▏        | 25935/225000 [61:03:25<297:35:18,  5.38s/it]
 12%|█▏        | 25936/225000 [61:03:32<332:26:09,  6.01s/it]
 12%|█▏        | 25937/225000 [61:03:42<386:33:53,  6.99s/it]
 12%|█▏        | 25938/225000 [61:03:47<357:39:09,  6.47s/it]
 12%|█▏        | 25939/225000 [61:03:52<329:25:02,  5.96s/it]
 12%|█▏        | 25940/225000 [61:03:59<350:46:34,  6.34s/it]
 12%|█▏        | 25941/225000 [61:04:09<417:18:33,  7.55s/it]
 12%|█▏        | 25942/225000 [61:04:13<355:53:25,  6.44s/it]
 12%|█▏        | 25943/225000 [61:04:22<394:02:44,  7.13s/it]
 12%|█▏        | 25944/225000 [61:04:26<340:06:53,  6.15s/it]
 12%|█▏        | 25945/225000 [61:04:34<377:57:33,  6.84s/it]
 12%|█▏        | 25946/225000 [61:04:39<339:05:40,  6.13s/it]
 12%|█▏        | 25947/225000 [61:04:46<355:08:27,  6.42s/it]
 12%|█▏        | 25948/225000 [61:04:52<358:55:17,  6.49s/it]
 12%|█▏        | 25949/225000 [61:04:57<332:51:25,  6.02s/it]
 12%|█▏        | 25950/225000 [61:05:02<305:41:18,  5.53s/it]
                                                             
{'loss': '0.7109', 'grad_norm': '0.6568', 'learning_rate': '4.933e-05', 'epoch': '0.1153', 'num_input_tokens_seen': 105383000, 'train_runtime': '2.199e+05', 'train_tokens_per_second': '479.2'}

 12%|█▏        | 25950/225000 [61:05:02<305:41:18,  5.53s/it]
 12%|█▏        | 25951/225000 [61:05:07<292:20:29,  5.29s/it]
 12%|█▏        | 25952/225000 [61:05:19<404:50:03,  7.32s/it]
 12%|█▏        | 25953/225000 [61:05:23<361:30:53,  6.54s/it]
 12%|█▏        | 25954/225000 [61:05:30<368:33:18,  6.67s/it]
 12%|█▏        | 25955/225000 [61:05:35<341:32:11,  6.18s/it]
 12%|█▏        | 25956/225000 [61:05:44<384:13:34,  6.95s/it]
 12%|█▏        | 25957/225000 [61:05:48<331:13:40,  5.99s/it]
 12%|█▏        | 25958/225000 [61:05:57<388:32:15,  7.03s/it]
 12%|█▏        | 25959/225000 [61:06:03<373:39:02,  6.76s/it]
 12%|█▏        | 25960/225000 [61:06:08<335:27:29,  6.07s/it]
 12%|█▏        | 25961/225000 [61:06:17<380:01:05,  6.87s/it]
 12%|█▏        | 25962/225000 [61:06:21<346:37:19,  6.27s/it]
 12%|█▏        | 25963/225000 [61:06:27<330:08:16,  5.97s/it]
 12%|█▏        | 25964/225000 [61:06:32<310:28:57,  5.62s/it]
 12%|█▏        | 25965/225000 [61:06:36<287:42:33,  5.20s/it]
 12%|█▏        | 25966/225000 [61:06:41<283:50:19,  5.13s/it]
 12%|█▏        | 25967/225000 [61:06:51<361:26:55,  6.54s/it]
 12%|█▏        | 25968/225000 [61:07:01<420:29:22,  7.61s/it]
 12%|█▏        | 25969/225000 [61:07:04<352:28:41,  6.38s/it]
 12%|█▏        | 25970/225000 [61:07:13<391:16:08,  7.08s/it]
 12%|█▏        | 25971/225000 [61:07:16<330:52:08,  5.98s/it]
 12%|█▏        | 25972/225000 [61:07:23<346:53:01,  6.27s/it]
 12%|█▏        | 25973/225000 [61:07:29<333:05:26,  6.02s/it]
 12%|█▏        | 25974/225000 [61:07:33<310:58:29,  5.62s/it]
 12%|█▏        | 25975/225000 [61:07:38<291:13:53,  5.27s/it]
 12%|█▏        | 25976/225000 [61:07:49<394:02:33,  7.13s/it]
 12%|█▏        | 25977/225000 [61:07:54<361:25:12,  6.54s/it]
 12%|█▏        | 25978/225000 [61:08:01<368:37:41,  6.67s/it]
 12%|█▏        | 25979/225000 [61:08:06<333:04:06,  6.02s/it]
 12%|█▏        | 25980/225000 [61:08:10<301:10:58,  5.45s/it]
 12%|█▏        | 25981/225000 [61:08:21<385:23:04,  6.97s/it]
 12%|█▏        | 25982/225000 [61:08:25<347:53:36,  6.29s/it]
 12%|█▏        | 25983/225000 [61:08:29<307:25:53,  5.56s/it]
 12%|█▏        | 25984/225000 [61:08:40<394:48:02,  7.14s/it]
 12%|█▏        | 25985/225000 [61:08:44<344:18:24,  6.23s/it]
 12%|█▏        | 25986/225000 [61:08:49<329:06:55,  5.95s/it]
 12%|█▏        | 25987/225000 [61:08:54<306:37:32,  5.55s/it]
 12%|█▏        | 25988/225000 [61:08:58<285:45:31,  5.17s/it]
 12%|█▏        | 25989/225000 [61:09:08<357:10:50,  6.46s/it]
 12%|█▏        | 25990/225000 [61:09:13<338:51:07,  6.13s/it]
 12%|█▏        | 25991/225000 [61:09:18<316:06:43,  5.72s/it]
 12%|█▏        | 25992/225000 [61:09:23<303:02:47,  5.48s/it]
 12%|█▏        | 25993/225000 [61:09:28<302:52:09,  5.48s/it]
 12%|█▏        | 25994/225000 [61:09:36<346:59:21,  6.28s/it]
 12%|█▏        | 25995/225000 [61:09:49<448:07:43,  8.11s/it]
 12%|█▏        | 25996/225000 [61:09:53<390:12:44,  7.06s/it]
 12%|█▏        | 25997/225000 [61:09:59<361:40:51,  6.54s/it]
 12%|█▏        | 25998/225000 [61:10:04<342:54:53,  6.20s/it]
 12%|█▏        | 25999/225000 [61:10:10<336:28:44,  6.09s/it]
 12%|█▏        | 26000/225000 [61:10:15<318:17:23,  5.76s/it]
                                                             
{'loss': '0.7131', 'grad_norm': '0.7329', 'learning_rate': '4.932e-05', 'epoch': '0.1156', 'num_input_tokens_seen': 105525936, 'train_runtime': '2.202e+05', 'train_tokens_per_second': '479.2'}

 12%|█▏        | 26000/225000 [61:10:15<318:17:23,  5.76s/it]
 12%|█▏        | 26001/225000 [61:10:20<301:41:43,  5.46s/it]
 12%|█▏        | 26002/225000 [61:10:25<300:56:55,  5.44s/it]
 12%|█▏        | 26003/225000 [61:10:33<340:11:25,  6.15s/it]
 12%|█▏        | 26004/225000 [61:10:38<319:14:20,  5.78s/it]
 12%|█▏        | 26005/225000 [61:10:48<391:29:31,  7.08s/it]
 12%|█▏        | 26006/225000 [61:10:57<429:19:49,  7.77s/it]
 12%|█▏        | 26007/225000 [61:11:01<366:45:30,  6.64s/it]
 12%|█▏        | 26008/225000 [61:11:09<391:47:50,  7.09s/it]
 12%|█▏        | 26009/225000 [61:11:17<401:26:27,  7.26s/it]
 12%|█▏        | 26010/225000 [61:11:22<356:59:21,  6.46s/it]
 12%|█▏        | 26011/225000 [61:11:27<331:29:37,  6.00s/it]
 12%|█▏        | 26012/225000 [61:11:32<316:05:19,  5.72s/it]
 12%|█▏        | 26013/225000 [61:11:37<308:38:34,  5.58s/it]
 12%|█▏        | 26014/225000 [61:11:49<409:09:28,  7.40s/it]
 12%|█▏        | 26015/225000 [61:11:54<379:35:52,  6.87s/it]
 12%|█▏        | 26016/225000 [61:12:00<361:29:00,  6.54s/it]
 12%|█▏        | 26017/225000 [61:12:04<320:53:37,  5.81s/it]
 12%|█▏        | 26018/225000 [61:12:12<362:19:41,  6.56s/it]
 12%|█▏        | 26019/225000 [61:12:18<343:05:22,  6.21s/it]
 12%|█▏        | 26020/225000 [61:12:23<319:32:10,  5.78s/it]
 12%|█▏        | 26021/225000 [61:12:31<371:13:05,  6.72s/it]
 12%|█▏        | 26022/225000 [61:12:40<397:11:51,  7.19s/it]
 12%|█▏        | 26023/225000 [61:12:46<383:24:49,  6.94s/it]
 12%|█▏        | 26024/225000 [61:12:52<361:25:14,  6.54s/it]
 12%|█▏        | 26025/225000 [61:12:59<369:31:53,  6.69s/it]
 12%|█▏        | 26026/225000 [61:13:06<376:18:59,  6.81s/it]
 12%|█▏        | 26027/225000 [61:13:11<356:24:25,  6.45s/it]
 12%|█▏        | 26028/225000 [61:13:17<334:12:46,  6.05s/it]
 12%|█▏        | 26029/225000 [61:13:22<329:39:45,  5.96s/it]
 12%|█▏        | 26030/225000 [61:13:28<317:39:15,  5.75s/it]
 12%|█▏        | 26031/225000 [61:13:33<313:52:02,  5.68s/it]
 12%|█▏        | 26032/225000 [61:13:38<309:25:52,  5.60s/it]
 12%|█▏        | 26033/225000 [61:13:43<296:29:44,  5.36s/it]
 12%|█▏        | 26034/225000 [61:13:48<288:08:34,  5.21s/it]
 12%|█▏        | 26035/225000 [61:14:02<424:01:39,  7.67s/it]
 12%|█▏        | 26036/225000 [61:14:06<369:19:58,  6.68s/it]
 12%|█▏        | 26037/225000 [61:14:15<403:24:20,  7.30s/it]
 12%|█▏        | 26038/225000 [61:14:19<347:20:19,  6.28s/it]
 12%|█▏        | 26039/225000 [61:14:23<317:28:06,  5.74s/it]
 12%|█▏        | 26040/225000 [61:14:29<317:19:56,  5.74s/it]
 12%|█▏        | 26041/225000 [61:14:40<406:31:40,  7.36s/it]
 12%|█▏        | 26042/225000 [61:14:45<362:19:40,  6.56s/it]
 12%|█▏        | 26043/225000 [61:14:53<393:14:15,  7.12s/it]
 12%|█▏        | 26044/225000 [61:14:59<368:38:14,  6.67s/it]
 12%|█▏        | 26045/225000 [61:15:05<363:52:04,  6.58s/it]
 12%|█▏        | 26046/225000 [61:15:10<330:11:51,  5.97s/it]
 12%|█▏        | 26047/225000 [61:15:14<301:02:13,  5.45s/it]
 12%|█▏        | 26048/225000 [61:15:18<283:33:58,  5.13s/it]
 12%|█▏        | 26049/225000 [61:15:29<369:11:25,  6.68s/it]
 12%|█▏        | 26050/225000 [61:15:36<377:34:20,  6.83s/it]
                                                             
{'loss': '0.6956', 'grad_norm': '0.6844', 'learning_rate': '4.932e-05', 'epoch': '0.1158', 'num_input_tokens_seen': 105669008, 'train_runtime': '2.205e+05', 'train_tokens_per_second': '479.1'}

 12%|█▏        | 26050/225000 [61:15:36<377:34:20,  6.83s/it]
 12%|█▏        | 26051/225000 [61:15:43<382:09:16,  6.92s/it]
 12%|█▏        | 26052/225000 [61:15:49<362:45:26,  6.56s/it]
 12%|█▏        | 26053/225000 [61:15:55<354:24:35,  6.41s/it]
 12%|█▏        | 26054/225000 [61:16:00<343:53:27,  6.22s/it]
 12%|█▏        | 26055/225000 [61:16:07<347:35:52,  6.29s/it]
 12%|█▏        | 26056/225000 [61:16:11<305:06:39,  5.52s/it]
 12%|█▏        | 26057/225000 [61:16:21<379:55:08,  6.87s/it]
 12%|█▏        | 26058/225000 [61:16:28<396:21:42,  7.17s/it]
 12%|█▏        | 26059/225000 [61:16:33<357:44:33,  6.47s/it]
 12%|█▏        | 26060/225000 [61:16:38<334:46:55,  6.06s/it]
 12%|█▏        | 26061/225000 [61:16:44<321:13:33,  5.81s/it]
 12%|█▏        | 26062/225000 [61:16:49<313:56:52,  5.68s/it]
 12%|█▏        | 26063/225000 [61:16:55<312:27:05,  5.65s/it]
 12%|█▏        | 26064/225000 [61:17:00<304:20:14,  5.51s/it]
 12%|█▏        | 26065/225000 [61:17:09<361:06:37,  6.53s/it]
 12%|█▏        | 26066/225000 [61:17:21<453:42:20,  8.21s/it]
 12%|█▏        | 26067/225000 [61:17:27<417:08:30,  7.55s/it]
 12%|█▏        | 26068/225000 [61:17:33<388:47:45,  7.04s/it]
 12%|█▏        | 26069/225000 [61:17:37<336:54:41,  6.10s/it]
 12%|█▏        | 26070/225000 [61:17:41<312:49:24,  5.66s/it]
 12%|█▏        | 26071/225000 [61:17:46<294:19:06,  5.33s/it]
 12%|█▏        | 26072/225000 [61:17:58<407:12:30,  7.37s/it]
 12%|█▏        | 26073/225000 [61:18:02<358:49:00,  6.49s/it]
 12%|█▏        | 26074/225000 [61:18:12<417:04:25,  7.55s/it]
 12%|█▏        | 26075/225000 [61:18:16<353:11:56,  6.39s/it]
 12%|█▏        | 26076/225000 [61:18:20<312:56:28,  5.66s/it]
 12%|█▏        | 26077/225000 [61:18:24<286:09:58,  5.18s/it]
 12%|█▏        | 26078/225000 [61:18:37<414:27:21,  7.50s/it]
 12%|█▏        | 26079/225000 [61:18:41<363:38:06,  6.58s/it]
 12%|█▏        | 26080/225000 [61:18:47<340:08:26,  6.16s/it]
 12%|█▏        | 26081/225000 [61:18:52<325:23:35,  5.89s/it]
 12%|█▏        | 26082/225000 [61:18:57<315:51:05,  5.72s/it]
 12%|█▏        | 26083/225000 [61:19:02<308:31:23,  5.58s/it]
 12%|█▏        | 26084/225000 [61:19:08<301:28:57,  5.46s/it]
 12%|█▏        | 26085/225000 [61:19:12<288:14:55,  5.22s/it]
 12%|█▏        | 26086/225000 [61:19:20<336:49:01,  6.10s/it]
 12%|█▏        | 26087/225000 [61:19:34<453:07:17,  8.20s/it]
 12%|█▏        | 26088/225000 [61:19:37<376:05:56,  6.81s/it]
 12%|█▏        | 26089/225000 [61:19:44<382:05:22,  6.92s/it]
 12%|█▏        | 26090/225000 [61:19:51<374:07:49,  6.77s/it]
 12%|█▏        | 26091/225000 [61:19:54<318:57:48,  5.77s/it]
 12%|█▏        | 26092/225000 [61:20:00<322:52:21,  5.84s/it]
 12%|█▏        | 26093/225000 [61:20:09<370:57:28,  6.71s/it]
 12%|█▏        | 26094/225000 [61:20:13<334:25:05,  6.05s/it]
 12%|█▏        | 26095/225000 [61:20:18<307:29:37,  5.57s/it]
 12%|█▏        | 26096/225000 [61:20:23<298:50:37,  5.41s/it]
 12%|█▏        | 26097/225000 [61:20:28<297:53:51,  5.39s/it]
 12%|█▏        | 26098/225000 [61:20:36<336:28:49,  6.09s/it]
 12%|█▏        | 26099/225000 [61:20:49<455:12:14,  8.24s/it]
 12%|█▏        | 26100/225000 [61:20:52<371:42:19,  6.73s/it]
                                                             
{'loss': '0.7238', 'grad_norm': '0.6807', 'learning_rate': '4.931e-05', 'epoch': '0.116', 'num_input_tokens_seen': 105811192, 'train_runtime': '2.209e+05', 'train_tokens_per_second': '479.1'}

 12%|█▏        | 26100/225000 [61:20:52<371:42:19,  6.73s/it]
 12%|█▏        | 26101/225000 [61:21:00<382:04:48,  6.92s/it]
 12%|█▏        | 26102/225000 [61:21:05<348:03:35,  6.30s/it]
 12%|█▏        | 26103/225000 [61:21:10<332:27:51,  6.02s/it]
 12%|█▏        | 26104/225000 [61:21:15<312:01:09,  5.65s/it]
 12%|█▏        | 26105/225000 [61:21:19<295:35:19,  5.35s/it]
 12%|█▏        | 26106/225000 [61:21:24<289:26:08,  5.24s/it]
 12%|█▏        | 26107/225000 [61:21:31<317:57:42,  5.76s/it]
 12%|█▏        | 26108/225000 [61:21:45<444:32:05,  8.05s/it]
 12%|█▏        | 26109/225000 [61:21:48<367:53:34,  6.66s/it]
 12%|█▏        | 26110/225000 [61:21:54<357:48:19,  6.48s/it]
 12%|█▏        | 26111/225000 [61:22:02<374:14:36,  6.77s/it]
 12%|█▏        | 26112/225000 [61:22:07<355:32:16,  6.44s/it]
 12%|█▏        | 26113/225000 [61:22:16<386:52:44,  7.00s/it]
 12%|█▏        | 26114/225000 [61:22:23<392:13:22,  7.10s/it]
 12%|█▏        | 26115/225000 [61:22:26<331:01:18,  5.99s/it]
 12%|█▏        | 26116/225000 [61:22:34<361:02:25,  6.54s/it]
 12%|█▏        | 26117/225000 [61:22:38<318:58:34,  5.77s/it]
 12%|█▏        | 26118/225000 [61:22:45<333:01:41,  6.03s/it]
 12%|█▏        | 26119/225000 [61:22:53<375:03:34,  6.79s/it]
 12%|█▏        | 26120/225000 [61:22:57<317:54:57,  5.75s/it]
 12%|█▏        | 26121/225000 [61:23:05<363:36:57,  6.58s/it]
 12%|█▏        | 26122/225000 [61:23:11<348:47:12,  6.31s/it]
 12%|█▏        | 26123/225000 [61:23:16<334:47:28,  6.06s/it]
 12%|█▏        | 26124/225000 [61:23:22<332:17:11,  6.01s/it]
 12%|█▏        | 26125/225000 [61:23:30<363:08:59,  6.57s/it]
 12%|█▏        | 26126/225000 [61:23:36<353:06:47,  6.39s/it]
 12%|█▏        | 26127/225000 [61:23:42<338:24:10,  6.13s/it]
 12%|█▏        | 26128/225000 [61:23:51<396:54:03,  7.18s/it]
 12%|█▏        | 26129/225000 [61:23:55<334:01:09,  6.05s/it]
 12%|█▏        | 26130/225000 [61:24:03<377:30:10,  6.83s/it]
 12%|█▏        | 26131/225000 [61:24:07<322:40:35,  5.84s/it]
 12%|█▏        | 26132/225000 [61:24:16<383:01:27,  6.93s/it]
 12%|█▏        | 26133/225000 [61:24:21<345:57:15,  6.26s/it]
 12%|█▏        | 26134/225000 [61:24:27<333:04:03,  6.03s/it]
 12%|█▏        | 26135/225000 [61:24:31<306:50:28,  5.55s/it]
 12%|█▏        | 26136/225000 [61:24:37<309:53:28,  5.61s/it]
 12%|█▏        | 26137/225000 [61:24:42<305:08:25,  5.52s/it]
 12%|█▏        | 26138/225000 [61:24:47<297:49:40,  5.39s/it]
 12%|█▏        | 26139/225000 [61:24:52<296:02:38,  5.36s/it]
 12%|█▏        | 26140/225000 [61:25:05<410:52:30,  7.44s/it]
 12%|█▏        | 26141/225000 [61:25:11<388:02:22,  7.02s/it]
 12%|█▏        | 26142/225000 [61:25:15<345:18:55,  6.25s/it]
 12%|█▏        | 26143/225000 [61:25:21<340:46:23,  6.17s/it]
 12%|█▏        | 26144/225000 [61:25:26<318:20:07,  5.76s/it]
 12%|█▏        | 26145/225000 [61:25:31<305:35:02,  5.53s/it]
 12%|█▏        | 26146/225000 [61:25:36<304:50:05,  5.52s/it]
 12%|█▏        | 26147/225000 [61:25:43<328:42:57,  5.95s/it]
 12%|█▏        | 26148/225000 [61:25:57<452:42:35,  8.20s/it]
 12%|█▏        | 26149/225000 [61:26:03<410:27:07,  7.43s/it]
 12%|█▏        | 26150/225000 [61:26:09<393:37:32,  7.13s/it]
                                                             
{'loss': '0.722', 'grad_norm': '0.6935', 'learning_rate': '4.931e-05', 'epoch': '0.1162', 'num_input_tokens_seen': 105954328, 'train_runtime': '2.212e+05', 'train_tokens_per_second': '479.1'}

 12%|█▏        | 26150/225000 [61:26:09<393:37:32,  7.13s/it]
 12%|█▏        | 26151/225000 [61:26:17<407:02:40,  7.37s/it]
 12%|█▏        | 26152/225000 [61:26:24<406:41:18,  7.36s/it]
 12%|█▏        | 26153/225000 [61:26:33<432:20:38,  7.83s/it]
 12%|█▏        | 26154/225000 [61:26:38<391:16:26,  7.08s/it]
 12%|█▏        | 26155/225000 [61:26:43<349:41:56,  6.33s/it]
 12%|█▏        | 26156/225000 [61:26:54<432:15:50,  7.83s/it]
 12%|█▏        | 26157/225000 [61:27:00<388:46:40,  7.04s/it]
 12%|█▏        | 26158/225000 [61:27:05<365:08:52,  6.61s/it]
 12%|█▏        | 26159/225000 [61:27:10<326:59:58,  5.92s/it]
 12%|█▏        | 26160/225000 [61:27:14<305:11:30,  5.53s/it]
 12%|█▏        | 26161/225000 [61:27:21<322:58:01,  5.85s/it]
 12%|█▏        | 26162/225000 [61:27:31<391:05:53,  7.08s/it]
 12%|█▏        | 26163/225000 [61:27:35<345:25:13,  6.25s/it]
 12%|█▏        | 26164/225000 [61:27:40<325:04:42,  5.89s/it]
 12%|█▏        | 26165/225000 [61:27:49<378:54:05,  6.86s/it]
 12%|█▏        | 26166/225000 [61:27:56<377:40:21,  6.84s/it]
 12%|█▏        | 26167/225000 [61:28:00<325:33:14,  5.89s/it]
 12%|█▏        | 26168/225000 [61:28:05<311:49:24,  5.65s/it]
 12%|█▏        | 26169/225000 [61:28:10<304:49:15,  5.52s/it]
 12%|█▏        | 26170/225000 [61:28:15<294:46:10,  5.34s/it]
 12%|█▏        | 26171/225000 [61:28:24<361:54:46,  6.55s/it]
 12%|█▏        | 26172/225000 [61:28:33<395:54:04,  7.17s/it]
 12%|█▏        | 26173/225000 [61:28:38<365:22:50,  6.62s/it]
 12%|█▏        | 26174/225000 [61:28:46<378:32:21,  6.85s/it]
 12%|█▏        | 26175/225000 [61:28:51<361:39:12,  6.55s/it]
 12%|█▏        | 26176/225000 [61:28:57<342:43:02,  6.21s/it]
 12%|█▏        | 26177/225000 [61:29:02<326:10:08,  5.91s/it]
 12%|█▏        | 26178/225000 [61:29:07<312:39:31,  5.66s/it]
 12%|█▏        | 26179/225000 [61:29:12<308:16:14,  5.58s/it]
 12%|█▏        | 26180/225000 [61:29:18<307:13:23,  5.56s/it]
 12%|█▏        | 26181/225000 [61:29:23<301:19:29,  5.46s/it]
 12%|█▏        | 26182/225000 [61:29:33<380:42:35,  6.89s/it]
 12%|█▏        | 26183/225000 [61:29:42<413:37:31,  7.49s/it]
 12%|█▏        | 26184/225000 [61:29:49<399:16:47,  7.23s/it]
 12%|█▏        | 26185/225000 [61:29:56<389:55:07,  7.06s/it]
 12%|█▏        | 26186/225000 [61:30:00<340:11:10,  6.16s/it]
 12%|█▏        | 26187/225000 [61:30:09<384:35:06,  6.96s/it]
 12%|█▏        | 26188/225000 [61:30:14<352:00:17,  6.37s/it]
 12%|█▏        | 26189/225000 [61:30:18<319:53:55,  5.79s/it]
 12%|█▏        | 26190/225000 [61:30:22<296:14:58,  5.36s/it]
 12%|█▏        | 26191/225000 [61:30:32<368:40:23,  6.68s/it]
 12%|█▏        | 26192/225000 [61:30:40<395:27:03,  7.16s/it]
 12%|█▏        | 26193/225000 [61:30:47<392:31:23,  7.11s/it]
 12%|█▏        | 26194/225000 [61:30:54<393:11:36,  7.12s/it]
 12%|█▏        | 26195/225000 [61:30:58<340:37:10,  6.17s/it]
 12%|█▏        | 26196/225000 [61:31:03<309:29:37,  5.60s/it]
 12%|█▏        | 26197/225000 [61:31:12<376:25:52,  6.82s/it]
 12%|█▏        | 26198/225000 [61:31:16<329:03:03,  5.96s/it]
 12%|█▏        | 26199/225000 [61:31:24<354:57:43,  6.43s/it]
 12%|█▏        | 26200/225000 [61:31:31<368:27:16,  6.67s/it]
                                                             
{'loss': '0.7309', 'grad_norm': '0.6578', 'learning_rate': '4.93e-05', 'epoch': '0.1164', 'num_input_tokens_seen': 106096512, 'train_runtime': '2.215e+05', 'train_tokens_per_second': '479'}

 12%|█▏        | 26200/225000 [61:31:31<368:27:16,  6.67s/it]
 12%|█▏        | 26201/225000 [61:31:36<346:33:54,  6.28s/it]
 12%|█▏        | 26202/225000 [61:31:41<323:25:16,  5.86s/it]
 12%|█▏        | 26203/225000 [61:31:46<308:42:45,  5.59s/it]
 12%|█▏        | 26204/225000 [61:31:52<302:22:20,  5.48s/it]
 12%|█▏        | 26205/225000 [61:32:04<422:23:38,  7.65s/it]
 12%|█▏        | 26206/225000 [61:32:09<375:51:02,  6.81s/it]
 12%|█▏        | 26207/225000 [61:32:14<344:45:19,  6.24s/it]
 12%|█▏        | 26208/225000 [61:32:19<320:34:20,  5.81s/it]
 12%|█▏        | 26209/225000 [61:32:24<303:43:13,  5.50s/it]
 12%|█▏        | 26210/225000 [61:32:30<320:37:40,  5.81s/it]
 12%|█▏        | 26211/225000 [61:32:42<429:39:18,  7.78s/it]
 12%|█▏        | 26212/225000 [61:32:49<410:49:29,  7.44s/it]
 12%|█▏        | 26213/225000 [61:32:54<366:47:47,  6.64s/it]
 12%|█▏        | 26214/225000 [61:33:00<350:54:09,  6.35s/it]
 12%|█▏        | 26215/225000 [61:33:04<325:34:40,  5.90s/it]
 12%|█▏        | 26216/225000 [61:33:09<311:10:42,  5.64s/it]
 12%|█▏        | 26217/225000 [61:33:16<326:20:18,  5.91s/it]
 12%|█▏        | 26218/225000 [61:33:27<413:11:07,  7.48s/it]
 12%|█▏        | 26219/225000 [61:33:31<357:52:18,  6.48s/it]
 12%|█▏        | 26220/225000 [61:33:39<383:58:20,  6.95s/it]
 12%|█▏        | 26221/225000 [61:33:47<403:05:51,  7.30s/it]
 12%|█▏        | 26222/225000 [61:33:56<417:14:59,  7.56s/it]
 12%|█▏        | 26223/225000 [61:34:00<370:26:14,  6.71s/it]
 12%|█▏        | 26224/225000 [61:34:04<315:09:49,  5.71s/it]
 12%|█▏        | 26225/225000 [61:34:16<417:23:25,  7.56s/it]
 12%|█▏        | 26226/225000 [61:34:20<359:35:36,  6.51s/it]
 12%|█▏        | 26227/225000 [61:34:24<326:42:30,  5.92s/it]
 12%|█▏        | 26228/225000 [61:34:34<396:03:24,  7.17s/it]
 12%|█▏        | 26229/225000 [61:34:40<370:54:48,  6.72s/it]
 12%|█▏        | 26230/225000 [61:34:45<335:40:00,  6.08s/it]
 12%|█▏        | 26231/225000 [61:34:50<320:03:48,  5.80s/it]
 12%|█▏        | 26232/225000 [61:34:55<304:51:32,  5.52s/it]
 12%|█▏        | 26233/225000 [61:35:04<364:34:43,  6.60s/it]
 12%|█▏        | 26234/225000 [61:35:11<374:28:29,  6.78s/it]
 12%|█▏        | 26235/225000 [61:35:16<344:37:07,  6.24s/it]
 12%|█▏        | 26236/225000 [61:35:25<391:06:25,  7.08s/it]
 12%|█▏        | 26237/225000 [61:35:30<359:01:56,  6.50s/it]
 12%|█▏        | 26238/225000 [61:35:34<321:58:07,  5.83s/it]
 12%|█▏        | 26239/225000 [61:35:39<298:49:04,  5.41s/it]
 12%|█▏        | 26240/225000 [61:35:43<282:30:18,  5.12s/it]
 12%|█▏        | 26241/225000 [61:35:51<321:36:15,  5.83s/it]
 12%|█▏        | 26242/225000 [61:36:00<386:58:59,  7.01s/it]
 12%|█▏        | 26243/225000 [61:36:04<329:43:01,  5.97s/it]
 12%|█▏        | 26244/225000 [61:36:12<371:20:00,  6.73s/it]
 12%|█▏        | 26245/225000 [61:36:21<396:45:55,  7.19s/it]
 12%|█▏        | 26246/225000 [61:36:25<343:52:59,  6.23s/it]
 12%|█▏        | 26247/225000 [61:36:29<303:40:50,  5.50s/it]
 12%|█▏        | 26248/225000 [61:36:33<285:32:53,  5.17s/it]
 12%|█▏        | 26249/225000 [61:36:46<409:42:47,  7.42s/it]
 12%|█▏        | 26250/225000 [61:36:52<392:54:50,  7.12s/it]
                                                             
{'loss': '0.7208', 'grad_norm': '0.708', 'learning_rate': '4.93e-05', 'epoch': '0.1167', 'num_input_tokens_seen': 106240296, 'train_runtime': '2.218e+05', 'train_tokens_per_second': '479'}

 12%|█▏        | 26250/225000 [61:36:52<392:54:50,  7.12s/it]
 12%|█▏        | 26251/225000 [61:36:56<342:10:05,  6.20s/it]
 12%|█▏        | 26252/225000 [61:37:05<391:26:22,  7.09s/it]
 12%|█▏        | 26253/225000 [61:37:12<381:26:23,  6.91s/it]
 12%|█▏        | 26254/225000 [61:37:18<374:50:05,  6.79s/it]
 12%|█▏        | 26255/225000 [61:37:23<347:44:19,  6.30s/it]
 12%|█▏        | 26256/225000 [61:37:28<324:03:09,  5.87s/it]
 12%|█▏        | 26257/225000 [61:37:33<307:12:00,  5.56s/it]
 12%|█▏        | 26258/225000 [61:37:38<290:27:03,  5.26s/it]
 12%|█▏        | 26259/225000 [61:37:51<417:50:19,  7.57s/it]
 12%|█▏        | 26260/225000 [61:37:55<362:03:12,  6.56s/it]
 12%|█▏        | 26261/225000 [61:38:01<352:19:46,  6.38s/it]
 12%|█▏        | 26262/225000 [61:38:08<366:22:49,  6.64s/it]
 12%|█▏        | 26263/225000 [61:38:15<367:27:15,  6.66s/it]
 12%|█▏        | 26264/225000 [61:38:20<337:38:44,  6.12s/it]
 12%|█▏        | 26265/225000 [61:38:24<307:30:35,  5.57s/it]
 12%|█▏        | 26266/225000 [61:38:33<359:32:28,  6.51s/it]
 12%|█▏        | 26267/225000 [61:38:39<364:40:26,  6.61s/it]
 12%|█▏        | 26268/225000 [61:38:47<373:48:28,  6.77s/it]
 12%|█▏        | 26269/225000 [61:38:53<361:01:27,  6.54s/it]
 12%|█▏        | 26270/225000 [61:39:00<372:46:55,  6.75s/it]
 12%|█▏        | 26271/225000 [61:39:04<325:37:06,  5.90s/it]
 12%|█▏        | 26272/225000 [61:39:11<347:03:14,  6.29s/it]
 12%|█▏        | 26273/225000 [61:39:17<341:08:28,  6.18s/it]
 12%|█▏        | 26274/225000 [61:39:22<329:36:40,  5.97s/it]
 12%|█▏        | 26275/225000 [61:39:31<367:57:23,  6.67s/it]
 12%|█▏        | 26276/225000 [61:39:36<342:11:17,  6.20s/it]
 12%|█▏        | 26277/225000 [61:39:44<372:06:05,  6.74s/it]
 12%|█▏        | 26278/225000 [61:39:48<328:51:17,  5.96s/it]
 12%|█▏        | 26279/225000 [61:39:53<311:36:42,  5.65s/it]
 12%|█▏        | 26280/225000 [61:40:04<401:07:35,  7.27s/it]
 12%|█▏        | 26281/225000 [61:40:09<369:30:59,  6.69s/it]
 12%|█▏        | 26282/225000 [61:40:16<378:34:04,  6.86s/it]
 12%|█▏        | 26283/225000 [61:40:21<347:24:32,  6.29s/it]
 12%|█▏        | 26284/225000 [61:40:27<336:16:56,  6.09s/it]
 12%|█▏        | 26285/225000 [61:40:32<319:30:18,  5.79s/it]
 12%|█▏        | 26286/225000 [61:40:38<315:24:13,  5.71s/it]
 12%|█▏        | 26287/225000 [61:40:43<306:49:47,  5.56s/it]
 12%|█▏        | 26288/225000 [61:40:48<308:12:36,  5.58s/it]
 12%|█▏        | 26289/225000 [61:41:02<435:05:35,  7.88s/it]
 12%|█▏        | 26290/225000 [61:41:07<395:58:34,  7.17s/it]
 12%|█▏        | 26291/225000 [61:41:11<346:28:03,  6.28s/it]
 12%|█▏        | 26292/225000 [61:41:22<420:23:14,  7.62s/it]
 12%|█▏        | 26293/225000 [61:41:27<368:55:58,  6.68s/it]
 12%|█▏        | 26294/225000 [61:41:31<333:28:51,  6.04s/it]
 12%|█▏        | 26295/225000 [61:41:37<322:22:29,  5.84s/it]
 12%|█▏        | 26296/225000 [61:41:42<322:53:06,  5.85s/it]
 12%|█▏        | 26297/225000 [61:41:55<426:58:36,  7.74s/it]
 12%|█▏        | 26298/225000 [61:41:59<372:21:24,  6.75s/it]
 12%|█▏        | 26299/225000 [61:42:04<340:11:15,  6.16s/it]
 12%|█▏        | 26300/225000 [61:42:09<315:39:20,  5.72s/it]
                                                             
{'loss': '0.7245', 'grad_norm': '0.7603', 'learning_rate': '4.93e-05', 'epoch': '0.1169', 'num_input_tokens_seen': 106383872, 'train_runtime': '2.221e+05', 'train_tokens_per_second': '478.9'}

 12%|█▏        | 26300/225000 [61:42:09<315:39:20,  5.72s/it]
 12%|█▏        | 26301/225000 [61:42:18<384:29:16,  6.97s/it]
 12%|█▏        | 26302/225000 [61:42:27<418:53:18,  7.59s/it]
 12%|█▏        | 26303/225000 [61:42:31<357:06:17,  6.47s/it]
 12%|█▏        | 26304/225000 [61:42:38<358:16:26,  6.49s/it]
 12%|█▏        | 26305/225000 [61:42:46<381:20:42,  6.91s/it]
 12%|█▏        | 26306/225000 [61:42:50<344:37:08,  6.24s/it]
 12%|█▏        | 26307/225000 [61:42:55<321:09:55,  5.82s/it]
 12%|█▏        | 26308/225000 [61:43:01<319:30:01,  5.79s/it]
 12%|█▏        | 26309/225000 [61:43:06<306:08:37,  5.55s/it]
 12%|█▏        | 26310/225000 [61:43:11<301:52:08,  5.47s/it]
 12%|█▏        | 26311/225000 [61:43:16<294:02:26,  5.33s/it]
 12%|█▏        | 26312/225000 [61:43:21<292:43:08,  5.30s/it]
 12%|█▏        | 26313/225000 [61:43:27<296:34:42,  5.37s/it]
 12%|█▏        | 26314/225000 [61:43:39<413:13:16,  7.49s/it]
 12%|█▏        | 26315/225000 [61:43:46<399:57:52,  7.25s/it]
 12%|█▏        | 26316/225000 [61:43:50<346:47:53,  6.28s/it]
 12%|█▏        | 26317/225000 [61:44:00<413:11:50,  7.49s/it]
 12%|█▏        | 26318/225000 [61:44:05<357:53:17,  6.48s/it]
 12%|█▏        | 26319/225000 [61:44:09<329:42:11,  5.97s/it]
 12%|█▏        | 26320/225000 [61:44:14<302:21:04,  5.48s/it]
 12%|█▏        | 26321/225000 [61:44:18<284:02:17,  5.15s/it]
 12%|█▏        | 26322/225000 [61:44:23<282:22:52,  5.12s/it]
 12%|█▏        | 26323/225000 [61:44:28<284:35:10,  5.16s/it]
 12%|█▏        | 26324/225000 [61:44:34<293:11:18,  5.31s/it]
 12%|█▏        | 26325/225000 [61:44:44<366:09:15,  6.63s/it]
 12%|█▏        | 26326/225000 [61:44:57<477:21:08,  8.65s/it]
 12%|█▏        | 26327/225000 [61:45:01<393:13:45,  7.13s/it]
 12%|█▏        | 26328/225000 [61:45:06<364:33:13,  6.61s/it]
 12%|█▏        | 26329/225000 [61:45:12<350:50:26,  6.36s/it]
 12%|█▏        | 26330/225000 [61:45:18<339:44:12,  6.16s/it]
 12%|█▏        | 26331/225000 [61:45:25<365:57:00,  6.63s/it]
 12%|█▏        | 26332/225000 [61:45:31<343:32:48,  6.23s/it]
 12%|█▏        | 26333/225000 [61:45:36<326:53:15,  5.92s/it]
 12%|█▏        | 26334/225000 [61:45:41<317:27:09,  5.75s/it]
 12%|█▏        | 26335/225000 [61:45:47<312:15:58,  5.66s/it]
 12%|█▏        | 26336/225000 [61:45:53<325:11:27,  5.89s/it]
 12%|█▏        | 26337/225000 [61:46:02<374:36:11,  6.79s/it]
 12%|█▏        | 26338/225000 [61:46:07<354:21:57,  6.42s/it]
 12%|█▏        | 26339/225000 [61:46:13<339:46:30,  6.16s/it]
 12%|█▏        | 26340/225000 [61:46:18<314:40:39,  5.70s/it]
 12%|█▏        | 26341/225000 [61:46:23<306:09:20,  5.55s/it]
 12%|█▏        | 26342/225000 [61:46:29<312:46:03,  5.67s/it]
 12%|█▏        | 26343/225000 [61:46:42<442:26:31,  8.02s/it]
 12%|█▏        | 26344/225000 [61:46:47<381:11:16,  6.91s/it]
 12%|█▏        | 26345/225000 [61:46:51<338:26:03,  6.13s/it]
 12%|█▏        | 26346/225000 [61:46:56<323:08:40,  5.86s/it]
 12%|█▏        | 26347/225000 [61:47:08<416:10:24,  7.54s/it]
 12%|█▏        | 26348/225000 [61:47:12<365:21:37,  6.62s/it]
 12%|█▏        | 26349/225000 [61:47:18<352:23:37,  6.39s/it]
 12%|█▏        | 26350/225000 [61:47:26<382:54:39,  6.94s/it]
                                                             
{'loss': '0.7076', 'grad_norm': '0.6495', 'learning_rate': '4.929e-05', 'epoch': '0.1171', 'num_input_tokens_seen': 106524056, 'train_runtime': '2.224e+05', 'train_tokens_per_second': '478.9'}

 12%|█▏        | 26350/225000 [61:47:26<382:54:39,  6.94s/it]
 12%|█▏        | 26351/225000 [61:47:31<350:32:53,  6.35s/it]
 12%|█▏        | 26352/225000 [61:47:36<325:33:31,  5.90s/it]
 12%|█▏        | 26353/225000 [61:47:41<309:38:21,  5.61s/it]
 12%|█▏        | 26354/225000 [61:47:52<396:42:26,  7.19s/it]
 12%|█▏        | 26355/225000 [61:47:58<380:45:55,  6.90s/it]
 12%|█▏        | 26356/225000 [61:48:05<383:40:02,  6.95s/it]
 12%|█▏        | 26357/225000 [61:48:11<367:53:11,  6.67s/it]
 12%|█▏        | 26358/225000 [61:48:19<389:32:48,  7.06s/it]
 12%|█▏        | 26359/225000 [61:48:24<346:50:31,  6.29s/it]
 12%|█▏        | 26360/225000 [61:48:27<307:32:44,  5.57s/it]
 12%|█▏        | 26361/225000 [61:48:36<363:51:27,  6.59s/it]
 12%|█▏        | 26362/225000 [61:48:42<355:10:04,  6.44s/it]
 12%|█▏        | 26363/225000 [61:48:47<330:32:29,  5.99s/it]
 12%|█▏        | 26364/225000 [61:48:53<318:02:13,  5.76s/it]
 12%|█▏        | 26365/225000 [61:48:58<308:34:37,  5.59s/it]
 12%|█▏        | 26366/225000 [61:49:02<291:00:15,  5.27s/it]
 12%|█▏        | 26367/225000 [61:49:08<288:19:21,  5.23s/it]
 12%|█▏        | 26368/225000 [61:49:14<302:45:30,  5.49s/it]
 12%|█▏        | 26369/225000 [61:49:19<296:10:04,  5.37s/it]
 12%|█▏        | 26370/225000 [61:49:31<416:03:54,  7.54s/it]
 12%|█▏        | 26371/225000 [61:49:40<426:48:13,  7.74s/it]
 12%|█▏        | 26372/225000 [61:49:45<386:32:30,  7.01s/it]
 12%|█▏        | 26373/225000 [61:49:48<331:16:35,  6.00s/it]
 12%|█▏        | 26374/225000 [61:49:55<337:10:14,  6.11s/it]
 12%|█▏        | 26375/225000 [61:50:05<397:59:34,  7.21s/it]
 12%|█▏        | 26376/225000 [61:50:11<392:24:31,  7.11s/it]
 12%|█▏        | 26377/225000 [61:50:15<333:23:31,  6.04s/it]
 12%|█▏        | 26378/225000 [61:50:25<401:03:01,  7.27s/it]
 12%|█▏        | 26379/225000 [61:50:29<351:19:51,  6.37s/it]
 12%|█▏        | 26380/225000 [61:50:35<333:38:05,  6.05s/it]
 12%|█▏        | 26381/225000 [61:50:40<325:20:37,  5.90s/it]
 12%|█▏        | 26382/225000 [61:50:46<324:18:26,  5.88s/it]
 12%|█▏        | 26383/225000 [61:50:52<317:22:07,  5.75s/it]
 12%|█▏        | 26384/225000 [61:50:57<309:58:12,  5.62s/it]
 12%|█▏        | 26385/225000 [61:51:02<307:42:02,  5.58s/it]
 12%|█▏        | 26386/225000 [61:51:07<298:32:08,  5.41s/it]
 12%|█▏        | 26387/225000 [61:51:18<379:44:57,  6.88s/it]
 12%|█▏        | 26388/225000 [61:51:26<398:29:52,  7.22s/it]
 12%|█▏        | 26389/225000 [61:51:30<352:08:35,  6.38s/it]
 12%|█▏        | 26390/225000 [61:51:34<310:25:41,  5.63s/it]
 12%|█▏        | 26391/225000 [61:51:40<320:09:15,  5.80s/it]
 12%|█▏        | 26392/225000 [61:51:54<450:17:10,  8.16s/it]
 12%|█▏        | 26393/225000 [61:52:00<412:45:51,  7.48s/it]
 12%|█▏        | 26394/225000 [61:52:04<366:14:29,  6.64s/it]
 12%|█▏        | 26395/225000 [61:52:10<355:54:12,  6.45s/it]
 12%|█▏        | 26396/225000 [61:52:19<396:24:23,  7.19s/it]
 12%|█▏        | 26397/225000 [61:52:28<414:21:58,  7.51s/it]
 12%|█▏        | 26398/225000 [61:52:32<362:14:27,  6.57s/it]
 12%|█▏        | 26399/225000 [61:52:37<332:45:31,  6.03s/it]
 12%|█▏        | 26400/225000 [61:52:46<382:42:15,  6.94s/it]
                                                             
{'loss': '0.7252', 'grad_norm': '0.6125', 'learning_rate': '4.929e-05', 'epoch': '0.1173', 'num_input_tokens_seen': 106665104, 'train_runtime': '2.228e+05', 'train_tokens_per_second': '478.8'}

 12%|█▏        | 26400/225000 [61:52:46<382:42:15,  6.94s/it]
 12%|█▏        | 26401/225000 [61:52:51<360:03:25,  6.53s/it]
 12%|█▏        | 26402/225000 [61:52:57<341:46:32,  6.20s/it]
 12%|█▏        | 26403/225000 [61:53:05<377:24:42,  6.84s/it]
 12%|█▏        | 26404/225000 [61:53:11<352:35:33,  6.39s/it]
 12%|█▏        | 26405/225000 [61:53:16<342:35:53,  6.21s/it]
 12%|█▏        | 26406/225000 [61:53:26<400:31:25,  7.26s/it]
 12%|█▏        | 26407/225000 [61:53:31<361:09:21,  6.55s/it]
 12%|█▏        | 26408/225000 [61:53:36<333:52:38,  6.05s/it]
 12%|█▏        | 26409/225000 [61:53:41<317:59:42,  5.76s/it]
 12%|█▏        | 26410/225000 [61:53:53<417:51:25,  7.57s/it]
 12%|█▏        | 26411/225000 [61:53:59<392:30:28,  7.12s/it]
 12%|█▏        | 26412/225000 [61:54:03<347:45:38,  6.30s/it]
 12%|█▏        | 26413/225000 [61:54:10<349:08:17,  6.33s/it]
 12%|█▏        | 26414/225000 [61:54:14<310:30:51,  5.63s/it]
 12%|█▏        | 26415/225000 [61:54:24<394:27:18,  7.15s/it]
 12%|█▏        | 26416/225000 [61:54:31<389:49:14,  7.07s/it]
 12%|█▏        | 26417/225000 [61:54:36<352:52:22,  6.40s/it]
 12%|█▏        | 26418/225000 [61:54:40<319:23:47,  5.79s/it]
 12%|█▏        | 26419/225000 [61:54:45<302:25:33,  5.48s/it]
 12%|█▏        | 26420/225000 [61:54:55<376:50:20,  6.83s/it]
 12%|█▏        | 26421/225000 [61:55:01<363:03:57,  6.58s/it]
 12%|█▏        | 26422/225000 [61:55:05<323:22:18,  5.86s/it]
 12%|█▏        | 26423/225000 [61:55:09<294:59:23,  5.35s/it]
 12%|█▏        | 26424/225000 [61:55:17<338:56:21,  6.14s/it]
 12%|█▏        | 26425/225000 [61:55:27<393:01:26,  7.13s/it]
 12%|█▏        | 26426/225000 [61:55:31<342:14:25,  6.20s/it]
 12%|█▏        | 26427/225000 [61:55:35<309:35:46,  5.61s/it]
 12%|█▏        | 26428/225000 [61:55:46<398:42:16,  7.23s/it]
 12%|█▏        | 26429/225000 [61:55:50<336:35:36,  6.10s/it]
 12%|█▏        | 26430/225000 [61:55:54<310:52:57,  5.64s/it]
 12%|█▏        | 26431/225000 [61:55:59<301:49:22,  5.47s/it]
 12%|█▏        | 26432/225000 [61:56:04<297:27:58,  5.39s/it]
 12%|█▏        | 26433/225000 [61:56:10<294:49:14,  5.35s/it]
 12%|█▏        | 26434/225000 [61:56:15<294:35:52,  5.34s/it]
 12%|█▏        | 26435/225000 [61:56:28<422:32:06,  7.66s/it]
 12%|█▏        | 26436/225000 [61:56:33<383:27:10,  6.95s/it]
 12%|█▏        | 26437/225000 [61:56:37<324:00:10,  5.87s/it]
 12%|█▏        | 26438/225000 [61:56:42<313:49:36,  5.69s/it]
 12%|█▏        | 26439/225000 [61:56:51<374:19:51,  6.79s/it]
 12%|█▏        | 26440/225000 [61:56:58<379:20:13,  6.88s/it]
 12%|█▏        | 26441/225000 [61:57:02<329:20:58,  5.97s/it]
 12%|█▏        | 26442/225000 [61:57:07<303:47:50,  5.51s/it]
 12%|█▏        | 26443/225000 [61:57:15<342:35:22,  6.21s/it]
 12%|█▏        | 26444/225000 [61:57:24<403:00:40,  7.31s/it]
 12%|█▏        | 26445/225000 [61:57:28<344:20:52,  6.24s/it]
 12%|█▏        | 26446/225000 [61:57:37<391:56:55,  7.11s/it]
 12%|█▏        | 26447/225000 [61:57:43<370:47:26,  6.72s/it]
 12%|█▏        | 26448/225000 [61:57:49<352:48:54,  6.40s/it]
 12%|█▏        | 26449/225000 [61:57:53<315:40:43,  5.72s/it]
 12%|█▏        | 26450/225000 [61:58:04<397:38:18,  7.21s/it]
                                                             
{'loss': '0.6997', 'grad_norm': '0.7359', 'learning_rate': '4.928e-05', 'epoch': '0.1176', 'num_input_tokens_seen': 106807016, 'train_runtime': '2.231e+05', 'train_tokens_per_second': '478.8'}

 12%|█▏        | 26450/225000 [61:58:04<397:38:18,  7.21s/it]
 12%|█▏        | 26451/225000 [61:58:07<339:36:21,  6.16s/it]
 12%|█▏        | 26452/225000 [61:58:15<371:06:02,  6.73s/it]
 12%|█▏        | 26453/225000 [61:58:24<401:35:01,  7.28s/it]
 12%|█▏        | 26454/225000 [61:58:28<344:15:18,  6.24s/it]
 12%|█▏        | 26455/225000 [61:58:34<347:15:37,  6.30s/it]
 12%|█▏        | 26456/225000 [61:58:38<312:32:26,  5.67s/it]
 12%|█▏        | 26457/225000 [61:58:44<305:23:24,  5.54s/it]
 12%|█▏        | 26458/225000 [61:58:54<379:18:46,  6.88s/it]
 12%|█▏        | 26459/225000 [61:58:58<343:18:40,  6.23s/it]
 12%|█▏        | 26460/225000 [61:59:03<324:05:41,  5.88s/it]
 12%|█▏        | 26461/225000 [61:59:08<301:59:19,  5.48s/it]
 12%|█▏        | 26462/225000 [61:59:13<297:16:04,  5.39s/it]
 12%|█▏        | 26463/225000 [61:59:24<396:48:32,  7.20s/it]
 12%|█▏        | 26464/225000 [61:59:31<378:28:41,  6.86s/it]
 12%|█▏        | 26465/225000 [61:59:38<386:01:03,  7.00s/it]
 12%|█▏        | 26466/225000 [61:59:44<374:42:57,  6.79s/it]
 12%|█▏        | 26467/225000 [61:59:51<382:49:39,  6.94s/it]
 12%|█▏        | 26468/225000 [61:59:57<356:54:42,  6.47s/it]
 12%|█▏        | 26469/225000 [62:00:01<318:50:18,  5.78s/it]
 12%|█▏        | 26470/225000 [62:00:07<319:48:09,  5.80s/it]
 12%|█▏        | 26471/225000 [62:00:12<316:28:15,  5.74s/it]
 12%|█▏        | 26472/225000 [62:00:17<300:52:29,  5.46s/it]
 12%|█▏        | 26473/225000 [62:00:23<302:48:07,  5.49s/it]
 12%|█▏        | 26474/225000 [62:00:36<431:16:14,  7.82s/it]
 12%|█▏        | 26475/225000 [62:00:41<377:33:00,  6.85s/it]
 12%|█▏        | 26476/225000 [62:00:46<357:23:58,  6.48s/it]
 12%|█▏        | 26477/225000 [62:00:52<341:30:00,  6.19s/it]
 12%|█▏        | 26478/225000 [62:00:57<323:13:53,  5.86s/it]
 12%|█▏        | 26479/225000 [62:01:02<310:40:24,  5.63s/it]
 12%|█▏        | 26480/225000 [62:01:09<327:26:56,  5.94s/it]
 12%|█▏        | 26481/225000 [62:01:22<453:34:41,  8.23s/it]
 12%|█▏        | 26482/225000 [62:01:26<386:20:04,  7.01s/it]
 12%|█▏        | 26483/225000 [62:01:33<384:20:33,  6.97s/it]
 12%|█▏        | 26484/225000 [62:01:40<377:38:46,  6.85s/it]
 12%|█▏        | 26485/225000 [62:01:45<341:38:25,  6.20s/it]
 12%|█▏        | 26486/225000 [62:01:52<360:30:31,  6.54s/it]
 12%|█▏        | 26487/225000 [62:02:00<389:39:51,  7.07s/it]
 12%|█▏        | 26488/225000 [62:02:04<338:48:52,  6.14s/it]
 12%|█▏        | 26489/225000 [62:02:15<418:57:44,  7.60s/it]
 12%|█▏        | 26490/225000 [62:02:19<364:05:15,  6.60s/it]
 12%|█▏        | 26491/225000 [62:02:25<351:26:57,  6.37s/it]
 12%|█▏        | 26492/225000 [62:02:35<401:02:01,  7.27s/it]
 12%|█▏        | 26493/225000 [62:02:39<348:18:43,  6.32s/it]
 12%|█▏        | 26494/225000 [62:02:47<374:37:50,  6.79s/it]
 12%|█▏        | 26495/225000 [62:02:52<349:15:22,  6.33s/it]
 12%|█▏        | 26496/225000 [62:02:59<362:28:56,  6.57s/it]
 12%|█▏        | 26497/225000 [62:03:03<315:42:57,  5.73s/it]
 12%|█▏        | 26498/225000 [62:03:12<370:43:32,  6.72s/it]
 12%|█▏        | 26499/225000 [62:03:19<371:07:58,  6.73s/it]
 12%|█▏        | 26500/225000 [62:03:23<336:43:08,  6.11s/it]
                                                             
{'loss': '0.7117', 'grad_norm': '0.6722', 'learning_rate': '4.928e-05', 'epoch': '0.1178', 'num_input_tokens_seen': 106951376, 'train_runtime': '2.234e+05', 'train_tokens_per_second': '478.7'}

 12%|█▏        | 26500/225000 [62:03:23<336:43:08,  6.11s/it]
 12%|█▏        | 26501/225000 [62:03:30<345:54:41,  6.27s/it]
 12%|█▏        | 26502/225000 [62:03:34<303:34:04,  5.51s/it]
 12%|█▏        | 26503/225000 [62:03:45<400:20:56,  7.26s/it]
 12%|█▏        | 26504/225000 [62:03:48<336:59:14,  6.11s/it]
 12%|█▏        | 26505/225000 [62:03:56<368:28:14,  6.68s/it]
 12%|█▏        | 26506/225000 [62:04:04<381:34:46,  6.92s/it]
 12%|█▏        | 26507/225000 [62:04:08<333:46:24,  6.05s/it]
 12%|█▏        | 26508/225000 [62:04:19<418:58:24,  7.60s/it]
 12%|█▏        | 26509/225000 [62:04:23<354:10:20,  6.42s/it]
 12%|█▏        | 26510/225000 [62:04:28<341:09:56,  6.19s/it]
 12%|█▏        | 26511/225000 [62:04:38<396:28:03,  7.19s/it]
 12%|█▏        | 26512/225000 [62:04:42<343:19:51,  6.23s/it]
 12%|█▏        | 26513/225000 [62:04:51<383:29:25,  6.96s/it]
 12%|█▏        | 26514/225000 [62:04:56<349:52:55,  6.35s/it]
 12%|█▏        | 26515/225000 [62:05:01<327:49:37,  5.95s/it]
 12%|█▏        | 26516/225000 [62:05:05<304:25:58,  5.52s/it]
 12%|█▏        | 26517/225000 [62:05:12<328:49:43,  5.96s/it]
 12%|█▏        | 26518/225000 [62:05:23<411:10:02,  7.46s/it]
 12%|█▏        | 26519/225000 [62:05:27<354:31:45,  6.43s/it]
 12%|█▏        | 26520/225000 [62:05:31<313:23:28,  5.68s/it]
 12%|█▏        | 26521/225000 [62:05:40<371:26:30,  6.74s/it]
 12%|█▏        | 26522/225000 [62:05:46<357:18:21,  6.48s/it]
 12%|█▏        | 26523/225000 [62:05:50<318:55:32,  5.78s/it]
 12%|█▏        | 26524/225000 [62:06:01<394:44:01,  7.16s/it]
 12%|█▏        | 26525/225000 [62:06:06<363:20:46,  6.59s/it]
 12%|█▏        | 26526/225000 [62:06:13<365:22:58,  6.63s/it]
 12%|█▏        | 26527/225000 [62:06:16<315:36:39,  5.72s/it]
 12%|█▏        | 26528/225000 [62:06:24<347:41:31,  6.31s/it]
 12%|█▏        | 26529/225000 [62:06:30<343:59:29,  6.24s/it]
 12%|█▏        | 26530/225000 [62:06:34<303:31:58,  5.51s/it]
 12%|█▏        | 26531/225000 [62:06:38<287:42:50,  5.22s/it]
 12%|█▏        | 26532/225000 [62:06:47<338:50:49,  6.15s/it]
 12%|█▏        | 26533/225000 [62:06:52<319:23:54,  5.79s/it]
 12%|█▏        | 26534/225000 [62:06:58<337:07:06,  6.12s/it]
 12%|█▏        | 26535/225000 [62:07:10<420:11:43,  7.62s/it]
 12%|█▏        | 26536/225000 [62:07:15<383:35:51,  6.96s/it]
 12%|█▏        | 26537/225000 [62:07:20<355:00:47,  6.44s/it]
 12%|█▏        | 26538/225000 [62:07:28<385:27:38,  6.99s/it]
 12%|█▏        | 26539/225000 [62:07:34<366:51:57,  6.65s/it]
 12%|█▏        | 26540/225000 [62:07:43<402:10:11,  7.30s/it]
 12%|█▏        | 26541/225000 [62:07:47<352:36:41,  6.40s/it]
 12%|█▏        | 26542/225000 [62:07:51<314:15:50,  5.70s/it]
 12%|█▏        | 26543/225000 [62:08:02<392:02:45,  7.11s/it]
 12%|█▏        | 26544/225000 [62:08:05<330:01:22,  5.99s/it]
 12%|█▏        | 26545/225000 [62:08:14<375:17:12,  6.81s/it]
 12%|█▏        | 26546/225000 [62:08:19<338:22:55,  6.14s/it]
 12%|█▏        | 26547/225000 [62:08:27<378:28:15,  6.87s/it]
 12%|█▏        | 26548/225000 [62:08:32<339:44:09,  6.16s/it]
 12%|█▏        | 26549/225000 [62:08:36<306:38:22,  5.56s/it]
 12%|█▏        | 26550/225000 [62:08:47<393:48:06,  7.14s/it]
                                                             
{'loss': '0.7217', 'grad_norm': '0.7102', 'learning_rate': '4.927e-05', 'epoch': '0.118', 'num_input_tokens_seen': 107092936, 'train_runtime': '2.237e+05', 'train_tokens_per_second': '478.7'}

 12%|█▏        | 26550/225000 [62:08:47<393:48:06,  7.14s/it]
 12%|█▏        | 26551/225000 [62:08:51<340:12:05,  6.17s/it]
 12%|█▏        | 26552/225000 [62:08:55<306:59:00,  5.57s/it]
 12%|█▏        | 26553/225000 [62:09:06<403:09:05,  7.31s/it]
 12%|█▏        | 26554/225000 [62:09:12<371:44:38,  6.74s/it]
 12%|█▏        | 26555/225000 [62:09:17<353:25:47,  6.41s/it]
 12%|█▏        | 26556/225000 [62:09:22<332:00:33,  6.02s/it]
 12%|█▏        | 26557/225000 [62:09:28<325:38:08,  5.91s/it]
 12%|█▏        | 26558/225000 [62:09:33<311:54:12,  5.66s/it]
 12%|█▏        | 26559/225000 [62:09:38<305:35:01,  5.54s/it]
 12%|█▏        | 26560/225000 [62:09:44<314:51:19,  5.71s/it]
 12%|█▏        | 26561/225000 [62:09:53<356:15:56,  6.46s/it]
 12%|█▏        | 26562/225000 [62:10:00<371:34:07,  6.74s/it]
 12%|█▏        | 26563/225000 [62:10:06<359:39:08,  6.52s/it]
 12%|█▏        | 26564/225000 [62:10:11<341:31:53,  6.20s/it]
 12%|█▏        | 26565/225000 [62:10:22<418:33:31,  7.59s/it]
 12%|█▏        | 26566/225000 [62:10:31<429:46:57,  7.80s/it]
 12%|█▏        | 26567/225000 [62:10:35<371:51:07,  6.75s/it]
 12%|█▏        | 26568/225000 [62:10:45<421:47:34,  7.65s/it]
 12%|█▏        | 26569/225000 [62:10:49<366:46:30,  6.65s/it]
 12%|█▏        | 26570/225000 [62:10:53<327:38:36,  5.94s/it]
 12%|█▏        | 26571/225000 [62:11:00<334:16:49,  6.06s/it]
 12%|█▏        | 26572/225000 [62:11:10<409:01:16,  7.42s/it]
 12%|█▏        | 26573/225000 [62:11:14<345:45:16,  6.27s/it]
 12%|█▏        | 26574/225000 [62:11:22<384:10:55,  6.97s/it]
 12%|█▏        | 26575/225000 [62:11:27<343:27:08,  6.23s/it]
 12%|█▏        | 26576/225000 [62:11:32<320:06:42,  5.81s/it]
 12%|█▏        | 26577/225000 [62:11:37<304:45:58,  5.53s/it]
 12%|█▏        | 26578/225000 [62:11:46<364:46:27,  6.62s/it]
 12%|█▏        | 26579/225000 [62:11:55<415:37:56,  7.54s/it]
 12%|█▏        | 26580/225000 [62:12:01<380:29:48,  6.90s/it]
 12%|█▏        | 26581/225000 [62:12:08<385:15:47,  6.99s/it]
 12%|█▏        | 26582/225000 [62:12:14<366:06:28,  6.64s/it]
 12%|█▏        | 26583/225000 [62:12:20<359:54:02,  6.53s/it]
 12%|█▏        | 26584/225000 [62:12:26<355:34:34,  6.45s/it]
 12%|█▏        | 26585/225000 [62:12:35<390:46:03,  7.09s/it]
 12%|█▏        | 26586/225000 [62:12:38<332:07:18,  6.03s/it]
 12%|█▏        | 26587/225000 [62:12:47<369:34:50,  6.71s/it]
 12%|█▏        | 26588/225000 [62:12:50<315:21:36,  5.72s/it]
 12%|█▏        | 26589/225000 [62:13:00<384:48:34,  6.98s/it]
 12%|█▏        | 26590/225000 [62:13:05<349:14:56,  6.34s/it]
 12%|█▏        | 26591/225000 [62:13:10<324:09:56,  5.88s/it]
 12%|█▏        | 26592/225000 [62:13:15<313:52:15,  5.70s/it]
 12%|█▏        | 26593/225000 [62:13:20<298:22:50,  5.41s/it]
 12%|█▏        | 26594/225000 [62:13:25<300:37:44,  5.45s/it]
 12%|█▏        | 26595/225000 [62:13:38<427:27:00,  7.76s/it]
 12%|█▏        | 26596/225000 [62:13:43<374:27:16,  6.79s/it]
 12%|█▏        | 26597/225000 [62:13:48<347:11:55,  6.30s/it]
 12%|█▏        | 26598/225000 [62:13:57<388:58:09,  7.06s/it]
 12%|█▏        | 26599/225000 [62:14:00<328:18:09,  5.96s/it]
 12%|█▏        | 26600/225000 [62:14:10<386:42:30,  7.02s/it]
                                                             
{'loss': '0.7198', 'grad_norm': '0.6893', 'learning_rate': '4.927e-05', 'epoch': '0.1182', 'num_input_tokens_seen': 107235704, 'train_runtime': '2.241e+05', 'train_tokens_per_second': '478.6'}

 12%|█▏        | 26600/225000 [62:14:10<386:42:30,  7.02s/it]
 12%|█▏        | 26601/225000 [62:14:14<345:32:47,  6.27s/it]
 12%|█▏        | 26602/225000 [62:14:19<312:52:07,  5.68s/it]
 12%|█▏        | 26603/225000 [62:14:24<302:03:56,  5.48s/it]
 12%|█▏        | 26604/225000 [62:14:29<295:19:00,  5.36s/it]
 12%|█▏        | 26605/225000 [62:14:43<433:32:55,  7.87s/it]
 12%|█▏        | 26606/225000 [62:14:49<416:58:44,  7.57s/it]
 12%|█▏        | 26607/225000 [62:14:54<367:11:16,  6.66s/it]
 12%|█▏        | 26608/225000 [62:14:59<340:57:28,  6.19s/it]
 12%|█▏        | 26609/225000 [62:15:08<384:43:36,  6.98s/it]
 12%|█▏        | 26610/225000 [62:15:16<408:45:49,  7.42s/it]
 12%|█▏        | 26611/225000 [62:15:22<386:00:53,  7.00s/it]
 12%|█▏        | 26612/225000 [62:15:30<393:35:06,  7.14s/it]
 12%|█▏        | 26613/225000 [62:15:35<363:21:27,  6.59s/it]
 12%|█▏        | 26614/225000 [62:15:41<348:23:18,  6.32s/it]
 12%|█▏        | 26615/225000 [62:15:46<326:59:09,  5.93s/it]
 12%|█▏        | 26616/225000 [62:15:55<386:21:47,  7.01s/it]
 12%|█▏        | 26617/225000 [62:16:00<351:26:36,  6.38s/it]
 12%|█▏        | 26618/225000 [62:16:05<319:46:47,  5.80s/it]
 12%|█▏        | 26619/225000 [62:16:11<334:16:16,  6.07s/it]
 12%|█▏        | 26620/225000 [62:16:22<410:53:09,  7.46s/it]
 12%|█▏        | 26621/225000 [62:16:26<344:39:32,  6.25s/it]
 12%|█▏        | 26622/225000 [62:16:34<389:21:42,  7.07s/it]
 12%|█▏        | 26623/225000 [62:16:39<344:32:38,  6.25s/it]
 12%|█▏        | 26624/225000 [62:16:44<323:00:55,  5.86s/it]
 12%|█▏        | 26625/225000 [62:16:50<322:53:55,  5.86s/it]
 12%|█▏        | 26626/225000 [62:16:55<314:51:47,  5.71s/it]
 12%|█▏        | 26627/225000 [62:17:01<315:04:50,  5.72s/it]
 12%|█▏        | 26628/225000 [62:17:05<298:24:17,  5.42s/it]
 12%|█▏        | 26629/225000 [62:17:11<298:35:26,  5.42s/it]
 12%|█▏        | 26630/225000 [62:17:16<297:05:20,  5.39s/it]
 12%|█▏        | 26631/225000 [62:17:23<315:56:07,  5.73s/it]
 12%|█▏        | 26632/225000 [62:17:34<400:01:29,  7.26s/it]
 12%|█▏        | 26633/225000 [62:17:42<424:32:01,  7.70s/it]
 12%|█▏        | 26634/225000 [62:17:48<395:30:10,  7.18s/it]
 12%|█▏        | 26635/225000 [62:17:52<346:48:27,  6.29s/it]
 12%|█▏        | 26636/225000 [62:18:02<408:11:13,  7.41s/it]
 12%|█▏        | 26637/225000 [62:18:07<358:55:01,  6.51s/it]
 12%|█▏        | 26638/225000 [62:18:12<332:42:50,  6.04s/it]
 12%|█▏        | 26639/225000 [62:18:17<316:50:44,  5.75s/it]
 12%|█▏        | 26640/225000 [62:18:22<303:17:09,  5.50s/it]
 12%|█▏        | 26641/225000 [62:18:33<388:26:00,  7.05s/it]
 12%|█▏        | 26642/225000 [62:18:40<387:30:16,  7.03s/it]
 12%|█▏        | 26643/225000 [62:18:44<342:05:48,  6.21s/it]
 12%|█▏        | 26644/225000 [62:18:51<356:16:27,  6.47s/it]
 12%|█▏        | 26645/225000 [62:18:58<362:41:43,  6.58s/it]
 12%|█▏        | 26646/225000 [62:19:02<320:33:16,  5.82s/it]
 12%|█▏        | 26647/225000 [62:19:13<409:30:21,  7.43s/it]
 12%|█▏        | 26648/225000 [62:19:17<357:08:01,  6.48s/it]
 12%|█▏        | 26649/225000 [62:19:23<350:29:56,  6.36s/it]
 12%|█▏        | 26650/225000 [62:19:28<327:29:21,  5.94s/it]
                                                             
{'loss': '0.7129', 'grad_norm': '0.7276', 'learning_rate': '4.926e-05', 'epoch': '0.1184', 'num_input_tokens_seen': 107376120, 'train_runtime': '2.244e+05', 'train_tokens_per_second': '478.6'}

 12%|█▏        | 26650/225000 [62:19:28<327:29:21,  5.94s/it]
 12%|█▏        | 26651/225000 [62:19:33<310:48:05,  5.64s/it]
 12%|█▏        | 26652/225000 [62:19:38<291:04:36,  5.28s/it]
 12%|█▏        | 26653/225000 [62:19:43<286:37:55,  5.20s/it]
 12%|█▏        | 26654/225000 [62:19:55<411:49:38,  7.47s/it]
 12%|█▏        | 26655/225000 [62:20:03<407:03:59,  7.39s/it]
 12%|█▏        | 26656/225000 [62:20:10<410:58:30,  7.46s/it]
 12%|█▏        | 26657/225000 [62:20:14<343:20:40,  6.23s/it]
 12%|█▏        | 26658/225000 [62:20:20<340:36:55,  6.18s/it]
 12%|█▏        | 26659/225000 [62:20:27<356:54:35,  6.48s/it]
 12%|█▏        | 26660/225000 [62:20:33<359:47:02,  6.53s/it]
 12%|█▏        | 26661/225000 [62:20:41<373:14:28,  6.77s/it]
 12%|█▏        | 26662/225000 [62:20:45<325:10:09,  5.90s/it]
 12%|█▏        | 26663/225000 [62:20:49<303:54:40,  5.52s/it]
 12%|█▏        | 26664/225000 [62:20:59<373:17:38,  6.78s/it]
 12%|█▏        | 26665/225000 [62:21:05<367:06:51,  6.66s/it]
 12%|█▏        | 26666/225000 [62:21:12<359:47:56,  6.53s/it]
 12%|█▏        | 26667/225000 [62:21:16<319:44:35,  5.80s/it]
 12%|█▏        | 26668/225000 [62:21:25<380:21:27,  6.90s/it]
 12%|█▏        | 26669/225000 [62:21:29<323:01:30,  5.86s/it]
 12%|█▏        | 26670/225000 [62:21:38<388:06:26,  7.04s/it]
 12%|█▏        | 26671/225000 [62:21:42<333:21:26,  6.05s/it]
 12%|█▏        | 26672/225000 [62:21:52<394:26:32,  7.16s/it]
 12%|█▏        | 26673/225000 [62:21:59<397:27:26,  7.21s/it]
 12%|█▏        | 26674/225000 [62:22:03<341:34:15,  6.20s/it]
 12%|█▏        | 26675/225000 [62:22:08<318:22:59,  5.78s/it]
 12%|█▏        | 26676/225000 [62:22:17<376:59:04,  6.84s/it]
 12%|█▏        | 26677/225000 [62:22:23<351:41:52,  6.38s/it]
 12%|█▏        | 26678/225000 [62:22:27<325:47:16,  5.91s/it]
 12%|█▏        | 26679/225000 [62:22:33<318:56:40,  5.79s/it]
 12%|█▏        | 26680/225000 [62:22:38<313:24:14,  5.69s/it]
 12%|█▏        | 26681/225000 [62:22:44<310:59:03,  5.65s/it]
 12%|█▏        | 26682/225000 [62:22:56<422:52:19,  7.68s/it]
 12%|█▏        | 26683/225000 [62:23:03<409:29:57,  7.43s/it]
 12%|█▏        | 26684/225000 [62:23:07<356:12:15,  6.47s/it]
 12%|█▏        | 26685/225000 [62:23:12<331:37:21,  6.02s/it]
 12%|█▏        | 26686/225000 [62:23:17<305:46:26,  5.55s/it]
 12%|█▏        | 26687/225000 [62:23:27<387:49:00,  7.04s/it]
 12%|█▏        | 26688/225000 [62:23:37<439:20:43,  7.98s/it]
 12%|█▏        | 26689/225000 [62:23:42<380:41:35,  6.91s/it]
 12%|█▏        | 26690/225000 [62:23:47<349:37:31,  6.35s/it]
 12%|█▏        | 26691/225000 [62:23:52<326:02:19,  5.92s/it]
 12%|█▏        | 26692/225000 [62:24:05<437:37:35,  7.94s/it]
 12%|█▏        | 26693/225000 [62:24:10<402:12:36,  7.30s/it]
 12%|█▏        | 26694/225000 [62:24:17<391:25:57,  7.11s/it]
 12%|█▏        | 26695/225000 [62:24:22<349:22:57,  6.34s/it]
 12%|█▏        | 26696/225000 [62:24:28<349:05:46,  6.34s/it]
 12%|█▏        | 26697/225000 [62:24:32<318:47:13,  5.79s/it]
 12%|█▏        | 26698/225000 [62:24:37<297:07:47,  5.39s/it]
 12%|█▏        | 26699/225000 [62:24:41<284:33:18,  5.17s/it]
 12%|█▏        | 26700/225000 [62:24:53<383:53:41,  6.97s/it]
                                                             
{'loss': '0.7296', 'grad_norm': '0.7916', 'learning_rate': '4.926e-05', 'epoch': '0.1187', 'num_input_tokens_seen': 107518584, 'train_runtime': '2.247e+05', 'train_tokens_per_second': '478.5'}

 12%|█▏        | 26700/225000 [62:24:53<383:53:41,  6.97s/it]
 12%|█▏        | 26701/225000 [62:25:00<397:01:04,  7.21s/it]
 12%|█▏        | 26702/225000 [62:25:04<341:05:02,  6.19s/it]
 12%|█▏        | 26703/225000 [62:25:14<396:39:14,  7.20s/it]
 12%|█▏        | 26704/225000 [62:25:21<400:57:47,  7.28s/it]
 12%|█▏        | 26705/225000 [62:25:28<390:43:14,  7.09s/it]
 12%|█▏        | 26706/225000 [62:25:32<343:24:06,  6.23s/it]
 12%|█▏        | 26707/225000 [62:25:40<372:50:42,  6.77s/it]
 12%|█▏        | 26708/225000 [62:25:45<338:39:47,  6.15s/it]
 12%|█▏        | 26709/225000 [62:25:50<320:43:13,  5.82s/it]
 12%|█▏        | 26710/225000 [62:25:56<319:01:42,  5.79s/it]
 12%|█▏        | 26711/225000 [62:26:07<406:08:12,  7.37s/it]
 12%|█▏        | 26712/225000 [62:26:11<354:46:05,  6.44s/it]
 12%|█▏        | 26713/225000 [62:26:16<332:31:11,  6.04s/it]
 12%|█▏        | 26714/225000 [62:26:21<317:44:02,  5.77s/it]
 12%|█▏        | 26715/225000 [62:26:35<444:21:26,  8.07s/it]
 12%|█▏        | 26716/225000 [62:26:38<368:17:18,  6.69s/it]
 12%|█▏        | 26717/225000 [62:26:45<374:39:13,  6.80s/it]
 12%|█▏        | 26718/225000 [62:26:52<376:50:13,  6.84s/it]
 12%|█▏        | 26719/225000 [62:26:56<334:12:16,  6.07s/it]
 12%|█▏        | 26720/225000 [62:27:01<314:36:26,  5.71s/it]
 12%|█▏        | 26721/225000 [62:27:13<411:53:37,  7.48s/it]
 12%|█▏        | 26722/225000 [62:27:18<372:52:35,  6.77s/it]
 12%|█▏        | 26723/225000 [62:27:23<350:17:18,  6.36s/it]
 12%|█▏        | 26724/225000 [62:27:30<362:34:46,  6.58s/it]
 12%|█▏        | 26725/225000 [62:27:34<316:07:13,  5.74s/it]
 12%|█▏        | 26726/225000 [62:27:43<370:32:58,  6.73s/it]
 12%|█▏        | 26727/225000 [62:27:49<361:04:13,  6.56s/it]
 12%|█▏        | 26728/225000 [62:27:54<323:57:30,  5.88s/it]
 12%|█▏        | 26729/225000 [62:27:58<300:33:40,  5.46s/it]
 12%|█▏        | 26730/225000 [62:28:03<287:35:42,  5.22s/it]
 12%|█▏        | 26731/225000 [62:28:14<389:03:06,  7.06s/it]
 12%|█▏        | 26732/225000 [62:28:22<392:48:43,  7.13s/it]
 12%|█▏        | 26733/225000 [62:28:26<347:05:19,  6.30s/it]
 12%|█▏        | 26734/225000 [62:28:32<345:49:25,  6.28s/it]
 12%|█▏        | 26735/225000 [62:28:40<377:19:31,  6.85s/it]
 12%|█▏        | 26736/225000 [62:28:46<351:02:28,  6.37s/it]
 12%|█▏        | 26737/225000 [62:28:51<332:10:02,  6.03s/it]
 12%|█▏        | 26738/225000 [62:28:56<311:42:22,  5.66s/it]
 12%|█▏        | 26739/225000 [62:29:00<298:24:54,  5.42s/it]
 12%|█▏        | 26740/225000 [62:29:06<294:33:21,  5.35s/it]
 12%|█▏        | 26741/225000 [62:29:17<388:54:22,  7.06s/it]
 12%|█▏        | 26742/225000 [62:29:27<448:23:46,  8.14s/it]
 12%|█▏        | 26743/225000 [62:29:34<417:45:48,  7.59s/it]
 12%|█▏        | 26744/225000 [62:29:38<360:42:40,  6.55s/it]
 12%|█▏        | 26745/225000 [62:29:43<344:52:37,  6.26s/it]
 12%|█▏        | 26746/225000 [62:29:49<335:39:44,  6.10s/it]
 12%|█▏        | 26747/225000 [62:29:55<327:46:57,  5.95s/it]
 12%|█▏        | 26748/225000 [62:30:01<326:50:55,  5.94s/it]
 12%|█▏        | 26749/225000 [62:30:11<402:22:32,  7.31s/it]
 12%|█▏        | 26750/225000 [62:30:15<352:38:33,  6.40s/it]
                                                             
{'loss': '0.7086', 'grad_norm': '0.732', 'learning_rate': '4.926e-05', 'epoch': '0.1189', 'num_input_tokens_seen': 107661216, 'train_runtime': '2.25e+05', 'train_tokens_per_second': '478.5'}

 12%|█▏        | 26750/225000 [62:30:15<352:38:33,  6.40s/it]
 12%|█▏        | 26751/225000 [62:30:24<390:51:57,  7.10s/it]
 12%|█▏        | 26752/225000 [62:30:28<344:27:35,  6.26s/it]
 12%|█▏        | 26753/225000 [62:30:37<381:49:07,  6.93s/it]
 12%|█▏        | 26754/225000 [62:30:43<372:21:45,  6.76s/it]
 12%|█▏        | 26755/225000 [62:30:51<388:17:57,  7.05s/it]
 12%|█▏        | 26756/225000 [62:30:57<366:43:42,  6.66s/it]
 12%|█▏        | 26757/225000 [62:31:03<362:06:34,  6.58s/it]
 12%|█▏        | 26758/225000 [62:31:10<360:38:14,  6.55s/it]
 12%|█▏        | 26759/225000 [62:31:17<373:03:06,  6.77s/it]
 12%|█▏        | 26760/225000 [62:31:21<324:17:06,  5.89s/it]
 12%|█▏        | 26761/225000 [62:31:29<368:21:06,  6.69s/it]
 12%|█▏        | 26762/225000 [62:31:33<325:07:54,  5.90s/it]
 12%|█▏        | 26763/225000 [62:31:45<415:13:56,  7.54s/it]
 12%|█▏        | 26764/225000 [62:31:52<404:30:40,  7.35s/it]
 12%|█▏        | 26765/225000 [62:31:55<341:57:56,  6.21s/it]
 12%|█▏        | 26766/225000 [62:32:00<323:14:44,  5.87s/it]
 12%|█▏        | 26767/225000 [62:32:08<360:23:51,  6.54s/it]
 12%|█▏        | 26768/225000 [62:32:13<332:50:03,  6.04s/it]
 12%|█▏        | 26769/225000 [62:32:19<322:07:22,  5.85s/it]
 12%|█▏        | 26770/225000 [62:32:27<369:43:04,  6.71s/it]
 12%|█▏        | 26771/225000 [62:32:36<397:14:49,  7.21s/it]
 12%|█▏        | 26772/225000 [62:32:42<375:49:15,  6.83s/it]
 12%|█▏        | 26773/225000 [62:32:49<379:39:51,  6.90s/it]
 12%|█▏        | 26774/225000 [62:33:00<455:28:01,  8.27s/it]
 12%|█▏        | 26775/225000 [62:33:03<372:32:44,  6.77s/it]
 12%|█▏        | 26776/225000 [62:33:10<375:53:40,  6.83s/it]
 12%|█▏        | 26777/225000 [62:33:16<354:18:41,  6.43s/it]
 12%|█▏        | 26778/225000 [62:33:24<381:59:15,  6.94s/it]
 12%|█▏        | 26779/225000 [62:33:30<359:41:43,  6.53s/it]
 12%|█▏        | 26780/225000 [62:33:37<380:57:32,  6.92s/it]
 12%|█▏        | 26781/225000 [62:33:43<363:09:30,  6.60s/it]
 12%|█▏        | 26782/225000 [62:33:49<348:11:41,  6.32s/it]
 12%|█▏        | 26783/225000 [62:33:53<306:29:06,  5.57s/it]
 12%|█▏        | 26784/225000 [62:34:01<354:24:38,  6.44s/it]
 12%|█▏        | 26785/225000 [62:34:10<386:23:19,  7.02s/it]
 12%|█▏        | 26786/225000 [62:34:15<354:20:16,  6.44s/it]
 12%|█▏        | 26787/225000 [62:34:20<343:05:34,  6.23s/it]
 12%|█▏        | 26788/225000 [62:34:30<397:01:06,  7.21s/it]
 12%|█▏        | 26789/225000 [62:34:34<336:32:15,  6.11s/it]
 12%|█▏        | 26790/225000 [62:34:38<310:15:20,  5.64s/it]
 12%|█▏        | 26791/225000 [62:34:46<351:58:28,  6.39s/it]
 12%|█▏        | 26792/225000 [62:34:56<409:36:34,  7.44s/it]
 12%|█▏        | 26793/225000 [62:35:00<348:06:15,  6.32s/it]
 12%|█▏        | 26794/225000 [62:35:04<311:07:29,  5.65s/it]
 12%|█▏        | 26795/225000 [62:35:11<339:20:42,  6.16s/it]
 12%|█▏        | 26796/225000 [62:35:19<371:19:58,  6.74s/it]
 12%|█▏        | 26797/225000 [62:35:25<360:11:39,  6.54s/it]
 12%|█▏        | 26798/225000 [62:35:29<318:25:12,  5.78s/it]
 12%|█▏        | 26799/225000 [62:35:40<403:58:18,  7.34s/it]
 12%|█▏        | 26800/225000 [62:35:44<348:24:13,  6.33s/it]
                                                             
{'loss': '0.7236', 'grad_norm': '0.7304', 'learning_rate': '4.925e-05', 'epoch': '0.1191', 'num_input_tokens_seen': 107805208, 'train_runtime': '2.253e+05', 'train_tokens_per_second': '478.4'}

 12%|█▏        | 26800/225000 [62:35:44<348:24:13,  6.33s/it]
 12%|█▏        | 26801/225000 [62:35:53<384:23:07,  6.98s/it]
 12%|█▏        | 26802/225000 [62:36:01<406:12:38,  7.38s/it]
 12%|█▏        | 26803/225000 [62:36:06<360:31:54,  6.55s/it]
 12%|█▏        | 26804/225000 [62:36:11<331:14:24,  6.02s/it]
 12%|█▏        | 26805/225000 [62:36:15<304:13:02,  5.53s/it]
 12%|█▏        | 26806/225000 [62:36:24<369:17:35,  6.71s/it]
 12%|█▏        | 26807/225000 [62:36:32<378:16:01,  6.87s/it]
 12%|█▏        | 26808/225000 [62:36:37<350:09:21,  6.36s/it]
 12%|█▏        | 26809/225000 [62:36:42<337:34:10,  6.13s/it]
 12%|█▏        | 26810/225000 [62:36:48<324:03:25,  5.89s/it]
 12%|█▏        | 26811/225000 [62:36:53<314:21:48,  5.71s/it]
 12%|█▏        | 26812/225000 [62:36:58<306:12:38,  5.56s/it]
 12%|█▏        | 26813/225000 [62:37:06<345:15:47,  6.27s/it]
 12%|█▏        | 26814/225000 [62:37:13<353:24:41,  6.42s/it]
 12%|█▏        | 26815/225000 [62:37:18<324:46:45,  5.90s/it]
 12%|█▏        | 26816/225000 [62:37:24<339:46:41,  6.17s/it]
 12%|█▏        | 26817/225000 [62:37:35<415:11:13,  7.54s/it]
 12%|█▏        | 26818/225000 [62:37:40<375:06:01,  6.81s/it]
 12%|█▏        | 26819/225000 [62:37:45<340:09:40,  6.18s/it]
 12%|█▏        | 26820/225000 [62:37:50<319:39:24,  5.81s/it]
 12%|█▏        | 26821/225000 [62:38:00<392:21:37,  7.13s/it]
 12%|█▏        | 26822/225000 [62:38:06<378:56:39,  6.88s/it]
 12%|█▏        | 26823/225000 [62:38:11<335:53:03,  6.10s/it]
 12%|█▏        | 26824/225000 [62:38:16<317:20:16,  5.76s/it]
 12%|█▏        | 26825/225000 [62:38:20<297:37:50,  5.41s/it]
 12%|█▏        | 26826/225000 [62:38:26<297:04:02,  5.40s/it]
 12%|█▏        | 26827/225000 [62:38:39<426:28:32,  7.75s/it]
 12%|█▏        | 26828/225000 [62:38:43<361:21:18,  6.56s/it]
 12%|█▏        | 26829/225000 [62:38:51<389:39:56,  7.08s/it]
 12%|█▏        | 26830/225000 [62:38:59<403:19:12,  7.33s/it]
 12%|█▏        | 26831/225000 [62:39:05<383:09:28,  6.96s/it]
 12%|█▏        | 26832/225000 [62:39:10<352:04:14,  6.40s/it]
 12%|█▏        | 26833/225000 [62:39:14<314:49:34,  5.72s/it]
 12%|█▏        | 26834/225000 [62:39:24<377:21:08,  6.86s/it]
 12%|█▏        | 26835/225000 [62:39:28<337:58:25,  6.14s/it]
 12%|█▏        | 26836/225000 [62:39:33<315:13:33,  5.73s/it]
 12%|█▏        | 26837/225000 [62:39:39<313:00:02,  5.69s/it]
 12%|█▏        | 26838/225000 [62:39:43<299:22:14,  5.44s/it]
 12%|█▏        | 26839/225000 [62:39:50<314:35:53,  5.72s/it]
 12%|█▏        | 26840/225000 [62:40:03<439:49:15,  7.99s/it]
 12%|█▏        | 26841/225000 [62:40:09<404:46:05,  7.35s/it]
 12%|█▏        | 26842/225000 [62:40:14<362:35:48,  6.59s/it]
 12%|█▏        | 26843/225000 [62:40:19<337:57:32,  6.14s/it]
 12%|█▏        | 26844/225000 [62:40:24<320:59:16,  5.83s/it]
 12%|█▏        | 26845/225000 [62:40:35<411:22:44,  7.47s/it]
 12%|█▏        | 26846/225000 [62:40:42<391:20:09,  7.11s/it]
 12%|█▏        | 26847/225000 [62:40:48<376:53:49,  6.85s/it]
 12%|█▏        | 26848/225000 [62:40:52<335:18:45,  6.09s/it]
 12%|█▏        | 26849/225000 [62:40:57<320:16:03,  5.82s/it]
 12%|█▏        | 26850/225000 [62:41:02<309:28:23,  5.62s/it]
                                                             
{'loss': '0.7202', 'grad_norm': '0.6721', 'learning_rate': '4.925e-05', 'epoch': '0.1193', 'num_input_tokens_seen': 107949016, 'train_runtime': '2.257e+05', 'train_tokens_per_second': '478.4'}

 12%|█▏        | 26850/225000 [62:41:02<309:28:23,  5.62s/it]
 12%|█▏        | 26851/225000 [62:41:08<302:23:58,  5.49s/it]
 12%|█▏        | 26852/225000 [62:41:13<303:15:19,  5.51s/it]
 12%|█▏        | 26853/225000 [62:41:25<399:33:07,  7.26s/it]
 12%|█▏        | 26854/225000 [62:41:33<423:58:15,  7.70s/it]
 12%|█▏        | 26855/225000 [62:41:40<415:56:05,  7.56s/it]
 12%|█▏        | 26856/225000 [62:41:45<363:56:29,  6.61s/it]
 12%|█▏        | 26857/225000 [62:41:50<331:42:52,  6.03s/it]
 12%|█▏        | 26858/225000 [62:42:01<417:22:41,  7.58s/it]
 12%|█▏        | 26859/225000 [62:42:04<347:53:53,  6.32s/it]
 12%|█▏        | 26860/225000 [62:42:14<411:27:27,  7.48s/it]
 12%|█▏        | 26861/225000 [62:42:18<355:19:04,  6.46s/it]
 12%|█▏        | 26862/225000 [62:42:27<384:09:44,  6.98s/it]
 12%|█▏        | 26863/225000 [62:42:30<330:49:34,  6.01s/it]
 12%|█▏        | 26864/225000 [62:42:36<318:00:21,  5.78s/it]
 12%|█▏        | 26865/225000 [62:42:41<307:54:02,  5.59s/it]
 12%|█▏        | 26866/225000 [62:42:46<295:13:58,  5.36s/it]
 12%|█▏        | 26867/225000 [62:42:54<342:32:35,  6.22s/it]
 12%|█▏        | 26868/225000 [62:43:05<431:25:07,  7.84s/it]
 12%|█▏        | 26869/225000 [62:43:13<432:11:45,  7.85s/it]
 12%|█▏        | 26870/225000 [62:43:17<360:38:18,  6.55s/it]
 12%|█▏        | 26871/225000 [62:43:25<384:22:30,  6.98s/it]
 12%|█▏        | 26872/225000 [62:43:29<337:28:45,  6.13s/it]
 12%|█▏        | 26873/225000 [62:43:38<384:36:30,  6.99s/it]
 12%|█▏        | 26874/225000 [62:43:41<325:56:50,  5.92s/it]
 12%|█▏        | 26875/225000 [62:43:51<381:58:15,  6.94s/it]
 12%|█▏        | 26876/225000 [62:43:58<385:09:13,  7.00s/it]
 12%|█▏        | 26877/225000 [62:44:04<367:21:00,  6.67s/it]
 12%|█▏        | 26878/225000 [62:44:09<352:24:54,  6.40s/it]
 12%|█▏        | 26879/225000 [62:44:17<371:04:27,  6.74s/it]
 12%|█▏        | 26880/225000 [62:44:21<325:11:39,  5.91s/it]
 12%|█▏        | 26881/225000 [62:44:31<396:01:04,  7.20s/it]
 12%|█▏        | 26882/225000 [62:44:35<344:43:17,  6.26s/it]
 12%|█▏        | 26883/225000 [62:44:43<376:06:07,  6.83s/it]
 12%|█▏        | 26884/225000 [62:44:52<402:59:58,  7.32s/it]
 12%|█▏        | 26885/225000 [62:44:58<383:08:31,  6.96s/it]
 12%|█▏        | 26886/225000 [62:45:03<344:30:57,  6.26s/it]
 12%|█▏        | 26887/225000 [62:45:09<350:12:01,  6.36s/it]
 12%|█▏        | 26888/225000 [62:45:15<332:44:50,  6.05s/it]
 12%|█▏        | 26889/225000 [62:45:20<327:01:09,  5.94s/it]
 12%|█▏        | 26890/225000 [62:45:26<321:47:02,  5.85s/it]
 12%|█▏        | 26891/225000 [62:45:32<332:39:41,  6.05s/it]
 12%|█▏        | 26892/225000 [62:45:41<380:31:53,  6.91s/it]
 12%|█▏        | 26893/225000 [62:45:46<335:53:33,  6.10s/it]
 12%|█▏        | 26894/225000 [62:45:50<308:46:47,  5.61s/it]
 12%|█▏        | 26895/225000 [62:46:02<414:31:44,  7.53s/it]
 12%|█▏        | 26896/225000 [62:46:07<371:50:17,  6.76s/it]
 12%|█▏        | 26897/225000 [62:46:12<336:07:23,  6.11s/it]
 12%|█▏        | 26898/225000 [62:46:17<324:09:59,  5.89s/it]
 12%|█▏        | 26899/225000 [62:46:23<319:49:44,  5.81s/it]
 12%|█▏        | 26900/225000 [62:46:28<310:23:02,  5.64s/it]
                                                             
{'loss': '0.7129', 'grad_norm': '0.6093', 'learning_rate': '4.924e-05', 'epoch': '0.1196', 'num_input_tokens_seen': 108090536, 'train_runtime': '2.26e+05', 'train_tokens_per_second': '478.3'}

 12%|█▏        | 26900/225000 [62:46:28<310:23:02,  5.64s/it]
 12%|█▏        | 26901/225000 [62:46:38<392:28:27,  7.13s/it]
 12%|█▏        | 26902/225000 [62:46:46<398:15:22,  7.24s/it]
 12%|█▏        | 26903/225000 [62:46:50<342:31:46,  6.22s/it]
 12%|█▏        | 26904/225000 [62:46:54<307:32:00,  5.59s/it]
 12%|█▏        | 26905/225000 [62:47:04<390:32:23,  7.10s/it]
 12%|█▏        | 26906/225000 [62:47:09<346:00:03,  6.29s/it]
 12%|█▏        | 26907/225000 [62:47:14<321:21:06,  5.84s/it]
 12%|█▏        | 26908/225000 [62:47:18<298:47:17,  5.43s/it]
 12%|█▏        | 26909/225000 [62:47:27<347:41:40,  6.32s/it]
 12%|█▏        | 26910/225000 [62:47:37<418:56:56,  7.61s/it]
 12%|█▏        | 26911/225000 [62:47:42<365:19:11,  6.64s/it]
 12%|█▏        | 26912/225000 [62:47:47<338:17:34,  6.15s/it]
 12%|█▏        | 26913/225000 [62:47:52<320:56:11,  5.83s/it]
 12%|█▏        | 26914/225000 [62:47:57<314:42:10,  5.72s/it]
 12%|█▏        | 26915/225000 [62:48:02<301:49:17,  5.49s/it]
 12%|█▏        | 26916/225000 [62:48:14<414:11:25,  7.53s/it]
 12%|█▏        | 26917/225000 [62:48:20<381:33:58,  6.93s/it]
 12%|█▏        | 26918/225000 [62:48:25<343:43:59,  6.25s/it]
 12%|█▏        | 26919/225000 [62:48:29<321:33:24,  5.84s/it]
 12%|█▏        | 26920/225000 [62:48:35<309:38:31,  5.63s/it]
 12%|█▏        | 26921/225000 [62:48:42<336:30:23,  6.12s/it]
 12%|█▏        | 26922/225000 [62:48:54<439:20:11,  7.98s/it]
 12%|█▏        | 26923/225000 [62:49:00<404:11:52,  7.35s/it]
 12%|█▏        | 26924/225000 [62:49:04<344:43:03,  6.27s/it]
 12%|█▏        | 26925/225000 [62:49:13<395:49:31,  7.19s/it]
 12%|█▏        | 26926/225000 [62:49:18<353:59:57,  6.43s/it]
 12%|█▏        | 26927/225000 [62:49:27<397:31:20,  7.23s/it]
 12%|█▏        | 26928/225000 [62:49:31<349:13:04,  6.35s/it]
 12%|█▏        | 26929/225000 [62:49:35<302:09:18,  5.49s/it]
 12%|█▏        | 26930/225000 [62:49:42<336:41:11,  6.12s/it]
 12%|█▏        | 26931/225000 [62:49:49<339:49:20,  6.18s/it]
 12%|█▏        | 26932/225000 [62:49:58<391:05:23,  7.11s/it]
 12%|█▏        | 26933/225000 [62:50:02<336:12:23,  6.11s/it]
 12%|█▏        | 26934/225000 [62:50:09<358:44:27,  6.52s/it]
 12%|█▏        | 26935/225000 [62:50:14<325:58:08,  5.92s/it]
 12%|█▏        | 26936/225000 [62:50:18<307:14:49,  5.58s/it]
 12%|█▏        | 26937/225000 [62:50:29<390:49:48,  7.10s/it]
 12%|█▏        | 26938/225000 [62:50:33<343:03:52,  6.24s/it]
 12%|█▏        | 26939/225000 [62:50:38<310:27:27,  5.64s/it]
 12%|█▏        | 26940/225000 [62:50:42<285:31:23,  5.19s/it]
 12%|█▏        | 26941/225000 [62:50:46<278:56:30,  5.07s/it]
 12%|█▏        | 26942/225000 [62:50:51<276:58:47,  5.03s/it]
 12%|█▏        | 26943/225000 [62:51:03<380:17:02,  6.91s/it]
 12%|█▏        | 26944/225000 [62:51:12<412:40:12,  7.50s/it]
 12%|█▏        | 26945/225000 [62:51:19<405:56:46,  7.38s/it]
 12%|█▏        | 26946/225000 [62:51:23<349:08:55,  6.35s/it]
 12%|█▏        | 26947/225000 [62:51:32<401:42:56,  7.30s/it]
 12%|█▏        | 26948/225000 [62:51:37<360:29:56,  6.55s/it]
 12%|█▏        | 26949/225000 [62:51:42<333:43:24,  6.07s/it]
 12%|█▏        | 26950/225000 [62:51:47<311:50:45,  5.67s/it]
                                                             
{'loss': '0.6997', 'grad_norm': '0.6421', 'learning_rate': '4.924e-05', 'epoch': '0.1198', 'num_input_tokens_seen': 108231464, 'train_runtime': '2.263e+05', 'train_tokens_per_second': '478.2'}

 12%|█▏        | 26950/225000 [62:51:47<311:50:45,  5.67s/it]
 12%|█▏        | 26951/225000 [62:51:53<324:55:38,  5.91s/it]
 12%|█▏        | 26952/225000 [62:52:06<441:07:38,  8.02s/it]
 12%|█▏        | 26953/225000 [62:52:10<367:11:05,  6.67s/it]
 12%|█▏        | 26954/225000 [62:52:18<392:04:39,  7.13s/it]
 12%|█▏        | 26955/225000 [62:52:23<364:54:21,  6.63s/it]
 12%|█▏        | 26956/225000 [62:52:30<374:31:29,  6.81s/it]
 12%|█▏        | 26957/225000 [62:52:34<328:50:01,  5.98s/it]
 12%|█▏        | 26958/225000 [62:52:43<370:48:19,  6.74s/it]
 12%|█▏        | 26959/225000 [62:52:48<347:04:53,  6.31s/it]
 12%|█▏        | 26960/225000 [62:52:54<341:04:33,  6.20s/it]
 12%|█▏        | 26961/225000 [62:53:00<335:49:16,  6.10s/it]
 12%|█▏        | 26962/225000 [62:53:05<320:03:54,  5.82s/it]
 12%|█▏        | 26963/225000 [62:53:10<310:00:42,  5.64s/it]
 12%|█▏        | 26964/225000 [62:53:19<356:11:32,  6.48s/it]
 12%|█▏        | 26965/225000 [62:53:28<395:18:41,  7.19s/it]
 12%|█▏        | 26966/225000 [62:53:33<358:08:04,  6.51s/it]
 12%|█▏        | 26967/225000 [62:53:38<330:02:44,  6.00s/it]
 12%|█▏        | 26968/225000 [62:53:42<304:40:31,  5.54s/it]
 12%|█▏        | 26969/225000 [62:53:54<412:55:01,  7.51s/it]
 12%|█▏        | 26970/225000 [62:54:01<399:01:16,  7.25s/it]
 12%|█▏        | 26971/225000 [62:54:08<391:51:09,  7.12s/it]
 12%|█▏        | 26972/225000 [62:54:15<394:45:38,  7.18s/it]
 12%|█▏        | 26973/225000 [62:54:22<393:10:19,  7.15s/it]
 12%|█▏        | 26974/225000 [62:54:26<344:30:07,  6.26s/it]
 12%|█▏        | 26975/225000 [62:54:35<385:57:18,  7.02s/it]
 12%|█▏        | 26976/225000 [62:54:38<325:01:17,  5.91s/it]
 12%|█▏        | 26977/225000 [62:54:47<364:51:55,  6.63s/it]
 12%|█▏        | 26978/225000 [62:54:50<310:40:22,  5.65s/it]
 12%|█▏        | 26979/225000 [62:54:58<343:36:01,  6.25s/it]
 12%|█▏        | 26980/225000 [62:55:03<336:19:24,  6.11s/it]
 12%|█▏        | 26981/225000 [62:55:11<354:14:38,  6.44s/it]
 12%|█▏        | 26982/225000 [62:55:15<321:32:30,  5.85s/it]
 12%|█▏        | 26983/225000 [62:55:25<391:49:13,  7.12s/it]
 12%|█▏        | 26984/225000 [62:55:30<354:00:20,  6.44s/it]
 12%|█▏        | 26985/225000 [62:55:35<329:44:33,  5.99s/it]
 12%|█▏        | 26986/225000 [62:55:39<299:12:13,  5.44s/it]
 12%|█▏        | 26987/225000 [62:55:46<328:22:01,  5.97s/it]
 12%|█▏        | 26988/225000 [62:55:55<380:18:41,  6.91s/it]
 12%|█▏        | 26989/225000 [62:55:59<318:32:57,  5.79s/it]
 12%|█▏        | 26990/225000 [62:56:07<356:34:32,  6.48s/it]
 12%|█▏        | 26991/225000 [62:56:11<320:33:48,  5.83s/it]
 12%|█▏        | 26992/225000 [62:56:15<294:38:17,  5.36s/it]
 12%|█▏        | 26993/225000 [62:56:20<280:00:24,  5.09s/it]
 12%|█▏        | 26994/225000 [62:56:28<326:54:43,  5.94s/it]
 12%|█▏        | 26995/225000 [62:56:38<401:17:39,  7.30s/it]
 12%|█▏        | 26996/225000 [62:56:43<369:29:19,  6.72s/it]
 12%|█▏        | 26997/225000 [62:56:48<336:02:26,  6.11s/it]
 12%|█▏        | 26998/225000 [62:56:54<326:24:39,  5.93s/it]
 12%|█▏        | 26999/225000 [62:56:59<318:00:40,  5.78s/it]
 12%|█▏        | 27000/225000 [62:57:05<320:34:45,  5.83s/it]
                                                             
{'loss': '0.7207', 'grad_norm': '0.6393', 'learning_rate': '4.923e-05', 'epoch': '0.12', 'num_input_tokens_seen': 108372168, 'train_runtime': '2.266e+05', 'train_tokens_per_second': '478.2'}

 12%|█▏        | 27000/225000 [62:57:05<320:34:45,  5.83s/it]
 12%|█▏        | 27001/225000 [62:57:13<361:38:24,  6.58s/it]
 12%|█▏        | 27002/225000 [62:57:26<462:56:31,  8.42s/it]
 12%|█▏        | 27003/225000 [62:57:29<378:18:29,  6.88s/it]
 12%|█▏        | 27004/225000 [62:57:39<416:20:13,  7.57s/it]
 12%|█▏        | 27005/225000 [62:57:43<357:19:26,  6.50s/it]
 12%|█▏        | 27006/225000 [62:57:48<346:41:10,  6.30s/it]
 12%|█▏        | 27007/225000 [62:57:57<377:49:24,  6.87s/it]
 12%|█▏        | 27008/225000 [62:58:01<334:24:40,  6.08s/it]
 12%|█▏        | 27009/225000 [62:58:05<303:39:08,  5.52s/it]
 12%|█▏        | 27010/225000 [62:58:09<285:21:16,  5.19s/it]
 12%|█▏        | 27011/225000 [62:58:14<275:57:16,  5.02s/it]
 12%|█▏        | 27012/225000 [62:58:26<397:34:33,  7.23s/it]
 12%|█▏        | 27013/225000 [62:58:32<375:28:08,  6.83s/it]
 12%|█▏        | 27014/225000 [62:58:40<393:24:58,  7.15s/it]
 12%|█▏        | 27015/225000 [62:58:44<341:42:08,  6.21s/it]
 12%|█▏        | 27016/225000 [62:58:52<363:54:14,  6.62s/it]
 12%|█▏        | 27017/225000 [62:58:56<323:03:02,  5.87s/it]
 12%|█▏        | 27018/225000 [62:59:05<379:24:52,  6.90s/it]
 12%|█▏        | 27019/225000 [62:59:09<326:35:12,  5.94s/it]
 12%|█▏        | 27020/225000 [62:59:13<294:52:36,  5.36s/it]
 12%|█▏        | 27021/225000 [62:59:24<389:10:40,  7.08s/it]
 12%|█▏        | 27022/225000 [62:59:28<330:42:35,  6.01s/it]
 12%|█▏        | 27023/225000 [62:59:37<380:59:59,  6.93s/it]
 12%|█▏        | 27024/225000 [62:59:41<344:04:55,  6.26s/it]
 12%|█▏        | 27025/225000 [62:59:47<327:44:49,  5.96s/it]
 12%|█▏        | 27026/225000 [62:59:55<368:04:39,  6.69s/it]
 12%|█▏        | 27027/225000 [62:59:59<317:44:54,  5.78s/it]
 12%|█▏        | 27028/225000 [63:00:06<345:13:32,  6.28s/it]
 12%|█▏        | 27029/225000 [63:00:12<338:23:43,  6.15s/it]
 12%|█▏        | 27030/225000 [63:00:16<307:06:49,  5.58s/it]
 12%|█▏        | 27031/225000 [63:00:20<283:01:03,  5.15s/it]
 12%|█▏        | 27032/225000 [63:00:26<284:03:13,  5.17s/it]
 12%|█▏        | 27033/225000 [63:00:31<295:29:24,  5.37s/it]
 12%|█▏        | 27034/225000 [63:00:41<364:15:26,  6.62s/it]
 12%|█▏        | 27035/225000 [63:00:51<426:18:37,  7.75s/it]
 12%|█▏        | 27036/225000 [63:00:59<423:14:55,  7.70s/it]
 12%|█▏        | 27037/225000 [63:01:02<354:07:40,  6.44s/it]
 12%|█▏        | 27038/225000 [63:01:11<396:57:54,  7.22s/it]
 12%|█▏        | 27039/225000 [63:01:15<332:44:00,  6.05s/it]
 12%|█▏        | 27040/225000 [63:01:24<378:13:30,  6.88s/it]
 12%|█▏        | 27041/225000 [63:01:27<322:34:14,  5.87s/it]
 12%|█▏        | 27042/225000 [63:01:37<388:07:37,  7.06s/it]
 12%|█▏        | 27043/225000 [63:01:41<339:37:37,  6.18s/it]
 12%|█▏        | 27044/225000 [63:01:45<299:00:22,  5.44s/it]
 12%|█▏        | 27045/225000 [63:01:57<409:55:19,  7.45s/it]
 12%|█▏        | 27046/225000 [63:02:01<347:58:50,  6.33s/it]
 12%|█▏        | 27047/225000 [63:02:09<379:30:57,  6.90s/it]
 12%|█▏        | 27048/225000 [63:02:13<333:26:51,  6.06s/it]
 12%|█▏        | 27049/225000 [63:02:22<377:11:41,  6.86s/it]
 12%|█▏        | 27050/225000 [63:02:25<322:12:28,  5.86s/it]
                                                             
{'loss': '0.7065', 'grad_norm': '0.6762', 'learning_rate': '4.923e-05', 'epoch': '0.1202', 'num_input_tokens_seen': 108511728, 'train_runtime': '2.269e+05', 'train_tokens_per_second': '478.1'}

 12%|█▏        | 27050/225000 [63:02:25<322:12:28,  5.86s/it]
 12%|█▏        | 27051/225000 [63:02:33<357:45:38,  6.51s/it]
 12%|█▏        | 27052/225000 [63:02:38<322:17:56,  5.86s/it]
 12%|█▏        | 27053/225000 [63:02:46<358:49:26,  6.53s/it]
 12%|█▏        | 27054/225000 [63:02:53<369:09:29,  6.71s/it]
 12%|█▏        | 27055/225000 [63:02:57<329:20:38,  5.99s/it]
 12%|█▏        | 27056/225000 [63:03:01<301:00:43,  5.47s/it]
 12%|█▏        | 27057/225000 [63:03:11<366:42:09,  6.67s/it]
 12%|█▏        | 27058/225000 [63:03:17<364:53:19,  6.64s/it]
 12%|█▏        | 27059/225000 [63:03:21<317:51:29,  5.78s/it]
 12%|█▏        | 27060/225000 [63:03:28<342:29:57,  6.23s/it]
 12%|█▏        | 27061/225000 [63:03:37<379:40:49,  6.91s/it]
 12%|█▏        | 27062/225000 [63:03:41<335:01:43,  6.09s/it]
 12%|█▏        | 27063/225000 [63:03:46<310:54:51,  5.65s/it]
 12%|█▏        | 27064/225000 [63:03:51<310:09:22,  5.64s/it]
 12%|█▏        | 27065/225000 [63:04:03<408:35:44,  7.43s/it]
 12%|█▏        | 27066/225000 [63:04:09<383:52:06,  6.98s/it]
 12%|█▏        | 27067/225000 [63:04:15<362:20:26,  6.59s/it]
 12%|█▏        | 27068/225000 [63:04:24<410:55:20,  7.47s/it]
 12%|█▏        | 27069/225000 [63:04:29<364:25:38,  6.63s/it]
 12%|█▏        | 27070/225000 [63:04:34<338:13:29,  6.15s/it]
 12%|█▏        | 27071/225000 [63:04:39<318:41:43,  5.80s/it]
 12%|█▏        | 27072/225000 [63:04:44<302:35:17,  5.50s/it]
 12%|█▏        | 27073/225000 [63:04:50<312:38:32,  5.69s/it]
 12%|█▏        | 27074/225000 [63:05:02<425:18:24,  7.74s/it]
 12%|█▏        | 27075/225000 [63:05:06<365:46:04,  6.65s/it]
 12%|█▏        | 27076/225000 [63:05:16<418:37:35,  7.61s/it]
 12%|█▏        | 27077/225000 [63:05:20<360:19:56,  6.55s/it]
 12%|█▏        | 27078/225000 [63:05:25<323:16:57,  5.88s/it]
 12%|█▏        | 27079/225000 [63:05:37<429:19:20,  7.81s/it]
 12%|█▏        | 27080/225000 [63:05:40<353:50:36,  6.44s/it]
 12%|█▏        | 27081/225000 [63:05:46<336:55:25,  6.13s/it]
 12%|█▏        | 27082/225000 [63:05:51<320:51:37,  5.84s/it]
 12%|█▏        | 27083/225000 [63:05:58<351:05:00,  6.39s/it]
 12%|█▏        | 27084/225000 [63:06:04<337:26:54,  6.14s/it]
 12%|█▏        | 27085/225000 [63:06:10<344:12:32,  6.26s/it]
 12%|█▏        | 27086/225000 [63:06:16<328:33:07,  5.98s/it]
 12%|█▏        | 27087/225000 [63:06:20<306:33:45,  5.58s/it]
 12%|█▏        | 27088/225000 [63:06:32<412:46:11,  7.51s/it]
 12%|█▏        | 27089/225000 [63:06:40<406:01:10,  7.39s/it]
 12%|█▏        | 27090/225000 [63:06:43<348:49:56,  6.35s/it]
 12%|█▏        | 27091/225000 [63:06:48<322:57:21,  5.87s/it]
 12%|█▏        | 27092/225000 [63:06:52<295:20:49,  5.37s/it]
 12%|█▏        | 27093/225000 [63:07:05<414:06:08,  7.53s/it]
 12%|█▏        | 27094/225000 [63:07:11<383:28:15,  6.98s/it]
 12%|█▏        | 27095/225000 [63:07:18<381:18:58,  6.94s/it]
 12%|█▏        | 27096/225000 [63:07:25<389:58:59,  7.09s/it]
 12%|█▏        | 27097/225000 [63:07:29<339:08:25,  6.17s/it]
 12%|█▏        | 27098/225000 [63:07:33<307:43:23,  5.60s/it]
 12%|█▏        | 27099/225000 [63:07:38<289:43:19,  5.27s/it]
 12%|█▏        | 27100/225000 [63:07:51<417:08:42,  7.59s/it]
                                                             
{'loss': '0.709', 'grad_norm': '0.6146', 'learning_rate': '4.922e-05', 'epoch': '0.1204', 'num_input_tokens_seen': 108655008, 'train_runtime': '2.273e+05', 'train_tokens_per_second': '478.1'}

 12%|█▏        | 27100/225000 [63:07:51<417:08:42,  7.59s/it]
 12%|█▏        | 27101/225000 [63:07:55<365:57:56,  6.66s/it]
 12%|█▏        | 27102/225000 [63:08:01<345:04:13,  6.28s/it]
 12%|█▏        | 27103/225000 [63:08:10<394:21:13,  7.17s/it]
 12%|█▏        | 27104/225000 [63:08:15<355:47:18,  6.47s/it]
 12%|█▏        | 27105/225000 [63:08:20<331:46:00,  6.04s/it]
 12%|█▏        | 27106/225000 [63:08:29<381:39:03,  6.94s/it]
 12%|█▏        | 27107/225000 [63:08:34<358:19:03,  6.52s/it]
 12%|█▏        | 27108/225000 [63:08:39<327:31:57,  5.96s/it]
 12%|█▏        | 27109/225000 [63:08:48<385:26:28,  7.01s/it]
 12%|█▏        | 27110/225000 [63:08:52<326:01:18,  5.93s/it]
 12%|█▏        | 27111/225000 [63:09:00<359:20:19,  6.54s/it]
 12%|█▏        | 27112/225000 [63:09:06<357:37:07,  6.51s/it]
 12%|█▏        | 27113/225000 [63:09:13<354:12:05,  6.44s/it]
 12%|█▏        | 27114/225000 [63:09:18<345:12:34,  6.28s/it]
 12%|█▏        | 27115/225000 [63:09:27<382:32:44,  6.96s/it]
 12%|█▏        | 27116/225000 [63:09:31<334:55:21,  6.09s/it]
 12%|█▏        | 27117/225000 [63:09:36<312:33:06,  5.69s/it]
 12%|█▏        | 27118/225000 [63:09:40<289:28:36,  5.27s/it]
 12%|█▏        | 27119/225000 [63:09:52<405:37:18,  7.38s/it]
 12%|█▏        | 27120/225000 [63:09:58<383:26:46,  6.98s/it]
 12%|█▏        | 27121/225000 [63:10:04<355:11:19,  6.46s/it]
 12%|█▏        | 27122/225000 [63:10:09<330:09:49,  6.01s/it]
 12%|█▏        | 27123/225000 [63:10:14<310:52:43,  5.66s/it]
 12%|█▏        | 27124/225000 [63:10:18<289:06:55,  5.26s/it]
 12%|█▏        | 27125/225000 [63:10:23<291:36:13,  5.31s/it]
 12%|█▏        | 27126/225000 [63:10:31<330:51:54,  6.02s/it]
 12%|█▏        | 27127/225000 [63:10:43<425:10:53,  7.74s/it]
 12%|█▏        | 27128/225000 [63:10:47<375:58:50,  6.84s/it]
 12%|█▏        | 27129/225000 [63:10:55<385:02:05,  7.01s/it]
 12%|█▏        | 27130/225000 [63:10:59<341:56:17,  6.22s/it]
 12%|█▏        | 27131/225000 [63:11:09<392:43:41,  7.15s/it]
 12%|█▏        | 27132/225000 [63:11:14<369:27:29,  6.72s/it]
 12%|█▏        | 27133/225000 [63:11:21<369:11:21,  6.72s/it]
 12%|█▏        | 27134/225000 [63:11:25<330:36:45,  6.02s/it]
 12%|█▏        | 27135/225000 [63:11:31<317:34:17,  5.78s/it]
 12%|█▏        | 27136/225000 [63:11:36<305:29:06,  5.56s/it]
 12%|█▏        | 27137/225000 [63:11:47<405:13:52,  7.37s/it]
 12%|█▏        | 27138/225000 [63:11:52<362:21:49,  6.59s/it]
 12%|█▏        | 27139/225000 [63:11:58<344:59:28,  6.28s/it]
 12%|█▏        | 27140/225000 [63:12:03<333:10:10,  6.06s/it]
 12%|█▏        | 27141/225000 [63:12:09<323:08:09,  5.88s/it]
 12%|█▏        | 27142/225000 [63:12:14<313:22:30,  5.70s/it]
 12%|█▏        | 27143/225000 [63:12:20<315:53:14,  5.75s/it]
 12%|█▏        | 27144/225000 [63:12:30<395:25:51,  7.19s/it]
 12%|█▏        | 27145/225000 [63:12:38<397:34:22,  7.23s/it]
 12%|█▏        | 27146/225000 [63:12:44<377:07:57,  6.86s/it]
 12%|█▏        | 27147/225000 [63:12:49<358:30:40,  6.52s/it]
 12%|█▏        | 27148/225000 [63:12:55<344:19:18,  6.27s/it]
 12%|█▏        | 27149/225000 [63:13:01<338:16:49,  6.16s/it]
 12%|█▏        | 27150/225000 [63:13:07<330:27:06,  6.01s/it]
                                                             
{'loss': '0.7143', 'grad_norm': '0.6011', 'learning_rate': '4.922e-05', 'epoch': '0.1207', 'num_input_tokens_seen': 108797464, 'train_runtime': '2.276e+05', 'train_tokens_per_second': '478'}

 12%|█▏        | 27150/225000 [63:13:07<330:27:06,  6.01s/it]
 12%|█▏        | 27151/225000 [63:13:13<335:19:35,  6.10s/it]
 12%|█▏        | 27152/225000 [63:13:22<390:32:17,  7.11s/it]
 12%|█▏        | 27153/225000 [63:13:28<374:31:19,  6.81s/it]
 12%|█▏        | 27154/225000 [63:13:34<351:31:36,  6.40s/it]
 12%|█▏        | 27155/225000 [63:13:40<341:17:26,  6.21s/it]
 12%|█▏        | 27156/225000 [63:13:45<326:38:25,  5.94s/it]
 12%|█▏        | 27157/225000 [63:13:50<314:02:57,  5.71s/it]
 12%|█▏        | 27158/225000 [63:13:55<299:20:18,  5.45s/it]
 12%|█▏        | 27159/225000 [63:14:01<314:09:59,  5.72s/it]
 12%|█▏        | 27160/225000 [63:14:11<372:52:49,  6.79s/it]
 12%|█▏        | 27161/225000 [63:14:17<359:41:36,  6.55s/it]
 12%|█▏        | 27162/225000 [63:14:22<334:58:46,  6.10s/it]
 12%|█▏        | 27163/225000 [63:14:27<322:41:58,  5.87s/it]
 12%|█▏        | 27164/225000 [63:14:33<317:33:54,  5.78s/it]
 12%|█▏        | 27165/225000 [63:14:44<407:36:08,  7.42s/it]
 12%|█▏        | 27166/225000 [63:14:53<430:04:42,  7.83s/it]
 12%|█▏        | 27167/225000 [63:15:00<420:30:53,  7.65s/it]
 12%|█▏        | 27168/225000 [63:15:03<354:17:59,  6.45s/it]
 12%|█▏        | 27169/225000 [63:15:11<369:13:54,  6.72s/it]
 12%|█▏        | 27170/225000 [63:15:15<334:04:46,  6.08s/it]
 12%|█▏        | 27171/225000 [63:15:20<311:28:09,  5.67s/it]
 12%|█▏        | 27172/225000 [63:15:29<360:15:11,  6.56s/it]
 12%|█▏        | 27173/225000 [63:15:36<375:39:43,  6.84s/it]
 12%|█▏        | 27174/225000 [63:15:45<409:10:02,  7.45s/it]
 12%|█▏        | 27175/225000 [63:15:53<411:23:24,  7.49s/it]
 12%|█▏        | 27176/225000 [63:15:56<350:24:40,  6.38s/it]
 12%|█▏        | 27177/225000 [63:16:05<385:31:16,  7.02s/it]
 12%|█▏        | 27178/225000 [63:16:10<345:50:21,  6.29s/it]
 12%|█▏        | 27179/225000 [63:16:14<313:59:10,  5.71s/it]
 12%|█▏        | 27180/225000 [63:16:18<285:19:32,  5.19s/it]
 12%|█▏        | 27181/225000 [63:16:25<309:54:33,  5.64s/it]
 12%|█▏        | 27182/225000 [63:16:38<433:14:45,  7.88s/it]
 12%|█▏        | 27183/225000 [63:16:42<366:06:22,  6.66s/it]
 12%|█▏        | 27184/225000 [63:16:45<321:01:37,  5.84s/it]
 12%|█▏        | 27185/225000 [63:16:51<312:43:53,  5.69s/it]
 12%|█▏        | 27186/225000 [63:16:56<305:54:30,  5.57s/it]
 12%|█▏        | 27187/225000 [63:17:01<297:25:46,  5.41s/it]
 12%|█▏        | 27188/225000 [63:17:13<407:13:21,  7.41s/it]
 12%|█▏        | 27189/225000 [63:17:18<364:48:31,  6.64s/it]
 12%|█▏        | 27190/225000 [63:17:23<342:54:08,  6.24s/it]
 12%|█▏        | 27191/225000 [63:17:28<323:51:59,  5.89s/it]
 12%|█▏        | 27192/225000 [63:17:33<304:17:23,  5.54s/it]
 12%|█▏        | 27193/225000 [63:17:38<288:22:59,  5.25s/it]
 12%|█▏        | 27194/225000 [63:17:43<283:28:27,  5.16s/it]
 12%|█▏        | 27195/225000 [63:17:54<388:33:42,  7.07s/it]
 12%|█▏        | 27196/225000 [63:18:02<404:26:54,  7.36s/it]
 12%|█▏        | 27197/225000 [63:18:08<386:33:09,  7.04s/it]
 12%|█▏        | 27198/225000 [63:18:14<357:56:52,  6.51s/it]
 12%|█▏        | 27199/225000 [63:18:21<361:35:18,  6.58s/it]
 12%|█▏        | 27200/225000 [63:18:27<365:17:42,  6.65s/it]
                                                             
{'loss': '0.729', 'grad_norm': '0.6415', 'learning_rate': '4.922e-05', 'epoch': '0.1209', 'num_input_tokens_seen': 108938656, 'train_runtime': '2.279e+05', 'train_tokens_per_second': '478'}

 12%|█▏        | 27200/225000 [63:18:27<365:17:42,  6.65s/it]
 12%|█▏        | 27201/225000 [63:18:31<324:17:10,  5.90s/it]
 12%|█▏        | 27202/225000 [63:18:37<317:21:11,  5.78s/it]
 12%|█▏        | 27203/225000 [63:18:42<303:39:31,  5.53s/it]
 12%|█▏        | 27204/225000 [63:18:47<295:12:58,  5.37s/it]
 12%|█▏        | 27205/225000 [63:18:52<298:25:52,  5.43s/it]
 12%|█▏        | 27206/225000 [63:18:59<315:26:53,  5.74s/it]
 12%|█▏        | 27207/225000 [63:19:12<433:15:17,  7.89s/it]
 12%|█▏        | 27208/225000 [63:19:17<389:10:06,  7.08s/it]
 12%|█▏        | 27209/225000 [63:19:21<344:31:10,  6.27s/it]
 12%|█▏        | 27210/225000 [63:19:33<429:06:07,  7.81s/it]
 12%|█▏        | 27211/225000 [63:19:36<358:30:15,  6.53s/it]
 12%|█▏        | 27212/225000 [63:19:45<397:18:31,  7.23s/it]
 12%|█▏        | 27213/225000 [63:19:50<356:18:55,  6.49s/it]
 12%|█▏        | 27214/225000 [63:19:57<358:01:15,  6.52s/it]
 12%|█▏        | 27215/225000 [63:20:02<342:30:00,  6.23s/it]
 12%|█▏        | 27216/225000 [63:20:09<346:41:23,  6.31s/it]
 12%|█▏        | 27217/225000 [63:20:14<324:31:21,  5.91s/it]
 12%|█▏        | 27218/225000 [63:20:18<305:21:12,  5.56s/it]
 12%|█▏        | 27219/225000 [63:20:23<284:01:21,  5.17s/it]
 12%|█▏        | 27220/225000 [63:20:28<281:26:04,  5.12s/it]
 12%|█▏        | 27221/225000 [63:20:39<391:20:03,  7.12s/it]
 12%|█▏        | 27222/225000 [63:20:44<343:49:01,  6.26s/it]
 12%|█▏        | 27223/225000 [63:20:53<402:20:35,  7.32s/it]
 12%|█▏        | 27224/225000 [63:20:58<348:23:42,  6.34s/it]
 12%|█▏        | 27225/225000 [63:21:05<373:29:46,  6.80s/it]
 12%|█▏        | 27226/225000 [63:21:13<393:39:39,  7.17s/it]
 12%|█▏        | 27227/225000 [63:21:20<388:37:40,  7.07s/it]
 12%|█▏        | 27228/225000 [63:21:24<331:11:22,  6.03s/it]
 12%|█▏        | 27229/225000 [63:21:32<362:35:17,  6.60s/it]
 12%|█▏        | 27230/225000 [63:21:36<321:07:42,  5.85s/it]
 12%|█▏        | 27231/225000 [63:21:41<311:32:32,  5.67s/it]
 12%|█▏        | 27232/225000 [63:21:49<353:09:23,  6.43s/it]
 12%|█▏        | 27233/225000 [63:21:56<355:41:35,  6.47s/it]
 12%|█▏        | 27234/225000 [63:22:02<353:33:13,  6.44s/it]
 12%|█▏        | 27235/225000 [63:22:10<367:07:10,  6.68s/it]
 12%|█▏        | 27236/225000 [63:22:13<321:02:21,  5.84s/it]
 12%|█▏        | 27237/225000 [63:22:25<412:48:27,  7.51s/it]
 12%|█▏        | 27238/225000 [63:22:30<372:25:46,  6.78s/it]
 12%|█▏        | 27239/225000 [63:22:36<368:24:57,  6.71s/it]
 12%|█▏        | 27240/225000 [63:22:41<331:25:48,  6.03s/it]
 12%|█▏        | 27241/225000 [63:22:50<389:09:02,  7.08s/it]
 12%|█▏        | 27242/225000 [63:22:54<339:15:51,  6.18s/it]
 12%|█▏        | 27243/225000 [63:22:59<311:17:00,  5.67s/it]
 12%|█▏        | 27244/225000 [63:23:08<368:57:52,  6.72s/it]
 12%|█▏        | 27245/225000 [63:23:14<359:17:35,  6.54s/it]
 12%|█▏        | 27246/225000 [63:23:19<333:54:03,  6.08s/it]
 12%|█▏        | 27247/225000 [63:23:24<308:12:07,  5.61s/it]
 12%|█▏        | 27248/225000 [63:23:36<413:50:24,  7.53s/it]
 12%|█▏        | 27249/225000 [63:23:41<375:39:17,  6.84s/it]
 12%|█▏        | 27250/225000 [63:23:46<345:39:35,  6.29s/it]
                                                             
{'loss': '0.7392', 'grad_norm': '0.5958', 'learning_rate': '4.921e-05', 'epoch': '0.1211', 'num_input_tokens_seen': 109080840, 'train_runtime': '2.282e+05', 'train_tokens_per_second': '477.9'}

 12%|█▏        | 27250/225000 [63:23:46<345:39:35,  6.29s/it]
 12%|█▏        | 27251/225000 [63:23:51<320:50:19,  5.84s/it]
 12%|█▏        | 27252/225000 [63:23:56<308:25:35,  5.61s/it]
 12%|█▏        | 27253/225000 [63:24:01<300:38:53,  5.47s/it]
 12%|█▏        | 27254/225000 [63:24:06<287:28:49,  5.23s/it]
 12%|█▏        | 27255/225000 [63:24:16<362:50:53,  6.61s/it]
 12%|█▏        | 27256/225000 [63:24:26<431:08:44,  7.85s/it]
 12%|█▏        | 27257/225000 [63:24:34<421:42:55,  7.68s/it]
 12%|█▏        | 27258/225000 [63:24:37<359:29:45,  6.54s/it]
 12%|█▏        | 27259/225000 [63:24:44<354:04:18,  6.45s/it]
 12%|█▏        | 27260/225000 [63:24:50<345:41:01,  6.29s/it]
 12%|█▏        | 27261/225000 [63:24:57<363:06:36,  6.61s/it]
 12%|█▏        | 27262/225000 [63:25:02<330:12:31,  6.01s/it]
 12%|█▏        | 27263/225000 [63:25:06<304:22:26,  5.54s/it]
 12%|█▏        | 27264/225000 [63:25:11<299:06:13,  5.45s/it]
 12%|█▏        | 27265/225000 [63:25:23<406:08:58,  7.39s/it]
 12%|█▏        | 27266/225000 [63:25:27<348:46:58,  6.35s/it]
 12%|█▏        | 27267/225000 [63:25:35<369:17:44,  6.72s/it]
 12%|█▏        | 27268/225000 [63:25:41<367:23:58,  6.69s/it]
 12%|█▏        | 27269/225000 [63:25:45<318:57:26,  5.81s/it]
 12%|█▏        | 27270/225000 [63:25:55<387:01:57,  7.05s/it]
 12%|█▏        | 27271/225000 [63:25:59<338:00:54,  6.15s/it]
 12%|█▏        | 27272/225000 [63:26:04<317:45:32,  5.79s/it]
 12%|█▏        | 27273/225000 [63:26:10<316:17:21,  5.76s/it]
 12%|█▏        | 27274/225000 [63:26:14<300:46:38,  5.48s/it]
 12%|█▏        | 27275/225000 [63:26:20<295:44:55,  5.38s/it]
 12%|█▏        | 27276/225000 [63:26:25<297:52:15,  5.42s/it]
 12%|█▏        | 27277/225000 [63:26:37<402:01:24,  7.32s/it]
 12%|█▏        | 27278/225000 [63:26:42<368:50:36,  6.72s/it]
 12%|█▏        | 27279/225000 [63:26:49<376:50:09,  6.86s/it]
 12%|█▏        | 27280/225000 [63:26:56<375:48:02,  6.84s/it]
 12%|█▏        | 27281/225000 [63:27:00<327:37:47,  5.97s/it]
 12%|█▏        | 27282/225000 [63:27:04<297:27:29,  5.42s/it]
 12%|█▏        | 27283/225000 [63:27:09<284:57:10,  5.19s/it]
 12%|█▏        | 27284/225000 [63:27:14<278:22:50,  5.07s/it]
 12%|█▏        | 27285/225000 [63:27:23<340:17:00,  6.20s/it]
 12%|█▏        | 27286/225000 [63:27:35<440:50:16,  8.03s/it]
 12%|█▏        | 27287/225000 [63:27:39<373:43:40,  6.80s/it]
 12%|█▏        | 27288/225000 [63:27:42<321:24:10,  5.85s/it]
 12%|█▏        | 27289/225000 [63:27:53<403:49:00,  7.35s/it]
 12%|█▏        | 27290/225000 [63:27:58<366:37:04,  6.68s/it]
 12%|█▏        | 27291/225000 [63:28:06<375:04:32,  6.83s/it]
 12%|█▏        | 27292/225000 [63:28:10<338:41:40,  6.17s/it]
 12%|█▏        | 27293/225000 [63:28:15<317:01:32,  5.77s/it]
 12%|█▏        | 27294/225000 [63:28:20<303:40:58,  5.53s/it]
 12%|█▏        | 27295/225000 [63:28:25<302:19:53,  5.51s/it]
 12%|█▏        | 27296/225000 [63:28:37<394:46:11,  7.19s/it]
 12%|█▏        | 27297/225000 [63:28:41<354:44:08,  6.46s/it]
 12%|█▏        | 27298/225000 [63:28:47<347:53:22,  6.33s/it]
 12%|█▏        | 27299/225000 [63:28:55<371:56:38,  6.77s/it]
 12%|█▏        | 27300/225000 [63:29:01<362:21:52,  6.60s/it]
                                                             
{'loss': '0.7331', 'grad_norm': '0.6969', 'learning_rate': '4.921e-05', 'epoch': '0.1213', 'num_input_tokens_seen': 109220400, 'train_runtime': '2.285e+05', 'train_tokens_per_second': '477.9'}

 12%|█▏        | 27300/225000 [63:29:01<362:21:52,  6.60s/it]
 12%|█▏        | 27301/225000 [63:29:08<356:17:44,  6.49s/it]
 12%|█▏        | 27302/225000 [63:29:12<320:02:53,  5.83s/it]
 12%|█▏        | 27303/225000 [63:29:20<357:38:11,  6.51s/it]
 12%|█▏        | 27304/225000 [63:29:28<389:11:48,  7.09s/it]
 12%|█▏        | 27305/225000 [63:29:35<375:04:33,  6.83s/it]
 12%|█▏        | 27306/225000 [63:29:39<338:27:49,  6.16s/it]
 12%|█▏        | 27307/225000 [63:29:44<319:40:52,  5.82s/it]
 12%|█▏        | 27308/225000 [63:29:56<421:10:53,  7.67s/it]
 12%|█▏        | 27309/225000 [63:30:00<362:55:42,  6.61s/it]
 12%|█▏        | 27310/225000 [63:30:07<366:14:34,  6.67s/it]
 12%|█▏        | 27311/225000 [63:30:16<399:04:20,  7.27s/it]
 12%|█▏        | 27312/225000 [63:30:21<357:48:15,  6.52s/it]
 12%|█▏        | 27313/225000 [63:30:26<331:49:58,  6.04s/it]
 12%|█▏        | 27314/225000 [63:30:30<307:29:49,  5.60s/it]
 12%|█▏        | 27315/225000 [63:30:42<405:44:23,  7.39s/it]
 12%|█▏        | 27316/225000 [63:30:45<345:50:14,  6.30s/it]
 12%|█▏        | 27317/225000 [63:30:53<374:14:42,  6.82s/it]
 12%|█▏        | 27318/225000 [63:30:57<324:58:30,  5.92s/it]
 12%|█▏        | 27319/225000 [63:31:08<409:06:52,  7.45s/it]
 12%|█▏        | 27320/225000 [63:31:14<380:51:47,  6.94s/it]
 12%|█▏        | 27321/225000 [63:31:21<376:33:43,  6.86s/it]
 12%|█▏        | 27322/225000 [63:31:26<350:10:39,  6.38s/it]
 12%|█▏        | 27323/225000 [63:31:31<333:31:53,  6.07s/it]
 12%|█▏        | 27324/225000 [63:31:37<326:28:26,  5.95s/it]
 12%|█▏        | 27325/225000 [63:31:43<325:36:08,  5.93s/it]
 12%|█▏        | 27326/225000 [63:31:48<313:26:09,  5.71s/it]
 12%|█▏        | 27327/225000 [63:31:53<306:12:16,  5.58s/it]
 12%|█▏        | 27328/225000 [63:32:03<378:52:42,  6.90s/it]
 12%|█▏        | 27329/225000 [63:32:13<416:01:50,  7.58s/it]
 12%|█▏        | 27330/225000 [63:32:19<404:22:38,  7.36s/it]
 12%|█▏        | 27331/225000 [63:32:25<375:33:48,  6.84s/it]
 12%|█▏        | 27332/225000 [63:32:30<348:44:00,  6.35s/it]
 12%|█▏        | 27333/225000 [63:32:35<325:54:22,  5.94s/it]
 12%|█▏        | 27334/225000 [63:32:40<311:45:53,  5.68s/it]
 12%|█▏        | 27335/225000 [63:32:50<373:31:34,  6.80s/it]
 12%|█▏        | 27336/225000 [63:32:58<401:21:10,  7.31s/it]
 12%|█▏        | 27337/225000 [63:33:04<372:36:58,  6.79s/it]
 12%|█▏        | 27338/225000 [63:33:11<380:15:22,  6.93s/it]
 12%|█▏        | 27339/225000 [63:33:16<353:28:34,  6.44s/it]
 12%|█▏        | 27340/225000 [63:33:23<355:17:22,  6.47s/it]
 12%|█▏        | 27341/225000 [63:33:27<309:40:22,  5.64s/it]
 12%|█▏        | 27342/225000 [63:33:36<380:36:24,  6.93s/it]
 12%|█▏        | 27343/225000 [63:33:41<335:45:54,  6.12s/it]
 12%|█▏        | 27344/225000 [63:33:45<308:44:56,  5.62s/it]
 12%|█▏        | 27345/225000 [63:33:50<297:24:58,  5.42s/it]
 12%|█▏        | 27346/225000 [63:34:02<402:35:33,  7.33s/it]
 12%|█▏        | 27347/225000 [63:34:07<370:26:07,  6.75s/it]
 12%|█▏        | 27348/225000 [63:34:12<339:15:41,  6.18s/it]
 12%|█▏        | 27349/225000 [63:34:17<319:35:22,  5.82s/it]
 12%|█▏        | 27350/225000 [63:34:22<304:54:14,  5.55s/it]
                                                             
{'loss': '0.7069', 'grad_norm': '0.7538', 'learning_rate': '4.92e-05', 'epoch': '0.1216', 'num_input_tokens_seen': 109364480, 'train_runtime': '2.289e+05', 'train_tokens_per_second': '477.9'}

 12%|█▏        | 27350/225000 [63:34:22<304:54:14,  5.55s/it]
 12%|█▏        | 27351/225000 [63:34:27<296:11:17,  5.39s/it]
 12%|█▏        | 27352/225000 [63:34:33<299:00:42,  5.45s/it]
 12%|█▏        | 27353/225000 [63:34:44<403:34:28,  7.35s/it]
 12%|█▏        | 27354/225000 [63:34:51<396:41:39,  7.23s/it]
 12%|█▏        | 27355/225000 [63:34:55<342:22:15,  6.24s/it]
 12%|█▏        | 27356/225000 [63:35:00<322:04:45,  5.87s/it]
 12%|█▏        | 27357/225000 [63:35:07<328:28:04,  5.98s/it]
 12%|█▏        | 27358/225000 [63:35:18<410:24:54,  7.48s/it]
 12%|█▏        | 27359/225000 [63:35:23<377:34:39,  6.88s/it]
 12%|█▏        | 27360/225000 [63:35:30<376:02:17,  6.85s/it]
 12%|█▏        | 27361/225000 [63:35:36<365:59:07,  6.67s/it]
 12%|█▏        | 27362/225000 [63:35:44<394:37:18,  7.19s/it]
 12%|█▏        | 27363/225000 [63:35:50<374:49:18,  6.83s/it]
 12%|█▏        | 27364/225000 [63:35:55<341:21:52,  6.22s/it]
 12%|█▏        | 27365/225000 [63:36:00<314:00:44,  5.72s/it]
 12%|█▏        | 27366/225000 [63:36:05<305:24:55,  5.56s/it]
 12%|█▏        | 27367/225000 [63:36:10<303:29:03,  5.53s/it]
 12%|█▏        | 27368/225000 [63:36:15<291:26:46,  5.31s/it]
 12%|█▏        | 27369/225000 [63:36:22<311:22:55,  5.67s/it]
 12%|█▏        | 27370/225000 [63:36:34<427:51:44,  7.79s/it]
 12%|█▏        | 27371/225000 [63:36:40<388:24:35,  7.08s/it]
 12%|█▏        | 27372/225000 [63:36:48<406:48:47,  7.41s/it]
 12%|█▏        | 27373/225000 [63:36:55<393:03:40,  7.16s/it]
 12%|█▏        | 27374/225000 [63:36:58<329:17:20,  6.00s/it]
 12%|█▏        | 27375/225000 [63:37:06<368:52:31,  6.72s/it]
 12%|█▏        | 27376/225000 [63:37:13<369:16:32,  6.73s/it]
 12%|█▏        | 27377/225000 [63:37:18<345:45:06,  6.30s/it]
 12%|█▏        | 27378/225000 [63:37:24<331:06:15,  6.03s/it]
 12%|█▏        | 27379/225000 [63:37:29<325:48:10,  5.94s/it]
 12%|█▏        | 27380/225000 [63:37:35<322:05:42,  5.87s/it]
 12%|█▏        | 27381/225000 [63:37:40<310:58:38,  5.67s/it]
 12%|█▏        | 27382/225000 [63:37:46<303:44:18,  5.53s/it]
 12%|█▏        | 27383/225000 [63:37:56<384:02:50,  7.00s/it]
 12%|█▏        | 27384/225000 [63:38:05<412:47:51,  7.52s/it]
 12%|█▏        | 27385/225000 [63:38:10<378:53:42,  6.90s/it]
 12%|█▏        | 27386/225000 [63:38:15<344:33:04,  6.28s/it]
 12%|█▏        | 27387/225000 [63:38:19<309:33:50,  5.64s/it]
 12%|█▏        | 27388/225000 [63:38:24<289:59:12,  5.28s/it]
 12%|█▏        | 27389/225000 [63:38:29<293:24:04,  5.35s/it]
 12%|█▏        | 27390/225000 [63:38:42<416:03:02,  7.58s/it]
 12%|█▏        | 27391/225000 [63:38:46<364:23:48,  6.64s/it]
 12%|█▏        | 27392/225000 [63:38:55<391:43:29,  7.14s/it]
 12%|█▏        | 27393/225000 [63:39:00<358:42:21,  6.53s/it]
 12%|█▏        | 27394/225000 [63:39:04<320:50:58,  5.85s/it]
 12%|█▏        | 27395/225000 [63:39:13<366:22:58,  6.67s/it]
 12%|█▏        | 27396/225000 [63:39:21<393:38:06,  7.17s/it]
 12%|█▏        | 27397/225000 [63:39:25<344:32:00,  6.28s/it]
 12%|█▏        | 27398/225000 [63:39:34<393:36:20,  7.17s/it]
 12%|█▏        | 27399/225000 [63:39:42<394:30:55,  7.19s/it]
 12%|█▏        | 27400/225000 [63:39:48<387:45:27,  7.06s/it]
                                                             
{'loss': '0.7255', 'grad_norm': '0.7216', 'learning_rate': '4.92e-05', 'epoch': '0.1218', 'num_input_tokens_seen': 109505912, 'train_runtime': '2.292e+05', 'train_tokens_per_second': '477.8'}

 12%|█▏        | 27400/225000 [63:39:48<387:45:27,  7.06s/it]
 12%|█▏        | 27401/225000 [63:39:53<344:26:34,  6.28s/it]
 12%|█▏        | 27402/225000 [63:39:57<316:09:01,  5.76s/it]
 12%|█▏        | 27403/225000 [63:40:08<397:24:26,  7.24s/it]
 12%|█▏        | 27404/225000 [63:40:12<348:50:16,  6.36s/it]
 12%|█▏        | 27405/225000 [63:40:21<390:01:10,  7.11s/it]
 12%|█▏        | 27406/225000 [63:40:28<385:50:50,  7.03s/it]
 12%|█▏        | 27407/225000 [63:40:33<343:47:58,  6.26s/it]
 12%|█▏        | 27408/225000 [63:40:41<387:13:56,  7.06s/it]
 12%|█▏        | 27409/225000 [63:40:45<335:03:52,  6.10s/it]
 12%|█▏        | 27410/225000 [63:40:53<363:01:14,  6.61s/it]
 12%|█▏        | 27411/225000 [63:40:57<322:54:16,  5.88s/it]
 12%|█▏        | 27412/225000 [63:41:02<294:58:39,  5.37s/it]
 12%|█▏        | 27413/225000 [63:41:13<390:30:53,  7.12s/it]
 12%|█▏        | 27414/225000 [63:41:17<337:49:49,  6.16s/it]
 12%|█▏        | 27415/225000 [63:41:20<299:09:23,  5.45s/it]
 12%|█▏        | 27416/225000 [63:41:31<383:00:43,  6.98s/it]
 12%|█▏        | 27417/225000 [63:41:35<334:15:05,  6.09s/it]
 12%|█▏        | 27418/225000 [63:41:44<385:40:33,  7.03s/it]
 12%|█▏        | 27419/225000 [63:41:52<392:57:37,  7.16s/it]
 12%|█▏        | 27420/225000 [63:41:56<348:39:37,  6.35s/it]
 12%|█▏        | 27421/225000 [63:42:00<312:40:21,  5.70s/it]
 12%|█▏        | 27422/225000 [63:42:13<420:09:38,  7.66s/it]
 12%|█▏        | 27423/225000 [63:42:16<355:57:35,  6.49s/it]
 12%|█▏        | 27424/225000 [63:42:24<375:41:27,  6.85s/it]
 12%|█▏        | 27425/225000 [63:42:29<344:04:34,  6.27s/it]
 12%|█▏        | 27426/225000 [63:42:36<352:31:36,  6.42s/it]
 12%|█▏        | 27427/225000 [63:42:44<378:07:22,  6.89s/it]
 12%|█▏        | 27428/225000 [63:42:48<339:57:30,  6.19s/it]
 12%|█▏        | 27429/225000 [63:42:53<321:48:34,  5.86s/it]
 12%|█▏        | 27430/225000 [63:42:59<311:30:35,  5.68s/it]
 12%|█▏        | 27431/225000 [63:43:08<364:50:37,  6.65s/it]
 12%|█▏        | 27432/225000 [63:43:16<403:31:16,  7.35s/it]
 12%|█▏        | 27433/225000 [63:43:23<392:31:01,  7.15s/it]
 12%|█▏        | 27434/225000 [63:43:27<335:53:40,  6.12s/it]
 12%|█▏        | 27435/225000 [63:43:36<382:57:36,  6.98s/it]
 12%|█▏        | 27436/225000 [63:43:39<327:12:58,  5.96s/it]
 12%|█▏        | 27437/225000 [63:43:44<298:23:00,  5.44s/it]
 12%|█▏        | 27438/225000 [63:43:52<340:14:32,  6.20s/it]
 12%|█▏        | 27439/225000 [63:44:00<372:50:54,  6.79s/it]
 12%|█▏        | 27440/225000 [63:44:08<390:28:05,  7.12s/it]
 12%|█▏        | 27441/225000 [63:44:15<388:30:13,  7.08s/it]
 12%|█▏        | 27442/225000 [63:44:19<339:56:19,  6.19s/it]
 12%|█▏        | 27443/225000 [63:44:23<307:57:12,  5.61s/it]
 12%|█▏        | 27444/225000 [63:44:31<340:22:06,  6.20s/it]
 12%|█▏        | 27445/225000 [63:44:42<421:40:20,  7.68s/it]
 12%|█▏        | 27446/225000 [63:44:49<408:38:04,  7.45s/it]
 12%|█▏        | 27447/225000 [63:44:53<352:21:34,  6.42s/it]
 12%|█▏        | 27448/225000 [63:45:02<397:23:19,  7.24s/it]
 12%|█▏        | 27449/225000 [63:45:07<370:07:07,  6.74s/it]
 12%|█▏        | 27450/225000 [63:45:12<331:06:39,  6.03s/it]
                                                             
{'loss': '0.7315', 'grad_norm': '0.5916', 'learning_rate': '4.919e-05', 'epoch': '0.122', 'num_input_tokens_seen': 109649928, 'train_runtime': '2.295e+05', 'train_tokens_per_second': '477.8'}

 12%|█▏        | 27450/225000 [63:45:12<331:06:39,  6.03s/it]
 12%|█▏        | 27451/225000 [63:45:16<304:27:50,  5.55s/it]
 12%|█▏        | 27452/225000 [63:45:21<285:07:49,  5.20s/it]
 12%|█▏        | 27453/225000 [63:45:33<408:01:22,  7.44s/it]
 12%|█▏        | 27454/225000 [63:45:37<347:22:48,  6.33s/it]
 12%|█▏        | 27455/225000 [63:45:43<338:19:34,  6.17s/it]
 12%|█▏        | 27456/225000 [63:45:49<334:41:57,  6.10s/it]
 12%|█▏        | 27457/225000 [63:45:54<327:32:55,  5.97s/it]
 12%|█▏        | 27458/225000 [63:46:00<313:38:13,  5.72s/it]
 12%|█▏        | 27459/225000 [63:46:08<356:24:42,  6.50s/it]
 12%|█▏        | 27460/225000 [63:46:15<366:32:10,  6.68s/it]
 12%|█▏        | 27461/225000 [63:46:20<345:48:59,  6.30s/it]
 12%|█▏        | 27462/225000 [63:46:31<410:57:51,  7.49s/it]
 12%|█▏        | 27463/225000 [63:46:36<370:06:10,  6.74s/it]
 12%|█▏        | 27464/225000 [63:46:40<331:25:30,  6.04s/it]
 12%|█▏        | 27465/225000 [63:46:51<415:33:56,  7.57s/it]
 12%|█▏        | 27466/225000 [63:46:56<371:03:45,  6.76s/it]
 12%|█▏        | 27467/225000 [63:47:01<335:47:22,  6.12s/it]
 12%|█▏        | 27468/225000 [63:47:06<320:07:00,  5.83s/it]
 12%|█▏        | 27469/225000 [63:47:11<304:55:37,  5.56s/it]
 12%|█▏        | 27470/225000 [63:47:16<292:14:51,  5.33s/it]
 12%|█▏        | 27471/225000 [63:47:21<287:47:51,  5.25s/it]
 12%|█▏        | 27472/225000 [63:47:26<284:26:39,  5.18s/it]
 12%|█▏        | 27473/225000 [63:47:33<327:54:10,  5.98s/it]
 12%|█▏        | 27474/225000 [63:47:45<420:37:55,  7.67s/it]
 12%|█▏        | 27475/225000 [63:47:51<396:27:28,  7.23s/it]
 12%|█▏        | 27476/225000 [63:47:58<388:36:44,  7.08s/it]
 12%|█▏        | 27477/225000 [63:48:04<367:19:06,  6.69s/it]
 12%|█▏        | 27478/225000 [63:48:11<380:38:29,  6.94s/it]
 12%|█▏        | 27479/225000 [63:48:15<332:26:28,  6.06s/it]
 12%|█▏        | 27480/225000 [63:48:22<338:00:14,  6.16s/it]
 12%|█▏        | 27481/225000 [63:48:31<384:58:38,  7.02s/it]
 12%|█▏        | 27482/225000 [63:48:35<334:34:10,  6.10s/it]
 12%|█▏        | 27483/225000 [63:48:38<295:16:08,  5.38s/it]
 12%|█▏        | 27484/225000 [63:48:50<398:50:22,  7.27s/it]
 12%|█▏        | 27485/225000 [63:48:53<332:27:27,  6.06s/it]
 12%|█▏        | 27486/225000 [63:49:03<386:45:59,  7.05s/it]
 12%|█▏        | 27487/225000 [63:49:06<331:34:28,  6.04s/it]
 12%|█▏        | 27488/225000 [63:49:16<397:08:20,  7.24s/it]
 12%|█▏        | 27489/225000 [63:49:20<339:03:45,  6.18s/it]
 12%|█▏        | 27490/225000 [63:49:24<308:35:43,  5.62s/it]
 12%|█▏        | 27491/225000 [63:49:31<317:59:55,  5.80s/it]
 12%|█▏        | 27492/225000 [63:49:40<368:15:30,  6.71s/it]
 12%|█▏        | 27493/225000 [63:49:47<382:48:15,  6.98s/it]
 12%|█▏        | 27494/225000 [63:49:51<330:44:59,  6.03s/it]
 12%|█▏        | 27495/225000 [63:49:55<304:45:16,  5.55s/it]
 12%|█▏        | 27496/225000 [63:50:01<303:19:51,  5.53s/it]
 12%|█▏        | 27497/225000 [63:50:06<291:41:26,  5.32s/it]
 12%|█▏        | 27498/225000 [63:50:11<293:49:22,  5.36s/it]
 12%|█▏        | 27499/225000 [63:50:18<315:24:52,  5.75s/it]
 12%|█▏        | 27500/225000 [63:50:29<407:03:48,  7.42s/it]
                                                             
{'loss': '0.7128', 'grad_norm': '0.8095', 'learning_rate': '4.919e-05', 'epoch': '0.1222', 'num_input_tokens_seen': 109791672, 'train_runtime': '2.298e+05', 'train_tokens_per_second': '477.7'}

 12%|█▏        | 27500/225000 [63:50:29<407:03:48,  7.42s/it]
 12%|█▏        | 27501/225000 [63:50:33<354:48:43,  6.47s/it]
 12%|█▏        | 27502/225000 [63:50:38<316:51:34,  5.78s/it]
 12%|█▏        | 27503/225000 [63:50:43<320:02:26,  5.83s/it]
 12%|█▏        | 27504/225000 [63:50:56<432:39:24,  7.89s/it]
 12%|█▏        | 27505/225000 [63:51:00<368:36:12,  6.72s/it]
 12%|█▏        | 27506/225000 [63:51:06<349:50:54,  6.38s/it]
 12%|█▏        | 27507/225000 [63:51:12<340:11:12,  6.20s/it]
 12%|█▏        | 27508/225000 [63:51:20<376:28:08,  6.86s/it]
 12%|█▏        | 27509/225000 [63:51:27<388:16:57,  7.08s/it]
 12%|█▏        | 27510/225000 [63:51:32<339:34:54,  6.19s/it]
 12%|█▏        | 27511/225000 [63:51:41<389:16:56,  7.10s/it]
 12%|█▏        | 27512/225000 [63:51:45<345:16:08,  6.29s/it]
 12%|█▏        | 27513/225000 [63:51:52<355:31:16,  6.48s/it]
 12%|█▏        | 27514/225000 [63:51:58<345:10:11,  6.29s/it]
 12%|█▏        | 27515/225000 [63:52:04<338:27:19,  6.17s/it]
 12%|█▏        | 27516/225000 [63:52:12<376:26:50,  6.86s/it]
 12%|█▏        | 27517/225000 [63:52:17<332:13:20,  6.06s/it]
 12%|█▏        | 27518/225000 [63:52:26<389:13:53,  7.10s/it]
 12%|█▏        | 27519/225000 [63:52:30<339:10:29,  6.18s/it]
 12%|█▏        | 27520/225000 [63:52:35<309:45:00,  5.65s/it]
 12%|█▏        | 27521/225000 [63:52:42<331:46:12,  6.05s/it]
 12%|█▏        | 27522/225000 [63:52:52<411:13:07,  7.50s/it]
 12%|█▏        | 27523/225000 [63:52:57<366:28:51,  6.68s/it]
 12%|█▏        | 27524/225000 [63:53:02<336:51:54,  6.14s/it]
 12%|█▏        | 27525/225000 [63:53:07<311:14:15,  5.67s/it]
 12%|█▏        | 27526/225000 [63:53:12<307:43:11,  5.61s/it]
 12%|█▏        | 27527/225000 [63:53:26<440:27:25,  8.03s/it]
 12%|█▏        | 27528/225000 [63:53:32<417:37:27,  7.61s/it]
 12%|█▏        | 27529/225000 [63:53:37<368:14:49,  6.71s/it]
 12%|█▏        | 27530/225000 [63:53:44<378:22:45,  6.90s/it]
 12%|█▏        | 27531/225000 [63:53:48<329:51:26,  6.01s/it]
 12%|█▏        | 27532/225000 [63:53:56<364:02:11,  6.64s/it]
 12%|█▏        | 27533/225000 [63:54:01<338:03:05,  6.16s/it]
 12%|█▏        | 27534/225000 [63:54:06<312:15:26,  5.69s/it]
 12%|█▏        | 27535/225000 [63:54:13<332:23:28,  6.06s/it]
 12%|█▏        | 27536/225000 [63:54:22<384:35:07,  7.01s/it]
 12%|█▏        | 27537/225000 [63:54:31<409:08:54,  7.46s/it]
 12%|█▏        | 27538/225000 [63:54:35<364:03:57,  6.64s/it]
 12%|█▏        | 27539/225000 [63:54:41<338:40:28,  6.17s/it]
 12%|█▏        | 27540/225000 [63:54:46<323:31:26,  5.90s/it]
 12%|█▏        | 27541/225000 [63:54:51<315:13:10,  5.75s/it]
 12%|█▏        | 27542/225000 [63:54:57<313:09:30,  5.71s/it]
 12%|█▏        | 27543/225000 [63:55:02<310:20:08,  5.66s/it]
 12%|█▏        | 27544/225000 [63:55:08<312:09:51,  5.69s/it]
 12%|█▏        | 27545/225000 [63:55:14<315:58:07,  5.76s/it]
 12%|█▏        | 27546/225000 [63:55:22<355:46:44,  6.49s/it]
 12%|█▏        | 27547/225000 [63:55:28<347:31:57,  6.34s/it]
 12%|█▏        | 27548/225000 [63:55:34<336:21:17,  6.13s/it]
 12%|█▏        | 27549/225000 [63:55:39<321:31:34,  5.86s/it]
 12%|█▏        | 27550/225000 [63:55:49<392:16:09,  7.15s/it]
                                                             
{'loss': '0.7222', 'grad_norm': '0.6485', 'learning_rate': '4.918e-05', 'epoch': '0.1224', 'num_input_tokens_seen': 109933760, 'train_runtime': '2.301e+05', 'train_tokens_per_second': '477.7'}

 12%|█▏        | 27550/225000 [63:55:49<392:16:09,  7.15s/it]
 12%|█▏        | 27551/225000 [63:56:01<468:51:07,  8.55s/it]
 12%|█▏        | 27552/225000 [63:56:08<450:33:58,  8.22s/it]
 12%|█▏        | 27553/225000 [63:56:14<406:45:05,  7.42s/it]
 12%|█▏        | 27554/225000 [63:56:19<358:42:58,  6.54s/it]
 12%|█▏        | 27555/225000 [63:56:23<322:57:51,  5.89s/it]
 12%|█▏        | 27556/225000 [63:56:31<359:15:30,  6.55s/it]
 12%|█▏        | 27557/225000 [63:56:40<404:51:02,  7.38s/it]
 12%|█▏        | 27558/225000 [63:56:44<338:06:00,  6.16s/it]
 12%|█▏        | 27559/225000 [63:56:48<313:59:39,  5.73s/it]
 12%|█▏        | 27560/225000 [63:56:53<304:15:14,  5.55s/it]
 12%|█▏        | 27561/225000 [63:56:59<301:35:15,  5.50s/it]
 12%|█▏        | 27562/225000 [63:57:10<396:27:28,  7.23s/it]
 12%|█▏        | 27563/225000 [63:57:17<388:24:07,  7.08s/it]
 12%|█▏        | 27564/225000 [63:57:25<406:24:15,  7.41s/it]
 12%|█▏        | 27565/225000 [63:57:28<341:20:06,  6.22s/it]
 12%|█▏        | 27566/225000 [63:57:37<375:59:22,  6.86s/it]
 12%|█▏        | 27567/225000 [63:57:41<329:09:48,  6.00s/it]
 12%|█▏        | 27568/225000 [63:57:50<387:24:26,  7.06s/it]
 12%|█▏        | 27569/225000 [63:57:55<350:41:37,  6.39s/it]
 12%|█▏        | 27570/225000 [63:57:59<311:18:14,  5.68s/it]
 12%|█▏        | 27571/225000 [63:58:08<368:22:58,  6.72s/it]
 12%|█▏        | 27572/225000 [63:58:17<402:42:58,  7.34s/it]
 12%|█▏        | 27573/225000 [63:58:21<343:59:55,  6.27s/it]
 12%|█▏        | 27574/225000 [63:58:27<333:48:03,  6.09s/it]
 12%|█▏        | 27575/225000 [63:58:34<357:30:34,  6.52s/it]
 12%|█▏        | 27576/225000 [63:58:38<318:41:35,  5.81s/it]
 12%|█▏        | 27577/225000 [63:58:44<315:43:00,  5.76s/it]
 12%|█▏        | 27578/225000 [63:58:53<363:45:14,  6.63s/it]
 12%|█▏        | 27579/225000 [63:58:57<331:18:16,  6.04s/it]
 12%|█▏        | 27580/225000 [63:59:03<320:37:21,  5.85s/it]
 12%|█▏        | 27581/225000 [63:59:15<427:40:34,  7.80s/it]
 12%|█▏        | 27582/225000 [63:59:19<369:21:25,  6.74s/it]
 12%|█▏        | 27583/225000 [63:59:24<331:31:07,  6.05s/it]
 12%|█▏        | 27584/225000 [63:59:28<309:47:07,  5.65s/it]
 12%|█▏        | 27585/225000 [63:59:34<309:35:57,  5.65s/it]
 12%|█▏        | 27586/225000 [63:59:41<337:51:14,  6.16s/it]
 12%|█▏        | 27587/225000 [63:59:52<417:57:20,  7.62s/it]
 12%|█▏        | 27588/225000 [63:59:57<360:35:33,  6.58s/it]
 12%|█▏        | 27589/225000 [64:00:01<322:51:11,  5.89s/it]
 12%|█▏        | 27590/225000 [64:00:12<401:49:17,  7.33s/it]
 12%|█▏        | 27591/225000 [64:00:15<335:59:15,  6.13s/it]
 12%|█▏        | 27592/225000 [64:00:25<396:00:30,  7.22s/it]
 12%|█▏        | 27593/225000 [64:00:32<403:29:36,  7.36s/it]
 12%|█▏        | 27594/225000 [64:00:36<338:59:59,  6.18s/it]
 12%|█▏        | 27595/225000 [64:00:42<343:17:23,  6.26s/it]
 12%|█▏        | 27596/225000 [64:00:50<362:50:20,  6.62s/it]
 12%|█▏        | 27597/225000 [64:00:54<318:21:10,  5.81s/it]
 12%|█▏        | 27598/225000 [64:00:59<315:05:56,  5.75s/it]
 12%|█▏        | 27599/225000 [64:01:10<398:39:55,  7.27s/it]
 12%|█▏        | 27600/225000 [64:01:14<342:46:33,  6.25s/it]
                                                             
{'loss': '0.7034', 'grad_norm': '0.6903', 'learning_rate': '4.918e-05', 'epoch': '0.1227', 'num_input_tokens_seen': 110075792, 'train_runtime': '2.305e+05', 'train_tokens_per_second': '477.6'}

 12%|█▏        | 27600/225000 [64:01:14<342:46:33,  6.25s/it]
 12%|█▏        | 27601/225000 [64:01:21<352:25:17,  6.43s/it]
 12%|█▏        | 27602/225000 [64:01:28<361:46:50,  6.60s/it]
 12%|█▏        | 27603/225000 [64:01:32<318:34:45,  5.81s/it]
 12%|█▏        | 27604/225000 [64:01:36<298:12:22,  5.44s/it]
 12%|█▏        | 27605/225000 [64:01:41<288:51:08,  5.27s/it]
 12%|█▏        | 27606/225000 [64:01:53<397:46:08,  7.25s/it]
 12%|█▏        | 27607/225000 [64:01:57<347:05:19,  6.33s/it]
 12%|█▏        | 27608/225000 [64:02:02<318:21:22,  5.81s/it]
 12%|█▏        | 27609/225000 [64:02:08<331:53:28,  6.05s/it]
 12%|█▏        | 27610/225000 [64:02:18<397:34:04,  7.25s/it]
 12%|█▏        | 27611/225000 [64:02:23<347:46:00,  6.34s/it]
 12%|█▏        | 27612/225000 [64:02:27<311:11:07,  5.68s/it]
 12%|█▏        | 27613/225000 [64:02:33<312:47:44,  5.70s/it]
 12%|█▏        | 27614/225000 [64:02:45<419:28:14,  7.65s/it]
 12%|█▏        | 27615/225000 [64:02:49<363:32:58,  6.63s/it]
 12%|█▏        | 27616/225000 [64:02:53<320:50:19,  5.85s/it]
 12%|█▏        | 27617/225000 [64:03:00<336:25:38,  6.14s/it]
 12%|█▏        | 27618/225000 [64:03:09<392:13:33,  7.15s/it]
 12%|█▏        | 27619/225000 [64:03:13<341:21:44,  6.23s/it]
 12%|█▏        | 27620/225000 [64:03:22<381:50:46,  6.96s/it]
 12%|█▏        | 27621/225000 [64:03:26<338:06:16,  6.17s/it]
 12%|█▏        | 27622/225000 [64:03:34<364:59:02,  6.66s/it]
 12%|█▏        | 27623/225000 [64:03:38<324:56:31,  5.93s/it]
 12%|█▏        | 27624/225000 [64:03:43<303:25:39,  5.53s/it]
 12%|█▏        | 27625/225000 [64:03:48<301:12:51,  5.49s/it]
 12%|█▏        | 27626/225000 [64:03:56<332:37:22,  6.07s/it]
 12%|█▏        | 27627/225000 [64:04:07<411:01:32,  7.50s/it]
 12%|█▏        | 27628/225000 [64:04:11<360:41:00,  6.58s/it]
 12%|█▏        | 27629/225000 [64:04:15<322:19:39,  5.88s/it]
 12%|█▏        | 27630/225000 [64:04:26<404:46:48,  7.38s/it]
 12%|█▏        | 27631/225000 [64:04:30<347:08:30,  6.33s/it]
 12%|█▏        | 27632/225000 [64:04:39<391:49:46,  7.15s/it]
 12%|█▏        | 27633/225000 [64:04:43<338:47:13,  6.18s/it]
 12%|█▏        | 27634/225000 [64:04:52<384:54:12,  7.02s/it]
 12%|█▏        | 27635/225000 [64:04:59<388:44:54,  7.09s/it]
 12%|█▏        | 27636/225000 [64:05:04<351:58:11,  6.42s/it]
 12%|█▏        | 27637/225000 [64:05:09<331:57:52,  6.06s/it]
 12%|█▏        | 27638/225000 [64:05:14<309:43:42,  5.65s/it]
 12%|█▏        | 27639/225000 [64:05:22<346:46:30,  6.33s/it]
 12%|█▏        | 27640/225000 [64:05:31<398:30:33,  7.27s/it]
 12%|█▏        | 27641/225000 [64:05:39<402:07:57,  7.34s/it]
 12%|█▏        | 27642/225000 [64:05:44<357:16:42,  6.52s/it]
 12%|█▏        | 27643/225000 [64:05:48<325:11:12,  5.93s/it]
 12%|█▏        | 27644/225000 [64:05:56<355:31:03,  6.49s/it]
 12%|█▏        | 27645/225000 [64:06:05<391:46:37,  7.15s/it]
 12%|█▏        | 27646/225000 [64:06:08<336:07:24,  6.13s/it]
 12%|█▏        | 27647/225000 [64:06:19<406:49:17,  7.42s/it]
 12%|█▏        | 27648/225000 [64:06:23<354:03:49,  6.46s/it]
 12%|█▏        | 27649/225000 [64:06:28<324:57:07,  5.93s/it]
 12%|█▏        | 27650/225000 [64:06:33<310:47:09,  5.67s/it]
                                                             
{'loss': '0.7335', 'grad_norm': '0.6792', 'learning_rate': '4.917e-05', 'epoch': '0.1229', 'num_input_tokens_seen': 110218848, 'train_runtime': '2.308e+05', 'train_tokens_per_second': '477.6'}

 12%|█▏        | 27650/225000 [64:06:33<310:47:09,  5.67s/it]
 12%|█▏        | 27651/225000 [64:06:44<396:09:20,  7.23s/it]
 12%|█▏        | 27652/225000 [64:06:49<373:12:22,  6.81s/it]
 12%|█▏        | 27653/225000 [64:06:57<387:48:13,  7.07s/it]
 12%|█▏        | 27654/225000 [64:07:02<357:31:12,  6.52s/it]
 12%|█▏        | 27655/225000 [64:07:08<344:26:41,  6.28s/it]
 12%|█▏        | 27656/225000 [64:07:15<354:34:09,  6.47s/it]
 12%|█▏        | 27657/225000 [64:07:20<323:13:31,  5.90s/it]
 12%|█▏        | 27658/225000 [64:07:24<297:46:23,  5.43s/it]
 12%|█▏        | 27659/225000 [64:07:35<384:27:01,  7.01s/it]
 12%|█▏        | 27660/225000 [64:07:39<334:59:05,  6.11s/it]
 12%|█▏        | 27661/225000 [64:07:43<305:20:15,  5.57s/it]
 12%|█▏        | 27662/225000 [64:07:47<287:25:39,  5.24s/it]
 12%|█▏        | 27663/225000 [64:07:57<360:38:11,  6.58s/it]
 12%|█▏        | 27664/225000 [64:08:03<345:36:23,  6.30s/it]
 12%|█▏        | 27665/225000 [64:08:11<381:33:58,  6.96s/it]
 12%|█▏        | 27666/225000 [64:08:15<328:30:56,  5.99s/it]
 12%|█▏        | 27667/225000 [64:08:20<306:50:33,  5.60s/it]
 12%|█▏        | 27668/225000 [64:08:25<297:11:43,  5.42s/it]
 12%|█▏        | 27669/225000 [64:08:36<400:24:13,  7.30s/it]
 12%|█▏        | 27670/225000 [64:08:41<355:06:31,  6.48s/it]
 12%|█▏        | 27671/225000 [64:08:47<340:36:31,  6.21s/it]
 12%|█▏        | 27672/225000 [64:08:52<330:35:06,  6.03s/it]
 12%|█▏        | 27673/225000 [64:08:58<330:07:17,  6.02s/it]
 12%|█▏        | 27674/225000 [64:09:03<313:25:27,  5.72s/it]
 12%|█▏        | 27675/225000 [64:09:14<401:02:55,  7.32s/it]
 12%|█▏        | 27676/225000 [64:09:24<448:12:11,  8.18s/it]
 12%|█▏        | 27677/225000 [64:09:28<380:34:06,  6.94s/it]
 12%|█▏        | 27678/225000 [64:09:32<327:37:52,  5.98s/it]
 12%|█▏        | 27679/225000 [64:09:36<293:29:21,  5.35s/it]
 12%|█▏        | 27680/225000 [64:09:46<369:40:58,  6.74s/it]
 12%|█▏        | 27681/225000 [64:09:53<379:02:20,  6.92s/it]
 12%|█▏        | 27682/225000 [64:09:59<355:14:24,  6.48s/it]
 12%|█▏        | 27683/225000 [64:10:06<361:36:42,  6.60s/it]
 12%|█▏        | 27684/225000 [64:10:15<402:50:09,  7.35s/it]
 12%|█▏        | 27685/225000 [64:10:21<377:11:57,  6.88s/it]
 12%|█▏        | 27686/225000 [64:10:26<347:01:58,  6.33s/it]
 12%|█▏        | 27687/225000 [64:10:30<318:39:57,  5.81s/it]
 12%|█▏        | 27688/225000 [64:10:35<305:36:38,  5.58s/it]
 12%|█▏        | 27689/225000 [64:10:40<289:21:07,  5.28s/it]
 12%|█▏        | 27690/225000 [64:10:47<324:51:33,  5.93s/it]
 12%|█▏        | 27691/225000 [64:10:59<419:41:59,  7.66s/it]
 12%|█▏        | 27692/225000 [64:11:04<382:28:24,  6.98s/it]
 12%|█▏        | 27693/225000 [64:11:09<335:13:56,  6.12s/it]
 12%|█▏        | 27694/225000 [64:11:16<363:21:05,  6.63s/it]
 12%|█▏        | 27695/225000 [64:11:23<360:34:11,  6.58s/it]
 12%|█▏        | 27696/225000 [64:11:31<392:34:49,  7.16s/it]
 12%|█▏        | 27697/225000 [64:11:36<349:38:16,  6.38s/it]
 12%|█▏        | 27698/225000 [64:11:40<319:44:10,  5.83s/it]
 12%|█▏        | 27699/225000 [64:11:46<317:07:23,  5.79s/it]
 12%|█▏        | 27700/225000 [64:11:58<414:14:04,  7.56s/it]
                                                             
{'loss': '0.7253', 'grad_norm': '0.6785', 'learning_rate': '4.917e-05', 'epoch': '0.1231', 'num_input_tokens_seen': 110361728, 'train_runtime': '2.311e+05', 'train_tokens_per_second': '477.5'}

 12%|█▏        | 27700/225000 [64:11:58<414:14:04,  7.56s/it]
 12%|█▏        | 27701/225000 [64:12:02<358:50:21,  6.55s/it]
 12%|█▏        | 27702/225000 [64:12:10<375:16:20,  6.85s/it]
 12%|█▏        | 27703/225000 [64:12:17<377:45:00,  6.89s/it]
 12%|█▏        | 27704/225000 [64:12:22<348:18:52,  6.36s/it]
 12%|█▏        | 27705/225000 [64:12:31<392:34:14,  7.16s/it]
 12%|█▏        | 27706/225000 [64:12:34<333:44:30,  6.09s/it]
 12%|█▏        | 27707/225000 [64:12:43<375:59:08,  6.86s/it]
 12%|█▏        | 27708/225000 [64:12:46<321:07:36,  5.86s/it]
 12%|█▏        | 27709/225000 [64:12:51<300:50:55,  5.49s/it]
 12%|█▏        | 27710/225000 [64:13:01<369:18:37,  6.74s/it]
 12%|█▏        | 27711/225000 [64:13:08<380:42:19,  6.95s/it]
 12%|█▏        | 27712/225000 [64:13:14<364:39:17,  6.65s/it]
 12%|█▏        | 27713/225000 [64:13:22<383:13:20,  6.99s/it]
 12%|█▏        | 27714/225000 [64:13:27<351:24:28,  6.41s/it]
 12%|█▏        | 27715/225000 [64:13:32<329:45:18,  6.02s/it]
 12%|█▏        | 27716/225000 [64:13:41<385:47:03,  7.04s/it]
 12%|█▏        | 27717/225000 [64:13:46<341:49:47,  6.24s/it]
 12%|█▏        | 27718/225000 [64:13:53<360:02:36,  6.57s/it]
 12%|█▏        | 27719/225000 [64:14:01<375:00:23,  6.84s/it]
 12%|█▏        | 27720/225000 [64:14:06<345:10:07,  6.30s/it]
 12%|█▏        | 27721/225000 [64:14:14<381:57:40,  6.97s/it]
 12%|█▏        | 27722/225000 [64:14:21<385:23:16,  7.03s/it]
 12%|█▏        | 27723/225000 [64:14:27<357:57:10,  6.53s/it]
 12%|█▏        | 27724/225000 [64:14:32<330:39:36,  6.03s/it]
 12%|█▏        | 27725/225000 [64:14:37<314:59:50,  5.75s/it]
 12%|█▏        | 27726/225000 [64:14:42<300:01:03,  5.47s/it]
 12%|█▏        | 27727/225000 [64:14:46<286:08:01,  5.22s/it]
 12%|█▏        | 27728/225000 [64:14:51<275:08:34,  5.02s/it]
 12%|█▏        | 27729/225000 [64:14:57<299:57:23,  5.47s/it]
 12%|█▏        | 27730/225000 [64:15:11<436:01:00,  7.96s/it]
 12%|█▏        | 27731/225000 [64:15:15<364:06:53,  6.64s/it]
 12%|█▏        | 27732/225000 [64:15:19<333:41:09,  6.09s/it]
 12%|█▏        | 27733/225000 [64:15:24<313:19:53,  5.72s/it]
 12%|█▏        | 27734/225000 [64:15:36<419:56:39,  7.66s/it]
 12%|█▏        | 27735/225000 [64:15:41<367:37:10,  6.71s/it]
 12%|█▏        | 27736/225000 [64:15:45<321:16:09,  5.86s/it]
 12%|█▏        | 27737/225000 [64:15:51<320:50:53,  5.86s/it]
 12%|█▏        | 27738/225000 [64:15:59<359:49:19,  6.57s/it]
 12%|█▏        | 27739/225000 [64:16:05<358:57:26,  6.55s/it]
 12%|█▏        | 27740/225000 [64:16:09<314:56:38,  5.75s/it]
 12%|█▏        | 27741/225000 [64:16:16<334:08:12,  6.10s/it]
 12%|█▏        | 27742/225000 [64:16:24<355:50:33,  6.49s/it]
 12%|█▏        | 27743/225000 [64:16:29<332:43:51,  6.07s/it]
 12%|█▏        | 27744/225000 [64:16:35<341:27:28,  6.23s/it]
 12%|█▏        | 27745/225000 [64:16:42<343:34:09,  6.27s/it]
 12%|█▏        | 27746/225000 [64:16:48<350:34:51,  6.40s/it]
 12%|█▏        | 27747/225000 [64:16:53<315:51:20,  5.76s/it]
 12%|█▏        | 27748/225000 [64:17:03<390:24:02,  7.13s/it]
 12%|█▏        | 27749/225000 [64:17:07<332:23:30,  6.07s/it]
 12%|█▏        | 27750/225000 [64:17:16<386:01:35,  7.05s/it]
                                                             
{'loss': '0.725', 'grad_norm': '0.6255', 'learning_rate': '4.917e-05', 'epoch': '0.1233', 'num_input_tokens_seen': 110504248, 'train_runtime': '2.314e+05', 'train_tokens_per_second': '477.5'}

 12%|█▏        | 27750/225000 [64:17:16<386:01:35,  7.05s/it]
 12%|█▏        | 27751/225000 [64:17:20<332:07:02,  6.06s/it]
 12%|█▏        | 27752/225000 [64:17:26<328:45:37,  6.00s/it]
 12%|█▏        | 27753/225000 [64:17:34<376:09:01,  6.87s/it]
 12%|█▏        | 27754/225000 [64:17:38<329:25:10,  6.01s/it]
 12%|█▏        | 27755/225000 [64:17:46<357:01:36,  6.52s/it]
 12%|█▏        | 27756/225000 [64:17:51<324:19:09,  5.92s/it]
 12%|█▏        | 27757/225000 [64:17:56<306:49:37,  5.60s/it]
 12%|█▏        | 27758/225000 [64:18:06<394:52:17,  7.21s/it]
 12%|█▏        | 27759/225000 [64:18:11<355:09:11,  6.48s/it]
 12%|█▏        | 27760/225000 [64:18:16<334:38:28,  6.11s/it]
 12%|█▏        | 27761/225000 [64:18:24<357:36:39,  6.53s/it]
 12%|█▏        | 27762/225000 [64:18:28<316:46:18,  5.78s/it]
 12%|█▏        | 27763/225000 [64:18:33<306:06:12,  5.59s/it]
 12%|█▏        | 27764/225000 [64:18:38<294:32:30,  5.38s/it]
 12%|█▏        | 27765/225000 [64:18:51<414:16:27,  7.56s/it]
 12%|█▏        | 27766/225000 [64:18:56<374:20:35,  6.83s/it]
 12%|█▏        | 27767/225000 [64:19:03<384:10:01,  7.01s/it]
 12%|█▏        | 27768/225000 [64:19:10<383:13:38,  6.99s/it]
 12%|█▏        | 27769/225000 [64:19:14<334:10:23,  6.10s/it]
 12%|█▏        | 27770/225000 [64:19:19<314:20:28,  5.74s/it]
 12%|█▏        | 27771/225000 [64:19:24<296:41:18,  5.42s/it]
 12%|█▏        | 27772/225000 [64:19:30<311:15:25,  5.68s/it]
 12%|█▏        | 27773/225000 [64:19:42<416:30:51,  7.60s/it]
 12%|█▏        | 27774/225000 [64:19:45<340:03:52,  6.21s/it]
 12%|█▏        | 27775/225000 [64:19:49<301:00:14,  5.49s/it]
 12%|█▏        | 27776/225000 [64:20:00<395:24:49,  7.22s/it]
 12%|█▏        | 27777/225000 [64:20:04<333:00:49,  6.08s/it]
 12%|█▏        | 27778/225000 [64:20:09<319:22:03,  5.83s/it]
 12%|█▏        | 27779/225000 [64:20:14<315:17:26,  5.76s/it]
 12%|█▏        | 27780/225000 [64:20:27<422:55:17,  7.72s/it]
 12%|█▏        | 27781/225000 [64:20:31<365:21:25,  6.67s/it]
 12%|█▏        | 27782/225000 [64:20:36<331:06:26,  6.04s/it]
 12%|█▏        | 27783/225000 [64:20:40<310:41:26,  5.67s/it]
 12%|█▏        | 27784/225000 [64:20:50<378:49:04,  6.91s/it]
 12%|█▏        | 27785/225000 [64:20:58<399:32:10,  7.29s/it]
 12%|█▏        | 27786/225000 [64:21:03<361:48:57,  6.60s/it]
 12%|█▏        | 27787/225000 [64:21:09<348:34:27,  6.36s/it]
 12%|█▏        | 27788/225000 [64:21:14<331:15:29,  6.05s/it]
 12%|█▏        | 27789/225000 [64:21:20<328:24:44,  6.00s/it]
 12%|█▏        | 27790/225000 [64:21:30<390:06:01,  7.12s/it]
 12%|█▏        | 27791/225000 [64:21:37<378:32:53,  6.91s/it]
 12%|█▏        | 27792/225000 [64:21:46<415:35:35,  7.59s/it]
 12%|█▏        | 27793/225000 [64:21:51<381:31:04,  6.96s/it]
 12%|█▏        | 27794/225000 [64:21:57<357:15:52,  6.52s/it]
 12%|█▏        | 27795/225000 [64:22:02<331:30:19,  6.05s/it]
 12%|█▏        | 27796/225000 [64:22:11<378:18:09,  6.91s/it]
 12%|█▏        | 27797/225000 [64:22:16<353:11:33,  6.45s/it]
 12%|█▏        | 27798/225000 [64:22:24<380:17:25,  6.94s/it]
 12%|█▏        | 27799/225000 [64:22:30<364:48:47,  6.66s/it]
 12%|█▏        | 27800/225000 [64:22:34<328:43:21,  6.00s/it]
                                                             
{'loss': '0.7373', 'grad_norm': '0.6422', 'learning_rate': '4.916e-05', 'epoch': '0.1236', 'num_input_tokens_seen': 110645784, 'train_runtime': '2.318e+05', 'train_tokens_per_second': '477.4'}

 12%|█▏        | 27800/225000 [64:22:34<328:43:21,  6.00s/it]
 12%|█▏        | 27801/225000 [64:22:40<316:44:07,  5.78s/it]
 12%|█▏        | 27802/225000 [64:22:47<346:26:05,  6.32s/it]
 12%|█▏        | 27803/225000 [64:22:55<366:25:14,  6.69s/it]
 12%|█▏        | 27804/225000 [64:23:02<375:27:15,  6.85s/it]
 12%|█▏        | 27805/225000 [64:23:07<343:34:51,  6.27s/it]
 12%|█▏        | 27806/225000 [64:23:15<369:42:54,  6.75s/it]
 12%|█▏        | 27807/225000 [64:23:20<348:33:24,  6.36s/it]
 12%|█▏        | 27808/225000 [64:23:26<334:56:21,  6.11s/it]
 12%|█▏        | 27809/225000 [64:23:30<307:24:26,  5.61s/it]
 12%|█▏        | 27810/225000 [64:23:36<300:48:18,  5.49s/it]
 12%|█▏        | 27811/225000 [64:23:41<299:29:35,  5.47s/it]
 12%|█▏        | 27812/225000 [64:23:46<300:37:14,  5.49s/it]
 12%|█▏        | 27813/225000 [64:23:52<307:10:26,  5.61s/it]
 12%|█▏        | 27814/225000 [64:23:58<303:49:22,  5.55s/it]
 12%|█▏        | 27815/225000 [64:24:08<376:21:54,  6.87s/it]
 12%|█▏        | 27816/225000 [64:24:19<443:13:00,  8.09s/it]
 12%|█▏        | 27817/225000 [64:24:23<387:06:02,  7.07s/it]
 12%|█▏        | 27818/225000 [64:24:28<343:45:18,  6.28s/it]
 12%|█▏        | 27819/225000 [64:24:33<329:13:59,  6.01s/it]
 12%|█▏        | 27820/225000 [64:24:39<322:31:18,  5.89s/it]
 12%|█▏        | 27821/225000 [64:24:44<318:09:57,  5.81s/it]
 12%|█▏        | 27822/225000 [64:24:53<356:15:24,  6.50s/it]
 12%|█▏        | 27823/225000 [64:25:05<452:27:55,  8.26s/it]
 12%|█▏        | 27824/225000 [64:25:10<404:00:35,  7.38s/it]
 12%|█▏        | 27825/225000 [64:25:15<367:26:14,  6.71s/it]
 12%|█▏        | 27826/225000 [64:25:20<341:38:21,  6.24s/it]
 12%|█▏        | 27827/225000 [64:25:25<320:12:06,  5.85s/it]
 12%|█▏        | 27828/225000 [64:25:31<311:53:42,  5.69s/it]
 12%|█▏        | 27829/225000 [64:25:41<385:31:29,  7.04s/it]
 12%|█▏        | 27830/225000 [64:25:50<414:34:55,  7.57s/it]
 12%|█▏        | 27831/225000 [64:25:56<390:18:25,  7.13s/it]
 12%|█▏        | 27832/225000 [64:26:00<344:37:31,  6.29s/it]
 12%|█▏        | 27833/225000 [64:26:09<380:29:57,  6.95s/it]
 12%|█▏        | 27834/225000 [64:26:15<366:17:58,  6.69s/it]
 12%|█▏        | 27835/225000 [64:26:22<381:58:51,  6.97s/it]
 12%|█▏        | 27836/225000 [64:26:29<377:56:39,  6.90s/it]
 12%|█▏        | 27837/225000 [64:26:33<331:15:13,  6.05s/it]
 12%|█▏        | 27838/225000 [64:26:43<400:06:12,  7.31s/it]
 12%|█▏        | 27839/225000 [64:26:47<346:19:02,  6.32s/it]
 12%|█▏        | 27840/225000 [64:26:53<340:26:31,  6.22s/it]
 12%|█▏        | 27841/225000 [64:27:01<368:42:23,  6.73s/it]
 12%|█▏        | 27842/225000 [64:27:06<330:00:13,  6.03s/it]
 12%|█▏        | 27843/225000 [64:27:16<393:38:44,  7.19s/it]
 12%|█▏        | 27844/225000 [64:27:19<330:45:06,  6.04s/it]
 12%|█▏        | 27845/225000 [64:27:28<375:15:00,  6.85s/it]
 12%|█▏        | 27846/225000 [64:27:32<338:42:52,  6.18s/it]
 12%|█▏        | 27847/225000 [64:27:41<377:26:12,  6.89s/it]
 12%|█▏        | 27848/225000 [64:27:48<381:10:54,  6.96s/it]
 12%|█▏        | 27849/225000 [64:27:52<332:54:57,  6.08s/it]
 12%|█▏        | 27850/225000 [64:28:00<357:13:47,  6.52s/it]
                                                             
{'loss': '0.7228', 'grad_norm': '0.7066', 'learning_rate': '4.916e-05', 'epoch': '0.1238', 'num_input_tokens_seen': 110789472, 'train_runtime': '2.321e+05', 'train_tokens_per_second': '477.4'}

 12%|█▏        | 27850/225000 [64:28:00<357:13:47,  6.52s/it]
 12%|█▏        | 27851/225000 [64:28:04<322:03:38,  5.88s/it]
 12%|█▏        | 27852/225000 [64:28:08<294:14:27,  5.37s/it]
 12%|█▏        | 27853/225000 [64:28:17<343:19:53,  6.27s/it]
 12%|█▏        | 27854/225000 [64:28:26<401:59:25,  7.34s/it]
 12%|█▏        | 27855/225000 [64:28:31<356:45:35,  6.51s/it]
 12%|█▏        | 27856/225000 [64:28:35<320:46:32,  5.86s/it]
 12%|█▏        | 27857/225000 [64:28:47<419:01:25,  7.65s/it]
 12%|█▏        | 27858/225000 [64:28:52<369:39:16,  6.75s/it]
 12%|█▏        | 27859/225000 [64:28:56<330:12:11,  6.03s/it]
 12%|█▏        | 27860/225000 [64:29:01<309:48:22,  5.66s/it]
 12%|█▏        | 27861/225000 [64:29:06<299:42:27,  5.47s/it]
 12%|█▏        | 27862/225000 [64:29:19<429:13:58,  7.84s/it]
 12%|█▏        | 27863/225000 [64:29:23<365:39:58,  6.68s/it]
 12%|█▏        | 27864/225000 [64:29:28<332:29:41,  6.07s/it]
 12%|█▏        | 27865/225000 [64:29:33<311:34:06,  5.69s/it]
 12%|█▏        | 27866/225000 [64:29:38<303:42:46,  5.55s/it]
 12%|█▏        | 27867/225000 [64:29:43<295:02:29,  5.39s/it]
 12%|█▏        | 27868/225000 [64:29:57<432:58:55,  7.91s/it]
 12%|█▏        | 27869/225000 [64:30:04<417:52:05,  7.63s/it]
 12%|█▏        | 27870/225000 [64:30:09<373:57:07,  6.83s/it]
 12%|█▏        | 27871/225000 [64:30:13<338:57:34,  6.19s/it]
 12%|█▏        | 27872/225000 [64:30:18<316:09:38,  5.77s/it]
 12%|█▏        | 27873/225000 [64:30:32<453:07:28,  8.28s/it]
 12%|█▏        | 27874/225000 [64:30:37<393:38:35,  7.19s/it]
 12%|█▏        | 27875/225000 [64:30:47<434:02:36,  7.93s/it]
 12%|█▏        | 27876/225000 [64:30:51<374:37:53,  6.84s/it]
 12%|█▏        | 27877/225000 [64:30:56<340:22:26,  6.22s/it]
 12%|█▏        | 27878/225000 [64:31:00<315:01:04,  5.75s/it]
 12%|█▏        | 27879/225000 [64:31:14<438:22:42,  8.01s/it]
 12%|█▏        | 27880/225000 [64:31:19<402:41:16,  7.35s/it]
 12%|█▏        | 27881/225000 [64:31:24<353:44:05,  6.46s/it]
 12%|█▏        | 27882/225000 [64:31:32<386:44:42,  7.06s/it]
 12%|█▏        | 27883/225000 [64:31:41<412:06:34,  7.53s/it]
 12%|█▏        | 27884/225000 [64:31:46<375:48:35,  6.86s/it]
 12%|█▏        | 27885/225000 [64:31:52<354:25:35,  6.47s/it]
 12%|█▏        | 27886/225000 [64:31:57<329:37:48,  6.02s/it]
 12%|█▏        | 27887/225000 [64:32:02<318:59:30,  5.83s/it]
 12%|█▏        | 27888/225000 [64:32:07<303:08:23,  5.54s/it]
 12%|█▏        | 27889/225000 [64:32:12<292:45:04,  5.35s/it]
 12%|█▏        | 27890/225000 [64:32:17<290:26:02,  5.30s/it]
 12%|█▏        | 27891/225000 [64:32:27<370:59:07,  6.78s/it]
 12%|█▏        | 27892/225000 [64:32:38<438:48:12,  8.01s/it]
 12%|█▏        | 27893/225000 [64:32:43<390:20:23,  7.13s/it]
 12%|█▏        | 27894/225000 [64:32:49<360:11:55,  6.58s/it]
 12%|█▏        | 27895/225000 [64:32:54<342:08:59,  6.25s/it]
 12%|█▏        | 27896/225000 [64:32:59<319:29:40,  5.84s/it]
 12%|█▏        | 27897/225000 [64:33:04<302:59:03,  5.53s/it]
 12%|█▏        | 27898/225000 [64:33:09<303:55:37,  5.55s/it]
 12%|█▏        | 27899/225000 [64:33:15<304:21:50,  5.56s/it]
 12%|█▏        | 27900/225000 [64:33:26<395:50:15,  7.23s/it]
                                                             
{'loss': '0.7475', 'grad_norm': '0.717', 'learning_rate': '4.915e-05', 'epoch': '0.124', 'num_input_tokens_seen': 110931624, 'train_runtime': '2.324e+05', 'train_tokens_per_second': '477.3'}

 12%|█▏        | 27900/225000 [64:33:26<395:50:15,  7.23s/it]
 12%|█▏        | 27901/225000 [64:33:35<428:18:49,  7.82s/it]
 12%|█▏        | 27902/225000 [64:33:40<380:00:19,  6.94s/it]
 12%|█▏        | 27903/225000 [64:33:45<346:43:53,  6.33s/it]
 12%|█▏        | 27904/225000 [64:33:50<321:53:01,  5.88s/it]
 12%|█▏        | 27905/225000 [64:33:57<338:33:46,  6.18s/it]
 12%|█▏        | 27906/225000 [64:34:08<417:50:56,  7.63s/it]
 12%|█▏        | 27907/225000 [64:34:11<353:28:01,  6.46s/it]
 12%|█▏        | 27908/225000 [64:34:17<339:35:26,  6.20s/it]
 12%|█▏        | 27909/225000 [64:34:23<333:18:53,  6.09s/it]
 12%|█▏        | 27910/225000 [64:34:28<322:16:19,  5.89s/it]
 12%|█▏        | 27911/225000 [64:34:34<321:04:45,  5.86s/it]
 12%|█▏        | 27912/225000 [64:34:46<414:00:26,  7.56s/it]
 12%|█▏        | 27913/225000 [64:34:55<437:43:47,  8.00s/it]
 12%|█▏        | 27914/225000 [64:34:58<365:34:42,  6.68s/it]
 12%|█▏        | 27915/225000 [64:35:08<422:41:54,  7.72s/it]
 12%|█▏        | 27916/225000 [64:35:12<360:28:04,  6.58s/it]
 12%|█▏        | 27917/225000 [64:35:22<407:51:43,  7.45s/it]
 12%|█▏        | 27918/225000 [64:35:25<345:28:01,  6.31s/it]
 12%|█▏        | 27919/225000 [64:35:33<371:21:58,  6.78s/it]
 12%|█▏        | 27920/225000 [64:35:38<342:14:40,  6.25s/it]
 12%|█▏        | 27921/225000 [64:35:47<376:37:53,  6.88s/it]
 12%|█▏        | 27922/225000 [64:35:51<326:27:17,  5.96s/it]
 12%|█▏        | 27923/225000 [64:36:01<393:06:43,  7.18s/it]
 12%|█▏        | 27924/225000 [64:36:05<340:08:24,  6.21s/it]
 12%|█▏        | 27925/225000 [64:36:10<331:34:14,  6.06s/it]
 12%|█▏        | 27926/225000 [64:36:16<329:07:06,  6.01s/it]
 12%|█▏        | 27927/225000 [64:36:22<325:34:06,  5.95s/it]
 12%|█▏        | 27928/225000 [64:36:28<324:23:17,  5.93s/it]
 12%|█▏        | 27929/225000 [64:36:35<339:57:59,  6.21s/it]
 12%|█▏        | 27930/225000 [64:36:41<343:13:54,  6.27s/it]
 12%|█▏        | 27931/225000 [64:36:46<327:26:19,  5.98s/it]
 12%|█▏        | 27932/225000 [64:36:51<303:28:15,  5.54s/it]
 12%|█▏        | 27933/225000 [64:37:03<409:16:47,  7.48s/it]
 12%|█▏        | 27934/225000 [64:37:08<377:30:58,  6.90s/it]
 12%|█▏        | 27935/225000 [64:37:14<360:01:37,  6.58s/it]
 12%|█▏        | 27936/225000 [64:37:19<332:41:58,  6.08s/it]
 12%|█▏        | 27937/225000 [64:37:30<409:11:24,  7.48s/it]
 12%|█▏        | 27938/225000 [64:37:34<352:40:17,  6.44s/it]
 12%|█▏        | 27939/225000 [64:37:39<327:46:45,  5.99s/it]
 12%|█▏        | 27940/225000 [64:37:44<307:24:56,  5.62s/it]
 12%|█▏        | 27941/225000 [64:37:57<431:57:37,  7.89s/it]
 12%|█▏        | 27942/225000 [64:38:01<366:37:29,  6.70s/it]
 12%|█▏        | 27943/225000 [64:38:11<423:50:02,  7.74s/it]
 12%|█▏        | 27944/225000 [64:38:15<356:34:49,  6.51s/it]
 12%|█▏        | 27945/225000 [64:38:20<336:58:36,  6.16s/it]
 12%|█▏        | 27946/225000 [64:38:26<331:23:28,  6.05s/it]
 12%|█▏        | 27947/225000 [64:38:31<322:10:36,  5.89s/it]
 12%|█▏        | 27948/225000 [64:38:37<313:14:53,  5.72s/it]
 12%|█▏        | 27949/225000 [64:38:47<384:18:53,  7.02s/it]
 12%|█▏        | 27950/225000 [64:38:57<437:21:21,  7.99s/it]
                                                             
{'loss': '0.7369', 'grad_norm': '0.7159', 'learning_rate': '4.915e-05', 'epoch': '0.1242', 'num_input_tokens_seen': 111074616, 'train_runtime': '2.327e+05', 'train_tokens_per_second': '477.3'}

 12%|█▏        | 27950/225000 [64:38:57<437:21:21,  7.99s/it]
 12%|█▏        | 27951/225000 [64:39:02<385:17:28,  7.04s/it]
 12%|█▏        | 27952/225000 [64:39:10<401:45:05,  7.34s/it]
 12%|█▏        | 27953/225000 [64:39:15<368:37:38,  6.73s/it]
 12%|█▏        | 27954/225000 [64:39:25<428:15:30,  7.82s/it]
 12%|█▏        | 27955/225000 [64:39:30<373:21:51,  6.82s/it]
 12%|█▏        | 27956/225000 [64:39:35<342:02:50,  6.25s/it]
 12%|█▏        | 27957/225000 [64:39:39<309:20:54,  5.65s/it]
 12%|█▏        | 27958/225000 [64:39:51<410:37:41,  7.50s/it]
 12%|█▏        | 27959/225000 [64:39:55<349:40:47,  6.39s/it]
 12%|█▏        | 27960/225000 [64:40:06<426:59:38,  7.80s/it]
 12%|█▏        | 27961/225000 [64:40:10<374:34:43,  6.84s/it]
 12%|█▏        | 27962/225000 [64:40:18<394:37:34,  7.21s/it]
 12%|█▏        | 27963/225000 [64:40:22<335:53:47,  6.14s/it]
 12%|█▏        | 27964/225000 [64:40:29<351:46:29,  6.43s/it]
 12%|█▏        | 27965/225000 [64:40:36<356:20:22,  6.51s/it]
 12%|█▏        | 27966/225000 [64:40:43<372:29:50,  6.81s/it]
 12%|█▏        | 27967/225000 [64:40:47<328:24:48,  6.00s/it]
 12%|█▏        | 27968/225000 [64:40:51<292:20:12,  5.34s/it]
 12%|█▏        | 27969/225000 [64:41:02<386:38:24,  7.06s/it]
 12%|█▏        | 27970/225000 [64:41:08<363:52:06,  6.65s/it]
 12%|█▏        | 27971/225000 [64:41:16<380:45:10,  6.96s/it]
 12%|█▏        | 27972/225000 [64:41:21<346:17:19,  6.33s/it]
 12%|█▏        | 27973/225000 [64:41:26<325:29:49,  5.95s/it]
 12%|█▏        | 27974/225000 [64:41:31<315:56:31,  5.77s/it]
 12%|█▏        | 27975/225000 [64:41:44<430:04:30,  7.86s/it]
 12%|█▏        | 27976/225000 [64:41:48<366:33:11,  6.70s/it]
 12%|█▏        | 27977/225000 [64:41:57<401:23:18,  7.33s/it]
 12%|█▏        | 27978/225000 [64:42:01<360:49:28,  6.59s/it]
 12%|█▏        | 27979/225000 [64:42:09<384:06:05,  7.02s/it]
 12%|█▏        | 27980/225000 [64:42:18<405:08:46,  7.40s/it]
 12%|█▏        | 27981/225000 [64:42:22<352:17:09,  6.44s/it]
 12%|█▏        | 27982/225000 [64:42:26<313:06:13,  5.72s/it]
 12%|█▏        | 27983/225000 [64:42:35<372:44:05,  6.81s/it]
 12%|█▏        | 27984/225000 [64:42:42<366:50:10,  6.70s/it]
 12%|█▏        | 27985/225000 [64:42:47<339:07:28,  6.20s/it]
 12%|█▏        | 27986/225000 [64:42:56<383:17:14,  7.00s/it]
 12%|█▏        | 27987/225000 [64:42:59<329:55:53,  6.03s/it]
 12%|█▏        | 27988/225000 [64:43:08<370:29:38,  6.77s/it]
 12%|█▏        | 27989/225000 [64:43:13<345:19:18,  6.31s/it]
 12%|█▏        | 27990/225000 [64:43:19<344:45:29,  6.30s/it]
 12%|█▏        | 27991/225000 [64:43:25<326:57:20,  5.97s/it]
 12%|█▏        | 27992/225000 [64:43:33<362:17:07,  6.62s/it]
 12%|█▏        | 27993/225000 [64:43:36<313:55:59,  5.74s/it]
 12%|█▏        | 27994/225000 [64:43:44<349:55:59,  6.39s/it]
 12%|█▏        | 27995/225000 [64:43:51<359:38:09,  6.57s/it]
 12%|█▏        | 27996/225000 [64:43:55<312:38:27,  5.71s/it]
 12%|█▏        | 27997/225000 [64:43:59<282:53:46,  5.17s/it]
 12%|█▏        | 27998/225000 [64:44:07<331:29:22,  6.06s/it]
 12%|█▏        | 27999/225000 [64:44:17<387:57:38,  7.09s/it]
 12%|█▏        | 28000/225000 [64:44:21<348:35:03,  6.37s/it]
                                                             
{'loss': '0.7464', 'grad_norm': '0.5725', 'learning_rate': '4.914e-05', 'epoch': '0.1244', 'num_input_tokens_seen': 111218896, 'train_runtime': '2.331e+05', 'train_tokens_per_second': '477.2'}

 12%|█▏        | 28000/225000 [64:44:21<348:35:03,  6.37s/it]
 12%|█▏        | 28001/225000 [64:44:31<405:27:38,  7.41s/it]
 12%|█▏        | 28002/225000 [64:44:35<349:29:44,  6.39s/it]
 12%|█▏        | 28003/225000 [64:44:40<320:37:59,  5.86s/it]
 12%|█▏        | 28004/225000 [64:44:45<306:13:10,  5.60s/it]
 12%|█▏        | 28005/225000 [64:44:58<425:08:30,  7.77s/it]
 12%|█▏        | 28006/225000 [64:45:02<374:44:46,  6.85s/it]
 12%|█▏        | 28007/225000 [64:45:07<341:58:06,  6.25s/it]
 12%|█▏        | 28008/225000 [64:45:12<317:30:45,  5.80s/it]
 12%|█▏        | 28009/225000 [64:45:17<299:48:48,  5.48s/it]
 12%|█▏        | 28010/225000 [64:45:29<408:30:35,  7.47s/it]
 12%|█▏        | 28011/225000 [64:45:36<409:10:11,  7.48s/it]
 12%|█▏        | 28012/225000 [64:45:40<349:17:27,  6.38s/it]
 12%|█▏        | 28013/225000 [64:45:45<330:54:20,  6.05s/it]
 12%|█▏        | 28014/225000 [64:45:51<328:34:52,  6.00s/it]
 12%|█▏        | 28015/225000 [64:45:57<325:52:45,  5.96s/it]
 12%|█▏        | 28016/225000 [64:46:05<360:18:05,  6.58s/it]
 12%|█▏        | 28017/225000 [64:46:11<346:27:04,  6.33s/it]
 12%|█▏        | 28018/225000 [64:46:16<331:41:07,  6.06s/it]
 12%|█▏        | 28019/225000 [64:46:22<322:54:11,  5.90s/it]
 12%|█▏        | 28020/225000 [64:46:27<311:35:23,  5.69s/it]
 12%|█▏        | 28021/225000 [64:46:35<346:18:00,  6.33s/it]
 12%|█▏        | 28022/225000 [64:46:48<466:30:29,  8.53s/it]
 12%|█▏        | 28023/225000 [64:46:53<396:04:08,  7.24s/it]
 12%|█▏        | 28024/225000 [64:47:00<393:11:19,  7.19s/it]
 12%|█▏        | 28025/225000 [64:47:04<348:08:59,  6.36s/it]
 12%|█▏        | 28026/225000 [64:47:12<374:58:44,  6.85s/it]
 12%|█▏        | 28027/225000 [64:47:16<317:46:34,  5.81s/it]
 12%|█▏        | 28028/225000 [64:47:25<373:31:47,  6.83s/it]
 12%|█▏        | 28029/225000 [64:47:30<348:51:49,  6.38s/it]
 12%|█▏        | 28030/225000 [64:47:36<338:12:27,  6.18s/it]
 12%|█▏        | 28031/225000 [64:47:41<327:10:18,  5.98s/it]
 12%|█▏        | 28032/225000 [64:47:47<314:26:28,  5.75s/it]
 12%|█▏        | 28033/225000 [64:47:52<306:02:37,  5.59s/it]
 12%|█▏        | 28034/225000 [64:47:58<310:25:54,  5.67s/it]
 12%|█▏        | 28035/225000 [64:48:06<359:29:43,  6.57s/it]
 12%|█▏        | 28036/225000 [64:48:15<394:15:08,  7.21s/it]
 12%|█▏        | 28037/225000 [64:48:19<347:17:32,  6.35s/it]
 12%|█▏        | 28038/225000 [64:48:24<321:15:44,  5.87s/it]
 12%|█▏        | 28039/225000 [64:48:34<385:16:33,  7.04s/it]
 12%|█▏        | 28040/225000 [64:48:42<402:23:39,  7.35s/it]
 12%|█▏        | 28041/225000 [64:48:49<396:18:35,  7.24s/it]
 12%|█▏        | 28042/225000 [64:48:53<347:40:02,  6.35s/it]
 12%|█▏        | 28043/225000 [64:48:58<323:15:06,  5.91s/it]
 12%|█▏        | 28044/225000 [64:49:03<308:24:47,  5.64s/it]
 12%|█▏        | 28045/225000 [64:49:08<295:50:14,  5.41s/it]
 12%|█▏        | 28046/225000 [64:49:19<385:33:51,  7.05s/it]
 12%|█▏        | 28047/225000 [64:49:27<395:46:53,  7.23s/it]
 12%|█▏        | 28048/225000 [64:49:31<351:20:14,  6.42s/it]
 12%|█▏        | 28049/225000 [64:49:35<317:24:23,  5.80s/it]
 12%|█▏        | 28050/225000 [64:49:46<404:07:48,  7.39s/it]
                                                             
{'loss': '0.7449', 'grad_norm': '0.6888', 'learning_rate': '4.914e-05', 'epoch': '0.1247', 'num_input_tokens_seen': 111362992, 'train_runtime': '2.334e+05', 'train_tokens_per_second': '477.2'}

 12%|█▏        | 28050/225000 [64:49:46<404:07:48,  7.39s/it]
 12%|█▏        | 28051/225000 [64:49:51<351:03:31,  6.42s/it]
 12%|█▏        | 28052/225000 [64:49:59<376:15:12,  6.88s/it]
 12%|█▏        | 28053/225000 [64:50:03<343:27:46,  6.28s/it]
 12%|█▏        | 28054/225000 [64:50:12<375:16:42,  6.86s/it]
 12%|█▏        | 28055/225000 [64:50:17<354:05:16,  6.47s/it]
 12%|█▏        | 28056/225000 [64:50:25<379:43:36,  6.94s/it]
 12%|█▏        | 28057/225000 [64:50:29<327:33:51,  5.99s/it]
 12%|█▏        | 28058/225000 [64:50:38<383:55:05,  7.02s/it]
 12%|█▏        | 28059/225000 [64:50:44<353:17:28,  6.46s/it]
 12%|█▏        | 28060/225000 [64:50:49<331:07:13,  6.05s/it]
 12%|█▏        | 28061/225000 [64:50:54<315:56:08,  5.78s/it]
 12%|█▏        | 28062/225000 [64:50:59<311:46:56,  5.70s/it]
 12%|█▏        | 28063/225000 [64:51:11<403:57:34,  7.38s/it]
 12%|█▏        | 28064/225000 [64:51:19<420:32:07,  7.69s/it]
 12%|█▏        | 28065/225000 [64:51:25<383:21:12,  7.01s/it]
 12%|█▏        | 28066/225000 [64:51:30<352:34:02,  6.45s/it]
 12%|█▏        | 28067/225000 [64:51:35<342:39:04,  6.26s/it]
 12%|█▏        | 28068/225000 [64:51:41<325:09:37,  5.94s/it]
 12%|█▏        | 28069/225000 [64:51:46<316:37:00,  5.79s/it]
 12%|█▏        | 28070/225000 [64:51:58<410:46:29,  7.51s/it]
 12%|█▏        | 28071/225000 [64:52:01<348:22:19,  6.37s/it]
 12%|█▏        | 28072/225000 [64:52:11<405:59:40,  7.42s/it]
 12%|█▏        | 28073/225000 [64:52:18<402:59:58,  7.37s/it]
 12%|█▏        | 28074/225000 [64:52:22<344:12:18,  6.29s/it]
 12%|█▏        | 28075/225000 [64:52:31<390:02:49,  7.13s/it]
 12%|█▏        | 28076/225000 [64:52:37<370:22:10,  6.77s/it]
 12%|█▏        | 28077/225000 [64:52:42<331:08:53,  6.05s/it]
 12%|█▏        | 28078/225000 [64:52:50<368:44:35,  6.74s/it]
 12%|█▏        | 28079/225000 [64:52:54<315:41:59,  5.77s/it]
 12%|█▏        | 28080/225000 [64:53:02<356:35:00,  6.52s/it]
 12%|█▏        | 28081/225000 [64:53:06<324:39:46,  5.94s/it]
 12%|█▏        | 28082/225000 [64:53:15<367:36:57,  6.72s/it]
 12%|█▏        | 28083/225000 [64:53:19<331:56:11,  6.07s/it]
 12%|█▏        | 28084/225000 [64:53:27<360:11:08,  6.58s/it]
 12%|█▏        | 28085/225000 [64:53:32<331:03:06,  6.05s/it]
 12%|█▏        | 28086/225000 [64:53:37<309:12:01,  5.65s/it]
 12%|█▏        | 28087/225000 [64:53:42<295:21:59,  5.40s/it]
 12%|█▏        | 28088/225000 [64:53:46<286:12:59,  5.23s/it]
 12%|█▏        | 28089/225000 [64:53:59<407:53:26,  7.46s/it]
 12%|█▏        | 28090/225000 [64:54:05<378:01:43,  6.91s/it]
 12%|█▏        | 28091/225000 [64:54:10<350:20:50,  6.41s/it]
 12%|█▏        | 28092/225000 [64:54:15<328:09:42,  6.00s/it]
 12%|█▏        | 28093/225000 [64:54:20<306:56:19,  5.61s/it]
 12%|█▏        | 28094/225000 [64:54:24<290:51:07,  5.32s/it]
 12%|█▏        | 28095/225000 [64:54:37<407:43:28,  7.45s/it]
 12%|█▏        | 28096/225000 [64:54:43<394:26:21,  7.21s/it]
 12%|█▏        | 28097/225000 [64:54:52<413:41:40,  7.56s/it]
 12%|█▏        | 28098/225000 [64:54:56<355:24:19,  6.50s/it]
 12%|█▏        | 28099/225000 [64:55:01<327:41:44,  5.99s/it]
 12%|█▏        | 28100/225000 [64:55:06<310:48:06,  5.68s/it]
                                                             
{'loss': '0.7271', 'grad_norm': '1.093', 'learning_rate': '4.913e-05', 'epoch': '0.1249', 'num_input_tokens_seen': 111506768, 'train_runtime': '2.337e+05', 'train_tokens_per_second': '477.1'}

 12%|█▏        | 28100/225000 [64:55:06<310:48:06,  5.68s/it]
 12%|█▏        | 28101/225000 [64:55:12<316:07:21,  5.78s/it]
 12%|█▏        | 28102/225000 [64:55:23<406:51:17,  7.44s/it]
 12%|█▏        | 28103/225000 [64:55:27<346:50:09,  6.34s/it]
 12%|█▏        | 28104/225000 [64:55:31<309:43:30,  5.66s/it]
 12%|█▏        | 28105/225000 [64:55:39<351:07:00,  6.42s/it]
 12%|█▏        | 28106/225000 [64:55:47<376:37:05,  6.89s/it]
 12%|█▏        | 28107/225000 [64:55:53<357:06:34,  6.53s/it]
 12%|█▏        | 28108/225000 [64:55:58<339:49:27,  6.21s/it]
 12%|█▏        | 28109/225000 [64:56:04<328:46:49,  6.01s/it]
 12%|█▏        | 28110/225000 [64:56:09<319:25:44,  5.84s/it]
 12%|█▏        | 28111/225000 [64:56:16<337:51:02,  6.18s/it]
 12%|█▏        | 28112/225000 [64:56:26<391:58:41,  7.17s/it]
 12%|█▏        | 28113/225000 [64:56:31<365:39:22,  6.69s/it]
 12%|█▏        | 28114/225000 [64:56:37<350:38:34,  6.41s/it]
 12%|█▏        | 28115/225000 [64:56:43<344:31:36,  6.30s/it]
 12%|█▏        | 28116/225000 [64:56:50<356:38:14,  6.52s/it]
 12%|█▏        | 28117/225000 [64:56:59<401:32:46,  7.34s/it]
 12%|█▏        | 28118/225000 [64:57:03<341:44:15,  6.25s/it]
 12%|█▏        | 28119/225000 [64:57:14<418:40:39,  7.66s/it]
 12%|█▏        | 28120/225000 [64:57:21<409:54:37,  7.50s/it]
 12%|█▏        | 28121/225000 [64:57:24<341:27:23,  6.24s/it]
 12%|█▏        | 28122/225000 [64:57:33<374:36:48,  6.85s/it]
 12%|█▏        | 28123/225000 [64:57:38<350:37:19,  6.41s/it]
 12%|█▏        | 28124/225000 [64:57:42<315:34:29,  5.77s/it]
 12%|█▎        | 28125/225000 [64:57:55<436:09:01,  7.98s/it]
 13%|█▎        | 28126/225000 [64:57:59<366:03:30,  6.69s/it]
 13%|█▎        | 28127/225000 [64:58:08<398:40:13,  7.29s/it]
 13%|█▎        | 28128/225000 [64:58:12<343:25:27,  6.28s/it]
 13%|█▎        | 28129/225000 [64:58:20<385:49:13,  7.06s/it]
 13%|█▎        | 28130/225000 [64:58:25<349:30:40,  6.39s/it]
 13%|█▎        | 28131/225000 [64:58:32<354:40:00,  6.49s/it]
 13%|█▎        | 28132/225000 [64:58:40<383:04:35,  7.01s/it]
 13%|█▎        | 28133/225000 [64:58:44<331:11:19,  6.06s/it]
 13%|█▎        | 28134/225000 [64:58:52<357:12:52,  6.53s/it]
 13%|█▎        | 28135/225000 [64:58:57<335:41:15,  6.14s/it]
 13%|█▎        | 28136/225000 [64:59:02<325:06:50,  5.95s/it]
 13%|█▎        | 28137/225000 [64:59:07<307:15:04,  5.62s/it]
 13%|█▎        | 28138/225000 [64:59:12<291:15:01,  5.33s/it]
 13%|█▎        | 28139/225000 [64:59:23<393:09:59,  7.19s/it]
 13%|█▎        | 28140/225000 [64:59:30<384:12:35,  7.03s/it]
 13%|█▎        | 28141/225000 [64:59:37<379:04:45,  6.93s/it]
 13%|█▎        | 28142/225000 [64:59:41<336:49:46,  6.16s/it]
 13%|█▎        | 28143/225000 [64:59:46<321:58:19,  5.89s/it]
 13%|█▎        | 28144/225000 [64:59:59<427:26:17,  7.82s/it]
 13%|█▎        | 28145/225000 [65:00:03<370:52:46,  6.78s/it]
 13%|█▎        | 28146/225000 [65:00:08<331:35:43,  6.06s/it]
 13%|█▎        | 28147/225000 [65:00:12<310:24:36,  5.68s/it]
 13%|█▎        | 28148/225000 [65:00:23<384:53:23,  7.04s/it]
 13%|█▎        | 28149/225000 [65:00:30<400:08:55,  7.32s/it]
 13%|█▎        | 28150/225000 [65:00:35<361:24:30,  6.61s/it]
                                                             
{'loss': '0.7273', 'grad_norm': '0.7073', 'learning_rate': '4.913e-05', 'epoch': '0.1251', 'num_input_tokens_seen': 111651664, 'train_runtime': '2.34e+05', 'train_tokens_per_second': '477.1'}

 13%|█▎        | 28150/225000 [65:00:35<361:24:30,  6.61s/it]
 13%|█▎        | 28151/225000 [65:00:39<315:57:58,  5.78s/it]
 13%|█▎        | 28152/225000 [65:00:47<343:18:48,  6.28s/it]
 13%|█▎        | 28153/225000 [65:00:56<396:47:13,  7.26s/it]
 13%|█▎        | 28154/225000 [65:01:04<398:27:14,  7.29s/it]
 13%|█▎        | 28155/225000 [65:01:07<339:40:39,  6.21s/it]
 13%|█▎        | 28156/225000 [65:01:16<384:51:09,  7.04s/it]
 13%|█▎        | 28157/225000 [65:01:21<344:05:57,  6.29s/it]
 13%|█▎        | 28158/225000 [65:01:28<356:47:04,  6.53s/it]
 13%|█▎        | 28159/225000 [65:01:33<333:58:39,  6.11s/it]
 13%|█▎        | 28160/225000 [65:01:38<323:02:06,  5.91s/it]
 13%|█▎        | 28161/225000 [65:01:44<311:18:36,  5.69s/it]
 13%|█▎        | 28162/225000 [65:01:48<296:36:40,  5.42s/it]
 13%|█▎        | 28163/225000 [65:01:54<291:46:46,  5.34s/it]
 13%|█▎        | 28164/225000 [65:02:02<346:10:19,  6.33s/it]
 13%|█▎        | 28165/225000 [65:02:14<439:10:45,  8.03s/it]
 13%|█▎        | 28166/225000 [65:02:19<378:55:33,  6.93s/it]
 13%|█▎        | 28167/225000 [65:02:23<330:03:27,  6.04s/it]
 13%|█▎        | 28168/225000 [65:02:28<312:46:13,  5.72s/it]
 13%|█▎        | 28169/225000 [65:02:33<305:15:01,  5.58s/it]
 13%|█▎        | 28170/225000 [65:02:38<292:56:47,  5.36s/it]
 13%|█▎        | 28171/225000 [65:02:44<305:56:22,  5.60s/it]
 13%|█▎        | 28172/225000 [65:02:57<430:01:21,  7.87s/it]
 13%|█▎        | 28173/225000 [65:03:03<405:00:27,  7.41s/it]
 13%|█▎        | 28174/225000 [65:03:07<352:05:42,  6.44s/it]
 13%|█▎        | 28175/225000 [65:03:14<359:03:44,  6.57s/it]
 13%|█▎        | 28176/225000 [65:03:23<393:00:23,  7.19s/it]
 13%|█▎        | 28177/225000 [65:03:30<382:30:35,  7.00s/it]
 13%|█▎        | 28178/225000 [65:03:35<352:54:32,  6.45s/it]
 13%|█▎        | 28179/225000 [65:03:44<395:42:56,  7.24s/it]
 13%|█▎        | 28180/225000 [65:03:51<399:39:22,  7.31s/it]
 13%|█▎        | 28181/225000 [65:03:56<359:33:18,  6.58s/it]
 13%|█▎        | 28182/225000 [65:04:03<368:20:44,  6.74s/it]
 13%|█▎        | 28183/225000 [65:04:07<317:19:37,  5.80s/it]
 13%|█▎        | 28184/225000 [65:04:15<355:03:58,  6.49s/it]
 13%|█▎        | 28185/225000 [65:04:22<369:29:10,  6.76s/it]
 13%|█▎        | 28186/225000 [65:04:27<342:10:49,  6.26s/it]
 13%|█▎        | 28187/225000 [65:04:33<322:47:21,  5.90s/it]
 13%|█▎        | 28188/225000 [65:04:38<319:11:15,  5.84s/it]
 13%|█▎        | 28189/225000 [65:04:44<312:53:36,  5.72s/it]
 13%|█▎        | 28190/225000 [65:04:56<413:31:45,  7.56s/it]
 13%|█▎        | 28191/225000 [65:05:02<388:15:05,  7.10s/it]
 13%|█▎        | 28192/225000 [65:05:08<384:24:21,  7.03s/it]
 13%|█▎        | 28193/225000 [65:05:14<359:28:39,  6.58s/it]
 13%|█▎        | 28194/225000 [65:05:22<378:22:04,  6.92s/it]
 13%|█▎        | 28195/225000 [65:05:27<350:13:28,  6.41s/it]
 13%|█▎        | 28196/225000 [65:05:31<317:49:40,  5.81s/it]
 13%|█▎        | 28197/225000 [65:05:36<292:09:38,  5.34s/it]
 13%|█▎        | 28198/225000 [65:05:48<406:11:39,  7.43s/it]
 13%|█▎        | 28199/225000 [65:05:52<349:50:17,  6.40s/it]
 13%|█▎        | 28200/225000 [65:05:57<323:27:28,  5.92s/it]
                                                             
{'loss': '0.727', 'grad_norm': '0.6956', 'learning_rate': '4.912e-05', 'epoch': '0.1253', 'num_input_tokens_seen': 111794400, 'train_runtime': '2.344e+05', 'train_tokens_per_second': '477'}

 13%|█▎        | 28200/225000 [65:05:57<323:27:28,  5.92s/it]
 13%|█▎        | 28201/225000 [65:06:02<315:21:38,  5.77s/it]
 13%|█▎        | 28202/225000 [65:06:07<306:26:54,  5.61s/it]
 13%|█▎        | 28203/225000 [65:06:13<310:55:34,  5.69s/it]
 13%|█▎        | 28204/225000 [65:06:26<425:21:16,  7.78s/it]
 13%|█▎        | 28205/225000 [65:06:32<402:11:58,  7.36s/it]
 13%|█▎        | 28206/225000 [65:06:39<398:03:23,  7.28s/it]
 13%|█▎        | 28207/225000 [65:06:43<338:48:38,  6.20s/it]
 13%|█▎        | 28208/225000 [65:06:53<407:18:33,  7.45s/it]
 13%|█▎        | 28209/225000 [65:06:57<342:54:24,  6.27s/it]
 13%|█▎        | 28210/225000 [65:07:04<355:31:58,  6.50s/it]
 13%|█▎        | 28211/225000 [65:07:09<324:56:51,  5.94s/it]
 13%|█▎        | 28212/225000 [65:07:16<349:34:56,  6.40s/it]
 13%|█▎        | 28213/225000 [65:07:25<395:15:31,  7.23s/it]
 13%|█▎        | 28214/225000 [65:07:29<341:05:25,  6.24s/it]
 13%|█▎        | 28215/225000 [65:07:36<351:47:46,  6.44s/it]
 13%|█▎        | 28216/225000 [65:07:43<359:47:46,  6.58s/it]
 13%|█▎        | 28217/225000 [65:07:50<372:12:04,  6.81s/it]
 13%|█▎        | 28218/225000 [65:07:56<357:50:35,  6.55s/it]
 13%|█▎        | 28219/225000 [65:08:03<363:51:17,  6.66s/it]
 13%|█▎        | 28220/225000 [65:08:08<342:52:59,  6.27s/it]
 13%|█▎        | 28221/225000 [65:08:17<374:08:02,  6.84s/it]
 13%|█▎        | 28222/225000 [65:08:20<318:52:58,  5.83s/it]
 13%|█▎        | 28223/225000 [65:08:25<300:49:33,  5.50s/it]
 13%|█▎        | 28224/225000 [65:08:32<322:33:07,  5.90s/it]
 13%|█▎        | 28225/225000 [65:08:38<321:40:49,  5.89s/it]
 13%|█▎        | 28226/225000 [65:08:44<337:15:37,  6.17s/it]
 13%|█▎        | 28227/225000 [65:08:50<321:34:54,  5.88s/it]
 13%|█▎        | 28228/225000 [65:08:59<383:58:32,  7.02s/it]
 13%|█▎        | 28229/225000 [65:09:05<357:15:57,  6.54s/it]
 13%|█▎        | 28230/225000 [65:09:09<319:18:07,  5.84s/it]
 13%|█▎        | 28231/225000 [65:09:14<306:04:56,  5.60s/it]
 13%|█▎        | 28232/225000 [65:09:28<438:11:26,  8.02s/it]
 13%|█▎        | 28233/225000 [65:09:32<375:00:37,  6.86s/it]
 13%|█▎        | 28234/225000 [65:09:36<327:17:09,  5.99s/it]
 13%|█▎        | 28235/225000 [65:09:43<346:02:19,  6.33s/it]
 13%|█▎        | 28236/225000 [65:09:54<425:19:22,  7.78s/it]
 13%|█▎        | 28237/225000 [65:09:58<365:27:31,  6.69s/it]
 13%|█▎        | 28238/225000 [65:10:08<414:34:27,  7.59s/it]
 13%|█▎        | 28239/225000 [65:10:13<378:44:24,  6.93s/it]
 13%|█▎        | 28240/225000 [65:10:18<337:34:10,  6.18s/it]
 13%|█▎        | 28241/225000 [65:10:25<349:24:43,  6.39s/it]
 13%|█▎        | 28242/225000 [65:10:29<316:24:00,  5.79s/it]
 13%|█▎        | 28243/225000 [65:10:36<345:00:22,  6.31s/it]
 13%|█▎        | 28244/225000 [65:10:45<379:30:35,  6.94s/it]
 13%|█▎        | 28245/225000 [65:10:49<331:59:43,  6.07s/it]
 13%|█▎        | 28246/225000 [65:10:56<354:58:46,  6.50s/it]
 13%|█▎        | 28247/225000 [65:11:02<339:40:46,  6.22s/it]
 13%|█▎        | 28248/225000 [65:11:12<405:30:57,  7.42s/it]
 13%|█▎        | 28249/225000 [65:11:17<357:25:13,  6.54s/it]
 13%|█▎        | 28250/225000 [65:11:21<319:18:47,  5.84s/it]
                                                             
{'loss': '0.7344', 'grad_norm': '0.7979', 'learning_rate': '4.912e-05', 'epoch': '0.1256', 'num_input_tokens_seen': 111938008, 'train_runtime': '2.347e+05', 'train_tokens_per_second': '477'}

 13%|█▎        | 28250/225000 [65:11:21<319:18:47,  5.84s/it]
 13%|█▎        | 28251/225000 [65:11:30<373:13:54,  6.83s/it]
 13%|█▎        | 28252/225000 [65:11:35<336:30:11,  6.16s/it]
 13%|█▎        | 28253/225000 [65:11:39<307:33:17,  5.63s/it]
 13%|█▎        | 28254/225000 [65:11:52<421:17:57,  7.71s/it]
 13%|█▎        | 28255/225000 [65:11:59<414:06:08,  7.58s/it]
 13%|█▎        | 28256/225000 [65:12:03<355:46:46,  6.51s/it]
 13%|█▎        | 28257/225000 [65:12:11<390:42:33,  7.15s/it]
 13%|█▎        | 28258/225000 [65:12:16<341:55:42,  6.26s/it]
 13%|█▎        | 28259/225000 [65:12:20<310:55:05,  5.69s/it]
 13%|█▎        | 28260/225000 [65:12:32<409:00:46,  7.48s/it]
 13%|█▎        | 28261/225000 [65:12:36<354:15:12,  6.48s/it]
 13%|█▎        | 28262/225000 [65:12:40<319:33:09,  5.85s/it]
 13%|█▎        | 28263/225000 [65:12:45<301:33:57,  5.52s/it]
 13%|█▎        | 28264/225000 [65:12:50<292:25:03,  5.35s/it]
 13%|█▎        | 28265/225000 [65:13:00<363:18:03,  6.65s/it]
 13%|█▎        | 28266/225000 [65:13:09<416:37:37,  7.62s/it]
 13%|█▎        | 28267/225000 [65:13:13<353:55:37,  6.48s/it]
 13%|█▎        | 28268/225000 [65:13:21<373:17:10,  6.83s/it]
 13%|█▎        | 28269/225000 [65:13:28<376:38:53,  6.89s/it]
 13%|█▎        | 28270/225000 [65:13:35<380:39:36,  6.97s/it]
 13%|█▎        | 28271/225000 [65:13:42<372:48:06,  6.82s/it]
 13%|█▎        | 28272/225000 [65:13:48<359:31:50,  6.58s/it]
 13%|█▎        | 28273/225000 [65:13:54<361:40:50,  6.62s/it]
 13%|█▎        | 28274/225000 [65:13:59<337:08:46,  6.17s/it]
 13%|█▎        | 28275/225000 [65:14:05<323:56:58,  5.93s/it]
 13%|█▎        | 28276/225000 [65:14:11<320:12:43,  5.86s/it]
 13%|█▎        | 28277/225000 [65:14:16<316:03:31,  5.78s/it]
 13%|█▎        | 28278/225000 [65:14:21<305:57:06,  5.60s/it]
 13%|█▎        | 28279/225000 [65:14:29<338:47:39,  6.20s/it]
 13%|█▎        | 28280/225000 [65:14:41<441:41:09,  8.08s/it]
 13%|█▎        | 28281/225000 [65:14:45<372:59:21,  6.83s/it]
 13%|█▎        | 28282/225000 [65:14:55<415:52:24,  7.61s/it]
 13%|█▎        | 28283/225000 [65:15:01<390:43:57,  7.15s/it]
 13%|█▎        | 28284/225000 [65:15:07<369:47:58,  6.77s/it]
 13%|█▎        | 28285/225000 [65:15:15<394:11:23,  7.21s/it]
 13%|█▎        | 28286/225000 [65:15:18<334:45:13,  6.13s/it]
 13%|█▎        | 28287/225000 [65:15:23<313:18:47,  5.73s/it]
 13%|█▎        | 28288/225000 [65:15:28<289:57:23,  5.31s/it]
 13%|█▎        | 28289/225000 [65:15:39<389:12:57,  7.12s/it]
 13%|█▎        | 28290/225000 [65:15:47<404:23:37,  7.40s/it]
 13%|█▎        | 28291/225000 [65:15:52<360:29:51,  6.60s/it]
 13%|█▎        | 28292/225000 [65:16:00<390:38:35,  7.15s/it]
 13%|█▎        | 28293/225000 [65:16:05<347:03:55,  6.35s/it]
 13%|█▎        | 28294/225000 [65:16:15<405:50:30,  7.43s/it]
 13%|█▎        | 28295/225000 [65:16:18<334:16:55,  6.12s/it]
 13%|█▎        | 28296/225000 [65:16:28<398:00:22,  7.28s/it]
 13%|█▎        | 28297/225000 [65:16:31<340:50:11,  6.24s/it]
 13%|█▎        | 28298/225000 [65:16:37<321:59:25,  5.89s/it]
 13%|█▎        | 28299/225000 [65:16:42<313:37:52,  5.74s/it]
 13%|█▎        | 28300/225000 [65:16:48<312:16:09,  5.72s/it]
                                                             
{'loss': '0.7362', 'grad_norm': '0.6277', 'learning_rate': '4.911e-05', 'epoch': '0.1258', 'num_input_tokens_seen': 112082960, 'train_runtime': '2.35e+05', 'train_tokens_per_second': '476.9'}

 13%|█▎        | 28300/225000 [65:16:48<312:16:09,  5.72s/it]
 13%|█▎        | 28301/225000 [65:16:53<310:18:38,  5.68s/it]
 13%|█▎        | 28302/225000 [65:16:58<295:33:18,  5.41s/it]
 13%|█▎        | 28303/225000 [65:17:09<393:57:16,  7.21s/it]
 13%|█▎        | 28304/225000 [65:17:16<387:31:06,  7.09s/it]
 13%|█▎        | 28305/225000 [65:17:21<344:12:34,  6.30s/it]
 13%|█▎        | 28306/225000 [65:17:25<308:28:31,  5.65s/it]
 13%|█▎        | 28307/225000 [65:17:34<365:37:00,  6.69s/it]
 13%|█▎        | 28308/225000 [65:17:43<404:14:03,  7.40s/it]
 13%|█▎        | 28309/225000 [65:17:51<417:02:52,  7.63s/it]
 13%|█▎        | 28310/225000 [65:17:56<369:00:39,  6.75s/it]
 13%|█▎        | 28311/225000 [65:18:01<335:56:36,  6.15s/it]
 13%|█▎        | 28312/225000 [65:18:05<313:37:58,  5.74s/it]
 13%|█▎        | 28313/225000 [65:18:11<317:28:44,  5.81s/it]
 13%|█▎        | 28314/225000 [65:18:23<413:39:56,  7.57s/it]
 13%|█▎        | 28315/225000 [65:18:28<373:23:53,  6.83s/it]
 13%|█▎        | 28316/225000 [65:18:33<345:11:16,  6.32s/it]
 13%|█▎        | 28317/225000 [65:18:38<312:46:01,  5.72s/it]
 13%|█▎        | 28318/225000 [65:18:45<339:40:44,  6.22s/it]
 13%|█▎        | 28319/225000 [65:18:57<427:47:29,  7.83s/it]
 13%|█▎        | 28320/225000 [65:19:01<370:48:46,  6.79s/it]
 13%|█▎        | 28321/225000 [65:19:06<335:31:34,  6.14s/it]
 13%|█▎        | 28322/225000 [65:19:10<303:52:03,  5.56s/it]
 13%|█▎        | 28323/225000 [65:19:21<390:42:21,  7.15s/it]
 13%|█▎        | 28324/225000 [65:19:26<356:30:20,  6.53s/it]
 13%|█▎        | 28325/225000 [65:19:34<382:27:49,  7.00s/it]
 13%|█▎        | 28326/225000 [65:19:41<392:22:22,  7.18s/it]
 13%|█▎        | 28327/225000 [65:19:46<352:48:44,  6.46s/it]
 13%|█▎        | 28328/225000 [65:19:56<403:36:55,  7.39s/it]
 13%|█▎        | 28329/225000 [65:20:00<344:58:19,  6.31s/it]
 13%|█▎        | 28330/225000 [65:20:09<390:00:21,  7.14s/it]
 13%|█▎        | 28331/225000 [65:20:12<328:33:01,  6.01s/it]
 13%|█▎        | 28332/225000 [65:20:17<310:31:09,  5.68s/it]
 13%|█▎        | 28333/225000 [65:20:22<303:56:21,  5.56s/it]
 13%|█▎        | 28334/225000 [65:20:29<320:58:27,  5.88s/it]
 13%|█▎        | 28335/225000 [65:20:41<421:09:30,  7.71s/it]
 13%|█▎        | 28336/225000 [65:20:47<394:38:16,  7.22s/it]
 13%|█▎        | 28337/225000 [65:20:55<404:36:26,  7.41s/it]
 13%|█▎        | 28338/225000 [65:21:00<362:10:57,  6.63s/it]
 13%|█▎        | 28339/225000 [65:21:04<319:20:57,  5.85s/it]
 13%|█▎        | 28340/225000 [65:21:08<299:05:09,  5.47s/it]
 13%|█▎        | 28341/225000 [65:21:17<361:19:04,  6.61s/it]
 13%|█▎        | 28342/225000 [65:21:25<375:53:09,  6.88s/it]
 13%|█▎        | 28343/225000 [65:21:30<350:12:44,  6.41s/it]
 13%|█▎        | 28344/225000 [65:21:35<329:02:32,  6.02s/it]
 13%|█▎        | 28345/225000 [65:21:41<322:34:10,  5.91s/it]
 13%|█▎        | 28346/225000 [65:21:46<305:42:10,  5.60s/it]
 13%|█▎        | 28347/225000 [65:21:51<295:44:39,  5.41s/it]
 13%|█▎        | 28348/225000 [65:21:56<296:30:00,  5.43s/it]
 13%|█▎        | 28349/225000 [65:22:01<286:59:49,  5.25s/it]
 13%|█▎        | 28350/225000 [65:22:08<313:40:57,  5.74s/it]
                                                             
{'loss': '0.7256', 'grad_norm': '0.7126', 'learning_rate': '4.911e-05', 'epoch': '0.126', 'num_input_tokens_seen': 112227800, 'train_runtime': '2.353e+05', 'train_tokens_per_second': '476.9'}

 13%|█▎        | 28350/225000 [65:22:08<313:40:57,  5.74s/it]
 13%|█▎        | 28351/225000 [65:22:20<414:39:48,  7.59s/it]
 13%|█▎        | 28352/225000 [65:22:27<403:26:25,  7.39s/it]
 13%|█▎        | 28353/225000 [65:22:34<407:50:51,  7.47s/it]
 13%|█▎        | 28354/225000 [65:22:43<418:54:09,  7.67s/it]
 13%|█▎        | 28355/225000 [65:22:47<365:58:03,  6.70s/it]
 13%|█▎        | 28356/225000 [65:22:56<395:22:21,  7.24s/it]
 13%|█▎        | 28357/225000 [65:23:02<381:34:24,  6.99s/it]
 13%|█▎        | 28358/225000 [65:23:09<381:26:14,  6.98s/it]
 13%|█▎        | 28359/225000 [65:23:14<348:19:27,  6.38s/it]
 13%|█▎        | 28360/225000 [65:23:19<333:16:51,  6.10s/it]
 13%|█▎        | 28361/225000 [65:23:24<312:33:37,  5.72s/it]
 13%|█▎        | 28362/225000 [65:23:35<399:58:33,  7.32s/it]
 13%|█▎        | 28363/225000 [65:23:43<412:45:52,  7.56s/it]
 13%|█▎        | 28364/225000 [65:23:50<403:19:57,  7.38s/it]
 13%|█▎        | 28365/225000 [65:23:56<367:36:46,  6.73s/it]
 13%|█▎        | 28366/225000 [65:24:03<378:56:17,  6.94s/it]
 13%|█▎        | 28367/225000 [65:24:08<346:50:01,  6.35s/it]
 13%|█▎        | 28368/225000 [65:24:17<392:46:41,  7.19s/it]
 13%|█▎        | 28369/225000 [65:24:21<340:08:33,  6.23s/it]
 13%|█▎        | 28370/225000 [65:24:25<303:37:44,  5.56s/it]
 13%|█▎        | 28371/225000 [65:24:34<358:45:06,  6.57s/it]
 13%|█▎        | 28372/225000 [65:24:40<342:26:44,  6.27s/it]
 13%|█▎        | 28373/225000 [65:24:49<390:32:07,  7.15s/it]
 13%|█▎        | 28374/225000 [65:24:53<336:58:48,  6.17s/it]
 13%|█▎        | 28375/225000 [65:24:58<320:46:01,  5.87s/it]
 13%|█▎        | 28376/225000 [65:25:03<307:15:36,  5.63s/it]
 13%|█▎        | 28377/225000 [65:25:08<295:41:30,  5.41s/it]
 13%|█▎        | 28378/225000 [65:25:21<422:08:43,  7.73s/it]
 13%|█▎        | 28379/225000 [65:25:26<370:23:11,  6.78s/it]
 13%|█▎        | 28380/225000 [65:25:32<370:35:41,  6.79s/it]
 13%|█▎        | 28381/225000 [65:25:38<345:45:57,  6.33s/it]
 13%|█▎        | 28382/225000 [65:25:43<328:53:48,  6.02s/it]
 13%|█▎        | 28383/225000 [65:25:48<320:28:40,  5.87s/it]
 13%|█▎        | 28384/225000 [65:25:54<313:08:39,  5.73s/it]
 13%|█▎        | 28385/225000 [65:26:01<330:32:54,  6.05s/it]
 13%|█▎        | 28386/225000 [65:26:13<431:45:43,  7.91s/it]
 13%|█▎        | 28387/225000 [65:26:17<366:35:19,  6.71s/it]
 13%|█▎        | 28388/225000 [65:26:26<413:24:25,  7.57s/it]
 13%|█▎        | 28389/225000 [65:26:32<384:34:57,  7.04s/it]
 13%|█▎        | 28390/225000 [65:26:39<378:16:19,  6.93s/it]
 13%|█▎        | 28391/225000 [65:26:46<380:43:34,  6.97s/it]
 13%|█▎        | 28392/225000 [65:26:52<360:21:51,  6.60s/it]
 13%|█▎        | 28393/225000 [65:26:56<325:54:35,  5.97s/it]
 13%|█▎        | 28394/225000 [65:27:01<305:19:28,  5.59s/it]
 13%|█▎        | 28395/225000 [65:27:07<321:09:15,  5.88s/it]
 13%|█▎        | 28396/225000 [65:27:21<444:06:07,  8.13s/it]
 13%|█▎        | 28397/225000 [65:27:27<405:20:28,  7.42s/it]
 13%|█▎        | 28398/225000 [65:27:31<353:08:05,  6.47s/it]
 13%|█▎        | 28399/225000 [65:27:39<374:20:27,  6.85s/it]
 13%|█▎        | 28400/225000 [65:27:44<345:59:10,  6.34s/it]
                                                             
{'loss': '0.7238', 'grad_norm': '0.6919', 'learning_rate': '4.91e-05', 'epoch': '0.1262', 'num_input_tokens_seen': 112372560, 'train_runtime': '2.357e+05', 'train_tokens_per_second': '476.8'}

 13%|█▎        | 28400/225000 [65:27:44<345:59:10,  6.34s/it]
 13%|█▎        | 28401/225000 [65:27:49<324:06:16,  5.93s/it]
 13%|█▎        | 28402/225000 [65:27:54<309:09:58,  5.66s/it]
 13%|█▎        | 28403/225000 [65:27:59<299:56:41,  5.49s/it]
 13%|█▎        | 28404/225000 [65:28:07<336:52:54,  6.17s/it]
 13%|█▎        | 28405/225000 [65:28:18<423:29:07,  7.75s/it]
 13%|█▎        | 28406/225000 [65:28:23<378:05:25,  6.92s/it]
 13%|█▎        | 28407/225000 [65:28:28<353:04:45,  6.47s/it]
 13%|█▎        | 28408/225000 [65:28:34<339:55:17,  6.22s/it]
 13%|█▎        | 28409/225000 [65:28:39<318:42:46,  5.84s/it]
 13%|█▎        | 28410/225000 [65:28:44<308:17:50,  5.65s/it]
 13%|█▎        | 28411/225000 [65:28:50<305:56:57,  5.60s/it]
 13%|█▎        | 28412/225000 [65:29:01<395:51:01,  7.25s/it]
 13%|█▎        | 28413/225000 [65:29:10<421:02:43,  7.71s/it]
 13%|█▎        | 28414/225000 [65:29:14<362:11:15,  6.63s/it]
 13%|█▎        | 28415/225000 [65:29:18<322:12:58,  5.90s/it]
 13%|█▎        | 28416/225000 [65:29:27<378:58:15,  6.94s/it]
 13%|█▎        | 28417/225000 [65:29:32<350:59:34,  6.43s/it]
 13%|█▎        | 28418/225000 [65:29:42<409:02:30,  7.49s/it]
 13%|█▎        | 28419/225000 [65:29:46<353:04:27,  6.47s/it]
 13%|█▎        | 28420/225000 [65:29:51<319:49:22,  5.86s/it]
 13%|█▎        | 28421/225000 [65:29:56<305:31:26,  5.60s/it]
 13%|█▎        | 28422/225000 [65:30:02<318:38:48,  5.84s/it]
 13%|█▎        | 28423/225000 [65:30:14<408:14:01,  7.48s/it]
 13%|█▎        | 28424/225000 [65:30:18<350:47:13,  6.42s/it]
 13%|█▎        | 28425/225000 [65:30:25<374:20:25,  6.86s/it]
 13%|█▎        | 28426/225000 [65:30:31<360:07:14,  6.60s/it]
 13%|█▎        | 28427/225000 [65:30:38<365:44:17,  6.70s/it]
 13%|█▎        | 28428/225000 [65:30:46<386:07:54,  7.07s/it]
 13%|█▎        | 28429/225000 [65:30:50<335:49:36,  6.15s/it]
 13%|█▎        | 28430/225000 [65:31:00<398:22:20,  7.30s/it]
 13%|█▎        | 28431/225000 [65:31:05<360:19:42,  6.60s/it]
 13%|█▎        | 28432/225000 [65:31:10<334:06:04,  6.12s/it]
 13%|█▎        | 28433/225000 [65:31:20<395:33:38,  7.24s/it]
 13%|█▎        | 28434/225000 [65:31:24<340:55:15,  6.24s/it]
 13%|█▎        | 28435/225000 [65:31:34<401:55:34,  7.36s/it]
 13%|█▎        | 28436/225000 [65:31:38<351:05:38,  6.43s/it]
 13%|█▎        | 28437/225000 [65:31:49<415:45:30,  7.61s/it]
 13%|█▎        | 28438/225000 [65:31:53<368:03:58,  6.74s/it]
 13%|█▎        | 28439/225000 [65:32:03<407:59:59,  7.47s/it]
 13%|█▎        | 28440/225000 [65:32:06<348:39:00,  6.39s/it]
 13%|█▎        | 28441/225000 [65:32:11<328:11:36,  6.01s/it]
 13%|█▎        | 28442/225000 [65:32:17<319:32:30,  5.85s/it]
 13%|█▎        | 28443/225000 [65:32:22<311:44:54,  5.71s/it]
 13%|█▎        | 28444/225000 [65:32:28<306:40:30,  5.62s/it]
 13%|█▎        | 28445/225000 [65:32:33<306:14:54,  5.61s/it]
 13%|█▎        | 28446/225000 [65:32:39<307:32:25,  5.63s/it]
 13%|█▎        | 28447/225000 [65:32:52<433:19:06,  7.94s/it]
 13%|█▎        | 28448/225000 [65:32:57<379:10:36,  6.94s/it]
 13%|█▎        | 28449/225000 [65:33:05<403:51:22,  7.40s/it]
 13%|█▎        | 28450/225000 [65:33:09<347:46:56,  6.37s/it]
                                                             
{'loss': '0.7389', 'grad_norm': '0.7196', 'learning_rate': '4.91e-05', 'epoch': '0.1264', 'num_input_tokens_seen': 112518016, 'train_runtime': '2.36e+05', 'train_tokens_per_second': '476.8'}

 13%|█▎        | 28450/225000 [65:33:09<347:46:56,  6.37s/it]
 13%|█▎        | 28451/225000 [65:33:19<395:02:45,  7.24s/it]
 13%|█▎        | 28452/225000 [65:33:26<391:36:07,  7.17s/it]
 13%|█▎        | 28453/225000 [65:33:31<362:12:31,  6.63s/it]
 13%|█▎        | 28454/225000 [65:33:38<374:54:18,  6.87s/it]
 13%|█▎        | 28455/225000 [65:33:44<351:31:30,  6.44s/it]
 13%|█▎        | 28456/225000 [65:33:51<359:56:15,  6.59s/it]
 13%|█▎        | 28457/225000 [65:33:56<337:30:21,  6.18s/it]
 13%|█▎        | 28458/225000 [65:34:04<366:17:56,  6.71s/it]
 13%|█▎        | 28459/225000 [65:34:09<334:21:15,  6.12s/it]
 13%|█▎        | 28460/225000 [65:34:14<319:25:57,  5.85s/it]
 13%|█▎        | 28461/225000 [65:34:21<331:53:13,  6.08s/it]
 13%|█▎        | 28462/225000 [65:34:32<425:49:43,  7.80s/it]
 13%|█▎        | 28463/225000 [65:34:37<371:17:13,  6.80s/it]
 13%|█▎        | 28464/225000 [65:34:41<333:15:32,  6.10s/it]
 13%|█▎        | 28465/225000 [65:34:52<405:15:30,  7.42s/it]
 13%|█▎        | 28466/225000 [65:34:58<386:23:24,  7.08s/it]
 13%|█▎        | 28467/225000 [65:35:05<377:15:00,  6.91s/it]
 13%|█▎        | 28468/225000 [65:35:10<348:48:15,  6.39s/it]
 13%|█▎        | 28469/225000 [65:35:15<326:47:07,  5.99s/it]
 13%|█▎        | 28470/225000 [65:35:20<311:11:28,  5.70s/it]
 13%|█▎        | 28471/225000 [65:35:25<301:47:26,  5.53s/it]
 13%|█▎        | 28472/225000 [65:35:32<328:12:51,  6.01s/it]
 13%|█▎        | 28473/225000 [65:35:45<432:59:50,  7.93s/it]
 13%|█▎        | 28474/225000 [65:35:48<361:36:13,  6.62s/it]
 13%|█▎        | 28475/225000 [65:35:56<382:22:41,  7.00s/it]
 13%|█▎        | 28476/225000 [65:36:02<358:52:45,  6.57s/it]
 13%|█▎        | 28477/225000 [65:36:07<340:25:53,  6.24s/it]
 13%|█▎        | 28478/225000 [65:36:17<397:59:27,  7.29s/it]
 13%|█▎        | 28479/225000 [65:36:21<350:58:16,  6.43s/it]
 13%|█▎        | 28480/225000 [65:36:26<318:09:52,  5.83s/it]
 13%|█▎        | 28481/225000 [65:36:30<297:41:56,  5.45s/it]
 13%|█▎        | 28482/225000 [65:36:37<321:53:11,  5.90s/it]
 13%|█▎        | 28483/225000 [65:36:48<405:06:30,  7.42s/it]
 13%|█▎        | 28484/225000 [65:36:52<344:39:23,  6.31s/it]
 13%|█▎        | 28485/225000 [65:37:02<408:25:09,  7.48s/it]
 13%|█▎        | 28486/225000 [65:37:06<350:03:00,  6.41s/it]
 13%|█▎        | 28487/225000 [65:37:16<408:13:39,  7.48s/it]
 13%|█▎        | 28488/225000 [65:37:20<353:20:08,  6.47s/it]
 13%|█▎        | 28489/225000 [65:37:27<358:22:34,  6.57s/it]
 13%|█▎        | 28490/225000 [65:37:34<366:10:17,  6.71s/it]
 13%|█▎        | 28491/225000 [65:37:40<349:10:05,  6.40s/it]
 13%|█▎        | 28492/225000 [65:37:48<374:17:13,  6.86s/it]
 13%|█▎        | 28493/225000 [65:37:53<354:10:08,  6.49s/it]
 13%|█▎        | 28494/225000 [65:37:58<329:33:32,  6.04s/it]
 13%|█▎        | 28495/225000 [65:38:09<400:32:21,  7.34s/it]
 13%|█▎        | 28496/225000 [65:38:14<368:32:56,  6.75s/it]
 13%|█▎        | 28497/225000 [65:38:19<347:50:13,  6.37s/it]
 13%|█▎        | 28498/225000 [65:38:29<397:50:17,  7.29s/it]
 13%|█▎        | 28499/225000 [65:38:33<339:20:32,  6.22s/it]
 13%|█▎        | 28500/225000 [65:38:41<382:35:50,  7.01s/it]
                                                             
{'loss': '0.7368', 'grad_norm': '0.845', 'learning_rate': '4.909e-05', 'epoch': '0.1267', 'num_input_tokens_seen': 112662488, 'train_runtime': '2.363e+05', 'train_tokens_per_second': '476.7'}

 13%|█▎        | 28500/225000 [65:38:41<382:35:50,  7.01s/it]
 13%|█▎        | 28501/225000 [65:38:47<355:48:34,  6.52s/it]
 13%|█▎        | 28502/225000 [65:38:56<395:30:36,  7.25s/it]
 13%|█▎        | 28503/225000 [65:39:03<391:56:21,  7.18s/it]
 13%|█▎        | 28504/225000 [65:39:06<332:19:42,  6.09s/it]
 13%|█▎        | 28505/225000 [65:39:15<377:08:39,  6.91s/it]
 13%|█▎        | 28506/225000 [65:39:19<335:50:23,  6.15s/it]
 13%|█▎        | 28507/225000 [65:39:24<313:20:46,  5.74s/it]
 13%|█▎        | 28508/225000 [65:39:30<315:32:14,  5.78s/it]
 13%|█▎        | 28509/225000 [65:39:43<432:20:07,  7.92s/it]
 13%|█▎        | 28510/225000 [65:39:46<358:52:12,  6.58s/it]
 13%|█▎        | 28511/225000 [65:39:51<332:18:03,  6.09s/it]
 13%|█▎        | 28512/225000 [65:39:56<309:04:07,  5.66s/it]
 13%|█▎        | 28513/225000 [65:40:01<301:23:17,  5.52s/it]
 13%|█▎        | 28514/225000 [65:40:06<294:25:18,  5.39s/it]
 13%|█▎        | 28515/225000 [65:40:19<405:41:26,  7.43s/it]
 13%|█▎        | 28516/225000 [65:40:26<401:28:00,  7.36s/it]
 13%|█▎        | 28517/225000 [65:40:33<393:01:59,  7.20s/it]
 13%|█▎        | 28518/225000 [65:40:41<409:28:20,  7.50s/it]
 13%|█▎        | 28519/225000 [65:40:45<355:02:04,  6.51s/it]
 13%|█▎        | 28520/225000 [65:40:55<413:07:22,  7.57s/it]
 13%|█▎        | 28521/225000 [65:40:59<351:52:45,  6.45s/it]
 13%|█▎        | 28522/225000 [65:41:04<335:54:03,  6.15s/it]
 13%|█▎        | 28523/225000 [65:41:12<366:02:16,  6.71s/it]
 13%|█▎        | 28524/225000 [65:41:16<316:22:42,  5.80s/it]
 13%|█▎        | 28525/225000 [65:41:25<367:38:04,  6.74s/it]
 13%|█▎        | 28526/225000 [65:41:29<318:49:58,  5.84s/it]
 13%|█▎        | 28527/225000 [65:41:38<373:10:46,  6.84s/it]
 13%|█▎        | 28528/225000 [65:41:45<380:47:15,  6.98s/it]
 13%|█▎        | 28529/225000 [65:41:51<362:47:50,  6.65s/it]
 13%|█▎        | 28530/225000 [65:41:58<374:15:15,  6.86s/it]
 13%|█▎        | 28531/225000 [65:42:02<322:10:39,  5.90s/it]
 13%|█▎        | 28532/225000 [65:42:06<287:48:10,  5.27s/it]
 13%|█▎        | 28533/225000 [65:42:17<385:35:49,  7.07s/it]
 13%|█▎        | 28534/225000 [65:42:22<355:10:54,  6.51s/it]
 13%|█▎        | 28535/225000 [65:42:26<316:35:21,  5.80s/it]
 13%|█▎        | 28536/225000 [65:42:31<303:48:17,  5.57s/it]
 13%|█▎        | 28537/225000 [65:42:37<300:28:18,  5.51s/it]
 13%|█▎        | 28538/225000 [65:42:42<291:49:16,  5.35s/it]
 13%|█▎        | 28539/225000 [65:42:51<357:03:54,  6.54s/it]
 13%|█▎        | 28540/225000 [65:43:02<433:49:57,  7.95s/it]
 13%|█▎        | 28541/225000 [65:43:09<404:46:42,  7.42s/it]
 13%|█▎        | 28542/225000 [65:43:14<368:57:22,  6.76s/it]
 13%|█▎        | 28543/225000 [65:43:18<326:54:53,  5.99s/it]
 13%|█▎        | 28544/225000 [65:43:26<361:37:21,  6.63s/it]
 13%|█▎        | 28545/225000 [65:43:36<421:06:18,  7.72s/it]
 13%|█▎        | 28546/225000 [65:43:40<356:36:55,  6.53s/it]
 13%|█▎        | 28547/225000 [65:43:49<395:49:37,  7.25s/it]
 13%|█▎        | 28548/225000 [65:43:56<386:53:40,  7.09s/it]
 13%|█▎        | 28549/225000 [65:44:01<350:18:58,  6.42s/it]
 13%|█▎        | 28550/225000 [65:44:05<322:00:31,  5.90s/it]
                                                             
{'loss': '0.7148', 'grad_norm': '0.6815', 'learning_rate': '4.909e-05', 'epoch': '0.1269', 'num_input_tokens_seen': 112806976, 'train_runtime': '2.366e+05', 'train_tokens_per_second': '476.7'}

 13%|█▎        | 28550/225000 [65:44:05<322:00:31,  5.90s/it]
 13%|█▎        | 28551/225000 [65:44:10<303:33:32,  5.56s/it]
 13%|█▎        | 28552/225000 [65:44:22<412:44:48,  7.56s/it]
 13%|█▎        | 28553/225000 [65:44:27<361:00:21,  6.62s/it]
 13%|█▎        | 28554/225000 [65:44:34<371:40:17,  6.81s/it]
 13%|█▎        | 28555/225000 [65:44:42<398:38:32,  7.31s/it]
 13%|█▎        | 28556/225000 [65:44:51<411:50:21,  7.55s/it]
 13%|█▎        | 28557/225000 [65:44:54<344:00:17,  6.30s/it]
 13%|█▎        | 28558/225000 [65:45:04<401:34:07,  7.36s/it]
 13%|█▎        | 28559/225000 [65:45:10<385:16:31,  7.06s/it]
 13%|█▎        | 28560/225000 [65:45:19<406:49:25,  7.46s/it]
 13%|█▎        | 28561/225000 [65:45:23<365:49:58,  6.70s/it]
 13%|█▎        | 28562/225000 [65:45:28<328:08:44,  6.01s/it]
 13%|█▎        | 28563/225000 [65:45:33<308:16:42,  5.65s/it]
 13%|█▎        | 28564/225000 [65:45:44<406:22:34,  7.45s/it]
 13%|█▎        | 28565/225000 [65:45:50<369:58:41,  6.78s/it]
 13%|█▎        | 28566/225000 [65:45:54<333:04:29,  6.10s/it]
 13%|█▎        | 28567/225000 [65:45:59<317:45:36,  5.82s/it]
 13%|█▎        | 28568/225000 [65:46:04<296:35:01,  5.44s/it]
 13%|█▎        | 28569/225000 [65:46:15<386:45:14,  7.09s/it]
 13%|█▎        | 28570/225000 [65:46:22<397:16:26,  7.28s/it]
 13%|█▎        | 28571/225000 [65:46:30<399:35:46,  7.32s/it]
 13%|█▎        | 28572/225000 [65:46:35<370:00:48,  6.78s/it]
 13%|█▎        | 28573/225000 [65:46:44<393:28:54,  7.21s/it]
 13%|█▎        | 28574/225000 [65:46:48<350:35:20,  6.43s/it]
 13%|█▎        | 28575/225000 [65:46:54<332:29:22,  6.09s/it]
 13%|█▎        | 28576/225000 [65:46:59<316:04:57,  5.79s/it]
 13%|█▎        | 28577/225000 [65:47:05<318:00:04,  5.83s/it]
 13%|█▎        | 28578/225000 [65:47:18<440:41:14,  8.08s/it]
 13%|█▎        | 28579/225000 [65:47:23<392:13:39,  7.19s/it]
 13%|█▎        | 28580/225000 [65:47:28<362:55:10,  6.65s/it]
 13%|█▎        | 28581/225000 [65:47:33<334:23:13,  6.13s/it]
 13%|█▎        | 28582/225000 [65:47:38<317:10:02,  5.81s/it]
 13%|█▎        | 28583/225000 [65:47:44<306:13:54,  5.61s/it]
 13%|█▎        | 28584/225000 [65:47:52<350:24:40,  6.42s/it]
 13%|█▎        | 28585/225000 [65:48:05<460:44:35,  8.44s/it]
 13%|█▎        | 28586/225000 [65:48:10<401:36:12,  7.36s/it]
 13%|█▎        | 28587/225000 [65:48:18<422:54:26,  7.75s/it]
 13%|█▎        | 28588/225000 [65:48:23<370:10:13,  6.78s/it]
 13%|█▎        | 28589/225000 [65:48:27<327:44:29,  6.01s/it]
 13%|█▎        | 28590/225000 [65:48:37<398:02:11,  7.30s/it]
 13%|█▎        | 28591/225000 [65:48:41<342:56:24,  6.29s/it]
 13%|█▎        | 28592/225000 [65:48:50<377:15:14,  6.91s/it]
 13%|█▎        | 28593/225000 [65:48:57<381:41:39,  7.00s/it]
 13%|█▎        | 28594/225000 [65:49:02<355:20:16,  6.51s/it]
 13%|█▎        | 28595/225000 [65:49:09<355:20:14,  6.51s/it]
 13%|█▎        | 28596/225000 [65:49:13<317:13:52,  5.81s/it]
 13%|█▎        | 28597/225000 [65:49:18<308:18:27,  5.65s/it]
 13%|█▎        | 28598/225000 [65:49:24<307:18:08,  5.63s/it]
 13%|█▎        | 28599/225000 [65:49:29<301:12:24,  5.52s/it]
 13%|█▎        | 28600/225000 [65:49:34<297:08:17,  5.45s/it]
                                                             
{'loss': '0.7148', 'grad_norm': '0.8088', 'learning_rate': '4.908e-05', 'epoch': '0.1271', 'num_input_tokens_seen': 112952424, 'train_runtime': '2.37e+05', 'train_tokens_per_second': '476.6'}

 13%|█▎        | 28600/225000 [65:49:34<297:08:17,  5.45s/it]
 13%|█▎        | 28601/225000 [65:49:45<387:06:05,  7.10s/it]
 13%|█▎        | 28602/225000 [65:49:54<411:55:58,  7.55s/it]
 13%|█▎        | 28603/225000 [65:49:59<376:06:03,  6.89s/it]
 13%|█▎        | 28604/225000 [65:50:05<352:39:05,  6.46s/it]
 13%|█▎        | 28605/225000 [65:50:15<405:49:12,  7.44s/it]
 13%|█▎        | 28606/225000 [65:50:19<363:02:34,  6.65s/it]
 13%|█▎        | 28607/225000 [65:50:24<328:47:03,  6.03s/it]
 13%|█▎        | 28608/225000 [65:50:29<308:42:03,  5.66s/it]
 13%|█▎        | 28609/225000 [65:50:38<359:37:25,  6.59s/it]
 13%|█▎        | 28610/225000 [65:50:48<418:56:10,  7.68s/it]
 13%|█▎        | 28611/225000 [65:50:52<370:21:25,  6.79s/it]
 13%|█▎        | 28612/225000 [65:50:58<355:03:40,  6.51s/it]
 13%|█▎        | 28613/225000 [65:51:03<327:06:54,  6.00s/it]
 13%|█▎        | 28614/225000 [65:51:08<304:11:58,  5.58s/it]
 13%|█▎        | 28615/225000 [65:51:13<293:38:55,  5.38s/it]
 13%|█▎        | 28616/225000 [65:51:25<409:12:38,  7.50s/it]
 13%|█▎        | 28617/225000 [65:51:31<380:45:56,  6.98s/it]
 13%|█▎        | 28618/225000 [65:51:35<336:12:44,  6.16s/it]
 13%|█▎        | 28619/225000 [65:51:39<301:44:51,  5.53s/it]
 13%|█▎        | 28620/225000 [65:51:51<411:46:26,  7.55s/it]
 13%|█▎        | 28621/225000 [65:51:56<360:38:14,  6.61s/it]
 13%|█▎        | 28622/225000 [65:52:00<314:21:56,  5.76s/it]
 13%|█▎        | 28623/225000 [65:52:12<426:38:19,  7.82s/it]
 13%|█▎        | 28624/225000 [65:52:19<401:57:26,  7.37s/it]
 13%|█▎        | 28625/225000 [65:52:22<345:41:02,  6.34s/it]
 13%|█▎        | 28626/225000 [65:52:27<314:23:21,  5.76s/it]
 13%|█▎        | 28627/225000 [65:52:38<400:24:56,  7.34s/it]
 13%|█▎        | 28628/225000 [65:52:42<353:22:53,  6.48s/it]
 13%|█▎        | 28629/225000 [65:52:49<354:01:15,  6.49s/it]
 13%|█▎        | 28630/225000 [65:52:58<399:15:38,  7.32s/it]
 13%|█▎        | 28631/225000 [65:53:02<350:12:44,  6.42s/it]
 13%|█▎        | 28632/225000 [65:53:07<315:24:34,  5.78s/it]
 13%|█▎        | 28633/225000 [65:53:18<406:35:05,  7.45s/it]
 13%|█▎        | 28634/225000 [65:53:22<350:30:03,  6.43s/it]
 13%|█▎        | 28635/225000 [65:53:32<406:31:57,  7.45s/it]
 13%|█▎        | 28636/225000 [65:53:35<339:21:48,  6.22s/it]
 13%|█▎        | 28637/225000 [65:53:42<351:33:32,  6.45s/it]
 13%|█▎        | 28638/225000 [65:53:46<312:11:47,  5.72s/it]
 13%|█▎        | 28639/225000 [65:53:56<382:44:42,  7.02s/it]
 13%|█▎        | 28640/225000 [65:54:00<325:19:35,  5.96s/it]
 13%|█▎        | 28641/225000 [65:54:04<302:46:07,  5.55s/it]
 13%|█▎        | 28642/225000 [65:54:09<290:28:18,  5.33s/it]
 13%|█▎        | 28643/225000 [65:54:22<405:45:17,  7.44s/it]
 13%|█▎        | 28644/225000 [65:54:26<352:33:15,  6.46s/it]
 13%|█▎        | 28645/225000 [65:54:34<387:11:59,  7.10s/it]
 13%|█▎        | 28646/225000 [65:54:39<340:08:05,  6.24s/it]
 13%|█▎        | 28647/225000 [65:54:43<315:27:02,  5.78s/it]
 13%|█▎        | 28648/225000 [65:54:50<336:51:23,  6.18s/it]
 13%|█▎        | 28649/225000 [65:54:58<362:34:58,  6.65s/it]
 13%|█▎        | 28650/225000 [65:55:03<338:52:51,  6.21s/it]
                                                             
{'loss': '0.7238', 'grad_norm': '0.6459', 'learning_rate': '4.908e-05', 'epoch': '0.1273', 'num_input_tokens_seen': 113095888, 'train_runtime': '2.373e+05', 'train_tokens_per_second': '476.6'}

 13%|█▎        | 28650/225000 [65:55:03<338:52:51,  6.21s/it]
 13%|█▎        | 28651/225000 [65:55:08<315:16:30,  5.78s/it]
 13%|█▎        | 28652/225000 [65:55:16<350:50:33,  6.43s/it]
 13%|█▎        | 28653/225000 [65:55:26<409:39:09,  7.51s/it]
 13%|█▎        | 28654/225000 [65:55:34<413:19:31,  7.58s/it]
 13%|█▎        | 28655/225000 [65:55:38<350:41:21,  6.43s/it]
 13%|█▎        | 28656/225000 [65:55:45<371:12:29,  6.81s/it]
 13%|█▎        | 28657/225000 [65:55:49<322:36:33,  5.92s/it]
 13%|█▎        | 28658/225000 [65:56:00<406:16:23,  7.45s/it]
 13%|█▎        | 28659/225000 [65:56:04<346:29:23,  6.35s/it]
 13%|█▎        | 28660/225000 [65:56:09<325:21:42,  5.97s/it]
 13%|█▎        | 28661/225000 [65:56:13<299:14:36,  5.49s/it]
 13%|█▎        | 28662/225000 [65:56:26<415:39:40,  7.62s/it]
 13%|█▎        | 28663/225000 [65:56:31<364:50:56,  6.69s/it]
 13%|█▎        | 28664/225000 [65:56:35<332:31:46,  6.10s/it]
 13%|█▎        | 28665/225000 [65:56:40<306:04:44,  5.61s/it]
 13%|█▎        | 28666/225000 [65:56:52<411:52:09,  7.55s/it]
 13%|█▎        | 28667/225000 [65:56:57<369:10:33,  6.77s/it]
 13%|█▎        | 28668/225000 [65:57:05<395:19:54,  7.25s/it]
 13%|█▎        | 28669/225000 [65:57:09<334:13:55,  6.13s/it]
 13%|█▎        | 28670/225000 [65:57:17<368:04:22,  6.75s/it]
 13%|█▎        | 28671/225000 [65:57:25<393:18:05,  7.21s/it]
 13%|█▎        | 28672/225000 [65:57:30<352:27:31,  6.46s/it]
 13%|█▎        | 28673/225000 [65:57:35<324:08:14,  5.94s/it]
 13%|█▎        | 28674/225000 [65:57:39<304:34:50,  5.59s/it]
 13%|█▎        | 28675/225000 [65:57:47<341:31:10,  6.26s/it]
 13%|█▎        | 28676/225000 [65:58:00<449:37:04,  8.24s/it]
 13%|█▎        | 28677/225000 [65:58:06<412:23:28,  7.56s/it]
 13%|█▎        | 28678/225000 [65:58:12<388:57:39,  7.13s/it]
 13%|█▎        | 28679/225000 [65:58:16<335:40:29,  6.16s/it]
 13%|█▎        | 28680/225000 [65:58:27<416:18:47,  7.63s/it]
 13%|█▎        | 28681/225000 [65:58:31<353:40:42,  6.49s/it]
 13%|█▎        | 28682/225000 [65:58:38<359:51:08,  6.60s/it]
 13%|█▎        | 28683/225000 [65:58:44<351:36:29,  6.45s/it]
 13%|█▎        | 28684/225000 [65:58:51<369:34:59,  6.78s/it]
 13%|█▎        | 28685/225000 [65:58:56<337:12:03,  6.18s/it]
 13%|█▎        | 28686/225000 [65:59:01<309:35:44,  5.68s/it]
 13%|█▎        | 28687/225000 [65:59:07<327:32:34,  6.01s/it]
 13%|█▎        | 28688/225000 [65:59:13<321:21:54,  5.89s/it]
 13%|█▎        | 28689/225000 [65:59:22<372:09:48,  6.82s/it]
 13%|█▎        | 28690/225000 [65:59:26<323:14:39,  5.93s/it]
 13%|█▎        | 28691/225000 [65:59:31<307:19:19,  5.64s/it]
 13%|█▎        | 28692/225000 [65:59:37<317:18:36,  5.82s/it]
 13%|█▎        | 28693/225000 [65:59:49<412:00:29,  7.56s/it]
 13%|█▎        | 28694/225000 [65:59:52<340:26:14,  6.24s/it]
 13%|█▎        | 28695/225000 [65:59:56<305:59:14,  5.61s/it]
 13%|█▎        | 28696/225000 [66:00:05<355:51:43,  6.53s/it]
 13%|█▎        | 28697/225000 [66:00:12<374:15:29,  6.86s/it]
 13%|█▎        | 28698/225000 [66:00:17<338:46:02,  6.21s/it]
 13%|█▎        | 28699/225000 [66:00:22<312:37:22,  5.73s/it]
 13%|█▎        | 28700/225000 [66:00:26<288:50:18,  5.30s/it]
                                                             
{'loss': '0.7359', 'grad_norm': '0.7666', 'learning_rate': '4.907e-05', 'epoch': '0.1276', 'num_input_tokens_seen': 113239640, 'train_runtime': '2.376e+05', 'train_tokens_per_second': '476.5'}

 13%|█▎        | 28700/225000 [66:00:26<288:50:18,  5.30s/it]
 13%|█▎        | 28701/225000 [66:00:37<376:30:34,  6.90s/it]
 13%|█▎        | 28702/225000 [66:00:44<390:23:41,  7.16s/it]
 13%|█▎        | 28703/225000 [66:00:48<338:44:46,  6.21s/it]
 13%|█▎        | 28704/225000 [66:00:53<304:36:06,  5.59s/it]
 13%|█▎        | 28705/225000 [66:01:03<379:21:04,  6.96s/it]
 13%|█▎        | 28706/225000 [66:01:10<381:04:20,  6.99s/it]
 13%|█▎        | 28707/225000 [66:01:16<375:22:25,  6.88s/it]
 13%|█▎        | 28708/225000 [66:01:23<373:03:29,  6.84s/it]
 13%|█▎        | 28709/225000 [66:01:27<332:38:40,  6.10s/it]
 13%|█▎        | 28710/225000 [66:01:32<314:33:59,  5.77s/it]
 13%|█▎        | 28711/225000 [66:01:44<402:46:24,  7.39s/it]
 13%|█▎        | 28712/225000 [66:01:47<342:34:35,  6.28s/it]
 13%|█▎        | 28713/225000 [66:01:52<312:35:07,  5.73s/it]
 13%|█▎        | 28714/225000 [66:01:57<298:47:56,  5.48s/it]
 13%|█▎        | 28715/225000 [66:02:01<280:30:56,  5.14s/it]
 13%|█▎        | 28716/225000 [66:02:13<397:10:45,  7.28s/it]
 13%|█▎        | 28717/225000 [66:02:18<360:00:07,  6.60s/it]
 13%|█▎        | 28718/225000 [66:02:25<354:17:15,  6.50s/it]
 13%|█▎        | 28719/225000 [66:02:32<368:07:14,  6.75s/it]
 13%|█▎        | 28720/225000 [66:02:38<356:17:16,  6.53s/it]
 13%|█▎        | 28721/225000 [66:02:45<356:26:05,  6.54s/it]
 13%|█▎        | 28722/225000 [66:02:49<314:40:17,  5.77s/it]
 13%|█▎        | 28723/225000 [66:02:53<297:17:41,  5.45s/it]
 13%|█▎        | 28724/225000 [66:02:59<302:23:43,  5.55s/it]
 13%|█▎        | 28725/225000 [66:03:05<304:06:43,  5.58s/it]
 13%|█▎        | 28726/225000 [66:03:10<297:22:56,  5.45s/it]
 13%|█▎        | 28727/225000 [66:03:15<292:33:53,  5.37s/it]
 13%|█▎        | 28728/225000 [66:03:25<372:59:13,  6.84s/it]
 13%|█▎        | 28729/225000 [66:03:34<409:34:28,  7.51s/it]
 13%|█▎        | 28730/225000 [66:03:42<411:08:21,  7.54s/it]
 13%|█▎        | 28731/225000 [66:03:46<357:59:45,  6.57s/it]
 13%|█▎        | 28732/225000 [66:03:50<317:41:11,  5.83s/it]
 13%|█▎        | 28733/225000 [66:03:58<349:41:39,  6.41s/it]
 13%|█▎        | 28734/225000 [66:04:09<419:46:34,  7.70s/it]
 13%|█▎        | 28735/225000 [66:04:13<368:25:15,  6.76s/it]
 13%|█▎        | 28736/225000 [66:04:18<340:54:11,  6.25s/it]
 13%|█▎        | 28737/225000 [66:04:24<322:49:58,  5.92s/it]
 13%|█▎        | 28738/225000 [66:04:29<313:27:05,  5.75s/it]
 13%|█▎        | 28739/225000 [66:04:34<310:19:15,  5.69s/it]
 13%|█▎        | 28740/225000 [66:04:49<451:12:07,  8.28s/it]
 13%|█▎        | 28741/225000 [66:04:54<407:54:15,  7.48s/it]
 13%|█▎        | 28742/225000 [66:05:00<383:57:15,  7.04s/it]
 13%|█▎        | 28743/225000 [66:05:07<369:39:43,  6.78s/it]
 13%|█▎        | 28744/225000 [66:05:14<372:39:11,  6.84s/it]
 13%|█▎        | 28745/225000 [66:05:18<333:32:01,  6.12s/it]
 13%|█▎        | 28746/225000 [66:05:24<324:17:44,  5.95s/it]
 13%|█▎        | 28747/225000 [66:05:29<314:42:49,  5.77s/it]
 13%|█▎        | 28748/225000 [66:05:35<315:44:17,  5.79s/it]
 13%|█▎        | 28749/225000 [66:05:40<306:02:52,  5.61s/it]
 13%|█▎        | 28750/225000 [66:05:54<436:33:14,  8.01s/it]
                                                             
{'loss': '0.7254', 'grad_norm': '0.7159', 'learning_rate': '4.907e-05', 'epoch': '0.1278', 'num_input_tokens_seen': 113384216, 'train_runtime': '2.38e+05', 'train_tokens_per_second': '476.5'}

 13%|█▎        | 28750/225000 [66:05:54<436:33:14,  8.01s/it]
 13%|█▎        | 28751/225000 [66:05:59<387:02:13,  7.10s/it]
 13%|█▎        | 28752/225000 [66:06:03<337:20:25,  6.19s/it]
 13%|█▎        | 28753/225000 [66:06:07<315:42:42,  5.79s/it]
 13%|█▎        | 28754/225000 [66:06:13<305:03:03,  5.60s/it]
 13%|█▎        | 28755/225000 [66:06:18<299:50:48,  5.50s/it]
 13%|█▎        | 28756/225000 [66:06:28<369:03:03,  6.77s/it]
 13%|█▎        | 28757/225000 [66:06:40<453:43:48,  8.32s/it]
 13%|█▎        | 28758/225000 [66:06:46<429:06:23,  7.87s/it]
 13%|█▎        | 28759/225000 [66:06:50<358:41:55,  6.58s/it]
 13%|█▎        | 28760/225000 [66:06:55<329:26:36,  6.04s/it]
 13%|█▎        | 28761/225000 [66:06:59<295:32:36,  5.42s/it]
 13%|█▎        | 28762/225000 [66:07:03<284:55:29,  5.23s/it]
 13%|█▎        | 28763/225000 [66:07:15<389:16:31,  7.14s/it]
 13%|█▎        | 28764/225000 [66:07:24<414:49:32,  7.61s/it]
 13%|█▎        | 28765/225000 [66:07:31<414:58:23,  7.61s/it]
 13%|█▎        | 28766/225000 [66:07:36<364:27:50,  6.69s/it]
 13%|█▎        | 28767/225000 [66:07:40<326:18:25,  5.99s/it]
 13%|█▎        | 28768/225000 [66:07:45<306:28:32,  5.62s/it]
 13%|█▎        | 28769/225000 [66:07:53<343:17:21,  6.30s/it]
 13%|█▎        | 28770/225000 [66:08:04<424:51:03,  7.79s/it]
 13%|█▎        | 28771/225000 [66:08:09<367:53:38,  6.75s/it]
 13%|█▎        | 28772/225000 [66:08:14<343:57:11,  6.31s/it]
 13%|█▎        | 28773/225000 [66:08:19<318:35:50,  5.85s/it]
 13%|█▎        | 28774/225000 [66:08:23<296:13:49,  5.43s/it]
 13%|█▎        | 28775/225000 [66:08:28<289:52:53,  5.32s/it]
 13%|█▎        | 28776/225000 [66:08:33<287:26:24,  5.27s/it]
 13%|█▎        | 28777/225000 [66:08:45<398:54:05,  7.32s/it]
 13%|█▎        | 28778/225000 [66:08:53<400:25:37,  7.35s/it]
 13%|█▎        | 28779/225000 [66:08:57<347:30:32,  6.38s/it]
 13%|█▎        | 28780/225000 [66:09:03<340:36:21,  6.25s/it]
 13%|█▎        | 28781/225000 [66:09:14<415:50:09,  7.63s/it]
 13%|█▎        | 28782/225000 [66:09:17<351:33:45,  6.45s/it]
 13%|█▎        | 28783/225000 [66:09:21<309:52:54,  5.69s/it]
 13%|█▎        | 28784/225000 [66:09:33<407:03:06,  7.47s/it]
 13%|█▎        | 28785/225000 [66:09:36<337:20:56,  6.19s/it]
 13%|█▎        | 28786/225000 [66:09:45<383:11:24,  7.03s/it]
 13%|█▎        | 28787/225000 [66:09:49<330:14:19,  6.06s/it]
 13%|█▎        | 28788/225000 [66:09:54<308:56:45,  5.67s/it]
 13%|█▎        | 28789/225000 [66:09:59<295:27:38,  5.42s/it]
 13%|█▎        | 28790/225000 [66:10:04<294:17:49,  5.40s/it]
 13%|█▎        | 28791/225000 [66:10:09<294:58:27,  5.41s/it]
 13%|█▎        | 28792/225000 [66:10:14<288:18:01,  5.29s/it]
 13%|█▎        | 28793/225000 [66:10:23<344:18:43,  6.32s/it]
 13%|█▎        | 28794/225000 [66:10:28<327:34:32,  6.01s/it]
 13%|█▎        | 28795/225000 [66:10:34<321:33:12,  5.90s/it]
 13%|█▎        | 28796/225000 [66:10:39<305:00:28,  5.60s/it]
 13%|█▎        | 28797/225000 [66:10:46<335:34:41,  6.16s/it]
 13%|█▎        | 28798/225000 [66:10:58<417:52:39,  7.67s/it]
 13%|█▎        | 28799/225000 [66:11:04<402:36:32,  7.39s/it]
 13%|█▎        | 28800/225000 [66:11:10<376:58:09,  6.92s/it]
                                                             
{'loss': '0.7305', 'grad_norm': '0.7655', 'learning_rate': '4.906e-05', 'epoch': '0.128', 'num_input_tokens_seen': 113526736, 'train_runtime': '2.383e+05', 'train_tokens_per_second': '476.5'}

 13%|█▎        | 28800/225000 [66:11:10<376:58:09,  6.92s/it]
 13%|█▎        | 28801/225000 [66:11:15<346:43:57,  6.36s/it]
 13%|█▎        | 28802/225000 [66:11:21<336:10:55,  6.17s/it]
 13%|█▎        | 28803/225000 [66:11:26<318:16:53,  5.84s/it]
 13%|█▎        | 28804/225000 [66:11:32<318:02:41,  5.84s/it]
 13%|█▎        | 28805/225000 [66:11:37<308:21:07,  5.66s/it]
 13%|█▎        | 28806/225000 [66:11:42<304:26:45,  5.59s/it]
 13%|█▎        | 28807/225000 [66:11:54<404:24:02,  7.42s/it]
 13%|█▎        | 28808/225000 [66:12:00<375:35:41,  6.89s/it]
 13%|█▎        | 28809/225000 [66:12:10<425:16:46,  7.80s/it]
 13%|█▎        | 28810/225000 [66:12:16<406:49:56,  7.47s/it]
 13%|█▎        | 28811/225000 [66:12:22<374:40:46,  6.88s/it]
 13%|█▎        | 28812/225000 [66:12:29<373:16:14,  6.85s/it]
 13%|█▎        | 28813/225000 [66:12:37<403:33:58,  7.41s/it]
 13%|█▎        | 28814/225000 [66:12:42<356:25:54,  6.54s/it]
 13%|█▎        | 28815/225000 [66:12:46<320:47:45,  5.89s/it]
 13%|█▎        | 28816/225000 [66:12:56<383:35:25,  7.04s/it]
 13%|█▎        | 28817/225000 [66:13:02<360:30:40,  6.62s/it]
 13%|█▎        | 28818/225000 [66:13:10<396:42:22,  7.28s/it]
 13%|█▎        | 28819/225000 [66:13:15<351:19:05,  6.45s/it]
 13%|█▎        | 28820/225000 [66:13:21<337:21:23,  6.19s/it]
 13%|█▎        | 28821/225000 [66:13:26<330:51:22,  6.07s/it]
 13%|█▎        | 28822/225000 [66:13:32<316:33:58,  5.81s/it]
 13%|█▎        | 28823/225000 [66:13:37<308:38:05,  5.66s/it]
 13%|█▎        | 28824/225000 [66:13:42<305:33:43,  5.61s/it]
 13%|█▎        | 28825/225000 [66:13:55<427:40:14,  7.85s/it]
 13%|█▎        | 28826/225000 [66:14:02<412:07:46,  7.56s/it]
 13%|█▎        | 28827/225000 [66:14:09<391:51:15,  7.19s/it]
 13%|█▎        | 28828/225000 [66:14:13<345:53:54,  6.35s/it]
 13%|█▎        | 28829/225000 [66:14:18<319:31:05,  5.86s/it]
 13%|█▎        | 28830/225000 [66:14:23<302:24:51,  5.55s/it]
 13%|█▎        | 28831/225000 [66:14:28<296:56:44,  5.45s/it]
 13%|█▎        | 28832/225000 [66:14:34<314:45:10,  5.78s/it]
 13%|█▎        | 28833/225000 [66:14:47<422:45:26,  7.76s/it]
 13%|█▎        | 28834/225000 [66:14:50<355:30:43,  6.52s/it]
 13%|█▎        | 28835/225000 [66:14:59<394:47:56,  7.25s/it]
 13%|█▎        | 28836/225000 [66:15:06<391:21:56,  7.18s/it]
 13%|█▎        | 28837/225000 [66:15:12<369:51:04,  6.79s/it]
 13%|█▎        | 28838/225000 [66:15:17<331:52:53,  6.09s/it]
 13%|█▎        | 28839/225000 [66:15:26<392:29:13,  7.20s/it]
 13%|█▎        | 28840/225000 [66:15:31<344:40:23,  6.33s/it]
 13%|█▎        | 28841/225000 [66:15:38<354:50:04,  6.51s/it]
 13%|█▎        | 28842/225000 [66:15:43<335:59:00,  6.17s/it]
 13%|█▎        | 28843/225000 [66:15:50<346:17:49,  6.36s/it]
 13%|█▎        | 28844/225000 [66:15:53<301:52:33,  5.54s/it]
 13%|█▎        | 28845/225000 [66:16:03<371:05:11,  6.81s/it]
 13%|█▎        | 28846/225000 [66:16:10<377:57:49,  6.94s/it]
 13%|█▎        | 28847/225000 [66:16:15<338:01:29,  6.20s/it]
 13%|█▎        | 28848/225000 [66:16:20<313:14:31,  5.75s/it]
 13%|█▎        | 28849/225000 [66:16:25<315:01:09,  5.78s/it]
 13%|█▎        | 28850/225000 [66:16:38<424:25:56,  7.79s/it]
                                                             
{'loss': '0.7393', 'grad_norm': '0.657', 'learning_rate': '4.906e-05', 'epoch': '0.1282', 'num_input_tokens_seen': 113671112, 'train_runtime': '2.386e+05', 'train_tokens_per_second': '476.4'}

 13%|█▎        | 28850/225000 [66:16:38<424:25:56,  7.79s/it]
 13%|█▎        | 28851/225000 [66:16:41<350:07:15,  6.43s/it]
 13%|█▎        | 28852/225000 [66:16:49<371:02:08,  6.81s/it]
 13%|█▎        | 28853/225000 [66:16:53<320:09:00,  5.88s/it]
 13%|█▎        | 28854/225000 [66:16:58<315:00:54,  5.78s/it]
 13%|█▎        | 28855/225000 [66:17:04<311:40:19,  5.72s/it]
 13%|█▎        | 28856/225000 [66:17:09<305:55:41,  5.61s/it]
 13%|█▎        | 28857/225000 [66:17:15<312:16:03,  5.73s/it]
 13%|█▎        | 28858/225000 [66:17:21<308:21:36,  5.66s/it]
 13%|█▎        | 28859/225000 [66:17:26<302:03:39,  5.54s/it]
 13%|█▎        | 28860/225000 [66:17:32<307:15:47,  5.64s/it]
 13%|█▎        | 28861/225000 [66:17:38<311:48:17,  5.72s/it]
 13%|█▎        | 28862/225000 [66:17:50<424:40:53,  7.79s/it]
 13%|█▎        | 28863/225000 [66:17:58<425:44:47,  7.81s/it]
 13%|█▎        | 28864/225000 [66:18:02<352:39:40,  6.47s/it]
 13%|█▎        | 28865/225000 [66:18:07<340:36:06,  6.25s/it]
 13%|█▎        | 28866/225000 [66:18:13<332:37:03,  6.11s/it]
 13%|█▎        | 28867/225000 [66:18:19<323:36:07,  5.94s/it]
 13%|█▎        | 28868/225000 [66:18:24<310:25:56,  5.70s/it]
 13%|█▎        | 28869/225000 [66:18:30<316:28:18,  5.81s/it]
 13%|█▎        | 28870/225000 [66:18:35<314:12:26,  5.77s/it]
 13%|█▎        | 28871/225000 [66:18:43<350:33:51,  6.43s/it]
 13%|█▎        | 28872/225000 [66:18:56<452:20:13,  8.30s/it]
 13%|█▎        | 28873/225000 [66:19:00<374:08:13,  6.87s/it]
 13%|█▎        | 28874/225000 [66:19:09<412:07:55,  7.56s/it]
 13%|█▎        | 28875/225000 [66:19:14<367:20:49,  6.74s/it]
 13%|█▎        | 28876/225000 [66:19:18<331:42:13,  6.09s/it]
 13%|█▎        | 28877/225000 [66:19:23<316:12:24,  5.80s/it]
 13%|█▎        | 28878/225000 [66:19:34<392:01:37,  7.20s/it]
 13%|█▎        | 28879/225000 [66:19:42<412:53:17,  7.58s/it]
 13%|█▎        | 28880/225000 [66:19:46<345:55:00,  6.35s/it]
 13%|█▎        | 28881/225000 [66:19:55<396:13:02,  7.27s/it]
 13%|█▎        | 28882/225000 [66:19:59<333:57:59,  6.13s/it]
 13%|█▎        | 28883/225000 [66:20:08<381:06:31,  7.00s/it]
 13%|█▎        | 28884/225000 [66:20:16<396:56:38,  7.29s/it]
 13%|█▎        | 28885/225000 [66:20:21<358:13:06,  6.58s/it]
 13%|█▎        | 28886/225000 [66:20:29<387:07:57,  7.11s/it]
 13%|█▎        | 28887/225000 [66:20:33<334:02:01,  6.13s/it]
 13%|█▎        | 28888/225000 [66:20:38<322:55:52,  5.93s/it]
 13%|█▎        | 28889/225000 [66:20:43<303:04:42,  5.56s/it]
 13%|█▎        | 28890/225000 [66:20:48<294:41:30,  5.41s/it]
 13%|█▎        | 28891/225000 [66:20:53<297:26:27,  5.46s/it]
 13%|█▎        | 28892/225000 [66:20:59<293:44:30,  5.39s/it]
 13%|█▎        | 28893/225000 [66:21:08<353:01:34,  6.48s/it]
 13%|█▎        | 28894/225000 [66:21:20<445:09:15,  8.17s/it]
 13%|█▎        | 28895/225000 [66:21:26<405:20:55,  7.44s/it]
 13%|█▎        | 28896/225000 [66:21:32<390:11:25,  7.16s/it]
 13%|█▎        | 28897/225000 [66:21:40<397:07:15,  7.29s/it]
 13%|█▎        | 28898/225000 [66:21:43<337:06:41,  6.19s/it]
 13%|█▎        | 28899/225000 [66:21:52<374:21:29,  6.87s/it]
 13%|█▎        | 28900/225000 [66:21:56<338:44:16,  6.22s/it]
                                                             
{'loss': '0.7164', 'grad_norm': '0.6278', 'learning_rate': '4.905e-05', 'epoch': '0.1284', 'num_input_tokens_seen': 113816728, 'train_runtime': '2.389e+05', 'train_tokens_per_second': '476.4'}

 13%|█▎        | 28900/225000 [66:21:56<338:44:16,  6.22s/it]
 13%|█▎        | 28901/225000 [66:22:07<408:30:45,  7.50s/it]
 13%|█▎        | 28902/225000 [66:22:11<349:18:29,  6.41s/it]
 13%|█▎        | 28903/225000 [66:22:19<374:26:15,  6.87s/it]
 13%|█▎        | 28904/225000 [66:22:24<344:52:31,  6.33s/it]
 13%|█▎        | 28905/225000 [66:22:28<311:47:47,  5.72s/it]
 13%|█▎        | 28906/225000 [66:22:40<411:17:55,  7.55s/it]
 13%|█▎        | 28907/225000 [66:22:44<355:19:34,  6.52s/it]
 13%|█▎        | 28908/225000 [66:22:48<314:31:01,  5.77s/it]
 13%|█▎        | 28909/225000 [66:22:54<321:46:18,  5.91s/it]
 13%|█▎        | 28910/225000 [66:23:04<380:51:19,  6.99s/it]
 13%|█▎        | 28911/225000 [66:23:10<370:46:10,  6.81s/it]
 13%|█▎        | 28912/225000 [66:23:16<357:40:42,  6.57s/it]
 13%|█▎        | 28913/225000 [66:23:21<327:29:48,  6.01s/it]
 13%|█▎        | 28914/225000 [66:23:31<397:00:28,  7.29s/it]
 13%|█▎        | 28915/225000 [66:23:35<336:07:25,  6.17s/it]
 13%|█▎        | 28916/225000 [66:23:45<402:26:04,  7.39s/it]
 13%|█▎        | 28917/225000 [66:23:49<349:50:12,  6.42s/it]
 13%|█▎        | 28918/225000 [66:23:56<348:38:40,  6.40s/it]
 13%|█▎        | 28919/225000 [66:24:03<366:17:15,  6.72s/it]
 13%|█▎        | 28920/225000 [66:24:08<330:03:45,  6.06s/it]
 13%|█▎        | 28921/225000 [66:24:17<389:47:54,  7.16s/it]
 13%|█▎        | 28922/225000 [66:24:23<358:51:54,  6.59s/it]
 13%|█▎        | 28923/225000 [66:24:30<370:09:59,  6.80s/it]
 13%|█▎        | 28924/225000 [66:24:34<331:39:27,  6.09s/it]
 13%|█▎        | 28925/225000 [66:24:43<370:37:06,  6.80s/it]
 13%|█▎        | 28926/225000 [66:24:48<342:46:17,  6.29s/it]
 13%|█▎        | 28927/225000 [66:24:53<325:51:10,  5.98s/it]
 13%|█▎        | 28928/225000 [66:24:58<312:15:05,  5.73s/it]
 13%|█▎        | 28929/225000 [66:25:04<304:40:49,  5.59s/it]
 13%|█▎        | 28930/225000 [66:25:12<349:27:12,  6.42s/it]
 13%|█▎        | 28931/225000 [66:25:21<396:43:54,  7.28s/it]
 13%|█▎        | 28932/225000 [66:25:25<342:09:38,  6.28s/it]
 13%|█▎        | 28933/225000 [66:25:33<364:19:33,  6.69s/it]
 13%|█▎        | 28934/225000 [66:25:43<416:01:52,  7.64s/it]
 13%|█▎        | 28935/225000 [66:25:47<367:43:25,  6.75s/it]
 13%|█▎        | 28936/225000 [66:25:52<328:25:41,  6.03s/it]
 13%|█▎        | 28937/225000 [66:26:01<385:42:19,  7.08s/it]
 13%|█▎        | 28938/225000 [66:26:10<414:08:23,  7.60s/it]
 13%|█▎        | 28939/225000 [66:26:14<349:10:08,  6.41s/it]
 13%|█▎        | 28940/225000 [66:26:17<303:13:19,  5.57s/it]
 13%|█▎        | 28941/225000 [66:26:26<358:14:38,  6.58s/it]
 13%|█▎        | 28942/225000 [66:26:33<369:58:21,  6.79s/it]
 13%|█▎        | 28943/225000 [66:26:37<321:31:20,  5.90s/it]
 13%|█▎        | 28944/225000 [66:26:48<398:54:10,  7.32s/it]
 13%|█▎        | 28945/225000 [66:26:51<336:26:14,  6.18s/it]
 13%|█▎        | 28946/225000 [66:27:00<376:41:46,  6.92s/it]
 13%|█▎        | 28947/225000 [66:27:06<361:02:20,  6.63s/it]
 13%|█▎        | 28948/225000 [66:27:14<384:28:17,  7.06s/it]
 13%|█▎        | 28949/225000 [66:27:18<334:51:43,  6.15s/it]
 13%|█▎        | 28950/225000 [66:27:22<305:29:37,  5.61s/it]
                                                             
{'loss': '0.7447', 'grad_norm': '0.9913', 'learning_rate': '4.905e-05', 'epoch': '0.1287', 'num_input_tokens_seen': 113962792, 'train_runtime': '2.392e+05', 'train_tokens_per_second': '476.3'}

 13%|█▎        | 28950/225000 [66:27:22<305:29:37,  5.61s/it]
 13%|█▎        | 28951/225000 [66:27:29<317:31:28,  5.83s/it]
 13%|█▎        | 28952/225000 [66:27:38<368:15:26,  6.76s/it]
 13%|█▎        | 28953/225000 [66:27:44<355:23:09,  6.53s/it]
 13%|█▎        | 28954/225000 [66:27:48<314:36:01,  5.78s/it]
 13%|█▎        | 28955/225000 [66:27:58<389:06:30,  7.15s/it]
 13%|█▎        | 28956/225000 [66:28:03<347:54:05,  6.39s/it]
 13%|█▎        | 28957/225000 [66:28:08<327:59:28,  6.02s/it]
 13%|█▎        | 28958/225000 [66:28:13<310:04:30,  5.69s/it]
 13%|█▎        | 28959/225000 [66:28:17<291:14:09,  5.35s/it]
 13%|█▎        | 28960/225000 [66:28:22<285:24:25,  5.24s/it]
 13%|█▎        | 28961/225000 [66:28:31<343:42:04,  6.31s/it]
 13%|█▎        | 28962/225000 [66:28:42<424:23:20,  7.79s/it]
 13%|█▎        | 28963/225000 [66:28:46<358:47:23,  6.59s/it]
 13%|█▎        | 28964/225000 [66:28:50<317:22:17,  5.83s/it]
 13%|█▎        | 28965/225000 [66:29:01<397:31:37,  7.30s/it]
 13%|█▎        | 28966/225000 [66:29:07<371:49:00,  6.83s/it]
 13%|█▎        | 28967/225000 [66:29:11<335:11:29,  6.16s/it]
 13%|█▎        | 28968/225000 [66:29:19<368:29:07,  6.77s/it]
 13%|█▎        | 28969/225000 [66:29:24<338:29:10,  6.22s/it]
 13%|█▎        | 28970/225000 [66:29:32<367:49:12,  6.75s/it]
 13%|█▎        | 28971/225000 [66:29:36<313:54:35,  5.76s/it]
 13%|█▎        | 28972/225000 [66:29:40<286:54:44,  5.27s/it]
 13%|█▎        | 28973/225000 [66:29:50<368:40:04,  6.77s/it]
 13%|█▎        | 28974/225000 [66:29:59<402:01:22,  7.38s/it]
 13%|█▎        | 28975/225000 [66:30:03<352:33:56,  6.47s/it]
 13%|█▎        | 28976/225000 [66:30:08<324:41:03,  5.96s/it]
 13%|█▎        | 28977/225000 [66:30:13<309:39:48,  5.69s/it]
 13%|█▎        | 28978/225000 [66:30:20<330:14:15,  6.06s/it]
 13%|█▎        | 28979/225000 [66:30:30<395:16:44,  7.26s/it]
 13%|█▎        | 28980/225000 [66:30:34<341:11:12,  6.27s/it]
 13%|█▎        | 28981/225000 [66:30:40<328:40:57,  6.04s/it]
 13%|█▎        | 28982/225000 [66:30:45<318:48:47,  5.86s/it]
 13%|█▎        | 28983/225000 [66:30:51<312:57:43,  5.75s/it]
 13%|█▎        | 28984/225000 [66:30:56<313:30:00,  5.76s/it]
 13%|█▎        | 28985/225000 [66:31:02<308:32:52,  5.67s/it]
 13%|█▎        | 28986/225000 [66:31:07<303:55:15,  5.58s/it]
 13%|█▎        | 28987/225000 [66:31:13<302:48:20,  5.56s/it]
 13%|█▎        | 28988/225000 [66:31:24<398:29:35,  7.32s/it]
 13%|█▎        | 28989/225000 [66:31:33<420:45:35,  7.73s/it]
 13%|█▎        | 28990/225000 [66:31:38<373:18:31,  6.86s/it]
 13%|█▎        | 28991/225000 [66:31:46<395:15:18,  7.26s/it]
 13%|█▎        | 28992/225000 [66:31:54<401:56:53,  7.38s/it]
 13%|█▎        | 28993/225000 [66:31:57<346:03:57,  6.36s/it]
 13%|█▎        | 28994/225000 [66:32:02<308:33:59,  5.67s/it]
 13%|█▎        | 28995/225000 [66:32:14<415:41:22,  7.63s/it]
 13%|█▎        | 28996/225000 [66:32:17<344:55:46,  6.34s/it]
 13%|█▎        | 28997/225000 [66:32:24<350:51:27,  6.44s/it]
 13%|█▎        | 28998/225000 [66:32:30<345:05:31,  6.34s/it]
 13%|█▎        | 28999/225000 [66:32:35<329:43:01,  6.06s/it]
 13%|█▎        | 29000/225000 [66:32:41<323:01:04,  5.93s/it]
                                                             
{'loss': '0.711', 'grad_norm': '0.846', 'learning_rate': '4.904e-05', 'epoch': '0.1289', 'num_input_tokens_seen': 114105760, 'train_runtime': '2.396e+05', 'train_tokens_per_second': '476.3'}

 13%|█▎        | 29000/225000 [66:32:41<323:01:04,  5.93s/it]
 13%|█▎        | 29001/225000 [66:32:46<305:33:45,  5.61s/it]
 13%|█▎        | 29002/225000 [66:32:51<299:54:32,  5.51s/it]
 13%|█▎        | 29003/225000 [66:32:57<302:03:02,  5.55s/it]
 13%|█▎        | 29004/225000 [66:33:09<418:33:25,  7.69s/it]
 13%|█▎        | 29005/225000 [66:33:15<379:03:51,  6.96s/it]
 13%|█▎        | 29006/225000 [66:33:21<366:26:26,  6.73s/it]
 13%|█▎        | 29007/225000 [66:33:29<382:30:16,  7.03s/it]
 13%|█▎        | 29008/225000 [66:33:33<337:17:14,  6.20s/it]
 13%|█▎        | 29009/225000 [66:33:40<357:50:51,  6.57s/it]
 13%|█▎        | 29010/225000 [66:33:48<370:17:33,  6.80s/it]
 13%|█▎        | 29011/225000 [66:33:52<331:48:24,  6.09s/it]
 13%|█▎        | 29012/225000 [66:33:57<313:47:39,  5.76s/it]
 13%|█▎        | 29013/225000 [66:34:04<329:13:47,  6.05s/it]
 13%|█▎        | 29014/225000 [66:34:09<317:05:04,  5.82s/it]
 13%|█▎        | 29015/225000 [66:34:15<312:14:34,  5.74s/it]
 13%|█▎        | 29016/225000 [66:34:20<303:30:17,  5.58s/it]
 13%|█▎        | 29017/225000 [66:34:25<301:50:55,  5.54s/it]
 13%|█▎        | 29018/225000 [66:34:30<296:11:10,  5.44s/it]
 13%|█▎        | 29019/225000 [66:34:38<338:45:49,  6.22s/it]
 13%|█▎        | 29020/225000 [66:34:52<458:04:09,  8.41s/it]
 13%|█▎        | 29021/225000 [66:34:59<427:53:51,  7.86s/it]
 13%|█▎        | 29022/225000 [66:35:05<412:49:41,  7.58s/it]
 13%|█▎        | 29023/225000 [66:35:10<356:47:23,  6.55s/it]
 13%|█▎        | 29024/225000 [66:35:21<428:16:07,  7.87s/it]
 13%|█▎        | 29025/225000 [66:35:24<358:02:04,  6.58s/it]
 13%|█▎        | 29026/225000 [66:35:29<331:37:49,  6.09s/it]
 13%|█▎        | 29027/225000 [66:35:34<315:24:59,  5.79s/it]
 13%|█▎        | 29028/225000 [66:35:40<320:12:56,  5.88s/it]
 13%|█▎        | 29029/225000 [66:35:45<308:20:32,  5.66s/it]
 13%|█▎        | 29030/225000 [66:35:51<307:37:09,  5.65s/it]
 13%|█▎        | 29031/225000 [66:35:57<304:38:35,  5.60s/it]
 13%|█▎        | 29032/225000 [66:36:02<301:42:09,  5.54s/it]
 13%|█▎        | 29033/225000 [66:36:14<408:09:57,  7.50s/it]
 13%|█▎        | 29034/225000 [66:36:21<396:17:26,  7.28s/it]
 13%|█▎        | 29035/225000 [66:36:26<359:37:24,  6.61s/it]
 13%|█▎        | 29036/225000 [66:36:36<422:25:09,  7.76s/it]
 13%|█▎        | 29037/225000 [66:36:41<367:19:54,  6.75s/it]
 13%|█▎        | 29038/225000 [66:36:45<325:25:17,  5.98s/it]
 13%|█▎        | 29039/225000 [66:36:50<304:06:43,  5.59s/it]
 13%|█▎        | 29040/225000 [66:37:01<392:18:43,  7.21s/it]
 13%|█▎        | 29041/225000 [66:37:05<346:45:38,  6.37s/it]
 13%|█▎        | 29042/225000 [66:37:09<309:53:47,  5.69s/it]
 13%|█▎        | 29043/225000 [66:37:19<380:49:40,  7.00s/it]
 13%|█▎        | 29044/225000 [66:37:25<364:01:56,  6.69s/it]
 13%|█▎        | 29045/225000 [66:37:32<372:52:31,  6.85s/it]
 13%|█▎        | 29046/225000 [66:37:38<356:36:21,  6.55s/it]
 13%|█▎        | 29047/225000 [66:37:46<377:48:52,  6.94s/it]
 13%|█▎        | 29048/225000 [66:37:53<372:25:14,  6.84s/it]
 13%|█▎        | 29049/225000 [66:37:59<367:37:19,  6.75s/it]
 13%|█▎        | 29050/225000 [66:38:05<360:44:32,  6.63s/it]
                                                             
{'loss': '0.7175', 'grad_norm': '0.7643', 'learning_rate': '4.904e-05', 'epoch': '0.1291', 'num_input_tokens_seen': 114249400, 'train_runtime': '2.399e+05', 'train_tokens_per_second': '476.3'}

 13%|█▎        | 29050/225000 [66:38:05<360:44:32,  6.63s/it]
 13%|█▎        | 29051/225000 [66:38:11<340:05:55,  6.25s/it]
 13%|█▎        | 29052/225000 [66:38:18<362:34:11,  6.66s/it]
 13%|█▎        | 29053/225000 [66:38:23<333:51:00,  6.13s/it]
 13%|█▎        | 29054/225000 [66:38:28<313:06:38,  5.75s/it]
 13%|█▎        | 29055/225000 [66:38:38<384:07:26,  7.06s/it]
 13%|█▎        | 29056/225000 [66:38:45<373:18:22,  6.86s/it]
 13%|█▎        | 29057/225000 [66:38:50<345:57:43,  6.36s/it]
 13%|█▎        | 29058/225000 [66:38:55<325:38:14,  5.98s/it]
 13%|█▎        | 29059/225000 [66:39:01<318:41:02,  5.86s/it]
 13%|█▎        | 29060/225000 [66:39:05<302:40:17,  5.56s/it]
 13%|█▎        | 29061/225000 [66:39:11<298:06:27,  5.48s/it]
 13%|█▎        | 29062/225000 [66:39:16<295:45:19,  5.43s/it]
 13%|█▎        | 29063/225000 [66:39:29<418:53:26,  7.70s/it]
 13%|█▎        | 29064/225000 [66:39:35<389:12:21,  7.15s/it]
 13%|█▎        | 29065/225000 [66:39:42<388:54:15,  7.15s/it]
 13%|█▎        | 29066/225000 [66:39:48<371:57:25,  6.83s/it]
 13%|█▎        | 29067/225000 [66:39:53<334:04:26,  6.14s/it]
 13%|█▎        | 29068/225000 [66:39:58<328:39:55,  6.04s/it]
 13%|█▎        | 29069/225000 [66:40:03<308:50:56,  5.67s/it]
 13%|█▎        | 29070/225000 [66:40:09<306:10:03,  5.63s/it]
 13%|█▎        | 29071/225000 [66:40:15<309:12:23,  5.68s/it]
 13%|█▎        | 29072/225000 [66:40:24<372:13:26,  6.84s/it]
 13%|█▎        | 29073/225000 [66:40:29<347:19:37,  6.38s/it]
 13%|█▎        | 29074/225000 [66:40:35<327:19:28,  6.01s/it]
 13%|█▎        | 29075/225000 [66:40:40<316:38:01,  5.82s/it]
 13%|█▎        | 29076/225000 [66:40:46<315:32:18,  5.80s/it]
 13%|█▎        | 29077/225000 [66:40:51<311:04:18,  5.72s/it]
 13%|█▎        | 29078/225000 [66:40:59<340:50:43,  6.26s/it]
 13%|█▎        | 29079/225000 [66:41:12<449:01:11,  8.25s/it]
 13%|█▎        | 29080/225000 [66:41:16<379:43:10,  6.98s/it]
 13%|█▎        | 29081/225000 [66:41:24<395:10:34,  7.26s/it]
 13%|█▎        | 29082/225000 [66:41:32<411:40:21,  7.56s/it]
 13%|█▎        | 29083/225000 [66:41:37<367:50:02,  6.76s/it]
 13%|█▎        | 29084/225000 [66:41:43<356:16:40,  6.55s/it]
 13%|█▎        | 29085/225000 [66:41:50<361:02:48,  6.63s/it]
 13%|█▎        | 29086/225000 [66:41:56<357:32:39,  6.57s/it]
 13%|█▎        | 29087/225000 [66:42:00<307:29:42,  5.65s/it]
 13%|█▎        | 29088/225000 [66:42:08<344:44:08,  6.33s/it]
 13%|█▎        | 29089/225000 [66:42:13<338:09:11,  6.21s/it]
 13%|█▎        | 29090/225000 [66:42:21<367:19:54,  6.75s/it]
 13%|█▎        | 29091/225000 [66:42:26<338:07:22,  6.21s/it]
 13%|█▎        | 29092/225000 [66:42:35<374:36:54,  6.88s/it]
 13%|█▎        | 29093/225000 [66:42:43<399:06:24,  7.33s/it]
 13%|█▎        | 29094/225000 [66:42:48<354:53:28,  6.52s/it]
 13%|█▎        | 29095/225000 [66:42:52<317:16:05,  5.83s/it]
 13%|█▎        | 29096/225000 [66:42:56<292:06:26,  5.37s/it]
 13%|█▎        | 29097/225000 [66:43:05<339:07:55,  6.23s/it]
 13%|█▎        | 29098/225000 [66:43:13<371:58:12,  6.84s/it]
 13%|█▎        | 29099/225000 [66:43:17<326:54:56,  6.01s/it]
 13%|█▎        | 29100/225000 [66:43:26<380:13:17,  6.99s/it]
                                                             
{'loss': '0.7229', 'grad_norm': '0.644', 'learning_rate': '4.903e-05', 'epoch': '0.1293', 'num_input_tokens_seen': 114392288, 'train_runtime': '2.402e+05', 'train_tokens_per_second': '476.2'}

 13%|█▎        | 29100/225000 [66:43:26<380:13:17,  6.99s/it]
 13%|█▎        | 29101/225000 [66:43:32<359:33:30,  6.61s/it]
 13%|█▎        | 29102/225000 [66:43:37<327:00:58,  6.01s/it]
 13%|█▎        | 29103/225000 [66:43:42<315:39:55,  5.80s/it]
 13%|█▎        | 29104/225000 [66:43:47<306:41:43,  5.64s/it]
 13%|█▎        | 29105/225000 [66:43:53<305:46:10,  5.62s/it]
 13%|█▎        | 29106/225000 [66:44:06<425:03:27,  7.81s/it]
 13%|█▎        | 29107/225000 [66:44:10<368:02:08,  6.76s/it]
 13%|█▎        | 29108/225000 [66:44:15<335:28:46,  6.17s/it]
 13%|█▎        | 29109/225000 [66:44:19<298:56:24,  5.49s/it]
 13%|█▎        | 29110/225000 [66:44:28<368:25:26,  6.77s/it]
 13%|█▎        | 29111/225000 [66:44:36<376:58:16,  6.93s/it]
 13%|█▎        | 29112/225000 [66:44:42<360:14:25,  6.62s/it]
 13%|█▎        | 29113/225000 [66:44:47<342:02:30,  6.29s/it]
 13%|█▎        | 29114/225000 [66:44:57<399:02:16,  7.33s/it]
 13%|█▎        | 29115/225000 [66:45:02<357:36:02,  6.57s/it]
 13%|█▎        | 29116/225000 [66:45:08<352:49:33,  6.48s/it]
 13%|█▎        | 29117/225000 [66:45:12<315:08:31,  5.79s/it]
 13%|█▎        | 29118/225000 [66:45:22<389:07:11,  7.15s/it]
 13%|█▎        | 29119/225000 [66:45:26<338:04:41,  6.21s/it]
 13%|█▎        | 29120/225000 [66:45:35<377:29:00,  6.94s/it]
 13%|█▎        | 29121/225000 [66:45:41<360:12:59,  6.62s/it]
 13%|█▎        | 29122/225000 [66:45:47<342:40:43,  6.30s/it]
 13%|█▎        | 29123/225000 [66:45:52<327:01:03,  6.01s/it]
 13%|█▎        | 29124/225000 [66:45:57<317:46:14,  5.84s/it]
 13%|█▎        | 29125/225000 [66:46:03<319:36:00,  5.87s/it]
 13%|█▎        | 29126/225000 [66:46:09<309:34:23,  5.69s/it]
 13%|█▎        | 29127/225000 [66:46:14<309:07:28,  5.68s/it]
 13%|█▎        | 29128/225000 [66:46:20<305:15:12,  5.61s/it]
 13%|█▎        | 29129/225000 [66:46:25<300:39:48,  5.53s/it]
 13%|█▎        | 29130/225000 [66:46:38<427:35:36,  7.86s/it]
 13%|█▎        | 29131/225000 [66:46:43<382:53:32,  7.04s/it]
 13%|█▎        | 29132/225000 [66:46:52<408:53:04,  7.52s/it]
 13%|█▎        | 29133/225000 [66:46:56<353:16:10,  6.49s/it]
 13%|█▎        | 29134/225000 [66:47:02<336:42:49,  6.19s/it]
 13%|█▎        | 29135/225000 [66:47:07<326:19:25,  6.00s/it]
 13%|█▎        | 29136/225000 [66:47:18<409:28:31,  7.53s/it]
 13%|█▎        | 29137/225000 [66:47:26<415:25:34,  7.64s/it]
 13%|█▎        | 29138/225000 [66:47:30<360:12:41,  6.62s/it]
 13%|█▎        | 29139/225000 [66:47:35<322:19:02,  5.92s/it]
 13%|█▎        | 29140/225000 [66:47:45<394:00:31,  7.24s/it]
 13%|█▎        | 29141/225000 [66:47:50<349:05:57,  6.42s/it]
 13%|█▎        | 29142/225000 [66:47:54<310:37:12,  5.71s/it]
 13%|█▎        | 29143/225000 [66:47:59<308:35:39,  5.67s/it]
 13%|█▎        | 29144/225000 [66:48:10<397:04:34,  7.30s/it]
 13%|█▎        | 29145/225000 [66:48:14<334:16:08,  6.14s/it]
 13%|█▎        | 29146/225000 [66:48:21<358:23:55,  6.59s/it]
 13%|█▎        | 29147/225000 [66:48:25<307:37:27,  5.65s/it]
 13%|█▎        | 29148/225000 [66:48:29<284:14:32,  5.22s/it]
 13%|█▎        | 29149/225000 [66:48:41<402:48:22,  7.40s/it]
 13%|█▎        | 29150/225000 [66:48:46<355:50:18,  6.54s/it]
                                                             
{'loss': '0.7159', 'grad_norm': '0.694', 'learning_rate': '4.903e-05', 'epoch': '0.1296', 'num_input_tokens_seen': 114536736, 'train_runtime': '2.405e+05', 'train_tokens_per_second': '476.2'}

 13%|█▎        | 29150/225000 [66:48:46<355:50:18,  6.54s/it]
 13%|█▎        | 29151/225000 [66:48:50<320:04:47,  5.88s/it]
 13%|█▎        | 29152/225000 [66:48:56<321:48:59,  5.92s/it]
 13%|█▎        | 29153/225000 [66:49:07<391:01:36,  7.19s/it]
 13%|█▎        | 29154/225000 [66:49:15<407:53:20,  7.50s/it]
 13%|█▎        | 29155/225000 [66:49:21<387:12:04,  7.12s/it]
 13%|█▎        | 29156/225000 [66:49:27<377:11:30,  6.93s/it]
 13%|█▎        | 29157/225000 [66:49:32<334:17:44,  6.15s/it]
 13%|█▎        | 29158/225000 [66:49:36<305:59:35,  5.62s/it]
 13%|█▎        | 29159/225000 [66:49:45<358:51:50,  6.60s/it]
 13%|█▎        | 29160/225000 [66:49:52<371:13:28,  6.82s/it]
 13%|█▎        | 29161/225000 [66:49:58<352:45:34,  6.48s/it]
 13%|█▎        | 29162/225000 [66:50:07<396:16:02,  7.28s/it]
 13%|█▎        | 29163/225000 [66:50:12<348:45:39,  6.41s/it]
 13%|█▎        | 29164/225000 [66:50:16<320:05:31,  5.88s/it]
 13%|█▎        | 29165/225000 [66:50:26<374:50:49,  6.89s/it]
 13%|█▎        | 29166/225000 [66:50:31<353:46:31,  6.50s/it]
 13%|█▎        | 29167/225000 [66:50:37<346:36:26,  6.37s/it]
 13%|█▎        | 29168/225000 [66:50:46<381:46:08,  7.02s/it]
 13%|█▎        | 29169/225000 [66:50:50<333:22:19,  6.13s/it]
 13%|█▎        | 29170/225000 [66:50:54<299:07:47,  5.50s/it]
 13%|█▎        | 29171/225000 [66:51:04<374:54:56,  6.89s/it]
 13%|█▎        | 29172/225000 [66:51:08<332:35:32,  6.11s/it]
 13%|█▎        | 29173/225000 [66:51:17<376:19:00,  6.92s/it]
 13%|█▎        | 29174/225000 [66:51:23<368:14:35,  6.77s/it]
 13%|█▎        | 29175/225000 [66:51:28<329:18:41,  6.05s/it]
 13%|█▎        | 29176/225000 [66:51:33<311:44:32,  5.73s/it]
 13%|█▎        | 29177/225000 [66:51:38<296:57:44,  5.46s/it]
 13%|█▎        | 29178/225000 [66:51:43<289:33:23,  5.32s/it]
 13%|█▎        | 29179/225000 [66:51:48<286:41:05,  5.27s/it]
 13%|█▎        | 29180/225000 [66:51:59<390:39:29,  7.18s/it]
 13%|█▎        | 29181/225000 [66:52:08<410:18:04,  7.54s/it]
 13%|█▎        | 29182/225000 [66:52:12<355:56:00,  6.54s/it]
 13%|█▎        | 29183/225000 [66:52:21<399:07:16,  7.34s/it]
 13%|█▎        | 29184/225000 [66:52:25<349:05:18,  6.42s/it]
 13%|█▎        | 29185/225000 [66:52:30<315:07:01,  5.79s/it]
 13%|█▎        | 29186/225000 [66:52:34<293:14:49,  5.39s/it]
 13%|█▎        | 29187/225000 [66:52:39<289:45:31,  5.33s/it]
 13%|█▎        | 29188/225000 [66:52:50<373:17:05,  6.86s/it]
 13%|█▎        | 29189/225000 [66:52:55<351:08:44,  6.46s/it]
 13%|█▎        | 29190/225000 [66:53:00<315:34:08,  5.80s/it]
 13%|█▎        | 29191/225000 [66:53:04<297:48:12,  5.48s/it]
 13%|█▎        | 29192/225000 [66:53:16<399:43:19,  7.35s/it]
 13%|█▎        | 29193/225000 [66:53:21<364:31:01,  6.70s/it]
 13%|█▎        | 29194/225000 [66:53:26<332:09:21,  6.11s/it]
 13%|█▎        | 29195/225000 [66:53:30<304:49:19,  5.60s/it]
 13%|█▎        | 29196/225000 [66:53:37<324:15:46,  5.96s/it]
 13%|█▎        | 29197/225000 [66:53:48<407:55:10,  7.50s/it]
 13%|█▎        | 29198/225000 [66:53:53<360:59:28,  6.64s/it]
 13%|█▎        | 29199/225000 [66:53:58<330:59:32,  6.09s/it]
 13%|█▎        | 29200/225000 [66:54:02<305:09:20,  5.61s/it]
                                                             
{'loss': '0.7201', 'grad_norm': '0.7016', 'learning_rate': '4.902e-05', 'epoch': '0.1298', 'num_input_tokens_seen': 114681568, 'train_runtime': '2.408e+05', 'train_tokens_per_second': '476.2'}

 13%|█▎        | 29200/225000 [66:54:02<305:09:20,  5.61s/it]
 13%|█▎        | 29201/225000 [66:54:10<346:59:15,  6.38s/it]
 13%|█▎        | 29202/225000 [66:54:19<377:41:31,  6.94s/it]
 13%|█▎        | 29203/225000 [66:54:23<331:07:34,  6.09s/it]
 13%|█▎        | 29204/225000 [66:54:28<313:11:46,  5.76s/it]
 13%|█▎        | 29205/225000 [66:54:33<306:29:46,  5.64s/it]
 13%|█▎        | 29206/225000 [66:54:40<323:25:50,  5.95s/it]
 13%|█▎        | 29207/225000 [66:54:53<433:47:50,  7.98s/it]
 13%|█▎        | 29208/225000 [66:54:56<363:16:14,  6.68s/it]
 13%|█▎        | 29209/225000 [66:55:05<400:02:31,  7.36s/it]
 13%|█▎        | 29210/225000 [66:55:09<348:29:36,  6.41s/it]
 13%|█▎        | 29211/225000 [66:55:14<315:28:48,  5.80s/it]
 13%|█▎        | 29212/225000 [66:55:20<319:48:34,  5.88s/it]
 13%|█▎        | 29213/225000 [66:55:31<404:23:49,  7.44s/it]
 13%|█▎        | 29214/225000 [66:55:37<390:47:13,  7.19s/it]
 13%|█▎        | 29215/225000 [66:55:42<350:23:34,  6.44s/it]
 13%|█▎        | 29216/225000 [66:55:51<384:35:41,  7.07s/it]
 13%|█▎        | 29217/225000 [66:55:54<329:14:30,  6.05s/it]
 13%|█▎        | 29218/225000 [66:55:59<308:05:45,  5.67s/it]
 13%|█▎        | 29219/225000 [66:56:04<293:21:26,  5.39s/it]
 13%|█▎        | 29220/225000 [66:56:10<306:41:21,  5.64s/it]
 13%|█▎        | 29221/225000 [66:56:22<404:24:52,  7.44s/it]
 13%|█▎        | 29222/225000 [66:56:28<393:21:46,  7.23s/it]
 13%|█▎        | 29223/225000 [66:56:33<351:51:58,  6.47s/it]
 13%|█▎        | 29224/225000 [66:56:41<376:37:02,  6.93s/it]
 13%|█▎        | 29225/225000 [66:56:45<331:37:57,  6.10s/it]
 13%|█▎        | 29226/225000 [66:56:50<306:51:54,  5.64s/it]
 13%|█▎        | 29227/225000 [66:56:58<340:30:21,  6.26s/it]
 13%|█▎        | 29228/225000 [66:57:05<361:14:18,  6.64s/it]
 13%|█▎        | 29229/225000 [66:57:13<384:21:04,  7.07s/it]
 13%|█▎        | 29230/225000 [66:57:17<334:40:38,  6.15s/it]
 13%|█▎        | 29231/225000 [66:57:22<310:50:29,  5.72s/it]
 13%|█▎        | 29232/225000 [66:57:26<288:17:03,  5.30s/it]
 13%|█▎        | 29233/225000 [66:57:35<343:53:31,  6.32s/it]
 13%|█▎        | 29234/225000 [66:57:43<364:32:39,  6.70s/it]
 13%|█▎        | 29235/225000 [66:57:49<357:16:07,  6.57s/it]
 13%|█▎        | 29236/225000 [66:57:57<380:25:54,  7.00s/it]
 13%|█▎        | 29237/225000 [66:58:04<376:01:54,  6.92s/it]
 13%|█▎        | 29238/225000 [66:58:09<352:45:42,  6.49s/it]
 13%|█▎        | 29239/225000 [66:58:15<343:06:38,  6.31s/it]
 13%|█▎        | 29240/225000 [66:58:22<360:55:15,  6.64s/it]
 13%|█▎        | 29241/225000 [66:58:27<332:24:20,  6.11s/it]
 13%|█▎        | 29242/225000 [66:58:33<322:37:43,  5.93s/it]
 13%|█▎        | 29243/225000 [66:58:38<304:40:18,  5.60s/it]
 13%|█▎        | 29244/225000 [66:58:42<290:19:04,  5.34s/it]
 13%|█▎        | 29245/225000 [66:58:52<361:58:32,  6.66s/it]
 13%|█▎        | 29246/225000 [66:59:00<390:22:49,  7.18s/it]
 13%|█▎        | 29247/225000 [66:59:07<376:44:52,  6.93s/it]
 13%|█▎        | 29248/225000 [66:59:11<329:04:26,  6.05s/it]
 13%|█▎        | 29249/225000 [66:59:20<375:17:56,  6.90s/it]
 13%|█▎        | 29250/225000 [66:59:27<378:35:36,  6.96s/it]
                                                             
{'loss': '0.7309', 'grad_norm': '0.6857', 'learning_rate': '4.902e-05', 'epoch': '0.13', 'num_input_tokens_seen': 114825528, 'train_runtime': '2.412e+05', 'train_tokens_per_second': '476.1'}

 13%|█▎        | 29250/225000 [66:59:27<378:35:36,  6.96s/it]
 13%|█▎        | 29251/225000 [66:59:33<359:22:15,  6.61s/it]
 13%|█▎        | 29252/225000 [66:59:37<331:15:48,  6.09s/it]
 13%|█▎        | 29253/225000 [66:59:42<305:19:29,  5.62s/it]
 13%|█▎        | 29254/225000 [66:59:53<397:19:12,  7.31s/it]
 13%|█▎        | 29255/225000 [66:59:57<348:39:13,  6.41s/it]
 13%|█▎        | 29256/225000 [67:00:02<323:35:15,  5.95s/it]
 13%|█▎        | 29257/225000 [67:00:10<356:42:23,  6.56s/it]
 13%|█▎        | 29258/225000 [67:00:19<386:04:22,  7.10s/it]
 13%|█▎        | 29259/225000 [67:00:24<363:17:02,  6.68s/it]
 13%|█▎        | 29260/225000 [67:00:31<356:37:28,  6.56s/it]
 13%|█▎        | 29261/225000 [67:00:40<397:53:55,  7.32s/it]
 13%|█▎        | 29262/225000 [67:00:45<369:32:52,  6.80s/it]
 13%|█▎        | 29263/225000 [67:00:50<336:53:48,  6.20s/it]
 13%|█▎        | 29264/225000 [67:00:56<338:36:49,  6.23s/it]
 13%|█▎        | 29265/225000 [67:01:06<399:22:14,  7.35s/it]
 13%|█▎        | 29266/225000 [67:01:10<346:06:47,  6.37s/it]
 13%|█▎        | 29267/225000 [67:01:15<309:55:09,  5.70s/it]
 13%|█▎        | 29268/225000 [67:01:26<405:57:13,  7.47s/it]
 13%|█▎        | 29269/225000 [67:01:30<346:04:17,  6.37s/it]
 13%|█▎        | 29270/225000 [67:01:36<338:46:40,  6.23s/it]
 13%|█▎        | 29271/225000 [67:01:40<301:17:27,  5.54s/it]
 13%|█▎        | 29272/225000 [67:01:44<283:32:09,  5.22s/it]
 13%|█▎        | 29273/225000 [67:01:50<295:28:50,  5.43s/it]
 13%|█▎        | 29274/225000 [67:02:02<395:22:43,  7.27s/it]
 13%|█▎        | 29275/225000 [67:02:07<365:53:31,  6.73s/it]
 13%|█▎        | 29276/225000 [67:02:15<381:33:02,  7.02s/it]
 13%|█▎        | 29277/225000 [67:02:21<359:05:22,  6.60s/it]
 13%|█▎        | 29278/225000 [67:02:28<377:25:00,  6.94s/it]
 13%|█▎        | 29279/225000 [67:02:35<376:23:08,  6.92s/it]
 13%|█▎        | 29280/225000 [67:02:40<338:17:26,  6.22s/it]
 13%|█▎        | 29281/225000 [67:02:46<333:42:52,  6.14s/it]
 13%|█▎        | 29282/225000 [67:02:52<341:13:09,  6.28s/it]
 13%|█▎        | 29283/225000 [67:02:58<325:36:56,  5.99s/it]
 13%|█▎        | 29284/225000 [67:03:03<315:26:55,  5.80s/it]
 13%|█▎        | 29285/225000 [67:03:08<296:34:23,  5.46s/it]
 13%|█▎        | 29286/225000 [67:03:13<292:35:54,  5.38s/it]
 13%|█▎        | 29287/225000 [67:03:24<384:40:37,  7.08s/it]
 13%|█▎        | 29288/225000 [67:03:31<382:40:39,  7.04s/it]
 13%|█▎        | 29289/225000 [67:03:37<375:27:31,  6.91s/it]
 13%|█▎        | 29290/225000 [67:03:41<325:24:22,  5.99s/it]
 13%|█▎        | 29291/225000 [67:03:51<384:54:35,  7.08s/it]
 13%|█▎        | 29292/225000 [67:03:55<341:49:36,  6.29s/it]
 13%|█▎        | 29293/225000 [67:04:04<385:14:43,  7.09s/it]
 13%|█▎        | 29294/225000 [67:04:08<329:18:37,  6.06s/it]
 13%|█▎        | 29295/225000 [67:04:12<303:29:50,  5.58s/it]
 13%|█▎        | 29296/225000 [67:04:20<331:32:36,  6.10s/it]
 13%|█▎        | 29297/225000 [67:04:29<375:24:20,  6.91s/it]
 13%|█▎        | 29298/225000 [67:04:36<383:40:14,  7.06s/it]
 13%|█▎        | 29299/225000 [67:04:41<354:46:47,  6.53s/it]
 13%|█▎        | 29300/225000 [67:04:46<321:37:48,  5.92s/it]
                                                             
{'loss': '0.7178', 'grad_norm': '0.774', 'learning_rate': '4.901e-05', 'epoch': '0.1302', 'num_input_tokens_seen': 114968336, 'train_runtime': '2.415e+05', 'train_tokens_per_second': '476.1'}

 13%|█▎        | 29300/225000 [67:04:46<321:37:48,  5.92s/it]
 13%|█▎        | 29301/225000 [67:04:50<297:32:42,  5.47s/it]
 13%|█▎        | 29302/225000 [67:04:54<277:02:21,  5.10s/it]
 13%|█▎        | 29303/225000 [67:05:07<397:00:24,  7.30s/it]
 13%|█▎        | 29304/225000 [67:05:12<364:45:34,  6.71s/it]
 13%|█▎        | 29305/225000 [67:05:20<378:08:58,  6.96s/it]
 13%|█▎        | 29306/225000 [67:05:24<342:08:20,  6.29s/it]
 13%|█▎        | 29307/225000 [67:05:29<320:36:05,  5.90s/it]
 13%|█▎        | 29308/225000 [67:05:35<311:37:27,  5.73s/it]
 13%|█▎        | 29309/225000 [67:05:40<296:32:34,  5.46s/it]
 13%|█▎        | 29310/225000 [67:05:46<305:10:02,  5.61s/it]
 13%|█▎        | 29311/225000 [67:05:59<428:51:55,  7.89s/it]
 13%|█▎        | 29312/225000 [67:06:03<371:44:08,  6.84s/it]
 13%|█▎        | 29313/225000 [67:06:08<332:45:37,  6.12s/it]
 13%|█▎        | 29314/225000 [67:06:18<396:07:54,  7.29s/it]
 13%|█▎        | 29315/225000 [67:06:21<336:59:43,  6.20s/it]
 13%|█▎        | 29316/225000 [67:06:30<373:41:48,  6.87s/it]
 13%|█▎        | 29317/225000 [67:06:34<330:46:20,  6.09s/it]
 13%|█▎        | 29318/225000 [67:06:42<359:37:13,  6.62s/it]
 13%|█▎        | 29319/225000 [67:06:49<361:35:28,  6.65s/it]
 13%|█▎        | 29320/225000 [67:06:53<327:21:12,  6.02s/it]
 13%|█▎        | 29321/225000 [67:06:59<326:53:20,  6.01s/it]
 13%|█▎        | 29322/225000 [67:07:08<369:36:12,  6.80s/it]
 13%|█▎        | 29323/225000 [67:07:11<318:21:10,  5.86s/it]
 13%|█▎        | 29324/225000 [67:07:15<281:41:04,  5.18s/it]
 13%|█▎        | 29325/225000 [67:07:25<354:28:10,  6.52s/it]
 13%|█▎        | 29326/225000 [67:07:28<308:38:16,  5.68s/it]
 13%|█▎        | 29327/225000 [67:07:37<357:46:32,  6.58s/it]
 13%|█▎        | 29328/225000 [67:07:43<355:19:02,  6.54s/it]
 13%|█▎        | 29329/225000 [67:07:49<333:32:43,  6.14s/it]
 13%|█▎        | 29330/225000 [67:07:57<363:12:22,  6.68s/it]
 13%|█▎        | 29331/225000 [67:08:01<328:01:42,  6.04s/it]
 13%|█▎        | 29332/225000 [67:08:06<305:36:51,  5.62s/it]
 13%|█▎        | 29333/225000 [67:08:10<283:17:37,  5.21s/it]
 13%|█▎        | 29334/225000 [67:08:16<292:48:26,  5.39s/it]
 13%|█▎        | 29335/225000 [67:08:29<424:00:34,  7.80s/it]
 13%|█▎        | 29336/225000 [67:08:36<404:20:03,  7.44s/it]
 13%|█▎        | 29337/225000 [67:08:41<370:18:02,  6.81s/it]
 13%|█▎        | 29338/225000 [67:08:45<322:18:17,  5.93s/it]
 13%|█▎        | 29339/225000 [67:08:54<377:52:20,  6.95s/it]
 13%|█▎        | 29340/225000 [67:08:59<334:09:38,  6.15s/it]
 13%|█▎        | 29341/225000 [67:09:03<305:57:34,  5.63s/it]
 13%|█▎        | 29342/225000 [67:09:09<308:21:39,  5.67s/it]
 13%|█▎        | 29343/225000 [67:09:15<307:25:18,  5.66s/it]
 13%|█▎        | 29344/225000 [67:09:20<308:33:57,  5.68s/it]
 13%|█▎        | 29345/225000 [67:09:26<311:24:39,  5.73s/it]
 13%|█▎        | 29346/225000 [67:09:32<312:29:12,  5.75s/it]
 13%|█▎        | 29347/225000 [67:09:37<304:33:00,  5.60s/it]
 13%|█▎        | 29348/225000 [67:09:50<420:26:18,  7.74s/it]
 13%|█▎        | 29349/225000 [67:09:58<427:09:18,  7.86s/it]
 13%|█▎        | 29350/225000 [67:10:04<393:39:21,  7.24s/it]
                                                             
{'loss': '0.7118', 'grad_norm': '0.7083', 'learning_rate': '4.901e-05', 'epoch': '0.1304', 'num_input_tokens_seen': 115111464, 'train_runtime': '2.418e+05', 'train_tokens_per_second': '476.1'}

 13%|█▎        | 29350/225000 [67:10:04<393:39:21,  7.24s/it]
 13%|█▎        | 29351/225000 [67:10:11<390:59:00,  7.19s/it]
 13%|█▎        | 29352/225000 [67:10:15<332:34:25,  6.12s/it]
 13%|█▎        | 29353/225000 [67:10:24<380:05:19,  6.99s/it]
 13%|█▎        | 29354/225000 [67:10:28<340:39:55,  6.27s/it]
 13%|█▎        | 29355/225000 [67:10:33<310:50:26,  5.72s/it]
 13%|█▎        | 29356/225000 [67:10:43<386:07:27,  7.10s/it]
 13%|█▎        | 29357/225000 [67:10:48<351:27:37,  6.47s/it]
 13%|█▎        | 29358/225000 [67:10:53<327:50:30,  6.03s/it]
 13%|█▎        | 29359/225000 [67:10:57<301:31:30,  5.55s/it]
 13%|█▎        | 29360/225000 [67:11:02<294:44:37,  5.42s/it]
 13%|█▎        | 29361/225000 [67:11:11<343:34:21,  6.32s/it]
 13%|█▎        | 29362/225000 [67:11:22<421:27:41,  7.76s/it]
 13%|█▎        | 29363/225000 [67:11:27<383:00:25,  7.05s/it]
 13%|█▎        | 29364/225000 [67:11:34<373:27:27,  6.87s/it]
 13%|█▎        | 29365/225000 [67:11:39<345:53:33,  6.36s/it]
 13%|█▎        | 29366/225000 [67:11:44<327:51:00,  6.03s/it]
 13%|█▎        | 29367/225000 [67:11:50<316:47:36,  5.83s/it]
 13%|█▎        | 29368/225000 [67:11:54<300:32:04,  5.53s/it]
 13%|█▎        | 29369/225000 [67:12:00<296:34:16,  5.46s/it]
 13%|█▎        | 29370/225000 [67:12:05<298:42:35,  5.50s/it]
 13%|█▎        | 29371/225000 [67:12:11<298:06:25,  5.49s/it]
 13%|█▎        | 29372/225000 [67:12:22<397:21:33,  7.31s/it]
 13%|█▎        | 29373/225000 [67:12:30<394:50:48,  7.27s/it]
 13%|█▎        | 29374/225000 [67:12:33<338:31:40,  6.23s/it]
 13%|█▎        | 29375/225000 [67:12:42<374:55:29,  6.90s/it]
 13%|█▎        | 29376/225000 [67:12:49<378:09:20,  6.96s/it]
 13%|█▎        | 29377/225000 [67:12:53<337:40:23,  6.21s/it]
 13%|█▎        | 29378/225000 [67:12:58<319:09:26,  5.87s/it]
 13%|█▎        | 29379/225000 [67:13:04<308:20:49,  5.67s/it]
 13%|█▎        | 29380/225000 [67:13:09<294:41:41,  5.42s/it]
 13%|█▎        | 29381/225000 [67:13:21<407:09:13,  7.49s/it]
 13%|█▎        | 29382/225000 [67:13:26<371:36:10,  6.84s/it]
 13%|█▎        | 29383/225000 [67:13:31<339:15:58,  6.24s/it]
 13%|█▎        | 29384/225000 [67:13:37<331:16:07,  6.10s/it]
 13%|█▎        | 29385/225000 [67:13:42<315:55:35,  5.81s/it]
 13%|█▎        | 29386/225000 [67:13:47<310:22:37,  5.71s/it]
 13%|█▎        | 29387/225000 [67:13:53<300:24:23,  5.53s/it]
 13%|█▎        | 29388/225000 [67:14:05<419:54:07,  7.73s/it]
 13%|█▎        | 29389/225000 [67:14:11<387:14:30,  7.13s/it]
 13%|█▎        | 29390/225000 [67:14:19<393:08:45,  7.24s/it]
 13%|█▎        | 29391/225000 [67:14:23<351:01:52,  6.46s/it]
 13%|█▎        | 29392/225000 [67:14:27<313:13:21,  5.76s/it]
 13%|█▎        | 29393/225000 [67:14:38<398:05:10,  7.33s/it]
 13%|█▎        | 29394/225000 [67:14:43<347:54:50,  6.40s/it]
 13%|█▎        | 29395/225000 [67:14:46<306:04:20,  5.63s/it]
 13%|█▎        | 29396/225000 [67:14:55<356:18:42,  6.56s/it]
 13%|█▎        | 29397/225000 [67:15:00<328:45:14,  6.05s/it]
 13%|█▎        | 29398/225000 [67:15:05<315:29:53,  5.81s/it]
 13%|█▎        | 29399/225000 [67:15:10<304:50:26,  5.61s/it]
 13%|█▎        | 29400/225000 [67:15:17<313:06:41,  5.76s/it]
                                                             
{'loss': '0.7383', 'grad_norm': '0.7306', 'learning_rate': '4.9e-05', 'epoch': '0.1307', 'num_input_tokens_seen': 115255576, 'train_runtime': '2.421e+05', 'train_tokens_per_second': '476'}

 13%|█▎        | 29400/225000 [67:15:17<313:06:41,  5.76s/it]
 13%|█▎        | 29401/225000 [67:15:27<397:53:08,  7.32s/it]
 13%|█▎        | 29402/225000 [67:15:37<438:10:32,  8.06s/it]
 13%|█▎        | 29403/225000 [67:15:42<375:58:12,  6.92s/it]
 13%|█▎        | 29404/225000 [67:15:46<334:25:51,  6.16s/it]
 13%|█▎        | 29405/225000 [67:15:50<304:09:45,  5.60s/it]
 13%|█▎        | 29406/225000 [67:16:01<386:38:10,  7.12s/it]
 13%|█▎        | 29407/225000 [67:16:07<376:50:17,  6.94s/it]
 13%|█▎        | 29408/225000 [67:16:12<332:53:05,  6.13s/it]
 13%|█▎        | 29409/225000 [67:16:16<299:43:15,  5.52s/it]
 13%|█▎        | 29410/225000 [67:16:27<392:25:57,  7.22s/it]
 13%|█▎        | 29411/225000 [67:16:33<371:33:52,  6.84s/it]
 13%|█▎        | 29412/225000 [67:16:40<380:52:14,  7.01s/it]
 13%|█▎        | 29413/225000 [67:16:44<333:52:07,  6.15s/it]
 13%|█▎        | 29414/225000 [67:16:53<368:27:41,  6.78s/it]
 13%|█▎        | 29415/225000 [67:16:56<315:09:31,  5.80s/it]
 13%|█▎        | 29416/225000 [67:17:05<360:58:42,  6.64s/it]
 13%|█▎        | 29417/225000 [67:17:10<339:55:14,  6.26s/it]
 13%|█▎        | 29418/225000 [67:17:17<357:33:12,  6.58s/it]
 13%|█▎        | 29419/225000 [67:17:22<325:55:19,  6.00s/it]
 13%|█▎        | 29420/225000 [67:17:26<298:42:16,  5.50s/it]
 13%|█▎        | 29421/225000 [67:17:31<283:21:27,  5.22s/it]
 13%|█▎        | 29422/225000 [67:17:39<322:53:42,  5.94s/it]
 13%|█▎        | 29423/225000 [67:17:50<404:48:46,  7.45s/it]
 13%|█▎        | 29424/225000 [67:17:53<343:13:24,  6.32s/it]
 13%|█▎        | 29425/225000 [67:18:00<353:09:20,  6.50s/it]
 13%|█▎        | 29426/225000 [67:18:04<311:34:30,  5.74s/it]
 13%|█▎        | 29427/225000 [67:18:15<392:03:27,  7.22s/it]
 13%|█▎        | 29428/225000 [67:18:19<340:34:32,  6.27s/it]
 13%|█▎        | 29429/225000 [67:18:29<402:16:39,  7.40s/it]
 13%|█▎        | 29430/225000 [67:18:33<344:08:06,  6.33s/it]
 13%|█▎        | 29431/225000 [67:18:37<309:34:25,  5.70s/it]
 13%|█▎        | 29432/225000 [67:18:48<394:59:14,  7.27s/it]
 13%|█▎        | 29433/225000 [67:18:53<364:27:57,  6.71s/it]
 13%|█▎        | 29434/225000 [67:18:58<332:19:59,  6.12s/it]
 13%|█▎        | 29435/225000 [67:19:03<313:25:32,  5.77s/it]
 13%|█▎        | 29436/225000 [67:19:08<295:00:01,  5.43s/it]
 13%|█▎        | 29437/225000 [67:19:20<402:10:42,  7.40s/it]
 13%|█▎        | 29438/225000 [67:19:25<369:30:33,  6.80s/it]
 13%|█▎        | 29439/225000 [67:19:30<341:58:11,  6.30s/it]
 13%|█▎        | 29440/225000 [67:19:35<321:40:51,  5.92s/it]
 13%|█▎        | 29441/225000 [67:19:40<302:55:16,  5.58s/it]
 13%|█▎        | 29442/225000 [67:19:45<292:04:11,  5.38s/it]
 13%|█▎        | 29443/225000 [67:19:50<291:00:29,  5.36s/it]
 13%|█▎        | 29444/225000 [67:20:02<393:37:10,  7.25s/it]
 13%|█▎        | 29445/225000 [67:20:12<432:56:56,  7.97s/it]
 13%|█▎        | 29446/225000 [67:20:15<364:06:37,  6.70s/it]
 13%|█▎        | 29447/225000 [67:20:20<327:26:22,  6.03s/it]
 13%|█▎        | 29448/225000 [67:20:30<389:01:21,  7.16s/it]
 13%|█▎        | 29449/225000 [67:20:34<345:55:29,  6.37s/it]
 13%|█▎        | 29450/225000 [67:20:43<381:13:18,  7.02s/it]
                                                             
{'loss': '0.7254', 'grad_norm': '0.6849', 'learning_rate': '4.9e-05', 'epoch': '0.1309', 'num_input_tokens_seen': 115399136, 'train_runtime': '2.424e+05', 'train_tokens_per_second': '476'}

 13%|█▎        | 29450/225000 [67:20:43<381:13:18,  7.02s/it]
 13%|█▎        | 29451/225000 [67:20:48<359:39:27,  6.62s/it]
 13%|█▎        | 29452/225000 [67:20:53<330:38:56,  6.09s/it]
 13%|█▎        | 29453/225000 [67:20:59<323:14:05,  5.95s/it]
 13%|█▎        | 29454/225000 [67:21:05<324:13:11,  5.97s/it]
 13%|█▎        | 29455/225000 [67:21:10<308:27:27,  5.68s/it]
 13%|█▎        | 29456/225000 [67:21:15<298:21:40,  5.49s/it]
 13%|█▎        | 29457/225000 [67:21:22<326:00:41,  6.00s/it]
 13%|█▎        | 29458/225000 [67:21:35<445:09:47,  8.20s/it]
 13%|█▎        | 29459/225000 [67:21:39<375:31:55,  6.91s/it]
 13%|█▎        | 29460/225000 [67:21:43<318:38:35,  5.87s/it]
 13%|█▎        | 29461/225000 [67:21:50<345:17:38,  6.36s/it]
 13%|█▎        | 29462/225000 [67:21:54<306:13:42,  5.64s/it]
 13%|█▎        | 29463/225000 [67:22:02<339:11:00,  6.24s/it]
 13%|█▎        | 29464/225000 [67:22:11<389:29:06,  7.17s/it]
 13%|█▎        | 29465/225000 [67:22:16<358:35:13,  6.60s/it]
 13%|█▎        | 29466/225000 [67:22:22<337:20:31,  6.21s/it]
 13%|█▎        | 29467/225000 [67:22:27<327:53:57,  6.04s/it]
 13%|█▎        | 29468/225000 [67:22:33<325:25:12,  5.99s/it]
 13%|█▎        | 29469/225000 [67:22:39<315:47:22,  5.81s/it]
 13%|█▎        | 29470/225000 [67:22:49<386:28:24,  7.12s/it]
 13%|█▎        | 29471/225000 [67:22:54<349:02:32,  6.43s/it]
 13%|█▎        | 29472/225000 [67:22:58<318:23:46,  5.86s/it]
 13%|█▎        | 29473/225000 [67:23:02<293:03:54,  5.40s/it]
 13%|█▎        | 29474/225000 [67:23:16<418:06:20,  7.70s/it]
 13%|█▎        | 29475/225000 [67:23:21<374:20:18,  6.89s/it]
 13%|█▎        | 29476/225000 [67:23:30<409:07:26,  7.53s/it]
 13%|█▎        | 29477/225000 [67:23:34<353:49:08,  6.51s/it]
 13%|█▎        | 29478/225000 [67:23:40<356:39:48,  6.57s/it]
 13%|█▎        | 29479/225000 [67:23:46<347:54:58,  6.41s/it]
 13%|█▎        | 29480/225000 [67:23:55<386:10:43,  7.11s/it]
 13%|█▎        | 29481/225000 [67:23:59<335:48:45,  6.18s/it]
 13%|█▎        | 29482/225000 [67:24:03<300:19:00,  5.53s/it]
 13%|█▎        | 29483/225000 [67:24:08<285:21:40,  5.25s/it]
 13%|█▎        | 29484/225000 [67:24:18<373:40:04,  6.88s/it]
 13%|█▎        | 29485/225000 [67:24:22<325:47:47,  6.00s/it]
 13%|█▎        | 29486/225000 [67:24:33<394:21:24,  7.26s/it]
 13%|█▎        | 29487/225000 [67:24:37<342:30:16,  6.31s/it]
 13%|█▎        | 29488/225000 [67:24:47<402:13:25,  7.41s/it]
 13%|█▎        | 29489/225000 [67:24:51<352:44:03,  6.49s/it]
 13%|█▎        | 29490/225000 [67:24:56<322:01:40,  5.93s/it]
 13%|█▎        | 29491/225000 [67:25:00<302:59:46,  5.58s/it]
 13%|█▎        | 29492/225000 [67:25:13<417:45:58,  7.69s/it]
 13%|█▎        | 29493/225000 [67:25:17<353:10:17,  6.50s/it]
 13%|█▎        | 29494/225000 [67:25:24<368:33:03,  6.79s/it]
 13%|█▎        | 29495/225000 [67:25:32<389:31:47,  7.17s/it]
 13%|█▎        | 29496/225000 [67:25:36<340:16:10,  6.27s/it]
 13%|█▎        | 29497/225000 [67:25:43<352:42:51,  6.49s/it]
 13%|█▎        | 29498/225000 [67:25:49<334:57:10,  6.17s/it]
 13%|█▎        | 29499/225000 [67:25:58<382:24:01,  7.04s/it]
 13%|█▎        | 29500/225000 [67:26:06<396:22:02,  7.30s/it]
                                                             
{'loss': '0.7219', 'grad_norm': '0.7403', 'learning_rate': '4.899e-05', 'epoch': '0.1311', 'num_input_tokens_seen': 115543152, 'train_runtime': '2.428e+05', 'train_tokens_per_second': '475.9'}

 13%|█▎        | 29500/225000 [67:26:06<396:22:02,  7.30s/it]
 13%|█▎        | 29501/225000 [67:26:10<345:46:34,  6.37s/it]
 13%|█▎        | 29502/225000 [67:26:17<350:17:14,  6.45s/it]
 13%|█▎        | 29503/225000 [67:26:22<323:20:28,  5.95s/it]
 13%|█▎        | 29504/225000 [67:26:28<333:41:48,  6.14s/it]
 13%|█▎        | 29505/225000 [67:26:34<329:53:59,  6.08s/it]
 13%|█▎        | 29506/225000 [67:26:41<343:24:46,  6.32s/it]
 13%|█▎        | 29507/225000 [67:26:47<338:36:37,  6.24s/it]
 13%|█▎        | 29508/225000 [67:26:56<380:01:12,  7.00s/it]
 13%|█▎        | 29509/225000 [67:27:02<375:29:27,  6.91s/it]
 13%|█▎        | 29510/225000 [67:27:07<336:10:53,  6.19s/it]
 13%|█▎        | 29511/225000 [67:27:12<322:59:54,  5.95s/it]
 13%|█▎        | 29512/225000 [67:27:18<315:07:52,  5.80s/it]
 13%|█▎        | 29513/225000 [67:27:23<310:54:15,  5.73s/it]
 13%|█▎        | 29514/225000 [67:27:28<299:21:11,  5.51s/it]
 13%|█▎        | 29515/225000 [67:27:34<294:58:48,  5.43s/it]
 13%|█▎        | 29516/225000 [67:27:46<400:48:38,  7.38s/it]
 13%|█▎        | 29517/225000 [67:27:55<431:01:34,  7.94s/it]
 13%|█▎        | 29518/225000 [67:28:00<382:27:29,  7.04s/it]
 13%|█▎        | 29519/225000 [67:28:04<340:56:18,  6.28s/it]
 13%|█▎        | 29520/225000 [67:28:09<320:20:40,  5.90s/it]
 13%|█▎        | 29521/225000 [67:28:19<388:53:05,  7.16s/it]
 13%|█▎        | 29522/225000 [67:28:26<372:51:56,  6.87s/it]
 13%|█▎        | 29523/225000 [67:28:30<328:21:40,  6.05s/it]
 13%|█▎        | 29524/225000 [67:28:34<300:37:15,  5.54s/it]
 13%|█▎        | 29525/225000 [67:28:46<412:46:47,  7.60s/it]
 13%|█▎        | 29526/225000 [67:28:51<371:40:28,  6.85s/it]
 13%|█▎        | 29527/225000 [67:28:56<341:26:31,  6.29s/it]
 13%|█▎        | 29528/225000 [67:29:02<332:15:07,  6.12s/it]
 13%|█▎        | 29529/225000 [67:29:08<323:37:37,  5.96s/it]
 13%|█▎        | 29530/225000 [67:29:13<311:59:20,  5.75s/it]
 13%|█▎        | 29531/225000 [67:29:19<307:52:12,  5.67s/it]
 13%|█▎        | 29532/225000 [67:29:24<299:35:56,  5.52s/it]
 13%|█▎        | 29533/225000 [67:29:29<297:44:51,  5.48s/it]
 13%|█▎        | 29534/225000 [67:29:42<418:53:40,  7.72s/it]
 13%|█▎        | 29535/225000 [67:29:49<408:30:06,  7.52s/it]
 13%|█▎        | 29536/225000 [67:29:57<418:07:16,  7.70s/it]
 13%|█▎        | 29537/225000 [67:30:04<406:46:29,  7.49s/it]
 13%|█▎        | 29538/225000 [67:30:10<374:36:59,  6.90s/it]
 13%|█▎        | 29539/225000 [67:30:17<383:17:34,  7.06s/it]
 13%|█▎        | 29540/225000 [67:30:21<337:49:22,  6.22s/it]
 13%|█▎        | 29541/225000 [67:30:27<320:31:31,  5.90s/it]
 13%|█▎        | 29542/225000 [67:30:31<298:23:46,  5.50s/it]
 13%|█▎        | 29543/225000 [67:30:38<315:23:52,  5.81s/it]
 13%|█▎        | 29544/225000 [67:30:49<411:37:16,  7.58s/it]
 13%|█▎        | 29545/225000 [67:30:54<357:13:45,  6.58s/it]
 13%|█▎        | 29546/225000 [67:30:58<325:36:57,  6.00s/it]
 13%|█▎        | 29547/225000 [67:31:05<339:47:48,  6.26s/it]
 13%|█▎        | 29548/225000 [67:31:15<404:37:17,  7.45s/it]
 13%|█▎        | 29549/225000 [67:31:19<341:47:43,  6.30s/it]
 13%|█▎        | 29550/225000 [67:31:25<335:32:26,  6.18s/it]
                                                             
{'loss': '0.7179', 'grad_norm': '0.662', 'learning_rate': '4.899e-05', 'epoch': '0.1313', 'num_input_tokens_seen': 115686832, 'train_runtime': '2.431e+05', 'train_tokens_per_second': '475.9'}

 13%|█▎        | 29550/225000 [67:31:25<335:32:26,  6.18s/it]
 13%|█▎        | 29551/225000 [67:31:34<387:59:00,  7.15s/it]
 13%|█▎        | 29552/225000 [67:31:39<352:21:57,  6.49s/it]
 13%|█▎        | 29553/225000 [67:31:44<327:22:33,  6.03s/it]
 13%|█▎        | 29554/225000 [67:31:49<313:38:52,  5.78s/it]
 13%|█▎        | 29555/225000 [67:31:55<303:11:52,  5.58s/it]
 13%|█▎        | 29556/225000 [67:32:00<306:19:21,  5.64s/it]
 13%|█▎        | 29557/225000 [67:32:06<310:34:12,  5.72s/it]
 13%|█▎        | 29558/225000 [67:32:18<401:36:06,  7.40s/it]
 13%|█▎        | 29559/225000 [67:32:23<374:32:22,  6.90s/it]
 13%|█▎        | 29560/225000 [67:32:28<333:11:00,  6.14s/it]
 13%|█▎        | 29561/225000 [67:32:33<321:21:49,  5.92s/it]
 13%|█▎        | 29562/225000 [67:32:38<298:51:23,  5.50s/it]
 13%|█▎        | 29563/225000 [67:32:44<311:32:34,  5.74s/it]
 13%|█▎        | 29564/225000 [67:32:57<434:08:38,  8.00s/it]
 13%|█▎        | 29565/225000 [67:33:02<387:25:30,  7.14s/it]
 13%|█▎        | 29566/225000 [67:33:08<364:23:29,  6.71s/it]
 13%|█▎        | 29567/225000 [67:33:13<344:34:05,  6.35s/it]
 13%|█▎        | 29568/225000 [67:33:19<337:21:06,  6.21s/it]
 13%|█▎        | 29569/225000 [67:33:25<327:33:08,  6.03s/it]
 13%|█▎        | 29570/225000 [67:33:31<331:55:47,  6.11s/it]
 13%|█▎        | 29571/225000 [67:33:44<440:32:06,  8.12s/it]
 13%|█▎        | 29572/225000 [67:33:50<404:44:47,  7.46s/it]
 13%|█▎        | 29573/225000 [67:33:58<409:21:12,  7.54s/it]
 13%|█▎        | 29574/225000 [67:34:03<376:25:39,  6.93s/it]
 13%|█▎        | 29575/225000 [67:34:09<349:57:44,  6.45s/it]
 13%|█▎        | 29576/225000 [67:34:13<325:20:40,  5.99s/it]
 13%|█▎        | 29577/225000 [67:34:18<306:03:45,  5.64s/it]
 13%|█▎        | 29578/225000 [67:34:25<330:33:17,  6.09s/it]
 13%|█▎        | 29579/225000 [67:34:35<393:07:26,  7.24s/it]
 13%|█▎        | 29580/225000 [67:34:39<336:04:09,  6.19s/it]
 13%|█▎        | 29581/225000 [67:34:43<292:10:49,  5.38s/it]
 13%|█▎        | 29582/225000 [67:34:54<393:43:49,  7.25s/it]
 13%|█▎        | 29583/225000 [67:35:01<381:17:35,  7.02s/it]
 13%|█▎        | 29584/225000 [67:35:06<359:52:57,  6.63s/it]
 13%|█▎        | 29585/225000 [67:35:11<329:47:40,  6.08s/it]
 13%|█▎        | 29586/225000 [67:35:18<346:15:34,  6.38s/it]
 13%|█▎        | 29587/225000 [67:35:28<395:40:53,  7.29s/it]
 13%|█▎        | 29588/225000 [67:35:33<360:24:01,  6.64s/it]
 13%|█▎        | 29589/225000 [67:35:38<329:39:37,  6.07s/it]
 13%|█▎        | 29590/225000 [67:35:43<311:31:53,  5.74s/it]
 13%|█▎        | 29591/225000 [67:35:48<302:57:51,  5.58s/it]
 13%|█▎        | 29592/225000 [67:35:53<298:32:21,  5.50s/it]
 13%|█▎        | 29593/225000 [67:36:01<338:08:24,  6.23s/it]
 13%|█▎        | 29594/225000 [67:36:12<420:34:18,  7.75s/it]
 13%|█▎        | 29595/225000 [67:36:17<368:52:41,  6.80s/it]
 13%|█▎        | 29596/225000 [67:36:21<326:43:57,  6.02s/it]
 13%|█▎        | 29597/225000 [67:36:32<405:43:44,  7.47s/it]
 13%|█▎        | 29598/225000 [67:36:38<375:16:11,  6.91s/it]
 13%|█▎        | 29599/225000 [67:36:41<326:37:39,  6.02s/it]
 13%|█▎        | 29600/225000 [67:36:51<384:48:49,  7.09s/it]
                                                             
{'loss': '0.7115', 'grad_norm': '0.6116', 'learning_rate': '4.898e-05', 'epoch': '0.1316', 'num_input_tokens_seen': 115831960, 'train_runtime': '2.434e+05', 'train_tokens_per_second': '475.9'}

 13%|█▎        | 29600/225000 [67:36:51<384:48:49,  7.09s/it]
 13%|█▎        | 29601/225000 [67:36:55<329:16:14,  6.07s/it]
 13%|█▎        | 29602/225000 [67:37:02<353:07:40,  6.51s/it]
 13%|█▎        | 29603/225000 [67:37:08<336:33:41,  6.20s/it]
 13%|█▎        | 29604/225000 [67:37:15<358:24:35,  6.60s/it]
 13%|█▎        | 29605/225000 [67:37:21<342:20:50,  6.31s/it]
 13%|█▎        | 29606/225000 [67:37:28<358:25:52,  6.60s/it]
 13%|█▎        | 29607/225000 [67:37:32<319:19:39,  5.88s/it]
 13%|█▎        | 29608/225000 [67:37:42<380:04:15,  7.00s/it]
 13%|█▎        | 29609/225000 [67:37:46<329:26:50,  6.07s/it]
 13%|█▎        | 29610/225000 [67:37:50<298:54:18,  5.51s/it]
 13%|█▎        | 29611/225000 [67:38:02<403:30:40,  7.43s/it]
 13%|█▎        | 29612/225000 [67:38:07<367:05:29,  6.76s/it]
 13%|█▎        | 29613/225000 [67:38:14<367:20:13,  6.77s/it]
 13%|█▎        | 29614/225000 [67:38:21<375:04:59,  6.91s/it]
 13%|█▎        | 29615/225000 [67:38:27<351:56:46,  6.48s/it]
 13%|█▎        | 29616/225000 [67:38:32<333:55:33,  6.15s/it]
 13%|█▎        | 29617/225000 [67:38:38<322:22:47,  5.94s/it]
 13%|█▎        | 29618/225000 [67:38:43<320:29:23,  5.91s/it]
 13%|█▎        | 29619/225000 [67:38:49<309:54:31,  5.71s/it]
 13%|█▎        | 29620/225000 [67:39:01<412:51:57,  7.61s/it]
 13%|█▎        | 29621/225000 [67:39:08<408:02:16,  7.52s/it]
 13%|█▎        | 29622/225000 [67:39:13<361:18:53,  6.66s/it]
 13%|█▎        | 29623/225000 [67:39:18<344:38:07,  6.35s/it]
 13%|█▎        | 29624/225000 [67:39:25<343:29:52,  6.33s/it]
 13%|█▎        | 29625/225000 [67:39:37<437:15:25,  8.06s/it]
 13%|█▎        | 29626/225000 [67:39:44<423:49:43,  7.81s/it]
 13%|█▎        | 29627/225000 [67:39:48<365:25:03,  6.73s/it]
 13%|█▎        | 29628/225000 [67:39:57<408:09:56,  7.52s/it]
 13%|█▎        | 29629/225000 [67:40:03<372:25:13,  6.86s/it]
 13%|█▎        | 29630/225000 [67:40:10<375:49:17,  6.93s/it]
 13%|█▎        | 29631/225000 [67:40:17<377:58:34,  6.96s/it]
 13%|█▎        | 29632/225000 [67:40:21<328:40:10,  6.06s/it]
 13%|█▎        | 29633/225000 [67:40:29<370:19:10,  6.82s/it]
 13%|█▎        | 29634/225000 [67:40:36<363:44:37,  6.70s/it]
 13%|█▎        | 29635/225000 [67:40:41<339:15:41,  6.25s/it]
 13%|█▎        | 29636/225000 [67:40:46<319:03:45,  5.88s/it]
 13%|█▎        | 29637/225000 [67:40:53<338:12:29,  6.23s/it]
 13%|█▎        | 29638/225000 [67:40:58<313:39:07,  5.78s/it]
 13%|█▎        | 29639/225000 [67:41:03<301:24:55,  5.55s/it]
 13%|█▎        | 29640/225000 [67:41:08<287:57:44,  5.31s/it]
 13%|█▎        | 29641/225000 [67:41:14<309:38:37,  5.71s/it]
 13%|█▎        | 29642/225000 [67:41:26<414:51:06,  7.64s/it]
 13%|█▎        | 29643/225000 [67:41:30<351:11:39,  6.47s/it]
 13%|█▎        | 29644/225000 [67:41:38<375:05:26,  6.91s/it]
 13%|█▎        | 29645/225000 [67:41:46<385:33:50,  7.11s/it]
 13%|█▎        | 29646/225000 [67:41:50<341:43:23,  6.30s/it]
 13%|█▎        | 29647/225000 [67:41:54<310:45:03,  5.73s/it]
 13%|█▎        | 29648/225000 [67:41:59<293:11:36,  5.40s/it]
 13%|█▎        | 29649/225000 [67:42:10<380:28:17,  7.01s/it]
 13%|█▎        | 29650/225000 [67:42:17<376:12:00,  6.93s/it]
                                                             
{'loss': '0.715', 'grad_norm': '0.6781', 'learning_rate': '4.898e-05', 'epoch': '0.1318', 'num_input_tokens_seen': 115976320, 'train_runtime': '2.437e+05', 'train_tokens_per_second': '475.8'}

 13%|█▎        | 29650/225000 [67:42:17<376:12:00,  6.93s/it]
 13%|█▎        | 29651/225000 [67:42:24<378:07:00,  6.97s/it]
 13%|█▎        | 29652/225000 [67:42:29<353:13:34,  6.51s/it]
 13%|█▎        | 29653/225000 [67:42:34<331:15:22,  6.10s/it]
 13%|█▎        | 29654/225000 [67:42:41<348:47:54,  6.43s/it]
 13%|█▎        | 29655/225000 [67:42:47<327:44:27,  6.04s/it]
 13%|█▎        | 29656/225000 [67:42:52<311:21:13,  5.74s/it]
 13%|█▎        | 29657/225000 [67:42:56<292:00:05,  5.38s/it]
 13%|█▎        | 29658/225000 [67:43:01<285:25:38,  5.26s/it]
 13%|█▎        | 29659/225000 [67:43:06<282:27:15,  5.21s/it]
 13%|█▎        | 29660/225000 [67:43:19<397:47:24,  7.33s/it]
 13%|█▎        | 29661/225000 [67:43:24<370:01:02,  6.82s/it]
 13%|█▎        | 29662/225000 [67:43:29<342:49:50,  6.32s/it]
 13%|█▎        | 29663/225000 [67:43:38<375:25:41,  6.92s/it]
 13%|█▎        | 29664/225000 [67:43:42<339:23:41,  6.25s/it]
 13%|█▎        | 29665/225000 [67:43:50<364:24:45,  6.72s/it]
 13%|█▎        | 29666/225000 [67:43:55<329:49:28,  6.08s/it]
 13%|█▎        | 29667/225000 [67:43:59<308:06:19,  5.68s/it]
 13%|█▎        | 29668/225000 [67:44:04<288:19:53,  5.31s/it]
 13%|█▎        | 29669/225000 [67:44:09<289:36:14,  5.34s/it]
 13%|█▎        | 29670/225000 [67:44:17<325:34:36,  6.00s/it]
 13%|█▎        | 29671/225000 [67:44:29<426:34:00,  7.86s/it]
 13%|█▎        | 29672/225000 [67:44:33<358:49:11,  6.61s/it]
 13%|█▎        | 29673/225000 [67:44:40<370:29:41,  6.83s/it]
 13%|█▎        | 29674/225000 [67:44:46<347:07:44,  6.40s/it]
 13%|█▎        | 29675/225000 [67:44:52<341:26:17,  6.29s/it]
 13%|█▎        | 29676/225000 [67:44:59<363:32:17,  6.70s/it]
 13%|█▎        | 29677/225000 [67:45:03<315:23:16,  5.81s/it]
 13%|█▎        | 29678/225000 [67:45:11<358:29:09,  6.61s/it]
 13%|█▎        | 29679/225000 [67:45:16<327:26:42,  6.04s/it]
 13%|█▎        | 29680/225000 [67:45:25<369:52:02,  6.82s/it]
 13%|█▎        | 29681/225000 [67:45:29<327:39:19,  6.04s/it]
 13%|█▎        | 29682/225000 [67:45:33<287:27:40,  5.30s/it]
 13%|█▎        | 29683/225000 [67:45:42<354:03:49,  6.53s/it]
 13%|█▎        | 29684/225000 [67:45:50<379:31:31,  7.00s/it]
 13%|█▎        | 29685/225000 [67:45:56<358:59:34,  6.62s/it]
 13%|█▎        | 29686/225000 [67:46:03<364:50:05,  6.72s/it]
 13%|█▎        | 29687/225000 [67:46:07<320:32:23,  5.91s/it]
 13%|█▎        | 29688/225000 [67:46:12<316:57:46,  5.84s/it]
 13%|█▎        | 29689/225000 [67:46:18<304:51:19,  5.62s/it]
 13%|█▎        | 29690/225000 [67:46:30<413:47:10,  7.63s/it]
 13%|█▎        | 29691/225000 [67:46:35<377:10:16,  6.95s/it]
 13%|█▎        | 29692/225000 [67:46:40<335:02:04,  6.18s/it]
 13%|█▎        | 29693/225000 [67:46:44<312:26:19,  5.76s/it]
 13%|█▎        | 29694/225000 [67:46:54<367:30:03,  6.77s/it]
 13%|█▎        | 29695/225000 [67:47:01<385:22:12,  7.10s/it]
 13%|█▎        | 29696/225000 [67:47:06<352:44:56,  6.50s/it]
 13%|█▎        | 29697/225000 [67:47:11<328:09:02,  6.05s/it]
 13%|█▎        | 29698/225000 [67:47:17<313:48:02,  5.78s/it]
 13%|█▎        | 29699/225000 [67:47:22<303:42:11,  5.60s/it]
 13%|█▎        | 29700/225000 [67:47:30<344:24:14,  6.35s/it]
                                                             
{'loss': '0.7259', 'grad_norm': '0.7019', 'learning_rate': '4.897e-05', 'epoch': '0.132', 'num_input_tokens_seen': 116117944, 'train_runtime': '2.441e+05', 'train_tokens_per_second': '475.8'}

 13%|█▎        | 29700/225000 [67:47:30<344:24:14,  6.35s/it]
 13%|█▎        | 29701/225000 [67:47:35<330:33:22,  6.09s/it]
 13%|█▎        | 29702/225000 [67:47:41<320:49:51,  5.91s/it]
 13%|█▎        | 29703/225000 [67:47:46<309:36:52,  5.71s/it]
 13%|█▎        | 29704/225000 [67:47:51<304:08:11,  5.61s/it]
 13%|█▎        | 29705/225000 [67:47:57<301:31:18,  5.56s/it]
 13%|█▎        | 29706/225000 [67:48:05<345:31:03,  6.37s/it]
 13%|█▎        | 29707/225000 [67:48:18<454:59:25,  8.39s/it]
 13%|█▎        | 29708/225000 [67:48:22<383:08:30,  7.06s/it]
 13%|█▎        | 29709/225000 [67:48:27<339:16:57,  6.25s/it]
 13%|█▎        | 29710/225000 [67:48:36<388:27:15,  7.16s/it]
 13%|█▎        | 29711/225000 [67:48:44<398:03:03,  7.34s/it]
 13%|█▎        | 29712/225000 [67:48:49<371:49:26,  6.85s/it]
 13%|█▎        | 29713/225000 [67:48:57<387:30:25,  7.14s/it]
 13%|█▎        | 29714/225000 [67:49:01<336:00:23,  6.19s/it]
 13%|█▎        | 29715/225000 [67:49:06<313:35:43,  5.78s/it]
 13%|█▎        | 29716/225000 [67:49:17<395:53:20,  7.30s/it]
 13%|█▎        | 29717/225000 [67:49:21<340:50:03,  6.28s/it]
 13%|█▎        | 29718/225000 [67:49:27<337:41:17,  6.23s/it]
 13%|█▎        | 29719/225000 [67:49:36<392:25:00,  7.23s/it]
 13%|█▎        | 29720/225000 [67:49:41<342:40:41,  6.32s/it]
 13%|█▎        | 29721/225000 [67:49:45<305:29:33,  5.63s/it]
 13%|█▎        | 29722/225000 [67:49:48<276:18:02,  5.09s/it]
 13%|█▎        | 29723/225000 [67:50:01<402:48:22,  7.43s/it]
 13%|█▎        | 29724/225000 [67:50:05<340:12:29,  6.27s/it]
 13%|█▎        | 29725/225000 [67:50:09<304:20:18,  5.61s/it]
 13%|█▎        | 29726/225000 [67:50:16<328:15:44,  6.05s/it]
 13%|█▎        | 29727/225000 [67:50:27<406:14:05,  7.49s/it]
 13%|█▎        | 29728/225000 [67:50:30<339:44:51,  6.26s/it]
 13%|█▎        | 29729/225000 [67:50:38<358:12:16,  6.60s/it]
 13%|█▎        | 29730/225000 [67:50:44<347:49:59,  6.41s/it]
 13%|█▎        | 29731/225000 [67:50:52<383:52:26,  7.08s/it]
 13%|█▎        | 29732/225000 [67:50:58<358:50:40,  6.62s/it]
 13%|█▎        | 29733/225000 [67:51:02<312:33:32,  5.76s/it]
 13%|█▎        | 29734/225000 [67:51:07<306:30:07,  5.65s/it]
 13%|█▎        | 29735/225000 [67:51:17<382:40:44,  7.06s/it]
 13%|█▎        | 29736/225000 [67:51:22<341:54:05,  6.30s/it]
 13%|█▎        | 29737/225000 [67:51:27<318:03:42,  5.86s/it]
 13%|█▎        | 29738/225000 [67:51:32<310:42:02,  5.73s/it]
 13%|█▎        | 29739/225000 [67:51:37<299:33:43,  5.52s/it]
 13%|█▎        | 29740/225000 [67:51:42<291:51:21,  5.38s/it]
 13%|█▎        | 29741/225000 [67:51:51<343:28:45,  6.33s/it]
 13%|█▎        | 29742/225000 [67:52:02<416:56:15,  7.69s/it]
 13%|█▎        | 29743/225000 [67:52:06<355:31:47,  6.55s/it]
 13%|█▎        | 29744/225000 [67:52:14<383:09:51,  7.06s/it]
 13%|█▎        | 29745/225000 [67:52:19<345:07:18,  6.36s/it]
 13%|█▎        | 29746/225000 [67:52:23<313:40:21,  5.78s/it]
 13%|█▎        | 29747/225000 [67:52:35<419:30:14,  7.73s/it]
 13%|█▎        | 29748/225000 [67:52:41<389:13:39,  7.18s/it]
 13%|█▎        | 29749/225000 [67:52:45<343:11:30,  6.33s/it]
 13%|█▎        | 29750/225000 [67:52:53<370:47:36,  6.84s/it]
                                                             
{'loss': '0.7509', 'grad_norm': '0.669', 'learning_rate': '4.897e-05', 'epoch': '0.1322', 'num_input_tokens_seen': 116260360, 'train_runtime': '2.444e+05', 'train_tokens_per_second': '475.7'}

 13%|█▎        | 29750/225000 [67:52:54<370:47:36,  6.84s/it]
 13%|█▎        | 29751/225000 [67:52:59<346:13:04,  6.38s/it]
 13%|█▎        | 29752/225000 [67:53:03<317:44:49,  5.86s/it]
 13%|█▎        | 29753/225000 [67:53:08<302:15:09,  5.57s/it]
 13%|█▎        | 29754/225000 [67:53:16<336:39:59,  6.21s/it]
 13%|█▎        | 29755/225000 [67:53:28<423:47:48,  7.81s/it]
 13%|█▎        | 29756/225000 [67:53:31<355:17:34,  6.55s/it]
 13%|█▎        | 29757/225000 [67:53:41<408:55:52,  7.54s/it]
 13%|█▎        | 29758/225000 [67:53:45<344:25:06,  6.35s/it]
 13%|█▎        | 29759/225000 [67:53:49<307:45:23,  5.67s/it]
 13%|█▎        | 29760/225000 [67:53:58<364:51:11,  6.73s/it]
 13%|█▎        | 29761/225000 [67:54:02<319:27:46,  5.89s/it]
 13%|█▎        | 29762/225000 [67:54:09<343:18:59,  6.33s/it]
 13%|█▎        | 29763/225000 [67:54:17<359:58:20,  6.64s/it]
 13%|█▎        | 29764/225000 [67:54:23<349:25:52,  6.44s/it]
 13%|█▎        | 29765/225000 [67:54:28<329:38:32,  6.08s/it]
 13%|█▎        | 29766/225000 [67:54:36<370:02:05,  6.82s/it]
 13%|█▎        | 29767/225000 [67:54:42<351:36:17,  6.48s/it]
 13%|█▎        | 29768/225000 [67:54:50<369:14:48,  6.81s/it]
 13%|█▎        | 29769/225000 [67:54:57<371:03:40,  6.84s/it]
 13%|█▎        | 29770/225000 [67:55:01<329:56:03,  6.08s/it]
 13%|█▎        | 29771/225000 [67:55:05<302:44:09,  5.58s/it]
 13%|█▎        | 29772/225000 [67:55:15<375:56:34,  6.93s/it]
 13%|█▎        | 29773/225000 [67:55:22<374:43:29,  6.91s/it]
 13%|█▎        | 29774/225000 [67:55:29<373:29:55,  6.89s/it]
 13%|█▎        | 29775/225000 [67:55:34<338:12:45,  6.24s/it]
 13%|█▎        | 29776/225000 [67:55:38<299:49:36,  5.53s/it]
 13%|█▎        | 29777/225000 [67:55:42<281:57:03,  5.20s/it]
 13%|█▎        | 29778/225000 [67:55:47<273:38:21,  5.05s/it]
 13%|█▎        | 29779/225000 [67:55:52<281:38:46,  5.19s/it]
 13%|█▎        | 29780/225000 [67:55:59<304:12:31,  5.61s/it]
 13%|█▎        | 29781/225000 [67:56:11<416:50:21,  7.69s/it]
 13%|█▎        | 29782/225000 [67:56:15<351:01:27,  6.47s/it]
 13%|█▎        | 29783/225000 [67:56:24<383:36:00,  7.07s/it]
 13%|█▎        | 29784/225000 [67:56:31<391:05:04,  7.21s/it]
 13%|█▎        | 29785/225000 [67:56:36<351:33:33,  6.48s/it]
 13%|█▎        | 29786/225000 [67:56:40<321:02:19,  5.92s/it]
 13%|█▎        | 29787/225000 [67:56:45<305:03:53,  5.63s/it]
 13%|█▎        | 29788/225000 [67:56:50<291:38:55,  5.38s/it]
 13%|█▎        | 29789/225000 [67:56:56<291:10:14,  5.37s/it]
 13%|█▎        | 29790/225000 [67:57:09<418:57:53,  7.73s/it]
 13%|█▎        | 29791/225000 [67:57:16<408:04:28,  7.53s/it]
 13%|█▎        | 29792/225000 [67:57:21<370:13:38,  6.83s/it]
 13%|█▎        | 29793/225000 [67:57:28<377:21:01,  6.96s/it]
 13%|█▎        | 29794/225000 [67:57:36<396:19:38,  7.31s/it]
 13%|█▎        | 29795/225000 [67:57:40<332:47:01,  6.14s/it]
 13%|█▎        | 29796/225000 [67:57:49<380:51:42,  7.02s/it]
 13%|█▎        | 29797/225000 [67:57:53<326:48:57,  6.03s/it]
 13%|█▎        | 29798/225000 [67:58:03<399:49:45,  7.37s/it]
 13%|█▎        | 29799/225000 [67:58:07<336:17:33,  6.20s/it]
 13%|█▎        | 29800/225000 [67:58:11<304:35:36,  5.62s/it]
                                                             
{'loss': '0.7262', 'grad_norm': '0.7579', 'learning_rate': '4.896e-05', 'epoch': '0.1324', 'num_input_tokens_seen': 116404640, 'train_runtime': '2.447e+05', 'train_tokens_per_second': '475.7'}

 13%|█▎        | 29800/225000 [67:58:11<304:35:36,  5.62s/it]
 13%|█▎        | 29801/225000 [67:58:23<406:51:28,  7.50s/it]
 13%|█▎        | 29802/225000 [67:58:27<353:42:04,  6.52s/it]
 13%|█▎        | 29803/225000 [67:58:35<385:14:06,  7.10s/it]
 13%|█▎        | 29804/225000 [67:58:41<362:40:36,  6.69s/it]
 13%|█▎        | 29805/225000 [67:58:48<370:05:51,  6.83s/it]
 13%|█▎        | 29806/225000 [67:58:53<330:18:02,  6.09s/it]
 13%|█▎        | 29807/225000 [67:58:57<308:49:32,  5.70s/it]
 13%|█▎        | 29808/225000 [67:59:02<297:43:54,  5.49s/it]
 13%|█▎        | 29809/225000 [67:59:14<401:21:09,  7.40s/it]
 13%|█▎        | 29810/225000 [67:59:20<376:34:51,  6.95s/it]
 13%|█▎        | 29811/225000 [67:59:29<398:34:09,  7.35s/it]
 13%|█▎        | 29812/225000 [67:59:33<355:12:52,  6.55s/it]
 13%|█▎        | 29813/225000 [67:59:38<335:01:44,  6.18s/it]
 13%|█▎        | 29814/225000 [67:59:43<313:47:36,  5.79s/it]
 13%|█▎        | 29815/225000 [67:59:48<295:33:29,  5.45s/it]
 13%|█▎        | 29816/225000 [67:59:52<279:00:13,  5.15s/it]
 13%|█▎        | 29817/225000 [68:00:04<386:26:54,  7.13s/it]
 13%|█▎        | 29818/225000 [68:00:09<352:43:37,  6.51s/it]
 13%|█▎        | 29819/225000 [68:00:19<397:33:16,  7.33s/it]
 13%|█▎        | 29820/225000 [68:00:23<346:04:11,  6.38s/it]
 13%|█▎        | 29821/225000 [68:00:31<380:11:37,  7.01s/it]
 13%|█▎        | 29822/225000 [68:00:36<342:13:47,  6.31s/it]
 13%|█▎        | 29823/225000 [68:00:44<372:57:21,  6.88s/it]
 13%|█▎        | 29824/225000 [68:00:50<358:17:30,  6.61s/it]
 13%|█▎        | 29825/225000 [68:00:55<326:50:53,  6.03s/it]
 13%|█▎        | 29826/225000 [68:01:02<353:55:52,  6.53s/it]
 13%|█▎        | 29827/225000 [68:01:10<363:40:40,  6.71s/it]
 13%|█▎        | 29828/225000 [68:01:13<312:59:47,  5.77s/it]
 13%|█▎        | 29829/225000 [68:01:17<281:28:53,  5.19s/it]
 13%|█▎        | 29830/225000 [68:01:27<359:23:32,  6.63s/it]
 13%|█▎        | 29831/225000 [68:01:33<357:20:45,  6.59s/it]
 13%|█▎        | 29832/225000 [68:01:38<317:29:24,  5.86s/it]
 13%|█▎        | 29833/225000 [68:01:47<367:06:34,  6.77s/it]
 13%|█▎        | 29834/225000 [68:01:53<356:27:31,  6.58s/it]
 13%|█▎        | 29835/225000 [68:01:57<324:43:01,  5.99s/it]
 13%|█▎        | 29836/225000 [68:02:02<307:18:14,  5.67s/it]
 13%|█▎        | 29837/225000 [68:02:08<316:09:04,  5.83s/it]
 13%|█▎        | 29838/225000 [68:02:21<429:32:24,  7.92s/it]
 13%|█▎        | 29839/225000 [68:02:28<416:18:58,  7.68s/it]
 13%|█▎        | 29840/225000 [68:02:32<353:55:54,  6.53s/it]
 13%|█▎        | 29841/225000 [68:02:41<397:58:46,  7.34s/it]
 13%|█▎        | 29842/225000 [68:02:45<343:39:09,  6.34s/it]
 13%|█▎        | 29843/225000 [68:02:49<305:18:01,  5.63s/it]
 13%|█▎        | 29844/225000 [68:02:59<378:31:03,  6.98s/it]
 13%|█▎        | 29845/225000 [68:03:04<332:13:07,  6.13s/it]
 13%|█▎        | 29846/225000 [68:03:09<316:42:40,  5.84s/it]
 13%|█▎        | 29847/225000 [68:03:18<366:40:34,  6.76s/it]
 13%|█▎        | 29848/225000 [68:03:23<349:32:57,  6.45s/it]
 13%|█▎        | 29849/225000 [68:03:31<366:57:40,  6.77s/it]
 13%|█▎        | 29850/225000 [68:03:36<345:10:44,  6.37s/it]
                                                             
{'loss': '0.7173', 'grad_norm': '0.6368', 'learning_rate': '4.896e-05', 'epoch': '0.1327', 'num_input_tokens_seen': 116547720, 'train_runtime': '2.45e+05', 'train_tokens_per_second': '475.7'}

 13%|█▎        | 29850/225000 [68:03:36<345:10:44,  6.37s/it]
 13%|█▎        | 29851/225000 [68:03:42<331:30:01,  6.12s/it]
 13%|█▎        | 29852/225000 [68:03:49<341:28:39,  6.30s/it]
 13%|█▎        | 29853/225000 [68:03:53<315:17:46,  5.82s/it]
 13%|█▎        | 29854/225000 [68:03:58<295:05:30,  5.44s/it]
 13%|█▎        | 29855/225000 [68:04:07<349:41:20,  6.45s/it]
 13%|█▎        | 29856/225000 [68:04:15<379:15:29,  7.00s/it]
 13%|█▎        | 29857/225000 [68:04:21<356:19:08,  6.57s/it]
 13%|█▎        | 29858/225000 [68:04:25<318:06:03,  5.87s/it]
 13%|█▎        | 29859/225000 [68:04:30<312:01:44,  5.76s/it]
 13%|█▎        | 29860/225000 [68:04:36<305:01:33,  5.63s/it]
 13%|█▎        | 29861/225000 [68:04:40<292:53:41,  5.40s/it]
 13%|█▎        | 29862/225000 [68:04:52<397:08:50,  7.33s/it]
 13%|█▎        | 29863/225000 [68:05:01<412:54:26,  7.62s/it]
 13%|█▎        | 29864/225000 [68:05:08<401:39:21,  7.41s/it]
 13%|█▎        | 29865/225000 [68:05:12<347:41:36,  6.41s/it]
 13%|█▎        | 29866/225000 [68:05:16<322:27:52,  5.95s/it]
 13%|█▎        | 29867/225000 [68:05:21<303:29:42,  5.60s/it]
 13%|█▎        | 29868/225000 [68:05:27<300:14:50,  5.54s/it]
 13%|█▎        | 29869/225000 [68:05:40<421:44:47,  7.78s/it]
 13%|█▎        | 29870/225000 [68:05:44<361:20:49,  6.67s/it]
 13%|█▎        | 29871/225000 [68:05:51<370:46:40,  6.84s/it]
 13%|█▎        | 29872/225000 [68:05:58<380:30:18,  7.02s/it]
 13%|█▎        | 29873/225000 [68:06:02<328:01:38,  6.05s/it]
 13%|█▎        | 29874/225000 [68:06:11<373:09:17,  6.88s/it]
 13%|█▎        | 29875/225000 [68:06:17<363:41:31,  6.71s/it]
 13%|█▎        | 29876/225000 [68:06:25<383:58:48,  7.08s/it]
 13%|█▎        | 29877/225000 [68:06:31<357:22:43,  6.59s/it]
 13%|█▎        | 29878/225000 [68:06:36<332:06:46,  6.13s/it]
 13%|█▎        | 29879/225000 [68:06:41<325:18:03,  6.00s/it]
 13%|█▎        | 29880/225000 [68:06:47<311:44:45,  5.75s/it]
 13%|█▎        | 29881/225000 [68:06:52<301:02:14,  5.55s/it]
 13%|█▎        | 29882/225000 [68:06:57<301:41:07,  5.57s/it]
 13%|█▎        | 29883/225000 [68:07:02<294:11:02,  5.43s/it]
 13%|█▎        | 29884/225000 [68:07:07<284:42:35,  5.25s/it]
 13%|█▎        | 29885/225000 [68:07:12<279:25:32,  5.16s/it]
 13%|█▎        | 29886/225000 [68:07:25<405:10:32,  7.48s/it]
 13%|█▎        | 29887/225000 [68:07:32<400:36:35,  7.39s/it]
 13%|█▎        | 29888/225000 [68:07:38<372:11:54,  6.87s/it]
 13%|█▎        | 29889/225000 [68:07:43<337:20:04,  6.22s/it]
 13%|█▎        | 29890/225000 [68:07:48<325:09:44,  6.00s/it]
 13%|█▎        | 29891/225000 [68:07:54<316:57:06,  5.85s/it]
 13%|█▎        | 29892/225000 [68:08:03<367:46:12,  6.79s/it]
 13%|█▎        | 29893/225000 [68:08:14<441:46:20,  8.15s/it]
 13%|█▎        | 29894/225000 [68:08:18<378:13:41,  6.98s/it]
 13%|█▎        | 29895/225000 [68:08:22<328:05:50,  6.05s/it]
 13%|█▎        | 29896/225000 [68:08:31<373:25:05,  6.89s/it]
 13%|█▎        | 29897/225000 [68:08:36<346:26:41,  6.39s/it]
 13%|█▎        | 29898/225000 [68:08:44<366:16:57,  6.76s/it]
 13%|█▎        | 29899/225000 [68:08:50<353:55:26,  6.53s/it]
 13%|█▎        | 29900/225000 [68:08:56<349:36:42,  6.45s/it]
                                                             
{'loss': '0.7411', 'grad_norm': '0.7431', 'learning_rate': '4.895e-05', 'epoch': '0.1329', 'num_input_tokens_seen': 116690800, 'train_runtime': '2.453e+05', 'train_tokens_per_second': '475.6'}

 13%|█▎        | 29900/225000 [68:08:56<349:36:42,  6.45s/it]
 13%|█▎        | 29901/225000 [68:09:04<373:52:28,  6.90s/it]
 13%|█▎        | 29902/225000 [68:09:08<332:14:27,  6.13s/it]
 13%|█▎        | 29903/225000 [68:09:12<300:18:28,  5.54s/it]
 13%|█▎        | 29904/225000 [68:09:22<361:25:27,  6.67s/it]
 13%|█▎        | 29905/225000 [68:09:28<358:58:37,  6.62s/it]
 13%|█▎        | 29906/225000 [68:09:32<318:47:41,  5.88s/it]
 13%|█▎        | 29907/225000 [68:09:37<300:46:53,  5.55s/it]
 13%|█▎        | 29908/225000 [68:09:42<288:24:20,  5.32s/it]
 13%|█▎        | 29909/225000 [68:09:55<417:54:36,  7.71s/it]
 13%|█▎        | 29910/225000 [68:10:00<364:01:43,  6.72s/it]
 13%|█▎        | 29911/225000 [68:10:06<352:38:28,  6.51s/it]
 13%|█▎        | 29912/225000 [68:10:11<336:13:17,  6.20s/it]
 13%|█▎        | 29913/225000 [68:10:17<332:43:10,  6.14s/it]
 13%|█▎        | 29914/225000 [68:10:27<395:45:47,  7.30s/it]
 13%|█▎        | 29915/225000 [68:10:33<369:45:58,  6.82s/it]
 13%|█▎        | 29916/225000 [68:10:39<352:54:20,  6.51s/it]
 13%|█▎        | 29917/225000 [68:10:44<333:52:37,  6.16s/it]
 13%|█▎        | 29918/225000 [68:10:49<316:01:35,  5.83s/it]
 13%|█▎        | 29919/225000 [68:10:54<303:54:42,  5.61s/it]
 13%|█▎        | 29920/225000 [68:11:00<309:31:02,  5.71s/it]
 13%|█▎        | 29921/225000 [68:11:11<386:51:46,  7.14s/it]
 13%|█▎        | 29922/225000 [68:11:17<379:17:04,  7.00s/it]
 13%|█▎        | 29923/225000 [68:11:22<345:42:13,  6.38s/it]
 13%|█▎        | 29924/225000 [68:11:27<321:06:40,  5.93s/it]
 13%|█▎        | 29925/225000 [68:11:38<406:51:50,  7.51s/it]
 13%|█▎        | 29926/225000 [68:11:44<383:24:10,  7.08s/it]
 13%|█▎        | 29927/225000 [68:11:49<341:30:43,  6.30s/it]
 13%|█▎        | 29928/225000 [68:11:55<343:52:40,  6.35s/it]
 13%|█▎        | 29929/225000 [68:12:02<356:59:27,  6.59s/it]
 13%|█▎        | 29930/225000 [68:12:06<308:36:05,  5.70s/it]
 13%|█▎        | 29931/225000 [68:12:15<368:17:40,  6.80s/it]
 13%|█▎        | 29932/225000 [68:12:19<319:30:53,  5.90s/it]
 13%|█▎        | 29933/225000 [68:12:24<296:30:16,  5.47s/it]
 13%|█▎        | 29934/225000 [68:12:29<288:07:27,  5.32s/it]
 13%|█▎        | 29935/225000 [68:12:34<285:13:37,  5.26s/it]
 13%|█▎        | 29936/225000 [68:12:48<428:25:57,  7.91s/it]
 13%|█▎        | 29937/225000 [68:12:53<383:46:57,  7.08s/it]
 13%|█▎        | 29938/225000 [68:12:58<348:44:57,  6.44s/it]
 13%|█▎        | 29939/225000 [68:13:03<321:18:29,  5.93s/it]
 13%|█▎        | 29940/225000 [68:13:08<309:04:12,  5.70s/it]
 13%|█▎        | 29941/225000 [68:13:13<299:24:18,  5.53s/it]
 13%|█▎        | 29942/225000 [68:13:19<313:29:33,  5.79s/it]
 13%|█▎        | 29943/225000 [68:13:31<400:45:00,  7.40s/it]
 13%|█▎        | 29944/225000 [68:13:36<360:49:00,  6.66s/it]
 13%|█▎        | 29945/225000 [68:13:41<334:27:22,  6.17s/it]
 13%|█▎        | 29946/225000 [68:13:45<312:40:39,  5.77s/it]
 13%|█▎        | 29947/225000 [68:13:50<300:39:02,  5.55s/it]
 13%|█▎        | 29948/225000 [68:13:56<294:49:40,  5.44s/it]
 13%|█▎        | 29949/225000 [68:14:06<380:35:20,  7.02s/it]
 13%|█▎        | 29950/225000 [68:14:14<398:14:25,  7.35s/it]
                                                             
{'loss': '0.7288', 'grad_norm': '0.6602', 'learning_rate': '4.895e-05', 'epoch': '0.1331', 'num_input_tokens_seen': 116834424, 'train_runtime': '2.457e+05', 'train_tokens_per_second': '475.6'}

 13%|█▎        | 29950/225000 [68:14:14<398:14:25,  7.35s/it]
 13%|█▎        | 29951/225000 [68:14:21<383:56:06,  7.09s/it]
 13%|█▎        | 29952/225000 [68:14:27<375:44:52,  6.94s/it]
 13%|█▎        | 29953/225000 [68:14:34<373:16:40,  6.89s/it]
 13%|█▎        | 29954/225000 [68:14:41<372:24:04,  6.87s/it]
 13%|█▎        | 29955/225000 [68:14:46<344:09:17,  6.35s/it]
 13%|█▎        | 29956/225000 [68:14:54<361:05:19,  6.66s/it]
 13%|█▎        | 29957/225000 [68:14:58<327:35:26,  6.05s/it]
 13%|█▎        | 29958/225000 [68:15:03<301:20:15,  5.56s/it]
 13%|█▎        | 29959/225000 [68:15:08<292:27:43,  5.40s/it]
 13%|█▎        | 29960/225000 [68:15:20<409:19:27,  7.56s/it]
 13%|█▎        | 29961/225000 [68:15:24<351:34:25,  6.49s/it]
 13%|█▎        | 29962/225000 [68:15:29<315:53:04,  5.83s/it]
 13%|█▎        | 29963/225000 [68:15:33<299:59:10,  5.54s/it]
 13%|█▎        | 29964/225000 [68:15:44<385:01:29,  7.11s/it]
 13%|█▎        | 29965/225000 [68:15:50<358:49:59,  6.62s/it]
 13%|█▎        | 29966/225000 [68:15:55<335:33:33,  6.19s/it]
 13%|█▎        | 29967/225000 [68:16:00<317:35:54,  5.86s/it]
 13%|█▎        | 29968/225000 [68:16:06<317:07:29,  5.85s/it]
 13%|█▎        | 29969/225000 [68:16:11<302:00:10,  5.57s/it]
 13%|█▎        | 29970/225000 [68:16:17<306:55:47,  5.67s/it]
 13%|█▎        | 29971/225000 [68:16:26<364:29:31,  6.73s/it]
 13%|█▎        | 29972/225000 [68:16:36<417:38:03,  7.71s/it]
 13%|█▎        | 29973/225000 [68:16:40<368:21:05,  6.80s/it]
 13%|█▎        | 29974/225000 [68:16:45<333:16:42,  6.15s/it]
 13%|█▎        | 29975/225000 [68:16:50<306:45:21,  5.66s/it]
 13%|█▎        | 29976/225000 [68:16:55<297:17:20,  5.49s/it]
 13%|█▎        | 29977/225000 [68:16:59<279:38:27,  5.16s/it]
 13%|█▎        | 29978/225000 [68:17:04<270:21:49,  4.99s/it]
 13%|█▎        | 29979/225000 [68:17:16<382:07:52,  7.05s/it]
 13%|█▎        | 29980/225000 [68:17:22<371:32:33,  6.86s/it]
 13%|█▎        | 29981/225000 [68:17:27<336:06:57,  6.20s/it]
 13%|█▎        | 29982/225000 [68:17:35<364:46:37,  6.73s/it]
 13%|█▎        | 29983/225000 [68:17:39<325:36:49,  6.01s/it]
 13%|█▎        | 29984/225000 [68:17:48<380:24:04,  7.02s/it]
 13%|█▎        | 29985/225000 [68:17:53<346:14:18,  6.39s/it]
 13%|█▎        | 29986/225000 [68:17:58<318:16:00,  5.88s/it]
 13%|█▎        | 29987/225000 [68:18:03<301:37:42,  5.57s/it]
 13%|█▎        | 29988/225000 [68:18:08<292:12:15,  5.39s/it]
 13%|█▎        | 29989/225000 [68:18:17<359:05:28,  6.63s/it]
 13%|█▎        | 29990/225000 [68:18:26<389:05:12,  7.18s/it]
 13%|█▎        | 29991/225000 [68:18:29<331:10:57,  6.11s/it]
 13%|█▎        | 29992/225000 [68:18:39<388:43:25,  7.18s/it]
 13%|█▎        | 29993/225000 [68:18:44<355:04:47,  6.56s/it]
 13%|█▎        | 29994/225000 [68:18:49<333:50:20,  6.16s/it]
 13%|█▎        | 29995/225000 [68:18:55<316:38:55,  5.85s/it]
 13%|█▎        | 29996/225000 [68:18:59<302:12:34,  5.58s/it]
 13%|█▎        | 29997/225000 [68:19:13<436:51:51,  8.07s/it]
 13%|█▎        | 29998/225000 [68:19:18<378:55:10,  7.00s/it]
 13%|█▎        | 29999/225000 [68:19:22<340:34:44,  6.29s/it]
 13%|█▎        | 30000/225000 [68:19:32<396:07:12,  7.31s/it]
                                                             
{'loss': '0.7221', 'grad_norm': '0.668', 'learning_rate': '4.894e-05', 'epoch': '0.1333', 'num_input_tokens_seen': 116975728, 'train_runtime': '2.46e+05', 'train_tokens_per_second': '475.6'}

 13%|█▎        | 30000/225000 [68:19:32<396:07:12,  7.31s/it]
 13%|█▎        | 30001/225000 [68:19:37<353:03:51,  6.52s/it]
 13%|█▎        | 30002/225000 [68:19:41<317:56:08,  5.87s/it]
 13%|█▎        | 30003/225000 [68:19:45<289:14:39,  5.34s/it]
 13%|█▎        | 30004/225000 [68:19:58<413:14:17,  7.63s/it]
 13%|█▎        | 30005/225000 [68:20:02<352:23:43,  6.51s/it]
 13%|█▎        | 30006/225000 [68:20:09<361:22:47,  6.67s/it]
 13%|█▎        | 30007/225000 [68:20:15<348:46:46,  6.44s/it]
 13%|█▎        | 30008/225000 [68:20:22<349:18:16,  6.45s/it]
 13%|█▎        | 30009/225000 [68:20:25<304:47:49,  5.63s/it]
 13%|█▎        | 30010/225000 [68:20:31<302:35:59,  5.59s/it]
 13%|█▎        | 30011/225000 [68:20:41<382:15:46,  7.06s/it]
 13%|█▎        | 30012/225000 [68:20:47<357:24:47,  6.60s/it]
 13%|█▎        | 30013/225000 [68:20:53<344:54:58,  6.37s/it]
 13%|█▎        | 30014/225000 [68:20:57<319:02:19,  5.89s/it]
 13%|█▎        | 30015/225000 [68:21:03<307:29:00,  5.68s/it]
 13%|█▎        | 30016/225000 [68:21:08<301:55:15,  5.57s/it]
 13%|█▎        | 30017/225000 [68:21:13<286:55:02,  5.30s/it]
 13%|█▎        | 30018/225000 [68:21:25<395:30:31,  7.30s/it]
 13%|█▎        | 30019/225000 [68:21:31<376:55:51,  6.96s/it]
 13%|█▎        | 30020/225000 [68:21:36<353:10:12,  6.52s/it]
 13%|█▎        | 30021/225000 [68:21:42<346:29:28,  6.40s/it]
 13%|█▎        | 30022/225000 [68:21:46<304:35:36,  5.62s/it]
 13%|█▎        | 30023/225000 [68:21:53<317:26:00,  5.86s/it]
 13%|█▎        | 30024/225000 [68:22:00<339:05:58,  6.26s/it]
 13%|█▎        | 30025/225000 [68:22:03<288:33:41,  5.33s/it]
 13%|█▎        | 30026/225000 [68:22:13<363:19:23,  6.71s/it]
 13%|█▎        | 30027/225000 [68:22:16<310:29:58,  5.73s/it]
 13%|█▎        | 30028/225000 [68:22:24<346:31:12,  6.40s/it]
 13%|█▎        | 30029/225000 [68:22:29<314:27:17,  5.81s/it]
 13%|█▎        | 30030/225000 [68:22:34<304:54:14,  5.63s/it]
 13%|█▎        | 30031/225000 [68:22:39<292:30:31,  5.40s/it]
 13%|█▎        | 30032/225000 [68:22:44<283:21:33,  5.23s/it]
 13%|█▎        | 30033/225000 [68:22:56<399:24:31,  7.37s/it]
 13%|█▎        | 30034/225000 [68:23:01<367:28:13,  6.79s/it]
 13%|█▎        | 30035/225000 [68:23:05<318:35:23,  5.88s/it]
 13%|█▎        | 30036/225000 [68:23:16<391:58:57,  7.24s/it]
 13%|█▎        | 30037/225000 [68:23:19<331:56:05,  6.13s/it]
 13%|█▎        | 30038/225000 [68:23:28<372:30:15,  6.88s/it]
 13%|█▎        | 30039/225000 [68:23:35<374:58:49,  6.92s/it]
 13%|█▎        | 30040/225000 [68:23:39<329:10:53,  6.08s/it]
 13%|█▎        | 30041/225000 [68:23:43<294:05:32,  5.43s/it]
 13%|█▎        | 30042/225000 [68:23:54<383:34:56,  7.08s/it]
 13%|█▎        | 30043/225000 [68:23:58<339:14:32,  6.26s/it]
 13%|█▎        | 30044/225000 [68:24:02<305:34:10,  5.64s/it]
 13%|█▎        | 30045/225000 [68:24:12<374:41:53,  6.92s/it]
 13%|█▎        | 30046/225000 [68:24:16<325:12:35,  6.01s/it]
 13%|█▎        | 30047/225000 [68:24:27<403:55:52,  7.46s/it]
 13%|█▎        | 30048/225000 [68:24:33<379:33:10,  7.01s/it]
 13%|█▎        | 30049/225000 [68:24:38<346:42:55,  6.40s/it]
 13%|█▎        | 30050/225000 [68:24:43<328:57:39,  6.07s/it]
                                                             
{'loss': '0.7339', 'grad_norm': '0.5922', 'learning_rate': '4.894e-05', 'epoch': '0.1336', 'num_input_tokens_seen': 117117224, 'train_runtime': '2.463e+05', 'train_tokens_per_second': '475.5'}

 13%|█▎        | 30050/225000 [68:24:43<328:57:39,  6.07s/it]
 13%|█▎        | 30051/225000 [68:24:48<302:21:26,  5.58s/it]
 13%|█▎        | 30052/225000 [68:24:53<293:11:54,  5.41s/it]
 13%|█▎        | 30053/225000 [68:24:58<292:00:55,  5.39s/it]
 13%|█▎        | 30054/225000 [68:25:03<284:10:56,  5.25s/it]
 13%|█▎        | 30055/225000 [68:25:16<417:22:38,  7.71s/it]
 13%|█▎        | 30056/225000 [68:25:23<399:13:45,  7.37s/it]
 13%|█▎        | 30057/225000 [68:25:27<352:52:20,  6.52s/it]
 13%|█▎        | 30058/225000 [68:25:34<360:53:04,  6.66s/it]
 13%|█▎        | 30059/225000 [68:25:38<315:29:57,  5.83s/it]
 13%|█▎        | 30060/225000 [68:25:43<298:59:44,  5.52s/it]
 13%|█▎        | 30061/225000 [68:25:52<358:04:27,  6.61s/it]
 13%|█▎        | 30062/225000 [68:26:00<383:52:30,  7.09s/it]
 13%|█▎        | 30063/225000 [68:26:05<350:34:32,  6.47s/it]
 13%|█▎        | 30064/225000 [68:26:12<351:36:20,  6.49s/it]
 13%|█▎        | 30065/225000 [68:26:21<392:44:01,  7.25s/it]
 13%|█▎        | 30066/225000 [68:26:25<340:06:38,  6.28s/it]
 13%|█▎        | 30067/225000 [68:26:29<303:57:52,  5.61s/it]
 13%|█▎        | 30068/225000 [68:26:39<378:25:10,  6.99s/it]
 13%|█▎        | 30069/225000 [68:26:46<376:50:22,  6.96s/it]
 13%|█▎        | 30070/225000 [68:26:53<366:17:46,  6.76s/it]
 13%|█▎        | 30071/225000 [68:26:56<321:12:26,  5.93s/it]
 13%|█▎        | 30072/225000 [68:27:01<290:56:20,  5.37s/it]
 13%|█▎        | 30073/225000 [68:27:11<378:48:52,  7.00s/it]
 13%|█▎        | 30074/225000 [68:27:15<328:45:19,  6.07s/it]
 13%|█▎        | 30075/225000 [68:27:24<371:44:30,  6.87s/it]
 13%|█▎        | 30076/225000 [68:27:29<349:02:35,  6.45s/it]
 13%|█▎        | 30077/225000 [68:27:35<332:02:55,  6.13s/it]
 13%|█▎        | 30078/225000 [68:27:42<349:02:15,  6.45s/it]
 13%|█▎        | 30079/225000 [68:27:47<328:08:27,  6.06s/it]
 13%|█▎        | 30080/225000 [68:27:53<324:53:12,  6.00s/it]
 13%|█▎        | 30081/225000 [68:27:59<324:33:49,  5.99s/it]
 13%|█▎        | 30082/225000 [68:28:08<365:42:01,  6.75s/it]
 13%|█▎        | 30083/225000 [68:28:11<316:26:57,  5.84s/it]
 13%|█▎        | 30084/225000 [68:28:21<378:19:16,  6.99s/it]
 13%|█▎        | 30085/225000 [68:28:25<325:18:47,  6.01s/it]
 13%|█▎        | 30086/225000 [68:28:29<299:12:54,  5.53s/it]
 13%|█▎        | 30087/225000 [68:28:35<299:29:18,  5.53s/it]
 13%|█▎        | 30088/225000 [68:28:39<289:02:02,  5.34s/it]
 13%|█▎        | 30089/225000 [68:28:45<289:03:29,  5.34s/it]
 13%|█▎        | 30090/225000 [68:28:50<288:47:38,  5.33s/it]
 13%|█▎        | 30091/225000 [68:29:03<403:52:30,  7.46s/it]
 13%|█▎        | 30092/225000 [68:29:11<424:46:49,  7.85s/it]
 13%|█▎        | 30093/225000 [68:29:16<366:03:58,  6.76s/it]
 13%|█▎        | 30094/225000 [68:29:20<325:15:47,  6.01s/it]
 13%|█▎        | 30095/225000 [68:29:30<395:02:27,  7.30s/it]
 13%|█▎        | 30096/225000 [68:29:35<348:37:46,  6.44s/it]
 13%|█▎        | 30097/225000 [68:29:39<312:56:08,  5.78s/it]
 13%|█▎        | 30098/225000 [68:29:51<412:19:13,  7.62s/it]
 13%|█▎        | 30099/225000 [68:29:54<348:41:52,  6.44s/it]
 13%|█▎        | 30100/225000 [68:30:04<394:11:06,  7.28s/it]
                                                             
{'loss': '0.7125', 'grad_norm': '0.7241', 'learning_rate': '4.893e-05', 'epoch': '0.1338', 'num_input_tokens_seen': 117261504, 'train_runtime': '2.466e+05', 'train_tokens_per_second': '475.5'}

 13%|█▎        | 30100/225000 [68:30:04<394:11:06,  7.28s/it]
 13%|█▎        | 30101/225000 [68:30:09<368:24:28,  6.80s/it]
 13%|█▎        | 30102/225000 [68:30:14<338:50:11,  6.26s/it]
 13%|█▎        | 30103/225000 [68:30:23<379:39:55,  7.01s/it]
 13%|█▎        | 30104/225000 [68:30:27<334:49:29,  6.18s/it]
 13%|█▎        | 30105/225000 [68:30:32<306:36:04,  5.66s/it]
 13%|█▎        | 30106/225000 [68:30:37<299:41:55,  5.54s/it]
 13%|█▎        | 30107/225000 [68:30:49<406:41:56,  7.51s/it]
 13%|█▎        | 30108/225000 [68:30:53<349:33:15,  6.46s/it]
 13%|█▎        | 30109/225000 [68:31:01<376:13:49,  6.95s/it]
 13%|█▎        | 30110/225000 [68:31:09<381:36:32,  7.05s/it]
 13%|█▎        | 30111/225000 [68:31:14<359:36:06,  6.64s/it]
 13%|█▎        | 30112/225000 [68:31:18<312:44:54,  5.78s/it]
 13%|█▎        | 30113/225000 [68:31:26<348:43:50,  6.44s/it]
 13%|█▎        | 30114/225000 [68:31:32<348:13:40,  6.43s/it]
 13%|█▎        | 30115/225000 [68:31:37<312:19:45,  5.77s/it]
 13%|█▎        | 30116/225000 [68:31:42<302:09:45,  5.58s/it]
 13%|█▎        | 30117/225000 [68:31:47<304:57:47,  5.63s/it]
 13%|█▎        | 30118/225000 [68:31:53<303:43:02,  5.61s/it]
 13%|█▎        | 30119/225000 [68:31:58<296:28:58,  5.48s/it]
 13%|█▎        | 30120/225000 [68:32:04<309:27:32,  5.72s/it]
 13%|█▎        | 30121/225000 [68:32:17<417:34:18,  7.71s/it]
 13%|█▎        | 30122/225000 [68:32:21<367:20:19,  6.79s/it]
 13%|█▎        | 30123/225000 [68:32:26<331:35:01,  6.13s/it]
 13%|█▎        | 30124/225000 [68:32:33<338:08:57,  6.25s/it]
 13%|█▎        | 30125/225000 [68:32:44<416:12:50,  7.69s/it]
 13%|█▎        | 30126/225000 [68:32:47<346:54:31,  6.41s/it]
 13%|█▎        | 30127/225000 [68:32:56<395:15:26,  7.30s/it]
 13%|█▎        | 30128/225000 [68:33:02<362:29:36,  6.70s/it]
 13%|█▎        | 30129/225000 [68:33:07<341:40:09,  6.31s/it]
 13%|█▎        | 30130/225000 [68:33:15<368:48:56,  6.81s/it]
 13%|█▎        | 30131/225000 [68:33:19<320:20:40,  5.92s/it]
 13%|█▎        | 30132/225000 [68:33:29<392:29:15,  7.25s/it]
 13%|█▎        | 30133/225000 [68:33:33<337:28:51,  6.23s/it]
 13%|█▎        | 30134/225000 [68:33:42<375:28:15,  6.94s/it]
 13%|█▎        | 30135/225000 [68:33:46<325:09:23,  6.01s/it]
 13%|█▎        | 30136/225000 [68:33:53<350:30:02,  6.48s/it]
 13%|█▎        | 30137/225000 [68:33:57<310:38:37,  5.74s/it]
 13%|█▎        | 30138/225000 [68:34:02<296:40:59,  5.48s/it]
 13%|█▎        | 30139/225000 [68:34:12<369:12:46,  6.82s/it]
 13%|█▎        | 30140/225000 [68:34:19<374:08:02,  6.91s/it]
 13%|█▎        | 30141/225000 [68:34:26<375:03:00,  6.93s/it]
 13%|█▎        | 30142/225000 [68:34:30<328:59:10,  6.08s/it]
 13%|█▎        | 30143/225000 [68:34:40<385:48:08,  7.13s/it]
 13%|█▎        | 30144/225000 [68:34:44<342:36:16,  6.33s/it]
 13%|█▎        | 30145/225000 [68:34:48<308:37:59,  5.70s/it]
 13%|█▎        | 30146/225000 [68:34:53<283:18:23,  5.23s/it]
 13%|█▎        | 30147/225000 [68:35:06<408:57:31,  7.56s/it]
 13%|█▎        | 30148/225000 [68:35:10<349:39:48,  6.46s/it]
 13%|█▎        | 30149/225000 [68:35:17<370:26:16,  6.84s/it]
 13%|█▎        | 30150/225000 [68:35:21<325:48:35,  6.02s/it]
                                                             
{'loss': '0.7003', 'grad_norm': '0.6637', 'learning_rate': '4.893e-05', 'epoch': '0.134', 'num_input_tokens_seen': 117405152, 'train_runtime': '2.469e+05', 'train_tokens_per_second': '475.5'}

 13%|█▎        | 30150/225000 [68:35:21<325:48:35,  6.02s/it]
 13%|█▎        | 30151/225000 [68:35:33<422:28:21,  7.81s/it]
 13%|█▎        | 30152/225000 [68:35:37<354:53:21,  6.56s/it]
 13%|█▎        | 30153/225000 [68:35:41<318:06:24,  5.88s/it]
 13%|█▎        | 30154/225000 [68:35:51<382:20:13,  7.06s/it]
 13%|█▎        | 30155/225000 [68:35:58<375:36:46,  6.94s/it]
 13%|█▎        | 30156/225000 [68:36:02<329:23:32,  6.09s/it]
 13%|█▎        | 30157/225000 [68:36:11<374:50:13,  6.93s/it]
 13%|█▎        | 30158/225000 [68:36:14<319:55:39,  5.91s/it]
 13%|█▎        | 30159/225000 [68:36:19<307:43:08,  5.69s/it]
 13%|█▎        | 30160/225000 [68:36:25<305:38:13,  5.65s/it]
 13%|█▎        | 30161/225000 [68:36:30<299:36:32,  5.54s/it]
 13%|█▎        | 30162/225000 [68:36:35<294:25:35,  5.44s/it]
 13%|█▎        | 30163/225000 [68:36:41<291:25:33,  5.38s/it]
 13%|█▎        | 30164/225000 [68:36:48<320:05:18,  5.91s/it]
 13%|█▎        | 30165/225000 [68:37:02<454:07:45,  8.39s/it]
 13%|█▎        | 30166/225000 [68:37:06<381:33:14,  7.05s/it]
 13%|█▎        | 30167/225000 [68:37:10<336:41:56,  6.22s/it]
 13%|█▎        | 30168/225000 [68:37:15<305:19:07,  5.64s/it]
 13%|█▎        | 30169/225000 [68:37:26<401:07:58,  7.41s/it]
 13%|█▎        | 30170/225000 [68:37:30<340:13:51,  6.29s/it]
 13%|█▎        | 30171/225000 [68:37:37<350:24:18,  6.47s/it]
 13%|█▎        | 30172/225000 [68:37:45<375:52:57,  6.95s/it]
 13%|█▎        | 30173/225000 [68:37:51<359:56:12,  6.65s/it]
 13%|█▎        | 30174/225000 [68:37:57<349:54:51,  6.47s/it]
 13%|█▎        | 30175/225000 [68:38:01<317:10:16,  5.86s/it]
 13%|█▎        | 30176/225000 [68:38:06<303:25:33,  5.61s/it]
 13%|█▎        | 30177/225000 [68:38:11<292:28:25,  5.40s/it]
 13%|█▎        | 30178/225000 [68:38:22<377:47:19,  6.98s/it]
 13%|█▎        | 30179/225000 [68:38:29<375:52:36,  6.95s/it]
 13%|█▎        | 30180/225000 [68:38:36<385:12:27,  7.12s/it]
 13%|█▎        | 30181/225000 [68:38:41<352:05:09,  6.51s/it]
 13%|█▎        | 30182/225000 [68:38:46<318:37:18,  5.89s/it]
 13%|█▎        | 30183/225000 [68:38:50<293:51:31,  5.43s/it]
 13%|█▎        | 30184/225000 [68:38:55<294:07:14,  5.44s/it]
 13%|█▎        | 30185/225000 [68:39:08<406:26:14,  7.51s/it]
 13%|█▎        | 30186/225000 [68:39:11<343:04:27,  6.34s/it]
 13%|█▎        | 30187/225000 [68:39:21<387:48:09,  7.17s/it]
 13%|█▎        | 30188/225000 [68:39:24<332:44:37,  6.15s/it]
 13%|█▎        | 30189/225000 [68:39:32<365:46:16,  6.76s/it]
 13%|█▎        | 30190/225000 [68:39:36<319:31:48,  5.90s/it]
 13%|█▎        | 30191/225000 [68:39:46<383:22:36,  7.08s/it]
 13%|█▎        | 30192/225000 [68:39:51<342:48:44,  6.34s/it]
 13%|█▎        | 30193/225000 [68:39:56<320:40:32,  5.93s/it]
 13%|█▎        | 30194/225000 [68:40:01<309:52:49,  5.73s/it]
 13%|█▎        | 30195/225000 [68:40:06<295:26:37,  5.46s/it]
 13%|█▎        | 30196/225000 [68:40:11<282:43:36,  5.22s/it]
 13%|█▎        | 30197/225000 [68:40:18<325:11:16,  6.01s/it]
 13%|█▎        | 30198/225000 [68:40:30<408:04:51,  7.54s/it]
 13%|█▎        | 30199/225000 [68:40:34<350:50:25,  6.48s/it]
 13%|█▎        | 30200/225000 [68:40:37<308:11:33,  5.70s/it]
                                                             
{'loss': '0.7162', 'grad_norm': '0.6571', 'learning_rate': '4.892e-05', 'epoch': '0.1342', 'num_input_tokens_seen': 117546168, 'train_runtime': '2.472e+05', 'train_tokens_per_second': '475.4'}

 13%|█▎        | 30200/225000 [68:40:37<308:11:33,  5.70s/it]
 13%|█▎        | 30201/225000 [68:40:42<290:02:10,  5.36s/it]
 13%|█▎        | 30202/225000 [68:40:52<358:05:47,  6.62s/it]
 13%|█▎        | 30203/225000 [68:41:01<399:25:13,  7.38s/it]
 13%|█▎        | 30204/225000 [68:41:04<334:44:37,  6.19s/it]
 13%|█▎        | 30205/225000 [68:41:13<376:53:34,  6.97s/it]
 13%|█▎        | 30206/225000 [68:41:17<323:51:40,  5.99s/it]
 13%|█▎        | 30207/225000 [68:41:21<291:24:38,  5.39s/it]
 13%|█▎        | 30208/225000 [68:41:25<283:43:28,  5.24s/it]
 13%|█▎        | 30209/225000 [68:41:31<285:43:40,  5.28s/it]
 13%|█▎        | 30210/225000 [68:41:38<310:33:39,  5.74s/it]
 13%|█▎        | 30211/225000 [68:41:52<449:30:05,  8.31s/it]
 13%|█▎        | 30212/225000 [68:41:58<409:19:44,  7.57s/it]
 13%|█▎        | 30213/225000 [68:42:06<415:06:03,  7.67s/it]
 13%|█▎        | 30214/225000 [68:42:10<360:24:56,  6.66s/it]
 13%|█▎        | 30215/225000 [68:42:19<393:03:22,  7.26s/it]
 13%|█▎        | 30216/225000 [68:42:24<368:49:26,  6.82s/it]
 13%|█▎        | 30217/225000 [68:42:30<355:54:36,  6.58s/it]
 13%|█▎        | 30218/225000 [68:42:37<362:58:15,  6.71s/it]
 13%|█▎        | 30219/225000 [68:42:41<312:01:47,  5.77s/it]
 13%|█▎        | 30220/225000 [68:42:49<340:31:18,  6.29s/it]
 13%|█▎        | 30221/225000 [68:42:56<363:17:48,  6.71s/it]
 13%|█▎        | 30222/225000 [68:43:02<354:12:55,  6.55s/it]
 13%|█▎        | 30223/225000 [68:43:07<322:34:22,  5.96s/it]
 13%|█▎        | 30224/225000 [68:43:12<302:39:44,  5.59s/it]
 13%|█▎        | 30225/225000 [68:43:16<288:03:11,  5.32s/it]
 13%|█▎        | 30226/225000 [68:43:21<283:16:02,  5.24s/it]
 13%|█▎        | 30227/225000 [68:43:29<319:22:37,  5.90s/it]
 13%|█▎        | 30228/225000 [68:43:41<416:53:01,  7.71s/it]
 13%|█▎        | 30229/225000 [68:43:45<353:50:16,  6.54s/it]
 13%|█▎        | 30230/225000 [68:43:53<384:49:53,  7.11s/it]
 13%|█▎        | 30231/225000 [68:43:57<335:28:54,  6.20s/it]
 13%|█▎        | 30232/225000 [68:44:03<324:49:49,  6.00s/it]
 13%|█▎        | 30233/225000 [68:44:13<385:55:40,  7.13s/it]
 13%|█▎        | 30234/225000 [68:44:17<343:46:34,  6.35s/it]
 13%|█▎        | 30235/225000 [68:44:25<375:05:59,  6.93s/it]
 13%|█▎        | 30236/225000 [68:44:29<323:45:44,  5.98s/it]
 13%|█▎        | 30237/225000 [68:44:40<396:18:33,  7.33s/it]
 13%|█▎        | 30238/225000 [68:44:43<335:25:16,  6.20s/it]
 13%|█▎        | 30239/225000 [68:44:54<410:39:51,  7.59s/it]
 13%|█▎        | 30240/225000 [68:44:57<342:05:54,  6.32s/it]
 13%|█▎        | 30241/225000 [68:45:03<333:21:23,  6.16s/it]
 13%|█▎        | 30242/225000 [68:45:11<363:51:08,  6.73s/it]
 13%|█▎        | 30243/225000 [68:45:16<337:00:18,  6.23s/it]
 13%|█▎        | 30244/225000 [68:45:22<327:18:55,  6.05s/it]
 13%|█▎        | 30245/225000 [68:45:27<308:55:18,  5.71s/it]
 13%|█▎        | 30246/225000 [68:45:32<307:31:34,  5.68s/it]
 13%|█▎        | 30247/225000 [68:45:46<430:18:11,  7.95s/it]
 13%|█▎        | 30248/225000 [68:45:51<384:24:12,  7.11s/it]
 13%|█▎        | 30249/225000 [68:45:56<351:56:01,  6.51s/it]
 13%|█▎        | 30250/225000 [68:46:06<405:39:50,  7.50s/it]
                                                             
{'loss': '0.7128', 'grad_norm': '0.6997', 'learning_rate': '4.892e-05', 'epoch': '0.1344', 'num_input_tokens_seen': 117690736, 'train_runtime': '2.476e+05', 'train_tokens_per_second': '475.4'}

 13%|█▎        | 30250/225000 [68:46:06<405:39:50,  7.50s/it]
 13%|█▎        | 30251/225000 [68:46:10<348:21:54,  6.44s/it]
 13%|█▎        | 30252/225000 [68:46:14<313:28:43,  5.79s/it]
 13%|█▎        | 30253/225000 [68:46:24<380:23:48,  7.03s/it]
 13%|█▎        | 30254/225000 [68:46:31<375:20:42,  6.94s/it]
 13%|█▎        | 30255/225000 [68:46:36<344:54:09,  6.38s/it]
 13%|█▎        | 30256/225000 [68:46:44<383:00:15,  7.08s/it]
 13%|█▎        | 30257/225000 [68:46:50<354:06:42,  6.55s/it]
 13%|█▎        | 30258/225000 [68:46:56<350:51:09,  6.49s/it]
 13%|█▎        | 30259/225000 [68:47:00<307:53:53,  5.69s/it]
 13%|█▎        | 30260/225000 [68:47:08<341:32:15,  6.31s/it]
 13%|█▎        | 30261/225000 [68:47:13<330:27:42,  6.11s/it]
 13%|█▎        | 30262/225000 [68:47:19<321:47:15,  5.95s/it]
 13%|█▎        | 30263/225000 [68:47:24<314:05:28,  5.81s/it]
 13%|█▎        | 30264/225000 [68:47:30<312:06:54,  5.77s/it]
 13%|█▎        | 30265/225000 [68:47:39<361:02:16,  6.67s/it]
 13%|█▎        | 30266/225000 [68:47:46<376:24:03,  6.96s/it]
 13%|█▎        | 30267/225000 [68:47:51<336:18:54,  6.22s/it]
 13%|█▎        | 30268/225000 [68:47:55<309:57:44,  5.73s/it]
 13%|█▎        | 30269/225000 [68:48:06<390:38:59,  7.22s/it]
 13%|█▎        | 30270/225000 [68:48:12<360:41:24,  6.67s/it]
 13%|█▎        | 30271/225000 [68:48:17<346:09:39,  6.40s/it]
 13%|█▎        | 30272/225000 [68:48:23<326:45:15,  6.04s/it]
 13%|█▎        | 30273/225000 [68:48:28<313:17:14,  5.79s/it]
 13%|█▎        | 30274/225000 [68:48:33<303:30:46,  5.61s/it]
 13%|█▎        | 30275/225000 [68:48:38<299:11:14,  5.53s/it]
 13%|█▎        | 30276/225000 [68:48:43<291:21:10,  5.39s/it]
 13%|█▎        | 30277/225000 [68:48:49<295:32:04,  5.46s/it]
 13%|█▎        | 30278/225000 [68:49:00<393:40:15,  7.28s/it]
 13%|█▎        | 30279/225000 [68:49:12<455:34:59,  8.42s/it]
 13%|█▎        | 30280/225000 [68:49:15<379:02:22,  7.01s/it]
 13%|█▎        | 30281/225000 [68:49:24<402:38:53,  7.44s/it]
 13%|█▎        | 30282/225000 [68:49:29<365:50:27,  6.76s/it]
 13%|█▎        | 30283/225000 [68:49:33<327:03:22,  6.05s/it]
 13%|█▎        | 30284/225000 [68:49:38<309:30:42,  5.72s/it]
 13%|█▎        | 30285/225000 [68:49:43<297:13:33,  5.50s/it]
 13%|█▎        | 30286/225000 [68:49:50<320:14:09,  5.92s/it]
 13%|█▎        | 30287/225000 [68:50:03<429:04:43,  7.93s/it]
 13%|█▎        | 30288/225000 [68:50:06<360:03:46,  6.66s/it]
 13%|█▎        | 30289/225000 [68:50:12<343:20:37,  6.35s/it]
 13%|█▎        | 30290/225000 [68:50:17<326:08:40,  6.03s/it]
 13%|█▎        | 30291/225000 [68:50:23<315:37:16,  5.84s/it]
 13%|█▎        | 30292/225000 [68:50:28<312:08:40,  5.77s/it]
 13%|█▎        | 30293/225000 [68:50:33<299:28:51,  5.54s/it]
 13%|█▎        | 30294/225000 [68:50:39<307:25:43,  5.68s/it]
 13%|█▎        | 30295/225000 [68:50:53<440:04:52,  8.14s/it]
 13%|█▎        | 30296/225000 [68:50:59<404:24:28,  7.48s/it]
 13%|█▎        | 30297/225000 [68:51:04<362:56:46,  6.71s/it]
 13%|█▎        | 30298/225000 [68:51:09<334:11:41,  6.18s/it]
 13%|█▎        | 30299/225000 [68:51:14<308:33:07,  5.71s/it]
 13%|█▎        | 30300/225000 [68:51:18<294:27:40,  5.44s/it]
                                                             
{'loss': '0.7101', 'grad_norm': '0.5755', 'learning_rate': '4.891e-05', 'epoch': '0.1347', 'num_input_tokens_seen': 117833288, 'train_runtime': '2.479e+05', 'train_tokens_per_second': '475.4'}

 13%|█▎        | 30300/225000 [68:51:18<294:27:40,  5.44s/it]
 13%|█▎        | 30301/225000 [68:51:27<342:20:37,  6.33s/it]
 13%|█▎        | 30302/225000 [68:51:38<426:20:50,  7.88s/it]
 13%|█▎        | 30303/225000 [68:51:45<405:11:21,  7.49s/it]
 13%|█▎        | 30304/225000 [68:51:48<339:54:31,  6.29s/it]
 13%|█▎        | 30305/225000 [68:51:58<387:39:58,  7.17s/it]
 13%|█▎        | 30306/225000 [68:52:01<328:56:52,  6.08s/it]
 13%|█▎        | 30307/225000 [68:52:10<366:50:19,  6.78s/it]
 13%|█▎        | 30308/225000 [68:52:14<327:59:07,  6.06s/it]
 13%|█▎        | 30309/225000 [68:52:18<302:05:07,  5.59s/it]
 13%|█▎        | 30310/225000 [68:52:26<329:40:34,  6.10s/it]
 13%|█▎        | 30311/225000 [68:52:35<379:39:41,  7.02s/it]
 13%|█▎        | 30312/225000 [68:52:40<347:10:45,  6.42s/it]
 13%|█▎        | 30313/225000 [68:52:46<333:52:52,  6.17s/it]
 13%|█▎        | 30314/225000 [68:52:53<355:14:18,  6.57s/it]
 13%|█▎        | 30315/225000 [68:52:58<325:02:18,  6.01s/it]
 13%|█▎        | 30316/225000 [68:53:02<302:36:21,  5.60s/it]
 13%|█▎        | 30317/225000 [68:53:07<285:19:09,  5.28s/it]
 13%|█▎        | 30318/225000 [68:53:17<357:50:21,  6.62s/it]
 13%|█▎        | 30319/225000 [68:53:22<342:27:50,  6.33s/it]
 13%|█▎        | 30320/225000 [68:53:27<311:17:46,  5.76s/it]
 13%|█▎        | 30321/225000 [68:53:34<341:08:51,  6.31s/it]
 13%|█▎        | 30322/225000 [68:53:45<407:32:28,  7.54s/it]
 13%|█▎        | 30323/225000 [68:53:49<350:10:06,  6.48s/it]
 13%|█▎        | 30324/225000 [68:53:58<397:43:54,  7.35s/it]
 13%|█▎        | 30325/225000 [68:54:02<339:01:43,  6.27s/it]
 13%|█▎        | 30326/225000 [68:54:11<379:22:30,  7.02s/it]
 13%|█▎        | 30327/225000 [68:54:15<341:27:44,  6.31s/it]
 13%|█▎        | 30328/225000 [68:54:23<365:41:00,  6.76s/it]
 13%|█▎        | 30329/225000 [68:54:29<346:23:35,  6.41s/it]
 13%|█▎        | 30330/225000 [68:54:34<331:33:48,  6.13s/it]
 13%|█▎        | 30331/225000 [68:54:39<316:25:17,  5.85s/it]
 13%|█▎        | 30332/225000 [68:54:45<317:20:22,  5.87s/it]
 13%|█▎        | 30333/225000 [68:54:56<400:36:02,  7.41s/it]
 13%|█▎        | 30334/225000 [68:55:01<352:27:16,  6.52s/it]
 13%|█▎        | 30335/225000 [68:55:05<321:43:42,  5.95s/it]
 13%|█▎        | 30336/225000 [68:55:10<296:36:52,  5.49s/it]
 13%|█▎        | 30337/225000 [68:55:21<396:46:57,  7.34s/it]
 13%|█▎        | 30338/225000 [68:55:28<386:52:50,  7.15s/it]
 13%|█▎        | 30339/225000 [68:55:33<348:40:45,  6.45s/it]
 13%|█▎        | 30340/225000 [68:55:40<356:50:06,  6.60s/it]
 13%|█▎        | 30341/225000 [68:55:45<332:40:15,  6.15s/it]
 13%|█▎        | 30342/225000 [68:55:50<316:51:49,  5.86s/it]
 13%|█▎        | 30343/225000 [68:55:56<308:04:38,  5.70s/it]
 13%|█▎        | 30344/225000 [68:56:01<305:43:17,  5.65s/it]
 13%|█▎        | 30345/225000 [68:56:06<294:57:10,  5.45s/it]
 13%|█▎        | 30346/225000 [68:56:11<287:37:00,  5.32s/it]
 13%|█▎        | 30347/225000 [68:56:17<292:13:29,  5.40s/it]
 13%|█▎        | 30348/225000 [68:56:22<298:19:33,  5.52s/it]
 13%|█▎        | 30349/225000 [68:56:28<301:13:20,  5.57s/it]
 13%|█▎        | 30350/225000 [68:56:33<296:01:06,  5.47s/it]
                                                             
{'loss': '0.7171', 'grad_norm': '0.6956', 'learning_rate': '4.89e-05', 'epoch': '0.1349', 'num_input_tokens_seen': 117976256, 'train_runtime': '2.482e+05', 'train_tokens_per_second': '475.3'}

 13%|█▎        | 30350/225000 [68:56:33<296:01:06,  5.47s/it]
 13%|█▎        | 30351/225000 [68:56:42<339:49:34,  6.29s/it]
 13%|█▎        | 30352/225000 [68:56:47<323:41:30,  5.99s/it]
 13%|█▎        | 30353/225000 [68:56:52<315:07:40,  5.83s/it]
 13%|█▎        | 30354/225000 [68:57:01<357:03:56,  6.60s/it]
 13%|█▎        | 30355/225000 [68:57:13<443:48:24,  8.21s/it]
 13%|█▎        | 30356/225000 [68:57:16<372:25:43,  6.89s/it]
 13%|█▎        | 30357/225000 [68:57:25<397:34:08,  7.35s/it]
 13%|█▎        | 30358/225000 [68:57:30<360:26:40,  6.67s/it]
 13%|█▎        | 30359/225000 [68:57:34<322:39:38,  5.97s/it]
 13%|█▎        | 30360/225000 [68:57:39<294:47:43,  5.45s/it]
 13%|█▎        | 30361/225000 [68:57:43<284:42:16,  5.27s/it]
 13%|█▎        | 30362/225000 [68:57:55<385:56:45,  7.14s/it]
 13%|█▎        | 30363/225000 [68:58:02<381:08:54,  7.05s/it]
 13%|█▎        | 30364/225000 [68:58:06<340:07:40,  6.29s/it]
 13%|█▎        | 30365/225000 [68:58:17<405:23:13,  7.50s/it]
 13%|█▎        | 30366/225000 [68:58:20<336:59:04,  6.23s/it]
 13%|█▎        | 30367/225000 [68:58:28<360:57:15,  6.68s/it]
 13%|█▎        | 30368/225000 [68:58:33<340:04:42,  6.29s/it]
 13%|█▎        | 30369/225000 [68:58:38<326:09:16,  6.03s/it]
 13%|█▎        | 30370/225000 [68:58:43<309:47:33,  5.73s/it]
 13%|█▎        | 30371/225000 [68:58:48<298:54:35,  5.53s/it]
 13%|█▎        | 30372/225000 [68:58:54<293:25:21,  5.43s/it]
 13%|█▎        | 30373/225000 [68:58:59<293:50:44,  5.44s/it]
 13%|█▎        | 30374/225000 [68:59:11<399:45:26,  7.39s/it]
 14%|█▎        | 30375/225000 [68:59:18<398:22:23,  7.37s/it]
 14%|█▎        | 30376/225000 [68:59:25<378:55:54,  7.01s/it]
 14%|█▎        | 30377/225000 [68:59:29<339:46:09,  6.28s/it]
 14%|█▎        | 30378/225000 [68:59:36<350:42:50,  6.49s/it]
 14%|█▎        | 30379/225000 [68:59:41<323:55:32,  5.99s/it]
 14%|█▎        | 30380/225000 [68:59:46<309:29:26,  5.72s/it]
 14%|█▎        | 30381/225000 [68:59:51<292:51:27,  5.42s/it]
 14%|█▎        | 30382/225000 [69:00:04<424:14:57,  7.85s/it]
 14%|█▎        | 30383/225000 [69:00:08<364:36:13,  6.74s/it]
 14%|█▎        | 30384/225000 [69:00:13<324:04:09,  5.99s/it]
 14%|█▎        | 30385/225000 [69:00:18<312:59:14,  5.79s/it]
 14%|█▎        | 30386/225000 [69:00:29<394:17:09,  7.29s/it]
 14%|█▎        | 30387/225000 [69:00:32<331:47:45,  6.14s/it]
 14%|█▎        | 30388/225000 [69:00:40<352:22:25,  6.52s/it]
 14%|█▎        | 30389/225000 [69:00:48<379:49:56,  7.03s/it]
 14%|█▎        | 30390/225000 [69:00:52<339:43:40,  6.28s/it]
 14%|█▎        | 30391/225000 [69:00:57<310:28:54,  5.74s/it]
 14%|█▎        | 30392/225000 [69:01:02<293:18:07,  5.43s/it]
 14%|█▎        | 30393/225000 [69:01:13<383:10:12,  7.09s/it]
 14%|█▎        | 30394/225000 [69:01:17<343:27:22,  6.35s/it]
 14%|█▎        | 30395/225000 [69:01:23<329:24:31,  6.09s/it]
 14%|█▎        | 30396/225000 [69:01:31<364:05:02,  6.74s/it]
 14%|█▎        | 30397/225000 [69:01:38<366:13:12,  6.77s/it]
 14%|█▎        | 30398/225000 [69:01:43<346:03:00,  6.40s/it]
 14%|█▎        | 30399/225000 [69:01:48<311:30:04,  5.76s/it]
 14%|█▎        | 30400/225000 [69:01:53<311:28:27,  5.76s/it]
                                                             
{'loss': '0.7105', 'grad_norm': '0.9258', 'learning_rate': '4.89e-05', 'epoch': '0.1351', 'num_input_tokens_seen': 118118448, 'train_runtime': '2.485e+05', 'train_tokens_per_second': '475.3'}

 14%|█▎        | 30400/225000 [69:01:53<311:28:27,  5.76s/it]
 14%|█▎        | 30401/225000 [69:02:03<377:08:27,  6.98s/it]
 14%|█▎        | 30402/225000 [69:02:07<323:53:33,  5.99s/it]
 14%|█▎        | 30403/225000 [69:02:15<359:37:21,  6.65s/it]
 14%|█▎        | 30404/225000 [69:02:22<371:30:01,  6.87s/it]
 14%|█▎        | 30405/225000 [69:02:28<353:38:02,  6.54s/it]
 14%|█▎        | 30406/225000 [69:02:36<379:29:57,  7.02s/it]
 14%|█▎        | 30407/225000 [69:02:43<379:31:00,  7.02s/it]
 14%|█▎        | 30408/225000 [69:02:47<326:15:49,  6.04s/it]
 14%|█▎        | 30409/225000 [69:02:53<316:09:35,  5.85s/it]
 14%|█▎        | 30410/225000 [69:02:57<296:40:13,  5.49s/it]
 14%|█▎        | 30411/225000 [69:03:02<293:24:17,  5.43s/it]
 14%|█▎        | 30412/225000 [69:03:15<414:31:43,  7.67s/it]
 14%|█▎        | 30413/225000 [69:03:19<351:40:03,  6.51s/it]
 14%|█▎        | 30414/225000 [69:03:26<359:48:47,  6.66s/it]
 14%|█▎        | 30415/225000 [69:03:33<365:02:35,  6.75s/it]
 14%|█▎        | 30416/225000 [69:03:37<323:50:54,  5.99s/it]
 14%|█▎        | 30417/225000 [69:03:41<286:56:02,  5.31s/it]
 14%|█▎        | 30418/225000 [69:03:52<382:35:59,  7.08s/it]
 14%|█▎        | 30419/225000 [69:03:56<323:37:56,  5.99s/it]
 14%|█▎        | 30420/225000 [69:04:04<362:31:46,  6.71s/it]
 14%|█▎        | 30421/225000 [69:04:10<349:23:25,  6.46s/it]
 14%|█▎        | 30422/225000 [69:04:14<315:03:49,  5.83s/it]
 14%|█▎        | 30423/225000 [69:04:23<359:13:17,  6.65s/it]
 14%|█▎        | 30424/225000 [69:04:30<370:49:28,  6.86s/it]
 14%|█▎        | 30425/225000 [69:04:36<350:24:23,  6.48s/it]
 14%|█▎        | 30426/225000 [69:04:41<332:25:58,  6.15s/it]
 14%|█▎        | 30427/225000 [69:04:47<319:50:15,  5.92s/it]
 14%|█▎        | 30428/225000 [69:04:52<316:07:04,  5.85s/it]
 14%|█▎        | 30429/225000 [69:04:58<314:19:08,  5.82s/it]
 14%|█▎        | 30430/225000 [69:05:04<310:56:07,  5.75s/it]
 14%|█▎        | 30431/225000 [69:05:09<306:54:57,  5.68s/it]
 14%|█▎        | 30432/225000 [69:05:19<378:00:09,  6.99s/it]
 14%|█▎        | 30433/225000 [69:05:26<378:09:36,  7.00s/it]
 14%|█▎        | 30434/225000 [69:05:30<333:16:24,  6.17s/it]
 14%|█▎        | 30435/225000 [69:05:35<310:23:10,  5.74s/it]
 14%|█▎        | 30436/225000 [69:05:42<332:27:17,  6.15s/it]
 14%|█▎        | 30437/225000 [69:05:55<432:38:08,  8.01s/it]
 14%|█▎        | 30438/225000 [69:05:59<367:28:52,  6.80s/it]
 14%|█▎        | 30439/225000 [69:06:03<332:22:29,  6.15s/it]
 14%|█▎        | 30440/225000 [69:06:08<312:31:46,  5.78s/it]
 14%|█▎        | 30441/225000 [69:06:13<299:30:18,  5.54s/it]
 14%|█▎        | 30442/225000 [69:06:18<285:42:54,  5.29s/it]
 14%|█▎        | 30443/225000 [69:06:23<281:54:19,  5.22s/it]
 14%|█▎        | 30444/225000 [69:06:36<404:54:18,  7.49s/it]
 14%|█▎        | 30445/225000 [69:06:43<403:59:41,  7.48s/it]
 14%|█▎        | 30446/225000 [69:06:50<398:30:54,  7.37s/it]
 14%|█▎        | 30447/225000 [69:06:55<350:18:32,  6.48s/it]
 14%|█▎        | 30448/225000 [69:06:59<322:34:31,  5.97s/it]
 14%|█▎        | 30449/225000 [69:07:04<306:23:01,  5.67s/it]
 14%|█▎        | 30450/225000 [69:07:10<302:34:31,  5.60s/it]
                                                             
{'loss': '0.715', 'grad_norm': '0.6661', 'learning_rate': '4.889e-05', 'epoch': '0.1353', 'num_input_tokens_seen': 118261488, 'train_runtime': '2.488e+05', 'train_tokens_per_second': '475.3'}

 14%|█▎        | 30450/225000 [69:07:10<302:34:31,  5.60s/it]
 14%|█▎        | 30451/225000 [69:07:16<311:25:50,  5.76s/it]
 14%|█▎        | 30452/225000 [69:07:29<423:48:54,  7.84s/it]
 14%|█▎        | 30453/225000 [69:07:34<384:53:19,  7.12s/it]
 14%|█▎        | 30454/225000 [69:07:39<347:59:08,  6.44s/it]
 14%|█▎        | 30455/225000 [69:07:43<315:31:15,  5.84s/it]
 14%|█▎        | 30456/225000 [69:07:55<412:10:41,  7.63s/it]
 14%|█▎        | 30457/225000 [69:08:02<390:15:34,  7.22s/it]
 14%|█▎        | 30458/225000 [69:08:05<332:23:40,  6.15s/it]
 14%|█▎        | 30459/225000 [69:08:10<306:27:58,  5.67s/it]
 14%|█▎        | 30460/225000 [69:08:21<403:19:22,  7.46s/it]
 14%|█▎        | 30461/225000 [69:08:26<354:49:11,  6.57s/it]
 14%|█▎        | 30462/225000 [69:08:31<328:44:10,  6.08s/it]
 14%|█▎        | 30463/225000 [69:08:37<326:59:07,  6.05s/it]
 14%|█▎        | 30464/225000 [69:08:42<321:43:45,  5.95s/it]
 14%|█▎        | 30465/225000 [69:08:48<319:04:05,  5.90s/it]
 14%|█▎        | 30466/225000 [69:08:58<387:53:49,  7.18s/it]
 14%|█▎        | 30467/225000 [69:09:04<358:07:07,  6.63s/it]
 14%|█▎        | 30468/225000 [69:09:09<335:33:31,  6.21s/it]
 14%|█▎        | 30469/225000 [69:09:14<316:44:41,  5.86s/it]
 14%|█▎        | 30470/225000 [69:09:20<313:47:58,  5.81s/it]
 14%|█▎        | 30471/225000 [69:09:27<335:43:57,  6.21s/it]
 14%|█▎        | 30472/225000 [69:09:40<447:49:02,  8.29s/it]
 14%|█▎        | 30473/225000 [69:09:45<401:14:58,  7.43s/it]
 14%|█▎        | 30474/225000 [69:09:49<337:37:44,  6.25s/it]
 14%|█▎        | 30475/225000 [69:09:58<391:22:53,  7.24s/it]
 14%|█▎        | 30476/225000 [69:10:03<350:58:28,  6.50s/it]
 14%|█▎        | 30477/225000 [69:10:08<323:21:25,  5.98s/it]
 14%|█▎        | 30478/225000 [69:10:13<303:24:11,  5.62s/it]
 14%|█▎        | 30479/225000 [69:10:25<408:03:25,  7.55s/it]
 14%|█▎        | 30480/225000 [69:10:30<376:36:25,  6.97s/it]
 14%|█▎        | 30481/225000 [69:10:38<392:16:58,  7.26s/it]
 14%|█▎        | 30482/225000 [69:10:44<372:56:59,  6.90s/it]
 14%|█▎        | 30483/225000 [69:10:50<350:00:45,  6.48s/it]
 14%|█▎        | 30484/225000 [69:10:58<374:55:53,  6.94s/it]
 14%|█▎        | 30485/225000 [69:11:02<330:13:12,  6.11s/it]
 14%|█▎        | 30486/225000 [69:11:07<301:36:48,  5.58s/it]
 14%|█▎        | 30487/225000 [69:11:11<281:31:09,  5.21s/it]
 14%|█▎        | 30488/225000 [69:11:15<265:43:10,  4.92s/it]
 14%|█▎        | 30489/225000 [69:11:27<380:42:44,  7.05s/it]
 14%|█▎        | 30490/225000 [69:11:31<335:59:35,  6.22s/it]
 14%|█▎        | 30491/225000 [69:11:36<310:24:39,  5.75s/it]
 14%|█▎        | 30492/225000 [69:11:41<292:24:55,  5.41s/it]
 14%|█▎        | 30493/225000 [69:11:45<278:01:05,  5.15s/it]
 14%|█▎        | 30494/225000 [69:11:53<326:20:34,  6.04s/it]
 14%|█▎        | 30495/225000 [69:12:07<443:16:19,  8.20s/it]
 14%|█▎        | 30496/225000 [69:12:11<377:41:58,  6.99s/it]
 14%|█▎        | 30497/225000 [69:12:21<433:09:56,  8.02s/it]
 14%|█▎        | 30498/225000 [69:12:28<407:13:13,  7.54s/it]
 14%|█▎        | 30499/225000 [69:12:32<357:55:01,  6.62s/it]
 14%|█▎        | 30500/225000 [69:12:37<336:00:53,  6.22s/it]
                                                             
{'loss': '0.703', 'grad_norm': '0.6791', 'learning_rate': '4.889e-05', 'epoch': '0.1356', 'num_input_tokens_seen': 118404496, 'train_runtime': '2.492e+05', 'train_tokens_per_second': '475.2'}

 14%|█▎        | 30500/225000 [69:12:37<336:00:53,  6.22s/it]
 14%|█▎        | 30501/225000 [69:12:42<311:47:04,  5.77s/it]
 14%|█▎        | 30502/225000 [69:12:47<296:55:19,  5.50s/it]
 14%|█▎        | 30503/225000 [69:12:52<283:40:55,  5.25s/it]
 14%|█▎        | 30504/225000 [69:13:05<413:12:07,  7.65s/it]
 14%|█▎        | 30505/225000 [69:13:11<389:41:35,  7.21s/it]
 14%|█▎        | 30506/225000 [69:13:15<340:04:36,  6.29s/it]
 14%|█▎        | 30507/225000 [69:13:26<415:29:00,  7.69s/it]
 14%|█▎        | 30508/225000 [69:13:30<356:57:49,  6.61s/it]
 14%|█▎        | 30509/225000 [69:13:38<374:19:54,  6.93s/it]
 14%|█▎        | 30510/225000 [69:13:46<393:35:08,  7.29s/it]
 14%|█▎        | 30511/225000 [69:13:49<328:27:02,  6.08s/it]
 14%|█▎        | 30512/225000 [69:13:57<349:44:33,  6.47s/it]
 14%|█▎        | 30513/225000 [69:14:01<318:00:21,  5.89s/it]
 14%|█▎        | 30514/225000 [69:14:06<306:00:21,  5.66s/it]
 14%|█▎        | 30515/225000 [69:14:11<291:30:43,  5.40s/it]
 14%|█▎        | 30516/225000 [69:14:16<286:38:07,  5.31s/it]
 14%|█▎        | 30517/225000 [69:14:22<291:17:57,  5.39s/it]
 14%|█▎        | 30518/225000 [69:14:27<286:51:17,  5.31s/it]
 14%|█▎        | 30519/225000 [69:14:40<411:56:14,  7.63s/it]
 14%|█▎        | 30520/225000 [69:14:46<388:47:23,  7.20s/it]
 14%|█▎        | 30521/225000 [69:14:50<337:45:52,  6.25s/it]
 14%|█▎        | 30522/225000 [69:15:00<398:00:35,  7.37s/it]
 14%|█▎        | 30523/225000 [69:15:06<367:02:51,  6.79s/it]
 14%|█▎        | 30524/225000 [69:15:12<362:26:31,  6.71s/it]
 14%|█▎        | 30525/225000 [69:15:19<367:48:37,  6.81s/it]
 14%|█▎        | 30526/225000 [69:15:26<363:30:44,  6.73s/it]
 14%|█▎        | 30527/225000 [69:15:32<357:52:11,  6.62s/it]
 14%|█▎        | 30528/225000 [69:15:36<312:21:13,  5.78s/it]
 14%|█▎        | 30529/225000 [69:15:47<391:05:42,  7.24s/it]
 14%|█▎        | 30530/225000 [69:15:51<352:03:52,  6.52s/it]
 14%|█▎        | 30531/225000 [69:16:02<422:00:33,  7.81s/it]
 14%|█▎        | 30532/225000 [69:16:06<361:19:58,  6.69s/it]
 14%|█▎        | 30533/225000 [69:16:14<378:40:31,  7.01s/it]
 14%|█▎        | 30534/225000 [69:16:18<328:23:20,  6.08s/it]
 14%|█▎        | 30535/225000 [69:16:28<394:32:23,  7.30s/it]
 14%|█▎        | 30536/225000 [69:16:34<371:43:11,  6.88s/it]
 14%|█▎        | 30537/225000 [69:16:42<384:25:01,  7.12s/it]
 14%|█▎        | 30538/225000 [69:16:46<341:31:45,  6.32s/it]
 14%|█▎        | 30539/225000 [69:16:51<311:33:52,  5.77s/it]
 14%|█▎        | 30540/225000 [69:16:56<299:02:32,  5.54s/it]
 14%|█▎        | 30541/225000 [69:17:01<290:38:52,  5.38s/it]
 14%|█▎        | 30542/225000 [69:17:09<345:14:03,  6.39s/it]
 14%|█▎        | 30543/225000 [69:17:18<383:11:33,  7.09s/it]
 14%|█▎        | 30544/225000 [69:17:23<341:06:14,  6.31s/it]
 14%|█▎        | 30545/225000 [69:17:30<355:32:10,  6.58s/it]
 14%|█▎        | 30546/225000 [69:17:35<325:35:17,  6.03s/it]
 14%|█▎        | 30547/225000 [69:17:41<331:33:06,  6.14s/it]
 14%|█▎        | 30548/225000 [69:17:50<376:15:51,  6.97s/it]
 14%|█▎        | 30549/225000 [69:17:55<352:22:52,  6.52s/it]
 14%|█▎        | 30550/225000 [69:18:03<372:58:20,  6.91s/it]
                                                             
{'loss': '0.7176', 'grad_norm': '0.6956', 'learning_rate': '4.888e-05', 'epoch': '0.1358', 'num_input_tokens_seen': 118547416, 'train_runtime': '2.495e+05', 'train_tokens_per_second': '475.2'}

 14%|█▎        | 30550/225000 [69:18:03<372:58:20,  6.91s/it]
 14%|█▎        | 30551/225000 [69:18:07<329:59:40,  6.11s/it]
 14%|█▎        | 30552/225000 [69:18:11<295:59:05,  5.48s/it]
 14%|█▎        | 30553/225000 [69:18:23<394:35:09,  7.31s/it]
 14%|█▎        | 30554/225000 [69:18:28<359:36:04,  6.66s/it]
 14%|█▎        | 30555/225000 [69:18:32<315:01:20,  5.83s/it]
 14%|█▎        | 30556/225000 [69:18:42<390:40:11,  7.23s/it]
 14%|█▎        | 30557/225000 [69:18:49<374:08:35,  6.93s/it]
 14%|█▎        | 30558/225000 [69:18:54<350:53:16,  6.50s/it]
 14%|█▎        | 30559/225000 [69:19:00<347:10:40,  6.43s/it]
 14%|█▎        | 30560/225000 [69:19:08<366:49:40,  6.79s/it]
 14%|█▎        | 30561/225000 [69:19:12<321:35:31,  5.95s/it]
 14%|█▎        | 30562/225000 [69:19:19<342:46:20,  6.35s/it]
 14%|█▎        | 30563/225000 [69:19:24<312:32:32,  5.79s/it]
 14%|█▎        | 30564/225000 [69:19:29<297:41:14,  5.51s/it]
 14%|█▎        | 30565/225000 [69:19:34<287:22:43,  5.32s/it]
 14%|█▎        | 30566/225000 [69:19:43<360:21:49,  6.67s/it]
 14%|█▎        | 30567/225000 [69:19:51<381:02:47,  7.06s/it]
 14%|█▎        | 30568/225000 [69:19:56<340:30:12,  6.30s/it]
 14%|█▎        | 30569/225000 [69:20:02<331:54:49,  6.15s/it]
 14%|█▎        | 30570/225000 [69:20:12<404:03:59,  7.48s/it]
 14%|█▎        | 30571/225000 [69:20:16<341:14:34,  6.32s/it]
 14%|█▎        | 30572/225000 [69:20:22<338:26:29,  6.27s/it]
 14%|█▎        | 30573/225000 [69:20:31<389:08:01,  7.21s/it]
 14%|█▎        | 30574/225000 [69:20:36<345:24:17,  6.40s/it]
 14%|█▎        | 30575/225000 [69:20:41<323:46:54,  6.00s/it]
 14%|█▎        | 30576/225000 [69:20:46<312:42:48,  5.79s/it]
 14%|█▎        | 30577/225000 [69:20:52<306:03:41,  5.67s/it]
 14%|█▎        | 30578/225000 [69:20:57<300:48:48,  5.57s/it]
 14%|█▎        | 30579/225000 [69:21:06<352:02:17,  6.52s/it]
 14%|█▎        | 30580/225000 [69:21:18<437:42:37,  8.10s/it]
 14%|█▎        | 30581/225000 [69:21:22<379:07:23,  7.02s/it]
 14%|█▎        | 30582/225000 [69:21:30<395:03:08,  7.32s/it]
 14%|█▎        | 30583/225000 [69:21:34<333:39:17,  6.18s/it]
 14%|█▎        | 30584/225000 [69:21:42<362:07:15,  6.71s/it]
 14%|█▎        | 30585/225000 [69:21:45<313:11:13,  5.80s/it]
 14%|█▎        | 30586/225000 [69:21:54<353:22:39,  6.54s/it]
 14%|█▎        | 30587/225000 [69:21:58<313:01:48,  5.80s/it]
 14%|█▎        | 30588/225000 [69:22:03<299:41:26,  5.55s/it]
 14%|█▎        | 30589/225000 [69:22:08<298:13:25,  5.52s/it]
 14%|█▎        | 30590/225000 [69:22:13<293:04:22,  5.43s/it]
 14%|█▎        | 30591/225000 [69:22:19<296:54:49,  5.50s/it]
 14%|█▎        | 30592/225000 [69:22:33<431:35:31,  7.99s/it]
 14%|█▎        | 30593/225000 [69:22:39<406:56:00,  7.54s/it]
 14%|█▎        | 30594/225000 [69:22:45<383:48:45,  7.11s/it]
 14%|█▎        | 30595/225000 [69:22:53<400:31:19,  7.42s/it]
 14%|█▎        | 30596/225000 [69:22:58<349:11:44,  6.47s/it]
 14%|█▎        | 30597/225000 [69:23:02<315:52:49,  5.85s/it]
 14%|█▎        | 30598/225000 [69:23:14<419:05:34,  7.76s/it]
 14%|█▎        | 30599/225000 [69:23:21<406:04:36,  7.52s/it]
 14%|█▎        | 30600/225000 [69:23:26<354:59:24,  6.57s/it]
                                                             
{'loss': '0.7085', 'grad_norm': '0.7589', 'learning_rate': '4.888e-05', 'epoch': '0.136', 'num_input_tokens_seen': 118689224, 'train_runtime': '2.498e+05', 'train_tokens_per_second': '475.1'}

 14%|█▎        | 30600/225000 [69:23:26<354:59:24,  6.57s/it]
 14%|█▎        | 30601/225000 [69:23:29<307:40:38,  5.70s/it]
 14%|█▎        | 30602/225000 [69:23:38<359:35:27,  6.66s/it]
 14%|█▎        | 30603/225000 [69:23:44<344:39:46,  6.38s/it]
 14%|█▎        | 30604/225000 [69:23:51<359:28:55,  6.66s/it]
 14%|█▎        | 30605/225000 [69:23:59<382:32:07,  7.08s/it]
 14%|█▎        | 30606/225000 [69:24:03<325:13:43,  6.02s/it]
 14%|█▎        | 30607/225000 [69:24:08<317:50:19,  5.89s/it]
 14%|█▎        | 30608/225000 [69:24:14<313:47:11,  5.81s/it]
 14%|█▎        | 30609/225000 [69:24:20<322:08:27,  5.97s/it]
 14%|█▎        | 30610/225000 [69:24:28<350:04:16,  6.48s/it]
 14%|█▎        | 30611/225000 [69:24:33<320:31:49,  5.94s/it]
 14%|█▎        | 30612/225000 [69:24:37<299:19:20,  5.54s/it]
 14%|█▎        | 30613/225000 [69:24:50<411:19:44,  7.62s/it]
 14%|█▎        | 30614/225000 [69:24:55<370:23:33,  6.86s/it]
 14%|█▎        | 30615/225000 [69:25:00<344:45:51,  6.39s/it]
 14%|█▎        | 30616/225000 [69:25:07<350:33:32,  6.49s/it]
 14%|█▎        | 30617/225000 [69:25:13<351:59:38,  6.52s/it]
 14%|█▎        | 30618/225000 [69:25:18<317:51:45,  5.89s/it]
 14%|█▎        | 30619/225000 [69:25:23<299:44:25,  5.55s/it]
 14%|█▎        | 30620/225000 [69:25:28<292:56:32,  5.43s/it]
 14%|█▎        | 30621/225000 [69:25:33<293:36:48,  5.44s/it]
 14%|█▎        | 30622/225000 [69:25:40<319:30:04,  5.92s/it]
 14%|█▎        | 30623/225000 [69:25:53<423:54:35,  7.85s/it]
 14%|█▎        | 30624/225000 [69:25:57<361:53:07,  6.70s/it]
 14%|█▎        | 30625/225000 [69:26:04<365:24:06,  6.77s/it]
 14%|█▎        | 30626/225000 [69:26:11<383:56:09,  7.11s/it]
 14%|█▎        | 30627/225000 [69:26:20<404:46:35,  7.50s/it]
 14%|█▎        | 30628/225000 [69:26:24<356:01:58,  6.59s/it]
 14%|█▎        | 30629/225000 [69:26:33<381:03:10,  7.06s/it]
 14%|█▎        | 30630/225000 [69:26:38<353:29:50,  6.55s/it]
 14%|█▎        | 30631/225000 [69:26:43<331:07:17,  6.13s/it]
 14%|█▎        | 30632/225000 [69:26:48<312:07:06,  5.78s/it]
 14%|█▎        | 30633/225000 [69:26:52<288:49:43,  5.35s/it]
 14%|█▎        | 30634/225000 [69:26:57<284:35:02,  5.27s/it]
 14%|█▎        | 30635/225000 [69:27:02<274:36:21,  5.09s/it]
 14%|█▎        | 30636/225000 [69:27:10<323:26:55,  5.99s/it]
 14%|█▎        | 30637/225000 [69:27:23<430:00:03,  7.96s/it]
 14%|█▎        | 30638/225000 [69:27:27<365:49:52,  6.78s/it]
 14%|█▎        | 30639/225000 [69:27:31<330:29:29,  6.12s/it]
 14%|█▎        | 30640/225000 [69:27:37<315:32:39,  5.84s/it]
 14%|█▎        | 30641/225000 [69:27:49<429:37:58,  7.96s/it]
 14%|█▎        | 30642/225000 [69:27:54<380:31:59,  7.05s/it]
 14%|█▎        | 30643/225000 [69:27:59<342:55:09,  6.35s/it]
 14%|█▎        | 30644/225000 [69:28:10<421:39:41,  7.81s/it]
 14%|█▎        | 30645/225000 [69:28:14<358:15:28,  6.64s/it]
 14%|█▎        | 30646/225000 [69:28:23<398:35:31,  7.38s/it]
 14%|█▎        | 30647/225000 [69:28:29<365:14:32,  6.77s/it]
 14%|█▎        | 30648/225000 [69:28:38<404:47:55,  7.50s/it]
 14%|█▎        | 30649/225000 [69:28:41<341:28:13,  6.33s/it]
 14%|█▎        | 30650/225000 [69:28:48<345:52:36,  6.41s/it]
                                                             
{'loss': '0.7226', 'grad_norm': '0.7033', 'learning_rate': '4.887e-05', 'epoch': '0.1362', 'num_input_tokens_seen': 118832152, 'train_runtime': '2.501e+05', 'train_tokens_per_second': '475.1'}

 14%|█▎        | 30650/225000 [69:28:48<345:52:36,  6.41s/it]
 14%|█▎        | 30651/225000 [69:28:54<332:05:29,  6.15s/it]
 14%|█▎        | 30652/225000 [69:28:58<307:01:14,  5.69s/it]
 14%|█▎        | 30653/225000 [69:29:04<312:34:09,  5.79s/it]
 14%|█▎        | 30654/225000 [69:29:10<305:29:11,  5.66s/it]
 14%|█▎        | 30655/225000 [69:29:14<288:49:50,  5.35s/it]
 14%|█▎        | 30656/225000 [69:29:21<305:22:05,  5.66s/it]
 14%|█▎        | 30657/225000 [69:29:33<418:04:43,  7.74s/it]
 14%|█▎        | 30658/225000 [69:29:38<364:20:30,  6.75s/it]
 14%|█▎        | 30659/225000 [69:29:45<366:42:36,  6.79s/it]
 14%|█▎        | 30660/225000 [69:29:49<329:30:56,  6.10s/it]
 14%|█▎        | 30661/225000 [69:29:54<318:29:27,  5.90s/it]
 14%|█▎        | 30662/225000 [69:30:07<418:43:44,  7.76s/it]
 14%|█▎        | 30663/225000 [69:30:11<360:43:43,  6.68s/it]
 14%|█▎        | 30664/225000 [69:30:19<378:56:03,  7.02s/it]
 14%|█▎        | 30665/225000 [69:30:24<346:46:58,  6.42s/it]
 14%|█▎        | 30666/225000 [69:30:29<324:57:25,  6.02s/it]
 14%|█▎        | 30667/225000 [69:30:33<301:35:57,  5.59s/it]
 14%|█▎        | 30668/225000 [69:30:40<322:07:54,  5.97s/it]
 14%|█▎        | 30669/225000 [69:30:53<433:06:44,  8.02s/it]
 14%|█▎        | 30670/225000 [69:30:58<392:12:06,  7.27s/it]
 14%|█▎        | 30671/225000 [69:31:03<353:17:08,  6.54s/it]
 14%|█▎        | 30672/225000 [69:31:08<332:21:15,  6.16s/it]
 14%|█▎        | 30673/225000 [69:31:14<314:59:42,  5.84s/it]
 14%|█▎        | 30674/225000 [69:31:19<305:05:51,  5.65s/it]
 14%|█▎        | 30675/225000 [69:31:24<302:30:45,  5.60s/it]
 14%|█▎        | 30676/225000 [69:31:35<379:59:52,  7.04s/it]
 14%|█▎        | 30677/225000 [69:31:44<412:31:17,  7.64s/it]
 14%|█▎        | 30678/225000 [69:31:49<375:49:35,  6.96s/it]
 14%|█▎        | 30679/225000 [69:31:54<347:34:45,  6.44s/it]
 14%|█▎        | 30680/225000 [69:32:00<338:25:25,  6.27s/it]
 14%|█▎        | 30681/225000 [69:32:06<322:39:00,  5.98s/it]
 14%|█▎        | 30682/225000 [69:32:11<315:46:45,  5.85s/it]
 14%|█▎        | 30683/225000 [69:32:16<308:00:04,  5.71s/it]
 14%|█▎        | 30684/225000 [69:32:22<304:05:22,  5.63s/it]
 14%|█▎        | 30685/225000 [69:32:33<385:02:37,  7.13s/it]
 14%|█▎        | 30686/225000 [69:32:43<432:02:25,  8.00s/it]
 14%|█▎        | 30687/225000 [69:32:46<362:51:43,  6.72s/it]
 14%|█▎        | 30688/225000 [69:32:57<420:20:44,  7.79s/it]
 14%|█▎        | 30689/225000 [69:33:03<398:20:54,  7.38s/it]
 14%|█▎        | 30690/225000 [69:33:07<338:27:39,  6.27s/it]
 14%|█▎        | 30691/225000 [69:33:16<391:45:03,  7.26s/it]
 14%|█▎        | 30692/225000 [69:33:23<376:52:08,  6.98s/it]
 14%|█▎        | 30693/225000 [69:33:27<336:32:52,  6.24s/it]
 14%|█▎        | 30694/225000 [69:33:32<312:12:11,  5.78s/it]
 14%|█▎        | 30695/225000 [69:33:37<301:59:39,  5.60s/it]
 14%|█▎        | 30696/225000 [69:33:49<409:56:21,  7.60s/it]
 14%|█▎        | 30697/225000 [69:33:53<350:48:09,  6.50s/it]
 14%|█▎        | 30698/225000 [69:33:57<301:32:24,  5.59s/it]
 14%|█▎        | 30699/225000 [69:34:05<352:42:22,  6.53s/it]
 14%|█▎        | 30700/225000 [69:34:13<366:29:44,  6.79s/it]
                                                             
{'loss': '0.7106', 'grad_norm': '0.6295', 'learning_rate': '4.887e-05', 'epoch': '0.1364', 'num_input_tokens_seen': 118976296, 'train_runtime': '2.505e+05', 'train_tokens_per_second': '475'}

 14%|█▎        | 30700/225000 [69:34:13<366:29:44,  6.79s/it]
 14%|█▎        | 30701/225000 [69:34:17<325:01:26,  6.02s/it]
 14%|█▎        | 30702/225000 [69:34:23<322:12:52,  5.97s/it]
 14%|█▎        | 30703/225000 [69:34:28<311:42:13,  5.78s/it]
 14%|█▎        | 30704/225000 [69:34:33<300:25:48,  5.57s/it]
 14%|█▎        | 30705/225000 [69:34:38<295:12:15,  5.47s/it]
 14%|█▎        | 30706/225000 [69:34:44<289:56:14,  5.37s/it]
 14%|█▎        | 30707/225000 [69:34:56<401:17:38,  7.44s/it]
 14%|█▎        | 30708/225000 [69:35:03<399:32:24,  7.40s/it]
 14%|█▎        | 30709/225000 [69:35:11<406:02:02,  7.52s/it]
 14%|█▎        | 30710/225000 [69:35:17<380:35:38,  7.05s/it]
 14%|█▎        | 30711/225000 [69:35:21<333:24:47,  6.18s/it]
 14%|█▎        | 30712/225000 [69:35:27<329:48:08,  6.11s/it]
 14%|█▎        | 30713/225000 [69:35:37<388:58:50,  7.21s/it]
 14%|█▎        | 30714/225000 [69:35:41<347:36:14,  6.44s/it]
 14%|█▎        | 30715/225000 [69:35:46<320:00:32,  5.93s/it]
 14%|█▎        | 30716/225000 [69:35:51<294:30:52,  5.46s/it]
 14%|█▎        | 30717/225000 [69:36:01<376:53:07,  6.98s/it]
 14%|█▎        | 30718/225000 [69:36:07<365:01:35,  6.76s/it]
 14%|█▎        | 30719/225000 [69:36:12<335:46:41,  6.22s/it]
 14%|█▎        | 30720/225000 [69:36:17<319:03:27,  5.91s/it]
 14%|█▎        | 30721/225000 [69:36:23<309:24:15,  5.73s/it]
 14%|█▎        | 30722/225000 [69:36:28<297:16:36,  5.51s/it]
 14%|█▎        | 30723/225000 [69:36:33<290:29:28,  5.38s/it]
 14%|█▎        | 30724/225000 [69:36:38<291:45:04,  5.41s/it]
 14%|█▎        | 30725/225000 [69:36:45<315:09:17,  5.84s/it]
 14%|█▎        | 30726/225000 [69:36:58<435:29:13,  8.07s/it]
 14%|█▎        | 30727/225000 [69:37:07<436:40:57,  8.09s/it]
 14%|█▎        | 30728/225000 [69:37:12<395:05:14,  7.32s/it]
 14%|█▎        | 30729/225000 [69:37:19<392:15:11,  7.27s/it]
 14%|█▎        | 30730/225000 [69:37:26<389:44:42,  7.22s/it]
 14%|█▎        | 30731/225000 [69:37:34<399:30:25,  7.40s/it]
 14%|█▎        | 30732/225000 [69:37:38<343:17:35,  6.36s/it]
 14%|█▎        | 30733/225000 [69:37:42<299:38:09,  5.55s/it]
 14%|█▎        | 30734/225000 [69:37:52<371:05:37,  6.88s/it]
 14%|█▎        | 30735/225000 [69:37:56<322:41:47,  5.98s/it]
 14%|█▎        | 30736/225000 [69:38:00<293:36:45,  5.44s/it]
 14%|█▎        | 30737/225000 [69:38:07<317:53:35,  5.89s/it]
 14%|█▎        | 30738/225000 [69:38:15<361:51:10,  6.71s/it]
 14%|█▎        | 30739/225000 [69:38:25<408:54:55,  7.58s/it]
 14%|█▎        | 30740/225000 [69:38:28<336:32:27,  6.24s/it]
 14%|█▎        | 30741/225000 [69:38:36<370:05:38,  6.86s/it]
 14%|█▎        | 30742/225000 [69:38:40<320:54:54,  5.95s/it]
 14%|█▎        | 30743/225000 [69:38:48<348:54:12,  6.47s/it]
 14%|█▎        | 30744/225000 [69:38:54<347:36:59,  6.44s/it]
 14%|█▎        | 30745/225000 [69:39:01<355:21:49,  6.59s/it]
 14%|█▎        | 30746/225000 [69:39:09<370:28:17,  6.87s/it]
 14%|█▎        | 30747/225000 [69:39:13<327:09:54,  6.06s/it]
 14%|█▎        | 30748/225000 [69:39:18<314:38:15,  5.83s/it]
 14%|█▎        | 30749/225000 [69:39:28<378:24:39,  7.01s/it]
 14%|█▎        | 30750/225000 [69:39:32<334:57:37,  6.21s/it]
                                                             
{'loss': '0.7096', 'grad_norm': '0.7445', 'learning_rate': '4.886e-05', 'epoch': '0.1367', 'num_input_tokens_seen': 119118736, 'train_runtime': '2.508e+05', 'train_tokens_per_second': '475'}

 14%|█▎        | 30750/225000 [69:39:32<334:57:37,  6.21s/it]
 14%|█▎        | 30751/225000 [69:39:37<315:26:55,  5.85s/it]
 14%|█▎        | 30752/225000 [69:39:43<307:10:03,  5.69s/it]
 14%|█▎        | 30753/225000 [69:39:51<347:58:46,  6.45s/it]
 14%|█▎        | 30754/225000 [69:39:59<375:41:12,  6.96s/it]
 14%|█▎        | 30755/225000 [69:40:04<338:11:11,  6.27s/it]
 14%|█▎        | 30756/225000 [69:40:09<318:51:20,  5.91s/it]
 14%|█▎        | 30757/225000 [69:40:13<299:08:09,  5.54s/it]
 14%|█▎        | 30758/225000 [69:40:27<435:30:04,  8.07s/it]
 14%|█▎        | 30759/225000 [69:40:32<377:45:25,  7.00s/it]
 14%|█▎        | 30760/225000 [69:40:39<380:09:14,  7.05s/it]
 14%|█▎        | 30761/225000 [69:40:43<331:47:02,  6.15s/it]
 14%|█▎        | 30762/225000 [69:40:47<300:49:08,  5.58s/it]
 14%|█▎        | 30763/225000 [69:40:55<329:37:19,  6.11s/it]
 14%|█▎        | 30764/225000 [69:41:05<393:05:32,  7.29s/it]
 14%|█▎        | 30765/225000 [69:41:09<337:28:24,  6.25s/it]
 14%|█▎        | 30766/225000 [69:41:17<372:18:00,  6.90s/it]
 14%|█▎        | 30767/225000 [69:41:23<349:22:13,  6.48s/it]
 14%|█▎        | 30768/225000 [69:41:28<334:44:58,  6.20s/it]
 14%|█▎        | 30769/225000 [69:41:33<313:38:39,  5.81s/it]
 14%|█▎        | 30770/225000 [69:41:39<312:08:43,  5.79s/it]
 14%|█▎        | 30771/225000 [69:41:44<307:39:02,  5.70s/it]
 14%|█▎        | 30772/225000 [69:41:55<397:10:56,  7.36s/it]
 14%|█▎        | 30773/225000 [69:42:03<407:39:42,  7.56s/it]
 14%|█▎        | 30774/225000 [69:42:08<359:40:52,  6.67s/it]
 14%|█▎        | 30775/225000 [69:42:15<366:26:12,  6.79s/it]
 14%|█▎        | 30776/225000 [69:42:24<400:40:04,  7.43s/it]
 14%|█▎        | 30777/225000 [69:42:29<354:12:32,  6.57s/it]
 14%|█▎        | 30778/225000 [69:42:34<331:50:14,  6.15s/it]
 14%|█▎        | 30779/225000 [69:42:39<316:45:21,  5.87s/it]
 14%|█▎        | 30780/225000 [69:42:44<303:46:04,  5.63s/it]
 14%|█▎        | 30781/225000 [69:42:50<301:34:25,  5.59s/it]
 14%|█▎        | 30782/225000 [69:43:01<390:36:16,  7.24s/it]
 14%|█▎        | 30783/225000 [69:43:04<332:53:22,  6.17s/it]
 14%|█▎        | 30784/225000 [69:43:14<391:30:11,  7.26s/it]
 14%|█▎        | 30785/225000 [69:43:18<329:14:07,  6.10s/it]
 14%|█▎        | 30786/225000 [69:43:22<297:22:15,  5.51s/it]
 14%|█▎        | 30787/225000 [69:43:27<295:23:16,  5.48s/it]
 14%|█▎        | 30788/225000 [69:43:32<288:17:30,  5.34s/it]
 14%|█▎        | 30789/225000 [69:43:41<346:01:31,  6.41s/it]
 14%|█▎        | 30790/225000 [69:43:53<438:23:27,  8.13s/it]
 14%|█▎        | 30791/225000 [69:43:58<385:10:37,  7.14s/it]
 14%|█▎        | 30792/225000 [69:44:04<368:45:28,  6.84s/it]
 14%|█▎        | 30793/225000 [69:44:09<337:40:10,  6.26s/it]
 14%|█▎        | 30794/225000 [69:44:14<313:59:50,  5.82s/it]
 14%|█▎        | 30795/225000 [69:44:19<298:53:04,  5.54s/it]
 14%|█▎        | 30796/225000 [69:44:26<320:40:22,  5.94s/it]
 14%|█▎        | 30797/225000 [69:44:38<429:07:59,  7.95s/it]
 14%|█▎        | 30798/225000 [69:44:42<369:12:38,  6.84s/it]
 14%|█▎        | 30799/225000 [69:44:47<333:57:30,  6.19s/it]
 14%|█▎        | 30800/225000 [69:44:52<305:50:14,  5.67s/it]
                                                             
{'loss': '0.7144', 'grad_norm': '0.7958', 'learning_rate': '4.886e-05', 'epoch': '0.1369', 'num_input_tokens_seen': 119263072, 'train_runtime': '2.511e+05', 'train_tokens_per_second': '475'}

 14%|█▎        | 30800/225000 [69:44:52<305:50:14,  5.67s/it]
 14%|█▎        | 30801/225000 [69:44:56<292:40:19,  5.43s/it]
 14%|█▎        | 30802/225000 [69:45:07<373:45:22,  6.93s/it]
 14%|█▎        | 30803/225000 [69:45:15<390:13:02,  7.23s/it]
 14%|█▎        | 30804/225000 [69:45:22<391:53:50,  7.26s/it]
 14%|█▎        | 30805/225000 [69:45:26<331:58:19,  6.15s/it]
 14%|█▎        | 30806/225000 [69:45:35<375:59:52,  6.97s/it]
 14%|█▎        | 30807/225000 [69:45:38<324:20:49,  6.01s/it]
 14%|█▎        | 30808/225000 [69:45:48<376:36:45,  6.98s/it]
 14%|█▎        | 30809/225000 [69:45:52<334:22:51,  6.20s/it]
 14%|█▎        | 30810/225000 [69:45:57<311:38:22,  5.78s/it]
 14%|█▎        | 30811/225000 [69:46:01<292:24:32,  5.42s/it]
 14%|█▎        | 30812/225000 [69:46:11<353:07:37,  6.55s/it]
 14%|█▎        | 30813/225000 [69:46:18<365:09:13,  6.77s/it]
 14%|█▎        | 30814/225000 [69:46:21<314:30:42,  5.83s/it]
 14%|█▎        | 30815/225000 [69:46:31<382:16:05,  7.09s/it]
 14%|█▎        | 30816/225000 [69:46:36<347:14:46,  6.44s/it]
 14%|█▎        | 30817/225000 [69:46:41<311:47:06,  5.78s/it]
 14%|█▎        | 30818/225000 [69:46:52<405:36:16,  7.52s/it]
 14%|█▎        | 30819/225000 [69:46:58<382:57:03,  7.10s/it]
 14%|█▎        | 30820/225000 [69:47:02<332:43:18,  6.17s/it]
 14%|█▎        | 30821/225000 [69:47:11<371:46:12,  6.89s/it]
 14%|█▎        | 30822/225000 [69:47:17<355:34:22,  6.59s/it]
 14%|█▎        | 30823/225000 [69:47:25<375:18:19,  6.96s/it]
 14%|█▎        | 30824/225000 [69:47:30<346:52:52,  6.43s/it]
 14%|█▎        | 30825/225000 [69:47:39<384:10:17,  7.12s/it]
 14%|█▎        | 30826/225000 [69:47:44<355:00:54,  6.58s/it]
 14%|█▎        | 30827/225000 [69:47:49<329:56:28,  6.12s/it]
 14%|█▎        | 30828/225000 [69:47:54<314:16:33,  5.83s/it]
 14%|█▎        | 30829/225000 [69:47:59<304:57:49,  5.65s/it]
 14%|█▎        | 30830/225000 [69:48:04<289:14:07,  5.36s/it]
 14%|█▎        | 30831/225000 [69:48:09<290:07:32,  5.38s/it]
 14%|█▎        | 30832/225000 [69:48:15<291:44:16,  5.41s/it]
 14%|█▎        | 30833/225000 [69:48:20<290:29:14,  5.39s/it]
 14%|█▎        | 30834/225000 [69:48:30<368:39:18,  6.84s/it]
 14%|█▎        | 30835/225000 [69:48:41<434:06:10,  8.05s/it]
 14%|█▎        | 30836/225000 [69:48:45<362:18:00,  6.72s/it]
 14%|█▎        | 30837/225000 [69:48:55<414:47:53,  7.69s/it]
 14%|█▎        | 30838/225000 [69:49:01<381:50:16,  7.08s/it]
 14%|█▎        | 30839/225000 [69:49:06<349:16:18,  6.48s/it]
 14%|█▎        | 30840/225000 [69:49:11<323:26:54,  6.00s/it]
 14%|█▎        | 30841/225000 [69:49:20<385:35:38,  7.15s/it]
 14%|█▎        | 30842/225000 [69:49:25<343:51:44,  6.38s/it]
 14%|█▎        | 30843/225000 [69:49:30<318:52:00,  5.91s/it]
 14%|█▎        | 30844/225000 [69:49:41<412:14:42,  7.64s/it]
 14%|█▎        | 30845/225000 [69:49:45<348:17:30,  6.46s/it]
 14%|█▎        | 30846/225000 [69:49:53<371:50:49,  6.89s/it]
 14%|█▎        | 30847/225000 [69:50:00<373:37:17,  6.93s/it]
 14%|█▎        | 30848/225000 [69:50:06<364:39:58,  6.76s/it]
 14%|█▎        | 30849/225000 [69:50:11<321:52:36,  5.97s/it]
 14%|█▎        | 30850/225000 [69:50:18<346:02:42,  6.42s/it]
                                                             
{'loss': '0.7102', 'grad_norm': '0.6395', 'learning_rate': '4.885e-05', 'epoch': '0.1371', 'num_input_tokens_seen': 119406976, 'train_runtime': '2.514e+05', 'train_tokens_per_second': '474.9'}

 14%|█▎        | 30850/225000 [69:50:18<346:02:42,  6.42s/it]
 14%|█▎        | 30851/225000 [69:50:28<397:54:59,  7.38s/it]
 14%|█▎        | 30852/225000 [69:50:31<336:07:02,  6.23s/it]
 14%|█▎        | 30853/225000 [69:50:36<316:27:31,  5.87s/it]
 14%|█▎        | 30854/225000 [69:50:44<346:41:16,  6.43s/it]
 14%|█▎        | 30855/225000 [69:50:51<358:19:06,  6.64s/it]
 14%|█▎        | 30856/225000 [69:50:56<337:58:51,  6.27s/it]
 14%|█▎        | 30857/225000 [69:51:02<332:58:15,  6.17s/it]
 14%|█▎        | 30858/225000 [69:51:10<363:38:44,  6.74s/it]
 14%|█▎        | 30859/225000 [69:51:19<397:49:02,  7.38s/it]
 14%|█▎        | 30860/225000 [69:51:25<364:31:50,  6.76s/it]
 14%|█▎        | 30861/225000 [69:51:30<338:33:08,  6.28s/it]
 14%|█▎        | 30862/225000 [69:51:35<323:23:56,  6.00s/it]
 14%|█▎        | 30863/225000 [69:51:40<310:33:45,  5.76s/it]
 14%|█▎        | 30864/225000 [69:51:45<299:15:03,  5.55s/it]
 14%|█▎        | 30865/225000 [69:51:52<323:00:53,  5.99s/it]
 14%|█▎        | 30866/225000 [69:52:05<435:43:54,  8.08s/it]
 14%|█▎        | 30867/225000 [69:52:10<380:10:48,  7.05s/it]
 14%|█▎        | 30868/225000 [69:52:18<401:02:42,  7.44s/it]
 14%|█▎        | 30869/225000 [69:52:24<366:27:17,  6.80s/it]
 14%|█▎        | 30870/225000 [69:52:28<332:42:47,  6.17s/it]
 14%|█▎        | 30871/225000 [69:52:33<301:54:57,  5.60s/it]
 14%|█▎        | 30872/225000 [69:52:42<364:57:11,  6.77s/it]
 14%|█▎        | 30873/225000 [69:52:49<364:57:33,  6.77s/it]
 14%|█▎        | 30874/225000 [69:52:53<316:32:05,  5.87s/it]
 14%|█▎        | 30875/225000 [69:53:03<380:31:49,  7.06s/it]
 14%|█▎        | 30876/225000 [69:53:06<325:51:18,  6.04s/it]
 14%|█▎        | 30877/225000 [69:53:13<340:29:00,  6.31s/it]
 14%|█▎        | 30878/225000 [69:53:20<349:38:47,  6.48s/it]
 14%|█▎        | 30879/225000 [69:53:24<307:08:22,  5.70s/it]
 14%|█▎        | 30880/225000 [69:53:28<286:40:15,  5.32s/it]
 14%|█▎        | 30881/225000 [69:53:38<357:05:42,  6.62s/it]
 14%|█▎        | 30882/225000 [69:53:46<378:25:57,  7.02s/it]
 14%|█▎        | 30883/225000 [69:53:51<351:27:20,  6.52s/it]
 14%|█▎        | 30884/225000 [69:53:57<334:46:17,  6.21s/it]
 14%|█▎        | 30885/225000 [69:54:02<317:00:04,  5.88s/it]
 14%|█▎        | 30886/225000 [69:54:07<306:21:03,  5.68s/it]
 14%|█▎        | 30887/225000 [69:54:13<308:55:17,  5.73s/it]
 14%|█▎        | 30888/225000 [69:54:26<423:51:59,  7.86s/it]
 14%|█▎        | 30889/225000 [69:54:31<382:19:40,  7.09s/it]
 14%|█▎        | 30890/225000 [69:54:38<372:52:40,  6.92s/it]
 14%|█▎        | 30891/225000 [69:54:44<371:00:55,  6.88s/it]
 14%|█▎        | 30892/225000 [69:54:48<324:25:05,  6.02s/it]
 14%|█▎        | 30893/225000 [69:54:55<331:32:01,  6.15s/it]
 14%|█▎        | 30894/225000 [69:55:05<399:52:23,  7.42s/it]
 14%|█▎        | 30895/225000 [69:55:10<363:37:45,  6.74s/it]
 14%|█▎        | 30896/225000 [69:55:16<344:58:19,  6.40s/it]
 14%|█▎        | 30897/225000 [69:55:20<308:57:48,  5.73s/it]
 14%|█▎        | 30898/225000 [69:55:25<297:45:03,  5.52s/it]
 14%|█▎        | 30899/225000 [69:55:32<313:59:09,  5.82s/it]
 14%|█▎        | 30900/225000 [69:55:45<430:51:40,  7.99s/it]
                                                             
{'loss': '0.7132', 'grad_norm': '0.8025', 'learning_rate': '4.884e-05', 'epoch': '0.1373', 'num_input_tokens_seen': 119551224, 'train_runtime': '2.517e+05', 'train_tokens_per_second': '474.9'}

 14%|█▎        | 30900/225000 [69:55:45<430:51:40,  7.99s/it]
 14%|█▎        | 30901/225000 [69:55:48<361:30:15,  6.70s/it]
 14%|█▎        | 30902/225000 [69:55:57<387:28:40,  7.19s/it]
 14%|█▎        | 30903/225000 [69:56:01<333:29:10,  6.19s/it]
 14%|█▎        | 30904/225000 [69:56:05<300:44:14,  5.58s/it]
 14%|█▎        | 30905/225000 [69:56:12<321:30:21,  5.96s/it]
 14%|█▎        | 30906/225000 [69:56:21<368:14:31,  6.83s/it]
 14%|█▎        | 30907/225000 [69:56:28<384:32:10,  7.13s/it]
 14%|█▎        | 30908/225000 [69:56:32<335:39:10,  6.23s/it]
 14%|█▎        | 30909/225000 [69:56:38<331:47:50,  6.15s/it]
 14%|█▎        | 30910/225000 [69:56:43<307:03:11,  5.70s/it]
 14%|█▎        | 30911/225000 [69:56:48<296:23:31,  5.50s/it]
 14%|█▎        | 30912/225000 [69:56:53<291:02:44,  5.40s/it]
 14%|█▎        | 30913/225000 [69:57:00<304:34:49,  5.65s/it]
 14%|█▎        | 30914/225000 [69:57:12<422:42:50,  7.84s/it]
 14%|█▎        | 30915/225000 [69:57:17<366:06:45,  6.79s/it]
 14%|█▎        | 30916/225000 [69:57:27<417:30:45,  7.74s/it]
 14%|█▎        | 30917/225000 [69:57:31<361:06:54,  6.70s/it]
 14%|█▎        | 30918/225000 [69:57:36<340:09:30,  6.31s/it]
 14%|█▎        | 30919/225000 [69:57:41<319:54:48,  5.93s/it]
 14%|█▎        | 30920/225000 [69:57:46<303:00:36,  5.62s/it]
 14%|█▎        | 30921/225000 [69:57:52<297:28:08,  5.52s/it]
 14%|█▎        | 30922/225000 [69:58:03<387:41:47,  7.19s/it]
 14%|█▎        | 30923/225000 [69:58:12<422:57:42,  7.85s/it]
 14%|█▎        | 30924/225000 [69:58:17<373:41:09,  6.93s/it]
 14%|█▎        | 30925/225000 [69:58:26<403:48:00,  7.49s/it]
 14%|█▎        | 30926/225000 [69:58:32<384:30:26,  7.13s/it]
 14%|█▎        | 30927/225000 [69:58:38<361:42:10,  6.71s/it]
 14%|█▎        | 30928/225000 [69:58:43<344:08:33,  6.38s/it]
 14%|█▎        | 30929/225000 [69:58:48<323:09:09,  5.99s/it]
 14%|█▎        | 30930/225000 [69:58:54<320:50:05,  5.95s/it]
 14%|█▎        | 30931/225000 [69:59:00<308:53:12,  5.73s/it]
 14%|█▎        | 30932/225000 [69:59:05<300:15:25,  5.57s/it]
 14%|█▎        | 30933/225000 [69:59:10<293:14:51,  5.44s/it]
 14%|█▎        | 30934/225000 [69:59:15<291:25:04,  5.41s/it]
 14%|█▎        | 30935/225000 [69:59:21<294:31:03,  5.46s/it]
 14%|█▎        | 30936/225000 [69:59:34<418:32:34,  7.76s/it]
 14%|█▎        | 30937/225000 [69:59:42<426:21:18,  7.91s/it]
 14%|█▍        | 30938/225000 [69:59:50<426:30:55,  7.91s/it]
 14%|█▍        | 30939/225000 [69:59:57<404:51:38,  7.51s/it]
 14%|█▍        | 30940/225000 [70:00:01<356:09:30,  6.61s/it]
 14%|█▍        | 30941/225000 [70:00:06<326:42:41,  6.06s/it]
 14%|█▍        | 30942/225000 [70:00:11<304:40:18,  5.65s/it]
 14%|█▍        | 30943/225000 [70:00:21<376:39:36,  6.99s/it]
 14%|█▍        | 30944/225000 [70:00:28<388:15:55,  7.20s/it]
 14%|█▍        | 30945/225000 [70:00:32<327:19:41,  6.07s/it]
 14%|█▍        | 30946/225000 [70:00:37<317:53:55,  5.90s/it]
 14%|█▍        | 30947/225000 [70:00:48<391:09:26,  7.26s/it]
 14%|█▍        | 30948/225000 [70:00:52<338:20:26,  6.28s/it]
 14%|█▍        | 30949/225000 [70:01:01<378:47:43,  7.03s/it]
 14%|█▍        | 30950/225000 [70:01:04<326:04:24,  6.05s/it]
                                                             
{'loss': '0.7182', 'grad_norm': '0.6577', 'learning_rate': '4.884e-05', 'epoch': '0.1376', 'num_input_tokens_seen': 119694488, 'train_runtime': '2.521e+05', 'train_tokens_per_second': '474.9'}

 14%|█▍        | 30950/225000 [70:01:04<326:04:24,  6.05s/it]
 14%|█▍        | 30951/225000 [70:01:09<297:01:43,  5.51s/it]
 14%|█▍        | 30952/225000 [70:01:13<286:22:01,  5.31s/it]
 14%|█▍        | 30953/225000 [70:01:26<396:34:58,  7.36s/it]
 14%|█▍        | 30954/225000 [70:01:31<365:53:50,  6.79s/it]
 14%|█▍        | 30955/225000 [70:01:37<357:29:14,  6.63s/it]
 14%|█▍        | 30956/225000 [70:01:44<358:32:45,  6.65s/it]
 14%|█▍        | 30957/225000 [70:01:50<349:43:36,  6.49s/it]
 14%|█▍        | 30958/225000 [70:01:55<320:03:32,  5.94s/it]
 14%|█▍        | 30959/225000 [70:02:04<378:28:37,  7.02s/it]
 14%|█▍        | 30960/225000 [70:02:08<327:45:11,  6.08s/it]
 14%|█▍        | 30961/225000 [70:02:18<380:51:02,  7.07s/it]
 14%|█▍        | 30962/225000 [70:02:22<338:28:47,  6.28s/it]
 14%|█▍        | 30963/225000 [70:02:29<345:20:21,  6.41s/it]
 14%|█▍        | 30964/225000 [70:02:37<374:34:54,  6.95s/it]
 14%|█▍        | 30965/225000 [70:02:41<332:20:55,  6.17s/it]
 14%|█▍        | 30966/225000 [70:02:46<306:23:30,  5.68s/it]
 14%|█▍        | 30967/225000 [70:02:51<291:59:16,  5.42s/it]
 14%|█▍        | 30968/225000 [70:03:04<413:11:55,  7.67s/it]
 14%|█▍        | 30969/225000 [70:03:07<349:36:03,  6.49s/it]
 14%|█▍        | 30970/225000 [70:03:11<305:27:01,  5.67s/it]
 14%|█▍        | 30971/225000 [70:03:19<350:33:47,  6.50s/it]
 14%|█▍        | 30972/225000 [70:03:24<316:36:22,  5.87s/it]
 14%|█▍        | 30973/225000 [70:03:34<379:29:05,  7.04s/it]
 14%|█▍        | 30974/225000 [70:03:42<395:36:12,  7.34s/it]
 14%|█▍        | 30975/225000 [70:03:45<338:45:10,  6.29s/it]
 14%|█▍        | 30976/225000 [70:03:49<300:25:33,  5.57s/it]
 14%|█▍        | 30977/225000 [70:04:00<389:08:12,  7.22s/it]
 14%|█▍        | 30978/225000 [70:04:05<352:16:27,  6.54s/it]
 14%|█▍        | 30979/225000 [70:04:10<320:17:37,  5.94s/it]
 14%|█▍        | 30980/225000 [70:04:15<303:31:47,  5.63s/it]
 14%|█▍        | 30981/225000 [70:04:21<318:43:54,  5.91s/it]
 14%|█▍        | 30982/225000 [70:04:34<424:34:31,  7.88s/it]
 14%|█▍        | 30983/225000 [70:04:38<370:23:55,  6.87s/it]
 14%|█▍        | 30984/225000 [70:04:43<331:36:16,  6.15s/it]
 14%|█▍        | 30985/225000 [70:04:53<395:27:07,  7.34s/it]
 14%|█▍        | 30986/225000 [70:04:58<354:16:58,  6.57s/it]
 14%|█▍        | 30987/225000 [70:05:05<357:20:04,  6.63s/it]
 14%|█▍        | 30988/225000 [70:05:08<313:29:49,  5.82s/it]
 14%|█▍        | 30989/225000 [70:05:20<406:28:23,  7.54s/it]
 14%|█▍        | 30990/225000 [70:05:25<368:33:43,  6.84s/it]
 14%|█▍        | 30991/225000 [70:05:30<336:05:21,  6.24s/it]
 14%|█▍        | 30992/225000 [70:05:35<316:45:02,  5.88s/it]
 14%|█▍        | 30993/225000 [70:05:40<300:16:51,  5.57s/it]
 14%|█▍        | 30994/225000 [70:05:45<290:14:18,  5.39s/it]
 14%|█▍        | 30995/225000 [70:05:50<285:20:49,  5.29s/it]
 14%|█▍        | 30996/225000 [70:06:02<401:20:46,  7.45s/it]
 14%|█▍        | 30997/225000 [70:06:10<396:21:48,  7.36s/it]
 14%|█▍        | 30998/225000 [70:06:17<389:49:03,  7.23s/it]
 14%|█▍        | 30999/225000 [70:06:22<353:37:31,  6.56s/it]
 14%|█▍        | 31000/225000 [70:06:27<327:56:17,  6.09s/it]
                                                             
{'loss': '0.7105', 'grad_norm': '0.7509', 'learning_rate': '4.883e-05', 'epoch': '0.1378', 'num_input_tokens_seen': 119836624, 'train_runtime': '2.524e+05', 'train_tokens_per_second': '474.8'}

 14%|█▍        | 31000/225000 [70:06:27<327:56:17,  6.09s/it]
 14%|█▍        | 31001/225000 [70:06:31<307:41:34,  5.71s/it]
 14%|█▍        | 31002/225000 [70:06:38<322:14:06,  5.98s/it]
 14%|█▍        | 31003/225000 [70:06:49<410:29:25,  7.62s/it]
 14%|█▍        | 31004/225000 [70:06:53<346:28:39,  6.43s/it]
 14%|█▍        | 31005/225000 [70:07:01<367:08:17,  6.81s/it]
 14%|█▍        | 31006/225000 [70:07:07<356:20:09,  6.61s/it]
 14%|█▍        | 31007/225000 [70:07:14<361:27:41,  6.71s/it]
 14%|█▍        | 31008/225000 [70:07:24<417:08:50,  7.74s/it]
 14%|█▍        | 31009/225000 [70:07:27<346:58:53,  6.44s/it]
 14%|█▍        | 31010/225000 [70:07:32<320:05:28,  5.94s/it]
 14%|█▍        | 31011/225000 [70:07:42<377:13:00,  7.00s/it]
 14%|█▍        | 31012/225000 [70:07:46<332:33:36,  6.17s/it]
 14%|█▍        | 31013/225000 [70:07:50<304:12:40,  5.65s/it]
 14%|█▍        | 31014/225000 [70:08:01<378:28:47,  7.02s/it]
 14%|█▍        | 31015/225000 [70:08:06<360:04:40,  6.68s/it]
 14%|█▍        | 31016/225000 [70:08:11<325:18:45,  6.04s/it]
 14%|█▍        | 31017/225000 [70:08:16<310:47:56,  5.77s/it]
 14%|█▍        | 31018/225000 [70:08:21<302:05:47,  5.61s/it]
 14%|█▍        | 31019/225000 [70:08:26<292:47:08,  5.43s/it]
 14%|█▍        | 31020/225000 [70:08:31<286:16:21,  5.31s/it]
 14%|█▍        | 31021/225000 [70:08:43<385:10:38,  7.15s/it]
 14%|█▍        | 31022/225000 [70:08:52<409:57:05,  7.61s/it]
 14%|█▍        | 31023/225000 [70:08:56<361:21:54,  6.71s/it]
 14%|█▍        | 31024/225000 [70:09:01<327:42:57,  6.08s/it]
 14%|█▍        | 31025/225000 [70:09:06<312:35:48,  5.80s/it]
 14%|█▍        | 31026/225000 [70:09:11<294:32:23,  5.47s/it]
 14%|█▍        | 31027/225000 [70:09:23<414:48:46,  7.70s/it]
 14%|█▍        | 31028/225000 [70:09:29<376:38:05,  6.99s/it]
 14%|█▍        | 31029/225000 [70:09:34<340:24:48,  6.32s/it]
 14%|█▍        | 31030/225000 [70:09:39<322:48:36,  5.99s/it]
 14%|█▍        | 31031/225000 [70:09:44<304:31:13,  5.65s/it]
 14%|█▍        | 31032/225000 [70:09:57<435:37:57,  8.09s/it]
 14%|█▍        | 31033/225000 [70:10:04<418:40:16,  7.77s/it]
 14%|█▍        | 31034/225000 [70:10:10<377:14:58,  7.00s/it]
 14%|█▍        | 31035/225000 [70:10:15<357:16:21,  6.63s/it]
 14%|█▍        | 31036/225000 [70:10:21<345:28:11,  6.41s/it]
 14%|█▍        | 31037/225000 [70:10:29<366:26:36,  6.80s/it]
 14%|█▍        | 31038/225000 [70:10:33<321:47:28,  5.97s/it]
 14%|█▍        | 31039/225000 [70:10:44<401:36:38,  7.45s/it]
 14%|█▍        | 31040/225000 [70:10:48<347:51:05,  6.46s/it]
 14%|█▍        | 31041/225000 [70:10:57<394:58:14,  7.33s/it]
 14%|█▍        | 31042/225000 [70:11:01<333:04:26,  6.18s/it]
 14%|█▍        | 31043/225000 [70:11:07<334:01:09,  6.20s/it]
 14%|█▍        | 31044/225000 [70:11:15<356:58:39,  6.63s/it]
 14%|█▍        | 31045/225000 [70:11:22<357:38:25,  6.64s/it]
 14%|█▍        | 31046/225000 [70:11:26<325:43:45,  6.05s/it]
 14%|█▍        | 31047/225000 [70:11:31<307:32:08,  5.71s/it]
 14%|█▍        | 31048/225000 [70:11:42<391:23:04,  7.26s/it]
 14%|█▍        | 31049/225000 [70:11:50<397:36:31,  7.38s/it]
 14%|█▍        | 31050/225000 [70:11:54<354:28:49,  6.58s/it]
                                                             
{'loss': '0.723', 'grad_norm': '0.6396', 'learning_rate': '4.883e-05', 'epoch': '0.138', 'num_input_tokens_seen': 119979824, 'train_runtime': '2.527e+05', 'train_tokens_per_second': '474.8'}

 14%|█▍        | 31050/225000 [70:11:54<354:28:49,  6.58s/it]
 14%|█▍        | 31051/225000 [70:11:59<330:04:52,  6.13s/it]
 14%|█▍        | 31052/225000 [70:12:04<307:27:49,  5.71s/it]
 14%|█▍        | 31053/225000 [70:12:09<292:53:40,  5.44s/it]
 14%|█▍        | 31054/225000 [70:12:14<284:02:38,  5.27s/it]
 14%|█▍        | 31055/225000 [70:12:20<291:00:07,  5.40s/it]
 14%|█▍        | 31056/225000 [70:12:31<389:43:20,  7.23s/it]
 14%|█▍        | 31057/225000 [70:12:38<391:57:31,  7.28s/it]
 14%|█▍        | 31058/225000 [70:12:44<367:00:19,  6.81s/it]
 14%|█▍        | 31059/225000 [70:12:53<393:17:26,  7.30s/it]
 14%|█▍        | 31060/225000 [70:12:56<337:13:10,  6.26s/it]
 14%|█▍        | 31061/225000 [70:13:05<374:22:15,  6.95s/it]
 14%|█▍        | 31062/225000 [70:13:10<345:23:24,  6.41s/it]
 14%|█▍        | 31063/225000 [70:13:14<306:29:36,  5.69s/it]
 14%|█▍        | 31064/225000 [70:13:25<387:59:11,  7.20s/it]
 14%|█▍        | 31065/225000 [70:13:30<348:01:59,  6.46s/it]
 14%|█▍        | 31066/225000 [70:13:34<307:28:12,  5.71s/it]
 14%|█▍        | 31067/225000 [70:13:38<294:08:37,  5.46s/it]
 14%|█▍        | 31068/225000 [70:13:50<393:21:26,  7.30s/it]
 14%|█▍        | 31069/225000 [70:13:54<336:13:03,  6.24s/it]
 14%|█▍        | 31070/225000 [70:14:03<381:39:30,  7.08s/it]
 14%|█▍        | 31071/225000 [70:14:10<380:06:13,  7.06s/it]
 14%|█▍        | 31072/225000 [70:14:15<343:28:32,  6.38s/it]
 14%|█▍        | 31073/225000 [70:14:20<320:37:09,  5.95s/it]
 14%|█▍        | 31074/225000 [70:14:25<311:07:33,  5.78s/it]
 14%|█▍        | 31075/225000 [70:14:30<291:27:45,  5.41s/it]
 14%|█▍        | 31076/225000 [70:14:35<289:13:26,  5.37s/it]
 14%|█▍        | 31077/225000 [70:14:40<290:45:06,  5.40s/it]
 14%|█▍        | 31078/225000 [70:14:48<334:19:59,  6.21s/it]
 14%|█▍        | 31079/225000 [70:15:00<417:23:40,  7.75s/it]
 14%|█▍        | 31080/225000 [70:15:07<405:11:18,  7.52s/it]
 14%|█▍        | 31081/225000 [70:15:12<363:40:25,  6.75s/it]
 14%|█▍        | 31082/225000 [70:15:17<333:41:16,  6.19s/it]
 14%|█▍        | 31083/225000 [70:15:21<312:58:07,  5.81s/it]
 14%|█▍        | 31084/225000 [70:15:32<393:26:06,  7.30s/it]
 14%|█▍        | 31085/225000 [70:15:39<386:09:43,  7.17s/it]
 14%|█▍        | 31086/225000 [70:15:43<328:20:22,  6.10s/it]
 14%|█▍        | 31087/225000 [70:15:54<411:12:12,  7.63s/it]
 14%|█▍        | 31088/225000 [70:16:00<383:14:51,  7.12s/it]
 14%|█▍        | 31089/225000 [70:16:04<332:54:08,  6.18s/it]
 14%|█▍        | 31090/225000 [70:16:08<306:38:30,  5.69s/it]
 14%|█▍        | 31091/225000 [70:16:19<378:15:30,  7.02s/it]
 14%|█▍        | 31092/225000 [70:16:27<405:00:05,  7.52s/it]
 14%|█▍        | 31093/225000 [70:16:31<345:36:10,  6.42s/it]
 14%|█▍        | 31094/225000 [70:16:35<310:44:45,  5.77s/it]
 14%|█▍        | 31095/225000 [70:16:40<294:13:53,  5.46s/it]
 14%|█▍        | 31096/225000 [70:16:51<377:24:37,  7.01s/it]
 14%|█▍        | 31097/225000 [70:16:59<400:37:41,  7.44s/it]
 14%|█▍        | 31098/225000 [70:17:04<356:41:14,  6.62s/it]
 14%|█▍        | 31099/225000 [70:17:09<339:17:42,  6.30s/it]
 14%|█▍        | 31100/225000 [70:17:15<325:09:58,  6.04s/it]
                                                             
{'loss': '0.7134', 'grad_norm': '0.6577', 'learning_rate': '4.882e-05', 'epoch': '0.1382', 'num_input_tokens_seen': 120122992, 'train_runtime': '2.53e+05', 'train_tokens_per_second': '474.7'}

 14%|█▍        | 31100/225000 [70:17:15<325:09:58,  6.04s/it]
 14%|█▍        | 31101/225000 [70:17:20<306:42:11,  5.69s/it]
 14%|█▍        | 31102/225000 [70:17:32<407:19:15,  7.56s/it]
 14%|█▍        | 31103/225000 [70:17:41<434:12:43,  8.06s/it]
 14%|█▍        | 31104/225000 [70:17:45<367:02:36,  6.81s/it]
 14%|█▍        | 31105/225000 [70:17:51<352:19:02,  6.54s/it]
 14%|█▍        | 31106/225000 [70:17:59<380:23:41,  7.06s/it]
 14%|█▍        | 31107/225000 [70:18:07<393:56:37,  7.31s/it]
 14%|█▍        | 31108/225000 [70:18:12<354:00:49,  6.57s/it]
 14%|█▍        | 31109/225000 [70:18:16<323:49:40,  6.01s/it]
 14%|█▍        | 31110/225000 [70:18:21<302:19:59,  5.61s/it]
 14%|█▍        | 31111/225000 [70:18:26<294:13:14,  5.46s/it]
 14%|█▍        | 31112/225000 [70:18:32<293:38:21,  5.45s/it]
 14%|█▍        | 31113/225000 [70:18:43<397:21:09,  7.38s/it]
 14%|█▍        | 31114/225000 [70:18:48<358:38:54,  6.66s/it]
 14%|█▍        | 31115/225000 [70:18:53<326:12:21,  6.06s/it]
 14%|█▍        | 31116/225000 [70:18:59<321:34:51,  5.97s/it]
 14%|█▍        | 31117/225000 [70:19:04<312:34:10,  5.80s/it]
 14%|█▍        | 31118/225000 [70:19:14<379:59:23,  7.06s/it]
 14%|█▍        | 31119/225000 [70:19:22<394:52:14,  7.33s/it]
 14%|█▍        | 31120/225000 [70:19:28<363:17:45,  6.75s/it]
 14%|█▍        | 31121/225000 [70:19:33<345:41:43,  6.42s/it]
 14%|█▍        | 31122/225000 [70:19:39<333:41:54,  6.20s/it]
 14%|█▍        | 31123/225000 [70:19:44<318:29:27,  5.91s/it]
 14%|█▍        | 31124/225000 [70:19:49<306:28:50,  5.69s/it]
 14%|█▍        | 31125/225000 [70:20:03<427:13:22,  7.93s/it]
 14%|█▍        | 31126/225000 [70:20:07<374:59:44,  6.96s/it]
 14%|█▍        | 31127/225000 [70:20:15<381:24:02,  7.08s/it]
 14%|█▍        | 31128/225000 [70:20:21<371:35:39,  6.90s/it]
 14%|█▍        | 31129/225000 [70:20:26<334:50:53,  6.22s/it]
 14%|█▍        | 31130/225000 [70:20:30<308:00:27,  5.72s/it]
 14%|█▍        | 31131/225000 [70:20:40<373:46:37,  6.94s/it]
 14%|█▍        | 31132/225000 [70:20:51<433:28:18,  8.05s/it]
 14%|█▍        | 31133/225000 [70:20:54<363:06:58,  6.74s/it]
 14%|█▍        | 31134/225000 [70:21:02<379:44:11,  7.05s/it]
 14%|█▍        | 31135/225000 [70:21:07<344:35:55,  6.40s/it]
 14%|█▍        | 31136/225000 [70:21:14<360:52:59,  6.70s/it]
 14%|█▍        | 31137/225000 [70:21:20<334:45:59,  6.22s/it]
 14%|█▍        | 31138/225000 [70:21:25<326:06:48,  6.06s/it]
 14%|█▍        | 31139/225000 [70:21:31<318:20:31,  5.91s/it]
 14%|█▍        | 31140/225000 [70:21:36<308:54:40,  5.74s/it]
 14%|█▍        | 31141/225000 [70:21:41<301:21:03,  5.60s/it]
 14%|█▍        | 31142/225000 [70:21:47<299:32:37,  5.56s/it]
 14%|█▍        | 31143/225000 [70:21:52<292:41:29,  5.44s/it]
 14%|█▍        | 31144/225000 [70:22:05<420:58:53,  7.82s/it]
 14%|█▍        | 31145/225000 [70:22:12<407:42:41,  7.57s/it]
 14%|█▍        | 31146/225000 [70:22:17<361:07:47,  6.71s/it]
 14%|█▍        | 31147/225000 [70:22:21<324:26:24,  6.03s/it]
 14%|█▍        | 31148/225000 [70:22:26<305:25:57,  5.67s/it]
 14%|█▍        | 31149/225000 [70:22:33<320:49:29,  5.96s/it]
 14%|█▍        | 31150/225000 [70:22:46<441:43:32,  8.20s/it]
                                                             
{'loss': '0.7298', 'grad_norm': '0.6432', 'learning_rate': '4.882e-05', 'epoch': '0.1384', 'num_input_tokens_seen': 120265872, 'train_runtime': '2.534e+05', 'train_tokens_per_second': '474.7'}

 14%|█▍        | 31150/225000 [70:22:46<441:43:32,  8.20s/it]
 14%|█▍        | 31151/225000 [70:22:50<374:12:28,  6.95s/it]
 14%|█▍        | 31152/225000 [70:22:56<344:00:01,  6.39s/it]
 14%|█▍        | 31153/225000 [70:23:00<320:07:35,  5.95s/it]
 14%|█▍        | 31154/225000 [70:23:06<306:29:40,  5.69s/it]
 14%|█▍        | 31155/225000 [70:23:10<291:39:51,  5.42s/it]
 14%|█▍        | 31156/225000 [70:23:24<425:18:13,  7.90s/it]
 14%|█▍        | 31157/225000 [70:23:28<361:59:17,  6.72s/it]
 14%|█▍        | 31158/225000 [70:23:32<317:31:34,  5.90s/it]
 14%|█▍        | 31159/225000 [70:23:42<381:17:47,  7.08s/it]
 14%|█▍        | 31160/225000 [70:23:48<374:51:45,  6.96s/it]
 14%|█▍        | 31161/225000 [70:23:56<386:37:07,  7.18s/it]
 14%|█▍        | 31162/225000 [70:24:01<342:11:30,  6.36s/it]
 14%|█▍        | 31163/225000 [70:24:05<307:14:25,  5.71s/it]
 14%|█▍        | 31164/225000 [70:24:16<398:18:14,  7.40s/it]
 14%|█▍        | 31165/225000 [70:24:24<399:30:13,  7.42s/it]
 14%|█▍        | 31166/225000 [70:24:27<341:52:00,  6.35s/it]
 14%|█▍        | 31167/225000 [70:24:36<381:26:44,  7.08s/it]
 14%|█▍        | 31168/225000 [70:24:40<331:18:54,  6.15s/it]
 14%|█▍        | 31169/225000 [70:24:45<303:26:05,  5.64s/it]
 14%|█▍        | 31170/225000 [70:24:53<341:40:35,  6.35s/it]
 14%|█▍        | 31171/225000 [70:25:03<404:08:58,  7.51s/it]
 14%|█▍        | 31172/225000 [70:25:07<350:08:42,  6.50s/it]
 14%|█▍        | 31173/225000 [70:25:12<332:21:22,  6.17s/it]
 14%|█▍        | 31174/225000 [70:25:18<325:16:20,  6.04s/it]
 14%|█▍        | 31175/225000 [70:25:24<320:49:22,  5.96s/it]
 14%|█▍        | 31176/225000 [70:25:29<309:31:11,  5.75s/it]
 14%|█▍        | 31177/225000 [70:25:35<303:01:18,  5.63s/it]
 14%|█▍        | 31178/225000 [70:25:41<320:06:15,  5.95s/it]
 14%|█▍        | 31179/225000 [70:25:53<418:31:00,  7.77s/it]
 14%|█▍        | 31180/225000 [70:25:58<373:11:03,  6.93s/it]
 14%|█▍        | 31181/225000 [70:26:04<352:30:20,  6.55s/it]
 14%|█▍        | 31182/225000 [70:26:09<325:04:33,  6.04s/it]
 14%|█▍        | 31183/225000 [70:26:14<311:14:57,  5.78s/it]
 14%|█▍        | 31184/225000 [70:26:19<302:20:16,  5.62s/it]
 14%|█▍        | 31185/225000 [70:26:31<405:55:30,  7.54s/it]
 14%|█▍        | 31186/225000 [70:26:38<388:51:56,  7.22s/it]
 14%|█▍        | 31187/225000 [70:26:46<404:14:04,  7.51s/it]
 14%|█▍        | 31188/225000 [70:26:50<353:12:09,  6.56s/it]
 14%|█▍        | 31189/225000 [70:27:00<403:34:05,  7.50s/it]
 14%|█▍        | 31190/225000 [70:27:06<375:22:25,  6.97s/it]
 14%|█▍        | 31191/225000 [70:27:13<386:06:24,  7.17s/it]
 14%|█▍        | 31192/225000 [70:27:21<388:48:41,  7.22s/it]
 14%|█▍        | 31193/225000 [70:27:25<339:32:44,  6.31s/it]
 14%|█▍        | 31194/225000 [70:27:30<321:47:17,  5.98s/it]
 14%|█▍        | 31195/225000 [70:27:34<295:16:12,  5.48s/it]
 14%|█▍        | 31196/225000 [70:27:45<376:09:28,  6.99s/it]
 14%|█▍        | 31197/225000 [70:27:52<387:05:01,  7.19s/it]
 14%|█▍        | 31198/225000 [70:27:57<341:16:47,  6.34s/it]
 14%|█▍        | 31199/225000 [70:28:02<324:47:31,  6.03s/it]
 14%|█▍        | 31200/225000 [70:28:12<387:23:42,  7.20s/it]
                                                             
{'loss': '0.7027', 'grad_norm': '0.7211', 'learning_rate': '4.881e-05', 'epoch': '0.1387', 'num_input_tokens_seen': 120409952, 'train_runtime': '2.537e+05', 'train_tokens_per_second': '474.6'}

 14%|█▍        | 31200/225000 [70:28:12<387:23:42,  7.20s/it]
 14%|█▍        | 31201/225000 [70:28:16<340:05:27,  6.32s/it]
 14%|█▍        | 31202/225000 [70:28:21<320:39:20,  5.96s/it]
 14%|█▍        | 31203/225000 [70:28:32<400:33:03,  7.44s/it]
 14%|█▍        | 31204/225000 [70:28:36<346:55:04,  6.44s/it]
 14%|█▍        | 31205/225000 [70:28:41<322:59:06,  6.00s/it]
 14%|█▍        | 31206/225000 [70:28:52<389:35:08,  7.24s/it]
 14%|█▍        | 31207/225000 [70:28:55<335:08:11,  6.23s/it]
 14%|█▍        | 31208/225000 [70:29:00<307:57:25,  5.72s/it]
 14%|█▍        | 31209/225000 [70:29:05<296:12:32,  5.50s/it]
 14%|█▍        | 31210/225000 [70:29:18<413:03:59,  7.67s/it]
 14%|█▍        | 31211/225000 [70:29:22<361:33:26,  6.72s/it]
 14%|█▍        | 31212/225000 [70:29:30<379:26:09,  7.05s/it]
 14%|█▍        | 31213/225000 [70:29:34<338:18:36,  6.28s/it]
 14%|█▍        | 31214/225000 [70:29:39<309:21:54,  5.75s/it]
 14%|█▍        | 31215/225000 [70:29:47<345:15:48,  6.41s/it]
 14%|█▍        | 31216/225000 [70:29:57<400:26:05,  7.44s/it]
 14%|█▍        | 31217/225000 [70:30:01<342:14:07,  6.36s/it]
 14%|█▍        | 31218/225000 [70:30:10<384:23:55,  7.14s/it]
 14%|█▍        | 31219/225000 [70:30:17<381:09:32,  7.08s/it]
 14%|█▍        | 31220/225000 [70:30:20<327:33:21,  6.09s/it]
 14%|█▍        | 31221/225000 [70:30:28<354:46:14,  6.59s/it]
 14%|█▍        | 31222/225000 [70:30:33<330:41:08,  6.14s/it]
 14%|█▍        | 31223/225000 [70:30:39<318:22:03,  5.91s/it]
 14%|█▍        | 31224/225000 [70:30:44<311:02:53,  5.78s/it]
 14%|█▍        | 31225/225000 [70:30:51<338:32:27,  6.29s/it]
 14%|█▍        | 31226/225000 [70:31:02<406:27:59,  7.55s/it]
 14%|█▍        | 31227/225000 [70:31:08<375:33:59,  6.98s/it]
 14%|█▍        | 31228/225000 [70:31:15<378:54:50,  7.04s/it]
 14%|█▍        | 31229/225000 [70:31:22<381:29:38,  7.09s/it]
 14%|█▍        | 31230/225000 [70:31:28<361:10:54,  6.71s/it]
 14%|█▍        | 31231/225000 [70:31:34<354:47:19,  6.59s/it]
 14%|█▍        | 31232/225000 [70:31:42<378:25:26,  7.03s/it]
 14%|█▍        | 31233/225000 [70:31:47<349:40:43,  6.50s/it]
 14%|█▍        | 31234/225000 [70:31:55<363:19:58,  6.75s/it]
 14%|█▍        | 31235/225000 [70:31:59<315:34:31,  5.86s/it]
 14%|█▍        | 31236/225000 [70:32:06<338:20:58,  6.29s/it]
 14%|█▍        | 31237/225000 [70:32:13<354:11:41,  6.58s/it]
 14%|█▍        | 31238/225000 [70:32:21<371:42:20,  6.91s/it]
 14%|█▍        | 31239/225000 [70:32:24<319:52:20,  5.94s/it]
 14%|█▍        | 31240/225000 [70:32:34<376:23:22,  6.99s/it]
 14%|█▍        | 31241/225000 [70:32:38<333:26:16,  6.20s/it]
 14%|█▍        | 31242/225000 [70:32:42<299:01:06,  5.56s/it]
 14%|█▍        | 31243/225000 [70:32:55<407:50:59,  7.58s/it]
 14%|█▍        | 31244/225000 [70:32:58<343:53:06,  6.39s/it]
 14%|█▍        | 31245/225000 [70:33:03<317:16:30,  5.90s/it]
 14%|█▍        | 31246/225000 [70:33:10<332:21:52,  6.18s/it]
 14%|█▍        | 31247/225000 [70:33:15<317:51:53,  5.91s/it]
 14%|█▍        | 31248/225000 [70:33:20<299:42:39,  5.57s/it]
 14%|█▍        | 31249/225000 [70:33:25<297:46:40,  5.53s/it]
 14%|█▍        | 31250/225000 [70:33:30<288:19:10,  5.36s/it]
                                                             
{'loss': '0.721', 'grad_norm': '0.816', 'learning_rate': '4.881e-05', 'epoch': '0.1389', 'num_input_tokens_seen': 120553264, 'train_runtime': '2.54e+05', 'train_tokens_per_second': '474.6'}

 14%|█▍        | 31250/225000 [70:33:30<288:19:10,  5.36s/it]
 14%|█▍        | 31251/225000 [70:33:36<289:48:48,  5.38s/it]
 14%|█▍        | 31252/225000 [70:33:41<293:59:54,  5.46s/it]
 14%|█▍        | 31253/225000 [70:33:54<405:29:17,  7.53s/it]
 14%|█▍        | 31254/225000 [70:34:02<414:06:40,  7.69s/it]
 14%|█▍        | 31255/225000 [70:34:06<350:27:19,  6.51s/it]
 14%|█▍        | 31256/225000 [70:34:11<327:43:52,  6.09s/it]
 14%|█▍        | 31257/225000 [70:34:23<422:12:41,  7.85s/it]
 14%|█▍        | 31258/225000 [70:34:26<347:22:58,  6.45s/it]
 14%|█▍        | 31259/225000 [70:34:35<393:05:38,  7.30s/it]
 14%|█▍        | 31260/225000 [70:34:39<337:37:16,  6.27s/it]
 14%|█▍        | 31261/225000 [70:34:44<314:15:42,  5.84s/it]
 14%|█▍        | 31262/225000 [70:34:54<384:13:19,  7.14s/it]
 14%|█▍        | 31263/225000 [70:35:01<380:47:32,  7.08s/it]
 14%|█▍        | 31264/225000 [70:35:04<323:22:39,  6.01s/it]
 14%|█▍        | 31265/225000 [70:35:13<359:55:25,  6.69s/it]
 14%|█▍        | 31266/225000 [70:35:18<331:45:26,  6.16s/it]
 14%|█▍        | 31267/225000 [70:35:23<319:32:44,  5.94s/it]
 14%|█▍        | 31268/225000 [70:35:28<309:10:20,  5.75s/it]
 14%|█▍        | 31269/225000 [70:35:34<309:39:14,  5.75s/it]
 14%|█▍        | 31270/225000 [70:35:39<299:38:06,  5.57s/it]
 14%|█▍        | 31271/225000 [70:35:44<292:27:48,  5.43s/it]
 14%|█▍        | 31272/225000 [70:35:57<408:52:37,  7.60s/it]
 14%|█▍        | 31273/225000 [70:36:03<387:20:27,  7.20s/it]
 14%|█▍        | 31274/225000 [70:36:11<396:46:57,  7.37s/it]
 14%|█▍        | 31275/225000 [70:36:18<383:34:37,  7.13s/it]
 14%|█▍        | 31276/225000 [70:36:24<375:07:07,  6.97s/it]
 14%|█▍        | 31277/225000 [70:36:31<367:20:38,  6.83s/it]
 14%|█▍        | 31278/225000 [70:36:35<321:18:06,  5.97s/it]
 14%|█▍        | 31279/225000 [70:36:44<372:14:50,  6.92s/it]
 14%|█▍        | 31280/225000 [70:36:47<317:59:33,  5.91s/it]
 14%|█▍        | 31281/225000 [70:36:58<387:12:24,  7.20s/it]
 14%|█▍        | 31282/225000 [70:37:02<335:49:30,  6.24s/it]
 14%|█▍        | 31283/225000 [70:37:07<316:08:06,  5.87s/it]
 14%|█▍        | 31284/225000 [70:37:12<303:07:04,  5.63s/it]
 14%|█▍        | 31285/225000 [70:37:16<289:05:37,  5.37s/it]
 14%|█▍        | 31286/225000 [70:37:21<281:54:40,  5.24s/it]
 14%|█▍        | 31287/225000 [70:37:34<395:13:04,  7.34s/it]
 14%|█▍        | 31288/225000 [70:37:40<383:50:52,  7.13s/it]
 14%|█▍        | 31289/225000 [70:37:47<378:28:35,  7.03s/it]
 14%|█▍        | 31290/225000 [70:37:51<330:43:31,  6.15s/it]
 14%|█▍        | 31291/225000 [70:37:55<301:28:32,  5.60s/it]
 14%|█▍        | 31292/225000 [70:38:08<416:42:11,  7.74s/it]
 14%|█▍        | 31293/225000 [70:38:13<371:10:25,  6.90s/it]
 14%|█▍        | 31294/225000 [70:38:17<330:30:49,  6.14s/it]
 14%|█▍        | 31295/225000 [70:38:22<309:50:28,  5.76s/it]
 14%|█▍        | 31296/225000 [70:38:33<388:42:23,  7.22s/it]
 14%|█▍        | 31297/225000 [70:38:39<375:29:39,  6.98s/it]
 14%|█▍        | 31298/225000 [70:38:43<327:51:00,  6.09s/it]
 14%|█▍        | 31299/225000 [70:38:52<370:15:38,  6.88s/it]
 14%|█▍        | 31300/225000 [70:38:57<341:22:46,  6.34s/it]
                                                             
{'loss': '0.7054', 'grad_norm': '0.7686', 'learning_rate': '4.88e-05', 'epoch': '0.1391', 'num_input_tokens_seen': 120696880, 'train_runtime': '2.543e+05', 'train_tokens_per_second': '474.6'}

 14%|█▍        | 31300/225000 [70:38:57<341:22:46,  6.34s/it]
 14%|█▍        | 31301/225000 [70:39:05<369:42:06,  6.87s/it]
 14%|█▍        | 31302/225000 [70:39:11<351:34:54,  6.53s/it]
 14%|█▍        | 31303/225000 [70:39:15<312:20:31,  5.81s/it]
 14%|█▍        | 31304/225000 [70:39:25<384:11:12,  7.14s/it]
 14%|█▍        | 31305/225000 [70:39:29<327:05:58,  6.08s/it]
 14%|█▍        | 31306/225000 [70:39:34<315:44:17,  5.87s/it]
 14%|█▍        | 31307/225000 [70:39:40<313:24:50,  5.83s/it]
 14%|█▍        | 31308/225000 [70:39:45<302:23:31,  5.62s/it]
 14%|█▍        | 31309/225000 [70:39:51<299:37:43,  5.57s/it]
 14%|█▍        | 31310/225000 [70:40:04<419:09:11,  7.79s/it]
 14%|█▍        | 31311/225000 [70:40:09<379:29:11,  7.05s/it]
 14%|█▍        | 31312/225000 [70:40:13<329:40:32,  6.13s/it]
 14%|█▍        | 31313/225000 [70:40:24<411:01:57,  7.64s/it]
 14%|█▍        | 31314/225000 [70:40:30<373:46:59,  6.95s/it]
 14%|█▍        | 31315/225000 [70:40:35<345:01:04,  6.41s/it]
 14%|█▍        | 31316/225000 [70:40:40<320:11:05,  5.95s/it]
 14%|█▍        | 31317/225000 [70:40:45<316:11:37,  5.88s/it]
 14%|█▍        | 31318/225000 [70:40:51<309:37:50,  5.76s/it]
 14%|█▍        | 31319/225000 [70:40:57<310:29:49,  5.77s/it]
 14%|█▍        | 31320/225000 [70:41:02<309:05:12,  5.75s/it]
 14%|█▍        | 31321/225000 [70:41:08<306:08:43,  5.69s/it]
 14%|█▍        | 31322/225000 [70:41:18<374:34:29,  6.96s/it]
 14%|█▍        | 31323/225000 [70:41:30<468:13:56,  8.70s/it]
 14%|█▍        | 31324/225000 [70:41:34<386:42:54,  7.19s/it]
 14%|█▍        | 31325/225000 [70:41:38<330:13:30,  6.14s/it]
 14%|█▍        | 31326/225000 [70:41:48<394:20:28,  7.33s/it]
 14%|█▍        | 31327/225000 [70:41:53<362:55:04,  6.75s/it]
 14%|█▍        | 31328/225000 [70:42:01<379:44:22,  7.06s/it]
 14%|█▍        | 31329/225000 [70:42:09<395:49:02,  7.36s/it]
 14%|█▍        | 31330/225000 [70:42:13<340:52:06,  6.34s/it]
 14%|█▍        | 31331/225000 [70:42:23<397:53:02,  7.40s/it]
 14%|█▍        | 31332/225000 [70:42:27<340:53:14,  6.34s/it]
 14%|█▍        | 31333/225000 [70:42:32<315:09:54,  5.86s/it]
 14%|█▍        | 31334/225000 [70:42:41<367:34:07,  6.83s/it]
 14%|█▍        | 31335/225000 [70:42:49<387:35:16,  7.20s/it]
 14%|█▍        | 31336/225000 [70:42:53<342:21:15,  6.36s/it]
 14%|█▍        | 31337/225000 [70:43:01<372:15:19,  6.92s/it]
 14%|█▍        | 31338/225000 [70:43:08<368:16:30,  6.85s/it]
 14%|█▍        | 31339/225000 [70:43:16<382:42:13,  7.11s/it]
 14%|█▍        | 31340/225000 [70:43:21<349:25:23,  6.50s/it]
 14%|█▍        | 31341/225000 [70:43:29<377:56:50,  7.03s/it]
 14%|█▍        | 31342/225000 [70:43:33<332:03:06,  6.17s/it]
 14%|█▍        | 31343/225000 [70:43:37<298:59:38,  5.56s/it]
 14%|█▍        | 31344/225000 [70:43:48<385:43:31,  7.17s/it]
 14%|█▍        | 31345/225000 [70:43:53<341:28:10,  6.35s/it]
 14%|█▍        | 31346/225000 [70:44:01<370:54:07,  6.90s/it]
 14%|█▍        | 31347/225000 [70:44:05<323:19:47,  6.01s/it]
 14%|█▍        | 31348/225000 [70:44:16<400:09:24,  7.44s/it]
 14%|█▍        | 31349/225000 [70:44:20<343:51:07,  6.39s/it]
 14%|█▍        | 31350/225000 [70:44:28<376:09:54,  6.99s/it]
                                                             
{'loss': '0.7024', 'grad_norm': '0.709', 'learning_rate': '4.88e-05', 'epoch': '0.1393', 'num_input_tokens_seen': 120841320, 'train_runtime': '2.547e+05', 'train_tokens_per_second': '474.5'}

 14%|█▍        | 31350/225000 [70:44:28<376:09:54,  6.99s/it]
 14%|█▍        | 31351/225000 [70:44:34<354:23:17,  6.59s/it]
 14%|█▍        | 31352/225000 [70:44:38<319:52:54,  5.95s/it]
 14%|█▍        | 31353/225000 [70:44:43<306:13:08,  5.69s/it]
 14%|█▍        | 31354/225000 [70:44:49<304:34:31,  5.66s/it]
 14%|█▍        | 31355/225000 [70:44:58<362:20:07,  6.74s/it]
 14%|█▍        | 31356/225000 [70:45:04<354:03:53,  6.58s/it]
 14%|█▍        | 31357/225000 [70:45:10<345:25:01,  6.42s/it]
 14%|█▍        | 31358/225000 [70:45:18<362:32:35,  6.74s/it]
 14%|█▍        | 31359/225000 [70:45:23<342:37:15,  6.37s/it]
 14%|█▍        | 31360/225000 [70:45:31<364:29:08,  6.78s/it]
 14%|█▍        | 31361/225000 [70:45:39<382:36:34,  7.11s/it]
 14%|█▍        | 31362/225000 [70:45:43<328:17:41,  6.10s/it]
 14%|█▍        | 31363/225000 [70:45:53<396:50:37,  7.38s/it]
 14%|█▍        | 31364/225000 [70:45:58<352:36:28,  6.56s/it]
 14%|█▍        | 31365/225000 [70:46:03<325:25:12,  6.05s/it]
 14%|█▍        | 31366/225000 [70:46:09<327:32:08,  6.09s/it]
 14%|█▍        | 31367/225000 [70:46:20<406:26:58,  7.56s/it]
 14%|█▍        | 31368/225000 [70:46:23<340:54:55,  6.34s/it]
 14%|█▍        | 31369/225000 [70:46:30<343:50:44,  6.39s/it]
 14%|█▍        | 31370/225000 [70:46:40<401:12:32,  7.46s/it]
 14%|█▍        | 31371/225000 [70:46:44<345:35:24,  6.43s/it]
 14%|█▍        | 31372/225000 [70:46:48<309:41:37,  5.76s/it]
 14%|█▍        | 31373/225000 [70:46:59<391:45:02,  7.28s/it]
 14%|█▍        | 31374/225000 [70:47:02<333:09:11,  6.19s/it]
 14%|█▍        | 31375/225000 [70:47:11<371:58:44,  6.92s/it]
 14%|█▍        | 31376/225000 [70:47:17<361:00:00,  6.71s/it]
 14%|█▍        | 31377/225000 [70:47:22<330:44:44,  6.15s/it]
 14%|█▍        | 31378/225000 [70:47:27<307:43:40,  5.72s/it]
 14%|█▍        | 31379/225000 [70:47:32<298:21:24,  5.55s/it]
 14%|█▍        | 31380/225000 [70:47:37<294:23:10,  5.47s/it]
 14%|█▍        | 31381/225000 [70:47:42<284:06:30,  5.28s/it]
 14%|█▍        | 31382/225000 [70:47:50<333:02:18,  6.19s/it]
 14%|█▍        | 31383/225000 [70:48:03<438:47:04,  8.16s/it]
 14%|█▍        | 31384/225000 [70:48:07<374:30:57,  6.96s/it]
 14%|█▍        | 31385/225000 [70:48:14<366:19:17,  6.81s/it]
 14%|█▍        | 31386/225000 [70:48:18<330:55:17,  6.15s/it]
 14%|█▍        | 31387/225000 [70:48:27<376:39:01,  7.00s/it]
 14%|█▍        | 31388/225000 [70:48:36<401:07:31,  7.46s/it]
 14%|█▍        | 31389/225000 [70:48:40<350:37:17,  6.52s/it]
 14%|█▍        | 31390/225000 [70:48:49<389:56:44,  7.25s/it]
 14%|█▍        | 31391/225000 [70:48:53<342:26:46,  6.37s/it]
 14%|█▍        | 31392/225000 [70:48:58<316:42:26,  5.89s/it]
 14%|█▍        | 31393/225000 [70:49:09<399:23:14,  7.43s/it]
 14%|█▍        | 31394/225000 [70:49:13<343:19:31,  6.38s/it]
 14%|█▍        | 31395/225000 [70:49:18<314:00:30,  5.84s/it]
 14%|█▍        | 31396/225000 [70:49:23<299:33:23,  5.57s/it]
 14%|█▍        | 31397/225000 [70:49:28<297:09:00,  5.53s/it]
 14%|█▍        | 31398/225000 [70:49:34<309:21:17,  5.75s/it]
 14%|█▍        | 31399/225000 [70:49:46<410:21:10,  7.63s/it]
 14%|█▍        | 31400/225000 [70:49:50<348:46:58,  6.49s/it]
                                                             
{'loss': '0.7114', 'grad_norm': '0.6302', 'learning_rate': '4.879e-05', 'epoch': '0.1396', 'num_input_tokens_seen': 120985064, 'train_runtime': '2.55e+05', 'train_tokens_per_second': '474.5'}

 14%|█▍        | 31400/225000 [70:49:50<348:46:58,  6.49s/it]
 14%|█▍        | 31401/225000 [70:49:54<309:42:25,  5.76s/it]
 14%|█▍        | 31402/225000 [70:50:05<391:10:52,  7.27s/it]
 14%|█▍        | 31403/225000 [70:50:09<336:27:42,  6.26s/it]
 14%|█▍        | 31404/225000 [70:50:14<319:31:39,  5.94s/it]
 14%|█▍        | 31405/225000 [70:50:23<360:18:37,  6.70s/it]
 14%|█▍        | 31406/225000 [70:50:27<328:07:06,  6.10s/it]
 14%|█▍        | 31407/225000 [70:50:34<338:37:27,  6.30s/it]
 14%|█▍        | 31408/225000 [70:50:39<321:11:34,  5.97s/it]
 14%|█▍        | 31409/225000 [70:50:47<348:01:34,  6.47s/it]
 14%|█▍        | 31410/225000 [70:50:52<327:57:44,  6.10s/it]
 14%|█▍        | 31411/225000 [70:51:01<368:12:20,  6.85s/it]
 14%|█▍        | 31412/225000 [70:51:04<310:56:59,  5.78s/it]
 14%|█▍        | 31413/225000 [70:51:08<283:58:56,  5.28s/it]
 14%|█▍        | 31414/225000 [70:51:20<381:10:25,  7.09s/it]
 14%|█▍        | 31415/225000 [70:51:24<337:08:37,  6.27s/it]
 14%|█▍        | 31416/225000 [70:51:28<308:29:00,  5.74s/it]
 14%|█▍        | 31417/225000 [70:51:33<290:05:29,  5.39s/it]
 14%|█▍        | 31418/225000 [70:51:38<282:00:13,  5.24s/it]
 14%|█▍        | 31419/225000 [70:51:45<308:07:20,  5.73s/it]
 14%|█▍        | 31420/225000 [70:51:57<410:36:03,  7.64s/it]
 14%|█▍        | 31421/225000 [70:52:00<342:28:19,  6.37s/it]
 14%|█▍        | 31422/225000 [70:52:05<309:58:45,  5.76s/it]
 14%|█▍        | 31423/225000 [70:52:15<392:20:09,  7.30s/it]
 14%|█▍        | 31424/225000 [70:52:21<365:03:33,  6.79s/it]
 14%|█▍        | 31425/225000 [70:52:27<352:29:28,  6.56s/it]
 14%|█▍        | 31426/225000 [70:52:35<380:51:36,  7.08s/it]
 14%|█▍        | 31427/225000 [70:52:41<361:10:33,  6.72s/it]
 14%|█▍        | 31428/225000 [70:52:46<330:26:00,  6.15s/it]
 14%|█▍        | 31429/225000 [70:52:50<302:15:07,  5.62s/it]
 14%|█▍        | 31430/225000 [70:53:02<397:24:21,  7.39s/it]
 14%|█▍        | 31431/225000 [70:53:08<373:47:54,  6.95s/it]
 14%|█▍        | 31432/225000 [70:53:12<333:04:09,  6.19s/it]
 14%|█▍        | 31433/225000 [70:53:17<300:38:15,  5.59s/it]
 14%|█▍        | 31434/225000 [70:53:27<381:39:30,  7.10s/it]
 14%|█▍        | 31435/225000 [70:53:32<349:36:44,  6.50s/it]
 14%|█▍        | 31436/225000 [70:53:37<317:52:18,  5.91s/it]
 14%|█▍        | 31437/225000 [70:53:46<378:53:06,  7.05s/it]
 14%|█▍        | 31438/225000 [70:53:51<332:56:28,  6.19s/it]
 14%|█▍        | 31439/225000 [70:53:58<345:36:00,  6.43s/it]
 14%|█▍        | 31440/225000 [70:54:03<322:42:24,  6.00s/it]
 14%|█▍        | 31441/225000 [70:54:08<314:26:03,  5.85s/it]
 14%|█▍        | 31442/225000 [70:54:18<374:07:04,  6.96s/it]
 14%|█▍        | 31443/225000 [70:54:21<318:49:37,  5.93s/it]
 14%|█▍        | 31444/225000 [70:54:30<360:12:26,  6.70s/it]
 14%|█▍        | 31445/225000 [70:54:35<339:34:05,  6.32s/it]
 14%|█▍        | 31446/225000 [70:54:40<318:21:38,  5.92s/it]
 14%|█▍        | 31447/225000 [70:54:48<342:22:31,  6.37s/it]
 14%|█▍        | 31448/225000 [70:54:52<315:25:42,  5.87s/it]
 14%|█▍        | 31449/225000 [70:55:00<346:40:48,  6.45s/it]
 14%|█▍        | 31450/225000 [70:55:04<302:44:21,  5.63s/it]
                                                             
{'loss': '0.734', 'grad_norm': '0.761', 'learning_rate': '4.878e-05', 'epoch': '0.1398', 'num_input_tokens_seen': 121127520, 'train_runtime': '2.553e+05', 'train_tokens_per_second': '474.4'}

 14%|█▍        | 31450/225000 [70:55:04<302:44:21,  5.63s/it]
 14%|█▍        | 31451/225000 [70:55:15<393:35:07,  7.32s/it]
 14%|█▍        | 31452/225000 [70:55:19<333:57:08,  6.21s/it]
 14%|█▍        | 31453/225000 [70:55:24<324:35:28,  6.04s/it]
 14%|█▍        | 31454/225000 [70:55:30<314:28:24,  5.85s/it]
 14%|█▍        | 31455/225000 [70:55:36<316:45:51,  5.89s/it]
 14%|█▍        | 31456/225000 [70:55:41<313:50:16,  5.84s/it]
 14%|█▍        | 31457/225000 [70:55:47<310:50:14,  5.78s/it]
 14%|█▍        | 31458/225000 [70:55:53<307:45:22,  5.72s/it]
 14%|█▍        | 31459/225000 [70:56:03<382:18:17,  7.11s/it]
 14%|█▍        | 31460/225000 [70:56:10<375:33:48,  6.99s/it]
 14%|█▍        | 31461/225000 [70:56:14<326:31:13,  6.07s/it]
 14%|█▍        | 31462/225000 [70:56:24<397:43:25,  7.40s/it]
 14%|█▍        | 31463/225000 [70:56:30<376:33:32,  7.00s/it]
 14%|█▍        | 31464/225000 [70:56:34<331:08:21,  6.16s/it]
 14%|█▍        | 31465/225000 [70:56:44<383:18:26,  7.13s/it]
 14%|█▍        | 31466/225000 [70:56:49<351:27:58,  6.54s/it]
 14%|█▍        | 31467/225000 [70:56:54<329:27:39,  6.13s/it]
 14%|█▍        | 31468/225000 [70:56:59<314:16:34,  5.85s/it]
 14%|█▍        | 31469/225000 [70:57:04<294:31:06,  5.48s/it]
 14%|█▍        | 31470/225000 [70:57:09<292:52:14,  5.45s/it]
 14%|█▍        | 31471/225000 [70:57:14<285:21:46,  5.31s/it]
 14%|█▍        | 31472/225000 [70:57:28<419:10:51,  7.80s/it]
 14%|█▍        | 31473/225000 [70:57:33<370:25:58,  6.89s/it]
 14%|█▍        | 31474/225000 [70:57:39<365:19:41,  6.80s/it]
 14%|█▍        | 31475/225000 [70:57:46<368:26:18,  6.85s/it]
 14%|█▍        | 31476/225000 [70:57:51<337:14:41,  6.27s/it]
 14%|█▍        | 31477/225000 [70:57:56<311:26:25,  5.79s/it]
 14%|█▍        | 31478/225000 [70:58:01<297:17:29,  5.53s/it]
 14%|█▍        | 31479/225000 [70:58:12<381:56:39,  7.11s/it]
 14%|█▍        | 31480/225000 [70:58:19<380:53:16,  7.09s/it]
 14%|█▍        | 31481/225000 [70:58:24<359:28:44,  6.69s/it]
 14%|█▍        | 31482/225000 [70:58:31<365:08:00,  6.79s/it]
 14%|█▍        | 31483/225000 [70:58:35<321:10:38,  5.97s/it]
 14%|█▍        | 31484/225000 [70:58:44<370:46:36,  6.90s/it]
 14%|█▍        | 31485/225000 [70:58:51<371:26:06,  6.91s/it]
 14%|█▍        | 31486/225000 [70:58:57<345:14:56,  6.42s/it]
 14%|█▍        | 31487/225000 [70:59:01<316:47:07,  5.89s/it]
 14%|█▍        | 31488/225000 [70:59:07<308:26:29,  5.74s/it]
 14%|█▍        | 31489/225000 [70:59:19<407:27:56,  7.58s/it]
 14%|█▍        | 31490/225000 [70:59:24<370:47:19,  6.90s/it]
 14%|█▍        | 31491/225000 [70:59:29<336:52:38,  6.27s/it]
 14%|█▍        | 31492/225000 [70:59:33<312:57:34,  5.82s/it]
 14%|█▍        | 31493/225000 [70:59:39<310:35:03,  5.78s/it]
 14%|█▍        | 31494/225000 [70:59:49<372:03:37,  6.92s/it]
 14%|█▍        | 31495/225000 [70:59:59<420:42:19,  7.83s/it]
 14%|█▍        | 31496/225000 [71:00:02<355:24:02,  6.61s/it]
 14%|█▍        | 31497/225000 [71:00:08<338:29:04,  6.30s/it]
 14%|█▍        | 31498/225000 [71:00:17<383:13:58,  7.13s/it]
 14%|█▍        | 31499/225000 [71:00:22<344:45:13,  6.41s/it]
 14%|█▍        | 31500/225000 [71:00:30<367:05:39,  6.83s/it]
                                                             
{'loss': '0.7276', 'grad_norm': '0.6365', 'learning_rate': '4.878e-05', 'epoch': '0.14', 'num_input_tokens_seen': 121272760, 'train_runtime': '2.556e+05', 'train_tokens_per_second': '474.4'}

 14%|█▍        | 31500/225000 [71:00:30<367:05:39,  6.83s/it]
 14%|█▍        | 31501/225000 [71:00:35<338:56:57,  6.31s/it]
 14%|█▍        | 31502/225000 [71:00:46<410:49:44,  7.64s/it]
 14%|█▍        | 31503/225000 [71:00:49<347:27:46,  6.46s/it]
 14%|█▍        | 31504/225000 [71:00:58<385:12:39,  7.17s/it]
 14%|█▍        | 31505/225000 [71:01:02<334:17:06,  6.22s/it]
 14%|█▍        | 31506/225000 [71:01:06<293:36:22,  5.46s/it]
 14%|█▍        | 31507/225000 [71:01:15<358:56:54,  6.68s/it]
 14%|█▍        | 31508/225000 [71:01:20<333:33:56,  6.21s/it]
 14%|█▍        | 31509/225000 [71:01:26<320:11:50,  5.96s/it]
 14%|█▍        | 31510/225000 [71:01:31<307:46:17,  5.73s/it]
 14%|█▍        | 31511/225000 [71:01:36<290:41:57,  5.41s/it]
 14%|█▍        | 31512/225000 [71:01:41<288:05:17,  5.36s/it]
 14%|█▍        | 31513/225000 [71:01:46<288:55:15,  5.38s/it]
 14%|█▍        | 31514/225000 [71:01:52<301:24:26,  5.61s/it]
 14%|█▍        | 31515/225000 [71:02:03<387:29:55,  7.21s/it]
 14%|█▍        | 31516/225000 [71:02:11<402:34:50,  7.49s/it]
 14%|█▍        | 31517/225000 [71:02:19<409:21:03,  7.62s/it]
 14%|█▍        | 31518/225000 [71:02:25<381:31:02,  7.10s/it]
 14%|█▍        | 31519/225000 [71:02:30<338:53:55,  6.31s/it]
 14%|█▍        | 31520/225000 [71:02:34<309:41:39,  5.76s/it]
 14%|█▍        | 31521/225000 [71:02:39<290:30:07,  5.41s/it]
 14%|█▍        | 31522/225000 [71:02:47<338:11:13,  6.29s/it]
 14%|█▍        | 31523/225000 [71:02:59<420:29:55,  7.82s/it]
 14%|█▍        | 31524/225000 [71:03:03<357:57:50,  6.66s/it]
 14%|█▍        | 31525/225000 [71:03:09<352:18:13,  6.56s/it]
 14%|█▍        | 31526/225000 [71:03:19<405:14:17,  7.54s/it]
 14%|█▍        | 31527/225000 [71:03:23<348:18:54,  6.48s/it]
 14%|█▍        | 31528/225000 [71:03:30<362:19:28,  6.74s/it]
 14%|█▍        | 31529/225000 [71:03:37<367:53:13,  6.85s/it]
 14%|█▍        | 31530/225000 [71:03:41<318:57:12,  5.93s/it]
 14%|█▍        | 31531/225000 [71:03:50<377:44:58,  7.03s/it]
 14%|█▍        | 31532/225000 [71:03:55<334:44:18,  6.23s/it]
 14%|█▍        | 31533/225000 [71:03:59<299:53:21,  5.58s/it]
 14%|█▍        | 31534/225000 [71:04:10<382:32:00,  7.12s/it]
 14%|█▍        | 31535/225000 [71:04:15<359:37:18,  6.69s/it]
 14%|█▍        | 31536/225000 [71:04:22<353:41:38,  6.58s/it]
 14%|█▍        | 31537/225000 [71:04:30<375:53:15,  6.99s/it]
 14%|█▍        | 31538/225000 [71:04:33<321:33:34,  5.98s/it]
 14%|█▍        | 31539/225000 [71:04:41<348:40:08,  6.49s/it]
 14%|█▍        | 31540/225000 [71:04:46<332:10:17,  6.18s/it]
 14%|█▍        | 31541/225000 [71:04:51<303:59:26,  5.66s/it]
 14%|█▍        | 31542/225000 [71:04:56<289:06:09,  5.38s/it]
 14%|█▍        | 31543/225000 [71:05:00<281:11:51,  5.23s/it]
 14%|█▍        | 31544/225000 [71:05:05<274:08:55,  5.10s/it]
 14%|█▍        | 31545/225000 [71:05:14<335:55:19,  6.25s/it]
 14%|█▍        | 31546/225000 [71:05:22<366:55:43,  6.83s/it]
 14%|█▍        | 31547/225000 [71:05:26<321:14:25,  5.98s/it]
 14%|█▍        | 31548/225000 [71:05:37<389:04:13,  7.24s/it]
 14%|█▍        | 31549/225000 [71:05:41<338:29:56,  6.30s/it]
 14%|█▍        | 31550/225000 [71:05:45<307:56:08,  5.73s/it]
                                                             
{'loss': '0.7215', 'grad_norm': '0.6592', 'learning_rate': '4.877e-05', 'epoch': '0.1402', 'num_input_tokens_seen': 121416984, 'train_runtime': '2.559e+05', 'train_tokens_per_second': '474.4'}

 14%|█▍        | 31550/225000 [71:05:45<307:56:08,  5.73s/it]
 14%|█▍        | 31551/225000 [71:05:55<375:28:47,  6.99s/it]
 14%|█▍        | 31552/225000 [71:06:01<353:59:13,  6.59s/it]
 14%|█▍        | 31553/225000 [71:06:07<356:32:34,  6.64s/it]
 14%|█▍        | 31554/225000 [71:06:14<355:53:36,  6.62s/it]
 14%|█▍        | 31555/225000 [71:06:18<310:07:37,  5.77s/it]
 14%|█▍        | 31556/225000 [71:06:26<350:54:17,  6.53s/it]
 14%|█▍        | 31557/225000 [71:06:33<363:05:51,  6.76s/it]
 14%|█▍        | 31558/225000 [71:06:37<317:37:13,  5.91s/it]
 14%|█▍        | 31559/225000 [71:06:46<370:44:30,  6.90s/it]
 14%|█▍        | 31560/225000 [71:06:52<355:56:35,  6.62s/it]
 14%|█▍        | 31561/225000 [71:06:57<330:17:37,  6.15s/it]
 14%|█▍        | 31562/225000 [71:07:06<372:00:11,  6.92s/it]
 14%|█▍        | 31563/225000 [71:07:11<330:45:55,  6.16s/it]
 14%|█▍        | 31564/225000 [71:07:16<315:22:15,  5.87s/it]
 14%|█▍        | 31565/225000 [71:07:22<315:04:02,  5.86s/it]
 14%|█▍        | 31566/225000 [71:07:31<376:38:52,  7.01s/it]
 14%|█▍        | 31567/225000 [71:07:36<336:17:19,  6.26s/it]
 14%|█▍        | 31568/225000 [71:07:40<307:31:26,  5.72s/it]
 14%|█▍        | 31569/225000 [71:07:49<358:04:35,  6.66s/it]
 14%|█▍        | 31570/225000 [71:07:56<360:09:11,  6.70s/it]
 14%|█▍        | 31571/225000 [71:08:00<323:46:24,  6.03s/it]
 14%|█▍        | 31572/225000 [71:08:10<377:58:26,  7.03s/it]
 14%|█▍        | 31573/225000 [71:08:17<387:30:05,  7.21s/it]
 14%|█▍        | 31574/225000 [71:08:25<388:01:16,  7.22s/it]
 14%|█▍        | 31575/225000 [71:08:29<337:16:05,  6.28s/it]
 14%|█▍        | 31576/225000 [71:08:33<303:32:46,  5.65s/it]
 14%|█▍        | 31577/225000 [71:08:43<379:57:39,  7.07s/it]
 14%|█▍        | 31578/225000 [71:08:46<315:57:31,  5.88s/it]
 14%|█▍        | 31579/225000 [71:08:52<305:15:20,  5.68s/it]
 14%|█▍        | 31580/225000 [71:08:57<299:55:15,  5.58s/it]
 14%|█▍        | 31581/225000 [71:09:06<353:07:15,  6.57s/it]
 14%|█▍        | 31582/225000 [71:09:12<351:21:27,  6.54s/it]
 14%|█▍        | 31583/225000 [71:09:17<321:28:30,  5.98s/it]
 14%|█▍        | 31584/225000 [71:09:21<297:36:56,  5.54s/it]
 14%|█▍        | 31585/225000 [71:09:32<377:29:12,  7.03s/it]
 14%|█▍        | 31586/225000 [71:09:37<340:34:03,  6.34s/it]
 14%|█▍        | 31587/225000 [71:09:42<330:08:25,  6.14s/it]
 14%|█▍        | 31588/225000 [71:09:55<431:07:38,  8.02s/it]
 14%|█▍        | 31589/225000 [71:10:00<391:31:48,  7.29s/it]
 14%|█▍        | 31590/225000 [71:10:05<346:52:48,  6.46s/it]
 14%|█▍        | 31591/225000 [71:10:09<314:31:07,  5.85s/it]
 14%|█▍        | 31592/225000 [71:10:14<289:15:17,  5.38s/it]
 14%|█▍        | 31593/225000 [71:10:25<378:30:23,  7.05s/it]
 14%|█▍        | 31594/225000 [71:10:32<389:47:49,  7.26s/it]
 14%|█▍        | 31595/225000 [71:10:36<336:55:10,  6.27s/it]
 14%|█▍        | 31596/225000 [71:10:47<405:32:58,  7.55s/it]
 14%|█▍        | 31597/225000 [71:10:54<394:22:39,  7.34s/it]
 14%|█▍        | 31598/225000 [71:10:58<340:47:26,  6.34s/it]
 14%|█▍        | 31599/225000 [71:11:02<306:21:34,  5.70s/it]
 14%|█▍        | 31600/225000 [71:11:14<407:11:26,  7.58s/it]
                                                             
{'loss': '0.748', 'grad_norm': '0.7019', 'learning_rate': '4.877e-05', 'epoch': '0.1404', 'num_input_tokens_seen': 121561056, 'train_runtime': '2.563e+05', 'train_tokens_per_second': '474.3'}

 14%|█▍        | 31600/225000 [71:11:14<407:11:26,  7.58s/it]
 14%|█▍        | 31601/225000 [71:11:18<349:58:52,  6.51s/it]
 14%|█▍        | 31602/225000 [71:11:22<307:40:03,  5.73s/it]
 14%|█▍        | 31603/225000 [71:11:33<393:02:24,  7.32s/it]
 14%|█▍        | 31604/225000 [71:11:39<370:18:22,  6.89s/it]
 14%|█▍        | 31605/225000 [71:11:44<346:31:13,  6.45s/it]
 14%|█▍        | 31606/225000 [71:11:52<364:44:03,  6.79s/it]
 14%|█▍        | 31607/225000 [71:12:00<381:20:13,  7.10s/it]
 14%|█▍        | 31608/225000 [71:12:03<322:51:06,  6.01s/it]
 14%|█▍        | 31609/225000 [71:12:09<326:42:40,  6.08s/it]
 14%|█▍        | 31610/225000 [71:12:14<298:36:18,  5.56s/it]
 14%|█▍        | 31611/225000 [71:12:23<367:37:41,  6.84s/it]
 14%|█▍        | 31612/225000 [71:12:29<350:42:34,  6.53s/it]
 14%|█▍        | 31613/225000 [71:12:34<319:22:37,  5.95s/it]
 14%|█▍        | 31614/225000 [71:12:38<296:55:18,  5.53s/it]
 14%|█▍        | 31615/225000 [71:12:43<289:09:14,  5.38s/it]
 14%|█▍        | 31616/225000 [71:12:52<334:24:07,  6.23s/it]
 14%|█▍        | 31617/225000 [71:13:01<381:04:32,  7.09s/it]
 14%|█▍        | 31618/225000 [71:13:08<380:10:31,  7.08s/it]
 14%|█▍        | 31619/225000 [71:13:12<334:46:42,  6.23s/it]
 14%|█▍        | 31620/225000 [71:13:17<310:25:28,  5.78s/it]
 14%|█▍        | 31621/225000 [71:13:22<300:00:31,  5.59s/it]
 14%|█▍        | 31622/225000 [71:13:27<297:27:55,  5.54s/it]
 14%|█▍        | 31623/225000 [71:13:39<390:01:06,  7.26s/it]
 14%|█▍        | 31624/225000 [71:13:45<370:02:13,  6.89s/it]
 14%|█▍        | 31625/225000 [71:13:49<322:33:32,  6.00s/it]
 14%|█▍        | 31626/225000 [71:13:59<386:55:06,  7.20s/it]
 14%|█▍        | 31627/225000 [71:14:02<330:42:45,  6.16s/it]
 14%|█▍        | 31628/225000 [71:14:11<378:40:14,  7.05s/it]
 14%|█▍        | 31629/225000 [71:14:15<321:31:58,  5.99s/it]
 14%|█▍        | 31630/225000 [71:14:23<362:48:01,  6.75s/it]
 14%|█▍        | 31631/225000 [71:14:28<331:58:21,  6.18s/it]
 14%|█▍        | 31632/225000 [71:14:33<303:02:35,  5.64s/it]
 14%|█▍        | 31633/225000 [71:14:39<315:57:51,  5.88s/it]
 14%|█▍        | 31634/225000 [71:14:49<386:01:05,  7.19s/it]
 14%|█▍        | 31635/225000 [71:14:53<327:45:51,  6.10s/it]
 14%|█▍        | 31636/225000 [71:15:04<411:42:25,  7.67s/it]
 14%|█▍        | 31637/225000 [71:15:09<358:50:54,  6.68s/it]
 14%|█▍        | 31638/225000 [71:15:16<363:50:06,  6.77s/it]
 14%|█▍        | 31639/225000 [71:15:21<340:00:45,  6.33s/it]
 14%|█▍        | 31640/225000 [71:15:25<308:07:26,  5.74s/it]
 14%|█▍        | 31641/225000 [71:15:35<370:35:49,  6.90s/it]
 14%|█▍        | 31642/225000 [71:15:40<346:36:49,  6.45s/it]
 14%|█▍        | 31643/225000 [71:15:49<390:35:29,  7.27s/it]
 14%|█▍        | 31644/225000 [71:15:54<339:26:52,  6.32s/it]
 14%|█▍        | 31645/225000 [71:16:02<380:50:20,  7.09s/it]
 14%|█▍        | 31646/225000 [71:16:07<336:51:34,  6.27s/it]
 14%|█▍        | 31647/225000 [71:16:11<302:28:20,  5.63s/it]
 14%|█▍        | 31648/225000 [71:16:16<287:34:46,  5.35s/it]
 14%|█▍        | 31649/225000 [71:16:21<282:08:09,  5.25s/it]
 14%|█▍        | 31650/225000 [71:16:33<390:34:31,  7.27s/it]
                                                             
{'loss': '0.71', 'grad_norm': '0.6497', 'learning_rate': '4.876e-05', 'epoch': '0.1407', 'num_input_tokens_seen': 121705120, 'train_runtime': '2.566e+05', 'train_tokens_per_second': '474.3'}

 14%|█▍        | 31650/225000 [71:16:33<390:34:31,  7.27s/it]
 14%|█▍        | 31651/225000 [71:16:37<346:51:16,  6.46s/it]
 14%|█▍        | 31652/225000 [71:16:42<314:28:25,  5.86s/it]
 14%|█▍        | 31653/225000 [71:16:47<302:57:57,  5.64s/it]
 14%|█▍        | 31654/225000 [71:16:52<291:05:14,  5.42s/it]
 14%|█▍        | 31655/225000 [71:16:59<317:39:12,  5.91s/it]
 14%|█▍        | 31656/225000 [71:17:11<420:34:33,  7.83s/it]
 14%|█▍        | 31657/225000 [71:17:18<400:23:13,  7.46s/it]
 14%|█▍        | 31658/225000 [71:17:24<389:01:43,  7.24s/it]
 14%|█▍        | 31659/225000 [71:17:31<381:07:58,  7.10s/it]
 14%|█▍        | 31660/225000 [71:17:35<335:55:26,  6.25s/it]
 14%|█▍        | 31661/225000 [71:17:45<388:31:24,  7.23s/it]
 14%|█▍        | 31662/225000 [71:17:49<341:04:37,  6.35s/it]
 14%|█▍        | 31663/225000 [71:17:54<313:10:29,  5.83s/it]
 14%|█▍        | 31664/225000 [71:17:58<289:21:59,  5.39s/it]
 14%|█▍        | 31665/225000 [71:18:10<390:43:36,  7.28s/it]
 14%|█▍        | 31666/225000 [71:18:15<359:35:23,  6.70s/it]
 14%|█▍        | 31667/225000 [71:18:19<318:04:36,  5.92s/it]
 14%|█▍        | 31668/225000 [71:18:25<319:24:13,  5.95s/it]
 14%|█▍        | 31669/225000 [71:18:35<372:01:41,  6.93s/it]
 14%|█▍        | 31670/225000 [71:18:38<317:02:13,  5.90s/it]
 14%|█▍        | 31671/225000 [71:18:43<295:38:50,  5.51s/it]
 14%|█▍        | 31672/225000 [71:18:54<397:37:00,  7.40s/it]
 14%|█▍        | 31673/225000 [71:18:58<331:56:48,  6.18s/it]
 14%|█▍        | 31674/225000 [71:19:06<364:05:52,  6.78s/it]
 14%|█▍        | 31675/225000 [71:19:12<345:07:05,  6.43s/it]
 14%|█▍        | 31676/225000 [71:19:19<352:48:04,  6.57s/it]
 14%|█▍        | 31677/225000 [71:19:23<324:08:19,  6.04s/it]
 14%|█▍        | 31678/225000 [71:19:28<297:46:59,  5.55s/it]
 14%|█▍        | 31679/225000 [71:19:33<290:11:05,  5.40s/it]
 14%|█▍        | 31680/225000 [71:19:39<299:54:28,  5.58s/it]
 14%|█▍        | 31681/225000 [71:19:45<305:42:27,  5.69s/it]
 14%|█▍        | 31682/225000 [71:19:51<310:06:25,  5.77s/it]
 14%|█▍        | 31683/225000 [71:20:05<441:27:23,  8.22s/it]
 14%|█▍        | 31684/225000 [71:20:10<399:57:37,  7.45s/it]
 14%|█▍        | 31685/225000 [71:20:19<418:08:38,  7.79s/it]
 14%|█▍        | 31686/225000 [71:20:24<368:15:41,  6.86s/it]
 14%|█▍        | 31687/225000 [71:20:28<328:07:00,  6.11s/it]
 14%|█▍        | 31688/225000 [71:20:33<311:59:17,  5.81s/it]
 14%|█▍        | 31689/225000 [71:20:37<288:47:11,  5.38s/it]
 14%|█▍        | 31690/225000 [71:20:44<307:52:47,  5.73s/it]
 14%|█▍        | 31691/225000 [71:20:56<406:43:05,  7.57s/it]
 14%|█▍        | 31692/225000 [71:21:01<370:13:38,  6.89s/it]
 14%|█▍        | 31693/225000 [71:21:08<375:49:42,  7.00s/it]
 14%|█▍        | 31694/225000 [71:21:12<322:22:39,  6.00s/it]
 14%|█▍        | 31695/225000 [71:21:18<317:36:21,  5.91s/it]
 14%|█▍        | 31696/225000 [71:21:28<383:05:43,  7.13s/it]
 14%|█▍        | 31697/225000 [71:21:32<340:47:44,  6.35s/it]
 14%|█▍        | 31698/225000 [71:21:37<312:08:22,  5.81s/it]
 14%|█▍        | 31699/225000 [71:21:48<399:22:16,  7.44s/it]
 14%|█▍        | 31700/225000 [71:21:52<340:26:16,  6.34s/it]
                                                             
{'loss': '0.7065', 'grad_norm': '0.6591', 'learning_rate': '4.876e-05', 'epoch': '0.1409', 'num_input_tokens_seen': 121849232, 'train_runtime': '2.569e+05', 'train_tokens_per_second': '474.3'}

 14%|█▍        | 31700/225000 [71:21:52<340:26:16,  6.34s/it]
 14%|█▍        | 31701/225000 [71:21:56<312:27:51,  5.82s/it]
 14%|█▍        | 31702/225000 [71:22:01<288:56:09,  5.38s/it]
 14%|█▍        | 31703/225000 [71:22:11<359:49:37,  6.70s/it]
 14%|█▍        | 31704/225000 [71:22:20<395:58:52,  7.37s/it]
 14%|█▍        | 31705/225000 [71:22:23<330:33:43,  6.16s/it]
 14%|█▍        | 31706/225000 [71:22:31<365:39:31,  6.81s/it]
 14%|█▍        | 31707/225000 [71:22:35<313:27:56,  5.84s/it]
 14%|█▍        | 31708/225000 [71:22:44<364:30:35,  6.79s/it]
 14%|█▍        | 31709/225000 [71:22:51<373:47:06,  6.96s/it]
 14%|█▍        | 31710/225000 [71:22:55<319:32:13,  5.95s/it]
 14%|█▍        | 31711/225000 [71:23:00<311:30:25,  5.80s/it]
 14%|█▍        | 31712/225000 [71:23:06<307:35:01,  5.73s/it]
 14%|█▍        | 31713/225000 [71:23:12<312:40:35,  5.82s/it]
 14%|█▍        | 31714/225000 [71:23:18<313:43:12,  5.84s/it]
 14%|█▍        | 31715/225000 [71:23:23<307:19:48,  5.72s/it]
 14%|█▍        | 31716/225000 [71:23:34<393:45:39,  7.33s/it]
 14%|█▍        | 31717/225000 [71:23:39<349:47:01,  6.51s/it]
 14%|█▍        | 31718/225000 [71:23:43<313:51:56,  5.85s/it]
 14%|█▍        | 31719/225000 [71:23:47<288:54:20,  5.38s/it]
 14%|█▍        | 31720/225000 [71:23:56<335:13:53,  6.24s/it]
 14%|█▍        | 31721/225000 [71:24:05<383:13:49,  7.14s/it]
 14%|█▍        | 31722/225000 [71:24:11<371:11:44,  6.91s/it]
 14%|█▍        | 31723/225000 [71:24:19<389:34:07,  7.26s/it]
 14%|█▍        | 31724/225000 [71:24:28<407:06:16,  7.58s/it]
 14%|█▍        | 31725/225000 [71:24:32<349:49:53,  6.52s/it]
 14%|█▍        | 31726/225000 [71:24:36<308:05:18,  5.74s/it]
 14%|█▍        | 31727/225000 [71:24:40<288:35:53,  5.38s/it]
 14%|█▍        | 31728/225000 [71:24:50<358:13:23,  6.67s/it]
 14%|█▍        | 31729/225000 [71:24:59<394:52:37,  7.36s/it]
 14%|█▍        | 31730/225000 [71:25:05<384:30:05,  7.16s/it]
 14%|█▍        | 31731/225000 [71:25:10<337:32:59,  6.29s/it]
 14%|█▍        | 31732/225000 [71:25:15<325:03:39,  6.05s/it]
 14%|█▍        | 31733/225000 [71:25:25<389:09:45,  7.25s/it]
 14%|█▍        | 31734/225000 [71:25:30<345:53:08,  6.44s/it]
 14%|█▍        | 31735/225000 [71:25:38<379:32:01,  7.07s/it]
 14%|█▍        | 31736/225000 [71:25:46<382:45:06,  7.13s/it]
 14%|█▍        | 31737/225000 [71:25:54<395:25:20,  7.37s/it]
 14%|█▍        | 31738/225000 [71:25:58<345:08:40,  6.43s/it]
 14%|█▍        | 31739/225000 [71:26:02<309:02:11,  5.76s/it]
 14%|█▍        | 31740/225000 [71:26:10<353:34:21,  6.59s/it]
 14%|█▍        | 31741/225000 [71:26:18<362:16:17,  6.75s/it]
 14%|█▍        | 31742/225000 [71:26:24<349:24:07,  6.51s/it]
 14%|█▍        | 31743/225000 [71:26:32<375:15:24,  6.99s/it]
 14%|█▍        | 31744/225000 [71:26:36<324:30:41,  6.05s/it]
 14%|█▍        | 31745/225000 [71:26:45<373:42:25,  6.96s/it]
 14%|█▍        | 31746/225000 [71:26:48<322:28:20,  6.01s/it]
 14%|█▍        | 31747/225000 [71:26:57<364:24:53,  6.79s/it]
 14%|█▍        | 31748/225000 [71:27:01<313:17:22,  5.84s/it]
 14%|█▍        | 31749/225000 [71:27:09<351:08:25,  6.54s/it]
 14%|█▍        | 31750/225000 [71:27:13<306:38:58,  5.71s/it]
                                                             
{'loss': '0.7229', 'grad_norm': '0.7355', 'learning_rate': '4.875e-05', 'epoch': '0.1411', 'num_input_tokens_seen': 121994016, 'train_runtime': '2.572e+05', 'train_tokens_per_second': '474.3'}

 14%|█▍        | 31750/225000 [71:27:13<306:38:58,  5.71s/it]
 14%|█▍        | 31751/225000 [71:27:22<371:33:53,  6.92s/it]
 14%|█▍        | 31752/225000 [71:27:26<316:58:59,  5.91s/it]
 14%|█▍        | 31753/225000 [71:27:31<301:09:17,  5.61s/it]
 14%|█▍        | 31754/225000 [71:27:36<292:29:00,  5.45s/it]
 14%|█▍        | 31755/225000 [71:27:46<365:04:35,  6.80s/it]
 14%|█▍        | 31756/225000 [71:27:54<381:00:53,  7.10s/it]
 14%|█▍        | 31757/225000 [71:28:00<370:42:06,  6.91s/it]
 14%|█▍        | 31758/225000 [71:28:05<333:32:30,  6.21s/it]
 14%|█▍        | 31759/225000 [71:28:10<314:59:34,  5.87s/it]
 14%|█▍        | 31760/225000 [71:28:15<298:44:31,  5.57s/it]
 14%|█▍        | 31761/225000 [71:28:25<371:04:47,  6.91s/it]
 14%|█▍        | 31762/225000 [71:28:29<335:13:32,  6.25s/it]
 14%|█▍        | 31763/225000 [71:28:35<332:57:51,  6.20s/it]
 14%|█▍        | 31764/225000 [71:28:46<403:03:10,  7.51s/it]
 14%|█▍        | 31765/225000 [71:28:51<366:03:23,  6.82s/it]
 14%|█▍        | 31766/225000 [71:28:58<365:48:42,  6.82s/it]
 14%|█▍        | 31767/225000 [71:29:05<374:27:26,  6.98s/it]
 14%|█▍        | 31768/225000 [71:29:11<359:24:50,  6.70s/it]
 14%|█▍        | 31769/225000 [71:29:18<351:37:03,  6.55s/it]
 14%|█▍        | 31770/225000 [71:29:22<316:09:52,  5.89s/it]
 14%|█▍        | 31771/225000 [71:29:32<379:42:17,  7.07s/it]
 14%|█▍        | 31772/225000 [71:29:35<318:14:17,  5.93s/it]
 14%|█▍        | 31773/225000 [71:29:41<321:26:11,  5.99s/it]
 14%|█▍        | 31774/225000 [71:29:49<358:34:23,  6.68s/it]
 14%|█▍        | 31775/225000 [71:29:56<354:01:06,  6.60s/it]
 14%|█▍        | 31776/225000 [71:30:04<382:54:46,  7.13s/it]
 14%|█▍        | 31777/225000 [71:30:09<351:32:52,  6.55s/it]
 14%|█▍        | 31778/225000 [71:30:15<331:59:50,  6.19s/it]
 14%|█▍        | 31779/225000 [71:30:24<376:39:45,  7.02s/it]
 14%|█▍        | 31780/225000 [71:30:28<325:19:27,  6.06s/it]
 14%|█▍        | 31781/225000 [71:30:38<391:04:18,  7.29s/it]
 14%|█▍        | 31782/225000 [71:30:42<335:53:57,  6.26s/it]
 14%|█▍        | 31783/225000 [71:30:46<311:07:38,  5.80s/it]
 14%|█▍        | 31784/225000 [71:30:55<354:00:41,  6.60s/it]
 14%|█▍        | 31785/225000 [71:30:59<309:55:33,  5.77s/it]
 14%|█▍        | 31786/225000 [71:31:04<305:17:44,  5.69s/it]
 14%|█▍        | 31787/225000 [71:31:13<360:42:04,  6.72s/it]
 14%|█▍        | 31788/225000 [71:31:17<319:37:18,  5.96s/it]
 14%|█▍        | 31789/225000 [71:31:23<305:38:35,  5.69s/it]
 14%|█▍        | 31790/225000 [71:31:28<305:12:21,  5.69s/it]
 14%|█▍        | 31791/225000 [71:31:34<306:22:53,  5.71s/it]
 14%|█▍        | 31792/225000 [71:31:39<302:41:42,  5.64s/it]
 14%|█▍        | 31793/225000 [71:31:45<302:41:09,  5.64s/it]
 14%|█▍        | 31794/225000 [71:31:53<335:05:19,  6.24s/it]
 14%|█▍        | 31795/225000 [71:32:04<421:31:38,  7.85s/it]
 14%|█▍        | 31796/225000 [71:32:11<409:19:15,  7.63s/it]
 14%|█▍        | 31797/225000 [71:32:15<344:34:37,  6.42s/it]
 14%|█▍        | 31798/225000 [71:32:24<388:37:04,  7.24s/it]
 14%|█▍        | 31799/225000 [71:32:29<356:42:11,  6.65s/it]
 14%|█▍        | 31800/225000 [71:32:34<326:46:59,  6.09s/it]
                                                             
{'loss': '0.7206', 'grad_norm': '0.6951', 'learning_rate': '4.874e-05', 'epoch': '0.1413', 'num_input_tokens_seen': 122138672, 'train_runtime': '2.576e+05', 'train_tokens_per_second': '474.2'}

 14%|█▍        | 31800/225000 [71:32:34<326:46:59,  6.09s/it]
 14%|█▍        | 31801/225000 [71:32:39<312:04:29,  5.82s/it]
 14%|█▍        | 31802/225000 [71:32:44<295:26:23,  5.51s/it]
 14%|█▍        | 31803/225000 [71:32:53<350:00:47,  6.52s/it]
 14%|█▍        | 31804/225000 [71:33:03<398:54:16,  7.43s/it]
 14%|█▍        | 31805/225000 [71:33:07<349:58:52,  6.52s/it]
 14%|█▍        | 31806/225000 [71:33:12<319:36:52,  5.96s/it]
 14%|█▍        | 31807/225000 [71:33:17<306:04:52,  5.70s/it]
 14%|█▍        | 31808/225000 [71:33:22<298:18:03,  5.56s/it]
 14%|█▍        | 31809/225000 [71:33:32<372:30:37,  6.94s/it]
 14%|█▍        | 31810/225000 [71:33:41<407:09:19,  7.59s/it]
 14%|█▍        | 31811/225000 [71:33:46<360:36:23,  6.72s/it]
 14%|█▍        | 31812/225000 [71:33:50<320:04:47,  5.96s/it]
 14%|█▍        | 31813/225000 [71:33:58<346:47:28,  6.46s/it]
 14%|█▍        | 31814/225000 [71:34:07<383:36:37,  7.15s/it]
 14%|█▍        | 31815/225000 [71:34:10<324:22:50,  6.04s/it]
 14%|█▍        | 31816/225000 [71:34:20<384:36:30,  7.17s/it]
 14%|█▍        | 31817/225000 [71:34:26<363:17:07,  6.77s/it]
 14%|█▍        | 31818/225000 [71:34:31<339:31:11,  6.33s/it]
 14%|█▍        | 31819/225000 [71:34:36<320:14:30,  5.97s/it]
 14%|█▍        | 31820/225000 [71:34:42<317:44:18,  5.92s/it]
 14%|█▍        | 31821/225000 [71:34:48<326:19:01,  6.08s/it]
 14%|█▍        | 31822/225000 [71:34:56<359:43:44,  6.70s/it]
 14%|█▍        | 31823/225000 [71:35:01<325:07:57,  6.06s/it]
 14%|█▍        | 31824/225000 [71:35:06<303:39:04,  5.66s/it]
 14%|█▍        | 31825/225000 [71:35:18<404:35:53,  7.54s/it]
 14%|█▍        | 31826/225000 [71:35:21<339:39:51,  6.33s/it]
 14%|█▍        | 31827/225000 [71:35:29<369:38:38,  6.89s/it]
 14%|█▍        | 31828/225000 [71:35:35<349:05:30,  6.51s/it]
 14%|█▍        | 31829/225000 [71:35:41<337:57:29,  6.30s/it]
 14%|█▍        | 31830/225000 [71:35:46<326:02:36,  6.08s/it]
 14%|█▍        | 31831/225000 [71:35:52<322:22:36,  6.01s/it]
 14%|█▍        | 31832/225000 [71:35:57<310:03:37,  5.78s/it]
 14%|█▍        | 31833/225000 [71:36:10<420:33:05,  7.84s/it]
 14%|█▍        | 31834/225000 [71:36:14<352:37:46,  6.57s/it]
 14%|█▍        | 31835/225000 [71:36:24<405:12:40,  7.55s/it]
 14%|█▍        | 31836/225000 [71:36:29<365:17:13,  6.81s/it]
 14%|█▍        | 31837/225000 [71:36:36<376:51:19,  7.02s/it]
 14%|█▍        | 31838/225000 [71:36:41<343:59:11,  6.41s/it]
 14%|█▍        | 31839/225000 [71:36:48<351:26:17,  6.55s/it]
 14%|█▍        | 31840/225000 [71:36:54<338:06:43,  6.30s/it]
 14%|█▍        | 31841/225000 [71:37:01<354:57:50,  6.62s/it]
 14%|█▍        | 31842/225000 [71:37:05<317:59:25,  5.93s/it]
 14%|█▍        | 31843/225000 [71:37:10<296:40:30,  5.53s/it]
 14%|█▍        | 31844/225000 [71:37:21<386:56:41,  7.21s/it]
 14%|█▍        | 31845/225000 [71:37:26<354:21:49,  6.60s/it]
 14%|█▍        | 31846/225000 [71:37:33<351:37:09,  6.55s/it]
 14%|█▍        | 31847/225000 [71:37:40<360:00:53,  6.71s/it]
 14%|█▍        | 31848/225000 [71:37:46<344:37:41,  6.42s/it]
 14%|█▍        | 31849/225000 [71:37:53<356:25:20,  6.64s/it]
 14%|█▍        | 31850/225000 [71:37:58<336:23:38,  6.27s/it]
                                                             
{'loss': '0.7186', 'grad_norm': '0.7469', 'learning_rate': '4.874e-05', 'epoch': '0.1416', 'num_input_tokens_seen': 122283456, 'train_runtime': '2.579e+05', 'train_tokens_per_second': '474.2'}

 14%|█▍        | 31850/225000 [71:37:58<336:23:38,  6.27s/it]
 14%|█▍        | 31851/225000 [71:38:07<378:15:33,  7.05s/it]
 14%|█▍        | 31852/225000 [71:38:11<322:00:38,  6.00s/it]
 14%|█▍        | 31853/225000 [71:38:20<380:20:37,  7.09s/it]
 14%|█▍        | 31854/225000 [71:38:25<340:24:02,  6.34s/it]
 14%|█▍        | 31855/225000 [71:38:30<317:55:35,  5.93s/it]
 14%|█▍        | 31856/225000 [71:38:39<373:14:39,  6.96s/it]
 14%|█▍        | 31857/225000 [71:38:48<399:57:30,  7.45s/it]
 14%|█▍        | 31858/225000 [71:38:52<350:35:57,  6.53s/it]
 14%|█▍        | 31859/225000 [71:38:59<354:42:40,  6.61s/it]
 14%|█▍        | 31860/225000 [71:39:03<312:00:15,  5.82s/it]
 14%|█▍        | 31861/225000 [71:39:08<294:17:52,  5.49s/it]
 14%|█▍        | 31862/225000 [71:39:20<406:13:58,  7.57s/it]
 14%|█▍        | 31863/225000 [71:39:24<342:38:03,  6.39s/it]
 14%|█▍        | 31864/225000 [71:39:32<368:40:42,  6.87s/it]
 14%|█▍        | 31865/225000 [71:39:40<392:03:06,  7.31s/it]
 14%|█▍        | 31866/225000 [71:39:45<357:35:54,  6.67s/it]
 14%|█▍        | 31867/225000 [71:39:50<327:40:36,  6.11s/it]
 14%|█▍        | 31868/225000 [71:39:55<317:47:06,  5.92s/it]
 14%|█▍        | 31869/225000 [71:40:00<297:13:34,  5.54s/it]
 14%|█▍        | 31870/225000 [71:40:05<287:41:32,  5.36s/it]
 14%|█▍        | 31871/225000 [71:40:10<284:50:22,  5.31s/it]
 14%|█▍        | 31872/225000 [71:40:20<362:18:17,  6.75s/it]
 14%|█▍        | 31873/225000 [71:40:31<422:21:25,  7.87s/it]
 14%|█▍        | 31874/225000 [71:40:36<378:05:57,  7.05s/it]
 14%|█▍        | 31875/225000 [71:40:42<356:52:59,  6.65s/it]
 14%|█▍        | 31876/225000 [71:40:47<337:31:53,  6.29s/it]
 14%|█▍        | 31877/225000 [71:40:55<360:50:39,  6.73s/it]
 14%|█▍        | 31878/225000 [71:41:02<361:45:56,  6.74s/it]
 14%|█▍        | 31879/225000 [71:41:11<406:02:53,  7.57s/it]
 14%|█▍        | 31880/225000 [71:41:15<343:41:48,  6.41s/it]
 14%|█▍        | 31881/225000 [71:41:27<437:07:50,  8.15s/it]
 14%|█▍        | 31882/225000 [71:41:31<362:48:39,  6.76s/it]
 14%|█▍        | 31883/225000 [71:41:36<340:17:53,  6.34s/it]
 14%|█▍        | 31884/225000 [71:41:41<325:04:46,  6.06s/it]
 14%|█▍        | 31885/225000 [71:41:47<312:58:31,  5.83s/it]
 14%|█▍        | 31886/225000 [71:41:52<312:16:24,  5.82s/it]
 14%|█▍        | 31887/225000 [71:41:58<310:37:00,  5.79s/it]
 14%|█▍        | 31888/225000 [71:42:04<304:56:04,  5.68s/it]
 14%|█▍        | 31889/225000 [71:42:09<299:08:17,  5.58s/it]
 14%|█▍        | 31890/225000 [71:42:16<318:52:57,  5.94s/it]
 14%|█▍        | 31891/225000 [71:42:28<427:31:15,  7.97s/it]
 14%|█▍        | 31892/225000 [71:42:34<385:21:53,  7.18s/it]
 14%|█▍        | 31893/225000 [71:42:41<384:04:06,  7.16s/it]
 14%|█▍        | 31894/225000 [71:42:46<355:30:22,  6.63s/it]
 14%|█▍        | 31895/225000 [71:42:54<370:11:06,  6.90s/it]
 14%|█▍        | 31896/225000 [71:43:02<389:12:56,  7.26s/it]
 14%|█▍        | 31897/225000 [71:43:06<341:25:47,  6.37s/it]
 14%|█▍        | 31898/225000 [71:43:11<308:45:39,  5.76s/it]
 14%|█▍        | 31899/225000 [71:43:15<289:08:36,  5.39s/it]
 14%|█▍        | 31900/225000 [71:43:22<319:33:39,  5.96s/it]
                                                             
{'loss': '0.729', 'grad_norm': '0.7191', 'learning_rate': '4.873e-05', 'epoch': '0.1418', 'num_input_tokens_seen': 122429056, 'train_runtime': '2.582e+05', 'train_tokens_per_second': '474.2'}

 14%|█▍        | 31900/225000 [71:43:22<319:33:39,  5.96s/it]
 14%|█▍        | 31901/225000 [71:43:32<379:23:03,  7.07s/it]
 14%|█▍        | 31902/225000 [71:43:36<323:52:04,  6.04s/it]
 14%|█▍        | 31903/225000 [71:43:40<304:09:52,  5.67s/it]
 14%|█▍        | 31904/225000 [71:43:46<296:42:07,  5.53s/it]
 14%|█▍        | 31905/225000 [71:43:57<384:26:41,  7.17s/it]
 14%|█▍        | 31906/225000 [71:44:04<386:02:10,  7.20s/it]
 14%|█▍        | 31907/225000 [71:44:10<371:21:59,  6.92s/it]
 14%|█▍        | 31908/225000 [71:44:18<388:31:15,  7.24s/it]
 14%|█▍        | 31909/225000 [71:44:26<393:49:55,  7.34s/it]
 14%|█▍        | 31910/225000 [71:44:30<337:06:10,  6.28s/it]
 14%|█▍        | 31911/225000 [71:44:35<325:22:51,  6.07s/it]
 14%|█▍        | 31912/225000 [71:44:41<317:45:21,  5.92s/it]
 14%|█▍        | 31913/225000 [71:44:46<313:13:44,  5.84s/it]
 14%|█▍        | 31914/225000 [71:44:52<309:08:07,  5.76s/it]
 14%|█▍        | 31915/225000 [71:44:58<318:47:41,  5.94s/it]
 14%|█▍        | 31916/225000 [71:45:04<315:47:54,  5.89s/it]
 14%|█▍        | 31917/225000 [71:45:10<315:46:00,  5.89s/it]
 14%|█▍        | 31918/225000 [71:45:22<419:51:55,  7.83s/it]
 14%|█▍        | 31919/225000 [71:45:28<379:24:17,  7.07s/it]
 14%|█▍        | 31920/225000 [71:45:35<379:27:31,  7.08s/it]
 14%|█▍        | 31921/225000 [71:45:38<324:46:01,  6.06s/it]
 14%|█▍        | 31922/225000 [71:45:43<293:26:43,  5.47s/it]
 14%|█▍        | 31923/225000 [71:45:55<398:43:04,  7.43s/it]
 14%|█▍        | 31924/225000 [71:45:58<342:13:48,  6.38s/it]
 14%|█▍        | 31925/225000 [71:46:06<353:53:10,  6.60s/it]
 14%|█▍        | 31926/225000 [71:46:12<356:17:12,  6.64s/it]
 14%|█▍        | 31927/225000 [71:46:20<378:54:40,  7.07s/it]
 14%|█▍        | 31928/225000 [71:46:25<339:36:42,  6.33s/it]
 14%|█▍        | 31929/225000 [71:46:30<312:50:20,  5.83s/it]
 14%|█▍        | 31930/225000 [71:46:34<296:18:28,  5.52s/it]
 14%|█▍        | 31931/225000 [71:46:43<349:48:48,  6.52s/it]
 14%|█▍        | 31932/225000 [71:46:52<378:38:18,  7.06s/it]
 14%|█▍        | 31933/225000 [71:47:00<398:28:00,  7.43s/it]
 14%|█▍        | 31934/225000 [71:47:05<366:27:41,  6.83s/it]
 14%|█▍        | 31935/225000 [71:47:12<371:24:52,  6.93s/it]
 14%|█▍        | 31936/225000 [71:47:19<362:25:36,  6.76s/it]
 14%|█▍        | 31937/225000 [71:47:26<364:09:46,  6.79s/it]
 14%|█▍        | 31938/225000 [71:47:34<389:10:15,  7.26s/it]
 14%|█▍        | 31939/225000 [71:47:40<361:00:37,  6.73s/it]
 14%|█▍        | 31940/225000 [71:47:47<377:48:41,  7.05s/it]
 14%|█▍        | 31941/225000 [71:47:52<341:33:03,  6.37s/it]
 14%|█▍        | 31942/225000 [71:48:01<379:51:44,  7.08s/it]
 14%|█▍        | 31943/225000 [71:48:05<324:22:32,  6.05s/it]
 14%|█▍        | 31944/225000 [71:48:15<397:33:26,  7.41s/it]
 14%|█▍        | 31945/225000 [71:48:19<335:16:31,  6.25s/it]
 14%|█▍        | 31946/225000 [71:48:23<304:56:27,  5.69s/it]
 14%|█▍        | 31947/225000 [71:48:28<297:04:09,  5.54s/it]
 14%|█▍        | 31948/225000 [71:48:40<399:13:50,  7.44s/it]
 14%|█▍        | 31949/225000 [71:48:48<404:24:19,  7.54s/it]
 14%|█▍        | 31950/225000 [71:48:54<382:42:32,  7.14s/it]
                                                             
{'loss': '0.7428', 'grad_norm': '0.7753', 'learning_rate': '4.873e-05', 'epoch': '0.142', 'num_input_tokens_seen': 122575936, 'train_runtime': '2.585e+05', 'train_tokens_per_second': '474.1'}

 14%|█▍        | 31950/225000 [71:48:54<382:42:32,  7.14s/it]
 14%|█▍        | 31951/225000 [71:48:59<339:49:27,  6.34s/it]
 14%|█▍        | 31952/225000 [71:49:03<309:12:03,  5.77s/it]
 14%|█▍        | 31953/225000 [71:49:11<348:18:29,  6.50s/it]
 14%|█▍        | 31954/225000 [71:49:16<315:36:11,  5.89s/it]
 14%|█▍        | 31955/225000 [71:49:20<290:06:54,  5.41s/it]
 14%|█▍        | 31956/225000 [71:49:25<278:47:24,  5.20s/it]
 14%|█▍        | 31957/225000 [71:49:37<390:25:38,  7.28s/it]
 14%|█▍        | 31958/225000 [71:49:42<354:59:04,  6.62s/it]
 14%|█▍        | 31959/225000 [71:49:47<325:34:29,  6.07s/it]
 14%|█▍        | 31960/225000 [71:49:57<387:10:42,  7.22s/it]
 14%|█▍        | 31961/225000 [71:50:01<342:19:56,  6.38s/it]
 14%|█▍        | 31962/225000 [71:50:05<309:09:23,  5.77s/it]
 14%|█▍        | 31963/225000 [71:50:10<288:56:04,  5.39s/it]
 14%|█▍        | 31964/225000 [71:50:23<408:26:54,  7.62s/it]
 14%|█▍        | 31965/225000 [71:50:26<345:22:04,  6.44s/it]
 14%|█▍        | 31966/225000 [71:50:34<370:21:45,  6.91s/it]
 14%|█▍        | 31967/225000 [71:50:38<319:23:20,  5.96s/it]
 14%|█▍        | 31968/225000 [71:50:44<315:33:44,  5.89s/it]
 14%|█▍        | 31969/225000 [71:50:54<384:32:34,  7.17s/it]
 14%|█▍        | 31970/225000 [71:51:00<368:09:30,  6.87s/it]
 14%|█▍        | 31971/225000 [71:51:07<375:29:00,  7.00s/it]
 14%|█▍        | 31972/225000 [71:51:14<375:28:51,  7.00s/it]
 14%|█▍        | 31973/225000 [71:51:19<338:39:06,  6.32s/it]
 14%|█▍        | 31974/225000 [71:51:24<319:11:16,  5.95s/it]
 14%|█▍        | 31975/225000 [71:51:29<306:24:09,  5.71s/it]
 14%|█▍        | 31976/225000 [71:51:34<294:13:33,  5.49s/it]
 14%|█▍        | 31977/225000 [71:51:47<404:45:38,  7.55s/it]
 14%|█▍        | 31978/225000 [71:51:53<391:30:55,  7.30s/it]
 14%|█▍        | 31979/225000 [71:51:58<343:22:14,  6.40s/it]
 14%|█▍        | 31980/225000 [71:52:03<323:09:15,  6.03s/it]
 14%|█▍        | 31981/225000 [71:52:08<311:53:28,  5.82s/it]
 14%|█▍        | 31982/225000 [71:52:15<320:48:15,  5.98s/it]
 14%|█▍        | 31983/225000 [71:52:28<441:29:07,  8.23s/it]
 14%|█▍        | 31984/225000 [71:52:32<372:39:19,  6.95s/it]
 14%|█▍        | 31985/225000 [71:52:40<383:13:06,  7.15s/it]
 14%|█▍        | 31986/225000 [71:52:44<331:04:07,  6.17s/it]
 14%|█▍        | 31987/225000 [71:52:52<374:44:52,  6.99s/it]
 14%|█▍        | 31988/225000 [71:52:57<328:55:23,  6.13s/it]
 14%|█▍        | 31989/225000 [71:53:01<301:05:24,  5.62s/it]
 14%|█▍        | 31990/225000 [71:53:06<283:55:13,  5.30s/it]
 14%|█▍        | 31991/225000 [71:53:15<358:35:23,  6.69s/it]
 14%|█▍        | 31992/225000 [71:53:25<403:48:57,  7.53s/it]
 14%|█▍        | 31993/225000 [71:53:29<348:58:39,  6.51s/it]
 14%|█▍        | 31994/225000 [71:53:34<325:03:46,  6.06s/it]
 14%|█▍        | 31995/225000 [71:53:39<311:11:17,  5.80s/it]
 14%|█▍        | 31996/225000 [71:53:45<311:40:07,  5.81s/it]
 14%|█▍        | 31997/225000 [71:53:50<301:07:06,  5.62s/it]
 14%|█▍        | 31998/225000 [71:53:56<299:59:51,  5.60s/it]
 14%|█▍        | 31999/225000 [71:54:01<300:19:56,  5.60s/it]
 14%|█▍        | 32000/225000 [71:54:14<417:55:15,  7.80s/it]
                                                             
{'loss': '0.7133', 'grad_norm': '0.6291', 'learning_rate': '4.872e-05', 'epoch': '0.1422', 'num_input_tokens_seen': 122721008, 'train_runtime': '2.589e+05', 'train_tokens_per_second': '474.1'}

 14%|█▍        | 32000/225000 [71:54:14<417:55:15,  7.80s/it]
 14%|█▍        | 32001/225000 [71:54:20<387:18:00,  7.22s/it]
 14%|█▍        | 32002/225000 [71:54:31<435:33:53,  8.12s/it]
 14%|█▍        | 32003/225000 [71:54:34<361:29:51,  6.74s/it]
 14%|█▍        | 32004/225000 [71:54:42<382:25:17,  7.13s/it]
 14%|█▍        | 32005/225000 [71:54:46<335:56:19,  6.27s/it]
 14%|█▍        | 32006/225000 [71:54:51<310:49:05,  5.80s/it]
 14%|█▍        | 32007/225000 [71:54:55<289:26:26,  5.40s/it]
 14%|█▍        | 32008/225000 [71:55:05<357:19:48,  6.67s/it]
 14%|█▍        | 32009/225000 [71:55:11<339:08:02,  6.33s/it]
 14%|█▍        | 32010/225000 [71:55:16<320:53:59,  5.99s/it]
 14%|█▍        | 32011/225000 [71:55:21<308:51:16,  5.76s/it]
 14%|█▍        | 32012/225000 [71:55:26<298:19:45,  5.57s/it]
 14%|█▍        | 32013/225000 [71:55:38<404:13:58,  7.54s/it]
 14%|█▍        | 32014/225000 [71:55:45<382:11:26,  7.13s/it]
 14%|█▍        | 32015/225000 [71:55:50<350:58:49,  6.55s/it]
 14%|█▍        | 32016/225000 [71:55:55<327:40:49,  6.11s/it]
 14%|█▍        | 32017/225000 [71:55:59<302:40:35,  5.65s/it]
 14%|█▍        | 32018/225000 [71:56:11<406:25:49,  7.58s/it]
 14%|█▍        | 32019/225000 [71:56:18<393:22:02,  7.34s/it]
 14%|█▍        | 32020/225000 [71:56:23<346:32:01,  6.46s/it]
 14%|█▍        | 32021/225000 [71:56:30<368:27:56,  6.87s/it]
 14%|█▍        | 32022/225000 [71:56:35<328:28:45,  6.13s/it]
 14%|█▍        | 32023/225000 [71:56:39<297:18:07,  5.55s/it]
 14%|█▍        | 32024/225000 [71:56:47<330:27:21,  6.16s/it]
 14%|█▍        | 32025/225000 [71:56:57<395:54:33,  7.39s/it]
 14%|█▍        | 32026/225000 [71:57:01<348:04:46,  6.49s/it]
 14%|█▍        | 32027/225000 [71:57:06<317:44:08,  5.93s/it]
 14%|█▍        | 32028/225000 [71:57:11<297:43:18,  5.55s/it]
 14%|█▍        | 32029/225000 [71:57:16<288:42:23,  5.39s/it]
 14%|█▍        | 32030/225000 [71:57:27<387:07:31,  7.22s/it]
 14%|█▍        | 32031/225000 [71:57:36<408:20:45,  7.62s/it]
 14%|█▍        | 32032/225000 [71:57:40<348:20:46,  6.50s/it]
 14%|█▍        | 32033/225000 [71:57:44<319:32:33,  5.96s/it]
 14%|█▍        | 32034/225000 [71:57:50<312:21:58,  5.83s/it]
 14%|█▍        | 32035/225000 [71:58:01<394:50:32,  7.37s/it]
 14%|█▍        | 32036/225000 [71:58:06<365:58:56,  6.83s/it]
 14%|█▍        | 32037/225000 [71:58:12<344:10:09,  6.42s/it]
 14%|█▍        | 32038/225000 [71:58:17<332:06:33,  6.20s/it]
 14%|█▍        | 32039/225000 [71:58:23<324:58:23,  6.06s/it]
 14%|█▍        | 32040/225000 [71:58:28<310:44:52,  5.80s/it]
 14%|█▍        | 32041/225000 [71:58:34<310:28:16,  5.79s/it]
 14%|█▍        | 32042/225000 [71:58:40<311:15:24,  5.81s/it]
 14%|█▍        | 32043/225000 [71:58:45<306:32:12,  5.72s/it]
 14%|█▍        | 32044/225000 [71:58:52<320:49:25,  5.99s/it]
 14%|█▍        | 32045/225000 [71:58:58<324:51:27,  6.06s/it]
 14%|█▍        | 32046/225000 [71:59:12<451:21:11,  8.42s/it]
 14%|█▍        | 32047/225000 [71:59:17<392:03:00,  7.31s/it]
 14%|█▍        | 32048/225000 [71:59:24<393:00:22,  7.33s/it]
 14%|█▍        | 32049/225000 [71:59:30<367:11:40,  6.85s/it]
 14%|█▍        | 32050/225000 [71:59:38<388:30:27,  7.25s/it]
                                                             
{'loss': '0.7272', 'grad_norm': '0.8297', 'learning_rate': '4.872e-05', 'epoch': '0.1424', 'num_input_tokens_seen': 122865656, 'train_runtime': '2.592e+05', 'train_tokens_per_second': '474.1'}

 14%|█▍        | 32050/225000 [71:59:38<388:30:27,  7.25s/it]
 14%|█▍        | 32051/225000 [71:59:43<349:50:52,  6.53s/it]
 14%|█▍        | 32052/225000 [71:59:51<369:26:09,  6.89s/it]
 14%|█▍        | 32053/225000 [71:59:55<321:11:03,  5.99s/it]
 14%|█▍        | 32054/225000 [71:59:59<289:14:16,  5.40s/it]
 14%|█▍        | 32055/225000 [72:00:10<383:54:50,  7.16s/it]
 14%|█▍        | 32056/225000 [72:00:14<330:10:11,  6.16s/it]
 14%|█▍        | 32057/225000 [72:00:19<307:52:49,  5.74s/it]
 14%|█▍        | 32058/225000 [72:00:23<286:16:11,  5.34s/it]
 14%|█▍        | 32059/225000 [72:00:28<281:44:56,  5.26s/it]
 14%|█▍        | 32060/225000 [72:00:41<396:45:38,  7.40s/it]
 14%|█▍        | 32061/225000 [72:00:48<402:38:22,  7.51s/it]
 14%|█▍        | 32062/225000 [72:00:54<375:41:04,  7.01s/it]
 14%|█▍        | 32063/225000 [72:01:01<381:28:00,  7.12s/it]
 14%|█▍        | 32064/225000 [72:01:09<385:33:49,  7.19s/it]
 14%|█▍        | 32065/225000 [72:01:13<340:07:03,  6.35s/it]
 14%|█▍        | 32066/225000 [72:01:18<311:44:05,  5.82s/it]
 14%|█▍        | 32067/225000 [72:01:28<388:50:33,  7.26s/it]
 14%|█▍        | 32068/225000 [72:01:34<357:44:23,  6.68s/it]
 14%|█▍        | 32069/225000 [72:01:41<361:10:18,  6.74s/it]
 14%|█▍        | 32070/225000 [72:01:45<319:23:08,  5.96s/it]
 14%|█▍        | 32071/225000 [72:01:49<284:13:45,  5.30s/it]
 14%|█▍        | 32072/225000 [72:01:56<320:31:35,  5.98s/it]
 14%|█▍        | 32073/225000 [72:02:00<282:15:16,  5.27s/it]
 14%|█▍        | 32074/225000 [72:02:11<375:42:31,  7.01s/it]
 14%|█▍        | 32075/225000 [72:02:15<331:46:32,  6.19s/it]
 14%|█▍        | 32076/225000 [72:02:22<347:49:08,  6.49s/it]
 14%|█▍        | 32077/225000 [72:02:27<313:25:26,  5.85s/it]
 14%|█▍        | 32078/225000 [72:02:37<391:59:32,  7.31s/it]
 14%|█▍        | 32079/225000 [72:02:42<355:04:13,  6.63s/it]
 14%|█▍        | 32080/225000 [72:02:49<351:54:32,  6.57s/it]
 14%|█▍        | 32081/225000 [72:02:56<363:39:03,  6.79s/it]
 14%|█▍        | 32082/225000 [72:03:02<352:11:07,  6.57s/it]
 14%|█▍        | 32083/225000 [72:03:08<348:33:36,  6.50s/it]
 14%|█▍        | 32084/225000 [72:03:16<356:41:39,  6.66s/it]
 14%|█▍        | 32085/225000 [72:03:21<331:31:33,  6.19s/it]
 14%|█▍        | 32086/225000 [72:03:28<350:37:05,  6.54s/it]
 14%|█▍        | 32087/225000 [72:03:34<334:37:11,  6.24s/it]
 14%|█▍        | 32088/225000 [72:03:38<299:51:55,  5.60s/it]
 14%|█▍        | 32089/225000 [72:03:48<379:47:37,  7.09s/it]
 14%|█▍        | 32090/225000 [72:03:52<335:19:53,  6.26s/it]
 14%|█▍        | 32091/225000 [72:04:00<362:52:50,  6.77s/it]
 14%|█▍        | 32092/225000 [72:04:06<349:47:19,  6.53s/it]
 14%|█▍        | 32093/225000 [72:04:12<339:00:55,  6.33s/it]
 14%|█▍        | 32094/225000 [72:04:18<324:50:52,  6.06s/it]
 14%|█▍        | 32095/225000 [72:04:23<312:42:28,  5.84s/it]
 14%|█▍        | 32096/225000 [72:04:28<297:20:26,  5.55s/it]
 14%|█▍        | 32097/225000 [72:04:33<292:17:34,  5.45s/it]
 14%|█▍        | 32098/225000 [72:04:39<295:56:16,  5.52s/it]
 14%|█▍        | 32099/225000 [72:04:44<293:41:51,  5.48s/it]
 14%|█▍        | 32100/225000 [72:04:50<301:22:42,  5.62s/it]
                                                             
{'loss': '0.7461', 'grad_norm': '0.6477', 'learning_rate': '4.871e-05', 'epoch': '0.1427', 'num_input_tokens_seen': 123010136, 'train_runtime': '2.595e+05', 'train_tokens_per_second': '474'}

 14%|█▍        | 32100/225000 [72:04:50<301:22:42,  5.62s/it]
 14%|█▍        | 32101/225000 [72:05:04<429:18:41,  8.01s/it]
 14%|█▍        | 32102/225000 [72:05:10<407:08:45,  7.60s/it]
 14%|█▍        | 32103/225000 [72:05:18<401:10:54,  7.49s/it]
 14%|█▍        | 32104/225000 [72:05:21<338:03:32,  6.31s/it]
 14%|█▍        | 32105/225000 [72:05:30<372:23:57,  6.95s/it]
 14%|█▍        | 32106/225000 [72:05:33<320:13:55,  5.98s/it]
 14%|█▍        | 32107/225000 [72:05:43<375:06:32,  7.00s/it]
 14%|█▍        | 32108/225000 [72:05:50<385:38:46,  7.20s/it]
 14%|█▍        | 32109/225000 [72:05:54<332:34:44,  6.21s/it]
 14%|█▍        | 32110/225000 [72:06:04<390:23:20,  7.29s/it]
 14%|█▍        | 32111/225000 [72:06:09<345:24:29,  6.45s/it]
 14%|█▍        | 32112/225000 [72:06:14<323:21:51,  6.04s/it]
 14%|█▍        | 32113/225000 [72:06:19<316:26:47,  5.91s/it]
 14%|█▍        | 32114/225000 [72:06:25<311:37:42,  5.82s/it]
 14%|█▍        | 32115/225000 [72:06:31<311:29:54,  5.81s/it]
 14%|█▍        | 32116/225000 [72:06:36<304:36:05,  5.69s/it]
 14%|█▍        | 32117/225000 [72:06:42<303:43:41,  5.67s/it]
 14%|█▍        | 32118/225000 [72:06:48<319:35:30,  5.96s/it]
 14%|█▍        | 32119/225000 [72:07:02<436:18:17,  8.14s/it]
 14%|█▍        | 32120/225000 [72:07:06<384:42:47,  7.18s/it]
 14%|█▍        | 32121/225000 [72:07:11<343:34:01,  6.41s/it]
 14%|█▍        | 32122/225000 [72:07:17<338:51:44,  6.32s/it]
 14%|█▍        | 32123/225000 [72:07:27<390:20:25,  7.29s/it]
 14%|█▍        | 32124/225000 [72:07:31<340:06:25,  6.35s/it]
 14%|█▍        | 32125/225000 [72:07:37<330:45:33,  6.17s/it]
 14%|█▍        | 32126/225000 [72:07:42<314:24:26,  5.87s/it]
 14%|█▍        | 32127/225000 [72:07:47<305:23:51,  5.70s/it]
 14%|█▍        | 32128/225000 [72:07:53<304:06:42,  5.68s/it]
 14%|█▍        | 32129/225000 [72:07:58<298:58:12,  5.58s/it]
 14%|█▍        | 32130/225000 [72:08:10<400:20:42,  7.47s/it]
 14%|█▍        | 32131/225000 [72:08:16<373:48:14,  6.98s/it]
 14%|█▍        | 32132/225000 [72:08:24<388:05:21,  7.24s/it]
 14%|█▍        | 32133/225000 [72:08:28<336:42:02,  6.28s/it]
 14%|█▍        | 32134/225000 [72:08:33<316:01:40,  5.90s/it]
 14%|█▍        | 32135/225000 [72:08:38<300:19:50,  5.61s/it]
 14%|█▍        | 32136/225000 [72:08:43<293:26:27,  5.48s/it]
 14%|█▍        | 32137/225000 [72:08:48<289:27:53,  5.40s/it]
 14%|█▍        | 32138/225000 [72:08:54<292:26:15,  5.46s/it]
 14%|█▍        | 32139/225000 [72:09:07<414:06:25,  7.73s/it]
 14%|█▍        | 32140/225000 [72:09:14<405:44:01,  7.57s/it]
 14%|█▍        | 32141/225000 [72:09:22<418:46:22,  7.82s/it]
 14%|█▍        | 32142/225000 [72:09:26<353:36:05,  6.60s/it]
 14%|█▍        | 32143/225000 [72:09:36<404:52:22,  7.56s/it]
 14%|█▍        | 32144/225000 [72:09:42<376:02:31,  7.02s/it]
 14%|█▍        | 32145/225000 [72:09:47<346:02:13,  6.46s/it]
 14%|█▍        | 32146/225000 [72:09:52<331:48:16,  6.19s/it]
 14%|█▍        | 32147/225000 [72:09:58<318:36:26,  5.95s/it]
 14%|█▍        | 32148/225000 [72:10:03<307:25:37,  5.74s/it]
 14%|█▍        | 32149/225000 [72:10:09<313:42:51,  5.86s/it]
 14%|█▍        | 32150/225000 [72:10:14<301:09:05,  5.62s/it]
                                                             
{'loss': '0.7159', 'grad_norm': '0.7517', 'learning_rate': '4.87e-05', 'epoch': '0.1429', 'num_input_tokens_seen': 123154192, 'train_runtime': '2.598e+05', 'train_tokens_per_second': '474'}

 14%|█▍        | 32150/225000 [72:10:14<301:09:05,  5.62s/it]
 14%|█▍        | 32151/225000 [72:10:19<294:21:30,  5.49s/it]
 14%|█▍        | 32152/225000 [72:10:25<291:10:09,  5.44s/it]
 14%|█▍        | 32153/225000 [72:10:30<285:23:56,  5.33s/it]
 14%|█▍        | 32154/225000 [72:10:43<409:10:17,  7.64s/it]
 14%|█▍        | 32155/225000 [72:10:49<392:42:07,  7.33s/it]
 14%|█▍        | 32156/225000 [72:10:54<350:54:40,  6.55s/it]
 14%|█▍        | 32157/225000 [72:10:59<321:40:11,  6.00s/it]
 14%|█▍        | 32158/225000 [72:11:10<411:58:13,  7.69s/it]
 14%|█▍        | 32159/225000 [72:11:16<379:33:20,  7.09s/it]
 14%|█▍        | 32160/225000 [72:11:22<357:44:52,  6.68s/it]
 14%|█▍        | 32161/225000 [72:11:29<371:28:06,  6.93s/it]
 14%|█▍        | 32162/225000 [72:11:33<321:37:14,  6.00s/it]
 14%|█▍        | 32163/225000 [72:11:41<348:09:13,  6.50s/it]
 14%|█▍        | 32164/225000 [72:11:49<368:58:44,  6.89s/it]
 14%|█▍        | 32165/225000 [72:11:53<327:58:10,  6.12s/it]
 14%|█▍        | 32166/225000 [72:11:57<297:49:03,  5.56s/it]
 14%|█▍        | 32167/225000 [72:12:08<378:23:30,  7.06s/it]
 14%|█▍        | 32168/225000 [72:12:14<359:03:05,  6.70s/it]
 14%|█▍        | 32169/225000 [72:12:19<343:35:30,  6.41s/it]
 14%|█▍        | 32170/225000 [72:12:25<329:19:38,  6.15s/it]
 14%|█▍        | 32171/225000 [72:12:30<312:42:06,  5.84s/it]
 14%|█▍        | 32172/225000 [72:12:35<300:17:57,  5.61s/it]
 14%|█▍        | 32173/225000 [72:12:40<292:11:08,  5.45s/it]
 14%|█▍        | 32174/225000 [72:12:53<403:02:02,  7.52s/it]
 14%|█▍        | 32175/225000 [72:12:57<353:36:55,  6.60s/it]
 14%|█▍        | 32176/225000 [72:13:02<320:57:34,  5.99s/it]
 14%|█▍        | 32177/225000 [72:13:12<383:31:30,  7.16s/it]
 14%|█▍        | 32178/225000 [72:13:18<373:25:42,  6.97s/it]
 14%|█▍        | 32179/225000 [72:13:24<356:59:50,  6.67s/it]
 14%|█▍        | 32180/225000 [72:13:31<361:43:25,  6.75s/it]
 14%|█▍        | 32181/225000 [72:13:35<324:24:28,  6.06s/it]
 14%|█▍        | 32182/225000 [72:13:41<318:42:32,  5.95s/it]
 14%|█▍        | 32183/225000 [72:13:47<311:29:47,  5.82s/it]
 14%|█▍        | 32184/225000 [72:13:52<311:57:29,  5.82s/it]
 14%|█▍        | 32185/225000 [72:13:57<293:13:10,  5.47s/it]
 14%|█▍        | 32186/225000 [72:14:02<291:47:58,  5.45s/it]
 14%|█▍        | 32187/225000 [72:14:08<295:55:20,  5.53s/it]
 14%|█▍        | 32188/225000 [72:14:15<309:41:37,  5.78s/it]
 14%|█▍        | 32189/225000 [72:14:27<415:58:47,  7.77s/it]
 14%|█▍        | 32190/225000 [72:14:33<390:31:29,  7.29s/it]
 14%|█▍        | 32191/225000 [72:14:37<338:11:07,  6.31s/it]
 14%|█▍        | 32192/225000 [72:14:48<405:36:20,  7.57s/it]
 14%|█▍        | 32193/225000 [72:14:53<362:36:16,  6.77s/it]
 14%|█▍        | 32194/225000 [72:15:01<394:09:20,  7.36s/it]
 14%|█▍        | 32195/225000 [72:15:06<349:20:13,  6.52s/it]
 14%|█▍        | 32196/225000 [72:15:11<320:21:46,  5.98s/it]
 14%|█▍        | 32197/225000 [72:15:16<309:17:01,  5.77s/it]
 14%|█▍        | 32198/225000 [72:15:28<408:59:50,  7.64s/it]
 14%|█▍        | 32199/225000 [72:15:33<365:26:19,  6.82s/it]
 14%|█▍        | 32200/225000 [72:15:40<368:25:12,  6.88s/it]
                                                             
{'loss': '0.7194', 'grad_norm': '0.7464', 'learning_rate': '4.87e-05', 'epoch': '0.1431', 'num_input_tokens_seen': 123297984, 'train_runtime': '2.601e+05', 'train_tokens_per_second': '474'}

 14%|█▍        | 32200/225000 [72:15:40<368:25:12,  6.88s/it]
 14%|█▍        | 32201/225000 [72:15:43<313:47:00,  5.86s/it]
 14%|█▍        | 32202/225000 [72:15:53<373:40:16,  6.98s/it]
 14%|█▍        | 32203/225000 [72:16:00<372:19:37,  6.95s/it]
 14%|█▍        | 32204/225000 [72:16:05<352:36:07,  6.58s/it]
 14%|█▍        | 32205/225000 [72:16:12<352:58:30,  6.59s/it]
 14%|█▍        | 32206/225000 [72:16:19<363:50:46,  6.79s/it]
 14%|█▍        | 32207/225000 [72:16:24<321:25:03,  6.00s/it]
 14%|█▍        | 32208/225000 [72:16:28<293:04:18,  5.47s/it]
 14%|█▍        | 32209/225000 [72:16:36<336:29:05,  6.28s/it]
 14%|█▍        | 32210/225000 [72:16:43<349:48:46,  6.53s/it]
 14%|█▍        | 32211/225000 [72:16:49<346:25:13,  6.47s/it]
 14%|█▍        | 32212/225000 [72:16:54<318:33:49,  5.95s/it]
 14%|█▍        | 32213/225000 [72:16:59<307:11:46,  5.74s/it]
 14%|█▍        | 32214/225000 [72:17:04<295:54:07,  5.53s/it]
 14%|█▍        | 32215/225000 [72:17:10<294:50:28,  5.51s/it]
 14%|█▍        | 32216/225000 [72:17:15<290:12:36,  5.42s/it]
 14%|█▍        | 32217/225000 [72:17:20<289:14:02,  5.40s/it]
 14%|█▍        | 32218/225000 [72:17:26<288:50:12,  5.39s/it]
 14%|█▍        | 32219/225000 [72:17:31<289:02:23,  5.40s/it]
 14%|█▍        | 32220/225000 [72:17:42<381:48:24,  7.13s/it]
 14%|█▍        | 32221/225000 [72:17:50<386:15:36,  7.21s/it]
 14%|█▍        | 32222/225000 [72:17:57<384:13:19,  7.18s/it]
 14%|█▍        | 32223/225000 [72:18:02<354:35:44,  6.62s/it]
 14%|█▍        | 32224/225000 [72:18:09<351:53:54,  6.57s/it]
 14%|█▍        | 32225/225000 [72:18:17<380:59:16,  7.11s/it]
 14%|█▍        | 32226/225000 [72:18:23<355:26:00,  6.64s/it]
 14%|█▍        | 32227/225000 [72:18:30<367:56:17,  6.87s/it]
 14%|█▍        | 32228/225000 [72:18:35<333:18:12,  6.22s/it]
 14%|█▍        | 32229/225000 [72:18:39<309:14:45,  5.78s/it]
 14%|█▍        | 32230/225000 [72:18:46<318:22:38,  5.95s/it]
 14%|█▍        | 32231/225000 [72:18:56<390:44:23,  7.30s/it]
 14%|█▍        | 32232/225000 [72:18:59<323:10:15,  6.04s/it]
 14%|█▍        | 32233/225000 [72:19:10<394:56:09,  7.38s/it]
 14%|█▍        | 32234/225000 [72:19:14<339:01:46,  6.33s/it]
 14%|█▍        | 32235/225000 [72:19:18<302:27:25,  5.65s/it]
 14%|█▍        | 32236/225000 [72:19:28<368:40:16,  6.89s/it]
 14%|█▍        | 32237/225000 [72:19:34<364:23:35,  6.81s/it]
 14%|█▍        | 32238/225000 [72:19:38<315:56:40,  5.90s/it]
 14%|█▍        | 32239/225000 [72:19:47<366:55:51,  6.85s/it]
 14%|█▍        | 32240/225000 [72:19:51<321:05:09,  6.00s/it]
 14%|█▍        | 32241/225000 [72:19:55<287:43:20,  5.37s/it]
 14%|█▍        | 32242/225000 [72:20:05<357:31:49,  6.68s/it]
 14%|█▍        | 32243/225000 [72:20:09<322:00:50,  6.01s/it]
 14%|█▍        | 32244/225000 [72:20:14<307:50:46,  5.75s/it]
 14%|█▍        | 32245/225000 [72:20:24<379:28:21,  7.09s/it]
 14%|█▍        | 32246/225000 [72:20:30<349:13:02,  6.52s/it]
 14%|█▍        | 32247/225000 [72:20:35<329:18:55,  6.15s/it]
 14%|█▍        | 32248/225000 [72:20:40<309:57:47,  5.79s/it]
 14%|█▍        | 32249/225000 [72:20:45<295:45:58,  5.52s/it]
 14%|█▍        | 32250/225000 [72:20:51<300:12:24,  5.61s/it]
                                                             
{'loss': '0.7322', 'grad_norm': '0.6484', 'learning_rate': '4.869e-05', 'epoch': '0.1433', 'num_input_tokens_seen': 123441792, 'train_runtime': '2.605e+05', 'train_tokens_per_second': '474'}

 14%|█▍        | 32250/225000 [72:20:51<300:12:24,  5.61s/it]
 14%|█▍        | 32251/225000 [72:20:58<321:47:16,  6.01s/it]
 14%|█▍        | 32252/225000 [72:21:08<385:48:19,  7.21s/it]
 14%|█▍        | 32253/225000 [72:21:11<333:36:52,  6.23s/it]
 14%|█▍        | 32254/225000 [72:21:22<394:15:44,  7.36s/it]
 14%|█▍        | 32255/225000 [72:21:30<405:32:37,  7.57s/it]
 14%|█▍        | 32256/225000 [72:21:34<355:27:27,  6.64s/it]
 14%|█▍        | 32257/225000 [72:21:39<327:25:41,  6.12s/it]
 14%|█▍        | 32258/225000 [72:21:44<310:34:14,  5.80s/it]
 14%|█▍        | 32259/225000 [72:21:56<408:22:49,  7.63s/it]
 14%|█▍        | 32260/225000 [72:22:01<372:10:20,  6.95s/it]
 14%|█▍        | 32261/225000 [72:22:07<350:56:12,  6.55s/it]
 14%|█▍        | 32262/225000 [72:22:12<325:23:52,  6.08s/it]
 14%|█▍        | 32263/225000 [72:22:17<310:03:18,  5.79s/it]
 14%|█▍        | 32264/225000 [72:22:22<289:50:18,  5.41s/it]
 14%|█▍        | 32265/225000 [72:22:32<374:27:05,  6.99s/it]
 14%|█▍        | 32266/225000 [72:22:39<378:55:32,  7.08s/it]
 14%|█▍        | 32267/225000 [72:22:43<322:13:24,  6.02s/it]
 14%|█▍        | 32268/225000 [72:22:53<384:36:23,  7.18s/it]
 14%|█▍        | 32269/225000 [72:22:57<332:09:19,  6.20s/it]
 14%|█▍        | 32270/225000 [72:23:06<384:55:10,  7.19s/it]
 14%|█▍        | 32271/225000 [72:23:12<361:35:06,  6.75s/it]
 14%|█▍        | 32272/225000 [72:23:20<383:48:18,  7.17s/it]
 14%|█▍        | 32273/225000 [72:23:28<392:43:40,  7.34s/it]
 14%|█▍        | 32274/225000 [72:23:32<336:40:27,  6.29s/it]
 14%|█▍        | 32275/225000 [72:23:37<315:44:27,  5.90s/it]
 14%|█▍        | 32276/225000 [72:23:41<295:55:53,  5.53s/it]
 14%|█▍        | 32277/225000 [72:23:46<285:37:53,  5.34s/it]
 14%|█▍        | 32278/225000 [72:23:54<326:08:08,  6.09s/it]
 14%|█▍        | 32279/225000 [72:24:05<395:17:56,  7.38s/it]
 14%|█▍        | 32280/225000 [72:24:08<339:47:15,  6.35s/it]
 14%|█▍        | 32281/225000 [72:24:17<372:07:34,  6.95s/it]
 14%|█▍        | 32282/225000 [72:24:23<366:31:58,  6.85s/it]
 14%|█▍        | 32283/225000 [72:24:27<319:08:39,  5.96s/it]
 14%|█▍        | 32284/225000 [72:24:39<402:38:53,  7.52s/it]
 14%|█▍        | 32285/225000 [72:24:42<343:24:00,  6.41s/it]
 14%|█▍        | 32286/225000 [72:24:50<365:19:23,  6.82s/it]
 14%|█▍        | 32287/225000 [72:24:55<338:08:35,  6.32s/it]
 14%|█▍        | 32288/225000 [72:25:00<318:04:03,  5.94s/it]
 14%|█▍        | 32289/225000 [72:25:05<303:03:05,  5.66s/it]
 14%|█▍        | 32290/225000 [72:25:10<294:55:42,  5.51s/it]
 14%|█▍        | 32291/225000 [72:25:15<279:44:13,  5.23s/it]
 14%|█▍        | 32292/225000 [72:25:20<276:33:08,  5.17s/it]
 14%|█▍        | 32293/225000 [72:25:31<367:34:07,  6.87s/it]
 14%|█▍        | 32294/225000 [72:25:41<415:06:05,  7.75s/it]
 14%|█▍        | 32295/225000 [72:25:47<385:23:33,  7.20s/it]
 14%|█▍        | 32296/225000 [72:25:51<341:08:57,  6.37s/it]
 14%|█▍        | 32297/225000 [72:26:01<399:09:56,  7.46s/it]
 14%|█▍        | 32298/225000 [72:26:05<347:06:13,  6.48s/it]
 14%|█▍        | 32299/225000 [72:26:10<323:41:31,  6.05s/it]
 14%|█▍        | 32300/225000 [72:26:16<314:10:34,  5.87s/it]
                                                             
{'loss': '0.7203', 'grad_norm': '0.6933', 'learning_rate': '4.869e-05', 'epoch': '0.1436', 'num_input_tokens_seen': 123587792, 'train_runtime': '2.608e+05', 'train_tokens_per_second': '473.9'}

 14%|█▍        | 32300/225000 [72:26:16<314:10:34,  5.87s/it]
 14%|█▍        | 32301/225000 [72:26:21<308:22:03,  5.76s/it]
 14%|█▍        | 32302/225000 [72:26:27<309:58:24,  5.79s/it]
 14%|█▍        | 32303/225000 [72:26:33<308:23:57,  5.76s/it]
 14%|█▍        | 32304/225000 [72:26:40<329:09:19,  6.15s/it]
 14%|█▍        | 32305/225000 [72:26:45<312:37:32,  5.84s/it]
 14%|█▍        | 32306/225000 [72:26:58<426:23:50,  7.97s/it]
 14%|█▍        | 32307/225000 [72:27:04<388:24:55,  7.26s/it]
 14%|█▍        | 32308/225000 [72:27:08<343:56:33,  6.43s/it]
 14%|█▍        | 32309/225000 [72:27:14<343:54:19,  6.43s/it]
 14%|█▍        | 32310/225000 [72:27:23<385:56:30,  7.21s/it]
 14%|█▍        | 32311/225000 [72:27:27<325:21:04,  6.08s/it]
 14%|█▍        | 32312/225000 [72:27:37<396:54:47,  7.42s/it]
 14%|█▍        | 32313/225000 [72:27:45<392:07:47,  7.33s/it]
 14%|█▍        | 32314/225000 [72:27:49<344:14:00,  6.43s/it]
 14%|█▍        | 32315/225000 [72:27:54<320:33:36,  5.99s/it]
 14%|█▍        | 32316/225000 [72:27:59<313:08:46,  5.85s/it]
 14%|█▍        | 32317/225000 [72:28:05<303:43:12,  5.67s/it]
 14%|█▍        | 32318/225000 [72:28:13<342:06:42,  6.39s/it]
 14%|█▍        | 32319/225000 [72:28:24<416:30:12,  7.78s/it]
 14%|█▍        | 32320/225000 [72:28:27<349:42:16,  6.53s/it]
 14%|█▍        | 32321/225000 [72:28:35<363:10:16,  6.79s/it]
 14%|█▍        | 32322/225000 [72:28:42<374:11:27,  6.99s/it]
 14%|█▍        | 32323/225000 [72:28:47<332:38:38,  6.22s/it]
 14%|█▍        | 32324/225000 [72:28:52<316:07:50,  5.91s/it]
 14%|█▍        | 32325/225000 [72:28:57<306:36:14,  5.73s/it]
 14%|█▍        | 32326/225000 [72:29:02<300:03:02,  5.61s/it]
 14%|█▍        | 32327/225000 [72:29:07<290:46:49,  5.43s/it]
 14%|█▍        | 32328/225000 [72:29:17<350:59:47,  6.56s/it]
 14%|█▍        | 32329/225000 [72:29:29<438:37:13,  8.20s/it]
 14%|█▍        | 32330/225000 [72:29:34<391:04:02,  7.31s/it]
 14%|█▍        | 32331/225000 [72:29:40<367:46:06,  6.87s/it]
 14%|█▍        | 32332/225000 [72:29:45<337:41:50,  6.31s/it]
 14%|█▍        | 32333/225000 [72:29:50<315:56:31,  5.90s/it]
 14%|█▍        | 32334/225000 [72:29:55<309:18:29,  5.78s/it]
 14%|█▍        | 32335/225000 [72:30:00<296:20:48,  5.54s/it]
 14%|█▍        | 32336/225000 [72:30:05<287:38:19,  5.37s/it]
 14%|█▍        | 32337/225000 [72:30:10<285:31:50,  5.34s/it]
 14%|█▍        | 32338/225000 [72:30:24<424:02:30,  7.92s/it]
 14%|█▍        | 32339/225000 [72:30:32<421:25:22,  7.87s/it]
 14%|█▍        | 32340/225000 [72:30:40<422:14:28,  7.89s/it]
 14%|█▍        | 32341/225000 [72:30:44<361:43:26,  6.76s/it]
 14%|█▍        | 32342/225000 [72:30:51<365:45:10,  6.83s/it]
 14%|█▍        | 32343/225000 [72:30:58<368:57:55,  6.89s/it]
 14%|█▍        | 32344/225000 [72:31:04<353:10:11,  6.60s/it]
 14%|█▍        | 32345/225000 [72:31:08<303:37:47,  5.67s/it]
 14%|█▍        | 32346/225000 [72:31:16<352:45:22,  6.59s/it]
 14%|█▍        | 32347/225000 [72:31:22<342:38:35,  6.40s/it]
 14%|█▍        | 32348/225000 [72:31:29<339:34:30,  6.35s/it]
 14%|█▍        | 32349/225000 [72:31:34<318:04:17,  5.94s/it]
 14%|█▍        | 32350/225000 [72:31:39<309:31:16,  5.78s/it]
                                                             
{'loss': '0.727', 'grad_norm': '0.7285', 'learning_rate': '4.868e-05', 'epoch': '0.1438', 'num_input_tokens_seen': 123732568, 'train_runtime': '2.611e+05', 'train_tokens_per_second': '473.9'}

 14%|█▍        | 32350/225000 [72:31:39<309:31:16,  5.78s/it]
 14%|█▍        | 32351/225000 [72:31:44<299:50:17,  5.60s/it]
 14%|█▍        | 32352/225000 [72:31:49<294:44:59,  5.51s/it]
 14%|█▍        | 32353/225000 [72:31:55<288:08:03,  5.38s/it]
 14%|█▍        | 32354/225000 [72:32:05<376:14:35,  7.03s/it]
 14%|█▍        | 32355/225000 [72:32:17<445:16:24,  8.32s/it]
 14%|█▍        | 32356/225000 [72:32:21<375:01:27,  7.01s/it]
 14%|█▍        | 32357/225000 [72:32:25<324:40:24,  6.07s/it]
 14%|█▍        | 32358/225000 [72:32:35<397:11:23,  7.42s/it]
 14%|█▍        | 32359/225000 [72:32:43<398:16:05,  7.44s/it]
 14%|█▍        | 32360/225000 [72:32:47<347:48:32,  6.50s/it]
 14%|█▍        | 32361/225000 [72:32:52<317:22:33,  5.93s/it]
 14%|█▍        | 32362/225000 [72:32:56<290:27:59,  5.43s/it]
 14%|█▍        | 32363/225000 [72:33:08<391:59:13,  7.33s/it]
 14%|█▍        | 32364/225000 [72:33:11<326:51:10,  6.11s/it]
 14%|█▍        | 32365/225000 [72:33:18<338:06:27,  6.32s/it]
 14%|█▍        | 32366/225000 [72:33:26<371:39:52,  6.95s/it]
 14%|█▍        | 32367/225000 [72:33:29<312:30:00,  5.84s/it]
 14%|█▍        | 32368/225000 [72:33:37<346:00:01,  6.47s/it]
 14%|█▍        | 32369/225000 [72:33:44<352:49:37,  6.59s/it]
 14%|█▍        | 32370/225000 [72:33:51<362:04:49,  6.77s/it]
 14%|█▍        | 32371/225000 [72:33:56<328:35:12,  6.14s/it]
 14%|█▍        | 32372/225000 [72:34:02<332:22:12,  6.21s/it]
 14%|█▍        | 32373/225000 [72:34:11<375:01:44,  7.01s/it]
 14%|█▍        | 32374/225000 [72:34:16<336:57:30,  6.30s/it]
 14%|█▍        | 32375/225000 [72:34:24<366:39:07,  6.85s/it]
 14%|█▍        | 32376/225000 [72:34:29<329:34:20,  6.16s/it]
 14%|█▍        | 32377/225000 [72:34:33<302:38:19,  5.66s/it]
 14%|█▍        | 32378/225000 [72:34:38<291:01:42,  5.44s/it]
 14%|█▍        | 32379/225000 [72:34:49<376:50:44,  7.04s/it]
 14%|█▍        | 32380/225000 [72:34:55<357:16:05,  6.68s/it]
 14%|█▍        | 32381/225000 [72:34:59<317:37:28,  5.94s/it]
 14%|█▍        | 32382/225000 [72:35:08<377:21:08,  7.05s/it]
 14%|█▍        | 32383/225000 [72:35:15<372:38:34,  6.96s/it]
 14%|█▍        | 32384/225000 [72:35:19<319:13:19,  5.97s/it]
 14%|█▍        | 32385/225000 [72:35:23<295:31:52,  5.52s/it]
 14%|█▍        | 32386/225000 [72:35:28<281:17:29,  5.26s/it]
 14%|█▍        | 32387/225000 [72:35:41<399:10:23,  7.46s/it]
 14%|█▍        | 32388/225000 [72:35:44<340:40:12,  6.37s/it]
 14%|█▍        | 32389/225000 [72:35:54<386:42:48,  7.23s/it]
 14%|█▍        | 32390/225000 [72:35:57<329:05:53,  6.15s/it]
 14%|█▍        | 32391/225000 [72:36:07<384:19:23,  7.18s/it]
 14%|█▍        | 32392/225000 [72:36:13<370:44:52,  6.93s/it]
 14%|█▍        | 32393/225000 [72:36:17<324:52:44,  6.07s/it]
 14%|█▍        | 32394/225000 [72:36:22<300:33:11,  5.62s/it]
 14%|█▍        | 32395/225000 [72:36:31<355:39:17,  6.65s/it]
 14%|█▍        | 32396/225000 [72:36:38<355:57:29,  6.65s/it]
 14%|█▍        | 32397/225000 [72:36:42<315:42:53,  5.90s/it]
 14%|█▍        | 32398/225000 [72:36:46<292:42:11,  5.47s/it]
 14%|█▍        | 32399/225000 [72:36:51<288:05:51,  5.38s/it]
 14%|█▍        | 32400/225000 [72:36:56<278:46:00,  5.21s/it]
                                                             
{'loss': '0.7177', 'grad_norm': '0.7917', 'learning_rate': '4.867e-05', 'epoch': '0.144', 'num_input_tokens_seen': 123878600, 'train_runtime': '2.614e+05', 'train_tokens_per_second': '473.9'}

 14%|█▍        | 32400/225000 [72:36:56<278:46:00,  5.21s/it]
 14%|█▍        | 32401/225000 [72:37:01<275:21:36,  5.15s/it]
 14%|█▍        | 32402/225000 [72:37:14<396:38:02,  7.41s/it]
 14%|█▍        | 32403/225000 [72:37:20<369:06:54,  6.90s/it]
 14%|█▍        | 32404/225000 [72:37:25<348:09:47,  6.51s/it]
 14%|█▍        | 32405/225000 [72:37:33<368:49:32,  6.89s/it]
 14%|█▍        | 32406/225000 [72:37:39<354:05:17,  6.62s/it]
 14%|█▍        | 32407/225000 [72:37:46<362:24:35,  6.77s/it]
 14%|█▍        | 32408/225000 [72:37:54<378:01:06,  7.07s/it]
 14%|█▍        | 32409/225000 [72:37:58<333:54:06,  6.24s/it]
 14%|█▍        | 32410/225000 [72:38:02<302:49:53,  5.66s/it]
 14%|█▍        | 32411/225000 [72:38:13<388:07:59,  7.26s/it]
 14%|█▍        | 32412/225000 [72:38:18<348:09:58,  6.51s/it]
 14%|█▍        | 32413/225000 [72:38:23<320:50:09,  6.00s/it]
 14%|█▍        | 32414/225000 [72:38:28<301:28:03,  5.64s/it]
 14%|█▍        | 32415/225000 [72:38:34<307:53:31,  5.76s/it]
 14%|█▍        | 32416/225000 [72:38:46<417:45:20,  7.81s/it]
 14%|█▍        | 32417/225000 [72:38:50<355:20:43,  6.64s/it]
 14%|█▍        | 32418/225000 [72:39:00<405:15:00,  7.58s/it]
 14%|█▍        | 32419/225000 [72:39:04<352:28:11,  6.59s/it]
 14%|█▍        | 32420/225000 [72:39:12<376:15:12,  7.03s/it]
 14%|█▍        | 32421/225000 [72:39:17<332:51:13,  6.22s/it]
 14%|█▍        | 32422/225000 [72:39:21<300:40:14,  5.62s/it]
 14%|█▍        | 32423/225000 [72:39:33<397:05:19,  7.42s/it]
 14%|█▍        | 32424/225000 [72:39:36<340:37:56,  6.37s/it]
 14%|█▍        | 32425/225000 [72:39:45<373:36:00,  6.98s/it]
 14%|█▍        | 32426/225000 [72:39:50<351:23:14,  6.57s/it]
 14%|█▍        | 32427/225000 [72:39:55<325:05:20,  6.08s/it]
 14%|█▍        | 32428/225000 [72:40:00<308:20:42,  5.76s/it]
 14%|█▍        | 32429/225000 [72:40:05<289:20:57,  5.41s/it]
 14%|█▍        | 32430/225000 [72:40:10<287:53:22,  5.38s/it]
 14%|█▍        | 32431/225000 [72:40:18<317:30:47,  5.94s/it]
 14%|█▍        | 32432/225000 [72:40:31<431:37:15,  8.07s/it]
 14%|█▍        | 32433/225000 [72:40:35<369:52:50,  6.91s/it]
 14%|█▍        | 32434/225000 [72:40:41<349:43:20,  6.54s/it]
 14%|█▍        | 32435/225000 [72:40:46<330:44:04,  6.18s/it]
 14%|█▍        | 32436/225000 [72:40:51<311:19:36,  5.82s/it]
 14%|█▍        | 32437/225000 [72:40:56<295:09:44,  5.52s/it]
 14%|█▍        | 32438/225000 [72:41:07<380:39:36,  7.12s/it]
 14%|█▍        | 32439/225000 [72:41:12<356:51:38,  6.67s/it]
 14%|█▍        | 32440/225000 [72:41:16<311:48:48,  5.83s/it]
 14%|█▍        | 32441/225000 [72:41:26<380:33:09,  7.11s/it]
 14%|█▍        | 32442/225000 [72:41:31<343:14:10,  6.42s/it]
 14%|█▍        | 32443/225000 [72:41:36<315:17:37,  5.89s/it]
 14%|█▍        | 32444/225000 [72:41:40<294:05:31,  5.50s/it]
 14%|█▍        | 32445/225000 [72:41:53<412:09:53,  7.71s/it]
 14%|█▍        | 32446/225000 [72:41:58<366:00:08,  6.84s/it]
 14%|█▍        | 32447/225000 [72:42:03<331:07:19,  6.19s/it]
 14%|█▍        | 32448/225000 [72:42:07<304:51:46,  5.70s/it]
 14%|█▍        | 32449/225000 [72:42:17<366:29:20,  6.85s/it]
 14%|█▍        | 32450/225000 [72:42:24<378:20:18,  7.07s/it]
                                                             
{'loss': '0.7262', 'grad_norm': '0.7331', 'learning_rate': '4.867e-05', 'epoch': '0.1442', 'num_input_tokens_seen': 124024400, 'train_runtime': '2.617e+05', 'train_tokens_per_second': '473.8'}

 14%|█▍        | 32450/225000 [72:42:24<378:20:18,  7.07s/it]
 14%|█▍        | 32451/225000 [72:42:28<332:22:00,  6.21s/it]
 14%|█▍        | 32452/225000 [72:42:32<296:42:12,  5.55s/it]
 14%|█▍        | 32453/225000 [72:42:42<358:20:53,  6.70s/it]
 14%|█▍        | 32454/225000 [72:42:49<371:24:12,  6.94s/it]
 14%|█▍        | 32455/225000 [72:42:54<331:52:57,  6.21s/it]
 14%|█▍        | 32456/225000 [72:42:59<322:57:52,  6.04s/it]
 14%|█▍        | 32457/225000 [72:43:09<386:07:10,  7.22s/it]
 14%|█▍        | 32458/225000 [72:43:13<333:30:54,  6.24s/it]
 14%|█▍        | 32459/225000 [72:43:20<340:33:08,  6.37s/it]
 14%|█▍        | 32460/225000 [72:43:28<369:47:57,  6.91s/it]
 14%|█▍        | 32461/225000 [72:43:34<345:27:50,  6.46s/it]
 14%|█▍        | 32462/225000 [72:43:38<316:17:55,  5.91s/it]
 14%|█▍        | 32463/225000 [72:43:45<335:30:22,  6.27s/it]
 14%|█▍        | 32464/225000 [72:43:51<319:02:08,  5.97s/it]
 14%|█▍        | 32465/225000 [72:43:56<311:25:06,  5.82s/it]
 14%|█▍        | 32466/225000 [72:44:04<352:19:57,  6.59s/it]
 14%|█▍        | 32467/225000 [72:44:12<364:39:06,  6.82s/it]
 14%|█▍        | 32468/225000 [72:44:19<368:37:43,  6.89s/it]
 14%|█▍        | 32469/225000 [72:44:27<391:02:40,  7.31s/it]
 14%|█▍        | 32470/225000 [72:44:33<359:57:53,  6.73s/it]
 14%|█▍        | 32471/225000 [72:44:38<341:07:11,  6.38s/it]
 14%|█▍        | 32472/225000 [72:44:44<330:23:29,  6.18s/it]
 14%|█▍        | 32473/225000 [72:44:49<319:47:11,  5.98s/it]
 14%|█▍        | 32474/225000 [72:44:55<314:02:40,  5.87s/it]
 14%|█▍        | 32475/225000 [72:45:01<309:06:12,  5.78s/it]
 14%|█▍        | 32476/225000 [72:45:06<299:54:13,  5.61s/it]
 14%|█▍        | 32477/225000 [72:45:16<370:52:41,  6.94s/it]
 14%|█▍        | 32478/225000 [72:45:26<415:54:19,  7.78s/it]
 14%|█▍        | 32479/225000 [72:45:30<370:07:46,  6.92s/it]
 14%|█▍        | 32480/225000 [72:45:36<349:36:54,  6.54s/it]
 14%|█▍        | 32481/225000 [72:45:41<328:27:00,  6.14s/it]
 14%|█▍        | 32482/225000 [72:45:47<318:44:25,  5.96s/it]
 14%|█▍        | 32483/225000 [72:45:57<390:57:50,  7.31s/it]
 14%|█▍        | 32484/225000 [72:46:04<379:45:05,  7.10s/it]
 14%|█▍        | 32485/225000 [72:46:11<383:49:56,  7.18s/it]
 14%|█▍        | 32486/225000 [72:46:16<344:19:50,  6.44s/it]
 14%|█▍        | 32487/225000 [72:46:21<326:44:57,  6.11s/it]
 14%|█▍        | 32488/225000 [72:46:27<314:55:25,  5.89s/it]
 14%|█▍        | 32489/225000 [72:46:34<336:06:12,  6.29s/it]
 14%|█▍        | 32490/225000 [72:46:45<411:52:40,  7.70s/it]
 14%|█▍        | 32491/225000 [72:46:53<424:48:49,  7.94s/it]
 14%|█▍        | 32492/225000 [72:46:57<358:16:57,  6.70s/it]
 14%|█▍        | 32493/225000 [72:47:06<388:01:36,  7.26s/it]
 14%|█▍        | 32494/225000 [72:47:10<344:16:01,  6.44s/it]
 14%|█▍        | 32495/225000 [72:47:15<315:03:55,  5.89s/it]
 14%|█▍        | 32496/225000 [72:47:20<294:08:16,  5.50s/it]
 14%|█▍        | 32497/225000 [72:47:26<302:44:27,  5.66s/it]
 14%|█▍        | 32498/225000 [72:47:36<386:39:31,  7.23s/it]
 14%|█▍        | 32499/225000 [72:47:40<332:04:39,  6.21s/it]
 14%|█▍        | 32500/225000 [72:47:49<374:40:59,  7.01s/it]
                                                             
{'loss': '0.7234', 'grad_norm': '0.622', 'learning_rate': '4.866e-05', 'epoch': '0.1444', 'num_input_tokens_seen': 124171040, 'train_runtime': '2.621e+05', 'train_tokens_per_second': '473.8'}

 14%|█▍        | 32500/225000 [72:47:49<374:40:59,  7.01s/it]
 14%|█▍        | 32501/225000 [72:47:55<358:18:42,  6.70s/it]
 14%|█▍        | 32502/225000 [72:48:00<328:36:23,  6.15s/it]
 14%|█▍        | 32503/225000 [72:48:09<372:20:07,  6.96s/it]
 14%|█▍        | 32504/225000 [72:48:13<323:19:55,  6.05s/it]
 14%|█▍        | 32505/225000 [72:48:20<347:24:12,  6.50s/it]
 14%|█▍        | 32506/225000 [72:48:24<304:06:18,  5.69s/it]
 14%|█▍        | 32507/225000 [72:48:32<339:58:06,  6.36s/it]
 14%|█▍        | 32508/225000 [72:48:40<361:26:31,  6.76s/it]
 14%|█▍        | 32509/225000 [72:48:46<350:12:27,  6.55s/it]
 14%|█▍        | 32510/225000 [72:48:53<366:42:50,  6.86s/it]
 14%|█▍        | 32511/225000 [72:48:57<321:51:26,  6.02s/it]
 14%|█▍        | 32512/225000 [72:49:02<295:48:48,  5.53s/it]
 14%|█▍        | 32513/225000 [72:49:08<301:00:58,  5.63s/it]
 14%|█▍        | 32514/225000 [72:49:17<360:23:59,  6.74s/it]
 14%|█▍        | 32515/225000 [72:49:21<322:40:55,  6.04s/it]
 14%|█▍        | 32516/225000 [72:49:27<315:13:58,  5.90s/it]
 14%|█▍        | 32517/225000 [72:49:32<299:07:21,  5.59s/it]
 14%|█▍        | 32518/225000 [72:49:37<291:51:28,  5.46s/it]
 14%|█▍        | 32519/225000 [72:49:47<370:23:35,  6.93s/it]
 14%|█▍        | 32520/225000 [72:49:55<384:30:22,  7.19s/it]
 14%|█▍        | 32521/225000 [72:50:03<393:55:54,  7.37s/it]
 14%|█▍        | 32522/225000 [72:50:10<389:36:20,  7.29s/it]
 14%|█▍        | 32523/225000 [72:50:15<346:32:50,  6.48s/it]
 14%|█▍        | 32524/225000 [72:50:20<329:02:41,  6.15s/it]
 14%|█▍        | 32525/225000 [72:50:26<324:58:00,  6.08s/it]
 14%|█▍        | 32526/225000 [72:50:32<317:28:47,  5.94s/it]
 14%|█▍        | 32527/225000 [72:50:37<303:57:26,  5.69s/it]
 14%|█▍        | 32528/225000 [72:50:42<297:54:26,  5.57s/it]
 14%|█▍        | 32529/225000 [72:50:47<288:52:50,  5.40s/it]
 14%|█▍        | 32530/225000 [72:50:59<390:59:27,  7.31s/it]
 14%|█▍        | 32531/225000 [72:51:07<401:06:02,  7.50s/it]
 14%|█▍        | 32532/225000 [72:51:11<358:22:30,  6.70s/it]
 14%|█▍        | 32533/225000 [72:51:16<324:02:47,  6.06s/it]
 14%|█▍        | 32534/225000 [72:51:28<415:53:48,  7.78s/it]
 14%|█▍        | 32535/225000 [72:51:32<354:41:00,  6.63s/it]
 14%|█▍        | 32536/225000 [72:51:40<383:01:56,  7.16s/it]
 14%|█▍        | 32537/225000 [72:51:44<326:04:34,  6.10s/it]
 14%|█▍        | 32538/225000 [72:51:54<398:21:23,  7.45s/it]
 14%|█▍        | 32539/225000 [72:51:58<339:44:19,  6.35s/it]
 14%|█▍        | 32540/225000 [72:52:06<355:27:13,  6.65s/it]
 14%|█▍        | 32541/225000 [72:52:12<347:54:51,  6.51s/it]
 14%|█▍        | 32542/225000 [72:52:20<383:55:10,  7.18s/it]
 14%|█▍        | 32543/225000 [72:52:24<327:50:31,  6.13s/it]
 14%|█▍        | 32544/225000 [72:52:29<308:49:45,  5.78s/it]
 14%|█▍        | 32545/225000 [72:52:34<286:31:00,  5.36s/it]
 14%|█▍        | 32546/225000 [72:52:39<288:05:08,  5.39s/it]
 14%|█▍        | 32547/225000 [72:52:44<283:35:11,  5.30s/it]
 14%|█▍        | 32548/225000 [72:52:52<324:11:32,  6.06s/it]
 14%|█▍        | 32549/225000 [72:53:04<419:32:10,  7.85s/it]
 14%|█▍        | 32550/225000 [72:53:08<355:33:31,  6.65s/it]
                                                             
{'loss': '0.7134', 'grad_norm': '0.7978', 'learning_rate': '4.866e-05', 'epoch': '0.1447', 'num_input_tokens_seen': 124317368, 'train_runtime': '2.624e+05', 'train_tokens_per_second': '473.8'}

 14%|█▍        | 32550/225000 [72:53:08<355:33:31,  6.65s/it]
 14%|█▍        | 32551/225000 [72:53:13<334:53:10,  6.26s/it]
 14%|█▍        | 32552/225000 [72:53:18<319:57:32,  5.99s/it]
 14%|█▍        | 32553/225000 [72:53:25<334:05:59,  6.25s/it]
 14%|█▍        | 32554/225000 [72:53:34<379:08:13,  7.09s/it]
 14%|█▍        | 32555/225000 [72:53:40<347:44:15,  6.51s/it]
 14%|█▍        | 32556/225000 [72:53:45<331:12:00,  6.20s/it]
 14%|█▍        | 32557/225000 [72:53:51<320:07:01,  5.99s/it]
 14%|█▍        | 32558/225000 [72:53:57<321:50:51,  6.02s/it]
 14%|█▍        | 32559/225000 [72:54:02<314:01:50,  5.87s/it]
 14%|█▍        | 32560/225000 [72:54:14<415:20:31,  7.77s/it]
 14%|█▍        | 32561/225000 [72:54:20<380:18:28,  7.11s/it]
 14%|█▍        | 32562/225000 [72:54:26<357:27:27,  6.69s/it]
 14%|█▍        | 32563/225000 [72:54:32<358:01:32,  6.70s/it]
 14%|█▍        | 32564/225000 [72:54:41<389:36:20,  7.29s/it]
 14%|█▍        | 32565/225000 [72:54:45<339:44:06,  6.36s/it]
 14%|█▍        | 32566/225000 [72:54:56<408:12:14,  7.64s/it]
 14%|█▍        | 32567/225000 [72:55:02<387:51:03,  7.26s/it]
 14%|█▍        | 32568/225000 [72:55:08<366:29:07,  6.86s/it]
 14%|█▍        | 32569/225000 [72:55:16<385:34:23,  7.21s/it]
 14%|█▍        | 32570/225000 [72:55:22<362:17:32,  6.78s/it]
 14%|█▍        | 32571/225000 [72:55:29<370:01:02,  6.92s/it]
 14%|█▍        | 32572/225000 [72:55:33<327:06:15,  6.12s/it]
 14%|█▍        | 32573/225000 [72:55:44<400:29:04,  7.49s/it]
 14%|█▍        | 32574/225000 [72:55:49<359:16:20,  6.72s/it]
 14%|█▍        | 32575/225000 [72:55:54<328:43:43,  6.15s/it]
 14%|█▍        | 32576/225000 [72:55:59<304:50:59,  5.70s/it]
 14%|█▍        | 32577/225000 [72:56:04<306:31:08,  5.73s/it]
 14%|█▍        | 32578/225000 [72:56:10<308:17:10,  5.77s/it]
 14%|█▍        | 32579/225000 [72:56:23<425:35:11,  7.96s/it]
 14%|█▍        | 32580/225000 [72:56:28<374:24:53,  7.00s/it]
 14%|█▍        | 32581/225000 [72:56:32<327:28:28,  6.13s/it]
 14%|█▍        | 32582/225000 [72:56:43<403:51:57,  7.56s/it]
 14%|█▍        | 32583/225000 [72:56:47<350:17:29,  6.55s/it]
 14%|█▍        | 32584/225000 [72:56:52<323:11:54,  6.05s/it]
 14%|█▍        | 32585/225000 [72:56:57<311:39:12,  5.83s/it]
 14%|█▍        | 32586/225000 [72:57:03<306:55:02,  5.74s/it]
 14%|█▍        | 32587/225000 [72:57:08<292:46:25,  5.48s/it]
 14%|█▍        | 32588/225000 [72:57:13<289:12:44,  5.41s/it]
 14%|█▍        | 32589/225000 [72:57:26<415:17:39,  7.77s/it]
 14%|█▍        | 32590/225000 [72:57:32<386:40:09,  7.23s/it]
 14%|█▍        | 32591/225000 [72:57:36<335:13:56,  6.27s/it]
 14%|█▍        | 32592/225000 [72:57:43<337:37:13,  6.32s/it]
 14%|█▍        | 32593/225000 [72:57:53<399:14:41,  7.47s/it]
 14%|█▍        | 32594/225000 [72:57:57<337:39:55,  6.32s/it]
 14%|█▍        | 32595/225000 [72:58:05<378:43:14,  7.09s/it]
 14%|█▍        | 32596/225000 [72:58:10<336:45:07,  6.30s/it]
 14%|█▍        | 32597/225000 [72:58:18<371:33:24,  6.95s/it]
 14%|█▍        | 32598/225000 [72:58:23<332:42:54,  6.23s/it]
 14%|█▍        | 32599/225000 [72:58:29<324:19:04,  6.07s/it]
 14%|█▍        | 32600/225000 [72:58:34<310:01:24,  5.80s/it]
                                                             
{'loss': '0.7331', 'grad_norm': '0.6936', 'learning_rate': '4.865e-05', 'epoch': '0.1449', 'num_input_tokens_seen': 124465120, 'train_runtime': '2.627e+05', 'train_tokens_per_second': '473.8'}

 14%|█▍        | 32600/225000 [72:58:34<310:01:24,  5.80s/it]
 14%|█▍        | 32601/225000 [72:58:40<310:09:25,  5.80s/it]
 14%|█▍        | 32602/225000 [72:58:44<295:02:22,  5.52s/it]
 14%|█▍        | 32603/225000 [72:58:51<309:43:46,  5.80s/it]
 14%|█▍        | 32604/225000 [72:59:03<417:19:31,  7.81s/it]
 14%|█▍        | 32605/225000 [72:59:08<369:29:22,  6.91s/it]
 14%|█▍        | 32606/225000 [72:59:13<328:25:55,  6.15s/it]
 14%|█▍        | 32607/225000 [72:59:17<301:40:26,  5.64s/it]
 14%|█▍        | 32608/225000 [72:59:28<382:10:29,  7.15s/it]
 14%|█▍        | 32609/225000 [72:59:34<370:11:07,  6.93s/it]
 14%|█▍        | 32610/225000 [72:59:44<413:47:22,  7.74s/it]
 14%|█▍        | 32611/225000 [72:59:48<351:41:45,  6.58s/it]
 14%|█▍        | 32612/225000 [72:59:56<383:55:51,  7.18s/it]
 14%|█▍        | 32613/225000 [73:00:01<340:37:38,  6.37s/it]
 14%|█▍        | 32614/225000 [73:00:07<345:32:49,  6.47s/it]
 14%|█▍        | 32615/225000 [73:00:16<376:11:47,  7.04s/it]
 14%|█▍        | 32616/225000 [73:00:20<330:30:29,  6.18s/it]
 14%|█▍        | 32617/225000 [73:00:28<358:32:54,  6.71s/it]
 14%|█▍        | 32618/225000 [73:00:32<319:09:01,  5.97s/it]
 14%|█▍        | 32619/225000 [73:00:43<401:11:47,  7.51s/it]
 14%|█▍        | 32620/225000 [73:00:47<344:14:39,  6.44s/it]
 14%|█▍        | 32621/225000 [73:00:56<387:16:25,  7.25s/it]
 14%|█▍        | 32622/225000 [73:01:00<330:07:41,  6.18s/it]
 14%|█▍        | 32623/225000 [73:01:04<294:16:23,  5.51s/it]
 14%|█▍        | 32624/225000 [73:01:14<364:58:59,  6.83s/it]
 14%|█▍        | 32625/225000 [73:01:18<323:45:11,  6.06s/it]
 15%|█▍        | 32626/225000 [73:01:23<302:22:34,  5.66s/it]
 15%|█▍        | 32627/225000 [73:01:28<295:57:59,  5.54s/it]
 15%|█▍        | 32628/225000 [73:01:33<278:49:17,  5.22s/it]
 15%|█▍        | 32629/225000 [73:01:43<359:13:37,  6.72s/it]
 15%|█▍        | 32630/225000 [73:01:50<373:36:27,  6.99s/it]
 15%|█▍        | 32631/225000 [73:01:55<334:01:10,  6.25s/it]
 15%|█▍        | 32632/225000 [73:02:00<310:08:20,  5.80s/it]
 15%|█▍        | 32633/225000 [73:02:04<286:43:09,  5.37s/it]
 15%|█▍        | 32634/225000 [73:02:17<403:44:47,  7.56s/it]
 15%|█▍        | 32635/225000 [73:02:22<367:15:29,  6.87s/it]
 15%|█▍        | 32636/225000 [73:02:30<386:29:00,  7.23s/it]
 15%|█▍        | 32637/225000 [73:02:35<352:10:39,  6.59s/it]
 15%|█▍        | 32638/225000 [73:02:41<337:42:47,  6.32s/it]
 15%|█▍        | 32639/225000 [73:02:46<316:02:05,  5.91s/it]
 15%|█▍        | 32640/225000 [73:02:51<305:28:37,  5.72s/it]
 15%|█▍        | 32641/225000 [73:02:56<293:17:36,  5.49s/it]
 15%|█▍        | 32642/225000 [73:03:02<293:42:49,  5.50s/it]
 15%|█▍        | 32643/225000 [73:03:06<279:45:49,  5.24s/it]
 15%|█▍        | 32644/225000 [73:03:13<301:57:55,  5.65s/it]
 15%|█▍        | 32645/225000 [73:03:24<389:43:14,  7.29s/it]
 15%|█▍        | 32646/225000 [73:03:29<355:37:39,  6.66s/it]
 15%|█▍        | 32647/225000 [73:03:34<334:34:47,  6.26s/it]
 15%|█▍        | 32648/225000 [73:03:39<315:37:45,  5.91s/it]
 15%|█▍        | 32649/225000 [73:03:52<415:23:17,  7.77s/it]
 15%|█▍        | 32650/225000 [73:03:55<348:56:50,  6.53s/it]
                                                             
{'loss': '0.7274', 'grad_norm': '0.8064', 'learning_rate': '4.865e-05', 'epoch': '0.1451', 'num_input_tokens_seen': 124609384, 'train_runtime': '2.63e+05', 'train_tokens_per_second': '473.7'}

 15%|█▍        | 32650/225000 [73:03:55<348:56:50,  6.53s/it]
 15%|█▍        | 32651/225000 [73:04:03<368:33:48,  6.90s/it]
 15%|█▍        | 32652/225000 [73:04:07<319:07:50,  5.97s/it]
 15%|█▍        | 32653/225000 [73:04:17<390:57:51,  7.32s/it]
 15%|█▍        | 32654/225000 [73:04:22<351:00:53,  6.57s/it]
 15%|█▍        | 32655/225000 [73:04:27<324:07:13,  6.07s/it]
 15%|█▍        | 32656/225000 [73:04:32<300:52:59,  5.63s/it]
 15%|█▍        | 32657/225000 [73:04:36<287:56:38,  5.39s/it]
 15%|█▍        | 32658/225000 [73:04:41<276:40:13,  5.18s/it]
 15%|█▍        | 32659/225000 [73:04:49<314:48:32,  5.89s/it]
 15%|█▍        | 32660/225000 [73:04:59<383:45:28,  7.18s/it]
 15%|█▍        | 32661/225000 [73:05:02<324:44:39,  6.08s/it]
 15%|█▍        | 32662/225000 [73:05:11<372:52:37,  6.98s/it]
 15%|█▍        | 32663/225000 [73:05:15<325:30:39,  6.09s/it]
 15%|█▍        | 32664/225000 [73:05:22<340:13:46,  6.37s/it]
 15%|█▍        | 32665/225000 [73:05:28<328:24:49,  6.15s/it]
 15%|█▍        | 32666/225000 [73:05:35<338:49:28,  6.34s/it]
 15%|█▍        | 32667/225000 [73:05:41<326:47:37,  6.12s/it]
 15%|█▍        | 32668/225000 [73:05:46<311:58:49,  5.84s/it]
 15%|█▍        | 32669/225000 [73:05:51<300:02:52,  5.62s/it]
 15%|█▍        | 32670/225000 [73:05:56<298:43:05,  5.59s/it]
 15%|█▍        | 32671/225000 [73:06:01<291:08:28,  5.45s/it]
 15%|█▍        | 32672/225000 [73:06:14<409:01:18,  7.66s/it]
 15%|█▍        | 32673/225000 [73:06:22<413:10:27,  7.73s/it]
 15%|█▍        | 32674/225000 [73:06:29<400:41:25,  7.50s/it]
 15%|█▍        | 32675/225000 [73:06:35<372:59:01,  6.98s/it]
 15%|█▍        | 32676/225000 [73:06:40<350:06:06,  6.55s/it]
 15%|█▍        | 32677/225000 [73:06:45<324:57:18,  6.08s/it]
 15%|█▍        | 32678/225000 [73:06:51<311:20:43,  5.83s/it]
 15%|█▍        | 32679/225000 [73:06:56<302:53:01,  5.67s/it]
 15%|█▍        | 32680/225000 [73:07:01<293:34:17,  5.50s/it]
 15%|█▍        | 32681/225000 [73:07:06<289:52:42,  5.43s/it]
 15%|█▍        | 32682/225000 [73:07:18<396:31:26,  7.42s/it]
 15%|█▍        | 32683/225000 [73:07:25<390:51:03,  7.32s/it]
 15%|█▍        | 32684/225000 [73:07:31<360:08:36,  6.74s/it]
 15%|█▍        | 32685/225000 [73:07:38<364:23:39,  6.82s/it]
 15%|█▍        | 32686/225000 [73:07:43<337:57:02,  6.33s/it]
 15%|█▍        | 32687/225000 [73:07:49<324:25:24,  6.07s/it]
 15%|█▍        | 32688/225000 [73:07:54<320:14:29,  5.99s/it]
 15%|█▍        | 32689/225000 [73:07:59<306:23:29,  5.74s/it]
 15%|█▍        | 32690/225000 [73:08:08<357:31:43,  6.69s/it]
 15%|█▍        | 32691/225000 [73:08:19<416:54:32,  7.80s/it]
 15%|█▍        | 32692/225000 [73:08:23<359:19:38,  6.73s/it]
 15%|█▍        | 32693/225000 [73:08:28<329:43:08,  6.17s/it]
 15%|█▍        | 32694/225000 [73:08:38<392:43:05,  7.35s/it]
 15%|█▍        | 32695/225000 [73:08:43<361:29:48,  6.77s/it]
 15%|█▍        | 32696/225000 [73:08:49<342:18:04,  6.41s/it]
 15%|█▍        | 32697/225000 [73:08:54<323:34:08,  6.06s/it]
 15%|█▍        | 32698/225000 [73:08:59<302:12:37,  5.66s/it]
 15%|█▍        | 32699/225000 [73:09:04<296:57:28,  5.56s/it]
 15%|█▍        | 32700/225000 [73:09:17<406:17:34,  7.61s/it]
                                                             
{'loss': '0.7339', 'grad_norm': '0.7073', 'learning_rate': '4.864e-05', 'epoch': '0.1453', 'num_input_tokens_seen': 124753504, 'train_runtime': '2.634e+05', 'train_tokens_per_second': '473.7'}

 15%|█▍        | 32700/225000 [73:09:17<406:17:34,  7.61s/it]
 15%|█▍        | 32701/225000 [73:09:20<339:26:16,  6.35s/it]
 15%|█▍        | 32702/225000 [73:09:28<371:55:24,  6.96s/it]
 15%|█▍        | 32703/225000 [73:09:36<380:45:40,  7.13s/it]
 15%|█▍        | 32704/225000 [73:09:40<324:41:17,  6.08s/it]
 15%|█▍        | 32705/225000 [73:09:48<358:07:15,  6.70s/it]
 15%|█▍        | 32706/225000 [73:09:54<358:31:32,  6.71s/it]
 15%|█▍        | 32707/225000 [73:10:01<354:40:04,  6.64s/it]
 15%|█▍        | 32708/225000 [73:10:05<308:34:30,  5.78s/it]
 15%|█▍        | 32709/225000 [73:10:14<371:26:00,  6.95s/it]
 15%|█▍        | 32710/225000 [73:10:19<326:42:46,  6.12s/it]
 15%|█▍        | 32711/225000 [73:10:23<296:36:51,  5.55s/it]
 15%|█▍        | 32712/225000 [73:10:33<374:20:29,  7.01s/it]
 15%|█▍        | 32713/225000 [73:10:37<321:06:22,  6.01s/it]
 15%|█▍        | 32714/225000 [73:10:45<355:41:53,  6.66s/it]
 15%|█▍        | 32715/225000 [73:10:53<367:53:09,  6.89s/it]
 15%|█▍        | 32716/225000 [73:10:58<346:54:32,  6.49s/it]
 15%|█▍        | 32717/225000 [73:11:03<326:14:27,  6.11s/it]
 15%|█▍        | 32718/225000 [73:11:09<321:16:42,  6.02s/it]
 15%|█▍        | 32719/225000 [73:11:15<314:41:13,  5.89s/it]
 15%|█▍        | 32720/225000 [73:11:20<303:42:35,  5.69s/it]
 15%|█▍        | 32721/225000 [73:11:27<331:17:33,  6.20s/it]
 15%|█▍        | 32722/225000 [73:11:35<351:47:26,  6.59s/it]
 15%|█▍        | 32723/225000 [73:11:39<314:09:42,  5.88s/it]
 15%|█▍        | 32724/225000 [73:11:44<305:15:12,  5.72s/it]
 15%|█▍        | 32725/225000 [73:11:50<302:12:47,  5.66s/it]
 15%|█▍        | 32726/225000 [73:11:56<302:48:48,  5.67s/it]
 15%|█▍        | 32727/225000 [73:12:05<355:05:04,  6.65s/it]
 15%|█▍        | 32728/225000 [73:12:16<436:36:44,  8.17s/it]
 15%|█▍        | 32729/225000 [73:12:20<369:43:45,  6.92s/it]
 15%|█▍        | 32730/225000 [73:12:24<326:43:27,  6.12s/it]
 15%|█▍        | 32731/225000 [73:12:35<398:27:50,  7.46s/it]
 15%|█▍        | 32732/225000 [73:12:40<360:57:38,  6.76s/it]
 15%|█▍        | 32733/225000 [73:12:44<319:38:28,  5.98s/it]
 15%|█▍        | 32734/225000 [73:12:50<310:27:18,  5.81s/it]
 15%|█▍        | 32735/225000 [73:12:55<307:38:54,  5.76s/it]
 15%|█▍        | 32736/225000 [73:13:01<304:52:05,  5.71s/it]
 15%|█▍        | 32737/225000 [73:13:07<302:12:52,  5.66s/it]
 15%|█▍        | 32738/225000 [73:13:17<378:18:29,  7.08s/it]
 15%|█▍        | 32739/225000 [73:13:28<438:04:22,  8.20s/it]
 15%|█▍        | 32740/225000 [73:13:32<378:51:36,  7.09s/it]
 15%|█▍        | 32741/225000 [73:13:41<400:22:56,  7.50s/it]
 15%|█▍        | 32742/225000 [73:13:45<353:36:19,  6.62s/it]
 15%|█▍        | 32743/225000 [73:13:50<325:08:38,  6.09s/it]
 15%|█▍        | 32744/225000 [73:13:54<296:30:22,  5.55s/it]
 15%|█▍        | 32745/225000 [73:13:59<283:48:11,  5.31s/it]
 15%|█▍        | 32746/225000 [73:14:11<384:07:23,  7.19s/it]
 15%|█▍        | 32747/225000 [73:14:17<361:34:33,  6.77s/it]
 15%|█▍        | 32748/225000 [73:14:23<356:10:30,  6.67s/it]
 15%|█▍        | 32749/225000 [73:14:32<397:04:02,  7.44s/it]
 15%|█▍        | 32750/225000 [73:14:36<343:25:39,  6.43s/it]
                                                             
{'loss': '0.7448', 'grad_norm': '0.6589', 'learning_rate': '4.863e-05', 'epoch': '0.1456', 'num_input_tokens_seen': 124900000, 'train_runtime': '2.637e+05', 'train_tokens_per_second': '473.7'}

 15%|█▍        | 32750/225000 [73:14:36<343:25:39,  6.43s/it]
 15%|█▍        | 32751/225000 [73:14:41<323:05:35,  6.05s/it]
 15%|█▍        | 32752/225000 [73:14:50<357:06:07,  6.69s/it]
 15%|█▍        | 32753/225000 [73:14:57<362:09:59,  6.78s/it]
 15%|█▍        | 32754/225000 [73:15:02<334:33:32,  6.26s/it]
 15%|█▍        | 32755/225000 [73:15:08<327:08:05,  6.13s/it]
 15%|█▍        | 32756/225000 [73:15:14<330:06:22,  6.18s/it]
 15%|█▍        | 32757/225000 [73:15:18<297:34:55,  5.57s/it]
 15%|█▍        | 32758/225000 [73:15:26<338:41:16,  6.34s/it]
 15%|█▍        | 32759/225000 [73:15:31<308:22:45,  5.77s/it]
 15%|█▍        | 32760/225000 [73:15:36<296:46:35,  5.56s/it]
 15%|█▍        | 32761/225000 [73:15:46<366:24:28,  6.86s/it]
 15%|█▍        | 32762/225000 [73:15:54<385:58:26,  7.23s/it]
 15%|█▍        | 32763/225000 [73:15:58<336:20:54,  6.30s/it]
 15%|█▍        | 32764/225000 [73:16:02<306:58:42,  5.75s/it]
 15%|█▍        | 32765/225000 [73:16:15<422:14:28,  7.91s/it]
 15%|█▍        | 32766/225000 [73:16:19<363:36:32,  6.81s/it]
 15%|█▍        | 32767/225000 [73:16:27<379:12:54,  7.10s/it]
 15%|█▍        | 32768/225000 [73:16:32<338:26:35,  6.34s/it]
 15%|█▍        | 32769/225000 [73:16:37<314:35:00,  5.89s/it]
 15%|█▍        | 32770/225000 [73:16:45<349:15:27,  6.54s/it]
 15%|█▍        | 32771/225000 [73:16:51<351:50:33,  6.59s/it]
 15%|█▍        | 32772/225000 [73:16:58<360:07:21,  6.74s/it]
 15%|█▍        | 32773/225000 [73:17:05<361:51:46,  6.78s/it]
 15%|█▍        | 32774/225000 [73:17:10<320:31:44,  6.00s/it]
 15%|█▍        | 32775/225000 [73:17:14<298:25:39,  5.59s/it]
 15%|█▍        | 32776/225000 [73:17:25<376:17:46,  7.05s/it]
 15%|█▍        | 32777/225000 [73:17:30<343:19:21,  6.43s/it]
 15%|█▍        | 32778/225000 [73:17:34<316:03:14,  5.92s/it]
 15%|█▍        | 32779/225000 [73:17:39<299:31:18,  5.61s/it]
 15%|█▍        | 32780/225000 [73:17:45<296:19:29,  5.55s/it]
 15%|█▍        | 32781/225000 [73:17:49<285:11:03,  5.34s/it]
 15%|█▍        | 32782/225000 [73:17:55<284:58:06,  5.34s/it]
 15%|█▍        | 32783/225000 [73:18:03<338:50:14,  6.35s/it]
 15%|█▍        | 32784/225000 [73:18:14<399:27:21,  7.48s/it]
 15%|█▍        | 32785/225000 [73:18:18<349:42:45,  6.55s/it]
 15%|█▍        | 32786/225000 [73:18:22<311:06:06,  5.83s/it]
 15%|█▍        | 32787/225000 [73:18:27<288:08:25,  5.40s/it]
 15%|█▍        | 32788/225000 [73:18:32<282:53:48,  5.30s/it]
 15%|█▍        | 32789/225000 [73:18:44<403:40:49,  7.56s/it]
 15%|█▍        | 32790/225000 [73:18:48<341:58:55,  6.41s/it]
 15%|█▍        | 32791/225000 [73:18:56<358:44:52,  6.72s/it]
 15%|█▍        | 32792/225000 [73:18:59<306:20:06,  5.74s/it]
 15%|█▍        | 32793/225000 [73:19:09<375:09:08,  7.03s/it]
 15%|█▍        | 32794/225000 [73:19:13<321:45:08,  6.03s/it]
 15%|█▍        | 32795/225000 [73:19:18<307:55:11,  5.77s/it]
 15%|█▍        | 32796/225000 [73:19:23<296:04:06,  5.55s/it]
 15%|█▍        | 32797/225000 [73:19:28<291:46:12,  5.46s/it]
 15%|█▍        | 32798/225000 [73:19:41<411:14:27,  7.70s/it]
 15%|█▍        | 32799/225000 [73:19:47<378:28:59,  7.09s/it]
 15%|█▍        | 32800/225000 [73:19:51<337:42:33,  6.33s/it]
                                                             
{'loss': '0.7337', 'grad_norm': '0.6854', 'learning_rate': '4.863e-05', 'epoch': '0.1458', 'num_input_tokens_seen': 125044848, 'train_runtime': '2.64e+05', 'train_tokens_per_second': '473.7'}

 15%|█▍        | 32800/225000 [73:19:51<337:42:33,  6.33s/it]
 15%|█▍        | 32801/225000 [73:19:56<316:26:13,  5.93s/it]
 15%|█▍        | 32802/225000 [73:20:04<347:01:54,  6.50s/it]
 15%|█▍        | 32803/225000 [73:20:14<394:43:38,  7.39s/it]
 15%|█▍        | 32804/225000 [73:20:17<331:46:30,  6.21s/it]
 15%|█▍        | 32805/225000 [73:20:26<377:47:55,  7.08s/it]
 15%|█▍        | 32806/225000 [73:20:30<329:04:45,  6.16s/it]
 15%|█▍        | 32807/225000 [73:20:39<373:05:10,  6.99s/it]
 15%|█▍        | 32808/225000 [73:20:43<325:49:10,  6.10s/it]
 15%|█▍        | 32809/225000 [73:20:51<353:03:05,  6.61s/it]
 15%|█▍        | 32810/225000 [73:20:57<350:17:36,  6.56s/it]
 15%|█▍        | 32811/225000 [73:21:01<301:36:22,  5.65s/it]
 15%|█▍        | 32812/225000 [73:21:12<390:22:19,  7.31s/it]
 15%|█▍        | 32813/225000 [73:21:17<350:47:21,  6.57s/it]
 15%|█▍        | 32814/225000 [73:21:22<322:16:07,  6.04s/it]
 15%|█▍        | 32815/225000 [73:21:26<293:20:33,  5.49s/it]
 15%|█▍        | 32816/225000 [73:21:31<279:31:12,  5.24s/it]
 15%|█▍        | 32817/225000 [73:21:43<390:14:22,  7.31s/it]
 15%|█▍        | 32818/225000 [73:21:49<366:13:16,  6.86s/it]
 15%|█▍        | 32819/225000 [73:21:53<329:06:53,  6.17s/it]
 15%|█▍        | 32820/225000 [73:21:57<292:07:31,  5.47s/it]
 15%|█▍        | 32821/225000 [73:22:06<348:52:38,  6.54s/it]
 15%|█▍        | 32822/225000 [73:22:14<374:35:29,  7.02s/it]
 15%|█▍        | 32823/225000 [73:22:18<325:30:42,  6.10s/it]
 15%|█▍        | 32824/225000 [73:22:26<356:36:40,  6.68s/it]
 15%|█▍        | 32825/225000 [73:22:35<394:24:44,  7.39s/it]
 15%|█▍        | 32826/225000 [73:22:43<393:43:39,  7.38s/it]
 15%|█▍        | 32827/225000 [73:22:47<344:45:10,  6.46s/it]
 15%|█▍        | 32828/225000 [73:22:51<311:31:14,  5.84s/it]
 15%|█▍        | 32829/225000 [73:23:01<379:42:57,  7.11s/it]
 15%|█▍        | 32830/225000 [73:23:06<336:11:43,  6.30s/it]
 15%|█▍        | 32831/225000 [73:23:14<368:35:07,  6.90s/it]
 15%|█▍        | 32832/225000 [73:23:18<318:37:29,  5.97s/it]
 15%|█▍        | 32833/225000 [73:23:22<287:02:46,  5.38s/it]
 15%|█▍        | 32834/225000 [73:23:32<370:09:25,  6.93s/it]
 15%|█▍        | 32835/225000 [73:23:37<338:44:29,  6.35s/it]
 15%|█▍        | 32836/225000 [73:23:47<386:32:55,  7.24s/it]
 15%|█▍        | 32837/225000 [73:23:51<336:29:58,  6.30s/it]
 15%|█▍        | 32838/225000 [73:23:54<290:29:42,  5.44s/it]
 15%|█▍        | 32839/225000 [73:24:05<375:56:46,  7.04s/it]
 15%|█▍        | 32840/225000 [73:24:10<347:17:11,  6.51s/it]
 15%|█▍        | 32841/225000 [73:24:16<337:56:51,  6.33s/it]
 15%|█▍        | 32842/225000 [73:24:24<360:05:08,  6.75s/it]
 15%|█▍        | 32843/225000 [73:24:30<342:26:00,  6.42s/it]
 15%|█▍        | 32844/225000 [73:24:35<324:53:16,  6.09s/it]
 15%|█▍        | 32845/225000 [73:24:40<317:06:58,  5.94s/it]
 15%|█▍        | 32846/225000 [73:24:51<382:24:10,  7.16s/it]
 15%|█▍        | 32847/225000 [73:24:55<332:41:54,  6.23s/it]
 15%|█▍        | 32848/225000 [73:24:58<294:07:36,  5.51s/it]
 15%|█▍        | 32849/225000 [73:25:10<389:00:55,  7.29s/it]
 15%|█▍        | 32850/225000 [73:25:14<345:07:22,  6.47s/it]
                                                             
{'loss': '0.7326', 'grad_norm': '0.7125', 'learning_rate': '4.862e-05', 'epoch': '0.146', 'num_input_tokens_seen': 125191488, 'train_runtime': '2.643e+05', 'train_tokens_per_second': '473.6'}

 15%|█▍        | 32850/225000 [73:25:14<345:07:22,  6.47s/it]
 15%|█▍        | 32851/225000 [73:25:21<354:37:29,  6.64s/it]
 15%|█▍        | 32852/225000 [73:25:25<311:19:01,  5.83s/it]
 15%|█▍        | 32853/225000 [73:25:30<296:53:33,  5.56s/it]
 15%|█▍        | 32854/225000 [73:25:36<295:05:13,  5.53s/it]
 15%|█▍        | 32855/225000 [73:25:41<294:33:52,  5.52s/it]
 15%|█▍        | 32856/225000 [73:25:47<294:10:12,  5.51s/it]
 15%|█▍        | 32857/225000 [73:25:52<297:07:41,  5.57s/it]
 15%|█▍        | 32858/225000 [73:25:59<305:09:33,  5.72s/it]
 15%|█▍        | 32859/225000 [73:26:07<343:16:42,  6.43s/it]
 15%|█▍        | 32860/225000 [73:26:20<450:54:01,  8.45s/it]
 15%|█▍        | 32861/225000 [73:26:25<393:51:04,  7.38s/it]
 15%|█▍        | 32862/225000 [73:26:34<417:26:13,  7.82s/it]
 15%|█▍        | 32863/225000 [73:26:38<365:39:13,  6.85s/it]
 15%|█▍        | 32864/225000 [73:26:43<336:07:17,  6.30s/it]
 15%|█▍        | 32865/225000 [73:26:52<372:58:20,  6.99s/it]
 15%|█▍        | 32866/225000 [73:26:59<373:37:12,  7.00s/it]
 15%|█▍        | 32867/225000 [73:27:04<339:50:11,  6.37s/it]
 15%|█▍        | 32868/225000 [73:27:13<383:16:29,  7.18s/it]
 15%|█▍        | 32869/225000 [73:27:16<325:05:32,  6.09s/it]
 15%|█▍        | 32870/225000 [73:27:26<378:26:51,  7.09s/it]
 15%|█▍        | 32871/225000 [73:27:29<325:22:19,  6.10s/it]
 15%|█▍        | 32872/225000 [73:27:39<374:19:05,  7.01s/it]
 15%|█▍        | 32873/225000 [73:27:43<329:53:44,  6.18s/it]
 15%|█▍        | 32874/225000 [73:27:52<370:00:35,  6.93s/it]
 15%|█▍        | 32875/225000 [73:27:56<329:08:20,  6.17s/it]
 15%|█▍        | 32876/225000 [73:28:04<362:11:02,  6.79s/it]
 15%|█▍        | 32877/225000 [73:28:09<326:07:59,  6.11s/it]
 15%|█▍        | 32878/225000 [73:28:16<350:40:57,  6.57s/it]
 15%|█▍        | 32879/225000 [73:28:23<354:07:17,  6.64s/it]
 15%|█▍        | 32880/225000 [73:28:28<320:33:56,  6.01s/it]
 15%|█▍        | 32881/225000 [73:28:35<336:09:00,  6.30s/it]
 15%|█▍        | 32882/225000 [73:28:40<313:45:37,  5.88s/it]
 15%|█▍        | 32883/225000 [73:28:45<306:59:31,  5.75s/it]
 15%|█▍        | 32884/225000 [73:28:50<299:03:05,  5.60s/it]
 15%|█▍        | 32885/225000 [73:28:56<295:43:13,  5.54s/it]
 15%|█▍        | 32886/225000 [73:29:02<304:03:18,  5.70s/it]
 15%|█▍        | 32887/225000 [73:29:08<309:05:00,  5.79s/it]
 15%|█▍        | 32888/225000 [73:29:14<310:05:26,  5.81s/it]
 15%|█▍        | 32889/225000 [73:29:19<302:55:13,  5.68s/it]
 15%|█▍        | 32890/225000 [73:29:25<305:00:04,  5.72s/it]
 15%|█▍        | 32891/225000 [73:29:37<405:23:05,  7.60s/it]
 15%|█▍        | 32892/225000 [73:29:45<411:21:27,  7.71s/it]
 15%|█▍        | 32893/225000 [73:29:53<423:06:34,  7.93s/it]
 15%|█▍        | 32894/225000 [73:29:58<372:16:01,  6.98s/it]
 15%|█▍        | 32895/225000 [73:30:02<332:39:23,  6.23s/it]
 15%|█▍        | 32896/225000 [73:30:07<301:12:50,  5.64s/it]
 15%|█▍        | 32897/225000 [73:30:18<398:14:43,  7.46s/it]
 15%|█▍        | 32898/225000 [73:30:24<363:04:27,  6.80s/it]
 15%|█▍        | 32899/225000 [73:30:30<351:31:44,  6.59s/it]
 15%|█▍        | 32900/225000 [73:30:37<368:02:07,  6.90s/it]
                                                             
{'loss': '0.7435', 'grad_norm': '0.6976', 'learning_rate': '4.862e-05', 'epoch': '0.1462', 'num_input_tokens_seen': 125337416, 'train_runtime': '2.646e+05', 'train_tokens_per_second': '473.6'}

 15%|█▍        | 32900/225000 [73:30:37<368:02:07,  6.90s/it]
 15%|█▍        | 32901/225000 [73:30:41<318:02:28,  5.96s/it]
 15%|█▍        | 32902/225000 [73:30:50<371:05:13,  6.95s/it]
 15%|█▍        | 32903/225000 [73:30:57<371:32:15,  6.96s/it]
 15%|█▍        | 32904/225000 [73:31:02<330:30:44,  6.19s/it]
 15%|█▍        | 32905/225000 [73:31:08<324:51:48,  6.09s/it]
 15%|█▍        | 32906/225000 [73:31:13<315:31:39,  5.91s/it]
 15%|█▍        | 32907/225000 [73:31:19<307:47:59,  5.77s/it]
 15%|█▍        | 32908/225000 [73:31:24<302:57:02,  5.68s/it]
 15%|█▍        | 32909/225000 [73:31:34<373:23:58,  7.00s/it]
 15%|█▍        | 32910/225000 [73:31:41<367:13:16,  6.88s/it]
 15%|█▍        | 32911/225000 [73:31:45<323:21:16,  6.06s/it]
 15%|█▍        | 32912/225000 [73:31:55<394:17:02,  7.39s/it]
 15%|█▍        | 32913/225000 [73:31:59<340:42:15,  6.39s/it]
 15%|█▍        | 32914/225000 [73:32:08<381:24:16,  7.15s/it]
 15%|█▍        | 32915/225000 [73:32:13<344:00:46,  6.45s/it]
 15%|█▍        | 32916/225000 [73:32:19<329:13:00,  6.17s/it]
 15%|█▍        | 32917/225000 [73:32:23<307:35:27,  5.76s/it]
 15%|█▍        | 32918/225000 [73:32:28<293:15:35,  5.50s/it]
 15%|█▍        | 32919/225000 [73:32:33<275:07:38,  5.16s/it]
 15%|█▍        | 32920/225000 [73:32:41<328:03:54,  6.15s/it]
 15%|█▍        | 32921/225000 [73:32:53<421:26:57,  7.90s/it]
 15%|█▍        | 32922/225000 [73:33:00<404:56:05,  7.59s/it]
 15%|█▍        | 32923/225000 [73:33:04<347:15:40,  6.51s/it]
 15%|█▍        | 32924/225000 [73:33:12<379:01:02,  7.10s/it]
 15%|█▍        | 32925/225000 [73:33:17<345:06:03,  6.47s/it]
 15%|█▍        | 32926/225000 [73:33:22<312:48:55,  5.86s/it]
 15%|█▍        | 32927/225000 [73:33:27<300:20:40,  5.63s/it]
 15%|█▍        | 32928/225000 [73:33:32<298:08:01,  5.59s/it]
 15%|█▍        | 32929/225000 [73:33:38<290:46:10,  5.45s/it]
 15%|█▍        | 32930/225000 [73:33:52<427:02:42,  8.00s/it]
 15%|█▍        | 32931/225000 [73:33:55<351:12:16,  6.58s/it]
 15%|█▍        | 32932/225000 [73:34:01<349:38:08,  6.55s/it]
 15%|█▍        | 32933/225000 [73:34:07<335:24:16,  6.29s/it]
 15%|█▍        | 32934/225000 [73:34:16<375:55:34,  7.05s/it]
 15%|█▍        | 32935/225000 [73:34:20<330:00:07,  6.19s/it]
 15%|█▍        | 32936/225000 [73:34:29<379:42:26,  7.12s/it]
 15%|█▍        | 32937/225000 [73:34:33<324:01:46,  6.07s/it]
 15%|█▍        | 32938/225000 [73:34:38<308:10:14,  5.78s/it]
 15%|█▍        | 32939/225000 [73:34:44<306:32:35,  5.75s/it]
 15%|█▍        | 32940/225000 [73:34:49<302:35:50,  5.67s/it]
 15%|█▍        | 32941/225000 [73:34:55<301:49:42,  5.66s/it]
 15%|█▍        | 32942/225000 [73:35:00<299:10:37,  5.61s/it]
 15%|█▍        | 32943/225000 [73:35:06<299:04:59,  5.61s/it]
 15%|█▍        | 32944/225000 [73:35:19<419:28:21,  7.86s/it]
 15%|█▍        | 32945/225000 [73:35:26<403:24:23,  7.56s/it]
 15%|█▍        | 32946/225000 [73:35:30<348:06:17,  6.53s/it]
 15%|█▍        | 32947/225000 [73:35:34<314:35:18,  5.90s/it]
 15%|█▍        | 32948/225000 [73:35:43<352:30:58,  6.61s/it]
 15%|█▍        | 32949/225000 [73:35:51<387:01:31,  7.25s/it]
 15%|█▍        | 32950/225000 [73:36:01<421:05:39,  7.89s/it]
                                                             
{'loss': '0.7264', 'grad_norm': '0.6676', 'learning_rate': '4.861e-05', 'epoch': '0.1464', 'num_input_tokens_seen': 125484296, 'train_runtime': '2.65e+05', 'train_tokens_per_second': '473.6'}

 15%|█▍        | 32950/225000 [73:36:01<421:05:39,  7.89s/it]
 15%|█▍        | 32951/225000 [73:36:05<357:04:34,  6.69s/it]
 15%|█▍        | 32952/225000 [73:36:10<333:12:08,  6.25s/it]
 15%|█▍        | 32953/225000 [73:36:15<318:58:40,  5.98s/it]
 15%|█▍        | 32954/225000 [73:36:21<308:29:09,  5.78s/it]
 15%|█▍        | 32955/225000 [73:36:26<305:07:25,  5.72s/it]
 15%|█▍        | 32956/225000 [73:36:32<300:30:47,  5.63s/it]
 15%|█▍        | 32957/225000 [73:36:37<297:51:21,  5.58s/it]
 15%|█▍        | 32958/225000 [73:36:45<332:37:43,  6.24s/it]
 15%|█▍        | 32959/225000 [73:36:58<449:05:56,  8.42s/it]
 15%|█▍        | 32960/225000 [73:37:04<399:37:56,  7.49s/it]
 15%|█▍        | 32961/225000 [73:37:08<346:11:49,  6.49s/it]
 15%|█▍        | 32962/225000 [73:37:21<445:36:30,  8.35s/it]
 15%|█▍        | 32963/225000 [73:37:24<369:03:52,  6.92s/it]
 15%|█▍        | 32964/225000 [73:37:29<339:29:31,  6.36s/it]
 15%|█▍        | 32965/225000 [73:37:35<324:19:37,  6.08s/it]
 15%|█▍        | 32966/225000 [73:37:40<307:26:09,  5.76s/it]
 15%|█▍        | 32967/225000 [73:37:45<304:02:35,  5.70s/it]
 15%|█▍        | 32968/225000 [73:37:51<305:37:09,  5.73s/it]
 15%|█▍        | 32969/225000 [73:37:56<301:09:11,  5.65s/it]
 15%|█▍        | 32970/225000 [73:38:02<303:40:18,  5.69s/it]
 15%|█▍        | 32971/225000 [73:38:15<417:59:47,  7.84s/it]
 15%|█▍        | 32972/225000 [73:38:22<404:40:09,  7.59s/it]
 15%|█▍        | 32973/225000 [73:38:28<380:51:37,  7.14s/it]
 15%|█▍        | 32974/225000 [73:38:36<391:27:30,  7.34s/it]
 15%|█▍        | 32975/225000 [73:38:41<350:09:26,  6.56s/it]
 15%|█▍        | 32976/225000 [73:38:46<331:26:02,  6.21s/it]
 15%|█▍        | 32977/225000 [73:38:52<322:27:26,  6.05s/it]
 15%|█▍        | 32978/225000 [73:38:57<305:35:36,  5.73s/it]
 15%|█▍        | 32979/225000 [73:39:02<297:51:39,  5.58s/it]
 15%|█▍        | 32980/225000 [73:39:15<420:55:27,  7.89s/it]
 15%|█▍        | 32981/225000 [73:39:22<396:11:07,  7.43s/it]
 15%|█▍        | 32982/225000 [73:39:28<384:51:40,  7.22s/it]
 15%|█▍        | 32983/225000 [73:39:37<411:35:52,  7.72s/it]
 15%|█▍        | 32984/225000 [73:39:41<343:14:32,  6.44s/it]
 15%|█▍        | 32985/225000 [73:39:50<389:12:44,  7.30s/it]
 15%|█▍        | 32986/225000 [73:39:55<356:25:52,  6.68s/it]
 15%|█▍        | 32987/225000 [73:39:59<310:11:58,  5.82s/it]
 15%|█▍        | 32988/225000 [73:40:07<346:40:07,  6.50s/it]
 15%|█▍        | 32989/225000 [73:40:13<332:30:25,  6.23s/it]
 15%|█▍        | 32990/225000 [73:40:21<357:14:56,  6.70s/it]
 15%|█▍        | 32991/225000 [73:40:28<368:31:35,  6.91s/it]
 15%|█▍        | 32992/225000 [73:40:34<355:29:22,  6.67s/it]
 15%|█▍        | 32993/225000 [73:40:40<343:56:31,  6.45s/it]
 15%|█▍        | 32994/225000 [73:40:45<329:18:01,  6.17s/it]
 15%|█▍        | 32995/225000 [73:40:51<324:30:13,  6.08s/it]
 15%|█▍        | 32996/225000 [73:40:57<313:59:30,  5.89s/it]
 15%|█▍        | 32997/225000 [73:41:03<311:32:11,  5.84s/it]
 15%|█▍        | 32998/225000 [73:41:08<313:09:43,  5.87s/it]
 15%|█▍        | 32999/225000 [73:41:17<355:49:04,  6.67s/it]
 15%|█▍        | 33000/225000 [73:41:23<342:10:30,  6.42s/it]
                                                             
{'loss': '0.7158', 'grad_norm': '0.683', 'learning_rate': '4.86e-05', 'epoch': '0.1467', 'num_input_tokens_seen': 125632240, 'train_runtime': '2.653e+05', 'train_tokens_per_second': '473.6'}

 15%|█▍        | 33000/225000 [73:41:23<342:10:30,  6.42s/it]
 15%|█▍        | 33001/225000 [73:41:32<393:27:28,  7.38s/it]
 15%|█▍        | 33002/225000 [73:41:38<358:27:35,  6.72s/it]
 15%|█▍        | 33003/225000 [73:41:43<340:14:14,  6.38s/it]
 15%|█▍        | 33004/225000 [73:41:48<321:29:56,  6.03s/it]
 15%|█▍        | 33005/225000 [73:42:00<404:16:05,  7.58s/it]
 15%|█▍        | 33006/225000 [73:42:05<367:29:59,  6.89s/it]
 15%|█▍        | 33007/225000 [73:42:09<316:58:35,  5.94s/it]
 15%|█▍        | 33008/225000 [73:42:20<396:27:45,  7.43s/it]
 15%|█▍        | 33009/225000 [73:42:24<346:50:15,  6.50s/it]
 15%|█▍        | 33010/225000 [73:42:31<355:16:15,  6.66s/it]
 15%|█▍        | 33011/225000 [73:42:38<366:27:41,  6.87s/it]
 15%|█▍        | 33012/225000 [73:42:44<343:44:46,  6.45s/it]
 15%|█▍        | 33013/225000 [73:42:49<318:39:03,  5.98s/it]
 15%|█▍        | 33014/225000 [73:42:53<300:24:37,  5.63s/it]
 15%|█▍        | 33015/225000 [73:42:59<295:27:43,  5.54s/it]
 15%|█▍        | 33016/225000 [73:43:04<290:44:00,  5.45s/it]
 15%|█▍        | 33017/225000 [73:43:16<392:36:39,  7.36s/it]
 15%|█▍        | 33018/225000 [73:43:23<384:13:35,  7.20s/it]
 15%|█▍        | 33019/225000 [73:43:27<343:19:38,  6.44s/it]
 15%|█▍        | 33020/225000 [73:43:32<315:25:53,  5.91s/it]
 15%|█▍        | 33021/225000 [73:43:37<296:20:19,  5.56s/it]
 15%|█▍        | 33022/225000 [73:43:47<368:56:38,  6.92s/it]
 15%|█▍        | 33023/225000 [73:43:56<405:42:09,  7.61s/it]
 15%|█▍        | 33024/225000 [73:44:00<352:01:36,  6.60s/it]
 15%|█▍        | 33025/225000 [73:44:09<382:25:02,  7.17s/it]
 15%|█▍        | 33026/225000 [73:44:14<355:29:03,  6.67s/it]
 15%|█▍        | 33027/225000 [73:44:21<351:02:32,  6.58s/it]
 15%|█▍        | 33028/225000 [73:44:26<329:12:20,  6.17s/it]
 15%|█▍        | 33029/225000 [73:44:31<315:07:37,  5.91s/it]
 15%|█▍        | 33030/225000 [73:44:36<303:49:37,  5.70s/it]
 15%|█▍        | 33031/225000 [73:44:42<300:55:36,  5.64s/it]
 15%|█▍        | 33032/225000 [73:44:47<299:18:09,  5.61s/it]
 15%|█▍        | 33033/225000 [73:44:53<297:38:25,  5.58s/it]
 15%|█▍        | 33034/225000 [73:44:58<295:28:57,  5.54s/it]
 15%|█▍        | 33035/225000 [73:45:06<324:38:02,  6.09s/it]
 15%|█▍        | 33036/225000 [73:45:18<428:02:24,  8.03s/it]
 15%|█▍        | 33037/225000 [73:45:23<380:34:09,  7.14s/it]
 15%|█▍        | 33038/225000 [73:45:28<345:07:05,  6.47s/it]
 15%|█▍        | 33039/225000 [73:45:33<322:19:17,  6.04s/it]
 15%|█▍        | 33040/225000 [73:45:39<316:19:02,  5.93s/it]
 15%|█▍        | 33041/225000 [73:45:44<298:28:11,  5.60s/it]
 15%|█▍        | 33042/225000 [73:45:57<412:54:26,  7.74s/it]
 15%|█▍        | 33043/225000 [73:46:03<392:56:46,  7.37s/it]
 15%|█▍        | 33044/225000 [73:46:07<337:36:51,  6.33s/it]
 15%|█▍        | 33045/225000 [73:46:12<310:32:16,  5.82s/it]
 15%|█▍        | 33046/225000 [73:46:18<322:07:08,  6.04s/it]
 15%|█▍        | 33047/225000 [73:46:30<413:12:00,  7.75s/it]
 15%|█▍        | 33048/225000 [73:46:33<345:34:59,  6.48s/it]
 15%|█▍        | 33049/225000 [73:46:42<381:28:31,  7.15s/it]
 15%|█▍        | 33050/225000 [73:46:46<334:42:29,  6.28s/it]
                                                             
{'loss': '0.7435', 'grad_norm': '0.6942', 'learning_rate': '4.86e-05', 'epoch': '0.1469', 'num_input_tokens_seen': 125777784, 'train_runtime': '2.656e+05', 'train_tokens_per_second': '473.5'}

 15%|█▍        | 33050/225000 [73:46:46<334:42:29,  6.28s/it]
 15%|█▍        | 33051/225000 [73:46:54<349:05:23,  6.55s/it]
 15%|█▍        | 33052/225000 [73:47:01<364:56:36,  6.84s/it]
 15%|█▍        | 33053/225000 [73:47:08<363:13:16,  6.81s/it]
 15%|█▍        | 33054/225000 [73:47:13<337:24:45,  6.33s/it]
 15%|█▍        | 33055/225000 [73:47:21<363:22:46,  6.82s/it]
 15%|█▍        | 33056/225000 [73:47:25<319:15:12,  5.99s/it]
 15%|█▍        | 33057/225000 [73:47:31<311:05:43,  5.83s/it]
 15%|█▍        | 33058/225000 [73:47:36<305:49:25,  5.74s/it]
 15%|█▍        | 33059/225000 [73:47:42<303:43:16,  5.70s/it]
 15%|█▍        | 33060/225000 [73:47:47<297:53:42,  5.59s/it]
 15%|█▍        | 33061/225000 [73:47:52<289:33:28,  5.43s/it]
 15%|█▍        | 33062/225000 [73:47:58<293:46:03,  5.51s/it]
 15%|█▍        | 33063/225000 [73:48:08<363:14:03,  6.81s/it]
 15%|█▍        | 33064/225000 [73:48:13<345:45:59,  6.49s/it]
 15%|█▍        | 33065/225000 [73:48:18<323:51:23,  6.07s/it]
 15%|█▍        | 33066/225000 [73:48:24<312:21:53,  5.86s/it]
 15%|█▍        | 33067/225000 [73:48:30<317:50:51,  5.96s/it]
 15%|█▍        | 33068/225000 [73:48:43<438:07:44,  8.22s/it]
 15%|█▍        | 33069/225000 [73:48:48<385:55:08,  7.24s/it]
 15%|█▍        | 33070/225000 [73:48:53<348:38:05,  6.54s/it]
 15%|█▍        | 33071/225000 [73:48:59<329:16:08,  6.18s/it]
 15%|█▍        | 33072/225000 [73:49:04<317:41:35,  5.96s/it]
 15%|█▍        | 33073/225000 [73:49:13<366:05:55,  6.87s/it]
 15%|█▍        | 33074/225000 [73:49:23<408:36:59,  7.66s/it]
 15%|█▍        | 33075/225000 [73:49:30<410:32:45,  7.70s/it]
 15%|█▍        | 33076/225000 [73:49:35<358:28:04,  6.72s/it]
 15%|█▍        | 33077/225000 [73:49:40<338:03:29,  6.34s/it]
 15%|█▍        | 33078/225000 [73:49:46<322:59:21,  6.06s/it]
 15%|█▍        | 33079/225000 [73:49:51<313:49:54,  5.89s/it]
 15%|█▍        | 33080/225000 [73:49:57<311:07:36,  5.84s/it]
 15%|█▍        | 33081/225000 [73:50:06<367:11:35,  6.89s/it]
 15%|█▍        | 33082/225000 [73:50:16<416:53:58,  7.82s/it]
 15%|█▍        | 33083/225000 [73:50:22<380:00:59,  7.13s/it]
 15%|█▍        | 33084/225000 [73:50:27<341:55:15,  6.41s/it]
 15%|█▍        | 33085/225000 [73:50:32<327:17:05,  6.14s/it]
 15%|█▍        | 33086/225000 [73:50:38<317:31:41,  5.96s/it]
 15%|█▍        | 33087/225000 [73:50:43<307:36:38,  5.77s/it]
 15%|█▍        | 33088/225000 [73:50:48<301:36:37,  5.66s/it]
 15%|█▍        | 33089/225000 [73:50:54<303:21:11,  5.69s/it]
 15%|█▍        | 33090/225000 [73:51:07<417:03:22,  7.82s/it]
 15%|█▍        | 33091/225000 [73:51:11<366:19:46,  6.87s/it]
 15%|█▍        | 33092/225000 [73:51:21<411:53:03,  7.73s/it]
 15%|█▍        | 33093/225000 [73:51:25<350:29:51,  6.58s/it]
 15%|█▍        | 33094/225000 [73:51:30<318:36:28,  5.98s/it]
 15%|█▍        | 33095/225000 [73:51:36<323:46:22,  6.07s/it]
 15%|█▍        | 33096/225000 [73:51:46<389:39:03,  7.31s/it]
 15%|█▍        | 33097/225000 [73:51:50<337:36:39,  6.33s/it]
 15%|█▍        | 33098/225000 [73:51:55<315:31:56,  5.92s/it]
 15%|█▍        | 33099/225000 [73:52:07<411:29:20,  7.72s/it]
 15%|█▍        | 33100/225000 [73:52:12<365:24:49,  6.86s/it]
                                                             
{'loss': '0.7288', 'grad_norm': '0.6256', 'learning_rate': '4.859e-05', 'epoch': '0.1471', 'num_input_tokens_seen': 125924496, 'train_runtime': '2.659e+05', 'train_tokens_per_second': '473.5'}

 15%|█▍        | 33100/225000 [73:52:12<365:24:49,  6.86s/it]
 15%|█▍        | 33101/225000 [73:52:18<351:13:16,  6.59s/it]
 15%|█▍        | 33102/225000 [73:52:25<359:45:16,  6.75s/it]
 15%|█▍        | 33103/225000 [73:52:31<352:58:18,  6.62s/it]
 15%|█▍        | 33104/225000 [73:52:37<335:10:13,  6.29s/it]
 15%|█▍        | 33105/225000 [73:52:46<385:13:28,  7.23s/it]
 15%|█▍        | 33106/225000 [73:52:50<332:50:07,  6.24s/it]
 15%|█▍        | 33107/225000 [73:52:59<376:21:48,  7.06s/it]
 15%|█▍        | 33108/225000 [73:53:03<323:44:31,  6.07s/it]
 15%|█▍        | 33109/225000 [73:53:12<372:51:31,  7.00s/it]
 15%|█▍        | 33110/225000 [73:53:17<340:36:11,  6.39s/it]
 15%|█▍        | 33111/225000 [73:53:22<312:31:24,  5.86s/it]
 15%|█▍        | 33112/225000 [73:53:27<299:54:11,  5.63s/it]
 15%|█▍        | 33113/225000 [73:53:32<290:54:02,  5.46s/it]
 15%|█▍        | 33114/225000 [73:53:37<284:59:55,  5.35s/it]
 15%|█▍        | 33115/225000 [73:53:50<402:33:09,  7.55s/it]
 15%|█▍        | 33116/225000 [73:53:54<352:11:33,  6.61s/it]
 15%|█▍        | 33117/225000 [73:54:02<378:22:05,  7.10s/it]
 15%|█▍        | 33118/225000 [73:54:09<376:12:41,  7.06s/it]
 15%|█▍        | 33119/225000 [73:54:17<380:16:49,  7.13s/it]
 15%|█▍        | 33120/225000 [73:54:23<370:06:37,  6.94s/it]
 15%|█▍        | 33121/225000 [73:54:27<328:42:12,  6.17s/it]
 15%|█▍        | 33122/225000 [73:54:32<301:48:38,  5.66s/it]
 15%|█▍        | 33123/225000 [73:54:41<357:59:42,  6.72s/it]
 15%|█▍        | 33124/225000 [73:54:49<375:40:10,  7.05s/it]
 15%|█▍        | 33125/225000 [73:54:58<405:33:52,  7.61s/it]
 15%|█▍        | 33126/225000 [73:55:02<352:43:13,  6.62s/it]
 15%|█▍        | 33127/225000 [73:55:11<387:52:50,  7.28s/it]
 15%|█▍        | 33128/225000 [73:55:16<355:04:25,  6.66s/it]
 15%|█▍        | 33129/225000 [73:55:21<320:19:49,  6.01s/it]
 15%|█▍        | 33130/225000 [73:55:25<295:32:34,  5.55s/it]
 15%|█▍        | 33131/225000 [73:55:35<364:50:04,  6.85s/it]
 15%|█▍        | 33132/225000 [73:55:42<367:33:48,  6.90s/it]
 15%|█▍        | 33133/225000 [73:55:46<318:01:24,  5.97s/it]
 15%|█▍        | 33134/225000 [73:55:55<374:08:34,  7.02s/it]
 15%|█▍        | 33135/225000 [73:56:01<347:23:58,  6.52s/it]
 15%|█▍        | 33136/225000 [73:56:06<327:49:36,  6.15s/it]
 15%|█▍        | 33137/225000 [73:56:11<317:16:15,  5.95s/it]
 15%|█▍        | 33138/225000 [73:56:17<314:55:45,  5.91s/it]
 15%|█▍        | 33139/225000 [73:56:22<303:26:21,  5.69s/it]
 15%|█▍        | 33140/225000 [73:56:27<293:28:09,  5.51s/it]
 15%|█▍        | 33141/225000 [73:56:33<298:16:09,  5.60s/it]
 15%|█▍        | 33142/225000 [73:56:45<394:30:31,  7.40s/it]
 15%|█▍        | 33143/225000 [73:56:51<379:13:46,  7.12s/it]
 15%|█▍        | 33144/225000 [73:57:00<407:30:37,  7.65s/it]
 15%|█▍        | 33145/225000 [73:57:05<359:33:29,  6.75s/it]
 15%|█▍        | 33146/225000 [73:57:13<382:04:18,  7.17s/it]
 15%|█▍        | 33147/225000 [73:57:19<362:52:01,  6.81s/it]
 15%|█▍        | 33148/225000 [73:57:24<331:20:16,  6.22s/it]
 15%|█▍        | 33149/225000 [73:57:32<366:08:15,  6.87s/it]
 15%|█▍        | 33150/225000 [73:57:40<378:38:25,  7.11s/it]
                                                             
{'loss': '0.7287', 'grad_norm': '0.7271', 'learning_rate': '4.859e-05', 'epoch': '0.1473', 'num_input_tokens_seen': 126070832, 'train_runtime': '2.663e+05', 'train_tokens_per_second': '473.5'}

 15%|█▍        | 33150/225000 [73:57:40<378:38:25,  7.11s/it]
 15%|█▍        | 33151/225000 [73:57:47<382:14:10,  7.17s/it]
 15%|█▍        | 33152/225000 [73:57:54<377:31:00,  7.08s/it]
 15%|█▍        | 33153/225000 [73:58:00<350:34:51,  6.58s/it]
 15%|█▍        | 33154/225000 [73:58:07<364:33:46,  6.84s/it]
 15%|█▍        | 33155/225000 [73:58:11<322:04:33,  6.04s/it]
 15%|█▍        | 33156/225000 [73:58:21<385:07:49,  7.23s/it]
 15%|█▍        | 33157/225000 [73:58:26<341:54:43,  6.42s/it]
 15%|█▍        | 33158/225000 [73:58:31<317:32:27,  5.96s/it]
 15%|█▍        | 33159/225000 [73:58:38<339:25:28,  6.37s/it]
 15%|█▍        | 33160/225000 [73:58:47<377:37:33,  7.09s/it]
 15%|█▍        | 33161/225000 [73:58:51<341:48:01,  6.41s/it]
 15%|█▍        | 33162/225000 [73:58:56<318:26:30,  5.98s/it]
 15%|█▍        | 33163/225000 [73:59:01<297:17:54,  5.58s/it]
 15%|█▍        | 33164/225000 [73:59:08<317:49:16,  5.96s/it]
 15%|█▍        | 33165/225000 [73:59:20<409:35:16,  7.69s/it]
 15%|█▍        | 33166/225000 [73:59:25<373:56:12,  7.02s/it]
 15%|█▍        | 33167/225000 [73:59:29<324:27:03,  6.09s/it]
 15%|█▍        | 33168/225000 [73:59:34<302:10:03,  5.67s/it]
 15%|█▍        | 33169/225000 [73:59:40<305:36:03,  5.74s/it]
 15%|█▍        | 33170/225000 [73:59:46<308:30:04,  5.79s/it]
 15%|█▍        | 33171/225000 [73:59:53<329:00:29,  6.17s/it]
 15%|█▍        | 33172/225000 [73:59:58<323:07:47,  6.06s/it]
 15%|█▍        | 33173/225000 [74:00:11<425:19:18,  7.98s/it]
 15%|█▍        | 33174/225000 [74:00:14<355:31:06,  6.67s/it]
 15%|█▍        | 33175/225000 [74:00:22<370:11:32,  6.95s/it]
 15%|█▍        | 33176/225000 [74:00:27<341:52:18,  6.42s/it]
 15%|█▍        | 33177/225000 [74:00:32<314:19:34,  5.90s/it]
 15%|█▍        | 33178/225000 [74:00:37<298:32:49,  5.60s/it]
 15%|█▍        | 33179/225000 [74:00:44<330:44:07,  6.21s/it]
 15%|█▍        | 33180/225000 [74:00:55<396:53:32,  7.45s/it]
 15%|█▍        | 33181/225000 [74:01:00<353:19:27,  6.63s/it]
 15%|█▍        | 33182/225000 [74:01:07<367:55:11,  6.91s/it]
 15%|█▍        | 33183/225000 [74:01:12<341:05:08,  6.40s/it]
 15%|█▍        | 33184/225000 [74:01:17<313:05:27,  5.88s/it]
 15%|█▍        | 33185/225000 [74:01:22<296:48:43,  5.57s/it]
 15%|█▍        | 33186/225000 [74:01:32<363:59:53,  6.83s/it]
 15%|█▍        | 33187/225000 [74:01:40<391:43:40,  7.35s/it]
 15%|█▍        | 33188/225000 [74:01:44<334:16:13,  6.27s/it]
 15%|█▍        | 33189/225000 [74:01:52<355:25:53,  6.67s/it]
 15%|█▍        | 33190/225000 [74:01:58<346:06:21,  6.50s/it]
 15%|█▍        | 33191/225000 [74:02:05<363:59:21,  6.83s/it]
 15%|█▍        | 33192/225000 [74:02:10<326:34:28,  6.13s/it]
 15%|█▍        | 33193/225000 [74:02:14<302:08:17,  5.67s/it]
 15%|█▍        | 33194/225000 [74:02:20<296:20:13,  5.56s/it]
 15%|█▍        | 33195/225000 [74:02:24<285:03:31,  5.35s/it]
 15%|█▍        | 33196/225000 [74:02:30<283:21:07,  5.32s/it]
 15%|█▍        | 33197/225000 [74:02:40<354:50:45,  6.66s/it]
 15%|█▍        | 33198/225000 [74:02:48<388:17:00,  7.29s/it]
 15%|█▍        | 33199/225000 [74:02:55<378:43:10,  7.11s/it]
 15%|█▍        | 33200/225000 [74:02:59<331:58:11,  6.23s/it]
                                                             
{'loss': '0.7348', 'grad_norm': '0.7492', 'learning_rate': '4.858e-05', 'epoch': '0.1476', 'num_input_tokens_seen': 126216056, 'train_runtime': '2.666e+05', 'train_tokens_per_second': '473.5'}

 15%|█▍        | 33200/225000 [74:02:59<331:58:11,  6.23s/it]
 15%|█▍        | 33201/225000 [74:03:03<291:55:29,  5.48s/it]
 15%|█▍        | 33202/225000 [74:03:13<372:37:05,  6.99s/it]
 15%|█▍        | 33203/225000 [74:03:18<339:01:04,  6.36s/it]
 15%|█▍        | 33204/225000 [74:03:27<373:35:07,  7.01s/it]
 15%|█▍        | 33205/225000 [74:03:31<336:01:36,  6.31s/it]
 15%|█▍        | 33206/225000 [74:03:36<308:39:40,  5.79s/it]
 15%|█▍        | 33207/225000 [74:03:41<298:15:49,  5.60s/it]
 15%|█▍        | 33208/225000 [74:03:46<286:01:03,  5.37s/it]
 15%|█▍        | 33209/225000 [74:03:51<272:26:53,  5.11s/it]
 15%|█▍        | 33210/225000 [74:04:03<391:55:28,  7.36s/it]
 15%|█▍        | 33211/225000 [74:04:10<386:53:28,  7.26s/it]
 15%|█▍        | 33212/225000 [74:04:18<389:34:53,  7.31s/it]
 15%|█▍        | 33213/225000 [74:04:23<350:51:02,  6.59s/it]
 15%|█▍        | 33214/225000 [74:04:31<384:31:04,  7.22s/it]
 15%|█▍        | 33215/225000 [74:04:37<355:30:51,  6.67s/it]
 15%|█▍        | 33216/225000 [74:04:42<329:59:41,  6.19s/it]
 15%|█▍        | 33217/225000 [74:04:47<311:51:27,  5.85s/it]
 15%|█▍        | 33218/225000 [74:04:51<293:14:59,  5.50s/it]
 15%|█▍        | 33219/225000 [74:05:03<383:40:01,  7.20s/it]
 15%|█▍        | 33220/225000 [74:05:08<358:29:46,  6.73s/it]
 15%|█▍        | 33221/225000 [74:05:16<382:17:22,  7.18s/it]
 15%|█▍        | 33222/225000 [74:05:21<339:41:54,  6.38s/it]
 15%|█▍        | 33223/225000 [74:05:28<350:30:36,  6.58s/it]
 15%|█▍        | 33224/225000 [74:05:36<379:31:37,  7.12s/it]
 15%|█▍        | 33225/225000 [74:05:40<327:54:45,  6.16s/it]
 15%|█▍        | 33226/225000 [74:05:50<378:18:36,  7.10s/it]
 15%|█▍        | 33227/225000 [74:05:55<345:10:26,  6.48s/it]
 15%|█▍        | 33228/225000 [74:06:00<327:03:31,  6.14s/it]
 15%|█▍        | 33229/225000 [74:06:05<309:41:07,  5.81s/it]
 15%|█▍        | 33230/225000 [74:06:10<301:55:42,  5.67s/it]
 15%|█▍        | 33231/225000 [74:06:16<298:22:17,  5.60s/it]
 15%|█▍        | 33232/225000 [74:06:22<304:17:40,  5.71s/it]
 15%|█▍        | 33233/225000 [74:06:27<294:27:07,  5.53s/it]
 15%|█▍        | 33234/225000 [74:06:32<290:07:12,  5.45s/it]
 15%|█▍        | 33235/225000 [74:06:43<373:34:25,  7.01s/it]
 15%|█▍        | 33236/225000 [74:06:53<429:23:04,  8.06s/it]
 15%|█▍        | 33237/225000 [74:06:57<356:03:38,  6.68s/it]
 15%|█▍        | 33238/225000 [74:07:05<380:40:09,  7.15s/it]
 15%|█▍        | 33239/225000 [74:07:10<339:41:09,  6.38s/it]
 15%|█▍        | 33240/225000 [74:07:14<316:10:15,  5.94s/it]
 15%|█▍        | 33241/225000 [74:07:19<294:47:47,  5.53s/it]
 15%|█▍        | 33242/225000 [74:07:26<323:11:15,  6.07s/it]
 15%|█▍        | 33243/225000 [74:07:38<417:25:39,  7.84s/it]
 15%|█▍        | 33244/225000 [74:07:45<403:37:34,  7.58s/it]
 15%|█▍        | 33245/225000 [74:07:49<344:39:53,  6.47s/it]
 15%|█▍        | 33246/225000 [74:07:54<319:23:24,  6.00s/it]
 15%|█▍        | 33247/225000 [74:08:04<378:43:36,  7.11s/it]
 15%|█▍        | 33248/225000 [74:08:09<354:04:52,  6.65s/it]
 15%|█▍        | 33249/225000 [74:08:17<367:01:24,  6.89s/it]
 15%|█▍        | 33250/225000 [74:08:21<322:24:45,  6.05s/it]
                                                             
{'loss': '0.7026', 'grad_norm': '0.6452', 'learning_rate': '4.857e-05', 'epoch': '0.1478', 'num_input_tokens_seen': 126360408, 'train_runtime': '2.669e+05', 'train_tokens_per_second': '473.4'}

 15%|█▍        | 33250/225000 [74:08:21<322:24:45,  6.05s/it]
 15%|█▍        | 33251/225000 [74:08:31<381:09:08,  7.16s/it]
 15%|█▍        | 33252/225000 [74:08:35<335:15:03,  6.29s/it]
 15%|█▍        | 33253/225000 [74:08:45<389:08:27,  7.31s/it]
 15%|█▍        | 33254/225000 [74:08:51<379:02:38,  7.12s/it]
 15%|█▍        | 33255/225000 [74:08:56<334:57:32,  6.29s/it]
 15%|█▍        | 33256/225000 [74:09:00<303:47:42,  5.70s/it]
 15%|█▍        | 33257/225000 [74:09:04<281:29:35,  5.29s/it]
 15%|█▍        | 33258/225000 [74:09:14<354:13:48,  6.65s/it]
 15%|█▍        | 33259/225000 [74:09:22<373:52:05,  7.02s/it]
 15%|█▍        | 33260/225000 [74:09:27<333:51:07,  6.27s/it]
 15%|█▍        | 33261/225000 [74:09:31<305:42:49,  5.74s/it]
 15%|█▍        | 33262/225000 [74:09:36<292:50:37,  5.50s/it]
 15%|█▍        | 33263/225000 [74:09:41<282:55:37,  5.31s/it]
 15%|█▍        | 33264/225000 [74:09:53<394:34:57,  7.41s/it]
 15%|█▍        | 33265/225000 [74:09:59<370:01:07,  6.95s/it]
 15%|█▍        | 33266/225000 [74:10:08<405:21:57,  7.61s/it]
 15%|█▍        | 33267/225000 [74:10:12<343:30:55,  6.45s/it]
 15%|█▍        | 33268/225000 [74:10:20<366:24:54,  6.88s/it]
 15%|█▍        | 33269/225000 [74:10:24<324:26:29,  6.09s/it]
 15%|█▍        | 33270/225000 [74:10:36<423:01:55,  7.94s/it]
 15%|█▍        | 33271/225000 [74:10:40<350:33:37,  6.58s/it]
 15%|█▍        | 33272/225000 [74:10:47<353:55:08,  6.65s/it]
 15%|█▍        | 33273/225000 [74:10:51<316:51:41,  5.95s/it]
 15%|█▍        | 33274/225000 [74:10:55<289:16:25,  5.43s/it]
 15%|█▍        | 33275/225000 [74:11:00<285:27:49,  5.36s/it]
 15%|█▍        | 33276/225000 [74:11:05<283:13:37,  5.32s/it]
 15%|█▍        | 33277/225000 [74:11:11<293:31:14,  5.51s/it]
 15%|█▍        | 33278/225000 [74:11:17<301:37:07,  5.66s/it]
 15%|█▍        | 33279/225000 [74:11:32<439:47:49,  8.26s/it]
 15%|█▍        | 33280/225000 [74:11:37<388:57:09,  7.30s/it]
 15%|█▍        | 33281/225000 [74:11:46<423:28:54,  7.95s/it]
 15%|█▍        | 33282/225000 [74:11:51<367:36:59,  6.90s/it]
 15%|█▍        | 33283/225000 [74:12:00<404:54:12,  7.60s/it]
 15%|█▍        | 33284/225000 [74:12:06<383:22:19,  7.20s/it]
 15%|█▍        | 33285/225000 [74:12:10<327:09:03,  6.14s/it]
 15%|█▍        | 33286/225000 [74:12:14<293:33:57,  5.51s/it]
 15%|█▍        | 33287/225000 [74:12:23<341:28:42,  6.41s/it]
 15%|█▍        | 33288/225000 [74:12:32<396:32:24,  7.45s/it]
 15%|█▍        | 33289/225000 [74:12:36<335:39:47,  6.30s/it]
 15%|█▍        | 33290/225000 [74:12:40<298:46:02,  5.61s/it]
 15%|█▍        | 33291/225000 [74:12:50<370:45:06,  6.96s/it]
 15%|█▍        | 33292/225000 [74:12:54<322:05:54,  6.05s/it]
 15%|█▍        | 33293/225000 [74:13:01<332:05:15,  6.24s/it]
 15%|█▍        | 33294/225000 [74:13:10<374:47:35,  7.04s/it]
 15%|█▍        | 33295/225000 [74:13:15<346:06:10,  6.50s/it]
 15%|█▍        | 33296/225000 [74:13:21<334:05:29,  6.27s/it]
 15%|█▍        | 33297/225000 [74:13:26<324:56:47,  6.10s/it]
 15%|█▍        | 33298/225000 [74:13:32<321:34:49,  6.04s/it]
 15%|█▍        | 33299/225000 [74:13:38<315:47:07,  5.93s/it]
 15%|█▍        | 33300/225000 [74:13:43<302:07:57,  5.67s/it]
                                                             
{'loss': '0.725', 'grad_norm': '0.6587', 'learning_rate': '4.857e-05', 'epoch': '0.148', 'num_input_tokens_seen': 126507144, 'train_runtime': '2.672e+05', 'train_tokens_per_second': '473.4'}

 15%|█▍        | 33300/225000 [74:13:43<302:07:57,  5.67s/it]
 15%|█▍        | 33301/225000 [74:13:48<298:48:42,  5.61s/it]
 15%|█▍        | 33302/225000 [74:13:54<294:26:04,  5.53s/it]
 15%|█▍        | 33303/225000 [74:14:00<312:46:10,  5.87s/it]
 15%|█▍        | 33304/225000 [74:14:13<427:40:32,  8.03s/it]
 15%|█▍        | 33305/225000 [74:14:19<392:57:35,  7.38s/it]
 15%|█▍        | 33306/225000 [74:14:26<374:57:17,  7.04s/it]
 15%|█▍        | 33307/225000 [74:14:33<374:01:07,  7.02s/it]
 15%|█▍        | 33308/225000 [74:14:40<383:22:11,  7.20s/it]
 15%|█▍        | 33309/225000 [74:14:44<334:52:26,  6.29s/it]
 15%|█▍        | 33310/225000 [74:14:52<362:33:07,  6.81s/it]
 15%|█▍        | 33311/225000 [74:14:58<342:44:00,  6.44s/it]
 15%|█▍        | 33312/225000 [74:15:02<311:42:35,  5.85s/it]
 15%|█▍        | 33313/225000 [74:15:08<308:07:14,  5.79s/it]
 15%|█▍        | 33314/225000 [74:15:18<374:37:50,  7.04s/it]
 15%|█▍        | 33315/225000 [74:15:23<349:32:14,  6.56s/it]
 15%|█▍        | 33316/225000 [74:15:29<325:41:35,  6.12s/it]
 15%|█▍        | 33317/225000 [74:15:34<313:55:53,  5.90s/it]
 15%|█▍        | 33318/225000 [74:15:40<312:59:26,  5.88s/it]
 15%|█▍        | 33319/225000 [74:15:53<424:49:31,  7.98s/it]
 15%|█▍        | 33320/225000 [74:15:57<366:10:24,  6.88s/it]
 15%|█▍        | 33321/225000 [74:16:01<327:53:07,  6.16s/it]
 15%|█▍        | 33322/225000 [74:16:06<300:01:22,  5.63s/it]
 15%|█▍        | 33323/225000 [74:16:17<394:37:40,  7.41s/it]
 15%|█▍        | 33324/225000 [74:16:23<371:18:28,  6.97s/it]
 15%|█▍        | 33325/225000 [74:16:29<347:39:50,  6.53s/it]
 15%|█▍        | 33326/225000 [74:16:37<373:06:23,  7.01s/it]
 15%|█▍        | 33327/225000 [74:16:42<337:59:29,  6.35s/it]
 15%|█▍        | 33328/225000 [74:16:47<314:25:17,  5.91s/it]
 15%|█▍        | 33329/225000 [74:16:51<288:31:22,  5.42s/it]
 15%|█▍        | 33330/225000 [74:17:03<398:36:02,  7.49s/it]
 15%|█▍        | 33331/225000 [74:17:07<335:30:32,  6.30s/it]
 15%|█▍        | 33332/225000 [74:17:11<304:51:40,  5.73s/it]
 15%|█▍        | 33333/225000 [74:17:23<398:21:31,  7.48s/it]
 15%|█▍        | 33334/225000 [74:17:27<341:20:14,  6.41s/it]
 15%|█▍        | 33335/225000 [74:17:31<310:31:28,  5.83s/it]
 15%|█▍        | 33336/225000 [74:17:41<381:51:03,  7.17s/it]
 15%|█▍        | 33337/225000 [74:17:46<342:15:39,  6.43s/it]
 15%|█▍        | 33338/225000 [74:17:51<318:34:52,  5.98s/it]
 15%|█▍        | 33339/225000 [74:17:56<297:15:18,  5.58s/it]
 15%|█▍        | 33340/225000 [74:18:01<289:01:20,  5.43s/it]
 15%|█▍        | 33341/225000 [74:18:14<412:16:25,  7.74s/it]
 15%|█▍        | 33342/225000 [74:18:19<361:12:23,  6.78s/it]
 15%|█▍        | 33343/225000 [74:18:27<395:01:32,  7.42s/it]
 15%|█▍        | 33344/225000 [74:18:33<366:35:16,  6.89s/it]
 15%|█▍        | 33345/225000 [74:18:39<345:10:54,  6.48s/it]
 15%|█▍        | 33346/225000 [74:18:46<365:11:44,  6.86s/it]
 15%|█▍        | 33347/225000 [74:18:50<315:11:41,  5.92s/it]
 15%|█▍        | 33348/225000 [74:18:59<366:25:24,  6.88s/it]
 15%|█▍        | 33349/225000 [74:19:07<375:26:52,  7.05s/it]
 15%|█▍        | 33350/225000 [74:19:11<330:19:12,  6.20s/it]
                                                             
{'loss': '0.7263', 'grad_norm': '0.7069', 'learning_rate': '4.856e-05', 'epoch': '0.1482', 'num_input_tokens_seen': 126652832, 'train_runtime': '2.676e+05', 'train_tokens_per_second': '473.4'}

 15%|█▍        | 33350/225000 [74:19:11<330:19:12,  6.20s/it]
 15%|█▍        | 33351/225000 [74:19:19<365:35:48,  6.87s/it]
 15%|█▍        | 33352/225000 [74:19:23<319:17:01,  6.00s/it]
 15%|█▍        | 33353/225000 [74:19:28<291:37:33,  5.48s/it]
 15%|█▍        | 33354/225000 [74:19:37<353:23:17,  6.64s/it]
 15%|█▍        | 33355/225000 [74:19:42<331:03:34,  6.22s/it]
 15%|█▍        | 33356/225000 [74:19:49<334:38:42,  6.29s/it]
 15%|█▍        | 33357/225000 [74:19:56<349:42:18,  6.57s/it]
 15%|█▍        | 33358/225000 [74:20:01<332:06:50,  6.24s/it]
 15%|█▍        | 33359/225000 [74:20:08<339:04:21,  6.37s/it]
 15%|█▍        | 33360/225000 [74:20:15<349:04:34,  6.56s/it]
 15%|█▍        | 33361/225000 [74:20:22<360:14:11,  6.77s/it]
 15%|█▍        | 33362/225000 [74:20:27<324:25:32,  6.09s/it]
 15%|█▍        | 33363/225000 [74:20:31<303:28:27,  5.70s/it]
 15%|█▍        | 33364/225000 [74:20:36<286:05:01,  5.37s/it]
 15%|█▍        | 33365/225000 [74:20:47<381:55:43,  7.17s/it]
 15%|█▍        | 33366/225000 [74:20:53<356:05:32,  6.69s/it]
 15%|█▍        | 33367/225000 [74:20:59<346:30:10,  6.51s/it]
 15%|█▍        | 33368/225000 [74:21:07<369:43:13,  6.95s/it]
 15%|█▍        | 33369/225000 [74:21:11<327:28:22,  6.15s/it]
 15%|█▍        | 33370/225000 [74:21:19<347:11:33,  6.52s/it]
 15%|█▍        | 33371/225000 [74:21:26<351:22:26,  6.60s/it]
 15%|█▍        | 33372/225000 [74:21:33<366:27:25,  6.88s/it]
 15%|█▍        | 33373/225000 [74:21:38<337:47:41,  6.35s/it]
 15%|█▍        | 33374/225000 [74:21:43<312:38:00,  5.87s/it]
 15%|█▍        | 33375/225000 [74:21:54<391:51:35,  7.36s/it]
 15%|█▍        | 33376/225000 [74:21:57<332:25:52,  6.25s/it]
 15%|█▍        | 33377/225000 [74:22:03<325:36:15,  6.12s/it]
 15%|█▍        | 33378/225000 [74:22:11<357:22:15,  6.71s/it]
 15%|█▍        | 33379/225000 [74:22:16<331:12:23,  6.22s/it]
 15%|█▍        | 33380/225000 [74:22:22<321:46:52,  6.05s/it]
 15%|█▍        | 33381/225000 [74:22:27<307:34:40,  5.78s/it]
 15%|█▍        | 33382/225000 [74:22:38<384:10:38,  7.22s/it]
 15%|█▍        | 33383/225000 [74:22:47<414:46:56,  7.79s/it]
 15%|█▍        | 33384/225000 [74:22:51<351:05:34,  6.60s/it]
 15%|█▍        | 33385/225000 [74:23:00<386:41:59,  7.27s/it]
 15%|█▍        | 33386/225000 [74:23:03<329:08:07,  6.18s/it]
 15%|█▍        | 33387/225000 [74:23:12<373:10:24,  7.01s/it]
 15%|█▍        | 33388/225000 [74:23:16<327:25:19,  6.15s/it]
 15%|█▍        | 33389/225000 [74:23:25<369:10:55,  6.94s/it]
 15%|█▍        | 33390/225000 [74:23:30<337:19:21,  6.34s/it]
 15%|█▍        | 33391/225000 [74:23:38<367:30:53,  6.90s/it]
 15%|█▍        | 33392/225000 [74:23:43<332:35:21,  6.25s/it]
 15%|█▍        | 33393/225000 [74:23:51<366:23:25,  6.88s/it]
 15%|█▍        | 33394/225000 [74:23:58<364:04:31,  6.84s/it]
 15%|█▍        | 33395/225000 [74:24:05<365:42:32,  6.87s/it]
 15%|█▍        | 33396/225000 [74:24:10<328:34:03,  6.17s/it]
 15%|█▍        | 33397/225000 [74:24:17<352:17:11,  6.62s/it]
 15%|█▍        | 33398/225000 [74:24:22<328:45:38,  6.18s/it]
 15%|█▍        | 33399/225000 [74:24:28<316:11:34,  5.94s/it]
 15%|█▍        | 33400/225000 [74:24:34<320:54:35,  6.03s/it]
                                                             
{'loss': '0.7369', 'grad_norm': '0.603', 'learning_rate': '4.855e-05', 'epoch': '0.1484', 'num_input_tokens_seen': 126799328, 'train_runtime': '2.679e+05', 'train_tokens_per_second': '473.4'}

 15%|█▍        | 33400/225000 [74:24:34<320:54:35,  6.03s/it]
 15%|█▍        | 33401/225000 [74:24:43<371:07:53,  6.97s/it]
 15%|█▍        | 33402/225000 [74:24:49<357:55:20,  6.73s/it]
 15%|█▍        | 33403/225000 [74:24:55<348:28:26,  6.55s/it]
 15%|█▍        | 33404/225000 [74:24:59<307:38:26,  5.78s/it]
 15%|█▍        | 33405/225000 [74:25:09<366:38:24,  6.89s/it]
 15%|█▍        | 33406/225000 [74:25:13<320:57:58,  6.03s/it]
 15%|█▍        | 33407/225000 [74:25:22<375:55:45,  7.06s/it]
 15%|█▍        | 33408/225000 [74:25:29<372:32:04,  7.00s/it]
 15%|█▍        | 33409/225000 [74:25:37<388:18:06,  7.30s/it]
 15%|█▍        | 33410/225000 [74:25:45<394:29:39,  7.41s/it]
 15%|█▍        | 33411/225000 [74:25:49<340:09:54,  6.39s/it]
 15%|█▍        | 33412/225000 [74:25:56<347:16:52,  6.53s/it]
 15%|█▍        | 33413/225000 [74:26:03<360:27:32,  6.77s/it]
 15%|█▍        | 33414/225000 [74:26:07<314:35:53,  5.91s/it]
 15%|█▍        | 33415/225000 [74:26:16<365:30:14,  6.87s/it]
 15%|█▍        | 33416/225000 [74:26:21<333:09:01,  6.26s/it]
 15%|█▍        | 33417/225000 [74:26:26<310:28:11,  5.83s/it]
 15%|█▍        | 33418/225000 [74:26:33<335:22:26,  6.30s/it]
 15%|█▍        | 33419/225000 [74:26:44<401:05:01,  7.54s/it]
 15%|█▍        | 33420/225000 [74:26:48<344:27:02,  6.47s/it]
 15%|█▍        | 33421/225000 [74:26:56<371:16:27,  6.98s/it]
 15%|█▍        | 33422/225000 [74:27:01<336:33:09,  6.32s/it]
 15%|█▍        | 33423/225000 [74:27:05<307:52:01,  5.79s/it]
 15%|█▍        | 33424/225000 [74:27:10<298:33:30,  5.61s/it]
 15%|█▍        | 33425/225000 [74:27:15<281:53:46,  5.30s/it]
 15%|█▍        | 33426/225000 [74:27:20<283:07:08,  5.32s/it]
 15%|█▍        | 33427/225000 [74:27:33<401:00:03,  7.54s/it]
 15%|█▍        | 33428/225000 [74:27:40<391:42:51,  7.36s/it]
 15%|█▍        | 33429/225000 [74:27:47<393:10:40,  7.39s/it]
 15%|█▍        | 33430/225000 [74:27:53<362:11:39,  6.81s/it]
 15%|█▍        | 33431/225000 [74:27:57<327:15:20,  6.15s/it]
 15%|█▍        | 33432/225000 [74:28:02<300:51:13,  5.65s/it]
 15%|█▍        | 33433/225000 [74:28:07<298:19:13,  5.61s/it]
 15%|█▍        | 33434/225000 [74:28:19<390:56:57,  7.35s/it]
 15%|█▍        | 33435/225000 [74:28:23<341:09:15,  6.41s/it]
 15%|█▍        | 33436/225000 [74:28:33<396:41:17,  7.45s/it]
 15%|█▍        | 33437/225000 [74:28:38<359:28:43,  6.76s/it]
 15%|█▍        | 33438/225000 [74:28:43<331:18:32,  6.23s/it]
 15%|█▍        | 33439/225000 [74:28:50<343:36:34,  6.46s/it]
 15%|█▍        | 33440/225000 [74:28:59<387:04:39,  7.27s/it]
 15%|█▍        | 33441/225000 [74:29:06<376:18:46,  7.07s/it]
 15%|█▍        | 33442/225000 [74:29:10<335:42:02,  6.31s/it]
 15%|█▍        | 33443/225000 [74:29:15<309:42:22,  5.82s/it]
 15%|█▍        | 33444/225000 [74:29:19<285:03:16,  5.36s/it]
 15%|█▍        | 33445/225000 [74:29:32<404:39:39,  7.61s/it]
 15%|█▍        | 33446/225000 [74:29:36<338:01:20,  6.35s/it]
 15%|█▍        | 33447/225000 [74:29:44<366:29:05,  6.89s/it]
 15%|█▍        | 33448/225000 [74:29:51<374:27:40,  7.04s/it]
 15%|█▍        | 33449/225000 [74:29:56<334:19:03,  6.28s/it]
 15%|█▍        | 33450/225000 [74:30:00<300:35:53,  5.65s/it]
                                                             
{'loss': '0.7581', 'grad_norm': '0.7296', 'learning_rate': '4.855e-05', 'epoch': '0.1487', 'num_input_tokens_seen': 126946384, 'train_runtime': '2.682e+05', 'train_tokens_per_second': '473.3'}

 15%|█▍        | 33450/225000 [74:30:00<300:35:53,  5.65s/it]
 15%|█▍        | 33451/225000 [74:30:10<376:25:16,  7.07s/it]
 15%|█▍        | 33452/225000 [74:30:14<328:03:30,  6.17s/it]
 15%|█▍        | 33453/225000 [74:30:23<363:33:55,  6.83s/it]
 15%|█▍        | 33454/225000 [74:30:30<373:32:07,  7.02s/it]
 15%|█▍        | 33455/225000 [74:30:38<384:33:44,  7.23s/it]
 15%|█▍        | 33456/225000 [74:30:43<354:23:26,  6.66s/it]
 15%|█▍        | 33457/225000 [74:30:52<390:13:50,  7.33s/it]
 15%|█▍        | 33458/225000 [74:30:57<344:18:01,  6.47s/it]
 15%|█▍        | 33459/225000 [74:31:01<306:48:48,  5.77s/it]
 15%|█▍        | 33460/225000 [74:31:05<289:31:25,  5.44s/it]
 15%|█▍        | 33461/225000 [74:31:18<404:50:15,  7.61s/it]
 15%|█▍        | 33462/225000 [74:31:22<344:07:48,  6.47s/it]
 15%|█▍        | 33463/225000 [74:31:27<328:28:45,  6.17s/it]
 15%|█▍        | 33464/225000 [74:31:33<318:05:54,  5.98s/it]
 15%|█▍        | 33465/225000 [74:31:40<341:20:25,  6.42s/it]
 15%|█▍        | 33466/225000 [74:31:48<360:06:10,  6.77s/it]
 15%|█▍        | 33467/225000 [74:31:57<393:34:58,  7.40s/it]
 15%|█▍        | 33468/225000 [74:32:02<362:13:51,  6.81s/it]
 15%|█▍        | 33469/225000 [74:32:06<314:35:03,  5.91s/it]
 15%|█▍        | 33470/225000 [74:32:11<304:24:17,  5.72s/it]
 15%|█▍        | 33471/225000 [74:32:20<359:44:15,  6.76s/it]
 15%|█▍        | 33472/225000 [74:32:24<316:44:50,  5.95s/it]
 15%|█▍        | 33473/225000 [74:32:30<312:25:05,  5.87s/it]
 15%|█▍        | 33474/225000 [74:32:35<299:54:44,  5.64s/it]
 15%|█▍        | 33475/225000 [74:32:41<304:50:09,  5.73s/it]
 15%|█▍        | 33476/225000 [74:32:50<354:16:42,  6.66s/it]
 15%|█▍        | 33477/225000 [74:33:00<411:10:44,  7.73s/it]
 15%|█▍        | 33478/225000 [74:33:03<338:32:20,  6.36s/it]
 15%|█▍        | 33479/225000 [74:33:13<394:31:33,  7.42s/it]
 15%|█▍        | 33480/225000 [74:33:18<355:47:37,  6.69s/it]
 15%|█▍        | 33481/225000 [74:33:25<353:55:52,  6.65s/it]
 15%|█▍        | 33482/225000 [74:33:32<366:20:50,  6.89s/it]
 15%|█▍        | 33483/225000 [74:33:37<336:17:50,  6.32s/it]
 15%|█▍        | 33484/225000 [74:33:42<310:58:47,  5.85s/it]
 15%|█▍        | 33485/225000 [74:33:48<307:04:58,  5.77s/it]
 15%|█▍        | 33486/225000 [74:33:53<305:43:17,  5.75s/it]
 15%|█▍        | 33487/225000 [74:33:59<302:25:57,  5.69s/it]
 15%|█▍        | 33488/225000 [74:34:05<311:08:24,  5.85s/it]
 15%|█▍        | 33489/225000 [74:34:17<406:13:35,  7.64s/it]
 15%|█▍        | 33490/225000 [74:34:26<422:33:28,  7.94s/it]
 15%|█▍        | 33491/225000 [74:34:29<358:21:05,  6.74s/it]
 15%|█▍        | 33492/225000 [74:34:35<337:25:24,  6.34s/it]
 15%|█▍        | 33493/225000 [74:34:40<317:47:44,  5.97s/it]
 15%|█▍        | 33494/225000 [74:34:47<340:29:18,  6.40s/it]
 15%|█▍        | 33495/225000 [74:35:00<442:07:41,  8.31s/it]
 15%|█▍        | 33496/225000 [74:35:08<429:57:22,  8.08s/it]
 15%|█▍        | 33497/225000 [74:35:12<371:09:51,  6.98s/it]
 15%|█▍        | 33498/225000 [74:35:17<335:30:41,  6.31s/it]
 15%|█▍        | 33499/225000 [74:35:28<404:56:48,  7.61s/it]
 15%|█▍        | 33500/225000 [74:35:34<382:10:07,  7.18s/it]
                                                             
{'loss': '0.7246', 'grad_norm': '0.6789', 'learning_rate': '4.854e-05', 'epoch': '0.1489', 'num_input_tokens_seen': 127095080, 'train_runtime': '2.685e+05', 'train_tokens_per_second': '473.3'}

 15%|█▍        | 33500/225000 [74:35:34<382:10:07,  7.18s/it]
 15%|█▍        | 33501/225000 [74:35:38<332:30:02,  6.25s/it]
 15%|█▍        | 33502/225000 [74:35:44<327:06:37,  6.15s/it]
 15%|█▍        | 33503/225000 [74:35:52<363:41:06,  6.84s/it]
 15%|█▍        | 33504/225000 [74:35:57<331:21:20,  6.23s/it]
 15%|█▍        | 33505/225000 [74:36:02<310:51:50,  5.84s/it]
 15%|█▍        | 33506/225000 [74:36:07<299:31:29,  5.63s/it]
 15%|█▍        | 33507/225000 [74:36:12<285:03:13,  5.36s/it]
 15%|█▍        | 33508/225000 [74:36:24<401:46:21,  7.55s/it]
 15%|█▍        | 33509/225000 [74:36:32<395:51:00,  7.44s/it]
 15%|█▍        | 33510/225000 [74:36:35<337:55:38,  6.35s/it]
 15%|█▍        | 33511/225000 [74:36:45<393:13:22,  7.39s/it]
 15%|█▍        | 33512/225000 [74:36:50<355:31:51,  6.68s/it]
 15%|█▍        | 33513/225000 [74:36:59<387:50:04,  7.29s/it]
 15%|█▍        | 33514/225000 [74:37:03<342:31:43,  6.44s/it]
 15%|█▍        | 33515/225000 [74:37:08<315:28:51,  5.93s/it]
 15%|█▍        | 33516/225000 [74:37:16<349:14:19,  6.57s/it]
 15%|█▍        | 33517/225000 [74:37:23<349:52:25,  6.58s/it]
 15%|█▍        | 33518/225000 [74:37:29<347:06:28,  6.53s/it]
 15%|█▍        | 33519/225000 [74:37:36<347:17:38,  6.53s/it]
 15%|█▍        | 33520/225000 [74:37:41<319:29:31,  6.01s/it]
 15%|█▍        | 33521/225000 [74:37:45<301:13:46,  5.66s/it]
 15%|█▍        | 33522/225000 [74:37:50<284:55:58,  5.36s/it]
 15%|█▍        | 33523/225000 [74:37:55<278:51:44,  5.24s/it]
 15%|█▍        | 33524/225000 [74:38:06<363:11:15,  6.83s/it]
 15%|█▍        | 33525/225000 [74:38:14<382:24:44,  7.19s/it]
 15%|█▍        | 33526/225000 [74:38:21<380:07:34,  7.15s/it]
 15%|█▍        | 33527/225000 [74:38:25<328:18:36,  6.17s/it]
 15%|█▍        | 33528/225000 [74:38:29<303:27:41,  5.71s/it]
 15%|█▍        | 33529/225000 [74:38:36<329:02:47,  6.19s/it]
 15%|█▍        | 33530/225000 [74:38:46<379:47:21,  7.14s/it]
 15%|█▍        | 33531/225000 [74:38:54<402:09:15,  7.56s/it]
 15%|█▍        | 33532/225000 [74:38:59<356:56:30,  6.71s/it]
 15%|█▍        | 33533/225000 [74:39:03<318:33:09,  5.99s/it]
 15%|█▍        | 33534/225000 [74:39:08<299:59:57,  5.64s/it]
 15%|█▍        | 33535/225000 [74:39:13<284:37:12,  5.35s/it]
 15%|█▍        | 33536/225000 [74:39:21<334:34:13,  6.29s/it]
 15%|█▍        | 33537/225000 [74:39:32<399:19:20,  7.51s/it]
 15%|█▍        | 33538/225000 [74:39:38<375:10:04,  7.05s/it]
 15%|█▍        | 33539/225000 [74:39:42<330:56:51,  6.22s/it]
 15%|█▍        | 33540/225000 [74:39:46<300:15:49,  5.65s/it]
 15%|█▍        | 33541/225000 [74:39:55<348:00:23,  6.54s/it]
 15%|█▍        | 33542/225000 [74:40:06<417:17:04,  7.85s/it]
 15%|█▍        | 33543/225000 [74:40:10<351:59:39,  6.62s/it]
 15%|█▍        | 33544/225000 [74:40:19<388:38:09,  7.31s/it]
 15%|█▍        | 33545/225000 [74:40:24<356:19:31,  6.70s/it]
 15%|█▍        | 33546/225000 [74:40:29<326:38:52,  6.14s/it]
 15%|█▍        | 33547/225000 [74:40:34<306:33:31,  5.76s/it]
 15%|█▍        | 33548/225000 [74:40:40<311:09:13,  5.85s/it]
 15%|█▍        | 33549/225000 [74:40:53<427:28:25,  8.04s/it]
 15%|█▍        | 33550/225000 [74:40:57<369:32:18,  6.95s/it]
                                                             
{'loss': '0.7326', 'grad_norm': '0.6245', 'learning_rate': '4.854e-05', 'epoch': '0.1491', 'num_input_tokens_seen': 127241552, 'train_runtime': '2.689e+05', 'train_tokens_per_second': '473.3'}

 15%|█▍        | 33550/225000 [74:40:57<369:32:18,  6.95s/it]
 15%|█▍        | 33551/225000 [74:41:04<362:57:17,  6.82s/it]
 15%|█▍        | 33552/225000 [74:41:10<350:46:36,  6.60s/it]
 15%|█▍        | 33553/225000 [74:41:16<342:19:20,  6.44s/it]
 15%|█▍        | 33554/225000 [74:41:23<362:34:27,  6.82s/it]
 15%|█▍        | 33555/225000 [74:41:28<323:50:53,  6.09s/it]
 15%|█▍        | 33556/225000 [74:41:32<296:54:10,  5.58s/it]
 15%|█▍        | 33557/225000 [74:41:42<365:32:59,  6.87s/it]
 15%|█▍        | 33558/225000 [74:41:49<365:27:46,  6.87s/it]
 15%|█▍        | 33559/225000 [74:41:53<318:58:59,  6.00s/it]
 15%|█▍        | 33560/225000 [74:41:57<293:39:34,  5.52s/it]
 15%|█▍        | 33561/225000 [74:42:05<324:50:02,  6.11s/it]
 15%|█▍        | 33562/225000 [74:42:14<378:57:59,  7.13s/it]
 15%|█▍        | 33563/225000 [74:42:19<338:17:16,  6.36s/it]
 15%|█▍        | 33564/225000 [74:42:28<386:35:06,  7.27s/it]
 15%|█▍        | 33565/225000 [74:42:32<336:32:15,  6.33s/it]
 15%|█▍        | 33566/225000 [74:42:37<307:28:01,  5.78s/it]
 15%|█▍        | 33567/225000 [74:42:46<363:27:46,  6.84s/it]
 15%|█▍        | 33568/225000 [74:42:53<368:33:13,  6.93s/it]
 15%|█▍        | 33569/225000 [74:42:58<336:23:28,  6.33s/it]
 15%|█▍        | 33570/225000 [74:43:03<309:14:57,  5.82s/it]
 15%|█▍        | 33571/225000 [74:43:07<283:59:17,  5.34s/it]
 15%|█▍        | 33572/225000 [74:43:18<364:25:59,  6.85s/it]
 15%|█▍        | 33573/225000 [74:43:25<368:18:19,  6.93s/it]
 15%|█▍        | 33574/225000 [74:43:32<374:23:55,  7.04s/it]
 15%|█▍        | 33575/225000 [74:43:37<343:57:52,  6.47s/it]
 15%|█▍        | 33576/225000 [74:43:42<318:51:07,  6.00s/it]
 15%|█▍        | 33577/225000 [74:43:47<296:29:47,  5.58s/it]
 15%|█▍        | 33578/225000 [74:43:55<348:13:38,  6.55s/it]
 15%|█▍        | 33579/225000 [74:44:03<372:15:26,  7.00s/it]
 15%|█▍        | 33580/225000 [74:44:08<332:44:13,  6.26s/it]
 15%|█▍        | 33581/225000 [74:44:13<311:49:15,  5.86s/it]
 15%|█▍        | 33582/225000 [74:44:18<305:11:55,  5.74s/it]
 15%|█▍        | 33583/225000 [74:44:24<303:03:14,  5.70s/it]
 15%|█▍        | 33584/225000 [74:44:33<351:03:11,  6.60s/it]
 15%|█▍        | 33585/225000 [74:44:42<393:53:19,  7.41s/it]
 15%|█▍        | 33586/225000 [74:44:46<343:51:16,  6.47s/it]
 15%|█▍        | 33587/225000 [74:44:52<335:05:48,  6.30s/it]
 15%|█▍        | 33588/225000 [74:45:04<425:51:41,  8.01s/it]
 15%|█▍        | 33589/225000 [74:45:09<380:02:22,  7.15s/it]
 15%|█▍        | 33590/225000 [74:45:17<387:53:18,  7.30s/it]
 15%|█▍        | 33591/225000 [74:45:21<334:58:28,  6.30s/it]
 15%|█▍        | 33592/225000 [74:45:30<386:07:32,  7.26s/it]
 15%|█▍        | 33593/225000 [74:45:37<368:31:02,  6.93s/it]
 15%|█▍        | 33594/225000 [74:45:44<380:34:42,  7.16s/it]
 15%|█▍        | 33595/225000 [74:45:49<336:30:07,  6.33s/it]
 15%|█▍        | 33596/225000 [74:45:54<317:48:54,  5.98s/it]
 15%|█▍        | 33597/225000 [74:45:59<306:00:19,  5.76s/it]
 15%|█▍        | 33598/225000 [74:46:04<291:11:15,  5.48s/it]
 15%|█▍        | 33599/225000 [74:46:09<285:37:15,  5.37s/it]
 15%|█▍        | 33600/225000 [74:46:23<414:49:31,  7.80s/it]
                                                             
{'loss': '0.7257', 'grad_norm': '0.5824', 'learning_rate': '4.853e-05', 'epoch': '0.1493', 'num_input_tokens_seen': 127387656, 'train_runtime': '2.692e+05', 'train_tokens_per_second': '473.2'}

 15%|█▍        | 33600/225000 [74:46:23<414:49:31,  7.80s/it]
 15%|█▍        | 33601/225000 [74:46:27<356:11:14,  6.70s/it]
 15%|█▍        | 33602/225000 [74:46:31<318:08:50,  5.98s/it]
 15%|█▍        | 33603/225000 [74:46:38<327:15:22,  6.16s/it]
 15%|█▍        | 33604/225000 [74:46:44<329:18:49,  6.19s/it]
 15%|█▍        | 33605/225000 [74:46:49<321:07:00,  6.04s/it]
 15%|█▍        | 33606/225000 [74:47:00<389:18:08,  7.32s/it]
 15%|█▍        | 33607/225000 [74:47:04<331:30:06,  6.24s/it]
 15%|█▍        | 33608/225000 [74:47:08<310:24:19,  5.84s/it]
 15%|█▍        | 33609/225000 [74:47:13<297:00:45,  5.59s/it]
 15%|█▍        | 33610/225000 [74:47:27<422:49:02,  7.95s/it]
 15%|█▍        | 33611/225000 [74:47:31<358:10:00,  6.74s/it]
 15%|█▍        | 33612/225000 [74:47:39<388:03:38,  7.30s/it]
 15%|█▍        | 33613/225000 [74:47:43<331:45:04,  6.24s/it]
 15%|█▍        | 33614/225000 [74:47:48<316:46:06,  5.96s/it]
 15%|█▍        | 33615/225000 [74:47:54<304:11:57,  5.72s/it]
 15%|█▍        | 33616/225000 [74:47:59<303:16:21,  5.70s/it]
 15%|█▍        | 33617/225000 [74:48:05<300:31:45,  5.65s/it]
 15%|█▍        | 33618/225000 [74:48:13<348:11:33,  6.55s/it]
 15%|█▍        | 33619/225000 [74:48:24<418:57:36,  7.88s/it]
 15%|█▍        | 33620/225000 [74:48:29<368:18:51,  6.93s/it]
 15%|█▍        | 33621/225000 [74:48:34<337:30:46,  6.35s/it]
 15%|█▍        | 33622/225000 [74:48:39<310:25:32,  5.84s/it]
 15%|█▍        | 33623/225000 [74:48:51<410:20:09,  7.72s/it]
 15%|█▍        | 33624/225000 [74:48:56<362:03:14,  6.81s/it]
 15%|█▍        | 33625/225000 [74:49:00<324:35:23,  6.11s/it]
 15%|█▍        | 33626/225000 [74:49:08<359:35:48,  6.76s/it]
 15%|█▍        | 33627/225000 [74:49:15<351:37:33,  6.61s/it]
 15%|█▍        | 33628/225000 [74:49:25<409:59:50,  7.71s/it]
 15%|█▍        | 33629/225000 [74:49:29<350:03:37,  6.59s/it]
 15%|█▍        | 33630/225000 [74:49:33<317:20:22,  5.97s/it]
 15%|█▍        | 33631/225000 [74:49:39<303:29:00,  5.71s/it]
 15%|█▍        | 33632/225000 [74:49:44<292:56:03,  5.51s/it]
 15%|█▍        | 33633/225000 [74:49:52<346:03:29,  6.51s/it]
 15%|█▍        | 33634/225000 [74:50:03<417:06:26,  7.85s/it]
 15%|█▍        | 33635/225000 [74:50:10<394:04:03,  7.41s/it]
 15%|█▍        | 33636/225000 [74:50:15<359:03:49,  6.75s/it]
 15%|█▍        | 33637/225000 [74:50:20<338:15:07,  6.36s/it]
 15%|█▍        | 33638/225000 [74:50:25<308:48:03,  5.81s/it]
 15%|█▍        | 33639/225000 [74:50:30<304:07:14,  5.72s/it]
 15%|█▍        | 33640/225000 [74:50:36<294:42:01,  5.54s/it]
 15%|█▍        | 33641/225000 [74:50:40<282:38:13,  5.32s/it]
 15%|█▍        | 33642/225000 [74:50:53<393:18:48,  7.40s/it]
 15%|█▍        | 33643/225000 [74:51:01<409:11:46,  7.70s/it]
 15%|█▍        | 33644/225000 [74:51:05<356:25:49,  6.71s/it]
 15%|█▍        | 33645/225000 [74:51:10<329:00:13,  6.19s/it]
 15%|█▍        | 33646/225000 [74:51:15<304:51:24,  5.74s/it]
 15%|█▍        | 33647/225000 [74:51:22<327:37:44,  6.16s/it]
 15%|█▍        | 33648/225000 [74:51:33<397:35:20,  7.48s/it]
 15%|█▍        | 33649/225000 [74:51:37<340:30:28,  6.41s/it]
 15%|█▍        | 33650/225000 [74:51:47<409:16:08,  7.70s/it]
                                                             
{'loss': '0.7422', 'grad_norm': '0.7196', 'learning_rate': '4.852e-05', 'epoch': '0.1496', 'num_input_tokens_seen': 127534208, 'train_runtime': '2.695e+05', 'train_tokens_per_second': '473.2'}

 15%|█▍        | 33650/225000 [74:51:47<409:16:08,  7.70s/it]
 15%|█▍        | 33651/225000 [74:51:51<344:44:36,  6.49s/it]
 15%|█▍        | 33652/225000 [74:52:00<377:26:59,  7.10s/it]
 15%|█▍        | 33653/225000 [74:52:03<321:47:49,  6.05s/it]
 15%|█▍        | 33654/225000 [74:52:13<387:20:05,  7.29s/it]
 15%|█▍        | 33655/225000 [74:52:18<347:21:50,  6.54s/it]
 15%|█▍        | 33656/225000 [74:52:25<349:38:40,  6.58s/it]
 15%|█▍        | 33657/225000 [74:52:31<349:57:47,  6.58s/it]
 15%|█▍        | 33658/225000 [74:52:35<301:16:19,  5.67s/it]
 15%|█▍        | 33659/225000 [74:52:42<323:07:02,  6.08s/it]
 15%|█▍        | 33660/225000 [74:52:47<298:46:47,  5.62s/it]
 15%|█▍        | 33661/225000 [74:52:56<357:46:35,  6.73s/it]
 15%|█▍        | 33662/225000 [74:53:00<315:54:21,  5.94s/it]
 15%|█▍        | 33663/225000 [74:53:10<375:09:06,  7.06s/it]
 15%|█▍        | 33664/225000 [74:53:14<331:56:36,  6.25s/it]
 15%|█▍        | 33665/225000 [74:53:18<303:00:28,  5.70s/it]
 15%|█▍        | 33666/225000 [74:53:29<374:45:12,  7.05s/it]
 15%|█▍        | 33667/225000 [74:53:32<317:24:31,  5.97s/it]
 15%|█▍        | 33668/225000 [74:53:43<397:06:23,  7.47s/it]
 15%|█▍        | 33669/225000 [74:53:51<400:41:18,  7.54s/it]
 15%|█▍        | 33670/225000 [74:53:55<349:20:30,  6.57s/it]
 15%|█▍        | 33671/225000 [74:54:04<380:49:53,  7.17s/it]
 15%|█▍        | 33672/225000 [74:54:08<331:14:44,  6.23s/it]
 15%|█▍        | 33673/225000 [74:54:12<300:33:57,  5.66s/it]
 15%|█▍        | 33674/225000 [74:54:20<333:31:07,  6.28s/it]
 15%|█▍        | 33675/225000 [74:54:30<401:29:53,  7.55s/it]
 15%|█▍        | 33676/225000 [74:54:33<332:01:41,  6.25s/it]
 15%|█▍        | 33677/225000 [74:54:38<303:53:48,  5.72s/it]
 15%|█▍        | 33678/225000 [74:54:43<289:59:39,  5.46s/it]
 15%|█▍        | 33679/225000 [74:54:48<280:10:56,  5.27s/it]
 15%|█▍        | 33680/225000 [74:54:57<350:35:03,  6.60s/it]
 15%|█▍        | 33681/225000 [74:55:08<417:24:02,  7.85s/it]
 15%|█▍        | 33682/225000 [74:55:14<382:57:49,  7.21s/it]
 15%|█▍        | 33683/225000 [74:55:20<363:16:17,  6.84s/it]
 15%|█▍        | 33684/225000 [74:55:23<312:49:48,  5.89s/it]
 15%|█▍        | 33685/225000 [74:55:35<402:04:41,  7.57s/it]
 15%|█▍        | 33686/225000 [74:55:38<337:18:38,  6.35s/it]
 15%|█▍        | 33687/225000 [74:55:42<298:38:27,  5.62s/it]
 15%|█▍        | 33688/225000 [74:55:50<333:10:25,  6.27s/it]
 15%|█▍        | 33689/225000 [74:55:58<358:15:20,  6.74s/it]
 15%|█▍        | 33690/225000 [74:56:03<329:27:45,  6.20s/it]
 15%|█▍        | 33691/225000 [74:56:08<306:04:12,  5.76s/it]
 15%|█▍        | 33692/225000 [74:56:13<292:11:21,  5.50s/it]
 15%|█▍        | 33693/225000 [74:56:25<408:19:50,  7.68s/it]
 15%|█▍        | 33694/225000 [74:56:30<360:53:49,  6.79s/it]
 15%|█▍        | 33695/225000 [74:56:36<348:44:36,  6.56s/it]
 15%|█▍        | 33696/225000 [74:56:43<359:50:30,  6.77s/it]
 15%|█▍        | 33697/225000 [74:56:47<316:30:36,  5.96s/it]
 15%|█▍        | 33698/225000 [74:56:58<388:29:27,  7.31s/it]
 15%|█▍        | 33699/225000 [74:57:02<331:59:46,  6.25s/it]
 15%|█▍        | 33700/225000 [74:57:10<361:09:20,  6.80s/it]
                                                             
{'loss': '0.7327', 'grad_norm': '0.7998', 'learning_rate': '4.852e-05', 'epoch': '0.1498', 'num_input_tokens_seen': 127678656, 'train_runtime': '2.698e+05', 'train_tokens_per_second': '473.2'}

 15%|█▍        | 33700/225000 [74:57:10<361:09:20,  6.80s/it]
 15%|█▍        | 33701/225000 [74:57:13<310:55:03,  5.85s/it]
 15%|█▍        | 33702/225000 [74:57:18<295:44:07,  5.57s/it]
 15%|█▍        | 33703/225000 [74:57:23<281:10:45,  5.29s/it]
 15%|█▍        | 33704/225000 [74:57:29<287:27:30,  5.41s/it]
 15%|█▍        | 33705/225000 [74:57:35<297:32:29,  5.60s/it]
 15%|█▍        | 33706/225000 [74:57:47<407:02:32,  7.66s/it]
 15%|█▍        | 33707/225000 [74:57:54<397:02:53,  7.47s/it]
 15%|█▍        | 33708/225000 [74:57:59<354:41:43,  6.68s/it]
 15%|█▍        | 33709/225000 [74:58:04<333:21:07,  6.27s/it]
 15%|█▍        | 33710/225000 [74:58:09<310:19:50,  5.84s/it]
 15%|█▍        | 33711/225000 [74:58:14<303:31:03,  5.71s/it]
 15%|█▍        | 33712/225000 [74:58:20<303:42:21,  5.72s/it]
 15%|█▍        | 33713/225000 [74:58:26<299:17:18,  5.63s/it]
 15%|█▍        | 33714/225000 [74:58:31<294:27:16,  5.54s/it]
 15%|█▍        | 33715/225000 [74:58:37<297:50:06,  5.61s/it]
 15%|█▍        | 33716/225000 [74:58:43<301:45:13,  5.68s/it]
 15%|█▍        | 33717/225000 [74:58:55<416:20:38,  7.84s/it]
 15%|█▍        | 33718/225000 [74:59:01<381:49:46,  7.19s/it]
 15%|█▍        | 33719/225000 [74:59:10<400:49:41,  7.54s/it]
 15%|█▍        | 33720/225000 [74:59:14<355:43:22,  6.69s/it]
 15%|█▍        | 33721/225000 [74:59:20<334:52:28,  6.30s/it]
 15%|█▍        | 33722/225000 [74:59:25<325:33:42,  6.13s/it]
 15%|█▍        | 33723/225000 [74:59:31<313:48:32,  5.91s/it]
 15%|█▍        | 33724/225000 [74:59:36<301:59:21,  5.68s/it]
 15%|█▍        | 33725/225000 [74:59:41<297:50:44,  5.61s/it]
 15%|█▍        | 33726/225000 [74:59:47<295:35:24,  5.56s/it]
 15%|█▍        | 33727/225000 [74:59:58<390:04:12,  7.34s/it]
 15%|█▍        | 33728/225000 [75:00:05<377:23:47,  7.10s/it]
 15%|█▍        | 33729/225000 [75:00:10<346:16:15,  6.52s/it]
 15%|█▍        | 33730/225000 [75:00:16<332:43:17,  6.26s/it]
 15%|█▍        | 33731/225000 [75:00:22<342:19:38,  6.44s/it]
 15%|█▍        | 33732/225000 [75:00:28<324:31:17,  6.11s/it]
 15%|█▍        | 33733/225000 [75:00:33<314:32:18,  5.92s/it]
 15%|█▍        | 33734/225000 [75:00:39<308:15:23,  5.80s/it]
 15%|█▍        | 33735/225000 [75:00:44<304:58:14,  5.74s/it]
 15%|█▍        | 33736/225000 [75:00:53<352:55:41,  6.64s/it]
 15%|█▍        | 33737/225000 [75:01:05<438:20:06,  8.25s/it]
 15%|█▍        | 33738/225000 [75:01:09<367:31:22,  6.92s/it]
 15%|█▍        | 33739/225000 [75:01:16<368:42:49,  6.94s/it]
 15%|█▍        | 33740/225000 [75:01:20<325:57:06,  6.14s/it]
 15%|█▍        | 33741/225000 [75:01:28<354:53:57,  6.68s/it]
 15%|█▍        | 33742/225000 [75:01:35<362:22:36,  6.82s/it]
 15%|█▍        | 33743/225000 [75:01:41<344:37:21,  6.49s/it]
 15%|█▍        | 33744/225000 [75:01:46<315:11:57,  5.93s/it]
 15%|█▍        | 33745/225000 [75:01:51<301:09:43,  5.67s/it]
 15%|█▍        | 33746/225000 [75:01:56<290:33:38,  5.47s/it]
 15%|█▍        | 33747/225000 [75:02:01<281:58:07,  5.31s/it]
 15%|█▍        | 33748/225000 [75:02:13<400:00:10,  7.53s/it]
 15%|█▍        | 33749/225000 [75:02:20<385:24:59,  7.25s/it]
 15%|█▌        | 33750/225000 [75:02:26<359:54:11,  6.77s/it]
                                                             
{'loss': '0.7339', 'grad_norm': '0.6765', 'learning_rate': '4.851e-05', 'epoch': '0.15', 'num_input_tokens_seen': 127823432, 'train_runtime': '2.701e+05', 'train_tokens_per_second': '473.2'}

 15%|█▌        | 33750/225000 [75:02:26<359:54:11,  6.77s/it]
 15%|█▌        | 33751/225000 [75:02:31<335:18:38,  6.31s/it]
 15%|█▌        | 33752/225000 [75:02:36<317:43:34,  5.98s/it]
 15%|█▌        | 33753/225000 [75:02:42<311:13:04,  5.86s/it]
 15%|█▌        | 33754/225000 [75:02:47<296:17:10,  5.58s/it]
 15%|█▌        | 33755/225000 [75:02:52<289:25:05,  5.45s/it]
 15%|█▌        | 33756/225000 [75:02:57<287:40:53,  5.42s/it]
 15%|█▌        | 33757/225000 [75:03:09<385:52:31,  7.26s/it]
 15%|█▌        | 33758/225000 [75:03:16<384:14:55,  7.23s/it]
 15%|█▌        | 33759/225000 [75:03:23<388:06:21,  7.31s/it]
 15%|█▌        | 33760/225000 [75:03:28<353:51:18,  6.66s/it]
 15%|█▌        | 33761/225000 [75:03:38<398:10:29,  7.50s/it]
 15%|█▌        | 33762/225000 [75:03:42<346:03:01,  6.51s/it]
 15%|█▌        | 33763/225000 [75:03:47<317:46:21,  5.98s/it]
 15%|█▌        | 33764/225000 [75:03:51<288:55:14,  5.44s/it]
 15%|█▌        | 33765/225000 [75:03:55<271:30:53,  5.11s/it]
 15%|█▌        | 33766/225000 [75:04:08<387:08:16,  7.29s/it]
 15%|█▌        | 33767/225000 [75:04:13<353:23:39,  6.65s/it]
 15%|█▌        | 33768/225000 [75:04:17<318:06:48,  5.99s/it]
 15%|█▌        | 33769/225000 [75:04:22<294:19:05,  5.54s/it]
 15%|█▌        | 33770/225000 [75:04:27<293:08:03,  5.52s/it]
 15%|█▌        | 33771/225000 [75:04:33<292:51:58,  5.51s/it]
 15%|█▌        | 33772/225000 [75:04:41<334:53:45,  6.30s/it]
 15%|█▌        | 33773/225000 [75:04:53<424:10:23,  7.99s/it]
 15%|█▌        | 33774/225000 [75:04:57<365:26:08,  6.88s/it]
 15%|█▌        | 33775/225000 [75:05:02<329:06:38,  6.20s/it]
 15%|█▌        | 33776/225000 [75:05:09<343:03:21,  6.46s/it]
 15%|█▌        | 33777/225000 [75:05:19<405:53:06,  7.64s/it]
 15%|█▌        | 33778/225000 [75:05:24<357:05:40,  6.72s/it]
 15%|█▌        | 33779/225000 [75:05:29<329:17:28,  6.20s/it]
 15%|█▌        | 33780/225000 [75:05:34<321:27:42,  6.05s/it]
 15%|█▌        | 33781/225000 [75:05:40<314:32:15,  5.92s/it]
 15%|█▌        | 33782/225000 [75:05:46<309:07:46,  5.82s/it]
 15%|█▌        | 33783/225000 [75:05:52<312:10:13,  5.88s/it]
 15%|█▌        | 33784/225000 [75:06:01<368:01:26,  6.93s/it]
 15%|█▌        | 33785/225000 [75:06:11<421:49:25,  7.94s/it]
 15%|█▌        | 33786/225000 [75:06:16<375:52:11,  7.08s/it]
 15%|█▌        | 33787/225000 [75:06:22<346:06:18,  6.52s/it]
 15%|█▌        | 33788/225000 [75:06:27<323:12:46,  6.09s/it]
 15%|█▌        | 33789/225000 [75:06:32<313:02:24,  5.89s/it]
 15%|█▌        | 33790/225000 [75:06:39<321:12:13,  6.05s/it]
 15%|█▌        | 33791/225000 [75:06:50<405:33:16,  7.64s/it]
 15%|█▌        | 33792/225000 [75:06:54<345:40:26,  6.51s/it]
 15%|█▌        | 33793/225000 [75:07:00<342:27:04,  6.45s/it]
 15%|█▌        | 33794/225000 [75:07:09<375:58:16,  7.08s/it]
 15%|█▌        | 33795/225000 [75:07:12<321:28:37,  6.05s/it]
 15%|█▌        | 33796/225000 [75:07:21<363:01:59,  6.84s/it]
 15%|█▌        | 33797/225000 [75:07:27<354:51:19,  6.68s/it]
 15%|█▌        | 33798/225000 [75:07:35<366:10:25,  6.89s/it]
 15%|█▌        | 33799/225000 [75:07:41<349:15:28,  6.58s/it]
 15%|█▌        | 33800/225000 [75:07:50<388:23:10,  7.31s/it]
                                                             
{'loss': '0.7435', 'grad_norm': '0.8369', 'learning_rate': '4.851e-05', 'epoch': '0.1502', 'num_input_tokens_seen': 127969120, 'train_runtime': '2.705e+05', 'train_tokens_per_second': '473.1'}

 15%|█▌        | 33800/225000 [75:07:50<388:23:10,  7.31s/it]
 15%|█▌        | 33801/225000 [75:07:55<359:04:50,  6.76s/it]
 15%|█▌        | 33802/225000 [75:08:00<331:30:19,  6.24s/it]
 15%|█▌        | 33803/225000 [75:08:05<318:03:18,  5.99s/it]
 15%|█▌        | 33804/225000 [75:08:11<312:12:56,  5.88s/it]
 15%|█▌        | 33805/225000 [75:08:16<296:10:59,  5.58s/it]
 15%|█▌        | 33806/225000 [75:08:21<287:52:07,  5.42s/it]
 15%|█▌        | 33807/225000 [75:08:29<321:18:20,  6.05s/it]
 15%|█▌        | 33808/225000 [75:08:39<392:42:34,  7.39s/it]
 15%|█▌        | 33809/225000 [75:08:46<377:33:59,  7.11s/it]
 15%|█▌        | 33810/225000 [75:08:51<348:30:17,  6.56s/it]
 15%|█▌        | 33811/225000 [75:08:56<324:28:38,  6.11s/it]
 15%|█▌        | 33812/225000 [75:09:05<374:53:39,  7.06s/it]
 15%|█▌        | 33813/225000 [75:09:11<356:10:17,  6.71s/it]
 15%|█▌        | 33814/225000 [75:09:17<348:56:35,  6.57s/it]
 15%|█▌        | 33815/225000 [75:09:22<317:40:34,  5.98s/it]
 15%|█▌        | 33816/225000 [75:09:28<322:22:46,  6.07s/it]
 15%|█▌        | 33817/225000 [75:09:40<408:15:13,  7.69s/it]
 15%|█▌        | 33818/225000 [75:09:44<351:16:01,  6.61s/it]
 15%|█▌        | 33819/225000 [75:09:54<412:41:07,  7.77s/it]
 15%|█▌        | 33820/225000 [75:09:58<352:42:38,  6.64s/it]
 15%|█▌        | 33821/225000 [75:10:07<382:48:44,  7.21s/it]
 15%|█▌        | 33822/225000 [75:10:11<337:11:09,  6.35s/it]
 15%|█▌        | 33823/225000 [75:10:18<351:48:44,  6.62s/it]
 15%|█▌        | 33824/225000 [75:10:23<326:07:45,  6.14s/it]
 15%|█▌        | 33825/225000 [75:10:29<311:11:04,  5.86s/it]
 15%|█▌        | 33826/225000 [75:10:34<307:37:16,  5.79s/it]
 15%|█▌        | 33827/225000 [75:10:39<298:23:37,  5.62s/it]
 15%|█▌        | 33828/225000 [75:10:45<292:26:31,  5.51s/it]
 15%|█▌        | 33829/225000 [75:10:54<358:39:25,  6.75s/it]
 15%|█▌        | 33830/225000 [75:11:04<397:53:29,  7.49s/it]
 15%|█▌        | 33831/225000 [75:11:08<342:56:41,  6.46s/it]
 15%|█▌        | 33832/225000 [75:11:14<338:03:13,  6.37s/it]
 15%|█▌        | 33833/225000 [75:11:22<371:48:54,  7.00s/it]
 15%|█▌        | 33834/225000 [75:11:27<343:11:51,  6.46s/it]
 15%|█▌        | 33835/225000 [75:11:33<322:27:41,  6.07s/it]
 15%|█▌        | 33836/225000 [75:11:38<309:08:28,  5.82s/it]
 15%|█▌        | 33837/225000 [75:11:48<370:36:30,  6.98s/it]
 15%|█▌        | 33838/225000 [75:11:56<397:21:52,  7.48s/it]
 15%|█▌        | 33839/225000 [75:12:00<337:58:26,  6.36s/it]
 15%|█▌        | 33840/225000 [75:12:08<363:47:14,  6.85s/it]
 15%|█▌        | 33841/225000 [75:12:13<338:58:12,  6.38s/it]
 15%|█▌        | 33842/225000 [75:12:18<311:46:25,  5.87s/it]
 15%|█▌        | 33843/225000 [75:12:30<404:58:46,  7.63s/it]
 15%|█▌        | 33844/225000 [75:12:36<387:14:10,  7.29s/it]
 15%|█▌        | 33845/225000 [75:12:41<343:02:38,  6.46s/it]
 15%|█▌        | 33846/225000 [75:12:49<368:45:57,  6.94s/it]
 15%|█▌        | 33847/225000 [75:12:53<329:37:05,  6.21s/it]
 15%|█▌        | 33848/225000 [75:13:03<379:14:02,  7.14s/it]
 15%|█▌        | 33849/225000 [75:13:06<317:32:49,  5.98s/it]
 15%|█▌        | 33850/225000 [75:13:15<362:43:35,  6.83s/it]
                                                             
{'loss': '0.7382', 'grad_norm': '0.6785', 'learning_rate': '4.85e-05', 'epoch': '0.1504', 'num_input_tokens_seen': 128116224, 'train_runtime': '2.708e+05', 'train_tokens_per_second': '473.1'}

 15%|█▌        | 33850/225000 [75:13:15<362:43:35,  6.83s/it]
 15%|█▌        | 33851/225000 [75:13:20<344:20:00,  6.48s/it]
 15%|█▌        | 33852/225000 [75:13:26<330:36:46,  6.23s/it]
 15%|█▌        | 33853/225000 [75:13:32<326:04:19,  6.14s/it]
 15%|█▌        | 33854/225000 [75:13:38<323:33:30,  6.09s/it]
 15%|█▌        | 33855/225000 [75:13:44<320:10:02,  6.03s/it]
 15%|█▌        | 33856/225000 [75:13:50<316:30:46,  5.96s/it]
 15%|█▌        | 33857/225000 [75:13:55<309:34:48,  5.83s/it]
 15%|█▌        | 33858/225000 [75:14:04<366:29:18,  6.90s/it]
 15%|█▌        | 33859/225000 [75:14:15<425:42:26,  8.02s/it]
 15%|█▌        | 33860/225000 [75:14:20<377:08:34,  7.10s/it]
 15%|█▌        | 33861/225000 [75:14:26<358:48:25,  6.76s/it]
 15%|█▌        | 33862/225000 [75:14:30<321:12:51,  6.05s/it]
 15%|█▌        | 33863/225000 [75:14:35<303:47:32,  5.72s/it]
 15%|█▌        | 33864/225000 [75:14:40<293:32:02,  5.53s/it]
 15%|█▌        | 33865/225000 [75:14:50<365:53:19,  6.89s/it]
 15%|█▌        | 33866/225000 [75:15:00<400:19:53,  7.54s/it]
 15%|█▌        | 33867/225000 [75:15:03<340:40:59,  6.42s/it]
 15%|█▌        | 33868/225000 [75:15:10<350:21:40,  6.60s/it]
 15%|█▌        | 33869/225000 [75:15:20<395:44:58,  7.45s/it]
 15%|█▌        | 33870/225000 [75:15:28<412:22:04,  7.77s/it]
 15%|█▌        | 33871/225000 [75:15:33<361:38:17,  6.81s/it]
 15%|█▌        | 33872/225000 [75:15:37<315:41:05,  5.95s/it]
 15%|█▌        | 33873/225000 [75:15:43<320:57:16,  6.05s/it]
 15%|█▌        | 33874/225000 [75:15:53<386:08:08,  7.27s/it]
 15%|█▌        | 33875/225000 [75:15:58<342:11:25,  6.45s/it]
 15%|█▌        | 33876/225000 [75:16:08<395:19:13,  7.45s/it]
 15%|█▌        | 33877/225000 [75:16:13<368:10:43,  6.94s/it]
 15%|█▌        | 33878/225000 [75:16:18<326:46:12,  6.16s/it]
 15%|█▌        | 33879/225000 [75:16:23<307:27:42,  5.79s/it]
 15%|█▌        | 33880/225000 [75:16:28<305:59:08,  5.76s/it]
 15%|█▌        | 33881/225000 [75:16:34<300:00:34,  5.65s/it]
 15%|█▌        | 33882/225000 [75:16:38<286:41:45,  5.40s/it]
 15%|█▌        | 33883/225000 [75:16:51<407:36:26,  7.68s/it]
 15%|█▌        | 33884/225000 [75:16:57<372:24:10,  7.01s/it]
 15%|█▌        | 33885/225000 [75:17:04<373:38:53,  7.04s/it]
 15%|█▌        | 33886/225000 [75:17:09<337:52:15,  6.36s/it]
 15%|█▌        | 33887/225000 [75:17:18<380:32:32,  7.17s/it]
 15%|█▌        | 33888/225000 [75:17:22<337:19:44,  6.35s/it]
 15%|█▌        | 33889/225000 [75:17:27<304:36:47,  5.74s/it]
 15%|█▌        | 33890/225000 [75:17:34<329:52:00,  6.21s/it]
 15%|█▌        | 33891/225000 [75:17:44<387:03:03,  7.29s/it]
 15%|█▌        | 33892/225000 [75:17:49<356:14:28,  6.71s/it]
 15%|█▌        | 33893/225000 [75:17:55<340:55:27,  6.42s/it]
 15%|█▌        | 33894/225000 [75:18:00<320:40:53,  6.04s/it]
 15%|█▌        | 33895/225000 [75:18:05<302:14:58,  5.69s/it]
 15%|█▌        | 33896/225000 [75:18:10<287:34:55,  5.42s/it]
 15%|█▌        | 33897/225000 [75:18:19<355:15:17,  6.69s/it]
 15%|█▌        | 33898/225000 [75:18:30<422:23:57,  7.96s/it]
 15%|█▌        | 33899/225000 [75:18:37<406:38:13,  7.66s/it]
 15%|█▌        | 33900/225000 [75:18:43<375:02:11,  7.07s/it]
                                                             
{'loss': '0.7489', 'grad_norm': '0.7317', 'learning_rate': '4.849e-05', 'epoch': '0.1507', 'num_input_tokens_seen': 128263600, 'train_runtime': '2.711e+05', 'train_tokens_per_second': '473.1'}

 15%|█▌        | 33900/225000 [75:18:43<375:02:11,  7.07s/it]
 15%|█▌        | 33901/225000 [75:18:48<337:06:01,  6.35s/it]
 15%|█▌        | 33902/225000 [75:18:56<368:00:25,  6.93s/it]
 15%|█▌        | 33903/225000 [75:19:03<367:41:53,  6.93s/it]
 15%|█▌        | 33904/225000 [75:19:09<362:24:23,  6.83s/it]
 15%|█▌        | 33905/225000 [75:19:16<364:01:56,  6.86s/it]
 15%|█▌        | 33906/225000 [75:19:20<318:48:54,  6.01s/it]
 15%|█▌        | 33907/225000 [75:19:32<404:08:09,  7.61s/it]
 15%|█▌        | 33908/225000 [75:19:36<357:07:37,  6.73s/it]
 15%|█▌        | 33909/225000 [75:19:41<328:38:34,  6.19s/it]
 15%|█▌        | 33910/225000 [75:19:46<301:38:04,  5.68s/it]
 15%|█▌        | 33911/225000 [75:19:53<328:06:13,  6.18s/it]
 15%|█▌        | 33912/225000 [75:20:02<374:37:57,  7.06s/it]
 15%|█▌        | 33913/225000 [75:20:07<331:35:27,  6.25s/it]
 15%|█▌        | 33914/225000 [75:20:16<382:52:04,  7.21s/it]
 15%|█▌        | 33915/225000 [75:20:22<361:24:59,  6.81s/it]
 15%|█▌        | 33916/225000 [75:20:27<335:39:58,  6.32s/it]
 15%|█▌        | 33917/225000 [75:20:36<374:50:25,  7.06s/it]
 15%|█▌        | 33918/225000 [75:20:42<366:46:16,  6.91s/it]
 15%|█▌        | 33919/225000 [75:20:49<361:34:58,  6.81s/it]
 15%|█▌        | 33920/225000 [75:20:53<310:34:11,  5.85s/it]
 15%|█▌        | 33921/225000 [75:21:02<366:52:51,  6.91s/it]
 15%|█▌        | 33922/225000 [75:21:09<371:17:41,  7.00s/it]
 15%|█▌        | 33923/225000 [75:21:16<361:32:50,  6.81s/it]
 15%|█▌        | 33924/225000 [75:21:20<330:38:09,  6.23s/it]
 15%|█▌        | 33925/225000 [75:21:25<302:00:08,  5.69s/it]
 15%|█▌        | 33926/225000 [75:21:29<274:38:14,  5.17s/it]
 15%|█▌        | 33927/225000 [75:21:40<362:43:54,  6.83s/it]
 15%|█▌        | 33928/225000 [75:21:46<354:51:52,  6.69s/it]
 15%|█▌        | 33929/225000 [75:21:50<309:39:13,  5.83s/it]
 15%|█▌        | 33930/225000 [75:21:59<366:08:14,  6.90s/it]
 15%|█▌        | 33931/225000 [75:22:03<321:17:27,  6.05s/it]
 15%|█▌        | 33932/225000 [75:22:11<346:36:29,  6.53s/it]
 15%|█▌        | 33933/225000 [75:22:18<351:48:52,  6.63s/it]
 15%|█▌        | 33934/225000 [75:22:22<317:17:16,  5.98s/it]
 15%|█▌        | 33935/225000 [75:22:27<297:28:38,  5.60s/it]
 15%|█▌        | 33936/225000 [75:22:31<281:17:28,  5.30s/it]
 15%|█▌        | 33937/225000 [75:22:44<390:34:27,  7.36s/it]
 15%|█▌        | 33938/225000 [75:22:48<346:20:54,  6.53s/it]
 15%|█▌        | 33939/225000 [75:22:54<328:08:30,  6.18s/it]
 15%|█▌        | 33940/225000 [75:22:58<306:58:25,  5.78s/it]
 15%|█▌        | 33941/225000 [75:23:03<292:29:35,  5.51s/it]
 15%|█▌        | 33942/225000 [75:23:09<293:04:19,  5.52s/it]
 15%|█▌        | 33943/225000 [75:23:14<285:54:34,  5.39s/it]
 15%|█▌        | 33944/225000 [75:23:19<285:14:24,  5.37s/it]
 15%|█▌        | 33945/225000 [75:23:25<294:58:51,  5.56s/it]
 15%|█▌        | 33946/225000 [75:23:39<421:24:13,  7.94s/it]
 15%|█▌        | 33947/225000 [75:23:47<421:47:27,  7.95s/it]
 15%|█▌        | 33948/225000 [75:23:51<356:50:46,  6.72s/it]
 15%|█▌        | 33949/225000 [75:23:55<316:32:20,  5.96s/it]
 15%|█▌        | 33950/225000 [75:24:00<297:42:39,  5.61s/it]
                                                             
{'loss': '0.7309', 'grad_norm': '0.7682', 'learning_rate': '4.849e-05', 'epoch': '0.1509', 'num_input_tokens_seen': 128408608, 'train_runtime': '2.714e+05', 'train_tokens_per_second': '473.1'}

 15%|█▌        | 33950/225000 [75:24:00<297:42:39,  5.61s/it]
 15%|█▌        | 33951/225000 [75:24:11<386:56:31,  7.29s/it]
 15%|█▌        | 33952/225000 [75:24:17<372:58:34,  7.03s/it]
 15%|█▌        | 33953/225000 [75:24:22<329:57:00,  6.22s/it]
 15%|█▌        | 33954/225000 [75:24:27<312:33:38,  5.89s/it]
 15%|█▌        | 33955/225000 [75:24:31<291:47:22,  5.50s/it]
 15%|█▌        | 33956/225000 [75:24:43<395:18:23,  7.45s/it]
 15%|█▌        | 33957/225000 [75:24:50<384:14:53,  7.24s/it]
 15%|█▌        | 33958/225000 [75:24:58<402:31:32,  7.59s/it]
 15%|█▌        | 33959/225000 [75:25:04<373:19:37,  7.04s/it]
 15%|█▌        | 33960/225000 [75:25:13<404:34:45,  7.62s/it]
 15%|█▌        | 33961/225000 [75:25:19<379:52:39,  7.16s/it]
 15%|█▌        | 33962/225000 [75:25:23<319:55:29,  6.03s/it]
 15%|█▌        | 33963/225000 [75:25:31<354:54:37,  6.69s/it]
 15%|█▌        | 33964/225000 [75:25:36<325:46:06,  6.14s/it]
 15%|█▌        | 33965/225000 [75:25:40<303:52:19,  5.73s/it]
 15%|█▌        | 33966/225000 [75:25:46<293:09:33,  5.52s/it]
 15%|█▌        | 33967/225000 [75:25:55<350:33:33,  6.61s/it]
 15%|█▌        | 33968/225000 [75:26:03<381:10:34,  7.18s/it]
 15%|█▌        | 33969/225000 [75:26:11<387:04:21,  7.29s/it]
 15%|█▌        | 33970/225000 [75:26:15<337:27:10,  6.36s/it]
 15%|█▌        | 33971/225000 [75:26:20<318:33:16,  6.00s/it]
 15%|█▌        | 33972/225000 [75:26:26<309:53:02,  5.84s/it]
 15%|█▌        | 33973/225000 [75:26:38<409:24:53,  7.72s/it]
 15%|█▌        | 33974/225000 [75:26:43<379:27:57,  7.15s/it]
 15%|█▌        | 33975/225000 [75:26:51<382:20:50,  7.21s/it]
 15%|█▌        | 33976/225000 [75:26:55<335:24:22,  6.32s/it]
 15%|█▌        | 33977/225000 [75:27:00<306:06:36,  5.77s/it]
 15%|█▌        | 33978/225000 [75:27:10<380:34:50,  7.17s/it]
 15%|█▌        | 33979/225000 [75:27:16<356:26:42,  6.72s/it]
 15%|█▌        | 33980/225000 [75:27:24<378:47:33,  7.14s/it]
 15%|█▌        | 33981/225000 [75:27:28<335:31:33,  6.32s/it]
 15%|█▌        | 33982/225000 [75:27:33<312:47:53,  5.90s/it]
 15%|█▌        | 33983/225000 [75:27:44<388:29:01,  7.32s/it]
 15%|█▌        | 33984/225000 [75:27:49<350:17:13,  6.60s/it]
 15%|█▌        | 33985/225000 [75:27:53<318:21:28,  6.00s/it]
 15%|█▌        | 33986/225000 [75:27:58<299:55:20,  5.65s/it]
 15%|█▌        | 33987/225000 [75:28:04<306:21:58,  5.77s/it]
 15%|█▌        | 33988/225000 [75:28:17<424:01:33,  7.99s/it]
 15%|█▌        | 33989/225000 [75:28:22<365:05:53,  6.88s/it]
 15%|█▌        | 33990/225000 [75:28:26<317:48:21,  5.99s/it]
 15%|█▌        | 33991/225000 [75:28:31<306:48:21,  5.78s/it]
 15%|█▌        | 33992/225000 [75:28:36<299:56:40,  5.65s/it]
 15%|█▌        | 33993/225000 [75:28:41<286:05:18,  5.39s/it]
 15%|█▌        | 33994/225000 [75:28:46<281:11:53,  5.30s/it]
 15%|█▌        | 33995/225000 [75:28:51<280:45:12,  5.29s/it]
 15%|█▌        | 33996/225000 [75:29:01<358:31:07,  6.76s/it]
 15%|█▌        | 33997/225000 [75:29:13<428:47:30,  8.08s/it]
 15%|█▌        | 33998/225000 [75:29:16<357:41:49,  6.74s/it]
 15%|█▌        | 33999/225000 [75:29:25<396:43:49,  7.48s/it]
 15%|█▌        | 34000/225000 [75:29:31<371:28:05,  7.00s/it]
                                                             
{'loss': '0.7247', 'grad_norm': '0.7156', 'learning_rate': '4.848e-05', 'epoch': '0.1511', 'num_input_tokens_seen': 128554544, 'train_runtime': '2.718e+05', 'train_tokens_per_second': '473'}

 15%|█▌        | 34000/225000 [75:29:31<371:28:05,  7.00s/it]
 15%|█▌        | 34001/225000 [75:29:36<341:10:56,  6.43s/it]
 15%|█▌        | 34002/225000 [75:29:43<342:57:38,  6.46s/it]
 15%|█▌        | 34003/225000 [75:29:49<329:34:30,  6.21s/it]
 15%|█▌        | 34004/225000 [75:29:54<315:04:45,  5.94s/it]
 15%|█▌        | 34005/225000 [75:29:59<307:24:49,  5.79s/it]
 15%|█▌        | 34006/225000 [75:30:05<310:22:56,  5.85s/it]
 15%|█▌        | 34007/225000 [75:30:11<310:06:04,  5.85s/it]
 15%|█▌        | 34008/225000 [75:30:17<314:27:36,  5.93s/it]
 15%|█▌        | 34009/225000 [75:30:23<305:47:34,  5.76s/it]
 15%|█▌        | 34010/225000 [75:30:36<417:59:16,  7.88s/it]
 15%|█▌        | 34011/225000 [75:30:44<431:55:41,  8.14s/it]
 15%|█▌        | 34012/225000 [75:30:48<356:29:57,  6.72s/it]
 15%|█▌        | 34013/225000 [75:30:57<396:37:20,  7.48s/it]
 15%|█▌        | 34014/225000 [75:31:02<351:43:53,  6.63s/it]
 15%|█▌        | 34015/225000 [75:31:06<324:28:12,  6.12s/it]
 15%|█▌        | 34016/225000 [75:31:11<305:59:33,  5.77s/it]
 15%|█▌        | 34017/225000 [75:31:16<291:16:02,  5.49s/it]
 15%|█▌        | 34018/225000 [75:31:27<375:21:45,  7.08s/it]
 15%|█▌        | 34019/225000 [75:31:34<380:20:56,  7.17s/it]
 15%|█▌        | 34020/225000 [75:31:41<375:58:42,  7.09s/it]
 15%|█▌        | 34021/225000 [75:31:48<369:58:53,  6.97s/it]
 15%|█▌        | 34022/225000 [75:31:53<335:50:47,  6.33s/it]
 15%|█▌        | 34023/225000 [75:31:58<309:09:27,  5.83s/it]
 15%|█▌        | 34024/225000 [75:32:02<292:53:11,  5.52s/it]
 15%|█▌        | 34025/225000 [75:32:13<374:33:39,  7.06s/it]
 15%|█▌        | 34026/225000 [75:32:20<381:12:38,  7.19s/it]
 15%|█▌        | 34027/225000 [75:32:28<388:09:00,  7.32s/it]
 15%|█▌        | 34028/225000 [75:32:32<338:03:14,  6.37s/it]
 15%|█▌        | 34029/225000 [75:32:38<322:41:36,  6.08s/it]
 15%|█▌        | 34030/225000 [75:32:43<317:23:12,  5.98s/it]
 15%|█▌        | 34031/225000 [75:32:57<433:27:25,  8.17s/it]
 15%|█▌        | 34032/225000 [75:33:00<361:34:53,  6.82s/it]
 15%|█▌        | 34033/225000 [75:33:07<359:09:18,  6.77s/it]
 15%|█▌        | 34034/225000 [75:33:12<324:44:23,  6.12s/it]
 15%|█▌        | 34035/225000 [75:33:17<311:41:28,  5.88s/it]
 15%|█▌        | 34036/225000 [75:33:22<302:22:14,  5.70s/it]
 15%|█▌        | 34037/225000 [75:33:28<300:47:14,  5.67s/it]
 15%|█▌        | 34038/225000 [75:33:34<301:53:56,  5.69s/it]
 15%|█▌        | 34039/225000 [75:33:43<355:29:32,  6.70s/it]
 15%|█▌        | 34040/225000 [75:33:52<404:38:06,  7.63s/it]
 15%|█▌        | 34041/225000 [75:33:58<366:25:39,  6.91s/it]
 15%|█▌        | 34042/225000 [75:34:02<333:38:34,  6.29s/it]
 15%|█▌        | 34043/225000 [75:34:08<313:59:29,  5.92s/it]
 15%|█▌        | 34044/225000 [75:34:13<309:30:12,  5.83s/it]
 15%|█▌        | 34045/225000 [75:34:24<388:46:44,  7.33s/it]
 15%|█▌        | 34046/225000 [75:34:29<345:37:39,  6.52s/it]
 15%|█▌        | 34047/225000 [75:34:33<314:48:34,  5.94s/it]
 15%|█▌        | 34048/225000 [75:34:37<288:33:52,  5.44s/it]
 15%|█▌        | 34049/225000 [75:34:44<311:47:23,  5.88s/it]
 15%|█▌        | 34050/225000 [75:34:58<435:38:55,  8.21s/it]
                                                             
{'loss': '0.7477', 'grad_norm': '0.662', 'learning_rate': '4.847e-05', 'epoch': '0.1513', 'num_input_tokens_seen': 128701144, 'train_runtime': '2.721e+05', 'train_tokens_per_second': '473'}

 15%|█▌        | 34050/225000 [75:34:58<435:38:55,  8.21s/it]
 15%|█▌        | 34051/225000 [75:35:02<367:19:03,  6.93s/it]
 15%|█▌        | 34052/225000 [75:35:08<350:04:54,  6.60s/it]
 15%|█▌        | 34053/225000 [75:35:15<364:35:22,  6.87s/it]
 15%|█▌        | 34054/225000 [75:35:19<321:26:32,  6.06s/it]
 15%|█▌        | 34055/225000 [75:35:24<301:47:04,  5.69s/it]
 15%|█▌        | 34056/225000 [75:35:34<373:15:24,  7.04s/it]
 15%|█▌        | 34057/225000 [75:35:39<331:47:03,  6.26s/it]
 15%|█▌        | 34058/225000 [75:35:44<311:41:31,  5.88s/it]
 15%|█▌        | 34059/225000 [75:35:49<299:43:58,  5.65s/it]
 15%|█▌        | 34060/225000 [75:35:55<305:55:49,  5.77s/it]
 15%|█▌        | 34061/225000 [75:36:06<393:22:28,  7.42s/it]
 15%|█▌        | 34062/225000 [75:36:11<347:05:56,  6.54s/it]
 15%|█▌        | 34063/225000 [75:36:16<326:41:52,  6.16s/it]
 15%|█▌        | 34064/225000 [75:36:21<306:55:04,  5.79s/it]
 15%|█▌        | 34065/225000 [75:36:26<292:35:38,  5.52s/it]
 15%|█▌        | 34066/225000 [75:36:34<340:36:05,  6.42s/it]
 15%|█▌        | 34067/225000 [75:36:45<404:25:44,  7.63s/it]
 15%|█▌        | 34068/225000 [75:36:52<399:41:49,  7.54s/it]
 15%|█▌        | 34069/225000 [75:36:58<369:43:26,  6.97s/it]
 15%|█▌        | 34070/225000 [75:37:03<338:23:31,  6.38s/it]
 15%|█▌        | 34071/225000 [75:37:08<323:05:37,  6.09s/it]
 15%|█▌        | 34072/225000 [75:37:13<305:57:41,  5.77s/it]
 15%|█▌        | 34073/225000 [75:37:18<293:59:53,  5.54s/it]
 15%|█▌        | 34074/225000 [75:37:24<290:21:33,  5.47s/it]
 15%|█▌        | 34075/225000 [75:37:36<400:37:37,  7.55s/it]
 15%|█▌        | 34076/225000 [75:37:40<342:49:06,  6.46s/it]
 15%|█▌        | 34077/225000 [75:37:47<353:16:37,  6.66s/it]
 15%|█▌        | 34078/225000 [75:37:55<379:47:50,  7.16s/it]
 15%|█▌        | 34079/225000 [75:38:00<345:39:07,  6.52s/it]
 15%|█▌        | 34080/225000 [75:38:06<327:36:03,  6.18s/it]
 15%|█▌        | 34081/225000 [75:38:11<307:10:21,  5.79s/it]
 15%|█▌        | 34082/225000 [75:38:16<297:37:49,  5.61s/it]
 15%|█▌        | 34083/225000 [75:38:21<288:07:24,  5.43s/it]
 15%|█▌        | 34084/225000 [75:38:35<422:36:17,  7.97s/it]
 15%|█▌        | 34085/225000 [75:38:39<357:22:21,  6.74s/it]
 15%|█▌        | 34086/225000 [75:38:50<423:33:30,  7.99s/it]
 15%|█▌        | 34087/225000 [75:38:54<365:30:55,  6.89s/it]
 15%|█▌        | 34088/225000 [75:39:00<352:06:53,  6.64s/it]
 15%|█▌        | 34089/225000 [75:39:04<306:53:14,  5.79s/it]
 15%|█▌        | 34090/225000 [75:39:12<342:19:59,  6.46s/it]
 15%|█▌        | 34091/225000 [75:39:21<382:34:16,  7.21s/it]
 15%|█▌        | 34092/225000 [75:39:25<338:30:35,  6.38s/it]
 15%|█▌        | 34093/225000 [75:39:30<307:05:00,  5.79s/it]
 15%|█▌        | 34094/225000 [75:39:41<402:00:54,  7.58s/it]
 15%|█▌        | 34095/225000 [75:39:45<343:57:41,  6.49s/it]
 15%|█▌        | 34096/225000 [75:39:54<374:51:34,  7.07s/it]
 15%|█▌        | 34097/225000 [75:39:59<348:15:19,  6.57s/it]
 15%|█▌        | 34098/225000 [75:40:04<317:51:44,  5.99s/it]
 15%|█▌        | 34099/225000 [75:40:12<352:21:06,  6.64s/it]
 15%|█▌        | 34100/225000 [75:40:19<354:59:27,  6.69s/it]
                                                             
{'loss': '0.7523', 'grad_norm': '0.7435', 'learning_rate': '4.847e-05', 'epoch': '0.1516', 'num_input_tokens_seen': 128846312, 'train_runtime': '2.724e+05', 'train_tokens_per_second': '473'}

 15%|█▌        | 34100/225000 [75:40:19<354:59:27,  6.69s/it]
 15%|█▌        | 34101/225000 [75:40:24<332:07:11,  6.26s/it]
 15%|█▌        | 34102/225000 [75:40:29<318:53:13,  6.01s/it]
 15%|█▌        | 34103/225000 [75:40:34<296:19:45,  5.59s/it]
 15%|█▌        | 34104/225000 [75:40:44<372:23:58,  7.02s/it]
 15%|█▌        | 34105/225000 [75:40:48<325:38:03,  6.14s/it]
 15%|█▌        | 34106/225000 [75:40:57<357:29:49,  6.74s/it]
 15%|█▌        | 34107/225000 [75:41:04<375:30:29,  7.08s/it]
 15%|█▌        | 34108/225000 [75:41:12<380:19:03,  7.17s/it]
 15%|█▌        | 34109/225000 [75:41:16<338:27:15,  6.38s/it]
 15%|█▌        | 34110/225000 [75:41:22<319:26:11,  6.02s/it]
 15%|█▌        | 34111/225000 [75:41:26<293:04:08,  5.53s/it]
 15%|█▌        | 34112/225000 [75:41:31<281:46:52,  5.31s/it]
 15%|█▌        | 34113/225000 [75:41:43<398:13:04,  7.51s/it]
 15%|█▌        | 34114/225000 [75:41:50<379:07:08,  7.15s/it]
 15%|█▌        | 34115/225000 [75:41:55<349:32:02,  6.59s/it]
 15%|█▌        | 34116/225000 [75:42:01<332:27:25,  6.27s/it]
 15%|█▌        | 34117/225000 [75:42:06<315:18:03,  5.95s/it]
 15%|█▌        | 34118/225000 [75:42:16<377:01:36,  7.11s/it]
 15%|█▌        | 34119/225000 [75:42:19<325:00:13,  6.13s/it]
 15%|█▌        | 34120/225000 [75:42:26<333:48:43,  6.30s/it]
 15%|█▌        | 34121/225000 [75:42:34<365:17:57,  6.89s/it]
 15%|█▌        | 34122/225000 [75:42:41<359:49:08,  6.79s/it]
 15%|█▌        | 34123/225000 [75:42:45<309:40:26,  5.84s/it]
 15%|█▌        | 34124/225000 [75:42:50<308:28:04,  5.82s/it]
 15%|█▌        | 34125/225000 [75:42:59<351:39:10,  6.63s/it]
 15%|█▌        | 34126/225000 [75:43:04<330:09:33,  6.23s/it]
 15%|█▌        | 34127/225000 [75:43:09<315:52:59,  5.96s/it]
 15%|█▌        | 34128/225000 [75:43:15<308:33:32,  5.82s/it]
 15%|█▌        | 34129/225000 [75:43:20<299:21:00,  5.65s/it]
 15%|█▌        | 34130/225000 [75:43:26<301:36:23,  5.69s/it]
 15%|█▌        | 34131/225000 [75:43:31<295:38:29,  5.58s/it]
 15%|█▌        | 34132/225000 [75:43:37<295:50:33,  5.58s/it]
 15%|█▌        | 34133/225000 [75:43:43<304:33:40,  5.74s/it]
 15%|█▌        | 34134/225000 [75:43:58<452:20:07,  8.53s/it]
 15%|█▌        | 34135/225000 [75:44:03<388:44:55,  7.33s/it]
 15%|█▌        | 34136/225000 [75:44:09<372:35:06,  7.03s/it]
 15%|█▌        | 34137/225000 [75:44:17<387:35:22,  7.31s/it]
 15%|█▌        | 34138/225000 [75:44:22<360:40:38,  6.80s/it]
 15%|█▌        | 34139/225000 [75:44:27<329:59:45,  6.22s/it]
 15%|█▌        | 34140/225000 [75:44:32<309:43:41,  5.84s/it]
 15%|█▌        | 34141/225000 [75:44:38<314:36:26,  5.93s/it]
 15%|█▌        | 34142/225000 [75:44:50<405:07:17,  7.64s/it]
 15%|█▌        | 34143/225000 [75:44:54<341:39:13,  6.44s/it]
 15%|█▌        | 34144/225000 [75:45:04<402:18:55,  7.59s/it]
 15%|█▌        | 34145/225000 [75:45:10<375:12:07,  7.08s/it]
 15%|█▌        | 34146/225000 [75:45:16<356:38:50,  6.73s/it]
 15%|█▌        | 34147/225000 [75:45:21<337:34:17,  6.37s/it]
 15%|█▌        | 34148/225000 [75:45:28<350:02:50,  6.60s/it]
 15%|█▌        | 34149/225000 [75:45:35<354:04:23,  6.68s/it]
 15%|█▌        | 34150/225000 [75:45:40<319:11:54,  6.02s/it]
                                                             
{'loss': '0.7287', 'grad_norm': '0.739', 'learning_rate': '4.846e-05', 'epoch': '0.1518', 'num_input_tokens_seen': 128991184, 'train_runtime': '2.727e+05', 'train_tokens_per_second': '472.9'}

 15%|█▌        | 34150/225000 [75:45:40<319:11:54,  6.02s/it]
 15%|█▌        | 34151/225000 [75:45:45<301:03:05,  5.68s/it]
 15%|█▌        | 34152/225000 [75:45:49<287:04:13,  5.42s/it]
 15%|█▌        | 34153/225000 [75:46:02<400:37:28,  7.56s/it]
 15%|█▌        | 34154/225000 [75:46:08<378:11:10,  7.13s/it]
 15%|█▌        | 34155/225000 [75:46:13<339:45:35,  6.41s/it]
 15%|█▌        | 34156/225000 [75:46:18<314:44:39,  5.94s/it]
 15%|█▌        | 34157/225000 [75:46:23<301:38:27,  5.69s/it]
 15%|█▌        | 34158/225000 [75:46:28<287:09:17,  5.42s/it]
 15%|█▌        | 34159/225000 [75:46:33<286:42:58,  5.41s/it]
 15%|█▌        | 34160/225000 [75:46:46<409:50:44,  7.73s/it]
 15%|█▌        | 34161/225000 [75:46:53<399:16:22,  7.53s/it]
 15%|█▌        | 34162/225000 [75:47:01<403:29:40,  7.61s/it]
 15%|█▌        | 34163/225000 [75:47:09<406:41:22,  7.67s/it]
 15%|█▌        | 34164/225000 [75:47:13<348:29:15,  6.57s/it]
 15%|█▌        | 34165/225000 [75:47:17<314:31:08,  5.93s/it]
 15%|█▌        | 34166/225000 [75:47:27<381:01:36,  7.19s/it]
 15%|█▌        | 34167/225000 [75:47:31<329:54:29,  6.22s/it]
 15%|█▌        | 34168/225000 [75:47:36<300:54:06,  5.68s/it]
 15%|█▌        | 34169/225000 [75:47:44<347:44:29,  6.56s/it]
 15%|█▌        | 34170/225000 [75:47:50<328:29:32,  6.20s/it]
 15%|█▌        | 34171/225000 [75:47:57<351:57:08,  6.64s/it]
 15%|█▌        | 34172/225000 [75:48:07<398:04:35,  7.51s/it]
 15%|█▌        | 34173/225000 [75:48:11<337:39:49,  6.37s/it]
 15%|█▌        | 34174/225000 [75:48:20<390:04:59,  7.36s/it]
 15%|█▌        | 34175/225000 [75:48:25<352:10:57,  6.64s/it]
 15%|█▌        | 34176/225000 [75:48:30<315:14:36,  5.95s/it]
 15%|█▌        | 34177/225000 [75:48:35<299:39:10,  5.65s/it]
 15%|█▌        | 34178/225000 [75:48:40<289:05:10,  5.45s/it]
 15%|█▌        | 34179/225000 [75:48:45<281:35:43,  5.31s/it]
 15%|█▌        | 34180/225000 [75:48:51<300:19:36,  5.67s/it]
 15%|█▌        | 34181/225000 [75:49:03<400:14:17,  7.55s/it]
 15%|█▌        | 34182/225000 [75:49:07<343:59:12,  6.49s/it]
 15%|█▌        | 34183/225000 [75:49:17<400:27:59,  7.56s/it]
 15%|█▌        | 34184/225000 [75:49:21<338:34:26,  6.39s/it]
 15%|█▌        | 34185/225000 [75:49:30<392:13:54,  7.40s/it]
 15%|█▌        | 34186/225000 [75:49:37<381:44:29,  7.20s/it]
 15%|█▌        | 34187/225000 [75:49:43<353:16:19,  6.67s/it]
 15%|█▌        | 34188/225000 [75:49:51<372:25:00,  7.03s/it]
 15%|█▌        | 34189/225000 [75:49:55<339:23:12,  6.40s/it]
 15%|█▌        | 34190/225000 [75:50:01<322:00:46,  6.08s/it]
 15%|█▌        | 34191/225000 [75:50:06<306:24:04,  5.78s/it]
 15%|█▌        | 34192/225000 [75:50:11<303:17:03,  5.72s/it]
 15%|█▌        | 34193/225000 [75:50:17<299:19:57,  5.65s/it]
 15%|█▌        | 34194/225000 [75:50:22<291:20:11,  5.50s/it]
 15%|█▌        | 34195/225000 [75:50:27<287:14:48,  5.42s/it]
 15%|█▌        | 34196/225000 [75:50:33<284:50:48,  5.37s/it]
 15%|█▌        | 34197/225000 [75:50:38<289:02:20,  5.45s/it]
 15%|█▌        | 34198/225000 [75:50:51<408:18:44,  7.70s/it]
 15%|█▌        | 34199/225000 [75:50:59<415:42:33,  7.84s/it]
 15%|█▌        | 34200/225000 [75:51:06<401:33:33,  7.58s/it]
                                                             
{'loss': '0.7256', 'grad_norm': '0.6591', 'learning_rate': '4.846e-05', 'epoch': '0.152', 'num_input_tokens_seen': 129138112, 'train_runtime': '2.731e+05', 'train_tokens_per_second': '472.9'}

 15%|█▌        | 34200/225000 [75:51:06<401:33:33,  7.58s/it]
 15%|█▌        | 34201/225000 [75:51:12<365:41:14,  6.90s/it]
 15%|█▌        | 34202/225000 [75:51:19<376:09:12,  7.10s/it]
 15%|█▌        | 34203/225000 [75:51:23<328:56:36,  6.21s/it]
 15%|█▌        | 34204/225000 [75:51:28<301:00:48,  5.68s/it]
 15%|█▌        | 34205/225000 [75:51:33<286:39:18,  5.41s/it]
 15%|█▌        | 34206/225000 [75:51:46<413:04:24,  7.79s/it]
 15%|█▌        | 34207/225000 [75:51:52<378:38:16,  7.14s/it]
 15%|█▌        | 34208/225000 [75:51:56<340:57:37,  6.43s/it]
 15%|█▌        | 34209/225000 [75:52:01<314:33:41,  5.94s/it]
 15%|█▌        | 34210/225000 [75:52:06<291:44:49,  5.50s/it]
 15%|█▌        | 34211/225000 [75:52:14<337:34:03,  6.37s/it]
 15%|█▌        | 34212/225000 [75:52:23<375:03:09,  7.08s/it]
 15%|█▌        | 34213/225000 [75:52:33<425:31:17,  8.03s/it]
 15%|█▌        | 34214/225000 [75:52:36<349:17:31,  6.59s/it]
 15%|█▌        | 34215/225000 [75:52:42<343:46:29,  6.49s/it]
 15%|█▌        | 34216/225000 [75:52:46<301:46:57,  5.69s/it]
 15%|█▌        | 34217/225000 [75:52:51<286:02:51,  5.40s/it]
 15%|█▌        | 34218/225000 [75:53:00<342:08:21,  6.46s/it]
 15%|█▌        | 34219/225000 [75:53:07<360:03:18,  6.79s/it]
 15%|█▌        | 34220/225000 [75:53:12<322:23:08,  6.08s/it]
 15%|█▌        | 34221/225000 [75:53:19<340:34:12,  6.43s/it]
 15%|█▌        | 34222/225000 [75:53:29<402:59:15,  7.60s/it]
 15%|█▌        | 34223/225000 [75:53:34<348:06:05,  6.57s/it]
 15%|█▌        | 34224/225000 [75:53:42<375:03:20,  7.08s/it]
 15%|█▌        | 34225/225000 [75:53:48<353:24:19,  6.67s/it]
 15%|█▌        | 34226/225000 [75:53:55<360:02:43,  6.79s/it]
 15%|█▌        | 34227/225000 [75:53:59<324:04:16,  6.12s/it]
 15%|█▌        | 34228/225000 [75:54:10<390:28:42,  7.37s/it]
 15%|█▌        | 34229/225000 [75:54:16<369:13:19,  6.97s/it]
 15%|█▌        | 34230/225000 [75:54:21<338:21:39,  6.39s/it]
 15%|█▌        | 34231/225000 [75:54:28<352:44:36,  6.66s/it]
 15%|█▌        | 34232/225000 [75:54:35<355:49:06,  6.71s/it]
 15%|█▌        | 34233/225000 [75:54:42<359:11:37,  6.78s/it]
 15%|█▌        | 34234/225000 [75:54:50<377:47:23,  7.13s/it]
 15%|█▌        | 34235/225000 [75:54:54<327:32:01,  6.18s/it]
 15%|█▌        | 34236/225000 [75:54:58<300:22:06,  5.67s/it]
 15%|█▌        | 34237/225000 [75:55:03<287:12:47,  5.42s/it]
 15%|█▌        | 34238/225000 [75:55:12<353:55:58,  6.68s/it]
 15%|█▌        | 34239/225000 [75:55:20<370:58:49,  7.00s/it]
 15%|█▌        | 34240/225000 [75:55:27<374:15:59,  7.06s/it]
 15%|█▌        | 34241/225000 [75:55:31<317:49:39,  6.00s/it]
 15%|█▌        | 34242/225000 [75:55:40<366:34:37,  6.92s/it]
 15%|█▌        | 34243/225000 [75:55:44<326:02:53,  6.15s/it]
 15%|█▌        | 34244/225000 [75:55:49<306:34:33,  5.79s/it]
 15%|█▌        | 34245/225000 [75:55:53<279:08:28,  5.27s/it]
 15%|█▌        | 34246/225000 [75:56:06<399:17:00,  7.54s/it]
 15%|█▌        | 34247/225000 [75:56:11<363:07:53,  6.85s/it]
 15%|█▌        | 34248/225000 [75:56:16<328:55:56,  6.21s/it]
 15%|█▌        | 34249/225000 [75:56:26<380:08:49,  7.17s/it]
 15%|█▌        | 34250/225000 [75:56:30<329:31:42,  6.22s/it]
                                                             
{'loss': '0.72', 'grad_norm': '0.7188', 'learning_rate': '4.845e-05', 'epoch': '0.1522', 'num_input_tokens_seen': 129283928, 'train_runtime': '2.734e+05', 'train_tokens_per_second': '472.9'}

 15%|█▌        | 34250/225000 [75:56:30<329:31:42,  6.22s/it]
 15%|█▌        | 34251/225000 [75:56:34<301:03:16,  5.68s/it]
 15%|█▌        | 34252/225000 [75:56:39<293:00:00,  5.53s/it]
 15%|█▌        | 34253/225000 [75:56:44<286:45:58,  5.41s/it]
 15%|█▌        | 34254/225000 [75:56:50<287:10:47,  5.42s/it]
 15%|█▌        | 34255/225000 [75:56:56<303:43:23,  5.73s/it]
 15%|█▌        | 34256/225000 [75:57:09<418:53:23,  7.91s/it]
 15%|█▌        | 34257/225000 [75:57:13<360:03:56,  6.80s/it]
 15%|█▌        | 34258/225000 [75:57:18<321:22:04,  6.07s/it]
 15%|█▌        | 34259/225000 [75:57:22<292:58:33,  5.53s/it]
 15%|█▌        | 34260/225000 [75:57:29<323:01:47,  6.10s/it]
 15%|█▌        | 34261/225000 [75:57:41<406:40:44,  7.68s/it]
 15%|█▌        | 34262/225000 [75:57:45<348:28:11,  6.58s/it]
 15%|█▌        | 34263/225000 [75:57:49<312:57:36,  5.91s/it]
 15%|█▌        | 34264/225000 [75:57:59<368:08:02,  6.95s/it]
 15%|█▌        | 34265/225000 [75:58:07<395:32:25,  7.47s/it]
 15%|█▌        | 34266/225000 [75:58:11<337:19:43,  6.37s/it]
 15%|█▌        | 34267/225000 [75:58:21<391:17:34,  7.39s/it]
 15%|█▌        | 34268/225000 [75:58:27<365:33:53,  6.90s/it]
 15%|█▌        | 34269/225000 [75:58:31<319:05:26,  6.02s/it]
 15%|█▌        | 34270/225000 [75:58:39<355:54:29,  6.72s/it]
 15%|█▌        | 34271/225000 [75:58:47<385:45:11,  7.28s/it]
 15%|█▌        | 34272/225000 [75:58:54<375:35:48,  7.09s/it]
 15%|█▌        | 34273/225000 [75:59:02<383:25:59,  7.24s/it]
 15%|█▌        | 34274/225000 [75:59:06<329:48:53,  6.23s/it]
 15%|█▌        | 34275/225000 [75:59:12<340:10:21,  6.42s/it]
 15%|█▌        | 34276/225000 [75:59:17<309:13:08,  5.84s/it]
 15%|█▌        | 34277/225000 [75:59:22<290:00:16,  5.47s/it]
 15%|█▌        | 34278/225000 [75:59:26<278:51:42,  5.26s/it]
 15%|█▌        | 34279/225000 [75:59:35<335:50:26,  6.34s/it]
 15%|█▌        | 34280/225000 [75:59:44<374:08:12,  7.06s/it]
 15%|█▌        | 34281/225000 [75:59:48<332:06:24,  6.27s/it]
 15%|█▌        | 34282/225000 [75:59:57<369:03:55,  6.97s/it]
 15%|█▌        | 34283/225000 [76:00:01<321:10:27,  6.06s/it]
 15%|█▌        | 34284/225000 [76:00:05<293:48:35,  5.55s/it]
 15%|█▌        | 34285/225000 [76:00:15<354:11:54,  6.69s/it]
 15%|█▌        | 34286/225000 [76:00:21<352:57:12,  6.66s/it]
 15%|█▌        | 34287/225000 [76:00:26<318:52:56,  6.02s/it]
 15%|█▌        | 34288/225000 [76:00:30<297:22:38,  5.61s/it]
 15%|█▌        | 34289/225000 [76:00:37<307:32:06,  5.81s/it]
 15%|█▌        | 34290/225000 [76:00:49<414:49:05,  7.83s/it]
 15%|█▌        | 34291/225000 [76:00:53<345:21:49,  6.52s/it]
 15%|█▌        | 34292/225000 [76:01:02<387:09:02,  7.31s/it]
 15%|█▌        | 34293/225000 [76:01:06<340:53:19,  6.43s/it]
 15%|█▌        | 34294/225000 [76:01:11<316:46:37,  5.98s/it]
 15%|█▌        | 34295/225000 [76:01:20<358:55:25,  6.78s/it]
 15%|█▌        | 34296/225000 [76:01:28<380:10:39,  7.18s/it]
 15%|█▌        | 34297/225000 [76:01:32<338:52:24,  6.40s/it]
 15%|█▌        | 34298/225000 [76:01:37<313:46:29,  5.92s/it]
 15%|█▌        | 34299/225000 [76:01:42<301:22:54,  5.69s/it]
 15%|█▌        | 34300/225000 [76:01:48<300:25:27,  5.67s/it]
                                                             
{'loss': '0.7282', 'grad_norm': '0.7573', 'learning_rate': '4.844e-05', 'epoch': '0.1524', 'num_input_tokens_seen': 129429608, 'train_runtime': '2.737e+05', 'train_tokens_per_second': '472.9'}

 15%|█▌        | 34300/225000 [76:01:48<300:25:27,  5.67s/it]
 15%|█▌        | 34301/225000 [76:02:00<395:53:35,  7.47s/it]
 15%|█▌        | 34302/225000 [76:02:06<382:30:37,  7.22s/it]
 15%|█▌        | 34303/225000 [76:02:14<384:39:03,  7.26s/it]
 15%|█▌        | 34304/225000 [76:02:20<368:50:51,  6.96s/it]
 15%|█▌        | 34305/225000 [76:02:28<388:00:14,  7.32s/it]
 15%|█▌        | 34306/225000 [76:02:33<344:18:28,  6.50s/it]
 15%|█▌        | 34307/225000 [76:02:38<323:32:13,  6.11s/it]
 15%|█▌        | 34308/225000 [76:02:47<371:11:00,  7.01s/it]
 15%|█▌        | 34309/225000 [76:02:55<382:38:55,  7.22s/it]
 15%|█▌        | 34310/225000 [76:03:01<372:01:21,  7.02s/it]
 15%|█▌        | 34311/225000 [76:03:08<373:33:10,  7.05s/it]
 15%|█▌        | 34312/225000 [76:03:15<365:09:49,  6.89s/it]
 15%|█▌        | 34313/225000 [76:03:19<320:27:26,  6.05s/it]
 15%|█▌        | 34314/225000 [76:03:23<289:26:07,  5.46s/it]
 15%|█▌        | 34315/225000 [76:03:36<403:02:23,  7.61s/it]
 15%|█▌        | 34316/225000 [76:03:42<381:28:44,  7.20s/it]
 15%|█▌        | 34317/225000 [76:03:46<333:34:11,  6.30s/it]
 15%|█▌        | 34318/225000 [76:03:53<349:55:05,  6.61s/it]
 15%|█▌        | 34319/225000 [76:03:58<309:43:19,  5.85s/it]
 15%|█▌        | 34320/225000 [76:04:05<336:49:06,  6.36s/it]
 15%|█▌        | 34321/225000 [76:04:13<358:47:54,  6.77s/it]
 15%|█▌        | 34322/225000 [76:04:19<349:21:56,  6.60s/it]
 15%|█▌        | 34323/225000 [76:04:26<351:40:24,  6.64s/it]
 15%|█▌        | 34324/225000 [76:04:30<319:42:23,  6.04s/it]
 15%|█▌        | 34325/225000 [76:04:35<303:37:35,  5.73s/it]
 15%|█▌        | 34326/225000 [76:04:41<304:16:37,  5.74s/it]
 15%|█▌        | 34327/225000 [76:04:46<282:14:56,  5.33s/it]
 15%|█▌        | 34328/225000 [76:04:58<390:03:06,  7.36s/it]
 15%|█▌        | 34329/225000 [76:05:02<348:54:30,  6.59s/it]
 15%|█▌        | 34330/225000 [76:05:11<375:41:27,  7.09s/it]
 15%|█▌        | 34331/225000 [76:05:16<344:53:26,  6.51s/it]
 15%|█▌        | 34332/225000 [76:05:20<313:08:00,  5.91s/it]
 15%|█▌        | 34333/225000 [76:05:25<297:00:12,  5.61s/it]
 15%|█▌        | 34334/225000 [76:05:30<288:54:48,  5.46s/it]
 15%|█▌        | 34335/225000 [76:05:35<279:10:23,  5.27s/it]
 15%|█▌        | 34336/225000 [76:05:45<346:22:17,  6.54s/it]
 15%|█▌        | 34337/225000 [76:05:56<418:21:20,  7.90s/it]
 15%|█▌        | 34338/225000 [76:05:59<347:07:20,  6.55s/it]
 15%|█▌        | 34339/225000 [76:06:04<315:47:20,  5.96s/it]
 15%|█▌        | 34340/225000 [76:06:09<306:39:32,  5.79s/it]
 15%|█▌        | 34341/225000 [76:06:14<291:49:47,  5.51s/it]
 15%|█▌        | 34342/225000 [76:06:27<405:46:08,  7.66s/it]
 15%|█▌        | 34343/225000 [76:06:32<375:29:44,  7.09s/it]
 15%|█▌        | 34344/225000 [76:06:40<383:54:56,  7.25s/it]
 15%|█▌        | 34345/225000 [76:06:45<344:31:48,  6.51s/it]
 15%|█▌        | 34346/225000 [76:06:50<317:07:09,  5.99s/it]
 15%|█▌        | 34347/225000 [76:06:55<300:17:26,  5.67s/it]
 15%|█▌        | 34348/225000 [76:07:01<310:55:53,  5.87s/it]
 15%|█▌        | 34349/225000 [76:07:13<410:12:05,  7.75s/it]
 15%|█▌        | 34350/225000 [76:07:17<351:46:30,  6.64s/it]
                                                             
{'loss': '0.7353', 'grad_norm': '0.7631', 'learning_rate': '4.844e-05', 'epoch': '0.1527', 'num_input_tokens_seen': 129575560, 'train_runtime': '2.74e+05', 'train_tokens_per_second': '472.8'}

 15%|█▌        | 34350/225000 [76:07:17<351:46:30,  6.64s/it]
 15%|█▌        | 34351/225000 [76:07:23<341:07:44,  6.44s/it]
 15%|█▌        | 34352/225000 [76:07:32<385:40:22,  7.28s/it]
 15%|█▌        | 34353/225000 [76:07:38<353:36:17,  6.68s/it]
 15%|█▌        | 34354/225000 [76:07:46<385:53:01,  7.29s/it]
 15%|█▌        | 34355/225000 [76:07:52<361:16:18,  6.82s/it]
 15%|█▌        | 34356/225000 [76:07:57<327:48:22,  6.19s/it]
 15%|█▌        | 34357/225000 [76:08:02<306:57:11,  5.80s/it]
 15%|█▌        | 34358/225000 [76:08:07<294:01:49,  5.55s/it]
 15%|█▌        | 34359/225000 [76:08:12<288:21:30,  5.45s/it]
 15%|█▌        | 34360/225000 [76:08:17<286:31:39,  5.41s/it]
 15%|█▌        | 34361/225000 [76:08:22<282:42:42,  5.34s/it]
 15%|█▌        | 34362/225000 [76:08:33<371:47:50,  7.02s/it]
 15%|█▌        | 34363/225000 [76:08:42<400:06:01,  7.56s/it]
 15%|█▌        | 34364/225000 [76:08:47<355:35:02,  6.71s/it]
 15%|█▌        | 34365/225000 [76:08:54<366:56:02,  6.93s/it]
 15%|█▌        | 34366/225000 [76:08:58<320:59:55,  6.06s/it]
 15%|█▌        | 34367/225000 [76:09:05<330:02:26,  6.23s/it]
 15%|█▌        | 34368/225000 [76:09:14<375:28:10,  7.09s/it]
 15%|█▌        | 34369/225000 [76:09:20<358:34:04,  6.77s/it]
 15%|█▌        | 34370/225000 [76:09:26<340:06:53,  6.42s/it]
 15%|█▌        | 34371/225000 [76:09:31<317:49:38,  6.00s/it]
 15%|█▌        | 34372/225000 [76:09:40<375:54:57,  7.10s/it]
 15%|█▌        | 34373/225000 [76:09:47<364:29:00,  6.88s/it]
 15%|█▌        | 34374/225000 [76:09:54<373:14:30,  7.05s/it]
 15%|█▌        | 34375/225000 [76:09:59<330:39:18,  6.24s/it]
 15%|█▌        | 34376/225000 [76:10:06<342:51:53,  6.48s/it]
 15%|█▌        | 34377/225000 [76:10:13<366:01:17,  6.91s/it]
 15%|█▌        | 34378/225000 [76:10:18<326:01:42,  6.16s/it]
 15%|█▌        | 34379/225000 [76:10:23<304:17:24,  5.75s/it]
 15%|█▌        | 34380/225000 [76:10:28<291:04:27,  5.50s/it]
 15%|█▌        | 34381/225000 [76:10:35<325:40:06,  6.15s/it]
 15%|█▌        | 34382/225000 [76:10:45<384:44:08,  7.27s/it]
 15%|█▌        | 34383/225000 [76:10:49<327:01:38,  6.18s/it]
 15%|█▌        | 34384/225000 [76:10:59<389:48:30,  7.36s/it]
 15%|█▌        | 34385/225000 [76:11:04<347:54:51,  6.57s/it]
 15%|█▌        | 34386/225000 [76:11:10<347:27:19,  6.56s/it]
 15%|█▌        | 34387/225000 [76:11:15<316:31:28,  5.98s/it]
 15%|█▌        | 34388/225000 [76:11:20<300:15:12,  5.67s/it]
 15%|█▌        | 34389/225000 [76:11:25<292:12:35,  5.52s/it]
 15%|█▌        | 34390/225000 [76:11:37<390:31:33,  7.38s/it]
 15%|█▌        | 34391/225000 [76:11:41<343:55:47,  6.50s/it]
 15%|█▌        | 34392/225000 [76:11:49<363:16:16,  6.86s/it]
 15%|█▌        | 34393/225000 [76:11:53<319:28:56,  6.03s/it]
 15%|█▌        | 34394/225000 [76:12:04<402:20:59,  7.60s/it]
 15%|█▌        | 34395/225000 [76:12:10<376:53:54,  7.12s/it]
 15%|█▌        | 34396/225000 [76:12:16<359:08:31,  6.78s/it]
 15%|█▌        | 34397/225000 [76:12:21<332:56:01,  6.29s/it]
 15%|█▌        | 34398/225000 [76:12:30<371:18:02,  7.01s/it]
 15%|█▌        | 34399/225000 [76:12:36<349:04:55,  6.59s/it]
 15%|█▌        | 34400/225000 [76:12:41<333:51:38,  6.31s/it]
                                                             
{'loss': '0.7538', 'grad_norm': '0.7391', 'learning_rate': '4.843e-05', 'epoch': '0.1529', 'num_input_tokens_seen': 129720832, 'train_runtime': '2.744e+05', 'train_tokens_per_second': '472.8'}

 15%|█▌        | 34400/225000 [76:12:41<333:51:38,  6.31s/it]
 15%|█▌        | 34401/225000 [76:12:45<297:00:13,  5.61s/it]
 15%|█▌        | 34402/225000 [76:12:50<277:32:04,  5.24s/it]
 15%|█▌        | 34403/225000 [76:12:55<286:41:05,  5.41s/it]
 15%|█▌        | 34404/225000 [76:13:08<403:59:48,  7.63s/it]
 15%|█▌        | 34405/225000 [76:13:12<340:51:50,  6.44s/it]
 15%|█▌        | 34406/225000 [76:13:18<337:25:29,  6.37s/it]
 15%|█▌        | 34407/225000 [76:13:26<366:13:24,  6.92s/it]
 15%|█▌        | 34408/225000 [76:13:30<309:44:13,  5.85s/it]
 15%|█▌        | 34409/225000 [76:13:35<298:15:14,  5.63s/it]
 15%|█▌        | 34410/225000 [76:13:40<293:18:07,  5.54s/it]
 15%|█▌        | 34411/225000 [76:13:53<403:50:31,  7.63s/it]
 15%|█▌        | 34412/225000 [76:13:57<350:06:07,  6.61s/it]
 15%|█▌        | 34413/225000 [76:14:03<346:03:21,  6.54s/it]
 15%|█▌        | 34414/225000 [76:14:07<308:42:26,  5.83s/it]
 15%|█▌        | 34415/225000 [76:14:18<378:32:35,  7.15s/it]
 15%|█▌        | 34416/225000 [76:14:21<315:41:07,  5.96s/it]
 15%|█▌        | 34417/225000 [76:14:29<356:32:15,  6.73s/it]
 15%|█▌        | 34418/225000 [76:14:34<328:33:06,  6.21s/it]
 15%|█▌        | 34419/225000 [76:14:39<312:17:53,  5.90s/it]
 15%|█▌        | 34420/225000 [76:14:44<295:28:32,  5.58s/it]
 15%|█▌        | 34421/225000 [76:14:51<312:11:08,  5.90s/it]
 15%|█▌        | 34422/225000 [76:15:03<402:47:05,  7.61s/it]
 15%|█▌        | 34423/225000 [76:15:07<349:54:54,  6.61s/it]
 15%|█▌        | 34424/225000 [76:15:11<316:43:59,  5.98s/it]
 15%|█▌        | 34425/225000 [76:15:21<376:17:50,  7.11s/it]
 15%|█▌        | 34426/225000 [76:15:28<377:45:46,  7.14s/it]
 15%|█▌        | 34427/225000 [76:15:34<359:47:46,  6.80s/it]
 15%|█▌        | 34428/225000 [76:15:44<408:28:27,  7.72s/it]
 15%|█▌        | 34429/225000 [76:15:49<364:09:43,  6.88s/it]
 15%|█▌        | 34430/225000 [76:15:54<332:17:50,  6.28s/it]
 15%|█▌        | 34431/225000 [76:15:59<320:10:48,  6.05s/it]
 15%|█▌        | 34432/225000 [76:16:05<306:40:44,  5.79s/it]
 15%|█▌        | 34433/225000 [76:16:10<292:42:00,  5.53s/it]
 15%|█▌        | 34434/225000 [76:16:20<373:29:45,  7.06s/it]
 15%|█▌        | 34435/225000 [76:16:28<384:36:07,  7.27s/it]
 15%|█▌        | 34436/225000 [76:16:34<367:24:29,  6.94s/it]
 15%|█▌        | 34437/225000 [76:16:38<327:12:12,  6.18s/it]
 15%|█▌        | 34438/225000 [76:16:44<323:01:04,  6.10s/it]
 15%|█▌        | 34439/225000 [76:16:57<427:10:40,  8.07s/it]
 15%|█▌        | 34440/225000 [76:17:01<353:58:54,  6.69s/it]
 15%|█▌        | 34441/225000 [76:17:09<378:39:09,  7.15s/it]
 15%|█▌        | 34442/225000 [76:17:13<338:42:40,  6.40s/it]
 15%|█▌        | 34443/225000 [76:17:20<348:57:57,  6.59s/it]
 15%|█▌        | 34444/225000 [76:17:29<372:36:24,  7.04s/it]
 15%|█▌        | 34445/225000 [76:17:34<339:51:46,  6.42s/it]
 15%|█▌        | 34446/225000 [76:17:38<314:00:07,  5.93s/it]
 15%|█▌        | 34447/225000 [76:17:43<296:41:10,  5.61s/it]
 15%|█▌        | 34448/225000 [76:17:55<396:54:15,  7.50s/it]
 15%|█▌        | 34449/225000 [76:18:02<381:37:59,  7.21s/it]
 15%|█▌        | 34450/225000 [76:18:06<331:43:14,  6.27s/it]
                                                             
{'loss': '0.7438', 'grad_norm': '0.76', 'learning_rate': '4.842e-05', 'epoch': '0.1531', 'num_input_tokens_seen': 129867456, 'train_runtime': '2.747e+05', 'train_tokens_per_second': '472.8'}

 15%|█▌        | 34450/225000 [76:18:06<331:43:14,  6.27s/it]
 15%|█▌        | 34451/225000 [76:18:16<390:56:52,  7.39s/it]
 15%|█▌        | 34452/225000 [76:18:20<347:00:53,  6.56s/it]
 15%|█▌        | 34453/225000 [76:18:25<310:18:21,  5.86s/it]
 15%|█▌        | 34454/225000 [76:18:33<357:41:50,  6.76s/it]
 15%|█▌        | 34455/225000 [76:18:40<349:41:30,  6.61s/it]
 15%|█▌        | 34456/225000 [76:18:45<326:03:00,  6.16s/it]
 15%|█▌        | 34457/225000 [76:18:50<310:25:10,  5.86s/it]
 15%|█▌        | 34458/225000 [76:18:58<347:35:02,  6.57s/it]
 15%|█▌        | 34459/225000 [76:19:08<396:38:29,  7.49s/it]
 15%|█▌        | 34460/225000 [76:19:12<346:52:47,  6.55s/it]
 15%|█▌        | 34461/225000 [76:19:21<376:26:33,  7.11s/it]
 15%|█▌        | 34462/225000 [76:19:26<351:07:30,  6.63s/it]
 15%|█▌        | 34463/225000 [76:19:33<356:18:18,  6.73s/it]
 15%|█▌        | 34464/225000 [76:19:38<335:57:51,  6.35s/it]
 15%|█▌        | 34465/225000 [76:19:43<314:01:41,  5.93s/it]
 15%|█▌        | 34466/225000 [76:19:52<360:40:35,  6.81s/it]
 15%|█▌        | 34467/225000 [76:19:57<320:05:55,  6.05s/it]
 15%|█▌        | 34468/225000 [76:20:01<291:20:22,  5.50s/it]
 15%|█▌        | 34469/225000 [76:20:12<386:43:31,  7.31s/it]
 15%|█▌        | 34470/225000 [76:20:18<356:12:45,  6.73s/it]
 15%|█▌        | 34471/225000 [76:20:23<336:07:32,  6.35s/it]
 15%|█▌        | 34472/225000 [76:20:28<316:38:48,  5.98s/it]
 15%|█▌        | 34473/225000 [76:20:34<307:41:04,  5.81s/it]
 15%|█▌        | 34474/225000 [76:20:39<298:10:09,  5.63s/it]
 15%|█▌        | 34475/225000 [76:20:45<297:11:12,  5.62s/it]
 15%|█▌        | 34476/225000 [76:20:50<292:47:07,  5.53s/it]
 15%|█▌        | 34477/225000 [76:21:04<423:11:44,  8.00s/it]
 15%|█▌        | 34478/225000 [76:21:10<390:48:13,  7.38s/it]
 15%|█▌        | 34479/225000 [76:21:14<336:17:05,  6.35s/it]
 15%|█▌        | 34480/225000 [76:21:23<380:13:32,  7.18s/it]
 15%|█▌        | 34481/225000 [76:21:28<354:36:31,  6.70s/it]
 15%|█▌        | 34482/225000 [76:21:34<335:38:52,  6.34s/it]
 15%|█▌        | 34483/225000 [76:21:39<319:44:32,  6.04s/it]
 15%|█▌        | 34484/225000 [76:21:45<311:08:08,  5.88s/it]
 15%|█▌        | 34485/225000 [76:21:50<302:55:17,  5.72s/it]
 15%|█▌        | 34486/225000 [76:21:55<297:38:52,  5.62s/it]
 15%|█▌        | 34487/225000 [76:22:01<291:22:36,  5.51s/it]
 15%|█▌        | 34488/225000 [76:22:06<297:48:42,  5.63s/it]
 15%|█▌        | 34489/225000 [76:22:19<413:56:49,  7.82s/it]
 15%|█▌        | 34490/225000 [76:22:25<386:30:37,  7.30s/it]
 15%|█▌        | 34491/225000 [76:22:30<343:01:24,  6.48s/it]
 15%|█▌        | 34492/225000 [76:22:36<330:54:33,  6.25s/it]
 15%|█▌        | 34493/225000 [76:22:46<393:11:47,  7.43s/it]
 15%|█▌        | 34494/225000 [76:22:50<345:56:33,  6.54s/it]
 15%|█▌        | 34495/225000 [76:22:55<310:59:14,  5.88s/it]
 15%|█▌        | 34496/225000 [76:23:05<377:16:52,  7.13s/it]
 15%|█▌        | 34497/225000 [76:23:12<379:22:14,  7.17s/it]
 15%|█▌        | 34498/225000 [76:23:19<381:53:45,  7.22s/it]
 15%|█▌        | 34499/225000 [76:23:24<335:12:37,  6.33s/it]
 15%|█▌        | 34500/225000 [76:23:28<301:58:52,  5.71s/it]
                                                             
{'loss': '0.7622', 'grad_norm': '0.7872', 'learning_rate': '4.842e-05', 'epoch': '0.1533', 'num_input_tokens_seen': 130013160, 'train_runtime': '2.75e+05', 'train_tokens_per_second': '472.8'}

 15%|█▌        | 34500/225000 [76:23:28<301:58:52,  5.71s/it]
 15%|█▌        | 34501/225000 [76:23:39<390:37:32,  7.38s/it]
 15%|█▌        | 34502/225000 [76:23:46<374:02:00,  7.07s/it]
 15%|█▌        | 34503/225000 [76:23:50<339:39:07,  6.42s/it]
 15%|█▌        | 34504/225000 [76:23:56<319:46:49,  6.04s/it]
 15%|█▌        | 34505/225000 [76:24:03<333:50:51,  6.31s/it]
 15%|█▌        | 34506/225000 [76:24:10<347:15:20,  6.56s/it]
 15%|█▌        | 34507/225000 [76:24:14<319:02:12,  6.03s/it]
 15%|█▌        | 34508/225000 [76:24:20<303:25:47,  5.73s/it]
 15%|█▌        | 34509/225000 [76:24:25<298:24:21,  5.64s/it]
 15%|█▌        | 34510/225000 [76:24:38<415:10:34,  7.85s/it]
 15%|█▌        | 34511/225000 [76:24:43<366:17:30,  6.92s/it]
 15%|█▌        | 34512/225000 [76:24:47<330:21:56,  6.24s/it]
 15%|█▌        | 34513/225000 [76:24:53<320:12:13,  6.05s/it]
 15%|█▌        | 34514/225000 [76:25:03<389:35:25,  7.36s/it]
 15%|█▌        | 34515/225000 [76:25:08<340:00:05,  6.43s/it]
 15%|█▌        | 34516/225000 [76:25:12<305:18:55,  5.77s/it]
 15%|█▌        | 34517/225000 [76:25:23<385:29:44,  7.29s/it]
 15%|█▌        | 34518/225000 [76:25:27<330:59:28,  6.26s/it]
 15%|█▌        | 34519/225000 [76:25:35<359:39:42,  6.80s/it]
 15%|█▌        | 34520/225000 [76:25:39<318:27:32,  6.02s/it]
 15%|█▌        | 34521/225000 [76:25:47<354:41:00,  6.70s/it]
 15%|█▌        | 34522/225000 [76:25:50<300:32:27,  5.68s/it]
 15%|█▌        | 34523/225000 [76:25:58<327:00:05,  6.18s/it]
 15%|█▌        | 34524/225000 [76:26:02<298:46:58,  5.65s/it]
 15%|█▌        | 34525/225000 [76:26:07<288:11:17,  5.45s/it]
 15%|█▌        | 34526/225000 [76:26:12<282:03:45,  5.33s/it]
 15%|█▌        | 34527/225000 [76:26:23<367:04:15,  6.94s/it]
 15%|█▌        | 34528/225000 [76:26:29<359:25:23,  6.79s/it]
 15%|█▌        | 34529/225000 [76:26:34<320:45:06,  6.06s/it]
 15%|█▌        | 34530/225000 [76:26:38<294:01:28,  5.56s/it]
 15%|█▌        | 34531/225000 [76:26:48<357:01:54,  6.75s/it]
 15%|█▌        | 34532/225000 [76:26:55<368:51:48,  6.97s/it]
 15%|█▌        | 34533/225000 [76:27:01<347:24:53,  6.57s/it]
 15%|█▌        | 34534/225000 [76:27:07<339:07:03,  6.41s/it]
 15%|█▌        | 34535/225000 [76:27:12<328:38:09,  6.21s/it]
 15%|█▌        | 34536/225000 [76:27:18<320:47:24,  6.06s/it]
 15%|█▌        | 34537/225000 [76:27:24<316:29:53,  5.98s/it]
 15%|█▌        | 34538/225000 [76:27:34<387:59:31,  7.33s/it]
 15%|█▌        | 34539/225000 [76:27:41<375:36:19,  7.10s/it]
 15%|█▌        | 34540/225000 [76:27:50<405:22:46,  7.66s/it]
 15%|█▌        | 34541/225000 [76:27:54<345:49:25,  6.54s/it]
 15%|█▌        | 34542/225000 [76:28:04<399:17:12,  7.55s/it]
 15%|█▌        | 34543/225000 [76:28:07<333:20:33,  6.30s/it]
 15%|█▌        | 34544/225000 [76:28:16<371:11:15,  7.02s/it]
 15%|█▌        | 34545/225000 [76:28:20<323:43:26,  6.12s/it]
 15%|█▌        | 34546/225000 [76:28:24<296:50:50,  5.61s/it]
 15%|█▌        | 34547/225000 [76:28:35<371:11:01,  7.02s/it]
 15%|█▌        | 34548/225000 [76:28:39<324:57:19,  6.14s/it]
 15%|█▌        | 34549/225000 [76:28:43<292:02:24,  5.52s/it]
 15%|█▌        | 34550/225000 [76:28:48<283:05:31,  5.35s/it]
                                                             
{'loss': '0.7475', 'grad_norm': '0.6799', 'learning_rate': '4.841e-05', 'epoch': '0.1536', 'num_input_tokens_seen': 130156336, 'train_runtime': '2.753e+05', 'train_tokens_per_second': '472.7'}

 15%|█▌        | 34550/225000 [76:28:48<283:05:31,  5.35s/it]
 15%|█▌        | 34551/225000 [76:28:58<360:29:48,  6.81s/it]
 15%|█▌        | 34552/225000 [76:29:03<325:33:26,  6.15s/it]
 15%|█▌        | 34553/225000 [76:29:10<345:26:32,  6.53s/it]
 15%|█▌        | 34554/225000 [76:29:15<328:25:06,  6.21s/it]
 15%|█▌        | 34555/225000 [76:29:24<366:58:57,  6.94s/it]
 15%|█▌        | 34556/225000 [76:29:30<343:45:44,  6.50s/it]
 15%|█▌        | 34557/225000 [76:29:38<372:44:44,  7.05s/it]
 15%|█▌        | 34558/225000 [76:29:45<373:25:57,  7.06s/it]
 15%|█▌        | 34559/225000 [76:29:50<339:25:45,  6.42s/it]
 15%|█▌        | 34560/225000 [76:29:55<313:15:43,  5.92s/it]
 15%|█▌        | 34561/225000 [76:29:59<293:51:33,  5.56s/it]
 15%|█▌        | 34562/225000 [76:30:10<381:33:19,  7.21s/it]
 15%|█▌        | 34563/225000 [76:30:15<339:58:22,  6.43s/it]
 15%|█▌        | 34564/225000 [76:30:20<313:43:31,  5.93s/it]
 15%|█▌        | 34565/225000 [76:30:24<288:22:31,  5.45s/it]
 15%|█▌        | 34566/225000 [76:30:29<281:14:16,  5.32s/it]
 15%|█▌        | 34567/225000 [76:30:38<330:40:01,  6.25s/it]
 15%|█▌        | 34568/225000 [76:30:47<386:05:11,  7.30s/it]
 15%|█▌        | 34569/225000 [76:30:51<332:49:08,  6.29s/it]
 15%|█▌        | 34570/225000 [76:30:59<351:38:46,  6.65s/it]
 15%|█▌        | 34571/225000 [76:31:05<350:31:27,  6.63s/it]
 15%|█▌        | 34572/225000 [76:31:12<355:56:33,  6.73s/it]
 15%|█▌        | 34573/225000 [76:31:19<353:58:42,  6.69s/it]
 15%|█▌        | 34574/225000 [76:31:23<313:33:08,  5.93s/it]
 15%|█▌        | 34575/225000 [76:31:32<368:01:27,  6.96s/it]
 15%|█▌        | 34576/225000 [76:31:36<311:41:51,  5.89s/it]
 15%|█▌        | 34577/225000 [76:31:40<282:53:50,  5.35s/it]
 15%|█▌        | 34578/225000 [76:31:45<282:26:02,  5.34s/it]
 15%|█▌        | 34579/225000 [76:31:57<382:01:19,  7.22s/it]
 15%|█▌        | 34580/225000 [76:32:02<342:16:42,  6.47s/it]
 15%|█▌        | 34581/225000 [76:32:10<366:12:14,  6.92s/it]
 15%|█▌        | 34582/225000 [76:32:14<333:35:26,  6.31s/it]
 15%|█▌        | 34583/225000 [76:32:20<327:44:18,  6.20s/it]
 15%|█▌        | 34584/225000 [76:32:31<402:50:47,  7.62s/it]
 15%|█▌        | 34585/225000 [76:32:38<387:15:25,  7.32s/it]
 15%|█▌        | 34586/225000 [76:32:42<331:55:21,  6.28s/it]
 15%|█▌        | 34587/225000 [76:32:51<374:08:18,  7.07s/it]
 15%|█▌        | 34588/225000 [76:32:55<330:36:49,  6.25s/it]
 15%|█▌        | 34589/225000 [76:32:59<301:41:18,  5.70s/it]
 15%|█▌        | 34590/225000 [76:33:06<307:27:42,  5.81s/it]
 15%|█▌        | 34591/225000 [76:33:15<358:08:44,  6.77s/it]
 15%|█▌        | 34592/225000 [76:33:19<328:08:07,  6.20s/it]
 15%|█▌        | 34593/225000 [76:33:25<313:10:55,  5.92s/it]
 15%|█▌        | 34594/225000 [76:33:30<296:45:12,  5.61s/it]
 15%|█▌        | 34595/225000 [76:33:35<299:00:15,  5.65s/it]
 15%|█▌        | 34596/225000 [76:33:48<411:17:14,  7.78s/it]
 15%|█▌        | 34597/225000 [76:33:53<371:47:30,  7.03s/it]
 15%|█▌        | 34598/225000 [76:33:58<328:19:15,  6.21s/it]
 15%|█▌        | 34599/225000 [76:34:02<297:29:59,  5.62s/it]
 15%|█▌        | 34600/225000 [76:34:08<303:25:50,  5.74s/it]
                                                             
{'loss': '0.7412', 'grad_norm': '0.7058', 'learning_rate': '4.84e-05', 'epoch': '0.1538', 'num_input_tokens_seen': 130301600, 'train_runtime': '2.756e+05', 'train_tokens_per_second': '472.7'}

 15%|█▌        | 34600/225000 [76:34:08<303:25:50,  5.74s/it]
 15%|█▌        | 34601/225000 [76:34:18<379:14:19,  7.17s/it]
 15%|█▌        | 34602/225000 [76:34:22<326:47:27,  6.18s/it]
 15%|█▌        | 34603/225000 [76:34:26<287:52:41,  5.44s/it]
 15%|█▌        | 34604/225000 [76:34:35<352:32:34,  6.67s/it]
 15%|█▌        | 34605/225000 [76:34:41<337:35:46,  6.38s/it]
 15%|█▌        | 34606/225000 [76:34:45<300:35:26,  5.68s/it]
 15%|█▌        | 34607/225000 [76:34:55<359:32:08,  6.80s/it]
 15%|█▌        | 34608/225000 [76:35:00<334:29:35,  6.32s/it]
 15%|█▌        | 34609/225000 [76:35:04<304:39:14,  5.76s/it]
 15%|█▌        | 34610/225000 [76:35:09<286:02:11,  5.41s/it]
 15%|█▌        | 34611/225000 [76:35:20<378:43:06,  7.16s/it]
 15%|█▌        | 34612/225000 [76:35:28<384:18:33,  7.27s/it]
 15%|█▌        | 34613/225000 [76:35:33<345:55:54,  6.54s/it]
 15%|█▌        | 34614/225000 [76:35:38<330:18:09,  6.25s/it]
 15%|█▌        | 34615/225000 [76:35:43<304:30:07,  5.76s/it]
 15%|█▌        | 34616/225000 [76:35:48<297:16:51,  5.62s/it]
 15%|█▌        | 34617/225000 [76:36:00<395:52:44,  7.49s/it]
 15%|█▌        | 34618/225000 [76:36:04<347:25:50,  6.57s/it]
 15%|█▌        | 34619/225000 [76:36:10<327:09:31,  6.19s/it]
 15%|█▌        | 34620/225000 [76:36:15<318:22:09,  6.02s/it]
 15%|█▌        | 34621/225000 [76:36:22<327:55:10,  6.20s/it]
 15%|█▌        | 34622/225000 [76:36:27<306:20:53,  5.79s/it]
 15%|█▌        | 34623/225000 [76:36:40<420:14:54,  7.95s/it]
 15%|█▌        | 34624/225000 [76:36:44<367:17:01,  6.95s/it]
 15%|█▌        | 34625/225000 [76:36:49<325:30:05,  6.16s/it]
 15%|█▌        | 34626/225000 [76:36:53<304:17:56,  5.75s/it]
 15%|█▌        | 34627/225000 [76:36:59<298:05:28,  5.64s/it]
 15%|█▌        | 34628/225000 [76:37:12<417:16:00,  7.89s/it]
 15%|█▌        | 34629/225000 [76:37:16<357:21:56,  6.76s/it]
 15%|█▌        | 34630/225000 [76:37:20<312:43:02,  5.91s/it]
 15%|█▌        | 34631/225000 [76:37:28<347:02:27,  6.56s/it]
 15%|█▌        | 34632/225000 [76:37:36<366:29:17,  6.93s/it]
 15%|█▌        | 34633/225000 [76:37:42<354:09:24,  6.70s/it]
 15%|█▌        | 34634/225000 [76:37:51<396:01:46,  7.49s/it]
 15%|█▌        | 34635/225000 [76:37:56<358:10:13,  6.77s/it]
 15%|█▌        | 34636/225000 [76:38:01<321:22:44,  6.08s/it]
 15%|█▌        | 34637/225000 [76:38:06<306:58:15,  5.81s/it]
 15%|█▌        | 34638/225000 [76:38:13<322:16:42,  6.09s/it]
 15%|█▌        | 34639/225000 [76:38:23<394:43:52,  7.46s/it]
 15%|█▌        | 34640/225000 [76:38:27<336:56:30,  6.37s/it]
 15%|█▌        | 34641/225000 [76:38:31<297:33:33,  5.63s/it]
 15%|█▌        | 34642/225000 [76:38:40<348:28:01,  6.59s/it]
 15%|█▌        | 34643/225000 [76:38:49<381:15:18,  7.21s/it]
 15%|█▌        | 34644/225000 [76:38:53<339:32:21,  6.42s/it]
 15%|█▌        | 34645/225000 [76:38:58<308:35:01,  5.84s/it]
 15%|█▌        | 34646/225000 [76:39:04<310:38:03,  5.87s/it]
 15%|█▌        | 34647/225000 [76:39:14<388:49:24,  7.35s/it]
 15%|█▌        | 34648/225000 [76:39:21<372:06:26,  7.04s/it]
 15%|█▌        | 34649/225000 [76:39:25<325:34:24,  6.16s/it]
 15%|█▌        | 34650/225000 [76:39:29<295:39:27,  5.59s/it]
                                                             
{'loss': '0.7673', 'grad_norm': '0.8623', 'learning_rate': '4.84e-05', 'epoch': '0.154', 'num_input_tokens_seen': 130447792, 'train_runtime': '2.76e+05', 'train_tokens_per_second': '472.7'}

 15%|█▌        | 34650/225000 [76:39:29<295:39:27,  5.59s/it]
 15%|█▌        | 34651/225000 [76:39:38<354:22:51,  6.70s/it]
 15%|█▌        | 34652/225000 [76:39:44<336:34:20,  6.37s/it]
 15%|█▌        | 34653/225000 [76:39:52<357:47:16,  6.77s/it]
 15%|█▌        | 34654/225000 [76:39:55<306:01:51,  5.79s/it]
 15%|█▌        | 34655/225000 [76:40:04<349:34:47,  6.61s/it]
 15%|█▌        | 34656/225000 [76:40:09<325:21:44,  6.15s/it]
 15%|█▌        | 34657/225000 [76:40:15<318:50:26,  6.03s/it]
 15%|█▌        | 34658/225000 [76:40:19<298:03:06,  5.64s/it]
 15%|█▌        | 34659/225000 [76:40:25<292:29:34,  5.53s/it]
 15%|█▌        | 34660/225000 [76:40:30<287:12:11,  5.43s/it]
 15%|█▌        | 34661/225000 [76:40:35<280:10:40,  5.30s/it]
 15%|█▌        | 34662/225000 [76:40:48<403:42:10,  7.64s/it]
 15%|█▌        | 34663/225000 [76:40:54<378:53:34,  7.17s/it]
 15%|█▌        | 34664/225000 [76:40:57<320:41:47,  6.07s/it]
 15%|█▌        | 34665/225000 [76:41:02<288:33:35,  5.46s/it]
 15%|█▌        | 34666/225000 [76:41:12<374:21:20,  7.08s/it]
 15%|█▌        | 34667/225000 [76:41:17<334:24:23,  6.33s/it]
 15%|█▌        | 34668/225000 [76:41:21<293:39:49,  5.55s/it]
 15%|█▌        | 34669/225000 [76:41:31<366:44:18,  6.94s/it]
 15%|█▌        | 34670/225000 [76:41:36<334:07:04,  6.32s/it]
 15%|█▌        | 34671/225000 [76:41:41<321:07:31,  6.07s/it]
 15%|█▌        | 34672/225000 [76:41:46<305:11:38,  5.77s/it]
 15%|█▌        | 34673/225000 [76:41:52<299:57:46,  5.67s/it]
 15%|█▌        | 34674/225000 [76:42:00<344:04:13,  6.51s/it]
 15%|█▌        | 34675/225000 [76:42:06<332:09:10,  6.28s/it]
 15%|█▌        | 34676/225000 [76:42:11<319:29:46,  6.04s/it]
 15%|█▌        | 34677/225000 [76:42:21<371:39:39,  7.03s/it]
 15%|█▌        | 34678/225000 [76:42:29<384:09:03,  7.27s/it]
 15%|█▌        | 34679/225000 [76:42:35<362:53:59,  6.86s/it]
 15%|█▌        | 34680/225000 [76:42:39<328:48:09,  6.22s/it]
 15%|█▌        | 34681/225000 [76:42:44<309:42:38,  5.86s/it]
 15%|█▌        | 34682/225000 [76:42:49<299:46:25,  5.67s/it]
 15%|█▌        | 34683/225000 [76:42:55<294:57:40,  5.58s/it]
 15%|█▌        | 34684/225000 [76:43:05<362:53:09,  6.86s/it]
 15%|█▌        | 34685/225000 [76:43:15<418:32:53,  7.92s/it]
 15%|█▌        | 34686/225000 [76:43:19<354:58:56,  6.71s/it]
 15%|█▌        | 34687/225000 [76:43:23<314:43:18,  5.95s/it]
 15%|█▌        | 34688/225000 [76:43:31<338:15:43,  6.40s/it]
 15%|█▌        | 34689/225000 [76:43:40<388:39:25,  7.35s/it]
 15%|█▌        | 34690/225000 [76:43:44<336:58:40,  6.37s/it]
 15%|█▌        | 34691/225000 [76:43:49<303:52:50,  5.75s/it]
 15%|█▌        | 34692/225000 [76:43:58<368:54:48,  6.98s/it]
 15%|█▌        | 34693/225000 [76:44:04<348:25:01,  6.59s/it]
 15%|█▌        | 34694/225000 [76:44:08<309:45:36,  5.86s/it]
 15%|█▌        | 34695/225000 [76:44:18<379:20:19,  7.18s/it]
 15%|█▌        | 34696/225000 [76:44:24<352:10:47,  6.66s/it]
 15%|█▌        | 34697/225000 [76:44:30<338:35:11,  6.41s/it]
 15%|█▌        | 34698/225000 [76:44:37<354:04:53,  6.70s/it]
 15%|█▌        | 34699/225000 [76:44:41<316:45:31,  5.99s/it]
 15%|█▌        | 34700/225000 [76:44:46<296:18:23,  5.61s/it]
                                                             
{'loss': '0.7427', 'grad_norm': '0.8443', 'learning_rate': '4.839e-05', 'epoch': '0.1542', 'num_input_tokens_seen': 130591080, 'train_runtime': '2.763e+05', 'train_tokens_per_second': '472.7'}

 15%|█▌        | 34700/225000 [76:44:46<296:18:23,  5.61s/it]
 15%|█▌        | 34701/225000 [76:44:57<384:11:00,  7.27s/it]
 15%|█▌        | 34702/225000 [76:45:02<346:47:03,  6.56s/it]
 15%|█▌        | 34703/225000 [76:45:07<320:33:43,  6.06s/it]
 15%|█▌        | 34704/225000 [76:45:12<298:32:01,  5.65s/it]
 15%|█▌        | 34705/225000 [76:45:17<286:30:53,  5.42s/it]
 15%|█▌        | 34706/225000 [76:45:22<277:30:21,  5.25s/it]
 15%|█▌        | 34707/225000 [76:45:29<318:46:19,  6.03s/it]
 15%|█▌        | 34708/225000 [76:45:40<396:32:34,  7.50s/it]
 15%|█▌        | 34709/225000 [76:45:44<337:27:47,  6.38s/it]
 15%|█▌        | 34710/225000 [76:45:54<385:22:15,  7.29s/it]
 15%|█▌        | 34711/225000 [76:45:58<343:55:11,  6.51s/it]
 15%|█▌        | 34712/225000 [76:46:03<323:11:10,  6.11s/it]
 15%|█▌        | 34713/225000 [76:46:09<312:29:40,  5.91s/it]
 15%|█▌        | 34714/225000 [76:46:20<398:55:17,  7.55s/it]
 15%|█▌        | 34715/225000 [76:46:24<346:24:28,  6.55s/it]
 15%|█▌        | 34716/225000 [76:46:29<310:50:39,  5.88s/it]
 15%|█▌        | 34717/225000 [76:46:39<382:09:49,  7.23s/it]
 15%|█▌        | 34718/225000 [76:46:43<326:35:21,  6.18s/it]
 15%|█▌        | 34719/225000 [76:46:47<297:04:22,  5.62s/it]
 15%|█▌        | 34720/225000 [76:46:59<388:15:45,  7.35s/it]
 15%|█▌        | 34721/225000 [76:47:02<327:09:07,  6.19s/it]
 15%|█▌        | 34722/225000 [76:47:11<369:25:48,  6.99s/it]
 15%|█▌        | 34723/225000 [76:47:16<335:51:29,  6.35s/it]
 15%|█▌        | 34724/225000 [76:47:21<313:52:00,  5.94s/it]
 15%|█▌        | 34725/225000 [76:47:27<314:36:48,  5.95s/it]
 15%|█▌        | 34726/225000 [76:47:38<398:42:01,  7.54s/it]
 15%|█▌        | 34727/225000 [76:47:42<346:47:45,  6.56s/it]
 15%|█▌        | 34728/225000 [76:47:46<308:19:03,  5.83s/it]
 15%|█▌        | 34729/225000 [76:47:57<383:47:29,  7.26s/it]
 15%|█▌        | 34730/225000 [76:48:03<358:48:36,  6.79s/it]
 15%|█▌        | 34731/225000 [76:48:08<330:03:06,  6.24s/it]
 15%|█▌        | 34732/225000 [76:48:13<314:36:00,  5.95s/it]
 15%|█▌        | 34733/225000 [76:48:19<315:31:35,  5.97s/it]
 15%|█▌        | 34734/225000 [76:48:24<296:16:42,  5.61s/it]
 15%|█▌        | 34735/225000 [76:48:29<292:27:02,  5.53s/it]
 15%|█▌        | 34736/225000 [76:48:34<289:06:37,  5.47s/it]
 15%|█▌        | 34737/225000 [76:48:48<414:47:26,  7.85s/it]
 15%|█▌        | 34738/225000 [76:48:53<378:28:17,  7.16s/it]
 15%|█▌        | 34739/225000 [76:48:59<348:42:57,  6.60s/it]
 15%|█▌        | 34740/225000 [76:49:04<328:10:08,  6.21s/it]
 15%|█▌        | 34741/225000 [76:49:09<309:20:07,  5.85s/it]
 15%|█▌        | 34742/225000 [76:49:15<305:31:59,  5.78s/it]
 15%|█▌        | 34743/225000 [76:49:23<351:49:45,  6.66s/it]
 15%|█▌        | 34744/225000 [76:49:34<423:03:46,  8.01s/it]
 15%|█▌        | 34745/225000 [76:49:38<360:41:24,  6.82s/it]
 15%|█▌        | 34746/225000 [76:49:43<330:40:52,  6.26s/it]
 15%|█▌        | 34747/225000 [76:49:49<317:21:10,  6.01s/it]
 15%|█▌        | 34748/225000 [76:49:54<308:08:12,  5.83s/it]
 15%|█▌        | 34749/225000 [76:50:04<364:46:57,  6.90s/it]
 15%|█▌        | 34750/225000 [76:50:14<420:02:13,  7.95s/it]
                                                             
{'loss': '0.7395', 'grad_norm': '0.721', 'learning_rate': '4.839e-05', 'epoch': '0.1544', 'num_input_tokens_seen': 130737160, 'train_runtime': '2.766e+05', 'train_tokens_per_second': '472.6'}

 15%|█▌        | 34750/225000 [76:50:14<420:02:13,  7.95s/it]
 15%|█▌        | 34751/225000 [76:50:19<371:06:04,  7.02s/it]
 15%|█▌        | 34752/225000 [76:50:24<337:06:24,  6.38s/it]
 15%|█▌        | 34753/225000 [76:50:29<323:34:26,  6.12s/it]
 15%|█▌        | 34754/225000 [76:50:34<299:56:55,  5.68s/it]
 15%|█▌        | 34755/225000 [76:50:39<291:00:37,  5.51s/it]
 15%|█▌        | 34756/225000 [76:50:44<288:28:38,  5.46s/it]
 15%|█▌        | 34757/225000 [76:50:50<287:07:16,  5.43s/it]
 15%|█▌        | 34758/225000 [76:50:58<330:58:13,  6.26s/it]
 15%|█▌        | 34759/225000 [76:51:09<405:17:30,  7.67s/it]
 15%|█▌        | 34760/225000 [76:51:12<340:09:21,  6.44s/it]
 15%|█▌        | 34761/225000 [76:51:20<363:28:06,  6.88s/it]
 15%|█▌        | 34762/225000 [76:51:24<315:32:36,  5.97s/it]
 15%|█▌        | 34763/225000 [76:51:34<383:19:38,  7.25s/it]
 15%|█▌        | 34764/225000 [76:51:38<330:40:03,  6.26s/it]
 15%|█▌        | 34765/225000 [76:51:48<376:22:25,  7.12s/it]
 15%|█▌        | 34766/225000 [76:51:52<330:39:00,  6.26s/it]
 15%|█▌        | 34767/225000 [76:51:58<335:34:34,  6.35s/it]
 15%|█▌        | 34768/225000 [76:52:07<366:51:38,  6.94s/it]
 15%|█▌        | 34769/225000 [76:52:15<384:16:40,  7.27s/it]
 15%|█▌        | 34770/225000 [76:52:18<323:55:43,  6.13s/it]
 15%|█▌        | 34771/225000 [76:52:25<337:21:04,  6.38s/it]
 15%|█▌        | 34772/225000 [76:52:31<328:10:44,  6.21s/it]
 15%|█▌        | 34773/225000 [76:52:39<355:41:44,  6.73s/it]
 15%|█▌        | 34774/225000 [76:52:45<341:42:17,  6.47s/it]
 15%|█▌        | 34775/225000 [76:52:51<332:16:21,  6.29s/it]
 15%|█▌        | 34776/225000 [76:52:57<327:10:07,  6.19s/it]
 15%|█▌        | 34777/225000 [76:53:02<319:38:51,  6.05s/it]
 15%|█▌        | 34778/225000 [76:53:07<303:43:21,  5.75s/it]
 15%|█▌        | 34779/225000 [76:53:12<290:00:13,  5.49s/it]
 15%|█▌        | 34780/225000 [76:53:17<280:03:26,  5.30s/it]
 15%|█▌        | 34781/225000 [76:53:29<383:56:51,  7.27s/it]
 15%|█▌        | 34782/225000 [76:53:36<378:24:48,  7.16s/it]
 15%|█▌        | 34783/225000 [76:53:41<344:26:04,  6.52s/it]
 15%|█▌        | 34784/225000 [76:53:46<318:31:19,  6.03s/it]
 15%|█▌        | 34785/225000 [76:53:51<301:33:41,  5.71s/it]
 15%|█▌        | 34786/225000 [76:53:58<328:31:38,  6.22s/it]
 15%|█▌        | 34787/225000 [76:54:10<415:08:46,  7.86s/it]
 15%|█▌        | 34788/225000 [76:54:13<348:01:09,  6.59s/it]
 15%|█▌        | 34789/225000 [76:54:22<373:36:58,  7.07s/it]
 15%|█▌        | 34790/225000 [76:54:27<349:04:15,  6.61s/it]
 15%|█▌        | 34791/225000 [76:54:34<347:47:46,  6.58s/it]
 15%|█▌        | 34792/225000 [76:54:38<315:49:29,  5.98s/it]
 15%|█▌        | 34793/225000 [76:54:43<289:44:36,  5.48s/it]
 15%|█▌        | 34794/225000 [76:54:53<371:10:51,  7.03s/it]
 15%|█▌        | 34795/225000 [76:54:59<344:05:07,  6.51s/it]
 15%|█▌        | 34796/225000 [76:55:04<320:36:34,  6.07s/it]
 15%|█▌        | 34797/225000 [76:55:09<308:23:48,  5.84s/it]
 15%|█▌        | 34798/225000 [76:55:14<290:51:42,  5.51s/it]
 15%|█▌        | 34799/225000 [76:55:19<287:09:06,  5.44s/it]
 15%|█▌        | 34800/225000 [76:55:24<275:12:21,  5.21s/it]
                                                             
{'loss': '0.7351', 'grad_norm': '0.7444', 'learning_rate': '4.838e-05', 'epoch': '0.1547', 'num_input_tokens_seen': 130880696, 'train_runtime': '2.769e+05', 'train_tokens_per_second': '472.6'}

 15%|█▌        | 34800/225000 [76:55:24<275:12:21,  5.21s/it]
 15%|█▌        | 34801/225000 [76:55:35<380:39:00,  7.20s/it]
 15%|█▌        | 34802/225000 [76:55:44<409:04:30,  7.74s/it]
 15%|█▌        | 34803/225000 [76:55:48<348:15:28,  6.59s/it]
 15%|█▌        | 34804/225000 [76:55:57<387:02:29,  7.33s/it]
 15%|█▌        | 34805/225000 [76:56:01<328:56:12,  6.23s/it]
 15%|█▌        | 34806/225000 [76:56:09<352:05:55,  6.66s/it]
 15%|█▌        | 34807/225000 [76:56:16<365:09:58,  6.91s/it]
 15%|█▌        | 34808/225000 [76:56:24<373:35:26,  7.07s/it]
 15%|█▌        | 34809/225000 [76:56:31<371:36:37,  7.03s/it]
 15%|█▌        | 34810/225000 [76:56:37<355:48:19,  6.73s/it]
 15%|█▌        | 34811/225000 [76:56:43<351:06:04,  6.65s/it]
 15%|█▌        | 34812/225000 [76:56:50<350:53:34,  6.64s/it]
 15%|█▌        | 34813/225000 [76:56:58<372:30:15,  7.05s/it]
 15%|█▌        | 34814/225000 [76:57:03<340:14:17,  6.44s/it]
 15%|█▌        | 34815/225000 [76:57:09<342:50:44,  6.49s/it]
 15%|█▌        | 34816/225000 [76:57:14<313:51:47,  5.94s/it]
 15%|█▌        | 34817/225000 [76:57:19<294:35:57,  5.58s/it]
 15%|█▌        | 34818/225000 [76:57:26<323:11:54,  6.12s/it]
 15%|█▌        | 34819/225000 [76:57:34<357:15:23,  6.76s/it]
 15%|█▌        | 34820/225000 [76:57:42<372:29:13,  7.05s/it]
 15%|█▌        | 34821/225000 [76:57:47<339:35:08,  6.43s/it]
 15%|█▌        | 34822/225000 [76:57:52<317:45:29,  6.02s/it]
 15%|█▌        | 34823/225000 [76:57:57<297:48:13,  5.64s/it]
 15%|█▌        | 34824/225000 [76:58:01<276:22:49,  5.23s/it]
 15%|█▌        | 34825/225000 [76:58:12<370:31:54,  7.01s/it]
 15%|█▌        | 34826/225000 [76:58:19<362:55:53,  6.87s/it]
 15%|█▌        | 34827/225000 [76:58:27<388:11:57,  7.35s/it]
 15%|█▌        | 34828/225000 [76:58:34<371:47:16,  7.04s/it]
 15%|█▌        | 34829/225000 [76:58:39<348:10:21,  6.59s/it]
 15%|█▌        | 34830/225000 [76:58:44<314:50:17,  5.96s/it]
 15%|█▌        | 34831/225000 [76:58:48<296:20:40,  5.61s/it]
 15%|█▌        | 34832/225000 [76:58:59<376:02:58,  7.12s/it]
 15%|█▌        | 34833/225000 [76:59:04<343:32:07,  6.50s/it]
 15%|█▌        | 34834/225000 [76:59:12<365:27:40,  6.92s/it]
 15%|█▌        | 34835/225000 [76:59:20<383:08:34,  7.25s/it]
 15%|█▌        | 34836/225000 [76:59:24<324:51:06,  6.15s/it]
 15%|█▌        | 34837/225000 [76:59:29<318:35:03,  6.03s/it]
 15%|█▌        | 34838/225000 [76:59:37<336:14:44,  6.37s/it]
 15%|█▌        | 34839/225000 [76:59:42<313:45:16,  5.94s/it]
 15%|█▌        | 34840/225000 [76:59:47<300:09:55,  5.68s/it]
 15%|█▌        | 34841/225000 [76:59:58<383:38:32,  7.26s/it]
 15%|█▌        | 34842/225000 [77:00:02<341:22:51,  6.46s/it]
 15%|█▌        | 34843/225000 [77:00:09<344:30:21,  6.52s/it]
 15%|█▌        | 34844/225000 [77:00:13<313:22:18,  5.93s/it]
 15%|█▌        | 34845/225000 [77:00:22<363:47:02,  6.89s/it]
 15%|█▌        | 34846/225000 [77:00:27<330:07:36,  6.25s/it]
 15%|█▌        | 34847/225000 [77:00:32<302:37:44,  5.73s/it]
 15%|█▌        | 34848/225000 [77:00:37<287:25:12,  5.44s/it]
 15%|█▌        | 34849/225000 [77:00:41<273:05:29,  5.17s/it]
 15%|█▌        | 34850/225000 [77:00:54<400:23:35,  7.58s/it]
                                                             
{'loss': '0.7443', 'grad_norm': '0.8325', 'learning_rate': '4.837e-05', 'epoch': '0.1549', 'num_input_tokens_seen': 131025200, 'train_runtime': '2.773e+05', 'train_tokens_per_second': '472.6'}

 15%|█▌        | 34850/225000 [77:00:54<400:23:35,  7.58s/it]
 15%|█▌        | 34851/225000 [77:00:59<350:03:51,  6.63s/it]
 15%|█▌        | 34852/225000 [77:01:06<362:29:40,  6.86s/it]
 15%|█▌        | 34853/225000 [77:01:11<334:24:34,  6.33s/it]
 15%|█▌        | 34854/225000 [77:01:18<342:15:43,  6.48s/it]
 15%|█▌        | 34855/225000 [77:01:25<344:24:01,  6.52s/it]
 15%|█▌        | 34856/225000 [77:01:29<316:37:01,  5.99s/it]
 15%|█▌        | 34857/225000 [77:01:34<297:02:14,  5.62s/it]
 15%|█▌        | 34858/225000 [77:01:46<389:59:56,  7.38s/it]
 15%|█▌        | 34859/225000 [77:01:50<341:45:38,  6.47s/it]
 15%|█▌        | 34860/225000 [77:01:58<366:01:41,  6.93s/it]
 15%|█▌        | 34861/225000 [77:02:02<314:05:41,  5.95s/it]
 15%|█▌        | 34862/225000 [77:02:12<379:35:27,  7.19s/it]
 15%|█▌        | 34863/225000 [77:02:15<320:18:16,  6.06s/it]
 15%|█▌        | 34864/225000 [77:02:24<363:11:04,  6.88s/it]
 15%|█▌        | 34865/225000 [77:02:29<331:23:02,  6.27s/it]
 15%|█▌        | 34866/225000 [77:02:34<310:06:07,  5.87s/it]
 15%|█▌        | 34867/225000 [77:02:38<291:54:26,  5.53s/it]
 15%|█▌        | 34868/225000 [77:02:43<282:03:48,  5.34s/it]
 15%|█▌        | 34869/225000 [77:02:49<288:25:33,  5.46s/it]
 15%|█▌        | 34870/225000 [77:03:01<398:00:02,  7.54s/it]
 15%|█▌        | 34871/225000 [77:03:05<340:16:42,  6.44s/it]
 15%|█▌        | 34872/225000 [77:03:14<377:06:13,  7.14s/it]
 15%|█▌        | 34873/225000 [77:03:18<329:00:39,  6.23s/it]
 15%|█▌        | 34874/225000 [77:03:25<343:18:33,  6.50s/it]
 16%|█▌        | 34875/225000 [77:03:31<325:56:46,  6.17s/it]
 16%|█▌        | 34876/225000 [77:03:36<308:02:28,  5.83s/it]
 16%|█▌        | 34877/225000 [77:03:41<292:19:06,  5.54s/it]
 16%|█▌        | 34878/225000 [77:03:46<291:23:32,  5.52s/it]
 16%|█▌        | 34879/225000 [77:03:52<299:17:10,  5.67s/it]
 16%|█▌        | 34880/225000 [77:04:02<367:02:34,  6.95s/it]
 16%|█▌        | 34881/225000 [77:04:06<321:05:22,  6.08s/it]
 16%|█▌        | 34882/225000 [77:04:12<313:56:48,  5.94s/it]
 16%|█▌        | 34883/225000 [77:04:25<427:49:31,  8.10s/it]
 16%|█▌        | 34884/225000 [77:04:31<402:34:03,  7.62s/it]
 16%|█▌        | 34885/225000 [77:04:35<335:54:37,  6.36s/it]
 16%|█▌        | 34886/225000 [77:04:42<353:17:40,  6.69s/it]
 16%|█▌        | 34887/225000 [77:04:47<321:52:45,  6.10s/it]
 16%|█▌        | 34888/225000 [77:04:52<300:16:08,  5.69s/it]
 16%|█▌        | 34889/225000 [77:05:04<397:50:51,  7.53s/it]
 16%|█▌        | 34890/225000 [77:05:08<342:00:11,  6.48s/it]
 16%|█▌        | 34891/225000 [77:05:16<369:39:17,  7.00s/it]
 16%|█▌        | 34892/225000 [77:05:20<327:53:31,  6.21s/it]
 16%|█▌        | 34893/225000 [77:05:25<301:52:45,  5.72s/it]
 16%|█▌        | 34894/225000 [77:05:31<310:35:53,  5.88s/it]
 16%|█▌        | 34895/225000 [77:05:41<376:56:42,  7.14s/it]
 16%|█▌        | 34896/225000 [77:05:45<327:43:47,  6.21s/it]
 16%|█▌        | 34897/225000 [77:05:54<368:56:55,  6.99s/it]
 16%|█▌        | 34898/225000 [77:05:59<335:05:13,  6.35s/it]
 16%|█▌        | 34899/225000 [77:06:08<374:10:25,  7.09s/it]
 16%|█▌        | 34900/225000 [77:06:15<375:45:27,  7.12s/it]
                                                             
{'loss': '0.7428', 'grad_norm': '0.662', 'learning_rate': '4.837e-05', 'epoch': '0.1551', 'num_input_tokens_seen': 131169328, 'train_runtime': '2.776e+05', 'train_tokens_per_second': '472.6'}

 16%|█▌        | 34900/225000 [77:06:15<375:45:27,  7.12s/it]
 16%|█▌        | 34901/225000 [77:06:20<348:27:58,  6.60s/it]
 16%|█▌        | 34902/225000 [77:06:26<328:48:49,  6.23s/it]
 16%|█▌        | 34903/225000 [77:06:31<310:55:07,  5.89s/it]
 16%|█▌        | 34904/225000 [77:06:36<298:55:04,  5.66s/it]
 16%|█▌        | 34905/225000 [77:06:41<294:12:53,  5.57s/it]
 16%|█▌        | 34906/225000 [77:06:50<343:29:47,  6.51s/it]
 16%|█▌        | 34907/225000 [77:07:01<415:04:05,  7.86s/it]
 16%|█▌        | 34908/225000 [77:07:05<363:08:48,  6.88s/it]
 16%|█▌        | 34909/225000 [77:07:10<328:55:05,  6.23s/it]
 16%|█▌        | 34910/225000 [77:07:16<321:40:53,  6.09s/it]
 16%|█▌        | 34911/225000 [77:07:27<406:38:04,  7.70s/it]
 16%|█▌        | 34912/225000 [77:07:32<353:51:59,  6.70s/it]
 16%|█▌        | 34913/225000 [77:07:36<317:17:19,  6.01s/it]
 16%|█▌        | 34914/225000 [77:07:41<305:15:38,  5.78s/it]
 16%|█▌        | 34915/225000 [77:07:53<393:04:11,  7.44s/it]
 16%|█▌        | 34916/225000 [77:07:59<371:27:11,  7.03s/it]
 16%|█▌        | 34917/225000 [77:08:06<374:42:32,  7.10s/it]
 16%|█▌        | 34918/225000 [77:08:12<349:27:49,  6.62s/it]
 16%|█▌        | 34919/225000 [77:08:16<315:43:13,  5.98s/it]
 16%|█▌        | 34920/225000 [77:08:25<361:46:58,  6.85s/it]
 16%|█▌        | 34921/225000 [77:08:29<324:42:07,  6.15s/it]
 16%|█▌        | 34922/225000 [77:08:34<302:28:47,  5.73s/it]
 16%|█▌        | 34923/225000 [77:08:39<290:21:48,  5.50s/it]
 16%|█▌        | 34924/225000 [77:08:51<395:58:49,  7.50s/it]
 16%|█▌        | 34925/225000 [77:08:55<337:21:50,  6.39s/it]
 16%|█▌        | 34926/225000 [77:09:01<323:40:33,  6.13s/it]
 16%|█▌        | 34927/225000 [77:09:06<308:23:04,  5.84s/it]
 16%|█▌        | 34928/225000 [77:09:12<320:05:13,  6.06s/it]
 16%|█▌        | 34929/225000 [77:09:24<405:15:06,  7.68s/it]
 16%|█▌        | 34930/225000 [77:09:28<352:51:01,  6.68s/it]
 16%|█▌        | 34931/225000 [77:09:35<353:31:35,  6.70s/it]
 16%|█▌        | 34932/225000 [77:09:41<338:15:45,  6.41s/it]
 16%|█▌        | 34933/225000 [77:09:51<398:12:14,  7.54s/it]
 16%|█▌        | 34934/225000 [77:09:54<332:01:21,  6.29s/it]
 16%|█▌        | 34935/225000 [77:10:02<358:43:16,  6.79s/it]
 16%|█▌        | 34936/225000 [77:10:09<360:34:30,  6.83s/it]
 16%|█▌        | 34937/225000 [77:10:13<314:11:53,  5.95s/it]
 16%|█▌        | 34938/225000 [77:10:25<412:39:15,  7.82s/it]
 16%|█▌        | 34939/225000 [77:10:29<345:00:01,  6.53s/it]
 16%|█▌        | 34940/225000 [77:10:33<308:45:24,  5.85s/it]
 16%|█▌        | 34941/225000 [77:10:38<291:27:27,  5.52s/it]
 16%|█▌        | 34942/225000 [77:10:43<288:46:57,  5.47s/it]
 16%|█▌        | 34943/225000 [77:10:48<278:40:14,  5.28s/it]
 16%|█▌        | 34944/225000 [77:10:56<327:34:27,  6.20s/it]
 16%|█▌        | 34945/225000 [77:11:08<417:21:31,  7.91s/it]
 16%|█▌        | 34946/225000 [77:11:12<352:58:02,  6.69s/it]
 16%|█▌        | 34947/225000 [77:11:20<379:01:49,  7.18s/it]
 16%|█▌        | 34948/225000 [77:11:25<346:16:16,  6.56s/it]
 16%|█▌        | 34949/225000 [77:11:32<350:25:56,  6.64s/it]
 16%|█▌        | 34950/225000 [77:11:37<325:50:48,  6.17s/it]
                                                             
{'loss': '0.7647', 'grad_norm': '0.7226', 'learning_rate': '4.836e-05', 'epoch': '0.1553', 'num_input_tokens_seen': 131313696, 'train_runtime': '2.779e+05', 'train_tokens_per_second': '472.5'}

 16%|█▌        | 34950/225000 [77:11:37<325:50:48,  6.17s/it]
 16%|█▌        | 34951/225000 [77:11:44<326:38:36,  6.19s/it]
 16%|█▌        | 34952/225000 [77:11:54<397:29:16,  7.53s/it]
 16%|█▌        | 34953/225000 [77:11:58<338:43:38,  6.42s/it]
 16%|█▌        | 34954/225000 [77:12:05<345:26:41,  6.54s/it]
 16%|█▌        | 34955/225000 [77:12:13<368:47:49,  6.99s/it]
 16%|█▌        | 34956/225000 [77:12:20<367:45:09,  6.97s/it]
 16%|█▌        | 34957/225000 [77:12:25<335:14:46,  6.35s/it]
 16%|█▌        | 34958/225000 [77:12:30<313:01:08,  5.93s/it]
 16%|█▌        | 34959/225000 [77:12:35<302:01:29,  5.72s/it]
 16%|█▌        | 34960/225000 [77:12:40<289:11:26,  5.48s/it]
 16%|█▌        | 34961/225000 [77:12:52<388:25:01,  7.36s/it]
 16%|█▌        | 34962/225000 [77:12:55<333:18:59,  6.31s/it]
 16%|█▌        | 34963/225000 [77:12:59<295:39:57,  5.60s/it]
 16%|█▌        | 34964/225000 [77:13:12<410:58:27,  7.79s/it]
 16%|█▌        | 34965/225000 [77:13:17<367:55:23,  6.97s/it]
 16%|█▌        | 34966/225000 [77:13:25<377:59:49,  7.16s/it]
 16%|█▌        | 34967/225000 [77:13:29<333:47:51,  6.32s/it]
 16%|█▌        | 34968/225000 [77:13:34<304:04:38,  5.76s/it]
 16%|█▌        | 34969/225000 [77:13:39<292:33:45,  5.54s/it]
 16%|█▌        | 34970/225000 [77:13:44<285:11:16,  5.40s/it]
 16%|█▌        | 34971/225000 [77:13:56<398:04:56,  7.54s/it]
 16%|█▌        | 34972/225000 [77:14:01<347:40:44,  6.59s/it]
 16%|█▌        | 34973/225000 [77:14:05<308:05:10,  5.84s/it]
 16%|█▌        | 34974/225000 [77:14:17<406:32:19,  7.70s/it]
 16%|█▌        | 34975/225000 [77:14:20<339:19:56,  6.43s/it]
 16%|█▌        | 34976/225000 [77:14:29<372:46:06,  7.06s/it]
 16%|█▌        | 34977/225000 [77:14:33<327:18:49,  6.20s/it]
 16%|█▌        | 34978/225000 [77:14:37<296:03:48,  5.61s/it]
 16%|█▌        | 34979/225000 [77:14:50<410:26:50,  7.78s/it]
 16%|█▌        | 34980/225000 [77:14:53<340:38:56,  6.45s/it]
 16%|█▌        | 34981/225000 [77:14:59<326:21:55,  6.18s/it]
 16%|█▌        | 34982/225000 [77:15:06<345:25:30,  6.54s/it]
 16%|█▌        | 34983/225000 [77:15:12<335:41:35,  6.36s/it]
 16%|█▌        | 34984/225000 [77:15:16<293:57:31,  5.57s/it]
 16%|█▌        | 34985/225000 [77:15:22<300:06:47,  5.69s/it]
 16%|█▌        | 34986/225000 [77:15:30<343:16:23,  6.50s/it]
 16%|█▌        | 34987/225000 [77:15:36<336:11:49,  6.37s/it]
 16%|█▌        | 34988/225000 [77:15:42<321:58:43,  6.10s/it]
 16%|█▌        | 34989/225000 [77:15:49<340:08:38,  6.44s/it]
 16%|█▌        | 34990/225000 [77:15:55<332:08:58,  6.29s/it]
 16%|█▌        | 34991/225000 [77:16:00<305:53:46,  5.80s/it]
 16%|█▌        | 34992/225000 [77:16:08<339:44:33,  6.44s/it]
 16%|█▌        | 34993/225000 [77:16:12<299:07:23,  5.67s/it]
 16%|█▌        | 34994/225000 [77:16:20<335:09:24,  6.35s/it]
 16%|█▌        | 34995/225000 [77:16:27<349:54:50,  6.63s/it]
 16%|█▌        | 34996/225000 [77:16:31<312:24:29,  5.92s/it]
 16%|█▌        | 34997/225000 [77:16:37<312:23:34,  5.92s/it]
 16%|█▌        | 34998/225000 [77:16:46<354:32:32,  6.72s/it]
 16%|█▌        | 34999/225000 [77:16:49<309:22:33,  5.86s/it]
 16%|█▌        | 35000/225000 [77:17:00<383:40:01,  7.27s/it]
                                                             
{'loss': '0.7417', 'grad_norm': '0.8447', 'learning_rate': '4.835e-05', 'epoch': '0.1556', 'num_input_tokens_seen': 131457904, 'train_runtime': '2.782e+05', 'train_tokens_per_second': '472.5'}

 16%|█▌        | 35000/225000 [77:17:00<383:40:01,  7.27s/it]
 16%|█▌        | 35001/225000 [77:17:04<337:35:28,  6.40s/it]
 16%|█▌        | 35002/225000 [77:17:09<307:23:22,  5.82s/it]
 16%|█▌        | 35003/225000 [77:17:17<344:56:01,  6.54s/it]
 16%|█▌        | 35004/225000 [77:17:20<295:40:25,  5.60s/it]
 16%|█▌        | 35005/225000 [77:17:30<359:31:48,  6.81s/it]
 16%|█▌        | 35006/225000 [77:17:34<321:02:39,  6.08s/it]
 16%|█▌        | 35007/225000 [77:17:39<294:14:19,  5.58s/it]
 16%|█▌        | 35008/225000 [77:17:44<285:37:11,  5.41s/it]
 16%|█▌        | 35009/225000 [77:17:49<276:26:42,  5.24s/it]
 16%|█▌        | 35010/225000 [77:18:02<399:25:49,  7.57s/it]
 16%|█▌        | 35011/225000 [77:18:06<343:46:25,  6.51s/it]
 16%|█▌        | 35012/225000 [77:18:10<300:55:44,  5.70s/it]
 16%|█▌        | 35013/225000 [77:18:18<343:06:35,  6.50s/it]
 16%|█▌        | 35014/225000 [77:18:26<372:14:19,  7.05s/it]
 16%|█▌        | 35015/225000 [77:18:30<317:36:17,  6.02s/it]
 16%|█▌        | 35016/225000 [77:18:36<317:47:05,  6.02s/it]
 16%|█▌        | 35017/225000 [77:18:41<306:41:46,  5.81s/it]
 16%|█▌        | 35018/225000 [77:18:47<298:20:08,  5.65s/it]
 16%|█▌        | 35019/225000 [77:18:52<298:49:33,  5.66s/it]
 16%|█▌        | 35020/225000 [77:18:58<293:42:46,  5.57s/it]
 16%|█▌        | 35021/225000 [77:19:11<421:10:14,  7.98s/it]
 16%|█▌        | 35022/225000 [77:19:18<409:08:50,  7.75s/it]
 16%|█▌        | 35023/225000 [77:19:26<403:10:56,  7.64s/it]
 16%|█▌        | 35024/225000 [77:19:32<380:16:39,  7.21s/it]
 16%|█▌        | 35025/225000 [77:19:37<342:50:48,  6.50s/it]
 16%|█▌        | 35026/225000 [77:19:41<311:16:22,  5.90s/it]
 16%|█▌        | 35027/225000 [77:19:46<291:00:52,  5.51s/it]
 16%|█▌        | 35028/225000 [77:19:58<397:09:46,  7.53s/it]
 16%|█▌        | 35029/225000 [77:20:04<376:16:13,  7.13s/it]
 16%|█▌        | 35030/225000 [77:20:09<330:00:46,  6.25s/it]
 16%|█▌        | 35031/225000 [77:20:19<389:51:30,  7.39s/it]
 16%|█▌        | 35032/225000 [77:20:22<332:06:18,  6.29s/it]
 16%|█▌        | 35033/225000 [77:20:32<378:02:39,  7.16s/it]
 16%|█▌        | 35034/225000 [77:20:35<321:31:19,  6.09s/it]
 16%|█▌        | 35035/225000 [77:20:44<367:23:32,  6.96s/it]
 16%|█▌        | 35036/225000 [77:20:48<319:36:01,  6.06s/it]
 16%|█▌        | 35037/225000 [77:20:52<293:46:52,  5.57s/it]
 16%|█▌        | 35038/225000 [77:20:57<279:57:24,  5.31s/it]
 16%|█▌        | 35039/225000 [77:21:03<282:04:21,  5.35s/it]
 16%|█▌        | 35040/225000 [77:21:08<289:06:50,  5.48s/it]
 16%|█▌        | 35041/225000 [77:21:16<317:28:09,  6.02s/it]
 16%|█▌        | 35042/225000 [77:21:29<433:13:00,  8.21s/it]
 16%|█▌        | 35043/225000 [77:21:34<388:22:49,  7.36s/it]
 16%|█▌        | 35044/225000 [77:21:38<336:41:22,  6.38s/it]
 16%|█▌        | 35045/225000 [77:21:45<337:37:53,  6.40s/it]
 16%|█▌        | 35046/225000 [77:21:54<377:20:11,  7.15s/it]
 16%|█▌        | 35047/225000 [77:21:58<324:06:05,  6.14s/it]
 16%|█▌        | 35048/225000 [77:22:07<380:21:02,  7.21s/it]
 16%|█▌        | 35049/225000 [77:22:11<321:11:10,  6.09s/it]
 16%|█▌        | 35050/225000 [77:22:15<289:23:46,  5.48s/it]
                                                             
{'loss': '0.7206', 'grad_norm': '0.6234', 'learning_rate': '4.835e-05', 'epoch': '0.1558', 'num_input_tokens_seen': 131601648, 'train_runtime': '2.785e+05', 'train_tokens_per_second': '472.5'}

 16%|█▌        | 35050/225000 [77:22:15<289:23:46,  5.48s/it]
 16%|█▌        | 35051/225000 [77:22:23<324:21:30,  6.15s/it]
 16%|█▌        | 35052/225000 [77:22:31<363:35:50,  6.89s/it]
 16%|█▌        | 35053/225000 [77:22:35<322:12:42,  6.11s/it]
 16%|█▌        | 35054/225000 [77:22:46<397:18:37,  7.53s/it]
 16%|█▌        | 35055/225000 [77:22:50<338:18:07,  6.41s/it]
 16%|█▌        | 35056/225000 [77:23:00<393:45:45,  7.46s/it]
 16%|█▌        | 35057/225000 [77:23:04<337:09:53,  6.39s/it]
 16%|█▌        | 35058/225000 [77:23:08<302:51:08,  5.74s/it]
 16%|█▌        | 35059/225000 [77:23:19<381:03:39,  7.22s/it]
 16%|█▌        | 35060/225000 [77:23:23<330:31:02,  6.26s/it]
 16%|█▌        | 35061/225000 [77:23:31<360:34:39,  6.83s/it]
 16%|█▌        | 35062/225000 [77:23:35<319:47:46,  6.06s/it]
 16%|█▌        | 35063/225000 [77:23:43<350:29:27,  6.64s/it]
 16%|█▌        | 35064/225000 [77:23:48<325:34:48,  6.17s/it]
 16%|█▌        | 35065/225000 [77:23:54<311:40:35,  5.91s/it]
 16%|█▌        | 35066/225000 [77:23:59<303:14:25,  5.75s/it]
 16%|█▌        | 35067/225000 [77:24:04<292:25:30,  5.54s/it]
 16%|█▌        | 35068/225000 [77:24:10<292:04:33,  5.54s/it]
 16%|█▌        | 35069/225000 [77:24:15<296:43:00,  5.62s/it]
 16%|█▌        | 35070/225000 [77:24:29<416:17:01,  7.89s/it]
 16%|█▌        | 35071/225000 [77:24:36<410:14:35,  7.78s/it]
 16%|█▌        | 35072/225000 [77:24:41<362:10:49,  6.86s/it]
 16%|█▌        | 35073/225000 [77:24:50<391:26:01,  7.42s/it]
 16%|█▌        | 35074/225000 [77:24:54<342:18:53,  6.49s/it]
 16%|█▌        | 35075/225000 [77:24:58<311:53:38,  5.91s/it]
 16%|█▌        | 35076/225000 [77:25:03<297:29:12,  5.64s/it]
 16%|█▌        | 35077/225000 [77:25:10<309:50:20,  5.87s/it]
 16%|█▌        | 35078/225000 [77:25:22<410:10:25,  7.77s/it]
 16%|█▌        | 35079/225000 [77:25:26<348:06:03,  6.60s/it]
 16%|█▌        | 35080/225000 [77:25:33<361:35:20,  6.85s/it]
 16%|█▌        | 35081/225000 [77:25:37<317:27:03,  6.02s/it]
 16%|█▌        | 35082/225000 [77:25:49<411:13:59,  7.80s/it]
 16%|█▌        | 35083/225000 [77:25:54<358:41:44,  6.80s/it]
 16%|█▌        | 35084/225000 [77:25:58<319:34:38,  6.06s/it]
 16%|█▌        | 35085/225000 [77:26:03<295:18:48,  5.60s/it]
 16%|█▌        | 35086/225000 [77:26:11<330:07:20,  6.26s/it]
 16%|█▌        | 35087/225000 [77:26:21<392:13:34,  7.44s/it]
 16%|█▌        | 35088/225000 [77:26:26<363:19:54,  6.89s/it]
 16%|█▌        | 35089/225000 [77:26:33<356:09:36,  6.75s/it]
 16%|█▌        | 35090/225000 [77:26:37<316:10:55,  5.99s/it]
 16%|█▌        | 35091/225000 [77:26:42<294:15:09,  5.58s/it]
 16%|█▌        | 35092/225000 [77:26:52<371:41:10,  7.05s/it]
 16%|█▌        | 35093/225000 [77:26:57<333:08:15,  6.32s/it]
 16%|█▌        | 35094/225000 [77:27:04<343:07:08,  6.50s/it]
 16%|█▌        | 35095/225000 [77:27:11<356:09:51,  6.75s/it]
 16%|█▌        | 35096/225000 [77:27:16<324:41:23,  6.16s/it]
 16%|█▌        | 35097/225000 [77:27:21<317:09:20,  6.01s/it]
 16%|█▌        | 35098/225000 [77:27:27<304:25:06,  5.77s/it]
 16%|█▌        | 35099/225000 [77:27:31<289:57:08,  5.50s/it]
 16%|█▌        | 35100/225000 [77:27:37<295:28:00,  5.60s/it]
                                                             
{'loss': '0.745', 'grad_norm': '0.7806', 'learning_rate': '4.834e-05', 'epoch': '0.156', 'num_input_tokens_seen': 131746264, 'train_runtime': '2.789e+05', 'train_tokens_per_second': '472.4'}

 16%|█▌        | 35100/225000 [77:27:37<295:28:00,  5.60s/it]
 16%|█▌        | 35101/225000 [77:27:42<286:43:20,  5.44s/it]
 16%|█▌        | 35102/225000 [77:27:52<350:16:36,  6.64s/it]
 16%|█▌        | 35103/225000 [77:28:02<401:32:09,  7.61s/it]
 16%|█▌        | 35104/225000 [77:28:06<342:22:20,  6.49s/it]
 16%|█▌        | 35105/225000 [77:28:15<394:31:29,  7.48s/it]
 16%|█▌        | 35106/225000 [77:28:19<337:09:59,  6.39s/it]
 16%|█▌        | 35107/225000 [77:28:24<309:38:58,  5.87s/it]
 16%|█▌        | 35108/225000 [77:28:28<288:23:03,  5.47s/it]
 16%|█▌        | 35109/225000 [77:28:39<366:44:02,  6.95s/it]
 16%|█▌        | 35110/225000 [77:28:43<325:40:19,  6.17s/it]
 16%|█▌        | 35111/225000 [77:28:50<330:06:29,  6.26s/it]
 16%|█▌        | 35112/225000 [77:29:00<393:22:56,  7.46s/it]
 16%|█▌        | 35113/225000 [77:29:07<382:07:21,  7.24s/it]
 16%|█▌        | 35114/225000 [77:29:11<342:16:29,  6.49s/it]
 16%|█▌        | 35115/225000 [77:29:16<317:47:13,  6.02s/it]
 16%|█▌        | 35116/225000 [77:29:21<294:00:10,  5.57s/it]
 16%|█▌        | 35117/225000 [77:29:26<285:21:27,  5.41s/it]
 16%|█▌        | 35118/225000 [77:29:34<334:17:01,  6.34s/it]
 16%|█▌        | 35119/225000 [77:29:45<396:59:16,  7.53s/it]
 16%|█▌        | 35120/225000 [77:29:49<354:11:08,  6.72s/it]
 16%|█▌        | 35121/225000 [77:29:57<362:57:18,  6.88s/it]
 16%|█▌        | 35122/225000 [77:30:01<319:36:31,  6.06s/it]
 16%|█▌        | 35123/225000 [77:30:10<364:20:18,  6.91s/it]
 16%|█▌        | 35124/225000 [77:30:15<341:45:56,  6.48s/it]
 16%|█▌        | 35125/225000 [77:30:20<308:15:50,  5.84s/it]
 16%|█▌        | 35126/225000 [77:30:28<352:57:38,  6.69s/it]
 16%|█▌        | 35127/225000 [77:30:34<333:36:25,  6.33s/it]
 16%|█▌        | 35128/225000 [77:30:43<375:20:55,  7.12s/it]
 16%|█▌        | 35129/225000 [77:30:48<342:30:47,  6.49s/it]
 16%|█▌        | 35130/225000 [77:30:52<312:48:14,  5.93s/it]
 16%|█▌        | 35131/225000 [77:30:57<295:01:44,  5.59s/it]
 16%|█▌        | 35132/225000 [77:31:08<382:30:11,  7.25s/it]
 16%|█▌        | 35133/225000 [77:31:14<353:51:49,  6.71s/it]
 16%|█▌        | 35134/225000 [77:31:19<335:34:10,  6.36s/it]
 16%|█▌        | 35135/225000 [77:31:28<376:29:34,  7.14s/it]
 16%|█▌        | 35136/225000 [77:31:32<320:57:22,  6.09s/it]
 16%|█▌        | 35137/225000 [77:31:40<353:03:30,  6.69s/it]
 16%|█▌        | 35138/225000 [77:31:48<370:12:51,  7.02s/it]
 16%|█▌        | 35139/225000 [77:31:51<313:00:22,  5.93s/it]
 16%|█▌        | 35140/225000 [77:32:00<358:18:36,  6.79s/it]
 16%|█▌        | 35141/225000 [77:32:04<307:32:59,  5.83s/it]
 16%|█▌        | 35142/225000 [77:32:13<360:30:50,  6.84s/it]
 16%|█▌        | 35143/225000 [77:32:17<312:46:44,  5.93s/it]
 16%|█▌        | 35144/225000 [77:32:22<302:44:58,  5.74s/it]
 16%|█▌        | 35145/225000 [77:32:32<378:16:37,  7.17s/it]
 16%|█▌        | 35146/225000 [77:32:39<364:03:50,  6.90s/it]
 16%|█▌        | 35147/225000 [77:32:42<315:22:16,  5.98s/it]
 16%|█▌        | 35148/225000 [77:32:46<279:27:52,  5.30s/it]
 16%|█▌        | 35149/225000 [77:32:54<316:41:01,  6.01s/it]
 16%|█▌        | 35150/225000 [77:33:01<340:33:18,  6.46s/it]
                                                             
{'loss': '0.7384', 'grad_norm': '0.6566', 'learning_rate': '4.833e-05', 'epoch': '0.1562', 'num_input_tokens_seen': 131890144, 'train_runtime': '2.792e+05', 'train_tokens_per_second': '472.4'}

 16%|█▌        | 35150/225000 [77:33:01<340:33:18,  6.46s/it]
 16%|█▌        | 35151/225000 [77:33:07<335:56:47,  6.37s/it]
 16%|█▌        | 35152/225000 [77:33:12<301:29:31,  5.72s/it]
 16%|█▌        | 35153/225000 [77:33:19<328:11:34,  6.22s/it]
 16%|█▌        | 35154/225000 [77:33:29<378:49:53,  7.18s/it]
 16%|█▌        | 35155/225000 [77:33:37<397:02:52,  7.53s/it]
 16%|█▌        | 35156/225000 [77:33:42<357:51:58,  6.79s/it]
 16%|█▌        | 35157/225000 [77:33:47<323:22:24,  6.13s/it]
 16%|█▌        | 35158/225000 [77:33:51<291:05:33,  5.52s/it]
 16%|█▌        | 35159/225000 [77:34:00<348:36:11,  6.61s/it]
 16%|█▌        | 35160/225000 [77:34:08<380:39:00,  7.22s/it]
 16%|█▌        | 35161/225000 [77:34:13<331:45:52,  6.29s/it]
 16%|█▌        | 35162/225000 [77:34:19<332:20:03,  6.30s/it]
 16%|█▌        | 35163/225000 [77:34:27<357:55:57,  6.79s/it]
 16%|█▌        | 35164/225000 [77:34:32<326:07:50,  6.18s/it]
 16%|█▌        | 35165/225000 [77:34:39<348:17:55,  6.61s/it]
 16%|█▌        | 35166/225000 [77:34:47<363:23:01,  6.89s/it]
 16%|█▌        | 35167/225000 [77:34:51<321:50:27,  6.10s/it]
 16%|█▌        | 35168/225000 [77:35:00<371:17:52,  7.04s/it]
 16%|█▌        | 35169/225000 [77:35:04<323:22:31,  6.13s/it]
 16%|█▌        | 35170/225000 [77:35:14<381:37:42,  7.24s/it]
 16%|█▌        | 35171/225000 [77:35:19<351:21:07,  6.66s/it]
 16%|█▌        | 35172/225000 [77:35:24<326:54:35,  6.20s/it]
 16%|█▌        | 35173/225000 [77:35:29<300:51:11,  5.71s/it]
 16%|█▌        | 35174/225000 [77:35:34<291:15:11,  5.52s/it]
 16%|█▌        | 35175/225000 [77:35:44<353:07:37,  6.70s/it]
 16%|█▌        | 35176/225000 [77:35:52<381:49:22,  7.24s/it]
 16%|█▌        | 35177/225000 [77:35:59<375:23:38,  7.12s/it]
 16%|█▌        | 35178/225000 [77:36:05<353:35:28,  6.71s/it]
 16%|█▌        | 35179/225000 [77:36:11<347:16:26,  6.59s/it]
 16%|█▌        | 35180/225000 [77:36:18<356:08:55,  6.75s/it]
 16%|█▌        | 35181/225000 [77:36:22<311:10:48,  5.90s/it]
 16%|█▌        | 35182/225000 [77:36:31<359:33:06,  6.82s/it]
 16%|█▌        | 35183/225000 [77:36:34<301:43:57,  5.72s/it]
 16%|█▌        | 35184/225000 [77:36:42<339:42:18,  6.44s/it]
 16%|█▌        | 35185/225000 [77:36:46<304:42:46,  5.78s/it]
 16%|█▌        | 35186/225000 [77:36:51<278:35:36,  5.28s/it]
 16%|█▌        | 35187/225000 [77:37:02<367:59:50,  6.98s/it]
 16%|█▌        | 35188/225000 [77:37:07<346:50:45,  6.58s/it]
 16%|█▌        | 35189/225000 [77:37:12<313:57:06,  5.95s/it]
 16%|█▌        | 35190/225000 [77:37:16<288:31:52,  5.47s/it]
 16%|█▌        | 35191/225000 [77:37:24<328:00:40,  6.22s/it]
 16%|█▌        | 35192/225000 [77:37:34<391:29:39,  7.43s/it]
 16%|█▌        | 35193/225000 [77:37:38<335:04:10,  6.36s/it]
 16%|█▌        | 35194/225000 [77:37:43<317:10:30,  6.02s/it]
 16%|█▌        | 35195/225000 [77:37:49<304:47:00,  5.78s/it]
 16%|█▌        | 35196/225000 [77:37:54<298:00:04,  5.65s/it]
 16%|█▌        | 35197/225000 [77:38:00<302:03:57,  5.73s/it]
 16%|█▌        | 35198/225000 [77:38:14<430:30:36,  8.17s/it]
 16%|█▌        | 35199/225000 [77:38:18<371:08:05,  7.04s/it]
 16%|█▌        | 35200/225000 [77:38:26<380:44:41,  7.22s/it]
                                                             
{'loss': '0.7384', 'grad_norm': '0.7288', 'learning_rate': '4.833e-05', 'epoch': '0.1564', 'num_input_tokens_seen': 132034688, 'train_runtime': '2.795e+05', 'train_tokens_per_second': '472.4'}

 16%|█▌        | 35200/225000 [77:38:26<380:44:41,  7.22s/it]
 16%|█▌        | 35201/225000 [77:38:30<340:04:18,  6.45s/it]
 16%|█▌        | 35202/225000 [77:38:38<363:57:02,  6.90s/it]
 16%|█▌        | 35203/225000 [77:38:43<334:07:18,  6.34s/it]
 16%|█▌        | 35204/225000 [77:38:51<350:18:53,  6.64s/it]
 16%|█▌        | 35205/225000 [77:38:57<352:03:44,  6.68s/it]
 16%|█▌        | 35206/225000 [77:39:02<324:33:36,  6.16s/it]
 16%|█▌        | 35207/225000 [77:39:07<301:47:39,  5.72s/it]
 16%|█▌        | 35208/225000 [77:39:18<386:57:01,  7.34s/it]
 16%|█▌        | 35209/225000 [77:39:24<356:53:36,  6.77s/it]
 16%|█▌        | 35210/225000 [77:39:30<342:37:42,  6.50s/it]
 16%|█▌        | 35211/225000 [77:39:38<373:33:44,  7.09s/it]
 16%|█▌        | 35212/225000 [77:39:43<341:40:39,  6.48s/it]
 16%|█▌        | 35213/225000 [77:39:48<314:18:31,  5.96s/it]
 16%|█▌        | 35214/225000 [77:39:53<294:35:48,  5.59s/it]
 16%|█▌        | 35215/225000 [77:39:58<285:32:56,  5.42s/it]
 16%|█▌        | 35216/225000 [77:40:02<272:27:19,  5.17s/it]
 16%|█▌        | 35217/225000 [77:40:07<269:03:58,  5.10s/it]
 16%|█▌        | 35218/225000 [77:40:20<392:29:58,  7.45s/it]
 16%|█▌        | 35219/225000 [77:40:27<384:06:06,  7.29s/it]
 16%|█▌        | 35220/225000 [77:40:32<352:09:39,  6.68s/it]
 16%|█▌        | 35221/225000 [77:40:41<386:14:29,  7.33s/it]
 16%|█▌        | 35222/225000 [77:40:47<360:51:54,  6.85s/it]
 16%|█▌        | 35223/225000 [77:40:52<338:08:23,  6.41s/it]
 16%|█▌        | 35224/225000 [77:40:58<322:01:14,  6.11s/it]
 16%|█▌        | 35225/225000 [77:41:03<306:30:39,  5.81s/it]
 16%|█▌        | 35226/225000 [77:41:08<298:54:44,  5.67s/it]
 16%|█▌        | 35227/225000 [77:41:13<281:49:31,  5.35s/it]
 16%|█▌        | 35228/225000 [77:41:18<284:59:35,  5.41s/it]
 16%|█▌        | 35229/225000 [77:41:23<282:08:21,  5.35s/it]
 16%|█▌        | 35230/225000 [77:41:29<287:57:32,  5.46s/it]
 16%|█▌        | 35231/225000 [77:41:41<389:15:50,  7.38s/it]
 16%|█▌        | 35232/225000 [77:41:51<428:15:54,  8.12s/it]
 16%|█▌        | 35233/225000 [77:41:55<366:06:23,  6.95s/it]
 16%|█▌        | 35234/225000 [77:42:04<400:41:45,  7.60s/it]
 16%|█▌        | 35235/225000 [77:42:10<378:01:54,  7.17s/it]
 16%|█▌        | 35236/225000 [77:42:15<344:35:45,  6.54s/it]
 16%|█▌        | 35237/225000 [77:42:27<418:30:17,  7.94s/it]
 16%|█▌        | 35238/225000 [77:42:30<351:39:07,  6.67s/it]
 16%|█▌        | 35239/225000 [77:42:39<383:48:03,  7.28s/it]
 16%|█▌        | 35240/225000 [77:42:45<369:29:06,  7.01s/it]
 16%|█▌        | 35241/225000 [77:42:55<407:33:13,  7.73s/it]
 16%|█▌        | 35242/225000 [77:42:58<343:23:11,  6.51s/it]
 16%|█▌        | 35243/225000 [77:43:03<311:19:51,  5.91s/it]
 16%|█▌        | 35244/225000 [77:43:12<367:50:29,  6.98s/it]
 16%|█▌        | 35245/225000 [77:43:17<326:40:24,  6.20s/it]
 16%|█▌        | 35246/225000 [77:43:21<297:33:44,  5.65s/it]
 16%|█▌        | 35247/225000 [77:43:27<307:40:43,  5.84s/it]
 16%|█▌        | 35248/225000 [77:43:38<386:11:55,  7.33s/it]
 16%|█▌        | 35249/225000 [77:43:42<333:50:52,  6.33s/it]
 16%|█▌        | 35250/225000 [77:43:47<306:12:43,  5.81s/it]
                                                             
{'loss': '0.7495', 'grad_norm': '0.6161', 'learning_rate': '4.832e-05', 'epoch': '0.1567', 'num_input_tokens_seen': 132178672, 'train_runtime': '2.798e+05', 'train_tokens_per_second': '472.4'}

 16%|█▌        | 35250/225000 [77:43:47<306:12:43,  5.81s/it]
 16%|█▌        | 35251/225000 [77:43:58<394:57:29,  7.49s/it]
 16%|█▌        | 35252/225000 [77:44:03<344:33:05,  6.54s/it]
 16%|█▌        | 35253/225000 [77:44:07<310:55:14,  5.90s/it]
 16%|█▌        | 35254/225000 [77:44:17<372:25:52,  7.07s/it]
 16%|█▌        | 35255/225000 [77:44:23<352:00:29,  6.68s/it]
 16%|█▌        | 35256/225000 [77:44:28<330:54:43,  6.28s/it]
 16%|█▌        | 35257/225000 [77:44:37<371:45:08,  7.05s/it]
 16%|█▌        | 35258/225000 [77:44:42<337:01:16,  6.39s/it]
 16%|█▌        | 35259/225000 [77:44:49<347:11:18,  6.59s/it]
 16%|█▌        | 35260/225000 [77:44:53<317:48:12,  6.03s/it]
 16%|█▌        | 35261/225000 [77:44:58<296:52:37,  5.63s/it]
 16%|█▌        | 35262/225000 [77:45:03<282:23:34,  5.36s/it]
 16%|█▌        | 35263/225000 [77:45:16<403:00:42,  7.65s/it]
 16%|█▌        | 35264/225000 [77:45:20<355:31:51,  6.75s/it]
 16%|█▌        | 35265/225000 [77:45:26<332:00:36,  6.30s/it]
 16%|█▌        | 35266/225000 [77:45:37<406:29:13,  7.71s/it]
 16%|█▌        | 35267/225000 [77:45:40<334:05:26,  6.34s/it]
 16%|█▌        | 35268/225000 [77:45:47<353:40:45,  6.71s/it]
 16%|█▌        | 35269/225000 [77:45:51<302:04:43,  5.73s/it]
 16%|█▌        | 35270/225000 [77:46:01<365:41:45,  6.94s/it]
 16%|█▌        | 35271/225000 [77:46:04<313:14:06,  5.94s/it]
 16%|█▌        | 35272/225000 [77:46:10<313:42:27,  5.95s/it]
 16%|█▌        | 35273/225000 [77:46:17<328:10:19,  6.23s/it]
 16%|█▌        | 35274/225000 [77:46:22<303:03:26,  5.75s/it]
 16%|█▌        | 35275/225000 [77:46:32<374:52:14,  7.11s/it]
 16%|█▌        | 35276/225000 [77:46:36<318:17:50,  6.04s/it]
 16%|█▌        | 35277/225000 [77:46:41<303:42:59,  5.76s/it]
 16%|█▌        | 35278/225000 [77:46:46<296:06:36,  5.62s/it]
 16%|█▌        | 35279/225000 [77:46:51<284:02:41,  5.39s/it]
 16%|█▌        | 35280/225000 [77:46:56<273:28:48,  5.19s/it]
 16%|█▌        | 35281/225000 [77:47:09<399:11:44,  7.57s/it]
 16%|█▌        | 35282/225000 [77:47:14<368:32:18,  6.99s/it]
 16%|█▌        | 35283/225000 [77:47:20<350:58:52,  6.66s/it]
 16%|█▌        | 35284/225000 [77:47:27<359:27:54,  6.82s/it]
 16%|█▌        | 35285/225000 [77:47:32<318:04:54,  6.04s/it]
 16%|█▌        | 35286/225000 [77:47:38<327:29:10,  6.21s/it]
 16%|█▌        | 35287/225000 [77:47:48<389:23:31,  7.39s/it]
 16%|█▌        | 35288/225000 [77:47:52<327:35:44,  6.22s/it]
 16%|█▌        | 35289/225000 [77:47:59<336:56:33,  6.39s/it]
 16%|█▌        | 35290/225000 [77:48:04<319:03:53,  6.05s/it]
 16%|█▌        | 35291/225000 [77:48:09<302:32:52,  5.74s/it]
 16%|█▌        | 35292/225000 [77:48:14<290:48:53,  5.52s/it]
 16%|█▌        | 35293/225000 [77:48:19<286:14:54,  5.43s/it]
 16%|█▌        | 35294/225000 [77:48:25<288:39:35,  5.48s/it]
 16%|█▌        | 35295/225000 [77:48:36<387:15:57,  7.35s/it]
 16%|█▌        | 35296/225000 [77:48:46<425:01:41,  8.07s/it]
 16%|█▌        | 35297/225000 [77:48:50<362:31:23,  6.88s/it]
 16%|█▌        | 35298/225000 [77:49:01<426:47:21,  8.10s/it]
 16%|█▌        | 35299/225000 [77:49:06<369:33:23,  7.01s/it]
 16%|█▌        | 35300/225000 [77:49:09<316:03:59,  6.00s/it]
                                                             
{'loss': '0.747', 'grad_norm': '0.7665', 'learning_rate': '4.831e-05', 'epoch': '0.1569', 'num_input_tokens_seen': 132324112, 'train_runtime': '2.801e+05', 'train_tokens_per_second': '472.3'}

 16%|█▌        | 35300/225000 [77:49:09<316:03:59,  6.00s/it]
 16%|█▌        | 35301/225000 [77:49:14<293:59:30,  5.58s/it]
 16%|█▌        | 35302/225000 [77:49:26<392:17:18,  7.44s/it]
 16%|█▌        | 35303/225000 [77:49:29<330:39:27,  6.28s/it]
 16%|█▌        | 35304/225000 [77:49:38<368:34:39,  6.99s/it]
 16%|█▌        | 35305/225000 [77:49:42<323:30:43,  6.14s/it]
 16%|█▌        | 35306/225000 [77:49:48<312:35:47,  5.93s/it]
 16%|█▌        | 35307/225000 [77:49:53<304:33:15,  5.78s/it]
 16%|█▌        | 35308/225000 [77:49:59<305:50:53,  5.80s/it]
 16%|█▌        | 35309/225000 [77:50:04<295:44:17,  5.61s/it]
 16%|█▌        | 35310/225000 [77:50:11<313:54:07,  5.96s/it]
 16%|█▌        | 35311/225000 [77:50:24<430:25:51,  8.17s/it]
 16%|█▌        | 35312/225000 [77:50:31<407:26:56,  7.73s/it]
 16%|█▌        | 35313/225000 [77:50:35<357:41:46,  6.79s/it]
 16%|█▌        | 35314/225000 [77:50:41<345:44:40,  6.56s/it]
 16%|█▌        | 35315/225000 [77:50:50<384:26:41,  7.30s/it]
 16%|█▌        | 35316/225000 [77:50:54<329:59:04,  6.26s/it]
 16%|█▌        | 35317/225000 [77:50:59<311:09:39,  5.91s/it]
 16%|█▌        | 35318/225000 [77:51:08<347:33:20,  6.60s/it]
 16%|█▌        | 35319/225000 [77:51:13<326:11:34,  6.19s/it]
 16%|█▌        | 35320/225000 [77:51:18<307:58:02,  5.85s/it]
 16%|█▌        | 35321/225000 [77:51:23<302:14:31,  5.74s/it]
 16%|█▌        | 35322/225000 [77:51:29<305:39:55,  5.80s/it]
 16%|█▌        | 35323/225000 [77:51:35<310:17:28,  5.89s/it]
 16%|█▌        | 35324/225000 [77:51:42<324:18:23,  6.16s/it]
 16%|█▌        | 35325/225000 [77:51:55<431:23:10,  8.19s/it]
 16%|█▌        | 35326/225000 [77:52:00<376:51:46,  7.15s/it]
 16%|█▌        | 35327/225000 [77:52:04<334:42:40,  6.35s/it]
 16%|█▌        | 35328/225000 [77:52:09<313:50:23,  5.96s/it]
 16%|█▌        | 35329/225000 [77:52:14<298:55:00,  5.67s/it]
 16%|█▌        | 35330/225000 [77:52:19<290:31:43,  5.51s/it]
 16%|█▌        | 35331/225000 [77:52:30<370:48:29,  7.04s/it]
 16%|█▌        | 35332/225000 [77:52:38<392:37:43,  7.45s/it]
 16%|█▌        | 35333/225000 [77:52:43<349:45:45,  6.64s/it]
 16%|█▌        | 35334/225000 [77:52:49<332:27:12,  6.31s/it]
 16%|█▌        | 35335/225000 [77:52:54<318:05:54,  6.04s/it]
 16%|█▌        | 35336/225000 [77:53:00<308:29:08,  5.86s/it]
 16%|█▌        | 35337/225000 [77:53:05<301:31:17,  5.72s/it]
 16%|█▌        | 35338/225000 [77:53:10<297:36:41,  5.65s/it]
 16%|█▌        | 35339/225000 [77:53:16<290:40:13,  5.52s/it]
 16%|█▌        | 35340/225000 [77:53:26<372:57:54,  7.08s/it]
 16%|█▌        | 35341/225000 [77:53:35<402:47:30,  7.65s/it]
 16%|█▌        | 35342/225000 [77:53:41<367:44:20,  6.98s/it]
 16%|█▌        | 35343/225000 [77:53:46<342:16:31,  6.50s/it]
 16%|█▌        | 35344/225000 [77:53:52<324:26:09,  6.16s/it]
 16%|█▌        | 35345/225000 [77:53:57<314:48:20,  5.98s/it]
 16%|█▌        | 35346/225000 [77:54:10<418:08:34,  7.94s/it]
 16%|█▌        | 35347/225000 [77:54:15<383:51:22,  7.29s/it]
 16%|█▌        | 35348/225000 [77:54:22<368:06:55,  6.99s/it]
 16%|█▌        | 35349/225000 [77:54:28<362:16:49,  6.88s/it]
 16%|█▌        | 35350/225000 [77:54:36<379:23:46,  7.20s/it]
                                                             
{'loss': '0.7395', 'grad_norm': '0.8321', 'learning_rate': '4.831e-05', 'epoch': '0.1571', 'num_input_tokens_seen': 132468512, 'train_runtime': '2.805e+05', 'train_tokens_per_second': '472.3'}

 16%|█▌        | 35350/225000 [77:54:36<379:23:46,  7.20s/it]
 16%|█▌        | 35351/225000 [77:54:43<372:10:27,  7.06s/it]
 16%|█▌        | 35352/225000 [77:54:48<344:12:31,  6.53s/it]
 16%|█▌        | 35353/225000 [77:54:53<321:09:55,  6.10s/it]
 16%|█▌        | 35354/225000 [77:54:58<303:10:13,  5.76s/it]
 16%|█▌        | 35355/225000 [77:55:04<296:33:04,  5.63s/it]
 16%|█▌        | 35356/225000 [77:55:13<354:56:53,  6.74s/it]
 16%|█▌        | 35357/225000 [77:55:19<349:42:20,  6.64s/it]
 16%|█▌        | 35358/225000 [77:55:25<326:15:06,  6.19s/it]
 16%|█▌        | 35359/225000 [77:55:35<390:09:19,  7.41s/it]
 16%|█▌        | 35360/225000 [77:55:40<350:54:07,  6.66s/it]
 16%|█▌        | 35361/225000 [77:55:47<360:09:22,  6.84s/it]
 16%|█▌        | 35362/225000 [77:55:52<325:14:47,  6.17s/it]
 16%|█▌        | 35363/225000 [77:55:57<314:37:01,  5.97s/it]
 16%|█▌        | 35364/225000 [77:56:03<309:10:11,  5.87s/it]
 16%|█▌        | 35365/225000 [77:56:08<304:01:36,  5.77s/it]
 16%|█▌        | 35366/225000 [77:56:13<293:02:28,  5.56s/it]
 16%|█▌        | 35367/225000 [77:56:26<399:02:35,  7.58s/it]
 16%|█▌        | 35368/225000 [77:56:29<334:38:40,  6.35s/it]
 16%|█▌        | 35369/225000 [77:56:37<354:13:47,  6.72s/it]
 16%|█▌        | 35370/225000 [77:56:44<366:52:57,  6.97s/it]
 16%|█▌        | 35371/225000 [77:56:49<324:40:53,  6.16s/it]
 16%|█▌        | 35372/225000 [77:56:56<343:13:40,  6.52s/it]
 16%|█▌        | 35373/225000 [77:57:03<360:12:28,  6.84s/it]
 16%|█▌        | 35374/225000 [77:57:08<326:47:19,  6.20s/it]
 16%|█▌        | 35375/225000 [77:57:12<295:09:29,  5.60s/it]
 16%|█▌        | 35376/225000 [77:57:24<388:36:23,  7.38s/it]
 16%|█▌        | 35377/225000 [77:57:30<370:51:28,  7.04s/it]
 16%|█▌        | 35378/225000 [77:57:35<336:42:52,  6.39s/it]
 16%|█▌        | 35379/225000 [77:57:40<314:15:21,  5.97s/it]
 16%|█▌        | 35380/225000 [77:57:45<304:40:09,  5.78s/it]
 16%|█▌        | 35381/225000 [77:57:51<299:58:49,  5.70s/it]
 16%|█▌        | 35382/225000 [77:57:56<286:41:36,  5.44s/it]
 16%|█▌        | 35383/225000 [77:58:01<281:53:04,  5.35s/it]
 16%|█▌        | 35384/225000 [77:58:07<291:02:16,  5.53s/it]
 16%|█▌        | 35385/225000 [77:58:19<400:23:27,  7.60s/it]
 16%|█▌        | 35386/225000 [77:58:27<409:00:41,  7.77s/it]
 16%|█▌        | 35387/225000 [77:58:31<346:43:44,  6.58s/it]
 16%|█▌        | 35388/225000 [77:58:36<316:37:44,  6.01s/it]
 16%|█▌        | 35389/225000 [77:58:41<296:58:47,  5.64s/it]
 16%|█▌        | 35390/225000 [77:58:45<284:04:33,  5.39s/it]
 16%|█▌        | 35391/225000 [77:58:53<325:47:32,  6.19s/it]
 16%|█▌        | 35392/225000 [77:59:06<419:17:58,  7.96s/it]
 16%|█▌        | 35393/225000 [77:59:10<355:29:32,  6.75s/it]
 16%|█▌        | 35394/225000 [77:59:17<360:41:36,  6.85s/it]
 16%|█▌        | 35395/225000 [77:59:22<337:16:07,  6.40s/it]
 16%|█▌        | 35396/225000 [77:59:27<308:58:35,  5.87s/it]
 16%|█▌        | 35397/225000 [77:59:31<287:49:59,  5.47s/it]
 16%|█▌        | 35398/225000 [77:59:38<309:50:23,  5.88s/it]
 16%|█▌        | 35399/225000 [77:59:49<385:18:56,  7.32s/it]
 16%|█▌        | 35400/225000 [77:59:56<383:56:11,  7.29s/it]
                                                             
{'loss': '0.7341', 'grad_norm': '0.7508', 'learning_rate': '4.83e-05', 'epoch': '0.1573', 'num_input_tokens_seen': 132612112, 'train_runtime': '2.808e+05', 'train_tokens_per_second': '472.3'}

 16%|█▌        | 35400/225000 [77:59:56<383:56:11,  7.29s/it]
 16%|█▌        | 35401/225000 [78:00:00<337:26:15,  6.41s/it]
 16%|█▌        | 35402/225000 [78:00:05<305:47:30,  5.81s/it]
 16%|█▌        | 35403/225000 [78:00:12<323:16:55,  6.14s/it]
 16%|█▌        | 35404/225000 [78:00:20<359:31:42,  6.83s/it]
 16%|█▌        | 35405/225000 [78:00:26<349:30:22,  6.64s/it]
 16%|█▌        | 35406/225000 [78:00:35<377:07:39,  7.16s/it]
 16%|█▌        | 35407/225000 [78:00:38<317:22:15,  6.03s/it]
 16%|█▌        | 35408/225000 [78:00:42<292:16:04,  5.55s/it]
 16%|█▌        | 35409/225000 [78:00:49<317:14:55,  6.02s/it]
 16%|█▌        | 35410/225000 [78:01:00<388:17:21,  7.37s/it]
 16%|█▌        | 35411/225000 [78:01:04<333:03:58,  6.32s/it]
 16%|█▌        | 35412/225000 [78:01:12<366:40:11,  6.96s/it]
 16%|█▌        | 35413/225000 [78:01:17<331:29:45,  6.29s/it]
 16%|█▌        | 35414/225000 [78:01:24<347:40:51,  6.60s/it]
 16%|█▌        | 35415/225000 [78:01:29<313:13:56,  5.95s/it]
 16%|█▌        | 35416/225000 [78:01:34<302:54:12,  5.75s/it]
 16%|█▌        | 35417/225000 [78:01:39<289:25:55,  5.50s/it]
 16%|█▌        | 35418/225000 [78:01:45<290:19:46,  5.51s/it]
 16%|█▌        | 35419/225000 [78:01:56<376:37:24,  7.15s/it]
 16%|█▌        | 35420/225000 [78:01:59<323:13:36,  6.14s/it]
 16%|█▌        | 35421/225000 [78:02:06<329:43:20,  6.26s/it]
 16%|█▌        | 35422/225000 [78:02:15<381:33:07,  7.25s/it]
 16%|█▌        | 35423/225000 [78:02:21<349:48:59,  6.64s/it]
 16%|█▌        | 35424/225000 [78:02:26<333:42:37,  6.34s/it]
 16%|█▌        | 35425/225000 [78:02:31<316:14:20,  6.01s/it]
 16%|█▌        | 35426/225000 [78:02:37<307:22:27,  5.84s/it]
 16%|█▌        | 35427/225000 [78:02:42<301:48:36,  5.73s/it]
 16%|█▌        | 35428/225000 [78:02:48<299:15:53,  5.68s/it]
 16%|█▌        | 35429/225000 [78:02:53<293:41:53,  5.58s/it]
 16%|█▌        | 35430/225000 [78:03:06<402:40:55,  7.65s/it]
 16%|█▌        | 35431/225000 [78:03:13<396:55:57,  7.54s/it]
 16%|█▌        | 35432/225000 [78:03:21<398:48:04,  7.57s/it]
 16%|█▌        | 35433/225000 [78:03:25<354:04:31,  6.72s/it]
 16%|█▌        | 35434/225000 [78:03:33<359:54:27,  6.83s/it]
 16%|█▌        | 35435/225000 [78:03:40<375:58:25,  7.14s/it]
 16%|█▌        | 35436/225000 [78:03:44<322:20:50,  6.12s/it]
 16%|█▌        | 35437/225000 [78:03:49<301:35:40,  5.73s/it]
 16%|█▌        | 35438/225000 [78:03:54<291:58:22,  5.54s/it]
 16%|█▌        | 35439/225000 [78:04:04<365:19:48,  6.94s/it]
 16%|█▌        | 35440/225000 [78:04:12<374:45:37,  7.12s/it]
 16%|█▌        | 35441/225000 [78:04:16<336:14:54,  6.39s/it]
 16%|█▌        | 35442/225000 [78:04:21<306:47:28,  5.83s/it]
 16%|█▌        | 35443/225000 [78:04:26<291:00:44,  5.53s/it]
 16%|█▌        | 35444/225000 [78:04:37<375:36:59,  7.13s/it]
 16%|█▌        | 35445/225000 [78:04:44<370:08:31,  7.03s/it]
 16%|█▌        | 35446/225000 [78:04:48<333:38:33,  6.34s/it]
 16%|█▌        | 35447/225000 [78:05:00<412:09:13,  7.83s/it]
 16%|█▌        | 35448/225000 [78:05:03<346:01:28,  6.57s/it]
 16%|█▌        | 35449/225000 [78:05:10<357:42:37,  6.79s/it]
 16%|█▌        | 35450/225000 [78:05:16<332:21:39,  6.31s/it]
                                                             
{'loss': '0.7502', 'grad_norm': '0.7523', 'learning_rate': '4.829e-05', 'epoch': '0.1576', 'num_input_tokens_seen': 132755872, 'train_runtime': '2.811e+05', 'train_tokens_per_second': '472.2'}

 16%|█▌        | 35450/225000 [78:05:16<332:21:39,  6.31s/it]
 16%|█▌        | 35451/225000 [78:05:20<308:13:20,  5.85s/it]
 16%|█▌        | 35452/225000 [78:05:25<290:14:17,  5.51s/it]
 16%|█▌        | 35453/225000 [78:05:30<281:45:23,  5.35s/it]
 16%|█▌        | 35454/225000 [78:05:37<302:24:00,  5.74s/it]
 16%|█▌        | 35455/225000 [78:05:50<416:05:01,  7.90s/it]
 16%|█▌        | 35456/225000 [78:05:53<340:46:16,  6.47s/it]
 16%|█▌        | 35457/225000 [78:06:01<364:36:17,  6.92s/it]
 16%|█▌        | 35458/225000 [78:06:06<330:52:18,  6.28s/it]
 16%|█▌        | 35459/225000 [78:06:11<312:27:56,  5.93s/it]
 16%|█▌        | 35460/225000 [78:06:16<297:26:43,  5.65s/it]
 16%|█▌        | 35461/225000 [78:06:25<346:50:53,  6.59s/it]
 16%|█▌        | 35462/225000 [78:06:33<376:13:35,  7.15s/it]
 16%|█▌        | 35463/225000 [78:06:37<329:05:23,  6.25s/it]
 16%|█▌        | 35464/225000 [78:06:48<402:28:57,  7.64s/it]
 16%|█▌        | 35465/225000 [78:06:51<336:08:28,  6.38s/it]
 16%|█▌        | 35466/225000 [78:07:01<380:27:49,  7.23s/it]
 16%|█▌        | 35467/225000 [78:07:05<328:12:14,  6.23s/it]
 16%|█▌        | 35468/225000 [78:07:13<362:44:56,  6.89s/it]
 16%|█▌        | 35469/225000 [78:07:17<312:37:05,  5.94s/it]
 16%|█▌        | 35470/225000 [78:07:28<389:06:35,  7.39s/it]
 16%|█▌        | 35471/225000 [78:07:32<337:27:38,  6.41s/it]
 16%|█▌        | 35472/225000 [78:07:36<299:26:19,  5.69s/it]
 16%|█▌        | 35473/225000 [78:07:40<273:37:46,  5.20s/it]
 16%|█▌        | 35474/225000 [78:07:44<259:44:43,  4.93s/it]
 16%|█▌        | 35475/225000 [78:07:57<382:59:25,  7.27s/it]
 16%|█▌        | 35476/225000 [78:08:02<343:38:31,  6.53s/it]
 16%|█▌        | 35477/225000 [78:08:06<309:31:09,  5.88s/it]
 16%|█▌        | 35478/225000 [78:08:10<288:22:21,  5.48s/it]
 16%|█▌        | 35479/225000 [78:08:22<385:34:48,  7.32s/it]
 16%|█▌        | 35480/225000 [78:08:27<344:14:46,  6.54s/it]
 16%|█▌        | 35481/225000 [78:08:31<306:22:12,  5.82s/it]
 16%|█▌        | 35482/225000 [78:08:40<350:51:39,  6.66s/it]
 16%|█▌        | 35483/225000 [78:08:47<357:03:43,  6.78s/it]
 16%|█▌        | 35484/225000 [78:08:50<308:05:25,  5.85s/it]
 16%|█▌        | 35485/225000 [78:08:58<341:57:01,  6.50s/it]
 16%|█▌        | 35486/225000 [78:09:07<375:44:18,  7.14s/it]
 16%|█▌        | 35487/225000 [78:09:11<327:02:16,  6.21s/it]
 16%|█▌        | 35488/225000 [78:09:17<326:42:42,  6.21s/it]
 16%|█▌        | 35489/225000 [78:09:24<342:26:32,  6.51s/it]
 16%|█▌        | 35490/225000 [78:09:31<351:04:49,  6.67s/it]
 16%|█▌        | 35491/225000 [78:09:38<353:55:47,  6.72s/it]
 16%|█▌        | 35492/225000 [78:09:44<332:48:46,  6.32s/it]
 16%|█▌        | 35493/225000 [78:09:49<312:23:14,  5.93s/it]
 16%|█▌        | 35494/225000 [78:09:54<303:27:20,  5.76s/it]
 16%|█▌        | 35495/225000 [78:10:00<301:35:15,  5.73s/it]
 16%|█▌        | 35496/225000 [78:10:05<292:37:26,  5.56s/it]
 16%|█▌        | 35497/225000 [78:10:10<287:33:46,  5.46s/it]
 16%|█▌        | 35498/225000 [78:10:15<284:15:52,  5.40s/it]
 16%|█▌        | 35499/225000 [78:10:21<280:44:47,  5.33s/it]
 16%|█▌        | 35500/225000 [78:10:34<402:52:12,  7.65s/it]
                                                             
{'loss': '0.7455', 'grad_norm': '0.6566', 'learning_rate': '4.829e-05', 'epoch': '0.1578', 'num_input_tokens_seen': 132899240, 'train_runtime': '2.814e+05', 'train_tokens_per_second': '472.2'}

 16%|█▌        | 35500/225000 [78:10:34<402:52:12,  7.65s/it]
 16%|█▌        | 35501/225000 [78:10:40<385:19:33,  7.32s/it]
 16%|█▌        | 35502/225000 [78:10:47<371:48:07,  7.06s/it]
 16%|█▌        | 35503/225000 [78:10:54<379:10:10,  7.20s/it]
 16%|█▌        | 35504/225000 [78:10:59<343:12:46,  6.52s/it]
 16%|█▌        | 35505/225000 [78:11:05<328:44:36,  6.25s/it]
 16%|█▌        | 35506/225000 [78:11:10<312:37:33,  5.94s/it]
 16%|█▌        | 35507/225000 [78:11:15<304:08:54,  5.78s/it]
 16%|█▌        | 35508/225000 [78:11:21<309:00:02,  5.87s/it]
 16%|█▌        | 35509/225000 [78:11:27<305:49:19,  5.81s/it]
 16%|█▌        | 35510/225000 [78:11:36<351:12:20,  6.67s/it]
 16%|█▌        | 35511/225000 [78:11:46<401:57:14,  7.64s/it]
 16%|█▌        | 35512/225000 [78:11:50<347:18:37,  6.60s/it]
 16%|█▌        | 35513/225000 [78:11:59<395:11:44,  7.51s/it]
 16%|█▌        | 35514/225000 [78:12:03<340:05:57,  6.46s/it]
 16%|█▌        | 35515/225000 [78:12:10<339:33:24,  6.45s/it]
 16%|█▌        | 35516/225000 [78:12:18<369:17:01,  7.02s/it]
 16%|█▌        | 35517/225000 [78:12:24<348:58:48,  6.63s/it]
 16%|█▌        | 35518/225000 [78:12:32<366:42:31,  6.97s/it]
 16%|█▌        | 35519/225000 [78:12:36<330:12:05,  6.27s/it]
 16%|█▌        | 35520/225000 [78:12:41<302:43:24,  5.75s/it]
 16%|█▌        | 35521/225000 [78:12:45<274:15:52,  5.21s/it]
 16%|█▌        | 35522/225000 [78:12:58<394:13:26,  7.49s/it]
 16%|█▌        | 35523/225000 [78:13:03<354:32:46,  6.74s/it]
 16%|█▌        | 35524/225000 [78:13:08<340:10:44,  6.46s/it]
 16%|█▌        | 35525/225000 [78:13:18<386:48:45,  7.35s/it]
 16%|█▌        | 35526/225000 [78:13:23<347:23:52,  6.60s/it]
 16%|█▌        | 35527/225000 [78:13:31<371:34:48,  7.06s/it]
 16%|█▌        | 35528/225000 [78:13:37<352:22:11,  6.70s/it]
 16%|█▌        | 35529/225000 [78:13:42<322:38:39,  6.13s/it]
 16%|█▌        | 35530/225000 [78:13:47<309:50:42,  5.89s/it]
 16%|█▌        | 35531/225000 [78:13:52<300:45:24,  5.71s/it]
 16%|█▌        | 35532/225000 [78:13:57<290:11:27,  5.51s/it]
 16%|█▌        | 35533/225000 [78:14:02<279:47:17,  5.32s/it]
 16%|█▌        | 35534/225000 [78:14:16<410:57:35,  7.81s/it]
 16%|█▌        | 35535/225000 [78:14:20<363:39:17,  6.91s/it]
 16%|█▌        | 35536/225000 [78:14:27<353:46:44,  6.72s/it]
 16%|█▌        | 35537/225000 [78:14:35<378:03:43,  7.18s/it]
 16%|█▌        | 35538/225000 [78:14:41<363:25:08,  6.91s/it]
 16%|█▌        | 35539/225000 [78:14:46<335:23:05,  6.37s/it]
 16%|█▌        | 35540/225000 [78:14:52<317:31:16,  6.03s/it]
 16%|█▌        | 35541/225000 [78:14:56<297:05:30,  5.65s/it]
 16%|█▌        | 35542/225000 [78:15:01<282:18:22,  5.36s/it]
 16%|█▌        | 35543/225000 [78:15:12<368:51:48,  7.01s/it]
 16%|█▌        | 35544/225000 [78:15:20<386:18:09,  7.34s/it]
 16%|█▌        | 35545/225000 [78:15:24<332:19:17,  6.31s/it]
 16%|█▌        | 35546/225000 [78:15:31<350:45:43,  6.67s/it]
 16%|█▌        | 35547/225000 [78:15:39<357:22:04,  6.79s/it]
 16%|█▌        | 35548/225000 [78:15:43<322:27:04,  6.13s/it]
 16%|█▌        | 35549/225000 [78:15:50<340:52:56,  6.48s/it]
 16%|█▌        | 35550/225000 [78:15:55<309:48:17,  5.89s/it]
                                                             
{'loss': '0.7249', 'grad_norm': '0.7081', 'learning_rate': '4.828e-05', 'epoch': '0.158', 'num_input_tokens_seen': 133044712, 'train_runtime': '2.818e+05', 'train_tokens_per_second': '472.2'}

 16%|█▌        | 35550/225000 [78:15:55<309:48:17,  5.89s/it]
 16%|█▌        | 35551/225000 [78:16:00<290:33:06,  5.52s/it]
 16%|█▌        | 35552/225000 [78:16:11<385:16:33,  7.32s/it]
 16%|█▌        | 35553/225000 [78:16:15<330:15:12,  6.28s/it]
 16%|█▌        | 35554/225000 [78:16:25<386:58:13,  7.35s/it]
 16%|█▌        | 35555/225000 [78:16:29<339:17:18,  6.45s/it]
 16%|█▌        | 35556/225000 [78:16:37<366:25:05,  6.96s/it]
 16%|█▌        | 35557/225000 [78:16:41<311:57:13,  5.93s/it]
 16%|█▌        | 35558/225000 [78:16:45<286:35:49,  5.45s/it]
 16%|█▌        | 35559/225000 [78:16:56<373:37:01,  7.10s/it]
 16%|█▌        | 35560/225000 [78:17:00<321:36:51,  6.11s/it]
 16%|█▌        | 35561/225000 [78:17:09<370:23:18,  7.04s/it]
 16%|█▌        | 35562/225000 [78:17:14<336:35:36,  6.40s/it]
 16%|█▌        | 35563/225000 [78:17:18<303:51:38,  5.77s/it]
 16%|█▌        | 35564/225000 [78:17:28<360:18:45,  6.85s/it]
 16%|█▌        | 35565/225000 [78:17:34<352:29:20,  6.70s/it]
 16%|█▌        | 35566/225000 [78:17:38<308:37:16,  5.87s/it]
 16%|█▌        | 35567/225000 [78:17:44<318:32:49,  6.05s/it]
 16%|█▌        | 35568/225000 [78:17:52<343:03:37,  6.52s/it]
 16%|█▌        | 35569/225000 [78:17:56<304:24:00,  5.78s/it]
 16%|█▌        | 35570/225000 [78:18:00<279:28:44,  5.31s/it]
 16%|█▌        | 35571/225000 [78:18:11<369:21:36,  7.02s/it]
 16%|█▌        | 35572/225000 [78:18:16<332:42:58,  6.32s/it]
 16%|█▌        | 35573/225000 [78:18:21<307:14:35,  5.84s/it]
 16%|█▌        | 35574/225000 [78:18:26<292:19:20,  5.56s/it]
 16%|█▌        | 35575/225000 [78:18:37<376:31:29,  7.16s/it]
 16%|█▌        | 35576/225000 [78:18:43<364:43:53,  6.93s/it]
 16%|█▌        | 35577/225000 [78:18:51<389:02:36,  7.39s/it]
 16%|█▌        | 35578/225000 [78:18:55<334:48:59,  6.36s/it]
 16%|█▌        | 35579/225000 [78:19:04<371:55:49,  7.07s/it]
 16%|█▌        | 35580/225000 [78:19:08<324:49:48,  6.17s/it]
 16%|█▌        | 35581/225000 [78:19:15<339:56:24,  6.46s/it]
 16%|█▌        | 35582/225000 [78:19:23<354:37:59,  6.74s/it]
 16%|█▌        | 35583/225000 [78:19:26<302:39:10,  5.75s/it]
 16%|█▌        | 35584/225000 [78:19:34<343:03:28,  6.52s/it]
 16%|█▌        | 35585/225000 [78:19:39<315:27:29,  6.00s/it]
 16%|█▌        | 35586/225000 [78:19:44<296:13:03,  5.63s/it]
 16%|█▌        | 35587/225000 [78:19:49<287:16:29,  5.46s/it]
 16%|█▌        | 35588/225000 [78:19:55<300:06:39,  5.70s/it]
 16%|█▌        | 35589/225000 [78:20:08<417:11:54,  7.93s/it]
 16%|█▌        | 35590/225000 [78:20:15<393:55:13,  7.49s/it]
 16%|█▌        | 35591/225000 [78:20:20<362:27:55,  6.89s/it]
 16%|█▌        | 35592/225000 [78:20:24<310:01:26,  5.89s/it]
 16%|█▌        | 35593/225000 [78:20:36<406:22:37,  7.72s/it]
 16%|█▌        | 35594/225000 [78:20:39<339:30:38,  6.45s/it]
 16%|█▌        | 35595/225000 [78:20:44<314:27:44,  5.98s/it]
 16%|█▌        | 35596/225000 [78:20:54<375:52:15,  7.14s/it]
 16%|█▌        | 35597/225000 [78:20:58<329:47:16,  6.27s/it]
 16%|█▌        | 35598/225000 [78:21:08<382:28:54,  7.27s/it]
 16%|█▌        | 35599/225000 [78:21:12<332:25:33,  6.32s/it]
 16%|█▌        | 35600/225000 [78:21:20<350:05:17,  6.65s/it]
                                                             
{'loss': '0.7336', 'grad_norm': '0.7127', 'learning_rate': '4.827e-05', 'epoch': '0.1582', 'num_input_tokens_seen': 133189840, 'train_runtime': '2.821e+05', 'train_tokens_per_second': '472.2'}

 16%|█▌        | 35600/225000 [78:21:20<350:05:17,  6.65s/it]
 16%|█▌        | 35601/225000 [78:21:27<365:34:51,  6.95s/it]
 16%|█▌        | 35602/225000 [78:21:31<313:33:30,  5.96s/it]
 16%|█▌        | 35603/225000 [78:21:36<297:40:38,  5.66s/it]
 16%|█▌        | 35604/225000 [78:21:42<299:06:29,  5.69s/it]
 16%|█▌        | 35605/225000 [78:21:55<415:39:00,  7.90s/it]
 16%|█▌        | 35606/225000 [78:21:59<352:06:52,  6.69s/it]
 16%|█▌        | 35607/225000 [78:22:07<374:36:22,  7.12s/it]
 16%|█▌        | 35608/225000 [78:22:13<364:54:25,  6.94s/it]
 16%|█▌        | 35609/225000 [78:22:18<329:42:46,  6.27s/it]
 16%|█▌        | 35610/225000 [78:22:22<303:24:06,  5.77s/it]
 16%|█▌        | 35611/225000 [78:22:28<297:39:21,  5.66s/it]
 16%|█▌        | 35612/225000 [78:22:40<396:48:34,  7.54s/it]
 16%|█▌        | 35613/225000 [78:22:43<333:42:49,  6.34s/it]
 16%|█▌        | 35614/225000 [78:22:51<355:07:36,  6.75s/it]
 16%|█▌        | 35615/225000 [78:22:55<315:59:28,  6.01s/it]
 16%|█▌        | 35616/225000 [78:23:00<292:08:50,  5.55s/it]
 16%|█▌        | 35617/225000 [78:23:04<270:21:13,  5.14s/it]
 16%|█▌        | 35618/225000 [78:23:16<380:18:23,  7.23s/it]
 16%|█▌        | 35619/225000 [78:23:24<386:46:38,  7.35s/it]
 16%|█▌        | 35620/225000 [78:23:30<365:41:42,  6.95s/it]
 16%|█▌        | 35621/225000 [78:23:35<332:58:35,  6.33s/it]
 16%|█▌        | 35622/225000 [78:23:45<394:14:49,  7.49s/it]
 16%|█▌        | 35623/225000 [78:23:49<334:12:54,  6.35s/it]
 16%|█▌        | 35624/225000 [78:23:57<367:15:32,  6.98s/it]
 16%|█▌        | 35625/225000 [78:24:05<376:36:30,  7.16s/it]
 16%|█▌        | 35626/225000 [78:24:09<338:00:45,  6.43s/it]
 16%|█▌        | 35627/225000 [78:24:14<313:27:51,  5.96s/it]
 16%|█▌        | 35628/225000 [78:24:19<296:14:22,  5.63s/it]
 16%|█▌        | 35629/225000 [78:24:24<289:27:47,  5.50s/it]
 16%|█▌        | 35630/225000 [78:24:29<280:11:43,  5.33s/it]
 16%|█▌        | 35631/225000 [78:24:42<395:21:49,  7.52s/it]
 16%|█▌        | 35632/225000 [78:24:46<340:19:27,  6.47s/it]
 16%|█▌        | 35633/225000 [78:24:55<384:00:44,  7.30s/it]
 16%|█▌        | 35634/225000 [78:24:58<319:55:10,  6.08s/it]
 16%|█▌        | 35635/225000 [78:25:04<318:27:20,  6.05s/it]
 16%|█▌        | 35636/225000 [78:25:12<351:29:24,  6.68s/it]
 16%|█▌        | 35637/225000 [78:25:17<314:50:17,  5.99s/it]
 16%|█▌        | 35638/225000 [78:25:25<346:15:40,  6.58s/it]
 16%|█▌        | 35639/225000 [78:25:31<341:14:46,  6.49s/it]
 16%|█▌        | 35640/225000 [78:25:38<344:25:55,  6.55s/it]
 16%|█▌        | 35641/225000 [78:25:43<332:42:09,  6.33s/it]
 16%|█▌        | 35642/225000 [78:25:52<367:23:23,  6.98s/it]
 16%|█▌        | 35643/225000 [78:25:56<324:15:53,  6.16s/it]
 16%|█▌        | 35644/225000 [78:26:05<370:42:50,  7.05s/it]
 16%|█▌        | 35645/225000 [78:26:10<338:15:16,  6.43s/it]
 16%|█▌        | 35646/225000 [78:26:14<298:48:05,  5.68s/it]
 16%|█▌        | 35647/225000 [78:26:22<326:21:50,  6.20s/it]
 16%|█▌        | 35648/225000 [78:26:28<334:15:28,  6.35s/it]
 16%|█▌        | 35649/225000 [78:26:32<296:30:52,  5.64s/it]
 16%|█▌        | 35650/225000 [78:26:42<361:37:49,  6.88s/it]
                                                             
{'loss': '0.7352', 'grad_norm': '0.7468', 'learning_rate': '4.827e-05', 'epoch': '0.1584', 'num_input_tokens_seen': 133333264, 'train_runtime': '2.824e+05', 'train_tokens_per_second': '472.1'}

 16%|█▌        | 35650/225000 [78:26:42<361:37:49,  6.88s/it]
 16%|█▌        | 35651/225000 [78:26:47<331:19:33,  6.30s/it]
 16%|█▌        | 35652/225000 [78:26:52<312:15:29,  5.94s/it]
 16%|█▌        | 35653/225000 [78:26:57<295:16:12,  5.61s/it]
 16%|█▌        | 35654/225000 [78:27:06<344:48:26,  6.56s/it]
 16%|█▌        | 35655/225000 [78:27:14<373:54:17,  7.11s/it]
 16%|█▌        | 35656/225000 [78:27:18<327:08:00,  6.22s/it]
 16%|█▌        | 35657/225000 [78:27:25<331:52:45,  6.31s/it]
 16%|█▌        | 35658/225000 [78:27:34<375:49:58,  7.15s/it]
 16%|█▌        | 35659/225000 [78:27:39<348:28:32,  6.63s/it]
 16%|█▌        | 35660/225000 [78:27:44<322:36:30,  6.13s/it]
 16%|█▌        | 35661/225000 [78:27:49<304:32:57,  5.79s/it]
 16%|█▌        | 35662/225000 [78:27:55<300:40:04,  5.72s/it]
 16%|█▌        | 35663/225000 [78:28:01<300:00:31,  5.70s/it]
 16%|█▌        | 35664/225000 [78:28:06<302:35:54,  5.75s/it]
 16%|█▌        | 35665/225000 [78:28:11<289:56:43,  5.51s/it]
 16%|█▌        | 35666/225000 [78:28:24<407:41:29,  7.75s/it]
 16%|█▌        | 35667/225000 [78:28:31<391:07:45,  7.44s/it]
 16%|█▌        | 35668/225000 [78:28:40<411:33:04,  7.83s/it]
 16%|█▌        | 35669/225000 [78:28:43<346:10:09,  6.58s/it]
 16%|█▌        | 35670/225000 [78:28:48<313:30:03,  5.96s/it]
 16%|█▌        | 35671/225000 [78:28:59<397:32:13,  7.56s/it]
 16%|█▌        | 35672/225000 [78:29:03<342:59:05,  6.52s/it]
 16%|█▌        | 35673/225000 [78:29:11<364:00:00,  6.92s/it]
 16%|█▌        | 35674/225000 [78:29:17<339:57:17,  6.46s/it]
 16%|█▌        | 35675/225000 [78:29:24<357:31:44,  6.80s/it]
 16%|█▌        | 35676/225000 [78:29:29<327:12:16,  6.22s/it]
 16%|█▌        | 35677/225000 [78:29:35<316:00:05,  6.01s/it]
 16%|█▌        | 35678/225000 [78:29:39<297:36:35,  5.66s/it]
 16%|█▌        | 35679/225000 [78:29:44<282:21:14,  5.37s/it]
 16%|█▌        | 35680/225000 [78:29:49<280:03:13,  5.33s/it]
 16%|█▌        | 35681/225000 [78:30:02<400:06:24,  7.61s/it]
 16%|█▌        | 35682/225000 [78:30:08<370:15:17,  7.04s/it]
 16%|█▌        | 35683/225000 [78:30:13<335:24:34,  6.38s/it]
 16%|█▌        | 35684/225000 [78:30:17<302:29:34,  5.75s/it]
 16%|█▌        | 35685/225000 [78:30:27<359:52:43,  6.84s/it]
 16%|█▌        | 35686/225000 [78:30:35<379:29:37,  7.22s/it]
 16%|█▌        | 35687/225000 [78:30:41<366:02:40,  6.96s/it]
 16%|█▌        | 35688/225000 [78:30:48<368:17:54,  7.00s/it]
 16%|█▌        | 35689/225000 [78:30:52<324:46:12,  6.18s/it]
 16%|█▌        | 35690/225000 [78:30:58<313:58:42,  5.97s/it]
 16%|█▌        | 35691/225000 [78:31:08<381:42:42,  7.26s/it]
 16%|█▌        | 35692/225000 [78:31:12<328:30:46,  6.25s/it]
 16%|█▌        | 35693/225000 [78:31:17<315:03:16,  5.99s/it]
 16%|█▌        | 35694/225000 [78:31:27<366:23:34,  6.97s/it]
 16%|█▌        | 35695/225000 [78:31:33<351:09:15,  6.68s/it]
 16%|█▌        | 35696/225000 [78:31:39<341:58:49,  6.50s/it]
 16%|█▌        | 35697/225000 [78:31:43<313:00:32,  5.95s/it]
 16%|█▌        | 35698/225000 [78:31:51<336:42:07,  6.40s/it]
 16%|█▌        | 35699/225000 [78:32:00<381:01:02,  7.25s/it]
 16%|█▌        | 35700/225000 [78:32:07<380:28:37,  7.24s/it]
                                                             
{'loss': '0.7429', 'grad_norm': '0.76', 'learning_rate': '4.826e-05', 'epoch': '0.1587', 'num_input_tokens_seen': 133478624, 'train_runtime': '2.827e+05', 'train_tokens_per_second': '472.1'}

 16%|█▌        | 35700/225000 [78:32:07<380:28:37,  7.24s/it]
 16%|█▌        | 35701/225000 [78:32:13<356:15:50,  6.78s/it]
 16%|█▌        | 35702/225000 [78:32:18<335:15:53,  6.38s/it]
 16%|█▌        | 35703/225000 [78:32:23<305:35:16,  5.81s/it]
 16%|█▌        | 35704/225000 [78:32:28<288:40:08,  5.49s/it]
 16%|█▌        | 35705/225000 [78:32:33<287:15:38,  5.46s/it]
 16%|█▌        | 35706/225000 [78:32:45<385:55:10,  7.34s/it]
 16%|█▌        | 35707/225000 [78:32:49<342:26:29,  6.51s/it]
 16%|█▌        | 35708/225000 [78:32:55<335:57:17,  6.39s/it]
 16%|█▌        | 35709/225000 [78:33:03<353:45:50,  6.73s/it]
 16%|█▌        | 35710/225000 [78:33:07<311:31:41,  5.92s/it]
 16%|█▌        | 35711/225000 [78:33:18<389:06:53,  7.40s/it]
 16%|█▌        | 35712/225000 [78:33:22<336:56:37,  6.41s/it]
 16%|█▌        | 35713/225000 [78:33:29<344:24:10,  6.55s/it]
 16%|█▌        | 35714/225000 [78:33:35<346:32:53,  6.59s/it]
 16%|█▌        | 35715/225000 [78:33:39<304:25:22,  5.79s/it]
 16%|█▌        | 35716/225000 [78:33:49<360:42:22,  6.86s/it]
 16%|█▌        | 35717/225000 [78:33:56<367:58:13,  7.00s/it]
 16%|█▌        | 35718/225000 [78:34:00<319:45:08,  6.08s/it]
 16%|█▌        | 35719/225000 [78:34:07<332:12:56,  6.32s/it]
 16%|█▌        | 35720/225000 [78:34:15<363:06:02,  6.91s/it]
 16%|█▌        | 35721/225000 [78:34:19<306:46:49,  5.83s/it]
 16%|█▌        | 35722/225000 [78:34:27<341:13:18,  6.49s/it]
 16%|█▌        | 35723/225000 [78:34:35<369:14:14,  7.02s/it]
 16%|█▌        | 35724/225000 [78:34:40<338:30:38,  6.44s/it]
 16%|█▌        | 35725/225000 [78:34:44<309:45:05,  5.89s/it]
 16%|█▌        | 35726/225000 [78:34:49<289:53:54,  5.51s/it]
 16%|█▌        | 35727/225000 [78:34:55<291:12:33,  5.54s/it]
 16%|█▌        | 35728/225000 [78:34:59<276:55:11,  5.27s/it]
 16%|█▌        | 35729/225000 [78:35:06<298:41:37,  5.68s/it]
 16%|█▌        | 35730/225000 [78:35:19<411:40:07,  7.83s/it]
 16%|█▌        | 35731/225000 [78:35:23<347:35:02,  6.61s/it]
 16%|█▌        | 35732/225000 [78:35:28<326:15:23,  6.21s/it]
 16%|█▌        | 35733/225000 [78:35:33<310:13:51,  5.90s/it]
 16%|█▌        | 35734/225000 [78:35:38<294:34:43,  5.60s/it]
 16%|█▌        | 35735/225000 [78:35:44<294:44:27,  5.61s/it]
 16%|█▌        | 35736/225000 [78:35:49<289:56:27,  5.51s/it]
 16%|█▌        | 35737/225000 [78:35:54<288:22:13,  5.49s/it]
 16%|█▌        | 35738/225000 [78:36:07<399:14:37,  7.59s/it]
 16%|█▌        | 35739/225000 [78:36:14<399:14:24,  7.59s/it]
 16%|█▌        | 35740/225000 [78:36:20<361:31:33,  6.88s/it]
 16%|█▌        | 35741/225000 [78:36:25<340:17:09,  6.47s/it]
 16%|█▌        | 35742/225000 [78:36:30<320:40:12,  6.10s/it]
 16%|█▌        | 35743/225000 [78:36:36<310:10:15,  5.90s/it]
 16%|█▌        | 35744/225000 [78:36:41<295:40:56,  5.62s/it]
 16%|█▌        | 35745/225000 [78:36:46<295:37:05,  5.62s/it]
 16%|█▌        | 35746/225000 [78:36:52<290:19:19,  5.52s/it]
 16%|█▌        | 35747/225000 [78:36:57<288:53:48,  5.50s/it]
 16%|█▌        | 35748/225000 [78:37:09<386:07:53,  7.35s/it]
 16%|█▌        | 35749/225000 [78:37:16<376:31:24,  7.16s/it]
 16%|█▌        | 35750/225000 [78:37:20<340:15:47,  6.47s/it]
                                                             
{'loss': '0.7265', 'grad_norm': '0.6743', 'learning_rate': '4.825e-05', 'epoch': '0.1589', 'num_input_tokens_seen': 133621768, 'train_runtime': '2.83e+05', 'train_tokens_per_second': '472.1'}

 16%|█▌        | 35750/225000 [78:37:20<340:15:47,  6.47s/it]
 16%|█▌        | 35751/225000 [78:37:26<330:58:23,  6.30s/it]
 16%|█▌        | 35752/225000 [78:37:39<428:25:12,  8.15s/it]
 16%|█▌        | 35753/225000 [78:37:43<359:13:43,  6.83s/it]
 16%|█▌        | 35754/225000 [78:37:52<399:10:44,  7.59s/it]
 16%|█▌        | 35755/225000 [78:37:56<339:47:30,  6.46s/it]
 16%|█▌        | 35756/225000 [78:38:04<373:56:10,  7.11s/it]
 16%|█▌        | 35757/225000 [78:38:09<327:35:07,  6.23s/it]
 16%|█▌        | 35758/225000 [78:38:13<299:40:39,  5.70s/it]
 16%|█▌        | 35759/225000 [78:38:23<368:43:36,  7.01s/it]
 16%|█▌        | 35760/225000 [78:38:29<349:44:52,  6.65s/it]
 16%|█▌        | 35761/225000 [78:38:37<369:32:24,  7.03s/it]
 16%|█▌        | 35762/225000 [78:38:42<343:55:24,  6.54s/it]
 16%|█▌        | 35763/225000 [78:38:49<352:36:51,  6.71s/it]
 16%|█▌        | 35764/225000 [78:38:58<380:38:43,  7.24s/it]
 16%|█▌        | 35765/225000 [78:39:02<328:15:59,  6.24s/it]
 16%|█▌        | 35766/225000 [78:39:11<377:30:29,  7.18s/it]
 16%|█▌        | 35767/225000 [78:39:18<367:27:24,  6.99s/it]
 16%|█▌        | 35768/225000 [78:39:24<363:07:17,  6.91s/it]
 16%|█▌        | 35769/225000 [78:39:30<338:23:27,  6.44s/it]
 16%|█▌        | 35770/225000 [78:39:34<311:29:44,  5.93s/it]
 16%|█▌        | 35771/225000 [78:39:40<300:48:27,  5.72s/it]
 16%|█▌        | 35772/225000 [78:39:45<296:34:19,  5.64s/it]
 16%|█▌        | 35773/225000 [78:39:51<296:01:16,  5.63s/it]
 16%|█▌        | 35774/225000 [78:39:56<289:52:21,  5.51s/it]
 16%|█▌        | 35775/225000 [78:40:02<292:50:59,  5.57s/it]
 16%|█▌        | 35776/225000 [78:40:07<284:32:51,  5.41s/it]
 16%|█▌        | 35777/225000 [78:40:17<354:34:17,  6.75s/it]
 16%|█▌        | 35778/225000 [78:40:29<448:42:41,  8.54s/it]
 16%|█▌        | 35779/225000 [78:40:33<374:43:16,  7.13s/it]
 16%|█▌        | 35780/225000 [78:40:37<328:47:49,  6.26s/it]
 16%|█▌        | 35781/225000 [78:40:48<397:05:13,  7.55s/it]
 16%|█▌        | 35782/225000 [78:40:53<361:10:14,  6.87s/it]
 16%|█▌        | 35783/225000 [78:40:59<341:32:35,  6.50s/it]
 16%|█▌        | 35784/225000 [78:41:04<318:34:12,  6.06s/it]
 16%|█▌        | 35785/225000 [78:41:10<319:12:30,  6.07s/it]
 16%|█▌        | 35786/225000 [78:41:23<427:05:02,  8.13s/it]
 16%|█▌        | 35787/225000 [78:41:27<371:58:00,  7.08s/it]
 16%|█▌        | 35788/225000 [78:41:33<341:03:43,  6.49s/it]
 16%|█▌        | 35789/225000 [78:41:38<317:39:58,  6.04s/it]
 16%|█▌        | 35790/225000 [78:41:43<303:52:18,  5.78s/it]
 16%|█▌        | 35791/225000 [78:41:48<287:18:46,  5.47s/it]
 16%|█▌        | 35792/225000 [78:41:58<361:25:48,  6.88s/it]
 16%|█▌        | 35793/225000 [78:42:07<392:22:35,  7.47s/it]
 16%|█▌        | 35794/225000 [78:42:13<371:31:50,  7.07s/it]
 16%|█▌        | 35795/225000 [78:42:23<419:44:54,  7.99s/it]
 16%|█▌        | 35796/225000 [78:42:26<348:40:07,  6.63s/it]
 16%|█▌        | 35797/225000 [78:42:35<384:53:03,  7.32s/it]
 16%|█▌        | 35798/225000 [78:42:41<361:17:07,  6.87s/it]
 16%|█▌        | 35799/225000 [78:42:47<344:02:49,  6.55s/it]
 16%|█▌        | 35800/225000 [78:42:52<328:00:37,  6.24s/it]
                                                             
{'loss': '0.7237', 'grad_norm': '0.7258', 'learning_rate': '4.825e-05', 'epoch': '0.1591', 'num_input_tokens_seen': 133767952, 'train_runtime': '2.834e+05', 'train_tokens_per_second': '472.1'}

 16%|█▌        | 35800/225000 [78:42:52<328:00:37,  6.24s/it]
 16%|█▌        | 35801/225000 [78:42:58<316:50:35,  6.03s/it]
 16%|█▌        | 35802/225000 [78:43:03<299:47:04,  5.70s/it]
 16%|█▌        | 35803/225000 [78:43:08<284:16:32,  5.41s/it]
 16%|█▌        | 35804/225000 [78:43:15<315:31:40,  6.00s/it]
 16%|█▌        | 35805/225000 [78:43:28<422:00:29,  8.03s/it]
 16%|█▌        | 35806/225000 [78:43:32<355:31:37,  6.76s/it]
 16%|█▌        | 35807/225000 [78:43:40<378:54:54,  7.21s/it]
 16%|█▌        | 35808/225000 [78:43:44<328:41:23,  6.25s/it]
 16%|█▌        | 35809/225000 [78:43:49<306:46:12,  5.84s/it]
 16%|█▌        | 35810/225000 [78:43:53<290:01:35,  5.52s/it]
 16%|█▌        | 35811/225000 [78:43:59<285:05:38,  5.42s/it]
 16%|█▌        | 35812/225000 [78:44:04<280:16:56,  5.33s/it]
 16%|█▌        | 35813/225000 [78:44:13<344:58:23,  6.56s/it]
 16%|█▌        | 35814/225000 [78:44:22<387:54:14,  7.38s/it]
 16%|█▌        | 35815/225000 [78:44:27<349:57:55,  6.66s/it]
 16%|█▌        | 35816/225000 [78:44:32<324:20:01,  6.17s/it]
 16%|█▌        | 35817/225000 [78:44:37<301:12:33,  5.73s/it]
 16%|█▌        | 35818/225000 [78:44:43<298:26:27,  5.68s/it]
 16%|█▌        | 35819/225000 [78:44:54<389:11:57,  7.41s/it]
 16%|█▌        | 35820/225000 [78:45:01<380:59:30,  7.25s/it]
 16%|█▌        | 35821/225000 [78:45:08<383:27:13,  7.30s/it]
 16%|█▌        | 35822/225000 [78:45:13<342:39:50,  6.52s/it]
 16%|█▌        | 35823/225000 [78:45:20<351:28:43,  6.69s/it]
 16%|█▌        | 35824/225000 [78:45:28<360:41:32,  6.86s/it]
 16%|█▌        | 35825/225000 [78:45:35<373:01:07,  7.10s/it]
 16%|█▌        | 35826/225000 [78:45:39<323:41:04,  6.16s/it]
 16%|█▌        | 35827/225000 [78:45:49<384:13:07,  7.31s/it]
 16%|█▌        | 35828/225000 [78:45:53<336:59:17,  6.41s/it]
 16%|█▌        | 35829/225000 [78:46:02<367:37:39,  7.00s/it]
 16%|█▌        | 35830/225000 [78:46:06<318:41:19,  6.06s/it]
 16%|█▌        | 35831/225000 [78:46:12<329:45:58,  6.28s/it]
 16%|█▌        | 35832/225000 [78:46:21<367:02:40,  6.99s/it]
 16%|█▌        | 35833/225000 [78:46:25<320:24:56,  6.10s/it]
 16%|█▌        | 35834/225000 [78:46:33<345:50:58,  6.58s/it]
 16%|█▌        | 35835/225000 [78:46:38<320:14:10,  6.09s/it]
 16%|█▌        | 35836/225000 [78:46:43<304:09:40,  5.79s/it]
 16%|█▌        | 35837/225000 [78:46:49<302:35:20,  5.76s/it]
 16%|█▌        | 35838/225000 [78:46:54<292:46:26,  5.57s/it]
 16%|█▌        | 35839/225000 [78:47:04<372:43:47,  7.09s/it]
 16%|█▌        | 35840/225000 [78:47:14<410:15:03,  7.81s/it]
 16%|█▌        | 35841/225000 [78:47:18<358:29:38,  6.82s/it]
 16%|█▌        | 35842/225000 [78:47:23<328:10:51,  6.25s/it]
 16%|█▌        | 35843/225000 [78:47:28<312:01:54,  5.94s/it]
 16%|█▌        | 35844/225000 [78:47:34<298:23:08,  5.68s/it]
 16%|█▌        | 35845/225000 [78:47:46<408:30:36,  7.77s/it]
 16%|█▌        | 35846/225000 [78:47:52<379:03:49,  7.21s/it]
 16%|█▌        | 35847/225000 [78:47:57<340:54:53,  6.49s/it]
 16%|█▌        | 35848/225000 [78:48:03<332:12:11,  6.32s/it]
 16%|█▌        | 35849/225000 [78:48:08<317:31:56,  6.04s/it]
 16%|█▌        | 35850/225000 [78:48:14<312:39:18,  5.95s/it]
                                                             
{'loss': '0.7273', 'grad_norm': '0.6584', 'learning_rate': '4.824e-05', 'epoch': '0.1593', 'num_input_tokens_seen': 133914016, 'train_runtime': '2.837e+05', 'train_tokens_per_second': '472'}

 16%|█▌        | 35850/225000 [78:48:14<312:39:18,  5.95s/it]
 16%|█▌        | 35851/225000 [78:48:19<299:22:17,  5.70s/it]
 16%|█▌        | 35852/225000 [78:48:29<367:36:51,  7.00s/it]
 16%|█▌        | 35853/225000 [78:48:40<431:23:13,  8.21s/it]
 16%|█▌        | 35854/225000 [78:48:44<356:21:08,  6.78s/it]
 16%|█▌        | 35855/225000 [78:48:52<389:30:22,  7.41s/it]
 16%|█▌        | 35856/225000 [78:48:56<334:47:10,  6.37s/it]
 16%|█▌        | 35857/225000 [78:49:02<326:39:22,  6.22s/it]
 16%|█▌        | 35858/225000 [78:49:12<385:24:05,  7.34s/it]
 16%|█▌        | 35859/225000 [78:49:17<341:39:05,  6.50s/it]
 16%|█▌        | 35860/225000 [78:49:22<316:10:21,  6.02s/it]
 16%|█▌        | 35861/225000 [78:49:26<288:36:03,  5.49s/it]
 16%|█▌        | 35862/225000 [78:49:39<405:53:59,  7.73s/it]
 16%|█▌        | 35863/225000 [78:49:43<350:35:20,  6.67s/it]
 16%|█▌        | 35864/225000 [78:49:51<375:23:34,  7.15s/it]
 16%|█▌        | 35865/225000 [78:49:56<339:02:04,  6.45s/it]
 16%|█▌        | 35866/225000 [78:50:01<306:28:07,  5.83s/it]
 16%|█▌        | 35867/225000 [78:50:10<359:51:40,  6.85s/it]
 16%|█▌        | 35868/225000 [78:50:14<320:21:25,  6.10s/it]
 16%|█▌        | 35869/225000 [78:50:19<295:45:42,  5.63s/it]
 16%|█▌        | 35870/225000 [78:50:30<386:49:42,  7.36s/it]
 16%|█▌        | 35871/225000 [78:50:34<328:31:36,  6.25s/it]
 16%|█▌        | 35872/225000 [78:50:41<351:44:08,  6.70s/it]
 16%|█▌        | 35873/225000 [78:50:47<326:40:12,  6.22s/it]
 16%|█▌        | 35874/225000 [78:50:51<298:58:01,  5.69s/it]
 16%|█▌        | 35875/225000 [78:51:02<382:48:27,  7.29s/it]
 16%|█▌        | 35876/225000 [78:51:06<334:46:38,  6.37s/it]
 16%|█▌        | 35877/225000 [78:51:11<302:41:25,  5.76s/it]
 16%|█▌        | 35878/225000 [78:51:15<281:28:50,  5.36s/it]
 16%|█▌        | 35879/225000 [78:51:24<342:30:56,  6.52s/it]
 16%|█▌        | 35880/225000 [78:51:33<377:39:54,  7.19s/it]
 16%|█▌        | 35881/225000 [78:51:37<326:26:21,  6.21s/it]
 16%|█▌        | 35882/225000 [78:51:46<372:14:02,  7.09s/it]
 16%|█▌        | 35883/225000 [78:51:51<340:17:58,  6.48s/it]
 16%|█▌        | 35884/225000 [78:51:56<312:37:47,  5.95s/it]
 16%|█▌        | 35885/225000 [78:52:00<287:56:50,  5.48s/it]
 16%|█▌        | 35886/225000 [78:52:06<292:34:07,  5.57s/it]
 16%|█▌        | 35887/225000 [78:52:19<412:14:35,  7.85s/it]
 16%|█▌        | 35888/225000 [78:52:25<378:21:39,  7.20s/it]
 16%|█▌        | 35889/225000 [78:52:32<381:10:20,  7.26s/it]
 16%|█▌        | 35890/225000 [78:52:36<325:40:24,  6.20s/it]
 16%|█▌        | 35891/225000 [78:52:47<394:29:52,  7.51s/it]
 16%|█▌        | 35892/225000 [78:52:51<341:34:24,  6.50s/it]
 16%|█▌        | 35893/225000 [78:52:59<361:45:43,  6.89s/it]
 16%|█▌        | 35894/225000 [78:53:05<358:20:11,  6.82s/it]
 16%|█▌        | 35895/225000 [78:53:10<330:18:49,  6.29s/it]
 16%|█▌        | 35896/225000 [78:53:15<303:41:04,  5.78s/it]
 16%|█▌        | 35897/225000 [78:53:24<358:56:14,  6.83s/it]
 16%|█▌        | 35898/225000 [78:53:31<357:59:26,  6.82s/it]
 16%|█▌        | 35899/225000 [78:53:39<383:34:09,  7.30s/it]
 16%|█▌        | 35900/225000 [78:53:43<334:24:29,  6.37s/it]
                                                             
{'loss': '0.7267', 'grad_norm': '0.6593', 'learning_rate': '4.823e-05', 'epoch': '0.1596', 'num_input_tokens_seen': 134058912, 'train_runtime': '2.84e+05', 'train_tokens_per_second': '472'}

 16%|█▌        | 35900/225000 [78:53:43<334:24:29,  6.37s/it]
 16%|█▌        | 35901/225000 [78:53:47<293:30:38,  5.59s/it]
 16%|█▌        | 35902/225000 [78:53:56<350:53:19,  6.68s/it]
 16%|█▌        | 35903/225000 [78:54:03<343:00:03,  6.53s/it]
 16%|█▌        | 35904/225000 [78:54:10<357:45:04,  6.81s/it]
 16%|█▌        | 35905/225000 [78:54:16<350:35:36,  6.67s/it]
 16%|█▌        | 35906/225000 [78:54:21<323:27:13,  6.16s/it]
 16%|█▌        | 35907/225000 [78:54:29<338:36:45,  6.45s/it]
 16%|█▌        | 35908/225000 [78:54:36<346:41:11,  6.60s/it]
 16%|█▌        | 35909/225000 [78:54:40<318:08:40,  6.06s/it]
 16%|█▌        | 35910/225000 [78:54:45<299:30:11,  5.70s/it]
 16%|█▌        | 35911/225000 [78:54:50<280:12:17,  5.33s/it]
 16%|█▌        | 35912/225000 [78:54:58<324:44:47,  6.18s/it]
 16%|█▌        | 35913/225000 [78:55:07<373:12:39,  7.11s/it]
 16%|█▌        | 35914/225000 [78:55:11<317:06:58,  6.04s/it]
 16%|█▌        | 35915/225000 [78:55:19<360:33:20,  6.86s/it]
 16%|█▌        | 35916/225000 [78:55:24<323:18:30,  6.16s/it]
 16%|█▌        | 35917/225000 [78:55:29<300:44:37,  5.73s/it]
 16%|█▌        | 35918/225000 [78:55:34<287:19:34,  5.47s/it]
 16%|█▌        | 35919/225000 [78:55:39<281:39:03,  5.36s/it]
 16%|█▌        | 35920/225000 [78:55:52<402:39:49,  7.67s/it]
 16%|█▌        | 35921/225000 [78:55:56<343:28:34,  6.54s/it]
 16%|█▌        | 35922/225000 [78:56:04<373:16:01,  7.11s/it]
 16%|█▌        | 35923/225000 [78:56:09<333:30:22,  6.35s/it]
 16%|█▌        | 35924/225000 [78:56:17<359:04:39,  6.84s/it]
 16%|█▌        | 35925/225000 [78:56:20<310:22:52,  5.91s/it]
 16%|█▌        | 35926/225000 [78:56:28<335:55:34,  6.40s/it]
 16%|█▌        | 35927/225000 [78:56:33<313:55:50,  5.98s/it]
 16%|█▌        | 35928/225000 [78:56:44<391:15:47,  7.45s/it]
 16%|█▌        | 35929/225000 [78:56:47<328:25:16,  6.25s/it]
 16%|█▌        | 35930/225000 [78:56:56<367:34:18,  7.00s/it]
 16%|█▌        | 35931/225000 [78:56:59<308:28:56,  5.87s/it]
 16%|█▌        | 35932/225000 [78:57:07<337:54:32,  6.43s/it]
 16%|█▌        | 35933/225000 [78:57:13<338:26:34,  6.44s/it]
 16%|█▌        | 35934/225000 [78:57:18<302:54:29,  5.77s/it]
 16%|█▌        | 35935/225000 [78:57:26<344:28:21,  6.56s/it]
 16%|█▌        | 35936/225000 [78:57:34<360:47:24,  6.87s/it]
 16%|█▌        | 35937/225000 [78:57:42<377:38:48,  7.19s/it]
 16%|█▌        | 35938/225000 [78:57:49<376:59:29,  7.18s/it]
 16%|█▌        | 35939/225000 [78:57:53<324:33:31,  6.18s/it]
 16%|█▌        | 35940/225000 [78:58:01<358:18:09,  6.82s/it]
 16%|█▌        | 35941/225000 [78:58:06<326:36:56,  6.22s/it]
 16%|█▌        | 35942/225000 [78:58:14<366:06:52,  6.97s/it]
 16%|█▌        | 35943/225000 [78:58:19<321:43:51,  6.13s/it]
 16%|█▌        | 35944/225000 [78:58:25<327:49:34,  6.24s/it]
 16%|█▌        | 35945/225000 [78:58:35<380:02:42,  7.24s/it]
 16%|█▌        | 35946/225000 [78:58:38<318:54:10,  6.07s/it]
 16%|█▌        | 35947/225000 [78:58:48<381:31:30,  7.27s/it]
 16%|█▌        | 35948/225000 [78:58:52<334:12:53,  6.36s/it]
 16%|█▌        | 35949/225000 [78:58:58<326:47:45,  6.22s/it]
 16%|█▌        | 35950/225000 [78:59:09<401:30:00,  7.65s/it]
                                                             
{'loss': '0.7278', 'grad_norm': '0.7365', 'learning_rate': '4.823e-05', 'epoch': '0.1598', 'num_input_tokens_seen': 134203320, 'train_runtime': '2.843e+05', 'train_tokens_per_second': '472'}

 16%|█▌        | 35950/225000 [78:59:09<401:30:00,  7.65s/it]
 16%|█▌        | 35951/225000 [78:59:13<344:20:17,  6.56s/it]
 16%|█▌        | 35952/225000 [78:59:18<310:11:25,  5.91s/it]
 16%|█▌        | 35953/225000 [78:59:22<294:59:15,  5.62s/it]
 16%|█▌        | 35954/225000 [78:59:34<392:51:41,  7.48s/it]
 16%|█▌        | 35955/225000 [78:59:39<352:46:32,  6.72s/it]
 16%|█▌        | 35956/225000 [78:59:45<333:34:52,  6.35s/it]
 16%|█▌        | 35957/225000 [78:59:50<320:21:43,  6.10s/it]
 16%|█▌        | 35958/225000 [78:59:59<369:01:06,  7.03s/it]
 16%|█▌        | 35959/225000 [79:00:04<328:52:09,  6.26s/it]
 16%|█▌        | 35960/225000 [79:00:09<304:07:21,  5.79s/it]
 16%|█▌        | 35961/225000 [79:00:13<289:10:29,  5.51s/it]
 16%|█▌        | 35962/225000 [79:00:19<284:13:39,  5.41s/it]
 16%|█▌        | 35963/225000 [79:00:30<375:10:47,  7.14s/it]
 16%|█▌        | 35964/225000 [79:00:38<390:18:04,  7.43s/it]
 16%|█▌        | 35965/225000 [79:00:42<341:40:29,  6.51s/it]
 16%|█▌        | 35966/225000 [79:00:47<319:23:36,  6.08s/it]
 16%|█▌        | 35967/225000 [79:00:53<304:37:28,  5.80s/it]
 16%|█▌        | 35968/225000 [79:00:58<292:24:52,  5.57s/it]
 16%|█▌        | 35969/225000 [79:01:03<294:55:15,  5.62s/it]
 16%|█▌        | 35970/225000 [79:01:09<288:31:31,  5.49s/it]
 16%|█▌        | 35971/225000 [79:01:14<289:30:17,  5.51s/it]
 16%|█▌        | 35972/225000 [79:01:20<295:15:20,  5.62s/it]
 16%|█▌        | 35973/225000 [79:01:33<417:12:07,  7.95s/it]
 16%|█▌        | 35974/225000 [79:01:40<395:55:26,  7.54s/it]
 16%|█▌        | 35975/225000 [79:01:45<360:50:18,  6.87s/it]
 16%|█▌        | 35976/225000 [79:01:54<389:42:24,  7.42s/it]
 16%|█▌        | 35977/225000 [79:01:59<347:33:23,  6.62s/it]
 16%|█▌        | 35978/225000 [79:02:04<319:43:31,  6.09s/it]
 16%|█▌        | 35979/225000 [79:02:08<297:46:48,  5.67s/it]
 16%|█▌        | 35980/225000 [79:02:13<285:02:48,  5.43s/it]
 16%|█▌        | 35981/225000 [79:02:20<304:15:42,  5.79s/it]
 16%|█▌        | 35982/225000 [79:02:32<411:41:21,  7.84s/it]
 16%|█▌        | 35983/225000 [79:02:36<344:04:11,  6.55s/it]
 16%|█▌        | 35984/225000 [79:02:41<318:15:18,  6.06s/it]
 16%|█▌        | 35985/225000 [79:02:52<401:30:32,  7.65s/it]
 16%|█▌        | 35986/225000 [79:02:56<349:06:19,  6.65s/it]
 16%|█▌        | 35987/225000 [79:03:01<313:05:08,  5.96s/it]
 16%|█▌        | 35988/225000 [79:03:09<353:13:53,  6.73s/it]
 16%|█▌        | 35989/225000 [79:03:17<375:42:36,  7.16s/it]
 16%|█▌        | 35990/225000 [79:03:22<331:00:34,  6.30s/it]
 16%|█▌        | 35991/225000 [79:03:31<382:00:24,  7.28s/it]
 16%|█▌        | 35992/225000 [79:03:37<357:36:34,  6.81s/it]
 16%|█▌        | 35993/225000 [79:03:42<328:00:10,  6.25s/it]
 16%|█▌        | 35994/225000 [79:03:47<300:22:20,  5.72s/it]
 16%|█▌        | 35995/225000 [79:03:51<282:52:35,  5.39s/it]
 16%|█▌        | 35996/225000 [79:04:02<361:33:48,  6.89s/it]
 16%|█▌        | 35997/225000 [79:04:10<392:14:24,  7.47s/it]
 16%|█▌        | 35998/225000 [79:04:15<345:17:28,  6.58s/it]
 16%|█▌        | 35999/225000 [79:04:20<316:59:52,  6.04s/it]
 16%|█▌        | 36000/225000 [79:04:25<302:20:17,  5.76s/it]
                                                             
{'loss': '0.7048', 'grad_norm': '0.7074', 'learning_rate': '4.822e-05', 'epoch': '0.16', 'num_input_tokens_seen': 134346928, 'train_runtime': '2.847e+05', 'train_tokens_per_second': '471.9'}

 16%|█▌        | 36000/225000 [79:04:25<302:20:17,  5.76s/it]
 16%|█▌        | 36001/225000 [79:04:29<285:33:07,  5.44s/it]
 16%|█▌        | 36002/225000 [79:04:35<284:30:13,  5.42s/it]
 16%|█▌        | 36003/225000 [79:04:47<388:01:10,  7.39s/it]
 16%|█▌        | 36004/225000 [79:04:54<380:13:56,  7.24s/it]
 16%|█▌        | 36005/225000 [79:05:00<365:28:58,  6.96s/it]
 16%|█▌        | 36006/225000 [79:05:04<323:59:40,  6.17s/it]
 16%|█▌        | 36007/225000 [79:05:09<293:45:16,  5.60s/it]
 16%|█▌        | 36008/225000 [79:05:14<297:32:59,  5.67s/it]
 16%|█▌        | 36009/225000 [79:05:25<375:36:53,  7.15s/it]
 16%|█▌        | 36010/225000 [79:05:30<341:44:25,  6.51s/it]
 16%|█▌        | 36011/225000 [79:05:35<319:17:19,  6.08s/it]
 16%|█▌        | 36012/225000 [79:05:48<428:44:22,  8.17s/it]
 16%|█▌        | 36013/225000 [79:05:53<373:05:20,  7.11s/it]
 16%|█▌        | 36014/225000 [79:05:58<336:43:10,  6.41s/it]
 16%|█▌        | 36015/225000 [79:06:02<306:19:53,  5.84s/it]
 16%|█▌        | 36016/225000 [79:06:13<390:13:08,  7.43s/it]
 16%|█▌        | 36017/225000 [79:06:17<328:22:48,  6.26s/it]
 16%|█▌        | 36018/225000 [79:06:26<383:19:16,  7.30s/it]
 16%|█▌        | 36019/225000 [79:06:31<333:00:20,  6.34s/it]
 16%|█▌        | 36020/225000 [79:06:36<325:43:48,  6.21s/it]
 16%|█▌        | 36021/225000 [79:06:44<349:38:49,  6.66s/it]
 16%|█▌        | 36022/225000 [79:06:48<300:56:56,  5.73s/it]
 16%|█▌        | 36023/225000 [79:06:56<342:40:43,  6.53s/it]
 16%|█▌        | 36024/225000 [79:07:00<306:44:27,  5.84s/it]
 16%|█▌        | 36025/225000 [79:07:05<288:51:16,  5.50s/it]
 16%|█▌        | 36026/225000 [79:07:17<397:22:40,  7.57s/it]
 16%|█▌        | 36027/225000 [79:07:21<332:13:29,  6.33s/it]
 16%|█▌        | 36028/225000 [79:07:30<375:56:44,  7.16s/it]
 16%|█▌        | 36029/225000 [79:07:34<332:28:26,  6.33s/it]
 16%|█▌        | 36030/225000 [79:07:39<303:45:36,  5.79s/it]
 16%|█▌        | 36031/225000 [79:07:48<362:18:54,  6.90s/it]
 16%|█▌        | 36032/225000 [79:07:56<377:45:11,  7.20s/it]
 16%|█▌        | 36033/225000 [79:08:01<342:54:40,  6.53s/it]
 16%|█▌        | 36034/225000 [79:08:06<309:49:32,  5.90s/it]
 16%|█▌        | 36035/225000 [79:08:14<339:09:40,  6.46s/it]
 16%|█▌        | 36036/225000 [79:08:22<372:59:02,  7.11s/it]
 16%|█▌        | 36037/225000 [79:08:29<372:51:07,  7.10s/it]
 16%|█▌        | 36038/225000 [79:08:34<340:44:00,  6.49s/it]
 16%|█▌        | 36039/225000 [79:08:38<304:19:23,  5.80s/it]
 16%|█▌        | 36040/225000 [79:08:45<309:55:57,  5.90s/it]
 16%|█▌        | 36041/225000 [79:08:56<391:01:26,  7.45s/it]
 16%|█▌        | 36042/225000 [79:09:00<343:36:01,  6.55s/it]
 16%|█▌        | 36043/225000 [79:09:05<314:22:30,  5.99s/it]
 16%|█▌        | 36044/225000 [79:09:11<319:05:36,  6.08s/it]
 16%|█▌        | 36045/225000 [79:09:23<403:22:13,  7.69s/it]
 16%|█▌        | 36046/225000 [79:09:26<342:29:44,  6.53s/it]
 16%|█▌        | 36047/225000 [79:09:35<375:15:43,  7.15s/it]
 16%|█▌        | 36048/225000 [79:09:41<359:08:08,  6.84s/it]
 16%|█▌        | 36049/225000 [79:09:46<333:32:05,  6.35s/it]
 16%|█▌        | 36050/225000 [79:09:51<313:04:18,  5.96s/it]
                                                             
{'loss': '0.7273', 'grad_norm': '0.7345', 'learning_rate': '4.821e-05', 'epoch': '0.1602', 'num_input_tokens_seen': 134492656, 'train_runtime': '2.85e+05', 'train_tokens_per_second': '471.9'}

 16%|█▌        | 36050/225000 [79:09:51<313:04:18,  5.96s/it]
 16%|█▌        | 36051/225000 [79:09:56<294:15:07,  5.61s/it]
 16%|█▌        | 36052/225000 [79:10:01<277:12:58,  5.28s/it]
 16%|█▌        | 36053/225000 [79:10:12<369:26:42,  7.04s/it]
 16%|█▌        | 36054/225000 [79:10:20<382:42:23,  7.29s/it]
 16%|█▌        | 36055/225000 [79:10:24<330:53:00,  6.30s/it]
 16%|█▌        | 36056/225000 [79:10:33<380:24:49,  7.25s/it]
 16%|█▌        | 36057/225000 [79:10:39<364:12:56,  6.94s/it]
 16%|█▌        | 36058/225000 [79:10:47<376:43:36,  7.18s/it]
 16%|█▌        | 36059/225000 [79:10:51<320:01:57,  6.10s/it]
 16%|█▌        | 36060/225000 [79:11:00<366:01:00,  6.97s/it]
 16%|█▌        | 36061/225000 [79:11:06<354:29:49,  6.75s/it]
 16%|█▌        | 36062/225000 [79:11:10<305:08:09,  5.81s/it]
 16%|█▌        | 36063/225000 [79:11:19<366:28:58,  6.98s/it]
 16%|█▌        | 36064/225000 [79:11:23<318:25:30,  6.07s/it]
 16%|█▌        | 36065/225000 [79:11:34<401:09:36,  7.64s/it]
 16%|█▌        | 36066/225000 [79:11:38<341:12:14,  6.50s/it]
 16%|█▌        | 36067/225000 [79:11:48<384:45:06,  7.33s/it]
 16%|█▌        | 36068/225000 [79:11:52<345:26:39,  6.58s/it]
 16%|█▌        | 36069/225000 [79:12:02<396:11:41,  7.55s/it]
 16%|█▌        | 36070/225000 [79:12:06<329:53:08,  6.29s/it]
 16%|█▌        | 36071/225000 [79:12:10<302:15:28,  5.76s/it]
 16%|█▌        | 36072/225000 [79:12:15<296:15:47,  5.65s/it]
 16%|█▌        | 36073/225000 [79:12:20<284:12:26,  5.42s/it]
 16%|█▌        | 36074/225000 [79:12:32<390:17:36,  7.44s/it]
 16%|█▌        | 36075/225000 [79:12:38<356:25:19,  6.79s/it]
 16%|█▌        | 36076/225000 [79:12:44<341:19:55,  6.50s/it]
 16%|█▌        | 36077/225000 [79:12:49<323:15:19,  6.16s/it]
 16%|█▌        | 36078/225000 [79:12:54<307:58:28,  5.87s/it]
 16%|█▌        | 36079/225000 [79:13:00<300:40:10,  5.73s/it]
 16%|█▌        | 36080/225000 [79:13:05<298:19:55,  5.68s/it]
 16%|█▌        | 36081/225000 [79:13:11<297:51:57,  5.68s/it]
 16%|█▌        | 36082/225000 [79:13:16<297:40:45,  5.67s/it]
 16%|█▌        | 36083/225000 [79:13:22<292:07:43,  5.57s/it]
 16%|█▌        | 36084/225000 [79:13:34<402:09:14,  7.66s/it]
 16%|█▌        | 36085/225000 [79:13:40<371:43:43,  7.08s/it]
 16%|█▌        | 36086/225000 [79:13:44<329:48:26,  6.28s/it]
 16%|█▌        | 36087/225000 [79:13:50<310:51:07,  5.92s/it]
 16%|█▌        | 36088/225000 [79:14:01<393:45:12,  7.50s/it]
 16%|█▌        | 36089/225000 [79:14:05<346:36:34,  6.61s/it]
 16%|█▌        | 36090/225000 [79:14:14<381:41:44,  7.27s/it]
 16%|█▌        | 36091/225000 [79:14:19<338:15:25,  6.45s/it]
 16%|█▌        | 36092/225000 [79:14:26<356:54:14,  6.80s/it]
 16%|█▌        | 36093/225000 [79:14:34<372:41:10,  7.10s/it]
 16%|█▌        | 36094/225000 [79:14:41<373:01:09,  7.11s/it]
 16%|█▌        | 36095/225000 [79:14:46<333:02:44,  6.35s/it]
 16%|█▌        | 36096/225000 [79:14:50<304:31:14,  5.80s/it]
 16%|█▌        | 36097/225000 [79:14:55<288:14:29,  5.49s/it]
 16%|█▌        | 36098/225000 [79:15:03<326:05:40,  6.21s/it]
 16%|█▌        | 36099/225000 [79:15:12<374:33:41,  7.14s/it]
 16%|█▌        | 36100/225000 [79:15:21<401:05:40,  7.64s/it]
                                                             
{'loss': '0.7285', 'grad_norm': '0.7729', 'learning_rate': '4.821e-05', 'epoch': '0.1604', 'num_input_tokens_seen': 134638424, 'train_runtime': '2.853e+05', 'train_tokens_per_second': '471.9'}

 16%|█▌        | 36100/225000 [79:15:21<401:05:40,  7.64s/it]
 16%|█▌        | 36101/225000 [79:15:25<343:07:58,  6.54s/it]
 16%|█▌        | 36102/225000 [79:15:29<302:19:06,  5.76s/it]
 16%|█▌        | 36103/225000 [79:15:40<379:15:31,  7.23s/it]
 16%|█▌        | 36104/225000 [79:15:45<342:52:13,  6.53s/it]
 16%|█▌        | 36105/225000 [79:15:49<308:08:22,  5.87s/it]
 16%|█▌        | 36106/225000 [79:15:54<290:54:25,  5.54s/it]
 16%|█▌        | 36107/225000 [79:15:59<283:41:54,  5.41s/it]
 16%|█▌        | 36108/225000 [79:16:07<336:18:00,  6.41s/it]
 16%|█▌        | 36109/225000 [79:16:17<381:52:49,  7.28s/it]
 16%|█▌        | 36110/225000 [79:16:21<330:07:38,  6.29s/it]
 16%|█▌        | 36111/225000 [79:16:31<389:23:30,  7.42s/it]
 16%|█▌        | 36112/225000 [79:16:37<371:07:57,  7.07s/it]
 16%|█▌        | 36113/225000 [79:16:42<333:34:52,  6.36s/it]
 16%|█▌        | 36114/225000 [79:16:46<303:20:10,  5.78s/it]
 16%|█▌        | 36115/225000 [79:16:56<358:18:03,  6.83s/it]
 16%|█▌        | 36116/225000 [79:17:02<349:56:42,  6.67s/it]
 16%|█▌        | 36117/225000 [79:17:11<383:36:52,  7.31s/it]
 16%|█▌        | 36118/225000 [79:17:15<333:55:32,  6.36s/it]
 16%|█▌        | 36119/225000 [79:17:20<311:10:09,  5.93s/it]
 16%|█▌        | 36120/225000 [79:17:31<398:04:29,  7.59s/it]
 16%|█▌        | 36121/225000 [79:17:34<329:49:31,  6.29s/it]
 16%|█▌        | 36122/225000 [79:17:38<290:14:02,  5.53s/it]
 16%|█▌        | 36123/225000 [79:17:43<278:44:14,  5.31s/it]
 16%|█▌        | 36124/225000 [79:17:52<339:11:17,  6.46s/it]
 16%|█▌        | 36125/225000 [79:17:57<317:54:40,  6.06s/it]
 16%|█▌        | 36126/225000 [79:18:02<297:52:56,  5.68s/it]
 16%|█▌        | 36127/225000 [79:18:13<389:08:54,  7.42s/it]
 16%|█▌        | 36128/225000 [79:18:19<359:39:04,  6.86s/it]
 16%|█▌        | 36129/225000 [79:18:23<321:44:19,  6.13s/it]
 16%|█▌        | 36130/225000 [79:18:28<295:56:45,  5.64s/it]
 16%|█▌        | 36131/225000 [79:18:39<387:25:50,  7.38s/it]
 16%|█▌        | 36132/225000 [79:18:44<344:37:33,  6.57s/it]
 16%|█▌        | 36133/225000 [79:18:49<323:51:58,  6.17s/it]
 16%|█▌        | 36134/225000 [79:18:54<307:21:19,  5.86s/it]
 16%|█▌        | 36135/225000 [79:19:04<369:41:28,  7.05s/it]
 16%|█▌        | 36136/225000 [79:19:12<374:28:40,  7.14s/it]
 16%|█▌        | 36137/225000 [79:19:16<328:31:18,  6.26s/it]
 16%|█▌        | 36138/225000 [79:19:21<308:12:28,  5.87s/it]
 16%|█▌        | 36139/225000 [79:19:25<289:09:34,  5.51s/it]
 16%|█▌        | 36140/225000 [79:19:36<371:17:14,  7.08s/it]
 16%|█▌        | 36141/225000 [79:19:43<372:55:31,  7.11s/it]
 16%|█▌        | 36142/225000 [79:19:50<359:17:01,  6.85s/it]
 16%|█▌        | 36143/225000 [79:19:56<357:31:07,  6.82s/it]
 16%|█▌        | 36144/225000 [79:20:01<328:52:51,  6.27s/it]
 16%|█▌        | 36145/225000 [79:20:06<304:55:49,  5.81s/it]
 16%|█▌        | 36146/225000 [79:20:16<373:16:47,  7.12s/it]
 16%|█▌        | 36147/225000 [79:20:24<382:41:08,  7.29s/it]
 16%|█▌        | 36148/225000 [79:20:30<357:06:17,  6.81s/it]
 16%|█▌        | 36149/225000 [79:20:35<328:35:37,  6.26s/it]
 16%|█▌        | 36150/225000 [79:20:39<300:41:58,  5.73s/it]
                                                             
{'loss': '0.726', 'grad_norm': '0.7537', 'learning_rate': '4.82e-05', 'epoch': '0.1607', 'num_input_tokens_seen': 134782944, 'train_runtime': '2.856e+05', 'train_tokens_per_second': '471.9'}

 16%|█▌        | 36150/225000 [79:20:39<300:41:58,  5.73s/it]
 16%|█▌        | 36151/225000 [79:20:44<281:31:26,  5.37s/it]
 16%|█▌        | 36152/225000 [79:20:49<273:20:00,  5.21s/it]
 16%|█▌        | 36153/225000 [79:20:53<266:38:49,  5.08s/it]
 16%|█▌        | 36154/225000 [79:20:59<279:55:58,  5.34s/it]
 16%|█▌        | 36155/225000 [79:21:11<380:33:24,  7.25s/it]
 16%|█▌        | 36156/225000 [79:21:21<418:05:35,  7.97s/it]
 16%|█▌        | 36157/225000 [79:21:28<407:41:40,  7.77s/it]
 16%|█▌        | 36158/225000 [79:21:32<350:28:48,  6.68s/it]
 16%|█▌        | 36159/225000 [79:21:36<312:26:44,  5.96s/it]
 16%|█▌        | 36160/225000 [79:21:46<366:52:48,  6.99s/it]
 16%|█▌        | 36161/225000 [79:21:52<357:43:44,  6.82s/it]
 16%|█▌        | 36162/225000 [79:22:00<381:23:32,  7.27s/it]
 16%|█▌        | 36163/225000 [79:22:06<358:48:20,  6.84s/it]
 16%|█▌        | 36164/225000 [79:22:11<328:39:42,  6.27s/it]
 16%|█▌        | 36165/225000 [79:22:16<312:34:15,  5.96s/it]
 16%|█▌        | 36166/225000 [79:22:21<296:22:43,  5.65s/it]
 16%|█▌        | 36167/225000 [79:22:26<272:17:53,  5.19s/it]
 16%|█▌        | 36168/225000 [79:22:32<297:59:34,  5.68s/it]
 16%|█▌        | 36169/225000 [79:22:46<424:10:46,  8.09s/it]
 16%|█▌        | 36170/225000 [79:22:50<352:53:54,  6.73s/it]
 16%|█▌        | 36171/225000 [79:22:58<380:19:04,  7.25s/it]
 16%|█▌        | 36172/225000 [79:23:02<332:26:17,  6.34s/it]
 16%|█▌        | 36173/225000 [79:23:10<360:15:19,  6.87s/it]
 16%|█▌        | 36174/225000 [79:23:18<379:11:41,  7.23s/it]
 16%|█▌        | 36175/225000 [79:23:23<340:01:05,  6.48s/it]
 16%|█▌        | 36176/225000 [79:23:28<314:12:54,  5.99s/it]
 16%|█▌        | 36177/225000 [79:23:32<286:53:30,  5.47s/it]
 16%|█▌        | 36178/225000 [79:23:36<263:45:12,  5.03s/it]
 16%|█▌        | 36179/225000 [79:23:47<351:59:32,  6.71s/it]
 16%|█▌        | 36180/225000 [79:23:56<392:23:41,  7.48s/it]
 16%|█▌        | 36181/225000 [79:24:00<333:44:46,  6.36s/it]
 16%|█▌        | 36182/225000 [79:24:11<401:25:16,  7.65s/it]
 16%|█▌        | 36183/225000 [79:24:18<395:20:23,  7.54s/it]
 16%|█▌        | 36184/225000 [79:24:23<349:28:42,  6.66s/it]
 16%|█▌        | 36185/225000 [79:24:27<309:53:27,  5.91s/it]
 16%|█▌        | 36186/225000 [79:24:34<339:32:57,  6.47s/it]
 16%|█▌        | 36187/225000 [79:24:42<362:39:41,  6.91s/it]
 16%|█▌        | 36188/225000 [79:24:47<319:31:35,  6.09s/it]
 16%|█▌        | 36189/225000 [79:24:56<375:24:45,  7.16s/it]
 16%|█▌        | 36190/225000 [79:25:00<318:11:36,  6.07s/it]
 16%|█▌        | 36191/225000 [79:25:09<363:08:10,  6.92s/it]
 16%|█▌        | 36192/225000 [79:25:14<345:25:58,  6.59s/it]
 16%|█▌        | 36193/225000 [79:25:23<376:00:42,  7.17s/it]
 16%|█▌        | 36194/225000 [79:25:27<327:51:38,  6.25s/it]
 16%|█▌        | 36195/225000 [79:25:35<356:52:28,  6.80s/it]
 16%|█▌        | 36196/225000 [79:25:40<331:17:37,  6.32s/it]
 16%|█▌        | 36197/225000 [79:25:45<307:07:33,  5.86s/it]
 16%|█▌        | 36198/225000 [79:25:50<293:21:57,  5.59s/it]
 16%|█▌        | 36199/225000 [79:25:55<279:33:48,  5.33s/it]
 16%|█▌        | 36200/225000 [79:26:00<282:30:22,  5.39s/it]
                                                             
{'loss': '0.7178', 'grad_norm': '0.9571', 'learning_rate': '4.819e-05', 'epoch': '0.1609', 'num_input_tokens_seen': 134927152, 'train_runtime': '2.86e+05', 'train_tokens_per_second': '471.8'}

 16%|█▌        | 36200/225000 [79:26:00<282:30:22,  5.39s/it]
 16%|█▌        | 36201/225000 [79:26:12<375:13:35,  7.15s/it]
 16%|█▌        | 36202/225000 [79:26:16<337:09:42,  6.43s/it]
 16%|█▌        | 36203/225000 [79:26:21<312:43:57,  5.96s/it]
 16%|█▌        | 36204/225000 [79:26:26<292:52:24,  5.58s/it]
 16%|█▌        | 36205/225000 [79:26:34<335:34:55,  6.40s/it]
 16%|█▌        | 36206/225000 [79:26:45<400:08:23,  7.63s/it]
 16%|█▌        | 36207/225000 [79:26:52<387:18:34,  7.39s/it]
 16%|█▌        | 36208/225000 [79:26:56<346:38:42,  6.61s/it]
 16%|█▌        | 36209/225000 [79:27:01<320:27:38,  6.11s/it]
 16%|█▌        | 36210/225000 [79:27:07<306:10:04,  5.84s/it]
 16%|█▌        | 36211/225000 [79:27:17<374:47:45,  7.15s/it]
 16%|█▌        | 36212/225000 [79:27:26<412:29:39,  7.87s/it]
 16%|█▌        | 36213/225000 [79:27:33<393:16:21,  7.50s/it]
 16%|█▌        | 36214/225000 [79:27:37<343:48:41,  6.56s/it]
 16%|█▌        | 36215/225000 [79:27:45<356:31:37,  6.80s/it]
 16%|█▌        | 36216/225000 [79:27:54<390:18:39,  7.44s/it]
 16%|█▌        | 36217/225000 [79:27:57<331:30:28,  6.32s/it]
 16%|█▌        | 36218/225000 [79:28:06<362:45:05,  6.92s/it]
 16%|█▌        | 36219/225000 [79:28:14<383:01:59,  7.30s/it]
 16%|█▌        | 36220/225000 [79:28:18<339:42:32,  6.48s/it]
 16%|█▌        | 36221/225000 [79:28:22<303:00:38,  5.78s/it]
 16%|█▌        | 36222/225000 [79:28:27<283:17:51,  5.40s/it]
 16%|█▌        | 36223/225000 [79:28:39<390:38:52,  7.45s/it]
 16%|█▌        | 36224/225000 [79:28:44<341:58:07,  6.52s/it]
 16%|█▌        | 36225/225000 [79:28:53<388:59:05,  7.42s/it]
 16%|█▌        | 36226/225000 [79:28:58<354:08:03,  6.75s/it]
 16%|█▌        | 36227/225000 [79:29:04<343:04:00,  6.54s/it]
 16%|█▌        | 36228/225000 [79:29:09<312:36:04,  5.96s/it]
 16%|█▌        | 36229/225000 [79:29:17<340:51:06,  6.50s/it]
 16%|█▌        | 36230/225000 [79:29:21<311:25:11,  5.94s/it]
 16%|█▌        | 36231/225000 [79:29:26<295:40:22,  5.64s/it]
 16%|█▌        | 36232/225000 [79:29:32<294:22:43,  5.61s/it]
 16%|█▌        | 36233/225000 [79:29:45<418:21:01,  7.98s/it]
 16%|█▌        | 36234/225000 [79:29:49<352:35:00,  6.72s/it]
 16%|█▌        | 36235/225000 [79:29:57<372:24:05,  7.10s/it]
 16%|█▌        | 36236/225000 [79:30:01<328:07:38,  6.26s/it]
 16%|█▌        | 36237/225000 [79:30:10<370:49:07,  7.07s/it]
 16%|█▌        | 36238/225000 [79:30:14<312:45:27,  5.96s/it]
 16%|█▌        | 36239/225000 [79:30:21<338:29:07,  6.46s/it]
 16%|█▌        | 36240/225000 [79:30:27<318:19:53,  6.07s/it]
 16%|█▌        | 36241/225000 [79:30:32<301:23:33,  5.75s/it]
 16%|█▌        | 36242/225000 [79:30:37<294:41:18,  5.62s/it]
 16%|█▌        | 36243/225000 [79:30:43<302:51:41,  5.78s/it]
 16%|█▌        | 36244/225000 [79:30:56<416:02:55,  7.93s/it]
 16%|█▌        | 36245/225000 [79:31:00<354:20:47,  6.76s/it]
 16%|█▌        | 36246/225000 [79:31:08<373:27:49,  7.12s/it]
 16%|█▌        | 36247/225000 [79:31:12<325:40:52,  6.21s/it]
 16%|█▌        | 36248/225000 [79:31:21<372:03:38,  7.10s/it]
 16%|█▌        | 36249/225000 [79:31:27<346:28:48,  6.61s/it]
 16%|█▌        | 36250/225000 [79:31:32<331:20:37,  6.32s/it]
                                                             
{'loss': '0.7082', 'grad_norm': '0.7907', 'learning_rate': '4.819e-05', 'epoch': '0.1611', 'num_input_tokens_seen': 135071496, 'train_runtime': '2.863e+05', 'train_tokens_per_second': '471.8'}

 16%|█▌        | 36250/225000 [79:31:32<331:20:37,  6.32s/it]
 16%|█▌        | 36251/225000 [79:31:40<359:34:07,  6.86s/it]
 16%|█▌        | 36252/225000 [79:31:45<321:46:01,  6.14s/it]
 16%|█▌        | 36253/225000 [79:31:53<358:57:31,  6.85s/it]
 16%|█▌        | 36254/225000 [79:32:00<357:22:20,  6.82s/it]
 16%|█▌        | 36255/225000 [79:32:05<329:28:50,  6.28s/it]
 16%|█▌        | 36256/225000 [79:32:13<349:45:22,  6.67s/it]
 16%|█▌        | 36257/225000 [79:32:17<308:32:51,  5.89s/it]
 16%|█▌        | 36258/225000 [79:32:22<293:15:12,  5.59s/it]
 16%|█▌        | 36259/225000 [79:32:27<289:39:20,  5.52s/it]
 16%|█▌        | 36260/225000 [79:32:36<344:23:03,  6.57s/it]
 16%|█▌        | 36261/225000 [79:32:43<350:00:24,  6.68s/it]
 16%|█▌        | 36262/225000 [79:32:48<316:39:55,  6.04s/it]
 16%|█▌        | 36263/225000 [79:32:52<289:28:53,  5.52s/it]
 16%|█▌        | 36264/225000 [79:33:01<347:14:27,  6.62s/it]
 16%|█▌        | 36265/225000 [79:33:11<399:34:22,  7.62s/it]
 16%|█▌        | 36266/225000 [79:33:15<339:17:54,  6.47s/it]
 16%|█▌        | 36267/225000 [79:33:25<394:38:32,  7.53s/it]
 16%|█▌        | 36268/225000 [79:33:29<349:11:25,  6.66s/it]
 16%|█▌        | 36269/225000 [79:33:35<328:15:01,  6.26s/it]
 16%|█▌        | 36270/225000 [79:33:47<417:42:36,  7.97s/it]
 16%|█▌        | 36271/225000 [79:33:51<353:38:40,  6.75s/it]
 16%|█▌        | 36272/225000 [79:34:00<395:20:34,  7.54s/it]
 16%|█▌        | 36273/225000 [79:34:07<380:03:08,  7.25s/it]
 16%|█▌        | 36274/225000 [79:34:11<329:26:37,  6.28s/it]
 16%|█▌        | 36275/225000 [79:34:15<301:37:52,  5.75s/it]
 16%|█▌        | 36276/225000 [79:34:19<275:08:29,  5.25s/it]
 16%|█▌        | 36277/225000 [79:34:31<379:41:52,  7.24s/it]
 16%|█▌        | 36278/225000 [79:34:36<344:36:51,  6.57s/it]
 16%|█▌        | 36279/225000 [79:34:45<374:20:01,  7.14s/it]
 16%|█▌        | 36280/225000 [79:34:51<369:22:04,  7.05s/it]
 16%|█▌        | 36281/225000 [79:34:56<333:48:06,  6.37s/it]
 16%|█▌        | 36282/225000 [79:35:01<301:36:11,  5.75s/it]
 16%|█▌        | 36283/225000 [79:35:10<364:56:39,  6.96s/it]
 16%|█▌        | 36284/225000 [79:35:17<359:00:18,  6.85s/it]
 16%|█▌        | 36285/225000 [79:35:21<323:50:46,  6.18s/it]
 16%|█▌        | 36286/225000 [79:35:26<298:17:02,  5.69s/it]
 16%|█▌        | 36287/225000 [79:35:37<379:26:42,  7.24s/it]
 16%|█▌        | 36288/225000 [79:35:41<333:34:34,  6.36s/it]
 16%|█▌        | 36289/225000 [79:35:48<347:32:21,  6.63s/it]
 16%|█▌        | 36290/225000 [79:35:53<313:56:07,  5.99s/it]
 16%|█▌        | 36291/225000 [79:36:02<358:46:22,  6.84s/it]
 16%|█▌        | 36292/225000 [79:36:07<327:02:06,  6.24s/it]
 16%|█▌        | 36293/225000 [79:36:15<361:37:08,  6.90s/it]
 16%|█▌        | 36294/225000 [79:36:20<330:34:08,  6.31s/it]
 16%|█▌        | 36295/225000 [79:36:25<307:27:47,  5.87s/it]
 16%|█▌        | 36296/225000 [79:36:29<288:39:16,  5.51s/it]
 16%|█▌        | 36297/225000 [79:36:34<276:22:16,  5.27s/it]
 16%|█▌        | 36298/225000 [79:36:47<392:37:04,  7.49s/it]
 16%|█▌        | 36299/225000 [79:36:52<355:22:13,  6.78s/it]
 16%|█▌        | 36300/225000 [79:36:57<325:09:29,  6.20s/it]
                                                             
{'loss': '0.7241', 'grad_norm': '0.7757', 'learning_rate': '4.818e-05', 'epoch': '0.1613', 'num_input_tokens_seen': 135215984, 'train_runtime': '2.866e+05', 'train_tokens_per_second': '471.8'}

 16%|█▌        | 36300/225000 [79:36:57<325:09:29,  6.20s/it]
 16%|█▌        | 36301/225000 [79:37:02<304:53:13,  5.82s/it]
 16%|█▌        | 36302/225000 [79:37:07<290:44:40,  5.55s/it]
 16%|█▌        | 36303/225000 [79:37:17<362:26:31,  6.91s/it]
 16%|█▌        | 36304/225000 [79:37:27<407:24:22,  7.77s/it]
 16%|█▌        | 36305/225000 [79:37:33<385:29:14,  7.35s/it]
 16%|█▌        | 36306/225000 [79:37:37<340:15:19,  6.49s/it]
 16%|█▌        | 36307/225000 [79:37:42<315:52:08,  6.03s/it]
 16%|█▌        | 36308/225000 [79:37:54<399:18:27,  7.62s/it]
 16%|█▌        | 36309/225000 [79:37:59<356:57:34,  6.81s/it]
 16%|█▌        | 36310/225000 [79:38:04<328:25:52,  6.27s/it]
 16%|█▌        | 36311/225000 [79:38:09<309:18:29,  5.90s/it]
 16%|█▌        | 36312/225000 [79:38:20<400:17:05,  7.64s/it]
 16%|█▌        | 36313/225000 [79:38:25<358:35:38,  6.84s/it]
 16%|█▌        | 36314/225000 [79:38:30<325:56:25,  6.22s/it]
 16%|█▌        | 36315/225000 [79:38:35<305:16:25,  5.82s/it]
 16%|█▌        | 36316/225000 [79:38:41<307:47:12,  5.87s/it]
 16%|█▌        | 36317/225000 [79:38:53<408:34:11,  7.80s/it]
 16%|█▌        | 36318/225000 [79:39:00<389:07:02,  7.42s/it]
 16%|█▌        | 36319/225000 [79:39:05<353:57:43,  6.75s/it]
 16%|█▌        | 36320/225000 [79:39:10<324:34:20,  6.19s/it]
 16%|█▌        | 36321/225000 [79:39:19<364:14:59,  6.95s/it]
 16%|█▌        | 36322/225000 [79:39:28<395:44:52,  7.55s/it]
 16%|█▌        | 36323/225000 [79:39:31<334:39:46,  6.39s/it]
 16%|█▌        | 36324/225000 [79:39:39<353:40:55,  6.75s/it]
 16%|█▌        | 36325/225000 [79:39:45<337:24:15,  6.44s/it]
 16%|█▌        | 36326/225000 [79:39:50<315:35:27,  6.02s/it]
 16%|█▌        | 36327/225000 [79:39:54<284:58:23,  5.44s/it]
 16%|█▌        | 36328/225000 [79:40:05<376:49:59,  7.19s/it]
 16%|█▌        | 36329/225000 [79:40:09<321:22:01,  6.13s/it]
 16%|█▌        | 36330/225000 [79:40:18<369:50:09,  7.06s/it]
 16%|█▌        | 36331/225000 [79:40:22<331:57:15,  6.33s/it]
 16%|█▌        | 36332/225000 [79:40:27<304:51:21,  5.82s/it]
 16%|█▌        | 36333/225000 [79:40:32<290:28:37,  5.54s/it]
 16%|█▌        | 36334/225000 [79:40:37<282:46:21,  5.40s/it]
 16%|█▌        | 36335/225000 [79:40:42<277:23:33,  5.29s/it]
 16%|█▌        | 36336/225000 [79:40:54<382:45:57,  7.30s/it]
 16%|█▌        | 36337/225000 [79:41:03<407:59:36,  7.79s/it]
 16%|█▌        | 36338/225000 [79:41:07<348:40:29,  6.65s/it]
 16%|█▌        | 36339/225000 [79:41:18<409:03:48,  7.81s/it]
 16%|█▌        | 36340/225000 [79:41:22<354:08:31,  6.76s/it]
 16%|█▌        | 36341/225000 [79:41:26<315:08:04,  6.01s/it]
 16%|█▌        | 36342/225000 [79:41:37<385:44:10,  7.36s/it]
 16%|█▌        | 36343/225000 [79:41:41<336:39:06,  6.42s/it]
 16%|█▌        | 36344/225000 [79:41:46<311:51:30,  5.95s/it]
 16%|█▌        | 36345/225000 [79:41:51<295:29:42,  5.64s/it]
 16%|█▌        | 36346/225000 [79:42:03<398:01:09,  7.60s/it]
 16%|█▌        | 36347/225000 [79:42:07<338:48:25,  6.47s/it]
 16%|█▌        | 36348/225000 [79:42:14<359:12:32,  6.85s/it]
 16%|█▌        | 36349/225000 [79:42:22<375:39:05,  7.17s/it]
 16%|█▌        | 36350/225000 [79:42:26<325:24:50,  6.21s/it]
                                                             
{'loss': '0.7141', 'grad_norm': '0.6626', 'learning_rate': '4.817e-05', 'epoch': '0.1616', 'num_input_tokens_seen': 135362240, 'train_runtime': '2.869e+05', 'train_tokens_per_second': '471.7'}

 16%|█▌        | 36350/225000 [79:42:26<325:24:50,  6.21s/it]
 16%|█▌        | 36351/225000 [79:42:35<363:23:36,  6.93s/it]
 16%|█▌        | 36352/225000 [79:42:40<328:46:44,  6.27s/it]
 16%|█▌        | 36353/225000 [79:42:45<308:58:43,  5.90s/it]
 16%|█▌        | 36354/225000 [79:42:55<378:04:54,  7.22s/it]
 16%|█▌        | 36355/225000 [79:42:59<331:58:10,  6.34s/it]
 16%|█▌        | 36356/225000 [79:43:04<309:19:36,  5.90s/it]
 16%|█▌        | 36357/225000 [79:43:08<281:08:53,  5.37s/it]
 16%|█▌        | 36358/225000 [79:43:18<355:06:48,  6.78s/it]
 16%|█▌        | 36359/225000 [79:43:26<375:44:02,  7.17s/it]
 16%|█▌        | 36360/225000 [79:43:33<363:33:44,  6.94s/it]
 16%|█▌        | 36361/225000 [79:43:38<340:46:46,  6.50s/it]
 16%|█▌        | 36362/225000 [79:43:48<388:33:22,  7.42s/it]
 16%|█▌        | 36363/225000 [79:43:54<372:54:46,  7.12s/it]
 16%|█▌        | 36364/225000 [79:43:58<324:31:51,  6.19s/it]
 16%|█▌        | 36365/225000 [79:44:03<296:38:56,  5.66s/it]
 16%|█▌        | 36366/225000 [79:44:10<318:44:58,  6.08s/it]
 16%|█▌        | 36367/225000 [79:44:20<380:47:34,  7.27s/it]
 16%|█▌        | 36368/225000 [79:44:24<337:28:26,  6.44s/it]
 16%|█▌        | 36369/225000 [79:44:29<303:05:46,  5.78s/it]
 16%|█▌        | 36370/225000 [79:44:38<363:18:25,  6.93s/it]
 16%|█▌        | 36371/225000 [79:44:44<340:45:55,  6.50s/it]
 16%|█▌        | 36372/225000 [79:44:49<319:13:27,  6.09s/it]
 16%|█▌        | 36373/225000 [79:44:53<295:18:54,  5.64s/it]
 16%|█▌        | 36374/225000 [79:44:58<287:09:49,  5.48s/it]
 16%|█▌        | 36375/225000 [79:45:05<307:28:05,  5.87s/it]
 16%|█▌        | 36376/225000 [79:45:19<428:17:04,  8.17s/it]
 16%|█▌        | 36377/225000 [79:45:23<368:03:22,  7.02s/it]
 16%|█▌        | 36378/225000 [79:45:30<358:41:45,  6.85s/it]
 16%|█▌        | 36379/225000 [79:45:34<321:13:49,  6.13s/it]
 16%|█▌        | 36380/225000 [79:45:42<355:32:07,  6.79s/it]
 16%|█▌        | 36381/225000 [79:45:50<368:59:29,  7.04s/it]
 16%|█▌        | 36382/225000 [79:45:58<391:15:24,  7.47s/it]
 16%|█▌        | 36383/225000 [79:46:05<372:05:53,  7.10s/it]
 16%|█▌        | 36384/225000 [79:46:10<341:08:08,  6.51s/it]
 16%|█▌        | 36385/225000 [79:46:15<322:05:43,  6.15s/it]
 16%|█▌        | 36386/225000 [79:46:21<312:31:07,  5.96s/it]
 16%|█▌        | 36387/225000 [79:46:26<304:02:43,  5.80s/it]
 16%|█▌        | 36388/225000 [79:46:38<407:03:30,  7.77s/it]
 16%|█▌        | 36389/225000 [79:46:45<392:01:32,  7.48s/it]
 16%|█▌        | 36390/225000 [79:46:50<355:37:29,  6.79s/it]
 16%|█▌        | 36391/225000 [79:46:55<327:42:00,  6.25s/it]
 16%|█▌        | 36392/225000 [79:47:06<393:21:57,  7.51s/it]
 16%|█▌        | 36393/225000 [79:47:14<399:37:57,  7.63s/it]
 16%|█▌        | 36394/225000 [79:47:20<379:30:24,  7.24s/it]
 16%|█▌        | 36395/225000 [79:47:24<323:59:09,  6.18s/it]
 16%|█▌        | 36396/225000 [79:47:33<372:52:00,  7.12s/it]
 16%|█▌        | 36397/225000 [79:47:40<368:03:29,  7.03s/it]
 16%|█▌        | 36398/225000 [79:47:45<330:53:49,  6.32s/it]
 16%|█▌        | 36399/225000 [79:47:49<300:42:23,  5.74s/it]
 16%|█▌        | 36400/225000 [79:47:57<336:57:17,  6.43s/it]
                                                             
{'loss': '0.7188', 'grad_norm': '0.8198', 'learning_rate': '4.817e-05', 'epoch': '0.1618', 'num_input_tokens_seen': 135508056, 'train_runtime': '2.873e+05', 'train_tokens_per_second': '471.7'}

 16%|█▌        | 36400/225000 [79:47:57<336:57:17,  6.43s/it]
 16%|█▌        | 36401/225000 [79:48:06<377:09:13,  7.20s/it]
 16%|█▌        | 36402/225000 [79:48:11<346:09:16,  6.61s/it]
 16%|█▌        | 36403/225000 [79:48:16<322:40:41,  6.16s/it]
 16%|█▌        | 36404/225000 [79:48:22<315:01:49,  6.01s/it]
 16%|█▌        | 36405/225000 [79:48:28<307:31:23,  5.87s/it]
 16%|█▌        | 36406/225000 [79:48:33<297:49:02,  5.68s/it]
 16%|█▌        | 36407/225000 [79:48:38<294:04:14,  5.61s/it]
 16%|█▌        | 36408/225000 [79:48:44<296:11:39,  5.65s/it]
 16%|█▌        | 36409/225000 [79:48:49<292:53:30,  5.59s/it]
 16%|█▌        | 36410/225000 [79:49:01<389:23:23,  7.43s/it]
 16%|█▌        | 36411/225000 [79:49:12<447:56:34,  8.55s/it]
 16%|█▌        | 36412/225000 [79:49:16<366:29:30,  7.00s/it]
 16%|█▌        | 36413/225000 [79:49:21<336:10:14,  6.42s/it]
 16%|█▌        | 36414/225000 [79:49:30<378:11:26,  7.22s/it]
 16%|█▌        | 36415/225000 [79:49:34<336:37:08,  6.43s/it]
 16%|█▌        | 36416/225000 [79:49:39<306:16:12,  5.85s/it]
 16%|█▌        | 36417/225000 [79:49:48<349:54:17,  6.68s/it]
 16%|█▌        | 36418/225000 [79:49:57<395:03:12,  7.54s/it]
 16%|█▌        | 36419/225000 [79:50:01<337:36:26,  6.44s/it]
 16%|█▌        | 36420/225000 [79:50:06<317:43:07,  6.07s/it]
 16%|█▌        | 36421/225000 [79:50:11<303:49:20,  5.80s/it]
 16%|█▌        | 36422/225000 [79:50:16<290:32:36,  5.55s/it]
 16%|█▌        | 36423/225000 [79:50:22<291:49:41,  5.57s/it]
 16%|█▌        | 36424/225000 [79:50:27<281:29:44,  5.37s/it]
 16%|█▌        | 36425/225000 [79:50:40<404:04:20,  7.71s/it]
 16%|█▌        | 36426/225000 [79:50:48<402:02:11,  7.68s/it]
 16%|█▌        | 36427/225000 [79:50:53<359:53:47,  6.87s/it]
 16%|█▌        | 36428/225000 [79:50:57<319:13:32,  6.09s/it]
 16%|█▌        | 36429/225000 [79:51:08<400:26:09,  7.64s/it]
 16%|█▌        | 36430/225000 [79:51:14<372:41:00,  7.11s/it]
 16%|█▌        | 36431/225000 [79:51:21<368:46:18,  7.04s/it]
 16%|█▌        | 36432/225000 [79:51:28<369:53:56,  7.06s/it]
 16%|█▌        | 36433/225000 [79:51:33<344:36:31,  6.58s/it]
 16%|█▌        | 36434/225000 [79:51:43<386:13:51,  7.37s/it]
 16%|█▌        | 36435/225000 [79:51:48<349:22:23,  6.67s/it]
 16%|█▌        | 36436/225000 [79:51:54<337:53:33,  6.45s/it]
 16%|█▌        | 36437/225000 [79:51:59<321:25:58,  6.14s/it]
 16%|█▌        | 36438/225000 [79:52:05<318:10:42,  6.07s/it]
 16%|█▌        | 36439/225000 [79:52:11<309:17:11,  5.90s/it]
 16%|█▌        | 36440/225000 [79:52:16<303:27:14,  5.79s/it]
 16%|█▌        | 36441/225000 [79:52:22<299:25:29,  5.72s/it]
 16%|█▌        | 36442/225000 [79:52:28<313:19:36,  5.98s/it]
 16%|█▌        | 36443/225000 [79:52:42<440:42:25,  8.41s/it]
 16%|█▌        | 36444/225000 [79:52:46<369:17:15,  7.05s/it]
 16%|█▌        | 36445/225000 [79:52:55<403:07:30,  7.70s/it]
 16%|█▌        | 36446/225000 [79:53:00<349:34:44,  6.67s/it]
 16%|█▌        | 36447/225000 [79:53:04<318:14:29,  6.08s/it]
 16%|█▌        | 36448/225000 [79:53:13<366:45:44,  7.00s/it]
 16%|█▌        | 36449/225000 [79:53:20<352:04:50,  6.72s/it]
 16%|█▌        | 36450/225000 [79:53:25<333:45:28,  6.37s/it]
                                                             
{'loss': '0.7199', 'grad_norm': '0.6635', 'learning_rate': '4.816e-05', 'epoch': '0.162', 'num_input_tokens_seen': 135653176, 'train_runtime': '2.876e+05', 'train_tokens_per_second': '471.7'}

 16%|█▌        | 36450/225000 [79:53:25<333:45:28,  6.37s/it]
 16%|█▌        | 36451/225000 [79:53:32<340:46:05,  6.51s/it]
 16%|█▌        | 36452/225000 [79:53:37<320:20:19,  6.12s/it]
 16%|█▌        | 36453/225000 [79:53:47<374:59:31,  7.16s/it]
 16%|█▌        | 36454/225000 [79:53:53<360:32:18,  6.88s/it]
 16%|█▌        | 36455/225000 [79:53:57<309:45:54,  5.91s/it]
 16%|█▌        | 36456/225000 [79:54:05<355:50:34,  6.79s/it]
 16%|█▌        | 36457/225000 [79:54:12<348:18:58,  6.65s/it]
 16%|█▌        | 36458/225000 [79:54:19<351:31:05,  6.71s/it]
 16%|█▌        | 36459/225000 [79:54:22<299:55:59,  5.73s/it]
 16%|█▌        | 36460/225000 [79:54:33<375:18:58,  7.17s/it]
 16%|█▌        | 36461/225000 [79:54:37<339:24:47,  6.48s/it]
 16%|█▌        | 36462/225000 [79:54:42<307:06:23,  5.86s/it]
 16%|█▌        | 36463/225000 [79:54:50<341:05:54,  6.51s/it]
 16%|█▌        | 36464/225000 [79:54:54<302:58:13,  5.79s/it]
 16%|█▌        | 36465/225000 [79:54:59<292:19:04,  5.58s/it]
 16%|█▌        | 36466/225000 [79:55:11<389:09:22,  7.43s/it]
 16%|█▌        | 36467/225000 [79:55:17<375:12:57,  7.16s/it]
 16%|█▌        | 36468/225000 [79:55:21<325:55:57,  6.22s/it]
 16%|█▌        | 36469/225000 [79:55:29<348:59:16,  6.66s/it]
 16%|█▌        | 36470/225000 [79:55:34<313:59:04,  6.00s/it]
 16%|█▌        | 36471/225000 [79:55:42<346:58:07,  6.63s/it]
 16%|█▌        | 36472/225000 [79:55:47<319:03:58,  6.09s/it]
 16%|█▌        | 36473/225000 [79:55:51<294:28:59,  5.62s/it]
 16%|█▌        | 36474/225000 [79:55:56<285:07:43,  5.44s/it]
 16%|█▌        | 36475/225000 [79:56:09<399:34:14,  7.63s/it]
 16%|█▌        | 36476/225000 [79:56:14<354:30:39,  6.77s/it]
 16%|█▌        | 36477/225000 [79:56:22<379:18:28,  7.24s/it]
 16%|█▌        | 36478/225000 [79:56:26<322:58:02,  6.17s/it]
 16%|█▌        | 36479/225000 [79:56:35<368:10:52,  7.03s/it]
 16%|█▌        | 36480/225000 [79:56:39<332:48:55,  6.36s/it]
 16%|█▌        | 36481/225000 [79:56:44<306:50:28,  5.86s/it]
 16%|█▌        | 36482/225000 [79:56:55<391:39:08,  7.48s/it]
 16%|█▌        | 36483/225000 [79:57:03<388:38:25,  7.42s/it]
 16%|█▌        | 36484/225000 [79:57:06<329:56:38,  6.30s/it]
 16%|█▌        | 36485/225000 [79:57:11<304:46:50,  5.82s/it]
 16%|█▌        | 36486/225000 [79:57:16<295:27:14,  5.64s/it]
 16%|█▌        | 36487/225000 [79:57:22<289:34:17,  5.53s/it]
 16%|█▌        | 36488/225000 [79:57:27<293:47:13,  5.61s/it]
 16%|█▌        | 36489/225000 [79:57:37<351:55:47,  6.72s/it]
 16%|█▌        | 36490/225000 [79:57:47<408:20:27,  7.80s/it]
 16%|█▌        | 36491/225000 [79:57:51<348:18:04,  6.65s/it]
 16%|█▌        | 36492/225000 [79:57:56<321:42:07,  6.14s/it]
 16%|█▌        | 36493/225000 [79:58:03<335:42:02,  6.41s/it]
 16%|█▌        | 36494/225000 [79:58:16<441:01:58,  8.42s/it]
 16%|█▌        | 36495/225000 [79:58:22<399:39:39,  7.63s/it]
 16%|█▌        | 36496/225000 [79:58:26<351:05:49,  6.71s/it]
 16%|█▌        | 36497/225000 [79:58:34<364:27:16,  6.96s/it]
 16%|█▌        | 36498/225000 [79:58:38<313:54:48,  6.00s/it]
 16%|█▌        | 36499/225000 [79:58:46<349:54:48,  6.68s/it]
 16%|█▌        | 36500/225000 [79:58:50<312:16:58,  5.96s/it]
                                                             
{'loss': '0.7022', 'grad_norm': '0.6447', 'learning_rate': '4.815e-05', 'epoch': '0.1622', 'num_input_tokens_seen': 135798200, 'train_runtime': '2.879e+05', 'train_tokens_per_second': '471.6'}

 16%|█▌        | 36500/225000 [79:58:50<312:16:58,  5.96s/it]
 16%|█▌        | 36501/225000 [79:58:55<287:56:53,  5.50s/it]
 16%|█▌        | 36502/225000 [79:59:00<279:00:10,  5.33s/it]
 16%|█▌        | 36503/225000 [79:59:11<376:34:01,  7.19s/it]
 16%|█▌        | 36504/225000 [79:59:16<343:38:40,  6.56s/it]
 16%|█▌        | 36505/225000 [79:59:21<311:18:27,  5.95s/it]
 16%|█▌        | 36506/225000 [79:59:26<299:44:35,  5.72s/it]
 16%|█▌        | 36507/225000 [79:59:31<285:22:38,  5.45s/it]
 16%|█▌        | 36508/225000 [79:59:44<409:55:14,  7.83s/it]
 16%|█▌        | 36509/225000 [79:59:49<358:34:48,  6.85s/it]
 16%|█▌        | 36510/225000 [79:59:56<372:48:48,  7.12s/it]
 16%|█▌        | 36511/225000 [80:00:01<339:43:34,  6.49s/it]
 16%|█▌        | 36512/225000 [80:00:06<316:03:58,  6.04s/it]
 16%|█▌        | 36513/225000 [80:00:12<304:07:28,  5.81s/it]
 16%|█▌        | 36514/225000 [80:00:17<292:56:25,  5.60s/it]
 16%|█▌        | 36515/225000 [80:00:28<382:19:10,  7.30s/it]
 16%|█▌        | 36516/225000 [80:00:35<377:12:02,  7.20s/it]
 16%|█▌        | 36517/225000 [80:00:40<349:23:07,  6.67s/it]
 16%|█▌        | 36518/225000 [80:00:47<346:18:13,  6.61s/it]
 16%|█▌        | 36519/225000 [80:00:51<307:31:18,  5.87s/it]
 16%|█▌        | 36520/225000 [80:01:00<356:29:32,  6.81s/it]
 16%|█▌        | 36521/225000 [80:01:08<368:29:00,  7.04s/it]
 16%|█▌        | 36522/225000 [80:01:12<324:36:39,  6.20s/it]
 16%|█▌        | 36523/225000 [80:01:21<375:47:07,  7.18s/it]
 16%|█▌        | 36524/225000 [80:01:27<343:44:56,  6.57s/it]
 16%|█▌        | 36525/225000 [80:01:31<316:00:28,  6.04s/it]
 16%|█▌        | 36526/225000 [80:01:36<301:21:56,  5.76s/it]
 16%|█▌        | 36527/225000 [80:01:45<349:02:52,  6.67s/it]
 16%|█▌        | 36528/225000 [80:01:55<391:57:26,  7.49s/it]
 16%|█▌        | 36529/225000 [80:02:02<394:29:13,  7.54s/it]
 16%|█▌        | 36530/225000 [80:02:06<339:15:45,  6.48s/it]
 16%|█▌        | 36531/225000 [80:02:15<381:42:45,  7.29s/it]
 16%|█▌        | 36532/225000 [80:02:20<341:07:50,  6.52s/it]
 16%|█▌        | 36533/225000 [80:02:25<318:31:05,  6.08s/it]
 16%|█▌        | 36534/225000 [80:02:30<297:46:43,  5.69s/it]
 16%|█▌        | 36535/225000 [80:02:38<330:20:50,  6.31s/it]
 16%|█▌        | 36536/225000 [80:02:47<381:56:33,  7.30s/it]
 16%|█▌        | 36537/225000 [80:02:52<339:58:52,  6.49s/it]
 16%|█▌        | 36538/225000 [80:03:01<377:40:08,  7.21s/it]
 16%|█▌        | 36539/225000 [80:03:08<377:45:22,  7.22s/it]
 16%|█▌        | 36540/225000 [80:03:14<360:03:19,  6.88s/it]
 16%|█▌        | 36541/225000 [80:03:18<306:37:24,  5.86s/it]
 16%|█▌        | 36542/225000 [80:03:28<376:07:13,  7.18s/it]
 16%|█▌        | 36543/225000 [80:03:32<325:33:15,  6.22s/it]
 16%|█▌        | 36544/225000 [80:03:41<366:35:58,  7.00s/it]
 16%|█▌        | 36545/225000 [80:03:45<329:27:52,  6.29s/it]
 16%|█▌        | 36546/225000 [80:03:50<302:56:09,  5.79s/it]
 16%|█▌        | 36547/225000 [80:03:55<290:48:08,  5.56s/it]
 16%|█▌        | 36548/225000 [80:04:06<370:33:39,  7.08s/it]
 16%|█▌        | 36549/225000 [80:04:13<367:35:17,  7.02s/it]
 16%|█▌        | 36550/225000 [80:04:19<358:35:57,  6.85s/it]
                                                             
{'loss': '0.6862', 'grad_norm': '0.7629', 'learning_rate': '4.814e-05', 'epoch': '0.1624', 'num_input_tokens_seen': 135944472, 'train_runtime': '2.883e+05', 'train_tokens_per_second': '471.6'}

 16%|█▌        | 36550/225000 [80:04:19<358:35:57,  6.85s/it]
 16%|█▌        | 36551/225000 [80:04:23<321:49:39,  6.15s/it]
 16%|█▌        | 36552/225000 [80:04:28<295:33:48,  5.65s/it]
 16%|█▌        | 36553/225000 [80:04:37<346:12:00,  6.61s/it]
 16%|█▌        | 36554/225000 [80:04:47<396:09:24,  7.57s/it]
 16%|█▌        | 36555/225000 [80:04:50<335:40:36,  6.41s/it]
 16%|█▌        | 36556/225000 [80:04:59<373:29:56,  7.14s/it]
 16%|█▌        | 36557/225000 [80:05:05<345:16:21,  6.60s/it]
 16%|█▌        | 36558/225000 [80:05:10<321:21:45,  6.14s/it]
 16%|█▌        | 36559/225000 [80:05:15<306:32:55,  5.86s/it]
 16%|█▌        | 36560/225000 [80:05:20<301:25:09,  5.76s/it]
 16%|█▌        | 36561/225000 [80:05:26<294:23:37,  5.62s/it]
 16%|█▌        | 36562/225000 [80:05:31<296:17:43,  5.66s/it]
 16%|█▋        | 36563/225000 [80:05:41<364:07:35,  6.96s/it]
 16%|█▋        | 36564/225000 [80:05:51<409:03:59,  7.82s/it]
 16%|█▋        | 36565/225000 [80:05:55<345:36:38,  6.60s/it]
 16%|█▋        | 36566/225000 [80:06:04<387:30:17,  7.40s/it]
 16%|█▋        | 36567/225000 [80:06:09<354:20:34,  6.77s/it]
 16%|█▋        | 36568/225000 [80:06:13<309:45:08,  5.92s/it]
 16%|█▋        | 36569/225000 [80:06:24<378:34:22,  7.23s/it]
 16%|█▋        | 36570/225000 [80:06:28<324:35:08,  6.20s/it]
 16%|█▋        | 36571/225000 [80:06:37<375:00:43,  7.16s/it]
 16%|█▋        | 36572/225000 [80:06:41<323:54:58,  6.19s/it]
 16%|█▋        | 36573/225000 [80:06:47<325:48:49,  6.22s/it]
 16%|█▋        | 36574/225000 [80:06:56<373:47:58,  7.14s/it]
 16%|█▋        | 36575/225000 [80:07:00<320:22:13,  6.12s/it]
 16%|█▋        | 36576/225000 [80:07:09<362:10:12,  6.92s/it]
 16%|█▋        | 36577/225000 [80:07:16<370:40:46,  7.08s/it]
 16%|█▋        | 36578/225000 [80:07:21<326:25:16,  6.24s/it]
 16%|█▋        | 36579/225000 [80:07:28<346:34:44,  6.62s/it]
 16%|█▋        | 36580/225000 [80:07:35<343:31:59,  6.56s/it]
 16%|█▋        | 36581/225000 [80:07:41<346:21:32,  6.62s/it]
 16%|█▋        | 36582/225000 [80:07:45<300:37:38,  5.74s/it]
 16%|█▋        | 36583/225000 [80:07:50<291:40:35,  5.57s/it]
 16%|█▋        | 36584/225000 [80:08:02<387:23:17,  7.40s/it]
 16%|█▋        | 36585/225000 [80:08:05<320:41:02,  6.13s/it]
 16%|█▋        | 36586/225000 [80:08:13<343:18:13,  6.56s/it]
 16%|█▋        | 36587/225000 [80:08:18<319:48:36,  6.11s/it]
 16%|█▋        | 36588/225000 [80:08:22<291:05:05,  5.56s/it]
 16%|█▋        | 36589/225000 [80:08:27<283:59:00,  5.43s/it]
 16%|█▋        | 36590/225000 [80:08:32<281:20:32,  5.38s/it]
 16%|█▋        | 36591/225000 [80:08:45<398:07:10,  7.61s/it]
 16%|█▋        | 36592/225000 [80:08:50<354:37:41,  6.78s/it]
 16%|█▋        | 36593/225000 [80:08:54<310:55:03,  5.94s/it]
 16%|█▋        | 36594/225000 [80:08:59<301:57:44,  5.77s/it]
 16%|█▋        | 36595/225000 [80:09:11<398:25:25,  7.61s/it]
 16%|█▋        | 36596/225000 [80:09:15<339:20:05,  6.48s/it]
 16%|█▋        | 36597/225000 [80:09:19<294:46:51,  5.63s/it]
 16%|█▋        | 36598/225000 [80:09:29<366:23:49,  7.00s/it]
 16%|█▋        | 36599/225000 [80:09:33<325:58:41,  6.23s/it]
 16%|█▋        | 36600/225000 [80:09:38<293:05:24,  5.60s/it]
                                                             
{'loss': '0.6938', 'grad_norm': '0.6931', 'learning_rate': '4.814e-05', 'epoch': '0.1627', 'num_input_tokens_seen': 136088688, 'train_runtime': '2.886e+05', 'train_tokens_per_second': '471.6'}

 16%|█▋        | 36600/225000 [80:09:38<293:05:24,  5.60s/it]
 16%|█▋        | 36601/225000 [80:09:49<383:03:32,  7.32s/it]
 16%|█▋        | 36602/225000 [80:09:53<339:14:47,  6.48s/it]
 16%|█▋        | 36603/225000 [80:09:58<305:25:13,  5.84s/it]
 16%|█▋        | 36604/225000 [80:10:02<282:55:06,  5.41s/it]
 16%|█▋        | 36605/225000 [80:10:13<365:16:40,  6.98s/it]
 16%|█▋        | 36606/225000 [80:10:17<319:28:13,  6.10s/it]
 16%|█▋        | 36607/225000 [80:10:21<294:31:15,  5.63s/it]
 16%|█▋        | 36608/225000 [80:10:32<372:03:01,  7.11s/it]
 16%|█▋        | 36609/225000 [80:10:39<365:26:41,  6.98s/it]
 16%|█▋        | 36610/225000 [80:10:44<336:24:35,  6.43s/it]
 16%|█▋        | 36611/225000 [80:10:49<321:13:15,  6.14s/it]
 16%|█▋        | 36612/225000 [80:10:59<375:46:08,  7.18s/it]
 16%|█▋        | 36613/225000 [80:11:05<359:35:01,  6.87s/it]
 16%|█▋        | 36614/225000 [80:11:13<378:22:28,  7.23s/it]
 16%|█▋        | 36615/225000 [80:11:20<377:19:28,  7.21s/it]
 16%|█▋        | 36616/225000 [80:11:26<355:18:59,  6.79s/it]
 16%|█▋        | 36617/225000 [80:11:33<357:37:14,  6.83s/it]
 16%|█▋        | 36618/225000 [80:11:40<361:43:11,  6.91s/it]
 16%|█▋        | 36619/225000 [80:11:47<368:08:51,  7.04s/it]
 16%|█▋        | 36620/225000 [80:11:52<332:14:35,  6.35s/it]
 16%|█▋        | 36621/225000 [80:11:57<314:43:32,  6.01s/it]
 16%|█▋        | 36622/225000 [80:12:03<301:16:49,  5.76s/it]
 16%|█▋        | 36623/225000 [80:12:07<282:41:51,  5.40s/it]
 16%|█▋        | 36624/225000 [80:12:14<307:15:02,  5.87s/it]
 16%|█▋        | 36625/225000 [80:12:25<383:07:24,  7.32s/it]
 16%|█▋        | 36626/225000 [80:12:30<351:11:28,  6.71s/it]
 16%|█▋        | 36627/225000 [80:12:36<335:52:42,  6.42s/it]
 16%|█▋        | 36628/225000 [80:12:40<299:17:12,  5.72s/it]
 16%|█▋        | 36629/225000 [80:12:51<377:29:28,  7.21s/it]
 16%|█▋        | 36630/225000 [80:12:57<370:47:17,  7.09s/it]
 16%|█▋        | 36631/225000 [80:13:05<380:23:03,  7.27s/it]
 16%|█▋        | 36632/225000 [80:13:10<345:46:41,  6.61s/it]
 16%|█▋        | 36633/225000 [80:13:17<343:51:59,  6.57s/it]
 16%|█▋        | 36634/225000 [80:13:21<311:50:58,  5.96s/it]
 16%|█▋        | 36635/225000 [80:13:26<302:10:35,  5.78s/it]
 16%|█▋        | 36636/225000 [80:13:32<296:17:17,  5.66s/it]
 16%|█▋        | 36637/225000 [80:13:38<305:54:06,  5.85s/it]
 16%|█▋        | 36638/225000 [80:13:51<420:11:35,  8.03s/it]
 16%|█▋        | 36639/225000 [80:13:56<364:13:53,  6.96s/it]
 16%|█▋        | 36640/225000 [80:14:02<357:15:38,  6.83s/it]
 16%|█▋        | 36641/225000 [80:14:06<309:48:20,  5.92s/it]
 16%|█▋        | 36642/225000 [80:14:10<281:37:47,  5.38s/it]
 16%|█▋        | 36643/225000 [80:14:21<373:38:49,  7.14s/it]
 16%|█▋        | 36644/225000 [80:14:27<346:55:50,  6.63s/it]
 16%|█▋        | 36645/225000 [80:14:33<338:50:01,  6.48s/it]
 16%|█▋        | 36646/225000 [80:14:43<392:37:06,  7.50s/it]
 16%|█▋        | 36647/225000 [80:14:47<340:22:39,  6.51s/it]
 16%|█▋        | 36648/225000 [80:14:56<380:29:11,  7.27s/it]
 16%|█▋        | 36649/225000 [80:15:02<353:00:33,  6.75s/it]
 16%|█▋        | 36650/225000 [80:15:06<311:06:25,  5.95s/it]
                                                             
{'loss': '0.6836', 'grad_norm': '0.6855', 'learning_rate': '4.813e-05', 'epoch': '0.1629', 'num_input_tokens_seen': 136234376, 'train_runtime': '2.889e+05', 'train_tokens_per_second': '471.6'}

 16%|█▋        | 36650/225000 [80:15:06<311:06:25,  5.95s/it]
 16%|█▋        | 36651/225000 [80:15:10<284:55:20,  5.45s/it]
 16%|█▋        | 36652/225000 [80:15:19<339:40:52,  6.49s/it]
 16%|█▋        | 36653/225000 [80:15:29<390:01:17,  7.45s/it]
 16%|█▋        | 36654/225000 [80:15:32<322:33:34,  6.17s/it]
 16%|█▋        | 36655/225000 [80:15:37<313:19:39,  5.99s/it]
 16%|█▋        | 36656/225000 [80:15:45<343:40:01,  6.57s/it]
 16%|█▋        | 36657/225000 [80:15:50<319:14:41,  6.10s/it]
 16%|█▋        | 36658/225000 [80:15:56<309:54:34,  5.92s/it]
 16%|█▋        | 36659/225000 [80:16:01<301:14:33,  5.76s/it]
 16%|█▋        | 36660/225000 [80:16:10<352:12:23,  6.73s/it]
 16%|█▋        | 36661/225000 [80:16:17<347:21:40,  6.64s/it]
 16%|█▋        | 36662/225000 [80:16:21<317:12:13,  6.06s/it]
 16%|█▋        | 36663/225000 [80:16:26<291:44:41,  5.58s/it]
 16%|█▋        | 36664/225000 [80:16:31<283:22:34,  5.42s/it]
 16%|█▋        | 36665/225000 [80:16:37<287:27:25,  5.49s/it]
 16%|█▋        | 36666/225000 [80:16:49<393:17:17,  7.52s/it]
 16%|█▋        | 36667/225000 [80:16:55<367:21:26,  7.02s/it]
 16%|█▋        | 36668/225000 [80:17:02<370:59:54,  7.09s/it]
 16%|█▋        | 36669/225000 [80:17:08<357:38:36,  6.84s/it]
 16%|█▋        | 36670/225000 [80:17:17<383:23:08,  7.33s/it]
 16%|█▋        | 36671/225000 [80:17:23<365:56:08,  7.00s/it]
 16%|█▋        | 36672/225000 [80:17:29<357:31:09,  6.83s/it]
 16%|█▋        | 36673/225000 [80:17:33<310:55:43,  5.94s/it]
 16%|█▋        | 36674/225000 [80:17:41<338:10:05,  6.46s/it]
 16%|█▋        | 36675/225000 [80:17:47<340:55:50,  6.52s/it]
 16%|█▋        | 36676/225000 [80:17:57<394:37:43,  7.54s/it]
 16%|█▋        | 36677/225000 [80:18:01<331:43:34,  6.34s/it]
 16%|█▋        | 36678/225000 [80:18:06<316:08:50,  6.04s/it]
 16%|█▋        | 36679/225000 [80:18:12<303:30:56,  5.80s/it]
 16%|█▋        | 36680/225000 [80:18:16<290:21:37,  5.55s/it]
 16%|█▋        | 36681/225000 [80:18:25<332:26:50,  6.36s/it]
 16%|█▋        | 36682/225000 [80:18:37<422:09:56,  8.07s/it]
 16%|█▋        | 36683/225000 [80:18:42<372:15:11,  7.12s/it]
 16%|█▋        | 36684/225000 [80:18:47<339:03:00,  6.48s/it]
 16%|█▋        | 36685/225000 [80:18:52<322:28:59,  6.16s/it]
 16%|█▋        | 36686/225000 [80:18:58<310:51:23,  5.94s/it]
 16%|█▋        | 36687/225000 [80:19:03<302:32:44,  5.78s/it]
 16%|█▋        | 36688/225000 [80:19:13<375:42:36,  7.18s/it]
 16%|█▋        | 36689/225000 [80:19:23<421:26:35,  8.06s/it]
 16%|█▋        | 36690/225000 [80:19:29<374:51:18,  7.17s/it]
 16%|█▋        | 36691/225000 [80:19:34<342:21:40,  6.55s/it]
 16%|█▋        | 36692/225000 [80:19:38<315:08:44,  6.02s/it]
 16%|█▋        | 36693/225000 [80:19:44<310:57:01,  5.94s/it]
 16%|█▋        | 36694/225000 [80:19:57<423:23:16,  8.09s/it]
 16%|█▋        | 36695/225000 [80:20:01<361:28:31,  6.91s/it]
 16%|█▋        | 36696/225000 [80:20:10<389:17:11,  7.44s/it]
 16%|█▋        | 36697/225000 [80:20:14<331:41:46,  6.34s/it]
 16%|█▋        | 36698/225000 [80:20:20<329:22:05,  6.30s/it]
 16%|█▋        | 36699/225000 [80:20:28<351:42:36,  6.72s/it]
 16%|█▋        | 36700/225000 [80:20:32<307:23:21,  5.88s/it]
                                                             
{'loss': '0.694', 'grad_norm': '0.7261', 'learning_rate': '4.812e-05', 'epoch': '0.1631', 'num_input_tokens_seen': 136378384, 'train_runtime': '2.892e+05', 'train_tokens_per_second': '471.5'}

 16%|█▋        | 36700/225000 [80:20:32<307:23:21,  5.88s/it]
 16%|█▋        | 36701/225000 [80:20:42<368:31:42,  7.05s/it]
 16%|█▋        | 36702/225000 [80:20:45<317:25:49,  6.07s/it]
 16%|█▋        | 36703/225000 [80:20:55<378:52:43,  7.24s/it]
 16%|█▋        | 36704/225000 [80:21:01<347:56:54,  6.65s/it]
 16%|█▋        | 36705/225000 [80:21:07<347:57:44,  6.65s/it]
 16%|█▋        | 36706/225000 [80:21:11<310:31:41,  5.94s/it]
 16%|█▋        | 36707/225000 [80:21:19<340:19:11,  6.51s/it]
 16%|█▋        | 36708/225000 [80:21:27<360:02:55,  6.88s/it]
 16%|█▋        | 36709/225000 [80:21:34<362:30:55,  6.93s/it]
 16%|█▋        | 36710/225000 [80:21:38<310:09:42,  5.93s/it]
 16%|█▋        | 36711/225000 [80:21:47<356:24:45,  6.81s/it]
 16%|█▋        | 36712/225000 [80:21:50<304:53:30,  5.83s/it]
 16%|█▋        | 36713/225000 [80:22:00<376:02:21,  7.19s/it]
 16%|█▋        | 36714/225000 [80:22:07<365:33:43,  6.99s/it]
 16%|█▋        | 36715/225000 [80:22:12<327:25:51,  6.26s/it]
 16%|█▋        | 36716/225000 [80:22:16<293:41:57,  5.62s/it]
 16%|█▋        | 36717/225000 [80:22:27<386:00:26,  7.38s/it]
 16%|█▋        | 36718/225000 [80:22:32<344:12:03,  6.58s/it]
 16%|█▋        | 36719/225000 [80:22:39<356:19:40,  6.81s/it]
 16%|█▋        | 36720/225000 [80:22:44<321:23:43,  6.15s/it]
 16%|█▋        | 36721/225000 [80:22:53<370:57:50,  7.09s/it]
 16%|█▋        | 36722/225000 [80:23:00<367:39:17,  7.03s/it]
 16%|█▋        | 36723/225000 [80:23:07<361:31:34,  6.91s/it]
 16%|█▋        | 36724/225000 [80:23:15<377:09:03,  7.21s/it]
 16%|█▋        | 36725/225000 [80:23:21<367:45:06,  7.03s/it]
 16%|█▋        | 36726/225000 [80:23:25<321:47:47,  6.15s/it]
 16%|█▋        | 36727/225000 [80:23:30<300:26:15,  5.74s/it]
 16%|█▋        | 36728/225000 [80:23:42<398:34:33,  7.62s/it]
 16%|█▋        | 36729/225000 [80:23:47<357:40:50,  6.84s/it]
 16%|█▋        | 36730/225000 [80:23:51<305:55:58,  5.85s/it]
 16%|█▋        | 36731/225000 [80:24:00<359:05:13,  6.87s/it]
 16%|█▋        | 36732/225000 [80:24:04<321:57:56,  6.16s/it]
 16%|█▋        | 36733/225000 [80:24:11<326:44:02,  6.25s/it]
 16%|█▋        | 36734/225000 [80:24:19<353:19:22,  6.76s/it]
 16%|█▋        | 36735/225000 [80:24:25<348:37:42,  6.67s/it]
 16%|█▋        | 36736/225000 [80:24:30<325:20:12,  6.22s/it]
 16%|█▋        | 36737/225000 [80:24:36<309:08:20,  5.91s/it]
 16%|█▋        | 36738/225000 [80:24:40<292:44:13,  5.60s/it]
 16%|█▋        | 36739/225000 [80:24:45<283:23:44,  5.42s/it]
 16%|█▋        | 36740/225000 [80:24:53<321:22:22,  6.15s/it]
 16%|█▋        | 36741/225000 [80:25:06<416:38:42,  7.97s/it]
 16%|█▋        | 36742/225000 [80:25:09<346:23:16,  6.62s/it]
 16%|█▋        | 36743/225000 [80:25:18<377:20:53,  7.22s/it]
 16%|█▋        | 36744/225000 [80:25:22<333:59:34,  6.39s/it]
 16%|█▋        | 36745/225000 [80:25:27<311:29:55,  5.96s/it]
 16%|█▋        | 36746/225000 [80:25:33<305:34:02,  5.84s/it]
 16%|█▋        | 36747/225000 [80:25:38<291:04:21,  5.57s/it]
 16%|█▋        | 36748/225000 [80:25:43<282:27:51,  5.40s/it]
 16%|█▋        | 36749/225000 [80:25:55<390:11:21,  7.46s/it]
 16%|█▋        | 36750/225000 [80:26:02<386:36:47,  7.39s/it]
                                                             
{'loss': '0.6968', 'grad_norm': '0.7929', 'learning_rate': '4.812e-05', 'epoch': '0.1633', 'num_input_tokens_seen': 136523808, 'train_runtime': '2.896e+05', 'train_tokens_per_second': '471.5'}

 16%|█▋        | 36750/225000 [80:26:02<386:36:47,  7.39s/it]
 16%|█▋        | 36751/225000 [80:26:06<337:56:45,  6.46s/it]
 16%|█▋        | 36752/225000 [80:26:15<364:41:30,  6.97s/it]
 16%|█▋        | 36753/225000 [80:26:21<358:06:04,  6.85s/it]
 16%|█▋        | 36754/225000 [80:26:27<347:52:35,  6.65s/it]
 16%|█▋        | 36755/225000 [80:26:34<347:00:13,  6.64s/it]
 16%|█▋        | 36756/225000 [80:26:39<323:18:32,  6.18s/it]
 16%|█▋        | 36757/225000 [80:26:43<294:09:30,  5.63s/it]
 16%|█▋        | 36758/225000 [80:26:54<366:35:08,  7.01s/it]
 16%|█▋        | 36759/225000 [80:27:03<401:39:54,  7.68s/it]
 16%|█▋        | 36760/225000 [80:27:08<366:58:51,  7.02s/it]
 16%|█▋        | 36761/225000 [80:27:13<330:26:28,  6.32s/it]
 16%|█▋        | 36762/225000 [80:27:19<321:06:31,  6.14s/it]
 16%|█▋        | 36763/225000 [80:27:24<313:12:03,  5.99s/it]
 16%|█▋        | 36764/225000 [80:27:30<307:55:57,  5.89s/it]
 16%|█▋        | 36765/225000 [80:27:36<309:51:12,  5.93s/it]
 16%|█▋        | 36766/225000 [80:27:42<310:11:44,  5.93s/it]
 16%|█▋        | 36767/225000 [80:27:54<407:14:29,  7.79s/it]
 16%|█▋        | 36768/225000 [80:28:00<385:55:06,  7.38s/it]
 16%|█▋        | 36769/225000 [80:28:05<342:44:32,  6.56s/it]
 16%|█▋        | 36770/225000 [80:28:10<311:05:37,  5.95s/it]
 16%|█▋        | 36771/225000 [80:28:18<356:11:12,  6.81s/it]
 16%|█▋        | 36772/225000 [80:28:27<383:03:45,  7.33s/it]
 16%|█▋        | 36773/225000 [80:28:31<335:54:16,  6.42s/it]
 16%|█▋        | 36774/225000 [80:28:36<314:39:19,  6.02s/it]
 16%|█▋        | 36775/225000 [80:28:43<318:08:22,  6.08s/it]
 16%|█▋        | 36776/225000 [80:28:53<378:59:30,  7.25s/it]
 16%|█▋        | 36777/225000 [80:28:57<332:47:33,  6.37s/it]
 16%|█▋        | 36778/225000 [80:29:06<373:07:48,  7.14s/it]
 16%|█▋        | 36779/225000 [80:29:12<355:59:15,  6.81s/it]
 16%|█▋        | 36780/225000 [80:29:19<353:23:22,  6.76s/it]
 16%|█▋        | 36781/225000 [80:29:24<332:21:41,  6.36s/it]
 16%|█▋        | 36782/225000 [80:29:29<317:08:28,  6.07s/it]
 16%|█▋        | 36783/225000 [80:29:35<305:10:12,  5.84s/it]
 16%|█▋        | 36784/225000 [80:29:40<291:26:52,  5.57s/it]
 16%|█▋        | 36785/225000 [80:29:45<286:00:50,  5.47s/it]
 16%|█▋        | 36786/225000 [80:29:50<278:28:47,  5.33s/it]
 16%|█▋        | 36787/225000 [80:30:00<358:10:57,  6.85s/it]
 16%|█▋        | 36788/225000 [80:30:13<448:45:22,  8.58s/it]
 16%|█▋        | 36789/225000 [80:30:17<384:30:22,  7.35s/it]
 16%|█▋        | 36790/225000 [80:30:23<351:30:07,  6.72s/it]
 16%|█▋        | 36791/225000 [80:30:28<327:09:24,  6.26s/it]
 16%|█▋        | 36792/225000 [80:30:33<315:55:39,  6.04s/it]
 16%|█▋        | 36793/225000 [80:30:39<310:42:41,  5.94s/it]
 16%|█▋        | 36794/225000 [80:30:45<306:01:30,  5.85s/it]
 16%|█▋        | 36795/225000 [80:30:50<297:16:48,  5.69s/it]
 16%|█▋        | 36796/225000 [80:30:59<346:37:10,  6.63s/it]
 16%|█▋        | 36797/225000 [80:31:08<391:46:08,  7.49s/it]
 16%|█▋        | 36798/225000 [80:31:13<343:51:29,  6.58s/it]
 16%|█▋        | 36799/225000 [80:31:18<317:56:34,  6.08s/it]
 16%|█▋        | 36800/225000 [80:31:26<355:50:20,  6.81s/it]
                                                             
{'loss': '0.7082', 'grad_norm': '0.7856', 'learning_rate': '4.811e-05', 'epoch': '0.1636', 'num_input_tokens_seen': 136668960, 'train_runtime': '2.899e+05', 'train_tokens_per_second': '471.5'}

 16%|█▋        | 36800/225000 [80:31:26<355:50:20,  6.81s/it]
 16%|█▋        | 36801/225000 [80:31:37<420:48:13,  8.05s/it]
 16%|█▋        | 36802/225000 [80:31:43<383:38:50,  7.34s/it]
 16%|█▋        | 36803/225000 [80:31:50<380:49:05,  7.28s/it]
 16%|█▋        | 36804/225000 [80:31:55<339:33:36,  6.50s/it]
 16%|█▋        | 36805/225000 [80:32:08<443:05:07,  8.48s/it]
 16%|█▋        | 36806/225000 [80:32:13<400:50:43,  7.67s/it]
 16%|█▋        | 36807/225000 [80:32:19<366:20:28,  7.01s/it]
 16%|█▋        | 36808/225000 [80:32:23<326:20:00,  6.24s/it]
 16%|█▋        | 36809/225000 [80:32:28<306:18:00,  5.86s/it]
 16%|█▋        | 36810/225000 [80:32:39<374:54:09,  7.17s/it]
 16%|█▋        | 36811/225000 [80:32:45<369:20:02,  7.07s/it]
 16%|█▋        | 36812/225000 [80:32:49<321:22:42,  6.15s/it]
 16%|█▋        | 36813/225000 [80:32:59<380:12:25,  7.27s/it]
 16%|█▋        | 36814/225000 [80:33:05<353:18:15,  6.76s/it]
 16%|█▋        | 36815/225000 [80:33:10<335:11:32,  6.41s/it]
 16%|█▋        | 36816/225000 [80:33:16<327:51:09,  6.27s/it]
 16%|█▋        | 36817/225000 [80:33:21<295:47:25,  5.66s/it]
 16%|█▋        | 36818/225000 [80:33:26<290:23:40,  5.56s/it]
 16%|█▋        | 36819/225000 [80:33:31<283:15:55,  5.42s/it]
 16%|█▋        | 36820/225000 [80:33:36<282:40:15,  5.41s/it]
 16%|█▋        | 36821/225000 [80:33:42<281:00:55,  5.38s/it]
 16%|█▋        | 36822/225000 [80:33:47<282:12:11,  5.40s/it]
 16%|█▋        | 36823/225000 [80:33:57<353:19:17,  6.76s/it]
 16%|█▋        | 36824/225000 [80:34:09<433:17:33,  8.29s/it]
 16%|█▋        | 36825/225000 [80:34:13<373:11:13,  7.14s/it]
 16%|█▋        | 36826/225000 [80:34:21<383:58:35,  7.35s/it]
 16%|█▋        | 36827/225000 [80:34:26<336:47:24,  6.44s/it]
 16%|█▋        | 36828/225000 [80:34:35<387:47:48,  7.42s/it]
 16%|█▋        | 36829/225000 [80:34:40<343:42:47,  6.58s/it]
 16%|█▋        | 36830/225000 [80:34:45<313:57:01,  6.01s/it]
 16%|█▋        | 36831/225000 [80:34:58<423:21:07,  8.10s/it]
 16%|█▋        | 36832/225000 [80:35:04<404:24:17,  7.74s/it]
 16%|█▋        | 36833/225000 [80:35:11<386:05:19,  7.39s/it]
 16%|█▋        | 36834/225000 [80:35:16<355:46:43,  6.81s/it]
 16%|█▋        | 36835/225000 [80:35:20<307:51:01,  5.89s/it]
 16%|█▋        | 36836/225000 [80:35:29<354:01:46,  6.77s/it]
 16%|█▋        | 36837/225000 [80:35:34<329:03:59,  6.30s/it]
 16%|█▋        | 36838/225000 [80:35:41<342:36:55,  6.56s/it]
 16%|█▋        | 36839/225000 [80:35:49<357:39:25,  6.84s/it]
 16%|█▋        | 36840/225000 [80:35:55<346:59:27,  6.64s/it]
 16%|█▋        | 36841/225000 [80:36:01<327:54:25,  6.27s/it]
 16%|█▋        | 36842/225000 [80:36:06<315:42:43,  6.04s/it]
 16%|█▋        | 36843/225000 [80:36:12<309:06:53,  5.91s/it]
 16%|█▋        | 36844/225000 [80:36:19<331:58:25,  6.35s/it]
 16%|█▋        | 36845/225000 [80:36:24<317:48:12,  6.08s/it]
 16%|█▋        | 36846/225000 [80:36:30<307:04:09,  5.88s/it]
 16%|█▋        | 36847/225000 [80:36:35<302:17:35,  5.78s/it]
 16%|█▋        | 36848/225000 [80:36:41<299:48:10,  5.74s/it]
 16%|█▋        | 36849/225000 [80:36:48<325:13:34,  6.22s/it]
 16%|█▋        | 36850/225000 [80:36:54<313:08:58,  5.99s/it]
                                                             
{'loss': '0.6969', 'grad_norm': '0.9641', 'learning_rate': '4.81e-05', 'epoch': '0.1638', 'num_input_tokens_seen': 136815880, 'train_runtime': '2.902e+05', 'train_tokens_per_second': '471.4'}

 16%|█▋        | 36850/225000 [80:36:54<313:08:58,  5.99s/it]
 16%|█▋        | 36851/225000 [80:37:00<311:55:30,  5.97s/it]
 16%|█▋        | 36852/225000 [80:37:08<350:08:17,  6.70s/it]
 16%|█▋        | 36853/225000 [80:37:19<418:52:08,  8.01s/it]
 16%|█▋        | 36854/225000 [80:37:23<359:20:28,  6.88s/it]
 16%|█▋        | 36855/225000 [80:37:28<319:09:23,  6.11s/it]
 16%|█▋        | 36856/225000 [80:37:32<292:03:52,  5.59s/it]
 16%|█▋        | 36857/225000 [80:37:44<388:39:55,  7.44s/it]
 16%|█▋        | 36858/225000 [80:37:51<383:23:45,  7.34s/it]
 16%|█▋        | 36859/225000 [80:37:56<353:41:48,  6.77s/it]
 16%|█▋        | 36860/225000 [80:38:02<334:37:00,  6.40s/it]
 16%|█▋        | 36861/225000 [80:38:07<319:18:37,  6.11s/it]
 16%|█▋        | 36862/225000 [80:38:12<302:25:13,  5.79s/it]
 16%|█▋        | 36863/225000 [80:38:18<301:19:17,  5.77s/it]
 16%|█▋        | 36864/225000 [80:38:28<363:10:48,  6.95s/it]
 16%|█▋        | 36865/225000 [80:38:35<365:57:33,  7.00s/it]
 16%|█▋        | 36866/225000 [80:38:43<375:13:12,  7.18s/it]
 16%|█▋        | 36867/225000 [80:38:46<321:47:52,  6.16s/it]
 16%|█▋        | 36868/225000 [80:38:55<362:08:15,  6.93s/it]
 16%|█▋        | 36869/225000 [80:39:03<378:00:25,  7.23s/it]
 16%|█▋        | 36870/225000 [80:39:07<320:46:40,  6.14s/it]
 16%|█▋        | 36871/225000 [80:39:10<282:42:43,  5.41s/it]
 16%|█▋        | 36872/225000 [80:39:20<351:18:43,  6.72s/it]
 16%|█▋        | 36873/225000 [80:39:24<305:35:42,  5.85s/it]
 16%|█▋        | 36874/225000 [80:39:33<362:59:33,  6.95s/it]
 16%|█▋        | 36875/225000 [80:39:38<320:40:05,  6.14s/it]
 16%|█▋        | 36876/225000 [80:39:42<299:19:24,  5.73s/it]
 16%|█▋        | 36877/225000 [80:39:47<282:40:09,  5.41s/it]
 16%|█▋        | 36878/225000 [80:39:59<381:40:39,  7.30s/it]
 16%|█▋        | 36879/225000 [80:40:05<356:44:20,  6.83s/it]
 16%|█▋        | 36880/225000 [80:40:12<366:29:14,  7.01s/it]
 16%|█▋        | 36881/225000 [80:40:20<384:42:27,  7.36s/it]
 16%|█▋        | 36882/225000 [80:40:25<338:27:28,  6.48s/it]
 16%|█▋        | 36883/225000 [80:40:30<324:16:11,  6.21s/it]
 16%|█▋        | 36884/225000 [80:40:36<314:49:09,  6.02s/it]
 16%|█▋        | 36885/225000 [80:40:43<328:04:09,  6.28s/it]
 16%|█▋        | 36886/225000 [80:40:50<346:03:49,  6.62s/it]
 16%|█▋        | 36887/225000 [80:40:54<304:43:16,  5.83s/it]
 16%|█▋        | 36888/225000 [80:41:03<358:23:06,  6.86s/it]
 16%|█▋        | 36889/225000 [80:41:10<361:02:09,  6.91s/it]
 16%|█▋        | 36890/225000 [80:41:15<325:24:57,  6.23s/it]
 16%|█▋        | 36891/225000 [80:41:19<294:14:35,  5.63s/it]
 16%|█▋        | 36892/225000 [80:41:24<282:40:26,  5.41s/it]
 16%|█▋        | 36893/225000 [80:41:37<394:02:47,  7.54s/it]
 16%|█▋        | 36894/225000 [80:41:44<386:13:35,  7.39s/it]
 16%|█▋        | 36895/225000 [80:41:47<325:08:54,  6.22s/it]
 16%|█▋        | 36896/225000 [80:41:54<339:15:55,  6.49s/it]
 16%|█▋        | 36897/225000 [80:41:59<316:00:02,  6.05s/it]
 16%|█▋        | 36898/225000 [80:42:07<342:16:46,  6.55s/it]
 16%|█▋        | 36899/225000 [80:42:14<351:47:21,  6.73s/it]
 16%|█▋        | 36900/225000 [80:42:19<315:46:33,  6.04s/it]
                                                             
{'loss': '0.6993', 'grad_norm': '0.7032', 'learning_rate': '4.81e-05', 'epoch': '0.164', 'num_input_tokens_seen': 136963048, 'train_runtime': '2.905e+05', 'train_tokens_per_second': '471.4'}

 16%|█▋        | 36900/225000 [80:42:19<315:46:33,  6.04s/it]
 16%|█▋        | 36901/225000 [80:42:23<290:59:32,  5.57s/it]
 16%|█▋        | 36902/225000 [80:42:30<304:21:28,  5.83s/it]
 16%|█▋        | 36903/225000 [80:42:40<374:31:30,  7.17s/it]
 16%|█▋        | 36904/225000 [80:42:44<325:48:51,  6.24s/it]
 16%|█▋        | 36905/225000 [80:42:49<312:34:52,  5.98s/it]
 16%|█▋        | 36906/225000 [80:42:59<369:35:13,  7.07s/it]
 16%|█▋        | 36907/225000 [80:43:03<319:43:25,  6.12s/it]
 16%|█▋        | 36908/225000 [80:43:06<280:46:38,  5.37s/it]
 16%|█▋        | 36909/225000 [80:43:18<370:57:02,  7.10s/it]
 16%|█▋        | 36910/225000 [80:43:22<329:48:15,  6.31s/it]
 16%|█▋        | 36911/225000 [80:43:27<306:00:58,  5.86s/it]
 16%|█▋        | 36912/225000 [80:43:38<395:19:01,  7.57s/it]
 16%|█▋        | 36913/225000 [80:43:43<342:24:50,  6.55s/it]
 16%|█▋        | 36914/225000 [80:43:47<310:01:55,  5.93s/it]
 16%|█▋        | 36915/225000 [80:43:58<384:45:48,  7.36s/it]
 16%|█▋        | 36916/225000 [80:44:03<343:58:07,  6.58s/it]
 16%|█▋        | 36917/225000 [80:44:09<342:31:17,  6.56s/it]
 16%|█▋        | 36918/225000 [80:44:16<351:33:52,  6.73s/it]
 16%|█▋        | 36919/225000 [80:44:21<321:56:03,  6.16s/it]
 16%|█▋        | 36920/225000 [80:44:25<294:55:01,  5.64s/it]
 16%|█▋        | 36921/225000 [80:44:33<317:48:07,  6.08s/it]
 16%|█▋        | 36922/225000 [80:44:42<372:25:15,  7.13s/it]
 16%|█▋        | 36923/225000 [80:44:46<314:59:55,  6.03s/it]
 16%|█▋        | 36924/225000 [80:44:50<289:02:07,  5.53s/it]
 16%|█▋        | 36925/225000 [80:45:00<357:09:29,  6.84s/it]
 16%|█▋        | 36926/225000 [80:45:04<310:28:52,  5.94s/it]
 16%|█▋        | 36927/225000 [80:45:13<356:27:44,  6.82s/it]
 16%|█▋        | 36928/225000 [80:45:20<367:54:25,  7.04s/it]
 16%|█▋        | 36929/225000 [80:45:24<317:50:13,  6.08s/it]
 16%|█▋        | 36930/225000 [80:45:33<370:53:02,  7.10s/it]
 16%|█▋        | 36931/225000 [80:45:37<316:31:35,  6.06s/it]
 16%|█▋        | 36932/225000 [80:45:41<289:36:25,  5.54s/it]
 16%|█▋        | 36933/225000 [80:45:51<355:25:54,  6.80s/it]
 16%|█▋        | 36934/225000 [80:45:56<327:32:07,  6.27s/it]
 16%|█▋        | 36935/225000 [80:46:01<303:07:25,  5.80s/it]
 16%|█▋        | 36936/225000 [80:46:09<333:54:43,  6.39s/it]
 16%|█▋        | 36937/225000 [80:46:17<372:01:16,  7.12s/it]
 16%|█▋        | 36938/225000 [80:46:24<370:22:43,  7.09s/it]
 16%|█▋        | 36939/225000 [80:46:28<315:45:33,  6.04s/it]
 16%|█▋        | 36940/225000 [80:46:37<364:07:06,  6.97s/it]
 16%|█▋        | 36941/225000 [80:46:44<363:54:06,  6.97s/it]
 16%|█▋        | 36942/225000 [80:46:49<337:27:32,  6.46s/it]
 16%|█▋        | 36943/225000 [80:46:55<320:32:01,  6.14s/it]
 16%|█▋        | 36944/225000 [80:47:00<313:03:41,  5.99s/it]
 16%|█▋        | 36945/225000 [80:47:05<296:11:01,  5.67s/it]
 16%|█▋        | 36946/225000 [80:47:10<284:58:09,  5.46s/it]
 16%|█▋        | 36947/225000 [80:47:16<283:39:56,  5.43s/it]
 16%|█▋        | 36948/225000 [80:47:25<345:01:32,  6.61s/it]
 16%|█▋        | 36949/225000 [80:47:35<399:30:43,  7.65s/it]
 16%|█▋        | 36950/225000 [80:47:42<394:16:38,  7.55s/it]
                                                             
{'loss': '0.7285', 'grad_norm': '0.7415', 'learning_rate': '4.809e-05', 'epoch': '0.1642', 'num_input_tokens_seen': 137108448, 'train_runtime': '2.909e+05', 'train_tokens_per_second': '471.4'}

 16%|█▋        | 36950/225000 [80:47:42<394:16:38,  7.55s/it]
 16%|█▋        | 36951/225000 [80:47:46<335:17:34,  6.42s/it]
 16%|█▋        | 36952/225000 [80:47:56<382:07:35,  7.32s/it]
 16%|█▋        | 36953/225000 [80:48:01<353:49:15,  6.77s/it]
 16%|█▋        | 36954/225000 [80:48:06<327:44:02,  6.27s/it]
 16%|█▋        | 36955/225000 [80:48:13<336:54:07,  6.45s/it]
 16%|█▋        | 36956/225000 [80:48:18<306:04:01,  5.86s/it]
 16%|█▋        | 36957/225000 [80:48:22<282:46:25,  5.41s/it]
 16%|█▋        | 36958/225000 [80:48:34<386:27:22,  7.40s/it]
 16%|█▋        | 36959/225000 [80:48:37<321:57:41,  6.16s/it]
 16%|█▋        | 36960/225000 [80:48:47<374:19:36,  7.17s/it]
 16%|█▋        | 36961/225000 [80:48:50<317:08:02,  6.07s/it]
 16%|█▋        | 36962/225000 [80:49:00<377:10:31,  7.22s/it]
 16%|█▋        | 36963/225000 [80:49:05<339:03:25,  6.49s/it]
 16%|█▋        | 36964/225000 [80:49:10<308:49:10,  5.91s/it]
 16%|█▋        | 36965/225000 [80:49:19<365:22:34,  7.00s/it]
 16%|█▋        | 36966/225000 [80:49:25<354:42:37,  6.79s/it]
 16%|█▋        | 36967/225000 [80:49:29<308:33:05,  5.91s/it]
 16%|█▋        | 36968/225000 [80:49:40<380:35:34,  7.29s/it]
 16%|█▋        | 36969/225000 [80:49:44<332:09:39,  6.36s/it]
 16%|█▋        | 36970/225000 [80:49:48<297:27:47,  5.70s/it]
 16%|█▋        | 36971/225000 [80:50:00<388:36:40,  7.44s/it]
 16%|█▋        | 36972/225000 [80:50:03<332:26:54,  6.37s/it]
 16%|█▋        | 36973/225000 [80:50:08<297:59:07,  5.71s/it]
 16%|█▋        | 36974/225000 [80:50:18<370:29:19,  7.09s/it]
 16%|█▋        | 36975/225000 [80:50:22<318:03:53,  6.09s/it]
 16%|█▋        | 36976/225000 [80:50:27<304:02:52,  5.82s/it]
 16%|█▋        | 36977/225000 [80:50:33<303:06:10,  5.80s/it]
 16%|█▋        | 36978/225000 [80:50:38<297:03:51,  5.69s/it]
 16%|█▋        | 36979/225000 [80:50:43<292:23:27,  5.60s/it]
 16%|█▋        | 36980/225000 [80:50:49<289:35:22,  5.54s/it]
 16%|█▋        | 36981/225000 [80:51:00<380:07:34,  7.28s/it]
 16%|█▋        | 36982/225000 [80:51:07<368:37:33,  7.06s/it]
 16%|█▋        | 36983/225000 [80:51:15<388:46:58,  7.44s/it]
 16%|█▋        | 36984/225000 [80:51:19<339:28:41,  6.50s/it]
 16%|█▋        | 36985/225000 [80:51:27<348:43:48,  6.68s/it]
 16%|█▋        | 36986/225000 [80:51:36<388:57:57,  7.45s/it]
 16%|█▋        | 36987/225000 [80:51:40<344:03:51,  6.59s/it]
 16%|█▋        | 36988/225000 [80:51:45<319:36:15,  6.12s/it]
 16%|█▋        | 36989/225000 [80:51:50<303:51:07,  5.82s/it]
 16%|█▋        | 36990/225000 [80:51:56<296:57:38,  5.69s/it]
 16%|█▋        | 36991/225000 [80:52:09<406:22:15,  7.78s/it]
 16%|█▋        | 36992/225000 [80:52:15<391:23:34,  7.49s/it]
 16%|█▋        | 36993/225000 [80:52:19<337:07:07,  6.46s/it]
 16%|█▋        | 36994/225000 [80:52:29<389:25:24,  7.46s/it]
 16%|█▋        | 36995/225000 [80:52:33<330:49:44,  6.33s/it]
 16%|█▋        | 36996/225000 [80:52:38<318:01:17,  6.09s/it]
 16%|█▋        | 36997/225000 [80:52:47<363:29:40,  6.96s/it]
 16%|█▋        | 36998/225000 [80:52:53<343:35:00,  6.58s/it]
 16%|█▋        | 36999/225000 [80:52:58<319:28:13,  6.12s/it]
 16%|█▋        | 37000/225000 [80:53:04<311:36:38,  5.97s/it]
                                                             
{'loss': '0.6772', 'grad_norm': '0.8138', 'learning_rate': '4.808e-05', 'epoch': '0.1644', 'num_input_tokens_seen': 137254408, 'train_runtime': '2.912e+05', 'train_tokens_per_second': '471.4'}

 16%|█▋        | 37000/225000 [80:53:04<311:36:38,  5.97s/it]
 16%|█▋        | 37001/225000 [80:53:09<300:35:36,  5.76s/it]
 16%|█▋        | 37002/225000 [80:53:15<301:35:22,  5.78s/it]
 16%|█▋        | 37003/225000 [80:53:20<294:10:13,  5.63s/it]
 16%|█▋        | 37004/225000 [80:53:26<293:01:36,  5.61s/it]
 16%|█▋        | 37005/225000 [80:53:38<401:20:46,  7.69s/it]
 16%|█▋        | 37006/225000 [80:53:45<394:05:16,  7.55s/it]
 16%|█▋        | 37007/225000 [80:53:54<411:07:03,  7.87s/it]
 16%|█▋        | 37008/225000 [80:54:02<404:02:33,  7.74s/it]
 16%|█▋        | 37009/225000 [80:54:07<375:22:01,  7.19s/it]
 16%|█▋        | 37010/225000 [80:54:14<369:30:27,  7.08s/it]
 16%|█▋        | 37011/225000 [80:54:20<350:19:02,  6.71s/it]
 16%|█▋        | 37012/225000 [80:54:27<354:47:10,  6.79s/it]
 16%|█▋        | 37013/225000 [80:54:33<339:43:43,  6.51s/it]
 16%|█▋        | 37014/225000 [80:54:40<348:26:02,  6.67s/it]
 16%|█▋        | 37015/225000 [80:54:44<306:15:34,  5.87s/it]
 16%|█▋        | 37016/225000 [80:54:49<291:56:36,  5.59s/it]
 16%|█▋        | 37017/225000 [80:54:54<285:26:31,  5.47s/it]
 16%|█▋        | 37018/225000 [80:55:00<287:13:45,  5.50s/it]
 16%|█▋        | 37019/225000 [80:55:06<294:37:55,  5.64s/it]
 16%|█▋        | 37020/225000 [80:55:11<290:35:09,  5.57s/it]
 16%|█▋        | 37021/225000 [80:55:22<380:23:07,  7.28s/it]
 16%|█▋        | 37022/225000 [80:55:32<422:32:36,  8.09s/it]
 16%|█▋        | 37023/225000 [80:55:36<356:08:35,  6.82s/it]
 16%|█▋        | 37024/225000 [80:55:46<400:36:56,  7.67s/it]
 16%|█▋        | 37025/225000 [80:55:52<374:07:25,  7.17s/it]
 16%|█▋        | 37026/225000 [80:55:57<337:55:36,  6.47s/it]
 16%|█▋        | 37027/225000 [80:56:05<371:48:33,  7.12s/it]
 16%|█▋        | 37028/225000 [80:56:09<319:41:22,  6.12s/it]
 16%|█▋        | 37029/225000 [80:56:17<348:24:50,  6.67s/it]
 16%|█▋        | 37030/225000 [80:56:25<365:06:17,  6.99s/it]
 16%|█▋        | 37031/225000 [80:56:29<315:10:34,  6.04s/it]
 16%|█▋        | 37032/225000 [80:56:37<346:44:52,  6.64s/it]
 16%|█▋        | 37033/225000 [80:56:41<315:06:23,  6.04s/it]
 16%|█▋        | 37034/225000 [80:56:49<349:08:45,  6.69s/it]
 16%|█▋        | 37035/225000 [80:56:54<321:29:52,  6.16s/it]
 16%|█▋        | 37036/225000 [80:56:59<303:55:54,  5.82s/it]
 16%|█▋        | 37037/225000 [80:57:04<289:21:57,  5.54s/it]
 16%|█▋        | 37038/225000 [80:57:10<286:02:13,  5.48s/it]
 16%|█▋        | 37039/225000 [80:57:15<278:44:44,  5.34s/it]
 16%|█▋        | 37040/225000 [80:57:27<390:30:11,  7.48s/it]
 16%|█▋        | 37041/225000 [80:57:32<348:18:22,  6.67s/it]
 16%|█▋        | 37042/225000 [80:57:36<309:23:41,  5.93s/it]
 16%|█▋        | 37043/225000 [80:57:45<352:47:30,  6.76s/it]
 16%|█▋        | 37044/225000 [80:57:52<353:28:09,  6.77s/it]
 16%|█▋        | 37045/225000 [80:57:59<362:39:08,  6.95s/it]
 16%|█▋        | 37046/225000 [80:58:04<328:54:29,  6.30s/it]
 16%|█▋        | 37047/225000 [80:58:09<316:32:43,  6.06s/it]
 16%|█▋        | 37048/225000 [80:58:14<303:23:02,  5.81s/it]
 16%|█▋        | 37049/225000 [80:58:20<293:06:16,  5.61s/it]
 16%|█▋        | 37050/225000 [80:58:24<276:28:01,  5.30s/it]
                                                             
{'loss': '0.6825', 'grad_norm': '0.7433', 'learning_rate': '4.807e-05', 'epoch': '0.1647', 'num_input_tokens_seen': 137400200, 'train_runtime': '2.915e+05', 'train_tokens_per_second': '471.3'}

 16%|█▋        | 37050/225000 [80:58:24<276:28:01,  5.30s/it]
 16%|█▋        | 37051/225000 [80:58:33<339:00:35,  6.49s/it]
 16%|█▋        | 37052/225000 [80:58:44<410:03:48,  7.85s/it]
 16%|█▋        | 37053/225000 [80:58:49<350:24:21,  6.71s/it]
 16%|█▋        | 37054/225000 [80:58:59<409:20:55,  7.84s/it]
 16%|█▋        | 37055/225000 [80:59:02<339:00:15,  6.49s/it]
 16%|█▋        | 37056/225000 [80:59:08<325:04:53,  6.23s/it]
 16%|█▋        | 37057/225000 [80:59:14<320:32:26,  6.14s/it]
 16%|█▋        | 37058/225000 [80:59:20<312:11:12,  5.98s/it]
 16%|█▋        | 37059/225000 [80:59:25<304:58:13,  5.84s/it]
 16%|█▋        | 37060/225000 [80:59:31<307:27:39,  5.89s/it]
 16%|█▋        | 37061/225000 [80:59:37<301:13:28,  5.77s/it]
 16%|█▋        | 37062/225000 [80:59:43<309:37:36,  5.93s/it]
 16%|█▋        | 37063/225000 [80:59:57<431:34:30,  8.27s/it]
 16%|█▋        | 37064/225000 [81:00:04<418:36:40,  8.02s/it]
 16%|█▋        | 37065/225000 [81:00:11<402:19:17,  7.71s/it]
 16%|█▋        | 37066/225000 [81:00:16<355:02:11,  6.80s/it]
 16%|█▋        | 37067/225000 [81:00:20<322:40:37,  6.18s/it]
 16%|█▋        | 37068/225000 [81:00:25<299:45:43,  5.74s/it]
 16%|█▋        | 37069/225000 [81:00:30<284:26:35,  5.45s/it]
 16%|█▋        | 37070/225000 [81:00:41<375:17:52,  7.19s/it]
 16%|█▋        | 37071/225000 [81:00:48<366:50:34,  7.03s/it]
 16%|█▋        | 37072/225000 [81:00:53<338:41:23,  6.49s/it]
 16%|█▋        | 37073/225000 [81:00:59<323:59:36,  6.21s/it]
 16%|█▋        | 37074/225000 [81:01:04<314:58:57,  6.03s/it]
 16%|█▋        | 37075/225000 [81:01:09<302:58:17,  5.80s/it]
 16%|█▋        | 37076/225000 [81:01:14<287:33:26,  5.51s/it]
 16%|█▋        | 37077/225000 [81:01:20<287:17:29,  5.50s/it]
 16%|█▋        | 37078/225000 [81:01:30<361:19:36,  6.92s/it]
 16%|█▋        | 37079/225000 [81:01:38<383:53:48,  7.35s/it]
 16%|█▋        | 37080/225000 [81:01:46<389:30:59,  7.46s/it]
 16%|█▋        | 37081/225000 [81:01:51<342:16:00,  6.56s/it]
 16%|█▋        | 37082/225000 [81:01:55<307:28:02,  5.89s/it]
 16%|█▋        | 37083/225000 [81:01:59<279:17:25,  5.35s/it]
 16%|█▋        | 37084/225000 [81:02:03<266:38:21,  5.11s/it]
 16%|█▋        | 37085/225000 [81:02:08<258:36:36,  4.95s/it]
 16%|█▋        | 37086/225000 [81:02:21<382:02:14,  7.32s/it]
 16%|█▋        | 37087/225000 [81:02:28<379:45:19,  7.28s/it]
 16%|█▋        | 37088/225000 [81:02:34<363:13:27,  6.96s/it]
 16%|█▋        | 37089/225000 [81:02:41<366:31:18,  7.02s/it]
 16%|█▋        | 37090/225000 [81:02:49<379:21:20,  7.27s/it]
 16%|█▋        | 37091/225000 [81:02:54<337:56:38,  6.47s/it]
 16%|█▋        | 37092/225000 [81:02:58<306:11:00,  5.87s/it]
 16%|█▋        | 37093/225000 [81:03:03<285:15:30,  5.47s/it]
 16%|█▋        | 37094/225000 [81:03:13<350:17:21,  6.71s/it]
 16%|█▋        | 37095/225000 [81:03:21<375:27:00,  7.19s/it]
 16%|█▋        | 37096/225000 [81:03:25<330:08:55,  6.33s/it]
 16%|█▋        | 37097/225000 [81:03:37<411:36:40,  7.89s/it]
 16%|█▋        | 37098/225000 [81:03:41<355:15:09,  6.81s/it]
 16%|█▋        | 37099/225000 [81:03:49<372:35:42,  7.14s/it]
 16%|█▋        | 37100/225000 [81:03:52<313:05:50,  6.00s/it]
                                                             
{'loss': '0.7014', 'grad_norm': '0.7433', 'learning_rate': '4.807e-05', 'epoch': '0.1649', 'num_input_tokens_seen': 137547048, 'train_runtime': '2.918e+05', 'train_tokens_per_second': '471.3'}

 16%|█▋        | 37100/225000 [81:03:52<313:05:50,  6.00s/it]
 16%|█▋        | 37101/225000 [81:04:02<370:04:06,  7.09s/it]
 16%|█▋        | 37102/225000 [81:04:06<328:06:45,  6.29s/it]
 16%|█▋        | 37103/225000 [81:04:12<318:37:53,  6.10s/it]
 16%|█▋        | 37104/225000 [81:04:18<313:14:42,  6.00s/it]
 16%|█▋        | 37105/225000 [81:04:29<393:15:02,  7.53s/it]
 16%|█▋        | 37106/225000 [81:04:35<374:56:27,  7.18s/it]
 16%|█▋        | 37107/225000 [81:04:39<325:45:18,  6.24s/it]
 16%|█▋        | 37108/225000 [81:04:44<299:51:37,  5.75s/it]
 16%|█▋        | 37109/225000 [81:04:49<286:08:58,  5.48s/it]
 16%|█▋        | 37110/225000 [81:05:00<381:01:23,  7.30s/it]
 16%|█▋        | 37111/225000 [81:05:04<331:56:33,  6.36s/it]
 16%|█▋        | 37112/225000 [81:05:11<341:03:31,  6.53s/it]
 16%|█▋        | 37113/225000 [81:05:21<384:00:02,  7.36s/it]
 16%|█▋        | 37114/225000 [81:05:26<347:17:15,  6.65s/it]
 16%|█▋        | 37115/225000 [81:05:33<350:50:56,  6.72s/it]
 16%|█▋        | 37116/225000 [81:05:38<332:05:47,  6.36s/it]
 16%|█▋        | 37117/225000 [81:05:43<317:43:24,  6.09s/it]
 16%|█▋        | 37118/225000 [81:05:49<302:54:16,  5.80s/it]
 16%|█▋        | 37119/225000 [81:05:54<299:27:40,  5.74s/it]
 16%|█▋        | 37120/225000 [81:06:08<421:35:13,  8.08s/it]
 16%|█▋        | 37121/225000 [81:06:13<382:23:11,  7.33s/it]
 16%|█▋        | 37122/225000 [81:06:21<390:56:56,  7.49s/it]
 16%|█▋        | 37123/225000 [81:06:26<351:36:33,  6.74s/it]
 16%|█▋        | 37124/225000 [81:06:31<321:54:16,  6.17s/it]
 16%|█▋        | 37125/225000 [81:06:37<322:53:33,  6.19s/it]
 17%|█▋        | 37126/225000 [81:06:48<396:43:50,  7.60s/it]
 17%|█▋        | 37127/225000 [81:06:53<346:23:36,  6.64s/it]
 17%|█▋        | 37128/225000 [81:06:57<313:58:07,  6.02s/it]
 17%|█▋        | 37129/225000 [81:07:02<291:27:55,  5.59s/it]
 17%|█▋        | 37130/225000 [81:07:13<388:58:20,  7.45s/it]
 17%|█▋        | 37131/225000 [81:07:20<372:42:44,  7.14s/it]
 17%|█▋        | 37132/225000 [81:07:24<320:41:48,  6.15s/it]
 17%|█▋        | 37133/225000 [81:07:34<386:12:25,  7.40s/it]
 17%|█▋        | 37134/225000 [81:07:38<331:59:54,  6.36s/it]
 17%|█▋        | 37135/225000 [81:07:42<301:59:57,  5.79s/it]
 17%|█▋        | 37136/225000 [81:07:51<351:05:04,  6.73s/it]
 17%|█▋        | 37137/225000 [81:07:55<305:31:04,  5.85s/it]
 17%|█▋        | 37138/225000 [81:08:00<294:42:47,  5.65s/it]
 17%|█▋        | 37139/225000 [81:08:06<291:36:20,  5.59s/it]
 17%|█▋        | 37140/225000 [81:08:11<293:02:28,  5.62s/it]
 17%|█▋        | 37141/225000 [81:08:25<413:59:44,  7.93s/it]
 17%|█▋        | 37142/225000 [81:08:29<360:14:28,  6.90s/it]
 17%|█▋        | 37143/225000 [81:08:35<338:09:34,  6.48s/it]
 17%|█▋        | 37144/225000 [81:08:40<321:21:03,  6.16s/it]
 17%|█▋        | 37145/225000 [81:08:49<367:00:59,  7.03s/it]
 17%|█▋        | 37146/225000 [81:08:56<359:47:09,  6.89s/it]
 17%|█▋        | 37147/225000 [81:09:01<326:32:37,  6.26s/it]
 17%|█▋        | 37148/225000 [81:09:05<301:29:38,  5.78s/it]
 17%|█▋        | 37149/225000 [81:09:17<401:31:31,  7.69s/it]
 17%|█▋        | 37150/225000 [81:09:22<354:49:10,  6.80s/it]
                                                             
{'loss': '0.6835', 'grad_norm': '0.6555', 'learning_rate': '4.806e-05', 'epoch': '0.1651', 'num_input_tokens_seen': 137694776, 'train_runtime': '2.922e+05', 'train_tokens_per_second': '471.3'}

 17%|█▋        | 37150/225000 [81:09:22<354:49:10,  6.80s/it]
 17%|█▋        | 37151/225000 [81:09:30<371:20:48,  7.12s/it]
 17%|█▋        | 37152/225000 [81:09:33<313:29:46,  6.01s/it]
 17%|█▋        | 37153/225000 [81:09:42<354:34:24,  6.80s/it]
 17%|█▋        | 37154/225000 [81:09:49<362:26:07,  6.95s/it]
 17%|█▋        | 37155/225000 [81:09:53<317:52:49,  6.09s/it]
 17%|█▋        | 37156/225000 [81:09:58<289:34:02,  5.55s/it]
 17%|█▋        | 37157/225000 [81:10:07<347:18:17,  6.66s/it]
 17%|█▋        | 37158/225000 [81:10:14<357:15:56,  6.85s/it]
 17%|█▋        | 37159/225000 [81:10:23<380:57:18,  7.30s/it]
 17%|█▋        | 37160/225000 [81:10:26<326:06:45,  6.25s/it]
 17%|█▋        | 37161/225000 [81:10:31<297:53:48,  5.71s/it]
 17%|█▋        | 37162/225000 [81:10:39<330:58:04,  6.34s/it]
 17%|█▋        | 37163/225000 [81:10:49<385:29:37,  7.39s/it]
 17%|█▋        | 37164/225000 [81:10:53<340:47:23,  6.53s/it]
 17%|█▋        | 37165/225000 [81:10:58<316:35:14,  6.07s/it]
 17%|█▋        | 37166/225000 [81:11:03<302:01:31,  5.79s/it]
 17%|█▋        | 37167/225000 [81:11:09<298:17:34,  5.72s/it]
 17%|█▋        | 37168/225000 [81:11:14<287:44:24,  5.51s/it]
 17%|█▋        | 37169/225000 [81:11:19<282:52:26,  5.42s/it]
 17%|█▋        | 37170/225000 [81:11:33<409:31:34,  7.85s/it]
 17%|█▋        | 37171/225000 [81:11:38<372:30:29,  7.14s/it]
 17%|█▋        | 37172/225000 [81:11:42<320:55:47,  6.15s/it]
 17%|█▋        | 37173/225000 [81:11:52<387:05:44,  7.42s/it]
 17%|█▋        | 37174/225000 [81:11:59<371:00:48,  7.11s/it]
 17%|█▋        | 37175/225000 [81:12:03<332:20:07,  6.37s/it]
 17%|█▋        | 37176/225000 [81:12:12<363:42:30,  6.97s/it]
 17%|█▋        | 37177/225000 [81:12:15<313:33:15,  6.01s/it]
 17%|█▋        | 37178/225000 [81:12:20<288:27:18,  5.53s/it]
 17%|█▋        | 37179/225000 [81:12:30<358:47:08,  6.88s/it]
 17%|█▋        | 37180/225000 [81:12:35<332:57:25,  6.38s/it]
 17%|█▋        | 37181/225000 [81:12:40<305:05:19,  5.85s/it]
 17%|█▋        | 37182/225000 [81:12:45<292:14:15,  5.60s/it]
 17%|█▋        | 37183/225000 [81:12:52<319:38:41,  6.13s/it]
 17%|█▋        | 37184/225000 [81:13:01<359:50:17,  6.90s/it]
 17%|█▋        | 37185/225000 [81:13:06<335:09:45,  6.42s/it]
 17%|█▋        | 37186/225000 [81:13:17<401:29:37,  7.70s/it]
 17%|█▋        | 37187/225000 [81:13:21<354:44:45,  6.80s/it]
 17%|█▋        | 37188/225000 [81:13:30<379:40:24,  7.28s/it]
 17%|█▋        | 37189/225000 [81:13:34<328:17:22,  6.29s/it]
 17%|█▋        | 37190/225000 [81:13:39<315:57:05,  6.06s/it]
 17%|█▋        | 37191/225000 [81:13:45<305:16:58,  5.85s/it]
 17%|█▋        | 37192/225000 [81:13:50<292:18:36,  5.60s/it]
 17%|█▋        | 37193/225000 [81:13:55<290:20:16,  5.57s/it]
 17%|█▋        | 37194/225000 [81:14:01<289:41:27,  5.55s/it]
 17%|█▋        | 37195/225000 [81:14:06<289:46:49,  5.55s/it]
 17%|█▋        | 37196/225000 [81:14:17<372:59:38,  7.15s/it]
 17%|█▋        | 37197/225000 [81:14:27<414:37:02,  7.95s/it]
 17%|█▋        | 37198/225000 [81:14:32<368:00:41,  7.05s/it]
 17%|█▋        | 37199/225000 [81:14:37<340:34:47,  6.53s/it]
 17%|█▋        | 37200/225000 [81:14:43<321:40:21,  6.17s/it]
                                                             
{'loss': '0.6791', 'grad_norm': '0.6609', 'learning_rate': '4.805e-05', 'epoch': '0.1653', 'num_input_tokens_seen': 137840856, 'train_runtime': '2.925e+05', 'train_tokens_per_second': '471.3'}

 17%|█▋        | 37200/225000 [81:14:43<321:40:21,  6.17s/it]
 17%|█▋        | 37201/225000 [81:14:48<311:27:26,  5.97s/it]
 17%|█▋        | 37202/225000 [81:14:53<300:34:21,  5.76s/it]
 17%|█▋        | 37203/225000 [81:15:00<319:37:51,  6.13s/it]
 17%|█▋        | 37204/225000 [81:15:13<415:23:57,  7.96s/it]
 17%|█▋        | 37205/225000 [81:15:18<370:03:01,  7.09s/it]
 17%|█▋        | 37206/225000 [81:15:24<355:36:56,  6.82s/it]
 17%|█▋        | 37207/225000 [81:15:31<369:40:25,  7.09s/it]
 17%|█▋        | 37208/225000 [81:15:36<329:32:16,  6.32s/it]
 17%|█▋        | 37209/225000 [81:15:41<304:27:12,  5.84s/it]
 17%|█▋        | 37210/225000 [81:15:46<291:22:46,  5.59s/it]
 17%|█▋        | 37211/225000 [81:15:58<392:25:25,  7.52s/it]
 17%|█▋        | 37212/225000 [81:16:03<358:32:14,  6.87s/it]
 17%|█▋        | 37213/225000 [81:16:11<380:50:25,  7.30s/it]
 17%|█▋        | 37214/225000 [81:16:15<323:36:31,  6.20s/it]
 17%|█▋        | 37215/225000 [81:16:24<368:55:59,  7.07s/it]
 17%|█▋        | 37216/225000 [81:16:32<379:13:33,  7.27s/it]
 17%|█▋        | 37217/225000 [81:16:37<349:10:29,  6.69s/it]
 17%|█▋        | 37218/225000 [81:16:43<330:09:46,  6.33s/it]
 17%|█▋        | 37219/225000 [81:16:48<310:00:46,  5.94s/it]
 17%|█▋        | 37220/225000 [81:16:53<291:50:24,  5.59s/it]
 17%|█▋        | 37221/225000 [81:16:58<287:57:03,  5.52s/it]
 17%|█▋        | 37222/225000 [81:17:03<281:53:53,  5.40s/it]
 17%|█▋        | 37223/225000 [81:17:09<289:43:25,  5.55s/it]
 17%|█▋        | 37224/225000 [81:17:22<409:42:02,  7.85s/it]
 17%|█▋        | 37225/225000 [81:17:27<358:19:02,  6.87s/it]
 17%|█▋        | 37226/225000 [81:17:34<365:35:14,  7.01s/it]
 17%|█▋        | 37227/225000 [81:17:39<332:11:22,  6.37s/it]
 17%|█▋        | 37228/225000 [81:17:44<313:20:28,  6.01s/it]
 17%|█▋        | 37229/225000 [81:17:49<300:11:30,  5.76s/it]
 17%|█▋        | 37230/225000 [81:17:54<279:43:52,  5.36s/it]
 17%|█▋        | 37231/225000 [81:17:58<268:18:44,  5.14s/it]
 17%|█▋        | 37232/225000 [81:18:12<406:48:57,  7.80s/it]
 17%|█▋        | 37233/225000 [81:18:18<370:10:00,  7.10s/it]
 17%|█▋        | 37234/225000 [81:18:25<370:13:57,  7.10s/it]
 17%|█▋        | 37235/225000 [81:18:29<327:38:00,  6.28s/it]
 17%|█▋        | 37236/225000 [81:18:35<311:49:42,  5.98s/it]
 17%|█▋        | 37237/225000 [81:18:40<298:55:51,  5.73s/it]
 17%|█▋        | 37238/225000 [81:18:46<300:09:30,  5.76s/it]
 17%|█▋        | 37239/225000 [81:18:58<407:27:23,  7.81s/it]
 17%|█▋        | 37240/225000 [81:19:04<370:17:13,  7.10s/it]
 17%|█▋        | 37241/225000 [81:19:09<342:35:43,  6.57s/it]
 17%|█▋        | 37242/225000 [81:19:15<327:41:10,  6.28s/it]
 17%|█▋        | 37243/225000 [81:19:20<311:11:12,  5.97s/it]
 17%|█▋        | 37244/225000 [81:19:25<295:11:13,  5.66s/it]
 17%|█▋        | 37245/225000 [81:19:30<290:57:43,  5.58s/it]
 17%|█▋        | 37246/225000 [81:19:36<293:50:15,  5.63s/it]
 17%|█▋        | 37247/225000 [81:19:41<293:11:59,  5.62s/it]
 17%|█▋        | 37248/225000 [81:19:53<379:14:20,  7.27s/it]
 17%|█▋        | 37249/225000 [81:19:59<368:40:54,  7.07s/it]
 17%|█▋        | 37250/225000 [81:20:04<328:50:44,  6.31s/it]
                                                             
{'loss': '0.701', 'grad_norm': '0.857', 'learning_rate': '4.805e-05', 'epoch': '0.1656', 'num_input_tokens_seen': 137986288, 'train_runtime': '2.928e+05', 'train_tokens_per_second': '471.3'}

 17%|█▋        | 37250/225000 [81:20:04<328:50:44,  6.31s/it]
 17%|█▋        | 37251/225000 [81:20:08<300:17:19,  5.76s/it]
 17%|█▋        | 37252/225000 [81:20:20<397:52:00,  7.63s/it]
 17%|█▋        | 37253/225000 [81:20:24<338:23:30,  6.49s/it]
 17%|█▋        | 37254/225000 [81:20:32<366:54:17,  7.04s/it]
 17%|█▋        | 37255/225000 [81:20:37<331:36:35,  6.36s/it]
 17%|█▋        | 37256/225000 [81:20:42<310:28:15,  5.95s/it]
 17%|█▋        | 37257/225000 [81:20:47<295:39:59,  5.67s/it]
 17%|█▋        | 37258/225000 [81:20:52<283:12:15,  5.43s/it]
 17%|█▋        | 37259/225000 [81:21:05<396:19:16,  7.60s/it]
 17%|█▋        | 37260/225000 [81:21:10<368:55:24,  7.07s/it]
 17%|█▋        | 37261/225000 [81:21:17<355:22:19,  6.81s/it]
 17%|█▋        | 37262/225000 [81:21:22<335:22:48,  6.43s/it]
 17%|█▋        | 37263/225000 [81:21:28<323:41:30,  6.21s/it]
 17%|█▋        | 37264/225000 [81:21:34<322:31:40,  6.18s/it]
 17%|█▋        | 37265/225000 [81:21:39<308:06:39,  5.91s/it]
 17%|█▋        | 37266/225000 [81:21:45<302:38:43,  5.80s/it]
 17%|█▋        | 37267/225000 [81:21:52<329:43:17,  6.32s/it]
 17%|█▋        | 37268/225000 [81:22:06<443:47:20,  8.51s/it]
 17%|█▋        | 37269/225000 [81:22:10<376:10:57,  7.21s/it]
 17%|█▋        | 37270/225000 [81:22:16<359:36:08,  6.90s/it]
 17%|█▋        | 37271/225000 [81:22:20<303:56:31,  5.83s/it]
 17%|█▋        | 37272/225000 [81:22:31<391:01:09,  7.50s/it]
 17%|█▋        | 37273/225000 [81:22:35<329:49:49,  6.33s/it]
 17%|█▋        | 37274/225000 [81:22:44<373:07:18,  7.16s/it]
 17%|█▋        | 37275/225000 [81:22:49<347:55:32,  6.67s/it]
 17%|█▋        | 37276/225000 [81:22:57<359:10:03,  6.89s/it]
 17%|█▋        | 37277/225000 [81:23:00<305:09:07,  5.85s/it]
 17%|█▋        | 37278/225000 [81:23:10<373:09:10,  7.16s/it]
 17%|█▋        | 37279/225000 [81:23:14<317:44:34,  6.09s/it]
 17%|█▋        | 37280/225000 [81:23:23<362:12:39,  6.95s/it]
 17%|█▋        | 37281/225000 [81:23:26<309:52:34,  5.94s/it]
 17%|█▋        | 37282/225000 [81:23:32<295:36:10,  5.67s/it]
 17%|█▋        | 37283/225000 [81:23:36<284:39:49,  5.46s/it]
 17%|█▋        | 37284/225000 [81:23:49<399:57:11,  7.67s/it]
 17%|█▋        | 37285/225000 [81:23:54<350:41:40,  6.73s/it]
 17%|█▋        | 37286/225000 [81:23:58<315:35:06,  6.05s/it]
 17%|█▋        | 37287/225000 [81:24:03<295:02:01,  5.66s/it]
 17%|█▋        | 37288/225000 [81:24:08<286:04:05,  5.49s/it]
 17%|█▋        | 37289/225000 [81:24:20<389:05:29,  7.46s/it]
 17%|█▋        | 37290/225000 [81:24:24<335:43:21,  6.44s/it]
 17%|█▋        | 37291/225000 [81:24:29<309:32:10,  5.94s/it]
 17%|█▋        | 37292/225000 [81:24:35<309:03:44,  5.93s/it]
 17%|█▋        | 37293/225000 [81:24:48<415:45:24,  7.97s/it]
 17%|█▋        | 37294/225000 [81:24:52<358:04:36,  6.87s/it]
 17%|█▋        | 37295/225000 [81:24:56<310:53:46,  5.96s/it]
 17%|█▋        | 37296/225000 [81:25:05<363:48:19,  6.98s/it]
 17%|█▋        | 37297/225000 [81:25:11<339:14:42,  6.51s/it]
 17%|█▋        | 37298/225000 [81:25:17<330:21:25,  6.34s/it]
 17%|█▋        | 37299/225000 [81:25:22<314:45:18,  6.04s/it]
 17%|█▋        | 37300/225000 [81:25:27<306:39:14,  5.88s/it]
                                                             
{'loss': '0.6852', 'grad_norm': '0.6799', 'learning_rate': '4.804e-05', 'epoch': '0.1658', 'num_input_tokens_seen': 138131816, 'train_runtime': '2.931e+05', 'train_tokens_per_second': '471.2'}

 17%|█▋        | 37300/225000 [81:25:27<306:39:14,  5.88s/it]
 17%|█▋        | 37301/225000 [81:25:37<358:39:54,  6.88s/it]
 17%|█▋        | 37302/225000 [81:25:48<429:04:13,  8.23s/it]
 17%|█▋        | 37303/225000 [81:25:52<360:40:26,  6.92s/it]
 17%|█▋        | 37304/225000 [81:25:56<315:19:07,  6.05s/it]
 17%|█▋        | 37305/225000 [81:26:05<364:18:46,  6.99s/it]
 17%|█▋        | 37306/225000 [81:26:13<380:06:50,  7.29s/it]
 17%|█▋        | 37307/225000 [81:26:18<340:55:18,  6.54s/it]
 17%|█▋        | 37308/225000 [81:26:23<325:01:51,  6.23s/it]
 17%|█▋        | 37309/225000 [81:26:29<311:03:22,  5.97s/it]
 17%|█▋        | 37310/225000 [81:26:34<300:05:27,  5.76s/it]
 17%|█▋        | 37311/225000 [81:26:39<292:41:11,  5.61s/it]
 17%|█▋        | 37312/225000 [81:26:45<292:22:43,  5.61s/it]
 17%|█▋        | 37313/225000 [81:26:51<298:32:45,  5.73s/it]
 17%|█▋        | 37314/225000 [81:26:57<301:27:58,  5.78s/it]
 17%|█▋        | 37315/225000 [81:27:04<324:40:57,  6.23s/it]
 17%|█▋        | 37316/225000 [81:27:17<432:41:27,  8.30s/it]
 17%|█▋        | 37317/225000 [81:27:23<388:10:36,  7.45s/it]
 17%|█▋        | 37318/225000 [81:27:26<333:05:35,  6.39s/it]
 17%|█▋        | 37319/225000 [81:27:31<308:57:22,  5.93s/it]
 17%|█▋        | 37320/225000 [81:27:36<294:54:43,  5.66s/it]
 17%|█▋        | 37321/225000 [81:27:46<358:43:15,  6.88s/it]
 17%|█▋        | 37322/225000 [81:27:56<412:47:02,  7.92s/it]
 17%|█▋        | 37323/225000 [81:28:03<390:27:21,  7.49s/it]
 17%|█▋        | 37324/225000 [81:28:08<345:44:42,  6.63s/it]
 17%|█▋        | 37325/225000 [81:28:17<396:12:52,  7.60s/it]
 17%|█▋        | 37326/225000 [81:28:22<353:46:39,  6.79s/it]
 17%|█▋        | 37327/225000 [81:28:31<389:57:17,  7.48s/it]
 17%|█▋        | 37328/225000 [81:28:36<337:30:33,  6.47s/it]
 17%|█▋        | 37329/225000 [81:28:45<389:58:39,  7.48s/it]
 17%|█▋        | 37330/225000 [81:28:49<333:57:38,  6.41s/it]
 17%|█▋        | 37331/225000 [81:28:54<308:01:02,  5.91s/it]
 17%|█▋        | 37332/225000 [81:28:59<290:39:36,  5.58s/it]
 17%|█▋        | 37333/225000 [81:29:10<383:14:24,  7.35s/it]
 17%|█▋        | 37334/225000 [81:29:15<347:05:04,  6.66s/it]
 17%|█▋        | 37335/225000 [81:29:23<370:20:37,  7.10s/it]
 17%|█▋        | 37336/225000 [81:29:30<361:18:45,  6.93s/it]
 17%|█▋        | 37337/225000 [81:29:35<334:08:27,  6.41s/it]
 17%|█▋        | 37338/225000 [81:29:41<317:58:58,  6.10s/it]
 17%|█▋        | 37339/225000 [81:29:46<301:03:01,  5.78s/it]
 17%|█▋        | 37340/225000 [81:29:51<294:09:22,  5.64s/it]
 17%|█▋        | 37341/225000 [81:30:01<363:09:15,  6.97s/it]
 17%|█▋        | 37342/225000 [81:30:10<399:14:49,  7.66s/it]
 17%|█▋        | 37343/225000 [81:30:14<333:49:36,  6.40s/it]
 17%|█▋        | 37344/225000 [81:30:23<377:29:59,  7.24s/it]
 17%|█▋        | 37345/225000 [81:30:28<345:28:35,  6.63s/it]
 17%|█▋        | 37346/225000 [81:30:34<329:03:49,  6.31s/it]
 17%|█▋        | 37347/225000 [81:30:39<314:11:40,  6.03s/it]
 17%|█▋        | 37348/225000 [81:30:45<305:10:32,  5.85s/it]
 17%|█▋        | 37349/225000 [81:30:50<293:43:25,  5.63s/it]
 17%|█▋        | 37350/225000 [81:30:55<287:27:02,  5.51s/it]
                                                             
{'loss': '0.691', 'grad_norm': '0.6927', 'learning_rate': '4.803e-05', 'epoch': '0.166', 'num_input_tokens_seen': 138277688, 'train_runtime': '2.935e+05', 'train_tokens_per_second': '471.2'}

 17%|█▋        | 37350/225000 [81:30:55<287:27:02,  5.51s/it]
 17%|█▋        | 37351/225000 [81:31:01<293:07:45,  5.62s/it]
 17%|█▋        | 37352/225000 [81:31:11<359:42:17,  6.90s/it]
 17%|█▋        | 37353/225000 [81:31:23<439:43:42,  8.44s/it]
 17%|█▋        | 37354/225000 [81:31:26<367:23:34,  7.05s/it]
 17%|█▋        | 37355/225000 [81:31:33<361:31:27,  6.94s/it]
 17%|█▋        | 37356/225000 [81:31:41<377:40:59,  7.25s/it]
 17%|█▋        | 37357/225000 [81:31:46<341:25:09,  6.55s/it]
 17%|█▋        | 37358/225000 [81:31:51<322:53:10,  6.19s/it]
 17%|█▋        | 37359/225000 [81:31:59<344:52:30,  6.62s/it]
 17%|█▋        | 37360/225000 [81:32:08<384:16:01,  7.37s/it]
 17%|█▋        | 37361/225000 [81:32:12<334:32:23,  6.42s/it]
 17%|█▋        | 37362/225000 [81:32:16<296:24:16,  5.69s/it]
 17%|█▋        | 37363/225000 [81:32:27<379:30:43,  7.28s/it]
 17%|█▋        | 37364/225000 [81:32:34<372:22:29,  7.14s/it]
 17%|█▋        | 37365/225000 [81:32:39<330:44:46,  6.35s/it]
 17%|█▋        | 37366/225000 [81:32:43<300:52:20,  5.77s/it]
 17%|█▋        | 37367/225000 [81:32:48<280:23:55,  5.38s/it]
 17%|█▋        | 37368/225000 [81:32:55<308:14:41,  5.91s/it]
 17%|█▋        | 37369/225000 [81:33:08<423:34:28,  8.13s/it]
 17%|█▋        | 37370/225000 [81:33:13<380:51:54,  7.31s/it]
 17%|█▋        | 37371/225000 [81:33:19<359:56:14,  6.91s/it]
 17%|█▋        | 37372/225000 [81:33:25<334:35:44,  6.42s/it]
 17%|█▋        | 37373/225000 [81:33:34<387:45:48,  7.44s/it]
 17%|█▋        | 37374/225000 [81:33:39<339:07:03,  6.51s/it]
 17%|█▋        | 37375/225000 [81:33:45<328:55:44,  6.31s/it]
 17%|█▋        | 37376/225000 [81:33:50<320:17:33,  6.15s/it]
 17%|█▋        | 37377/225000 [81:33:56<306:33:12,  5.88s/it]
 17%|█▋        | 37378/225000 [81:34:04<345:12:25,  6.62s/it]
 17%|█▋        | 37379/225000 [81:34:10<333:35:27,  6.40s/it]
 17%|█▋        | 37380/225000 [81:34:17<339:25:44,  6.51s/it]
 17%|█▋        | 37381/225000 [81:34:23<341:07:22,  6.55s/it]
 17%|█▋        | 37382/225000 [81:34:29<322:21:38,  6.19s/it]
 17%|█▋        | 37383/225000 [81:34:40<396:30:21,  7.61s/it]
 17%|█▋        | 37384/225000 [81:34:46<371:41:07,  7.13s/it]
 17%|█▋        | 37385/225000 [81:34:52<364:32:28,  6.99s/it]
 17%|█▋        | 37386/225000 [81:34:59<363:20:16,  6.97s/it]
 17%|█▋        | 37387/225000 [81:35:06<366:22:41,  7.03s/it]
 17%|█▋        | 37388/225000 [81:35:13<366:49:52,  7.04s/it]
 17%|█▋        | 37389/225000 [81:35:18<334:44:26,  6.42s/it]
 17%|█▋        | 37390/225000 [81:35:23<302:41:16,  5.81s/it]
 17%|█▋        | 37391/225000 [81:35:28<286:33:47,  5.50s/it]
 17%|█▋        | 37392/225000 [81:35:38<366:50:43,  7.04s/it]
 17%|█▋        | 37393/225000 [81:35:46<374:39:39,  7.19s/it]
 17%|█▋        | 37394/225000 [81:35:54<385:10:52,  7.39s/it]
 17%|█▋        | 37395/225000 [81:35:58<338:59:49,  6.51s/it]
 17%|█▋        | 37396/225000 [81:36:07<377:12:16,  7.24s/it]
 17%|█▋        | 37397/225000 [81:36:11<329:06:55,  6.32s/it]
 17%|█▋        | 37398/225000 [81:36:15<293:00:24,  5.62s/it]
 17%|█▋        | 37399/225000 [81:36:26<380:55:06,  7.31s/it]
 17%|█▋        | 37400/225000 [81:36:30<323:32:48,  6.21s/it]
                                                             
{'loss': '0.6897', 'grad_norm': '0.7639', 'learning_rate': '4.803e-05', 'epoch': '0.1662', 'num_input_tokens_seen': 138424112, 'train_runtime': '2.938e+05', 'train_tokens_per_second': '471.2'}

 17%|█▋        | 37400/225000 [81:36:30<323:32:48,  6.21s/it]
 17%|█▋        | 37401/225000 [81:36:38<349:33:45,  6.71s/it]
 17%|█▋        | 37402/225000 [81:36:42<310:30:30,  5.96s/it]
 17%|█▋        | 37403/225000 [81:36:52<376:35:10,  7.23s/it]
 17%|█▋        | 37404/225000 [81:36:56<324:12:51,  6.22s/it]
 17%|█▋        | 37405/225000 [81:37:05<358:42:47,  6.88s/it]
 17%|█▋        | 37406/225000 [81:37:11<354:14:11,  6.80s/it]
 17%|█▋        | 37407/225000 [81:37:14<299:20:51,  5.74s/it]
 17%|█▋        | 37408/225000 [81:37:21<308:06:41,  5.91s/it]
 17%|█▋        | 37409/225000 [81:37:28<334:02:09,  6.41s/it]
 17%|█▋        | 37410/225000 [81:37:32<297:32:21,  5.71s/it]
 17%|█▋        | 37411/225000 [81:37:42<354:15:41,  6.80s/it]
 17%|█▋        | 37412/225000 [81:37:46<319:43:14,  6.14s/it]
 17%|█▋        | 37413/225000 [81:37:54<338:04:27,  6.49s/it]
 17%|█▋        | 37414/225000 [81:37:59<320:03:05,  6.14s/it]
 17%|█▋        | 37415/225000 [81:38:04<307:58:42,  5.91s/it]
 17%|█▋        | 37416/225000 [81:38:10<301:39:10,  5.79s/it]
 17%|█▋        | 37417/225000 [81:38:15<294:58:16,  5.66s/it]
 17%|█▋        | 37418/225000 [81:38:21<292:21:57,  5.61s/it]
 17%|█▋        | 37419/225000 [81:38:31<367:40:09,  7.06s/it]
 17%|█▋        | 37420/225000 [81:38:40<396:31:04,  7.61s/it]
 17%|█▋        | 37421/225000 [81:38:49<414:12:14,  7.95s/it]
 17%|█▋        | 37422/225000 [81:38:52<345:30:00,  6.63s/it]
 17%|█▋        | 37423/225000 [81:38:59<350:34:27,  6.73s/it]
 17%|█▋        | 37424/225000 [81:39:08<374:15:11,  7.18s/it]
 17%|█▋        | 37425/225000 [81:39:12<327:55:35,  6.29s/it]
 17%|█▋        | 37426/225000 [81:39:16<301:33:16,  5.79s/it]
 17%|█▋        | 37427/225000 [81:39:26<356:31:08,  6.84s/it]
 17%|█▋        | 37428/225000 [81:39:34<387:04:33,  7.43s/it]
 17%|█▋        | 37429/225000 [81:39:39<342:07:31,  6.57s/it]
 17%|█▋        | 37430/225000 [81:39:47<365:34:28,  7.02s/it]
 17%|█▋        | 37431/225000 [81:39:55<377:18:02,  7.24s/it]
 17%|█▋        | 37432/225000 [81:39:59<334:05:28,  6.41s/it]
 17%|█▋        | 37433/225000 [81:40:09<382:20:19,  7.34s/it]
 17%|█▋        | 37434/225000 [81:40:13<334:17:16,  6.42s/it]
 17%|█▋        | 37435/225000 [81:40:22<367:09:57,  7.05s/it]
 17%|█▋        | 37436/225000 [81:40:28<358:36:29,  6.88s/it]
 17%|█▋        | 37437/225000 [81:40:32<303:40:24,  5.83s/it]
 17%|█▋        | 37438/225000 [81:40:40<345:55:10,  6.64s/it]
 17%|█▋        | 37439/225000 [81:40:45<321:50:09,  6.18s/it]
 17%|█▋        | 37440/225000 [81:40:51<313:36:50,  6.02s/it]
 17%|█▋        | 37441/225000 [81:40:57<310:43:39,  5.96s/it]
 17%|█▋        | 37442/225000 [81:41:02<295:59:22,  5.68s/it]
 17%|█▋        | 37443/225000 [81:41:07<285:51:41,  5.49s/it]
 17%|█▋        | 37444/225000 [81:41:15<335:17:21,  6.44s/it]
 17%|█▋        | 37445/225000 [81:41:27<412:12:18,  7.91s/it]
 17%|█▋        | 37446/225000 [81:41:32<364:21:00,  6.99s/it]
 17%|█▋        | 37447/225000 [81:41:37<341:03:38,  6.55s/it]
 17%|█▋        | 37448/225000 [81:41:43<327:14:39,  6.28s/it]
 17%|█▋        | 37449/225000 [81:41:52<369:14:06,  7.09s/it]
 17%|█▋        | 37450/225000 [81:41:57<347:31:46,  6.67s/it]
                                                             
{'loss': '0.7014', 'grad_norm': '0.6642', 'learning_rate': '4.802e-05', 'epoch': '0.1664', 'num_input_tokens_seen': 138571728, 'train_runtime': '2.941e+05', 'train_tokens_per_second': '471.1'}

 17%|█▋        | 37450/225000 [81:41:57<347:31:46,  6.67s/it]
 17%|█▋        | 37451/225000 [81:42:02<316:48:26,  6.08s/it]
 17%|█▋        | 37452/225000 [81:42:07<293:00:51,  5.62s/it]
 17%|█▋        | 37453/225000 [81:42:19<401:02:56,  7.70s/it]
 17%|█▋        | 37454/225000 [81:42:24<349:32:37,  6.71s/it]
 17%|█▋        | 37455/225000 [81:42:32<372:20:35,  7.15s/it]
 17%|█▋        | 37456/225000 [81:42:37<343:07:53,  6.59s/it]
 17%|█▋        | 37457/225000 [81:42:42<315:31:15,  6.06s/it]
 17%|█▋        | 37458/225000 [81:42:47<297:34:24,  5.71s/it]
 17%|█▋        | 37459/225000 [81:42:52<292:39:18,  5.62s/it]
 17%|█▋        | 37460/225000 [81:43:00<326:53:34,  6.28s/it]
 17%|█▋        | 37461/225000 [81:43:11<406:19:38,  7.80s/it]
 17%|█▋        | 37462/225000 [81:43:15<349:26:28,  6.71s/it]
 17%|█▋        | 37463/225000 [81:43:22<352:03:59,  6.76s/it]
 17%|█▋        | 37464/225000 [81:43:27<326:09:47,  6.26s/it]
 17%|█▋        | 37465/225000 [81:43:33<309:33:56,  5.94s/it]
 17%|█▋        | 37466/225000 [81:43:38<299:35:35,  5.75s/it]
 17%|█▋        | 37467/225000 [81:43:48<373:42:05,  7.17s/it]
 17%|█▋        | 37468/225000 [81:43:56<376:46:19,  7.23s/it]
 17%|█▋        | 37469/225000 [81:44:02<365:56:42,  7.02s/it]
 17%|█▋        | 37470/225000 [81:44:07<321:48:35,  6.18s/it]
 17%|█▋        | 37471/225000 [81:44:12<305:51:56,  5.87s/it]
 17%|█▋        | 37472/225000 [81:44:23<391:38:53,  7.52s/it]
 17%|█▋        | 37473/225000 [81:44:27<339:45:33,  6.52s/it]
 17%|█▋        | 37474/225000 [81:44:35<351:43:26,  6.75s/it]
 17%|█▋        | 37475/225000 [81:44:41<353:20:16,  6.78s/it]
 17%|█▋        | 37476/225000 [81:44:50<388:25:55,  7.46s/it]
 17%|█▋        | 37477/225000 [81:44:54<334:00:31,  6.41s/it]
 17%|█▋        | 37478/225000 [81:44:58<292:52:44,  5.62s/it]
 17%|█▋        | 37479/225000 [81:45:04<290:50:42,  5.58s/it]
 17%|█▋        | 37480/225000 [81:45:09<282:02:47,  5.41s/it]
 17%|█▋        | 37481/225000 [81:45:14<278:07:43,  5.34s/it]
 17%|█▋        | 37482/225000 [81:45:26<389:35:17,  7.48s/it]
 17%|█▋        | 37483/225000 [81:45:32<361:33:13,  6.94s/it]
 17%|█▋        | 37484/225000 [81:45:37<326:57:09,  6.28s/it]
 17%|█▋        | 37485/225000 [81:45:45<354:29:52,  6.81s/it]
 17%|█▋        | 37486/225000 [81:45:53<371:07:15,  7.12s/it]
 17%|█▋        | 37487/225000 [81:45:57<332:54:34,  6.39s/it]
 17%|█▋        | 37488/225000 [81:46:02<311:40:05,  5.98s/it]
 17%|█▋        | 37489/225000 [81:46:13<376:51:42,  7.24s/it]
 17%|█▋        | 37490/225000 [81:46:18<353:46:41,  6.79s/it]
 17%|█▋        | 37491/225000 [81:46:27<380:45:17,  7.31s/it]
 17%|█▋        | 37492/225000 [81:46:31<332:39:49,  6.39s/it]
 17%|█▋        | 37493/225000 [81:46:37<323:07:31,  6.20s/it]
 17%|█▋        | 37494/225000 [81:46:46<369:04:28,  7.09s/it]
 17%|█▋        | 37495/225000 [81:46:50<327:47:49,  6.29s/it]
 17%|█▋        | 37496/225000 [81:46:55<307:33:30,  5.90s/it]
 17%|█▋        | 37497/225000 [81:47:00<290:14:16,  5.57s/it]
 17%|█▋        | 37498/225000 [81:47:05<280:45:46,  5.39s/it]
 17%|█▋        | 37499/225000 [81:47:14<331:12:20,  6.36s/it]
 17%|█▋        | 37500/225000 [81:47:24<387:33:11,  7.44s/it]
                                                             
{'loss': '0.6904', 'grad_norm': '0.7815', 'learning_rate': '4.801e-05', 'epoch': '0.1667', 'num_input_tokens_seen': 138718352, 'train_runtime': '2.944e+05', 'train_tokens_per_second': '471.1'}

 17%|█▋        | 37500/225000 [81:47:24<387:33:11,  7.44s/it]
 17%|█▋        | 37501/225000 [81:47:28<339:46:01,  6.52s/it]
 17%|█▋        | 37502/225000 [81:47:37<375:18:07,  7.21s/it]
 17%|█▋        | 37503/225000 [81:47:43<364:54:48,  7.01s/it]
 17%|█▋        | 37504/225000 [81:47:50<363:11:23,  6.97s/it]
 17%|█▋        | 37505/225000 [81:47:58<369:57:08,  7.10s/it]
 17%|█▋        | 37506/225000 [81:48:06<380:21:43,  7.30s/it]
 17%|█▋        | 37507/225000 [81:48:10<340:31:05,  6.54s/it]
 17%|█▋        | 37508/225000 [81:48:17<342:37:54,  6.58s/it]
 17%|█▋        | 37509/225000 [81:48:27<398:35:55,  7.65s/it]
 17%|█▋        | 37510/225000 [81:48:33<377:32:04,  7.25s/it]
 17%|█▋        | 37511/225000 [81:48:40<362:14:13,  6.96s/it]
 17%|█▋        | 37512/225000 [81:48:47<365:45:04,  7.02s/it]
 17%|█▋        | 37513/225000 [81:48:54<367:54:39,  7.06s/it]
 17%|█▋        | 37514/225000 [81:49:02<380:03:57,  7.30s/it]
 17%|█▋        | 37515/225000 [81:49:06<330:26:24,  6.34s/it]
 17%|█▋        | 37516/225000 [81:49:11<305:09:54,  5.86s/it]
 17%|█▋        | 37517/225000 [81:49:19<345:36:57,  6.64s/it]
 17%|█▋        | 37518/225000 [81:49:25<330:10:14,  6.34s/it]
 17%|█▋        | 37519/225000 [81:49:30<313:07:27,  6.01s/it]
 17%|█▋        | 37520/225000 [81:49:35<295:54:41,  5.68s/it]
 17%|█▋        | 37521/225000 [81:49:41<294:33:26,  5.66s/it]
 17%|█▋        | 37522/225000 [81:49:46<291:24:02,  5.60s/it]
 17%|█▋        | 37523/225000 [81:49:54<329:14:35,  6.32s/it]
 17%|█▋        | 37524/225000 [81:50:05<396:53:28,  7.62s/it]
 17%|█▋        | 37525/225000 [81:50:09<342:48:32,  6.58s/it]
 17%|█▋        | 37526/225000 [81:50:21<428:01:40,  8.22s/it]
 17%|█▋        | 37527/225000 [81:50:25<367:40:08,  7.06s/it]
 17%|█▋        | 37528/225000 [81:50:30<328:51:49,  6.32s/it]
 17%|█▋        | 37529/225000 [81:50:35<303:10:06,  5.82s/it]
 17%|█▋        | 37530/225000 [81:50:42<329:16:56,  6.32s/it]
 17%|█▋        | 37531/225000 [81:50:52<386:53:50,  7.43s/it]
 17%|█▋        | 37532/225000 [81:50:57<346:04:00,  6.65s/it]
 17%|█▋        | 37533/225000 [81:51:03<330:50:39,  6.35s/it]
 17%|█▋        | 37534/225000 [81:51:08<322:20:16,  6.19s/it]
 17%|█▋        | 37535/225000 [81:51:15<336:01:03,  6.45s/it]
 17%|█▋        | 37536/225000 [81:51:21<329:39:16,  6.33s/it]
 17%|█▋        | 37537/225000 [81:51:27<320:29:33,  6.15s/it]
 17%|█▋        | 37538/225000 [81:51:36<356:37:38,  6.85s/it]
 17%|█▋        | 37539/225000 [81:51:44<374:19:05,  7.19s/it]
 17%|█▋        | 37540/225000 [81:51:51<377:12:57,  7.24s/it]
 17%|█▋        | 37541/225000 [81:51:55<327:16:05,  6.28s/it]
 17%|█▋        | 37542/225000 [81:52:05<387:16:15,  7.44s/it]
 17%|█▋        | 37543/225000 [81:52:13<385:43:49,  7.41s/it]
 17%|█▋        | 37544/225000 [81:52:17<332:38:24,  6.39s/it]
 17%|█▋        | 37545/225000 [81:52:21<302:14:12,  5.80s/it]
 17%|█▋        | 37546/225000 [81:52:26<294:22:43,  5.65s/it]
 17%|█▋        | 37547/225000 [81:52:38<385:37:29,  7.41s/it]
 17%|█▋        | 37548/225000 [81:52:43<344:07:03,  6.61s/it]
 17%|█▋        | 37549/225000 [81:52:50<359:39:05,  6.91s/it]
 17%|█▋        | 37550/225000 [81:52:54<316:47:43,  6.08s/it]
                                                             
{'loss': '0.7035', 'grad_norm': '0.7139', 'learning_rate': '4.8e-05', 'epoch': '0.1669', 'num_input_tokens_seen': 138864912, 'train_runtime': '2.948e+05', 'train_tokens_per_second': '471.1'}

 17%|█▋        | 37550/225000 [81:52:54<316:47:43,  6.08s/it]
 17%|█▋        | 37551/225000 [81:52:59<296:29:01,  5.69s/it]
 17%|█▋        | 37552/225000 [81:53:05<302:02:47,  5.80s/it]
 17%|█▋        | 37553/225000 [81:53:16<379:02:48,  7.28s/it]
 17%|█▋        | 37554/225000 [81:53:21<339:48:48,  6.53s/it]
 17%|█▋        | 37555/225000 [81:53:25<310:05:00,  5.96s/it]
 17%|█▋        | 37556/225000 [81:53:35<370:25:40,  7.11s/it]
 17%|█▋        | 37557/225000 [81:53:42<369:28:30,  7.10s/it]
 17%|█▋        | 37558/225000 [81:53:50<388:24:00,  7.46s/it]
 17%|█▋        | 37559/225000 [81:53:56<352:46:29,  6.78s/it]
 17%|█▋        | 37560/225000 [81:54:05<390:23:27,  7.50s/it]
 17%|█▋        | 37561/225000 [81:54:09<337:10:34,  6.48s/it]
 17%|█▋        | 37562/225000 [81:54:13<304:58:08,  5.86s/it]
 17%|█▋        | 37563/225000 [81:54:23<370:07:59,  7.11s/it]
 17%|█▋        | 37564/225000 [81:54:28<334:40:17,  6.43s/it]
 17%|█▋        | 37565/225000 [81:54:33<303:32:44,  5.83s/it]
 17%|█▋        | 37566/225000 [81:54:37<281:09:49,  5.40s/it]
 17%|█▋        | 37567/225000 [81:54:49<390:25:16,  7.50s/it]
 17%|█▋        | 37568/225000 [81:54:54<347:21:05,  6.67s/it]
 17%|█▋        | 37569/225000 [81:54:58<306:48:15,  5.89s/it]
 17%|█▋        | 37570/225000 [81:55:10<401:59:28,  7.72s/it]
 17%|█▋        | 37571/225000 [81:55:14<345:23:20,  6.63s/it]
 17%|█▋        | 37572/225000 [81:55:23<383:43:35,  7.37s/it]
 17%|█▋        | 37573/225000 [81:55:28<336:32:16,  6.46s/it]
 17%|█▋        | 37574/225000 [81:55:34<327:52:07,  6.30s/it]
 17%|█▋        | 37575/225000 [81:55:44<384:50:49,  7.39s/it]
 17%|█▋        | 37576/225000 [81:55:47<328:36:31,  6.31s/it]
 17%|█▋        | 37577/225000 [81:55:52<296:57:22,  5.70s/it]
 17%|█▋        | 37578/225000 [81:55:56<282:29:37,  5.43s/it]
 17%|█▋        | 37579/225000 [81:56:05<327:39:35,  6.29s/it]
 17%|█▋        | 37580/225000 [81:56:13<363:35:52,  6.98s/it]
 17%|█▋        | 37581/225000 [81:56:18<333:45:33,  6.41s/it]
 17%|█▋        | 37582/225000 [81:56:28<388:46:28,  7.47s/it]
 17%|█▋        | 37583/225000 [81:56:32<332:17:17,  6.38s/it]
 17%|█▋        | 37584/225000 [81:56:37<302:19:47,  5.81s/it]
 17%|█▋        | 37585/225000 [81:56:42<290:03:02,  5.57s/it]
 17%|█▋        | 37586/225000 [81:56:53<377:25:27,  7.25s/it]
 17%|█▋        | 37587/225000 [81:57:02<411:31:17,  7.90s/it]
 17%|█▋        | 37588/225000 [81:57:10<411:20:49,  7.90s/it]
 17%|█▋        | 37589/225000 [81:57:14<348:08:31,  6.69s/it]
 17%|█▋        | 37590/225000 [81:57:18<301:03:09,  5.78s/it]
 17%|█▋        | 37591/225000 [81:57:29<379:54:44,  7.30s/it]
 17%|█▋        | 37592/225000 [81:57:32<323:25:38,  6.21s/it]
 17%|█▋        | 37593/225000 [81:57:40<354:56:18,  6.82s/it]
 17%|█▋        | 37594/225000 [81:57:45<312:50:53,  6.01s/it]
 17%|█▋        | 37595/225000 [81:57:50<302:48:14,  5.82s/it]
 17%|█▋        | 37596/225000 [81:57:55<294:34:57,  5.66s/it]
 17%|█▋        | 37597/225000 [81:58:01<300:36:56,  5.77s/it]
 17%|█▋        | 37598/225000 [81:58:15<432:08:27,  8.30s/it]
 17%|█▋        | 37599/225000 [81:58:20<380:10:38,  7.30s/it]
 17%|█▋        | 37600/225000 [81:58:27<369:12:28,  7.09s/it]
                                                             
{'loss': '0.7005', 'grad_norm': '0.683', 'learning_rate': '4.8e-05', 'epoch': '0.1671', 'num_input_tokens_seen': 139010280, 'train_runtime': '2.951e+05', 'train_tokens_per_second': '471.1'}

 17%|█▋        | 37600/225000 [81:58:27<369:12:28,  7.09s/it]
 17%|█▋        | 37601/225000 [81:58:35<382:40:03,  7.35s/it]
 17%|█▋        | 37602/225000 [81:58:39<329:07:08,  6.32s/it]
 17%|█▋        | 37603/225000 [81:58:47<353:24:00,  6.79s/it]
 17%|█▋        | 37604/225000 [81:58:54<353:27:45,  6.79s/it]
 17%|█▋        | 37605/225000 [81:59:02<371:13:41,  7.13s/it]
 17%|█▋        | 37606/225000 [81:59:07<338:39:23,  6.51s/it]
 17%|█▋        | 37607/225000 [81:59:13<334:02:30,  6.42s/it]
 17%|█▋        | 37608/225000 [81:59:22<373:39:59,  7.18s/it]
 17%|█▋        | 37609/225000 [81:59:26<321:45:43,  6.18s/it]
 17%|█▋        | 37610/225000 [81:59:31<311:01:20,  5.98s/it]
 17%|█▋        | 37611/225000 [81:59:37<307:24:00,  5.91s/it]
 17%|█▋        | 37612/225000 [81:59:42<302:23:33,  5.81s/it]
 17%|█▋        | 37613/225000 [81:59:48<302:27:49,  5.81s/it]
 17%|█▋        | 37614/225000 [81:59:57<355:52:31,  6.84s/it]
 17%|█▋        | 37615/225000 [82:00:03<342:12:33,  6.57s/it]
 17%|█▋        | 37616/225000 [82:00:11<363:14:51,  6.98s/it]
 17%|█▋        | 37617/225000 [82:00:18<354:08:21,  6.80s/it]
 17%|█▋        | 37618/225000 [82:00:26<379:44:28,  7.30s/it]
 17%|█▋        | 37619/225000 [82:00:31<336:31:58,  6.47s/it]
 17%|█▋        | 37620/225000 [82:00:35<310:19:40,  5.96s/it]
 17%|█▋        | 37621/225000 [82:00:40<291:53:03,  5.61s/it]
 17%|█▋        | 37622/225000 [82:00:49<344:17:47,  6.61s/it]
 17%|█▋        | 37623/225000 [82:00:59<389:45:35,  7.49s/it]
 17%|█▋        | 37624/225000 [82:01:07<398:08:31,  7.65s/it]
 17%|█▋        | 37625/225000 [82:01:11<346:24:28,  6.66s/it]
 17%|█▋        | 37626/225000 [82:01:20<374:34:16,  7.20s/it]
 17%|█▋        | 37627/225000 [82:01:26<359:19:00,  6.90s/it]
 17%|█▋        | 37628/225000 [82:01:33<361:09:41,  6.94s/it]
 17%|█▋        | 37629/225000 [82:01:37<317:22:24,  6.10s/it]
 17%|█▋        | 37630/225000 [82:01:42<297:05:01,  5.71s/it]
 17%|█▋        | 37631/225000 [82:01:46<276:44:40,  5.32s/it]
 17%|█▋        | 37632/225000 [82:01:59<398:58:46,  7.67s/it]
 17%|█▋        | 37633/225000 [82:02:05<364:04:52,  7.00s/it]
 17%|█▋        | 37634/225000 [82:02:13<380:20:09,  7.31s/it]
 17%|█▋        | 37635/225000 [82:02:17<328:28:41,  6.31s/it]
 17%|█▋        | 37636/225000 [82:02:21<298:15:42,  5.73s/it]
 17%|█▋        | 37637/225000 [82:02:33<388:45:36,  7.47s/it]
 17%|█▋        | 37638/225000 [82:02:37<334:36:46,  6.43s/it]
 17%|█▋        | 37639/225000 [82:02:44<348:15:38,  6.69s/it]
 17%|█▋        | 37640/225000 [82:02:52<362:46:25,  6.97s/it]
 17%|█▋        | 37641/225000 [82:02:56<315:36:58,  6.06s/it]
 17%|█▋        | 37642/225000 [82:02:59<281:56:35,  5.42s/it]
 17%|█▋        | 37643/225000 [82:03:11<379:51:19,  7.30s/it]
 17%|█▋        | 37644/225000 [82:03:18<366:08:54,  7.04s/it]
 17%|█▋        | 37645/225000 [82:03:24<361:46:44,  6.95s/it]
 17%|█▋        | 37646/225000 [82:03:28<316:36:00,  6.08s/it]
 17%|█▋        | 37647/225000 [82:03:40<410:45:16,  7.89s/it]
 17%|█▋        | 37648/225000 [82:03:46<366:04:38,  7.03s/it]
 17%|█▋        | 37649/225000 [82:03:51<343:24:17,  6.60s/it]
 17%|█▋        | 37650/225000 [82:03:56<310:57:54,  5.98s/it]
                                                             
{'loss': '0.7094', 'grad_norm': '0.8589', 'learning_rate': '4.799e-05', 'epoch': '0.1673', 'num_input_tokens_seen': 139156384, 'train_runtime': '2.954e+05', 'train_tokens_per_second': '471'}

 17%|█▋        | 37650/225000 [82:03:56<310:57:54,  5.98s/it]
 17%|█▋        | 37651/225000 [82:04:00<287:10:18,  5.52s/it]
 17%|█▋        | 37652/225000 [82:04:08<319:54:03,  6.15s/it]
 17%|█▋        | 37653/225000 [82:04:18<391:23:23,  7.52s/it]
 17%|█▋        | 37654/225000 [82:04:26<389:18:04,  7.48s/it]
 17%|█▋        | 37655/225000 [82:04:31<345:50:44,  6.65s/it]
 17%|█▋        | 37656/225000 [82:04:36<320:49:08,  6.16s/it]
 17%|█▋        | 37657/225000 [82:04:40<300:10:09,  5.77s/it]
 17%|█▋        | 37658/225000 [82:04:53<409:39:31,  7.87s/it]
 17%|█▋        | 37659/225000 [82:04:58<364:00:02,  6.99s/it]
 17%|█▋        | 37660/225000 [82:05:08<407:58:16,  7.84s/it]
 17%|█▋        | 37661/225000 [82:05:12<352:46:35,  6.78s/it]
 17%|█▋        | 37662/225000 [82:05:20<374:01:29,  7.19s/it]
 17%|█▋        | 37663/225000 [82:05:25<328:37:19,  6.32s/it]
 17%|█▋        | 37664/225000 [82:05:35<389:48:23,  7.49s/it]
 17%|█▋        | 37665/225000 [82:05:41<366:01:09,  7.03s/it]
 17%|█▋        | 37666/225000 [82:05:44<309:40:16,  5.95s/it]
 17%|█▋        | 37667/225000 [82:05:49<293:00:04,  5.63s/it]
 17%|█▋        | 37668/225000 [82:05:54<279:07:30,  5.36s/it]
 17%|█▋        | 37669/225000 [82:06:00<287:43:47,  5.53s/it]
 17%|█▋        | 37670/225000 [82:06:12<391:41:46,  7.53s/it]
 17%|█▋        | 37671/225000 [82:06:16<331:45:20,  6.38s/it]
 17%|█▋        | 37672/225000 [82:06:26<395:16:35,  7.60s/it]
 17%|█▋        | 37673/225000 [82:06:31<354:04:34,  6.80s/it]
 17%|█▋        | 37674/225000 [82:06:36<319:02:53,  6.13s/it]
 17%|█▋        | 37675/225000 [82:06:40<296:22:27,  5.70s/it]
 17%|█▋        | 37676/225000 [82:06:44<271:06:02,  5.21s/it]
 17%|█▋        | 37677/225000 [82:06:52<308:16:08,  5.92s/it]
 17%|█▋        | 37678/225000 [82:07:03<384:55:19,  7.40s/it]
 17%|█▋        | 37679/225000 [82:07:07<337:33:59,  6.49s/it]
 17%|█▋        | 37680/225000 [82:07:11<302:30:36,  5.81s/it]
 17%|█▋        | 37681/225000 [82:07:23<387:05:30,  7.44s/it]
 17%|█▋        | 37682/225000 [82:07:31<397:26:43,  7.64s/it]
 17%|█▋        | 37683/225000 [82:07:37<370:08:37,  7.11s/it]
 17%|█▋        | 37684/225000 [82:07:46<398:34:59,  7.66s/it]
 17%|█▋        | 37685/225000 [82:07:52<385:50:34,  7.42s/it]
 17%|█▋        | 37686/225000 [82:07:57<336:56:59,  6.48s/it]
 17%|█▋        | 37687/225000 [82:08:01<301:58:55,  5.80s/it]
 17%|█▋        | 37688/225000 [82:08:12<381:12:35,  7.33s/it]
 17%|█▋        | 37689/225000 [82:08:16<332:59:15,  6.40s/it]
 17%|█▋        | 37690/225000 [82:08:21<305:58:11,  5.88s/it]
 17%|█▋        | 37691/225000 [82:08:26<296:21:49,  5.70s/it]
 17%|█▋        | 37692/225000 [82:08:38<392:34:15,  7.55s/it]
 17%|█▋        | 37693/225000 [82:08:42<335:50:40,  6.45s/it]
 17%|█▋        | 37694/225000 [82:08:46<294:47:47,  5.67s/it]
 17%|█▋        | 37695/225000 [82:08:56<366:28:50,  7.04s/it]
 17%|█▋        | 37696/225000 [82:09:00<317:37:55,  6.10s/it]
 17%|█▋        | 37697/225000 [82:09:06<320:09:21,  6.15s/it]
 17%|█▋        | 37698/225000 [82:09:12<323:59:51,  6.23s/it]
 17%|█▋        | 37699/225000 [82:09:17<293:52:43,  5.65s/it]
 17%|█▋        | 37700/225000 [82:09:23<302:00:04,  5.80s/it]
                                                             
{'loss': '0.688', 'grad_norm': '0.6873', 'learning_rate': '4.798e-05', 'epoch': '0.1676', 'num_input_tokens_seen': 139303256, 'train_runtime': '2.958e+05', 'train_tokens_per_second': '471'}

 17%|█▋        | 37700/225000 [82:09:23<302:00:04,  5.80s/it]
 17%|█▋        | 37701/225000 [82:09:34<389:13:37,  7.48s/it]
 17%|█▋        | 37702/225000 [82:09:38<330:10:29,  6.35s/it]
 17%|█▋        | 37703/225000 [82:09:45<345:10:26,  6.63s/it]
 17%|█▋        | 37704/225000 [82:09:53<367:13:15,  7.06s/it]
 17%|█▋        | 37705/225000 [82:09:59<339:19:02,  6.52s/it]
 17%|█▋        | 37706/225000 [82:10:05<341:56:50,  6.57s/it]
 17%|█▋        | 37707/225000 [82:10:10<316:45:14,  6.09s/it]
 17%|█▋        | 37708/225000 [82:10:15<293:50:34,  5.65s/it]
 17%|█▋        | 37709/225000 [82:10:22<320:36:17,  6.16s/it]
 17%|█▋        | 37710/225000 [82:10:34<413:33:55,  7.95s/it]
 17%|█▋        | 37711/225000 [82:10:39<362:59:10,  6.98s/it]
 17%|█▋        | 37712/225000 [82:10:44<330:51:16,  6.36s/it]
 17%|█▋        | 37713/225000 [82:10:49<312:17:07,  6.00s/it]
 17%|█▋        | 37714/225000 [82:10:54<298:10:18,  5.73s/it]
 17%|█▋        | 37715/225000 [82:11:00<294:46:27,  5.67s/it]
 17%|█▋        | 37716/225000 [82:11:05<290:53:11,  5.59s/it]
 17%|█▋        | 37717/225000 [82:11:19<413:13:11,  7.94s/it]
 17%|█▋        | 37718/225000 [82:11:23<362:15:23,  6.96s/it]
 17%|█▋        | 37719/225000 [82:11:30<351:02:58,  6.75s/it]
 17%|█▋        | 37720/225000 [82:11:39<392:17:41,  7.54s/it]
 17%|█▋        | 37721/225000 [82:11:43<344:50:18,  6.63s/it]
 17%|█▋        | 37722/225000 [82:11:47<303:49:36,  5.84s/it]
 17%|█▋        | 37723/225000 [82:11:52<288:56:39,  5.55s/it]
 17%|█▋        | 37724/225000 [82:12:04<387:13:42,  7.44s/it]
 17%|█▋        | 37725/225000 [82:12:09<350:47:05,  6.74s/it]
 17%|█▋        | 37726/225000 [82:12:15<333:08:43,  6.40s/it]
 17%|█▋        | 37727/225000 [82:12:20<312:53:37,  6.01s/it]
 17%|█▋        | 37728/225000 [82:12:25<301:02:06,  5.79s/it]
 17%|█▋        | 37729/225000 [82:12:30<289:07:23,  5.56s/it]
 17%|█▋        | 37730/225000 [82:12:42<379:48:30,  7.30s/it]
 17%|█▋        | 37731/225000 [82:12:50<388:30:53,  7.47s/it]
 17%|█▋        | 37732/225000 [82:12:55<356:29:01,  6.85s/it]
 17%|█▋        | 37733/225000 [82:13:01<344:08:23,  6.62s/it]
 17%|█▋        | 37734/225000 [82:13:11<399:50:14,  7.69s/it]
 17%|█▋        | 37735/225000 [82:13:17<366:33:50,  7.05s/it]
 17%|█▋        | 37736/225000 [82:13:21<330:34:21,  6.35s/it]
 17%|█▋        | 37737/225000 [82:13:26<300:45:53,  5.78s/it]
 17%|█▋        | 37738/225000 [82:13:35<355:14:12,  6.83s/it]
 17%|█▋        | 37739/225000 [82:13:44<378:09:11,  7.27s/it]
 17%|█▋        | 37740/225000 [82:13:51<377:28:29,  7.26s/it]
 17%|█▋        | 37741/225000 [82:13:56<340:51:31,  6.55s/it]
 17%|█▋        | 37742/225000 [82:14:04<374:10:05,  7.19s/it]
 17%|█▋        | 37743/225000 [82:14:09<327:00:21,  6.29s/it]
 17%|█▋        | 37744/225000 [82:14:19<393:44:15,  7.57s/it]
 17%|█▋        | 37745/225000 [82:14:23<333:44:47,  6.42s/it]
 17%|█▋        | 37746/225000 [82:14:31<363:21:46,  6.99s/it]
 17%|█▋        | 37747/225000 [82:14:37<339:01:01,  6.52s/it]
 17%|█▋        | 37748/225000 [82:14:41<300:49:28,  5.78s/it]
 17%|█▋        | 37749/225000 [82:14:51<367:49:18,  7.07s/it]
 17%|█▋        | 37750/225000 [82:14:55<318:31:07,  6.12s/it]
                                                             
{'loss': '0.707', 'grad_norm': '0.7866', 'learning_rate': '4.798e-05', 'epoch': '0.1678', 'num_input_tokens_seen': 139448808, 'train_runtime': '2.961e+05', 'train_tokens_per_second': '471'}

 17%|█▋        | 37750/225000 [82:14:55<318:31:07,  6.12s/it]
 17%|█▋        | 37751/225000 [82:15:04<368:54:09,  7.09s/it]
 17%|█▋        | 37752/225000 [82:15:08<328:41:59,  6.32s/it]
 17%|█▋        | 37753/225000 [82:15:17<367:47:42,  7.07s/it]
 17%|█▋        | 37754/225000 [82:15:23<347:24:26,  6.68s/it]
 17%|█▋        | 37755/225000 [82:15:28<319:32:22,  6.14s/it]
 17%|█▋        | 37756/225000 [82:15:33<307:58:42,  5.92s/it]
 17%|█▋        | 37757/225000 [82:15:39<299:37:15,  5.76s/it]
 17%|█▋        | 37758/225000 [82:15:45<299:44:33,  5.76s/it]
 17%|█▋        | 37759/225000 [82:15:50<293:13:00,  5.64s/it]
 17%|█▋        | 37760/225000 [82:15:59<354:37:45,  6.82s/it]
 17%|█▋        | 37761/225000 [82:16:10<415:55:38,  8.00s/it]
 17%|█▋        | 37762/225000 [82:16:15<367:24:50,  7.06s/it]
 17%|█▋        | 37763/225000 [82:16:20<328:35:30,  6.32s/it]
 17%|█▋        | 37764/225000 [82:16:24<302:37:52,  5.82s/it]
 17%|█▋        | 37765/225000 [82:16:31<321:32:04,  6.18s/it]
 17%|█▋        | 37766/225000 [82:16:44<427:17:58,  8.22s/it]
 17%|█▋        | 37767/225000 [82:16:48<358:15:00,  6.89s/it]
 17%|█▋        | 37768/225000 [82:16:57<393:31:49,  7.57s/it]
 17%|█▋        | 37769/225000 [82:17:01<337:05:46,  6.48s/it]
 17%|█▋        | 37770/225000 [82:17:09<360:11:37,  6.93s/it]
 17%|█▋        | 37771/225000 [82:17:15<342:07:29,  6.58s/it]
 17%|█▋        | 37772/225000 [82:17:24<387:21:17,  7.45s/it]
 17%|█▋        | 37773/225000 [82:17:28<333:06:43,  6.41s/it]
 17%|█▋        | 37774/225000 [82:17:33<311:46:20,  5.99s/it]
 17%|█▋        | 37775/225000 [82:17:41<339:48:34,  6.53s/it]
 17%|█▋        | 37776/225000 [82:17:51<392:18:43,  7.54s/it]
 17%|█▋        | 37777/225000 [82:17:57<373:00:26,  7.17s/it]
 17%|█▋        | 37778/225000 [82:18:01<321:22:07,  6.18s/it]
 17%|█▋        | 37779/225000 [82:18:11<381:26:53,  7.33s/it]
 17%|█▋        | 37780/225000 [82:18:16<340:52:43,  6.55s/it]
 17%|█▋        | 37781/225000 [82:18:21<311:49:59,  6.00s/it]
 17%|█▋        | 37782/225000 [82:18:26<295:16:08,  5.68s/it]
 17%|█▋        | 37783/225000 [82:18:30<278:35:30,  5.36s/it]
 17%|█▋        | 37784/225000 [82:18:41<368:29:31,  7.09s/it]
 17%|█▋        | 37785/225000 [82:18:47<345:30:23,  6.64s/it]
 17%|█▋        | 37786/225000 [82:18:52<316:24:38,  6.08s/it]
 17%|█▋        | 37787/225000 [82:18:57<303:43:25,  5.84s/it]
 17%|█▋        | 37788/225000 [82:19:02<290:33:56,  5.59s/it]
 17%|█▋        | 37789/225000 [82:19:08<289:15:58,  5.56s/it]
 17%|█▋        | 37790/225000 [82:19:21<406:05:59,  7.81s/it]
 17%|█▋        | 37791/225000 [82:19:28<394:58:18,  7.60s/it]
 17%|█▋        | 37792/225000 [82:19:31<333:24:10,  6.41s/it]
 17%|█▋        | 37793/225000 [82:19:37<322:45:11,  6.21s/it]
 17%|█▋        | 37794/225000 [82:19:42<309:29:41,  5.95s/it]
 17%|█▋        | 37795/225000 [82:19:48<298:35:17,  5.74s/it]
 17%|█▋        | 37796/225000 [82:19:55<325:40:11,  6.26s/it]
 17%|█▋        | 37797/225000 [82:20:06<397:39:37,  7.65s/it]
 17%|█▋        | 37798/225000 [82:20:10<347:23:31,  6.68s/it]
 17%|█▋        | 37799/225000 [82:20:15<313:54:46,  6.04s/it]
 17%|█▋        | 37800/225000 [82:20:26<394:56:03,  7.59s/it]
                                                             
{'loss': '0.6979', 'grad_norm': '0.8366', 'learning_rate': '4.797e-05', 'epoch': '0.168', 'num_input_tokens_seen': 139594952, 'train_runtime': '2.964e+05', 'train_tokens_per_second': '470.9'}

 17%|█▋        | 37800/225000 [82:20:26<394:56:03,  7.59s/it]
 17%|█▋        | 37801/225000 [82:20:31<355:30:36,  6.84s/it]
 17%|█▋        | 37802/225000 [82:20:36<323:12:10,  6.22s/it]
 17%|█▋        | 37803/225000 [82:20:41<297:37:27,  5.72s/it]
 17%|█▋        | 37804/225000 [82:20:46<285:21:55,  5.49s/it]
 17%|█▋        | 37805/225000 [82:20:52<297:26:37,  5.72s/it]
 17%|█▋        | 37806/225000 [82:21:05<417:08:00,  8.02s/it]
 17%|█▋        | 37807/225000 [82:21:12<393:32:14,  7.57s/it]
 17%|█▋        | 37808/225000 [82:21:16<345:53:17,  6.65s/it]
 17%|█▋        | 37809/225000 [82:21:21<315:39:07,  6.07s/it]
 17%|█▋        | 37810/225000 [82:21:31<377:00:14,  7.25s/it]
 17%|█▋        | 37811/225000 [82:21:39<393:59:31,  7.58s/it]
 17%|█▋        | 37812/225000 [82:21:44<351:57:07,  6.77s/it]
 17%|█▋        | 37813/225000 [82:21:50<334:17:05,  6.43s/it]
 17%|█▋        | 37814/225000 [82:21:55<313:41:29,  6.03s/it]
 17%|█▋        | 37815/225000 [82:22:00<299:24:00,  5.76s/it]
 17%|█▋        | 37816/225000 [82:22:06<297:59:33,  5.73s/it]
 17%|█▋        | 37817/225000 [82:22:11<296:12:44,  5.70s/it]
 17%|█▋        | 37818/225000 [82:22:23<392:59:44,  7.56s/it]
 17%|█▋        | 37819/225000 [82:22:33<434:52:40,  8.36s/it]
 17%|█▋        | 37820/225000 [82:22:38<374:36:11,  7.20s/it]
 17%|█▋        | 37821/225000 [82:22:42<330:54:20,  6.36s/it]
 17%|█▋        | 37822/225000 [82:22:53<394:39:46,  7.59s/it]
 17%|█▋        | 37823/225000 [82:22:57<345:42:32,  6.65s/it]
 17%|█▋        | 37824/225000 [82:23:02<313:41:36,  6.03s/it]
 17%|█▋        | 37825/225000 [82:23:12<377:54:39,  7.27s/it]
 17%|█▋        | 37826/225000 [82:23:16<328:55:39,  6.33s/it]
 17%|█▋        | 37827/225000 [82:23:20<296:17:38,  5.70s/it]
 17%|█▋        | 37828/225000 [82:23:26<287:16:47,  5.53s/it]
 17%|█▋        | 37829/225000 [82:23:31<292:43:49,  5.63s/it]
 17%|█▋        | 37830/225000 [82:23:43<383:04:38,  7.37s/it]
 17%|█▋        | 37831/225000 [82:23:51<398:03:01,  7.66s/it]
 17%|█▋        | 37832/225000 [82:23:55<335:51:52,  6.46s/it]
 17%|█▋        | 37833/225000 [82:24:04<370:50:03,  7.13s/it]
 17%|█▋        | 37834/225000 [82:24:08<332:52:20,  6.40s/it]
 17%|█▋        | 37835/225000 [82:24:13<302:32:24,  5.82s/it]
 17%|█▋        | 37836/225000 [82:24:25<408:42:34,  7.86s/it]
 17%|█▋        | 37837/225000 [82:24:31<381:08:32,  7.33s/it]
 17%|█▋        | 37838/225000 [82:24:38<376:03:38,  7.23s/it]
 17%|█▋        | 37839/225000 [82:24:44<356:03:33,  6.85s/it]
 17%|█▋        | 37840/225000 [82:24:53<391:13:01,  7.53s/it]
 17%|█▋        | 37841/225000 [82:24:59<357:17:57,  6.87s/it]
 17%|█▋        | 37842/225000 [82:25:03<313:45:09,  6.04s/it]
 17%|█▋        | 37843/225000 [82:25:08<306:05:53,  5.89s/it]
 17%|█▋        | 37844/225000 [82:25:15<312:54:06,  6.02s/it]
 17%|█▋        | 37845/225000 [82:25:27<402:44:15,  7.75s/it]
 17%|█▋        | 37846/225000 [82:25:30<342:48:29,  6.59s/it]
 17%|█▋        | 37847/225000 [82:25:42<416:11:07,  8.01s/it]
 17%|█▋        | 37848/225000 [82:25:47<375:01:38,  7.21s/it]
 17%|█▋        | 37849/225000 [82:25:51<323:52:04,  6.23s/it]
 17%|█▋        | 37850/225000 [82:25:59<358:48:33,  6.90s/it]
                                                             
{'loss': '0.7022', 'grad_norm': '0.7153', 'learning_rate': '4.796e-05', 'epoch': '0.1682', 'num_input_tokens_seen': 139741296, 'train_runtime': '2.968e+05', 'train_tokens_per_second': '470.9'}

 17%|█▋        | 37850/225000 [82:26:00<358:48:33,  6.90s/it]
 17%|█▋        | 37851/225000 [82:26:07<361:47:46,  6.96s/it]
 17%|█▋        | 37852/225000 [82:26:10<313:36:21,  6.03s/it]
 17%|█▋        | 37853/225000 [82:26:15<284:08:02,  5.47s/it]
 17%|█▋        | 37854/225000 [82:26:27<391:59:41,  7.54s/it]
 17%|█▋        | 37855/225000 [82:26:32<348:34:43,  6.71s/it]
 17%|█▋        | 37856/225000 [82:26:37<320:40:32,  6.17s/it]
 17%|█▋        | 37857/225000 [82:26:41<297:38:02,  5.73s/it]
 17%|█▋        | 37858/225000 [82:26:47<295:36:10,  5.69s/it]
 17%|█▋        | 37859/225000 [82:26:53<293:43:50,  5.65s/it]
 17%|█▋        | 37860/225000 [82:27:04<384:37:06,  7.40s/it]
 17%|█▋        | 37861/225000 [82:27:12<394:24:04,  7.59s/it]
 17%|█▋        | 37862/225000 [82:27:18<374:42:01,  7.21s/it]
 17%|█▋        | 37863/225000 [82:27:22<323:17:04,  6.22s/it]
 17%|█▋        | 37864/225000 [82:27:29<325:19:47,  6.26s/it]
 17%|█▋        | 37865/225000 [82:27:39<385:16:03,  7.41s/it]
 17%|█▋        | 37866/225000 [82:27:43<333:00:08,  6.41s/it]
 17%|█▋        | 37867/225000 [82:27:50<341:33:01,  6.57s/it]
 17%|█▋        | 37868/225000 [82:27:58<372:33:36,  7.17s/it]
 17%|█▋        | 37869/225000 [82:28:03<342:13:32,  6.58s/it]
 17%|█▋        | 37870/225000 [82:28:09<325:30:18,  6.26s/it]
 17%|█▋        | 37871/225000 [82:28:15<315:18:05,  6.07s/it]
 17%|█▋        | 37872/225000 [82:28:20<302:34:47,  5.82s/it]
 17%|█▋        | 37873/225000 [82:28:26<304:03:54,  5.85s/it]
 17%|█▋        | 37874/225000 [82:28:34<340:59:30,  6.56s/it]
 17%|█▋        | 37875/225000 [82:28:44<392:11:25,  7.55s/it]
 17%|█▋        | 37876/225000 [82:28:48<337:32:46,  6.49s/it]
 17%|█▋        | 37877/225000 [82:28:56<367:57:07,  7.08s/it]
 17%|█▋        | 37878/225000 [82:29:01<332:35:49,  6.40s/it]
 17%|█▋        | 37879/225000 [82:29:06<316:07:59,  6.08s/it]
 17%|█▋        | 37880/225000 [82:29:11<298:09:36,  5.74s/it]
 17%|█▋        | 37881/225000 [82:29:24<407:12:15,  7.83s/it]
 17%|█▋        | 37882/225000 [82:29:30<373:58:58,  7.20s/it]
 17%|█▋        | 37883/225000 [82:29:36<359:54:19,  6.92s/it]
 17%|█▋        | 37884/225000 [82:29:44<380:03:50,  7.31s/it]
 17%|█▋        | 37885/225000 [82:29:51<376:17:36,  7.24s/it]
 17%|█▋        | 37886/225000 [82:29:57<342:57:02,  6.60s/it]
 17%|█▋        | 37887/225000 [82:30:05<375:47:10,  7.23s/it]
 17%|█▋        | 37888/225000 [82:30:13<390:17:16,  7.51s/it]
 17%|█▋        | 37889/225000 [82:30:21<385:06:18,  7.41s/it]
 17%|█▋        | 37890/225000 [82:30:28<380:11:01,  7.31s/it]
 17%|█▋        | 37891/225000 [82:30:32<328:06:38,  6.31s/it]
 17%|█▋        | 37892/225000 [82:30:40<357:55:02,  6.89s/it]
 17%|█▋        | 37893/225000 [82:30:45<337:22:01,  6.49s/it]
 17%|█▋        | 37894/225000 [82:30:54<372:39:38,  7.17s/it]
 17%|█▋        | 37895/225000 [82:30:58<319:02:01,  6.14s/it]
 17%|█▋        | 37896/225000 [82:31:08<383:14:12,  7.37s/it]
 17%|█▋        | 37897/225000 [82:31:15<381:46:47,  7.35s/it]
 17%|█▋        | 37898/225000 [82:31:20<330:54:29,  6.37s/it]
 17%|█▋        | 37899/225000 [82:31:24<296:19:20,  5.70s/it]
 17%|█▋        | 37900/225000 [82:31:34<368:49:55,  7.10s/it]
                                                             
{'loss': '0.7124', 'grad_norm': '0.7308', 'learning_rate': '4.795e-05', 'epoch': '0.1684', 'num_input_tokens_seen': 139888080, 'train_runtime': '2.971e+05', 'train_tokens_per_second': '470.9'}

 17%|█▋        | 37900/225000 [82:31:34<368:49:55,  7.10s/it]
 17%|█▋        | 37901/225000 [82:31:40<356:13:08,  6.85s/it]
 17%|█▋        | 37902/225000 [82:31:47<348:27:14,  6.70s/it]
 17%|█▋        | 37903/225000 [82:31:54<359:51:30,  6.92s/it]
 17%|█▋        | 37904/225000 [82:31:59<333:50:13,  6.42s/it]
 17%|█▋        | 37905/225000 [82:32:04<313:13:32,  6.03s/it]
 17%|█▋        | 37906/225000 [82:32:10<307:42:40,  5.92s/it]
 17%|█▋        | 37907/225000 [82:32:16<302:39:44,  5.82s/it]
 17%|█▋        | 37908/225000 [82:32:25<349:27:17,  6.72s/it]
 17%|█▋        | 37909/225000 [82:32:33<381:06:21,  7.33s/it]
 17%|█▋        | 37910/225000 [82:32:38<339:35:41,  6.53s/it]
 17%|█▋        | 37911/225000 [82:32:42<306:14:22,  5.89s/it]
 17%|█▋        | 37912/225000 [82:32:54<387:35:23,  7.46s/it]
 17%|█▋        | 37913/225000 [82:32:59<354:00:55,  6.81s/it]
 17%|█▋        | 37914/225000 [82:33:04<326:08:45,  6.28s/it]
 17%|█▋        | 37915/225000 [82:33:09<308:57:56,  5.95s/it]
 17%|█▋        | 37916/225000 [82:33:15<305:04:47,  5.87s/it]
 17%|█▋        | 37917/225000 [82:33:23<349:17:03,  6.72s/it]
 17%|█▋        | 37918/225000 [82:33:35<423:05:42,  8.14s/it]
 17%|█▋        | 37919/225000 [82:33:39<361:31:37,  6.96s/it]
 17%|█▋        | 37920/225000 [82:33:47<384:29:27,  7.40s/it]
 17%|█▋        | 37921/225000 [82:33:56<402:22:53,  7.74s/it]
 17%|█▋        | 37922/225000 [82:34:01<351:44:06,  6.77s/it]
 17%|█▋        | 37923/225000 [82:34:08<360:28:24,  6.94s/it]
 17%|█▋        | 37924/225000 [82:34:13<334:00:45,  6.43s/it]
 17%|█▋        | 37925/225000 [82:34:18<312:44:41,  6.02s/it]
 17%|█▋        | 37926/225000 [82:34:23<294:14:22,  5.66s/it]
 17%|█▋        | 37927/225000 [82:34:28<286:54:10,  5.52s/it]
 17%|█▋        | 37928/225000 [82:34:33<279:47:51,  5.38s/it]
 17%|█▋        | 37929/225000 [82:34:42<332:15:41,  6.39s/it]
 17%|█▋        | 37930/225000 [82:34:53<401:15:24,  7.72s/it]
 17%|█▋        | 37931/225000 [82:34:57<346:21:59,  6.67s/it]
 17%|█▋        | 37932/225000 [82:35:01<304:14:01,  5.85s/it]
 17%|█▋        | 37933/225000 [82:35:13<397:30:12,  7.65s/it]
 17%|█▋        | 37934/225000 [82:35:19<374:48:50,  7.21s/it]
 17%|█▋        | 37935/225000 [82:35:25<355:44:42,  6.85s/it]
 17%|█▋        | 37936/225000 [82:35:33<378:14:10,  7.28s/it]
 17%|█▋        | 37937/225000 [82:35:38<343:01:32,  6.60s/it]
 17%|█▋        | 37938/225000 [82:35:43<316:50:19,  6.10s/it]
 17%|█▋        | 37939/225000 [82:35:48<299:33:30,  5.77s/it]
 17%|█▋        | 37940/225000 [82:35:53<280:20:05,  5.40s/it]
 17%|█▋        | 37941/225000 [82:36:06<398:56:39,  7.68s/it]
 17%|█▋        | 37942/225000 [82:36:10<352:41:27,  6.79s/it]
 17%|█▋        | 37943/225000 [82:36:15<317:29:22,  6.11s/it]
 17%|█▋        | 37944/225000 [82:36:26<397:02:18,  7.64s/it]
 17%|█▋        | 37945/225000 [82:36:30<343:56:47,  6.62s/it]
 17%|█▋        | 37946/225000 [82:36:38<356:48:06,  6.87s/it]
 17%|█▋        | 37947/225000 [82:36:42<307:47:39,  5.92s/it]
 17%|█▋        | 37948/225000 [82:36:51<359:32:26,  6.92s/it]
 17%|█▋        | 37949/225000 [82:36:58<365:25:23,  7.03s/it]
 17%|█▋        | 37950/225000 [82:37:02<311:11:07,  5.99s/it]
                                                             
{'loss': '0.6988', 'grad_norm': '0.6982', 'learning_rate': '4.795e-05', 'epoch': '0.1687', 'num_input_tokens_seen': 140034464, 'train_runtime': '2.974e+05', 'train_tokens_per_second': '470.8'}

 17%|█▋        | 37950/225000 [82:37:02<311:11:07,  5.99s/it]
 17%|█▋        | 37951/225000 [82:37:06<282:54:42,  5.45s/it]
 17%|█▋        | 37952/225000 [82:37:13<309:14:49,  5.95s/it]
 17%|█▋        | 37953/225000 [82:37:24<394:56:32,  7.60s/it]
 17%|█▋        | 37954/225000 [82:37:33<414:21:32,  7.98s/it]
 17%|█▋        | 37955/225000 [82:37:40<386:29:52,  7.44s/it]
 17%|█▋        | 37956/225000 [82:37:45<353:31:27,  6.80s/it]
 17%|█▋        | 37957/225000 [82:37:53<369:40:50,  7.12s/it]
 17%|█▋        | 37958/225000 [82:37:57<319:54:26,  6.16s/it]
 17%|█▋        | 37959/225000 [82:38:02<300:29:54,  5.78s/it]
 17%|█▋        | 37960/225000 [82:38:11<351:31:44,  6.77s/it]
 17%|█▋        | 37961/225000 [82:38:16<328:19:36,  6.32s/it]
 17%|█▋        | 37962/225000 [82:38:21<311:22:38,  5.99s/it]
 17%|█▋        | 37963/225000 [82:38:26<298:10:06,  5.74s/it]
 17%|█▋        | 37964/225000 [82:38:32<292:38:39,  5.63s/it]
 17%|█▋        | 37965/225000 [82:38:37<294:18:43,  5.66s/it]
 17%|█▋        | 37966/225000 [82:38:46<344:35:27,  6.63s/it]
 17%|█▋        | 37967/225000 [82:38:59<442:00:48,  8.51s/it]
 17%|█▋        | 37968/225000 [82:39:03<376:13:23,  7.24s/it]
 17%|█▋        | 37969/225000 [82:39:11<383:36:39,  7.38s/it]
 17%|█▋        | 37970/225000 [82:39:19<384:52:51,  7.41s/it]
 17%|█▋        | 37971/225000 [82:39:23<340:40:53,  6.56s/it]
 17%|█▋        | 37972/225000 [82:39:35<421:26:16,  8.11s/it]
 17%|█▋        | 37973/225000 [82:39:39<365:39:48,  7.04s/it]
 17%|█▋        | 37974/225000 [82:39:44<334:02:41,  6.43s/it]
 17%|█▋        | 37975/225000 [82:39:50<315:19:58,  6.07s/it]
 17%|█▋        | 37976/225000 [82:39:55<300:05:53,  5.78s/it]
 17%|█▋        | 37977/225000 [82:40:00<285:44:28,  5.50s/it]
 17%|█▋        | 37978/225000 [82:40:08<335:24:48,  6.46s/it]
 17%|█▋        | 37979/225000 [82:40:18<389:12:59,  7.49s/it]
 17%|█▋        | 37980/225000 [82:40:22<332:27:20,  6.40s/it]
 17%|█▋        | 37981/225000 [82:40:27<316:17:50,  6.09s/it]
 17%|█▋        | 37982/225000 [82:40:33<302:12:34,  5.82s/it]
 17%|█▋        | 37983/225000 [82:40:38<295:20:28,  5.69s/it]
 17%|█▋        | 37984/225000 [82:40:48<364:17:58,  7.01s/it]
 17%|█▋        | 37985/225000 [82:40:58<405:58:44,  7.82s/it]
 17%|█▋        | 37986/225000 [82:41:02<345:59:01,  6.66s/it]
 17%|█▋        | 37987/225000 [82:41:08<340:33:05,  6.56s/it]
 17%|█▋        | 37988/225000 [82:41:17<372:27:45,  7.17s/it]
 17%|█▋        | 37989/225000 [82:41:25<383:30:14,  7.38s/it]
 17%|█▋        | 37990/225000 [82:41:31<364:36:04,  7.02s/it]
 17%|█▋        | 37991/225000 [82:41:39<379:56:53,  7.31s/it]
 17%|█▋        | 37992/225000 [82:41:43<327:40:59,  6.31s/it]
 17%|█▋        | 37993/225000 [82:41:52<380:40:11,  7.33s/it]
 17%|█▋        | 37994/225000 [82:41:57<337:25:31,  6.50s/it]
 17%|█▋        | 37995/225000 [82:42:01<305:04:41,  5.87s/it]
 17%|█▋        | 37996/225000 [82:42:07<297:58:55,  5.74s/it]
 17%|█▋        | 37997/225000 [82:42:12<291:18:22,  5.61s/it]
 17%|█▋        | 37998/225000 [82:42:20<326:27:17,  6.28s/it]
 17%|█▋        | 37999/225000 [82:42:33<436:33:09,  8.40s/it]
 17%|█▋        | 38000/225000 [82:42:38<383:00:34,  7.37s/it]
                                                             
{'loss': '0.6921', 'grad_norm': '0.6614', 'learning_rate': '4.794e-05', 'epoch': '0.1689', 'num_input_tokens_seen': 140181240, 'train_runtime': '2.978e+05', 'train_tokens_per_second': '470.8'}

 17%|█▋        | 38000/225000 [82:42:38<383:00:34,  7.37s/it]
 17%|█▋        | 38001/225000 [82:42:43<341:25:46,  6.57s/it]
 17%|█▋        | 38002/225000 [82:42:48<311:06:35,  5.99s/it]
 17%|█▋        | 38003/225000 [82:42:53<294:31:45,  5.67s/it]
 17%|█▋        | 38004/225000 [82:43:06<410:45:30,  7.91s/it]
 17%|█▋        | 38005/225000 [82:43:12<387:16:07,  7.46s/it]
 17%|█▋        | 38006/225000 [82:43:17<350:24:11,  6.75s/it]
 17%|█▋        | 38007/225000 [82:43:23<333:09:25,  6.41s/it]
 17%|█▋        | 38008/225000 [82:43:28<312:52:32,  6.02s/it]
 17%|█▋        | 38009/225000 [82:43:33<301:02:02,  5.80s/it]
 17%|█▋        | 38010/225000 [82:43:39<300:31:02,  5.79s/it]
 17%|█▋        | 38011/225000 [82:43:44<294:26:33,  5.67s/it]
 17%|█▋        | 38012/225000 [82:43:56<392:13:45,  7.55s/it]
 17%|█▋        | 38013/225000 [82:44:06<425:51:21,  8.20s/it]
 17%|█▋        | 38014/225000 [82:44:12<394:56:39,  7.60s/it]
 17%|█▋        | 38015/225000 [82:44:18<373:30:32,  7.19s/it]
 17%|█▋        | 38016/225000 [82:44:26<383:52:10,  7.39s/it]
 17%|█▋        | 38017/225000 [82:44:31<339:45:07,  6.54s/it]
 17%|█▋        | 38018/225000 [82:44:36<325:46:31,  6.27s/it]
 17%|█▋        | 38019/225000 [82:44:42<315:12:39,  6.07s/it]
 17%|█▋        | 38020/225000 [82:44:47<303:30:15,  5.84s/it]
 17%|█▋        | 38021/225000 [82:44:57<355:57:21,  6.85s/it]
 17%|█▋        | 38022/225000 [82:45:09<448:10:12,  8.63s/it]
 17%|█▋        | 38023/225000 [82:45:14<384:29:13,  7.40s/it]
 17%|█▋        | 38024/225000 [82:45:18<338:51:44,  6.52s/it]
 17%|█▋        | 38025/225000 [82:45:23<305:19:24,  5.88s/it]
 17%|█▋        | 38026/225000 [82:45:28<289:01:30,  5.56s/it]
 17%|█▋        | 38027/225000 [82:45:35<318:51:43,  6.14s/it]
 17%|█▋        | 38028/225000 [82:45:48<419:10:39,  8.07s/it]
 17%|█▋        | 38029/225000 [82:45:52<358:14:04,  6.90s/it]
 17%|█▋        | 38030/225000 [82:46:02<416:34:44,  8.02s/it]
 17%|█▋        | 38031/225000 [82:46:09<400:02:45,  7.70s/it]
 17%|█▋        | 38032/225000 [82:46:13<343:21:43,  6.61s/it]
 17%|█▋        | 38033/225000 [82:46:18<307:36:44,  5.92s/it]
 17%|█▋        | 38034/225000 [82:46:30<408:38:48,  7.87s/it]
 17%|█▋        | 38035/225000 [82:46:35<356:44:27,  6.87s/it]
 17%|█▋        | 38036/225000 [82:46:38<306:44:22,  5.91s/it]
 17%|█▋        | 38037/225000 [82:46:50<398:47:03,  7.68s/it]
 17%|█▋        | 38038/225000 [82:46:54<338:51:44,  6.52s/it]
 17%|█▋        | 38039/225000 [82:47:00<326:42:20,  6.29s/it]
 17%|█▋        | 38040/225000 [82:47:09<365:53:33,  7.05s/it]
 17%|█▋        | 38041/225000 [82:47:13<322:51:38,  6.22s/it]
 17%|█▋        | 38042/225000 [82:47:17<295:14:11,  5.68s/it]
 17%|█▋        | 38043/225000 [82:47:26<344:15:32,  6.63s/it]
 17%|█▋        | 38044/225000 [82:47:34<361:00:10,  6.95s/it]
 17%|█▋        | 38045/225000 [82:47:41<365:37:17,  7.04s/it]
 17%|█▋        | 38046/225000 [82:47:45<317:51:02,  6.12s/it]
 17%|█▋        | 38047/225000 [82:47:56<391:51:18,  7.55s/it]
 17%|█▋        | 38048/225000 [82:48:00<342:09:29,  6.59s/it]
 17%|█▋        | 38049/225000 [82:48:08<353:41:31,  6.81s/it]
 17%|█▋        | 38050/225000 [82:48:13<325:31:26,  6.27s/it]
                                                             
{'loss': '0.7034', 'grad_norm': '0.793', 'learning_rate': '4.793e-05', 'epoch': '0.1691', 'num_input_tokens_seen': 140328272, 'train_runtime': '2.981e+05', 'train_tokens_per_second': '470.8'}

 17%|█▋        | 38050/225000 [82:48:13<325:31:26,  6.27s/it]
 17%|█▋        | 38051/225000 [82:48:20<342:13:18,  6.59s/it]
 17%|█▋        | 38052/225000 [82:48:25<316:00:42,  6.09s/it]
 17%|█▋        | 38053/225000 [82:48:33<344:43:41,  6.64s/it]
 17%|█▋        | 38054/225000 [82:48:38<320:13:03,  6.17s/it]
 17%|█▋        | 38055/225000 [82:48:43<304:34:41,  5.87s/it]
 17%|█▋        | 38056/225000 [82:48:48<298:01:10,  5.74s/it]
 17%|█▋        | 38057/225000 [82:48:53<285:01:18,  5.49s/it]
 17%|█▋        | 38058/225000 [82:48:58<276:51:22,  5.33s/it]
 17%|█▋        | 38059/225000 [82:49:10<369:57:29,  7.12s/it]
 17%|█▋        | 38060/225000 [82:49:16<358:10:21,  6.90s/it]
 17%|█▋        | 38061/225000 [82:49:21<322:28:27,  6.21s/it]
 17%|█▋        | 38062/225000 [82:49:26<311:13:48,  5.99s/it]
 17%|█▋        | 38063/225000 [82:49:40<429:19:39,  8.27s/it]
 17%|█▋        | 38064/225000 [82:49:45<379:58:44,  7.32s/it]
 17%|█▋        | 38065/225000 [82:49:50<347:04:20,  6.68s/it]
 17%|█▋        | 38066/225000 [82:49:55<325:46:39,  6.27s/it]
 17%|█▋        | 38067/225000 [82:50:00<306:20:01,  5.90s/it]
 17%|█▋        | 38068/225000 [82:50:12<394:23:03,  7.60s/it]
 17%|█▋        | 38069/225000 [82:50:16<335:10:02,  6.45s/it]
 17%|█▋        | 38070/225000 [82:50:21<311:41:36,  6.00s/it]
 17%|█▋        | 38071/225000 [82:50:25<289:49:35,  5.58s/it]
 17%|█▋        | 38072/225000 [82:50:32<305:53:16,  5.89s/it]
 17%|█▋        | 38073/225000 [82:50:45<413:03:45,  7.96s/it]
 17%|█▋        | 38074/225000 [82:50:50<375:53:33,  7.24s/it]
 17%|█▋        | 38075/225000 [82:50:59<393:07:08,  7.57s/it]
 17%|█▋        | 38076/225000 [82:51:04<355:44:41,  6.85s/it]
 17%|█▋        | 38077/225000 [82:51:08<318:50:53,  6.14s/it]
 17%|█▋        | 38078/225000 [82:51:18<369:15:08,  7.11s/it]
 17%|█▋        | 38079/225000 [82:51:22<334:38:30,  6.45s/it]
 17%|█▋        | 38080/225000 [82:51:27<310:11:16,  5.97s/it]
 17%|█▋        | 38081/225000 [82:51:32<296:08:03,  5.70s/it]
 17%|█▋        | 38082/225000 [82:51:40<320:29:42,  6.17s/it]
 17%|█▋        | 38083/225000 [82:51:48<350:40:06,  6.75s/it]
 17%|█▋        | 38084/225000 [82:51:53<329:20:58,  6.34s/it]
 17%|█▋        | 38085/225000 [82:52:01<357:37:50,  6.89s/it]
 17%|█▋        | 38086/225000 [82:52:06<321:21:35,  6.19s/it]
 17%|█▋        | 38087/225000 [82:52:10<293:50:52,  5.66s/it]
 17%|█▋        | 38088/225000 [82:52:21<372:57:45,  7.18s/it]
 17%|█▋        | 38089/225000 [82:52:28<370:39:50,  7.14s/it]
 17%|█▋        | 38090/225000 [82:52:36<386:57:27,  7.45s/it]
 17%|█▋        | 38091/225000 [82:52:45<401:51:16,  7.74s/it]
 17%|█▋        | 38092/225000 [82:52:49<342:05:39,  6.59s/it]
 17%|█▋        | 38093/225000 [82:52:58<381:11:19,  7.34s/it]
 17%|█▋        | 38094/225000 [82:53:04<358:04:59,  6.90s/it]
 17%|█▋        | 38095/225000 [82:53:11<369:11:37,  7.11s/it]
 17%|█▋        | 38096/225000 [82:53:16<330:42:38,  6.37s/it]
 17%|█▋        | 38097/225000 [82:53:25<378:47:14,  7.30s/it]
 17%|█▋        | 38098/225000 [82:53:29<322:38:43,  6.21s/it]
 17%|█▋        | 38099/225000 [82:53:38<360:12:22,  6.94s/it]
 17%|█▋        | 38100/225000 [82:53:42<321:17:11,  6.19s/it]
                                                             
{'loss': '0.7054', 'grad_norm': '0.7463', 'learning_rate': '4.792e-05', 'epoch': '0.1693', 'num_input_tokens_seen': 140474448, 'train_runtime': '2.984e+05', 'train_tokens_per_second': '470.7'}

 17%|█▋        | 38100/225000 [82:53:42<321:17:11,  6.19s/it]
 17%|█▋        | 38101/225000 [82:53:46<290:56:29,  5.60s/it]
 17%|█▋        | 38102/225000 [82:53:58<389:12:31,  7.50s/it]
 17%|█▋        | 38103/225000 [82:54:03<348:12:02,  6.71s/it]
 17%|█▋        | 38104/225000 [82:54:08<327:28:32,  6.31s/it]
 17%|█▋        | 38105/225000 [82:54:14<310:26:04,  5.98s/it]
 17%|█▋        | 38106/225000 [82:54:19<295:20:50,  5.69s/it]
 17%|█▋        | 38107/225000 [82:54:24<285:27:26,  5.50s/it]
 17%|█▋        | 38108/225000 [82:54:35<374:03:23,  7.21s/it]
 17%|█▋        | 38109/225000 [82:54:45<425:31:36,  8.20s/it]
 17%|█▋        | 38110/225000 [82:54:50<365:51:01,  7.05s/it]
 17%|█▋        | 38111/225000 [82:54:57<368:34:03,  7.10s/it]
 17%|█▋        | 38112/225000 [82:55:02<335:13:29,  6.46s/it]
 17%|█▋        | 38113/225000 [82:55:07<317:49:39,  6.12s/it]
 17%|█▋        | 38114/225000 [82:55:12<298:59:28,  5.76s/it]
 17%|█▋        | 38115/225000 [82:55:23<373:13:47,  7.19s/it]
 17%|█▋        | 38116/225000 [82:55:31<387:38:08,  7.47s/it]
 17%|█▋        | 38117/225000 [82:55:40<410:08:49,  7.90s/it]
 17%|█▋        | 38118/225000 [82:55:43<345:32:14,  6.66s/it]
 17%|█▋        | 38119/225000 [82:55:49<329:37:15,  6.35s/it]
 17%|█▋        | 38120/225000 [82:55:54<304:02:36,  5.86s/it]
 17%|█▋        | 38121/225000 [82:56:01<325:10:15,  6.26s/it]
 17%|█▋        | 38122/225000 [82:56:14<433:04:49,  8.34s/it]
 17%|█▋        | 38123/225000 [82:56:18<364:24:23,  7.02s/it]
 17%|█▋        | 38124/225000 [82:56:27<388:26:25,  7.48s/it]
 17%|█▋        | 38125/225000 [82:56:32<348:18:25,  6.71s/it]
 17%|█▋        | 38126/225000 [82:56:38<339:07:23,  6.53s/it]
 17%|█▋        | 38127/225000 [82:56:43<317:15:25,  6.11s/it]
 17%|█▋        | 38128/225000 [82:56:49<313:05:24,  6.03s/it]
 17%|█▋        | 38129/225000 [82:57:01<408:18:00,  7.87s/it]
 17%|█▋        | 38130/225000 [82:57:04<332:52:35,  6.41s/it]
 17%|█▋        | 38131/225000 [82:57:09<313:38:13,  6.04s/it]
 17%|█▋        | 38132/225000 [82:57:15<307:32:51,  5.92s/it]
 17%|█▋        | 38133/225000 [82:57:20<296:34:17,  5.71s/it]
 17%|█▋        | 38134/225000 [82:57:25<288:59:48,  5.57s/it]
 17%|█▋        | 38135/225000 [82:57:39<419:04:45,  8.07s/it]
 17%|█▋        | 38136/225000 [82:57:46<400:44:03,  7.72s/it]
 17%|█▋        | 38137/225000 [82:57:54<405:09:46,  7.81s/it]
 17%|█▋        | 38138/225000 [82:58:01<386:13:57,  7.44s/it]
 17%|█▋        | 38139/225000 [82:58:06<355:35:38,  6.85s/it]
 17%|█▋        | 38140/225000 [82:58:11<325:50:41,  6.28s/it]
 17%|█▋        | 38141/225000 [82:58:22<405:37:27,  7.81s/it]
 17%|█▋        | 38142/225000 [82:58:28<365:06:35,  7.03s/it]
 17%|█▋        | 38143/225000 [82:58:33<343:29:45,  6.62s/it]
 17%|█▋        | 38144/225000 [82:58:39<332:07:28,  6.40s/it]
 17%|█▋        | 38145/225000 [82:58:45<318:53:54,  6.14s/it]
 17%|█▋        | 38146/225000 [82:58:50<310:32:02,  5.98s/it]
 17%|█▋        | 38147/225000 [82:58:55<297:46:02,  5.74s/it]
 17%|█▋        | 38148/225000 [82:59:01<300:25:15,  5.79s/it]
 17%|█▋        | 38149/225000 [82:59:14<401:41:31,  7.74s/it]
 17%|█▋        | 38150/225000 [82:59:20<386:44:33,  7.45s/it]
                                                             
{'loss': '0.7067', 'grad_norm': '0.6405', 'learning_rate': '4.792e-05', 'epoch': '0.1696', 'num_input_tokens_seen': 140621288, 'train_runtime': '2.988e+05', 'train_tokens_per_second': '470.7'}

 17%|█▋        | 38150/225000 [82:59:20<386:44:33,  7.45s/it]
 17%|█▋        | 38151/225000 [82:59:28<385:21:19,  7.42s/it]
 17%|█▋        | 38152/225000 [82:59:32<334:09:04,  6.44s/it]
 17%|█▋        | 38153/225000 [82:59:37<305:20:25,  5.88s/it]
 17%|█▋        | 38154/225000 [82:59:48<399:30:49,  7.70s/it]
 17%|█▋        | 38155/225000 [82:59:53<354:34:45,  6.83s/it]
 17%|█▋        | 38156/225000 [82:59:58<326:40:51,  6.29s/it]
 17%|█▋        | 38157/225000 [83:00:03<303:32:52,  5.85s/it]
 17%|█▋        | 38158/225000 [83:00:14<388:02:58,  7.48s/it]
 17%|█▋        | 38159/225000 [83:00:18<329:19:05,  6.35s/it]
 17%|█▋        | 38160/225000 [83:00:27<363:25:38,  7.00s/it]
 17%|█▋        | 38161/225000 [83:00:34<372:32:26,  7.18s/it]
 17%|█▋        | 38162/225000 [83:00:40<343:08:12,  6.61s/it]
 17%|█▋        | 38163/225000 [83:00:45<323:07:45,  6.23s/it]
 17%|█▋        | 38164/225000 [83:00:50<308:48:39,  5.95s/it]
 17%|█▋        | 38165/225000 [83:00:55<297:30:00,  5.73s/it]
 17%|█▋        | 38166/225000 [83:01:00<281:14:13,  5.42s/it]
 17%|█▋        | 38167/225000 [83:01:11<362:14:04,  6.98s/it]
 17%|█▋        | 38168/225000 [83:01:20<396:59:19,  7.65s/it]
 17%|█▋        | 38169/225000 [83:01:24<337:50:13,  6.51s/it]
 17%|█▋        | 38170/225000 [83:01:35<409:55:22,  7.90s/it]
 17%|█▋        | 38171/225000 [83:01:41<380:05:52,  7.32s/it]
 17%|█▋        | 38172/225000 [83:01:49<388:22:45,  7.48s/it]
 17%|█▋        | 38173/225000 [83:01:54<358:11:28,  6.90s/it]
 17%|█▋        | 38174/225000 [83:02:00<337:12:04,  6.50s/it]
 17%|█▋        | 38175/225000 [83:02:06<325:16:50,  6.27s/it]
 17%|█▋        | 38176/225000 [83:02:11<316:11:52,  6.09s/it]
 17%|█▋        | 38177/225000 [83:02:20<359:29:29,  6.93s/it]
 17%|█▋        | 38178/225000 [83:02:27<361:49:54,  6.97s/it]
 17%|█▋        | 38179/225000 [83:02:33<339:47:33,  6.55s/it]
 17%|█▋        | 38180/225000 [83:02:42<385:35:00,  7.43s/it]
 17%|█▋        | 38181/225000 [83:02:48<354:21:02,  6.83s/it]
 17%|█▋        | 38182/225000 [83:02:53<324:23:02,  6.25s/it]
 17%|█▋        | 38183/225000 [83:02:58<314:31:10,  6.06s/it]
 17%|█▋        | 38184/225000 [83:03:04<305:58:49,  5.90s/it]
 17%|█▋        | 38185/225000 [83:03:09<295:51:16,  5.70s/it]
 17%|█▋        | 38186/225000 [83:03:19<359:04:02,  6.92s/it]
 17%|█▋        | 38187/225000 [83:03:27<382:17:12,  7.37s/it]
 17%|█▋        | 38188/225000 [83:03:33<362:35:53,  6.99s/it]
 17%|█▋        | 38189/225000 [83:03:42<394:31:58,  7.60s/it]
 17%|█▋        | 38190/225000 [83:03:47<347:56:57,  6.71s/it]
 17%|█▋        | 38191/225000 [83:03:56<383:08:15,  7.38s/it]
 17%|█▋        | 38192/225000 [83:04:02<365:23:11,  7.04s/it]
 17%|█▋        | 38193/225000 [83:04:11<387:35:10,  7.47s/it]
 17%|█▋        | 38194/225000 [83:04:17<369:28:48,  7.12s/it]
 17%|█▋        | 38195/225000 [83:04:21<329:29:43,  6.35s/it]
 17%|█▋        | 38196/225000 [83:04:32<402:35:41,  7.76s/it]
 17%|█▋        | 38197/225000 [83:04:39<379:43:11,  7.32s/it]
 17%|█▋        | 38198/225000 [83:04:42<324:02:50,  6.24s/it]
 17%|█▋        | 38199/225000 [83:04:51<357:56:46,  6.90s/it]
 17%|█▋        | 38200/225000 [83:04:56<327:57:25,  6.32s/it]
                                                             
{'loss': '0.7007', 'grad_norm': '0.6867', 'learning_rate': '4.791e-05', 'epoch': '0.1698', 'num_input_tokens_seen': 140768224, 'train_runtime': '2.991e+05', 'train_tokens_per_second': '470.6'}

 17%|█▋        | 38200/225000 [83:04:56<327:57:25,  6.32s/it]
 17%|█▋        | 38201/225000 [83:05:01<307:08:10,  5.92s/it]
 17%|█▋        | 38202/225000 [83:05:06<290:58:31,  5.61s/it]
 17%|█▋        | 38203/225000 [83:05:11<279:52:09,  5.39s/it]
 17%|█▋        | 38204/225000 [83:05:20<345:58:01,  6.67s/it]
 17%|█▋        | 38205/225000 [83:05:28<368:48:59,  7.11s/it]
 17%|█▋        | 38206/225000 [83:05:36<380:51:02,  7.34s/it]
 17%|█▋        | 38207/225000 [83:05:42<348:14:49,  6.71s/it]
 17%|█▋        | 38208/225000 [83:05:49<361:38:47,  6.97s/it]
 17%|█▋        | 38209/225000 [83:05:54<330:35:32,  6.37s/it]
 17%|█▋        | 38210/225000 [83:05:59<313:03:27,  6.03s/it]
 17%|█▋        | 38211/225000 [83:06:05<303:44:20,  5.85s/it]
 17%|█▋        | 38212/225000 [83:06:10<289:16:30,  5.58s/it]
 17%|█▋        | 38213/225000 [83:06:23<406:30:09,  7.83s/it]
 17%|█▋        | 38214/225000 [83:06:29<380:46:18,  7.34s/it]
 17%|█▋        | 38215/225000 [83:06:39<416:30:35,  8.03s/it]
 17%|█▋        | 38216/225000 [83:06:42<347:34:12,  6.70s/it]
 17%|█▋        | 38217/225000 [83:06:51<384:14:52,  7.41s/it]
 17%|█▋        | 38218/225000 [83:06:59<382:01:25,  7.36s/it]
 17%|█▋        | 38219/225000 [83:07:08<412:50:34,  7.96s/it]
 17%|█▋        | 38220/225000 [83:07:13<361:47:03,  6.97s/it]
 17%|█▋        | 38221/225000 [83:07:17<325:09:55,  6.27s/it]
 17%|█▋        | 38222/225000 [83:07:23<313:31:54,  6.04s/it]
 17%|█▋        | 38223/225000 [83:07:32<364:32:26,  7.03s/it]
 17%|█▋        | 38224/225000 [83:07:37<330:05:20,  6.36s/it]
 17%|█▋        | 38225/225000 [83:07:42<316:22:29,  6.10s/it]
 17%|█▋        | 38226/225000 [83:07:48<312:10:49,  6.02s/it]
 17%|█▋        | 38227/225000 [83:07:54<308:29:43,  5.95s/it]
 17%|█▋        | 38228/225000 [83:07:59<300:00:07,  5.78s/it]
 17%|█▋        | 38229/225000 [83:08:05<300:07:31,  5.78s/it]
 17%|█▋        | 38230/225000 [83:08:10<293:31:00,  5.66s/it]
 17%|█▋        | 38231/225000 [83:08:21<372:33:43,  7.18s/it]
 17%|█▋        | 38232/225000 [83:08:32<421:09:42,  8.12s/it]
 17%|█▋        | 38233/225000 [83:08:35<351:37:41,  6.78s/it]
 17%|█▋        | 38234/225000 [83:08:44<378:24:09,  7.29s/it]
 17%|█▋        | 38235/225000 [83:08:48<328:48:35,  6.34s/it]
 17%|█▋        | 38236/225000 [83:08:55<336:48:32,  6.49s/it]
 17%|█▋        | 38237/225000 [83:09:03<362:00:06,  6.98s/it]
 17%|█▋        | 38238/225000 [83:09:10<372:24:44,  7.18s/it]
 17%|█▋        | 38239/225000 [83:09:15<326:58:19,  6.30s/it]
 17%|█▋        | 38240/225000 [83:09:20<307:05:57,  5.92s/it]
 17%|█▋        | 38241/225000 [83:09:25<298:03:56,  5.75s/it]
 17%|█▋        | 38242/225000 [83:09:30<288:43:00,  5.57s/it]
 17%|█▋        | 38243/225000 [83:09:41<373:37:45,  7.20s/it]
 17%|█▋        | 38244/225000 [83:09:49<376:41:59,  7.26s/it]
 17%|█▋        | 38245/225000 [83:09:53<326:40:00,  6.30s/it]
 17%|█▋        | 38246/225000 [83:09:57<295:39:02,  5.70s/it]
 17%|█▋        | 38247/225000 [83:10:09<395:40:50,  7.63s/it]
 17%|█▋        | 38248/225000 [83:10:14<360:32:45,  6.95s/it]
 17%|█▋        | 38249/225000 [83:10:23<388:38:23,  7.49s/it]
 17%|█▋        | 38250/225000 [83:10:27<333:14:44,  6.42s/it]
                                                             
{'loss': '0.7007', 'grad_norm': '0.7272', 'learning_rate': '4.79e-05', 'epoch': '0.17', 'num_input_tokens_seen': 140914520, 'train_runtime': '2.994e+05', 'train_tokens_per_second': '470.6'}

 17%|█▋        | 38250/225000 [83:10:27<333:14:44,  6.42s/it]
 17%|█▋        | 38251/225000 [83:10:36<367:23:33,  7.08s/it]
 17%|█▋        | 38252/225000 [83:10:41<332:44:30,  6.41s/it]
 17%|█▋        | 38253/225000 [83:10:45<306:42:41,  5.91s/it]
 17%|█▋        | 38254/225000 [83:10:50<288:48:45,  5.57s/it]
 17%|█▋        | 38255/225000 [83:11:02<391:43:52,  7.55s/it]
 17%|█▋        | 38256/225000 [83:11:07<348:45:22,  6.72s/it]
 17%|█▋        | 38257/225000 [83:11:17<394:16:31,  7.60s/it]
 17%|█▋        | 38258/225000 [83:11:21<335:09:07,  6.46s/it]
 17%|█▋        | 38259/225000 [83:11:26<322:21:07,  6.21s/it]
 17%|█▋        | 38260/225000 [83:11:36<375:34:51,  7.24s/it]
 17%|█▋        | 38261/225000 [83:11:39<317:28:51,  6.12s/it]
 17%|█▋        | 38262/225000 [83:11:48<359:30:09,  6.93s/it]
 17%|█▋        | 38263/225000 [83:11:53<325:00:50,  6.27s/it]
 17%|█▋        | 38264/225000 [83:11:58<301:03:23,  5.80s/it]
 17%|█▋        | 38265/225000 [83:12:02<287:37:12,  5.54s/it]
 17%|█▋        | 38266/225000 [83:12:15<388:24:28,  7.49s/it]
 17%|█▋        | 38267/225000 [83:12:18<334:00:12,  6.44s/it]
 17%|█▋        | 38268/225000 [83:12:27<358:33:17,  6.91s/it]
 17%|█▋        | 38269/225000 [83:12:32<335:19:41,  6.46s/it]
 17%|█▋        | 38270/225000 [83:12:39<346:56:42,  6.69s/it]
 17%|█▋        | 38271/225000 [83:12:43<301:07:48,  5.81s/it]
 17%|█▋        | 38272/225000 [83:12:54<388:53:40,  7.50s/it]
 17%|█▋        | 38273/225000 [83:12:59<350:58:29,  6.77s/it]
 17%|█▋        | 38274/225000 [83:13:04<324:08:49,  6.25s/it]
 17%|█▋        | 38275/225000 [83:13:09<305:08:15,  5.88s/it]
 17%|█▋        | 38276/225000 [83:13:15<292:20:29,  5.64s/it]
 17%|█▋        | 38277/225000 [83:13:19<278:04:25,  5.36s/it]
 17%|█▋        | 38278/225000 [83:13:31<378:53:14,  7.30s/it]
 17%|█▋        | 38279/225000 [83:13:40<402:30:25,  7.76s/it]
 17%|█▋        | 38280/225000 [83:13:46<370:31:39,  7.14s/it]
 17%|█▋        | 38281/225000 [83:13:49<315:28:43,  6.08s/it]
 17%|█▋        | 38282/225000 [83:13:55<305:49:07,  5.90s/it]
 17%|█▋        | 38283/225000 [83:14:00<298:10:48,  5.75s/it]
 17%|█▋        | 38284/225000 [83:14:06<292:57:05,  5.65s/it]
 17%|█▋        | 38285/225000 [83:14:11<287:52:02,  5.55s/it]
 17%|█▋        | 38286/225000 [83:14:17<292:28:35,  5.64s/it]
 17%|█▋        | 38287/225000 [83:14:30<410:30:17,  7.91s/it]
 17%|█▋        | 38288/225000 [83:14:36<384:55:24,  7.42s/it]
 17%|█▋        | 38289/225000 [83:14:41<339:41:34,  6.55s/it]
 17%|█▋        | 38290/225000 [83:14:46<316:30:55,  6.10s/it]
 17%|█▋        | 38291/225000 [83:14:51<301:27:47,  5.81s/it]
 17%|█▋        | 38292/225000 [83:14:56<286:19:59,  5.52s/it]
 17%|█▋        | 38293/225000 [83:15:08<393:38:40,  7.59s/it]
 17%|█▋        | 38294/225000 [83:15:14<359:42:03,  6.94s/it]
 17%|█▋        | 38295/225000 [83:15:23<404:28:44,  7.80s/it]
 17%|█▋        | 38296/225000 [83:15:27<346:58:42,  6.69s/it]
 17%|█▋        | 38297/225000 [83:15:37<385:28:58,  7.43s/it]
 17%|█▋        | 38298/225000 [83:15:43<363:43:17,  7.01s/it]
 17%|█▋        | 38299/225000 [83:15:50<369:51:21,  7.13s/it]
 17%|█▋        | 38300/225000 [83:15:55<330:39:26,  6.38s/it]
                                                             
{'loss': '0.6959', 'grad_norm': '0.8519', 'learning_rate': '4.79e-05', 'epoch': '0.1702', 'num_input_tokens_seen': 141062208, 'train_runtime': '2.998e+05', 'train_tokens_per_second': '470.6'}

 17%|█▋        | 38300/225000 [83:15:55<330:39:26,  6.38s/it]
 17%|█▋        | 38301/225000 [83:16:05<384:21:37,  7.41s/it]
 17%|█▋        | 38302/225000 [83:16:10<359:12:14,  6.93s/it]
 17%|█▋        | 38303/225000 [83:16:15<328:19:16,  6.33s/it]
 17%|█▋        | 38304/225000 [83:16:20<304:45:27,  5.88s/it]
 17%|█▋        | 38305/225000 [83:16:25<291:11:53,  5.62s/it]
 17%|█▋        | 38306/225000 [83:16:30<280:07:00,  5.40s/it]
 17%|█▋        | 38307/225000 [83:16:35<270:08:34,  5.21s/it]
 17%|█▋        | 38308/225000 [83:16:44<331:26:53,  6.39s/it]
 17%|█▋        | 38309/225000 [83:16:56<421:13:49,  8.12s/it]
 17%|█▋        | 38310/225000 [83:17:00<355:42:19,  6.86s/it]
 17%|█▋        | 38311/225000 [83:17:05<320:53:05,  6.19s/it]
 17%|█▋        | 38312/225000 [83:17:09<300:55:23,  5.80s/it]
 17%|█▋        | 38313/225000 [83:17:14<282:44:57,  5.45s/it]
 17%|█▋        | 38314/225000 [83:17:19<267:58:24,  5.17s/it]
 17%|█▋        | 38315/225000 [83:17:31<384:21:13,  7.41s/it]
 17%|█▋        | 38316/225000 [83:17:38<380:40:07,  7.34s/it]
 17%|█▋        | 38317/225000 [83:17:43<331:38:22,  6.40s/it]
 17%|█▋        | 38318/225000 [83:17:52<385:20:47,  7.43s/it]
 17%|█▋        | 38319/225000 [83:17:56<321:51:20,  6.21s/it]
 17%|█▋        | 38320/225000 [83:18:04<353:22:53,  6.81s/it]
 17%|█▋        | 38321/225000 [83:18:08<312:00:53,  6.02s/it]
 17%|█▋        | 38322/225000 [83:18:13<289:42:24,  5.59s/it]
 17%|█▋        | 38323/225000 [83:18:19<294:14:02,  5.67s/it]
 17%|█▋        | 38324/225000 [83:18:32<412:09:03,  7.95s/it]
 17%|█▋        | 38325/225000 [83:18:36<346:34:39,  6.68s/it]
 17%|█▋        | 38326/225000 [83:18:42<339:01:00,  6.54s/it]
 17%|█▋        | 38327/225000 [83:18:47<322:54:27,  6.23s/it]
 17%|█▋        | 38328/225000 [83:18:57<371:49:27,  7.17s/it]
 17%|█▋        | 38329/225000 [83:19:01<319:43:16,  6.17s/it]
 17%|█▋        | 38330/225000 [83:19:09<356:20:40,  6.87s/it]
 17%|█▋        | 38331/225000 [83:19:13<315:53:17,  6.09s/it]
 17%|█▋        | 38332/225000 [83:19:22<355:38:04,  6.86s/it]
 17%|█▋        | 38333/225000 [83:19:28<339:50:24,  6.55s/it]
 17%|█▋        | 38334/225000 [83:19:32<305:10:33,  5.89s/it]
 17%|█▋        | 38335/225000 [83:19:38<310:07:28,  5.98s/it]
 17%|█▋        | 38336/225000 [83:19:50<392:57:41,  7.58s/it]
 17%|█▋        | 38337/225000 [83:19:53<331:26:56,  6.39s/it]
 17%|█▋        | 38338/225000 [83:19:59<316:42:29,  6.11s/it]
 17%|█▋        | 38339/225000 [83:20:04<307:07:07,  5.92s/it]
 17%|█▋        | 38340/225000 [83:20:10<305:22:48,  5.89s/it]
 17%|█▋        | 38341/225000 [83:20:16<307:33:17,  5.93s/it]
 17%|█▋        | 38342/225000 [83:20:22<300:47:46,  5.80s/it]
 17%|█▋        | 38343/225000 [83:20:27<293:35:09,  5.66s/it]
 17%|█▋        | 38344/225000 [83:20:39<389:22:38,  7.51s/it]
 17%|█▋        | 38345/225000 [83:20:43<346:33:47,  6.68s/it]
 17%|█▋        | 38346/225000 [83:20:48<316:16:34,  6.10s/it]
 17%|█▋        | 38347/225000 [83:20:53<300:57:32,  5.80s/it]
 17%|█▋        | 38348/225000 [83:21:05<390:14:40,  7.53s/it]
 17%|█▋        | 38349/225000 [83:21:10<359:53:43,  6.94s/it]
 17%|█▋        | 38350/225000 [83:21:15<320:41:17,  6.19s/it]
                                                             
{'loss': '0.6988', 'grad_norm': '0.8075', 'learning_rate': '4.789e-05', 'epoch': '0.1704', 'num_input_tokens_seen': 141207360, 'train_runtime': '3.001e+05', 'train_tokens_per_second': '470.6'}

 17%|█▋        | 38350/225000 [83:21:15<320:41:17,  6.19s/it]
 17%|█▋        | 38351/225000 [83:21:20<299:58:50,  5.79s/it]
 17%|█▋        | 38352/225000 [83:21:24<280:59:19,  5.42s/it]
 17%|█▋        | 38353/225000 [83:21:38<403:09:50,  7.78s/it]
 17%|█▋        | 38354/225000 [83:21:42<347:34:13,  6.70s/it]
 17%|█▋        | 38355/225000 [83:21:51<383:06:12,  7.39s/it]
 17%|█▋        | 38356/225000 [83:21:55<327:41:40,  6.32s/it]
 17%|█▋        | 38357/225000 [83:22:03<363:42:21,  7.02s/it]
 17%|█▋        | 38358/225000 [83:22:07<318:37:09,  6.15s/it]
 17%|█▋        | 38359/225000 [83:22:13<312:45:56,  6.03s/it]
 17%|█▋        | 38360/225000 [83:22:25<396:50:53,  7.65s/it]
 17%|█▋        | 38361/225000 [83:22:29<347:34:31,  6.70s/it]
 17%|█▋        | 38362/225000 [83:22:34<319:01:36,  6.15s/it]
 17%|█▋        | 38363/225000 [83:22:41<336:30:35,  6.49s/it]
 17%|█▋        | 38364/225000 [83:22:49<364:23:36,  7.03s/it]
 17%|█▋        | 38365/225000 [83:22:54<322:29:42,  6.22s/it]
 17%|█▋        | 38366/225000 [83:22:59<301:09:35,  5.81s/it]
 17%|█▋        | 38367/225000 [83:23:06<331:03:53,  6.39s/it]
 17%|█▋        | 38368/225000 [83:23:16<382:32:14,  7.38s/it]
 17%|█▋        | 38369/225000 [83:23:23<367:30:22,  7.09s/it]
 17%|█▋        | 38370/225000 [83:23:29<350:47:52,  6.77s/it]
 17%|█▋        | 38371/225000 [83:23:33<315:30:04,  6.09s/it]
 17%|█▋        | 38372/225000 [83:23:42<353:10:55,  6.81s/it]
 17%|█▋        | 38373/225000 [83:23:49<363:56:05,  7.02s/it]
 17%|█▋        | 38374/225000 [83:23:53<319:29:38,  6.16s/it]
 17%|█▋        | 38375/225000 [83:24:04<394:37:29,  7.61s/it]
 17%|█▋        | 38376/225000 [83:24:09<346:29:27,  6.68s/it]
 17%|█▋        | 38377/225000 [83:24:13<305:54:38,  5.90s/it]
 17%|█▋        | 38378/225000 [83:24:24<389:46:14,  7.52s/it]
 17%|█▋        | 38379/225000 [83:24:30<367:12:06,  7.08s/it]
 17%|█▋        | 38380/225000 [83:24:35<340:27:27,  6.57s/it]
 17%|█▋        | 38381/225000 [83:24:41<319:45:31,  6.17s/it]
 17%|█▋        | 38382/225000 [83:24:46<301:13:29,  5.81s/it]
 17%|█▋        | 38383/225000 [83:24:51<292:15:54,  5.64s/it]
 17%|█▋        | 38384/225000 [83:24:56<284:08:59,  5.48s/it]
 17%|█▋        | 38385/225000 [83:25:01<283:12:47,  5.46s/it]
 17%|█▋        | 38386/225000 [83:25:15<414:01:19,  7.99s/it]
 17%|█▋        | 38387/225000 [83:25:23<410:05:38,  7.91s/it]
 17%|█▋        | 38388/225000 [83:25:29<386:23:34,  7.45s/it]
 17%|█▋        | 38389/225000 [83:25:35<350:10:38,  6.76s/it]
 17%|█▋        | 38390/225000 [83:25:43<376:08:14,  7.26s/it]
 17%|█▋        | 38391/225000 [83:25:49<359:00:27,  6.93s/it]
 17%|█▋        | 38392/225000 [83:25:54<324:38:58,  6.26s/it]
 17%|█▋        | 38393/225000 [83:25:59<299:36:53,  5.78s/it]
 17%|█▋        | 38394/225000 [83:26:04<299:36:01,  5.78s/it]
 17%|█▋        | 38395/225000 [83:26:16<392:16:03,  7.57s/it]
 17%|█▋        | 38396/225000 [83:26:24<393:38:04,  7.59s/it]
 17%|█▋        | 38397/225000 [83:26:29<353:29:29,  6.82s/it]
 17%|█▋        | 38398/225000 [83:26:33<315:17:18,  6.08s/it]
 17%|█▋        | 38399/225000 [83:26:38<292:32:19,  5.64s/it]
 17%|█▋        | 38400/225000 [83:26:47<353:04:36,  6.81s/it]
                                                             
{'loss': '0.6961', 'grad_norm': '0.7058', 'learning_rate': '4.788e-05', 'epoch': '0.1707', 'num_input_tokens_seen': 141353200, 'train_runtime': '3.004e+05', 'train_tokens_per_second': '470.5'}

 17%|█▋        | 38400/225000 [83:26:47<353:04:36,  6.81s/it]
 17%|█▋        | 38401/225000 [83:26:57<401:50:14,  7.75s/it]
 17%|█▋        | 38402/225000 [83:27:05<409:44:51,  7.91s/it]
 17%|█▋        | 38403/225000 [83:27:10<354:19:32,  6.84s/it]
 17%|█▋        | 38404/225000 [83:27:15<324:04:05,  6.25s/it]
 17%|█▋        | 38405/225000 [83:27:25<391:07:33,  7.55s/it]
 17%|█▋        | 38406/225000 [83:27:30<354:58:23,  6.85s/it]
 17%|█▋        | 38407/225000 [83:27:36<327:46:56,  6.32s/it]
 17%|█▋        | 38408/225000 [83:27:41<306:29:20,  5.91s/it]
 17%|█▋        | 38409/225000 [83:27:45<288:56:09,  5.57s/it]
 17%|█▋        | 38410/225000 [83:27:50<279:59:21,  5.40s/it]
 17%|█▋        | 38411/225000 [83:28:03<397:01:41,  7.66s/it]
 17%|█▋        | 38412/225000 [83:28:09<368:08:52,  7.10s/it]
 17%|█▋        | 38413/225000 [83:28:14<331:37:11,  6.40s/it]
 17%|█▋        | 38414/225000 [83:28:18<304:28:05,  5.87s/it]
 17%|█▋        | 38415/225000 [83:28:25<313:39:19,  6.05s/it]
 17%|█▋        | 38416/225000 [83:28:37<412:06:00,  7.95s/it]
 17%|█▋        | 38417/225000 [83:28:41<346:33:56,  6.69s/it]
 17%|█▋        | 38418/225000 [83:28:49<369:22:29,  7.13s/it]
 17%|█▋        | 38419/225000 [83:28:55<341:57:05,  6.60s/it]
 17%|█▋        | 38420/225000 [83:29:03<364:51:46,  7.04s/it]
 17%|█▋        | 38421/225000 [83:29:08<335:33:39,  6.47s/it]
 17%|█▋        | 38422/225000 [83:29:13<314:25:04,  6.07s/it]
 17%|█▋        | 38423/225000 [83:29:17<290:01:33,  5.60s/it]
 17%|█▋        | 38424/225000 [83:29:30<403:00:28,  7.78s/it]
 17%|█▋        | 38425/225000 [83:29:34<343:25:36,  6.63s/it]
 17%|█▋        | 38426/225000 [83:29:42<367:43:19,  7.10s/it]
 17%|█▋        | 38427/225000 [83:29:48<341:16:24,  6.59s/it]
 17%|█▋        | 38428/225000 [83:29:53<314:21:58,  6.07s/it]
 17%|█▋        | 38429/225000 [83:29:58<301:16:28,  5.81s/it]
 17%|█▋        | 38430/225000 [83:30:03<285:29:05,  5.51s/it]
 17%|█▋        | 38431/225000 [83:30:16<402:32:39,  7.77s/it]
 17%|█▋        | 38432/225000 [83:30:20<351:47:23,  6.79s/it]
 17%|█▋        | 38433/225000 [83:30:30<394:28:00,  7.61s/it]
 17%|█▋        | 38434/225000 [83:30:35<359:13:14,  6.93s/it]
 17%|█▋        | 38435/225000 [83:30:41<337:53:03,  6.52s/it]
 17%|█▋        | 38436/225000 [83:30:47<329:05:22,  6.35s/it]
 17%|█▋        | 38437/225000 [83:30:52<315:37:08,  6.09s/it]
 17%|█▋        | 38438/225000 [83:30:57<302:27:39,  5.84s/it]
 17%|█▋        | 38439/225000 [83:31:04<313:17:56,  6.05s/it]
 17%|█▋        | 38440/225000 [83:31:18<432:45:51,  8.35s/it]
 17%|█▋        | 38441/225000 [83:31:24<405:23:09,  7.82s/it]
 17%|█▋        | 38442/225000 [83:31:29<363:07:38,  7.01s/it]
 17%|█▋        | 38443/225000 [83:31:35<337:30:57,  6.51s/it]
 17%|█▋        | 38444/225000 [83:31:40<316:46:21,  6.11s/it]
 17%|█▋        | 38445/225000 [83:31:45<302:53:39,  5.85s/it]
 17%|█▋        | 38446/225000 [83:31:57<391:40:29,  7.56s/it]
 17%|█▋        | 38447/225000 [83:32:03<367:24:43,  7.09s/it]
 17%|█▋        | 38448/225000 [83:32:12<406:15:08,  7.84s/it]
 17%|█▋        | 38449/225000 [83:32:17<351:54:57,  6.79s/it]
 17%|█▋        | 38450/225000 [83:32:21<315:22:35,  6.09s/it]
                                                             
{'loss': '0.6952', 'grad_norm': '0.749', 'learning_rate': '4.787e-05', 'epoch': '0.1709', 'num_input_tokens_seen': 141502552, 'train_runtime': '3.007e+05', 'train_tokens_per_second': '470.5'}

 17%|█▋        | 38450/225000 [83:32:21<315:22:35,  6.09s/it]
 17%|█▋        | 38451/225000 [83:32:28<336:09:50,  6.49s/it]
 17%|█▋        | 38452/225000 [83:32:38<382:33:23,  7.38s/it]
 17%|█▋        | 38453/225000 [83:32:43<349:17:12,  6.74s/it]
 17%|█▋        | 38454/225000 [83:32:52<375:33:11,  7.25s/it]
 17%|█▋        | 38455/225000 [83:32:57<344:27:45,  6.65s/it]
 17%|█▋        | 38456/225000 [83:33:02<320:29:59,  6.19s/it]
 17%|█▋        | 38457/225000 [83:33:07<300:50:08,  5.81s/it]
 17%|█▋        | 38458/225000 [83:33:12<291:30:17,  5.63s/it]
 17%|█▋        | 38459/225000 [83:33:18<298:08:49,  5.75s/it]
 17%|█▋        | 38460/225000 [83:33:30<400:56:49,  7.74s/it]
 17%|█▋        | 38461/225000 [83:33:34<341:50:12,  6.60s/it]
 17%|█▋        | 38462/225000 [83:33:41<348:52:00,  6.73s/it]
 17%|█▋        | 38463/225000 [83:33:50<383:07:18,  7.39s/it]
 17%|█▋        | 38464/225000 [83:33:55<337:25:00,  6.51s/it]
 17%|█▋        | 38465/225000 [83:34:00<322:39:51,  6.23s/it]
 17%|█▋        | 38466/225000 [83:34:06<312:54:26,  6.04s/it]
 17%|█▋        | 38467/225000 [83:34:11<304:08:22,  5.87s/it]
 17%|█▋        | 38468/225000 [83:34:18<307:58:34,  5.94s/it]
 17%|█▋        | 38469/225000 [83:34:23<303:31:12,  5.86s/it]
 17%|█▋        | 38470/225000 [83:34:29<303:59:44,  5.87s/it]
 17%|█▋        | 38471/225000 [83:34:35<298:55:54,  5.77s/it]
 17%|█▋        | 38472/225000 [83:34:47<402:14:27,  7.76s/it]
 17%|█▋        | 38473/225000 [83:34:56<413:59:18,  7.99s/it]
 17%|█▋        | 38474/225000 [83:35:01<369:40:40,  7.13s/it]
 17%|█▋        | 38475/225000 [83:35:09<383:40:24,  7.41s/it]
 17%|█▋        | 38476/225000 [83:35:13<339:39:57,  6.56s/it]
 17%|█▋        | 38477/225000 [83:35:18<315:33:38,  6.09s/it]
 17%|█▋        | 38478/225000 [83:35:23<298:04:25,  5.75s/it]
 17%|█▋        | 38479/225000 [83:35:28<288:10:32,  5.56s/it]
 17%|█▋        | 38480/225000 [83:35:34<281:54:09,  5.44s/it]
 17%|█▋        | 38481/225000 [83:35:43<344:32:09,  6.65s/it]
 17%|█▋        | 38482/225000 [83:35:53<392:18:29,  7.57s/it]
 17%|█▋        | 38483/225000 [83:35:56<330:50:52,  6.39s/it]
 17%|█▋        | 38484/225000 [83:36:05<368:48:52,  7.12s/it]
 17%|█▋        | 38485/225000 [83:36:11<341:06:00,  6.58s/it]
 17%|█▋        | 38486/225000 [83:36:15<309:13:21,  5.97s/it]
 17%|█▋        | 38487/225000 [83:36:20<290:01:18,  5.60s/it]
 17%|█▋        | 38488/225000 [83:36:25<278:53:44,  5.38s/it]
 17%|█▋        | 38489/225000 [83:36:33<326:55:44,  6.31s/it]
 17%|█▋        | 38490/225000 [83:36:43<381:47:27,  7.37s/it]
 17%|█▋        | 38491/225000 [83:36:48<339:51:57,  6.56s/it]
 17%|█▋        | 38492/225000 [83:36:53<320:30:04,  6.19s/it]
 17%|█▋        | 38493/225000 [83:36:58<299:50:21,  5.79s/it]
 17%|█▋        | 38494/225000 [83:37:06<335:39:52,  6.48s/it]
 17%|█▋        | 38495/225000 [83:37:16<389:54:12,  7.53s/it]
 17%|█▋        | 38496/225000 [83:37:20<333:09:20,  6.43s/it]
 17%|█▋        | 38497/225000 [83:37:24<301:53:14,  5.83s/it]
 17%|█▋        | 38498/225000 [83:37:32<339:15:42,  6.55s/it]
 17%|█▋        | 38499/225000 [83:37:41<371:37:04,  7.17s/it]
 17%|█▋        | 38500/225000 [83:37:45<326:39:37,  6.31s/it]
                                                             
{'loss': '0.7096', 'grad_norm': '0.7538', 'learning_rate': '4.787e-05', 'epoch': '0.1711', 'num_input_tokens_seen': 141648960, 'train_runtime': '3.011e+05', 'train_tokens_per_second': '470.5'}

 17%|█▋        | 38500/225000 [83:37:45<326:39:37,  6.31s/it]
 17%|█▋        | 38501/225000 [83:37:53<347:56:11,  6.72s/it]
 17%|█▋        | 38502/225000 [83:38:01<362:19:16,  6.99s/it]
 17%|█▋        | 38503/225000 [83:38:05<320:50:58,  6.19s/it]
 17%|█▋        | 38504/225000 [83:38:09<291:26:22,  5.63s/it]
 17%|█▋        | 38505/225000 [83:38:14<276:52:22,  5.34s/it]
 17%|█▋        | 38506/225000 [83:38:19<273:48:18,  5.29s/it]
 17%|█▋        | 38507/225000 [83:38:26<302:04:05,  5.83s/it]
 17%|█▋        | 38508/225000 [83:38:40<421:06:00,  8.13s/it]
 17%|█▋        | 38509/225000 [83:38:43<345:38:58,  6.67s/it]
 17%|█▋        | 38510/225000 [83:38:51<370:14:13,  7.15s/it]
 17%|█▋        | 38511/225000 [83:38:55<312:46:16,  6.04s/it]
 17%|█▋        | 38512/225000 [83:39:00<307:36:53,  5.94s/it]
 17%|█▋        | 38513/225000 [83:39:06<299:27:27,  5.78s/it]
 17%|█▋        | 38514/225000 [83:39:11<292:35:24,  5.65s/it]
 17%|█▋        | 38515/225000 [83:39:17<291:46:05,  5.63s/it]
 17%|█▋        | 38516/225000 [83:39:22<291:16:23,  5.62s/it]
 17%|█▋        | 38517/225000 [83:39:36<407:55:34,  7.87s/it]
 17%|█▋        | 38518/225000 [83:39:40<362:52:53,  7.01s/it]
 17%|█▋        | 38519/225000 [83:39:45<330:16:53,  6.38s/it]
 17%|█▋        | 38520/225000 [83:39:50<306:25:44,  5.92s/it]
 17%|█▋        | 38521/225000 [83:39:55<289:05:42,  5.58s/it]
 17%|█▋        | 38522/225000 [83:40:04<335:49:14,  6.48s/it]
 17%|█▋        | 38523/225000 [83:40:15<411:59:39,  7.95s/it]
 17%|█▋        | 38524/225000 [83:40:18<338:55:53,  6.54s/it]
 17%|█▋        | 38525/225000 [83:40:26<354:27:17,  6.84s/it]
 17%|█▋        | 38526/225000 [83:40:33<366:45:23,  7.08s/it]
 17%|█▋        | 38527/225000 [83:40:38<322:46:23,  6.23s/it]
 17%|█▋        | 38528/225000 [83:40:42<295:06:31,  5.70s/it]
 17%|█▋        | 38529/225000 [83:40:54<393:17:28,  7.59s/it]
 17%|█▋        | 38530/225000 [83:40:58<335:20:44,  6.47s/it]
 17%|█▋        | 38531/225000 [83:41:02<291:36:52,  5.63s/it]
 17%|█▋        | 38532/225000 [83:41:13<383:21:18,  7.40s/it]
 17%|█▋        | 38533/225000 [83:41:18<337:21:59,  6.51s/it]
 17%|█▋        | 38534/225000 [83:41:23<313:21:00,  6.05s/it]
 17%|█▋        | 38535/225000 [83:41:28<309:24:21,  5.97s/it]
 17%|█▋        | 38536/225000 [83:41:34<305:12:04,  5.89s/it]
 17%|█▋        | 38537/225000 [83:41:39<296:44:11,  5.73s/it]
 17%|█▋        | 38538/225000 [83:41:45<288:48:24,  5.58s/it]
 17%|█▋        | 38539/225000 [83:41:50<288:44:55,  5.57s/it]
 17%|█▋        | 38540/225000 [83:42:03<396:58:15,  7.66s/it]
 17%|█▋        | 38541/225000 [83:42:10<394:47:24,  7.62s/it]
 17%|█▋        | 38542/225000 [83:42:16<368:42:24,  7.12s/it]
 17%|█▋        | 38543/225000 [83:42:22<346:38:59,  6.69s/it]
 17%|█▋        | 38544/225000 [83:42:28<334:42:09,  6.46s/it]
 17%|█▋        | 38545/225000 [83:42:32<302:33:02,  5.84s/it]
 17%|█▋        | 38546/225000 [83:42:43<373:24:01,  7.21s/it]
 17%|█▋        | 38547/225000 [83:42:49<352:46:01,  6.81s/it]
 17%|█▋        | 38548/225000 [83:42:57<371:40:30,  7.18s/it]
 17%|█▋        | 38549/225000 [83:43:01<333:15:23,  6.43s/it]
 17%|█▋        | 38550/225000 [83:43:06<307:29:48,  5.94s/it]
                                                             
{'loss': '0.6931', 'grad_norm': '0.7192', 'learning_rate': '4.786e-05', 'epoch': '0.1713', 'num_input_tokens_seen': 141794040, 'train_runtime': '3.014e+05', 'train_tokens_per_second': '470.5'}

 17%|█▋        | 38550/225000 [83:43:06<307:29:48,  5.94s/it]
 17%|█▋        | 38551/225000 [83:43:11<287:45:14,  5.56s/it]
 17%|█▋        | 38552/225000 [83:43:16<281:56:33,  5.44s/it]
 17%|█▋        | 38553/225000 [83:43:22<291:23:48,  5.63s/it]
 17%|█▋        | 38554/225000 [83:43:35<408:09:50,  7.88s/it]
 17%|█▋        | 38555/225000 [83:43:41<383:51:32,  7.41s/it]
 17%|█▋        | 38556/225000 [83:43:47<356:42:43,  6.89s/it]
 17%|█▋        | 38557/225000 [83:43:52<331:34:38,  6.40s/it]
 17%|█▋        | 38558/225000 [83:44:03<393:41:24,  7.60s/it]
 17%|█▋        | 38559/225000 [83:44:08<350:35:55,  6.77s/it]
 17%|█▋        | 38560/225000 [83:44:13<324:28:08,  6.27s/it]
 17%|█▋        | 38561/225000 [83:44:18<313:33:49,  6.05s/it]
 17%|█▋        | 38562/225000 [83:44:23<295:50:04,  5.71s/it]
 17%|█▋        | 38563/225000 [83:44:29<291:46:33,  5.63s/it]
 17%|█▋        | 38564/225000 [83:44:34<282:56:39,  5.46s/it]
 17%|█▋        | 38565/225000 [83:44:43<348:40:41,  6.73s/it]
 17%|█▋        | 38566/225000 [83:44:55<425:37:28,  8.22s/it]
 17%|█▋        | 38567/225000 [83:44:59<359:10:42,  6.94s/it]
 17%|█▋        | 38568/225000 [83:45:05<343:31:10,  6.63s/it]
 17%|█▋        | 38569/225000 [83:45:10<315:39:26,  6.10s/it]
 17%|█▋        | 38570/225000 [83:45:15<299:08:58,  5.78s/it]
 17%|█▋        | 38571/225000 [83:45:22<325:25:35,  6.28s/it]
 17%|█▋        | 38572/225000 [83:45:32<385:59:22,  7.45s/it]
 17%|█▋        | 38573/225000 [83:45:36<328:40:55,  6.35s/it]
 17%|█▋        | 38574/225000 [83:45:46<378:27:41,  7.31s/it]
 17%|█▋        | 38575/225000 [83:45:50<333:09:05,  6.43s/it]
 17%|█▋        | 38576/225000 [83:45:57<335:09:03,  6.47s/it]
 17%|█▋        | 38577/225000 [83:46:05<365:46:27,  7.06s/it]
 17%|█▋        | 38578/225000 [83:46:12<361:21:12,  6.98s/it]
 17%|█▋        | 38579/225000 [83:46:18<348:31:47,  6.73s/it]
 17%|█▋        | 38580/225000 [83:46:24<341:19:22,  6.59s/it]
 17%|█▋        | 38581/225000 [83:46:30<321:01:17,  6.20s/it]
 17%|█▋        | 38582/225000 [83:46:36<320:37:10,  6.19s/it]
 17%|█▋        | 38583/225000 [83:46:45<365:29:42,  7.06s/it]
 17%|█▋        | 38584/225000 [83:46:51<349:46:54,  6.75s/it]
 17%|█▋        | 38585/225000 [83:46:56<322:31:31,  6.23s/it]
 17%|█▋        | 38586/225000 [83:47:01<304:07:59,  5.87s/it]
 17%|█▋        | 38587/225000 [83:47:06<288:45:54,  5.58s/it]
 17%|█▋        | 38588/225000 [83:47:10<270:38:17,  5.23s/it]
 17%|█▋        | 38589/225000 [83:47:23<388:37:51,  7.51s/it]
 17%|█▋        | 38590/225000 [83:47:29<360:27:36,  6.96s/it]
 17%|█▋        | 38591/225000 [83:47:35<343:12:55,  6.63s/it]
 17%|█▋        | 38592/225000 [83:47:43<364:33:16,  7.04s/it]
 17%|█▋        | 38593/225000 [83:47:50<365:30:21,  7.06s/it]
 17%|█▋        | 38594/225000 [83:47:55<332:55:04,  6.43s/it]
 17%|█▋        | 38595/225000 [83:47:59<304:12:11,  5.88s/it]
 17%|█▋        | 38596/225000 [83:48:05<294:14:30,  5.68s/it]
 17%|█▋        | 38597/225000 [83:48:10<293:34:07,  5.67s/it]
 17%|█▋        | 38598/225000 [83:48:23<411:40:08,  7.95s/it]
 17%|█▋        | 38599/225000 [83:48:30<384:52:13,  7.43s/it]
 17%|█▋        | 38600/225000 [83:48:34<330:05:27,  6.38s/it]
                                                             
{'loss': '0.6776', 'grad_norm': '0.8228', 'learning_rate': '4.785e-05', 'epoch': '0.1716', 'num_input_tokens_seen': 141941768, 'train_runtime': '3.017e+05', 'train_tokens_per_second': '470.5'}

 17%|█▋        | 38600/225000 [83:48:34<330:05:27,  6.38s/it]
 17%|█▋        | 38601/225000 [83:48:43<384:28:51,  7.43s/it]
 17%|█▋        | 38602/225000 [83:48:50<372:04:23,  7.19s/it]
 17%|█▋        | 38603/225000 [83:48:54<324:39:52,  6.27s/it]
 17%|█▋        | 38604/225000 [83:49:01<325:36:21,  6.29s/it]
 17%|█▋        | 38605/225000 [83:49:06<307:17:23,  5.93s/it]
 17%|█▋        | 38606/225000 [83:49:11<298:47:43,  5.77s/it]
 17%|█▋        | 38607/225000 [83:49:16<285:57:02,  5.52s/it]
 17%|█▋        | 38608/225000 [83:49:21<273:41:15,  5.29s/it]
 17%|█▋        | 38609/225000 [83:49:26<270:07:17,  5.22s/it]
 17%|█▋        | 38610/225000 [83:49:39<389:38:17,  7.53s/it]
 17%|█▋        | 38611/225000 [83:49:45<373:54:16,  7.22s/it]
 17%|█▋        | 38612/225000 [83:49:49<317:38:31,  6.14s/it]
 17%|█▋        | 38613/225000 [83:49:59<383:36:01,  7.41s/it]
 17%|█▋        | 38614/225000 [83:50:06<375:37:11,  7.26s/it]
 17%|█▋        | 38615/225000 [83:50:11<332:59:25,  6.43s/it]
 17%|█▋        | 38616/225000 [83:50:17<333:29:04,  6.44s/it]
 17%|█▋        | 38617/225000 [83:50:21<295:51:16,  5.71s/it]
 17%|█▋        | 38618/225000 [83:50:25<272:00:08,  5.25s/it]
 17%|█▋        | 38619/225000 [83:50:38<386:50:27,  7.47s/it]
 17%|█▋        | 38620/225000 [83:50:41<318:37:55,  6.15s/it]
 17%|█▋        | 38621/225000 [83:50:49<345:06:55,  6.67s/it]
 17%|█▋        | 38622/225000 [83:50:56<358:53:28,  6.93s/it]
 17%|█▋        | 38623/225000 [83:51:01<317:17:26,  6.13s/it]
 17%|█▋        | 38624/225000 [83:51:11<378:34:43,  7.31s/it]
 17%|█▋        | 38625/225000 [83:51:17<356:22:35,  6.88s/it]
 17%|█▋        | 38626/225000 [83:51:22<335:25:16,  6.48s/it]
 17%|█▋        | 38627/225000 [83:51:28<319:42:58,  6.18s/it]
 17%|█▋        | 38628/225000 [83:51:33<313:33:00,  6.06s/it]
 17%|█▋        | 38629/225000 [83:51:39<299:36:20,  5.79s/it]
 17%|█▋        | 38630/225000 [83:51:44<288:33:57,  5.57s/it]
 17%|█▋        | 38631/225000 [83:51:54<356:26:15,  6.89s/it]
 17%|█▋        | 38632/225000 [83:51:59<337:34:24,  6.52s/it]
 17%|█▋        | 38633/225000 [83:52:05<329:15:21,  6.36s/it]
 17%|█▋        | 38634/225000 [83:52:10<308:41:15,  5.96s/it]
 17%|█▋        | 38635/225000 [83:52:16<306:28:49,  5.92s/it]
 17%|█▋        | 38636/225000 [83:52:22<299:45:36,  5.79s/it]
 17%|█▋        | 38637/225000 [83:52:31<348:30:01,  6.73s/it]
 17%|█▋        | 38638/225000 [83:52:44<451:46:44,  8.73s/it]
 17%|█▋        | 38639/225000 [83:52:48<376:13:54,  7.27s/it]
 17%|█▋        | 38640/225000 [83:52:55<376:06:28,  7.27s/it]
 17%|█▋        | 38641/225000 [83:53:00<342:22:48,  6.61s/it]
 17%|█▋        | 38642/225000 [83:53:05<317:42:17,  6.14s/it]
 17%|█▋        | 38643/225000 [83:53:10<303:19:07,  5.86s/it]
 17%|█▋        | 38644/225000 [83:53:21<373:53:14,  7.22s/it]
 17%|█▋        | 38645/225000 [83:53:29<395:03:42,  7.63s/it]
 17%|█▋        | 38646/225000 [83:53:36<373:49:25,  7.22s/it]
 17%|█▋        | 38647/225000 [83:53:41<348:32:39,  6.73s/it]
 17%|█▋        | 38648/225000 [83:53:47<339:15:15,  6.55s/it]
 17%|█▋        | 38649/225000 [83:53:53<321:53:02,  6.22s/it]
 17%|█▋        | 38650/225000 [83:53:58<308:47:04,  5.97s/it]
                                                             
{'loss': '0.6966', 'grad_norm': '0.7273', 'learning_rate': '4.784e-05', 'epoch': '0.1718', 'num_input_tokens_seen': 142089600, 'train_runtime': '3.02e+05', 'train_tokens_per_second': '470.4'}

 17%|█▋        | 38650/225000 [83:53:58<308:47:04,  5.97s/it]
 17%|█▋        | 38651/225000 [83:54:03<298:13:45,  5.76s/it]
 17%|█▋        | 38652/225000 [83:54:09<291:49:45,  5.64s/it]
 17%|█▋        | 38653/225000 [83:54:20<379:37:07,  7.33s/it]
 17%|█▋        | 38654/225000 [83:54:27<380:34:12,  7.35s/it]
 17%|█▋        | 38655/225000 [83:54:32<339:35:40,  6.56s/it]
 17%|█▋        | 38656/225000 [83:54:37<318:39:45,  6.16s/it]
 17%|█▋        | 38657/225000 [83:54:43<306:18:37,  5.92s/it]
 17%|█▋        | 38658/225000 [83:54:51<337:22:34,  6.52s/it]
 17%|█▋        | 38659/225000 [83:55:03<421:14:08,  8.14s/it]
 17%|█▋        | 38660/225000 [83:55:06<355:31:29,  6.87s/it]
 17%|█▋        | 38661/225000 [83:55:12<335:06:05,  6.47s/it]
 17%|█▋        | 38662/225000 [83:55:21<371:44:03,  7.18s/it]
 17%|█▋        | 38663/225000 [83:55:25<330:29:49,  6.39s/it]
 17%|█▋        | 38664/225000 [83:55:31<313:03:29,  6.05s/it]
 17%|█▋        | 38665/225000 [83:55:36<300:28:34,  5.81s/it]
 17%|█▋        | 38666/225000 [83:55:49<412:27:52,  7.97s/it]
 17%|█▋        | 38667/225000 [83:55:53<355:11:06,  6.86s/it]
 17%|█▋        | 38668/225000 [83:55:58<326:11:52,  6.30s/it]
 17%|█▋        | 38669/225000 [83:56:03<310:38:15,  6.00s/it]
 17%|█▋        | 38670/225000 [83:56:16<413:06:05,  7.98s/it]
 17%|█▋        | 38671/225000 [83:56:20<354:26:29,  6.85s/it]
 17%|█▋        | 38672/225000 [83:56:28<373:13:14,  7.21s/it]
 17%|█▋        | 38673/225000 [83:56:35<362:07:26,  7.00s/it]
 17%|█▋        | 38674/225000 [83:56:39<317:24:33,  6.13s/it]
 17%|█▋        | 38675/225000 [83:56:45<316:57:31,  6.12s/it]
 17%|█▋        | 38676/225000 [83:56:55<382:57:45,  7.40s/it]
 17%|█▋        | 38677/225000 [83:57:01<353:15:02,  6.83s/it]
 17%|█▋        | 38678/225000 [83:57:08<360:54:01,  6.97s/it]
 17%|█▋        | 38679/225000 [83:57:16<374:28:14,  7.24s/it]
 17%|█▋        | 38680/225000 [83:57:21<336:00:26,  6.49s/it]
 17%|█▋        | 38681/225000 [83:57:29<361:00:50,  6.98s/it]
 17%|█▋        | 38682/225000 [83:57:32<307:28:22,  5.94s/it]
 17%|█▋        | 38683/225000 [83:57:42<358:14:00,  6.92s/it]
 17%|█▋        | 38684/225000 [83:57:45<307:57:16,  5.95s/it]
 17%|█▋        | 38685/225000 [83:57:54<353:03:49,  6.82s/it]
 17%|█▋        | 38686/225000 [83:57:58<308:58:47,  5.97s/it]
 17%|█▋        | 38687/225000 [83:58:05<324:17:28,  6.27s/it]
 17%|█▋        | 38688/225000 [83:58:11<316:14:04,  6.11s/it]
 17%|█▋        | 38689/225000 [83:58:16<307:50:57,  5.95s/it]
 17%|█▋        | 38690/225000 [83:58:24<331:43:52,  6.41s/it]
 17%|█▋        | 38691/225000 [83:58:34<393:35:00,  7.61s/it]
 17%|█▋        | 38692/225000 [83:58:39<341:46:01,  6.60s/it]
 17%|█▋        | 38693/225000 [83:58:43<311:03:48,  6.01s/it]
 17%|█▋        | 38694/225000 [83:58:55<393:45:40,  7.61s/it]
 17%|█▋        | 38695/225000 [83:58:58<331:31:47,  6.41s/it]
 17%|█▋        | 38696/225000 [83:59:06<353:10:34,  6.82s/it]
 17%|█▋        | 38697/225000 [83:59:10<307:42:24,  5.95s/it]
 17%|█▋        | 38698/225000 [83:59:22<398:33:15,  7.70s/it]
 17%|█▋        | 38699/225000 [83:59:26<343:49:40,  6.64s/it]
 17%|█▋        | 38700/225000 [83:59:30<307:07:36,  5.93s/it]
                                                             
{'loss': '0.7002', 'grad_norm': '0.6534', 'learning_rate': '4.784e-05', 'epoch': '0.172', 'num_input_tokens_seen': 142237064, 'train_runtime': '3.024e+05', 'train_tokens_per_second': '470.4'}

 17%|█▋        | 38700/225000 [83:59:30<307:07:36,  5.93s/it]
 17%|█▋        | 38701/225000 [83:59:41<383:13:52,  7.41s/it]
 17%|█▋        | 38702/225000 [83:59:47<363:31:08,  7.02s/it]
 17%|█▋        | 38703/225000 [83:59:52<326:09:57,  6.30s/it]
 17%|█▋        | 38704/225000 [83:59:56<294:42:41,  5.70s/it]
 17%|█▋        | 38705/225000 [84:00:01<277:24:44,  5.36s/it]
 17%|█▋        | 38706/225000 [84:00:11<351:17:27,  6.79s/it]
 17%|█▋        | 38707/225000 [84:00:20<394:30:06,  7.62s/it]
 17%|█▋        | 38708/225000 [84:00:24<330:52:04,  6.39s/it]
 17%|█▋        | 38709/225000 [84:00:34<383:07:29,  7.40s/it]
 17%|█▋        | 38710/225000 [84:00:38<332:36:35,  6.43s/it]
 17%|█▋        | 38711/225000 [84:00:42<302:30:22,  5.85s/it]
 17%|█▋        | 38712/225000 [84:00:51<352:16:54,  6.81s/it]
 17%|█▋        | 38713/225000 [84:00:58<355:36:58,  6.87s/it]
 17%|█▋        | 38714/225000 [84:01:04<334:35:16,  6.47s/it]
 17%|█▋        | 38715/225000 [84:01:12<361:23:18,  6.98s/it]
 17%|█▋        | 38716/225000 [84:01:18<350:27:41,  6.77s/it]
 17%|█▋        | 38717/225000 [84:01:25<350:03:22,  6.76s/it]
 17%|█▋        | 38718/225000 [84:01:31<338:10:54,  6.54s/it]
 17%|█▋        | 38719/225000 [84:01:41<387:08:52,  7.48s/it]
 17%|█▋        | 38720/225000 [84:01:45<330:30:34,  6.39s/it]
 17%|█▋        | 38721/225000 [84:01:53<364:17:15,  7.04s/it]
 17%|█▋        | 38722/225000 [84:01:58<330:12:44,  6.38s/it]
 17%|█▋        | 38723/225000 [84:02:02<297:39:30,  5.75s/it]
 17%|█▋        | 38724/225000 [84:02:13<367:41:19,  7.11s/it]
 17%|█▋        | 38725/225000 [84:02:17<323:13:01,  6.25s/it]
 17%|█▋        | 38726/225000 [84:02:21<285:16:49,  5.51s/it]
 17%|█▋        | 38727/225000 [84:02:33<387:51:37,  7.50s/it]
 17%|█▋        | 38728/225000 [84:02:38<354:56:17,  6.86s/it]
 17%|█▋        | 38729/225000 [84:02:46<368:38:34,  7.12s/it]
 17%|█▋        | 38730/225000 [84:02:50<325:39:29,  6.29s/it]
 17%|█▋        | 38731/225000 [84:02:55<297:00:25,  5.74s/it]
 17%|█▋        | 38732/225000 [84:03:04<347:28:51,  6.72s/it]
 17%|█▋        | 38733/225000 [84:03:13<388:08:11,  7.50s/it]
 17%|█▋        | 38734/225000 [84:03:20<373:53:53,  7.23s/it]
 17%|█▋        | 38735/225000 [84:03:24<335:49:39,  6.49s/it]
 17%|█▋        | 38736/225000 [84:03:30<317:18:07,  6.13s/it]
 17%|█▋        | 38737/225000 [84:03:35<301:43:59,  5.83s/it]
 17%|█▋        | 38738/225000 [84:03:40<295:42:19,  5.72s/it]
 17%|█▋        | 38739/225000 [84:03:45<288:39:37,  5.58s/it]
 17%|█▋        | 38740/225000 [84:03:51<293:43:52,  5.68s/it]
 17%|█▋        | 38741/225000 [84:03:59<327:13:46,  6.32s/it]
 17%|█▋        | 38742/225000 [84:04:12<427:14:03,  8.26s/it]
 17%|█▋        | 38743/225000 [84:04:15<353:51:47,  6.84s/it]
 17%|█▋        | 38744/225000 [84:04:24<373:21:09,  7.22s/it]
 17%|█▋        | 38745/225000 [84:04:28<326:35:50,  6.31s/it]
 17%|█▋        | 38746/225000 [84:04:35<339:27:59,  6.56s/it]
 17%|█▋        | 38747/225000 [84:04:43<362:25:56,  7.01s/it]
 17%|█▋        | 38748/225000 [84:04:47<311:38:20,  6.02s/it]
 17%|█▋        | 38749/225000 [84:04:56<357:57:29,  6.92s/it]
 17%|█▋        | 38750/225000 [84:05:01<326:50:51,  6.32s/it]
                                                             
{'loss': '0.682', 'grad_norm': '0.6901', 'learning_rate': '4.783e-05', 'epoch': '0.1722', 'num_input_tokens_seen': 142383952, 'train_runtime': '3.027e+05', 'train_tokens_per_second': '470.4'}

 17%|█▋        | 38750/225000 [84:05:01<326:50:51,  6.32s/it]
 17%|█▋        | 38751/225000 [84:05:05<300:16:09,  5.80s/it]
 17%|█▋        | 38752/225000 [84:05:11<295:03:18,  5.70s/it]
 17%|█▋        | 38753/225000 [84:05:23<391:45:05,  7.57s/it]
 17%|█▋        | 38754/225000 [84:05:29<370:36:13,  7.16s/it]
 17%|█▋        | 38755/225000 [84:05:34<344:08:09,  6.65s/it]
 17%|█▋        | 38756/225000 [84:05:39<312:56:59,  6.05s/it]
 17%|█▋        | 38757/225000 [84:05:49<383:17:26,  7.41s/it]
 17%|█▋        | 38758/225000 [84:05:57<387:45:01,  7.50s/it]
 17%|█▋        | 38759/225000 [84:06:04<379:00:54,  7.33s/it]
 17%|█▋        | 38760/225000 [84:06:11<372:07:53,  7.19s/it]
 17%|█▋        | 38761/225000 [84:06:15<327:53:27,  6.34s/it]
 17%|█▋        | 38762/225000 [84:06:20<304:18:17,  5.88s/it]
 17%|█▋        | 38763/225000 [84:06:25<285:28:57,  5.52s/it]
 17%|█▋        | 38764/225000 [84:06:30<280:49:33,  5.43s/it]
 17%|█▋        | 38765/225000 [84:06:36<291:26:32,  5.63s/it]
 17%|█▋        | 38766/225000 [84:06:49<409:04:33,  7.91s/it]
 17%|█▋        | 38767/225000 [84:06:54<359:52:40,  6.96s/it]
 17%|█▋        | 38768/225000 [84:06:59<327:57:45,  6.34s/it]
 17%|█▋        | 38769/225000 [84:07:04<307:32:46,  5.95s/it]
 17%|█▋        | 38770/225000 [84:07:12<335:57:58,  6.49s/it]
 17%|█▋        | 38771/225000 [84:07:23<403:09:12,  7.79s/it]
 17%|█▋        | 38772/225000 [84:07:30<400:09:23,  7.74s/it]
 17%|█▋        | 38773/225000 [84:07:34<342:39:06,  6.62s/it]
 17%|█▋        | 38774/225000 [84:07:40<336:06:08,  6.50s/it]
 17%|█▋        | 38775/225000 [84:07:48<359:28:50,  6.95s/it]
 17%|█▋        | 38776/225000 [84:07:53<317:46:11,  6.14s/it]
 17%|█▋        | 38777/225000 [84:08:00<328:49:09,  6.36s/it]
 17%|█▋        | 38778/225000 [84:08:10<387:20:08,  7.49s/it]
 17%|█▋        | 38779/225000 [84:08:14<341:32:12,  6.60s/it]
 17%|█▋        | 38780/225000 [84:08:23<369:45:55,  7.15s/it]
 17%|█▋        | 38781/225000 [84:08:29<357:08:36,  6.90s/it]
 17%|█▋        | 38782/225000 [84:08:33<313:00:36,  6.05s/it]
 17%|█▋        | 38783/225000 [84:08:41<347:07:45,  6.71s/it]
 17%|█▋        | 38784/225000 [84:08:50<380:07:53,  7.35s/it]
 17%|█▋        | 38785/225000 [84:08:56<353:35:22,  6.84s/it]
 17%|█▋        | 38786/225000 [84:09:03<357:05:10,  6.90s/it]
 17%|█▋        | 38787/225000 [84:09:08<328:30:30,  6.35s/it]
 17%|█▋        | 38788/225000 [84:09:12<295:31:58,  5.71s/it]
 17%|█▋        | 38789/225000 [84:09:22<360:30:54,  6.97s/it]
 17%|█▋        | 38790/225000 [84:09:29<357:12:41,  6.91s/it]
 17%|█▋        | 38791/225000 [84:09:37<383:14:37,  7.41s/it]
 17%|█▋        | 38792/225000 [84:09:42<336:35:19,  6.51s/it]
 17%|█▋        | 38793/225000 [84:09:46<308:10:42,  5.96s/it]
 17%|█▋        | 38794/225000 [84:09:57<379:02:39,  7.33s/it]
 17%|█▋        | 38795/225000 [84:10:02<335:59:00,  6.50s/it]
 17%|█▋        | 38796/225000 [84:10:10<362:10:50,  7.00s/it]
 17%|█▋        | 38797/225000 [84:10:17<362:19:42,  7.01s/it]
 17%|█▋        | 38798/225000 [84:10:21<317:06:34,  6.13s/it]
 17%|█▋        | 38799/225000 [84:10:25<287:56:09,  5.57s/it]
 17%|█▋        | 38800/225000 [84:10:37<391:30:34,  7.57s/it]
                                                             
{'loss': '0.702', 'grad_norm': '0.7249', 'learning_rate': '4.782e-05', 'epoch': '0.1724', 'num_input_tokens_seen': 142533256, 'train_runtime': '3.03e+05', 'train_tokens_per_second': '470.3'}

 17%|█▋        | 38800/225000 [84:10:37<391:30:34,  7.57s/it]
 17%|█▋        | 38801/225000 [84:10:42<351:25:16,  6.79s/it]
 17%|█▋        | 38802/225000 [84:10:47<322:56:05,  6.24s/it]
 17%|█▋        | 38803/225000 [84:10:52<293:37:16,  5.68s/it]
 17%|█▋        | 38804/225000 [84:11:04<397:11:08,  7.68s/it]
 17%|█▋        | 38805/225000 [84:11:08<333:59:04,  6.46s/it]
 17%|█▋        | 38806/225000 [84:11:12<308:55:54,  5.97s/it]
 17%|█▋        | 38807/225000 [84:11:21<351:36:08,  6.80s/it]
 17%|█▋        | 38808/225000 [84:11:28<355:16:41,  6.87s/it]
 17%|█▋        | 38809/225000 [84:11:33<326:34:45,  6.31s/it]
 17%|█▋        | 38810/225000 [84:11:38<306:42:11,  5.93s/it]
 17%|█▋        | 38811/225000 [84:11:43<293:02:21,  5.67s/it]
 17%|█▋        | 38812/225000 [84:11:48<274:54:47,  5.32s/it]
 17%|█▋        | 38813/225000 [84:12:01<404:24:16,  7.82s/it]
 17%|█▋        | 38814/225000 [84:12:08<379:44:31,  7.34s/it]
 17%|█▋        | 38815/225000 [84:12:11<323:58:32,  6.26s/it]
 17%|█▋        | 38816/225000 [84:12:21<375:10:40,  7.25s/it]
 17%|█▋        | 38817/225000 [84:12:26<333:48:38,  6.45s/it]
 17%|█▋        | 38818/225000 [84:12:30<304:09:36,  5.88s/it]
 17%|█▋        | 38819/225000 [84:12:35<288:17:30,  5.57s/it]
 17%|█▋        | 38820/225000 [84:12:45<354:05:46,  6.85s/it]
 17%|█▋        | 38821/225000 [84:12:52<360:17:08,  6.97s/it]
 17%|█▋        | 38822/225000 [84:12:56<314:35:43,  6.08s/it]
 17%|█▋        | 38823/225000 [84:13:06<371:57:27,  7.19s/it]
 17%|█▋        | 38824/225000 [84:13:13<376:03:48,  7.27s/it]
 17%|█▋        | 38825/225000 [84:13:19<345:11:59,  6.68s/it]
 17%|█▋        | 38826/225000 [84:13:25<346:47:46,  6.71s/it]
 17%|█▋        | 38827/225000 [84:13:33<368:55:33,  7.13s/it]
 17%|█▋        | 38828/225000 [84:13:38<332:05:40,  6.42s/it]
 17%|█▋        | 38829/225000 [84:13:43<303:40:45,  5.87s/it]
 17%|█▋        | 38830/225000 [84:13:48<298:28:27,  5.77s/it]
 17%|█▋        | 38831/225000 [84:14:01<400:35:38,  7.75s/it]
 17%|█▋        | 38832/225000 [84:14:05<342:15:09,  6.62s/it]
 17%|█▋        | 38833/225000 [84:14:15<398:26:32,  7.70s/it]
 17%|█▋        | 38834/225000 [84:14:19<345:18:52,  6.68s/it]
 17%|█▋        | 38835/225000 [84:14:23<306:17:41,  5.92s/it]
 17%|█▋        | 38836/225000 [84:14:28<292:45:01,  5.66s/it]
 17%|█▋        | 38837/225000 [84:14:36<321:41:59,  6.22s/it]
 17%|█▋        | 38838/225000 [84:14:47<393:00:10,  7.60s/it]
 17%|█▋        | 38839/225000 [84:14:52<352:01:34,  6.81s/it]
 17%|█▋        | 38840/225000 [84:14:57<329:29:34,  6.37s/it]
 17%|█▋        | 38841/225000 [84:15:02<307:55:31,  5.95s/it]
 17%|█▋        | 38842/225000 [84:15:07<296:01:09,  5.72s/it]
 17%|█▋        | 38843/225000 [84:15:12<282:34:58,  5.46s/it]
 17%|█▋        | 38844/225000 [84:15:23<362:54:55,  7.02s/it]
 17%|█▋        | 38845/225000 [84:15:33<414:43:15,  8.02s/it]
 17%|█▋        | 38846/225000 [84:15:37<354:10:58,  6.85s/it]
 17%|█▋        | 38847/225000 [84:15:41<311:11:35,  6.02s/it]
 17%|█▋        | 38848/225000 [84:15:47<305:42:45,  5.91s/it]
 17%|█▋        | 38849/225000 [84:16:01<424:37:41,  8.21s/it]
 17%|█▋        | 38850/225000 [84:16:05<363:51:32,  7.04s/it]
                                                             
{'loss': '0.6911', 'grad_norm': '0.5935', 'learning_rate': '4.781e-05', 'epoch': '0.1727', 'num_input_tokens_seen': 142681632, 'train_runtime': '3.034e+05', 'train_tokens_per_second': '470.3'}

 17%|█▋        | 38850/225000 [84:16:05<363:51:32,  7.04s/it]
 17%|█▋        | 38851/225000 [84:16:12<366:08:22,  7.08s/it]
 17%|█▋        | 38852/225000 [84:16:20<377:38:07,  7.30s/it]
 17%|█▋        | 38853/225000 [84:16:24<327:31:44,  6.33s/it]
 17%|█▋        | 38854/225000 [84:16:29<306:58:03,  5.94s/it]
 17%|█▋        | 38855/225000 [84:16:34<285:28:50,  5.52s/it]
 17%|█▋        | 38856/225000 [84:16:45<373:47:47,  7.23s/it]
 17%|█▋        | 38857/225000 [84:16:49<325:33:30,  6.30s/it]
 17%|█▋        | 38858/225000 [84:16:54<311:30:30,  6.02s/it]
 17%|█▋        | 38859/225000 [84:17:01<325:39:03,  6.30s/it]
 17%|█▋        | 38860/225000 [84:17:07<313:56:32,  6.07s/it]
 17%|█▋        | 38861/225000 [84:17:12<309:15:31,  5.98s/it]
 17%|█▋        | 38862/225000 [84:17:18<303:48:15,  5.88s/it]
 17%|█▋        | 38863/225000 [84:17:24<299:24:18,  5.79s/it]
 17%|█▋        | 38864/225000 [84:17:29<293:28:02,  5.68s/it]
 17%|█▋        | 38865/225000 [84:17:43<416:24:43,  8.05s/it]
 17%|█▋        | 38866/225000 [84:17:48<378:55:56,  7.33s/it]
 17%|█▋        | 38867/225000 [84:17:52<327:42:38,  6.34s/it]
 17%|█▋        | 38868/225000 [84:17:57<301:15:28,  5.83s/it]
 17%|█▋        | 38869/225000 [84:18:02<294:24:19,  5.69s/it]
 17%|█▋        | 38870/225000 [84:18:12<357:06:14,  6.91s/it]
 17%|█▋        | 38871/225000 [84:18:21<389:41:53,  7.54s/it]
 17%|█▋        | 38872/225000 [84:18:28<377:30:00,  7.30s/it]
 17%|█▋        | 38873/225000 [84:18:32<330:39:04,  6.40s/it]
 17%|█▋        | 38874/225000 [84:18:41<374:35:41,  7.25s/it]
 17%|█▋        | 38875/225000 [84:18:46<325:47:47,  6.30s/it]
 17%|█▋        | 38876/225000 [84:18:54<359:43:34,  6.96s/it]
 17%|█▋        | 38877/225000 [84:18:59<331:15:34,  6.41s/it]
 17%|█▋        | 38878/225000 [84:19:05<317:26:08,  6.14s/it]
 17%|█▋        | 38879/225000 [84:19:10<303:31:09,  5.87s/it]
 17%|█▋        | 38880/225000 [84:19:15<293:04:57,  5.67s/it]
 17%|█▋        | 38881/225000 [84:19:20<284:24:46,  5.50s/it]
 17%|█▋        | 38882/225000 [84:19:26<282:58:54,  5.47s/it]
 17%|█▋        | 38883/225000 [84:19:37<377:09:32,  7.30s/it]
 17%|█▋        | 38884/225000 [84:19:46<398:24:23,  7.71s/it]
 17%|█▋        | 38885/225000 [84:19:51<355:15:17,  6.87s/it]
 17%|█▋        | 38886/225000 [84:19:58<368:01:05,  7.12s/it]
 17%|█▋        | 38887/225000 [84:20:06<380:20:58,  7.36s/it]
 17%|█▋        | 38888/225000 [84:20:11<338:34:48,  6.55s/it]
 17%|█▋        | 38889/225000 [84:20:16<314:05:46,  6.08s/it]
 17%|█▋        | 38890/225000 [84:20:21<293:52:12,  5.68s/it]
 17%|█▋        | 38891/225000 [84:20:25<278:47:41,  5.39s/it]
 17%|█▋        | 38892/225000 [84:20:36<351:12:09,  6.79s/it]
 17%|█▋        | 38893/225000 [84:20:45<398:48:46,  7.71s/it]
 17%|█▋        | 38894/225000 [84:20:49<341:06:35,  6.60s/it]
 17%|█▋        | 38895/225000 [84:20:53<302:11:50,  5.85s/it]
 17%|█▋        | 38896/225000 [84:21:05<389:08:57,  7.53s/it]
 17%|█▋        | 38897/225000 [84:21:09<340:37:16,  6.59s/it]
 17%|█▋        | 38898/225000 [84:21:19<384:51:38,  7.44s/it]
 17%|█▋        | 38899/225000 [84:21:23<329:55:18,  6.38s/it]
 17%|█▋        | 38900/225000 [84:21:28<312:19:01,  6.04s/it]
                                                             
{'loss': '0.7081', 'grad_norm': '0.7225', 'learning_rate': '4.781e-05', 'epoch': '0.1729', 'num_input_tokens_seen': 142828768, 'train_runtime': '3.037e+05', 'train_tokens_per_second': '470.3'}

 17%|█▋        | 38900/225000 [84:21:28<312:19:01,  6.04s/it]
 17%|█▋        | 38901/225000 [84:21:37<364:22:58,  7.05s/it]
 17%|█▋        | 38902/225000 [84:21:41<316:33:57,  6.12s/it]
 17%|█▋        | 38903/225000 [84:21:46<300:57:58,  5.82s/it]
 17%|█▋        | 38904/225000 [84:21:52<290:35:00,  5.62s/it]
 17%|█▋        | 38905/225000 [84:22:02<366:08:34,  7.08s/it]
 17%|█▋        | 38906/225000 [84:22:09<363:27:24,  7.03s/it]
 17%|█▋        | 38907/225000 [84:22:16<362:53:33,  7.02s/it]
 17%|█▋        | 38908/225000 [84:22:23<362:06:59,  7.01s/it]
 17%|█▋        | 38909/225000 [84:22:28<329:10:16,  6.37s/it]
 17%|█▋        | 38910/225000 [84:22:33<305:04:28,  5.90s/it]
 17%|█▋        | 38911/225000 [84:22:38<290:22:35,  5.62s/it]
 17%|█▋        | 38912/225000 [84:22:43<286:30:37,  5.54s/it]
 17%|█▋        | 38913/225000 [84:22:49<289:15:17,  5.60s/it]
 17%|█▋        | 38914/225000 [84:22:57<331:06:20,  6.41s/it]
 17%|█▋        | 38915/225000 [84:23:09<416:29:23,  8.06s/it]
 17%|█▋        | 38916/225000 [84:23:13<360:43:48,  6.98s/it]
 17%|█▋        | 38917/225000 [84:23:20<354:36:44,  6.86s/it]
 17%|█▋        | 38918/225000 [84:23:25<321:19:58,  6.22s/it]
 17%|█▋        | 38919/225000 [84:23:35<387:57:18,  7.51s/it]
 17%|█▋        | 38920/225000 [84:23:40<343:01:46,  6.64s/it]
 17%|█▋        | 38921/225000 [84:23:49<378:46:53,  7.33s/it]
 17%|█▋        | 38922/225000 [84:23:53<328:16:46,  6.35s/it]
 17%|█▋        | 38923/225000 [84:23:57<299:40:30,  5.80s/it]
 17%|█▋        | 38924/225000 [84:24:06<344:30:23,  6.67s/it]
 17%|█▋        | 38925/225000 [84:24:15<377:39:44,  7.31s/it]
 17%|█▋        | 38926/225000 [84:24:19<328:09:56,  6.35s/it]
 17%|█▋        | 38927/225000 [84:24:23<290:00:06,  5.61s/it]
 17%|█▋        | 38928/225000 [84:24:34<383:03:18,  7.41s/it]
 17%|█▋        | 38929/225000 [84:24:40<354:28:23,  6.86s/it]
 17%|█▋        | 38930/225000 [84:24:45<324:57:35,  6.29s/it]
 17%|█▋        | 38931/225000 [84:24:50<304:46:44,  5.90s/it]
 17%|█▋        | 38932/225000 [84:24:55<290:55:04,  5.63s/it]
 17%|█▋        | 38933/225000 [84:25:00<283:15:37,  5.48s/it]
 17%|█▋        | 38934/225000 [84:25:05<278:38:30,  5.39s/it]
 17%|█▋        | 38935/225000 [84:25:19<401:36:59,  7.77s/it]
 17%|█▋        | 38936/225000 [84:25:24<360:04:20,  6.97s/it]
 17%|█▋        | 38937/225000 [84:25:30<346:18:46,  6.70s/it]
 17%|█▋        | 38938/225000 [84:25:38<369:26:10,  7.15s/it]
 17%|█▋        | 38939/225000 [84:25:43<344:19:16,  6.66s/it]
 17%|█▋        | 38940/225000 [84:25:49<327:49:27,  6.34s/it]
 17%|█▋        | 38941/225000 [84:25:54<312:07:53,  6.04s/it]
 17%|█▋        | 38942/225000 [84:25:59<298:17:45,  5.77s/it]
 17%|█▋        | 38943/225000 [84:26:10<375:42:09,  7.27s/it]
 17%|█▋        | 38944/225000 [84:26:20<408:11:52,  7.90s/it]
 17%|█▋        | 38945/225000 [84:26:24<349:34:56,  6.76s/it]
 17%|█▋        | 38946/225000 [84:26:33<392:53:10,  7.60s/it]
 17%|█▋        | 38947/225000 [84:26:37<330:29:14,  6.39s/it]
 17%|█▋        | 38948/225000 [84:26:46<376:18:02,  7.28s/it]
 17%|█▋        | 38949/225000 [84:26:51<331:05:43,  6.41s/it]
 17%|█▋        | 38950/225000 [84:26:59<368:14:24,  7.13s/it]
                                                             
{'loss': '0.7019', 'grad_norm': '0.7779', 'learning_rate': '4.78e-05', 'epoch': '0.1731', 'num_input_tokens_seen': 142976568, 'train_runtime': '3.04e+05', 'train_tokens_per_second': '470.3'}

 17%|█▋        | 38950/225000 [84:26:59<368:14:24,  7.13s/it]
 17%|█▋        | 38951/225000 [84:27:04<323:10:44,  6.25s/it]
 17%|█▋        | 38952/225000 [84:27:08<297:50:00,  5.76s/it]
 17%|█▋        | 38953/225000 [84:27:13<283:17:11,  5.48s/it]
 17%|█▋        | 38954/225000 [84:27:26<395:55:32,  7.66s/it]
 17%|█▋        | 38955/225000 [84:27:29<333:10:00,  6.45s/it]
 17%|█▋        | 38956/225000 [84:27:38<373:57:03,  7.24s/it]
 17%|█▋        | 38957/225000 [84:27:43<325:37:00,  6.30s/it]
 17%|█▋        | 38958/225000 [84:27:49<322:37:02,  6.24s/it]
 17%|█▋        | 38959/225000 [84:27:57<348:29:48,  6.74s/it]
 17%|█▋        | 38960/225000 [84:28:01<304:23:55,  5.89s/it]
 17%|█▋        | 38961/225000 [84:28:12<385:18:35,  7.46s/it]
 17%|█▋        | 38962/225000 [84:28:16<334:11:46,  6.47s/it]
 17%|█▋        | 38963/225000 [84:28:20<299:41:45,  5.80s/it]
 17%|█▋        | 38964/225000 [84:28:31<384:36:16,  7.44s/it]
 17%|█▋        | 38965/225000 [84:28:35<331:07:20,  6.41s/it]
 17%|█▋        | 38966/225000 [84:28:45<382:41:36,  7.41s/it]
 17%|█▋        | 38967/225000 [84:28:49<330:13:35,  6.39s/it]
 17%|█▋        | 38968/225000 [84:28:58<367:05:48,  7.10s/it]
 17%|█▋        | 38969/225000 [84:29:03<335:17:03,  6.49s/it]
 17%|█▋        | 38970/225000 [84:29:09<322:35:52,  6.24s/it]
 17%|█▋        | 38971/225000 [84:29:14<309:02:15,  5.98s/it]
 17%|█▋        | 38972/225000 [84:29:19<292:49:31,  5.67s/it]
 17%|█▋        | 38973/225000 [84:29:24<285:30:00,  5.53s/it]
 17%|█▋        | 38974/225000 [84:29:29<282:53:28,  5.47s/it]
 17%|█▋        | 38975/225000 [84:29:34<274:46:34,  5.32s/it]
 17%|█▋        | 38976/225000 [84:29:48<406:38:35,  7.87s/it]
 17%|█▋        | 38977/225000 [84:29:52<347:59:27,  6.73s/it]
 17%|█▋        | 38978/225000 [84:30:02<394:08:45,  7.63s/it]
 17%|█▋        | 38979/225000 [84:30:05<330:20:00,  6.39s/it]
 17%|█▋        | 38980/225000 [84:30:13<347:35:13,  6.73s/it]
 17%|█▋        | 38981/225000 [84:30:17<305:55:38,  5.92s/it]
 17%|█▋        | 38982/225000 [84:30:26<350:58:06,  6.79s/it]
 17%|█▋        | 38983/225000 [84:30:30<312:00:23,  6.04s/it]
 17%|█▋        | 38984/225000 [84:30:40<364:47:41,  7.06s/it]
 17%|█▋        | 38985/225000 [84:30:45<345:30:51,  6.69s/it]
 17%|█▋        | 38986/225000 [84:30:55<383:14:11,  7.42s/it]
 17%|█▋        | 38987/225000 [84:30:59<331:49:11,  6.42s/it]
 17%|█▋        | 38988/225000 [84:31:06<341:21:58,  6.61s/it]
 17%|█▋        | 38989/225000 [84:31:13<345:41:40,  6.69s/it]
 17%|█▋        | 38990/225000 [84:31:17<314:21:51,  6.08s/it]
 17%|█▋        | 38991/225000 [84:31:22<289:34:48,  5.60s/it]
 17%|█▋        | 38992/225000 [84:31:27<281:14:41,  5.44s/it]
 17%|█▋        | 38993/225000 [84:31:37<360:01:40,  6.97s/it]
 17%|█▋        | 38994/225000 [84:31:44<361:51:35,  7.00s/it]
 17%|█▋        | 38995/225000 [84:31:51<359:17:23,  6.95s/it]
 17%|█▋        | 38996/225000 [84:31:55<315:04:07,  6.10s/it]
 17%|█▋        | 38997/225000 [84:32:00<297:34:50,  5.76s/it]
 17%|█▋        | 38998/225000 [84:32:11<372:32:45,  7.21s/it]
 17%|█▋        | 38999/225000 [84:32:16<341:25:20,  6.61s/it]
 17%|█▋        | 39000/225000 [84:32:22<334:03:50,  6.47s/it]
                                                             
{'loss': '0.712', 'grad_norm': '0.7661', 'learning_rate': '4.779e-05', 'epoch': '0.1733', 'num_input_tokens_seen': 143124016, 'train_runtime': '3.043e+05', 'train_tokens_per_second': '470.3'}

 17%|█▋        | 39000/225000 [84:32:22<334:03:50,  6.47s/it]
 17%|█▋        | 39001/225000 [84:32:32<381:58:13,  7.39s/it]
 17%|█▋        | 39002/225000 [84:32:38<356:52:47,  6.91s/it]
 17%|█▋        | 39003/225000 [84:32:41<310:55:14,  6.02s/it]
 17%|█▋        | 39004/225000 [84:32:52<379:53:06,  7.35s/it]
 17%|█▋        | 39005/225000 [84:32:55<317:12:50,  6.14s/it]
 17%|█▋        | 39006/225000 [84:33:03<344:18:36,  6.66s/it]
 17%|█▋        | 39007/225000 [84:33:07<307:20:52,  5.95s/it]
 17%|█▋        | 39008/225000 [84:33:17<360:04:37,  6.97s/it]
 17%|█▋        | 39009/225000 [84:33:22<338:04:37,  6.54s/it]
 17%|█▋        | 39010/225000 [84:33:28<324:38:51,  6.28s/it]
 17%|█▋        | 39011/225000 [84:33:34<312:53:17,  6.06s/it]
 17%|█▋        | 39012/225000 [84:33:41<330:29:14,  6.40s/it]
 17%|█▋        | 39013/225000 [84:33:46<315:43:07,  6.11s/it]
 17%|█▋        | 39014/225000 [84:33:52<306:42:53,  5.94s/it]
 17%|█▋        | 39015/225000 [84:33:58<305:55:02,  5.92s/it]
 17%|█▋        | 39016/225000 [84:34:04<305:59:41,  5.92s/it]
 17%|█▋        | 39017/225000 [84:34:09<294:20:12,  5.70s/it]
 17%|█▋        | 39018/225000 [84:34:14<288:12:44,  5.58s/it]
 17%|█▋        | 39019/225000 [84:34:22<320:43:14,  6.21s/it]
 17%|█▋        | 39020/225000 [84:34:35<432:07:24,  8.36s/it]
 17%|█▋        | 39021/225000 [84:34:39<366:33:07,  7.10s/it]
 17%|█▋        | 39022/225000 [84:34:47<378:45:25,  7.33s/it]
 17%|█▋        | 39023/225000 [84:34:51<330:52:19,  6.40s/it]
 17%|█▋        | 39024/225000 [84:34:57<315:42:14,  6.11s/it]
 17%|█▋        | 39025/225000 [84:35:02<302:07:24,  5.85s/it]
 17%|█▋        | 39026/225000 [84:35:08<299:26:23,  5.80s/it]
 17%|█▋        | 39027/225000 [84:35:13<292:22:55,  5.66s/it]
 17%|█▋        | 39028/225000 [84:35:22<347:46:42,  6.73s/it]
 17%|█▋        | 39029/225000 [84:35:34<421:10:03,  8.15s/it]
 17%|█▋        | 39030/225000 [84:35:38<365:14:18,  7.07s/it]
 17%|█▋        | 39031/225000 [84:35:48<399:46:47,  7.74s/it]
 17%|█▋        | 39032/225000 [84:35:53<360:21:53,  6.98s/it]
 17%|█▋        | 39033/225000 [84:35:58<326:10:50,  6.31s/it]
 17%|█▋        | 39034/225000 [84:36:04<326:51:38,  6.33s/it]
 17%|█▋        | 39035/225000 [84:36:15<402:38:34,  7.79s/it]
 17%|█▋        | 39036/225000 [84:36:20<350:59:08,  6.79s/it]
 17%|█▋        | 39037/225000 [84:36:24<320:43:50,  6.21s/it]
 17%|█▋        | 39038/225000 [84:36:29<298:07:08,  5.77s/it]
 17%|█▋        | 39039/225000 [84:36:38<352:08:56,  6.82s/it]
 17%|█▋        | 39040/225000 [84:36:49<405:01:49,  7.84s/it]
 17%|█▋        | 39041/225000 [84:36:52<339:44:37,  6.58s/it]
 17%|█▋        | 39042/225000 [84:37:03<411:20:33,  7.96s/it]
 17%|█▋        | 39043/225000 [84:37:08<353:36:27,  6.85s/it]
 17%|█▋        | 39044/225000 [84:37:16<378:37:54,  7.33s/it]
 17%|█▋        | 39045/225000 [84:37:25<395:36:29,  7.66s/it]
 17%|█▋        | 39046/225000 [84:37:29<351:22:35,  6.80s/it]
 17%|█▋        | 39047/225000 [84:37:34<324:10:51,  6.28s/it]
 17%|█▋        | 39048/225000 [84:37:39<293:42:29,  5.69s/it]
 17%|█▋        | 39049/225000 [84:37:51<398:36:45,  7.72s/it]
 17%|█▋        | 39050/225000 [84:37:56<360:27:16,  6.98s/it]
                                                             
{'loss': '0.7048', 'grad_norm': '0.599', 'learning_rate': '4.778e-05', 'epoch': '0.1736', 'num_input_tokens_seen': 143273504, 'train_runtime': '3.047e+05', 'train_tokens_per_second': '470.2'}

 17%|█▋        | 39050/225000 [84:37:56<360:27:16,  6.98s/it]
 17%|█▋        | 39051/225000 [84:38:02<333:57:51,  6.47s/it]
 17%|█▋        | 39052/225000 [84:38:11<369:58:54,  7.16s/it]
 17%|█▋        | 39053/225000 [84:38:14<313:22:52,  6.07s/it]
 17%|█▋        | 39054/225000 [84:38:23<363:01:46,  7.03s/it]
 17%|█▋        | 39055/225000 [84:38:30<353:36:12,  6.85s/it]
 17%|█▋        | 39056/225000 [84:38:34<313:23:04,  6.07s/it]
 17%|█▋        | 39057/225000 [84:38:38<286:46:37,  5.55s/it]
 17%|█▋        | 39058/225000 [84:38:43<270:22:20,  5.23s/it]
 17%|█▋        | 39059/225000 [84:38:55<377:14:00,  7.30s/it]
 17%|█▋        | 39060/225000 [84:39:00<341:28:45,  6.61s/it]
 17%|█▋        | 39061/225000 [84:39:07<341:17:01,  6.61s/it]
 17%|█▋        | 39062/225000 [84:39:16<380:07:43,  7.36s/it]
 17%|█▋        | 39063/225000 [84:39:20<337:34:08,  6.54s/it]
 17%|█▋        | 39064/225000 [84:39:26<318:59:31,  6.18s/it]
 17%|█▋        | 39065/225000 [84:39:31<308:20:20,  5.97s/it]
 17%|█▋        | 39066/225000 [84:39:36<295:21:44,  5.72s/it]
 17%|█▋        | 39067/225000 [84:39:42<293:35:30,  5.68s/it]
 17%|█▋        | 39068/225000 [84:39:47<292:09:18,  5.66s/it]
 17%|█▋        | 39069/225000 [84:39:52<281:35:10,  5.45s/it]
 17%|█▋        | 39070/225000 [84:39:58<279:17:57,  5.41s/it]
 17%|█▋        | 39071/225000 [84:40:03<282:40:22,  5.47s/it]
 17%|█▋        | 39072/225000 [84:40:14<362:21:00,  7.02s/it]
 17%|█▋        | 39073/225000 [84:40:23<397:47:21,  7.70s/it]
 17%|█▋        | 39074/225000 [84:40:31<405:14:58,  7.85s/it]
 17%|█▋        | 39075/225000 [84:40:37<366:48:51,  7.10s/it]
 17%|█▋        | 39076/225000 [84:40:42<338:18:38,  6.55s/it]
 17%|█▋        | 39077/225000 [84:40:48<324:47:25,  6.29s/it]
 17%|█▋        | 39078/225000 [84:40:53<304:19:47,  5.89s/it]
 17%|█▋        | 39079/225000 [84:41:04<382:02:43,  7.40s/it]
 17%|█▋        | 39080/225000 [84:41:12<404:12:38,  7.83s/it]
 17%|█▋        | 39081/225000 [84:41:20<393:29:26,  7.62s/it]
 17%|█▋        | 39082/225000 [84:41:23<334:58:01,  6.49s/it]
 17%|█▋        | 39083/225000 [84:41:34<402:11:50,  7.79s/it]
 17%|█▋        | 39084/225000 [84:41:38<338:26:52,  6.55s/it]
 17%|█▋        | 39085/225000 [84:41:42<305:48:07,  5.92s/it]
 17%|█▋        | 39086/225000 [84:41:54<390:46:29,  7.57s/it]
 17%|█▋        | 39087/225000 [84:42:00<363:44:00,  7.04s/it]
 17%|█▋        | 39088/225000 [84:42:04<330:08:28,  6.39s/it]
 17%|█▋        | 39089/225000 [84:42:10<315:58:18,  6.12s/it]
 17%|█▋        | 39090/225000 [84:42:16<309:09:59,  5.99s/it]
 17%|█▋        | 39091/225000 [84:42:21<293:09:31,  5.68s/it]
 17%|█▋        | 39092/225000 [84:42:28<317:26:04,  6.15s/it]
 17%|█▋        | 39093/225000 [84:42:39<393:04:57,  7.61s/it]
 17%|█▋        | 39094/225000 [84:42:43<334:32:48,  6.48s/it]
 17%|█▋        | 39095/225000 [84:42:53<400:06:58,  7.75s/it]
 17%|█▋        | 39096/225000 [84:42:59<362:46:13,  7.02s/it]
 17%|█▋        | 39097/225000 [84:43:04<341:17:33,  6.61s/it]
 17%|█▋        | 39098/225000 [84:43:10<318:20:23,  6.16s/it]
 17%|█▋        | 39099/225000 [84:43:15<304:20:51,  5.89s/it]
 17%|█▋        | 39100/225000 [84:43:20<289:56:19,  5.61s/it]
                                                             
{'loss': '0.6934', 'grad_norm': '0.7373', 'learning_rate': '4.778e-05', 'epoch': '0.1738', 'num_input_tokens_seen': 143420856, 'train_runtime': '3.05e+05', 'train_tokens_per_second': '470.2'}

 17%|█▋        | 39100/225000 [84:43:20<289:56:19,  5.61s/it]
 17%|█▋        | 39101/225000 [84:43:26<300:49:19,  5.83s/it]
 17%|█▋        | 39102/225000 [84:43:40<419:58:50,  8.13s/it]
 17%|█▋        | 39103/225000 [84:43:44<369:09:34,  7.15s/it]
 17%|█▋        | 39104/225000 [84:43:49<326:44:23,  6.33s/it]
 17%|█▋        | 39105/225000 [84:43:54<307:48:43,  5.96s/it]
 17%|█▋        | 39106/225000 [84:43:59<290:39:22,  5.63s/it]
 17%|█▋        | 39107/225000 [84:44:10<370:49:31,  7.18s/it]
 17%|█▋        | 39108/225000 [84:44:16<352:51:44,  6.83s/it]
 17%|█▋        | 39109/225000 [84:44:20<310:54:40,  6.02s/it]
 17%|█▋        | 39110/225000 [84:44:25<294:09:42,  5.70s/it]
 17%|█▋        | 39111/225000 [84:44:37<393:53:18,  7.63s/it]
 17%|█▋        | 39112/225000 [84:44:43<364:52:24,  7.07s/it]
 17%|█▋        | 39113/225000 [84:44:49<352:50:44,  6.83s/it]
 17%|█▋        | 39114/225000 [84:44:55<334:44:53,  6.48s/it]
 17%|█▋        | 39115/225000 [84:44:59<296:58:29,  5.75s/it]
 17%|█▋        | 39116/225000 [84:45:03<269:28:03,  5.22s/it]
 17%|█▋        | 39117/225000 [84:45:13<346:14:49,  6.71s/it]
 17%|█▋        | 39118/225000 [84:45:20<355:27:20,  6.88s/it]
 17%|█▋        | 39119/225000 [84:45:28<371:26:03,  7.19s/it]
 17%|█▋        | 39120/225000 [84:45:31<310:44:15,  6.02s/it]
 17%|█▋        | 39121/225000 [84:45:41<365:21:46,  7.08s/it]
 17%|█▋        | 39122/225000 [84:45:46<331:52:23,  6.43s/it]
 17%|█▋        | 39123/225000 [84:45:51<317:03:20,  6.14s/it]
 17%|█▋        | 39124/225000 [84:45:55<286:49:11,  5.56s/it]
 17%|█▋        | 39125/225000 [84:46:03<319:14:52,  6.18s/it]
 17%|█▋        | 39126/225000 [84:46:12<362:59:41,  7.03s/it]
 17%|█▋        | 39127/225000 [84:46:18<343:15:51,  6.65s/it]
 17%|█▋        | 39128/225000 [84:46:25<353:53:16,  6.85s/it]
 17%|█▋        | 39129/225000 [84:46:32<355:49:01,  6.89s/it]
 17%|█▋        | 39130/225000 [84:46:36<310:35:58,  6.02s/it]
 17%|█▋        | 39131/225000 [84:46:40<284:09:40,  5.50s/it]
 17%|█▋        | 39132/225000 [84:46:52<374:36:21,  7.26s/it]
 17%|█▋        | 39133/225000 [84:46:56<334:06:58,  6.47s/it]
 17%|█▋        | 39134/225000 [84:47:06<381:18:39,  7.39s/it]
 17%|█▋        | 39135/225000 [84:47:09<316:43:31,  6.13s/it]
 17%|█▋        | 39136/225000 [84:47:17<340:01:43,  6.59s/it]
 17%|█▋        | 39137/225000 [84:47:23<340:39:24,  6.60s/it]
 17%|█▋        | 39138/225000 [84:47:30<340:28:55,  6.59s/it]
 17%|█▋        | 39139/225000 [84:47:37<350:06:44,  6.78s/it]
 17%|█▋        | 39140/225000 [84:47:42<322:38:28,  6.25s/it]
 17%|█▋        | 39141/225000 [84:47:47<308:32:31,  5.98s/it]
 17%|█▋        | 39142/225000 [84:47:53<302:44:58,  5.86s/it]
 17%|█▋        | 39143/225000 [84:47:58<294:55:53,  5.71s/it]
 17%|█▋        | 39144/225000 [84:48:04<288:27:32,  5.59s/it]
 17%|█▋        | 39145/225000 [84:48:16<399:07:34,  7.73s/it]
 17%|█▋        | 39146/225000 [84:48:22<369:18:39,  7.15s/it]
 17%|█▋        | 39147/225000 [84:48:27<336:10:23,  6.51s/it]
 17%|█▋        | 39148/225000 [84:48:32<311:08:37,  6.03s/it]
 17%|█▋        | 39149/225000 [84:48:41<358:26:34,  6.94s/it]
 17%|█▋        | 39150/225000 [84:48:49<365:58:06,  7.09s/it]
                                                             
{'loss': '0.6985', 'grad_norm': '0.7136', 'learning_rate': '4.777e-05', 'epoch': '0.174', 'num_input_tokens_seen': 143567456, 'train_runtime': '3.053e+05', 'train_tokens_per_second': '470.2'}

 17%|█▋        | 39150/225000 [84:48:49<365:58:06,  7.09s/it]
 17%|█▋        | 39151/225000 [84:48:57<382:52:31,  7.42s/it]
 17%|█▋        | 39152/225000 [84:49:01<334:40:45,  6.48s/it]
 17%|█▋        | 39153/225000 [84:49:06<301:37:06,  5.84s/it]
 17%|█▋        | 39154/225000 [84:49:16<375:49:12,  7.28s/it]
 17%|█▋        | 39155/225000 [84:49:20<325:12:31,  6.30s/it]
 17%|█▋        | 39156/225000 [84:49:29<365:55:32,  7.09s/it]
 17%|█▋        | 39157/225000 [84:49:36<360:50:11,  6.99s/it]
 17%|█▋        | 39158/225000 [84:49:42<347:23:42,  6.73s/it]
 17%|█▋        | 39159/225000 [84:49:47<321:35:01,  6.23s/it]
 17%|█▋        | 39160/225000 [84:49:52<307:14:18,  5.95s/it]
 17%|█▋        | 39161/225000 [84:49:58<301:59:05,  5.85s/it]
 17%|█▋        | 39162/225000 [84:50:04<302:26:18,  5.86s/it]
 17%|█▋        | 39163/225000 [84:50:09<298:05:14,  5.77s/it]
 17%|█▋        | 39164/225000 [84:50:15<292:57:52,  5.68s/it]
 17%|█▋        | 39165/225000 [84:50:29<418:09:28,  8.10s/it]
 17%|█▋        | 39166/225000 [84:50:34<383:28:15,  7.43s/it]
 17%|█▋        | 39167/225000 [84:50:41<368:21:20,  7.14s/it]
 17%|█▋        | 39168/225000 [84:50:47<352:08:40,  6.82s/it]
 17%|█▋        | 39169/225000 [84:50:52<325:17:00,  6.30s/it]
 17%|█▋        | 39170/225000 [84:50:57<300:10:46,  5.82s/it]
 17%|█▋        | 39171/225000 [84:51:08<383:57:12,  7.44s/it]
 17%|█▋        | 39172/225000 [84:51:13<346:01:44,  6.70s/it]
 17%|█▋        | 39173/225000 [84:51:18<315:37:32,  6.11s/it]
 17%|█▋        | 39174/225000 [84:51:23<305:02:48,  5.91s/it]
 17%|█▋        | 39175/225000 [84:51:28<295:06:44,  5.72s/it]
 17%|█▋        | 39176/225000 [84:51:33<278:44:15,  5.40s/it]
 17%|█▋        | 39177/225000 [84:51:43<345:25:48,  6.69s/it]
 17%|█▋        | 39178/225000 [84:51:53<392:34:50,  7.61s/it]
 17%|█▋        | 39179/225000 [84:51:57<337:18:13,  6.53s/it]
 17%|█▋        | 39180/225000 [84:52:04<357:39:48,  6.93s/it]
 17%|█▋        | 39181/225000 [84:52:12<375:12:57,  7.27s/it]
 17%|█▋        | 39182/225000 [84:52:17<336:25:27,  6.52s/it]
 17%|█▋        | 39183/225000 [84:52:22<307:37:11,  5.96s/it]
 17%|█▋        | 39184/225000 [84:52:27<287:48:31,  5.58s/it]
 17%|█▋        | 39185/225000 [84:52:39<394:45:08,  7.65s/it]
 17%|█▋        | 39186/225000 [84:52:46<376:40:58,  7.30s/it]
 17%|█▋        | 39187/225000 [84:52:50<327:54:04,  6.35s/it]
 17%|█▋        | 39188/225000 [84:52:59<369:16:32,  7.15s/it]
 17%|█▋        | 39189/225000 [84:53:04<335:03:14,  6.49s/it]
 17%|█▋        | 39190/225000 [84:53:13<376:21:47,  7.29s/it]
 17%|█▋        | 39191/225000 [84:53:18<336:17:29,  6.52s/it]
 17%|█▋        | 39192/225000 [84:53:25<346:43:52,  6.72s/it]
 17%|█▋        | 39193/225000 [84:53:32<357:08:35,  6.92s/it]
 17%|█▋        | 39194/225000 [84:53:36<312:08:48,  6.05s/it]
 17%|█▋        | 39195/225000 [84:53:40<282:24:00,  5.47s/it]
 17%|█▋        | 39196/225000 [84:53:52<375:17:54,  7.27s/it]
 17%|█▋        | 39197/225000 [84:53:58<352:38:55,  6.83s/it]
 17%|█▋        | 39198/225000 [84:54:02<318:46:06,  6.18s/it]
 17%|█▋        | 39199/225000 [84:54:07<298:18:35,  5.78s/it]
 17%|█▋        | 39200/225000 [84:54:12<286:25:40,  5.55s/it]
                                                             
{'loss': '0.6894', 'grad_norm': '0.7209', 'learning_rate': '4.776e-05', 'epoch': '0.1742', 'num_input_tokens_seen': 143715016, 'train_runtime': '3.057e+05', 'train_tokens_per_second': '470.2'}

 17%|█▋        | 39200/225000 [84:54:12<286:25:40,  5.55s/it]
 17%|█▋        | 39201/225000 [84:54:22<349:58:44,  6.78s/it]
 17%|█▋        | 39202/225000 [84:54:33<413:24:41,  8.01s/it]
 17%|█▋        | 39203/225000 [84:54:39<390:44:54,  7.57s/it]
 17%|█▋        | 39204/225000 [84:54:46<386:15:17,  7.48s/it]
 17%|█▋        | 39205/225000 [84:54:51<336:28:31,  6.52s/it]
 17%|█▋        | 39206/225000 [84:54:58<355:40:45,  6.89s/it]
 17%|█▋        | 39207/225000 [84:55:03<313:07:27,  6.07s/it]
 17%|█▋        | 39208/225000 [84:55:11<354:05:05,  6.86s/it]
 17%|█▋        | 39209/225000 [84:55:17<340:38:13,  6.60s/it]
 17%|█▋        | 39210/225000 [84:55:25<357:40:16,  6.93s/it]
 17%|█▋        | 39211/225000 [84:55:29<311:58:31,  6.05s/it]
 17%|█▋        | 39212/225000 [84:55:34<298:17:24,  5.78s/it]
 17%|█▋        | 39213/225000 [84:55:40<293:24:57,  5.69s/it]
 17%|█▋        | 39214/225000 [84:55:44<279:09:38,  5.41s/it]
 17%|█▋        | 39215/225000 [84:55:58<411:44:53,  7.98s/it]
 17%|█▋        | 39216/225000 [84:56:04<378:39:43,  7.34s/it]
 17%|█▋        | 39217/225000 [84:56:13<394:39:10,  7.65s/it]
 17%|█▋        | 39218/225000 [84:56:20<389:46:54,  7.55s/it]
 17%|█▋        | 39219/225000 [84:56:25<346:10:50,  6.71s/it]
 17%|█▋        | 39220/225000 [84:56:34<380:14:01,  7.37s/it]
 17%|█▋        | 39221/225000 [84:56:42<392:52:43,  7.61s/it]
 17%|█▋        | 39222/225000 [84:56:46<342:07:27,  6.63s/it]
 17%|█▋        | 39223/225000 [84:56:55<371:02:01,  7.19s/it]
 17%|█▋        | 39224/225000 [84:56:59<329:22:14,  6.38s/it]
 17%|█▋        | 39225/225000 [84:57:03<295:17:23,  5.72s/it]
 17%|█▋        | 39226/225000 [84:57:13<358:30:32,  6.95s/it]
 17%|█▋        | 39227/225000 [84:57:19<341:52:30,  6.63s/it]
 17%|█▋        | 39228/225000 [84:57:24<322:19:55,  6.25s/it]
 17%|█▋        | 39229/225000 [84:57:34<370:53:06,  7.19s/it]
 17%|█▋        | 39230/225000 [84:57:41<369:01:18,  7.15s/it]
 17%|█▋        | 39231/225000 [84:57:46<344:47:37,  6.68s/it]
 17%|█▋        | 39232/225000 [84:57:52<330:11:41,  6.40s/it]
 17%|█▋        | 39233/225000 [84:58:01<369:08:10,  7.15s/it]
 17%|█▋        | 39234/225000 [84:58:10<393:57:45,  7.63s/it]
 17%|█▋        | 39235/225000 [84:58:14<336:15:37,  6.52s/it]
 17%|█▋        | 39236/225000 [84:58:22<357:34:32,  6.93s/it]
 17%|█▋        | 39237/225000 [84:58:27<329:50:57,  6.39s/it]
 17%|█▋        | 39238/225000 [84:58:32<308:00:57,  5.97s/it]
 17%|█▋        | 39239/225000 [84:58:37<300:57:16,  5.83s/it]
 17%|█▋        | 39240/225000 [84:58:42<287:37:08,  5.57s/it]
 17%|█▋        | 39241/225000 [84:58:53<370:15:50,  7.18s/it]
 17%|█▋        | 39242/225000 [84:59:01<388:39:35,  7.53s/it]
 17%|█▋        | 39243/225000 [84:59:06<347:26:30,  6.73s/it]
 17%|█▋        | 39244/225000 [84:59:11<324:10:30,  6.28s/it]
 17%|█▋        | 39245/225000 [84:59:16<303:41:56,  5.89s/it]
 17%|█▋        | 39246/225000 [84:59:22<295:35:24,  5.73s/it]
 17%|█▋        | 39247/225000 [84:59:35<404:03:19,  7.83s/it]
 17%|█▋        | 39248/225000 [84:59:39<353:26:27,  6.85s/it]
 17%|█▋        | 39249/225000 [84:59:44<325:48:01,  6.31s/it]
 17%|█▋        | 39250/225000 [84:59:50<320:28:15,  6.21s/it]
                                                             
{'loss': '0.7042', 'grad_norm': '0.6613', 'learning_rate': '4.775e-05', 'epoch': '0.1744', 'num_input_tokens_seen': 143863144, 'train_runtime': '3.06e+05', 'train_tokens_per_second': '470.2'}

 17%|█▋        | 39250/225000 [84:59:50<320:28:15,  6.21s/it]
 17%|█▋        | 39251/225000 [84:59:56<313:08:54,  6.07s/it]
 17%|█▋        | 39252/225000 [85:00:02<306:02:39,  5.93s/it]
 17%|█▋        | 39253/225000 [85:00:07<301:23:25,  5.84s/it]
 17%|█▋        | 39254/225000 [85:00:14<317:29:19,  6.15s/it]
 17%|█▋        | 39255/225000 [85:00:20<313:22:50,  6.07s/it]
 17%|█▋        | 39256/225000 [85:00:33<423:40:54,  8.21s/it]
 17%|█▋        | 39257/225000 [85:00:41<413:32:05,  8.01s/it]
 17%|█▋        | 39258/225000 [85:00:46<370:41:08,  7.18s/it]
 17%|█▋        | 39259/225000 [85:00:51<342:04:12,  6.63s/it]
 17%|█▋        | 39260/225000 [85:00:57<324:30:00,  6.29s/it]
 17%|█▋        | 39261/225000 [85:01:02<310:44:02,  6.02s/it]
 17%|█▋        | 39262/225000 [85:01:08<303:14:41,  5.88s/it]
 17%|█▋        | 39263/225000 [85:01:13<293:59:38,  5.70s/it]
 17%|█▋        | 39264/225000 [85:01:18<286:48:58,  5.56s/it]
 17%|█▋        | 39265/225000 [85:01:29<369:21:30,  7.16s/it]
 17%|█▋        | 39266/225000 [85:01:40<428:34:09,  8.31s/it]
 17%|█▋        | 39267/225000 [85:01:44<353:14:23,  6.85s/it]
 17%|█▋        | 39268/225000 [85:01:49<331:52:34,  6.43s/it]
 17%|█▋        | 39269/225000 [85:01:56<344:38:37,  6.68s/it]
 17%|█▋        | 39270/225000 [85:02:04<354:13:26,  6.87s/it]
 17%|█▋        | 39271/225000 [85:02:09<333:31:30,  6.46s/it]
 17%|█▋        | 39272/225000 [85:02:20<408:36:10,  7.92s/it]
 17%|█▋        | 39273/225000 [85:02:24<347:08:29,  6.73s/it]
 17%|█▋        | 39274/225000 [85:02:31<351:43:29,  6.82s/it]
 17%|█▋        | 39275/225000 [85:02:38<354:28:10,  6.87s/it]
 17%|█▋        | 39276/225000 [85:02:47<375:36:28,  7.28s/it]
 17%|█▋        | 39277/225000 [85:02:51<333:58:32,  6.47s/it]
 17%|█▋        | 39278/225000 [85:02:56<304:24:55,  5.90s/it]
 17%|█▋        | 39279/225000 [85:03:01<293:50:51,  5.70s/it]
 17%|█▋        | 39280/225000 [85:03:14<403:26:19,  7.82s/it]
 17%|█▋        | 39281/225000 [85:03:18<341:10:53,  6.61s/it]
 17%|█▋        | 39282/225000 [85:03:22<308:08:26,  5.97s/it]
 17%|█▋        | 39283/225000 [85:03:28<301:16:12,  5.84s/it]
 17%|█▋        | 39284/225000 [85:03:40<410:31:41,  7.96s/it]
 17%|█▋        | 39285/225000 [85:03:45<360:54:27,  7.00s/it]
 17%|█▋        | 39286/225000 [85:03:50<328:29:11,  6.37s/it]
 17%|█▋        | 39287/225000 [85:03:55<307:56:51,  5.97s/it]
 17%|█▋        | 39288/225000 [85:04:02<324:05:42,  6.28s/it]
 17%|█▋        | 39289/225000 [85:04:13<402:19:29,  7.80s/it]
 17%|█▋        | 39290/225000 [85:04:20<383:45:04,  7.44s/it]
 17%|█▋        | 39291/225000 [85:04:25<351:56:26,  6.82s/it]
 17%|█▋        | 39292/225000 [85:04:33<367:59:21,  7.13s/it]
 17%|█▋        | 39293/225000 [85:04:37<320:17:01,  6.21s/it]
 17%|█▋        | 39294/225000 [85:04:48<382:13:49,  7.41s/it]
 17%|█▋        | 39295/225000 [85:04:51<324:47:01,  6.30s/it]
 17%|█▋        | 39296/225000 [85:05:02<390:00:27,  7.56s/it]
 17%|█▋        | 39297/225000 [85:05:08<374:35:50,  7.26s/it]
 17%|█▋        | 39298/225000 [85:05:12<317:49:18,  6.16s/it]
 17%|█▋        | 39299/225000 [85:05:21<363:36:17,  7.05s/it]
 17%|█▋        | 39300/225000 [85:05:28<355:18:12,  6.89s/it]
                                                             
{'loss': '0.6974', 'grad_norm': '0.6695', 'learning_rate': '4.775e-05', 'epoch': '0.1747', 'num_input_tokens_seen': 144010416, 'train_runtime': '3.063e+05', 'train_tokens_per_second': '470.1'}

 17%|█▋        | 39300/225000 [85:05:28<355:18:12,  6.89s/it]
 17%|█▋        | 39301/225000 [85:05:32<311:22:40,  6.04s/it]
 17%|█▋        | 39302/225000 [85:05:37<294:39:25,  5.71s/it]
 17%|█▋        | 39303/225000 [85:05:41<278:17:38,  5.40s/it]
 17%|█▋        | 39304/225000 [85:05:46<270:37:12,  5.25s/it]
 17%|█▋        | 39305/225000 [85:05:58<378:50:40,  7.34s/it]
 17%|█▋        | 39306/225000 [85:06:04<347:59:38,  6.75s/it]
 17%|█▋        | 39307/225000 [85:06:14<394:45:49,  7.65s/it]
 17%|█▋        | 39308/225000 [85:06:17<338:08:15,  6.56s/it]
 17%|█▋        | 39309/225000 [85:06:24<342:36:28,  6.64s/it]
 17%|█▋        | 39310/225000 [85:06:34<396:19:46,  7.68s/it]
 17%|█▋        | 39311/225000 [85:06:38<334:39:49,  6.49s/it]
 17%|█▋        | 39312/225000 [85:06:47<368:02:46,  7.14s/it]
 17%|█▋        | 39313/225000 [85:06:52<341:44:44,  6.63s/it]
 17%|█▋        | 39314/225000 [85:07:00<366:18:28,  7.10s/it]
 17%|█▋        | 39315/225000 [85:07:05<332:29:59,  6.45s/it]
 17%|█▋        | 39316/225000 [85:07:15<377:56:27,  7.33s/it]
 17%|█▋        | 39317/225000 [85:07:22<380:13:57,  7.37s/it]
 17%|█▋        | 39318/225000 [85:07:27<345:48:40,  6.70s/it]
 17%|█▋        | 39319/225000 [85:07:32<316:10:32,  6.13s/it]
 17%|█▋        | 39320/225000 [85:07:37<299:48:14,  5.81s/it]
 17%|█▋        | 39321/225000 [85:07:46<348:18:20,  6.75s/it]
 17%|█▋        | 39322/225000 [85:07:57<405:10:43,  7.86s/it]
 17%|█▋        | 39323/225000 [85:08:00<338:46:23,  6.57s/it]
 17%|█▋        | 39324/225000 [85:08:05<311:03:56,  6.03s/it]
 17%|█▋        | 39325/225000 [85:08:17<402:20:40,  7.80s/it]
 17%|█▋        | 39326/225000 [85:08:22<364:53:48,  7.07s/it]
 17%|█▋        | 39327/225000 [85:08:26<318:23:33,  6.17s/it]
 17%|█▋        | 39328/225000 [85:08:36<378:34:34,  7.34s/it]
 17%|█▋        | 39329/225000 [85:08:40<327:07:41,  6.34s/it]
 17%|█▋        | 39330/225000 [85:08:50<383:52:53,  7.44s/it]
 17%|█▋        | 39331/225000 [85:08:54<324:22:52,  6.29s/it]
 17%|█▋        | 39332/225000 [85:09:00<311:58:05,  6.05s/it]
 17%|█▋        | 39333/225000 [85:09:05<307:10:36,  5.96s/it]
 17%|█▋        | 39334/225000 [85:09:10<293:00:28,  5.68s/it]
 17%|█▋        | 39335/225000 [85:09:23<396:02:09,  7.68s/it]
 17%|█▋        | 39336/225000 [85:09:29<373:04:49,  7.23s/it]
 17%|█▋        | 39337/225000 [85:09:39<411:25:08,  7.98s/it]
 17%|█▋        | 39338/225000 [85:09:43<352:43:14,  6.84s/it]
 17%|█▋        | 39339/225000 [85:09:51<379:34:49,  7.36s/it]
 17%|█▋        | 39340/225000 [85:09:57<354:55:07,  6.88s/it]
 17%|█▋        | 39341/225000 [85:10:05<365:40:14,  7.09s/it]
 17%|█▋        | 39342/225000 [85:10:11<353:01:28,  6.85s/it]
 17%|█▋        | 39343/225000 [85:10:20<389:14:11,  7.55s/it]
 17%|█▋        | 39344/225000 [85:10:27<372:22:53,  7.22s/it]
 17%|█▋        | 39345/225000 [85:10:32<344:21:14,  6.68s/it]
 17%|█▋        | 39346/225000 [85:10:41<374:30:08,  7.26s/it]
 17%|█▋        | 39347/225000 [85:10:47<362:50:55,  7.04s/it]
 17%|█▋        | 39348/225000 [85:10:52<331:56:27,  6.44s/it]
 17%|█▋        | 39349/225000 [85:10:57<307:25:58,  5.96s/it]
 17%|█▋        | 39350/225000 [85:11:02<294:13:35,  5.71s/it]
                                                             
{'loss': '0.696', 'grad_norm': '0.7079', 'learning_rate': '4.774e-05', 'epoch': '0.1749', 'num_input_tokens_seen': 144160160, 'train_runtime': '3.067e+05', 'train_tokens_per_second': '470.1'}

 17%|█▋        | 39350/225000 [85:11:02<294:13:35,  5.71s/it]
 17%|█▋        | 39351/225000 [85:11:08<296:28:27,  5.75s/it]
 17%|█▋        | 39352/225000 [85:11:20<388:28:53,  7.53s/it]
 17%|█▋        | 39353/225000 [85:11:28<396:10:20,  7.68s/it]
 17%|█▋        | 39354/225000 [85:11:32<343:59:42,  6.67s/it]
 17%|█▋        | 39355/225000 [85:11:41<384:18:23,  7.45s/it]
 17%|█▋        | 39356/225000 [85:11:50<398:13:00,  7.72s/it]
 17%|█▋        | 39357/225000 [85:11:55<355:33:44,  6.90s/it]
 17%|█▋        | 39358/225000 [85:12:00<327:02:17,  6.34s/it]
 17%|█▋        | 39359/225000 [85:12:05<313:13:08,  6.07s/it]
 17%|█▋        | 39360/225000 [85:12:19<427:11:02,  8.28s/it]
 17%|█▋        | 39361/225000 [85:12:24<379:57:41,  7.37s/it]
 17%|█▋        | 39362/225000 [85:12:29<349:54:55,  6.79s/it]
 17%|█▋        | 39363/225000 [85:12:35<327:26:24,  6.35s/it]
 17%|█▋        | 39364/225000 [85:12:40<310:35:50,  6.02s/it]
 17%|█▋        | 39365/225000 [85:12:45<303:02:19,  5.88s/it]
 17%|█▋        | 39366/225000 [85:12:51<299:16:36,  5.80s/it]
 17%|█▋        | 39367/225000 [85:12:56<290:18:32,  5.63s/it]
 17%|█▋        | 39368/225000 [85:13:03<308:37:28,  5.99s/it]
 17%|█▋        | 39369/225000 [85:13:15<402:03:38,  7.80s/it]
 17%|█▋        | 39370/225000 [85:13:19<345:54:23,  6.71s/it]
 17%|█▋        | 39371/225000 [85:13:26<341:23:41,  6.62s/it]
 17%|█▋        | 39372/225000 [85:13:35<381:38:16,  7.40s/it]
 17%|█▋        | 39373/225000 [85:13:39<324:41:13,  6.30s/it]
 17%|█▋        | 39374/225000 [85:13:47<351:09:39,  6.81s/it]
 18%|█▊        | 39375/225000 [85:13:53<339:33:54,  6.59s/it]
 18%|█▊        | 39376/225000 [85:14:03<392:11:26,  7.61s/it]
 18%|█▊        | 39377/225000 [85:14:06<328:08:33,  6.36s/it]
 18%|█▊        | 39378/225000 [85:14:11<313:08:51,  6.07s/it]
 18%|█▊        | 39379/225000 [85:14:20<350:58:20,  6.81s/it]
 18%|█▊        | 39380/225000 [85:14:24<314:22:28,  6.10s/it]
 18%|█▊        | 39381/225000 [85:14:30<300:49:20,  5.83s/it]
 18%|█▊        | 39382/225000 [85:14:35<295:41:15,  5.73s/it]
 18%|█▊        | 39383/225000 [85:14:41<302:49:41,  5.87s/it]
 18%|█▊        | 39384/225000 [85:14:54<415:12:12,  8.05s/it]
 18%|█▊        | 39385/225000 [85:15:00<376:36:17,  7.30s/it]
 18%|█▊        | 39386/225000 [85:15:08<389:43:14,  7.56s/it]
 18%|█▊        | 39387/225000 [85:15:14<360:54:37,  7.00s/it]
 18%|█▊        | 39388/225000 [85:15:20<344:25:12,  6.68s/it]
 18%|█▊        | 39389/225000 [85:15:29<379:44:54,  7.37s/it]
 18%|█▊        | 39390/225000 [85:15:32<317:35:54,  6.16s/it]
 18%|█▊        | 39391/225000 [85:15:40<345:47:06,  6.71s/it]
 18%|█▊        | 39392/225000 [85:15:47<351:13:29,  6.81s/it]
 18%|█▊        | 39393/225000 [85:15:53<343:40:12,  6.67s/it]
 18%|█▊        | 39394/225000 [85:16:00<333:56:20,  6.48s/it]
 18%|█▊        | 39395/225000 [85:16:04<305:10:37,  5.92s/it]
 18%|█▊        | 39396/225000 [85:16:18<427:01:02,  8.28s/it]
 18%|█▊        | 39397/225000 [85:16:22<360:31:47,  6.99s/it]
 18%|█▊        | 39398/225000 [85:16:29<354:58:33,  6.89s/it]
 18%|█▊        | 39399/225000 [85:16:37<374:35:24,  7.27s/it]
 18%|█▊        | 39400/225000 [85:16:45<389:02:06,  7.55s/it]
                                                             
{'loss': '0.6872', 'grad_norm': '0.7688', 'learning_rate': '4.773e-05', 'epoch': '0.1751', 'num_input_tokens_seen': 144309112, 'train_runtime': '3.07e+05', 'train_tokens_per_second': '470.1'}

 18%|█▊        | 39400/225000 [85:16:45<389:02:06,  7.55s/it]
 18%|█▊        | 39401/225000 [85:16:49<328:14:39,  6.37s/it]
 18%|█▊        | 39402/225000 [85:16:58<374:08:04,  7.26s/it]
 18%|█▊        | 39403/225000 [85:17:02<320:38:15,  6.22s/it]
 18%|█▊        | 39404/225000 [85:17:12<383:09:50,  7.43s/it]
 18%|█▊        | 39405/225000 [85:17:16<334:30:53,  6.49s/it]
 18%|█▊        | 39406/225000 [85:17:23<341:46:53,  6.63s/it]
 18%|█▊        | 39407/225000 [85:17:31<363:20:31,  7.05s/it]
 18%|█▊        | 39408/225000 [85:17:35<308:47:06,  5.99s/it]
 18%|█▊        | 39409/225000 [85:17:44<357:59:30,  6.94s/it]
 18%|█▊        | 39410/225000 [85:17:48<319:44:32,  6.20s/it]
 18%|█▊        | 39411/225000 [85:17:53<289:28:55,  5.62s/it]
 18%|█▊        | 39412/225000 [85:17:57<278:29:28,  5.40s/it]
 18%|█▊        | 39413/225000 [85:18:02<265:36:23,  5.15s/it]
 18%|█▊        | 39414/225000 [85:18:15<392:39:27,  7.62s/it]
 18%|█▊        | 39415/225000 [85:18:22<374:52:00,  7.27s/it]
 18%|█▊        | 39416/225000 [85:18:30<391:39:53,  7.60s/it]
 18%|█▊        | 39417/225000 [85:18:34<337:47:42,  6.55s/it]
 18%|█▊        | 39418/225000 [85:18:44<378:53:35,  7.35s/it]
 18%|█▊        | 39419/225000 [85:18:50<361:57:22,  7.02s/it]
 18%|█▊        | 39420/225000 [85:18:55<336:23:35,  6.53s/it]
 18%|█▊        | 39421/225000 [85:19:01<320:28:19,  6.22s/it]
 18%|█▊        | 39422/225000 [85:19:06<310:08:52,  6.02s/it]
 18%|█▊        | 39423/225000 [85:19:12<301:29:45,  5.85s/it]
 18%|█▊        | 39424/225000 [85:19:17<300:24:12,  5.83s/it]
 18%|█▊        | 39425/225000 [85:19:27<350:45:13,  6.80s/it]
 18%|█▊        | 39426/225000 [85:19:39<430:24:15,  8.35s/it]
 18%|█▊        | 39427/225000 [85:19:42<359:27:59,  6.97s/it]
 18%|█▊        | 39428/225000 [85:19:49<357:39:52,  6.94s/it]
 18%|█▊        | 39429/225000 [85:19:54<322:47:53,  6.26s/it]
 18%|█▊        | 39430/225000 [85:19:59<300:20:24,  5.83s/it]
 18%|█▊        | 39431/225000 [85:20:03<279:22:19,  5.42s/it]
 18%|█▊        | 39432/225000 [85:20:16<394:57:24,  7.66s/it]
 18%|█▊        | 39433/225000 [85:20:20<340:37:40,  6.61s/it]
 18%|█▊        | 39434/225000 [85:20:31<400:41:29,  7.77s/it]
 18%|█▊        | 39435/225000 [85:20:35<343:03:23,  6.66s/it]
 18%|█▊        | 39436/225000 [85:20:40<322:13:38,  6.25s/it]
 18%|█▊        | 39437/225000 [85:20:46<310:45:15,  6.03s/it]
 18%|█▊        | 39438/225000 [85:20:51<306:19:59,  5.94s/it]
 18%|█▊        | 39439/225000 [85:20:57<303:27:37,  5.89s/it]
 18%|█▊        | 39440/225000 [85:21:02<287:21:31,  5.57s/it]
 18%|█▊        | 39441/225000 [85:21:16<414:20:27,  8.04s/it]
 18%|█▊        | 39442/225000 [85:21:24<424:06:28,  8.23s/it]
 18%|█▊        | 39443/225000 [85:21:31<407:20:11,  7.90s/it]
 18%|█▊        | 39444/225000 [85:21:36<362:50:36,  7.04s/it]
 18%|█▊        | 39445/225000 [85:21:42<331:59:11,  6.44s/it]
 18%|█▊        | 39446/225000 [85:21:47<310:30:49,  6.02s/it]
 18%|█▊        | 39447/225000 [85:21:52<300:27:22,  5.83s/it]
 18%|█▊        | 39448/225000 [85:21:58<297:45:02,  5.78s/it]
 18%|█▊        | 39449/225000 [85:22:03<295:16:48,  5.73s/it]
 18%|█▊        | 39450/225000 [85:22:11<325:52:59,  6.32s/it]
                                                             
{'loss': '0.6848', 'grad_norm': '0.9459', 'learning_rate': '4.772e-05', 'epoch': '0.1753', 'num_input_tokens_seen': 144456336, 'train_runtime': '3.073e+05', 'train_tokens_per_second': '470'}

 18%|█▊        | 39450/225000 [85:22:11<325:52:59,  6.32s/it]
 18%|█▊        | 39451/225000 [85:22:24<436:24:18,  8.47s/it]
 18%|█▊        | 39452/225000 [85:22:29<372:16:25,  7.22s/it]
 18%|█▊        | 39453/225000 [85:22:33<326:28:44,  6.33s/it]
 18%|█▊        | 39454/225000 [85:22:41<355:25:25,  6.90s/it]
 18%|█▊        | 39455/225000 [85:22:51<393:48:34,  7.64s/it]
 18%|█▊        | 39456/225000 [85:22:57<375:36:58,  7.29s/it]
 18%|█▊        | 39457/225000 [85:23:03<351:48:38,  6.83s/it]
 18%|█▊        | 39458/225000 [85:23:08<329:41:43,  6.40s/it]
 18%|█▊        | 39459/225000 [85:23:13<308:08:21,  5.98s/it]
 18%|█▊        | 39460/225000 [85:23:19<299:32:51,  5.81s/it]
 18%|█▊        | 39461/225000 [85:23:25<306:14:49,  5.94s/it]
 18%|█▊        | 39462/225000 [85:23:38<418:25:49,  8.12s/it]
 18%|█▊        | 39463/225000 [85:23:41<345:14:34,  6.70s/it]
 18%|█▊        | 39464/225000 [85:23:47<320:19:43,  6.22s/it]
 18%|█▊        | 39465/225000 [85:23:55<347:41:11,  6.75s/it]
 18%|█▊        | 39466/225000 [85:24:00<327:14:06,  6.35s/it]
 18%|█▊        | 39467/225000 [85:24:06<318:37:47,  6.18s/it]
 18%|█▊        | 39468/225000 [85:24:11<299:14:50,  5.81s/it]
 18%|█▊        | 39469/225000 [85:24:24<412:52:43,  8.01s/it]
 18%|█▊        | 39470/225000 [85:24:33<428:04:38,  8.31s/it]
 18%|█▊        | 39471/225000 [85:24:39<390:54:32,  7.59s/it]
 18%|█▊        | 39472/225000 [85:24:43<347:14:13,  6.74s/it]
 18%|█▊        | 39473/225000 [85:24:52<372:18:05,  7.22s/it]
 18%|█▊        | 39474/225000 [85:24:57<341:35:48,  6.63s/it]
 18%|█▊        | 39475/225000 [85:25:05<354:40:00,  6.88s/it]
 18%|█▊        | 39476/225000 [85:25:10<326:48:59,  6.34s/it]
 18%|█▊        | 39477/225000 [85:25:15<304:37:26,  5.91s/it]
 18%|█▊        | 39478/225000 [85:25:22<323:04:39,  6.27s/it]
 18%|█▊        | 39479/225000 [85:25:33<396:03:57,  7.69s/it]
 18%|█▊        | 39480/225000 [85:25:38<354:42:29,  6.88s/it]
 18%|█▊        | 39481/225000 [85:25:43<327:21:37,  6.35s/it]
 18%|█▊        | 39482/225000 [85:25:48<305:17:37,  5.92s/it]
 18%|█▊        | 39483/225000 [85:25:53<291:04:52,  5.65s/it]
 18%|█▊        | 39484/225000 [85:26:06<402:45:12,  7.82s/it]
 18%|█▊        | 39485/225000 [85:26:10<345:16:37,  6.70s/it]
 18%|█▊        | 39486/225000 [85:26:19<381:54:58,  7.41s/it]
 18%|█▊        | 39487/225000 [85:26:26<376:10:42,  7.30s/it]
 18%|█▊        | 39488/225000 [85:26:32<352:56:18,  6.85s/it]
 18%|█▊        | 39489/225000 [85:26:41<396:50:34,  7.70s/it]
 18%|█▊        | 39490/225000 [85:26:45<341:49:03,  6.63s/it]
 18%|█▊        | 39491/225000 [85:26:50<313:21:14,  6.08s/it]
 18%|█▊        | 39492/225000 [85:27:01<389:51:13,  7.57s/it]
 18%|█▊        | 39493/225000 [85:27:05<335:25:47,  6.51s/it]
 18%|█▊        | 39494/225000 [85:27:14<375:18:02,  7.28s/it]
 18%|█▊        | 39495/225000 [85:27:22<375:25:11,  7.29s/it]
 18%|█▊        | 39496/225000 [85:27:25<314:31:18,  6.10s/it]
 18%|█▊        | 39497/225000 [85:27:34<355:42:14,  6.90s/it]
 18%|█▊        | 39498/225000 [85:27:38<318:39:13,  6.18s/it]
 18%|█▊        | 39499/225000 [85:27:43<295:51:34,  5.74s/it]
 18%|█▊        | 39500/225000 [85:27:48<283:31:58,  5.50s/it]
                                                             
{'loss': '0.6943', 'grad_norm': '0.9313', 'learning_rate': '4.772e-05', 'epoch': '0.1756', 'num_input_tokens_seen': 144604144, 'train_runtime': '3.077e+05', 'train_tokens_per_second': '470'}

 18%|█▊        | 39500/225000 [85:27:48<283:31:58,  5.50s/it]
 18%|█▊        | 39501/225000 [85:27:57<342:51:39,  6.65s/it]
 18%|█▊        | 39502/225000 [85:28:05<361:58:07,  7.02s/it]
 18%|█▊        | 39503/225000 [85:28:14<385:40:50,  7.49s/it]
 18%|█▊        | 39504/225000 [85:28:20<366:01:43,  7.10s/it]
 18%|█▊        | 39505/225000 [85:28:24<322:49:59,  6.27s/it]
 18%|█▊        | 39506/225000 [85:28:29<295:26:40,  5.73s/it]
 18%|█▊        | 39507/225000 [85:28:33<278:47:01,  5.41s/it]
 18%|█▊        | 39508/225000 [85:28:39<281:21:26,  5.46s/it]
 18%|█▊        | 39509/225000 [85:28:45<288:20:50,  5.60s/it]
 18%|█▊        | 39510/225000 [85:28:50<286:16:18,  5.56s/it]
 18%|█▊        | 39511/225000 [85:29:00<355:48:53,  6.91s/it]
 18%|█▊        | 39512/225000 [85:29:12<428:03:12,  8.31s/it]
 18%|█▊        | 39513/225000 [85:29:16<356:37:04,  6.92s/it]
 18%|█▊        | 39514/225000 [85:29:25<387:50:05,  7.53s/it]
 18%|█▊        | 39515/225000 [85:29:30<349:09:50,  6.78s/it]
 18%|█▊        | 39516/225000 [85:29:35<324:00:01,  6.29s/it]
 18%|█▊        | 39517/225000 [85:29:40<306:25:54,  5.95s/it]
 18%|█▊        | 39518/225000 [85:29:51<388:48:51,  7.55s/it]
 18%|█▊        | 39519/225000 [85:29:57<358:18:36,  6.95s/it]
 18%|█▊        | 39520/225000 [85:30:02<324:52:03,  6.31s/it]
 18%|█▊        | 39521/225000 [85:30:07<310:34:40,  6.03s/it]
 18%|█▊        | 39522/225000 [85:30:12<300:30:03,  5.83s/it]
 18%|█▊        | 39523/225000 [85:30:18<290:59:04,  5.65s/it]
 18%|█▊        | 39524/225000 [85:30:30<391:48:50,  7.60s/it]
 18%|█▊        | 39525/225000 [85:30:37<383:35:46,  7.45s/it]
 18%|█▊        | 39526/225000 [85:30:42<343:51:17,  6.67s/it]
 18%|█▊        | 39527/225000 [85:30:47<321:21:34,  6.24s/it]
 18%|█▊        | 39528/225000 [85:30:52<298:31:41,  5.79s/it]
 18%|█▊        | 39529/225000 [85:30:58<302:48:28,  5.88s/it]
 18%|█▊        | 39530/225000 [85:31:11<417:49:37,  8.11s/it]
 18%|█▊        | 39531/225000 [85:31:18<395:46:23,  7.68s/it]
 18%|█▊        | 39532/225000 [85:31:23<364:36:59,  7.08s/it]
 18%|█▊        | 39533/225000 [85:31:32<381:27:00,  7.40s/it]
 18%|█▊        | 39534/225000 [85:31:37<354:51:04,  6.89s/it]
 18%|█▊        | 39535/225000 [85:31:43<333:07:23,  6.47s/it]
 18%|█▊        | 39536/225000 [85:31:48<308:46:01,  5.99s/it]
 18%|█▊        | 39537/225000 [85:31:53<295:47:29,  5.74s/it]
 18%|█▊        | 39538/225000 [85:32:00<313:06:11,  6.08s/it]
 18%|█▊        | 39539/225000 [85:32:12<404:40:51,  7.86s/it]
 18%|█▊        | 39540/225000 [85:32:16<351:01:15,  6.81s/it]
 18%|█▊        | 39541/225000 [85:32:24<368:13:13,  7.15s/it]
 18%|█▊        | 39542/225000 [85:32:30<356:58:51,  6.93s/it]
 18%|█▊        | 39543/225000 [85:32:35<322:12:56,  6.25s/it]
 18%|█▊        | 39544/225000 [85:32:40<301:13:36,  5.85s/it]
 18%|█▊        | 39545/225000 [85:32:45<287:39:38,  5.58s/it]
 18%|█▊        | 39546/225000 [85:32:53<331:12:08,  6.43s/it]
 18%|█▊        | 39547/225000 [85:33:07<439:53:02,  8.54s/it]
 18%|█▊        | 39548/225000 [85:33:10<365:23:37,  7.09s/it]
 18%|█▊        | 39549/225000 [85:33:18<369:14:06,  7.17s/it]
 18%|█▊        | 39550/225000 [85:33:23<332:57:47,  6.46s/it]
                                                             
{'loss': '0.6844', 'grad_norm': '0.7894', 'learning_rate': '4.771e-05', 'epoch': '0.1758', 'num_input_tokens_seen': 144751232, 'train_runtime': '3.08e+05', 'train_tokens_per_second': '470'}

 18%|█▊        | 39550/225000 [85:33:23<332:57:47,  6.46s/it]
 18%|█▊        | 39551/225000 [85:33:27<304:48:54,  5.92s/it]
 18%|█▊        | 39552/225000 [85:33:38<377:18:18,  7.32s/it]
 18%|█▊        | 39553/225000 [85:33:43<348:16:08,  6.76s/it]
 18%|█▊        | 39554/225000 [85:33:50<339:15:34,  6.59s/it]
 18%|█▊        | 39555/225000 [85:33:55<326:30:50,  6.34s/it]
 18%|█▊        | 39556/225000 [85:34:01<313:07:29,  6.08s/it]
 18%|█▊        | 39557/225000 [85:34:09<342:23:28,  6.65s/it]
 18%|█▊        | 39558/225000 [85:34:18<389:01:44,  7.55s/it]
 18%|█▊        | 39559/225000 [85:34:22<335:48:33,  6.52s/it]
 18%|█▊        | 39560/225000 [85:34:30<346:25:09,  6.73s/it]
 18%|█▊        | 39561/225000 [85:34:38<369:55:48,  7.18s/it]
 18%|█▊        | 39562/225000 [85:34:46<386:55:13,  7.51s/it]
 18%|█▊        | 39563/225000 [85:34:50<332:50:53,  6.46s/it]
 18%|█▊        | 39564/225000 [85:34:59<367:39:09,  7.14s/it]
 18%|█▊        | 39565/225000 [85:35:04<333:49:01,  6.48s/it]
 18%|█▊        | 39566/225000 [85:35:09<308:25:22,  5.99s/it]
 18%|█▊        | 39567/225000 [85:35:13<287:06:06,  5.57s/it]
 18%|█▊        | 39568/225000 [85:35:23<350:57:12,  6.81s/it]
 18%|█▊        | 39569/225000 [85:35:32<388:20:17,  7.54s/it]
 18%|█▊        | 39570/225000 [85:35:36<334:00:35,  6.48s/it]
 18%|█▊        | 39571/225000 [85:35:46<386:23:34,  7.50s/it]
 18%|█▊        | 39572/225000 [85:35:52<357:58:22,  6.95s/it]
 18%|█▊        | 39573/225000 [85:35:57<327:08:20,  6.35s/it]
 18%|█▊        | 39574/225000 [85:36:01<295:49:23,  5.74s/it]
 18%|█▊        | 39575/225000 [85:36:12<380:12:52,  7.38s/it]
 18%|█▊        | 39576/225000 [85:36:17<333:18:14,  6.47s/it]
 18%|█▊        | 39577/225000 [85:36:22<322:10:09,  6.25s/it]
 18%|█▊        | 39578/225000 [85:36:31<357:08:25,  6.93s/it]
 18%|█▊        | 39579/225000 [85:36:38<362:08:13,  7.03s/it]
 18%|█▊        | 39580/225000 [85:36:45<365:45:26,  7.10s/it]
 18%|█▊        | 39581/225000 [85:36:50<321:34:13,  6.24s/it]
 18%|█▊        | 39582/225000 [85:36:59<366:35:30,  7.12s/it]
 18%|█▊        | 39583/225000 [85:37:04<338:38:53,  6.58s/it]
 18%|█▊        | 39584/225000 [85:37:10<320:25:23,  6.22s/it]
 18%|█▊        | 39585/225000 [85:37:15<308:31:32,  5.99s/it]
 18%|█▊        | 39586/225000 [85:37:20<297:07:25,  5.77s/it]
 18%|█▊        | 39587/225000 [85:37:26<289:29:00,  5.62s/it]
 18%|█▊        | 39588/225000 [85:37:31<287:50:27,  5.59s/it]
 18%|█▊        | 39589/225000 [85:37:44<404:11:52,  7.85s/it]
 18%|█▊        | 39590/225000 [85:37:49<355:55:43,  6.91s/it]
 18%|█▊        | 39591/225000 [85:37:54<327:12:34,  6.35s/it]
 18%|█▊        | 39592/225000 [85:37:59<306:00:19,  5.94s/it]
 18%|█▊        | 39593/225000 [85:38:05<304:33:46,  5.91s/it]
 18%|█▊        | 39594/225000 [85:38:18<416:50:51,  8.09s/it]
 18%|█▊        | 39595/225000 [85:38:22<346:55:02,  6.74s/it]
 18%|█▊        | 39596/225000 [85:38:30<378:04:41,  7.34s/it]
 18%|█▊        | 39597/225000 [85:38:33<314:08:45,  6.10s/it]
 18%|█▊        | 39598/225000 [85:38:42<345:46:10,  6.71s/it]
 18%|█▊        | 39599/225000 [85:38:47<326:42:42,  6.34s/it]
 18%|█▊        | 39600/225000 [85:38:56<360:00:21,  6.99s/it]
                                                             
{'loss': '0.6916', 'grad_norm': '0.7307', 'learning_rate': '4.77e-05', 'epoch': '0.176', 'num_input_tokens_seen': 144899968, 'train_runtime': '3.083e+05', 'train_tokens_per_second': '469.9'}

 18%|█▊        | 39600/225000 [85:38:56<360:00:21,  6.99s/it]
 18%|█▊        | 39601/225000 [85:39:04<377:40:31,  7.33s/it]
 18%|█▊        | 39602/225000 [85:39:07<321:03:59,  6.23s/it]
 18%|█▊        | 39603/225000 [85:39:17<380:27:50,  7.39s/it]
 18%|█▊        | 39604/225000 [85:39:21<320:14:18,  6.22s/it]
 18%|█▊        | 39605/225000 [85:39:28<333:03:38,  6.47s/it]
 18%|█▊        | 39606/225000 [85:39:33<310:23:39,  6.03s/it]
 18%|█▊        | 39607/225000 [85:39:38<298:01:16,  5.79s/it]
 18%|█▊        | 39608/225000 [85:39:44<295:48:53,  5.74s/it]
 18%|█▊        | 39609/225000 [85:39:49<291:59:35,  5.67s/it]
 18%|█▊        | 39610/225000 [85:40:02<395:55:56,  7.69s/it]
 18%|█▊        | 39611/225000 [85:40:09<381:34:49,  7.41s/it]
 18%|█▊        | 39612/225000 [85:40:13<340:27:07,  6.61s/it]
 18%|█▊        | 39613/225000 [85:40:19<319:32:56,  6.21s/it]
 18%|█▊        | 39614/225000 [85:40:24<304:06:14,  5.91s/it]
 18%|█▊        | 39615/225000 [85:40:29<296:46:22,  5.76s/it]
 18%|█▊        | 39616/225000 [85:40:34<288:11:23,  5.60s/it]
 18%|█▊        | 39617/225000 [85:40:40<288:30:44,  5.60s/it]
 18%|█▊        | 39618/225000 [85:40:51<376:07:03,  7.30s/it]
 18%|█▊        | 39619/225000 [85:41:02<425:17:54,  8.26s/it]
 18%|█▊        | 39620/225000 [85:41:05<351:16:41,  6.82s/it]
 18%|█▊        | 39621/225000 [85:41:13<368:23:44,  7.15s/it]
 18%|█▊        | 39622/225000 [85:41:21<382:56:06,  7.44s/it]
 18%|█▊        | 39623/225000 [85:41:27<363:20:37,  7.06s/it]
 18%|█▊        | 39624/225000 [85:41:32<318:44:15,  6.19s/it]
 18%|█▊        | 39625/225000 [85:41:38<317:03:45,  6.16s/it]
 18%|█▊        | 39626/225000 [85:41:43<309:31:56,  6.01s/it]
 18%|█▊        | 39627/225000 [85:41:49<303:24:42,  5.89s/it]
 18%|█▊        | 39628/225000 [85:41:59<360:14:43,  7.00s/it]
 18%|█▊        | 39629/225000 [85:42:04<340:14:29,  6.61s/it]
 18%|█▊        | 39630/225000 [85:42:10<319:42:56,  6.21s/it]
 18%|█▊        | 39631/225000 [85:42:19<374:59:02,  7.28s/it]
 18%|█▊        | 39632/225000 [85:42:25<351:39:57,  6.83s/it]
 18%|█▊        | 39633/225000 [85:42:31<342:03:18,  6.64s/it]
 18%|█▊        | 39634/225000 [85:42:37<321:04:24,  6.24s/it]
 18%|█▊        | 39635/225000 [85:42:49<410:56:48,  7.98s/it]
 18%|█▊        | 39636/225000 [85:42:55<388:38:08,  7.55s/it]
 18%|█▊        | 39637/225000 [85:43:00<353:01:28,  6.86s/it]
 18%|█▊        | 39638/225000 [85:43:08<359:12:56,  6.98s/it]
 18%|█▊        | 39639/225000 [85:43:12<322:44:37,  6.27s/it]
 18%|█▊        | 39640/225000 [85:43:18<308:27:13,  5.99s/it]
 18%|█▊        | 39641/225000 [85:43:23<299:46:06,  5.82s/it]
 18%|█▊        | 39642/225000 [85:43:29<298:59:54,  5.81s/it]
 18%|█▊        | 39643/225000 [85:43:34<293:05:11,  5.69s/it]
 18%|█▊        | 39644/225000 [85:43:40<293:02:02,  5.69s/it]
 18%|█▊        | 39645/225000 [85:43:53<413:50:12,  8.04s/it]
 18%|█▊        | 39646/225000 [85:43:59<377:33:16,  7.33s/it]
 18%|█▊        | 39647/225000 [85:44:04<335:20:58,  6.51s/it]
 18%|█▊        | 39648/225000 [85:44:09<309:56:55,  6.02s/it]
 18%|█▊        | 39649/225000 [85:44:20<390:55:52,  7.59s/it]
 18%|█▊        | 39650/225000 [85:44:25<360:04:25,  6.99s/it]
                                                             
{'loss': '0.6842', 'grad_norm': '0.6002', 'learning_rate': '4.769e-05', 'epoch': '0.1762', 'num_input_tokens_seen': 145048416, 'train_runtime': '3.087e+05', 'train_tokens_per_second': '469.9'}

 18%|█▊        | 39650/225000 [85:44:25<360:04:25,  6.99s/it]
 18%|█▊        | 39651/225000 [85:44:31<330:05:43,  6.41s/it]
 18%|█▊        | 39652/225000 [85:44:36<310:50:12,  6.04s/it]
 18%|█▊        | 39653/225000 [85:44:40<290:19:20,  5.64s/it]
 18%|█▊        | 39654/225000 [85:44:45<277:21:58,  5.39s/it]
 18%|█▊        | 39655/225000 [85:44:51<277:05:45,  5.38s/it]
 18%|█▊        | 39656/225000 [85:45:02<376:02:08,  7.30s/it]
 18%|█▊        | 39657/225000 [85:45:10<383:04:07,  7.44s/it]
 18%|█▊        | 39658/225000 [85:45:15<337:53:32,  6.56s/it]
 18%|█▊        | 39659/225000 [85:45:25<390:44:53,  7.59s/it]
 18%|█▊        | 39660/225000 [85:45:30<357:35:06,  6.95s/it]
 18%|█▊        | 39661/225000 [85:45:34<312:01:00,  6.06s/it]
 18%|█▊        | 39662/225000 [85:45:40<316:31:15,  6.15s/it]
 18%|█▊        | 39663/225000 [85:45:51<384:27:19,  7.47s/it]
 18%|█▊        | 39664/225000 [85:45:59<386:07:41,  7.50s/it]
 18%|█▊        | 39665/225000 [85:46:04<358:06:22,  6.96s/it]
 18%|█▊        | 39666/225000 [85:46:11<352:26:54,  6.85s/it]
 18%|█▊        | 39667/225000 [85:46:20<383:14:25,  7.44s/it]
 18%|█▊        | 39668/225000 [85:46:27<382:24:25,  7.43s/it]
 18%|█▊        | 39669/225000 [85:46:32<342:58:53,  6.66s/it]
 18%|█▊        | 39670/225000 [85:46:41<383:16:24,  7.45s/it]
 18%|█▊        | 39671/225000 [85:46:46<338:15:20,  6.57s/it]
 18%|█▊        | 39672/225000 [85:46:52<332:50:33,  6.47s/it]
 18%|█▊        | 39673/225000 [85:47:00<360:11:26,  7.00s/it]
 18%|█▊        | 39674/225000 [85:47:06<347:39:37,  6.75s/it]
 18%|█▊        | 39675/225000 [85:47:11<317:04:50,  6.16s/it]
 18%|█▊        | 39676/225000 [85:47:16<302:35:51,  5.88s/it]
 18%|█▊        | 39677/225000 [85:47:22<293:43:25,  5.71s/it]
 18%|█▊        | 39678/225000 [85:47:27<284:52:24,  5.53s/it]
 18%|█▊        | 39679/225000 [85:47:32<282:53:07,  5.50s/it]
 18%|█▊        | 39680/225000 [85:47:43<370:46:52,  7.20s/it]
 18%|█▊        | 39681/225000 [85:47:52<384:55:28,  7.48s/it]
 18%|█▊        | 39682/225000 [85:47:57<348:57:17,  6.78s/it]
 18%|█▊        | 39683/225000 [85:48:05<368:41:09,  7.16s/it]
 18%|█▊        | 39684/225000 [85:48:10<333:09:21,  6.47s/it]
 18%|█▊        | 39685/225000 [85:48:14<298:35:11,  5.80s/it]
 18%|█▊        | 39686/225000 [85:48:20<310:37:04,  6.03s/it]
 18%|█▊        | 39687/225000 [85:48:30<360:25:20,  7.00s/it]
 18%|█▊        | 39688/225000 [85:48:34<320:09:12,  6.22s/it]
 18%|█▊        | 39689/225000 [85:48:45<399:52:27,  7.77s/it]
 18%|█▊        | 39690/225000 [85:48:51<359:18:26,  6.98s/it]
 18%|█▊        | 39691/225000 [85:48:56<328:58:55,  6.39s/it]
 18%|█▊        | 39692/225000 [85:49:01<312:47:21,  6.08s/it]
 18%|█▊        | 39693/225000 [85:49:06<300:38:48,  5.84s/it]
 18%|█▊        | 39694/225000 [85:49:11<288:27:36,  5.60s/it]
 18%|█▊        | 39695/225000 [85:49:25<409:16:47,  7.95s/it]
 18%|█▊        | 39696/225000 [85:49:29<354:15:59,  6.88s/it]
 18%|█▊        | 39697/225000 [85:49:37<370:04:19,  7.19s/it]
 18%|█▊        | 39698/225000 [85:49:44<361:41:02,  7.03s/it]
 18%|█▊        | 39699/225000 [85:49:50<352:03:13,  6.84s/it]
 18%|█▊        | 39700/225000 [85:49:56<334:52:12,  6.51s/it]
                                                             
{'loss': '0.6924', 'grad_norm': '0.6262', 'learning_rate': '4.769e-05', 'epoch': '0.1764', 'num_input_tokens_seen': 145197504, 'train_runtime': '3.09e+05', 'train_tokens_per_second': '469.9'}

 18%|█▊        | 39700/225000 [85:49:56<334:52:12,  6.51s/it]
 18%|█▊        | 39701/225000 [85:50:05<382:01:09,  7.42s/it]
 18%|█▊        | 39702/225000 [85:50:09<325:16:49,  6.32s/it]
 18%|█▊        | 39703/225000 [85:50:15<313:51:57,  6.10s/it]
 18%|█▊        | 39704/225000 [85:50:25<374:16:12,  7.27s/it]
 18%|█▊        | 39705/225000 [85:50:29<323:42:56,  6.29s/it]
 18%|█▊        | 39706/225000 [85:50:39<389:23:01,  7.57s/it]
 18%|█▊        | 39707/225000 [85:50:43<336:31:13,  6.54s/it]
 18%|█▊        | 39708/225000 [85:50:48<310:21:49,  6.03s/it]
 18%|█▊        | 39709/225000 [85:50:53<292:13:21,  5.68s/it]
 18%|█▊        | 39710/225000 [85:51:05<392:26:54,  7.62s/it]
 18%|█▊        | 39711/225000 [85:51:10<352:04:55,  6.84s/it]
 18%|█▊        | 39712/225000 [85:51:15<313:09:17,  6.08s/it]
 18%|█▊        | 39713/225000 [85:51:19<288:06:03,  5.60s/it]
 18%|█▊        | 39714/225000 [85:51:24<278:15:58,  5.41s/it]
 18%|█▊        | 39715/225000 [85:51:34<349:10:04,  6.78s/it]
 18%|█▊        | 39716/225000 [85:51:43<382:33:54,  7.43s/it]
 18%|█▊        | 39717/225000 [85:51:48<353:45:52,  6.87s/it]
 18%|█▊        | 39718/225000 [85:51:54<325:59:01,  6.33s/it]
 18%|█▊        | 39719/225000 [85:51:58<299:22:29,  5.82s/it]
 18%|█▊        | 39720/225000 [85:52:04<295:03:48,  5.73s/it]
 18%|█▊        | 39721/225000 [85:52:16<393:06:39,  7.64s/it]
 18%|█▊        | 39722/225000 [85:52:20<335:30:42,  6.52s/it]
 18%|█▊        | 39723/225000 [85:52:25<317:45:38,  6.17s/it]
 18%|█▊        | 39724/225000 [85:52:38<426:12:01,  8.28s/it]
 18%|█▊        | 39725/225000 [85:52:43<377:17:48,  7.33s/it]
 18%|█▊        | 39726/225000 [85:52:49<349:20:23,  6.79s/it]
 18%|█▊        | 39727/225000 [85:52:54<316:02:38,  6.14s/it]
 18%|█▊        | 39728/225000 [85:53:01<340:05:44,  6.61s/it]
 18%|█▊        | 39729/225000 [85:53:09<364:05:47,  7.07s/it]
 18%|█▊        | 39730/225000 [85:53:15<344:43:55,  6.70s/it]
 18%|█▊        | 39731/225000 [85:53:20<308:14:36,  5.99s/it]
 18%|█▊        | 39732/225000 [85:53:29<362:17:46,  7.04s/it]
 18%|█▊        | 39733/225000 [85:53:35<339:38:18,  6.60s/it]
 18%|█▊        | 39734/225000 [85:53:40<327:41:38,  6.37s/it]
 18%|█▊        | 39735/225000 [85:53:47<332:47:13,  6.47s/it]
 18%|█▊        | 39736/225000 [85:53:52<306:12:22,  5.95s/it]
 18%|█▊        | 39737/225000 [85:53:57<294:43:30,  5.73s/it]
 18%|█▊        | 39738/225000 [85:54:02<286:31:17,  5.57s/it]
 18%|█▊        | 39739/225000 [85:54:10<327:25:34,  6.36s/it]
 18%|█▊        | 39740/225000 [85:54:24<430:19:54,  8.36s/it]
 18%|█▊        | 39741/225000 [85:54:28<366:53:16,  7.13s/it]
 18%|█▊        | 39742/225000 [85:54:33<339:11:04,  6.59s/it]
 18%|█▊        | 39743/225000 [85:54:39<324:52:44,  6.31s/it]
 18%|█▊        | 39744/225000 [85:54:50<392:59:51,  7.64s/it]
 18%|█▊        | 39745/225000 [85:54:54<348:45:06,  6.78s/it]
 18%|█▊        | 39746/225000 [85:54:59<318:35:23,  6.19s/it]
 18%|█▊        | 39747/225000 [85:55:06<329:20:54,  6.40s/it]
 18%|█▊        | 39748/225000 [85:55:19<437:21:44,  8.50s/it]
 18%|█▊        | 39749/225000 [85:55:23<364:33:41,  7.08s/it]
 18%|█▊        | 39750/225000 [85:55:33<414:38:06,  8.06s/it]
                                                             
{'loss': '0.6882', 'grad_norm': '0.6641', 'learning_rate': '4.768e-05', 'epoch': '0.1767', 'num_input_tokens_seen': 145346112, 'train_runtime': '3.093e+05', 'train_tokens_per_second': '469.9'}

 18%|█▊        | 39750/225000 [85:55:33<414:38:06,  8.06s/it]
 18%|█▊        | 39751/225000 [85:55:40<392:09:43,  7.62s/it]
 18%|█▊        | 39752/225000 [85:55:44<340:15:48,  6.61s/it]
 18%|█▊        | 39753/225000 [85:55:52<361:32:25,  7.03s/it]
 18%|█▊        | 39754/225000 [85:55:57<325:50:14,  6.33s/it]
 18%|█▊        | 39755/225000 [85:56:01<295:38:22,  5.75s/it]
 18%|█▊        | 39756/225000 [85:56:13<388:59:13,  7.56s/it]
 18%|█▊        | 39757/225000 [85:56:20<383:47:48,  7.46s/it]
 18%|█▊        | 39758/225000 [85:56:25<332:42:49,  6.47s/it]
 18%|█▊        | 39759/225000 [85:56:29<297:56:42,  5.79s/it]
 18%|█▊        | 39760/225000 [85:56:40<382:45:35,  7.44s/it]
 18%|█▊        | 39761/225000 [85:56:44<326:25:39,  6.34s/it]
 18%|█▊        | 39762/225000 [85:56:52<358:30:19,  6.97s/it]
 18%|█▊        | 39763/225000 [85:56:56<311:58:30,  6.06s/it]
 18%|█▊        | 39764/225000 [85:57:06<362:44:03,  7.05s/it]
 18%|█▊        | 39765/225000 [85:57:11<335:55:17,  6.53s/it]
 18%|█▊        | 39766/225000 [85:57:21<394:54:50,  7.68s/it]
 18%|█▊        | 39767/225000 [85:57:25<326:48:43,  6.35s/it]
 18%|█▊        | 39768/225000 [85:57:34<374:40:41,  7.28s/it]
 18%|█▊        | 39769/225000 [85:57:40<349:47:11,  6.80s/it]
 18%|█▊        | 39770/225000 [85:57:44<308:49:34,  6.00s/it]
 18%|█▊        | 39771/225000 [85:57:51<329:18:44,  6.40s/it]
 18%|█▊        | 39772/225000 [85:57:55<290:50:55,  5.65s/it]
 18%|█▊        | 39773/225000 [85:58:06<368:33:03,  7.16s/it]
 18%|█▊        | 39774/225000 [85:58:09<313:57:39,  6.10s/it]
 18%|█▊        | 39775/225000 [85:58:17<341:54:10,  6.65s/it]
 18%|█▊        | 39776/225000 [85:58:22<314:38:20,  6.12s/it]
 18%|█▊        | 39777/225000 [85:58:27<294:42:58,  5.73s/it]
 18%|█▊        | 39778/225000 [85:58:32<284:24:06,  5.53s/it]
 18%|█▊        | 39779/225000 [85:58:41<335:16:46,  6.52s/it]
 18%|█▊        | 39780/225000 [85:58:52<403:37:08,  7.84s/it]
 18%|█▊        | 39781/225000 [85:58:55<339:19:47,  6.60s/it]
 18%|█▊        | 39782/225000 [85:59:04<366:51:00,  7.13s/it]
 18%|█▊        | 39783/225000 [85:59:12<384:20:55,  7.47s/it]
 18%|█▊        | 39784/225000 [85:59:17<350:58:16,  6.82s/it]
 18%|█▊        | 39785/225000 [85:59:22<321:21:22,  6.25s/it]
 18%|█▊        | 39786/225000 [85:59:27<304:21:03,  5.92s/it]
 18%|█▊        | 39787/225000 [85:59:33<299:11:04,  5.82s/it]
 18%|█▊        | 39788/225000 [85:59:38<291:39:05,  5.67s/it]
 18%|█▊        | 39789/225000 [85:59:43<282:00:40,  5.48s/it]
 18%|█▊        | 39790/225000 [85:59:52<326:26:51,  6.35s/it]
 18%|█▊        | 39791/225000 [86:00:03<397:02:54,  7.72s/it]
 18%|█▊        | 39792/225000 [86:00:08<353:03:19,  6.86s/it]
 18%|█▊        | 39793/225000 [86:00:12<317:13:19,  6.17s/it]
 18%|█▊        | 39794/225000 [86:00:17<291:23:45,  5.66s/it]
 18%|█▊        | 39795/225000 [86:00:26<355:33:15,  6.91s/it]
 18%|█▊        | 39796/225000 [86:00:34<371:11:26,  7.22s/it]
 18%|█▊        | 39797/225000 [86:00:39<324:55:27,  6.32s/it]
 18%|█▊        | 39798/225000 [86:00:49<391:36:35,  7.61s/it]
 18%|█▊        | 39799/225000 [86:00:54<345:30:54,  6.72s/it]
 18%|█▊        | 39800/225000 [86:01:00<332:25:12,  6.46s/it]
                                                             
{'loss': '0.6898', 'grad_norm': '0.7394', 'learning_rate': '4.767e-05', 'epoch': '0.1769', 'num_input_tokens_seen': 145493608, 'train_runtime': '3.097e+05', 'train_tokens_per_second': '469.8'}

 18%|█▊        | 39800/225000 [86:01:00<332:25:12,  6.46s/it]
 18%|█▊        | 39801/225000 [86:01:07<342:43:08,  6.66s/it]
 18%|█▊        | 39802/225000 [86:01:14<347:10:02,  6.75s/it]
 18%|█▊        | 39803/225000 [86:01:20<340:59:06,  6.63s/it]
 18%|█▊        | 39804/225000 [86:01:29<382:56:53,  7.44s/it]
 18%|█▊        | 39805/225000 [86:01:33<320:50:38,  6.24s/it]
 18%|█▊        | 39806/225000 [86:01:40<335:27:47,  6.52s/it]
 18%|█▊        | 39807/225000 [86:01:45<306:05:52,  5.95s/it]
 18%|█▊        | 39808/225000 [86:01:56<384:40:01,  7.48s/it]
 18%|█▊        | 39809/225000 [86:02:00<337:11:01,  6.55s/it]
 18%|█▊        | 39810/225000 [86:02:05<305:54:27,  5.95s/it]
 18%|█▊        | 39811/225000 [86:02:09<280:31:54,  5.45s/it]
 18%|█▊        | 39812/225000 [86:02:13<262:07:01,  5.10s/it]
 18%|█▊        | 39813/225000 [86:02:24<346:44:32,  6.74s/it]
 18%|█▊        | 39814/225000 [86:02:30<336:46:47,  6.55s/it]
 18%|█▊        | 39815/225000 [86:02:38<355:47:51,  6.92s/it]
 18%|█▊        | 39816/225000 [86:02:44<350:32:16,  6.81s/it]
 18%|█▊        | 39817/225000 [86:02:50<333:36:26,  6.49s/it]
 18%|█▊        | 39818/225000 [86:02:55<306:36:35,  5.96s/it]
 18%|█▊        | 39819/225000 [86:03:05<379:53:44,  7.39s/it]
 18%|█▊        | 39820/225000 [86:03:10<341:32:46,  6.64s/it]
 18%|█▊        | 39821/225000 [86:03:18<359:26:33,  6.99s/it]
 18%|█▊        | 39822/225000 [86:03:22<314:42:54,  6.12s/it]
 18%|█▊        | 39823/225000 [86:03:26<285:57:06,  5.56s/it]
 18%|█▊        | 39824/225000 [86:03:37<370:06:23,  7.20s/it]
 18%|█▊        | 39825/225000 [86:03:41<318:32:48,  6.19s/it]
 18%|█▊        | 39826/225000 [86:03:46<299:38:09,  5.83s/it]
 18%|█▊        | 39827/225000 [86:03:52<292:15:43,  5.68s/it]
 18%|█▊        | 39828/225000 [86:03:57<294:20:40,  5.72s/it]
 18%|█▊        | 39829/225000 [86:04:03<292:37:09,  5.69s/it]
 18%|█▊        | 39830/225000 [86:04:13<352:45:05,  6.86s/it]
 18%|█▊        | 39831/225000 [86:04:23<413:01:15,  8.03s/it]
 18%|█▊        | 39832/225000 [86:04:27<347:06:24,  6.75s/it]
 18%|█▊        | 39833/225000 [86:04:35<357:14:48,  6.95s/it]
 18%|█▊        | 39834/225000 [86:04:42<363:41:53,  7.07s/it]
 18%|█▊        | 39835/225000 [86:04:50<381:20:37,  7.41s/it]
 18%|█▊        | 39836/225000 [86:04:54<326:35:29,  6.35s/it]
 18%|█▊        | 39837/225000 [86:04:59<303:23:46,  5.90s/it]
 18%|█▊        | 39838/225000 [86:05:05<313:42:35,  6.10s/it]
 18%|█▊        | 39839/225000 [86:05:17<401:37:55,  7.81s/it]
 18%|█▊        | 39840/225000 [86:05:21<337:49:17,  6.57s/it]
 18%|█▊        | 39841/225000 [86:05:28<352:35:31,  6.86s/it]
 18%|█▊        | 39842/225000 [86:05:32<303:09:24,  5.89s/it]
 18%|█▊        | 39843/225000 [86:05:42<362:02:43,  7.04s/it]
 18%|█▊        | 39844/225000 [86:05:45<307:31:16,  5.98s/it]
 18%|█▊        | 39845/225000 [86:05:50<290:02:00,  5.64s/it]
 18%|█▊        | 39846/225000 [86:05:57<308:57:13,  6.01s/it]
 18%|█▊        | 39847/225000 [86:06:08<384:58:48,  7.49s/it]
 18%|█▊        | 39848/225000 [86:06:12<329:02:25,  6.40s/it]
 18%|█▊        | 39849/225000 [86:06:20<350:28:24,  6.81s/it]
 18%|█▊        | 39850/225000 [86:06:27<364:33:09,  7.09s/it]
                                                             
{'loss': '0.6938', 'grad_norm': '0.695', 'learning_rate': '4.766e-05', 'epoch': '0.1771', 'num_input_tokens_seen': 145640272, 'train_runtime': '3.1e+05', 'train_tokens_per_second': '469.8'}

 18%|█▊        | 39850/225000 [86:06:27<364:33:09,  7.09s/it]
 18%|█▊        | 39851/225000 [86:06:32<328:32:29,  6.39s/it]
 18%|█▊        | 39852/225000 [86:06:36<292:30:00,  5.69s/it]
 18%|█▊        | 39853/225000 [86:06:41<284:20:59,  5.53s/it]
 18%|█▊        | 39854/225000 [86:06:47<281:06:25,  5.47s/it]
 18%|█▊        | 39855/225000 [86:06:57<357:17:01,  6.95s/it]
 18%|█▊        | 39856/225000 [86:07:07<405:47:22,  7.89s/it]
 18%|█▊        | 39857/225000 [86:07:12<355:31:53,  6.91s/it]
 18%|█▊        | 39858/225000 [86:07:19<367:26:56,  7.14s/it]
 18%|█▊        | 39859/225000 [86:07:25<342:42:12,  6.66s/it]
 18%|█▊        | 39860/225000 [86:07:30<322:59:41,  6.28s/it]
 18%|█▊        | 39861/225000 [86:07:36<306:09:28,  5.95s/it]
 18%|█▊        | 39862/225000 [86:07:41<303:31:19,  5.90s/it]
 18%|█▊        | 39863/225000 [86:07:47<298:11:27,  5.80s/it]
 18%|█▊        | 39864/225000 [86:07:54<321:01:07,  6.24s/it]
 18%|█▊        | 39865/225000 [86:08:08<432:13:24,  8.40s/it]
 18%|█▊        | 39866/225000 [86:08:13<390:56:50,  7.60s/it]
 18%|█▊        | 39867/225000 [86:08:23<419:55:28,  8.17s/it]
 18%|█▊        | 39868/225000 [86:08:28<373:48:04,  7.27s/it]
 18%|█▊        | 39869/225000 [86:08:33<344:34:04,  6.70s/it]
 18%|█▊        | 39870/225000 [86:08:39<320:08:08,  6.23s/it]
 18%|█▊        | 39871/225000 [86:08:44<310:47:06,  6.04s/it]
 18%|█▊        | 39872/225000 [86:08:49<299:58:40,  5.83s/it]
 18%|█▊        | 39873/225000 [86:09:03<414:03:11,  8.05s/it]
 18%|█▊        | 39874/225000 [86:09:11<413:29:53,  8.04s/it]
 18%|█▊        | 39875/225000 [86:09:19<416:08:27,  8.09s/it]
 18%|█▊        | 39876/225000 [86:09:23<349:55:39,  6.80s/it]
 18%|█▊        | 39877/225000 [86:09:30<360:50:01,  7.02s/it]
 18%|█▊        | 39878/225000 [86:09:37<360:03:24,  7.00s/it]
 18%|█▊        | 39879/225000 [86:09:42<326:53:29,  6.36s/it]
 18%|█▊        | 39880/225000 [86:09:47<308:40:43,  6.00s/it]
 18%|█▊        | 39881/225000 [86:09:54<319:27:52,  6.21s/it]
 18%|█▊        | 39882/225000 [86:10:05<395:37:45,  7.69s/it]
 18%|█▊        | 39883/225000 [86:10:11<364:13:38,  7.08s/it]
 18%|█▊        | 39884/225000 [86:10:16<334:28:37,  6.50s/it]
 18%|█▊        | 39885/225000 [86:10:25<367:40:36,  7.15s/it]
 18%|█▊        | 39886/225000 [86:10:28<313:44:58,  6.10s/it]
 18%|█▊        | 39887/225000 [86:10:39<383:30:16,  7.46s/it]
 18%|█▊        | 39888/225000 [86:10:43<326:50:03,  6.36s/it]
 18%|█▊        | 39889/225000 [86:10:50<335:23:15,  6.52s/it]
 18%|█▊        | 39890/225000 [86:10:57<345:12:35,  6.71s/it]
 18%|█▊        | 39891/225000 [86:11:02<316:28:46,  6.15s/it]
 18%|█▊        | 39892/225000 [86:11:06<296:08:24,  5.76s/it]
 18%|█▊        | 39893/225000 [86:11:11<285:31:25,  5.55s/it]
 18%|█▊        | 39894/225000 [86:11:17<284:50:57,  5.54s/it]
 18%|█▊        | 39895/225000 [86:11:30<394:31:27,  7.67s/it]
 18%|█▊        | 39896/225000 [86:11:35<361:28:00,  7.03s/it]
 18%|█▊        | 39897/225000 [86:11:41<349:00:04,  6.79s/it]
 18%|█▊        | 39898/225000 [86:11:47<335:00:21,  6.52s/it]
 18%|█▊        | 39899/225000 [86:11:53<316:13:45,  6.15s/it]
 18%|█▊        | 39900/225000 [86:11:59<317:25:15,  6.17s/it]
                                                             
{'loss': '0.6779', 'grad_norm': '0.6948', 'learning_rate': '4.765e-05', 'epoch': '0.1773', 'num_input_tokens_seen': 145788760, 'train_runtime': '3.103e+05', 'train_tokens_per_second': '469.8'}

 18%|█▊        | 39900/225000 [86:11:59<317:25:15,  6.17s/it]
 18%|█▊        | 39901/225000 [86:12:04<303:42:48,  5.91s/it]
 18%|█▊        | 39902/225000 [86:12:10<301:22:00,  5.86s/it]
 18%|█▊        | 39903/225000 [86:12:15<295:53:47,  5.75s/it]
 18%|█▊        | 39904/225000 [86:12:21<301:10:58,  5.86s/it]
 18%|█▊        | 39905/225000 [86:12:34<404:25:01,  7.87s/it]
 18%|█▊        | 39906/225000 [86:12:39<367:57:55,  7.16s/it]
 18%|█▊        | 39907/225000 [86:12:48<387:47:46,  7.54s/it]
 18%|█▊        | 39908/225000 [86:12:53<347:25:32,  6.76s/it]
 18%|█▊        | 39909/225000 [86:12:58<320:02:01,  6.22s/it]
 18%|█▊        | 39910/225000 [86:13:03<302:57:39,  5.89s/it]
 18%|█▊        | 39911/225000 [86:13:16<414:43:23,  8.07s/it]
 18%|█▊        | 39912/225000 [86:13:21<373:32:38,  7.27s/it]
 18%|█▊        | 39913/225000 [86:13:29<373:16:31,  7.26s/it]
 18%|█▊        | 39914/225000 [86:13:34<345:07:53,  6.71s/it]
 18%|█▊        | 39915/225000 [86:13:39<321:20:50,  6.25s/it]
 18%|█▊        | 39916/225000 [86:13:44<292:27:25,  5.69s/it]
 18%|█▊        | 39917/225000 [86:13:55<373:01:55,  7.26s/it]
 18%|█▊        | 39918/225000 [86:14:02<369:17:57,  7.18s/it]
 18%|█▊        | 39919/225000 [86:14:06<320:11:41,  6.23s/it]
 18%|█▊        | 39920/225000 [86:14:16<387:50:13,  7.54s/it]
 18%|█▊        | 39921/225000 [86:14:22<353:01:23,  6.87s/it]
 18%|█▊        | 39922/225000 [86:14:29<357:07:50,  6.95s/it]
 18%|█▊        | 39923/225000 [86:14:33<314:19:12,  6.11s/it]
 18%|█▊        | 39924/225000 [86:14:37<290:54:29,  5.66s/it]
 18%|█▊        | 39925/225000 [86:14:42<278:05:40,  5.41s/it]
 18%|█▊        | 39926/225000 [86:14:55<388:31:53,  7.56s/it]
 18%|█▊        | 39927/225000 [86:15:02<378:57:09,  7.37s/it]
 18%|█▊        | 39928/225000 [86:15:06<337:58:05,  6.57s/it]
 18%|█▊        | 39929/225000 [86:15:14<359:39:36,  7.00s/it]
 18%|█▊        | 39930/225000 [86:15:19<328:15:04,  6.39s/it]
 18%|█▊        | 39931/225000 [86:15:28<359:45:57,  7.00s/it]
 18%|█▊        | 39932/225000 [86:15:35<358:39:24,  6.98s/it]
 18%|█▊        | 39933/225000 [86:15:40<325:03:34,  6.32s/it]
 18%|█▊        | 39934/225000 [86:15:44<300:24:45,  5.84s/it]
 18%|█▊        | 39935/225000 [86:15:48<274:52:32,  5.35s/it]
 18%|█▊        | 39936/225000 [86:15:53<269:10:12,  5.24s/it]
 18%|█▊        | 39937/225000 [86:16:05<365:11:16,  7.10s/it]
 18%|█▊        | 39938/225000 [86:16:11<344:24:15,  6.70s/it]
 18%|█▊        | 39939/225000 [86:16:15<312:15:06,  6.07s/it]
 18%|█▊        | 39940/225000 [86:16:20<290:51:09,  5.66s/it]
 18%|█▊        | 39941/225000 [86:16:25<282:13:13,  5.49s/it]
 18%|█▊        | 39942/225000 [86:16:30<278:18:36,  5.41s/it]
 18%|█▊        | 39943/225000 [86:16:43<396:10:33,  7.71s/it]
 18%|█▊        | 39944/225000 [86:16:50<376:52:37,  7.33s/it]
 18%|█▊        | 39945/225000 [86:16:57<367:55:49,  7.16s/it]
 18%|█▊        | 39946/225000 [86:17:03<364:07:21,  7.08s/it]
 18%|█▊        | 39947/225000 [86:17:08<325:30:17,  6.33s/it]
 18%|█▊        | 39948/225000 [86:17:13<302:42:53,  5.89s/it]
 18%|█▊        | 39949/225000 [86:17:24<388:24:54,  7.56s/it]
 18%|█▊        | 39950/225000 [86:17:29<339:05:08,  6.60s/it]
                                                             
{'loss': '0.6796', 'grad_norm': '0.7173', 'learning_rate': '4.765e-05', 'epoch': '0.1776', 'num_input_tokens_seen': 145936672, 'train_runtime': '3.106e+05', 'train_tokens_per_second': '469.8'}

 18%|█▊        | 39950/225000 [86:17:29<339:05:08,  6.60s/it]
 18%|█▊        | 39951/225000 [86:17:34<316:30:48,  6.16s/it]
 18%|█▊        | 39952/225000 [86:17:39<306:40:04,  5.97s/it]
 18%|█▊        | 39953/225000 [86:17:45<307:02:59,  5.97s/it]
 18%|█▊        | 39954/225000 [86:17:53<340:03:27,  6.62s/it]
 18%|█▊        | 39955/225000 [86:18:07<444:12:36,  8.64s/it]
 18%|█▊        | 39956/225000 [86:18:11<378:36:25,  7.37s/it]
 18%|█▊        | 39957/225000 [86:18:15<330:44:06,  6.43s/it]
 18%|█▊        | 39958/225000 [86:18:25<381:54:18,  7.43s/it]
 18%|█▊        | 39959/225000 [86:18:30<342:03:46,  6.65s/it]
 18%|█▊        | 39960/225000 [86:18:35<310:58:33,  6.05s/it]
 18%|█▊        | 39961/225000 [86:18:39<288:04:17,  5.60s/it]
 18%|█▊        | 39962/225000 [86:18:53<412:23:27,  8.02s/it]
 18%|█▊        | 39963/225000 [86:18:58<364:53:22,  7.10s/it]
 18%|█▊        | 39964/225000 [86:19:02<322:26:20,  6.27s/it]
 18%|█▊        | 39965/225000 [86:19:07<301:28:14,  5.87s/it]
 18%|█▊        | 39966/225000 [86:19:16<352:33:42,  6.86s/it]
 18%|█▊        | 39967/225000 [86:19:25<380:38:09,  7.41s/it]
 18%|█▊        | 39968/225000 [86:19:30<342:09:41,  6.66s/it]
 18%|█▊        | 39969/225000 [86:19:35<312:46:49,  6.09s/it]
 18%|█▊        | 39970/225000 [86:19:46<385:55:14,  7.51s/it]
 18%|█▊        | 39971/225000 [86:19:49<328:32:15,  6.39s/it]
 18%|█▊        | 39972/225000 [86:19:58<356:43:13,  6.94s/it]
 18%|█▊        | 39973/225000 [86:20:03<339:59:33,  6.62s/it]
 18%|█▊        | 39974/225000 [86:20:08<305:48:02,  5.95s/it]
 18%|█▊        | 39975/225000 [86:20:18<365:36:11,  7.11s/it]
 18%|█▊        | 39976/225000 [86:20:22<328:10:29,  6.39s/it]
 18%|█▊        | 39977/225000 [86:20:27<296:55:35,  5.78s/it]
 18%|█▊        | 39978/225000 [86:20:36<357:20:09,  6.95s/it]
 18%|█▊        | 39979/225000 [86:20:43<351:40:28,  6.84s/it]
 18%|█▊        | 39980/225000 [86:20:51<375:41:04,  7.31s/it]
 18%|█▊        | 39981/225000 [86:20:57<343:16:00,  6.68s/it]
 18%|█▊        | 39982/225000 [86:21:01<315:40:01,  6.14s/it]
 18%|█▊        | 39983/225000 [86:21:06<291:22:40,  5.67s/it]
 18%|█▊        | 39984/225000 [86:21:11<285:47:14,  5.56s/it]
 18%|█▊        | 39985/225000 [86:21:20<328:40:29,  6.40s/it]
 18%|█▊        | 39986/225000 [86:21:32<418:30:14,  8.14s/it]
 18%|█▊        | 39987/225000 [86:21:36<356:43:17,  6.94s/it]
 18%|█▊        | 39988/225000 [86:21:44<377:45:38,  7.35s/it]
 18%|█▊        | 39989/225000 [86:21:49<340:12:16,  6.62s/it]
 18%|█▊        | 39990/225000 [86:21:57<362:56:16,  7.06s/it]
 18%|█▊        | 39991/225000 [86:22:03<334:13:13,  6.50s/it]
 18%|█▊        | 39992/225000 [86:22:09<337:11:24,  6.56s/it]
 18%|█▊        | 39993/225000 [86:22:13<298:01:12,  5.80s/it]
 18%|█▊        | 39994/225000 [86:22:19<305:02:33,  5.94s/it]
 18%|█▊        | 39995/225000 [86:22:30<371:44:48,  7.23s/it]
 18%|█▊        | 39996/225000 [86:22:34<326:07:26,  6.35s/it]
 18%|█▊        | 39997/225000 [86:22:41<339:26:50,  6.61s/it]
 18%|█▊        | 39998/225000 [86:22:48<343:50:09,  6.69s/it]
 18%|█▊        | 39999/225000 [86:22:52<304:43:29,  5.93s/it]
 18%|█▊        | 40000/225000 [86:23:04<390:53:06,  7.61s/it]
                                                             
{'loss': '0.6851', 'grad_norm': '0.6731', 'learning_rate': '4.764e-05', 'epoch': '0.1778', 'num_input_tokens_seen': 146083776, 'train_runtime': '3.11e+05', 'train_tokens_per_second': '469.7'}

 18%|█▊        | 40000/225000 [86:23:04<390:53:06,  7.61s/it]
 18%|█▊        | 40001/225000 [86:23:10<366:43:27,  7.14s/it]
 18%|█▊        | 40002/225000 [86:23:14<321:51:57,  6.26s/it]
 18%|█▊        | 40003/225000 [86:23:18<286:56:53,  5.58s/it]
 18%|█▊        | 40004/225000 [86:23:30<388:42:00,  7.56s/it]
 18%|█▊        | 40005/225000 [86:23:38<386:24:49,  7.52s/it]
 18%|█▊        | 40006/225000 [86:23:42<333:53:28,  6.50s/it]
 18%|█▊        | 40007/225000 [86:23:51<379:53:03,  7.39s/it]
 18%|█▊        | 40008/225000 [86:23:57<356:32:00,  6.94s/it]
 18%|█▊        | 40009/225000 [86:24:01<315:38:41,  6.14s/it]
 18%|█▊        | 40010/225000 [86:24:11<367:23:58,  7.15s/it]
 18%|█▊        | 40011/225000 [86:24:16<328:57:58,  6.40s/it]
 18%|█▊        | 40012/225000 [86:24:21<307:20:05,  5.98s/it]
 18%|█▊        | 40013/225000 [86:24:26<302:09:43,  5.88s/it]
 18%|█▊        | 40014/225000 [86:24:34<327:17:44,  6.37s/it]
 18%|█▊        | 40015/225000 [86:24:47<429:48:25,  8.36s/it]
 18%|█▊        | 40016/225000 [86:24:53<397:20:44,  7.73s/it]
 18%|█▊        | 40017/225000 [86:24:59<375:33:27,  7.31s/it]
 18%|█▊        | 40018/225000 [86:25:08<399:13:30,  7.77s/it]
 18%|█▊        | 40019/225000 [86:25:14<376:08:07,  7.32s/it]
 18%|█▊        | 40020/225000 [86:25:22<374:48:34,  7.29s/it]
 18%|█▊        | 40021/225000 [86:25:27<340:00:29,  6.62s/it]
 18%|█▊        | 40022/225000 [86:25:37<391:22:57,  7.62s/it]
 18%|█▊        | 40023/225000 [86:25:41<344:06:09,  6.70s/it]
 18%|█▊        | 40024/225000 [86:25:51<389:48:37,  7.59s/it]
 18%|█▊        | 40025/225000 [86:25:55<334:51:16,  6.52s/it]
 18%|█▊        | 40026/225000 [86:26:05<392:53:36,  7.65s/it]
 18%|█▊        | 40027/225000 [86:26:09<332:50:34,  6.48s/it]
 18%|█▊        | 40028/225000 [86:26:19<392:08:52,  7.63s/it]
 18%|█▊        | 40029/225000 [86:26:23<333:09:17,  6.48s/it]
 18%|█▊        | 40030/225000 [86:26:28<309:51:50,  6.03s/it]
 18%|█▊        | 40031/225000 [86:26:33<290:50:27,  5.66s/it]
 18%|█▊        | 40032/225000 [86:26:38<275:51:51,  5.37s/it]
 18%|█▊        | 40033/225000 [86:26:51<396:58:35,  7.73s/it]
 18%|█▊        | 40034/225000 [86:26:56<358:28:31,  6.98s/it]
 18%|█▊        | 40035/225000 [86:27:01<335:48:36,  6.54s/it]
 18%|█▊        | 40036/225000 [86:27:07<313:13:04,  6.10s/it]
 18%|█▊        | 40037/225000 [86:27:12<302:39:39,  5.89s/it]
 18%|█▊        | 40038/225000 [86:27:26<423:01:32,  8.23s/it]
 18%|█▊        | 40039/225000 [86:27:30<357:41:44,  6.96s/it]
 18%|█▊        | 40040/225000 [86:27:34<316:58:52,  6.17s/it]
 18%|█▊        | 40041/225000 [86:27:45<386:39:27,  7.53s/it]
 18%|█▊        | 40042/225000 [86:27:50<358:20:59,  6.97s/it]
 18%|█▊        | 40043/225000 [86:27:55<322:13:24,  6.27s/it]
 18%|█▊        | 40044/225000 [86:28:03<347:00:45,  6.75s/it]
 18%|█▊        | 40045/225000 [86:28:08<320:08:58,  6.23s/it]
 18%|█▊        | 40046/225000 [86:28:13<304:46:23,  5.93s/it]
 18%|█▊        | 40047/225000 [86:28:18<287:09:14,  5.59s/it]
 18%|█▊        | 40048/225000 [86:28:23<286:01:41,  5.57s/it]
 18%|█▊        | 40049/225000 [86:28:34<366:20:56,  7.13s/it]
 18%|█▊        | 40050/225000 [86:28:44<408:43:47,  7.96s/it]
                                                             
{'loss': '0.678', 'grad_norm': '0.636', 'learning_rate': '4.763e-05', 'epoch': '0.178', 'num_input_tokens_seen': 146233440, 'train_runtime': '3.113e+05', 'train_tokens_per_second': '469.7'}

 18%|█▊        | 40050/225000 [86:28:44<408:43:47,  7.96s/it]
 18%|█▊        | 40051/225000 [86:28:50<380:25:45,  7.40s/it]
 18%|█▊        | 40052/225000 [86:28:57<364:13:27,  7.09s/it]
 18%|█▊        | 40053/225000 [86:29:01<323:39:14,  6.30s/it]
 18%|█▊        | 40054/225000 [86:29:12<393:04:46,  7.65s/it]
 18%|█▊        | 40055/225000 [86:29:16<343:33:35,  6.69s/it]
 18%|█▊        | 40056/225000 [86:29:21<312:16:59,  6.08s/it]
 18%|█▊        | 40057/225000 [86:29:27<313:46:13,  6.11s/it]
 18%|█▊        | 40058/225000 [86:29:38<392:06:55,  7.63s/it]
 18%|█▊        | 40059/225000 [86:29:42<337:45:34,  6.57s/it]
 18%|█▊        | 40060/225000 [86:29:48<319:23:30,  6.22s/it]
 18%|█▊        | 40061/225000 [86:29:59<390:41:41,  7.61s/it]
 18%|█▊        | 40062/225000 [86:30:03<345:10:07,  6.72s/it]
 18%|█▊        | 40063/225000 [86:30:08<318:20:01,  6.20s/it]
 18%|█▊        | 40064/225000 [86:30:17<357:42:25,  6.96s/it]
 18%|█▊        | 40065/225000 [86:30:24<357:20:20,  6.96s/it]
 18%|█▊        | 40066/225000 [86:30:30<336:59:10,  6.56s/it]
 18%|█▊        | 40067/225000 [86:30:35<315:31:20,  6.14s/it]
 18%|█▊        | 40068/225000 [86:30:39<293:24:05,  5.71s/it]
 18%|█▊        | 40069/225000 [86:30:45<283:14:32,  5.51s/it]
 18%|█▊        | 40070/225000 [86:30:58<399:30:13,  7.78s/it]
 18%|█▊        | 40071/225000 [86:31:04<376:21:01,  7.33s/it]
 18%|█▊        | 40072/225000 [86:31:12<381:56:41,  7.44s/it]
 18%|█▊        | 40073/225000 [86:31:17<356:12:49,  6.93s/it]
 18%|█▊        | 40074/225000 [86:31:21<312:45:07,  6.09s/it]
 18%|█▊        | 40075/225000 [86:31:27<309:33:15,  6.03s/it]
 18%|█▊        | 40076/225000 [86:31:36<350:38:58,  6.83s/it]
 18%|█▊        | 40077/225000 [86:31:42<343:32:28,  6.69s/it]
 18%|█▊        | 40078/225000 [86:31:49<349:36:59,  6.81s/it]
 18%|█▊        | 40079/225000 [86:31:54<322:49:17,  6.28s/it]
 18%|█▊        | 40080/225000 [86:31:59<301:49:30,  5.88s/it]
 18%|█▊        | 40081/225000 [86:32:05<289:47:09,  5.64s/it]
 18%|█▊        | 40082/225000 [86:32:10<294:04:31,  5.73s/it]
 18%|█▊        | 40083/225000 [86:32:26<451:59:28,  8.80s/it]
 18%|█▊        | 40084/225000 [86:32:31<392:03:26,  7.63s/it]
 18%|█▊        | 40085/225000 [86:32:39<393:01:04,  7.65s/it]
 18%|█▊        | 40086/225000 [86:32:44<345:23:07,  6.72s/it]
 18%|█▊        | 40087/225000 [86:32:48<306:22:02,  5.96s/it]
 18%|█▊        | 40088/225000 [86:33:01<415:13:07,  8.08s/it]
 18%|█▊        | 40089/225000 [86:33:05<348:47:05,  6.79s/it]
 18%|█▊        | 40090/225000 [86:33:10<332:04:57,  6.47s/it]
 18%|█▊        | 40091/225000 [86:33:16<314:01:11,  6.11s/it]
 18%|█▊        | 40092/225000 [86:33:21<299:25:55,  5.83s/it]
 18%|█▊        | 40093/225000 [86:33:26<287:45:53,  5.60s/it]
 18%|█▊        | 40094/225000 [86:33:39<411:09:04,  8.00s/it]
 18%|█▊        | 40095/225000 [86:33:45<379:51:24,  7.40s/it]
 18%|█▊        | 40096/225000 [86:33:54<399:42:48,  7.78s/it]
 18%|█▊        | 40097/225000 [86:33:59<354:06:07,  6.89s/it]
 18%|█▊        | 40098/225000 [86:34:03<312:27:16,  6.08s/it]
 18%|█▊        | 40099/225000 [86:34:08<297:09:23,  5.79s/it]
 18%|█▊        | 40100/225000 [86:34:14<291:47:35,  5.68s/it]
                                                             
{'loss': '0.6943', 'grad_norm': '0.7258', 'learning_rate': '4.762e-05', 'epoch': '0.1782', 'num_input_tokens_seen': 146382336, 'train_runtime': '3.117e+05', 'train_tokens_per_second': '469.7'}

 18%|█▊        | 40100/225000 [86:34:14<291:47:35,  5.68s/it]
 18%|█▊        | 40101/225000 [86:34:26<390:18:28,  7.60s/it]
 18%|█▊        | 40102/225000 [86:34:31<350:27:58,  6.82s/it]
 18%|█▊        | 40103/225000 [86:34:36<332:12:00,  6.47s/it]
 18%|█▊        | 40104/225000 [86:34:42<314:07:40,  6.12s/it]
 18%|█▊        | 40105/225000 [86:34:47<310:09:27,  6.04s/it]
 18%|█▊        | 40106/225000 [86:34:53<299:03:01,  5.82s/it]
 18%|█▊        | 40107/225000 [86:34:58<293:40:09,  5.72s/it]
 18%|█▊        | 40108/225000 [86:35:09<375:23:40,  7.31s/it]
 18%|█▊        | 40109/225000 [86:35:19<411:05:25,  8.00s/it]
 18%|█▊        | 40110/225000 [86:35:23<352:28:51,  6.86s/it]
 18%|█▊        | 40111/225000 [86:35:31<362:32:00,  7.06s/it]
 18%|█▊        | 40112/225000 [86:35:39<384:22:29,  7.48s/it]
 18%|█▊        | 40113/225000 [86:35:48<398:14:11,  7.75s/it]
 18%|█▊        | 40114/225000 [86:35:52<341:05:07,  6.64s/it]
 18%|█▊        | 40115/225000 [86:36:00<370:07:12,  7.21s/it]
 18%|█▊        | 40116/225000 [86:36:04<319:54:20,  6.23s/it]
 18%|█▊        | 40117/225000 [86:36:15<388:46:38,  7.57s/it]
 18%|█▊        | 40118/225000 [86:36:19<344:08:44,  6.70s/it]
 18%|█▊        | 40119/225000 [86:36:28<369:40:02,  7.20s/it]
 18%|█▊        | 40120/225000 [86:36:32<319:11:45,  6.22s/it]
 18%|█▊        | 40121/225000 [86:36:40<355:39:54,  6.93s/it]
 18%|█▊        | 40122/225000 [86:36:45<320:18:56,  6.24s/it]
 18%|█▊        | 40123/225000 [86:36:50<301:55:54,  5.88s/it]
 18%|█▊        | 40124/225000 [86:36:54<278:10:54,  5.42s/it]
 18%|█▊        | 40125/225000 [86:37:02<317:01:28,  6.17s/it]
 18%|█▊        | 40126/225000 [86:37:14<404:17:59,  7.87s/it]
 18%|█▊        | 40127/225000 [86:37:18<348:46:25,  6.79s/it]
 18%|█▊        | 40128/225000 [86:37:28<398:39:15,  7.76s/it]
 18%|█▊        | 40129/225000 [86:37:36<389:08:38,  7.58s/it]
 18%|█▊        | 40130/225000 [86:37:40<334:23:07,  6.51s/it]
 18%|█▊        | 40131/225000 [86:37:47<350:34:35,  6.83s/it]
 18%|█▊        | 40132/225000 [86:37:55<367:21:55,  7.15s/it]
 18%|█▊        | 40133/225000 [86:37:59<318:53:15,  6.21s/it]
 18%|█▊        | 40134/225000 [86:38:04<296:51:15,  5.78s/it]
 18%|█▊        | 40135/225000 [86:38:09<281:53:04,  5.49s/it]
 18%|█▊        | 40136/225000 [86:38:14<280:13:59,  5.46s/it]
 18%|█▊        | 40137/225000 [86:38:19<270:51:14,  5.27s/it]
 18%|█▊        | 40138/225000 [86:38:28<334:45:49,  6.52s/it]
 18%|█▊        | 40139/225000 [86:38:38<390:29:41,  7.60s/it]
 18%|█▊        | 40140/225000 [86:38:46<387:57:49,  7.56s/it]
 18%|█▊        | 40141/225000 [86:38:50<329:00:19,  6.41s/it]
 18%|█▊        | 40142/225000 [86:38:55<317:12:04,  6.18s/it]
 18%|█▊        | 40143/225000 [86:39:06<383:01:40,  7.46s/it]
 18%|█▊        | 40144/225000 [86:39:10<336:48:34,  6.56s/it]
 18%|█▊        | 40145/225000 [86:39:15<317:09:28,  6.18s/it]
 18%|█▊        | 40146/225000 [86:39:28<416:09:13,  8.10s/it]
 18%|█▊        | 40147/225000 [86:39:32<348:17:26,  6.78s/it]
 18%|█▊        | 40148/225000 [86:39:37<318:41:01,  6.21s/it]
 18%|█▊        | 40149/225000 [86:39:48<395:25:18,  7.70s/it]
 18%|█▊        | 40150/225000 [86:39:52<340:42:15,  6.64s/it]
                                                             
{'loss': '0.6879', 'grad_norm': '0.7341', 'learning_rate': '4.762e-05', 'epoch': '0.1784', 'num_input_tokens_seen': 146530576, 'train_runtime': '3.12e+05', 'train_tokens_per_second': '469.7'}

 18%|█▊        | 40150/225000 [86:39:52<340:42:15,  6.64s/it]
 18%|█▊        | 40151/225000 [86:39:56<301:02:35,  5.86s/it]
 18%|█▊        | 40152/225000 [86:40:07<374:52:19,  7.30s/it]
 18%|█▊        | 40153/225000 [86:40:11<326:12:31,  6.35s/it]
 18%|█▊        | 40154/225000 [86:40:15<294:15:09,  5.73s/it]
 18%|█▊        | 40155/225000 [86:40:27<384:02:26,  7.48s/it]
 18%|█▊        | 40156/225000 [86:40:32<352:08:43,  6.86s/it]
 18%|█▊        | 40157/225000 [86:40:38<334:33:00,  6.52s/it]
 18%|█▊        | 40158/225000 [86:40:43<319:33:23,  6.22s/it]
 18%|█▊        | 40159/225000 [86:40:49<306:14:39,  5.96s/it]
 18%|█▊        | 40160/225000 [86:40:54<294:18:14,  5.73s/it]
 18%|█▊        | 40161/225000 [86:40:59<292:18:47,  5.69s/it]
 18%|█▊        | 40162/225000 [86:41:12<396:15:34,  7.72s/it]
 18%|█▊        | 40163/225000 [86:41:19<384:05:47,  7.48s/it]
 18%|█▊        | 40164/225000 [86:41:28<409:31:30,  7.98s/it]
 18%|█▊        | 40165/225000 [86:41:32<353:15:30,  6.88s/it]
 18%|█▊        | 40166/225000 [86:41:37<318:47:15,  6.21s/it]
 18%|█▊        | 40167/225000 [86:41:41<292:50:35,  5.70s/it]
 18%|█▊        | 40168/225000 [86:41:52<374:41:17,  7.30s/it]
 18%|█▊        | 40169/225000 [86:41:58<353:43:13,  6.89s/it]
 18%|█▊        | 40170/225000 [86:42:04<335:51:51,  6.54s/it]
 18%|█▊        | 40171/225000 [86:42:10<319:35:13,  6.22s/it]
 18%|█▊        | 40172/225000 [86:42:15<311:59:19,  6.08s/it]
 18%|█▊        | 40173/225000 [86:42:21<306:40:54,  5.97s/it]
 18%|█▊        | 40174/225000 [86:42:27<303:17:36,  5.91s/it]
 18%|█▊        | 40175/225000 [86:42:32<299:11:04,  5.83s/it]
 18%|█▊        | 40176/225000 [86:42:38<298:11:34,  5.81s/it]
 18%|█▊        | 40177/225000 [86:42:44<298:04:01,  5.81s/it]
 18%|█▊        | 40178/225000 [86:42:57<407:04:13,  7.93s/it]
 18%|█▊        | 40179/225000 [86:43:05<403:34:47,  7.86s/it]
 18%|█▊        | 40180/225000 [86:43:08<338:31:40,  6.59s/it]
 18%|█▊        | 40181/225000 [86:43:18<393:46:29,  7.67s/it]
 18%|█▊        | 40182/225000 [86:43:22<330:33:35,  6.44s/it]
 18%|█▊        | 40183/225000 [86:43:29<338:25:40,  6.59s/it]
 18%|█▊        | 40184/225000 [86:43:38<384:24:47,  7.49s/it]
 18%|█▊        | 40185/225000 [86:43:45<366:01:37,  7.13s/it]
 18%|█▊        | 40186/225000 [86:43:53<389:53:04,  7.59s/it]
 18%|█▊        | 40187/225000 [86:43:57<332:47:03,  6.48s/it]
 18%|█▊        | 40188/225000 [86:44:06<370:32:33,  7.22s/it]
 18%|█▊        | 40189/225000 [86:44:10<322:38:31,  6.28s/it]
 18%|█▊        | 40190/225000 [86:44:16<308:40:33,  6.01s/it]
 18%|█▊        | 40191/225000 [86:44:25<360:53:05,  7.03s/it]
 18%|█▊        | 40192/225000 [86:44:34<392:29:10,  7.65s/it]
 18%|█▊        | 40193/225000 [86:44:40<360:53:56,  7.03s/it]
 18%|█▊        | 40194/225000 [86:44:45<336:58:26,  6.56s/it]
 18%|█▊        | 40195/225000 [86:44:51<321:04:35,  6.25s/it]
 18%|█▊        | 40196/225000 [86:44:56<304:13:49,  5.93s/it]
 18%|█▊        | 40197/225000 [86:45:01<290:23:01,  5.66s/it]
 18%|█▊        | 40198/225000 [86:45:07<290:54:02,  5.67s/it]
 18%|█▊        | 40199/225000 [86:45:12<290:18:20,  5.66s/it]
 18%|█▊        | 40200/225000 [86:45:18<285:51:35,  5.57s/it]
                                                             
{'loss': '0.6796', 'grad_norm': '0.7327', 'learning_rate': '4.761e-05', 'epoch': '0.1787', 'num_input_tokens_seen': 146677792, 'train_runtime': '3.123e+05', 'train_tokens_per_second': '469.6'}

 18%|█▊        | 40200/225000 [86:45:18<285:51:35,  5.57s/it]
 18%|█▊        | 40201/225000 [86:45:31<409:49:05,  7.98s/it]
 18%|█▊        | 40202/225000 [86:45:38<385:13:44,  7.50s/it]
 18%|█▊        | 40203/225000 [86:45:42<333:33:31,  6.50s/it]
 18%|█▊        | 40204/225000 [86:45:53<410:57:34,  8.01s/it]
 18%|█▊        | 40205/225000 [86:45:58<357:21:07,  6.96s/it]
 18%|█▊        | 40206/225000 [86:46:08<398:47:14,  7.77s/it]
 18%|█▊        | 40207/225000 [86:46:13<355:46:16,  6.93s/it]
 18%|█▊        | 40208/225000 [86:46:22<401:34:00,  7.82s/it]
 18%|█▊        | 40209/225000 [86:46:27<357:02:03,  6.96s/it]
 18%|█▊        | 40210/225000 [86:46:33<333:18:58,  6.49s/it]
 18%|█▊        | 40211/225000 [86:46:43<387:18:41,  7.55s/it]
 18%|█▊        | 40212/225000 [86:46:47<334:17:54,  6.51s/it]
 18%|█▊        | 40213/225000 [86:46:54<338:29:18,  6.59s/it]
 18%|█▊        | 40214/225000 [86:47:01<350:00:00,  6.82s/it]
 18%|█▊        | 40215/225000 [86:47:07<334:18:36,  6.51s/it]
 18%|█▊        | 40216/225000 [86:47:14<345:39:11,  6.73s/it]
 18%|█▊        | 40217/225000 [86:47:22<359:49:20,  7.01s/it]
 18%|█▊        | 40218/225000 [86:47:27<330:55:28,  6.45s/it]
 18%|█▊        | 40219/225000 [86:47:35<355:48:49,  6.93s/it]
 18%|█▊        | 40220/225000 [86:47:39<312:18:42,  6.08s/it]
 18%|█▊        | 40221/225000 [86:47:44<302:22:36,  5.89s/it]
 18%|█▊        | 40222/225000 [86:47:50<296:24:51,  5.77s/it]
 18%|█▊        | 40223/225000 [86:47:55<290:27:36,  5.66s/it]
 18%|█▊        | 40224/225000 [86:48:02<307:08:56,  5.98s/it]
 18%|█▊        | 40225/225000 [86:48:17<437:12:35,  8.52s/it]
 18%|█▊        | 40226/225000 [86:48:24<419:45:16,  8.18s/it]
 18%|█▊        | 40227/225000 [86:48:28<356:05:01,  6.94s/it]
 18%|█▊        | 40228/225000 [86:48:37<393:06:35,  7.66s/it]
 18%|█▊        | 40229/225000 [86:48:42<340:42:52,  6.64s/it]
 18%|█▊        | 40230/225000 [86:48:52<395:56:59,  7.71s/it]
 18%|█▊        | 40231/225000 [86:48:56<334:39:51,  6.52s/it]
 18%|█▊        | 40232/225000 [86:49:00<309:06:22,  6.02s/it]
 18%|█▊        | 40233/225000 [86:49:11<373:43:07,  7.28s/it]
 18%|█▊        | 40234/225000 [86:49:16<342:31:02,  6.67s/it]
 18%|█▊        | 40235/225000 [86:49:20<310:02:52,  6.04s/it]
 18%|█▊        | 40236/225000 [86:49:25<288:14:38,  5.62s/it]
 18%|█▊        | 40237/225000 [86:49:36<370:44:32,  7.22s/it]
 18%|█▊        | 40238/225000 [86:49:46<408:31:49,  7.96s/it]
 18%|█▊        | 40239/225000 [86:49:53<402:15:53,  7.84s/it]
 18%|█▊        | 40240/225000 [86:50:00<388:33:20,  7.57s/it]
 18%|█▊        | 40241/225000 [86:50:05<352:09:47,  6.86s/it]
 18%|█▊        | 40242/225000 [86:50:09<302:26:23,  5.89s/it]
 18%|█▊        | 40243/225000 [86:50:19<364:07:27,  7.09s/it]
 18%|█▊        | 40244/225000 [86:50:23<313:25:18,  6.11s/it]
 18%|█▊        | 40245/225000 [86:50:32<363:23:19,  7.08s/it]
 18%|█▊        | 40246/225000 [86:50:36<317:57:14,  6.20s/it]
 18%|█▊        | 40247/225000 [86:50:46<372:36:01,  7.26s/it]
 18%|█▊        | 40248/225000 [86:50:51<331:02:21,  6.45s/it]
 18%|█▊        | 40249/225000 [86:50:55<301:32:04,  5.88s/it]
 18%|█▊        | 40250/225000 [86:51:00<288:03:51,  5.61s/it]
                                                             
{'loss': '0.6711', 'grad_norm': '0.7341', 'learning_rate': '4.76e-05', 'epoch': '0.1789', 'num_input_tokens_seen': 146827944, 'train_runtime': '3.127e+05', 'train_tokens_per_second': '469.6'}

 18%|█▊        | 40250/225000 [86:51:00<288:03:51,  5.61s/it]
 18%|█▊        | 40251/225000 [86:51:09<340:47:51,  6.64s/it]
 18%|█▊        | 40252/225000 [86:51:18<380:56:24,  7.42s/it]
 18%|█▊        | 40253/225000 [86:51:26<385:12:34,  7.51s/it]
 18%|█▊        | 40254/225000 [86:51:30<331:14:02,  6.45s/it]
 18%|█▊        | 40255/225000 [86:51:36<327:03:27,  6.37s/it]
 18%|█▊        | 40256/225000 [86:51:45<368:25:22,  7.18s/it]
 18%|█▊        | 40257/225000 [86:51:50<331:55:21,  6.47s/it]
 18%|█▊        | 40258/225000 [86:51:55<301:20:12,  5.87s/it]
 18%|█▊        | 40259/225000 [86:52:00<291:15:23,  5.68s/it]
 18%|█▊        | 40260/225000 [86:52:13<413:39:42,  8.06s/it]
 18%|█▊        | 40261/225000 [86:52:18<366:58:03,  7.15s/it]
 18%|█▊        | 40262/225000 [86:52:26<379:01:58,  7.39s/it]
 18%|█▊        | 40263/225000 [86:52:31<341:11:32,  6.65s/it]
 18%|█▊        | 40264/225000 [86:52:40<366:06:37,  7.13s/it]
 18%|█▊        | 40265/225000 [86:52:46<352:12:54,  6.86s/it]
 18%|█▊        | 40266/225000 [86:52:53<356:37:37,  6.95s/it]
 18%|█▊        | 40267/225000 [86:52:58<332:43:59,  6.48s/it]
 18%|█▊        | 40268/225000 [86:53:04<323:09:24,  6.30s/it]
 18%|█▊        | 40269/225000 [86:53:14<369:45:14,  7.21s/it]
 18%|█▊        | 40270/225000 [86:53:17<316:44:20,  6.17s/it]
 18%|█▊        | 40271/225000 [86:53:26<349:32:32,  6.81s/it]
 18%|█▊        | 40272/225000 [86:53:30<315:38:51,  6.15s/it]
 18%|█▊        | 40273/225000 [86:53:35<293:44:59,  5.72s/it]
 18%|█▊        | 40274/225000 [86:53:49<415:01:48,  8.09s/it]
 18%|█▊        | 40275/225000 [86:53:53<363:14:41,  7.08s/it]
 18%|█▊        | 40276/225000 [86:53:59<340:46:12,  6.64s/it]
 18%|█▊        | 40277/225000 [86:54:05<325:05:20,  6.34s/it]
 18%|█▊        | 40278/225000 [86:54:10<310:39:52,  6.05s/it]
 18%|█▊        | 40279/225000 [86:54:16<307:36:28,  5.99s/it]
 18%|█▊        | 40280/225000 [86:54:22<302:50:38,  5.90s/it]
 18%|█▊        | 40281/225000 [86:54:27<298:02:09,  5.81s/it]
 18%|█▊        | 40282/225000 [86:54:33<295:35:09,  5.76s/it]
 18%|█▊        | 40283/225000 [86:54:46<410:55:43,  8.01s/it]
 18%|█▊        | 40284/225000 [86:54:54<405:30:03,  7.90s/it]
 18%|█▊        | 40285/225000 [86:54:58<352:16:25,  6.87s/it]
 18%|█▊        | 40286/225000 [86:55:03<329:07:36,  6.41s/it]
 18%|█▊        | 40287/225000 [86:55:16<419:13:16,  8.17s/it]
 18%|█▊        | 40288/225000 [86:55:23<400:11:47,  7.80s/it]
 18%|█▊        | 40289/225000 [86:55:26<334:54:57,  6.53s/it]
 18%|█▊        | 40290/225000 [86:55:31<308:02:54,  6.00s/it]
 18%|█▊        | 40291/225000 [86:55:36<286:01:14,  5.57s/it]
 18%|█▊        | 40292/225000 [86:55:41<281:29:56,  5.49s/it]
 18%|█▊        | 40293/225000 [86:55:50<341:09:13,  6.65s/it]
 18%|█▊        | 40294/225000 [86:56:01<397:09:55,  7.74s/it]
 18%|█▊        | 40295/225000 [86:56:04<337:23:07,  6.58s/it]
 18%|█▊        | 40296/225000 [86:56:15<397:41:09,  7.75s/it]
 18%|█▊        | 40297/225000 [86:56:19<341:02:00,  6.65s/it]
 18%|█▊        | 40298/225000 [86:56:24<323:27:14,  6.30s/it]
 18%|█▊        | 40299/225000 [86:56:34<379:19:20,  7.39s/it]
 18%|█▊        | 40300/225000 [86:56:39<336:44:29,  6.56s/it]
                                                             
{'loss': '0.6768', 'grad_norm': '0.7118', 'learning_rate': '4.759e-05', 'epoch': '0.1791', 'num_input_tokens_seen': 146975944, 'train_runtime': '3.13e+05', 'train_tokens_per_second': '469.6'}

 18%|█▊        | 40300/225000 [86:56:39<336:44:29,  6.56s/it]
 18%|█▊        | 40301/225000 [86:56:46<337:12:53,  6.57s/it]
 18%|█▊        | 40302/225000 [86:56:56<391:04:55,  7.62s/it]
 18%|█▊        | 40303/225000 [86:57:01<349:14:49,  6.81s/it]
 18%|█▊        | 40304/225000 [86:57:06<325:55:17,  6.35s/it]
 18%|█▊        | 40305/225000 [86:57:10<295:39:34,  5.76s/it]
 18%|█▊        | 40306/225000 [86:57:15<286:16:29,  5.58s/it]
 18%|█▊        | 40307/225000 [86:57:25<351:21:01,  6.85s/it]
 18%|█▊        | 40308/225000 [86:57:36<413:14:38,  8.05s/it]
 18%|█▊        | 40309/225000 [86:57:40<356:11:54,  6.94s/it]
 18%|█▊        | 40310/225000 [86:57:46<329:03:02,  6.41s/it]
 18%|█▊        | 40311/225000 [86:57:51<313:49:24,  6.12s/it]
 18%|█▊        | 40312/225000 [86:58:05<427:27:52,  8.33s/it]
 18%|█▊        | 40313/225000 [86:58:09<372:51:04,  7.27s/it]
 18%|█▊        | 40314/225000 [86:58:15<346:12:00,  6.75s/it]
 18%|█▊        | 40315/225000 [86:58:20<319:56:55,  6.24s/it]
 18%|█▊        | 40316/225000 [86:58:32<413:26:55,  8.06s/it]
 18%|█▊        | 40317/225000 [86:58:39<397:09:59,  7.74s/it]
 18%|█▊        | 40318/225000 [86:58:44<355:31:02,  6.93s/it]
 18%|█▊        | 40319/225000 [86:58:49<320:08:21,  6.24s/it]
 18%|█▊        | 40320/225000 [86:58:54<300:06:03,  5.85s/it]
 18%|█▊        | 40321/225000 [86:59:05<377:41:18,  7.36s/it]
 18%|█▊        | 40322/225000 [86:59:13<397:37:37,  7.75s/it]
 18%|█▊        | 40323/225000 [86:59:17<336:06:01,  6.55s/it]
 18%|█▊        | 40324/225000 [86:59:27<385:10:30,  7.51s/it]
 18%|█▊        | 40325/225000 [86:59:30<324:55:44,  6.33s/it]
 18%|█▊        | 40326/225000 [86:59:39<351:26:27,  6.85s/it]
 18%|█▊        | 40327/225000 [86:59:43<314:12:14,  6.13s/it]
 18%|█▊        | 40328/225000 [86:59:51<346:15:21,  6.75s/it]
 18%|█▊        | 40329/225000 [86:59:56<309:59:30,  6.04s/it]
 18%|█▊        | 40330/225000 [87:00:05<361:05:58,  7.04s/it]
 18%|█▊        | 40331/225000 [87:00:12<361:33:27,  7.05s/it]
 18%|█▊        | 40332/225000 [87:00:17<323:20:19,  6.30s/it]
 18%|█▊        | 40333/225000 [87:00:21<292:05:45,  5.69s/it]
 18%|█▊        | 40334/225000 [87:00:33<384:44:40,  7.50s/it]
 18%|█▊        | 40335/225000 [87:00:39<374:53:41,  7.31s/it]
 18%|█▊        | 40336/225000 [87:00:45<348:04:11,  6.79s/it]
 18%|█▊        | 40337/225000 [87:00:49<312:47:23,  6.10s/it]
 18%|█▊        | 40338/225000 [87:00:55<297:38:30,  5.80s/it]
 18%|█▊        | 40339/225000 [87:00:59<282:47:26,  5.51s/it]
 18%|█▊        | 40340/225000 [87:01:13<402:07:50,  7.84s/it]
 18%|█▊        | 40341/225000 [87:01:17<351:56:36,  6.86s/it]
 18%|█▊        | 40342/225000 [87:01:22<312:42:56,  6.10s/it]
 18%|█▊        | 40343/225000 [87:01:27<298:49:29,  5.83s/it]
 18%|█▊        | 40344/225000 [87:01:32<293:59:46,  5.73s/it]
 18%|█▊        | 40345/225000 [87:01:38<289:47:03,  5.65s/it]
 18%|█▊        | 40346/225000 [87:01:49<382:27:07,  7.46s/it]
 18%|█▊        | 40347/225000 [87:01:55<358:05:51,  6.98s/it]
 18%|█▊        | 40348/225000 [87:02:01<339:23:11,  6.62s/it]
 18%|█▊        | 40349/225000 [87:02:07<331:05:51,  6.46s/it]
 18%|█▊        | 40350/225000 [87:02:13<326:58:14,  6.37s/it]
                                                             
{'loss': '0.6884', 'grad_norm': '0.6626', 'learning_rate': '4.758e-05', 'epoch': '0.1793', 'num_input_tokens_seen': 147126552, 'train_runtime': '3.133e+05', 'train_tokens_per_second': '469.6'}

 18%|█▊        | 40350/225000 [87:02:13<326:58:14,  6.37s/it]
 18%|█▊        | 40351/225000 [87:02:20<325:04:53,  6.34s/it]
 18%|█▊        | 40352/225000 [87:02:25<316:51:48,  6.18s/it]
 18%|█▊        | 40353/225000 [87:02:31<314:14:58,  6.13s/it]
 18%|█▊        | 40354/225000 [87:02:42<378:46:49,  7.38s/it]
 18%|█▊        | 40355/225000 [87:02:53<432:27:37,  8.43s/it]
 18%|█▊        | 40356/225000 [87:02:57<369:45:07,  7.21s/it]
 18%|█▊        | 40357/225000 [87:03:04<371:47:05,  7.25s/it]
 18%|█▊        | 40358/225000 [87:03:10<351:43:19,  6.86s/it]
 18%|█▊        | 40359/225000 [87:03:15<318:57:16,  6.22s/it]
 18%|█▊        | 40360/225000 [87:03:20<302:18:05,  5.89s/it]
 18%|█▊        | 40361/225000 [87:03:34<421:35:33,  8.22s/it]
 18%|█▊        | 40362/225000 [87:03:39<382:28:31,  7.46s/it]
 18%|█▊        | 40363/225000 [87:03:44<334:13:25,  6.52s/it]
 18%|█▊        | 40364/225000 [87:03:50<333:40:08,  6.51s/it]
 18%|█▊        | 40365/225000 [87:04:02<413:10:41,  8.06s/it]
 18%|█▊        | 40366/225000 [87:04:05<343:57:53,  6.71s/it]
 18%|█▊        | 40367/225000 [87:04:12<347:26:48,  6.77s/it]
 18%|█▊        | 40368/225000 [87:04:16<303:12:18,  5.91s/it]
 18%|█▊        | 40369/225000 [87:04:27<372:20:24,  7.26s/it]
 18%|█▊        | 40370/225000 [87:04:32<343:33:10,  6.70s/it]
 18%|█▊        | 40371/225000 [87:04:36<305:54:32,  5.96s/it]
 18%|█▊        | 40372/225000 [87:04:41<288:33:57,  5.63s/it]
 18%|█▊        | 40373/225000 [87:04:47<292:02:55,  5.69s/it]
 18%|█▊        | 40374/225000 [87:04:53<294:03:31,  5.73s/it]
 18%|█▊        | 40375/225000 [87:04:59<293:23:41,  5.72s/it]
 18%|█▊        | 40376/225000 [87:05:11<394:31:00,  7.69s/it]
 18%|█▊        | 40377/225000 [87:05:19<403:10:20,  7.86s/it]
 18%|█▊        | 40378/225000 [87:05:23<348:04:35,  6.79s/it]
 18%|█▊        | 40379/225000 [87:05:28<312:34:25,  6.10s/it]
 18%|█▊        | 40380/225000 [87:05:33<300:12:18,  5.85s/it]
 18%|█▊        | 40381/225000 [87:05:46<415:32:31,  8.10s/it]
 18%|█▊        | 40382/225000 [87:05:50<346:20:17,  6.75s/it]
 18%|█▊        | 40383/225000 [87:05:59<376:05:43,  7.33s/it]
 18%|█▊        | 40384/225000 [87:06:03<329:40:57,  6.43s/it]
 18%|█▊        | 40385/225000 [87:06:10<344:17:40,  6.71s/it]
 18%|█▊        | 40386/225000 [87:06:18<350:13:32,  6.83s/it]
 18%|█▊        | 40387/225000 [87:06:25<367:05:06,  7.16s/it]
 18%|█▊        | 40388/225000 [87:06:31<336:21:01,  6.56s/it]
 18%|█▊        | 40389/225000 [87:06:36<315:05:16,  6.14s/it]
 18%|█▊        | 40390/225000 [87:06:41<305:07:27,  5.95s/it]
 18%|█▊        | 40391/225000 [87:06:46<291:59:47,  5.69s/it]
 18%|█▊        | 40392/225000 [87:06:52<285:26:17,  5.57s/it]
 18%|█▊        | 40393/225000 [87:07:02<364:20:35,  7.11s/it]
 18%|█▊        | 40394/225000 [87:07:12<402:49:38,  7.86s/it]
 18%|█▊        | 40395/225000 [87:07:17<359:33:32,  7.01s/it]
 18%|█▊        | 40396/225000 [87:07:22<334:52:10,  6.53s/it]
 18%|█▊        | 40397/225000 [87:07:28<317:36:54,  6.19s/it]
 18%|█▊        | 40398/225000 [87:07:41<420:52:55,  8.21s/it]
 18%|█▊        | 40399/225000 [87:07:45<353:13:50,  6.89s/it]
 18%|█▊        | 40400/225000 [87:07:53<379:29:35,  7.40s/it]
                                                             
{'loss': '0.6785', 'grad_norm': '0.7807', 'learning_rate': '4.758e-05', 'epoch': '0.1796', 'num_input_tokens_seen': 147275168, 'train_runtime': '3.137e+05', 'train_tokens_per_second': '469.5'}

 18%|█▊        | 40400/225000 [87:07:53<379:29:35,  7.40s/it]
 18%|█▊        | 40401/225000 [87:07:59<357:05:50,  6.96s/it]
 18%|█▊        | 40402/225000 [87:08:04<328:22:13,  6.40s/it]
 18%|█▊        | 40403/225000 [87:08:10<312:01:07,  6.08s/it]
 18%|█▊        | 40404/225000 [87:08:15<304:18:16,  5.93s/it]
 18%|█▊        | 40405/225000 [87:08:21<299:49:22,  5.85s/it]
 18%|█▊        | 40406/225000 [87:08:27<309:41:57,  6.04s/it]
 18%|█▊        | 40407/225000 [87:08:39<399:06:24,  7.78s/it]
 18%|█▊        | 40408/225000 [87:08:46<380:05:20,  7.41s/it]
 18%|█▊        | 40409/225000 [87:08:53<373:19:24,  7.28s/it]
 18%|█▊        | 40410/225000 [87:09:00<370:54:21,  7.23s/it]
 18%|█▊        | 40411/225000 [87:09:04<331:44:34,  6.47s/it]
 18%|█▊        | 40412/225000 [87:09:15<390:51:02,  7.62s/it]
 18%|█▊        | 40413/225000 [87:09:18<325:21:29,  6.35s/it]
 18%|█▊        | 40414/225000 [87:09:22<287:21:10,  5.60s/it]
 18%|█▊        | 40415/225000 [87:09:32<360:45:48,  7.04s/it]
 18%|█▊        | 40416/225000 [87:09:40<373:10:16,  7.28s/it]
 18%|█▊        | 40417/225000 [87:09:44<324:24:31,  6.33s/it]
 18%|█▊        | 40418/225000 [87:09:49<306:07:53,  5.97s/it]
 18%|█▊        | 40419/225000 [87:09:54<289:29:28,  5.65s/it]
 18%|█▊        | 40420/225000 [87:10:00<282:40:29,  5.51s/it]
 18%|█▊        | 40421/225000 [87:10:08<324:34:06,  6.33s/it]
 18%|█▊        | 40422/225000 [87:10:18<390:23:33,  7.61s/it]
 18%|█▊        | 40423/225000 [87:10:24<361:18:15,  7.05s/it]
 18%|█▊        | 40424/225000 [87:10:30<338:45:51,  6.61s/it]
 18%|█▊        | 40425/225000 [87:10:39<382:42:23,  7.46s/it]
 18%|█▊        | 40426/225000 [87:10:45<353:34:25,  6.90s/it]
 18%|█▊        | 40427/225000 [87:10:52<361:43:37,  7.06s/it]
 18%|█▊        | 40428/225000 [87:10:57<322:31:51,  6.29s/it]
 18%|█▊        | 40429/225000 [87:11:02<315:09:31,  6.15s/it]
 18%|█▊        | 40430/225000 [87:11:11<358:18:52,  6.99s/it]
 18%|█▊        | 40431/225000 [87:11:17<333:19:02,  6.50s/it]
 18%|█▊        | 40432/225000 [87:11:25<358:42:06,  7.00s/it]
 18%|█▊        | 40433/225000 [87:11:32<356:54:44,  6.96s/it]
 18%|█▊        | 40434/225000 [87:11:36<313:47:30,  6.12s/it]
 18%|█▊        | 40435/225000 [87:11:40<287:41:03,  5.61s/it]
 18%|█▊        | 40436/225000 [87:11:50<348:50:37,  6.80s/it]
 18%|█▊        | 40437/225000 [87:11:56<341:01:55,  6.65s/it]
 18%|█▊        | 40438/225000 [87:12:01<306:02:58,  5.97s/it]
 18%|█▊        | 40439/225000 [87:12:05<278:40:15,  5.44s/it]
 18%|█▊        | 40440/225000 [87:12:15<358:31:29,  6.99s/it]
 18%|█▊        | 40441/225000 [87:12:22<352:06:39,  6.87s/it]
 18%|█▊        | 40442/225000 [87:12:26<311:14:32,  6.07s/it]
 18%|█▊        | 40443/225000 [87:12:36<373:59:16,  7.30s/it]
 18%|█▊        | 40444/225000 [87:12:41<339:54:22,  6.63s/it]
 18%|█▊        | 40445/225000 [87:12:49<351:40:46,  6.86s/it]
 18%|█▊        | 40446/225000 [87:12:53<312:16:42,  6.09s/it]
 18%|█▊        | 40447/225000 [87:12:57<282:33:29,  5.51s/it]
 18%|█▊        | 40448/225000 [87:13:06<326:28:09,  6.37s/it]
 18%|█▊        | 40449/225000 [87:13:14<362:31:59,  7.07s/it]
 18%|█▊        | 40450/225000 [87:13:22<370:57:44,  7.24s/it]
                                                             
{'loss': '0.6754', 'grad_norm': '0.6711', 'learning_rate': '4.757e-05', 'epoch': '0.1798', 'num_input_tokens_seen': 147423584, 'train_runtime': '3.14e+05', 'train_tokens_per_second': '469.5'}

 18%|█▊        | 40450/225000 [87:13:22<370:57:44,  7.24s/it]
 18%|█▊        | 40451/225000 [87:13:27<336:12:24,  6.56s/it]
 18%|█▊        | 40452/225000 [87:13:34<345:26:27,  6.74s/it]
 18%|█▊        | 40453/225000 [87:13:43<377:24:46,  7.36s/it]
 18%|█▊        | 40454/225000 [87:13:48<347:31:43,  6.78s/it]
 18%|█▊        | 40455/225000 [87:13:56<366:18:42,  7.15s/it]
 18%|█▊        | 40456/225000 [87:14:01<325:40:30,  6.35s/it]
 18%|█▊        | 40457/225000 [87:14:09<346:56:17,  6.77s/it]
 18%|█▊        | 40458/225000 [87:14:13<316:55:01,  6.18s/it]
 18%|█▊        | 40459/225000 [87:14:19<301:36:41,  5.88s/it]
 18%|█▊        | 40460/225000 [87:14:26<326:42:14,  6.37s/it]
 18%|█▊        | 40461/225000 [87:14:37<401:40:36,  7.84s/it]
 18%|█▊        | 40462/225000 [87:14:41<343:13:37,  6.70s/it]
 18%|█▊        | 40463/225000 [87:14:50<370:06:27,  7.22s/it]
 18%|█▊        | 40464/225000 [87:14:55<334:33:06,  6.53s/it]
 18%|█▊        | 40465/225000 [87:15:00<309:34:29,  6.04s/it]
 18%|█▊        | 40466/225000 [87:15:05<294:31:25,  5.75s/it]
 18%|█▊        | 40467/225000 [87:15:10<292:02:58,  5.70s/it]
 18%|█▊        | 40468/225000 [87:15:16<290:38:56,  5.67s/it]
 18%|█▊        | 40469/225000 [87:15:29<405:07:17,  7.90s/it]
 18%|█▊        | 40470/225000 [87:15:36<387:38:30,  7.56s/it]
 18%|█▊        | 40471/225000 [87:15:43<378:01:34,  7.37s/it]
 18%|█▊        | 40472/225000 [87:15:48<342:02:14,  6.67s/it]
 18%|█▊        | 40473/225000 [87:15:53<324:20:24,  6.33s/it]
 18%|█▊        | 40474/225000 [87:15:58<304:53:49,  5.95s/it]
 18%|█▊        | 40475/225000 [87:16:09<376:30:49,  7.35s/it]
 18%|█▊        | 40476/225000 [87:16:18<399:16:25,  7.79s/it]
 18%|█▊        | 40477/225000 [87:16:22<347:55:13,  6.79s/it]
 18%|█▊        | 40478/225000 [87:16:27<314:42:53,  6.14s/it]
 18%|█▊        | 40479/225000 [87:16:32<294:06:10,  5.74s/it]
 18%|█▊        | 40480/225000 [87:16:41<351:59:22,  6.87s/it]
 18%|█▊        | 40481/225000 [87:16:51<390:32:38,  7.62s/it]
 18%|█▊        | 40482/225000 [87:16:55<340:38:59,  6.65s/it]
 18%|█▊        | 40483/225000 [87:17:00<314:10:57,  6.13s/it]
 18%|█▊        | 40484/225000 [87:17:08<342:34:55,  6.68s/it]
 18%|█▊        | 40485/225000 [87:17:18<389:22:24,  7.60s/it]
 18%|█▊        | 40486/225000 [87:17:26<404:33:00,  7.89s/it]
 18%|█▊        | 40487/225000 [87:17:31<358:19:41,  6.99s/it]
 18%|█▊        | 40488/225000 [87:17:36<330:16:56,  6.44s/it]
 18%|█▊        | 40489/225000 [87:17:42<314:48:43,  6.14s/it]
 18%|█▊        | 40490/225000 [87:17:47<301:46:21,  5.89s/it]
 18%|█▊        | 40491/225000 [87:17:52<288:19:13,  5.63s/it]
 18%|█▊        | 40492/225000 [87:17:57<281:53:39,  5.50s/it]
 18%|█▊        | 40493/225000 [87:18:07<351:19:28,  6.85s/it]
 18%|█▊        | 40494/225000 [87:18:17<396:21:53,  7.73s/it]
 18%|█▊        | 40495/225000 [87:18:22<347:53:22,  6.79s/it]
 18%|█▊        | 40496/225000 [87:18:27<333:52:57,  6.51s/it]
 18%|█▊        | 40497/225000 [87:18:39<409:26:36,  7.99s/it]
 18%|█▊        | 40498/225000 [87:18:44<367:48:48,  7.18s/it]
 18%|█▊        | 40499/225000 [87:18:49<333:28:20,  6.51s/it]
 18%|█▊        | 40500/225000 [87:18:54<311:14:03,  6.07s/it]
                                                             
{'loss': '0.6963', 'grad_norm': '0.8379', 'learning_rate': '4.756e-05', 'epoch': '0.18', 'num_input_tokens_seen': 147572856, 'train_runtime': '3.143e+05', 'train_tokens_per_second': '469.5'}

 18%|█▊        | 40500/225000 [87:18:54<311:14:03,  6.07s/it]
 18%|█▊        | 40501/225000 [87:19:05<381:44:53,  7.45s/it]
 18%|█▊        | 40502/225000 [87:19:09<331:10:28,  6.46s/it]
 18%|█▊        | 40503/225000 [87:19:15<327:13:32,  6.38s/it]
 18%|█▊        | 40504/225000 [87:19:23<351:38:25,  6.86s/it]
 18%|█▊        | 40505/225000 [87:19:27<307:49:40,  6.01s/it]
 18%|█▊        | 40506/225000 [87:19:32<295:05:58,  5.76s/it]
 18%|█▊        | 40507/225000 [87:19:39<314:55:38,  6.15s/it]
 18%|█▊        | 40508/225000 [87:19:50<390:17:16,  7.62s/it]
 18%|█▊        | 40509/225000 [87:19:55<348:18:54,  6.80s/it]
 18%|█▊        | 40510/225000 [87:20:00<314:15:37,  6.13s/it]
 18%|█▊        | 40511/225000 [87:20:14<431:50:21,  8.43s/it]
 18%|█▊        | 40512/225000 [87:20:18<368:25:12,  7.19s/it]
 18%|█▊        | 40513/225000 [87:20:26<377:36:59,  7.37s/it]
 18%|█▊        | 40514/225000 [87:20:30<336:26:29,  6.57s/it]
 18%|█▊        | 40515/225000 [87:20:35<307:00:02,  5.99s/it]
 18%|█▊        | 40516/225000 [87:20:47<395:34:42,  7.72s/it]
 18%|█▊        | 40517/225000 [87:20:51<339:46:02,  6.63s/it]
 18%|█▊        | 40518/225000 [87:20:56<308:03:38,  6.01s/it]
 18%|█▊        | 40519/225000 [87:21:05<362:06:15,  7.07s/it]
 18%|█▊        | 40520/225000 [87:21:11<343:54:06,  6.71s/it]
 18%|█▊        | 40521/225000 [87:21:15<308:33:58,  6.02s/it]
 18%|█▊        | 40522/225000 [87:21:21<298:31:10,  5.83s/it]
 18%|█▊        | 40523/225000 [87:21:26<295:57:09,  5.78s/it]
 18%|█▊        | 40524/225000 [87:21:32<299:37:14,  5.85s/it]
 18%|█▊        | 40525/225000 [87:21:38<302:07:49,  5.90s/it]
 18%|█▊        | 40526/225000 [87:21:52<414:39:35,  8.09s/it]
 18%|█▊        | 40527/225000 [87:21:58<385:34:26,  7.52s/it]
 18%|█▊        | 40528/225000 [87:22:05<377:43:45,  7.37s/it]
 18%|█▊        | 40529/225000 [87:22:09<332:01:54,  6.48s/it]
 18%|█▊        | 40530/225000 [87:22:14<299:57:18,  5.85s/it]
 18%|█▊        | 40531/225000 [87:22:22<339:23:05,  6.62s/it]
 18%|█▊        | 40532/225000 [87:22:29<349:28:46,  6.82s/it]
 18%|█▊        | 40533/225000 [87:22:34<321:12:59,  6.27s/it]
 18%|█▊        | 40534/225000 [87:22:40<315:19:02,  6.15s/it]
 18%|█▊        | 40535/225000 [87:22:46<308:49:35,  6.03s/it]
 18%|█▊        | 40536/225000 [87:22:51<297:23:28,  5.80s/it]
 18%|█▊        | 40537/225000 [87:22:57<291:12:26,  5.68s/it]
 18%|█▊        | 40538/225000 [87:23:02<293:32:12,  5.73s/it]
 18%|█▊        | 40539/225000 [87:23:09<311:18:25,  6.08s/it]
 18%|█▊        | 40540/225000 [87:23:22<420:12:07,  8.20s/it]
 18%|█▊        | 40541/225000 [87:23:28<372:39:30,  7.27s/it]
 18%|█▊        | 40542/225000 [87:23:33<338:00:15,  6.60s/it]
 18%|█▊        | 40543/225000 [87:23:38<316:36:53,  6.18s/it]
 18%|█▊        | 40544/225000 [87:23:43<307:02:04,  5.99s/it]
 18%|█▊        | 40545/225000 [87:23:52<352:44:02,  6.88s/it]
 18%|█▊        | 40546/225000 [87:24:01<382:48:58,  7.47s/it]
 18%|█▊        | 40547/225000 [87:24:07<354:00:53,  6.91s/it]
 18%|█▊        | 40548/225000 [87:24:12<332:09:09,  6.48s/it]
 18%|█▊        | 40549/225000 [87:24:23<398:39:21,  7.78s/it]
 18%|█▊        | 40550/225000 [87:24:29<369:16:50,  7.21s/it]
                                                             
{'loss': '0.6945', 'grad_norm': '0.7848', 'learning_rate': '4.755e-05', 'epoch': '0.1802', 'num_input_tokens_seen': 147721056, 'train_runtime': '3.147e+05', 'train_tokens_per_second': '469.4'}

 18%|█▊        | 40550/225000 [87:24:29<369:16:50,  7.21s/it]
 18%|█▊        | 40551/225000 [87:24:35<347:39:16,  6.79s/it]
 18%|█▊        | 40552/225000 [87:24:40<330:43:24,  6.45s/it]
 18%|█▊        | 40553/225000 [87:24:52<404:16:15,  7.89s/it]
 18%|█▊        | 40554/225000 [87:24:57<372:25:11,  7.27s/it]
 18%|█▊        | 40555/225000 [87:25:03<343:38:50,  6.71s/it]
 18%|█▊        | 40556/225000 [87:25:09<328:31:13,  6.41s/it]
 18%|█▊        | 40557/225000 [87:25:15<321:51:55,  6.28s/it]
 18%|█▊        | 40558/225000 [87:25:20<313:17:21,  6.11s/it]
 18%|█▊        | 40559/225000 [87:25:26<307:25:03,  6.00s/it]
 18%|█▊        | 40560/225000 [87:25:33<315:35:21,  6.16s/it]
 18%|█▊        | 40561/225000 [87:25:46<419:30:27,  8.19s/it]
 18%|█▊        | 40562/225000 [87:25:51<376:15:44,  7.34s/it]
 18%|█▊        | 40563/225000 [87:25:55<334:18:41,  6.53s/it]
 18%|█▊        | 40564/225000 [87:26:00<305:47:52,  5.97s/it]
 18%|█▊        | 40565/225000 [87:26:05<287:45:05,  5.62s/it]
 18%|█▊        | 40566/225000 [87:26:12<311:15:57,  6.08s/it]
 18%|█▊        | 40567/225000 [87:26:23<377:56:24,  7.38s/it]
 18%|█▊        | 40568/225000 [87:26:27<325:49:52,  6.36s/it]
 18%|█▊        | 40569/225000 [87:26:32<306:24:16,  5.98s/it]
 18%|█▊        | 40570/225000 [87:26:44<406:27:10,  7.93s/it]
 18%|█▊        | 40571/225000 [87:26:49<354:49:05,  6.93s/it]
 18%|█▊        | 40572/225000 [87:26:56<357:34:06,  6.98s/it]
 18%|█▊        | 40573/225000 [87:27:01<328:54:40,  6.42s/it]
 18%|█▊        | 40574/225000 [87:27:10<363:40:42,  7.10s/it]
 18%|█▊        | 40575/225000 [87:27:15<332:14:26,  6.49s/it]
 18%|█▊        | 40576/225000 [87:27:20<307:51:24,  6.01s/it]
 18%|█▊        | 40577/225000 [87:27:31<398:24:10,  7.78s/it]
 18%|█▊        | 40578/225000 [87:27:35<340:22:19,  6.64s/it]
 18%|█▊        | 40579/225000 [87:27:40<306:40:00,  5.99s/it]
 18%|█▊        | 40580/225000 [87:27:45<299:31:24,  5.85s/it]
 18%|█▊        | 40581/225000 [87:27:56<375:19:20,  7.33s/it]
 18%|█▊        | 40582/225000 [87:28:00<328:05:31,  6.40s/it]
 18%|█▊        | 40583/225000 [87:28:06<317:56:08,  6.21s/it]
 18%|█▊        | 40584/225000 [87:28:12<308:39:27,  6.03s/it]
 18%|█▊        | 40585/225000 [87:28:18<307:22:23,  6.00s/it]
 18%|█▊        | 40586/225000 [87:28:23<301:30:00,  5.89s/it]
 18%|█▊        | 40587/225000 [87:28:36<408:26:20,  7.97s/it]
 18%|█▊        | 40588/225000 [87:28:42<375:27:25,  7.33s/it]
 18%|█▊        | 40589/225000 [87:28:50<387:51:41,  7.57s/it]
 18%|█▊        | 40590/225000 [87:28:57<371:32:43,  7.25s/it]
 18%|█▊        | 40591/225000 [87:29:04<376:27:34,  7.35s/it]
 18%|█▊        | 40592/225000 [87:29:10<352:16:32,  6.88s/it]
 18%|█▊        | 40593/225000 [87:29:16<335:36:08,  6.55s/it]
 18%|█▊        | 40594/225000 [87:29:22<323:35:35,  6.32s/it]
 18%|█▊        | 40595/225000 [87:29:27<307:59:04,  6.01s/it]
 18%|█▊        | 40596/225000 [87:29:32<300:50:14,  5.87s/it]
 18%|█▊        | 40597/225000 [87:29:40<320:47:11,  6.26s/it]
 18%|█▊        | 40598/225000 [87:29:50<385:14:30,  7.52s/it]
 18%|█▊        | 40599/225000 [87:29:56<355:29:55,  6.94s/it]
 18%|█▊        | 40600/225000 [87:30:01<336:37:35,  6.57s/it]
                                                             
{'loss': '0.6761', 'grad_norm': '0.7437', 'learning_rate': '4.755e-05', 'epoch': '0.1804', 'num_input_tokens_seen': 147869976, 'train_runtime': '3.15e+05', 'train_tokens_per_second': '469.4'}

 18%|█▊        | 40600/225000 [87:30:01<336:37:35,  6.57s/it]
 18%|█▊        | 40601/225000 [87:30:07<321:16:14,  6.27s/it]
 18%|█▊        | 40602/225000 [87:30:13<312:16:10,  6.10s/it]
 18%|█▊        | 40603/225000 [87:30:20<338:58:48,  6.62s/it]
 18%|█▊        | 40604/225000 [87:30:26<321:30:49,  6.28s/it]
 18%|█▊        | 40605/225000 [87:30:31<307:22:14,  6.00s/it]
 18%|█▊        | 40606/225000 [87:30:37<303:28:02,  5.92s/it]
 18%|█▊        | 40607/225000 [87:30:43<303:08:04,  5.92s/it]
 18%|█▊        | 40608/225000 [87:30:48<295:44:27,  5.77s/it]
 18%|█▊        | 40609/225000 [87:31:01<405:30:12,  7.92s/it]
 18%|█▊        | 40610/225000 [87:31:10<412:10:56,  8.05s/it]
 18%|█▊        | 40611/225000 [87:31:16<391:56:50,  7.65s/it]
 18%|█▊        | 40612/225000 [87:31:22<355:34:10,  6.94s/it]
 18%|█▊        | 40613/225000 [87:31:27<328:17:50,  6.41s/it]
 18%|█▊        | 40614/225000 [87:31:32<310:38:32,  6.07s/it]
 18%|█▊        | 40615/225000 [87:31:39<317:31:23,  6.20s/it]
 18%|█▊        | 40616/225000 [87:31:51<409:44:58,  8.00s/it]
 18%|█▊        | 40617/225000 [87:31:55<353:21:27,  6.90s/it]
 18%|█▊        | 40618/225000 [87:32:00<317:42:36,  6.20s/it]
 18%|█▊        | 40619/225000 [87:32:04<287:23:01,  5.61s/it]
 18%|█▊        | 40620/225000 [87:32:16<394:14:32,  7.70s/it]
 18%|█▊        | 40621/225000 [87:32:22<358:33:16,  7.00s/it]
 18%|█▊        | 40622/225000 [87:32:27<336:16:52,  6.57s/it]
 18%|█▊        | 40623/225000 [87:32:33<320:17:20,  6.25s/it]
 18%|█▊        | 40624/225000 [87:32:38<308:58:38,  6.03s/it]
 18%|█▊        | 40625/225000 [87:32:44<306:44:52,  5.99s/it]
 18%|█▊        | 40626/225000 [87:32:50<297:32:40,  5.81s/it]
 18%|█▊        | 40627/225000 [87:32:55<289:34:11,  5.65s/it]
 18%|█▊        | 40628/225000 [87:33:02<307:56:41,  6.01s/it]
 18%|█▊        | 40629/225000 [87:33:16<436:39:06,  8.53s/it]
 18%|█▊        | 40630/225000 [87:33:21<376:57:55,  7.36s/it]
 18%|█▊        | 40631/225000 [87:33:31<414:56:06,  8.10s/it]
 18%|█▊        | 40632/225000 [87:33:37<381:22:27,  7.45s/it]
 18%|█▊        | 40633/225000 [87:33:43<363:19:07,  7.09s/it]
 18%|█▊        | 40634/225000 [87:33:47<316:13:27,  6.17s/it]
 18%|█▊        | 40635/225000 [87:33:58<384:34:36,  7.51s/it]
 18%|█▊        | 40636/225000 [87:34:01<324:14:34,  6.33s/it]
 18%|█▊        | 40637/225000 [87:34:09<344:07:27,  6.72s/it]
 18%|█▊        | 40638/225000 [87:34:13<305:50:00,  5.97s/it]
 18%|█▊        | 40639/225000 [87:34:18<291:42:26,  5.70s/it]
 18%|█▊        | 40640/225000 [87:34:23<281:05:28,  5.49s/it]
 18%|█▊        | 40641/225000 [87:34:28<278:44:24,  5.44s/it]
 18%|█▊        | 40642/225000 [87:34:34<282:23:20,  5.51s/it]
 18%|█▊        | 40643/225000 [87:34:47<400:31:35,  7.82s/it]
 18%|█▊        | 40644/225000 [87:34:54<385:46:18,  7.53s/it]
 18%|█▊        | 40645/225000 [87:35:01<379:15:44,  7.41s/it]
 18%|█▊        | 40646/225000 [87:35:05<325:53:02,  6.36s/it]
 18%|█▊        | 40647/225000 [87:35:11<316:21:29,  6.18s/it]
 18%|█▊        | 40648/225000 [87:35:20<359:43:05,  7.02s/it]
 18%|█▊        | 40649/225000 [87:35:27<363:52:09,  7.11s/it]
 18%|█▊        | 40650/225000 [87:35:31<313:12:11,  6.12s/it]
                                                             
{'loss': '0.6787', 'grad_norm': '0.8232', 'learning_rate': '4.754e-05', 'epoch': '0.1807', 'num_input_tokens_seen': 148017656, 'train_runtime': '3.153e+05', 'train_tokens_per_second': '469.4'}

 18%|█▊        | 40650/225000 [87:35:31<313:12:11,  6.12s/it]
 18%|█▊        | 40651/225000 [87:35:36<297:18:18,  5.81s/it]
 18%|█▊        | 40652/225000 [87:35:41<288:45:36,  5.64s/it]
 18%|█▊        | 40653/225000 [87:35:47<284:12:07,  5.55s/it]
 18%|█▊        | 40654/225000 [87:35:52<285:18:53,  5.57s/it]
 18%|█▊        | 40655/225000 [87:35:57<278:04:23,  5.43s/it]
 18%|█▊        | 40656/225000 [87:36:10<392:36:17,  7.67s/it]
 18%|█▊        | 40657/225000 [87:36:16<362:13:05,  7.07s/it]
 18%|█▊        | 40658/225000 [87:36:25<390:25:55,  7.62s/it]
 18%|█▊        | 40659/225000 [87:36:30<344:00:13,  6.72s/it]
 18%|█▊        | 40660/225000 [87:36:38<373:53:30,  7.30s/it]
 18%|█▊        | 40661/225000 [87:36:44<346:39:26,  6.77s/it]
 18%|█▊        | 40662/225000 [87:36:48<314:15:59,  6.14s/it]
 18%|█▊        | 40663/225000 [87:36:54<305:35:44,  5.97s/it]
 18%|█▊        | 40664/225000 [87:36:59<296:12:28,  5.78s/it]
 18%|█▊        | 40665/225000 [87:37:06<302:50:25,  5.91s/it]
 18%|█▊        | 40666/225000 [87:37:19<414:50:43,  8.10s/it]
 18%|█▊        | 40667/225000 [87:37:25<381:22:28,  7.45s/it]
 18%|█▊        | 40668/225000 [87:37:34<414:37:39,  8.10s/it]
 18%|█▊        | 40669/225000 [87:37:38<341:17:27,  6.67s/it]
 18%|█▊        | 40670/225000 [87:37:45<356:51:46,  6.97s/it]
 18%|█▊        | 40671/225000 [87:37:52<355:45:39,  6.95s/it]
 18%|█▊        | 40672/225000 [87:37:57<324:04:07,  6.33s/it]
 18%|█▊        | 40673/225000 [87:38:08<401:21:28,  7.84s/it]
 18%|█▊        | 40674/225000 [87:38:16<395:17:21,  7.72s/it]
 18%|█▊        | 40675/225000 [87:38:20<347:17:10,  6.78s/it]
 18%|█▊        | 40676/225000 [87:38:27<347:54:02,  6.79s/it]
 18%|█▊        | 40677/225000 [87:38:35<359:19:19,  7.02s/it]
 18%|█▊        | 40678/225000 [87:38:40<336:51:50,  6.58s/it]
 18%|█▊        | 40679/225000 [87:38:45<299:02:05,  5.84s/it]
 18%|█▊        | 40680/225000 [87:38:50<298:17:22,  5.83s/it]
 18%|█▊        | 40681/225000 [87:38:56<299:22:40,  5.85s/it]
 18%|█▊        | 40682/225000 [87:39:02<302:27:54,  5.91s/it]
 18%|█▊        | 40683/225000 [87:39:09<321:14:41,  6.27s/it]
 18%|█▊        | 40684/225000 [87:39:18<355:59:33,  6.95s/it]
 18%|█▊        | 40685/225000 [87:39:23<327:45:01,  6.40s/it]
 18%|█▊        | 40686/225000 [87:39:32<361:28:03,  7.06s/it]
 18%|█▊        | 40687/225000 [87:39:36<324:22:55,  6.34s/it]
 18%|█▊        | 40688/225000 [87:39:42<308:46:53,  6.03s/it]
 18%|█▊        | 40689/225000 [87:39:53<394:50:41,  7.71s/it]
 18%|█▊        | 40690/225000 [87:39:57<327:50:28,  6.40s/it]
 18%|█▊        | 40691/225000 [87:40:07<383:08:19,  7.48s/it]
 18%|█▊        | 40692/225000 [87:40:11<337:11:15,  6.59s/it]
 18%|█▊        | 40693/225000 [87:40:18<338:58:38,  6.62s/it]
 18%|█▊        | 40694/225000 [87:40:26<370:21:19,  7.23s/it]
 18%|█▊        | 40695/225000 [87:40:32<348:07:23,  6.80s/it]
 18%|█▊        | 40696/225000 [87:40:40<363:15:21,  7.10s/it]
 18%|█▊        | 40697/225000 [87:40:47<364:49:36,  7.13s/it]
 18%|█▊        | 40698/225000 [87:40:54<366:04:19,  7.15s/it]
 18%|█▊        | 40699/225000 [87:40:58<310:24:26,  6.06s/it]
 18%|█▊        | 40700/225000 [87:41:06<346:46:23,  6.77s/it]
                                                             
{'loss': '0.6742', 'grad_norm': '0.7244', 'learning_rate': '4.753e-05', 'epoch': '0.1809', 'num_input_tokens_seen': 148166224, 'train_runtime': '3.157e+05', 'train_tokens_per_second': '469.4'}

 18%|█▊        | 40700/225000 [87:41:06<346:46:23,  6.77s/it]
 18%|█▊        | 40701/225000 [87:41:12<334:04:41,  6.53s/it]
 18%|█▊        | 40702/225000 [87:41:18<320:33:50,  6.26s/it]
 18%|█▊        | 40703/225000 [87:41:25<334:02:33,  6.53s/it]
 18%|█▊        | 40704/225000 [87:41:31<325:28:25,  6.36s/it]
 18%|█▊        | 40705/225000 [87:41:36<300:37:52,  5.87s/it]
 18%|█▊        | 40706/225000 [87:41:41<288:38:21,  5.64s/it]
 18%|█▊        | 40707/225000 [87:41:53<387:22:55,  7.57s/it]
 18%|█▊        | 40708/225000 [87:41:58<342:06:50,  6.68s/it]
 18%|█▊        | 40709/225000 [87:42:02<313:19:51,  6.12s/it]
 18%|█▊        | 40710/225000 [87:42:12<361:57:48,  7.07s/it]
 18%|█▊        | 40711/225000 [87:42:18<347:36:25,  6.79s/it]
 18%|█▊        | 40712/225000 [87:42:22<311:37:44,  6.09s/it]
 18%|█▊        | 40713/225000 [87:42:28<308:49:12,  6.03s/it]
 18%|█▊        | 40714/225000 [87:42:34<307:36:08,  6.01s/it]
 18%|█▊        | 40715/225000 [87:42:47<418:03:15,  8.17s/it]
 18%|█▊        | 40716/225000 [87:42:54<393:30:52,  7.69s/it]
 18%|█▊        | 40717/225000 [87:42:58<342:23:04,  6.69s/it]
 18%|█▊        | 40718/225000 [87:43:03<309:52:42,  6.05s/it]
 18%|█▊        | 40719/225000 [87:43:12<365:08:56,  7.13s/it]
 18%|█▊        | 40720/225000 [87:43:19<359:39:54,  7.03s/it]
 18%|█▊        | 40721/225000 [87:43:29<395:26:15,  7.73s/it]
 18%|█▊        | 40722/225000 [87:43:33<349:25:24,  6.83s/it]
 18%|█▊        | 40723/225000 [87:43:38<320:13:09,  6.26s/it]
 18%|█▊        | 40724/225000 [87:43:43<295:36:22,  5.77s/it]
 18%|█▊        | 40725/225000 [87:43:48<278:00:06,  5.43s/it]
 18%|█▊        | 40726/225000 [87:43:55<314:37:57,  6.15s/it]
 18%|█▊        | 40727/225000 [87:44:06<381:31:36,  7.45s/it]
 18%|█▊        | 40728/225000 [87:44:13<374:35:59,  7.32s/it]
 18%|█▊        | 40729/225000 [87:44:17<322:22:05,  6.30s/it]
 18%|█▊        | 40730/225000 [87:44:26<374:07:53,  7.31s/it]
 18%|█▊        | 40731/225000 [87:44:30<321:51:38,  6.29s/it]
 18%|█▊        | 40732/225000 [87:44:35<300:25:14,  5.87s/it]
 18%|█▊        | 40733/225000 [87:44:45<356:05:40,  6.96s/it]
 18%|█▊        | 40734/225000 [87:44:49<309:31:44,  6.05s/it]
 18%|█▊        | 40735/225000 [87:44:57<347:41:10,  6.79s/it]
 18%|█▊        | 40736/225000 [87:45:04<342:02:03,  6.68s/it]
 18%|█▊        | 40737/225000 [87:45:11<351:26:44,  6.87s/it]
 18%|█▊        | 40738/225000 [87:45:19<368:21:15,  7.20s/it]
 18%|█▊        | 40739/225000 [87:45:23<324:17:52,  6.34s/it]
 18%|█▊        | 40740/225000 [87:45:28<294:26:43,  5.75s/it]
 18%|█▊        | 40741/225000 [87:45:35<317:45:42,  6.21s/it]
 18%|█▊        | 40742/225000 [87:45:44<360:55:58,  7.05s/it]
 18%|█▊        | 40743/225000 [87:45:49<327:11:47,  6.39s/it]
 18%|█▊        | 40744/225000 [87:45:53<301:10:43,  5.88s/it]
 18%|█▊        | 40745/225000 [87:45:59<295:21:57,  5.77s/it]
 18%|█▊        | 40746/225000 [87:46:11<384:05:59,  7.50s/it]
 18%|█▊        | 40747/225000 [87:46:17<361:16:59,  7.06s/it]
 18%|█▊        | 40748/225000 [87:46:23<355:29:04,  6.95s/it]
 18%|█▊        | 40749/225000 [87:46:28<316:01:03,  6.17s/it]
 18%|█▊        | 40750/225000 [87:46:32<286:39:16,  5.60s/it]
                                                             
{'loss': '0.6822', 'grad_norm': '0.7983', 'learning_rate': '4.752e-05', 'epoch': '0.1811', 'num_input_tokens_seen': 148314144, 'train_runtime': '3.16e+05', 'train_tokens_per_second': '469.4'}

 18%|█▊        | 40750/225000 [87:46:32<286:39:16,  5.60s/it]
 18%|█▊        | 40751/225000 [87:46:37<274:10:42,  5.36s/it]
 18%|█▊        | 40752/225000 [87:46:44<305:07:01,  5.96s/it]
 18%|█▊        | 40753/225000 [87:46:56<398:59:15,  7.80s/it]
 18%|█▊        | 40754/225000 [87:47:01<349:52:58,  6.84s/it]
 18%|█▊        | 40755/225000 [87:47:07<339:56:58,  6.64s/it]
 18%|█▊        | 40756/225000 [87:47:16<374:17:43,  7.31s/it]
 18%|█▊        | 40757/225000 [87:47:20<328:46:50,  6.42s/it]
 18%|█▊        | 40758/225000 [87:47:29<371:53:54,  7.27s/it]
 18%|█▊        | 40759/225000 [87:47:35<349:51:34,  6.84s/it]
 18%|█▊        | 40760/225000 [87:47:43<370:57:11,  7.25s/it]
 18%|█▊        | 40761/225000 [87:47:48<333:43:53,  6.52s/it]
 18%|█▊        | 40762/225000 [87:47:53<310:18:54,  6.06s/it]
 18%|█▊        | 40763/225000 [87:48:00<320:13:36,  6.26s/it]
 18%|█▊        | 40764/225000 [87:48:11<386:52:03,  7.56s/it]
 18%|█▊        | 40765/225000 [87:48:15<343:37:12,  6.71s/it]
 18%|█▊        | 40766/225000 [87:48:20<308:07:39,  6.02s/it]
 18%|█▊        | 40767/225000 [87:48:31<384:18:06,  7.51s/it]
 18%|█▊        | 40768/225000 [87:48:36<357:05:35,  6.98s/it]
 18%|█▊        | 40769/225000 [87:48:41<319:48:33,  6.25s/it]
 18%|█▊        | 40770/225000 [87:48:51<371:20:31,  7.26s/it]
 18%|█▊        | 40771/225000 [87:48:57<362:29:10,  7.08s/it]
 18%|█▊        | 40772/225000 [87:49:03<335:05:30,  6.55s/it]
 18%|█▊        | 40773/225000 [87:49:08<315:30:41,  6.17s/it]
 18%|█▊        | 40774/225000 [87:49:13<301:46:45,  5.90s/it]
 18%|█▊        | 40775/225000 [87:49:19<305:45:51,  5.98s/it]
 18%|█▊        | 40776/225000 [87:49:25<302:48:52,  5.92s/it]
 18%|█▊        | 40777/225000 [87:49:31<297:06:10,  5.81s/it]
 18%|█▊        | 40778/225000 [87:49:36<297:27:30,  5.81s/it]
 18%|█▊        | 40779/225000 [87:49:49<408:24:56,  7.98s/it]
 18%|█▊        | 40780/225000 [87:49:57<409:30:10,  8.00s/it]
 18%|█▊        | 40781/225000 [87:50:06<411:14:04,  8.04s/it]
 18%|█▊        | 40782/225000 [87:50:10<350:53:36,  6.86s/it]
 18%|█▊        | 40783/225000 [87:50:14<309:45:38,  6.05s/it]
 18%|█▊        | 40784/225000 [87:50:19<295:19:21,  5.77s/it]
 18%|█▊        | 40785/225000 [87:50:25<303:11:50,  5.93s/it]
 18%|█▊        | 40786/225000 [87:50:38<402:01:16,  7.86s/it]
 18%|█▊        | 40787/225000 [87:50:43<365:19:12,  7.14s/it]
 18%|█▊        | 40788/225000 [87:50:49<342:07:12,  6.69s/it]
 18%|█▊        | 40789/225000 [87:50:54<320:59:39,  6.27s/it]
 18%|█▊        | 40790/225000 [87:50:59<298:28:09,  5.83s/it]
 18%|█▊        | 40791/225000 [87:51:12<406:32:46,  7.95s/it]
 18%|█▊        | 40792/225000 [87:51:17<365:03:22,  7.13s/it]
 18%|█▊        | 40793/225000 [87:51:21<319:22:47,  6.24s/it]
 18%|█▊        | 40794/225000 [87:51:26<293:52:43,  5.74s/it]
 18%|█▊        | 40795/225000 [87:51:31<290:37:12,  5.68s/it]
 18%|█▊        | 40796/225000 [87:51:43<386:36:00,  7.56s/it]
 18%|█▊        | 40797/225000 [87:51:47<328:36:15,  6.42s/it]
 18%|█▊        | 40798/225000 [87:51:52<304:32:50,  5.95s/it]
 18%|█▊        | 40799/225000 [87:51:56<284:55:59,  5.57s/it]
 18%|█▊        | 40800/225000 [87:52:09<393:14:35,  7.69s/it]
                                                             
{'loss': '0.6862', 'grad_norm': '0.7066', 'learning_rate': '4.751e-05', 'epoch': '0.1813', 'num_input_tokens_seen': 148464328, 'train_runtime': '3.163e+05', 'train_tokens_per_second': '469.3'}

 18%|█▊        | 40800/225000 [87:52:09<393:14:35,  7.69s/it]
 18%|█▊        | 40801/225000 [87:52:13<338:19:24,  6.61s/it]
 18%|█▊        | 40802/225000 [87:52:17<301:21:43,  5.89s/it]
 18%|█▊        | 40803/225000 [87:52:26<348:17:19,  6.81s/it]
 18%|█▊        | 40804/225000 [87:52:35<375:57:01,  7.35s/it]
 18%|█▊        | 40805/225000 [87:52:39<329:28:36,  6.44s/it]
 18%|█▊        | 40806/225000 [87:52:47<342:07:53,  6.69s/it]
 18%|█▊        | 40807/225000 [87:52:51<306:43:59,  6.00s/it]
 18%|█▊        | 40808/225000 [87:52:55<283:14:24,  5.54s/it]
 18%|█▊        | 40809/225000 [87:53:00<276:24:30,  5.40s/it]
 18%|█▊        | 40810/225000 [87:53:12<374:08:43,  7.31s/it]
 18%|█▊        | 40811/225000 [87:53:18<355:00:37,  6.94s/it]
 18%|█▊        | 40812/225000 [87:53:23<319:56:40,  6.25s/it]
 18%|█▊        | 40813/225000 [87:53:29<312:30:42,  6.11s/it]
 18%|█▊        | 40814/225000 [87:53:34<304:46:23,  5.96s/it]
 18%|█▊        | 40815/225000 [87:53:47<412:40:23,  8.07s/it]
 18%|█▊        | 40816/225000 [87:53:54<396:55:23,  7.76s/it]
 18%|█▊        | 40817/225000 [87:54:00<358:00:38,  7.00s/it]
 18%|█▊        | 40818/225000 [87:54:05<331:11:24,  6.47s/it]
 18%|█▊        | 40819/225000 [87:54:10<315:41:35,  6.17s/it]
 18%|█▊        | 40820/225000 [87:54:16<301:12:33,  5.89s/it]
 18%|█▊        | 40821/225000 [87:54:25<353:37:56,  6.91s/it]
 18%|█▊        | 40822/225000 [87:54:34<393:08:52,  7.68s/it]
 18%|█▊        | 40823/225000 [87:54:40<364:45:45,  7.13s/it]
 18%|█▊        | 40824/225000 [87:54:47<365:26:25,  7.14s/it]
 18%|█▊        | 40825/225000 [87:54:53<337:16:40,  6.59s/it]
 18%|█▊        | 40826/225000 [87:54:58<322:40:32,  6.31s/it]
 18%|█▊        | 40827/225000 [87:55:04<307:32:47,  6.01s/it]
 18%|█▊        | 40828/225000 [87:55:09<300:31:45,  5.87s/it]
 18%|█▊        | 40829/225000 [87:55:14<288:09:50,  5.63s/it]
 18%|█▊        | 40830/225000 [87:55:19<280:06:36,  5.48s/it]
 18%|█▊        | 40831/225000 [87:55:34<424:54:09,  8.31s/it]
 18%|█▊        | 40832/225000 [87:55:39<378:12:35,  7.39s/it]
 18%|█▊        | 40833/225000 [87:55:49<412:05:30,  8.06s/it]
 18%|█▊        | 40834/225000 [87:55:57<407:40:03,  7.97s/it]
 18%|█▊        | 40835/225000 [87:56:01<355:10:36,  6.94s/it]
 18%|█▊        | 40836/225000 [87:56:06<319:52:25,  6.25s/it]
 18%|█▊        | 40837/225000 [87:56:11<293:03:37,  5.73s/it]
 18%|█▊        | 40838/225000 [87:56:24<405:48:56,  7.93s/it]
 18%|█▊        | 40839/225000 [87:56:28<354:19:39,  6.93s/it]
 18%|█▊        | 40840/225000 [87:56:33<316:02:24,  6.18s/it]
 18%|█▊        | 40841/225000 [87:56:37<289:56:23,  5.67s/it]
 18%|█▊        | 40842/225000 [87:56:42<279:27:28,  5.46s/it]
 18%|█▊        | 40843/225000 [87:56:51<331:56:18,  6.49s/it]
 18%|█▊        | 40844/225000 [87:57:01<388:13:17,  7.59s/it]
 18%|█▊        | 40845/225000 [87:57:05<329:06:13,  6.43s/it]
 18%|█▊        | 40846/225000 [87:57:10<306:46:09,  6.00s/it]
 18%|█▊        | 40847/225000 [87:57:15<289:41:50,  5.66s/it]
 18%|█▊        | 40848/225000 [87:57:23<328:56:51,  6.43s/it]
 18%|█▊        | 40849/225000 [87:57:33<390:31:04,  7.63s/it]
 18%|█▊        | 40850/225000 [87:57:39<363:18:55,  7.10s/it]
                                                             
{'loss': '0.7081', 'grad_norm': '0.6824', 'learning_rate': '4.751e-05', 'epoch': '0.1816', 'num_input_tokens_seen': 148614744, 'train_runtime': '3.167e+05', 'train_tokens_per_second': '469.3'}

 18%|█▊        | 40850/225000 [87:57:39<363:18:55,  7.10s/it]
 18%|█▊        | 40851/225000 [87:57:46<361:49:11,  7.07s/it]
 18%|█▊        | 40852/225000 [87:57:53<356:51:21,  6.98s/it]
 18%|█▊        | 40853/225000 [87:57:58<324:00:46,  6.33s/it]
 18%|█▊        | 40854/225000 [87:58:03<303:46:55,  5.94s/it]
 18%|█▊        | 40855/225000 [87:58:08<291:51:10,  5.71s/it]
 18%|█▊        | 40856/225000 [87:58:13<281:04:05,  5.49s/it]
 18%|█▊        | 40857/225000 [87:58:25<380:59:29,  7.45s/it]
 18%|█▊        | 40858/225000 [87:58:29<323:53:28,  6.33s/it]
 18%|█▊        | 40859/225000 [87:58:38<370:10:51,  7.24s/it]
 18%|█▊        | 40860/225000 [87:58:42<320:49:06,  6.27s/it]
 18%|█▊        | 40861/225000 [87:58:46<281:55:22,  5.51s/it]
 18%|█▊        | 40862/225000 [87:58:57<371:37:14,  7.27s/it]
 18%|█▊        | 40863/225000 [87:59:01<323:41:44,  6.33s/it]
 18%|█▊        | 40864/225000 [87:59:06<304:28:01,  5.95s/it]
 18%|█▊        | 40865/225000 [87:59:12<301:25:04,  5.89s/it]
 18%|█▊        | 40866/225000 [87:59:20<325:16:33,  6.36s/it]
 18%|█▊        | 40867/225000 [87:59:28<358:34:01,  7.01s/it]
 18%|█▊        | 40868/225000 [87:59:33<320:01:26,  6.26s/it]
 18%|█▊        | 40869/225000 [87:59:40<336:53:22,  6.59s/it]
 18%|█▊        | 40870/225000 [87:59:49<372:57:42,  7.29s/it]
 18%|█▊        | 40871/225000 [87:59:57<389:18:36,  7.61s/it]
 18%|█▊        | 40872/225000 [88:00:03<360:41:15,  7.05s/it]
 18%|█▊        | 40873/225000 [88:00:11<372:39:59,  7.29s/it]
 18%|█▊        | 40874/225000 [88:00:18<375:04:07,  7.33s/it]
 18%|█▊        | 40875/225000 [88:00:23<339:36:14,  6.64s/it]
 18%|█▊        | 40876/225000 [88:00:29<326:08:46,  6.38s/it]
 18%|█▊        | 40877/225000 [88:00:38<369:33:11,  7.23s/it]
 18%|█▊        | 40878/225000 [88:00:42<316:55:10,  6.20s/it]
 18%|█▊        | 40879/225000 [88:00:53<383:03:10,  7.49s/it]
 18%|█▊        | 40880/225000 [88:00:56<323:09:10,  6.32s/it]
 18%|█▊        | 40881/225000 [88:01:01<295:08:36,  5.77s/it]
 18%|█▊        | 40882/225000 [88:01:06<282:16:03,  5.52s/it]
 18%|█▊        | 40883/225000 [88:01:18<392:39:29,  7.68s/it]
 18%|█▊        | 40884/225000 [88:01:22<329:51:18,  6.45s/it]
 18%|█▊        | 40885/225000 [88:01:32<377:20:17,  7.38s/it]
 18%|█▊        | 40886/225000 [88:01:36<332:12:22,  6.50s/it]
 18%|█▊        | 40887/225000 [88:01:40<301:32:43,  5.90s/it]
 18%|█▊        | 40888/225000 [88:01:45<284:06:33,  5.56s/it]
 18%|█▊        | 40889/225000 [88:01:51<283:24:33,  5.54s/it]
 18%|█▊        | 40890/225000 [88:02:03<394:31:27,  7.71s/it]
 18%|█▊        | 40891/225000 [88:02:08<341:28:24,  6.68s/it]
 18%|█▊        | 40892/225000 [88:02:17<379:52:49,  7.43s/it]
 18%|█▊        | 40893/225000 [88:02:22<350:18:15,  6.85s/it]
 18%|█▊        | 40894/225000 [88:02:28<328:39:01,  6.43s/it]
 18%|█▊        | 40895/225000 [88:02:33<305:52:29,  5.98s/it]
 18%|█▊        | 40896/225000 [88:02:38<297:37:02,  5.82s/it]
 18%|█▊        | 40897/225000 [88:02:44<293:59:41,  5.75s/it]
 18%|█▊        | 40898/225000 [88:02:52<330:37:38,  6.47s/it]
 18%|█▊        | 40899/225000 [88:03:04<420:07:58,  8.22s/it]
 18%|█▊        | 40900/225000 [88:03:08<349:52:30,  6.84s/it]
                                                             
{'loss': '0.6875', 'grad_norm': '0.8053', 'learning_rate': '4.75e-05', 'epoch': '0.1818', 'num_input_tokens_seen': 148762888, 'train_runtime': '3.17e+05', 'train_tokens_per_second': '469.3'}

 18%|█▊        | 40900/225000 [88:03:08<349:52:30,  6.84s/it]
 18%|█▊        | 40901/225000 [88:03:17<382:01:14,  7.47s/it]
 18%|█▊        | 40902/225000 [88:03:24<374:35:10,  7.32s/it]
 18%|█▊        | 40903/225000 [88:03:28<330:33:17,  6.46s/it]
 18%|█▊        | 40904/225000 [88:03:33<297:59:43,  5.83s/it]
 18%|█▊        | 40905/225000 [88:03:41<339:04:15,  6.63s/it]
 18%|█▊        | 40906/225000 [88:03:51<384:15:18,  7.51s/it]
 18%|█▊        | 40907/225000 [88:03:56<344:22:12,  6.73s/it]
 18%|█▊        | 40908/225000 [88:04:01<326:49:23,  6.39s/it]
 18%|█▊        | 40909/225000 [88:04:10<357:15:24,  6.99s/it]
 18%|█▊        | 40910/225000 [88:04:15<336:55:19,  6.59s/it]
 18%|█▊        | 40911/225000 [88:04:25<380:22:36,  7.44s/it]
 18%|█▊        | 40912/225000 [88:04:32<383:35:52,  7.50s/it]
 18%|█▊        | 40913/225000 [88:04:37<340:18:00,  6.65s/it]
 18%|█▊        | 40914/225000 [88:04:42<315:20:21,  6.17s/it]
 18%|█▊        | 40915/225000 [88:04:47<299:03:56,  5.85s/it]
 18%|█▊        | 40916/225000 [88:04:56<340:57:11,  6.67s/it]
 18%|█▊        | 40917/225000 [88:05:05<374:41:04,  7.33s/it]
 18%|█▊        | 40918/225000 [88:05:08<319:09:19,  6.24s/it]
 18%|█▊        | 40919/225000 [88:05:20<396:39:57,  7.76s/it]
 18%|█▊        | 40920/225000 [88:05:24<339:55:08,  6.65s/it]
 18%|█▊        | 40921/225000 [88:05:33<375:26:45,  7.34s/it]
 18%|█▊        | 40922/225000 [88:05:38<338:56:37,  6.63s/it]
 18%|█▊        | 40923/225000 [88:05:43<320:41:45,  6.27s/it]
 18%|█▊        | 40924/225000 [88:05:48<300:32:05,  5.88s/it]
 18%|█▊        | 40925/225000 [88:05:53<282:39:07,  5.53s/it]
 18%|█▊        | 40926/225000 [88:06:04<373:04:22,  7.30s/it]
 18%|█▊        | 40927/225000 [88:06:08<326:56:37,  6.39s/it]
 18%|█▊        | 40928/225000 [88:06:19<391:46:31,  7.66s/it]
 18%|█▊        | 40929/225000 [88:06:23<336:37:35,  6.58s/it]
 18%|█▊        | 40930/225000 [88:06:28<303:28:26,  5.94s/it]
 18%|█▊        | 40931/225000 [88:06:37<356:43:48,  6.98s/it]
 18%|█▊        | 40932/225000 [88:06:43<337:25:09,  6.60s/it]
 18%|█▊        | 40933/225000 [88:06:48<316:59:41,  6.20s/it]
 18%|█▊        | 40934/225000 [88:06:53<300:28:40,  5.88s/it]
 18%|█▊        | 40935/225000 [88:06:58<280:50:50,  5.49s/it]
 18%|█▊        | 40936/225000 [88:07:09<374:50:18,  7.33s/it]
 18%|█▊        | 40937/225000 [88:07:14<336:32:20,  6.58s/it]
 18%|█▊        | 40938/225000 [88:07:19<306:07:15,  5.99s/it]
 18%|█▊        | 40939/225000 [88:07:27<337:24:44,  6.60s/it]
 18%|█▊        | 40940/225000 [88:07:35<369:48:31,  7.23s/it]
 18%|█▊        | 40941/225000 [88:07:42<359:30:29,  7.03s/it]
 18%|█▊        | 40942/225000 [88:07:48<342:52:54,  6.71s/it]
 18%|█▊        | 40943/225000 [88:07:55<346:31:58,  6.78s/it]
 18%|█▊        | 40944/225000 [88:08:03<361:50:10,  7.08s/it]
 18%|█▊        | 40945/225000 [88:08:07<312:34:08,  6.11s/it]
 18%|█▊        | 40946/225000 [88:08:15<354:03:02,  6.93s/it]
 18%|█▊        | 40947/225000 [88:08:21<334:45:48,  6.55s/it]
 18%|█▊        | 40948/225000 [88:08:28<340:21:09,  6.66s/it]
 18%|█▊        | 40949/225000 [88:08:34<326:57:38,  6.40s/it]
 18%|█▊        | 40950/225000 [88:08:40<330:36:17,  6.47s/it]
                                                             
{'loss': '0.6797', 'grad_norm': '0.7162', 'learning_rate': '4.749e-05', 'epoch': '0.182', 'num_input_tokens_seen': 148911048, 'train_runtime': '3.173e+05', 'train_tokens_per_second': '469.3'}

 18%|█▊        | 40950/225000 [88:08:40<330:36:17,  6.47s/it]
 18%|█▊        | 40951/225000 [88:08:44<291:32:16,  5.70s/it]
 18%|█▊        | 40952/225000 [88:08:50<287:32:10,  5.62s/it]
 18%|█▊        | 40953/225000 [88:08:55<281:22:55,  5.50s/it]
 18%|█▊        | 40954/225000 [88:09:00<272:01:09,  5.32s/it]
 18%|█▊        | 40955/225000 [88:09:05<268:31:18,  5.25s/it]
 18%|█▊        | 40956/225000 [88:09:10<270:13:01,  5.29s/it]
 18%|█▊        | 40957/225000 [88:09:23<389:50:32,  7.63s/it]
 18%|█▊        | 40958/225000 [88:09:29<364:10:22,  7.12s/it]
 18%|█▊        | 40959/225000 [88:09:37<366:23:34,  7.17s/it]
 18%|█▊        | 40960/225000 [88:09:43<352:12:54,  6.89s/it]
 18%|█▊        | 40961/225000 [88:09:50<362:37:27,  7.09s/it]
 18%|█▊        | 40962/225000 [88:09:56<343:34:24,  6.72s/it]
 18%|█▊        | 40963/225000 [88:10:04<355:41:56,  6.96s/it]
 18%|█▊        | 40964/225000 [88:10:08<314:32:16,  6.15s/it]
 18%|█▊        | 40965/225000 [88:10:15<323:35:17,  6.33s/it]
 18%|█▊        | 40966/225000 [88:10:24<361:12:48,  7.07s/it]
 18%|█▊        | 40967/225000 [88:10:28<315:20:43,  6.17s/it]
 18%|█▊        | 40968/225000 [88:10:32<290:53:40,  5.69s/it]
 18%|█▊        | 40969/225000 [88:10:40<329:01:22,  6.44s/it]
 18%|█▊        | 40970/225000 [88:10:46<309:20:46,  6.05s/it]
 18%|█▊        | 40971/225000 [88:10:51<300:08:14,  5.87s/it]
 18%|█▊        | 40972/225000 [88:10:56<286:08:21,  5.60s/it]
 18%|█▊        | 40973/225000 [88:11:06<358:08:45,  7.01s/it]
 18%|█▊        | 40974/225000 [88:11:16<398:03:43,  7.79s/it]
 18%|█▊        | 40975/225000 [88:11:24<410:38:05,  8.03s/it]
 18%|█▊        | 40976/225000 [88:11:29<365:02:54,  7.14s/it]
 18%|█▊        | 40977/225000 [88:11:35<343:59:34,  6.73s/it]
 18%|█▊        | 40978/225000 [88:11:40<315:19:45,  6.17s/it]
 18%|█▊        | 40979/225000 [88:11:45<300:53:48,  5.89s/it]
 18%|█▊        | 40980/225000 [88:11:56<374:30:06,  7.33s/it]
 18%|█▊        | 40981/225000 [88:12:03<375:25:49,  7.34s/it]
 18%|█▊        | 40982/225000 [88:12:09<345:20:05,  6.76s/it]
 18%|█▊        | 40983/225000 [88:12:16<351:21:00,  6.87s/it]
 18%|█▊        | 40984/225000 [88:12:24<370:34:24,  7.25s/it]
 18%|█▊        | 40985/225000 [88:12:28<319:00:56,  6.24s/it]
 18%|█▊        | 40986/225000 [88:12:37<368:44:34,  7.21s/it]
 18%|█▊        | 40987/225000 [88:12:43<337:02:00,  6.59s/it]
 18%|█▊        | 40988/225000 [88:12:49<333:49:38,  6.53s/it]
 18%|█▊        | 40989/225000 [88:12:57<349:43:45,  6.84s/it]
 18%|█▊        | 40990/225000 [88:13:01<309:32:12,  6.06s/it]
 18%|█▊        | 40991/225000 [88:13:05<283:23:47,  5.54s/it]
 18%|█▊        | 40992/225000 [88:13:16<367:54:37,  7.20s/it]
 18%|█▊        | 40993/225000 [88:13:23<359:25:10,  7.03s/it]
 18%|█▊        | 40994/225000 [88:13:28<330:02:59,  6.46s/it]
 18%|█▊        | 40995/225000 [88:13:37<367:27:34,  7.19s/it]
 18%|█▊        | 40996/225000 [88:13:42<336:28:30,  6.58s/it]
 18%|█▊        | 40997/225000 [88:13:49<349:56:51,  6.85s/it]
 18%|█▊        | 40998/225000 [88:13:55<337:21:27,  6.60s/it]
 18%|█▊        | 40999/225000 [88:14:03<351:08:05,  6.87s/it]
 18%|█▊        | 41000/225000 [88:14:11<366:15:03,  7.17s/it]
                                                             
{'loss': '0.6666', 'grad_norm': '0.6783', 'learning_rate': '4.748e-05', 'epoch': '0.1822', 'num_input_tokens_seen': 149058952, 'train_runtime': '3.177e+05', 'train_tokens_per_second': '469.3'}

 18%|█▊        | 41000/225000 [88:14:11<366:15:03,  7.17s/it]
 18%|█▊        | 41001/225000 [88:14:15<312:26:08,  6.11s/it]
 18%|█▊        | 41002/225000 [88:14:22<339:41:57,  6.65s/it]
 18%|█▊        | 41003/225000 [88:14:27<313:41:50,  6.14s/it]
 18%|█▊        | 41004/225000 [88:14:37<363:27:55,  7.11s/it]
 18%|█▊        | 41005/225000 [88:14:42<334:10:52,  6.54s/it]
 18%|█▊        | 41006/225000 [88:14:47<314:58:18,  6.16s/it]
 18%|█▊        | 41007/225000 [88:14:53<313:47:04,  6.14s/it]
 18%|█▊        | 41008/225000 [88:14:58<296:11:28,  5.80s/it]
 18%|█▊        | 41009/225000 [88:15:04<292:48:47,  5.73s/it]
 18%|█▊        | 41010/225000 [88:15:11<318:16:00,  6.23s/it]
 18%|█▊        | 41011/225000 [88:15:24<413:02:47,  8.08s/it]
 18%|█▊        | 41012/225000 [88:15:30<386:47:41,  7.57s/it]
 18%|█▊        | 41013/225000 [88:15:34<331:58:01,  6.50s/it]
 18%|█▊        | 41014/225000 [88:15:44<385:53:28,  7.55s/it]
 18%|█▊        | 41015/225000 [88:15:49<341:13:14,  6.68s/it]
 18%|█▊        | 41016/225000 [88:15:56<354:03:05,  6.93s/it]
 18%|█▊        | 41017/225000 [88:16:01<322:36:06,  6.31s/it]
 18%|█▊        | 41018/225000 [88:16:06<302:46:01,  5.92s/it]
 18%|█▊        | 41019/225000 [88:16:11<290:26:04,  5.68s/it]
 18%|█▊        | 41020/225000 [88:16:17<286:47:20,  5.61s/it]
 18%|█▊        | 41021/225000 [88:16:26<344:02:41,  6.73s/it]
 18%|█▊        | 41022/225000 [88:16:37<413:56:39,  8.10s/it]
 18%|█▊        | 41023/225000 [88:16:43<376:48:20,  7.37s/it]
 18%|█▊        | 41024/225000 [88:16:50<377:32:59,  7.39s/it]
 18%|█▊        | 41025/225000 [88:16:55<330:23:33,  6.47s/it]
 18%|█▊        | 41026/225000 [88:17:02<341:03:52,  6.67s/it]
 18%|█▊        | 41027/225000 [88:17:09<350:57:17,  6.87s/it]
 18%|█▊        | 41028/225000 [88:17:13<306:30:52,  6.00s/it]
 18%|█▊        | 41029/225000 [88:17:21<328:16:30,  6.42s/it]
 18%|█▊        | 41030/225000 [88:17:30<367:45:13,  7.20s/it]
 18%|█▊        | 41031/225000 [88:17:34<332:29:54,  6.51s/it]
 18%|█▊        | 41032/225000 [88:17:40<310:29:13,  6.08s/it]
 18%|█▊        | 41033/225000 [88:17:45<301:11:42,  5.89s/it]
 18%|█▊        | 41034/225000 [88:17:50<288:34:40,  5.65s/it]
 18%|█▊        | 41035/225000 [88:18:00<361:08:53,  7.07s/it]
 18%|█▊        | 41036/225000 [88:18:11<407:19:02,  7.97s/it]
 18%|█▊        | 41037/225000 [88:18:15<359:56:16,  7.04s/it]
 18%|█▊        | 41038/225000 [88:18:21<332:19:05,  6.50s/it]
 18%|█▊        | 41039/225000 [88:18:26<318:52:14,  6.24s/it]
 18%|█▊        | 41040/225000 [88:18:32<314:02:11,  6.15s/it]
 18%|█▊        | 41041/225000 [88:18:38<302:16:58,  5.92s/it]
 18%|█▊        | 41042/225000 [88:18:46<345:38:52,  6.76s/it]
 18%|█▊        | 41043/225000 [88:18:58<420:27:58,  8.23s/it]
 18%|█▊        | 41044/225000 [88:19:02<351:13:40,  6.87s/it]
 18%|█▊        | 41045/225000 [88:19:06<317:48:33,  6.22s/it]
 18%|█▊        | 41046/225000 [88:19:17<384:23:06,  7.52s/it]
 18%|█▊        | 41047/225000 [88:19:22<343:51:14,  6.73s/it]
 18%|█▊        | 41048/225000 [88:19:27<315:36:25,  6.18s/it]
 18%|█▊        | 41049/225000 [88:19:38<389:36:33,  7.62s/it]
 18%|█▊        | 41050/225000 [88:19:42<332:03:08,  6.50s/it]
                                                             
{'loss': '0.7064', 'grad_norm': '0.7743', 'learning_rate': '4.747e-05', 'epoch': '0.1824', 'num_input_tokens_seen': 149209080, 'train_runtime': '3.18e+05', 'train_tokens_per_second': '469.2'}

 18%|█▊        | 41050/225000 [88:19:42<332:03:08,  6.50s/it]
 18%|█▊        | 41051/225000 [88:19:50<355:23:57,  6.96s/it]
 18%|█▊        | 41052/225000 [88:19:57<359:50:59,  7.04s/it]
 18%|█▊        | 41053/225000 [88:20:02<327:05:39,  6.40s/it]
 18%|█▊        | 41054/225000 [88:20:07<309:07:33,  6.05s/it]
 18%|█▊        | 41055/225000 [88:20:12<290:20:52,  5.68s/it]
 18%|█▊        | 41056/225000 [88:20:17<289:55:56,  5.67s/it]
 18%|█▊        | 41057/225000 [88:20:30<395:47:18,  7.75s/it]
 18%|█▊        | 41058/225000 [88:20:36<372:58:26,  7.30s/it]
 18%|█▊        | 41059/225000 [88:20:43<367:40:40,  7.20s/it]
 18%|█▊        | 41060/225000 [88:20:48<327:01:30,  6.40s/it]
 18%|█▊        | 41061/225000 [88:20:56<349:27:05,  6.84s/it]
 18%|█▊        | 41062/225000 [88:21:04<366:27:44,  7.17s/it]
 18%|█▊        | 41063/225000 [88:21:09<335:29:28,  6.57s/it]
 18%|█▊        | 41064/225000 [88:21:13<306:26:50,  6.00s/it]
 18%|█▊        | 41065/225000 [88:21:18<286:58:59,  5.62s/it]
 18%|█▊        | 41066/225000 [88:21:23<276:35:20,  5.41s/it]
 18%|█▊        | 41067/225000 [88:21:32<326:03:55,  6.38s/it]
 18%|█▊        | 41068/225000 [88:21:38<331:11:32,  6.48s/it]
 18%|█▊        | 41069/225000 [88:21:48<379:03:42,  7.42s/it]
 18%|█▊        | 41070/225000 [88:21:52<329:49:22,  6.46s/it]
 18%|█▊        | 41071/225000 [88:21:57<302:49:52,  5.93s/it]
 18%|█▊        | 41072/225000 [88:22:06<354:40:20,  6.94s/it]
 18%|█▊        | 41073/225000 [88:22:15<379:04:58,  7.42s/it]
 18%|█▊        | 41074/225000 [88:22:20<347:47:27,  6.81s/it]
 18%|█▊        | 41075/225000 [88:22:26<326:41:28,  6.39s/it]
 18%|█▊        | 41076/225000 [88:22:31<305:08:26,  5.97s/it]
 18%|█▊        | 41077/225000 [88:22:36<293:25:59,  5.74s/it]
 18%|█▊        | 41078/225000 [88:22:42<292:32:52,  5.73s/it]
 18%|█▊        | 41079/225000 [88:22:47<285:10:11,  5.58s/it]
 18%|█▊        | 41080/225000 [88:23:00<400:21:27,  7.84s/it]
 18%|█▊        | 41081/225000 [88:23:06<369:17:13,  7.23s/it]
 18%|█▊        | 41082/225000 [88:23:09<315:48:32,  6.18s/it]
 18%|█▊        | 41083/225000 [88:23:20<387:50:19,  7.59s/it]
 18%|█▊        | 41084/225000 [88:23:28<386:10:06,  7.56s/it]
 18%|█▊        | 41085/225000 [88:23:32<341:31:30,  6.69s/it]
 18%|█▊        | 41086/225000 [88:23:37<314:22:04,  6.15s/it]
 18%|█▊        | 41087/225000 [88:23:42<294:40:39,  5.77s/it]
 18%|█▊        | 41088/225000 [88:23:52<361:10:08,  7.07s/it]
 18%|█▊        | 41089/225000 [88:24:00<373:21:50,  7.31s/it]
 18%|█▊        | 41090/225000 [88:24:05<331:35:18,  6.49s/it]
 18%|█▊        | 41091/225000 [88:24:09<303:31:53,  5.94s/it]
 18%|█▊        | 41092/225000 [88:24:19<360:58:40,  7.07s/it]
 18%|█▊        | 41093/225000 [88:24:26<358:16:40,  7.01s/it]
 18%|█▊        | 41094/225000 [88:24:35<387:35:42,  7.59s/it]
 18%|█▊        | 41095/225000 [88:24:41<357:09:00,  6.99s/it]
 18%|█▊        | 41096/225000 [88:24:46<332:44:10,  6.51s/it]
 18%|█▊        | 41097/225000 [88:24:50<293:00:00,  5.74s/it]
 18%|█▊        | 41098/225000 [88:25:00<354:46:47,  6.95s/it]
 18%|█▊        | 41099/225000 [88:25:03<300:13:28,  5.88s/it]
 18%|█▊        | 41100/225000 [88:25:07<274:04:35,  5.37s/it]
                                                             
{'loss': '0.6802', 'grad_norm': '0.7478', 'learning_rate': '4.747e-05', 'epoch': '0.1827', 'num_input_tokens_seen': 149357600, 'train_runtime': '3.183e+05', 'train_tokens_per_second': '469.2'}

 18%|█▊        | 41100/225000 [88:25:07<274:04:35,  5.37s/it]
 18%|█▊        | 41101/225000 [88:25:15<311:39:09,  6.10s/it]
 18%|█▊        | 41102/225000 [88:25:25<371:43:41,  7.28s/it]
 18%|█▊        | 41103/225000 [88:25:30<330:26:17,  6.47s/it]
 18%|█▊        | 41104/225000 [88:25:34<300:52:15,  5.89s/it]
 18%|█▊        | 41105/225000 [88:25:40<301:34:44,  5.90s/it]
 18%|█▊        | 41106/225000 [88:25:53<404:04:56,  7.91s/it]
 18%|█▊        | 41107/225000 [88:25:57<351:15:01,  6.88s/it]
 18%|█▊        | 41108/225000 [88:26:02<317:30:30,  6.22s/it]
 18%|█▊        | 41109/225000 [88:26:07<299:30:29,  5.86s/it]
 18%|█▊        | 41110/225000 [88:26:16<350:15:11,  6.86s/it]
 18%|█▊        | 41111/225000 [88:26:25<388:10:02,  7.60s/it]
 18%|█▊        | 41112/225000 [88:26:33<392:08:03,  7.68s/it]
 18%|█▊        | 41113/225000 [88:26:40<372:55:15,  7.30s/it]
 18%|█▊        | 41114/225000 [88:26:45<336:15:25,  6.58s/it]
 18%|█▊        | 41115/225000 [88:26:50<318:35:25,  6.24s/it]
 18%|█▊        | 41116/225000 [88:26:56<312:15:04,  6.11s/it]
 18%|█▊        | 41117/225000 [88:27:01<302:36:13,  5.92s/it]
 18%|█▊        | 41118/225000 [88:27:07<294:04:27,  5.76s/it]
 18%|█▊        | 41119/225000 [88:27:12<295:31:28,  5.79s/it]
 18%|█▊        | 41120/225000 [88:27:23<360:25:00,  7.06s/it]
 18%|█▊        | 41121/225000 [88:27:32<395:43:03,  7.75s/it]
 18%|█▊        | 41122/225000 [88:27:36<346:11:24,  6.78s/it]
 18%|█▊        | 41123/225000 [88:27:42<334:37:25,  6.55s/it]
 18%|█▊        | 41124/225000 [88:27:53<400:02:25,  7.83s/it]
 18%|█▊        | 41125/225000 [88:27:59<368:41:58,  7.22s/it]
 18%|█▊        | 41126/225000 [88:28:04<342:08:40,  6.70s/it]
 18%|█▊        | 41127/225000 [88:28:13<364:54:36,  7.14s/it]
 18%|█▊        | 41128/225000 [88:28:20<368:43:20,  7.22s/it]
 18%|█▊        | 41129/225000 [88:28:24<319:10:58,  6.25s/it]
 18%|█▊        | 41130/225000 [88:28:33<363:16:09,  7.11s/it]
 18%|█▊        | 41131/225000 [88:28:38<329:12:24,  6.45s/it]
 18%|█▊        | 41132/225000 [88:28:43<303:39:52,  5.95s/it]
 18%|█▊        | 41133/225000 [88:28:48<287:42:51,  5.63s/it]
 18%|█▊        | 41134/225000 [88:28:54<303:58:41,  5.95s/it]
 18%|█▊        | 41135/225000 [88:29:05<377:57:32,  7.40s/it]
 18%|█▊        | 41136/225000 [88:29:09<324:43:51,  6.36s/it]
 18%|█▊        | 41137/225000 [88:29:14<294:53:32,  5.77s/it]
 18%|█▊        | 41138/225000 [88:29:25<388:12:41,  7.60s/it]
 18%|█▊        | 41139/225000 [88:29:29<326:11:28,  6.39s/it]
 18%|█▊        | 41140/225000 [88:29:37<357:15:32,  7.00s/it]
 18%|█▊        | 41141/225000 [88:29:41<308:31:01,  6.04s/it]
 18%|█▊        | 41142/225000 [88:29:52<379:50:34,  7.44s/it]
 18%|█▊        | 41143/225000 [88:29:56<332:32:56,  6.51s/it]
 18%|█▊        | 41144/225000 [88:30:00<293:56:24,  5.76s/it]
 18%|█▊        | 41145/225000 [88:30:11<367:25:28,  7.19s/it]
 18%|█▊        | 41146/225000 [88:30:14<311:08:16,  6.09s/it]
 18%|█▊        | 41147/225000 [88:30:25<379:15:39,  7.43s/it]
 18%|█▊        | 41148/225000 [88:30:29<332:25:28,  6.51s/it]
 18%|█▊        | 41149/225000 [88:30:36<339:07:50,  6.64s/it]
 18%|█▊        | 41150/225000 [88:30:41<303:31:35,  5.94s/it]
                                                             
{'loss': '0.6709', 'grad_norm': '0.8023', 'learning_rate': '4.746e-05', 'epoch': '0.1829', 'num_input_tokens_seen': 149508024, 'train_runtime': '3.186e+05', 'train_tokens_per_second': '469.2'}

 18%|█▊        | 41150/225000 [88:30:41<303:31:35,  5.94s/it]
 18%|█▊        | 41151/225000 [88:30:51<372:35:10,  7.30s/it]
 18%|█▊        | 41152/225000 [88:30:56<345:35:16,  6.77s/it]
 18%|█▊        | 41153/225000 [88:31:02<332:21:43,  6.51s/it]
 18%|█▊        | 41154/225000 [88:31:08<315:30:13,  6.18s/it]
 18%|█▊        | 41155/225000 [88:31:13<301:56:32,  5.91s/it]
 18%|█▊        | 41156/225000 [88:31:19<306:30:27,  6.00s/it]
 18%|█▊        | 41157/225000 [88:31:25<304:32:41,  5.96s/it]
 18%|█▊        | 41158/225000 [88:31:31<297:10:59,  5.82s/it]
 18%|█▊        | 41159/225000 [88:31:36<290:04:19,  5.68s/it]
 18%|█▊        | 41160/225000 [88:31:42<289:12:49,  5.66s/it]
 18%|█▊        | 41161/225000 [88:31:47<279:48:48,  5.48s/it]
 18%|█▊        | 41162/225000 [88:31:53<298:51:25,  5.85s/it]
 18%|█▊        | 41163/225000 [88:32:04<367:42:14,  7.20s/it]
 18%|█▊        | 41164/225000 [88:32:13<401:58:51,  7.87s/it]
 18%|█▊        | 41165/225000 [88:32:20<385:19:27,  7.55s/it]
 18%|█▊        | 41166/225000 [88:32:26<356:44:56,  6.99s/it]
 18%|█▊        | 41167/225000 [88:32:34<370:45:02,  7.26s/it]
 18%|█▊        | 41168/225000 [88:32:39<347:07:22,  6.80s/it]
 18%|█▊        | 41169/225000 [88:32:46<349:54:36,  6.85s/it]
 18%|█▊        | 41170/225000 [88:32:54<370:36:37,  7.26s/it]
 18%|█▊        | 41171/225000 [88:32:59<328:50:44,  6.44s/it]
 18%|█▊        | 41172/225000 [88:33:04<303:10:51,  5.94s/it]
 18%|█▊        | 41173/225000 [88:33:14<370:16:21,  7.25s/it]
 18%|█▊        | 41174/225000 [88:33:21<362:53:08,  7.11s/it]
 18%|█▊        | 41175/225000 [88:33:28<369:50:30,  7.24s/it]
 18%|█▊        | 41176/225000 [88:33:35<354:43:48,  6.95s/it]
 18%|█▊        | 41177/225000 [88:33:43<383:31:34,  7.51s/it]
 18%|█▊        | 41178/225000 [88:33:47<323:50:51,  6.34s/it]
 18%|█▊        | 41179/225000 [88:33:53<315:43:49,  6.18s/it]
 18%|█▊        | 41180/225000 [88:34:00<331:27:53,  6.49s/it]
 18%|█▊        | 41181/225000 [88:34:06<327:12:28,  6.41s/it]
 18%|█▊        | 41182/225000 [88:34:12<322:10:45,  6.31s/it]
 18%|█▊        | 41183/225000 [88:34:21<356:46:09,  6.99s/it]
 18%|█▊        | 41184/225000 [88:34:26<326:53:47,  6.40s/it]
 18%|█▊        | 41185/225000 [88:34:31<299:47:12,  5.87s/it]
 18%|█▊        | 41186/225000 [88:34:41<375:30:12,  7.35s/it]
 18%|█▊        | 41187/225000 [88:34:46<330:47:00,  6.48s/it]
 18%|█▊        | 41188/225000 [88:34:51<310:10:53,  6.07s/it]
 18%|█▊        | 41189/225000 [88:34:56<296:38:30,  5.81s/it]
 18%|█▊        | 41190/225000 [88:35:08<394:52:41,  7.73s/it]
 18%|█▊        | 41191/225000 [88:35:15<374:17:20,  7.33s/it]
 18%|█▊        | 41192/225000 [88:35:21<354:18:09,  6.94s/it]
 18%|█▊        | 41193/225000 [88:35:29<369:12:45,  7.23s/it]
 18%|█▊        | 41194/225000 [88:35:35<346:52:05,  6.79s/it]
 18%|█▊        | 41195/225000 [88:35:42<361:04:41,  7.07s/it]
 18%|█▊        | 41196/225000 [88:35:47<318:21:21,  6.24s/it]
 18%|█▊        | 41197/225000 [88:35:52<301:06:31,  5.90s/it]
 18%|█▊        | 41198/225000 [88:35:57<288:01:30,  5.64s/it]
 18%|█▊        | 41199/225000 [88:36:02<288:09:50,  5.64s/it]
 18%|█▊        | 41200/225000 [88:36:08<288:27:13,  5.65s/it]
                                                             
{'loss': '0.6937', 'grad_norm': '0.9712', 'learning_rate': '4.745e-05', 'epoch': '0.1831', 'num_input_tokens_seen': 149656368, 'train_runtime': '3.19e+05', 'train_tokens_per_second': '469.2'}

 18%|█▊        | 41200/225000 [88:36:08<288:27:13,  5.65s/it]
 18%|█▊        | 41201/225000 [88:36:14<289:30:04,  5.67s/it]
 18%|█▊        | 41202/225000 [88:36:23<336:42:26,  6.59s/it]
 18%|█▊        | 41203/225000 [88:36:29<340:52:53,  6.68s/it]
 18%|█▊        | 41204/225000 [88:36:39<389:44:54,  7.63s/it]
 18%|█▊        | 41205/225000 [88:36:45<359:50:38,  7.05s/it]
 18%|█▊        | 41206/225000 [88:36:50<325:56:52,  6.38s/it]
 18%|█▊        | 41207/225000 [88:37:02<412:42:16,  8.08s/it]
 18%|█▊        | 41208/225000 [88:37:06<358:31:11,  7.02s/it]
 18%|█▊        | 41209/225000 [88:37:14<372:27:54,  7.30s/it]
 18%|█▊        | 41210/225000 [88:37:22<373:00:55,  7.31s/it]
 18%|█▊        | 41211/225000 [88:37:30<383:58:09,  7.52s/it]
 18%|█▊        | 41212/225000 [88:37:33<323:14:56,  6.33s/it]
 18%|█▊        | 41213/225000 [88:37:43<376:46:31,  7.38s/it]
 18%|█▊        | 41214/225000 [88:37:47<317:40:48,  6.22s/it]
 18%|█▊        | 41215/225000 [88:37:52<306:47:47,  6.01s/it]
 18%|█▊        | 41216/225000 [88:38:01<358:24:34,  7.02s/it]
 18%|█▊        | 41217/225000 [88:38:11<393:56:29,  7.72s/it]
 18%|█▊        | 41218/225000 [88:38:15<333:51:15,  6.54s/it]
 18%|█▊        | 41219/225000 [88:38:24<377:36:57,  7.40s/it]
 18%|█▊        | 41220/225000 [88:38:28<332:52:55,  6.52s/it]
 18%|█▊        | 41221/225000 [88:38:34<310:46:58,  6.09s/it]
 18%|█▊        | 41222/225000 [88:38:44<381:51:24,  7.48s/it]
 18%|█▊        | 41223/225000 [88:38:48<324:20:53,  6.35s/it]
 18%|█▊        | 41224/225000 [88:38:58<375:44:18,  7.36s/it]
 18%|█▊        | 41225/225000 [88:39:03<340:52:52,  6.68s/it]
 18%|█▊        | 41226/225000 [88:39:08<317:06:56,  6.21s/it]
 18%|█▊        | 41227/225000 [88:39:13<298:06:28,  5.84s/it]
 18%|█▊        | 41228/225000 [88:39:26<402:35:19,  7.89s/it]
 18%|█▊        | 41229/225000 [88:39:29<340:40:14,  6.67s/it]
 18%|█▊        | 41230/225000 [88:39:39<379:40:55,  7.44s/it]
 18%|█▊        | 41231/225000 [88:39:43<327:04:22,  6.41s/it]
 18%|█▊        | 41232/225000 [88:39:48<307:59:10,  6.03s/it]
 18%|█▊        | 41233/225000 [88:39:54<306:37:26,  6.01s/it]
 18%|█▊        | 41234/225000 [88:39:59<296:17:07,  5.80s/it]
 18%|█▊        | 41235/225000 [88:40:05<294:58:52,  5.78s/it]
 18%|█▊        | 41236/225000 [88:40:10<291:44:50,  5.72s/it]
 18%|█▊        | 41237/225000 [88:40:16<290:34:33,  5.69s/it]
 18%|█▊        | 41238/225000 [88:40:22<290:23:26,  5.69s/it]
 18%|█▊        | 41239/225000 [88:40:31<346:31:23,  6.79s/it]
 18%|█▊        | 41240/225000 [88:40:43<431:32:19,  8.45s/it]
 18%|█▊        | 41241/225000 [88:40:48<367:42:10,  7.20s/it]
 18%|█▊        | 41242/225000 [88:40:55<366:58:54,  7.19s/it]
 18%|█▊        | 41243/225000 [88:41:02<362:10:00,  7.10s/it]
 18%|█▊        | 41244/225000 [88:41:06<315:57:41,  6.19s/it]
 18%|█▊        | 41245/225000 [88:41:10<288:00:28,  5.64s/it]
 18%|█▊        | 41246/225000 [88:41:22<383:05:16,  7.51s/it]
 18%|█▊        | 41247/225000 [88:41:26<330:38:10,  6.48s/it]
 18%|█▊        | 41248/225000 [88:41:32<319:17:57,  6.26s/it]
 18%|█▊        | 41249/225000 [88:41:38<311:58:37,  6.11s/it]
 18%|█▊        | 41250/225000 [88:41:49<398:08:59,  7.80s/it]
                                                             
{'loss': '0.6986', 'grad_norm': '0.7642', 'learning_rate': '4.744e-05', 'epoch': '0.1833', 'num_input_tokens_seen': 149805512, 'train_runtime': '3.193e+05', 'train_tokens_per_second': '469.2'}

 18%|█▊        | 41250/225000 [88:41:49<398:08:59,  7.80s/it]
 18%|█▊        | 41251/225000 [88:41:53<331:21:59,  6.49s/it]
 18%|█▊        | 41252/225000 [88:42:00<338:30:13,  6.63s/it]
 18%|█▊        | 41253/225000 [88:42:08<360:13:19,  7.06s/it]
 18%|█▊        | 41254/225000 [88:42:12<324:52:12,  6.36s/it]
 18%|█▊        | 41255/225000 [88:42:17<297:18:54,  5.83s/it]
 18%|█▊        | 41256/225000 [88:42:28<377:32:24,  7.40s/it]
 18%|█▊        | 41257/225000 [88:42:33<333:27:11,  6.53s/it]
 18%|█▊        | 41258/225000 [88:42:37<307:31:51,  6.03s/it]
 18%|█▊        | 41259/225000 [88:42:42<288:28:35,  5.65s/it]
 18%|█▊        | 41260/225000 [88:42:47<277:10:22,  5.43s/it]
 18%|█▊        | 41261/225000 [88:42:52<273:54:49,  5.37s/it]
 18%|█▊        | 41262/225000 [88:42:58<270:42:19,  5.30s/it]
 18%|█▊        | 41263/225000 [88:43:11<391:48:54,  7.68s/it]
 18%|█▊        | 41264/225000 [88:43:19<406:08:56,  7.96s/it]
 18%|█▊        | 41265/225000 [88:43:23<344:03:43,  6.74s/it]
 18%|█▊        | 41266/225000 [88:43:33<384:06:43,  7.53s/it]
 18%|█▊        | 41267/225000 [88:43:37<341:39:30,  6.69s/it]
 18%|█▊        | 41268/225000 [88:43:47<390:44:18,  7.66s/it]
 18%|█▊        | 41269/225000 [88:43:53<353:49:46,  6.93s/it]
 18%|█▊        | 41270/225000 [88:43:58<331:35:23,  6.50s/it]
 18%|█▊        | 41271/225000 [88:44:04<318:35:35,  6.24s/it]
 18%|█▊        | 41272/225000 [88:44:09<299:11:16,  5.86s/it]
 18%|█▊        | 41273/225000 [88:44:22<407:20:45,  7.98s/it]
 18%|█▊        | 41274/225000 [88:44:27<375:27:12,  7.36s/it]
 18%|█▊        | 41275/225000 [88:44:32<335:15:17,  6.57s/it]
 18%|█▊        | 41276/225000 [88:44:38<328:55:35,  6.45s/it]
 18%|█▊        | 41277/225000 [88:44:48<370:55:57,  7.27s/it]
 18%|█▊        | 41278/225000 [88:44:54<362:20:33,  7.10s/it]
 18%|█▊        | 41279/225000 [88:45:00<335:41:36,  6.58s/it]
 18%|█▊        | 41280/225000 [88:45:07<353:49:57,  6.93s/it]
 18%|█▊        | 41281/225000 [88:45:12<315:54:36,  6.19s/it]
 18%|█▊        | 41282/225000 [88:45:16<291:26:26,  5.71s/it]
 18%|█▊        | 41283/225000 [88:45:28<378:16:20,  7.41s/it]
 18%|█▊        | 41284/225000 [88:45:32<329:09:41,  6.45s/it]
 18%|█▊        | 41285/225000 [88:45:40<347:57:33,  6.82s/it]
 18%|█▊        | 41286/225000 [88:45:45<324:42:26,  6.36s/it]
 18%|█▊        | 41287/225000 [88:45:50<304:47:10,  5.97s/it]
 18%|█▊        | 41288/225000 [88:45:56<299:44:03,  5.87s/it]
 18%|█▊        | 41289/225000 [88:46:01<287:10:31,  5.63s/it]
 18%|█▊        | 41290/225000 [88:46:13<387:42:53,  7.60s/it]
 18%|█▊        | 41291/225000 [88:46:20<383:50:09,  7.52s/it]
 18%|█▊        | 41292/225000 [88:46:24<332:11:42,  6.51s/it]
 18%|█▊        | 41293/225000 [88:46:30<310:24:42,  6.08s/it]
 18%|█▊        | 41294/225000 [88:46:35<306:00:54,  6.00s/it]
 18%|█▊        | 41295/225000 [88:46:48<414:58:03,  8.13s/it]
 18%|█▊        | 41296/225000 [88:46:55<384:37:29,  7.54s/it]
 18%|█▊        | 41297/225000 [88:47:00<353:22:23,  6.93s/it]
 18%|█▊        | 41298/225000 [88:47:05<324:03:35,  6.35s/it]
 18%|█▊        | 41299/225000 [88:47:10<308:12:50,  6.04s/it]
 18%|█▊        | 41300/225000 [88:47:16<299:33:57,  5.87s/it]
                                                             
{'loss': '0.6962', 'grad_norm': '0.7601', 'learning_rate': '4.743e-05', 'epoch': '0.1836', 'num_input_tokens_seen': 149954976, 'train_runtime': '3.196e+05', 'train_tokens_per_second': '469.1'}

 18%|█▊        | 41300/225000 [88:47:16<299:33:57,  5.87s/it]
 18%|█▊        | 41301/225000 [88:47:21<295:13:20,  5.79s/it]
 18%|█▊        | 41302/225000 [88:47:27<293:15:08,  5.75s/it]
 18%|█▊        | 41303/225000 [88:47:36<336:20:55,  6.59s/it]
 18%|█▊        | 41304/225000 [88:47:47<406:07:49,  7.96s/it]
 18%|█▊        | 41305/225000 [88:47:52<367:55:28,  7.21s/it]
 18%|█▊        | 41306/225000 [88:47:58<345:55:16,  6.78s/it]
 18%|█▊        | 41307/225000 [88:48:06<367:36:39,  7.20s/it]
 18%|█▊        | 41308/225000 [88:48:11<336:36:46,  6.60s/it]
 18%|█▊        | 41309/225000 [88:48:15<295:34:24,  5.79s/it]
 18%|█▊        | 41310/225000 [88:48:26<364:09:25,  7.14s/it]
 18%|█▊        | 41311/225000 [88:48:30<322:41:50,  6.32s/it]
 18%|█▊        | 41312/225000 [88:48:34<291:28:31,  5.71s/it]
 18%|█▊        | 41313/225000 [88:48:40<287:01:20,  5.63s/it]
 18%|█▊        | 41314/225000 [88:48:45<288:45:07,  5.66s/it]
 18%|█▊        | 41315/225000 [88:48:51<293:06:39,  5.74s/it]
 18%|█▊        | 41316/225000 [88:49:04<401:06:18,  7.86s/it]
 18%|█▊        | 41317/225000 [88:49:11<389:49:56,  7.64s/it]
 18%|█▊        | 41318/225000 [88:49:17<352:39:54,  6.91s/it]
 18%|█▊        | 41319/225000 [88:49:21<318:15:30,  6.24s/it]
 18%|█▊        | 41320/225000 [88:49:26<300:13:15,  5.88s/it]
 18%|█▊        | 41321/225000 [88:49:31<287:39:56,  5.64s/it]
 18%|█▊        | 41322/225000 [88:49:44<397:21:56,  7.79s/it]
 18%|█▊        | 41323/225000 [88:49:50<361:52:04,  7.09s/it]
 18%|█▊        | 41324/225000 [88:49:58<376:19:07,  7.38s/it]
 18%|█▊        | 41325/225000 [88:50:03<339:48:24,  6.66s/it]
 18%|█▊        | 41326/225000 [88:50:08<316:03:19,  6.19s/it]
 18%|█▊        | 41327/225000 [88:50:13<298:39:30,  5.85s/it]
 18%|█▊        | 41328/225000 [88:50:18<285:30:35,  5.60s/it]
 18%|█▊        | 41329/225000 [88:50:31<401:29:29,  7.87s/it]
 18%|█▊        | 41330/225000 [88:50:36<362:30:37,  7.11s/it]
 18%|█▊        | 41331/225000 [88:50:45<391:05:47,  7.67s/it]
 18%|█▊        | 41332/225000 [88:50:50<338:27:22,  6.63s/it]
 18%|█▊        | 41333/225000 [88:50:56<338:19:01,  6.63s/it]
 18%|█▊        | 41334/225000 [88:51:05<376:31:35,  7.38s/it]
 18%|█▊        | 41335/225000 [88:51:09<319:13:29,  6.26s/it]
 18%|█▊        | 41336/225000 [88:51:13<291:10:49,  5.71s/it]
 18%|█▊        | 41337/225000 [88:51:22<334:57:23,  6.57s/it]
 18%|█▊        | 41338/225000 [88:51:27<310:56:28,  6.09s/it]
 18%|█▊        | 41339/225000 [88:51:37<371:48:38,  7.29s/it]
 18%|█▊        | 41340/225000 [88:51:40<313:23:33,  6.14s/it]
 18%|█▊        | 41341/225000 [88:51:50<358:51:16,  7.03s/it]
 18%|█▊        | 41342/225000 [88:51:57<363:51:06,  7.13s/it]
 18%|█▊        | 41343/225000 [88:52:01<313:21:01,  6.14s/it]
 18%|█▊        | 41344/225000 [88:52:09<339:19:45,  6.65s/it]
 18%|█▊        | 41345/225000 [88:52:12<296:47:37,  5.82s/it]
 18%|█▊        | 41346/225000 [88:52:21<340:22:36,  6.67s/it]
 18%|█▊        | 41347/225000 [88:52:28<344:12:45,  6.75s/it]
 18%|█▊        | 41348/225000 [88:52:33<310:49:20,  6.09s/it]
 18%|█▊        | 41349/225000 [88:52:41<352:52:58,  6.92s/it]
 18%|█▊        | 41350/225000 [88:52:46<309:53:00,  6.07s/it]
                                                             
{'loss': '0.6842', 'grad_norm': '0.702', 'learning_rate': '4.743e-05', 'epoch': '0.1838', 'num_input_tokens_seen': 150102720, 'train_runtime': '3.2e+05', 'train_tokens_per_second': '469.1'}

 18%|█▊        | 41350/225000 [88:52:46<309:53:00,  6.07s/it]
 18%|█▊        | 41351/225000 [88:52:54<353:39:40,  6.93s/it]
 18%|█▊        | 41352/225000 [88:52:59<319:08:26,  6.26s/it]
 18%|█▊        | 41353/225000 [88:53:03<282:32:10,  5.54s/it]
 18%|█▊        | 41354/225000 [88:53:14<362:04:04,  7.10s/it]
 18%|█▊        | 41355/225000 [88:53:18<323:20:07,  6.34s/it]
 18%|█▊        | 41356/225000 [88:53:23<303:52:39,  5.96s/it]
 18%|█▊        | 41357/225000 [88:53:28<282:36:04,  5.54s/it]
 18%|█▊        | 41358/225000 [88:53:40<375:25:56,  7.36s/it]
 18%|█▊        | 41359/225000 [88:53:45<344:30:21,  6.75s/it]
 18%|█▊        | 41360/225000 [88:53:50<314:27:42,  6.16s/it]
 18%|█▊        | 41361/225000 [88:54:00<378:34:50,  7.42s/it]
 18%|█▊        | 41362/225000 [88:54:04<327:24:45,  6.42s/it]
 18%|█▊        | 41363/225000 [88:54:14<374:14:55,  7.34s/it]
 18%|█▊        | 41364/225000 [88:54:18<323:10:16,  6.34s/it]
 18%|█▊        | 41365/225000 [88:54:24<316:21:00,  6.20s/it]
 18%|█▊        | 41366/225000 [88:54:34<384:15:36,  7.53s/it]
 18%|█▊        | 41367/225000 [88:54:38<327:14:09,  6.42s/it]
 18%|█▊        | 41368/225000 [88:54:46<345:27:39,  6.77s/it]
 18%|█▊        | 41369/225000 [88:54:54<371:10:31,  7.28s/it]
 18%|█▊        | 41370/225000 [88:54:57<311:38:22,  6.11s/it]
 18%|█▊        | 41371/225000 [88:55:03<302:33:38,  5.93s/it]
 18%|█▊        | 41372/225000 [88:55:08<295:16:06,  5.79s/it]
 18%|█▊        | 41373/225000 [88:55:14<289:37:08,  5.68s/it]
 18%|█▊        | 41374/225000 [88:55:23<344:07:23,  6.75s/it]
 18%|█▊        | 41375/225000 [88:55:32<372:15:17,  7.30s/it]
 18%|█▊        | 41376/225000 [88:55:41<401:06:38,  7.86s/it]
 18%|█▊        | 41377/225000 [88:55:45<347:40:01,  6.82s/it]
 18%|█▊        | 41378/225000 [88:55:55<390:44:39,  7.66s/it]
 18%|█▊        | 41379/225000 [88:55:59<335:10:14,  6.57s/it]
 18%|█▊        | 41380/225000 [88:56:03<302:10:41,  5.92s/it]
 18%|█▊        | 41381/225000 [88:56:15<387:11:31,  7.59s/it]
 18%|█▊        | 41382/225000 [88:56:20<354:33:11,  6.95s/it]
 18%|█▊        | 41383/225000 [88:56:26<340:28:04,  6.68s/it]
 18%|█▊        | 41384/225000 [88:56:32<328:32:01,  6.44s/it]
 18%|█▊        | 41385/225000 [88:56:38<318:15:32,  6.24s/it]
 18%|█▊        | 41386/225000 [88:56:44<311:41:46,  6.11s/it]
 18%|█▊        | 41387/225000 [88:56:49<305:44:30,  5.99s/it]
 18%|█▊        | 41388/225000 [88:56:55<293:13:49,  5.75s/it]
 18%|█▊        | 41389/225000 [88:57:00<295:25:04,  5.79s/it]
 18%|█▊        | 41390/225000 [88:57:06<289:38:34,  5.68s/it]
 18%|█▊        | 41391/225000 [88:57:13<314:38:24,  6.17s/it]
 18%|█▊        | 41392/225000 [88:57:26<408:41:40,  8.01s/it]
 18%|█▊        | 41393/225000 [88:57:30<350:32:39,  6.87s/it]
 18%|█▊        | 41394/225000 [88:57:34<313:29:06,  6.15s/it]
 18%|█▊        | 41395/225000 [88:57:46<405:51:58,  7.96s/it]
 18%|█▊        | 41396/225000 [88:57:52<364:09:41,  7.14s/it]
 18%|█▊        | 41397/225000 [88:58:00<382:06:28,  7.49s/it]
 18%|█▊        | 41398/225000 [88:58:05<342:01:35,  6.71s/it]
 18%|█▊        | 41399/225000 [88:58:10<318:12:22,  6.24s/it]
 18%|█▊        | 41400/225000 [88:58:15<306:30:15,  6.01s/it]
                                                             
{'loss': '0.6753', 'grad_norm': '0.6951', 'learning_rate': '4.742e-05', 'epoch': '0.184', 'num_input_tokens_seen': 150252800, 'train_runtime': '3.203e+05', 'train_tokens_per_second': '469.1'}

 18%|█▊        | 41400/225000 [88:58:15<306:30:15,  6.01s/it]
 18%|█▊        | 41401/225000 [88:58:22<319:46:09,  6.27s/it]
 18%|█▊        | 41402/225000 [88:58:34<408:59:39,  8.02s/it]
 18%|█▊        | 41403/225000 [88:58:40<370:08:24,  7.26s/it]
 18%|█▊        | 41404/225000 [88:58:46<345:06:37,  6.77s/it]
 18%|█▊        | 41405/225000 [88:58:51<326:48:20,  6.41s/it]
 18%|█▊        | 41406/225000 [88:58:57<313:23:55,  6.15s/it]
 18%|█▊        | 41407/225000 [88:59:02<299:18:46,  5.87s/it]
 18%|█▊        | 41408/225000 [88:59:07<293:44:00,  5.76s/it]
 18%|█▊        | 41409/225000 [88:59:13<287:22:48,  5.64s/it]
 18%|█▊        | 41410/225000 [88:59:26<401:34:50,  7.87s/it]
 18%|█▊        | 41411/225000 [88:59:33<384:33:01,  7.54s/it]
 18%|█▊        | 41412/225000 [88:59:38<345:41:00,  6.78s/it]
 18%|█▊        | 41413/225000 [88:59:43<328:08:24,  6.43s/it]
 18%|█▊        | 41414/225000 [88:59:51<344:20:46,  6.75s/it]
 18%|█▊        | 41415/225000 [88:59:58<357:12:02,  7.00s/it]
 18%|█▊        | 41416/225000 [89:00:03<322:45:48,  6.33s/it]
 18%|█▊        | 41417/225000 [89:00:10<329:53:41,  6.47s/it]
 18%|█▊        | 41418/225000 [89:00:21<396:37:55,  7.78s/it]
 18%|█▊        | 41419/225000 [89:00:26<362:25:10,  7.11s/it]
 18%|█▊        | 41420/225000 [89:00:32<337:46:57,  6.62s/it]
 18%|█▊        | 41421/225000 [89:00:39<346:29:03,  6.79s/it]
 18%|█▊        | 41422/225000 [89:00:44<323:42:09,  6.35s/it]
 18%|█▊        | 41423/225000 [89:00:50<317:13:04,  6.22s/it]
 18%|█▊        | 41424/225000 [89:01:01<395:51:20,  7.76s/it]
 18%|█▊        | 41425/225000 [89:01:06<348:02:29,  6.83s/it]
 18%|█▊        | 41426/225000 [89:01:11<313:16:26,  6.14s/it]
 18%|█▊        | 41427/225000 [89:01:19<348:19:10,  6.83s/it]
 18%|█▊        | 41428/225000 [89:01:27<363:41:48,  7.13s/it]
 18%|█▊        | 41429/225000 [89:01:33<355:01:02,  6.96s/it]
 18%|█▊        | 41430/225000 [89:01:41<356:01:39,  6.98s/it]
 18%|█▊        | 41431/225000 [89:01:48<364:45:04,  7.15s/it]
 18%|█▊        | 41432/225000 [89:01:53<326:07:05,  6.40s/it]
 18%|█▊        | 41433/225000 [89:01:58<301:54:03,  5.92s/it]
 18%|█▊        | 41434/225000 [89:02:09<380:16:03,  7.46s/it]
 18%|█▊        | 41435/225000 [89:02:14<347:40:07,  6.82s/it]
 18%|█▊        | 41436/225000 [89:02:18<311:40:26,  6.11s/it]
 18%|█▊        | 41437/225000 [89:02:28<365:19:20,  7.16s/it]
 18%|█▊        | 41438/225000 [89:02:37<396:16:41,  7.77s/it]
 18%|█▊        | 41439/225000 [89:02:41<330:53:43,  6.49s/it]
 18%|█▊        | 41440/225000 [89:02:50<374:41:05,  7.35s/it]
 18%|█▊        | 41441/225000 [89:02:55<335:11:36,  6.57s/it]
 18%|█▊        | 41442/225000 [89:03:01<336:21:02,  6.60s/it]
 18%|█▊        | 41443/225000 [89:03:09<345:13:44,  6.77s/it]
 18%|█▊        | 41444/225000 [89:03:14<329:39:21,  6.47s/it]
 18%|█▊        | 41445/225000 [89:03:19<308:47:38,  6.06s/it]
 18%|█▊        | 41446/225000 [89:03:29<362:02:31,  7.10s/it]
 18%|█▊        | 41447/225000 [89:03:33<312:56:27,  6.14s/it]
 18%|█▊        | 41448/225000 [89:03:39<308:18:54,  6.05s/it]
 18%|█▊        | 41449/225000 [89:03:48<358:58:43,  7.04s/it]
 18%|█▊        | 41450/225000 [89:03:52<315:39:23,  6.19s/it]
                                                             
{'loss': '0.6837', 'grad_norm': '0.8205', 'learning_rate': '4.741e-05', 'epoch': '0.1842', 'num_input_tokens_seen': 150401864, 'train_runtime': '3.206e+05', 'train_tokens_per_second': '469.1'}

 18%|█▊        | 41450/225000 [89:03:52<315:39:23,  6.19s/it]
 18%|█▊        | 41451/225000 [89:03:57<293:44:01,  5.76s/it]
 18%|█▊        | 41452/225000 [89:04:02<283:06:26,  5.55s/it]
 18%|█▊        | 41453/225000 [89:04:15<389:02:41,  7.63s/it]
 18%|█▊        | 41454/225000 [89:04:19<334:41:49,  6.56s/it]
 18%|█▊        | 41455/225000 [89:04:28<375:31:05,  7.37s/it]
 18%|█▊        | 41456/225000 [89:04:31<317:23:08,  6.23s/it]
 18%|█▊        | 41457/225000 [89:04:40<353:24:40,  6.93s/it]
 18%|█▊        | 41458/225000 [89:04:45<322:03:32,  6.32s/it]
 18%|█▊        | 41459/225000 [89:04:50<303:36:00,  5.95s/it]
 18%|█▊        | 41460/225000 [89:04:55<290:12:05,  5.69s/it]
 18%|█▊        | 41461/225000 [89:05:00<283:48:50,  5.57s/it]
 18%|█▊        | 41462/225000 [89:05:12<371:54:05,  7.29s/it]
 18%|█▊        | 41463/225000 [89:05:19<376:24:34,  7.38s/it]
 18%|█▊        | 41464/225000 [89:05:25<347:06:35,  6.81s/it]
 18%|█▊        | 41465/225000 [89:05:34<376:43:05,  7.39s/it]
 18%|█▊        | 41466/225000 [89:05:38<336:46:20,  6.61s/it]
 18%|█▊        | 41467/225000 [89:05:46<358:38:56,  7.03s/it]
 18%|█▊        | 41468/225000 [89:05:54<371:10:59,  7.28s/it]
 18%|█▊        | 41469/225000 [89:06:00<344:37:19,  6.76s/it]
 18%|█▊        | 41470/225000 [89:06:07<355:13:50,  6.97s/it]
 18%|█▊        | 41471/225000 [89:06:13<340:23:36,  6.68s/it]
 18%|█▊        | 41472/225000 [89:06:20<345:49:56,  6.78s/it]
 18%|█▊        | 41473/225000 [89:06:25<314:01:41,  6.16s/it]
 18%|█▊        | 41474/225000 [89:06:30<291:05:04,  5.71s/it]
 18%|█▊        | 41475/225000 [89:06:35<280:24:45,  5.50s/it]
 18%|█▊        | 41476/225000 [89:06:46<363:33:16,  7.13s/it]
 18%|█▊        | 41477/225000 [89:06:52<350:56:01,  6.88s/it]
 18%|█▊        | 41478/225000 [89:07:00<376:34:42,  7.39s/it]
 18%|█▊        | 41479/225000 [89:07:04<325:10:08,  6.38s/it]
 18%|█▊        | 41480/225000 [89:07:08<288:01:20,  5.65s/it]
 18%|█▊        | 41481/225000 [89:07:17<339:19:47,  6.66s/it]
 18%|█▊        | 41482/225000 [89:07:24<336:44:47,  6.61s/it]
 18%|█▊        | 41483/225000 [89:07:31<341:25:00,  6.70s/it]
 18%|█▊        | 41484/225000 [89:07:40<373:52:34,  7.33s/it]
 18%|█▊        | 41485/225000 [89:07:46<366:33:43,  7.19s/it]
 18%|█▊        | 41486/225000 [89:07:51<326:00:43,  6.40s/it]
 18%|█▊        | 41487/225000 [89:07:55<293:43:03,  5.76s/it]
 18%|█▊        | 41488/225000 [89:08:08<392:54:33,  7.71s/it]
 18%|█▊        | 41489/225000 [89:08:12<336:27:12,  6.60s/it]
 18%|█▊        | 41490/225000 [89:08:17<323:46:22,  6.35s/it]
 18%|█▊        | 41491/225000 [89:08:25<349:19:28,  6.85s/it]
 18%|█▊        | 41492/225000 [89:08:33<358:01:15,  7.02s/it]
 18%|█▊        | 41493/225000 [89:08:37<308:18:38,  6.05s/it]
 18%|█▊        | 41494/225000 [89:08:46<363:29:12,  7.13s/it]
 18%|█▊        | 41495/225000 [89:08:51<320:29:56,  6.29s/it]
 18%|█▊        | 41496/225000 [89:08:55<291:00:08,  5.71s/it]
 18%|█▊        | 41497/225000 [89:09:07<394:54:52,  7.75s/it]
 18%|█▊        | 41498/225000 [89:09:12<343:10:02,  6.73s/it]
 18%|█▊        | 41499/225000 [89:09:18<337:16:28,  6.62s/it]
 18%|█▊        | 41500/225000 [89:09:27<374:30:19,  7.35s/it]
                                                             
{'loss': '0.6822', 'grad_norm': '0.7901', 'learning_rate': '4.74e-05', 'epoch': '0.1844', 'num_input_tokens_seen': 150551200, 'train_runtime': '3.21e+05', 'train_tokens_per_second': '469.1'}

 18%|█▊        | 41500/225000 [89:09:27<374:30:19,  7.35s/it]
 18%|█▊        | 41501/225000 [89:09:32<331:44:46,  6.51s/it]
 18%|█▊        | 41502/225000 [89:09:41<376:17:23,  7.38s/it]
 18%|█▊        | 41503/225000 [89:09:46<336:41:56,  6.61s/it]
 18%|█▊        | 41504/225000 [89:09:54<352:19:13,  6.91s/it]
 18%|█▊        | 41505/225000 [89:09:58<309:41:28,  6.08s/it]
 18%|█▊        | 41506/225000 [89:10:08<371:41:35,  7.29s/it]
 18%|█▊        | 41507/225000 [89:10:14<360:17:59,  7.07s/it]
 18%|█▊        | 41508/225000 [89:10:20<340:08:15,  6.67s/it]
 18%|█▊        | 41509/225000 [89:10:26<325:18:47,  6.38s/it]
 18%|█▊        | 41510/225000 [89:10:31<315:05:31,  6.18s/it]
 18%|█▊        | 41511/225000 [89:10:37<301:51:53,  5.92s/it]
 18%|█▊        | 41512/225000 [89:10:42<296:00:09,  5.81s/it]
 18%|█▊        | 41513/225000 [89:10:49<313:29:15,  6.15s/it]
 18%|█▊        | 41514/225000 [89:11:02<418:59:43,  8.22s/it]
 18%|█▊        | 41515/225000 [89:11:08<376:27:26,  7.39s/it]
 18%|█▊        | 41516/225000 [89:11:12<327:28:49,  6.43s/it]
 18%|█▊        | 41517/225000 [89:11:17<300:03:21,  5.89s/it]
 18%|█▊        | 41518/225000 [89:11:29<393:47:49,  7.73s/it]
 18%|█▊        | 41519/225000 [89:11:34<363:18:36,  7.13s/it]
 18%|█▊        | 41520/225000 [89:11:40<347:29:08,  6.82s/it]
 18%|█▊        | 41521/225000 [89:11:50<384:09:37,  7.54s/it]
 18%|█▊        | 41522/225000 [89:11:54<335:21:51,  6.58s/it]
 18%|█▊        | 41523/225000 [89:11:59<315:05:53,  6.18s/it]
 18%|█▊        | 41524/225000 [89:12:09<369:09:04,  7.24s/it]
 18%|█▊        | 41525/225000 [89:12:17<379:47:21,  7.45s/it]
 18%|█▊        | 41526/225000 [89:12:24<376:09:43,  7.38s/it]
 18%|█▊        | 41527/225000 [89:12:27<314:12:39,  6.17s/it]
 18%|█▊        | 41528/225000 [89:12:37<359:49:01,  7.06s/it]
 18%|█▊        | 41529/225000 [89:12:44<364:16:35,  7.15s/it]
 18%|█▊        | 41530/225000 [89:12:49<327:40:59,  6.43s/it]
 18%|█▊        | 41531/225000 [89:12:57<357:20:13,  7.01s/it]
 18%|█▊        | 41532/225000 [89:13:02<322:17:12,  6.32s/it]
 18%|█▊        | 41533/225000 [89:13:07<299:50:22,  5.88s/it]
 18%|█▊        | 41534/225000 [89:13:15<334:11:21,  6.56s/it]
 18%|█▊        | 41535/225000 [89:13:23<361:37:40,  7.10s/it]
 18%|█▊        | 41536/225000 [89:13:28<319:56:49,  6.28s/it]
 18%|█▊        | 41537/225000 [89:13:32<291:46:47,  5.73s/it]
 18%|█▊        | 41538/225000 [89:13:43<367:49:21,  7.22s/it]
 18%|█▊        | 41539/225000 [89:13:46<316:06:19,  6.20s/it]
 18%|█▊        | 41540/225000 [89:13:52<300:40:54,  5.90s/it]
 18%|█▊        | 41541/225000 [89:14:01<358:56:13,  7.04s/it]
 18%|█▊        | 41542/225000 [89:14:06<314:39:57,  6.17s/it]
 18%|█▊        | 41543/225000 [89:14:16<378:43:17,  7.43s/it]
 18%|█▊        | 41544/225000 [89:14:21<336:47:18,  6.61s/it]
 18%|█▊        | 41545/225000 [89:14:25<305:10:19,  5.99s/it]
 18%|█▊        | 41546/225000 [89:14:30<283:53:05,  5.57s/it]
 18%|█▊        | 41547/225000 [89:14:40<348:16:28,  6.83s/it]
 18%|█▊        | 41548/225000 [89:14:49<394:10:03,  7.74s/it]
 18%|█▊        | 41549/225000 [89:14:55<354:52:43,  6.96s/it]
 18%|█▊        | 41550/225000 [89:14:59<324:10:14,  6.36s/it]
                                                             
{'loss': '0.6998', 'grad_norm': '0.8689', 'learning_rate': '4.739e-05', 'epoch': '0.1847', 'num_input_tokens_seen': 150699824, 'train_runtime': '3.213e+05', 'train_tokens_per_second': '469'}

 18%|█▊        | 41550/225000 [89:14:59<324:10:14,  6.36s/it]
 18%|█▊        | 41551/225000 [89:15:04<299:51:50,  5.88s/it]
 18%|█▊        | 41552/225000 [89:15:16<386:52:00,  7.59s/it]
 18%|█▊        | 41553/225000 [89:15:22<366:06:40,  7.18s/it]
 18%|█▊        | 41554/225000 [89:15:26<315:03:37,  6.18s/it]
 18%|█▊        | 41555/225000 [89:15:36<381:30:40,  7.49s/it]
 18%|█▊        | 41556/225000 [89:15:41<333:31:21,  6.55s/it]
 18%|█▊        | 41557/225000 [89:15:45<298:35:27,  5.86s/it]
 18%|█▊        | 41558/225000 [89:15:56<369:15:22,  7.25s/it]
 18%|█▊        | 41559/225000 [89:15:59<316:28:42,  6.21s/it]
 18%|█▊        | 41560/225000 [89:16:08<350:14:24,  6.87s/it]
 18%|█▊        | 41561/225000 [89:16:15<358:02:32,  7.03s/it]
 18%|█▊        | 41562/225000 [89:16:20<320:21:42,  6.29s/it]
 18%|█▊        | 41563/225000 [89:16:25<305:16:30,  5.99s/it]
 18%|█▊        | 41564/225000 [89:16:36<376:30:53,  7.39s/it]
 18%|█▊        | 41565/225000 [89:16:40<336:07:18,  6.60s/it]
 18%|█▊        | 41566/225000 [89:16:45<313:23:45,  6.15s/it]
 18%|█▊        | 41567/225000 [89:16:50<288:37:10,  5.66s/it]
 18%|█▊        | 41568/225000 [89:16:55<277:32:33,  5.45s/it]
 18%|█▊        | 41569/225000 [89:17:00<274:49:06,  5.39s/it]
 18%|█▊        | 41570/225000 [89:17:06<278:58:09,  5.48s/it]
 18%|█▊        | 41571/225000 [89:17:16<351:54:30,  6.91s/it]
 18%|█▊        | 41572/225000 [89:17:27<416:51:26,  8.18s/it]
 18%|█▊        | 41573/225000 [89:17:32<358:39:29,  7.04s/it]
 18%|█▊        | 41574/225000 [89:17:37<328:40:17,  6.45s/it]
 18%|█▊        | 41575/225000 [89:17:42<306:49:52,  6.02s/it]
 18%|█▊        | 41576/225000 [89:17:47<301:40:15,  5.92s/it]
 18%|█▊        | 41577/225000 [89:17:53<292:51:22,  5.75s/it]
 18%|█▊        | 41578/225000 [89:17:59<292:52:32,  5.75s/it]
 18%|█▊        | 41579/225000 [89:18:12<408:59:34,  8.03s/it]
 18%|█▊        | 41580/225000 [89:18:21<425:20:40,  8.35s/it]
 18%|█▊        | 41581/225000 [89:18:26<380:37:08,  7.47s/it]
 18%|█▊        | 41582/225000 [89:18:32<345:16:36,  6.78s/it]
 18%|█▊        | 41583/225000 [89:18:37<320:24:30,  6.29s/it]
 18%|█▊        | 41584/225000 [89:18:42<309:38:51,  6.08s/it]
 18%|█▊        | 41585/225000 [89:18:47<293:28:05,  5.76s/it]
 18%|█▊        | 41586/225000 [89:19:00<401:10:53,  7.87s/it]
 18%|█▊        | 41587/225000 [89:19:08<399:01:17,  7.83s/it]
 18%|█▊        | 41588/225000 [89:19:15<382:31:04,  7.51s/it]
 18%|█▊        | 41589/225000 [89:19:19<330:25:33,  6.49s/it]
 18%|█▊        | 41590/225000 [89:19:28<379:35:11,  7.45s/it]
 18%|█▊        | 41591/225000 [89:19:34<350:20:16,  6.88s/it]
 18%|█▊        | 41592/225000 [89:19:40<342:12:41,  6.72s/it]
 18%|█▊        | 41593/225000 [89:19:45<305:50:21,  6.00s/it]
 18%|█▊        | 41594/225000 [89:19:51<309:19:01,  6.07s/it]
 18%|█▊        | 41595/225000 [89:20:02<380:27:33,  7.47s/it]
 18%|█▊        | 41596/225000 [89:20:06<329:34:03,  6.47s/it]
 18%|█▊        | 41597/225000 [89:20:10<297:37:46,  5.84s/it]
 18%|█▊        | 41598/225000 [89:20:19<342:17:12,  6.72s/it]
 18%|█▊        | 41599/225000 [89:20:28<375:53:56,  7.38s/it]
 18%|█▊        | 41600/225000 [89:20:32<328:45:53,  6.45s/it]
                                                             
{'loss': '0.6911', 'grad_norm': '0.7456', 'learning_rate': '4.739e-05', 'epoch': '0.1849', 'num_input_tokens_seen': 150849168, 'train_runtime': '3.216e+05', 'train_tokens_per_second': '469'}

 18%|█▊        | 41600/225000 [89:20:32<328:45:53,  6.45s/it]
 18%|█▊        | 41601/225000 [89:20:36<296:19:17,  5.82s/it]
 18%|█▊        | 41602/225000 [89:20:49<394:45:56,  7.75s/it]
 18%|█▊        | 41603/225000 [89:20:52<329:59:25,  6.48s/it]
 18%|█▊        | 41604/225000 [89:21:01<363:43:07,  7.14s/it]
 18%|█▊        | 41605/225000 [89:21:05<322:05:04,  6.32s/it]
 18%|█▊        | 41606/225000 [89:21:10<295:26:18,  5.80s/it]
 18%|█▊        | 41607/225000 [89:21:20<356:58:46,  7.01s/it]
 18%|█▊        | 41608/225000 [89:21:24<311:07:34,  6.11s/it]
 18%|█▊        | 41609/225000 [89:21:30<313:12:49,  6.15s/it]
 18%|█▊        | 41610/225000 [89:21:41<388:02:00,  7.62s/it]
 18%|█▊        | 41611/225000 [89:21:45<338:09:07,  6.64s/it]
 18%|█▊        | 41612/225000 [89:21:51<321:27:44,  6.31s/it]
 18%|█▊        | 41613/225000 [89:22:01<385:36:37,  7.57s/it]
 18%|█▊        | 41614/225000 [89:22:05<330:33:31,  6.49s/it]
 18%|█▊        | 41615/225000 [89:22:14<356:07:16,  6.99s/it]
 18%|█▊        | 41616/225000 [89:22:18<321:27:30,  6.31s/it]
 18%|█▊        | 41617/225000 [89:22:26<341:49:15,  6.71s/it]
 18%|█▊        | 41618/225000 [89:22:34<360:33:35,  7.08s/it]
 18%|█▊        | 41619/225000 [89:22:39<324:15:11,  6.37s/it]
 18%|█▊        | 41620/225000 [89:22:43<296:42:49,  5.82s/it]
 18%|█▊        | 41621/225000 [89:22:48<287:34:59,  5.65s/it]
 18%|█▊        | 41622/225000 [89:22:58<355:16:20,  6.97s/it]
 18%|█▊        | 41623/225000 [89:23:06<361:29:39,  7.10s/it]
 18%|█▊        | 41624/225000 [89:23:13<365:23:20,  7.17s/it]
 18%|█▊        | 41625/225000 [89:23:20<360:21:12,  7.07s/it]
 19%|█▊        | 41626/225000 [89:23:27<365:58:23,  7.18s/it]
 19%|█▊        | 41627/225000 [89:23:32<326:25:52,  6.41s/it]
 19%|█▊        | 41628/225000 [89:23:42<378:49:35,  7.44s/it]
 19%|█▊        | 41629/225000 [89:23:45<320:29:37,  6.29s/it]
 19%|█▊        | 41630/225000 [89:23:56<382:24:17,  7.51s/it]
 19%|█▊        | 41631/225000 [89:24:01<352:46:19,  6.93s/it]
 19%|█▊        | 41632/225000 [89:24:08<351:58:54,  6.91s/it]
 19%|█▊        | 41633/225000 [89:24:12<305:05:05,  5.99s/it]
 19%|█▊        | 41634/225000 [89:24:22<357:39:35,  7.02s/it]
 19%|█▊        | 41635/225000 [89:24:27<338:59:18,  6.66s/it]
 19%|█▊        | 41636/225000 [89:24:36<362:20:16,  7.11s/it]
 19%|█▊        | 41637/225000 [89:24:39<309:23:19,  6.07s/it]
 19%|█▊        | 41638/225000 [89:24:48<347:24:10,  6.82s/it]
 19%|█▊        | 41639/225000 [89:24:53<319:24:19,  6.27s/it]
 19%|█▊        | 41640/225000 [89:24:58<298:38:03,  5.86s/it]
 19%|█▊        | 41641/225000 [89:25:03<288:34:29,  5.67s/it]
 19%|█▊        | 41642/225000 [89:25:08<286:54:01,  5.63s/it]
 19%|█▊        | 41643/225000 [89:25:14<291:56:58,  5.73s/it]
 19%|█▊        | 41644/225000 [89:25:28<416:41:06,  8.18s/it]
 19%|█▊        | 41645/225000 [89:25:32<354:52:55,  6.97s/it]
 19%|█▊        | 41646/225000 [89:25:41<375:48:37,  7.38s/it]
 19%|█▊        | 41647/225000 [89:25:45<333:37:35,  6.55s/it]
 19%|█▊        | 41648/225000 [89:25:55<380:54:49,  7.48s/it]
 19%|█▊        | 41649/225000 [89:26:01<353:55:53,  6.95s/it]
 19%|█▊        | 41650/225000 [89:26:05<311:40:20,  6.12s/it]
                                                             
{'loss': '0.6914', 'grad_norm': '0.7524', 'learning_rate': '4.738e-05', 'epoch': '0.1851', 'num_input_tokens_seen': 150997640, 'train_runtime': '3.22e+05', 'train_tokens_per_second': '469'}

 19%|█▊        | 41650/225000 [89:26:05<311:40:20,  6.12s/it]
 19%|█▊        | 41651/225000 [89:26:10<298:15:07,  5.86s/it]
 19%|█▊        | 41652/225000 [89:26:20<359:59:55,  7.07s/it]
 19%|█▊        | 41653/225000 [89:26:24<319:50:40,  6.28s/it]
 19%|█▊        | 41654/225000 [89:26:32<334:00:29,  6.56s/it]
 19%|█▊        | 41655/225000 [89:26:38<335:10:12,  6.58s/it]
 19%|█▊        | 41656/225000 [89:26:43<307:20:43,  6.03s/it]
 19%|█▊        | 41657/225000 [89:26:53<365:35:23,  7.18s/it]
 19%|█▊        | 41658/225000 [89:26:58<334:23:59,  6.57s/it]
 19%|█▊        | 41659/225000 [89:27:07<363:29:21,  7.14s/it]
 19%|█▊        | 41660/225000 [89:27:11<319:56:38,  6.28s/it]
 19%|█▊        | 41661/225000 [89:27:16<298:23:02,  5.86s/it]
 19%|█▊        | 41662/225000 [89:27:25<349:29:35,  6.86s/it]
 19%|█▊        | 41663/225000 [89:27:35<398:52:08,  7.83s/it]
 19%|█▊        | 41664/225000 [89:27:39<345:39:58,  6.79s/it]
 19%|█▊        | 41665/225000 [89:27:43<303:37:56,  5.96s/it]
 19%|█▊        | 41666/225000 [89:27:53<354:46:40,  6.97s/it]
 19%|█▊        | 41667/225000 [89:28:00<360:24:13,  7.08s/it]
 19%|█▊        | 41668/225000 [89:28:04<315:52:27,  6.20s/it]
 19%|█▊        | 41669/225000 [89:28:12<337:16:05,  6.62s/it]
 19%|█▊        | 41670/225000 [89:28:21<370:50:10,  7.28s/it]
 19%|█▊        | 41671/225000 [89:28:25<323:23:06,  6.35s/it]
 19%|█▊        | 41672/225000 [89:28:31<323:27:28,  6.35s/it]
 19%|█▊        | 41673/225000 [89:28:41<380:25:23,  7.47s/it]
 19%|█▊        | 41674/225000 [89:28:45<322:55:49,  6.34s/it]
 19%|█▊        | 41675/225000 [89:28:53<355:01:41,  6.97s/it]
 19%|█▊        | 41676/225000 [89:28:58<319:19:45,  6.27s/it]
 19%|█▊        | 41677/225000 [89:29:03<293:29:34,  5.76s/it]
 19%|█▊        | 41678/225000 [89:29:12<355:06:01,  6.97s/it]
 19%|█▊        | 41679/225000 [89:29:20<368:12:10,  7.23s/it]
 19%|█▊        | 41680/225000 [89:29:27<359:19:11,  7.06s/it]
 19%|█▊        | 41681/225000 [89:29:32<335:45:44,  6.59s/it]
 19%|█▊        | 41682/225000 [89:29:38<315:41:21,  6.20s/it]
 19%|█▊        | 41683/225000 [89:29:43<302:21:12,  5.94s/it]
 19%|█▊        | 41684/225000 [89:29:55<394:39:35,  7.75s/it]
 19%|█▊        | 41685/225000 [89:30:02<389:09:08,  7.64s/it]
 19%|█▊        | 41686/225000 [89:30:06<323:27:50,  6.35s/it]
 19%|█▊        | 41687/225000 [89:30:14<347:47:41,  6.83s/it]
 19%|█▊        | 41688/225000 [89:30:21<350:33:42,  6.88s/it]
 19%|█▊        | 41689/225000 [89:30:25<307:59:52,  6.05s/it]
 19%|█▊        | 41690/225000 [89:30:32<320:44:37,  6.30s/it]
 19%|█▊        | 41691/225000 [89:30:39<338:23:41,  6.65s/it]
 19%|█▊        | 41692/225000 [89:30:44<304:47:43,  5.99s/it]
 19%|█▊        | 41693/225000 [89:30:48<280:34:59,  5.51s/it]
 19%|█▊        | 41694/225000 [89:31:01<392:01:22,  7.70s/it]
 19%|█▊        | 41695/225000 [89:31:04<328:22:33,  6.45s/it]
 19%|█▊        | 41696/225000 [89:31:09<309:05:38,  6.07s/it]
 19%|█▊        | 41697/225000 [89:31:21<389:01:08,  7.64s/it]
 19%|█▊        | 41698/225000 [89:31:24<328:06:05,  6.44s/it]
 19%|█▊        | 41699/225000 [89:31:30<319:59:33,  6.28s/it]
 19%|█▊        | 41700/225000 [89:31:40<368:39:22,  7.24s/it]
                                                             
{'loss': '0.7021', 'grad_norm': '0.7936', 'learning_rate': '4.737e-05', 'epoch': '0.1853', 'num_input_tokens_seen': 151146704, 'train_runtime': '3.223e+05', 'train_tokens_per_second': '469'}

 19%|█▊        | 41700/225000 [89:31:40<368:39:22,  7.24s/it]
 19%|█▊        | 41701/225000 [89:31:44<316:04:42,  6.21s/it]
 19%|█▊        | 41702/225000 [89:31:53<361:20:33,  7.10s/it]
 19%|█▊        | 41703/225000 [89:31:59<342:27:41,  6.73s/it]
 19%|█▊        | 41704/225000 [89:32:03<299:25:24,  5.88s/it]
 19%|█▊        | 41705/225000 [89:32:07<276:07:49,  5.42s/it]
 19%|█▊        | 41706/225000 [89:32:13<280:50:31,  5.52s/it]
 19%|█▊        | 41707/225000 [89:32:18<272:21:50,  5.35s/it]
 19%|█▊        | 41708/225000 [89:32:29<359:37:31,  7.06s/it]
 19%|█▊        | 41709/225000 [89:32:40<418:11:48,  8.21s/it]
 19%|█▊        | 41710/225000 [89:32:44<362:05:37,  7.11s/it]
 19%|█▊        | 41711/225000 [89:32:50<339:05:08,  6.66s/it]
 19%|█▊        | 41712/225000 [89:33:00<389:13:27,  7.64s/it]
 19%|█▊        | 41713/225000 [89:33:05<351:24:14,  6.90s/it]
 19%|█▊        | 41714/225000 [89:33:10<332:46:41,  6.54s/it]
 19%|█▊        | 41715/225000 [89:33:16<319:26:49,  6.27s/it]
 19%|█▊        | 41716/225000 [89:33:22<314:53:00,  6.18s/it]
 19%|█▊        | 41717/225000 [89:33:28<305:57:40,  6.01s/it]
 19%|█▊        | 41718/225000 [89:33:35<323:38:39,  6.36s/it]
 19%|█▊        | 41719/225000 [89:33:49<435:20:44,  8.55s/it]
 19%|█▊        | 41720/225000 [89:33:53<380:14:18,  7.47s/it]
 19%|█▊        | 41721/225000 [89:34:01<388:13:21,  7.63s/it]
 19%|█▊        | 41722/225000 [89:34:09<381:24:30,  7.49s/it]
 19%|█▊        | 41723/225000 [89:34:15<362:17:07,  7.12s/it]
 19%|█▊        | 41724/225000 [89:34:20<337:46:54,  6.63s/it]
 19%|█▊        | 41725/225000 [89:34:26<321:16:49,  6.31s/it]
 19%|█▊        | 41726/225000 [89:34:31<304:38:59,  5.98s/it]
 19%|█▊        | 41727/225000 [89:34:36<292:23:15,  5.74s/it]
 19%|█▊        | 41728/225000 [89:34:41<280:31:34,  5.51s/it]
 19%|█▊        | 41729/225000 [89:34:52<360:15:43,  7.08s/it]
 19%|█▊        | 41730/225000 [89:35:01<394:45:29,  7.75s/it]
 19%|█▊        | 41731/225000 [89:35:07<362:01:13,  7.11s/it]
 19%|█▊        | 41732/225000 [89:35:14<366:05:36,  7.19s/it]
 19%|█▊        | 41733/225000 [89:35:18<317:20:59,  6.23s/it]
 19%|█▊        | 41734/225000 [89:35:29<378:21:17,  7.43s/it]
 19%|█▊        | 41735/225000 [89:35:32<321:56:19,  6.32s/it]
 19%|█▊        | 41736/225000 [89:35:40<345:14:46,  6.78s/it]
 19%|█▊        | 41737/225000 [89:35:47<340:40:37,  6.69s/it]
 19%|█▊        | 41738/225000 [89:35:51<311:44:24,  6.12s/it]
 19%|█▊        | 41739/225000 [89:35:57<295:28:29,  5.80s/it]
 19%|█▊        | 41740/225000 [89:36:02<296:56:34,  5.83s/it]
 19%|█▊        | 41741/225000 [89:36:15<406:24:27,  7.98s/it]
 19%|█▊        | 41742/225000 [89:36:20<349:37:06,  6.87s/it]
 19%|█▊        | 41743/225000 [89:36:26<339:06:23,  6.66s/it]
 19%|█▊        | 41744/225000 [89:36:33<346:20:21,  6.80s/it]
 19%|█▊        | 41745/225000 [89:36:39<328:11:15,  6.45s/it]
 19%|█▊        | 41746/225000 [89:36:43<302:11:13,  5.94s/it]
 19%|█▊        | 41747/225000 [89:36:48<285:38:56,  5.61s/it]
 19%|█▊        | 41748/225000 [89:36:54<289:36:03,  5.69s/it]
 19%|█▊        | 41749/225000 [89:37:07<401:52:54,  7.90s/it]
 19%|█▊        | 41750/225000 [89:37:11<341:09:38,  6.70s/it]
                                                             
{'loss': '0.6945', 'grad_norm': '0.7597', 'learning_rate': '4.736e-05', 'epoch': '0.1856', 'num_input_tokens_seen': 151295896, 'train_runtime': '3.226e+05', 'train_tokens_per_second': '468.9'}

 19%|█▊        | 41750/225000 [89:37:11<341:09:38,  6.70s/it]
 19%|█▊        | 41751/225000 [89:37:20<376:49:45,  7.40s/it]
 19%|█▊        | 41752/225000 [89:37:24<319:30:38,  6.28s/it]
 19%|█▊        | 41753/225000 [89:37:32<353:33:26,  6.95s/it]
 19%|█▊        | 41754/225000 [89:37:40<371:08:44,  7.29s/it]
 19%|█▊        | 41755/225000 [89:37:44<320:10:18,  6.29s/it]
 19%|█▊        | 41756/225000 [89:37:49<299:26:05,  5.88s/it]
 19%|█▊        | 41757/225000 [89:37:55<301:15:08,  5.92s/it]
 19%|█▊        | 41758/225000 [89:38:01<299:03:02,  5.88s/it]
 19%|█▊        | 41759/225000 [89:38:07<294:32:05,  5.79s/it]
 19%|█▊        | 41760/225000 [89:38:13<309:20:41,  6.08s/it]
 19%|█▊        | 41761/225000 [89:38:27<429:25:30,  8.44s/it]
 19%|█▊        | 41762/225000 [89:38:33<386:38:37,  7.60s/it]
 19%|█▊        | 41763/225000 [89:38:39<367:22:41,  7.22s/it]
 19%|█▊        | 41764/225000 [89:38:50<414:01:58,  8.13s/it]
 19%|█▊        | 41765/225000 [89:38:55<379:33:18,  7.46s/it]
 19%|█▊        | 41766/225000 [89:39:01<344:05:59,  6.76s/it]
 19%|█▊        | 41767/225000 [89:39:06<322:38:58,  6.34s/it]
 19%|█▊        | 41768/225000 [89:39:11<305:59:26,  6.01s/it]
 19%|█▊        | 41769/225000 [89:39:24<410:23:42,  8.06s/it]
 19%|█▊        | 41770/225000 [89:39:29<366:11:36,  7.19s/it]
 19%|█▊        | 41771/225000 [89:39:34<329:17:47,  6.47s/it]
 19%|█▊        | 41772/225000 [89:39:39<308:50:19,  6.07s/it]
 19%|█▊        | 41773/225000 [89:39:44<288:43:14,  5.67s/it]
 19%|█▊        | 41774/225000 [89:39:51<303:53:05,  5.97s/it]
 19%|█▊        | 41775/225000 [89:40:04<411:37:17,  8.09s/it]
 19%|█▊        | 41776/225000 [89:40:07<344:10:47,  6.76s/it]
 19%|█▊        | 41777/225000 [89:40:17<391:16:36,  7.69s/it]
 19%|█▊        | 41778/225000 [89:40:23<369:44:41,  7.26s/it]
 19%|█▊        | 41779/225000 [89:40:30<355:06:36,  6.98s/it]
 19%|█▊        | 41780/225000 [89:40:35<325:26:10,  6.39s/it]
 19%|█▊        | 41781/225000 [89:40:40<307:53:45,  6.05s/it]
 19%|█▊        | 41782/225000 [89:40:45<293:41:58,  5.77s/it]
 19%|█▊        | 41783/225000 [89:40:50<281:29:21,  5.53s/it]
 19%|█▊        | 41784/225000 [89:40:55<279:09:48,  5.49s/it]
 19%|█▊        | 41785/225000 [89:41:08<390:51:16,  7.68s/it]
 19%|█▊        | 41786/225000 [89:41:15<377:11:56,  7.41s/it]
 19%|█▊        | 41787/225000 [89:41:19<329:11:08,  6.47s/it]
 19%|█▊        | 41788/225000 [89:41:26<339:11:35,  6.66s/it]
 19%|█▊        | 41789/225000 [89:41:36<391:29:42,  7.69s/it]
 19%|█▊        | 41790/225000 [89:41:41<340:25:18,  6.69s/it]
 19%|█▊        | 41791/225000 [89:41:46<316:13:28,  6.21s/it]
 19%|█▊        | 41792/225000 [89:41:56<377:40:23,  7.42s/it]
 19%|█▊        | 41793/225000 [89:42:04<383:19:02,  7.53s/it]
 19%|█▊        | 41794/225000 [89:42:08<328:24:57,  6.45s/it]
 19%|█▊        | 41795/225000 [89:42:15<338:09:01,  6.64s/it]
 19%|█▊        | 41796/225000 [89:42:24<369:50:43,  7.27s/it]
 19%|█▊        | 41797/225000 [89:42:28<319:14:51,  6.27s/it]
 19%|█▊        | 41798/225000 [89:42:39<390:56:39,  7.68s/it]
 19%|█▊        | 41799/225000 [89:42:43<335:23:38,  6.59s/it]
 19%|█▊        | 41800/225000 [89:42:49<335:44:24,  6.60s/it]
                                                             
{'loss': '0.6918', 'grad_norm': '0.723', 'learning_rate': '4.735e-05', 'epoch': '0.1858', 'num_input_tokens_seen': 151445536, 'train_runtime': '3.23e+05', 'train_tokens_per_second': '468.9'}

 19%|█▊        | 41800/225000 [89:42:49<335:44:24,  6.60s/it]
 19%|█▊        | 41801/225000 [89:42:56<341:49:48,  6.72s/it]
 19%|█▊        | 41802/225000 [89:43:03<334:53:01,  6.58s/it]
 19%|█▊        | 41803/225000 [89:43:12<372:54:11,  7.33s/it]
 19%|█▊        | 41804/225000 [89:43:16<327:34:08,  6.44s/it]
 19%|█▊        | 41805/225000 [89:43:21<308:59:52,  6.07s/it]
 19%|█▊        | 41806/225000 [89:43:26<291:35:55,  5.73s/it]
 19%|█▊        | 41807/225000 [89:43:31<281:55:06,  5.54s/it]
 19%|█▊        | 41808/225000 [89:43:44<388:35:46,  7.64s/it]
 19%|█▊        | 41809/225000 [89:43:50<375:01:36,  7.37s/it]
 19%|█▊        | 41810/225000 [89:43:55<337:40:40,  6.64s/it]
 19%|█▊        | 41811/225000 [89:44:01<324:32:59,  6.38s/it]
 19%|█▊        | 41812/225000 [89:44:12<389:57:52,  7.66s/it]
 19%|█▊        | 41813/225000 [89:44:16<336:28:39,  6.61s/it]
 19%|█▊        | 41814/225000 [89:44:26<381:03:53,  7.49s/it]
 19%|█▊        | 41815/225000 [89:44:30<337:26:26,  6.63s/it]
 19%|█▊        | 41816/225000 [89:44:35<304:14:58,  5.98s/it]
 19%|█▊        | 41817/225000 [89:44:46<382:15:51,  7.51s/it]
 19%|█▊        | 41818/225000 [89:44:55<402:20:50,  7.91s/it]
 19%|█▊        | 41819/225000 [89:44:58<335:35:19,  6.60s/it]
 19%|█▊        | 41820/225000 [89:45:08<389:58:10,  7.66s/it]
 19%|█▊        | 41821/225000 [89:45:12<333:58:36,  6.56s/it]
 19%|█▊        | 41822/225000 [89:45:17<307:27:04,  6.04s/it]
 19%|█▊        | 41823/225000 [89:45:26<352:45:30,  6.93s/it]
 19%|█▊        | 41824/225000 [89:45:31<323:28:30,  6.36s/it]
 19%|█▊        | 41825/225000 [89:45:40<364:35:01,  7.17s/it]
 19%|█▊        | 41826/225000 [89:45:45<326:35:29,  6.42s/it]
 19%|█▊        | 41827/225000 [89:45:49<299:21:56,  5.88s/it]
 19%|█▊        | 41828/225000 [89:45:54<276:04:57,  5.43s/it]
 19%|█▊        | 41829/225000 [89:46:05<370:57:36,  7.29s/it]
 19%|█▊        | 41830/225000 [89:46:09<320:31:17,  6.30s/it]
 19%|█▊        | 41831/225000 [89:46:15<307:13:27,  6.04s/it]
 19%|█▊        | 41832/225000 [89:46:19<280:28:45,  5.51s/it]
 19%|█▊        | 41833/225000 [89:46:24<267:52:58,  5.27s/it]
 19%|█▊        | 41834/225000 [89:46:33<323:03:09,  6.35s/it]
 19%|█▊        | 41835/225000 [89:46:42<367:10:02,  7.22s/it]
 19%|█▊        | 41836/225000 [89:46:49<358:28:06,  7.05s/it]
 19%|█▊        | 41837/225000 [89:46:55<345:11:32,  6.78s/it]
 19%|█▊        | 41838/225000 [89:47:04<383:16:21,  7.53s/it]
 19%|█▊        | 41839/225000 [89:47:08<332:16:53,  6.53s/it]
 19%|█▊        | 41840/225000 [89:47:20<404:29:39,  7.95s/it]
 19%|█▊        | 41841/225000 [89:47:23<340:44:43,  6.70s/it]
 19%|█▊        | 41842/225000 [89:47:28<316:46:36,  6.23s/it]
 19%|█▊        | 41843/225000 [89:47:38<365:24:25,  7.18s/it]
 19%|█▊        | 41844/225000 [89:47:42<319:30:28,  6.28s/it]
 19%|█▊        | 41845/225000 [89:47:51<359:35:05,  7.07s/it]
 19%|█▊        | 41846/225000 [89:47:55<316:02:27,  6.21s/it]
 19%|█▊        | 41847/225000 [89:47:59<286:08:04,  5.62s/it]
 19%|█▊        | 41848/225000 [89:48:11<377:42:33,  7.42s/it]
 19%|█▊        | 41849/225000 [89:48:16<337:04:16,  6.63s/it]
 19%|█▊        | 41850/225000 [89:48:24<356:34:51,  7.01s/it]
                                                             
{'loss': '0.6884', 'grad_norm': '0.7051', 'learning_rate': '4.734e-05', 'epoch': '0.186', 'num_input_tokens_seen': 151595472, 'train_runtime': '3.233e+05', 'train_tokens_per_second': '468.9'}

 19%|█▊        | 41850/225000 [89:48:24<356:34:51,  7.01s/it]
 19%|█▊        | 41851/225000 [89:48:30<342:06:18,  6.72s/it]
 19%|█▊        | 41852/225000 [89:48:35<315:34:20,  6.20s/it]
 19%|█▊        | 41853/225000 [89:48:40<298:27:58,  5.87s/it]
 19%|█▊        | 41854/225000 [89:48:45<295:11:35,  5.80s/it]
 19%|█▊        | 41855/225000 [89:48:51<296:02:50,  5.82s/it]
 19%|█▊        | 41856/225000 [89:48:57<298:25:12,  5.87s/it]
 19%|█▊        | 41857/225000 [89:49:03<292:55:12,  5.76s/it]
 19%|█▊        | 41858/225000 [89:49:16<405:22:49,  7.97s/it]
 19%|█▊        | 41859/225000 [89:49:23<397:23:23,  7.81s/it]
 19%|█▊        | 41860/225000 [89:49:28<350:37:44,  6.89s/it]
 19%|█▊        | 41861/225000 [89:49:33<313:12:29,  6.16s/it]
 19%|█▊        | 41862/225000 [89:49:41<351:02:39,  6.90s/it]
 19%|█▊        | 41863/225000 [89:49:50<377:46:52,  7.43s/it]
 19%|█▊        | 41864/225000 [89:49:53<314:48:27,  6.19s/it]
 19%|█▊        | 41865/225000 [89:50:02<355:10:33,  6.98s/it]
 19%|█▊        | 41866/225000 [89:50:07<318:11:41,  6.25s/it]
 19%|█▊        | 41867/225000 [89:50:15<353:47:58,  6.95s/it]
 19%|█▊        | 41868/225000 [89:50:19<310:50:04,  6.11s/it]
 19%|█▊        | 41869/225000 [89:50:27<340:55:59,  6.70s/it]
 19%|█▊        | 41870/225000 [89:50:37<379:55:17,  7.47s/it]
 19%|█▊        | 41871/225000 [89:50:41<330:43:24,  6.50s/it]
 19%|█▊        | 41872/225000 [89:50:46<303:52:37,  5.97s/it]
 19%|█▊        | 41873/225000 [89:50:51<288:41:20,  5.68s/it]
 19%|█▊        | 41874/225000 [89:50:56<292:37:38,  5.75s/it]
 19%|█▊        | 41875/225000 [89:51:09<396:20:28,  7.79s/it]
 19%|█▊        | 41876/225000 [89:51:13<333:49:12,  6.56s/it]
 19%|█▊        | 41877/225000 [89:51:17<304:49:20,  5.99s/it]
 19%|█▊        | 41878/225000 [89:51:27<356:57:32,  7.02s/it]
 19%|█▊        | 41879/225000 [89:51:35<381:58:21,  7.51s/it]
 19%|█▊        | 41880/225000 [89:51:39<326:23:51,  6.42s/it]
 19%|█▊        | 41881/225000 [89:51:43<291:31:28,  5.73s/it]
 19%|█▊        | 41882/225000 [89:51:54<360:45:08,  7.09s/it]
 19%|█▊        | 41883/225000 [89:52:02<374:25:05,  7.36s/it]
 19%|█▊        | 41884/225000 [89:52:06<324:16:27,  6.38s/it]
 19%|█▊        | 41885/225000 [89:52:10<287:43:14,  5.66s/it]
 19%|█▊        | 41886/225000 [89:52:21<374:42:46,  7.37s/it]
 19%|█▊        | 41887/225000 [89:52:26<331:38:04,  6.52s/it]
 19%|█▊        | 41888/225000 [89:52:30<302:01:32,  5.94s/it]
 19%|█▊        | 41889/225000 [89:52:39<346:12:56,  6.81s/it]
 19%|█▊        | 41890/225000 [89:52:47<363:17:21,  7.14s/it]
 19%|█▊        | 41891/225000 [89:52:51<320:23:21,  6.30s/it]
 19%|█▊        | 41892/225000 [89:53:03<404:27:42,  7.95s/it]
 19%|█▊        | 41893/225000 [89:53:08<359:14:41,  7.06s/it]
 19%|█▊        | 41894/225000 [89:53:15<358:12:50,  7.04s/it]
 19%|█▊        | 41895/225000 [89:53:22<361:04:07,  7.10s/it]
 19%|█▊        | 41896/225000 [89:53:28<341:40:58,  6.72s/it]
 19%|█▊        | 41897/225000 [89:53:35<342:47:43,  6.74s/it]
 19%|█▊        | 41898/225000 [89:53:40<311:23:47,  6.12s/it]
 19%|█▊        | 41899/225000 [89:53:44<290:25:45,  5.71s/it]
 19%|█▊        | 41900/225000 [89:53:49<279:31:08,  5.50s/it]
                                                             
{'loss': '0.6922', 'grad_norm': '1.077', 'learning_rate': '4.734e-05', 'epoch': '0.1862', 'num_input_tokens_seen': 151743536, 'train_runtime': '3.236e+05', 'train_tokens_per_second': '468.9'}

 19%|█▊        | 41900/225000 [89:53:49<279:31:08,  5.50s/it]
 19%|█▊        | 41901/225000 [89:53:55<279:47:13,  5.50s/it]
 19%|█▊        | 41902/225000 [89:54:08<396:36:36,  7.80s/it]
 19%|█▊        | 41903/225000 [89:54:15<378:19:41,  7.44s/it]
 19%|█▊        | 41904/225000 [89:54:22<381:36:06,  7.50s/it]
 19%|█▊        | 41905/225000 [89:54:27<335:04:16,  6.59s/it]
 19%|█▊        | 41906/225000 [89:54:35<361:35:26,  7.11s/it]
 19%|█▊        | 41907/225000 [89:54:42<356:45:12,  7.01s/it]
 19%|█▊        | 41908/225000 [89:54:47<332:58:32,  6.55s/it]
 19%|█▊        | 41909/225000 [89:54:55<357:07:57,  7.02s/it]
 19%|█▊        | 41910/225000 [89:55:01<331:23:25,  6.52s/it]
 19%|█▊        | 41911/225000 [89:55:06<311:51:46,  6.13s/it]
 19%|█▊        | 41912/225000 [89:55:11<296:52:24,  5.84s/it]
 19%|█▊        | 41913/225000 [89:55:17<294:02:03,  5.78s/it]
 19%|█▊        | 41914/225000 [89:55:23<293:21:49,  5.77s/it]
 19%|█▊        | 41915/225000 [89:55:36<412:04:11,  8.10s/it]
 19%|█▊        | 41916/225000 [89:55:41<367:36:33,  7.23s/it]
 19%|█▊        | 41917/225000 [89:55:50<382:07:19,  7.51s/it]
 19%|█▊        | 41918/225000 [89:55:55<352:12:35,  6.93s/it]
 19%|█▊        | 41919/225000 [89:56:01<333:49:59,  6.56s/it]
 19%|█▊        | 41920/225000 [89:56:09<361:59:52,  7.12s/it]
 19%|█▊        | 41921/225000 [89:56:15<348:47:55,  6.86s/it]
 19%|█▊        | 41922/225000 [89:56:20<311:40:51,  6.13s/it]
 19%|█▊        | 41923/225000 [89:56:26<308:31:11,  6.07s/it]
 19%|█▊        | 41924/225000 [89:56:34<342:13:57,  6.73s/it]
 19%|█▊        | 41925/225000 [89:56:39<308:40:16,  6.07s/it]
 19%|█▊        | 41926/225000 [89:56:43<285:27:59,  5.61s/it]
 19%|█▊        | 41927/225000 [89:56:56<390:29:59,  7.68s/it]
 19%|█▊        | 41928/225000 [89:57:00<341:23:50,  6.71s/it]
 19%|█▊        | 41929/225000 [89:57:04<303:30:23,  5.97s/it]
 19%|█▊        | 41930/225000 [89:57:16<396:28:14,  7.80s/it]
 19%|█▊        | 41931/225000 [89:57:21<341:24:02,  6.71s/it]
 19%|█▊        | 41932/225000 [89:57:27<334:59:05,  6.59s/it]
 19%|█▊        | 41933/225000 [89:57:37<382:07:51,  7.51s/it]
 19%|█▊        | 41934/225000 [89:57:41<332:42:16,  6.54s/it]
 19%|█▊        | 41935/225000 [89:57:49<359:58:20,  7.08s/it]
 19%|█▊        | 41936/225000 [89:57:54<320:29:44,  6.30s/it]
 19%|█▊        | 41937/225000 [89:57:59<307:39:13,  6.05s/it]
 19%|█▊        | 41938/225000 [89:58:07<339:05:40,  6.67s/it]
 19%|█▊        | 41939/225000 [89:58:16<370:46:22,  7.29s/it]
 19%|█▊        | 41940/225000 [89:58:21<338:33:24,  6.66s/it]
 19%|█▊        | 41941/225000 [89:58:26<306:23:10,  6.03s/it]
 19%|█▊        | 41942/225000 [89:58:34<334:02:43,  6.57s/it]
 19%|█▊        | 41943/225000 [89:58:42<367:29:29,  7.23s/it]
 19%|█▊        | 41944/225000 [89:58:49<365:37:42,  7.19s/it]
 19%|█▊        | 41945/225000 [89:58:55<339:40:03,  6.68s/it]
 19%|█▊        | 41946/225000 [89:58:59<294:56:20,  5.80s/it]
 19%|█▊        | 41947/225000 [89:59:09<364:40:26,  7.17s/it]
 19%|█▊        | 41948/225000 [89:59:12<307:55:33,  6.06s/it]
 19%|█▊        | 41949/225000 [89:59:20<327:42:51,  6.45s/it]
 19%|█▊        | 41950/225000 [89:59:29<370:48:34,  7.29s/it]
                                                             
{'loss': '0.6959', 'grad_norm': '0.825', 'learning_rate': '4.733e-05', 'epoch': '0.1864', 'num_input_tokens_seen': 151892928, 'train_runtime': '3.24e+05', 'train_tokens_per_second': '468.8'}

 19%|█▊        | 41950/225000 [89:59:29<370:48:34,  7.29s/it]
 19%|█▊        | 41951/225000 [89:59:33<318:15:12,  6.26s/it]
 19%|█▊        | 41952/225000 [89:59:38<293:06:18,  5.76s/it]
 19%|█▊        | 41953/225000 [89:59:50<401:40:17,  7.90s/it]
 19%|█▊        | 41954/225000 [89:59:57<378:54:25,  7.45s/it]
 19%|█▊        | 41955/225000 [90:00:00<317:18:04,  6.24s/it]
 19%|█▊        | 41956/225000 [90:00:10<370:15:04,  7.28s/it]
 19%|█▊        | 41957/225000 [90:00:17<363:02:25,  7.14s/it]
 19%|█▊        | 41958/225000 [90:00:21<321:40:02,  6.33s/it]
 19%|█▊        | 41959/225000 [90:00:26<297:36:44,  5.85s/it]
 19%|█▊        | 41960/225000 [90:00:30<277:13:47,  5.45s/it]
 19%|█▊        | 41961/225000 [90:00:44<399:13:08,  7.85s/it]
 19%|█▊        | 41962/225000 [90:00:48<348:10:56,  6.85s/it]
 19%|█▊        | 41963/225000 [90:00:55<339:33:34,  6.68s/it]
 19%|█▊        | 41964/225000 [90:00:59<295:27:19,  5.81s/it]
 19%|█▊        | 41965/225000 [90:01:09<369:41:49,  7.27s/it]
 19%|█▊        | 41966/225000 [90:01:15<342:36:57,  6.74s/it]
 19%|█▊        | 41967/225000 [90:01:20<318:12:03,  6.26s/it]
 19%|█▊        | 41968/225000 [90:01:25<298:54:45,  5.88s/it]
 19%|█▊        | 41969/225000 [90:01:30<293:53:10,  5.78s/it]
 19%|█▊        | 41970/225000 [90:01:36<285:33:27,  5.62s/it]
 19%|█▊        | 41971/225000 [90:01:47<371:10:51,  7.30s/it]
 19%|█▊        | 41972/225000 [90:01:54<376:17:07,  7.40s/it]
 19%|█▊        | 41973/225000 [90:01:59<326:46:36,  6.43s/it]
 19%|█▊        | 41974/225000 [90:02:08<377:44:02,  7.43s/it]
 19%|█▊        | 41975/225000 [90:02:15<368:23:59,  7.25s/it]
 19%|█▊        | 41976/225000 [90:02:21<339:24:55,  6.68s/it]
 19%|█▊        | 41977/225000 [90:02:29<361:40:22,  7.11s/it]
 19%|█▊        | 41978/225000 [90:02:37<385:17:04,  7.58s/it]
 19%|█▊        | 41979/225000 [90:02:42<337:21:18,  6.64s/it]
 19%|█▊        | 41980/225000 [90:02:50<365:09:36,  7.18s/it]
 19%|█▊        | 41981/225000 [90:02:55<332:25:51,  6.54s/it]
 19%|█▊        | 41982/225000 [90:03:02<339:40:35,  6.68s/it]
 19%|█▊        | 41983/225000 [90:03:07<309:04:01,  6.08s/it]
 19%|█▊        | 41984/225000 [90:03:12<287:30:15,  5.66s/it]
 19%|█▊        | 41985/225000 [90:03:22<353:30:15,  6.95s/it]
 19%|█▊        | 41986/225000 [90:03:30<376:47:10,  7.41s/it]
 19%|█▊        | 41987/225000 [90:03:37<371:46:49,  7.31s/it]
 19%|█▊        | 41988/225000 [90:03:42<339:35:06,  6.68s/it]
 19%|█▊        | 41989/225000 [90:03:47<310:59:24,  6.12s/it]
 19%|█▊        | 41990/225000 [90:03:59<391:39:40,  7.70s/it]
 19%|█▊        | 41991/225000 [90:04:04<359:58:07,  7.08s/it]
 19%|█▊        | 41992/225000 [90:04:09<326:03:34,  6.41s/it]
 19%|█▊        | 41993/225000 [90:04:14<302:48:32,  5.96s/it]
 19%|█▊        | 41994/225000 [90:04:19<289:14:05,  5.69s/it]
 19%|█▊        | 41995/225000 [90:04:26<305:07:49,  6.00s/it]
 19%|█▊        | 41996/225000 [90:04:39<417:14:53,  8.21s/it]
 19%|█▊        | 41997/225000 [90:04:44<365:47:37,  7.20s/it]
 19%|█▊        | 41998/225000 [90:04:50<343:03:33,  6.75s/it]
 19%|█▊        | 41999/225000 [90:04:56<333:13:46,  6.56s/it]
 19%|█▊        | 42000/225000 [90:05:02<325:14:24,  6.40s/it]
                                                             
{'loss': '0.6793', 'grad_norm': '0.7181', 'learning_rate': '4.732e-05', 'epoch': '0.1867', 'num_input_tokens_seen': 152043960, 'train_runtime': '3.243e+05', 'train_tokens_per_second': '468.8'}

 19%|█▊        | 42000/225000 [90:05:02<325:14:24,  6.40s/it]
 19%|█▊        | 42001/225000 [90:05:07<305:24:13,  6.01s/it]
 19%|█▊        | 42002/225000 [90:05:13<300:24:13,  5.91s/it]
 19%|█▊        | 42003/225000 [90:05:18<293:51:12,  5.78s/it]
 19%|█▊        | 42004/225000 [90:05:28<362:06:55,  7.12s/it]
 19%|█▊        | 42005/225000 [90:05:41<444:43:45,  8.75s/it]
 19%|█▊        | 42006/225000 [90:05:47<400:29:13,  7.88s/it]
 19%|█▊        | 42007/225000 [90:05:52<356:05:01,  7.01s/it]
 19%|█▊        | 42008/225000 [90:05:59<360:04:33,  7.08s/it]
 19%|█▊        | 42009/225000 [90:06:07<382:08:58,  7.52s/it]
 19%|█▊        | 42010/225000 [90:06:11<327:38:10,  6.45s/it]
 19%|█▊        | 42011/225000 [90:06:22<384:56:25,  7.57s/it]
 19%|█▊        | 42012/225000 [90:06:26<332:02:22,  6.53s/it]
 19%|█▊        | 42013/225000 [90:06:35<381:27:18,  7.50s/it]
 19%|█▊        | 42014/225000 [90:06:41<353:42:11,  6.96s/it]
 19%|█▊        | 42015/225000 [90:06:46<326:26:48,  6.42s/it]
 19%|█▊        | 42016/225000 [90:06:51<304:34:28,  5.99s/it]
 19%|█▊        | 42017/225000 [90:06:56<282:35:00,  5.56s/it]
 19%|█▊        | 42018/225000 [90:07:06<349:46:57,  6.88s/it]
 19%|█▊        | 42019/225000 [90:07:14<361:36:21,  7.11s/it]
 19%|█▊        | 42020/225000 [90:07:20<358:28:28,  7.05s/it]
 19%|█▊        | 42021/225000 [90:07:26<339:06:55,  6.67s/it]
 19%|█▊        | 42022/225000 [90:07:32<328:40:44,  6.47s/it]
 19%|█▊        | 42023/225000 [90:07:38<316:07:10,  6.22s/it]
 19%|█▊        | 42024/225000 [90:07:44<308:12:58,  6.06s/it]
 19%|█▊        | 42025/225000 [90:07:49<298:42:50,  5.88s/it]
 19%|█▊        | 42026/225000 [90:07:55<299:19:55,  5.89s/it]
 19%|█▊        | 42027/225000 [90:08:00<290:45:03,  5.72s/it]
 19%|█▊        | 42028/225000 [90:08:06<293:01:02,  5.77s/it]
 19%|█▊        | 42029/225000 [90:08:13<307:55:02,  6.06s/it]
 19%|█▊        | 42030/225000 [90:08:18<298:45:17,  5.88s/it]
 19%|█▊        | 42031/225000 [90:08:29<379:29:23,  7.47s/it]
 19%|█▊        | 42032/225000 [90:08:40<422:25:50,  8.31s/it]
 19%|█▊        | 42033/225000 [90:08:43<349:58:10,  6.89s/it]
 19%|█▊        | 42034/225000 [90:08:51<366:47:47,  7.22s/it]
 19%|█▊        | 42035/225000 [90:09:00<383:40:50,  7.55s/it]
 19%|█▊        | 42036/225000 [90:09:04<333:48:01,  6.57s/it]
 19%|█▊        | 42037/225000 [90:09:08<302:39:34,  5.96s/it]
 19%|█▊        | 42038/225000 [90:09:17<341:59:38,  6.73s/it]
 19%|█▊        | 42039/225000 [90:09:27<385:40:00,  7.59s/it]
 19%|█▊        | 42040/225000 [90:09:31<337:10:35,  6.63s/it]
 19%|█▊        | 42041/225000 [90:09:41<389:49:36,  7.67s/it]
 19%|█▊        | 42042/225000 [90:09:47<356:34:03,  7.02s/it]
 19%|█▊        | 42043/225000 [90:09:52<329:21:12,  6.48s/it]
 19%|█▊        | 42044/225000 [90:09:57<311:40:29,  6.13s/it]
 19%|█▊        | 42045/225000 [90:10:02<299:24:00,  5.89s/it]
 19%|█▊        | 42046/225000 [90:10:08<298:01:04,  5.86s/it]
 19%|█▊        | 42047/225000 [90:10:17<341:40:07,  6.72s/it]
 19%|█▊        | 42048/225000 [90:10:27<392:27:51,  7.72s/it]
 19%|█▊        | 42049/225000 [90:10:31<340:04:15,  6.69s/it]
 19%|█▊        | 42050/225000 [90:10:42<399:03:06,  7.85s/it]
                                                             
{'loss': '0.6811', 'grad_norm': '0.6145', 'learning_rate': '4.731e-05', 'epoch': '0.1869', 'num_input_tokens_seen': 152194456, 'train_runtime': '3.246e+05', 'train_tokens_per_second': '468.8'}

 19%|█▊        | 42050/225000 [90:10:42<399:03:06,  7.85s/it]
 19%|█▊        | 42051/225000 [90:10:47<364:20:46,  7.17s/it]
 19%|█▊        | 42052/225000 [90:10:54<355:19:01,  6.99s/it]
 19%|█▊        | 42053/225000 [90:10:59<326:03:20,  6.42s/it]
 19%|█▊        | 42054/225000 [90:11:04<306:31:38,  6.03s/it]
 19%|█▊        | 42055/225000 [90:11:09<289:14:37,  5.69s/it]
 19%|█▊        | 42056/225000 [90:11:14<280:35:27,  5.52s/it]
 19%|█▊        | 42057/225000 [90:11:23<334:57:57,  6.59s/it]
 19%|█▊        | 42058/225000 [90:11:35<415:15:17,  8.17s/it]
 19%|█▊        | 42059/225000 [90:11:39<352:06:20,  6.93s/it]
 19%|█▊        | 42060/225000 [90:11:45<335:48:56,  6.61s/it]
 19%|█▊        | 42061/225000 [90:11:54<377:55:33,  7.44s/it]
 19%|█▊        | 42062/225000 [90:12:00<348:09:12,  6.85s/it]
 19%|█▊        | 42063/225000 [90:12:06<328:54:46,  6.47s/it]
 19%|█▊        | 42064/225000 [90:12:10<305:26:06,  6.01s/it]
 19%|█▊        | 42065/225000 [90:12:16<298:08:08,  5.87s/it]
 19%|█▊        | 42066/225000 [90:12:21<290:52:20,  5.72s/it]
 19%|█▊        | 42067/225000 [90:12:33<377:25:49,  7.43s/it]
 19%|█▊        | 42068/225000 [90:12:43<414:53:07,  8.16s/it]
 19%|█▊        | 42069/225000 [90:12:49<387:47:53,  7.63s/it]
 19%|█▊        | 42070/225000 [90:12:56<376:15:54,  7.40s/it]
 19%|█▊        | 42071/225000 [90:13:01<338:17:46,  6.66s/it]
 19%|█▊        | 42072/225000 [90:13:06<311:43:40,  6.13s/it]
 19%|█▊        | 42073/225000 [90:13:17<385:10:31,  7.58s/it]
 19%|█▊        | 42074/225000 [90:13:21<340:53:07,  6.71s/it]
 19%|█▊        | 42075/225000 [90:13:26<309:15:51,  6.09s/it]
 19%|█▊        | 42076/225000 [90:13:32<306:18:47,  6.03s/it]
 19%|█▊        | 42077/225000 [90:13:44<397:33:39,  7.82s/it]
 19%|█▊        | 42078/225000 [90:13:48<334:19:13,  6.58s/it]
 19%|█▊        | 42079/225000 [90:13:58<394:24:08,  7.76s/it]
 19%|█▊        | 42080/225000 [90:14:03<349:57:28,  6.89s/it]
 19%|█▊        | 42081/225000 [90:14:08<328:42:18,  6.47s/it]
 19%|█▊        | 42082/225000 [90:14:14<308:04:31,  6.06s/it]
 19%|█▊        | 42083/225000 [90:14:19<298:03:04,  5.87s/it]
 19%|█▊        | 42084/225000 [90:14:25<294:01:38,  5.79s/it]
 19%|█▊        | 42085/225000 [90:14:30<291:27:11,  5.74s/it]
 19%|█▊        | 42086/225000 [90:14:38<321:41:47,  6.33s/it]
 19%|█▊        | 42087/225000 [90:14:51<430:05:07,  8.46s/it]
 19%|█▊        | 42088/225000 [90:14:57<383:03:05,  7.54s/it]
 19%|█▊        | 42089/225000 [90:15:02<343:25:43,  6.76s/it]
 19%|█▊        | 42090/225000 [90:15:07<314:58:21,  6.20s/it]
 19%|█▊        | 42091/225000 [90:15:11<294:56:10,  5.80s/it]
 19%|█▊        | 42092/225000 [90:15:21<346:13:49,  6.81s/it]
 19%|█▊        | 42093/225000 [90:15:32<421:39:59,  8.30s/it]
 19%|█▊        | 42094/225000 [90:15:40<406:19:53,  8.00s/it]
 19%|█▊        | 42095/225000 [90:15:44<343:24:48,  6.76s/it]
 19%|█▊        | 42096/225000 [90:15:48<308:08:21,  6.06s/it]
 19%|█▊        | 42097/225000 [90:15:52<282:10:32,  5.55s/it]
 19%|█▊        | 42098/225000 [90:16:03<362:11:38,  7.13s/it]
 19%|█▊        | 42099/225000 [90:16:09<340:13:33,  6.70s/it]
 19%|█▊        | 42100/225000 [90:16:19<386:18:05,  7.60s/it]
                                                             
{'loss': '0.6838', 'grad_norm': '0.7156', 'learning_rate': '4.73e-05', 'epoch': '0.1871', 'num_input_tokens_seen': 152346032, 'train_runtime': '3.25e+05', 'train_tokens_per_second': '468.8'}

 19%|█▊        | 42100/225000 [90:16:19<386:18:05,  7.60s/it]
 19%|█▊        | 42101/225000 [90:16:23<332:29:19,  6.54s/it]
 19%|█▊        | 42102/225000 [90:16:27<304:51:36,  6.00s/it]
 19%|█▊        | 42103/225000 [90:16:32<290:55:27,  5.73s/it]
 19%|█▊        | 42104/225000 [90:16:41<337:37:15,  6.65s/it]
 19%|█▊        | 42105/225000 [90:16:53<417:48:58,  8.22s/it]
 19%|█▊        | 42106/225000 [90:17:00<394:04:34,  7.76s/it]
 19%|█▊        | 42107/225000 [90:17:04<347:02:16,  6.83s/it]
 19%|█▊        | 42108/225000 [90:17:09<314:14:21,  6.19s/it]
 19%|█▊        | 42109/225000 [90:17:14<297:26:56,  5.85s/it]
 19%|█▊        | 42110/225000 [90:17:27<408:47:01,  8.05s/it]
 19%|█▊        | 42111/225000 [90:17:36<410:44:50,  8.09s/it]
 19%|█▊        | 42112/225000 [90:17:40<351:32:00,  6.92s/it]
 19%|█▊        | 42113/225000 [90:17:44<315:00:45,  6.20s/it]
 19%|█▊        | 42114/225000 [90:17:54<369:46:31,  7.28s/it]
 19%|█▊        | 42115/225000 [90:18:02<383:01:45,  7.54s/it]
 19%|█▊        | 42116/225000 [90:18:06<326:30:22,  6.43s/it]
 19%|█▊        | 42117/225000 [90:18:14<355:20:17,  6.99s/it]
 19%|█▊        | 42118/225000 [90:18:19<319:36:47,  6.29s/it]
 19%|█▊        | 42119/225000 [90:18:30<392:42:10,  7.73s/it]
 19%|█▊        | 42120/225000 [90:18:34<330:26:14,  6.50s/it]
 19%|█▊        | 42121/225000 [90:18:38<291:31:43,  5.74s/it]
 19%|█▊        | 42122/225000 [90:18:49<380:14:58,  7.49s/it]
 19%|█▊        | 42123/225000 [90:18:53<327:42:15,  6.45s/it]
 19%|█▊        | 42124/225000 [90:19:03<379:33:00,  7.47s/it]
 19%|█▊        | 42125/225000 [90:19:07<330:05:52,  6.50s/it]
 19%|█▊        | 42126/225000 [90:19:12<304:58:35,  6.00s/it]
 19%|█▊        | 42127/225000 [90:19:17<290:33:19,  5.72s/it]
 19%|█▊        | 42128/225000 [90:19:22<278:31:24,  5.48s/it]
 19%|█▊        | 42129/225000 [90:19:35<392:07:59,  7.72s/it]
 19%|█▊        | 42130/225000 [90:19:41<355:39:35,  7.00s/it]
 19%|█▊        | 42131/225000 [90:19:47<340:35:42,  6.71s/it]
 19%|█▊        | 42132/225000 [90:19:52<325:59:04,  6.42s/it]
 19%|█▊        | 42133/225000 [90:19:58<310:39:44,  6.12s/it]
 19%|█▊        | 42134/225000 [90:20:03<301:05:12,  5.93s/it]
 19%|█▊        | 42135/225000 [90:20:09<301:19:50,  5.93s/it]
 19%|█▊        | 42136/225000 [90:20:21<397:50:45,  7.83s/it]
 19%|█▊        | 42137/225000 [90:20:28<371:42:28,  7.32s/it]
 19%|█▊        | 42138/225000 [90:20:38<413:30:21,  8.14s/it]
 19%|█▊        | 42139/225000 [90:20:44<391:17:47,  7.70s/it]
 19%|█▊        | 42140/225000 [90:20:49<352:09:34,  6.93s/it]
 19%|█▊        | 42141/225000 [90:20:54<323:32:37,  6.37s/it]
 19%|█▊        | 42142/225000 [90:21:02<348:20:26,  6.86s/it]
 19%|█▊        | 42143/225000 [90:21:15<430:53:40,  8.48s/it]
 19%|█▊        | 42144/225000 [90:21:19<371:28:59,  7.31s/it]
 19%|█▊        | 42145/225000 [90:21:24<335:42:54,  6.61s/it]
 19%|█▊        | 42146/225000 [90:21:29<305:36:54,  6.02s/it]
 19%|█▊        | 42147/225000 [90:21:37<336:55:00,  6.63s/it]
 19%|█▊        | 42148/225000 [90:21:45<356:00:46,  7.01s/it]
 19%|█▊        | 42149/225000 [90:21:54<393:02:12,  7.74s/it]
 19%|█▊        | 42150/225000 [90:21:59<350:50:22,  6.91s/it]
                                                             
{'loss': '0.6899', 'grad_norm': '0.7981', 'learning_rate': '4.73e-05', 'epoch': '0.1873', 'num_input_tokens_seen': 152496936, 'train_runtime': '3.253e+05', 'train_tokens_per_second': '468.8'}

 19%|█▊        | 42150/225000 [90:21:59<350:50:22,  6.91s/it]
 19%|█▊        | 42151/225000 [90:22:05<328:50:59,  6.47s/it]
 19%|█▊        | 42152/225000 [90:22:10<306:26:30,  6.03s/it]
 19%|█▊        | 42153/225000 [90:22:15<296:35:08,  5.84s/it]
 19%|█▊        | 42154/225000 [90:22:20<287:00:43,  5.65s/it]
 19%|█▊        | 42155/225000 [90:22:30<345:03:18,  6.79s/it]
 19%|█▊        | 42156/225000 [90:22:40<397:43:05,  7.83s/it]
 19%|█▊        | 42157/225000 [90:22:45<347:23:57,  6.84s/it]
 19%|█▊        | 42158/225000 [90:22:49<315:54:10,  6.22s/it]
 19%|█▊        | 42159/225000 [90:23:03<421:56:47,  8.31s/it]
 19%|█▊        | 42160/225000 [90:23:06<354:49:16,  6.99s/it]
 19%|█▊        | 42161/225000 [90:23:15<371:52:43,  7.32s/it]
 19%|█▊        | 42162/225000 [90:23:20<341:45:48,  6.73s/it]
 19%|█▊        | 42163/225000 [90:23:25<313:27:09,  6.17s/it]
 19%|█▊        | 42164/225000 [90:23:34<367:15:53,  7.23s/it]
 19%|█▊        | 42165/225000 [90:23:38<316:10:29,  6.23s/it]
 19%|█▊        | 42166/225000 [90:23:43<291:31:49,  5.74s/it]
 19%|█▊        | 42167/225000 [90:23:54<370:19:06,  7.29s/it]
 19%|█▊        | 42168/225000 [90:23:59<341:56:49,  6.73s/it]
 19%|█▊        | 42169/225000 [90:24:04<305:31:25,  6.02s/it]
 19%|█▊        | 42170/225000 [90:24:11<332:23:13,  6.54s/it]
 19%|█▊        | 42171/225000 [90:24:20<363:33:47,  7.16s/it]
 19%|█▊        | 42172/225000 [90:24:25<338:02:52,  6.66s/it]
 19%|█▊        | 42173/225000 [90:24:30<301:45:19,  5.94s/it]
 19%|█▊        | 42174/225000 [90:24:41<376:14:23,  7.41s/it]
 19%|█▊        | 42175/225000 [90:24:45<322:59:42,  6.36s/it]
 19%|█▊        | 42176/225000 [90:24:48<283:31:35,  5.58s/it]
 19%|█▊        | 42177/225000 [90:24:59<361:37:49,  7.12s/it]
 19%|█▊        | 42178/225000 [90:25:04<331:44:44,  6.53s/it]
 19%|█▊        | 42179/225000 [90:25:12<354:58:05,  6.99s/it]
 19%|█▊        | 42180/225000 [90:25:19<345:47:37,  6.81s/it]
 19%|█▊        | 42181/225000 [90:25:23<316:24:05,  6.23s/it]
 19%|█▊        | 42182/225000 [90:25:28<295:28:04,  5.82s/it]
 19%|█▊        | 42183/225000 [90:25:36<326:42:26,  6.43s/it]
 19%|█▊        | 42184/225000 [90:25:45<366:29:43,  7.22s/it]
 19%|█▊        | 42185/225000 [90:25:49<319:40:47,  6.30s/it]
 19%|█▊        | 42186/225000 [90:25:55<312:08:46,  6.15s/it]
 19%|█▊        | 42187/225000 [90:26:01<303:45:24,  5.98s/it]
 19%|█▉        | 42188/225000 [90:26:07<303:20:24,  5.97s/it]
 19%|█▉        | 42189/225000 [90:26:17<367:17:49,  7.23s/it]
 19%|█▉        | 42190/225000 [90:26:30<452:40:47,  8.91s/it]
 19%|█▉        | 42191/225000 [90:26:33<369:08:31,  7.27s/it]
 19%|█▉        | 42192/225000 [90:26:38<325:21:27,  6.41s/it]
 19%|█▉        | 42193/225000 [90:26:47<376:37:50,  7.42s/it]
 19%|█▉        | 42194/225000 [90:26:53<352:16:32,  6.94s/it]
 19%|█▉        | 42195/225000 [90:26:58<327:56:14,  6.46s/it]
 19%|█▉        | 42196/225000 [90:27:03<302:11:40,  5.95s/it]
 19%|█▉        | 42197/225000 [90:27:16<412:35:34,  8.13s/it]
 19%|█▉        | 42198/225000 [90:27:21<354:30:38,  6.98s/it]
 19%|█▉        | 42199/225000 [90:27:25<314:10:06,  6.19s/it]
 19%|█▉        | 42200/225000 [90:27:35<365:21:42,  7.20s/it]
                                                             
{'loss': '0.6934', 'grad_norm': '0.7001', 'learning_rate': '4.729e-05', 'epoch': '0.1876', 'num_input_tokens_seen': 152647464, 'train_runtime': '3.257e+05', 'train_tokens_per_second': '468.7'}

 19%|█▉        | 42200/225000 [90:27:35<365:21:42,  7.20s/it]
 19%|█▉        | 42201/225000 [90:27:40<341:01:31,  6.72s/it]
 19%|█▉        | 42202/225000 [90:27:48<349:41:48,  6.89s/it]
 19%|█▉        | 42203/225000 [90:27:55<363:20:58,  7.16s/it]
 19%|█▉        | 42204/225000 [90:28:04<380:43:55,  7.50s/it]
 19%|█▉        | 42205/225000 [90:28:08<335:55:47,  6.62s/it]
 19%|█▉        | 42206/225000 [90:28:13<301:20:36,  5.93s/it]
 19%|█▉        | 42207/225000 [90:28:24<378:42:47,  7.46s/it]
 19%|█▉        | 42208/225000 [90:28:29<346:36:18,  6.83s/it]
 19%|█▉        | 42209/225000 [90:28:34<314:26:18,  6.19s/it]
 19%|█▉        | 42210/225000 [90:28:38<293:08:06,  5.77s/it]
 19%|█▉        | 42211/225000 [90:28:43<280:16:48,  5.52s/it]
 19%|█▉        | 42212/225000 [90:28:51<316:06:01,  6.23s/it]
 19%|█▉        | 42213/225000 [90:29:03<405:32:58,  7.99s/it]
 19%|█▉        | 42214/225000 [90:29:07<339:58:41,  6.70s/it]
 19%|█▉        | 42215/225000 [90:29:15<359:16:08,  7.08s/it]
 19%|█▉        | 42216/225000 [90:29:21<346:23:14,  6.82s/it]
 19%|█▉        | 42217/225000 [90:29:28<352:34:40,  6.94s/it]
 19%|█▉        | 42218/225000 [90:29:36<365:43:34,  7.20s/it]
 19%|█▉        | 42219/225000 [90:29:41<322:13:05,  6.35s/it]
 19%|█▉        | 42220/225000 [90:29:49<359:13:04,  7.08s/it]
 19%|█▉        | 42221/225000 [90:29:58<389:10:58,  7.67s/it]
 19%|█▉        | 42222/225000 [90:30:02<328:43:00,  6.47s/it]
 19%|█▉        | 42223/225000 [90:30:11<359:44:15,  7.09s/it]
 19%|█▉        | 42224/225000 [90:30:20<394:27:20,  7.77s/it]
 19%|█▉        | 42225/225000 [90:30:24<344:14:30,  6.78s/it]
 19%|█▉        | 42226/225000 [90:30:34<393:18:12,  7.75s/it]
 19%|█▉        | 42227/225000 [90:30:40<352:50:55,  6.95s/it]
 19%|█▉        | 42228/225000 [90:30:45<323:14:36,  6.37s/it]
 19%|█▉        | 42229/225000 [90:30:49<300:55:55,  5.93s/it]
 19%|█▉        | 42230/225000 [90:30:55<294:14:31,  5.80s/it]
 19%|█▉        | 42231/225000 [90:31:00<282:59:32,  5.57s/it]
 19%|█▉        | 42232/225000 [90:31:06<290:03:27,  5.71s/it]
 19%|█▉        | 42233/225000 [90:31:19<404:05:36,  7.96s/it]
 19%|█▉        | 42234/225000 [90:31:24<351:23:36,  6.92s/it]
 19%|█▉        | 42235/225000 [90:31:34<397:21:13,  7.83s/it]
 19%|█▉        | 42236/225000 [90:31:38<350:25:03,  6.90s/it]
 19%|█▉        | 42237/225000 [90:31:43<320:22:14,  6.31s/it]
 19%|█▉        | 42238/225000 [90:31:48<300:10:43,  5.91s/it]
 19%|█▉        | 42239/225000 [90:31:53<280:09:07,  5.52s/it]
 19%|█▉        | 42240/225000 [90:32:04<363:00:15,  7.15s/it]
 19%|█▉        | 42241/225000 [90:32:13<389:53:51,  7.68s/it]
 19%|█▉        | 42242/225000 [90:32:17<335:22:35,  6.61s/it]
 19%|█▉        | 42243/225000 [90:32:22<313:17:00,  6.17s/it]
 19%|█▉        | 42244/225000 [90:32:28<303:56:12,  5.99s/it]
 19%|█▉        | 42245/225000 [90:32:33<298:20:22,  5.88s/it]
 19%|█▉        | 42246/225000 [90:32:39<295:08:50,  5.81s/it]
 19%|█▉        | 42247/225000 [90:32:50<378:01:31,  7.45s/it]
 19%|█▉        | 42248/225000 [90:32:59<396:36:17,  7.81s/it]
 19%|█▉        | 42249/225000 [90:33:07<400:25:33,  7.89s/it]
 19%|█▉        | 42250/225000 [90:33:11<350:39:46,  6.91s/it]
                                                             
{'loss': '0.6995', 'grad_norm': '0.6929', 'learning_rate': '4.728e-05', 'epoch': '0.1878', 'num_input_tokens_seen': 152798600, 'train_runtime': '3.26e+05', 'train_tokens_per_second': '468.7'}

 19%|█▉        | 42250/225000 [90:33:11<350:39:46,  6.91s/it]
 19%|█▉        | 42251/225000 [90:33:19<364:01:34,  7.17s/it]
 19%|█▉        | 42252/225000 [90:33:24<333:50:32,  6.58s/it]
 19%|█▉        | 42253/225000 [90:33:29<302:36:44,  5.96s/it]
 19%|█▉        | 42254/225000 [90:33:37<335:21:38,  6.61s/it]
 19%|█▉        | 42255/225000 [90:33:46<371:18:37,  7.31s/it]
 19%|█▉        | 42256/225000 [90:33:53<362:33:21,  7.14s/it]
 19%|█▉        | 42257/225000 [90:34:00<358:34:25,  7.06s/it]
 19%|█▉        | 42258/225000 [90:34:04<317:01:34,  6.25s/it]
 19%|█▉        | 42259/225000 [90:34:09<293:09:12,  5.78s/it]
 19%|█▉        | 42260/225000 [90:34:13<277:54:47,  5.47s/it]
 19%|█▉        | 42261/225000 [90:34:24<347:41:32,  6.85s/it]
 19%|█▉        | 42262/225000 [90:34:32<372:59:58,  7.35s/it]
 19%|█▉        | 42263/225000 [90:34:36<323:11:41,  6.37s/it]
 19%|█▉        | 42264/225000 [90:34:46<374:09:02,  7.37s/it]
 19%|█▉        | 42265/225000 [90:34:51<334:16:09,  6.59s/it]
 19%|█▉        | 42266/225000 [90:34:56<316:05:28,  6.23s/it]
 19%|█▉        | 42267/225000 [90:35:01<296:38:52,  5.84s/it]
 19%|█▉        | 42268/225000 [90:35:13<394:27:25,  7.77s/it]
 19%|█▉        | 42269/225000 [90:35:20<377:28:50,  7.44s/it]
 19%|█▉        | 42270/225000 [90:35:26<352:56:30,  6.95s/it]
 19%|█▉        | 42271/225000 [90:35:31<324:07:19,  6.39s/it]
 19%|█▉        | 42272/225000 [90:35:35<298:19:24,  5.88s/it]
 19%|█▉        | 42273/225000 [90:35:42<315:26:54,  6.21s/it]
 19%|█▉        | 42274/225000 [90:35:56<421:42:55,  8.31s/it]
 19%|█▉        | 42275/225000 [90:36:01<382:25:26,  7.53s/it]
 19%|█▉        | 42276/225000 [90:36:07<360:22:50,  7.10s/it]
 19%|█▉        | 42277/225000 [90:36:12<324:56:45,  6.40s/it]
 19%|█▉        | 42278/225000 [90:36:17<302:42:01,  5.96s/it]
 19%|█▉        | 42279/225000 [90:36:22<288:18:22,  5.68s/it]
 19%|█▉        | 42280/225000 [90:36:36<409:40:45,  8.07s/it]
 19%|█▉        | 42281/225000 [90:36:40<349:28:53,  6.89s/it]
 19%|█▉        | 42282/225000 [90:36:49<387:36:42,  7.64s/it]
 19%|█▉        | 42283/225000 [90:36:53<328:09:57,  6.47s/it]
 19%|█▉        | 42284/225000 [90:36:57<294:27:20,  5.80s/it]
 19%|█▉        | 42285/225000 [90:37:02<280:17:23,  5.52s/it]
 19%|█▉        | 42286/225000 [90:37:11<331:22:01,  6.53s/it]
 19%|█▉        | 42287/225000 [90:37:22<398:23:05,  7.85s/it]
 19%|█▉        | 42288/225000 [90:37:26<342:28:51,  6.75s/it]
 19%|█▉        | 42289/225000 [90:37:35<370:48:37,  7.31s/it]
 19%|█▉        | 42290/225000 [90:37:40<341:22:07,  6.73s/it]
 19%|█▉        | 42291/225000 [90:37:46<330:07:38,  6.50s/it]
 19%|█▉        | 42292/225000 [90:37:51<310:47:49,  6.12s/it]
 19%|█▉        | 42293/225000 [90:37:56<293:52:41,  5.79s/it]
 19%|█▉        | 42294/225000 [90:38:01<274:10:44,  5.40s/it]
 19%|█▉        | 42295/225000 [90:38:13<380:10:33,  7.49s/it]
 19%|█▉        | 42296/225000 [90:38:19<354:39:16,  6.99s/it]
 19%|█▉        | 42297/225000 [90:38:23<312:08:05,  6.15s/it]
 19%|█▉        | 42298/225000 [90:38:28<289:06:39,  5.70s/it]
 19%|█▉        | 42299/225000 [90:38:40<389:31:54,  7.68s/it]
 19%|█▉        | 42300/225000 [90:38:44<328:18:58,  6.47s/it]
                                                             
{'loss': '0.6988', 'grad_norm': '0.8101', 'learning_rate': '4.727e-05', 'epoch': '0.188', 'num_input_tokens_seen': 152948488, 'train_runtime': '3.263e+05', 'train_tokens_per_second': '468.7'}

 19%|█▉        | 42300/225000 [90:38:44<328:18:58,  6.47s/it]
 19%|█▉        | 42301/225000 [90:38:53<369:15:15,  7.28s/it]
 19%|█▉        | 42302/225000 [90:38:58<337:29:25,  6.65s/it]
 19%|█▉        | 42303/225000 [90:39:07<374:52:06,  7.39s/it]
 19%|█▉        | 42304/225000 [90:39:15<373:03:00,  7.35s/it]
 19%|█▉        | 42305/225000 [90:39:23<385:38:32,  7.60s/it]
 19%|█▉        | 42306/225000 [90:39:27<336:15:39,  6.63s/it]
 19%|█▉        | 42307/225000 [90:39:31<298:43:11,  5.89s/it]
 19%|█▉        | 42308/225000 [90:39:36<285:48:34,  5.63s/it]
 19%|█▉        | 42309/225000 [90:39:50<401:58:13,  7.92s/it]
 19%|█▉        | 42310/225000 [90:39:53<341:17:38,  6.73s/it]
 19%|█▉        | 42311/225000 [90:39:58<311:30:44,  6.14s/it]
 19%|█▉        | 42312/225000 [90:40:03<296:53:59,  5.85s/it]
 19%|█▉        | 42313/225000 [90:40:16<393:11:24,  7.75s/it]
 19%|█▉        | 42314/225000 [90:40:19<331:53:34,  6.54s/it]
 19%|█▉        | 42315/225000 [90:40:26<340:57:34,  6.72s/it]
 19%|█▉        | 42316/225000 [90:40:33<342:24:09,  6.75s/it]
 19%|█▉        | 42317/225000 [90:40:42<371:43:36,  7.33s/it]
 19%|█▉        | 42318/225000 [90:40:46<328:13:58,  6.47s/it]
 19%|█▉        | 42319/225000 [90:40:56<374:29:09,  7.38s/it]
 19%|█▉        | 42320/225000 [90:41:00<329:54:11,  6.50s/it]
 19%|█▉        | 42321/225000 [90:41:06<313:26:04,  6.18s/it]
 19%|█▉        | 42322/225000 [90:41:15<356:12:14,  7.02s/it]
 19%|█▉        | 42323/225000 [90:41:22<355:46:59,  7.01s/it]
 19%|█▉        | 42324/225000 [90:41:27<328:36:14,  6.48s/it]
 19%|█▉        | 42325/225000 [90:41:32<311:32:14,  6.14s/it]
 19%|█▉        | 42326/225000 [90:41:37<296:02:40,  5.83s/it]
 19%|█▉        | 42327/225000 [90:41:43<283:54:27,  5.60s/it]
 19%|█▉        | 42328/225000 [90:41:52<346:32:05,  6.83s/it]
 19%|█▉        | 42329/225000 [90:42:01<368:57:44,  7.27s/it]
 19%|█▉        | 42330/225000 [90:42:05<328:57:11,  6.48s/it]
 19%|█▉        | 42331/225000 [90:42:15<380:27:22,  7.50s/it]
 19%|█▉        | 42332/225000 [90:42:22<373:41:41,  7.36s/it]
 19%|█▉        | 42333/225000 [90:42:27<342:17:06,  6.75s/it]
 19%|█▉        | 42334/225000 [90:42:32<314:41:49,  6.20s/it]
 19%|█▉        | 42335/225000 [90:42:38<300:15:22,  5.92s/it]
 19%|█▉        | 42336/225000 [90:42:45<328:46:28,  6.48s/it]
 19%|█▉        | 42337/225000 [90:42:57<410:10:48,  8.08s/it]
 19%|█▉        | 42338/225000 [90:43:01<345:47:05,  6.81s/it]
 19%|█▉        | 42339/225000 [90:43:06<312:44:59,  6.16s/it]
 19%|█▉        | 42340/225000 [90:43:16<380:38:11,  7.50s/it]
 19%|█▉        | 42341/225000 [90:43:22<349:07:55,  6.88s/it]
 19%|█▉        | 42342/225000 [90:43:27<317:23:06,  6.26s/it]
 19%|█▉        | 42343/225000 [90:43:32<302:36:19,  5.96s/it]
 19%|█▉        | 42344/225000 [90:43:37<289:29:16,  5.71s/it]
 19%|█▉        | 42345/225000 [90:43:49<379:00:29,  7.47s/it]
 19%|█▉        | 42346/225000 [90:43:53<328:00:44,  6.46s/it]
 19%|█▉        | 42347/225000 [90:44:00<341:12:41,  6.73s/it]
 19%|█▉        | 42348/225000 [90:44:07<341:05:01,  6.72s/it]
 19%|█▉        | 42349/225000 [90:44:15<366:41:14,  7.23s/it]
 19%|█▉        | 42350/225000 [90:44:19<321:23:22,  6.33s/it]
                                                             
{'loss': '0.696', 'grad_norm': '0.6218', 'learning_rate': '4.726e-05', 'epoch': '0.1882', 'num_input_tokens_seen': 153099512, 'train_runtime': '3.267e+05', 'train_tokens_per_second': '468.7'}

 19%|█▉        | 42350/225000 [90:44:19<321:23:22,  6.33s/it]
 19%|█▉        | 42351/225000 [90:44:28<356:37:29,  7.03s/it]
 19%|█▉        | 42352/225000 [90:44:34<345:39:15,  6.81s/it]
 19%|█▉        | 42353/225000 [90:44:43<368:19:50,  7.26s/it]
 19%|█▉        | 42354/225000 [90:44:47<320:45:08,  6.32s/it]
 19%|█▉        | 42355/225000 [90:44:55<348:53:09,  6.88s/it]
 19%|█▉        | 42356/225000 [90:45:03<360:37:27,  7.11s/it]
 19%|█▉        | 42357/225000 [90:45:08<332:40:07,  6.56s/it]
 19%|█▉        | 42358/225000 [90:45:14<319:57:10,  6.31s/it]
 19%|█▉        | 42359/225000 [90:45:22<359:35:23,  7.09s/it]
 19%|█▉        | 42360/225000 [90:45:26<309:09:03,  6.09s/it]
 19%|█▉        | 42361/225000 [90:45:34<335:54:08,  6.62s/it]
 19%|█▉        | 42362/225000 [90:45:42<360:50:59,  7.11s/it]
 19%|█▉        | 42363/225000 [90:45:50<367:46:53,  7.25s/it]
 19%|█▉        | 42364/225000 [90:45:55<329:02:26,  6.49s/it]
 19%|█▉        | 42365/225000 [90:46:00<308:40:43,  6.08s/it]
 19%|█▉        | 42366/225000 [90:46:04<285:16:25,  5.62s/it]
 19%|█▉        | 42367/225000 [90:46:09<267:15:42,  5.27s/it]
 19%|█▉        | 42368/225000 [90:46:22<388:46:14,  7.66s/it]
 19%|█▉        | 42369/225000 [90:46:26<328:11:31,  6.47s/it]
 19%|█▉        | 42370/225000 [90:46:33<335:55:34,  6.62s/it]
 19%|█▉        | 42371/225000 [90:46:37<301:31:16,  5.94s/it]
 19%|█▉        | 42372/225000 [90:46:48<378:25:09,  7.46s/it]
 19%|█▉        | 42373/225000 [90:46:53<339:47:43,  6.70s/it]
 19%|█▉        | 42374/225000 [90:47:01<362:37:15,  7.15s/it]
 19%|█▉        | 42375/225000 [90:47:09<372:41:38,  7.35s/it]
 19%|█▉        | 42376/225000 [90:47:14<336:58:18,  6.64s/it]
 19%|█▉        | 42377/225000 [90:47:19<310:11:55,  6.11s/it]
 19%|█▉        | 42378/225000 [90:47:31<400:27:28,  7.89s/it]
 19%|█▉        | 42379/225000 [90:47:37<379:54:58,  7.49s/it]
 19%|█▉        | 42380/225000 [90:47:42<335:44:49,  6.62s/it]
 19%|█▉        | 42381/225000 [90:47:50<350:42:41,  6.91s/it]
 19%|█▉        | 42382/225000 [90:47:55<327:02:07,  6.45s/it]
 19%|█▉        | 42383/225000 [90:48:00<308:32:17,  6.08s/it]
 19%|█▉        | 42384/225000 [90:48:05<294:27:01,  5.80s/it]
 19%|█▉        | 42385/225000 [90:48:10<283:05:07,  5.58s/it]
 19%|█▉        | 42386/225000 [90:48:16<286:35:21,  5.65s/it]
 19%|█▉        | 42387/225000 [90:48:27<366:57:10,  7.23s/it]
 19%|█▉        | 42388/225000 [90:48:36<396:10:02,  7.81s/it]
 19%|█▉        | 42389/225000 [90:48:44<396:16:12,  7.81s/it]
 19%|█▉        | 42390/225000 [90:48:48<339:14:02,  6.69s/it]
 19%|█▉        | 42391/225000 [90:48:58<380:37:24,  7.50s/it]
 19%|█▉        | 42392/225000 [90:49:04<362:58:19,  7.16s/it]
 19%|█▉        | 42393/225000 [90:49:09<328:12:44,  6.47s/it]
 19%|█▉        | 42394/225000 [90:49:14<306:21:19,  6.04s/it]
 19%|█▉        | 42395/225000 [90:49:19<290:43:25,  5.73s/it]
 19%|█▉        | 42396/225000 [90:49:24<282:52:27,  5.58s/it]
 19%|█▉        | 42397/225000 [90:49:36<384:50:31,  7.59s/it]
 19%|█▉        | 42398/225000 [90:49:41<337:56:02,  6.66s/it]
 19%|█▉        | 42399/225000 [90:49:49<360:04:34,  7.10s/it]
 19%|█▉        | 42400/225000 [90:49:54<326:14:17,  6.43s/it]
                                                             
{'loss': '0.6876', 'grad_norm': '0.7466', 'learning_rate': '4.725e-05', 'epoch': '0.1884', 'num_input_tokens_seen': 153248888, 'train_runtime': '3.27e+05', 'train_tokens_per_second': '468.7'}

 19%|█▉        | 42400/225000 [90:49:54<326:14:17,  6.43s/it]
 19%|█▉        | 42401/225000 [90:49:58<294:24:39,  5.80s/it]
 19%|█▉        | 42402/225000 [90:50:09<370:19:18,  7.30s/it]
 19%|█▉        | 42403/225000 [90:50:14<338:13:07,  6.67s/it]
 19%|█▉        | 42404/225000 [90:50:23<370:26:33,  7.30s/it]
 19%|█▉        | 42405/225000 [90:50:27<320:04:48,  6.31s/it]
 19%|█▉        | 42406/225000 [90:50:31<290:53:48,  5.74s/it]
 19%|█▉        | 42407/225000 [90:50:42<366:53:09,  7.23s/it]
 19%|█▉        | 42408/225000 [90:50:51<387:33:12,  7.64s/it]
 19%|█▉        | 42409/225000 [90:50:55<343:48:45,  6.78s/it]
 19%|█▉        | 42410/225000 [90:51:00<312:39:31,  6.16s/it]
 19%|█▉        | 42411/225000 [90:51:05<297:09:43,  5.86s/it]
 19%|█▉        | 42412/225000 [90:51:10<277:06:36,  5.46s/it]
 19%|█▉        | 42413/225000 [90:51:17<307:08:57,  6.06s/it]
 19%|█▉        | 42414/225000 [90:51:30<404:00:46,  7.97s/it]
 19%|█▉        | 42415/225000 [90:51:36<373:23:42,  7.36s/it]
 19%|█▉        | 42416/225000 [90:51:40<329:38:44,  6.50s/it]
 19%|█▉        | 42417/225000 [90:51:50<381:33:49,  7.52s/it]
 19%|█▉        | 42418/225000 [90:51:55<334:38:51,  6.60s/it]
 19%|█▉        | 42419/225000 [90:51:59<306:22:03,  6.04s/it]
 19%|█▉        | 42420/225000 [90:52:06<320:44:59,  6.32s/it]
 19%|█▉        | 42421/225000 [90:52:17<391:22:31,  7.72s/it]
 19%|█▉        | 42422/225000 [90:52:21<331:29:47,  6.54s/it]
 19%|█▉        | 42423/225000 [90:52:29<360:05:07,  7.10s/it]
 19%|█▉        | 42424/225000 [90:52:35<343:00:04,  6.76s/it]
 19%|█▉        | 42425/225000 [90:52:43<353:00:23,  6.96s/it]
 19%|█▉        | 42426/225000 [90:52:53<400:17:09,  7.89s/it]
 19%|█▉        | 42427/225000 [90:52:58<350:46:33,  6.92s/it]
 19%|█▉        | 42428/225000 [90:53:02<305:58:01,  6.03s/it]
 19%|█▉        | 42429/225000 [90:53:11<364:35:09,  7.19s/it]
 19%|█▉        | 42430/225000 [90:53:20<389:54:51,  7.69s/it]
 19%|█▉        | 42431/225000 [90:53:25<345:07:45,  6.81s/it]
 19%|█▉        | 42432/225000 [90:53:34<380:58:56,  7.51s/it]
 19%|█▉        | 42433/225000 [90:53:39<335:39:57,  6.62s/it]
 19%|█▉        | 42434/225000 [90:53:43<298:28:33,  5.89s/it]
 19%|█▉        | 42435/225000 [90:53:54<376:22:35,  7.42s/it]
 19%|█▉        | 42436/225000 [90:54:00<351:33:47,  6.93s/it]
 19%|█▉        | 42437/225000 [90:54:05<324:24:42,  6.40s/it]
 19%|█▉        | 42438/225000 [90:54:15<386:06:40,  7.61s/it]
 19%|█▉        | 42439/225000 [90:54:21<355:26:22,  7.01s/it]
 19%|█▉        | 42440/225000 [90:54:25<311:13:23,  6.14s/it]
 19%|█▉        | 42441/225000 [90:54:31<307:13:32,  6.06s/it]
 19%|█▉        | 42442/225000 [90:54:36<300:51:04,  5.93s/it]
 19%|█▉        | 42443/225000 [90:54:42<287:58:44,  5.68s/it]
 19%|█▉        | 42444/225000 [90:54:46<274:35:46,  5.42s/it]
 19%|█▉        | 42445/225000 [90:54:59<380:14:07,  7.50s/it]
 19%|█▉        | 42446/225000 [90:55:06<383:40:14,  7.57s/it]
 19%|█▉        | 42447/225000 [90:55:14<380:24:23,  7.50s/it]
 19%|█▉        | 42448/225000 [90:55:20<359:00:30,  7.08s/it]
 19%|█▉        | 42449/225000 [90:55:24<309:52:21,  6.11s/it]
 19%|█▉        | 42450/225000 [90:55:28<288:56:37,  5.70s/it]
                                                             
{'loss': '0.6763', 'grad_norm': '0.6964', 'learning_rate': '4.725e-05', 'epoch': '0.1887', 'num_input_tokens_seen': 153400536, 'train_runtime': '3.273e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42450/225000 [90:55:28<288:56:37,  5.70s/it]
 19%|█▉        | 42451/225000 [90:55:33<271:28:01,  5.35s/it]
 19%|█▉        | 42452/225000 [90:55:45<369:11:34,  7.28s/it]
 19%|█▉        | 42453/225000 [90:55:52<362:01:06,  7.14s/it]
 19%|█▉        | 42454/225000 [90:55:56<318:36:12,  6.28s/it]
 19%|█▉        | 42455/225000 [90:56:07<398:36:08,  7.86s/it]
 19%|█▉        | 42456/225000 [90:56:11<335:52:00,  6.62s/it]
 19%|█▉        | 42457/225000 [90:56:20<362:56:11,  7.16s/it]
 19%|█▉        | 42458/225000 [90:56:24<316:47:42,  6.25s/it]
 19%|█▉        | 42459/225000 [90:56:28<291:39:02,  5.75s/it]
 19%|█▉        | 42460/225000 [90:56:39<370:03:53,  7.30s/it]
 19%|█▉        | 42461/225000 [90:56:44<330:39:13,  6.52s/it]
 19%|█▉        | 42462/225000 [90:56:48<300:51:51,  5.93s/it]
 19%|█▉        | 42463/225000 [90:56:55<313:10:06,  6.18s/it]
 19%|█▉        | 42464/225000 [90:57:06<380:14:27,  7.50s/it]
 19%|█▉        | 42465/225000 [90:57:10<331:21:37,  6.54s/it]
 19%|█▉        | 42466/225000 [90:57:16<317:51:17,  6.27s/it]
 19%|█▉        | 42467/225000 [90:57:26<380:57:05,  7.51s/it]
 19%|█▉        | 42468/225000 [90:57:30<327:20:06,  6.46s/it]
 19%|█▉        | 42469/225000 [90:57:38<355:21:59,  7.01s/it]
 19%|█▉        | 42470/225000 [90:57:44<335:31:17,  6.62s/it]
 19%|█▉        | 42471/225000 [90:57:49<314:06:50,  6.20s/it]
 19%|█▉        | 42472/225000 [90:57:55<303:15:17,  5.98s/it]
 19%|█▉        | 42473/225000 [90:58:01<302:00:53,  5.96s/it]
 19%|█▉        | 42474/225000 [90:58:06<298:43:07,  5.89s/it]
 19%|█▉        | 42475/225000 [90:58:18<385:03:28,  7.59s/it]
 19%|█▉        | 42476/225000 [90:58:25<381:35:29,  7.53s/it]
 19%|█▉        | 42477/225000 [90:58:29<325:35:28,  6.42s/it]
 19%|█▉        | 42478/225000 [90:58:41<401:30:25,  7.92s/it]
 19%|█▉        | 42479/225000 [90:58:45<350:36:33,  6.92s/it]
 19%|█▉        | 42480/225000 [90:58:50<313:12:38,  6.18s/it]
 19%|█▉        | 42481/225000 [90:59:00<368:41:17,  7.27s/it]
 19%|█▉        | 42482/225000 [90:59:06<352:32:09,  6.95s/it]
 19%|█▉        | 42483/225000 [90:59:11<332:07:21,  6.55s/it]
 19%|█▉        | 42484/225000 [90:59:17<318:41:38,  6.29s/it]
 19%|█▉        | 42485/225000 [90:59:22<305:45:58,  6.03s/it]
 19%|█▉        | 42486/225000 [90:59:34<382:16:48,  7.54s/it]
 19%|█▉        | 42487/225000 [90:59:40<366:40:38,  7.23s/it]
 19%|█▉        | 42488/225000 [90:59:48<379:04:55,  7.48s/it]
 19%|█▉        | 42489/225000 [90:59:54<353:58:30,  6.98s/it]
 19%|█▉        | 42490/225000 [91:00:02<365:46:16,  7.21s/it]
 19%|█▉        | 42491/225000 [91:00:06<323:03:54,  6.37s/it]
 19%|█▉        | 42492/225000 [91:00:17<396:46:07,  7.83s/it]
 19%|█▉        | 42493/225000 [91:00:21<341:10:26,  6.73s/it]
 19%|█▉        | 42494/225000 [91:00:27<321:58:43,  6.35s/it]
 19%|█▉        | 42495/225000 [91:00:33<313:22:11,  6.18s/it]
 19%|█▉        | 42496/225000 [91:00:38<304:55:21,  6.01s/it]
 19%|█▉        | 42497/225000 [91:00:44<298:09:58,  5.88s/it]
 19%|█▉        | 42498/225000 [91:00:50<299:12:43,  5.90s/it]
 19%|█▉        | 42499/225000 [91:00:56<303:40:12,  5.99s/it]
 19%|█▉        | 42500/225000 [91:01:09<403:17:16,  7.96s/it]
                                                             
{'loss': '0.6966', 'grad_norm': '0.6825', 'learning_rate': '4.724e-05', 'epoch': '0.1889', 'num_input_tokens_seen': 153551824, 'train_runtime': '3.277e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42500/225000 [91:01:09<403:17:16,  7.96s/it]
 19%|█▉        | 42501/225000 [91:01:15<373:48:47,  7.37s/it]
 19%|█▉        | 42502/225000 [91:01:22<375:31:13,  7.41s/it]
 19%|█▉        | 42503/225000 [91:01:29<369:28:43,  7.29s/it]
 19%|█▉        | 42504/225000 [91:01:35<341:08:00,  6.73s/it]
 19%|█▉        | 42505/225000 [91:01:40<322:24:16,  6.36s/it]
 19%|█▉        | 42506/225000 [91:01:45<308:14:35,  6.08s/it]
 19%|█▉        | 42507/225000 [91:01:51<302:26:54,  5.97s/it]
 19%|█▉        | 42508/225000 [91:01:56<292:10:59,  5.76s/it]
 19%|█▉        | 42509/225000 [91:02:10<416:47:22,  8.22s/it]
 19%|█▉        | 42510/225000 [91:02:17<393:40:55,  7.77s/it]
 19%|█▉        | 42511/225000 [91:02:23<371:54:40,  7.34s/it]
 19%|█▉        | 42512/225000 [91:02:31<369:11:44,  7.28s/it]
 19%|█▉        | 42513/225000 [91:02:38<377:53:59,  7.45s/it]
 19%|█▉        | 42514/225000 [91:02:42<324:00:01,  6.39s/it]
 19%|█▉        | 42515/225000 [91:02:53<386:14:13,  7.62s/it]
 19%|█▉        | 42516/225000 [91:02:57<335:04:26,  6.61s/it]
 19%|█▉        | 42517/225000 [91:03:03<325:04:08,  6.41s/it]
 19%|█▉        | 42518/225000 [91:03:09<318:00:16,  6.27s/it]
 19%|█▉        | 42519/225000 [91:03:15<313:38:16,  6.19s/it]
 19%|█▉        | 42520/225000 [91:03:21<303:59:35,  6.00s/it]
 19%|█▉        | 42521/225000 [91:03:26<302:02:09,  5.96s/it]
 19%|█▉        | 42522/225000 [91:03:39<405:20:14,  8.00s/it]
 19%|█▉        | 42523/225000 [91:03:45<373:25:13,  7.37s/it]
 19%|█▉        | 42524/225000 [91:03:50<335:16:54,  6.61s/it]
 19%|█▉        | 42525/225000 [91:03:55<306:37:48,  6.05s/it]
 19%|█▉        | 42526/225000 [91:04:00<290:52:26,  5.74s/it]
 19%|█▉        | 42527/225000 [91:04:11<378:24:08,  7.47s/it]
 19%|█▉        | 42528/225000 [91:04:18<374:49:08,  7.39s/it]
 19%|█▉        | 42529/225000 [91:04:22<319:03:36,  6.29s/it]
 19%|█▉        | 42530/225000 [91:04:30<344:27:32,  6.80s/it]
 19%|█▉        | 42531/225000 [91:04:35<319:53:36,  6.31s/it]
 19%|█▉        | 42532/225000 [91:04:44<351:11:30,  6.93s/it]
 19%|█▉        | 42533/225000 [91:04:49<330:19:21,  6.52s/it]
 19%|█▉        | 42534/225000 [91:04:55<318:13:19,  6.28s/it]
 19%|█▉        | 42535/225000 [91:05:00<306:07:46,  6.04s/it]
 19%|█▉        | 42536/225000 [91:05:06<306:00:38,  6.04s/it]
 19%|█▉        | 42537/225000 [91:05:12<291:27:28,  5.75s/it]
 19%|█▉        | 42538/225000 [91:05:17<290:23:46,  5.73s/it]
 19%|█▉        | 42539/225000 [91:05:25<320:03:09,  6.31s/it]
 19%|█▉        | 42540/225000 [91:05:37<412:07:47,  8.13s/it]
 19%|█▉        | 42541/225000 [91:05:42<363:28:48,  7.17s/it]
 19%|█▉        | 42542/225000 [91:05:47<334:58:01,  6.61s/it]
 19%|█▉        | 42543/225000 [91:05:53<314:51:21,  6.21s/it]
 19%|█▉        | 42544/225000 [91:06:00<323:37:44,  6.39s/it]
 19%|█▉        | 42545/225000 [91:06:14<441:33:15,  8.71s/it]
 19%|█▉        | 42546/225000 [91:06:18<371:58:49,  7.34s/it]
 19%|█▉        | 42547/225000 [91:06:25<367:38:45,  7.25s/it]
 19%|█▉        | 42548/225000 [91:06:33<375:32:01,  7.41s/it]
 19%|█▉        | 42549/225000 [91:06:38<341:56:48,  6.75s/it]
 19%|█▉        | 42550/225000 [91:06:44<333:41:09,  6.58s/it]
                                                             
{'loss': '0.682', 'grad_norm': '0.69', 'learning_rate': '4.723e-05', 'epoch': '0.1891', 'num_input_tokens_seen': 153702752, 'train_runtime': '3.28e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42550/225000 [91:06:44<333:41:09,  6.58s/it]
 19%|█▉        | 42551/225000 [91:06:51<340:30:10,  6.72s/it]
 19%|█▉        | 42552/225000 [91:06:57<328:04:49,  6.47s/it]
 19%|█▉        | 42553/225000 [91:07:05<358:29:16,  7.07s/it]
 19%|█▉        | 42554/225000 [91:07:13<367:17:29,  7.25s/it]
 19%|█▉        | 42555/225000 [91:07:18<328:35:05,  6.48s/it]
 19%|█▉        | 42556/225000 [91:07:24<325:18:58,  6.42s/it]
 19%|█▉        | 42557/225000 [91:07:34<376:16:38,  7.42s/it]
 19%|█▉        | 42558/225000 [91:07:38<320:17:43,  6.32s/it]
 19%|█▉        | 42559/225000 [91:07:46<346:02:10,  6.83s/it]
 19%|█▉        | 42560/225000 [91:07:52<339:05:17,  6.69s/it]
 19%|█▉        | 42561/225000 [91:07:58<335:53:38,  6.63s/it]
 19%|█▉        | 42562/225000 [91:08:07<366:32:59,  7.23s/it]
 19%|█▉        | 42563/225000 [91:08:11<323:08:51,  6.38s/it]
 19%|█▉        | 42564/225000 [91:08:19<347:24:44,  6.86s/it]
 19%|█▉        | 42565/225000 [91:08:24<310:34:56,  6.13s/it]
 19%|█▉        | 42566/225000 [91:08:29<291:02:23,  5.74s/it]
 19%|█▉        | 42567/225000 [91:08:35<305:08:52,  6.02s/it]
 19%|█▉        | 42568/225000 [91:08:47<395:56:16,  7.81s/it]
 19%|█▉        | 42569/225000 [91:08:51<337:49:00,  6.67s/it]
 19%|█▉        | 42570/225000 [91:09:01<376:55:39,  7.44s/it]
 19%|█▉        | 42571/225000 [91:09:06<345:07:15,  6.81s/it]
 19%|█▉        | 42572/225000 [91:09:11<323:23:51,  6.38s/it]
 19%|█▉        | 42573/225000 [91:09:17<314:07:08,  6.20s/it]
 19%|█▉        | 42574/225000 [91:09:23<304:18:23,  6.01s/it]
 19%|█▉        | 42575/225000 [91:09:28<297:23:55,  5.87s/it]
 19%|█▉        | 42576/225000 [91:09:34<295:13:12,  5.83s/it]
 19%|█▉        | 42577/225000 [91:09:40<302:58:08,  5.98s/it]
 19%|█▉        | 42578/225000 [91:09:54<417:44:27,  8.24s/it]
 19%|█▉        | 42579/225000 [91:09:58<360:19:42,  7.11s/it]
 19%|█▉        | 42580/225000 [91:10:05<349:25:07,  6.90s/it]
 19%|█▉        | 42581/225000 [91:10:16<411:51:48,  8.13s/it]
 19%|█▉        | 42582/225000 [91:10:21<366:55:01,  7.24s/it]
 19%|█▉        | 42583/225000 [91:10:26<334:13:18,  6.60s/it]
 19%|█▉        | 42584/225000 [91:10:31<308:05:37,  6.08s/it]
 19%|█▉        | 42585/225000 [91:10:36<291:55:58,  5.76s/it]
 19%|█▉        | 42586/225000 [91:10:41<285:46:16,  5.64s/it]
 19%|█▉        | 42587/225000 [91:10:46<272:52:07,  5.39s/it]
 19%|█▉        | 42588/225000 [91:10:56<344:41:13,  6.80s/it]
 19%|█▉        | 42589/225000 [91:11:08<417:49:51,  8.25s/it]
 19%|█▉        | 42590/225000 [91:11:12<363:29:06,  7.17s/it]
 19%|█▉        | 42591/225000 [91:11:21<381:01:13,  7.52s/it]
 19%|█▉        | 42592/225000 [91:11:25<329:47:38,  6.51s/it]
 19%|█▉        | 42593/225000 [91:11:36<398:32:50,  7.87s/it]
 19%|█▉        | 42594/225000 [91:11:40<334:36:41,  6.60s/it]
 19%|█▉        | 42595/225000 [91:11:44<307:18:56,  6.07s/it]
 19%|█▉        | 42596/225000 [91:11:50<306:46:06,  6.05s/it]
 19%|█▉        | 42597/225000 [91:11:56<299:44:07,  5.92s/it]
 19%|█▉        | 42598/225000 [91:12:02<294:29:25,  5.81s/it]
 19%|█▉        | 42599/225000 [91:12:07<294:59:18,  5.82s/it]
 19%|█▉        | 42600/225000 [91:12:13<288:59:55,  5.70s/it]
                                                             
{'loss': '0.7015', 'grad_norm': '0.8544', 'learning_rate': '4.722e-05', 'epoch': '0.1893', 'num_input_tokens_seen': 153854072, 'train_runtime': '3.283e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42600/225000 [91:12:13<288:59:55,  5.70s/it]
 19%|█▉        | 42601/225000 [91:12:25<382:49:51,  7.56s/it]
 19%|█▉        | 42602/225000 [91:12:30<340:54:45,  6.73s/it]
 19%|█▉        | 42603/225000 [91:12:39<376:50:07,  7.44s/it]
 19%|█▉        | 42604/225000 [91:12:44<340:43:25,  6.72s/it]
 19%|█▉        | 42605/225000 [91:12:49<314:26:22,  6.21s/it]
 19%|█▉        | 42606/225000 [91:12:54<294:52:36,  5.82s/it]
 19%|█▉        | 42607/225000 [91:12:58<279:23:19,  5.51s/it]
 19%|█▉        | 42608/225000 [91:13:09<350:47:31,  6.92s/it]
 19%|█▉        | 42609/225000 [91:13:19<406:16:32,  8.02s/it]
 19%|█▉        | 42610/225000 [91:13:26<389:15:27,  7.68s/it]
 19%|█▉        | 42611/225000 [91:13:31<342:13:12,  6.75s/it]
 19%|█▉        | 42612/225000 [91:13:36<318:56:30,  6.30s/it]
 19%|█▉        | 42613/225000 [91:13:40<289:50:24,  5.72s/it]
 19%|█▉        | 42614/225000 [91:13:54<407:47:58,  8.05s/it]
 19%|█▉        | 42615/225000 [91:13:59<361:50:08,  7.14s/it]
 19%|█▉        | 42616/225000 [91:14:03<325:01:10,  6.42s/it]
 19%|█▉        | 42617/225000 [91:14:08<298:36:54,  5.89s/it]
 19%|█▉        | 42618/225000 [91:14:13<285:57:40,  5.64s/it]
 19%|█▉        | 42619/225000 [91:14:26<387:58:10,  7.66s/it]
 19%|█▉        | 42620/225000 [91:14:31<347:07:18,  6.85s/it]
 19%|█▉        | 42621/225000 [91:14:40<387:09:30,  7.64s/it]
 19%|█▉        | 42622/225000 [91:14:45<339:58:51,  6.71s/it]
 19%|█▉        | 42623/225000 [91:14:49<307:37:33,  6.07s/it]
 19%|█▉        | 42624/225000 [91:14:53<281:01:19,  5.55s/it]
 19%|█▉        | 42625/225000 [91:15:06<392:28:48,  7.75s/it]
 19%|█▉        | 42626/225000 [91:15:11<344:39:26,  6.80s/it]
 19%|█▉        | 42627/225000 [91:15:16<314:18:04,  6.20s/it]
 19%|█▉        | 42628/225000 [91:15:21<292:14:27,  5.77s/it]
 19%|█▉        | 42629/225000 [91:15:32<371:42:54,  7.34s/it]
 19%|█▉        | 42630/225000 [91:15:39<366:49:33,  7.24s/it]
 19%|█▉        | 42631/225000 [91:15:46<377:18:29,  7.45s/it]
 19%|█▉        | 42632/225000 [91:15:50<321:45:49,  6.35s/it]
 19%|█▉        | 42633/225000 [91:16:00<365:51:22,  7.22s/it]
 19%|█▉        | 42634/225000 [91:16:05<333:04:17,  6.58s/it]
 19%|█▉        | 42635/225000 [91:16:10<309:37:46,  6.11s/it]
 19%|█▉        | 42636/225000 [91:16:15<297:23:39,  5.87s/it]
 19%|█▉        | 42637/225000 [91:16:20<287:36:39,  5.68s/it]
 19%|█▉        | 42638/225000 [91:16:26<288:19:55,  5.69s/it]
 19%|█▉        | 42639/225000 [91:16:35<347:16:15,  6.86s/it]
 19%|█▉        | 42640/225000 [91:16:47<415:31:14,  8.20s/it]
 19%|█▉        | 42641/225000 [91:16:52<373:31:43,  7.37s/it]
 19%|█▉        | 42642/225000 [91:16:58<352:00:50,  6.95s/it]
 19%|█▉        | 42643/225000 [91:17:04<327:26:41,  6.46s/it]
 19%|█▉        | 42644/225000 [91:17:09<319:02:52,  6.30s/it]
 19%|█▉        | 42645/225000 [91:17:19<372:43:30,  7.36s/it]
 19%|█▉        | 42646/225000 [91:17:27<373:15:19,  7.37s/it]
 19%|█▉        | 42647/225000 [91:17:31<325:16:44,  6.42s/it]
 19%|█▉        | 42648/225000 [91:17:42<392:00:42,  7.74s/it]
 19%|█▉        | 42649/225000 [91:17:47<348:56:42,  6.89s/it]
 19%|█▉        | 42650/225000 [91:17:54<359:49:59,  7.10s/it]
                                                             
{'loss': '0.6731', 'grad_norm': '0.7364', 'learning_rate': '4.721e-05', 'epoch': '0.1896', 'num_input_tokens_seen': 154005264, 'train_runtime': '3.287e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42650/225000 [91:17:54<359:49:59,  7.10s/it]
 19%|█▉        | 42651/225000 [91:17:58<310:21:46,  6.13s/it]
 19%|█▉        | 42652/225000 [91:18:07<350:05:20,  6.91s/it]
 19%|█▉        | 42653/225000 [91:18:14<355:46:50,  7.02s/it]
 19%|█▉        | 42654/225000 [91:18:19<327:25:23,  6.46s/it]
 19%|█▉        | 42655/225000 [91:18:24<306:47:05,  6.06s/it]
 19%|█▉        | 42656/225000 [91:18:29<291:45:07,  5.76s/it]
 19%|█▉        | 42657/225000 [91:18:38<335:35:53,  6.63s/it]
 19%|█▉        | 42658/225000 [91:18:49<408:47:59,  8.07s/it]
 19%|█▉        | 42659/225000 [91:18:53<343:52:32,  6.79s/it]
 19%|█▉        | 42660/225000 [91:19:01<358:32:17,  7.08s/it]
 19%|█▉        | 42661/225000 [91:19:06<323:05:37,  6.38s/it]
 19%|█▉        | 42662/225000 [91:19:11<299:55:49,  5.92s/it]
 19%|█▉        | 42663/225000 [91:19:15<282:44:38,  5.58s/it]
 19%|█▉        | 42664/225000 [91:19:28<384:36:19,  7.59s/it]
 19%|█▉        | 42665/225000 [91:19:35<384:46:55,  7.60s/it]
 19%|█▉        | 42666/225000 [91:19:40<346:45:30,  6.85s/it]
 19%|█▉        | 42667/225000 [91:19:48<352:27:17,  6.96s/it]
 19%|█▉        | 42668/225000 [91:19:53<327:01:09,  6.46s/it]
 19%|█▉        | 42669/225000 [91:19:58<306:38:29,  6.05s/it]
 19%|█▉        | 42670/225000 [91:20:03<289:52:32,  5.72s/it]
 19%|█▉        | 42671/225000 [91:20:08<278:54:45,  5.51s/it]
 19%|█▉        | 42672/225000 [91:20:20<376:05:18,  7.43s/it]
 19%|█▉        | 42673/225000 [91:20:28<385:04:29,  7.60s/it]
 19%|█▉        | 42674/225000 [91:20:35<373:45:52,  7.38s/it]
 19%|█▉        | 42675/225000 [91:20:41<353:53:35,  6.99s/it]
 19%|█▉        | 42676/225000 [91:20:48<356:33:40,  7.04s/it]
 19%|█▉        | 42677/225000 [91:20:55<359:46:44,  7.10s/it]
 19%|█▉        | 42678/225000 [91:21:00<325:25:04,  6.43s/it]
 19%|█▉        | 42679/225000 [91:21:05<306:32:31,  6.05s/it]
 19%|█▉        | 42680/225000 [91:21:11<303:44:23,  6.00s/it]
 19%|█▉        | 42681/225000 [91:21:24<406:23:59,  8.02s/it]
 19%|█▉        | 42682/225000 [91:21:28<344:34:16,  6.80s/it]
 19%|█▉        | 42683/225000 [91:21:37<382:44:05,  7.56s/it]
 19%|█▉        | 42684/225000 [91:21:42<338:19:11,  6.68s/it]
 19%|█▉        | 42685/225000 [91:21:47<308:42:24,  6.10s/it]
 19%|█▉        | 42686/225000 [91:21:52<291:30:40,  5.76s/it]
 19%|█▉        | 42687/225000 [91:22:04<397:33:42,  7.85s/it]
 19%|█▉        | 42688/225000 [91:22:09<343:06:54,  6.78s/it]
 19%|█▉        | 42689/225000 [91:22:13<313:58:09,  6.20s/it]
 19%|█▉        | 42690/225000 [91:22:19<298:54:48,  5.90s/it]
 19%|█▉        | 42691/225000 [91:22:23<282:55:22,  5.59s/it]
 19%|█▉        | 42692/225000 [91:22:36<396:15:41,  7.82s/it]
 19%|█▉        | 42693/225000 [91:22:43<369:26:14,  7.30s/it]
 19%|█▉        | 42694/225000 [91:22:48<340:24:08,  6.72s/it]
 19%|█▉        | 42695/225000 [91:22:53<317:23:23,  6.27s/it]
 19%|█▉        | 42696/225000 [91:22:59<311:58:37,  6.16s/it]
 19%|█▉        | 42697/225000 [91:23:05<306:07:03,  6.05s/it]
 19%|█▉        | 42698/225000 [91:23:11<314:24:53,  6.21s/it]
 19%|█▉        | 42699/225000 [91:23:25<429:30:04,  8.48s/it]
 19%|█▉        | 42700/225000 [91:23:30<369:04:51,  7.29s/it]
                                                             
{'loss': '0.6874', 'grad_norm': '0.6325', 'learning_rate': '4.72e-05', 'epoch': '0.1898', 'num_input_tokens_seen': 154158416, 'train_runtime': '3.29e+05', 'train_tokens_per_second': '468.6'}

 19%|█▉        | 42700/225000 [91:23:30<369:04:51,  7.29s/it]
 19%|█▉        | 42701/225000 [91:23:34<319:58:27,  6.32s/it]
 19%|█▉        | 42702/225000 [91:23:39<311:07:24,  6.14s/it]
 19%|█▉        | 42703/225000 [91:23:51<394:51:25,  7.80s/it]
 19%|█▉        | 42704/225000 [91:23:55<332:21:18,  6.56s/it]
 19%|█▉        | 42705/225000 [91:24:03<349:45:36,  6.91s/it]
 19%|█▉        | 42706/225000 [91:24:07<311:50:58,  6.16s/it]
 19%|█▉        | 42707/225000 [91:24:15<336:12:36,  6.64s/it]
 19%|█▉        | 42708/225000 [91:24:24<374:03:44,  7.39s/it]
 19%|█▉        | 42709/225000 [91:24:30<348:35:23,  6.88s/it]
 19%|█▉        | 42710/225000 [91:24:37<356:47:02,  7.05s/it]
 19%|█▉        | 42711/225000 [91:24:43<341:24:14,  6.74s/it]
 19%|█▉        | 42712/225000 [91:24:50<349:12:10,  6.90s/it]
 19%|█▉        | 42713/225000 [91:24:55<317:04:23,  6.26s/it]
 19%|█▉        | 42714/225000 [91:25:00<294:27:15,  5.82s/it]
 19%|█▉        | 42715/225000 [91:25:09<349:16:01,  6.90s/it]
 19%|█▉        | 42716/225000 [91:25:15<338:58:43,  6.69s/it]
 19%|█▉        | 42717/225000 [91:25:26<392:19:31,  7.75s/it]
 19%|█▉        | 42718/225000 [91:25:30<344:39:46,  6.81s/it]
 19%|█▉        | 42719/225000 [91:25:35<315:07:01,  6.22s/it]
 19%|█▉        | 42720/225000 [91:25:40<296:40:50,  5.86s/it]
 19%|█▉        | 42721/225000 [91:25:51<366:10:47,  7.23s/it]
 19%|█▉        | 42722/225000 [91:25:58<364:17:10,  7.19s/it]
 19%|█▉        | 42723/225000 [91:26:02<325:33:05,  6.43s/it]
 19%|█▉        | 42724/225000 [91:26:07<296:54:02,  5.86s/it]
 19%|█▉        | 42725/225000 [91:26:17<359:19:11,  7.10s/it]
 19%|█▉        | 42726/225000 [91:26:23<345:57:50,  6.83s/it]
 19%|█▉        | 42727/225000 [91:26:29<331:27:04,  6.55s/it]
 19%|█▉        | 42728/225000 [91:26:34<313:26:48,  6.19s/it]
 19%|█▉        | 42729/225000 [91:26:39<296:07:38,  5.85s/it]
 19%|█▉        | 42730/225000 [91:26:49<358:32:01,  7.08s/it]
 19%|█▉        | 42731/225000 [91:26:59<394:20:46,  7.79s/it]
 19%|█▉        | 42732/225000 [91:27:03<335:01:03,  6.62s/it]
 19%|█▉        | 42733/225000 [91:27:12<382:03:30,  7.55s/it]
 19%|█▉        | 42734/225000 [91:27:17<331:51:22,  6.55s/it]
 19%|█▉        | 42735/225000 [91:27:21<305:47:01,  6.04s/it]
 19%|█▉        | 42736/225000 [91:27:26<290:45:34,  5.74s/it]
 19%|█▉        | 42737/225000 [91:27:35<334:31:18,  6.61s/it]
 19%|█▉        | 42738/225000 [91:27:46<402:58:45,  7.96s/it]
 19%|█▉        | 42739/225000 [91:27:51<349:18:02,  6.90s/it]
 19%|█▉        | 42740/225000 [91:27:59<375:32:34,  7.42s/it]
 19%|█▉        | 42741/225000 [91:28:04<329:50:57,  6.52s/it]
 19%|█▉        | 42742/225000 [91:28:08<300:13:20,  5.93s/it]
 19%|█▉        | 42743/225000 [91:28:14<290:02:45,  5.73s/it]
 19%|█▉        | 42744/225000 [91:28:27<406:13:18,  8.02s/it]
 19%|█▉        | 42745/225000 [91:28:32<362:41:21,  7.16s/it]
 19%|█▉        | 42746/225000 [91:28:37<333:14:51,  6.58s/it]
 19%|█▉        | 42747/225000 [91:28:43<315:21:55,  6.23s/it]
 19%|█▉        | 42748/225000 [91:28:48<305:44:18,  6.04s/it]
 19%|█▉        | 42749/225000 [91:28:54<299:17:15,  5.91s/it]
 19%|█▉        | 42750/225000 [91:29:06<392:59:18,  7.76s/it]
                                                             
{'loss': '0.6921', 'grad_norm': '0.7447', 'learning_rate': '4.72e-05', 'epoch': '0.19', 'num_input_tokens_seen': 154310832, 'train_runtime': '3.293e+05', 'train_tokens_per_second': '468.5'}

 19%|█▉        | 42750/225000 [91:29:06<392:59:18,  7.76s/it]
 19%|█▉        | 42751/225000 [91:29:12<369:52:56,  7.31s/it]
 19%|█▉        | 42752/225000 [91:29:20<383:35:18,  7.58s/it]
 19%|█▉        | 42753/225000 [91:29:26<354:33:39,  7.00s/it]
 19%|█▉        | 42754/225000 [91:29:32<342:15:58,  6.76s/it]
 19%|█▉        | 42755/225000 [91:29:37<314:31:19,  6.21s/it]
 19%|█▉        | 42756/225000 [91:29:48<387:03:03,  7.65s/it]
 19%|█▉        | 42757/225000 [91:29:52<331:57:35,  6.56s/it]
 19%|█▉        | 42758/225000 [91:30:03<397:58:04,  7.86s/it]
 19%|█▉        | 42759/225000 [91:30:07<338:31:23,  6.69s/it]
 19%|█▉        | 42760/225000 [91:30:12<316:34:32,  6.25s/it]
 19%|█▉        | 42761/225000 [91:30:18<308:04:10,  6.09s/it]
 19%|█▉        | 42762/225000 [91:30:24<303:00:59,  5.99s/it]
 19%|█▉        | 42763/225000 [91:30:30<300:57:48,  5.95s/it]
 19%|█▉        | 42764/225000 [91:30:41<380:10:42,  7.51s/it]
 19%|█▉        | 42765/225000 [91:30:45<336:57:52,  6.66s/it]
 19%|█▉        | 42766/225000 [91:30:51<322:18:15,  6.37s/it]
 19%|█▉        | 42767/225000 [91:31:03<409:01:30,  8.08s/it]
 19%|█▉        | 42768/225000 [91:31:08<364:35:29,  7.20s/it]
 19%|█▉        | 42769/225000 [91:31:14<336:00:58,  6.64s/it]
 19%|█▉        | 42770/225000 [91:31:19<314:51:46,  6.22s/it]
 19%|█▉        | 42771/225000 [91:31:24<303:19:26,  5.99s/it]
 19%|█▉        | 42772/225000 [91:31:33<347:10:59,  6.86s/it]
 19%|█▉        | 42773/225000 [91:31:42<369:52:08,  7.31s/it]
 19%|█▉        | 42774/225000 [91:31:49<364:57:27,  7.21s/it]
 19%|█▉        | 42775/225000 [91:31:54<334:55:13,  6.62s/it]
 19%|█▉        | 42776/225000 [91:32:02<354:19:27,  7.00s/it]
 19%|█▉        | 42777/225000 [91:32:05<301:25:19,  5.95s/it]
 19%|█▉        | 42778/225000 [91:32:16<368:30:59,  7.28s/it]
 19%|█▉        | 42779/225000 [91:32:20<320:13:42,  6.33s/it]
 19%|█▉        | 42780/225000 [91:32:29<371:38:32,  7.34s/it]
 19%|█▉        | 42781/225000 [91:32:34<323:40:21,  6.39s/it]
 19%|█▉        | 42782/225000 [91:32:38<290:20:52,  5.74s/it]
 19%|█▉        | 42783/225000 [91:32:49<378:31:42,  7.48s/it]
 19%|█▉        | 42784/225000 [91:32:54<330:52:15,  6.54s/it]
 19%|█▉        | 42785/225000 [91:33:02<355:09:56,  7.02s/it]
 19%|█▉        | 42786/225000 [91:33:09<350:05:20,  6.92s/it]
 19%|█▉        | 42787/225000 [91:33:14<325:23:14,  6.43s/it]
 19%|█▉        | 42788/225000 [91:33:18<295:54:16,  5.85s/it]
 19%|█▉        | 42789/225000 [91:33:23<284:14:55,  5.62s/it]
 19%|█▉        | 42790/225000 [91:33:31<313:46:16,  6.20s/it]
 19%|█▉        | 42791/225000 [91:33:43<404:52:47,  8.00s/it]
 19%|█▉        | 42792/225000 [91:33:47<339:14:47,  6.70s/it]
 19%|█▉        | 42793/225000 [91:33:53<329:53:30,  6.52s/it]
 19%|█▉        | 42794/225000 [91:34:00<345:13:47,  6.82s/it]
 19%|█▉        | 42795/225000 [91:34:05<312:52:15,  6.18s/it]
 19%|█▉        | 42796/225000 [91:34:09<280:12:34,  5.54s/it]
 19%|█▉        | 42797/225000 [91:34:22<390:42:27,  7.72s/it]
 19%|█▉        | 42798/225000 [91:34:27<355:02:28,  7.02s/it]
 19%|█▉        | 42799/225000 [91:34:32<325:40:44,  6.43s/it]
 19%|█▉        | 42800/225000 [91:34:38<307:46:24,  6.08s/it]
                                                             
{'loss': '0.6766', 'grad_norm': '0.7958', 'learning_rate': '4.719e-05', 'epoch': '0.1902', 'num_input_tokens_seen': 154462336, 'train_runtime': '3.297e+05', 'train_tokens_per_second': '468.5'}

 19%|█▉        | 42800/225000 [91:34:38<307:46:24,  6.08s/it]
 19%|█▉        | 42801/225000 [91:34:43<291:03:20,  5.75s/it]
 19%|█▉        | 42802/225000 [91:34:48<279:50:25,  5.53s/it]
 19%|█▉        | 42803/225000 [91:35:02<409:27:58,  8.09s/it]
 19%|█▉        | 42804/225000 [91:35:07<365:14:05,  7.22s/it]
 19%|█▉        | 42805/225000 [91:35:14<357:40:32,  7.07s/it]
 19%|█▉        | 42806/225000 [91:35:20<340:46:32,  6.73s/it]
 19%|█▉        | 42807/225000 [91:35:26<334:21:07,  6.61s/it]
 19%|█▉        | 42808/225000 [91:35:32<328:01:01,  6.48s/it]
 19%|█▉        | 42809/225000 [91:35:38<312:16:50,  6.17s/it]
 19%|█▉        | 42810/225000 [91:35:49<387:59:11,  7.67s/it]
 19%|█▉        | 42811/225000 [91:35:53<343:59:50,  6.80s/it]
 19%|█▉        | 42812/225000 [91:35:58<313:31:41,  6.20s/it]
 19%|█▉        | 42813/225000 [91:36:10<393:11:49,  7.77s/it]
 19%|█▉        | 42814/225000 [91:36:14<346:29:03,  6.85s/it]
 19%|█▉        | 42815/225000 [91:36:23<376:43:42,  7.44s/it]
 19%|█▉        | 42816/225000 [91:36:29<344:13:44,  6.80s/it]
 19%|█▉        | 42817/225000 [91:36:33<316:17:49,  6.25s/it]
 19%|█▉        | 42818/225000 [91:36:38<292:26:36,  5.78s/it]
 19%|█▉        | 42819/225000 [91:36:43<283:20:43,  5.60s/it]
 19%|█▉        | 42820/225000 [91:36:48<271:43:21,  5.37s/it]
 19%|█▉        | 42821/225000 [91:36:55<295:07:19,  5.83s/it]
 19%|█▉        | 42822/225000 [91:37:08<406:30:20,  8.03s/it]
 19%|█▉        | 42823/225000 [91:37:13<351:48:17,  6.95s/it]
 19%|█▉        | 42824/225000 [91:37:17<314:49:47,  6.22s/it]
 19%|█▉        | 42825/225000 [91:37:23<307:08:39,  6.07s/it]
 19%|█▉        | 42826/225000 [91:37:35<402:21:39,  7.95s/it]
 19%|█▉        | 42827/225000 [91:37:40<349:27:07,  6.91s/it]
 19%|█▉        | 42828/225000 [91:37:45<318:50:20,  6.30s/it]
 19%|█▉        | 42829/225000 [91:37:50<298:33:48,  5.90s/it]
 19%|█▉        | 42830/225000 [91:37:55<296:19:37,  5.86s/it]
 19%|█▉        | 42831/225000 [91:38:08<392:56:19,  7.77s/it]
 19%|█▉        | 42832/225000 [91:38:14<374:45:58,  7.41s/it]
 19%|█▉        | 42833/225000 [91:38:22<385:31:53,  7.62s/it]
 19%|█▉        | 42834/225000 [91:38:27<336:34:54,  6.65s/it]
 19%|█▉        | 42835/225000 [91:38:31<303:57:13,  6.01s/it]
 19%|█▉        | 42836/225000 [91:38:36<283:04:44,  5.59s/it]
 19%|█▉        | 42837/225000 [91:38:41<280:56:28,  5.55s/it]
 19%|█▉        | 42838/225000 [91:38:54<388:40:33,  7.68s/it]
 19%|█▉        | 42839/225000 [91:39:01<384:59:11,  7.61s/it]
 19%|█▉        | 42840/225000 [91:39:08<370:43:22,  7.33s/it]
 19%|█▉        | 42841/225000 [91:39:12<326:54:55,  6.46s/it]
 19%|█▉        | 42842/225000 [91:39:17<297:52:09,  5.89s/it]
 19%|█▉        | 42843/225000 [91:39:26<343:44:39,  6.79s/it]
 19%|█▉        | 42844/225000 [91:39:35<381:36:16,  7.54s/it]
 19%|█▉        | 42845/225000 [91:39:44<401:50:04,  7.94s/it]
 19%|█▉        | 42846/225000 [91:39:48<337:58:46,  6.68s/it]
 19%|█▉        | 42847/225000 [91:39:56<366:07:55,  7.24s/it]
 19%|█▉        | 42848/225000 [91:40:01<324:17:40,  6.41s/it]
 19%|█▉        | 42849/225000 [91:40:05<297:05:25,  5.87s/it]
 19%|█▉        | 42850/225000 [91:40:18<394:22:15,  7.79s/it]
                                                             
{'loss': '0.705', 'grad_norm': '0.7662', 'learning_rate': '4.718e-05', 'epoch': '0.1904', 'num_input_tokens_seen': 154613200, 'train_runtime': '3.3e+05', 'train_tokens_per_second': '468.5'}

 19%|█▉        | 42850/225000 [91:40:18<394:22:15,  7.79s/it]
 19%|█▉        | 42851/225000 [91:40:21<328:03:23,  6.48s/it]
 19%|█▉        | 42852/225000 [91:40:27<317:06:49,  6.27s/it]
 19%|█▉        | 42853/225000 [91:40:32<301:39:16,  5.96s/it]
 19%|█▉        | 42854/225000 [91:40:37<290:56:58,  5.75s/it]
 19%|█▉        | 42855/225000 [91:40:43<287:12:47,  5.68s/it]
 19%|█▉        | 42856/225000 [91:40:49<290:44:28,  5.75s/it]
 19%|█▉        | 42857/225000 [91:41:02<400:18:57,  7.91s/it]
 19%|█▉        | 42858/225000 [91:41:09<389:22:04,  7.70s/it]
 19%|█▉        | 42859/225000 [91:41:16<377:16:32,  7.46s/it]
 19%|█▉        | 42860/225000 [91:41:21<344:36:43,  6.81s/it]
 19%|█▉        | 42861/225000 [91:41:28<343:30:45,  6.79s/it]
 19%|█▉        | 42862/225000 [91:41:36<362:42:40,  7.17s/it]
 19%|█▉        | 42863/225000 [91:41:43<362:49:38,  7.17s/it]
 19%|█▉        | 42864/225000 [91:41:49<340:44:07,  6.73s/it]
 19%|█▉        | 42865/225000 [91:41:59<388:52:51,  7.69s/it]
 19%|█▉        | 42866/225000 [91:42:03<334:09:57,  6.61s/it]
 19%|█▉        | 42867/225000 [91:42:12<377:06:01,  7.45s/it]
 19%|█▉        | 42868/225000 [91:42:17<341:45:14,  6.76s/it]
 19%|█▉        | 42869/225000 [91:42:23<317:15:32,  6.27s/it]
 19%|█▉        | 42870/225000 [91:42:31<345:39:48,  6.83s/it]
 19%|█▉        | 42871/225000 [91:42:39<371:46:14,  7.35s/it]
 19%|█▉        | 42872/225000 [91:42:43<321:56:15,  6.36s/it]
 19%|█▉        | 42873/225000 [91:42:54<383:26:29,  7.58s/it]
 19%|█▉        | 42874/225000 [91:42:58<327:03:59,  6.46s/it]
 19%|█▉        | 42875/225000 [91:43:06<362:47:22,  7.17s/it]
 19%|█▉        | 42876/225000 [91:43:11<330:21:01,  6.53s/it]
 19%|█▉        | 42877/225000 [91:43:17<309:26:59,  6.12s/it]
 19%|█▉        | 42878/225000 [91:43:21<286:26:33,  5.66s/it]
 19%|█▉        | 42879/225000 [91:43:33<379:43:57,  7.51s/it]
 19%|█▉        | 42880/225000 [91:43:40<372:15:05,  7.36s/it]
 19%|█▉        | 42881/225000 [91:43:49<390:15:15,  7.71s/it]
 19%|█▉        | 42882/225000 [91:43:56<385:25:34,  7.62s/it]
 19%|█▉        | 42883/225000 [91:44:03<373:58:14,  7.39s/it]
 19%|█▉        | 42884/225000 [91:44:08<333:20:13,  6.59s/it]
 19%|█▉        | 42885/225000 [91:44:13<310:07:30,  6.13s/it]
 19%|█▉        | 42886/225000 [91:44:18<292:31:46,  5.78s/it]
 19%|█▉        | 42887/225000 [91:44:25<321:11:05,  6.35s/it]
 19%|█▉        | 42888/225000 [91:44:36<381:34:48,  7.54s/it]
 19%|█▉        | 42889/225000 [91:44:40<338:50:24,  6.70s/it]
 19%|█▉        | 42890/225000 [91:44:45<301:52:17,  5.97s/it]
 19%|█▉        | 42891/225000 [91:44:55<373:58:47,  7.39s/it]
 19%|█▉        | 42892/225000 [91:45:00<336:13:21,  6.65s/it]
 19%|█▉        | 42893/225000 [91:45:06<317:21:39,  6.27s/it]
 19%|█▉        | 42894/225000 [91:45:10<295:02:29,  5.83s/it]
 19%|█▉        | 42895/225000 [91:45:21<371:47:56,  7.35s/it]
 19%|█▉        | 42896/225000 [91:45:28<361:47:05,  7.15s/it]
 19%|█▉        | 42897/225000 [91:45:36<374:54:13,  7.41s/it]
 19%|█▉        | 42898/225000 [91:45:40<321:35:43,  6.36s/it]
 19%|█▉        | 42899/225000 [91:45:49<369:46:58,  7.31s/it]
 19%|█▉        | 42900/225000 [91:45:55<338:00:51,  6.68s/it]
                                                             
{'loss': '0.6768', 'grad_norm': '0.6696', 'learning_rate': '4.717e-05', 'epoch': '0.1907', 'num_input_tokens_seen': 154764520, 'train_runtime': '3.304e+05', 'train_tokens_per_second': '468.5'}

 19%|█▉        | 42900/225000 [91:45:55<338:00:51,  6.68s/it]
 19%|█▉        | 42901/225000 [91:45:59<301:25:16,  5.96s/it]
 19%|█▉        | 42902/225000 [91:46:10<377:21:26,  7.46s/it]
 19%|█▉        | 42903/225000 [91:46:15<337:42:07,  6.68s/it]
 19%|█▉        | 42904/225000 [91:46:23<366:22:06,  7.24s/it]
 19%|█▉        | 42905/225000 [91:46:31<380:40:06,  7.53s/it]
 19%|█▉        | 42906/225000 [91:46:37<349:48:44,  6.92s/it]
 19%|█▉        | 42907/225000 [91:46:45<364:10:03,  7.20s/it]
 19%|█▉        | 42908/225000 [91:46:49<311:53:55,  6.17s/it]
 19%|█▉        | 42909/225000 [91:46:58<362:15:11,  7.16s/it]
 19%|█▉        | 42910/225000 [91:47:02<319:27:44,  6.32s/it]
 19%|█▉        | 42911/225000 [91:47:11<347:17:20,  6.87s/it]
 19%|█▉        | 42912/225000 [91:47:18<361:54:13,  7.16s/it]
 19%|█▉        | 42913/225000 [91:47:24<346:14:33,  6.85s/it]
 19%|█▉        | 42914/225000 [91:47:29<317:24:26,  6.28s/it]
 19%|█▉        | 42915/225000 [91:47:35<304:28:57,  6.02s/it]
 19%|█▉        | 42916/225000 [91:47:40<293:27:06,  5.80s/it]
 19%|█▉        | 42917/225000 [91:47:46<290:21:08,  5.74s/it]
 19%|█▉        | 42918/225000 [91:47:51<288:29:44,  5.70s/it]
 19%|█▉        | 42919/225000 [91:47:57<286:14:53,  5.66s/it]
 19%|█▉        | 42920/225000 [91:48:10<402:39:32,  7.96s/it]
 19%|█▉        | 42921/225000 [91:48:16<374:23:52,  7.40s/it]
 19%|█▉        | 42922/225000 [91:48:24<375:00:42,  7.41s/it]
 19%|█▉        | 42923/225000 [91:48:32<384:32:44,  7.60s/it]
 19%|█▉        | 42924/225000 [91:48:37<349:22:32,  6.91s/it]
 19%|█▉        | 42925/225000 [91:48:43<328:57:35,  6.50s/it]
 19%|█▉        | 42926/225000 [91:48:49<328:28:35,  6.49s/it]
 19%|█▉        | 42927/225000 [91:48:57<349:27:40,  6.91s/it]
 19%|█▉        | 42928/225000 [91:49:04<349:31:30,  6.91s/it]
 19%|█▉        | 42929/225000 [91:49:09<327:48:32,  6.48s/it]
 19%|█▉        | 42930/225000 [91:49:15<313:19:52,  6.20s/it]
 19%|█▉        | 42931/225000 [91:49:20<298:13:25,  5.90s/it]
 19%|█▉        | 42932/225000 [91:49:26<297:25:37,  5.88s/it]
 19%|█▉        | 42933/225000 [91:49:31<288:09:18,  5.70s/it]
 19%|█▉        | 42934/225000 [91:49:37<285:23:12,  5.64s/it]
 19%|█▉        | 42935/225000 [91:49:43<290:50:59,  5.75s/it]
 19%|█▉        | 42936/225000 [91:49:56<398:47:33,  7.89s/it]
 19%|█▉        | 42937/225000 [91:50:05<413:35:58,  8.18s/it]
 19%|█▉        | 42938/225000 [91:50:10<370:18:06,  7.32s/it]
 19%|█▉        | 42939/225000 [91:50:15<336:03:34,  6.65s/it]
 19%|█▉        | 42940/225000 [91:50:20<313:34:22,  6.20s/it]
 19%|█▉        | 42941/225000 [91:50:26<304:17:49,  6.02s/it]
 19%|█▉        | 42942/225000 [91:50:38<405:35:03,  8.02s/it]
 19%|█▉        | 42943/225000 [91:50:44<368:00:54,  7.28s/it]
 19%|█▉        | 42944/225000 [91:50:52<386:15:27,  7.64s/it]
 19%|█▉        | 42945/225000 [91:50:57<333:23:55,  6.59s/it]
 19%|█▉        | 42946/225000 [91:51:02<319:52:15,  6.33s/it]
 19%|█▉        | 42947/225000 [91:51:08<309:36:42,  6.12s/it]
 19%|█▉        | 42948/225000 [91:51:13<301:09:45,  5.96s/it]
 19%|█▉        | 42949/225000 [91:51:19<296:52:43,  5.87s/it]
 19%|█▉        | 42950/225000 [91:51:25<292:31:58,  5.78s/it]
                                                             
{'loss': '0.6746', 'grad_norm': '0.6504', 'learning_rate': '4.716e-05', 'epoch': '0.1909', 'num_input_tokens_seen': 154915152, 'train_runtime': '3.307e+05', 'train_tokens_per_second': '468.5'}

 19%|█▉        | 42950/225000 [91:51:25<292:31:58,  5.78s/it]
 19%|█▉        | 42951/225000 [91:51:38<400:28:30,  7.92s/it]
 19%|█▉        | 42952/225000 [91:51:46<401:35:37,  7.94s/it]
 19%|█▉        | 42953/225000 [91:51:52<380:50:05,  7.53s/it]
 19%|█▉        | 42954/225000 [91:52:02<410:20:10,  8.11s/it]
 19%|█▉        | 42955/225000 [91:52:05<341:31:21,  6.75s/it]
 19%|█▉        | 42956/225000 [91:52:12<348:19:13,  6.89s/it]
 19%|█▉        | 42957/225000 [91:52:18<322:13:12,  6.37s/it]
 19%|█▉        | 42958/225000 [91:52:25<339:32:51,  6.71s/it]
 19%|█▉        | 42959/225000 [91:52:29<290:19:13,  5.74s/it]
 19%|█▉        | 42960/225000 [91:52:39<364:50:51,  7.22s/it]
 19%|█▉        | 42961/225000 [91:52:46<351:24:45,  6.95s/it]
 19%|█▉        | 42962/225000 [91:52:50<305:55:53,  6.05s/it]
 19%|█▉        | 42963/225000 [91:53:00<380:33:44,  7.53s/it]
 19%|█▉        | 42964/225000 [91:53:04<324:31:37,  6.42s/it]
 19%|█▉        | 42965/225000 [91:53:13<355:58:02,  7.04s/it]
 19%|█▉        | 42966/225000 [91:53:16<304:35:55,  6.02s/it]
 19%|█▉        | 42967/225000 [91:53:23<309:09:35,  6.11s/it]
 19%|█▉        | 42968/225000 [91:53:35<402:39:41,  7.96s/it]
 19%|█▉        | 42969/225000 [91:53:39<342:44:49,  6.78s/it]
 19%|█▉        | 42970/225000 [91:53:45<322:28:15,  6.38s/it]
 19%|█▉        | 42971/225000 [91:53:54<364:05:50,  7.20s/it]
 19%|█▉        | 42972/225000 [91:54:01<359:16:46,  7.11s/it]
 19%|█▉        | 42973/225000 [91:54:07<348:16:03,  6.89s/it]
 19%|█▉        | 42974/225000 [91:54:13<331:06:12,  6.55s/it]
 19%|█▉        | 42975/225000 [91:54:17<297:03:48,  5.88s/it]
 19%|█▉        | 42976/225000 [91:54:28<376:09:16,  7.44s/it]
 19%|█▉        | 42977/225000 [91:54:33<331:04:59,  6.55s/it]
 19%|█▉        | 42978/225000 [91:54:37<301:55:29,  5.97s/it]
 19%|█▉        | 42979/225000 [91:54:43<296:12:56,  5.86s/it]
 19%|█▉        | 42980/225000 [91:54:53<367:33:41,  7.27s/it]
 19%|█▉        | 42981/225000 [91:54:57<317:10:36,  6.27s/it]
 19%|█▉        | 42982/225000 [91:55:08<391:19:14,  7.74s/it]
 19%|█▉        | 42983/225000 [91:55:13<346:36:23,  6.86s/it]
 19%|█▉        | 42984/225000 [91:55:18<311:19:25,  6.16s/it]
 19%|█▉        | 42985/225000 [91:55:23<293:00:56,  5.80s/it]
 19%|█▉        | 42986/225000 [91:55:32<338:59:39,  6.70s/it]
 19%|█▉        | 42987/225000 [91:55:42<395:14:01,  7.82s/it]
 19%|█▉        | 42988/225000 [91:55:46<337:55:43,  6.68s/it]
 19%|█▉        | 42989/225000 [91:55:57<401:03:46,  7.93s/it]
 19%|█▉        | 42990/225000 [91:56:02<358:07:18,  7.08s/it]
 19%|█▉        | 42991/225000 [91:56:09<359:11:42,  7.10s/it]
 19%|█▉        | 42992/225000 [91:56:17<377:38:40,  7.47s/it]
 19%|█▉        | 42993/225000 [91:56:21<323:20:01,  6.40s/it]
 19%|█▉        | 42994/225000 [91:56:31<373:14:40,  7.38s/it]
 19%|█▉        | 42995/225000 [91:56:35<326:22:20,  6.46s/it]
 19%|█▉        | 42996/225000 [91:56:40<300:39:50,  5.95s/it]
 19%|█▉        | 42997/225000 [91:56:44<276:17:21,  5.46s/it]
 19%|█▉        | 42998/225000 [91:56:58<392:40:36,  7.77s/it]
 19%|█▉        | 42999/225000 [91:57:04<373:31:45,  7.39s/it]
 19%|█▉        | 43000/225000 [91:57:08<325:25:14,  6.44s/it]
                                                             
{'loss': '0.6846', 'grad_norm': '0.8025', 'learning_rate': '4.715e-05', 'epoch': '0.1911', 'num_input_tokens_seen': 155066016, 'train_runtime': '3.31e+05', 'train_tokens_per_second': '468.4'}

 19%|█▉        | 43000/225000 [91:57:08<325:25:14,  6.44s/it]
 19%|█▉        | 43001/225000 [91:57:18<370:58:11,  7.34s/it]
 19%|█▉        | 43002/225000 [91:57:23<334:04:27,  6.61s/it]
 19%|█▉        | 43003/225000 [91:57:30<341:06:59,  6.75s/it]
 19%|█▉        | 43004/225000 [91:57:38<360:09:35,  7.12s/it]
 19%|█▉        | 43005/225000 [91:57:47<398:58:26,  7.89s/it]
 19%|█▉        | 43006/225000 [91:57:52<355:06:54,  7.02s/it]
 19%|█▉        | 43007/225000 [91:57:57<318:54:07,  6.31s/it]
 19%|█▉        | 43008/225000 [91:58:02<295:16:25,  5.84s/it]
 19%|█▉        | 43009/225000 [91:58:13<375:44:58,  7.43s/it]
 19%|█▉        | 43010/225000 [91:58:19<352:16:49,  6.97s/it]
 19%|█▉        | 43011/225000 [91:58:25<334:42:40,  6.62s/it]
 19%|█▉        | 43012/225000 [91:58:35<395:03:23,  7.81s/it]
 19%|█▉        | 43013/225000 [91:58:43<395:46:12,  7.83s/it]
 19%|█▉        | 43014/225000 [91:58:48<354:35:44,  7.01s/it]
 19%|█▉        | 43015/225000 [91:58:56<372:19:04,  7.37s/it]
 19%|█▉        | 43016/225000 [91:59:01<332:28:24,  6.58s/it]
 19%|█▉        | 43017/225000 [91:59:05<298:38:42,  5.91s/it]
 19%|█▉        | 43018/225000 [91:59:14<342:52:03,  6.78s/it]
 19%|█▉        | 43019/225000 [91:59:21<345:20:46,  6.83s/it]
 19%|█▉        | 43020/225000 [91:59:26<314:24:51,  6.22s/it]
 19%|█▉        | 43021/225000 [91:59:31<297:32:34,  5.89s/it]
 19%|█▉        | 43022/225000 [91:59:41<361:30:45,  7.15s/it]
 19%|█▉        | 43023/225000 [91:59:49<374:34:27,  7.41s/it]
 19%|█▉        | 43024/225000 [91:59:58<393:04:37,  7.78s/it]
 19%|█▉        | 43025/225000 [92:00:03<358:29:31,  7.09s/it]
 19%|█▉        | 43026/225000 [92:00:09<341:40:04,  6.76s/it]
 19%|█▉        | 43027/225000 [92:00:14<310:15:16,  6.14s/it]
 19%|█▉        | 43028/225000 [92:00:19<288:30:07,  5.71s/it]
 19%|█▉        | 43029/225000 [92:00:32<400:52:23,  7.93s/it]
 19%|█▉        | 43030/225000 [92:00:39<383:03:11,  7.58s/it]
 19%|█▉        | 43031/225000 [92:00:45<359:10:58,  7.11s/it]
 19%|█▉        | 43032/225000 [92:00:52<362:16:54,  7.17s/it]
 19%|█▉        | 43033/225000 [92:00:56<316:21:30,  6.26s/it]
 19%|█▉        | 43034/225000 [92:01:01<290:00:08,  5.74s/it]
 19%|█▉        | 43035/225000 [92:01:09<328:39:06,  6.50s/it]
 19%|█▉        | 43036/225000 [92:01:18<375:44:08,  7.43s/it]
 19%|█▉        | 43037/225000 [92:01:26<372:20:26,  7.37s/it]
 19%|█▉        | 43038/225000 [92:01:31<334:23:34,  6.62s/it]
 19%|█▉        | 43039/225000 [92:01:39<369:21:02,  7.31s/it]
 19%|█▉        | 43040/225000 [92:01:45<347:47:51,  6.88s/it]
 19%|█▉        | 43041/225000 [92:01:50<318:31:34,  6.30s/it]
 19%|█▉        | 43042/225000 [92:01:57<332:28:25,  6.58s/it]
 19%|█▉        | 43043/225000 [92:02:03<311:36:17,  6.17s/it]
 19%|█▉        | 43044/225000 [92:02:14<386:22:58,  7.64s/it]
 19%|█▉        | 43045/225000 [92:02:18<337:11:00,  6.67s/it]
 19%|█▉        | 43046/225000 [92:02:28<386:47:04,  7.65s/it]
 19%|█▉        | 43047/225000 [92:02:33<345:23:24,  6.83s/it]
 19%|█▉        | 43048/225000 [92:02:42<381:13:55,  7.54s/it]
 19%|█▉        | 43049/225000 [92:02:47<343:10:07,  6.79s/it]
 19%|█▉        | 43050/225000 [92:02:53<320:32:25,  6.34s/it]
                                                             
{'loss': '0.7081', 'grad_norm': '0.7529', 'learning_rate': '4.714e-05', 'epoch': '0.1913', 'num_input_tokens_seen': 155218408, 'train_runtime': '3.314e+05', 'train_tokens_per_second': '468.4'}

 19%|█▉        | 43050/225000 [92:02:53<320:32:25,  6.34s/it]
 19%|█▉        | 43051/225000 [92:02:58<306:05:49,  6.06s/it]
 19%|█▉        | 43052/225000 [92:03:03<294:16:58,  5.82s/it]
 19%|█▉        | 43053/225000 [92:03:12<346:16:55,  6.85s/it]
 19%|█▉        | 43054/225000 [92:03:25<426:11:02,  8.43s/it]
 19%|█▉        | 43055/225000 [92:03:28<349:41:37,  6.92s/it]
 19%|█▉        | 43056/225000 [92:03:34<332:15:34,  6.57s/it]
 19%|█▉        | 43057/225000 [92:03:42<364:12:10,  7.21s/it]
 19%|█▉        | 43058/225000 [92:03:50<370:30:55,  7.33s/it]
 19%|█▉        | 43059/225000 [92:03:58<378:38:12,  7.49s/it]
 19%|█▉        | 43060/225000 [92:04:02<331:39:44,  6.56s/it]
 19%|█▉        | 43061/225000 [92:04:07<307:35:26,  6.09s/it]
 19%|█▉        | 43062/225000 [92:04:18<383:01:15,  7.58s/it]
 19%|█▉        | 43063/225000 [92:04:24<354:32:05,  7.02s/it]
 19%|█▉        | 43064/225000 [92:04:31<355:09:59,  7.03s/it]
 19%|█▉        | 43065/225000 [92:04:35<308:14:03,  6.10s/it]
 19%|█▉        | 43066/225000 [92:04:46<374:50:44,  7.42s/it]
 19%|█▉        | 43067/225000 [92:04:49<321:18:05,  6.36s/it]
 19%|█▉        | 43068/225000 [92:04:59<364:08:05,  7.21s/it]
 19%|█▉        | 43069/225000 [92:05:03<318:04:29,  6.29s/it]
 19%|█▉        | 43070/225000 [92:05:07<290:25:47,  5.75s/it]
 19%|█▉        | 43071/225000 [92:05:18<369:54:17,  7.32s/it]
 19%|█▉        | 43072/225000 [92:05:24<342:25:29,  6.78s/it]
 19%|█▉        | 43073/225000 [92:05:28<306:33:30,  6.07s/it]
 19%|█▉        | 43074/225000 [92:05:36<327:51:30,  6.49s/it]
 19%|█▉        | 43075/225000 [92:05:44<358:10:11,  7.09s/it]
 19%|█▉        | 43076/225000 [92:05:53<383:19:48,  7.59s/it]
 19%|█▉        | 43077/225000 [92:05:57<332:56:59,  6.59s/it]
 19%|█▉        | 43078/225000 [92:06:04<337:18:39,  6.67s/it]
 19%|█▉        | 43079/225000 [92:06:12<356:21:37,  7.05s/it]
 19%|█▉        | 43080/225000 [92:06:17<319:35:32,  6.32s/it]
 19%|█▉        | 43081/225000 [92:06:21<294:37:18,  5.83s/it]
 19%|█▉        | 43082/225000 [92:06:26<280:17:16,  5.55s/it]
 19%|█▉        | 43083/225000 [92:06:39<391:36:30,  7.75s/it]
 19%|█▉        | 43084/225000 [92:06:45<361:07:54,  7.15s/it]
 19%|█▉        | 43085/225000 [92:06:53<380:18:04,  7.53s/it]
 19%|█▉        | 43086/225000 [92:06:57<328:59:57,  6.51s/it]
 19%|█▉        | 43087/225000 [92:07:07<373:46:35,  7.40s/it]
 19%|█▉        | 43088/225000 [92:07:14<371:22:32,  7.35s/it]
 19%|█▉        | 43089/225000 [92:07:18<318:25:05,  6.30s/it]
 19%|█▉        | 43090/225000 [92:07:23<305:35:19,  6.05s/it]
 19%|█▉        | 43091/225000 [92:07:34<377:12:34,  7.47s/it]
 19%|█▉        | 43092/225000 [92:07:41<369:56:15,  7.32s/it]
 19%|█▉        | 43093/225000 [92:07:47<349:51:07,  6.92s/it]
 19%|█▉        | 43094/225000 [92:07:56<375:56:33,  7.44s/it]
 19%|█▉        | 43095/225000 [92:08:00<334:31:22,  6.62s/it]
 19%|█▉        | 43096/225000 [92:08:05<310:45:51,  6.15s/it]
 19%|█▉        | 43097/225000 [92:08:13<335:09:28,  6.63s/it]
 19%|█▉        | 43098/225000 [92:08:21<357:19:13,  7.07s/it]
 19%|█▉        | 43099/225000 [92:08:26<327:24:13,  6.48s/it]
 19%|█▉        | 43100/225000 [92:08:31<300:02:46,  5.94s/it]
                                                             
{'loss': '0.6964', 'grad_norm': '0.6924', 'learning_rate': '4.714e-05', 'epoch': '0.1916', 'num_input_tokens_seen': 155369704, 'train_runtime': '3.317e+05', 'train_tokens_per_second': '468.4'}

 19%|█▉        | 43100/225000 [92:08:31<300:02:46,  5.94s/it]
 19%|█▉        | 43101/225000 [92:08:41<359:27:20,  7.11s/it]
 19%|█▉        | 43102/225000 [92:08:47<347:26:23,  6.88s/it]
 19%|█▉        | 43103/225000 [92:08:54<342:33:37,  6.78s/it]
 19%|█▉        | 43104/225000 [92:09:03<381:26:41,  7.55s/it]
 19%|█▉        | 43105/225000 [92:09:08<339:27:58,  6.72s/it]
 19%|█▉        | 43106/225000 [92:09:12<300:47:33,  5.95s/it]
 19%|█▉        | 43107/225000 [92:09:18<302:00:40,  5.98s/it]
 19%|█▉        | 43108/225000 [92:09:28<361:09:06,  7.15s/it]
 19%|█▉        | 43109/225000 [92:09:32<315:28:52,  6.24s/it]
 19%|█▉        | 43110/225000 [92:09:37<290:37:38,  5.75s/it]
 19%|█▉        | 43111/225000 [92:09:43<295:30:28,  5.85s/it]
 19%|█▉        | 43112/225000 [92:09:51<336:57:47,  6.67s/it]
 19%|█▉        | 43113/225000 [92:10:02<388:50:59,  7.70s/it]
 19%|█▉        | 43114/225000 [92:10:05<330:41:58,  6.55s/it]
 19%|█▉        | 43115/225000 [92:10:17<403:01:05,  7.98s/it]
 19%|█▉        | 43116/225000 [92:10:21<343:32:53,  6.80s/it]
 19%|█▉        | 43117/225000 [92:10:31<392:21:04,  7.77s/it]
 19%|█▉        | 43118/225000 [92:10:35<344:50:00,  6.83s/it]
 19%|█▉        | 43119/225000 [92:10:40<305:26:00,  6.05s/it]
 19%|█▉        | 43120/225000 [92:10:49<361:34:47,  7.16s/it]
 19%|█▉        | 43121/225000 [92:10:55<332:42:42,  6.59s/it]
 19%|█▉        | 43122/225000 [92:11:00<314:38:57,  6.23s/it]
 19%|█▉        | 43123/225000 [92:11:05<295:22:07,  5.85s/it]
 19%|█▉        | 43124/225000 [92:11:13<327:10:36,  6.48s/it]
 19%|█▉        | 43125/225000 [92:11:25<415:15:23,  8.22s/it]
 19%|█▉        | 43126/225000 [92:11:29<351:34:41,  6.96s/it]
 19%|█▉        | 43127/225000 [92:11:40<409:41:09,  8.11s/it]
 19%|█▉        | 43128/225000 [92:11:44<348:13:10,  6.89s/it]
 19%|█▉        | 43129/225000 [92:11:54<393:07:06,  7.78s/it]
 19%|█▉        | 43130/225000 [92:11:58<342:04:20,  6.77s/it]
 19%|█▉        | 43131/225000 [92:12:04<317:40:51,  6.29s/it]
 19%|█▉        | 43132/225000 [92:12:08<290:49:38,  5.76s/it]
 19%|█▉        | 43133/225000 [92:12:21<401:23:15,  7.95s/it]
 19%|█▉        | 43134/225000 [92:12:26<354:29:03,  7.02s/it]
 19%|█▉        | 43135/225000 [92:12:31<329:08:50,  6.52s/it]
 19%|█▉        | 43136/225000 [92:12:42<390:01:02,  7.72s/it]
 19%|█▉        | 43137/225000 [92:12:47<347:49:23,  6.89s/it]
 19%|█▉        | 43138/225000 [92:12:52<315:43:35,  6.25s/it]
 19%|█▉        | 43139/225000 [92:12:56<295:43:59,  5.85s/it]
 19%|█▉        | 43140/225000 [92:13:03<305:49:18,  6.05s/it]
 19%|█▉        | 43141/225000 [92:13:15<403:17:50,  7.98s/it]
 19%|█▉        | 43142/225000 [92:13:22<381:34:57,  7.55s/it]
 19%|█▉        | 43143/225000 [92:13:26<322:38:09,  6.39s/it]
 19%|█▉        | 43144/225000 [92:13:30<288:09:45,  5.70s/it]
 19%|█▉        | 43145/225000 [92:13:41<365:50:05,  7.24s/it]
 19%|█▉        | 43146/225000 [92:13:46<329:52:49,  6.53s/it]
 19%|█▉        | 43147/225000 [92:13:50<302:49:47,  5.99s/it]
 19%|█▉        | 43148/225000 [92:13:57<309:29:17,  6.13s/it]
 19%|█▉        | 43149/225000 [92:14:09<397:15:32,  7.86s/it]
 19%|█▉        | 43150/225000 [92:14:13<339:04:55,  6.71s/it]
                                                             
{'loss': '0.6727', 'grad_norm': '0.7011', 'learning_rate': '4.713e-05', 'epoch': '0.1918', 'num_input_tokens_seen': 155522744, 'train_runtime': '3.321e+05', 'train_tokens_per_second': '468.4'}

 19%|█▉        | 43150/225000 [92:14:13<339:04:55,  6.71s/it]
 19%|█▉        | 43151/225000 [92:14:20<344:26:51,  6.82s/it]
 19%|█▉        | 43152/225000 [92:14:30<400:22:19,  7.93s/it]
 19%|█▉        | 43153/225000 [92:14:35<351:44:44,  6.96s/it]
 19%|█▉        | 43154/225000 [92:14:43<368:01:00,  7.29s/it]
 19%|█▉        | 43155/225000 [92:14:49<346:19:47,  6.86s/it]
 19%|█▉        | 43156/225000 [92:14:53<311:14:47,  6.16s/it]
 19%|█▉        | 43157/225000 [92:14:58<295:20:27,  5.85s/it]
 19%|█▉        | 43158/225000 [92:15:04<286:31:35,  5.67s/it]
 19%|█▉        | 43159/225000 [92:15:12<325:23:24,  6.44s/it]
 19%|█▉        | 43160/225000 [92:15:22<383:09:05,  7.59s/it]
 19%|█▉        | 43161/225000 [92:15:26<332:11:09,  6.58s/it]
 19%|█▉        | 43162/225000 [92:15:31<298:18:07,  5.91s/it]
 19%|█▉        | 43163/225000 [92:15:43<386:32:00,  7.65s/it]
 19%|█▉        | 43164/225000 [92:15:46<325:43:50,  6.45s/it]
 19%|█▉        | 43165/225000 [92:15:52<315:49:10,  6.25s/it]
 19%|█▉        | 43166/225000 [92:15:58<315:01:03,  6.24s/it]
 19%|█▉        | 43167/225000 [92:16:06<338:35:20,  6.70s/it]
 19%|█▉        | 43168/225000 [92:16:11<308:33:46,  6.11s/it]
 19%|█▉        | 43169/225000 [92:16:19<337:26:11,  6.68s/it]
 19%|█▉        | 43170/225000 [92:16:28<371:47:55,  7.36s/it]
 19%|█▉        | 43171/225000 [92:16:32<328:21:55,  6.50s/it]
 19%|█▉        | 43172/225000 [92:16:40<343:38:02,  6.80s/it]
 19%|█▉        | 43173/225000 [92:16:49<384:30:06,  7.61s/it]
 19%|█▉        | 43174/225000 [92:16:55<361:35:45,  7.16s/it]
 19%|█▉        | 43175/225000 [92:17:02<360:07:28,  7.13s/it]
 19%|█▉        | 43176/225000 [92:17:08<334:53:19,  6.63s/it]
 19%|█▉        | 43177/225000 [92:17:14<321:23:54,  6.36s/it]
 19%|█▉        | 43178/225000 [92:17:18<299:55:43,  5.94s/it]
 19%|█▉        | 43179/225000 [92:17:24<292:58:04,  5.80s/it]
 19%|█▉        | 43180/225000 [92:17:29<276:30:22,  5.47s/it]
 19%|█▉        | 43181/225000 [92:17:34<276:41:26,  5.48s/it]
 19%|█▉        | 43182/225000 [92:17:40<281:13:32,  5.57s/it]
 19%|█▉        | 43183/225000 [92:17:50<344:15:54,  6.82s/it]
 19%|█▉        | 43184/225000 [92:18:03<444:41:30,  8.81s/it]
 19%|█▉        | 43185/225000 [92:18:09<403:53:26,  8.00s/it]
 19%|█▉        | 43186/225000 [92:18:15<367:53:37,  7.28s/it]
 19%|█▉        | 43187/225000 [92:18:20<333:51:20,  6.61s/it]
 19%|█▉        | 43188/225000 [92:18:25<315:43:02,  6.25s/it]
 19%|█▉        | 43189/225000 [92:18:31<311:44:04,  6.17s/it]
 19%|█▉        | 43190/225000 [92:18:36<294:16:35,  5.83s/it]
 19%|█▉        | 43191/225000 [92:18:42<292:10:42,  5.79s/it]
 19%|█▉        | 43192/225000 [92:18:47<282:37:02,  5.60s/it]
 19%|█▉        | 43193/225000 [92:19:00<387:59:09,  7.68s/it]
 19%|█▉        | 43194/225000 [92:19:09<406:14:35,  8.04s/it]
 19%|█▉        | 43195/225000 [92:19:13<357:39:42,  7.08s/it]
 19%|█▉        | 43196/225000 [92:19:22<377:47:19,  7.48s/it]
 19%|█▉        | 43197/225000 [92:19:28<351:39:26,  6.96s/it]
 19%|█▉        | 43198/225000 [92:19:38<404:34:42,  8.01s/it]
 19%|█▉        | 43199/225000 [92:19:42<345:04:25,  6.83s/it]
 19%|█▉        | 43200/225000 [92:19:47<320:55:54,  6.36s/it]
                                                             
{'loss': '0.7015', 'grad_norm': '0.811', 'learning_rate': '4.712e-05', 'epoch': '0.192', 'num_input_tokens_seen': 155673232, 'train_runtime': '3.324e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43200/225000 [92:19:47<320:55:54,  6.36s/it]
 19%|█▉        | 43201/225000 [92:19:53<306:56:38,  6.08s/it]
 19%|█▉        | 43202/225000 [92:19:58<296:57:28,  5.88s/it]
 19%|█▉        | 43203/225000 [92:20:04<302:46:09,  6.00s/it]
 19%|█▉        | 43204/225000 [92:20:18<419:40:01,  8.31s/it]
 19%|█▉        | 43205/225000 [92:20:23<371:27:02,  7.36s/it]
 19%|█▉        | 43206/225000 [92:20:32<397:04:29,  7.86s/it]
 19%|█▉        | 43207/225000 [92:20:38<365:56:23,  7.25s/it]
 19%|█▉        | 43208/225000 [92:20:45<357:56:25,  7.09s/it]
 19%|█▉        | 43209/225000 [92:20:53<374:50:57,  7.42s/it]
 19%|█▉        | 43210/225000 [92:20:57<328:34:25,  6.51s/it]
 19%|█▉        | 43211/225000 [92:21:03<316:49:49,  6.27s/it]
 19%|█▉        | 43212/225000 [92:21:08<294:20:28,  5.83s/it]
 19%|█▉        | 43213/225000 [92:21:18<354:20:55,  7.02s/it]
 19%|█▉        | 43214/225000 [92:21:24<342:11:34,  6.78s/it]
 19%|█▉        | 43215/225000 [92:21:33<379:11:48,  7.51s/it]
 19%|█▉        | 43216/225000 [92:21:37<322:37:22,  6.39s/it]
 19%|█▉        | 43217/225000 [92:21:42<295:43:43,  5.86s/it]
 19%|█▉        | 43218/225000 [92:21:49<321:19:18,  6.36s/it]
 19%|█▉        | 43219/225000 [92:21:59<370:37:31,  7.34s/it]
 19%|█▉        | 43220/225000 [92:22:04<340:52:09,  6.75s/it]
 19%|█▉        | 43221/225000 [92:22:09<316:03:45,  6.26s/it]
 19%|█▉        | 43222/225000 [92:22:15<305:30:23,  6.05s/it]
 19%|█▉        | 43223/225000 [92:22:20<291:59:30,  5.78s/it]
 19%|█▉        | 43224/225000 [92:22:25<284:02:48,  5.63s/it]
 19%|█▉        | 43225/225000 [92:22:38<393:21:57,  7.79s/it]
 19%|█▉        | 43226/225000 [92:22:43<356:41:34,  7.06s/it]
 19%|█▉        | 43227/225000 [92:22:48<322:42:08,  6.39s/it]
 19%|█▉        | 43228/225000 [92:22:53<296:30:31,  5.87s/it]
 19%|█▉        | 43229/225000 [92:23:05<390:41:23,  7.74s/it]
 19%|█▉        | 43230/225000 [92:23:10<342:26:10,  6.78s/it]
 19%|█▉        | 43231/225000 [92:23:15<315:37:58,  6.25s/it]
 19%|█▉        | 43232/225000 [92:23:19<283:13:10,  5.61s/it]
 19%|█▉        | 43233/225000 [92:23:29<351:49:20,  6.97s/it]
 19%|█▉        | 43234/225000 [92:23:37<373:33:09,  7.40s/it]
 19%|█▉        | 43235/225000 [92:23:42<340:59:03,  6.75s/it]
 19%|█▉        | 43236/225000 [92:23:47<314:49:46,  6.24s/it]
 19%|█▉        | 43237/225000 [92:23:53<300:52:01,  5.96s/it]
 19%|█▉        | 43238/225000 [92:23:58<296:26:11,  5.87s/it]
 19%|█▉        | 43239/225000 [92:24:04<291:35:16,  5.78s/it]
 19%|█▉        | 43240/225000 [92:24:15<368:40:05,  7.30s/it]
 19%|█▉        | 43241/225000 [92:24:24<402:41:15,  7.98s/it]
 19%|█▉        | 43242/225000 [92:24:32<388:58:48,  7.70s/it]
 19%|█▉        | 43243/225000 [92:24:38<375:03:36,  7.43s/it]
 19%|█▉        | 43244/225000 [92:24:43<334:34:53,  6.63s/it]
 19%|█▉        | 43245/225000 [92:24:51<354:22:39,  7.02s/it]
 19%|█▉        | 43246/225000 [92:24:56<322:09:08,  6.38s/it]
 19%|█▉        | 43247/225000 [92:25:01<303:36:02,  6.01s/it]
 19%|█▉        | 43248/225000 [92:25:06<287:30:52,  5.69s/it]
 19%|█▉        | 43249/225000 [92:25:14<329:12:14,  6.52s/it]
 19%|█▉        | 43250/225000 [92:25:26<400:14:51,  7.93s/it]
                                                             
{'loss': '0.6973', 'grad_norm': '0.7889', 'learning_rate': '4.711e-05', 'epoch': '0.1922', 'num_input_tokens_seen': 155824608, 'train_runtime': '3.327e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43250/225000 [92:25:26<400:14:51,  7.93s/it]
 19%|█▉        | 43251/225000 [92:25:30<343:18:09,  6.80s/it]
 19%|█▉        | 43252/225000 [92:25:38<365:45:51,  7.24s/it]
 19%|█▉        | 43253/225000 [92:25:44<350:25:28,  6.94s/it]
 19%|█▉        | 43254/225000 [92:25:49<319:04:07,  6.32s/it]
 19%|█▉        | 43255/225000 [92:25:56<320:53:27,  6.36s/it]
 19%|█▉        | 43256/225000 [92:26:08<408:29:29,  8.09s/it]
 19%|█▉        | 43257/225000 [92:26:11<341:42:11,  6.77s/it]
 19%|█▉        | 43258/225000 [92:26:17<318:59:07,  6.32s/it]
 19%|█▉        | 43259/225000 [92:26:22<310:21:45,  6.15s/it]
 19%|█▉        | 43260/225000 [92:26:28<302:21:49,  5.99s/it]
 19%|█▉        | 43261/225000 [92:26:33<292:21:54,  5.79s/it]
 19%|█▉        | 43262/225000 [92:26:39<294:05:32,  5.83s/it]
 19%|█▉        | 43263/225000 [92:26:49<353:31:20,  7.00s/it]
 19%|█▉        | 43264/225000 [92:27:00<411:29:39,  8.15s/it]
 19%|█▉        | 43265/225000 [92:27:04<350:04:23,  6.93s/it]
 19%|█▉        | 43266/225000 [92:27:13<385:43:03,  7.64s/it]
 19%|█▉        | 43267/225000 [92:27:18<333:46:07,  6.61s/it]
 19%|█▉        | 43268/225000 [92:27:22<303:06:30,  6.00s/it]
 19%|█▉        | 43269/225000 [92:27:27<281:36:57,  5.58s/it]
 19%|█▉        | 43270/225000 [92:27:40<393:46:46,  7.80s/it]
 19%|█▉        | 43271/225000 [92:27:45<361:23:34,  7.16s/it]
 19%|█▉        | 43272/225000 [92:27:51<333:16:40,  6.60s/it]
 19%|█▉        | 43273/225000 [92:27:56<314:55:43,  6.24s/it]
 19%|█▉        | 43274/225000 [92:28:01<300:36:43,  5.96s/it]
 19%|█▉        | 43275/225000 [92:28:06<285:29:31,  5.66s/it]
 19%|█▉        | 43276/225000 [92:28:12<282:53:11,  5.60s/it]
 19%|█▉        | 43277/225000 [92:28:17<282:34:12,  5.60s/it]
 19%|█▉        | 43278/225000 [92:28:31<406:05:56,  8.05s/it]
 19%|█▉        | 43279/225000 [92:28:37<378:37:15,  7.50s/it]
 19%|█▉        | 43280/225000 [92:28:47<407:09:28,  8.07s/it]
 19%|█▉        | 43281/225000 [92:28:53<373:23:27,  7.40s/it]
 19%|█▉        | 43282/225000 [92:28:58<346:52:32,  6.87s/it]
 19%|█▉        | 43283/225000 [92:29:02<307:44:10,  6.10s/it]
 19%|█▉        | 43284/225000 [92:29:08<292:56:22,  5.80s/it]
 19%|█▉        | 43285/225000 [92:29:18<355:37:29,  7.05s/it]
 19%|█▉        | 43286/225000 [92:29:25<360:30:28,  7.14s/it]
 19%|█▉        | 43287/225000 [92:29:30<330:31:12,  6.55s/it]
 19%|█▉        | 43288/225000 [92:29:38<358:27:48,  7.10s/it]
 19%|█▉        | 43289/225000 [92:29:42<311:05:54,  6.16s/it]
 19%|█▉        | 43290/225000 [92:29:48<294:51:27,  5.84s/it]
 19%|█▉        | 43291/225000 [92:29:52<281:00:22,  5.57s/it]
 19%|█▉        | 43292/225000 [92:30:05<394:05:57,  7.81s/it]
 19%|█▉        | 43293/225000 [92:30:12<380:09:54,  7.53s/it]
 19%|█▉        | 43294/225000 [92:30:20<376:23:19,  7.46s/it]
 19%|█▉        | 43295/225000 [92:30:24<329:04:55,  6.52s/it]
 19%|█▉        | 43296/225000 [92:30:29<301:23:05,  5.97s/it]
 19%|█▉        | 43297/225000 [92:30:35<301:42:10,  5.98s/it]
 19%|█▉        | 43298/225000 [92:30:48<409:15:02,  8.11s/it]
 19%|█▉        | 43299/225000 [92:30:51<341:52:20,  6.77s/it]
 19%|█▉        | 43300/225000 [92:31:01<389:50:56,  7.72s/it]
                                                             
{'loss': '0.7079', 'grad_norm': '0.7423', 'learning_rate': '4.71e-05', 'epoch': '0.1924', 'num_input_tokens_seen': 155975128, 'train_runtime': '3.331e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43300/225000 [92:31:01<389:50:56,  7.72s/it]
 19%|█▉        | 43301/225000 [92:31:06<337:00:14,  6.68s/it]
 19%|█▉        | 43302/225000 [92:31:15<376:48:35,  7.47s/it]
 19%|█▉        | 43303/225000 [92:31:20<342:21:02,  6.78s/it]
 19%|█▉        | 43304/225000 [92:31:25<312:47:41,  6.20s/it]
 19%|█▉        | 43305/225000 [92:31:30<290:05:47,  5.75s/it]
 19%|█▉        | 43306/225000 [92:31:42<389:21:41,  7.71s/it]
 19%|█▉        | 43307/225000 [92:31:47<356:50:29,  7.07s/it]
 19%|█▉        | 43308/225000 [92:31:52<320:49:08,  6.36s/it]
 19%|█▉        | 43309/225000 [92:31:57<292:34:49,  5.80s/it]
 19%|█▉        | 43310/225000 [92:32:02<280:23:32,  5.56s/it]
 19%|█▉        | 43311/225000 [92:32:12<359:40:57,  7.13s/it]
 19%|█▉        | 43312/225000 [92:32:21<382:02:53,  7.57s/it]
 19%|█▉        | 43313/225000 [92:32:29<386:36:39,  7.66s/it]
 19%|█▉        | 43314/225000 [92:32:33<327:57:59,  6.50s/it]
 19%|█▉        | 43315/225000 [92:32:38<302:11:22,  5.99s/it]
 19%|█▉        | 43316/225000 [92:32:43<292:59:03,  5.81s/it]
 19%|█▉        | 43317/225000 [92:32:49<291:58:06,  5.79s/it]
 19%|█▉        | 43318/225000 [92:32:54<293:11:39,  5.81s/it]
 19%|█▉        | 43319/225000 [92:33:00<291:56:10,  5.78s/it]
 19%|█▉        | 43320/225000 [92:33:06<292:45:47,  5.80s/it]
 19%|█▉        | 43321/225000 [92:33:19<398:05:05,  7.89s/it]
 19%|█▉        | 43322/225000 [92:33:27<405:25:24,  8.03s/it]
 19%|█▉        | 43323/225000 [92:33:32<356:34:32,  7.07s/it]
 19%|█▉        | 43324/225000 [92:33:38<336:03:32,  6.66s/it]
 19%|█▉        | 43325/225000 [92:33:43<320:11:49,  6.34s/it]
 19%|█▉        | 43326/225000 [92:33:49<306:16:29,  6.07s/it]
 19%|█▉        | 43327/225000 [92:33:54<297:50:55,  5.90s/it]
 19%|█▉        | 43328/225000 [92:34:00<293:23:57,  5.81s/it]
 19%|█▉        | 43329/225000 [92:34:13<403:23:27,  7.99s/it]
 19%|█▉        | 43330/225000 [92:34:19<375:00:34,  7.43s/it]
 19%|█▉        | 43331/225000 [92:34:26<366:16:38,  7.26s/it]
 19%|█▉        | 43332/225000 [92:34:31<328:07:49,  6.50s/it]
 19%|█▉        | 43333/225000 [92:34:36<304:40:56,  6.04s/it]
 19%|█▉        | 43334/225000 [92:34:42<303:19:31,  6.01s/it]
 19%|█▉        | 43335/225000 [92:34:47<295:41:05,  5.86s/it]
 19%|█▉        | 43336/225000 [92:34:59<388:58:42,  7.71s/it]
 19%|█▉        | 43337/225000 [92:35:06<378:50:24,  7.51s/it]
 19%|█▉        | 43338/225000 [92:35:12<353:50:16,  7.01s/it]
 19%|█▉        | 43339/225000 [92:35:18<331:42:26,  6.57s/it]
 19%|█▉        | 43340/225000 [92:35:23<316:33:47,  6.27s/it]
 19%|█▉        | 43341/225000 [92:35:29<307:36:03,  6.10s/it]
 19%|█▉        | 43342/225000 [92:35:40<380:15:28,  7.54s/it]
 19%|█▉        | 43343/225000 [92:35:50<414:47:31,  8.22s/it]
 19%|█▉        | 43344/225000 [92:35:54<354:30:47,  7.03s/it]
 19%|█▉        | 43345/225000 [92:36:04<399:27:55,  7.92s/it]
 19%|█▉        | 43346/225000 [92:36:07<332:52:35,  6.60s/it]
 19%|█▉        | 43347/225000 [92:36:15<345:15:12,  6.84s/it]
 19%|█▉        | 43348/225000 [92:36:20<315:28:58,  6.25s/it]
 19%|█▉        | 43349/225000 [92:36:24<293:56:40,  5.83s/it]
 19%|█▉        | 43350/225000 [92:36:31<307:30:12,  6.09s/it]
                                                             
{'loss': '0.6782', 'grad_norm': '0.7136', 'learning_rate': '4.709e-05', 'epoch': '0.1927', 'num_input_tokens_seen': 156127216, 'train_runtime': '3.334e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43350/225000 [92:36:31<307:30:12,  6.09s/it]
 19%|█▉        | 43351/225000 [92:36:42<380:40:33,  7.54s/it]
 19%|█▉        | 43352/225000 [92:36:48<361:27:36,  7.16s/it]
 19%|█▉        | 43353/225000 [92:36:55<350:14:33,  6.94s/it]
 19%|█▉        | 43354/225000 [92:37:03<376:42:50,  7.47s/it]
 19%|█▉        | 43355/225000 [92:37:07<322:01:17,  6.38s/it]
 19%|█▉        | 43356/225000 [92:37:17<369:59:17,  7.33s/it]
 19%|█▉        | 43357/225000 [92:37:22<331:54:53,  6.58s/it]
 19%|█▉        | 43358/225000 [92:37:27<311:07:13,  6.17s/it]
 19%|█▉        | 43359/225000 [92:37:32<295:52:41,  5.86s/it]
 19%|█▉        | 43360/225000 [92:37:37<280:45:15,  5.56s/it]
 19%|█▉        | 43361/225000 [92:37:42<277:26:19,  5.50s/it]
 19%|█▉        | 43362/225000 [92:37:51<326:07:54,  6.46s/it]
 19%|█▉        | 43363/225000 [92:38:05<437:19:11,  8.67s/it]
 19%|█▉        | 43364/225000 [92:38:09<376:11:31,  7.46s/it]
 19%|█▉        | 43365/225000 [92:38:14<331:04:06,  6.56s/it]
 19%|█▉        | 43366/225000 [92:38:20<317:34:37,  6.29s/it]
 19%|█▉        | 43367/225000 [92:38:30<388:02:29,  7.69s/it]
 19%|█▉        | 43368/225000 [92:38:35<335:51:30,  6.66s/it]
 19%|█▉        | 43369/225000 [92:38:44<379:53:37,  7.53s/it]
 19%|█▉        | 43370/225000 [92:38:49<331:52:27,  6.58s/it]
 19%|█▉        | 43371/225000 [92:38:53<303:21:10,  6.01s/it]
 19%|█▉        | 43372/225000 [92:39:01<329:21:20,  6.53s/it]
 19%|█▉        | 43373/225000 [92:39:11<376:34:20,  7.46s/it]
 19%|█▉        | 43374/225000 [92:39:16<344:32:35,  6.83s/it]
 19%|█▉        | 43375/225000 [92:39:22<328:23:41,  6.51s/it]
 19%|█▉        | 43376/225000 [92:39:27<307:00:43,  6.09s/it]
 19%|█▉        | 43377/225000 [92:39:32<287:17:35,  5.69s/it]
 19%|█▉        | 43378/225000 [92:39:37<286:50:05,  5.69s/it]
 19%|█▉        | 43379/225000 [92:39:43<293:43:59,  5.82s/it]
 19%|█▉        | 43380/225000 [92:39:57<409:03:43,  8.11s/it]
 19%|█▉        | 43381/225000 [92:40:01<353:37:59,  7.01s/it]
 19%|█▉        | 43382/225000 [92:40:09<356:11:47,  7.06s/it]
 19%|█▉        | 43383/225000 [92:40:14<328:11:42,  6.51s/it]
 19%|█▉        | 43384/225000 [92:40:25<407:13:33,  8.07s/it]
 19%|█▉        | 43385/225000 [92:40:30<350:15:35,  6.94s/it]
 19%|█▉        | 43386/225000 [92:40:34<307:15:25,  6.09s/it]
 19%|█▉        | 43387/225000 [92:40:47<415:36:07,  8.24s/it]
 19%|█▉        | 43388/225000 [92:40:52<367:24:11,  7.28s/it]
 19%|█▉        | 43389/225000 [92:40:57<327:16:10,  6.49s/it]
 19%|█▉        | 43390/225000 [92:41:02<303:09:13,  6.01s/it]
 19%|█▉        | 43391/225000 [92:41:07<289:37:21,  5.74s/it]
 19%|█▉        | 43392/225000 [92:41:12<280:21:02,  5.56s/it]
 19%|█▉        | 43393/225000 [92:41:17<274:40:15,  5.44s/it]
 19%|█▉        | 43394/225000 [92:41:23<283:50:01,  5.63s/it]
 19%|█▉        | 43395/225000 [92:41:36<385:55:08,  7.65s/it]
 19%|█▉        | 43396/225000 [92:41:43<389:25:18,  7.72s/it]
 19%|█▉        | 43397/225000 [92:41:50<368:14:55,  7.30s/it]
 19%|█▉        | 43398/225000 [92:41:56<353:36:26,  7.01s/it]
 19%|█▉        | 43399/225000 [92:42:00<308:50:13,  6.12s/it]
 19%|█▉        | 43400/225000 [92:42:11<380:45:21,  7.55s/it]
                                                             
{'loss': '0.7146', 'grad_norm': '0.6692', 'learning_rate': '4.709e-05', 'epoch': '0.1929', 'num_input_tokens_seen': 156277792, 'train_runtime': '3.337e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43400/225000 [92:42:11<380:45:21,  7.55s/it]
 19%|█▉        | 43401/225000 [92:42:16<335:49:03,  6.66s/it]
 19%|█▉        | 43402/225000 [92:42:25<370:09:32,  7.34s/it]
 19%|█▉        | 43403/225000 [92:42:30<342:11:00,  6.78s/it]
 19%|█▉        | 43404/225000 [92:42:36<332:25:23,  6.59s/it]
 19%|█▉        | 43405/225000 [92:42:41<311:59:56,  6.19s/it]
 19%|█▉        | 43406/225000 [92:42:47<299:16:45,  5.93s/it]
 19%|█▉        | 43407/225000 [92:42:53<299:18:42,  5.93s/it]
 19%|█▉        | 43408/225000 [92:42:58<294:31:56,  5.84s/it]
 19%|█▉        | 43409/225000 [92:43:04<298:36:26,  5.92s/it]
 19%|█▉        | 43410/225000 [92:43:18<409:13:15,  8.11s/it]
 19%|█▉        | 43411/225000 [92:43:25<395:56:21,  7.85s/it]
 19%|█▉        | 43412/225000 [92:43:29<333:59:30,  6.62s/it]
 19%|█▉        | 43413/225000 [92:43:38<373:28:27,  7.40s/it]
 19%|█▉        | 43414/225000 [92:43:43<335:35:43,  6.65s/it]
 19%|█▉        | 43415/225000 [92:43:50<351:18:39,  6.96s/it]
 19%|█▉        | 43416/225000 [92:43:57<351:58:54,  6.98s/it]
 19%|█▉        | 43417/225000 [92:44:03<337:25:26,  6.69s/it]
 19%|█▉        | 43418/225000 [92:44:10<339:11:23,  6.72s/it]
 19%|█▉        | 43419/225000 [92:44:17<338:26:20,  6.71s/it]
 19%|█▉        | 43420/225000 [92:44:27<391:13:54,  7.76s/it]
 19%|█▉        | 43421/225000 [92:44:32<342:48:57,  6.80s/it]
 19%|█▉        | 43422/225000 [92:44:37<314:23:04,  6.23s/it]
 19%|█▉        | 43423/225000 [92:44:42<294:26:49,  5.84s/it]
 19%|█▉        | 43424/225000 [92:44:53<377:06:11,  7.48s/it]
 19%|█▉        | 43425/225000 [92:45:00<376:54:15,  7.47s/it]
 19%|█▉        | 43426/225000 [92:45:04<322:10:50,  6.39s/it]
 19%|█▉        | 43427/225000 [92:45:10<306:33:54,  6.08s/it]
 19%|█▉        | 43428/225000 [92:45:19<355:46:21,  7.05s/it]
 19%|█▉        | 43429/225000 [92:45:24<321:46:18,  6.38s/it]
 19%|█▉        | 43430/225000 [92:45:29<306:45:06,  6.08s/it]
 19%|█▉        | 43431/225000 [92:45:35<297:26:46,  5.90s/it]
 19%|█▉        | 43432/225000 [92:45:48<407:32:12,  8.08s/it]
 19%|█▉        | 43433/225000 [92:45:51<339:55:13,  6.74s/it]
 19%|█▉        | 43434/225000 [92:46:00<371:33:24,  7.37s/it]
 19%|█▉        | 43435/225000 [92:46:05<331:52:30,  6.58s/it]
 19%|█▉        | 43436/225000 [92:46:12<340:07:58,  6.74s/it]
 19%|█▉        | 43437/225000 [92:46:22<382:38:37,  7.59s/it]
 19%|█▉        | 43438/225000 [92:46:27<352:01:56,  6.98s/it]
 19%|█▉        | 43439/225000 [92:46:32<326:53:06,  6.48s/it]
 19%|█▉        | 43440/225000 [92:46:37<303:21:49,  6.02s/it]
 19%|█▉        | 43441/225000 [92:46:43<292:00:52,  5.79s/it]
 19%|█▉        | 43442/225000 [92:46:53<359:40:29,  7.13s/it]
 19%|█▉        | 43443/225000 [92:47:03<408:54:17,  8.11s/it]
 19%|█▉        | 43444/225000 [92:47:11<404:58:14,  8.03s/it]
 19%|█▉        | 43445/225000 [92:47:15<345:44:03,  6.86s/it]
 19%|█▉        | 43446/225000 [92:47:24<368:49:48,  7.31s/it]
 19%|█▉        | 43447/225000 [92:47:28<329:57:23,  6.54s/it]
 19%|█▉        | 43448/225000 [92:47:33<294:56:05,  5.85s/it]
 19%|█▉        | 43449/225000 [92:47:38<291:27:43,  5.78s/it]
 19%|█▉        | 43450/225000 [92:47:44<290:38:35,  5.76s/it]
                                                             
{'loss': '0.7002', 'grad_norm': '0.6548', 'learning_rate': '4.708e-05', 'epoch': '0.1931', 'num_input_tokens_seen': 156429408, 'train_runtime': '3.341e+05', 'train_tokens_per_second': '468.3'}

 19%|█▉        | 43450/225000 [92:47:44<290:38:35,  5.76s/it]
 19%|█▉        | 43451/225000 [92:47:50<288:26:11,  5.72s/it]
 19%|█▉        | 43452/225000 [92:47:57<318:00:19,  6.31s/it]
 19%|█▉        | 43453/225000 [92:48:09<402:22:30,  7.98s/it]
 19%|█▉        | 43454/225000 [92:48:14<352:16:47,  6.99s/it]
 19%|█▉        | 43455/225000 [92:48:19<322:34:04,  6.40s/it]
 19%|█▉        | 43456/225000 [92:48:27<348:47:20,  6.92s/it]
 19%|█▉        | 43457/225000 [92:48:38<417:52:41,  8.29s/it]
 19%|█▉        | 43458/225000 [92:48:43<368:11:06,  7.30s/it]
 19%|█▉        | 43459/225000 [92:48:49<345:23:11,  6.85s/it]
 19%|█▉        | 43460/225000 [92:48:55<329:30:47,  6.53s/it]
 19%|█▉        | 43461/225000 [92:49:00<310:01:08,  6.15s/it]
 19%|█▉        | 43462/225000 [92:49:06<300:10:51,  5.95s/it]
 19%|█▉        | 43463/225000 [92:49:11<293:41:46,  5.82s/it]
 19%|█▉        | 43464/225000 [92:49:23<387:48:07,  7.69s/it]
 19%|█▉        | 43465/225000 [92:49:30<368:42:26,  7.31s/it]
 19%|█▉        | 43466/225000 [92:49:34<324:10:06,  6.43s/it]
 19%|█▉        | 43467/225000 [92:49:47<420:08:06,  8.33s/it]
 19%|█▉        | 43468/225000 [92:49:51<351:04:37,  6.96s/it]
 19%|█▉        | 43469/225000 [92:49:57<337:41:35,  6.70s/it]
 19%|█▉        | 43470/225000 [92:50:05<360:58:58,  7.16s/it]
 19%|█▉        | 43471/225000 [92:50:10<324:50:29,  6.44s/it]
 19%|█▉        | 43472/225000 [92:50:14<289:00:36,  5.73s/it]
 19%|█▉        | 43473/225000 [92:50:25<364:42:41,  7.23s/it]
 19%|█▉        | 43474/225000 [92:50:30<337:27:50,  6.69s/it]
 19%|█▉        | 43475/225000 [92:50:39<371:24:40,  7.37s/it]
 19%|█▉        | 43476/225000 [92:50:44<331:50:27,  6.58s/it]
 19%|█▉        | 43477/225000 [92:50:49<305:04:55,  6.05s/it]
 19%|█▉        | 43478/225000 [92:50:55<305:01:30,  6.05s/it]
 19%|█▉        | 43479/225000 [92:51:08<420:57:02,  8.35s/it]
 19%|█▉        | 43480/225000 [92:51:13<360:40:21,  7.15s/it]
 19%|█▉        | 43481/225000 [92:51:17<319:19:20,  6.33s/it]
 19%|█▉        | 43482/225000 [92:51:28<381:57:20,  7.58s/it]
 19%|█▉        | 43483/225000 [92:51:32<336:12:09,  6.67s/it]
 19%|█▉        | 43484/225000 [92:51:42<385:12:01,  7.64s/it]
 19%|█▉        | 43485/225000 [92:51:46<325:01:35,  6.45s/it]
 19%|█▉        | 43486/225000 [92:51:54<354:14:04,  7.03s/it]
 19%|█▉        | 43487/225000 [92:52:02<362:46:40,  7.20s/it]
 19%|█▉        | 43488/225000 [92:52:05<309:29:08,  6.14s/it]
 19%|█▉        | 43489/225000 [92:52:10<293:54:32,  5.83s/it]
 19%|█▉        | 43490/225000 [92:52:20<345:44:57,  6.86s/it]
 19%|█▉        | 43491/225000 [92:52:24<302:10:03,  5.99s/it]
 19%|█▉        | 43492/225000 [92:52:34<375:36:25,  7.45s/it]
 19%|█▉        | 43493/225000 [92:52:39<324:35:18,  6.44s/it]
 19%|█▉        | 43494/225000 [92:52:48<367:31:06,  7.29s/it]
 19%|█▉        | 43495/225000 [92:52:53<334:14:43,  6.63s/it]
 19%|█▉        | 43496/225000 [92:52:57<302:12:12,  5.99s/it]
 19%|█▉        | 43497/225000 [92:53:08<372:27:41,  7.39s/it]
 19%|█▉        | 43498/225000 [92:53:13<337:52:32,  6.70s/it]
 19%|█▉        | 43499/225000 [92:53:18<312:11:29,  6.19s/it]
 19%|█▉        | 43500/225000 [92:53:29<386:08:43,  7.66s/it]
                                                             
{'loss': '0.6936', 'grad_norm': '1.13', 'learning_rate': '4.707e-05', 'epoch': '0.1933', 'num_input_tokens_seen': 156580544, 'train_runtime': '3.344e+05', 'train_tokens_per_second': '468.2'}

 19%|█▉        | 43500/225000 [92:53:29<386:08:43,  7.66s/it]
 19%|█▉        | 43501/225000 [92:53:35<354:21:43,  7.03s/it]
 19%|█▉        | 43502/225000 [92:53:41<348:06:32,  6.90s/it]
 19%|█▉        | 43503/225000 [92:53:48<346:10:34,  6.87s/it]
 19%|█▉        | 43504/225000 [92:53:53<318:18:05,  6.31s/it]
 19%|█▉        | 43505/225000 [92:53:58<293:07:09,  5.81s/it]
 19%|█▉        | 43506/225000 [92:54:02<273:18:59,  5.42s/it]
 19%|█▉        | 43507/225000 [92:54:14<360:16:42,  7.15s/it]
 19%|█▉        | 43508/225000 [92:54:21<364:24:38,  7.23s/it]
 19%|█▉        | 43509/225000 [92:54:25<314:04:13,  6.23s/it]
 19%|█▉        | 43510/225000 [92:54:30<295:12:13,  5.86s/it]
 19%|█▉        | 43511/225000 [92:54:36<294:57:56,  5.85s/it]
 19%|█▉        | 43512/225000 [92:54:41<292:19:28,  5.80s/it]
 19%|█▉        | 43513/225000 [92:54:52<372:33:12,  7.39s/it]
 19%|█▉        | 43514/225000 [92:55:02<408:18:31,  8.10s/it]
 19%|█▉        | 43515/225000 [92:55:08<379:38:55,  7.53s/it]
 19%|█▉        | 43516/225000 [92:55:13<329:45:13,  6.54s/it]
 19%|█▉        | 43517/225000 [92:55:22<378:51:49,  7.52s/it]
 19%|█▉        | 43518/225000 [92:55:30<379:33:32,  7.53s/it]
 19%|█▉        | 43519/225000 [92:55:38<380:48:12,  7.55s/it]
 19%|█▉        | 43520/225000 [92:55:42<339:29:03,  6.73s/it]
 19%|█▉        | 43521/225000 [92:55:51<365:45:43,  7.26s/it]
 19%|█▉        | 43522/225000 [92:55:55<317:50:57,  6.31s/it]
 19%|█▉        | 43523/225000 [92:56:05<378:54:56,  7.52s/it]
 19%|█▉        | 43524/225000 [92:56:12<365:30:38,  7.25s/it]
 19%|█▉        | 43525/225000 [92:56:19<369:04:26,  7.32s/it]
 19%|█▉        | 43526/225000 [92:56:26<355:22:35,  7.05s/it]
 19%|█▉        | 43527/225000 [92:56:33<355:09:02,  7.05s/it]
 19%|█▉        | 43528/225000 [92:56:41<369:23:14,  7.33s/it]
 19%|█▉        | 43529/225000 [92:56:45<320:17:29,  6.35s/it]
 19%|█▉        | 43530/225000 [92:56:55<379:13:52,  7.52s/it]
 19%|█▉        | 43531/225000 [92:57:01<353:00:13,  7.00s/it]
 19%|█▉        | 43532/225000 [92:57:05<304:31:51,  6.04s/it]
 19%|█▉        | 43533/225000 [92:57:15<361:42:03,  7.18s/it]
 19%|█▉        | 43534/225000 [92:57:18<308:07:42,  6.11s/it]
 19%|█▉        | 43535/225000 [92:57:24<297:28:16,  5.90s/it]
 19%|█▉        | 43536/225000 [92:57:29<293:16:12,  5.82s/it]
 19%|█▉        | 43537/225000 [92:57:35<292:18:09,  5.80s/it]
 19%|█▉        | 43538/225000 [92:57:41<299:38:11,  5.94s/it]
 19%|█▉        | 43539/225000 [92:57:55<409:11:18,  8.12s/it]
 19%|█▉        | 43540/225000 [92:58:00<361:54:30,  7.18s/it]
 19%|█▉        | 43541/225000 [92:58:05<333:03:25,  6.61s/it]
 19%|█▉        | 43542/225000 [92:58:10<307:45:23,  6.11s/it]
 19%|█▉        | 43543/225000 [92:58:16<304:51:09,  6.05s/it]
 19%|█▉        | 43544/225000 [92:58:29<412:31:04,  8.18s/it]
 19%|█▉        | 43545/225000 [92:58:34<360:26:03,  7.15s/it]
 19%|█▉        | 43546/225000 [92:58:38<315:04:05,  6.25s/it]
 19%|█▉        | 43547/225000 [92:58:46<346:04:41,  6.87s/it]
 19%|█▉        | 43548/225000 [92:58:52<326:45:35,  6.48s/it]
 19%|█▉        | 43549/225000 [92:58:57<315:47:28,  6.27s/it]
 19%|█▉        | 43550/225000 [92:59:09<392:47:12,  7.79s/it]
                                                             
{'loss': '0.7088', 'grad_norm': '0.6064', 'learning_rate': '4.706e-05', 'epoch': '0.1936', 'num_input_tokens_seen': 156731968, 'train_runtime': '3.347e+05', 'train_tokens_per_second': '468.2'}

 19%|█▉        | 43550/225000 [92:59:09<392:47:12,  7.79s/it]
 19%|█▉        | 43551/225000 [92:59:14<347:25:37,  6.89s/it]
 19%|█▉        | 43552/225000 [92:59:18<310:51:02,  6.17s/it]
 19%|█▉        | 43553/225000 [92:59:25<319:47:25,  6.34s/it]
 19%|█▉        | 43554/225000 [92:59:35<383:37:47,  7.61s/it]
 19%|█▉        | 43555/225000 [92:59:43<391:35:17,  7.77s/it]
 19%|█▉        | 43556/225000 [92:59:49<350:41:06,  6.96s/it]
 19%|█▉        | 43557/225000 [92:59:54<326:12:54,  6.47s/it]
 19%|█▉        | 43558/225000 [92:59:59<304:17:06,  6.04s/it]
 19%|█▉        | 43559/225000 [93:00:11<400:29:44,  7.95s/it]
 19%|█▉        | 43560/225000 [93:00:15<337:53:34,  6.70s/it]
 19%|█▉        | 43561/225000 [93:00:20<313:34:27,  6.22s/it]
 19%|█▉        | 43562/225000 [93:00:26<309:30:59,  6.14s/it]
 19%|█▉        | 43563/225000 [93:00:32<299:06:37,  5.93s/it]
 19%|█▉        | 43564/225000 [93:00:37<287:19:40,  5.70s/it]
 19%|█▉        | 43565/225000 [93:00:49<388:09:05,  7.70s/it]
 19%|█▉        | 43566/225000 [93:00:55<358:01:43,  7.10s/it]
 19%|█▉        | 43567/225000 [93:00:59<318:20:24,  6.32s/it]
 19%|█▉        | 43568/225000 [93:01:04<295:04:02,  5.85s/it]
 19%|█▉        | 43569/225000 [93:01:13<342:56:16,  6.80s/it]
 19%|█▉        | 43570/225000 [93:01:23<394:49:09,  7.83s/it]
 19%|█▉        | 43571/225000 [93:01:28<343:06:26,  6.81s/it]
 19%|█▉        | 43572/225000 [93:01:33<322:40:36,  6.40s/it]
 19%|█▉        | 43573/225000 [93:01:39<316:23:44,  6.28s/it]
 19%|█▉        | 43574/225000 [93:01:45<308:28:12,  6.12s/it]
 19%|█▉        | 43575/225000 [93:01:50<297:05:05,  5.90s/it]
 19%|█▉        | 43576/225000 [93:01:56<296:01:42,  5.87s/it]
 19%|█▉        | 43577/225000 [93:02:02<295:07:39,  5.86s/it]
 19%|█▉        | 43578/225000 [93:02:15<411:01:05,  8.16s/it]
 19%|█▉        | 43579/225000 [93:02:21<373:06:15,  7.40s/it]
 19%|█▉        | 43580/225000 [93:02:26<334:24:35,  6.64s/it]
 19%|█▉        | 43581/225000 [93:02:31<313:10:16,  6.21s/it]
 19%|█▉        | 43582/225000 [93:02:45<420:26:10,  8.34s/it]
 19%|█▉        | 43583/225000 [93:02:50<384:14:34,  7.62s/it]
 19%|█▉        | 43584/225000 [93:02:56<359:36:51,  7.14s/it]
 19%|█▉        | 43585/225000 [93:03:01<324:23:40,  6.44s/it]
 19%|█▉        | 43586/225000 [93:03:10<353:08:51,  7.01s/it]
 19%|█▉        | 43587/225000 [93:03:17<363:00:37,  7.20s/it]
 19%|█▉        | 43588/225000 [93:03:26<379:42:01,  7.53s/it]
 19%|█▉        | 43589/225000 [93:03:30<336:56:39,  6.69s/it]
 19%|█▉        | 43590/225000 [93:03:36<319:52:26,  6.35s/it]
 19%|█▉        | 43591/225000 [93:03:41<297:35:28,  5.91s/it]
 19%|█▉        | 43592/225000 [93:03:49<328:44:14,  6.52s/it]
 19%|█▉        | 43593/225000 [93:04:02<431:12:28,  8.56s/it]
 19%|█▉        | 43594/225000 [93:04:05<351:19:43,  6.97s/it]
 19%|█▉        | 43595/225000 [93:04:10<312:43:39,  6.21s/it]
 19%|█▉        | 43596/225000 [93:04:15<293:58:23,  5.83s/it]
 19%|█▉        | 43597/225000 [93:04:28<404:21:03,  8.02s/it]
 19%|█▉        | 43598/225000 [93:04:32<353:10:54,  7.01s/it]
 19%|█▉        | 43599/225000 [93:04:39<347:08:14,  6.89s/it]
 19%|█▉        | 43600/225000 [93:04:46<348:27:57,  6.92s/it]
                                                             
{'loss': '0.6948', 'grad_norm': '0.8158', 'learning_rate': '4.705e-05', 'epoch': '0.1938', 'num_input_tokens_seen': 156883328, 'train_runtime': '3.351e+05', 'train_tokens_per_second': '468.2'}

 19%|█▉        | 43600/225000 [93:04:46<348:27:57,  6.92s/it]
 19%|█▉        | 43601/225000 [93:04:50<309:44:47,  6.15s/it]
 19%|█▉        | 43602/225000 [93:05:02<388:21:07,  7.71s/it]
 19%|█▉        | 43603/225000 [93:05:05<329:11:18,  6.53s/it]
 19%|█▉        | 43604/225000 [93:05:15<380:11:49,  7.55s/it]
 19%|█▉        | 43605/225000 [93:05:19<317:51:54,  6.31s/it]
 19%|█▉        | 43606/225000 [93:05:28<358:45:54,  7.12s/it]
 19%|█▉        | 43607/225000 [93:05:32<308:28:48,  6.12s/it]
 19%|█▉        | 43608/225000 [93:05:36<285:49:53,  5.67s/it]
 19%|█▉        | 43609/225000 [93:05:42<283:43:01,  5.63s/it]
 19%|█▉        | 43610/225000 [93:05:54<379:50:49,  7.54s/it]
 19%|█▉        | 43611/225000 [93:06:01<372:43:59,  7.40s/it]
 19%|█▉        | 43612/225000 [93:06:10<406:17:25,  8.06s/it]
 19%|█▉        | 43613/225000 [93:06:15<347:21:14,  6.89s/it]
 19%|█▉        | 43614/225000 [93:06:22<356:10:05,  7.07s/it]
 19%|█▉        | 43615/225000 [93:06:29<357:13:20,  7.09s/it]
 19%|█▉        | 43616/225000 [93:06:34<315:30:33,  6.26s/it]
 19%|█▉        | 43617/225000 [93:06:38<283:55:25,  5.64s/it]
 19%|█▉        | 43618/225000 [93:06:48<359:59:22,  7.14s/it]
 19%|█▉        | 43619/225000 [93:06:54<330:00:44,  6.55s/it]
 19%|█▉        | 43620/225000 [93:07:04<385:00:10,  7.64s/it]
 19%|█▉        | 43621/225000 [93:07:08<334:16:36,  6.63s/it]
 19%|█▉        | 43622/225000 [93:07:13<306:07:37,  6.08s/it]
 19%|█▉        | 43623/225000 [93:07:25<391:15:27,  7.77s/it]
 19%|█▉        | 43624/225000 [93:07:28<326:07:18,  6.47s/it]
 19%|█▉        | 43625/225000 [93:07:36<355:49:44,  7.06s/it]
 19%|█▉        | 43626/225000 [93:07:42<332:51:15,  6.61s/it]
 19%|█▉        | 43627/225000 [93:07:49<342:29:15,  6.80s/it]
 19%|█▉        | 43628/225000 [93:07:56<335:03:56,  6.65s/it]
 19%|█▉        | 43629/225000 [93:08:00<309:46:22,  6.15s/it]
 19%|█▉        | 43630/225000 [93:08:06<297:48:00,  5.91s/it]
 19%|█▉        | 43631/225000 [93:08:12<296:06:44,  5.88s/it]
 19%|█▉        | 43632/225000 [93:08:18<302:38:13,  6.01s/it]
 19%|█▉        | 43633/225000 [93:08:31<415:06:50,  8.24s/it]
 19%|█▉        | 43634/225000 [93:08:36<365:27:01,  7.25s/it]
 19%|█▉        | 43635/225000 [93:08:41<326:38:54,  6.48s/it]
 19%|█▉        | 43636/225000 [93:08:46<298:21:26,  5.92s/it]
 19%|█▉        | 43637/225000 [93:08:54<334:22:12,  6.64s/it]
 19%|█▉        | 43638/225000 [93:09:06<410:08:23,  8.14s/it]
 19%|█▉        | 43639/225000 [93:09:11<373:02:47,  7.40s/it]
 19%|█▉        | 43640/225000 [93:09:17<345:59:30,  6.87s/it]
 19%|█▉        | 43641/225000 [93:09:22<321:22:21,  6.38s/it]
 19%|█▉        | 43642/225000 [93:09:27<303:47:11,  6.03s/it]
 19%|█▉        | 43643/225000 [93:09:32<289:34:23,  5.75s/it]
 19%|█▉        | 43644/225000 [93:09:38<288:07:42,  5.72s/it]
 19%|█▉        | 43645/225000 [93:09:44<297:23:02,  5.90s/it]
 19%|█▉        | 43646/225000 [93:09:58<407:15:30,  8.08s/it]
 19%|█▉        | 43647/225000 [93:10:04<376:05:09,  7.47s/it]
 19%|█▉        | 43648/225000 [93:10:11<370:39:40,  7.36s/it]
 19%|█▉        | 43649/225000 [93:10:17<351:31:07,  6.98s/it]
 19%|█▉        | 43650/225000 [93:10:24<349:33:18,  6.94s/it]
                                                             
{'loss': '0.7021', 'grad_norm': '0.7031', 'learning_rate': '4.704e-05', 'epoch': '0.194', 'num_input_tokens_seen': 157034496, 'train_runtime': '3.354e+05', 'train_tokens_per_second': '468.2'}

 19%|█▉        | 43650/225000 [93:10:24<349:33:18,  6.94s/it]
 19%|█▉        | 43651/225000 [93:10:32<368:34:56,  7.32s/it]
 19%|█▉        | 43652/225000 [93:10:37<338:33:24,  6.72s/it]
 19%|█▉        | 43653/225000 [93:10:46<376:48:56,  7.48s/it]
 19%|█▉        | 43654/225000 [93:10:54<382:32:04,  7.59s/it]
 19%|█▉        | 43655/225000 [93:11:00<354:19:56,  7.03s/it]
 19%|█▉        | 43656/225000 [93:11:08<367:02:24,  7.29s/it]
 19%|█▉        | 43657/225000 [93:11:14<345:40:51,  6.86s/it]
 19%|█▉        | 43658/225000 [93:11:20<337:32:04,  6.70s/it]
 19%|█▉        | 43659/225000 [93:11:28<361:03:39,  7.17s/it]
 19%|█▉        | 43660/225000 [93:11:33<321:28:43,  6.38s/it]
 19%|█▉        | 43661/225000 [93:11:37<289:53:41,  5.76s/it]
 19%|█▉        | 43662/225000 [93:11:46<330:35:43,  6.56s/it]
 19%|█▉        | 43663/225000 [93:11:55<367:10:35,  7.29s/it]
 19%|█▉        | 43664/225000 [93:12:00<337:12:53,  6.69s/it]
 19%|█▉        | 43665/225000 [93:12:05<317:41:53,  6.31s/it]
 19%|█▉        | 43666/225000 [93:12:11<306:11:59,  6.08s/it]
 19%|█▉        | 43667/225000 [93:12:17<299:42:49,  5.95s/it]
 19%|█▉        | 43668/225000 [93:12:22<296:01:08,  5.88s/it]
 19%|█▉        | 43669/225000 [93:12:28<293:56:41,  5.84s/it]
 19%|█▉        | 43670/225000 [93:12:34<301:50:46,  5.99s/it]
 19%|█▉        | 43671/225000 [93:12:48<409:54:35,  8.14s/it]
 19%|█▉        | 43672/225000 [93:12:53<373:47:35,  7.42s/it]
 19%|█▉        | 43673/225000 [93:12:58<337:44:09,  6.71s/it]
 19%|█▉        | 43674/225000 [93:13:05<337:04:44,  6.69s/it]
 19%|█▉        | 43675/225000 [93:13:13<351:03:28,  6.97s/it]
 19%|█▉        | 43676/225000 [93:13:21<374:38:10,  7.44s/it]
 19%|█▉        | 43677/225000 [93:13:26<337:57:50,  6.71s/it]
 19%|█▉        | 43678/225000 [93:13:31<316:01:29,  6.27s/it]
 19%|█▉        | 43679/225000 [93:13:37<303:50:28,  6.03s/it]
 19%|█▉        | 43680/225000 [93:13:42<287:37:02,  5.71s/it]
 19%|█▉        | 43681/225000 [93:13:47<276:17:40,  5.49s/it]
 19%|█▉        | 43682/225000 [93:14:00<392:39:54,  7.80s/it]
 19%|█▉        | 43683/225000 [93:14:06<365:30:59,  7.26s/it]
 19%|█▉        | 43684/225000 [93:14:10<315:24:32,  6.26s/it]
 19%|█▉        | 43685/225000 [93:14:21<393:00:32,  7.80s/it]
 19%|█▉        | 43686/225000 [93:14:26<350:54:25,  6.97s/it]
 19%|█▉        | 43687/225000 [93:14:32<333:52:30,  6.63s/it]
 19%|█▉        | 43688/225000 [93:14:38<316:45:39,  6.29s/it]
 19%|█▉        | 43689/225000 [93:14:43<310:11:35,  6.16s/it]
 19%|█▉        | 43690/225000 [93:14:49<299:47:27,  5.95s/it]
 19%|█▉        | 43691/225000 [93:14:57<338:26:18,  6.72s/it]
 19%|█▉        | 43692/225000 [93:15:09<417:37:53,  8.29s/it]
 19%|█▉        | 43693/225000 [93:15:14<362:28:33,  7.20s/it]
 19%|█▉        | 43694/225000 [93:15:20<345:57:40,  6.87s/it]
 19%|█▉        | 43695/225000 [93:15:33<436:59:21,  8.68s/it]
 19%|█▉        | 43696/225000 [93:15:38<386:55:26,  7.68s/it]
 19%|█▉        | 43697/225000 [93:15:44<361:17:18,  7.17s/it]
 19%|█▉        | 43698/225000 [93:15:50<344:25:14,  6.84s/it]
 19%|█▉        | 43699/225000 [93:15:56<330:43:47,  6.57s/it]
 19%|█▉        | 43700/225000 [93:16:08<405:04:34,  8.04s/it]
                                                             
{'loss': '0.7107', 'grad_norm': '0.7527', 'learning_rate': '4.703e-05', 'epoch': '0.1942', 'num_input_tokens_seen': 157188976, 'train_runtime': '3.358e+05', 'train_tokens_per_second': '468.1'}

 19%|█▉        | 43700/225000 [93:16:08<405:04:34,  8.04s/it]
 19%|█▉        | 43701/225000 [93:16:12<350:51:34,  6.97s/it]
 19%|█▉        | 43702/225000 [93:16:21<382:52:04,  7.60s/it]
 19%|█▉        | 43703/225000 [93:16:26<332:26:04,  6.60s/it]
 19%|█▉        | 43704/225000 [93:16:30<296:54:28,  5.90s/it]
 19%|█▉        | 43705/225000 [93:16:36<298:25:13,  5.93s/it]
 19%|█▉        | 43706/225000 [93:16:47<379:52:03,  7.54s/it]
 19%|█▉        | 43707/225000 [93:16:54<361:11:57,  7.17s/it]
 19%|█▉        | 43708/225000 [93:16:59<329:46:20,  6.55s/it]
 19%|█▉        | 43709/225000 [93:17:03<290:18:10,  5.76s/it]
 19%|█▉        | 43710/225000 [93:17:08<286:15:56,  5.68s/it]
 19%|█▉        | 43711/225000 [93:17:20<376:07:00,  7.47s/it]
 19%|█▉        | 43712/225000 [93:17:24<325:47:46,  6.47s/it]
 19%|█▉        | 43713/225000 [93:17:34<386:02:30,  7.67s/it]
 19%|█▉        | 43714/225000 [93:17:40<351:51:57,  6.99s/it]
 19%|█▉        | 43715/225000 [93:17:44<316:09:54,  6.28s/it]
 19%|█▉        | 43716/225000 [93:17:49<292:35:38,  5.81s/it]
 19%|█▉        | 43717/225000 [93:17:55<291:58:31,  5.80s/it]
 19%|█▉        | 43718/225000 [93:18:08<405:57:37,  8.06s/it]
 19%|█▉        | 43719/225000 [93:18:12<340:48:00,  6.77s/it]
 19%|█▉        | 43720/225000 [93:18:16<303:50:52,  6.03s/it]
 19%|█▉        | 43721/225000 [93:18:29<407:44:19,  8.10s/it]
 19%|█▉        | 43722/225000 [93:18:33<345:08:05,  6.85s/it]
 19%|█▉        | 43723/225000 [93:18:37<305:53:20,  6.07s/it]
 19%|█▉        | 43724/225000 [93:18:44<312:33:30,  6.21s/it]
 19%|█▉        | 43725/225000 [93:18:55<391:10:04,  7.77s/it]
 19%|█▉        | 43726/225000 [93:19:00<342:15:39,  6.80s/it]
 19%|█▉        | 43727/225000 [93:19:05<314:37:09,  6.25s/it]
 19%|█▉        | 43728/225000 [93:19:10<297:15:32,  5.90s/it]
 19%|█▉        | 43729/225000 [93:19:15<291:30:42,  5.79s/it]
 19%|█▉        | 43730/225000 [93:19:21<284:51:39,  5.66s/it]
 19%|█▉        | 43731/225000 [93:19:34<392:39:29,  7.80s/it]
 19%|█▉        | 43732/225000 [93:19:41<384:39:27,  7.64s/it]
 19%|█▉        | 43733/225000 [93:19:46<341:59:36,  6.79s/it]
 19%|█▉        | 43734/225000 [93:19:50<310:23:54,  6.16s/it]
 19%|█▉        | 43735/225000 [93:19:55<290:43:35,  5.77s/it]
 19%|█▉        | 43736/225000 [93:20:00<280:20:32,  5.57s/it]
 19%|█▉        | 43737/225000 [93:20:11<353:41:33,  7.02s/it]
 19%|█▉        | 43738/225000 [93:20:22<424:59:01,  8.44s/it]
 19%|█▉        | 43739/225000 [93:20:28<385:47:18,  7.66s/it]
 19%|█▉        | 43740/225000 [93:20:33<341:44:39,  6.79s/it]
 19%|█▉        | 43741/225000 [93:20:40<347:40:27,  6.91s/it]
 19%|█▉        | 43742/225000 [93:20:49<369:46:25,  7.34s/it]
 19%|█▉        | 43743/225000 [93:20:57<387:11:01,  7.69s/it]
 19%|█▉        | 43744/225000 [93:21:02<340:29:44,  6.76s/it]
 19%|█▉        | 43745/225000 [93:21:07<311:09:59,  6.18s/it]
 19%|█▉        | 43746/225000 [93:21:15<344:13:13,  6.84s/it]
 19%|█▉        | 43747/225000 [93:21:25<386:13:33,  7.67s/it]
 19%|█▉        | 43748/225000 [93:21:32<389:14:37,  7.73s/it]
 19%|█▉        | 43749/225000 [93:21:38<350:53:03,  6.97s/it]
 19%|█▉        | 43750/225000 [93:21:45<361:45:12,  7.19s/it]
                                                             
{'loss': '0.6952', 'grad_norm': '0.7046', 'learning_rate': '4.702e-05', 'epoch': '0.1944', 'num_input_tokens_seen': 157339840, 'train_runtime': '3.361e+05', 'train_tokens_per_second': '468.1'}

 19%|█▉        | 43750/225000 [93:21:45<361:45:12,  7.19s/it]
 19%|█▉        | 43751/225000 [93:21:50<328:37:59,  6.53s/it]
 19%|█▉        | 43752/225000 [93:21:55<305:12:33,  6.06s/it]
 19%|█▉        | 43753/225000 [93:22:00<291:00:03,  5.78s/it]
 19%|█▉        | 43754/225000 [93:22:06<285:24:23,  5.67s/it]
 19%|█▉        | 43755/225000 [93:22:11<279:52:51,  5.56s/it]
 19%|█▉        | 43756/225000 [93:22:24<389:51:16,  7.74s/it]
 19%|█▉        | 43757/225000 [93:22:28<340:54:26,  6.77s/it]
 19%|█▉        | 43758/225000 [93:22:36<349:45:15,  6.95s/it]
 19%|█▉        | 43759/225000 [93:22:45<377:51:03,  7.51s/it]
 19%|█▉        | 43760/225000 [93:22:49<328:07:32,  6.52s/it]
 19%|█▉        | 43761/225000 [93:22:57<357:20:14,  7.10s/it]
 19%|█▉        | 43762/225000 [93:23:04<352:47:14,  7.01s/it]
 19%|█▉        | 43763/225000 [93:23:08<311:10:32,  6.18s/it]
 19%|█▉        | 43764/225000 [93:23:17<356:12:31,  7.08s/it]
 19%|█▉        | 43765/225000 [93:23:22<323:24:22,  6.42s/it]
 19%|█▉        | 43766/225000 [93:23:31<355:21:17,  7.06s/it]
 19%|█▉        | 43767/225000 [93:23:37<336:52:33,  6.69s/it]
 19%|█▉        | 43768/225000 [93:23:45<356:18:00,  7.08s/it]
 19%|█▉        | 43769/225000 [93:23:51<345:05:56,  6.86s/it]
 19%|█▉        | 43770/225000 [93:23:57<336:33:19,  6.69s/it]
 19%|█▉        | 43771/225000 [93:24:02<308:25:15,  6.13s/it]
 19%|█▉        | 43772/225000 [93:24:07<293:24:33,  5.83s/it]
 19%|█▉        | 43773/225000 [93:24:17<354:56:31,  7.05s/it]
 19%|█▉        | 43774/225000 [93:24:26<385:31:12,  7.66s/it]
 19%|█▉        | 43775/225000 [93:24:32<354:57:41,  7.05s/it]
 19%|█▉        | 43776/225000 [93:24:38<336:14:39,  6.68s/it]
 19%|█▉        | 43777/225000 [93:24:43<319:02:52,  6.34s/it]
 19%|█▉        | 43778/225000 [93:24:49<306:28:16,  6.09s/it]
 19%|█▉        | 43779/225000 [93:24:54<291:50:33,  5.80s/it]
 19%|█▉        | 43780/225000 [93:25:00<300:39:53,  5.97s/it]
 19%|█▉        | 43781/225000 [93:25:14<414:59:15,  8.24s/it]
 19%|█▉        | 43782/225000 [93:25:19<365:24:59,  7.26s/it]
 19%|█▉        | 43783/225000 [93:25:28<392:29:17,  7.80s/it]
 19%|█▉        | 43784/225000 [93:25:33<352:49:15,  7.01s/it]
 19%|█▉        | 43785/225000 [93:25:38<323:17:00,  6.42s/it]
 19%|█▉        | 43786/225000 [93:25:43<298:21:04,  5.93s/it]
 19%|█▉        | 43787/225000 [93:25:49<301:37:42,  5.99s/it]
 19%|█▉        | 43788/225000 [93:26:02<412:02:53,  8.19s/it]
 19%|█▉        | 43789/225000 [93:26:06<347:14:30,  6.90s/it]
 19%|█▉        | 43790/225000 [93:26:15<379:38:35,  7.54s/it]
 19%|█▉        | 43791/225000 [93:26:21<346:11:23,  6.88s/it]
 19%|█▉        | 43792/225000 [93:26:25<306:13:07,  6.08s/it]
 19%|█▉        | 43793/225000 [93:26:37<392:58:36,  7.81s/it]
 19%|█▉        | 43794/225000 [93:26:41<337:45:23,  6.71s/it]
 19%|█▉        | 43795/225000 [93:26:49<366:46:13,  7.29s/it]
 19%|█▉        | 43796/225000 [93:26:55<343:49:27,  6.83s/it]
 19%|█▉        | 43797/225000 [93:27:04<372:51:32,  7.41s/it]
 19%|█▉        | 43798/225000 [93:27:09<337:58:12,  6.71s/it]
 19%|█▉        | 43799/225000 [93:27:17<356:46:18,  7.09s/it]
 19%|█▉        | 43800/225000 [93:27:25<366:34:35,  7.28s/it]
                                                             
{'loss': '0.7151', 'grad_norm': '0.8209', 'learning_rate': '4.702e-05', 'epoch': '0.1947', 'num_input_tokens_seen': 157490312, 'train_runtime': '3.364e+05', 'train_tokens_per_second': '468.1'}

 19%|█▉        | 43800/225000 [93:27:25<366:34:35,  7.28s/it]
 19%|█▉        | 43801/225000 [93:27:30<329:58:25,  6.56s/it]
 19%|█▉        | 43802/225000 [93:27:38<355:12:24,  7.06s/it]
 19%|█▉        | 43803/225000 [93:27:42<310:46:50,  6.17s/it]
 19%|█▉        | 43804/225000 [93:27:47<287:17:50,  5.71s/it]
 19%|█▉        | 43805/225000 [93:27:51<270:40:39,  5.38s/it]
 19%|█▉        | 43806/225000 [93:28:02<352:51:41,  7.01s/it]
 19%|█▉        | 43807/225000 [93:28:09<357:44:04,  7.11s/it]
 19%|█▉        | 43808/225000 [93:28:16<358:41:41,  7.13s/it]
 19%|█▉        | 43809/225000 [93:28:22<327:44:42,  6.51s/it]
 19%|█▉        | 43810/225000 [93:28:27<304:39:51,  6.05s/it]
 19%|█▉        | 43811/225000 [93:28:31<287:58:09,  5.72s/it]
 19%|█▉        | 43812/225000 [93:28:36<276:32:57,  5.49s/it]
 19%|█▉        | 43813/225000 [93:28:48<361:16:31,  7.18s/it]
 19%|█▉        | 43814/225000 [93:28:55<359:56:08,  7.15s/it]
 19%|█▉        | 43815/225000 [93:29:03<383:47:29,  7.63s/it]
 19%|█▉        | 43816/225000 [93:29:08<341:03:11,  6.78s/it]
 19%|█▉        | 43817/225000 [93:29:13<313:42:24,  6.23s/it]
 19%|█▉        | 43818/225000 [93:29:19<301:03:17,  5.98s/it]
 19%|█▉        | 43819/225000 [93:29:24<298:08:33,  5.92s/it]
 19%|█▉        | 43820/225000 [93:29:30<294:11:37,  5.85s/it]
 19%|█▉        | 43821/225000 [93:29:43<403:53:23,  8.03s/it]
 19%|█▉        | 43822/225000 [93:29:50<381:55:15,  7.59s/it]
 19%|█▉        | 43823/225000 [93:29:53<321:16:38,  6.38s/it]
 19%|█▉        | 43824/225000 [93:30:04<382:57:02,  7.61s/it]
 19%|█▉        | 43825/225000 [93:30:08<326:39:13,  6.49s/it]
 19%|█▉        | 43826/225000 [93:30:12<288:32:54,  5.73s/it]
 19%|█▉        | 43827/225000 [93:30:22<367:00:21,  7.29s/it]
 19%|█▉        | 43828/225000 [93:30:28<342:53:20,  6.81s/it]
 19%|█▉        | 43829/225000 [93:30:34<325:53:28,  6.48s/it]
 19%|█▉        | 43830/225000 [93:30:40<313:30:44,  6.23s/it]
 19%|█▉        | 43831/225000 [93:30:45<305:18:46,  6.07s/it]
 19%|█▉        | 43832/225000 [93:30:51<302:58:07,  6.02s/it]
 19%|█▉        | 43833/225000 [93:30:58<311:43:12,  6.19s/it]
 19%|█▉        | 43834/225000 [93:31:03<302:01:29,  6.00s/it]
 19%|█▉        | 43835/225000 [93:31:09<297:11:34,  5.91s/it]
 19%|█▉        | 43836/225000 [93:31:15<294:15:00,  5.85s/it]
 19%|█▉        | 43837/225000 [93:31:23<329:27:22,  6.55s/it]
 19%|█▉        | 43838/225000 [93:31:36<436:01:45,  8.66s/it]
 19%|█▉        | 43839/225000 [93:31:41<380:57:34,  7.57s/it]
 19%|█▉        | 43840/225000 [93:31:46<328:17:23,  6.52s/it]
 19%|█▉        | 43841/225000 [93:31:54<364:06:24,  7.24s/it]
 19%|█▉        | 43842/225000 [93:32:01<351:23:16,  6.98s/it]
 19%|█▉        | 43843/225000 [93:32:06<321:07:28,  6.38s/it]
 19%|█▉        | 43844/225000 [93:32:15<370:53:44,  7.37s/it]
 19%|█▉        | 43845/225000 [93:32:20<327:05:29,  6.50s/it]
 19%|█▉        | 43846/225000 [93:32:24<292:55:18,  5.82s/it]
 19%|█▉        | 43847/225000 [93:32:36<387:58:22,  7.71s/it]
 19%|█▉        | 43848/225000 [93:32:44<391:51:18,  7.79s/it]
 19%|█▉        | 43849/225000 [93:32:48<327:38:42,  6.51s/it]
 19%|█▉        | 43850/225000 [93:32:53<301:26:11,  5.99s/it]
                                                             
{'loss': '0.7088', 'grad_norm': '0.7941', 'learning_rate': '4.701e-05', 'epoch': '0.1949', 'num_input_tokens_seen': 157642056, 'train_runtime': '3.368e+05', 'train_tokens_per_second': '468.1'}

 19%|█▉        | 43850/225000 [93:32:53<301:26:11,  5.99s/it]
 19%|█▉        | 43851/225000 [93:33:03<373:14:25,  7.42s/it]
 19%|█▉        | 43852/225000 [93:33:07<322:21:32,  6.41s/it]
 19%|█▉        | 43853/225000 [93:33:16<361:34:52,  7.19s/it]
 19%|█▉        | 43854/225000 [93:33:25<386:23:08,  7.68s/it]
 19%|█▉        | 43855/225000 [93:33:31<354:02:06,  7.04s/it]
 19%|█▉        | 43856/225000 [93:33:36<332:55:48,  6.62s/it]
 19%|█▉        | 43857/225000 [93:33:42<320:13:51,  6.36s/it]
 19%|█▉        | 43858/225000 [93:33:48<308:57:31,  6.14s/it]
 19%|█▉        | 43859/225000 [93:33:54<304:06:46,  6.04s/it]
 19%|█▉        | 43860/225000 [93:33:59<299:45:28,  5.96s/it]
 19%|█▉        | 43861/225000 [93:34:05<297:28:35,  5.91s/it]
 19%|█▉        | 43862/225000 [93:34:11<296:42:35,  5.90s/it]
 19%|█▉        | 43863/225000 [93:34:17<299:03:05,  5.94s/it]
 19%|█▉        | 43864/225000 [93:34:29<382:52:44,  7.61s/it]
 19%|█▉        | 43865/225000 [93:34:38<411:34:09,  8.18s/it]
 19%|█▉        | 43866/225000 [93:34:45<391:00:47,  7.77s/it]
 19%|█▉        | 43867/225000 [93:34:49<338:22:37,  6.73s/it]
 19%|█▉        | 43868/225000 [93:34:57<355:15:41,  7.06s/it]
 19%|█▉        | 43869/225000 [93:35:05<369:16:09,  7.34s/it]
 19%|█▉        | 43870/225000 [93:35:10<327:35:06,  6.51s/it]
 19%|█▉        | 43871/225000 [93:35:14<300:19:09,  5.97s/it]
 19%|█▉        | 43872/225000 [93:35:22<331:44:00,  6.59s/it]
 19%|█▉        | 43873/225000 [93:35:33<388:34:20,  7.72s/it]
 19%|█▉        | 43874/225000 [93:35:37<330:05:07,  6.56s/it]
 20%|█▉        | 43875/225000 [93:35:47<395:47:15,  7.87s/it]
 20%|█▉        | 43876/225000 [93:35:54<381:59:17,  7.59s/it]
 20%|█▉        | 43877/225000 [93:35:58<327:54:17,  6.52s/it]
 20%|█▉        | 43878/225000 [93:36:03<294:16:00,  5.85s/it]
 20%|█▉        | 43879/225000 [93:36:12<343:42:41,  6.83s/it]
 20%|█▉        | 43880/225000 [93:36:18<327:33:44,  6.51s/it]
 20%|█▉        | 43881/225000 [93:36:26<357:13:42,  7.10s/it]
 20%|█▉        | 43882/225000 [93:36:31<317:38:28,  6.31s/it]
 20%|█▉        | 43883/225000 [93:36:35<291:31:56,  5.79s/it]
 20%|█▉        | 43884/225000 [93:36:41<289:04:07,  5.75s/it]
 20%|█▉        | 43885/225000 [93:36:45<271:55:21,  5.40s/it]
 20%|█▉        | 43886/225000 [93:36:54<326:24:54,  6.49s/it]
 20%|█▉        | 43887/225000 [93:37:07<420:07:51,  8.35s/it]
 20%|█▉        | 43888/225000 [93:37:11<347:53:42,  6.92s/it]
 20%|█▉        | 43889/225000 [93:37:15<306:59:03,  6.10s/it]
 20%|█▉        | 43890/225000 [93:37:20<285:45:22,  5.68s/it]
 20%|█▉        | 43891/225000 [93:37:32<385:12:23,  7.66s/it]
 20%|█▉        | 43892/225000 [93:37:37<352:23:48,  7.00s/it]
 20%|█▉        | 43893/225000 [93:37:44<349:19:42,  6.94s/it]
 20%|█▉        | 43894/225000 [93:37:49<322:13:42,  6.41s/it]
 20%|█▉        | 43895/225000 [93:37:57<342:03:54,  6.80s/it]
 20%|█▉        | 43896/225000 [93:38:02<321:19:13,  6.39s/it]
 20%|█▉        | 43897/225000 [93:38:07<297:55:52,  5.92s/it]
 20%|█▉        | 43898/225000 [93:38:14<316:26:20,  6.29s/it]
 20%|█▉        | 43899/225000 [93:38:25<387:08:35,  7.70s/it]
 20%|█▉        | 43900/225000 [93:38:30<339:18:42,  6.75s/it]
                                                             
{'loss': '0.6951', 'grad_norm': '0.7164', 'learning_rate': '4.7e-05', 'epoch': '0.1951', 'num_input_tokens_seen': 157794376, 'train_runtime': '3.371e+05', 'train_tokens_per_second': '468.1'}

 20%|█▉        | 43900/225000 [93:38:30<339:18:42,  6.75s/it]
 20%|█▉        | 43901/225000 [93:38:35<309:00:15,  6.14s/it]
 20%|█▉        | 43902/225000 [93:38:39<287:08:17,  5.71s/it]
 20%|█▉        | 43903/225000 [93:38:45<287:26:39,  5.71s/it]
 20%|█▉        | 43904/225000 [93:38:59<405:19:38,  8.06s/it]
 20%|█▉        | 43905/225000 [93:39:04<361:57:57,  7.20s/it]
 20%|█▉        | 43906/225000 [93:39:09<328:17:24,  6.53s/it]
 20%|█▉        | 43907/225000 [93:39:14<303:08:12,  6.03s/it]
 20%|█▉        | 43908/225000 [93:39:24<365:13:35,  7.26s/it]
 20%|█▉        | 43909/225000 [93:39:30<351:28:35,  6.99s/it]
 20%|█▉        | 43910/225000 [93:39:36<337:57:49,  6.72s/it]
 20%|█▉        | 43911/225000 [93:39:45<369:53:18,  7.35s/it]
 20%|█▉        | 43912/225000 [93:39:49<315:53:51,  6.28s/it]
 20%|█▉        | 43913/225000 [93:39:59<381:53:54,  7.59s/it]
 20%|█▉        | 43914/225000 [93:40:03<326:30:08,  6.49s/it]
 20%|█▉        | 43915/225000 [93:40:11<344:26:55,  6.85s/it]
 20%|█▉        | 43916/225000 [93:40:16<323:16:48,  6.43s/it]
 20%|█▉        | 43917/225000 [93:40:28<393:27:31,  7.82s/it]
 20%|█▉        | 43918/225000 [93:40:32<336:04:22,  6.68s/it]
 20%|█▉        | 43919/225000 [93:40:41<371:45:50,  7.39s/it]
 20%|█▉        | 43920/225000 [93:40:45<319:38:38,  6.35s/it]
 20%|█▉        | 43921/225000 [93:40:50<310:21:31,  6.17s/it]
 20%|█▉        | 43922/225000 [93:40:56<304:14:26,  6.05s/it]
 20%|█▉        | 43923/225000 [93:41:02<302:01:07,  6.00s/it]
 20%|█▉        | 43924/225000 [93:41:08<298:02:11,  5.93s/it]
 20%|█▉        | 43925/225000 [93:41:13<293:39:13,  5.84s/it]
 20%|█▉        | 43926/225000 [93:41:19<291:12:33,  5.79s/it]
 20%|█▉        | 43927/225000 [93:41:29<358:57:37,  7.14s/it]
 20%|█▉        | 43928/225000 [93:41:39<402:21:12,  8.00s/it]
 20%|█▉        | 43929/225000 [93:41:43<343:28:49,  6.83s/it]
 20%|█▉        | 43930/225000 [93:41:53<385:40:06,  7.67s/it]
 20%|█▉        | 43931/225000 [93:42:00<370:12:13,  7.36s/it]
 20%|█▉        | 43932/225000 [93:42:03<313:56:32,  6.24s/it]
 20%|█▉        | 43933/225000 [93:42:13<368:37:27,  7.33s/it]
 20%|█▉        | 43934/225000 [93:42:18<328:21:37,  6.53s/it]
 20%|█▉        | 43935/225000 [93:42:23<304:12:50,  6.05s/it]
 20%|█▉        | 43936/225000 [93:42:31<339:51:04,  6.76s/it]
 20%|█▉        | 43937/225000 [93:42:41<381:35:35,  7.59s/it]
 20%|█▉        | 43938/225000 [93:42:49<385:16:18,  7.66s/it]
 20%|█▉        | 43939/225000 [93:42:52<329:16:56,  6.55s/it]
 20%|█▉        | 43940/225000 [93:42:58<312:03:55,  6.20s/it]
 20%|█▉        | 43941/225000 [93:43:03<290:50:10,  5.78s/it]
 20%|█▉        | 43942/225000 [93:43:08<278:09:10,  5.53s/it]
 20%|█▉        | 43943/225000 [93:43:13<282:54:53,  5.63s/it]
 20%|█▉        | 43944/225000 [93:43:27<396:03:37,  7.88s/it]
 20%|█▉        | 43945/225000 [93:43:32<365:55:02,  7.28s/it]
 20%|█▉        | 43946/225000 [93:43:39<359:04:38,  7.14s/it]
 20%|█▉        | 43947/225000 [93:43:43<306:07:01,  6.09s/it]
 20%|█▉        | 43948/225000 [93:43:52<358:24:11,  7.13s/it]
 20%|█▉        | 43949/225000 [93:43:58<339:11:51,  6.74s/it]
 20%|█▉        | 43950/225000 [93:44:06<352:58:12,  7.02s/it]
                                                             
{'loss': '0.7037', 'grad_norm': '0.7245', 'learning_rate': '4.699e-05', 'epoch': '0.1953', 'num_input_tokens_seen': 157945952, 'train_runtime': '3.374e+05', 'train_tokens_per_second': '468.1'}

 20%|█▉        | 43950/225000 [93:44:06<352:58:12,  7.02s/it]
 20%|█▉        | 43951/225000 [93:44:14<361:21:48,  7.19s/it]
 20%|█▉        | 43952/225000 [93:44:17<310:26:05,  6.17s/it]
 20%|█▉        | 43953/225000 [93:44:21<276:51:15,  5.51s/it]
 20%|█▉        | 43954/225000 [93:44:30<330:13:07,  6.57s/it]
 20%|█▉        | 43955/225000 [93:44:38<341:55:14,  6.80s/it]
 20%|█▉        | 43956/225000 [93:44:43<321:48:23,  6.40s/it]
 20%|█▉        | 43957/225000 [93:44:50<331:32:57,  6.59s/it]
 20%|█▉        | 43958/225000 [93:44:56<323:17:21,  6.43s/it]
 20%|█▉        | 43959/225000 [93:45:02<316:16:54,  6.29s/it]
 20%|█▉        | 43960/225000 [93:45:08<310:04:33,  6.17s/it]
 20%|█▉        | 43961/225000 [93:45:14<306:09:08,  6.09s/it]
 20%|█▉        | 43962/225000 [93:45:19<297:00:40,  5.91s/it]
 20%|█▉        | 43963/225000 [93:45:25<296:26:31,  5.89s/it]
 20%|█▉        | 43964/225000 [93:45:31<291:33:41,  5.80s/it]
 20%|█▉        | 43965/225000 [93:45:40<333:36:36,  6.63s/it]
 20%|█▉        | 43966/225000 [93:45:53<440:21:38,  8.76s/it]
 20%|█▉        | 43967/225000 [93:45:58<373:02:15,  7.42s/it]
 20%|█▉        | 43968/225000 [93:46:02<327:24:12,  6.51s/it]
 20%|█▉        | 43969/225000 [93:46:08<319:29:31,  6.35s/it]
 20%|█▉        | 43970/225000 [93:46:18<381:55:09,  7.59s/it]
 20%|█▉        | 43971/225000 [93:46:23<333:55:37,  6.64s/it]
 20%|█▉        | 43972/225000 [93:46:27<300:22:01,  5.97s/it]
 20%|█▉        | 43973/225000 [93:46:36<348:03:10,  6.92s/it]
 20%|█▉        | 43974/225000 [93:46:44<359:48:09,  7.16s/it]
 20%|█▉        | 43975/225000 [93:46:49<320:14:13,  6.37s/it]
 20%|█▉        | 43976/225000 [93:46:58<362:52:31,  7.22s/it]
 20%|█▉        | 43977/225000 [93:47:04<353:00:27,  7.02s/it]
 20%|█▉        | 43978/225000 [93:47:08<306:13:15,  6.09s/it]
 20%|█▉        | 43979/225000 [93:47:18<368:16:03,  7.32s/it]
 20%|█▉        | 43980/225000 [93:47:23<326:04:46,  6.48s/it]
 20%|█▉        | 43981/225000 [93:47:31<349:05:03,  6.94s/it]
 20%|█▉        | 43982/225000 [93:47:38<344:38:01,  6.85s/it]
 20%|█▉        | 43983/225000 [93:47:44<333:43:15,  6.64s/it]
 20%|█▉        | 43984/225000 [93:47:49<305:08:21,  6.07s/it]
 20%|█▉        | 43985/225000 [93:47:53<286:22:09,  5.70s/it]
 20%|█▉        | 43986/225000 [93:47:58<271:42:44,  5.40s/it]
 20%|█▉        | 43987/225000 [93:48:11<385:24:33,  7.67s/it]
 20%|█▉        | 43988/225000 [93:48:15<332:26:08,  6.61s/it]
 20%|█▉        | 43989/225000 [93:48:22<335:33:31,  6.67s/it]
 20%|█▉        | 43990/225000 [93:48:27<314:53:28,  6.26s/it]
 20%|█▉        | 43991/225000 [93:48:33<310:46:53,  6.18s/it]
 20%|█▉        | 43992/225000 [93:48:44<378:18:36,  7.52s/it]
 20%|█▉        | 43993/225000 [93:48:48<325:19:46,  6.47s/it]
 20%|█▉        | 43994/225000 [93:48:55<340:10:47,  6.77s/it]
 20%|█▉        | 43995/225000 [93:49:00<309:28:11,  6.16s/it]
 20%|█▉        | 43996/225000 [93:49:09<350:54:33,  6.98s/it]
 20%|█▉        | 43997/225000 [93:49:13<308:48:00,  6.14s/it]
 20%|█▉        | 43998/225000 [93:49:19<299:52:31,  5.96s/it]
 20%|█▉        | 43999/225000 [93:49:24<287:17:48,  5.71s/it]
 20%|█▉        | 44000/225000 [93:49:30<286:15:08,  5.69s/it]
                                                             
{'loss': '0.6934', 'grad_norm': '0.6764', 'learning_rate': '4.698e-05', 'epoch': '0.1956', 'num_input_tokens_seen': 158096200, 'train_runtime': '3.378e+05', 'train_tokens_per_second': '468.1'}

 20%|█▉        | 44000/225000 [93:49:30<286:15:08,  5.69s/it]
 20%|█▉        | 44001/225000 [93:49:35<283:35:06,  5.64s/it]
 20%|█▉        | 44002/225000 [93:49:48<395:55:30,  7.87s/it]
 20%|█▉        | 44003/225000 [93:49:55<384:08:45,  7.64s/it]
 20%|█▉        | 44004/225000 [93:50:02<363:57:36,  7.24s/it]
 20%|█▉        | 44005/225000 [93:50:10<383:29:42,  7.63s/it]
 20%|█▉        | 44006/225000 [93:50:14<332:39:55,  6.62s/it]
 20%|█▉        | 44007/225000 [93:50:23<362:55:49,  7.22s/it]
 20%|█▉        | 44008/225000 [93:50:28<322:59:55,  6.42s/it]
 20%|█▉        | 44009/225000 [93:50:33<315:48:22,  6.28s/it]
 20%|█▉        | 44010/225000 [93:50:40<311:58:36,  6.21s/it]
 20%|█▉        | 44011/225000 [93:50:45<302:41:50,  6.02s/it]
 20%|█▉        | 44012/225000 [93:50:51<295:48:00,  5.88s/it]
 20%|█▉        | 44013/225000 [93:50:56<294:14:29,  5.85s/it]
 20%|█▉        | 44014/225000 [93:51:02<291:32:55,  5.80s/it]
 20%|█▉        | 44015/225000 [93:51:08<289:53:40,  5.77s/it]
 20%|█▉        | 44016/225000 [93:51:17<341:32:10,  6.79s/it]
 20%|█▉        | 44017/225000 [93:51:30<433:17:43,  8.62s/it]
 20%|█▉        | 44018/225000 [93:51:33<356:05:09,  7.08s/it]
 20%|█▉        | 44019/225000 [93:51:39<327:03:49,  6.51s/it]
 20%|█▉        | 44020/225000 [93:51:44<308:03:04,  6.13s/it]
 20%|█▉        | 44021/225000 [93:51:49<294:28:22,  5.86s/it]
 20%|█▉        | 44022/225000 [93:51:58<334:11:10,  6.65s/it]
 20%|█▉        | 44023/225000 [93:52:10<428:06:22,  8.52s/it]
 20%|█▉        | 44024/225000 [93:52:14<356:30:20,  7.09s/it]
 20%|█▉        | 44025/225000 [93:52:20<332:08:53,  6.61s/it]
 20%|█▉        | 44026/225000 [93:52:25<317:35:26,  6.32s/it]
 20%|█▉        | 44027/225000 [93:52:30<299:29:15,  5.96s/it]
 20%|█▉        | 44028/225000 [93:52:36<295:23:59,  5.88s/it]
 20%|█▉        | 44029/225000 [93:52:48<384:09:42,  7.64s/it]
 20%|█▉        | 44030/225000 [93:52:57<409:40:39,  8.15s/it]
 20%|█▉        | 44031/225000 [93:53:02<358:04:05,  7.12s/it]
 20%|█▉        | 44032/225000 [93:53:07<331:32:37,  6.60s/it]
 20%|█▉        | 44033/225000 [93:53:13<315:13:48,  6.27s/it]
 20%|█▉        | 44034/225000 [93:53:19<309:30:34,  6.16s/it]
 20%|█▉        | 44035/225000 [93:53:24<300:29:03,  5.98s/it]
 20%|█▉        | 44036/225000 [93:53:34<361:40:10,  7.19s/it]
 20%|█▉        | 44037/225000 [93:53:39<323:24:04,  6.43s/it]
 20%|█▉        | 44038/225000 [93:53:49<385:08:08,  7.66s/it]
 20%|█▉        | 44039/225000 [93:53:55<347:56:11,  6.92s/it]
 20%|█▉        | 44040/225000 [93:54:02<351:06:22,  6.98s/it]
 20%|█▉        | 44041/225000 [93:54:06<302:55:32,  6.03s/it]
 20%|█▉        | 44042/225000 [93:54:13<327:26:42,  6.51s/it]
 20%|█▉        | 44043/225000 [93:54:19<310:01:40,  6.17s/it]
 20%|█▉        | 44044/225000 [93:54:25<307:01:29,  6.11s/it]
 20%|█▉        | 44045/225000 [93:54:30<293:22:04,  5.84s/it]
 20%|█▉        | 44046/225000 [93:54:35<281:47:23,  5.61s/it]
 20%|█▉        | 44047/225000 [93:54:49<408:28:05,  8.13s/it]
 20%|█▉        | 44048/225000 [93:54:54<357:49:12,  7.12s/it]
 20%|█▉        | 44049/225000 [93:55:03<388:58:20,  7.74s/it]
 20%|█▉        | 44050/225000 [93:55:07<343:33:19,  6.84s/it]
                                                             
{'loss': '0.7049', 'grad_norm': '0.7057', 'learning_rate': '4.697e-05', 'epoch': '0.1958', 'num_input_tokens_seen': 158247416, 'train_runtime': '3.381e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44050/225000 [93:55:08<343:33:19,  6.84s/it]
 20%|█▉        | 44051/225000 [93:55:16<375:37:54,  7.47s/it]
 20%|█▉        | 44052/225000 [93:55:23<363:02:12,  7.22s/it]
 20%|█▉        | 44053/225000 [93:55:28<324:35:31,  6.46s/it]
 20%|█▉        | 44054/225000 [93:55:33<299:07:10,  5.95s/it]
 20%|█▉        | 44055/225000 [93:55:38<287:25:14,  5.72s/it]
 20%|█▉        | 44056/225000 [93:55:43<282:12:29,  5.61s/it]
 20%|█▉        | 44057/225000 [93:55:55<371:36:26,  7.39s/it]
 20%|█▉        | 44058/225000 [93:56:03<384:01:13,  7.64s/it]
 20%|█▉        | 44059/225000 [93:56:08<346:57:48,  6.90s/it]
 20%|█▉        | 44060/225000 [93:56:13<318:35:12,  6.34s/it]
 20%|█▉        | 44061/225000 [93:56:18<292:38:16,  5.82s/it]
 20%|█▉        | 44062/225000 [93:56:31<401:49:35,  7.99s/it]
 20%|█▉        | 44063/225000 [93:56:36<358:15:52,  7.13s/it]
 20%|█▉        | 44064/225000 [93:56:41<327:45:29,  6.52s/it]
 20%|█▉        | 44065/225000 [93:56:46<307:17:32,  6.11s/it]
 20%|█▉        | 44066/225000 [93:56:58<400:10:57,  7.96s/it]
 20%|█▉        | 44067/225000 [93:57:03<347:40:49,  6.92s/it]
 20%|█▉        | 44068/225000 [93:57:10<348:46:28,  6.94s/it]
 20%|█▉        | 44069/225000 [93:57:19<381:24:46,  7.59s/it]
 20%|█▉        | 44070/225000 [93:57:23<328:02:07,  6.53s/it]
 20%|█▉        | 44071/225000 [93:57:28<298:44:47,  5.94s/it]
 20%|█▉        | 44072/225000 [93:57:34<304:41:57,  6.06s/it]
 20%|█▉        | 44073/225000 [93:57:45<375:14:09,  7.47s/it]
 20%|█▉        | 44074/225000 [93:57:51<352:12:31,  7.01s/it]
 20%|█▉        | 44075/225000 [93:57:57<345:34:05,  6.88s/it]
 20%|█▉        | 44076/225000 [93:58:05<356:21:45,  7.09s/it]
 20%|█▉        | 44077/225000 [93:58:11<348:26:35,  6.93s/it]
 20%|█▉        | 44078/225000 [93:58:18<342:48:40,  6.82s/it]
 20%|█▉        | 44079/225000 [93:58:23<317:42:19,  6.32s/it]
 20%|█▉        | 44080/225000 [93:58:29<308:33:39,  6.14s/it]
 20%|█▉        | 44081/225000 [93:58:34<300:52:24,  5.99s/it]
 20%|█▉        | 44082/225000 [93:58:40<297:48:54,  5.93s/it]
 20%|█▉        | 44083/225000 [93:58:46<297:48:58,  5.93s/it]
 20%|█▉        | 44084/225000 [93:58:58<385:54:49,  7.68s/it]
 20%|█▉        | 44085/225000 [93:59:03<350:07:43,  6.97s/it]
 20%|█▉        | 44086/225000 [93:59:11<358:54:18,  7.14s/it]
 20%|█▉        | 44087/225000 [93:59:16<325:04:17,  6.47s/it]
 20%|█▉        | 44088/225000 [93:59:22<323:11:37,  6.43s/it]
 20%|█▉        | 44089/225000 [93:59:32<380:27:03,  7.57s/it]
 20%|█▉        | 44090/225000 [93:59:37<332:57:01,  6.63s/it]
 20%|█▉        | 44091/225000 [93:59:43<331:09:07,  6.59s/it]
 20%|█▉        | 44092/225000 [93:59:53<378:13:04,  7.53s/it]
 20%|█▉        | 44093/225000 [93:59:58<338:19:10,  6.73s/it]
 20%|█▉        | 44094/225000 [94:00:06<363:25:25,  7.23s/it]
 20%|█▉        | 44095/225000 [94:00:14<375:51:03,  7.48s/it]
 20%|█▉        | 44096/225000 [94:00:19<334:23:15,  6.65s/it]
 20%|█▉        | 44097/225000 [94:00:24<312:45:16,  6.22s/it]
 20%|█▉        | 44098/225000 [94:00:30<302:05:56,  6.01s/it]
 20%|█▉        | 44099/225000 [94:00:35<293:11:55,  5.83s/it]
 20%|█▉        | 44100/225000 [94:00:40<285:50:07,  5.69s/it]
                                                             
{'loss': '0.6972', 'grad_norm': '0.8036', 'learning_rate': '4.696e-05', 'epoch': '0.196', 'num_input_tokens_seen': 158399136, 'train_runtime': '3.384e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44100/225000 [94:00:40<285:50:07,  5.69s/it]
 20%|█▉        | 44101/225000 [94:00:46<278:40:19,  5.55s/it]
 20%|█▉        | 44102/225000 [94:00:52<285:10:46,  5.68s/it]
 20%|█▉        | 44103/225000 [94:01:05<400:34:53,  7.97s/it]
 20%|█▉        | 44104/225000 [94:01:12<384:11:41,  7.65s/it]
 20%|█▉        | 44105/225000 [94:01:17<342:44:10,  6.82s/it]
 20%|█▉        | 44106/225000 [94:01:21<310:35:42,  6.18s/it]
 20%|█▉        | 44107/225000 [94:01:26<293:28:44,  5.84s/it]
 20%|█▉        | 44108/225000 [94:01:39<391:06:04,  7.78s/it]
 20%|█▉        | 44109/225000 [94:01:47<398:41:52,  7.93s/it]
 20%|█▉        | 44110/225000 [94:01:52<358:06:12,  7.13s/it]
 20%|█▉        | 44111/225000 [94:01:59<357:33:26,  7.12s/it]
 20%|█▉        | 44112/225000 [94:02:05<337:53:50,  6.72s/it]
 20%|█▉        | 44113/225000 [94:02:11<328:27:27,  6.54s/it]
 20%|█▉        | 44114/225000 [94:02:20<355:19:38,  7.07s/it]
 20%|█▉        | 44115/225000 [94:02:24<308:57:40,  6.15s/it]
 20%|█▉        | 44116/225000 [94:02:35<382:01:36,  7.60s/it]
 20%|█▉        | 44117/225000 [94:02:39<330:57:06,  6.59s/it]
 20%|█▉        | 44118/225000 [94:02:49<377:37:44,  7.52s/it]
 20%|█▉        | 44119/225000 [94:02:54<351:34:15,  7.00s/it]
 20%|█▉        | 44120/225000 [94:03:01<347:47:06,  6.92s/it]
 20%|█▉        | 44121/225000 [94:03:06<312:51:03,  6.23s/it]
 20%|█▉        | 44122/225000 [94:03:11<294:26:45,  5.86s/it]
 20%|█▉        | 44123/225000 [94:03:16<279:48:35,  5.57s/it]
 20%|█▉        | 44124/225000 [94:03:23<309:14:17,  6.15s/it]
 20%|█▉        | 44125/225000 [94:03:35<394:23:20,  7.85s/it]
 20%|█▉        | 44126/225000 [94:03:40<356:23:36,  7.09s/it]
 20%|█▉        | 44127/225000 [94:03:48<361:09:40,  7.19s/it]
 20%|█▉        | 44128/225000 [94:03:51<308:32:08,  6.14s/it]
 20%|█▉        | 44129/225000 [94:04:01<364:23:15,  7.25s/it]
 20%|█▉        | 44130/225000 [94:04:08<365:49:33,  7.28s/it]
 20%|█▉        | 44131/225000 [94:04:12<307:57:36,  6.13s/it]
 20%|█▉        | 44132/225000 [94:04:16<273:34:02,  5.45s/it]
 20%|█▉        | 44133/225000 [94:04:28<371:01:36,  7.38s/it]
 20%|█▉        | 44134/225000 [94:04:32<323:01:13,  6.43s/it]
 20%|█▉        | 44135/225000 [94:04:36<291:35:38,  5.80s/it]
 20%|█▉        | 44136/225000 [94:04:47<363:32:45,  7.24s/it]
 20%|█▉        | 44137/225000 [94:04:53<344:46:07,  6.86s/it]
 20%|█▉        | 44138/225000 [94:04:58<322:23:49,  6.42s/it]
 20%|█▉        | 44139/225000 [94:05:03<299:16:06,  5.96s/it]
 20%|█▉        | 44140/225000 [94:05:08<287:37:05,  5.73s/it]
 20%|█▉        | 44141/225000 [94:05:14<281:25:15,  5.60s/it]
 20%|█▉        | 44142/225000 [94:05:27<396:06:00,  7.88s/it]
 20%|█▉        | 44143/225000 [94:05:32<359:29:47,  7.16s/it]
 20%|█▉        | 44144/225000 [94:05:36<313:51:18,  6.25s/it]
 20%|█▉        | 44145/225000 [94:05:41<287:14:43,  5.72s/it]
 20%|█▉        | 44146/225000 [94:05:54<404:16:01,  8.05s/it]
 20%|█▉        | 44147/225000 [94:06:02<406:01:16,  8.08s/it]
 20%|█▉        | 44148/225000 [94:06:06<340:12:12,  6.77s/it]
 20%|█▉        | 44149/225000 [94:06:11<314:20:15,  6.26s/it]
 20%|█▉        | 44150/225000 [94:06:19<338:03:15,  6.73s/it]
                                                             
{'loss': '0.7063', 'grad_norm': '0.7779', 'learning_rate': '4.696e-05', 'epoch': '0.1962', 'num_input_tokens_seen': 158550688, 'train_runtime': '3.388e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44150/225000 [94:06:19<338:03:15,  6.73s/it]
 20%|█▉        | 44151/225000 [94:06:23<302:34:15,  6.02s/it]
 20%|█▉        | 44152/225000 [94:06:28<284:30:51,  5.66s/it]
 20%|█▉        | 44153/225000 [94:06:35<304:07:06,  6.05s/it]
 20%|█▉        | 44154/225000 [94:06:46<370:26:26,  7.37s/it]
 20%|█▉        | 44155/225000 [94:06:50<319:20:47,  6.36s/it]
 20%|█▉        | 44156/225000 [94:07:00<383:05:48,  7.63s/it]
 20%|█▉        | 44157/225000 [94:07:07<376:43:47,  7.50s/it]
 20%|█▉        | 44158/225000 [94:07:12<327:10:48,  6.51s/it]
 20%|█▉        | 44159/225000 [94:07:16<299:25:25,  5.96s/it]
 20%|█▉        | 44160/225000 [94:07:26<358:26:55,  7.14s/it]
 20%|█▉        | 44161/225000 [94:07:34<367:18:02,  7.31s/it]
 20%|█▉        | 44162/225000 [94:07:38<319:36:44,  6.36s/it]
 20%|█▉        | 44163/225000 [94:07:43<294:50:31,  5.87s/it]
 20%|█▉        | 44164/225000 [94:07:56<398:06:57,  7.93s/it]
 20%|█▉        | 44165/225000 [94:08:00<347:28:45,  6.92s/it]
 20%|█▉        | 44166/225000 [94:08:08<367:54:16,  7.32s/it]
 20%|█▉        | 44167/225000 [94:08:15<354:34:37,  7.06s/it]
 20%|█▉        | 44168/225000 [94:08:21<343:15:09,  6.83s/it]
 20%|█▉        | 44169/225000 [94:08:26<313:06:10,  6.23s/it]
 20%|█▉        | 44170/225000 [94:08:37<381:50:31,  7.60s/it]
 20%|█▉        | 44171/225000 [94:08:41<328:56:23,  6.55s/it]
 20%|█▉        | 44172/225000 [94:08:45<293:31:28,  5.84s/it]
 20%|█▉        | 44173/225000 [94:08:50<273:28:37,  5.44s/it]
 20%|█▉        | 44174/225000 [94:09:02<372:14:55,  7.41s/it]
 20%|█▉        | 44175/225000 [94:09:07<342:24:06,  6.82s/it]
 20%|█▉        | 44176/225000 [94:09:11<300:08:49,  5.98s/it]
 20%|█▉        | 44177/225000 [94:09:15<268:58:32,  5.36s/it]
 20%|█▉        | 44178/225000 [94:09:27<367:34:53,  7.32s/it]
 20%|█▉        | 44179/225000 [94:09:33<343:25:20,  6.84s/it]
 20%|█▉        | 44180/225000 [94:09:40<349:54:12,  6.97s/it]
 20%|█▉        | 44181/225000 [94:09:46<336:52:18,  6.71s/it]
 20%|█▉        | 44182/225000 [94:09:51<311:15:25,  6.20s/it]
 20%|█▉        | 44183/225000 [94:09:56<291:16:32,  5.80s/it]
 20%|█▉        | 44184/225000 [94:10:01<280:16:17,  5.58s/it]
 20%|█▉        | 44185/225000 [94:10:07<287:18:52,  5.72s/it]
 20%|█▉        | 44186/225000 [94:10:20<393:07:02,  7.83s/it]
 20%|█▉        | 44187/225000 [94:10:24<335:50:47,  6.69s/it]
 20%|█▉        | 44188/225000 [94:10:34<389:24:21,  7.75s/it]
 20%|█▉        | 44189/225000 [94:10:39<351:42:28,  7.00s/it]
 20%|█▉        | 44190/225000 [94:10:44<319:04:35,  6.35s/it]
 20%|█▉        | 44191/225000 [94:10:49<301:41:25,  6.01s/it]
 20%|█▉        | 44192/225000 [94:11:01<390:11:15,  7.77s/it]
 20%|█▉        | 44193/225000 [94:11:07<354:53:41,  7.07s/it]
 20%|█▉        | 44194/225000 [94:11:11<318:28:51,  6.34s/it]
 20%|█▉        | 44195/225000 [94:11:16<294:43:14,  5.87s/it]
 20%|█▉        | 44196/225000 [94:11:27<377:30:52,  7.52s/it]
 20%|█▉        | 44197/225000 [94:11:32<342:04:17,  6.81s/it]
 20%|█▉        | 44198/225000 [94:11:38<323:08:08,  6.43s/it]
 20%|█▉        | 44199/225000 [94:11:47<366:34:11,  7.30s/it]
 20%|█▉        | 44200/225000 [94:11:51<314:10:32,  6.26s/it]
                                                             
{'loss': '0.7121', 'grad_norm': '0.7584', 'learning_rate': '4.695e-05', 'epoch': '0.1964', 'num_input_tokens_seen': 158700384, 'train_runtime': '3.391e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44200/225000 [94:11:51<314:10:32,  6.26s/it]
 20%|█▉        | 44201/225000 [94:11:56<295:22:47,  5.88s/it]
 20%|█▉        | 44202/225000 [94:12:07<366:40:39,  7.30s/it]
 20%|█▉        | 44203/225000 [94:12:12<336:43:58,  6.70s/it]
 20%|█▉        | 44204/225000 [94:12:17<314:55:33,  6.27s/it]
 20%|█▉        | 44205/225000 [94:12:27<368:58:44,  7.35s/it]
 20%|█▉        | 44206/225000 [94:12:32<337:18:48,  6.72s/it]
 20%|█▉        | 44207/225000 [94:12:38<317:19:29,  6.32s/it]
 20%|█▉        | 44208/225000 [94:12:43<297:50:18,  5.93s/it]
 20%|█▉        | 44209/225000 [94:12:48<289:05:49,  5.76s/it]
 20%|█▉        | 44210/225000 [94:12:53<274:47:51,  5.47s/it]
 20%|█▉        | 44211/225000 [94:12:58<272:33:10,  5.43s/it]
 20%|█▉        | 44212/225000 [94:13:09<351:32:05,  7.00s/it]
 20%|█▉        | 44213/225000 [94:13:18<378:42:46,  7.54s/it]
 20%|█▉        | 44214/225000 [94:13:25<369:25:57,  7.36s/it]
 20%|█▉        | 44215/225000 [94:13:29<327:32:32,  6.52s/it]
 20%|█▉        | 44216/225000 [94:13:34<301:16:26,  6.00s/it]
 20%|█▉        | 44217/225000 [94:13:43<342:14:20,  6.82s/it]
 20%|█▉        | 44218/225000 [94:13:51<363:18:56,  7.23s/it]
 20%|█▉        | 44219/225000 [94:13:59<376:49:38,  7.50s/it]
 20%|█▉        | 44220/225000 [94:14:04<342:09:43,  6.81s/it]
 20%|█▉        | 44221/225000 [94:14:10<330:16:43,  6.58s/it]
 20%|█▉        | 44222/225000 [94:14:16<322:32:25,  6.42s/it]
 20%|█▉        | 44223/225000 [94:14:22<313:34:15,  6.24s/it]
 20%|█▉        | 44224/225000 [94:14:28<305:51:57,  6.09s/it]
 20%|█▉        | 44225/225000 [94:14:34<300:56:39,  5.99s/it]
 20%|█▉        | 44226/225000 [94:14:39<286:36:50,  5.71s/it]
 20%|█▉        | 44227/225000 [94:14:51<387:32:59,  7.72s/it]
 20%|█▉        | 44228/225000 [94:14:58<375:12:11,  7.47s/it]
 20%|█▉        | 44229/225000 [94:15:03<328:47:46,  6.55s/it]
 20%|█▉        | 44230/225000 [94:15:12<368:41:27,  7.34s/it]
 20%|█▉        | 44231/225000 [94:15:21<395:03:58,  7.87s/it]
 20%|█▉        | 44232/225000 [94:15:29<398:22:03,  7.93s/it]
 20%|█▉        | 44233/225000 [94:15:33<339:13:06,  6.76s/it]
 20%|█▉        | 44234/225000 [94:15:41<353:56:08,  7.05s/it]
 20%|█▉        | 44235/225000 [94:15:49<379:07:14,  7.55s/it]
 20%|█▉        | 44236/225000 [94:15:53<326:16:37,  6.50s/it]
 20%|█▉        | 44237/225000 [94:16:01<344:56:21,  6.87s/it]
 20%|█▉        | 44238/225000 [94:16:07<334:05:15,  6.65s/it]
 20%|█▉        | 44239/225000 [94:16:13<320:50:53,  6.39s/it]
 20%|█▉        | 44240/225000 [94:16:17<285:38:09,  5.69s/it]
 20%|█▉        | 44241/225000 [94:16:29<376:03:27,  7.49s/it]
 20%|█▉        | 44242/225000 [94:16:35<359:39:30,  7.16s/it]
 20%|█▉        | 44243/225000 [94:16:41<331:59:07,  6.61s/it]
 20%|█▉        | 44244/225000 [94:16:45<304:44:37,  6.07s/it]
 20%|█▉        | 44245/225000 [94:16:55<356:45:37,  7.11s/it]
 20%|█▉        | 44246/225000 [94:16:59<312:46:53,  6.23s/it]
 20%|█▉        | 44247/225000 [94:17:04<296:41:07,  5.91s/it]
 20%|█▉        | 44248/225000 [94:17:10<288:16:31,  5.74s/it]
 20%|█▉        | 44249/225000 [94:17:21<373:20:53,  7.44s/it]
 20%|█▉        | 44250/225000 [94:17:25<327:51:35,  6.53s/it]
                                                             
{'loss': '0.6947', 'grad_norm': '1.082', 'learning_rate': '4.694e-05', 'epoch': '0.1967', 'num_input_tokens_seen': 158850616, 'train_runtime': '3.394e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44250/225000 [94:17:25<327:51:35,  6.53s/it]
 20%|█▉        | 44251/225000 [94:17:34<355:26:59,  7.08s/it]
 20%|█▉        | 44252/225000 [94:17:40<340:38:28,  6.78s/it]
 20%|█▉        | 44253/225000 [94:17:49<375:30:38,  7.48s/it]
 20%|█▉        | 44254/225000 [94:17:53<319:06:04,  6.36s/it]
 20%|█▉        | 44255/225000 [94:18:02<369:06:06,  7.35s/it]
 20%|█▉        | 44256/225000 [94:18:10<375:13:49,  7.47s/it]
 20%|█▉        | 44257/225000 [94:18:14<322:50:31,  6.43s/it]
 20%|█▉        | 44258/225000 [94:18:18<292:30:30,  5.83s/it]
 20%|█▉        | 44259/225000 [94:18:25<301:51:24,  6.01s/it]
 20%|█▉        | 44260/225000 [94:18:36<380:46:22,  7.58s/it]
 20%|█▉        | 44261/225000 [94:18:40<324:10:13,  6.46s/it]
 20%|█▉        | 44262/225000 [94:18:49<367:21:44,  7.32s/it]
 20%|█▉        | 44263/225000 [94:18:56<353:20:52,  7.04s/it]
 20%|█▉        | 44264/225000 [94:19:00<308:45:58,  6.15s/it]
 20%|█▉        | 44265/225000 [94:19:10<366:34:15,  7.30s/it]
 20%|█▉        | 44266/225000 [94:19:14<316:09:37,  6.30s/it]
 20%|█▉        | 44267/225000 [94:19:20<321:45:28,  6.41s/it]
 20%|█▉        | 44268/225000 [94:19:28<337:36:19,  6.72s/it]
 20%|█▉        | 44269/225000 [94:19:34<321:07:10,  6.40s/it]
 20%|█▉        | 44270/225000 [94:19:39<311:06:12,  6.20s/it]
 20%|█▉        | 44271/225000 [94:19:44<294:07:37,  5.86s/it]
 20%|█▉        | 44272/225000 [94:19:50<290:53:31,  5.79s/it]
 20%|█▉        | 44273/225000 [94:19:55<282:32:09,  5.63s/it]
 20%|█▉        | 44274/225000 [94:20:01<279:49:17,  5.57s/it]
 20%|█▉        | 44275/225000 [94:20:08<309:20:55,  6.16s/it]
 20%|█▉        | 44276/225000 [94:20:21<416:30:44,  8.30s/it]
 20%|█▉        | 44277/225000 [94:20:26<363:05:01,  7.23s/it]
 20%|█▉        | 44278/225000 [94:20:36<396:33:25,  7.90s/it]
 20%|█▉        | 44279/225000 [94:20:39<332:19:55,  6.62s/it]
 20%|█▉        | 44280/225000 [94:20:48<368:36:19,  7.34s/it]
 20%|█▉        | 44281/225000 [94:20:52<314:54:29,  6.27s/it]
 20%|█▉        | 44282/225000 [94:20:59<327:37:26,  6.53s/it]
 20%|█▉        | 44283/225000 [94:21:08<357:15:20,  7.12s/it]
 20%|█▉        | 44284/225000 [94:21:13<322:41:56,  6.43s/it]
 20%|█▉        | 44285/225000 [94:21:18<300:54:56,  5.99s/it]
 20%|█▉        | 44286/225000 [94:21:22<285:25:12,  5.69s/it]
 20%|█▉        | 44287/225000 [94:21:36<395:51:56,  7.89s/it]
 20%|█▉        | 44288/225000 [94:21:41<364:08:11,  7.25s/it]
 20%|█▉        | 44289/225000 [94:21:45<313:18:15,  6.24s/it]
 20%|█▉        | 44290/225000 [94:21:56<378:43:02,  7.54s/it]
 20%|█▉        | 44291/225000 [94:22:01<347:48:50,  6.93s/it]
 20%|█▉        | 44292/225000 [94:22:06<310:16:54,  6.18s/it]
 20%|█▉        | 44293/225000 [94:22:11<294:39:00,  5.87s/it]
 20%|█▉        | 44294/225000 [94:22:16<282:46:28,  5.63s/it]
 20%|█▉        | 44295/225000 [94:22:28<376:35:43,  7.50s/it]
 20%|█▉        | 44296/225000 [94:22:36<390:15:33,  7.77s/it]
 20%|█▉        | 44297/225000 [94:22:42<362:22:37,  7.22s/it]
 20%|█▉        | 44298/225000 [94:22:47<330:34:37,  6.59s/it]
 20%|█▉        | 44299/225000 [94:22:53<311:41:54,  6.21s/it]
 20%|█▉        | 44300/225000 [94:22:57<289:56:31,  5.78s/it]
                                                             
{'loss': '0.6974', 'grad_norm': '0.6702', 'learning_rate': '4.693e-05', 'epoch': '0.1969', 'num_input_tokens_seen': 159002032, 'train_runtime': '3.398e+05', 'train_tokens_per_second': '468'}

 20%|█▉        | 44300/225000 [94:22:57<289:56:31,  5.78s/it]
 20%|█▉        | 44301/225000 [94:23:10<398:06:57,  7.93s/it]
 20%|█▉        | 44302/225000 [94:23:14<336:39:59,  6.71s/it]
 20%|█▉        | 44303/225000 [94:23:23<367:20:41,  7.32s/it]
 20%|█▉        | 44304/225000 [94:23:27<321:05:23,  6.40s/it]
 20%|█▉        | 44305/225000 [94:23:33<310:11:40,  6.18s/it]
 20%|█▉        | 44306/225000 [94:23:41<345:29:29,  6.88s/it]
 20%|█▉        | 44307/225000 [94:23:50<376:22:11,  7.50s/it]
 20%|█▉        | 44308/225000 [94:23:56<349:49:19,  6.97s/it]
 20%|█▉        | 44309/225000 [94:24:05<376:21:23,  7.50s/it]
 20%|█▉        | 44310/225000 [94:24:10<349:42:56,  6.97s/it]
 20%|█▉        | 44311/225000 [94:24:16<333:57:29,  6.65s/it]
 20%|█▉        | 44312/225000 [94:24:22<311:24:43,  6.20s/it]
 20%|█▉        | 44313/225000 [94:24:27<306:40:21,  6.11s/it]
 20%|█▉        | 44314/225000 [94:24:32<290:18:25,  5.78s/it]
 20%|█▉        | 44315/225000 [94:24:38<289:56:50,  5.78s/it]
 20%|█▉        | 44316/225000 [94:24:49<372:23:34,  7.42s/it]
 20%|█▉        | 44317/225000 [94:24:58<386:44:55,  7.71s/it]
 20%|█▉        | 44318/225000 [94:25:06<386:57:23,  7.71s/it]
 20%|█▉        | 44319/225000 [94:25:10<332:07:38,  6.62s/it]
 20%|█▉        | 44320/225000 [94:25:14<302:58:55,  6.04s/it]
 20%|█▉        | 44321/225000 [94:25:19<286:02:41,  5.70s/it]
 20%|█▉        | 44322/225000 [94:25:26<303:13:11,  6.04s/it]
 20%|█▉        | 44323/225000 [94:25:40<415:07:15,  8.27s/it]
 20%|█▉        | 44324/225000 [94:25:44<353:11:42,  7.04s/it]
 20%|█▉        | 44325/225000 [94:25:53<384:08:12,  7.65s/it]
 20%|█▉        | 44326/225000 [94:25:57<333:03:21,  6.64s/it]
 20%|█▉        | 44327/225000 [94:26:01<298:45:35,  5.95s/it]
 20%|█▉        | 44328/225000 [94:26:07<288:12:22,  5.74s/it]
 20%|█▉        | 44329/225000 [94:26:12<284:38:52,  5.67s/it]
 20%|█▉        | 44330/225000 [94:26:17<277:52:09,  5.54s/it]
 20%|█▉        | 44331/225000 [94:26:30<389:19:37,  7.76s/it]
 20%|█▉        | 44332/225000 [94:26:37<378:38:12,  7.54s/it]
 20%|█▉        | 44333/225000 [94:26:46<399:45:45,  7.97s/it]
 20%|█▉        | 44334/225000 [94:26:50<342:45:24,  6.83s/it]
 20%|█▉        | 44335/225000 [94:26:55<306:59:20,  6.12s/it]
 20%|█▉        | 44336/225000 [94:27:03<336:11:09,  6.70s/it]
 20%|█▉        | 44337/225000 [94:27:12<378:20:35,  7.54s/it]
 20%|█▉        | 44338/225000 [94:27:18<343:36:44,  6.85s/it]
 20%|█▉        | 44339/225000 [94:27:24<330:34:14,  6.59s/it]
 20%|█▉        | 44340/225000 [94:27:29<310:15:35,  6.18s/it]
 20%|█▉        | 44341/225000 [94:27:35<307:32:45,  6.13s/it]
 20%|█▉        | 44342/225000 [94:27:40<298:14:20,  5.94s/it]
 20%|█▉        | 44343/225000 [94:27:46<296:30:10,  5.91s/it]
 20%|█▉        | 44344/225000 [94:28:00<407:03:08,  8.11s/it]
 20%|█▉        | 44345/225000 [94:28:05<368:26:42,  7.34s/it]
 20%|█▉        | 44346/225000 [94:28:11<352:33:25,  7.03s/it]
 20%|█▉        | 44347/225000 [94:28:17<335:09:24,  6.68s/it]
 20%|█▉        | 44348/225000 [94:28:23<318:44:11,  6.35s/it]
 20%|█▉        | 44349/225000 [94:28:28<307:08:55,  6.12s/it]
 20%|█▉        | 44350/225000 [94:28:41<411:18:33,  8.20s/it]
                                                             
{'loss': '0.6837', 'grad_norm': '0.6404', 'learning_rate': '4.692e-05', 'epoch': '0.1971', 'num_input_tokens_seen': 159154120, 'train_runtime': '3.401e+05', 'train_tokens_per_second': '467.9'}

 20%|█▉        | 44350/225000 [94:28:41<411:18:33,  8.20s/it]
 20%|█▉        | 44351/225000 [94:28:50<420:50:20,  8.39s/it]
 20%|█▉        | 44352/225000 [94:28:54<354:16:50,  7.06s/it]
 20%|█▉        | 44353/225000 [94:29:04<397:04:13,  7.91s/it]
 20%|█▉        | 44354/225000 [94:29:08<335:14:22,  6.68s/it]
 20%|█▉        | 44355/225000 [94:29:12<290:53:14,  5.80s/it]
 20%|█▉        | 44356/225000 [94:29:22<356:26:37,  7.10s/it]
 20%|█▉        | 44357/225000 [94:29:29<356:52:31,  7.11s/it]
 20%|█▉        | 44358/225000 [94:29:35<336:27:52,  6.71s/it]
 20%|█▉        | 44359/225000 [94:29:43<366:33:41,  7.31s/it]
 20%|█▉        | 44360/225000 [94:29:47<317:44:07,  6.33s/it]
 20%|█▉        | 44361/225000 [94:29:56<351:00:44,  7.00s/it]
 20%|█▉        | 44362/225000 [94:30:01<323:35:31,  6.45s/it]
 20%|█▉        | 44363/225000 [94:30:07<308:10:34,  6.14s/it]
 20%|█▉        | 44364/225000 [94:30:12<295:07:01,  5.88s/it]
 20%|█▉        | 44365/225000 [94:30:24<381:52:21,  7.61s/it]
 20%|█▉        | 44366/225000 [94:30:28<334:59:22,  6.68s/it]
 20%|█▉        | 44367/225000 [94:30:32<299:17:43,  5.96s/it]
 20%|█▉        | 44368/225000 [94:30:37<278:33:59,  5.55s/it]
 20%|█▉        | 44369/225000 [94:30:51<402:37:01,  8.02s/it]
 20%|█▉        | 44370/225000 [94:30:55<343:10:34,  6.84s/it]
 20%|█▉        | 44371/225000 [94:30:59<304:54:10,  6.08s/it]
 20%|█▉        | 44372/225000 [94:31:09<358:16:10,  7.14s/it]
 20%|█▉        | 44373/225000 [94:31:15<349:02:38,  6.96s/it]
 20%|█▉        | 44374/225000 [94:31:22<351:37:24,  7.01s/it]
 20%|█▉        | 44375/225000 [94:31:26<306:51:24,  6.12s/it]
 20%|█▉        | 44376/225000 [94:31:35<337:43:06,  6.73s/it]
 20%|█▉        | 44377/225000 [94:31:39<306:38:31,  6.11s/it]
 20%|█▉        | 44378/225000 [94:31:44<279:35:28,  5.57s/it]
 20%|█▉        | 44379/225000 [94:31:54<357:05:47,  7.12s/it]
 20%|█▉        | 44380/225000 [94:32:01<347:12:47,  6.92s/it]
 20%|█▉        | 44381/225000 [94:32:09<364:12:42,  7.26s/it]
 20%|█▉        | 44382/225000 [94:32:13<311:11:02,  6.20s/it]
 20%|█▉        | 44383/225000 [94:32:20<332:48:04,  6.63s/it]
 20%|█▉        | 44384/225000 [94:32:28<349:51:01,  6.97s/it]
 20%|█▉        | 44385/225000 [94:32:36<367:38:25,  7.33s/it]
 20%|█▉        | 44386/225000 [94:32:41<325:31:24,  6.49s/it]
 20%|█▉        | 44387/225000 [94:32:45<295:41:41,  5.89s/it]
 20%|█▉        | 44388/225000 [94:32:51<293:08:46,  5.84s/it]
 20%|█▉        | 44389/225000 [94:33:03<383:25:24,  7.64s/it]
 20%|█▉        | 44390/225000 [94:33:07<336:17:11,  6.70s/it]
 20%|█▉        | 44391/225000 [94:33:12<310:33:24,  6.19s/it]
 20%|█▉        | 44392/225000 [94:33:17<296:33:59,  5.91s/it]
 20%|█▉        | 44393/225000 [94:33:23<291:14:42,  5.81s/it]
 20%|█▉        | 44394/225000 [94:33:29<286:42:29,  5.71s/it]
 20%|█▉        | 44395/225000 [94:33:34<276:45:10,  5.52s/it]
 20%|█▉        | 44396/225000 [94:33:39<273:14:27,  5.45s/it]
 20%|█▉        | 44397/225000 [94:33:52<389:09:09,  7.76s/it]
 20%|█▉        | 44398/225000 [94:33:59<377:26:39,  7.52s/it]
 20%|█▉        | 44399/225000 [94:34:08<399:06:52,  7.96s/it]
 20%|█▉        | 44400/225000 [94:34:15<384:09:21,  7.66s/it]
                                                             
{'loss': '0.7007', 'grad_norm': '0.9215', 'learning_rate': '4.691e-05', 'epoch': '0.1973', 'num_input_tokens_seen': 159303128, 'train_runtime': '3.405e+05', 'train_tokens_per_second': '467.9'}

 20%|█▉        | 44400/225000 [94:34:15<384:09:21,  7.66s/it]
 20%|█▉        | 44401/225000 [94:34:19<331:16:45,  6.60s/it]
 20%|█▉        | 44402/225000 [94:34:23<294:44:14,  5.88s/it]
 20%|█▉        | 44403/225000 [94:34:32<332:17:08,  6.62s/it]
 20%|█▉        | 44404/225000 [94:34:42<387:12:27,  7.72s/it]
 20%|█▉        | 44405/225000 [94:34:49<372:01:56,  7.42s/it]
 20%|█▉        | 44406/225000 [94:34:54<346:06:58,  6.90s/it]
 20%|█▉        | 44407/225000 [94:34:59<319:42:52,  6.37s/it]
 20%|█▉        | 44408/225000 [94:35:04<298:29:23,  5.95s/it]
 20%|█▉        | 44409/225000 [94:35:16<387:44:41,  7.73s/it]
 20%|█▉        | 44410/225000 [94:35:22<357:28:20,  7.13s/it]
 20%|█▉        | 44411/225000 [94:35:26<310:44:27,  6.19s/it]
 20%|█▉        | 44412/225000 [94:35:31<289:09:40,  5.76s/it]
 20%|█▉        | 44413/225000 [94:35:41<362:38:42,  7.23s/it]
 20%|█▉        | 44414/225000 [94:35:49<373:06:33,  7.44s/it]
 20%|█▉        | 44415/225000 [94:35:53<323:19:33,  6.45s/it]
 20%|█▉        | 44416/225000 [94:36:03<375:07:59,  7.48s/it]
 20%|█▉        | 44417/225000 [94:36:08<335:34:32,  6.69s/it]
 20%|█▉        | 44418/225000 [94:36:16<355:54:53,  7.10s/it]
 20%|█▉        | 44419/225000 [94:36:22<342:33:19,  6.83s/it]
 20%|█▉        | 44420/225000 [94:36:27<315:05:11,  6.28s/it]
 20%|█▉        | 44421/225000 [94:36:32<293:53:38,  5.86s/it]
 20%|█▉        | 44422/225000 [94:36:37<278:05:50,  5.54s/it]
 20%|█▉        | 44423/225000 [94:36:49<378:25:21,  7.54s/it]
 20%|█▉        | 44424/225000 [94:36:58<397:12:13,  7.92s/it]
 20%|█▉        | 44425/225000 [94:37:02<342:25:00,  6.83s/it]
 20%|█▉        | 44426/225000 [94:37:07<301:34:47,  6.01s/it]
 20%|█▉        | 44427/225000 [94:37:12<286:21:59,  5.71s/it]
 20%|█▉        | 44428/225000 [94:37:22<364:05:38,  7.26s/it]
 20%|█▉        | 44429/225000 [94:37:30<362:34:53,  7.23s/it]
 20%|█▉        | 44430/225000 [94:37:34<317:02:02,  6.32s/it]
 20%|█▉        | 44431/225000 [94:37:41<337:55:39,  6.74s/it]
 20%|█▉        | 44432/225000 [94:37:50<367:42:01,  7.33s/it]
 20%|█▉        | 44433/225000 [94:37:57<363:54:55,  7.26s/it]
 20%|█▉        | 44434/225000 [94:38:03<346:02:38,  6.90s/it]
 20%|█▉        | 44435/225000 [94:38:08<313:20:26,  6.25s/it]
 20%|█▉        | 44436/225000 [94:38:13<293:33:48,  5.85s/it]
 20%|█▉        | 44437/225000 [94:38:20<312:18:46,  6.23s/it]
 20%|█▉        | 44438/225000 [94:38:33<417:42:00,  8.33s/it]
 20%|█▉        | 44439/225000 [94:38:37<348:24:26,  6.95s/it]
 20%|█▉        | 44440/225000 [94:38:47<387:24:23,  7.72s/it]
 20%|█▉        | 44441/225000 [94:38:53<360:26:59,  7.19s/it]
 20%|█▉        | 44442/225000 [94:38:59<342:42:58,  6.83s/it]
 20%|█▉        | 44443/225000 [94:39:08<381:33:52,  7.61s/it]
 20%|█▉        | 44444/225000 [94:39:12<329:02:41,  6.56s/it]
 20%|█▉        | 44445/225000 [94:39:17<298:37:45,  5.95s/it]
 20%|█▉        | 44446/225000 [94:39:26<350:04:47,  6.98s/it]
 20%|█▉        | 44447/225000 [94:39:33<350:55:28,  7.00s/it]
 20%|█▉        | 44448/225000 [94:39:40<348:07:47,  6.94s/it]
 20%|█▉        | 44449/225000 [94:39:45<314:44:12,  6.28s/it]
 20%|█▉        | 44450/225000 [94:39:53<353:01:45,  7.04s/it]
                                                             
{'loss': '0.7028', 'grad_norm': '0.7449', 'learning_rate': '4.69e-05', 'epoch': '0.1976', 'num_input_tokens_seen': 159454408, 'train_runtime': '3.408e+05', 'train_tokens_per_second': '467.9'}

 20%|█▉        | 44450/225000 [94:39:53<353:01:45,  7.04s/it]
 20%|█▉        | 44451/225000 [94:40:01<354:35:18,  7.07s/it]
 20%|█▉        | 44452/225000 [94:40:09<370:06:57,  7.38s/it]
 20%|█▉        | 44453/225000 [94:40:13<321:52:43,  6.42s/it]
 20%|█▉        | 44454/225000 [94:40:18<297:33:29,  5.93s/it]
 20%|█▉        | 44455/225000 [94:40:22<279:27:52,  5.57s/it]
 20%|█▉        | 44456/225000 [94:40:35<390:46:41,  7.79s/it]
 20%|█▉        | 44457/225000 [94:40:41<362:41:00,  7.23s/it]
 20%|█▉        | 44458/225000 [94:40:50<383:18:52,  7.64s/it]
 20%|█▉        | 44459/225000 [94:40:55<345:55:37,  6.90s/it]
 20%|█▉        | 44460/225000 [94:41:00<317:59:24,  6.34s/it]
 20%|█▉        | 44461/225000 [94:41:06<307:05:19,  6.12s/it]
 20%|█▉        | 44462/225000 [94:41:11<298:36:01,  5.95s/it]
 20%|█▉        | 44463/225000 [94:41:16<286:14:33,  5.71s/it]
 20%|█▉        | 44464/225000 [94:41:28<379:28:55,  7.57s/it]
 20%|█▉        | 44465/225000 [94:41:37<399:43:51,  7.97s/it]
 20%|█▉        | 44466/225000 [94:41:44<385:12:04,  7.68s/it]
 20%|█▉        | 44467/225000 [94:41:51<373:36:46,  7.45s/it]
 20%|█▉        | 44468/225000 [94:41:56<334:32:47,  6.67s/it]
 20%|█▉        | 44469/225000 [94:42:01<304:31:21,  6.07s/it]
 20%|█▉        | 44470/225000 [94:42:08<319:26:00,  6.37s/it]
 20%|█▉        | 44471/225000 [94:42:20<410:44:44,  8.19s/it]
 20%|█▉        | 44472/225000 [94:42:24<349:53:02,  6.98s/it]
 20%|█▉        | 44473/225000 [94:42:30<329:31:11,  6.57s/it]
 20%|█▉        | 44474/225000 [94:42:42<418:40:29,  8.35s/it]
 20%|█▉        | 44475/225000 [94:42:46<354:46:06,  7.07s/it]
 20%|█▉        | 44476/225000 [94:42:53<345:50:13,  6.90s/it]
 20%|█▉        | 44477/225000 [94:43:02<383:26:29,  7.65s/it]
 20%|█▉        | 44478/225000 [94:43:08<346:14:04,  6.90s/it]
 20%|█▉        | 44479/225000 [94:43:14<346:34:30,  6.91s/it]
 20%|█▉        | 44480/225000 [94:43:19<316:49:06,  6.32s/it]
 20%|█▉        | 44481/225000 [94:43:27<328:53:57,  6.56s/it]
 20%|█▉        | 44482/225000 [94:43:32<309:06:17,  6.16s/it]
 20%|█▉        | 44483/225000 [94:43:38<305:31:21,  6.09s/it]
 20%|█▉        | 44484/225000 [94:43:43<300:48:14,  6.00s/it]
 20%|█▉        | 44485/225000 [94:43:49<298:54:02,  5.96s/it]
 20%|█▉        | 44486/225000 [94:43:55<291:52:59,  5.82s/it]
 20%|█▉        | 44487/225000 [94:44:06<372:33:43,  7.43s/it]
 20%|█▉        | 44488/225000 [94:44:13<366:04:14,  7.30s/it]
 20%|█▉        | 44489/225000 [94:44:24<423:25:51,  8.44s/it]
 20%|█▉        | 44490/225000 [94:44:28<360:58:53,  7.20s/it]
 20%|█▉        | 44491/225000 [94:44:36<370:48:15,  7.40s/it]
 20%|█▉        | 44492/225000 [94:44:40<317:14:20,  6.33s/it]
 20%|█▉        | 44493/225000 [94:44:49<349:08:49,  6.96s/it]
 20%|█▉        | 44494/225000 [94:44:55<335:52:35,  6.70s/it]
 20%|█▉        | 44495/225000 [94:45:05<387:37:27,  7.73s/it]
 20%|█▉        | 44496/225000 [94:45:09<328:49:18,  6.56s/it]
 20%|█▉        | 44497/225000 [94:45:17<349:24:05,  6.97s/it]
 20%|█▉        | 44498/225000 [94:45:21<314:10:47,  6.27s/it]
 20%|█▉        | 44499/225000 [94:45:32<387:04:58,  7.72s/it]
 20%|█▉        | 44500/225000 [94:45:37<346:20:16,  6.91s/it]
                                                             
{'loss': '0.694', 'grad_norm': '0.8209', 'learning_rate': '4.689e-05', 'epoch': '0.1978', 'num_input_tokens_seen': 159606416, 'train_runtime': '3.411e+05', 'train_tokens_per_second': '467.9'}

 20%|█▉        | 44500/225000 [94:45:37<346:20:16,  6.91s/it]
 20%|█▉        | 44501/225000 [94:45:42<312:18:22,  6.23s/it]
 20%|█▉        | 44502/225000 [94:45:46<286:32:00,  5.71s/it]
 20%|█▉        | 44503/225000 [94:45:59<395:41:05,  7.89s/it]
 20%|█▉        | 44504/225000 [94:46:04<349:06:30,  6.96s/it]
 20%|█▉        | 44505/225000 [94:46:11<339:58:02,  6.78s/it]
 20%|█▉        | 44506/225000 [94:46:17<340:46:19,  6.80s/it]
 20%|█▉        | 44507/225000 [94:46:23<319:54:39,  6.38s/it]
 20%|█▉        | 44508/225000 [94:46:29<314:02:01,  6.26s/it]
 20%|█▉        | 44509/225000 [94:46:34<301:56:40,  6.02s/it]
 20%|█▉        | 44510/225000 [94:46:40<296:03:42,  5.91s/it]
 20%|█▉        | 44511/225000 [94:46:49<345:21:40,  6.89s/it]
 20%|█▉        | 44512/225000 [94:46:57<359:05:29,  7.16s/it]
 20%|█▉        | 44513/225000 [94:47:02<331:59:23,  6.62s/it]
 20%|█▉        | 44514/225000 [94:47:09<326:57:12,  6.52s/it]
 20%|█▉        | 44515/225000 [94:47:17<351:50:31,  7.02s/it]
 20%|█▉        | 44516/225000 [94:47:24<352:56:49,  7.04s/it]
 20%|█▉        | 44517/225000 [94:47:30<334:09:16,  6.67s/it]
 20%|█▉        | 44518/225000 [94:47:35<319:32:30,  6.37s/it]
 20%|█▉        | 44519/225000 [94:47:41<307:07:22,  6.13s/it]
 20%|█▉        | 44520/225000 [94:47:46<298:09:26,  5.95s/it]
 20%|█▉        | 44521/225000 [94:47:51<286:06:06,  5.71s/it]
 20%|█▉        | 44522/225000 [94:47:57<290:17:55,  5.79s/it]
 20%|█▉        | 44523/225000 [94:48:04<294:43:18,  5.88s/it]
 20%|█▉        | 44524/225000 [94:48:09<293:11:07,  5.85s/it]
 20%|█▉        | 44525/225000 [94:48:18<342:53:33,  6.84s/it]
 20%|█▉        | 44526/225000 [94:48:31<426:20:52,  8.50s/it]
 20%|█▉        | 44527/225000 [94:48:35<354:43:54,  7.08s/it]
 20%|█▉        | 44528/225000 [94:48:43<381:04:44,  7.60s/it]
 20%|█▉        | 44529/225000 [94:48:51<385:35:35,  7.69s/it]
 20%|█▉        | 44530/225000 [94:48:57<349:09:57,  6.97s/it]
 20%|█▉        | 44531/225000 [94:49:06<390:04:35,  7.78s/it]
 20%|█▉        | 44532/225000 [94:49:10<330:02:55,  6.58s/it]
 20%|█▉        | 44533/225000 [94:49:18<353:51:20,  7.06s/it]
 20%|█▉        | 44534/225000 [94:49:24<335:54:45,  6.70s/it]
 20%|█▉        | 44535/225000 [94:49:31<344:03:24,  6.86s/it]
 20%|█▉        | 44536/225000 [94:49:36<303:19:07,  6.05s/it]
 20%|█▉        | 44537/225000 [94:49:40<283:21:04,  5.65s/it]
 20%|█▉        | 44538/225000 [94:49:52<369:15:13,  7.37s/it]
 20%|█▉        | 44539/225000 [94:49:56<322:24:55,  6.43s/it]
 20%|█▉        | 44540/225000 [94:50:01<298:48:16,  5.96s/it]
 20%|█▉        | 44541/225000 [94:50:06<287:15:14,  5.73s/it]
 20%|█▉        | 44542/225000 [94:50:11<278:17:08,  5.55s/it]
 20%|█▉        | 44543/225000 [94:50:23<368:22:06,  7.35s/it]
 20%|█▉        | 44544/225000 [94:50:32<405:03:52,  8.08s/it]
 20%|█▉        | 44545/225000 [94:50:37<353:39:59,  7.06s/it]
 20%|█▉        | 44546/225000 [94:50:44<354:32:26,  7.07s/it]
 20%|█▉        | 44547/225000 [94:50:49<323:22:16,  6.45s/it]
 20%|█▉        | 44548/225000 [94:50:55<314:46:58,  6.28s/it]
 20%|█▉        | 44549/225000 [94:51:01<303:10:48,  6.05s/it]
 20%|█▉        | 44550/225000 [94:51:06<293:14:08,  5.85s/it]
                                                             
{'loss': '0.6992', 'grad_norm': '0.7268', 'learning_rate': '4.689e-05', 'epoch': '0.198', 'num_input_tokens_seen': 159757088, 'train_runtime': '3.415e+05', 'train_tokens_per_second': '467.9'}

 20%|█▉        | 44550/225000 [94:51:06<293:14:08,  5.85s/it]
 20%|█▉        | 44551/225000 [94:51:12<290:35:54,  5.80s/it]
 20%|█▉        | 44552/225000 [94:51:26<416:10:46,  8.30s/it]
 20%|█▉        | 44553/225000 [94:51:33<393:36:09,  7.85s/it]
 20%|█▉        | 44554/225000 [94:51:38<351:18:44,  7.01s/it]
 20%|█▉        | 44555/225000 [94:51:43<333:39:24,  6.66s/it]
 20%|█▉        | 44556/225000 [94:51:49<314:31:11,  6.27s/it]
 20%|█▉        | 44557/225000 [94:51:55<317:46:19,  6.34s/it]
 20%|█▉        | 44558/225000 [94:52:00<298:00:30,  5.95s/it]
 20%|█▉        | 44559/225000 [94:52:06<290:05:48,  5.79s/it]
 20%|█▉        | 44560/225000 [94:52:11<286:22:51,  5.71s/it]
 20%|█▉        | 44561/225000 [94:52:24<390:08:48,  7.78s/it]
 20%|█▉        | 44562/225000 [94:52:33<408:11:53,  8.14s/it]
 20%|█▉        | 44563/225000 [94:52:39<379:37:14,  7.57s/it]
 20%|█▉        | 44564/225000 [94:52:49<412:14:12,  8.22s/it]
 20%|█▉        | 44565/225000 [94:52:57<405:16:41,  8.09s/it]
 20%|█▉        | 44566/225000 [94:53:02<364:45:27,  7.28s/it]
 20%|█▉        | 44567/225000 [94:53:07<323:22:28,  6.45s/it]
 20%|█▉        | 44568/225000 [94:53:11<297:13:27,  5.93s/it]
 20%|█▉        | 44569/225000 [94:53:21<347:02:16,  6.92s/it]
 20%|█▉        | 44570/225000 [94:53:27<339:22:10,  6.77s/it]
 20%|█▉        | 44571/225000 [94:53:34<350:05:30,  6.99s/it]
 20%|█▉        | 44572/225000 [94:53:41<347:02:40,  6.92s/it]
 20%|█▉        | 44573/225000 [94:53:45<304:10:34,  6.07s/it]
 20%|█▉        | 44574/225000 [94:53:54<338:32:11,  6.75s/it]
 20%|█▉        | 44575/225000 [94:53:58<307:48:04,  6.14s/it]
 20%|█▉        | 44576/225000 [94:54:02<276:27:41,  5.52s/it]
 20%|█▉        | 44577/225000 [94:54:15<389:47:28,  7.78s/it]
 20%|█▉        | 44578/225000 [94:54:19<329:56:10,  6.58s/it]
 20%|█▉        | 44579/225000 [94:54:28<369:48:32,  7.38s/it]
 20%|█▉        | 44580/225000 [94:54:34<342:30:02,  6.83s/it]
 20%|█▉        | 44581/225000 [94:54:42<353:18:00,  7.05s/it]
 20%|█▉        | 44582/225000 [94:54:46<319:07:39,  6.37s/it]
 20%|█▉        | 44583/225000 [94:54:51<289:58:41,  5.79s/it]
 20%|█▉        | 44584/225000 [94:54:55<270:36:37,  5.40s/it]
 20%|█▉        | 44585/225000 [94:55:03<307:31:59,  6.14s/it]
 20%|█▉        | 44586/225000 [94:55:15<398:16:15,  7.95s/it]
 20%|█▉        | 44587/225000 [94:55:19<331:28:24,  6.61s/it]
 20%|█▉        | 44588/225000 [94:55:24<314:24:51,  6.27s/it]
 20%|█▉        | 44589/225000 [94:55:29<297:17:22,  5.93s/it]
 20%|█▉        | 44590/225000 [94:55:35<297:25:14,  5.93s/it]
 20%|█▉        | 44591/225000 [94:55:41<290:54:15,  5.80s/it]
 20%|█▉        | 44592/225000 [94:55:49<323:26:27,  6.45s/it]
 20%|█▉        | 44593/225000 [94:55:59<385:21:35,  7.69s/it]
 20%|█▉        | 44594/225000 [94:56:04<335:45:39,  6.70s/it]
 20%|█▉        | 44595/225000 [94:56:08<296:09:44,  5.91s/it]
 20%|█▉        | 44596/225000 [94:56:14<297:10:35,  5.93s/it]
 20%|█▉        | 44597/225000 [94:56:27<406:32:53,  8.11s/it]
 20%|█▉        | 44598/225000 [94:56:35<397:14:19,  7.93s/it]
 20%|█▉        | 44599/225000 [94:56:40<354:21:52,  7.07s/it]
 20%|█▉        | 44600/225000 [94:56:45<324:07:18,  6.47s/it]
                                                             
{'loss': '0.7091', 'grad_norm': '0.5978', 'learning_rate': '4.688e-05', 'epoch': '0.1982', 'num_input_tokens_seen': 159907400, 'train_runtime': '3.418e+05', 'train_tokens_per_second': '467.8'}

 20%|█▉        | 44600/225000 [94:56:45<324:07:18,  6.47s/it]
 20%|█▉        | 44601/225000 [94:56:50<307:09:46,  6.13s/it]
 20%|█▉        | 44602/225000 [94:56:55<293:43:22,  5.86s/it]
 20%|█▉        | 44603/225000 [94:57:03<319:00:05,  6.37s/it]
 20%|█▉        | 44604/225000 [94:57:12<362:40:32,  7.24s/it]
 20%|█▉        | 44605/225000 [94:57:18<342:49:53,  6.84s/it]
 20%|█▉        | 44606/225000 [94:57:29<398:54:22,  7.96s/it]
 20%|█▉        | 44607/225000 [94:57:36<396:12:20,  7.91s/it]
 20%|█▉        | 44608/225000 [94:57:43<376:53:54,  7.52s/it]
 20%|█▉        | 44609/225000 [94:57:47<331:28:14,  6.62s/it]
 20%|█▉        | 44610/225000 [94:57:52<292:54:32,  5.85s/it]
 20%|█▉        | 44611/225000 [94:58:03<378:31:04,  7.55s/it]
 20%|█▉        | 44612/225000 [94:58:07<323:22:31,  6.45s/it]
 20%|█▉        | 44613/225000 [94:58:11<286:25:33,  5.72s/it]
 20%|█▉        | 44614/225000 [94:58:23<387:27:23,  7.73s/it]
 20%|█▉        | 44615/225000 [94:58:30<365:08:16,  7.29s/it]
 20%|█▉        | 44616/225000 [94:58:34<315:46:41,  6.30s/it]
 20%|█▉        | 44617/225000 [94:58:38<285:12:55,  5.69s/it]
 20%|█▉        | 44618/225000 [94:58:43<278:53:55,  5.57s/it]
 20%|█▉        | 44619/225000 [94:58:53<343:43:19,  6.86s/it]
 20%|█▉        | 44620/225000 [94:58:58<314:56:32,  6.29s/it]
 20%|█▉        | 44621/225000 [94:59:08<363:41:52,  7.26s/it]
 20%|█▉        | 44622/225000 [94:59:15<360:19:25,  7.19s/it]
 20%|█▉        | 44623/225000 [94:59:22<370:57:28,  7.40s/it]
 20%|█▉        | 44624/225000 [94:59:27<326:03:59,  6.51s/it]
 20%|█▉        | 44625/225000 [94:59:32<300:10:50,  5.99s/it]
 20%|█▉        | 44626/225000 [94:59:36<277:54:22,  5.55s/it]
 20%|█▉        | 44627/225000 [94:59:43<293:36:43,  5.86s/it]
 20%|█▉        | 44628/225000 [94:59:54<367:04:48,  7.33s/it]
 20%|█▉        | 44629/225000 [94:59:58<324:06:38,  6.47s/it]
 20%|█▉        | 44630/225000 [95:00:02<291:02:17,  5.81s/it]
 20%|█▉        | 44631/225000 [95:00:13<362:46:26,  7.24s/it]
 20%|█▉        | 44632/225000 [95:00:22<395:29:56,  7.89s/it]
 20%|█▉        | 44633/225000 [95:00:26<336:07:08,  6.71s/it]
 20%|█▉        | 44634/225000 [95:00:31<314:20:04,  6.27s/it]
 20%|█▉        | 44635/225000 [95:00:43<398:44:06,  7.96s/it]
 20%|█▉        | 44636/225000 [95:00:50<375:22:28,  7.49s/it]
 20%|█▉        | 44637/225000 [95:00:55<334:34:01,  6.68s/it]
 20%|█▉        | 44638/225000 [95:01:04<380:41:08,  7.60s/it]
 20%|█▉        | 44639/225000 [95:01:08<327:25:35,  6.54s/it]
 20%|█▉        | 44640/225000 [95:01:20<411:19:35,  8.21s/it]
 20%|█▉        | 44641/225000 [95:01:24<345:04:36,  6.89s/it]
 20%|█▉        | 44642/225000 [95:01:32<360:06:20,  7.19s/it]
 20%|█▉        | 44643/225000 [95:01:38<336:47:16,  6.72s/it]
 20%|█▉        | 44644/225000 [95:01:42<302:23:10,  6.04s/it]
 20%|█▉        | 44645/225000 [95:01:55<398:55:13,  7.96s/it]
 20%|█▉        | 44646/225000 [95:02:01<367:05:55,  7.33s/it]
 20%|█▉        | 44647/225000 [95:02:05<322:39:08,  6.44s/it]
 20%|█▉        | 44648/225000 [95:02:10<296:39:33,  5.92s/it]
 20%|█▉        | 44649/225000 [95:02:21<384:28:20,  7.67s/it]
 20%|█▉        | 44650/225000 [95:02:25<324:14:19,  6.47s/it]
                                                             
{'loss': '0.7084', 'grad_norm': '0.7777', 'learning_rate': '4.687e-05', 'epoch': '0.1984', 'num_input_tokens_seen': 160058632, 'train_runtime': '3.421e+05', 'train_tokens_per_second': '467.8'}

 20%|█▉        | 44650/225000 [95:02:25<324:14:19,  6.47s/it]
 20%|█▉        | 44651/225000 [95:02:34<366:19:51,  7.31s/it]
 20%|█▉        | 44652/225000 [95:02:40<346:49:35,  6.92s/it]
 20%|█▉        | 44653/225000 [95:02:48<363:59:36,  7.27s/it]
 20%|█▉        | 44654/225000 [95:02:52<313:37:47,  6.26s/it]
 20%|█▉        | 44655/225000 [95:03:02<371:01:48,  7.41s/it]
 20%|█▉        | 44656/225000 [95:03:07<325:27:52,  6.50s/it]
 20%|█▉        | 44657/225000 [95:03:13<322:55:56,  6.45s/it]
 20%|█▉        | 44658/225000 [95:03:22<358:43:50,  7.16s/it]
 20%|█▉        | 44659/225000 [95:03:26<314:19:48,  6.27s/it]
 20%|█▉        | 44660/225000 [95:03:32<315:21:31,  6.30s/it]
 20%|█▉        | 44661/225000 [95:03:41<343:30:55,  6.86s/it]
 20%|█▉        | 44662/225000 [95:03:46<316:25:29,  6.32s/it]
 20%|█▉        | 44663/225000 [95:03:51<295:54:49,  5.91s/it]
 20%|█▉        | 44664/225000 [95:03:56<290:41:04,  5.80s/it]
 20%|█▉        | 44665/225000 [95:04:09<397:41:26,  7.94s/it]
 20%|█▉        | 44666/225000 [95:04:15<369:37:27,  7.38s/it]
 20%|█▉        | 44667/225000 [95:04:21<342:33:02,  6.84s/it]
 20%|█▉        | 44668/225000 [95:04:35<448:53:17,  8.96s/it]
 20%|█▉        | 44669/225000 [95:04:39<375:48:20,  7.50s/it]
 20%|█▉        | 44670/225000 [95:04:46<369:08:13,  7.37s/it]
 20%|█▉        | 44671/225000 [95:04:51<335:05:56,  6.69s/it]
 20%|█▉        | 44672/225000 [95:05:01<380:21:12,  7.59s/it]
 20%|█▉        | 44673/225000 [95:05:05<333:11:46,  6.65s/it]
 20%|█▉        | 44674/225000 [95:05:10<304:57:19,  6.09s/it]
 20%|█▉        | 44675/225000 [95:05:15<288:01:33,  5.75s/it]
 20%|█▉        | 44676/225000 [95:05:20<285:06:19,  5.69s/it]
 20%|█▉        | 44677/225000 [95:05:28<311:55:59,  6.23s/it]
 20%|█▉        | 44678/225000 [95:05:41<415:03:14,  8.29s/it]
 20%|█▉        | 44679/225000 [95:05:46<362:14:49,  7.23s/it]
 20%|█▉        | 44680/225000 [95:05:51<335:49:07,  6.70s/it]
 20%|█▉        | 44681/225000 [95:06:00<371:46:07,  7.42s/it]
 20%|█▉        | 44682/225000 [95:06:08<375:11:07,  7.49s/it]
 20%|█▉        | 44683/225000 [95:06:15<366:40:50,  7.32s/it]
 20%|█▉        | 44684/225000 [95:06:19<316:00:47,  6.31s/it]
 20%|█▉        | 44685/225000 [95:06:29<370:12:06,  7.39s/it]
 20%|█▉        | 44686/225000 [95:06:33<320:48:46,  6.41s/it]
 20%|█▉        | 44687/225000 [95:06:41<340:27:18,  6.80s/it]
 20%|█▉        | 44688/225000 [95:06:47<340:22:32,  6.80s/it]
 20%|█▉        | 44689/225000 [95:06:52<309:38:50,  6.18s/it]
 20%|█▉        | 44690/225000 [95:06:57<293:24:54,  5.86s/it]
 20%|█▉        | 44691/225000 [95:07:02<281:42:41,  5.62s/it]
 20%|█▉        | 44692/225000 [95:07:08<288:56:43,  5.77s/it]
 20%|█▉        | 44693/225000 [95:07:21<396:29:08,  7.92s/it]
 20%|█▉        | 44694/225000 [95:07:28<377:20:54,  7.53s/it]
 20%|█▉        | 44695/225000 [95:07:32<325:53:01,  6.51s/it]
 20%|█▉        | 44696/225000 [95:07:42<371:20:06,  7.41s/it]
 20%|█▉        | 44697/225000 [95:07:46<323:52:14,  6.47s/it]
 20%|█▉        | 44698/225000 [95:07:50<294:06:56,  5.87s/it]
 20%|█▉        | 44699/225000 [95:08:02<382:45:27,  7.64s/it]
 20%|█▉        | 44700/225000 [95:08:07<336:45:57,  6.72s/it]
                                                             
{'loss': '0.6922', 'grad_norm': '0.7833', 'learning_rate': '4.686e-05', 'epoch': '0.1987', 'num_input_tokens_seen': 160209416, 'train_runtime': '3.425e+05', 'train_tokens_per_second': '467.8'}

 20%|█▉        | 44700/225000 [95:08:07<336:45:57,  6.72s/it]
 20%|█▉        | 44701/225000 [95:08:13<331:59:49,  6.63s/it]
 20%|█▉        | 44702/225000 [95:08:22<369:34:16,  7.38s/it]
 20%|█▉        | 44703/225000 [95:08:27<326:12:12,  6.51s/it]
 20%|█▉        | 44704/225000 [95:08:32<302:14:55,  6.04s/it]
 20%|█▉        | 44705/225000 [95:08:37<287:31:37,  5.74s/it]
 20%|█▉        | 44706/225000 [95:08:42<273:58:20,  5.47s/it]
 20%|█▉        | 44707/225000 [95:08:47<276:33:30,  5.52s/it]
 20%|█▉        | 44708/225000 [95:09:00<391:51:53,  7.82s/it]
 20%|█▉        | 44709/225000 [95:09:07<377:18:17,  7.53s/it]
 20%|█▉        | 44710/225000 [95:09:16<388:41:24,  7.76s/it]
 20%|█▉        | 44711/225000 [95:09:22<375:28:06,  7.50s/it]
 20%|█▉        | 44712/225000 [95:09:30<372:27:27,  7.44s/it]
 20%|█▉        | 44713/225000 [95:09:35<333:41:34,  6.66s/it]
 20%|█▉        | 44714/225000 [95:09:40<313:14:28,  6.25s/it]
 20%|█▉        | 44715/225000 [95:09:45<300:08:06,  5.99s/it]
 20%|█▉        | 44716/225000 [95:09:50<283:40:48,  5.66s/it]
 20%|█▉        | 44717/225000 [95:09:55<275:06:08,  5.49s/it]
 20%|█▉        | 44718/225000 [95:10:06<356:31:18,  7.12s/it]
 20%|█▉        | 44719/225000 [95:10:16<399:19:35,  7.97s/it]
 20%|█▉        | 44720/225000 [95:10:23<389:09:07,  7.77s/it]
 20%|█▉        | 44721/225000 [95:10:29<359:59:20,  7.19s/it]
 20%|█▉        | 44722/225000 [95:10:34<321:24:59,  6.42s/it]
 20%|█▉        | 44723/225000 [95:10:39<297:52:57,  5.95s/it]
 20%|█▉        | 44724/225000 [95:10:45<298:15:32,  5.96s/it]
 20%|█▉        | 44725/225000 [95:10:50<295:42:18,  5.91s/it]
 20%|█▉        | 44726/225000 [95:10:56<295:45:47,  5.91s/it]
 20%|█▉        | 44727/225000 [95:11:06<348:24:54,  6.96s/it]
 20%|█▉        | 44728/225000 [95:11:16<391:41:01,  7.82s/it]
 20%|█▉        | 44729/225000 [95:11:20<343:37:51,  6.86s/it]
 20%|█▉        | 44730/225000 [95:11:25<315:10:45,  6.29s/it]
 20%|█▉        | 44731/225000 [95:11:36<382:44:08,  7.64s/it]
 20%|█▉        | 44732/225000 [95:11:43<376:15:38,  7.51s/it]
 20%|█▉        | 44733/225000 [95:11:48<341:01:30,  6.81s/it]
 20%|█▉        | 44734/225000 [95:11:54<318:54:08,  6.37s/it]
 20%|█▉        | 44735/225000 [95:11:59<295:47:19,  5.91s/it]
 20%|█▉        | 44736/225000 [95:12:06<326:02:45,  6.51s/it]
 20%|█▉        | 44737/225000 [95:12:19<414:00:15,  8.27s/it]
 20%|█▉        | 44738/225000 [95:12:23<346:36:14,  6.92s/it]
 20%|█▉        | 44739/225000 [95:12:28<320:21:58,  6.40s/it]
 20%|█▉        | 44740/225000 [95:12:36<346:07:26,  6.91s/it]
 20%|█▉        | 44741/225000 [95:12:44<368:21:20,  7.36s/it]
 20%|█▉        | 44742/225000 [95:12:49<322:50:51,  6.45s/it]
 20%|█▉        | 44743/225000 [95:12:57<347:24:20,  6.94s/it]
 20%|█▉        | 44744/225000 [95:13:05<368:04:16,  7.35s/it]
 20%|█▉        | 44745/225000 [95:13:10<326:07:49,  6.51s/it]
 20%|█▉        | 44746/225000 [95:13:14<296:47:17,  5.93s/it]
 20%|█▉        | 44747/225000 [95:13:25<375:18:45,  7.50s/it]
 20%|█▉        | 44748/225000 [95:13:31<344:30:04,  6.88s/it]
 20%|█▉        | 44749/225000 [95:13:38<351:49:45,  7.03s/it]
 20%|█▉        | 44750/225000 [95:13:43<312:38:46,  6.24s/it]
                                                             
{'loss': '0.6906', 'grad_norm': '0.654', 'learning_rate': '4.685e-05', 'epoch': '0.1989', 'num_input_tokens_seen': 160359248, 'train_runtime': '3.428e+05', 'train_tokens_per_second': '467.8'}

 20%|█▉        | 44750/225000 [95:13:43<312:38:46,  6.24s/it]
 20%|█▉        | 44751/225000 [95:13:48<301:28:01,  6.02s/it]
 20%|█▉        | 44752/225000 [95:13:53<290:25:26,  5.80s/it]
 20%|█▉        | 44753/225000 [95:13:59<281:37:38,  5.62s/it]
 20%|█▉        | 44754/225000 [95:14:12<392:16:54,  7.83s/it]
 20%|█▉        | 44755/225000 [95:14:18<374:27:18,  7.48s/it]
 20%|█▉        | 44756/225000 [95:14:23<335:59:16,  6.71s/it]
 20%|█▉        | 44757/225000 [95:14:28<304:20:53,  6.08s/it]
 20%|█▉        | 44758/225000 [95:14:39<382:48:24,  7.65s/it]
 20%|█▉        | 44759/225000 [95:14:43<330:59:12,  6.61s/it]
 20%|█▉        | 44760/225000 [95:14:48<301:46:24,  6.03s/it]
 20%|█▉        | 44761/225000 [95:14:53<286:59:07,  5.73s/it]
 20%|█▉        | 44762/225000 [95:15:02<331:19:06,  6.62s/it]
 20%|█▉        | 44763/225000 [95:15:14<413:22:25,  8.26s/it]
 20%|█▉        | 44764/225000 [95:15:19<363:46:43,  7.27s/it]
 20%|█▉        | 44765/225000 [95:15:24<329:05:38,  6.57s/it]
 20%|█▉        | 44766/225000 [95:15:32<360:31:20,  7.20s/it]
 20%|█▉        | 44767/225000 [95:15:42<394:56:35,  7.89s/it]
 20%|█▉        | 44768/225000 [95:15:47<353:14:50,  7.06s/it]
 20%|█▉        | 44769/225000 [95:15:56<376:55:23,  7.53s/it]
 20%|█▉        | 44770/225000 [95:16:05<405:05:07,  8.09s/it]
 20%|█▉        | 44771/225000 [95:16:09<341:04:48,  6.81s/it]
 20%|█▉        | 44772/225000 [95:16:17<360:04:10,  7.19s/it]
 20%|█▉        | 44773/225000 [95:16:22<332:18:16,  6.64s/it]
 20%|█▉        | 44774/225000 [95:16:30<350:53:36,  7.01s/it]
 20%|█▉        | 44775/225000 [95:16:34<304:49:47,  6.09s/it]
 20%|█▉        | 44776/225000 [95:16:45<372:21:17,  7.44s/it]
 20%|█▉        | 44777/225000 [95:16:49<329:12:52,  6.58s/it]
 20%|█▉        | 44778/225000 [95:16:54<300:02:31,  5.99s/it]
 20%|█▉        | 44779/225000 [95:17:02<326:24:24,  6.52s/it]
 20%|█▉        | 44780/225000 [95:17:09<348:07:04,  6.95s/it]
 20%|█▉        | 44781/225000 [95:17:18<378:09:59,  7.55s/it]
 20%|█▉        | 44782/225000 [95:17:23<328:49:32,  6.57s/it]
 20%|█▉        | 44783/225000 [95:17:27<292:36:28,  5.85s/it]
 20%|█▉        | 44784/225000 [95:17:39<390:28:02,  7.80s/it]
 20%|█▉        | 44785/225000 [95:17:43<326:20:23,  6.52s/it]
 20%|█▉        | 44786/225000 [95:17:52<372:13:10,  7.44s/it]
 20%|█▉        | 44787/225000 [95:17:57<334:31:16,  6.68s/it]
 20%|█▉        | 44788/225000 [95:18:02<307:42:11,  6.15s/it]
 20%|█▉        | 44789/225000 [95:18:07<288:19:47,  5.76s/it]
 20%|█▉        | 44790/225000 [95:18:12<270:57:10,  5.41s/it]
 20%|█▉        | 44791/225000 [95:18:22<346:24:48,  6.92s/it]
 20%|█▉        | 44792/225000 [95:18:33<400:39:32,  8.00s/it]
 20%|█▉        | 44793/225000 [95:18:39<379:30:56,  7.58s/it]
 20%|█▉        | 44794/225000 [95:18:45<351:07:07,  7.01s/it]
 20%|█▉        | 44795/225000 [95:18:49<314:46:50,  6.29s/it]
 20%|█▉        | 44796/225000 [95:18:54<290:09:00,  5.80s/it]
 20%|█▉        | 44797/225000 [95:18:59<277:49:00,  5.55s/it]
 20%|█▉        | 44798/225000 [95:19:08<335:36:32,  6.70s/it]
 20%|█▉        | 44799/225000 [95:19:21<424:19:44,  8.48s/it]
 20%|█▉        | 44800/225000 [95:19:25<351:19:54,  7.02s/it]
                                                             
{'loss': '0.698', 'grad_norm': '0.8028', 'learning_rate': '4.684e-05', 'epoch': '0.1991', 'num_input_tokens_seen': 160511752, 'train_runtime': '3.432e+05', 'train_tokens_per_second': '467.7'}

 20%|█▉        | 44800/225000 [95:19:25<351:19:54,  7.02s/it]
 20%|█▉        | 44801/225000 [95:19:29<315:05:52,  6.30s/it]
 20%|█▉        | 44802/225000 [95:19:40<385:23:22,  7.70s/it]
 20%|█▉        | 44803/225000 [95:19:47<374:08:16,  7.47s/it]
 20%|█▉        | 44804/225000 [95:19:52<329:49:05,  6.59s/it]
 20%|█▉        | 44805/225000 [95:20:00<354:46:38,  7.09s/it]
 20%|█▉        | 44806/225000 [95:20:04<303:11:45,  6.06s/it]
 20%|█▉        | 44807/225000 [95:20:12<339:12:09,  6.78s/it]
 20%|█▉        | 44808/225000 [95:20:17<315:42:29,  6.31s/it]
 20%|█▉        | 44809/225000 [95:20:22<297:26:43,  5.94s/it]
 20%|█▉        | 44810/225000 [95:20:28<289:24:36,  5.78s/it]
 20%|█▉        | 44811/225000 [95:20:33<286:25:00,  5.72s/it]
 20%|█▉        | 44812/225000 [95:20:43<351:14:45,  7.02s/it]
 20%|█▉        | 44813/225000 [95:20:53<384:52:16,  7.69s/it]
 20%|█▉        | 44814/225000 [95:21:01<388:59:50,  7.77s/it]
 20%|█▉        | 44815/225000 [95:21:05<341:51:19,  6.83s/it]
 20%|█▉        | 44816/225000 [95:21:14<374:44:28,  7.49s/it]
 20%|█▉        | 44817/225000 [95:21:19<329:13:44,  6.58s/it]
 20%|█▉        | 44818/225000 [95:21:26<344:27:01,  6.88s/it]
 20%|█▉        | 44819/225000 [95:21:31<307:17:57,  6.14s/it]
 20%|█▉        | 44820/225000 [95:21:42<383:27:52,  7.66s/it]
 20%|█▉        | 44821/225000 [95:21:45<320:26:46,  6.40s/it]
 20%|█▉        | 44822/225000 [95:21:50<293:05:31,  5.86s/it]
 20%|█▉        | 44823/225000 [95:21:55<274:08:13,  5.48s/it]
 20%|█▉        | 44824/225000 [95:22:06<359:02:41,  7.17s/it]
 20%|█▉        | 44825/225000 [95:22:12<338:06:53,  6.76s/it]
 20%|█▉        | 44826/225000 [95:22:19<345:48:50,  6.91s/it]
 20%|█▉        | 44827/225000 [95:22:24<315:41:24,  6.31s/it]
 20%|█▉        | 44828/225000 [95:22:30<308:56:01,  6.17s/it]
 20%|█▉        | 44829/225000 [95:22:35<298:45:26,  5.97s/it]
 20%|█▉        | 44830/225000 [95:22:41<302:07:42,  6.04s/it]
 20%|█▉        | 44831/225000 [95:22:47<298:57:44,  5.97s/it]
 20%|█▉        | 44832/225000 [95:22:55<334:13:12,  6.68s/it]
 20%|█▉        | 44833/225000 [95:23:08<420:13:03,  8.40s/it]
 20%|█▉        | 44834/225000 [95:23:12<362:36:11,  7.25s/it]
 20%|█▉        | 44835/225000 [95:23:24<426:18:29,  8.52s/it]
 20%|█▉        | 44836/225000 [95:23:30<394:55:57,  7.89s/it]
 20%|█▉        | 44837/225000 [95:23:39<405:25:06,  8.10s/it]
 20%|█▉        | 44838/225000 [95:23:44<359:58:45,  7.19s/it]
 20%|█▉        | 44839/225000 [95:23:49<321:56:02,  6.43s/it]
 20%|█▉        | 44840/225000 [95:23:53<291:15:41,  5.82s/it]
 20%|█▉        | 44841/225000 [95:24:05<382:09:35,  7.64s/it]
 20%|█▉        | 44842/225000 [95:24:09<323:42:37,  6.47s/it]
 20%|█▉        | 44843/225000 [95:24:18<361:44:52,  7.23s/it]
 20%|█▉        | 44844/225000 [95:24:22<313:13:22,  6.26s/it]
 20%|█▉        | 44845/225000 [95:24:31<353:53:24,  7.07s/it]
 20%|█▉        | 44846/225000 [95:24:35<314:34:41,  6.29s/it]
 20%|█▉        | 44847/225000 [95:24:39<286:53:02,  5.73s/it]
 20%|█▉        | 44848/225000 [95:24:48<335:55:46,  6.71s/it]
 20%|█▉        | 44849/225000 [95:24:56<344:13:40,  6.88s/it]
 20%|█▉        | 44850/225000 [95:25:03<352:43:56,  7.05s/it]
                                                             
{'loss': '0.6913', 'grad_norm': '0.7183', 'learning_rate': '4.683e-05', 'epoch': '0.1993', 'num_input_tokens_seen': 160662936, 'train_runtime': '3.435e+05', 'train_tokens_per_second': '467.7'}

 20%|█▉        | 44850/225000 [95:25:03<352:43:56,  7.05s/it]
 20%|█▉        | 44851/225000 [95:25:08<324:10:00,  6.48s/it]
 20%|█▉        | 44852/225000 [95:25:14<310:25:57,  6.20s/it]
 20%|█▉        | 44853/225000 [95:25:19<297:55:26,  5.95s/it]
 20%|█▉        | 44854/225000 [95:25:25<295:23:04,  5.90s/it]
 20%|█▉        | 44855/225000 [95:25:31<293:10:04,  5.86s/it]
 20%|█▉        | 44856/225000 [95:25:36<289:48:14,  5.79s/it]
 20%|█▉        | 44857/225000 [95:25:42<286:50:24,  5.73s/it]
 20%|█▉        | 44858/225000 [95:25:48<286:49:16,  5.73s/it]
 20%|█▉        | 44859/225000 [95:26:02<409:44:58,  8.19s/it]
 20%|█▉        | 44860/225000 [95:26:10<406:46:04,  8.13s/it]
 20%|█▉        | 44861/225000 [95:26:18<402:31:29,  8.04s/it]
 20%|█▉        | 44862/225000 [95:26:23<358:27:45,  7.16s/it]
 20%|█▉        | 44863/225000 [95:26:28<326:11:58,  6.52s/it]
 20%|█▉        | 44864/225000 [95:26:33<304:56:30,  6.09s/it]
 20%|█▉        | 44865/225000 [95:26:38<284:27:32,  5.68s/it]
 20%|█▉        | 44866/225000 [95:26:46<321:21:42,  6.42s/it]
 20%|█▉        | 44867/225000 [95:26:57<395:08:42,  7.90s/it]
 20%|█▉        | 44868/225000 [95:27:03<366:15:55,  7.32s/it]
 20%|█▉        | 44869/225000 [95:27:08<334:05:33,  6.68s/it]
 20%|█▉        | 44870/225000 [95:27:13<311:32:19,  6.23s/it]
 20%|█▉        | 44871/225000 [95:27:18<293:58:15,  5.88s/it]
 20%|█▉        | 44872/225000 [95:27:27<333:13:24,  6.66s/it]
 20%|█▉        | 44873/225000 [95:27:39<410:55:31,  8.21s/it]
 20%|█▉        | 44874/225000 [95:27:43<347:13:31,  6.94s/it]
 20%|█▉        | 44875/225000 [95:27:51<372:59:36,  7.45s/it]
 20%|█▉        | 44876/225000 [95:27:57<348:14:03,  6.96s/it]
 20%|█▉        | 44877/225000 [95:28:06<372:39:15,  7.45s/it]
 20%|█▉        | 44878/225000 [95:28:12<357:31:27,  7.15s/it]
 20%|█▉        | 44879/225000 [95:28:17<325:22:15,  6.50s/it]
 20%|█▉        | 44880/225000 [95:28:23<308:06:04,  6.16s/it]
 20%|█▉        | 44881/225000 [95:28:27<288:16:36,  5.76s/it]
 20%|█▉        | 44882/225000 [95:28:35<316:47:56,  6.33s/it]
 20%|█▉        | 44883/225000 [95:28:45<373:43:30,  7.47s/it]
 20%|█▉        | 44884/225000 [95:28:49<319:24:12,  6.38s/it]
 20%|█▉        | 44885/225000 [95:28:59<378:44:37,  7.57s/it]
 20%|█▉        | 44886/225000 [95:29:06<372:00:11,  7.44s/it]
 20%|█▉        | 44887/225000 [95:29:11<335:18:16,  6.70s/it]
 20%|█▉        | 44888/225000 [95:29:17<314:01:25,  6.28s/it]
 20%|█▉        | 44889/225000 [95:29:22<292:49:36,  5.85s/it]
 20%|█▉        | 44890/225000 [95:29:27<278:52:44,  5.57s/it]
 20%|█▉        | 44891/225000 [95:29:34<308:41:04,  6.17s/it]
 20%|█▉        | 44892/225000 [95:29:48<418:29:35,  8.36s/it]
 20%|█▉        | 44893/225000 [95:29:53<380:10:59,  7.60s/it]
 20%|█▉        | 44894/225000 [95:29:58<339:23:34,  6.78s/it]
 20%|█▉        | 44895/225000 [95:30:04<316:32:21,  6.33s/it]
 20%|█▉        | 44896/225000 [95:30:09<298:07:20,  5.96s/it]
 20%|█▉        | 44897/225000 [95:30:16<317:57:53,  6.36s/it]
 20%|█▉        | 44898/225000 [95:30:27<387:29:54,  7.75s/it]
 20%|█▉        | 44899/225000 [95:30:33<359:54:42,  7.19s/it]
 20%|█▉        | 44900/225000 [95:30:41<374:02:16,  7.48s/it]
                                                             
{'loss': '0.68', 'grad_norm': '0.5705', 'learning_rate': '4.682e-05', 'epoch': '0.1996', 'num_input_tokens_seen': 160814096, 'train_runtime': '3.438e+05', 'train_tokens_per_second': '467.7'}

 20%|█▉        | 44900/225000 [95:30:41<374:02:16,  7.48s/it]
 20%|█▉        | 44901/225000 [95:30:49<377:03:52,  7.54s/it]
 20%|█▉        | 44902/225000 [95:30:53<330:15:29,  6.60s/it]
 20%|█▉        | 44903/225000 [95:31:00<337:45:10,  6.75s/it]
 20%|█▉        | 44904/225000 [95:31:07<343:53:12,  6.87s/it]
 20%|█▉        | 44905/225000 [95:31:16<372:00:49,  7.44s/it]
 20%|█▉        | 44906/225000 [95:31:21<333:37:30,  6.67s/it]
 20%|█▉        | 44907/225000 [95:31:27<328:17:35,  6.56s/it]
 20%|█▉        | 44908/225000 [95:31:33<312:58:41,  6.26s/it]
 20%|█▉        | 44909/225000 [95:31:38<299:07:12,  5.98s/it]
 20%|█▉        | 44910/225000 [95:31:43<288:11:26,  5.76s/it]
 20%|█▉        | 44911/225000 [95:31:49<283:19:39,  5.66s/it]
 20%|█▉        | 44912/225000 [95:31:59<356:32:00,  7.13s/it]
 20%|█▉        | 44913/225000 [95:32:09<394:19:26,  7.88s/it]
 20%|█▉        | 44914/225000 [95:32:14<347:18:16,  6.94s/it]
 20%|█▉        | 44915/225000 [95:32:19<326:26:30,  6.53s/it]
 20%|█▉        | 44916/225000 [95:32:25<311:25:44,  6.23s/it]
 20%|█▉        | 44917/225000 [95:32:33<338:42:50,  6.77s/it]
 20%|█▉        | 44918/225000 [95:32:39<329:41:56,  6.59s/it]
 20%|█▉        | 44919/225000 [95:32:45<316:07:59,  6.32s/it]
 20%|█▉        | 44920/225000 [95:32:50<302:24:03,  6.05s/it]
 20%|█▉        | 44921/225000 [95:32:56<296:58:33,  5.94s/it]
 20%|█▉        | 44922/225000 [95:33:01<291:56:48,  5.84s/it]
 20%|█▉        | 44923/225000 [95:33:15<405:22:24,  8.10s/it]
 20%|█▉        | 44924/225000 [95:33:23<400:02:38,  8.00s/it]
 20%|█▉        | 44925/225000 [95:33:29<371:37:30,  7.43s/it]
 20%|█▉        | 44926/225000 [95:33:36<367:05:44,  7.34s/it]
 20%|█▉        | 44927/225000 [95:33:44<373:06:56,  7.46s/it]
 20%|█▉        | 44928/225000 [95:33:50<359:26:46,  7.19s/it]
 20%|█▉        | 44929/225000 [95:33:57<357:55:55,  7.16s/it]
 20%|█▉        | 44930/225000 [95:34:02<329:37:16,  6.59s/it]
 20%|█▉        | 44931/225000 [95:34:07<298:33:58,  5.97s/it]
 20%|█▉        | 44932/225000 [95:34:14<308:17:46,  6.16s/it]
 20%|█▉        | 44933/225000 [95:34:25<391:09:27,  7.82s/it]
 20%|█▉        | 44934/225000 [95:34:29<334:55:59,  6.70s/it]
 20%|█▉        | 44935/225000 [95:34:33<295:51:25,  5.92s/it]
 20%|█▉        | 44936/225000 [95:34:44<364:12:05,  7.28s/it]
 20%|█▉        | 44937/225000 [95:34:49<331:29:14,  6.63s/it]
 20%|█▉        | 44938/225000 [95:34:59<375:19:13,  7.50s/it]
 20%|█▉        | 44939/225000 [95:35:03<335:37:41,  6.71s/it]
 20%|█▉        | 44940/225000 [95:35:09<326:20:44,  6.52s/it]
 20%|█▉        | 44941/225000 [95:35:19<377:32:33,  7.55s/it]
 20%|█▉        | 44942/225000 [95:35:24<338:44:45,  6.77s/it]
 20%|█▉        | 44943/225000 [95:35:33<373:20:51,  7.46s/it]
 20%|█▉        | 44944/225000 [95:35:40<363:43:05,  7.27s/it]
 20%|█▉        | 44945/225000 [95:35:45<328:30:34,  6.57s/it]
 20%|█▉        | 44946/225000 [95:35:50<303:46:37,  6.07s/it]
 20%|█▉        | 44947/225000 [95:35:55<286:26:23,  5.73s/it]
 20%|█▉        | 44948/225000 [95:36:00<276:37:05,  5.53s/it]
 20%|█▉        | 44949/225000 [95:36:13<384:07:21,  7.68s/it]
 20%|█▉        | 44950/225000 [95:36:20<375:25:33,  7.51s/it]
                                                             
{'loss': '0.6771', 'grad_norm': '0.7461', 'learning_rate': '4.681e-05', 'epoch': '0.1998', 'num_input_tokens_seen': 160965152, 'train_runtime': '3.442e+05', 'train_tokens_per_second': '467.7'}

 20%|█▉        | 44950/225000 [95:36:20<375:25:33,  7.51s/it]
 20%|█▉        | 44951/225000 [95:36:29<399:05:25,  7.98s/it]
 20%|█▉        | 44952/225000 [95:36:33<344:01:55,  6.88s/it]
 20%|█▉        | 44953/225000 [95:36:38<313:15:12,  6.26s/it]
 20%|█▉        | 44954/225000 [95:36:50<400:27:38,  8.01s/it]
 20%|█▉        | 44955/225000 [95:36:54<337:03:32,  6.74s/it]
 20%|█▉        | 44956/225000 [95:36:58<300:10:54,  6.00s/it]
 20%|█▉        | 44957/225000 [95:37:09<372:21:57,  7.45s/it]
 20%|█▉        | 44958/225000 [95:37:14<335:28:51,  6.71s/it]
 20%|█▉        | 44959/225000 [95:37:24<381:03:11,  7.62s/it]
 20%|█▉        | 44960/225000 [95:37:29<350:37:55,  7.01s/it]
 20%|█▉        | 44961/225000 [95:37:34<319:16:52,  6.38s/it]
 20%|█▉        | 44962/225000 [95:37:39<292:52:10,  5.86s/it]
 20%|█▉        | 44963/225000 [95:37:48<333:22:07,  6.67s/it]
 20%|█▉        | 44964/225000 [95:37:54<333:51:48,  6.68s/it]
 20%|█▉        | 44965/225000 [95:38:02<356:04:36,  7.12s/it]
 20%|█▉        | 44966/225000 [95:38:08<333:20:46,  6.67s/it]
 20%|█▉        | 44967/225000 [95:38:12<292:47:17,  5.85s/it]
 20%|█▉        | 44968/225000 [95:38:18<299:18:12,  5.99s/it]
 20%|█▉        | 44969/225000 [95:38:27<346:23:51,  6.93s/it]
 20%|█▉        | 44970/225000 [95:38:31<301:07:15,  6.02s/it]
 20%|█▉        | 44971/225000 [95:38:36<289:23:15,  5.79s/it]
 20%|█▉        | 44972/225000 [95:38:42<284:32:16,  5.69s/it]
 20%|█▉        | 44973/225000 [95:38:54<383:01:51,  7.66s/it]
 20%|█▉        | 44974/225000 [95:39:00<355:15:10,  7.10s/it]
 20%|█▉        | 44975/225000 [95:39:05<319:54:44,  6.40s/it]
 20%|█▉        | 44976/225000 [95:39:10<308:09:30,  6.16s/it]
 20%|█▉        | 44977/225000 [95:39:20<353:33:14,  7.07s/it]
 20%|█▉        | 44978/225000 [95:39:30<409:16:08,  8.18s/it]
 20%|█▉        | 44979/225000 [95:39:35<350:31:42,  7.01s/it]
 20%|█▉        | 44980/225000 [95:39:44<382:10:35,  7.64s/it]
 20%|█▉        | 44981/225000 [95:39:51<379:12:09,  7.58s/it]
 20%|█▉        | 44982/225000 [95:39:58<375:10:15,  7.50s/it]
 20%|█▉        | 44983/225000 [95:40:05<365:39:30,  7.31s/it]
 20%|█▉        | 44984/225000 [95:40:10<324:21:41,  6.49s/it]
 20%|█▉        | 44985/225000 [95:40:15<304:56:03,  6.10s/it]
 20%|█▉        | 44986/225000 [95:40:28<409:34:22,  8.19s/it]
 20%|█▉        | 44987/225000 [95:40:33<356:58:47,  7.14s/it]
 20%|█▉        | 44988/225000 [95:40:39<334:37:45,  6.69s/it]
 20%|█▉        | 44989/225000 [95:40:44<318:57:09,  6.38s/it]
 20%|█▉        | 44990/225000 [95:40:49<298:53:34,  5.98s/it]
 20%|█▉        | 44991/225000 [95:40:55<291:14:38,  5.82s/it]
 20%|█▉        | 44992/225000 [95:41:00<290:29:09,  5.81s/it]
 20%|█▉        | 44993/225000 [95:41:08<321:07:54,  6.42s/it]
 20%|█▉        | 44994/225000 [95:41:22<432:19:05,  8.65s/it]
 20%|█▉        | 44995/225000 [95:41:26<365:00:28,  7.30s/it]
 20%|█▉        | 44996/225000 [95:41:32<334:18:24,  6.69s/it]
 20%|█▉        | 44997/225000 [95:41:41<372:39:48,  7.45s/it]
 20%|█▉        | 44998/225000 [95:41:47<359:18:04,  7.19s/it]
 20%|█▉        | 44999/225000 [95:41:53<341:36:46,  6.83s/it]
 20%|██        | 45000/225000 [95:42:03<382:37:06,  7.65s/it]
                                                             
{'loss': '0.69', 'grad_norm': '0.8652', 'learning_rate': '4.681e-05', 'epoch': '0.2', 'num_input_tokens_seen': 161117680, 'train_runtime': '3.445e+05', 'train_tokens_per_second': '467.7'}

 20%|██        | 45000/225000 [95:42:03<382:37:06,  7.65s/it]
 20%|██        | 45001/225000 [95:42:08<344:23:23,  6.89s/it]
 20%|██        | 45002/225000 [95:42:13<322:54:21,  6.46s/it]
 20%|██        | 45003/225000 [95:42:19<302:59:23,  6.06s/it]
 20%|██        | 45004/225000 [95:42:24<290:43:14,  5.81s/it]
 20%|██        | 45005/225000 [95:42:29<283:23:54,  5.67s/it]
 20%|██        | 45006/225000 [95:42:35<280:29:23,  5.61s/it]
 20%|██        | 45007/225000 [95:42:40<272:37:16,  5.45s/it]
 20%|██        | 45008/225000 [95:42:52<378:02:02,  7.56s/it]
 20%|██        | 45009/225000 [95:43:02<407:02:35,  8.14s/it]
 20%|██        | 45010/225000 [95:43:07<365:34:38,  7.31s/it]
 20%|██        | 45011/225000 [95:43:13<339:54:49,  6.80s/it]
 20%|██        | 45012/225000 [95:43:18<315:10:15,  6.30s/it]
 20%|██        | 45013/225000 [95:43:29<391:30:21,  7.83s/it]
 20%|██        | 45014/225000 [95:43:33<332:21:00,  6.65s/it]
 20%|██        | 45015/225000 [95:43:40<332:46:13,  6.66s/it]
 20%|██        | 45016/225000 [95:43:49<368:18:57,  7.37s/it]
 20%|██        | 45017/225000 [95:43:54<328:00:56,  6.56s/it]
 20%|██        | 45018/225000 [95:43:58<301:18:08,  6.03s/it]
 20%|██        | 45019/225000 [95:44:09<372:01:37,  7.44s/it]
 20%|██        | 45020/225000 [95:44:15<343:36:47,  6.87s/it]
 20%|██        | 45021/225000 [95:44:19<309:35:01,  6.19s/it]
 20%|██        | 45022/225000 [95:44:24<284:18:44,  5.69s/it]
 20%|██        | 45023/225000 [95:44:31<310:33:08,  6.21s/it]
 20%|██        | 45024/225000 [95:44:42<380:40:07,  7.61s/it]
 20%|██        | 45025/225000 [95:44:48<348:41:34,  6.97s/it]
 20%|██        | 45026/225000 [95:44:53<329:44:19,  6.60s/it]
 20%|██        | 45027/225000 [95:44:59<316:43:27,  6.34s/it]
 20%|██        | 45028/225000 [95:45:05<308:34:00,  6.17s/it]
 20%|██        | 45029/225000 [95:45:10<296:14:24,  5.93s/it]
 20%|██        | 45030/225000 [95:45:16<290:50:54,  5.82s/it]
 20%|██        | 45031/225000 [95:45:29<397:32:42,  7.95s/it]
 20%|██        | 45032/225000 [95:45:36<388:15:36,  7.77s/it]
 20%|██        | 45033/225000 [95:45:42<364:45:30,  7.30s/it]
 20%|██        | 45034/225000 [95:45:48<341:20:09,  6.83s/it]
 20%|██        | 45035/225000 [95:45:57<376:11:48,  7.53s/it]
 20%|██        | 45036/225000 [95:46:04<369:52:09,  7.40s/it]
 20%|██        | 45037/225000 [95:46:10<352:37:02,  7.05s/it]
 20%|██        | 45038/225000 [95:46:18<359:58:39,  7.20s/it]
 20%|██        | 45039/225000 [95:46:23<321:56:56,  6.44s/it]
 20%|██        | 45040/225000 [95:46:27<295:14:22,  5.91s/it]
 20%|██        | 45041/225000 [95:46:38<369:47:29,  7.40s/it]
 20%|██        | 45042/225000 [95:46:42<315:09:31,  6.30s/it]
 20%|██        | 45043/225000 [95:46:53<381:44:15,  7.64s/it]
 20%|██        | 45044/225000 [95:46:56<324:21:01,  6.49s/it]
 20%|██        | 45045/225000 [95:47:04<346:28:28,  6.93s/it]
 20%|██        | 45046/225000 [95:47:10<334:03:00,  6.68s/it]
 20%|██        | 45047/225000 [95:47:18<348:03:28,  6.96s/it]
 20%|██        | 45048/225000 [95:47:23<312:49:41,  6.26s/it]
 20%|██        | 45049/225000 [95:47:27<285:06:44,  5.70s/it]
 20%|██        | 45050/225000 [95:47:33<294:58:38,  5.90s/it]
                                                             
{'loss': '0.702', 'grad_norm': '0.6787', 'learning_rate': '4.68e-05', 'epoch': '0.2002', 'num_input_tokens_seen': 161268288, 'train_runtime': '3.449e+05', 'train_tokens_per_second': '467.6'}

 20%|██        | 45050/225000 [95:47:33<294:58:38,  5.90s/it]
 20%|██        | 45051/225000 [95:47:45<371:41:30,  7.44s/it]
 20%|██        | 45052/225000 [95:47:48<312:28:47,  6.25s/it]
 20%|██        | 45053/225000 [95:47:57<354:37:25,  7.09s/it]
 20%|██        | 45054/225000 [95:48:05<361:02:35,  7.22s/it]
 20%|██        | 45055/225000 [95:48:11<351:05:14,  7.02s/it]
 20%|██        | 45056/225000 [95:48:17<331:04:08,  6.62s/it]
 20%|██        | 45057/225000 [95:48:22<313:05:46,  6.26s/it]
 20%|██        | 45058/225000 [95:48:27<296:05:11,  5.92s/it]
 20%|██        | 45059/225000 [95:48:33<285:38:54,  5.71s/it]
 20%|██        | 45060/225000 [95:48:37<272:21:32,  5.45s/it]
 20%|██        | 45061/225000 [95:48:48<347:47:12,  6.96s/it]
 20%|██        | 45062/225000 [95:48:58<393:28:05,  7.87s/it]
 20%|██        | 45063/225000 [95:49:03<351:11:23,  7.03s/it]
 20%|██        | 45064/225000 [95:49:11<361:52:41,  7.24s/it]
 20%|██        | 45065/225000 [95:49:17<346:26:12,  6.93s/it]
 20%|██        | 45066/225000 [95:49:24<353:54:30,  7.08s/it]
 20%|██        | 45067/225000 [95:49:29<313:11:29,  6.27s/it]
 20%|██        | 45068/225000 [95:49:36<333:20:44,  6.67s/it]
 20%|██        | 45069/225000 [95:49:41<307:59:32,  6.16s/it]
 20%|██        | 45070/225000 [95:49:46<287:09:04,  5.75s/it]
 20%|██        | 45071/225000 [95:49:51<272:16:07,  5.45s/it]
 20%|██        | 45072/225000 [95:50:02<353:12:54,  7.07s/it]
 20%|██        | 45073/225000 [95:50:09<351:38:14,  7.04s/it]
 20%|██        | 45074/225000 [95:50:17<367:56:30,  7.36s/it]
 20%|██        | 45075/225000 [95:50:22<340:35:14,  6.81s/it]
 20%|██        | 45076/225000 [95:50:28<330:25:52,  6.61s/it]
 20%|██        | 45077/225000 [95:50:34<321:19:32,  6.43s/it]
 20%|██        | 45078/225000 [95:50:40<309:37:13,  6.20s/it]
 20%|██        | 45079/225000 [95:50:48<340:35:48,  6.81s/it]
 20%|██        | 45080/225000 [95:50:56<359:41:44,  7.20s/it]
 20%|██        | 45081/225000 [95:51:02<327:52:37,  6.56s/it]
 20%|██        | 45082/225000 [95:51:07<318:54:23,  6.38s/it]
 20%|██        | 45083/225000 [95:51:13<307:21:10,  6.15s/it]
 20%|██        | 45084/225000 [95:51:19<304:27:14,  6.09s/it]
 20%|██        | 45085/225000 [95:51:25<305:57:19,  6.12s/it]
 20%|██        | 45086/225000 [95:51:39<415:42:12,  8.32s/it]
 20%|██        | 45087/225000 [95:51:47<409:02:50,  8.18s/it]
 20%|██        | 45088/225000 [95:51:50<345:11:10,  6.91s/it]
 20%|██        | 45089/225000 [95:51:55<310:19:47,  6.21s/it]
 20%|██        | 45090/225000 [95:52:00<285:08:45,  5.71s/it]
 20%|██        | 45091/225000 [95:52:13<400:53:12,  8.02s/it]
 20%|██        | 45092/225000 [95:52:17<341:33:45,  6.83s/it]
 20%|██        | 45093/225000 [95:52:25<353:25:34,  7.07s/it]
 20%|██        | 45094/225000 [95:52:29<314:52:05,  6.30s/it]
 20%|██        | 45095/225000 [95:52:34<292:08:18,  5.85s/it]
 20%|██        | 45096/225000 [95:52:45<367:44:17,  7.36s/it]
 20%|██        | 45097/225000 [95:52:51<345:23:51,  6.91s/it]
 20%|██        | 45098/225000 [95:52:56<317:55:55,  6.36s/it]
 20%|██        | 45099/225000 [95:53:01<293:27:57,  5.87s/it]
 20%|██        | 45100/225000 [95:53:06<285:06:28,  5.71s/it]
                                                             
{'loss': '0.692', 'grad_norm': '0.801', 'learning_rate': '4.679e-05', 'epoch': '0.2004', 'num_input_tokens_seen': 161418120, 'train_runtime': '3.452e+05', 'train_tokens_per_second': '467.6'}

 20%|██        | 45100/225000 [95:53:06<285:06:28,  5.71s/it]
 20%|██        | 45101/225000 [95:53:17<365:32:31,  7.31s/it]
 20%|██        | 45102/225000 [95:53:27<399:38:49,  8.00s/it]
 20%|██        | 45103/225000 [95:53:33<371:47:59,  7.44s/it]
 20%|██        | 45104/225000 [95:53:40<369:46:09,  7.40s/it]
 20%|██        | 45105/225000 [95:53:48<371:53:56,  7.44s/it]
 20%|██        | 45106/225000 [95:53:55<373:50:23,  7.48s/it]
 20%|██        | 45107/225000 [95:54:00<327:49:48,  6.56s/it]
 20%|██        | 45108/225000 [95:54:08<355:23:05,  7.11s/it]
 20%|██        | 45109/225000 [95:54:13<331:00:48,  6.62s/it]
 20%|██        | 45110/225000 [95:54:19<312:14:13,  6.25s/it]
 20%|██        | 45111/225000 [95:54:24<296:11:30,  5.93s/it]
 20%|██        | 45112/225000 [95:54:30<300:59:31,  6.02s/it]
 20%|██        | 45113/225000 [95:54:35<287:29:31,  5.75s/it]
 20%|██        | 45114/225000 [95:54:41<280:34:23,  5.62s/it]
 20%|██        | 45115/225000 [95:54:52<360:17:08,  7.21s/it]
 20%|██        | 45116/225000 [95:55:01<388:51:01,  7.78s/it]
 20%|██        | 45117/225000 [95:55:08<385:05:29,  7.71s/it]
 20%|██        | 45118/225000 [95:55:16<392:58:09,  7.86s/it]
 20%|██        | 45119/225000 [95:55:21<351:01:53,  7.03s/it]
 20%|██        | 45120/225000 [95:55:26<307:44:14,  6.16s/it]
 20%|██        | 45121/225000 [95:55:36<363:39:37,  7.28s/it]
 20%|██        | 45122/225000 [95:55:42<352:33:22,  7.06s/it]
 20%|██        | 45123/225000 [95:55:47<319:54:30,  6.40s/it]
 20%|██        | 45124/225000 [95:55:57<380:49:18,  7.62s/it]
 20%|██        | 45125/225000 [95:56:02<335:53:49,  6.72s/it]
 20%|██        | 45126/225000 [95:56:11<377:05:37,  7.55s/it]
 20%|██        | 45127/225000 [95:56:17<340:01:43,  6.81s/it]
 20%|██        | 45128/225000 [95:56:21<309:42:10,  6.20s/it]
 20%|██        | 45129/225000 [95:56:27<298:57:55,  5.98s/it]
 20%|██        | 45130/225000 [95:56:40<399:19:02,  7.99s/it]
 20%|██        | 45131/225000 [95:56:44<339:38:53,  6.80s/it]
 20%|██        | 45132/225000 [95:56:53<385:20:12,  7.71s/it]
 20%|██        | 45133/225000 [95:56:57<324:01:16,  6.49s/it]
 20%|██        | 45134/225000 [95:57:03<316:42:51,  6.34s/it]
 20%|██        | 45135/225000 [95:57:11<338:58:25,  6.78s/it]
 20%|██        | 45136/225000 [95:57:16<311:37:45,  6.24s/it]
 20%|██        | 45137/225000 [95:57:23<333:04:46,  6.67s/it]
 20%|██        | 45138/225000 [95:57:31<349:37:40,  7.00s/it]
 20%|██        | 45139/225000 [95:57:36<315:12:21,  6.31s/it]
 20%|██        | 45140/225000 [95:57:45<350:46:21,  7.02s/it]
 20%|██        | 45141/225000 [95:57:52<356:20:30,  7.13s/it]
 20%|██        | 45142/225000 [95:58:01<380:28:57,  7.62s/it]
 20%|██        | 45143/225000 [95:58:08<381:09:15,  7.63s/it]
 20%|██        | 45144/225000 [95:58:12<324:36:53,  6.50s/it]
 20%|██        | 45145/225000 [95:58:17<303:22:47,  6.07s/it]
 20%|██        | 45146/225000 [95:58:26<339:54:46,  6.80s/it]
 20%|██        | 45147/225000 [95:58:31<308:12:23,  6.17s/it]
 20%|██        | 45148/225000 [95:58:35<287:23:40,  5.75s/it]
 20%|██        | 45149/225000 [95:58:41<286:02:30,  5.73s/it]
 20%|██        | 45150/225000 [95:58:54<399:20:46,  7.99s/it]
                                                             
{'loss': '0.6867', 'grad_norm': '0.8271', 'learning_rate': '4.678e-05', 'epoch': '0.2007', 'num_input_tokens_seen': 161570096, 'train_runtime': '3.455e+05', 'train_tokens_per_second': '467.6'}

 20%|██        | 45150/225000 [95:58:54<399:20:46,  7.99s/it]
 20%|██        | 45151/225000 [95:58:58<331:51:12,  6.64s/it]
 20%|██        | 45152/225000 [95:59:04<332:48:08,  6.66s/it]
 20%|██        | 45153/225000 [95:59:11<338:03:56,  6.77s/it]
 20%|██        | 45154/225000 [95:59:19<352:27:35,  7.06s/it]
 20%|██        | 45155/225000 [95:59:26<342:44:28,  6.86s/it]
 20%|██        | 45156/225000 [95:59:36<388:31:57,  7.78s/it]
 20%|██        | 45157/225000 [95:59:40<340:02:54,  6.81s/it]
 20%|██        | 45158/225000 [95:59:46<326:15:01,  6.53s/it]
 20%|██        | 45159/225000 [95:59:52<311:47:01,  6.24s/it]
 20%|██        | 45160/225000 [95:59:57<307:03:44,  6.15s/it]
 20%|██        | 45161/225000 [96:00:03<304:58:53,  6.11s/it]
 20%|██        | 45162/225000 [96:00:09<296:31:06,  5.94s/it]
 20%|██        | 45163/225000 [96:00:16<312:16:06,  6.25s/it]
 20%|██        | 45164/225000 [96:00:29<416:52:32,  8.35s/it]
 20%|██        | 45165/225000 [96:00:36<394:19:33,  7.89s/it]
 20%|██        | 45166/225000 [96:00:40<339:30:15,  6.80s/it]
 20%|██        | 45167/225000 [96:00:50<378:55:22,  7.59s/it]
 20%|██        | 45168/225000 [96:00:57<373:20:10,  7.47s/it]
 20%|██        | 45169/225000 [96:01:02<340:34:29,  6.82s/it]
 20%|██        | 45170/225000 [96:01:07<317:11:39,  6.35s/it]
 20%|██        | 45171/225000 [96:01:12<296:35:48,  5.94s/it]
 20%|██        | 45172/225000 [96:01:18<294:31:07,  5.90s/it]
 20%|██        | 45173/225000 [96:01:24<289:11:56,  5.79s/it]
 20%|██        | 45174/225000 [96:01:36<384:54:26,  7.71s/it]
 20%|██        | 45175/225000 [96:01:44<392:44:51,  7.86s/it]
 20%|██        | 45176/225000 [96:01:49<350:07:23,  7.01s/it]
 20%|██        | 45177/225000 [96:01:54<323:48:52,  6.48s/it]
 20%|██        | 45178/225000 [96:02:00<308:06:45,  6.17s/it]
 20%|██        | 45179/225000 [96:02:05<292:42:45,  5.86s/it]
 20%|██        | 45180/225000 [96:02:13<322:20:47,  6.45s/it]
 20%|██        | 45181/225000 [96:02:25<410:31:19,  8.22s/it]
 20%|██        | 45182/225000 [96:02:30<360:25:53,  7.22s/it]
 20%|██        | 45183/225000 [96:02:34<314:00:23,  6.29s/it]
 20%|██        | 45184/225000 [96:02:44<373:56:51,  7.49s/it]
 20%|██        | 45185/225000 [96:02:51<363:51:22,  7.28s/it]
 20%|██        | 45186/225000 [96:02:59<363:27:26,  7.28s/it]
 20%|██        | 45187/225000 [96:03:03<320:20:10,  6.41s/it]
 20%|██        | 45188/225000 [96:03:08<297:35:21,  5.96s/it]
 20%|██        | 45189/225000 [96:03:13<291:38:47,  5.84s/it]
 20%|██        | 45190/225000 [96:03:21<310:59:48,  6.23s/it]
 20%|██        | 45191/225000 [96:03:33<400:31:41,  8.02s/it]
 20%|██        | 45192/225000 [96:03:38<352:21:17,  7.05s/it]
 20%|██        | 45193/225000 [96:03:43<330:39:39,  6.62s/it]
 20%|██        | 45194/225000 [96:03:48<306:08:41,  6.13s/it]
 20%|██        | 45195/225000 [96:03:54<301:54:14,  6.04s/it]
 20%|██        | 45196/225000 [96:04:07<412:03:56,  8.25s/it]
 20%|██        | 45197/225000 [96:04:12<354:56:29,  7.11s/it]
 20%|██        | 45198/225000 [96:04:20<373:05:02,  7.47s/it]
 20%|██        | 45199/225000 [96:04:27<366:52:05,  7.35s/it]
 20%|██        | 45200/225000 [96:04:32<328:11:11,  6.57s/it]
                                                             
{'loss': '0.7056', 'grad_norm': '0.788', 'learning_rate': '4.677e-05', 'epoch': '0.2009', 'num_input_tokens_seen': 161722360, 'train_runtime': '3.459e+05', 'train_tokens_per_second': '467.6'}

 20%|██        | 45200/225000 [96:04:32<328:11:11,  6.57s/it]
 20%|██        | 45201/225000 [96:04:37<309:34:09,  6.20s/it]
 20%|██        | 45202/225000 [96:04:42<292:35:37,  5.86s/it]
 20%|██        | 45203/225000 [96:04:50<326:10:40,  6.53s/it]
 20%|██        | 45204/225000 [96:05:04<436:41:41,  8.74s/it]
 20%|██        | 45205/225000 [96:05:10<395:34:22,  7.92s/it]
 20%|██        | 45206/225000 [96:05:14<329:24:33,  6.60s/it]
 20%|██        | 45207/225000 [96:05:18<297:11:29,  5.95s/it]
 20%|██        | 45208/225000 [96:05:23<279:42:27,  5.60s/it]
 20%|██        | 45209/225000 [96:05:36<384:49:06,  7.71s/it]
 20%|██        | 45210/225000 [96:05:41<354:29:51,  7.10s/it]
 20%|██        | 45211/225000 [96:05:47<334:54:16,  6.71s/it]
 20%|██        | 45212/225000 [96:05:56<360:02:27,  7.21s/it]
 20%|██        | 45213/225000 [96:06:00<320:33:47,  6.42s/it]
 20%|██        | 45214/225000 [96:06:05<300:01:12,  6.01s/it]
 20%|██        | 45215/225000 [96:06:10<284:16:59,  5.69s/it]
 20%|██        | 45216/225000 [96:06:23<390:49:31,  7.83s/it]
 20%|██        | 45217/225000 [96:06:27<330:38:51,  6.62s/it]
 20%|██        | 45218/225000 [96:06:32<316:19:47,  6.33s/it]
 20%|██        | 45219/225000 [96:06:39<323:38:41,  6.48s/it]
 20%|██        | 45220/225000 [96:06:48<363:35:25,  7.28s/it]
 20%|██        | 45221/225000 [96:06:56<374:46:42,  7.50s/it]
 20%|██        | 45222/225000 [96:07:00<320:14:07,  6.41s/it]
 20%|██        | 45223/225000 [96:07:10<365:32:31,  7.32s/it]
 20%|██        | 45224/225000 [96:07:15<330:56:47,  6.63s/it]
 20%|██        | 45225/225000 [96:07:20<306:28:44,  6.14s/it]
 20%|██        | 45226/225000 [96:07:25<289:14:01,  5.79s/it]
 20%|██        | 45227/225000 [96:07:30<275:30:06,  5.52s/it]
 20%|██        | 45228/225000 [96:07:42<374:46:23,  7.50s/it]
 20%|██        | 45229/225000 [96:07:48<353:00:35,  7.07s/it]
 20%|██        | 45230/225000 [96:07:52<307:32:11,  6.16s/it]
 20%|██        | 45231/225000 [96:07:57<288:54:45,  5.79s/it]
 20%|██        | 45232/225000 [96:08:02<282:53:50,  5.67s/it]
 20%|██        | 45233/225000 [96:08:07<277:04:16,  5.55s/it]
 20%|██        | 45234/225000 [96:08:20<386:04:40,  7.73s/it]
 20%|██        | 45235/225000 [96:08:28<386:37:57,  7.74s/it]
 20%|██        | 45236/225000 [96:08:33<338:32:52,  6.78s/it]
 20%|██        | 45237/225000 [96:08:44<403:53:09,  8.09s/it]
 20%|██        | 45238/225000 [96:08:48<352:41:31,  7.06s/it]
 20%|██        | 45239/225000 [96:08:54<324:42:29,  6.50s/it]
 20%|██        | 45240/225000 [96:09:00<319:11:59,  6.39s/it]
 20%|██        | 45241/225000 [96:09:06<314:19:50,  6.30s/it]
 20%|██        | 45242/225000 [96:09:19<414:09:42,  8.29s/it]
 20%|██        | 45243/225000 [96:09:23<354:33:39,  7.10s/it]
 20%|██        | 45244/225000 [96:09:33<394:39:59,  7.90s/it]
 20%|██        | 45245/225000 [96:09:37<337:44:26,  6.76s/it]
 20%|██        | 45246/225000 [96:09:41<303:23:06,  6.08s/it]
 20%|██        | 45247/225000 [96:09:47<291:01:58,  5.83s/it]
 20%|██        | 45248/225000 [96:09:56<346:35:38,  6.94s/it]
 20%|██        | 45249/225000 [96:10:06<384:35:09,  7.70s/it]
 20%|██        | 45250/225000 [96:10:13<385:44:40,  7.73s/it]
                                                             
{'loss': '0.6953', 'grad_norm': '0.8031', 'learning_rate': '4.676e-05', 'epoch': '0.2011', 'num_input_tokens_seen': 161874376, 'train_runtime': '3.462e+05', 'train_tokens_per_second': '467.6'}

 20%|██        | 45250/225000 [96:10:13<385:44:40,  7.73s/it]
 20%|██        | 45251/225000 [96:10:19<353:05:09,  7.07s/it]
 20%|██        | 45252/225000 [96:10:25<335:45:34,  6.72s/it]
 20%|██        | 45253/225000 [96:10:31<319:46:41,  6.40s/it]
 20%|██        | 45254/225000 [96:10:38<331:35:46,  6.64s/it]
 20%|██        | 45255/225000 [96:10:47<364:35:09,  7.30s/it]
 20%|██        | 45256/225000 [96:10:52<341:26:40,  6.84s/it]
 20%|██        | 45257/225000 [96:10:57<303:39:58,  6.08s/it]
 20%|██        | 45258/225000 [96:11:08<383:55:30,  7.69s/it]
 20%|██        | 45259/225000 [96:11:13<337:20:08,  6.76s/it]
 20%|██        | 45260/225000 [96:11:20<352:28:49,  7.06s/it]
 20%|██        | 45261/225000 [96:11:25<317:23:26,  6.36s/it]
 20%|██        | 45262/225000 [96:11:30<292:49:22,  5.86s/it]
 20%|██        | 45263/225000 [96:11:42<393:20:55,  7.88s/it]
 20%|██        | 45264/225000 [96:11:47<337:54:55,  6.77s/it]
 20%|██        | 45265/225000 [96:11:54<351:27:01,  7.04s/it]
 20%|██        | 45266/225000 [96:12:01<353:39:01,  7.08s/it]
 20%|██        | 45267/225000 [96:12:07<336:40:55,  6.74s/it]
 20%|██        | 45268/225000 [96:12:13<325:24:19,  6.52s/it]
 20%|██        | 45269/225000 [96:12:19<313:40:16,  6.28s/it]
 20%|██        | 45270/225000 [96:12:25<310:40:16,  6.22s/it]
 20%|██        | 45271/225000 [96:12:31<308:19:30,  6.18s/it]
 20%|██        | 45272/225000 [96:12:37<303:56:10,  6.09s/it]
 20%|██        | 45273/225000 [96:12:48<381:00:39,  7.63s/it]
 20%|██        | 45274/225000 [96:12:55<362:55:09,  7.27s/it]
 20%|██        | 45275/225000 [96:13:00<330:00:07,  6.61s/it]
 20%|██        | 45276/225000 [96:13:05<301:22:18,  6.04s/it]
 20%|██        | 45277/225000 [96:13:15<359:16:05,  7.20s/it]
 20%|██        | 45278/225000 [96:13:24<394:09:41,  7.90s/it]
 20%|██        | 45279/225000 [96:13:31<377:44:15,  7.57s/it]
 20%|██        | 45280/225000 [96:13:36<335:11:16,  6.71s/it]
 20%|██        | 45281/225000 [96:13:40<298:08:11,  5.97s/it]
 20%|██        | 45282/225000 [96:13:47<311:04:08,  6.23s/it]
 20%|██        | 45283/225000 [96:13:57<379:50:04,  7.61s/it]
 20%|██        | 45284/225000 [96:14:01<319:18:47,  6.40s/it]
 20%|██        | 45285/225000 [96:14:11<374:23:29,  7.50s/it]
 20%|██        | 45286/225000 [96:14:16<333:17:33,  6.68s/it]
 20%|██        | 45287/225000 [96:14:20<300:37:05,  6.02s/it]
 20%|██        | 45288/225000 [96:14:26<292:02:41,  5.85s/it]
 20%|██        | 45289/225000 [96:14:31<285:10:25,  5.71s/it]
 20%|██        | 45290/225000 [96:14:37<279:20:47,  5.60s/it]
 20%|██        | 45291/225000 [96:14:51<415:13:45,  8.32s/it]
 20%|██        | 45292/225000 [96:14:57<383:24:02,  7.68s/it]
 20%|██        | 45293/225000 [96:15:02<331:23:20,  6.64s/it]
 20%|██        | 45294/225000 [96:15:09<341:56:32,  6.85s/it]
 20%|██        | 45295/225000 [96:15:17<366:51:52,  7.35s/it]
 20%|██        | 45296/225000 [96:15:22<321:29:52,  6.44s/it]
 20%|██        | 45297/225000 [96:15:26<294:16:03,  5.90s/it]
 20%|██        | 45298/225000 [96:15:39<396:15:54,  7.94s/it]
 20%|██        | 45299/225000 [96:15:45<358:59:38,  7.19s/it]
 20%|██        | 45300/225000 [96:15:51<350:41:57,  7.03s/it]
                                                             
{'loss': '0.6864', 'grad_norm': '0.6338', 'learning_rate': '4.675e-05', 'epoch': '0.2013', 'num_input_tokens_seen': 162026112, 'train_runtime': '3.466e+05', 'train_tokens_per_second': '467.5'}

 20%|██        | 45300/225000 [96:15:51<350:41:57,  7.03s/it]
 20%|██        | 45301/225000 [96:15:56<322:26:39,  6.46s/it]
 20%|██        | 45302/225000 [96:16:01<303:05:46,  6.07s/it]
 20%|██        | 45303/225000 [96:16:07<292:39:23,  5.86s/it]
 20%|██        | 45304/225000 [96:16:12<280:27:36,  5.62s/it]
 20%|██        | 45305/225000 [96:16:22<342:25:43,  6.86s/it]
 20%|██        | 45306/225000 [96:16:31<375:57:37,  7.53s/it]
 20%|██        | 45307/225000 [96:16:35<329:05:59,  6.59s/it]
 20%|██        | 45308/225000 [96:16:45<374:14:05,  7.50s/it]
 20%|██        | 45309/225000 [96:16:52<374:40:18,  7.51s/it]
 20%|██        | 45310/225000 [96:16:56<324:59:48,  6.51s/it]
 20%|██        | 45311/225000 [96:17:06<373:14:44,  7.48s/it]
 20%|██        | 45312/225000 [96:17:11<328:01:50,  6.57s/it]
 20%|██        | 45313/225000 [96:17:16<307:56:11,  6.17s/it]
 20%|██        | 45314/225000 [96:17:20<283:04:01,  5.67s/it]
 20%|██        | 45315/225000 [96:17:31<361:57:26,  7.25s/it]
 20%|██        | 45316/225000 [96:17:40<377:07:26,  7.56s/it]
 20%|██        | 45317/225000 [96:17:45<351:32:36,  7.04s/it]
 20%|██        | 45318/225000 [96:17:50<316:35:59,  6.34s/it]
 20%|██        | 45319/225000 [96:17:55<289:58:40,  5.81s/it]
 20%|██        | 45320/225000 [96:18:00<287:52:26,  5.77s/it]
 20%|██        | 45321/225000 [96:18:12<375:42:22,  7.53s/it]
 20%|██        | 45322/225000 [96:18:19<362:49:33,  7.27s/it]
 20%|██        | 45323/225000 [96:18:23<323:26:24,  6.48s/it]
 20%|██        | 45324/225000 [96:18:34<386:26:44,  7.74s/it]
 20%|██        | 45325/225000 [96:18:39<339:09:07,  6.80s/it]
 20%|██        | 45326/225000 [96:18:43<309:55:56,  6.21s/it]
 20%|██        | 45327/225000 [96:18:50<317:29:00,  6.36s/it]
 20%|██        | 45328/225000 [96:19:01<388:56:34,  7.79s/it]
 20%|██        | 45329/225000 [96:19:10<406:41:59,  8.15s/it]
 20%|██        | 45330/225000 [96:19:14<336:48:11,  6.75s/it]
 20%|██        | 45331/225000 [96:19:19<309:38:07,  6.20s/it]
 20%|██        | 45332/225000 [96:19:23<281:39:27,  5.64s/it]
 20%|██        | 45333/225000 [96:19:28<269:32:00,  5.40s/it]
 20%|██        | 45334/225000 [96:19:40<374:30:41,  7.50s/it]
 20%|██        | 45335/225000 [96:19:47<365:26:07,  7.32s/it]
 20%|██        | 45336/225000 [96:19:54<358:35:35,  7.19s/it]
 20%|██        | 45337/225000 [96:20:03<385:19:54,  7.72s/it]
 20%|██        | 45338/225000 [96:20:08<349:39:22,  7.01s/it]
 20%|██        | 45339/225000 [96:20:15<337:41:22,  6.77s/it]
 20%|██        | 45340/225000 [96:20:20<314:06:56,  6.29s/it]
 20%|██        | 45341/225000 [96:20:25<299:50:54,  6.01s/it]
 20%|██        | 45342/225000 [96:20:31<291:34:26,  5.84s/it]
 20%|██        | 45343/225000 [96:20:36<289:00:28,  5.79s/it]
 20%|██        | 45344/225000 [96:20:42<288:22:05,  5.78s/it]
 20%|██        | 45345/225000 [96:20:54<382:13:05,  7.66s/it]
 20%|██        | 45346/225000 [96:21:01<371:20:11,  7.44s/it]
 20%|██        | 45347/225000 [96:21:06<330:57:14,  6.63s/it]
 20%|██        | 45348/225000 [96:21:11<306:54:57,  6.15s/it]
 20%|██        | 45349/225000 [96:21:16<295:11:35,  5.92s/it]
 20%|██        | 45350/225000 [96:21:22<289:58:30,  5.81s/it]
                                                             
{'loss': '0.6903', 'grad_norm': '0.8309', 'learning_rate': '4.674e-05', 'epoch': '0.2016', 'num_input_tokens_seen': 162177728, 'train_runtime': '3.469e+05', 'train_tokens_per_second': '467.5'}

 20%|██        | 45350/225000 [96:21:22<289:58:30,  5.81s/it]
 20%|██        | 45351/225000 [96:21:35<402:10:54,  8.06s/it]
 20%|██        | 45352/225000 [96:21:43<395:22:14,  7.92s/it]
 20%|██        | 45353/225000 [96:21:47<340:33:35,  6.82s/it]
 20%|██        | 45354/225000 [96:21:52<311:18:41,  6.24s/it]
 20%|██        | 45355/225000 [96:22:00<349:07:06,  7.00s/it]
 20%|██        | 45356/225000 [96:22:08<363:10:55,  7.28s/it]
 20%|██        | 45357/225000 [96:22:16<372:45:31,  7.47s/it]
 20%|██        | 45358/225000 [96:22:21<335:13:00,  6.72s/it]
 20%|██        | 45359/225000 [96:22:26<311:10:17,  6.24s/it]
 20%|██        | 45360/225000 [96:22:32<296:29:01,  5.94s/it]
 20%|██        | 45361/225000 [96:22:37<287:10:10,  5.75s/it]
 20%|██        | 45362/225000 [96:22:49<385:03:20,  7.72s/it]
 20%|██        | 45363/225000 [96:22:55<355:21:06,  7.12s/it]
 20%|██        | 45364/225000 [96:22:59<315:44:26,  6.33s/it]
 20%|██        | 45365/225000 [96:23:04<295:05:44,  5.91s/it]
 20%|██        | 45366/225000 [96:23:12<326:36:38,  6.55s/it]
 20%|██        | 45367/225000 [96:23:25<420:51:22,  8.43s/it]
 20%|██        | 45368/225000 [96:23:29<356:08:39,  7.14s/it]
 20%|██        | 45369/225000 [96:23:38<384:03:15,  7.70s/it]
 20%|██        | 45370/225000 [96:23:43<342:24:01,  6.86s/it]
 20%|██        | 45371/225000 [96:23:51<349:46:13,  7.01s/it]
 20%|██        | 45372/225000 [96:23:57<339:24:45,  6.80s/it]
 20%|██        | 45373/225000 [96:24:03<324:36:33,  6.51s/it]
 20%|██        | 45374/225000 [96:24:09<312:34:32,  6.26s/it]
 20%|██        | 45375/225000 [96:24:15<310:21:55,  6.22s/it]
 20%|██        | 45376/225000 [96:24:21<306:16:39,  6.14s/it]
 20%|██        | 45377/225000 [96:24:26<295:52:30,  5.93s/it]
 20%|██        | 45378/225000 [96:24:31<288:08:06,  5.77s/it]
 20%|██        | 45379/225000 [96:24:37<288:49:41,  5.79s/it]
 20%|██        | 45380/225000 [96:24:47<354:30:12,  7.11s/it]
 20%|██        | 45381/225000 [96:24:59<421:59:17,  8.46s/it]
 20%|██        | 45382/225000 [96:25:03<353:02:29,  7.08s/it]
 20%|██        | 45383/225000 [96:25:13<396:27:55,  7.95s/it]
 20%|██        | 45384/225000 [96:25:17<333:35:12,  6.69s/it]
 20%|██        | 45385/225000 [96:25:27<388:56:43,  7.80s/it]
 20%|██        | 45386/225000 [96:25:32<348:46:55,  6.99s/it]
 20%|██        | 45387/225000 [96:25:37<318:48:34,  6.39s/it]
 20%|██        | 45388/225000 [96:25:42<292:02:32,  5.85s/it]
 20%|██        | 45389/225000 [96:25:50<331:55:43,  6.65s/it]
 20%|██        | 45390/225000 [96:26:01<390:19:21,  7.82s/it]
 20%|██        | 45391/225000 [96:26:05<338:10:29,  6.78s/it]
 20%|██        | 45392/225000 [96:26:12<341:40:32,  6.85s/it]
 20%|██        | 45393/225000 [96:26:18<324:01:40,  6.49s/it]
 20%|██        | 45394/225000 [96:26:24<313:40:38,  6.29s/it]
 20%|██        | 45395/225000 [96:26:29<304:25:41,  6.10s/it]
 20%|██        | 45396/225000 [96:26:35<300:26:24,  6.02s/it]
 20%|██        | 45397/225000 [96:26:41<298:03:35,  5.97s/it]
 20%|██        | 45398/225000 [96:26:47<294:40:05,  5.91s/it]
 20%|██        | 45399/225000 [96:26:53<292:50:12,  5.87s/it]
 20%|██        | 45400/225000 [96:27:06<399:48:29,  8.01s/it]
                                                             
{'loss': '0.7021', 'grad_norm': '0.6315', 'learning_rate': '4.673e-05', 'epoch': '0.2018', 'num_input_tokens_seen': 162328728, 'train_runtime': '3.472e+05', 'train_tokens_per_second': '467.5'}

 20%|██        | 45400/225000 [96:27:06<399:48:29,  8.01s/it]
 20%|██        | 45401/225000 [96:27:14<401:29:48,  8.05s/it]
 20%|██        | 45402/225000 [96:27:18<349:10:30,  7.00s/it]
 20%|██        | 45403/225000 [96:27:23<317:58:07,  6.37s/it]
 20%|██        | 45404/225000 [96:27:35<394:31:36,  7.91s/it]
 20%|██        | 45405/225000 [96:27:39<349:09:51,  7.00s/it]
 20%|██        | 45406/225000 [96:27:44<314:07:44,  6.30s/it]
 20%|██        | 45407/225000 [96:27:49<291:38:55,  5.85s/it]
 20%|██        | 45408/225000 [96:27:54<281:14:24,  5.64s/it]
 20%|██        | 45409/225000 [96:28:03<327:14:58,  6.56s/it]
 20%|██        | 45410/225000 [96:28:13<386:46:24,  7.75s/it]
 20%|██        | 45411/225000 [96:28:21<388:13:11,  7.78s/it]
 20%|██        | 45412/225000 [96:28:29<389:36:38,  7.81s/it]
 20%|██        | 45413/225000 [96:28:34<340:59:09,  6.84s/it]
 20%|██        | 45414/225000 [96:28:42<370:31:19,  7.43s/it]
 20%|██        | 45415/225000 [96:28:47<324:49:13,  6.51s/it]
 20%|██        | 45416/225000 [96:28:58<387:47:31,  7.77s/it]
 20%|██        | 45417/225000 [96:29:05<389:23:35,  7.81s/it]
 20%|██        | 45418/225000 [96:29:09<331:25:39,  6.64s/it]
 20%|██        | 45419/225000 [96:29:15<317:40:23,  6.37s/it]
 20%|██        | 45420/225000 [96:29:25<370:34:39,  7.43s/it]
 20%|██        | 45421/225000 [96:29:32<360:19:05,  7.22s/it]
 20%|██        | 45422/225000 [96:29:38<341:13:52,  6.84s/it]
 20%|██        | 45423/225000 [96:29:43<313:30:22,  6.28s/it]
 20%|██        | 45424/225000 [96:29:47<287:24:42,  5.76s/it]
 20%|██        | 45425/225000 [96:29:59<373:19:09,  7.48s/it]
 20%|██        | 45426/225000 [96:30:04<340:09:44,  6.82s/it]
 20%|██        | 45427/225000 [96:30:09<317:09:37,  6.36s/it]
 20%|██        | 45428/225000 [96:30:15<301:49:43,  6.05s/it]
 20%|██        | 45429/225000 [96:30:20<295:03:54,  5.92s/it]
 20%|██        | 45430/225000 [96:30:25<282:40:42,  5.67s/it]
 20%|██        | 45431/225000 [96:30:31<278:05:53,  5.58s/it]
 20%|██        | 45432/225000 [96:30:43<385:41:37,  7.73s/it]
 20%|██        | 45433/225000 [96:30:51<385:27:58,  7.73s/it]
 20%|██        | 45434/225000 [96:30:58<369:51:31,  7.42s/it]
 20%|██        | 45435/225000 [96:31:03<333:04:57,  6.68s/it]
 20%|██        | 45436/225000 [96:31:08<315:59:09,  6.34s/it]
 20%|██        | 45437/225000 [96:31:13<299:03:22,  6.00s/it]
 20%|██        | 45438/225000 [96:31:27<412:21:28,  8.27s/it]
 20%|██        | 45439/225000 [96:31:33<383:17:56,  7.68s/it]
 20%|██        | 45440/225000 [96:31:42<395:29:37,  7.93s/it]
 20%|██        | 45441/225000 [96:31:46<332:33:33,  6.67s/it]
 20%|██        | 45442/225000 [96:31:54<358:11:28,  7.18s/it]
 20%|██        | 45443/225000 [96:31:59<323:25:49,  6.48s/it]
 20%|██        | 45444/225000 [96:32:04<296:19:52,  5.94s/it]
 20%|██        | 45445/225000 [96:32:12<338:14:52,  6.78s/it]
 20%|██        | 45446/225000 [96:32:17<311:08:01,  6.24s/it]
 20%|██        | 45447/225000 [96:32:23<298:25:30,  5.98s/it]
 20%|██        | 45448/225000 [96:32:29<298:09:15,  5.98s/it]
 20%|██        | 45449/225000 [96:32:34<289:02:11,  5.80s/it]
 20%|██        | 45450/225000 [96:32:41<313:46:20,  6.29s/it]
                                                             
{'loss': '0.7101', 'grad_norm': '0.7432', 'learning_rate': '4.672e-05', 'epoch': '0.202', 'num_input_tokens_seen': 162480808, 'train_runtime': '3.476e+05', 'train_tokens_per_second': '467.5'}

 20%|██        | 45450/225000 [96:32:41<313:46:20,  6.29s/it]
 20%|██        | 45451/225000 [96:32:54<412:51:26,  8.28s/it]
 20%|██        | 45452/225000 [96:33:00<375:02:38,  7.52s/it]
 20%|██        | 45453/225000 [96:33:09<397:18:49,  7.97s/it]
 20%|██        | 45454/225000 [96:33:14<349:35:45,  7.01s/it]
 20%|██        | 45455/225000 [96:33:19<314:52:02,  6.31s/it]
 20%|██        | 45456/225000 [96:33:24<296:36:40,  5.95s/it]
 20%|██        | 45457/225000 [96:33:32<328:18:20,  6.58s/it]
 20%|██        | 45458/225000 [96:33:42<385:17:05,  7.73s/it]
 20%|██        | 45459/225000 [96:33:47<340:26:52,  6.83s/it]
 20%|██        | 45460/225000 [96:33:52<322:53:23,  6.47s/it]
 20%|██        | 45461/225000 [96:33:58<304:46:35,  6.11s/it]
 20%|██        | 45462/225000 [96:34:03<297:01:23,  5.96s/it]
 20%|██        | 45463/225000 [96:34:09<290:41:27,  5.83s/it]
 20%|██        | 45464/225000 [96:34:15<294:54:27,  5.91s/it]
 20%|██        | 45465/225000 [96:34:29<416:39:16,  8.35s/it]
 20%|██        | 45466/225000 [96:34:37<410:33:33,  8.23s/it]
 20%|██        | 45467/225000 [96:34:44<394:05:22,  7.90s/it]
 20%|██        | 45468/225000 [96:34:49<348:21:48,  6.99s/it]
 20%|██        | 45469/225000 [96:34:57<361:38:16,  7.25s/it]
 20%|██        | 45470/225000 [96:35:01<322:23:40,  6.46s/it]
 20%|██        | 45471/225000 [96:35:06<299:33:32,  6.01s/it]
 20%|██        | 45472/225000 [96:35:11<279:16:16,  5.60s/it]
 20%|██        | 45473/225000 [96:35:24<387:56:52,  7.78s/it]
 20%|██        | 45474/225000 [96:35:31<373:43:39,  7.49s/it]
 20%|██        | 45475/225000 [96:35:35<326:28:13,  6.55s/it]
 20%|██        | 45476/225000 [96:35:47<410:06:55,  8.22s/it]
 20%|██        | 45477/225000 [96:35:52<362:09:18,  7.26s/it]
 20%|██        | 45478/225000 [96:35:58<333:22:59,  6.69s/it]
 20%|██        | 45479/225000 [96:36:03<317:40:21,  6.37s/it]
 20%|██        | 45480/225000 [96:36:08<298:10:41,  5.98s/it]
 20%|██        | 45481/225000 [96:36:13<281:06:54,  5.64s/it]
 20%|██        | 45482/225000 [96:36:25<374:17:35,  7.51s/it]
 20%|██        | 45483/225000 [96:36:31<354:05:18,  7.10s/it]
 20%|██        | 45484/225000 [96:36:39<367:43:26,  7.37s/it]
 20%|██        | 45485/225000 [96:36:47<369:59:10,  7.42s/it]
 20%|██        | 45486/225000 [96:36:55<386:25:45,  7.75s/it]
 20%|██        | 45487/225000 [96:37:02<367:18:59,  7.37s/it]
 20%|██        | 45488/225000 [96:37:09<360:29:46,  7.23s/it]
 20%|██        | 45489/225000 [96:37:14<327:10:02,  6.56s/it]
 20%|██        | 45490/225000 [96:37:18<298:11:14,  5.98s/it]
 20%|██        | 45491/225000 [96:37:28<352:42:50,  7.07s/it]
 20%|██        | 45492/225000 [96:37:34<343:02:44,  6.88s/it]
 20%|██        | 45493/225000 [96:37:44<389:30:24,  7.81s/it]
 20%|██        | 45494/225000 [96:37:49<339:29:42,  6.81s/it]
 20%|██        | 45495/225000 [96:37:58<376:44:02,  7.56s/it]
 20%|██        | 45496/225000 [96:38:04<346:47:27,  6.95s/it]
 20%|██        | 45497/225000 [96:38:11<354:10:12,  7.10s/it]
 20%|██        | 45498/225000 [96:38:18<354:35:53,  7.11s/it]
 20%|██        | 45499/225000 [96:38:24<332:42:23,  6.67s/it]
 20%|██        | 45500/225000 [96:38:34<384:30:05,  7.71s/it]
                                                             
{'loss': '0.6942', 'grad_norm': '0.6983', 'learning_rate': '4.672e-05', 'epoch': '0.2022', 'num_input_tokens_seen': 162633904, 'train_runtime': '3.479e+05', 'train_tokens_per_second': '467.5'}

 20%|██        | 45500/225000 [96:38:34<384:30:05,  7.71s/it]
 20%|██        | 45501/225000 [96:38:40<353:39:52,  7.09s/it]
 20%|██        | 45502/225000 [96:38:45<334:59:18,  6.72s/it]
 20%|██        | 45503/225000 [96:38:50<309:40:09,  6.21s/it]
 20%|██        | 45504/225000 [96:38:55<289:26:59,  5.81s/it]
 20%|██        | 45505/225000 [96:39:05<342:11:58,  6.86s/it]
 20%|██        | 45506/225000 [96:39:13<358:49:51,  7.20s/it]
 20%|██        | 45507/225000 [96:39:17<318:27:11,  6.39s/it]
 20%|██        | 45508/225000 [96:39:27<372:07:57,  7.46s/it]
 20%|██        | 45509/225000 [96:39:33<342:26:20,  6.87s/it]
 20%|██        | 45510/225000 [96:39:38<320:00:27,  6.42s/it]
 20%|██        | 45511/225000 [96:39:43<307:12:25,  6.16s/it]
 20%|██        | 45512/225000 [96:39:55<391:13:33,  7.85s/it]
 20%|██        | 45513/225000 [96:39:59<335:33:54,  6.73s/it]
 20%|██        | 45514/225000 [96:40:04<301:54:12,  6.06s/it]
 20%|██        | 45515/225000 [96:40:11<324:30:17,  6.51s/it]
 20%|██        | 45516/225000 [96:40:22<379:55:50,  7.62s/it]
 20%|██        | 45517/225000 [96:40:27<342:15:12,  6.86s/it]
 20%|██        | 45518/225000 [96:40:32<318:49:17,  6.39s/it]
 20%|██        | 45519/225000 [96:40:37<302:37:50,  6.07s/it]
 20%|██        | 45520/225000 [96:40:43<299:48:50,  6.01s/it]
 20%|██        | 45521/225000 [96:40:53<352:04:50,  7.06s/it]
 20%|██        | 45522/225000 [96:41:02<386:04:57,  7.74s/it]
 20%|██        | 45523/225000 [96:41:08<355:46:09,  7.14s/it]
 20%|██        | 45524/225000 [96:41:14<341:08:36,  6.84s/it]
 20%|██        | 45525/225000 [96:41:20<324:18:16,  6.51s/it]
 20%|██        | 45526/225000 [96:41:25<309:33:07,  6.21s/it]
 20%|██        | 45527/225000 [96:41:30<294:55:11,  5.92s/it]
 20%|██        | 45528/225000 [96:41:45<425:32:43,  8.54s/it]
 20%|██        | 45529/225000 [96:41:51<391:54:44,  7.86s/it]
 20%|██        | 45530/225000 [96:41:55<333:18:12,  6.69s/it]
 20%|██        | 45531/225000 [96:42:02<331:47:20,  6.66s/it]
 20%|██        | 45532/225000 [96:42:11<369:13:56,  7.41s/it]
 20%|██        | 45533/225000 [96:42:16<331:08:33,  6.64s/it]
 20%|██        | 45534/225000 [96:42:21<310:37:24,  6.23s/it]
 20%|██        | 45535/225000 [96:42:26<286:57:25,  5.76s/it]
 20%|██        | 45536/225000 [96:42:38<380:28:22,  7.63s/it]
 20%|██        | 45537/225000 [96:42:44<351:48:53,  7.06s/it]
 20%|██        | 45538/225000 [96:42:48<319:41:29,  6.41s/it]
 20%|██        | 45539/225000 [96:42:54<312:31:28,  6.27s/it]
 20%|██        | 45540/225000 [96:43:06<397:45:38,  7.98s/it]
 20%|██        | 45541/225000 [96:43:13<371:32:02,  7.45s/it]
 20%|██        | 45542/225000 [96:43:18<334:15:04,  6.71s/it]
 20%|██        | 45543/225000 [96:43:23<310:05:12,  6.22s/it]
 20%|██        | 45544/225000 [96:43:33<365:52:08,  7.34s/it]
 20%|██        | 45545/225000 [96:43:36<310:29:51,  6.23s/it]
 20%|██        | 45546/225000 [96:43:46<363:12:28,  7.29s/it]
 20%|██        | 45547/225000 [96:43:50<307:40:56,  6.17s/it]
 20%|██        | 45548/225000 [96:43:54<281:34:55,  5.65s/it]
 20%|██        | 45549/225000 [96:44:05<366:44:53,  7.36s/it]
 20%|██        | 45550/225000 [96:44:09<312:39:21,  6.27s/it]
                                                             
{'loss': '0.6881', 'grad_norm': '0.7303', 'learning_rate': '4.671e-05', 'epoch': '0.2024', 'num_input_tokens_seen': 162785296, 'train_runtime': '3.482e+05', 'train_tokens_per_second': '467.4'}

 20%|██        | 45550/225000 [96:44:09<312:39:21,  6.27s/it]
 20%|██        | 45551/225000 [96:44:14<297:31:59,  5.97s/it]
 20%|██        | 45552/225000 [96:44:19<285:24:18,  5.73s/it]
 20%|██        | 45553/225000 [96:44:32<386:11:16,  7.75s/it]
 20%|██        | 45554/225000 [96:44:36<332:09:26,  6.66s/it]
 20%|██        | 45555/225000 [96:44:40<294:50:55,  5.92s/it]
 20%|██        | 45556/225000 [96:44:53<400:34:08,  8.04s/it]
 20%|██        | 45557/225000 [96:45:00<381:27:04,  7.65s/it]
 20%|██        | 45558/225000 [96:45:05<346:15:11,  6.95s/it]
 20%|██        | 45559/225000 [96:45:14<373:23:50,  7.49s/it]
 20%|██        | 45560/225000 [96:45:21<360:40:47,  7.24s/it]
 20%|██        | 45561/225000 [96:45:30<386:14:13,  7.75s/it]
 20%|██        | 45562/225000 [96:45:35<348:49:17,  7.00s/it]
 20%|██        | 45563/225000 [96:45:40<324:40:39,  6.51s/it]
 20%|██        | 45564/225000 [96:45:45<302:37:00,  6.07s/it]
 20%|██        | 45565/225000 [96:45:50<283:59:31,  5.70s/it]
 20%|██        | 45566/225000 [96:46:00<340:37:36,  6.83s/it]
 20%|██        | 45567/225000 [96:46:09<378:33:21,  7.60s/it]
 20%|██        | 45568/225000 [96:46:14<335:45:46,  6.74s/it]
 20%|██        | 45569/225000 [96:46:20<321:40:08,  6.45s/it]
 20%|██        | 45570/225000 [96:46:25<313:05:16,  6.28s/it]
 20%|██        | 45571/225000 [96:46:36<370:47:58,  7.44s/it]
 20%|██        | 45572/225000 [96:46:41<334:51:12,  6.72s/it]
 20%|██        | 45573/225000 [96:46:45<294:22:00,  5.91s/it]
 20%|██        | 45574/225000 [96:46:49<276:33:49,  5.55s/it]
 20%|██        | 45575/225000 [96:47:01<370:28:21,  7.43s/it]
 20%|██        | 45576/225000 [96:47:07<347:22:49,  6.97s/it]
 20%|██        | 45577/225000 [96:47:12<317:55:04,  6.38s/it]
 20%|██        | 45578/225000 [96:47:17<297:44:48,  5.97s/it]
 20%|██        | 45579/225000 [96:47:25<328:09:24,  6.58s/it]
 20%|██        | 45580/225000 [96:47:35<373:01:29,  7.48s/it]
 20%|██        | 45581/225000 [96:47:39<326:37:50,  6.55s/it]
 20%|██        | 45582/225000 [96:47:44<295:44:36,  5.93s/it]
 20%|██        | 45583/225000 [96:47:56<386:47:44,  7.76s/it]
 20%|██        | 45584/225000 [96:48:01<355:37:41,  7.14s/it]
 20%|██        | 45585/225000 [96:48:06<314:40:04,  6.31s/it]
 20%|██        | 45586/225000 [96:48:15<358:06:11,  7.19s/it]
 20%|██        | 45587/225000 [96:48:23<376:18:45,  7.55s/it]
 20%|██        | 45588/225000 [96:48:29<342:08:22,  6.87s/it]
 20%|██        | 45589/225000 [96:48:34<324:29:00,  6.51s/it]
 20%|██        | 45590/225000 [96:48:44<378:03:40,  7.59s/it]
 20%|██        | 45591/225000 [96:48:51<357:47:43,  7.18s/it]
 20%|██        | 45592/225000 [96:48:59<372:37:16,  7.48s/it]
 20%|██        | 45593/225000 [96:49:03<323:53:06,  6.50s/it]
 20%|██        | 45594/225000 [96:49:07<290:27:11,  5.83s/it]
 20%|██        | 45595/225000 [96:49:18<366:41:43,  7.36s/it]
 20%|██        | 45596/225000 [96:49:22<313:19:02,  6.29s/it]
 20%|██        | 45597/225000 [96:49:26<286:04:26,  5.74s/it]
 20%|██        | 45598/225000 [96:49:35<329:29:52,  6.61s/it]
 20%|██        | 45599/225000 [96:49:42<341:36:49,  6.86s/it]
 20%|██        | 45600/225000 [96:49:47<311:45:59,  6.26s/it]
                                                             
{'loss': '0.6939', 'grad_norm': '0.8347', 'learning_rate': '4.67e-05', 'epoch': '0.2027', 'num_input_tokens_seen': 162935832, 'train_runtime': '3.486e+05', 'train_tokens_per_second': '467.4'}

 20%|██        | 45600/225000 [96:49:47<311:45:59,  6.26s/it]
 20%|██        | 45601/225000 [96:49:53<305:45:55,  6.14s/it]
 20%|██        | 45602/225000 [96:49:59<294:18:47,  5.91s/it]
 20%|██        | 45603/225000 [96:50:09<361:49:23,  7.26s/it]
 20%|██        | 45604/225000 [96:50:19<397:47:15,  7.98s/it]
 20%|██        | 45605/225000 [96:50:23<351:40:55,  7.06s/it]
 20%|██        | 45606/225000 [96:50:28<320:12:30,  6.43s/it]
 20%|██        | 45607/225000 [96:50:34<299:51:10,  6.02s/it]
 20%|██        | 45608/225000 [96:50:40<302:36:37,  6.07s/it]
 20%|██        | 45609/225000 [96:50:53<415:19:48,  8.33s/it]
 20%|██        | 45610/225000 [96:50:58<357:58:30,  7.18s/it]
 20%|██        | 45611/225000 [96:51:03<335:23:25,  6.73s/it]
 20%|██        | 45612/225000 [96:51:09<323:44:13,  6.50s/it]
 20%|██        | 45613/225000 [96:51:16<322:25:12,  6.47s/it]
 20%|██        | 45614/225000 [96:51:22<313:43:21,  6.30s/it]
 20%|██        | 45615/225000 [96:51:28<310:18:27,  6.23s/it]
 20%|██        | 45616/225000 [96:51:34<304:30:24,  6.11s/it]
 20%|██        | 45617/225000 [96:51:41<324:09:56,  6.51s/it]
 20%|██        | 45618/225000 [96:51:54<421:34:18,  8.46s/it]
 20%|██        | 45619/225000 [96:51:58<352:58:39,  7.08s/it]
 20%|██        | 45620/225000 [96:52:07<379:32:02,  7.62s/it]
 20%|██        | 45621/225000 [96:52:11<327:18:48,  6.57s/it]
 20%|██        | 45622/225000 [96:52:20<369:02:57,  7.41s/it]
 20%|██        | 45623/225000 [96:52:27<357:46:56,  7.18s/it]
 20%|██        | 45624/225000 [96:52:33<333:33:01,  6.69s/it]
 20%|██        | 45625/225000 [96:52:38<316:46:42,  6.36s/it]
 20%|██        | 45626/225000 [96:52:44<315:26:11,  6.33s/it]
 20%|██        | 45627/225000 [96:52:50<298:47:43,  6.00s/it]
 20%|██        | 45628/225000 [96:52:55<297:25:05,  5.97s/it]
 20%|██        | 45629/225000 [96:53:01<289:31:28,  5.81s/it]
 20%|██        | 45630/225000 [96:53:12<370:31:08,  7.44s/it]
 20%|██        | 45631/225000 [96:53:21<395:31:58,  7.94s/it]
 20%|██        | 45632/225000 [96:53:28<375:23:05,  7.53s/it]
 20%|██        | 45633/225000 [96:53:34<359:17:40,  7.21s/it]
 20%|██        | 45634/225000 [96:53:42<370:05:32,  7.43s/it]
 20%|██        | 45635/225000 [96:53:47<331:17:30,  6.65s/it]
 20%|██        | 45636/225000 [96:53:52<310:33:06,  6.23s/it]
 20%|██        | 45637/225000 [96:53:58<305:43:11,  6.14s/it]
 20%|██        | 45638/225000 [96:54:12<426:08:58,  8.55s/it]
 20%|██        | 45639/225000 [96:54:16<351:21:19,  7.05s/it]
 20%|██        | 45640/225000 [96:54:24<364:51:15,  7.32s/it]
 20%|██        | 45641/225000 [96:54:29<325:54:45,  6.54s/it]
 20%|██        | 45642/225000 [96:54:34<300:44:42,  6.04s/it]
 20%|██        | 45643/225000 [96:54:39<294:16:11,  5.91s/it]
 20%|██        | 45644/225000 [96:54:48<334:11:43,  6.71s/it]
 20%|██        | 45645/225000 [96:54:58<388:58:54,  7.81s/it]
 20%|██        | 45646/225000 [96:55:05<371:51:33,  7.46s/it]
 20%|██        | 45647/225000 [96:55:11<346:29:36,  6.95s/it]
 20%|██        | 45648/225000 [96:55:17<334:09:38,  6.71s/it]
 20%|██        | 45649/225000 [96:55:26<368:52:35,  7.40s/it]
 20%|██        | 45650/225000 [96:55:30<318:55:49,  6.40s/it]
                                                             
{'loss': '0.7039', 'grad_norm': '0.7785', 'learning_rate': '4.669e-05', 'epoch': '0.2029', 'num_input_tokens_seen': 163088800, 'train_runtime': '3.489e+05', 'train_tokens_per_second': '467.4'}

 20%|██        | 45650/225000 [96:55:30<318:55:49,  6.40s/it]
 20%|██        | 45651/225000 [96:55:38<345:55:21,  6.94s/it]
 20%|██        | 45652/225000 [96:55:42<304:17:12,  6.11s/it]
 20%|██        | 45653/225000 [96:55:46<276:57:31,  5.56s/it]
 20%|██        | 45654/225000 [96:55:52<272:12:49,  5.46s/it]
 20%|██        | 45655/225000 [96:56:00<309:51:23,  6.22s/it]
 20%|██        | 45656/225000 [96:56:13<413:06:54,  8.29s/it]
 20%|██        | 45657/225000 [96:56:17<349:15:18,  7.01s/it]
 20%|██        | 45658/225000 [96:56:23<334:04:19,  6.71s/it]
 20%|██        | 45659/225000 [96:56:31<352:57:21,  7.09s/it]
 20%|██        | 45660/225000 [96:56:35<316:32:19,  6.35s/it]
 20%|██        | 45661/225000 [96:56:46<375:31:48,  7.54s/it]
 20%|██        | 45662/225000 [96:56:50<330:49:29,  6.64s/it]
 20%|██        | 45663/225000 [96:56:55<306:49:54,  6.16s/it]
 20%|██        | 45664/225000 [96:57:00<290:28:24,  5.83s/it]
 20%|██        | 45665/225000 [96:57:13<388:21:21,  7.80s/it]
 20%|██        | 45666/225000 [96:57:18<347:54:47,  6.98s/it]
 20%|██        | 45667/225000 [96:57:23<314:50:33,  6.32s/it]
 20%|██        | 45668/225000 [96:57:29<319:41:41,  6.42s/it]
 20%|██        | 45669/225000 [96:57:40<391:25:52,  7.86s/it]
 20%|██        | 45670/225000 [96:57:45<337:44:41,  6.78s/it]
 20%|██        | 45671/225000 [96:57:51<325:08:15,  6.53s/it]
 20%|██        | 45672/225000 [96:58:01<382:20:25,  7.68s/it]
 20%|██        | 45673/225000 [96:58:05<326:00:34,  6.54s/it]
 20%|██        | 45674/225000 [96:58:15<372:39:45,  7.48s/it]
 20%|██        | 45675/225000 [96:58:19<327:26:47,  6.57s/it]
 20%|██        | 45676/225000 [96:58:24<302:20:57,  6.07s/it]
 20%|██        | 45677/225000 [96:58:30<299:37:36,  6.02s/it]
 20%|██        | 45678/225000 [96:58:39<341:30:14,  6.86s/it]
 20%|██        | 45679/225000 [96:58:45<330:24:47,  6.63s/it]
 20%|██        | 45680/225000 [96:58:54<367:30:29,  7.38s/it]
 20%|██        | 45681/225000 [96:58:58<321:10:46,  6.45s/it]
 20%|██        | 45682/225000 [96:59:08<379:45:31,  7.62s/it]
 20%|██        | 45683/225000 [96:59:13<334:11:09,  6.71s/it]
 20%|██        | 45684/225000 [96:59:23<376:39:16,  7.56s/it]
 20%|██        | 45685/225000 [96:59:27<332:29:02,  6.68s/it]
 20%|██        | 45686/225000 [96:59:32<303:10:30,  6.09s/it]
 20%|██        | 45687/225000 [96:59:37<287:02:22,  5.76s/it]
 20%|██        | 45688/225000 [96:59:42<281:52:00,  5.66s/it]
 20%|██        | 45689/225000 [96:59:50<312:14:54,  6.27s/it]
 20%|██        | 45690/225000 [97:00:03<412:51:39,  8.29s/it]
 20%|██        | 45691/225000 [97:00:10<393:08:03,  7.89s/it]
 20%|██        | 45692/225000 [97:00:15<346:24:19,  6.95s/it]
 20%|██        | 45693/225000 [97:00:19<307:42:30,  6.18s/it]
 20%|██        | 45694/225000 [97:00:30<384:13:33,  7.71s/it]
 20%|██        | 45695/225000 [97:00:38<377:39:20,  7.58s/it]
 20%|██        | 45696/225000 [97:00:43<339:31:49,  6.82s/it]
 20%|██        | 45697/225000 [97:00:48<315:07:46,  6.33s/it]
 20%|██        | 45698/225000 [97:01:00<398:31:08,  8.00s/it]
 20%|██        | 45699/225000 [97:01:04<338:14:58,  6.79s/it]
 20%|██        | 45700/225000 [97:01:12<363:12:08,  7.29s/it]
                                                             
{'loss': '0.7031', 'grad_norm': '0.7731', 'learning_rate': '4.668e-05', 'epoch': '0.2031', 'num_input_tokens_seen': 163240808, 'train_runtime': '3.493e+05', 'train_tokens_per_second': '467.4'}

 20%|██        | 45700/225000 [97:01:12<363:12:08,  7.29s/it]
 20%|██        | 45701/225000 [97:01:17<328:33:20,  6.60s/it]
 20%|██        | 45702/225000 [97:01:24<331:40:58,  6.66s/it]
 20%|██        | 45703/225000 [97:01:29<298:36:59,  6.00s/it]
 20%|██        | 45704/225000 [97:01:33<280:18:06,  5.63s/it]
 20%|██        | 45705/225000 [97:01:46<384:52:15,  7.73s/it]
 20%|██        | 45706/225000 [97:01:51<340:26:51,  6.84s/it]
 20%|██        | 45707/225000 [97:01:55<307:20:21,  6.17s/it]
 20%|██        | 45708/225000 [97:02:00<284:41:46,  5.72s/it]
 20%|██        | 45709/225000 [97:02:12<376:10:28,  7.55s/it]
 20%|██        | 45710/225000 [97:02:17<348:42:36,  7.00s/it]
 20%|██        | 45711/225000 [97:02:21<303:45:06,  6.10s/it]
 20%|██        | 45712/225000 [97:02:31<357:16:33,  7.17s/it]
 20%|██        | 45713/225000 [97:02:36<315:22:03,  6.33s/it]
 20%|██        | 45714/225000 [97:02:41<298:55:46,  6.00s/it]
 20%|██        | 45715/225000 [97:02:52<375:56:39,  7.55s/it]
 20%|██        | 45716/225000 [97:02:56<330:04:13,  6.63s/it]
 20%|██        | 45717/225000 [97:03:05<366:28:20,  7.36s/it]
 20%|██        | 45718/225000 [97:03:11<341:23:25,  6.86s/it]
 20%|██        | 45719/225000 [97:03:17<332:40:10,  6.68s/it]
 20%|██        | 45720/225000 [97:03:27<378:30:36,  7.60s/it]
 20%|██        | 45721/225000 [97:03:31<328:07:08,  6.59s/it]
 20%|██        | 45722/225000 [97:03:36<296:34:29,  5.96s/it]
 20%|██        | 45723/225000 [97:03:47<372:05:36,  7.47s/it]
 20%|██        | 45724/225000 [97:03:51<325:43:43,  6.54s/it]
 20%|██        | 45725/225000 [97:03:59<339:18:41,  6.81s/it]
 20%|██        | 45726/225000 [97:04:05<337:32:15,  6.78s/it]
 20%|██        | 45727/225000 [97:04:14<366:07:56,  7.35s/it]
 20%|██        | 45728/225000 [97:04:22<381:38:05,  7.66s/it]
 20%|██        | 45729/225000 [97:04:28<348:28:58,  7.00s/it]
 20%|██        | 45730/225000 [97:04:34<327:25:30,  6.58s/it]
 20%|██        | 45731/225000 [97:04:39<314:09:25,  6.31s/it]
 20%|██        | 45732/225000 [97:04:48<358:20:22,  7.20s/it]
 20%|██        | 45733/225000 [97:04:55<346:26:26,  6.96s/it]
 20%|██        | 45734/225000 [97:04:59<306:33:44,  6.16s/it]
 20%|██        | 45735/225000 [97:05:04<287:02:52,  5.76s/it]
 20%|██        | 45736/225000 [97:05:17<396:04:01,  7.95s/it]
 20%|██        | 45737/225000 [97:05:22<349:53:53,  7.03s/it]
 20%|██        | 45738/225000 [97:05:27<316:35:01,  6.36s/it]
 20%|██        | 45739/225000 [97:05:32<297:37:30,  5.98s/it]
 20%|██        | 45740/225000 [97:05:37<288:53:02,  5.80s/it]
 20%|██        | 45741/225000 [97:05:45<312:04:24,  6.27s/it]
 20%|██        | 45742/225000 [97:05:57<398:01:46,  7.99s/it]
 20%|██        | 45743/225000 [97:06:01<339:05:50,  6.81s/it]
 20%|██        | 45744/225000 [97:06:11<399:34:59,  8.02s/it]
 20%|██        | 45745/225000 [97:06:16<340:32:50,  6.84s/it]
 20%|██        | 45746/225000 [97:06:21<326:34:57,  6.56s/it]
 20%|██        | 45747/225000 [97:06:33<404:29:21,  8.12s/it]
 20%|██        | 45748/225000 [97:06:37<338:15:12,  6.79s/it]
 20%|██        | 45749/225000 [97:06:47<391:03:36,  7.85s/it]
 20%|██        | 45750/225000 [97:06:52<345:09:58,  6.93s/it]
                                                             
{'loss': '0.6946', 'grad_norm': '0.7286', 'learning_rate': '4.667e-05', 'epoch': '0.2033', 'num_input_tokens_seen': 163392912, 'train_runtime': '3.496e+05', 'train_tokens_per_second': '467.4'}

 20%|██        | 45750/225000 [97:06:52<345:09:58,  6.93s/it]
 20%|██        | 45751/225000 [97:06:57<311:38:47,  6.26s/it]
 20%|██        | 45752/225000 [97:07:03<306:19:02,  6.15s/it]
 20%|██        | 45753/225000 [97:07:13<372:07:28,  7.47s/it]
 20%|██        | 45754/225000 [97:07:21<383:53:55,  7.71s/it]
 20%|██        | 45755/225000 [97:07:27<349:52:37,  7.03s/it]
 20%|██        | 45756/225000 [97:07:32<327:57:02,  6.59s/it]
 20%|██        | 45757/225000 [97:07:38<308:25:15,  6.19s/it]
 20%|██        | 45758/225000 [97:07:42<282:00:10,  5.66s/it]
 20%|██        | 45759/225000 [97:07:47<267:10:59,  5.37s/it]
 20%|██        | 45760/225000 [97:08:00<379:58:47,  7.63s/it]
 20%|██        | 45761/225000 [97:08:06<353:40:24,  7.10s/it]
 20%|██        | 45762/225000 [97:08:14<374:46:38,  7.53s/it]
 20%|██        | 45763/225000 [97:08:19<341:38:23,  6.86s/it]
 20%|██        | 45764/225000 [97:08:24<307:13:28,  6.17s/it]
 20%|██        | 45765/225000 [97:08:29<287:32:05,  5.78s/it]
 20%|██        | 45766/225000 [97:08:35<286:12:43,  5.75s/it]
 20%|██        | 45767/225000 [97:08:45<358:15:45,  7.20s/it]
 20%|██        | 45768/225000 [97:08:53<370:08:36,  7.43s/it]
 20%|██        | 45769/225000 [97:08:57<321:08:40,  6.45s/it]
 20%|██        | 45770/225000 [97:09:09<406:50:47,  8.17s/it]
 20%|██        | 45771/225000 [97:09:15<366:38:56,  7.36s/it]
 20%|██        | 45772/225000 [97:09:21<353:07:15,  7.09s/it]
 20%|██        | 45773/225000 [97:09:25<305:58:39,  6.15s/it]
 20%|██        | 45774/225000 [97:09:35<363:22:55,  7.30s/it]
 20%|██        | 45775/225000 [97:09:43<369:14:10,  7.42s/it]
 20%|██        | 45776/225000 [97:09:48<329:48:11,  6.62s/it]
 20%|██        | 45777/225000 [97:09:52<298:59:23,  6.01s/it]
 20%|██        | 45778/225000 [97:09:57<279:13:25,  5.61s/it]
 20%|██        | 45779/225000 [97:10:11<403:09:51,  8.10s/it]
 20%|██        | 45780/225000 [97:10:15<339:19:54,  6.82s/it]
 20%|██        | 45781/225000 [97:10:24<381:20:23,  7.66s/it]
 20%|██        | 45782/225000 [97:10:29<331:06:38,  6.65s/it]
 20%|██        | 45783/225000 [97:10:39<386:03:48,  7.76s/it]
 20%|██        | 45784/225000 [97:10:46<378:02:51,  7.59s/it]
 20%|██        | 45785/225000 [97:10:51<333:33:00,  6.70s/it]
 20%|██        | 45786/225000 [97:10:55<302:07:33,  6.07s/it]
 20%|██        | 45787/225000 [97:11:04<346:49:12,  6.97s/it]
 20%|██        | 45788/225000 [97:11:13<374:30:01,  7.52s/it]
 20%|██        | 45789/225000 [97:11:17<319:31:56,  6.42s/it]
 20%|██        | 45790/225000 [97:11:22<293:32:26,  5.90s/it]
 20%|██        | 45791/225000 [97:11:27<283:57:17,  5.70s/it]
 20%|██        | 45792/225000 [97:11:33<290:41:19,  5.84s/it]
 20%|██        | 45793/225000 [97:11:39<291:53:19,  5.86s/it]
 20%|██        | 45794/225000 [97:11:44<281:27:41,  5.65s/it]
 20%|██        | 45795/225000 [97:11:57<389:27:31,  7.82s/it]
 20%|██        | 45796/225000 [97:12:04<378:52:31,  7.61s/it]
 20%|██        | 45797/225000 [97:12:13<388:36:36,  7.81s/it]
 20%|██        | 45798/225000 [97:12:17<337:58:03,  6.79s/it]
 20%|██        | 45799/225000 [97:12:23<329:02:16,  6.61s/it]
 20%|██        | 45800/225000 [97:12:34<387:13:31,  7.78s/it]
                                                             
{'loss': '0.6877', 'grad_norm': '0.6799', 'learning_rate': '4.666e-05', 'epoch': '0.2036', 'num_input_tokens_seen': 163546024, 'train_runtime': '3.5e+05', 'train_tokens_per_second': '467.3'}

 20%|██        | 45800/225000 [97:12:34<387:13:31,  7.78s/it]
 20%|██        | 45801/225000 [97:12:38<339:52:17,  6.83s/it]
 20%|██        | 45802/225000 [97:12:44<318:29:59,  6.40s/it]
 20%|██        | 45803/225000 [97:12:49<303:12:41,  6.09s/it]
 20%|██        | 45804/225000 [97:12:57<331:36:33,  6.66s/it]
 20%|██        | 45805/225000 [97:13:06<369:12:31,  7.42s/it]
 20%|██        | 45806/225000 [97:13:14<372:42:34,  7.49s/it]
 20%|██        | 45807/225000 [97:13:19<336:01:28,  6.75s/it]
 20%|██        | 45808/225000 [97:13:24<314:26:36,  6.32s/it]
 20%|██        | 45809/225000 [97:13:29<291:30:11,  5.86s/it]
 20%|██        | 45810/225000 [97:13:43<406:47:48,  8.17s/it]
 20%|██        | 45811/225000 [97:13:46<340:17:15,  6.84s/it]
 20%|██        | 45812/225000 [97:13:54<351:42:11,  7.07s/it]
 20%|██        | 45813/225000 [97:14:01<354:10:46,  7.12s/it]
 20%|██        | 45814/225000 [97:14:09<358:53:33,  7.21s/it]
 20%|██        | 45815/225000 [97:14:14<328:03:32,  6.59s/it]
 20%|██        | 45816/225000 [97:14:21<343:52:40,  6.91s/it]
 20%|██        | 45817/225000 [97:14:30<365:51:05,  7.35s/it]
 20%|██        | 45818/225000 [97:14:36<342:45:11,  6.89s/it]
 20%|██        | 45819/225000 [97:14:44<363:52:54,  7.31s/it]
 20%|██        | 45820/225000 [97:14:50<343:26:41,  6.90s/it]
 20%|██        | 45821/225000 [97:14:58<366:13:48,  7.36s/it]
 20%|██        | 45822/225000 [97:15:03<322:40:48,  6.48s/it]
 20%|██        | 45823/225000 [97:15:11<350:23:47,  7.04s/it]
 20%|██        | 45824/225000 [97:15:18<348:17:06,  7.00s/it]
 20%|██        | 45825/225000 [97:15:22<305:02:11,  6.13s/it]
 20%|██        | 45826/225000 [97:15:31<341:47:46,  6.87s/it]
 20%|██        | 45827/225000 [97:15:38<353:17:08,  7.10s/it]
 20%|██        | 45828/225000 [97:15:44<331:17:32,  6.66s/it]
 20%|██        | 45829/225000 [97:15:49<311:50:44,  6.27s/it]
 20%|██        | 45830/225000 [97:15:54<294:41:45,  5.92s/it]
 20%|██        | 45831/225000 [97:15:59<279:59:25,  5.63s/it]
 20%|██        | 45832/225000 [97:16:08<322:08:09,  6.47s/it]
 20%|██        | 45833/225000 [97:16:19<388:02:13,  7.80s/it]
 20%|██        | 45834/225000 [97:16:25<374:17:55,  7.52s/it]
 20%|██        | 45835/225000 [97:16:29<321:48:02,  6.47s/it]
 20%|██        | 45836/225000 [97:16:39<373:21:06,  7.50s/it]
 20%|██        | 45837/225000 [97:16:47<378:42:07,  7.61s/it]
 20%|██        | 45838/225000 [97:16:51<323:19:45,  6.50s/it]
 20%|██        | 45839/225000 [97:16:57<316:34:06,  6.36s/it]
 20%|██        | 45840/225000 [97:17:06<351:28:13,  7.06s/it]
 20%|██        | 45841/225000 [97:17:11<319:15:51,  6.42s/it]
 20%|██        | 45842/225000 [97:17:22<388:38:18,  7.81s/it]
 20%|██        | 45843/225000 [97:17:29<372:32:53,  7.49s/it]
 20%|██        | 45844/225000 [97:17:37<380:45:03,  7.65s/it]
 20%|██        | 45845/225000 [97:17:43<359:19:18,  7.22s/it]
 20%|██        | 45846/225000 [97:17:49<341:05:12,  6.85s/it]
 20%|██        | 45847/225000 [97:17:57<364:06:39,  7.32s/it]
 20%|██        | 45848/225000 [97:18:03<339:28:14,  6.82s/it]
 20%|██        | 45849/225000 [97:18:07<292:46:44,  5.88s/it]
 20%|██        | 45850/225000 [97:18:17<359:39:26,  7.23s/it]
                                                             
{'loss': '0.7021', 'grad_norm': '0.6936', 'learning_rate': '4.665e-05', 'epoch': '0.2038', 'num_input_tokens_seen': 163698544, 'train_runtime': '3.503e+05', 'train_tokens_per_second': '467.3'}

 20%|██        | 45850/225000 [97:18:17<359:39:26,  7.23s/it]
 20%|██        | 45851/225000 [97:18:24<352:09:38,  7.08s/it]
 20%|██        | 45852/225000 [97:18:28<314:32:54,  6.32s/it]
 20%|██        | 45853/225000 [97:18:33<288:42:58,  5.80s/it]
 20%|██        | 45854/225000 [97:18:38<271:59:09,  5.47s/it]
 20%|██        | 45855/225000 [97:18:46<319:42:30,  6.42s/it]
 20%|██        | 45856/225000 [97:18:56<377:22:40,  7.58s/it]
 20%|██        | 45857/225000 [97:19:02<353:03:11,  7.09s/it]
 20%|██        | 45858/225000 [97:19:12<388:00:49,  7.80s/it]
 20%|██        | 45859/225000 [97:19:16<329:44:22,  6.63s/it]
 20%|██        | 45860/225000 [97:19:26<376:27:13,  7.57s/it]
 20%|██        | 45861/225000 [97:19:31<342:15:14,  6.88s/it]
 20%|██        | 45862/225000 [97:19:38<341:16:02,  6.86s/it]
 20%|██        | 45863/225000 [97:19:46<363:38:51,  7.31s/it]
 20%|██        | 45864/225000 [97:19:50<312:25:17,  6.28s/it]
 20%|██        | 45865/225000 [97:19:59<358:28:19,  7.20s/it]
 20%|██        | 45866/225000 [97:20:03<314:47:27,  6.33s/it]
 20%|██        | 45867/225000 [97:20:12<340:42:46,  6.85s/it]
 20%|██        | 45868/225000 [97:20:16<310:46:11,  6.25s/it]
 20%|██        | 45869/225000 [97:20:21<289:59:54,  5.83s/it]
 20%|██        | 45870/225000 [97:20:26<278:49:04,  5.60s/it]
 20%|██        | 45871/225000 [97:20:33<295:19:23,  5.94s/it]
 20%|██        | 45872/225000 [97:20:47<410:29:06,  8.25s/it]
 20%|██        | 45873/225000 [97:20:52<367:19:15,  7.38s/it]
 20%|██        | 45874/225000 [97:20:59<362:21:42,  7.28s/it]
 20%|██        | 45875/225000 [97:21:06<353:01:09,  7.09s/it]
 20%|██        | 45876/225000 [97:21:12<343:59:42,  6.91s/it]
 20%|██        | 45877/225000 [97:21:17<304:19:29,  6.12s/it]
 20%|██        | 45878/225000 [97:21:23<307:20:17,  6.18s/it]
 20%|██        | 45879/225000 [97:21:34<387:14:21,  7.78s/it]
 20%|██        | 45880/225000 [97:21:40<351:47:58,  7.07s/it]
 20%|██        | 45881/225000 [97:21:46<343:02:52,  6.89s/it]
 20%|██        | 45882/225000 [97:21:52<326:52:36,  6.57s/it]
 20%|██        | 45883/225000 [97:21:59<325:23:22,  6.54s/it]
 20%|██        | 45884/225000 [97:22:08<364:45:58,  7.33s/it]
 20%|██        | 45885/225000 [97:22:13<332:35:37,  6.68s/it]
 20%|██        | 45886/225000 [97:22:18<305:52:47,  6.15s/it]
 20%|██        | 45887/225000 [97:22:28<361:26:40,  7.26s/it]
 20%|██        | 45888/225000 [97:22:36<381:32:55,  7.67s/it]
 20%|██        | 45889/225000 [97:22:41<333:34:09,  6.70s/it]
 20%|██        | 45890/225000 [97:22:45<296:56:05,  5.97s/it]
 20%|██        | 45891/225000 [97:22:55<356:02:34,  7.16s/it]
 20%|██        | 45892/225000 [97:23:03<364:10:14,  7.32s/it]
 20%|██        | 45893/225000 [97:23:10<364:40:08,  7.33s/it]
 20%|██        | 45894/225000 [97:23:16<345:34:22,  6.95s/it]
 20%|██        | 45895/225000 [97:23:21<318:31:31,  6.40s/it]
 20%|██        | 45896/225000 [97:23:26<299:42:03,  6.02s/it]
 20%|██        | 45897/225000 [97:23:32<290:33:38,  5.84s/it]
 20%|██        | 45898/225000 [97:23:37<281:19:10,  5.65s/it]
 20%|██        | 45899/225000 [97:23:50<388:34:07,  7.81s/it]
 20%|██        | 45900/225000 [97:23:55<345:31:33,  6.95s/it]
                                                             
{'loss': '0.7079', 'grad_norm': '0.6429', 'learning_rate': '4.664e-05', 'epoch': '0.204', 'num_input_tokens_seen': 163849160, 'train_runtime': '3.506e+05', 'train_tokens_per_second': '467.3'}

 20%|██        | 45900/225000 [97:23:55<345:31:33,  6.95s/it]
 20%|██        | 45901/225000 [97:24:02<354:41:51,  7.13s/it]
 20%|██        | 45902/225000 [97:24:07<324:48:30,  6.53s/it]
 20%|██        | 45903/225000 [97:24:13<313:40:58,  6.31s/it]
 20%|██        | 45904/225000 [97:24:19<307:18:38,  6.18s/it]
 20%|██        | 45905/225000 [97:24:25<304:33:08,  6.12s/it]
 20%|██        | 45906/225000 [97:24:38<411:24:24,  8.27s/it]
 20%|██        | 45907/225000 [97:24:46<402:30:20,  8.09s/it]
 20%|██        | 45908/225000 [97:24:52<374:06:30,  7.52s/it]
 20%|██        | 45909/225000 [97:25:02<405:14:42,  8.15s/it]
 20%|██        | 45910/225000 [97:25:07<362:56:10,  7.30s/it]
 20%|██        | 45911/225000 [97:25:13<338:55:19,  6.81s/it]
 20%|██        | 45912/225000 [97:25:19<327:11:42,  6.58s/it]
 20%|██        | 45913/225000 [97:25:24<312:08:46,  6.27s/it]
 20%|██        | 45914/225000 [97:25:30<296:10:02,  5.95s/it]
 20%|██        | 45915/225000 [97:25:35<292:51:52,  5.89s/it]
 20%|██        | 45916/225000 [97:25:42<298:23:42,  6.00s/it]
 20%|██        | 45917/225000 [97:25:48<306:12:38,  6.16s/it]
 20%|██        | 45918/225000 [97:26:03<439:06:30,  8.83s/it]
 20%|██        | 45919/225000 [97:26:09<397:15:16,  7.99s/it]
 20%|██        | 45920/225000 [97:26:17<397:47:57,  8.00s/it]
 20%|██        | 45921/225000 [97:26:23<365:46:19,  7.35s/it]
 20%|██        | 45922/225000 [97:26:28<336:14:56,  6.76s/it]
 20%|██        | 45923/225000 [97:26:34<318:09:54,  6.40s/it]
 20%|██        | 45924/225000 [97:26:40<312:27:06,  6.28s/it]
 20%|██        | 45925/225000 [97:26:45<298:40:21,  6.00s/it]
 20%|██        | 45926/225000 [97:26:59<417:08:18,  8.39s/it]
 20%|██        | 45927/225000 [97:27:04<357:15:50,  7.18s/it]
 20%|██        | 45928/225000 [97:27:11<361:44:12,  7.27s/it]
 20%|██        | 45929/225000 [97:27:17<335:17:16,  6.74s/it]
 20%|██        | 45930/225000 [97:27:22<314:07:50,  6.32s/it]
 20%|██        | 45931/225000 [97:27:34<397:31:01,  7.99s/it]
 20%|██        | 45932/225000 [97:27:37<326:46:12,  6.57s/it]
 20%|██        | 45933/225000 [97:27:43<309:27:43,  6.22s/it]
 20%|██        | 45934/225000 [97:27:47<289:05:58,  5.81s/it]
 20%|██        | 45935/225000 [97:27:53<284:28:19,  5.72s/it]
 20%|██        | 45936/225000 [97:27:59<285:45:31,  5.75s/it]
 20%|██        | 45937/225000 [97:28:08<340:55:23,  6.85s/it]
 20%|██        | 45938/225000 [97:28:19<396:47:29,  7.98s/it]
 20%|██        | 45939/225000 [97:28:26<383:49:40,  7.72s/it]
 20%|██        | 45940/225000 [97:28:33<375:58:58,  7.56s/it]
 20%|██        | 45941/225000 [97:28:39<347:54:52,  6.99s/it]
 20%|██        | 45942/225000 [97:28:44<320:29:56,  6.44s/it]
 20%|██        | 45943/225000 [97:28:50<311:44:53,  6.27s/it]
 20%|██        | 45944/225000 [97:28:55<301:27:41,  6.06s/it]
 20%|██        | 45945/225000 [97:29:01<289:59:33,  5.83s/it]
 20%|██        | 45946/225000 [97:29:07<293:28:16,  5.90s/it]
 20%|██        | 45947/225000 [97:29:21<414:36:50,  8.34s/it]
 20%|██        | 45948/225000 [97:29:29<420:10:16,  8.45s/it]
 20%|██        | 45949/225000 [97:29:35<380:49:07,  7.66s/it]
 20%|██        | 45950/225000 [97:29:43<389:55:42,  7.84s/it]
                                                             
{'loss': '0.6927', 'grad_norm': '0.6844', 'learning_rate': '4.663e-05', 'epoch': '0.2042', 'num_input_tokens_seen': 164001992, 'train_runtime': '3.51e+05', 'train_tokens_per_second': '467.3'}

 20%|██        | 45950/225000 [97:29:43<389:55:42,  7.84s/it]
 20%|██        | 45951/225000 [97:29:48<345:10:58,  6.94s/it]
 20%|██        | 45952/225000 [97:29:53<310:55:52,  6.25s/it]
 20%|██        | 45953/225000 [97:29:58<296:37:19,  5.96s/it]
 20%|██        | 45954/225000 [97:30:12<411:07:15,  8.27s/it]
 20%|██        | 45955/225000 [97:30:18<373:41:33,  7.51s/it]
 20%|██        | 45956/225000 [97:30:25<374:01:29,  7.52s/it]
 20%|██        | 45957/225000 [97:30:31<347:39:34,  6.99s/it]
 20%|██        | 45958/225000 [97:30:36<324:47:40,  6.53s/it]
 20%|██        | 45959/225000 [97:30:42<307:17:11,  6.18s/it]
 20%|██        | 45960/225000 [97:30:47<293:26:07,  5.90s/it]
 20%|██        | 45961/225000 [97:30:52<283:31:57,  5.70s/it]
 20%|██        | 45962/225000 [97:30:58<283:01:27,  5.69s/it]
 20%|██        | 45963/225000 [97:31:03<279:20:17,  5.62s/it]
 20%|██        | 45964/225000 [97:31:16<385:36:04,  7.75s/it]
 20%|██        | 45965/225000 [97:31:27<435:36:56,  8.76s/it]
 20%|██        | 45966/225000 [97:31:31<357:46:40,  7.19s/it]
 20%|██        | 45967/225000 [97:31:40<395:29:50,  7.95s/it]
 20%|██        | 45968/225000 [97:31:48<382:58:21,  7.70s/it]
 20%|██        | 45969/225000 [97:31:51<323:51:06,  6.51s/it]
 20%|██        | 45970/225000 [97:32:01<366:01:55,  7.36s/it]
 20%|██        | 45971/225000 [97:32:06<340:09:51,  6.84s/it]
 20%|██        | 45972/225000 [97:32:11<316:10:59,  6.36s/it]
 20%|██        | 45973/225000 [97:32:16<289:14:50,  5.82s/it]
 20%|██        | 45974/225000 [97:32:21<270:17:35,  5.44s/it]
 20%|██        | 45975/225000 [97:32:33<370:34:09,  7.45s/it]
 20%|██        | 45976/225000 [97:32:39<356:25:46,  7.17s/it]
 20%|██        | 45977/225000 [97:32:44<313:32:01,  6.30s/it]
 20%|██        | 45978/225000 [97:32:48<289:09:25,  5.81s/it]
 20%|██        | 45979/225000 [97:33:01<389:51:45,  7.84s/it]
 20%|██        | 45980/225000 [97:33:05<336:08:03,  6.76s/it]
 20%|██        | 45981/225000 [97:33:14<370:36:01,  7.45s/it]
 20%|██        | 45982/225000 [97:33:18<322:33:10,  6.49s/it]
 20%|██        | 45983/225000 [97:33:26<335:16:23,  6.74s/it]
 20%|██        | 45984/225000 [97:33:35<377:48:52,  7.60s/it]
 20%|██        | 45985/225000 [97:33:40<331:29:01,  6.67s/it]
 20%|██        | 45986/225000 [97:33:45<305:58:16,  6.15s/it]
 20%|██        | 45987/225000 [97:33:55<371:07:16,  7.46s/it]
 20%|██        | 45988/225000 [97:34:01<346:54:07,  6.98s/it]
 20%|██        | 45989/225000 [97:34:10<371:33:17,  7.47s/it]
 20%|██        | 45990/225000 [97:34:14<323:18:29,  6.50s/it]
 20%|██        | 45991/225000 [97:34:19<304:49:07,  6.13s/it]
 20%|██        | 45992/225000 [97:34:24<287:18:22,  5.78s/it]
 20%|██        | 45993/225000 [97:34:37<391:43:26,  7.88s/it]
 20%|██        | 45994/225000 [97:34:43<369:40:44,  7.43s/it]
 20%|██        | 45995/225000 [97:34:47<316:09:57,  6.36s/it]
 20%|██        | 45996/225000 [97:35:01<425:31:21,  8.56s/it]
 20%|██        | 45997/225000 [97:35:05<360:27:27,  7.25s/it]
 20%|██        | 45998/225000 [97:35:09<309:06:29,  6.22s/it]
 20%|██        | 45999/225000 [97:35:20<382:42:16,  7.70s/it]
 20%|██        | 46000/225000 [97:35:25<346:24:42,  6.97s/it]
                                                             
{'loss': '0.7042', 'grad_norm': '0.7445', 'learning_rate': '4.662e-05', 'epoch': '0.2044', 'num_input_tokens_seen': 164154648, 'train_runtime': '3.513e+05', 'train_tokens_per_second': '467.2'}

 20%|██        | 46000/225000 [97:35:25<346:24:42,  6.97s/it]
 20%|██        | 46001/225000 [97:35:30<317:56:44,  6.39s/it]
 20%|██        | 46002/225000 [97:35:35<298:45:19,  6.01s/it]
 20%|██        | 46003/225000 [97:35:40<284:11:57,  5.72s/it]
 20%|██        | 46004/225000 [97:35:53<392:22:32,  7.89s/it]
 20%|██        | 46005/225000 [97:36:02<399:23:38,  8.03s/it]
 20%|██        | 46006/225000 [97:36:09<388:28:03,  7.81s/it]
 20%|██        | 46007/225000 [97:36:15<356:09:46,  7.16s/it]
 20%|██        | 46008/225000 [97:36:20<333:47:07,  6.71s/it]
 20%|██        | 46009/225000 [97:36:26<323:15:09,  6.50s/it]
 20%|██        | 46010/225000 [97:36:32<303:12:51,  6.10s/it]
 20%|██        | 46011/225000 [97:36:37<289:00:46,  5.81s/it]
 20%|██        | 46012/225000 [97:36:42<284:57:08,  5.73s/it]
 20%|██        | 46013/225000 [97:36:56<409:34:10,  8.24s/it]
 20%|██        | 46014/225000 [97:37:02<376:53:31,  7.58s/it]
 20%|██        | 46015/225000 [97:37:11<393:14:25,  7.91s/it]
 20%|██        | 46016/225000 [97:37:17<356:12:49,  7.16s/it]
 20%|██        | 46017/225000 [97:37:22<336:47:51,  6.77s/it]
 20%|██        | 46018/225000 [97:37:33<391:31:45,  7.88s/it]
 20%|██        | 46019/225000 [97:37:37<333:02:21,  6.70s/it]
 20%|██        | 46020/225000 [97:37:46<364:39:04,  7.33s/it]
 20%|██        | 46021/225000 [97:37:49<312:59:29,  6.30s/it]
 20%|██        | 46022/225000 [97:38:01<386:20:37,  7.77s/it]
 20%|██        | 46023/225000 [97:38:05<332:07:45,  6.68s/it]
 20%|██        | 46024/225000 [97:38:10<316:56:24,  6.38s/it]
 20%|██        | 46025/225000 [97:38:16<300:15:44,  6.04s/it]
 20%|██        | 46026/225000 [97:38:21<291:22:38,  5.86s/it]
 20%|██        | 46027/225000 [97:38:34<398:31:51,  8.02s/it]
 20%|██        | 46028/225000 [97:38:40<366:46:18,  7.38s/it]
 20%|██        | 46029/225000 [97:38:46<350:04:29,  7.04s/it]
 20%|██        | 46030/225000 [97:38:50<302:19:04,  6.08s/it]
 20%|██        | 46031/225000 [97:39:01<379:36:28,  7.64s/it]
 20%|██        | 46032/225000 [97:39:08<361:17:16,  7.27s/it]
 20%|██        | 46033/225000 [97:39:12<316:36:51,  6.37s/it]
 20%|██        | 46034/225000 [97:39:18<301:45:20,  6.07s/it]
 20%|██        | 46035/225000 [97:39:23<293:34:37,  5.91s/it]
 20%|██        | 46036/225000 [97:39:36<392:21:05,  7.89s/it]
 20%|██        | 46037/225000 [97:39:41<355:17:49,  7.15s/it]
 20%|██        | 46038/225000 [97:39:45<306:20:21,  6.16s/it]
 20%|██        | 46039/225000 [97:39:56<383:48:05,  7.72s/it]
 20%|██        | 46040/225000 [97:40:01<343:21:46,  6.91s/it]
 20%|██        | 46041/225000 [97:40:08<339:36:21,  6.83s/it]
 20%|██        | 46042/225000 [97:40:17<370:41:29,  7.46s/it]
 20%|██        | 46043/225000 [97:40:23<359:08:09,  7.22s/it]
 20%|██        | 46044/225000 [97:40:29<339:17:14,  6.83s/it]
 20%|██        | 46045/225000 [97:40:39<388:03:47,  7.81s/it]
 20%|██        | 46046/225000 [97:40:45<348:46:01,  7.02s/it]
 20%|██        | 46047/225000 [97:40:53<362:33:48,  7.29s/it]
 20%|██        | 46048/225000 [97:40:58<334:54:21,  6.74s/it]
 20%|██        | 46049/225000 [97:41:06<358:13:48,  7.21s/it]
 20%|██        | 46050/225000 [97:41:11<318:49:19,  6.41s/it]
                                                             
{'loss': '0.6972', 'grad_norm': '0.8159', 'learning_rate': '4.662e-05', 'epoch': '0.2047', 'num_input_tokens_seen': 164307192, 'train_runtime': '3.517e+05', 'train_tokens_per_second': '467.2'}

 20%|██        | 46050/225000 [97:41:11<318:49:19,  6.41s/it]
 20%|██        | 46051/225000 [97:41:21<368:41:33,  7.42s/it]
 20%|██        | 46052/225000 [97:41:25<316:41:30,  6.37s/it]
 20%|██        | 46053/225000 [97:41:34<361:19:33,  7.27s/it]
 20%|██        | 46054/225000 [97:41:39<324:22:39,  6.53s/it]
 20%|██        | 46055/225000 [97:41:44<298:33:38,  6.01s/it]
 20%|██        | 46056/225000 [97:41:49<286:41:54,  5.77s/it]
 20%|██        | 46057/225000 [97:41:54<279:38:08,  5.63s/it]
 20%|██        | 46058/225000 [97:42:05<362:17:11,  7.29s/it]
 20%|██        | 46059/225000 [97:42:16<410:36:16,  8.26s/it]
 20%|██        | 46060/225000 [97:42:22<388:06:57,  7.81s/it]
 20%|██        | 46061/225000 [97:42:27<332:03:53,  6.68s/it]
 20%|██        | 46062/225000 [97:42:35<358:37:40,  7.22s/it]
 20%|██        | 46063/225000 [97:42:40<325:47:11,  6.55s/it]
 20%|██        | 46064/225000 [97:42:45<298:39:25,  6.01s/it]
 20%|██        | 46065/225000 [97:42:50<284:47:53,  5.73s/it]
 20%|██        | 46066/225000 [97:43:03<392:22:21,  7.89s/it]
 20%|██        | 46067/225000 [97:43:09<363:49:05,  7.32s/it]
 20%|██        | 46068/225000 [97:43:16<363:10:57,  7.31s/it]
 20%|██        | 46069/225000 [97:43:20<313:42:48,  6.31s/it]
 20%|██        | 46070/225000 [97:43:25<299:54:45,  6.03s/it]
 20%|██        | 46071/225000 [97:43:36<367:33:49,  7.40s/it]
 20%|██        | 46072/225000 [97:43:41<326:14:30,  6.56s/it]
 20%|██        | 46073/225000 [97:43:50<366:54:17,  7.38s/it]
 20%|██        | 46074/225000 [97:43:54<321:48:10,  6.47s/it]
 20%|██        | 46075/225000 [97:43:59<291:03:55,  5.86s/it]
 20%|██        | 46076/225000 [97:44:13<413:01:57,  8.31s/it]
 20%|██        | 46077/225000 [97:44:18<370:09:30,  7.45s/it]
 20%|██        | 46078/225000 [97:44:24<350:30:15,  7.05s/it]
 20%|██        | 46079/225000 [97:44:31<349:50:35,  7.04s/it]
 20%|██        | 46080/225000 [97:44:39<364:19:21,  7.33s/it]
 20%|██        | 46081/225000 [97:44:47<371:39:58,  7.48s/it]
 20%|██        | 46082/225000 [97:44:54<360:58:39,  7.26s/it]
 20%|██        | 46083/225000 [97:44:58<320:20:43,  6.45s/it]
 20%|██        | 46084/225000 [97:45:04<304:03:20,  6.12s/it]
 20%|██        | 46085/225000 [97:45:09<288:32:52,  5.81s/it]
 20%|██        | 46086/225000 [97:45:14<278:46:09,  5.61s/it]
 20%|██        | 46087/225000 [97:45:23<329:29:23,  6.63s/it]
 20%|██        | 46088/225000 [97:45:35<409:36:50,  8.24s/it]
 20%|██        | 46089/225000 [97:45:41<382:57:58,  7.71s/it]
 20%|██        | 46090/225000 [97:45:47<354:26:20,  7.13s/it]
 20%|██        | 46091/225000 [97:45:53<329:09:50,  6.62s/it]
 20%|██        | 46092/225000 [97:45:57<293:58:42,  5.92s/it]
 20%|██        | 46093/225000 [97:46:05<330:14:12,  6.65s/it]
 20%|██        | 46094/225000 [97:46:16<390:22:13,  7.86s/it]
 20%|██        | 46095/225000 [97:46:20<328:27:28,  6.61s/it]
 20%|██        | 46096/225000 [97:46:29<363:09:19,  7.31s/it]
 20%|██        | 46097/225000 [97:46:34<336:50:16,  6.78s/it]
 20%|██        | 46098/225000 [97:46:40<317:41:36,  6.39s/it]
 20%|██        | 46099/225000 [97:46:45<300:36:12,  6.05s/it]
 20%|██        | 46100/225000 [97:46:51<297:14:09,  5.98s/it]
                                                             
{'loss': '0.6877', 'grad_norm': '0.8462', 'learning_rate': '4.661e-05', 'epoch': '0.2049', 'num_input_tokens_seen': 164459192, 'train_runtime': '3.52e+05', 'train_tokens_per_second': '467.2'}

 20%|██        | 46100/225000 [97:46:51<297:14:09,  5.98s/it]
 20%|██        | 46101/225000 [97:46:56<288:11:49,  5.80s/it]
 20%|██        | 46102/225000 [97:47:08<385:58:24,  7.77s/it]
 20%|██        | 46103/225000 [97:47:15<362:32:56,  7.30s/it]
 20%|██        | 46104/225000 [97:47:20<326:57:37,  6.58s/it]
 20%|██        | 46105/225000 [97:47:25<305:57:32,  6.16s/it]
 20%|██        | 46106/225000 [97:47:30<287:13:52,  5.78s/it]
 20%|██        | 46107/225000 [97:47:39<338:42:42,  6.82s/it]
 20%|██        | 46108/225000 [97:47:51<418:39:31,  8.43s/it]
 20%|██        | 46109/225000 [97:47:55<350:39:04,  7.06s/it]
 20%|██        | 46110/225000 [97:48:00<314:20:17,  6.33s/it]
 20%|██        | 46111/225000 [97:48:05<297:36:57,  5.99s/it]
 20%|██        | 46112/225000 [97:48:16<371:38:09,  7.48s/it]
 20%|██        | 46113/225000 [97:48:25<405:48:42,  8.17s/it]
 20%|██        | 46114/225000 [97:48:33<395:53:48,  7.97s/it]
 20%|██        | 46115/225000 [97:48:37<334:12:08,  6.73s/it]
 20%|██        | 46116/225000 [97:48:45<353:57:47,  7.12s/it]
 20%|██        | 46117/225000 [97:48:51<333:39:46,  6.71s/it]
 20%|██        | 46118/225000 [97:48:59<357:42:21,  7.20s/it]
 20%|██        | 46119/225000 [97:49:03<311:49:38,  6.28s/it]
 20%|██        | 46120/225000 [97:49:09<305:16:33,  6.14s/it]
 20%|██        | 46121/225000 [97:49:19<370:58:59,  7.47s/it]
 20%|██        | 46122/225000 [97:49:25<346:20:57,  6.97s/it]
 20%|██        | 46123/225000 [97:49:32<345:30:23,  6.95s/it]
 20%|██        | 46124/225000 [97:49:38<329:31:59,  6.63s/it]
 20%|██        | 46125/225000 [97:49:47<360:47:13,  7.26s/it]
 21%|██        | 46126/225000 [97:49:52<326:18:09,  6.57s/it]
 21%|██        | 46127/225000 [97:49:56<290:22:07,  5.84s/it]
 21%|██        | 46128/225000 [97:50:04<322:40:01,  6.49s/it]
 21%|██        | 46129/225000 [97:50:09<306:25:56,  6.17s/it]
 21%|██        | 46130/225000 [97:50:18<348:47:27,  7.02s/it]
 21%|██        | 46131/225000 [97:50:25<347:11:08,  6.99s/it]
 21%|██        | 46132/225000 [97:50:31<332:26:45,  6.69s/it]
 21%|██        | 46133/225000 [97:50:37<321:22:15,  6.47s/it]
 21%|██        | 46134/225000 [97:50:43<311:59:17,  6.28s/it]
 21%|██        | 46135/225000 [97:50:52<348:31:22,  7.01s/it]
 21%|██        | 46136/225000 [97:50:58<332:14:48,  6.69s/it]
 21%|██        | 46137/225000 [97:51:02<291:56:12,  5.88s/it]
 21%|██        | 46138/225000 [97:51:06<275:36:38,  5.55s/it]
 21%|██        | 46139/225000 [97:51:14<308:38:48,  6.21s/it]
 21%|██        | 46140/225000 [97:51:25<373:14:10,  7.51s/it]
 21%|██        | 46141/225000 [97:51:29<329:21:26,  6.63s/it]
 21%|██        | 46142/225000 [97:51:37<347:34:07,  7.00s/it]
 21%|██        | 46143/225000 [97:51:42<313:21:05,  6.31s/it]
 21%|██        | 46144/225000 [97:51:47<292:42:28,  5.89s/it]
 21%|██        | 46145/225000 [97:51:56<348:36:04,  7.02s/it]
 21%|██        | 46146/225000 [97:52:05<371:02:54,  7.47s/it]
 21%|██        | 46147/225000 [97:52:09<320:46:04,  6.46s/it]
 21%|██        | 46148/225000 [97:52:14<306:19:36,  6.17s/it]
 21%|██        | 46149/225000 [97:52:20<289:54:31,  5.84s/it]
 21%|██        | 46150/225000 [97:52:29<339:30:48,  6.83s/it]
                                                             
{'loss': '0.6881', 'grad_norm': '0.8386', 'learning_rate': '4.66e-05', 'epoch': '0.2051', 'num_input_tokens_seen': 164609000, 'train_runtime': '3.523e+05', 'train_tokens_per_second': '467.2'}

 21%|██        | 46150/225000 [97:52:29<339:30:48,  6.83s/it]
 21%|██        | 46151/225000 [97:52:40<400:56:20,  8.07s/it]
 21%|██        | 46152/225000 [97:52:45<352:52:54,  7.10s/it]
 21%|██        | 46153/225000 [97:52:49<315:10:32,  6.34s/it]
 21%|██        | 46154/225000 [97:52:54<292:51:15,  5.89s/it]
 21%|██        | 46155/225000 [97:53:01<304:08:52,  6.12s/it]
 21%|██        | 46156/225000 [97:53:14<412:01:45,  8.29s/it]
 21%|██        | 46157/225000 [97:53:19<357:37:23,  7.20s/it]
 21%|██        | 46158/225000 [97:53:23<316:48:44,  6.38s/it]
 21%|██        | 46159/225000 [97:53:34<379:56:41,  7.65s/it]
 21%|██        | 46160/225000 [97:53:38<336:40:31,  6.78s/it]
 21%|██        | 46161/225000 [97:53:43<310:05:48,  6.24s/it]
 21%|██        | 46162/225000 [97:53:55<390:28:02,  7.86s/it]
 21%|██        | 46163/225000 [97:54:01<367:00:17,  7.39s/it]
 21%|██        | 46164/225000 [97:54:08<362:56:23,  7.31s/it]
 21%|██        | 46165/225000 [97:54:13<322:13:48,  6.49s/it]
 21%|██        | 46166/225000 [97:54:18<294:54:15,  5.94s/it]
 21%|██        | 46167/225000 [97:54:23<279:26:42,  5.63s/it]
 21%|██        | 46168/225000 [97:54:34<367:31:09,  7.40s/it]
 21%|██        | 46169/225000 [97:54:41<352:41:50,  7.10s/it]
 21%|██        | 46170/225000 [97:54:47<343:04:00,  6.91s/it]
 21%|██        | 46171/225000 [97:54:56<373:17:18,  7.51s/it]
 21%|██        | 46172/225000 [97:55:00<324:16:41,  6.53s/it]
 21%|██        | 46173/225000 [97:55:11<385:33:02,  7.76s/it]
 21%|██        | 46174/225000 [97:55:15<328:25:01,  6.61s/it]
 21%|██        | 46175/225000 [97:55:19<289:17:34,  5.82s/it]
 21%|██        | 46176/225000 [97:55:31<387:33:13,  7.80s/it]
 21%|██        | 46177/225000 [97:55:35<334:13:00,  6.73s/it]
 21%|██        | 46178/225000 [97:55:45<376:58:07,  7.59s/it]
 21%|██        | 46179/225000 [97:55:50<345:32:32,  6.96s/it]
 21%|██        | 46180/225000 [97:56:00<383:22:30,  7.72s/it]
 21%|██        | 46181/225000 [97:56:06<358:05:21,  7.21s/it]
 21%|██        | 46182/225000 [97:56:13<352:26:45,  7.10s/it]
 21%|██        | 46183/225000 [97:56:17<309:17:18,  6.23s/it]
 21%|██        | 46184/225000 [97:56:24<322:39:03,  6.50s/it]
 21%|██        | 46185/225000 [97:56:32<347:51:35,  7.00s/it]
 21%|██        | 46186/225000 [97:56:41<367:26:43,  7.40s/it]
 21%|██        | 46187/225000 [97:56:46<335:24:37,  6.75s/it]
 21%|██        | 46188/225000 [97:56:52<323:38:25,  6.52s/it]
 21%|██        | 46189/225000 [97:56:57<306:43:37,  6.18s/it]
 21%|██        | 46190/225000 [97:57:07<357:46:03,  7.20s/it]
 21%|██        | 46191/225000 [97:57:14<351:33:18,  7.08s/it]
 21%|██        | 46192/225000 [97:57:18<313:46:38,  6.32s/it]
 21%|██        | 46193/225000 [97:57:23<299:09:04,  6.02s/it]
 21%|██        | 46194/225000 [97:57:37<416:41:34,  8.39s/it]
 21%|██        | 46195/225000 [97:57:43<381:48:41,  7.69s/it]
 21%|██        | 46196/225000 [97:57:51<381:39:04,  7.68s/it]
 21%|██        | 46197/225000 [97:57:58<368:50:53,  7.43s/it]
 21%|██        | 46198/225000 [97:58:01<310:40:50,  6.26s/it]
 21%|██        | 46199/225000 [97:58:07<300:53:01,  6.06s/it]
 21%|██        | 46200/225000 [97:58:12<289:37:52,  5.83s/it]
                                                             
{'loss': '0.6845', 'grad_norm': '0.8177', 'learning_rate': '4.659e-05', 'epoch': '0.2053', 'num_input_tokens_seen': 164760344, 'train_runtime': '3.527e+05', 'train_tokens_per_second': '467.1'}

 21%|██        | 46200/225000 [97:58:12<289:37:52,  5.83s/it]
 21%|██        | 46201/225000 [97:58:18<283:39:10,  5.71s/it]
 21%|██        | 46202/225000 [97:58:23<282:52:39,  5.70s/it]
 21%|██        | 46203/225000 [97:58:36<386:28:55,  7.78s/it]
 21%|██        | 46204/225000 [97:58:42<357:45:48,  7.20s/it]
 21%|██        | 46205/225000 [97:58:49<362:12:06,  7.29s/it]
 21%|██        | 46206/225000 [97:58:54<326:25:57,  6.57s/it]
 21%|██        | 46207/225000 [97:59:06<396:40:11,  7.99s/it]
 21%|██        | 46208/225000 [97:59:10<336:08:09,  6.77s/it]
 21%|██        | 46209/225000 [97:59:15<320:11:27,  6.45s/it]
 21%|██        | 46210/225000 [97:59:24<361:54:01,  7.29s/it]
 21%|██        | 46211/225000 [97:59:32<372:55:19,  7.51s/it]
 21%|██        | 46212/225000 [97:59:37<323:42:40,  6.52s/it]
 21%|██        | 46213/225000 [97:59:41<286:17:06,  5.76s/it]
 21%|██        | 46214/225000 [97:59:52<369:14:43,  7.44s/it]
 21%|██        | 46215/225000 [97:59:56<316:14:23,  6.37s/it]
 21%|██        | 46216/225000 [98:00:00<283:04:01,  5.70s/it]
 21%|██        | 46217/225000 [98:00:11<367:25:31,  7.40s/it]
 21%|██        | 46218/225000 [98:00:15<314:35:29,  6.33s/it]
 21%|██        | 46219/225000 [98:00:25<359:23:17,  7.24s/it]
 21%|██        | 46220/225000 [98:00:33<378:22:36,  7.62s/it]
 21%|██        | 46221/225000 [98:00:39<359:12:20,  7.23s/it]
 21%|██        | 46222/225000 [98:00:44<313:54:03,  6.32s/it]
 21%|██        | 46223/225000 [98:00:49<301:26:30,  6.07s/it]
 21%|██        | 46224/225000 [98:00:55<303:46:59,  6.12s/it]
 21%|██        | 46225/225000 [98:01:01<300:14:18,  6.05s/it]
 21%|██        | 46226/225000 [98:01:15<409:35:11,  8.25s/it]
 21%|██        | 46227/225000 [98:01:18<343:50:34,  6.92s/it]
 21%|██        | 46228/225000 [98:01:29<392:05:48,  7.90s/it]
 21%|██        | 46229/225000 [98:01:34<352:40:07,  7.10s/it]
 21%|██        | 46230/225000 [98:01:40<333:02:56,  6.71s/it]
 21%|██        | 46231/225000 [98:01:45<319:18:32,  6.43s/it]
 21%|██        | 46232/225000 [98:01:51<308:09:13,  6.21s/it]
 21%|██        | 46233/225000 [98:01:57<297:29:45,  5.99s/it]
 21%|██        | 46234/225000 [98:02:02<288:41:53,  5.81s/it]
 21%|██        | 46235/225000 [98:02:07<278:09:32,  5.60s/it]
 21%|██        | 46236/225000 [98:02:12<270:34:56,  5.45s/it]
 21%|██        | 46237/225000 [98:02:21<324:54:58,  6.54s/it]
 21%|██        | 46238/225000 [98:02:35<425:45:37,  8.57s/it]
 21%|██        | 46239/225000 [98:02:40<372:35:29,  7.50s/it]
 21%|██        | 46240/225000 [98:02:45<344:25:36,  6.94s/it]
 21%|██        | 46241/225000 [98:02:51<327:13:13,  6.59s/it]
 21%|██        | 46242/225000 [98:02:57<312:51:52,  6.30s/it]
 21%|██        | 46243/225000 [98:03:03<306:06:49,  6.16s/it]
 21%|██        | 46244/225000 [98:03:09<304:44:11,  6.14s/it]
 21%|██        | 46245/225000 [98:03:15<305:14:14,  6.15s/it]
 21%|██        | 46246/225000 [98:03:24<345:36:28,  6.96s/it]
 21%|██        | 46247/225000 [98:03:37<441:12:13,  8.89s/it]
 21%|██        | 46248/225000 [98:03:41<372:26:59,  7.50s/it]
 21%|██        | 46249/225000 [98:03:49<376:53:02,  7.59s/it]
 21%|██        | 46250/225000 [98:03:54<339:15:54,  6.83s/it]
                                                             
{'loss': '0.6809', 'grad_norm': '0.7776', 'learning_rate': '4.658e-05', 'epoch': '0.2056', 'num_input_tokens_seen': 164912536, 'train_runtime': '3.53e+05', 'train_tokens_per_second': '467.1'}

 21%|██        | 46250/225000 [98:03:54<339:15:54,  6.83s/it]
 21%|██        | 46251/225000 [98:03:59<305:17:00,  6.15s/it]
 21%|██        | 46252/225000 [98:04:03<284:51:18,  5.74s/it]
 21%|██        | 46253/225000 [98:04:17<396:07:49,  7.98s/it]
 21%|██        | 46254/225000 [98:04:22<360:40:47,  7.26s/it]
 21%|██        | 46255/225000 [98:04:27<322:56:10,  6.50s/it]
 21%|██        | 46256/225000 [98:04:32<300:55:06,  6.06s/it]
 21%|██        | 46257/225000 [98:04:40<327:00:48,  6.59s/it]
 21%|██        | 46258/225000 [98:04:51<396:35:45,  7.99s/it]
 21%|██        | 46259/225000 [98:04:55<336:54:42,  6.79s/it]
 21%|██        | 46260/225000 [98:05:06<398:45:12,  8.03s/it]
 21%|██        | 46261/225000 [98:05:11<346:39:15,  6.98s/it]
 21%|██        | 46262/225000 [98:05:15<312:05:34,  6.29s/it]
 21%|██        | 46263/225000 [98:05:23<328:58:30,  6.63s/it]
 21%|██        | 46264/225000 [98:05:32<376:21:56,  7.58s/it]
 21%|██        | 46265/225000 [98:05:37<338:38:30,  6.82s/it]
 21%|██        | 46266/225000 [98:05:47<373:29:52,  7.52s/it]
 21%|██        | 46267/225000 [98:05:51<327:52:21,  6.60s/it]
 21%|██        | 46268/225000 [98:05:56<297:37:57,  5.99s/it]
 21%|██        | 46269/225000 [98:06:01<280:43:56,  5.65s/it]
 21%|██        | 46270/225000 [98:06:12<370:49:28,  7.47s/it]
 21%|██        | 46271/225000 [98:06:20<373:14:46,  7.52s/it]
 21%|██        | 46272/225000 [98:06:24<321:10:36,  6.47s/it]
 21%|██        | 46273/225000 [98:06:36<401:43:55,  8.09s/it]
 21%|██        | 46274/225000 [98:06:40<346:54:32,  6.99s/it]
 21%|██        | 46275/225000 [98:06:46<335:18:19,  6.75s/it]
 21%|██        | 46276/225000 [98:06:51<309:06:42,  6.23s/it]
 21%|██        | 46277/225000 [98:06:57<295:30:49,  5.95s/it]
 21%|██        | 46278/225000 [98:07:02<283:35:14,  5.71s/it]
 21%|██        | 46279/225000 [98:07:11<327:32:50,  6.60s/it]
 21%|██        | 46280/225000 [98:07:20<374:31:20,  7.54s/it]
 21%|██        | 46281/225000 [98:07:25<336:17:50,  6.77s/it]
 21%|██        | 46282/225000 [98:07:30<302:20:45,  6.09s/it]
 21%|██        | 46283/225000 [98:07:40<368:01:54,  7.41s/it]
 21%|██        | 46284/225000 [98:07:48<376:50:16,  7.59s/it]
 21%|██        | 46285/225000 [98:07:53<332:47:45,  6.70s/it]
 21%|██        | 46286/225000 [98:07:58<306:09:14,  6.17s/it]
 21%|██        | 46287/225000 [98:08:06<329:47:42,  6.64s/it]
 21%|██        | 46288/225000 [98:08:14<360:56:31,  7.27s/it]
 21%|██        | 46289/225000 [98:08:19<319:51:47,  6.44s/it]
 21%|██        | 46290/225000 [98:08:24<299:47:27,  6.04s/it]
 21%|██        | 46291/225000 [98:08:29<291:53:29,  5.88s/it]
 21%|██        | 46292/225000 [98:08:35<288:42:08,  5.82s/it]
 21%|██        | 46293/225000 [98:08:41<283:17:27,  5.71s/it]
 21%|██        | 46294/225000 [98:08:54<395:58:43,  7.98s/it]
 21%|██        | 46295/225000 [98:09:03<419:14:36,  8.45s/it]
 21%|██        | 46296/225000 [98:09:08<362:27:30,  7.30s/it]
 21%|██        | 46297/225000 [98:09:13<328:19:16,  6.61s/it]
 21%|██        | 46298/225000 [98:09:21<344:21:42,  6.94s/it]
 21%|██        | 46299/225000 [98:09:32<410:22:26,  8.27s/it]
 21%|██        | 46300/225000 [98:09:36<348:30:42,  7.02s/it]
                                                             
{'loss': '0.6983', 'grad_norm': '0.7393', 'learning_rate': '4.657e-05', 'epoch': '0.2058', 'num_input_tokens_seen': 165064848, 'train_runtime': '3.534e+05', 'train_tokens_per_second': '467.1'}

 21%|██        | 46300/225000 [98:09:36<348:30:42,  7.02s/it]
 21%|██        | 46301/225000 [98:09:41<319:16:24,  6.43s/it]
 21%|██        | 46302/225000 [98:09:51<374:18:56,  7.54s/it]
 21%|██        | 46303/225000 [98:09:56<330:56:12,  6.67s/it]
 21%|██        | 46304/225000 [98:10:01<303:59:30,  6.12s/it]
 21%|██        | 46305/225000 [98:10:11<366:04:19,  7.37s/it]
 21%|██        | 46306/225000 [98:10:19<373:10:30,  7.52s/it]
 21%|██        | 46307/225000 [98:10:24<333:37:25,  6.72s/it]
 21%|██        | 46308/225000 [98:10:32<350:53:48,  7.07s/it]
 21%|██        | 46309/225000 [98:10:40<370:22:39,  7.46s/it]
 21%|██        | 46310/225000 [98:10:44<320:37:48,  6.46s/it]
 21%|██        | 46311/225000 [98:10:54<369:51:47,  7.45s/it]
 21%|██        | 46312/225000 [98:10:58<325:58:12,  6.57s/it]
 21%|██        | 46313/225000 [98:11:04<305:23:13,  6.15s/it]
 21%|██        | 46314/225000 [98:11:14<374:12:16,  7.54s/it]
 21%|██        | 46315/225000 [98:11:20<350:20:04,  7.06s/it]
 21%|██        | 46316/225000 [98:11:24<306:45:07,  6.18s/it]
 21%|██        | 46317/225000 [98:11:30<290:27:57,  5.85s/it]
 21%|██        | 46318/225000 [98:11:35<279:31:43,  5.63s/it]
 21%|██        | 46319/225000 [98:11:45<347:13:24,  7.00s/it]
 21%|██        | 46320/225000 [98:11:53<371:28:24,  7.48s/it]
 21%|██        | 46321/225000 [98:11:58<321:59:23,  6.49s/it]
 21%|██        | 46322/225000 [98:12:07<366:30:48,  7.38s/it]
 21%|██        | 46323/225000 [98:12:11<315:56:48,  6.37s/it]
 21%|██        | 46324/225000 [98:12:21<370:19:33,  7.46s/it]
 21%|██        | 46325/225000 [98:12:27<345:49:59,  6.97s/it]
 21%|██        | 46326/225000 [98:12:33<327:33:26,  6.60s/it]
 21%|██        | 46327/225000 [98:12:38<309:21:45,  6.23s/it]
 21%|██        | 46328/225000 [98:12:43<296:58:09,  5.98s/it]
 21%|██        | 46329/225000 [98:12:57<405:23:38,  8.17s/it]
 21%|██        | 46330/225000 [98:13:03<374:02:00,  7.54s/it]
 21%|██        | 46331/225000 [98:13:08<335:21:10,  6.76s/it]
 21%|██        | 46332/225000 [98:13:14<329:47:47,  6.65s/it]
 21%|██        | 46333/225000 [98:13:26<401:15:21,  8.08s/it]
 21%|██        | 46334/225000 [98:13:31<356:43:22,  7.19s/it]
 21%|██        | 46335/225000 [98:13:36<329:56:17,  6.65s/it]
 21%|██        | 46336/225000 [98:13:45<360:39:23,  7.27s/it]
 21%|██        | 46337/225000 [98:13:56<414:25:56,  8.35s/it]
 21%|██        | 46338/225000 [98:14:01<366:59:35,  7.39s/it]
 21%|██        | 46339/225000 [98:14:06<337:33:26,  6.80s/it]
 21%|██        | 46340/225000 [98:14:11<314:25:10,  6.34s/it]
 21%|██        | 46341/225000 [98:14:18<313:00:49,  6.31s/it]
 21%|██        | 46342/225000 [98:14:31<416:45:10,  8.40s/it]
 21%|██        | 46343/225000 [98:14:35<346:43:38,  6.99s/it]
 21%|██        | 46344/225000 [98:14:43<371:26:41,  7.48s/it]
 21%|██        | 46345/225000 [98:14:48<326:27:51,  6.58s/it]
 21%|██        | 46346/225000 [98:14:57<370:09:36,  7.46s/it]
 21%|██        | 46347/225000 [98:15:02<332:11:02,  6.69s/it]
 21%|██        | 46348/225000 [98:15:06<295:19:22,  5.95s/it]
 21%|██        | 46349/225000 [98:15:14<316:57:41,  6.39s/it]
 21%|██        | 46350/225000 [98:15:22<347:45:26,  7.01s/it]
                                                             
{'loss': '0.7079', 'grad_norm': '0.695', 'learning_rate': '4.656e-05', 'epoch': '0.206', 'num_input_tokens_seen': 165217240, 'train_runtime': '3.537e+05', 'train_tokens_per_second': '467.1'}

 21%|██        | 46350/225000 [98:15:22<347:45:26,  7.01s/it]
 21%|██        | 46351/225000 [98:15:29<339:40:06,  6.84s/it]
 21%|██        | 46352/225000 [98:15:37<353:08:54,  7.12s/it]
 21%|██        | 46353/225000 [98:15:43<336:31:13,  6.78s/it]
 21%|██        | 46354/225000 [98:15:48<310:24:43,  6.26s/it]
 21%|██        | 46355/225000 [98:15:52<288:54:36,  5.82s/it]
 21%|██        | 46356/225000 [98:15:57<273:13:12,  5.51s/it]
 21%|██        | 46357/225000 [98:16:09<367:30:30,  7.41s/it]
 21%|██        | 46358/225000 [98:16:15<351:08:13,  7.08s/it]
 21%|██        | 46359/225000 [98:16:23<360:50:06,  7.27s/it]
 21%|██        | 46360/225000 [98:16:31<377:51:37,  7.61s/it]
 21%|██        | 46361/225000 [98:16:36<336:44:24,  6.79s/it]
 21%|██        | 46362/225000 [98:16:44<350:02:21,  7.05s/it]
 21%|██        | 46363/225000 [98:16:49<316:16:22,  6.37s/it]
 21%|██        | 46364/225000 [98:16:54<294:55:00,  5.94s/it]
 21%|██        | 46365/225000 [98:16:59<279:47:50,  5.64s/it]
 21%|██        | 46366/225000 [98:17:04<276:47:36,  5.58s/it]
 21%|██        | 46367/225000 [98:17:17<390:22:46,  7.87s/it]
 21%|██        | 46368/225000 [98:17:22<349:57:34,  7.05s/it]
 21%|██        | 46369/225000 [98:17:27<316:07:23,  6.37s/it]
 21%|██        | 46370/225000 [98:17:40<406:50:29,  8.20s/it]
 21%|██        | 46371/225000 [98:17:45<359:36:12,  7.25s/it]
 21%|██        | 46372/225000 [98:17:50<325:24:37,  6.56s/it]
 21%|██        | 46373/225000 [98:17:55<306:47:02,  6.18s/it]
 21%|██        | 46374/225000 [98:18:00<286:14:02,  5.77s/it]
 21%|██        | 46375/225000 [98:18:11<361:52:18,  7.29s/it]
 21%|██        | 46376/225000 [98:18:18<358:26:02,  7.22s/it]
 21%|██        | 46377/225000 [98:18:27<385:21:54,  7.77s/it]
 21%|██        | 46378/225000 [98:18:32<352:52:52,  7.11s/it]
 21%|██        | 46379/225000 [98:18:40<362:09:19,  7.30s/it]
 21%|██        | 46380/225000 [98:18:46<337:38:04,  6.80s/it]
 21%|██        | 46381/225000 [98:18:53<350:50:11,  7.07s/it]
 21%|██        | 46382/225000 [98:18:58<313:29:36,  6.32s/it]
 21%|██        | 46383/225000 [98:19:07<354:11:40,  7.14s/it]
 21%|██        | 46384/225000 [98:19:13<334:05:52,  6.73s/it]
 21%|██        | 46385/225000 [98:19:20<336:40:04,  6.79s/it]
 21%|██        | 46386/225000 [98:19:27<350:10:14,  7.06s/it]
 21%|██        | 46387/225000 [98:19:32<311:23:55,  6.28s/it]
 21%|██        | 46388/225000 [98:19:40<341:54:50,  6.89s/it]
 21%|██        | 46389/225000 [98:19:49<378:49:47,  7.64s/it]
 21%|██        | 46390/225000 [98:19:57<378:35:25,  7.63s/it]
 21%|██        | 46391/225000 [98:20:02<344:00:45,  6.93s/it]
 21%|██        | 46392/225000 [98:20:08<317:44:04,  6.40s/it]
 21%|██        | 46393/225000 [98:20:13<302:00:35,  6.09s/it]
 21%|██        | 46394/225000 [98:20:22<350:31:56,  7.07s/it]
 21%|██        | 46395/225000 [98:20:29<339:46:12,  6.85s/it]
 21%|██        | 46396/225000 [98:20:33<307:59:35,  6.21s/it]
 21%|██        | 46397/225000 [98:20:38<289:38:10,  5.84s/it]
 21%|██        | 46398/225000 [98:20:50<378:41:03,  7.63s/it]
 21%|██        | 46399/225000 [98:20:57<369:02:53,  7.44s/it]
 21%|██        | 46400/225000 [98:21:01<315:55:49,  6.37s/it]
                                                             
{'loss': '0.6891', 'grad_norm': '0.7807', 'learning_rate': '4.655e-05', 'epoch': '0.2062', 'num_input_tokens_seen': 165368768, 'train_runtime': '3.541e+05', 'train_tokens_per_second': '467.1'}

 21%|██        | 46400/225000 [98:21:01<315:55:49,  6.37s/it]
 21%|██        | 46401/225000 [98:21:14<412:16:46,  8.31s/it]
 21%|██        | 46402/225000 [98:21:18<345:05:21,  6.96s/it]
 21%|██        | 46403/225000 [98:21:26<366:37:59,  7.39s/it]
 21%|██        | 46404/225000 [98:21:31<333:36:25,  6.72s/it]
 21%|██        | 46405/225000 [98:21:36<305:49:03,  6.16s/it]
 21%|██        | 46406/225000 [98:21:43<315:08:49,  6.35s/it]
 21%|██        | 46407/225000 [98:21:54<391:15:12,  7.89s/it]
 21%|██        | 46408/225000 [98:22:02<381:22:12,  7.69s/it]
 21%|██        | 46409/225000 [98:22:09<374:05:16,  7.54s/it]
 21%|██        | 46410/225000 [98:22:16<375:35:59,  7.57s/it]
 21%|██        | 46411/225000 [98:22:20<320:26:02,  6.46s/it]
 21%|██        | 46412/225000 [98:22:28<345:21:26,  6.96s/it]
 21%|██        | 46413/225000 [98:22:35<347:11:58,  7.00s/it]
 21%|██        | 46414/225000 [98:22:40<305:01:12,  6.15s/it]
 21%|██        | 46415/225000 [98:22:44<273:25:11,  5.51s/it]
 21%|██        | 46416/225000 [98:22:56<372:24:49,  7.51s/it]
 21%|██        | 46417/225000 [98:23:01<335:27:08,  6.76s/it]
 21%|██        | 46418/225000 [98:23:09<358:52:31,  7.23s/it]
 21%|██        | 46419/225000 [98:23:13<315:22:33,  6.36s/it]
 21%|██        | 46420/225000 [98:23:19<306:23:30,  6.18s/it]
 21%|██        | 46421/225000 [98:23:25<301:36:21,  6.08s/it]
 21%|██        | 46422/225000 [98:23:32<306:24:04,  6.18s/it]
 21%|██        | 46423/225000 [98:23:38<313:24:10,  6.32s/it]
 21%|██        | 46424/225000 [98:23:51<409:18:38,  8.25s/it]
 21%|██        | 46425/225000 [98:23:58<388:04:03,  7.82s/it]
 21%|██        | 46426/225000 [98:24:02<328:39:26,  6.63s/it]
 21%|██        | 46427/225000 [98:24:08<330:26:58,  6.66s/it]
 21%|██        | 46428/225000 [98:24:18<381:51:08,  7.70s/it]
 21%|██        | 46429/225000 [98:24:23<329:45:26,  6.65s/it]
 21%|██        | 46430/225000 [98:24:31<354:41:10,  7.15s/it]
 21%|██        | 46431/225000 [98:24:38<353:32:58,  7.13s/it]
 21%|██        | 46432/225000 [98:24:46<361:01:46,  7.28s/it]
 21%|██        | 46433/225000 [98:24:50<313:45:20,  6.33s/it]
 21%|██        | 46434/225000 [98:24:59<360:15:38,  7.26s/it]
 21%|██        | 46435/225000 [98:25:04<326:06:32,  6.57s/it]
 21%|██        | 46436/225000 [98:25:09<305:00:25,  6.15s/it]
 21%|██        | 46437/225000 [98:25:15<301:15:56,  6.07s/it]
 21%|██        | 46438/225000 [98:25:21<302:44:48,  6.10s/it]
 21%|██        | 46439/225000 [98:25:27<293:06:04,  5.91s/it]
 21%|██        | 46440/225000 [98:25:40<395:26:38,  7.97s/it]
 21%|██        | 46441/225000 [98:25:47<381:20:37,  7.69s/it]
 21%|██        | 46442/225000 [98:25:52<352:31:59,  7.11s/it]
 21%|██        | 46443/225000 [98:26:01<380:26:23,  7.67s/it]
 21%|██        | 46444/225000 [98:26:05<326:39:05,  6.59s/it]
 21%|██        | 46445/225000 [98:26:09<287:52:32,  5.80s/it]
 21%|██        | 46446/225000 [98:26:20<362:29:17,  7.31s/it]
 21%|██        | 46447/225000 [98:26:26<337:13:10,  6.80s/it]
 21%|██        | 46448/225000 [98:26:35<377:31:39,  7.61s/it]
 21%|██        | 46449/225000 [98:26:40<339:22:51,  6.84s/it]
 21%|██        | 46450/225000 [98:26:46<315:56:13,  6.37s/it]
                                                             
{'loss': '0.699', 'grad_norm': '0.6587', 'learning_rate': '4.654e-05', 'epoch': '0.2064', 'num_input_tokens_seen': 165520016, 'train_runtime': '3.544e+05', 'train_tokens_per_second': '467'}

 21%|██        | 46450/225000 [98:26:46<315:56:13,  6.37s/it]
 21%|██        | 46451/225000 [98:26:50<292:01:07,  5.89s/it]
 21%|██        | 46452/225000 [98:26:56<280:53:43,  5.66s/it]
 21%|██        | 46453/225000 [98:27:01<275:09:42,  5.55s/it]
 21%|██        | 46454/225000 [98:27:11<347:34:32,  7.01s/it]
 21%|██        | 46455/225000 [98:27:23<416:27:23,  8.40s/it]
 21%|██        | 46456/225000 [98:27:31<406:39:45,  8.20s/it]
 21%|██        | 46457/225000 [98:27:36<362:34:12,  7.31s/it]
 21%|██        | 46458/225000 [98:27:45<392:08:21,  7.91s/it]
 21%|██        | 46459/225000 [98:27:49<334:51:28,  6.75s/it]
 21%|██        | 46460/225000 [98:27:58<364:06:24,  7.34s/it]
 21%|██        | 46461/225000 [98:28:02<321:52:10,  6.49s/it]
 21%|██        | 46462/225000 [98:28:07<290:55:07,  5.87s/it]
 21%|██        | 46463/225000 [98:28:16<333:13:40,  6.72s/it]
 21%|██        | 46464/225000 [98:28:25<372:18:34,  7.51s/it]
 21%|██        | 46465/225000 [98:28:29<326:13:48,  6.58s/it]
 21%|██        | 46466/225000 [98:28:38<361:57:04,  7.30s/it]
 21%|██        | 46467/225000 [98:28:42<310:54:51,  6.27s/it]
 21%|██        | 46468/225000 [98:28:54<386:13:40,  7.79s/it]
 21%|██        | 46469/225000 [98:29:01<381:40:19,  7.70s/it]
 21%|██        | 46470/225000 [98:29:04<318:34:29,  6.42s/it]
 21%|██        | 46471/225000 [98:29:09<290:40:00,  5.86s/it]
 21%|██        | 46472/225000 [98:29:14<274:41:24,  5.54s/it]
 21%|██        | 46473/225000 [98:29:19<269:03:56,  5.43s/it]
 21%|██        | 46474/225000 [98:29:32<382:22:14,  7.71s/it]
 21%|██        | 46475/225000 [98:29:38<355:59:55,  7.18s/it]
 21%|██        | 46476/225000 [98:29:43<322:44:54,  6.51s/it]
 21%|██        | 46477/225000 [98:29:48<296:28:55,  5.98s/it]
 21%|██        | 46478/225000 [98:29:56<337:32:49,  6.81s/it]
 21%|██        | 46479/225000 [98:30:08<402:20:59,  8.11s/it]
 21%|██        | 46480/225000 [98:30:14<382:21:43,  7.71s/it]
 21%|██        | 46481/225000 [98:30:19<331:55:31,  6.69s/it]
 21%|██        | 46482/225000 [98:30:24<308:38:05,  6.22s/it]
 21%|██        | 46483/225000 [98:30:29<297:08:41,  5.99s/it]
 21%|██        | 46484/225000 [98:30:40<366:49:25,  7.40s/it]
 21%|██        | 46485/225000 [98:30:45<326:38:20,  6.59s/it]
 21%|██        | 46486/225000 [98:30:53<360:24:53,  7.27s/it]
 21%|██        | 46487/225000 [98:30:58<322:44:39,  6.51s/it]
 21%|██        | 46488/225000 [98:31:03<300:17:12,  6.06s/it]
 21%|██        | 46489/225000 [98:31:08<278:34:40,  5.62s/it]
 21%|██        | 46490/225000 [98:31:21<390:34:53,  7.88s/it]
 21%|██        | 46491/225000 [98:31:26<344:08:51,  6.94s/it]
 21%|██        | 46492/225000 [98:31:30<311:04:09,  6.27s/it]
 21%|██        | 46493/225000 [98:31:36<299:57:23,  6.05s/it]
 21%|██        | 46494/225000 [98:31:47<372:23:57,  7.51s/it]
 21%|██        | 46495/225000 [98:31:56<401:45:37,  8.10s/it]
 21%|██        | 46496/225000 [98:32:03<383:50:59,  7.74s/it]
 21%|██        | 46497/225000 [98:32:08<333:03:47,  6.72s/it]
 21%|██        | 46498/225000 [98:32:17<378:24:23,  7.63s/it]
 21%|██        | 46499/225000 [98:32:22<328:00:06,  6.62s/it]
 21%|██        | 46500/225000 [98:32:28<319:23:57,  6.44s/it]
                                                             
{'loss': '0.7011', 'grad_norm': '0.8498', 'learning_rate': '4.653e-05', 'epoch': '0.2067', 'num_input_tokens_seen': 165671200, 'train_runtime': '3.547e+05', 'train_tokens_per_second': '467'}

 21%|██        | 46500/225000 [98:32:28<319:23:57,  6.44s/it]
 21%|██        | 46501/225000 [98:32:34<320:59:00,  6.47s/it]
 21%|██        | 46502/225000 [98:32:42<343:14:47,  6.92s/it]
 21%|██        | 46503/225000 [98:32:48<328:25:50,  6.62s/it]
 21%|██        | 46504/225000 [98:32:55<334:41:30,  6.75s/it]
 21%|██        | 46505/225000 [98:33:02<341:51:53,  6.89s/it]
 21%|██        | 46506/225000 [98:33:08<316:29:57,  6.38s/it]
 21%|██        | 46507/225000 [98:33:17<366:23:00,  7.39s/it]
 21%|██        | 46508/225000 [98:33:21<310:40:47,  6.27s/it]
 21%|██        | 46509/225000 [98:33:30<352:39:47,  7.11s/it]
 21%|██        | 46510/225000 [98:33:37<354:49:58,  7.16s/it]
 21%|██        | 46511/225000 [98:33:44<341:56:37,  6.90s/it]
 21%|██        | 46512/225000 [98:33:52<361:34:18,  7.29s/it]
 21%|██        | 46513/225000 [98:33:58<348:38:54,  7.03s/it]
 21%|██        | 46514/225000 [98:34:04<338:07:19,  6.82s/it]
 21%|██        | 46515/225000 [98:34:10<316:22:53,  6.38s/it]
 21%|██        | 46516/225000 [98:34:18<336:58:56,  6.80s/it]
 21%|██        | 46517/225000 [98:34:23<317:02:55,  6.39s/it]
 21%|██        | 46518/225000 [98:34:32<353:54:18,  7.14s/it]
 21%|██        | 46519/225000 [98:34:36<310:46:52,  6.27s/it]
 21%|██        | 46520/225000 [98:34:43<315:04:52,  6.36s/it]
 21%|██        | 46521/225000 [98:34:52<364:49:32,  7.36s/it]
 21%|██        | 46522/225000 [98:34:57<316:59:20,  6.39s/it]
 21%|██        | 46523/225000 [98:35:01<293:27:34,  5.92s/it]
 21%|██        | 46524/225000 [98:35:06<277:24:22,  5.60s/it]
 21%|██        | 46525/225000 [98:35:17<350:02:18,  7.06s/it]
 21%|██        | 46526/225000 [98:35:23<343:45:52,  6.93s/it]
 21%|██        | 46527/225000 [98:35:32<361:54:14,  7.30s/it]
 21%|██        | 46528/225000 [98:35:38<353:33:50,  7.13s/it]
 21%|██        | 46529/225000 [98:35:43<319:20:23,  6.44s/it]
 21%|██        | 46530/225000 [98:35:48<291:27:02,  5.88s/it]
 21%|██        | 46531/225000 [98:35:57<338:40:17,  6.83s/it]
 21%|██        | 46532/225000 [98:36:04<339:48:18,  6.85s/it]
 21%|██        | 46533/225000 [98:36:11<350:38:08,  7.07s/it]
 21%|██        | 46534/225000 [98:36:17<330:59:32,  6.68s/it]
 21%|██        | 46535/225000 [98:36:22<309:59:45,  6.25s/it]
 21%|██        | 46536/225000 [98:36:27<290:34:59,  5.86s/it]
 21%|██        | 46537/225000 [98:36:33<283:34:02,  5.72s/it]
 21%|██        | 46538/225000 [98:36:37<268:13:31,  5.41s/it]
 21%|██        | 46539/225000 [98:36:48<348:16:27,  7.03s/it]
 21%|██        | 46540/225000 [98:36:58<391:14:15,  7.89s/it]
 21%|██        | 46541/225000 [98:37:02<332:25:01,  6.71s/it]
 21%|██        | 46542/225000 [98:37:14<410:27:34,  8.28s/it]
 21%|██        | 46543/225000 [98:37:18<343:22:58,  6.93s/it]
 21%|██        | 46544/225000 [98:37:27<382:51:59,  7.72s/it]
 21%|██        | 46545/225000 [98:37:32<332:19:28,  6.70s/it]
 21%|██        | 46546/225000 [98:37:37<314:54:25,  6.35s/it]
 21%|██        | 46547/225000 [98:37:42<293:32:10,  5.92s/it]
 21%|██        | 46548/225000 [98:37:47<282:15:14,  5.69s/it]
 21%|██        | 46549/225000 [98:37:57<346:24:59,  6.99s/it]
 21%|██        | 46550/225000 [98:38:06<381:45:56,  7.70s/it]
                                                             
{'loss': '0.6669', 'grad_norm': '0.7804', 'learning_rate': '4.652e-05', 'epoch': '0.2069', 'num_input_tokens_seen': 165820784, 'train_runtime': '3.551e+05', 'train_tokens_per_second': '467'}

 21%|██        | 46550/225000 [98:38:06<381:45:56,  7.70s/it]
 21%|██        | 46551/225000 [98:38:11<332:13:51,  6.70s/it]
 21%|██        | 46552/225000 [98:38:15<299:09:04,  6.04s/it]
 21%|██        | 46553/225000 [98:38:27<379:37:33,  7.66s/it]
 21%|██        | 46554/225000 [98:38:31<324:15:44,  6.54s/it]
 21%|██        | 46555/225000 [98:38:41<375:49:38,  7.58s/it]
 21%|██        | 46556/225000 [98:38:46<346:44:45,  7.00s/it]
 21%|██        | 46557/225000 [98:38:54<350:41:11,  7.07s/it]
 21%|██        | 46558/225000 [98:38:58<306:56:07,  6.19s/it]
 21%|██        | 46559/225000 [98:39:06<342:37:45,  6.91s/it]
 21%|██        | 46560/225000 [98:39:12<329:51:33,  6.65s/it]
 21%|██        | 46561/225000 [98:39:17<302:58:13,  6.11s/it]
 21%|██        | 46562/225000 [98:39:28<364:44:14,  7.36s/it]
 21%|██        | 46563/225000 [98:39:35<364:00:25,  7.34s/it]
 21%|██        | 46564/225000 [98:39:39<321:00:22,  6.48s/it]
 21%|██        | 46565/225000 [98:39:49<369:12:17,  7.45s/it]
 21%|██        | 46566/225000 [98:39:54<326:44:03,  6.59s/it]
 21%|██        | 46567/225000 [98:39:58<300:05:20,  6.05s/it]
 21%|██        | 46568/225000 [98:40:03<275:49:59,  5.57s/it]
 21%|██        | 46569/225000 [98:40:15<379:31:32,  7.66s/it]
 21%|██        | 46570/225000 [98:40:22<360:29:51,  7.27s/it]
 21%|██        | 46571/225000 [98:40:28<350:49:58,  7.08s/it]
 21%|██        | 46572/225000 [98:40:32<302:05:31,  6.10s/it]
 21%|██        | 46573/225000 [98:40:39<319:46:54,  6.45s/it]
 21%|██        | 46574/225000 [98:40:47<341:56:20,  6.90s/it]
 21%|██        | 46575/225000 [98:40:52<306:38:50,  6.19s/it]
 21%|██        | 46576/225000 [98:40:56<278:25:41,  5.62s/it]
 21%|██        | 46577/225000 [98:41:08<374:02:14,  7.55s/it]
 21%|██        | 46578/225000 [98:41:13<326:00:45,  6.58s/it]
 21%|██        | 46579/225000 [98:41:18<312:43:37,  6.31s/it]
 21%|██        | 46580/225000 [98:41:26<337:49:39,  6.82s/it]
 21%|██        | 46581/225000 [98:41:32<321:07:35,  6.48s/it]
 21%|██        | 46582/225000 [98:41:37<306:03:20,  6.18s/it]
 21%|██        | 46583/225000 [98:41:45<334:07:05,  6.74s/it]
 21%|██        | 46584/225000 [98:41:54<364:43:57,  7.36s/it]
 21%|██        | 46585/225000 [98:41:58<311:28:54,  6.28s/it]
 21%|██        | 46586/225000 [98:42:03<292:39:52,  5.91s/it]
 21%|██        | 46587/225000 [98:42:08<273:49:00,  5.53s/it]
 21%|██        | 46588/225000 [98:42:19<363:45:12,  7.34s/it]
 21%|██        | 46589/225000 [98:42:26<352:49:38,  7.12s/it]
 21%|██        | 46590/225000 [98:42:34<363:07:32,  7.33s/it]
 21%|██        | 46591/225000 [98:42:37<310:46:36,  6.27s/it]
 21%|██        | 46592/225000 [98:42:42<287:40:14,  5.80s/it]
 21%|██        | 46593/225000 [98:42:48<285:01:07,  5.75s/it]
 21%|██        | 46594/225000 [98:42:54<284:09:40,  5.73s/it]
 21%|██        | 46595/225000 [98:42:59<284:12:51,  5.74s/it]
 21%|██        | 46596/225000 [98:43:09<345:05:58,  6.96s/it]
 21%|██        | 46597/225000 [98:43:18<380:14:12,  7.67s/it]
 21%|██        | 46598/225000 [98:43:22<321:34:07,  6.49s/it]
 21%|██        | 46599/225000 [98:43:27<296:30:03,  5.98s/it]
 21%|██        | 46600/225000 [98:43:36<335:19:35,  6.77s/it]
                                                             
{'loss': '0.7005', 'grad_norm': '0.7263', 'learning_rate': '4.651e-05', 'epoch': '0.2071', 'num_input_tokens_seen': 165970976, 'train_runtime': '3.554e+05', 'train_tokens_per_second': '467'}

 21%|██        | 46600/225000 [98:43:36<335:19:35,  6.77s/it]
 21%|██        | 46601/225000 [98:43:47<398:30:41,  8.04s/it]
 21%|██        | 46602/225000 [98:43:51<345:31:21,  6.97s/it]
 21%|██        | 46603/225000 [98:43:57<336:51:47,  6.80s/it]
 21%|██        | 46604/225000 [98:44:08<398:47:00,  8.05s/it]
 21%|██        | 46605/225000 [98:44:14<363:42:28,  7.34s/it]
 21%|██        | 46606/225000 [98:44:20<338:10:09,  6.82s/it]
 21%|██        | 46607/225000 [98:44:28<354:30:40,  7.15s/it]
 21%|██        | 46608/225000 [98:44:32<320:17:24,  6.46s/it]
 21%|██        | 46609/225000 [98:44:41<357:42:20,  7.22s/it]
 21%|██        | 46610/225000 [98:44:46<324:22:10,  6.55s/it]
 21%|██        | 46611/225000 [98:44:55<351:40:32,  7.10s/it]
 21%|██        | 46612/225000 [98:45:03<369:13:40,  7.45s/it]
 21%|██        | 46613/225000 [98:45:08<324:10:43,  6.54s/it]
 21%|██        | 46614/225000 [98:45:13<306:36:54,  6.19s/it]
 21%|██        | 46615/225000 [98:45:24<376:58:18,  7.61s/it]
 21%|██        | 46616/225000 [98:45:31<376:08:36,  7.59s/it]
 21%|██        | 46617/225000 [98:45:36<336:01:54,  6.78s/it]
 21%|██        | 46618/225000 [98:45:42<315:22:30,  6.36s/it]
 21%|██        | 46619/225000 [98:45:47<303:32:49,  6.13s/it]
 21%|██        | 46620/225000 [98:45:52<282:50:29,  5.71s/it]
 21%|██        | 46621/225000 [98:45:57<272:57:23,  5.51s/it]
 21%|██        | 46622/225000 [98:46:10<389:01:22,  7.85s/it]
 21%|██        | 46623/225000 [98:46:14<334:26:40,  6.75s/it]
 21%|██        | 46624/225000 [98:46:21<325:47:09,  6.58s/it]
 21%|██        | 46625/225000 [98:46:31<386:27:30,  7.80s/it]
 21%|██        | 46626/225000 [98:46:35<328:18:27,  6.63s/it]
 21%|██        | 46627/225000 [98:46:40<295:04:35,  5.96s/it]
 21%|██        | 46628/225000 [98:46:51<381:13:08,  7.69s/it]
 21%|██        | 46629/225000 [98:46:55<321:27:46,  6.49s/it]
 21%|██        | 46630/225000 [98:47:06<381:16:12,  7.70s/it]
 21%|██        | 46631/225000 [98:47:10<338:25:49,  6.83s/it]
 21%|██        | 46632/225000 [98:47:16<321:46:32,  6.49s/it]
 21%|██        | 46633/225000 [98:47:27<384:12:09,  7.75s/it]
 21%|██        | 46634/225000 [98:47:31<333:11:32,  6.72s/it]
 21%|██        | 46635/225000 [98:47:36<310:19:35,  6.26s/it]
 21%|██        | 46636/225000 [98:47:41<287:08:09,  5.80s/it]
 21%|██        | 46637/225000 [98:47:54<401:57:47,  8.11s/it]
 21%|██        | 46638/225000 [98:47:58<336:27:18,  6.79s/it]
 21%|██        | 46639/225000 [98:48:05<332:45:37,  6.72s/it]
 21%|██        | 46640/225000 [98:48:10<308:11:56,  6.22s/it]
 21%|██        | 46641/225000 [98:48:16<307:10:23,  6.20s/it]
 21%|██        | 46642/225000 [98:48:28<391:36:19,  7.90s/it]
 21%|██        | 46643/225000 [98:48:32<336:55:30,  6.80s/it]
 21%|██        | 46644/225000 [98:48:41<369:30:29,  7.46s/it]
 21%|██        | 46645/225000 [98:48:46<325:47:32,  6.58s/it]
 21%|██        | 46646/225000 [98:48:50<293:13:17,  5.92s/it]
 21%|██        | 46647/225000 [98:48:59<340:32:33,  6.87s/it]
 21%|██        | 46648/225000 [98:49:08<368:57:04,  7.45s/it]
 21%|██        | 46649/225000 [98:49:12<320:39:10,  6.47s/it]
 21%|██        | 46650/225000 [98:49:20<342:31:35,  6.91s/it]
                                                             
{'loss': '0.6987', 'grad_norm': '1.038', 'learning_rate': '4.65e-05', 'epoch': '0.2073', 'num_input_tokens_seen': 166121912, 'train_runtime': '3.558e+05', 'train_tokens_per_second': '466.9'}

 21%|██        | 46650/225000 [98:49:20<342:31:35,  6.91s/it]
 21%|██        | 46651/225000 [98:49:24<304:34:26,  6.15s/it]
 21%|██        | 46652/225000 [98:49:29<284:13:33,  5.74s/it]
 21%|██        | 46653/225000 [98:49:33<262:24:42,  5.30s/it]
 21%|██        | 46654/225000 [98:49:46<366:30:12,  7.40s/it]
 21%|██        | 46655/225000 [98:49:52<356:04:51,  7.19s/it]
 21%|██        | 46656/225000 [98:49:57<319:59:29,  6.46s/it]
 21%|██        | 46657/225000 [98:50:02<298:48:20,  6.03s/it]
 21%|██        | 46658/225000 [98:50:15<394:44:54,  7.97s/it]
 21%|██        | 46659/225000 [98:50:20<350:08:08,  7.07s/it]
 21%|██        | 46660/225000 [98:50:25<321:31:30,  6.49s/it]
 21%|██        | 46661/225000 [98:50:29<293:12:55,  5.92s/it]
 21%|██        | 46662/225000 [98:50:34<272:50:03,  5.51s/it]
 21%|██        | 46663/225000 [98:50:47<389:18:45,  7.86s/it]
 21%|██        | 46664/225000 [98:50:52<340:28:54,  6.87s/it]
 21%|██        | 46665/225000 [98:50:59<337:53:13,  6.82s/it]
 21%|██        | 46666/225000 [98:51:04<321:38:28,  6.49s/it]
 21%|██        | 46667/225000 [98:51:13<350:05:45,  7.07s/it]
 21%|██        | 46668/225000 [98:51:16<301:23:44,  6.08s/it]
 21%|██        | 46669/225000 [98:51:24<327:40:00,  6.61s/it]
 21%|██        | 46670/225000 [98:51:33<361:31:40,  7.30s/it]
 21%|██        | 46671/225000 [98:51:38<320:51:28,  6.48s/it]
 21%|██        | 46672/225000 [98:51:42<290:31:32,  5.86s/it]
 21%|██        | 46673/225000 [98:51:51<332:47:14,  6.72s/it]
 21%|██        | 46674/225000 [98:52:00<370:21:34,  7.48s/it]
 21%|██        | 46675/225000 [98:52:06<339:59:53,  6.86s/it]
 21%|██        | 46676/225000 [98:52:11<318:13:24,  6.42s/it]
 21%|██        | 46677/225000 [98:52:17<304:57:27,  6.16s/it]
 21%|██        | 46678/225000 [98:52:22<292:40:17,  5.91s/it]
 21%|██        | 46679/225000 [98:52:27<286:10:50,  5.78s/it]
 21%|██        | 46680/225000 [98:52:38<363:29:43,  7.34s/it]
 21%|██        | 46681/225000 [98:52:48<399:34:04,  8.07s/it]
 21%|██        | 46682/225000 [98:52:53<360:23:14,  7.28s/it]
 21%|██        | 46683/225000 [98:52:59<335:23:16,  6.77s/it]
 21%|██        | 46684/225000 [98:53:04<312:55:57,  6.32s/it]
 21%|██        | 46685/225000 [98:53:10<301:04:35,  6.08s/it]
 21%|██        | 46686/225000 [98:53:16<299:16:36,  6.04s/it]
 21%|██        | 46687/225000 [98:53:21<288:56:41,  5.83s/it]
 21%|██        | 46688/225000 [98:53:35<402:21:32,  8.12s/it]
 21%|██        | 46689/225000 [98:53:41<381:46:02,  7.71s/it]
 21%|██        | 46690/225000 [98:53:46<336:23:59,  6.79s/it]
 21%|██        | 46691/225000 [98:53:57<396:58:53,  8.01s/it]
 21%|██        | 46692/225000 [98:54:02<360:42:28,  7.28s/it]
 21%|██        | 46693/225000 [98:54:08<336:17:29,  6.79s/it]
 21%|██        | 46694/225000 [98:54:17<373:50:57,  7.55s/it]
 21%|██        | 46695/225000 [98:54:24<360:50:29,  7.29s/it]
 21%|██        | 46696/225000 [98:54:29<324:02:48,  6.54s/it]
 21%|██        | 46697/225000 [98:54:33<294:29:58,  5.95s/it]
 21%|██        | 46698/225000 [98:54:43<347:34:01,  7.02s/it]
 21%|██        | 46699/225000 [98:54:50<353:54:57,  7.15s/it]
 21%|██        | 46700/225000 [98:54:59<371:36:13,  7.50s/it]
                                                             
{'loss': '0.6775', 'grad_norm': '0.7407', 'learning_rate': '4.649e-05', 'epoch': '0.2076', 'num_input_tokens_seen': 166272008, 'train_runtime': '3.561e+05', 'train_tokens_per_second': '466.9'}

 21%|██        | 46700/225000 [98:54:59<371:36:13,  7.50s/it]
 21%|██        | 46701/225000 [98:55:03<330:13:39,  6.67s/it]
 21%|██        | 46702/225000 [98:55:09<312:05:57,  6.30s/it]
 21%|██        | 46703/225000 [98:55:15<302:50:12,  6.11s/it]
 21%|██        | 46704/225000 [98:55:24<345:49:01,  6.98s/it]
 21%|██        | 46705/225000 [98:55:29<327:37:41,  6.62s/it]
 21%|██        | 46706/225000 [98:55:38<355:30:48,  7.18s/it]
 21%|██        | 46707/225000 [98:55:45<349:55:57,  7.07s/it]
 21%|██        | 46708/225000 [98:55:52<355:29:18,  7.18s/it]
 21%|██        | 46709/225000 [98:55:58<330:51:32,  6.68s/it]
 21%|██        | 46710/225000 [98:56:08<382:11:45,  7.72s/it]
 21%|██        | 46711/225000 [98:56:13<340:21:17,  6.87s/it]
 21%|██        | 46712/225000 [98:56:18<314:53:16,  6.36s/it]
 21%|██        | 46713/225000 [98:56:25<321:29:02,  6.49s/it]
 21%|██        | 46714/225000 [98:56:34<366:34:47,  7.40s/it]
 21%|██        | 46715/225000 [98:56:42<366:08:57,  7.39s/it]
 21%|██        | 46716/225000 [98:56:47<337:48:03,  6.82s/it]
 21%|██        | 46717/225000 [98:56:52<316:54:07,  6.40s/it]
 21%|██        | 46718/225000 [98:56:58<303:58:25,  6.14s/it]
 21%|██        | 46719/225000 [98:57:08<362:23:22,  7.32s/it]
 21%|██        | 46720/225000 [98:57:16<378:37:52,  7.65s/it]
 21%|██        | 46721/225000 [98:57:22<350:48:20,  7.08s/it]
 21%|██        | 46722/225000 [98:57:30<355:35:59,  7.18s/it]
 21%|██        | 46723/225000 [98:57:35<322:45:12,  6.52s/it]
 21%|██        | 46724/225000 [98:57:40<301:26:34,  6.09s/it]
 21%|██        | 46725/225000 [98:57:44<278:46:01,  5.63s/it]
 21%|██        | 46726/225000 [98:57:56<372:03:58,  7.51s/it]
 21%|██        | 46727/225000 [98:58:04<374:34:49,  7.56s/it]
 21%|██        | 46728/225000 [98:58:11<368:17:29,  7.44s/it]
 21%|██        | 46729/225000 [98:58:15<311:53:03,  6.30s/it]
 21%|██        | 46730/225000 [98:58:24<359:37:08,  7.26s/it]
 21%|██        | 46731/225000 [98:58:31<355:52:12,  7.19s/it]
 21%|██        | 46732/225000 [98:58:37<338:00:06,  6.83s/it]
 21%|██        | 46733/225000 [98:58:46<375:33:27,  7.58s/it]
 21%|██        | 46734/225000 [98:58:52<340:41:11,  6.88s/it]
 21%|██        | 46735/225000 [98:58:57<310:04:51,  6.26s/it]
 21%|██        | 46736/225000 [98:59:02<292:57:30,  5.92s/it]
 21%|██        | 46737/225000 [98:59:07<279:38:27,  5.65s/it]
 21%|██        | 46738/225000 [98:59:16<332:10:00,  6.71s/it]
 21%|██        | 46739/225000 [98:59:29<424:22:41,  8.57s/it]
 21%|██        | 46740/225000 [98:59:33<357:15:11,  7.21s/it]
 21%|██        | 46741/225000 [98:59:40<359:43:18,  7.26s/it]
 21%|██        | 46742/225000 [98:59:48<363:07:38,  7.33s/it]
 21%|██        | 46743/225000 [98:59:56<370:29:49,  7.48s/it]
 21%|██        | 46744/225000 [99:00:00<332:53:11,  6.72s/it]
 21%|██        | 46745/225000 [99:00:06<313:47:23,  6.34s/it]
 21%|██        | 46746/225000 [99:00:11<301:14:31,  6.08s/it]
 21%|██        | 46747/225000 [99:00:17<288:56:13,  5.84s/it]
 21%|██        | 46748/225000 [99:00:22<280:59:03,  5.67s/it]
 21%|██        | 46749/225000 [99:00:27<277:17:57,  5.60s/it]
 21%|██        | 46750/225000 [99:00:38<346:22:32,  7.00s/it]
                                                             
{'loss': '0.6804', 'grad_norm': '0.6304', 'learning_rate': '4.649e-05', 'epoch': '0.2078', 'num_input_tokens_seen': 166423136, 'train_runtime': '3.564e+05', 'train_tokens_per_second': '466.9'}

 21%|██        | 46750/225000 [99:00:38<346:22:32,  7.00s/it]
 21%|██        | 46751/225000 [99:00:48<398:58:58,  8.06s/it]
 21%|██        | 46752/225000 [99:00:53<346:14:18,  6.99s/it]
 21%|██        | 46753/225000 [99:00:58<322:05:25,  6.51s/it]
 21%|██        | 46754/225000 [99:01:03<303:15:24,  6.12s/it]
 21%|██        | 46755/225000 [99:01:12<346:29:59,  7.00s/it]
 21%|██        | 46756/225000 [99:01:24<413:48:54,  8.36s/it]
 21%|██        | 46757/225000 [99:01:28<346:49:11,  7.00s/it]
 21%|██        | 46758/225000 [99:01:33<316:30:17,  6.39s/it]
 21%|██        | 46759/225000 [99:01:38<296:02:54,  5.98s/it]
 21%|██        | 46760/225000 [99:01:43<284:28:37,  5.75s/it]
 21%|██        | 46761/225000 [99:01:49<285:29:59,  5.77s/it]
 21%|██        | 46762/225000 [99:02:02<399:26:29,  8.07s/it]
 21%|██        | 46763/225000 [99:02:08<370:40:14,  7.49s/it]
 21%|██        | 46764/225000 [99:02:16<374:54:18,  7.57s/it]
 21%|██        | 46765/225000 [99:02:24<378:15:48,  7.64s/it]
 21%|██        | 46766/225000 [99:02:29<339:06:06,  6.85s/it]
 21%|██        | 46767/225000 [99:02:37<353:25:58,  7.14s/it]
 21%|██        | 46768/225000 [99:02:41<307:37:44,  6.21s/it]
 21%|██        | 46769/225000 [99:02:46<292:18:50,  5.90s/it]
 21%|██        | 46770/225000 [99:02:54<327:41:08,  6.62s/it]
 21%|██        | 46771/225000 [99:03:02<347:55:31,  7.03s/it]
 21%|██        | 46772/225000 [99:03:11<368:55:19,  7.45s/it]
 21%|██        | 46773/225000 [99:03:17<355:07:46,  7.17s/it]
 21%|██        | 46774/225000 [99:03:21<305:52:21,  6.18s/it]
 21%|██        | 46775/225000 [99:03:27<307:14:47,  6.21s/it]
 21%|██        | 46776/225000 [99:03:36<338:49:17,  6.84s/it]
 21%|██        | 46777/225000 [99:03:44<358:01:44,  7.23s/it]
 21%|██        | 46778/225000 [99:03:48<311:34:02,  6.29s/it]
 21%|██        | 46779/225000 [99:03:57<348:21:29,  7.04s/it]
 21%|██        | 46780/225000 [99:04:02<320:23:05,  6.47s/it]
 21%|██        | 46781/225000 [99:04:08<313:28:55,  6.33s/it]
 21%|██        | 46782/225000 [99:04:13<299:24:49,  6.05s/it]
 21%|██        | 46783/225000 [99:04:18<286:57:23,  5.80s/it]
 21%|██        | 46784/225000 [99:04:24<280:21:01,  5.66s/it]
 21%|██        | 46785/225000 [99:04:29<272:06:59,  5.50s/it]
 21%|██        | 46786/225000 [99:04:42<380:35:41,  7.69s/it]
 21%|██        | 46787/225000 [99:04:51<400:41:53,  8.09s/it]
 21%|██        | 46788/225000 [99:04:55<339:29:46,  6.86s/it]
 21%|██        | 46789/225000 [99:05:07<424:23:20,  8.57s/it]
 21%|██        | 46790/225000 [99:05:12<361:51:53,  7.31s/it]
 21%|██        | 46791/225000 [99:05:20<377:54:58,  7.63s/it]
 21%|██        | 46792/225000 [99:05:24<327:23:15,  6.61s/it]
 21%|██        | 46793/225000 [99:05:33<359:26:30,  7.26s/it]
 21%|██        | 46794/225000 [99:05:41<370:37:53,  7.49s/it]
 21%|██        | 46795/225000 [99:05:48<370:19:11,  7.48s/it]
 21%|██        | 46796/225000 [99:05:55<349:45:39,  7.07s/it]
 21%|██        | 46797/225000 [99:05:59<314:07:42,  6.35s/it]
 21%|██        | 46798/225000 [99:06:08<349:59:17,  7.07s/it]
 21%|██        | 46799/225000 [99:06:12<312:17:28,  6.31s/it]
 21%|██        | 46800/225000 [99:06:23<374:44:24,  7.57s/it]
                                                             
{'loss': '0.677', 'grad_norm': '0.6508', 'learning_rate': '4.648e-05', 'epoch': '0.208', 'num_input_tokens_seen': 166573808, 'train_runtime': '3.568e+05', 'train_tokens_per_second': '466.9'}

 21%|██        | 46800/225000 [99:06:23<374:44:24,  7.57s/it]
 21%|██        | 46801/225000 [99:06:27<326:42:10,  6.60s/it]
 21%|██        | 46802/225000 [99:06:35<345:15:17,  6.97s/it]
 21%|██        | 46803/225000 [99:06:45<383:38:47,  7.75s/it]
 21%|██        | 46804/225000 [99:06:49<324:28:40,  6.56s/it]
 21%|██        | 46805/225000 [99:06:53<296:24:17,  5.99s/it]
 21%|██        | 46806/225000 [99:06:59<286:41:29,  5.79s/it]
 21%|██        | 46807/225000 [99:07:04<280:51:09,  5.67s/it]
 21%|██        | 46808/225000 [99:07:10<290:37:43,  5.87s/it]
 21%|██        | 46809/225000 [99:07:23<394:06:17,  7.96s/it]
 21%|██        | 46810/225000 [99:07:28<356:05:21,  7.19s/it]
 21%|██        | 46811/225000 [99:07:37<377:40:39,  7.63s/it]
 21%|██        | 46812/225000 [99:07:42<336:25:52,  6.80s/it]
 21%|██        | 46813/225000 [99:07:47<313:21:44,  6.33s/it]
 21%|██        | 46814/225000 [99:07:52<289:03:38,  5.84s/it]
 21%|██        | 46815/225000 [99:08:01<340:13:32,  6.87s/it]
 21%|██        | 46816/225000 [99:08:09<359:20:19,  7.26s/it]
 21%|██        | 46817/225000 [99:08:15<337:21:07,  6.82s/it]
 21%|██        | 46818/225000 [99:08:24<366:54:48,  7.41s/it]
 21%|██        | 46819/225000 [99:08:28<311:58:54,  6.30s/it]
 21%|██        | 46820/225000 [99:08:36<340:33:55,  6.88s/it]
 21%|██        | 46821/225000 [99:08:43<346:17:32,  7.00s/it]
 21%|██        | 46822/225000 [99:08:49<330:59:13,  6.69s/it]
 21%|██        | 46823/225000 [99:08:56<329:36:09,  6.66s/it]
 21%|██        | 46824/225000 [99:09:06<376:07:18,  7.60s/it]
 21%|██        | 46825/225000 [99:09:11<339:29:01,  6.86s/it]
 21%|██        | 46826/225000 [99:09:15<304:06:07,  6.14s/it]
 21%|██        | 46827/225000 [99:09:20<286:22:44,  5.79s/it]
 21%|██        | 46828/225000 [99:09:28<320:45:50,  6.48s/it]
 21%|██        | 46829/225000 [99:09:37<357:54:00,  7.23s/it]
 21%|██        | 46830/225000 [99:09:43<333:38:46,  6.74s/it]
 21%|██        | 46831/225000 [99:09:48<308:41:54,  6.24s/it]
 21%|██        | 46832/225000 [99:09:53<287:46:31,  5.81s/it]
 21%|██        | 46833/225000 [99:10:02<339:56:39,  6.87s/it]
 21%|██        | 46834/225000 [99:10:11<378:18:43,  7.64s/it]
 21%|██        | 46835/225000 [99:10:16<330:15:34,  6.67s/it]
 21%|██        | 46836/225000 [99:10:23<333:28:42,  6.74s/it]
 21%|██        | 46837/225000 [99:10:33<382:27:48,  7.73s/it]
 21%|██        | 46838/225000 [99:10:37<328:43:42,  6.64s/it]
 21%|██        | 46839/225000 [99:10:46<364:26:24,  7.36s/it]
 21%|██        | 46840/225000 [99:10:53<364:07:05,  7.36s/it]
 21%|██        | 46841/225000 [99:10:58<322:03:31,  6.51s/it]
 21%|██        | 46842/225000 [99:11:02<294:43:45,  5.96s/it]
 21%|██        | 46843/225000 [99:11:12<349:49:47,  7.07s/it]
 21%|██        | 46844/225000 [99:11:18<338:33:54,  6.84s/it]
 21%|██        | 46845/225000 [99:11:23<307:32:11,  6.21s/it]
 21%|██        | 46846/225000 [99:11:35<390:10:00,  7.88s/it]
 21%|██        | 46847/225000 [99:11:39<334:02:39,  6.75s/it]
 21%|██        | 46848/225000 [99:11:46<342:28:07,  6.92s/it]
 21%|██        | 46849/225000 [99:11:54<351:02:25,  7.09s/it]
 21%|██        | 46850/225000 [99:11:58<313:54:30,  6.34s/it]
                                                             
{'loss': '0.7015', 'grad_norm': '0.7791', 'learning_rate': '4.647e-05', 'epoch': '0.2082', 'num_input_tokens_seen': 166725208, 'train_runtime': '3.571e+05', 'train_tokens_per_second': '466.9'}

 21%|██        | 46850/225000 [99:11:58<313:54:30,  6.34s/it]
 21%|██        | 46851/225000 [99:12:03<293:59:52,  5.94s/it]
 21%|██        | 46852/225000 [99:12:08<273:39:59,  5.53s/it]
 21%|██        | 46853/225000 [99:12:14<272:38:33,  5.51s/it]
 21%|██        | 46854/225000 [99:12:20<291:59:21,  5.90s/it]
 21%|██        | 46855/225000 [99:12:33<394:53:04,  7.98s/it]
 21%|██        | 46856/225000 [99:12:42<404:21:39,  8.17s/it]
 21%|██        | 46857/225000 [99:12:45<336:09:42,  6.79s/it]
 21%|██        | 46858/225000 [99:12:50<305:57:33,  6.18s/it]
 21%|██        | 46859/225000 [99:13:03<401:20:22,  8.11s/it]
 21%|██        | 46860/225000 [99:13:07<339:34:33,  6.86s/it]
 21%|██        | 46861/225000 [99:13:17<390:12:35,  7.89s/it]
 21%|██        | 46862/225000 [99:13:22<347:33:05,  7.02s/it]
 21%|██        | 46863/225000 [99:13:28<326:02:44,  6.59s/it]
 21%|██        | 46864/225000 [99:13:33<303:19:59,  6.13s/it]
 21%|██        | 46865/225000 [99:13:46<415:31:55,  8.40s/it]
 21%|██        | 46866/225000 [99:13:50<351:52:58,  7.11s/it]
 21%|██        | 46867/225000 [99:13:59<375:05:38,  7.58s/it]
 21%|██        | 46868/225000 [99:14:06<359:58:49,  7.28s/it]
 21%|██        | 46869/225000 [99:14:13<354:17:51,  7.16s/it]
 21%|██        | 46870/225000 [99:14:17<318:43:53,  6.44s/it]
 21%|██        | 46871/225000 [99:14:28<389:15:36,  7.87s/it]
 21%|██        | 46872/225000 [99:14:33<345:43:02,  6.99s/it]
 21%|██        | 46873/225000 [99:14:38<311:41:21,  6.30s/it]
 21%|██        | 46874/225000 [99:14:43<294:17:51,  5.95s/it]
 21%|██        | 46875/225000 [99:14:48<277:41:59,  5.61s/it]
 21%|██        | 46876/225000 [99:15:01<383:40:34,  7.75s/it]
 21%|██        | 46877/225000 [99:15:07<355:18:15,  7.18s/it]
 21%|██        | 46878/225000 [99:15:15<376:15:42,  7.60s/it]
 21%|██        | 46879/225000 [99:15:19<322:58:56,  6.53s/it]
 21%|██        | 46880/225000 [99:15:29<368:57:29,  7.46s/it]
 21%|██        | 46881/225000 [99:15:33<318:03:39,  6.43s/it]
 21%|██        | 46882/225000 [99:15:40<322:53:12,  6.53s/it]
 21%|██        | 46883/225000 [99:15:44<296:50:18,  6.00s/it]
 21%|██        | 46884/225000 [99:15:50<286:39:08,  5.79s/it]
 21%|██        | 46885/225000 [99:16:00<356:23:42,  7.20s/it]
 21%|██        | 46886/225000 [99:16:08<358:36:02,  7.25s/it]
 21%|██        | 46887/225000 [99:16:13<326:04:37,  6.59s/it]
 21%|██        | 46888/225000 [99:16:18<308:20:57,  6.23s/it]
 21%|██        | 46889/225000 [99:16:23<292:59:42,  5.92s/it]
 21%|██        | 46890/225000 [99:16:29<292:42:34,  5.92s/it]
 21%|██        | 46891/225000 [99:16:42<400:47:45,  8.10s/it]
 21%|██        | 46892/225000 [99:16:48<359:41:10,  7.27s/it]
 21%|██        | 46893/225000 [99:16:55<358:49:11,  7.25s/it]
 21%|██        | 46894/225000 [99:17:00<325:30:23,  6.58s/it]
 21%|██        | 46895/225000 [99:17:05<299:25:55,  6.05s/it]
 21%|██        | 46896/225000 [99:17:12<319:20:06,  6.45s/it]
 21%|██        | 46897/225000 [99:17:24<400:46:37,  8.10s/it]
 21%|██        | 46898/225000 [99:17:30<367:22:19,  7.43s/it]
 21%|██        | 46899/225000 [99:17:37<356:33:40,  7.21s/it]
 21%|██        | 46900/225000 [99:17:42<330:39:31,  6.68s/it]
                                                             
{'loss': '0.6955', 'grad_norm': '0.7193', 'learning_rate': '4.646e-05', 'epoch': '0.2084', 'num_input_tokens_seen': 166877896, 'train_runtime': '3.575e+05', 'train_tokens_per_second': '466.8'}

 21%|██        | 46900/225000 [99:17:42<330:39:31,  6.68s/it]
 21%|██        | 46901/225000 [99:17:51<367:59:55,  7.44s/it]
 21%|██        | 46902/225000 [99:17:57<337:30:17,  6.82s/it]
 21%|██        | 46903/225000 [99:18:01<307:25:42,  6.21s/it]
 21%|██        | 46904/225000 [99:18:07<290:51:01,  5.88s/it]
 21%|██        | 46905/225000 [99:18:12<280:19:19,  5.67s/it]
 21%|██        | 46906/225000 [99:18:18<283:13:05,  5.72s/it]
 21%|██        | 46907/225000 [99:18:28<349:47:15,  7.07s/it]
 21%|██        | 46908/225000 [99:18:39<405:05:24,  8.19s/it]
 21%|██        | 46909/225000 [99:18:43<344:27:43,  6.96s/it]
 21%|██        | 46910/225000 [99:18:54<409:53:14,  8.29s/it]
 21%|██        | 46911/225000 [99:18:58<345:07:50,  6.98s/it]
 21%|██        | 46912/225000 [99:19:05<348:29:01,  7.04s/it]
 21%|██        | 46913/225000 [99:19:11<332:45:53,  6.73s/it]
 21%|██        | 46914/225000 [99:19:18<330:00:21,  6.67s/it]
 21%|██        | 46915/225000 [99:19:23<310:41:02,  6.28s/it]
 21%|██        | 46916/225000 [99:19:28<291:32:01,  5.89s/it]
 21%|██        | 46917/225000 [99:19:33<283:39:42,  5.73s/it]
 21%|██        | 46918/225000 [99:19:38<271:52:47,  5.50s/it]
 21%|██        | 46919/225000 [99:19:44<273:05:58,  5.52s/it]
 21%|██        | 46920/225000 [99:19:56<362:41:28,  7.33s/it]
 21%|██        | 46921/225000 [99:20:05<401:57:52,  8.13s/it]
 21%|██        | 46922/225000 [99:20:13<397:48:04,  8.04s/it]
 21%|██        | 46923/225000 [99:20:20<370:18:06,  7.49s/it]
 21%|██        | 46924/225000 [99:20:25<347:33:19,  7.03s/it]
 21%|██        | 46925/225000 [99:20:33<358:06:18,  7.24s/it]
 21%|██        | 46926/225000 [99:20:42<375:25:39,  7.59s/it]
 21%|██        | 46927/225000 [99:20:48<357:22:22,  7.22s/it]
 21%|██        | 46928/225000 [99:20:53<324:42:33,  6.56s/it]
 21%|██        | 46929/225000 [99:21:05<398:11:33,  8.05s/it]
 21%|██        | 46930/225000 [99:21:10<354:11:02,  7.16s/it]
 21%|██        | 46931/225000 [99:21:14<320:19:13,  6.48s/it]
 21%|██        | 46932/225000 [99:21:19<295:12:39,  5.97s/it]
 21%|██        | 46933/225000 [99:21:32<400:41:30,  8.10s/it]
 21%|██        | 46934/225000 [99:21:36<339:44:09,  6.87s/it]
 21%|██        | 46935/225000 [99:21:41<305:05:57,  6.17s/it]
 21%|██        | 46936/225000 [99:21:46<294:03:48,  5.95s/it]
 21%|██        | 46937/225000 [99:21:58<379:55:09,  7.68s/it]
 21%|██        | 46938/225000 [99:22:02<330:28:48,  6.68s/it]
 21%|██        | 46939/225000 [99:22:11<356:30:45,  7.21s/it]
 21%|██        | 46940/225000 [99:22:15<316:09:05,  6.39s/it]
 21%|██        | 46941/225000 [99:22:20<287:19:58,  5.81s/it]
 21%|██        | 46942/225000 [99:22:25<286:02:12,  5.78s/it]
 21%|██        | 46943/225000 [99:22:39<405:49:17,  8.20s/it]
 21%|██        | 46944/225000 [99:22:44<345:49:18,  6.99s/it]
 21%|██        | 46945/225000 [99:22:47<297:22:51,  6.01s/it]
 21%|██        | 46946/225000 [99:22:58<365:26:42,  7.39s/it]
 21%|██        | 46947/225000 [99:23:05<354:55:15,  7.18s/it]
 21%|██        | 46948/225000 [99:23:09<319:50:22,  6.47s/it]
 21%|██        | 46949/225000 [99:23:21<398:57:03,  8.07s/it]
 21%|██        | 46950/225000 [99:23:25<338:49:49,  6.85s/it]
                                                             
{'loss': '0.6965', 'grad_norm': '0.8005', 'learning_rate': '4.645e-05', 'epoch': '0.2087', 'num_input_tokens_seen': 167029072, 'train_runtime': '3.578e+05', 'train_tokens_per_second': '466.8'}

 21%|██        | 46950/225000 [99:23:25<338:49:49,  6.85s/it]
 21%|██        | 46951/225000 [99:23:35<377:51:47,  7.64s/it]
 21%|██        | 46952/225000 [99:23:39<329:35:04,  6.66s/it]
 21%|██        | 46953/225000 [99:23:44<310:25:29,  6.28s/it]
 21%|██        | 46954/225000 [99:23:50<297:15:28,  6.01s/it]
 21%|██        | 46955/225000 [99:23:55<280:08:05,  5.66s/it]
 21%|██        | 46956/225000 [99:24:06<360:39:13,  7.29s/it]
 21%|██        | 46957/225000 [99:24:15<387:05:29,  7.83s/it]
 21%|██        | 46958/225000 [99:24:19<329:43:38,  6.67s/it]
 21%|██        | 46959/225000 [99:24:29<377:23:43,  7.63s/it]
 21%|██        | 46960/225000 [99:24:34<347:32:18,  7.03s/it]
 21%|██        | 46961/225000 [99:24:41<348:35:47,  7.05s/it]
 21%|██        | 46962/225000 [99:24:50<374:04:41,  7.56s/it]
 21%|██        | 46963/225000 [99:24:54<321:39:45,  6.50s/it]
 21%|██        | 46964/225000 [99:24:59<296:26:04,  5.99s/it]
 21%|██        | 46965/225000 [99:25:08<344:14:56,  6.96s/it]
 21%|██        | 46966/225000 [99:25:18<386:24:38,  7.81s/it]
 21%|██        | 46967/225000 [99:25:25<376:12:03,  7.61s/it]
 21%|██        | 46968/225000 [99:25:31<349:52:50,  7.07s/it]
 21%|██        | 46969/225000 [99:25:38<353:35:04,  7.15s/it]
 21%|██        | 46970/225000 [99:25:45<351:52:49,  7.12s/it]
 21%|██        | 46971/225000 [99:25:50<314:43:29,  6.36s/it]
 21%|██        | 46972/225000 [99:25:55<290:12:07,  5.87s/it]
 21%|██        | 46973/225000 [99:26:07<386:57:36,  7.82s/it]
 21%|██        | 46974/225000 [99:26:11<335:23:35,  6.78s/it]
 21%|██        | 46975/225000 [99:26:16<299:43:40,  6.06s/it]
 21%|██        | 46976/225000 [99:26:20<275:02:58,  5.56s/it]
 21%|██        | 46977/225000 [99:26:31<355:37:18,  7.19s/it]
 21%|██        | 46978/225000 [99:26:39<366:12:12,  7.41s/it]
 21%|██        | 46979/225000 [99:26:44<336:52:25,  6.81s/it]
 21%|██        | 46980/225000 [99:26:50<322:40:10,  6.53s/it]
 21%|██        | 46981/225000 [99:26:56<307:16:18,  6.21s/it]
 21%|██        | 46982/225000 [99:27:02<306:30:29,  6.20s/it]
 21%|██        | 46983/225000 [99:27:09<319:27:42,  6.46s/it]
 21%|██        | 46984/225000 [99:27:15<306:18:10,  6.19s/it]
 21%|██        | 46985/225000 [99:27:21<312:40:03,  6.32s/it]
 21%|██        | 46986/225000 [99:27:27<300:34:22,  6.08s/it]
 21%|██        | 46987/225000 [99:27:37<359:00:12,  7.26s/it]
 21%|██        | 46988/225000 [99:27:45<371:25:51,  7.51s/it]
 21%|██        | 46989/225000 [99:27:50<337:29:26,  6.83s/it]
 21%|██        | 46990/225000 [99:27:56<319:53:28,  6.47s/it]
 21%|██        | 46991/225000 [99:28:02<313:51:00,  6.35s/it]
 21%|██        | 46992/225000 [99:28:07<304:00:29,  6.15s/it]
 21%|██        | 46993/225000 [99:28:13<301:44:00,  6.10s/it]
 21%|██        | 46994/225000 [99:28:26<399:27:44,  8.08s/it]
 21%|██        | 46995/225000 [99:28:32<367:40:48,  7.44s/it]
 21%|██        | 46996/225000 [99:28:36<312:50:37,  6.33s/it]
 21%|██        | 46997/225000 [99:28:49<407:26:52,  8.24s/it]
 21%|██        | 46998/225000 [99:28:55<376:27:29,  7.61s/it]
 21%|██        | 46999/225000 [99:29:00<344:18:32,  6.96s/it]
 21%|██        | 47000/225000 [99:29:08<352:50:24,  7.14s/it]
                                                             
{'loss': '0.6917', 'grad_norm': '0.7915', 'learning_rate': '4.644e-05', 'epoch': '0.2089', 'num_input_tokens_seen': 167181320, 'train_runtime': '3.581e+05', 'train_tokens_per_second': '466.8'}

 21%|██        | 47000/225000 [99:29:08<352:50:24,  7.14s/it]
 21%|██        | 47001/225000 [99:29:16<368:53:41,  7.46s/it]
 21%|██        | 47002/225000 [99:29:21<336:23:37,  6.80s/it]
 21%|██        | 47003/225000 [99:29:29<357:07:22,  7.22s/it]
 21%|██        | 47004/225000 [99:29:34<317:20:38,  6.42s/it]
 21%|██        | 47005/225000 [99:29:40<305:34:57,  6.18s/it]
 21%|██        | 47006/225000 [99:29:51<378:07:31,  7.65s/it]
 21%|██        | 47007/225000 [99:29:55<331:48:50,  6.71s/it]
 21%|██        | 47008/225000 [99:30:03<356:16:49,  7.21s/it]
 21%|██        | 47009/225000 [99:30:09<330:06:23,  6.68s/it]
 21%|██        | 47010/225000 [99:30:18<365:39:08,  7.40s/it]
 21%|██        | 47011/225000 [99:30:24<350:30:16,  7.09s/it]
 21%|██        | 47012/225000 [99:30:28<300:03:04,  6.07s/it]
 21%|██        | 47013/225000 [99:30:40<382:33:06,  7.74s/it]
 21%|██        | 47014/225000 [99:30:44<326:26:45,  6.60s/it]
 21%|██        | 47015/225000 [99:30:50<325:38:47,  6.59s/it]
 21%|██        | 47016/225000 [99:30:57<329:32:20,  6.67s/it]
 21%|██        | 47017/225000 [99:31:06<366:01:34,  7.40s/it]
 21%|██        | 47018/225000 [99:31:11<326:17:19,  6.60s/it]
 21%|██        | 47019/225000 [99:31:16<297:57:24,  6.03s/it]
 21%|██        | 47020/225000 [99:31:20<278:16:49,  5.63s/it]
 21%|██        | 47021/225000 [99:31:31<356:00:10,  7.20s/it]
 21%|██        | 47022/225000 [99:31:35<312:09:40,  6.31s/it]
 21%|██        | 47023/225000 [99:31:43<336:26:55,  6.81s/it]
 21%|██        | 47024/225000 [99:31:51<341:42:41,  6.91s/it]
 21%|██        | 47025/225000 [99:31:57<341:55:12,  6.92s/it]
 21%|██        | 47026/225000 [99:32:06<360:43:46,  7.30s/it]
 21%|██        | 47027/225000 [99:32:10<323:15:48,  6.54s/it]
 21%|██        | 47028/225000 [99:32:15<300:21:26,  6.08s/it]
 21%|██        | 47029/225000 [99:32:20<283:40:32,  5.74s/it]
 21%|██        | 47030/225000 [99:32:28<306:53:59,  6.21s/it]
 21%|██        | 47031/225000 [99:32:37<358:20:12,  7.25s/it]
 21%|██        | 47032/225000 [99:32:42<320:02:16,  6.47s/it]
 21%|██        | 47033/225000 [99:32:47<295:44:49,  5.98s/it]
 21%|██        | 47034/225000 [99:32:55<326:20:29,  6.60s/it]
 21%|██        | 47035/225000 [99:33:06<397:34:25,  8.04s/it]
 21%|██        | 47036/225000 [99:33:10<339:16:38,  6.86s/it]
 21%|██        | 47037/225000 [99:33:20<373:44:58,  7.56s/it]
 21%|██        | 47038/225000 [99:33:25<341:05:52,  6.90s/it]
 21%|██        | 47039/225000 [99:33:32<336:44:31,  6.81s/it]
 21%|██        | 47040/225000 [99:33:36<295:28:18,  5.98s/it]
 21%|██        | 47041/225000 [99:33:44<333:28:30,  6.75s/it]
 21%|██        | 47042/225000 [99:33:52<349:21:31,  7.07s/it]
 21%|██        | 47043/225000 [99:33:56<306:51:35,  6.21s/it]
 21%|██        | 47044/225000 [99:34:05<346:54:35,  7.02s/it]
 21%|██        | 47045/225000 [99:34:15<383:32:27,  7.76s/it]
 21%|██        | 47046/225000 [99:34:22<383:06:05,  7.75s/it]
 21%|██        | 47047/225000 [99:34:26<320:53:16,  6.49s/it]
 21%|██        | 47048/225000 [99:34:31<304:09:16,  6.15s/it]
 21%|██        | 47049/225000 [99:34:41<361:48:51,  7.32s/it]
 21%|██        | 47050/225000 [99:34:47<333:57:06,  6.76s/it]
                                                             
{'loss': '0.6942', 'grad_norm': '0.7837', 'learning_rate': '4.643e-05', 'epoch': '0.2091', 'num_input_tokens_seen': 167333312, 'train_runtime': '3.585e+05', 'train_tokens_per_second': '466.8'}

 21%|██        | 47050/225000 [99:34:47<333:57:06,  6.76s/it]
 21%|██        | 47051/225000 [99:34:55<362:38:49,  7.34s/it]
 21%|██        | 47052/225000 [99:35:00<323:35:28,  6.55s/it]
 21%|██        | 47053/225000 [99:35:04<291:39:56,  5.90s/it]
 21%|██        | 47054/225000 [99:35:09<276:36:16,  5.60s/it]
 21%|██        | 47055/225000 [99:35:19<344:09:49,  6.96s/it]
 21%|██        | 47056/225000 [99:35:29<382:04:33,  7.73s/it]
 21%|██        | 47057/225000 [99:35:36<375:25:22,  7.60s/it]
 21%|██        | 47058/225000 [99:35:42<343:05:10,  6.94s/it]
 21%|██        | 47059/225000 [99:35:49<341:44:18,  6.91s/it]
 21%|██        | 47060/225000 [99:35:53<308:33:20,  6.24s/it]
 21%|██        | 47061/225000 [99:36:01<332:04:37,  6.72s/it]
 21%|██        | 47062/225000 [99:36:10<358:13:01,  7.25s/it]
 21%|██        | 47063/225000 [99:36:14<317:53:50,  6.43s/it]
 21%|██        | 47064/225000 [99:36:25<390:43:22,  7.91s/it]
 21%|██        | 47065/225000 [99:36:30<335:57:23,  6.80s/it]
 21%|██        | 47066/225000 [99:36:35<308:22:42,  6.24s/it]
 21%|██        | 47067/225000 [99:36:39<288:56:20,  5.85s/it]
 21%|██        | 47068/225000 [99:36:44<274:13:21,  5.55s/it]
 21%|██        | 47069/225000 [99:36:55<357:28:20,  7.23s/it]
 21%|██        | 47070/225000 [99:37:02<346:12:31,  7.00s/it]
 21%|██        | 47071/225000 [99:37:10<355:09:45,  7.19s/it]
 21%|██        | 47072/225000 [99:37:15<326:17:58,  6.60s/it]
 21%|██        | 47073/225000 [99:37:25<372:41:54,  7.54s/it]
 21%|██        | 47074/225000 [99:37:29<321:28:29,  6.50s/it]
 21%|██        | 47075/225000 [99:37:33<294:44:23,  5.96s/it]
 21%|██        | 47076/225000 [99:37:45<373:47:22,  7.56s/it]
 21%|██        | 47077/225000 [99:37:53<379:05:59,  7.67s/it]
 21%|██        | 47078/225000 [99:37:56<317:15:23,  6.42s/it]
 21%|██        | 47079/225000 [99:38:01<292:01:52,  5.91s/it]
 21%|██        | 47080/225000 [99:38:05<268:56:48,  5.44s/it]
 21%|██        | 47081/225000 [99:38:17<362:10:40,  7.33s/it]
 21%|██        | 47082/225000 [99:38:22<333:06:19,  6.74s/it]
 21%|██        | 47083/225000 [99:38:27<305:55:23,  6.19s/it]
 21%|██        | 47084/225000 [99:38:32<292:30:43,  5.92s/it]
 21%|██        | 47085/225000 [99:38:40<320:52:01,  6.49s/it]
 21%|██        | 47086/225000 [99:38:51<387:33:55,  7.84s/it]
 21%|██        | 47087/225000 [99:38:55<332:21:33,  6.73s/it]
 21%|██        | 47088/225000 [99:39:02<332:03:15,  6.72s/it]
 21%|██        | 47089/225000 [99:39:10<350:16:40,  7.09s/it]
 21%|██        | 47090/225000 [99:39:17<345:38:19,  6.99s/it]
 21%|██        | 47091/225000 [99:39:24<350:33:38,  7.09s/it]
 21%|██        | 47092/225000 [99:39:30<326:44:41,  6.61s/it]
 21%|██        | 47093/225000 [99:39:38<349:49:02,  7.08s/it]
 21%|██        | 47094/225000 [99:39:45<345:55:04,  7.00s/it]
 21%|██        | 47095/225000 [99:39:54<384:30:34,  7.78s/it]
 21%|██        | 47096/225000 [99:39:58<331:30:14,  6.71s/it]
 21%|██        | 47097/225000 [99:40:03<296:54:46,  6.01s/it]
 21%|██        | 47098/225000 [99:40:13<359:00:07,  7.26s/it]
 21%|██        | 47099/225000 [99:40:19<344:12:09,  6.97s/it]
 21%|██        | 47100/225000 [99:40:25<326:41:07,  6.61s/it]
                                                             
{'loss': '0.7091', 'grad_norm': '0.6939', 'learning_rate': '4.642e-05', 'epoch': '0.2093', 'num_input_tokens_seen': 167484320, 'train_runtime': '3.588e+05', 'train_tokens_per_second': '466.8'}

 21%|██        | 47100/225000 [99:40:25<326:41:07,  6.61s/it]
 21%|██        | 47101/225000 [99:40:32<327:02:52,  6.62s/it]
 21%|██        | 47102/225000 [99:40:37<315:24:46,  6.38s/it]
 21%|██        | 47103/225000 [99:40:47<355:55:50,  7.20s/it]
 21%|██        | 47104/225000 [99:40:51<308:30:06,  6.24s/it]
 21%|██        | 47105/225000 [99:40:55<277:44:06,  5.62s/it]
 21%|██        | 47106/225000 [99:41:04<337:40:44,  6.83s/it]
 21%|██        | 47107/225000 [99:41:10<319:27:23,  6.46s/it]
 21%|██        | 47108/225000 [99:41:15<296:54:04,  6.01s/it]
 21%|██        | 47109/225000 [99:41:20<285:23:00,  5.78s/it]
 21%|██        | 47110/225000 [99:41:25<272:26:00,  5.51s/it]
 21%|██        | 47111/225000 [99:41:37<372:15:47,  7.53s/it]
 21%|██        | 47112/225000 [99:41:46<387:25:36,  7.84s/it]
 21%|██        | 47113/225000 [99:41:51<350:30:29,  7.09s/it]
 21%|██        | 47114/225000 [99:41:57<326:32:16,  6.61s/it]
 21%|██        | 47115/225000 [99:42:02<306:54:20,  6.21s/it]
 21%|██        | 47116/225000 [99:42:07<294:26:05,  5.96s/it]
 21%|██        | 47117/225000 [99:42:13<284:57:21,  5.77s/it]
 21%|██        | 47118/225000 [99:42:18<285:20:05,  5.77s/it]
 21%|██        | 47119/225000 [99:42:24<286:18:55,  5.79s/it]
 21%|██        | 47120/225000 [99:42:38<398:27:49,  8.06s/it]
 21%|██        | 47121/225000 [99:42:45<388:53:48,  7.87s/it]
 21%|██        | 47122/225000 [99:42:49<337:01:56,  6.82s/it]
 21%|██        | 47123/225000 [99:43:01<399:52:50,  8.09s/it]
 21%|██        | 47124/225000 [99:43:06<359:13:43,  7.27s/it]
 21%|██        | 47125/225000 [99:43:10<317:26:57,  6.42s/it]
 21%|██        | 47126/225000 [99:43:15<291:41:06,  5.90s/it]
 21%|██        | 47127/225000 [99:43:20<282:51:38,  5.72s/it]
 21%|██        | 47128/225000 [99:43:26<275:56:10,  5.58s/it]
 21%|██        | 47129/225000 [99:43:39<392:16:07,  7.94s/it]
 21%|██        | 47130/225000 [99:43:46<372:15:51,  7.53s/it]
 21%|██        | 47131/225000 [99:43:55<394:05:43,  7.98s/it]
 21%|██        | 47132/225000 [99:44:02<381:35:42,  7.72s/it]
 21%|██        | 47133/225000 [99:44:06<325:00:22,  6.58s/it]
 21%|██        | 47134/225000 [99:44:15<359:19:25,  7.27s/it]
 21%|██        | 47135/225000 [99:44:20<335:19:10,  6.79s/it]
 21%|██        | 47136/225000 [99:44:26<317:19:00,  6.42s/it]
 21%|██        | 47137/225000 [99:44:31<301:08:22,  6.10s/it]
 21%|██        | 47138/225000 [99:44:37<294:07:56,  5.95s/it]
 21%|██        | 47139/225000 [99:44:42<284:55:17,  5.77s/it]
 21%|██        | 47140/225000 [99:44:48<282:58:06,  5.73s/it]
 21%|██        | 47141/225000 [99:44:57<337:17:52,  6.83s/it]
 21%|██        | 47142/225000 [99:45:09<417:51:20,  8.46s/it]
 21%|██        | 47143/225000 [99:45:15<368:30:14,  7.46s/it]
 21%|██        | 47144/225000 [99:45:19<330:21:17,  6.69s/it]
 21%|██        | 47145/225000 [99:45:26<326:55:40,  6.62s/it]
 21%|██        | 47146/225000 [99:45:40<434:11:19,  8.79s/it]
 21%|██        | 47147/225000 [99:45:45<381:54:20,  7.73s/it]
 21%|██        | 47148/225000 [99:45:49<326:46:05,  6.61s/it]
 21%|██        | 47149/225000 [99:46:01<403:07:36,  8.16s/it]
 21%|██        | 47150/225000 [99:46:07<376:54:49,  7.63s/it]
                                                             
{'loss': '0.703', 'grad_norm': '0.7485', 'learning_rate': '4.641e-05', 'epoch': '0.2096', 'num_input_tokens_seen': 167635120, 'train_runtime': '3.592e+05', 'train_tokens_per_second': '466.7'}

 21%|██        | 47150/225000 [99:46:07<376:54:49,  7.63s/it]
 21%|██        | 47151/225000 [99:46:12<341:09:41,  6.91s/it]
 21%|██        | 47152/225000 [99:46:17<300:42:57,  6.09s/it]
 21%|██        | 47153/225000 [99:46:27<365:46:29,  7.40s/it]
 21%|██        | 47154/225000 [99:46:32<327:08:59,  6.62s/it]
 21%|██        | 47155/225000 [99:46:39<339:01:36,  6.86s/it]
 21%|██        | 47156/225000 [99:46:48<364:36:12,  7.38s/it]
 21%|██        | 47157/225000 [99:46:52<316:49:52,  6.41s/it]
 21%|██        | 47158/225000 [99:46:57<292:05:11,  5.91s/it]
 21%|██        | 47159/225000 [99:47:02<280:57:47,  5.69s/it]
 21%|██        | 47160/225000 [99:47:11<328:46:03,  6.66s/it]
 21%|██        | 47161/225000 [99:47:23<413:28:51,  8.37s/it]
 21%|██        | 47162/225000 [99:47:27<352:29:03,  7.14s/it]
 21%|██        | 47163/225000 [99:47:32<315:41:50,  6.39s/it]
 21%|██        | 47164/225000 [99:47:39<317:38:05,  6.43s/it]
 21%|██        | 47165/225000 [99:47:51<412:33:53,  8.35s/it]
 21%|██        | 47166/225000 [99:47:58<383:30:43,  7.76s/it]
 21%|██        | 47167/225000 [99:48:03<348:36:14,  7.06s/it]
 21%|██        | 47168/225000 [99:48:09<329:19:49,  6.67s/it]
 21%|██        | 47169/225000 [99:48:15<312:44:56,  6.33s/it]
 21%|██        | 47170/225000 [99:48:20<306:13:35,  6.20s/it]
 21%|██        | 47171/225000 [99:48:26<292:29:31,  5.92s/it]
 21%|██        | 47172/225000 [99:48:34<326:23:27,  6.61s/it]
 21%|██        | 47173/225000 [99:48:46<407:09:44,  8.24s/it]
 21%|██        | 47174/225000 [99:48:50<352:09:46,  7.13s/it]
 21%|██        | 47175/225000 [99:48:55<316:19:12,  6.40s/it]
 21%|██        | 47176/225000 [99:49:00<292:28:35,  5.92s/it]
 21%|██        | 47177/225000 [99:49:06<297:58:06,  6.03s/it]
 21%|██        | 47178/225000 [99:49:20<405:54:41,  8.22s/it]
 21%|██        | 47179/225000 [99:49:23<338:02:53,  6.84s/it]
 21%|██        | 47180/225000 [99:49:29<314:47:07,  6.37s/it]
 21%|██        | 47181/225000 [99:49:34<299:56:56,  6.07s/it]
 21%|██        | 47182/225000 [99:49:40<293:25:51,  5.94s/it]
 21%|██        | 47183/225000 [99:49:45<290:10:11,  5.87s/it]
 21%|██        | 47184/225000 [99:49:53<323:35:58,  6.55s/it]
 21%|██        | 47185/225000 [99:50:05<404:57:53,  8.20s/it]
 21%|██        | 47186/225000 [99:50:10<346:14:26,  7.01s/it]
 21%|██        | 47187/225000 [99:50:18<362:52:58,  7.35s/it]
 21%|██        | 47188/225000 [99:50:23<335:00:13,  6.78s/it]
 21%|██        | 47189/225000 [99:50:29<319:04:10,  6.46s/it]
 21%|██        | 47190/225000 [99:50:35<308:20:13,  6.24s/it]
 21%|██        | 47191/225000 [99:50:46<386:16:20,  7.82s/it]
 21%|██        | 47192/225000 [99:50:52<351:44:42,  7.12s/it]
 21%|██        | 47193/225000 [99:51:02<395:41:49,  8.01s/it]
 21%|██        | 47194/225000 [99:51:08<374:52:33,  7.59s/it]
 21%|██        | 47195/225000 [99:51:12<320:51:33,  6.50s/it]
 21%|██        | 47196/225000 [99:51:16<282:30:54,  5.72s/it]
 21%|██        | 47197/225000 [99:51:29<383:07:06,  7.76s/it]
 21%|██        | 47198/225000 [99:51:33<332:09:17,  6.73s/it]
 21%|██        | 47199/225000 [99:51:38<301:18:40,  6.10s/it]
 21%|██        | 47200/225000 [99:51:47<351:00:57,  7.11s/it]
                                                             
{'loss': '0.7174', 'grad_norm': '0.7502', 'learning_rate': '4.64e-05', 'epoch': '0.2098', 'num_input_tokens_seen': 167785024, 'train_runtime': '3.595e+05', 'train_tokens_per_second': '466.7'}

 21%|██        | 47200/225000 [99:51:47<351:00:57,  7.11s/it]
 21%|██        | 47201/225000 [99:51:56<380:56:38,  7.71s/it]
 21%|██        | 47202/225000 [99:52:05<398:26:08,  8.07s/it]
 21%|██        | 47203/225000 [99:52:09<334:42:35,  6.78s/it]
 21%|██        | 47204/225000 [99:52:14<308:06:54,  6.24s/it]
 21%|██        | 47205/225000 [99:52:18<282:42:26,  5.72s/it]
 21%|██        | 47206/225000 [99:52:24<276:40:14,  5.60s/it]
 21%|██        | 47207/225000 [99:52:34<340:39:26,  6.90s/it]
 21%|██        | 47208/225000 [99:52:44<389:15:08,  7.88s/it]
 21%|██        | 47209/225000 [99:52:48<340:11:08,  6.89s/it]
 21%|██        | 47210/225000 [99:52:58<373:14:06,  7.56s/it]
 21%|██        | 47211/225000 [99:53:02<330:03:50,  6.68s/it]
 21%|██        | 47212/225000 [99:53:07<302:29:54,  6.13s/it]
 21%|██        | 47213/225000 [99:53:12<282:05:47,  5.71s/it]
 21%|██        | 47214/225000 [99:53:26<406:28:19,  8.23s/it]
 21%|██        | 47215/225000 [99:53:31<361:22:03,  7.32s/it]
 21%|██        | 47216/225000 [99:53:35<311:20:23,  6.30s/it]
 21%|██        | 47217/225000 [99:53:46<380:16:30,  7.70s/it]
 21%|██        | 47218/225000 [99:53:50<330:14:24,  6.69s/it]
 21%|██        | 47219/225000 [99:53:55<295:05:02,  5.98s/it]
 21%|██        | 47220/225000 [99:54:03<338:09:32,  6.85s/it]
 21%|██        | 47221/225000 [99:54:11<342:44:17,  6.94s/it]
 21%|██        | 47222/225000 [99:54:20<377:13:24,  7.64s/it]
 21%|██        | 47223/225000 [99:54:23<315:15:41,  6.38s/it]
 21%|██        | 47224/225000 [99:54:33<361:34:58,  7.32s/it]
 21%|██        | 47225/225000 [99:54:38<327:03:39,  6.62s/it]
 21%|██        | 47226/225000 [99:54:43<308:18:27,  6.24s/it]
 21%|██        | 47227/225000 [99:54:48<291:00:22,  5.89s/it]
 21%|██        | 47228/225000 [99:54:57<337:46:51,  6.84s/it]
 21%|██        | 47229/225000 [99:55:09<410:56:29,  8.32s/it]
 21%|██        | 47230/225000 [99:55:14<355:43:23,  7.20s/it]
 21%|██        | 47231/225000 [99:55:18<315:39:39,  6.39s/it]
 21%|██        | 47232/225000 [99:55:23<290:04:52,  5.87s/it]
 21%|██        | 47233/225000 [99:55:31<328:31:00,  6.65s/it]
 21%|██        | 47234/225000 [99:55:42<382:02:06,  7.74s/it]
 21%|██        | 47235/225000 [99:55:47<346:43:44,  7.02s/it]
 21%|██        | 47236/225000 [99:55:52<310:11:38,  6.28s/it]
 21%|██        | 47237/225000 [99:56:03<379:49:16,  7.69s/it]
 21%|██        | 47238/225000 [99:56:11<386:13:13,  7.82s/it]
 21%|██        | 47239/225000 [99:56:19<386:48:41,  7.83s/it]
 21%|██        | 47240/225000 [99:56:23<333:09:32,  6.75s/it]
 21%|██        | 47241/225000 [99:56:30<341:12:39,  6.91s/it]
 21%|██        | 47242/225000 [99:56:35<307:49:16,  6.23s/it]
 21%|██        | 47243/225000 [99:56:43<332:58:52,  6.74s/it]
 21%|██        | 47244/225000 [99:56:51<355:40:05,  7.20s/it]
 21%|██        | 47245/225000 [99:56:58<351:48:48,  7.13s/it]
 21%|██        | 47246/225000 [99:57:03<320:52:58,  6.50s/it]
 21%|██        | 47247/225000 [99:57:11<341:08:45,  6.91s/it]
 21%|██        | 47248/225000 [99:57:15<309:23:26,  6.27s/it]
 21%|██        | 47249/225000 [99:57:20<286:40:23,  5.81s/it]
 21%|██        | 47250/225000 [99:57:28<321:04:18,  6.50s/it]
                                                             
{'loss': '0.6925', 'grad_norm': '0.7685', 'learning_rate': '4.639e-05', 'epoch': '0.21', 'num_input_tokens_seen': 167935104, 'train_runtime': '3.598e+05', 'train_tokens_per_second': '466.7'}

 21%|██        | 47250/225000 [99:57:28<321:04:18,  6.50s/it]
 21%|██        | 47251/225000 [99:57:38<372:49:56,  7.55s/it]
 21%|██        | 47252/225000 [99:57:45<362:43:55,  7.35s/it]
 21%|██        | 47253/225000 [99:57:51<346:33:58,  7.02s/it]
 21%|██        | 47254/225000 [99:57:58<345:47:59,  7.00s/it]
 21%|██        | 47255/225000 [99:58:04<327:49:33,  6.64s/it]
 21%|██        | 47256/225000 [99:58:08<290:02:21,  5.87s/it]
 21%|██        | 47257/225000 [99:58:14<288:17:41,  5.84s/it]
 21%|██        | 47258/225000 [99:58:19<281:44:02,  5.71s/it]
 21%|██        | 47259/225000 [99:58:27<301:25:11,  6.11s/it]
 21%|██        | 47260/225000 [99:58:40<412:58:15,  8.36s/it]
 21%|██        | 47261/225000 [99:58:44<352:11:07,  7.13s/it]
 21%|██        | 47262/225000 [99:58:49<312:45:20,  6.33s/it]
 21%|██        | 47263/225000 [99:58:58<359:53:05,  7.29s/it]
 21%|██        | 47264/225000 [99:59:06<368:35:02,  7.47s/it]
 21%|██        | 47265/225000 [99:59:13<360:07:41,  7.29s/it]
 21%|██        | 47266/225000 [99:59:17<313:24:54,  6.35s/it]
 21%|██        | 47267/225000 [99:59:22<286:25:30,  5.80s/it]
 21%|██        | 47268/225000 [99:59:36<407:04:18,  8.25s/it]
 21%|██        | 47269/225000 [99:59:41<356:43:38,  7.23s/it]
 21%|██        | 47270/225000 [99:59:45<318:45:24,  6.46s/it]
 21%|██        | 47271/225000 [99:59:50<296:11:02,  6.00s/it]
 21%|██        | 47272/225000 [99:59:56<292:35:54,  5.93s/it]
 21%|██        | 47273/225000 [100:00:08<386:25:49,  7.83s/it]
 21%|██        | 47274/225000 [100:00:12<328:27:59,  6.65s/it]
 21%|██        | 47275/225000 [100:00:21<360:26:26,  7.30s/it]
 21%|██        | 47276/225000 [100:00:25<314:34:12,  6.37s/it]
 21%|██        | 47277/225000 [100:00:37<390:01:39,  7.90s/it]
 21%|██        | 47278/225000 [100:00:41<334:52:09,  6.78s/it]
 21%|██        | 47279/225000 [100:00:50<377:14:25,  7.64s/it]
 21%|██        | 47280/225000 [100:00:56<345:17:48,  6.99s/it]
 21%|██        | 47281/225000 [100:01:02<328:08:48,  6.65s/it]
 21%|██        | 47282/225000 [100:01:07<305:36:49,  6.19s/it]
 21%|██        | 47283/225000 [100:01:12<296:01:41,  6.00s/it]
 21%|██        | 47284/225000 [100:01:18<282:13:53,  5.72s/it]
 21%|██        | 47285/225000 [100:01:23<281:12:20,  5.70s/it]
 21%|██        | 47286/225000 [100:01:32<334:30:10,  6.78s/it]
 21%|██        | 47287/225000 [100:01:45<415:52:32,  8.42s/it]
 21%|██        | 47288/225000 [100:01:48<339:22:36,  6.87s/it]
 21%|██        | 47289/225000 [100:01:54<331:26:31,  6.71s/it]
 21%|██        | 47290/225000 [100:02:03<355:40:23,  7.21s/it]
 21%|██        | 47291/225000 [100:02:08<334:18:13,  6.77s/it]
 21%|██        | 47292/225000 [100:02:15<333:52:55,  6.76s/it]
 21%|██        | 47293/225000 [100:02:21<322:20:42,  6.53s/it]
 21%|██        | 47294/225000 [100:02:30<349:42:26,  7.08s/it]
 21%|██        | 47295/225000 [100:02:39<378:25:50,  7.67s/it]
 21%|██        | 47296/225000 [100:02:45<356:13:49,  7.22s/it]
 21%|██        | 47297/225000 [100:02:50<334:29:29,  6.78s/it]
 21%|██        | 47298/225000 [100:02:59<356:08:14,  7.21s/it]
 21%|██        | 47299/225000 [100:03:03<317:49:59,  6.44s/it]
 21%|██        | 47300/225000 [100:03:08<293:29:57,  5.95s/it]
                                                              
{'loss': '0.7225', 'grad_norm': '0.7022', 'learning_rate': '4.638e-05', 'epoch': '0.2102', 'num_input_tokens_seen': 168084912, 'train_runtime': '3.602e+05', 'train_tokens_per_second': '466.7'}

 21%|██        | 47300/225000 [100:03:08<293:29:57,  5.95s/it]
 21%|██        | 47301/225000 [100:03:17<331:19:03,  6.71s/it]
 21%|██        | 47302/225000 [100:03:26<374:07:47,  7.58s/it]
 21%|██        | 47303/225000 [100:03:32<341:39:29,  6.92s/it]
 21%|██        | 47304/225000 [100:03:37<314:51:30,  6.38s/it]
 21%|██        | 47305/225000 [100:03:42<296:29:57,  6.01s/it]
 21%|██        | 47306/225000 [100:03:47<287:58:08,  5.83s/it]
 21%|██        | 47307/225000 [100:03:52<277:50:50,  5.63s/it]
 21%|██        | 47308/225000 [100:04:03<350:14:28,  7.10s/it]
 21%|██        | 47309/225000 [100:04:12<384:26:11,  7.79s/it]
 21%|██        | 47310/225000 [100:04:17<339:52:43,  6.89s/it]
 21%|██        | 47311/225000 [100:04:22<308:55:50,  6.26s/it]
 21%|██        | 47312/225000 [100:04:27<294:27:42,  5.97s/it]
 21%|██        | 47313/225000 [100:04:33<285:54:16,  5.79s/it]
 21%|██        | 47314/225000 [100:04:45<377:08:37,  7.64s/it]
 21%|██        | 47315/225000 [100:04:53<385:11:14,  7.80s/it]
 21%|██        | 47316/225000 [100:05:01<384:36:36,  7.79s/it]
 21%|██        | 47317/225000 [100:05:05<330:44:22,  6.70s/it]
 21%|██        | 47318/225000 [100:05:09<293:20:35,  5.94s/it]
 21%|██        | 47319/225000 [100:05:21<386:35:28,  7.83s/it]
 21%|██        | 47320/225000 [100:05:26<335:44:43,  6.80s/it]
 21%|██        | 47321/225000 [100:05:30<301:26:15,  6.11s/it]
 21%|██        | 47322/225000 [100:05:42<387:25:48,  7.85s/it]
 21%|██        | 47323/225000 [100:05:46<328:29:00,  6.66s/it]
 21%|██        | 47324/225000 [100:05:54<352:48:28,  7.15s/it]
 21%|██        | 47325/225000 [100:05:59<316:24:51,  6.41s/it]
 21%|██        | 47326/225000 [100:06:10<385:07:18,  7.80s/it]
 21%|██        | 47327/225000 [100:06:14<325:26:22,  6.59s/it]
 21%|██        | 47328/225000 [100:06:18<294:33:54,  5.97s/it]
 21%|██        | 47329/225000 [100:06:25<303:58:39,  6.16s/it]
 21%|██        | 47330/225000 [100:06:35<369:13:48,  7.48s/it]
 21%|██        | 47331/225000 [100:06:41<340:11:28,  6.89s/it]
 21%|██        | 47332/225000 [100:06:47<336:17:45,  6.81s/it]
 21%|██        | 47333/225000 [100:06:55<342:51:06,  6.95s/it]
 21%|██        | 47334/225000 [100:07:01<340:04:40,  6.89s/it]
 21%|██        | 47335/225000 [100:07:08<331:02:46,  6.71s/it]
 21%|██        | 47336/225000 [100:07:15<342:37:30,  6.94s/it]
 21%|██        | 47337/225000 [100:07:23<359:04:18,  7.28s/it]
 21%|██        | 47338/225000 [100:07:28<314:24:45,  6.37s/it]
 21%|██        | 47339/225000 [100:07:36<344:44:19,  6.99s/it]
 21%|██        | 47340/225000 [100:07:42<325:10:50,  6.59s/it]
 21%|██        | 47341/225000 [100:07:51<363:03:45,  7.36s/it]
 21%|██        | 47342/225000 [100:07:56<325:59:34,  6.61s/it]
 21%|██        | 47343/225000 [100:08:01<307:48:10,  6.24s/it]
 21%|██        | 47344/225000 [100:08:06<294:33:31,  5.97s/it]
 21%|██        | 47345/225000 [100:08:12<289:00:48,  5.86s/it]
 21%|██        | 47346/225000 [100:08:17<283:02:40,  5.74s/it]
 21%|██        | 47347/225000 [100:08:27<335:18:01,  6.79s/it]
 21%|██        | 47348/225000 [100:08:39<413:54:16,  8.39s/it]
 21%|██        | 47349/225000 [100:08:43<346:02:15,  7.01s/it]
 21%|██        | 47350/225000 [100:08:52<382:15:51,  7.75s/it]
                                                              
{'loss': '0.6954', 'grad_norm': '0.6849', 'learning_rate': '4.637e-05', 'epoch': '0.2104', 'num_input_tokens_seen': 168234568, 'train_runtime': '3.605e+05', 'train_tokens_per_second': '466.6'}

 21%|██        | 47350/225000 [100:08:52<382:15:51,  7.75s/it]
 21%|██        | 47351/225000 [100:08:58<352:33:26,  7.14s/it]
 21%|██        | 47352/225000 [100:09:02<312:03:15,  6.32s/it]
 21%|██        | 47353/225000 [100:09:07<290:16:44,  5.88s/it]
 21%|██        | 47354/225000 [100:09:18<370:14:44,  7.50s/it]
 21%|██        | 47355/225000 [100:09:24<346:18:35,  7.02s/it]
 21%|██        | 47356/225000 [100:09:34<382:56:19,  7.76s/it]
 21%|██        | 47357/225000 [100:09:40<369:05:36,  7.48s/it]
 21%|██        | 47358/225000 [100:09:45<327:42:06,  6.64s/it]
 21%|██        | 47359/225000 [100:09:51<311:42:45,  6.32s/it]
 21%|██        | 47360/225000 [100:10:01<368:54:47,  7.48s/it]
 21%|██        | 47361/225000 [100:10:08<356:46:37,  7.23s/it]
 21%|██        | 47362/225000 [100:10:14<338:29:36,  6.86s/it]
 21%|██        | 47363/225000 [100:10:22<356:31:31,  7.23s/it]
 21%|██        | 47364/225000 [100:10:29<355:01:46,  7.20s/it]
 21%|██        | 47365/225000 [100:10:34<332:51:51,  6.75s/it]
 21%|██        | 47366/225000 [100:10:40<316:30:09,  6.41s/it]
 21%|██        | 47367/225000 [100:10:46<304:33:58,  6.17s/it]
 21%|██        | 47368/225000 [100:10:51<297:52:20,  6.04s/it]
 21%|██        | 47369/225000 [100:10:57<284:21:43,  5.76s/it]
 21%|██        | 47370/225000 [100:11:02<280:13:13,  5.68s/it]
 21%|██        | 47371/225000 [100:11:15<392:54:02,  7.96s/it]
 21%|██        | 47372/225000 [100:11:19<335:06:03,  6.79s/it]
 21%|██        | 47373/225000 [100:11:25<315:27:42,  6.39s/it]
 21%|██        | 47374/225000 [100:11:35<364:36:02,  7.39s/it]
 21%|██        | 47375/225000 [100:11:39<316:30:30,  6.41s/it]
 21%|██        | 47376/225000 [100:11:48<364:48:43,  7.39s/it]
 21%|██        | 47377/225000 [100:11:53<324:26:23,  6.58s/it]
 21%|██        | 47378/225000 [100:12:03<373:44:30,  7.57s/it]
 21%|██        | 47379/225000 [100:12:10<365:33:44,  7.41s/it]
 21%|██        | 47380/225000 [100:12:17<355:48:49,  7.21s/it]
 21%|██        | 47381/225000 [100:12:23<334:31:56,  6.78s/it]
 21%|██        | 47382/225000 [100:12:32<374:44:12,  7.60s/it]
 21%|██        | 47383/225000 [100:12:37<341:23:31,  6.92s/it]
 21%|██        | 47384/225000 [100:12:44<337:40:10,  6.84s/it]
 21%|██        | 47385/225000 [100:12:49<316:29:32,  6.41s/it]
 21%|██        | 47386/225000 [100:12:54<288:49:55,  5.85s/it]
 21%|██        | 47387/225000 [100:13:01<300:06:59,  6.08s/it]
 21%|██        | 47388/225000 [100:13:14<402:32:34,  8.16s/it]
 21%|██        | 47389/225000 [100:13:18<352:48:49,  7.15s/it]
 21%|██        | 47390/225000 [100:13:23<321:52:29,  6.52s/it]
 21%|██        | 47391/225000 [100:13:29<300:13:40,  6.09s/it]
 21%|██        | 47392/225000 [100:13:34<287:29:24,  5.83s/it]
 21%|██        | 47393/225000 [100:13:47<397:51:21,  8.06s/it]
 21%|██        | 47394/225000 [100:13:51<342:52:55,  6.95s/it]
 21%|██        | 47395/225000 [100:14:01<380:07:46,  7.71s/it]
 21%|██        | 47396/225000 [100:14:07<360:02:23,  7.30s/it]
 21%|██        | 47397/225000 [100:14:15<364:51:33,  7.40s/it]
 21%|██        | 47398/225000 [100:14:19<320:47:21,  6.50s/it]
 21%|██        | 47399/225000 [100:14:24<294:55:37,  5.98s/it]
 21%|██        | 47400/225000 [100:14:35<372:37:18,  7.55s/it]
                                                              
{'loss': '0.6778', 'grad_norm': '0.8543', 'learning_rate': '4.636e-05', 'epoch': '0.2107', 'num_input_tokens_seen': 168385744, 'train_runtime': '3.609e+05', 'train_tokens_per_second': '466.6'}

 21%|██        | 47400/225000 [100:14:35<372:37:18,  7.55s/it]
 21%|██        | 47401/225000 [100:14:41<342:10:06,  6.94s/it]
 21%|██        | 47402/225000 [100:14:49<356:10:55,  7.22s/it]
 21%|██        | 47403/225000 [100:14:55<344:41:37,  6.99s/it]
 21%|██        | 47404/225000 [100:15:01<329:47:05,  6.68s/it]
 21%|██        | 47405/225000 [100:15:09<355:56:28,  7.22s/it]
 21%|██        | 47406/225000 [100:15:14<318:20:43,  6.45s/it]
 21%|██        | 47407/225000 [100:15:19<298:21:24,  6.05s/it]
 21%|██        | 47408/225000 [100:15:24<282:42:34,  5.73s/it]
 21%|██        | 47409/225000 [100:15:29<273:15:58,  5.54s/it]
 21%|██        | 47410/225000 [100:15:41<367:33:22,  7.45s/it]
 21%|██        | 47411/225000 [100:15:48<351:18:38,  7.12s/it]
 21%|██        | 47412/225000 [100:15:53<325:15:13,  6.59s/it]
 21%|██        | 47413/225000 [100:15:58<303:56:45,  6.16s/it]
 21%|██        | 47414/225000 [100:16:11<405:32:48,  8.22s/it]
 21%|██        | 47415/225000 [100:16:15<344:10:44,  6.98s/it]
 21%|██        | 47416/225000 [100:16:19<303:00:07,  6.14s/it]
 21%|██        | 47417/225000 [100:16:24<277:25:47,  5.62s/it]
 21%|██        | 47418/225000 [100:16:36<369:25:46,  7.49s/it]
 21%|██        | 47419/225000 [100:16:41<337:40:03,  6.85s/it]
 21%|██        | 47420/225000 [100:16:47<319:44:49,  6.48s/it]
 21%|██        | 47421/225000 [100:16:52<303:28:42,  6.15s/it]
 21%|██        | 47422/225000 [100:16:58<297:48:00,  6.04s/it]
 21%|██        | 47423/225000 [100:17:05<321:32:54,  6.52s/it]
 21%|██        | 47424/225000 [100:17:11<309:01:47,  6.26s/it]
 21%|██        | 47425/225000 [100:17:20<342:04:04,  6.93s/it]
 21%|██        | 47426/225000 [100:17:25<320:50:56,  6.50s/it]
 21%|██        | 47427/225000 [100:17:31<310:11:11,  6.29s/it]
 21%|██        | 47428/225000 [100:17:40<344:56:13,  6.99s/it]
 21%|██        | 47429/225000 [100:17:47<355:26:45,  7.21s/it]
 21%|██        | 47430/225000 [100:17:52<314:36:16,  6.38s/it]
 21%|██        | 47431/225000 [100:17:57<294:50:44,  5.98s/it]
 21%|██        | 47432/225000 [100:18:09<385:16:36,  7.81s/it]
 21%|██        | 47433/225000 [100:18:15<366:54:30,  7.44s/it]
 21%|██        | 47434/225000 [100:18:24<389:55:43,  7.91s/it]
 21%|██        | 47435/225000 [100:18:29<346:45:14,  7.03s/it]
 21%|██        | 47436/225000 [100:18:35<326:15:22,  6.61s/it]
 21%|██        | 47437/225000 [100:18:40<302:04:23,  6.12s/it]
 21%|██        | 47438/225000 [100:18:48<324:45:47,  6.58s/it]
 21%|██        | 47439/225000 [100:18:57<372:32:14,  7.55s/it]
 21%|██        | 47440/225000 [100:19:03<345:22:17,  7.00s/it]
 21%|██        | 47441/225000 [100:19:09<327:25:08,  6.64s/it]
 21%|██        | 47442/225000 [100:19:14<307:51:54,  6.24s/it]
 21%|██        | 47443/225000 [100:19:19<291:25:07,  5.91s/it]
 21%|██        | 47444/225000 [100:19:25<283:48:50,  5.75s/it]
 21%|██        | 47445/225000 [100:19:30<279:38:56,  5.67s/it]
 21%|██        | 47446/225000 [100:19:36<278:48:19,  5.65s/it]
 21%|██        | 47447/225000 [100:19:48<372:42:39,  7.56s/it]
 21%|██        | 47448/225000 [100:19:57<390:23:13,  7.92s/it]
 21%|██        | 47449/225000 [100:20:01<331:30:36,  6.72s/it]
 21%|██        | 47450/225000 [100:20:12<407:57:21,  8.27s/it]
                                                              
{'loss': '0.685', 'grad_norm': '0.652', 'learning_rate': '4.635e-05', 'epoch': '0.2109', 'num_input_tokens_seen': 168534480, 'train_runtime': '3.612e+05', 'train_tokens_per_second': '466.6'}

 21%|██        | 47450/225000 [100:20:12<407:57:21,  8.27s/it]
 21%|██        | 47451/225000 [100:20:17<350:01:18,  7.10s/it]
 21%|██        | 47452/225000 [100:20:21<310:24:29,  6.29s/it]
 21%|██        | 47453/225000 [100:20:30<340:15:15,  6.90s/it]
 21%|██        | 47454/225000 [100:20:39<370:36:48,  7.51s/it]
 21%|██        | 47455/225000 [100:20:43<318:52:50,  6.47s/it]
 21%|██        | 47456/225000 [100:20:47<292:56:40,  5.94s/it]
 21%|██        | 47457/225000 [100:20:54<298:16:07,  6.05s/it]
 21%|██        | 47458/225000 [100:21:06<392:08:07,  7.95s/it]
 21%|██        | 47459/225000 [100:21:10<340:05:08,  6.90s/it]
 21%|██        | 47460/225000 [100:21:15<303:27:30,  6.15s/it]
 21%|██        | 47461/225000 [100:21:26<379:17:48,  7.69s/it]
 21%|██        | 47462/225000 [100:21:32<356:03:07,  7.22s/it]
 21%|██        | 47463/225000 [100:21:39<354:12:56,  7.18s/it]
 21%|██        | 47464/225000 [100:21:47<365:25:42,  7.41s/it]
 21%|██        | 47465/225000 [100:21:52<322:23:41,  6.54s/it]
 21%|██        | 47466/225000 [100:21:56<295:41:56,  6.00s/it]
 21%|██        | 47467/225000 [100:22:02<283:35:49,  5.75s/it]
 21%|██        | 47468/225000 [100:22:15<391:31:17,  7.94s/it]
 21%|██        | 47469/225000 [100:22:21<371:23:44,  7.53s/it]
 21%|██        | 47470/225000 [100:22:28<358:59:32,  7.28s/it]
 21%|██        | 47471/225000 [100:22:33<320:43:09,  6.50s/it]
 21%|██        | 47472/225000 [100:22:40<336:24:45,  6.82s/it]
 21%|██        | 47473/225000 [100:22:47<335:00:33,  6.79s/it]
 21%|██        | 47474/225000 [100:22:54<336:28:29,  6.82s/it]
 21%|██        | 47475/225000 [100:23:00<332:12:45,  6.74s/it]
 21%|██        | 47476/225000 [100:23:05<300:45:20,  6.10s/it]
 21%|██        | 47477/225000 [100:23:10<278:52:32,  5.66s/it]
 21%|██        | 47478/225000 [100:23:15<273:49:38,  5.55s/it]
 21%|██        | 47479/225000 [100:23:27<376:36:27,  7.64s/it]
 21%|██        | 47480/225000 [100:23:31<316:37:29,  6.42s/it]
 21%|██        | 47481/225000 [100:23:35<282:58:28,  5.74s/it]
 21%|██        | 47482/225000 [100:23:40<265:26:44,  5.38s/it]
 21%|██        | 47483/225000 [100:23:49<316:36:37,  6.42s/it]
 21%|██        | 47484/225000 [100:23:58<358:43:12,  7.27s/it]
 21%|██        | 47485/225000 [100:24:02<306:34:35,  6.22s/it]
 21%|██        | 47486/225000 [100:24:11<356:27:59,  7.23s/it]
 21%|██        | 47487/225000 [100:24:16<323:21:19,  6.56s/it]
 21%|██        | 47488/225000 [100:24:22<312:41:43,  6.34s/it]
 21%|██        | 47489/225000 [100:24:27<293:18:28,  5.95s/it]
 21%|██        | 47490/225000 [100:24:32<281:18:03,  5.70s/it]
 21%|██        | 47491/225000 [100:24:37<273:15:44,  5.54s/it]
 21%|██        | 47492/225000 [100:24:43<272:07:05,  5.52s/it]
 21%|██        | 47493/225000 [100:24:50<299:44:30,  6.08s/it]
 21%|██        | 47494/225000 [100:25:03<403:24:56,  8.18s/it]
 21%|██        | 47495/225000 [100:25:09<360:05:58,  7.30s/it]
 21%|██        | 47496/225000 [100:25:17<382:49:51,  7.76s/it]
 21%|██        | 47497/225000 [100:25:21<328:59:36,  6.67s/it]
 21%|██        | 47498/225000 [100:25:29<336:08:31,  6.82s/it]
 21%|██        | 47499/225000 [100:25:38<374:48:37,  7.60s/it]
 21%|██        | 47500/225000 [100:25:43<340:55:58,  6.91s/it]
                                                              
{'loss': '0.7092', 'grad_norm': '0.7361', 'learning_rate': '4.634e-05', 'epoch': '0.2111', 'num_input_tokens_seen': 168682904, 'train_runtime': '3.615e+05', 'train_tokens_per_second': '466.6'}

 21%|██        | 47500/225000 [100:25:43<340:55:58,  6.91s/it]
 21%|██        | 47501/225000 [100:25:48<312:17:31,  6.33s/it]
 21%|██        | 47502/225000 [100:25:53<294:27:42,  5.97s/it]
 21%|██        | 47503/225000 [100:25:59<282:40:52,  5.73s/it]
 21%|██        | 47504/225000 [100:26:12<388:23:52,  7.88s/it]
 21%|██        | 47505/225000 [100:26:17<354:48:47,  7.20s/it]
 21%|██        | 47506/225000 [100:26:21<310:07:53,  6.29s/it]
 21%|██        | 47507/225000 [100:26:34<411:05:41,  8.34s/it]
 21%|██        | 47508/225000 [100:26:38<346:33:15,  7.03s/it]
 21%|██        | 47509/225000 [100:26:44<328:26:36,  6.66s/it]
 21%|██        | 47510/225000 [100:26:50<308:19:02,  6.25s/it]
 21%|██        | 47511/225000 [100:26:56<318:31:45,  6.46s/it]
 21%|██        | 47512/225000 [100:27:06<367:06:46,  7.45s/it]
 21%|██        | 47513/225000 [100:27:13<362:39:52,  7.36s/it]
 21%|██        | 47514/225000 [100:27:19<331:42:15,  6.73s/it]
 21%|██        | 47515/225000 [100:27:27<351:26:34,  7.13s/it]
 21%|██        | 47516/225000 [100:27:35<362:09:57,  7.35s/it]
 21%|██        | 47517/225000 [100:27:38<310:07:43,  6.29s/it]
 21%|██        | 47518/225000 [100:27:49<372:40:26,  7.56s/it]
 21%|██        | 47519/225000 [100:27:52<314:24:16,  6.38s/it]
 21%|██        | 47520/225000 [100:28:02<356:09:35,  7.22s/it]
 21%|██        | 47521/225000 [100:28:07<334:04:45,  6.78s/it]
 21%|██        | 47522/225000 [100:28:11<289:04:21,  5.86s/it]
 21%|██        | 47523/225000 [100:28:16<280:34:55,  5.69s/it]
 21%|██        | 47524/225000 [100:28:22<277:56:16,  5.64s/it]
 21%|██        | 47525/225000 [100:28:34<368:20:25,  7.47s/it]
 21%|██        | 47526/225000 [100:28:41<360:16:01,  7.31s/it]
 21%|██        | 47527/225000 [100:28:45<323:51:55,  6.57s/it]
 21%|██        | 47528/225000 [100:28:51<306:49:18,  6.22s/it]
 21%|██        | 47529/225000 [100:29:02<386:02:04,  7.83s/it]
 21%|██        | 47530/225000 [100:29:07<332:04:33,  6.74s/it]
 21%|██        | 47531/225000 [100:29:14<337:18:02,  6.84s/it]
 21%|██        | 47532/225000 [100:29:20<326:29:07,  6.62s/it]
 21%|██        | 47533/225000 [100:29:24<296:27:13,  6.01s/it]
 21%|██        | 47534/225000 [100:29:35<363:53:26,  7.38s/it]
 21%|██        | 47535/225000 [100:29:41<347:50:21,  7.06s/it]
 21%|██        | 47536/225000 [100:29:45<301:28:49,  6.12s/it]
 21%|██        | 47537/225000 [100:29:54<347:35:58,  7.05s/it]
 21%|██        | 47538/225000 [100:30:02<354:19:25,  7.19s/it]
 21%|██        | 47539/225000 [100:30:08<333:55:38,  6.77s/it]
 21%|██        | 47540/225000 [100:30:13<317:58:14,  6.45s/it]
 21%|██        | 47541/225000 [100:30:19<298:42:39,  6.06s/it]
 21%|██        | 47542/225000 [100:30:29<358:05:40,  7.26s/it]
 21%|██        | 47543/225000 [100:30:34<328:34:55,  6.67s/it]
 21%|██        | 47544/225000 [100:30:39<300:04:17,  6.09s/it]
 21%|██        | 47545/225000 [100:30:44<291:21:32,  5.91s/it]
 21%|██        | 47546/225000 [100:30:56<371:41:11,  7.54s/it]
 21%|██        | 47547/225000 [100:31:01<338:32:15,  6.87s/it]
 21%|██        | 47548/225000 [100:31:06<306:51:21,  6.23s/it]
 21%|██        | 47549/225000 [100:31:11<290:06:32,  5.89s/it]
 21%|██        | 47550/225000 [100:31:16<281:06:41,  5.70s/it]
                                                              
{'loss': '0.692', 'grad_norm': '0.8093', 'learning_rate': '4.633e-05', 'epoch': '0.2113', 'num_input_tokens_seen': 168832320, 'train_runtime': '3.619e+05', 'train_tokens_per_second': '466.5'}

 21%|██        | 47550/225000 [100:31:16<281:06:41,  5.70s/it]
 21%|██        | 47551/225000 [100:31:22<279:48:43,  5.68s/it]
 21%|██        | 47552/225000 [100:31:35<388:11:34,  7.88s/it]
 21%|██        | 47553/225000 [100:31:41<366:59:44,  7.45s/it]
 21%|██        | 47554/225000 [100:31:45<319:42:31,  6.49s/it]
 21%|██        | 47555/225000 [100:31:51<302:11:38,  6.13s/it]
 21%|██        | 47556/225000 [100:32:03<392:37:34,  7.97s/it]
 21%|██        | 47557/225000 [100:32:07<340:32:58,  6.91s/it]
 21%|██        | 47558/225000 [100:32:11<299:24:08,  6.07s/it]
 21%|██        | 47559/225000 [100:32:24<398:48:29,  8.09s/it]
 21%|██        | 47560/225000 [100:32:28<334:19:35,  6.78s/it]
 21%|██        | 47561/225000 [100:32:33<307:18:04,  6.23s/it]
 21%|██        | 47562/225000 [100:32:43<367:45:17,  7.46s/it]
 21%|██        | 47563/225000 [100:32:47<310:38:42,  6.30s/it]
 21%|██        | 47564/225000 [100:32:55<344:24:02,  6.99s/it]
 21%|██        | 47565/225000 [100:33:00<303:50:00,  6.16s/it]
 21%|██        | 47566/225000 [100:33:04<279:30:47,  5.67s/it]
 21%|██        | 47567/225000 [100:33:14<341:07:48,  6.92s/it]
 21%|██        | 47568/225000 [100:33:20<330:12:46,  6.70s/it]
 21%|██        | 47569/225000 [100:33:28<348:21:49,  7.07s/it]
 21%|██        | 47570/225000 [100:33:33<311:32:25,  6.32s/it]
 21%|██        | 47571/225000 [100:33:37<288:13:04,  5.85s/it]
 21%|██        | 47572/225000 [100:33:48<362:01:26,  7.35s/it]
 21%|██        | 47573/225000 [100:33:56<364:22:17,  7.39s/it]
 21%|██        | 47574/225000 [100:34:03<361:42:52,  7.34s/it]
 21%|██        | 47575/225000 [100:34:08<332:13:03,  6.74s/it]
 21%|██        | 47576/225000 [100:34:14<312:08:12,  6.33s/it]
 21%|██        | 47577/225000 [100:34:19<302:08:59,  6.13s/it]
 21%|██        | 47578/225000 [100:34:25<293:11:18,  5.95s/it]
 21%|██        | 47579/225000 [100:34:30<281:47:46,  5.72s/it]
 21%|██        | 47580/225000 [100:34:40<336:57:44,  6.84s/it]
 21%|██        | 47581/225000 [100:34:49<372:48:22,  7.56s/it]
 21%|██        | 47582/225000 [100:34:55<357:35:25,  7.26s/it]
 21%|██        | 47583/225000 [100:35:02<346:47:18,  7.04s/it]
 21%|██        | 47584/225000 [100:35:10<359:40:51,  7.30s/it]
 21%|██        | 47585/225000 [100:35:13<306:43:15,  6.22s/it]
 21%|██        | 47586/225000 [100:35:23<360:59:19,  7.33s/it]
 21%|██        | 47587/225000 [100:35:27<306:29:15,  6.22s/it]
 21%|██        | 47588/225000 [100:35:32<283:28:13,  5.75s/it]
 21%|██        | 47589/225000 [100:35:36<269:10:31,  5.46s/it]
 21%|██        | 47590/225000 [100:35:42<269:46:05,  5.47s/it]
 21%|██        | 47591/225000 [100:35:47<264:28:56,  5.37s/it]
 21%|██        | 47592/225000 [100:35:55<307:12:04,  6.23s/it]
 21%|██        | 47593/225000 [100:36:07<391:18:46,  7.94s/it]
 21%|██        | 47594/225000 [100:36:11<326:38:52,  6.63s/it]
 21%|██        | 47595/225000 [100:36:19<356:04:54,  7.23s/it]
 21%|██        | 47596/225000 [100:36:27<359:07:04,  7.29s/it]
 21%|██        | 47597/225000 [100:36:31<316:36:03,  6.42s/it]
 21%|██        | 47598/225000 [100:36:37<307:06:57,  6.23s/it]
 21%|██        | 47599/225000 [100:36:42<294:55:13,  5.98s/it]
 21%|██        | 47600/225000 [100:36:48<294:43:57,  5.98s/it]
                                                              
{'loss': '0.7088', 'grad_norm': '0.6588', 'learning_rate': '4.633e-05', 'epoch': '0.2116', 'num_input_tokens_seen': 168982552, 'train_runtime': '3.622e+05', 'train_tokens_per_second': '466.5'}

 21%|██        | 47600/225000 [100:36:48<294:43:57,  5.98s/it]
 21%|██        | 47601/225000 [100:36:54<289:19:18,  5.87s/it]
 21%|██        | 47602/225000 [100:37:05<357:30:07,  7.25s/it]
 21%|██        | 47603/225000 [100:37:14<388:14:12,  7.88s/it]
 21%|██        | 47604/225000 [100:37:20<365:29:18,  7.42s/it]
 21%|██        | 47605/225000 [100:37:27<360:04:52,  7.31s/it]
 21%|██        | 47606/225000 [100:37:31<308:51:18,  6.27s/it]
 21%|██        | 47607/225000 [100:37:40<350:57:12,  7.12s/it]
 21%|██        | 47608/225000 [100:37:47<351:49:56,  7.14s/it]
 21%|██        | 47609/225000 [100:37:54<349:40:28,  7.10s/it]
 21%|██        | 47610/225000 [100:38:00<320:46:51,  6.51s/it]
 21%|██        | 47611/225000 [100:38:07<337:16:20,  6.84s/it]
 21%|██        | 47612/225000 [100:38:13<318:28:29,  6.46s/it]
 21%|██        | 47613/225000 [100:38:19<314:50:00,  6.39s/it]
 21%|██        | 47614/225000 [100:38:23<285:14:56,  5.79s/it]
 21%|██        | 47615/225000 [100:38:29<276:43:00,  5.62s/it]
 21%|██        | 47616/225000 [100:38:41<374:30:16,  7.60s/it]
 21%|██        | 47617/225000 [100:38:46<333:30:06,  6.77s/it]
 21%|██        | 47618/225000 [100:38:50<303:51:17,  6.17s/it]
 21%|██        | 47619/225000 [100:38:55<281:16:22,  5.71s/it]
 21%|██        | 47620/225000 [100:39:01<281:51:25,  5.72s/it]
 21%|██        | 47621/225000 [100:39:14<392:09:14,  7.96s/it]
 21%|██        | 47622/225000 [100:39:18<328:33:59,  6.67s/it]
 21%|██        | 47623/225000 [100:39:26<353:02:31,  7.17s/it]
 21%|██        | 47624/225000 [100:39:31<322:39:41,  6.55s/it]
 21%|██        | 47625/225000 [100:39:39<349:03:53,  7.08s/it]
 21%|██        | 47626/225000 [100:39:44<309:34:16,  6.28s/it]
 21%|██        | 47627/225000 [100:39:52<344:33:22,  6.99s/it]
 21%|██        | 47628/225000 [100:39:56<293:17:25,  5.95s/it]
 21%|██        | 47629/225000 [100:40:00<268:50:37,  5.46s/it]
 21%|██        | 47630/225000 [100:40:12<355:52:36,  7.22s/it]
 21%|██        | 47631/225000 [100:40:17<333:03:46,  6.76s/it]
 21%|██        | 47632/225000 [100:40:22<307:14:28,  6.24s/it]
 21%|██        | 47633/225000 [100:40:28<292:32:25,  5.94s/it]
 21%|██        | 47634/225000 [100:40:33<281:20:09,  5.71s/it]
 21%|██        | 47635/225000 [100:40:42<334:46:33,  6.79s/it]
 21%|██        | 47636/225000 [100:40:53<390:38:07,  7.93s/it]
 21%|██        | 47637/225000 [100:40:59<371:36:35,  7.54s/it]
 21%|██        | 47638/225000 [100:41:06<354:35:01,  7.20s/it]
 21%|██        | 47639/225000 [100:41:10<315:15:17,  6.40s/it]
 21%|██        | 47640/225000 [100:41:18<336:53:02,  6.84s/it]
 21%|██        | 47641/225000 [100:41:23<315:07:08,  6.40s/it]
 21%|██        | 47642/225000 [100:41:31<338:59:37,  6.88s/it]
 21%|██        | 47643/225000 [100:41:38<339:05:31,  6.88s/it]
 21%|██        | 47644/225000 [100:41:45<334:39:48,  6.79s/it]
 21%|██        | 47645/225000 [100:41:50<305:16:26,  6.20s/it]
 21%|██        | 47646/225000 [100:41:55<289:10:42,  5.87s/it]
 21%|██        | 47647/225000 [100:42:00<276:24:16,  5.61s/it]
 21%|██        | 47648/225000 [100:42:13<383:00:24,  7.77s/it]
 21%|██        | 47649/225000 [100:42:18<352:04:26,  7.15s/it]
 21%|██        | 47650/225000 [100:42:27<379:30:57,  7.70s/it]
                                                              
{'loss': '0.6902', 'grad_norm': '0.7445', 'learning_rate': '4.632e-05', 'epoch': '0.2118', 'num_input_tokens_seen': 169130736, 'train_runtime': '3.625e+05', 'train_tokens_per_second': '466.5'}

 21%|██        | 47650/225000 [100:42:27<379:30:57,  7.70s/it]
 21%|██        | 47651/225000 [100:42:31<326:58:41,  6.64s/it]
 21%|██        | 47652/225000 [100:42:39<336:56:57,  6.84s/it]
 21%|██        | 47653/225000 [100:42:47<363:23:58,  7.38s/it]
 21%|██        | 47654/225000 [100:42:56<374:13:27,  7.60s/it]
 21%|██        | 47655/225000 [100:43:00<320:48:54,  6.51s/it]
 21%|██        | 47656/225000 [100:43:05<299:23:46,  6.08s/it]
 21%|██        | 47657/225000 [100:43:15<365:00:51,  7.41s/it]
 21%|██        | 47658/225000 [100:43:19<320:11:20,  6.50s/it]
 21%|██        | 47659/225000 [100:43:24<293:16:11,  5.95s/it]
 21%|██        | 47660/225000 [100:43:31<301:15:38,  6.12s/it]
 21%|██        | 47661/225000 [100:43:41<360:44:59,  7.32s/it]
 21%|██        | 47662/225000 [100:43:47<339:58:46,  6.90s/it]
 21%|██        | 47663/225000 [100:43:55<358:13:36,  7.27s/it]
 21%|██        | 47664/225000 [100:43:59<313:57:03,  6.37s/it]
 21%|██        | 47665/225000 [100:44:07<338:22:07,  6.87s/it]
 21%|██        | 47666/225000 [100:44:16<370:40:51,  7.53s/it]
 21%|██        | 47667/225000 [100:44:23<364:15:45,  7.39s/it]
 21%|██        | 47668/225000 [100:44:29<332:28:03,  6.75s/it]
 21%|██        | 47669/225000 [100:44:33<292:46:07,  5.94s/it]
 21%|██        | 47670/225000 [100:44:39<304:33:12,  6.18s/it]
 21%|██        | 47671/225000 [100:44:51<387:50:48,  7.87s/it]
 21%|██        | 47672/225000 [100:44:55<331:04:39,  6.72s/it]
 21%|██        | 47673/225000 [100:45:03<344:36:01,  7.00s/it]
 21%|██        | 47674/225000 [100:45:11<355:05:06,  7.21s/it]
 21%|██        | 47675/225000 [100:45:16<327:22:52,  6.65s/it]
 21%|██        | 47676/225000 [100:45:24<343:38:42,  6.98s/it]
 21%|██        | 47677/225000 [100:45:28<305:31:26,  6.20s/it]
 21%|██        | 47678/225000 [100:45:32<277:25:30,  5.63s/it]
 21%|██        | 47679/225000 [100:45:39<287:56:16,  5.85s/it]
 21%|██        | 47680/225000 [100:45:50<372:20:18,  7.56s/it]
 21%|██        | 47681/225000 [100:45:54<319:15:21,  6.48s/it]
 21%|██        | 47682/225000 [100:45:59<288:59:29,  5.87s/it]
 21%|██        | 47683/225000 [100:46:06<311:51:34,  6.33s/it]
 21%|██        | 47684/225000 [100:46:15<346:43:23,  7.04s/it]
 21%|██        | 47685/225000 [100:46:19<301:34:36,  6.12s/it]
 21%|██        | 47686/225000 [100:46:24<288:22:40,  5.85s/it]
 21%|██        | 47687/225000 [100:46:38<409:27:48,  8.31s/it]
 21%|██        | 47688/225000 [100:46:42<343:54:16,  6.98s/it]
 21%|██        | 47689/225000 [100:46:50<363:12:51,  7.37s/it]
 21%|██        | 47690/225000 [100:46:55<332:24:15,  6.75s/it]
 21%|██        | 47691/225000 [100:47:04<358:35:57,  7.28s/it]
 21%|██        | 47692/225000 [100:47:10<336:58:42,  6.84s/it]
 21%|██        | 47693/225000 [100:47:15<319:01:43,  6.48s/it]
 21%|██        | 47694/225000 [100:47:24<353:15:43,  7.17s/it]
 21%|██        | 47695/225000 [100:47:28<306:11:56,  6.22s/it]
 21%|██        | 47696/225000 [100:47:33<286:44:53,  5.82s/it]
 21%|██        | 47697/225000 [100:47:38<268:37:36,  5.45s/it]
 21%|██        | 47698/225000 [100:47:46<308:51:08,  6.27s/it]
 21%|██        | 47699/225000 [100:47:55<350:21:27,  7.11s/it]
 21%|██        | 47700/225000 [100:47:59<306:28:43,  6.22s/it]
                                                              
{'loss': '0.7031', 'grad_norm': '0.7468', 'learning_rate': '4.631e-05', 'epoch': '0.212', 'num_input_tokens_seen': 169280488, 'train_runtime': '3.629e+05', 'train_tokens_per_second': '466.5'}

 21%|██        | 47700/225000 [100:47:59<306:28:43,  6.22s/it]
 21%|██        | 47701/225000 [100:48:06<321:59:04,  6.54s/it]
 21%|██        | 47702/225000 [100:48:16<374:20:52,  7.60s/it]
 21%|██        | 47703/225000 [100:48:20<322:00:28,  6.54s/it]
 21%|██        | 47704/225000 [100:48:31<380:28:22,  7.73s/it]
 21%|██        | 47705/225000 [100:48:37<352:05:55,  7.15s/it]
 21%|██        | 47706/225000 [100:48:45<365:10:49,  7.42s/it]
 21%|██        | 47707/225000 [100:48:51<345:19:17,  7.01s/it]
 21%|██        | 47708/225000 [100:48:58<345:59:04,  7.03s/it]
 21%|██        | 47709/225000 [100:49:04<325:37:01,  6.61s/it]
 21%|██        | 47710/225000 [100:49:11<335:02:07,  6.80s/it]
 21%|██        | 47711/225000 [100:49:16<310:00:38,  6.30s/it]
 21%|██        | 47712/225000 [100:49:21<297:56:45,  6.05s/it]
 21%|██        | 47713/225000 [100:49:27<292:10:27,  5.93s/it]
 21%|██        | 47714/225000 [100:49:33<287:06:37,  5.83s/it]
 21%|██        | 47715/225000 [100:49:38<273:16:03,  5.55s/it]
 21%|██        | 47716/225000 [100:49:43<265:10:09,  5.38s/it]
 21%|██        | 47717/225000 [100:49:56<380:38:42,  7.73s/it]
 21%|██        | 47718/225000 [100:50:02<353:22:34,  7.18s/it]
 21%|██        | 47719/225000 [100:50:07<324:17:59,  6.59s/it]
 21%|██        | 47720/225000 [100:50:18<393:54:01,  8.00s/it]
 21%|██        | 47721/225000 [100:50:23<340:17:30,  6.91s/it]
 21%|██        | 47722/225000 [100:50:27<299:14:44,  6.08s/it]
 21%|██        | 47723/225000 [100:50:36<340:39:45,  6.92s/it]
 21%|██        | 47724/225000 [100:50:42<329:40:30,  6.69s/it]
 21%|██        | 47725/225000 [100:50:51<370:15:52,  7.52s/it]
 21%|██        | 47726/225000 [100:50:55<310:40:00,  6.31s/it]
 21%|██        | 47727/225000 [100:51:04<356:50:55,  7.25s/it]
 21%|██        | 47728/225000 [100:51:10<337:42:54,  6.86s/it]
 21%|██        | 47729/225000 [100:51:17<332:58:31,  6.76s/it]
 21%|██        | 47730/225000 [100:51:23<334:44:16,  6.80s/it]
 21%|██        | 47731/225000 [100:51:29<313:39:46,  6.37s/it]
 21%|██        | 47732/225000 [100:51:34<299:12:41,  6.08s/it]
 21%|██        | 47733/225000 [100:51:39<285:02:08,  5.79s/it]
 21%|██        | 47734/225000 [100:51:45<290:21:07,  5.90s/it]
 21%|██        | 47735/225000 [100:51:51<282:03:38,  5.73s/it]
 21%|██        | 47736/225000 [100:51:57<287:13:07,  5.83s/it]
 21%|██        | 47737/225000 [100:52:06<332:28:19,  6.75s/it]
 21%|██        | 47738/225000 [100:52:17<400:08:25,  8.13s/it]
 21%|██        | 47739/225000 [100:52:21<342:42:48,  6.96s/it]
 21%|██        | 47740/225000 [100:52:28<339:22:32,  6.89s/it]
 21%|██        | 47741/225000 [100:52:38<390:35:36,  7.93s/it]
 21%|██        | 47742/225000 [100:52:44<347:54:23,  7.07s/it]
 21%|██        | 47743/225000 [100:52:51<353:16:02,  7.17s/it]
 21%|██        | 47744/225000 [100:52:56<327:22:33,  6.65s/it]
 21%|██        | 47745/225000 [100:53:02<306:20:50,  6.22s/it]
 21%|██        | 47746/225000 [100:53:07<301:02:34,  6.11s/it]
 21%|██        | 47747/225000 [100:53:13<291:56:58,  5.93s/it]
 21%|██        | 47748/225000 [100:53:18<283:28:36,  5.76s/it]
 21%|██        | 47749/225000 [100:53:24<280:45:59,  5.70s/it]
 21%|██        | 47750/225000 [100:53:36<382:20:43,  7.77s/it]
                                                              
{'loss': '0.6986', 'grad_norm': '0.8311', 'learning_rate': '4.63e-05', 'epoch': '0.2122', 'num_input_tokens_seen': 169428296, 'train_runtime': '3.632e+05', 'train_tokens_per_second': '466.5'}

 21%|██        | 47750/225000 [100:53:36<382:20:43,  7.77s/it]
 21%|██        | 47751/225000 [100:53:44<382:50:05,  7.78s/it]
 21%|██        | 47752/225000 [100:53:51<364:05:36,  7.39s/it]
 21%|██        | 47753/225000 [100:53:58<365:45:22,  7.43s/it]
 21%|██        | 47754/225000 [100:54:03<318:57:28,  6.48s/it]
 21%|██        | 47755/225000 [100:54:13<383:05:27,  7.78s/it]
 21%|██        | 47756/225000 [100:54:20<360:40:49,  7.33s/it]
 21%|██        | 47757/225000 [100:54:23<304:49:21,  6.19s/it]
 21%|██        | 47758/225000 [100:54:32<344:08:52,  6.99s/it]
 21%|██        | 47759/225000 [100:54:36<292:37:10,  5.94s/it]
 21%|██        | 47760/225000 [100:54:46<356:52:38,  7.25s/it]
 21%|██        | 47761/225000 [100:54:50<312:34:08,  6.35s/it]
 21%|██        | 47762/225000 [100:54:54<274:38:43,  5.58s/it]
 21%|██        | 47763/225000 [100:55:06<376:33:27,  7.65s/it]
 21%|██        | 47764/225000 [100:55:10<317:22:42,  6.45s/it]
 21%|██        | 47765/225000 [100:55:18<335:37:30,  6.82s/it]
 21%|██        | 47766/225000 [100:55:22<298:47:36,  6.07s/it]
 21%|██        | 47767/225000 [100:55:27<279:40:59,  5.68s/it]
 21%|██        | 47768/225000 [100:55:31<264:59:19,  5.38s/it]
 21%|██        | 47769/225000 [100:55:41<321:34:58,  6.53s/it]
 21%|██        | 47770/225000 [100:55:49<346:34:19,  7.04s/it]
 21%|██        | 47771/225000 [100:55:54<319:42:52,  6.49s/it]
 21%|██        | 47772/225000 [100:55:59<303:17:20,  6.16s/it]
 21%|██        | 47773/225000 [100:56:05<295:16:09,  6.00s/it]
 21%|██        | 47774/225000 [100:56:10<284:02:41,  5.77s/it]
 21%|██        | 47775/225000 [100:56:23<385:28:18,  7.83s/it]
 21%|██        | 47776/225000 [100:56:28<348:51:24,  7.09s/it]
 21%|██        | 47777/225000 [100:56:34<320:55:21,  6.52s/it]
 21%|██        | 47778/225000 [100:56:39<301:32:00,  6.13s/it]
 21%|██        | 47779/225000 [100:56:44<290:29:20,  5.90s/it]
 21%|██        | 47780/225000 [100:56:53<339:51:19,  6.90s/it]
 21%|██        | 47781/225000 [100:57:04<398:42:15,  8.10s/it]
 21%|██        | 47782/225000 [100:57:12<388:43:51,  7.90s/it]
 21%|██        | 47783/225000 [100:57:17<346:59:31,  7.05s/it]
 21%|██        | 47784/225000 [100:57:24<346:46:27,  7.04s/it]
 21%|██        | 47785/225000 [100:57:33<379:20:21,  7.71s/it]
 21%|██        | 47786/225000 [100:57:37<327:58:52,  6.66s/it]
 21%|██        | 47787/225000 [100:57:42<301:36:07,  6.13s/it]
 21%|██        | 47788/225000 [100:57:48<292:08:39,  5.93s/it]
 21%|██        | 47789/225000 [100:57:59<379:43:58,  7.71s/it]
 21%|██        | 47790/225000 [100:58:04<339:48:00,  6.90s/it]
 21%|██        | 47791/225000 [100:58:10<314:31:06,  6.39s/it]
 21%|██        | 47792/225000 [100:58:15<305:36:17,  6.21s/it]
 21%|██        | 47793/225000 [100:58:28<399:58:14,  8.13s/it]
 21%|██        | 47794/225000 [100:58:32<336:31:12,  6.84s/it]
 21%|██        | 47795/225000 [100:58:40<351:22:17,  7.14s/it]
 21%|██        | 47796/225000 [100:58:44<309:28:55,  6.29s/it]
 21%|██        | 47797/225000 [100:58:48<279:51:16,  5.69s/it]
 21%|██        | 47798/225000 [100:59:01<377:44:27,  7.67s/it]
 21%|██        | 47799/225000 [100:59:07<357:31:43,  7.26s/it]
 21%|██        | 47800/225000 [100:59:11<315:47:30,  6.42s/it]
                                                              
{'loss': '0.6847', 'grad_norm': '0.6532', 'learning_rate': '4.629e-05', 'epoch': '0.2124', 'num_input_tokens_seen': 169577392, 'train_runtime': '3.636e+05', 'train_tokens_per_second': '466.4'}

 21%|██        | 47800/225000 [100:59:11<315:47:30,  6.42s/it]
 21%|██        | 47801/225000 [100:59:21<368:00:22,  7.48s/it]
 21%|██        | 47802/225000 [100:59:25<309:39:46,  6.29s/it]
 21%|██        | 47803/225000 [100:59:30<297:15:38,  6.04s/it]
 21%|██        | 47804/225000 [100:59:36<293:55:55,  5.97s/it]
 21%|██        | 47805/225000 [100:59:42<285:50:14,  5.81s/it]
 21%|██        | 47806/225000 [100:59:47<280:29:19,  5.70s/it]
 21%|██        | 47807/225000 [100:59:59<380:39:11,  7.73s/it]
 21%|██        | 47808/225000 [101:00:08<396:16:57,  8.05s/it]
 21%|██        | 47809/225000 [101:00:12<338:24:17,  6.88s/it]
 21%|██        | 47810/225000 [101:00:17<309:58:45,  6.30s/it]
 21%|██        | 47811/225000 [101:00:22<291:09:32,  5.92s/it]
 21%|██        | 47812/225000 [101:00:28<289:13:14,  5.88s/it]
 21%|██▏       | 47813/225000 [101:00:34<286:49:54,  5.83s/it]
 21%|██▏       | 47814/225000 [101:00:48<406:21:46,  8.26s/it]
 21%|██▏       | 47815/225000 [101:00:55<391:08:59,  7.95s/it]
 21%|██▏       | 47816/225000 [101:01:00<342:42:18,  6.96s/it]
 21%|██▏       | 47817/225000 [101:01:05<320:19:58,  6.51s/it]
 21%|██▏       | 47818/225000 [101:01:17<394:54:34,  8.02s/it]
 21%|██▏       | 47819/225000 [101:01:22<355:47:23,  7.23s/it]
 21%|██▏       | 47820/225000 [101:01:27<321:54:23,  6.54s/it]
 21%|██▏       | 47821/225000 [101:01:32<298:32:39,  6.07s/it]
 21%|██▏       | 47822/225000 [101:01:37<277:13:04,  5.63s/it]
 21%|██▏       | 47823/225000 [101:01:49<380:06:11,  7.72s/it]
 21%|██▏       | 47824/225000 [101:01:54<336:44:20,  6.84s/it]
 21%|██▏       | 47825/225000 [101:02:03<362:31:55,  7.37s/it]
 21%|██▏       | 47826/225000 [101:02:09<347:07:40,  7.05s/it]
 21%|██▏       | 47827/225000 [101:02:16<355:02:22,  7.21s/it]
 21%|██▏       | 47828/225000 [101:02:20<305:01:33,  6.20s/it]
 21%|██▏       | 47829/225000 [101:02:31<371:29:33,  7.55s/it]
 21%|██▏       | 47830/225000 [101:02:35<312:31:21,  6.35s/it]
 21%|██▏       | 47831/225000 [101:02:44<357:12:43,  7.26s/it]
 21%|██▏       | 47832/225000 [101:02:48<305:19:12,  6.20s/it]
 21%|██▏       | 47833/225000 [101:02:54<312:16:35,  6.35s/it]
 21%|██▏       | 47834/225000 [101:03:02<333:49:56,  6.78s/it]
 21%|██▏       | 47835/225000 [101:03:09<331:00:42,  6.73s/it]
 21%|██▏       | 47836/225000 [101:03:13<297:20:58,  6.04s/it]
 21%|██▏       | 47837/225000 [101:03:20<314:59:39,  6.40s/it]
 21%|██▏       | 47838/225000 [101:03:30<358:28:58,  7.28s/it]
 21%|██▏       | 47839/225000 [101:03:38<366:30:41,  7.45s/it]
 21%|██▏       | 47840/225000 [101:03:43<335:57:33,  6.83s/it]
 21%|██▏       | 47841/225000 [101:03:51<349:19:42,  7.10s/it]
 21%|██▏       | 47842/225000 [101:03:55<308:03:46,  6.26s/it]
 21%|██▏       | 47843/225000 [101:04:00<296:45:24,  6.03s/it]
 21%|██▏       | 47844/225000 [101:04:06<287:45:31,  5.85s/it]
 21%|██▏       | 47845/225000 [101:04:11<281:45:42,  5.73s/it]
 21%|██▏       | 47846/225000 [101:04:16<271:17:34,  5.51s/it]
 21%|██▏       | 47847/225000 [101:04:22<268:44:27,  5.46s/it]
 21%|██▏       | 47848/225000 [101:04:29<299:29:37,  6.09s/it]
 21%|██▏       | 47849/225000 [101:04:43<411:05:59,  8.35s/it]
 21%|██▏       | 47850/225000 [101:04:49<372:58:30,  7.58s/it]
                                                              
{'loss': '0.7018', 'grad_norm': '0.7676', 'learning_rate': '4.628e-05', 'epoch': '0.2127', 'num_input_tokens_seen': 169725656, 'train_runtime': '3.639e+05', 'train_tokens_per_second': '466.4'}

 21%|██▏       | 47850/225000 [101:04:49<372:58:30,  7.58s/it]
 21%|██▏       | 47851/225000 [101:04:53<324:34:40,  6.60s/it]
 21%|██▏       | 47852/225000 [101:04:57<292:49:09,  5.95s/it]
 21%|██▏       | 47853/225000 [101:05:10<396:40:04,  8.06s/it]
 21%|██▏       | 47854/225000 [101:05:15<352:14:21,  7.16s/it]
 21%|██▏       | 47855/225000 [101:05:21<330:11:09,  6.71s/it]
 21%|██▏       | 47856/225000 [101:05:26<304:06:19,  6.18s/it]
 21%|██▏       | 47857/225000 [101:05:31<281:54:38,  5.73s/it]
 21%|██▏       | 47858/225000 [101:05:36<271:20:32,  5.51s/it]
 21%|██▏       | 47859/225000 [101:05:47<350:29:23,  7.12s/it]
 21%|██▏       | 47860/225000 [101:05:56<387:54:09,  7.88s/it]
 21%|██▏       | 47861/225000 [101:06:05<400:48:20,  8.15s/it]
 21%|██▏       | 47862/225000 [101:06:09<344:29:13,  7.00s/it]
 21%|██▏       | 47863/225000 [101:06:15<329:35:35,  6.70s/it]
 21%|██▏       | 47864/225000 [101:06:22<332:06:47,  6.75s/it]
 21%|██▏       | 47865/225000 [101:06:26<292:10:16,  5.94s/it]
 21%|██▏       | 47866/225000 [101:06:35<326:40:55,  6.64s/it]
 21%|██▏       | 47867/225000 [101:06:43<350:23:51,  7.12s/it]
 21%|██▏       | 47868/225000 [101:06:51<370:11:34,  7.52s/it]
 21%|██▏       | 47869/225000 [101:06:56<323:15:26,  6.57s/it]
 21%|██▏       | 47870/225000 [101:07:00<292:17:05,  5.94s/it]
 21%|██▏       | 47871/225000 [101:07:09<335:04:27,  6.81s/it]
 21%|██▏       | 47872/225000 [101:07:17<360:58:08,  7.34s/it]
 21%|██▏       | 47873/225000 [101:07:25<357:56:07,  7.27s/it]
 21%|██▏       | 47874/225000 [101:07:29<319:37:01,  6.50s/it]
 21%|██▏       | 47875/225000 [101:07:39<364:55:20,  7.42s/it]
 21%|██▏       | 47876/225000 [101:07:43<321:03:15,  6.53s/it]
 21%|██▏       | 47877/225000 [101:07:51<343:10:30,  6.97s/it]
 21%|██▏       | 47878/225000 [101:07:56<302:47:31,  6.15s/it]
 21%|██▏       | 47879/225000 [101:08:03<319:50:58,  6.50s/it]
 21%|██▏       | 47880/225000 [101:08:10<334:58:20,  6.81s/it]
 21%|██▏       | 47881/225000 [101:08:15<297:38:44,  6.05s/it]
 21%|██▏       | 47882/225000 [101:08:24<338:49:56,  6.89s/it]
 21%|██▏       | 47883/225000 [101:08:28<308:05:09,  6.26s/it]
 21%|██▏       | 47884/225000 [101:08:33<285:33:17,  5.80s/it]
 21%|██▏       | 47885/225000 [101:08:37<264:43:31,  5.38s/it]
 21%|██▏       | 47886/225000 [101:08:42<255:14:46,  5.19s/it]
 21%|██▏       | 47887/225000 [101:08:55<370:59:37,  7.54s/it]
 21%|██▏       | 47888/225000 [101:09:02<360:03:46,  7.32s/it]
 21%|██▏       | 47889/225000 [101:09:09<348:11:56,  7.08s/it]
 21%|██▏       | 47890/225000 [101:09:14<331:17:21,  6.73s/it]
 21%|██▏       | 47891/225000 [101:09:20<307:54:38,  6.26s/it]
 21%|██▏       | 47892/225000 [101:09:26<302:39:26,  6.15s/it]
 21%|██▏       | 47893/225000 [101:09:31<287:36:15,  5.85s/it]
 21%|██▏       | 47894/225000 [101:09:36<278:05:24,  5.65s/it]
 21%|██▏       | 47895/225000 [101:09:42<281:58:35,  5.73s/it]
 21%|██▏       | 47896/225000 [101:09:55<393:02:25,  7.99s/it]
 21%|██▏       | 47897/225000 [101:09:59<340:53:22,  6.93s/it]
 21%|██▏       | 47898/225000 [101:10:07<351:30:34,  7.15s/it]
 21%|██▏       | 47899/225000 [101:10:12<315:01:32,  6.40s/it]
 21%|██▏       | 47900/225000 [101:10:17<299:17:09,  6.08s/it]
                                                              
{'loss': '0.7043', 'grad_norm': '0.6958', 'learning_rate': '4.627e-05', 'epoch': '0.2129', 'num_input_tokens_seen': 169873216, 'train_runtime': '3.642e+05', 'train_tokens_per_second': '466.4'}

 21%|██▏       | 47900/225000 [101:10:17<299:17:09,  6.08s/it]
 21%|██▏       | 47901/225000 [101:10:30<405:51:15,  8.25s/it]
 21%|██▏       | 47902/225000 [101:10:38<399:52:08,  8.13s/it]
 21%|██▏       | 47903/225000 [101:10:43<346:52:38,  7.05s/it]
 21%|██▏       | 47904/225000 [101:10:51<360:27:17,  7.33s/it]
 21%|██▏       | 47905/225000 [101:10:55<316:49:14,  6.44s/it]
 21%|██▏       | 47906/225000 [101:11:00<290:18:29,  5.90s/it]
 21%|██▏       | 47907/225000 [101:11:12<378:51:34,  7.70s/it]
 21%|██▏       | 47908/225000 [101:11:17<345:05:19,  7.02s/it]
 21%|██▏       | 47909/225000 [101:11:22<311:01:42,  6.32s/it]
 21%|██▏       | 47910/225000 [101:11:26<285:21:55,  5.80s/it]
 21%|██▏       | 47911/225000 [101:11:37<352:37:30,  7.17s/it]
 21%|██▏       | 47912/225000 [101:11:45<369:19:07,  7.51s/it]
 21%|██▏       | 47913/225000 [101:11:51<342:54:58,  6.97s/it]
 21%|██▏       | 47914/225000 [101:11:56<316:27:39,  6.43s/it]
 21%|██▏       | 47915/225000 [101:12:03<327:43:39,  6.66s/it]
 21%|██▏       | 47916/225000 [101:12:12<366:46:54,  7.46s/it]
 21%|██▏       | 47917/225000 [101:12:17<326:41:34,  6.64s/it]
 21%|██▏       | 47918/225000 [101:12:22<300:42:51,  6.11s/it]
 21%|██▏       | 47919/225000 [101:12:27<279:25:27,  5.68s/it]
 21%|██▏       | 47920/225000 [101:12:40<385:47:36,  7.84s/it]
 21%|██▏       | 47921/225000 [101:12:44<341:18:10,  6.94s/it]
 21%|██▏       | 47922/225000 [101:12:53<358:16:33,  7.28s/it]
 21%|██▏       | 47923/225000 [101:12:57<312:45:45,  6.36s/it]
 21%|██▏       | 47924/225000 [101:13:06<348:05:40,  7.08s/it]
 21%|██▏       | 47925/225000 [101:13:09<299:28:34,  6.09s/it]
 21%|██▏       | 47926/225000 [101:13:21<382:09:58,  7.77s/it]
 21%|██▏       | 47927/225000 [101:13:25<322:54:22,  6.56s/it]
 21%|██▏       | 47928/225000 [101:13:35<374:57:24,  7.62s/it]
 21%|██▏       | 47929/225000 [101:13:39<321:51:34,  6.54s/it]
 21%|██▏       | 47930/225000 [101:13:45<309:19:46,  6.29s/it]
 21%|██▏       | 47931/225000 [101:13:53<347:41:50,  7.07s/it]
 21%|██▏       | 47932/225000 [101:13:59<329:44:20,  6.70s/it]
 21%|██▏       | 47933/225000 [101:14:05<316:37:43,  6.44s/it]
 21%|██▏       | 47934/225000 [101:14:11<304:11:29,  6.18s/it]
 21%|██▏       | 47935/225000 [101:14:17<310:13:06,  6.31s/it]
 21%|██▏       | 47936/225000 [101:14:27<354:27:56,  7.21s/it]
 21%|██▏       | 47937/225000 [101:14:32<325:33:01,  6.62s/it]
 21%|██▏       | 47938/225000 [101:14:37<310:15:43,  6.31s/it]
 21%|██▏       | 47939/225000 [101:14:43<301:48:01,  6.14s/it]
 21%|██▏       | 47940/225000 [101:14:49<301:15:48,  6.13s/it]
 21%|██▏       | 47941/225000 [101:14:59<356:17:02,  7.24s/it]
 21%|██▏       | 47942/225000 [101:15:07<357:58:34,  7.28s/it]
 21%|██▏       | 47943/225000 [101:15:12<332:39:10,  6.76s/it]
 21%|██▏       | 47944/225000 [101:15:17<309:09:29,  6.29s/it]
 21%|██▏       | 47945/225000 [101:15:23<297:26:32,  6.05s/it]
 21%|██▏       | 47946/225000 [101:15:29<298:10:26,  6.06s/it]
 21%|██▏       | 47947/225000 [101:15:34<288:02:26,  5.86s/it]
 21%|██▏       | 47948/225000 [101:15:47<397:02:54,  8.07s/it]
 21%|██▏       | 47949/225000 [101:15:54<381:25:58,  7.76s/it]
 21%|██▏       | 47950/225000 [101:15:59<340:01:35,  6.91s/it]
                                                              
{'loss': '0.6946', 'grad_norm': '0.5989', 'learning_rate': '4.626e-05', 'epoch': '0.2131', 'num_input_tokens_seen': 170022568, 'train_runtime': '3.646e+05', 'train_tokens_per_second': '466.4'}

 21%|██▏       | 47950/225000 [101:15:59<340:01:35,  6.91s/it]
 21%|██▏       | 47951/225000 [101:16:04<304:07:30,  6.18s/it]
 21%|██▏       | 47952/225000 [101:16:15<380:43:26,  7.74s/it]
 21%|██▏       | 47953/225000 [101:16:20<337:03:15,  6.85s/it]
 21%|██▏       | 47954/225000 [101:16:25<313:46:57,  6.38s/it]
 21%|██▏       | 47955/225000 [101:16:30<294:19:21,  5.98s/it]
 21%|██▏       | 47956/225000 [101:16:35<281:20:00,  5.72s/it]
 21%|██▏       | 47957/225000 [101:16:41<274:59:58,  5.59s/it]
 21%|██▏       | 47958/225000 [101:16:52<359:44:19,  7.31s/it]
 21%|██▏       | 47959/225000 [101:17:01<375:37:54,  7.64s/it]
 21%|██▏       | 47960/225000 [101:17:04<315:42:10,  6.42s/it]
 21%|██▏       | 47961/225000 [101:17:13<351:03:29,  7.14s/it]
 21%|██▏       | 47962/225000 [101:17:20<344:11:04,  7.00s/it]
 21%|██▏       | 47963/225000 [101:17:27<349:54:06,  7.12s/it]
 21%|██▏       | 47964/225000 [101:17:31<304:23:18,  6.19s/it]
 21%|██▏       | 47965/225000 [101:17:41<359:54:55,  7.32s/it]
 21%|██▏       | 47966/225000 [101:17:48<363:10:04,  7.39s/it]
 21%|██▏       | 47967/225000 [101:17:55<346:41:19,  7.05s/it]
 21%|██▏       | 47968/225000 [101:18:00<319:15:22,  6.49s/it]
 21%|██▏       | 47969/225000 [101:18:04<287:21:40,  5.84s/it]
 21%|██▏       | 47970/225000 [101:18:09<277:43:57,  5.65s/it]
 21%|██▏       | 47971/225000 [101:18:14<260:47:50,  5.30s/it]
 21%|██▏       | 47972/225000 [101:18:25<341:27:18,  6.94s/it]
 21%|██▏       | 47973/225000 [101:18:35<384:22:23,  7.82s/it]
 21%|██▏       | 47974/225000 [101:18:39<341:03:23,  6.94s/it]
 21%|██▏       | 47975/225000 [101:18:47<355:52:10,  7.24s/it]
 21%|██▏       | 47976/225000 [101:18:52<318:31:05,  6.48s/it]
 21%|██▏       | 47977/225000 [101:18:57<293:06:54,  5.96s/it]
 21%|██▏       | 47978/225000 [101:19:10<400:00:02,  8.13s/it]
 21%|██▏       | 47979/225000 [101:19:14<344:54:49,  7.01s/it]
 21%|██▏       | 47980/225000 [101:19:23<359:45:19,  7.32s/it]
 21%|██▏       | 47981/225000 [101:19:28<330:43:05,  6.73s/it]
 21%|██▏       | 47982/225000 [101:19:33<305:00:08,  6.20s/it]
 21%|██▏       | 47983/225000 [101:19:38<284:07:07,  5.78s/it]
 21%|██▏       | 47984/225000 [101:19:47<332:38:21,  6.76s/it]
 21%|██▏       | 47985/225000 [101:19:56<369:40:16,  7.52s/it]
 21%|██▏       | 47986/225000 [101:20:02<353:01:59,  7.18s/it]
 21%|██▏       | 47987/225000 [101:20:08<335:20:58,  6.82s/it]
 21%|██▏       | 47988/225000 [101:20:17<357:09:45,  7.26s/it]
 21%|██▏       | 47989/225000 [101:20:25<371:12:02,  7.55s/it]
 21%|██▏       | 47990/225000 [101:20:29<320:46:35,  6.52s/it]
 21%|██▏       | 47991/225000 [101:20:36<321:54:27,  6.55s/it]
 21%|██▏       | 47992/225000 [101:20:44<348:37:41,  7.09s/it]
 21%|██▏       | 47993/225000 [101:20:50<327:13:19,  6.66s/it]
 21%|██▏       | 47994/225000 [101:20:56<327:45:55,  6.67s/it]
 21%|██▏       | 47995/225000 [101:21:03<324:09:57,  6.59s/it]
 21%|██▏       | 47996/225000 [101:21:11<348:46:24,  7.09s/it]
 21%|██▏       | 47997/225000 [101:21:19<360:14:37,  7.33s/it]
 21%|██▏       | 47998/225000 [101:21:22<302:10:02,  6.15s/it]
 21%|██▏       | 47999/225000 [101:21:32<354:07:36,  7.20s/it]
 21%|██▏       | 48000/225000 [101:21:37<325:16:40,  6.62s/it]
                                                              
{'loss': '0.7051', 'grad_norm': '0.7008', 'learning_rate': '4.625e-05', 'epoch': '0.2133', 'num_input_tokens_seen': 170169552, 'train_runtime': '3.649e+05', 'train_tokens_per_second': '466.3'}

 21%|██▏       | 48000/225000 [101:21:37<325:16:40,  6.62s/it]
 21%|██▏       | 48001/225000 [101:21:42<296:41:23,  6.03s/it]
 21%|██▏       | 48002/225000 [101:21:47<283:18:31,  5.76s/it]
 21%|██▏       | 48003/225000 [101:21:58<363:45:37,  7.40s/it]
 21%|██▏       | 48004/225000 [101:22:06<369:58:42,  7.53s/it]
 21%|██▏       | 48005/225000 [101:22:10<316:36:39,  6.44s/it]
 21%|██▏       | 48006/225000 [101:22:14<281:18:46,  5.72s/it]
 21%|██▏       | 48007/225000 [101:22:26<368:10:20,  7.49s/it]
 21%|██▏       | 48008/225000 [101:22:30<324:29:21,  6.60s/it]
 21%|██▏       | 48009/225000 [101:22:35<293:36:35,  5.97s/it]
 21%|██▏       | 48010/225000 [101:22:46<371:30:12,  7.56s/it]
 21%|██▏       | 48011/225000 [101:22:50<326:24:02,  6.64s/it]
 21%|██▏       | 48012/225000 [101:23:00<366:26:36,  7.45s/it]
 21%|██▏       | 48013/225000 [101:23:04<319:10:01,  6.49s/it]
 21%|██▏       | 48014/225000 [101:23:12<346:21:43,  7.05s/it]
 21%|██▏       | 48015/225000 [101:23:20<353:15:56,  7.19s/it]
 21%|██▏       | 48016/225000 [101:23:24<308:17:24,  6.27s/it]
 21%|██▏       | 48017/225000 [101:23:34<370:53:13,  7.54s/it]
 21%|██▏       | 48018/225000 [101:23:38<319:34:30,  6.50s/it]
 21%|██▏       | 48019/225000 [101:23:43<283:40:55,  5.77s/it]
 21%|██▏       | 48020/225000 [101:23:54<360:28:35,  7.33s/it]
 21%|██▏       | 48021/225000 [101:23:58<311:48:49,  6.34s/it]
 21%|██▏       | 48022/225000 [101:24:05<324:02:17,  6.59s/it]
 21%|██▏       | 48023/225000 [101:24:13<349:50:56,  7.12s/it]
 21%|██▏       | 48024/225000 [101:24:17<305:12:19,  6.21s/it]
 21%|██▏       | 48025/225000 [101:24:22<282:37:25,  5.75s/it]
 21%|██▏       | 48026/225000 [101:24:28<287:26:45,  5.85s/it]
 21%|██▏       | 48027/225000 [101:24:37<334:29:03,  6.80s/it]
 21%|██▏       | 48028/225000 [101:24:45<350:20:58,  7.13s/it]
 21%|██▏       | 48029/225000 [101:24:49<304:30:40,  6.19s/it]
 21%|██▏       | 48030/225000 [101:25:00<382:32:57,  7.78s/it]
 21%|██▏       | 48031/225000 [101:25:04<323:26:13,  6.58s/it]
 21%|██▏       | 48032/225000 [101:25:10<306:29:14,  6.23s/it]
 21%|██▏       | 48033/225000 [101:25:15<293:42:20,  5.97s/it]
 21%|██▏       | 48034/225000 [101:25:21<288:18:59,  5.87s/it]
 21%|██▏       | 48035/225000 [101:25:26<280:19:34,  5.70s/it]
 21%|██▏       | 48036/225000 [101:25:32<281:31:38,  5.73s/it]
 21%|██▏       | 48037/225000 [101:25:40<327:37:22,  6.66s/it]
 21%|██▏       | 48038/225000 [101:25:51<390:51:37,  7.95s/it]
 21%|██▏       | 48039/225000 [101:25:56<335:24:53,  6.82s/it]
 21%|██▏       | 48040/225000 [101:26:00<301:59:50,  6.14s/it]
 21%|██▏       | 48041/225000 [101:26:09<348:25:36,  7.09s/it]
 21%|██▏       | 48042/225000 [101:26:16<335:33:58,  6.83s/it]
 21%|██▏       | 48043/225000 [101:26:25<365:31:40,  7.44s/it]
 21%|██▏       | 48044/225000 [101:26:29<315:58:23,  6.43s/it]
 21%|██▏       | 48045/225000 [101:26:33<286:45:46,  5.83s/it]
 21%|██▏       | 48046/225000 [101:26:41<320:01:45,  6.51s/it]
 21%|██▏       | 48047/225000 [101:26:51<376:11:01,  7.65s/it]
 21%|██▏       | 48048/225000 [101:27:00<385:45:17,  7.85s/it]
 21%|██▏       | 48049/225000 [101:27:03<313:08:44,  6.37s/it]
 21%|██▏       | 48050/225000 [101:27:07<285:12:24,  5.80s/it]
                                                              
{'loss': '0.69', 'grad_norm': '0.7059', 'learning_rate': '4.624e-05', 'epoch': '0.2136', 'num_input_tokens_seen': 170317472, 'train_runtime': '3.652e+05', 'train_tokens_per_second': '466.3'}

 21%|██▏       | 48050/225000 [101:27:07<285:12:24,  5.80s/it]
 21%|██▏       | 48051/225000 [101:27:12<273:29:33,  5.56s/it]
 21%|██▏       | 48052/225000 [101:27:23<352:10:38,  7.17s/it]
 21%|██▏       | 48053/225000 [101:27:27<303:59:03,  6.18s/it]
 21%|██▏       | 48054/225000 [101:27:34<310:32:01,  6.32s/it]
 21%|██▏       | 48055/225000 [101:27:45<378:59:07,  7.71s/it]
 21%|██▏       | 48056/225000 [101:27:50<347:00:12,  7.06s/it]
 21%|██▏       | 48057/225000 [101:27:55<316:53:49,  6.45s/it]
 21%|██▏       | 48058/225000 [101:28:01<300:42:06,  6.12s/it]
 21%|██▏       | 48059/225000 [101:28:06<297:39:22,  6.06s/it]
 21%|██▏       | 48060/225000 [101:28:12<288:45:22,  5.88s/it]
 21%|██▏       | 48061/225000 [101:28:17<279:12:36,  5.68s/it]
 21%|██▏       | 48062/225000 [101:28:24<303:43:34,  6.18s/it]
 21%|██▏       | 48063/225000 [101:28:37<390:51:21,  7.95s/it]
 21%|██▏       | 48064/225000 [101:28:42<350:05:39,  7.12s/it]
 21%|██▏       | 48065/225000 [101:28:48<334:25:28,  6.80s/it]
 21%|██▏       | 48066/225000 [101:28:53<306:17:52,  6.23s/it]
 21%|██▏       | 48067/225000 [101:28:58<292:00:41,  5.94s/it]
 21%|██▏       | 48068/225000 [101:29:03<283:56:23,  5.78s/it]
 21%|██▏       | 48069/225000 [101:29:16<388:44:55,  7.91s/it]
 21%|██▏       | 48070/225000 [101:29:23<375:33:34,  7.64s/it]
 21%|██▏       | 48071/225000 [101:29:27<323:44:21,  6.59s/it]
 21%|██▏       | 48072/225000 [101:29:33<307:35:52,  6.26s/it]
 21%|██▏       | 48073/225000 [101:29:46<408:06:20,  8.30s/it]
 21%|██▏       | 48074/225000 [101:29:50<339:38:11,  6.91s/it]
 21%|██▏       | 48075/225000 [101:30:00<387:46:56,  7.89s/it]
 21%|██▏       | 48076/225000 [101:30:05<345:36:42,  7.03s/it]
 21%|██▏       | 48077/225000 [101:30:10<319:23:14,  6.50s/it]
 21%|██▏       | 48078/225000 [101:30:15<298:14:03,  6.07s/it]
 21%|██▏       | 48079/225000 [101:30:20<282:41:23,  5.75s/it]
 21%|██▏       | 48080/225000 [101:30:26<278:02:23,  5.66s/it]
 21%|██▏       | 48081/225000 [101:30:31<276:34:33,  5.63s/it]
 21%|██▏       | 48082/225000 [101:30:40<330:40:15,  6.73s/it]
 21%|██▏       | 48083/225000 [101:30:51<394:30:15,  8.03s/it]
 21%|██▏       | 48084/225000 [101:30:56<349:44:39,  7.12s/it]
 21%|██▏       | 48085/225000 [101:31:06<388:51:24,  7.91s/it]
 21%|██▏       | 48086/225000 [101:31:12<352:30:38,  7.17s/it]
 21%|██▏       | 48087/225000 [101:31:18<335:12:11,  6.82s/it]
 21%|██▏       | 48088/225000 [101:31:23<314:35:28,  6.40s/it]
 21%|██▏       | 48089/225000 [101:31:29<303:41:36,  6.18s/it]
 21%|██▏       | 48090/225000 [101:31:41<389:53:46,  7.93s/it]
 21%|██▏       | 48091/225000 [101:31:45<333:55:26,  6.80s/it]
 21%|██▏       | 48092/225000 [101:31:54<364:18:40,  7.41s/it]
 21%|██▏       | 48093/225000 [101:31:57<307:06:55,  6.25s/it]
 21%|██▏       | 48094/225000 [101:32:09<385:35:52,  7.85s/it]
 21%|██▏       | 48095/225000 [101:32:13<337:24:17,  6.87s/it]
 21%|██▏       | 48096/225000 [101:32:18<309:34:31,  6.30s/it]
 21%|██▏       | 48097/225000 [101:32:23<286:55:44,  5.84s/it]
 21%|██▏       | 48098/225000 [101:32:33<342:33:02,  6.97s/it]
 21%|██▏       | 48099/225000 [101:32:42<373:42:50,  7.61s/it]
 21%|██▏       | 48100/225000 [101:32:46<317:25:34,  6.46s/it]
                                                              
{'loss': '0.7094', 'grad_norm': '0.8687', 'learning_rate': '4.623e-05', 'epoch': '0.2138', 'num_input_tokens_seen': 170465048, 'train_runtime': '3.656e+05', 'train_tokens_per_second': '466.3'}

 21%|██▏       | 48100/225000 [101:32:46<317:25:34,  6.46s/it]
 21%|██▏       | 48101/225000 [101:32:57<393:23:05,  8.01s/it]
 21%|██▏       | 48102/225000 [101:33:02<338:36:05,  6.89s/it]
 21%|██▏       | 48103/225000 [101:33:10<358:39:09,  7.30s/it]
 21%|██▏       | 48104/225000 [101:33:14<313:43:28,  6.38s/it]
 21%|██▏       | 48105/225000 [101:33:19<294:09:54,  5.99s/it]
 21%|██▏       | 48106/225000 [101:33:24<272:02:22,  5.54s/it]
 21%|██▏       | 48107/225000 [101:33:35<364:03:04,  7.41s/it]
 21%|██▏       | 48108/225000 [101:33:41<329:45:18,  6.71s/it]
 21%|██▏       | 48109/225000 [101:33:45<298:52:36,  6.08s/it]
 21%|██▏       | 48110/225000 [101:33:56<364:09:29,  7.41s/it]
 21%|██▏       | 48111/225000 [101:34:04<379:03:51,  7.71s/it]
 21%|██▏       | 48112/225000 [101:34:09<336:02:27,  6.84s/it]
 21%|██▏       | 48113/225000 [101:34:15<320:01:49,  6.51s/it]
 21%|██▏       | 48114/225000 [101:34:23<348:27:47,  7.09s/it]
 21%|██▏       | 48115/225000 [101:34:31<357:36:26,  7.28s/it]
 21%|██▏       | 48116/225000 [101:34:35<311:09:52,  6.33s/it]
 21%|██▏       | 48117/225000 [101:34:45<361:24:08,  7.36s/it]
 21%|██▏       | 48118/225000 [101:34:50<330:07:37,  6.72s/it]
 21%|██▏       | 48119/225000 [101:34:56<317:06:57,  6.45s/it]
 21%|██▏       | 48120/225000 [101:35:03<328:41:53,  6.69s/it]
 21%|██▏       | 48121/225000 [101:35:09<312:38:16,  6.36s/it]
 21%|██▏       | 48122/225000 [101:35:16<328:51:20,  6.69s/it]
 21%|██▏       | 48123/225000 [101:35:24<352:09:42,  7.17s/it]
 21%|██▏       | 48124/225000 [101:35:30<333:14:38,  6.78s/it]
 21%|██▏       | 48125/225000 [101:35:37<333:06:40,  6.78s/it]
 21%|██▏       | 48126/225000 [101:35:41<291:43:33,  5.94s/it]
 21%|██▏       | 48127/225000 [101:35:49<322:21:15,  6.56s/it]
 21%|██▏       | 48128/225000 [101:35:57<339:52:25,  6.92s/it]
 21%|██▏       | 48129/225000 [101:36:01<298:42:41,  6.08s/it]
 21%|██▏       | 48130/225000 [101:36:06<285:55:52,  5.82s/it]
 21%|██▏       | 48131/225000 [101:36:11<279:34:55,  5.69s/it]
 21%|██▏       | 48132/225000 [101:36:19<310:18:10,  6.32s/it]
 21%|██▏       | 48133/225000 [101:36:32<409:10:10,  8.33s/it]
 21%|██▏       | 48134/225000 [101:36:36<340:04:53,  6.92s/it]
 21%|██▏       | 48135/225000 [101:36:40<306:00:15,  6.23s/it]
 21%|██▏       | 48136/225000 [101:36:46<294:54:29,  6.00s/it]
 21%|██▏       | 48137/225000 [101:36:57<368:51:44,  7.51s/it]
 21%|██▏       | 48138/225000 [101:37:01<322:17:58,  6.56s/it]
 21%|██▏       | 48139/225000 [101:37:06<294:31:05,  5.99s/it]
 21%|██▏       | 48140/225000 [101:37:14<317:33:11,  6.46s/it]
 21%|██▏       | 48141/225000 [101:37:21<335:42:05,  6.83s/it]
 21%|██▏       | 48142/225000 [101:37:25<292:35:51,  5.96s/it]
 21%|██▏       | 48143/225000 [101:37:37<381:27:07,  7.76s/it]
 21%|██▏       | 48144/225000 [101:37:42<340:12:54,  6.93s/it]
 21%|██▏       | 48145/225000 [101:37:50<358:13:27,  7.29s/it]
 21%|██▏       | 48146/225000 [101:37:54<306:42:46,  6.24s/it]
 21%|██▏       | 48147/225000 [101:38:05<379:26:07,  7.72s/it]
 21%|██▏       | 48148/225000 [101:38:09<318:49:12,  6.49s/it]
 21%|██▏       | 48149/225000 [101:38:17<342:49:20,  6.98s/it]
 21%|██▏       | 48150/225000 [101:38:21<300:24:33,  6.12s/it]
                                                              
{'loss': '0.7087', 'grad_norm': '0.7633', 'learning_rate': '4.622e-05', 'epoch': '0.214', 'num_input_tokens_seen': 170613200, 'train_runtime': '3.659e+05', 'train_tokens_per_second': '466.3'}

 21%|██▏       | 48150/225000 [101:38:21<300:24:33,  6.12s/it]
 21%|██▏       | 48151/225000 [101:38:25<274:30:30,  5.59s/it]
 21%|██▏       | 48152/225000 [101:38:33<298:54:52,  6.08s/it]
 21%|██▏       | 48153/225000 [101:38:42<347:03:25,  7.06s/it]
 21%|██▏       | 48154/225000 [101:38:47<313:13:02,  6.38s/it]
 21%|██▏       | 48155/225000 [101:38:52<299:52:19,  6.10s/it]
 21%|██▏       | 48156/225000 [101:38:58<289:41:25,  5.90s/it]
 21%|██▏       | 48157/225000 [101:39:04<292:25:19,  5.95s/it]
 21%|██▏       | 48158/225000 [101:39:09<284:06:26,  5.78s/it]
 21%|██▏       | 48159/225000 [101:39:14<277:42:34,  5.65s/it]
 21%|██▏       | 48160/225000 [101:39:28<387:45:40,  7.89s/it]
 21%|██▏       | 48161/225000 [101:39:34<367:41:41,  7.49s/it]
 21%|██▏       | 48162/225000 [101:39:39<321:51:47,  6.55s/it]
 21%|██▏       | 48163/225000 [101:39:46<332:11:51,  6.76s/it]
 21%|██▏       | 48164/225000 [101:39:54<353:08:41,  7.19s/it]
 21%|██▏       | 48165/225000 [101:40:02<361:54:45,  7.37s/it]
 21%|██▏       | 48166/225000 [101:40:08<345:22:59,  7.03s/it]
 21%|██▏       | 48167/225000 [101:40:14<327:00:25,  6.66s/it]
 21%|██▏       | 48168/225000 [101:40:19<298:54:51,  6.09s/it]
 21%|██▏       | 48169/225000 [101:40:24<293:38:13,  5.98s/it]
 21%|██▏       | 48170/225000 [101:40:30<289:45:43,  5.90s/it]
 21%|██▏       | 48171/225000 [101:40:36<285:42:44,  5.82s/it]
 21%|██▏       | 48172/225000 [101:40:41<284:28:01,  5.79s/it]
 21%|██▏       | 48173/225000 [101:40:47<280:07:41,  5.70s/it]
 21%|██▏       | 48174/225000 [101:40:51<265:05:39,  5.40s/it]
 21%|██▏       | 48175/225000 [101:41:04<377:04:57,  7.68s/it]
 21%|██▏       | 48176/225000 [101:41:13<395:52:56,  8.06s/it]
 21%|██▏       | 48177/225000 [101:41:17<332:56:40,  6.78s/it]
 21%|██▏       | 48178/225000 [101:41:27<375:48:31,  7.65s/it]
 21%|██▏       | 48179/225000 [101:41:32<340:16:54,  6.93s/it]
 21%|██▏       | 48180/225000 [101:41:37<306:42:54,  6.24s/it]
 21%|██▏       | 48181/225000 [101:41:41<281:39:40,  5.73s/it]
 21%|██▏       | 48182/225000 [101:41:48<290:09:25,  5.91s/it]
 21%|██▏       | 48183/225000 [101:41:58<355:36:43,  7.24s/it]
 21%|██▏       | 48184/225000 [101:42:02<304:06:10,  6.19s/it]
 21%|██▏       | 48185/225000 [101:42:12<359:24:43,  7.32s/it]
 21%|██▏       | 48186/225000 [101:42:16<317:58:50,  6.47s/it]
 21%|██▏       | 48187/225000 [101:42:21<290:44:27,  5.92s/it]
 21%|██▏       | 48188/225000 [101:42:34<390:16:12,  7.95s/it]
 21%|██▏       | 48189/225000 [101:42:37<324:34:52,  6.61s/it]
 21%|██▏       | 48190/225000 [101:42:41<289:43:52,  5.90s/it]
 21%|██▏       | 48191/225000 [101:42:50<327:34:53,  6.67s/it]
 21%|██▏       | 48192/225000 [101:42:56<329:06:27,  6.70s/it]
 21%|██▏       | 48193/225000 [101:43:05<353:06:26,  7.19s/it]
 21%|██▏       | 48194/225000 [101:43:10<326:35:16,  6.65s/it]
 21%|██▏       | 48195/225000 [101:43:17<328:12:23,  6.68s/it]
 21%|██▏       | 48196/225000 [101:43:21<285:02:35,  5.80s/it]
 21%|██▏       | 48197/225000 [101:43:27<291:50:56,  5.94s/it]
 21%|██▏       | 48198/225000 [101:43:38<364:37:54,  7.42s/it]
 21%|██▏       | 48199/225000 [101:43:42<314:47:00,  6.41s/it]
 21%|██▏       | 48200/225000 [101:43:47<298:22:22,  6.08s/it]
                                                              
{'loss': '0.687', 'grad_norm': '0.8315', 'learning_rate': '4.621e-05', 'epoch': '0.2142', 'num_input_tokens_seen': 170758744, 'train_runtime': '3.662e+05', 'train_tokens_per_second': '466.3'}

 21%|██▏       | 48200/225000 [101:43:47<298:22:22,  6.08s/it]
 21%|██▏       | 48201/225000 [101:43:53<297:19:01,  6.05s/it]
 21%|██▏       | 48202/225000 [101:44:02<343:41:19,  7.00s/it]
 21%|██▏       | 48203/225000 [101:44:08<320:01:30,  6.52s/it]
 21%|██▏       | 48204/225000 [101:44:17<353:21:10,  7.20s/it]
 21%|██▏       | 48205/225000 [101:44:22<329:31:57,  6.71s/it]
 21%|██▏       | 48206/225000 [101:44:31<367:18:05,  7.48s/it]
 21%|██▏       | 48207/225000 [101:44:39<363:42:55,  7.41s/it]
 21%|██▏       | 48208/225000 [101:44:44<326:09:22,  6.64s/it]
 21%|██▏       | 48209/225000 [101:44:48<297:57:44,  6.07s/it]
 21%|██▏       | 48210/225000 [101:44:54<296:32:53,  6.04s/it]
 21%|██▏       | 48211/225000 [101:45:05<369:05:11,  7.52s/it]
 21%|██▏       | 48212/225000 [101:45:10<329:07:23,  6.70s/it]
 21%|██▏       | 48213/225000 [101:45:16<314:40:04,  6.41s/it]
 21%|██▏       | 48214/225000 [101:45:21<299:47:01,  6.10s/it]
 21%|██▏       | 48215/225000 [101:45:26<280:44:15,  5.72s/it]
 21%|██▏       | 48216/225000 [101:45:32<280:58:54,  5.72s/it]
 21%|██▏       | 48217/225000 [101:45:40<323:34:21,  6.59s/it]
 21%|██▏       | 48218/225000 [101:45:52<398:22:25,  8.11s/it]
 21%|██▏       | 48219/225000 [101:45:55<331:09:46,  6.74s/it]
 21%|██▏       | 48220/225000 [101:46:03<348:33:16,  7.10s/it]
 21%|██▏       | 48221/225000 [101:46:09<325:03:41,  6.62s/it]
 21%|██▏       | 48222/225000 [101:46:14<309:49:03,  6.31s/it]
 21%|██▏       | 48223/225000 [101:46:20<298:44:20,  6.08s/it]
 21%|██▏       | 48224/225000 [101:46:25<287:46:52,  5.86s/it]
 21%|██▏       | 48225/225000 [101:46:31<289:43:26,  5.90s/it]
 21%|██▏       | 48226/225000 [101:46:37<286:36:07,  5.84s/it]
 21%|██▏       | 48227/225000 [101:46:44<304:46:08,  6.21s/it]
 21%|██▏       | 48228/225000 [101:46:51<321:32:26,  6.55s/it]
 21%|██▏       | 48229/225000 [101:47:02<375:13:08,  7.64s/it]
 21%|██▏       | 48230/225000 [101:47:06<330:15:24,  6.73s/it]
 21%|██▏       | 48231/225000 [101:47:11<293:34:42,  5.98s/it]
 21%|██▏       | 48232/225000 [101:47:24<402:15:02,  8.19s/it]
 21%|██▏       | 48233/225000 [101:47:29<358:01:59,  7.29s/it]
 21%|██▏       | 48234/225000 [101:47:36<347:52:34,  7.08s/it]
 21%|██▏       | 48235/225000 [101:47:43<350:22:08,  7.14s/it]
 21%|██▏       | 48236/225000 [101:47:47<306:16:29,  6.24s/it]
 21%|██▏       | 48237/225000 [101:47:56<348:16:41,  7.09s/it]
 21%|██▏       | 48238/225000 [101:48:01<321:25:06,  6.55s/it]
 21%|██▏       | 48239/225000 [101:48:11<365:59:27,  7.45s/it]
 21%|██▏       | 48240/225000 [101:48:17<339:05:43,  6.91s/it]
 21%|██▏       | 48241/225000 [101:48:23<331:18:34,  6.75s/it]
 21%|██▏       | 48242/225000 [101:48:28<311:45:21,  6.35s/it]
 21%|██▏       | 48243/225000 [101:48:34<295:10:10,  6.01s/it]
 21%|██▏       | 48244/225000 [101:48:39<285:37:40,  5.82s/it]
 21%|██▏       | 48245/225000 [101:48:44<276:39:43,  5.63s/it]
 21%|██▏       | 48246/225000 [101:48:57<382:20:32,  7.79s/it]
 21%|██▏       | 48247/225000 [101:49:05<384:01:36,  7.82s/it]
 21%|██▏       | 48248/225000 [101:49:13<381:45:19,  7.78s/it]
 21%|██▏       | 48249/225000 [101:49:17<325:31:08,  6.63s/it]
 21%|██▏       | 48250/225000 [101:49:27<388:14:44,  7.91s/it]
                                                              
{'loss': '0.6848', 'grad_norm': '0.7687', 'learning_rate': '4.62e-05', 'epoch': '0.2144', 'num_input_tokens_seen': 170905904, 'train_runtime': '3.666e+05', 'train_tokens_per_second': '466.2'}

 21%|██▏       | 48250/225000 [101:49:27<388:14:44,  7.91s/it]
 21%|██▏       | 48251/225000 [101:49:31<320:41:37,  6.53s/it]
 21%|██▏       | 48252/225000 [101:49:39<345:50:54,  7.04s/it]
 21%|██▏       | 48253/225000 [101:49:44<320:27:46,  6.53s/it]
 21%|██▏       | 48254/225000 [101:49:49<292:27:32,  5.96s/it]
 21%|██▏       | 48255/225000 [101:49:54<283:45:24,  5.78s/it]
 21%|██▏       | 48256/225000 [101:50:02<313:01:56,  6.38s/it]
 21%|██▏       | 48257/225000 [101:50:11<356:35:54,  7.26s/it]
 21%|██▏       | 48258/225000 [101:50:17<328:38:27,  6.69s/it]
 21%|██▏       | 48259/225000 [101:50:22<307:42:46,  6.27s/it]
 21%|██▏       | 48260/225000 [101:50:27<291:37:16,  5.94s/it]
 21%|██▏       | 48261/225000 [101:50:33<285:01:49,  5.81s/it]
 21%|██▏       | 48262/225000 [101:50:46<395:18:17,  8.05s/it]
 21%|██▏       | 48263/225000 [101:50:51<352:29:44,  7.18s/it]
 21%|██▏       | 48264/225000 [101:50:55<305:55:06,  6.23s/it]
 21%|██▏       | 48265/225000 [101:51:07<385:36:15,  7.85s/it]
 21%|██▏       | 48266/225000 [101:51:10<322:04:04,  6.56s/it]
 21%|██▏       | 48267/225000 [101:51:18<338:11:55,  6.89s/it]
 21%|██▏       | 48268/225000 [101:51:22<295:31:53,  6.02s/it]
 21%|██▏       | 48269/225000 [101:51:27<282:08:41,  5.75s/it]
 21%|██▏       | 48270/225000 [101:51:36<325:29:38,  6.63s/it]
 21%|██▏       | 48271/225000 [101:51:40<297:04:49,  6.05s/it]
 21%|██▏       | 48272/225000 [101:51:45<278:07:00,  5.67s/it]
 21%|██▏       | 48273/225000 [101:51:50<269:24:11,  5.49s/it]
 21%|██▏       | 48274/225000 [101:51:59<311:31:10,  6.35s/it]
 21%|██▏       | 48275/225000 [101:52:12<411:41:48,  8.39s/it]
 21%|██▏       | 48276/225000 [101:52:15<337:15:43,  6.87s/it]
 21%|██▏       | 48277/225000 [101:52:20<314:12:12,  6.40s/it]
 21%|██▏       | 48278/225000 [101:52:26<307:00:11,  6.25s/it]
 21%|██▏       | 48279/225000 [101:52:32<296:19:51,  6.04s/it]
 21%|██▏       | 48280/225000 [101:52:37<288:22:03,  5.87s/it]
 21%|██▏       | 48281/225000 [101:52:42<271:38:04,  5.53s/it]
 21%|██▏       | 48282/225000 [101:52:52<340:56:47,  6.95s/it]
 21%|██▏       | 48283/225000 [101:53:02<385:03:22,  7.84s/it]
 21%|██▏       | 48284/225000 [101:53:09<364:04:28,  7.42s/it]
 21%|██▏       | 48285/225000 [101:53:16<356:37:47,  7.27s/it]
 21%|██▏       | 48286/225000 [101:53:22<341:23:18,  6.95s/it]
 21%|██▏       | 48287/225000 [101:53:28<322:15:05,  6.56s/it]
 21%|██▏       | 48288/225000 [101:53:32<295:40:53,  6.02s/it]
 21%|██▏       | 48289/225000 [101:53:37<275:45:49,  5.62s/it]
 21%|██▏       | 48290/225000 [101:53:42<267:06:49,  5.44s/it]
 21%|██▏       | 48291/225000 [101:53:52<336:26:43,  6.85s/it]
 21%|██▏       | 48292/225000 [101:54:02<382:50:24,  7.80s/it]
 21%|██▏       | 48293/225000 [101:54:06<326:35:13,  6.65s/it]
 21%|██▏       | 48294/225000 [101:54:16<368:06:07,  7.50s/it]
 21%|██▏       | 48295/225000 [101:54:21<343:12:47,  6.99s/it]
 21%|██▏       | 48296/225000 [101:54:26<303:50:40,  6.19s/it]
 21%|██▏       | 48297/225000 [101:54:31<292:44:04,  5.96s/it]
 21%|██▏       | 48298/225000 [101:54:43<380:30:57,  7.75s/it]
 21%|██▏       | 48299/225000 [101:54:49<357:49:52,  7.29s/it]
 21%|██▏       | 48300/225000 [101:54:54<317:25:27,  6.47s/it]
                                                              
{'loss': '0.699', 'grad_norm': '0.7379', 'learning_rate': '4.619e-05', 'epoch': '0.2147', 'num_input_tokens_seen': 171052776, 'train_runtime': '3.669e+05', 'train_tokens_per_second': '466.2'}

 21%|██▏       | 48300/225000 [101:54:54<317:25:27,  6.47s/it]
 21%|██▏       | 48301/225000 [101:55:01<334:13:11,  6.81s/it]
 21%|██▏       | 48302/225000 [101:55:06<306:14:23,  6.24s/it]
 21%|██▏       | 48303/225000 [101:55:11<283:14:59,  5.77s/it]
 21%|██▏       | 48304/225000 [101:55:19<313:18:26,  6.38s/it]
 21%|██▏       | 48305/225000 [101:55:28<358:05:26,  7.30s/it]
 21%|██▏       | 48306/225000 [101:55:34<329:09:58,  6.71s/it]
 21%|██▏       | 48307/225000 [101:55:38<300:21:13,  6.12s/it]
 21%|██▏       | 48308/225000 [101:55:44<287:03:03,  5.85s/it]
 21%|██▏       | 48309/225000 [101:55:48<272:46:39,  5.56s/it]
 21%|██▏       | 48310/225000 [101:55:56<308:24:33,  6.28s/it]
 21%|██▏       | 48311/225000 [101:56:07<374:03:42,  7.62s/it]
 21%|██▏       | 48312/225000 [101:56:13<354:37:26,  7.23s/it]
 21%|██▏       | 48313/225000 [101:56:18<319:25:28,  6.51s/it]
 21%|██▏       | 48314/225000 [101:56:23<293:26:07,  5.98s/it]
 21%|██▏       | 48315/225000 [101:56:31<315:20:19,  6.43s/it]
 21%|██▏       | 48316/225000 [101:56:41<379:21:32,  7.73s/it]
 21%|██▏       | 48317/225000 [101:56:45<320:59:50,  6.54s/it]
 21%|██▏       | 48318/225000 [101:56:51<307:22:56,  6.26s/it]
 21%|██▏       | 48319/225000 [101:56:56<292:14:14,  5.95s/it]
 21%|██▏       | 48320/225000 [101:57:01<281:37:51,  5.74s/it]
 21%|██▏       | 48321/225000 [101:57:06<265:54:12,  5.42s/it]
 21%|██▏       | 48322/225000 [101:57:12<274:02:53,  5.58s/it]
 21%|██▏       | 48323/225000 [101:57:25<383:36:08,  7.82s/it]
 21%|██▏       | 48324/225000 [101:57:31<358:51:04,  7.31s/it]
 21%|██▏       | 48325/225000 [101:57:38<359:11:52,  7.32s/it]
 21%|██▏       | 48326/225000 [101:57:43<314:04:31,  6.40s/it]
 21%|██▏       | 48327/225000 [101:57:47<280:43:31,  5.72s/it]
 21%|██▏       | 48328/225000 [101:58:00<386:38:18,  7.88s/it]
 21%|██▏       | 48329/225000 [101:58:06<366:03:11,  7.46s/it]
 21%|██▏       | 48330/225000 [101:58:11<326:50:09,  6.66s/it]
 21%|██▏       | 48331/225000 [101:58:16<305:28:40,  6.22s/it]
 21%|██▏       | 48332/225000 [101:58:23<315:12:01,  6.42s/it]
 21%|██▏       | 48333/225000 [101:58:35<395:14:01,  8.05s/it]
 21%|██▏       | 48334/225000 [101:58:39<331:19:09,  6.75s/it]
 21%|██▏       | 48335/225000 [101:58:46<343:24:01,  7.00s/it]
 21%|██▏       | 48336/225000 [101:58:52<323:32:40,  6.59s/it]
 21%|██▏       | 48337/225000 [101:59:01<368:40:02,  7.51s/it]
 21%|██▏       | 48338/225000 [101:59:06<321:29:12,  6.55s/it]
 21%|██▏       | 48339/225000 [101:59:10<286:02:43,  5.83s/it]
 21%|██▏       | 48340/225000 [101:59:17<310:54:43,  6.34s/it]
 21%|██▏       | 48341/225000 [101:59:25<335:26:55,  6.84s/it]
 21%|██▏       | 48342/225000 [101:59:30<299:57:03,  6.11s/it]
 21%|██▏       | 48343/225000 [101:59:37<312:02:42,  6.36s/it]
 21%|██▏       | 48344/225000 [101:59:46<353:35:04,  7.21s/it]
 21%|██▏       | 48345/225000 [101:59:50<305:44:05,  6.23s/it]
 21%|██▏       | 48346/225000 [101:59:56<296:51:27,  6.05s/it]
 21%|██▏       | 48347/225000 [102:00:00<280:05:16,  5.71s/it]
 21%|██▏       | 48348/225000 [102:00:11<356:02:26,  7.26s/it]
 21%|██▏       | 48349/225000 [102:00:19<361:14:03,  7.36s/it]
 21%|██▏       | 48350/225000 [102:00:24<327:18:00,  6.67s/it]
                                                              
{'loss': '0.7059', 'grad_norm': '0.8087', 'learning_rate': '4.618e-05', 'epoch': '0.2149', 'num_input_tokens_seen': 171199048, 'train_runtime': '3.672e+05', 'train_tokens_per_second': '466.2'}

 21%|██▏       | 48350/225000 [102:00:24<327:18:00,  6.67s/it]
 21%|██▏       | 48351/225000 [102:00:29<297:54:26,  6.07s/it]
 21%|██▏       | 48352/225000 [102:00:34<284:09:49,  5.79s/it]
 21%|██▏       | 48353/225000 [102:00:42<317:26:57,  6.47s/it]
 21%|██▏       | 48354/225000 [102:00:54<394:42:23,  8.04s/it]
 21%|██▏       | 48355/225000 [102:00:58<342:32:16,  6.98s/it]
 21%|██▏       | 48356/225000 [102:01:03<319:35:29,  6.51s/it]
 21%|██▏       | 48357/225000 [102:01:09<306:48:13,  6.25s/it]
 21%|██▏       | 48358/225000 [102:01:14<291:18:11,  5.94s/it]
 21%|██▏       | 48359/225000 [102:01:20<288:00:44,  5.87s/it]
 21%|██▏       | 48360/225000 [102:01:25<277:02:43,  5.65s/it]
 21%|██▏       | 48361/225000 [102:01:31<279:26:29,  5.70s/it]
 21%|██▏       | 48362/225000 [102:01:39<319:28:13,  6.51s/it]
 21%|██▏       | 48363/225000 [102:01:53<419:58:57,  8.56s/it]
 21%|██▏       | 48364/225000 [102:01:56<344:37:20,  7.02s/it]
 21%|██▏       | 48365/225000 [102:02:01<313:30:35,  6.39s/it]
 21%|██▏       | 48366/225000 [102:02:07<304:00:56,  6.20s/it]
 21%|██▏       | 48367/225000 [102:02:12<289:43:50,  5.91s/it]
 21%|██▏       | 48368/225000 [102:02:18<288:08:44,  5.87s/it]
 21%|██▏       | 48369/225000 [102:02:23<281:50:32,  5.74s/it]
 21%|██▏       | 48370/225000 [102:02:29<278:38:08,  5.68s/it]
 21%|██▏       | 48371/225000 [102:02:39<345:17:16,  7.04s/it]
 21%|██▏       | 48372/225000 [102:02:51<410:56:24,  8.38s/it]
 21%|██▏       | 48373/225000 [102:02:55<360:03:15,  7.34s/it]
 21%|██▏       | 48374/225000 [102:03:00<323:15:10,  6.59s/it]
 22%|██▏       | 48375/225000 [102:03:05<300:01:52,  6.12s/it]
 22%|██▏       | 48376/225000 [102:03:11<288:10:49,  5.87s/it]
 22%|██▏       | 48377/225000 [102:03:22<370:34:12,  7.55s/it]
 22%|██▏       | 48378/225000 [102:03:31<385:50:26,  7.86s/it]
 22%|██▏       | 48379/225000 [102:03:35<339:47:41,  6.93s/it]
 22%|██▏       | 48380/225000 [102:03:40<300:03:02,  6.12s/it]
 22%|██▏       | 48381/225000 [102:03:44<279:02:40,  5.69s/it]
 22%|██▏       | 48382/225000 [102:03:50<275:47:17,  5.62s/it]
 22%|██▏       | 48383/225000 [102:03:59<330:03:02,  6.73s/it]
 22%|██▏       | 48384/225000 [102:04:09<371:33:16,  7.57s/it]
 22%|██▏       | 48385/225000 [102:04:13<319:25:15,  6.51s/it]
 22%|██▏       | 48386/225000 [102:04:23<375:11:37,  7.65s/it]
 22%|██▏       | 48387/225000 [102:04:27<316:23:45,  6.45s/it]
 22%|██▏       | 48388/225000 [102:04:36<359:36:32,  7.33s/it]
 22%|██▏       | 48389/225000 [102:04:40<307:44:17,  6.27s/it]
 22%|██▏       | 48390/225000 [102:04:47<322:21:50,  6.57s/it]
 22%|██▏       | 48391/225000 [102:04:51<290:52:17,  5.93s/it]
 22%|██▏       | 48392/225000 [102:05:02<356:56:35,  7.28s/it]
 22%|██▏       | 48393/225000 [102:05:08<334:55:28,  6.83s/it]
 22%|██▏       | 48394/225000 [102:05:15<346:08:41,  7.06s/it]
 22%|██▏       | 48395/225000 [102:05:22<344:02:21,  7.01s/it]
 22%|██▏       | 48396/225000 [102:05:27<318:50:41,  6.50s/it]
 22%|██▏       | 48397/225000 [102:05:31<281:45:46,  5.74s/it]
 22%|██▏       | 48398/225000 [102:05:41<333:31:35,  6.80s/it]
 22%|██▏       | 48399/225000 [102:05:49<348:03:44,  7.10s/it]
 22%|██▏       | 48400/225000 [102:05:54<321:44:16,  6.56s/it]
                                                              
{'loss': '0.722', 'grad_norm': '0.833', 'learning_rate': '4.617e-05', 'epoch': '0.2151', 'num_input_tokens_seen': 171346328, 'train_runtime': '3.676e+05', 'train_tokens_per_second': '466.2'}

 22%|██▏       | 48400/225000 [102:05:54<321:44:16,  6.56s/it]
 22%|██▏       | 48401/225000 [102:06:01<337:55:04,  6.89s/it]
 22%|██▏       | 48402/225000 [102:06:09<340:44:14,  6.95s/it]
 22%|██▏       | 48403/225000 [102:06:13<309:38:35,  6.31s/it]
 22%|██▏       | 48404/225000 [102:06:18<283:17:29,  5.78s/it]
 22%|██▏       | 48405/225000 [102:06:23<276:03:16,  5.63s/it]
 22%|██▏       | 48406/225000 [102:06:29<275:20:57,  5.61s/it]
 22%|██▏       | 48407/225000 [102:06:34<267:06:35,  5.45s/it]
 22%|██▏       | 48408/225000 [102:06:47<380:03:08,  7.75s/it]
 22%|██▏       | 48409/225000 [102:06:52<345:13:51,  7.04s/it]
 22%|██▏       | 48410/225000 [102:07:00<357:09:31,  7.28s/it]
 22%|██▏       | 48411/225000 [102:07:07<353:33:29,  7.21s/it]
 22%|██▏       | 48412/225000 [102:07:12<322:38:29,  6.58s/it]
 22%|██▏       | 48413/225000 [102:07:18<302:43:06,  6.17s/it]
 22%|██▏       | 48414/225000 [102:07:23<287:06:03,  5.85s/it]
 22%|██▏       | 48415/225000 [102:07:29<291:49:52,  5.95s/it]
 22%|██▏       | 48416/225000 [102:07:43<405:36:52,  8.27s/it]
 22%|██▏       | 48417/225000 [102:07:46<334:23:12,  6.82s/it]
 22%|██▏       | 48418/225000 [102:07:51<310:17:16,  6.33s/it]
 22%|██▏       | 48419/225000 [102:07:56<293:41:08,  5.99s/it]
 22%|██▏       | 48420/225000 [102:08:02<286:46:36,  5.85s/it]
 22%|██▏       | 48421/225000 [102:08:08<284:41:09,  5.80s/it]
 22%|██▏       | 48422/225000 [102:08:12<271:09:38,  5.53s/it]
 22%|██▏       | 48423/225000 [102:08:26<383:56:56,  7.83s/it]
 22%|██▏       | 48424/225000 [102:08:33<379:27:35,  7.74s/it]
 22%|██▏       | 48425/225000 [102:08:40<373:10:31,  7.61s/it]
 22%|██▏       | 48426/225000 [102:08:46<338:12:40,  6.90s/it]
 22%|██▏       | 48427/225000 [102:08:50<298:35:57,  6.09s/it]
 22%|██▏       | 48428/225000 [102:09:03<406:01:40,  8.28s/it]
 22%|██▏       | 48429/225000 [102:09:07<337:01:14,  6.87s/it]
 22%|██▏       | 48430/225000 [102:09:11<299:27:28,  6.11s/it]
 22%|██▏       | 48431/225000 [102:09:21<356:08:30,  7.26s/it]
 22%|██▏       | 48432/225000 [102:09:26<315:09:21,  6.43s/it]
 22%|██▏       | 48433/225000 [102:09:30<286:25:27,  5.84s/it]
 22%|██▏       | 48434/225000 [102:09:37<300:17:16,  6.12s/it]
 22%|██▏       | 48435/225000 [102:09:48<375:11:10,  7.65s/it]
 22%|██▏       | 48436/225000 [102:09:52<323:22:17,  6.59s/it]
 22%|██▏       | 48437/225000 [102:09:59<329:59:23,  6.73s/it]
 22%|██▏       | 48438/225000 [102:10:05<310:05:31,  6.32s/it]
 22%|██▏       | 48439/225000 [102:10:12<332:27:36,  6.78s/it]
 22%|██▏       | 48440/225000 [102:10:21<361:31:55,  7.37s/it]
 22%|██▏       | 48441/225000 [102:10:26<321:19:06,  6.55s/it]
 22%|██▏       | 48442/225000 [102:10:35<358:01:55,  7.30s/it]
 22%|██▏       | 48443/225000 [102:10:41<346:35:31,  7.07s/it]
 22%|██▏       | 48444/225000 [102:10:46<311:55:38,  6.36s/it]
 22%|██▏       | 48445/225000 [102:10:55<352:08:06,  7.18s/it]
 22%|██▏       | 48446/225000 [102:10:59<300:19:09,  6.12s/it]
 22%|██▏       | 48447/225000 [102:11:08<341:29:07,  6.96s/it]
 22%|██▏       | 48448/225000 [102:11:15<346:28:16,  7.06s/it]
 22%|██▏       | 48449/225000 [102:11:21<335:18:50,  6.84s/it]
 22%|██▏       | 48450/225000 [102:11:26<308:46:21,  6.30s/it]
                                                              
{'loss': '0.7174', 'grad_norm': '0.8028', 'learning_rate': '4.616e-05', 'epoch': '0.2153', 'num_input_tokens_seen': 171495584, 'train_runtime': '3.679e+05', 'train_tokens_per_second': '466.2'}

 22%|██▏       | 48450/225000 [102:11:26<308:46:21,  6.30s/it]
 22%|██▏       | 48451/225000 [102:11:32<297:39:04,  6.07s/it]
 22%|██▏       | 48452/225000 [102:11:37<283:36:29,  5.78s/it]
 22%|██▏       | 48453/225000 [102:11:42<270:43:50,  5.52s/it]
 22%|██▏       | 48454/225000 [102:11:47<267:37:58,  5.46s/it]
 22%|██▏       | 48455/225000 [102:12:01<385:06:20,  7.85s/it]
 22%|██▏       | 48456/225000 [102:12:06<340:51:25,  6.95s/it]
 22%|██▏       | 48457/225000 [102:12:13<347:18:49,  7.08s/it]
 22%|██▏       | 48458/225000 [102:12:17<299:14:57,  6.10s/it]
 22%|██▏       | 48459/225000 [102:12:25<330:45:56,  6.74s/it]
 22%|██▏       | 48460/225000 [102:12:29<291:00:39,  5.93s/it]
 22%|██▏       | 48461/225000 [102:12:40<359:00:37,  7.32s/it]
 22%|██▏       | 48462/225000 [102:12:45<326:51:21,  6.67s/it]
 22%|██▏       | 48463/225000 [102:12:48<280:16:54,  5.72s/it]
 22%|██▏       | 48464/225000 [102:12:58<334:25:58,  6.82s/it]
 22%|██▏       | 48465/225000 [102:13:06<353:15:01,  7.20s/it]
 22%|██▏       | 48466/225000 [102:13:13<348:00:51,  7.10s/it]
 22%|██▏       | 48467/225000 [102:13:17<312:06:52,  6.36s/it]
 22%|██▏       | 48468/225000 [102:13:22<293:55:04,  5.99s/it]
 22%|██▏       | 48469/225000 [102:13:28<280:24:18,  5.72s/it]
 22%|██▏       | 48470/225000 [102:13:32<269:00:01,  5.49s/it]
 22%|██▏       | 48471/225000 [102:13:45<367:45:16,  7.50s/it]
 22%|██▏       | 48472/225000 [102:13:51<347:26:57,  7.09s/it]
 22%|██▏       | 48473/225000 [102:13:59<358:45:56,  7.32s/it]
 22%|██▏       | 48474/225000 [102:14:04<323:10:10,  6.59s/it]
 22%|██▏       | 48475/225000 [102:14:09<300:28:10,  6.13s/it]
 22%|██▏       | 48476/225000 [102:14:13<281:09:15,  5.73s/it]
 22%|██▏       | 48477/225000 [102:14:19<284:32:46,  5.80s/it]
 22%|██▏       | 48478/225000 [102:14:31<371:40:17,  7.58s/it]
 22%|██▏       | 48479/225000 [102:14:36<326:50:03,  6.67s/it]
 22%|██▏       | 48480/225000 [102:14:43<343:25:52,  7.00s/it]
 22%|██▏       | 48481/225000 [102:14:51<351:47:45,  7.17s/it]
 22%|██▏       | 48482/225000 [102:14:58<348:40:37,  7.11s/it]
 22%|██▏       | 48483/225000 [102:15:02<297:27:05,  6.07s/it]
 22%|██▏       | 48484/225000 [102:15:06<270:08:40,  5.51s/it]
 22%|██▏       | 48485/225000 [102:15:16<343:17:48,  7.00s/it]
 22%|██▏       | 48486/225000 [102:15:20<293:38:38,  5.99s/it]
 22%|██▏       | 48487/225000 [102:15:28<330:34:35,  6.74s/it]
 22%|██▏       | 48488/225000 [102:15:32<289:08:48,  5.90s/it]
 22%|██▏       | 48489/225000 [102:15:42<347:05:46,  7.08s/it]
 22%|██▏       | 48490/225000 [102:15:48<333:53:54,  6.81s/it]
 22%|██▏       | 48491/225000 [102:15:55<326:29:12,  6.66s/it]
 22%|██▏       | 48492/225000 [102:16:03<346:39:03,  7.07s/it]
 22%|██▏       | 48493/225000 [102:16:07<311:16:46,  6.35s/it]
 22%|██▏       | 48494/225000 [102:16:13<299:49:38,  6.12s/it]
 22%|██▏       | 48495/225000 [102:16:18<283:23:22,  5.78s/it]
 22%|██▏       | 48496/225000 [102:16:30<383:04:45,  7.81s/it]
 22%|██▏       | 48497/225000 [102:16:34<327:10:16,  6.67s/it]
 22%|██▏       | 48498/225000 [102:16:43<358:05:24,  7.30s/it]
 22%|██▏       | 48499/225000 [102:16:49<329:56:10,  6.73s/it]
 22%|██▏       | 48500/225000 [102:16:54<308:30:29,  6.29s/it]
                                                              
{'loss': '0.6889', 'grad_norm': '0.7706', 'learning_rate': '4.615e-05', 'epoch': '0.2156', 'num_input_tokens_seen': 171641192, 'train_runtime': '3.682e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48500/225000 [102:16:54<308:30:29,  6.29s/it]
 22%|██▏       | 48501/225000 [102:17:00<298:45:43,  6.09s/it]
 22%|██▏       | 48502/225000 [102:17:05<286:49:52,  5.85s/it]
 22%|██▏       | 48503/225000 [102:17:10<273:10:22,  5.57s/it]
 22%|██▏       | 48504/225000 [102:17:15<268:53:00,  5.48s/it]
 22%|██▏       | 48505/225000 [102:17:28<375:54:26,  7.67s/it]
 22%|██▏       | 48506/225000 [102:17:34<358:58:46,  7.32s/it]
 22%|██▏       | 48507/225000 [102:17:38<310:01:09,  6.32s/it]
 22%|██▏       | 48508/225000 [102:17:43<279:14:44,  5.70s/it]
 22%|██▏       | 48509/225000 [102:17:50<308:16:48,  6.29s/it]
 22%|██▏       | 48510/225000 [102:17:59<349:45:58,  7.13s/it]
 22%|██▏       | 48511/225000 [102:18:04<319:52:30,  6.52s/it]
 22%|██▏       | 48512/225000 [102:18:10<300:45:09,  6.13s/it]
 22%|██▏       | 48513/225000 [102:18:15<288:25:37,  5.88s/it]
 22%|██▏       | 48514/225000 [102:18:20<282:25:52,  5.76s/it]
 22%|██▏       | 48515/225000 [102:18:26<281:51:23,  5.75s/it]
 22%|██▏       | 48516/225000 [102:18:31<274:07:30,  5.59s/it]
 22%|██▏       | 48517/225000 [102:18:45<387:54:57,  7.91s/it]
 22%|██▏       | 48518/225000 [102:18:51<363:25:37,  7.41s/it]
 22%|██▏       | 48519/225000 [102:18:56<321:26:59,  6.56s/it]
 22%|██▏       | 48520/225000 [102:19:00<297:10:35,  6.06s/it]
 22%|██▏       | 48521/225000 [102:19:05<279:15:18,  5.70s/it]
 22%|██▏       | 48522/225000 [102:19:17<368:04:49,  7.51s/it]
 22%|██▏       | 48523/225000 [102:19:22<328:23:21,  6.70s/it]
 22%|██▏       | 48524/225000 [102:19:26<289:16:14,  5.90s/it]
 22%|██▏       | 48525/225000 [102:19:38<381:20:27,  7.78s/it]
 22%|██▏       | 48526/225000 [102:19:43<346:04:46,  7.06s/it]
 22%|██▏       | 48527/225000 [102:19:48<315:46:19,  6.44s/it]
 22%|██▏       | 48528/225000 [102:19:53<291:06:51,  5.94s/it]
 22%|██▏       | 48529/225000 [102:19:58<273:03:53,  5.57s/it]
 22%|██▏       | 48530/225000 [102:20:06<304:14:37,  6.21s/it]
 22%|██▏       | 48531/225000 [102:20:16<365:43:43,  7.46s/it]
 22%|██▏       | 48532/225000 [102:20:20<309:24:26,  6.31s/it]
 22%|██▏       | 48533/225000 [102:20:28<335:43:18,  6.85s/it]
 22%|██▏       | 48534/225000 [102:20:34<332:44:16,  6.79s/it]
 22%|██▏       | 48535/225000 [102:20:40<323:13:59,  6.59s/it]
 22%|██▏       | 48536/225000 [102:20:46<302:28:11,  6.17s/it]
 22%|██▏       | 48537/225000 [102:20:50<281:16:15,  5.74s/it]
 22%|██▏       | 48538/225000 [102:20:57<294:48:44,  6.01s/it]
 22%|██▏       | 48539/225000 [102:21:07<352:42:09,  7.20s/it]
 22%|██▏       | 48540/225000 [102:21:14<350:44:03,  7.16s/it]
 22%|██▏       | 48541/225000 [102:21:20<331:18:57,  6.76s/it]
 22%|██▏       | 48542/225000 [102:21:25<302:12:07,  6.17s/it]
 22%|██▏       | 48543/225000 [102:21:28<267:53:17,  5.47s/it]
 22%|██▏       | 48544/225000 [102:21:40<353:22:25,  7.21s/it]
 22%|██▏       | 48545/225000 [102:21:46<335:01:26,  6.84s/it]
 22%|██▏       | 48546/225000 [102:21:51<306:39:32,  6.26s/it]
 22%|██▏       | 48547/225000 [102:22:01<362:51:39,  7.40s/it]
 22%|██▏       | 48548/225000 [102:22:09<373:22:36,  7.62s/it]
 22%|██▏       | 48549/225000 [102:22:12<314:25:29,  6.41s/it]
 22%|██▏       | 48550/225000 [102:22:21<352:13:03,  7.19s/it]
                                                              
{'loss': '0.7023', 'grad_norm': '0.7962', 'learning_rate': '4.614e-05', 'epoch': '0.2158', 'num_input_tokens_seen': 171788112, 'train_runtime': '3.685e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48550/225000 [102:22:21<352:13:03,  7.19s/it]
 22%|██▏       | 48551/225000 [102:22:25<300:52:30,  6.14s/it]
 22%|██▏       | 48552/225000 [102:22:32<315:14:08,  6.43s/it]
 22%|██▏       | 48553/225000 [102:22:38<307:25:30,  6.27s/it]
 22%|██▏       | 48554/225000 [102:22:48<354:26:02,  7.23s/it]
 22%|██▏       | 48555/225000 [102:22:52<312:53:26,  6.38s/it]
 22%|██▏       | 48556/225000 [102:22:59<321:24:29,  6.56s/it]
 22%|██▏       | 48557/225000 [102:23:07<345:37:56,  7.05s/it]
 22%|██▏       | 48558/225000 [102:23:12<312:18:15,  6.37s/it]
 22%|██▏       | 48559/225000 [102:23:20<334:12:20,  6.82s/it]
 22%|██▏       | 48560/225000 [102:23:28<350:03:41,  7.14s/it]
 22%|██▏       | 48561/225000 [102:23:33<319:33:54,  6.52s/it]
 22%|██▏       | 48562/225000 [102:23:38<294:01:18,  6.00s/it]
 22%|██▏       | 48563/225000 [102:23:43<283:37:52,  5.79s/it]
 22%|██▏       | 48564/225000 [102:23:47<263:27:11,  5.38s/it]
 22%|██▏       | 48565/225000 [102:23:56<313:03:35,  6.39s/it]
 22%|██▏       | 48566/225000 [102:24:06<372:12:24,  7.59s/it]
 22%|██▏       | 48567/225000 [102:24:10<319:09:16,  6.51s/it]
 22%|██▏       | 48568/225000 [102:24:20<368:08:40,  7.51s/it]
 22%|██▏       | 48569/225000 [102:24:25<323:22:56,  6.60s/it]
 22%|██▏       | 48570/225000 [102:24:29<288:52:19,  5.89s/it]
 22%|██▏       | 48571/225000 [102:24:39<343:24:35,  7.01s/it]
 22%|██▏       | 48572/225000 [102:24:46<345:10:36,  7.04s/it]
 22%|██▏       | 48573/225000 [102:24:50<309:29:25,  6.32s/it]
 22%|██▏       | 48574/225000 [102:24:55<290:22:26,  5.93s/it]
 22%|██▏       | 48575/225000 [102:25:02<300:49:13,  6.14s/it]
 22%|██▏       | 48576/225000 [102:25:14<391:51:13,  8.00s/it]
 22%|██▏       | 48577/225000 [102:25:18<326:53:30,  6.67s/it]
 22%|██▏       | 48578/225000 [102:25:24<315:36:58,  6.44s/it]
 22%|██▏       | 48579/225000 [102:25:33<355:20:37,  7.25s/it]
 22%|██▏       | 48580/225000 [102:25:38<319:57:08,  6.53s/it]
 22%|██▏       | 48581/225000 [102:25:44<313:48:29,  6.40s/it]
 22%|██▏       | 48582/225000 [102:25:54<369:25:41,  7.54s/it]
 22%|██▏       | 48583/225000 [102:25:58<321:36:46,  6.56s/it]
 22%|██▏       | 48584/225000 [102:26:08<369:14:05,  7.53s/it]
 22%|██▏       | 48585/225000 [102:26:12<321:31:51,  6.56s/it]
 22%|██▏       | 48586/225000 [102:26:18<303:13:47,  6.19s/it]
 22%|██▏       | 48587/225000 [102:26:23<293:37:51,  5.99s/it]
 22%|██▏       | 48588/225000 [102:26:29<283:58:47,  5.80s/it]
 22%|██▏       | 48589/225000 [102:26:34<284:20:06,  5.80s/it]
 22%|██▏       | 48590/225000 [102:26:39<272:23:01,  5.56s/it]
 22%|██▏       | 48591/225000 [102:26:45<265:27:13,  5.42s/it]
 22%|██▏       | 48592/225000 [102:26:57<364:03:44,  7.43s/it]
 22%|██▏       | 48593/225000 [102:27:05<375:45:06,  7.67s/it]
 22%|██▏       | 48594/225000 [102:27:09<329:26:03,  6.72s/it]
 22%|██▏       | 48595/225000 [102:27:15<306:18:04,  6.25s/it]
 22%|██▏       | 48596/225000 [102:27:19<281:58:27,  5.75s/it]
 22%|██▏       | 48597/225000 [102:27:30<362:19:49,  7.39s/it]
 22%|██▏       | 48598/225000 [102:27:38<369:52:33,  7.55s/it]
 22%|██▏       | 48599/225000 [102:27:45<357:36:09,  7.30s/it]
 22%|██▏       | 48600/225000 [102:27:53<360:28:21,  7.36s/it]
                                                              
{'loss': '0.7169', 'grad_norm': '0.6351', 'learning_rate': '4.613e-05', 'epoch': '0.216', 'num_input_tokens_seen': 171935272, 'train_runtime': '3.689e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48600/225000 [102:27:53<360:28:21,  7.36s/it]
 22%|██▏       | 48601/225000 [102:27:59<341:23:56,  6.97s/it]
 22%|██▏       | 48602/225000 [102:28:04<323:37:37,  6.60s/it]
 22%|██▏       | 48603/225000 [102:28:10<307:41:00,  6.28s/it]
 22%|██▏       | 48604/225000 [102:28:16<301:08:16,  6.15s/it]
 22%|██▏       | 48605/225000 [102:28:21<292:19:03,  5.97s/it]
 22%|██▏       | 48606/225000 [102:28:26<280:03:28,  5.72s/it]
 22%|██▏       | 48607/225000 [102:28:32<279:30:35,  5.70s/it]
 22%|██▏       | 48608/225000 [102:28:39<298:34:16,  6.09s/it]
 22%|██▏       | 48609/225000 [102:28:45<290:27:15,  5.93s/it]
 22%|██▏       | 48610/225000 [102:28:50<281:51:09,  5.75s/it]
 22%|██▏       | 48611/225000 [102:28:58<323:15:43,  6.60s/it]
 22%|██▏       | 48612/225000 [102:29:04<312:08:50,  6.37s/it]
 22%|██▏       | 48613/225000 [102:29:11<313:38:48,  6.40s/it]
 22%|██▏       | 48614/225000 [102:29:23<398:46:24,  8.14s/it]
 22%|██▏       | 48615/225000 [102:29:27<342:20:22,  6.99s/it]
 22%|██▏       | 48616/225000 [102:29:37<385:47:33,  7.87s/it]
 22%|██▏       | 48617/225000 [102:29:42<337:09:29,  6.88s/it]
 22%|██▏       | 48618/225000 [102:29:48<322:17:29,  6.58s/it]
 22%|██▏       | 48619/225000 [102:29:56<355:11:57,  7.25s/it]
 22%|██▏       | 48620/225000 [102:30:04<357:29:52,  7.30s/it]
 22%|██▏       | 48621/225000 [102:30:08<308:18:34,  6.29s/it]
 22%|██▏       | 48622/225000 [102:30:16<340:42:09,  6.95s/it]
 22%|██▏       | 48623/225000 [102:30:22<315:21:44,  6.44s/it]
 22%|██▏       | 48624/225000 [102:30:28<309:23:28,  6.31s/it]
 22%|██▏       | 48625/225000 [102:30:33<293:20:00,  5.99s/it]
 22%|██▏       | 48626/225000 [102:30:38<281:55:03,  5.75s/it]
 22%|██▏       | 48627/225000 [102:30:43<266:29:08,  5.44s/it]
 22%|██▏       | 48628/225000 [102:30:55<359:48:00,  7.34s/it]
 22%|██▏       | 48629/225000 [102:31:03<377:07:59,  7.70s/it]
 22%|██▏       | 48630/225000 [102:31:07<321:57:27,  6.57s/it]
 22%|██▏       | 48631/225000 [102:31:15<340:05:52,  6.94s/it]
 22%|██▏       | 48632/225000 [102:31:23<351:20:29,  7.17s/it]
 22%|██▏       | 48633/225000 [102:31:30<350:44:18,  7.16s/it]
 22%|██▏       | 48634/225000 [102:31:35<324:08:18,  6.62s/it]
 22%|██▏       | 48635/225000 [102:31:42<333:32:29,  6.81s/it]
 22%|██▏       | 48636/225000 [102:31:49<331:13:49,  6.76s/it]
 22%|██▏       | 48637/225000 [102:31:53<290:50:21,  5.94s/it]
 22%|██▏       | 48638/225000 [102:32:03<351:54:23,  7.18s/it]
 22%|██▏       | 48639/225000 [102:32:07<300:01:09,  6.12s/it]
 22%|██▏       | 48640/225000 [102:32:16<347:26:01,  7.09s/it]
 22%|██▏       | 48641/225000 [102:32:20<295:14:41,  6.03s/it]
 22%|██▏       | 48642/225000 [102:32:27<321:17:11,  6.56s/it]
 22%|██▏       | 48643/225000 [102:32:32<288:02:06,  5.88s/it]
 22%|██▏       | 48644/225000 [102:32:42<359:28:25,  7.34s/it]
 22%|██▏       | 48645/225000 [102:32:46<310:54:44,  6.35s/it]
 22%|██▏       | 48646/225000 [102:32:55<346:23:57,  7.07s/it]
 22%|██▏       | 48647/225000 [102:32:59<297:22:00,  6.07s/it]
 22%|██▏       | 48648/225000 [102:33:03<265:14:43,  5.41s/it]
 22%|██▏       | 48649/225000 [102:33:15<363:27:17,  7.42s/it]
 22%|██▏       | 48650/225000 [102:33:19<308:42:25,  6.30s/it]
                                                              
{'loss': '0.7087', 'grad_norm': '0.7583', 'learning_rate': '4.612e-05', 'epoch': '0.2162', 'num_input_tokens_seen': 172082392, 'train_runtime': '3.692e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48650/225000 [102:33:19<308:42:25,  6.30s/it]
 22%|██▏       | 48651/225000 [102:33:23<277:37:43,  5.67s/it]
 22%|██▏       | 48652/225000 [102:33:35<369:39:15,  7.55s/it]
 22%|██▏       | 48653/225000 [102:33:41<344:09:09,  7.03s/it]
 22%|██▏       | 48654/225000 [102:33:46<318:50:39,  6.51s/it]
 22%|██▏       | 48655/225000 [102:33:54<347:15:38,  7.09s/it]
 22%|██▏       | 48656/225000 [102:33:58<297:24:37,  6.07s/it]
 22%|██▏       | 48657/225000 [102:34:03<284:55:06,  5.82s/it]
 22%|██▏       | 48658/225000 [102:34:11<315:24:46,  6.44s/it]
 22%|██▏       | 48659/225000 [102:34:19<333:05:58,  6.80s/it]
 22%|██▏       | 48660/225000 [102:34:25<321:19:02,  6.56s/it]
 22%|██▏       | 48661/225000 [102:34:32<330:06:44,  6.74s/it]
 22%|██▏       | 48662/225000 [102:34:38<326:06:14,  6.66s/it]
 22%|██▏       | 48663/225000 [102:34:43<295:58:09,  6.04s/it]
 22%|██▏       | 48664/225000 [102:34:47<270:53:30,  5.53s/it]
 22%|██▏       | 48665/225000 [102:34:59<365:26:48,  7.46s/it]
 22%|██▏       | 48666/225000 [102:35:03<317:45:19,  6.49s/it]
 22%|██▏       | 48667/225000 [102:35:08<284:51:26,  5.82s/it]
 22%|██▏       | 48668/225000 [102:35:13<270:40:03,  5.53s/it]
 22%|██▏       | 48669/225000 [102:35:23<343:55:25,  7.02s/it]
 22%|██▏       | 48670/225000 [102:35:30<342:45:04,  7.00s/it]
 22%|██▏       | 48671/225000 [102:35:35<316:22:18,  6.46s/it]
 22%|██▏       | 48672/225000 [102:35:41<307:01:52,  6.27s/it]
 22%|██▏       | 48673/225000 [102:35:47<300:10:38,  6.13s/it]
 22%|██▏       | 48674/225000 [102:35:52<291:34:20,  5.95s/it]
 22%|██▏       | 48675/225000 [102:35:58<292:40:10,  5.98s/it]
 22%|██▏       | 48676/225000 [102:36:09<359:48:38,  7.35s/it]
 22%|██▏       | 48677/225000 [102:36:15<346:14:50,  7.07s/it]
 22%|██▏       | 48678/225000 [102:36:20<311:41:37,  6.36s/it]
 22%|██▏       | 48679/225000 [102:36:25<284:50:36,  5.82s/it]
 22%|██▏       | 48680/225000 [102:36:30<274:41:57,  5.61s/it]
 22%|██▏       | 48681/225000 [102:36:36<279:08:12,  5.70s/it]
 22%|██▏       | 48682/225000 [102:36:47<364:28:44,  7.44s/it]
 22%|██▏       | 48683/225000 [102:36:54<359:44:56,  7.35s/it]
 22%|██▏       | 48684/225000 [102:37:04<390:30:05,  7.97s/it]
 22%|██▏       | 48685/225000 [102:37:08<341:47:41,  6.98s/it]
 22%|██▏       | 48686/225000 [102:37:14<319:50:51,  6.53s/it]
 22%|██▏       | 48687/225000 [102:37:19<304:31:30,  6.22s/it]
 22%|██▏       | 48688/225000 [102:37:25<298:07:15,  6.09s/it]
 22%|██▏       | 48689/225000 [102:37:30<286:03:15,  5.84s/it]
 22%|██▏       | 48690/225000 [102:37:36<279:33:29,  5.71s/it]
 22%|██▏       | 48691/225000 [102:37:47<365:55:14,  7.47s/it]
 22%|██▏       | 48692/225000 [102:37:58<405:59:45,  8.29s/it]
 22%|██▏       | 48693/225000 [102:38:01<341:19:59,  6.97s/it]
 22%|██▏       | 48694/225000 [102:38:06<309:59:21,  6.33s/it]
 22%|██▏       | 48695/225000 [102:38:12<297:44:11,  6.08s/it]
 22%|██▏       | 48696/225000 [102:38:19<313:13:25,  6.40s/it]
 22%|██▏       | 48697/225000 [102:38:32<403:43:50,  8.24s/it]
 22%|██▏       | 48698/225000 [102:38:35<336:15:02,  6.87s/it]
 22%|██▏       | 48699/225000 [102:38:46<393:10:16,  8.03s/it]
 22%|██▏       | 48700/225000 [102:38:49<322:43:39,  6.59s/it]
                                                              
{'loss': '0.6921', 'grad_norm': '0.7928', 'learning_rate': '4.611e-05', 'epoch': '0.2164', 'num_input_tokens_seen': 172228856, 'train_runtime': '3.695e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48700/225000 [102:38:49<322:43:39,  6.59s/it]
 22%|██▏       | 48701/225000 [102:38:58<360:37:29,  7.36s/it]
 22%|██▏       | 48702/225000 [102:39:02<311:03:37,  6.35s/it]
 22%|██▏       | 48703/225000 [102:39:11<347:02:19,  7.09s/it]
 22%|██▏       | 48704/225000 [102:39:15<304:22:29,  6.22s/it]
 22%|██▏       | 48705/225000 [102:39:21<295:11:45,  6.03s/it]
 22%|██▏       | 48706/225000 [102:39:33<378:24:56,  7.73s/it]
 22%|██▏       | 48707/225000 [102:39:38<349:21:40,  7.13s/it]
 22%|██▏       | 48708/225000 [102:39:45<348:51:25,  7.12s/it]
 22%|██▏       | 48709/225000 [102:39:54<364:29:22,  7.44s/it]
 22%|██▏       | 48710/225000 [102:40:00<348:33:20,  7.12s/it]
 22%|██▏       | 48711/225000 [102:40:04<299:35:18,  6.12s/it]
 22%|██▏       | 48712/225000 [102:40:12<334:54:23,  6.84s/it]
 22%|██▏       | 48713/225000 [102:40:17<299:13:33,  6.11s/it]
 22%|██▏       | 48714/225000 [102:40:22<290:40:55,  5.94s/it]
 22%|██▏       | 48715/225000 [102:40:30<320:11:19,  6.54s/it]
 22%|██▏       | 48716/225000 [102:40:36<315:20:11,  6.44s/it]
 22%|██▏       | 48717/225000 [102:40:40<279:51:06,  5.72s/it]
 22%|██▏       | 48718/225000 [102:40:50<343:54:58,  7.02s/it]
 22%|██▏       | 48719/225000 [102:40:55<309:02:19,  6.31s/it]
 22%|██▏       | 48720/225000 [102:41:05<354:14:36,  7.23s/it]
 22%|██▏       | 48721/225000 [102:41:12<355:05:58,  7.25s/it]
 22%|██▏       | 48722/225000 [102:41:16<317:11:43,  6.48s/it]
 22%|██▏       | 48723/225000 [102:41:21<289:39:03,  5.92s/it]
 22%|██▏       | 48724/225000 [102:41:28<304:55:28,  6.23s/it]
 22%|██▏       | 48725/225000 [102:41:37<346:00:09,  7.07s/it]
 22%|██▏       | 48726/225000 [102:41:42<315:05:57,  6.44s/it]
 22%|██▏       | 48727/225000 [102:41:47<288:37:59,  5.89s/it]
 22%|██▏       | 48728/225000 [102:41:51<272:58:43,  5.58s/it]
 22%|██▏       | 48729/225000 [102:42:02<349:31:00,  7.14s/it]
 22%|██▏       | 48730/225000 [102:42:11<366:59:35,  7.50s/it]
 22%|██▏       | 48731/225000 [102:42:14<312:20:53,  6.38s/it]
 22%|██▏       | 48732/225000 [102:42:24<358:17:00,  7.32s/it]
 22%|██▏       | 48733/225000 [102:42:31<353:16:05,  7.21s/it]
 22%|██▏       | 48734/225000 [102:42:39<363:14:44,  7.42s/it]
 22%|██▏       | 48735/225000 [102:42:43<319:22:51,  6.52s/it]
 22%|██▏       | 48736/225000 [102:42:52<350:38:37,  7.16s/it]
 22%|██▏       | 48737/225000 [102:42:58<333:45:39,  6.82s/it]
 22%|██▏       | 48738/225000 [102:43:03<305:49:15,  6.25s/it]
 22%|██▏       | 48739/225000 [102:43:11<338:58:18,  6.92s/it]
 22%|██▏       | 48740/225000 [102:43:15<290:16:28,  5.93s/it]
 22%|██▏       | 48741/225000 [102:43:20<283:38:59,  5.79s/it]
 22%|██▏       | 48742/225000 [102:43:26<279:39:14,  5.71s/it]
 22%|██▏       | 48743/225000 [102:43:31<273:21:03,  5.58s/it]
 22%|██▏       | 48744/225000 [102:43:41<338:50:46,  6.92s/it]
 22%|██▏       | 48745/225000 [102:43:51<384:42:31,  7.86s/it]
 22%|██▏       | 48746/225000 [102:43:56<339:12:45,  6.93s/it]
 22%|██▏       | 48747/225000 [102:44:01<305:09:34,  6.23s/it]
 22%|██▏       | 48748/225000 [102:44:06<290:38:41,  5.94s/it]
 22%|██▏       | 48749/225000 [102:44:11<277:26:01,  5.67s/it]
 22%|██▏       | 48750/225000 [102:44:22<364:00:14,  7.43s/it]
                                                              
{'loss': '0.7076', 'grad_norm': '0.6927', 'learning_rate': '4.61e-05', 'epoch': '0.2167', 'num_input_tokens_seen': 172378080, 'train_runtime': '3.699e+05', 'train_tokens_per_second': '466.1'}

 22%|██▏       | 48750/225000 [102:44:22<364:00:14,  7.43s/it]
 22%|██▏       | 48751/225000 [102:44:29<344:22:42,  7.03s/it]
 22%|██▏       | 48752/225000 [102:44:34<322:53:10,  6.60s/it]
 22%|██▏       | 48753/225000 [102:44:39<293:24:24,  5.99s/it]
 22%|██▏       | 48754/225000 [102:44:50<364:20:33,  7.44s/it]
 22%|██▏       | 48755/225000 [102:44:55<340:24:44,  6.95s/it]
 22%|██▏       | 48756/225000 [102:45:02<342:21:12,  6.99s/it]
 22%|██▏       | 48757/225000 [102:45:10<350:48:55,  7.17s/it]
 22%|██▏       | 48758/225000 [102:45:17<344:20:51,  7.03s/it]
 22%|██▏       | 48759/225000 [102:45:24<350:00:12,  7.15s/it]
 22%|██▏       | 48760/225000 [102:45:29<321:25:51,  6.57s/it]
 22%|██▏       | 48761/225000 [102:45:34<297:12:04,  6.07s/it]
 22%|██▏       | 48762/225000 [102:45:39<278:44:12,  5.69s/it]
 22%|██▏       | 48763/225000 [102:45:52<389:12:01,  7.95s/it]
 22%|██▏       | 48764/225000 [102:45:57<348:37:37,  7.12s/it]
 22%|██▏       | 48765/225000 [102:46:02<304:42:00,  6.22s/it]
 22%|██▏       | 48766/225000 [102:46:08<311:30:31,  6.36s/it]
 22%|██▏       | 48767/225000 [102:46:17<345:08:53,  7.05s/it]
 22%|██▏       | 48768/225000 [102:46:24<344:21:21,  7.03s/it]
 22%|██▏       | 48769/225000 [102:46:30<333:46:13,  6.82s/it]
 22%|██▏       | 48770/225000 [102:46:34<292:53:03,  5.98s/it]
 22%|██▏       | 48771/225000 [102:46:42<318:18:58,  6.50s/it]
 22%|██▏       | 48772/225000 [102:46:50<346:26:23,  7.08s/it]
 22%|██▏       | 48773/225000 [102:46:56<330:20:58,  6.75s/it]
 22%|██▏       | 48774/225000 [102:47:01<298:38:11,  6.10s/it]
 22%|██▏       | 48775/225000 [102:47:12<367:03:00,  7.50s/it]
 22%|██▏       | 48776/225000 [102:47:16<321:10:47,  6.56s/it]
 22%|██▏       | 48777/225000 [102:47:21<298:11:43,  6.09s/it]
 22%|██▏       | 48778/225000 [102:47:26<279:49:46,  5.72s/it]
 22%|██▏       | 48779/225000 [102:47:33<294:22:30,  6.01s/it]
 22%|██▏       | 48780/225000 [102:47:45<390:51:16,  7.98s/it]
 22%|██▏       | 48781/225000 [102:47:49<333:07:57,  6.81s/it]
 22%|██▏       | 48782/225000 [102:47:59<370:47:21,  7.57s/it]
 22%|██▏       | 48783/225000 [102:48:04<336:43:04,  6.88s/it]
 22%|██▏       | 48784/225000 [102:48:11<339:12:01,  6.93s/it]
 22%|██▏       | 48785/225000 [102:48:15<295:22:27,  6.03s/it]
 22%|██▏       | 48786/225000 [102:48:26<374:09:36,  7.64s/it]
 22%|██▏       | 48787/225000 [102:48:32<347:58:03,  7.11s/it]
 22%|██▏       | 48788/225000 [102:48:36<297:18:11,  6.07s/it]
 22%|██▏       | 48789/225000 [102:48:44<328:47:57,  6.72s/it]
 22%|██▏       | 48790/225000 [102:48:48<288:12:35,  5.89s/it]
 22%|██▏       | 48791/225000 [102:48:58<344:59:04,  7.05s/it]
 22%|██▏       | 48792/225000 [102:49:06<365:22:30,  7.46s/it]
 22%|██▏       | 48793/225000 [102:49:11<325:39:24,  6.65s/it]
 22%|██▏       | 48794/225000 [102:49:18<334:14:02,  6.83s/it]
 22%|██▏       | 48795/225000 [102:49:24<314:15:28,  6.42s/it]
 22%|██▏       | 48796/225000 [102:49:31<327:43:20,  6.70s/it]
 22%|██▏       | 48797/225000 [102:49:36<307:38:43,  6.29s/it]
 22%|██▏       | 48798/225000 [102:49:42<304:20:37,  6.22s/it]
 22%|██▏       | 48799/225000 [102:49:48<294:26:46,  6.02s/it]
 22%|██▏       | 48800/225000 [102:49:58<358:48:26,  7.33s/it]
                                                              
{'loss': '0.715', 'grad_norm': '0.7749', 'learning_rate': '4.609e-05', 'epoch': '0.2169', 'num_input_tokens_seen': 172526592, 'train_runtime': '3.702e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 48800/225000 [102:49:58<358:48:26,  7.33s/it]
 22%|██▏       | 48801/225000 [102:50:04<337:41:57,  6.90s/it]
 22%|██▏       | 48802/225000 [102:50:10<321:07:32,  6.56s/it]
 22%|██▏       | 48803/225000 [102:50:16<309:07:05,  6.32s/it]
 22%|██▏       | 48804/225000 [102:50:21<293:12:14,  5.99s/it]
 22%|██▏       | 48805/225000 [102:50:26<279:34:27,  5.71s/it]
 22%|██▏       | 48806/225000 [102:50:31<271:48:30,  5.55s/it]
 22%|██▏       | 48807/225000 [102:50:44<383:20:23,  7.83s/it]
 22%|██▏       | 48808/225000 [102:50:49<340:53:44,  6.97s/it]
 22%|██▏       | 48809/225000 [102:50:58<368:32:20,  7.53s/it]
 22%|██▏       | 48810/225000 [102:51:05<364:31:51,  7.45s/it]
 22%|██▏       | 48811/225000 [102:51:13<361:33:56,  7.39s/it]
 22%|██▏       | 48812/225000 [102:51:16<308:07:07,  6.30s/it]
 22%|██▏       | 48813/225000 [102:51:26<357:18:47,  7.30s/it]
 22%|██▏       | 48814/225000 [102:51:31<321:03:15,  6.56s/it]
 22%|██▏       | 48815/225000 [102:51:35<288:51:37,  5.90s/it]
 22%|██▏       | 48816/225000 [102:51:40<266:18:46,  5.44s/it]
 22%|██▏       | 48817/225000 [102:51:44<253:41:12,  5.18s/it]
 22%|██▏       | 48818/225000 [102:51:55<338:54:20,  6.92s/it]
 22%|██▏       | 48819/225000 [102:52:03<357:52:19,  7.31s/it]
 22%|██▏       | 48820/225000 [102:52:08<322:45:05,  6.59s/it]
 22%|██▏       | 48821/225000 [102:52:13<293:45:20,  6.00s/it]
 22%|██▏       | 48822/225000 [102:52:24<361:08:33,  7.38s/it]
 22%|██▏       | 48823/225000 [102:52:32<375:06:50,  7.67s/it]
 22%|██▏       | 48824/225000 [102:52:37<334:00:57,  6.83s/it]
 22%|██▏       | 48825/225000 [102:52:42<309:32:59,  6.33s/it]
 22%|██▏       | 48826/225000 [102:52:55<403:25:23,  8.24s/it]
 22%|██▏       | 48827/225000 [102:52:59<350:00:23,  7.15s/it]
 22%|██▏       | 48828/225000 [102:53:06<340:22:28,  6.96s/it]
 22%|██▏       | 48829/225000 [102:53:12<337:11:53,  6.89s/it]
 22%|██▏       | 48830/225000 [102:53:18<309:57:43,  6.33s/it]
 22%|██▏       | 48831/225000 [102:53:22<285:49:44,  5.84s/it]
 22%|██▏       | 48832/225000 [102:53:31<324:44:43,  6.64s/it]
 22%|██▏       | 48833/225000 [102:53:41<376:56:22,  7.70s/it]
 22%|██▏       | 48834/225000 [102:53:45<317:23:22,  6.49s/it]
 22%|██▏       | 48835/225000 [102:53:53<353:21:56,  7.22s/it]
 22%|██▏       | 48836/225000 [102:53:58<312:59:50,  6.40s/it]
 22%|██▏       | 48837/225000 [102:54:06<340:19:00,  6.95s/it]
 22%|██▏       | 48838/225000 [102:54:14<358:05:22,  7.32s/it]
 22%|██▏       | 48839/225000 [102:54:19<318:20:57,  6.51s/it]
 22%|██▏       | 48840/225000 [102:54:25<309:05:09,  6.32s/it]
 22%|██▏       | 48841/225000 [102:54:31<299:42:37,  6.12s/it]
 22%|██▏       | 48842/225000 [102:54:36<287:25:09,  5.87s/it]
 22%|██▏       | 48843/225000 [102:54:41<279:34:36,  5.71s/it]
 22%|██▏       | 48844/225000 [102:54:46<267:57:47,  5.48s/it]
 22%|██▏       | 48845/225000 [102:54:51<267:08:35,  5.46s/it]
 22%|██▏       | 48846/225000 [102:55:01<323:53:09,  6.62s/it]
 22%|██▏       | 48847/225000 [102:55:12<385:41:38,  7.88s/it]
 22%|██▏       | 48848/225000 [102:55:16<337:25:31,  6.90s/it]
 22%|██▏       | 48849/225000 [102:55:22<316:13:15,  6.46s/it]
 22%|██▏       | 48850/225000 [102:55:27<299:05:54,  6.11s/it]
                                                              
{'loss': '0.7075', 'grad_norm': '0.766', 'learning_rate': '4.608e-05', 'epoch': '0.2171', 'num_input_tokens_seen': 172675312, 'train_runtime': '3.705e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 48850/225000 [102:55:27<299:05:54,  6.11s/it]
 22%|██▏       | 48851/225000 [102:55:32<289:27:13,  5.92s/it]
 22%|██▏       | 48852/225000 [102:55:38<281:58:32,  5.76s/it]
 22%|██▏       | 48853/225000 [102:55:44<289:30:29,  5.92s/it]
 22%|██▏       | 48854/225000 [102:55:58<399:49:46,  8.17s/it]
 22%|██▏       | 48855/225000 [102:56:03<359:00:50,  7.34s/it]
 22%|██▏       | 48856/225000 [102:56:11<366:11:33,  7.48s/it]
 22%|██▏       | 48857/225000 [102:56:15<319:11:08,  6.52s/it]
 22%|██▏       | 48858/225000 [102:56:23<338:35:54,  6.92s/it]
 22%|██▏       | 48859/225000 [102:56:28<305:45:20,  6.25s/it]
 22%|██▏       | 48860/225000 [102:56:32<282:49:25,  5.78s/it]
 22%|██▏       | 48861/225000 [102:56:38<283:46:46,  5.80s/it]
 22%|██▏       | 48862/225000 [102:56:43<274:10:57,  5.60s/it]
 22%|██▏       | 48863/225000 [102:56:56<373:30:19,  7.63s/it]
 22%|██▏       | 48864/225000 [102:57:05<396:30:55,  8.10s/it]
 22%|██▏       | 48865/225000 [102:57:11<362:00:58,  7.40s/it]
 22%|██▏       | 48866/225000 [102:57:17<353:25:52,  7.22s/it]
 22%|██▏       | 48867/225000 [102:57:23<332:14:03,  6.79s/it]
 22%|██▏       | 48868/225000 [102:57:31<341:22:47,  6.98s/it]
 22%|██▏       | 48869/225000 [102:57:35<299:10:16,  6.11s/it]
 22%|██▏       | 48870/225000 [102:57:41<298:35:52,  6.10s/it]
 22%|██▏       | 48871/225000 [102:57:48<322:16:33,  6.59s/it]
 22%|██▏       | 48872/225000 [102:57:53<295:28:47,  6.04s/it]
 22%|██▏       | 48873/225000 [102:57:58<279:31:00,  5.71s/it]
 22%|██▏       | 48874/225000 [102:58:03<265:06:04,  5.42s/it]
 22%|██▏       | 48875/225000 [102:58:14<348:53:34,  7.13s/it]
 22%|██▏       | 48876/225000 [102:58:22<365:16:44,  7.47s/it]
 22%|██▏       | 48877/225000 [102:58:26<306:19:36,  6.26s/it]
 22%|██▏       | 48878/225000 [102:58:31<289:59:24,  5.93s/it]
 22%|██▏       | 48879/225000 [102:58:36<282:30:40,  5.77s/it]
 22%|██▏       | 48880/225000 [102:58:46<341:44:06,  6.99s/it]
 22%|██▏       | 48881/225000 [102:58:55<372:56:51,  7.62s/it]
 22%|██▏       | 48882/225000 [102:59:02<357:25:25,  7.31s/it]
 22%|██▏       | 48883/225000 [102:59:08<337:02:03,  6.89s/it]
 22%|██▏       | 48884/225000 [102:59:13<308:57:20,  6.32s/it]
 22%|██▏       | 48885/225000 [102:59:17<281:30:07,  5.75s/it]
 22%|██▏       | 48886/225000 [102:59:28<357:32:30,  7.31s/it]
 22%|██▏       | 48887/225000 [102:59:37<374:46:24,  7.66s/it]
 22%|██▏       | 48888/225000 [102:59:41<326:14:23,  6.67s/it]
 22%|██▏       | 48889/225000 [102:59:49<340:59:09,  6.97s/it]
 22%|██▏       | 48890/225000 [102:59:53<299:08:33,  6.12s/it]
 22%|██▏       | 48891/225000 [103:00:00<321:44:40,  6.58s/it]
 22%|██▏       | 48892/225000 [103:00:07<320:38:32,  6.55s/it]
 22%|██▏       | 48893/225000 [103:00:15<349:33:12,  7.15s/it]
 22%|██▏       | 48894/225000 [103:00:20<309:44:54,  6.33s/it]
 22%|██▏       | 48895/225000 [103:00:24<283:14:09,  5.79s/it]
 22%|██▏       | 48896/225000 [103:00:34<342:45:43,  7.01s/it]
 22%|██▏       | 48897/225000 [103:00:43<372:26:27,  7.61s/it]
 22%|██▏       | 48898/225000 [103:00:48<334:53:52,  6.85s/it]
 22%|██▏       | 48899/225000 [103:00:54<311:33:18,  6.37s/it]
 22%|██▏       | 48900/225000 [103:00:59<291:47:04,  5.96s/it]
                                                              
{'loss': '0.7039', 'grad_norm': '1.087', 'learning_rate': '4.607e-05', 'epoch': '0.2173', 'num_input_tokens_seen': 172822624, 'train_runtime': '3.709e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 48900/225000 [103:00:59<291:47:04,  5.96s/it]
 22%|██▏       | 48901/225000 [103:01:03<275:15:35,  5.63s/it]
 22%|██▏       | 48902/225000 [103:01:15<355:59:39,  7.28s/it]
 22%|██▏       | 48903/225000 [103:01:20<332:37:52,  6.80s/it]
 22%|██▏       | 48904/225000 [103:01:29<362:36:41,  7.41s/it]
 22%|██▏       | 48905/225000 [103:01:33<310:22:41,  6.35s/it]
 22%|██▏       | 48906/225000 [103:01:41<333:30:51,  6.82s/it]
 22%|██▏       | 48907/225000 [103:01:45<292:45:53,  5.99s/it]
 22%|██▏       | 48908/225000 [103:01:55<353:37:37,  7.23s/it]
 22%|██▏       | 48909/225000 [103:02:02<343:17:12,  7.02s/it]
 22%|██▏       | 48910/225000 [103:02:07<317:31:45,  6.49s/it]
 22%|██▏       | 48911/225000 [103:02:12<298:03:39,  6.09s/it]
 22%|██▏       | 48912/225000 [103:02:17<285:23:18,  5.83s/it]
 22%|██▏       | 48913/225000 [103:02:22<275:48:03,  5.64s/it]
 22%|██▏       | 48914/225000 [103:02:32<327:24:48,  6.69s/it]
 22%|██▏       | 48915/225000 [103:02:42<378:46:34,  7.74s/it]
 22%|██▏       | 48916/225000 [103:02:46<325:47:17,  6.66s/it]
 22%|██▏       | 48917/225000 [103:02:53<330:51:39,  6.76s/it]
 22%|██▏       | 48918/225000 [103:03:01<343:40:01,  7.03s/it]
 22%|██▏       | 48919/225000 [103:03:07<341:34:18,  6.98s/it]
 22%|██▏       | 48920/225000 [103:03:12<307:36:43,  6.29s/it]
 22%|██▏       | 48921/225000 [103:03:18<299:06:02,  6.12s/it]
 22%|██▏       | 48922/225000 [103:03:23<291:02:34,  5.95s/it]
 22%|██▏       | 48923/225000 [103:03:29<284:29:33,  5.82s/it]
 22%|██▏       | 48924/225000 [103:03:35<288:37:21,  5.90s/it]
 22%|██▏       | 48925/225000 [103:03:43<326:05:38,  6.67s/it]
 22%|██▏       | 48926/225000 [103:03:54<383:14:10,  7.84s/it]
 22%|██▏       | 48927/225000 [103:03:59<346:34:51,  7.09s/it]
 22%|██▏       | 48928/225000 [103:04:05<321:03:02,  6.56s/it]
 22%|██▏       | 48929/225000 [103:04:10<308:21:16,  6.30s/it]
 22%|██▏       | 48930/225000 [103:04:15<291:26:47,  5.96s/it]
 22%|██▏       | 48931/225000 [103:04:24<332:18:28,  6.79s/it]
 22%|██▏       | 48932/225000 [103:04:32<345:27:17,  7.06s/it]
 22%|██▏       | 48933/225000 [103:04:39<349:38:48,  7.15s/it]
 22%|██▏       | 48934/225000 [103:04:45<326:35:39,  6.68s/it]
 22%|██▏       | 48935/225000 [103:04:55<383:01:29,  7.83s/it]
 22%|██▏       | 48936/225000 [103:05:00<334:03:45,  6.83s/it]
 22%|██▏       | 48937/225000 [103:05:05<307:42:43,  6.29s/it]
 22%|██▏       | 48938/225000 [103:05:13<333:29:49,  6.82s/it]
 22%|██▏       | 48939/225000 [103:05:24<395:26:08,  8.09s/it]
 22%|██▏       | 48940/225000 [103:05:28<331:32:03,  6.78s/it]
 22%|██▏       | 48941/225000 [103:05:37<371:59:54,  7.61s/it]
 22%|██▏       | 48942/225000 [103:05:41<320:02:47,  6.54s/it]
 22%|██▏       | 48943/225000 [103:05:50<352:42:35,  7.21s/it]
 22%|██▏       | 48944/225000 [103:05:54<308:36:26,  6.31s/it]
 22%|██▏       | 48945/225000 [103:05:58<274:10:15,  5.61s/it]
 22%|██▏       | 48946/225000 [103:06:04<279:17:58,  5.71s/it]
 22%|██▏       | 48947/225000 [103:06:14<340:55:46,  6.97s/it]
 22%|██▏       | 48948/225000 [103:06:21<345:37:33,  7.07s/it]
 22%|██▏       | 48949/225000 [103:06:28<335:41:54,  6.86s/it]
 22%|██▏       | 48950/225000 [103:06:35<346:12:27,  7.08s/it]
                                                              
{'loss': '0.7069', 'grad_norm': '0.747', 'learning_rate': '4.606e-05', 'epoch': '0.2176', 'num_input_tokens_seen': 172971952, 'train_runtime': '3.712e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 48950/225000 [103:06:35<346:12:27,  7.08s/it]
 22%|██▏       | 48951/225000 [103:06:43<349:08:28,  7.14s/it]
 22%|██▏       | 48952/225000 [103:06:50<346:49:13,  7.09s/it]
 22%|██▏       | 48953/225000 [103:06:54<303:02:22,  6.20s/it]
 22%|██▏       | 48954/225000 [103:06:59<283:27:43,  5.80s/it]
 22%|██▏       | 48955/225000 [103:07:04<283:26:42,  5.80s/it]
 22%|██▏       | 48956/225000 [103:07:11<290:24:11,  5.94s/it]
 22%|██▏       | 48957/225000 [103:07:23<390:25:04,  7.98s/it]
 22%|██▏       | 48958/225000 [103:07:27<325:30:42,  6.66s/it]
 22%|██▏       | 48959/225000 [103:07:32<307:37:14,  6.29s/it]
 22%|██▏       | 48960/225000 [103:07:42<352:39:40,  7.21s/it]
 22%|██▏       | 48961/225000 [103:07:48<339:06:04,  6.93s/it]
 22%|██▏       | 48962/225000 [103:07:55<340:30:28,  6.96s/it]
 22%|██▏       | 48963/225000 [103:08:01<328:56:23,  6.73s/it]
 22%|██▏       | 48964/225000 [103:08:10<351:08:41,  7.18s/it]
 22%|██▏       | 48965/225000 [103:08:14<316:58:31,  6.48s/it]
 22%|██▏       | 48966/225000 [103:08:20<302:11:13,  6.18s/it]
 22%|██▏       | 48967/225000 [103:08:28<323:25:40,  6.61s/it]
 22%|██▏       | 48968/225000 [103:08:32<287:58:19,  5.89s/it]
 22%|██▏       | 48969/225000 [103:08:37<276:22:55,  5.65s/it]
 22%|██▏       | 48970/225000 [103:08:47<345:40:58,  7.07s/it]
 22%|██▏       | 48971/225000 [103:08:55<359:47:56,  7.36s/it]
 22%|██▏       | 48972/225000 [103:09:04<379:33:50,  7.76s/it]
 22%|██▏       | 48973/225000 [103:09:09<332:55:43,  6.81s/it]
 22%|██▏       | 48974/225000 [103:09:13<297:01:09,  6.07s/it]
 22%|██▏       | 48975/225000 [103:09:18<276:39:00,  5.66s/it]
 22%|██▏       | 48976/225000 [103:09:22<261:52:54,  5.36s/it]
 22%|██▏       | 48977/225000 [103:09:34<357:29:55,  7.31s/it]
 22%|██▏       | 48978/225000 [103:09:41<357:27:17,  7.31s/it]
 22%|██▏       | 48979/225000 [103:09:45<309:58:24,  6.34s/it]
 22%|██▏       | 48980/225000 [103:09:50<288:04:42,  5.89s/it]
 22%|██▏       | 48981/225000 [103:09:56<280:12:21,  5.73s/it]
 22%|██▏       | 48982/225000 [103:10:05<333:37:16,  6.82s/it]
 22%|██▏       | 48983/225000 [103:10:14<367:10:37,  7.51s/it]
 22%|██▏       | 48984/225000 [103:10:19<327:00:13,  6.69s/it]
 22%|██▏       | 48985/225000 [103:10:24<301:50:09,  6.17s/it]
 22%|██▏       | 48986/225000 [103:10:32<337:01:06,  6.89s/it]
 22%|██▏       | 48987/225000 [103:10:42<377:59:43,  7.73s/it]
 22%|██▏       | 48988/225000 [103:10:47<336:21:20,  6.88s/it]
 22%|██▏       | 48989/225000 [103:10:51<300:25:40,  6.14s/it]
 22%|██▏       | 48990/225000 [103:10:56<274:11:32,  5.61s/it]
 22%|██▏       | 48991/225000 [103:11:09<381:19:24,  7.80s/it]
 22%|██▏       | 48992/225000 [103:11:14<347:55:34,  7.12s/it]
 22%|██▏       | 48993/225000 [103:11:18<305:04:45,  6.24s/it]
 22%|██▏       | 48994/225000 [103:11:29<369:49:03,  7.56s/it]
 22%|██▏       | 48995/225000 [103:11:36<367:05:49,  7.51s/it]
 22%|██▏       | 48996/225000 [103:11:40<315:20:52,  6.45s/it]
 22%|██▏       | 48997/225000 [103:11:48<326:49:16,  6.68s/it]
 22%|██▏       | 48998/225000 [103:11:55<328:36:51,  6.72s/it]
 22%|██▏       | 48999/225000 [103:12:03<352:25:49,  7.21s/it]
 22%|██▏       | 49000/225000 [103:12:07<307:10:48,  6.28s/it]
                                                              
{'loss': '0.7017', 'grad_norm': '0.7396', 'learning_rate': '4.605e-05', 'epoch': '0.2178', 'num_input_tokens_seen': 173121640, 'train_runtime': '3.715e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 49000/225000 [103:12:07<307:10:48,  6.28s/it]
 22%|██▏       | 49001/225000 [103:12:14<322:27:36,  6.60s/it]
 22%|██▏       | 49002/225000 [103:12:20<315:10:48,  6.45s/it]
 22%|██▏       | 49003/225000 [103:12:29<351:41:21,  7.19s/it]
 22%|██▏       | 49004/225000 [103:12:37<361:19:10,  7.39s/it]
 22%|██▏       | 49005/225000 [103:12:41<316:03:49,  6.47s/it]
 22%|██▏       | 49006/225000 [103:12:50<339:16:45,  6.94s/it]
 22%|██▏       | 49007/225000 [103:12:55<315:27:45,  6.45s/it]
 22%|██▏       | 49008/225000 [103:12:59<282:05:06,  5.77s/it]
 22%|██▏       | 49009/225000 [103:13:06<305:10:32,  6.24s/it]
 22%|██▏       | 49010/225000 [103:13:15<338:56:06,  6.93s/it]
 22%|██▏       | 49011/225000 [103:13:19<302:35:42,  6.19s/it]
 22%|██▏       | 49012/225000 [103:13:24<285:51:50,  5.85s/it]
 22%|██▏       | 49013/225000 [103:13:30<277:40:11,  5.68s/it]
 22%|██▏       | 49014/225000 [103:13:34<260:26:09,  5.33s/it]
 22%|██▏       | 49015/225000 [103:13:40<261:31:11,  5.35s/it]
 22%|██▏       | 49016/225000 [103:13:52<368:26:17,  7.54s/it]
 22%|██▏       | 49017/225000 [103:13:57<328:52:39,  6.73s/it]
 22%|██▏       | 49018/225000 [103:14:06<358:42:00,  7.34s/it]
 22%|██▏       | 49019/225000 [103:14:10<315:44:32,  6.46s/it]
 22%|██▏       | 49020/225000 [103:14:16<310:14:34,  6.35s/it]
 22%|██▏       | 49021/225000 [103:14:27<366:46:17,  7.50s/it]
 22%|██▏       | 49022/225000 [103:14:31<314:38:53,  6.44s/it]
 22%|██▏       | 49023/225000 [103:14:35<293:00:01,  5.99s/it]
 22%|██▏       | 49024/225000 [103:14:41<288:18:19,  5.90s/it]
 22%|██▏       | 49025/225000 [103:14:47<281:00:57,  5.75s/it]
 22%|██▏       | 49026/225000 [103:14:52<279:57:48,  5.73s/it]
 22%|██▏       | 49027/225000 [103:15:02<335:08:16,  6.86s/it]
 22%|██▏       | 49028/225000 [103:15:13<399:17:48,  8.17s/it]
 22%|██▏       | 49029/225000 [103:15:16<331:14:09,  6.78s/it]
 22%|██▏       | 49030/225000 [103:15:21<303:26:16,  6.21s/it]
 22%|██▏       | 49031/225000 [103:15:26<282:53:41,  5.79s/it]
 22%|██▏       | 49032/225000 [103:15:32<279:07:57,  5.71s/it]
 22%|██▏       | 49033/225000 [103:15:45<386:31:33,  7.91s/it]
 22%|██▏       | 49034/225000 [103:15:49<335:22:14,  6.86s/it]
 22%|██▏       | 49035/225000 [103:15:55<313:29:15,  6.41s/it]
 22%|██▏       | 49036/225000 [103:16:05<373:05:27,  7.63s/it]
 22%|██▏       | 49037/225000 [103:16:13<375:10:42,  7.68s/it]
 22%|██▏       | 49038/225000 [103:16:17<323:53:17,  6.63s/it]
 22%|██▏       | 49039/225000 [103:16:29<396:41:54,  8.12s/it]
 22%|██▏       | 49040/225000 [103:16:33<336:54:48,  6.89s/it]
 22%|██▏       | 49041/225000 [103:16:41<353:58:47,  7.24s/it]
 22%|██▏       | 49042/225000 [103:16:47<338:46:36,  6.93s/it]
 22%|██▏       | 49043/225000 [103:16:56<367:38:00,  7.52s/it]
 22%|██▏       | 49044/225000 [103:17:01<339:34:31,  6.95s/it]
 22%|██▏       | 49045/225000 [103:17:09<343:41:26,  7.03s/it]
 22%|██▏       | 49046/225000 [103:17:13<309:32:13,  6.33s/it]
 22%|██▏       | 49047/225000 [103:17:18<281:49:12,  5.77s/it]
 22%|██▏       | 49048/225000 [103:17:23<272:43:53,  5.58s/it]
 22%|██▏       | 49049/225000 [103:17:34<355:09:05,  7.27s/it]
 22%|██▏       | 49050/225000 [103:17:39<314:35:36,  6.44s/it]
                                                              
{'loss': '0.692', 'grad_norm': '0.7825', 'learning_rate': '4.604e-05', 'epoch': '0.218', 'num_input_tokens_seen': 173269200, 'train_runtime': '3.719e+05', 'train_tokens_per_second': '466'}

 22%|██▏       | 49050/225000 [103:17:39<314:35:36,  6.44s/it]
 22%|██▏       | 49051/225000 [103:17:48<365:10:12,  7.47s/it]
 22%|██▏       | 49052/225000 [103:17:52<307:13:17,  6.29s/it]
 22%|██▏       | 49053/225000 [103:18:02<357:59:07,  7.32s/it]
 22%|██▏       | 49054/225000 [103:18:08<349:30:36,  7.15s/it]
 22%|██▏       | 49055/225000 [103:18:12<294:26:22,  6.02s/it]
 22%|██▏       | 49056/225000 [103:18:21<335:41:02,  6.87s/it]
 22%|██▏       | 49057/225000 [103:18:28<335:50:52,  6.87s/it]
 22%|██▏       | 49058/225000 [103:18:32<304:35:22,  6.23s/it]
 22%|██▏       | 49059/225000 [103:18:37<275:08:01,  5.63s/it]
 22%|██▏       | 49060/225000 [103:18:43<285:38:52,  5.84s/it]
 22%|██▏       | 49061/225000 [103:18:53<349:04:07,  7.14s/it]
 22%|██▏       | 49062/225000 [103:19:01<358:10:04,  7.33s/it]
 22%|██▏       | 49063/225000 [103:19:07<341:36:59,  6.99s/it]
 22%|██▏       | 49064/225000 [103:19:11<299:14:43,  6.12s/it]
 22%|██▏       | 49065/225000 [103:19:16<285:17:07,  5.84s/it]
 22%|██▏       | 49066/225000 [103:19:22<278:18:13,  5.69s/it]
 22%|██▏       | 49067/225000 [103:19:28<286:34:13,  5.86s/it]
 22%|██▏       | 49068/225000 [103:19:41<394:54:11,  8.08s/it]
 22%|██▏       | 49069/225000 [103:19:48<371:43:23,  7.61s/it]
 22%|██▏       | 49070/225000 [103:19:52<319:11:02,  6.53s/it]
 22%|██▏       | 49071/225000 [103:19:56<280:34:59,  5.74s/it]
 22%|██▏       | 49072/225000 [103:20:04<317:53:44,  6.51s/it]
 22%|██▏       | 49073/225000 [103:20:09<304:01:02,  6.22s/it]
 22%|██▏       | 49074/225000 [103:20:14<285:09:50,  5.84s/it]
 22%|██▏       | 49075/225000 [103:20:23<320:34:09,  6.56s/it]
 22%|██▏       | 49076/225000 [103:20:27<293:45:29,  6.01s/it]
 22%|██▏       | 49077/225000 [103:20:33<283:58:00,  5.81s/it]
 22%|██▏       | 49078/225000 [103:20:38<280:25:58,  5.74s/it]
 22%|██▏       | 49079/225000 [103:20:44<278:53:46,  5.71s/it]
 22%|██▏       | 49080/225000 [103:20:50<279:06:49,  5.71s/it]
 22%|██▏       | 49081/225000 [103:21:03<393:51:26,  8.06s/it]
 22%|██▏       | 49082/225000 [103:21:08<348:33:52,  7.13s/it]
 22%|██▏       | 49083/225000 [103:21:15<342:17:48,  7.00s/it]
 22%|██▏       | 49084/225000 [103:21:21<332:11:21,  6.80s/it]
 22%|██▏       | 49085/225000 [103:21:29<352:43:49,  7.22s/it]
 22%|██▏       | 49086/225000 [103:21:37<360:01:28,  7.37s/it]
 22%|██▏       | 49087/225000 [103:21:41<306:46:32,  6.28s/it]
 22%|██▏       | 49088/225000 [103:21:46<291:48:36,  5.97s/it]
 22%|██▏       | 49089/225000 [103:21:55<330:19:56,  6.76s/it]
 22%|██▏       | 49090/225000 [103:21:59<297:43:31,  6.09s/it]
 22%|██▏       | 49091/225000 [103:22:04<281:00:28,  5.75s/it]
 22%|██▏       | 49092/225000 [103:22:10<279:44:09,  5.72s/it]
 22%|██▏       | 49093/225000 [103:22:23<396:03:24,  8.11s/it]
 22%|██▏       | 49094/225000 [103:22:28<337:42:23,  6.91s/it]
 22%|██▏       | 49095/225000 [103:22:34<333:35:30,  6.83s/it]
 22%|██▏       | 49096/225000 [103:22:42<346:14:10,  7.09s/it]
 22%|██▏       | 49097/225000 [103:22:46<299:57:23,  6.14s/it]
 22%|██▏       | 49098/225000 [103:22:56<351:59:33,  7.20s/it]
 22%|██▏       | 49099/225000 [103:23:02<342:59:59,  7.02s/it]
 22%|██▏       | 49100/225000 [103:23:08<332:33:20,  6.81s/it]
                                                              
{'loss': '0.7185', 'grad_norm': '0.852', 'learning_rate': '4.603e-05', 'epoch': '0.2182', 'num_input_tokens_seen': 173415968, 'train_runtime': '3.722e+05', 'train_tokens_per_second': '465.9'}

 22%|██▏       | 49100/225000 [103:23:08<332:33:20,  6.81s/it]
 22%|██▏       | 49101/225000 [103:23:13<306:11:52,  6.27s/it]
 22%|██▏       | 49102/225000 [103:23:18<284:00:47,  5.81s/it]
 22%|██▏       | 49103/225000 [103:23:26<316:52:44,  6.49s/it]
 22%|██▏       | 49104/225000 [103:23:37<380:18:58,  7.78s/it]
 22%|██▏       | 49105/225000 [103:23:41<327:04:03,  6.69s/it]
 22%|██▏       | 49106/225000 [103:23:49<348:04:24,  7.12s/it]
 22%|██▏       | 49107/225000 [103:23:55<329:09:11,  6.74s/it]
 22%|██▏       | 49108/225000 [103:24:03<338:35:37,  6.93s/it]
 22%|██▏       | 49109/225000 [103:24:09<336:00:40,  6.88s/it]
 22%|██▏       | 49110/225000 [103:24:16<328:39:16,  6.73s/it]
 22%|██▏       | 49111/225000 [103:24:20<297:11:27,  6.08s/it]
 22%|██▏       | 49112/225000 [103:24:29<333:24:18,  6.82s/it]
 22%|██▏       | 49113/225000 [103:24:36<334:21:41,  6.84s/it]
 22%|██▏       | 49114/225000 [103:24:44<348:21:29,  7.13s/it]
 22%|██▏       | 49115/225000 [103:24:48<315:15:34,  6.45s/it]
 22%|██▏       | 49116/225000 [103:24:53<290:37:53,  5.95s/it]
 22%|██▏       | 49117/225000 [103:25:02<334:12:27,  6.84s/it]
 22%|██▏       | 49118/225000 [103:25:11<357:56:09,  7.33s/it]
 22%|██▏       | 49119/225000 [103:25:16<327:55:47,  6.71s/it]
 22%|██▏       | 49120/225000 [103:25:21<305:20:37,  6.25s/it]
 22%|██▏       | 49121/225000 [103:25:26<294:06:38,  6.02s/it]
 22%|██▏       | 49122/225000 [103:25:34<317:50:23,  6.51s/it]
 22%|██▏       | 49123/225000 [103:25:44<371:46:24,  7.61s/it]
 22%|██▏       | 49124/225000 [103:25:48<318:59:21,  6.53s/it]
 22%|██▏       | 49125/225000 [103:25:56<338:09:04,  6.92s/it]
 22%|██▏       | 49126/225000 [103:26:05<361:56:17,  7.41s/it]
 22%|██▏       | 49127/225000 [103:26:10<326:30:06,  6.68s/it]
 22%|██▏       | 49128/225000 [103:26:15<302:32:29,  6.19s/it]
 22%|██▏       | 49129/225000 [103:26:22<313:58:16,  6.43s/it]
 22%|██▏       | 49130/225000 [103:26:32<369:28:50,  7.56s/it]
 22%|██▏       | 49131/225000 [103:26:37<335:21:40,  6.86s/it]
 22%|██▏       | 49132/225000 [103:26:42<306:47:01,  6.28s/it]
 22%|██▏       | 49133/225000 [103:26:47<289:38:22,  5.93s/it]
 22%|██▏       | 49134/225000 [103:26:52<275:30:02,  5.64s/it]
 22%|██▏       | 49135/225000 [103:27:05<382:13:28,  7.82s/it]
 22%|██▏       | 49136/225000 [103:27:10<343:47:32,  7.04s/it]
 22%|██▏       | 49137/225000 [103:27:18<359:07:39,  7.35s/it]
 22%|██▏       | 49138/225000 [103:27:22<307:20:46,  6.29s/it]
 22%|██▏       | 49139/225000 [103:27:33<368:29:48,  7.54s/it]
 22%|██▏       | 49140/225000 [103:27:39<345:19:26,  7.07s/it]
 22%|██▏       | 49141/225000 [103:27:48<373:41:04,  7.65s/it]
 22%|██▏       | 49142/225000 [103:27:54<357:29:34,  7.32s/it]
 22%|██▏       | 49143/225000 [103:28:01<345:30:11,  7.07s/it]
 22%|██▏       | 49144/225000 [103:28:06<318:15:06,  6.52s/it]
 22%|██▏       | 49145/225000 [103:28:11<303:42:43,  6.22s/it]
 22%|██▏       | 49146/225000 [103:28:21<356:55:20,  7.31s/it]
 22%|██▏       | 49147/225000 [103:28:25<310:37:18,  6.36s/it]
 22%|██▏       | 49148/225000 [103:28:30<279:07:57,  5.71s/it]
 22%|██▏       | 49149/225000 [103:28:40<350:29:07,  7.18s/it]
 22%|██▏       | 49150/225000 [103:28:45<315:03:20,  6.45s/it]
                                                              
{'loss': '0.6938', 'grad_norm': '0.7914', 'learning_rate': '4.602e-05', 'epoch': '0.2184', 'num_input_tokens_seen': 173565112, 'train_runtime': '3.725e+05', 'train_tokens_per_second': '465.9'}

 22%|██▏       | 49150/225000 [103:28:45<315:03:20,  6.45s/it]
 22%|██▏       | 49151/225000 [103:28:53<344:01:15,  7.04s/it]
 22%|██▏       | 49152/225000 [103:29:01<359:31:27,  7.36s/it]
 22%|██▏       | 49153/225000 [103:29:08<352:00:40,  7.21s/it]
 22%|██▏       | 49154/225000 [103:29:12<299:41:31,  6.14s/it]
 22%|██▏       | 49155/225000 [103:29:20<332:03:20,  6.80s/it]
 22%|██▏       | 49156/225000 [103:29:24<291:08:37,  5.96s/it]
 22%|██▏       | 49157/225000 [103:29:31<308:12:30,  6.31s/it]
 22%|██▏       | 49158/225000 [103:29:40<336:02:00,  6.88s/it]
 22%|██▏       | 49159/225000 [103:29:46<334:02:53,  6.84s/it]
 22%|██▏       | 49160/225000 [103:29:51<299:35:39,  6.13s/it]
 22%|██▏       | 49161/225000 [103:29:56<283:53:05,  5.81s/it]
 22%|██▏       | 49162/225000 [103:30:05<333:11:12,  6.82s/it]
 22%|██▏       | 49163/225000 [103:30:12<327:32:46,  6.71s/it]
 22%|██▏       | 49164/225000 [103:30:16<296:51:44,  6.08s/it]
 22%|██▏       | 49165/225000 [103:30:24<325:56:52,  6.67s/it]
 22%|██▏       | 49166/225000 [103:30:28<290:52:50,  5.96s/it]
 22%|██▏       | 49167/225000 [103:30:33<273:18:56,  5.60s/it]
 22%|██▏       | 49168/225000 [103:30:42<317:41:03,  6.50s/it]
 22%|██▏       | 49169/225000 [103:30:53<384:29:45,  7.87s/it]
 22%|██▏       | 49170/225000 [103:30:57<324:53:41,  6.65s/it]
 22%|██▏       | 49171/225000 [103:31:01<285:18:36,  5.84s/it]
 22%|██▏       | 49172/225000 [103:31:11<355:18:16,  7.27s/it]
 22%|██▏       | 49173/225000 [103:31:17<334:27:38,  6.85s/it]
 22%|██▏       | 49174/225000 [103:31:23<313:40:33,  6.42s/it]
 22%|██▏       | 49175/225000 [103:31:32<359:51:35,  7.37s/it]
 22%|██▏       | 49176/225000 [103:31:36<306:14:44,  6.27s/it]
 22%|██▏       | 49177/225000 [103:31:41<293:05:22,  6.00s/it]
 22%|██▏       | 49178/225000 [103:31:46<275:32:53,  5.64s/it]
 22%|██▏       | 49179/225000 [103:31:58<366:44:43,  7.51s/it]
 22%|██▏       | 49180/225000 [103:32:05<366:54:00,  7.51s/it]
 22%|██▏       | 49181/225000 [103:32:11<331:16:12,  6.78s/it]
 22%|██▏       | 49182/225000 [103:32:19<358:59:40,  7.35s/it]
 22%|██▏       | 49183/225000 [103:32:23<313:15:06,  6.41s/it]
 22%|██▏       | 49184/225000 [103:32:30<315:55:09,  6.47s/it]
 22%|██▏       | 49185/225000 [103:32:35<290:28:27,  5.95s/it]
 22%|██▏       | 49186/225000 [103:32:42<307:03:58,  6.29s/it]
 22%|██▏       | 49187/225000 [103:32:49<315:09:27,  6.45s/it]
 22%|██▏       | 49188/225000 [103:32:55<318:05:38,  6.51s/it]
 22%|██▏       | 49189/225000 [103:33:03<331:55:10,  6.80s/it]
 22%|██▏       | 49190/225000 [103:33:10<344:07:28,  7.05s/it]
 22%|██▏       | 49191/225000 [103:33:16<329:01:10,  6.74s/it]
 22%|██▏       | 49192/225000 [103:33:23<333:55:33,  6.84s/it]
 22%|██▏       | 49193/225000 [103:33:31<344:09:28,  7.05s/it]
 22%|██▏       | 49194/225000 [103:33:35<302:42:05,  6.20s/it]
 22%|██▏       | 49195/225000 [103:33:40<281:09:00,  5.76s/it]
 22%|██▏       | 49196/225000 [103:33:45<264:38:19,  5.42s/it]
 22%|██▏       | 49197/225000 [103:33:52<295:58:00,  6.06s/it]
 22%|██▏       | 49198/225000 [103:34:03<363:24:54,  7.44s/it]
 22%|██▏       | 49199/225000 [103:34:09<338:57:03,  6.94s/it]
 22%|██▏       | 49200/225000 [103:34:16<352:19:40,  7.21s/it]
                                                              
{'loss': '0.7391', 'grad_norm': '0.7783', 'learning_rate': '4.601e-05', 'epoch': '0.2187', 'num_input_tokens_seen': 173712592, 'train_runtime': '3.729e+05', 'train_tokens_per_second': '465.9'}

 22%|██▏       | 49200/225000 [103:34:16<352:19:40,  7.21s/it]
 22%|██▏       | 49201/225000 [103:34:21<309:51:32,  6.35s/it]
 22%|██▏       | 49202/225000 [103:34:27<305:25:53,  6.25s/it]
 22%|██▏       | 49203/225000 [103:34:38<374:46:10,  7.67s/it]
 22%|██▏       | 49204/225000 [103:34:42<319:47:04,  6.55s/it]
 22%|██▏       | 49205/225000 [103:34:52<374:51:43,  7.68s/it]
 22%|██▏       | 49206/225000 [103:34:58<345:13:47,  7.07s/it]
 22%|██▏       | 49207/225000 [103:35:06<358:11:42,  7.34s/it]
 22%|██▏       | 49208/225000 [103:35:10<317:19:45,  6.50s/it]
 22%|██▏       | 49209/225000 [103:35:15<290:23:41,  5.95s/it]
 22%|██▏       | 49210/225000 [103:35:23<319:35:44,  6.54s/it]
 22%|██▏       | 49211/225000 [103:35:31<349:59:34,  7.17s/it]
 22%|██▏       | 49212/225000 [103:35:38<334:44:34,  6.86s/it]
 22%|██▏       | 49213/225000 [103:35:46<354:21:06,  7.26s/it]
 22%|██▏       | 49214/225000 [103:35:50<310:58:47,  6.37s/it]
 22%|██▏       | 49215/225000 [103:35:54<283:02:29,  5.80s/it]
 22%|██▏       | 49216/225000 [103:36:06<369:00:51,  7.56s/it]
 22%|██▏       | 49217/225000 [103:36:09<306:19:42,  6.27s/it]
 22%|██▏       | 49218/225000 [103:36:13<269:08:17,  5.51s/it]
 22%|██▏       | 49219/225000 [103:36:26<371:09:28,  7.60s/it]
 22%|██▏       | 49220/225000 [103:36:33<363:57:33,  7.45s/it]
 22%|██▏       | 49221/225000 [103:36:38<333:08:57,  6.82s/it]
 22%|██▏       | 49222/225000 [103:36:43<298:39:25,  6.12s/it]
 22%|██▏       | 49223/225000 [103:36:48<282:24:50,  5.78s/it]
 22%|██▏       | 49224/225000 [103:36:54<297:37:53,  6.10s/it]
 22%|██▏       | 49225/225000 [103:37:07<395:47:13,  8.11s/it]
 22%|██▏       | 49226/225000 [103:37:11<336:59:49,  6.90s/it]
 22%|██▏       | 49227/225000 [103:37:21<376:07:31,  7.70s/it]
 22%|██▏       | 49228/225000 [103:37:25<330:57:49,  6.78s/it]
 22%|██▏       | 49229/225000 [103:37:30<297:55:11,  6.10s/it]
 22%|██▏       | 49230/225000 [103:37:35<282:47:42,  5.79s/it]
 22%|██▏       | 49231/225000 [103:37:46<361:10:09,  7.40s/it]
 22%|██▏       | 49232/225000 [103:37:51<317:11:31,  6.50s/it]
 22%|██▏       | 49233/225000 [103:37:56<305:15:51,  6.25s/it]
 22%|██▏       | 49234/225000 [103:38:02<298:28:46,  6.11s/it]
 22%|██▏       | 49235/225000 [103:38:11<332:40:32,  6.81s/it]
 22%|██▏       | 49236/225000 [103:38:20<371:24:50,  7.61s/it]
 22%|██▏       | 49237/225000 [103:38:27<361:41:00,  7.41s/it]
 22%|██▏       | 49238/225000 [103:38:31<314:31:39,  6.44s/it]
 22%|██▏       | 49239/225000 [103:38:37<300:28:44,  6.15s/it]
 22%|██▏       | 49240/225000 [103:38:42<284:32:46,  5.83s/it]
 22%|██▏       | 49241/225000 [103:38:47<271:01:15,  5.55s/it]
 22%|██▏       | 49242/225000 [103:38:54<296:44:19,  6.08s/it]
 22%|██▏       | 49243/225000 [103:39:08<414:55:41,  8.50s/it]
 22%|██▏       | 49244/225000 [103:39:12<351:43:53,  7.20s/it]
 22%|██▏       | 49245/225000 [103:39:22<389:30:59,  7.98s/it]
 22%|██▏       | 49246/225000 [103:39:27<347:39:54,  7.12s/it]
 22%|██▏       | 49247/225000 [103:39:35<356:55:40,  7.31s/it]
 22%|██▏       | 49248/225000 [103:39:42<346:54:35,  7.11s/it]
 22%|██▏       | 49249/225000 [103:39:46<312:21:28,  6.40s/it]
 22%|██▏       | 49250/225000 [103:39:56<355:20:12,  7.28s/it]
                                                              
{'loss': '0.7144', 'grad_norm': '0.7798', 'learning_rate': '4.6e-05', 'epoch': '0.2189', 'num_input_tokens_seen': 173861512, 'train_runtime': '3.732e+05', 'train_tokens_per_second': '465.9'}

 22%|██▏       | 49250/225000 [103:39:56<355:20:12,  7.28s/it]
 22%|██▏       | 49251/225000 [103:40:02<343:14:03,  7.03s/it]
 22%|██▏       | 49252/225000 [103:40:08<321:29:48,  6.59s/it]
 22%|██▏       | 49253/225000 [103:40:13<299:32:27,  6.14s/it]
 22%|██▏       | 49254/225000 [103:40:20<320:03:52,  6.56s/it]
 22%|██▏       | 49255/225000 [103:40:27<323:39:16,  6.63s/it]
 22%|██▏       | 49256/225000 [103:40:35<347:08:25,  7.11s/it]
 22%|██▏       | 49257/225000 [103:40:39<303:20:29,  6.21s/it]
 22%|██▏       | 49258/225000 [103:40:44<278:10:12,  5.70s/it]
 22%|██▏       | 49259/225000 [103:40:48<256:04:40,  5.25s/it]
 22%|██▏       | 49260/225000 [103:41:01<364:02:39,  7.46s/it]
 22%|██▏       | 49261/225000 [103:41:05<321:55:30,  6.59s/it]
 22%|██▏       | 49262/225000 [103:41:09<284:54:03,  5.84s/it]
 22%|██▏       | 49263/225000 [103:41:18<321:59:02,  6.60s/it]
 22%|██▏       | 49264/225000 [103:41:25<325:29:23,  6.67s/it]
 22%|██▏       | 49265/225000 [103:41:31<330:00:01,  6.76s/it]
 22%|██▏       | 49266/225000 [103:41:39<335:52:58,  6.88s/it]
 22%|██▏       | 49267/225000 [103:41:47<350:25:04,  7.18s/it]
 22%|██▏       | 49268/225000 [103:41:51<311:18:19,  6.38s/it]
 22%|██▏       | 49269/225000 [103:41:56<286:09:26,  5.86s/it]
 22%|██▏       | 49270/225000 [103:42:08<374:11:24,  7.67s/it]
 22%|██▏       | 49271/225000 [103:42:12<321:04:48,  6.58s/it]
 22%|██▏       | 49272/225000 [103:42:21<359:29:42,  7.36s/it]
 22%|██▏       | 49273/225000 [103:42:25<307:31:58,  6.30s/it]
 22%|██▏       | 49274/225000 [103:42:33<338:22:48,  6.93s/it]
 22%|██▏       | 49275/225000 [103:42:38<305:06:04,  6.25s/it]
 22%|██▏       | 49276/225000 [103:42:42<280:22:00,  5.74s/it]
 22%|██▏       | 49277/225000 [103:42:47<264:41:51,  5.42s/it]
 22%|██▏       | 49278/225000 [103:42:59<358:18:15,  7.34s/it]
 22%|██▏       | 49279/225000 [103:43:05<341:39:09,  7.00s/it]
 22%|██▏       | 49280/225000 [103:43:10<314:01:39,  6.43s/it]
 22%|██▏       | 49281/225000 [103:43:15<292:27:59,  5.99s/it]
 22%|██▏       | 49282/225000 [103:43:20<276:36:31,  5.67s/it]
 22%|██▏       | 49283/225000 [103:43:31<351:15:45,  7.20s/it]
 22%|██▏       | 49284/225000 [103:43:38<351:58:41,  7.21s/it]
 22%|██▏       | 49285/225000 [103:43:43<323:25:10,  6.63s/it]
 22%|██▏       | 49286/225000 [103:43:48<302:13:16,  6.19s/it]
 22%|██▏       | 49287/225000 [103:43:53<285:29:04,  5.85s/it]
 22%|██▏       | 49288/225000 [103:43:58<272:35:05,  5.58s/it]
 22%|██▏       | 49289/225000 [103:44:11<376:45:52,  7.72s/it]
 22%|██▏       | 49290/225000 [103:44:17<350:14:58,  7.18s/it]
 22%|██▏       | 49291/225000 [103:44:22<322:17:11,  6.60s/it]
 22%|██▏       | 49292/225000 [103:44:27<297:48:14,  6.10s/it]
 22%|██▏       | 49293/225000 [103:44:35<320:28:08,  6.57s/it]
 22%|██▏       | 49294/225000 [103:44:47<395:27:36,  8.10s/it]
 22%|██▏       | 49295/225000 [103:44:50<332:24:34,  6.81s/it]
 22%|██▏       | 49296/225000 [103:44:59<366:46:04,  7.51s/it]
 22%|██▏       | 49297/225000 [103:45:07<365:31:48,  7.49s/it]
 22%|██▏       | 49298/225000 [103:45:11<308:23:07,  6.32s/it]
 22%|██▏       | 49299/225000 [103:45:21<365:51:28,  7.50s/it]
 22%|██▏       | 49300/225000 [103:45:26<328:22:21,  6.73s/it]
                                                              
{'loss': '0.7083', 'grad_norm': '0.729', 'learning_rate': '4.599e-05', 'epoch': '0.2191', 'num_input_tokens_seen': 174009248, 'train_runtime': '3.735e+05', 'train_tokens_per_second': '465.9'}

 22%|██▏       | 49300/225000 [103:45:26<328:22:21,  6.73s/it]
 22%|██▏       | 49301/225000 [103:45:30<293:56:16,  6.02s/it]
 22%|██▏       | 49302/225000 [103:45:40<353:56:05,  7.25s/it]
 22%|██▏       | 49303/225000 [103:45:45<313:33:06,  6.42s/it]
 22%|██▏       | 49304/225000 [103:45:49<285:36:25,  5.85s/it]
 22%|██▏       | 49305/225000 [103:45:54<264:08:20,  5.41s/it]
 22%|██▏       | 49306/225000 [103:46:03<316:52:36,  6.49s/it]
 22%|██▏       | 49307/225000 [103:46:11<345:02:02,  7.07s/it]
 22%|██▏       | 49308/225000 [103:46:15<298:12:06,  6.11s/it]
 22%|██▏       | 49309/225000 [103:46:27<379:54:59,  7.78s/it]
 22%|██▏       | 49310/225000 [103:46:33<360:19:21,  7.38s/it]
 22%|██▏       | 49311/225000 [103:46:37<311:56:55,  6.39s/it]
 22%|██▏       | 49312/225000 [103:46:41<276:27:23,  5.66s/it]
 22%|██▏       | 49313/225000 [103:46:47<276:03:38,  5.66s/it]
 22%|██▏       | 49314/225000 [103:46:59<377:32:08,  7.74s/it]
 22%|██▏       | 49315/225000 [103:47:03<316:50:47,  6.49s/it]
 22%|██▏       | 49316/225000 [103:47:10<327:29:14,  6.71s/it]
 22%|██▏       | 49317/225000 [103:47:14<291:33:50,  5.97s/it]
 22%|██▏       | 49318/225000 [103:47:19<276:04:15,  5.66s/it]
 22%|██▏       | 49319/225000 [103:47:31<366:31:03,  7.51s/it]
 22%|██▏       | 49320/225000 [103:47:36<333:16:38,  6.83s/it]
 22%|██▏       | 49321/225000 [103:47:42<317:11:48,  6.50s/it]
 22%|██▏       | 49322/225000 [103:47:48<302:03:02,  6.19s/it]
 22%|██▏       | 49323/225000 [103:47:59<376:57:53,  7.72s/it]
 22%|██▏       | 49324/225000 [103:48:02<316:25:49,  6.48s/it]
 22%|██▏       | 49325/225000 [103:48:12<363:32:01,  7.45s/it]
 22%|██▏       | 49326/225000 [103:48:18<343:46:28,  7.04s/it]
 22%|██▏       | 49327/225000 [103:48:25<342:44:06,  7.02s/it]
 22%|██▏       | 49328/225000 [103:48:30<302:38:39,  6.20s/it]
 22%|██▏       | 49329/225000 [103:48:34<284:36:49,  5.83s/it]
 22%|██▏       | 49330/225000 [103:48:39<272:33:57,  5.59s/it]
 22%|██▏       | 49331/225000 [103:48:45<269:29:24,  5.52s/it]
 22%|██▏       | 49332/225000 [103:48:55<329:42:31,  6.76s/it]
 22%|██▏       | 49333/225000 [103:49:04<371:30:51,  7.61s/it]
 22%|██▏       | 49334/225000 [103:49:11<367:59:28,  7.54s/it]
 22%|██▏       | 49335/225000 [103:49:16<317:06:07,  6.50s/it]
 22%|██▏       | 49336/225000 [103:49:24<341:03:16,  6.99s/it]
 22%|██▏       | 49337/225000 [103:49:29<318:05:24,  6.52s/it]
 22%|██▏       | 49338/225000 [103:49:34<288:20:07,  5.91s/it]
 22%|██▏       | 49339/225000 [103:49:42<326:31:59,  6.69s/it]
 22%|██▏       | 49340/225000 [103:49:50<348:01:45,  7.13s/it]
 22%|██▏       | 49341/225000 [103:49:56<320:36:39,  6.57s/it]
 22%|██▏       | 49342/225000 [103:50:00<293:06:57,  6.01s/it]
 22%|██▏       | 49343/225000 [103:50:05<277:11:47,  5.68s/it]
 22%|██▏       | 49344/225000 [103:50:11<285:06:29,  5.84s/it]
 22%|██▏       | 49345/225000 [103:50:25<394:26:50,  8.08s/it]
 22%|██▏       | 49346/225000 [103:50:28<325:51:31,  6.68s/it]
 22%|██▏       | 49347/225000 [103:50:37<359:45:00,  7.37s/it]
 22%|██▏       | 49348/225000 [103:50:42<329:48:20,  6.76s/it]
 22%|██▏       | 49349/225000 [103:50:48<306:48:44,  6.29s/it]
 22%|██▏       | 49350/225000 [103:50:56<330:42:24,  6.78s/it]
                                                              
{'loss': '0.6814', 'grad_norm': '0.7651', 'learning_rate': '4.598e-05', 'epoch': '0.2193', 'num_input_tokens_seen': 174155936, 'train_runtime': '3.739e+05', 'train_tokens_per_second': '465.8'}

 22%|██▏       | 49350/225000 [103:50:56<330:42:24,  6.78s/it]
 22%|██▏       | 49351/225000 [103:51:00<298:14:42,  6.11s/it]
 22%|██▏       | 49352/225000 [103:51:05<285:09:06,  5.84s/it]
 22%|██▏       | 49353/225000 [103:51:10<274:11:45,  5.62s/it]
 22%|██▏       | 49354/225000 [103:51:19<320:40:52,  6.57s/it]
 22%|██▏       | 49355/225000 [103:51:30<385:03:56,  7.89s/it]
 22%|██▏       | 49356/225000 [103:51:34<324:43:37,  6.66s/it]
 22%|██▏       | 49357/225000 [103:51:44<370:39:43,  7.60s/it]
 22%|██▏       | 49358/225000 [103:51:48<323:11:41,  6.62s/it]
 22%|██▏       | 49359/225000 [103:51:56<337:58:57,  6.93s/it]
 22%|██▏       | 49360/225000 [103:52:02<326:17:38,  6.69s/it]
 22%|██▏       | 49361/225000 [103:52:07<301:23:31,  6.18s/it]
 22%|██▏       | 49362/225000 [103:52:12<288:39:41,  5.92s/it]
 22%|██▏       | 49363/225000 [103:52:18<282:03:15,  5.78s/it]
 22%|██▏       | 49364/225000 [103:52:22<269:09:00,  5.52s/it]
 22%|██▏       | 49365/225000 [103:52:29<287:54:25,  5.90s/it]
 22%|██▏       | 49366/225000 [103:52:42<390:49:51,  8.01s/it]
 22%|██▏       | 49367/225000 [103:52:47<339:22:56,  6.96s/it]
 22%|██▏       | 49368/225000 [103:52:51<304:31:36,  6.24s/it]
 22%|██▏       | 49369/225000 [103:52:59<324:37:39,  6.65s/it]
 22%|██▏       | 49370/225000 [103:53:06<337:44:16,  6.92s/it]
 22%|██▏       | 49371/225000 [103:53:16<372:34:26,  7.64s/it]
 22%|██▏       | 49372/225000 [103:53:20<317:16:39,  6.50s/it]
 22%|██▏       | 49373/225000 [103:53:27<326:08:53,  6.69s/it]
 22%|██▏       | 49374/225000 [103:53:34<338:31:03,  6.94s/it]
 22%|██▏       | 49375/225000 [103:53:43<363:39:20,  7.45s/it]
 22%|██▏       | 49376/225000 [103:53:48<321:50:57,  6.60s/it]
 22%|██▏       | 49377/225000 [103:53:53<306:49:57,  6.29s/it]
 22%|██▏       | 49378/225000 [103:53:59<293:55:04,  6.02s/it]
 22%|██▏       | 49379/225000 [103:54:04<285:13:38,  5.85s/it]
 22%|██▏       | 49380/225000 [103:54:10<282:26:07,  5.79s/it]
 22%|██▏       | 49381/225000 [103:54:16<285:01:56,  5.84s/it]
 22%|██▏       | 49382/225000 [103:54:21<272:56:13,  5.59s/it]
 22%|██▏       | 49383/225000 [103:54:34<384:37:25,  7.88s/it]
 22%|██▏       | 49384/225000 [103:54:39<339:06:38,  6.95s/it]
 22%|██▏       | 49385/225000 [103:54:47<357:52:23,  7.34s/it]
 22%|██▏       | 49386/225000 [103:54:51<314:50:12,  6.45s/it]
 22%|██▏       | 49387/225000 [103:54:56<283:23:07,  5.81s/it]
 22%|██▏       | 49388/225000 [103:55:03<302:35:54,  6.20s/it]
 22%|██▏       | 49389/225000 [103:55:16<405:05:44,  8.30s/it]
 22%|██▏       | 49390/225000 [103:55:20<342:26:35,  7.02s/it]
 22%|██▏       | 49391/225000 [103:55:28<363:34:42,  7.45s/it]
 22%|██▏       | 49392/225000 [103:55:33<327:00:13,  6.70s/it]
 22%|██▏       | 49393/225000 [103:55:39<306:38:52,  6.29s/it]
 22%|██▏       | 49394/225000 [103:55:44<292:19:41,  5.99s/it]
 22%|██▏       | 49395/225000 [103:55:57<396:18:53,  8.12s/it]
 22%|██▏       | 49396/225000 [103:56:01<331:46:28,  6.80s/it]
 22%|██▏       | 49397/225000 [103:56:10<373:51:40,  7.66s/it]
 22%|██▏       | 49398/225000 [103:56:14<318:16:19,  6.52s/it]
 22%|██▏       | 49399/225000 [103:56:20<310:34:10,  6.37s/it]
 22%|██▏       | 49400/225000 [103:56:29<344:35:26,  7.06s/it]
                                                              
{'loss': '0.7008', 'grad_norm': '0.6471', 'learning_rate': '4.598e-05', 'epoch': '0.2196', 'num_input_tokens_seen': 174304192, 'train_runtime': '3.742e+05', 'train_tokens_per_second': '465.8'}

 22%|██▏       | 49400/225000 [103:56:29<344:35:26,  7.06s/it]
 22%|██▏       | 49401/225000 [103:56:35<327:50:15,  6.72s/it]
 22%|██▏       | 49402/225000 [103:56:42<340:23:33,  6.98s/it]
 22%|██▏       | 49403/225000 [103:56:50<346:14:34,  7.10s/it]
 22%|██▏       | 49404/225000 [103:56:55<318:44:00,  6.53s/it]
 22%|██▏       | 49405/225000 [103:57:00<294:51:11,  6.05s/it]
 22%|██▏       | 49406/225000 [103:57:08<319:59:39,  6.56s/it]
 22%|██▏       | 49407/225000 [103:57:19<387:02:42,  7.94s/it]
 22%|██▏       | 49408/225000 [103:57:23<325:11:35,  6.67s/it]
 22%|██▏       | 49409/225000 [103:57:27<286:01:15,  5.86s/it]
 22%|██▏       | 49410/225000 [103:57:32<279:59:23,  5.74s/it]
 22%|██▏       | 49411/225000 [103:57:37<274:46:53,  5.63s/it]
 22%|██▏       | 49412/225000 [103:57:51<388:28:59,  7.96s/it]
 22%|██▏       | 49413/225000 [103:57:57<362:40:05,  7.44s/it]
 22%|██▏       | 49414/225000 [103:58:05<376:14:17,  7.71s/it]
 22%|██▏       | 49415/225000 [103:58:11<345:03:05,  7.07s/it]
 22%|██▏       | 49416/225000 [103:58:20<369:29:51,  7.58s/it]
 22%|██▏       | 49417/225000 [103:58:25<333:55:51,  6.85s/it]
 22%|██▏       | 49418/225000 [103:58:30<303:19:18,  6.22s/it]
 22%|██▏       | 49419/225000 [103:58:35<291:32:54,  5.98s/it]
 22%|██▏       | 49420/225000 [103:58:48<393:18:59,  8.06s/it]
 22%|██▏       | 49421/225000 [103:58:53<354:48:41,  7.27s/it]
 22%|██▏       | 49422/225000 [103:58:58<312:13:18,  6.40s/it]
 22%|██▏       | 49423/225000 [103:59:06<338:01:36,  6.93s/it]
 22%|██▏       | 49424/225000 [103:59:12<332:31:54,  6.82s/it]
 22%|██▏       | 49425/225000 [103:59:18<308:00:25,  6.32s/it]
 22%|██▏       | 49426/225000 [103:59:23<295:18:52,  6.06s/it]
 22%|██▏       | 49427/225000 [103:59:28<282:22:05,  5.79s/it]
 22%|██▏       | 49428/225000 [103:59:34<279:31:28,  5.73s/it]
 22%|██▏       | 49429/225000 [103:59:38<262:34:27,  5.38s/it]
 22%|██▏       | 49430/225000 [103:59:44<271:46:58,  5.57s/it]
 22%|██▏       | 49431/225000 [103:59:56<362:02:35,  7.42s/it]
 22%|██▏       | 49432/225000 [104:00:04<366:36:30,  7.52s/it]
 22%|██▏       | 49433/225000 [104:00:11<356:47:50,  7.32s/it]
 22%|██▏       | 49434/225000 [104:00:20<389:18:21,  7.98s/it]
 22%|██▏       | 49435/225000 [104:00:27<368:25:33,  7.55s/it]
 22%|██▏       | 49436/225000 [104:00:31<312:35:32,  6.41s/it]
 22%|██▏       | 49437/225000 [104:00:43<394:31:43,  8.09s/it]
 22%|██▏       | 49438/225000 [104:00:46<331:35:27,  6.80s/it]
 22%|██▏       | 49439/225000 [104:00:55<362:14:14,  7.43s/it]
 22%|██▏       | 49440/225000 [104:01:00<325:15:32,  6.67s/it]
 22%|██▏       | 49441/225000 [104:01:05<304:32:21,  6.24s/it]
 22%|██▏       | 49442/225000 [104:01:11<292:51:35,  6.01s/it]
 22%|██▏       | 49443/225000 [104:01:16<283:55:36,  5.82s/it]
 22%|██▏       | 49444/225000 [104:01:22<277:19:37,  5.69s/it]
 22%|██▏       | 49445/225000 [104:01:34<376:28:54,  7.72s/it]
 22%|██▏       | 49446/225000 [104:01:42<378:29:35,  7.76s/it]
 22%|██▏       | 49447/225000 [104:01:51<395:35:38,  8.11s/it]
 22%|██▏       | 49448/225000 [104:01:55<335:01:44,  6.87s/it]
 22%|██▏       | 49449/225000 [104:02:00<306:48:32,  6.29s/it]
 22%|██▏       | 49450/225000 [104:02:11<373:44:54,  7.66s/it]
                                                              
{'loss': '0.6929', 'grad_norm': '0.7973', 'learning_rate': '4.597e-05', 'epoch': '0.2198', 'num_input_tokens_seen': 174454680, 'train_runtime': '3.745e+05', 'train_tokens_per_second': '465.8'}

 22%|██▏       | 49450/225000 [104:02:11<373:44:54,  7.66s/it]
 22%|██▏       | 49451/225000 [104:02:15<328:53:34,  6.74s/it]
 22%|██▏       | 49452/225000 [104:02:21<308:16:33,  6.32s/it]
 22%|██▏       | 49453/225000 [104:02:26<289:54:02,  5.95s/it]
 22%|██▏       | 49454/225000 [104:02:31<281:52:37,  5.78s/it]
 22%|██▏       | 49455/225000 [104:02:36<275:43:49,  5.65s/it]
 22%|██▏       | 49456/225000 [104:02:43<282:24:28,  5.79s/it]
 22%|██▏       | 49457/225000 [104:02:50<308:51:58,  6.33s/it]
 22%|██▏       | 49458/225000 [104:02:58<328:17:39,  6.73s/it]
 22%|██▏       | 49459/225000 [104:03:03<311:27:51,  6.39s/it]
 22%|██▏       | 49460/225000 [104:03:08<291:49:53,  5.98s/it]
 22%|██▏       | 49461/225000 [104:03:17<331:06:20,  6.79s/it]
 22%|██▏       | 49462/225000 [104:03:29<407:49:42,  8.36s/it]
 22%|██▏       | 49463/225000 [104:03:33<340:57:37,  6.99s/it]
 22%|██▏       | 49464/225000 [104:03:42<378:11:14,  7.76s/it]
 22%|██▏       | 49465/225000 [104:03:47<334:38:16,  6.86s/it]
 22%|██▏       | 49466/225000 [104:03:52<302:42:00,  6.21s/it]
 22%|██▏       | 49467/225000 [104:03:56<276:52:54,  5.68s/it]
 22%|██▏       | 49468/225000 [104:04:06<341:36:01,  7.01s/it]
 22%|██▏       | 49469/225000 [104:04:15<362:34:35,  7.44s/it]
 22%|██▏       | 49470/225000 [104:04:20<322:12:54,  6.61s/it]
 22%|██▏       | 49471/225000 [104:04:26<324:31:23,  6.66s/it]
 22%|██▏       | 49472/225000 [104:04:35<355:03:16,  7.28s/it]
 22%|██▏       | 49473/225000 [104:04:43<360:53:48,  7.40s/it]
 22%|██▏       | 49474/225000 [104:04:48<323:15:58,  6.63s/it]
 22%|██▏       | 49475/225000 [104:04:55<330:25:03,  6.78s/it]
 22%|██▏       | 49476/225000 [104:04:59<301:09:22,  6.18s/it]
 22%|██▏       | 49477/225000 [104:05:04<280:04:43,  5.74s/it]
 22%|██▏       | 49478/225000 [104:05:09<264:36:20,  5.43s/it]
 22%|██▏       | 49479/225000 [104:05:22<370:51:19,  7.61s/it]
 22%|██▏       | 49480/225000 [104:05:27<335:59:41,  6.89s/it]
 22%|██▏       | 49481/225000 [104:05:33<324:27:16,  6.65s/it]
 22%|██▏       | 49482/225000 [104:05:41<339:05:35,  6.96s/it]
 22%|██▏       | 49483/225000 [104:05:46<311:45:03,  6.39s/it]
 22%|██▏       | 49484/225000 [104:05:50<288:54:51,  5.93s/it]
 22%|██▏       | 49485/225000 [104:05:59<325:57:39,  6.69s/it]
 22%|██▏       | 49486/225000 [104:06:07<350:55:58,  7.20s/it]
 22%|██▏       | 49487/225000 [104:06:16<378:20:21,  7.76s/it]
 22%|██▏       | 49488/225000 [104:06:20<324:05:35,  6.65s/it]
 22%|██▏       | 49489/225000 [104:06:29<356:35:21,  7.31s/it]
 22%|██▏       | 49490/225000 [104:06:38<380:19:14,  7.80s/it]
 22%|██▏       | 49491/225000 [104:06:42<326:33:20,  6.70s/it]
 22%|██▏       | 49492/225000 [104:06:50<340:25:04,  6.98s/it]
 22%|██▏       | 49493/225000 [104:06:57<340:51:36,  6.99s/it]
 22%|██▏       | 49494/225000 [104:07:04<337:52:45,  6.93s/it]
 22%|██▏       | 49495/225000 [104:07:08<302:50:45,  6.21s/it]
 22%|██▏       | 49496/225000 [104:07:13<284:43:29,  5.84s/it]
 22%|██▏       | 49497/225000 [104:07:18<269:39:34,  5.53s/it]
 22%|██▏       | 49498/225000 [104:07:31<380:59:58,  7.82s/it]
 22%|██▏       | 49499/225000 [104:07:36<336:46:47,  6.91s/it]
 22%|██▏       | 49500/225000 [104:07:42<319:22:31,  6.55s/it]
                                                              
{'loss': '0.7101', 'grad_norm': '0.704', 'learning_rate': '4.596e-05', 'epoch': '0.22', 'num_input_tokens_seen': 174602720, 'train_runtime': '3.749e+05', 'train_tokens_per_second': '465.8'}

 22%|██▏       | 49500/225000 [104:07:42<319:22:31,  6.55s/it]
 22%|██▏       | 49501/225000 [104:07:48<312:36:55,  6.41s/it]
 22%|██▏       | 49502/225000 [104:07:53<300:16:49,  6.16s/it]
 22%|██▏       | 49503/225000 [104:07:59<289:14:19,  5.93s/it]
 22%|██▏       | 49504/225000 [104:08:04<283:05:54,  5.81s/it]
 22%|██▏       | 49505/225000 [104:08:10<275:24:03,  5.65s/it]
 22%|██▏       | 49506/225000 [104:08:21<354:07:52,  7.26s/it]
 22%|██▏       | 49507/225000 [104:08:29<371:39:47,  7.62s/it]
 22%|██▏       | 49508/225000 [104:08:34<327:42:13,  6.72s/it]
 22%|██▏       | 49509/225000 [104:08:39<301:32:21,  6.19s/it]
 22%|██▏       | 49510/225000 [104:08:43<276:44:10,  5.68s/it]
 22%|██▏       | 49511/225000 [104:08:54<354:29:25,  7.27s/it]
 22%|██▏       | 49512/225000 [104:09:03<376:08:10,  7.72s/it]
 22%|██▏       | 49513/225000 [104:09:07<322:07:14,  6.61s/it]
 22%|██▏       | 49514/225000 [104:09:17<374:26:24,  7.68s/it]
 22%|██▏       | 49515/225000 [104:09:22<338:16:36,  6.94s/it]
 22%|██▏       | 49516/225000 [104:09:31<357:45:43,  7.34s/it]
 22%|██▏       | 49517/225000 [104:09:34<306:38:33,  6.29s/it]
 22%|██▏       | 49518/225000 [104:09:45<366:38:45,  7.52s/it]
 22%|██▏       | 49519/225000 [104:09:49<317:21:51,  6.51s/it]
 22%|██▏       | 49520/225000 [104:09:59<374:52:13,  7.69s/it]
 22%|██▏       | 49521/225000 [104:10:04<335:09:20,  6.88s/it]
 22%|██▏       | 49522/225000 [104:10:09<303:46:14,  6.23s/it]
 22%|██▏       | 49523/225000 [104:10:14<280:36:05,  5.76s/it]
 22%|██▏       | 49524/225000 [104:10:18<264:00:41,  5.42s/it]
 22%|██▏       | 49525/225000 [104:10:27<304:32:28,  6.25s/it]
 22%|██▏       | 49526/225000 [104:10:38<375:41:21,  7.71s/it]
 22%|██▏       | 49527/225000 [104:10:42<328:46:38,  6.75s/it]
 22%|██▏       | 49528/225000 [104:10:47<294:28:14,  6.04s/it]
 22%|██▏       | 49529/225000 [104:10:56<348:31:47,  7.15s/it]
 22%|██▏       | 49530/225000 [104:11:03<339:19:08,  6.96s/it]
 22%|██▏       | 49531/225000 [104:11:07<298:58:10,  6.13s/it]
 22%|██▏       | 49532/225000 [104:11:18<373:59:21,  7.67s/it]
 22%|██▏       | 49533/225000 [104:11:23<329:28:18,  6.76s/it]
 22%|██▏       | 49534/225000 [104:11:27<295:37:13,  6.07s/it]
 22%|██▏       | 49535/225000 [104:11:36<331:44:37,  6.81s/it]
 22%|██▏       | 49536/225000 [104:11:45<359:32:37,  7.38s/it]
 22%|██▏       | 49537/225000 [104:11:49<319:55:24,  6.56s/it]
 22%|██▏       | 49538/225000 [104:11:58<356:33:55,  7.32s/it]
 22%|██▏       | 49539/225000 [104:12:02<304:36:46,  6.25s/it]
 22%|██▏       | 49540/225000 [104:12:07<276:59:52,  5.68s/it]
 22%|██▏       | 49541/225000 [104:12:11<256:20:58,  5.26s/it]
 22%|██▏       | 49542/225000 [104:12:22<345:21:45,  7.09s/it]
 22%|██▏       | 49543/225000 [104:12:30<361:09:05,  7.41s/it]
 22%|██▏       | 49544/225000 [104:12:34<303:28:54,  6.23s/it]
 22%|██▏       | 49545/225000 [104:12:39<287:35:39,  5.90s/it]
 22%|██▏       | 49546/225000 [104:12:49<355:24:46,  7.29s/it]
 22%|██▏       | 49547/225000 [104:12:54<309:05:10,  6.34s/it]
 22%|██▏       | 49548/225000 [104:12:59<287:47:57,  5.91s/it]
 22%|██▏       | 49549/225000 [104:13:10<362:21:29,  7.44s/it]
 22%|██▏       | 49550/225000 [104:13:17<361:02:54,  7.41s/it]
                                                              
{'loss': '0.7144', 'grad_norm': '0.6716', 'learning_rate': '4.595e-05', 'epoch': '0.2202', 'num_input_tokens_seen': 174750768, 'train_runtime': '3.752e+05', 'train_tokens_per_second': '465.8'}

 22%|██▏       | 49550/225000 [104:13:17<361:02:54,  7.41s/it]
 22%|██▏       | 49551/225000 [104:13:22<333:52:00,  6.85s/it]
 22%|██▏       | 49552/225000 [104:13:28<314:47:56,  6.46s/it]
 22%|██▏       | 49553/225000 [104:13:34<306:22:28,  6.29s/it]
 22%|██▏       | 49554/225000 [104:13:39<288:36:51,  5.92s/it]
 22%|██▏       | 49555/225000 [104:13:44<278:39:22,  5.72s/it]
 22%|██▏       | 49556/225000 [104:13:50<276:11:43,  5.67s/it]
 22%|██▏       | 49557/225000 [104:13:57<296:40:40,  6.09s/it]
 22%|██▏       | 49558/225000 [104:14:10<398:17:19,  8.17s/it]
 22%|██▏       | 49559/225000 [104:14:15<355:00:10,  7.28s/it]
 22%|██▏       | 49560/225000 [104:14:23<370:51:09,  7.61s/it]
 22%|██▏       | 49561/225000 [104:14:30<363:08:51,  7.45s/it]
 22%|██▏       | 49562/225000 [104:14:34<305:39:30,  6.27s/it]
 22%|██▏       | 49563/225000 [104:14:39<284:30:06,  5.84s/it]
 22%|██▏       | 49564/225000 [104:14:45<286:19:47,  5.88s/it]
 22%|██▏       | 49565/225000 [104:14:50<279:35:47,  5.74s/it]
 22%|██▏       | 49566/225000 [104:14:56<277:24:53,  5.69s/it]
 22%|██▏       | 49567/225000 [104:15:08<379:02:40,  7.78s/it]
 22%|██▏       | 49568/225000 [104:15:15<359:57:49,  7.39s/it]
 22%|██▏       | 49569/225000 [104:15:20<324:21:44,  6.66s/it]
 22%|██▏       | 49570/225000 [104:15:24<294:01:00,  6.03s/it]
 22%|██▏       | 49571/225000 [104:15:37<396:39:23,  8.14s/it]
 22%|██▏       | 49572/225000 [104:15:42<348:43:51,  7.16s/it]
 22%|██▏       | 49573/225000 [104:15:48<330:46:23,  6.79s/it]
 22%|██▏       | 49574/225000 [104:15:54<321:19:13,  6.59s/it]
 22%|██▏       | 49575/225000 [104:16:00<304:17:54,  6.24s/it]
 22%|██▏       | 49576/225000 [104:16:06<299:01:01,  6.14s/it]
 22%|██▏       | 49577/225000 [104:16:11<289:46:54,  5.95s/it]
 22%|██▏       | 49578/225000 [104:16:17<286:11:31,  5.87s/it]
 22%|██▏       | 49579/225000 [104:16:25<312:38:32,  6.42s/it]
 22%|██▏       | 49580/225000 [104:16:37<400:40:12,  8.22s/it]
 22%|██▏       | 49581/225000 [104:16:41<343:11:44,  7.04s/it]
 22%|██▏       | 49582/225000 [104:16:47<321:08:22,  6.59s/it]
 22%|██▏       | 49583/225000 [104:16:52<306:01:04,  6.28s/it]
 22%|██▏       | 49584/225000 [104:16:58<295:17:22,  6.06s/it]
 22%|██▏       | 49585/225000 [104:17:03<286:55:52,  5.89s/it]
 22%|██▏       | 49586/225000 [104:17:10<290:40:20,  5.97s/it]
 22%|██▏       | 49587/225000 [104:17:24<409:41:38,  8.41s/it]
 22%|██▏       | 49588/225000 [104:17:30<374:18:21,  7.68s/it]
 22%|██▏       | 49589/225000 [104:17:35<337:07:02,  6.92s/it]
 22%|██▏       | 49590/225000 [104:17:39<301:57:19,  6.20s/it]
 22%|██▏       | 49591/225000 [104:17:50<370:19:16,  7.60s/it]
 22%|██▏       | 49592/225000 [104:17:58<375:54:51,  7.72s/it]
 22%|██▏       | 49593/225000 [104:18:04<352:29:54,  7.23s/it]
 22%|██▏       | 49594/225000 [104:18:12<354:44:37,  7.28s/it]
 22%|██▏       | 49595/225000 [104:18:16<305:19:04,  6.27s/it]
 22%|██▏       | 49596/225000 [104:18:25<344:18:16,  7.07s/it]
 22%|██▏       | 49597/225000 [104:18:32<346:49:09,  7.12s/it]
 22%|██▏       | 49598/225000 [104:18:38<328:56:57,  6.75s/it]
 22%|██▏       | 49599/225000 [104:18:42<292:09:53,  6.00s/it]
 22%|██▏       | 49600/225000 [104:18:49<301:18:15,  6.18s/it]
                                                              
{'loss': '0.7009', 'grad_norm': '0.6922', 'learning_rate': '4.594e-05', 'epoch': '0.2204', 'num_input_tokens_seen': 174899544, 'train_runtime': '3.755e+05', 'train_tokens_per_second': '465.7'}

 22%|██▏       | 49600/225000 [104:18:49<301:18:15,  6.18s/it]
 22%|██▏       | 49601/225000 [104:18:57<340:40:26,  6.99s/it]
 22%|██▏       | 49602/225000 [104:19:01<293:02:57,  6.01s/it]
 22%|██▏       | 49603/225000 [104:19:07<290:54:14,  5.97s/it]
 22%|██▏       | 49604/225000 [104:19:20<392:57:45,  8.07s/it]
 22%|██▏       | 49605/225000 [104:19:23<326:02:41,  6.69s/it]
 22%|██▏       | 49606/225000 [104:19:31<341:42:00,  7.01s/it]
 22%|██▏       | 49607/225000 [104:19:39<356:12:44,  7.31s/it]
 22%|██▏       | 49608/225000 [104:19:43<303:02:27,  6.22s/it]
 22%|██▏       | 49609/225000 [104:19:50<317:46:31,  6.52s/it]
 22%|██▏       | 49610/225000 [104:19:57<315:35:32,  6.48s/it]
 22%|██▏       | 49611/225000 [104:20:03<315:44:25,  6.48s/it]
 22%|██▏       | 49612/225000 [104:20:08<301:20:06,  6.19s/it]
 22%|██▏       | 49613/225000 [104:20:14<284:45:42,  5.85s/it]
 22%|██▏       | 49614/225000 [104:20:19<279:30:50,  5.74s/it]
 22%|██▏       | 49615/225000 [104:20:30<353:10:15,  7.25s/it]
 22%|██▏       | 49616/225000 [104:20:37<355:02:35,  7.29s/it]
 22%|██▏       | 49617/225000 [104:20:44<344:03:51,  7.06s/it]
 22%|██▏       | 49618/225000 [104:20:50<337:50:20,  6.93s/it]
 22%|██▏       | 49619/225000 [104:20:58<349:40:55,  7.18s/it]
 22%|██▏       | 49620/225000 [104:21:02<297:59:58,  6.12s/it]
 22%|██▏       | 49621/225000 [104:21:08<301:57:52,  6.20s/it]
 22%|██▏       | 49622/225000 [104:21:18<351:11:55,  7.21s/it]
 22%|██▏       | 49623/225000 [104:21:22<309:15:12,  6.35s/it]
 22%|██▏       | 49624/225000 [104:21:29<316:03:06,  6.49s/it]
 22%|██▏       | 49625/225000 [104:21:38<354:51:13,  7.28s/it]
 22%|██▏       | 49626/225000 [104:21:44<335:17:22,  6.88s/it]
 22%|██▏       | 49627/225000 [104:21:49<312:34:02,  6.42s/it]
 22%|██▏       | 49628/225000 [104:21:53<279:30:07,  5.74s/it]
 22%|██▏       | 49629/225000 [104:22:01<307:41:20,  6.32s/it]
 22%|██▏       | 49630/225000 [104:22:11<358:31:08,  7.36s/it]
 22%|██▏       | 49631/225000 [104:22:15<311:36:16,  6.40s/it]
 22%|██▏       | 49632/225000 [104:22:24<355:42:50,  7.30s/it]
 22%|██▏       | 49633/225000 [104:22:28<305:09:38,  6.26s/it]
 22%|██▏       | 49634/225000 [104:22:39<365:21:31,  7.50s/it]
 22%|██▏       | 49635/225000 [104:22:43<314:37:08,  6.46s/it]
 22%|██▏       | 49636/225000 [104:22:48<304:14:02,  6.25s/it]
 22%|██▏       | 49637/225000 [104:22:58<352:41:03,  7.24s/it]
 22%|██▏       | 49638/225000 [104:23:05<342:50:42,  7.04s/it]
 22%|██▏       | 49639/225000 [104:23:09<303:11:57,  6.22s/it]
 22%|██▏       | 49640/225000 [104:23:14<280:23:38,  5.76s/it]
 22%|██▏       | 49641/225000 [104:23:25<366:19:32,  7.52s/it]
 22%|██▏       | 49642/225000 [104:23:30<326:51:25,  6.71s/it]
 22%|██▏       | 49643/225000 [104:23:38<341:25:56,  7.01s/it]
 22%|██▏       | 49644/225000 [104:23:43<310:53:08,  6.38s/it]
 22%|██▏       | 49645/225000 [104:23:48<293:13:03,  6.02s/it]
 22%|██▏       | 49646/225000 [104:23:59<372:53:39,  7.66s/it]
 22%|██▏       | 49647/225000 [104:24:03<318:38:55,  6.54s/it]
 22%|██▏       | 49648/225000 [104:24:12<345:49:38,  7.10s/it]
 22%|██▏       | 49649/225000 [104:24:17<325:36:39,  6.68s/it]
 22%|██▏       | 49650/225000 [104:24:24<325:51:02,  6.69s/it]
                                                              
{'loss': '0.7147', 'grad_norm': '0.7074', 'learning_rate': '4.593e-05', 'epoch': '0.2207', 'num_input_tokens_seen': 175046752, 'train_runtime': '3.759e+05', 'train_tokens_per_second': '465.7'}

 22%|██▏       | 49650/225000 [104:24:24<325:51:02,  6.69s/it]
 22%|██▏       | 49651/225000 [104:24:32<341:14:27,  7.01s/it]
 22%|██▏       | 49652/225000 [104:24:38<323:51:00,  6.65s/it]
 22%|██▏       | 49653/225000 [104:24:43<310:00:40,  6.36s/it]
 22%|██▏       | 49654/225000 [104:24:49<297:49:15,  6.11s/it]
 22%|██▏       | 49655/225000 [104:24:54<285:24:29,  5.86s/it]
 22%|██▏       | 49656/225000 [104:25:04<345:20:03,  7.09s/it]
 22%|██▏       | 49657/225000 [104:25:13<373:22:06,  7.67s/it]
 22%|██▏       | 49658/225000 [104:25:17<320:01:03,  6.57s/it]
 22%|██▏       | 49659/225000 [104:25:22<293:54:42,  6.03s/it]
 22%|██▏       | 49660/225000 [104:25:27<287:20:50,  5.90s/it]
 22%|██▏       | 49661/225000 [104:25:37<346:13:02,  7.11s/it]
 22%|██▏       | 49662/225000 [104:25:48<393:03:13,  8.07s/it]
 22%|██▏       | 49663/225000 [104:25:51<322:10:31,  6.61s/it]
 22%|██▏       | 49664/225000 [104:26:00<354:37:16,  7.28s/it]
 22%|██▏       | 49665/225000 [104:26:04<304:48:53,  6.26s/it]
 22%|██▏       | 49666/225000 [104:26:16<396:26:22,  8.14s/it]
 22%|██▏       | 49667/225000 [104:26:20<335:57:52,  6.90s/it]
 22%|██▏       | 49668/225000 [104:26:26<324:22:35,  6.66s/it]
 22%|██▏       | 49669/225000 [104:26:35<361:10:00,  7.42s/it]
 22%|██▏       | 49670/225000 [104:26:40<317:38:02,  6.52s/it]
 22%|██▏       | 49671/225000 [104:26:44<287:37:49,  5.91s/it]
 22%|██▏       | 49672/225000 [104:26:55<351:29:56,  7.22s/it]
 22%|██▏       | 49673/225000 [104:27:01<339:41:41,  6.97s/it]
 22%|██▏       | 49674/225000 [104:27:05<299:15:15,  6.14s/it]
 22%|██▏       | 49675/225000 [104:27:15<352:36:05,  7.24s/it]
 22%|██▏       | 49676/225000 [104:27:20<313:13:06,  6.43s/it]
 22%|██▏       | 49677/225000 [104:27:24<281:06:26,  5.77s/it]
 22%|██▏       | 49678/225000 [104:27:32<317:20:34,  6.52s/it]
 22%|██▏       | 49679/225000 [104:27:39<321:58:51,  6.61s/it]
 22%|██▏       | 49680/225000 [104:27:45<309:18:19,  6.35s/it]
 22%|██▏       | 49681/225000 [104:27:55<367:34:00,  7.55s/it]
 22%|██▏       | 49682/225000 [104:27:59<317:56:35,  6.53s/it]
 22%|██▏       | 49683/225000 [104:28:04<294:26:09,  6.05s/it]
 22%|██▏       | 49684/225000 [104:28:10<286:37:50,  5.89s/it]
 22%|██▏       | 49685/225000 [104:28:15<275:57:03,  5.67s/it]
 22%|██▏       | 49686/225000 [104:28:23<316:57:48,  6.51s/it]
 22%|██▏       | 49687/225000 [104:28:36<406:59:31,  8.36s/it]
 22%|██▏       | 49688/225000 [104:28:40<347:36:39,  7.14s/it]
 22%|██▏       | 49689/225000 [104:28:49<372:59:08,  7.66s/it]
 22%|██▏       | 49690/225000 [104:28:53<316:04:40,  6.49s/it]
 22%|██▏       | 49691/225000 [104:29:02<354:23:22,  7.28s/it]
 22%|██▏       | 49692/225000 [104:29:06<308:25:18,  6.33s/it]
 22%|██▏       | 49693/225000 [104:29:16<365:05:18,  7.50s/it]
 22%|██▏       | 49694/225000 [104:29:21<330:15:59,  6.78s/it]
 22%|██▏       | 49695/225000 [104:29:26<303:37:05,  6.23s/it]
 22%|██▏       | 49696/225000 [104:29:32<295:16:18,  6.06s/it]
 22%|██▏       | 49697/225000 [104:29:38<289:50:03,  5.95s/it]
 22%|██▏       | 49698/225000 [104:29:43<283:50:39,  5.83s/it]
 22%|██▏       | 49699/225000 [104:29:49<282:58:13,  5.81s/it]
 22%|██▏       | 49700/225000 [104:30:01<372:05:38,  7.64s/it]
                                                              
{'loss': '0.6988', 'grad_norm': '0.8298', 'learning_rate': '4.592e-05', 'epoch': '0.2209', 'num_input_tokens_seen': 175195872, 'train_runtime': '3.762e+05', 'train_tokens_per_second': '465.7'}

 22%|██▏       | 49700/225000 [104:30:01<372:05:38,  7.64s/it]
 22%|██▏       | 49701/225000 [104:30:12<417:54:32,  8.58s/it]
 22%|██▏       | 49702/225000 [104:30:17<373:17:20,  7.67s/it]
 22%|██▏       | 49703/225000 [104:30:27<402:41:47,  8.27s/it]
 22%|██▏       | 49704/225000 [104:30:32<359:05:23,  7.37s/it]
 22%|██▏       | 49705/225000 [104:30:37<327:19:45,  6.72s/it]
 22%|██▏       | 49706/225000 [104:30:43<305:44:05,  6.28s/it]
 22%|██▏       | 49707/225000 [104:30:53<370:11:39,  7.60s/it]
 22%|██▏       | 49708/225000 [104:31:02<384:54:31,  7.90s/it]
 22%|██▏       | 49709/225000 [104:31:07<341:47:33,  7.02s/it]
 22%|██▏       | 49710/225000 [104:31:15<362:04:30,  7.44s/it]
 22%|██▏       | 49711/225000 [104:31:23<370:57:40,  7.62s/it]
 22%|██▏       | 49712/225000 [104:31:28<324:06:19,  6.66s/it]
 22%|██▏       | 49713/225000 [104:31:33<296:23:26,  6.09s/it]
 22%|██▏       | 49714/225000 [104:31:37<271:30:55,  5.58s/it]
 22%|██▏       | 49715/225000 [104:31:51<399:17:09,  8.20s/it]
 22%|██▏       | 49716/225000 [104:31:58<380:44:12,  7.82s/it]
 22%|██▏       | 49717/225000 [104:32:04<358:41:18,  7.37s/it]
 22%|██▏       | 49718/225000 [104:32:14<383:04:46,  7.87s/it]
 22%|██▏       | 49719/225000 [104:32:18<337:21:29,  6.93s/it]
 22%|██▏       | 49720/225000 [104:32:23<310:09:10,  6.37s/it]
 22%|██▏       | 49721/225000 [104:32:32<347:51:07,  7.14s/it]
 22%|██▏       | 49722/225000 [104:32:41<371:16:36,  7.63s/it]
 22%|██▏       | 49723/225000 [104:32:46<336:27:54,  6.91s/it]
 22%|██▏       | 49724/225000 [104:32:52<316:53:19,  6.51s/it]
 22%|██▏       | 49725/225000 [104:32:57<296:19:06,  6.09s/it]
 22%|██▏       | 49726/225000 [104:33:02<281:05:06,  5.77s/it]
 22%|██▏       | 49727/225000 [104:33:09<304:52:57,  6.26s/it]
 22%|██▏       | 49728/225000 [104:33:22<403:54:35,  8.30s/it]
 22%|██▏       | 49729/225000 [104:33:27<346:40:08,  7.12s/it]
 22%|██▏       | 49730/225000 [104:33:36<381:28:13,  7.84s/it]
 22%|██▏       | 49731/225000 [104:33:41<341:54:35,  7.02s/it]
 22%|██▏       | 49732/225000 [104:33:50<362:13:16,  7.44s/it]
 22%|██▏       | 49733/225000 [104:33:55<333:51:39,  6.86s/it]
 22%|██▏       | 49734/225000 [104:34:00<306:25:46,  6.29s/it]
 22%|██▏       | 49735/225000 [104:34:06<301:41:31,  6.20s/it]
 22%|██▏       | 49736/225000 [104:34:12<292:16:39,  6.00s/it]
 22%|██▏       | 49737/225000 [104:34:17<285:14:19,  5.86s/it]
 22%|██▏       | 49738/225000 [104:34:23<274:58:24,  5.65s/it]
 22%|██▏       | 49739/225000 [104:34:28<275:42:37,  5.66s/it]
 22%|██▏       | 49740/225000 [104:34:40<368:45:58,  7.57s/it]
 22%|██▏       | 49741/225000 [104:34:49<383:24:09,  7.88s/it]
 22%|██▏       | 49742/225000 [104:34:57<382:39:20,  7.86s/it]
 22%|██▏       | 49743/225000 [104:35:01<331:40:11,  6.81s/it]
 22%|██▏       | 49744/225000 [104:35:07<315:21:45,  6.48s/it]
 22%|██▏       | 49745/225000 [104:35:20<417:03:01,  8.57s/it]
 22%|██▏       | 49746/225000 [104:35:24<348:07:07,  7.15s/it]
 22%|██▏       | 49747/225000 [104:35:33<370:45:53,  7.62s/it]
 22%|██▏       | 49748/225000 [104:35:40<370:04:32,  7.60s/it]
 22%|██▏       | 49749/225000 [104:35:44<315:29:55,  6.48s/it]
 22%|██▏       | 49750/225000 [104:35:53<343:23:00,  7.05s/it]
                                                              
{'loss': '0.7142', 'grad_norm': '0.7943', 'learning_rate': '4.591e-05', 'epoch': '0.2211', 'num_input_tokens_seen': 175346656, 'train_runtime': '3.766e+05', 'train_tokens_per_second': '465.7'}

 22%|██▏       | 49750/225000 [104:35:53<343:23:00,  7.05s/it]
 22%|██▏       | 49751/225000 [104:35:59<338:46:55,  6.96s/it]
 22%|██▏       | 49752/225000 [104:36:03<295:29:59,  6.07s/it]
 22%|██▏       | 49753/225000 [104:36:13<350:30:54,  7.20s/it]
 22%|██▏       | 49754/225000 [104:36:18<315:46:41,  6.49s/it]
 22%|██▏       | 49755/225000 [104:36:24<310:30:56,  6.38s/it]
 22%|██▏       | 49756/225000 [104:36:33<346:17:41,  7.11s/it]
 22%|██▏       | 49757/225000 [104:36:37<299:33:09,  6.15s/it]
 22%|██▏       | 49758/225000 [104:36:42<287:21:46,  5.90s/it]
 22%|██▏       | 49759/225000 [104:36:47<273:27:01,  5.62s/it]
 22%|██▏       | 49760/225000 [104:36:53<271:18:29,  5.57s/it]
 22%|██▏       | 49761/225000 [104:36:58<274:53:05,  5.65s/it]
 22%|██▏       | 49762/225000 [104:37:04<272:21:12,  5.60s/it]
 22%|██▏       | 49763/225000 [104:37:16<374:46:45,  7.70s/it]
 22%|██▏       | 49764/225000 [104:37:23<361:47:56,  7.43s/it]
 22%|██▏       | 49765/225000 [104:37:28<323:56:27,  6.65s/it]
 22%|██▏       | 49766/225000 [104:37:33<296:34:34,  6.09s/it]
 22%|██▏       | 49767/225000 [104:37:38<282:56:22,  5.81s/it]
 22%|██▏       | 49768/225000 [104:37:46<318:09:45,  6.54s/it]
 22%|██▏       | 49769/225000 [104:38:00<428:17:20,  8.80s/it]
 22%|██▏       | 49770/225000 [104:38:06<380:43:37,  7.82s/it]
 22%|██▏       | 49771/225000 [104:38:12<349:05:22,  7.17s/it]
 22%|██▏       | 49772/225000 [104:38:16<316:07:16,  6.49s/it]
 22%|██▏       | 49773/225000 [104:38:21<292:10:12,  6.00s/it]
 22%|██▏       | 49774/225000 [104:38:26<279:23:18,  5.74s/it]
 22%|██▏       | 49775/225000 [104:38:33<297:04:14,  6.10s/it]
 22%|██▏       | 49776/225000 [104:38:46<394:24:52,  8.10s/it]
 22%|██▏       | 49777/225000 [104:38:50<335:24:02,  6.89s/it]
 22%|██▏       | 49778/225000 [104:38:54<295:25:05,  6.07s/it]
 22%|██▏       | 49779/225000 [104:39:08<401:40:44,  8.25s/it]
 22%|██▏       | 49780/225000 [104:39:12<350:24:18,  7.20s/it]
 22%|██▏       | 49781/225000 [104:39:17<317:28:02,  6.52s/it]
 22%|██▏       | 49782/225000 [104:39:22<295:17:21,  6.07s/it]
 22%|██▏       | 49783/225000 [104:39:31<338:04:43,  6.95s/it]
 22%|██▏       | 49784/225000 [104:39:41<377:51:15,  7.76s/it]
 22%|██▏       | 49785/225000 [104:39:45<323:40:53,  6.65s/it]
 22%|██▏       | 49786/225000 [104:39:55<377:35:05,  7.76s/it]
 22%|██▏       | 49787/225000 [104:40:00<332:39:38,  6.83s/it]
 22%|██▏       | 49788/225000 [104:40:06<312:37:31,  6.42s/it]
 22%|██▏       | 49789/225000 [104:40:11<290:58:16,  5.98s/it]
 22%|██▏       | 49790/225000 [104:40:16<276:50:16,  5.69s/it]
 22%|██▏       | 49791/225000 [104:40:28<377:50:05,  7.76s/it]
 22%|██▏       | 49792/225000 [104:40:35<359:12:57,  7.38s/it]
 22%|██▏       | 49793/225000 [104:40:44<384:43:52,  7.91s/it]
 22%|██▏       | 49794/225000 [104:40:48<333:14:55,  6.85s/it]
 22%|██▏       | 49795/225000 [104:40:55<337:40:52,  6.94s/it]
 22%|██▏       | 49796/225000 [104:41:02<331:25:51,  6.81s/it]
 22%|██▏       | 49797/225000 [104:41:07<311:31:17,  6.40s/it]
 22%|██▏       | 49798/225000 [104:41:13<296:16:59,  6.09s/it]
 22%|██▏       | 49799/225000 [104:41:18<278:49:19,  5.73s/it]
 22%|██▏       | 49800/225000 [104:41:23<269:49:58,  5.54s/it]
                                                              
{'loss': '0.7079', 'grad_norm': '0.7968', 'learning_rate': '4.59e-05', 'epoch': '0.2213', 'num_input_tokens_seen': 175494544, 'train_runtime': '3.769e+05', 'train_tokens_per_second': '465.6'}

 22%|██▏       | 49800/225000 [104:41:23<269:49:58,  5.54s/it]
 22%|██▏       | 49801/225000 [104:41:29<284:20:34,  5.84s/it]
 22%|██▏       | 49802/225000 [104:41:42<391:04:54,  8.04s/it]
 22%|██▏       | 49803/225000 [104:41:48<361:29:36,  7.43s/it]
 22%|██▏       | 49804/225000 [104:41:55<344:00:55,  7.07s/it]
 22%|██▏       | 49805/225000 [104:42:03<360:51:31,  7.42s/it]
 22%|██▏       | 49806/225000 [104:42:07<309:27:58,  6.36s/it]
 22%|██▏       | 49807/225000 [104:42:16<348:01:12,  7.15s/it]
 22%|██▏       | 49808/225000 [104:42:24<360:23:24,  7.41s/it]
 22%|██▏       | 49809/225000 [104:42:30<337:56:22,  6.94s/it]
 22%|██▏       | 49810/225000 [104:42:35<310:47:17,  6.39s/it]
 22%|██▏       | 49811/225000 [104:42:40<291:02:48,  5.98s/it]
 22%|██▏       | 49812/225000 [104:42:45<281:20:25,  5.78s/it]
 22%|██▏       | 49813/225000 [104:42:50<276:02:00,  5.67s/it]
 22%|██▏       | 49814/225000 [104:43:03<376:02:32,  7.73s/it]
 22%|██▏       | 49815/225000 [104:43:09<358:09:34,  7.36s/it]
 22%|██▏       | 49816/225000 [104:43:16<349:19:24,  7.18s/it]
 22%|██▏       | 49817/225000 [104:43:23<341:02:58,  7.01s/it]
 22%|██▏       | 49818/225000 [104:43:30<338:10:34,  6.95s/it]
 22%|██▏       | 49819/225000 [104:43:37<341:28:14,  7.02s/it]
 22%|██▏       | 49820/225000 [104:43:43<334:14:50,  6.87s/it]
 22%|██▏       | 49821/225000 [104:43:50<331:32:58,  6.81s/it]
 22%|██▏       | 49822/225000 [104:43:56<313:34:25,  6.44s/it]
 22%|██▏       | 49823/225000 [104:44:00<283:38:20,  5.83s/it]
 22%|██▏       | 49824/225000 [104:44:05<270:28:04,  5.56s/it]
 22%|██▏       | 49825/225000 [104:44:15<334:20:54,  6.87s/it]
 22%|██▏       | 49826/225000 [104:44:22<341:01:51,  7.01s/it]
 22%|██▏       | 49827/225000 [104:44:27<314:00:53,  6.45s/it]
 22%|██▏       | 49828/225000 [104:44:32<287:46:32,  5.91s/it]
 22%|██▏       | 49829/225000 [104:44:37<274:43:39,  5.65s/it]
 22%|██▏       | 49830/225000 [104:44:47<334:03:17,  6.87s/it]
 22%|██▏       | 49831/225000 [104:44:55<349:52:09,  7.19s/it]
 22%|██▏       | 49832/225000 [104:45:00<318:19:09,  6.54s/it]
 22%|██▏       | 49833/225000 [104:45:05<299:10:41,  6.15s/it]
 22%|██▏       | 49834/225000 [104:45:10<284:42:06,  5.85s/it]
 22%|██▏       | 49835/225000 [104:45:24<397:52:48,  8.18s/it]
 22%|██▏       | 49836/225000 [104:45:29<362:32:19,  7.45s/it]
 22%|██▏       | 49837/225000 [104:45:38<374:10:48,  7.69s/it]
 22%|██▏       | 49838/225000 [104:45:42<321:44:01,  6.61s/it]
 22%|██▏       | 49839/225000 [104:45:53<384:49:15,  7.91s/it]
 22%|██▏       | 49840/225000 [104:45:57<336:01:37,  6.91s/it]
 22%|██▏       | 49841/225000 [104:46:01<296:32:04,  6.09s/it]
 22%|██▏       | 49842/225000 [104:46:07<284:58:54,  5.86s/it]
 22%|██▏       | 49843/225000 [104:46:15<315:08:22,  6.48s/it]
 22%|██▏       | 49844/225000 [104:46:28<408:29:20,  8.40s/it]
 22%|██▏       | 49845/225000 [104:46:31<339:51:57,  6.99s/it]
 22%|██▏       | 49846/225000 [104:46:35<294:37:54,  6.06s/it]
 22%|██▏       | 49847/225000 [104:46:45<345:59:23,  7.11s/it]
 22%|██▏       | 49848/225000 [104:46:51<336:51:25,  6.92s/it]
 22%|██▏       | 49849/225000 [104:46:57<313:58:37,  6.45s/it]
 22%|██▏       | 49850/225000 [104:47:02<295:22:52,  6.07s/it]
                                                              
{'loss': '0.7012', 'grad_norm': '0.6922', 'learning_rate': '4.589e-05', 'epoch': '0.2216', 'num_input_tokens_seen': 175642272, 'train_runtime': '3.772e+05', 'train_tokens_per_second': '465.6'}

 22%|██▏       | 49850/225000 [104:47:02<295:22:52,  6.07s/it]
 22%|██▏       | 49851/225000 [104:47:07<283:12:43,  5.82s/it]
 22%|██▏       | 49852/225000 [104:47:12<275:49:58,  5.67s/it]
 22%|██▏       | 49853/225000 [104:47:25<374:07:58,  7.69s/it]
 22%|██▏       | 49854/225000 [104:47:33<376:37:52,  7.74s/it]
 22%|██▏       | 49855/225000 [104:47:40<367:23:19,  7.55s/it]
 22%|██▏       | 49856/225000 [104:47:47<362:32:44,  7.45s/it]
 22%|██▏       | 49857/225000 [104:47:53<346:25:41,  7.12s/it]
 22%|██▏       | 49858/225000 [104:48:03<385:30:51,  7.92s/it]
 22%|██▏       | 49859/225000 [104:48:08<336:58:42,  6.93s/it]
 22%|██▏       | 49860/225000 [104:48:12<298:27:19,  6.13s/it]
 22%|██▏       | 49861/225000 [104:48:24<382:15:17,  7.86s/it]
 22%|██▏       | 49862/225000 [104:48:28<324:48:30,  6.68s/it]
 22%|██▏       | 49863/225000 [104:48:33<301:34:37,  6.20s/it]
 22%|██▏       | 49864/225000 [104:48:42<341:44:45,  7.02s/it]
 22%|██▏       | 49865/225000 [104:48:46<301:50:59,  6.20s/it]
 22%|██▏       | 49866/225000 [104:48:51<283:38:04,  5.83s/it]
 22%|██▏       | 49867/225000 [104:49:01<346:46:57,  7.13s/it]
 22%|██▏       | 49868/225000 [104:49:09<355:49:31,  7.31s/it]
 22%|██▏       | 49869/225000 [104:49:16<346:44:34,  7.13s/it]
 22%|██▏       | 49870/225000 [104:49:20<311:41:16,  6.41s/it]
 22%|██▏       | 49871/225000 [104:49:25<284:42:25,  5.85s/it]
 22%|██▏       | 49872/225000 [104:49:36<365:21:55,  7.51s/it]
 22%|██▏       | 49873/225000 [104:49:43<352:53:48,  7.25s/it]
 22%|██▏       | 49874/225000 [104:49:48<316:58:42,  6.52s/it]
 22%|██▏       | 49875/225000 [104:49:58<373:04:42,  7.67s/it]
 22%|██▏       | 49876/225000 [104:50:03<335:17:39,  6.89s/it]
 22%|██▏       | 49877/225000 [104:50:08<303:10:35,  6.23s/it]
 22%|██▏       | 49878/225000 [104:50:13<288:42:05,  5.93s/it]
 22%|██▏       | 49879/225000 [104:50:26<393:02:53,  8.08s/it]
 22%|██▏       | 49880/225000 [104:50:30<333:28:31,  6.86s/it]
 22%|██▏       | 49881/225000 [104:50:39<362:54:50,  7.46s/it]
 22%|██▏       | 49882/225000 [104:50:43<312:26:09,  6.42s/it]
 22%|██▏       | 49883/225000 [104:50:53<361:32:28,  7.43s/it]
 22%|██▏       | 49884/225000 [104:50:58<334:58:30,  6.89s/it]
 22%|██▏       | 49885/225000 [104:51:04<318:31:54,  6.55s/it]
 22%|██▏       | 49886/225000 [104:51:11<324:29:53,  6.67s/it]
 22%|██▏       | 49887/225000 [104:51:17<305:26:52,  6.28s/it]
 22%|██▏       | 49888/225000 [104:51:22<295:42:37,  6.08s/it]
 22%|██▏       | 49889/225000 [104:51:27<281:09:57,  5.78s/it]
 22%|██▏       | 49890/225000 [104:51:41<392:29:01,  8.07s/it]
 22%|██▏       | 49891/225000 [104:51:49<394:25:51,  8.11s/it]
 22%|██▏       | 49892/225000 [104:51:56<387:09:16,  7.96s/it]
 22%|██▏       | 49893/225000 [104:52:00<324:47:25,  6.68s/it]
 22%|██▏       | 49894/225000 [104:52:09<363:56:36,  7.48s/it]
 22%|██▏       | 49895/225000 [104:52:16<351:10:55,  7.22s/it]
 22%|██▏       | 49896/225000 [104:52:22<337:45:09,  6.94s/it]
 22%|██▏       | 49897/225000 [104:52:27<299:18:18,  6.15s/it]
 22%|██▏       | 49898/225000 [104:52:31<278:47:22,  5.73s/it]
 22%|██▏       | 49899/225000 [104:52:41<338:59:49,  6.97s/it]
 22%|██▏       | 49900/225000 [104:52:50<362:15:42,  7.45s/it]
                                                              
{'loss': '0.7046', 'grad_norm': '0.7379', 'learning_rate': '4.588e-05', 'epoch': '0.2218', 'num_input_tokens_seen': 175791296, 'train_runtime': '3.776e+05', 'train_tokens_per_second': '465.6'}

 22%|██▏       | 49900/225000 [104:52:50<362:15:42,  7.45s/it]
 22%|██▏       | 49901/225000 [104:52:54<320:57:24,  6.60s/it]
 22%|██▏       | 49902/225000 [104:53:00<312:16:00,  6.42s/it]
 22%|██▏       | 49903/225000 [104:53:06<299:52:05,  6.17s/it]
 22%|██▏       | 49904/225000 [104:53:12<294:08:13,  6.05s/it]
 22%|██▏       | 49905/225000 [104:53:18<292:27:20,  6.01s/it]
 22%|██▏       | 49906/225000 [104:53:28<359:27:46,  7.39s/it]
 22%|██▏       | 49907/225000 [104:53:35<342:35:48,  7.04s/it]
 22%|██▏       | 49908/225000 [104:53:42<353:25:09,  7.27s/it]
 22%|██▏       | 49909/225000 [104:53:47<313:09:50,  6.44s/it]
 22%|██▏       | 49910/225000 [104:53:52<296:39:48,  6.10s/it]
 22%|██▏       | 49911/225000 [104:54:06<403:24:57,  8.29s/it]
 22%|██▏       | 49912/225000 [104:54:09<337:51:24,  6.95s/it]
 22%|██▏       | 49913/225000 [104:54:18<358:25:31,  7.37s/it]
 22%|██▏       | 49914/225000 [104:54:26<371:18:27,  7.63s/it]
 22%|██▏       | 49915/225000 [104:54:31<329:03:12,  6.77s/it]
 22%|██▏       | 49916/225000 [104:54:40<368:35:30,  7.58s/it]
 22%|██▏       | 49917/225000 [104:54:47<361:00:51,  7.42s/it]
 22%|██▏       | 49918/225000 [104:54:52<315:48:39,  6.49s/it]
 22%|██▏       | 49919/225000 [104:54:56<283:51:06,  5.84s/it]
 22%|██▏       | 49920/225000 [104:55:06<351:17:42,  7.22s/it]
 22%|██▏       | 49921/225000 [104:55:11<315:56:12,  6.50s/it]
 22%|██▏       | 49922/225000 [104:55:16<296:48:26,  6.10s/it]
 22%|██▏       | 49923/225000 [104:55:26<345:31:25,  7.10s/it]
 22%|██▏       | 49924/225000 [104:55:33<353:02:44,  7.26s/it]
 22%|██▏       | 49925/225000 [104:55:41<354:31:49,  7.29s/it]
 22%|██▏       | 49926/225000 [104:55:50<377:46:00,  7.77s/it]
 22%|██▏       | 49927/225000 [104:55:55<336:36:41,  6.92s/it]
 22%|██▏       | 49928/225000 [104:56:01<335:24:42,  6.90s/it]
 22%|██▏       | 49929/225000 [104:56:05<289:19:56,  5.95s/it]
 22%|██▏       | 49930/225000 [104:56:15<343:01:18,  7.05s/it]
 22%|██▏       | 49931/225000 [104:56:19<307:21:49,  6.32s/it]
 22%|██▏       | 49932/225000 [104:56:24<281:26:43,  5.79s/it]
 22%|██▏       | 49933/225000 [104:56:29<276:37:07,  5.69s/it]
 22%|██▏       | 49934/225000 [104:56:42<376:57:37,  7.75s/it]
 22%|██▏       | 49935/225000 [104:56:48<355:41:24,  7.31s/it]
 22%|██▏       | 49936/225000 [104:56:53<321:14:11,  6.61s/it]
 22%|██▏       | 49937/225000 [104:56:58<294:30:38,  6.06s/it]
 22%|██▏       | 49938/225000 [104:57:03<274:53:50,  5.65s/it]
 22%|██▏       | 49939/225000 [104:57:14<353:49:01,  7.28s/it]
 22%|██▏       | 49940/225000 [104:57:21<346:09:47,  7.12s/it]
 22%|██▏       | 49941/225000 [104:57:25<303:28:03,  6.24s/it]
 22%|██▏       | 49942/225000 [104:57:34<342:36:34,  7.05s/it]
 22%|██▏       | 49943/225000 [104:57:40<335:11:32,  6.89s/it]
 22%|██▏       | 49944/225000 [104:57:47<334:20:59,  6.88s/it]
 22%|██▏       | 49945/225000 [104:57:51<296:59:50,  6.11s/it]
 22%|██▏       | 49946/225000 [104:58:01<347:13:55,  7.14s/it]
 22%|██▏       | 49947/225000 [104:58:09<367:46:13,  7.56s/it]
 22%|██▏       | 49948/225000 [104:58:13<310:41:27,  6.39s/it]
 22%|██▏       | 49949/225000 [104:58:22<340:22:04,  7.00s/it]
 22%|██▏       | 49950/225000 [104:58:25<289:08:51,  5.95s/it]
                                                              
{'loss': '0.6876', 'grad_norm': '0.7755', 'learning_rate': '4.587e-05', 'epoch': '0.222', 'num_input_tokens_seen': 175938400, 'train_runtime': '3.779e+05', 'train_tokens_per_second': '465.6'}

 22%|██▏       | 49950/225000 [104:58:25<289:08:51,  5.95s/it]
 22%|██▏       | 49951/225000 [104:58:30<275:37:27,  5.67s/it]
 22%|██▏       | 49952/225000 [104:58:35<263:19:32,  5.42s/it]
 22%|██▏       | 49953/225000 [104:58:40<256:02:49,  5.27s/it]
 22%|██▏       | 49954/225000 [104:58:49<316:46:46,  6.51s/it]
 22%|██▏       | 49955/225000 [104:59:00<374:31:40,  7.70s/it]
 22%|██▏       | 49956/225000 [104:59:06<356:05:09,  7.32s/it]
 22%|██▏       | 49957/225000 [104:59:12<327:36:32,  6.74s/it]
 22%|██▏       | 49958/225000 [104:59:17<308:21:01,  6.34s/it]
 22%|██▏       | 49959/225000 [104:59:23<297:26:37,  6.12s/it]
 22%|██▏       | 49960/225000 [104:59:33<365:05:44,  7.51s/it]
 22%|██▏       | 49961/225000 [104:59:42<375:47:31,  7.73s/it]
 22%|██▏       | 49962/225000 [104:59:49<369:28:21,  7.60s/it]
 22%|██▏       | 49963/225000 [104:59:54<328:23:54,  6.75s/it]
 22%|██▏       | 49964/225000 [104:59:58<291:37:00,  6.00s/it]
 22%|██▏       | 49965/225000 [105:00:03<273:41:59,  5.63s/it]
 22%|██▏       | 49966/225000 [105:00:08<265:51:58,  5.47s/it]
 22%|██▏       | 49967/225000 [105:00:17<316:21:36,  6.51s/it]
 22%|██▏       | 49968/225000 [105:00:30<410:26:31,  8.44s/it]
 22%|██▏       | 49969/225000 [105:00:33<344:11:09,  7.08s/it]
 22%|██▏       | 49970/225000 [105:00:44<391:59:45,  8.06s/it]
 22%|██▏       | 49971/225000 [105:00:48<337:31:19,  6.94s/it]
 22%|██▏       | 49972/225000 [105:00:54<318:53:55,  6.56s/it]
 22%|██▏       | 49973/225000 [105:01:04<376:55:28,  7.75s/it]
 22%|██▏       | 49974/225000 [105:01:09<326:42:33,  6.72s/it]
 22%|██▏       | 49975/225000 [105:01:18<362:21:55,  7.45s/it]
 22%|██▏       | 49976/225000 [105:01:27<386:28:33,  7.95s/it]
 22%|██▏       | 49977/225000 [105:01:31<325:53:00,  6.70s/it]
 22%|██▏       | 49978/225000 [105:01:39<341:38:17,  7.03s/it]
 22%|██▏       | 49979/225000 [105:01:45<330:52:43,  6.81s/it]
 22%|██▏       | 49980/225000 [105:01:51<323:11:02,  6.65s/it]
 22%|██▏       | 49981/225000 [105:01:57<308:34:31,  6.35s/it]
 22%|██▏       | 49982/225000 [105:02:05<332:08:46,  6.83s/it]
 22%|██▏       | 49983/225000 [105:02:11<325:51:57,  6.70s/it]
 22%|██▏       | 49984/225000 [105:02:16<304:15:05,  6.26s/it]
 22%|██▏       | 49985/225000 [105:02:24<328:26:01,  6.76s/it]
 22%|██▏       | 49986/225000 [105:02:31<324:17:43,  6.67s/it]
 22%|██▏       | 49987/225000 [105:02:40<362:07:12,  7.45s/it]
 22%|██▏       | 49988/225000 [105:02:44<311:23:22,  6.41s/it]
 22%|██▏       | 49989/225000 [105:02:51<322:28:38,  6.63s/it]
 22%|██▏       | 49990/225000 [105:02:58<332:07:21,  6.83s/it]
 22%|██▏       | 49991/225000 [105:03:03<297:44:19,  6.12s/it]
 22%|██▏       | 49992/225000 [105:03:11<329:45:33,  6.78s/it]
 22%|██▏       | 49993/225000 [105:03:15<288:08:07,  5.93s/it]
 22%|██▏       | 49994/225000 [105:03:25<338:19:37,  6.96s/it]
 22%|██▏       | 49995/225000 [105:03:31<328:16:27,  6.75s/it]
 22%|██▏       | 49996/225000 [105:03:36<302:35:32,  6.22s/it]
 22%|██▏       | 49997/225000 [105:03:44<326:18:09,  6.71s/it]
 22%|██▏       | 49998/225000 [105:03:55<399:40:51,  8.22s/it]
 22%|██▏       | 49999/225000 [105:04:02<377:45:19,  7.77s/it]
 22%|██▏       | 50000/225000 [105:04:07<334:53:16,  6.89s/it]
                                                              
{'loss': '0.6884', 'grad_norm': '0.5941', 'learning_rate': '4.585e-05', 'epoch': '0.2222', 'num_input_tokens_seen': 176087432, 'train_runtime': '3.782e+05', 'train_tokens_per_second': '465.5'}

 22%|██▏       | 50000/225000 [105:04:07<334:53:16,  6.89s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                              
{'eval_loss': '1.174', 'eval_runtime': '464.5', 'eval_samples_per_second': '1.076', 'eval_steps_per_second': '0.538', 'epoch': '0.2222', 'num_input_tokens_seen': 176087432}

 22%|██▏       | 50000/225000 [105:11:51<334:53:16,  6.89s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-50000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 22%|██▏       | 50001/225000 [105:11:57<7093:40:19, 145.93s/it]
 22%|██▏       | 50002/225000 [105:12:02<5039:05:16, 103.66s/it]
 22%|██▏       | 50003/225000 [105:12:07<3598:53:03, 74.04s/it] 
 22%|██▏       | 50004/225000 [105:12:12<2595:18:41, 53.39s/it]
 22%|██▏       | 50005/225000 [105:12:17<1889:22:19, 38.87s/it]
 22%|██▏       | 50006/225000 [105:12:23<1399:07:44, 28.78s/it]
 22%|██▏       | 50007/225000 [105:12:35<1165:33:42, 23.98s/it]
 22%|██▏       | 50008/225000 [105:12:39<872:15:37, 17.94s/it] 
 22%|██▏       | 50009/225000 [105:12:45<688:44:46, 14.17s/it]
 22%|██▏       | 50010/225000 [105:12:50<560:31:01, 11.53s/it]
 22%|██▏       | 50011/225000 [105:12:56<477:20:22,  9.82s/it]
 22%|██▏       | 50012/225000 [105:13:01<408:40:30,  8.41s/it]
 22%|██▏       | 50013/225000 [105:13:07<376:44:28,  7.75s/it]
 22%|██▏       | 50014/225000 [105:13:20<451:29:43,  9.29s/it]
 22%|██▏       | 50015/225000 [105:13:26<396:42:53,  8.16s/it]
 22%|██▏       | 50016/225000 [105:13:34<396:03:03,  8.15s/it]
 22%|██▏       | 50017/225000 [105:13:42<397:06:06,  8.17s/it]
 22%|██▏       | 50018/225000 [105:13:46<341:17:28,  7.02s/it]
 22%|██▏       | 50019/225000 [105:13:54<355:22:53,  7.31s/it]
 22%|██▏       | 50020/225000 [105:14:02<359:02:04,  7.39s/it]
 22%|██▏       | 50021/225000 [105:14:06<305:55:23,  6.29s/it]
 22%|██▏       | 50022/225000 [105:14:14<334:37:19,  6.88s/it]
 22%|██▏       | 50023/225000 [105:14:22<358:53:23,  7.38s/it]
 22%|██▏       | 50024/225000 [105:14:27<324:22:24,  6.67s/it]
 22%|██▏       | 50025/225000 [105:14:33<303:21:45,  6.24s/it]
 22%|██▏       | 50026/225000 [105:14:38<296:00:17,  6.09s/it]
 22%|██▏       | 50027/225000 [105:14:44<288:44:34,  5.94s/it]
 22%|██▏       | 50028/225000 [105:14:49<269:46:13,  5.55s/it]
 22%|██▏       | 50029/225000 [105:14:54<264:43:58,  5.45s/it]
 22%|██▏       | 50030/225000 [105:14:59<262:00:11,  5.39s/it]
 22%|██▏       | 50031/225000 [105:15:12<370:41:51,  7.63s/it]
 22%|██▏       | 50032/225000 [105:15:21<392:37:36,  8.08s/it]
 22%|██▏       | 50033/225000 [105:15:26<342:28:59,  7.05s/it]
 22%|██▏       | 50034/225000 [105:15:30<308:10:08,  6.34s/it]
 22%|██▏       | 50035/225000 [105:15:35<283:19:48,  5.83s/it]
 22%|██▏       | 50036/225000 [105:15:45<338:58:17,  6.97s/it]
 22%|██▏       | 50037/225000 [105:15:55<382:05:28,  7.86s/it]
 22%|██▏       | 50038/225000 [105:15:59<328:00:29,  6.75s/it]
 22%|██▏       | 50039/225000 [105:16:03<297:52:18,  6.13s/it]
 22%|██▏       | 50040/225000 [105:16:09<287:17:05,  5.91s/it]
 22%|██▏       | 50041/225000 [105:16:14<274:52:47,  5.66s/it]
 22%|██▏       | 50042/225000 [105:16:27<385:13:32,  7.93s/it]
 22%|██▏       | 50043/225000 [105:16:35<382:12:06,  7.86s/it]
 22%|██▏       | 50044/225000 [105:16:43<391:44:31,  8.06s/it]
 22%|██▏       | 50045/225000 [105:16:49<351:54:02,  7.24s/it]
 22%|██▏       | 50046/225000 [105:16:54<318:20:21,  6.55s/it]
 22%|██▏       | 50047/225000 [105:16:58<293:20:51,  6.04s/it]
 22%|██▏       | 50048/225000 [105:17:03<278:15:50,  5.73s/it]
 22%|██▏       | 50049/225000 [105:17:09<277:08:03,  5.70s/it]
 22%|██▏       | 50050/225000 [105:17:18<321:31:16,  6.62s/it]
                                                              
{'loss': '0.7062', 'grad_norm': '0.6305', 'learning_rate': '4.584e-05', 'epoch': '0.2224', 'num_input_tokens_seen': 176237952, 'train_runtime': '3.79e+05', 'train_tokens_per_second': '465'}

 22%|██▏       | 50050/225000 [105:17:18<321:31:16,  6.62s/it]
 22%|██▏       | 50051/225000 [105:17:28<375:33:36,  7.73s/it]
 22%|██▏       | 50052/225000 [105:17:34<351:11:12,  7.23s/it]
 22%|██▏       | 50053/225000 [105:17:42<353:23:09,  7.27s/it]
 22%|██▏       | 50054/225000 [105:17:48<341:07:17,  7.02s/it]
 22%|██▏       | 50055/225000 [105:17:57<363:01:49,  7.47s/it]
 22%|██▏       | 50056/225000 [105:18:02<332:14:41,  6.84s/it]
 22%|██▏       | 50057/225000 [105:18:09<335:53:44,  6.91s/it]
 22%|██▏       | 50058/225000 [105:18:15<327:23:21,  6.74s/it]
 22%|██▏       | 50059/225000 [105:18:21<308:34:37,  6.35s/it]
 22%|██▏       | 50060/225000 [105:18:26<290:17:11,  5.97s/it]
 22%|██▏       | 50061/225000 [105:18:31<282:42:14,  5.82s/it]
 22%|██▏       | 50062/225000 [105:18:44<388:00:35,  7.98s/it]
 22%|██▏       | 50063/225000 [105:18:51<374:37:55,  7.71s/it]
 22%|██▏       | 50064/225000 [105:18:55<311:31:39,  6.41s/it]
 22%|██▏       | 50065/225000 [105:19:04<353:51:36,  7.28s/it]
 22%|██▏       | 50066/225000 [105:19:08<310:24:10,  6.39s/it]
 22%|██▏       | 50067/225000 [105:19:13<283:01:34,  5.82s/it]
 22%|██▏       | 50068/225000 [105:19:18<264:22:57,  5.44s/it]
 22%|██▏       | 50069/225000 [105:19:22<253:03:06,  5.21s/it]
 22%|██▏       | 50070/225000 [105:19:35<359:10:48,  7.39s/it]
 22%|██▏       | 50071/225000 [105:19:44<384:23:50,  7.91s/it]
 22%|██▏       | 50072/225000 [105:19:49<347:56:13,  7.16s/it]
 22%|██▏       | 50073/225000 [105:19:57<357:14:40,  7.35s/it]
 22%|██▏       | 50074/225000 [105:20:02<320:57:31,  6.61s/it]
 22%|██▏       | 50075/225000 [105:20:07<299:02:35,  6.15s/it]
 22%|██▏       | 50076/225000 [105:20:12<282:10:35,  5.81s/it]
 22%|██▏       | 50077/225000 [105:20:17<272:41:42,  5.61s/it]
 22%|██▏       | 50078/225000 [105:20:22<268:16:30,  5.52s/it]
 22%|██▏       | 50079/225000 [105:20:34<362:29:32,  7.46s/it]
 22%|██▏       | 50080/225000 [105:20:44<388:06:01,  7.99s/it]
 22%|██▏       | 50081/225000 [105:20:49<348:23:58,  7.17s/it]
 22%|██▏       | 50082/225000 [105:20:54<320:42:01,  6.60s/it]
 22%|██▏       | 50083/225000 [105:20:59<301:21:14,  6.20s/it]
 22%|██▏       | 50084/225000 [105:21:05<287:34:02,  5.92s/it]
 22%|██▏       | 50085/225000 [105:21:10<275:50:10,  5.68s/it]
 22%|██▏       | 50086/225000 [105:21:22<378:12:31,  7.78s/it]
 22%|██▏       | 50087/225000 [105:21:29<360:41:49,  7.42s/it]
 22%|██▏       | 50088/225000 [105:21:33<316:04:08,  6.51s/it]
 22%|██▏       | 50089/225000 [105:21:42<339:24:56,  6.99s/it]
 22%|██▏       | 50090/225000 [105:21:50<362:53:46,  7.47s/it]
 22%|██▏       | 50091/225000 [105:21:54<315:28:03,  6.49s/it]
 22%|██▏       | 50092/225000 [105:22:05<376:48:11,  7.76s/it]
 22%|██▏       | 50093/225000 [105:22:13<381:08:11,  7.84s/it]
 22%|██▏       | 50094/225000 [105:22:18<331:13:01,  6.82s/it]
 22%|██▏       | 50095/225000 [105:22:24<323:25:56,  6.66s/it]
 22%|██▏       | 50096/225000 [105:22:33<363:04:27,  7.47s/it]
 22%|██▏       | 50097/225000 [105:22:37<307:47:14,  6.34s/it]
 22%|██▏       | 50098/225000 [105:22:41<276:36:36,  5.69s/it]
 22%|██▏       | 50099/225000 [105:22:48<289:13:36,  5.95s/it]
 22%|██▏       | 50100/225000 [105:22:59<365:01:14,  7.51s/it]
                                                              
{'loss': '0.6973', 'grad_norm': '0.7673', 'learning_rate': '4.583e-05', 'epoch': '0.2227', 'num_input_tokens_seen': 176386120, 'train_runtime': '3.794e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50100/225000 [105:22:59<365:01:14,  7.51s/it]
 22%|██▏       | 50101/225000 [105:23:02<308:20:14,  6.35s/it]
 22%|██▏       | 50102/225000 [105:23:11<346:39:26,  7.14s/it]
 22%|██▏       | 50103/225000 [105:23:15<297:07:34,  6.12s/it]
 22%|██▏       | 50104/225000 [105:23:26<367:23:50,  7.56s/it]
 22%|██▏       | 50105/225000 [105:23:30<312:50:05,  6.44s/it]
 22%|██▏       | 50106/225000 [105:23:40<365:37:31,  7.53s/it]
 22%|██▏       | 50107/225000 [105:23:43<307:18:29,  6.33s/it]
 22%|██▏       | 50108/225000 [105:23:53<351:43:54,  7.24s/it]
 22%|██▏       | 50109/225000 [105:23:58<319:26:15,  6.58s/it]
 22%|██▏       | 50110/225000 [105:24:04<319:41:44,  6.58s/it]
 22%|██▏       | 50111/225000 [105:24:10<306:22:00,  6.31s/it]
 22%|██▏       | 50112/225000 [105:24:19<338:07:02,  6.96s/it]
 22%|██▏       | 50113/225000 [105:24:23<305:55:53,  6.30s/it]
 22%|██▏       | 50114/225000 [105:24:28<282:18:02,  5.81s/it]
 22%|██▏       | 50115/225000 [105:24:34<280:07:38,  5.77s/it]
 22%|██▏       | 50116/225000 [105:24:46<379:29:27,  7.81s/it]
 22%|██▏       | 50117/225000 [105:24:51<335:59:27,  6.92s/it]
 22%|██▏       | 50118/225000 [105:24:57<314:08:57,  6.47s/it]
 22%|██▏       | 50119/225000 [105:25:02<298:47:56,  6.15s/it]
 22%|██▏       | 50120/225000 [105:25:07<286:01:55,  5.89s/it]
 22%|██▏       | 50121/225000 [105:25:17<347:06:30,  7.15s/it]
 22%|██▏       | 50122/225000 [105:25:27<383:12:02,  7.89s/it]
 22%|██▏       | 50123/225000 [105:25:31<323:25:07,  6.66s/it]
 22%|██▏       | 50124/225000 [105:25:41<369:22:24,  7.60s/it]
 22%|██▏       | 50125/225000 [105:25:49<383:50:38,  7.90s/it]
 22%|██▏       | 50126/225000 [105:25:53<328:26:13,  6.76s/it]
 22%|██▏       | 50127/225000 [105:25:58<300:51:01,  6.19s/it]
 22%|██▏       | 50128/225000 [105:26:03<277:18:56,  5.71s/it]
 22%|██▏       | 50129/225000 [105:26:08<270:38:08,  5.57s/it]
 22%|██▏       | 50130/225000 [105:26:14<274:42:29,  5.66s/it]
 22%|██▏       | 50131/225000 [105:26:27<390:58:30,  8.05s/it]
 22%|██▏       | 50132/225000 [105:26:35<381:20:48,  7.85s/it]
 22%|██▏       | 50133/225000 [105:26:43<381:09:04,  7.85s/it]
 22%|██▏       | 50134/225000 [105:26:48<339:12:28,  6.98s/it]
 22%|██▏       | 50135/225000 [105:26:53<317:28:22,  6.54s/it]
 22%|██▏       | 50136/225000 [105:26:58<299:05:31,  6.16s/it]
 22%|██▏       | 50137/225000 [105:27:04<288:38:08,  5.94s/it]
 22%|██▏       | 50138/225000 [105:27:09<282:26:37,  5.81s/it]
 22%|██▏       | 50139/225000 [105:27:20<349:25:00,  7.19s/it]
 22%|██▏       | 50140/225000 [105:27:30<400:12:54,  8.24s/it]
 22%|██▏       | 50141/225000 [105:27:34<329:14:06,  6.78s/it]
 22%|██▏       | 50142/225000 [105:27:43<364:52:43,  7.51s/it]
 22%|██▏       | 50143/225000 [105:27:51<367:25:45,  7.56s/it]
 22%|██▏       | 50144/225000 [105:27:56<337:45:26,  6.95s/it]
 22%|██▏       | 50145/225000 [105:28:02<318:31:57,  6.56s/it]
 22%|██▏       | 50146/225000 [105:28:07<299:45:25,  6.17s/it]
 22%|██▏       | 50147/225000 [105:28:12<287:48:44,  5.93s/it]
 22%|██▏       | 50148/225000 [105:28:18<279:54:34,  5.76s/it]
 22%|██▏       | 50149/225000 [105:28:31<391:07:59,  8.05s/it]
 22%|██▏       | 50150/225000 [105:28:35<334:45:40,  6.89s/it]
                                                              
{'loss': '0.7183', 'grad_norm': '0.6191', 'learning_rate': '4.582e-05', 'epoch': '0.2229', 'num_input_tokens_seen': 176536864, 'train_runtime': '3.797e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50150/225000 [105:28:35<334:45:40,  6.89s/it]
 22%|██▏       | 50151/225000 [105:28:43<337:36:40,  6.95s/it]
 22%|██▏       | 50152/225000 [105:28:47<302:31:15,  6.23s/it]
 22%|██▏       | 50153/225000 [105:28:56<345:18:13,  7.11s/it]
 22%|██▏       | 50154/225000 [105:29:04<360:56:20,  7.43s/it]
 22%|██▏       | 50155/225000 [105:29:09<314:58:27,  6.49s/it]
 22%|██▏       | 50156/225000 [105:29:14<290:47:13,  5.99s/it]
 22%|██▏       | 50157/225000 [105:29:19<277:12:14,  5.71s/it]
 22%|██▏       | 50158/225000 [105:29:24<271:51:58,  5.60s/it]
 22%|██▏       | 50159/225000 [105:29:37<379:15:44,  7.81s/it]
 22%|██▏       | 50160/225000 [105:29:43<352:10:40,  7.25s/it]
 22%|██▏       | 50161/225000 [105:29:49<331:48:41,  6.83s/it]
 22%|██▏       | 50162/225000 [105:29:54<312:49:35,  6.44s/it]
 22%|██▏       | 50163/225000 [105:30:05<383:22:06,  7.89s/it]
 22%|██▏       | 50164/225000 [105:30:09<320:54:39,  6.61s/it]
 22%|██▏       | 50165/225000 [105:30:17<337:59:09,  6.96s/it]
 22%|██▏       | 50166/225000 [105:30:21<303:10:43,  6.24s/it]
 22%|██▏       | 50167/225000 [105:30:26<280:46:56,  5.78s/it]
 22%|██▏       | 50168/225000 [105:30:34<316:37:25,  6.52s/it]
 22%|██▏       | 50169/225000 [105:30:45<381:25:15,  7.85s/it]
 22%|██▏       | 50170/225000 [105:30:51<355:41:05,  7.32s/it]
 22%|██▏       | 50171/225000 [105:30:56<315:08:11,  6.49s/it]
 22%|██▏       | 50172/225000 [105:31:06<365:22:43,  7.52s/it]
 22%|██▏       | 50173/225000 [105:31:12<347:09:52,  7.15s/it]
 22%|██▏       | 50174/225000 [105:31:17<313:25:17,  6.45s/it]
 22%|██▏       | 50175/225000 [105:31:23<301:19:23,  6.20s/it]
 22%|██▏       | 50176/225000 [105:31:28<286:15:52,  5.89s/it]
 22%|██▏       | 50177/225000 [105:31:33<271:45:16,  5.60s/it]
 22%|██▏       | 50178/225000 [105:31:40<290:46:07,  5.99s/it]
 22%|██▏       | 50179/225000 [105:31:53<400:16:49,  8.24s/it]
 22%|██▏       | 50180/225000 [105:32:00<373:02:33,  7.68s/it]
 22%|██▏       | 50181/225000 [105:32:06<352:11:45,  7.25s/it]
 22%|██▏       | 50182/225000 [105:32:11<324:24:15,  6.68s/it]
 22%|██▏       | 50183/225000 [105:32:17<310:08:12,  6.39s/it]
 22%|██▏       | 50184/225000 [105:32:27<369:33:38,  7.61s/it]
 22%|██▏       | 50185/225000 [105:32:31<319:13:52,  6.57s/it]
 22%|██▏       | 50186/225000 [105:32:42<372:15:16,  7.67s/it]
 22%|██▏       | 50187/225000 [105:32:46<324:39:07,  6.69s/it]
 22%|██▏       | 50188/225000 [105:32:54<346:17:42,  7.13s/it]
 22%|██▏       | 50189/225000 [105:32:59<311:01:05,  6.41s/it]
 22%|██▏       | 50190/225000 [105:33:07<339:01:30,  6.98s/it]
 22%|██▏       | 50191/225000 [105:33:12<308:36:44,  6.36s/it]
 22%|██▏       | 50192/225000 [105:33:17<287:23:21,  5.92s/it]
 22%|██▏       | 50193/225000 [105:33:23<281:20:53,  5.79s/it]
 22%|██▏       | 50194/225000 [105:33:28<270:12:01,  5.56s/it]
 22%|██▏       | 50195/225000 [105:33:38<345:17:54,  7.11s/it]
 22%|██▏       | 50196/225000 [105:33:45<339:00:31,  6.98s/it]
 22%|██▏       | 50197/225000 [105:33:52<337:41:30,  6.95s/it]
 22%|██▏       | 50198/225000 [105:33:59<339:13:51,  6.99s/it]
 22%|██▏       | 50199/225000 [105:34:04<314:12:54,  6.47s/it]
 22%|██▏       | 50200/225000 [105:34:12<330:46:00,  6.81s/it]
                                                              
{'loss': '0.7036', 'grad_norm': '0.7629', 'learning_rate': '4.581e-05', 'epoch': '0.2231', 'num_input_tokens_seen': 176685512, 'train_runtime': '3.801e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50200/225000 [105:34:12<330:46:00,  6.81s/it]
 22%|██▏       | 50201/225000 [105:34:16<290:53:16,  5.99s/it]
 22%|██▏       | 50202/225000 [105:34:24<324:35:51,  6.69s/it]
 22%|██▏       | 50203/225000 [105:34:29<301:55:56,  6.22s/it]
 22%|██▏       | 50204/225000 [105:34:34<277:35:57,  5.72s/it]
 22%|██▏       | 50205/225000 [105:34:44<340:46:35,  7.02s/it]
 22%|██▏       | 50206/225000 [105:34:49<310:18:22,  6.39s/it]
 22%|██▏       | 50207/225000 [105:34:53<273:52:21,  5.64s/it]
 22%|██▏       | 50208/225000 [105:35:03<338:26:56,  6.97s/it]
 22%|██▏       | 50209/225000 [105:35:09<332:21:40,  6.85s/it]
 22%|██▏       | 50210/225000 [105:35:14<299:17:42,  6.16s/it]
 22%|██▏       | 50211/225000 [105:35:22<327:48:59,  6.75s/it]
 22%|██▏       | 50212/225000 [105:35:26<284:33:35,  5.86s/it]
 22%|██▏       | 50213/225000 [105:35:30<266:20:05,  5.49s/it]
 22%|██▏       | 50214/225000 [105:35:43<367:37:31,  7.57s/it]
 22%|██▏       | 50215/225000 [105:35:50<355:30:35,  7.32s/it]
 22%|██▏       | 50216/225000 [105:35:55<323:42:32,  6.67s/it]
 22%|██▏       | 50217/225000 [105:36:01<310:23:27,  6.39s/it]
 22%|██▏       | 50218/225000 [105:36:06<301:34:29,  6.21s/it]
 22%|██▏       | 50219/225000 [105:36:12<291:51:40,  6.01s/it]
 22%|██▏       | 50220/225000 [105:36:18<289:41:27,  5.97s/it]
 22%|██▏       | 50221/225000 [105:36:23<282:37:56,  5.82s/it]
 22%|██▏       | 50222/225000 [105:36:31<314:34:42,  6.48s/it]
 22%|██▏       | 50223/225000 [105:36:42<370:00:03,  7.62s/it]
 22%|██▏       | 50224/225000 [105:36:51<393:36:51,  8.11s/it]
 22%|██▏       | 50225/225000 [105:36:55<344:06:57,  7.09s/it]
 22%|██▏       | 50226/225000 [105:37:01<319:04:05,  6.57s/it]
 22%|██▏       | 50227/225000 [105:37:06<304:49:56,  6.28s/it]
 22%|██▏       | 50228/225000 [105:37:12<288:01:49,  5.93s/it]
 22%|██▏       | 50229/225000 [105:37:17<285:14:45,  5.88s/it]
 22%|██▏       | 50230/225000 [105:37:23<278:42:45,  5.74s/it]
 22%|██▏       | 50231/225000 [105:37:31<309:39:36,  6.38s/it]
 22%|██▏       | 50232/225000 [105:37:44<405:48:25,  8.36s/it]
 22%|██▏       | 50233/225000 [105:37:47<339:30:15,  6.99s/it]
 22%|██▏       | 50234/225000 [105:37:53<319:39:03,  6.58s/it]
 22%|██▏       | 50235/225000 [105:38:01<339:57:12,  7.00s/it]
 22%|██▏       | 50236/225000 [105:38:09<355:22:04,  7.32s/it]
 22%|██▏       | 50237/225000 [105:38:13<302:00:19,  6.22s/it]
 22%|██▏       | 50238/225000 [105:38:18<282:44:20,  5.82s/it]
 22%|██▏       | 50239/225000 [105:38:23<271:00:12,  5.58s/it]
 22%|██▏       | 50240/225000 [105:38:28<268:07:32,  5.52s/it]
 22%|██▏       | 50241/225000 [105:38:39<348:01:28,  7.17s/it]
 22%|██▏       | 50242/225000 [105:38:47<366:26:10,  7.55s/it]
 22%|██▏       | 50243/225000 [105:38:55<360:53:34,  7.43s/it]
 22%|██▏       | 50244/225000 [105:38:59<323:13:02,  6.66s/it]
 22%|██▏       | 50245/225000 [105:39:07<332:03:17,  6.84s/it]
 22%|██▏       | 50246/225000 [105:39:13<326:04:23,  6.72s/it]
 22%|██▏       | 50247/225000 [105:39:19<318:32:59,  6.56s/it]
 22%|██▏       | 50248/225000 [105:39:23<281:39:35,  5.80s/it]
 22%|██▏       | 50249/225000 [105:39:33<339:35:16,  7.00s/it]
 22%|██▏       | 50250/225000 [105:39:41<353:37:58,  7.29s/it]
                                                              
{'loss': '0.6995', 'grad_norm': '0.7086', 'learning_rate': '4.58e-05', 'epoch': '0.2233', 'num_input_tokens_seen': 176833424, 'train_runtime': '3.804e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50250/225000 [105:39:41<353:37:58,  7.29s/it]
 22%|██▏       | 50251/225000 [105:39:48<353:11:22,  7.28s/it]
 22%|██▏       | 50252/225000 [105:39:57<367:23:32,  7.57s/it]
 22%|██▏       | 50253/225000 [105:40:02<335:09:06,  6.90s/it]
 22%|██▏       | 50254/225000 [105:40:08<319:32:17,  6.58s/it]
 22%|██▏       | 50255/225000 [105:40:14<309:49:41,  6.38s/it]
 22%|██▏       | 50256/225000 [105:40:24<371:16:23,  7.65s/it]
 22%|██▏       | 50257/225000 [105:40:30<348:48:57,  7.19s/it]
 22%|██▏       | 50258/225000 [105:40:36<318:32:00,  6.56s/it]
 22%|██▏       | 50259/225000 [105:40:40<293:43:03,  6.05s/it]
 22%|██▏       | 50260/225000 [105:40:52<378:26:37,  7.80s/it]
 22%|██▏       | 50261/225000 [105:40:59<362:49:27,  7.47s/it]
 22%|██▏       | 50262/225000 [105:41:05<343:30:59,  7.08s/it]
 22%|██▏       | 50263/225000 [105:41:12<335:49:27,  6.92s/it]
 22%|██▏       | 50264/225000 [105:41:21<363:33:34,  7.49s/it]
 22%|██▏       | 50265/225000 [105:41:26<332:21:28,  6.85s/it]
 22%|██▏       | 50266/225000 [105:41:32<323:44:59,  6.67s/it]
 22%|██▏       | 50267/225000 [105:41:38<311:07:33,  6.41s/it]
 22%|██▏       | 50268/225000 [105:41:43<292:28:53,  6.03s/it]
 22%|██▏       | 50269/225000 [105:41:49<286:20:37,  5.90s/it]
 22%|██▏       | 50270/225000 [105:42:00<360:37:38,  7.43s/it]
 22%|██▏       | 50271/225000 [105:42:08<371:22:20,  7.65s/it]
 22%|██▏       | 50272/225000 [105:42:12<325:46:05,  6.71s/it]
 22%|██▏       | 50273/225000 [105:42:21<352:11:47,  7.26s/it]
 22%|██▏       | 50274/225000 [105:42:25<308:12:33,  6.35s/it]
 22%|██▏       | 50275/225000 [105:42:33<334:22:03,  6.89s/it]
 22%|██▏       | 50276/225000 [105:42:39<318:55:53,  6.57s/it]
 22%|██▏       | 50277/225000 [105:42:48<350:40:45,  7.23s/it]
 22%|██▏       | 50278/225000 [105:42:54<334:25:53,  6.89s/it]
 22%|██▏       | 50279/225000 [105:43:02<344:06:37,  7.09s/it]
 22%|██▏       | 50280/225000 [105:43:06<311:53:13,  6.43s/it]
 22%|██▏       | 50281/225000 [105:43:12<302:28:40,  6.23s/it]
 22%|██▏       | 50282/225000 [105:43:18<295:32:14,  6.09s/it]
 22%|██▏       | 50283/225000 [105:43:23<288:06:34,  5.94s/it]
 22%|██▏       | 50284/225000 [105:43:32<319:33:06,  6.58s/it]
 22%|██▏       | 50285/225000 [105:43:43<390:58:51,  8.06s/it]
 22%|██▏       | 50286/225000 [105:43:47<334:07:58,  6.88s/it]
 22%|██▏       | 50287/225000 [105:43:57<372:09:29,  7.67s/it]
 22%|██▏       | 50288/225000 [105:44:00<315:15:00,  6.50s/it]
 22%|██▏       | 50289/225000 [105:44:09<350:19:01,  7.22s/it]
 22%|██▏       | 50290/225000 [105:44:16<335:55:25,  6.92s/it]
 22%|██▏       | 50291/225000 [105:44:19<285:14:26,  5.88s/it]
 22%|██▏       | 50292/225000 [105:44:28<332:30:26,  6.85s/it]
 22%|██▏       | 50293/225000 [105:44:32<294:59:18,  6.08s/it]
 22%|██▏       | 50294/225000 [105:44:40<310:08:29,  6.39s/it]
 22%|██▏       | 50295/225000 [105:44:44<280:02:24,  5.77s/it]
 22%|██▏       | 50296/225000 [105:44:49<273:16:35,  5.63s/it]
 22%|██▏       | 50297/225000 [105:45:00<350:37:27,  7.23s/it]
 22%|██▏       | 50298/225000 [105:45:04<295:24:36,  6.09s/it]
 22%|██▏       | 50299/225000 [105:45:12<327:31:19,  6.75s/it]
 22%|██▏       | 50300/225000 [105:45:17<300:22:00,  6.19s/it]
                                                              
{'loss': '0.7105', 'grad_norm': '0.7759', 'learning_rate': '4.579e-05', 'epoch': '0.2236', 'num_input_tokens_seen': 176982016, 'train_runtime': '3.807e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50300/225000 [105:45:17<300:22:00,  6.19s/it]
 22%|██▏       | 50301/225000 [105:45:22<285:45:49,  5.89s/it]
 22%|██▏       | 50302/225000 [105:45:27<272:52:16,  5.62s/it]
 22%|██▏       | 50303/225000 [105:45:41<392:57:56,  8.10s/it]
 22%|██▏       | 50304/225000 [105:45:45<337:42:06,  6.96s/it]
 22%|██▏       | 50305/225000 [105:45:50<301:33:18,  6.21s/it]
 22%|██▏       | 50306/225000 [105:46:00<367:12:37,  7.57s/it]
 22%|██▏       | 50307/225000 [105:46:06<339:52:24,  7.00s/it]
 22%|██▏       | 50308/225000 [105:46:11<307:51:59,  6.34s/it]
 22%|██▏       | 50309/225000 [105:46:16<284:13:54,  5.86s/it]
 22%|██▏       | 50310/225000 [105:46:22<291:04:09,  6.00s/it]
 22%|██▏       | 50311/225000 [105:46:34<380:59:00,  7.85s/it]
 22%|██▏       | 50312/225000 [105:46:38<318:18:00,  6.56s/it]
 22%|██▏       | 50313/225000 [105:46:47<362:03:32,  7.46s/it]
 22%|██▏       | 50314/225000 [105:46:53<343:03:52,  7.07s/it]
 22%|██▏       | 50315/225000 [105:46:59<328:33:35,  6.77s/it]
 22%|██▏       | 50316/225000 [105:47:07<346:30:21,  7.14s/it]
 22%|██▏       | 50317/225000 [105:47:15<349:32:58,  7.20s/it]
 22%|██▏       | 50318/225000 [105:47:20<319:28:27,  6.58s/it]
 22%|██▏       | 50319/225000 [105:47:25<295:18:04,  6.09s/it]
 22%|██▏       | 50320/225000 [105:47:29<271:48:29,  5.60s/it]
 22%|██▏       | 50321/225000 [105:47:34<254:28:15,  5.24s/it]
 22%|██▏       | 50322/225000 [105:47:46<352:01:53,  7.26s/it]
 22%|██▏       | 50323/225000 [105:47:53<359:28:56,  7.41s/it]
 22%|██▏       | 50324/225000 [105:47:59<332:20:00,  6.85s/it]
 22%|██▏       | 50325/225000 [105:48:04<310:28:19,  6.40s/it]
 22%|██▏       | 50326/225000 [105:48:09<291:44:18,  6.01s/it]
 22%|██▏       | 50327/225000 [105:48:15<281:36:34,  5.80s/it]
 22%|██▏       | 50328/225000 [105:48:20<276:13:17,  5.69s/it]
 22%|██▏       | 50329/225000 [105:48:33<380:01:22,  7.83s/it]
 22%|██▏       | 50330/225000 [105:48:41<383:43:56,  7.91s/it]
 22%|██▏       | 50331/225000 [105:48:46<335:37:40,  6.92s/it]
 22%|██▏       | 50332/225000 [105:48:55<376:34:46,  7.76s/it]
 22%|██▏       | 50333/225000 [105:48:59<320:36:49,  6.61s/it]
 22%|██▏       | 50334/225000 [105:49:06<323:37:41,  6.67s/it]
 22%|██▏       | 50335/225000 [105:49:17<380:40:27,  7.85s/it]
 22%|██▏       | 50336/225000 [105:49:23<352:17:59,  7.26s/it]
 22%|██▏       | 50337/225000 [105:49:27<306:44:41,  6.32s/it]
 22%|██▏       | 50338/225000 [105:49:35<341:28:58,  7.04s/it]
 22%|██▏       | 50339/225000 [105:49:40<307:34:13,  6.34s/it]
 22%|██▏       | 50340/225000 [105:49:49<340:37:37,  7.02s/it]
 22%|██▏       | 50341/225000 [105:49:53<295:22:43,  6.09s/it]
 22%|██▏       | 50342/225000 [105:49:58<281:54:53,  5.81s/it]
 22%|██▏       | 50343/225000 [105:50:03<273:30:49,  5.64s/it]
 22%|██▏       | 50344/225000 [105:50:08<268:17:17,  5.53s/it]
 22%|██▏       | 50345/225000 [105:50:20<358:24:58,  7.39s/it]
 22%|██▏       | 50346/225000 [105:50:28<370:17:41,  7.63s/it]
 22%|██▏       | 50347/225000 [105:50:34<342:44:55,  7.06s/it]
 22%|██▏       | 50348/225000 [105:50:40<320:07:52,  6.60s/it]
 22%|██▏       | 50349/225000 [105:50:45<303:10:50,  6.25s/it]
 22%|██▏       | 50350/225000 [105:50:51<297:03:00,  6.12s/it]
                                                              
{'loss': '0.6968', 'grad_norm': '0.7544', 'learning_rate': '4.578e-05', 'epoch': '0.2238', 'num_input_tokens_seen': 177131568, 'train_runtime': '3.811e+05', 'train_tokens_per_second': '464.9'}

 22%|██▏       | 50350/225000 [105:50:51<297:03:00,  6.12s/it]
 22%|██▏       | 50351/225000 [105:50:56<290:23:53,  5.99s/it]
 22%|██▏       | 50352/225000 [105:51:02<283:01:13,  5.83s/it]
 22%|██▏       | 50353/225000 [105:51:08<282:08:17,  5.82s/it]
 22%|██▏       | 50354/225000 [105:51:13<276:05:50,  5.69s/it]
 22%|██▏       | 50355/225000 [105:51:23<340:39:17,  7.02s/it]
 22%|██▏       | 50356/225000 [105:51:34<395:41:40,  8.16s/it]
 22%|██▏       | 50357/225000 [105:51:42<396:16:35,  8.17s/it]
 22%|██▏       | 50358/225000 [105:51:46<337:33:33,  6.96s/it]
 22%|██▏       | 50359/225000 [105:51:51<307:45:03,  6.34s/it]
 22%|██▏       | 50360/225000 [105:52:03<390:17:39,  8.05s/it]
 22%|██▏       | 50361/225000 [105:52:07<329:20:54,  6.79s/it]
 22%|██▏       | 50362/225000 [105:52:13<309:23:31,  6.38s/it]
 22%|██▏       | 50363/225000 [105:52:19<307:00:33,  6.33s/it]
 22%|██▏       | 50364/225000 [105:52:24<296:34:20,  6.11s/it]
 22%|██▏       | 50365/225000 [105:52:30<293:25:55,  6.05s/it]
 22%|██▏       | 50366/225000 [105:52:36<291:43:15,  6.01s/it]
 22%|██▏       | 50367/225000 [105:52:42<290:28:30,  5.99s/it]
 22%|██▏       | 50368/225000 [105:52:56<406:49:17,  8.39s/it]
 22%|██▏       | 50369/225000 [105:53:02<376:56:27,  7.77s/it]
 22%|██▏       | 50370/225000 [105:53:08<337:24:00,  6.96s/it]
 22%|██▏       | 50371/225000 [105:53:18<383:03:47,  7.90s/it]
 22%|██▏       | 50372/225000 [105:53:23<339:40:07,  7.00s/it]
 22%|██▏       | 50373/225000 [105:53:28<311:51:42,  6.43s/it]
 22%|██▏       | 50374/225000 [105:53:33<293:29:22,  6.05s/it]
 22%|██▏       | 50375/225000 [105:53:44<367:25:16,  7.57s/it]
 22%|██▏       | 50376/225000 [105:53:52<369:31:42,  7.62s/it]
 22%|██▏       | 50377/225000 [105:53:56<326:30:15,  6.73s/it]
 22%|██▏       | 50378/225000 [105:54:05<350:53:22,  7.23s/it]
 22%|██▏       | 50379/225000 [105:54:13<372:31:08,  7.68s/it]
 22%|██▏       | 50380/225000 [105:54:20<351:50:09,  7.25s/it]
 22%|██▏       | 50381/225000 [105:54:25<322:45:55,  6.65s/it]
 22%|██▏       | 50382/225000 [105:54:30<300:34:08,  6.20s/it]
 22%|██▏       | 50383/225000 [105:54:35<287:19:02,  5.92s/it]
 22%|██▏       | 50384/225000 [105:54:41<279:20:47,  5.76s/it]
 22%|██▏       | 50385/225000 [105:54:53<379:45:29,  7.83s/it]
 22%|██▏       | 50386/225000 [105:54:58<332:19:29,  6.85s/it]
 22%|██▏       | 50387/225000 [105:55:04<316:56:46,  6.53s/it]
 22%|██▏       | 50388/225000 [105:55:10<305:38:51,  6.30s/it]
 22%|██▏       | 50389/225000 [105:55:15<294:02:56,  6.06s/it]
 22%|██▏       | 50390/225000 [105:55:20<284:27:18,  5.86s/it]
 22%|██▏       | 50391/225000 [105:55:26<274:43:49,  5.66s/it]
 22%|██▏       | 50392/225000 [105:55:37<357:28:05,  7.37s/it]
 22%|██▏       | 50393/225000 [105:55:44<350:42:52,  7.23s/it]
 22%|██▏       | 50394/225000 [105:55:51<343:28:07,  7.08s/it]
 22%|██▏       | 50395/225000 [105:55:55<308:31:44,  6.36s/it]
 22%|██▏       | 50396/225000 [105:56:00<288:07:04,  5.94s/it]
 22%|██▏       | 50397/225000 [105:56:06<280:09:41,  5.78s/it]
 22%|██▏       | 50398/225000 [105:56:19<387:38:45,  7.99s/it]
 22%|██▏       | 50399/225000 [105:56:26<373:53:23,  7.71s/it]
 22%|██▏       | 50400/225000 [105:56:31<331:37:22,  6.84s/it]
                                                              
{'loss': '0.6951', 'grad_norm': '0.7486', 'learning_rate': '4.577e-05', 'epoch': '0.224', 'num_input_tokens_seen': 177282688, 'train_runtime': '3.814e+05', 'train_tokens_per_second': '464.8'}

 22%|██▏       | 50400/225000 [105:56:31<331:37:22,  6.84s/it]
 22%|██▏       | 50401/225000 [105:56:39<349:21:31,  7.20s/it]
 22%|██▏       | 50402/225000 [105:56:43<313:33:24,  6.47s/it]
 22%|██▏       | 50403/225000 [105:56:49<298:11:01,  6.15s/it]
 22%|██▏       | 50404/225000 [105:56:54<283:44:31,  5.85s/it]
 22%|██▏       | 50405/225000 [105:57:05<356:11:09,  7.34s/it]
 22%|██▏       | 50406/225000 [105:57:13<371:36:36,  7.66s/it]
 22%|██▏       | 50407/225000 [105:57:20<365:04:35,  7.53s/it]
 22%|██▏       | 50408/225000 [105:57:25<325:17:04,  6.71s/it]
 22%|██▏       | 50409/225000 [105:57:31<306:21:27,  6.32s/it]
 22%|██▏       | 50410/225000 [105:57:35<283:58:24,  5.86s/it]
 22%|██▏       | 50411/225000 [105:57:46<353:16:20,  7.28s/it]
 22%|██▏       | 50412/225000 [105:57:54<357:42:07,  7.38s/it]
 22%|██▏       | 50413/225000 [105:57:59<322:41:42,  6.65s/it]
 22%|██▏       | 50414/225000 [105:58:06<339:07:28,  6.99s/it]
 22%|██▏       | 50415/225000 [105:58:11<298:14:59,  6.15s/it]
 22%|██▏       | 50416/225000 [105:58:15<278:31:02,  5.74s/it]
 22%|██▏       | 50417/225000 [105:58:25<339:37:03,  7.00s/it]
 22%|██▏       | 50418/225000 [105:58:34<356:32:18,  7.35s/it]
 22%|██▏       | 50419/225000 [105:58:42<367:01:08,  7.57s/it]
 22%|██▏       | 50420/225000 [105:58:46<316:44:29,  6.53s/it]
 22%|██▏       | 50421/225000 [105:58:50<282:54:18,  5.83s/it]
 22%|██▏       | 50422/225000 [105:59:01<355:51:24,  7.34s/it]
 22%|██▏       | 50423/225000 [105:59:06<328:10:09,  6.77s/it]
 22%|██▏       | 50424/225000 [105:59:11<302:07:27,  6.23s/it]
 22%|██▏       | 50425/225000 [105:59:16<287:31:46,  5.93s/it]
 22%|██▏       | 50426/225000 [105:59:21<270:14:39,  5.57s/it]
 22%|██▏       | 50427/225000 [105:59:26<265:38:41,  5.48s/it]
 22%|██▏       | 50428/225000 [105:59:40<388:43:33,  8.02s/it]
 22%|██▏       | 50429/225000 [105:59:46<353:03:26,  7.28s/it]
 22%|██▏       | 50430/225000 [105:59:51<324:50:46,  6.70s/it]
 22%|██▏       | 50431/225000 [105:59:57<305:23:27,  6.30s/it]
 22%|██▏       | 50432/225000 [106:00:02<299:11:49,  6.17s/it]
 22%|██▏       | 50433/225000 [106:00:09<298:49:51,  6.16s/it]
 22%|██▏       | 50434/225000 [106:00:15<301:35:12,  6.22s/it]
 22%|██▏       | 50435/225000 [106:00:28<405:11:19,  8.36s/it]
 22%|██▏       | 50436/225000 [106:00:33<347:52:19,  7.17s/it]
 22%|██▏       | 50437/225000 [106:00:38<323:37:41,  6.67s/it]
 22%|██▏       | 50438/225000 [106:00:50<394:53:19,  8.14s/it]
 22%|██▏       | 50439/225000 [106:00:55<349:18:51,  7.20s/it]
 22%|██▏       | 50440/225000 [106:01:03<357:26:39,  7.37s/it]
 22%|██▏       | 50441/225000 [106:01:12<380:36:50,  7.85s/it]
 22%|██▏       | 50442/225000 [106:01:16<331:08:41,  6.83s/it]
 22%|██▏       | 50443/225000 [106:01:21<301:57:23,  6.23s/it]
 22%|██▏       | 50444/225000 [106:01:26<279:28:00,  5.76s/it]
 22%|██▏       | 50445/225000 [106:01:31<275:57:11,  5.69s/it]
 22%|██▏       | 50446/225000 [106:01:44<386:43:51,  7.98s/it]
 22%|██▏       | 50447/225000 [106:01:51<364:16:26,  7.51s/it]
 22%|██▏       | 50448/225000 [106:01:58<363:13:21,  7.49s/it]
 22%|██▏       | 50449/225000 [106:02:02<308:32:03,  6.36s/it]
 22%|██▏       | 50450/225000 [106:02:07<294:57:05,  6.08s/it]
                                                              
{'loss': '0.6963', 'grad_norm': '0.8169', 'learning_rate': '4.576e-05', 'epoch': '0.2242', 'num_input_tokens_seen': 177432960, 'train_runtime': '3.817e+05', 'train_tokens_per_second': '464.8'}

 22%|██▏       | 50450/225000 [106:02:07<294:57:05,  6.08s/it]
 22%|██▏       | 50451/225000 [106:02:17<351:50:45,  7.26s/it]
 22%|██▏       | 50452/225000 [106:02:21<303:17:48,  6.26s/it]
 22%|██▏       | 50453/225000 [106:02:28<315:36:24,  6.51s/it]
 22%|██▏       | 50454/225000 [106:02:33<290:48:37,  6.00s/it]
 22%|██▏       | 50455/225000 [106:02:38<271:59:57,  5.61s/it]
 22%|██▏       | 50456/225000 [106:02:43<260:27:47,  5.37s/it]
 22%|██▏       | 50457/225000 [106:02:55<359:09:03,  7.41s/it]
 22%|██▏       | 50458/225000 [106:03:01<345:11:28,  7.12s/it]
 22%|██▏       | 50459/225000 [106:03:07<319:20:04,  6.59s/it]
 22%|██▏       | 50460/225000 [106:03:13<319:56:37,  6.60s/it]
 22%|██▏       | 50461/225000 [106:03:19<300:46:41,  6.20s/it]
 22%|██▏       | 50462/225000 [106:03:24<290:07:03,  5.98s/it]
 22%|██▏       | 50463/225000 [106:03:29<282:17:57,  5.82s/it]
 22%|██▏       | 50464/225000 [106:03:39<341:20:16,  7.04s/it]
 22%|██▏       | 50465/225000 [106:03:49<384:13:10,  7.93s/it]
 22%|██▏       | 50466/225000 [106:03:53<327:33:59,  6.76s/it]
 22%|██▏       | 50467/225000 [106:03:59<313:54:42,  6.47s/it]
 22%|██▏       | 50468/225000 [106:04:11<385:52:42,  7.96s/it]
 22%|██▏       | 50469/225000 [106:04:14<324:56:27,  6.70s/it]
 22%|██▏       | 50470/225000 [106:04:22<340:48:40,  7.03s/it]
 22%|██▏       | 50471/225000 [106:04:28<319:32:03,  6.59s/it]
 22%|██▏       | 50472/225000 [106:04:35<328:20:25,  6.77s/it]
 22%|██▏       | 50473/225000 [106:04:40<299:30:58,  6.18s/it]
 22%|██▏       | 50474/225000 [106:04:44<277:19:46,  5.72s/it]
 22%|██▏       | 50475/225000 [106:04:49<257:20:09,  5.31s/it]
 22%|██▏       | 50476/225000 [106:05:01<364:46:16,  7.52s/it]
 22%|██▏       | 50477/225000 [106:05:06<320:08:15,  6.60s/it]
 22%|██▏       | 50478/225000 [106:05:11<295:46:25,  6.10s/it]
 22%|██▏       | 50479/225000 [106:05:16<278:31:49,  5.75s/it]
 22%|██▏       | 50480/225000 [106:05:21<266:37:56,  5.50s/it]
 22%|██▏       | 50481/225000 [106:05:29<308:10:03,  6.36s/it]
 22%|██▏       | 50482/225000 [106:05:40<372:33:23,  7.69s/it]
 22%|██▏       | 50483/225000 [106:05:43<313:27:16,  6.47s/it]
 22%|██▏       | 50484/225000 [106:05:48<290:02:16,  5.98s/it]
 22%|██▏       | 50485/225000 [106:05:53<271:53:05,  5.61s/it]
 22%|██▏       | 50486/225000 [106:05:58<267:58:49,  5.53s/it]
 22%|██▏       | 50487/225000 [106:06:08<324:59:01,  6.70s/it]
 22%|██▏       | 50488/225000 [106:06:18<382:07:13,  7.88s/it]
 22%|██▏       | 50489/225000 [106:06:24<348:29:22,  7.19s/it]
 22%|██▏       | 50490/225000 [106:06:30<336:07:58,  6.93s/it]
 22%|██▏       | 50491/225000 [106:06:36<318:38:06,  6.57s/it]
 22%|██▏       | 50492/225000 [106:06:44<341:22:09,  7.04s/it]
 22%|██▏       | 50493/225000 [106:06:52<348:17:41,  7.19s/it]
 22%|██▏       | 50494/225000 [106:06:57<321:36:45,  6.63s/it]
 22%|██▏       | 50495/225000 [106:07:02<301:54:28,  6.23s/it]
 22%|██▏       | 50496/225000 [106:07:08<289:09:50,  5.97s/it]
 22%|██▏       | 50497/225000 [106:07:13<285:47:09,  5.90s/it]
 22%|██▏       | 50498/225000 [106:07:19<277:45:17,  5.73s/it]
 22%|██▏       | 50499/225000 [106:07:28<327:35:09,  6.76s/it]
 22%|██▏       | 50500/225000 [106:07:39<395:08:52,  8.15s/it]
                                                              
{'loss': '0.6946', 'grad_norm': '0.6953', 'learning_rate': '4.575e-05', 'epoch': '0.2244', 'num_input_tokens_seen': 177580032, 'train_runtime': '3.821e+05', 'train_tokens_per_second': '464.8'}

 22%|██▏       | 50500/225000 [106:07:39<395:08:52,  8.15s/it]
 22%|██▏       | 50501/225000 [106:07:44<348:10:24,  7.18s/it]
 22%|██▏       | 50502/225000 [106:07:49<311:19:30,  6.42s/it]
 22%|██▏       | 50503/225000 [106:08:02<406:37:30,  8.39s/it]
 22%|██▏       | 50504/225000 [106:08:07<365:22:51,  7.54s/it]
 22%|██▏       | 50505/225000 [106:08:16<373:56:00,  7.71s/it]
 22%|██▏       | 50506/225000 [106:08:20<332:02:50,  6.85s/it]
 22%|██▏       | 50507/225000 [106:08:25<305:26:13,  6.30s/it]
 22%|██▏       | 50508/225000 [106:08:38<393:01:15,  8.11s/it]
 22%|██▏       | 50509/225000 [106:08:42<340:01:34,  7.02s/it]
 22%|██▏       | 50510/225000 [106:08:50<343:58:55,  7.10s/it]
 22%|██▏       | 50511/225000 [106:08:58<360:19:22,  7.43s/it]
 22%|██▏       | 50512/225000 [106:09:04<340:13:02,  7.02s/it]
 22%|██▏       | 50513/225000 [106:09:12<353:23:19,  7.29s/it]
 22%|██▏       | 50514/225000 [106:09:16<316:45:41,  6.54s/it]
 22%|██▏       | 50515/225000 [106:09:21<290:33:01,  5.99s/it]
 22%|██▏       | 50516/225000 [106:09:26<276:40:42,  5.71s/it]
 22%|██▏       | 50517/225000 [106:09:31<269:32:47,  5.56s/it]
 22%|██▏       | 50518/225000 [106:09:44<376:25:13,  7.77s/it]
 22%|██▏       | 50519/225000 [106:09:50<343:24:56,  7.09s/it]
 22%|██▏       | 50520/225000 [106:09:54<297:34:00,  6.14s/it]
 22%|██▏       | 50521/225000 [106:10:04<361:46:36,  7.46s/it]
 22%|██▏       | 50522/225000 [106:10:08<304:30:55,  6.28s/it]
 22%|██▏       | 50523/225000 [106:10:14<303:19:52,  6.26s/it]
 22%|██▏       | 50524/225000 [106:10:19<286:09:23,  5.90s/it]
 22%|██▏       | 50525/225000 [106:10:24<271:37:50,  5.60s/it]
 22%|██▏       | 50526/225000 [106:10:29<260:23:57,  5.37s/it]
 22%|██▏       | 50527/225000 [106:10:38<313:57:03,  6.48s/it]
 22%|██▏       | 50528/225000 [106:10:50<398:21:55,  8.22s/it]
 22%|██▏       | 50529/225000 [106:10:54<333:47:20,  6.89s/it]
 22%|██▏       | 50530/225000 [106:10:59<300:33:32,  6.20s/it]
 22%|██▏       | 50531/225000 [106:11:03<277:48:44,  5.73s/it]
 22%|██▏       | 50532/225000 [106:11:14<343:49:24,  7.09s/it]
 22%|██▏       | 50533/225000 [106:11:22<360:45:12,  7.44s/it]
 22%|██▏       | 50534/225000 [106:11:27<321:34:48,  6.64s/it]
 22%|██▏       | 50535/225000 [106:11:31<294:05:47,  6.07s/it]
 22%|██▏       | 50536/225000 [106:11:39<315:42:09,  6.51s/it]
 22%|██▏       | 50537/225000 [106:11:50<380:24:26,  7.85s/it]
 22%|██▏       | 50538/225000 [106:11:54<322:46:47,  6.66s/it]
 22%|██▏       | 50539/225000 [106:12:04<375:43:06,  7.75s/it]
 22%|██▏       | 50540/225000 [106:12:09<336:14:09,  6.94s/it]
 22%|██▏       | 50541/225000 [106:12:14<313:04:25,  6.46s/it]
 22%|██▏       | 50542/225000 [106:12:21<312:30:20,  6.45s/it]
 22%|██▏       | 50543/225000 [106:12:30<346:21:31,  7.15s/it]
 22%|██▏       | 50544/225000 [106:12:34<311:45:00,  6.43s/it]
 22%|██▏       | 50545/225000 [106:12:39<285:33:55,  5.89s/it]
 22%|██▏       | 50546/225000 [106:12:52<393:54:19,  8.13s/it]
 22%|██▏       | 50547/225000 [106:12:56<333:01:41,  6.87s/it]
 22%|██▏       | 50548/225000 [106:13:01<295:57:50,  6.11s/it]
 22%|██▏       | 50549/225000 [106:13:10<337:31:52,  6.97s/it]
 22%|██▏       | 50550/225000 [106:13:17<348:15:16,  7.19s/it]
                                                              
{'loss': '0.71', 'grad_norm': '0.6341', 'learning_rate': '4.574e-05', 'epoch': '0.2247', 'num_input_tokens_seen': 177730296, 'train_runtime': '3.824e+05', 'train_tokens_per_second': '464.8'}

 22%|██▏       | 50550/225000 [106:13:17<348:15:16,  7.19s/it]
 22%|██▏       | 50551/225000 [106:13:25<349:47:48,  7.22s/it]
 22%|██▏       | 50552/225000 [106:13:29<315:13:53,  6.51s/it]
 22%|██▏       | 50553/225000 [106:13:38<344:29:18,  7.11s/it]
 22%|██▏       | 50554/225000 [106:13:44<323:40:20,  6.68s/it]
 22%|██▏       | 50555/225000 [106:13:50<322:18:07,  6.65s/it]
 22%|██▏       | 50556/225000 [106:13:56<309:30:10,  6.39s/it]
 22%|██▏       | 50557/225000 [106:14:04<336:37:49,  6.95s/it]
 22%|██▏       | 50558/225000 [106:14:09<305:44:35,  6.31s/it]
 22%|██▏       | 50559/225000 [106:14:14<285:27:07,  5.89s/it]
 22%|██▏       | 50560/225000 [106:14:19<279:33:31,  5.77s/it]
 22%|██▏       | 50561/225000 [106:14:25<272:57:55,  5.63s/it]
 22%|██▏       | 50562/225000 [106:14:36<360:08:34,  7.43s/it]
 22%|██▏       | 50563/225000 [106:14:43<346:23:55,  7.15s/it]
 22%|██▏       | 50564/225000 [106:14:50<346:24:34,  7.15s/it]
 22%|██▏       | 50565/225000 [106:14:55<321:47:47,  6.64s/it]
 22%|██▏       | 50566/225000 [106:15:05<361:18:29,  7.46s/it]
 22%|██▏       | 50567/225000 [106:15:09<317:43:22,  6.56s/it]
 22%|██▏       | 50568/225000 [106:15:14<284:38:24,  5.87s/it]
 22%|██▏       | 50569/225000 [106:15:26<377:35:46,  7.79s/it]
 22%|██▏       | 50570/225000 [106:15:31<341:00:30,  7.04s/it]
 22%|██▏       | 50571/225000 [106:15:37<318:22:20,  6.57s/it]
 22%|██▏       | 50572/225000 [106:15:42<299:15:14,  6.18s/it]
 22%|██▏       | 50573/225000 [106:15:49<315:21:28,  6.51s/it]
 22%|██▏       | 50574/225000 [106:16:02<411:42:56,  8.50s/it]
 22%|██▏       | 50575/225000 [106:16:06<338:57:58,  7.00s/it]
 22%|██▏       | 50576/225000 [106:16:15<369:53:16,  7.63s/it]
 22%|██▏       | 50577/225000 [106:16:20<327:38:25,  6.76s/it]
 22%|██▏       | 50578/225000 [106:16:26<317:34:29,  6.55s/it]
 22%|██▏       | 50579/225000 [106:16:36<369:33:30,  7.63s/it]
 22%|██▏       | 50580/225000 [106:16:41<331:14:34,  6.84s/it]
 22%|██▏       | 50581/225000 [106:16:50<365:13:06,  7.54s/it]
 22%|██▏       | 50582/225000 [106:16:55<324:00:04,  6.69s/it]
 22%|██▏       | 50583/225000 [106:17:00<299:42:09,  6.19s/it]
 22%|██▏       | 50584/225000 [106:17:05<280:32:35,  5.79s/it]
 22%|██▏       | 50585/225000 [106:17:17<381:34:31,  7.88s/it]
 22%|██▏       | 50586/225000 [106:17:21<321:04:03,  6.63s/it]
 22%|██▏       | 50587/225000 [106:17:25<285:17:22,  5.89s/it]
 22%|██▏       | 50588/225000 [106:17:38<386:18:50,  7.97s/it]
 22%|██▏       | 50589/225000 [106:17:43<341:50:52,  7.06s/it]
 22%|██▏       | 50590/225000 [106:17:48<307:55:25,  6.36s/it]
 22%|██▏       | 50591/225000 [106:17:52<278:19:33,  5.74s/it]
 22%|██▏       | 50592/225000 [106:17:57<265:18:09,  5.48s/it]
 22%|██▏       | 50593/225000 [106:18:10<373:21:23,  7.71s/it]
 22%|██▏       | 50594/225000 [106:18:15<341:38:37,  7.05s/it]
 22%|██▏       | 50595/225000 [106:18:20<311:57:56,  6.44s/it]
 22%|██▏       | 50596/225000 [106:18:25<287:45:42,  5.94s/it]
 22%|██▏       | 50597/225000 [106:18:30<271:01:59,  5.59s/it]
 22%|██▏       | 50598/225000 [106:18:43<379:31:16,  7.83s/it]
 22%|██▏       | 50599/225000 [106:18:48<339:32:32,  7.01s/it]
 22%|██▏       | 50600/225000 [106:18:54<330:07:11,  6.81s/it]
                                                              
{'loss': '0.698', 'grad_norm': '0.6887', 'learning_rate': '4.573e-05', 'epoch': '0.2249', 'num_input_tokens_seen': 177879480, 'train_runtime': '3.827e+05', 'train_tokens_per_second': '464.8'}

 22%|██▏       | 50600/225000 [106:18:54<330:07:11,  6.81s/it]
 22%|██▏       | 50601/225000 [106:19:04<364:55:37,  7.53s/it]
 22%|██▏       | 50602/225000 [106:19:07<308:06:33,  6.36s/it]
 22%|██▏       | 50603/225000 [106:19:16<341:42:48,  7.05s/it]
 22%|██▏       | 50604/225000 [106:19:21<309:45:18,  6.39s/it]
 22%|██▏       | 50605/225000 [106:19:26<292:47:07,  6.04s/it]
 22%|██▏       | 50606/225000 [106:19:31<276:39:06,  5.71s/it]
 22%|██▏       | 50607/225000 [106:19:36<273:14:03,  5.64s/it]
 22%|██▏       | 50608/225000 [106:19:49<380:09:59,  7.85s/it]
 22%|██▏       | 50609/225000 [106:19:56<358:05:26,  7.39s/it]
 22%|██▏       | 50610/225000 [106:20:01<329:02:36,  6.79s/it]
 22%|██▏       | 50611/225000 [106:20:06<307:12:55,  6.34s/it]
 22%|██▏       | 50612/225000 [106:20:12<294:34:14,  6.08s/it]
 22%|██▏       | 50613/225000 [106:20:17<283:51:33,  5.86s/it]
 22%|██▏       | 50614/225000 [106:20:29<375:01:48,  7.74s/it]
 22%|██▏       | 50615/225000 [106:20:36<365:49:13,  7.55s/it]
 22%|██▏       | 50616/225000 [106:20:40<314:40:28,  6.50s/it]
 22%|██▏       | 50617/225000 [106:20:49<351:06:25,  7.25s/it]
 22%|██▏       | 50618/225000 [106:20:54<309:34:17,  6.39s/it]
 22%|██▏       | 50619/225000 [106:21:01<326:54:06,  6.75s/it]
 22%|██▏       | 50620/225000 [106:21:11<374:46:12,  7.74s/it]
 22%|██▏       | 50621/225000 [106:21:18<357:53:02,  7.39s/it]
 22%|██▏       | 50622/225000 [106:21:24<331:43:24,  6.85s/it]
 22%|██▏       | 50623/225000 [106:21:31<332:38:15,  6.87s/it]
 22%|██▏       | 50624/225000 [106:21:39<353:31:03,  7.30s/it]
 22%|██▎       | 50625/225000 [106:21:43<311:33:35,  6.43s/it]
 23%|██▎       | 50626/225000 [106:21:49<295:16:04,  6.10s/it]
 23%|██▎       | 50627/225000 [106:21:55<295:12:10,  6.09s/it]
 23%|██▎       | 50628/225000 [106:22:01<294:42:30,  6.08s/it]
 23%|██▎       | 50629/225000 [106:22:06<286:47:29,  5.92s/it]
 23%|██▎       | 50630/225000 [106:22:19<379:52:08,  7.84s/it]
 23%|██▎       | 50631/225000 [106:22:25<362:35:52,  7.49s/it]
 23%|██▎       | 50632/225000 [106:22:34<378:19:01,  7.81s/it]
 23%|██▎       | 50633/225000 [106:22:39<346:52:40,  7.16s/it]
 23%|██▎       | 50634/225000 [106:22:46<337:24:22,  6.97s/it]
 23%|██▎       | 50635/225000 [106:22:55<370:45:15,  7.65s/it]
 23%|██▎       | 50636/225000 [106:22:59<313:25:05,  6.47s/it]
 23%|██▎       | 50637/225000 [106:23:05<309:50:06,  6.40s/it]
 23%|██▎       | 50638/225000 [106:23:10<294:04:39,  6.07s/it]
 23%|██▎       | 50639/225000 [106:23:15<278:16:35,  5.75s/it]
 23%|██▎       | 50640/225000 [106:23:20<261:22:06,  5.40s/it]
 23%|██▎       | 50641/225000 [106:23:30<333:54:21,  6.89s/it]
 23%|██▎       | 50642/225000 [106:23:39<352:30:58,  7.28s/it]
 23%|██▎       | 50643/225000 [106:23:46<354:15:27,  7.31s/it]
 23%|██▎       | 50644/225000 [106:23:52<335:24:40,  6.93s/it]
 23%|██▎       | 50645/225000 [106:24:01<360:26:27,  7.44s/it]
 23%|██▎       | 50646/225000 [106:24:04<303:20:44,  6.26s/it]
 23%|██▎       | 50647/225000 [106:24:12<327:09:06,  6.75s/it]
 23%|██▎       | 50648/225000 [106:24:20<340:32:41,  7.03s/it]
 23%|██▎       | 50649/225000 [106:24:24<296:09:08,  6.11s/it]
 23%|██▎       | 50650/225000 [106:24:32<333:10:02,  6.88s/it]
                                                              
{'loss': '0.6994', 'grad_norm': '0.7369', 'learning_rate': '4.572e-05', 'epoch': '0.2251', 'num_input_tokens_seen': 178028656, 'train_runtime': '3.831e+05', 'train_tokens_per_second': '464.7'}

 23%|██▎       | 50650/225000 [106:24:32<333:10:02,  6.88s/it]
 23%|██▎       | 50651/225000 [106:24:38<309:52:54,  6.40s/it]
 23%|██▎       | 50652/225000 [106:24:42<276:48:17,  5.72s/it]
 23%|██▎       | 50653/225000 [106:24:52<339:14:59,  7.00s/it]
 23%|██▎       | 50654/225000 [106:25:00<355:43:13,  7.35s/it]
 23%|██▎       | 50655/225000 [106:25:04<311:13:17,  6.43s/it]
 23%|██▎       | 50656/225000 [106:25:10<308:46:53,  6.38s/it]
 23%|██▎       | 50657/225000 [106:25:19<342:46:54,  7.08s/it]
 23%|██▎       | 50658/225000 [106:25:25<319:36:30,  6.60s/it]
 23%|██▎       | 50659/225000 [106:25:29<292:37:36,  6.04s/it]
 23%|██▎       | 50660/225000 [106:25:34<273:04:20,  5.64s/it]
 23%|██▎       | 50661/225000 [106:25:40<283:11:34,  5.85s/it]
 23%|██▎       | 50662/225000 [106:25:52<364:52:02,  7.53s/it]
 23%|██▎       | 50663/225000 [106:25:59<360:15:21,  7.44s/it]
 23%|██▎       | 50664/225000 [106:26:04<319:04:25,  6.59s/it]
 23%|██▎       | 50665/225000 [106:26:09<293:23:02,  6.06s/it]
 23%|██▎       | 50666/225000 [106:26:20<376:32:53,  7.78s/it]
 23%|██▎       | 50667/225000 [106:26:24<313:57:22,  6.48s/it]
 23%|██▎       | 50668/225000 [106:26:33<351:18:21,  7.25s/it]
 23%|██▎       | 50669/225000 [106:26:37<311:11:51,  6.43s/it]
 23%|██▎       | 50670/225000 [106:26:42<291:54:02,  6.03s/it]
 23%|██▎       | 50671/225000 [106:26:47<275:13:50,  5.68s/it]
 23%|██▎       | 50672/225000 [106:26:55<304:16:39,  6.28s/it]
 23%|██▎       | 50673/225000 [106:27:07<386:41:39,  7.99s/it]
 23%|██▎       | 50674/225000 [106:27:12<337:46:34,  6.98s/it]
 23%|██▎       | 50675/225000 [106:27:21<374:39:39,  7.74s/it]
 23%|██▎       | 50676/225000 [106:27:25<323:23:56,  6.68s/it]
 23%|██▎       | 50677/225000 [106:27:30<290:00:12,  5.99s/it]
 23%|██▎       | 50678/225000 [106:27:42<383:54:08,  7.93s/it]
 23%|██▎       | 50679/225000 [106:27:46<319:07:15,  6.59s/it]
 23%|██▎       | 50680/225000 [106:27:50<284:47:47,  5.88s/it]
 23%|██▎       | 50681/225000 [106:27:55<274:31:58,  5.67s/it]
 23%|██▎       | 50682/225000 [106:28:01<283:55:56,  5.86s/it]
 23%|██▎       | 50683/225000 [106:28:15<393:44:27,  8.13s/it]
 23%|██▎       | 50684/225000 [106:28:19<339:48:35,  7.02s/it]
 23%|██▎       | 50685/225000 [106:28:24<302:07:43,  6.24s/it]
 23%|██▎       | 50686/225000 [106:28:29<287:30:16,  5.94s/it]
 23%|██▎       | 50687/225000 [106:28:41<373:02:07,  7.70s/it]
 23%|██▎       | 50688/225000 [106:28:46<345:11:01,  7.13s/it]
 23%|██▎       | 50689/225000 [106:28:52<318:04:23,  6.57s/it]
 23%|██▎       | 50690/225000 [106:28:58<308:05:42,  6.36s/it]
 23%|██▎       | 50691/225000 [106:29:03<294:54:05,  6.09s/it]
 23%|██▎       | 50692/225000 [106:29:08<280:42:59,  5.80s/it]
 23%|██▎       | 50693/225000 [106:29:14<275:48:48,  5.70s/it]
 23%|██▎       | 50694/225000 [106:29:19<273:48:42,  5.66s/it]
 23%|██▎       | 50695/225000 [106:29:29<331:49:21,  6.85s/it]
 23%|██▎       | 50696/225000 [106:29:42<418:27:24,  8.64s/it]
 23%|██▎       | 50697/225000 [106:29:45<346:53:03,  7.16s/it]
 23%|██▎       | 50698/225000 [106:29:57<406:52:03,  8.40s/it]
 23%|██▎       | 50699/225000 [106:30:01<350:46:34,  7.24s/it]
 23%|██▎       | 50700/225000 [106:30:06<310:14:58,  6.41s/it]
                                                              
{'loss': '0.7096', 'grad_norm': '0.8506', 'learning_rate': '4.571e-05', 'epoch': '0.2253', 'num_input_tokens_seen': 178177304, 'train_runtime': '3.834e+05', 'train_tokens_per_second': '464.7'}

 23%|██▎       | 50700/225000 [106:30:06<310:14:58,  6.41s/it]
 23%|██▎       | 50701/225000 [106:30:11<292:12:04,  6.04s/it]
 23%|██▎       | 50702/225000 [106:30:17<295:05:09,  6.09s/it]
 23%|██▎       | 50703/225000 [106:30:30<387:12:18,  8.00s/it]
 23%|██▎       | 50704/225000 [106:30:36<372:21:46,  7.69s/it]
 23%|██▎       | 50705/225000 [106:30:42<334:51:52,  6.92s/it]
 23%|██▎       | 50706/225000 [106:30:49<347:28:41,  7.18s/it]
 23%|██▎       | 50707/225000 [106:30:54<315:38:30,  6.52s/it]
 23%|██▎       | 50708/225000 [106:31:04<364:31:35,  7.53s/it]
 23%|██▎       | 50709/225000 [106:31:12<366:04:57,  7.56s/it]
 23%|██▎       | 50710/225000 [106:31:16<309:10:20,  6.39s/it]
 23%|██▎       | 50711/225000 [106:31:25<348:49:45,  7.21s/it]
 23%|██▎       | 50712/225000 [106:31:28<295:03:20,  6.09s/it]
 23%|██▎       | 50713/225000 [106:31:37<339:16:20,  7.01s/it]
 23%|██▎       | 50714/225000 [106:31:46<362:43:48,  7.49s/it]
 23%|██▎       | 50715/225000 [106:31:52<343:56:55,  7.10s/it]
 23%|██▎       | 50716/225000 [106:31:56<302:37:17,  6.25s/it]
 23%|██▎       | 50717/225000 [106:32:01<278:33:02,  5.75s/it]
 23%|██▎       | 50718/225000 [106:32:10<332:48:58,  6.87s/it]
 23%|██▎       | 50719/225000 [106:32:15<294:38:30,  6.09s/it]
 23%|██▎       | 50720/225000 [106:32:19<275:48:17,  5.70s/it]
 23%|██▎       | 50721/225000 [106:32:24<263:14:16,  5.44s/it]
 23%|██▎       | 50722/225000 [106:32:30<263:21:32,  5.44s/it]
 23%|██▎       | 50723/225000 [106:32:42<368:34:26,  7.61s/it]
 23%|██▎       | 50724/225000 [106:32:52<390:12:03,  8.06s/it]
 23%|██▎       | 50725/225000 [106:32:59<377:33:47,  7.80s/it]
 23%|██▎       | 50726/225000 [106:33:03<331:18:05,  6.84s/it]
 23%|██▎       | 50727/225000 [106:33:09<310:38:35,  6.42s/it]
 23%|██▎       | 50728/225000 [106:33:20<380:31:25,  7.86s/it]
 23%|██▎       | 50729/225000 [106:33:25<336:47:41,  6.96s/it]
 23%|██▎       | 50730/225000 [106:33:29<302:23:58,  6.25s/it]
 23%|██▎       | 50731/225000 [106:33:42<393:53:29,  8.14s/it]
 23%|██▎       | 50732/225000 [106:33:47<354:31:54,  7.32s/it]
 23%|██▎       | 50733/225000 [106:33:54<346:49:52,  7.16s/it]
 23%|██▎       | 50734/225000 [106:33:58<302:28:48,  6.25s/it]
 23%|██▎       | 50735/225000 [106:34:08<355:23:41,  7.34s/it]
 23%|██▎       | 50736/225000 [106:34:12<310:05:06,  6.41s/it]
 23%|██▎       | 50737/225000 [106:34:19<305:17:13,  6.31s/it]
 23%|██▎       | 50738/225000 [106:34:30<378:25:40,  7.82s/it]
 23%|██▎       | 50739/225000 [106:34:35<343:15:22,  7.09s/it]
 23%|██▎       | 50740/225000 [106:34:41<325:08:56,  6.72s/it]
 23%|██▎       | 50741/225000 [106:34:47<306:16:24,  6.33s/it]
 23%|██▎       | 50742/225000 [106:34:52<294:15:48,  6.08s/it]
 23%|██▎       | 50743/225000 [106:34:58<291:21:45,  6.02s/it]
 23%|██▎       | 50744/225000 [106:35:03<281:40:05,  5.82s/it]
 23%|██▎       | 50745/225000 [106:35:16<383:15:25,  7.92s/it]
 23%|██▎       | 50746/225000 [106:35:24<380:49:39,  7.87s/it]
 23%|██▎       | 50747/225000 [106:35:29<334:40:36,  6.91s/it]
 23%|██▎       | 50748/225000 [106:35:34<309:57:03,  6.40s/it]
 23%|██▎       | 50749/225000 [106:35:39<289:43:44,  5.99s/it]
 23%|██▎       | 50750/225000 [106:35:47<323:05:07,  6.67s/it]
                                                              
{'loss': '0.7165', 'grad_norm': '0.7716', 'learning_rate': '4.57e-05', 'epoch': '0.2256', 'num_input_tokens_seen': 178327408, 'train_runtime': '3.837e+05', 'train_tokens_per_second': '464.7'}

 23%|██▎       | 50750/225000 [106:35:47<323:05:07,  6.67s/it]
 23%|██▎       | 50751/225000 [106:36:00<414:33:00,  8.56s/it]
 23%|██▎       | 50752/225000 [106:36:07<391:05:14,  8.08s/it]
 23%|██▎       | 50753/225000 [106:36:11<335:17:16,  6.93s/it]
 23%|██▎       | 50754/225000 [106:36:15<295:50:06,  6.11s/it]
 23%|██▎       | 50755/225000 [106:36:28<386:04:37,  7.98s/it]
 23%|██▎       | 50756/225000 [106:36:31<323:03:50,  6.67s/it]
 23%|██▎       | 50757/225000 [106:36:39<338:13:18,  6.99s/it]
 23%|██▎       | 50758/225000 [106:36:45<318:26:39,  6.58s/it]
 23%|██▎       | 50759/225000 [106:36:50<301:47:04,  6.24s/it]
 23%|██▎       | 50760/225000 [106:36:56<291:24:22,  6.02s/it]
 23%|██▎       | 50761/225000 [106:37:02<292:00:08,  6.03s/it]
 23%|██▎       | 50762/225000 [106:37:07<280:44:15,  5.80s/it]
 23%|██▎       | 50763/225000 [106:37:13<279:06:15,  5.77s/it]
 23%|██▎       | 50764/225000 [106:37:18<273:12:53,  5.65s/it]
 23%|██▎       | 50765/225000 [106:37:28<340:12:55,  7.03s/it]
 23%|██▎       | 50766/225000 [106:37:39<398:14:52,  8.23s/it]
 23%|██▎       | 50767/225000 [106:37:47<383:35:35,  7.93s/it]
 23%|██▎       | 50768/225000 [106:37:52<341:24:09,  7.05s/it]
 23%|██▎       | 50769/225000 [106:38:01<373:55:20,  7.73s/it]
 23%|██▎       | 50770/225000 [106:38:06<333:24:13,  6.89s/it]
 23%|██▎       | 50771/225000 [106:38:11<305:59:03,  6.32s/it]
 23%|██▎       | 50772/225000 [106:38:16<286:00:57,  5.91s/it]
 23%|██▎       | 50773/225000 [106:38:26<354:27:50,  7.32s/it]
 23%|██▎       | 50774/225000 [106:38:34<366:21:16,  7.57s/it]
 23%|██▎       | 50775/225000 [106:38:40<338:14:22,  6.99s/it]
 23%|██▎       | 50776/225000 [106:38:46<316:44:36,  6.54s/it]
 23%|██▎       | 50777/225000 [106:38:53<330:22:47,  6.83s/it]
 23%|██▎       | 50778/225000 [106:39:04<395:38:57,  8.18s/it]
 23%|██▎       | 50779/225000 [106:39:11<371:11:51,  7.67s/it]
 23%|██▎       | 50780/225000 [106:39:16<332:43:46,  6.88s/it]
 23%|██▎       | 50781/225000 [106:39:21<309:59:53,  6.41s/it]
 23%|██▎       | 50782/225000 [106:39:26<284:50:39,  5.89s/it]
 23%|██▎       | 50783/225000 [106:39:32<284:58:17,  5.89s/it]
 23%|██▎       | 50784/225000 [106:39:45<396:35:27,  8.20s/it]
 23%|██▎       | 50785/225000 [106:39:51<357:11:21,  7.38s/it]
 23%|██▎       | 50786/225000 [106:39:56<318:31:31,  6.58s/it]
 23%|██▎       | 50787/225000 [106:40:02<316:35:23,  6.54s/it]
 23%|██▎       | 50788/225000 [106:40:12<360:40:14,  7.45s/it]
 23%|██▎       | 50789/225000 [106:40:20<366:55:49,  7.58s/it]
 23%|██▎       | 50790/225000 [106:40:25<332:35:18,  6.87s/it]
 23%|██▎       | 50791/225000 [106:40:30<308:34:22,  6.38s/it]
 23%|██▎       | 50792/225000 [106:40:35<289:48:21,  5.99s/it]
 23%|██▎       | 50793/225000 [106:40:41<291:16:24,  6.02s/it]
 23%|██▎       | 50794/225000 [106:40:46<281:42:43,  5.82s/it]
 23%|██▎       | 50795/225000 [106:40:52<272:24:44,  5.63s/it]
 23%|██▎       | 50796/225000 [106:41:03<361:09:58,  7.46s/it]
 23%|██▎       | 50797/225000 [106:41:12<378:58:57,  7.83s/it]
 23%|██▎       | 50798/225000 [106:41:17<332:07:53,  6.86s/it]
 23%|██▎       | 50799/225000 [106:41:21<295:46:55,  6.11s/it]
 23%|██▎       | 50800/225000 [106:41:26<273:41:19,  5.66s/it]
                                                              
{'loss': '0.703', 'grad_norm': '0.8023', 'learning_rate': '4.569e-05', 'epoch': '0.2258', 'num_input_tokens_seen': 178478448, 'train_runtime': '3.841e+05', 'train_tokens_per_second': '464.7'}

 23%|██▎       | 50800/225000 [106:41:26<273:41:19,  5.66s/it]
 23%|██▎       | 50801/225000 [106:41:39<380:31:16,  7.86s/it]
 23%|██▎       | 50802/225000 [106:41:43<326:50:22,  6.75s/it]
 23%|██▎       | 50803/225000 [106:41:52<360:23:14,  7.45s/it]
 23%|██▎       | 50804/225000 [106:41:57<320:52:53,  6.63s/it]
 23%|██▎       | 50805/225000 [106:42:02<304:15:00,  6.29s/it]
 23%|██▎       | 50806/225000 [106:42:07<283:40:47,  5.86s/it]
 23%|██▎       | 50807/225000 [106:42:12<274:54:10,  5.68s/it]
 23%|██▎       | 50808/225000 [106:42:17<267:32:04,  5.53s/it]
 23%|██▎       | 50809/225000 [106:42:24<279:24:05,  5.77s/it]
 23%|██▎       | 50810/225000 [106:42:36<375:34:47,  7.76s/it]
 23%|██▎       | 50811/225000 [106:42:44<371:52:47,  7.69s/it]
 23%|██▎       | 50812/225000 [106:42:49<343:09:27,  7.09s/it]
 23%|██▎       | 50813/225000 [106:42:55<316:03:25,  6.53s/it]
 23%|██▎       | 50814/225000 [106:43:00<302:31:31,  6.25s/it]
 23%|██▎       | 50815/225000 [106:43:05<287:12:56,  5.94s/it]
 23%|██▎       | 50816/225000 [106:43:18<390:17:59,  8.07s/it]
 23%|██▎       | 50817/225000 [106:43:23<336:05:46,  6.95s/it]
 23%|██▎       | 50818/225000 [106:43:27<297:28:12,  6.15s/it]
 23%|██▎       | 50819/225000 [106:43:39<384:21:45,  7.94s/it]
 23%|██▎       | 50820/225000 [106:43:44<333:30:44,  6.89s/it]
 23%|██▎       | 50821/225000 [106:43:48<303:32:00,  6.27s/it]
 23%|██▎       | 50822/225000 [106:43:53<284:07:03,  5.87s/it]
 23%|██▎       | 50823/225000 [106:43:59<278:08:19,  5.75s/it]
 23%|██▎       | 50824/225000 [106:44:09<345:09:54,  7.13s/it]
 23%|██▎       | 50825/225000 [106:44:18<373:51:01,  7.73s/it]
 23%|██▎       | 50826/225000 [106:44:23<327:35:00,  6.77s/it]
 23%|██▎       | 50827/225000 [106:44:28<299:01:56,  6.18s/it]
 23%|██▎       | 50828/225000 [106:44:34<300:17:17,  6.21s/it]
 23%|██▎       | 50829/225000 [106:44:47<402:04:05,  8.31s/it]
 23%|██▎       | 50830/225000 [106:44:52<356:43:49,  7.37s/it]
 23%|██▎       | 50831/225000 [106:44:57<312:48:08,  6.47s/it]
 23%|██▎       | 50832/225000 [106:45:02<289:35:13,  5.99s/it]
 23%|██▎       | 50833/225000 [106:45:10<328:24:24,  6.79s/it]
 23%|██▎       | 50834/225000 [106:45:22<395:33:07,  8.18s/it]
 23%|██▎       | 50835/225000 [106:45:25<331:07:27,  6.84s/it]
 23%|██▎       | 50836/225000 [106:45:32<331:43:53,  6.86s/it]
 23%|██▎       | 50837/225000 [106:45:40<342:07:01,  7.07s/it]
 23%|██▎       | 50838/225000 [106:45:46<333:05:14,  6.89s/it]
 23%|██▎       | 50839/225000 [106:45:50<291:19:51,  6.02s/it]
 23%|██▎       | 50840/225000 [106:46:01<353:50:59,  7.31s/it]
 23%|██▎       | 50841/225000 [106:46:05<303:56:33,  6.28s/it]
 23%|██▎       | 50842/225000 [106:46:13<338:51:35,  7.00s/it]
 23%|██▎       | 50843/225000 [106:46:20<339:47:37,  7.02s/it]
 23%|██▎       | 50844/225000 [106:46:25<311:00:18,  6.43s/it]
 23%|██▎       | 50845/225000 [106:46:35<357:19:29,  7.39s/it]
 23%|██▎       | 50846/225000 [106:46:40<320:56:43,  6.63s/it]
 23%|██▎       | 50847/225000 [106:46:45<295:29:11,  6.11s/it]
 23%|██▎       | 50848/225000 [106:46:50<285:30:05,  5.90s/it]
 23%|██▎       | 50849/225000 [106:46:55<276:24:17,  5.71s/it]
 23%|██▎       | 50850/225000 [106:47:06<347:19:53,  7.18s/it]
                                                              
{'loss': '0.6918', 'grad_norm': '0.7416', 'learning_rate': '4.568e-05', 'epoch': '0.226', 'num_input_tokens_seen': 178627408, 'train_runtime': '3.844e+05', 'train_tokens_per_second': '464.7'}

 23%|██▎       | 50850/225000 [106:47:06<347:19:53,  7.18s/it]
 23%|██▎       | 50851/225000 [106:47:15<380:56:52,  7.87s/it]
 23%|██▎       | 50852/225000 [106:47:23<374:24:21,  7.74s/it]
 23%|██▎       | 50853/225000 [106:47:28<333:48:12,  6.90s/it]
 23%|██▎       | 50854/225000 [106:47:35<335:31:16,  6.94s/it]
 23%|██▎       | 50855/225000 [106:47:41<317:15:04,  6.56s/it]
 23%|██▎       | 50856/225000 [106:47:48<336:48:13,  6.96s/it]
 23%|██▎       | 50857/225000 [106:47:53<301:41:23,  6.24s/it]
 23%|██▎       | 50858/225000 [106:47:58<290:18:47,  6.00s/it]
 23%|██▎       | 50859/225000 [106:48:05<294:44:18,  6.09s/it]
 23%|██▎       | 50860/225000 [106:48:10<281:01:43,  5.81s/it]
 23%|██▎       | 50861/225000 [106:48:22<367:57:44,  7.61s/it]
 23%|██▎       | 50862/225000 [106:48:29<357:12:43,  7.38s/it]
 23%|██▎       | 50863/225000 [106:48:32<306:19:15,  6.33s/it]
 23%|██▎       | 50864/225000 [106:48:43<365:36:24,  7.56s/it]
 23%|██▎       | 50865/225000 [106:48:47<312:29:35,  6.46s/it]
 23%|██▎       | 50866/225000 [106:48:52<297:16:20,  6.15s/it]
 23%|██▎       | 50867/225000 [106:48:58<286:18:54,  5.92s/it]
 23%|██▎       | 50868/225000 [106:49:11<391:06:51,  8.09s/it]
 23%|██▎       | 50869/225000 [106:49:15<337:45:23,  6.98s/it]
 23%|██▎       | 50870/225000 [106:49:19<297:45:25,  6.16s/it]
 23%|██▎       | 50871/225000 [106:49:30<363:59:55,  7.53s/it]
 23%|██▎       | 50872/225000 [106:49:35<325:22:22,  6.73s/it]
 23%|██▎       | 50873/225000 [106:49:40<300:46:08,  6.22s/it]
 23%|██▎       | 50874/225000 [106:49:45<285:47:59,  5.91s/it]
 23%|██▎       | 50875/225000 [106:49:51<286:22:41,  5.92s/it]
 23%|██▎       | 50876/225000 [106:50:04<383:22:15,  7.93s/it]
 23%|██▎       | 50877/225000 [106:50:09<352:05:30,  7.28s/it]
 23%|██▎       | 50878/225000 [106:50:14<309:50:12,  6.41s/it]
 23%|██▎       | 50879/225000 [106:50:19<294:31:02,  6.09s/it]
 23%|██▎       | 50880/225000 [106:50:30<362:14:21,  7.49s/it]
 23%|██▎       | 50881/225000 [106:50:39<384:04:10,  7.94s/it]
 23%|██▎       | 50882/225000 [106:50:43<331:28:58,  6.85s/it]
 23%|██▎       | 50883/225000 [106:50:48<299:05:58,  6.18s/it]
 23%|██▎       | 50884/225000 [106:51:00<389:52:39,  8.06s/it]
 23%|██▎       | 50885/225000 [106:51:05<334:14:33,  6.91s/it]
 23%|██▎       | 50886/225000 [106:51:10<313:16:04,  6.48s/it]
 23%|██▎       | 50887/225000 [106:51:15<297:02:59,  6.14s/it]
 23%|██▎       | 50888/225000 [106:51:24<337:38:48,  6.98s/it]
 23%|██▎       | 50889/225000 [106:51:36<399:45:11,  8.27s/it]
 23%|██▎       | 50890/225000 [106:51:40<345:18:36,  7.14s/it]
 23%|██▎       | 50891/225000 [106:51:45<309:35:41,  6.40s/it]
 23%|██▎       | 50892/225000 [106:51:52<314:47:08,  6.51s/it]
 23%|██▎       | 50893/225000 [106:52:03<381:55:57,  7.90s/it]
 23%|██▎       | 50894/225000 [106:52:07<331:09:43,  6.85s/it]
 23%|██▎       | 50895/225000 [106:52:17<371:36:58,  7.68s/it]
 23%|██▎       | 50896/225000 [106:52:22<340:35:34,  7.04s/it]
 23%|██▎       | 50897/225000 [106:52:31<371:57:16,  7.69s/it]
 23%|██▎       | 50898/225000 [106:52:35<318:15:30,  6.58s/it]
 23%|██▎       | 50899/225000 [106:52:42<313:35:43,  6.48s/it]
 23%|██▎       | 50900/225000 [106:52:49<329:36:25,  6.82s/it]
                                                              
{'loss': '0.7046', 'grad_norm': '0.7858', 'learning_rate': '4.567e-05', 'epoch': '0.2262', 'num_input_tokens_seen': 178778976, 'train_runtime': '3.848e+05', 'train_tokens_per_second': '464.6'}

 23%|██▎       | 50900/225000 [106:52:49<329:36:25,  6.82s/it]
 23%|██▎       | 50901/225000 [106:52:54<305:23:42,  6.31s/it]
 23%|██▎       | 50902/225000 [106:53:04<358:02:56,  7.40s/it]
 23%|██▎       | 50903/225000 [106:53:08<304:40:02,  6.30s/it]
 23%|██▎       | 50904/225000 [106:53:18<352:13:32,  7.28s/it]
 23%|██▎       | 50905/225000 [106:53:24<331:22:04,  6.85s/it]
 23%|██▎       | 50906/225000 [106:53:28<298:16:42,  6.17s/it]
 23%|██▎       | 50907/225000 [106:53:33<274:31:37,  5.68s/it]
 23%|██▎       | 50908/225000 [106:53:44<356:47:25,  7.38s/it]
 23%|██▎       | 50909/225000 [106:53:49<326:13:52,  6.75s/it]
 23%|██▎       | 50910/225000 [106:53:55<310:44:13,  6.43s/it]
 23%|██▎       | 50911/225000 [106:54:00<291:59:21,  6.04s/it]
 23%|██▎       | 50912/225000 [106:54:10<345:09:15,  7.14s/it]
 23%|██▎       | 50913/225000 [106:54:18<355:26:24,  7.35s/it]
 23%|██▎       | 50914/225000 [106:54:22<311:34:28,  6.44s/it]
 23%|██▎       | 50915/225000 [106:54:33<371:58:49,  7.69s/it]
 23%|██▎       | 50916/225000 [106:54:36<313:00:50,  6.47s/it]
 23%|██▎       | 50917/225000 [106:54:45<347:08:38,  7.18s/it]
 23%|██▎       | 50918/225000 [106:54:52<338:34:00,  7.00s/it]
 23%|██▎       | 50919/225000 [106:54:56<305:09:25,  6.31s/it]
 23%|██▎       | 50920/225000 [106:55:01<278:23:09,  5.76s/it]
 23%|██▎       | 50921/225000 [106:55:07<289:28:54,  5.99s/it]
 23%|██▎       | 50922/225000 [106:55:19<376:18:47,  7.78s/it]
 23%|██▎       | 50923/225000 [106:55:27<369:10:48,  7.63s/it]
 23%|██▎       | 50924/225000 [106:55:30<311:28:48,  6.44s/it]
 23%|██▎       | 50925/225000 [106:55:39<346:30:04,  7.17s/it]
 23%|██▎       | 50926/225000 [106:55:48<366:47:07,  7.59s/it]
 23%|██▎       | 50927/225000 [106:55:54<349:43:23,  7.23s/it]
 23%|██▎       | 50928/225000 [106:55:59<318:37:37,  6.59s/it]
 23%|██▎       | 50929/225000 [106:56:05<301:37:31,  6.24s/it]
 23%|██▎       | 50930/225000 [106:56:10<290:27:42,  6.01s/it]
 23%|██▎       | 50931/225000 [106:56:15<279:29:53,  5.78s/it]
 23%|██▎       | 50932/225000 [106:56:21<273:35:36,  5.66s/it]
 23%|██▎       | 50933/225000 [106:56:26<269:55:34,  5.58s/it]
 23%|██▎       | 50934/225000 [106:56:32<269:24:02,  5.57s/it]
 23%|██▎       | 50935/225000 [106:56:37<265:09:44,  5.48s/it]
 23%|██▎       | 50936/225000 [106:56:50<379:03:48,  7.84s/it]
 23%|██▎       | 50937/225000 [106:56:57<368:22:46,  7.62s/it]
 23%|██▎       | 50938/225000 [106:57:06<379:38:44,  7.85s/it]
 23%|██▎       | 50939/225000 [106:57:14<382:09:46,  7.90s/it]
 23%|██▎       | 50940/225000 [106:57:18<329:39:22,  6.82s/it]
 23%|██▎       | 50941/225000 [106:57:23<303:23:59,  6.28s/it]
 23%|██▎       | 50942/225000 [106:57:35<393:09:38,  8.13s/it]
 23%|██▎       | 50943/225000 [106:57:40<334:03:02,  6.91s/it]
 23%|██▎       | 50944/225000 [106:57:44<305:02:06,  6.31s/it]
 23%|██▎       | 50945/225000 [106:57:50<294:29:38,  6.09s/it]
 23%|██▎       | 50946/225000 [106:57:56<289:43:14,  5.99s/it]
 23%|██▎       | 50947/225000 [106:58:02<297:17:25,  6.15s/it]
 23%|██▎       | 50948/225000 [106:58:17<421:35:04,  8.72s/it]
 23%|██▎       | 50949/225000 [106:58:22<365:57:41,  7.57s/it]
 23%|██▎       | 50950/225000 [106:58:31<388:09:54,  8.03s/it]
                                                              
{'loss': '0.6925', 'grad_norm': '0.9183', 'learning_rate': '4.566e-05', 'epoch': '0.2264', 'num_input_tokens_seen': 178929808, 'train_runtime': '3.851e+05', 'train_tokens_per_second': '464.6'}

 23%|██▎       | 50950/225000 [106:58:31<388:09:54,  8.03s/it]
 23%|██▎       | 50951/225000 [106:58:35<326:24:13,  6.75s/it]
 23%|██▎       | 50952/225000 [106:58:46<392:49:48,  8.13s/it]
 23%|██▎       | 50953/225000 [106:58:49<322:59:35,  6.68s/it]
 23%|██▎       | 50954/225000 [106:58:55<301:56:19,  6.25s/it]
 23%|██▎       | 50955/225000 [106:59:00<293:36:16,  6.07s/it]
 23%|██▎       | 50956/225000 [106:59:06<284:44:07,  5.89s/it]
 23%|██▎       | 50957/225000 [106:59:11<271:01:21,  5.61s/it]
 23%|██▎       | 50958/225000 [106:59:16<267:04:27,  5.52s/it]
 23%|██▎       | 50959/225000 [106:59:22<266:52:45,  5.52s/it]
 23%|██▎       | 50960/225000 [106:59:30<308:39:54,  6.38s/it]
 23%|██▎       | 50961/225000 [106:59:44<413:21:23,  8.55s/it]
 23%|██▎       | 50962/225000 [106:59:49<369:19:10,  7.64s/it]
 23%|██▎       | 50963/225000 [106:59:53<322:48:24,  6.68s/it]
 23%|██▎       | 50964/225000 [107:00:02<348:48:01,  7.22s/it]
 23%|██▎       | 50965/225000 [107:00:10<365:31:13,  7.56s/it]
 23%|██▎       | 50966/225000 [107:00:15<324:22:29,  6.71s/it]
 23%|██▎       | 50967/225000 [107:00:20<295:25:06,  6.11s/it]
 23%|██▎       | 50968/225000 [107:00:30<360:12:19,  7.45s/it]
 23%|██▎       | 50969/225000 [107:00:40<399:04:41,  8.26s/it]
 23%|██▎       | 50970/225000 [107:00:46<359:33:14,  7.44s/it]
 23%|██▎       | 50971/225000 [107:00:51<329:59:51,  6.83s/it]
 23%|██▎       | 50972/225000 [107:00:57<312:56:32,  6.47s/it]
 23%|██▎       | 50973/225000 [107:01:08<379:53:07,  7.86s/it]
 23%|██▎       | 50974/225000 [107:01:12<323:57:36,  6.70s/it]
 23%|██▎       | 50975/225000 [107:01:22<371:33:46,  7.69s/it]
 23%|██▎       | 50976/225000 [107:01:26<322:42:58,  6.68s/it]
 23%|██▎       | 50977/225000 [107:01:31<284:56:10,  5.89s/it]
 23%|██▎       | 50978/225000 [107:01:35<270:34:52,  5.60s/it]
 23%|██▎       | 50979/225000 [107:01:44<319:15:38,  6.60s/it]
 23%|██▎       | 50980/225000 [107:01:54<363:50:34,  7.53s/it]
 23%|██▎       | 50981/225000 [107:02:01<361:35:51,  7.48s/it]
 23%|██▎       | 50982/225000 [107:02:05<310:51:02,  6.43s/it]
 23%|██▎       | 50983/225000 [107:02:13<324:55:02,  6.72s/it]
 23%|██▎       | 50984/225000 [107:02:20<326:33:20,  6.76s/it]
 23%|██▎       | 50985/225000 [107:02:24<292:46:22,  6.06s/it]
 23%|██▎       | 50986/225000 [107:02:34<343:43:31,  7.11s/it]
 23%|██▎       | 50987/225000 [107:02:40<332:53:40,  6.89s/it]
 23%|██▎       | 50988/225000 [107:02:48<349:58:06,  7.24s/it]
 23%|██▎       | 50989/225000 [107:02:53<322:35:57,  6.67s/it]
 23%|██▎       | 50990/225000 [107:02:59<307:16:25,  6.36s/it]
 23%|██▎       | 50991/225000 [107:03:04<292:08:18,  6.04s/it]
 23%|██▎       | 50992/225000 [107:03:10<290:42:55,  6.01s/it]
 23%|██▎       | 50993/225000 [107:03:16<279:46:56,  5.79s/it]
 23%|██▎       | 50994/225000 [107:03:21<279:23:07,  5.78s/it]
 23%|██▎       | 50995/225000 [107:03:28<292:55:24,  6.06s/it]
 23%|██▎       | 50996/225000 [107:03:34<287:02:06,  5.94s/it]
 23%|██▎       | 50997/225000 [107:03:46<385:14:37,  7.97s/it]
 23%|██▎       | 50998/225000 [107:03:52<346:14:14,  7.16s/it]
 23%|██▎       | 50999/225000 [107:03:56<309:00:48,  6.39s/it]
 23%|██▎       | 51000/225000 [107:04:06<352:33:14,  7.29s/it]
                                                              
{'loss': '0.7105', 'grad_norm': '0.7181', 'learning_rate': '4.565e-05', 'epoch': '0.2267', 'num_input_tokens_seen': 179078944, 'train_runtime': '3.854e+05', 'train_tokens_per_second': '464.6'}

 23%|██▎       | 51000/225000 [107:04:06<352:33:14,  7.29s/it]
 23%|██▎       | 51001/225000 [107:04:16<395:30:43,  8.18s/it]
 23%|██▎       | 51002/225000 [107:04:21<352:49:02,  7.30s/it]
 23%|██▎       | 51003/225000 [107:04:26<322:15:55,  6.67s/it]
 23%|██▎       | 51004/225000 [107:04:32<305:27:30,  6.32s/it]
 23%|██▎       | 51005/225000 [107:04:38<297:04:14,  6.15s/it]
 23%|██▎       | 51006/225000 [107:04:43<293:02:38,  6.06s/it]
 23%|██▎       | 51007/225000 [107:04:49<277:49:25,  5.75s/it]
 23%|██▎       | 51008/225000 [107:05:01<376:08:29,  7.78s/it]
 23%|██▎       | 51009/225000 [107:05:08<359:09:28,  7.43s/it]
 23%|██▎       | 51010/225000 [107:05:12<321:18:51,  6.65s/it]
 23%|██▎       | 51011/225000 [107:05:17<289:55:36,  6.00s/it]
 23%|██▎       | 51012/225000 [107:05:23<285:38:36,  5.91s/it]
 23%|██▎       | 51013/225000 [107:05:28<283:58:56,  5.88s/it]
 23%|██▎       | 51014/225000 [107:05:34<281:10:18,  5.82s/it]
 23%|██▎       | 51015/225000 [107:05:43<330:14:36,  6.83s/it]
 23%|██▎       | 51016/225000 [107:05:55<402:19:48,  8.32s/it]
 23%|██▎       | 51017/225000 [107:06:00<347:53:05,  7.20s/it]
 23%|██▎       | 51018/225000 [107:06:05<316:31:44,  6.55s/it]
 23%|██▎       | 51019/225000 [107:06:16<391:38:05,  8.10s/it]
 23%|██▎       | 51020/225000 [107:06:20<325:16:51,  6.73s/it]
 23%|██▎       | 51021/225000 [107:06:28<348:44:47,  7.22s/it]
 23%|██▎       | 51022/225000 [107:06:34<320:19:18,  6.63s/it]
 23%|██▎       | 51023/225000 [107:06:39<295:47:40,  6.12s/it]
 23%|██▎       | 51024/225000 [107:06:44<279:13:45,  5.78s/it]
 23%|██▎       | 51025/225000 [107:06:51<308:45:29,  6.39s/it]
 23%|██▎       | 51026/225000 [107:07:05<413:48:56,  8.56s/it]
 23%|██▎       | 51027/225000 [107:07:10<357:24:24,  7.40s/it]
 23%|██▎       | 51028/225000 [107:07:15<323:43:12,  6.70s/it]
 23%|██▎       | 51029/225000 [107:07:20<301:36:50,  6.24s/it]
 23%|██▎       | 51030/225000 [107:07:25<286:35:07,  5.93s/it]
 23%|██▎       | 51031/225000 [107:07:30<274:46:51,  5.69s/it]
 23%|██▎       | 51032/225000 [107:07:41<354:16:19,  7.33s/it]
 23%|██▎       | 51033/225000 [107:07:51<386:17:42,  7.99s/it]
 23%|██▎       | 51034/225000 [107:07:55<327:03:23,  6.77s/it]
 23%|██▎       | 51035/225000 [107:08:05<380:58:49,  7.88s/it]
 23%|██▎       | 51036/225000 [107:08:10<339:48:25,  7.03s/it]
 23%|██▎       | 51037/225000 [107:08:19<358:56:07,  7.43s/it]
 23%|██▎       | 51038/225000 [107:08:23<309:58:45,  6.41s/it]
 23%|██▎       | 51039/225000 [107:08:29<309:22:31,  6.40s/it]
 23%|██▎       | 51040/225000 [107:08:37<335:51:27,  6.95s/it]
 23%|██▎       | 51041/225000 [107:08:47<374:25:50,  7.75s/it]
 23%|██▎       | 51042/225000 [107:08:52<332:45:04,  6.89s/it]
 23%|██▎       | 51043/225000 [107:08:56<299:18:45,  6.19s/it]
 23%|██▎       | 51044/225000 [107:09:02<283:49:14,  5.87s/it]
 23%|██▎       | 51045/225000 [107:09:07<275:35:09,  5.70s/it]
 23%|██▎       | 51046/225000 [107:09:11<256:40:08,  5.31s/it]
 23%|██▎       | 51047/225000 [107:09:26<386:26:41,  8.00s/it]
 23%|██▎       | 51048/225000 [107:09:32<358:49:30,  7.43s/it]
 23%|██▎       | 51049/225000 [107:09:40<377:47:35,  7.82s/it]
 23%|██▎       | 51050/225000 [107:09:46<345:33:13,  7.15s/it]
                                                              
{'loss': '0.7089', 'grad_norm': '0.7609', 'learning_rate': '4.564e-05', 'epoch': '0.2269', 'num_input_tokens_seen': 179229320, 'train_runtime': '3.858e+05', 'train_tokens_per_second': '464.6'}

 23%|██▎       | 51050/225000 [107:09:46<345:33:13,  7.15s/it]
 23%|██▎       | 51051/225000 [107:09:52<333:01:26,  6.89s/it]
 23%|██▎       | 51052/225000 [107:10:01<356:13:33,  7.37s/it]
 23%|██▎       | 51053/225000 [107:10:05<313:40:00,  6.49s/it]
 23%|██▎       | 51054/225000 [107:10:13<331:46:53,  6.87s/it]
 23%|██▎       | 51055/225000 [107:10:18<307:26:26,  6.36s/it]
 23%|██▎       | 51056/225000 [107:10:23<290:22:01,  6.01s/it]
 23%|██▎       | 51057/225000 [107:10:28<272:28:18,  5.64s/it]
 23%|██▎       | 51058/225000 [107:10:34<271:10:10,  5.61s/it]
 23%|██▎       | 51059/225000 [107:10:45<357:19:51,  7.40s/it]
 23%|██▎       | 51060/225000 [107:10:50<315:17:18,  6.53s/it]
 23%|██▎       | 51061/225000 [107:10:57<327:37:37,  6.78s/it]
 23%|██▎       | 51062/225000 [107:11:07<372:57:15,  7.72s/it]
 23%|██▎       | 51063/225000 [107:11:11<317:11:45,  6.57s/it]
 23%|██▎       | 51064/225000 [107:11:21<364:22:55,  7.54s/it]
 23%|██▎       | 51065/225000 [107:11:24<306:43:11,  6.35s/it]
 23%|██▎       | 51066/225000 [107:11:33<348:16:58,  7.21s/it]
 23%|██▎       | 51067/225000 [107:11:39<318:07:56,  6.58s/it]
 23%|██▎       | 51068/225000 [107:11:43<288:44:10,  5.98s/it]
 23%|██▎       | 51069/225000 [107:11:48<277:10:51,  5.74s/it]
 23%|██▎       | 51070/225000 [107:11:55<287:47:58,  5.96s/it]
 23%|██▎       | 51071/225000 [107:12:08<386:47:37,  8.01s/it]
 23%|██▎       | 51072/225000 [107:12:14<357:16:05,  7.39s/it]
 23%|██▎       | 51073/225000 [107:12:19<322:41:03,  6.68s/it]
 23%|██▎       | 51074/225000 [107:12:24<298:57:55,  6.19s/it]
 23%|██▎       | 51075/225000 [107:12:37<408:26:22,  8.45s/it]
 23%|██▎       | 51076/225000 [107:12:42<354:15:03,  7.33s/it]
 23%|██▎       | 51077/225000 [107:12:46<311:34:07,  6.45s/it]
 23%|██▎       | 51078/225000 [107:12:56<358:14:32,  7.42s/it]
 23%|██▎       | 51079/225000 [107:13:03<347:40:44,  7.20s/it]
 23%|██▎       | 51080/225000 [107:13:07<307:32:37,  6.37s/it]
 23%|██▎       | 51081/225000 [107:13:18<366:26:20,  7.59s/it]
 23%|██▎       | 51082/225000 [107:13:24<350:27:21,  7.25s/it]
 23%|██▎       | 51083/225000 [107:13:33<368:25:16,  7.63s/it]
 23%|██▎       | 51084/225000 [107:13:36<314:17:03,  6.51s/it]
 23%|██▎       | 51085/225000 [107:13:45<342:41:55,  7.09s/it]
 23%|██▎       | 51086/225000 [107:13:50<319:57:38,  6.62s/it]
 23%|██▎       | 51087/225000 [107:13:56<302:59:00,  6.27s/it]
 23%|██▎       | 51088/225000 [107:14:01<288:05:58,  5.96s/it]
 23%|██▎       | 51089/225000 [107:14:07<280:57:43,  5.82s/it]
 23%|██▎       | 51090/225000 [107:14:12<273:16:16,  5.66s/it]
 23%|██▎       | 51091/225000 [107:14:26<389:42:05,  8.07s/it]
 23%|██▎       | 51092/225000 [107:14:31<344:52:34,  7.14s/it]
 23%|██▎       | 51093/225000 [107:14:38<353:12:09,  7.31s/it]
 23%|██▎       | 51094/225000 [107:14:43<316:57:45,  6.56s/it]
 23%|██▎       | 51095/225000 [107:14:48<290:47:14,  6.02s/it]
 23%|██▎       | 51096/225000 [107:14:53<271:38:56,  5.62s/it]
 23%|██▎       | 51097/225000 [107:15:04<351:06:38,  7.27s/it]
 23%|██▎       | 51098/225000 [107:15:11<357:41:28,  7.40s/it]
 23%|██▎       | 51099/225000 [107:15:16<311:02:43,  6.44s/it]
 23%|██▎       | 51100/225000 [107:15:27<376:43:04,  7.80s/it]
                                                              
{'loss': '0.714', 'grad_norm': '0.8337', 'learning_rate': '4.563e-05', 'epoch': '0.2271', 'num_input_tokens_seen': 179379992, 'train_runtime': '3.861e+05', 'train_tokens_per_second': '464.6'}

 23%|██▎       | 51100/225000 [107:15:27<376:43:04,  7.80s/it]
 23%|██▎       | 51101/225000 [107:15:32<342:19:25,  7.09s/it]
 23%|██▎       | 51102/225000 [107:15:39<347:33:02,  7.19s/it]
 23%|██▎       | 51103/225000 [107:15:46<338:20:02,  7.00s/it]
 23%|██▎       | 51104/225000 [107:15:54<350:08:35,  7.25s/it]
 23%|██▎       | 51105/225000 [107:16:01<346:02:08,  7.16s/it]
 23%|██▎       | 51106/225000 [107:16:05<307:19:44,  6.36s/it]
 23%|██▎       | 51107/225000 [107:16:15<355:23:10,  7.36s/it]
 23%|██▎       | 51108/225000 [107:16:20<319:44:28,  6.62s/it]
 23%|██▎       | 51109/225000 [107:16:30<364:31:05,  7.55s/it]
 23%|██▎       | 51110/225000 [107:16:33<311:13:11,  6.44s/it]
 23%|██▎       | 51111/225000 [107:16:43<358:50:36,  7.43s/it]
 23%|██▎       | 51112/225000 [107:16:47<307:22:50,  6.36s/it]
 23%|██▎       | 51113/225000 [107:16:53<295:12:05,  6.11s/it]
 23%|██▎       | 51114/225000 [107:17:01<325:00:08,  6.73s/it]
 23%|██▎       | 51115/225000 [107:17:04<281:06:51,  5.82s/it]
 23%|██▎       | 51116/225000 [107:17:15<350:20:49,  7.25s/it]
 23%|██▎       | 51117/225000 [107:17:20<311:23:38,  6.45s/it]
 23%|██▎       | 51118/225000 [107:17:25<292:33:15,  6.06s/it]
 23%|██▎       | 51119/225000 [107:17:30<279:32:14,  5.79s/it]
 23%|██▎       | 51120/225000 [107:17:38<316:21:20,  6.55s/it]
 23%|██▎       | 51121/225000 [107:17:52<415:27:55,  8.60s/it]
 23%|██▎       | 51122/225000 [107:17:55<346:03:06,  7.16s/it]
 23%|██▎       | 51123/225000 [107:18:05<377:54:42,  7.82s/it]
 23%|██▎       | 51124/225000 [107:18:08<310:52:36,  6.44s/it]
 23%|██▎       | 51125/225000 [107:18:17<352:31:38,  7.30s/it]
 23%|██▎       | 51126/225000 [107:18:23<336:17:59,  6.96s/it]
 23%|██▎       | 51127/225000 [107:18:30<328:36:28,  6.80s/it]
 23%|██▎       | 51128/225000 [107:18:37<334:14:23,  6.92s/it]
 23%|██▎       | 51129/225000 [107:18:44<335:30:44,  6.95s/it]
 23%|██▎       | 51130/225000 [107:18:48<296:27:59,  6.14s/it]
 23%|██▎       | 51131/225000 [107:18:53<274:33:22,  5.68s/it]
 23%|██▎       | 51132/225000 [107:19:01<310:11:45,  6.42s/it]
 23%|██▎       | 51133/225000 [107:19:11<367:24:32,  7.61s/it]
 23%|██▎       | 51134/225000 [107:19:17<332:46:52,  6.89s/it]
 23%|██▎       | 51135/225000 [107:19:22<309:40:47,  6.41s/it]
 23%|██▎       | 51136/225000 [107:19:27<288:43:49,  5.98s/it]
 23%|██▎       | 51137/225000 [107:19:32<279:13:43,  5.78s/it]
 23%|██▎       | 51138/225000 [107:19:38<271:44:02,  5.63s/it]
 23%|██▎       | 51139/225000 [107:19:43<270:53:43,  5.61s/it]
 23%|██▎       | 51140/225000 [107:19:52<319:13:27,  6.61s/it]
 23%|██▎       | 51141/225000 [107:20:05<416:42:48,  8.63s/it]
 23%|██▎       | 51142/225000 [107:20:10<354:41:01,  7.34s/it]
 23%|██▎       | 51143/225000 [107:20:14<310:03:38,  6.42s/it]
 23%|██▎       | 51144/225000 [107:20:24<360:11:38,  7.46s/it]
 23%|██▎       | 51145/225000 [107:20:32<367:28:46,  7.61s/it]
 23%|██▎       | 51146/225000 [107:20:36<319:08:08,  6.61s/it]
 23%|██▎       | 51147/225000 [107:20:45<345:46:39,  7.16s/it]
 23%|██▎       | 51148/225000 [107:20:50<317:01:25,  6.56s/it]
 23%|██▎       | 51149/225000 [107:20:59<348:45:06,  7.22s/it]
 23%|██▎       | 51150/225000 [107:21:03<302:49:30,  6.27s/it]
                                                              
{'loss': '0.7038', 'grad_norm': '0.8345', 'learning_rate': '4.562e-05', 'epoch': '0.2273', 'num_input_tokens_seen': 179529912, 'train_runtime': '3.865e+05', 'train_tokens_per_second': '464.5'}

 23%|██▎       | 51150/225000 [107:21:03<302:49:30,  6.27s/it]
 23%|██▎       | 51151/225000 [107:21:08<285:37:10,  5.91s/it]
 23%|██▎       | 51152/225000 [107:21:13<273:45:27,  5.67s/it]
 23%|██▎       | 51153/225000 [107:21:18<267:21:32,  5.54s/it]
 23%|██▎       | 51154/225000 [107:21:23<259:50:31,  5.38s/it]
 23%|██▎       | 51155/225000 [107:21:32<309:48:03,  6.42s/it]
 23%|██▎       | 51156/225000 [107:21:44<392:06:34,  8.12s/it]
 23%|██▎       | 51157/225000 [107:21:48<337:56:53,  7.00s/it]
 23%|██▎       | 51158/225000 [107:21:53<300:22:52,  6.22s/it]
 23%|██▎       | 51159/225000 [107:22:04<376:26:37,  7.80s/it]
 23%|██▎       | 51160/225000 [107:22:09<332:51:40,  6.89s/it]
 23%|██▎       | 51161/225000 [107:22:17<342:42:54,  7.10s/it]
 23%|██▎       | 51162/225000 [107:22:21<306:38:50,  6.35s/it]
 23%|██▎       | 51163/225000 [107:22:26<281:49:30,  5.84s/it]
 23%|██▎       | 51164/225000 [107:22:35<335:15:32,  6.94s/it]
 23%|██▎       | 51165/225000 [107:22:46<388:24:09,  8.04s/it]
 23%|██▎       | 51166/225000 [107:22:50<326:54:06,  6.77s/it]
 23%|██▎       | 51167/225000 [107:22:54<295:04:17,  6.11s/it]
 23%|██▎       | 51168/225000 [107:23:06<369:09:47,  7.65s/it]
 23%|██▎       | 51169/225000 [107:23:09<314:56:18,  6.52s/it]
 23%|██▎       | 51170/225000 [107:23:14<292:03:03,  6.05s/it]
 23%|██▎       | 51171/225000 [107:23:23<335:42:58,  6.95s/it]
 23%|██▎       | 51172/225000 [107:23:35<396:11:13,  8.21s/it]
 23%|██▎       | 51173/225000 [107:23:40<355:25:39,  7.36s/it]
 23%|██▎       | 51174/225000 [107:23:45<322:27:43,  6.68s/it]
 23%|██▎       | 51175/225000 [107:23:50<300:53:29,  6.23s/it]
 23%|██▎       | 51176/225000 [107:23:55<286:41:36,  5.94s/it]
 23%|██▎       | 51177/225000 [107:24:05<334:06:57,  6.92s/it]
 23%|██▎       | 51178/225000 [107:24:16<393:35:30,  8.15s/it]
 23%|██▎       | 51179/225000 [107:24:20<339:59:13,  7.04s/it]
 23%|██▎       | 51180/225000 [107:24:26<327:50:03,  6.79s/it]
 23%|██▎       | 51181/225000 [107:24:34<343:05:33,  7.11s/it]
 23%|██▎       | 51182/225000 [107:24:39<315:51:34,  6.54s/it]
 23%|██▎       | 51183/225000 [107:24:49<364:13:36,  7.54s/it]
 23%|██▎       | 51184/225000 [107:24:57<359:36:52,  7.45s/it]
 23%|██▎       | 51185/225000 [107:25:02<330:06:13,  6.84s/it]
 23%|██▎       | 51186/225000 [107:25:07<308:59:11,  6.40s/it]
 23%|██▎       | 51187/225000 [107:25:13<292:40:35,  6.06s/it]
 23%|██▎       | 51188/225000 [107:25:18<281:03:16,  5.82s/it]
 23%|██▎       | 51189/225000 [107:25:24<281:31:42,  5.83s/it]
 23%|██▎       | 51190/225000 [107:25:34<342:50:03,  7.10s/it]
 23%|██▎       | 51191/225000 [107:25:44<387:38:49,  8.03s/it]
 23%|██▎       | 51192/225000 [107:25:48<328:30:33,  6.80s/it]
 23%|██▎       | 51193/225000 [107:25:52<292:49:00,  6.07s/it]
 23%|██▎       | 51194/225000 [107:26:04<371:48:26,  7.70s/it]
 23%|██▎       | 51195/225000 [107:26:08<316:44:52,  6.56s/it]
 23%|██▎       | 51196/225000 [107:26:17<352:37:34,  7.30s/it]
 23%|██▎       | 51197/225000 [107:26:24<350:50:05,  7.27s/it]
 23%|██▎       | 51198/225000 [107:26:30<327:12:37,  6.78s/it]
 23%|██▎       | 51199/225000 [107:26:39<359:41:19,  7.45s/it]
 23%|██▎       | 51200/225000 [107:26:43<320:22:04,  6.64s/it]
                                                              
{'loss': '0.705', 'grad_norm': '0.7653', 'learning_rate': '4.561e-05', 'epoch': '0.2276', 'num_input_tokens_seen': 179678248, 'train_runtime': '3.868e+05', 'train_tokens_per_second': '464.5'}

 23%|██▎       | 51200/225000 [107:26:43<320:22:04,  6.64s/it]
 23%|██▎       | 51201/225000 [107:26:48<294:51:35,  6.11s/it]
 23%|██▎       | 51202/225000 [107:26:59<358:44:07,  7.43s/it]
 23%|██▎       | 51203/225000 [107:27:06<359:58:25,  7.46s/it]
 23%|██▎       | 51204/225000 [107:27:11<325:24:04,  6.74s/it]
 23%|██▎       | 51205/225000 [107:27:20<361:06:26,  7.48s/it]
 23%|██▎       | 51206/225000 [107:27:25<311:55:49,  6.46s/it]
 23%|██▎       | 51207/225000 [107:27:34<349:51:38,  7.25s/it]
 23%|██▎       | 51208/225000 [107:27:40<333:58:19,  6.92s/it]
 23%|██▎       | 51209/225000 [107:27:48<359:02:12,  7.44s/it]
 23%|██▎       | 51210/225000 [107:27:52<304:56:26,  6.32s/it]
 23%|██▎       | 51211/225000 [107:28:00<320:03:26,  6.63s/it]
 23%|██▎       | 51212/225000 [107:28:08<344:47:11,  7.14s/it]
 23%|██▎       | 51213/225000 [107:28:14<326:16:12,  6.76s/it]
 23%|██▎       | 51214/225000 [107:28:22<342:05:44,  7.09s/it]
 23%|██▎       | 51215/225000 [107:28:27<315:54:46,  6.54s/it]
 23%|██▎       | 51216/225000 [107:28:32<292:42:44,  6.06s/it]
 23%|██▎       | 51217/225000 [107:28:37<282:26:15,  5.85s/it]
 23%|██▎       | 51218/225000 [107:28:45<306:12:59,  6.34s/it]
 23%|██▎       | 51219/225000 [107:28:56<378:29:00,  7.84s/it]
 23%|██▎       | 51220/225000 [107:29:00<327:05:57,  6.78s/it]
 23%|██▎       | 51221/225000 [107:29:06<309:14:42,  6.41s/it]
 23%|██▎       | 51222/225000 [107:29:17<375:01:32,  7.77s/it]
 23%|██▎       | 51223/225000 [107:29:21<331:23:07,  6.87s/it]
 23%|██▎       | 51224/225000 [107:29:27<307:27:30,  6.37s/it]
 23%|██▎       | 51225/225000 [107:29:32<287:19:49,  5.95s/it]
 23%|██▎       | 51226/225000 [107:29:44<379:01:12,  7.85s/it]
 23%|██▎       | 51227/225000 [107:29:49<344:24:28,  7.13s/it]
 23%|██▎       | 51228/225000 [107:29:57<349:40:06,  7.24s/it]
 23%|██▎       | 51229/225000 [107:30:02<312:29:43,  6.47s/it]
 23%|██▎       | 51230/225000 [107:30:06<283:11:19,  5.87s/it]
 23%|██▎       | 51231/225000 [107:30:11<265:02:15,  5.49s/it]
 23%|██▎       | 51232/225000 [107:30:22<342:18:34,  7.09s/it]
 23%|██▎       | 51233/225000 [107:30:29<349:49:51,  7.25s/it]
 23%|██▎       | 51234/225000 [107:30:33<304:43:39,  6.31s/it]
 23%|██▎       | 51235/225000 [107:30:38<281:14:53,  5.83s/it]
 23%|██▎       | 51236/225000 [107:30:50<367:07:17,  7.61s/it]
 23%|██▎       | 51237/225000 [107:30:53<307:21:00,  6.37s/it]
 23%|██▎       | 51238/225000 [107:30:59<292:26:57,  6.06s/it]
 23%|██▎       | 51239/225000 [107:31:04<281:07:29,  5.82s/it]
 23%|██▎       | 51240/225000 [107:31:09<274:45:08,  5.69s/it]
 23%|██▎       | 51241/225000 [107:31:14<267:47:48,  5.55s/it]
 23%|██▎       | 51242/225000 [107:31:27<371:33:19,  7.70s/it]
 23%|██▎       | 51243/225000 [107:31:36<389:17:53,  8.07s/it]
 23%|██▎       | 51244/225000 [107:31:40<330:31:06,  6.85s/it]
 23%|██▎       | 51245/225000 [107:31:52<405:04:07,  8.39s/it]
 23%|██▎       | 51246/225000 [107:31:56<338:31:53,  7.01s/it]
 23%|██▎       | 51247/225000 [107:32:07<396:42:11,  8.22s/it]
 23%|██▎       | 51248/225000 [107:32:11<339:25:34,  7.03s/it]
 23%|██▎       | 51249/225000 [107:32:17<329:09:00,  6.82s/it]
 23%|██▎       | 51250/225000 [107:32:27<361:51:22,  7.50s/it]
                                                              
{'loss': '0.7114', 'grad_norm': '0.8894', 'learning_rate': '4.56e-05', 'epoch': '0.2278', 'num_input_tokens_seen': 179828776, 'train_runtime': '3.871e+05', 'train_tokens_per_second': '464.5'}

 23%|██▎       | 51250/225000 [107:32:27<361:51:22,  7.50s/it]
 23%|██▎       | 51251/225000 [107:32:31<324:23:29,  6.72s/it]
 23%|██▎       | 51252/225000 [107:32:36<300:04:06,  6.22s/it]
 23%|██▎       | 51253/225000 [107:32:42<284:22:00,  5.89s/it]
 23%|██▎       | 51254/225000 [107:32:48<295:05:51,  6.11s/it]
 23%|██▎       | 51255/225000 [107:33:01<392:31:08,  8.13s/it]
 23%|██▎       | 51256/225000 [107:33:06<340:46:39,  7.06s/it]
 23%|██▎       | 51257/225000 [107:33:10<297:44:46,  6.17s/it]
 23%|██▎       | 51258/225000 [107:33:21<371:12:32,  7.69s/it]
 23%|██▎       | 51259/225000 [107:33:25<322:34:10,  6.68s/it]
 23%|██▎       | 51260/225000 [107:33:33<332:48:41,  6.90s/it]
 23%|██▎       | 51261/225000 [107:33:40<341:59:35,  7.09s/it]
 23%|██▎       | 51262/225000 [107:33:46<322:08:45,  6.68s/it]
 23%|██▎       | 51263/225000 [107:33:51<302:36:16,  6.27s/it]
 23%|██▎       | 51264/225000 [107:33:57<295:04:59,  6.11s/it]
 23%|██▎       | 51265/225000 [107:34:03<286:23:03,  5.93s/it]
 23%|██▎       | 51266/225000 [107:34:08<275:21:00,  5.71s/it]
 23%|██▎       | 51267/225000 [107:34:13<268:38:36,  5.57s/it]
 23%|██▎       | 51268/225000 [107:34:19<281:04:31,  5.82s/it]
 23%|██▎       | 51269/225000 [107:34:32<380:33:08,  7.89s/it]
 23%|██▎       | 51270/225000 [107:34:38<352:28:56,  7.30s/it]
 23%|██▎       | 51271/225000 [107:34:43<325:06:36,  6.74s/it]
 23%|██▎       | 51272/225000 [107:34:49<312:27:45,  6.47s/it]
 23%|██▎       | 51273/225000 [107:34:55<300:04:20,  6.22s/it]
 23%|██▎       | 51274/225000 [107:35:00<286:11:53,  5.93s/it]
 23%|██▎       | 51275/225000 [107:35:06<282:30:40,  5.85s/it]
 23%|██▎       | 51276/225000 [107:35:17<363:37:48,  7.54s/it]
 23%|██▎       | 51277/225000 [107:35:27<387:47:56,  8.04s/it]
 23%|██▎       | 51278/225000 [107:35:36<412:27:07,  8.55s/it]
 23%|██▎       | 51279/225000 [107:35:41<357:23:35,  7.41s/it]
 23%|██▎       | 51280/225000 [107:35:46<326:24:40,  6.76s/it]
 23%|██▎       | 51281/225000 [107:35:52<307:27:40,  6.37s/it]
 23%|██▎       | 51282/225000 [107:35:57<297:34:17,  6.17s/it]
 23%|██▎       | 51283/225000 [107:36:09<378:19:32,  7.84s/it]
 23%|██▎       | 51284/225000 [107:36:19<407:06:31,  8.44s/it]
 23%|██▎       | 51285/225000 [107:36:24<351:28:55,  7.28s/it]
 23%|██▎       | 51286/225000 [107:36:35<408:11:58,  8.46s/it]
 23%|██▎       | 51287/225000 [107:36:38<338:29:15,  7.01s/it]
 23%|██▎       | 51288/225000 [107:36:46<348:53:05,  7.23s/it]
 23%|██▎       | 51289/225000 [107:36:51<310:44:31,  6.44s/it]
 23%|██▎       | 51290/225000 [107:36:55<285:01:27,  5.91s/it]
 23%|██▎       | 51291/225000 [107:37:01<273:34:13,  5.67s/it]
 23%|██▎       | 51292/225000 [107:37:11<338:25:27,  7.01s/it]
 23%|██▎       | 51293/225000 [107:37:19<352:49:57,  7.31s/it]
 23%|██▎       | 51294/225000 [107:37:23<316:07:35,  6.55s/it]
 23%|██▎       | 51295/225000 [107:37:29<301:11:56,  6.24s/it]
 23%|██▎       | 51296/225000 [107:37:34<284:27:09,  5.90s/it]
 23%|██▎       | 51297/225000 [107:37:43<327:15:00,  6.78s/it]
 23%|██▎       | 51298/225000 [107:37:54<396:13:57,  8.21s/it]
 23%|██▎       | 51299/225000 [107:38:00<356:04:56,  7.38s/it]
 23%|██▎       | 51300/225000 [107:38:06<341:57:33,  7.09s/it]
                                                              
{'loss': '0.7104', 'grad_norm': '0.8837', 'learning_rate': '4.559e-05', 'epoch': '0.228', 'num_input_tokens_seen': 179979792, 'train_runtime': '3.875e+05', 'train_tokens_per_second': '464.5'}

 23%|██▎       | 51300/225000 [107:38:06<341:57:33,  7.09s/it]
 23%|██▎       | 51301/225000 [107:38:13<339:15:03,  7.03s/it]
 23%|██▎       | 51302/225000 [107:38:18<307:15:06,  6.37s/it]
 23%|██▎       | 51303/225000 [107:38:27<346:40:31,  7.19s/it]
 23%|██▎       | 51304/225000 [107:38:35<352:55:38,  7.31s/it]
 23%|██▎       | 51305/225000 [107:38:39<301:55:27,  6.26s/it]
 23%|██▎       | 51306/225000 [107:38:44<292:55:45,  6.07s/it]
 23%|██▎       | 51307/225000 [107:38:50<285:28:18,  5.92s/it]
 23%|██▎       | 51308/225000 [107:38:55<281:03:14,  5.83s/it]
 23%|██▎       | 51309/225000 [107:39:01<275:15:38,  5.71s/it]
 23%|██▎       | 51310/225000 [107:39:07<288:28:57,  5.98s/it]
 23%|██▎       | 51311/225000 [107:39:21<405:59:48,  8.41s/it]
 23%|██▎       | 51312/225000 [107:39:27<362:48:02,  7.52s/it]
 23%|██▎       | 51313/225000 [107:39:32<327:36:36,  6.79s/it]
 23%|██▎       | 51314/225000 [107:39:41<362:23:57,  7.51s/it]
 23%|██▎       | 51315/225000 [107:39:46<318:40:27,  6.61s/it]
 23%|██▎       | 51316/225000 [107:39:51<297:13:01,  6.16s/it]
 23%|██▎       | 51317/225000 [107:39:55<273:13:52,  5.66s/it]
 23%|██▎       | 51318/225000 [107:40:04<323:39:40,  6.71s/it]
 23%|██▎       | 51319/225000 [107:40:14<363:46:08,  7.54s/it]
 23%|██▎       | 51320/225000 [107:40:19<325:12:00,  6.74s/it]
 23%|██▎       | 51321/225000 [107:40:23<288:56:02,  5.99s/it]
 23%|██▎       | 51322/225000 [107:40:37<399:16:54,  8.28s/it]
 23%|██▎       | 51323/225000 [107:40:41<335:22:10,  6.95s/it]
 23%|██▎       | 51324/225000 [107:40:47<325:01:57,  6.74s/it]
 23%|██▎       | 51325/225000 [107:40:54<330:32:29,  6.85s/it]
 23%|██▎       | 51326/225000 [107:41:01<338:14:13,  7.01s/it]
 23%|██▎       | 51327/225000 [107:41:09<350:10:27,  7.26s/it]
 23%|██▎       | 51328/225000 [107:41:14<313:42:28,  6.50s/it]
 23%|██▎       | 51329/225000 [107:41:22<330:34:15,  6.85s/it]
 23%|██▎       | 51330/225000 [107:41:26<299:42:06,  6.21s/it]
 23%|██▎       | 51331/225000 [107:41:31<281:19:33,  5.83s/it]
 23%|██▎       | 51332/225000 [107:41:42<360:51:03,  7.48s/it]
 23%|██▎       | 51333/225000 [107:41:47<321:07:47,  6.66s/it]
 23%|██▎       | 51334/225000 [107:41:52<295:55:47,  6.13s/it]
 23%|██▎       | 51335/225000 [107:41:58<284:51:25,  5.90s/it]
 23%|██▎       | 51336/225000 [107:42:03<273:00:19,  5.66s/it]
 23%|██▎       | 51337/225000 [107:42:11<308:09:09,  6.39s/it]
 23%|██▎       | 51338/225000 [107:42:21<369:33:26,  7.66s/it]
 23%|██▎       | 51339/225000 [107:42:31<396:58:58,  8.23s/it]
 23%|██▎       | 51340/225000 [107:42:36<347:18:05,  7.20s/it]
 23%|██▎       | 51341/225000 [107:42:40<307:52:58,  6.38s/it]
 23%|██▎       | 51342/225000 [107:42:45<288:03:19,  5.97s/it]
 23%|██▎       | 51343/225000 [107:42:51<283:41:23,  5.88s/it]
 23%|██▎       | 51344/225000 [107:43:01<344:10:26,  7.13s/it]
 23%|██▎       | 51345/225000 [107:43:09<356:34:35,  7.39s/it]
 23%|██▎       | 51346/225000 [107:43:14<328:05:56,  6.80s/it]
 23%|██▎       | 51347/225000 [107:43:26<401:06:54,  8.32s/it]
 23%|██▎       | 51348/225000 [107:43:30<336:43:16,  6.98s/it]
 23%|██▎       | 51349/225000 [107:43:39<368:13:20,  7.63s/it]
 23%|██▎       | 51350/225000 [107:43:45<337:42:00,  7.00s/it]
                                                              
{'loss': '0.7005', 'grad_norm': '0.9032', 'learning_rate': '4.558e-05', 'epoch': '0.2282', 'num_input_tokens_seen': 180130976, 'train_runtime': '3.878e+05', 'train_tokens_per_second': '464.5'}

 23%|██▎       | 51350/225000 [107:43:45<337:42:00,  7.00s/it]
 23%|██▎       | 51351/225000 [107:43:49<295:39:21,  6.13s/it]
 23%|██▎       | 51352/225000 [107:43:54<280:39:14,  5.82s/it]
 23%|██▎       | 51353/225000 [107:44:03<333:32:04,  6.91s/it]
 23%|██▎       | 51354/225000 [107:44:10<335:08:58,  6.95s/it]
 23%|██▎       | 51355/225000 [107:44:16<309:40:22,  6.42s/it]
 23%|██▎       | 51356/225000 [107:44:20<281:06:42,  5.83s/it]
 23%|██▎       | 51357/225000 [107:44:29<333:31:28,  6.91s/it]
 23%|██▎       | 51358/225000 [107:44:39<371:59:04,  7.71s/it]
 23%|██▎       | 51359/225000 [107:44:44<326:51:44,  6.78s/it]
 23%|██▎       | 51360/225000 [107:44:51<340:39:07,  7.06s/it]
 23%|██▎       | 51361/225000 [107:44:55<297:20:33,  6.16s/it]
 23%|██▎       | 51362/225000 [107:45:08<388:00:40,  8.04s/it]
 23%|██▎       | 51363/225000 [107:45:12<334:04:04,  6.93s/it]
 23%|██▎       | 51364/225000 [107:45:20<351:09:12,  7.28s/it]
 23%|██▎       | 51365/225000 [107:45:25<308:02:39,  6.39s/it]
 23%|██▎       | 51366/225000 [107:45:30<292:25:42,  6.06s/it]
 23%|██▎       | 51367/225000 [107:45:41<366:47:51,  7.60s/it]
 23%|██▎       | 51368/225000 [107:45:45<315:27:51,  6.54s/it]
 23%|██▎       | 51369/225000 [107:45:52<325:07:50,  6.74s/it]
 23%|██▎       | 51370/225000 [107:46:01<345:20:13,  7.16s/it]
 23%|██▎       | 51371/225000 [107:46:05<309:14:08,  6.41s/it]
 23%|██▎       | 51372/225000 [107:46:10<281:47:31,  5.84s/it]
 23%|██▎       | 51373/225000 [107:46:14<264:36:11,  5.49s/it]
 23%|██▎       | 51374/225000 [107:46:24<320:50:34,  6.65s/it]
 23%|██▎       | 51375/225000 [107:46:32<344:53:14,  7.15s/it]
 23%|██▎       | 51376/225000 [107:46:37<310:02:03,  6.43s/it]
 23%|██▎       | 51377/225000 [107:46:44<321:31:27,  6.67s/it]
 23%|██▎       | 51378/225000 [107:46:52<337:15:03,  6.99s/it]
 23%|██▎       | 51379/225000 [107:46:59<342:55:41,  7.11s/it]
 23%|██▎       | 51380/225000 [107:47:04<305:07:16,  6.33s/it]
 23%|██▎       | 51381/225000 [107:47:12<333:49:35,  6.92s/it]
 23%|██▎       | 51382/225000 [107:47:18<317:35:29,  6.59s/it]
 23%|██▎       | 51383/225000 [107:47:23<300:29:27,  6.23s/it]
 23%|██▎       | 51384/225000 [107:47:29<290:59:24,  6.03s/it]
 23%|██▎       | 51385/225000 [107:47:34<280:01:38,  5.81s/it]
 23%|██▎       | 51386/225000 [107:47:39<273:08:35,  5.66s/it]
 23%|██▎       | 51387/225000 [107:47:45<271:26:34,  5.63s/it]
 23%|██▎       | 51388/225000 [107:47:50<264:28:20,  5.48s/it]
 23%|██▎       | 51389/225000 [107:47:56<275:50:30,  5.72s/it]
 23%|██▎       | 51390/225000 [107:48:09<375:59:32,  7.80s/it]
 23%|██▎       | 51391/225000 [107:48:16<359:07:01,  7.45s/it]
 23%|██▎       | 51392/225000 [107:48:24<374:28:44,  7.77s/it]
 23%|██▎       | 51393/225000 [107:48:29<327:23:10,  6.79s/it]
 23%|██▎       | 51394/225000 [107:48:33<290:52:19,  6.03s/it]
 23%|██▎       | 51395/225000 [107:48:43<344:10:19,  7.14s/it]
 23%|██▎       | 51396/225000 [107:48:53<389:01:30,  8.07s/it]
 23%|██▎       | 51397/225000 [107:49:00<370:28:19,  7.68s/it]
 23%|██▎       | 51398/225000 [107:49:06<355:22:54,  7.37s/it]
 23%|██▎       | 51399/225000 [107:49:10<308:07:54,  6.39s/it]
 23%|██▎       | 51400/225000 [107:49:21<367:00:20,  7.61s/it]
                                                              
{'loss': '0.7064', 'grad_norm': '0.7179', 'learning_rate': '4.557e-05', 'epoch': '0.2284', 'num_input_tokens_seen': 180279432, 'train_runtime': '3.882e+05', 'train_tokens_per_second': '464.4'}

 23%|██▎       | 51400/225000 [107:49:21<367:00:20,  7.61s/it]
 23%|██▎       | 51401/225000 [107:49:25<322:12:21,  6.68s/it]
 23%|██▎       | 51402/225000 [107:49:35<370:03:55,  7.67s/it]
 23%|██▎       | 51403/225000 [107:49:44<381:57:15,  7.92s/it]
 23%|██▎       | 51404/225000 [107:49:49<337:33:34,  7.00s/it]
 23%|██▎       | 51405/225000 [107:49:54<313:27:21,  6.50s/it]
 23%|██▎       | 51406/225000 [107:49:58<284:43:11,  5.90s/it]
 23%|██▎       | 51407/225000 [107:50:04<281:53:41,  5.85s/it]
 23%|██▎       | 51408/225000 [107:50:18<397:48:34,  8.25s/it]
 23%|██▎       | 51409/225000 [107:50:22<329:26:16,  6.83s/it]
 23%|██▎       | 51410/225000 [107:50:30<359:24:47,  7.45s/it]
 23%|██▎       | 51411/225000 [107:50:36<326:11:50,  6.76s/it]
 23%|██▎       | 51412/225000 [107:50:45<357:26:57,  7.41s/it]
 23%|██▎       | 51413/225000 [107:50:49<315:23:06,  6.54s/it]
 23%|██▎       | 51414/225000 [107:50:54<294:05:43,  6.10s/it]
 23%|██▎       | 51415/225000 [107:51:03<331:55:01,  6.88s/it]
 23%|██▎       | 51416/225000 [107:51:12<363:28:18,  7.54s/it]
 23%|██▎       | 51417/225000 [107:51:20<375:20:10,  7.78s/it]
 23%|██▎       | 51418/225000 [107:51:25<337:35:55,  7.00s/it]
 23%|██▎       | 51419/225000 [107:51:31<311:53:01,  6.47s/it]
 23%|██▎       | 51420/225000 [107:51:37<304:54:18,  6.32s/it]
 23%|██▎       | 51421/225000 [107:51:42<297:24:25,  6.17s/it]
 23%|██▎       | 51422/225000 [107:51:48<286:23:38,  5.94s/it]
 23%|██▎       | 51423/225000 [107:52:01<388:02:48,  8.05s/it]
 23%|██▎       | 51424/225000 [107:52:08<377:08:52,  7.82s/it]
 23%|██▎       | 51425/225000 [107:52:18<407:49:53,  8.46s/it]
 23%|██▎       | 51426/225000 [107:52:22<347:09:41,  7.20s/it]
 23%|██▎       | 51427/225000 [107:52:28<319:35:36,  6.63s/it]
 23%|██▎       | 51428/225000 [107:52:38<380:09:33,  7.88s/it]
 23%|██▎       | 51429/225000 [107:52:43<324:47:12,  6.74s/it]
 23%|██▎       | 51430/225000 [107:52:47<287:04:02,  5.95s/it]
 23%|██▎       | 51431/225000 [107:52:59<383:00:22,  7.94s/it]
 23%|██▎       | 51432/225000 [107:53:04<336:33:06,  6.98s/it]
 23%|██▎       | 51433/225000 [107:53:12<350:25:41,  7.27s/it]
 23%|██▎       | 51434/225000 [107:53:17<325:29:20,  6.75s/it]
 23%|██▎       | 51435/225000 [107:53:23<312:43:39,  6.49s/it]
 23%|██▎       | 51436/225000 [107:53:29<300:55:00,  6.24s/it]
 23%|██▎       | 51437/225000 [107:53:39<348:31:43,  7.23s/it]
 23%|██▎       | 51438/225000 [107:53:42<299:10:56,  6.21s/it]
 23%|██▎       | 51439/225000 [107:53:53<363:07:24,  7.53s/it]
 23%|██▎       | 51440/225000 [107:53:58<330:48:44,  6.86s/it]
 23%|██▎       | 51441/225000 [107:54:07<352:12:11,  7.31s/it]
 23%|██▎       | 51442/225000 [107:54:11<316:59:51,  6.58s/it]
 23%|██▎       | 51443/225000 [107:54:16<282:23:55,  5.86s/it]
 23%|██▎       | 51444/225000 [107:54:22<289:27:40,  6.00s/it]
 23%|██▎       | 51445/225000 [107:54:32<350:44:14,  7.28s/it]
 23%|██▎       | 51446/225000 [107:54:38<332:15:18,  6.89s/it]
 23%|██▎       | 51447/225000 [107:54:44<315:05:29,  6.54s/it]
 23%|██▎       | 51448/225000 [107:54:49<300:20:03,  6.23s/it]
 23%|██▎       | 51449/225000 [107:54:55<288:14:17,  5.98s/it]
 23%|██▎       | 51450/225000 [107:55:00<278:45:01,  5.78s/it]
                                                              
{'loss': '0.7103', 'grad_norm': '0.8755', 'learning_rate': '4.556e-05', 'epoch': '0.2287', 'num_input_tokens_seen': 180430648, 'train_runtime': '3.885e+05', 'train_tokens_per_second': '464.4'}

 23%|██▎       | 51450/225000 [107:55:00<278:45:01,  5.78s/it]
 23%|██▎       | 51451/225000 [107:55:06<273:46:31,  5.68s/it]
 23%|██▎       | 51452/225000 [107:55:19<382:02:46,  7.92s/it]
 23%|██▎       | 51453/225000 [107:55:25<361:56:32,  7.51s/it]
 23%|██▎       | 51454/225000 [107:55:29<313:18:23,  6.50s/it]
 23%|██▎       | 51455/225000 [107:55:37<323:52:54,  6.72s/it]
 23%|██▎       | 51456/225000 [107:55:45<352:12:31,  7.31s/it]
 23%|██▎       | 51457/225000 [107:55:49<300:14:51,  6.23s/it]
 23%|██▎       | 51458/225000 [107:55:59<351:28:17,  7.29s/it]
 23%|██▎       | 51459/225000 [107:56:03<311:38:18,  6.46s/it]
 23%|██▎       | 51460/225000 [107:56:10<311:56:35,  6.47s/it]
 23%|██▎       | 51461/225000 [107:56:18<340:08:35,  7.06s/it]
 23%|██▎       | 51462/225000 [107:56:24<315:03:24,  6.54s/it]
 23%|██▎       | 51463/225000 [107:56:29<299:40:20,  6.22s/it]
 23%|██▎       | 51464/225000 [107:56:35<293:33:50,  6.09s/it]
 23%|██▎       | 51465/225000 [107:56:44<341:58:29,  7.09s/it]
 23%|██▎       | 51466/225000 [107:56:53<360:49:31,  7.49s/it]
 23%|██▎       | 51467/225000 [107:57:00<356:50:04,  7.40s/it]
 23%|██▎       | 51468/225000 [107:57:08<366:26:32,  7.60s/it]
 23%|██▎       | 51469/225000 [107:57:13<330:54:21,  6.86s/it]
 23%|██▎       | 51470/225000 [107:57:17<289:54:15,  6.01s/it]
 23%|██▎       | 51471/225000 [107:57:23<292:24:19,  6.07s/it]
 23%|██▎       | 51472/225000 [107:57:32<335:27:36,  6.96s/it]
 23%|██▎       | 51473/225000 [107:57:37<305:23:48,  6.34s/it]
 23%|██▎       | 51474/225000 [107:57:42<277:02:01,  5.75s/it]
 23%|██▎       | 51475/225000 [107:57:47<271:31:38,  5.63s/it]
 23%|██▎       | 51476/225000 [107:57:53<275:12:34,  5.71s/it]
 23%|██▎       | 51477/225000 [107:57:59<275:48:57,  5.72s/it]
 23%|██▎       | 51478/225000 [107:58:04<269:16:01,  5.59s/it]
 23%|██▎       | 51479/225000 [107:58:13<326:08:03,  6.77s/it]
 23%|██▎       | 51480/225000 [107:58:26<412:41:11,  8.56s/it]
 23%|██▎       | 51481/225000 [107:58:32<378:27:03,  7.85s/it]
 23%|██▎       | 51482/225000 [107:58:37<334:41:08,  6.94s/it]
 23%|██▎       | 51483/225000 [107:58:43<313:57:23,  6.51s/it]
 23%|██▎       | 51484/225000 [107:58:48<298:23:27,  6.19s/it]
 23%|██▎       | 51485/225000 [107:58:54<286:15:02,  5.94s/it]
 23%|██▎       | 51486/225000 [107:59:06<386:06:21,  8.01s/it]
 23%|██▎       | 51487/225000 [107:59:13<369:14:32,  7.66s/it]
 23%|██▎       | 51488/225000 [107:59:18<324:14:36,  6.73s/it]
 23%|██▎       | 51489/225000 [107:59:23<304:15:17,  6.31s/it]
 23%|██▎       | 51490/225000 [107:59:34<369:07:32,  7.66s/it]
 23%|██▎       | 51491/225000 [107:59:40<343:25:34,  7.13s/it]
 23%|██▎       | 51492/225000 [107:59:48<361:45:49,  7.51s/it]
 23%|██▎       | 51493/225000 [107:59:53<320:41:40,  6.65s/it]
 23%|██▎       | 51494/225000 [107:59:58<294:51:46,  6.12s/it]
 23%|██▎       | 51495/225000 [108:00:03<283:36:42,  5.88s/it]
 23%|██▎       | 51496/225000 [108:00:16<384:30:49,  7.98s/it]
 23%|██▎       | 51497/225000 [108:00:21<345:03:18,  7.16s/it]
 23%|██▎       | 51498/225000 [108:00:30<371:43:00,  7.71s/it]
 23%|██▎       | 51499/225000 [108:00:35<330:03:03,  6.85s/it]
 23%|██▎       | 51500/225000 [108:00:41<315:10:05,  6.54s/it]
                                                              
{'loss': '0.7079', 'grad_norm': '0.7115', 'learning_rate': '4.555e-05', 'epoch': '0.2289', 'num_input_tokens_seen': 180579528, 'train_runtime': '3.888e+05', 'train_tokens_per_second': '464.4'}

 23%|██▎       | 51500/225000 [108:00:41<315:10:05,  6.54s/it]
 23%|██▎       | 51501/225000 [108:00:53<392:56:25,  8.15s/it]
 23%|██▎       | 51502/225000 [108:00:57<334:04:00,  6.93s/it]
 23%|██▎       | 51503/225000 [108:01:04<343:16:19,  7.12s/it]
 23%|██▎       | 51504/225000 [108:01:10<327:30:42,  6.80s/it]
 23%|██▎       | 51505/225000 [108:01:15<294:04:19,  6.10s/it]
 23%|██▎       | 51506/225000 [108:01:22<309:35:49,  6.42s/it]
 23%|██▎       | 51507/225000 [108:01:31<344:56:22,  7.16s/it]
 23%|██▎       | 51508/225000 [108:01:35<305:59:01,  6.35s/it]
 23%|██▎       | 51509/225000 [108:01:44<331:55:03,  6.89s/it]
 23%|██▎       | 51510/225000 [108:01:52<353:21:34,  7.33s/it]
 23%|██▎       | 51511/225000 [108:01:57<314:01:32,  6.52s/it]
 23%|██▎       | 51512/225000 [108:02:04<325:16:59,  6.75s/it]
 23%|██▎       | 51513/225000 [108:02:08<284:03:00,  5.89s/it]
 23%|██▎       | 51514/225000 [108:02:13<267:49:42,  5.56s/it]
 23%|██▎       | 51515/225000 [108:02:18<265:04:53,  5.50s/it]
 23%|██▎       | 51516/225000 [108:02:28<336:48:49,  6.99s/it]
 23%|██▎       | 51517/225000 [108:02:40<400:39:34,  8.31s/it]
 23%|██▎       | 51518/225000 [108:02:44<344:51:48,  7.16s/it]
 23%|██▎       | 51519/225000 [108:02:49<306:22:14,  6.36s/it]
 23%|██▎       | 51520/225000 [108:02:58<344:02:33,  7.14s/it]
 23%|██▎       | 51521/225000 [108:03:07<376:43:28,  7.82s/it]
 23%|██▎       | 51522/225000 [108:03:12<331:44:43,  6.88s/it]
 23%|██▎       | 51523/225000 [108:03:17<311:53:18,  6.47s/it]
 23%|██▎       | 51524/225000 [108:03:23<301:22:35,  6.25s/it]
 23%|██▎       | 51525/225000 [108:03:29<290:13:46,  6.02s/it]
 23%|██▎       | 51526/225000 [108:03:35<301:16:46,  6.25s/it]
 23%|██▎       | 51527/225000 [108:03:48<387:18:13,  8.04s/it]
 23%|██▎       | 51528/225000 [108:03:51<324:51:45,  6.74s/it]
 23%|██▎       | 51529/225000 [108:04:00<351:40:47,  7.30s/it]
 23%|██▎       | 51530/225000 [108:04:07<345:31:29,  7.17s/it]
 23%|██▎       | 51531/225000 [108:04:14<346:05:10,  7.18s/it]
 23%|██▎       | 51532/225000 [108:04:22<352:59:43,  7.33s/it]
 23%|██▎       | 51533/225000 [108:04:29<350:11:18,  7.27s/it]
 23%|██▎       | 51534/225000 [108:04:33<310:10:35,  6.44s/it]
 23%|██▎       | 51535/225000 [108:04:38<287:28:12,  5.97s/it]
 23%|██▎       | 51536/225000 [108:04:43<268:36:42,  5.57s/it]
 23%|██▎       | 51537/225000 [108:04:56<378:25:55,  7.85s/it]
 23%|██▎       | 51538/225000 [108:04:59<316:24:10,  6.57s/it]
 23%|██▎       | 51539/225000 [108:05:08<350:27:59,  7.27s/it]
 23%|██▎       | 51540/225000 [108:05:12<302:08:18,  6.27s/it]
 23%|██▎       | 51541/225000 [108:05:19<303:26:38,  6.30s/it]
 23%|██▎       | 51542/225000 [108:05:29<360:36:27,  7.48s/it]
 23%|██▎       | 51543/225000 [108:05:33<316:34:24,  6.57s/it]
 23%|██▎       | 51544/225000 [108:05:38<290:35:38,  6.03s/it]
 23%|██▎       | 51545/225000 [108:05:49<363:39:29,  7.55s/it]
 23%|██▎       | 51546/225000 [108:05:55<331:32:10,  6.88s/it]
 23%|██▎       | 51547/225000 [108:05:59<291:04:42,  6.04s/it]
 23%|██▎       | 51548/225000 [108:06:03<268:09:33,  5.57s/it]
 23%|██▎       | 51549/225000 [108:06:13<327:40:31,  6.80s/it]
 23%|██▎       | 51550/225000 [108:06:22<358:34:19,  7.44s/it]
                                                              
{'loss': '0.7044', 'grad_norm': '0.7704', 'learning_rate': '4.554e-05', 'epoch': '0.2291', 'num_input_tokens_seen': 180729488, 'train_runtime': '3.892e+05', 'train_tokens_per_second': '464.4'}

 23%|██▎       | 51550/225000 [108:06:22<358:34:19,  7.44s/it]
 23%|██▎       | 51551/225000 [108:06:25<305:15:37,  6.34s/it]
 23%|██▎       | 51552/225000 [108:06:35<354:22:10,  7.36s/it]
 23%|██▎       | 51553/225000 [108:06:39<307:10:39,  6.38s/it]
 23%|██▎       | 51554/225000 [108:06:47<331:45:56,  6.89s/it]
 23%|██▎       | 51555/225000 [108:06:54<334:01:23,  6.93s/it]
 23%|██▎       | 51556/225000 [108:06:59<299:48:31,  6.22s/it]
 23%|██▎       | 51557/225000 [108:07:03<272:38:01,  5.66s/it]
 23%|██▎       | 51558/225000 [108:07:13<333:40:46,  6.93s/it]
 23%|██▎       | 51559/225000 [108:07:23<381:34:24,  7.92s/it]
 23%|██▎       | 51560/225000 [108:07:28<327:56:28,  6.81s/it]
 23%|██▎       | 51561/225000 [108:07:36<354:14:13,  7.35s/it]
 23%|██▎       | 51562/225000 [108:07:45<377:23:06,  7.83s/it]
 23%|██▎       | 51563/225000 [108:07:52<367:27:18,  7.63s/it]
 23%|██▎       | 51564/225000 [108:07:58<333:15:27,  6.92s/it]
 23%|██▎       | 51565/225000 [108:08:02<301:37:27,  6.26s/it]
 23%|██▎       | 51566/225000 [108:08:07<282:10:09,  5.86s/it]
 23%|██▎       | 51567/225000 [108:08:13<274:27:59,  5.70s/it]
 23%|██▎       | 51568/225000 [108:08:22<329:20:31,  6.84s/it]
 23%|██▎       | 51569/225000 [108:08:33<391:27:15,  8.13s/it]
 23%|██▎       | 51570/225000 [108:08:37<333:41:43,  6.93s/it]
 23%|██▎       | 51571/225000 [108:08:47<377:07:00,  7.83s/it]
 23%|██▎       | 51572/225000 [108:08:51<319:28:20,  6.63s/it]
 23%|██▎       | 51573/225000 [108:08:56<294:50:06,  6.12s/it]
 23%|██▎       | 51574/225000 [108:09:01<283:55:23,  5.89s/it]
 23%|██▎       | 51575/225000 [108:09:07<277:10:04,  5.75s/it]
 23%|██▎       | 51576/225000 [108:09:12<273:10:27,  5.67s/it]
 23%|██▎       | 51577/225000 [108:09:21<317:21:47,  6.59s/it]
 23%|██▎       | 51578/225000 [108:09:34<412:32:48,  8.56s/it]
 23%|██▎       | 51579/225000 [108:09:38<346:28:55,  7.19s/it]
 23%|██▎       | 51580/225000 [108:09:46<350:46:30,  7.28s/it]
 23%|██▎       | 51581/225000 [108:09:53<353:10:09,  7.33s/it]
 23%|██▎       | 51582/225000 [108:10:00<339:52:33,  7.06s/it]
 23%|██▎       | 51583/225000 [108:10:07<348:23:35,  7.23s/it]
 23%|██▎       | 51584/225000 [108:10:14<345:41:43,  7.18s/it]
 23%|██▎       | 51585/225000 [108:10:22<354:38:25,  7.36s/it]
 23%|██▎       | 51586/225000 [108:10:27<317:15:10,  6.59s/it]
 23%|██▎       | 51587/225000 [108:10:34<322:22:27,  6.69s/it]
 23%|██▎       | 51588/225000 [108:10:42<348:35:18,  7.24s/it]
 23%|██▎       | 51589/225000 [108:10:51<372:39:24,  7.74s/it]
 23%|██▎       | 51590/225000 [108:10:55<310:21:31,  6.44s/it]
 23%|██▎       | 51591/225000 [108:10:59<282:21:47,  5.86s/it]
 23%|██▎       | 51592/225000 [108:11:11<363:43:20,  7.55s/it]
 23%|██▎       | 51593/225000 [108:11:14<310:56:19,  6.46s/it]
 23%|██▎       | 51594/225000 [108:11:20<296:51:48,  6.16s/it]
 23%|██▎       | 51595/225000 [108:11:26<294:38:45,  6.12s/it]
 23%|██▎       | 51596/225000 [108:11:32<287:57:09,  5.98s/it]
 23%|██▎       | 51597/225000 [108:11:37<278:09:14,  5.77s/it]
 23%|██▎       | 51598/225000 [108:11:43<275:42:06,  5.72s/it]
 23%|██▎       | 51599/225000 [108:11:56<387:59:58,  8.06s/it]
 23%|██▎       | 51600/225000 [108:12:03<373:58:25,  7.76s/it]
                                                              
{'loss': '0.7137', 'grad_norm': '0.961', 'learning_rate': '4.553e-05', 'epoch': '0.2293', 'num_input_tokens_seen': 180880176, 'train_runtime': '3.895e+05', 'train_tokens_per_second': '464.4'}

 23%|██▎       | 51600/225000 [108:12:03<373:58:25,  7.76s/it]
 23%|██▎       | 51601/225000 [108:12:07<324:55:24,  6.75s/it]
 23%|██▎       | 51602/225000 [108:12:13<309:10:33,  6.42s/it]
 23%|██▎       | 51603/225000 [108:12:24<371:51:56,  7.72s/it]
 23%|██▎       | 51604/225000 [108:12:28<314:05:31,  6.52s/it]
 23%|██▎       | 51605/225000 [108:12:37<357:30:53,  7.42s/it]
 23%|██▎       | 51606/225000 [108:12:42<321:05:27,  6.67s/it]
 23%|██▎       | 51607/225000 [108:12:50<332:53:44,  6.91s/it]
 23%|██▎       | 51608/225000 [108:12:58<361:00:56,  7.50s/it]
 23%|██▎       | 51609/225000 [108:13:04<327:04:58,  6.79s/it]
 23%|██▎       | 51610/225000 [108:13:12<358:03:19,  7.43s/it]
 23%|██▎       | 51611/225000 [108:13:16<298:11:02,  6.19s/it]
 23%|██▎       | 51612/225000 [108:13:24<330:07:46,  6.85s/it]
 23%|██▎       | 51613/225000 [108:13:32<338:51:20,  7.04s/it]
 23%|██▎       | 51614/225000 [108:13:37<307:50:41,  6.39s/it]
 23%|██▎       | 51615/225000 [108:13:41<287:13:52,  5.96s/it]
 23%|██▎       | 51616/225000 [108:13:47<287:03:14,  5.96s/it]
 23%|██▎       | 51617/225000 [108:13:52<273:01:52,  5.67s/it]
 23%|██▎       | 51618/225000 [108:13:59<280:03:03,  5.81s/it]
 23%|██▎       | 51619/225000 [108:14:11<373:26:43,  7.75s/it]
 23%|██▎       | 51620/225000 [108:14:18<367:16:28,  7.63s/it]
 23%|██▎       | 51621/225000 [108:14:27<388:07:01,  8.06s/it]
 23%|██▎       | 51622/225000 [108:14:31<323:23:29,  6.71s/it]
 23%|██▎       | 51623/225000 [108:14:36<307:06:04,  6.38s/it]
 23%|██▎       | 51624/225000 [108:14:42<297:51:29,  6.18s/it]
 23%|██▎       | 51625/225000 [108:14:48<293:22:42,  6.09s/it]
 23%|██▎       | 51626/225000 [108:15:00<377:54:41,  7.85s/it]
 23%|██▎       | 51627/225000 [108:15:07<363:06:30,  7.54s/it]
 23%|██▎       | 51628/225000 [108:15:11<319:24:07,  6.63s/it]
 23%|██▎       | 51629/225000 [108:15:16<288:31:04,  5.99s/it]
 23%|██▎       | 51630/225000 [108:15:27<367:08:56,  7.62s/it]
 23%|██▎       | 51631/225000 [108:15:33<344:27:34,  7.15s/it]
 23%|██▎       | 51632/225000 [108:15:41<359:04:25,  7.46s/it]
 23%|██▎       | 51633/225000 [108:15:45<308:59:42,  6.42s/it]
 23%|██▎       | 51634/225000 [108:15:55<349:46:20,  7.26s/it]
 23%|██▎       | 51635/225000 [108:16:01<339:46:29,  7.06s/it]
 23%|██▎       | 51636/225000 [108:16:07<324:37:55,  6.74s/it]
 23%|██▎       | 51637/225000 [108:16:12<294:46:07,  6.12s/it]
 23%|██▎       | 51638/225000 [108:16:16<267:22:40,  5.55s/it]
 23%|██▎       | 51639/225000 [108:16:30<379:43:23,  7.89s/it]
 23%|██▎       | 51640/225000 [108:16:34<335:54:25,  6.98s/it]
 23%|██▎       | 51641/225000 [108:16:39<307:18:04,  6.38s/it]
 23%|██▎       | 51642/225000 [108:16:45<297:03:28,  6.17s/it]
 23%|██▎       | 51643/225000 [108:16:55<345:42:31,  7.18s/it]
 23%|██▎       | 51644/225000 [108:17:02<352:08:12,  7.31s/it]
 23%|██▎       | 51645/225000 [108:17:08<331:06:15,  6.88s/it]
 23%|██▎       | 51646/225000 [108:17:16<342:36:14,  7.11s/it]
 23%|██▎       | 51647/225000 [108:17:20<302:28:02,  6.28s/it]
 23%|██▎       | 51648/225000 [108:17:29<344:13:15,  7.15s/it]
 23%|██▎       | 51649/225000 [108:17:36<342:51:01,  7.12s/it]
 23%|██▎       | 51650/225000 [108:17:41<311:53:28,  6.48s/it]
                                                              
{'loss': '0.7101', 'grad_norm': '0.7854', 'learning_rate': '4.552e-05', 'epoch': '0.2296', 'num_input_tokens_seen': 181031160, 'train_runtime': '3.899e+05', 'train_tokens_per_second': '464.3'}

 23%|██▎       | 51650/225000 [108:17:41<311:53:28,  6.48s/it]
 23%|██▎       | 51651/225000 [108:17:46<283:12:56,  5.88s/it]
 23%|██▎       | 51652/225000 [108:17:51<276:53:42,  5.75s/it]
 23%|██▎       | 51653/225000 [108:18:00<318:14:27,  6.61s/it]
 23%|██▎       | 51654/225000 [108:18:13<415:23:29,  8.63s/it]
 23%|██▎       | 51655/225000 [108:18:20<392:51:52,  8.16s/it]
 23%|██▎       | 51656/225000 [108:18:26<353:49:26,  7.35s/it]
 23%|██▎       | 51657/225000 [108:18:33<355:52:31,  7.39s/it]
 23%|██▎       | 51658/225000 [108:18:40<353:04:52,  7.33s/it]
 23%|██▎       | 51659/225000 [108:18:50<383:01:47,  7.95s/it]
 23%|██▎       | 51660/225000 [108:18:55<338:53:42,  7.04s/it]
 23%|██▎       | 51661/225000 [108:19:01<332:26:11,  6.90s/it]
 23%|██▎       | 51662/225000 [108:19:05<286:59:53,  5.96s/it]
 23%|██▎       | 51663/225000 [108:19:15<348:20:38,  7.23s/it]
 23%|██▎       | 51664/225000 [108:19:21<323:44:41,  6.72s/it]
 23%|██▎       | 51665/225000 [108:19:28<327:02:57,  6.79s/it]
 23%|██▎       | 51666/225000 [108:19:34<326:27:01,  6.78s/it]
 23%|██▎       | 51667/225000 [108:19:39<288:00:14,  5.98s/it]
 23%|██▎       | 51668/225000 [108:19:48<333:30:54,  6.93s/it]
 23%|██▎       | 51669/225000 [108:19:54<328:12:15,  6.82s/it]
 23%|██▎       | 51670/225000 [108:20:01<320:21:32,  6.65s/it]
 23%|██▎       | 51671/225000 [108:20:09<349:57:50,  7.27s/it]
 23%|██▎       | 51672/225000 [108:20:14<308:02:54,  6.40s/it]
 23%|██▎       | 51673/225000 [108:20:25<377:12:01,  7.83s/it]
 23%|██▎       | 51674/225000 [108:20:29<327:10:18,  6.80s/it]
 23%|██▎       | 51675/225000 [108:20:33<288:43:15,  6.00s/it]
 23%|██▎       | 51676/225000 [108:20:38<268:19:33,  5.57s/it]
 23%|██▎       | 51677/225000 [108:20:50<369:32:46,  7.68s/it]
 23%|██▎       | 51678/225000 [108:20:55<330:01:08,  6.85s/it]
 23%|██▎       | 51679/225000 [108:21:03<347:39:05,  7.22s/it]
 23%|██▎       | 51680/225000 [108:21:08<311:54:21,  6.48s/it]
 23%|██▎       | 51681/225000 [108:21:14<303:27:59,  6.30s/it]
 23%|██▎       | 51682/225000 [108:21:20<291:47:09,  6.06s/it]
 23%|██▎       | 51683/225000 [108:21:25<286:13:04,  5.95s/it]
 23%|██▎       | 51684/225000 [108:21:31<283:53:33,  5.90s/it]
 23%|██▎       | 51685/225000 [108:21:42<351:58:13,  7.31s/it]
 23%|██▎       | 51686/225000 [108:21:53<409:31:43,  8.51s/it]
 23%|██▎       | 51687/225000 [108:21:57<340:33:06,  7.07s/it]
 23%|██▎       | 51688/225000 [108:22:07<390:46:55,  8.12s/it]
 23%|██▎       | 51689/225000 [108:22:11<330:44:20,  6.87s/it]
 23%|██▎       | 51690/225000 [108:22:17<309:39:50,  6.43s/it]
 23%|██▎       | 51691/225000 [108:22:22<295:49:41,  6.14s/it]
 23%|██▎       | 51692/225000 [108:22:28<287:44:49,  5.98s/it]
 23%|██▎       | 51693/225000 [108:22:36<315:06:51,  6.55s/it]
 23%|██▎       | 51694/225000 [108:22:48<395:43:08,  8.22s/it]
 23%|██▎       | 51695/225000 [108:22:51<328:36:43,  6.83s/it]
 23%|██▎       | 51696/225000 [108:23:00<357:36:00,  7.43s/it]
 23%|██▎       | 51697/225000 [108:23:08<362:43:56,  7.53s/it]
 23%|██▎       | 51698/225000 [108:23:14<335:56:16,  6.98s/it]
 23%|██▎       | 51699/225000 [108:23:18<302:22:16,  6.28s/it]
 23%|██▎       | 51700/225000 [108:23:27<338:00:16,  7.02s/it]
                                                              
{'loss': '0.7103', 'grad_norm': '0.7889', 'learning_rate': '4.551e-05', 'epoch': '0.2298', 'num_input_tokens_seen': 181183672, 'train_runtime': '3.902e+05', 'train_tokens_per_second': '464.3'}

 23%|██▎       | 51700/225000 [108:23:27<338:00:16,  7.02s/it]
 23%|██▎       | 51701/225000 [108:23:38<395:52:09,  8.22s/it]
 23%|██▎       | 51702/225000 [108:23:42<331:49:57,  6.89s/it]
 23%|██▎       | 51703/225000 [108:23:47<310:35:59,  6.45s/it]
 23%|██▎       | 51704/225000 [108:23:52<291:03:56,  6.05s/it]
 23%|██▎       | 51705/225000 [108:23:57<278:21:24,  5.78s/it]
 23%|██▎       | 51706/225000 [108:24:05<301:24:23,  6.26s/it]
 23%|██▎       | 51707/225000 [108:24:19<412:08:40,  8.56s/it]
 23%|██▎       | 51708/225000 [108:24:23<354:00:43,  7.35s/it]
 23%|██▎       | 51709/225000 [108:24:30<344:07:12,  7.15s/it]
 23%|██▎       | 51710/225000 [108:24:34<304:58:28,  6.34s/it]
 23%|██▎       | 51711/225000 [108:24:43<343:28:43,  7.14s/it]
 23%|██▎       | 51712/225000 [108:24:52<365:46:32,  7.60s/it]
 23%|██▎       | 51713/225000 [108:24:57<323:56:06,  6.73s/it]
 23%|██▎       | 51714/225000 [108:25:02<298:16:14,  6.20s/it]
 23%|██▎       | 51715/225000 [108:25:12<363:28:39,  7.55s/it]
 23%|██▎       | 51716/225000 [108:25:21<384:42:04,  7.99s/it]
 23%|██▎       | 51717/225000 [108:25:26<337:05:48,  7.00s/it]
 23%|██▎       | 51718/225000 [108:25:32<323:22:32,  6.72s/it]
 23%|██▎       | 51719/225000 [108:25:44<389:23:10,  8.09s/it]
 23%|██▎       | 51720/225000 [108:25:51<382:52:40,  7.95s/it]
 23%|██▎       | 51721/225000 [108:25:57<350:49:56,  7.29s/it]
 23%|██▎       | 51722/225000 [108:26:03<334:27:08,  6.95s/it]
 23%|██▎       | 51723/225000 [108:26:07<293:20:17,  6.09s/it]
 23%|██▎       | 51724/225000 [108:26:19<381:40:59,  7.93s/it]
 23%|██▎       | 51725/225000 [108:26:26<363:15:48,  7.55s/it]
 23%|██▎       | 51726/225000 [108:26:31<330:36:40,  6.87s/it]
 23%|██▎       | 51727/225000 [108:26:35<290:00:13,  6.03s/it]
 23%|██▎       | 51728/225000 [108:26:44<322:38:21,  6.70s/it]
 23%|██▎       | 51729/225000 [108:26:53<367:20:52,  7.63s/it]
 23%|██▎       | 51730/225000 [108:26:59<335:02:57,  6.96s/it]
 23%|██▎       | 51731/225000 [108:27:03<298:14:26,  6.20s/it]
 23%|██▎       | 51732/225000 [108:27:08<281:09:53,  5.84s/it]
 23%|██▎       | 51733/225000 [108:27:15<292:19:17,  6.07s/it]
 23%|██▎       | 51734/225000 [108:27:27<377:27:59,  7.84s/it]
 23%|██▎       | 51735/225000 [108:27:31<326:04:06,  6.77s/it]
 23%|██▎       | 51736/225000 [108:27:38<333:31:20,  6.93s/it]
 23%|██▎       | 51737/225000 [108:27:43<297:14:31,  6.18s/it]
 23%|██▎       | 51738/225000 [108:27:48<283:31:29,  5.89s/it]
 23%|██▎       | 51739/225000 [108:27:53<270:00:50,  5.61s/it]
 23%|██▎       | 51740/225000 [108:28:06<375:33:18,  7.80s/it]
 23%|██▎       | 51741/225000 [108:28:11<336:50:50,  7.00s/it]
 23%|██▎       | 51742/225000 [108:28:17<316:52:22,  6.58s/it]
 23%|██▎       | 51743/225000 [108:28:23<307:10:12,  6.38s/it]
 23%|██▎       | 51744/225000 [108:28:28<297:07:56,  6.17s/it]
 23%|██▎       | 51745/225000 [108:28:34<290:16:33,  6.03s/it]
 23%|██▎       | 51746/225000 [108:28:49<417:23:12,  8.67s/it]
 23%|██▎       | 51747/225000 [108:28:56<389:21:20,  8.09s/it]
 23%|██▎       | 51748/225000 [108:29:01<348:59:13,  7.25s/it]
 23%|██▎       | 51749/225000 [108:29:06<318:35:41,  6.62s/it]
 23%|██▎       | 51750/225000 [108:29:12<302:37:44,  6.29s/it]
                                                              
{'loss': '0.7126', 'grad_norm': '0.7801', 'learning_rate': '4.55e-05', 'epoch': '0.23', 'num_input_tokens_seen': 181335248, 'train_runtime': '3.906e+05', 'train_tokens_per_second': '464.3'}

 23%|██▎       | 51750/225000 [108:29:12<302:37:44,  6.29s/it]
 23%|██▎       | 51751/225000 [108:29:17<295:02:34,  6.13s/it]
 23%|██▎       | 51752/225000 [108:29:25<322:59:49,  6.71s/it]
 23%|██▎       | 51753/225000 [108:29:38<402:57:42,  8.37s/it]
 23%|██▎       | 51754/225000 [108:29:42<341:30:11,  7.10s/it]
 23%|██▎       | 51755/225000 [108:29:49<348:34:04,  7.24s/it]
 23%|██▎       | 51756/225000 [108:29:54<312:32:35,  6.49s/it]
 23%|██▎       | 51757/225000 [108:30:02<326:44:48,  6.79s/it]
 23%|██▎       | 51758/225000 [108:30:10<357:16:47,  7.42s/it]
 23%|██▎       | 51759/225000 [108:30:16<323:48:23,  6.73s/it]
 23%|██▎       | 51760/225000 [108:30:20<295:51:03,  6.15s/it]
 23%|██▎       | 51761/225000 [108:30:31<362:23:02,  7.53s/it]
 23%|██▎       | 51762/225000 [108:30:36<319:52:27,  6.65s/it]
 23%|██▎       | 51763/225000 [108:30:40<290:52:52,  6.04s/it]
 23%|██▎       | 51764/225000 [108:30:48<312:31:42,  6.49s/it]
 23%|██▎       | 51765/225000 [108:30:58<361:03:40,  7.50s/it]
 23%|██▎       | 51766/225000 [108:31:05<352:59:25,  7.34s/it]
 23%|██▎       | 51767/225000 [108:31:09<312:34:23,  6.50s/it]
 23%|██▎       | 51768/225000 [108:31:15<295:59:04,  6.15s/it]
 23%|██▎       | 51769/225000 [108:31:21<296:59:32,  6.17s/it]
 23%|██▎       | 51770/225000 [108:31:33<390:35:28,  8.12s/it]
 23%|██▎       | 51771/225000 [108:31:39<352:06:19,  7.32s/it]
 23%|██▎       | 51772/225000 [108:31:44<327:22:20,  6.80s/it]
 23%|██▎       | 51773/225000 [108:31:52<335:58:56,  6.98s/it]
 23%|██▎       | 51774/225000 [108:31:56<301:36:10,  6.27s/it]
 23%|██▎       | 51775/225000 [108:32:02<294:28:07,  6.12s/it]
 23%|██▎       | 51776/225000 [108:32:08<288:52:37,  6.00s/it]
 23%|██▎       | 51777/225000 [108:32:13<274:46:20,  5.71s/it]
 23%|██▎       | 51778/225000 [108:32:20<290:29:15,  6.04s/it]
 23%|██▎       | 51779/225000 [108:32:34<408:36:32,  8.49s/it]
 23%|██▎       | 51780/225000 [108:32:40<374:07:53,  7.78s/it]
 23%|██▎       | 51781/225000 [108:32:45<333:43:15,  6.94s/it]
 23%|██▎       | 51782/225000 [108:32:54<359:17:08,  7.47s/it]
 23%|██▎       | 51783/225000 [108:33:03<383:07:52,  7.96s/it]
 23%|██▎       | 51784/225000 [108:33:09<355:51:35,  7.40s/it]
 23%|██▎       | 51785/225000 [108:33:14<318:46:56,  6.63s/it]
 23%|██▎       | 51786/225000 [108:33:20<306:27:47,  6.37s/it]
 23%|██▎       | 51787/225000 [108:33:29<349:27:44,  7.26s/it]
 23%|██▎       | 51788/225000 [108:33:32<294:58:17,  6.13s/it]
 23%|██▎       | 51789/225000 [108:33:41<335:49:55,  6.98s/it]
 23%|██▎       | 51790/225000 [108:33:46<304:18:40,  6.32s/it]
 23%|██▎       | 51791/225000 [108:33:51<288:47:59,  6.00s/it]
 23%|██▎       | 51792/225000 [108:34:02<359:55:05,  7.48s/it]
 23%|██▎       | 51793/225000 [108:34:07<321:35:31,  6.68s/it]
 23%|██▎       | 51794/225000 [108:34:17<367:16:35,  7.63s/it]
 23%|██▎       | 51795/225000 [108:34:21<315:06:37,  6.55s/it]
 23%|██▎       | 51796/225000 [108:34:29<330:01:00,  6.86s/it]
 23%|██▎       | 51797/225000 [108:34:38<367:09:46,  7.63s/it]
 23%|██▎       | 51798/225000 [108:34:46<368:38:14,  7.66s/it]
 23%|██▎       | 51799/225000 [108:34:52<345:33:19,  7.18s/it]
 23%|██▎       | 51800/225000 [108:34:56<306:48:15,  6.38s/it]
                                                              
{'loss': '0.7198', 'grad_norm': '0.7967', 'learning_rate': '4.549e-05', 'epoch': '0.2302', 'num_input_tokens_seen': 181487936, 'train_runtime': '3.909e+05', 'train_tokens_per_second': '464.3'}

 23%|██▎       | 51800/225000 [108:34:56<306:48:15,  6.38s/it]
 23%|██▎       | 51801/225000 [108:35:01<276:33:56,  5.75s/it]
 23%|██▎       | 51802/225000 [108:35:14<393:08:35,  8.17s/it]
 23%|██▎       | 51803/225000 [108:35:18<325:23:04,  6.76s/it]
 23%|██▎       | 51804/225000 [108:35:26<349:56:47,  7.27s/it]
 23%|██▎       | 51805/225000 [108:35:31<308:34:04,  6.41s/it]
 23%|██▎       | 51806/225000 [108:35:41<359:38:35,  7.48s/it]
 23%|██▎       | 51807/225000 [108:35:44<304:00:35,  6.32s/it]
 23%|██▎       | 51808/225000 [108:35:55<359:37:14,  7.48s/it]
 23%|██▎       | 51809/225000 [108:36:00<330:34:09,  6.87s/it]
 23%|██▎       | 51810/225000 [108:36:08<343:59:28,  7.15s/it]
 23%|██▎       | 51811/225000 [108:36:14<323:25:34,  6.72s/it]
 23%|██▎       | 51812/225000 [108:36:18<291:22:58,  6.06s/it]
 23%|██▎       | 51813/225000 [108:36:23<273:29:25,  5.68s/it]
 23%|██▎       | 51814/225000 [108:36:33<335:10:46,  6.97s/it]
 23%|██▎       | 51815/225000 [108:36:40<341:12:33,  7.09s/it]
 23%|██▎       | 51816/225000 [108:36:46<317:18:22,  6.60s/it]
 23%|██▎       | 51817/225000 [108:36:51<303:32:07,  6.31s/it]
 23%|██▎       | 51818/225000 [108:36:57<289:43:55,  6.02s/it]
 23%|██▎       | 51819/225000 [108:37:02<282:17:20,  5.87s/it]
 23%|██▎       | 51820/225000 [108:37:07<273:28:51,  5.69s/it]
 23%|██▎       | 51821/225000 [108:37:18<346:43:40,  7.21s/it]
 23%|██▎       | 51822/225000 [108:37:28<387:20:46,  8.05s/it]
 23%|██▎       | 51823/225000 [108:37:32<323:32:28,  6.73s/it]
 23%|██▎       | 51824/225000 [108:37:42<371:29:32,  7.72s/it]
 23%|██▎       | 51825/225000 [108:37:46<318:15:47,  6.62s/it]
 23%|██▎       | 51826/225000 [108:37:51<293:07:10,  6.09s/it]
 23%|██▎       | 51827/225000 [108:38:03<378:33:11,  7.87s/it]
 23%|██▎       | 51828/225000 [108:38:07<331:27:42,  6.89s/it]
 23%|██▎       | 51829/225000 [108:38:13<312:41:52,  6.50s/it]
 23%|██▎       | 51830/225000 [108:38:18<292:56:06,  6.09s/it]
 23%|██▎       | 51831/225000 [108:38:28<349:51:29,  7.27s/it]
 23%|██▎       | 51832/225000 [108:38:37<370:11:17,  7.70s/it]
 23%|██▎       | 51833/225000 [108:38:41<314:12:06,  6.53s/it]
 23%|██▎       | 51834/225000 [108:38:50<359:01:53,  7.46s/it]
 23%|██▎       | 51835/225000 [108:38:54<311:12:08,  6.47s/it]
 23%|██▎       | 51836/225000 [108:39:05<366:48:51,  7.63s/it]
 23%|██▎       | 51837/225000 [108:39:08<309:40:34,  6.44s/it]
 23%|██▎       | 51838/225000 [108:39:13<276:59:38,  5.76s/it]
 23%|██▎       | 51839/225000 [108:39:25<375:50:51,  7.81s/it]
 23%|██▎       | 51840/225000 [108:39:29<316:41:47,  6.58s/it]
 23%|██▎       | 51841/225000 [108:39:36<324:03:03,  6.74s/it]
 23%|██▎       | 51842/225000 [108:39:43<322:36:07,  6.71s/it]
 23%|██▎       | 51843/225000 [108:39:47<293:05:28,  6.09s/it]
 23%|██▎       | 51844/225000 [108:39:54<307:01:21,  6.38s/it]
 23%|██▎       | 51845/225000 [108:40:04<349:45:16,  7.27s/it]
 23%|██▎       | 51846/225000 [108:40:09<320:40:32,  6.67s/it]
 23%|██▎       | 51847/225000 [108:40:18<348:19:42,  7.24s/it]
 23%|██▎       | 51848/225000 [108:40:26<360:29:48,  7.50s/it]
 23%|██▎       | 51849/225000 [108:40:31<321:57:59,  6.69s/it]
 23%|██▎       | 51850/225000 [108:40:35<295:24:42,  6.14s/it]
                                                              
{'loss': '0.7147', 'grad_norm': '0.6491', 'learning_rate': '4.547e-05', 'epoch': '0.2304', 'num_input_tokens_seen': 181638024, 'train_runtime': '3.912e+05', 'train_tokens_per_second': '464.3'}

 23%|██▎       | 51850/225000 [108:40:35<295:24:42,  6.14s/it]
 23%|██▎       | 51851/225000 [108:40:40<275:54:16,  5.74s/it]
 23%|██▎       | 51852/225000 [108:40:47<298:59:32,  6.22s/it]
 23%|██▎       | 51853/225000 [108:40:58<367:20:41,  7.64s/it]
 23%|██▎       | 51854/225000 [108:41:06<360:04:36,  7.49s/it]
 23%|██▎       | 51855/225000 [108:41:10<322:31:32,  6.71s/it]
 23%|██▎       | 51856/225000 [108:41:15<298:15:00,  6.20s/it]
 23%|██▎       | 51857/225000 [108:41:20<280:39:50,  5.84s/it]
 23%|██▎       | 51858/225000 [108:41:34<385:17:02,  8.01s/it]
 23%|██▎       | 51859/225000 [108:41:39<342:20:56,  7.12s/it]
 23%|██▎       | 51860/225000 [108:41:43<309:51:50,  6.44s/it]
 23%|██▎       | 51861/225000 [108:41:49<290:29:16,  6.04s/it]
 23%|██▎       | 51862/225000 [108:41:53<270:18:21,  5.62s/it]
 23%|██▎       | 51863/225000 [108:42:04<345:44:15,  7.19s/it]
 23%|██▎       | 51864/225000 [108:42:13<376:54:20,  7.84s/it]
 23%|██▎       | 51865/225000 [108:42:22<381:02:16,  7.92s/it]
 23%|██▎       | 51866/225000 [108:42:26<330:50:37,  6.88s/it]
 23%|██▎       | 51867/225000 [108:42:33<333:44:46,  6.94s/it]
 23%|██▎       | 51868/225000 [108:42:42<363:47:24,  7.56s/it]
 23%|██▎       | 51869/225000 [108:42:47<318:55:51,  6.63s/it]
 23%|██▎       | 51870/225000 [108:42:52<301:40:38,  6.27s/it]
 23%|██▎       | 51871/225000 [108:43:02<359:34:49,  7.48s/it]
 23%|██▎       | 51872/225000 [108:43:06<307:05:26,  6.39s/it]
 23%|██▎       | 51873/225000 [108:43:14<332:47:44,  6.92s/it]
 23%|██▎       | 51874/225000 [108:43:22<340:56:40,  7.09s/it]
 23%|██▎       | 51875/225000 [108:43:26<298:23:54,  6.20s/it]
 23%|██▎       | 51876/225000 [108:43:34<332:58:53,  6.92s/it]
 23%|██▎       | 51877/225000 [108:43:43<353:46:51,  7.36s/it]
 23%|██▎       | 51878/225000 [108:43:48<319:53:11,  6.65s/it]
 23%|██▎       | 51879/225000 [108:43:53<299:25:22,  6.23s/it]
 23%|██▎       | 51880/225000 [108:43:58<283:01:21,  5.89s/it]
 23%|██▎       | 51881/225000 [108:44:03<273:24:47,  5.69s/it]
 23%|██▎       | 51882/225000 [108:44:11<298:01:26,  6.20s/it]
 23%|██▎       | 51883/225000 [108:44:21<361:32:02,  7.52s/it]
 23%|██▎       | 51884/225000 [108:44:26<315:44:12,  6.57s/it]
 23%|██▎       | 51885/225000 [108:44:37<381:59:16,  7.94s/it]
 23%|██▎       | 51886/225000 [108:44:41<329:26:36,  6.85s/it]
 23%|██▎       | 51887/225000 [108:44:52<383:04:22,  7.97s/it]
 23%|██▎       | 51888/225000 [108:44:59<375:24:31,  7.81s/it]
 23%|██▎       | 51889/225000 [108:45:07<382:01:25,  7.94s/it]
 23%|██▎       | 51890/225000 [108:45:14<360:04:29,  7.49s/it]
 23%|██▎       | 51891/225000 [108:45:22<365:55:42,  7.61s/it]
 23%|██▎       | 51892/225000 [108:45:27<328:34:13,  6.83s/it]
 23%|██▎       | 51893/225000 [108:45:34<332:19:56,  6.91s/it]
 23%|██▎       | 51894/225000 [108:45:39<302:13:07,  6.29s/it]
 23%|██▎       | 51895/225000 [108:45:44<281:18:08,  5.85s/it]
 23%|██▎       | 51896/225000 [108:45:51<305:17:49,  6.35s/it]
 23%|██▎       | 51897/225000 [108:46:01<354:17:19,  7.37s/it]
 23%|██▎       | 51898/225000 [108:46:07<336:07:13,  6.99s/it]
 23%|██▎       | 51899/225000 [108:46:13<328:55:58,  6.84s/it]
 23%|██▎       | 51900/225000 [108:46:23<362:54:17,  7.55s/it]
                                                              
{'loss': '0.7074', 'grad_norm': '0.8432', 'learning_rate': '4.546e-05', 'epoch': '0.2307', 'num_input_tokens_seen': 181790536, 'train_runtime': '3.916e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 51900/225000 [108:46:23<362:54:17,  7.55s/it]
 23%|██▎       | 51901/225000 [108:46:29<346:00:57,  7.20s/it]
 23%|██▎       | 51902/225000 [108:46:34<309:18:39,  6.43s/it]
 23%|██▎       | 51903/225000 [108:46:42<332:05:51,  6.91s/it]
 23%|██▎       | 51904/225000 [108:46:47<303:33:14,  6.31s/it]
 23%|██▎       | 51905/225000 [108:46:51<277:14:28,  5.77s/it]
 23%|██▎       | 51906/225000 [108:47:00<328:29:48,  6.83s/it]
 23%|██▎       | 51907/225000 [108:47:08<343:11:25,  7.14s/it]
 23%|██▎       | 51908/225000 [108:47:13<309:24:08,  6.44s/it]
 23%|██▎       | 51909/225000 [108:47:18<293:44:54,  6.11s/it]
 23%|██▎       | 51910/225000 [108:47:28<350:58:13,  7.30s/it]
 23%|██▎       | 51911/225000 [108:47:36<357:27:53,  7.43s/it]
 23%|██▎       | 51912/225000 [108:47:40<308:35:29,  6.42s/it]
 23%|██▎       | 51913/225000 [108:47:48<330:56:02,  6.88s/it]
 23%|██▎       | 51914/225000 [108:47:57<363:48:26,  7.57s/it]
 23%|██▎       | 51915/225000 [108:48:05<369:06:46,  7.68s/it]
 23%|██▎       | 51916/225000 [108:48:09<315:44:04,  6.57s/it]
 23%|██▎       | 51917/225000 [108:48:18<341:08:24,  7.10s/it]
 23%|██▎       | 51918/225000 [108:48:23<311:50:50,  6.49s/it]
 23%|██▎       | 51919/225000 [108:48:27<276:47:01,  5.76s/it]
 23%|██▎       | 51920/225000 [108:48:37<344:25:55,  7.16s/it]
 23%|██▎       | 51921/225000 [108:48:43<324:24:11,  6.75s/it]
 23%|██▎       | 51922/225000 [108:48:50<329:43:39,  6.86s/it]
 23%|██▎       | 51923/225000 [108:48:59<352:44:25,  7.34s/it]
 23%|██▎       | 51924/225000 [108:49:03<314:17:55,  6.54s/it]
 23%|██▎       | 51925/225000 [108:49:08<285:12:39,  5.93s/it]
 23%|██▎       | 51926/225000 [108:49:15<301:49:16,  6.28s/it]
 23%|██▎       | 51927/225000 [108:49:25<363:14:51,  7.56s/it]
 23%|██▎       | 51928/225000 [108:49:31<337:12:43,  7.01s/it]
 23%|██▎       | 51929/225000 [108:49:36<301:49:59,  6.28s/it]
 23%|██▎       | 51930/225000 [108:49:41<283:14:02,  5.89s/it]
 23%|██▎       | 51931/225000 [108:49:45<264:26:48,  5.50s/it]
 23%|██▎       | 51932/225000 [108:49:58<365:16:50,  7.60s/it]
 23%|██▎       | 51933/225000 [108:50:04<352:45:53,  7.34s/it]
 23%|██▎       | 51934/225000 [108:50:12<361:43:25,  7.52s/it]
 23%|██▎       | 51935/225000 [108:50:19<351:26:28,  7.31s/it]
 23%|██▎       | 51936/225000 [108:50:25<323:59:27,  6.74s/it]
 23%|██▎       | 51937/225000 [108:50:32<335:45:16,  6.98s/it]
 23%|██▎       | 51938/225000 [108:50:37<297:56:28,  6.20s/it]
 23%|██▎       | 51939/225000 [108:50:42<283:28:43,  5.90s/it]
 23%|██▎       | 51940/225000 [108:50:47<278:37:42,  5.80s/it]
 23%|██▎       | 51941/225000 [108:50:52<268:03:30,  5.58s/it]
 23%|██▎       | 51942/225000 [108:51:05<371:19:35,  7.72s/it]
 23%|██▎       | 51943/225000 [108:51:12<353:33:15,  7.35s/it]
 23%|██▎       | 51944/225000 [108:51:15<298:46:31,  6.22s/it]
 23%|██▎       | 51945/225000 [108:51:21<294:07:20,  6.12s/it]
 23%|██▎       | 51946/225000 [108:51:31<352:56:09,  7.34s/it]
 23%|██▎       | 51947/225000 [108:51:38<349:16:37,  7.27s/it]
 23%|██▎       | 51948/225000 [108:51:44<321:39:07,  6.69s/it]
 23%|██▎       | 51949/225000 [108:51:50<314:36:25,  6.54s/it]
 23%|██▎       | 51950/225000 [108:51:59<347:27:29,  7.23s/it]
                                                              
{'loss': '0.6953', 'grad_norm': '0.7202', 'learning_rate': '4.545e-05', 'epoch': '0.2309', 'num_input_tokens_seen': 181941600, 'train_runtime': '3.919e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 51950/225000 [108:51:59<347:27:29,  7.23s/it]
 23%|██▎       | 51951/225000 [108:52:04<320:41:02,  6.67s/it]
 23%|██▎       | 51952/225000 [108:52:12<336:09:27,  6.99s/it]
 23%|██▎       | 51953/225000 [108:52:19<338:33:20,  7.04s/it]
 23%|██▎       | 51954/225000 [108:52:26<334:20:19,  6.96s/it]
 23%|██▎       | 51955/225000 [108:52:31<305:24:10,  6.35s/it]
 23%|██▎       | 51956/225000 [108:52:36<289:03:59,  6.01s/it]
 23%|██▎       | 51957/225000 [108:52:46<348:25:16,  7.25s/it]
 23%|██▎       | 51958/225000 [108:52:52<324:12:54,  6.75s/it]
 23%|██▎       | 51959/225000 [108:52:57<303:05:36,  6.31s/it]
 23%|██▎       | 51960/225000 [108:53:02<289:57:48,  6.03s/it]
 23%|██▎       | 51961/225000 [108:53:08<287:49:48,  5.99s/it]
 23%|██▎       | 51962/225000 [108:53:13<275:56:14,  5.74s/it]
 23%|██▎       | 51963/225000 [108:53:27<384:55:40,  8.01s/it]
 23%|██▎       | 51964/225000 [108:53:32<347:31:13,  7.23s/it]
 23%|██▎       | 51965/225000 [108:53:41<375:42:09,  7.82s/it]
 23%|██▎       | 51966/225000 [108:53:46<336:33:23,  7.00s/it]
 23%|██▎       | 51967/225000 [108:53:51<307:12:28,  6.39s/it]
 23%|██▎       | 51968/225000 [108:53:56<282:43:52,  5.88s/it]
 23%|██▎       | 51969/225000 [108:54:06<342:23:00,  7.12s/it]
 23%|██▎       | 51970/225000 [108:54:13<347:25:33,  7.23s/it]
 23%|██▎       | 51971/225000 [108:54:20<332:05:24,  6.91s/it]
 23%|██▎       | 51972/225000 [108:54:29<362:10:38,  7.54s/it]
 23%|██▎       | 51973/225000 [108:54:34<327:54:37,  6.82s/it]
 23%|██▎       | 51974/225000 [108:54:39<299:06:22,  6.22s/it]
 23%|██▎       | 51975/225000 [108:54:43<278:59:17,  5.80s/it]
 23%|██▎       | 51976/225000 [108:54:48<260:05:41,  5.41s/it]
 23%|██▎       | 51977/225000 [108:55:00<350:02:22,  7.28s/it]
 23%|██▎       | 51978/225000 [108:55:04<307:59:17,  6.41s/it]
 23%|██▎       | 51979/225000 [108:55:15<368:57:59,  7.68s/it]
 23%|██▎       | 51980/225000 [108:55:18<312:47:27,  6.51s/it]
 23%|██▎       | 51981/225000 [108:55:27<337:25:17,  7.02s/it]
 23%|██▎       | 51982/225000 [108:55:35<360:47:12,  7.51s/it]
 23%|██▎       | 51983/225000 [108:55:41<331:27:40,  6.90s/it]
 23%|██▎       | 51984/225000 [108:55:46<308:10:28,  6.41s/it]
 23%|██▎       | 51985/225000 [108:55:52<295:38:11,  6.15s/it]
 23%|██▎       | 51986/225000 [108:55:57<283:13:28,  5.89s/it]
 23%|██▎       | 51987/225000 [108:56:07<342:04:24,  7.12s/it]
 23%|██▎       | 51988/225000 [108:56:16<374:29:03,  7.79s/it]
 23%|██▎       | 51989/225000 [108:56:20<321:01:04,  6.68s/it]
 23%|██▎       | 51990/225000 [108:56:26<310:28:17,  6.46s/it]
 23%|██▎       | 51991/225000 [108:56:37<372:49:24,  7.76s/it]
 23%|██▎       | 51992/225000 [108:56:41<318:48:45,  6.63s/it]
 23%|██▎       | 51993/225000 [108:56:48<316:48:26,  6.59s/it]
 23%|██▎       | 51994/225000 [108:56:55<329:18:33,  6.85s/it]
 23%|██▎       | 51995/225000 [108:57:04<363:31:03,  7.56s/it]
 23%|██▎       | 51996/225000 [108:57:10<343:23:17,  7.15s/it]
 23%|██▎       | 51997/225000 [108:57:19<369:01:14,  7.68s/it]
 23%|██▎       | 51998/225000 [108:57:25<338:19:22,  7.04s/it]
 23%|██▎       | 51999/225000 [108:57:30<311:52:37,  6.49s/it]
 23%|██▎       | 52000/225000 [108:57:36<303:12:50,  6.31s/it]
                                                              
{'loss': '0.693', 'grad_norm': '0.8057', 'learning_rate': '4.544e-05', 'epoch': '0.2311', 'num_input_tokens_seen': 182092080, 'train_runtime': '3.923e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 52000/225000 [108:57:36<303:12:50,  6.31s/it]
 23%|██▎       | 52001/225000 [108:57:41<289:31:12,  6.02s/it]
 23%|██▎       | 52002/225000 [108:57:47<282:35:54,  5.88s/it]
 23%|██▎       | 52003/225000 [108:57:54<301:09:49,  6.27s/it]
 23%|██▎       | 52004/225000 [108:58:06<386:42:26,  8.05s/it]
 23%|██▎       | 52005/225000 [108:58:10<328:48:55,  6.84s/it]
 23%|██▎       | 52006/225000 [108:58:19<363:34:26,  7.57s/it]
 23%|██▎       | 52007/225000 [108:58:26<343:19:35,  7.14s/it]
 23%|██▎       | 52008/225000 [108:58:32<330:15:01,  6.87s/it]
 23%|██▎       | 52009/225000 [108:58:37<300:14:36,  6.25s/it]
 23%|██▎       | 52010/225000 [108:58:46<345:25:29,  7.19s/it]
 23%|██▎       | 52011/225000 [108:58:52<334:15:11,  6.96s/it]
 23%|██▎       | 52012/225000 [108:58:57<300:37:02,  6.26s/it]
 23%|██▎       | 52013/225000 [108:59:02<283:51:25,  5.91s/it]
 23%|██▎       | 52014/225000 [108:59:08<280:40:18,  5.84s/it]
 23%|██▎       | 52015/225000 [108:59:21<385:29:42,  8.02s/it]
 23%|██▎       | 52016/225000 [108:59:25<325:05:41,  6.77s/it]
 23%|██▎       | 52017/225000 [108:59:35<376:03:33,  7.83s/it]
 23%|██▎       | 52018/225000 [108:59:40<328:09:07,  6.83s/it]
 23%|██▎       | 52019/225000 [108:59:46<316:00:43,  6.58s/it]
 23%|██▎       | 52020/225000 [108:59:51<294:46:52,  6.13s/it]
 23%|██▎       | 52021/225000 [108:59:56<276:30:07,  5.75s/it]
 23%|██▎       | 52022/225000 [109:00:05<328:16:52,  6.83s/it]
 23%|██▎       | 52023/225000 [109:00:12<332:48:44,  6.93s/it]
 23%|██▎       | 52024/225000 [109:00:18<317:29:36,  6.61s/it]
 23%|██▎       | 52025/225000 [109:00:26<344:03:07,  7.16s/it]
 23%|██▎       | 52026/225000 [109:00:32<316:11:45,  6.58s/it]
 23%|██▎       | 52027/225000 [109:00:37<296:44:22,  6.18s/it]
 23%|██▎       | 52028/225000 [109:00:42<280:40:10,  5.84s/it]
 23%|██▎       | 52029/225000 [109:00:52<339:06:28,  7.06s/it]
 23%|██▎       | 52030/225000 [109:01:00<358:17:29,  7.46s/it]
 23%|██▎       | 52031/225000 [109:01:06<331:25:45,  6.90s/it]
 23%|██▎       | 52032/225000 [109:01:11<308:52:38,  6.43s/it]
 23%|██▎       | 52033/225000 [109:01:16<292:35:03,  6.09s/it]
 23%|██▎       | 52034/225000 [109:01:22<280:25:56,  5.84s/it]
 23%|██▎       | 52035/225000 [109:01:27<280:07:26,  5.83s/it]
 23%|██▎       | 52036/225000 [109:01:33<274:48:13,  5.72s/it]
 23%|██▎       | 52037/225000 [109:01:46<374:24:16,  7.79s/it]
 23%|██▎       | 52038/225000 [109:01:56<413:51:56,  8.61s/it]
 23%|██▎       | 52039/225000 [109:02:00<343:56:02,  7.16s/it]
 23%|██▎       | 52040/225000 [109:02:10<381:51:10,  7.95s/it]
 23%|██▎       | 52041/225000 [109:02:14<323:05:56,  6.73s/it]
 23%|██▎       | 52042/225000 [109:02:23<360:46:24,  7.51s/it]
 23%|██▎       | 52043/225000 [109:02:28<323:40:29,  6.74s/it]
 23%|██▎       | 52044/225000 [109:02:36<348:42:35,  7.26s/it]
 23%|██▎       | 52045/225000 [109:02:41<312:42:04,  6.51s/it]
 23%|██▎       | 52046/225000 [109:02:46<296:47:29,  6.18s/it]
 23%|██▎       | 52047/225000 [109:02:52<288:32:42,  6.01s/it]
 23%|██▎       | 52048/225000 [109:02:57<280:01:06,  5.83s/it]
 23%|██▎       | 52049/225000 [109:03:09<369:18:52,  7.69s/it]
 23%|██▎       | 52050/225000 [109:03:18<379:49:53,  7.91s/it]
                                                              
{'loss': '0.6947', 'grad_norm': '0.8422', 'learning_rate': '4.543e-05', 'epoch': '0.2313', 'num_input_tokens_seen': 182243192, 'train_runtime': '3.926e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 52050/225000 [109:03:18<379:49:53,  7.91s/it]
 23%|██▎       | 52051/225000 [109:03:23<333:19:26,  6.94s/it]
 23%|██▎       | 52052/225000 [109:03:28<305:29:17,  6.36s/it]
 23%|██▎       | 52053/225000 [109:03:33<290:56:48,  6.06s/it]
 23%|██▎       | 52054/225000 [109:03:38<276:27:21,  5.75s/it]
 23%|██▎       | 52055/225000 [109:03:45<293:56:31,  6.12s/it]
 23%|██▎       | 52056/225000 [109:03:59<411:20:14,  8.56s/it]
 23%|██▎       | 52057/225000 [109:04:06<381:22:20,  7.94s/it]
 23%|██▎       | 52058/225000 [109:04:11<340:50:03,  7.09s/it]
 23%|██▎       | 52059/225000 [109:04:16<309:51:22,  6.45s/it]
 23%|██▎       | 52060/225000 [109:04:21<291:38:34,  6.07s/it]
 23%|██▎       | 52061/225000 [109:04:26<276:08:59,  5.75s/it]
 23%|██▎       | 52062/225000 [109:04:39<381:08:25,  7.93s/it]
 23%|██▎       | 52063/225000 [109:04:45<347:43:01,  7.24s/it]
 23%|██▎       | 52064/225000 [109:04:49<312:17:15,  6.50s/it]
 23%|██▎       | 52065/225000 [109:04:54<287:43:37,  5.99s/it]
 23%|██▎       | 52066/225000 [109:05:03<333:45:36,  6.95s/it]
 23%|██▎       | 52067/225000 [109:05:14<387:41:26,  8.07s/it]
 23%|██▎       | 52068/225000 [109:05:19<344:03:19,  7.16s/it]
 23%|██▎       | 52069/225000 [109:05:25<332:24:06,  6.92s/it]
 23%|██▎       | 52070/225000 [109:05:34<356:51:33,  7.43s/it]
 23%|██▎       | 52071/225000 [109:05:41<350:48:23,  7.30s/it]
 23%|██▎       | 52072/225000 [109:05:48<344:13:36,  7.17s/it]
 23%|██▎       | 52073/225000 [109:05:52<296:43:56,  6.18s/it]
 23%|██▎       | 52074/225000 [109:05:56<267:32:35,  5.57s/it]
 23%|██▎       | 52075/225000 [109:06:08<365:16:50,  7.60s/it]
 23%|██▎       | 52076/225000 [109:06:12<309:01:21,  6.43s/it]
 23%|██▎       | 52077/225000 [109:06:17<286:03:27,  5.96s/it]
 23%|██▎       | 52078/225000 [109:06:22<271:54:41,  5.66s/it]
 23%|██▎       | 52079/225000 [109:06:33<350:27:50,  7.30s/it]
 23%|██▎       | 52080/225000 [109:06:38<312:51:01,  6.51s/it]
 23%|██▎       | 52081/225000 [109:06:47<354:21:35,  7.38s/it]
 23%|██▎       | 52082/225000 [109:06:52<315:52:58,  6.58s/it]
 23%|██▎       | 52083/225000 [109:06:57<303:19:29,  6.31s/it]
 23%|██▎       | 52084/225000 [109:07:03<293:51:08,  6.12s/it]
 23%|██▎       | 52085/225000 [109:07:08<282:54:29,  5.89s/it]
 23%|██▎       | 52086/225000 [109:07:14<282:03:57,  5.87s/it]
 23%|██▎       | 52087/225000 [109:07:20<278:51:39,  5.81s/it]
 23%|██▎       | 52088/225000 [109:07:30<339:32:58,  7.07s/it]
 23%|██▎       | 52089/225000 [109:07:40<389:19:22,  8.11s/it]
 23%|██▎       | 52090/225000 [109:07:44<328:10:03,  6.83s/it]
 23%|██▎       | 52091/225000 [109:07:49<297:41:21,  6.20s/it]
 23%|██▎       | 52092/225000 [109:07:55<288:22:00,  6.00s/it]
 23%|██▎       | 52093/225000 [109:08:05<355:33:45,  7.40s/it]
 23%|██▎       | 52094/225000 [109:08:13<361:52:49,  7.53s/it]
 23%|██▎       | 52095/225000 [109:08:17<315:47:34,  6.58s/it]
 23%|██▎       | 52096/225000 [109:08:22<286:34:18,  5.97s/it]
 23%|██▎       | 52097/225000 [109:08:32<343:01:01,  7.14s/it]
 23%|██▎       | 52098/225000 [109:08:41<374:53:00,  7.81s/it]
 23%|██▎       | 52099/225000 [109:08:46<335:34:09,  6.99s/it]
 23%|██▎       | 52100/225000 [109:08:51<309:05:33,  6.44s/it]
                                                              
{'loss': '0.7079', 'grad_norm': '0.8059', 'learning_rate': '4.542e-05', 'epoch': '0.2316', 'num_input_tokens_seen': 182393168, 'train_runtime': '3.929e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 52100/225000 [109:08:51<309:05:33,  6.44s/it]
 23%|██▎       | 52101/225000 [109:08:57<297:45:41,  6.20s/it]
 23%|██▎       | 52102/225000 [109:09:03<295:41:39,  6.16s/it]
 23%|██▎       | 52103/225000 [109:09:09<290:06:59,  6.04s/it]
 23%|██▎       | 52104/225000 [109:09:14<280:12:56,  5.83s/it]
 23%|██▎       | 52105/225000 [109:09:20<278:33:06,  5.80s/it]
 23%|██▎       | 52106/225000 [109:09:32<369:32:07,  7.69s/it]
 23%|██▎       | 52107/225000 [109:09:41<387:40:31,  8.07s/it]
 23%|██▎       | 52108/225000 [109:09:48<371:26:06,  7.73s/it]
 23%|██▎       | 52109/225000 [109:09:53<325:29:58,  6.78s/it]
 23%|██▎       | 52110/225000 [109:09:58<300:39:47,  6.26s/it]
 23%|██▎       | 52111/225000 [109:10:03<282:32:26,  5.88s/it]
 23%|██▎       | 52112/225000 [109:10:10<297:31:23,  6.20s/it]
 23%|██▎       | 52113/225000 [109:10:23<397:02:42,  8.27s/it]
 23%|██▎       | 52114/225000 [109:10:27<343:38:41,  7.16s/it]
 23%|██▎       | 52115/225000 [109:10:33<326:02:02,  6.79s/it]
 23%|██▎       | 52116/225000 [109:10:42<351:35:40,  7.32s/it]
 23%|██▎       | 52117/225000 [109:10:51<376:53:40,  7.85s/it]
 23%|██▎       | 52118/225000 [109:10:56<333:03:23,  6.94s/it]
 23%|██▎       | 52119/225000 [109:11:02<328:01:27,  6.83s/it]
 23%|██▎       | 52120/225000 [109:11:08<315:57:50,  6.58s/it]
 23%|██▎       | 52121/225000 [109:11:14<303:31:24,  6.32s/it]
 23%|██▎       | 52122/225000 [109:11:22<329:33:05,  6.86s/it]
 23%|██▎       | 52123/225000 [109:11:31<355:22:52,  7.40s/it]
 23%|██▎       | 52124/225000 [109:11:37<337:22:56,  7.03s/it]
 23%|██▎       | 52125/225000 [109:11:43<323:51:39,  6.74s/it]
 23%|██▎       | 52126/225000 [109:11:51<348:12:10,  7.25s/it]
 23%|██▎       | 52127/225000 [109:11:58<334:13:28,  6.96s/it]
 23%|██▎       | 52128/225000 [109:12:03<313:05:31,  6.52s/it]
 23%|██▎       | 52129/225000 [109:12:09<298:11:03,  6.21s/it]
 23%|██▎       | 52130/225000 [109:12:14<281:53:36,  5.87s/it]
 23%|██▎       | 52131/225000 [109:12:19<273:51:13,  5.70s/it]
 23%|██▎       | 52132/225000 [109:12:32<379:41:07,  7.91s/it]
 23%|██▎       | 52133/225000 [109:12:40<374:37:16,  7.80s/it]
 23%|██▎       | 52134/225000 [109:12:45<344:36:45,  7.18s/it]
 23%|██▎       | 52135/225000 [109:12:54<372:51:03,  7.76s/it]
 23%|██▎       | 52136/225000 [109:12:59<320:37:20,  6.68s/it]
 23%|██▎       | 52137/225000 [109:13:03<294:00:00,  6.12s/it]
 23%|██▎       | 52138/225000 [109:13:09<280:54:07,  5.85s/it]
 23%|██▎       | 52139/225000 [109:13:14<280:23:59,  5.84s/it]
 23%|██▎       | 52140/225000 [109:13:20<283:05:54,  5.90s/it]
 23%|██▎       | 52141/225000 [109:13:26<277:10:36,  5.77s/it]
 23%|██▎       | 52142/225000 [109:13:31<272:02:55,  5.67s/it]
 23%|██▎       | 52143/225000 [109:13:44<379:19:45,  7.90s/it]
 23%|██▎       | 52144/225000 [109:13:55<414:30:39,  8.63s/it]
 23%|██▎       | 52145/225000 [109:14:01<375:11:23,  7.81s/it]
 23%|██▎       | 52146/225000 [109:14:08<370:35:29,  7.72s/it]
 23%|██▎       | 52147/225000 [109:14:14<346:47:07,  7.22s/it]
 23%|██▎       | 52148/225000 [109:14:19<315:13:49,  6.57s/it]
 23%|██▎       | 52149/225000 [109:14:24<292:48:10,  6.10s/it]
 23%|██▎       | 52150/225000 [109:14:30<279:51:46,  5.83s/it]
                                                              
{'loss': '0.7145', 'grad_norm': '0.7694', 'learning_rate': '4.541e-05', 'epoch': '0.2318', 'num_input_tokens_seen': 182545712, 'train_runtime': '3.933e+05', 'train_tokens_per_second': '464.2'}

 23%|██▎       | 52150/225000 [109:14:30<279:51:46,  5.83s/it]
 23%|██▎       | 52151/225000 [109:14:36<287:34:57,  5.99s/it]
 23%|██▎       | 52152/225000 [109:14:49<391:00:45,  8.14s/it]
 23%|██▎       | 52153/225000 [109:14:54<350:12:54,  7.29s/it]
 23%|██▎       | 52154/225000 [109:15:02<354:28:37,  7.38s/it]
 23%|██▎       | 52155/225000 [109:15:10<360:20:49,  7.51s/it]
 23%|██▎       | 52156/225000 [109:15:17<361:48:52,  7.54s/it]
 23%|██▎       | 52157/225000 [109:15:21<306:20:10,  6.38s/it]
 23%|██▎       | 52158/225000 [109:15:26<291:02:05,  6.06s/it]
 23%|██▎       | 52159/225000 [109:15:32<285:52:18,  5.95s/it]
 23%|██▎       | 52160/225000 [109:15:37<274:04:52,  5.71s/it]
 23%|██▎       | 52161/225000 [109:15:43<269:07:01,  5.61s/it]
 23%|██▎       | 52162/225000 [109:15:49<280:20:09,  5.84s/it]
 23%|██▎       | 52163/225000 [109:16:02<388:16:23,  8.09s/it]
 23%|██▎       | 52164/225000 [109:16:10<379:42:44,  7.91s/it]
 23%|██▎       | 52165/225000 [109:16:14<327:05:27,  6.81s/it]
 23%|██▎       | 52166/225000 [109:16:19<294:02:30,  6.12s/it]
 23%|██▎       | 52167/225000 [109:16:30<365:07:33,  7.61s/it]
 23%|██▎       | 52168/225000 [109:16:34<316:40:49,  6.60s/it]
 23%|██▎       | 52169/225000 [109:16:40<311:20:16,  6.49s/it]
 23%|██▎       | 52170/225000 [109:16:51<380:31:48,  7.93s/it]
 23%|██▎       | 52171/225000 [109:16:56<333:09:37,  6.94s/it]
 23%|██▎       | 52172/225000 [109:17:01<310:10:12,  6.46s/it]
 23%|██▎       | 52173/225000 [109:17:06<289:35:06,  6.03s/it]
 23%|██▎       | 52174/225000 [109:17:15<329:44:45,  6.87s/it]
 23%|██▎       | 52175/225000 [109:17:25<374:55:08,  7.81s/it]
 23%|██▎       | 52176/225000 [109:17:30<329:26:32,  6.86s/it]
 23%|██▎       | 52177/225000 [109:17:35<303:32:45,  6.32s/it]
 23%|██▎       | 52178/225000 [109:17:45<362:48:26,  7.56s/it]
 23%|██▎       | 52179/225000 [109:17:53<356:48:40,  7.43s/it]
 23%|██▎       | 52180/225000 [109:17:57<317:26:45,  6.61s/it]
 23%|██▎       | 52181/225000 [109:18:02<292:53:35,  6.10s/it]
 23%|██▎       | 52182/225000 [109:18:07<280:07:14,  5.84s/it]
 23%|██▎       | 52183/225000 [109:18:20<383:13:23,  7.98s/it]
 23%|██▎       | 52184/225000 [109:18:26<343:46:35,  7.16s/it]
 23%|██▎       | 52185/225000 [109:18:33<350:44:37,  7.31s/it]
 23%|██▎       | 52186/225000 [109:18:37<304:27:57,  6.34s/it]
 23%|██▎       | 52187/225000 [109:18:47<357:27:31,  7.45s/it]
 23%|██▎       | 52188/225000 [109:18:52<316:31:31,  6.59s/it]
 23%|██▎       | 52189/225000 [109:18:58<302:03:53,  6.29s/it]
 23%|██▎       | 52190/225000 [109:19:08<362:57:45,  7.56s/it]
 23%|██▎       | 52191/225000 [109:19:16<361:52:53,  7.54s/it]
 23%|██▎       | 52192/225000 [109:19:22<351:42:37,  7.33s/it]
 23%|██▎       | 52193/225000 [109:19:27<310:57:41,  6.48s/it]
 23%|██▎       | 52194/225000 [109:19:34<323:48:31,  6.75s/it]
 23%|██▎       | 52195/225000 [109:19:39<300:10:31,  6.25s/it]
 23%|██▎       | 52196/225000 [109:19:45<286:46:03,  5.97s/it]
 23%|██▎       | 52197/225000 [109:19:49<268:04:16,  5.58s/it]
 23%|██▎       | 52198/225000 [109:19:54<260:07:49,  5.42s/it]
 23%|██▎       | 52199/225000 [109:20:01<272:44:39,  5.68s/it]
 23%|██▎       | 52200/225000 [109:20:14<389:06:59,  8.11s/it]
                                                              
{'loss': '0.7116', 'grad_norm': '0.691', 'learning_rate': '4.54e-05', 'epoch': '0.232', 'num_input_tokens_seen': 182695872, 'train_runtime': '3.936e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52200/225000 [109:20:14<389:06:59,  8.11s/it]
 23%|██▎       | 52201/225000 [109:20:19<344:10:40,  7.17s/it]
 23%|██▎       | 52202/225000 [109:20:28<362:21:44,  7.55s/it]
 23%|██▎       | 52203/225000 [109:20:33<328:16:12,  6.84s/it]
 23%|██▎       | 52204/225000 [109:20:38<302:09:17,  6.30s/it]
 23%|██▎       | 52205/225000 [109:20:43<288:21:22,  6.01s/it]
 23%|██▎       | 52206/225000 [109:20:49<284:06:50,  5.92s/it]
 23%|██▎       | 52207/225000 [109:21:02<390:53:07,  8.14s/it]
 23%|██▎       | 52208/225000 [109:21:11<390:15:12,  8.13s/it]
 23%|██▎       | 52209/225000 [109:21:15<341:03:09,  7.11s/it]
 23%|██▎       | 52210/225000 [109:21:20<307:36:52,  6.41s/it]
 23%|██▎       | 52211/225000 [109:21:25<292:57:54,  6.10s/it]
 23%|██▎       | 52212/225000 [109:21:37<370:47:57,  7.73s/it]
 23%|██▎       | 52213/225000 [109:21:44<368:01:41,  7.67s/it]
 23%|██▎       | 52214/225000 [109:21:49<322:32:44,  6.72s/it]
 23%|██▎       | 52215/225000 [109:21:54<301:06:24,  6.27s/it]
 23%|██▎       | 52216/225000 [109:21:59<284:16:58,  5.92s/it]
 23%|██▎       | 52217/225000 [109:22:05<281:39:36,  5.87s/it]
 23%|██▎       | 52218/225000 [109:22:19<395:05:00,  8.23s/it]
 23%|██▎       | 52219/225000 [109:22:24<351:26:21,  7.32s/it]
 23%|██▎       | 52220/225000 [109:22:32<360:54:35,  7.52s/it]
 23%|██▎       | 52221/225000 [109:22:36<309:21:25,  6.45s/it]
 23%|██▎       | 52222/225000 [109:22:47<369:10:53,  7.69s/it]
 23%|██▎       | 52223/225000 [109:22:53<349:34:56,  7.28s/it]
 23%|██▎       | 52224/225000 [109:22:58<320:25:42,  6.68s/it]
 23%|██▎       | 52225/225000 [109:23:04<303:16:03,  6.32s/it]
 23%|██▎       | 52226/225000 [109:23:10<297:13:56,  6.19s/it]
 23%|██▎       | 52227/225000 [109:23:15<292:14:16,  6.09s/it]
 23%|██▎       | 52228/225000 [109:23:21<284:07:07,  5.92s/it]
 23%|██▎       | 52229/225000 [109:23:27<286:37:55,  5.97s/it]
 23%|██▎       | 52230/225000 [109:23:32<277:35:48,  5.78s/it]
 23%|██▎       | 52231/225000 [109:23:38<280:12:16,  5.84s/it]
 23%|██▎       | 52232/225000 [109:23:52<392:26:12,  8.18s/it]
 23%|██▎       | 52233/225000 [109:23:59<381:46:23,  7.96s/it]
 23%|██▎       | 52234/225000 [109:24:06<366:39:09,  7.64s/it]
 23%|██▎       | 52235/225000 [109:24:15<386:57:29,  8.06s/it]
 23%|██▎       | 52236/225000 [109:24:20<334:16:45,  6.97s/it]
 23%|██▎       | 52237/225000 [109:24:24<300:11:48,  6.26s/it]
 23%|██▎       | 52238/225000 [109:24:36<378:43:17,  7.89s/it]
 23%|██▎       | 52239/225000 [109:24:39<315:02:02,  6.56s/it]
 23%|██▎       | 52240/225000 [109:24:47<332:27:43,  6.93s/it]
 23%|██▎       | 52241/225000 [109:24:52<298:44:11,  6.23s/it]
 23%|██▎       | 52242/225000 [109:25:03<372:41:09,  7.77s/it]
 23%|██▎       | 52243/225000 [109:25:06<307:26:06,  6.41s/it]
 23%|██▎       | 52244/225000 [109:25:16<355:32:25,  7.41s/it]
 23%|██▎       | 52245/225000 [109:25:24<361:19:04,  7.53s/it]
 23%|██▎       | 52246/225000 [109:25:28<309:29:32,  6.45s/it]
 23%|██▎       | 52247/225000 [109:25:35<319:23:09,  6.66s/it]
 23%|██▎       | 52248/225000 [109:25:41<303:29:13,  6.32s/it]
 23%|██▎       | 52249/225000 [109:25:46<290:25:28,  6.05s/it]
 23%|██▎       | 52250/225000 [109:25:52<295:44:42,  6.16s/it]
                                                              
{'loss': '0.6975', 'grad_norm': '0.7251', 'learning_rate': '4.539e-05', 'epoch': '0.2322', 'num_input_tokens_seen': 182847552, 'train_runtime': '3.94e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52250/225000 [109:25:52<295:44:42,  6.16s/it]
 23%|██▎       | 52251/225000 [109:26:01<334:04:01,  6.96s/it]
 23%|██▎       | 52252/225000 [109:26:09<348:54:00,  7.27s/it]
 23%|██▎       | 52253/225000 [109:26:15<328:16:36,  6.84s/it]
 23%|██▎       | 52254/225000 [109:26:21<309:44:32,  6.45s/it]
 23%|██▎       | 52255/225000 [109:26:27<312:41:04,  6.52s/it]
 23%|██▎       | 52256/225000 [109:26:37<352:01:56,  7.34s/it]
 23%|██▎       | 52257/225000 [109:26:41<304:18:01,  6.34s/it]
 23%|██▎       | 52258/225000 [109:26:51<365:47:48,  7.62s/it]
 23%|██▎       | 52259/225000 [109:26:56<325:11:30,  6.78s/it]
 23%|██▎       | 52260/225000 [109:27:01<303:01:24,  6.32s/it]
 23%|██▎       | 52261/225000 [109:27:07<292:15:06,  6.09s/it]
 23%|██▎       | 52262/225000 [109:27:12<282:51:22,  5.89s/it]
 23%|██▎       | 52263/225000 [109:27:18<273:39:37,  5.70s/it]
 23%|██▎       | 52264/225000 [109:27:23<274:19:52,  5.72s/it]
 23%|██▎       | 52265/225000 [109:27:29<271:59:42,  5.67s/it]
 23%|██▎       | 52266/225000 [109:27:34<268:04:11,  5.59s/it]
 23%|██▎       | 52267/225000 [109:27:48<379:06:03,  7.90s/it]
 23%|██▎       | 52268/225000 [109:27:58<410:15:43,  8.55s/it]
 23%|██▎       | 52269/225000 [109:28:01<339:41:34,  7.08s/it]
 23%|██▎       | 52270/225000 [109:28:10<368:04:56,  7.67s/it]
 23%|██▎       | 52271/225000 [109:28:15<321:14:04,  6.70s/it]
 23%|██▎       | 52272/225000 [109:28:19<292:06:26,  6.09s/it]
 23%|██▎       | 52273/225000 [109:28:26<301:57:05,  6.29s/it]
 23%|██▎       | 52274/225000 [109:28:35<336:12:53,  7.01s/it]
 23%|██▎       | 52275/225000 [109:28:44<369:37:23,  7.70s/it]
 23%|██▎       | 52276/225000 [109:28:49<323:40:02,  6.75s/it]
 23%|██▎       | 52277/225000 [109:28:57<347:42:48,  7.25s/it]
 23%|██▎       | 52278/225000 [109:29:03<321:56:31,  6.71s/it]
 23%|██▎       | 52279/225000 [109:29:11<344:30:31,  7.18s/it]
 23%|██▎       | 52280/225000 [109:29:17<326:11:30,  6.80s/it]
 23%|██▎       | 52281/225000 [109:29:23<314:16:01,  6.55s/it]
 23%|██▎       | 52282/225000 [109:29:29<317:12:39,  6.61s/it]
 23%|██▎       | 52283/225000 [109:29:34<289:02:28,  6.02s/it]
 23%|██▎       | 52284/225000 [109:29:39<273:30:31,  5.70s/it]
 23%|██▎       | 52285/225000 [109:29:44<261:03:48,  5.44s/it]
 23%|██▎       | 52286/225000 [109:29:55<346:58:49,  7.23s/it]
 23%|██▎       | 52287/225000 [109:30:02<340:59:53,  7.11s/it]
 23%|██▎       | 52288/225000 [109:30:08<321:58:47,  6.71s/it]
 23%|██▎       | 52289/225000 [109:30:17<357:43:26,  7.46s/it]
 23%|██▎       | 52290/225000 [109:30:21<309:40:36,  6.45s/it]
 23%|██▎       | 52291/225000 [109:30:25<272:28:46,  5.68s/it]
 23%|██▎       | 52292/225000 [109:30:30<266:13:25,  5.55s/it]
 23%|██▎       | 52293/225000 [109:30:35<259:59:11,  5.42s/it]
 23%|██▎       | 52294/225000 [109:30:48<369:16:05,  7.70s/it]
 23%|██▎       | 52295/225000 [109:30:54<344:24:21,  7.18s/it]
 23%|██▎       | 52296/225000 [109:30:59<312:57:25,  6.52s/it]
 23%|██▎       | 52297/225000 [109:31:05<293:36:03,  6.12s/it]
 23%|██▎       | 52298/225000 [109:31:10<278:30:33,  5.81s/it]
 23%|██▎       | 52299/225000 [109:31:24<396:15:18,  8.26s/it]
 23%|██▎       | 52300/225000 [109:31:29<352:16:20,  7.34s/it]
                                                              
{'loss': '0.7191', 'grad_norm': '0.754', 'learning_rate': '4.538e-05', 'epoch': '0.2324', 'num_input_tokens_seen': 182996776, 'train_runtime': '3.943e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52300/225000 [109:31:29<352:16:20,  7.34s/it]
 23%|██▎       | 52301/225000 [109:31:37<364:12:22,  7.59s/it]
 23%|██▎       | 52302/225000 [109:31:42<330:43:40,  6.89s/it]
 23%|██▎       | 52303/225000 [109:31:53<386:43:08,  8.06s/it]
 23%|██▎       | 52304/225000 [109:31:58<343:13:23,  7.15s/it]
 23%|██▎       | 52305/225000 [109:32:05<335:11:10,  6.99s/it]
 23%|██▎       | 52306/225000 [109:32:09<300:57:45,  6.27s/it]
 23%|██▎       | 52307/225000 [109:32:15<294:51:47,  6.15s/it]
 23%|██▎       | 52308/225000 [109:32:21<295:27:48,  6.16s/it]
 23%|██▎       | 52309/225000 [109:32:32<363:08:11,  7.57s/it]
 23%|██▎       | 52310/225000 [109:32:36<310:24:58,  6.47s/it]
 23%|██▎       | 52311/225000 [109:32:41<292:24:39,  6.10s/it]
 23%|██▎       | 52312/225000 [109:32:53<374:51:50,  7.81s/it]
 23%|██▎       | 52313/225000 [109:32:58<327:52:39,  6.84s/it]
 23%|██▎       | 52314/225000 [109:33:08<371:56:22,  7.75s/it]
 23%|██▎       | 52315/225000 [109:33:13<331:49:46,  6.92s/it]
 23%|██▎       | 52316/225000 [109:33:17<300:02:37,  6.26s/it]
 23%|██▎       | 52317/225000 [109:33:22<280:01:50,  5.84s/it]
 23%|██▎       | 52318/225000 [109:33:30<314:52:58,  6.56s/it]
 23%|██▎       | 52319/225000 [109:33:38<330:45:53,  6.90s/it]
 23%|██▎       | 52320/225000 [109:33:47<356:28:46,  7.43s/it]
 23%|██▎       | 52321/225000 [109:33:52<326:00:09,  6.80s/it]
 23%|██▎       | 52322/225000 [109:33:57<298:14:10,  6.22s/it]
 23%|██▎       | 52323/225000 [109:34:02<282:53:00,  5.90s/it]
 23%|██▎       | 52324/225000 [109:34:08<276:53:55,  5.77s/it]
 23%|██▎       | 52325/225000 [109:34:21<385:44:54,  8.04s/it]
 23%|██▎       | 52326/225000 [109:34:25<328:36:16,  6.85s/it]
 23%|██▎       | 52327/225000 [109:34:34<365:59:14,  7.63s/it]
 23%|██▎       | 52328/225000 [109:34:38<310:30:33,  6.47s/it]
 23%|██▎       | 52329/225000 [109:34:43<285:42:01,  5.96s/it]
 23%|██▎       | 52330/225000 [109:34:48<271:58:50,  5.67s/it]
 23%|██▎       | 52331/225000 [109:34:53<264:43:54,  5.52s/it]
 23%|██▎       | 52332/225000 [109:34:59<265:22:51,  5.53s/it]
 23%|██▎       | 52333/225000 [109:35:10<346:09:27,  7.22s/it]
 23%|██▎       | 52334/225000 [109:35:19<369:18:43,  7.70s/it]
 23%|██▎       | 52335/225000 [109:35:22<313:00:40,  6.53s/it]
 23%|██▎       | 52336/225000 [109:35:33<370:43:07,  7.73s/it]
 23%|██▎       | 52337/225000 [109:35:39<341:22:43,  7.12s/it]
 23%|██▎       | 52338/225000 [109:35:45<325:54:51,  6.80s/it]
 23%|██▎       | 52339/225000 [109:35:51<311:15:05,  6.49s/it]
 23%|██▎       | 52340/225000 [109:35:56<299:28:15,  6.24s/it]
 23%|██▎       | 52341/225000 [109:36:01<283:40:59,  5.91s/it]
 23%|██▎       | 52342/225000 [109:36:08<293:54:29,  6.13s/it]
 23%|██▎       | 52343/225000 [109:36:17<341:33:21,  7.12s/it]
 23%|██▎       | 52344/225000 [109:36:27<371:24:23,  7.74s/it]
 23%|██▎       | 52345/225000 [109:36:31<328:24:34,  6.85s/it]
 23%|██▎       | 52346/225000 [109:36:36<296:58:41,  6.19s/it]
 23%|██▎       | 52347/225000 [109:36:47<360:52:30,  7.52s/it]
 23%|██▎       | 52348/225000 [109:36:53<350:01:05,  7.30s/it]
 23%|██▎       | 52349/225000 [109:37:00<333:31:11,  6.95s/it]
 23%|██▎       | 52350/225000 [109:37:09<364:51:33,  7.61s/it]
                                                              
{'loss': '0.7156', 'grad_norm': '0.7699', 'learning_rate': '4.537e-05', 'epoch': '0.2327', 'num_input_tokens_seen': 183147968, 'train_runtime': '3.946e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52350/225000 [109:37:09<364:51:33,  7.61s/it]
 23%|██▎       | 52351/225000 [109:37:14<328:15:14,  6.84s/it]
 23%|██▎       | 52352/225000 [109:37:23<357:36:53,  7.46s/it]
 23%|██▎       | 52353/225000 [109:37:27<317:50:07,  6.63s/it]
 23%|██▎       | 52354/225000 [109:37:32<284:17:21,  5.93s/it]
 23%|██▎       | 52355/225000 [109:37:43<365:38:48,  7.62s/it]
 23%|██▎       | 52356/225000 [109:37:48<328:34:26,  6.85s/it]
 23%|██▎       | 52357/225000 [109:37:52<289:04:50,  6.03s/it]
 23%|██▎       | 52358/225000 [109:37:58<276:59:02,  5.78s/it]
 23%|██▎       | 52359/225000 [109:38:07<322:51:14,  6.73s/it]
 23%|██▎       | 52360/225000 [109:38:16<364:04:50,  7.59s/it]
 23%|██▎       | 52361/225000 [109:38:20<315:20:40,  6.58s/it]
 23%|██▎       | 52362/225000 [109:38:26<300:38:36,  6.27s/it]
 23%|██▎       | 52363/225000 [109:38:36<350:55:07,  7.32s/it]
 23%|██▎       | 52364/225000 [109:38:40<308:10:20,  6.43s/it]
 23%|██▎       | 52365/225000 [109:38:50<353:39:52,  7.38s/it]
 23%|██▎       | 52366/225000 [109:38:56<343:48:30,  7.17s/it]
 23%|██▎       | 52367/225000 [109:39:03<337:23:27,  7.04s/it]
 23%|██▎       | 52368/225000 [109:39:08<314:02:04,  6.55s/it]
 23%|██▎       | 52369/225000 [109:39:17<338:19:44,  7.06s/it]
 23%|██▎       | 52370/225000 [109:39:20<290:56:33,  6.07s/it]
 23%|██▎       | 52371/225000 [109:39:26<285:49:17,  5.96s/it]
 23%|██▎       | 52372/225000 [109:39:31<274:49:03,  5.73s/it]
 23%|██▎       | 52373/225000 [109:39:42<350:32:28,  7.31s/it]
 23%|██▎       | 52374/225000 [109:39:48<326:31:50,  6.81s/it]
 23%|██▎       | 52375/225000 [109:39:54<315:19:41,  6.58s/it]
 23%|██▎       | 52376/225000 [109:40:00<302:47:08,  6.31s/it]
 23%|██▎       | 52377/225000 [109:40:05<291:00:26,  6.07s/it]
 23%|██▎       | 52378/225000 [109:40:11<285:44:11,  5.96s/it]
 23%|██▎       | 52379/225000 [109:40:17<282:27:53,  5.89s/it]
 23%|██▎       | 52380/225000 [109:40:22<278:55:15,  5.82s/it]
 23%|██▎       | 52381/225000 [109:40:34<366:30:54,  7.64s/it]
 23%|██▎       | 52382/225000 [109:40:44<398:42:14,  8.32s/it]
 23%|██▎       | 52383/225000 [109:40:48<331:08:52,  6.91s/it]
 23%|██▎       | 52384/225000 [109:40:59<395:43:13,  8.25s/it]
 23%|██▎       | 52385/225000 [109:41:03<331:37:24,  6.92s/it]
 23%|██▎       | 52386/225000 [109:41:12<366:14:03,  7.64s/it]
 23%|██▎       | 52387/225000 [109:41:16<309:00:56,  6.44s/it]
 23%|██▎       | 52388/225000 [109:41:25<342:31:06,  7.14s/it]
 23%|██▎       | 52389/225000 [109:41:29<300:30:27,  6.27s/it]
 23%|██▎       | 52390/225000 [109:41:40<373:34:36,  7.79s/it]
 23%|██▎       | 52391/225000 [109:41:45<325:08:49,  6.78s/it]
 23%|██▎       | 52392/225000 [109:41:49<295:50:15,  6.17s/it]
 23%|██▎       | 52393/225000 [109:41:54<277:13:23,  5.78s/it]
 23%|██▎       | 52394/225000 [109:42:06<364:11:17,  7.60s/it]
 23%|██▎       | 52395/225000 [109:42:12<336:06:14,  7.01s/it]
 23%|██▎       | 52396/225000 [109:42:17<318:18:15,  6.64s/it]
 23%|██▎       | 52397/225000 [109:42:26<351:17:51,  7.33s/it]
 23%|██▎       | 52398/225000 [109:42:31<308:59:54,  6.44s/it]
 23%|██▎       | 52399/225000 [109:42:36<290:01:09,  6.05s/it]
 23%|██▎       | 52400/225000 [109:42:46<347:19:24,  7.24s/it]
                                                              
{'loss': '0.7078', 'grad_norm': '0.8395', 'learning_rate': '4.536e-05', 'epoch': '0.2329', 'num_input_tokens_seen': 183298584, 'train_runtime': '3.95e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52400/225000 [109:42:46<347:19:24,  7.24s/it]
 23%|██▎       | 52401/225000 [109:42:50<308:19:47,  6.43s/it]
 23%|██▎       | 52402/225000 [109:42:55<282:49:27,  5.90s/it]
 23%|██▎       | 52403/225000 [109:43:03<313:30:14,  6.54s/it]
 23%|██▎       | 52404/225000 [109:43:14<376:46:45,  7.86s/it]
 23%|██▎       | 52405/225000 [109:43:18<316:43:17,  6.61s/it]
 23%|██▎       | 52406/225000 [109:43:29<378:26:43,  7.89s/it]
 23%|██▎       | 52407/225000 [109:43:32<318:21:40,  6.64s/it]
 23%|██▎       | 52408/225000 [109:43:41<343:39:53,  7.17s/it]
 23%|██▎       | 52409/225000 [109:43:50<367:46:09,  7.67s/it]
 23%|██▎       | 52410/225000 [109:43:53<312:12:23,  6.51s/it]
 23%|██▎       | 52411/225000 [109:44:03<361:01:30,  7.53s/it]
 23%|██▎       | 52412/225000 [109:44:08<320:57:14,  6.69s/it]
 23%|██▎       | 52413/225000 [109:44:16<333:01:22,  6.95s/it]
 23%|██▎       | 52414/225000 [109:44:20<301:46:11,  6.29s/it]
 23%|██▎       | 52415/225000 [109:44:26<291:30:13,  6.08s/it]
 23%|██▎       | 52416/225000 [109:44:32<289:36:51,  6.04s/it]
 23%|██▎       | 52417/225000 [109:44:38<282:56:06,  5.90s/it]
 23%|██▎       | 52418/225000 [109:44:51<398:19:07,  8.31s/it]
 23%|██▎       | 52419/225000 [109:44:56<351:12:19,  7.33s/it]
 23%|██▎       | 52420/225000 [109:45:01<315:24:32,  6.58s/it]
 23%|██▎       | 52421/225000 [109:45:06<294:20:59,  6.14s/it]
 23%|██▎       | 52422/225000 [109:45:13<303:34:07,  6.33s/it]
 23%|██▎       | 52423/225000 [109:45:26<389:47:08,  8.13s/it]
 23%|██▎       | 52424/225000 [109:45:29<325:54:20,  6.80s/it]
 23%|██▎       | 52425/225000 [109:45:34<292:00:40,  6.09s/it]
 23%|██▎       | 52426/225000 [109:45:40<301:48:26,  6.30s/it]
 23%|██▎       | 52427/225000 [109:45:50<354:31:49,  7.40s/it]
 23%|██▎       | 52428/225000 [109:45:57<335:57:18,  7.01s/it]
 23%|██▎       | 52429/225000 [109:46:04<338:26:26,  7.06s/it]
 23%|██▎       | 52430/225000 [109:46:12<353:29:54,  7.37s/it]
 23%|██▎       | 52431/225000 [109:46:17<318:23:02,  6.64s/it]
 23%|██▎       | 52432/225000 [109:46:22<301:22:28,  6.29s/it]
 23%|██▎       | 52433/225000 [109:46:28<292:02:12,  6.09s/it]
 23%|██▎       | 52434/225000 [109:46:33<278:27:12,  5.81s/it]
 23%|██▎       | 52435/225000 [109:46:38<271:55:39,  5.67s/it]
 23%|██▎       | 52436/225000 [109:46:44<273:22:38,  5.70s/it]
 23%|██▎       | 52437/225000 [109:46:51<285:00:42,  5.95s/it]
 23%|██▎       | 52438/225000 [109:47:05<402:15:50,  8.39s/it]
 23%|██▎       | 52439/225000 [109:47:10<353:55:49,  7.38s/it]
 23%|██▎       | 52440/225000 [109:47:15<330:11:06,  6.89s/it]
 23%|██▎       | 52441/225000 [109:47:21<312:31:50,  6.52s/it]
 23%|██▎       | 52442/225000 [109:47:27<301:22:41,  6.29s/it]
 23%|██▎       | 52443/225000 [109:47:33<292:41:03,  6.11s/it]
 23%|██▎       | 52444/225000 [109:47:40<316:41:48,  6.61s/it]
 23%|██▎       | 52445/225000 [109:47:50<355:11:31,  7.41s/it]
 23%|██▎       | 52446/225000 [109:47:55<326:20:06,  6.81s/it]
 23%|██▎       | 52447/225000 [109:48:00<304:53:15,  6.36s/it]
 23%|██▎       | 52448/225000 [109:48:06<287:36:57,  6.00s/it]
 23%|██▎       | 52449/225000 [109:48:11<284:28:12,  5.94s/it]
 23%|██▎       | 52450/225000 [109:48:20<330:33:30,  6.90s/it]
                                                              
{'loss': '0.7139', 'grad_norm': '0.7212', 'learning_rate': '4.535e-05', 'epoch': '0.2331', 'num_input_tokens_seen': 183448784, 'train_runtime': '3.953e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52450/225000 [109:48:20<330:33:30,  6.90s/it]
 23%|██▎       | 52451/225000 [109:48:31<383:18:57,  8.00s/it]
 23%|██▎       | 52452/225000 [109:48:38<366:43:05,  7.65s/it]
 23%|██▎       | 52453/225000 [109:48:43<332:41:11,  6.94s/it]
 23%|██▎       | 52454/225000 [109:48:48<302:58:51,  6.32s/it]
 23%|██▎       | 52455/225000 [109:48:53<287:49:09,  6.01s/it]
 23%|██▎       | 52456/225000 [109:49:00<300:53:22,  6.28s/it]
 23%|██▎       | 52457/225000 [109:49:14<409:13:47,  8.54s/it]
 23%|██▎       | 52458/225000 [109:49:20<371:44:01,  7.76s/it]
 23%|██▎       | 52459/225000 [109:49:24<322:08:03,  6.72s/it]
 23%|██▎       | 52460/225000 [109:49:30<301:53:51,  6.30s/it]
 23%|██▎       | 52461/225000 [109:49:38<336:42:25,  7.03s/it]
 23%|██▎       | 52462/225000 [109:49:46<348:59:50,  7.28s/it]
 23%|██▎       | 52463/225000 [109:49:55<370:58:45,  7.74s/it]
 23%|██▎       | 52464/225000 [109:50:00<329:33:13,  6.88s/it]
 23%|██▎       | 52465/225000 [109:50:09<361:04:59,  7.53s/it]
 23%|██▎       | 52466/225000 [109:50:14<326:01:43,  6.80s/it]
 23%|██▎       | 52467/225000 [109:50:19<299:35:21,  6.25s/it]
 23%|██▎       | 52468/225000 [109:50:24<280:50:05,  5.86s/it]
 23%|██▎       | 52469/225000 [109:50:37<388:54:22,  8.11s/it]
 23%|██▎       | 52470/225000 [109:50:43<351:42:19,  7.34s/it]
 23%|██▎       | 52471/225000 [109:50:48<324:07:23,  6.76s/it]
 23%|██▎       | 52472/225000 [109:50:56<343:00:15,  7.16s/it]
 23%|██▎       | 52473/225000 [109:51:01<306:37:39,  6.40s/it]
 23%|██▎       | 52474/225000 [109:51:06<289:05:34,  6.03s/it]
 23%|██▎       | 52475/225000 [109:51:17<357:31:10,  7.46s/it]
 23%|██▎       | 52476/225000 [109:51:22<322:49:58,  6.74s/it]
 23%|██▎       | 52477/225000 [109:51:27<300:16:24,  6.27s/it]
 23%|██▎       | 52478/225000 [109:51:32<284:12:13,  5.93s/it]
 23%|██▎       | 52479/225000 [109:51:38<276:30:27,  5.77s/it]
 23%|██▎       | 52480/225000 [109:51:43<268:14:46,  5.60s/it]
 23%|██▎       | 52481/225000 [109:51:56<373:53:23,  7.80s/it]
 23%|██▎       | 52482/225000 [109:52:02<356:18:45,  7.44s/it]
 23%|██▎       | 52483/225000 [109:52:06<308:17:45,  6.43s/it]
 23%|██▎       | 52484/225000 [109:52:11<278:41:39,  5.82s/it]
 23%|██▎       | 52485/225000 [109:52:24<381:31:34,  7.96s/it]
 23%|██▎       | 52486/225000 [109:52:27<318:12:55,  6.64s/it]
 23%|██▎       | 52487/225000 [109:52:35<328:43:27,  6.86s/it]
 23%|██▎       | 52488/225000 [109:52:44<355:56:37,  7.43s/it]
 23%|██▎       | 52489/225000 [109:52:49<325:07:31,  6.78s/it]
 23%|██▎       | 52490/225000 [109:52:54<300:27:37,  6.27s/it]
 23%|██▎       | 52491/225000 [109:52:59<279:14:17,  5.83s/it]
 23%|██▎       | 52492/225000 [109:53:04<276:38:09,  5.77s/it]
 23%|██▎       | 52493/225000 [109:53:12<302:19:07,  6.31s/it]
 23%|██▎       | 52494/225000 [109:53:24<382:56:51,  7.99s/it]
 23%|██▎       | 52495/225000 [109:53:30<359:34:25,  7.50s/it]
 23%|██▎       | 52496/225000 [109:53:35<319:31:39,  6.67s/it]
 23%|██▎       | 52497/225000 [109:53:43<342:42:48,  7.15s/it]
 23%|██▎       | 52498/225000 [109:53:49<327:41:12,  6.84s/it]
 23%|██▎       | 52499/225000 [109:53:55<311:37:48,  6.50s/it]
 23%|██▎       | 52500/225000 [109:54:03<336:40:29,  7.03s/it]
                                                              
{'loss': '0.6998', 'grad_norm': '0.8096', 'learning_rate': '4.534e-05', 'epoch': '0.2333', 'num_input_tokens_seen': 183600440, 'train_runtime': '3.956e+05', 'train_tokens_per_second': '464.1'}

 23%|██▎       | 52500/225000 [109:54:03<336:40:29,  7.03s/it]
 23%|██▎       | 52501/225000 [109:54:07<294:44:28,  6.15s/it]
 23%|██▎       | 52502/225000 [109:54:13<290:16:03,  6.06s/it]
 23%|██▎       | 52503/225000 [109:54:19<283:28:13,  5.92s/it]
 23%|██▎       | 52504/225000 [109:54:25<286:21:18,  5.98s/it]
 23%|██▎       | 52505/225000 [109:54:30<280:31:52,  5.85s/it]
 23%|██▎       | 52506/225000 [109:54:36<274:28:23,  5.73s/it]
 23%|██▎       | 52507/225000 [109:54:42<275:13:52,  5.74s/it]
 23%|██▎       | 52508/225000 [109:54:54<375:25:06,  7.84s/it]
 23%|██▎       | 52509/225000 [109:55:04<399:22:24,  8.34s/it]
 23%|██▎       | 52510/225000 [109:55:09<350:43:04,  7.32s/it]
 23%|██▎       | 52511/225000 [109:55:14<316:30:27,  6.61s/it]
 23%|██▎       | 52512/225000 [109:55:18<286:19:03,  5.98s/it]
 23%|██▎       | 52513/225000 [109:55:26<315:03:42,  6.58s/it]
 23%|██▎       | 52514/225000 [109:55:35<351:50:27,  7.34s/it]
 23%|██▎       | 52515/225000 [109:55:41<331:48:43,  6.93s/it]
 23%|██▎       | 52516/225000 [109:55:47<312:19:58,  6.52s/it]
 23%|██▎       | 52517/225000 [109:55:52<296:18:54,  6.18s/it]
 23%|██▎       | 52518/225000 [109:55:58<296:09:24,  6.18s/it]
 23%|██▎       | 52519/225000 [109:56:04<291:05:45,  6.08s/it]
 23%|██▎       | 52520/225000 [109:56:10<288:29:28,  6.02s/it]
 23%|██▎       | 52521/225000 [109:56:16<280:41:34,  5.86s/it]
 23%|██▎       | 52522/225000 [109:56:23<299:42:06,  6.26s/it]
 23%|██▎       | 52523/225000 [109:56:37<406:27:03,  8.48s/it]
 23%|██▎       | 52524/225000 [109:56:43<374:15:26,  7.81s/it]
 23%|██▎       | 52525/225000 [109:56:53<408:14:40,  8.52s/it]
 23%|██▎       | 52526/225000 [109:57:00<381:16:02,  7.96s/it]
 23%|██▎       | 52527/225000 [109:57:05<347:36:15,  7.26s/it]
 23%|██▎       | 52528/225000 [109:57:10<316:24:54,  6.60s/it]
 23%|██▎       | 52529/225000 [109:57:16<298:28:29,  6.23s/it]
 23%|██▎       | 52530/225000 [109:57:21<286:20:20,  5.98s/it]
 23%|██▎       | 52531/225000 [109:57:29<315:58:42,  6.60s/it]
 23%|██▎       | 52532/225000 [109:57:42<399:42:32,  8.34s/it]
 23%|██▎       | 52533/225000 [109:57:46<345:38:23,  7.21s/it]
 23%|██▎       | 52534/225000 [109:57:51<315:30:19,  6.59s/it]
 23%|██▎       | 52535/225000 [109:58:02<382:53:30,  7.99s/it]
 23%|██▎       | 52536/225000 [109:58:06<318:55:14,  6.66s/it]
 23%|██▎       | 52537/225000 [109:58:16<368:55:31,  7.70s/it]
 23%|██▎       | 52538/225000 [109:58:20<318:57:00,  6.66s/it]
 23%|██▎       | 52539/225000 [109:58:29<343:35:15,  7.17s/it]
 23%|██▎       | 52540/225000 [109:58:36<346:00:48,  7.22s/it]
 23%|██▎       | 52541/225000 [109:58:43<346:48:26,  7.24s/it]
 23%|██▎       | 52542/225000 [109:58:48<314:58:28,  6.57s/it]
 23%|██▎       | 52543/225000 [109:58:53<292:31:21,  6.11s/it]
 23%|██▎       | 52544/225000 [109:58:59<279:21:41,  5.83s/it]
 23%|██▎       | 52545/225000 [109:59:04<273:58:08,  5.72s/it]
 23%|██▎       | 52546/225000 [109:59:09<267:49:49,  5.59s/it]
 23%|██▎       | 52547/225000 [109:59:20<339:04:12,  7.08s/it]
 23%|██▎       | 52548/225000 [109:59:28<358:12:38,  7.48s/it]
 23%|██▎       | 52549/225000 [109:59:33<316:00:49,  6.60s/it]
 23%|██▎       | 52550/225000 [109:59:40<320:24:51,  6.69s/it]
                                                              
{'loss': '0.7149', 'grad_norm': '0.7885', 'learning_rate': '4.533e-05', 'epoch': '0.2336', 'num_input_tokens_seen': 183752720, 'train_runtime': '3.96e+05', 'train_tokens_per_second': '464'}

 23%|██▎       | 52550/225000 [109:59:40<320:24:51,  6.69s/it]
 23%|██▎       | 52551/225000 [109:59:44<289:43:05,  6.05s/it]
 23%|██▎       | 52552/225000 [109:59:54<346:49:22,  7.24s/it]
 23%|██▎       | 52553/225000 [110:00:01<343:26:48,  7.17s/it]
 23%|██▎       | 52554/225000 [110:00:07<320:37:55,  6.69s/it]
 23%|██▎       | 52555/225000 [110:00:12<302:23:46,  6.31s/it]
 23%|██▎       | 52556/225000 [110:00:18<295:11:49,  6.16s/it]
 23%|██▎       | 52557/225000 [110:00:24<284:33:08,  5.94s/it]
 23%|██▎       | 52558/225000 [110:00:29<277:16:09,  5.79s/it]
 23%|██▎       | 52559/225000 [110:00:35<273:25:57,  5.71s/it]
 23%|██▎       | 52560/225000 [110:00:43<309:30:34,  6.46s/it]
 23%|██▎       | 52561/225000 [110:00:57<424:02:35,  8.85s/it]
 23%|██▎       | 52562/225000 [110:01:03<382:39:39,  7.99s/it]
 23%|██▎       | 52563/225000 [110:01:09<346:09:34,  7.23s/it]
 23%|██▎       | 52564/225000 [110:01:14<318:01:39,  6.64s/it]
 23%|██▎       | 52565/225000 [110:01:19<298:12:26,  6.23s/it]
 23%|██▎       | 52566/225000 [110:01:24<281:33:01,  5.88s/it]
 23%|██▎       | 52567/225000 [110:01:36<363:39:30,  7.59s/it]
 23%|██▎       | 52568/225000 [110:01:41<333:03:04,  6.95s/it]
 23%|██▎       | 52569/225000 [110:01:50<363:19:17,  7.59s/it]
 23%|██▎       | 52570/225000 [110:01:54<308:49:37,  6.45s/it]
 23%|██▎       | 52571/225000 [110:01:59<279:17:42,  5.83s/it]
 23%|██▎       | 52572/225000 [110:02:03<264:07:47,  5.51s/it]
 23%|██▎       | 52573/225000 [110:02:15<351:10:34,  7.33s/it]
 23%|██▎       | 52574/225000 [110:02:22<351:30:41,  7.34s/it]
 23%|██▎       | 52575/225000 [110:02:30<361:12:05,  7.54s/it]
 23%|██▎       | 52576/225000 [110:02:37<345:51:20,  7.22s/it]
 23%|██▎       | 52577/225000 [110:02:41<300:38:54,  6.28s/it]
 23%|██▎       | 52578/225000 [110:02:47<296:38:12,  6.19s/it]
 23%|██▎       | 52579/225000 [110:02:53<295:44:23,  6.17s/it]
 23%|██▎       | 52580/225000 [110:02:58<284:10:32,  5.93s/it]
 23%|██▎       | 52581/225000 [110:03:03<273:04:44,  5.70s/it]
 23%|██▎       | 52582/225000 [110:03:09<278:17:40,  5.81s/it]
 23%|██▎       | 52583/225000 [110:03:18<310:49:43,  6.49s/it]
 23%|██▎       | 52584/225000 [110:03:31<405:39:46,  8.47s/it]
 23%|██▎       | 52585/225000 [110:03:35<342:04:42,  7.14s/it]
 23%|██▎       | 52586/225000 [110:03:45<384:50:58,  8.04s/it]
 23%|██▎       | 52587/225000 [110:03:50<345:57:43,  7.22s/it]
 23%|██▎       | 52588/225000 [110:03:55<318:37:08,  6.65s/it]
 23%|██▎       | 52589/225000 [110:04:01<304:52:55,  6.37s/it]
 23%|██▎       | 52590/225000 [110:04:06<289:10:06,  6.04s/it]
 23%|██▎       | 52591/225000 [110:04:12<283:34:04,  5.92s/it]
 23%|██▎       | 52592/225000 [110:04:18<279:21:06,  5.83s/it]
 23%|██▎       | 52593/225000 [110:04:23<276:38:40,  5.78s/it]
 23%|██▎       | 52594/225000 [110:04:36<380:21:40,  7.94s/it]
 23%|██▎       | 52595/225000 [110:04:43<366:22:10,  7.65s/it]
 23%|██▎       | 52596/225000 [110:04:47<316:03:08,  6.60s/it]
 23%|██▎       | 52597/225000 [110:04:58<370:09:45,  7.73s/it]
 23%|██▎       | 52598/225000 [110:05:04<353:48:10,  7.39s/it]
 23%|██▎       | 52599/225000 [110:05:09<307:52:10,  6.43s/it]
 23%|██▎       | 52600/225000 [110:05:20<372:21:38,  7.78s/it]
                                                              
{'loss': '0.7085', 'grad_norm': '0.7451', 'learning_rate': '4.532e-05', 'epoch': '0.2338', 'num_input_tokens_seen': 183904648, 'train_runtime': '3.963e+05', 'train_tokens_per_second': '464'}

 23%|██▎       | 52600/225000 [110:05:20<372:21:38,  7.78s/it]
 23%|██▎       | 52601/225000 [110:05:24<327:29:39,  6.84s/it]
 23%|██▎       | 52602/225000 [110:05:29<296:37:03,  6.19s/it]
 23%|██▎       | 52603/225000 [110:05:40<370:13:31,  7.73s/it]
 23%|██▎       | 52604/225000 [110:05:46<336:48:28,  7.03s/it]
 23%|██▎       | 52605/225000 [110:05:51<314:37:19,  6.57s/it]
 23%|██▎       | 52606/225000 [110:05:56<296:52:20,  6.20s/it]
 23%|██▎       | 52607/225000 [110:06:02<291:49:06,  6.09s/it]
 23%|██▎       | 52608/225000 [110:06:08<282:40:58,  5.90s/it]
 23%|██▎       | 52609/225000 [110:06:13<276:54:38,  5.78s/it]
 23%|██▎       | 52610/225000 [110:06:19<270:51:44,  5.66s/it]
 23%|██▎       | 52611/225000 [110:06:24<265:56:09,  5.55s/it]
 23%|██▎       | 52612/225000 [110:06:37<380:33:39,  7.95s/it]
 23%|██▎       | 52613/225000 [110:06:47<408:59:34,  8.54s/it]
 23%|██▎       | 52614/225000 [110:06:52<348:03:53,  7.27s/it]
 23%|██▎       | 52615/225000 [110:06:57<313:24:11,  6.54s/it]
 23%|██▎       | 52616/225000 [110:07:08<389:31:24,  8.13s/it]
 23%|██▎       | 52617/225000 [110:07:15<373:34:25,  7.80s/it]
 23%|██▎       | 52618/225000 [110:07:21<337:15:00,  7.04s/it]
 23%|██▎       | 52619/225000 [110:07:26<309:51:30,  6.47s/it]
 23%|██▎       | 52620/225000 [110:07:31<293:33:59,  6.13s/it]
 23%|██▎       | 52621/225000 [110:07:36<277:43:17,  5.80s/it]
 23%|██▎       | 52622/225000 [110:07:47<347:52:45,  7.27s/it]
 23%|██▎       | 52623/225000 [110:07:58<408:40:30,  8.53s/it]
 23%|██▎       | 52624/225000 [110:08:03<356:04:55,  7.44s/it]
 23%|██▎       | 52625/225000 [110:08:09<331:16:24,  6.92s/it]
 23%|██▎       | 52626/225000 [110:08:14<309:40:01,  6.47s/it]
 23%|██▎       | 52627/225000 [110:08:20<292:23:49,  6.11s/it]
 23%|██▎       | 52628/225000 [110:08:26<294:39:58,  6.15s/it]
 23%|██▎       | 52629/225000 [110:08:39<397:39:04,  8.31s/it]
 23%|██▎       | 52630/225000 [110:08:43<329:49:18,  6.89s/it]
 23%|██▎       | 52631/225000 [110:08:52<360:26:00,  7.53s/it]
 23%|██▎       | 52632/225000 [110:08:57<322:10:34,  6.73s/it]
 23%|██▎       | 52633/225000 [110:09:02<295:22:37,  6.17s/it]
 23%|██▎       | 52634/225000 [110:09:10<330:53:50,  6.91s/it]
 23%|██▎       | 52635/225000 [110:09:19<356:36:15,  7.45s/it]
 23%|██▎       | 52636/225000 [110:09:25<332:43:59,  6.95s/it]
 23%|██▎       | 52637/225000 [110:09:30<316:19:57,  6.61s/it]
 23%|██▎       | 52638/225000 [110:09:41<365:34:01,  7.64s/it]
 23%|██▎       | 52639/225000 [110:09:45<323:43:54,  6.76s/it]
 23%|██▎       | 52640/225000 [110:09:50<289:37:39,  6.05s/it]
 23%|██▎       | 52641/225000 [110:10:02<376:39:18,  7.87s/it]
 23%|██▎       | 52642/225000 [110:10:06<327:19:17,  6.84s/it]
 23%|██▎       | 52643/225000 [110:10:13<329:12:20,  6.88s/it]
 23%|██▎       | 52644/225000 [110:10:20<335:26:31,  7.01s/it]
 23%|██▎       | 52645/225000 [110:10:25<294:30:09,  6.15s/it]
 23%|██▎       | 52646/225000 [110:10:35<356:48:01,  7.45s/it]
 23%|██▎       | 52647/225000 [110:10:39<308:08:08,  6.44s/it]
 23%|██▎       | 52648/225000 [110:10:49<354:46:43,  7.41s/it]
 23%|██▎       | 52649/225000 [110:10:56<349:10:54,  7.29s/it]
 23%|██▎       | 52650/225000 [110:11:02<332:51:17,  6.95s/it]
                                                              
{'loss': '0.7071', 'grad_norm': '0.6912', 'learning_rate': '4.531e-05', 'epoch': '0.234', 'num_input_tokens_seen': 184055448, 'train_runtime': '3.967e+05', 'train_tokens_per_second': '464'}

 23%|██▎       | 52650/225000 [110:11:02<332:51:17,  6.95s/it]
 23%|██▎       | 52651/225000 [110:11:09<329:44:57,  6.89s/it]
 23%|██▎       | 52652/225000 [110:11:16<340:26:25,  7.11s/it]
 23%|██▎       | 52653/225000 [110:11:24<351:36:23,  7.34s/it]
 23%|██▎       | 52654/225000 [110:11:30<322:20:11,  6.73s/it]
 23%|██▎       | 52655/225000 [110:11:34<288:00:47,  6.02s/it]
 23%|██▎       | 52656/225000 [110:11:39<267:44:55,  5.59s/it]
 23%|██▎       | 52657/225000 [110:11:43<247:46:12,  5.18s/it]
 23%|██▎       | 52658/225000 [110:11:52<309:20:33,  6.46s/it]
 23%|██▎       | 52659/225000 [110:12:03<377:50:40,  7.89s/it]
 23%|██▎       | 52660/225000 [110:12:08<324:07:27,  6.77s/it]
 23%|██▎       | 52661/225000 [110:12:12<289:43:44,  6.05s/it]
 23%|██▎       | 52662/225000 [110:12:20<322:52:08,  6.74s/it]
 23%|██▎       | 52663/225000 [110:12:28<343:12:46,  7.17s/it]
 23%|██▎       | 52664/225000 [110:12:33<306:51:54,  6.41s/it]
 23%|██▎       | 52665/225000 [110:12:43<350:27:29,  7.32s/it]
 23%|██▎       | 52666/225000 [110:12:46<301:06:43,  6.29s/it]
 23%|██▎       | 52667/225000 [110:12:53<297:55:10,  6.22s/it]
 23%|██▎       | 52668/225000 [110:12:58<289:18:11,  6.04s/it]
 23%|██▎       | 52669/225000 [110:13:09<358:39:03,  7.49s/it]
 23%|██▎       | 52670/225000 [110:13:15<332:19:27,  6.94s/it]
 23%|██▎       | 52671/225000 [110:13:24<371:18:28,  7.76s/it]
 23%|██▎       | 52672/225000 [110:13:31<361:53:06,  7.56s/it]
 23%|██▎       | 52673/225000 [110:13:37<331:10:55,  6.92s/it]
 23%|██▎       | 52674/225000 [110:13:45<350:36:39,  7.32s/it]
 23%|██▎       | 52675/225000 [110:13:51<324:47:16,  6.79s/it]
 23%|██▎       | 52676/225000 [110:13:55<293:21:17,  6.13s/it]
 23%|██▎       | 52677/225000 [110:14:06<362:48:12,  7.58s/it]
 23%|██▎       | 52678/225000 [110:14:11<319:45:05,  6.68s/it]
 23%|██▎       | 52679/225000 [110:14:15<290:06:46,  6.06s/it]
 23%|██▎       | 52680/225000 [110:14:20<272:34:19,  5.69s/it]
 23%|██▎       | 52681/225000 [110:14:32<357:51:00,  7.48s/it]
 23%|██▎       | 52682/225000 [110:14:41<384:34:23,  8.03s/it]
 23%|██▎       | 52683/225000 [110:14:49<387:36:27,  8.10s/it]
 23%|██▎       | 52684/225000 [110:14:53<327:46:17,  6.85s/it]
 23%|██▎       | 52685/225000 [110:14:58<300:16:13,  6.27s/it]
 23%|██▎       | 52686/225000 [110:15:04<292:16:42,  6.11s/it]
 23%|██▎       | 52687/225000 [110:15:16<377:08:57,  7.88s/it]
 23%|██▎       | 52688/225000 [110:15:23<368:15:43,  7.69s/it]
 23%|██▎       | 52689/225000 [110:15:27<313:11:20,  6.54s/it]
 23%|██▎       | 52690/225000 [110:15:37<359:30:44,  7.51s/it]
 23%|██▎       | 52691/225000 [110:15:41<309:35:17,  6.47s/it]
 23%|██▎       | 52692/225000 [110:15:51<364:55:01,  7.62s/it]
 23%|██▎       | 52693/225000 [110:15:56<326:46:36,  6.83s/it]
 23%|██▎       | 52694/225000 [110:16:05<357:11:15,  7.46s/it]
 23%|██▎       | 52695/225000 [110:16:09<306:54:16,  6.41s/it]
 23%|██▎       | 52696/225000 [110:16:19<353:32:09,  7.39s/it]
 23%|██▎       | 52697/225000 [110:16:24<324:06:23,  6.77s/it]
 23%|██▎       | 52698/225000 [110:16:31<332:07:49,  6.94s/it]
 23%|██▎       | 52699/225000 [110:16:36<301:06:11,  6.29s/it]
 23%|██▎       | 52700/225000 [110:16:45<332:35:47,  6.95s/it]
                                                              
{'loss': '0.7082', 'grad_norm': '0.8328', 'learning_rate': '4.53e-05', 'epoch': '0.2342', 'num_input_tokens_seen': 184205896, 'train_runtime': '3.97e+05', 'train_tokens_per_second': '464'}

 23%|██▎       | 52700/225000 [110:16:45<332:35:47,  6.95s/it]
 23%|██▎       | 52701/225000 [110:16:49<300:33:46,  6.28s/it]
 23%|██▎       | 52702/225000 [110:17:00<367:53:33,  7.69s/it]
 23%|██▎       | 52703/225000 [110:17:04<312:21:24,  6.53s/it]
 23%|██▎       | 52704/225000 [110:17:13<344:55:48,  7.21s/it]
 23%|██▎       | 52705/225000 [110:17:17<296:09:21,  6.19s/it]
 23%|██▎       | 52706/225000 [110:17:26<333:23:56,  6.97s/it]
 23%|██▎       | 52707/225000 [110:17:34<354:14:08,  7.40s/it]
 23%|██▎       | 52708/225000 [110:17:38<309:48:00,  6.47s/it]
 23%|██▎       | 52709/225000 [110:17:44<295:53:41,  6.18s/it]
 23%|██▎       | 52710/225000 [110:17:50<288:41:10,  6.03s/it]
 23%|██▎       | 52711/225000 [110:17:56<289:45:53,  6.05s/it]
 23%|██▎       | 52712/225000 [110:18:01<284:24:32,  5.94s/it]
 23%|██▎       | 52713/225000 [110:18:14<381:25:03,  7.97s/it]
 23%|██▎       | 52714/225000 [110:18:19<343:29:31,  7.18s/it]
 23%|██▎       | 52715/225000 [110:18:24<312:05:37,  6.52s/it]
 23%|██▎       | 52716/225000 [110:18:32<333:30:09,  6.97s/it]
 23%|██▎       | 52717/225000 [110:18:38<317:17:45,  6.63s/it]
 23%|██▎       | 52718/225000 [110:18:46<340:28:55,  7.11s/it]
 23%|██▎       | 52719/225000 [110:18:53<334:47:34,  7.00s/it]
 23%|██▎       | 52720/225000 [110:18:58<298:21:46,  6.23s/it]
 23%|██▎       | 52721/225000 [110:19:03<289:02:19,  6.04s/it]
 23%|██▎       | 52722/225000 [110:19:16<383:15:30,  8.01s/it]
 23%|██▎       | 52723/225000 [110:19:19<320:04:42,  6.69s/it]
 23%|██▎       | 52724/225000 [110:19:27<330:15:55,  6.90s/it]
 23%|██▎       | 52725/225000 [110:19:33<313:24:53,  6.55s/it]
 23%|██▎       | 52726/225000 [110:19:41<342:34:01,  7.16s/it]
 23%|██▎       | 52727/225000 [110:19:46<304:06:26,  6.35s/it]
 23%|██▎       | 52728/225000 [110:19:55<352:04:31,  7.36s/it]
 23%|██▎       | 52729/225000 [110:20:03<353:32:54,  7.39s/it]
 23%|██▎       | 52730/225000 [110:20:07<310:41:15,  6.49s/it]
 23%|██▎       | 52731/225000 [110:20:14<317:39:00,  6.64s/it]
 23%|██▎       | 52732/225000 [110:20:24<362:56:52,  7.58s/it]
 23%|██▎       | 52733/225000 [110:20:30<333:39:30,  6.97s/it]
 23%|██▎       | 52734/225000 [110:20:38<351:17:40,  7.34s/it]
 23%|██▎       | 52735/225000 [110:20:43<317:21:26,  6.63s/it]
 23%|██▎       | 52736/225000 [110:20:49<315:16:01,  6.59s/it]
 23%|██▎       | 52737/225000 [110:20:58<344:51:43,  7.21s/it]
 23%|██▎       | 52738/225000 [110:21:02<301:43:23,  6.31s/it]
 23%|██▎       | 52739/225000 [110:21:12<348:00:13,  7.27s/it]
 23%|██▎       | 52740/225000 [110:21:16<308:57:03,  6.46s/it]
 23%|██▎       | 52741/225000 [110:21:24<330:10:54,  6.90s/it]
 23%|██▎       | 52742/225000 [110:21:29<302:49:27,  6.33s/it]
 23%|██▎       | 52743/225000 [110:21:34<284:12:07,  5.94s/it]
 23%|██▎       | 52744/225000 [110:21:40<283:48:34,  5.93s/it]
 23%|██▎       | 52745/225000 [110:21:46<278:31:39,  5.82s/it]
 23%|██▎       | 52746/225000 [110:21:52<290:55:27,  6.08s/it]
 23%|██▎       | 52747/225000 [110:22:06<400:47:22,  8.38s/it]
 23%|██▎       | 52748/225000 [110:22:14<395:46:15,  8.27s/it]
 23%|██▎       | 52749/225000 [110:22:23<403:23:38,  8.43s/it]
 23%|██▎       | 52750/225000 [110:22:29<377:00:44,  7.88s/it]
                                                              
{'loss': '0.706', 'grad_norm': '0.7698', 'learning_rate': '4.528e-05', 'epoch': '0.2344', 'num_input_tokens_seen': 184356728, 'train_runtime': '3.973e+05', 'train_tokens_per_second': '464'}

 23%|██▎       | 52750/225000 [110:22:29<377:00:44,  7.88s/it]
 23%|██▎       | 52751/225000 [110:22:35<337:21:26,  7.05s/it]
 23%|██▎       | 52752/225000 [110:22:43<364:16:38,  7.61s/it]
 23%|██▎       | 52753/225000 [110:22:50<347:24:39,  7.26s/it]
 23%|██▎       | 52754/225000 [110:22:56<334:57:14,  7.00s/it]
 23%|██▎       | 52755/225000 [110:23:01<297:16:29,  6.21s/it]
 23%|██▎       | 52756/225000 [110:23:06<285:34:34,  5.97s/it]
 23%|██▎       | 52757/225000 [110:23:19<380:37:12,  7.96s/it]
 23%|██▎       | 52758/225000 [110:23:25<358:37:04,  7.50s/it]
 23%|██▎       | 52759/225000 [110:23:30<316:02:49,  6.61s/it]
 23%|██▎       | 52760/225000 [110:23:36<320:11:45,  6.69s/it]
 23%|██▎       | 52761/225000 [110:23:46<357:29:14,  7.47s/it]
 23%|██▎       | 52762/225000 [110:23:50<316:39:00,  6.62s/it]
 23%|██▎       | 52763/225000 [110:23:55<290:19:21,  6.07s/it]
 23%|██▎       | 52764/225000 [110:24:00<273:31:04,  5.72s/it]
 23%|██▎       | 52765/225000 [110:24:09<318:24:55,  6.66s/it]
 23%|██▎       | 52766/225000 [110:24:20<375:00:39,  7.84s/it]
 23%|██▎       | 52767/225000 [110:24:26<353:19:56,  7.39s/it]
 23%|██▎       | 52768/225000 [110:24:30<306:41:34,  6.41s/it]
 23%|██▎       | 52769/225000 [110:24:39<350:24:09,  7.32s/it]
 23%|██▎       | 52770/225000 [110:24:46<343:07:13,  7.17s/it]
 23%|██▎       | 52771/225000 [110:24:50<292:29:10,  6.11s/it]
 23%|██▎       | 52772/225000 [110:24:57<311:39:15,  6.51s/it]
 23%|██▎       | 52773/225000 [110:25:06<336:14:46,  7.03s/it]
 23%|██▎       | 52774/225000 [110:25:14<354:40:25,  7.41s/it]
 23%|██▎       | 52775/225000 [110:25:17<299:55:08,  6.27s/it]
 23%|██▎       | 52776/225000 [110:25:24<309:08:22,  6.46s/it]
 23%|██▎       | 52777/225000 [110:25:32<323:22:02,  6.76s/it]
 23%|██▎       | 52778/225000 [110:25:36<290:50:39,  6.08s/it]
 23%|██▎       | 52779/225000 [110:25:41<275:57:02,  5.77s/it]
 23%|██▎       | 52780/225000 [110:25:47<278:33:55,  5.82s/it]
 23%|██▎       | 52781/225000 [110:25:59<360:31:21,  7.54s/it]
 23%|██▎       | 52782/225000 [110:26:04<328:04:44,  6.86s/it]
 23%|██▎       | 52783/225000 [110:26:09<293:15:54,  6.13s/it]
 23%|██▎       | 52784/225000 [110:26:14<284:06:40,  5.94s/it]
 23%|██▎       | 52785/225000 [110:26:27<389:26:16,  8.14s/it]
 23%|██▎       | 52786/225000 [110:26:33<353:28:54,  7.39s/it]
 23%|██▎       | 52787/225000 [110:26:37<311:09:43,  6.50s/it]
 23%|██▎       | 52788/225000 [110:26:41<275:55:31,  5.77s/it]
 23%|██▎       | 52789/225000 [110:26:55<386:25:20,  8.08s/it]
 23%|██▎       | 52790/225000 [110:27:00<341:44:35,  7.14s/it]
 23%|██▎       | 52791/225000 [110:27:05<307:50:53,  6.44s/it]
 23%|██▎       | 52792/225000 [110:27:10<288:41:50,  6.04s/it]
 23%|██▎       | 52793/225000 [110:27:15<276:50:03,  5.79s/it]
 23%|██▎       | 52794/225000 [110:27:25<339:09:52,  7.09s/it]
 23%|██▎       | 52795/225000 [110:27:34<371:29:11,  7.77s/it]
 23%|██▎       | 52796/225000 [110:27:45<411:51:25,  8.61s/it]
 23%|██▎       | 52797/225000 [110:27:51<371:17:30,  7.76s/it]
 23%|██▎       | 52798/225000 [110:27:56<333:41:49,  6.98s/it]
 23%|██▎       | 52799/225000 [110:28:01<310:05:15,  6.48s/it]
 23%|██▎       | 52800/225000 [110:28:07<295:41:36,  6.18s/it]
                                                              
{'loss': '0.7285', 'grad_norm': '0.637', 'learning_rate': '4.527e-05', 'epoch': '0.2347', 'num_input_tokens_seen': 184506848, 'train_runtime': '3.977e+05', 'train_tokens_per_second': '463.9'}

 23%|██▎       | 52800/225000 [110:28:07<295:41:36,  6.18s/it]
 23%|██▎       | 52801/225000 [110:28:12<277:21:21,  5.80s/it]
 23%|██▎       | 52802/225000 [110:28:22<349:08:03,  7.30s/it]
 23%|██▎       | 52803/225000 [110:28:34<403:36:39,  8.44s/it]
 23%|██▎       | 52804/225000 [110:28:37<338:17:00,  7.07s/it]
 23%|██▎       | 52805/225000 [110:28:42<296:03:37,  6.19s/it]
 23%|██▎       | 52806/225000 [110:28:54<378:25:41,  7.91s/it]
 23%|██▎       | 52807/225000 [110:28:58<324:48:45,  6.79s/it]
 23%|██▎       | 52808/225000 [110:29:06<351:35:22,  7.35s/it]
 23%|██▎       | 52809/225000 [110:29:14<360:39:43,  7.54s/it]
 23%|██▎       | 52810/225000 [110:29:20<335:15:51,  7.01s/it]
 23%|██▎       | 52811/225000 [110:29:26<314:05:21,  6.57s/it]
 23%|██▎       | 52812/225000 [110:29:31<292:52:31,  6.12s/it]
 23%|██▎       | 52813/225000 [110:29:36<287:27:29,  6.01s/it]
 23%|██▎       | 52814/225000 [110:29:42<277:29:51,  5.80s/it]
 23%|██▎       | 52815/225000 [110:29:47<275:23:13,  5.76s/it]
 23%|██▎       | 52816/225000 [110:29:58<346:59:40,  7.25s/it]
 23%|██▎       | 52817/225000 [110:30:08<376:17:08,  7.87s/it]
 23%|██▎       | 52818/225000 [110:30:13<344:31:06,  7.20s/it]
 23%|██▎       | 52819/225000 [110:30:23<378:15:19,  7.91s/it]
 23%|██▎       | 52820/225000 [110:30:28<344:11:19,  7.20s/it]
 23%|██▎       | 52821/225000 [110:30:34<316:45:42,  6.62s/it]
 23%|██▎       | 52822/225000 [110:30:40<313:17:19,  6.55s/it]
 23%|██▎       | 52823/225000 [110:30:52<399:51:27,  8.36s/it]
 23%|██▎       | 52824/225000 [110:30:56<330:44:47,  6.92s/it]
 23%|██▎       | 52825/225000 [110:31:04<352:51:14,  7.38s/it]
 23%|██▎       | 52826/225000 [110:31:10<320:18:46,  6.70s/it]
 23%|██▎       | 52827/225000 [110:31:15<304:51:12,  6.37s/it]
 23%|██▎       | 52828/225000 [110:31:23<320:44:36,  6.71s/it]
 23%|██▎       | 52829/225000 [110:31:28<306:40:45,  6.41s/it]
 23%|██▎       | 52830/225000 [110:31:34<291:43:50,  6.10s/it]
 23%|██▎       | 52831/225000 [110:31:45<364:24:04,  7.62s/it]
 23%|██▎       | 52832/225000 [110:31:52<355:56:22,  7.44s/it]
 23%|██▎       | 52833/225000 [110:31:57<325:03:49,  6.80s/it]
 23%|██▎       | 52834/225000 [110:32:03<304:18:38,  6.36s/it]
 23%|██▎       | 52835/225000 [110:32:08<290:32:13,  6.08s/it]
 23%|██▎       | 52836/225000 [110:32:18<344:09:14,  7.20s/it]
 23%|██▎       | 52837/225000 [110:32:26<355:40:02,  7.44s/it]
 23%|██▎       | 52838/225000 [110:32:31<323:17:01,  6.76s/it]
 23%|██▎       | 52839/225000 [110:32:36<299:05:45,  6.25s/it]
 23%|██▎       | 52840/225000 [110:32:42<300:18:30,  6.28s/it]
 23%|██▎       | 52841/225000 [110:32:55<388:01:03,  8.11s/it]
 23%|██▎       | 52842/225000 [110:32:59<326:53:24,  6.84s/it]
 23%|██▎       | 52843/225000 [110:33:08<357:43:52,  7.48s/it]
 23%|██▎       | 52844/225000 [110:33:12<315:52:46,  6.61s/it]
 23%|██▎       | 52845/225000 [110:33:17<284:49:27,  5.96s/it]
 23%|██▎       | 52846/225000 [110:33:25<316:50:03,  6.63s/it]
 23%|██▎       | 52847/225000 [110:33:35<368:04:20,  7.70s/it]
 23%|██▎       | 52848/225000 [110:33:39<320:34:42,  6.70s/it]
 23%|██▎       | 52849/225000 [110:33:47<329:39:35,  6.89s/it]
 23%|██▎       | 52850/225000 [110:33:52<305:34:29,  6.39s/it]
                                                              
{'loss': '0.7074', 'grad_norm': '0.8205', 'learning_rate': '4.526e-05', 'epoch': '0.2349', 'num_input_tokens_seen': 184656688, 'train_runtime': '3.98e+05', 'train_tokens_per_second': '463.9'}

 23%|██▎       | 52850/225000 [110:33:52<305:34:29,  6.39s/it]
 23%|██▎       | 52851/225000 [110:33:57<281:33:49,  5.89s/it]
 23%|██▎       | 52852/225000 [110:34:05<308:46:06,  6.46s/it]
 23%|██▎       | 52853/225000 [110:34:18<412:09:24,  8.62s/it]
 23%|██▎       | 52854/225000 [110:34:25<391:00:47,  8.18s/it]
 23%|██▎       | 52855/225000 [110:34:30<337:51:28,  7.07s/it]
 23%|██▎       | 52856/225000 [110:34:34<300:36:10,  6.29s/it]
 23%|██▎       | 52857/225000 [110:34:40<291:14:21,  6.09s/it]
 23%|██▎       | 52858/225000 [110:34:53<393:37:52,  8.23s/it]
 23%|██▎       | 52859/225000 [110:34:58<348:44:58,  7.29s/it]
 23%|██▎       | 52860/225000 [110:35:06<356:25:09,  7.45s/it]
 23%|██▎       | 52861/225000 [110:35:12<332:08:10,  6.95s/it]
 23%|██▎       | 52862/225000 [110:35:18<322:00:19,  6.73s/it]
 23%|██▎       | 52863/225000 [110:35:26<333:16:44,  6.97s/it]
 23%|██▎       | 52864/225000 [110:35:29<287:46:50,  6.02s/it]
 23%|██▎       | 52865/225000 [110:35:39<344:16:49,  7.20s/it]
 23%|██▎       | 52866/225000 [110:35:45<317:08:20,  6.63s/it]
 23%|██▎       | 52867/225000 [110:35:50<299:55:45,  6.27s/it]
 23%|██▎       | 52868/225000 [110:35:56<297:45:47,  6.23s/it]
 23%|██▎       | 52869/225000 [110:36:05<338:32:31,  7.08s/it]
 23%|██▎       | 52870/225000 [110:36:12<338:42:56,  7.08s/it]
 23%|██▎       | 52871/225000 [110:36:18<311:11:20,  6.51s/it]
 23%|██▎       | 52872/225000 [110:36:26<345:12:06,  7.22s/it]
 23%|██▎       | 52873/225000 [110:36:34<355:02:08,  7.43s/it]
 23%|██▎       | 52874/225000 [110:36:42<356:58:14,  7.47s/it]
 24%|██▎       | 52875/225000 [110:36:49<353:06:17,  7.39s/it]
 24%|██▎       | 52876/225000 [110:36:57<359:23:43,  7.52s/it]
 24%|██▎       | 52877/225000 [110:37:02<323:32:56,  6.77s/it]
 24%|██▎       | 52878/225000 [110:37:07<292:46:35,  6.12s/it]
 24%|██▎       | 52879/225000 [110:37:12<277:43:27,  5.81s/it]
 24%|██▎       | 52880/225000 [110:37:19<297:11:20,  6.22s/it]
 24%|██▎       | 52881/225000 [110:37:30<368:19:18,  7.70s/it]
 24%|██▎       | 52882/225000 [110:37:34<317:59:30,  6.65s/it]
 24%|██▎       | 52883/225000 [110:37:39<296:28:19,  6.20s/it]
 24%|██▎       | 52884/225000 [110:37:52<389:10:58,  8.14s/it]
 24%|██▎       | 52885/225000 [110:37:57<340:25:50,  7.12s/it]
 24%|██▎       | 52886/225000 [110:38:01<297:40:35,  6.23s/it]
 24%|██▎       | 52887/225000 [110:38:09<332:14:53,  6.95s/it]
 24%|██▎       | 52888/225000 [110:38:19<362:02:14,  7.57s/it]
 24%|██▎       | 52889/225000 [110:38:23<314:46:17,  6.58s/it]
 24%|██▎       | 52890/225000 [110:38:27<274:49:53,  5.75s/it]
 24%|██▎       | 52891/225000 [110:38:40<390:09:27,  8.16s/it]
 24%|██▎       | 52892/225000 [110:38:44<325:49:26,  6.82s/it]
 24%|██▎       | 52893/225000 [110:38:48<286:37:58,  6.00s/it]
 24%|██▎       | 52894/225000 [110:39:00<365:14:26,  7.64s/it]
 24%|██▎       | 52895/225000 [110:39:05<338:06:13,  7.07s/it]
 24%|██▎       | 52896/225000 [110:39:11<321:31:38,  6.73s/it]
 24%|██▎       | 52897/225000 [110:39:20<354:39:33,  7.42s/it]
 24%|██▎       | 52898/225000 [110:39:25<316:31:39,  6.62s/it]
 24%|██▎       | 52899/225000 [110:39:30<287:20:53,  6.01s/it]
 24%|██▎       | 52900/225000 [110:39:38<313:45:44,  6.56s/it]
                                                              
{'loss': '0.6991', 'grad_norm': '0.6889', 'learning_rate': '4.525e-05', 'epoch': '0.2351', 'num_input_tokens_seen': 184806696, 'train_runtime': '3.984e+05', 'train_tokens_per_second': '463.9'}

 24%|██▎       | 52900/225000 [110:39:38<313:45:44,  6.56s/it]
 24%|██▎       | 52901/225000 [110:39:48<366:29:15,  7.67s/it]
 24%|██▎       | 52902/225000 [110:39:54<352:34:18,  7.38s/it]
 24%|██▎       | 52903/225000 [110:40:00<326:07:07,  6.82s/it]
 24%|██▎       | 52904/225000 [110:40:06<310:39:02,  6.50s/it]
 24%|██▎       | 52905/225000 [110:40:11<299:45:25,  6.27s/it]
 24%|██▎       | 52906/225000 [110:40:17<287:38:53,  6.02s/it]
 24%|██▎       | 52907/225000 [110:40:28<356:33:37,  7.46s/it]
 24%|██▎       | 52908/225000 [110:40:33<328:34:27,  6.87s/it]
 24%|██▎       | 52909/225000 [110:40:39<315:32:17,  6.60s/it]
 24%|██▎       | 52910/225000 [110:40:52<400:30:06,  8.38s/it]
 24%|██▎       | 52911/225000 [110:40:55<330:56:43,  6.92s/it]
 24%|██▎       | 52912/225000 [110:41:03<343:28:59,  7.19s/it]
 24%|██▎       | 52913/225000 [110:41:09<328:42:31,  6.88s/it]
 24%|██▎       | 52914/225000 [110:41:15<313:09:39,  6.55s/it]
 24%|██▎       | 52915/225000 [110:41:21<302:45:48,  6.33s/it]
 24%|██▎       | 52916/225000 [110:41:27<294:53:44,  6.17s/it]
 24%|██▎       | 52917/225000 [110:41:32<280:03:00,  5.86s/it]
 24%|██▎       | 52918/225000 [110:41:37<268:04:58,  5.61s/it]
 24%|██▎       | 52919/225000 [110:41:44<292:33:36,  6.12s/it]
 24%|██▎       | 52920/225000 [110:41:57<392:04:53,  8.20s/it]
 24%|██▎       | 52921/225000 [110:42:04<367:49:06,  7.69s/it]
 24%|██▎       | 52922/225000 [110:42:08<317:25:22,  6.64s/it]
 24%|██▎       | 52923/225000 [110:42:14<304:14:02,  6.36s/it]
 24%|██▎       | 52924/225000 [110:42:19<293:43:08,  6.14s/it]
 24%|██▎       | 52925/225000 [110:42:25<282:51:00,  5.92s/it]
 24%|██▎       | 52926/225000 [110:42:30<281:42:29,  5.89s/it]
 24%|██▎       | 52927/225000 [110:42:36<280:51:09,  5.88s/it]
 24%|██▎       | 52928/225000 [110:42:49<383:54:27,  8.03s/it]
 24%|██▎       | 52929/225000 [110:42:57<378:32:26,  7.92s/it]
 24%|██▎       | 52930/225000 [110:43:03<354:17:39,  7.41s/it]
 24%|██▎       | 52931/225000 [110:43:07<308:10:41,  6.45s/it]
 24%|██▎       | 52932/225000 [110:43:19<384:33:20,  8.05s/it]
 24%|██▎       | 52933/225000 [110:43:24<333:58:46,  6.99s/it]
 24%|██▎       | 52934/225000 [110:43:30<330:01:44,  6.90s/it]
 24%|██▎       | 52935/225000 [110:43:39<354:48:02,  7.42s/it]
 24%|██▎       | 52936/225000 [110:43:46<345:33:50,  7.23s/it]
 24%|██▎       | 52937/225000 [110:43:53<351:07:31,  7.35s/it]
 24%|██▎       | 52938/225000 [110:43:59<319:22:26,  6.68s/it]
 24%|██▎       | 52939/225000 [110:44:05<319:36:24,  6.69s/it]
 24%|██▎       | 52940/225000 [110:44:13<329:24:10,  6.89s/it]
 24%|██▎       | 52941/225000 [110:44:20<335:30:02,  7.02s/it]
 24%|██▎       | 52942/225000 [110:44:26<323:49:08,  6.78s/it]
 24%|██▎       | 52943/225000 [110:44:31<290:14:47,  6.07s/it]
 24%|██▎       | 52944/225000 [110:44:41<356:46:43,  7.47s/it]
 24%|██▎       | 52945/225000 [110:44:46<321:24:30,  6.73s/it]
 24%|██▎       | 52946/225000 [110:44:53<319:06:28,  6.68s/it]
 24%|██▎       | 52947/225000 [110:44:58<302:35:03,  6.33s/it]
 24%|██▎       | 52948/225000 [110:45:04<292:49:55,  6.13s/it]
 24%|██▎       | 52949/225000 [110:45:09<276:10:14,  5.78s/it]
 24%|██▎       | 52950/225000 [110:45:15<274:56:21,  5.75s/it]
                                                              
{'loss': '0.6975', 'grad_norm': '0.7466', 'learning_rate': '4.524e-05', 'epoch': '0.2353', 'num_input_tokens_seen': 184958848, 'train_runtime': '3.987e+05', 'train_tokens_per_second': '463.9'}

 24%|██▎       | 52950/225000 [110:45:15<274:56:21,  5.75s/it]
 24%|██▎       | 52951/225000 [110:45:20<273:45:44,  5.73s/it]
 24%|██▎       | 52952/225000 [110:45:26<274:41:33,  5.75s/it]
 24%|██▎       | 52953/225000 [110:45:39<373:17:21,  7.81s/it]
 24%|██▎       | 52954/225000 [110:45:46<371:13:13,  7.77s/it]
 24%|██▎       | 52955/225000 [110:45:52<344:56:20,  7.22s/it]
 24%|██▎       | 52956/225000 [110:45:58<323:50:58,  6.78s/it]
 24%|██▎       | 52957/225000 [110:46:03<294:06:51,  6.15s/it]
 24%|██▎       | 52958/225000 [110:46:08<280:57:56,  5.88s/it]
 24%|██▎       | 52959/225000 [110:46:19<346:32:15,  7.25s/it]
 24%|██▎       | 52960/225000 [110:46:25<341:50:21,  7.15s/it]
 24%|██▎       | 52961/225000 [110:46:33<342:42:06,  7.17s/it]
 24%|██▎       | 52962/225000 [110:46:37<307:02:58,  6.43s/it]
 24%|██▎       | 52963/225000 [110:46:43<290:01:34,  6.07s/it]
 24%|██▎       | 52964/225000 [110:46:48<284:12:10,  5.95s/it]
 24%|██▎       | 52965/225000 [110:46:53<271:57:59,  5.69s/it]
 24%|██▎       | 52966/225000 [110:46:59<265:30:55,  5.56s/it]
 24%|██▎       | 52967/225000 [110:47:12<373:43:57,  7.82s/it]
 24%|██▎       | 52968/225000 [110:47:17<337:13:47,  7.06s/it]
 24%|██▎       | 52969/225000 [110:47:25<348:54:46,  7.30s/it]
 24%|██▎       | 52970/225000 [110:47:29<301:30:38,  6.31s/it]
 24%|██▎       | 52971/225000 [110:47:41<386:21:06,  8.09s/it]
 24%|██▎       | 52972/225000 [110:47:46<342:03:02,  7.16s/it]
 24%|██▎       | 52973/225000 [110:47:51<308:19:04,  6.45s/it]
 24%|██▎       | 52974/225000 [110:47:56<283:48:40,  5.94s/it]
 24%|██▎       | 52975/225000 [110:48:01<277:04:04,  5.80s/it]
 24%|██▎       | 52976/225000 [110:48:06<270:52:31,  5.67s/it]
 24%|██▎       | 52977/225000 [110:48:12<274:37:19,  5.75s/it]
 24%|██▎       | 52978/225000 [110:48:21<321:58:37,  6.74s/it]
 24%|██▎       | 52979/225000 [110:48:35<423:08:44,  8.86s/it]
 24%|██▎       | 52980/225000 [110:48:40<358:36:30,  7.50s/it]
 24%|██▎       | 52981/225000 [110:48:44<318:23:58,  6.66s/it]
 24%|██▎       | 52982/225000 [110:48:56<388:12:42,  8.12s/it]
 24%|██▎       | 52983/225000 [110:49:02<356:13:20,  7.46s/it]
 24%|██▎       | 52984/225000 [110:49:07<321:10:42,  6.72s/it]
 24%|██▎       | 52985/225000 [110:49:16<352:43:02,  7.38s/it]
 24%|██▎       | 52986/225000 [110:49:20<304:38:44,  6.38s/it]
 24%|██▎       | 52987/225000 [110:49:29<352:45:34,  7.38s/it]
 24%|██▎       | 52988/225000 [110:49:33<295:01:05,  6.17s/it]
 24%|██▎       | 52989/225000 [110:49:42<343:38:47,  7.19s/it]
 24%|██▎       | 52990/225000 [110:49:51<370:14:28,  7.75s/it]
 24%|██▎       | 52991/225000 [110:49:55<318:24:20,  6.66s/it]
 24%|██▎       | 52992/225000 [110:50:05<355:08:07,  7.43s/it]
 24%|██▎       | 52993/225000 [110:50:09<306:24:07,  6.41s/it]
 24%|██▎       | 52994/225000 [110:50:13<274:15:09,  5.74s/it]
 24%|██▎       | 52995/225000 [110:50:21<311:30:00,  6.52s/it]
 24%|██▎       | 52996/225000 [110:50:32<372:19:23,  7.79s/it]
 24%|██▎       | 52997/225000 [110:50:37<336:07:21,  7.04s/it]
 24%|██▎       | 52998/225000 [110:50:43<310:35:56,  6.50s/it]
 24%|██▎       | 52999/225000 [110:50:48<290:31:38,  6.08s/it]
 24%|██▎       | 53000/225000 [110:50:54<287:53:56,  6.03s/it]
                                                              
{'loss': '0.7075', 'grad_norm': '0.7198', 'learning_rate': '4.523e-05', 'epoch': '0.2356', 'num_input_tokens_seen': 185109992, 'train_runtime': '3.991e+05', 'train_tokens_per_second': '463.9'}

 24%|██▎       | 53000/225000 [110:50:54<287:53:56,  6.03s/it]
 24%|██▎       | 53001/225000 [110:50:59<276:39:35,  5.79s/it]
 24%|██▎       | 53002/225000 [110:51:04<272:08:17,  5.70s/it]
 24%|██▎       | 53003/225000 [110:51:14<333:50:40,  6.99s/it]
 24%|██▎       | 53004/225000 [110:51:23<366:23:40,  7.67s/it]
 24%|██▎       | 53005/225000 [110:51:28<324:15:29,  6.79s/it]
 24%|██▎       | 53006/225000 [110:51:38<368:54:35,  7.72s/it]
 24%|██▎       | 53007/225000 [110:51:42<314:31:43,  6.58s/it]
 24%|██▎       | 53008/225000 [110:51:47<289:50:49,  6.07s/it]
 24%|██▎       | 53009/225000 [110:51:53<293:42:17,  6.15s/it]
 24%|██▎       | 53010/225000 [110:52:06<384:39:54,  8.05s/it]
 24%|██▎       | 53011/225000 [110:52:10<323:41:33,  6.78s/it]
 24%|██▎       | 53012/225000 [110:52:17<332:18:06,  6.96s/it]
 24%|██▎       | 53013/225000 [110:52:21<296:13:48,  6.20s/it]
 24%|██▎       | 53014/225000 [110:52:32<361:21:02,  7.56s/it]
 24%|██▎       | 53015/225000 [110:52:37<317:09:09,  6.64s/it]
 24%|██▎       | 53016/225000 [110:52:41<287:13:22,  6.01s/it]
 24%|██▎       | 53017/225000 [110:52:49<320:37:29,  6.71s/it]
 24%|██▎       | 53018/225000 [110:53:00<377:42:26,  7.91s/it]
 24%|██▎       | 53019/225000 [110:53:05<337:29:56,  7.06s/it]
 24%|██▎       | 53020/225000 [110:53:11<316:20:05,  6.62s/it]
 24%|██▎       | 53021/225000 [110:53:20<358:17:28,  7.50s/it]
 24%|██▎       | 53022/225000 [110:53:25<314:16:36,  6.58s/it]
 24%|██▎       | 53023/225000 [110:53:30<287:14:00,  6.01s/it]
 24%|██▎       | 53024/225000 [110:53:34<269:16:51,  5.64s/it]
 24%|██▎       | 53025/225000 [110:53:46<362:41:37,  7.59s/it]
 24%|██▎       | 53026/225000 [110:53:51<314:03:20,  6.57s/it]
 24%|██▎       | 53027/225000 [110:53:57<314:44:58,  6.59s/it]
 24%|██▎       | 53028/225000 [110:54:05<329:48:39,  6.90s/it]
 24%|██▎       | 53029/225000 [110:54:09<292:48:12,  6.13s/it]
 24%|██▎       | 53030/225000 [110:54:20<364:12:08,  7.62s/it]
 24%|██▎       | 53031/225000 [110:54:26<334:49:24,  7.01s/it]
 24%|██▎       | 53032/225000 [110:54:35<371:26:40,  7.78s/it]
 24%|██▎       | 53033/225000 [110:54:40<327:00:41,  6.85s/it]
 24%|██▎       | 53034/225000 [110:54:45<295:32:39,  6.19s/it]
 24%|██▎       | 53035/225000 [110:54:49<271:13:12,  5.68s/it]
 24%|██▎       | 53036/225000 [110:54:55<274:30:43,  5.75s/it]
 24%|██▎       | 53037/225000 [110:55:06<350:50:22,  7.34s/it]
 24%|██▎       | 53038/225000 [110:55:13<348:34:41,  7.30s/it]
 24%|██▎       | 53039/225000 [110:55:20<335:46:36,  7.03s/it]
 24%|██▎       | 53040/225000 [110:55:27<335:39:12,  7.03s/it]
 24%|██▎       | 53041/225000 [110:55:35<348:13:18,  7.29s/it]
 24%|██▎       | 53042/225000 [110:55:38<295:18:44,  6.18s/it]
 24%|██▎       | 53043/225000 [110:55:49<352:18:10,  7.38s/it]
 24%|██▎       | 53044/225000 [110:55:53<307:45:45,  6.44s/it]
 24%|██▎       | 53045/225000 [110:56:02<342:24:29,  7.17s/it]
 24%|██▎       | 53046/225000 [110:56:06<297:51:03,  6.24s/it]
 24%|██▎       | 53047/225000 [110:56:16<350:03:56,  7.33s/it]
 24%|██▎       | 53048/225000 [110:56:19<294:46:22,  6.17s/it]
 24%|██▎       | 53049/225000 [110:56:28<338:41:51,  7.09s/it]
 24%|██▎       | 53050/225000 [110:56:33<300:22:37,  6.29s/it]
                                                              
{'loss': '0.71', 'grad_norm': '0.7401', 'learning_rate': '4.522e-05', 'epoch': '0.2358', 'num_input_tokens_seen': 185259824, 'train_runtime': '3.994e+05', 'train_tokens_per_second': '463.9'}

 24%|██▎       | 53050/225000 [110:56:33<300:22:37,  6.29s/it]
 24%|██▎       | 53051/225000 [110:56:39<301:06:47,  6.30s/it]
 24%|██▎       | 53052/225000 [110:56:47<322:28:52,  6.75s/it]
 24%|██▎       | 53053/225000 [110:56:56<354:14:09,  7.42s/it]
 24%|██▎       | 53054/225000 [110:56:59<298:19:28,  6.25s/it]
 24%|██▎       | 53055/225000 [110:57:09<350:24:54,  7.34s/it]
 24%|██▎       | 53056/225000 [110:57:14<307:17:46,  6.43s/it]
 24%|██▎       | 53057/225000 [110:57:18<274:00:15,  5.74s/it]
 24%|██▎       | 53058/225000 [110:57:25<302:12:14,  6.33s/it]
 24%|██▎       | 53059/225000 [110:57:34<333:47:46,  6.99s/it]
 24%|██▎       | 53060/225000 [110:57:41<332:35:25,  6.96s/it]
 24%|██▎       | 53061/225000 [110:57:49<344:12:00,  7.21s/it]
 24%|██▎       | 53062/225000 [110:57:53<302:33:58,  6.34s/it]
 24%|██▎       | 53063/225000 [110:58:01<334:41:05,  7.01s/it]
 24%|██▎       | 53064/225000 [110:58:06<303:50:03,  6.36s/it]
 24%|██▎       | 53065/225000 [110:58:12<290:51:30,  6.09s/it]
 24%|██▎       | 53066/225000 [110:58:17<278:34:57,  5.83s/it]
 24%|██▎       | 53067/225000 [110:58:22<268:17:31,  5.62s/it]
 24%|██▎       | 53068/225000 [110:58:35<372:58:01,  7.81s/it]
 24%|██▎       | 53069/225000 [110:58:41<350:21:05,  7.34s/it]
 24%|██▎       | 53070/225000 [110:58:46<307:51:49,  6.45s/it]
 24%|██▎       | 53071/225000 [110:58:56<365:17:02,  7.65s/it]
 24%|██▎       | 53072/225000 [110:59:00<314:25:52,  6.58s/it]
 24%|██▎       | 53073/225000 [110:59:06<306:54:43,  6.43s/it]
 24%|██▎       | 53074/225000 [110:59:17<363:06:19,  7.60s/it]
 24%|██▎       | 53075/225000 [110:59:21<323:12:07,  6.77s/it]
 24%|██▎       | 53076/225000 [110:59:27<302:28:32,  6.33s/it]
 24%|██▎       | 53077/225000 [110:59:35<324:27:57,  6.79s/it]
 24%|██▎       | 53078/225000 [110:59:44<365:52:03,  7.66s/it]
 24%|██▎       | 53079/225000 [110:59:48<313:38:09,  6.57s/it]
 24%|██▎       | 53080/225000 [110:59:56<327:41:51,  6.86s/it]
 24%|██▎       | 53081/225000 [111:00:06<369:15:04,  7.73s/it]
 24%|██▎       | 53082/225000 [111:00:10<323:40:27,  6.78s/it]
 24%|██▎       | 53083/225000 [111:00:16<303:38:32,  6.36s/it]
 24%|██▎       | 53084/225000 [111:00:21<284:23:01,  5.96s/it]
 24%|██▎       | 53085/225000 [111:00:29<325:28:28,  6.82s/it]
 24%|██▎       | 53086/225000 [111:00:41<389:39:57,  8.16s/it]
 24%|██▎       | 53087/225000 [111:00:45<337:02:28,  7.06s/it]
 24%|██▎       | 53088/225000 [111:00:52<335:27:37,  7.02s/it]
 24%|██▎       | 53089/225000 [111:01:02<380:01:59,  7.96s/it]
 24%|██▎       | 53090/225000 [111:01:06<321:20:21,  6.73s/it]
 24%|██▎       | 53091/225000 [111:01:11<299:45:13,  6.28s/it]
 24%|██▎       | 53092/225000 [111:01:17<294:42:14,  6.17s/it]
 24%|██▎       | 53093/225000 [111:01:30<383:46:28,  8.04s/it]
 24%|██▎       | 53094/225000 [111:01:33<319:31:44,  6.69s/it]
 24%|██▎       | 53095/225000 [111:01:44<371:46:13,  7.79s/it]
 24%|██▎       | 53096/225000 [111:01:48<327:46:03,  6.86s/it]
 24%|██▎       | 53097/225000 [111:01:53<291:40:37,  6.11s/it]
 24%|██▎       | 53098/225000 [111:01:57<272:29:31,  5.71s/it]
 24%|██▎       | 53099/225000 [111:02:02<262:17:38,  5.49s/it]
 24%|██▎       | 53100/225000 [111:02:09<283:38:28,  5.94s/it]
                                                              
{'loss': '0.7142', 'grad_norm': '0.786', 'learning_rate': '4.521e-05', 'epoch': '0.236', 'num_input_tokens_seen': 185411144, 'train_runtime': '3.997e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53100/225000 [111:02:09<283:38:28,  5.94s/it]
 24%|██▎       | 53101/225000 [111:02:23<393:26:51,  8.24s/it]
 24%|██▎       | 53102/225000 [111:02:27<332:36:03,  6.97s/it]
 24%|██▎       | 53103/225000 [111:02:34<338:39:34,  7.09s/it]
 24%|██▎       | 53104/225000 [111:02:39<301:25:30,  6.31s/it]
 24%|██▎       | 53105/225000 [111:02:49<359:26:12,  7.53s/it]
 24%|██▎       | 53106/225000 [111:02:53<306:56:21,  6.43s/it]
 24%|██▎       | 53107/225000 [111:03:02<341:31:18,  7.15s/it]
 24%|██▎       | 53108/225000 [111:03:08<330:54:19,  6.93s/it]
 24%|██▎       | 53109/225000 [111:03:15<331:22:55,  6.94s/it]
 24%|██▎       | 53110/225000 [111:03:20<295:17:54,  6.18s/it]
 24%|██▎       | 53111/225000 [111:03:24<268:24:23,  5.62s/it]
 24%|██▎       | 53112/225000 [111:03:35<345:50:44,  7.24s/it]
 24%|██▎       | 53113/225000 [111:03:40<313:51:16,  6.57s/it]
 24%|██▎       | 53114/225000 [111:03:45<294:37:37,  6.17s/it]
 24%|██▎       | 53115/225000 [111:03:50<279:01:48,  5.84s/it]
 24%|██▎       | 53116/225000 [111:04:04<388:22:01,  8.13s/it]
 24%|██▎       | 53117/225000 [111:04:08<336:16:57,  7.04s/it]
 24%|██▎       | 53118/225000 [111:04:13<300:58:06,  6.30s/it]
 24%|██▎       | 53119/225000 [111:04:19<298:58:35,  6.26s/it]
 24%|██▎       | 53120/225000 [111:04:29<345:17:53,  7.23s/it]
 24%|██▎       | 53121/225000 [111:04:34<315:47:46,  6.61s/it]
 24%|██▎       | 53122/225000 [111:04:39<300:07:05,  6.29s/it]
 24%|██▎       | 53123/225000 [111:04:45<287:47:16,  6.03s/it]
 24%|██▎       | 53124/225000 [111:04:50<273:22:21,  5.73s/it]
 24%|██▎       | 53125/225000 [111:04:59<323:44:46,  6.78s/it]
 24%|██▎       | 53126/225000 [111:05:12<413:05:33,  8.65s/it]
 24%|██▎       | 53127/225000 [111:05:19<384:43:12,  8.06s/it]
 24%|██▎       | 53128/225000 [111:05:23<328:46:02,  6.89s/it]
 24%|██▎       | 53129/225000 [111:05:32<357:14:30,  7.48s/it]
 24%|██▎       | 53130/225000 [111:05:36<309:03:55,  6.47s/it]
 24%|██▎       | 53131/225000 [111:05:45<354:29:00,  7.43s/it]
 24%|██▎       | 53132/225000 [111:05:50<311:06:51,  6.52s/it]
 24%|██▎       | 53133/225000 [111:05:54<282:47:04,  5.92s/it]
 24%|██▎       | 53134/225000 [111:06:05<355:29:03,  7.45s/it]
 24%|██▎       | 53135/225000 [111:06:11<325:25:58,  6.82s/it]
 24%|██▎       | 53136/225000 [111:06:18<330:56:55,  6.93s/it]
 24%|██▎       | 53137/225000 [111:06:26<353:59:53,  7.42s/it]
 24%|██▎       | 53138/225000 [111:06:35<363:01:09,  7.60s/it]
 24%|██▎       | 53139/225000 [111:06:38<307:18:09,  6.44s/it]
 24%|██▎       | 53140/225000 [111:06:43<287:31:26,  6.02s/it]
 24%|██▎       | 53141/225000 [111:06:48<274:53:25,  5.76s/it]
 24%|██▎       | 53142/225000 [111:06:56<303:46:54,  6.36s/it]
 24%|██▎       | 53143/225000 [111:07:07<362:44:30,  7.60s/it]
 24%|██▎       | 53144/225000 [111:07:14<363:53:44,  7.62s/it]
 24%|██▎       | 53145/225000 [111:07:19<314:33:01,  6.59s/it]
 24%|██▎       | 53146/225000 [111:07:23<285:27:05,  5.98s/it]
 24%|██▎       | 53147/225000 [111:07:33<347:05:59,  7.27s/it]
 24%|██▎       | 53148/225000 [111:07:41<345:25:27,  7.24s/it]
 24%|██▎       | 53149/225000 [111:07:48<344:18:07,  7.21s/it]
 24%|██▎       | 53150/225000 [111:07:53<317:04:35,  6.64s/it]
                                                              
{'loss': '0.7088', 'grad_norm': '0.8329', 'learning_rate': '4.52e-05', 'epoch': '0.2362', 'num_input_tokens_seen': 185564472, 'train_runtime': '4.001e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53150/225000 [111:07:53<317:04:35,  6.64s/it]
 24%|██▎       | 53151/225000 [111:07:59<303:25:38,  6.36s/it]
 24%|██▎       | 53152/225000 [111:08:04<289:42:54,  6.07s/it]
 24%|██▎       | 53153/225000 [111:08:10<280:48:25,  5.88s/it]
 24%|██▎       | 53154/225000 [111:08:15<273:42:26,  5.73s/it]
 24%|██▎       | 53155/225000 [111:08:20<270:46:59,  5.67s/it]
 24%|██▎       | 53156/225000 [111:08:31<334:45:03,  7.01s/it]
 24%|██▎       | 53157/225000 [111:08:42<398:32:25,  8.35s/it]
 24%|██▎       | 53158/225000 [111:08:46<341:37:25,  7.16s/it]
 24%|██▎       | 53159/225000 [111:08:51<304:45:31,  6.38s/it]
 24%|██▎       | 53160/225000 [111:08:56<281:25:27,  5.90s/it]
 24%|██▎       | 53161/225000 [111:09:09<381:03:27,  7.98s/it]
 24%|██▎       | 53162/225000 [111:09:14<347:28:01,  7.28s/it]
 24%|██▎       | 53163/225000 [111:09:24<379:13:32,  7.94s/it]
 24%|██▎       | 53164/225000 [111:09:28<322:30:01,  6.76s/it]
 24%|██▎       | 53165/225000 [111:09:32<287:57:37,  6.03s/it]
 24%|██▎       | 53166/225000 [111:09:37<272:13:08,  5.70s/it]
 24%|██▎       | 53167/225000 [111:09:42<264:54:45,  5.55s/it]
 24%|██▎       | 53168/225000 [111:09:52<328:31:15,  6.88s/it]
 24%|██▎       | 53169/225000 [111:10:04<392:10:37,  8.22s/it]
 24%|██▎       | 53170/225000 [111:10:07<331:02:20,  6.94s/it]
 24%|██▎       | 53171/225000 [111:10:12<289:43:31,  6.07s/it]
 24%|██▎       | 53172/225000 [111:10:20<323:35:02,  6.78s/it]
 24%|██▎       | 53173/225000 [111:10:28<338:36:38,  7.09s/it]
 24%|██▎       | 53174/225000 [111:10:32<298:10:09,  6.25s/it]
 24%|██▎       | 53175/225000 [111:10:43<368:00:33,  7.71s/it]
 24%|██▎       | 53176/225000 [111:10:47<314:34:43,  6.59s/it]
 24%|██▎       | 53177/225000 [111:10:52<285:59:02,  5.99s/it]
 24%|██▎       | 53178/225000 [111:10:57<281:59:32,  5.91s/it]
 24%|██▎       | 53179/225000 [111:11:03<280:25:02,  5.88s/it]
 24%|██▎       | 53180/225000 [111:11:11<301:17:45,  6.31s/it]
 24%|██▎       | 53181/225000 [111:11:16<291:08:41,  6.10s/it]
 24%|██▎       | 53182/225000 [111:11:26<347:31:50,  7.28s/it]
 24%|██▎       | 53183/225000 [111:11:31<318:14:17,  6.67s/it]
 24%|██▎       | 53184/225000 [111:11:39<326:53:23,  6.85s/it]
 24%|██▎       | 53185/225000 [111:11:45<311:23:17,  6.52s/it]
 24%|██▎       | 53186/225000 [111:11:56<389:04:59,  8.15s/it]
 24%|██▎       | 53187/225000 [111:12:00<327:37:00,  6.86s/it]
 24%|██▎       | 53188/225000 [111:12:11<380:51:00,  7.98s/it]
 24%|██▎       | 53189/225000 [111:12:15<325:37:08,  6.82s/it]
 24%|██▎       | 53190/225000 [111:12:25<369:47:05,  7.75s/it]
 24%|██▎       | 53191/225000 [111:12:31<343:39:45,  7.20s/it]
 24%|██▎       | 53192/225000 [111:12:36<320:37:30,  6.72s/it]
 24%|██▎       | 53193/225000 [111:12:42<308:55:01,  6.47s/it]
 24%|██▎       | 53194/225000 [111:12:52<357:56:05,  7.50s/it]
 24%|██▎       | 53195/225000 [111:12:56<308:45:04,  6.47s/it]
 24%|██▎       | 53196/225000 [111:13:05<334:36:09,  7.01s/it]
 24%|██▎       | 53197/225000 [111:13:10<317:35:23,  6.65s/it]
 24%|██▎       | 53198/225000 [111:13:16<299:53:34,  6.28s/it]
 24%|██▎       | 53199/225000 [111:13:22<291:31:49,  6.11s/it]
 24%|██▎       | 53200/225000 [111:13:33<363:05:56,  7.61s/it]
                                                              
{'loss': '0.7132', 'grad_norm': '0.8434', 'learning_rate': '4.519e-05', 'epoch': '0.2364', 'num_input_tokens_seen': 185714928, 'train_runtime': '4.004e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53200/225000 [111:13:33<363:05:56,  7.61s/it]
 24%|██▎       | 53201/225000 [111:13:38<326:25:41,  6.84s/it]
 24%|██▎       | 53202/225000 [111:13:46<351:56:12,  7.37s/it]
 24%|██▎       | 53203/225000 [111:13:54<351:09:25,  7.36s/it]
 24%|██▎       | 53204/225000 [111:14:02<362:30:17,  7.60s/it]
 24%|██▎       | 53205/225000 [111:14:05<305:01:01,  6.39s/it]
 24%|██▎       | 53206/225000 [111:14:10<285:57:32,  5.99s/it]
 24%|██▎       | 53207/225000 [111:14:16<278:33:40,  5.84s/it]
 24%|██▎       | 53208/225000 [111:14:22<276:20:29,  5.79s/it]
 24%|██▎       | 53209/225000 [111:14:27<275:35:18,  5.78s/it]
 24%|██▎       | 53210/225000 [111:14:33<269:30:49,  5.65s/it]
 24%|██▎       | 53211/225000 [111:14:40<298:56:42,  6.26s/it]
 24%|██▎       | 53212/225000 [111:14:50<342:50:59,  7.18s/it]
 24%|██▎       | 53213/225000 [111:14:55<315:58:30,  6.62s/it]
 24%|██▎       | 53214/225000 [111:15:01<309:55:22,  6.49s/it]
 24%|██▎       | 53215/225000 [111:15:07<301:59:02,  6.33s/it]
 24%|██▎       | 53216/225000 [111:15:21<404:12:45,  8.47s/it]
 24%|██▎       | 53217/225000 [111:15:28<388:21:13,  8.14s/it]
 24%|██▎       | 53218/225000 [111:15:37<396:04:20,  8.30s/it]
 24%|██▎       | 53219/225000 [111:15:41<344:38:46,  7.22s/it]
 24%|██▎       | 53220/225000 [111:15:49<346:35:03,  7.26s/it]
 24%|██▎       | 53221/225000 [111:15:56<343:46:04,  7.20s/it]
 24%|██▎       | 53222/225000 [111:16:01<310:31:58,  6.51s/it]
 24%|██▎       | 53223/225000 [111:16:10<350:23:30,  7.34s/it]
 24%|██▎       | 53224/225000 [111:16:14<296:01:35,  6.20s/it]
 24%|██▎       | 53225/225000 [111:16:23<339:31:19,  7.12s/it]
 24%|██▎       | 53226/225000 [111:16:27<291:35:24,  6.11s/it]
 24%|██▎       | 53227/225000 [111:16:36<333:25:49,  6.99s/it]
 24%|██▎       | 53228/225000 [111:16:40<294:41:47,  6.18s/it]
 24%|██▎       | 53229/225000 [111:16:48<315:37:10,  6.61s/it]
 24%|██▎       | 53230/225000 [111:16:56<339:33:06,  7.12s/it]
 24%|██▎       | 53231/225000 [111:17:01<310:22:10,  6.50s/it]
 24%|██▎       | 53232/225000 [111:17:06<287:49:24,  6.03s/it]
 24%|██▎       | 53233/225000 [111:17:11<275:52:31,  5.78s/it]
 24%|██▎       | 53234/225000 [111:17:18<289:43:38,  6.07s/it]
 24%|██▎       | 53235/225000 [111:17:30<383:32:12,  8.04s/it]
 24%|██▎       | 53236/225000 [111:17:34<325:54:33,  6.83s/it]
 24%|██▎       | 53237/225000 [111:17:38<285:18:31,  5.98s/it]
 24%|██▎       | 53238/225000 [111:17:47<329:02:59,  6.90s/it]
 24%|██▎       | 53239/225000 [111:17:54<325:51:29,  6.83s/it]
 24%|██▎       | 53240/225000 [111:18:04<371:47:32,  7.79s/it]
 24%|██▎       | 53241/225000 [111:18:10<337:55:25,  7.08s/it]
 24%|██▎       | 53242/225000 [111:18:15<319:30:40,  6.70s/it]
 24%|██▎       | 53243/225000 [111:18:21<304:51:55,  6.39s/it]
 24%|██▎       | 53244/225000 [111:18:27<297:53:51,  6.24s/it]
 24%|██▎       | 53245/225000 [111:18:33<290:12:45,  6.08s/it]
 24%|██▎       | 53246/225000 [111:18:38<285:05:15,  5.98s/it]
 24%|██▎       | 53247/225000 [111:18:45<291:20:00,  6.11s/it]
 24%|██▎       | 53248/225000 [111:18:53<326:37:10,  6.85s/it]
 24%|██▎       | 53249/225000 [111:18:59<310:15:41,  6.50s/it]
 24%|██▎       | 53250/225000 [111:19:05<295:30:03,  6.19s/it]
                                                              
{'loss': '0.7023', 'grad_norm': '0.8177', 'learning_rate': '4.518e-05', 'epoch': '0.2367', 'num_input_tokens_seen': 185865528, 'train_runtime': '4.007e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53250/225000 [111:19:05<295:30:03,  6.19s/it]
 24%|██▎       | 53251/225000 [111:19:11<292:39:50,  6.13s/it]
 24%|██▎       | 53252/225000 [111:19:18<309:56:00,  6.50s/it]
 24%|██▎       | 53253/225000 [111:19:31<408:08:22,  8.56s/it]
 24%|██▎       | 53254/225000 [111:19:36<353:15:34,  7.40s/it]
 24%|██▎       | 53255/225000 [111:19:41<322:24:54,  6.76s/it]
 24%|██▎       | 53256/225000 [111:19:46<296:33:48,  6.22s/it]
 24%|██▎       | 53257/225000 [111:20:00<399:52:21,  8.38s/it]
 24%|██▎       | 53258/225000 [111:20:05<355:11:21,  7.45s/it]
 24%|██▎       | 53259/225000 [111:20:15<387:31:07,  8.12s/it]
 24%|██▎       | 53260/225000 [111:20:18<317:53:47,  6.66s/it]
 24%|██▎       | 53261/225000 [111:20:26<336:59:39,  7.06s/it]
 24%|██▎       | 53262/225000 [111:20:34<352:47:05,  7.40s/it]
 24%|██▎       | 53263/225000 [111:20:39<316:37:40,  6.64s/it]
 24%|██▎       | 53264/225000 [111:20:49<363:31:50,  7.62s/it]
 24%|██▎       | 53265/225000 [111:20:53<316:32:53,  6.64s/it]
 24%|██▎       | 53266/225000 [111:20:58<289:27:32,  6.07s/it]
 24%|██▎       | 53267/225000 [111:21:07<331:27:47,  6.95s/it]
 24%|██▎       | 53268/225000 [111:21:12<303:10:05,  6.36s/it]
 24%|██▎       | 53269/225000 [111:21:17<292:12:35,  6.13s/it]
 24%|██▎       | 53270/225000 [111:21:22<277:15:18,  5.81s/it]
 24%|██▎       | 53271/225000 [111:21:29<282:09:05,  5.91s/it]
 24%|██▎       | 53272/225000 [111:21:42<390:25:49,  8.18s/it]
 24%|██▎       | 53273/225000 [111:21:46<329:14:56,  6.90s/it]
 24%|██▎       | 53274/225000 [111:21:53<333:08:40,  6.98s/it]
 24%|██▎       | 53275/225000 [111:21:59<316:28:10,  6.63s/it]
 24%|██▎       | 53276/225000 [111:22:07<339:00:34,  7.11s/it]
 24%|██▎       | 53277/225000 [111:22:15<344:54:38,  7.23s/it]
 24%|██▎       | 53278/225000 [111:22:19<304:14:09,  6.38s/it]
 24%|██▎       | 53279/225000 [111:22:24<280:41:41,  5.88s/it]
 24%|██▎       | 53280/225000 [111:22:29<273:38:11,  5.74s/it]
 24%|██▎       | 53281/225000 [111:22:35<270:41:57,  5.68s/it]
 24%|██▎       | 53282/225000 [111:22:43<310:40:31,  6.51s/it]
 24%|██▎       | 53283/225000 [111:22:55<381:04:44,  7.99s/it]
 24%|██▎       | 53284/225000 [111:23:02<366:40:57,  7.69s/it]
 24%|██▎       | 53285/225000 [111:23:07<325:49:12,  6.83s/it]
 24%|██▎       | 53286/225000 [111:23:11<297:58:25,  6.25s/it]
 24%|██▎       | 53287/225000 [111:23:23<370:48:45,  7.77s/it]
 24%|██▎       | 53288/225000 [111:23:27<320:41:14,  6.72s/it]
 24%|██▎       | 53289/225000 [111:23:32<295:45:39,  6.20s/it]
 24%|██▎       | 53290/225000 [111:23:36<271:00:07,  5.68s/it]
 24%|██▎       | 53291/225000 [111:23:44<294:31:05,  6.17s/it]
 24%|██▎       | 53292/225000 [111:23:56<374:35:30,  7.85s/it]
 24%|██▎       | 53293/225000 [111:23:59<317:28:34,  6.66s/it]
 24%|██▎       | 53294/225000 [111:24:09<364:31:13,  7.64s/it]
 24%|██▎       | 53295/225000 [111:24:15<340:22:43,  7.14s/it]
 24%|██▎       | 53296/225000 [111:24:21<320:15:44,  6.71s/it]
 24%|██▎       | 53297/225000 [111:24:28<329:12:27,  6.90s/it]
 24%|██▎       | 53298/225000 [111:24:35<331:02:35,  6.94s/it]
 24%|██▎       | 53299/225000 [111:24:42<331:44:14,  6.96s/it]
 24%|██▎       | 53300/225000 [111:24:49<319:53:52,  6.71s/it]
                                                              
{'loss': '0.7096', 'grad_norm': '0.6862', 'learning_rate': '4.517e-05', 'epoch': '0.2369', 'num_input_tokens_seen': 186016624, 'train_runtime': '4.011e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53300/225000 [111:24:49<319:53:52,  6.71s/it]
 24%|██▎       | 53301/225000 [111:24:54<301:34:10,  6.32s/it]
 24%|██▎       | 53302/225000 [111:25:00<294:36:51,  6.18s/it]
 24%|██▎       | 53303/225000 [111:25:05<285:39:53,  5.99s/it]
 24%|██▎       | 53304/225000 [111:25:11<279:01:04,  5.85s/it]
 24%|██▎       | 53305/225000 [111:25:17<278:25:34,  5.84s/it]
 24%|██▎       | 53306/225000 [111:25:30<384:32:57,  8.06s/it]
 24%|██▎       | 53307/225000 [111:25:38<391:00:48,  8.20s/it]
 24%|██▎       | 53308/225000 [111:25:47<393:09:36,  8.24s/it]
 24%|██▎       | 53309/225000 [111:25:52<352:37:49,  7.39s/it]
 24%|██▎       | 53310/225000 [111:26:01<368:45:23,  7.73s/it]
 24%|██▎       | 53311/225000 [111:26:05<315:56:45,  6.62s/it]
 24%|██▎       | 53312/225000 [111:26:09<284:29:14,  5.97s/it]
 24%|██▎       | 53313/225000 [111:26:14<263:48:08,  5.53s/it]
 24%|██▎       | 53314/225000 [111:26:26<362:25:15,  7.60s/it]
 24%|██▎       | 53315/225000 [111:26:31<319:31:37,  6.70s/it]
 24%|██▎       | 53316/225000 [111:26:40<351:50:44,  7.38s/it]
 24%|██▎       | 53317/225000 [111:26:43<299:33:26,  6.28s/it]
 24%|██▎       | 53318/225000 [111:26:55<371:42:20,  7.79s/it]
 24%|██▎       | 53319/225000 [111:26:59<325:03:22,  6.82s/it]
 24%|██▎       | 53320/225000 [111:27:07<331:25:51,  6.95s/it]
 24%|██▎       | 53321/225000 [111:27:15<356:24:33,  7.47s/it]
 24%|██▎       | 53322/225000 [111:27:21<328:08:58,  6.88s/it]
 24%|██▎       | 53323/225000 [111:27:28<329:07:59,  6.90s/it]
 24%|██▎       | 53324/225000 [111:27:34<324:53:04,  6.81s/it]
 24%|██▎       | 53325/225000 [111:27:38<278:06:03,  5.83s/it]
 24%|██▎       | 53326/225000 [111:27:47<330:58:41,  6.94s/it]
 24%|██▎       | 53327/225000 [111:27:52<293:10:37,  6.15s/it]
 24%|██▎       | 53328/225000 [111:27:59<316:52:04,  6.64s/it]
 24%|██▎       | 53329/225000 [111:28:03<274:32:28,  5.76s/it]
 24%|██▎       | 53330/225000 [111:28:14<352:09:17,  7.38s/it]
 24%|██▎       | 53331/225000 [111:28:20<322:40:21,  6.77s/it]
 24%|██▎       | 53332/225000 [111:28:25<304:32:20,  6.39s/it]
 24%|██▎       | 53333/225000 [111:28:36<362:26:21,  7.60s/it]
 24%|██▎       | 53334/225000 [111:28:40<320:13:19,  6.72s/it]
 24%|██▎       | 53335/225000 [111:28:45<289:16:13,  6.07s/it]
 24%|██▎       | 53336/225000 [111:28:50<269:58:57,  5.66s/it]
 24%|██▎       | 53337/225000 [111:28:59<325:33:02,  6.83s/it]
 24%|██▎       | 53338/225000 [111:29:07<339:57:28,  7.13s/it]
 24%|██▎       | 53339/225000 [111:29:14<344:57:20,  7.23s/it]
 24%|██▎       | 53340/225000 [111:29:22<348:47:33,  7.31s/it]
 24%|██▎       | 53341/225000 [111:29:26<296:57:06,  6.23s/it]
 24%|██▎       | 53342/225000 [111:29:30<271:48:18,  5.70s/it]
 24%|██▎       | 53343/225000 [111:29:38<310:55:34,  6.52s/it]
 24%|██▎       | 53344/225000 [111:29:46<327:16:06,  6.86s/it]
 24%|██▎       | 53345/225000 [111:29:55<360:38:20,  7.56s/it]
 24%|██▎       | 53346/225000 [111:30:00<322:18:40,  6.76s/it]
 24%|██▎       | 53347/225000 [111:30:10<361:20:41,  7.58s/it]
 24%|██▎       | 53348/225000 [111:30:14<312:54:57,  6.56s/it]
 24%|██▎       | 53349/225000 [111:30:20<302:59:56,  6.35s/it]
 24%|██▎       | 53350/225000 [111:30:27<319:01:36,  6.69s/it]
                                                              
{'loss': '0.6975', 'grad_norm': '0.7495', 'learning_rate': '4.516e-05', 'epoch': '0.2371', 'num_input_tokens_seen': 186168448, 'train_runtime': '4.014e+05', 'train_tokens_per_second': '463.8'}

 24%|██▎       | 53350/225000 [111:30:27<319:01:36,  6.69s/it]
 24%|██▎       | 53351/225000 [111:30:32<297:45:10,  6.24s/it]
 24%|██▎       | 53352/225000 [111:30:37<274:20:16,  5.75s/it]
 24%|██▎       | 53353/225000 [111:30:43<270:11:15,  5.67s/it]
 24%|██▎       | 53354/225000 [111:30:56<378:04:55,  7.93s/it]
 24%|██▎       | 53355/225000 [111:31:01<341:28:01,  7.16s/it]
 24%|██▎       | 53356/225000 [111:31:05<299:46:37,  6.29s/it]
 24%|██▎       | 53357/225000 [111:31:16<357:07:05,  7.49s/it]
 24%|██▎       | 53358/225000 [111:31:20<315:03:34,  6.61s/it]
 24%|██▎       | 53359/225000 [111:31:26<297:39:00,  6.24s/it]
 24%|██▎       | 53360/225000 [111:31:31<288:43:00,  6.06s/it]
 24%|██▎       | 53361/225000 [111:31:44<386:10:12,  8.10s/it]
 24%|██▎       | 53362/225000 [111:31:49<342:34:43,  7.19s/it]
 24%|██▎       | 53363/225000 [111:31:56<344:37:20,  7.23s/it]
 24%|██▎       | 53364/225000 [111:32:03<328:29:42,  6.89s/it]
 24%|██▎       | 53365/225000 [111:32:08<313:56:22,  6.58s/it]
 24%|██▎       | 53366/225000 [111:32:17<343:52:35,  7.21s/it]
 24%|██▎       | 53367/225000 [111:32:25<349:55:13,  7.34s/it]
 24%|██▎       | 53368/225000 [111:32:30<322:10:18,  6.76s/it]
 24%|██▎       | 53369/225000 [111:32:35<302:13:13,  6.34s/it]
 24%|██▎       | 53370/225000 [111:32:41<288:48:24,  6.06s/it]
 24%|██▎       | 53371/225000 [111:32:46<273:28:39,  5.74s/it]
 24%|██▎       | 53372/225000 [111:32:52<273:21:44,  5.73s/it]
 24%|██▎       | 53373/225000 [111:33:03<356:36:09,  7.48s/it]
 24%|██▎       | 53374/225000 [111:33:13<391:25:53,  8.21s/it]
 24%|██▎       | 53375/225000 [111:33:19<363:53:51,  7.63s/it]
 24%|██▎       | 53376/225000 [111:33:25<336:11:53,  7.05s/it]
 24%|██▎       | 53377/225000 [111:33:33<353:14:39,  7.41s/it]
 24%|██▎       | 53378/225000 [111:33:37<300:41:06,  6.31s/it]
 24%|██▎       | 53379/225000 [111:33:47<346:33:44,  7.27s/it]
 24%|██▎       | 53380/225000 [111:33:52<321:16:23,  6.74s/it]
 24%|██▎       | 53381/225000 [111:33:58<308:10:07,  6.46s/it]
 24%|██▎       | 53382/225000 [111:34:06<334:23:45,  7.01s/it]
 24%|██▎       | 53383/225000 [111:34:11<307:57:14,  6.46s/it]
 24%|██▎       | 53384/225000 [111:34:16<285:01:21,  5.98s/it]
 24%|██▎       | 53385/225000 [111:34:27<360:49:51,  7.57s/it]
 24%|██▎       | 53386/225000 [111:34:32<312:59:30,  6.57s/it]
 24%|██▎       | 53387/225000 [111:34:38<306:50:13,  6.44s/it]
 24%|██▎       | 53388/225000 [111:34:47<346:52:54,  7.28s/it]
 24%|██▎       | 53389/225000 [111:34:54<336:00:41,  7.05s/it]
 24%|██▎       | 53390/225000 [111:35:01<337:05:22,  7.07s/it]
 24%|██▎       | 53391/225000 [111:35:09<359:14:54,  7.54s/it]
 24%|██▎       | 53392/225000 [111:35:14<311:13:16,  6.53s/it]
 24%|██▎       | 53393/225000 [111:35:19<291:12:57,  6.11s/it]
 24%|██▎       | 53394/225000 [111:35:24<282:19:28,  5.92s/it]
 24%|██▎       | 53395/225000 [111:35:36<366:54:34,  7.70s/it]
 24%|██▎       | 53396/225000 [111:35:39<306:16:54,  6.43s/it]
 24%|██▎       | 53397/225000 [111:35:48<341:33:43,  7.17s/it]
 24%|██▎       | 53398/225000 [111:35:53<298:54:39,  6.27s/it]
 24%|██▎       | 53399/225000 [111:35:59<296:32:03,  6.22s/it]
 24%|██▎       | 53400/225000 [111:36:09<358:01:37,  7.51s/it]
                                                              
{'loss': '0.695', 'grad_norm': '0.6786', 'learning_rate': '4.515e-05', 'epoch': '0.2373', 'num_input_tokens_seen': 186319880, 'train_runtime': '4.018e+05', 'train_tokens_per_second': '463.7'}

 24%|██▎       | 53400/225000 [111:36:09<358:01:37,  7.51s/it]
 24%|██▎       | 53401/225000 [111:36:15<333:26:14,  7.00s/it]
 24%|██▎       | 53402/225000 [111:36:21<317:14:38,  6.66s/it]
 24%|██▎       | 53403/225000 [111:36:26<303:37:32,  6.37s/it]
 24%|██▎       | 53404/225000 [111:36:34<315:52:04,  6.63s/it]
 24%|██▎       | 53405/225000 [111:36:43<348:15:16,  7.31s/it]
 24%|██▎       | 53406/225000 [111:36:49<337:01:26,  7.07s/it]
 24%|██▎       | 53407/225000 [111:36:54<311:52:17,  6.54s/it]
 24%|██▎       | 53408/225000 [111:37:00<297:48:27,  6.25s/it]
 24%|██▎       | 53409/225000 [111:37:11<366:03:52,  7.68s/it]
 24%|██▎       | 53410/225000 [111:37:18<355:35:25,  7.46s/it]
 24%|██▎       | 53411/225000 [111:37:22<311:11:35,  6.53s/it]
 24%|██▎       | 53412/225000 [111:37:30<328:47:42,  6.90s/it]
 24%|██▎       | 53413/225000 [111:37:39<360:33:17,  7.56s/it]
 24%|██▎       | 53414/225000 [111:37:45<332:54:41,  6.98s/it]
 24%|██▎       | 53415/225000 [111:37:50<312:12:26,  6.55s/it]
 24%|██▎       | 53416/225000 [111:37:55<289:09:27,  6.07s/it]
 24%|██▎       | 53417/225000 [111:38:01<283:54:33,  5.96s/it]
 24%|██▎       | 53418/225000 [111:38:07<278:56:15,  5.85s/it]
 24%|██▎       | 53419/225000 [111:38:12<269:59:22,  5.66s/it]
 24%|██▎       | 53420/225000 [111:38:21<313:06:17,  6.57s/it]
 24%|██▎       | 53421/225000 [111:38:31<368:46:51,  7.74s/it]
 24%|██▎       | 53422/225000 [111:38:38<354:27:37,  7.44s/it]
 24%|██▎       | 53423/225000 [111:38:44<337:44:35,  7.09s/it]
 24%|██▎       | 53424/225000 [111:38:52<357:57:32,  7.51s/it]
 24%|██▎       | 53425/225000 [111:38:59<349:17:09,  7.33s/it]
 24%|██▎       | 53426/225000 [111:39:05<329:34:12,  6.92s/it]
 24%|██▎       | 53427/225000 [111:39:10<292:26:36,  6.14s/it]
 24%|██▎       | 53428/225000 [111:39:15<287:53:00,  6.04s/it]
 24%|██▎       | 53429/225000 [111:39:26<352:53:38,  7.40s/it]
 24%|██▎       | 53430/225000 [111:39:34<358:35:39,  7.52s/it]
 24%|██▎       | 53431/225000 [111:39:38<310:40:06,  6.52s/it]
 24%|██▎       | 53432/225000 [111:39:43<283:35:21,  5.95s/it]
 24%|██▎       | 53433/225000 [111:39:49<283:51:11,  5.96s/it]
 24%|██▎       | 53434/225000 [111:40:02<383:53:48,  8.06s/it]
 24%|██▎       | 53435/225000 [111:40:07<343:05:43,  7.20s/it]
 24%|██▎       | 53436/225000 [111:40:14<338:33:32,  7.10s/it]
 24%|██▎       | 53437/225000 [111:40:21<342:47:50,  7.19s/it]
 24%|██▍       | 53438/225000 [111:40:28<344:05:53,  7.22s/it]
 24%|██▍       | 53439/225000 [111:40:33<307:21:59,  6.45s/it]
 24%|██▍       | 53440/225000 [111:40:43<360:29:15,  7.56s/it]
 24%|██▍       | 53441/225000 [111:40:50<356:24:52,  7.48s/it]
 24%|██▍       | 53442/225000 [111:40:55<312:30:57,  6.56s/it]
 24%|██▍       | 53443/225000 [111:41:00<288:25:28,  6.05s/it]
 24%|██▍       | 53444/225000 [111:41:10<346:35:36,  7.27s/it]
 24%|██▍       | 53445/225000 [111:41:14<304:35:23,  6.39s/it]
 24%|██▍       | 53446/225000 [111:41:19<282:28:08,  5.93s/it]
 24%|██▍       | 53447/225000 [111:41:31<371:22:05,  7.79s/it]
 24%|██▍       | 53448/225000 [111:41:37<340:09:04,  7.14s/it]
 24%|██▍       | 53449/225000 [111:41:43<330:58:23,  6.95s/it]
 24%|██▍       | 53450/225000 [111:41:48<302:21:18,  6.34s/it]
                                                              
{'loss': '0.7091', 'grad_norm': '0.8358', 'learning_rate': '4.513e-05', 'epoch': '0.2376', 'num_input_tokens_seen': 186472152, 'train_runtime': '4.021e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53450/225000 [111:41:48<302:21:18,  6.34s/it]
 24%|██▍       | 53451/225000 [111:41:55<309:25:42,  6.49s/it]
 24%|██▍       | 53452/225000 [111:42:07<379:57:33,  7.97s/it]
 24%|██▍       | 53453/225000 [111:42:14<368:57:14,  7.74s/it]
 24%|██▍       | 53454/225000 [111:42:18<320:22:27,  6.72s/it]
 24%|██▍       | 53455/225000 [111:42:25<322:41:17,  6.77s/it]
 24%|██▍       | 53456/225000 [111:42:31<311:36:09,  6.54s/it]
 24%|██▍       | 53457/225000 [111:42:38<314:34:34,  6.60s/it]
 24%|██▍       | 53458/225000 [111:42:42<283:35:13,  5.95s/it]
 24%|██▍       | 53459/225000 [111:42:49<294:08:22,  6.17s/it]
 24%|██▍       | 53460/225000 [111:43:00<362:34:57,  7.61s/it]
 24%|██▍       | 53461/225000 [111:43:05<328:43:17,  6.90s/it]
 24%|██▍       | 53462/225000 [111:43:10<308:23:33,  6.47s/it]
 24%|██▍       | 53463/225000 [111:43:16<296:23:13,  6.22s/it]
 24%|██▍       | 53464/225000 [111:43:23<300:28:24,  6.31s/it]
 24%|██▍       | 53465/225000 [111:43:28<292:54:52,  6.15s/it]
 24%|██▍       | 53466/225000 [111:43:34<285:16:58,  5.99s/it]
 24%|██▍       | 53467/225000 [111:43:47<387:00:29,  8.12s/it]
 24%|██▍       | 53468/225000 [111:43:52<337:04:57,  7.07s/it]
 24%|██▍       | 53469/225000 [111:43:56<296:14:41,  6.22s/it]
 24%|██▍       | 53470/225000 [111:44:01<283:59:38,  5.96s/it]
 24%|██▍       | 53471/225000 [111:44:15<392:09:11,  8.23s/it]
 24%|██▍       | 53472/225000 [111:44:20<351:41:41,  7.38s/it]
 24%|██▍       | 53473/225000 [111:44:29<366:56:25,  7.70s/it]
 24%|██▍       | 53474/225000 [111:44:33<320:13:57,  6.72s/it]
 24%|██▍       | 53475/225000 [111:44:42<344:22:40,  7.23s/it]
 24%|██▍       | 53476/225000 [111:44:50<355:07:08,  7.45s/it]
 24%|██▍       | 53477/225000 [111:44:54<314:31:02,  6.60s/it]
 24%|██▍       | 53478/225000 [111:44:59<286:14:06,  6.01s/it]
 24%|██▍       | 53479/225000 [111:45:04<273:27:17,  5.74s/it]
 24%|██▍       | 53480/225000 [111:45:10<273:12:54,  5.73s/it]
 24%|██▍       | 53481/225000 [111:45:15<271:43:26,  5.70s/it]
 24%|██▍       | 53482/225000 [111:45:28<371:46:25,  7.80s/it]
 24%|██▍       | 53483/225000 [111:45:38<404:30:02,  8.49s/it]
 24%|██▍       | 53484/225000 [111:45:45<385:02:06,  8.08s/it]
 24%|██▍       | 53485/225000 [111:45:49<324:11:20,  6.80s/it]
 24%|██▍       | 53486/225000 [111:45:53<288:12:57,  6.05s/it]
 24%|██▍       | 53487/225000 [111:46:06<382:45:51,  8.03s/it]
 24%|██▍       | 53488/225000 [111:46:11<333:58:55,  7.01s/it]
 24%|██▍       | 53489/225000 [111:46:17<330:30:55,  6.94s/it]
 24%|██▍       | 53490/225000 [111:46:24<328:22:27,  6.89s/it]
 24%|██▍       | 53491/225000 [111:46:31<331:21:11,  6.96s/it]
 24%|██▍       | 53492/225000 [111:46:39<343:14:29,  7.20s/it]
 24%|██▍       | 53493/225000 [111:46:43<301:25:16,  6.33s/it]
 24%|██▍       | 53494/225000 [111:46:48<277:43:53,  5.83s/it]
 24%|██▍       | 53495/225000 [111:46:53<261:30:03,  5.49s/it]
 24%|██▍       | 53496/225000 [111:47:01<307:26:23,  6.45s/it]
 24%|██▍       | 53497/225000 [111:47:13<376:51:17,  7.91s/it]
 24%|██▍       | 53498/225000 [111:47:16<314:03:47,  6.59s/it]
 24%|██▍       | 53499/225000 [111:47:25<338:57:26,  7.12s/it]
 24%|██▍       | 53500/225000 [111:47:30<314:49:07,  6.61s/it]
                                                              
{'loss': '0.7279', 'grad_norm': '0.8682', 'learning_rate': '4.512e-05', 'epoch': '0.2378', 'num_input_tokens_seen': 186624592, 'train_runtime': '4.025e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53500/225000 [111:47:30<314:49:07,  6.61s/it]
 24%|██▍       | 53501/225000 [111:47:39<353:21:00,  7.42s/it]
 24%|██▍       | 53502/225000 [111:47:44<316:17:48,  6.64s/it]
 24%|██▍       | 53503/225000 [111:47:49<297:51:25,  6.25s/it]
 24%|██▍       | 53504/225000 [111:47:57<312:25:54,  6.56s/it]
 24%|██▍       | 53505/225000 [111:48:06<355:15:13,  7.46s/it]
 24%|██▍       | 53506/225000 [111:48:12<326:25:31,  6.85s/it]
 24%|██▍       | 53507/225000 [111:48:19<336:10:52,  7.06s/it]
 24%|██▍       | 53508/225000 [111:48:24<297:27:59,  6.24s/it]
 24%|██▍       | 53509/225000 [111:48:28<267:57:32,  5.63s/it]
 24%|██▍       | 53510/225000 [111:48:40<358:59:43,  7.54s/it]
 24%|██▍       | 53511/225000 [111:48:46<341:23:07,  7.17s/it]
 24%|██▍       | 53512/225000 [111:48:56<384:17:05,  8.07s/it]
 24%|██▍       | 53513/225000 [111:49:00<325:05:16,  6.82s/it]
 24%|██▍       | 53514/225000 [111:49:09<349:10:31,  7.33s/it]
 24%|██▍       | 53515/225000 [111:49:13<308:49:03,  6.48s/it]
 24%|██▍       | 53516/225000 [111:49:17<276:30:03,  5.80s/it]
 24%|██▍       | 53517/225000 [111:49:24<287:41:14,  6.04s/it]
 24%|██▍       | 53518/225000 [111:49:35<352:37:27,  7.40s/it]
 24%|██▍       | 53519/225000 [111:49:39<308:26:08,  6.48s/it]
 24%|██▍       | 53520/225000 [111:49:47<337:12:57,  7.08s/it]
 24%|██▍       | 53521/225000 [111:49:53<319:16:30,  6.70s/it]
 24%|██▍       | 53522/225000 [111:49:57<282:40:18,  5.93s/it]
 24%|██▍       | 53523/225000 [111:50:03<274:38:31,  5.77s/it]
 24%|██▍       | 53524/225000 [111:50:14<356:00:30,  7.47s/it]
 24%|██▍       | 53525/225000 [111:50:19<321:24:15,  6.75s/it]
 24%|██▍       | 53526/225000 [111:50:28<349:33:10,  7.34s/it]
 24%|██▍       | 53527/225000 [111:50:33<313:49:57,  6.59s/it]
 24%|██▍       | 53528/225000 [111:50:37<281:01:34,  5.90s/it]
 24%|██▍       | 53529/225000 [111:50:45<303:38:03,  6.37s/it]
 24%|██▍       | 53530/225000 [111:50:56<383:16:00,  8.05s/it]
 24%|██▍       | 53531/225000 [111:51:04<370:04:46,  7.77s/it]
 24%|██▍       | 53532/225000 [111:51:08<319:43:29,  6.71s/it]
 24%|██▍       | 53533/225000 [111:51:17<353:50:44,  7.43s/it]
 24%|██▍       | 53534/225000 [111:51:21<309:09:24,  6.49s/it]
 24%|██▍       | 53535/225000 [111:51:26<290:33:33,  6.10s/it]
 24%|██▍       | 53536/225000 [111:51:38<366:53:05,  7.70s/it]
 24%|██▍       | 53537/225000 [111:51:44<338:52:25,  7.11s/it]
 24%|██▍       | 53538/225000 [111:51:49<319:09:50,  6.70s/it]
 24%|██▍       | 53539/225000 [111:51:55<298:40:00,  6.27s/it]
 24%|██▍       | 53540/225000 [111:52:00<289:44:23,  6.08s/it]
 24%|██▍       | 53541/225000 [111:52:13<387:33:55,  8.14s/it]
 24%|██▍       | 53542/225000 [111:52:18<337:22:22,  7.08s/it]
 24%|██▍       | 53543/225000 [111:52:25<336:04:03,  7.06s/it]
 24%|██▍       | 53544/225000 [111:52:30<311:02:52,  6.53s/it]
 24%|██▍       | 53545/225000 [111:52:35<288:25:18,  6.06s/it]
 24%|██▍       | 53546/225000 [111:52:41<279:33:18,  5.87s/it]
 24%|██▍       | 53547/225000 [111:52:48<297:58:48,  6.26s/it]
 24%|██▍       | 53548/225000 [111:52:58<353:22:18,  7.42s/it]
 24%|██▍       | 53549/225000 [111:53:03<316:33:26,  6.65s/it]
 24%|██▍       | 53550/225000 [111:53:08<294:09:02,  6.18s/it]
                                                              
{'loss': '0.7023', 'grad_norm': '0.7955', 'learning_rate': '4.511e-05', 'epoch': '0.238', 'num_input_tokens_seen': 186776288, 'train_runtime': '4.028e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53550/225000 [111:53:08<294:09:02,  6.18s/it]
 24%|██▍       | 53551/225000 [111:53:18<359:28:38,  7.55s/it]
 24%|██▍       | 53552/225000 [111:53:27<374:46:10,  7.87s/it]
 24%|██▍       | 53553/225000 [111:53:34<362:05:34,  7.60s/it]
 24%|██▍       | 53554/225000 [111:53:40<337:16:57,  7.08s/it]
 24%|██▍       | 53555/225000 [111:53:48<346:34:04,  7.28s/it]
 24%|██▍       | 53556/225000 [111:53:55<349:47:50,  7.35s/it]
 24%|██▍       | 53557/225000 [111:54:02<339:50:39,  7.14s/it]
 24%|██▍       | 53558/225000 [111:54:06<291:07:41,  6.11s/it]
 24%|██▍       | 53559/225000 [111:54:15<339:22:57,  7.13s/it]
 24%|██▍       | 53560/225000 [111:54:20<313:36:09,  6.59s/it]
 24%|██▍       | 53561/225000 [111:54:30<363:24:26,  7.63s/it]
 24%|██▍       | 53562/225000 [111:54:35<321:03:21,  6.74s/it]
 24%|██▍       | 53563/225000 [111:54:40<294:38:53,  6.19s/it]
 24%|██▍       | 53564/225000 [111:54:45<274:04:05,  5.76s/it]
 24%|██▍       | 53565/225000 [111:54:51<286:30:17,  6.02s/it]
 24%|██▍       | 53566/225000 [111:55:04<381:31:45,  8.01s/it]
 24%|██▍       | 53567/225000 [111:55:09<334:19:54,  7.02s/it]
 24%|██▍       | 53568/225000 [111:55:17<351:13:54,  7.38s/it]
 24%|██▍       | 53569/225000 [111:55:24<343:23:23,  7.21s/it]
 24%|██▍       | 53570/225000 [111:55:30<327:29:31,  6.88s/it]
 24%|██▍       | 53571/225000 [111:55:38<342:53:57,  7.20s/it]
 24%|██▍       | 53572/225000 [111:55:43<307:51:28,  6.47s/it]
 24%|██▍       | 53573/225000 [111:55:48<295:37:51,  6.21s/it]
 24%|██▍       | 53574/225000 [111:55:54<288:50:23,  6.07s/it]
 24%|██▍       | 53575/225000 [111:56:03<335:18:31,  7.04s/it]
 24%|██▍       | 53576/225000 [111:56:07<283:42:50,  5.96s/it]
 24%|██▍       | 53577/225000 [111:56:12<273:44:18,  5.75s/it]
 24%|██▍       | 53578/225000 [111:56:17<264:51:23,  5.56s/it]
 24%|██▍       | 53579/225000 [111:56:22<259:43:43,  5.45s/it]
 24%|██▍       | 53580/225000 [111:56:32<314:15:57,  6.60s/it]
 24%|██▍       | 53581/225000 [111:56:45<405:36:30,  8.52s/it]
 24%|██▍       | 53582/225000 [111:56:48<332:39:49,  6.99s/it]
 24%|██▍       | 53583/225000 [111:56:57<362:38:05,  7.62s/it]
 24%|██▍       | 53584/225000 [111:57:05<367:42:48,  7.72s/it]
 24%|██▍       | 53585/225000 [111:57:09<320:32:41,  6.73s/it]
 24%|██▍       | 53586/225000 [111:57:14<290:40:02,  6.10s/it]
 24%|██▍       | 53587/225000 [111:57:23<336:40:07,  7.07s/it]
 24%|██▍       | 53588/225000 [111:57:28<301:57:34,  6.34s/it]
 24%|██▍       | 53589/225000 [111:57:33<283:12:07,  5.95s/it]
 24%|██▍       | 53590/225000 [111:57:46<376:31:39,  7.91s/it]
 24%|██▍       | 53591/225000 [111:57:49<320:14:10,  6.73s/it]
 24%|██▍       | 53592/225000 [111:57:59<353:04:05,  7.42s/it]
 24%|██▍       | 53593/225000 [111:58:06<347:13:52,  7.29s/it]
 24%|██▍       | 53594/225000 [111:58:10<313:17:14,  6.58s/it]
 24%|██▍       | 53595/225000 [111:58:20<352:46:45,  7.41s/it]
 24%|██▍       | 53596/225000 [111:58:25<320:31:31,  6.73s/it]
 24%|██▍       | 53597/225000 [111:58:30<289:36:15,  6.08s/it]
 24%|██▍       | 53598/225000 [111:58:34<270:58:42,  5.69s/it]
 24%|██▍       | 53599/225000 [111:58:47<378:10:37,  7.94s/it]
 24%|██▍       | 53600/225000 [111:58:54<353:40:18,  7.43s/it]
                                                              
{'loss': '0.705', 'grad_norm': '0.9042', 'learning_rate': '4.51e-05', 'epoch': '0.2382', 'num_input_tokens_seen': 186927824, 'train_runtime': '4.031e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53600/225000 [111:58:54<353:40:18,  7.43s/it]
 24%|██▍       | 53601/225000 [111:59:00<343:36:35,  7.22s/it]
 24%|██▍       | 53602/225000 [111:59:06<319:48:43,  6.72s/it]
 24%|██▍       | 53603/225000 [111:59:16<363:31:46,  7.64s/it]
 24%|██▍       | 53604/225000 [111:59:20<317:09:03,  6.66s/it]
 24%|██▍       | 53605/225000 [111:59:28<336:58:26,  7.08s/it]
 24%|██▍       | 53606/225000 [111:59:36<348:43:48,  7.32s/it]
 24%|██▍       | 53607/225000 [111:59:43<348:26:08,  7.32s/it]
 24%|██▍       | 53608/225000 [111:59:49<329:37:52,  6.92s/it]
 24%|██▍       | 53609/225000 [111:59:56<325:01:54,  6.83s/it]
 24%|██▍       | 53610/225000 [112:00:01<302:19:35,  6.35s/it]
 24%|██▍       | 53611/225000 [112:00:07<292:30:04,  6.14s/it]
 24%|██▍       | 53612/225000 [112:00:12<280:33:51,  5.89s/it]
 24%|██▍       | 53613/225000 [112:00:17<271:13:43,  5.70s/it]
 24%|██▍       | 53614/225000 [112:00:23<268:53:57,  5.65s/it]
 24%|██▍       | 53615/225000 [112:00:34<348:10:07,  7.31s/it]
 24%|██▍       | 53616/225000 [112:00:42<357:27:39,  7.51s/it]
 24%|██▍       | 53617/225000 [112:00:50<359:50:09,  7.56s/it]
 24%|██▍       | 53618/225000 [112:00:55<327:47:04,  6.89s/it]
 24%|██▍       | 53619/225000 [112:01:04<352:57:56,  7.41s/it]
 24%|██▍       | 53620/225000 [112:01:10<332:04:06,  6.98s/it]
 24%|██▍       | 53621/225000 [112:01:14<291:41:50,  6.13s/it]
 24%|██▍       | 53622/225000 [112:01:26<382:45:20,  8.04s/it]
 24%|██▍       | 53623/225000 [112:01:30<324:56:46,  6.83s/it]
 24%|██▍       | 53624/225000 [112:01:36<306:13:40,  6.43s/it]
 24%|██▍       | 53625/225000 [112:01:41<290:14:01,  6.10s/it]
 24%|██▍       | 53626/225000 [112:01:47<281:07:19,  5.91s/it]
 24%|██▍       | 53627/225000 [112:01:52<276:39:41,  5.81s/it]
 24%|██▍       | 53628/225000 [112:01:58<270:15:32,  5.68s/it]
 24%|██▍       | 53629/225000 [112:02:06<307:01:11,  6.45s/it]
 24%|██▍       | 53630/225000 [112:02:19<397:10:21,  8.34s/it]
 24%|██▍       | 53631/225000 [112:02:23<342:34:37,  7.20s/it]
 24%|██▍       | 53632/225000 [112:02:32<366:33:30,  7.70s/it]
 24%|██▍       | 53633/225000 [112:02:39<362:32:42,  7.62s/it]
 24%|██▍       | 53634/225000 [112:02:45<337:55:30,  7.10s/it]
 24%|██▍       | 53635/225000 [112:02:52<327:39:10,  6.88s/it]
 24%|██▍       | 53636/225000 [112:02:57<305:49:43,  6.42s/it]
 24%|██▍       | 53637/225000 [112:03:03<293:47:14,  6.17s/it]
 24%|██▍       | 53638/225000 [112:03:08<284:49:32,  5.98s/it]
 24%|██▍       | 53639/225000 [112:03:14<276:38:03,  5.81s/it]
 24%|██▍       | 53640/225000 [112:03:27<383:45:20,  8.06s/it]
 24%|██▍       | 53641/225000 [112:03:32<347:50:37,  7.31s/it]
 24%|██▍       | 53642/225000 [112:03:39<335:38:56,  7.05s/it]
 24%|██▍       | 53643/225000 [112:03:48<358:00:29,  7.52s/it]
 24%|██▍       | 53644/225000 [112:03:53<323:11:11,  6.79s/it]
 24%|██▍       | 53645/225000 [112:04:04<386:55:55,  8.13s/it]
 24%|██▍       | 53646/225000 [112:04:09<350:00:45,  7.35s/it]
 24%|██▍       | 53647/225000 [112:04:15<323:36:04,  6.80s/it]
 24%|██▍       | 53648/225000 [112:04:20<305:04:11,  6.41s/it]
 24%|██▍       | 53649/225000 [112:04:26<290:55:36,  6.11s/it]
 24%|██▍       | 53650/225000 [112:04:32<285:36:39,  6.00s/it]
                                                              
{'loss': '0.7081', 'grad_norm': '0.8537', 'learning_rate': '4.509e-05', 'epoch': '0.2384', 'num_input_tokens_seen': 187080968, 'train_runtime': '4.035e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53650/225000 [112:04:32<285:36:39,  6.00s/it]
 24%|██▍       | 53651/225000 [112:04:38<292:17:04,  6.14s/it]
 24%|██▍       | 53652/225000 [112:04:51<391:17:39,  8.22s/it]
 24%|██▍       | 53653/225000 [112:04:57<361:45:35,  7.60s/it]
 24%|██▍       | 53654/225000 [112:05:05<367:41:23,  7.73s/it]
 24%|██▍       | 53655/225000 [112:05:09<314:16:41,  6.60s/it]
 24%|██▍       | 53656/225000 [112:05:19<354:31:04,  7.45s/it]
 24%|██▍       | 53657/225000 [112:05:24<324:28:44,  6.82s/it]
 24%|██▍       | 53658/225000 [112:05:28<287:07:32,  6.03s/it]
 24%|██▍       | 53659/225000 [112:05:35<290:47:30,  6.11s/it]
 24%|██▍       | 53660/225000 [112:05:45<351:36:33,  7.39s/it]
 24%|██▍       | 53661/225000 [112:05:50<311:45:20,  6.55s/it]
 24%|██▍       | 53662/225000 [112:05:54<288:42:44,  6.07s/it]
 24%|██▍       | 53663/225000 [112:06:04<340:33:36,  7.16s/it]
 24%|██▍       | 53664/225000 [112:06:12<353:53:31,  7.44s/it]
 24%|██▍       | 53665/225000 [112:06:19<344:29:48,  7.24s/it]
 24%|██▍       | 53666/225000 [112:06:26<342:38:34,  7.20s/it]
 24%|██▍       | 53667/225000 [112:06:31<307:19:08,  6.46s/it]
 24%|██▍       | 53668/225000 [112:06:42<375:13:02,  7.88s/it]
 24%|██▍       | 53669/225000 [112:06:46<323:44:52,  6.80s/it]
 24%|██▍       | 53670/225000 [112:06:56<358:36:45,  7.54s/it]
 24%|██▍       | 53671/225000 [112:07:00<311:52:41,  6.55s/it]
 24%|██▍       | 53672/225000 [112:07:09<355:37:41,  7.47s/it]
 24%|██▍       | 53673/225000 [112:07:15<321:45:47,  6.76s/it]
 24%|██▍       | 53674/225000 [112:07:19<293:25:06,  6.17s/it]
 24%|██▍       | 53675/225000 [112:07:25<280:05:58,  5.89s/it]
 24%|██▍       | 53676/225000 [112:07:37<377:55:03,  7.94s/it]
 24%|██▍       | 53677/225000 [112:07:42<330:21:56,  6.94s/it]
 24%|██▍       | 53678/225000 [112:07:46<285:57:31,  6.01s/it]
 24%|██▍       | 53679/225000 [112:07:58<370:45:38,  7.79s/it]
 24%|██▍       | 53680/225000 [112:08:03<339:11:03,  7.13s/it]
 24%|██▍       | 53681/225000 [112:08:09<314:16:11,  6.60s/it]
 24%|██▍       | 53682/225000 [112:08:14<290:59:36,  6.11s/it]
 24%|██▍       | 53683/225000 [112:08:19<276:52:01,  5.82s/it]
 24%|██▍       | 53684/225000 [112:08:25<283:52:27,  5.97s/it]
 24%|██▍       | 53685/225000 [112:08:31<277:13:48,  5.83s/it]
 24%|██▍       | 53686/225000 [112:08:36<277:58:08,  5.84s/it]
 24%|██▍       | 53687/225000 [112:08:48<353:59:46,  7.44s/it]
 24%|██▍       | 53688/225000 [112:08:57<378:13:07,  7.95s/it]
 24%|██▍       | 53689/225000 [112:09:05<378:23:15,  7.95s/it]
 24%|██▍       | 53690/225000 [112:09:09<324:21:18,  6.82s/it]
 24%|██▍       | 53691/225000 [112:09:20<380:51:23,  8.00s/it]
 24%|██▍       | 53692/225000 [112:09:26<349:55:47,  7.35s/it]
 24%|██▍       | 53693/225000 [112:09:30<303:20:25,  6.37s/it]
 24%|██▍       | 53694/225000 [112:09:40<365:05:16,  7.67s/it]
 24%|██▍       | 53695/225000 [112:09:49<381:50:29,  8.02s/it]
 24%|██▍       | 53696/225000 [112:09:53<324:34:04,  6.82s/it]
 24%|██▍       | 53697/225000 [112:09:57<285:34:44,  6.00s/it]
 24%|██▍       | 53698/225000 [112:10:08<355:23:04,  7.47s/it]
 24%|██▍       | 53699/225000 [112:10:12<306:24:45,  6.44s/it]
 24%|██▍       | 53700/225000 [112:10:17<285:33:25,  6.00s/it]
                                                              
{'loss': '0.7091', 'grad_norm': '0.7606', 'learning_rate': '4.508e-05', 'epoch': '0.2387', 'num_input_tokens_seen': 187233360, 'train_runtime': '4.038e+05', 'train_tokens_per_second': '463.7'}

 24%|██▍       | 53700/225000 [112:10:17<285:33:25,  6.00s/it]
 24%|██▍       | 53701/225000 [112:10:27<342:01:08,  7.19s/it]
 24%|██▍       | 53702/225000 [112:10:33<320:33:11,  6.74s/it]
 24%|██▍       | 53703/225000 [112:10:37<284:35:43,  5.98s/it]
 24%|██▍       | 53704/225000 [112:10:46<332:03:37,  6.98s/it]
 24%|██▍       | 53705/225000 [112:10:51<302:19:45,  6.35s/it]
 24%|██▍       | 53706/225000 [112:10:56<279:11:10,  5.87s/it]
 24%|██▍       | 53707/225000 [112:11:06<345:27:13,  7.26s/it]
 24%|██▍       | 53708/225000 [112:11:13<340:40:52,  7.16s/it]
 24%|██▍       | 53709/225000 [112:11:18<311:00:50,  6.54s/it]
 24%|██▍       | 53710/225000 [112:11:24<293:24:20,  6.17s/it]
 24%|██▍       | 53711/225000 [112:11:29<282:48:06,  5.94s/it]
 24%|██▍       | 53712/225000 [112:11:35<280:36:23,  5.90s/it]
 24%|██▍       | 53713/225000 [112:11:48<377:32:01,  7.93s/it]
 24%|██▍       | 53714/225000 [112:11:55<366:26:07,  7.70s/it]
 24%|██▍       | 53715/225000 [112:12:01<337:38:56,  7.10s/it]
 24%|██▍       | 53716/225000 [112:12:06<311:45:16,  6.55s/it]
 24%|██▍       | 53717/225000 [112:12:11<294:23:09,  6.19s/it]
 24%|██▍       | 53718/225000 [112:12:16<277:47:20,  5.84s/it]
 24%|██▍       | 53719/225000 [112:12:30<385:16:45,  8.10s/it]
 24%|██▍       | 53720/225000 [112:12:35<352:22:25,  7.41s/it]
 24%|██▍       | 53721/225000 [112:12:40<314:08:21,  6.60s/it]
 24%|██▍       | 53722/225000 [112:12:45<296:15:39,  6.23s/it]
 24%|██▍       | 53723/225000 [112:12:50<272:53:46,  5.74s/it]
 24%|██▍       | 53724/225000 [112:13:03<380:33:07,  8.00s/it]
 24%|██▍       | 53725/225000 [112:13:10<356:44:13,  7.50s/it]
 24%|██▍       | 53726/225000 [112:13:15<325:46:24,  6.85s/it]
 24%|██▍       | 53727/225000 [112:13:20<305:27:39,  6.42s/it]
 24%|██▍       | 53728/225000 [112:13:26<293:32:20,  6.17s/it]
 24%|██▍       | 53729/225000 [112:13:32<287:06:27,  6.03s/it]
 24%|██▍       | 53730/225000 [112:13:41<329:06:06,  6.92s/it]
 24%|██▍       | 53731/225000 [112:13:53<406:48:50,  8.55s/it]
 24%|██▍       | 53732/225000 [112:13:57<343:10:37,  7.21s/it]
 24%|██▍       | 53733/225000 [112:14:05<354:14:56,  7.45s/it]
 24%|██▍       | 53734/225000 [112:14:09<304:36:23,  6.40s/it]
 24%|██▍       | 53735/225000 [112:14:18<338:16:33,  7.11s/it]
 24%|██▍       | 53736/225000 [112:14:22<299:00:09,  6.29s/it]
 24%|██▍       | 53737/225000 [112:14:32<350:18:35,  7.36s/it]
 24%|██▍       | 53738/225000 [112:14:37<318:20:51,  6.69s/it]
 24%|██▍       | 53739/225000 [112:14:42<294:04:16,  6.18s/it]
 24%|██▍       | 53740/225000 [112:14:47<276:23:48,  5.81s/it]
 24%|██▍       | 53741/225000 [112:15:00<372:30:51,  7.83s/it]
 24%|██▍       | 53742/225000 [112:15:05<342:02:28,  7.19s/it]
 24%|██▍       | 53743/225000 [112:15:10<299:41:41,  6.30s/it]
 24%|██▍       | 53744/225000 [112:15:14<275:33:05,  5.79s/it]
 24%|██▍       | 53745/225000 [112:15:23<312:35:46,  6.57s/it]
 24%|██▍       | 53746/225000 [112:15:32<356:14:25,  7.49s/it]
 24%|██▍       | 53747/225000 [112:15:39<340:52:06,  7.17s/it]
 24%|██▍       | 53748/225000 [112:15:46<339:33:03,  7.14s/it]
 24%|██▍       | 53749/225000 [112:15:52<331:23:44,  6.97s/it]
 24%|██▍       | 53750/225000 [112:15:58<311:56:47,  6.56s/it]
                                                              
{'loss': '0.6973', 'grad_norm': '0.7095', 'learning_rate': '4.507e-05', 'epoch': '0.2389', 'num_input_tokens_seen': 187384672, 'train_runtime': '4.042e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 53750/225000 [112:15:58<311:56:47,  6.56s/it]
 24%|██▍       | 53751/225000 [112:16:03<289:49:54,  6.09s/it]
 24%|██▍       | 53752/225000 [112:16:08<276:22:04,  5.81s/it]
 24%|██▍       | 53753/225000 [112:16:16<314:11:36,  6.61s/it]
 24%|██▍       | 53754/225000 [112:16:25<346:28:57,  7.28s/it]
 24%|██▍       | 53755/225000 [112:16:32<342:59:04,  7.21s/it]
 24%|██▍       | 53756/225000 [112:16:38<319:10:13,  6.71s/it]
 24%|██▍       | 53757/225000 [112:16:44<304:45:35,  6.41s/it]
 24%|██▍       | 53758/225000 [112:16:49<289:49:39,  6.09s/it]
 24%|██▍       | 53759/225000 [112:16:55<283:01:05,  5.95s/it]
 24%|██▍       | 53760/225000 [112:17:05<340:18:05,  7.15s/it]
 24%|██▍       | 53761/225000 [112:17:12<343:02:56,  7.21s/it]
 24%|██▍       | 53762/225000 [112:17:17<318:19:05,  6.69s/it]
 24%|██▍       | 53763/225000 [112:17:23<296:01:02,  6.22s/it]
 24%|██▍       | 53764/225000 [112:17:28<282:37:08,  5.94s/it]
 24%|██▍       | 53765/225000 [112:17:35<296:57:08,  6.24s/it]
 24%|██▍       | 53766/225000 [112:17:48<395:31:19,  8.32s/it]
 24%|██▍       | 53767/225000 [112:17:54<358:01:28,  7.53s/it]
 24%|██▍       | 53768/225000 [112:17:58<313:19:03,  6.59s/it]
 24%|██▍       | 53769/225000 [112:18:04<299:47:38,  6.30s/it]
 24%|██▍       | 53770/225000 [112:18:16<392:30:24,  8.25s/it]
 24%|██▍       | 53771/225000 [112:18:21<338:49:32,  7.12s/it]
 24%|██▍       | 53772/225000 [112:18:27<330:58:20,  6.96s/it]
 24%|██▍       | 53773/225000 [112:18:37<361:53:55,  7.61s/it]
 24%|██▍       | 53774/225000 [112:18:43<345:37:05,  7.27s/it]
 24%|██▍       | 53775/225000 [112:18:48<313:28:15,  6.59s/it]
 24%|██▍       | 53776/225000 [112:18:53<290:07:20,  6.10s/it]
 24%|██▍       | 53777/225000 [112:18:58<276:57:30,  5.82s/it]
 24%|██▍       | 53778/225000 [112:19:07<316:22:05,  6.65s/it]
 24%|██▍       | 53779/225000 [112:19:16<353:11:19,  7.43s/it]
 24%|██▍       | 53780/225000 [112:19:20<304:32:48,  6.40s/it]
 24%|██▍       | 53781/225000 [112:19:27<307:33:49,  6.47s/it]
 24%|██▍       | 53782/225000 [112:19:37<367:04:57,  7.72s/it]
 24%|██▍       | 53783/225000 [112:19:42<325:18:43,  6.84s/it]
 24%|██▍       | 53784/225000 [112:19:47<296:21:23,  6.23s/it]
 24%|██▍       | 53785/225000 [112:19:52<280:21:34,  5.89s/it]
 24%|██▍       | 53786/225000 [112:19:58<285:28:16,  6.00s/it]
 24%|██▍       | 53787/225000 [112:20:11<379:11:34,  7.97s/it]
 24%|██▍       | 53788/225000 [112:20:15<329:56:34,  6.94s/it]
 24%|██▍       | 53789/225000 [112:20:22<327:44:28,  6.89s/it]
 24%|██▍       | 53790/225000 [112:20:29<331:56:31,  6.98s/it]
 24%|██▍       | 53791/225000 [112:20:33<288:44:24,  6.07s/it]
 24%|██▍       | 53792/225000 [112:20:43<339:04:37,  7.13s/it]
 24%|██▍       | 53793/225000 [112:20:47<296:35:15,  6.24s/it]
 24%|██▍       | 53794/225000 [112:20:52<271:51:31,  5.72s/it]
 24%|██▍       | 53795/225000 [112:21:01<318:55:32,  6.71s/it]
 24%|██▍       | 53796/225000 [112:21:08<334:49:53,  7.04s/it]
 24%|██▍       | 53797/225000 [112:21:16<344:21:41,  7.24s/it]
 24%|██▍       | 53798/225000 [112:21:24<356:08:11,  7.49s/it]
 24%|██▍       | 53799/225000 [112:21:31<340:29:08,  7.16s/it]
 24%|██▍       | 53800/225000 [112:21:36<313:58:44,  6.60s/it]
                                                              
{'loss': '0.721', 'grad_norm': '0.7556', 'learning_rate': '4.506e-05', 'epoch': '0.2391', 'num_input_tokens_seen': 187536456, 'train_runtime': '4.045e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 53800/225000 [112:21:36<313:58:44,  6.60s/it]
 24%|██▍       | 53801/225000 [112:21:41<296:11:57,  6.23s/it]
 24%|██▍       | 53802/225000 [112:21:46<282:30:07,  5.94s/it]
 24%|██▍       | 53803/225000 [112:21:52<277:09:47,  5.83s/it]
 24%|██▍       | 53804/225000 [112:22:05<380:24:47,  8.00s/it]
 24%|██▍       | 53805/225000 [112:22:10<342:31:40,  7.20s/it]
 24%|██▍       | 53806/225000 [112:22:15<306:11:43,  6.44s/it]
 24%|██▍       | 53807/225000 [112:22:22<316:15:58,  6.65s/it]
 24%|██▍       | 53808/225000 [112:22:35<402:15:04,  8.46s/it]
 24%|██▍       | 53809/225000 [112:22:39<332:45:18,  7.00s/it]
 24%|██▍       | 53810/225000 [112:22:47<347:43:10,  7.31s/it]
 24%|██▍       | 53811/225000 [112:22:53<329:20:59,  6.93s/it]
 24%|██▍       | 53812/225000 [112:23:02<357:52:52,  7.53s/it]
 24%|██▍       | 53813/225000 [112:23:06<311:34:49,  6.55s/it]
 24%|██▍       | 53814/225000 [112:23:11<298:30:00,  6.28s/it]
 24%|██▍       | 53815/225000 [112:23:17<294:37:02,  6.20s/it]
 24%|██▍       | 53816/225000 [112:23:23<290:10:35,  6.10s/it]
 24%|██▍       | 53817/225000 [112:23:29<288:58:29,  6.08s/it]
 24%|██▍       | 53818/225000 [112:23:35<285:14:40,  6.00s/it]
 24%|██▍       | 53819/225000 [112:23:45<340:22:32,  7.16s/it]
 24%|██▍       | 53820/225000 [112:23:58<425:47:21,  8.95s/it]
 24%|██▍       | 53821/225000 [112:24:02<356:20:17,  7.49s/it]
 24%|██▍       | 53822/225000 [112:24:07<312:54:01,  6.58s/it]
 24%|██▍       | 53823/225000 [112:24:18<381:08:47,  8.02s/it]
 24%|██▍       | 53824/225000 [112:24:24<349:25:05,  7.35s/it]
 24%|██▍       | 53825/225000 [112:24:31<344:48:21,  7.25s/it]
 24%|██▍       | 53826/225000 [112:24:39<362:47:24,  7.63s/it]
 24%|██▍       | 53827/225000 [112:24:44<316:57:48,  6.67s/it]
 24%|██▍       | 53828/225000 [112:24:49<293:31:39,  6.17s/it]
 24%|██▍       | 53829/225000 [112:25:01<383:20:35,  8.06s/it]
 24%|██▍       | 53830/225000 [112:25:05<325:14:05,  6.84s/it]
 24%|██▍       | 53831/225000 [112:25:09<287:13:36,  6.04s/it]
 24%|██▍       | 53832/225000 [112:25:19<336:18:41,  7.07s/it]
 24%|██▍       | 53833/225000 [112:25:27<345:57:58,  7.28s/it]
 24%|██▍       | 53834/225000 [112:25:33<337:20:53,  7.10s/it]
 24%|██▍       | 53835/225000 [112:25:42<356:03:49,  7.49s/it]
 24%|██▍       | 53836/225000 [112:25:48<333:19:26,  7.01s/it]
 24%|██▍       | 53837/225000 [112:25:52<300:46:10,  6.33s/it]
 24%|██▍       | 53838/225000 [112:25:58<292:16:51,  6.15s/it]
 24%|██▍       | 53839/225000 [112:26:02<266:41:20,  5.61s/it]
 24%|██▍       | 53840/225000 [112:26:16<374:18:42,  7.87s/it]
 24%|██▍       | 53841/225000 [112:26:21<341:05:32,  7.17s/it]
 24%|██▍       | 53842/225000 [112:26:26<307:52:36,  6.48s/it]
 24%|██▍       | 53843/225000 [112:26:31<286:45:22,  6.03s/it]
 24%|██▍       | 53844/225000 [112:26:37<279:48:24,  5.89s/it]
 24%|██▍       | 53845/225000 [112:26:46<328:03:01,  6.90s/it]
 24%|██▍       | 53846/225000 [112:26:56<369:28:44,  7.77s/it]
 24%|██▍       | 53847/225000 [112:27:00<323:45:45,  6.81s/it]
 24%|██▍       | 53848/225000 [112:27:04<287:22:23,  6.04s/it]
 24%|██▍       | 53849/225000 [112:27:10<274:00:22,  5.76s/it]
 24%|██▍       | 53850/225000 [112:27:22<371:24:07,  7.81s/it]
                                                              
{'loss': '0.7006', 'grad_norm': '0.8868', 'learning_rate': '4.505e-05', 'epoch': '0.2393', 'num_input_tokens_seen': 187688744, 'train_runtime': '4.048e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 53850/225000 [112:27:22<371:24:07,  7.81s/it]
 24%|██▍       | 53851/225000 [112:27:26<320:39:16,  6.74s/it]
 24%|██▍       | 53852/225000 [112:27:35<351:32:04,  7.39s/it]
 24%|██▍       | 53853/225000 [112:27:40<311:20:35,  6.55s/it]
 24%|██▍       | 53854/225000 [112:27:45<287:56:29,  6.06s/it]
 24%|██▍       | 53855/225000 [112:27:53<319:00:57,  6.71s/it]
 24%|██▍       | 53856/225000 [112:28:04<381:31:01,  8.03s/it]
 24%|██▍       | 53857/225000 [112:28:09<332:40:56,  7.00s/it]
 24%|██▍       | 53858/225000 [112:28:14<303:44:12,  6.39s/it]
 24%|██▍       | 53859/225000 [112:28:19<284:05:11,  5.98s/it]
 24%|██▍       | 53860/225000 [112:28:23<264:09:06,  5.56s/it]
 24%|██▍       | 53861/225000 [112:28:33<327:37:35,  6.89s/it]
 24%|██▍       | 53862/225000 [112:28:44<380:16:40,  8.00s/it]
 24%|██▍       | 53863/225000 [112:28:49<342:31:57,  7.21s/it]
 24%|██▍       | 53864/225000 [112:28:54<306:23:41,  6.45s/it]
 24%|██▍       | 53865/225000 [112:28:59<288:07:06,  6.06s/it]
 24%|██▍       | 53866/225000 [112:29:05<279:20:56,  5.88s/it]
 24%|██▍       | 53867/225000 [112:29:12<304:08:22,  6.40s/it]
 24%|██▍       | 53868/225000 [112:29:25<402:47:50,  8.47s/it]
 24%|██▍       | 53869/225000 [112:29:30<340:25:19,  7.16s/it]
 24%|██▍       | 53870/225000 [112:29:34<301:15:03,  6.34s/it]
 24%|██▍       | 53871/225000 [112:29:46<375:33:13,  7.90s/it]
 24%|██▍       | 53872/225000 [112:29:50<320:18:53,  6.74s/it]
 24%|██▍       | 53873/225000 [112:29:59<357:11:27,  7.51s/it]
 24%|██▍       | 53874/225000 [112:30:03<302:52:33,  6.37s/it]
 24%|██▍       | 53875/225000 [112:30:09<305:42:09,  6.43s/it]
 24%|██▍       | 53876/225000 [112:30:19<352:49:08,  7.42s/it]
 24%|██▍       | 53877/225000 [112:30:23<304:55:37,  6.41s/it]
 24%|██▍       | 53878/225000 [112:30:31<333:49:34,  7.02s/it]
 24%|██▍       | 53879/225000 [112:30:37<315:24:17,  6.64s/it]
 24%|██▍       | 53880/225000 [112:30:42<286:38:11,  6.03s/it]
 24%|██▍       | 53881/225000 [112:30:54<370:52:29,  7.80s/it]
 24%|██▍       | 53882/225000 [112:30:58<319:31:51,  6.72s/it]
 24%|██▍       | 53883/225000 [112:31:08<374:24:32,  7.88s/it]
 24%|██▍       | 53884/225000 [112:31:15<355:40:21,  7.48s/it]
 24%|██▍       | 53885/225000 [112:31:18<297:02:22,  6.25s/it]
 24%|██▍       | 53886/225000 [112:31:25<295:24:53,  6.22s/it]
 24%|██▍       | 53887/225000 [112:31:29<263:45:36,  5.55s/it]
 24%|██▍       | 53888/225000 [112:31:41<362:10:53,  7.62s/it]
 24%|██▍       | 53889/225000 [112:31:46<329:16:09,  6.93s/it]
 24%|██▍       | 53890/225000 [112:31:53<331:56:57,  6.98s/it]
 24%|██▍       | 53891/225000 [112:32:00<333:06:17,  7.01s/it]
 24%|██▍       | 53892/225000 [112:32:08<341:42:00,  7.19s/it]
 24%|██▍       | 53893/225000 [112:32:12<297:25:02,  6.26s/it]
 24%|██▍       | 53894/225000 [112:32:17<273:06:27,  5.75s/it]
 24%|██▍       | 53895/225000 [112:32:25<311:37:58,  6.56s/it]
 24%|██▍       | 53896/225000 [112:32:34<343:51:55,  7.23s/it]
 24%|██▍       | 53897/225000 [112:32:41<336:54:31,  7.09s/it]
 24%|██▍       | 53898/225000 [112:32:47<320:11:27,  6.74s/it]
 24%|██▍       | 53899/225000 [112:32:52<304:55:20,  6.42s/it]
 24%|██▍       | 53900/225000 [112:33:01<337:32:08,  7.10s/it]
                                                              
{'loss': '0.697', 'grad_norm': '0.8795', 'learning_rate': '4.504e-05', 'epoch': '0.2396', 'num_input_tokens_seen': 187841104, 'train_runtime': '4.052e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 53900/225000 [112:33:01<337:32:08,  7.10s/it]
 24%|██▍       | 53901/225000 [112:33:09<345:20:08,  7.27s/it]
 24%|██▍       | 53902/225000 [112:33:15<338:53:09,  7.13s/it]
 24%|██▍       | 53903/225000 [112:33:19<294:21:21,  6.19s/it]
 24%|██▍       | 53904/225000 [112:33:28<321:20:08,  6.76s/it]
 24%|██▍       | 53905/225000 [112:33:33<297:36:35,  6.26s/it]
 24%|██▍       | 53906/225000 [112:33:38<288:00:16,  6.06s/it]
 24%|██▍       | 53907/225000 [112:33:44<284:43:37,  5.99s/it]
 24%|██▍       | 53908/225000 [112:33:49<274:10:32,  5.77s/it]
 24%|██▍       | 53909/225000 [112:33:55<274:17:29,  5.77s/it]
 24%|██▍       | 53910/225000 [112:34:01<273:07:00,  5.75s/it]
 24%|██▍       | 53911/225000 [112:34:12<357:13:32,  7.52s/it]
 24%|██▍       | 53912/225000 [112:34:21<377:19:21,  7.94s/it]
 24%|██▍       | 53913/225000 [112:34:29<366:19:37,  7.71s/it]
 24%|██▍       | 53914/225000 [112:34:33<324:57:36,  6.84s/it]
 24%|██▍       | 53915/225000 [112:34:42<344:17:46,  7.24s/it]
 24%|██▍       | 53916/225000 [112:34:46<301:27:03,  6.34s/it]
 24%|██▍       | 53917/225000 [112:34:56<357:48:50,  7.53s/it]
 24%|██▍       | 53918/225000 [112:35:00<307:17:43,  6.47s/it]
 24%|██▍       | 53919/225000 [112:35:04<277:25:05,  5.84s/it]
 24%|██▍       | 53920/225000 [112:35:15<343:40:00,  7.23s/it]
 24%|██▍       | 53921/225000 [112:35:22<348:12:37,  7.33s/it]
 24%|██▍       | 53922/225000 [112:35:26<301:09:48,  6.34s/it]
 24%|██▍       | 53923/225000 [112:35:36<342:03:33,  7.20s/it]
 24%|██▍       | 53924/225000 [112:35:44<359:54:08,  7.57s/it]
 24%|██▍       | 53925/225000 [112:35:48<302:03:10,  6.36s/it]
 24%|██▍       | 53926/225000 [112:35:52<278:54:46,  5.87s/it]
 24%|██▍       | 53927/225000 [112:35:57<262:52:54,  5.53s/it]
 24%|██▍       | 53928/225000 [112:36:02<255:56:16,  5.39s/it]
 24%|██▍       | 53929/225000 [112:36:12<315:33:35,  6.64s/it]
 24%|██▍       | 53930/225000 [112:36:21<352:59:38,  7.43s/it]
 24%|██▍       | 53931/225000 [112:36:26<315:14:23,  6.63s/it]
 24%|██▍       | 53932/225000 [112:36:37<379:30:50,  7.99s/it]
 24%|██▍       | 53933/225000 [112:36:43<352:20:32,  7.41s/it]
 24%|██▍       | 53934/225000 [112:36:52<369:25:26,  7.77s/it]
 24%|██▍       | 53935/225000 [112:36:58<342:29:01,  7.21s/it]
 24%|██▍       | 53936/225000 [112:37:07<369:12:16,  7.77s/it]
 24%|██▍       | 53937/225000 [112:37:13<343:00:01,  7.22s/it]
 24%|██▍       | 53938/225000 [112:37:18<322:57:06,  6.80s/it]
 24%|██▍       | 53939/225000 [112:37:23<296:49:08,  6.25s/it]
 24%|██▍       | 53940/225000 [112:37:28<279:13:10,  5.88s/it]
 24%|██▍       | 53941/225000 [112:37:33<268:47:32,  5.66s/it]
 24%|██▍       | 53942/225000 [112:37:39<264:14:33,  5.56s/it]
 24%|██▍       | 53943/225000 [112:37:52<373:38:39,  7.86s/it]
 24%|██▍       | 53944/225000 [112:37:58<348:41:56,  7.34s/it]
 24%|██▍       | 53945/225000 [112:38:04<325:21:47,  6.85s/it]
 24%|██▍       | 53946/225000 [112:38:09<305:34:39,  6.43s/it]
 24%|██▍       | 53947/225000 [112:38:15<290:35:18,  6.12s/it]
 24%|██▍       | 53948/225000 [112:38:20<284:04:15,  5.98s/it]
 24%|██▍       | 53949/225000 [112:38:26<281:20:39,  5.92s/it]
 24%|██▍       | 53950/225000 [112:38:32<278:17:02,  5.86s/it]
                                                              
{'loss': '0.7126', 'grad_norm': '0.6768', 'learning_rate': '4.503e-05', 'epoch': '0.2398', 'num_input_tokens_seen': 187992880, 'train_runtime': '4.055e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 53950/225000 [112:38:32<278:17:02,  5.86s/it]
 24%|██▍       | 53951/225000 [112:38:45<388:27:21,  8.18s/it]
 24%|██▍       | 53952/225000 [112:38:51<351:31:05,  7.40s/it]
 24%|██▍       | 53953/225000 [112:38:56<314:23:30,  6.62s/it]
 24%|██▍       | 53954/225000 [112:39:09<405:26:17,  8.53s/it]
 24%|██▍       | 53955/225000 [112:39:12<332:09:30,  6.99s/it]
 24%|██▍       | 53956/225000 [112:39:21<361:24:55,  7.61s/it]
 24%|██▍       | 53957/225000 [112:39:25<304:23:44,  6.41s/it]
 24%|██▍       | 53958/225000 [112:39:34<336:37:11,  7.08s/it]
 24%|██▍       | 53959/225000 [112:39:37<287:25:50,  6.05s/it]
 24%|██▍       | 53960/225000 [112:39:46<327:23:54,  6.89s/it]
 24%|██▍       | 53961/225000 [112:39:55<353:42:48,  7.44s/it]
 24%|██▍       | 53962/225000 [112:39:58<299:04:22,  6.29s/it]
 24%|██▍       | 53963/225000 [112:40:07<331:46:06,  6.98s/it]
 24%|██▍       | 53964/225000 [112:40:11<287:32:18,  6.05s/it]
 24%|██▍       | 53965/225000 [112:40:15<264:46:26,  5.57s/it]
 24%|██▍       | 53966/225000 [112:40:27<354:21:36,  7.46s/it]
 24%|██▍       | 53967/225000 [112:40:31<303:59:37,  6.40s/it]
 24%|██▍       | 53968/225000 [112:40:36<280:44:56,  5.91s/it]
 24%|██▍       | 53969/225000 [112:40:42<279:27:45,  5.88s/it]
 24%|██▍       | 53970/225000 [112:40:46<264:12:46,  5.56s/it]
 24%|██▍       | 53971/225000 [112:40:59<368:57:42,  7.77s/it]
 24%|██▍       | 53972/225000 [112:41:04<327:26:08,  6.89s/it]
 24%|██▍       | 53973/225000 [112:41:11<325:07:21,  6.84s/it]
 24%|██▍       | 53974/225000 [112:41:18<333:15:08,  7.01s/it]
 24%|██▍       | 53975/225000 [112:41:23<293:03:19,  6.17s/it]
 24%|██▍       | 53976/225000 [112:41:32<339:43:41,  7.15s/it]
 24%|██▍       | 53977/225000 [112:41:38<329:42:12,  6.94s/it]
 24%|██▍       | 53978/225000 [112:41:44<307:34:51,  6.47s/it]
 24%|██▍       | 53979/225000 [112:41:50<296:21:19,  6.24s/it]
 24%|██▍       | 53980/225000 [112:41:55<281:02:02,  5.92s/it]
 24%|██▍       | 53981/225000 [112:42:00<276:20:02,  5.82s/it]
 24%|██▍       | 53982/225000 [112:42:09<322:46:05,  6.79s/it]
 24%|██▍       | 53983/225000 [112:42:19<369:53:18,  7.79s/it]
 24%|██▍       | 53984/225000 [112:42:25<331:57:24,  6.99s/it]
 24%|██▍       | 53985/225000 [112:42:29<296:43:28,  6.25s/it]
 24%|██▍       | 53986/225000 [112:42:36<312:00:51,  6.57s/it]
 24%|██▍       | 53987/225000 [112:42:47<368:58:31,  7.77s/it]
 24%|██▍       | 53988/225000 [112:42:52<326:46:00,  6.88s/it]
 24%|██▍       | 53989/225000 [112:42:57<299:54:38,  6.31s/it]
 24%|██▍       | 53990/225000 [112:43:02<279:47:38,  5.89s/it]
 24%|██▍       | 53991/225000 [112:43:12<344:46:13,  7.26s/it]
 24%|██▍       | 53992/225000 [112:43:18<330:17:41,  6.95s/it]
 24%|██▍       | 53993/225000 [112:43:26<336:43:37,  7.09s/it]
 24%|██▍       | 53994/225000 [112:43:30<299:01:19,  6.29s/it]
 24%|██▍       | 53995/225000 [112:43:38<317:41:10,  6.69s/it]
 24%|██▍       | 53996/225000 [112:43:42<283:49:40,  5.98s/it]
 24%|██▍       | 53997/225000 [112:43:50<314:56:19,  6.63s/it]
 24%|██▍       | 53998/225000 [112:43:58<335:31:36,  7.06s/it]
 24%|██▍       | 53999/225000 [112:44:03<305:10:45,  6.42s/it]
 24%|██▍       | 54000/225000 [112:44:08<287:06:03,  6.04s/it]
                                                              
{'loss': '0.7208', 'grad_norm': '0.7251', 'learning_rate': '4.501e-05', 'epoch': '0.24', 'num_input_tokens_seen': 188144544, 'train_runtime': '4.058e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 54000/225000 [112:44:08<287:06:03,  6.04s/it]
 24%|██▍       | 54001/225000 [112:44:13<269:59:08,  5.68s/it]
 24%|██▍       | 54002/225000 [112:44:19<268:39:48,  5.66s/it]
 24%|██▍       | 54003/225000 [112:44:24<260:35:53,  5.49s/it]
 24%|██▍       | 54004/225000 [112:44:32<299:33:14,  6.31s/it]
 24%|██▍       | 54005/225000 [112:44:45<387:05:30,  8.15s/it]
 24%|██▍       | 54006/225000 [112:44:50<341:11:17,  7.18s/it]
 24%|██▍       | 54007/225000 [112:44:54<305:46:41,  6.44s/it]
 24%|██▍       | 54008/225000 [112:44:59<280:30:02,  5.91s/it]
 24%|██▍       | 54009/225000 [112:45:05<278:37:58,  5.87s/it]
 24%|██▍       | 54010/225000 [112:45:11<279:21:37,  5.88s/it]
 24%|██▍       | 54011/225000 [112:45:23<371:16:55,  7.82s/it]
 24%|██▍       | 54012/225000 [112:45:32<390:54:31,  8.23s/it]
 24%|██▍       | 54013/225000 [112:45:38<351:21:28,  7.40s/it]
 24%|██▍       | 54014/225000 [112:45:42<314:23:06,  6.62s/it]
 24%|██▍       | 54015/225000 [112:45:49<313:59:10,  6.61s/it]
 24%|██▍       | 54016/225000 [112:46:01<391:25:40,  8.24s/it]
 24%|██▍       | 54017/225000 [112:46:05<335:40:09,  7.07s/it]
 24%|██▍       | 54018/225000 [112:46:13<338:35:19,  7.13s/it]
 24%|██▍       | 54019/225000 [112:46:17<295:22:46,  6.22s/it]
 24%|██▍       | 54020/225000 [112:46:27<345:31:43,  7.28s/it]
 24%|██▍       | 54021/225000 [112:46:34<342:52:15,  7.22s/it]
 24%|██▍       | 54022/225000 [112:46:40<326:51:13,  6.88s/it]
 24%|██▍       | 54023/225000 [112:46:45<304:11:11,  6.40s/it]
 24%|██▍       | 54024/225000 [112:46:53<322:04:04,  6.78s/it]
 24%|██▍       | 54025/225000 [112:46:56<278:55:07,  5.87s/it]
 24%|██▍       | 54026/225000 [112:47:07<348:57:35,  7.35s/it]
 24%|██▍       | 54027/225000 [112:47:12<315:59:39,  6.65s/it]
 24%|██▍       | 54028/225000 [112:47:19<317:51:14,  6.69s/it]
 24%|██▍       | 54029/225000 [112:47:25<301:35:51,  6.35s/it]
 24%|██▍       | 54030/225000 [112:47:31<297:03:11,  6.25s/it]
 24%|██▍       | 54031/225000 [112:47:37<295:04:24,  6.21s/it]
 24%|██▍       | 54032/225000 [112:47:43<292:58:40,  6.17s/it]
 24%|██▍       | 54033/225000 [112:47:51<321:34:40,  6.77s/it]
 24%|██▍       | 54034/225000 [112:47:57<304:34:57,  6.41s/it]
 24%|██▍       | 54035/225000 [112:48:03<303:12:52,  6.38s/it]
 24%|██▍       | 54036/225000 [112:48:09<298:06:20,  6.28s/it]
 24%|██▍       | 54037/225000 [112:48:14<286:31:06,  6.03s/it]
 24%|██▍       | 54038/225000 [112:48:28<399:39:46,  8.42s/it]
 24%|██▍       | 54039/225000 [112:48:36<382:54:16,  8.06s/it]
 24%|██▍       | 54040/225000 [112:48:41<347:32:01,  7.32s/it]
 24%|██▍       | 54041/225000 [112:48:50<370:45:41,  7.81s/it]
 24%|██▍       | 54042/225000 [112:48:55<324:11:32,  6.83s/it]
 24%|██▍       | 54043/225000 [112:48:59<291:37:01,  6.14s/it]
 24%|██▍       | 54044/225000 [112:49:08<328:12:23,  6.91s/it]
 24%|██▍       | 54045/225000 [112:49:17<353:20:18,  7.44s/it]
 24%|██▍       | 54046/225000 [112:49:24<353:56:13,  7.45s/it]
 24%|██▍       | 54047/225000 [112:49:29<312:59:20,  6.59s/it]
 24%|██▍       | 54048/225000 [112:49:34<296:14:16,  6.24s/it]
 24%|██▍       | 54049/225000 [112:49:46<373:49:59,  7.87s/it]
 24%|██▍       | 54050/225000 [112:49:52<345:21:54,  7.27s/it]
                                                              
{'loss': '0.7028', 'grad_norm': '0.8186', 'learning_rate': '4.5e-05', 'epoch': '0.2402', 'num_input_tokens_seen': 188296880, 'train_runtime': '4.062e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 54050/225000 [112:49:52<345:21:54,  7.27s/it]
 24%|██▍       | 54051/225000 [112:49:57<316:27:05,  6.66s/it]
 24%|██▍       | 54052/225000 [112:50:01<281:10:28,  5.92s/it]
 24%|██▍       | 54053/225000 [112:50:09<317:43:00,  6.69s/it]
 24%|██▍       | 54054/225000 [112:50:18<339:38:15,  7.15s/it]
 24%|██▍       | 54055/225000 [112:50:26<361:58:59,  7.62s/it]
 24%|██▍       | 54056/225000 [112:50:30<304:21:40,  6.41s/it]
 24%|██▍       | 54057/225000 [112:50:38<325:58:12,  6.86s/it]
 24%|██▍       | 54058/225000 [112:50:46<337:18:01,  7.10s/it]
 24%|██▍       | 54059/225000 [112:50:50<301:34:45,  6.35s/it]
 24%|██▍       | 54060/225000 [112:50:55<272:30:05,  5.74s/it]
 24%|██▍       | 54061/225000 [112:51:03<317:14:18,  6.68s/it]
 24%|██▍       | 54062/225000 [112:51:12<340:11:53,  7.16s/it]
 24%|██▍       | 54063/225000 [112:51:17<312:58:05,  6.59s/it]
 24%|██▍       | 54064/225000 [112:51:22<296:20:25,  6.24s/it]
 24%|██▍       | 54065/225000 [112:51:28<286:06:41,  6.03s/it]
 24%|██▍       | 54066/225000 [112:51:33<275:29:04,  5.80s/it]
 24%|██▍       | 54067/225000 [112:51:39<269:54:19,  5.68s/it]
 24%|██▍       | 54068/225000 [112:51:44<265:54:12,  5.60s/it]
 24%|██▍       | 54069/225000 [112:51:57<368:11:33,  7.75s/it]
 24%|██▍       | 54070/225000 [112:52:04<366:19:15,  7.72s/it]
 24%|██▍       | 54071/225000 [112:52:13<377:56:12,  7.96s/it]
 24%|██▍       | 54072/225000 [112:52:17<324:34:21,  6.84s/it]
 24%|██▍       | 54073/225000 [112:52:21<288:30:22,  6.08s/it]
 24%|██▍       | 54074/225000 [112:52:32<353:57:51,  7.46s/it]
 24%|██▍       | 54075/225000 [112:52:38<327:06:08,  6.89s/it]
 24%|██▍       | 54076/225000 [112:52:45<334:23:45,  7.04s/it]
 24%|██▍       | 54077/225000 [112:52:50<307:26:43,  6.48s/it]
 24%|██▍       | 54078/225000 [112:52:55<283:41:48,  5.98s/it]
 24%|██▍       | 54079/225000 [112:53:01<279:16:05,  5.88s/it]
 24%|██▍       | 54080/225000 [112:53:09<309:33:20,  6.52s/it]
 24%|██▍       | 54081/225000 [112:53:19<369:17:44,  7.78s/it]
 24%|██▍       | 54082/225000 [112:53:24<316:58:30,  6.68s/it]
 24%|██▍       | 54083/225000 [112:53:33<357:22:42,  7.53s/it]
 24%|██▍       | 54084/225000 [112:53:39<331:32:24,  6.98s/it]
 24%|██▍       | 54085/225000 [112:53:43<295:33:37,  6.23s/it]
 24%|██▍       | 54086/225000 [112:53:49<293:36:40,  6.18s/it]
 24%|██▍       | 54087/225000 [112:54:01<368:56:37,  7.77s/it]
 24%|██▍       | 54088/225000 [112:54:06<329:51:13,  6.95s/it]
 24%|██▍       | 54089/225000 [112:54:11<301:02:42,  6.34s/it]
 24%|██▍       | 54090/225000 [112:54:16<281:24:42,  5.93s/it]
 24%|██▍       | 54091/225000 [112:54:21<272:38:59,  5.74s/it]
 24%|██▍       | 54092/225000 [112:54:29<310:37:45,  6.54s/it]
 24%|██▍       | 54093/225000 [112:54:39<354:19:09,  7.46s/it]
 24%|██▍       | 54094/225000 [112:54:44<312:41:26,  6.59s/it]
 24%|██▍       | 54095/225000 [112:54:51<322:09:39,  6.79s/it]
 24%|██▍       | 54096/225000 [112:55:00<351:15:00,  7.40s/it]
 24%|██▍       | 54097/225000 [112:55:05<322:44:06,  6.80s/it]
 24%|██▍       | 54098/225000 [112:55:11<304:53:30,  6.42s/it]
 24%|██▍       | 54099/225000 [112:55:16<289:39:27,  6.10s/it]
 24%|██▍       | 54100/225000 [112:55:21<275:13:47,  5.80s/it]
                                                              
{'loss': '0.7011', 'grad_norm': '0.7772', 'learning_rate': '4.499e-05', 'epoch': '0.2404', 'num_input_tokens_seen': 188450208, 'train_runtime': '4.065e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 54100/225000 [112:55:21<275:13:47,  5.80s/it]
 24%|██▍       | 54101/225000 [112:55:26<267:33:42,  5.64s/it]
 24%|██▍       | 54102/225000 [112:55:32<267:27:59,  5.63s/it]
 24%|██▍       | 54103/225000 [112:55:39<292:27:18,  6.16s/it]
 24%|██▍       | 54104/225000 [112:55:53<403:21:29,  8.50s/it]
 24%|██▍       | 54105/225000 [112:55:58<346:09:13,  7.29s/it]
 24%|██▍       | 54106/225000 [112:56:05<345:21:06,  7.28s/it]
 24%|██▍       | 54107/225000 [112:56:11<320:29:53,  6.75s/it]
 24%|██▍       | 54108/225000 [112:56:19<342:41:52,  7.22s/it]
 24%|██▍       | 54109/225000 [112:56:25<332:13:58,  7.00s/it]
 24%|██▍       | 54110/225000 [112:56:32<334:28:29,  7.05s/it]
 24%|██▍       | 54111/225000 [112:56:36<289:47:43,  6.10s/it]
 24%|██▍       | 54112/225000 [112:56:47<350:39:30,  7.39s/it]
 24%|██▍       | 54113/225000 [112:56:51<306:34:50,  6.46s/it]
 24%|██▍       | 54114/225000 [112:56:57<297:29:29,  6.27s/it]
 24%|██▍       | 54115/225000 [112:57:06<340:08:59,  7.17s/it]
 24%|██▍       | 54116/225000 [112:57:13<333:24:25,  7.02s/it]
 24%|██▍       | 54117/225000 [112:57:17<289:56:48,  6.11s/it]
 24%|██▍       | 54118/225000 [112:57:26<340:18:55,  7.17s/it]
 24%|██▍       | 54119/225000 [112:57:30<289:46:38,  6.10s/it]
 24%|██▍       | 54120/225000 [112:57:41<352:57:30,  7.44s/it]
 24%|██▍       | 54121/225000 [112:57:46<316:59:07,  6.68s/it]
 24%|██▍       | 54122/225000 [112:57:50<291:15:56,  6.14s/it]
 24%|██▍       | 54123/225000 [112:57:58<311:38:52,  6.57s/it]
 24%|██▍       | 54124/225000 [112:58:09<373:52:36,  7.88s/it]
 24%|██▍       | 54125/225000 [112:58:14<327:52:24,  6.91s/it]
 24%|██▍       | 54126/225000 [112:58:18<297:16:10,  6.26s/it]
 24%|██▍       | 54127/225000 [112:58:29<356:37:27,  7.51s/it]
 24%|██▍       | 54128/225000 [112:58:35<332:07:22,  7.00s/it]
 24%|██▍       | 54129/225000 [112:58:40<309:34:13,  6.52s/it]
 24%|██▍       | 54130/225000 [112:58:46<304:35:32,  6.42s/it]
 24%|██▍       | 54131/225000 [112:58:52<291:26:59,  6.14s/it]
 24%|██▍       | 54132/225000 [112:59:04<384:40:45,  8.10s/it]
 24%|██▍       | 54133/225000 [112:59:08<323:45:29,  6.82s/it]
 24%|██▍       | 54134/225000 [112:59:13<297:05:42,  6.26s/it]
 24%|██▍       | 54135/225000 [112:59:20<313:29:53,  6.61s/it]
 24%|██▍       | 54136/225000 [112:59:30<358:39:35,  7.56s/it]
 24%|██▍       | 54137/225000 [112:59:35<315:21:03,  6.64s/it]
 24%|██▍       | 54138/225000 [112:59:43<343:22:07,  7.23s/it]
 24%|██▍       | 54139/225000 [112:59:47<289:39:50,  6.10s/it]
 24%|██▍       | 54140/225000 [112:59:56<326:18:41,  6.88s/it]
 24%|██▍       | 54141/225000 [113:00:01<309:33:56,  6.52s/it]
 24%|██▍       | 54142/225000 [113:00:09<333:58:30,  7.04s/it]
 24%|██▍       | 54143/225000 [113:00:14<296:42:32,  6.25s/it]
 24%|██▍       | 54144/225000 [113:00:19<279:49:08,  5.90s/it]
 24%|██▍       | 54145/225000 [113:00:24<273:53:21,  5.77s/it]
 24%|██▍       | 54146/225000 [113:00:31<285:32:47,  6.02s/it]
 24%|██▍       | 54147/225000 [113:00:45<399:43:34,  8.42s/it]
 24%|██▍       | 54148/225000 [113:00:49<335:21:29,  7.07s/it]
 24%|██▍       | 54149/225000 [113:00:58<365:05:12,  7.69s/it]
 24%|██▍       | 54150/225000 [113:01:02<311:36:03,  6.57s/it]
                                                              
{'loss': '0.7006', 'grad_norm': '0.7455', 'learning_rate': '4.498e-05', 'epoch': '0.2407', 'num_input_tokens_seen': 188603264, 'train_runtime': '4.069e+05', 'train_tokens_per_second': '463.6'}

 24%|██▍       | 54150/225000 [113:01:02<311:36:03,  6.57s/it]
 24%|██▍       | 54151/225000 [113:01:07<289:35:49,  6.10s/it]
 24%|██▍       | 54152/225000 [113:01:20<387:40:38,  8.17s/it]
 24%|██▍       | 54153/225000 [113:01:26<357:06:45,  7.52s/it]
 24%|██▍       | 54154/225000 [113:01:31<316:10:09,  6.66s/it]
 24%|██▍       | 54155/225000 [113:01:35<289:15:01,  6.10s/it]
 24%|██▍       | 54156/225000 [113:01:48<386:06:49,  8.14s/it]
 24%|██▍       | 54157/225000 [113:01:53<329:12:16,  6.94s/it]
 24%|██▍       | 54158/225000 [113:01:58<309:25:58,  6.52s/it]
 24%|██▍       | 54159/225000 [113:02:04<295:35:43,  6.23s/it]
 24%|██▍       | 54160/225000 [113:02:09<282:20:25,  5.95s/it]
 24%|██▍       | 54161/225000 [113:02:15<287:40:51,  6.06s/it]
 24%|██▍       | 54162/225000 [113:02:21<277:16:11,  5.84s/it]
 24%|██▍       | 54163/225000 [113:02:26<268:35:21,  5.66s/it]
 24%|██▍       | 54164/225000 [113:02:32<274:53:07,  5.79s/it]
 24%|██▍       | 54165/225000 [113:02:42<338:15:57,  7.13s/it]
 24%|██▍       | 54166/225000 [113:02:55<417:07:39,  8.79s/it]
 24%|██▍       | 54167/225000 [113:03:00<360:43:42,  7.60s/it]
 24%|██▍       | 54168/225000 [113:03:07<351:01:31,  7.40s/it]
 24%|██▍       | 54169/225000 [113:03:15<367:37:47,  7.75s/it]
 24%|██▍       | 54170/225000 [113:03:23<364:03:14,  7.67s/it]
 24%|██▍       | 54171/225000 [113:03:29<339:01:13,  7.14s/it]
 24%|██▍       | 54172/225000 [113:03:33<305:07:50,  6.43s/it]
 24%|██▍       | 54173/225000 [113:03:38<277:04:11,  5.84s/it]
 24%|██▍       | 54174/225000 [113:03:49<350:18:19,  7.38s/it]
 24%|██▍       | 54175/225000 [113:03:53<300:27:36,  6.33s/it]
 24%|██▍       | 54176/225000 [113:04:01<326:37:49,  6.88s/it]
 24%|██▍       | 54177/225000 [113:04:09<340:19:49,  7.17s/it]
 24%|██▍       | 54178/225000 [113:04:16<338:56:48,  7.14s/it]
 24%|██▍       | 54179/225000 [113:04:20<295:22:27,  6.22s/it]
 24%|██▍       | 54180/225000 [113:04:27<313:56:11,  6.62s/it]
 24%|██▍       | 54181/225000 [113:04:37<356:24:47,  7.51s/it]
 24%|██▍       | 54182/225000 [113:04:42<325:06:41,  6.85s/it]
 24%|██▍       | 54183/225000 [113:04:48<303:07:27,  6.39s/it]
 24%|██▍       | 54184/225000 [113:04:53<294:24:19,  6.20s/it]
 24%|██▍       | 54185/225000 [113:04:59<288:54:31,  6.09s/it]
 24%|██▍       | 54186/225000 [113:05:05<278:37:35,  5.87s/it]
 24%|██▍       | 54187/225000 [113:05:10<272:07:21,  5.74s/it]
 24%|██▍       | 54188/225000 [113:05:16<277:13:04,  5.84s/it]
 24%|██▍       | 54189/225000 [113:05:22<278:56:14,  5.88s/it]
 24%|██▍       | 54190/225000 [113:05:34<372:00:34,  7.84s/it]
 24%|██▍       | 54191/225000 [113:05:40<339:02:28,  7.15s/it]
 24%|██▍       | 54192/225000 [113:05:51<392:32:27,  8.27s/it]
 24%|██▍       | 54193/225000 [113:05:55<336:46:36,  7.10s/it]
 24%|██▍       | 54194/225000 [113:06:02<337:02:07,  7.10s/it]
 24%|██▍       | 54195/225000 [113:06:07<299:03:28,  6.30s/it]
 24%|██▍       | 54196/225000 [113:06:14<310:10:17,  6.54s/it]
 24%|██▍       | 54197/225000 [113:06:25<371:31:39,  7.83s/it]
 24%|██▍       | 54198/225000 [113:06:30<340:37:22,  7.18s/it]
 24%|██▍       | 54199/225000 [113:06:35<310:40:40,  6.55s/it]
 24%|██▍       | 54200/225000 [113:06:44<344:20:07,  7.26s/it]
                                                              
{'loss': '0.7032', 'grad_norm': '0.748', 'learning_rate': '4.497e-05', 'epoch': '0.2409', 'num_input_tokens_seen': 188756336, 'train_runtime': '4.072e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54200/225000 [113:06:44<344:20:07,  7.26s/it]
 24%|██▍       | 54201/225000 [113:06:48<300:27:21,  6.33s/it]
 24%|██▍       | 54202/225000 [113:06:53<275:57:30,  5.82s/it]
 24%|██▍       | 54203/225000 [113:07:04<349:16:05,  7.36s/it]
 24%|██▍       | 54204/225000 [113:07:08<298:25:07,  6.29s/it]
 24%|██▍       | 54205/225000 [113:07:16<325:48:51,  6.87s/it]
 24%|██▍       | 54206/225000 [113:07:21<297:42:58,  6.28s/it]
 24%|██▍       | 54207/225000 [113:07:25<261:49:30,  5.52s/it]
 24%|██▍       | 54208/225000 [113:07:30<258:37:37,  5.45s/it]
 24%|██▍       | 54209/225000 [113:07:36<266:01:03,  5.61s/it]
 24%|██▍       | 54210/225000 [113:07:41<262:31:29,  5.53s/it]
 24%|██▍       | 54211/225000 [113:07:52<341:42:52,  7.20s/it]
 24%|██▍       | 54212/225000 [113:08:03<384:04:39,  8.10s/it]
 24%|██▍       | 54213/225000 [113:08:07<328:53:40,  6.93s/it]
 24%|██▍       | 54214/225000 [113:08:15<345:30:02,  7.28s/it]
 24%|██▍       | 54215/225000 [113:08:22<345:02:34,  7.27s/it]
 24%|██▍       | 54216/225000 [113:08:29<337:22:17,  7.11s/it]
 24%|██▍       | 54217/225000 [113:08:37<349:39:01,  7.37s/it]
 24%|██▍       | 54218/225000 [113:08:41<308:35:08,  6.50s/it]
 24%|██▍       | 54219/225000 [113:08:46<282:25:13,  5.95s/it]
 24%|██▍       | 54220/225000 [113:08:51<274:40:53,  5.79s/it]
 24%|██▍       | 54221/225000 [113:09:04<373:02:08,  7.86s/it]
 24%|██▍       | 54222/225000 [113:09:08<316:55:24,  6.68s/it]
 24%|██▍       | 54223/225000 [113:09:15<320:16:45,  6.75s/it]
 24%|██▍       | 54224/225000 [113:09:23<334:03:24,  7.04s/it]
 24%|██▍       | 54225/225000 [113:09:30<343:49:17,  7.25s/it]
 24%|██▍       | 54226/225000 [113:09:34<295:50:09,  6.24s/it]
 24%|██▍       | 54227/225000 [113:09:43<337:00:49,  7.10s/it]
 24%|██▍       | 54228/225000 [113:09:51<343:52:45,  7.25s/it]
 24%|██▍       | 54229/225000 [113:09:57<322:02:07,  6.79s/it]
 24%|██▍       | 54230/225000 [113:10:05<346:30:34,  7.30s/it]
 24%|██▍       | 54231/225000 [113:10:09<293:38:51,  6.19s/it]
 24%|██▍       | 54232/225000 [113:10:17<326:53:50,  6.89s/it]
 24%|██▍       | 54233/225000 [113:10:22<298:13:05,  6.29s/it]
 24%|██▍       | 54234/225000 [113:10:27<281:49:39,  5.94s/it]
 24%|██▍       | 54235/225000 [113:10:32<265:53:03,  5.61s/it]
 24%|██▍       | 54236/225000 [113:10:37<255:15:39,  5.38s/it]
 24%|██▍       | 54237/225000 [113:10:42<250:23:45,  5.28s/it]
 24%|██▍       | 54238/225000 [113:10:52<317:46:41,  6.70s/it]
 24%|██▍       | 54239/225000 [113:11:02<368:58:33,  7.78s/it]
 24%|██▍       | 54240/225000 [113:11:06<311:44:51,  6.57s/it]
 24%|██▍       | 54241/225000 [113:11:18<380:20:06,  8.02s/it]
 24%|██▍       | 54242/225000 [113:11:21<315:40:57,  6.66s/it]
 24%|██▍       | 54243/225000 [113:11:26<291:56:14,  6.15s/it]
 24%|██▍       | 54244/225000 [113:11:38<368:01:25,  7.76s/it]
 24%|██▍       | 54245/225000 [113:11:41<311:51:29,  6.57s/it]
 24%|██▍       | 54246/225000 [113:11:50<335:32:47,  7.07s/it]
 24%|██▍       | 54247/225000 [113:11:56<324:51:46,  6.85s/it]
 24%|██▍       | 54248/225000 [113:12:05<357:34:04,  7.54s/it]
 24%|██▍       | 54249/225000 [113:12:10<320:09:37,  6.75s/it]
 24%|██▍       | 54250/225000 [113:12:15<291:08:40,  6.14s/it]
                                                              
{'loss': '0.704', 'grad_norm': '0.7433', 'learning_rate': '4.496e-05', 'epoch': '0.2411', 'num_input_tokens_seen': 188907424, 'train_runtime': '4.075e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54250/225000 [113:12:15<291:08:40,  6.14s/it]
 24%|██▍       | 54251/225000 [113:12:25<349:07:17,  7.36s/it]
 24%|██▍       | 54252/225000 [113:12:29<305:44:09,  6.45s/it]
 24%|██▍       | 54253/225000 [113:12:39<356:42:15,  7.52s/it]
 24%|██▍       | 54254/225000 [113:12:44<310:55:55,  6.56s/it]
 24%|██▍       | 54255/225000 [113:12:53<351:24:51,  7.41s/it]
 24%|██▍       | 54256/225000 [113:12:58<316:44:43,  6.68s/it]
 24%|██▍       | 54257/225000 [113:13:03<297:53:10,  6.28s/it]
 24%|██▍       | 54258/225000 [113:13:08<276:03:41,  5.82s/it]
 24%|██▍       | 54259/225000 [113:13:20<363:33:47,  7.67s/it]
 24%|██▍       | 54260/225000 [113:13:28<370:13:36,  7.81s/it]
 24%|██▍       | 54261/225000 [113:13:32<309:04:45,  6.52s/it]
 24%|██▍       | 54262/225000 [113:13:41<346:53:33,  7.31s/it]
 24%|██▍       | 54263/225000 [113:13:45<296:12:07,  6.25s/it]
 24%|██▍       | 54264/225000 [113:13:55<349:59:06,  7.38s/it]
 24%|██▍       | 54265/225000 [113:14:02<347:32:35,  7.33s/it]
 24%|██▍       | 54266/225000 [113:14:07<317:04:41,  6.69s/it]
 24%|██▍       | 54267/225000 [113:14:12<297:36:48,  6.28s/it]
 24%|██▍       | 54268/225000 [113:14:18<283:31:19,  5.98s/it]
 24%|██▍       | 54269/225000 [113:14:23<275:25:34,  5.81s/it]
 24%|██▍       | 54270/225000 [113:14:28<268:06:51,  5.65s/it]
 24%|██▍       | 54271/225000 [113:14:33<261:39:17,  5.52s/it]
 24%|██▍       | 54272/225000 [113:14:39<263:11:29,  5.55s/it]
 24%|██▍       | 54273/225000 [113:14:52<366:59:22,  7.74s/it]
 24%|██▍       | 54274/225000 [113:14:59<363:11:20,  7.66s/it]
 24%|██▍       | 54275/225000 [113:15:04<325:15:52,  6.86s/it]
 24%|██▍       | 54276/225000 [113:15:10<305:03:59,  6.43s/it]
 24%|██▍       | 54277/225000 [113:15:15<286:39:40,  6.04s/it]
 24%|██▍       | 54278/225000 [113:15:21<284:21:13,  6.00s/it]
 24%|██▍       | 54279/225000 [113:15:26<276:21:58,  5.83s/it]
 24%|██▍       | 54280/225000 [113:15:34<309:23:47,  6.52s/it]
 24%|██▍       | 54281/225000 [113:15:48<405:01:07,  8.54s/it]
 24%|██▍       | 54282/225000 [113:15:51<336:19:34,  7.09s/it]
 24%|██▍       | 54283/225000 [113:16:00<361:12:52,  7.62s/it]
 24%|██▍       | 54284/225000 [113:16:06<341:22:36,  7.20s/it]
 24%|██▍       | 54285/225000 [113:16:11<305:31:14,  6.44s/it]
 24%|██▍       | 54286/225000 [113:16:20<337:35:46,  7.12s/it]
 24%|██▍       | 54287/225000 [113:16:27<343:08:39,  7.24s/it]
 24%|██▍       | 54288/225000 [113:16:34<334:10:19,  7.05s/it]
 24%|██▍       | 54289/225000 [113:16:43<358:54:39,  7.57s/it]
 24%|██▍       | 54290/225000 [113:16:48<326:47:00,  6.89s/it]
 24%|██▍       | 54291/225000 [113:16:53<295:56:17,  6.24s/it]
 24%|██▍       | 54292/225000 [113:16:58<277:49:27,  5.86s/it]
 24%|██▍       | 54293/225000 [113:17:04<287:10:11,  6.06s/it]
 24%|██▍       | 54294/225000 [113:17:16<370:16:48,  7.81s/it]
 24%|██▍       | 54295/225000 [113:17:22<339:27:01,  7.16s/it]
 24%|██▍       | 54296/225000 [113:17:27<309:17:04,  6.52s/it]
 24%|██▍       | 54297/225000 [113:17:32<294:24:17,  6.21s/it]
 24%|██▍       | 54298/225000 [113:17:39<296:37:29,  6.26s/it]
 24%|██▍       | 54299/225000 [113:17:44<282:43:02,  5.96s/it]
 24%|██▍       | 54300/225000 [113:17:50<278:54:04,  5.88s/it]
                                                              
{'loss': '0.686', 'grad_norm': '0.6862', 'learning_rate': '4.495e-05', 'epoch': '0.2413', 'num_input_tokens_seen': 189059984, 'train_runtime': '4.079e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54300/225000 [113:17:50<278:54:04,  5.88s/it]
 24%|██▍       | 54301/225000 [113:17:56<282:38:58,  5.96s/it]
 24%|██▍       | 54302/225000 [113:18:06<347:22:51,  7.33s/it]
 24%|██▍       | 54303/225000 [113:18:18<410:18:02,  8.65s/it]
 24%|██▍       | 54304/225000 [113:18:23<363:00:18,  7.66s/it]
 24%|██▍       | 54305/225000 [113:18:29<339:02:02,  7.15s/it]
 24%|██▍       | 54306/225000 [113:18:35<318:12:57,  6.71s/it]
 24%|██▍       | 54307/225000 [113:18:43<334:42:13,  7.06s/it]
 24%|██▍       | 54308/225000 [113:18:47<296:08:21,  6.25s/it]
 24%|██▍       | 54309/225000 [113:18:54<299:12:27,  6.31s/it]
 24%|██▍       | 54310/225000 [113:19:03<334:52:09,  7.06s/it]
 24%|██▍       | 54311/225000 [113:19:07<295:03:17,  6.22s/it]
 24%|██▍       | 54312/225000 [113:19:17<346:57:35,  7.32s/it]
 24%|██▍       | 54313/225000 [113:19:24<344:36:01,  7.27s/it]
 24%|██▍       | 54314/225000 [113:19:29<314:51:48,  6.64s/it]
 24%|██▍       | 54315/225000 [113:19:34<292:22:35,  6.17s/it]
 24%|██▍       | 54316/225000 [113:19:42<322:51:46,  6.81s/it]
 24%|██▍       | 54317/225000 [113:19:47<290:59:55,  6.14s/it]
 24%|██▍       | 54318/225000 [113:19:53<292:01:09,  6.16s/it]
 24%|██▍       | 54319/225000 [113:20:04<358:10:12,  7.55s/it]
 24%|██▍       | 54320/225000 [113:20:09<328:09:29,  6.92s/it]
 24%|██▍       | 54321/225000 [113:20:15<313:29:27,  6.61s/it]
 24%|██▍       | 54322/225000 [113:20:19<277:04:16,  5.84s/it]
 24%|██▍       | 54323/225000 [113:20:24<262:53:31,  5.55s/it]
 24%|██▍       | 54324/225000 [113:20:34<329:21:53,  6.95s/it]
 24%|██▍       | 54325/225000 [113:20:42<332:52:28,  7.02s/it]
 24%|██▍       | 54326/225000 [113:20:51<359:27:39,  7.58s/it]
 24%|██▍       | 54327/225000 [113:20:55<309:14:33,  6.52s/it]
 24%|██▍       | 54328/225000 [113:21:03<336:25:59,  7.10s/it]
 24%|██▍       | 54329/225000 [113:21:11<354:52:29,  7.49s/it]
 24%|██▍       | 54330/225000 [113:21:16<309:09:40,  6.52s/it]
 24%|██▍       | 54331/225000 [113:21:20<282:03:47,  5.95s/it]
 24%|██▍       | 54332/225000 [113:21:31<348:42:51,  7.36s/it]
 24%|██▍       | 54333/225000 [113:21:35<308:44:17,  6.51s/it]
 24%|██▍       | 54334/225000 [113:21:44<342:55:50,  7.23s/it]
 24%|██▍       | 54335/225000 [113:21:49<304:58:02,  6.43s/it]
 24%|██▍       | 54336/225000 [113:21:54<284:16:50,  6.00s/it]
 24%|██▍       | 54337/225000 [113:21:59<265:15:08,  5.60s/it]
 24%|██▍       | 54338/225000 [113:22:11<359:58:48,  7.59s/it]
 24%|██▍       | 54339/225000 [113:22:17<343:55:15,  7.25s/it]
 24%|██▍       | 54340/225000 [113:22:22<300:31:05,  6.34s/it]
 24%|██▍       | 54341/225000 [113:22:31<349:43:17,  7.38s/it]
 24%|██▍       | 54342/225000 [113:22:36<317:11:22,  6.69s/it]
 24%|██▍       | 54343/225000 [113:22:45<339:55:27,  7.17s/it]
 24%|██▍       | 54344/225000 [113:22:49<302:52:18,  6.39s/it]
 24%|██▍       | 54345/225000 [113:22:54<279:28:45,  5.90s/it]
 24%|██▍       | 54346/225000 [113:22:58<258:18:10,  5.45s/it]
 24%|██▍       | 54347/225000 [113:23:03<251:39:09,  5.31s/it]
 24%|██▍       | 54348/225000 [113:23:14<326:09:03,  6.88s/it]
 24%|██▍       | 54349/225000 [113:23:24<371:51:27,  7.84s/it]
 24%|██▍       | 54350/225000 [113:23:29<330:16:46,  6.97s/it]
                                                              
{'loss': '0.713', 'grad_norm': '0.7917', 'learning_rate': '4.494e-05', 'epoch': '0.2416', 'num_input_tokens_seen': 189211160, 'train_runtime': '4.082e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54350/225000 [113:23:29<330:16:46,  6.97s/it]
 24%|██▍       | 54351/225000 [113:23:33<294:42:10,  6.22s/it]
 24%|██▍       | 54352/225000 [113:23:38<272:20:52,  5.75s/it]
 24%|██▍       | 54353/225000 [113:23:43<261:45:10,  5.52s/it]
 24%|██▍       | 54354/225000 [113:23:48<251:12:46,  5.30s/it]
 24%|██▍       | 54355/225000 [113:24:01<361:14:31,  7.62s/it]
 24%|██▍       | 54356/225000 [113:24:05<309:19:37,  6.53s/it]
 24%|██▍       | 54357/225000 [113:24:14<343:48:39,  7.25s/it]
 24%|██▍       | 54358/225000 [113:24:20<323:35:32,  6.83s/it]
 24%|██▍       | 54359/225000 [113:24:27<327:48:33,  6.92s/it]
 24%|██▍       | 54360/225000 [113:24:34<335:01:18,  7.07s/it]
 24%|██▍       | 54361/225000 [113:24:39<301:09:18,  6.35s/it]
 24%|██▍       | 54362/225000 [113:24:46<316:03:37,  6.67s/it]
 24%|██▍       | 54363/225000 [113:24:56<363:45:14,  7.67s/it]
 24%|██▍       | 54364/225000 [113:25:04<357:19:38,  7.54s/it]
 24%|██▍       | 54365/225000 [113:25:07<301:39:46,  6.36s/it]
 24%|██▍       | 54366/225000 [113:25:11<271:49:32,  5.73s/it]
 24%|██▍       | 54367/225000 [113:25:23<348:02:03,  7.34s/it]
 24%|██▍       | 54368/225000 [113:25:26<299:33:40,  6.32s/it]
 24%|██▍       | 54369/225000 [113:25:35<338:16:45,  7.14s/it]
 24%|██▍       | 54370/225000 [113:25:42<331:38:48,  7.00s/it]
 24%|██▍       | 54371/225000 [113:25:48<308:36:22,  6.51s/it]
 24%|██▍       | 54372/225000 [113:25:53<290:49:22,  6.14s/it]
 24%|██▍       | 54373/225000 [113:26:00<306:59:12,  6.48s/it]
 24%|██▍       | 54374/225000 [113:26:11<367:43:14,  7.76s/it]
 24%|██▍       | 54375/225000 [113:26:17<339:19:12,  7.16s/it]
 24%|██▍       | 54376/225000 [113:26:22<312:15:43,  6.59s/it]
 24%|██▍       | 54377/225000 [113:26:27<293:01:43,  6.18s/it]
 24%|██▍       | 54378/225000 [113:26:32<281:44:37,  5.94s/it]
 24%|██▍       | 54379/225000 [113:26:37<265:20:42,  5.60s/it]
 24%|██▍       | 54380/225000 [113:26:42<257:24:52,  5.43s/it]
 24%|██▍       | 54381/225000 [113:26:48<256:09:21,  5.40s/it]
 24%|██▍       | 54382/225000 [113:26:58<329:11:27,  6.95s/it]
 24%|██▍       | 54383/225000 [113:27:09<382:41:11,  8.07s/it]
 24%|██▍       | 54384/225000 [113:27:15<348:34:44,  7.36s/it]
 24%|██▍       | 54385/225000 [113:27:22<343:13:06,  7.24s/it]
 24%|██▍       | 54386/225000 [113:27:26<304:48:17,  6.43s/it]
 24%|██▍       | 54387/225000 [113:27:31<281:56:12,  5.95s/it]
 24%|██▍       | 54388/225000 [113:27:36<265:33:34,  5.60s/it]
 24%|██▍       | 54389/225000 [113:27:40<253:04:11,  5.34s/it]
 24%|██▍       | 54390/225000 [113:27:52<345:13:10,  7.28s/it]
 24%|██▍       | 54391/225000 [113:28:01<364:37:41,  7.69s/it]
 24%|██▍       | 54392/225000 [113:28:07<348:34:52,  7.36s/it]
 24%|██▍       | 54393/225000 [113:28:12<309:18:43,  6.53s/it]
 24%|██▍       | 54394/225000 [113:28:17<288:01:43,  6.08s/it]
 24%|██▍       | 54395/225000 [113:28:22<268:24:08,  5.66s/it]
 24%|██▍       | 54396/225000 [113:28:35<380:31:13,  8.03s/it]
 24%|██▍       | 54397/225000 [113:28:42<356:13:22,  7.52s/it]
 24%|██▍       | 54398/225000 [113:28:46<307:38:07,  6.49s/it]
 24%|██▍       | 54399/225000 [113:28:56<357:38:04,  7.55s/it]
 24%|██▍       | 54400/225000 [113:29:00<316:08:41,  6.67s/it]
                                                              
{'loss': '0.6959', 'grad_norm': '0.907', 'learning_rate': '4.493e-05', 'epoch': '0.2418', 'num_input_tokens_seen': 189362552, 'train_runtime': '4.085e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54400/225000 [113:29:00<316:08:41,  6.67s/it]
 24%|██▍       | 54401/225000 [113:29:05<288:45:46,  6.09s/it]
 24%|██▍       | 54402/225000 [113:29:17<377:54:15,  7.97s/it]
 24%|██▍       | 54403/225000 [113:29:22<334:04:01,  7.05s/it]
 24%|██▍       | 54404/225000 [113:29:28<310:07:35,  6.54s/it]
 24%|██▍       | 54405/225000 [113:29:35<320:18:04,  6.76s/it]
 24%|██▍       | 54406/225000 [113:29:40<290:29:48,  6.13s/it]
 24%|██▍       | 54407/225000 [113:29:50<344:27:50,  7.27s/it]
 24%|██▍       | 54408/225000 [113:29:57<340:05:29,  7.18s/it]
 24%|██▍       | 54409/225000 [113:30:04<341:55:02,  7.22s/it]
 24%|██▍       | 54410/225000 [113:30:11<339:24:50,  7.16s/it]
 24%|██▍       | 54411/225000 [113:30:15<300:07:07,  6.33s/it]
 24%|██▍       | 54412/225000 [113:30:26<364:21:48,  7.69s/it]
 24%|██▍       | 54413/225000 [113:30:32<332:48:19,  7.02s/it]
 24%|██▍       | 54414/225000 [113:30:38<317:10:03,  6.69s/it]
 24%|██▍       | 54415/225000 [113:30:46<339:06:32,  7.16s/it]
 24%|██▍       | 54416/225000 [113:30:51<314:40:46,  6.64s/it]
 24%|██▍       | 54417/225000 [113:30:57<297:31:29,  6.28s/it]
 24%|██▍       | 54418/225000 [113:31:02<283:37:07,  5.99s/it]
 24%|██▍       | 54419/225000 [113:31:07<268:21:00,  5.66s/it]
 24%|██▍       | 54420/225000 [113:31:13<267:47:41,  5.65s/it]
 24%|██▍       | 54421/225000 [113:31:19<273:58:56,  5.78s/it]
 24%|██▍       | 54422/225000 [113:31:30<355:24:13,  7.50s/it]
 24%|██▍       | 54423/225000 [113:31:41<404:48:18,  8.54s/it]
 24%|██▍       | 54424/225000 [113:31:48<376:18:07,  7.94s/it]
 24%|██▍       | 54425/225000 [113:31:53<339:09:51,  7.16s/it]
 24%|██▍       | 54426/225000 [113:32:01<350:36:38,  7.40s/it]
 24%|██▍       | 54427/225000 [113:32:08<342:01:52,  7.22s/it]
 24%|██▍       | 54428/225000 [113:32:15<338:49:28,  7.15s/it]
 24%|██▍       | 54429/225000 [113:32:20<307:45:08,  6.50s/it]
 24%|██▍       | 54430/225000 [113:32:29<353:35:43,  7.46s/it]
 24%|██▍       | 54431/225000 [113:32:33<305:11:59,  6.44s/it]
 24%|██▍       | 54432/225000 [113:32:42<331:52:56,  7.00s/it]
 24%|██▍       | 54433/225000 [113:32:48<315:19:54,  6.66s/it]
 24%|██▍       | 54434/225000 [113:32:54<305:42:01,  6.45s/it]
 24%|██▍       | 54435/225000 [113:33:01<315:53:20,  6.67s/it]
 24%|██▍       | 54436/225000 [113:33:09<334:26:14,  7.06s/it]
 24%|██▍       | 54437/225000 [113:33:14<310:08:46,  6.55s/it]
 24%|██▍       | 54438/225000 [113:33:20<296:50:30,  6.27s/it]
 24%|██▍       | 54439/225000 [113:33:25<281:30:38,  5.94s/it]
 24%|██▍       | 54440/225000 [113:33:30<267:30:17,  5.65s/it]
 24%|██▍       | 54441/225000 [113:33:38<307:10:27,  6.48s/it]
 24%|██▍       | 54442/225000 [113:33:50<384:11:14,  8.11s/it]
 24%|██▍       | 54443/225000 [113:33:55<332:24:02,  7.02s/it]
 24%|██▍       | 54444/225000 [113:34:04<370:45:14,  7.83s/it]
 24%|██▍       | 54445/225000 [113:34:10<336:03:52,  7.09s/it]
 24%|██▍       | 54446/225000 [113:34:13<282:48:11,  5.97s/it]
 24%|██▍       | 54447/225000 [113:34:24<348:54:14,  7.36s/it]
 24%|██▍       | 54448/225000 [113:34:31<354:48:24,  7.49s/it]
 24%|██▍       | 54449/225000 [113:34:35<297:05:24,  6.27s/it]
 24%|██▍       | 54450/225000 [113:34:43<317:32:04,  6.70s/it]
                                                              
{'loss': '0.7006', 'grad_norm': '0.739', 'learning_rate': '4.492e-05', 'epoch': '0.242', 'num_input_tokens_seen': 189514112, 'train_runtime': '4.089e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54450/225000 [113:34:43<317:32:04,  6.70s/it]
 24%|██▍       | 54451/225000 [113:34:48<294:53:43,  6.22s/it]
 24%|██▍       | 54452/225000 [113:34:53<276:41:31,  5.84s/it]
 24%|██▍       | 54453/225000 [113:35:03<339:02:30,  7.16s/it]
 24%|██▍       | 54454/225000 [113:35:12<361:52:36,  7.64s/it]
 24%|██▍       | 54455/225000 [113:35:15<307:44:58,  6.50s/it]
 24%|██▍       | 54456/225000 [113:35:25<346:29:34,  7.31s/it]
 24%|██▍       | 54457/225000 [113:35:29<308:03:07,  6.50s/it]
 24%|██▍       | 54458/225000 [113:35:34<284:01:42,  6.00s/it]
 24%|██▍       | 54459/225000 [113:35:42<305:58:26,  6.46s/it]
 24%|██▍       | 54460/225000 [113:35:51<347:33:12,  7.34s/it]
 24%|██▍       | 54461/225000 [113:35:55<304:09:31,  6.42s/it]
 24%|██▍       | 54462/225000 [113:36:03<321:07:12,  6.78s/it]
 24%|██▍       | 54463/225000 [113:36:08<297:24:38,  6.28s/it]
 24%|██▍       | 54464/225000 [113:36:13<273:47:51,  5.78s/it]
 24%|██▍       | 54465/225000 [113:36:18<263:23:44,  5.56s/it]
 24%|██▍       | 54466/225000 [113:36:27<321:58:00,  6.80s/it]
 24%|██▍       | 54467/225000 [113:36:36<350:17:53,  7.39s/it]
 24%|██▍       | 54468/225000 [113:36:42<321:20:20,  6.78s/it]
 24%|██▍       | 54469/225000 [113:36:49<336:55:10,  7.11s/it]
 24%|██▍       | 54470/225000 [113:36:54<298:06:04,  6.29s/it]
 24%|██▍       | 54471/225000 [113:37:03<343:47:39,  7.26s/it]
 24%|██▍       | 54472/225000 [113:37:09<314:13:34,  6.63s/it]
 24%|██▍       | 54473/225000 [113:37:16<328:13:46,  6.93s/it]
 24%|██▍       | 54474/225000 [113:37:22<308:05:15,  6.50s/it]
 24%|██▍       | 54475/225000 [113:37:27<295:11:12,  6.23s/it]
 24%|██▍       | 54476/225000 [113:37:32<278:21:11,  5.88s/it]
 24%|██▍       | 54477/225000 [113:37:41<314:37:07,  6.64s/it]
 24%|██▍       | 54478/225000 [113:37:53<387:57:12,  8.19s/it]
 24%|██▍       | 54479/225000 [113:37:57<330:32:50,  6.98s/it]
 24%|██▍       | 54480/225000 [113:38:03<325:44:10,  6.88s/it]
 24%|██▍       | 54481/225000 [113:38:11<339:12:47,  7.16s/it]
 24%|██▍       | 54482/225000 [113:38:20<361:33:07,  7.63s/it]
 24%|██▍       | 54483/225000 [113:38:28<363:52:12,  7.68s/it]
 24%|██▍       | 54484/225000 [113:38:31<308:05:06,  6.50s/it]
 24%|██▍       | 54485/225000 [113:38:36<283:44:10,  5.99s/it]
 24%|██▍       | 54486/225000 [113:38:49<376:10:40,  7.94s/it]
 24%|██▍       | 54487/225000 [113:38:54<332:42:32,  7.02s/it]
 24%|██▍       | 54488/225000 [113:38:59<304:43:53,  6.43s/it]
 24%|██▍       | 54489/225000 [113:39:03<280:43:33,  5.93s/it]
 24%|██▍       | 54490/225000 [113:39:09<274:08:33,  5.79s/it]
 24%|██▍       | 54491/225000 [113:39:14<260:43:31,  5.50s/it]
 24%|██▍       | 54492/225000 [113:39:25<349:21:31,  7.38s/it]
 24%|██▍       | 54493/225000 [113:39:34<373:01:19,  7.88s/it]
 24%|██▍       | 54494/225000 [113:39:39<320:40:50,  6.77s/it]
 24%|██▍       | 54495/225000 [113:39:43<292:21:57,  6.17s/it]
 24%|██▍       | 54496/225000 [113:39:49<280:56:52,  5.93s/it]
 24%|██▍       | 54497/225000 [113:40:03<393:47:47,  8.31s/it]
 24%|██▍       | 54498/225000 [113:40:09<364:46:06,  7.70s/it]
 24%|██▍       | 54499/225000 [113:40:16<354:39:11,  7.49s/it]
 24%|██▍       | 54500/225000 [113:40:22<332:54:52,  7.03s/it]
                                                              
{'loss': '0.704', 'grad_norm': '0.8341', 'learning_rate': '4.49e-05', 'epoch': '0.2422', 'num_input_tokens_seen': 189665392, 'train_runtime': '4.092e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54500/225000 [113:40:22<332:54:52,  7.03s/it]
 24%|██▍       | 54501/225000 [113:40:28<322:34:51,  6.81s/it]
 24%|██▍       | 54502/225000 [113:40:35<325:54:47,  6.88s/it]
 24%|██▍       | 54503/225000 [113:40:39<285:18:16,  6.02s/it]
 24%|██▍       | 54504/225000 [113:40:49<331:26:28,  7.00s/it]
 24%|██▍       | 54505/225000 [113:40:54<315:37:03,  6.66s/it]
 24%|██▍       | 54506/225000 [113:40:59<289:09:11,  6.11s/it]
 24%|██▍       | 54507/225000 [113:41:04<271:50:31,  5.74s/it]
 24%|██▍       | 54508/225000 [113:41:15<339:35:54,  7.17s/it]
 24%|██▍       | 54509/225000 [113:41:21<331:20:38,  7.00s/it]
 24%|██▍       | 54510/225000 [113:41:27<318:51:11,  6.73s/it]
 24%|██▍       | 54511/225000 [113:41:36<347:46:03,  7.34s/it]
 24%|██▍       | 54512/225000 [113:41:40<304:38:31,  6.43s/it]
 24%|██▍       | 54513/225000 [113:41:45<280:49:18,  5.93s/it]
 24%|██▍       | 54514/225000 [113:41:56<349:03:24,  7.37s/it]
 24%|██▍       | 54515/225000 [113:42:00<299:20:37,  6.32s/it]
 24%|██▍       | 54516/225000 [113:42:09<346:37:52,  7.32s/it]
 24%|██▍       | 54517/225000 [113:42:14<300:57:35,  6.36s/it]
 24%|██▍       | 54518/225000 [113:42:18<274:38:47,  5.80s/it]
 24%|██▍       | 54519/225000 [113:42:27<318:53:21,  6.73s/it]
 24%|██▍       | 54520/225000 [113:42:36<351:39:45,  7.43s/it]
 24%|██▍       | 54521/225000 [113:42:44<358:33:51,  7.57s/it]
 24%|██▍       | 54522/225000 [113:42:48<306:17:02,  6.47s/it]
 24%|██▍       | 54523/225000 [113:42:57<341:17:21,  7.21s/it]
 24%|██▍       | 54524/225000 [113:43:00<289:49:55,  6.12s/it]
 24%|██▍       | 54525/225000 [113:43:10<344:07:57,  7.27s/it]
 24%|██▍       | 54526/225000 [113:43:17<339:13:22,  7.16s/it]
 24%|██▍       | 54527/225000 [113:43:23<326:18:40,  6.89s/it]
 24%|██▍       | 54528/225000 [113:43:30<328:38:08,  6.94s/it]
 24%|██▍       | 54529/225000 [113:43:34<286:43:11,  6.05s/it]
 24%|██▍       | 54530/225000 [113:43:42<302:43:59,  6.39s/it]
 24%|██▍       | 54531/225000 [113:43:49<314:21:21,  6.64s/it]
 24%|██▍       | 54532/225000 [113:43:57<334:36:26,  7.07s/it]
 24%|██▍       | 54533/225000 [113:44:04<340:47:52,  7.20s/it]
 24%|██▍       | 54534/225000 [113:44:09<301:02:13,  6.36s/it]
 24%|██▍       | 54535/225000 [113:44:14<287:56:47,  6.08s/it]
 24%|██▍       | 54536/225000 [113:44:20<277:36:03,  5.86s/it]
 24%|██▍       | 54537/225000 [113:44:25<267:52:42,  5.66s/it]
 24%|██▍       | 54538/225000 [113:44:33<301:43:57,  6.37s/it]
 24%|██▍       | 54539/225000 [113:44:43<357:53:25,  7.56s/it]
 24%|██▍       | 54540/225000 [113:44:50<344:48:28,  7.28s/it]
 24%|██▍       | 54541/225000 [113:44:58<352:19:31,  7.44s/it]
 24%|██▍       | 54542/225000 [113:45:03<322:57:06,  6.82s/it]
 24%|██▍       | 54543/225000 [113:45:08<295:17:19,  6.24s/it]
 24%|██▍       | 54544/225000 [113:45:12<270:23:59,  5.71s/it]
 24%|██▍       | 54545/225000 [113:45:26<377:27:31,  7.97s/it]
 24%|██▍       | 54546/225000 [113:45:30<328:40:31,  6.94s/it]
 24%|██▍       | 54547/225000 [113:45:40<368:39:50,  7.79s/it]
 24%|██▍       | 54548/225000 [113:45:44<318:23:56,  6.72s/it]
 24%|██▍       | 54549/225000 [113:45:49<288:23:39,  6.09s/it]
 24%|██▍       | 54550/225000 [113:46:00<356:46:57,  7.54s/it]
                                                              
{'loss': '0.7297', 'grad_norm': '0.6864', 'learning_rate': '4.489e-05', 'epoch': '0.2424', 'num_input_tokens_seen': 189818120, 'train_runtime': '4.096e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54550/225000 [113:46:00<356:46:57,  7.54s/it]
 24%|██▍       | 54551/225000 [113:46:05<318:47:29,  6.73s/it]
 24%|██▍       | 54552/225000 [113:46:09<288:41:59,  6.10s/it]
 24%|██▍       | 54553/225000 [113:46:20<360:05:50,  7.61s/it]
 24%|██▍       | 54554/225000 [113:46:25<318:11:52,  6.72s/it]
 24%|██▍       | 54555/225000 [113:46:29<286:13:53,  6.05s/it]
 24%|██▍       | 54556/225000 [113:46:37<311:27:21,  6.58s/it]
 24%|██▍       | 54557/225000 [113:46:46<336:47:20,  7.11s/it]
 24%|██▍       | 54558/225000 [113:46:53<340:29:08,  7.19s/it]
 24%|██▍       | 54559/225000 [113:46:59<326:15:07,  6.89s/it]
 24%|██▍       | 54560/225000 [113:47:06<322:05:17,  6.80s/it]
 24%|██▍       | 54561/225000 [113:47:11<303:47:59,  6.42s/it]
 24%|██▍       | 54562/225000 [113:47:19<320:06:54,  6.76s/it]
 24%|██▍       | 54563/225000 [113:47:25<305:39:22,  6.46s/it]
 24%|██▍       | 54564/225000 [113:47:30<292:45:11,  6.18s/it]
 24%|██▍       | 54565/225000 [113:47:35<278:43:22,  5.89s/it]
 24%|██▍       | 54566/225000 [113:47:41<274:00:01,  5.79s/it]
 24%|██▍       | 54567/225000 [113:47:54<378:42:09,  8.00s/it]
 24%|██▍       | 54568/225000 [113:47:59<341:22:31,  7.21s/it]
 24%|██▍       | 54569/225000 [113:48:04<299:23:16,  6.32s/it]
 24%|██▍       | 54570/225000 [113:48:14<355:28:01,  7.51s/it]
 24%|██▍       | 54571/225000 [113:48:18<300:29:38,  6.35s/it]
 24%|██▍       | 54572/225000 [113:48:25<316:38:21,  6.69s/it]
 24%|██▍       | 54573/225000 [113:48:33<340:59:47,  7.20s/it]
 24%|██▍       | 54574/225000 [113:48:42<359:16:05,  7.59s/it]
 24%|██▍       | 54575/225000 [113:48:48<344:33:29,  7.28s/it]
 24%|██▍       | 54576/225000 [113:48:56<350:02:02,  7.39s/it]
 24%|██▍       | 54577/225000 [113:49:00<304:19:10,  6.43s/it]
 24%|██▍       | 54578/225000 [113:49:06<292:14:34,  6.17s/it]
 24%|██▍       | 54579/225000 [113:49:11<284:08:01,  6.00s/it]
 24%|██▍       | 54580/225000 [113:49:17<272:35:26,  5.76s/it]
 24%|██▍       | 54581/225000 [113:49:22<263:54:36,  5.57s/it]
 24%|██▍       | 54582/225000 [113:49:27<261:41:37,  5.53s/it]
 24%|██▍       | 54583/225000 [113:49:37<317:43:41,  6.71s/it]
 24%|██▍       | 54584/225000 [113:49:47<369:57:54,  7.82s/it]
 24%|██▍       | 54585/225000 [113:49:56<388:02:31,  8.20s/it]
 24%|██▍       | 54586/225000 [113:50:00<328:39:24,  6.94s/it]
 24%|██▍       | 54587/225000 [113:50:08<342:21:18,  7.23s/it]
 24%|██▍       | 54588/225000 [113:50:12<296:15:19,  6.26s/it]
 24%|██▍       | 54589/225000 [113:50:22<349:19:21,  7.38s/it]
 24%|██▍       | 54590/225000 [113:50:27<316:33:45,  6.69s/it]
 24%|██▍       | 54591/225000 [113:50:32<295:29:53,  6.24s/it]
 24%|██▍       | 54592/225000 [113:50:41<332:52:08,  7.03s/it]
 24%|██▍       | 54593/225000 [113:50:48<334:03:53,  7.06s/it]
 24%|██▍       | 54594/225000 [113:50:54<318:37:07,  6.73s/it]
 24%|██▍       | 54595/225000 [113:51:00<308:46:00,  6.52s/it]
 24%|██▍       | 54596/225000 [113:51:09<332:38:16,  7.03s/it]
 24%|██▍       | 54597/225000 [113:51:13<295:47:21,  6.25s/it]
 24%|██▍       | 54598/225000 [113:51:19<296:30:26,  6.26s/it]
 24%|██▍       | 54599/225000 [113:51:26<299:24:11,  6.33s/it]
 24%|██▍       | 54600/225000 [113:51:34<331:01:32,  6.99s/it]
                                                              
{'loss': '0.7019', 'grad_norm': '0.7428', 'learning_rate': '4.488e-05', 'epoch': '0.2427', 'num_input_tokens_seen': 189969984, 'train_runtime': '4.099e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54600/225000 [113:51:34<331:01:32,  6.99s/it]
 24%|██▍       | 54601/225000 [113:51:39<297:54:39,  6.29s/it]
 24%|██▍       | 54602/225000 [113:51:44<277:01:33,  5.85s/it]
 24%|██▍       | 54603/225000 [113:51:49<264:20:17,  5.58s/it]
 24%|██▍       | 54604/225000 [113:51:53<251:12:28,  5.31s/it]
 24%|██▍       | 54605/225000 [113:52:02<291:58:07,  6.17s/it]
 24%|██▍       | 54606/225000 [113:52:14<387:04:13,  8.18s/it]
 24%|██▍       | 54607/225000 [113:52:18<325:31:30,  6.88s/it]
 24%|██▍       | 54608/225000 [113:52:25<322:47:58,  6.82s/it]
 24%|██▍       | 54609/225000 [113:52:32<331:13:16,  7.00s/it]
 24%|██▍       | 54610/225000 [113:52:37<291:34:09,  6.16s/it]
 24%|██▍       | 54611/225000 [113:52:47<351:33:03,  7.43s/it]
 24%|██▍       | 54612/225000 [113:52:53<330:19:02,  6.98s/it]
 24%|██▍       | 54613/225000 [113:52:59<314:53:29,  6.65s/it]
 24%|██▍       | 54614/225000 [113:53:07<340:11:43,  7.19s/it]
 24%|██▍       | 54615/225000 [113:53:12<300:15:22,  6.34s/it]
 24%|██▍       | 54616/225000 [113:53:22<360:25:36,  7.62s/it]
 24%|██▍       | 54617/225000 [113:53:27<313:41:48,  6.63s/it]
 24%|██▍       | 54618/225000 [113:53:33<316:42:06,  6.69s/it]
 24%|██▍       | 54619/225000 [113:53:42<344:09:39,  7.27s/it]
 24%|██▍       | 54620/225000 [113:53:47<310:02:21,  6.55s/it]
 24%|██▍       | 54621/225000 [113:53:57<354:36:41,  7.49s/it]
 24%|██▍       | 54622/225000 [113:54:00<299:22:56,  6.33s/it]
 24%|██▍       | 54623/225000 [113:54:09<335:39:33,  7.09s/it]
 24%|██▍       | 54624/225000 [113:54:14<303:33:53,  6.41s/it]
 24%|██▍       | 54625/225000 [113:54:19<280:21:43,  5.92s/it]
 24%|██▍       | 54626/225000 [113:54:29<336:07:59,  7.10s/it]
 24%|██▍       | 54627/225000 [113:54:35<330:38:40,  6.99s/it]
 24%|██▍       | 54628/225000 [113:54:43<343:41:47,  7.26s/it]
 24%|██▍       | 54629/225000 [113:54:48<312:15:07,  6.60s/it]
 24%|██▍       | 54630/225000 [113:54:55<318:57:17,  6.74s/it]
 24%|██▍       | 54631/225000 [113:55:00<292:38:54,  6.18s/it]
 24%|██▍       | 54632/225000 [113:55:08<311:53:25,  6.59s/it]
 24%|██▍       | 54633/225000 [113:55:16<329:06:55,  6.95s/it]
 24%|██▍       | 54634/225000 [113:55:20<287:51:40,  6.08s/it]
 24%|██▍       | 54635/225000 [113:55:28<317:46:52,  6.72s/it]
 24%|██▍       | 54636/225000 [113:55:34<307:22:02,  6.50s/it]
 24%|██▍       | 54637/225000 [113:55:39<296:47:07,  6.27s/it]
 24%|██▍       | 54638/225000 [113:55:45<292:08:42,  6.17s/it]
 24%|██▍       | 54639/225000 [113:55:52<293:27:01,  6.20s/it]
 24%|██▍       | 54640/225000 [113:55:57<283:44:17,  6.00s/it]
 24%|██▍       | 54641/225000 [113:56:04<290:26:17,  6.14s/it]
 24%|██▍       | 54642/225000 [113:56:16<371:32:52,  7.85s/it]
 24%|██▍       | 54643/225000 [113:56:20<329:19:32,  6.96s/it]
 24%|██▍       | 54644/225000 [113:56:25<300:05:24,  6.34s/it]
 24%|██▍       | 54645/225000 [113:56:36<363:48:46,  7.69s/it]
 24%|██▍       | 54646/225000 [113:56:42<335:28:09,  7.09s/it]
 24%|██▍       | 54647/225000 [113:56:49<340:03:10,  7.19s/it]
 24%|██▍       | 54648/225000 [113:56:53<297:50:55,  6.29s/it]
 24%|██▍       | 54649/225000 [113:56:59<286:46:36,  6.06s/it]
 24%|██▍       | 54650/225000 [113:57:05<279:22:52,  5.90s/it]
                                                              
{'loss': '0.7076', 'grad_norm': '0.8019', 'learning_rate': '4.487e-05', 'epoch': '0.2429', 'num_input_tokens_seen': 190122272, 'train_runtime': '4.102e+05', 'train_tokens_per_second': '463.5'}

 24%|██▍       | 54650/225000 [113:57:05<279:22:52,  5.90s/it]
 24%|██▍       | 54651/225000 [113:57:10<274:15:27,  5.80s/it]
 24%|██▍       | 54652/225000 [113:57:15<267:58:48,  5.66s/it]
 24%|██▍       | 54653/225000 [113:57:21<263:04:49,  5.56s/it]
 24%|██▍       | 54654/225000 [113:57:28<284:06:59,  6.00s/it]
 24%|██▍       | 54655/225000 [113:57:41<390:37:12,  8.26s/it]
 24%|██▍       | 54656/225000 [113:57:47<350:30:56,  7.41s/it]
 24%|██▍       | 54657/225000 [113:57:51<304:15:15,  6.43s/it]
 24%|██▍       | 54658/225000 [113:58:02<375:19:23,  7.93s/it]
 24%|██▍       | 54659/225000 [113:58:08<349:12:44,  7.38s/it]
 24%|██▍       | 54660/225000 [113:58:13<303:55:48,  6.42s/it]
 24%|██▍       | 54661/225000 [113:58:18<289:04:00,  6.11s/it]
 24%|██▍       | 54662/225000 [113:58:23<280:09:39,  5.92s/it]
 24%|██▍       | 54663/225000 [113:58:29<276:09:16,  5.84s/it]
 24%|██▍       | 54664/225000 [113:58:35<277:36:33,  5.87s/it]
 24%|██▍       | 54665/225000 [113:58:44<323:10:27,  6.83s/it]
 24%|██▍       | 54666/225000 [113:58:56<398:36:20,  8.42s/it]
 24%|██▍       | 54667/225000 [113:59:00<335:25:23,  7.09s/it]
 24%|██▍       | 54668/225000 [113:59:08<347:13:39,  7.34s/it]
 24%|██▍       | 54669/225000 [113:59:15<342:30:16,  7.24s/it]
 24%|██▍       | 54670/225000 [113:59:20<312:46:12,  6.61s/it]
 24%|██▍       | 54671/225000 [113:59:25<279:27:31,  5.91s/it]
 24%|██▍       | 54672/225000 [113:59:35<342:51:21,  7.25s/it]
 24%|██▍       | 54673/225000 [113:59:39<295:39:23,  6.25s/it]
 24%|██▍       | 54674/225000 [113:59:48<333:54:51,  7.06s/it]
 24%|██▍       | 54675/225000 [113:59:53<313:02:22,  6.62s/it]
 24%|██▍       | 54676/225000 [113:59:59<298:14:28,  6.30s/it]
 24%|██▍       | 54677/225000 [114:00:04<285:39:14,  6.04s/it]
 24%|██▍       | 54678/225000 [114:00:09<272:57:52,  5.77s/it]
 24%|██▍       | 54679/225000 [114:00:15<266:10:09,  5.63s/it]
 24%|██▍       | 54680/225000 [114:00:21<278:21:47,  5.88s/it]
 24%|██▍       | 54681/225000 [114:00:34<372:52:13,  7.88s/it]
 24%|██▍       | 54682/225000 [114:00:40<342:46:00,  7.25s/it]
 24%|██▍       | 54683/225000 [114:00:47<338:24:27,  7.15s/it]
 24%|██▍       | 54684/225000 [114:00:53<334:15:43,  7.07s/it]
 24%|██▍       | 54685/225000 [114:01:01<337:48:08,  7.14s/it]
 24%|██▍       | 54686/225000 [114:01:07<326:37:48,  6.90s/it]
 24%|██▍       | 54687/225000 [114:01:13<315:41:20,  6.67s/it]
 24%|██▍       | 54688/225000 [114:01:18<295:53:07,  6.25s/it]
 24%|██▍       | 54689/225000 [114:01:23<277:07:12,  5.86s/it]
 24%|██▍       | 54690/225000 [114:01:34<348:22:12,  7.36s/it]
 24%|██▍       | 54691/225000 [114:01:39<314:17:16,  6.64s/it]
 24%|██▍       | 54692/225000 [114:01:44<289:06:11,  6.11s/it]
 24%|██▍       | 54693/225000 [114:01:49<276:21:45,  5.84s/it]
 24%|██▍       | 54694/225000 [114:01:59<328:19:41,  6.94s/it]
 24%|██▍       | 54695/225000 [114:02:06<334:14:29,  7.07s/it]
 24%|██▍       | 54696/225000 [114:02:10<294:23:10,  6.22s/it]
 24%|██▍       | 54697/225000 [114:02:19<330:39:34,  6.99s/it]
 24%|██▍       | 54698/225000 [114:02:26<333:58:24,  7.06s/it]
 24%|██▍       | 54699/225000 [114:02:30<290:14:25,  6.14s/it]
 24%|██▍       | 54700/225000 [114:02:40<342:45:52,  7.25s/it]
                                                              
{'loss': '0.7071', 'grad_norm': '0.8187', 'learning_rate': '4.486e-05', 'epoch': '0.2431', 'num_input_tokens_seen': 190272768, 'train_runtime': '4.106e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54700/225000 [114:02:40<342:45:52,  7.25s/it]
 24%|██▍       | 54701/225000 [114:02:44<293:39:55,  6.21s/it]
 24%|██▍       | 54702/225000 [114:02:52<322:29:42,  6.82s/it]
 24%|██▍       | 54703/225000 [114:03:00<333:58:52,  7.06s/it]
 24%|██▍       | 54704/225000 [114:03:06<313:22:55,  6.62s/it]
 24%|██▍       | 54705/225000 [114:03:12<310:31:19,  6.56s/it]
 24%|██▍       | 54706/225000 [114:03:16<281:24:43,  5.95s/it]
 24%|██▍       | 54707/225000 [114:03:22<268:43:42,  5.68s/it]
 24%|██▍       | 54708/225000 [114:03:34<361:07:28,  7.63s/it]
 24%|██▍       | 54709/225000 [114:03:38<317:02:26,  6.70s/it]
 24%|██▍       | 54710/225000 [114:03:43<290:04:39,  6.13s/it]
 24%|██▍       | 54711/225000 [114:03:48<268:59:12,  5.69s/it]
 24%|██▍       | 54712/225000 [114:03:54<276:14:40,  5.84s/it]
 24%|██▍       | 54713/225000 [114:04:07<381:35:07,  8.07s/it]
 24%|██▍       | 54714/225000 [114:04:14<364:07:22,  7.70s/it]
 24%|██▍       | 54715/225000 [114:04:19<325:14:17,  6.88s/it]
 24%|██▍       | 54716/225000 [114:04:23<292:29:58,  6.18s/it]
 24%|██▍       | 54717/225000 [114:04:28<264:27:27,  5.59s/it]
 24%|██▍       | 54718/225000 [114:04:38<336:56:30,  7.12s/it]
 24%|██▍       | 54719/225000 [114:04:42<293:19:06,  6.20s/it]
 24%|██▍       | 54720/225000 [114:04:52<334:45:28,  7.08s/it]
 24%|██▍       | 54721/225000 [114:04:58<321:53:40,  6.81s/it]
 24%|██▍       | 54722/225000 [114:05:04<318:07:50,  6.73s/it]
 24%|██▍       | 54723/225000 [114:05:11<313:00:51,  6.62s/it]
 24%|██▍       | 54724/225000 [114:05:17<312:01:36,  6.60s/it]
 24%|██▍       | 54725/225000 [114:05:22<285:53:44,  6.04s/it]
 24%|██▍       | 54726/225000 [114:05:29<306:41:04,  6.48s/it]
 24%|██▍       | 54727/225000 [114:05:35<286:45:58,  6.06s/it]
 24%|██▍       | 54728/225000 [114:05:40<272:56:59,  5.77s/it]
 24%|██▍       | 54729/225000 [114:05:45<260:18:51,  5.50s/it]
 24%|██▍       | 54730/225000 [114:05:54<321:58:50,  6.81s/it]
 24%|██▍       | 54731/225000 [114:06:03<347:28:00,  7.35s/it]
 24%|██▍       | 54732/225000 [114:06:06<291:49:00,  6.17s/it]
 24%|██▍       | 54733/225000 [114:06:15<331:38:07,  7.01s/it]
 24%|██▍       | 54734/225000 [114:06:23<344:35:21,  7.29s/it]
 24%|██▍       | 54735/225000 [114:06:31<348:20:26,  7.37s/it]
 24%|██▍       | 54736/225000 [114:06:37<331:17:02,  7.00s/it]
 24%|██▍       | 54737/225000 [114:06:42<308:09:59,  6.52s/it]
 24%|██▍       | 54738/225000 [114:06:48<291:05:41,  6.15s/it]
 24%|██▍       | 54739/225000 [114:06:53<282:35:51,  5.98s/it]
 24%|██▍       | 54740/225000 [114:07:03<339:32:49,  7.18s/it]
 24%|██▍       | 54741/225000 [114:07:12<360:52:52,  7.63s/it]
 24%|██▍       | 54742/225000 [114:07:18<339:19:54,  7.17s/it]
 24%|██▍       | 54743/225000 [114:07:23<307:03:14,  6.49s/it]
 24%|██▍       | 54744/225000 [114:07:32<337:28:02,  7.14s/it]
 24%|██▍       | 54745/225000 [114:07:38<322:36:21,  6.82s/it]
 24%|██▍       | 54746/225000 [114:07:44<311:49:13,  6.59s/it]
 24%|██▍       | 54747/225000 [114:07:54<360:04:29,  7.61s/it]
 24%|██▍       | 54748/225000 [114:07:58<319:29:17,  6.76s/it]
 24%|██▍       | 54749/225000 [114:08:04<306:52:08,  6.49s/it]
 24%|██▍       | 54750/225000 [114:08:11<316:02:53,  6.68s/it]
                                                              
{'loss': '0.696', 'grad_norm': '0.8465', 'learning_rate': '4.485e-05', 'epoch': '0.2433', 'num_input_tokens_seen': 190423720, 'train_runtime': '4.109e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54750/225000 [114:08:11<316:02:53,  6.68s/it]
 24%|██▍       | 54751/225000 [114:08:17<306:43:20,  6.49s/it]
 24%|██▍       | 54752/225000 [114:08:22<276:23:37,  5.84s/it]
 24%|██▍       | 54753/225000 [114:08:26<259:18:35,  5.48s/it]
 24%|██▍       | 54754/225000 [114:08:32<256:50:15,  5.43s/it]
 24%|██▍       | 54755/225000 [114:08:45<367:21:11,  7.77s/it]
 24%|██▍       | 54756/225000 [114:08:50<325:34:15,  6.88s/it]
 24%|██▍       | 54757/225000 [114:08:55<302:48:52,  6.40s/it]
 24%|██▍       | 54758/225000 [114:09:00<286:59:36,  6.07s/it]
 24%|██▍       | 54759/225000 [114:09:11<352:13:41,  7.45s/it]
 24%|██▍       | 54760/225000 [114:09:18<343:30:16,  7.26s/it]
 24%|██▍       | 54761/225000 [114:09:25<343:37:58,  7.27s/it]
 24%|██▍       | 54762/225000 [114:09:33<350:48:14,  7.42s/it]
 24%|██▍       | 54763/225000 [114:09:38<318:06:54,  6.73s/it]
 24%|██▍       | 54764/225000 [114:09:43<297:28:54,  6.29s/it]
 24%|██▍       | 54765/225000 [114:09:48<280:16:17,  5.93s/it]
 24%|██▍       | 54766/225000 [114:09:55<282:47:55,  5.98s/it]
 24%|██▍       | 54767/225000 [114:10:00<274:54:02,  5.81s/it]
 24%|██▍       | 54768/225000 [114:10:10<336:10:03,  7.11s/it]
 24%|██▍       | 54769/225000 [114:10:15<311:23:54,  6.59s/it]
 24%|██▍       | 54770/225000 [114:10:21<299:17:28,  6.33s/it]
 24%|██▍       | 54771/225000 [114:10:27<290:01:23,  6.13s/it]
 24%|██▍       | 54772/225000 [114:10:32<282:18:12,  5.97s/it]
 24%|██▍       | 54773/225000 [114:10:38<271:03:26,  5.73s/it]
 24%|██▍       | 54774/225000 [114:10:43<272:53:46,  5.77s/it]
 24%|██▍       | 54775/225000 [114:10:49<269:52:35,  5.71s/it]
 24%|██▍       | 54776/225000 [114:11:01<365:36:23,  7.73s/it]
 24%|██▍       | 54777/225000 [114:11:07<334:52:10,  7.08s/it]
 24%|██▍       | 54778/225000 [114:11:13<313:31:18,  6.63s/it]
 24%|██▍       | 54779/225000 [114:11:18<301:19:55,  6.37s/it]
 24%|██▍       | 54780/225000 [114:11:24<297:07:12,  6.28s/it]
 24%|██▍       | 54781/225000 [114:11:30<283:12:39,  5.99s/it]
 24%|██▍       | 54782/225000 [114:11:35<275:47:29,  5.83s/it]
 24%|██▍       | 54783/225000 [114:11:41<267:30:23,  5.66s/it]
 24%|██▍       | 54784/225000 [114:11:46<268:51:14,  5.69s/it]
 24%|██▍       | 54785/225000 [114:11:59<375:33:21,  7.94s/it]
 24%|██▍       | 54786/225000 [114:12:09<394:08:06,  8.34s/it]
 24%|██▍       | 54787/225000 [114:12:14<346:50:49,  7.34s/it]
 24%|██▍       | 54788/225000 [114:12:19<315:56:28,  6.68s/it]
 24%|██▍       | 54789/225000 [114:12:24<289:52:51,  6.13s/it]
 24%|██▍       | 54790/225000 [114:12:29<282:43:51,  5.98s/it]
 24%|██▍       | 54791/225000 [114:12:43<384:35:48,  8.13s/it]
 24%|██▍       | 54792/225000 [114:12:47<329:37:12,  6.97s/it]
 24%|██▍       | 54793/225000 [114:12:53<321:37:57,  6.80s/it]
 24%|██▍       | 54794/225000 [114:13:02<355:49:14,  7.53s/it]
 24%|██▍       | 54795/225000 [114:13:11<366:33:47,  7.75s/it]
 24%|██▍       | 54796/225000 [114:13:18<354:20:47,  7.49s/it]
 24%|██▍       | 54797/225000 [114:13:24<334:13:35,  7.07s/it]
 24%|██▍       | 54798/225000 [114:13:29<305:09:17,  6.45s/it]
 24%|██▍       | 54799/225000 [114:13:34<290:55:58,  6.15s/it]
 24%|██▍       | 54800/225000 [114:13:39<278:40:08,  5.89s/it]
                                                              
{'loss': '0.7147', 'grad_norm': '0.7871', 'learning_rate': '4.484e-05', 'epoch': '0.2436', 'num_input_tokens_seen': 190575472, 'train_runtime': '4.112e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54800/225000 [114:13:39<278:40:08,  5.89s/it]
 24%|██▍       | 54801/225000 [114:13:48<322:48:57,  6.83s/it]
 24%|██▍       | 54802/225000 [114:13:56<339:41:52,  7.19s/it]
 24%|██▍       | 54803/225000 [114:14:03<323:55:05,  6.85s/it]
 24%|██▍       | 54804/225000 [114:14:08<297:40:01,  6.30s/it]
 24%|██▍       | 54805/225000 [114:14:13<280:45:06,  5.94s/it]
 24%|██▍       | 54806/225000 [114:14:25<379:08:45,  8.02s/it]
 24%|██▍       | 54807/225000 [114:14:31<338:52:49,  7.17s/it]
 24%|██▍       | 54808/225000 [114:14:40<373:16:24,  7.90s/it]
 24%|██▍       | 54809/225000 [114:14:44<319:22:57,  6.76s/it]
 24%|██▍       | 54810/225000 [114:14:53<349:21:11,  7.39s/it]
 24%|██▍       | 54811/225000 [114:14:57<302:17:32,  6.39s/it]
 24%|██▍       | 54812/225000 [114:15:04<304:36:34,  6.44s/it]
 24%|██▍       | 54813/225000 [114:15:13<338:15:47,  7.16s/it]
 24%|██▍       | 54814/225000 [114:15:20<337:27:33,  7.14s/it]
 24%|██▍       | 54815/225000 [114:15:25<308:46:54,  6.53s/it]
 24%|██▍       | 54816/225000 [114:15:30<293:35:51,  6.21s/it]
 24%|██▍       | 54817/225000 [114:15:36<279:49:21,  5.92s/it]
 24%|██▍       | 54818/225000 [114:15:44<308:31:17,  6.53s/it]
 24%|██▍       | 54819/225000 [114:15:54<369:54:56,  7.83s/it]
 24%|██▍       | 54820/225000 [114:15:58<315:33:00,  6.68s/it]
 24%|██▍       | 54821/225000 [114:16:08<362:12:13,  7.66s/it]
 24%|██▍       | 54822/225000 [114:16:14<334:11:41,  7.07s/it]
 24%|██▍       | 54823/225000 [114:16:20<315:20:38,  6.67s/it]
 24%|██▍       | 54824/225000 [114:16:25<296:34:20,  6.27s/it]
 24%|██▍       | 54825/225000 [114:16:31<287:02:21,  6.07s/it]
 24%|██▍       | 54826/225000 [114:16:36<274:51:05,  5.81s/it]
 24%|██▍       | 54827/225000 [114:16:42<275:23:39,  5.83s/it]
 24%|██▍       | 54828/225000 [114:16:47<269:47:55,  5.71s/it]
 24%|██▍       | 54829/225000 [114:17:01<381:17:00,  8.07s/it]
 24%|██▍       | 54830/225000 [114:17:06<345:39:17,  7.31s/it]
 24%|██▍       | 54831/225000 [114:17:12<321:50:44,  6.81s/it]
 24%|██▍       | 54832/225000 [114:17:18<307:40:06,  6.51s/it]
 24%|██▍       | 54833/225000 [114:17:23<294:53:18,  6.24s/it]
 24%|██▍       | 54834/225000 [114:17:29<288:37:07,  6.11s/it]
 24%|██▍       | 54835/225000 [114:17:35<287:03:47,  6.07s/it]
 24%|██▍       | 54836/225000 [114:17:46<351:41:49,  7.44s/it]
 24%|██▍       | 54837/225000 [114:17:55<382:58:34,  8.10s/it]
 24%|██▍       | 54838/225000 [114:17:59<322:14:48,  6.82s/it]
 24%|██▍       | 54839/225000 [114:18:08<355:35:27,  7.52s/it]
 24%|██▍       | 54840/225000 [114:18:13<307:43:19,  6.51s/it]
 24%|██▍       | 54841/225000 [114:18:22<350:40:32,  7.42s/it]
 24%|██▍       | 54842/225000 [114:18:27<315:52:23,  6.68s/it]
 24%|██▍       | 54843/225000 [114:18:32<293:46:11,  6.22s/it]
 24%|██▍       | 54844/225000 [114:18:37<275:23:28,  5.83s/it]
 24%|██▍       | 54845/225000 [114:18:42<266:01:16,  5.63s/it]
 24%|██▍       | 54846/225000 [114:18:55<371:12:55,  7.85s/it]
 24%|██▍       | 54847/225000 [114:19:00<329:42:31,  6.98s/it]
 24%|██▍       | 54848/225000 [114:19:05<301:50:48,  6.39s/it]
 24%|██▍       | 54849/225000 [114:19:10<285:01:51,  6.03s/it]
 24%|██▍       | 54850/225000 [114:19:15<269:14:50,  5.70s/it]
                                                              
{'loss': '0.7072', 'grad_norm': '0.9712', 'learning_rate': '4.483e-05', 'epoch': '0.2438', 'num_input_tokens_seen': 190729136, 'train_runtime': '4.116e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54850/225000 [114:19:15<269:14:50,  5.70s/it]
 24%|██▍       | 54851/225000 [114:19:21<268:01:13,  5.67s/it]
 24%|██▍       | 54852/225000 [114:19:31<329:39:29,  6.97s/it]
 24%|██▍       | 54853/225000 [114:19:41<365:59:56,  7.74s/it]
 24%|██▍       | 54854/225000 [114:19:48<365:59:46,  7.74s/it]
 24%|██▍       | 54855/225000 [114:19:54<335:08:15,  7.09s/it]
 24%|██▍       | 54856/225000 [114:20:02<344:21:11,  7.29s/it]
 24%|██▍       | 54857/225000 [114:20:10<360:46:56,  7.63s/it]
 24%|██▍       | 54858/225000 [114:20:14<308:47:20,  6.53s/it]
 24%|██▍       | 54859/225000 [114:20:25<366:38:45,  7.76s/it]
 24%|██▍       | 54860/225000 [114:20:29<313:03:12,  6.62s/it]
 24%|██▍       | 54861/225000 [114:20:35<306:07:58,  6.48s/it]
 24%|██▍       | 54862/225000 [114:20:44<343:36:57,  7.27s/it]
 24%|██▍       | 54863/225000 [114:20:51<338:31:30,  7.16s/it]
 24%|██▍       | 54864/225000 [114:20:58<335:43:44,  7.10s/it]
 24%|██▍       | 54865/225000 [114:21:04<329:52:05,  6.98s/it]
 24%|██▍       | 54866/225000 [114:21:11<318:31:23,  6.74s/it]
 24%|██▍       | 54867/225000 [114:21:18<331:46:19,  7.02s/it]
 24%|██▍       | 54868/225000 [114:21:27<356:57:14,  7.55s/it]
 24%|██▍       | 54869/225000 [114:21:32<313:54:59,  6.64s/it]
 24%|██▍       | 54870/225000 [114:21:37<292:24:34,  6.19s/it]
 24%|██▍       | 54871/225000 [114:21:43<288:34:51,  6.11s/it]
 24%|██▍       | 54872/225000 [114:21:53<348:05:03,  7.37s/it]
 24%|██▍       | 54873/225000 [114:22:01<355:29:53,  7.52s/it]
 24%|██▍       | 54874/225000 [114:22:08<350:26:22,  7.42s/it]
 24%|██▍       | 54875/225000 [114:22:13<312:20:09,  6.61s/it]
 24%|██▍       | 54876/225000 [114:22:17<284:10:12,  6.01s/it]
 24%|██▍       | 54877/225000 [114:22:27<338:46:10,  7.17s/it]
 24%|██▍       | 54878/225000 [114:22:35<342:55:02,  7.26s/it]
 24%|██▍       | 54879/225000 [114:22:39<306:01:56,  6.48s/it]
 24%|██▍       | 54880/225000 [114:22:44<279:17:22,  5.91s/it]
 24%|██▍       | 54881/225000 [114:22:55<350:02:37,  7.41s/it]
 24%|██▍       | 54882/225000 [114:22:59<303:05:30,  6.41s/it]
 24%|██▍       | 54883/225000 [114:23:08<340:07:50,  7.20s/it]
 24%|██▍       | 54884/225000 [114:23:15<338:22:11,  7.16s/it]
 24%|██▍       | 54885/225000 [114:23:20<312:30:37,  6.61s/it]
 24%|██▍       | 54886/225000 [114:23:24<273:45:45,  5.79s/it]
 24%|██▍       | 54887/225000 [114:23:32<307:39:34,  6.51s/it]
 24%|██▍       | 54888/225000 [114:23:38<288:54:19,  6.11s/it]
 24%|██▍       | 54889/225000 [114:23:44<291:43:10,  6.17s/it]
 24%|██▍       | 54890/225000 [114:23:51<311:08:03,  6.58s/it]
 24%|██▍       | 54891/225000 [114:23:57<298:04:13,  6.31s/it]
 24%|██▍       | 54892/225000 [114:24:09<380:52:44,  8.06s/it]
 24%|██▍       | 54893/225000 [114:24:13<317:56:38,  6.73s/it]
 24%|██▍       | 54894/225000 [114:24:22<356:51:48,  7.55s/it]
 24%|██▍       | 54895/225000 [114:24:27<314:19:57,  6.65s/it]
 24%|██▍       | 54896/225000 [114:24:32<287:06:19,  6.08s/it]
 24%|██▍       | 54897/225000 [114:24:38<296:19:57,  6.27s/it]
 24%|██▍       | 54898/225000 [114:24:50<367:59:33,  7.79s/it]
 24%|██▍       | 54899/225000 [114:24:54<322:49:30,  6.83s/it]
 24%|██▍       | 54900/225000 [114:25:00<301:41:55,  6.39s/it]
                                                              
{'loss': '0.7175', 'grad_norm': '0.7855', 'learning_rate': '4.482e-05', 'epoch': '0.244', 'num_input_tokens_seen': 190882096, 'train_runtime': '4.119e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54900/225000 [114:25:00<301:41:55,  6.39s/it]
 24%|██▍       | 54901/225000 [114:25:05<289:31:16,  6.13s/it]
 24%|██▍       | 54902/225000 [114:25:11<280:40:47,  5.94s/it]
 24%|██▍       | 54903/225000 [114:25:16<271:43:53,  5.75s/it]
 24%|██▍       | 54904/225000 [114:25:21<267:35:57,  5.66s/it]
 24%|██▍       | 54905/225000 [114:25:31<323:54:32,  6.86s/it]
 24%|██▍       | 54906/225000 [114:25:44<406:03:41,  8.59s/it]
 24%|██▍       | 54907/225000 [114:25:47<332:59:07,  7.05s/it]
 24%|██▍       | 54908/225000 [114:25:55<349:48:51,  7.40s/it]
 24%|██▍       | 54909/225000 [114:26:02<344:25:47,  7.29s/it]
 24%|██▍       | 54910/225000 [114:26:08<314:06:25,  6.65s/it]
 24%|██▍       | 54911/225000 [114:26:12<286:43:04,  6.07s/it]
 24%|██▍       | 54912/225000 [114:26:16<259:07:03,  5.48s/it]
 24%|██▍       | 54913/225000 [114:26:29<356:02:28,  7.54s/it]
 24%|██▍       | 54914/225000 [114:26:36<344:58:27,  7.30s/it]
 24%|██▍       | 54915/225000 [114:26:41<319:55:45,  6.77s/it]
 24%|██▍       | 54916/225000 [114:26:46<291:28:11,  6.17s/it]
 24%|██▍       | 54917/225000 [114:26:57<367:50:51,  7.79s/it]
 24%|██▍       | 54918/225000 [114:27:01<306:53:30,  6.50s/it]
 24%|██▍       | 54919/225000 [114:27:09<327:32:05,  6.93s/it]
 24%|██▍       | 54920/225000 [114:27:18<356:14:25,  7.54s/it]
 24%|██▍       | 54921/225000 [114:27:22<303:16:08,  6.42s/it]
 24%|██▍       | 54922/225000 [114:27:32<361:03:26,  7.64s/it]
 24%|██▍       | 54923/225000 [114:27:36<307:35:11,  6.51s/it]
 24%|██▍       | 54924/225000 [114:27:43<316:34:34,  6.70s/it]
 24%|██▍       | 54925/225000 [114:27:51<326:30:09,  6.91s/it]
 24%|██▍       | 54926/225000 [114:27:55<291:29:51,  6.17s/it]
 24%|██▍       | 54927/225000 [114:28:07<373:45:35,  7.91s/it]
 24%|██▍       | 54928/225000 [114:28:11<317:07:38,  6.71s/it]
 24%|██▍       | 54929/225000 [114:28:20<350:49:39,  7.43s/it]
 24%|██▍       | 54930/225000 [114:28:24<301:57:32,  6.39s/it]
 24%|██▍       | 54931/225000 [114:28:32<325:15:21,  6.88s/it]
 24%|██▍       | 54932/225000 [114:28:36<279:43:33,  5.92s/it]
 24%|██▍       | 54933/225000 [114:28:42<286:36:10,  6.07s/it]
 24%|██▍       | 54934/225000 [114:28:50<310:12:05,  6.57s/it]
 24%|██▍       | 54935/225000 [114:28:57<322:06:20,  6.82s/it]
 24%|██▍       | 54936/225000 [114:29:03<307:11:07,  6.50s/it]
 24%|██▍       | 54937/225000 [114:29:11<332:21:38,  7.04s/it]
 24%|██▍       | 54938/225000 [114:29:16<298:03:31,  6.31s/it]
 24%|██▍       | 54939/225000 [114:29:24<320:23:28,  6.78s/it]
 24%|██▍       | 54940/225000 [114:29:28<279:50:21,  5.92s/it]
 24%|██▍       | 54941/225000 [114:29:39<350:45:36,  7.43s/it]
 24%|██▍       | 54942/225000 [114:29:42<296:02:48,  6.27s/it]
 24%|██▍       | 54943/225000 [114:29:49<308:43:00,  6.54s/it]
 24%|██▍       | 54944/225000 [114:29:56<314:51:36,  6.67s/it]
 24%|██▍       | 54945/225000 [114:30:00<275:46:00,  5.84s/it]
 24%|██▍       | 54946/225000 [114:30:10<336:43:50,  7.13s/it]
 24%|██▍       | 54947/225000 [114:30:15<301:54:02,  6.39s/it]
 24%|██▍       | 54948/225000 [114:30:22<311:20:29,  6.59s/it]
 24%|██▍       | 54949/225000 [114:30:27<286:43:58,  6.07s/it]
 24%|██▍       | 54950/225000 [114:30:32<266:13:45,  5.64s/it]
                                                              
{'loss': '0.7176', 'grad_norm': '0.9746', 'learning_rate': '4.48e-05', 'epoch': '0.2442', 'num_input_tokens_seen': 191032136, 'train_runtime': '4.122e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 54950/225000 [114:30:32<266:13:45,  5.64s/it]
 24%|██▍       | 54951/225000 [114:30:44<362:00:32,  7.66s/it]
 24%|██▍       | 54952/225000 [114:30:48<308:01:37,  6.52s/it]
 24%|██▍       | 54953/225000 [114:30:57<351:40:24,  7.45s/it]
 24%|██▍       | 54954/225000 [114:31:03<319:15:46,  6.76s/it]
 24%|██▍       | 54955/225000 [114:31:07<293:56:28,  6.22s/it]
 24%|██▍       | 54956/225000 [114:31:13<289:18:51,  6.13s/it]
 24%|██▍       | 54957/225000 [114:31:24<352:29:04,  7.46s/it]
 24%|██▍       | 54958/225000 [114:31:31<342:43:34,  7.26s/it]
 24%|██▍       | 54959/225000 [114:31:37<329:53:41,  6.98s/it]
 24%|██▍       | 54960/225000 [114:31:42<293:37:16,  6.22s/it]
 24%|██▍       | 54961/225000 [114:31:46<267:48:05,  5.67s/it]
 24%|██▍       | 54962/225000 [114:31:51<255:54:43,  5.42s/it]
 24%|██▍       | 54963/225000 [114:32:04<365:00:20,  7.73s/it]
 24%|██▍       | 54964/225000 [114:32:10<341:26:54,  7.23s/it]
 24%|██▍       | 54965/225000 [114:32:18<355:38:43,  7.53s/it]
 24%|██▍       | 54966/225000 [114:32:23<318:17:33,  6.74s/it]
 24%|██▍       | 54967/225000 [114:32:28<288:31:57,  6.11s/it]
 24%|██▍       | 54968/225000 [114:32:33<270:20:12,  5.72s/it]
 24%|██▍       | 54969/225000 [114:32:39<282:42:29,  5.99s/it]
 24%|██▍       | 54970/225000 [114:32:52<386:57:00,  8.19s/it]
 24%|██▍       | 54971/225000 [114:32:57<337:29:19,  7.15s/it]
 24%|██▍       | 54972/225000 [114:33:04<334:23:07,  7.08s/it]
 24%|██▍       | 54973/225000 [114:33:11<333:50:06,  7.07s/it]
 24%|██▍       | 54974/225000 [114:33:19<343:40:54,  7.28s/it]
 24%|██▍       | 54975/225000 [114:33:23<305:04:22,  6.46s/it]
 24%|██▍       | 54976/225000 [114:33:28<275:31:44,  5.83s/it]
 24%|██▍       | 54977/225000 [114:33:33<262:30:14,  5.56s/it]
 24%|██▍       | 54978/225000 [114:33:46<364:51:21,  7.73s/it]
 24%|██▍       | 54979/225000 [114:33:51<329:51:44,  6.98s/it]
 24%|██▍       | 54980/225000 [114:33:56<301:18:26,  6.38s/it]
 24%|██▍       | 54981/225000 [114:34:01<285:40:21,  6.05s/it]
 24%|██▍       | 54982/225000 [114:34:07<283:01:22,  5.99s/it]
 24%|██▍       | 54983/225000 [114:34:13<283:28:59,  6.00s/it]
 24%|██▍       | 54984/225000 [114:34:18<273:19:41,  5.79s/it]
 24%|██▍       | 54985/225000 [114:34:26<308:38:55,  6.54s/it]
 24%|██▍       | 54986/225000 [114:34:40<401:56:44,  8.51s/it]
 24%|██▍       | 54987/225000 [114:34:46<372:44:40,  7.89s/it]
 24%|██▍       | 54988/225000 [114:34:53<357:47:12,  7.58s/it]
 24%|██▍       | 54989/225000 [114:35:01<369:17:08,  7.82s/it]
 24%|██▍       | 54990/225000 [114:35:07<338:36:15,  7.17s/it]
 24%|██▍       | 54991/225000 [114:35:15<345:24:05,  7.31s/it]
 24%|██▍       | 54992/225000 [114:35:21<327:45:35,  6.94s/it]
 24%|██▍       | 54993/225000 [114:35:27<326:34:33,  6.92s/it]
 24%|██▍       | 54994/225000 [114:35:34<321:21:27,  6.80s/it]
 24%|██▍       | 54995/225000 [114:35:39<294:41:21,  6.24s/it]
 24%|██▍       | 54996/225000 [114:35:44<279:47:32,  5.92s/it]
 24%|██▍       | 54997/225000 [114:35:49<267:05:45,  5.66s/it]
 24%|██▍       | 54998/225000 [114:35:57<296:52:24,  6.29s/it]
 24%|██▍       | 54999/225000 [114:36:09<385:41:01,  8.17s/it]
 24%|██▍       | 55000/225000 [114:36:16<363:19:18,  7.69s/it]
                                                              
{'loss': '0.7142', 'grad_norm': '0.9041', 'learning_rate': '4.479e-05', 'epoch': '0.2444', 'num_input_tokens_seen': 191183152, 'train_runtime': '4.126e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 55000/225000 [114:36:16<363:19:18,  7.69s/it]
 24%|██▍       | 55001/225000 [114:36:19<301:47:44,  6.39s/it]
 24%|██▍       | 55002/225000 [114:36:26<305:05:17,  6.46s/it]
 24%|██▍       | 55003/225000 [114:36:31<285:39:27,  6.05s/it]
 24%|██▍       | 55004/225000 [114:36:36<272:29:55,  5.77s/it]
 24%|██▍       | 55005/225000 [114:36:42<266:04:17,  5.63s/it]
 24%|██▍       | 55006/225000 [114:36:47<260:08:46,  5.51s/it]
 24%|██▍       | 55007/225000 [114:36:54<280:18:16,  5.94s/it]
 24%|██▍       | 55008/225000 [114:37:07<385:31:10,  8.16s/it]
 24%|██▍       | 55009/225000 [114:37:14<370:44:09,  7.85s/it]
 24%|██▍       | 55010/225000 [114:37:19<328:47:17,  6.96s/it]
 24%|██▍       | 55011/225000 [114:37:27<339:53:38,  7.20s/it]
 24%|██▍       | 55012/225000 [114:37:35<356:12:18,  7.54s/it]
 24%|██▍       | 55013/225000 [114:37:42<346:43:35,  7.34s/it]
 24%|██▍       | 55014/225000 [114:37:47<305:25:26,  6.47s/it]
 24%|██▍       | 55015/225000 [114:37:51<276:10:01,  5.85s/it]
 24%|██▍       | 55016/225000 [114:37:56<264:00:04,  5.59s/it]
 24%|██▍       | 55017/225000 [114:38:09<367:12:58,  7.78s/it]
 24%|██▍       | 55018/225000 [114:38:13<323:24:56,  6.85s/it]
 24%|██▍       | 55019/225000 [114:38:18<294:32:09,  6.24s/it]
 24%|██▍       | 55020/225000 [114:38:29<359:34:39,  7.62s/it]
 24%|██▍       | 55021/225000 [114:38:33<307:45:57,  6.52s/it]
 24%|██▍       | 55022/225000 [114:38:41<332:31:29,  7.04s/it]
 24%|██▍       | 55023/225000 [114:38:45<290:51:54,  6.16s/it]
 24%|██▍       | 55024/225000 [114:38:57<366:03:19,  7.75s/it]
 24%|██▍       | 55025/225000 [114:39:01<316:00:02,  6.69s/it]
 24%|██▍       | 55026/225000 [114:39:09<326:58:21,  6.93s/it]
 24%|██▍       | 55027/225000 [114:39:15<318:39:01,  6.75s/it]
 24%|██▍       | 55028/225000 [114:39:23<331:26:43,  7.02s/it]
 24%|██▍       | 55029/225000 [114:39:27<297:24:00,  6.30s/it]
 24%|██▍       | 55030/225000 [114:39:32<277:21:34,  5.87s/it]
 24%|██▍       | 55031/225000 [114:39:37<263:02:48,  5.57s/it]
 24%|██▍       | 55032/225000 [114:39:42<256:10:51,  5.43s/it]
 24%|██▍       | 55033/225000 [114:39:48<266:58:40,  5.65s/it]
 24%|██▍       | 55034/225000 [114:40:02<381:41:18,  8.08s/it]
 24%|██▍       | 55035/225000 [114:40:08<357:11:01,  7.57s/it]
 24%|██▍       | 55036/225000 [114:40:13<314:35:35,  6.66s/it]
 24%|██▍       | 55037/225000 [114:40:18<292:13:37,  6.19s/it]
 24%|██▍       | 55038/225000 [114:40:24<289:46:05,  6.14s/it]
 24%|██▍       | 55039/225000 [114:40:37<388:43:30,  8.23s/it]
 24%|██▍       | 55040/225000 [114:40:42<345:52:09,  7.33s/it]
 24%|██▍       | 55041/225000 [114:40:48<322:29:50,  6.83s/it]
 24%|██▍       | 55042/225000 [114:40:52<278:25:12,  5.90s/it]
 24%|██▍       | 55043/225000 [114:41:01<332:39:46,  7.05s/it]
 24%|██▍       | 55044/225000 [114:41:09<333:07:18,  7.06s/it]
 24%|██▍       | 55045/225000 [114:41:17<351:17:20,  7.44s/it]
 24%|██▍       | 55046/225000 [114:41:22<325:40:44,  6.90s/it]
 24%|██▍       | 55047/225000 [114:41:27<296:15:19,  6.28s/it]
 24%|██▍       | 55048/225000 [114:41:32<269:36:31,  5.71s/it]
 24%|██▍       | 55049/225000 [114:41:36<256:10:20,  5.43s/it]
 24%|██▍       | 55050/225000 [114:41:51<379:33:26,  8.04s/it]
                                                              
{'loss': '0.7199', 'grad_norm': '0.764', 'learning_rate': '4.478e-05', 'epoch': '0.2447', 'num_input_tokens_seen': 191333144, 'train_runtime': '4.129e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 55050/225000 [114:41:51<379:33:26,  8.04s/it]
 24%|██▍       | 55051/225000 [114:41:54<320:20:59,  6.79s/it]
 24%|██▍       | 55052/225000 [114:42:02<327:20:26,  6.93s/it]
 24%|██▍       | 55053/225000 [114:42:07<299:57:59,  6.35s/it]
 24%|██▍       | 55054/225000 [114:42:12<279:05:22,  5.91s/it]
 24%|██▍       | 55055/225000 [114:42:25<389:18:48,  8.25s/it]
 24%|██▍       | 55056/225000 [114:42:30<340:21:23,  7.21s/it]
 24%|██▍       | 55057/225000 [114:42:35<302:59:23,  6.42s/it]
 24%|██▍       | 55058/225000 [114:42:44<346:54:49,  7.35s/it]
 24%|██▍       | 55059/225000 [114:42:50<327:33:54,  6.94s/it]
 24%|██▍       | 55060/225000 [114:42:59<358:00:20,  7.58s/it]
 24%|██▍       | 55061/225000 [114:43:04<317:12:53,  6.72s/it]
 24%|██▍       | 55062/225000 [114:43:12<341:36:21,  7.24s/it]
 24%|██▍       | 55063/225000 [114:43:17<302:49:10,  6.42s/it]
 24%|██▍       | 55064/225000 [114:43:23<293:03:26,  6.21s/it]
 24%|██▍       | 55065/225000 [114:43:28<280:00:56,  5.93s/it]
 24%|██▍       | 55066/225000 [114:43:33<271:52:38,  5.76s/it]
 24%|██▍       | 55067/225000 [114:43:44<342:10:09,  7.25s/it]
 24%|██▍       | 55068/225000 [114:43:51<337:33:26,  7.15s/it]
 24%|██▍       | 55069/225000 [114:43:57<327:37:13,  6.94s/it]
 24%|██▍       | 55070/225000 [114:44:03<302:14:34,  6.40s/it]
 24%|██▍       | 55071/225000 [114:44:09<299:56:50,  6.35s/it]
 24%|██▍       | 55072/225000 [114:44:13<276:06:32,  5.85s/it]
 24%|██▍       | 55073/225000 [114:44:21<306:03:24,  6.48s/it]
 24%|██▍       | 55074/225000 [114:44:33<384:06:31,  8.14s/it]
 24%|██▍       | 55075/225000 [114:44:41<372:02:42,  7.88s/it]
 24%|██▍       | 55076/225000 [114:44:47<355:13:08,  7.53s/it]
 24%|██▍       | 55077/225000 [114:44:52<317:01:30,  6.72s/it]
 24%|██▍       | 55078/225000 [114:44:58<297:50:36,  6.31s/it]
 24%|██▍       | 55079/225000 [114:45:03<281:42:43,  5.97s/it]
 24%|██▍       | 55080/225000 [114:45:08<268:31:46,  5.69s/it]
 24%|██▍       | 55081/225000 [114:45:18<338:05:35,  7.16s/it]
 24%|██▍       | 55082/225000 [114:45:27<357:12:12,  7.57s/it]
 24%|██▍       | 55083/225000 [114:45:35<362:31:30,  7.68s/it]
 24%|██▍       | 55084/225000 [114:45:40<324:39:34,  6.88s/it]
 24%|██▍       | 55085/225000 [114:45:49<356:26:36,  7.55s/it]
 24%|██▍       | 55086/225000 [114:45:53<309:13:18,  6.55s/it]
 24%|██▍       | 55087/225000 [114:45:57<276:38:28,  5.86s/it]
 24%|██▍       | 55088/225000 [114:46:02<255:58:39,  5.42s/it]
 24%|██▍       | 55089/225000 [114:46:12<329:52:14,  6.99s/it]
 24%|██▍       | 55090/225000 [114:46:20<342:22:08,  7.25s/it]
 24%|██▍       | 55091/225000 [114:46:24<297:33:51,  6.30s/it]
 24%|██▍       | 55092/225000 [114:46:29<279:53:07,  5.93s/it]
 24%|██▍       | 55093/225000 [114:46:35<268:23:24,  5.69s/it]
 24%|██▍       | 55094/225000 [114:46:39<255:51:03,  5.42s/it]
 24%|██▍       | 55095/225000 [114:46:45<257:34:28,  5.46s/it]
 24%|██▍       | 55096/225000 [114:46:52<281:25:49,  5.96s/it]
 24%|██▍       | 55097/225000 [114:47:06<388:03:52,  8.22s/it]
 24%|██▍       | 55098/225000 [114:47:11<352:50:21,  7.48s/it]
 24%|██▍       | 55099/225000 [114:47:17<328:15:19,  6.96s/it]
 24%|██▍       | 55100/225000 [114:47:24<327:56:32,  6.95s/it]
                                                              
{'loss': '0.7125', 'grad_norm': '0.7667', 'learning_rate': '4.477e-05', 'epoch': '0.2449', 'num_input_tokens_seen': 191486176, 'train_runtime': '4.132e+05', 'train_tokens_per_second': '463.4'}

 24%|██▍       | 55100/225000 [114:47:24<327:56:32,  6.95s/it]
 24%|██▍       | 55101/225000 [114:47:28<289:59:06,  6.14s/it]
 24%|██▍       | 55102/225000 [114:47:33<268:44:39,  5.69s/it]
 24%|██▍       | 55103/225000 [114:47:43<325:05:16,  6.89s/it]
 24%|██▍       | 55104/225000 [114:47:51<343:44:13,  7.28s/it]
 24%|██▍       | 55105/225000 [114:47:56<312:09:00,  6.61s/it]
 24%|██▍       | 55106/225000 [114:48:01<292:10:21,  6.19s/it]
 24%|██▍       | 55107/225000 [114:48:13<376:41:47,  7.98s/it]
 24%|██▍       | 55108/225000 [114:48:17<314:20:58,  6.66s/it]
 24%|██▍       | 55109/225000 [114:48:24<321:43:55,  6.82s/it]
 24%|██▍       | 55110/225000 [114:48:32<339:47:51,  7.20s/it]
 24%|██▍       | 55111/225000 [114:48:39<337:07:18,  7.14s/it]
 24%|██▍       | 55112/225000 [114:48:45<316:27:26,  6.71s/it]
 24%|██▍       | 55113/225000 [114:48:50<289:18:57,  6.13s/it]
 24%|██▍       | 55114/225000 [114:48:54<270:18:54,  5.73s/it]
 24%|██▍       | 55115/225000 [114:49:07<371:09:37,  7.87s/it]
 24%|██▍       | 55116/225000 [114:49:12<321:50:31,  6.82s/it]
 24%|██▍       | 55117/225000 [114:49:19<331:42:16,  7.03s/it]
 24%|██▍       | 55118/225000 [114:49:25<312:27:28,  6.62s/it]
 24%|██▍       | 55119/225000 [114:49:33<334:13:18,  7.08s/it]
 24%|██▍       | 55120/225000 [114:49:38<308:08:40,  6.53s/it]
 24%|██▍       | 55121/225000 [114:49:44<294:52:03,  6.25s/it]
 24%|██▍       | 55122/225000 [114:49:49<279:08:50,  5.92s/it]
 24%|██▍       | 55123/225000 [114:49:55<280:22:12,  5.94s/it]
 24%|██▍       | 55124/225000 [114:50:01<277:58:22,  5.89s/it]
 24%|██▍       | 55125/225000 [114:50:06<273:08:29,  5.79s/it]
 25%|██▍       | 55126/225000 [114:50:13<291:34:51,  6.18s/it]
 25%|██▍       | 55127/225000 [114:50:26<382:52:44,  8.11s/it]
 25%|██▍       | 55128/225000 [114:50:31<346:29:36,  7.34s/it]
 25%|██▍       | 55129/225000 [114:50:38<339:57:14,  7.20s/it]
 25%|██▍       | 55130/225000 [114:50:47<361:55:58,  7.67s/it]
 25%|██▍       | 55131/225000 [114:50:51<309:00:09,  6.55s/it]
 25%|██▍       | 55132/225000 [114:50:56<290:41:17,  6.16s/it]
 25%|██▍       | 55133/225000 [114:51:09<376:46:43,  7.99s/it]
 25%|██▍       | 55134/225000 [114:51:13<325:44:34,  6.90s/it]
 25%|██▍       | 55135/225000 [114:51:18<306:21:40,  6.49s/it]
 25%|██▍       | 55136/225000 [114:51:24<287:38:26,  6.10s/it]
 25%|██▍       | 55137/225000 [114:51:29<275:45:54,  5.84s/it]
 25%|██▍       | 55138/225000 [114:51:35<279:41:56,  5.93s/it]
 25%|██▍       | 55139/225000 [114:51:49<392:38:53,  8.32s/it]
 25%|██▍       | 55140/225000 [114:51:54<347:28:08,  7.36s/it]
 25%|██▍       | 55141/225000 [114:51:59<313:35:31,  6.65s/it]
 25%|██▍       | 55142/225000 [114:52:04<285:08:45,  6.04s/it]
 25%|██▍       | 55143/225000 [114:52:08<263:48:57,  5.59s/it]
 25%|██▍       | 55144/225000 [114:52:17<309:59:08,  6.57s/it]
 25%|██▍       | 55145/225000 [114:52:29<390:08:21,  8.27s/it]
 25%|██▍       | 55146/225000 [114:52:35<351:41:24,  7.45s/it]
 25%|██▍       | 55147/225000 [114:52:39<311:54:57,  6.61s/it]
 25%|██▍       | 55148/225000 [114:52:49<350:15:13,  7.42s/it]
 25%|██▍       | 55149/225000 [114:52:53<306:40:27,  6.50s/it]
 25%|██▍       | 55150/225000 [114:52:59<298:15:33,  6.32s/it]
                                                              
{'loss': '0.7049', 'grad_norm': '0.786', 'learning_rate': '4.476e-05', 'epoch': '0.2451', 'num_input_tokens_seen': 191637592, 'train_runtime': '4.136e+05', 'train_tokens_per_second': '463.4'}

 25%|██▍       | 55150/225000 [114:52:59<298:15:33,  6.32s/it]
 25%|██▍       | 55151/225000 [114:53:04<285:39:42,  6.05s/it]
 25%|██▍       | 55152/225000 [114:53:10<283:11:24,  6.00s/it]
 25%|██▍       | 55153/225000 [114:53:19<318:35:16,  6.75s/it]
 25%|██▍       | 55154/225000 [114:53:25<313:11:03,  6.64s/it]
 25%|██▍       | 55155/225000 [114:53:31<298:02:08,  6.32s/it]
 25%|██▍       | 55156/225000 [114:53:39<323:32:25,  6.86s/it]
 25%|██▍       | 55157/225000 [114:53:50<389:02:25,  8.25s/it]
 25%|██▍       | 55158/225000 [114:53:54<324:15:27,  6.87s/it]
 25%|██▍       | 55159/225000 [114:54:04<362:21:27,  7.68s/it]
 25%|██▍       | 55160/225000 [114:54:09<322:45:14,  6.84s/it]
 25%|██▍       | 55161/225000 [114:54:13<293:54:24,  6.23s/it]
 25%|██▍       | 55162/225000 [114:54:18<277:46:11,  5.89s/it]
 25%|██▍       | 55163/225000 [114:54:27<318:40:15,  6.75s/it]
 25%|██▍       | 55164/225000 [114:54:37<361:12:55,  7.66s/it]
 25%|██▍       | 55165/225000 [114:54:42<320:18:57,  6.79s/it]
 25%|██▍       | 55166/225000 [114:54:52<369:55:26,  7.84s/it]
 25%|██▍       | 55167/225000 [114:54:58<340:03:48,  7.21s/it]
 25%|██▍       | 55168/225000 [114:55:05<334:44:48,  7.10s/it]
 25%|██▍       | 55169/225000 [114:55:10<304:03:45,  6.45s/it]
 25%|██▍       | 55170/225000 [114:55:14<274:59:39,  5.83s/it]
 25%|██▍       | 55171/225000 [114:55:18<257:06:03,  5.45s/it]
 25%|██▍       | 55172/225000 [114:55:31<361:49:26,  7.67s/it]
 25%|██▍       | 55173/225000 [114:55:35<312:20:41,  6.62s/it]
 25%|██▍       | 55174/225000 [114:55:40<282:12:19,  5.98s/it]
 25%|██▍       | 55175/225000 [114:55:45<275:20:05,  5.84s/it]
 25%|██▍       | 55176/225000 [114:55:51<275:00:15,  5.83s/it]
 25%|██▍       | 55177/225000 [114:55:57<273:00:20,  5.79s/it]
 25%|██▍       | 55178/225000 [114:56:05<298:36:26,  6.33s/it]
 25%|██▍       | 55179/225000 [114:56:18<399:45:58,  8.47s/it]
 25%|██▍       | 55180/225000 [114:56:22<337:46:04,  7.16s/it]
 25%|██▍       | 55181/225000 [114:56:31<362:26:11,  7.68s/it]
 25%|██▍       | 55182/225000 [114:56:36<327:33:08,  6.94s/it]
 25%|██▍       | 55183/225000 [114:56:42<315:13:07,  6.68s/it]
 25%|██▍       | 55184/225000 [114:56:48<299:18:42,  6.35s/it]
 25%|██▍       | 55185/225000 [114:56:53<288:20:54,  6.11s/it]
 25%|██▍       | 55186/225000 [114:56:59<277:00:23,  5.87s/it]
 25%|██▍       | 55187/225000 [114:57:05<279:25:03,  5.92s/it]
 25%|██▍       | 55188/225000 [114:57:18<384:17:27,  8.15s/it]
 25%|██▍       | 55189/225000 [114:57:26<381:19:27,  8.08s/it]
 25%|██▍       | 55190/225000 [114:57:30<325:14:24,  6.90s/it]
 25%|██▍       | 55191/225000 [114:57:35<295:12:21,  6.26s/it]
 25%|██▍       | 55192/225000 [114:57:40<275:20:13,  5.84s/it]
 25%|██▍       | 55193/225000 [114:57:52<359:46:20,  7.63s/it]
 25%|██▍       | 55194/225000 [114:57:59<359:21:34,  7.62s/it]
 25%|██▍       | 55195/225000 [114:58:03<305:55:17,  6.49s/it]
 25%|██▍       | 55196/225000 [114:58:13<356:31:45,  7.56s/it]
 25%|██▍       | 55197/225000 [114:58:20<340:42:15,  7.22s/it]
 25%|██▍       | 55198/225000 [114:58:25<314:09:02,  6.66s/it]
 25%|██▍       | 55199/225000 [114:58:30<295:37:37,  6.27s/it]
 25%|██▍       | 55200/225000 [114:58:35<277:10:11,  5.88s/it]
                                                              
{'loss': '0.7084', 'grad_norm': '0.7595', 'learning_rate': '4.475e-05', 'epoch': '0.2453', 'num_input_tokens_seen': 191788768, 'train_runtime': '4.139e+05', 'train_tokens_per_second': '463.4'}

 25%|██▍       | 55200/225000 [114:58:35<277:10:11,  5.88s/it]
 25%|██▍       | 55201/225000 [114:58:48<376:13:49,  7.98s/it]
 25%|██▍       | 55202/225000 [114:58:52<324:49:50,  6.89s/it]
 25%|██▍       | 55203/225000 [114:59:02<360:05:01,  7.63s/it]
 25%|██▍       | 55204/225000 [114:59:06<315:03:47,  6.68s/it]
 25%|██▍       | 55205/225000 [114:59:12<295:13:34,  6.26s/it]
 25%|██▍       | 55206/225000 [114:59:23<364:19:54,  7.72s/it]
 25%|██▍       | 55207/225000 [114:59:27<308:23:29,  6.54s/it]
 25%|██▍       | 55208/225000 [114:59:35<333:54:11,  7.08s/it]
 25%|██▍       | 55209/225000 [114:59:40<306:50:35,  6.51s/it]
 25%|██▍       | 55210/225000 [114:59:45<280:36:27,  5.95s/it]
 25%|██▍       | 55211/225000 [114:59:49<263:16:04,  5.58s/it]
 25%|██▍       | 55212/225000 [114:59:59<324:15:12,  6.88s/it]
 25%|██▍       | 55213/225000 [115:00:09<362:20:28,  7.68s/it]
 25%|██▍       | 55214/225000 [115:00:12<304:21:10,  6.45s/it]
 25%|██▍       | 55215/225000 [115:00:21<328:32:15,  6.97s/it]
 25%|██▍       | 55216/225000 [115:00:28<340:13:36,  7.21s/it]
 25%|██▍       | 55217/225000 [115:00:34<315:35:20,  6.69s/it]
 25%|██▍       | 55218/225000 [115:00:38<278:36:05,  5.91s/it]
 25%|██▍       | 55219/225000 [115:00:47<318:38:52,  6.76s/it]
 25%|██▍       | 55220/225000 [115:00:53<318:45:59,  6.76s/it]
 25%|██▍       | 55221/225000 [115:00:58<286:54:50,  6.08s/it]
 25%|██▍       | 55222/225000 [115:01:02<264:55:47,  5.62s/it]
 25%|██▍       | 55223/225000 [115:01:08<260:09:57,  5.52s/it]
 25%|██▍       | 55224/225000 [115:01:18<321:49:07,  6.82s/it]
 25%|██▍       | 55225/225000 [115:01:28<371:46:33,  7.88s/it]
 25%|██▍       | 55226/225000 [115:01:33<331:12:23,  7.02s/it]
 25%|██▍       | 55227/225000 [115:01:38<298:49:36,  6.34s/it]
 25%|██▍       | 55228/225000 [115:01:43<280:41:29,  5.95s/it]
 25%|██▍       | 55229/225000 [115:01:48<271:24:43,  5.76s/it]
 25%|██▍       | 55230/225000 [115:02:00<364:09:16,  7.72s/it]
 25%|██▍       | 55231/225000 [115:02:05<313:12:57,  6.64s/it]
 25%|██▍       | 55232/225000 [115:02:09<282:56:35,  6.00s/it]
 25%|██▍       | 55233/225000 [115:02:19<332:50:58,  7.06s/it]
 25%|██▍       | 55234/225000 [115:02:26<332:35:26,  7.05s/it]
 25%|██▍       | 55235/225000 [115:02:30<292:01:02,  6.19s/it]
 25%|██▍       | 55236/225000 [115:02:42<383:21:11,  8.13s/it]
 25%|██▍       | 55237/225000 [115:02:46<316:06:04,  6.70s/it]
 25%|██▍       | 55238/225000 [115:02:51<288:30:44,  6.12s/it]
 25%|██▍       | 55239/225000 [115:02:55<264:49:47,  5.62s/it]
 25%|██▍       | 55240/225000 [115:03:08<370:30:22,  7.86s/it]
 25%|██▍       | 55241/225000 [115:03:12<314:55:58,  6.68s/it]
 25%|██▍       | 55242/225000 [115:03:21<352:24:42,  7.47s/it]
 25%|██▍       | 55243/225000 [115:03:26<307:32:31,  6.52s/it]
 25%|██▍       | 55244/225000 [115:03:37<372:47:05,  7.91s/it]
 25%|██▍       | 55245/225000 [115:03:41<314:43:12,  6.67s/it]
 25%|██▍       | 55246/225000 [115:03:45<280:36:53,  5.95s/it]
 25%|██▍       | 55247/225000 [115:03:49<257:02:10,  5.45s/it]
 25%|██▍       | 55248/225000 [115:04:02<365:17:43,  7.75s/it]
 25%|██▍       | 55249/225000 [115:04:07<322:40:14,  6.84s/it]
 25%|██▍       | 55250/225000 [115:04:12<297:56:22,  6.32s/it]
                                                              
{'loss': '0.7032', 'grad_norm': '0.7346', 'learning_rate': '4.474e-05', 'epoch': '0.2456', 'num_input_tokens_seen': 191939944, 'train_runtime': '4.143e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55250/225000 [115:04:12<297:56:22,  6.32s/it]
 25%|██▍       | 55251/225000 [115:04:17<273:32:24,  5.80s/it]
 25%|██▍       | 55252/225000 [115:04:22<261:13:37,  5.54s/it]
 25%|██▍       | 55253/225000 [115:04:27<257:31:08,  5.46s/it]
 25%|██▍       | 55254/225000 [115:04:41<376:40:28,  7.99s/it]
 25%|██▍       | 55255/225000 [115:04:46<339:56:04,  7.21s/it]
 25%|██▍       | 55256/225000 [115:04:55<359:56:18,  7.63s/it]
 25%|██▍       | 55257/225000 [115:05:01<338:54:46,  7.19s/it]
 25%|██▍       | 55258/225000 [115:05:06<305:53:28,  6.49s/it]
 25%|██▍       | 55259/225000 [115:05:10<278:24:38,  5.90s/it]
 25%|██▍       | 55260/225000 [115:05:15<259:52:25,  5.51s/it]
 25%|██▍       | 55261/225000 [115:05:26<336:25:17,  7.14s/it]
 25%|██▍       | 55262/225000 [115:05:32<319:52:31,  6.78s/it]
 25%|██▍       | 55263/225000 [115:05:42<364:27:40,  7.73s/it]
 25%|██▍       | 55264/225000 [115:05:47<324:19:12,  6.88s/it]
 25%|██▍       | 55265/225000 [115:05:54<327:39:12,  6.95s/it]
 25%|██▍       | 55266/225000 [115:05:58<284:20:41,  6.03s/it]
 25%|██▍       | 55267/225000 [115:06:04<288:59:53,  6.13s/it]
 25%|██▍       | 55268/225000 [115:06:12<312:16:17,  6.62s/it]
 25%|██▍       | 55269/225000 [115:06:17<287:55:44,  6.11s/it]
 25%|██▍       | 55270/225000 [115:06:21<267:49:07,  5.68s/it]
 25%|██▍       | 55271/225000 [115:06:29<302:23:31,  6.41s/it]
 25%|██▍       | 55272/225000 [115:06:40<362:02:03,  7.68s/it]
 25%|██▍       | 55273/225000 [115:06:49<384:19:09,  8.15s/it]
 25%|██▍       | 55274/225000 [115:06:53<318:40:41,  6.76s/it]
 25%|██▍       | 55275/225000 [115:06:57<281:32:10,  5.97s/it]
 25%|██▍       | 55276/225000 [115:07:02<269:04:29,  5.71s/it]
 25%|██▍       | 55277/225000 [115:07:08<267:12:13,  5.67s/it]
 25%|██▍       | 55278/225000 [115:07:13<268:00:10,  5.68s/it]
 25%|██▍       | 55279/225000 [115:07:19<267:59:32,  5.68s/it]
 25%|██▍       | 55280/225000 [115:07:28<311:32:03,  6.61s/it]
 25%|██▍       | 55281/225000 [115:07:40<388:04:14,  8.23s/it]
 25%|██▍       | 55282/225000 [115:07:45<338:43:56,  7.19s/it]
 25%|██▍       | 55283/225000 [115:07:53<349:57:37,  7.42s/it]
 25%|██▍       | 55284/225000 [115:08:00<350:12:36,  7.43s/it]
 25%|██▍       | 55285/225000 [115:08:05<308:59:18,  6.55s/it]
 25%|██▍       | 55286/225000 [115:08:10<297:15:36,  6.31s/it]
 25%|██▍       | 55287/225000 [115:08:16<287:10:11,  6.09s/it]
 25%|██▍       | 55288/225000 [115:08:21<276:37:33,  5.87s/it]
 25%|██▍       | 55289/225000 [115:08:26<268:32:59,  5.70s/it]
 25%|██▍       | 55290/225000 [115:08:35<312:28:20,  6.63s/it]
 25%|██▍       | 55291/225000 [115:08:46<376:45:15,  7.99s/it]
 25%|██▍       | 55292/225000 [115:08:51<327:44:47,  6.95s/it]
 25%|██▍       | 55293/225000 [115:09:01<369:22:09,  7.84s/it]
 25%|██▍       | 55294/225000 [115:09:05<322:39:02,  6.84s/it]
 25%|██▍       | 55295/225000 [115:09:17<386:20:07,  8.20s/it]
 25%|██▍       | 55296/225000 [115:09:21<325:42:43,  6.91s/it]
 25%|██▍       | 55297/225000 [115:09:29<346:22:06,  7.35s/it]
 25%|██▍       | 55298/225000 [115:09:33<302:55:14,  6.43s/it]
 25%|██▍       | 55299/225000 [115:09:43<348:06:48,  7.38s/it]
 25%|██▍       | 55300/225000 [115:09:48<313:57:36,  6.66s/it]
                                                              
{'loss': '0.7136', 'grad_norm': '0.8996', 'learning_rate': '4.473e-05', 'epoch': '0.2458', 'num_input_tokens_seen': 192090000, 'train_runtime': '4.146e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55300/225000 [115:09:48<313:57:36,  6.66s/it]
 25%|██▍       | 55301/225000 [115:09:55<322:21:28,  6.84s/it]
 25%|██▍       | 55302/225000 [115:09:59<286:32:42,  6.08s/it]
 25%|██▍       | 55303/225000 [115:10:08<322:35:50,  6.84s/it]
 25%|██▍       | 55304/225000 [115:10:16<339:32:53,  7.20s/it]
 25%|██▍       | 55305/225000 [115:10:20<294:47:04,  6.25s/it]
 25%|██▍       | 55306/225000 [115:10:32<367:39:51,  7.80s/it]
 25%|██▍       | 55307/225000 [115:10:36<313:06:46,  6.64s/it]
 25%|██▍       | 55308/225000 [115:10:39<272:27:47,  5.78s/it]
 25%|██▍       | 55309/225000 [115:10:51<357:53:08,  7.59s/it]
 25%|██▍       | 55310/225000 [115:10:55<302:56:11,  6.43s/it]
 25%|██▍       | 55311/225000 [115:11:03<330:13:49,  7.01s/it]
 25%|██▍       | 55312/225000 [115:11:08<300:53:27,  6.38s/it]
 25%|██▍       | 55313/225000 [115:11:17<340:51:58,  7.23s/it]
 25%|██▍       | 55314/225000 [115:11:23<323:54:16,  6.87s/it]
 25%|██▍       | 55315/225000 [115:11:32<348:08:20,  7.39s/it]
 25%|██▍       | 55316/225000 [115:11:36<303:09:06,  6.43s/it]
 25%|██▍       | 55317/225000 [115:11:42<293:13:58,  6.22s/it]
 25%|██▍       | 55318/225000 [115:11:51<327:57:38,  6.96s/it]
 25%|██▍       | 55319/225000 [115:11:59<350:28:09,  7.44s/it]
 25%|██▍       | 55320/225000 [115:12:04<313:09:47,  6.64s/it]
 25%|██▍       | 55321/225000 [115:12:09<287:39:53,  6.10s/it]
 25%|██▍       | 55322/225000 [115:12:14<270:32:38,  5.74s/it]
 25%|██▍       | 55323/225000 [115:12:20<281:53:06,  5.98s/it]
 25%|██▍       | 55324/225000 [115:12:32<367:51:52,  7.80s/it]
 25%|██▍       | 55325/225000 [115:12:38<338:04:59,  7.17s/it]
 25%|██▍       | 55326/225000 [115:12:44<322:04:06,  6.83s/it]
 25%|██▍       | 55327/225000 [115:12:51<321:17:56,  6.82s/it]
 25%|██▍       | 55328/225000 [115:12:58<325:00:00,  6.90s/it]
 25%|██▍       | 55329/225000 [115:13:05<326:54:01,  6.94s/it]
 25%|██▍       | 55330/225000 [115:13:11<313:30:54,  6.65s/it]
 25%|██▍       | 55331/225000 [115:13:15<283:47:34,  6.02s/it]
 25%|██▍       | 55332/225000 [115:13:20<266:26:20,  5.65s/it]
 25%|██▍       | 55333/225000 [115:13:25<256:03:38,  5.43s/it]
 25%|██▍       | 55334/225000 [115:13:37<352:45:08,  7.48s/it]
 25%|██▍       | 55335/225000 [115:13:43<331:02:38,  7.02s/it]
 25%|██▍       | 55336/225000 [115:13:47<287:06:43,  6.09s/it]
 25%|██▍       | 55337/225000 [115:13:53<275:48:25,  5.85s/it]
 25%|██▍       | 55338/225000 [115:14:03<345:46:35,  7.34s/it]
 25%|██▍       | 55339/225000 [115:14:11<347:55:50,  7.38s/it]
 25%|██▍       | 55340/225000 [115:14:15<304:59:25,  6.47s/it]
 25%|██▍       | 55341/225000 [115:14:23<327:04:36,  6.94s/it]
 25%|██▍       | 55342/225000 [115:14:30<322:59:55,  6.85s/it]
 25%|██▍       | 55343/225000 [115:14:34<286:39:30,  6.08s/it]
 25%|██▍       | 55344/225000 [115:14:44<345:05:57,  7.32s/it]
 25%|██▍       | 55345/225000 [115:14:52<343:57:39,  7.30s/it]
 25%|██▍       | 55346/225000 [115:15:01<369:59:58,  7.85s/it]
 25%|██▍       | 55347/225000 [115:15:05<322:46:28,  6.85s/it]
 25%|██▍       | 55348/225000 [115:15:12<318:38:48,  6.76s/it]
 25%|██▍       | 55349/225000 [115:15:16<282:32:48,  6.00s/it]
 25%|██▍       | 55350/225000 [115:15:21<274:34:21,  5.83s/it]
                                                              
{'loss': '0.741', 'grad_norm': '0.7519', 'learning_rate': '4.472e-05', 'epoch': '0.246', 'num_input_tokens_seen': 192242368, 'train_runtime': '4.149e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55350/225000 [115:15:21<274:34:21,  5.83s/it]
 25%|██▍       | 55351/225000 [115:15:27<269:49:39,  5.73s/it]
 25%|██▍       | 55352/225000 [115:15:33<274:43:02,  5.83s/it]
 25%|██▍       | 55353/225000 [115:15:39<271:43:08,  5.77s/it]
 25%|██▍       | 55354/225000 [115:15:53<390:47:07,  8.29s/it]
 25%|██▍       | 55355/225000 [115:15:59<357:08:44,  7.58s/it]
 25%|██▍       | 55356/225000 [115:16:07<371:27:59,  7.88s/it]
 25%|██▍       | 55357/225000 [115:16:12<331:56:45,  7.04s/it]
 25%|██▍       | 55358/225000 [115:16:18<313:46:43,  6.66s/it]
 25%|██▍       | 55359/225000 [115:16:24<298:36:11,  6.34s/it]
 25%|██▍       | 55360/225000 [115:16:30<295:34:03,  6.27s/it]
 25%|██▍       | 55361/225000 [115:16:35<285:16:25,  6.05s/it]
 25%|██▍       | 55362/225000 [115:16:41<275:49:37,  5.85s/it]
 25%|██▍       | 55363/225000 [115:16:50<318:09:29,  6.75s/it]
 25%|██▍       | 55364/225000 [115:16:56<310:51:28,  6.60s/it]
 25%|██▍       | 55365/225000 [115:17:05<345:43:04,  7.34s/it]
 25%|██▍       | 55366/225000 [115:17:13<352:14:56,  7.48s/it]
 25%|██▍       | 55367/225000 [115:17:18<314:29:56,  6.67s/it]
 25%|██▍       | 55368/225000 [115:17:29<377:45:25,  8.02s/it]
 25%|██▍       | 55369/225000 [115:17:33<325:57:11,  6.92s/it]
 25%|██▍       | 55370/225000 [115:17:41<339:31:28,  7.21s/it]
 25%|██▍       | 55371/225000 [115:17:49<345:22:05,  7.33s/it]
 25%|██▍       | 55372/225000 [115:17:53<303:56:42,  6.45s/it]
 25%|██▍       | 55373/225000 [115:17:59<299:01:59,  6.35s/it]
 25%|██▍       | 55374/225000 [115:18:04<284:51:13,  6.05s/it]
 25%|██▍       | 55375/225000 [115:18:10<283:02:21,  6.01s/it]
 25%|██▍       | 55376/225000 [115:18:20<330:23:02,  7.01s/it]
 25%|██▍       | 55377/225000 [115:18:25<307:58:12,  6.54s/it]
 25%|██▍       | 55378/225000 [115:18:30<290:41:32,  6.17s/it]
 25%|██▍       | 55379/225000 [115:18:36<283:26:08,  6.02s/it]
 25%|██▍       | 55380/225000 [115:18:42<282:44:01,  6.00s/it]
 25%|██▍       | 55381/225000 [115:18:48<276:16:23,  5.86s/it]
 25%|██▍       | 55382/225000 [115:18:57<329:27:31,  6.99s/it]
 25%|██▍       | 55383/225000 [115:19:07<371:44:14,  7.89s/it]
 25%|██▍       | 55384/225000 [115:19:12<332:16:34,  7.05s/it]
 25%|██▍       | 55385/225000 [115:19:17<301:42:41,  6.40s/it]
 25%|██▍       | 55386/225000 [115:19:22<276:27:51,  5.87s/it]
 25%|██▍       | 55387/225000 [115:19:30<314:25:18,  6.67s/it]
 25%|██▍       | 55388/225000 [115:19:39<347:02:30,  7.37s/it]
 25%|██▍       | 55389/225000 [115:19:44<310:20:35,  6.59s/it]
 25%|██▍       | 55390/225000 [115:19:55<364:37:51,  7.74s/it]
 25%|██▍       | 55391/225000 [115:20:00<332:25:41,  7.06s/it]
 25%|██▍       | 55392/225000 [115:20:06<316:37:55,  6.72s/it]
 25%|██▍       | 55393/225000 [115:20:14<333:28:11,  7.08s/it]
 25%|██▍       | 55394/225000 [115:20:20<320:03:26,  6.79s/it]
 25%|██▍       | 55395/225000 [115:20:27<324:24:06,  6.89s/it]
 25%|██▍       | 55396/225000 [115:20:32<297:58:08,  6.32s/it]
 25%|██▍       | 55397/225000 [115:20:37<282:24:18,  5.99s/it]
 25%|██▍       | 55398/225000 [115:20:43<271:33:30,  5.76s/it]
 25%|██▍       | 55399/225000 [115:20:48<265:38:22,  5.64s/it]
 25%|██▍       | 55400/225000 [115:21:02<378:45:57,  8.04s/it]
                                                              
{'loss': '0.7113', 'grad_norm': '0.8715', 'learning_rate': '4.47e-05', 'epoch': '0.2462', 'num_input_tokens_seen': 192393648, 'train_runtime': '4.153e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55400/225000 [115:21:02<378:45:57,  8.04s/it]
 25%|██▍       | 55401/225000 [115:21:08<352:10:35,  7.48s/it]
 25%|██▍       | 55402/225000 [115:21:12<311:49:22,  6.62s/it]
 25%|██▍       | 55403/225000 [115:21:17<280:52:59,  5.96s/it]
 25%|██▍       | 55404/225000 [115:21:28<360:45:45,  7.66s/it]
 25%|██▍       | 55405/225000 [115:21:33<322:56:17,  6.86s/it]
 25%|██▍       | 55406/225000 [115:21:41<338:57:22,  7.20s/it]
 25%|██▍       | 55407/225000 [115:21:46<298:16:34,  6.33s/it]
 25%|██▍       | 55408/225000 [115:21:54<322:52:51,  6.85s/it]
 25%|██▍       | 55409/225000 [115:21:59<293:18:47,  6.23s/it]
 25%|██▍       | 55410/225000 [115:22:09<358:05:13,  7.60s/it]
 25%|██▍       | 55411/225000 [115:22:15<328:08:57,  6.97s/it]
 25%|██▍       | 55412/225000 [115:22:20<306:11:02,  6.50s/it]
 25%|██▍       | 55413/225000 [115:22:30<347:14:51,  7.37s/it]
 25%|██▍       | 55414/225000 [115:22:34<299:48:13,  6.36s/it]
 25%|██▍       | 55415/225000 [115:22:43<337:42:31,  7.17s/it]
 25%|██▍       | 55416/225000 [115:22:47<302:21:38,  6.42s/it]
 25%|██▍       | 55417/225000 [115:22:55<321:34:58,  6.83s/it]
 25%|██▍       | 55418/225000 [115:22:59<281:53:37,  5.98s/it]
 25%|██▍       | 55419/225000 [115:23:08<324:32:32,  6.89s/it]
 25%|██▍       | 55420/225000 [115:23:13<300:52:21,  6.39s/it]
 25%|██▍       | 55421/225000 [115:23:19<283:50:11,  6.03s/it]
 25%|██▍       | 55422/225000 [115:23:24<271:05:42,  5.76s/it]
 25%|██▍       | 55423/225000 [115:23:29<266:18:24,  5.65s/it]
 25%|██▍       | 55424/225000 [115:23:35<274:15:52,  5.82s/it]
 25%|██▍       | 55425/225000 [115:23:49<388:01:36,  8.24s/it]
 25%|██▍       | 55426/225000 [115:23:53<331:31:15,  7.04s/it]
 25%|██▍       | 55427/225000 [115:23:58<293:51:09,  6.24s/it]
 25%|██▍       | 55428/225000 [115:24:09<361:06:43,  7.67s/it]
 25%|██▍       | 55429/225000 [115:24:16<359:23:53,  7.63s/it]
 25%|██▍       | 55430/225000 [115:24:21<315:07:03,  6.69s/it]
 25%|██▍       | 55431/225000 [115:24:26<294:48:34,  6.26s/it]
 25%|██▍       | 55432/225000 [115:24:31<282:38:28,  6.00s/it]
 25%|██▍       | 55433/225000 [115:24:36<265:25:11,  5.64s/it]
 25%|██▍       | 55434/225000 [115:24:43<287:02:08,  6.09s/it]
 25%|██▍       | 55435/225000 [115:24:56<385:01:31,  8.17s/it]
 25%|██▍       | 55436/225000 [115:25:01<330:10:48,  7.01s/it]
 25%|██▍       | 55437/225000 [115:25:11<374:38:13,  7.95s/it]
 25%|██▍       | 55438/225000 [115:25:15<317:12:14,  6.73s/it]
 25%|██▍       | 55439/225000 [115:25:22<329:17:35,  6.99s/it]
 25%|██▍       | 55440/225000 [115:25:30<334:29:08,  7.10s/it]
 25%|██▍       | 55441/225000 [115:25:37<331:07:42,  7.03s/it]
 25%|██▍       | 55442/225000 [115:25:42<302:47:53,  6.43s/it]
 25%|██▍       | 55443/225000 [115:25:50<327:04:06,  6.94s/it]
 25%|██▍       | 55444/225000 [115:25:58<341:57:46,  7.26s/it]
 25%|██▍       | 55445/225000 [115:26:02<301:41:42,  6.41s/it]
 25%|██▍       | 55446/225000 [115:26:06<268:38:32,  5.70s/it]
 25%|██▍       | 55447/225000 [115:26:15<308:03:40,  6.54s/it]
 25%|██▍       | 55448/225000 [115:26:19<279:06:22,  5.93s/it]
 25%|██▍       | 55449/225000 [115:26:29<332:12:40,  7.05s/it]
 25%|██▍       | 55450/225000 [115:26:37<343:06:13,  7.29s/it]
                                                              
{'loss': '0.7047', 'grad_norm': '0.7033', 'learning_rate': '4.469e-05', 'epoch': '0.2464', 'num_input_tokens_seen': 192543488, 'train_runtime': '4.156e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55450/225000 [115:26:37<343:06:13,  7.29s/it]
 25%|██▍       | 55451/225000 [115:26:42<313:04:18,  6.65s/it]
 25%|██▍       | 55452/225000 [115:26:48<298:27:29,  6.34s/it]
 25%|██▍       | 55453/225000 [115:26:53<281:06:59,  5.97s/it]
 25%|██▍       | 55454/225000 [115:26:58<274:58:41,  5.84s/it]
 25%|██▍       | 55455/225000 [115:27:03<265:45:38,  5.64s/it]
 25%|██▍       | 55456/225000 [115:27:17<372:47:31,  7.92s/it]
 25%|██▍       | 55457/225000 [115:27:21<319:29:46,  6.78s/it]
 25%|██▍       | 55458/225000 [115:27:25<286:23:03,  6.08s/it]
 25%|██▍       | 55459/225000 [115:27:35<342:52:32,  7.28s/it]
 25%|██▍       | 55460/225000 [115:27:39<297:08:08,  6.31s/it]
 25%|██▍       | 55461/225000 [115:27:45<282:31:12,  6.00s/it]
 25%|██▍       | 55462/225000 [115:27:55<344:24:59,  7.31s/it]
 25%|██▍       | 55463/225000 [115:27:59<294:22:37,  6.25s/it]
 25%|██▍       | 55464/225000 [115:28:05<297:07:13,  6.31s/it]
 25%|██▍       | 55465/225000 [115:28:10<282:39:00,  6.00s/it]
 25%|██▍       | 55466/225000 [115:28:17<288:12:28,  6.12s/it]
 25%|██▍       | 55467/225000 [115:28:23<287:13:30,  6.10s/it]
 25%|██▍       | 55468/225000 [115:28:36<388:19:06,  8.25s/it]
 25%|██▍       | 55469/225000 [115:28:41<334:42:30,  7.11s/it]
 25%|██▍       | 55470/225000 [115:28:50<363:31:54,  7.72s/it]
 25%|██▍       | 55471/225000 [115:28:54<309:03:31,  6.56s/it]
 25%|██▍       | 55472/225000 [115:29:03<354:30:04,  7.53s/it]
 25%|██▍       | 55473/225000 [115:29:07<301:49:47,  6.41s/it]
 25%|██▍       | 55474/225000 [115:29:18<361:25:15,  7.68s/it]
 25%|██▍       | 55475/225000 [115:29:22<312:22:40,  6.63s/it]
 25%|██▍       | 55476/225000 [115:29:26<278:18:11,  5.91s/it]
 25%|██▍       | 55477/225000 [115:29:33<294:29:45,  6.25s/it]
 25%|██▍       | 55478/225000 [115:29:43<339:01:26,  7.20s/it]
 25%|██▍       | 55479/225000 [115:29:49<321:26:28,  6.83s/it]
 25%|██▍       | 55480/225000 [115:29:56<324:29:51,  6.89s/it]
 25%|██▍       | 55481/225000 [115:30:00<293:28:07,  6.23s/it]
 25%|██▍       | 55482/225000 [115:30:05<273:53:07,  5.82s/it]
 25%|██▍       | 55483/225000 [115:30:17<362:27:59,  7.70s/it]
 25%|██▍       | 55484/225000 [115:30:22<323:50:47,  6.88s/it]
 25%|██▍       | 55485/225000 [115:30:27<297:24:53,  6.32s/it]
 25%|██▍       | 55486/225000 [115:30:32<274:38:25,  5.83s/it]
 25%|██▍       | 55487/225000 [115:30:39<291:15:40,  6.19s/it]
 25%|██▍       | 55488/225000 [115:30:52<383:08:21,  8.14s/it]
 25%|██▍       | 55489/225000 [115:30:59<368:12:50,  7.82s/it]
 25%|██▍       | 55490/225000 [115:31:03<317:28:04,  6.74s/it]
 25%|██▍       | 55491/225000 [115:31:07<279:17:12,  5.93s/it]
 25%|██▍       | 55492/225000 [115:31:11<255:04:42,  5.42s/it]
 25%|██▍       | 55493/225000 [115:31:23<340:47:20,  7.24s/it]
 25%|██▍       | 55494/225000 [115:31:27<299:54:51,  6.37s/it]
 25%|██▍       | 55495/225000 [115:31:37<347:47:40,  7.39s/it]
 25%|██▍       | 55496/225000 [115:31:41<308:24:15,  6.55s/it]
 25%|██▍       | 55497/225000 [115:31:49<325:23:47,  6.91s/it]
 25%|██▍       | 55498/225000 [115:31:55<309:31:06,  6.57s/it]
 25%|██▍       | 55499/225000 [115:32:02<319:34:13,  6.79s/it]
 25%|██▍       | 55500/225000 [115:32:12<361:36:09,  7.68s/it]
                                                              
{'loss': '0.7125', 'grad_norm': '0.6226', 'learning_rate': '4.468e-05', 'epoch': '0.2467', 'num_input_tokens_seen': 192694144, 'train_runtime': '4.159e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55500/225000 [115:32:12<361:36:09,  7.68s/it]
 25%|██▍       | 55501/225000 [115:32:16<313:45:13,  6.66s/it]
 25%|██▍       | 55502/225000 [115:32:21<282:09:59,  5.99s/it]
 25%|██▍       | 55503/225000 [115:32:26<270:08:05,  5.74s/it]
 25%|██▍       | 55504/225000 [115:32:36<332:49:39,  7.07s/it]
 25%|██▍       | 55505/225000 [115:32:43<330:31:10,  7.02s/it]
 25%|██▍       | 55506/225000 [115:32:47<287:18:20,  6.10s/it]
 25%|██▍       | 55507/225000 [115:32:55<315:56:49,  6.71s/it]
 25%|██▍       | 55508/225000 [115:33:00<284:50:36,  6.05s/it]
 25%|██▍       | 55509/225000 [115:33:08<319:53:16,  6.79s/it]
 25%|██▍       | 55510/225000 [115:33:15<321:59:46,  6.84s/it]
 25%|██▍       | 55511/225000 [115:33:22<323:51:42,  6.88s/it]
 25%|██▍       | 55512/225000 [115:33:29<320:12:22,  6.80s/it]
 25%|██▍       | 55513/225000 [115:33:35<320:24:57,  6.81s/it]
 25%|██▍       | 55514/225000 [115:33:41<297:24:21,  6.32s/it]
 25%|██▍       | 55515/225000 [115:33:46<277:57:34,  5.90s/it]
 25%|██▍       | 55516/225000 [115:33:50<261:17:13,  5.55s/it]
 25%|██▍       | 55517/225000 [115:34:03<365:40:32,  7.77s/it]
 25%|██▍       | 55518/225000 [115:34:07<308:39:08,  6.56s/it]
 25%|██▍       | 55519/225000 [115:34:18<365:41:20,  7.77s/it]
 25%|██▍       | 55520/225000 [115:34:24<351:35:16,  7.47s/it]
 25%|██▍       | 55521/225000 [115:34:28<299:19:04,  6.36s/it]
 25%|██▍       | 55522/225000 [115:34:37<339:10:21,  7.20s/it]
 25%|██▍       | 55523/225000 [115:34:42<303:14:51,  6.44s/it]
 25%|██▍       | 55524/225000 [115:34:51<338:48:52,  7.20s/it]
 25%|██▍       | 55525/225000 [115:34:55<301:46:30,  6.41s/it]
 25%|██▍       | 55526/225000 [115:35:01<285:17:36,  6.06s/it]
 25%|██▍       | 55527/225000 [115:35:06<275:06:02,  5.84s/it]
 25%|██▍       | 55528/225000 [115:35:12<271:26:11,  5.77s/it]
 25%|██▍       | 55529/225000 [115:35:17<265:57:32,  5.65s/it]
 25%|██▍       | 55530/225000 [115:35:22<259:55:07,  5.52s/it]
 25%|██▍       | 55531/225000 [115:35:36<380:27:37,  8.08s/it]
 25%|██▍       | 55532/225000 [115:35:43<356:46:57,  7.58s/it]
 25%|██▍       | 55533/225000 [115:35:47<317:05:53,  6.74s/it]
 25%|██▍       | 55534/225000 [115:35:52<289:11:36,  6.14s/it]
 25%|██▍       | 55535/225000 [115:36:04<362:51:28,  7.71s/it]
 25%|██▍       | 55536/225000 [115:36:08<316:01:56,  6.71s/it]
 25%|██▍       | 55537/225000 [115:36:15<320:25:28,  6.81s/it]
 25%|██▍       | 55538/225000 [115:36:21<315:40:46,  6.71s/it]
 25%|██▍       | 55539/225000 [115:36:26<288:13:06,  6.12s/it]
 25%|██▍       | 55540/225000 [115:36:31<272:27:28,  5.79s/it]
 25%|██▍       | 55541/225000 [115:36:38<286:10:37,  6.08s/it]
 25%|██▍       | 55542/225000 [115:36:50<365:13:00,  7.76s/it]
 25%|██▍       | 55543/225000 [115:36:55<325:48:45,  6.92s/it]
 25%|██▍       | 55544/225000 [115:37:02<327:33:56,  6.96s/it]
 25%|██▍       | 55545/225000 [115:37:10<342:27:17,  7.28s/it]
 25%|██▍       | 55546/225000 [115:37:17<335:36:04,  7.13s/it]
 25%|██▍       | 55547/225000 [115:37:23<326:46:04,  6.94s/it]
 25%|██▍       | 55548/225000 [115:37:27<284:24:48,  6.04s/it]
 25%|██▍       | 55549/225000 [115:37:32<268:25:45,  5.70s/it]
 25%|██▍       | 55550/225000 [115:37:40<304:36:09,  6.47s/it]
                                                              
{'loss': '0.7076', 'grad_norm': '0.7344', 'learning_rate': '4.467e-05', 'epoch': '0.2469', 'num_input_tokens_seen': 192843832, 'train_runtime': '4.163e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55550/225000 [115:37:40<304:36:09,  6.47s/it]
 25%|██▍       | 55551/225000 [115:37:52<382:05:47,  8.12s/it]
 25%|██▍       | 55552/225000 [115:37:56<315:46:20,  6.71s/it]
 25%|██▍       | 55553/225000 [115:38:01<297:17:24,  6.32s/it]
 25%|██▍       | 55554/225000 [115:38:07<289:40:45,  6.15s/it]
 25%|██▍       | 55555/225000 [115:38:12<276:19:41,  5.87s/it]
 25%|██▍       | 55556/225000 [115:38:19<298:55:51,  6.35s/it]
 25%|██▍       | 55557/225000 [115:38:32<381:46:56,  8.11s/it]
 25%|██▍       | 55558/225000 [115:38:39<365:25:31,  7.76s/it]
 25%|██▍       | 55559/225000 [115:38:45<343:36:58,  7.30s/it]
 25%|██▍       | 55560/225000 [115:38:51<327:30:24,  6.96s/it]
 25%|██▍       | 55561/225000 [115:38:57<308:57:11,  6.56s/it]
 25%|██▍       | 55562/225000 [115:39:03<300:34:17,  6.39s/it]
 25%|██▍       | 55563/225000 [115:39:12<347:38:21,  7.39s/it]
 25%|██▍       | 55564/225000 [115:39:16<300:42:49,  6.39s/it]
 25%|██▍       | 55565/225000 [115:39:21<271:04:35,  5.76s/it]
 25%|██▍       | 55566/225000 [115:39:28<295:42:48,  6.28s/it]
 25%|██▍       | 55567/225000 [115:39:37<330:46:51,  7.03s/it]
 25%|██▍       | 55568/225000 [115:39:41<293:01:08,  6.23s/it]
 25%|██▍       | 55569/225000 [115:39:50<325:03:54,  6.91s/it]
 25%|██▍       | 55570/225000 [115:39:56<314:45:57,  6.69s/it]
 25%|██▍       | 55571/225000 [115:40:02<311:45:34,  6.62s/it]
 25%|██▍       | 55572/225000 [115:40:10<328:57:23,  6.99s/it]
 25%|██▍       | 55573/225000 [115:40:15<296:37:30,  6.30s/it]
 25%|██▍       | 55574/225000 [115:40:19<269:51:05,  5.73s/it]
 25%|██▍       | 55575/225000 [115:40:26<280:47:40,  5.97s/it]
 25%|██▍       | 55576/225000 [115:40:36<336:00:14,  7.14s/it]
 25%|██▍       | 55577/225000 [115:40:44<354:56:07,  7.54s/it]
 25%|██▍       | 55578/225000 [115:40:48<298:22:14,  6.34s/it]
 25%|██▍       | 55579/225000 [115:40:53<286:22:36,  6.09s/it]
 25%|██▍       | 55580/225000 [115:40:59<279:22:57,  5.94s/it]
 25%|██▍       | 55581/225000 [115:41:03<260:24:01,  5.53s/it]
 25%|██▍       | 55582/225000 [115:41:09<261:31:13,  5.56s/it]
 25%|██▍       | 55583/225000 [115:41:19<322:44:58,  6.86s/it]
 25%|██▍       | 55584/225000 [115:41:29<375:01:11,  7.97s/it]
 25%|██▍       | 55585/225000 [115:41:33<315:19:58,  6.70s/it]
 25%|██▍       | 55586/225000 [115:41:43<358:06:04,  7.61s/it]
 25%|██▍       | 55587/225000 [115:41:51<368:02:00,  7.82s/it]
 25%|██▍       | 55588/225000 [115:41:55<307:43:47,  6.54s/it]
 25%|██▍       | 55589/225000 [115:42:03<326:34:28,  6.94s/it]
 25%|██▍       | 55590/225000 [115:42:07<290:22:12,  6.17s/it]
 25%|██▍       | 55591/225000 [115:42:15<316:23:13,  6.72s/it]
 25%|██▍       | 55592/225000 [115:42:21<311:19:12,  6.62s/it]
 25%|██▍       | 55593/225000 [115:42:27<297:46:37,  6.33s/it]
 25%|██▍       | 55594/225000 [115:42:35<320:08:59,  6.80s/it]
 25%|██▍       | 55595/225000 [115:42:39<283:18:18,  6.02s/it]
 25%|██▍       | 55596/225000 [115:42:48<320:55:27,  6.82s/it]
 25%|██▍       | 55597/225000 [115:42:52<279:50:59,  5.95s/it]
 25%|██▍       | 55598/225000 [115:42:57<270:40:01,  5.75s/it]
 25%|██▍       | 55599/225000 [115:43:02<261:34:07,  5.56s/it]
 25%|██▍       | 55600/225000 [115:43:15<366:13:59,  7.78s/it]
                                                              
{'loss': '0.7276', 'grad_norm': '0.8197', 'learning_rate': '4.466e-05', 'epoch': '0.2471', 'num_input_tokens_seen': 192993640, 'train_runtime': '4.166e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55600/225000 [115:43:15<366:13:59,  7.78s/it]
 25%|██▍       | 55601/225000 [115:43:20<318:05:31,  6.76s/it]
 25%|██▍       | 55602/225000 [115:43:29<351:07:00,  7.46s/it]
 25%|██▍       | 55603/225000 [115:43:34<317:45:31,  6.75s/it]
 25%|██▍       | 55604/225000 [115:43:40<308:10:51,  6.55s/it]
 25%|██▍       | 55605/225000 [115:43:45<293:34:40,  6.24s/it]
 25%|██▍       | 55606/225000 [115:43:54<329:56:08,  7.01s/it]
 25%|██▍       | 55607/225000 [115:44:02<337:32:14,  7.17s/it]
 25%|██▍       | 55608/225000 [115:44:08<325:53:43,  6.93s/it]
 25%|██▍       | 55609/225000 [115:44:12<288:46:22,  6.14s/it]
 25%|██▍       | 55610/225000 [115:44:17<266:52:37,  5.67s/it]
 25%|██▍       | 55611/225000 [115:44:28<337:04:29,  7.16s/it]
 25%|██▍       | 55612/225000 [115:44:35<343:20:59,  7.30s/it]
 25%|██▍       | 55613/225000 [115:44:39<298:27:19,  6.34s/it]
 25%|██▍       | 55614/225000 [115:44:50<355:26:16,  7.55s/it]
 25%|██▍       | 55615/225000 [115:44:54<306:27:22,  6.51s/it]
 25%|██▍       | 55616/225000 [115:45:01<318:47:02,  6.78s/it]
 25%|██▍       | 55617/225000 [115:45:07<312:19:24,  6.64s/it]
 25%|██▍       | 55618/225000 [115:45:15<322:28:39,  6.85s/it]
 25%|██▍       | 55619/225000 [115:45:21<315:15:07,  6.70s/it]
 25%|██▍       | 55620/225000 [115:45:28<310:54:18,  6.61s/it]
 25%|██▍       | 55621/225000 [115:45:34<307:50:24,  6.54s/it]
 25%|██▍       | 55622/225000 [115:45:38<276:21:59,  5.87s/it]
 25%|██▍       | 55623/225000 [115:45:43<267:01:51,  5.68s/it]
 25%|██▍       | 55624/225000 [115:45:49<258:46:24,  5.50s/it]
 25%|██▍       | 55625/225000 [115:45:57<305:00:12,  6.48s/it]
 25%|██▍       | 55626/225000 [115:46:10<396:56:48,  8.44s/it]
 25%|██▍       | 55627/225000 [115:46:15<337:21:28,  7.17s/it]
 25%|██▍       | 55628/225000 [115:46:18<291:16:35,  6.19s/it]
 25%|██▍       | 55629/225000 [115:46:27<317:41:23,  6.75s/it]
 25%|██▍       | 55630/225000 [115:46:34<328:25:26,  6.98s/it]
 25%|██▍       | 55631/225000 [115:46:41<324:28:21,  6.90s/it]
 25%|██▍       | 55632/225000 [115:46:47<311:03:42,  6.61s/it]
 25%|██▍       | 55633/225000 [115:46:53<303:29:08,  6.45s/it]
 25%|██▍       | 55634/225000 [115:46:59<294:49:34,  6.27s/it]
 25%|██▍       | 55635/225000 [115:47:09<347:05:55,  7.38s/it]
 25%|██▍       | 55636/225000 [115:47:13<307:44:15,  6.54s/it]
 25%|██▍       | 55637/225000 [115:47:18<282:46:25,  6.01s/it]
 25%|██▍       | 55638/225000 [115:47:27<323:06:36,  6.87s/it]
 25%|██▍       | 55639/225000 [115:47:33<317:58:28,  6.76s/it]
 25%|██▍       | 55640/225000 [115:47:38<294:56:05,  6.27s/it]
 25%|██▍       | 55641/225000 [115:47:44<283:59:47,  6.04s/it]
 25%|██▍       | 55642/225000 [115:47:49<272:56:44,  5.80s/it]
 25%|██▍       | 55643/225000 [115:48:01<356:10:05,  7.57s/it]
 25%|██▍       | 55644/225000 [115:48:05<301:49:15,  6.42s/it]
 25%|██▍       | 55645/225000 [115:48:14<347:58:22,  7.40s/it]
 25%|██▍       | 55646/225000 [115:48:19<312:50:24,  6.65s/it]
 25%|██▍       | 55647/225000 [115:48:29<362:26:23,  7.70s/it]
 25%|██▍       | 55648/225000 [115:48:33<310:13:49,  6.59s/it]
 25%|██▍       | 55649/225000 [115:48:39<301:13:42,  6.40s/it]
 25%|██▍       | 55650/225000 [115:48:48<327:28:22,  6.96s/it]
                                                              
{'loss': '0.7327', 'grad_norm': '0.6922', 'learning_rate': '4.465e-05', 'epoch': '0.2473', 'num_input_tokens_seen': 193144968, 'train_runtime': '4.169e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55650/225000 [115:48:48<327:28:22,  6.96s/it]
 25%|██▍       | 55651/225000 [115:48:52<289:56:31,  6.16s/it]
 25%|██▍       | 55652/225000 [115:48:57<281:58:44,  5.99s/it]
 25%|██▍       | 55653/225000 [115:49:03<279:47:42,  5.95s/it]
 25%|██▍       | 55654/225000 [115:49:12<311:51:22,  6.63s/it]
 25%|██▍       | 55655/225000 [115:49:18<313:48:44,  6.67s/it]
 25%|██▍       | 55656/225000 [115:49:29<366:25:03,  7.79s/it]
 25%|██▍       | 55657/225000 [115:49:34<336:53:31,  7.16s/it]
 25%|██▍       | 55658/225000 [115:49:39<295:16:40,  6.28s/it]
 25%|██▍       | 55659/225000 [115:49:48<343:49:49,  7.31s/it]
 25%|██▍       | 55660/225000 [115:49:53<304:46:56,  6.48s/it]
 25%|██▍       | 55661/225000 [115:50:02<341:30:45,  7.26s/it]
 25%|██▍       | 55662/225000 [115:50:10<352:29:27,  7.49s/it]
 25%|██▍       | 55663/225000 [115:50:14<297:25:59,  6.32s/it]
 25%|██▍       | 55664/225000 [115:50:21<310:43:59,  6.61s/it]
 25%|██▍       | 55665/225000 [115:50:26<290:02:19,  6.17s/it]
 25%|██▍       | 55666/225000 [115:50:31<278:13:15,  5.91s/it]
 25%|██▍       | 55667/225000 [115:50:37<272:19:49,  5.79s/it]
 25%|██▍       | 55668/225000 [115:50:42<267:29:08,  5.69s/it]
 25%|██▍       | 55669/225000 [115:50:48<261:04:20,  5.55s/it]
 25%|██▍       | 55670/225000 [115:51:01<370:51:38,  7.88s/it]
 25%|██▍       | 55671/225000 [115:51:08<353:54:05,  7.52s/it]
 25%|██▍       | 55672/225000 [115:51:12<311:41:15,  6.63s/it]
 25%|██▍       | 55673/225000 [115:51:17<283:34:35,  6.03s/it]
 25%|██▍       | 55674/225000 [115:51:29<365:46:24,  7.78s/it]
 25%|██▍       | 55675/225000 [115:51:35<343:51:44,  7.31s/it]
 25%|██▍       | 55676/225000 [115:51:40<315:56:17,  6.72s/it]
 25%|██▍       | 55677/225000 [115:51:45<289:37:59,  6.16s/it]
 25%|██▍       | 55678/225000 [115:51:50<270:48:55,  5.76s/it]
 25%|██▍       | 55679/225000 [115:51:59<326:16:59,  6.94s/it]
 25%|██▍       | 55680/225000 [115:52:07<336:05:27,  7.15s/it]
 25%|██▍       | 55681/225000 [115:52:11<288:00:23,  6.12s/it]
 25%|██▍       | 55682/225000 [115:52:21<342:24:05,  7.28s/it]
 25%|██▍       | 55683/225000 [115:52:25<297:50:40,  6.33s/it]
 25%|██▍       | 55684/225000 [115:52:29<269:30:25,  5.73s/it]
 25%|██▍       | 55685/225000 [115:52:38<316:54:23,  6.74s/it]
 25%|██▍       | 55686/225000 [115:52:42<279:32:12,  5.94s/it]
 25%|██▍       | 55687/225000 [115:52:52<336:55:20,  7.16s/it]
 25%|██▍       | 55688/225000 [115:52:59<322:12:48,  6.85s/it]
 25%|██▍       | 55689/225000 [115:53:04<307:54:01,  6.55s/it]
 25%|██▍       | 55690/225000 [115:53:09<279:12:04,  5.94s/it]
 25%|██▍       | 55691/225000 [115:53:14<270:01:36,  5.74s/it]
 25%|██▍       | 55692/225000 [115:53:19<256:11:56,  5.45s/it]
 25%|██▍       | 55693/225000 [115:53:31<348:07:31,  7.40s/it]
 25%|██▍       | 55694/225000 [115:53:38<349:28:29,  7.43s/it]
 25%|██▍       | 55695/225000 [115:53:42<301:03:32,  6.40s/it]
 25%|██▍       | 55696/225000 [115:53:52<352:33:25,  7.50s/it]
 25%|██▍       | 55697/225000 [115:53:56<301:05:40,  6.40s/it]
 25%|██▍       | 55698/225000 [115:54:01<277:29:12,  5.90s/it]
 25%|██▍       | 55699/225000 [115:54:06<261:45:26,  5.57s/it]
 25%|██▍       | 55700/225000 [115:54:17<336:56:39,  7.16s/it]
                                                              
{'loss': '0.728', 'grad_norm': '0.8514', 'learning_rate': '4.464e-05', 'epoch': '0.2476', 'num_input_tokens_seen': 193294952, 'train_runtime': '4.173e+05', 'train_tokens_per_second': '463.3'}

 25%|██▍       | 55700/225000 [115:54:17<336:56:39,  7.16s/it]
 25%|██▍       | 55701/225000 [115:54:23<318:04:47,  6.76s/it]
 25%|██▍       | 55702/225000 [115:54:32<349:34:32,  7.43s/it]
 25%|██▍       | 55703/225000 [115:54:38<330:34:23,  7.03s/it]
 25%|██▍       | 55704/225000 [115:54:42<296:39:13,  6.31s/it]
 25%|██▍       | 55705/225000 [115:54:52<346:09:47,  7.36s/it]
 25%|██▍       | 55706/225000 [115:54:56<295:54:36,  6.29s/it]
 25%|██▍       | 55707/225000 [115:55:04<323:50:18,  6.89s/it]
 25%|██▍       | 55708/225000 [115:55:12<343:15:30,  7.30s/it]
 25%|██▍       | 55709/225000 [115:55:16<295:20:06,  6.28s/it]
 25%|██▍       | 55710/225000 [115:55:26<339:24:28,  7.22s/it]
 25%|██▍       | 55711/225000 [115:55:31<314:47:29,  6.69s/it]
 25%|██▍       | 55712/225000 [115:55:35<280:17:49,  5.96s/it]
 25%|██▍       | 55713/225000 [115:55:44<323:06:18,  6.87s/it]
 25%|██▍       | 55714/225000 [115:55:49<292:52:23,  6.23s/it]
 25%|██▍       | 55715/225000 [115:55:54<273:46:37,  5.82s/it]
 25%|██▍       | 55716/225000 [115:56:01<286:57:34,  6.10s/it]
 25%|██▍       | 55717/225000 [115:56:12<365:33:57,  7.77s/it]
 25%|██▍       | 55718/225000 [115:56:18<338:18:08,  7.19s/it]
 25%|██▍       | 55719/225000 [115:56:25<333:08:29,  7.08s/it]
 25%|██▍       | 55720/225000 [115:56:30<297:23:32,  6.32s/it]
 25%|██▍       | 55721/225000 [115:56:35<276:57:35,  5.89s/it]
 25%|██▍       | 55722/225000 [115:56:46<348:14:56,  7.41s/it]
 25%|██▍       | 55723/225000 [115:56:53<342:48:01,  7.29s/it]
 25%|██▍       | 55724/225000 [115:57:00<340:32:08,  7.24s/it]
 25%|██▍       | 55725/225000 [115:57:04<298:04:18,  6.34s/it]
 25%|██▍       | 55726/225000 [115:57:12<316:16:56,  6.73s/it]
 25%|██▍       | 55727/225000 [115:57:16<279:01:39,  5.93s/it]
 25%|██▍       | 55728/225000 [115:57:25<330:24:32,  7.03s/it]
 25%|██▍       | 55729/225000 [115:57:33<346:36:05,  7.37s/it]
 25%|██▍       | 55730/225000 [115:57:38<306:15:57,  6.51s/it]
 25%|██▍       | 55731/225000 [115:57:44<301:46:50,  6.42s/it]
 25%|██▍       | 55732/225000 [115:57:51<311:54:29,  6.63s/it]
 25%|██▍       | 55733/225000 [115:58:00<342:15:09,  7.28s/it]
 25%|██▍       | 55734/225000 [115:58:04<301:25:40,  6.41s/it]
 25%|██▍       | 55735/225000 [115:58:09<274:25:57,  5.84s/it]
 25%|██▍       | 55736/225000 [115:58:15<285:03:14,  6.06s/it]
 25%|██▍       | 55737/225000 [115:58:24<326:23:06,  6.94s/it]
 25%|██▍       | 55738/225000 [115:58:30<309:26:35,  6.58s/it]
 25%|██▍       | 55739/225000 [115:58:37<310:17:56,  6.60s/it]
 25%|██▍       | 55740/225000 [115:58:44<316:12:57,  6.73s/it]
 25%|██▍       | 55741/225000 [115:58:50<308:15:33,  6.56s/it]
 25%|██▍       | 55742/225000 [115:58:54<278:05:28,  5.91s/it]
 25%|██▍       | 55743/225000 [115:59:05<346:59:51,  7.38s/it]
 25%|██▍       | 55744/225000 [115:59:12<338:31:41,  7.20s/it]
 25%|██▍       | 55745/225000 [115:59:21<361:18:38,  7.68s/it]
 25%|██▍       | 55746/225000 [115:59:28<356:41:42,  7.59s/it]
 25%|██▍       | 55747/225000 [115:59:34<325:45:12,  6.93s/it]
 25%|██▍       | 55748/225000 [115:59:38<294:57:24,  6.27s/it]
 25%|██▍       | 55749/225000 [115:59:49<351:08:41,  7.47s/it]
 25%|██▍       | 55750/225000 [115:59:54<316:28:33,  6.73s/it]
                                                              
{'loss': '0.7185', 'grad_norm': '0.8563', 'learning_rate': '4.462e-05', 'epoch': '0.2478', 'num_input_tokens_seen': 193446808, 'train_runtime': '4.176e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 55750/225000 [115:59:54<316:28:33,  6.73s/it]
 25%|██▍       | 55751/225000 [115:59:58<282:27:18,  6.01s/it]
 25%|██▍       | 55752/225000 [116:00:09<356:14:43,  7.58s/it]
 25%|██▍       | 55753/225000 [116:00:15<338:26:31,  7.20s/it]
 25%|██▍       | 55754/225000 [116:00:21<316:48:53,  6.74s/it]
 25%|██▍       | 55755/225000 [116:00:26<295:19:59,  6.28s/it]
 25%|██▍       | 55756/225000 [116:00:32<281:22:56,  5.99s/it]
 25%|██▍       | 55757/225000 [116:00:37<270:00:00,  5.74s/it]
 25%|██▍       | 55758/225000 [116:00:42<261:04:14,  5.55s/it]
 25%|██▍       | 55759/225000 [116:00:48<264:56:27,  5.64s/it]
 25%|██▍       | 55760/225000 [116:00:56<303:03:14,  6.45s/it]
 25%|██▍       | 55761/225000 [116:01:10<402:51:32,  8.57s/it]
 25%|██▍       | 55762/225000 [116:01:14<342:33:06,  7.29s/it]
 25%|██▍       | 55763/225000 [116:01:18<301:17:22,  6.41s/it]
 25%|██▍       | 55764/225000 [116:01:26<325:40:24,  6.93s/it]
 25%|██▍       | 55765/225000 [116:01:35<352:23:07,  7.50s/it]
 25%|██▍       | 55766/225000 [116:01:39<303:44:59,  6.46s/it]
 25%|██▍       | 55767/225000 [116:01:49<353:59:28,  7.53s/it]
 25%|██▍       | 55768/225000 [116:01:56<338:30:35,  7.20s/it]
 25%|██▍       | 55769/225000 [116:02:02<320:03:04,  6.81s/it]
 25%|██▍       | 55770/225000 [116:02:08<308:02:49,  6.55s/it]
 25%|██▍       | 55771/225000 [116:02:14<304:55:32,  6.49s/it]
 25%|██▍       | 55772/225000 [116:02:21<314:36:40,  6.69s/it]
 25%|██▍       | 55773/225000 [116:02:30<350:33:35,  7.46s/it]
 25%|██▍       | 55774/225000 [116:02:37<337:49:05,  7.19s/it]
 25%|██▍       | 55775/225000 [116:02:42<314:45:20,  6.70s/it]
 25%|██▍       | 55776/225000 [116:02:47<284:28:41,  6.05s/it]
 25%|██▍       | 55777/225000 [116:02:51<259:32:15,  5.52s/it]
 25%|██▍       | 55778/225000 [116:03:02<328:53:55,  7.00s/it]
 25%|██▍       | 55779/225000 [116:03:06<287:09:41,  6.11s/it]
 25%|██▍       | 55780/225000 [116:03:11<280:29:09,  5.97s/it]
 25%|██▍       | 55781/225000 [116:03:21<335:20:50,  7.13s/it]
 25%|██▍       | 55782/225000 [116:03:25<294:23:50,  6.26s/it]
 25%|██▍       | 55783/225000 [116:03:35<345:35:59,  7.35s/it]
 25%|██▍       | 55784/225000 [116:03:40<311:50:38,  6.63s/it]
 25%|██▍       | 55785/225000 [116:03:46<296:36:26,  6.31s/it]
 25%|██▍       | 55786/225000 [116:03:52<291:23:31,  6.20s/it]
 25%|██▍       | 55787/225000 [116:03:57<281:03:35,  5.98s/it]
 25%|██▍       | 55788/225000 [116:04:03<276:38:37,  5.89s/it]
 25%|██▍       | 55789/225000 [116:04:08<271:23:36,  5.77s/it]
 25%|██▍       | 55790/225000 [116:04:14<263:26:02,  5.60s/it]
 25%|██▍       | 55791/225000 [116:04:27<365:54:35,  7.78s/it]
 25%|██▍       | 55792/225000 [116:04:33<352:37:08,  7.50s/it]
 25%|██▍       | 55793/225000 [116:04:38<311:12:29,  6.62s/it]
 25%|██▍       | 55794/225000 [116:04:46<335:00:52,  7.13s/it]
 25%|██▍       | 55795/225000 [116:04:53<336:06:31,  7.15s/it]
 25%|██▍       | 55796/225000 [116:04:59<313:29:01,  6.67s/it]
 25%|██▍       | 55797/225000 [116:05:07<326:14:29,  6.94s/it]
 25%|██▍       | 55798/225000 [116:05:13<311:50:23,  6.63s/it]
 25%|██▍       | 55799/225000 [116:05:18<300:46:50,  6.40s/it]
 25%|██▍       | 55800/225000 [116:05:24<295:43:19,  6.29s/it]
                                                              
{'loss': '0.7175', 'grad_norm': '0.8055', 'learning_rate': '4.461e-05', 'epoch': '0.248', 'num_input_tokens_seen': 193596520, 'train_runtime': '4.179e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 55800/225000 [116:05:24<295:43:19,  6.29s/it]
 25%|██▍       | 55801/225000 [116:05:30<283:24:04,  6.03s/it]
 25%|██▍       | 55802/225000 [116:05:36<280:40:53,  5.97s/it]
 25%|██▍       | 55803/225000 [116:05:41<276:10:43,  5.88s/it]
 25%|██▍       | 55804/225000 [116:05:47<266:59:41,  5.68s/it]
 25%|██▍       | 55805/225000 [116:05:58<343:18:26,  7.30s/it]
 25%|██▍       | 55806/225000 [116:06:10<408:31:30,  8.69s/it]
 25%|██▍       | 55807/225000 [116:06:13<338:21:53,  7.20s/it]
 25%|██▍       | 55808/225000 [116:06:18<299:43:48,  6.38s/it]
 25%|██▍       | 55809/225000 [116:06:30<380:16:19,  8.09s/it]
 25%|██▍       | 55810/225000 [116:06:35<339:32:57,  7.22s/it]
 25%|██▍       | 55811/225000 [116:06:43<343:24:13,  7.31s/it]
 25%|██▍       | 55812/225000 [116:06:47<309:52:01,  6.59s/it]
 25%|██▍       | 55813/225000 [116:06:53<293:15:24,  6.24s/it]
 25%|██▍       | 55814/225000 [116:07:01<314:04:32,  6.68s/it]
 25%|██▍       | 55815/225000 [116:07:08<328:50:11,  7.00s/it]
 25%|██▍       | 55816/225000 [116:07:13<294:28:54,  6.27s/it]
 25%|██▍       | 55817/225000 [116:07:17<270:25:50,  5.75s/it]
 25%|██▍       | 55818/225000 [116:07:22<254:50:36,  5.42s/it]
 25%|██▍       | 55819/225000 [116:07:34<350:44:16,  7.46s/it]
 25%|██▍       | 55820/225000 [116:07:42<353:06:42,  7.51s/it]
 25%|██▍       | 55821/225000 [116:07:46<309:29:54,  6.59s/it]
 25%|██▍       | 55822/225000 [116:07:51<279:49:30,  5.95s/it]
 25%|██▍       | 55823/225000 [116:08:02<356:39:07,  7.59s/it]
 25%|██▍       | 55824/225000 [116:08:06<298:27:04,  6.35s/it]
 25%|██▍       | 55825/225000 [116:08:14<328:13:10,  6.98s/it]
 25%|██▍       | 55826/225000 [116:08:20<312:23:14,  6.65s/it]
 25%|██▍       | 55827/225000 [116:08:29<344:47:58,  7.34s/it]
 25%|██▍       | 55828/225000 [116:08:33<299:56:21,  6.38s/it]
 25%|██▍       | 55829/225000 [116:08:41<317:09:16,  6.75s/it]
 25%|██▍       | 55830/225000 [116:08:49<331:12:04,  7.05s/it]
 25%|██▍       | 55831/225000 [116:08:53<295:51:00,  6.30s/it]
 25%|██▍       | 55832/225000 [116:08:57<263:13:12,  5.60s/it]
 25%|██▍       | 55833/225000 [116:09:07<330:41:51,  7.04s/it]
 25%|██▍       | 55834/225000 [116:09:11<284:49:34,  6.06s/it]
 25%|██▍       | 55835/225000 [116:09:15<259:06:05,  5.51s/it]
 25%|██▍       | 55836/225000 [116:09:21<256:15:39,  5.45s/it]
 25%|██▍       | 55837/225000 [116:09:31<326:51:35,  6.96s/it]
 25%|██▍       | 55838/225000 [116:09:41<359:56:09,  7.66s/it]
 25%|██▍       | 55839/225000 [116:09:44<305:43:09,  6.51s/it]
 25%|██▍       | 55840/225000 [116:09:52<328:34:40,  6.99s/it]
 25%|██▍       | 55841/225000 [116:09:59<325:05:59,  6.92s/it]
 25%|██▍       | 55842/225000 [116:10:06<324:06:30,  6.90s/it]
 25%|██▍       | 55843/225000 [116:10:10<284:19:48,  6.05s/it]
 25%|██▍       | 55844/225000 [116:10:15<265:59:16,  5.66s/it]
 25%|██▍       | 55845/225000 [116:10:28<365:55:46,  7.79s/it]
 25%|██▍       | 55846/225000 [116:10:31<309:18:18,  6.58s/it]
 25%|██▍       | 55847/225000 [116:10:39<320:30:46,  6.82s/it]
 25%|██▍       | 55848/225000 [116:10:46<327:37:35,  6.97s/it]
 25%|██▍       | 55849/225000 [116:10:51<302:56:13,  6.45s/it]
 25%|██▍       | 55850/225000 [116:10:56<283:58:40,  6.04s/it]
                                                              
{'loss': '0.7169', 'grad_norm': '0.8357', 'learning_rate': '4.46e-05', 'epoch': '0.2482', 'num_input_tokens_seen': 193745640, 'train_runtime': '4.183e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 55850/225000 [116:10:56<283:58:40,  6.04s/it]
 25%|██▍       | 55851/225000 [116:11:02<275:43:52,  5.87s/it]
 25%|██▍       | 55852/225000 [116:11:07<269:10:38,  5.73s/it]
 25%|██▍       | 55853/225000 [116:11:13<266:07:17,  5.66s/it]
 25%|██▍       | 55854/225000 [116:11:18<262:16:46,  5.58s/it]
 25%|██▍       | 55855/225000 [116:11:24<269:51:27,  5.74s/it]
 25%|██▍       | 55856/225000 [116:11:38<381:12:59,  8.11s/it]
 25%|██▍       | 55857/225000 [116:11:47<387:31:13,  8.25s/it]
 25%|██▍       | 55858/225000 [116:11:54<376:58:59,  8.02s/it]
 25%|██▍       | 55859/225000 [116:11:59<338:27:16,  7.20s/it]
 25%|██▍       | 55860/225000 [116:12:04<300:54:53,  6.40s/it]
 25%|██▍       | 55861/225000 [116:12:15<364:44:14,  7.76s/it]
 25%|██▍       | 55862/225000 [116:12:21<342:18:11,  7.29s/it]
 25%|██▍       | 55863/225000 [116:12:25<302:33:55,  6.44s/it]
 25%|██▍       | 55864/225000 [116:12:30<276:53:21,  5.89s/it]
 25%|██▍       | 55865/225000 [116:12:35<262:28:10,  5.59s/it]
 25%|██▍       | 55866/225000 [116:12:48<369:18:36,  7.86s/it]
 25%|██▍       | 55867/225000 [116:12:53<322:14:57,  6.86s/it]
 25%|██▍       | 55868/225000 [116:12:58<302:22:11,  6.44s/it]
 25%|██▍       | 55869/225000 [116:13:03<283:09:16,  6.03s/it]
 25%|██▍       | 55870/225000 [116:13:08<267:38:16,  5.70s/it]
 25%|██▍       | 55871/225000 [116:13:16<304:29:27,  6.48s/it]
 25%|██▍       | 55872/225000 [116:13:27<367:16:18,  7.82s/it]
 25%|██▍       | 55873/225000 [116:13:32<319:28:17,  6.80s/it]
 25%|██▍       | 55874/225000 [116:13:36<286:28:02,  6.10s/it]
 25%|██▍       | 55875/225000 [116:13:46<338:20:55,  7.20s/it]
 25%|██▍       | 55876/225000 [116:13:52<316:54:36,  6.75s/it]
 25%|██▍       | 55877/225000 [116:13:56<287:46:15,  6.13s/it]
 25%|██▍       | 55878/225000 [116:14:09<378:57:24,  8.07s/it]
 25%|██▍       | 55879/225000 [116:14:16<360:57:46,  7.68s/it]
 25%|██▍       | 55880/225000 [116:14:21<332:54:00,  7.09s/it]
 25%|██▍       | 55881/225000 [116:14:27<316:19:42,  6.73s/it]
 25%|██▍       | 55882/225000 [116:14:31<278:52:20,  5.94s/it]
 25%|██▍       | 55883/225000 [116:14:41<328:27:20,  6.99s/it]
 25%|██▍       | 55884/225000 [116:14:47<318:22:40,  6.78s/it]
 25%|██▍       | 55885/225000 [116:14:52<296:06:33,  6.30s/it]
 25%|██▍       | 55886/225000 [116:14:58<281:58:24,  6.00s/it]
 25%|██▍       | 55887/225000 [116:15:03<273:59:25,  5.83s/it]
 25%|██▍       | 55888/225000 [116:15:16<369:05:22,  7.86s/it]
 25%|██▍       | 55889/225000 [116:15:19<312:40:30,  6.66s/it]
 25%|██▍       | 55890/225000 [116:15:27<327:07:40,  6.96s/it]
 25%|██▍       | 55891/225000 [116:15:34<332:12:57,  7.07s/it]
 25%|██▍       | 55892/225000 [116:15:39<289:14:12,  6.16s/it]
 25%|██▍       | 55893/225000 [116:15:48<334:33:46,  7.12s/it]
 25%|██▍       | 55894/225000 [116:15:52<293:22:03,  6.25s/it]
 25%|██▍       | 55895/225000 [116:15:57<271:22:13,  5.78s/it]
 25%|██▍       | 55896/225000 [116:16:08<351:40:10,  7.49s/it]
 25%|██▍       | 55897/225000 [116:16:12<304:02:43,  6.47s/it]
 25%|██▍       | 55898/225000 [116:16:22<348:03:44,  7.41s/it]
 25%|██▍       | 55899/225000 [116:16:29<341:49:43,  7.28s/it]
 25%|██▍       | 55900/225000 [116:16:33<298:06:06,  6.35s/it]
                                                              
{'loss': '0.6979', 'grad_norm': '0.7669', 'learning_rate': '4.459e-05', 'epoch': '0.2484', 'num_input_tokens_seen': 193896976, 'train_runtime': '4.186e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 55900/225000 [116:16:33<298:06:06,  6.35s/it]
 25%|██▍       | 55901/225000 [116:16:42<333:10:36,  7.09s/it]
 25%|██▍       | 55902/225000 [116:16:47<306:31:23,  6.53s/it]
 25%|██▍       | 55903/225000 [116:16:53<291:01:09,  6.20s/it]
 25%|██▍       | 55904/225000 [116:16:58<286:28:23,  6.10s/it]
 25%|██▍       | 55905/225000 [116:17:03<269:41:08,  5.74s/it]
 25%|██▍       | 55906/225000 [116:17:08<260:07:40,  5.54s/it]
 25%|██▍       | 55907/225000 [116:17:17<309:03:12,  6.58s/it]
 25%|██▍       | 55908/225000 [116:17:28<360:41:58,  7.68s/it]
 25%|██▍       | 55909/225000 [116:17:33<326:28:20,  6.95s/it]
 25%|██▍       | 55910/225000 [116:17:38<303:18:49,  6.46s/it]
 25%|██▍       | 55911/225000 [116:17:44<294:48:51,  6.28s/it]
 25%|██▍       | 55912/225000 [116:17:54<341:44:10,  7.28s/it]
 25%|██▍       | 55913/225000 [116:17:59<310:47:00,  6.62s/it]
 25%|██▍       | 55914/225000 [116:18:04<289:18:40,  6.16s/it]
 25%|██▍       | 55915/225000 [116:18:16<370:17:14,  7.88s/it]
 25%|██▍       | 55916/225000 [116:18:21<327:09:37,  6.97s/it]
 25%|██▍       | 55917/225000 [116:18:27<312:30:50,  6.65s/it]
 25%|██▍       | 55918/225000 [116:18:33<304:56:27,  6.49s/it]
 25%|██▍       | 55919/225000 [116:18:38<284:08:32,  6.05s/it]
 25%|██▍       | 55920/225000 [116:18:43<279:17:02,  5.95s/it]
 25%|██▍       | 55921/225000 [116:18:52<322:59:37,  6.88s/it]
 25%|██▍       | 55922/225000 [116:19:01<352:33:59,  7.51s/it]
 25%|██▍       | 55923/225000 [116:19:06<312:07:16,  6.65s/it]
 25%|██▍       | 55924/225000 [116:19:11<288:48:33,  6.15s/it]
 25%|██▍       | 55925/225000 [116:19:23<369:40:37,  7.87s/it]
 25%|██▍       | 55926/225000 [116:19:28<337:31:11,  7.19s/it]
 25%|██▍       | 55927/225000 [116:19:34<313:00:34,  6.66s/it]
 25%|██▍       | 55928/225000 [116:19:39<291:45:29,  6.21s/it]
 25%|██▍       | 55929/225000 [116:19:45<284:16:03,  6.05s/it]
 25%|██▍       | 55930/225000 [116:19:57<367:50:52,  7.83s/it]
 25%|██▍       | 55931/225000 [116:20:04<365:49:24,  7.79s/it]
 25%|██▍       | 55932/225000 [116:20:09<315:18:06,  6.71s/it]
 25%|██▍       | 55933/225000 [116:20:13<286:12:36,  6.09s/it]
 25%|██▍       | 55934/225000 [116:20:18<267:18:05,  5.69s/it]
 25%|██▍       | 55935/225000 [116:20:30<350:18:32,  7.46s/it]
 25%|██▍       | 55936/225000 [116:20:35<317:59:01,  6.77s/it]
 25%|██▍       | 55937/225000 [116:20:39<287:55:48,  6.13s/it]
 25%|██▍       | 55938/225000 [116:20:45<275:32:19,  5.87s/it]
 25%|██▍       | 55939/225000 [116:20:49<259:40:39,  5.53s/it]
 25%|██▍       | 55940/225000 [116:20:59<322:02:31,  6.86s/it]
 25%|██▍       | 55941/225000 [116:21:08<349:21:55,  7.44s/it]
 25%|██▍       | 55942/225000 [116:21:12<295:52:28,  6.30s/it]
 25%|██▍       | 55943/225000 [116:21:16<261:37:09,  5.57s/it]
 25%|██▍       | 55944/225000 [116:21:27<339:49:03,  7.24s/it]
 25%|██▍       | 55945/225000 [116:21:34<336:16:28,  7.16s/it]
 25%|██▍       | 55946/225000 [116:21:38<293:58:23,  6.26s/it]
 25%|██▍       | 55947/225000 [116:21:43<274:43:07,  5.85s/it]
 25%|██▍       | 55948/225000 [116:21:55<357:22:44,  7.61s/it]
 25%|██▍       | 55949/225000 [116:22:01<343:04:16,  7.31s/it]
 25%|██▍       | 55950/225000 [116:22:07<317:43:06,  6.77s/it]
                                                              
{'loss': '0.715', 'grad_norm': '0.7942', 'learning_rate': '4.458e-05', 'epoch': '0.2487', 'num_input_tokens_seen': 194048064, 'train_runtime': '4.189e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 55950/225000 [116:22:07<317:43:06,  6.77s/it]
 25%|██▍       | 55951/225000 [116:22:11<289:26:57,  6.16s/it]
 25%|██▍       | 55952/225000 [116:22:17<275:05:19,  5.86s/it]
 25%|██▍       | 55953/225000 [116:22:21<258:35:59,  5.51s/it]
 25%|██▍       | 55954/225000 [116:22:32<337:56:41,  7.20s/it]
 25%|██▍       | 55955/225000 [116:22:38<312:37:28,  6.66s/it]
 25%|██▍       | 55956/225000 [116:22:47<346:11:12,  7.37s/it]
 25%|██▍       | 55957/225000 [116:22:53<326:26:50,  6.95s/it]
 25%|██▍       | 55958/225000 [116:22:57<288:00:59,  6.13s/it]
 25%|██▍       | 55959/225000 [116:23:07<342:43:50,  7.30s/it]
 25%|██▍       | 55960/225000 [116:23:12<307:33:21,  6.55s/it]
 25%|██▍       | 55961/225000 [116:23:16<277:46:37,  5.92s/it]
 25%|██▍       | 55962/225000 [116:23:22<278:54:17,  5.94s/it]
 25%|██▍       | 55963/225000 [116:23:34<364:12:06,  7.76s/it]
 25%|██▍       | 55964/225000 [116:23:39<316:23:04,  6.74s/it]
 25%|██▍       | 55965/225000 [116:23:43<281:22:33,  5.99s/it]
 25%|██▍       | 55966/225000 [116:23:48<262:06:51,  5.58s/it]
 25%|██▍       | 55967/225000 [116:24:00<355:58:59,  7.58s/it]
 25%|██▍       | 55968/225000 [116:24:06<342:05:45,  7.29s/it]
 25%|██▍       | 55969/225000 [116:24:10<296:26:08,  6.31s/it]
 25%|██▍       | 55970/225000 [116:24:19<330:52:02,  7.05s/it]
 25%|██▍       | 55971/225000 [116:24:24<292:54:11,  6.24s/it]
 25%|██▍       | 55972/225000 [116:24:28<263:45:01,  5.62s/it]
 25%|██▍       | 55973/225000 [116:24:35<289:52:36,  6.17s/it]
 25%|██▍       | 55974/225000 [116:24:42<298:08:26,  6.35s/it]
 25%|██▍       | 55975/225000 [116:24:51<334:43:43,  7.13s/it]
 25%|██▍       | 55976/225000 [116:24:56<313:27:21,  6.68s/it]
 25%|██▍       | 55977/225000 [116:25:02<297:59:43,  6.35s/it]
 25%|██▍       | 55978/225000 [116:25:08<291:44:11,  6.21s/it]
 25%|██▍       | 55979/225000 [116:25:14<290:20:31,  6.18s/it]
 25%|██▍       | 55980/225000 [116:25:19<277:31:12,  5.91s/it]
 25%|██▍       | 55981/225000 [116:25:27<296:33:10,  6.32s/it]
 25%|██▍       | 55982/225000 [116:25:39<377:09:29,  8.03s/it]
 25%|██▍       | 55983/225000 [116:25:43<324:40:49,  6.92s/it]
 25%|██▍       | 55984/225000 [116:25:53<364:28:17,  7.76s/it]
 25%|██▍       | 55985/225000 [116:26:00<360:34:47,  7.68s/it]
 25%|██▍       | 55986/225000 [116:26:07<355:22:12,  7.57s/it]
 25%|██▍       | 55987/225000 [116:26:11<301:02:32,  6.41s/it]
 25%|██▍       | 55988/225000 [116:26:20<341:17:30,  7.27s/it]
 25%|██▍       | 55989/225000 [116:26:26<316:33:34,  6.74s/it]
 25%|██▍       | 55990/225000 [116:26:31<295:14:50,  6.29s/it]
 25%|██▍       | 55991/225000 [116:26:37<284:17:03,  6.06s/it]
 25%|██▍       | 55992/225000 [116:26:42<274:12:48,  5.84s/it]
 25%|██▍       | 55993/225000 [116:26:48<273:08:15,  5.82s/it]
 25%|██▍       | 55994/225000 [116:26:56<301:21:07,  6.42s/it]
 25%|██▍       | 55995/225000 [116:27:04<330:06:39,  7.03s/it]
 25%|██▍       | 55996/225000 [116:27:11<331:01:36,  7.05s/it]
 25%|██▍       | 55997/225000 [116:27:19<343:31:46,  7.32s/it]
 25%|██▍       | 55998/225000 [116:27:26<337:23:05,  7.19s/it]
 25%|██▍       | 55999/225000 [116:27:31<303:41:15,  6.47s/it]
 25%|██▍       | 56000/225000 [116:27:40<338:18:06,  7.21s/it]
                                                              
{'loss': '0.7284', 'grad_norm': '0.7608', 'learning_rate': '4.457e-05', 'epoch': '0.2489', 'num_input_tokens_seen': 194198096, 'train_runtime': '4.193e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 56000/225000 [116:27:40<338:18:06,  7.21s/it]
 25%|██▍       | 56001/225000 [116:27:44<290:24:39,  6.19s/it]
 25%|██▍       | 56002/225000 [116:27:53<334:23:58,  7.12s/it]
 25%|██▍       | 56003/225000 [116:27:57<294:28:40,  6.27s/it]
 25%|██▍       | 56004/225000 [116:28:07<341:17:19,  7.27s/it]
 25%|██▍       | 56005/225000 [116:28:11<294:39:42,  6.28s/it]
 25%|██▍       | 56006/225000 [116:28:15<269:48:28,  5.75s/it]
 25%|██▍       | 56007/225000 [116:28:24<309:43:00,  6.60s/it]
 25%|██▍       | 56008/225000 [116:28:33<342:56:26,  7.31s/it]
 25%|██▍       | 56009/225000 [116:28:37<298:37:14,  6.36s/it]
 25%|██▍       | 56010/225000 [116:28:41<269:25:54,  5.74s/it]
 25%|██▍       | 56011/225000 [116:28:54<366:47:45,  7.81s/it]
 25%|██▍       | 56012/225000 [116:28:58<316:22:24,  6.74s/it]
 25%|██▍       | 56013/225000 [116:29:06<338:20:57,  7.21s/it]
 25%|██▍       | 56014/225000 [116:29:14<342:17:57,  7.29s/it]
 25%|██▍       | 56015/225000 [116:29:20<323:07:55,  6.88s/it]
 25%|██▍       | 56016/225000 [116:29:28<337:03:38,  7.18s/it]
 25%|██▍       | 56017/225000 [116:29:34<321:56:49,  6.86s/it]
 25%|██▍       | 56018/225000 [116:29:39<293:11:44,  6.25s/it]
 25%|██▍       | 56019/225000 [116:29:43<271:07:49,  5.78s/it]
 25%|██▍       | 56020/225000 [116:29:52<315:20:31,  6.72s/it]
 25%|██▍       | 56021/225000 [116:30:01<346:46:22,  7.39s/it]
 25%|██▍       | 56022/225000 [116:30:06<303:54:14,  6.47s/it]
 25%|██▍       | 56023/225000 [116:30:16<362:49:46,  7.73s/it]
 25%|██▍       | 56024/225000 [116:30:20<308:04:18,  6.56s/it]
 25%|██▍       | 56025/225000 [116:30:28<328:21:15,  7.00s/it]
 25%|██▍       | 56026/225000 [116:30:34<314:14:47,  6.70s/it]
 25%|██▍       | 56027/225000 [116:30:42<337:00:32,  7.18s/it]
 25%|██▍       | 56028/225000 [116:30:48<313:40:39,  6.68s/it]
 25%|██▍       | 56029/225000 [116:30:54<300:35:53,  6.40s/it]
 25%|██▍       | 56030/225000 [116:31:01<307:52:17,  6.56s/it]
 25%|██▍       | 56031/225000 [116:31:08<320:16:45,  6.82s/it]
 25%|██▍       | 56032/225000 [116:31:13<299:41:46,  6.39s/it]
 25%|██▍       | 56033/225000 [116:31:20<307:08:57,  6.54s/it]
 25%|██▍       | 56034/225000 [116:31:28<330:51:48,  7.05s/it]
 25%|██▍       | 56035/225000 [116:31:32<283:45:49,  6.05s/it]
 25%|██▍       | 56036/225000 [116:31:38<285:18:05,  6.08s/it]
 25%|██▍       | 56037/225000 [116:31:48<334:14:38,  7.12s/it]
 25%|██▍       | 56038/225000 [116:31:52<287:28:22,  6.13s/it]
 25%|██▍       | 56039/225000 [116:31:56<255:17:51,  5.44s/it]
 25%|██▍       | 56040/225000 [116:32:05<315:30:24,  6.72s/it]
 25%|██▍       | 56041/225000 [116:32:10<289:35:49,  6.17s/it]
 25%|██▍       | 56042/225000 [116:32:15<276:32:41,  5.89s/it]
 25%|██▍       | 56043/225000 [116:32:21<268:28:48,  5.72s/it]
 25%|██▍       | 56044/225000 [116:32:34<370:38:15,  7.90s/it]
 25%|██▍       | 56045/225000 [116:32:39<328:43:22,  7.00s/it]
 25%|██▍       | 56046/225000 [116:32:43<294:53:16,  6.28s/it]
 25%|██▍       | 56047/225000 [116:32:48<276:13:32,  5.89s/it]
 25%|██▍       | 56048/225000 [116:33:01<380:56:58,  8.12s/it]
 25%|██▍       | 56049/225000 [116:33:07<351:14:04,  7.48s/it]
 25%|██▍       | 56050/225000 [116:33:12<310:00:06,  6.61s/it]
                                                              
{'loss': '0.7233', 'grad_norm': '0.7511', 'learning_rate': '4.456e-05', 'epoch': '0.2491', 'num_input_tokens_seen': 194348280, 'train_runtime': '4.196e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 56050/225000 [116:33:12<310:00:06,  6.61s/it]
 25%|██▍       | 56051/225000 [116:33:18<294:05:24,  6.27s/it]
 25%|██▍       | 56052/225000 [116:33:29<365:43:06,  7.79s/it]
 25%|██▍       | 56053/225000 [116:33:33<312:47:06,  6.66s/it]
 25%|██▍       | 56054/225000 [116:33:38<296:11:05,  6.31s/it]
 25%|██▍       | 56055/225000 [116:33:47<323:57:18,  6.90s/it]
 25%|██▍       | 56056/225000 [116:33:52<295:25:51,  6.30s/it]
 25%|██▍       | 56057/225000 [116:33:57<278:25:33,  5.93s/it]
 25%|██▍       | 56058/225000 [116:34:08<353:24:43,  7.53s/it]
 25%|██▍       | 56059/225000 [116:34:14<327:05:10,  6.97s/it]
 25%|██▍       | 56060/225000 [116:34:22<343:34:37,  7.32s/it]
 25%|██▍       | 56061/225000 [116:34:27<314:04:36,  6.69s/it]
 25%|██▍       | 56062/225000 [116:34:32<292:44:14,  6.24s/it]
 25%|██▍       | 56063/225000 [116:34:37<277:15:41,  5.91s/it]
 25%|██▍       | 56064/225000 [116:34:42<263:20:34,  5.61s/it]
 25%|██▍       | 56065/225000 [116:34:49<279:57:14,  5.97s/it]
 25%|██▍       | 56066/225000 [116:35:02<375:03:45,  7.99s/it]
 25%|██▍       | 56067/225000 [116:35:05<312:59:15,  6.67s/it]
 25%|██▍       | 56068/225000 [116:35:10<280:59:01,  5.99s/it]
 25%|██▍       | 56069/225000 [116:35:21<362:15:55,  7.72s/it]
 25%|██▍       | 56070/225000 [116:35:25<305:04:23,  6.50s/it]
 25%|██▍       | 56071/225000 [116:35:33<329:23:36,  7.02s/it]
 25%|██▍       | 56072/225000 [116:35:37<286:37:14,  6.11s/it]
 25%|██▍       | 56073/225000 [116:35:45<315:35:10,  6.73s/it]
 25%|██▍       | 56074/225000 [116:35:53<323:02:29,  6.88s/it]
 25%|██▍       | 56075/225000 [116:35:59<318:36:04,  6.79s/it]
 25%|██▍       | 56076/225000 [116:36:07<326:00:32,  6.95s/it]
 25%|██▍       | 56077/225000 [116:36:14<326:29:27,  6.96s/it]
 25%|██▍       | 56078/225000 [116:36:21<328:10:10,  6.99s/it]
 25%|██▍       | 56079/225000 [116:36:26<299:18:09,  6.38s/it]
 25%|██▍       | 56080/225000 [116:36:30<275:36:37,  5.87s/it]
 25%|██▍       | 56081/225000 [116:36:36<274:43:19,  5.85s/it]
 25%|██▍       | 56082/225000 [116:36:48<360:24:14,  7.68s/it]
 25%|██▍       | 56083/225000 [116:36:51<300:55:39,  6.41s/it]
 25%|██▍       | 56084/225000 [116:36:56<279:10:25,  5.95s/it]
 25%|██▍       | 56085/225000 [116:37:05<322:26:50,  6.87s/it]
 25%|██▍       | 56086/225000 [116:37:13<328:56:09,  7.01s/it]
 25%|██▍       | 56087/225000 [116:37:19<325:19:29,  6.93s/it]
 25%|██▍       | 56088/225000 [116:37:24<286:59:12,  6.12s/it]
 25%|██▍       | 56089/225000 [116:37:33<337:30:35,  7.19s/it]
 25%|██▍       | 56090/225000 [116:37:40<325:36:54,  6.94s/it]
 25%|██▍       | 56091/225000 [116:37:47<333:17:00,  7.10s/it]
 25%|██▍       | 56092/225000 [116:37:55<344:51:05,  7.35s/it]
 25%|██▍       | 56093/225000 [116:38:00<303:30:24,  6.47s/it]
 25%|██▍       | 56094/225000 [116:38:04<275:12:23,  5.87s/it]
 25%|██▍       | 56095/225000 [116:38:11<293:58:15,  6.27s/it]
 25%|██▍       | 56096/225000 [116:38:20<327:27:52,  6.98s/it]
 25%|██▍       | 56097/225000 [116:38:24<288:05:14,  6.14s/it]
 25%|██▍       | 56098/225000 [116:38:28<260:06:48,  5.54s/it]
 25%|██▍       | 56099/225000 [116:38:40<347:29:16,  7.41s/it]
 25%|██▍       | 56100/225000 [116:38:47<339:56:50,  7.25s/it]
                                                              
{'loss': '0.7202', 'grad_norm': '0.6258', 'learning_rate': '4.455e-05', 'epoch': '0.2493', 'num_input_tokens_seen': 194497128, 'train_runtime': '4.199e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 56100/225000 [116:38:47<339:56:50,  7.25s/it]
 25%|██▍       | 56101/225000 [116:38:55<346:49:43,  7.39s/it]
 25%|██▍       | 56102/225000 [116:39:01<334:46:05,  7.14s/it]
 25%|██▍       | 56103/225000 [116:39:06<298:14:57,  6.36s/it]
 25%|██▍       | 56104/225000 [116:39:14<326:07:58,  6.95s/it]
 25%|██▍       | 56105/225000 [116:39:22<337:35:01,  7.20s/it]
 25%|██▍       | 56106/225000 [116:39:26<298:05:30,  6.35s/it]
 25%|██▍       | 56107/225000 [116:39:35<340:13:11,  7.25s/it]
 25%|██▍       | 56108/225000 [116:39:40<305:05:06,  6.50s/it]
 25%|██▍       | 56109/225000 [116:39:47<313:38:35,  6.69s/it]
 25%|██▍       | 56110/225000 [116:39:56<337:05:13,  7.19s/it]
 25%|██▍       | 56111/225000 [116:39:59<289:03:44,  6.16s/it]
 25%|██▍       | 56112/225000 [116:40:05<281:28:26,  6.00s/it]
 25%|██▍       | 56113/225000 [116:40:11<282:02:44,  6.01s/it]
 25%|██▍       | 56114/225000 [116:40:17<276:14:02,  5.89s/it]
 25%|██▍       | 56115/225000 [116:40:24<297:09:35,  6.33s/it]
 25%|██▍       | 56116/225000 [116:40:34<353:55:06,  7.54s/it]
 25%|██▍       | 56117/225000 [116:40:41<336:18:38,  7.17s/it]
 25%|██▍       | 56118/225000 [116:40:48<333:13:47,  7.10s/it]
 25%|██▍       | 56119/225000 [116:40:53<311:58:01,  6.65s/it]
 25%|██▍       | 56120/225000 [116:41:00<310:04:05,  6.61s/it]
 25%|██▍       | 56121/225000 [116:41:09<343:28:23,  7.32s/it]
 25%|██▍       | 56122/225000 [116:41:13<294:17:10,  6.27s/it]
 25%|██▍       | 56123/225000 [116:41:22<338:37:38,  7.22s/it]
 25%|██▍       | 56124/225000 [116:41:26<292:20:54,  6.23s/it]
 25%|██▍       | 56125/225000 [116:41:31<273:46:35,  5.84s/it]
 25%|██▍       | 56126/225000 [116:41:42<343:31:06,  7.32s/it]
 25%|██▍       | 56127/225000 [116:41:47<310:12:38,  6.61s/it]
 25%|██▍       | 56128/225000 [116:41:52<289:00:32,  6.16s/it]
 25%|██▍       | 56129/225000 [116:41:57<278:39:12,  5.94s/it]
 25%|██▍       | 56130/225000 [116:42:02<269:45:41,  5.75s/it]
 25%|██▍       | 56131/225000 [116:42:08<268:15:44,  5.72s/it]
 25%|██▍       | 56132/225000 [116:42:19<343:01:29,  7.31s/it]
 25%|██▍       | 56133/225000 [116:42:29<376:44:31,  8.03s/it]
 25%|██▍       | 56134/225000 [116:42:33<315:39:22,  6.73s/it]
 25%|██▍       | 56135/225000 [116:42:42<358:38:56,  7.65s/it]
 25%|██▍       | 56136/225000 [116:42:48<333:55:10,  7.12s/it]
 25%|██▍       | 56137/225000 [116:42:55<323:42:13,  6.90s/it]
 25%|██▍       | 56138/225000 [116:43:02<324:08:34,  6.91s/it]
 25%|██▍       | 56139/225000 [116:43:10<345:38:48,  7.37s/it]
 25%|██▍       | 56140/225000 [116:43:14<301:19:39,  6.42s/it]
 25%|██▍       | 56141/225000 [116:43:19<275:51:34,  5.88s/it]
 25%|██▍       | 56142/225000 [116:43:30<352:48:19,  7.52s/it]
 25%|██▍       | 56143/225000 [116:43:34<300:27:17,  6.41s/it]
 25%|██▍       | 56144/225000 [116:43:39<274:43:42,  5.86s/it]
 25%|██▍       | 56145/225000 [116:43:43<255:39:02,  5.45s/it]
 25%|██▍       | 56146/225000 [116:43:56<354:24:40,  7.56s/it]
 25%|██▍       | 56147/225000 [116:43:59<299:53:32,  6.39s/it]
 25%|██▍       | 56148/225000 [116:44:08<335:29:31,  7.15s/it]
 25%|██▍       | 56149/225000 [116:44:13<302:55:15,  6.46s/it]
 25%|██▍       | 56150/225000 [116:44:18<276:12:59,  5.89s/it]
                                                              
{'loss': '0.7292', 'grad_norm': '0.7302', 'learning_rate': '4.453e-05', 'epoch': '0.2496', 'num_input_tokens_seen': 194647784, 'train_runtime': '4.203e+05', 'train_tokens_per_second': '463.2'}

 25%|██▍       | 56150/225000 [116:44:18<276:12:59,  5.89s/it]
 25%|██▍       | 56151/225000 [116:44:22<256:53:27,  5.48s/it]
 25%|██▍       | 56152/225000 [116:44:33<333:36:20,  7.11s/it]
 25%|██▍       | 56153/225000 [116:44:38<301:04:49,  6.42s/it]
 25%|██▍       | 56154/225000 [116:44:47<340:32:40,  7.26s/it]
 25%|██▍       | 56155/225000 [116:44:51<291:32:13,  6.22s/it]
 25%|██▍       | 56156/225000 [116:45:01<342:28:38,  7.30s/it]
 25%|██▍       | 56157/225000 [116:45:05<301:16:40,  6.42s/it]
 25%|██▍       | 56158/225000 [116:45:09<269:18:08,  5.74s/it]
 25%|██▍       | 56159/225000 [116:45:16<288:28:59,  6.15s/it]
 25%|██▍       | 56160/225000 [116:45:26<341:27:44,  7.28s/it]
 25%|██▍       | 56161/225000 [116:45:30<297:30:32,  6.34s/it]
 25%|██▍       | 56162/225000 [116:45:39<330:57:41,  7.06s/it]
 25%|██▍       | 56163/225000 [116:45:45<318:50:18,  6.80s/it]
 25%|██▍       | 56164/225000 [116:45:53<335:49:01,  7.16s/it]
 25%|██▍       | 56165/225000 [116:45:59<309:08:23,  6.59s/it]
 25%|██▍       | 56166/225000 [116:46:04<287:56:54,  6.14s/it]
 25%|██▍       | 56167/225000 [116:46:08<270:10:05,  5.76s/it]
 25%|██▍       | 56168/225000 [116:46:13<252:15:20,  5.38s/it]
 25%|██▍       | 56169/225000 [116:46:26<356:21:55,  7.60s/it]
 25%|██▍       | 56170/225000 [116:46:30<303:50:50,  6.48s/it]
 25%|██▍       | 56171/225000 [116:46:35<293:06:13,  6.25s/it]
 25%|██▍       | 56172/225000 [116:46:41<285:30:52,  6.09s/it]
 25%|██▍       | 56173/225000 [116:46:46<271:32:26,  5.79s/it]
 25%|██▍       | 56174/225000 [116:46:55<318:03:00,  6.78s/it]
 25%|██▍       | 56175/225000 [116:47:01<303:19:57,  6.47s/it]
 25%|██▍       | 56176/225000 [116:47:07<295:29:21,  6.30s/it]
 25%|██▍       | 56177/225000 [116:47:12<276:55:18,  5.91s/it]
 25%|██▍       | 56178/225000 [116:47:18<277:07:31,  5.91s/it]
 25%|██▍       | 56179/225000 [116:47:32<388:01:32,  8.27s/it]
 25%|██▍       | 56180/225000 [116:47:39<370:49:04,  7.91s/it]
 25%|██▍       | 56181/225000 [116:47:46<370:29:38,  7.90s/it]
 25%|██▍       | 56182/225000 [116:47:51<326:18:26,  6.96s/it]
 25%|██▍       | 56183/225000 [116:47:56<299:54:26,  6.40s/it]
 25%|██▍       | 56184/225000 [116:48:02<285:02:12,  6.08s/it]
 25%|██▍       | 56185/225000 [116:48:07<278:20:31,  5.94s/it]
 25%|██▍       | 56186/225000 [116:48:15<301:14:49,  6.42s/it]
 25%|██▍       | 56187/225000 [116:48:28<397:33:43,  8.48s/it]
 25%|██▍       | 56188/225000 [116:48:33<345:56:14,  7.38s/it]
 25%|██▍       | 56189/225000 [116:48:40<348:32:17,  7.43s/it]
 25%|██▍       | 56190/225000 [116:48:44<297:46:00,  6.35s/it]
 25%|██▍       | 56191/225000 [116:48:51<306:02:54,  6.53s/it]
 25%|██▍       | 56192/225000 [116:48:59<327:13:28,  6.98s/it]
 25%|██▍       | 56193/225000 [116:49:04<290:26:28,  6.19s/it]
 25%|██▍       | 56194/225000 [116:49:09<284:56:31,  6.08s/it]
 25%|██▍       | 56195/225000 [116:49:18<322:14:45,  6.87s/it]
 25%|██▍       | 56196/225000 [116:49:25<328:15:10,  7.00s/it]
 25%|██▍       | 56197/225000 [116:49:29<283:29:53,  6.05s/it]
 25%|██▍       | 56198/225000 [116:49:39<331:57:08,  7.08s/it]
 25%|██▍       | 56199/225000 [116:49:48<356:31:36,  7.60s/it]
 25%|██▍       | 56200/225000 [116:49:51<298:24:12,  6.36s/it]
                                                              
{'loss': '0.7291', 'grad_norm': '0.6661', 'learning_rate': '4.452e-05', 'epoch': '0.2498', 'num_input_tokens_seen': 194796904, 'train_runtime': '4.206e+05', 'train_tokens_per_second': '463.1'}

 25%|██▍       | 56200/225000 [116:49:51<298:24:12,  6.36s/it]
 25%|██▍       | 56201/225000 [116:49:59<324:39:30,  6.92s/it]
 25%|██▍       | 56202/225000 [116:50:04<296:59:56,  6.33s/it]
 25%|██▍       | 56203/225000 [116:50:09<279:14:34,  5.96s/it]
 25%|██▍       | 56204/225000 [116:50:16<295:05:03,  6.29s/it]
 25%|██▍       | 56205/225000 [116:50:27<359:33:37,  7.67s/it]
 25%|██▍       | 56206/225000 [116:50:33<326:41:18,  6.97s/it]
 25%|██▍       | 56207/225000 [116:50:39<311:33:27,  6.64s/it]
 25%|██▍       | 56208/225000 [116:50:44<290:33:33,  6.20s/it]
 25%|██▍       | 56209/225000 [116:50:49<273:10:44,  5.83s/it]
 25%|██▍       | 56210/225000 [116:50:54<268:31:22,  5.73s/it]
 25%|██▍       | 56211/225000 [116:51:03<313:07:49,  6.68s/it]
 25%|██▍       | 56212/225000 [116:51:16<405:46:43,  8.65s/it]
 25%|██▍       | 56213/225000 [116:51:22<364:37:43,  7.78s/it]
 25%|██▍       | 56214/225000 [116:51:28<334:40:30,  7.14s/it]
 25%|██▍       | 56215/225000 [116:51:33<311:45:57,  6.65s/it]
 25%|██▍       | 56216/225000 [116:51:39<294:45:20,  6.29s/it]
 25%|██▍       | 56217/225000 [116:51:44<280:52:26,  5.99s/it]
 25%|██▍       | 56218/225000 [116:51:55<351:12:45,  7.49s/it]
 25%|██▍       | 56219/225000 [116:52:01<325:52:38,  6.95s/it]
 25%|██▍       | 56220/225000 [116:52:05<293:30:24,  6.26s/it]
 25%|██▍       | 56221/225000 [116:52:10<271:42:57,  5.80s/it]
 25%|██▍       | 56222/225000 [116:52:22<365:42:46,  7.80s/it]
 25%|██▍       | 56223/225000 [116:52:26<306:41:02,  6.54s/it]
 25%|██▍       | 56224/225000 [116:52:31<286:06:35,  6.10s/it]
 25%|██▍       | 56225/225000 [116:52:37<278:24:22,  5.94s/it]
 25%|██▍       | 56226/225000 [116:52:42<270:40:58,  5.77s/it]
 25%|██▍       | 56227/225000 [116:52:47<258:41:17,  5.52s/it]
 25%|██▍       | 56228/225000 [116:52:52<257:44:05,  5.50s/it]
 25%|██▍       | 56229/225000 [116:52:58<262:21:35,  5.60s/it]
 25%|██▍       | 56230/225000 [116:53:08<315:42:58,  6.73s/it]
 25%|██▍       | 56231/225000 [116:53:20<399:15:41,  8.52s/it]
 25%|██▍       | 56232/225000 [116:53:26<360:06:07,  7.68s/it]
 25%|██▍       | 56233/225000 [116:53:31<323:47:56,  6.91s/it]
 25%|██▍       | 56234/225000 [116:53:38<328:51:13,  7.01s/it]
 25%|██▍       | 56235/225000 [116:53:42<283:54:33,  6.06s/it]
 25%|██▍       | 56236/225000 [116:53:51<315:54:45,  6.74s/it]
 25%|██▍       | 56237/225000 [116:53:57<316:01:15,  6.74s/it]
 25%|██▍       | 56238/225000 [116:54:02<292:00:45,  6.23s/it]
 25%|██▍       | 56239/225000 [116:54:08<282:03:23,  6.02s/it]
 25%|██▍       | 56240/225000 [116:54:20<362:52:11,  7.74s/it]
 25%|██▍       | 56241/225000 [116:54:24<314:10:48,  6.70s/it]
 25%|██▍       | 56242/225000 [116:54:30<302:57:48,  6.46s/it]
 25%|██▍       | 56243/225000 [116:54:35<290:22:29,  6.19s/it]
 25%|██▍       | 56244/225000 [116:54:41<281:59:57,  6.02s/it]
 25%|██▍       | 56245/225000 [116:54:47<278:28:41,  5.94s/it]
 25%|██▍       | 56246/225000 [116:54:53<277:59:49,  5.93s/it]
 25%|██▍       | 56247/225000 [116:54:59<282:08:46,  6.02s/it]
 25%|██▍       | 56248/225000 [116:55:04<274:54:55,  5.86s/it]
 25%|██▍       | 56249/225000 [116:55:12<295:22:35,  6.30s/it]
 25%|██▌       | 56250/225000 [116:55:26<401:09:41,  8.56s/it]
                                                              
{'loss': '0.7281', 'grad_norm': '0.7665', 'learning_rate': '4.451e-05', 'epoch': '0.25', 'num_input_tokens_seen': 194948816, 'train_runtime': '4.209e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56250/225000 [116:55:26<401:09:41,  8.56s/it]
 25%|██▌       | 56251/225000 [116:55:33<387:43:13,  8.27s/it]
 25%|██▌       | 56252/225000 [116:55:37<331:07:06,  7.06s/it]
 25%|██▌       | 56253/225000 [116:55:45<339:09:03,  7.24s/it]
 25%|██▌       | 56254/225000 [116:55:50<302:40:30,  6.46s/it]
 25%|██▌       | 56255/225000 [116:55:54<276:52:02,  5.91s/it]
 25%|██▌       | 56256/225000 [116:56:00<276:47:23,  5.91s/it]
 25%|██▌       | 56257/225000 [116:56:09<321:03:32,  6.85s/it]
 25%|██▌       | 56258/225000 [116:56:20<382:31:13,  8.16s/it]
 25%|██▌       | 56259/225000 [116:56:24<323:19:51,  6.90s/it]
 25%|██▌       | 56260/225000 [116:56:34<360:08:30,  7.68s/it]
 25%|██▌       | 56261/225000 [116:56:40<332:58:29,  7.10s/it]
 25%|██▌       | 56262/225000 [116:56:44<294:55:36,  6.29s/it]
 25%|██▌       | 56263/225000 [116:56:50<288:50:21,  6.16s/it]
 25%|██▌       | 56264/225000 [116:57:01<351:41:58,  7.50s/it]
 25%|██▌       | 56265/225000 [116:57:05<304:25:17,  6.49s/it]
 25%|██▌       | 56266/225000 [116:57:14<346:00:52,  7.38s/it]
 25%|██▌       | 56267/225000 [116:57:19<313:00:07,  6.68s/it]
 25%|██▌       | 56268/225000 [116:57:30<368:45:11,  7.87s/it]
 25%|██▌       | 56269/225000 [116:57:36<338:06:58,  7.21s/it]
 25%|██▌       | 56270/225000 [116:57:40<305:15:51,  6.51s/it]
 25%|██▌       | 56271/225000 [116:57:46<289:23:10,  6.17s/it]
 25%|██▌       | 56272/225000 [116:57:51<280:14:31,  5.98s/it]
 25%|██▌       | 56273/225000 [116:57:57<270:37:12,  5.77s/it]
 25%|██▌       | 56274/225000 [116:58:08<352:03:14,  7.51s/it]
 25%|██▌       | 56275/225000 [116:58:18<380:02:44,  8.11s/it]
 25%|██▌       | 56276/225000 [116:58:22<331:53:14,  7.08s/it]
 25%|██▌       | 56277/225000 [116:58:30<345:22:03,  7.37s/it]
 25%|██▌       | 56278/225000 [116:58:34<291:21:59,  6.22s/it]
 25%|██▌       | 56279/225000 [116:58:44<338:26:48,  7.22s/it]
 25%|██▌       | 56280/225000 [116:58:48<293:12:45,  6.26s/it]
 25%|██▌       | 56281/225000 [116:58:53<281:35:36,  6.01s/it]
 25%|██▌       | 56282/225000 [116:58:58<269:03:52,  5.74s/it]
 25%|██▌       | 56283/225000 [116:59:03<260:48:51,  5.57s/it]
 25%|██▌       | 56284/225000 [116:59:09<257:24:47,  5.49s/it]
 25%|██▌       | 56285/225000 [116:59:19<323:27:18,  6.90s/it]
 25%|██▌       | 56286/225000 [116:59:29<375:17:29,  8.01s/it]
 25%|██▌       | 56287/225000 [116:59:37<364:38:58,  7.78s/it]
 25%|██▌       | 56288/225000 [116:59:43<341:48:03,  7.29s/it]
 25%|██▌       | 56289/225000 [116:59:51<360:14:47,  7.69s/it]
 25%|██▌       | 56290/225000 [116:59:56<318:57:31,  6.81s/it]
 25%|██▌       | 56291/225000 [117:00:01<290:28:15,  6.20s/it]
 25%|██▌       | 56292/225000 [117:00:12<355:22:21,  7.58s/it]
 25%|██▌       | 56293/225000 [117:00:17<323:15:24,  6.90s/it]
 25%|██▌       | 56294/225000 [117:00:21<285:55:35,  6.10s/it]
 25%|██▌       | 56295/225000 [117:00:32<352:23:19,  7.52s/it]
 25%|██▌       | 56296/225000 [117:00:36<301:04:18,  6.42s/it]
 25%|██▌       | 56297/225000 [117:00:45<339:11:01,  7.24s/it]
 25%|██▌       | 56298/225000 [117:00:50<306:24:05,  6.54s/it]
 25%|██▌       | 56299/225000 [117:00:55<292:02:10,  6.23s/it]
 25%|██▌       | 56300/225000 [117:01:01<275:56:43,  5.89s/it]
                                                              
{'loss': '0.7129', 'grad_norm': '0.8238', 'learning_rate': '4.45e-05', 'epoch': '0.2502', 'num_input_tokens_seen': 195101960, 'train_runtime': '4.213e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56300/225000 [117:01:01<275:56:43,  5.89s/it]
 25%|██▌       | 56301/225000 [117:01:05<258:34:29,  5.52s/it]
 25%|██▌       | 56302/225000 [117:01:11<256:58:54,  5.48s/it]
 25%|██▌       | 56303/225000 [117:01:19<295:47:15,  6.31s/it]
 25%|██▌       | 56304/225000 [117:01:32<394:05:02,  8.41s/it]
 25%|██▌       | 56305/225000 [117:01:37<343:30:24,  7.33s/it]
 25%|██▌       | 56306/225000 [117:01:41<303:57:20,  6.49s/it]
 25%|██▌       | 56307/225000 [117:01:47<296:18:32,  6.32s/it]
 25%|██▌       | 56308/225000 [117:01:59<372:18:37,  7.95s/it]
 25%|██▌       | 56309/225000 [117:02:06<355:29:44,  7.59s/it]
 25%|██▌       | 56310/225000 [117:02:13<349:32:57,  7.46s/it]
 25%|██▌       | 56311/225000 [117:02:18<315:54:58,  6.74s/it]
 25%|██▌       | 56312/225000 [117:02:26<338:06:20,  7.22s/it]
 25%|██▌       | 56313/225000 [117:02:31<301:45:55,  6.44s/it]
 25%|██▌       | 56314/225000 [117:02:37<291:29:44,  6.22s/it]
 25%|██▌       | 56315/225000 [117:02:43<289:34:38,  6.18s/it]
 25%|██▌       | 56316/225000 [117:02:48<280:21:06,  5.98s/it]
 25%|██▌       | 56317/225000 [117:02:58<327:52:37,  7.00s/it]
 25%|██▌       | 56318/225000 [117:03:05<334:43:33,  7.14s/it]
 25%|██▌       | 56319/225000 [117:03:12<332:16:29,  7.09s/it]
 25%|██▌       | 56320/225000 [117:03:17<295:32:56,  6.31s/it]
 25%|██▌       | 56321/225000 [117:03:25<327:55:55,  7.00s/it]
 25%|██▌       | 56322/225000 [117:03:32<323:39:04,  6.91s/it]
 25%|██▌       | 56323/225000 [117:03:40<341:38:24,  7.29s/it]
 25%|██▌       | 56324/225000 [117:03:46<320:29:29,  6.84s/it]
 25%|██▌       | 56325/225000 [117:03:52<307:15:41,  6.56s/it]
 25%|██▌       | 56326/225000 [117:03:58<295:57:53,  6.32s/it]
 25%|██▌       | 56327/225000 [117:04:04<290:00:36,  6.19s/it]
 25%|██▌       | 56328/225000 [117:04:10<291:41:37,  6.23s/it]
 25%|██▌       | 56329/225000 [117:04:16<283:42:07,  6.06s/it]
 25%|██▌       | 56330/225000 [117:04:21<281:29:26,  6.01s/it]
 25%|██▌       | 56331/225000 [117:04:27<277:43:06,  5.93s/it]
 25%|██▌       | 56332/225000 [117:04:35<306:17:53,  6.54s/it]
 25%|██▌       | 56333/225000 [117:04:48<395:24:46,  8.44s/it]
 25%|██▌       | 56334/225000 [117:04:52<336:00:23,  7.17s/it]
 25%|██▌       | 56335/225000 [117:05:02<367:42:23,  7.85s/it]
 25%|██▌       | 56336/225000 [117:05:06<313:39:23,  6.69s/it]
 25%|██▌       | 56337/225000 [117:05:15<349:26:04,  7.46s/it]
 25%|██▌       | 56338/225000 [117:05:20<321:23:38,  6.86s/it]
 25%|██▌       | 56339/225000 [117:05:25<292:41:26,  6.25s/it]
 25%|██▌       | 56340/225000 [117:05:30<278:52:47,  5.95s/it]
 25%|██▌       | 56341/225000 [117:05:43<376:48:12,  8.04s/it]
 25%|██▌       | 56342/225000 [117:05:48<333:58:09,  7.13s/it]
 25%|██▌       | 56343/225000 [117:05:53<301:30:07,  6.44s/it]
 25%|██▌       | 56344/225000 [117:05:59<292:28:55,  6.24s/it]
 25%|██▌       | 56345/225000 [117:06:04<280:08:07,  5.98s/it]
 25%|██▌       | 56346/225000 [117:06:10<280:40:38,  5.99s/it]
 25%|██▌       | 56347/225000 [117:06:16<279:43:16,  5.97s/it]
 25%|██▌       | 56348/225000 [117:06:23<296:46:12,  6.33s/it]
 25%|██▌       | 56349/225000 [117:06:30<295:59:38,  6.32s/it]
 25%|██▌       | 56350/225000 [117:06:39<332:27:11,  7.10s/it]
                                                              
{'loss': '0.7161', 'grad_norm': '0.7095', 'learning_rate': '4.449e-05', 'epoch': '0.2504', 'num_input_tokens_seen': 195253736, 'train_runtime': '4.216e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56350/225000 [117:06:39<332:27:11,  7.10s/it]
 25%|██▌       | 56351/225000 [117:06:48<365:48:40,  7.81s/it]
 25%|██▌       | 56352/225000 [117:06:56<361:18:48,  7.71s/it]
 25%|██▌       | 56353/225000 [117:07:00<311:41:44,  6.65s/it]
 25%|██▌       | 56354/225000 [117:07:10<366:45:30,  7.83s/it]
 25%|██▌       | 56355/225000 [117:07:14<311:58:53,  6.66s/it]
 25%|██▌       | 56356/225000 [117:07:19<286:13:00,  6.11s/it]
 25%|██▌       | 56357/225000 [117:07:24<268:05:19,  5.72s/it]
 25%|██▌       | 56358/225000 [117:07:36<353:30:07,  7.55s/it]
 25%|██▌       | 56359/225000 [117:07:40<303:23:10,  6.48s/it]
 25%|██▌       | 56360/225000 [117:07:49<349:05:05,  7.45s/it]
 25%|██▌       | 56361/225000 [117:07:57<351:22:30,  7.50s/it]
 25%|██▌       | 56362/225000 [117:08:01<304:30:36,  6.50s/it]
 25%|██▌       | 56363/225000 [117:08:10<341:05:01,  7.28s/it]
 25%|██▌       | 56364/225000 [117:08:17<328:58:42,  7.02s/it]
 25%|██▌       | 56365/225000 [117:08:21<286:10:24,  6.11s/it]
 25%|██▌       | 56366/225000 [117:08:31<349:18:24,  7.46s/it]
 25%|██▌       | 56367/225000 [117:08:38<336:43:20,  7.19s/it]
 25%|██▌       | 56368/225000 [117:08:43<312:23:54,  6.67s/it]
 25%|██▌       | 56369/225000 [117:08:49<299:45:25,  6.40s/it]
 25%|██▌       | 56370/225000 [117:08:54<280:34:37,  5.99s/it]
 25%|██▌       | 56371/225000 [117:08:59<262:42:55,  5.61s/it]
 25%|██▌       | 56372/225000 [117:09:05<264:44:16,  5.65s/it]
 25%|██▌       | 56373/225000 [117:09:10<263:03:33,  5.62s/it]
 25%|██▌       | 56374/225000 [117:09:22<344:44:41,  7.36s/it]
 25%|██▌       | 56375/225000 [117:09:29<340:18:52,  7.27s/it]
 25%|██▌       | 56376/225000 [117:09:32<291:17:23,  6.22s/it]
 25%|██▌       | 56377/225000 [117:09:44<369:04:07,  7.88s/it]
 25%|██▌       | 56378/225000 [117:09:49<320:18:55,  6.84s/it]
 25%|██▌       | 56379/225000 [117:09:52<278:38:25,  5.95s/it]
 25%|██▌       | 56380/225000 [117:10:04<353:55:58,  7.56s/it]
 25%|██▌       | 56381/225000 [117:10:10<336:30:10,  7.18s/it]
 25%|██▌       | 56382/225000 [117:10:15<298:16:50,  6.37s/it]
 25%|██▌       | 56383/225000 [117:10:23<334:29:26,  7.14s/it]
 25%|██▌       | 56384/225000 [117:10:28<298:42:50,  6.38s/it]
 25%|██▌       | 56385/225000 [117:10:33<278:10:01,  5.94s/it]
 25%|██▌       | 56386/225000 [117:10:37<257:47:28,  5.50s/it]
 25%|██▌       | 56387/225000 [117:10:43<255:49:51,  5.46s/it]
 25%|██▌       | 56388/225000 [117:10:56<368:41:00,  7.87s/it]
 25%|██▌       | 56389/225000 [117:11:00<307:38:45,  6.57s/it]
 25%|██▌       | 56390/225000 [117:11:09<342:59:00,  7.32s/it]
 25%|██▌       | 56391/225000 [117:11:14<305:44:33,  6.53s/it]
 25%|██▌       | 56392/225000 [117:11:22<326:53:08,  6.98s/it]
 25%|██▌       | 56393/225000 [117:11:27<307:32:48,  6.57s/it]
 25%|██▌       | 56394/225000 [117:11:33<295:06:56,  6.30s/it]
 25%|██▌       | 56395/225000 [117:11:39<287:39:40,  6.14s/it]
 25%|██▌       | 56396/225000 [117:11:45<284:55:20,  6.08s/it]
 25%|██▌       | 56397/225000 [117:11:50<272:48:35,  5.83s/it]
 25%|██▌       | 56398/225000 [117:11:56<271:15:52,  5.79s/it]
 25%|██▌       | 56399/225000 [117:12:08<365:18:39,  7.80s/it]
 25%|██▌       | 56400/225000 [117:12:16<364:02:18,  7.77s/it]
                                                              
{'loss': '0.7042', 'grad_norm': '0.8967', 'learning_rate': '4.448e-05', 'epoch': '0.2507', 'num_input_tokens_seen': 195403880, 'train_runtime': '4.219e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56400/225000 [117:12:16<364:02:18,  7.77s/it]
 25%|██▌       | 56401/225000 [117:12:20<312:00:49,  6.66s/it]
 25%|██▌       | 56402/225000 [117:12:31<378:57:48,  8.09s/it]
 25%|██▌       | 56403/225000 [117:12:38<354:30:38,  7.57s/it]
 25%|██▌       | 56404/225000 [117:12:45<352:08:36,  7.52s/it]
 25%|██▌       | 56405/225000 [117:12:53<359:56:32,  7.69s/it]
 25%|██▌       | 56406/225000 [117:12:57<301:55:12,  6.45s/it]
 25%|██▌       | 56407/225000 [117:13:06<339:04:42,  7.24s/it]
 25%|██▌       | 56408/225000 [117:13:10<300:57:03,  6.43s/it]
 25%|██▌       | 56409/225000 [117:13:18<312:34:29,  6.67s/it]
 25%|██▌       | 56410/225000 [117:13:26<342:55:23,  7.32s/it]
 25%|██▌       | 56411/225000 [117:13:30<295:09:31,  6.30s/it]
 25%|██▌       | 56412/225000 [117:13:35<275:34:59,  5.88s/it]
 25%|██▌       | 56413/225000 [117:13:45<332:45:15,  7.11s/it]
 25%|██▌       | 56414/225000 [117:13:53<340:59:06,  7.28s/it]
 25%|██▌       | 56415/225000 [117:13:58<307:32:41,  6.57s/it]
 25%|██▌       | 56416/225000 [117:14:02<277:06:29,  5.92s/it]
 25%|██▌       | 56417/225000 [117:14:07<256:12:41,  5.47s/it]
 25%|██▌       | 56418/225000 [117:14:11<240:02:09,  5.13s/it]
 25%|██▌       | 56419/225000 [117:14:23<341:12:50,  7.29s/it]
 25%|██▌       | 56420/225000 [117:14:29<325:45:29,  6.96s/it]
 25%|██▌       | 56421/225000 [117:14:35<303:09:14,  6.47s/it]
 25%|██▌       | 56422/225000 [117:14:40<287:06:41,  6.13s/it]
 25%|██▌       | 56423/225000 [117:14:45<276:23:35,  5.90s/it]
 25%|██▌       | 56424/225000 [117:14:50<260:17:50,  5.56s/it]
 25%|██▌       | 56425/225000 [117:15:03<364:17:57,  7.78s/it]
 25%|██▌       | 56426/225000 [117:15:09<337:19:13,  7.20s/it]
 25%|██▌       | 56427/225000 [117:15:18<360:54:13,  7.71s/it]
 25%|██▌       | 56428/225000 [117:15:23<320:10:12,  6.84s/it]
 25%|██▌       | 56429/225000 [117:15:31<334:57:39,  7.15s/it]
 25%|██▌       | 56430/225000 [117:15:36<310:55:14,  6.64s/it]
 25%|██▌       | 56431/225000 [117:15:45<340:10:04,  7.26s/it]
 25%|██▌       | 56432/225000 [117:15:49<300:29:59,  6.42s/it]
 25%|██▌       | 56433/225000 [117:15:54<276:20:53,  5.90s/it]
 25%|██▌       | 56434/225000 [117:15:59<264:09:52,  5.64s/it]
 25%|██▌       | 56435/225000 [117:16:07<296:55:07,  6.34s/it]
 25%|██▌       | 56436/225000 [117:16:17<350:50:52,  7.49s/it]
 25%|██▌       | 56437/225000 [117:16:21<301:47:41,  6.45s/it]
 25%|██▌       | 56438/225000 [117:16:28<312:34:17,  6.68s/it]
 25%|██▌       | 56439/225000 [117:16:38<354:24:33,  7.57s/it]
 25%|██▌       | 56440/225000 [117:16:42<303:44:51,  6.49s/it]
 25%|██▌       | 56441/225000 [117:16:51<339:58:51,  7.26s/it]
 25%|██▌       | 56442/225000 [117:16:57<328:45:51,  7.02s/it]
 25%|██▌       | 56443/225000 [117:17:03<304:07:52,  6.50s/it]
 25%|██▌       | 56444/225000 [117:17:08<289:40:02,  6.19s/it]
 25%|██▌       | 56445/225000 [117:17:14<282:15:31,  6.03s/it]
 25%|██▌       | 56446/225000 [117:17:19<269:30:00,  5.76s/it]
 25%|██▌       | 56447/225000 [117:17:24<264:57:38,  5.66s/it]
 25%|██▌       | 56448/225000 [117:17:30<265:07:07,  5.66s/it]
 25%|██▌       | 56449/225000 [117:17:36<269:47:08,  5.76s/it]
 25%|██▌       | 56450/225000 [117:17:49<372:11:50,  7.95s/it]
                                                              
{'loss': '0.7313', 'grad_norm': '0.6888', 'learning_rate': '4.447e-05', 'epoch': '0.2509', 'num_input_tokens_seen': 195555424, 'train_runtime': '4.223e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56450/225000 [117:17:49<372:11:50,  7.95s/it]
 25%|██▌       | 56451/225000 [117:17:55<342:40:49,  7.32s/it]
 25%|██▌       | 56452/225000 [117:18:03<357:19:22,  7.63s/it]
 25%|██▌       | 56453/225000 [117:18:08<308:54:22,  6.60s/it]
 25%|██▌       | 56454/225000 [117:18:12<277:56:40,  5.94s/it]
 25%|██▌       | 56455/225000 [117:18:22<338:39:51,  7.23s/it]
 25%|██▌       | 56456/225000 [117:18:28<318:08:59,  6.80s/it]
 25%|██▌       | 56457/225000 [117:18:33<299:40:25,  6.40s/it]
 25%|██▌       | 56458/225000 [117:18:42<330:20:25,  7.06s/it]
 25%|██▌       | 56459/225000 [117:18:47<297:34:06,  6.36s/it]
 25%|██▌       | 56460/225000 [117:18:52<279:24:43,  5.97s/it]
 25%|██▌       | 56461/225000 [117:18:57<268:29:06,  5.73s/it]
 25%|██▌       | 56462/225000 [117:19:10<365:41:47,  7.81s/it]
 25%|██▌       | 56463/225000 [117:19:18<372:47:49,  7.96s/it]
 25%|██▌       | 56464/225000 [117:19:25<364:15:46,  7.78s/it]
 25%|██▌       | 56465/225000 [117:19:31<337:00:23,  7.20s/it]
 25%|██▌       | 56466/225000 [117:19:38<337:27:53,  7.21s/it]
 25%|██▌       | 56467/225000 [117:19:43<299:20:57,  6.39s/it]
 25%|██▌       | 56468/225000 [117:19:48<280:40:45,  6.00s/it]
 25%|██▌       | 56469/225000 [117:19:53<260:38:56,  5.57s/it]
 25%|██▌       | 56470/225000 [117:19:57<250:55:28,  5.36s/it]
 25%|██▌       | 56471/225000 [117:20:09<342:11:07,  7.31s/it]
 25%|██▌       | 56472/225000 [117:20:16<331:35:44,  7.08s/it]
 25%|██▌       | 56473/225000 [117:20:25<356:24:56,  7.61s/it]
 25%|██▌       | 56474/225000 [117:20:30<331:07:28,  7.07s/it]
 25%|██▌       | 56475/225000 [117:20:37<321:57:09,  6.88s/it]
 25%|██▌       | 56476/225000 [117:20:45<344:29:18,  7.36s/it]
 25%|██▌       | 56477/225000 [117:20:50<303:03:10,  6.47s/it]
 25%|██▌       | 56478/225000 [117:20:54<272:38:30,  5.82s/it]
 25%|██▌       | 56479/225000 [117:21:05<338:43:19,  7.24s/it]
 25%|██▌       | 56480/225000 [117:21:10<307:16:28,  6.56s/it]
 25%|██▌       | 56481/225000 [117:21:17<311:53:26,  6.66s/it]
 25%|██▌       | 56482/225000 [117:21:22<295:07:58,  6.30s/it]
 25%|██▌       | 56483/225000 [117:21:28<289:07:49,  6.18s/it]
 25%|██▌       | 56484/225000 [117:21:34<282:59:22,  6.05s/it]
 25%|██▌       | 56485/225000 [117:21:39<274:46:20,  5.87s/it]
 25%|██▌       | 56486/225000 [117:21:45<272:42:34,  5.83s/it]
 25%|██▌       | 56487/225000 [117:21:51<274:23:34,  5.86s/it]
 25%|██▌       | 56488/225000 [117:22:01<332:25:01,  7.10s/it]
 25%|██▌       | 56489/225000 [117:22:12<387:26:19,  8.28s/it]
 25%|██▌       | 56490/225000 [117:22:16<325:57:58,  6.96s/it]
 25%|██▌       | 56491/225000 [117:22:20<290:35:10,  6.21s/it]
 25%|██▌       | 56492/225000 [117:22:27<306:10:52,  6.54s/it]
 25%|██▌       | 56493/225000 [117:22:36<339:08:43,  7.25s/it]
 25%|██▌       | 56494/225000 [117:22:43<336:39:58,  7.19s/it]
 25%|██▌       | 56495/225000 [117:22:49<307:44:49,  6.57s/it]
 25%|██▌       | 56496/225000 [117:22:54<292:03:42,  6.24s/it]
 25%|██▌       | 56497/225000 [117:22:58<267:05:32,  5.71s/it]
 25%|██▌       | 56498/225000 [117:23:04<262:50:05,  5.62s/it]
 25%|██▌       | 56499/225000 [117:23:13<311:51:48,  6.66s/it]
 25%|██▌       | 56500/225000 [117:23:23<361:09:43,  7.72s/it]
                                                              
{'loss': '0.6992', 'grad_norm': '0.7428', 'learning_rate': '4.445e-05', 'epoch': '0.2511', 'num_input_tokens_seen': 195705640, 'train_runtime': '4.226e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56500/225000 [117:23:23<361:09:43,  7.72s/it]
 25%|██▌       | 56501/225000 [117:23:30<354:22:16,  7.57s/it]
 25%|██▌       | 56502/225000 [117:23:35<310:34:49,  6.64s/it]
 25%|██▌       | 56503/225000 [117:23:39<280:34:21,  5.99s/it]
 25%|██▌       | 56504/225000 [117:23:44<258:03:29,  5.51s/it]
 25%|██▌       | 56505/225000 [117:23:54<330:23:17,  7.06s/it]
 25%|██▌       | 56506/225000 [117:24:01<330:36:51,  7.06s/it]
 25%|██▌       | 56507/225000 [117:24:08<318:52:11,  6.81s/it]
 25%|██▌       | 56508/225000 [117:24:13<298:50:55,  6.39s/it]
 25%|██▌       | 56509/225000 [117:24:20<308:59:32,  6.60s/it]
 25%|██▌       | 56510/225000 [117:24:27<308:09:15,  6.58s/it]
 25%|██▌       | 56511/225000 [117:24:33<304:19:53,  6.50s/it]
 25%|██▌       | 56512/225000 [117:24:39<302:17:36,  6.46s/it]
 25%|██▌       | 56513/225000 [117:24:48<335:22:30,  7.17s/it]
 25%|██▌       | 56514/225000 [117:24:54<315:55:56,  6.75s/it]
 25%|██▌       | 56515/225000 [117:24:59<285:33:30,  6.10s/it]
 25%|██▌       | 56516/225000 [117:25:10<366:52:11,  7.84s/it]
 25%|██▌       | 56517/225000 [117:25:15<320:16:36,  6.84s/it]
 25%|██▌       | 56518/225000 [117:25:19<286:32:31,  6.12s/it]
 25%|██▌       | 56519/225000 [117:25:27<306:27:07,  6.55s/it]
 25%|██▌       | 56520/225000 [117:25:36<343:50:25,  7.35s/it]
 25%|██▌       | 56521/225000 [117:25:44<354:57:08,  7.58s/it]
 25%|██▌       | 56522/225000 [117:25:48<307:04:08,  6.56s/it]
 25%|██▌       | 56523/225000 [117:25:57<339:50:53,  7.26s/it]
 25%|██▌       | 56524/225000 [117:26:02<297:52:54,  6.37s/it]
 25%|██▌       | 56525/225000 [117:26:10<324:31:52,  6.93s/it]
 25%|██▌       | 56526/225000 [117:26:18<335:56:47,  7.18s/it]
 25%|██▌       | 56527/225000 [117:26:26<346:58:11,  7.41s/it]
 25%|██▌       | 56528/225000 [117:26:30<310:03:38,  6.63s/it]
 25%|██▌       | 56529/225000 [117:26:34<272:36:39,  5.83s/it]
 25%|██▌       | 56530/225000 [117:26:45<341:43:07,  7.30s/it]
 25%|██▌       | 56531/225000 [117:26:50<307:31:44,  6.57s/it]
 25%|██▌       | 56532/225000 [117:26:57<315:01:35,  6.73s/it]
 25%|██▌       | 56533/225000 [117:27:05<334:51:35,  7.16s/it]
 25%|██▌       | 56534/225000 [117:27:11<318:39:04,  6.81s/it]
 25%|██▌       | 56535/225000 [117:27:16<293:03:19,  6.26s/it]
 25%|██▌       | 56536/225000 [117:27:22<282:40:06,  6.04s/it]
 25%|██▌       | 56537/225000 [117:27:27<269:03:44,  5.75s/it]
 25%|██▌       | 56538/225000 [117:27:32<257:51:18,  5.51s/it]
 25%|██▌       | 56539/225000 [117:27:44<345:51:03,  7.39s/it]
 25%|██▌       | 56540/225000 [117:27:49<312:09:21,  6.67s/it]
 25%|██▌       | 56541/225000 [117:27:53<282:42:43,  6.04s/it]
 25%|██▌       | 56542/225000 [117:28:01<302:14:10,  6.46s/it]
 25%|██▌       | 56543/225000 [117:28:10<347:10:55,  7.42s/it]
 25%|██▌       | 56544/225000 [117:28:18<359:00:42,  7.67s/it]
 25%|██▌       | 56545/225000 [117:28:23<313:11:51,  6.69s/it]
 25%|██▌       | 56546/225000 [117:28:32<341:21:24,  7.30s/it]
 25%|██▌       | 56547/225000 [117:28:35<292:23:51,  6.25s/it]
 25%|██▌       | 56548/225000 [117:28:41<283:57:55,  6.07s/it]
 25%|██▌       | 56549/225000 [117:28:47<278:06:19,  5.94s/it]
 25%|██▌       | 56550/225000 [117:28:52<274:46:01,  5.87s/it]
                                                              
{'loss': '0.7257', 'grad_norm': '0.7209', 'learning_rate': '4.444e-05', 'epoch': '0.2513', 'num_input_tokens_seen': 195856712, 'train_runtime': '4.229e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56550/225000 [117:28:52<274:46:01,  5.87s/it]
 25%|██▌       | 56551/225000 [117:29:06<379:09:24,  8.10s/it]
 25%|██▌       | 56552/225000 [117:29:10<326:33:58,  6.98s/it]
 25%|██▌       | 56553/225000 [117:29:15<294:45:57,  6.30s/it]
 25%|██▌       | 56554/225000 [117:29:20<282:35:05,  6.04s/it]
 25%|██▌       | 56555/225000 [117:29:25<270:38:00,  5.78s/it]
 25%|██▌       | 56556/225000 [117:29:30<258:51:25,  5.53s/it]
 25%|██▌       | 56557/225000 [117:29:43<359:33:20,  7.68s/it]
 25%|██▌       | 56558/225000 [117:29:49<334:10:00,  7.14s/it]
 25%|██▌       | 56559/225000 [117:29:56<337:33:32,  7.21s/it]
 25%|██▌       | 56560/225000 [117:30:00<286:27:43,  6.12s/it]
 25%|██▌       | 56561/225000 [117:30:08<310:56:34,  6.65s/it]
 25%|██▌       | 56562/225000 [117:30:12<271:50:42,  5.81s/it]
 25%|██▌       | 56563/225000 [117:30:21<329:22:38,  7.04s/it]
 25%|██▌       | 56564/225000 [117:30:27<314:37:33,  6.72s/it]
 25%|██▌       | 56565/225000 [117:30:33<297:16:40,  6.35s/it]
 25%|██▌       | 56566/225000 [117:30:38<282:03:33,  6.03s/it]
 25%|██▌       | 56567/225000 [117:30:43<270:09:22,  5.77s/it]
 25%|██▌       | 56568/225000 [117:30:48<256:35:50,  5.48s/it]
 25%|██▌       | 56569/225000 [117:30:54<253:44:08,  5.42s/it]
 25%|██▌       | 56570/225000 [117:31:06<348:23:51,  7.45s/it]
 25%|██▌       | 56571/225000 [117:31:13<352:32:33,  7.54s/it]
 25%|██▌       | 56572/225000 [117:31:18<310:59:55,  6.65s/it]
 25%|██▌       | 56573/225000 [117:31:22<276:31:44,  5.91s/it]
 25%|██▌       | 56574/225000 [117:31:32<328:24:02,  7.02s/it]
 25%|██▌       | 56575/225000 [117:31:38<312:51:31,  6.69s/it]
 25%|██▌       | 56576/225000 [117:31:46<337:58:40,  7.22s/it]
 25%|██▌       | 56577/225000 [117:31:54<345:14:05,  7.38s/it]
 25%|██▌       | 56578/225000 [117:31:58<302:17:34,  6.46s/it]
 25%|██▌       | 56579/225000 [117:32:04<298:15:40,  6.38s/it]
 25%|██▌       | 56580/225000 [117:32:11<299:32:10,  6.40s/it]
 25%|██▌       | 56581/225000 [117:32:15<273:59:35,  5.86s/it]
 25%|██▌       | 56582/225000 [117:32:25<320:25:18,  6.85s/it]
 25%|██▌       | 56583/225000 [117:32:33<337:41:04,  7.22s/it]
 25%|██▌       | 56584/225000 [117:32:40<345:36:08,  7.39s/it]
 25%|██▌       | 56585/225000 [117:32:44<295:44:37,  6.32s/it]
 25%|██▌       | 56586/225000 [117:32:53<331:19:46,  7.08s/it]
 25%|██▌       | 56587/225000 [117:32:58<295:32:17,  6.32s/it]
 25%|██▌       | 56588/225000 [117:33:03<280:58:20,  6.01s/it]
 25%|██▌       | 56589/225000 [117:33:11<310:14:41,  6.63s/it]
 25%|██▌       | 56590/225000 [117:33:17<295:37:53,  6.32s/it]
 25%|██▌       | 56591/225000 [117:33:22<284:49:17,  6.09s/it]
 25%|██▌       | 56592/225000 [117:33:28<278:51:47,  5.96s/it]
 25%|██▌       | 56593/225000 [117:33:33<271:37:08,  5.81s/it]
 25%|██▌       | 56594/225000 [117:33:39<273:31:03,  5.85s/it]
 25%|██▌       | 56595/225000 [117:33:45<269:07:26,  5.75s/it]
 25%|██▌       | 56596/225000 [117:33:50<266:46:51,  5.70s/it]
 25%|██▌       | 56597/225000 [117:34:02<348:23:35,  7.45s/it]
 25%|██▌       | 56598/225000 [117:34:11<371:11:16,  7.94s/it]
 25%|██▌       | 56599/225000 [117:34:17<340:37:23,  7.28s/it]
 25%|██▌       | 56600/225000 [117:34:23<331:24:55,  7.08s/it]
                                                              
{'loss': '0.7336', 'grad_norm': '0.8307', 'learning_rate': '4.443e-05', 'epoch': '0.2516', 'num_input_tokens_seen': 196005912, 'train_runtime': '4.233e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56600/225000 [117:34:23<331:24:55,  7.08s/it]
 25%|██▌       | 56601/225000 [117:34:27<288:15:25,  6.16s/it]
 25%|██▌       | 56602/225000 [117:34:34<295:41:51,  6.32s/it]
 25%|██▌       | 56603/225000 [117:34:44<351:41:35,  7.52s/it]
 25%|██▌       | 56604/225000 [117:34:48<296:08:17,  6.33s/it]
 25%|██▌       | 56605/225000 [117:34:57<336:45:12,  7.20s/it]
 25%|██▌       | 56606/225000 [117:35:02<303:07:38,  6.48s/it]
 25%|██▌       | 56607/225000 [117:35:07<278:27:45,  5.95s/it]
 25%|██▌       | 56608/225000 [117:35:14<300:09:53,  6.42s/it]
 25%|██▌       | 56609/225000 [117:35:26<370:57:36,  7.93s/it]
 25%|██▌       | 56610/225000 [117:35:31<329:59:22,  7.05s/it]
 25%|██▌       | 56611/225000 [117:35:36<303:53:48,  6.50s/it]
 25%|██▌       | 56612/225000 [117:35:41<284:14:32,  6.08s/it]
 25%|██▌       | 56613/225000 [117:35:52<360:44:37,  7.71s/it]
 25%|██▌       | 56614/225000 [117:35:59<349:37:31,  7.47s/it]
 25%|██▌       | 56615/225000 [117:36:03<296:50:39,  6.35s/it]
 25%|██▌       | 56616/225000 [117:36:08<273:48:28,  5.85s/it]
 25%|██▌       | 56617/225000 [117:36:19<348:29:44,  7.45s/it]
 25%|██▌       | 56618/225000 [117:36:24<310:27:30,  6.64s/it]
 25%|██▌       | 56619/225000 [117:36:31<312:52:58,  6.69s/it]
 25%|██▌       | 56620/225000 [117:36:37<307:49:01,  6.58s/it]
 25%|██▌       | 56621/225000 [117:36:45<323:22:22,  6.91s/it]
 25%|██▌       | 56622/225000 [117:36:50<308:18:11,  6.59s/it]
 25%|██▌       | 56623/225000 [117:36:59<330:03:34,  7.06s/it]
 25%|██▌       | 56624/225000 [117:37:07<344:22:45,  7.36s/it]
 25%|██▌       | 56625/225000 [117:37:11<298:30:54,  6.38s/it]
 25%|██▌       | 56626/225000 [117:37:15<268:33:55,  5.74s/it]
 25%|██▌       | 56627/225000 [117:37:20<252:53:55,  5.41s/it]
 25%|██▌       | 56628/225000 [117:37:31<336:35:27,  7.20s/it]
 25%|██▌       | 56629/225000 [117:37:35<295:25:24,  6.32s/it]
 25%|██▌       | 56630/225000 [117:37:41<286:49:28,  6.13s/it]
 25%|██▌       | 56631/225000 [117:37:50<331:29:31,  7.09s/it]
 25%|██▌       | 56632/225000 [117:37:55<296:40:07,  6.34s/it]
 25%|██▌       | 56633/225000 [117:38:00<282:59:11,  6.05s/it]
 25%|██▌       | 56634/225000 [117:38:06<273:24:13,  5.85s/it]
 25%|██▌       | 56635/225000 [117:38:11<265:54:10,  5.69s/it]
 25%|██▌       | 56636/225000 [117:38:17<264:51:14,  5.66s/it]
 25%|██▌       | 56637/225000 [117:38:24<285:03:13,  6.10s/it]
 25%|██▌       | 56638/225000 [117:38:35<354:42:47,  7.58s/it]
 25%|██▌       | 56639/225000 [117:38:39<309:52:18,  6.63s/it]
 25%|██▌       | 56640/225000 [117:38:44<283:51:29,  6.07s/it]
 25%|██▌       | 56641/225000 [117:38:57<383:57:38,  8.21s/it]
 25%|██▌       | 56642/225000 [117:39:01<327:32:29,  7.00s/it]
 25%|██▌       | 56643/225000 [117:39:06<297:10:35,  6.35s/it]
 25%|██▌       | 56644/225000 [117:39:12<287:07:06,  6.14s/it]
 25%|██▌       | 56645/225000 [117:39:18<282:09:32,  6.03s/it]
 25%|██▌       | 56646/225000 [117:39:23<269:13:06,  5.76s/it]
 25%|██▌       | 56647/225000 [117:39:33<337:12:59,  7.21s/it]
 25%|██▌       | 56648/225000 [117:39:40<329:47:04,  7.05s/it]
 25%|██▌       | 56649/225000 [117:39:47<331:22:12,  7.09s/it]
 25%|██▌       | 56650/225000 [117:39:53<312:10:12,  6.68s/it]
                                                              
{'loss': '0.7364', 'grad_norm': '0.8977', 'learning_rate': '4.442e-05', 'epoch': '0.2518', 'num_input_tokens_seen': 196154936, 'train_runtime': '4.236e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56650/225000 [117:39:53<312:10:12,  6.68s/it]
 25%|██▌       | 56651/225000 [117:40:02<343:31:28,  7.35s/it]
 25%|██▌       | 56652/225000 [117:40:06<303:04:28,  6.48s/it]
 25%|██▌       | 56653/225000 [117:40:11<280:23:51,  6.00s/it]
 25%|██▌       | 56654/225000 [117:40:16<264:48:52,  5.66s/it]
 25%|██▌       | 56655/225000 [117:40:27<337:18:20,  7.21s/it]
 25%|██▌       | 56656/225000 [117:40:32<316:36:11,  6.77s/it]
 25%|██▌       | 56657/225000 [117:40:38<294:57:19,  6.31s/it]
 25%|██▌       | 56658/225000 [117:40:43<278:38:30,  5.96s/it]
 25%|██▌       | 56659/225000 [117:40:48<262:57:11,  5.62s/it]
 25%|██▌       | 56660/225000 [117:40:57<311:05:49,  6.65s/it]
 25%|██▌       | 56661/225000 [117:41:06<353:57:08,  7.57s/it]
 25%|██▌       | 56662/225000 [117:41:11<307:47:15,  6.58s/it]
 25%|██▌       | 56663/225000 [117:41:16<285:46:20,  6.11s/it]
 25%|██▌       | 56664/225000 [117:41:20<264:44:56,  5.66s/it]
 25%|██▌       | 56665/225000 [117:41:30<323:08:10,  6.91s/it]
 25%|██▌       | 56666/225000 [117:41:40<357:38:54,  7.65s/it]
 25%|██▌       | 56667/225000 [117:41:45<328:59:47,  7.04s/it]
 25%|██▌       | 56668/225000 [117:41:51<310:18:39,  6.64s/it]
 25%|██▌       | 56669/225000 [117:42:00<342:55:02,  7.33s/it]
 25%|██▌       | 56670/225000 [117:42:04<298:13:50,  6.38s/it]
 25%|██▌       | 56671/225000 [117:42:11<313:47:59,  6.71s/it]
 25%|██▌       | 56672/225000 [117:42:20<337:45:15,  7.22s/it]
 25%|██▌       | 56673/225000 [117:42:24<289:22:48,  6.19s/it]
 25%|██▌       | 56674/225000 [117:42:31<300:34:08,  6.43s/it]
 25%|██▌       | 56675/225000 [117:42:38<308:15:23,  6.59s/it]
 25%|██▌       | 56676/225000 [117:42:46<331:52:31,  7.10s/it]
 25%|██▌       | 56677/225000 [117:42:51<297:54:15,  6.37s/it]
 25%|██▌       | 56678/225000 [117:42:55<271:33:51,  5.81s/it]
 25%|██▌       | 56679/225000 [117:43:00<258:51:02,  5.54s/it]
 25%|██▌       | 56680/225000 [117:43:07<279:12:38,  5.97s/it]
 25%|██▌       | 56681/225000 [117:43:16<325:36:55,  6.96s/it]
 25%|██▌       | 56682/225000 [117:43:22<307:26:12,  6.58s/it]
 25%|██▌       | 56683/225000 [117:43:29<312:22:04,  6.68s/it]
 25%|██▌       | 56684/225000 [117:43:33<276:41:01,  5.92s/it]
 25%|██▌       | 56685/225000 [117:43:39<281:00:05,  6.01s/it]
 25%|██▌       | 56686/225000 [117:43:50<351:25:15,  7.52s/it]
 25%|██▌       | 56687/225000 [117:43:57<342:32:50,  7.33s/it]
 25%|██▌       | 56688/225000 [117:44:01<297:21:45,  6.36s/it]
 25%|██▌       | 56689/225000 [117:44:10<330:36:58,  7.07s/it]
 25%|██▌       | 56690/225000 [117:44:14<288:46:23,  6.18s/it]
 25%|██▌       | 56691/225000 [117:44:23<326:50:53,  6.99s/it]
 25%|██▌       | 56692/225000 [117:44:28<293:38:47,  6.28s/it]
 25%|██▌       | 56693/225000 [117:44:32<266:05:00,  5.69s/it]
 25%|██▌       | 56694/225000 [117:44:38<276:18:44,  5.91s/it]
 25%|██▌       | 56695/225000 [117:44:49<349:11:18,  7.47s/it]
 25%|██▌       | 56696/225000 [117:44:54<309:13:49,  6.61s/it]
 25%|██▌       | 56697/225000 [117:44:59<283:32:49,  6.07s/it]
 25%|██▌       | 56698/225000 [117:45:03<264:11:22,  5.65s/it]
 25%|██▌       | 56699/225000 [117:45:09<268:06:04,  5.73s/it]
 25%|██▌       | 56700/225000 [117:45:18<310:15:35,  6.64s/it]
                                                              
{'loss': '0.7209', 'grad_norm': '0.8087', 'learning_rate': '4.441e-05', 'epoch': '0.252', 'num_input_tokens_seen': 196302728, 'train_runtime': '4.239e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56700/225000 [117:45:18<310:15:35,  6.64s/it]
 25%|██▌       | 56701/225000 [117:45:25<310:05:33,  6.63s/it]
 25%|██▌       | 56702/225000 [117:45:32<319:25:08,  6.83s/it]
 25%|██▌       | 56703/225000 [117:45:40<341:02:03,  7.29s/it]
 25%|██▌       | 56704/225000 [117:45:45<308:22:38,  6.60s/it]
 25%|██▌       | 56705/225000 [117:45:55<346:18:09,  7.41s/it]
 25%|██▌       | 56706/225000 [117:45:59<304:26:00,  6.51s/it]
 25%|██▌       | 56707/225000 [117:46:05<291:49:03,  6.24s/it]
 25%|██▌       | 56708/225000 [117:46:10<279:32:30,  5.98s/it]
 25%|██▌       | 56709/225000 [117:46:16<279:17:40,  5.97s/it]
 25%|██▌       | 56710/225000 [117:46:21<270:33:54,  5.79s/it]
 25%|██▌       | 56711/225000 [117:46:27<264:54:33,  5.67s/it]
 25%|██▌       | 56712/225000 [117:46:35<301:49:48,  6.46s/it]
 25%|██▌       | 56713/225000 [117:46:47<384:41:47,  8.23s/it]
 25%|██▌       | 56714/225000 [117:46:52<332:01:23,  7.10s/it]
 25%|██▌       | 56715/225000 [117:46:56<290:22:50,  6.21s/it]
 25%|██▌       | 56716/225000 [117:47:02<287:37:16,  6.15s/it]
 25%|██▌       | 56717/225000 [117:47:13<360:41:29,  7.72s/it]
 25%|██▌       | 56718/225000 [117:47:17<306:23:10,  6.55s/it]
 25%|██▌       | 56719/225000 [117:47:27<344:44:22,  7.37s/it]
 25%|██▌       | 56720/225000 [117:47:33<333:41:02,  7.14s/it]
 25%|██▌       | 56721/225000 [117:47:38<302:20:44,  6.47s/it]
 25%|██▌       | 56722/225000 [117:47:43<277:55:33,  5.95s/it]
 25%|██▌       | 56723/225000 [117:47:48<265:52:31,  5.69s/it]
 25%|██▌       | 56724/225000 [117:47:53<257:42:02,  5.51s/it]
 25%|██▌       | 56725/225000 [117:47:58<256:01:26,  5.48s/it]
 25%|██▌       | 56726/225000 [117:48:04<256:49:52,  5.49s/it]
 25%|██▌       | 56727/225000 [117:48:17<366:25:47,  7.84s/it]
 25%|██▌       | 56728/225000 [117:48:22<327:45:15,  7.01s/it]
 25%|██▌       | 56729/225000 [117:48:27<294:24:36,  6.30s/it]
 25%|██▌       | 56730/225000 [117:48:38<361:04:15,  7.72s/it]
 25%|██▌       | 56731/225000 [117:48:44<330:15:53,  7.07s/it]
 25%|██▌       | 56732/225000 [117:48:48<292:45:26,  6.26s/it]
 25%|██▌       | 56733/225000 [117:48:55<300:57:55,  6.44s/it]
 25%|██▌       | 56734/225000 [117:49:05<357:17:46,  7.64s/it]
 25%|██▌       | 56735/225000 [117:49:10<312:48:44,  6.69s/it]
 25%|██▌       | 56736/225000 [117:49:20<361:40:20,  7.74s/it]
 25%|██▌       | 56737/225000 [117:49:24<315:38:51,  6.75s/it]
 25%|██▌       | 56738/225000 [117:49:29<279:52:46,  5.99s/it]
 25%|██▌       | 56739/225000 [117:49:34<278:35:02,  5.96s/it]
 25%|██▌       | 56740/225000 [117:49:46<352:10:29,  7.53s/it]
 25%|██▌       | 56741/225000 [117:49:49<299:16:49,  6.40s/it]
 25%|██▌       | 56742/225000 [117:49:58<328:13:28,  7.02s/it]
 25%|██▌       | 56743/225000 [117:50:02<292:01:24,  6.25s/it]
 25%|██▌       | 56744/225000 [117:50:07<271:05:30,  5.80s/it]
 25%|██▌       | 56745/225000 [117:50:14<290:27:47,  6.21s/it]
 25%|██▌       | 56746/225000 [117:50:25<358:09:09,  7.66s/it]
 25%|██▌       | 56747/225000 [117:50:30<321:54:18,  6.89s/it]
 25%|██▌       | 56748/225000 [117:50:35<295:12:16,  6.32s/it]
 25%|██▌       | 56749/225000 [117:50:40<274:35:27,  5.88s/it]
 25%|██▌       | 56750/225000 [117:50:50<335:55:42,  7.19s/it]
                                                              
{'loss': '0.7264', 'grad_norm': '0.8825', 'learning_rate': '4.44e-05', 'epoch': '0.2522', 'num_input_tokens_seen': 196453232, 'train_runtime': '4.243e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56750/225000 [117:50:50<335:55:42,  7.19s/it]
 25%|██▌       | 56751/225000 [117:51:00<363:09:26,  7.77s/it]
 25%|██▌       | 56752/225000 [117:51:04<310:52:48,  6.65s/it]
 25%|██▌       | 56753/225000 [117:51:13<353:05:51,  7.56s/it]
 25%|██▌       | 56754/225000 [117:51:17<302:56:22,  6.48s/it]
 25%|██▌       | 56755/225000 [117:51:25<321:34:39,  6.88s/it]
 25%|██▌       | 56756/225000 [117:51:31<308:08:38,  6.59s/it]
 25%|██▌       | 56757/225000 [117:51:37<300:10:02,  6.42s/it]
 25%|██▌       | 56758/225000 [117:51:42<283:24:31,  6.06s/it]
 25%|██▌       | 56759/225000 [117:51:48<276:08:50,  5.91s/it]
 25%|██▌       | 56760/225000 [117:51:53<260:30:39,  5.57s/it]
 25%|██▌       | 56761/225000 [117:52:05<359:51:15,  7.70s/it]
 25%|██▌       | 56762/225000 [117:52:12<341:40:25,  7.31s/it]
 25%|██▌       | 56763/225000 [117:52:17<313:14:42,  6.70s/it]
 25%|██▌       | 56764/225000 [117:52:22<290:37:15,  6.22s/it]
 25%|██▌       | 56765/225000 [117:52:28<281:52:46,  6.03s/it]
 25%|██▌       | 56766/225000 [117:52:33<266:28:16,  5.70s/it]
 25%|██▌       | 56767/225000 [117:52:37<255:38:15,  5.47s/it]
 25%|██▌       | 56768/225000 [117:52:50<351:36:44,  7.52s/it]
 25%|██▌       | 56769/225000 [117:52:57<340:54:07,  7.30s/it]
 25%|██▌       | 56770/225000 [117:53:02<320:28:08,  6.86s/it]
 25%|██▌       | 56771/225000 [117:53:10<324:39:32,  6.95s/it]
 25%|██▌       | 56772/225000 [117:53:15<299:05:24,  6.40s/it]
 25%|██▌       | 56773/225000 [117:53:20<286:24:47,  6.13s/it]
 25%|██▌       | 56774/225000 [117:53:25<270:44:33,  5.79s/it]
 25%|██▌       | 56775/225000 [117:53:30<262:38:08,  5.62s/it]
 25%|██▌       | 56776/225000 [117:53:37<272:48:38,  5.84s/it]
 25%|██▌       | 56777/225000 [117:53:50<377:28:18,  8.08s/it]
 25%|██▌       | 56778/225000 [117:53:55<333:55:04,  7.15s/it]
 25%|██▌       | 56779/225000 [117:53:59<293:20:46,  6.28s/it]
 25%|██▌       | 56780/225000 [117:54:05<280:13:56,  6.00s/it]
 25%|██▌       | 56781/225000 [117:54:17<367:37:26,  7.87s/it]
 25%|██▌       | 56782/225000 [117:54:21<309:55:06,  6.63s/it]
 25%|██▌       | 56783/225000 [117:54:31<356:59:33,  7.64s/it]
 25%|██▌       | 56784/225000 [117:54:38<353:43:33,  7.57s/it]
 25%|██▌       | 56785/225000 [117:54:43<319:41:40,  6.84s/it]
 25%|██▌       | 56786/225000 [117:54:48<290:44:30,  6.22s/it]
 25%|██▌       | 56787/225000 [117:54:53<280:51:03,  6.01s/it]
 25%|██▌       | 56788/225000 [117:54:58<266:51:57,  5.71s/it]
 25%|██▌       | 56789/225000 [117:55:11<365:50:00,  7.83s/it]
 25%|██▌       | 56790/225000 [117:55:17<331:35:28,  7.10s/it]
 25%|██▌       | 56791/225000 [117:55:21<294:48:11,  6.31s/it]
 25%|██▌       | 56792/225000 [117:55:26<271:41:58,  5.81s/it]
 25%|██▌       | 56793/225000 [117:55:38<359:22:44,  7.69s/it]
 25%|██▌       | 56794/225000 [117:55:42<306:24:29,  6.56s/it]
 25%|██▌       | 56795/225000 [117:55:46<269:08:59,  5.76s/it]
 25%|██▌       | 56796/225000 [117:55:57<341:29:56,  7.31s/it]
 25%|██▌       | 56797/225000 [117:56:02<309:56:44,  6.63s/it]
 25%|██▌       | 56798/225000 [117:56:07<290:05:29,  6.21s/it]
 25%|██▌       | 56799/225000 [117:56:11<267:15:03,  5.72s/it]
 25%|██▌       | 56800/225000 [117:56:16<255:41:28,  5.47s/it]
                                                              
{'loss': '0.7342', 'grad_norm': '0.8152', 'learning_rate': '4.439e-05', 'epoch': '0.2524', 'num_input_tokens_seen': 196603608, 'train_runtime': '4.246e+05', 'train_tokens_per_second': '463.1'}

 25%|██▌       | 56800/225000 [117:56:16<255:41:28,  5.47s/it]
 25%|██▌       | 56801/225000 [117:56:29<360:37:06,  7.72s/it]
 25%|██▌       | 56802/225000 [117:56:33<311:01:25,  6.66s/it]
 25%|██▌       | 56803/225000 [117:56:41<327:58:43,  7.02s/it]
 25%|██▌       | 56804/225000 [117:56:45<288:06:49,  6.17s/it]
 25%|██▌       | 56805/225000 [117:56:50<265:35:12,  5.68s/it]
 25%|██▌       | 56806/225000 [117:57:01<338:45:06,  7.25s/it]
 25%|██▌       | 56807/225000 [117:57:06<314:14:17,  6.73s/it]
 25%|██▌       | 56808/225000 [117:57:15<342:24:16,  7.33s/it]
 25%|██▌       | 56809/225000 [117:57:21<314:12:57,  6.73s/it]
 25%|██▌       | 56810/225000 [117:57:26<297:48:59,  6.37s/it]
 25%|██▌       | 56811/225000 [117:57:31<284:39:33,  6.09s/it]
 25%|██▌       | 56812/225000 [117:57:36<267:26:25,  5.72s/it]
 25%|██▌       | 56813/225000 [117:57:42<264:53:11,  5.67s/it]
 25%|██▌       | 56814/225000 [117:57:47<263:21:13,  5.64s/it]
 25%|██▌       | 56815/225000 [117:57:53<259:39:35,  5.56s/it]
 25%|██▌       | 56816/225000 [117:58:05<351:19:41,  7.52s/it]
 25%|██▌       | 56817/225000 [117:58:12<338:16:53,  7.24s/it]
 25%|██▌       | 56818/225000 [117:58:17<315:03:20,  6.74s/it]
 25%|██▌       | 56819/225000 [117:58:23<299:17:26,  6.41s/it]
 25%|██▌       | 56820/225000 [117:58:28<289:10:22,  6.19s/it]
 25%|██▌       | 56821/225000 [117:58:38<342:26:44,  7.33s/it]
 25%|██▌       | 56822/225000 [117:58:48<378:33:03,  8.10s/it]
 25%|██▌       | 56823/225000 [117:58:52<317:50:54,  6.80s/it]
 25%|██▌       | 56824/225000 [117:58:57<291:11:31,  6.23s/it]
 25%|██▌       | 56825/225000 [117:59:02<267:58:07,  5.74s/it]
 25%|██▌       | 56826/225000 [117:59:14<361:59:56,  7.75s/it]
 25%|██▌       | 56827/225000 [117:59:20<338:38:55,  7.25s/it]
 25%|██▌       | 56828/225000 [117:59:29<356:45:08,  7.64s/it]
 25%|██▌       | 56829/225000 [117:59:32<303:44:39,  6.50s/it]
 25%|██▌       | 56830/225000 [117:59:39<306:33:55,  6.56s/it]
 25%|██▌       | 56831/225000 [117:59:47<319:43:15,  6.84s/it]
 25%|██▌       | 56832/225000 [117:59:51<281:12:53,  6.02s/it]
 25%|██▌       | 56833/225000 [118:00:00<331:46:25,  7.10s/it]
 25%|██▌       | 56834/225000 [118:00:07<323:06:31,  6.92s/it]
 25%|██▌       | 56835/225000 [118:00:16<352:34:42,  7.55s/it]
 25%|██▌       | 56836/225000 [118:00:20<299:37:10,  6.41s/it]
 25%|██▌       | 56837/225000 [118:00:29<334:00:33,  7.15s/it]
 25%|██▌       | 56838/225000 [118:00:33<291:01:25,  6.23s/it]
 25%|██▌       | 56839/225000 [118:00:38<280:51:02,  6.01s/it]
 25%|██▌       | 56840/225000 [118:00:44<282:24:42,  6.05s/it]
 25%|██▌       | 56841/225000 [118:00:50<280:40:57,  6.01s/it]
 25%|██▌       | 56842/225000 [118:00:55<270:15:57,  5.79s/it]
 25%|██▌       | 56843/225000 [118:01:02<274:59:32,  5.89s/it]
 25%|██▌       | 56844/225000 [118:01:07<270:15:14,  5.79s/it]
 25%|██▌       | 56845/225000 [118:01:13<264:44:30,  5.67s/it]
 25%|██▌       | 56846/225000 [118:01:25<364:18:28,  7.80s/it]
 25%|██▌       | 56847/225000 [118:01:32<348:00:27,  7.45s/it]
 25%|██▌       | 56848/225000 [118:01:36<303:13:50,  6.49s/it]
 25%|██▌       | 56849/225000 [118:01:41<280:11:39,  6.00s/it]
 25%|██▌       | 56850/225000 [118:01:49<307:56:19,  6.59s/it]
                                                              
{'loss': '0.7312', 'grad_norm': '0.7527', 'learning_rate': '4.437e-05', 'epoch': '0.2527', 'num_input_tokens_seen': 196753472, 'train_runtime': '4.249e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 56850/225000 [118:01:49<307:56:19,  6.59s/it]
 25%|██▌       | 56851/225000 [118:02:00<367:54:04,  7.88s/it]
 25%|██▌       | 56852/225000 [118:02:04<313:34:19,  6.71s/it]
 25%|██▌       | 56853/225000 [118:02:08<280:59:44,  6.02s/it]
 25%|██▌       | 56854/225000 [118:02:17<312:27:01,  6.69s/it]
 25%|██▌       | 56855/225000 [118:02:24<326:20:10,  6.99s/it]
 25%|██▌       | 56856/225000 [118:02:33<348:01:47,  7.45s/it]
 25%|██▌       | 56857/225000 [118:02:37<305:17:30,  6.54s/it]
 25%|██▌       | 56858/225000 [118:02:47<350:47:41,  7.51s/it]
 25%|██▌       | 56859/225000 [118:02:51<298:40:36,  6.39s/it]
 25%|██▌       | 56860/225000 [118:02:55<266:04:32,  5.70s/it]
 25%|██▌       | 56861/225000 [118:03:06<340:39:30,  7.29s/it]
 25%|██▌       | 56862/225000 [118:03:13<341:20:33,  7.31s/it]
 25%|██▌       | 56863/225000 [118:03:17<293:17:44,  6.28s/it]
 25%|██▌       | 56864/225000 [118:03:22<273:17:00,  5.85s/it]
 25%|██▌       | 56865/225000 [118:03:29<287:24:57,  6.15s/it]
 25%|██▌       | 56866/225000 [118:03:40<360:55:23,  7.73s/it]
 25%|██▌       | 56867/225000 [118:03:44<311:09:41,  6.66s/it]
 25%|██▌       | 56868/225000 [118:03:55<369:14:13,  7.91s/it]
 25%|██▌       | 56869/225000 [118:04:01<333:54:45,  7.15s/it]
 25%|██▌       | 56870/225000 [118:04:05<297:39:27,  6.37s/it]
 25%|██▌       | 56871/225000 [118:04:13<317:03:01,  6.79s/it]
 25%|██▌       | 56872/225000 [118:04:17<283:12:14,  6.06s/it]
 25%|██▌       | 56873/225000 [118:04:23<274:22:39,  5.88s/it]
 25%|██▌       | 56874/225000 [118:04:28<266:07:58,  5.70s/it]
 25%|██▌       | 56875/225000 [118:04:34<266:35:45,  5.71s/it]
 25%|██▌       | 56876/225000 [118:04:39<262:34:40,  5.62s/it]
 25%|██▌       | 56877/225000 [118:04:44<259:05:15,  5.55s/it]
 25%|██▌       | 56878/225000 [118:04:59<378:25:14,  8.10s/it]
 25%|██▌       | 56879/225000 [118:05:04<343:51:44,  7.36s/it]
 25%|██▌       | 56880/225000 [118:05:09<310:50:50,  6.66s/it]
 25%|██▌       | 56881/225000 [118:05:18<335:03:23,  7.17s/it]
 25%|██▌       | 56882/225000 [118:05:25<337:35:33,  7.23s/it]
 25%|██▌       | 56883/225000 [118:05:29<294:08:53,  6.30s/it]
 25%|██▌       | 56884/225000 [118:05:34<268:32:16,  5.75s/it]
 25%|██▌       | 56885/225000 [118:05:45<353:48:25,  7.58s/it]
 25%|██▌       | 56886/225000 [118:05:50<306:32:18,  6.56s/it]
 25%|██▌       | 56887/225000 [118:05:55<284:36:41,  6.09s/it]
 25%|██▌       | 56888/225000 [118:06:00<270:35:53,  5.79s/it]
 25%|██▌       | 56889/225000 [118:06:11<344:43:10,  7.38s/it]
 25%|██▌       | 56890/225000 [118:06:15<296:17:58,  6.35s/it]
 25%|██▌       | 56891/225000 [118:06:21<296:01:16,  6.34s/it]
 25%|██▌       | 56892/225000 [118:06:31<342:20:03,  7.33s/it]
 25%|██▌       | 56893/225000 [118:06:35<303:01:12,  6.49s/it]
 25%|██▌       | 56894/225000 [118:06:46<363:05:12,  7.78s/it]
 25%|██▌       | 56895/225000 [118:06:49<301:42:08,  6.46s/it]
 25%|██▌       | 56896/225000 [118:06:58<326:04:19,  6.98s/it]
 25%|██▌       | 56897/225000 [118:07:01<278:43:39,  5.97s/it]
 25%|██▌       | 56898/225000 [118:07:11<338:05:26,  7.24s/it]
 25%|██▌       | 56899/225000 [118:07:16<295:46:48,  6.33s/it]
 25%|██▌       | 56900/225000 [118:07:20<264:32:14,  5.67s/it]
                                                              
{'loss': '0.7349', 'grad_norm': '0.7301', 'learning_rate': '4.436e-05', 'epoch': '0.2529', 'num_input_tokens_seen': 196903216, 'train_runtime': '4.252e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 56900/225000 [118:07:20<264:32:14,  5.67s/it]
 25%|██▌       | 56901/225000 [118:07:31<350:58:18,  7.52s/it]
 25%|██▌       | 56902/225000 [118:07:36<303:50:17,  6.51s/it]
 25%|██▌       | 56903/225000 [118:07:43<312:09:51,  6.69s/it]
 25%|██▌       | 56904/225000 [118:07:49<309:02:00,  6.62s/it]
 25%|██▌       | 56905/225000 [118:07:58<337:34:51,  7.23s/it]
 25%|██▌       | 56906/225000 [118:08:06<344:54:05,  7.39s/it]
 25%|██▌       | 56907/225000 [118:08:09<291:50:11,  6.25s/it]
 25%|██▌       | 56908/225000 [118:08:18<324:19:39,  6.95s/it]
 25%|██▌       | 56909/225000 [118:08:23<296:13:38,  6.34s/it]
 25%|██▌       | 56910/225000 [118:08:28<277:12:05,  5.94s/it]
 25%|██▌       | 56911/225000 [118:08:33<261:31:36,  5.60s/it]
 25%|██▌       | 56912/225000 [118:08:43<329:08:33,  7.05s/it]
 25%|██▌       | 56913/225000 [118:08:51<336:26:05,  7.21s/it]
 25%|██▌       | 56914/225000 [118:09:00<362:39:16,  7.77s/it]
 25%|██▌       | 56915/225000 [118:09:04<319:04:53,  6.83s/it]
 25%|██▌       | 56916/225000 [118:09:09<283:26:04,  6.07s/it]
 25%|██▌       | 56917/225000 [118:09:13<258:05:33,  5.53s/it]
 25%|██▌       | 56918/225000 [118:09:24<340:08:26,  7.29s/it]
 25%|██▌       | 56919/225000 [118:09:30<322:23:07,  6.90s/it]
 25%|██▌       | 56920/225000 [118:09:39<347:39:26,  7.45s/it]
 25%|██▌       | 56921/225000 [118:09:44<309:27:21,  6.63s/it]
 25%|██▌       | 56922/225000 [118:09:48<281:20:29,  6.03s/it]
 25%|██▌       | 56923/225000 [118:10:01<371:46:09,  7.96s/it]
 25%|██▌       | 56924/225000 [118:10:05<319:44:07,  6.85s/it]
 25%|██▌       | 56925/225000 [118:10:11<306:49:55,  6.57s/it]
 25%|██▌       | 56926/225000 [118:10:16<286:01:28,  6.13s/it]
 25%|██▌       | 56927/225000 [118:10:22<280:02:06,  6.00s/it]
 25%|██▌       | 56928/225000 [118:10:27<269:41:42,  5.78s/it]
 25%|██▌       | 56929/225000 [118:10:33<266:19:40,  5.70s/it]
 25%|██▌       | 56930/225000 [118:10:40<290:31:43,  6.22s/it]
 25%|██▌       | 56931/225000 [118:10:53<386:14:43,  8.27s/it]
 25%|██▌       | 56932/225000 [118:10:58<345:46:42,  7.41s/it]
 25%|██▌       | 56933/225000 [118:11:06<346:37:22,  7.42s/it]
 25%|██▌       | 56934/225000 [118:11:12<323:05:44,  6.92s/it]
 25%|██▌       | 56935/225000 [118:11:17<303:31:34,  6.50s/it]
 25%|██▌       | 56936/225000 [118:11:23<295:50:36,  6.34s/it]
 25%|██▌       | 56937/225000 [118:11:29<286:18:34,  6.13s/it]
 25%|██▌       | 56938/225000 [118:11:40<352:17:47,  7.55s/it]
 25%|██▌       | 56939/225000 [118:11:44<304:16:03,  6.52s/it]
 25%|██▌       | 56940/225000 [118:11:51<321:28:38,  6.89s/it]
 25%|██▌       | 56941/225000 [118:12:00<344:25:53,  7.38s/it]
 25%|██▌       | 56942/225000 [118:12:05<306:22:04,  6.56s/it]
 25%|██▌       | 56943/225000 [118:12:10<284:03:27,  6.08s/it]
 25%|██▌       | 56944/225000 [118:12:14<265:06:11,  5.68s/it]
 25%|██▌       | 56945/225000 [118:12:24<316:30:22,  6.78s/it]
 25%|██▌       | 56946/225000 [118:12:31<320:06:43,  6.86s/it]
 25%|██▌       | 56947/225000 [118:12:38<321:20:21,  6.88s/it]
 25%|██▌       | 56948/225000 [118:12:45<325:22:03,  6.97s/it]
 25%|██▌       | 56949/225000 [118:12:52<331:19:59,  7.10s/it]
 25%|██▌       | 56950/225000 [118:12:56<286:54:51,  6.15s/it]
                                                              
{'loss': '0.7139', 'grad_norm': '0.8551', 'learning_rate': '4.435e-05', 'epoch': '0.2531', 'num_input_tokens_seen': 197053704, 'train_runtime': '4.256e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 56950/225000 [118:12:56<286:54:51,  6.15s/it]
 25%|██▌       | 56951/225000 [118:13:05<330:22:08,  7.08s/it]
 25%|██▌       | 56952/225000 [118:13:10<289:16:52,  6.20s/it]
 25%|██▌       | 56953/225000 [118:13:14<259:17:37,  5.55s/it]
 25%|██▌       | 56954/225000 [118:13:25<341:16:09,  7.31s/it]
 25%|██▌       | 56955/225000 [118:13:31<318:39:09,  6.83s/it]
 25%|██▌       | 56956/225000 [118:13:35<289:18:52,  6.20s/it]
 25%|██▌       | 56957/225000 [118:13:41<277:47:12,  5.95s/it]
 25%|██▌       | 56958/225000 [118:13:46<272:18:28,  5.83s/it]
 25%|██▌       | 56959/225000 [118:13:52<268:23:30,  5.75s/it]
 25%|██▌       | 56960/225000 [118:13:57<261:38:32,  5.61s/it]
 25%|██▌       | 56961/225000 [118:14:03<257:35:40,  5.52s/it]
 25%|██▌       | 56962/225000 [118:14:08<261:44:20,  5.61s/it]
 25%|██▌       | 56963/225000 [118:14:21<360:24:17,  7.72s/it]
 25%|██▌       | 56964/225000 [118:14:28<346:45:50,  7.43s/it]
 25%|██▌       | 56965/225000 [118:14:33<319:30:14,  6.85s/it]
 25%|██▌       | 56966/225000 [118:14:39<300:45:23,  6.44s/it]
 25%|██▌       | 56967/225000 [118:14:45<292:13:04,  6.26s/it]
 25%|██▌       | 56968/225000 [118:14:53<326:09:02,  6.99s/it]
 25%|██▌       | 56969/225000 [118:14:59<302:17:25,  6.48s/it]
 25%|██▌       | 56970/225000 [118:15:04<284:04:03,  6.09s/it]
 25%|██▌       | 56971/225000 [118:15:14<345:29:27,  7.40s/it]
 25%|██▌       | 56972/225000 [118:15:21<340:04:40,  7.29s/it]
 25%|██▌       | 56973/225000 [118:15:27<315:43:19,  6.76s/it]
 25%|██▌       | 56974/225000 [118:15:33<306:36:21,  6.57s/it]
 25%|██▌       | 56975/225000 [118:15:38<280:23:11,  6.01s/it]
 25%|██▌       | 56976/225000 [118:15:48<337:34:57,  7.23s/it]
 25%|██▌       | 56977/225000 [118:15:53<304:41:30,  6.53s/it]
 25%|██▌       | 56978/225000 [118:15:58<285:45:22,  6.12s/it]
 25%|██▌       | 56979/225000 [118:16:03<268:01:20,  5.74s/it]
 25%|██▌       | 56980/225000 [118:16:15<358:48:15,  7.69s/it]
 25%|██▌       | 56981/225000 [118:16:20<324:35:17,  6.95s/it]
 25%|██▌       | 56982/225000 [118:16:29<346:31:47,  7.42s/it]
 25%|██▌       | 56983/225000 [118:16:34<317:53:55,  6.81s/it]
 25%|██▌       | 56984/225000 [118:16:39<287:30:11,  6.16s/it]
 25%|██▌       | 56985/225000 [118:16:43<268:22:33,  5.75s/it]
 25%|██▌       | 56986/225000 [118:16:48<255:32:56,  5.48s/it]
 25%|██▌       | 56987/225000 [118:16:53<243:33:50,  5.22s/it]
 25%|██▌       | 56988/225000 [118:16:59<262:01:48,  5.61s/it]
 25%|██▌       | 56989/225000 [118:17:12<362:52:33,  7.78s/it]
 25%|██▌       | 56990/225000 [118:17:18<337:07:54,  7.22s/it]
 25%|██▌       | 56991/225000 [118:17:26<346:06:48,  7.42s/it]
 25%|██▌       | 56992/225000 [118:17:30<301:58:00,  6.47s/it]
 25%|██▌       | 56993/225000 [118:17:34<262:29:27,  5.62s/it]
 25%|██▌       | 56994/225000 [118:17:44<330:17:17,  7.08s/it]
 25%|██▌       | 56995/225000 [118:17:48<283:57:51,  6.08s/it]
 25%|██▌       | 56996/225000 [118:17:56<310:47:18,  6.66s/it]
 25%|██▌       | 56997/225000 [118:18:05<346:11:37,  7.42s/it]
 25%|██▌       | 56998/225000 [118:18:09<287:13:39,  6.15s/it]
 25%|██▌       | 56999/225000 [118:18:16<305:42:26,  6.55s/it]
 25%|██▌       | 57000/225000 [118:18:22<296:00:07,  6.34s/it]
                                                              
{'loss': '0.7188', 'grad_norm': '0.8196', 'learning_rate': '4.434e-05', 'epoch': '0.2533', 'num_input_tokens_seen': 197202720, 'train_runtime': '4.259e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57000/225000 [118:18:22<296:00:07,  6.34s/it]
 25%|██▌       | 57001/225000 [118:18:28<287:42:19,  6.17s/it]
 25%|██▌       | 57002/225000 [118:18:33<283:34:40,  6.08s/it]
 25%|██▌       | 57003/225000 [118:18:39<278:45:31,  5.97s/it]
 25%|██▌       | 57004/225000 [118:18:45<272:30:15,  5.84s/it]
 25%|██▌       | 57005/225000 [118:18:50<262:22:55,  5.62s/it]
 25%|██▌       | 57006/225000 [118:19:01<342:12:54,  7.33s/it]
 25%|██▌       | 57007/225000 [118:19:08<340:59:49,  7.31s/it]
 25%|██▌       | 57008/225000 [118:19:15<329:33:09,  7.06s/it]
 25%|██▌       | 57009/225000 [118:19:21<311:57:05,  6.69s/it]
 25%|██▌       | 57010/225000 [118:19:26<294:47:37,  6.32s/it]
 25%|██▌       | 57011/225000 [118:19:31<278:10:52,  5.96s/it]
 25%|██▌       | 57012/225000 [118:19:36<267:32:26,  5.73s/it]
 25%|██▌       | 57013/225000 [118:19:49<368:02:53,  7.89s/it]
 25%|██▌       | 57014/225000 [118:19:56<343:30:23,  7.36s/it]
 25%|██▌       | 57015/225000 [118:20:02<327:07:21,  7.01s/it]
 25%|██▌       | 57016/225000 [118:20:08<313:27:01,  6.72s/it]
 25%|██▌       | 57017/225000 [118:20:12<280:15:51,  6.01s/it]
 25%|██▌       | 57018/225000 [118:20:17<260:12:26,  5.58s/it]
 25%|██▌       | 57019/225000 [118:20:24<290:56:48,  6.24s/it]
 25%|██▌       | 57020/225000 [118:20:35<345:57:48,  7.41s/it]
 25%|██▌       | 57021/225000 [118:20:39<309:25:04,  6.63s/it]
 25%|██▌       | 57022/225000 [118:20:46<302:35:37,  6.48s/it]
 25%|██▌       | 57023/225000 [118:20:52<297:38:25,  6.38s/it]
 25%|██▌       | 57024/225000 [118:20:59<304:45:40,  6.53s/it]
 25%|██▌       | 57025/225000 [118:21:07<330:56:05,  7.09s/it]
 25%|██▌       | 57026/225000 [118:21:12<297:47:15,  6.38s/it]
 25%|██▌       | 57027/225000 [118:21:17<276:28:35,  5.93s/it]
 25%|██▌       | 57028/225000 [118:21:21<261:17:06,  5.60s/it]
 25%|██▌       | 57029/225000 [118:21:28<276:31:41,  5.93s/it]
 25%|██▌       | 57030/225000 [118:21:41<375:21:08,  8.04s/it]
 25%|██▌       | 57031/225000 [118:21:45<313:53:48,  6.73s/it]
 25%|██▌       | 57032/225000 [118:21:50<295:53:06,  6.34s/it]
 25%|██▌       | 57033/225000 [118:21:55<277:31:25,  5.95s/it]
 25%|██▌       | 57034/225000 [118:22:01<272:42:44,  5.85s/it]
 25%|██▌       | 57035/225000 [118:22:07<271:25:18,  5.82s/it]
 25%|██▌       | 57036/225000 [118:22:19<366:50:26,  7.86s/it]
 25%|██▌       | 57037/225000 [118:22:26<350:08:27,  7.50s/it]
 25%|██▌       | 57038/225000 [118:22:31<318:05:38,  6.82s/it]
 25%|██▌       | 57039/225000 [118:22:36<284:44:35,  6.10s/it]
 25%|██▌       | 57040/225000 [118:22:40<256:32:15,  5.50s/it]
 25%|██▌       | 57041/225000 [118:22:51<335:13:16,  7.19s/it]
 25%|██▌       | 57042/225000 [118:23:00<365:51:49,  7.84s/it]
 25%|██▌       | 57043/225000 [118:23:07<356:15:01,  7.64s/it]
 25%|██▌       | 57044/225000 [118:23:13<325:12:54,  6.97s/it]
 25%|██▌       | 57045/225000 [118:23:18<305:44:22,  6.55s/it]
 25%|██▌       | 57046/225000 [118:23:24<292:29:27,  6.27s/it]
 25%|██▌       | 57047/225000 [118:23:31<306:45:34,  6.58s/it]
 25%|██▌       | 57048/225000 [118:23:41<346:58:44,  7.44s/it]
 25%|██▌       | 57049/225000 [118:23:45<299:26:54,  6.42s/it]
 25%|██▌       | 57050/225000 [118:23:56<363:10:24,  7.78s/it]
                                                              
{'loss': '0.7241', 'grad_norm': '0.6809', 'learning_rate': '4.433e-05', 'epoch': '0.2536', 'num_input_tokens_seen': 197352528, 'train_runtime': '4.262e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57050/225000 [118:23:56<363:10:24,  7.78s/it]
 25%|██▌       | 57051/225000 [118:23:59<302:31:10,  6.48s/it]
 25%|██▌       | 57052/225000 [118:24:07<321:29:28,  6.89s/it]
 25%|██▌       | 57053/225000 [118:24:12<289:24:20,  6.20s/it]
 25%|██▌       | 57054/225000 [118:24:16<265:51:10,  5.70s/it]
 25%|██▌       | 57055/225000 [118:24:21<253:20:39,  5.43s/it]
 25%|██▌       | 57056/225000 [118:24:33<345:17:00,  7.40s/it]
 25%|██▌       | 57057/225000 [118:24:37<302:50:15,  6.49s/it]
 25%|██▌       | 57058/225000 [118:24:42<275:02:54,  5.90s/it]
 25%|██▌       | 57059/225000 [118:24:52<340:43:22,  7.30s/it]
 25%|██▌       | 57060/225000 [118:24:59<325:50:53,  6.98s/it]
 25%|██▌       | 57061/225000 [118:25:03<284:57:22,  6.11s/it]
 25%|██▌       | 57062/225000 [118:25:13<341:22:43,  7.32s/it]
 25%|██▌       | 57063/225000 [118:25:17<299:11:32,  6.41s/it]
 25%|██▌       | 57064/225000 [118:25:25<314:02:14,  6.73s/it]
 25%|██▌       | 57065/225000 [118:25:31<311:32:20,  6.68s/it]
 25%|██▌       | 57066/225000 [118:25:37<298:02:33,  6.39s/it]
 25%|██▌       | 57067/225000 [118:25:42<283:29:03,  6.08s/it]
 25%|██▌       | 57068/225000 [118:25:48<275:38:45,  5.91s/it]
 25%|██▌       | 57069/225000 [118:25:53<274:12:49,  5.88s/it]
 25%|██▌       | 57070/225000 [118:25:59<270:20:50,  5.80s/it]
 25%|██▌       | 57071/225000 [118:26:11<356:58:45,  7.65s/it]
 25%|██▌       | 57072/225000 [118:26:17<329:48:00,  7.07s/it]
 25%|██▌       | 57073/225000 [118:26:21<289:30:50,  6.21s/it]
 25%|██▌       | 57074/225000 [118:26:25<265:02:04,  5.68s/it]
 25%|██▌       | 57075/225000 [118:26:34<303:37:35,  6.51s/it]
 25%|██▌       | 57076/225000 [118:26:43<343:23:44,  7.36s/it]
 25%|██▌       | 57077/225000 [118:26:49<320:04:52,  6.86s/it]
 25%|██▌       | 57078/225000 [118:26:56<326:49:25,  7.01s/it]
 25%|██▌       | 57079/225000 [118:27:05<344:55:43,  7.39s/it]
 25%|██▌       | 57080/225000 [118:27:10<320:31:13,  6.87s/it]
 25%|██▌       | 57081/225000 [118:27:18<326:57:15,  7.01s/it]
 25%|██▌       | 57082/225000 [118:27:24<314:40:05,  6.75s/it]
 25%|██▌       | 57083/225000 [118:27:32<331:12:27,  7.10s/it]
 25%|██▌       | 57084/225000 [118:27:35<282:21:23,  6.05s/it]
 25%|██▌       | 57085/225000 [118:27:44<324:26:48,  6.96s/it]
 25%|██▌       | 57086/225000 [118:27:49<290:42:27,  6.23s/it]
 25%|██▌       | 57087/225000 [118:27:53<264:55:37,  5.68s/it]
 25%|██▌       | 57088/225000 [118:28:02<311:36:38,  6.68s/it]
 25%|██▌       | 57089/225000 [118:28:09<314:21:48,  6.74s/it]
 25%|██▌       | 57090/225000 [118:28:13<278:43:43,  5.98s/it]
 25%|██▌       | 57091/225000 [118:28:21<296:42:11,  6.36s/it]
 25%|██▌       | 57092/225000 [118:28:29<324:10:16,  6.95s/it]
 25%|██▌       | 57093/225000 [118:28:34<295:36:46,  6.34s/it]
 25%|██▌       | 57094/225000 [118:28:39<281:30:31,  6.04s/it]
 25%|██▌       | 57095/225000 [118:28:44<267:15:18,  5.73s/it]
 25%|██▌       | 57096/225000 [118:28:50<262:41:30,  5.63s/it]
 25%|██▌       | 57097/225000 [118:28:55<259:35:43,  5.57s/it]
 25%|██▌       | 57098/225000 [118:29:00<257:40:24,  5.52s/it]
 25%|██▌       | 57099/225000 [118:29:14<375:00:02,  8.04s/it]
 25%|██▌       | 57100/225000 [118:29:21<349:45:45,  7.50s/it]
                                                              
{'loss': '0.7231', 'grad_norm': '0.8106', 'learning_rate': '4.432e-05', 'epoch': '0.2538', 'num_input_tokens_seen': 197502032, 'train_runtime': '4.266e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57100/225000 [118:29:21<349:45:45,  7.50s/it]
 25%|██▌       | 57101/225000 [118:29:25<314:04:33,  6.73s/it]
 25%|██▌       | 57102/225000 [118:29:35<352:59:50,  7.57s/it]
 25%|██▌       | 57103/225000 [118:29:40<310:20:48,  6.65s/it]
 25%|██▌       | 57104/225000 [118:29:44<282:55:57,  6.07s/it]
 25%|██▌       | 57105/225000 [118:29:56<359:21:59,  7.71s/it]
 25%|██▌       | 57106/225000 [118:29:59<301:46:35,  6.47s/it]
 25%|██▌       | 57107/225000 [118:30:09<349:42:39,  7.50s/it]
 25%|██▌       | 57108/225000 [118:30:14<311:09:25,  6.67s/it]
 25%|██▌       | 57109/225000 [118:30:19<283:15:20,  6.07s/it]
 25%|██▌       | 57110/225000 [118:30:24<274:45:54,  5.89s/it]
 25%|██▌       | 57111/225000 [118:30:37<368:55:22,  7.91s/it]
 25%|██▌       | 57112/225000 [118:30:40<306:58:27,  6.58s/it]
 25%|██▌       | 57113/225000 [118:30:50<347:07:37,  7.44s/it]
 25%|██▌       | 57114/225000 [118:30:56<327:31:05,  7.02s/it]
 25%|██▌       | 57115/225000 [118:31:01<299:11:14,  6.42s/it]
 25%|██▌       | 57116/225000 [118:31:09<325:15:12,  6.97s/it]
 25%|██▌       | 57117/225000 [118:31:14<301:09:52,  6.46s/it]
 25%|██▌       | 57118/225000 [118:31:20<287:55:13,  6.17s/it]
 25%|██▌       | 57119/225000 [118:31:25<275:50:11,  5.91s/it]
 25%|██▌       | 57120/225000 [118:31:31<270:21:16,  5.80s/it]
 25%|██▌       | 57121/225000 [118:31:36<267:03:27,  5.73s/it]
 25%|██▌       | 57122/225000 [118:31:41<256:23:08,  5.50s/it]
 25%|██▌       | 57123/225000 [118:31:47<266:53:23,  5.72s/it]
 25%|██▌       | 57124/225000 [118:32:00<369:12:32,  7.92s/it]
 25%|██▌       | 57125/225000 [118:32:05<324:12:56,  6.95s/it]
 25%|██▌       | 57126/225000 [118:32:09<285:15:39,  6.12s/it]
 25%|██▌       | 57127/225000 [118:32:20<343:45:41,  7.37s/it]
 25%|██▌       | 57128/225000 [118:32:24<304:10:51,  6.52s/it]
 25%|██▌       | 57129/225000 [118:32:29<274:57:48,  5.90s/it]
 25%|██▌       | 57130/225000 [118:32:34<272:22:11,  5.84s/it]
 25%|██▌       | 57131/225000 [118:32:45<343:09:44,  7.36s/it]
 25%|██▌       | 57132/225000 [118:32:49<299:01:06,  6.41s/it]
 25%|██▌       | 57133/225000 [118:33:00<360:04:55,  7.72s/it]
 25%|██▌       | 57134/225000 [118:33:05<317:01:30,  6.80s/it]
 25%|██▌       | 57135/225000 [118:33:09<284:10:51,  6.09s/it]
 25%|██▌       | 57136/225000 [118:33:14<270:34:07,  5.80s/it]
 25%|██▌       | 57137/225000 [118:33:26<347:23:45,  7.45s/it]
 25%|██▌       | 57138/225000 [118:33:32<330:50:58,  7.10s/it]
 25%|██▌       | 57139/225000 [118:33:37<296:53:07,  6.37s/it]
 25%|██▌       | 57140/225000 [118:33:41<267:41:41,  5.74s/it]
 25%|██▌       | 57141/225000 [118:33:53<357:01:25,  7.66s/it]
 25%|██▌       | 57142/225000 [118:33:57<300:57:32,  6.45s/it]
 25%|██▌       | 57143/225000 [118:34:02<283:11:32,  6.07s/it]
 25%|██▌       | 57144/225000 [118:34:07<273:59:09,  5.88s/it]
 25%|██▌       | 57145/225000 [118:34:16<318:52:29,  6.84s/it]
 25%|██▌       | 57146/225000 [118:34:21<291:13:49,  6.25s/it]
 25%|██▌       | 57147/225000 [118:34:31<341:22:50,  7.32s/it]
 25%|██▌       | 57148/225000 [118:34:37<320:41:03,  6.88s/it]
 25%|██▌       | 57149/225000 [118:34:43<307:57:05,  6.60s/it]
 25%|██▌       | 57150/225000 [118:34:48<293:36:31,  6.30s/it]
                                                              
{'loss': '0.737', 'grad_norm': '0.731', 'learning_rate': '4.43e-05', 'epoch': '0.254', 'num_input_tokens_seen': 197650304, 'train_runtime': '4.269e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57150/225000 [118:34:48<293:36:31,  6.30s/it]
 25%|██▌       | 57151/225000 [118:34:58<345:12:04,  7.40s/it]
 25%|██▌       | 57152/225000 [118:35:03<306:51:24,  6.58s/it]
 25%|██▌       | 57153/225000 [118:35:08<278:01:02,  5.96s/it]
 25%|██▌       | 57154/225000 [118:35:12<260:37:39,  5.59s/it]
 25%|██▌       | 57155/225000 [118:35:17<252:26:15,  5.41s/it]
 25%|██▌       | 57156/225000 [118:35:22<246:23:34,  5.28s/it]
 25%|██▌       | 57157/225000 [118:35:32<306:09:48,  6.57s/it]
 25%|██▌       | 57158/225000 [118:35:41<347:18:57,  7.45s/it]
 25%|██▌       | 57159/225000 [118:35:45<298:16:56,  6.40s/it]
 25%|██▌       | 57160/225000 [118:35:54<336:52:35,  7.23s/it]
 25%|██▌       | 57161/225000 [118:35:58<285:50:57,  6.13s/it]
 25%|██▌       | 57162/225000 [118:36:06<311:45:37,  6.69s/it]
 25%|██▌       | 57163/225000 [118:36:11<282:19:47,  6.06s/it]
 25%|██▌       | 57164/225000 [118:36:16<275:18:48,  5.91s/it]
 25%|██▌       | 57165/225000 [118:36:22<268:57:05,  5.77s/it]
 25%|██▌       | 57166/225000 [118:36:27<263:13:29,  5.65s/it]
 25%|██▌       | 57167/225000 [118:36:33<265:53:20,  5.70s/it]
 25%|██▌       | 57168/225000 [118:36:44<344:46:38,  7.40s/it]
 25%|██▌       | 57169/225000 [118:36:53<369:03:42,  7.92s/it]
 25%|██▌       | 57170/225000 [118:36:58<328:31:01,  7.05s/it]
 25%|██▌       | 57171/225000 [118:37:06<342:32:35,  7.35s/it]
 25%|██▌       | 57172/225000 [118:37:14<353:50:03,  7.59s/it]
 25%|██▌       | 57173/225000 [118:37:19<304:05:17,  6.52s/it]
 25%|██▌       | 57174/225000 [118:37:23<268:36:40,  5.76s/it]
 25%|██▌       | 57175/225000 [118:37:35<362:36:30,  7.78s/it]
 25%|██▌       | 57176/225000 [118:37:38<302:10:49,  6.48s/it]
 25%|██▌       | 57177/225000 [118:37:47<334:13:17,  7.17s/it]
 25%|██▌       | 57178/225000 [118:37:51<280:19:21,  6.01s/it]
 25%|██▌       | 57179/225000 [118:38:00<324:56:26,  6.97s/it]
 25%|██▌       | 57180/225000 [118:38:04<289:48:34,  6.22s/it]
 25%|██▌       | 57181/225000 [118:38:09<274:51:19,  5.90s/it]
 25%|██▌       | 57182/225000 [118:38:15<264:34:07,  5.68s/it]
 25%|██▌       | 57183/225000 [118:38:20<258:22:12,  5.54s/it]
 25%|██▌       | 57184/225000 [118:38:25<252:41:14,  5.42s/it]
 25%|██▌       | 57185/225000 [118:38:30<255:24:13,  5.48s/it]
 25%|██▌       | 57186/225000 [118:38:36<260:01:16,  5.58s/it]
 25%|██▌       | 57187/225000 [118:38:41<254:28:59,  5.46s/it]
 25%|██▌       | 57188/225000 [118:38:53<336:16:12,  7.21s/it]
 25%|██▌       | 57189/225000 [118:39:01<351:24:02,  7.54s/it]
 25%|██▌       | 57190/225000 [118:39:06<309:43:30,  6.64s/it]
 25%|██▌       | 57191/225000 [118:39:14<339:19:35,  7.28s/it]
 25%|██▌       | 57192/225000 [118:39:21<336:09:43,  7.21s/it]
 25%|██▌       | 57193/225000 [118:39:30<355:12:15,  7.62s/it]
 25%|██▌       | 57194/225000 [118:39:36<334:51:30,  7.18s/it]
 25%|██▌       | 57195/225000 [118:39:43<329:53:39,  7.08s/it]
 25%|██▌       | 57196/225000 [118:39:49<312:18:24,  6.70s/it]
 25%|██▌       | 57197/225000 [118:39:56<314:37:39,  6.75s/it]
 25%|██▌       | 57198/225000 [118:40:04<332:35:59,  7.14s/it]
 25%|██▌       | 57199/225000 [118:40:09<312:32:24,  6.71s/it]
 25%|██▌       | 57200/225000 [118:40:16<316:09:38,  6.78s/it]
                                                              
{'loss': '0.7293', 'grad_norm': '0.8139', 'learning_rate': '4.429e-05', 'epoch': '0.2542', 'num_input_tokens_seen': 197799152, 'train_runtime': '4.272e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57200/225000 [118:40:16<316:09:38,  6.78s/it]
 25%|██▌       | 57201/225000 [118:40:21<284:08:17,  6.10s/it]
 25%|██▌       | 57202/225000 [118:40:26<269:02:23,  5.77s/it]
 25%|██▌       | 57203/225000 [118:40:37<340:20:52,  7.30s/it]
 25%|██▌       | 57204/225000 [118:40:41<293:28:43,  6.30s/it]
 25%|██▌       | 57205/225000 [118:40:50<327:53:34,  7.03s/it]
 25%|██▌       | 57206/225000 [118:40:55<299:50:52,  6.43s/it]
 25%|██▌       | 57207/225000 [118:40:59<277:28:34,  5.95s/it]
 25%|██▌       | 57208/225000 [118:41:10<339:57:55,  7.29s/it]
 25%|██▌       | 57209/225000 [118:41:13<288:22:17,  6.19s/it]
 25%|██▌       | 57210/225000 [118:41:24<356:28:00,  7.65s/it]
 25%|██▌       | 57211/225000 [118:41:29<310:49:16,  6.67s/it]
 25%|██▌       | 57212/225000 [118:41:34<295:15:01,  6.33s/it]
 25%|██▌       | 57213/225000 [118:41:44<343:52:18,  7.38s/it]
 25%|██▌       | 57214/225000 [118:41:48<295:56:28,  6.35s/it]
 25%|██▌       | 57215/225000 [118:41:55<301:36:42,  6.47s/it]
 25%|██▌       | 57216/225000 [118:42:04<341:18:28,  7.32s/it]
 25%|██▌       | 57217/225000 [118:42:09<302:55:24,  6.50s/it]
 25%|██▌       | 57218/225000 [118:42:14<281:07:40,  6.03s/it]
 25%|██▌       | 57219/225000 [118:42:19<267:09:15,  5.73s/it]
 25%|██▌       | 57220/225000 [118:42:30<337:08:35,  7.23s/it]
 25%|██▌       | 57221/225000 [118:42:35<309:36:23,  6.64s/it]
 25%|██▌       | 57222/225000 [118:42:42<311:06:57,  6.68s/it]
 25%|██▌       | 57223/225000 [118:42:51<345:33:07,  7.41s/it]
 25%|██▌       | 57224/225000 [118:42:56<315:38:24,  6.77s/it]
 25%|██▌       | 57225/225000 [118:43:01<288:53:02,  6.20s/it]
 25%|██▌       | 57226/225000 [118:43:06<269:36:17,  5.79s/it]
 25%|██▌       | 57227/225000 [118:43:11<267:02:35,  5.73s/it]
 25%|██▌       | 57228/225000 [118:43:17<261:56:03,  5.62s/it]
 25%|██▌       | 57229/225000 [118:43:30<365:46:33,  7.85s/it]
 25%|██▌       | 57230/225000 [118:43:34<318:57:26,  6.84s/it]
 25%|██▌       | 57231/225000 [118:43:44<362:47:59,  7.78s/it]
 25%|██▌       | 57232/225000 [118:43:49<324:29:16,  6.96s/it]
 25%|██▌       | 57233/225000 [118:43:54<300:07:12,  6.44s/it]
 25%|██▌       | 57234/225000 [118:44:00<282:40:17,  6.07s/it]
 25%|██▌       | 57235/225000 [118:44:05<268:32:31,  5.76s/it]
 25%|██▌       | 57236/225000 [118:44:14<320:00:41,  6.87s/it]
 25%|██▌       | 57237/225000 [118:44:24<367:07:39,  7.88s/it]
 25%|██▌       | 57238/225000 [118:44:28<305:03:53,  6.55s/it]
 25%|██▌       | 57239/225000 [118:44:37<346:00:26,  7.43s/it]
 25%|██▌       | 57240/225000 [118:44:43<322:59:38,  6.93s/it]
 25%|██▌       | 57241/225000 [118:44:49<304:41:26,  6.54s/it]
 25%|██▌       | 57242/225000 [118:44:54<282:49:30,  6.07s/it]
 25%|██▌       | 57243/225000 [118:44:59<272:33:27,  5.85s/it]
 25%|██▌       | 57244/225000 [118:45:04<263:05:34,  5.65s/it]
 25%|██▌       | 57245/225000 [118:45:16<351:16:55,  7.54s/it]
 25%|██▌       | 57246/225000 [118:45:22<333:33:32,  7.16s/it]
 25%|██▌       | 57247/225000 [118:45:30<340:35:52,  7.31s/it]
 25%|██▌       | 57248/225000 [118:45:34<300:53:53,  6.46s/it]
 25%|██▌       | 57249/225000 [118:45:40<287:45:54,  6.18s/it]
 25%|██▌       | 57250/225000 [118:45:46<281:09:09,  6.03s/it]
                                                              
{'loss': '0.7198', 'grad_norm': '0.7279', 'learning_rate': '4.428e-05', 'epoch': '0.2544', 'num_input_tokens_seen': 197949576, 'train_runtime': '4.275e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57250/225000 [118:45:46<281:09:09,  6.03s/it]
 25%|██▌       | 57251/225000 [118:45:51<274:29:00,  5.89s/it]
 25%|██▌       | 57252/225000 [118:45:56<261:28:47,  5.61s/it]
 25%|██▌       | 57253/225000 [118:46:02<261:04:35,  5.60s/it]
 25%|██▌       | 57254/225000 [118:46:10<302:21:24,  6.49s/it]
 25%|██▌       | 57255/225000 [118:46:23<382:40:47,  8.21s/it]
 25%|██▌       | 57256/225000 [118:46:26<313:25:10,  6.73s/it]
 25%|██▌       | 57257/225000 [118:46:36<359:08:20,  7.71s/it]
 25%|██▌       | 57258/225000 [118:46:40<315:30:53,  6.77s/it]
 25%|██▌       | 57259/225000 [118:46:49<337:38:43,  7.25s/it]
 25%|██▌       | 57260/225000 [118:46:53<298:49:02,  6.41s/it]
 25%|██▌       | 57261/225000 [118:47:02<326:13:59,  7.00s/it]
 25%|██▌       | 57262/225000 [118:47:07<299:57:16,  6.44s/it]
 25%|██▌       | 57263/225000 [118:47:12<282:57:51,  6.07s/it]
 25%|██▌       | 57264/225000 [118:47:16<261:21:59,  5.61s/it]
 25%|██▌       | 57265/225000 [118:47:29<358:55:32,  7.70s/it]
 25%|██▌       | 57266/225000 [118:47:36<345:53:38,  7.42s/it]
 25%|██▌       | 57267/225000 [118:47:39<291:14:03,  6.25s/it]
 25%|██▌       | 57268/225000 [118:47:49<340:08:36,  7.30s/it]
 25%|██▌       | 57269/225000 [118:47:54<313:26:17,  6.73s/it]
 25%|██▌       | 57270/225000 [118:47:59<287:24:41,  6.17s/it]
 25%|██▌       | 57271/225000 [118:48:05<277:23:45,  5.95s/it]
 25%|██▌       | 57272/225000 [118:48:10<262:58:01,  5.64s/it]
 25%|██▌       | 57273/225000 [118:48:19<313:57:02,  6.74s/it]
 25%|██▌       | 57274/225000 [118:48:28<348:11:48,  7.47s/it]
 25%|██▌       | 57275/225000 [118:48:33<305:40:03,  6.56s/it]
 25%|██▌       | 57276/225000 [118:48:40<317:49:47,  6.82s/it]
 25%|██▌       | 57277/225000 [118:48:44<282:36:57,  6.07s/it]
 25%|██▌       | 57278/225000 [118:48:50<275:04:33,  5.90s/it]
 25%|██▌       | 57279/225000 [118:49:00<330:00:33,  7.08s/it]
 25%|██▌       | 57280/225000 [118:49:05<301:51:11,  6.48s/it]
 25%|██▌       | 57281/225000 [118:49:10<285:08:43,  6.12s/it]
 25%|██▌       | 57282/225000 [118:49:15<269:17:33,  5.78s/it]
 25%|██▌       | 57283/225000 [118:49:20<254:30:26,  5.46s/it]
 25%|██▌       | 57284/225000 [118:49:32<346:03:16,  7.43s/it]
 25%|██▌       | 57285/225000 [118:49:39<342:45:19,  7.36s/it]
 25%|██▌       | 57286/225000 [118:49:43<294:23:09,  6.32s/it]
 25%|██▌       | 57287/225000 [118:49:48<270:53:21,  5.81s/it]
 25%|██▌       | 57288/225000 [118:49:53<262:05:11,  5.63s/it]
 25%|██▌       | 57289/225000 [118:50:05<359:23:47,  7.71s/it]
 25%|██▌       | 57290/225000 [118:50:11<327:24:49,  7.03s/it]
 25%|██▌       | 57291/225000 [118:50:15<292:05:23,  6.27s/it]
 25%|██▌       | 57292/225000 [118:50:20<271:34:54,  5.83s/it]
 25%|██▌       | 57293/225000 [118:50:33<369:58:28,  7.94s/it]
 25%|██▌       | 57294/225000 [118:50:39<347:52:46,  7.47s/it]
 25%|██▌       | 57295/225000 [118:50:46<333:03:26,  7.15s/it]
 25%|██▌       | 57296/225000 [118:50:50<290:10:11,  6.23s/it]
 25%|██▌       | 57297/225000 [118:50:55<281:04:56,  6.03s/it]
 25%|██▌       | 57298/225000 [118:51:07<359:57:21,  7.73s/it]
 25%|██▌       | 57299/225000 [118:51:13<335:03:44,  7.19s/it]
 25%|██▌       | 57300/225000 [118:51:19<315:51:37,  6.78s/it]
                                                              
{'loss': '0.743', 'grad_norm': '0.7436', 'learning_rate': '4.427e-05', 'epoch': '0.2547', 'num_input_tokens_seen': 198098912, 'train_runtime': '4.279e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57300/225000 [118:51:19<315:51:37,  6.78s/it]
 25%|██▌       | 57301/225000 [118:51:26<323:37:59,  6.95s/it]
 25%|██▌       | 57302/225000 [118:51:32<306:37:46,  6.58s/it]
 25%|██▌       | 57303/225000 [118:51:36<270:41:30,  5.81s/it]
 25%|██▌       | 57304/225000 [118:51:41<261:35:46,  5.62s/it]
 25%|██▌       | 57305/225000 [118:51:52<337:11:59,  7.24s/it]
 25%|██▌       | 57306/225000 [118:51:57<311:09:18,  6.68s/it]
 25%|██▌       | 57307/225000 [118:52:04<309:35:40,  6.65s/it]
 25%|██▌       | 57308/225000 [118:52:10<297:15:10,  6.38s/it]
 25%|██▌       | 57309/225000 [118:52:15<284:07:25,  6.10s/it]
 25%|██▌       | 57310/225000 [118:52:21<274:49:32,  5.90s/it]
 25%|██▌       | 57311/225000 [118:52:26<272:52:10,  5.86s/it]
 25%|██▌       | 57312/225000 [118:52:32<268:56:59,  5.77s/it]
 25%|██▌       | 57313/225000 [118:52:38<273:06:44,  5.86s/it]
 25%|██▌       | 57314/225000 [118:52:52<383:25:10,  8.23s/it]
 25%|██▌       | 57315/225000 [118:52:57<337:55:08,  7.25s/it]
 25%|██▌       | 57316/225000 [118:53:04<343:23:12,  7.37s/it]
 25%|██▌       | 57317/225000 [118:53:11<330:07:12,  7.09s/it]
 25%|██▌       | 57318/225000 [118:53:15<290:48:45,  6.24s/it]
 25%|██▌       | 57319/225000 [118:53:20<264:46:51,  5.68s/it]
 25%|██▌       | 57320/225000 [118:53:24<249:01:52,  5.35s/it]
 25%|██▌       | 57321/225000 [118:53:32<289:09:35,  6.21s/it]
 25%|██▌       | 57322/225000 [118:53:44<368:01:18,  7.90s/it]
 25%|██▌       | 57323/225000 [118:53:49<320:02:21,  6.87s/it]
 25%|██▌       | 57324/225000 [118:53:54<299:12:47,  6.42s/it]
 25%|██▌       | 57325/225000 [118:54:03<331:20:32,  7.11s/it]
 25%|██▌       | 57326/225000 [118:54:11<342:10:39,  7.35s/it]
 25%|██▌       | 57327/225000 [118:54:18<348:47:40,  7.49s/it]
 25%|██▌       | 57328/225000 [118:54:22<297:59:46,  6.40s/it]
 25%|██▌       | 57329/225000 [118:54:27<280:41:34,  6.03s/it]
 25%|██▌       | 57330/225000 [118:54:38<342:44:59,  7.36s/it]
 25%|██▌       | 57331/225000 [118:54:45<336:35:59,  7.23s/it]
 25%|██▌       | 57332/225000 [118:54:49<291:35:11,  6.26s/it]
 25%|██▌       | 57333/225000 [118:54:54<269:37:20,  5.79s/it]
 25%|██▌       | 57334/225000 [118:54:58<250:18:06,  5.37s/it]
 25%|██▌       | 57335/225000 [118:55:09<330:25:11,  7.09s/it]
 25%|██▌       | 57336/225000 [118:55:16<332:50:26,  7.15s/it]
 25%|██▌       | 57337/225000 [118:55:21<305:38:46,  6.56s/it]
 25%|██▌       | 57338/225000 [118:55:27<284:54:27,  6.12s/it]
 25%|██▌       | 57339/225000 [118:55:31<264:54:32,  5.69s/it]
 25%|██▌       | 57340/225000 [118:55:37<267:13:16,  5.74s/it]
 25%|██▌       | 57341/225000 [118:55:49<350:41:51,  7.53s/it]
 25%|██▌       | 57342/225000 [118:55:53<310:21:03,  6.66s/it]
 25%|██▌       | 57343/225000 [118:55:58<276:37:38,  5.94s/it]
 25%|██▌       | 57344/225000 [118:56:05<296:39:53,  6.37s/it]
 25%|██▌       | 57345/225000 [118:56:15<344:48:17,  7.40s/it]
 25%|██▌       | 57346/225000 [118:56:18<290:36:20,  6.24s/it]
 25%|██▌       | 57347/225000 [118:56:26<304:03:34,  6.53s/it]
 25%|██▌       | 57348/225000 [118:56:30<277:06:15,  5.95s/it]
 25%|██▌       | 57349/225000 [118:56:35<259:38:06,  5.58s/it]
 25%|██▌       | 57350/225000 [118:56:48<365:55:14,  7.86s/it]
                                                              
{'loss': '0.7362', 'grad_norm': '0.9689', 'learning_rate': '4.426e-05', 'epoch': '0.2549', 'num_input_tokens_seen': 198247976, 'train_runtime': '4.282e+05', 'train_tokens_per_second': '463'}

 25%|██▌       | 57350/225000 [118:56:48<365:55:14,  7.86s/it]
 25%|██▌       | 57351/225000 [118:56:53<318:26:30,  6.84s/it]
 25%|██▌       | 57352/225000 [118:56:58<292:52:14,  6.29s/it]
 25%|██▌       | 57353/225000 [118:57:02<272:42:09,  5.86s/it]
 25%|██▌       | 57354/225000 [118:57:08<263:04:11,  5.65s/it]
 25%|██▌       | 57355/225000 [118:57:21<366:09:27,  7.86s/it]
 25%|██▌       | 57356/225000 [118:57:26<333:11:47,  7.16s/it]
 25%|██▌       | 57357/225000 [118:57:30<292:13:28,  6.28s/it]
 25%|██▌       | 57358/225000 [118:57:40<333:19:19,  7.16s/it]
 25%|██▌       | 57359/225000 [118:57:48<349:04:16,  7.50s/it]
 25%|██▌       | 57360/225000 [118:57:53<310:08:35,  6.66s/it]
 25%|██▌       | 57361/225000 [118:57:57<273:12:12,  5.87s/it]
 25%|██▌       | 57362/225000 [118:58:05<315:03:29,  6.77s/it]
 25%|██▌       | 57363/225000 [118:58:12<312:24:28,  6.71s/it]
 25%|██▌       | 57364/225000 [118:58:17<284:42:55,  6.11s/it]
 25%|██▌       | 57365/225000 [118:58:21<262:10:38,  5.63s/it]
 25%|██▌       | 57366/225000 [118:58:32<338:44:26,  7.27s/it]
 25%|██▌       | 57367/225000 [118:58:37<300:45:12,  6.46s/it]
 25%|██▌       | 57368/225000 [118:58:47<357:59:55,  7.69s/it]
 25%|██▌       | 57369/225000 [118:58:52<314:11:08,  6.75s/it]
 25%|██▌       | 57370/225000 [118:58:59<322:54:30,  6.93s/it]
 25%|██▌       | 57371/225000 [118:59:07<337:53:21,  7.26s/it]
 25%|██▌       | 57372/225000 [118:59:13<312:15:00,  6.71s/it]
 25%|██▌       | 57373/225000 [118:59:22<351:39:19,  7.55s/it]
 25%|██▌       | 57374/225000 [118:59:26<295:01:30,  6.34s/it]
 26%|██▌       | 57375/225000 [118:59:35<334:52:47,  7.19s/it]
 26%|██▌       | 57376/225000 [118:59:39<288:56:29,  6.21s/it]
 26%|██▌       | 57377/225000 [118:59:46<301:58:48,  6.49s/it]
 26%|██▌       | 57378/225000 [118:59:50<272:50:54,  5.86s/it]
 26%|██▌       | 57379/225000 [118:59:56<264:54:03,  5.69s/it]
 26%|██▌       | 57380/225000 [119:00:02<266:45:54,  5.73s/it]
 26%|██▌       | 57381/225000 [119:00:07<264:17:06,  5.68s/it]
 26%|██▌       | 57382/225000 [119:00:17<318:41:44,  6.84s/it]
 26%|██▌       | 57383/225000 [119:00:26<355:35:22,  7.64s/it]
 26%|██▌       | 57384/225000 [119:00:31<322:48:03,  6.93s/it]
 26%|██▌       | 57385/225000 [119:00:37<298:10:51,  6.40s/it]
 26%|██▌       | 57386/225000 [119:00:42<278:57:26,  5.99s/it]
 26%|██▌       | 57387/225000 [119:00:47<272:47:32,  5.86s/it]
 26%|██▌       | 57388/225000 [119:00:53<275:21:00,  5.91s/it]
 26%|██▌       | 57389/225000 [119:00:59<268:08:58,  5.76s/it]
 26%|██▌       | 57390/225000 [119:01:08<318:42:45,  6.85s/it]
 26%|██▌       | 57391/225000 [119:01:18<357:17:50,  7.67s/it]
 26%|██▌       | 57392/225000 [119:01:22<307:45:31,  6.61s/it]
 26%|██▌       | 57393/225000 [119:01:28<302:39:58,  6.50s/it]
 26%|██▌       | 57394/225000 [119:01:39<358:29:54,  7.70s/it]
 26%|██▌       | 57395/225000 [119:01:45<340:16:23,  7.31s/it]
 26%|██▌       | 57396/225000 [119:01:49<288:48:21,  6.20s/it]
 26%|██▌       | 57397/225000 [119:01:57<322:27:55,  6.93s/it]
 26%|██▌       | 57398/225000 [119:02:01<285:37:42,  6.14s/it]
 26%|██▌       | 57399/225000 [119:02:10<317:02:18,  6.81s/it]
 26%|██▌       | 57400/225000 [119:02:16<314:12:41,  6.75s/it]
                                                              
{'loss': '0.7256', 'grad_norm': '0.8346', 'learning_rate': '4.425e-05', 'epoch': '0.2551', 'num_input_tokens_seen': 198397112, 'train_runtime': '4.285e+05', 'train_tokens_per_second': '463'}

 26%|██▌       | 57400/225000 [119:02:16<314:12:41,  6.75s/it]
 26%|██▌       | 57401/225000 [119:02:21<282:52:50,  6.08s/it]
 26%|██▌       | 57402/225000 [119:02:27<284:29:10,  6.11s/it]
 26%|██▌       | 57403/225000 [119:02:36<318:16:18,  6.84s/it]
 26%|██▌       | 57404/225000 [119:02:44<342:52:49,  7.37s/it]
 26%|██▌       | 57405/225000 [119:02:48<293:02:49,  6.29s/it]
 26%|██▌       | 57406/225000 [119:02:57<336:23:40,  7.23s/it]
 26%|██▌       | 57407/225000 [119:03:02<302:25:13,  6.50s/it]
 26%|██▌       | 57408/225000 [119:03:07<278:37:40,  5.99s/it]
 26%|██▌       | 57409/225000 [119:03:13<276:56:30,  5.95s/it]
 26%|██▌       | 57410/225000 [119:03:19<283:22:16,  6.09s/it]
 26%|██▌       | 57411/225000 [119:03:32<377:34:44,  8.11s/it]
 26%|██▌       | 57412/225000 [119:03:36<315:41:51,  6.78s/it]
 26%|██▌       | 57413/225000 [119:03:45<354:11:00,  7.61s/it]
 26%|██▌       | 57414/225000 [119:03:50<314:56:16,  6.77s/it]
 26%|██▌       | 57415/225000 [119:03:55<289:33:50,  6.22s/it]
 26%|██▌       | 57416/225000 [119:04:00<265:30:24,  5.70s/it]
 26%|██▌       | 57417/225000 [119:04:05<260:50:21,  5.60s/it]
 26%|██▌       | 57418/225000 [119:04:10<251:18:25,  5.40s/it]
 26%|██▌       | 57419/225000 [119:04:21<326:55:40,  7.02s/it]
 26%|██▌       | 57420/225000 [119:04:29<339:15:39,  7.29s/it]
 26%|██▌       | 57421/225000 [119:04:33<298:29:57,  6.41s/it]
 26%|██▌       | 57422/225000 [119:04:38<284:42:10,  6.12s/it]
 26%|██▌       | 57423/225000 [119:04:44<280:19:31,  6.02s/it]
 26%|██▌       | 57424/225000 [119:04:50<270:16:36,  5.81s/it]
 26%|██▌       | 57425/225000 [119:04:55<260:15:49,  5.59s/it]
 26%|██▌       | 57426/225000 [119:05:08<365:22:29,  7.85s/it]
 26%|██▌       | 57427/225000 [119:05:14<338:15:52,  7.27s/it]
 26%|██▌       | 57428/225000 [119:05:18<295:20:43,  6.34s/it]
 26%|██▌       | 57429/225000 [119:05:24<297:12:26,  6.39s/it]
 26%|██▌       | 57430/225000 [119:05:33<323:50:57,  6.96s/it]
 26%|██▌       | 57431/225000 [119:05:37<294:26:15,  6.33s/it]
 26%|██▌       | 57432/225000 [119:05:42<271:06:10,  5.82s/it]
 26%|██▌       | 57433/225000 [119:05:51<314:22:52,  6.75s/it]
 26%|██▌       | 57434/225000 [119:06:02<368:20:06,  7.91s/it]
 26%|██▌       | 57435/225000 [119:06:06<319:39:47,  6.87s/it]
 26%|██▌       | 57436/225000 [119:06:10<283:27:15,  6.09s/it]
 26%|██▌       | 57437/225000 [119:06:18<298:13:47,  6.41s/it]
 26%|██▌       | 57438/225000 [119:06:28<357:43:16,  7.69s/it]
 26%|██▌       | 57439/225000 [119:06:34<335:43:30,  7.21s/it]
 26%|██▌       | 57440/225000 [119:06:38<291:55:26,  6.27s/it]
 26%|██▌       | 57441/225000 [119:06:44<280:43:19,  6.03s/it]
 26%|██▌       | 57442/225000 [119:06:55<350:15:50,  7.53s/it]
 26%|██▌       | 57443/225000 [119:06:59<304:05:36,  6.53s/it]
 26%|██▌       | 57444/225000 [119:07:09<346:33:35,  7.45s/it]
 26%|██▌       | 57445/225000 [119:07:13<301:42:36,  6.48s/it]
 26%|██▌       | 57446/225000 [119:07:19<292:21:01,  6.28s/it]
 26%|██▌       | 57447/225000 [119:07:24<284:48:44,  6.12s/it]
 26%|██▌       | 57448/225000 [119:07:30<280:48:32,  6.03s/it]
 26%|██▌       | 57449/225000 [119:07:36<275:27:04,  5.92s/it]
 26%|██▌       | 57450/225000 [119:07:48<365:05:46,  7.84s/it]
                                                              
{'loss': '0.727', 'grad_norm': '0.6107', 'learning_rate': '4.423e-05', 'epoch': '0.2553', 'num_input_tokens_seen': 198547832, 'train_runtime': '4.289e+05', 'train_tokens_per_second': '463'}

 26%|██▌       | 57450/225000 [119:07:48<365:05:46,  7.84s/it]
 26%|██▌       | 57451/225000 [119:07:52<313:10:55,  6.73s/it]
 26%|██▌       | 57452/225000 [119:08:01<340:13:07,  7.31s/it]
 26%|██▌       | 57453/225000 [119:08:05<293:07:43,  6.30s/it]
 26%|██▌       | 57454/225000 [119:08:14<326:24:46,  7.01s/it]
 26%|██▌       | 57455/225000 [119:08:18<294:07:10,  6.32s/it]
 26%|██▌       | 57456/225000 [119:08:27<324:36:32,  6.97s/it]
 26%|██▌       | 57457/225000 [119:08:33<310:36:18,  6.67s/it]
 26%|██▌       | 57458/225000 [119:08:38<294:37:09,  6.33s/it]
 26%|██▌       | 57459/225000 [119:08:43<266:46:30,  5.73s/it]
 26%|██▌       | 57460/225000 [119:08:51<304:52:57,  6.55s/it]
 26%|██▌       | 57461/225000 [119:08:59<324:04:01,  6.96s/it]
 26%|██▌       | 57462/225000 [119:09:04<295:21:20,  6.35s/it]
 26%|██▌       | 57463/225000 [119:09:08<265:42:47,  5.71s/it]
 26%|██▌       | 57464/225000 [119:09:20<354:33:04,  7.62s/it]
 26%|██▌       | 57465/225000 [119:09:24<301:43:26,  6.48s/it]
 26%|██▌       | 57466/225000 [119:09:33<329:10:20,  7.07s/it]
 26%|██▌       | 57467/225000 [119:09:38<302:16:29,  6.50s/it]
 26%|██▌       | 57468/225000 [119:09:43<279:34:46,  6.01s/it]
 26%|██▌       | 57469/225000 [119:09:47<261:33:18,  5.62s/it]
 26%|██▌       | 57470/225000 [119:10:01<368:56:39,  7.93s/it]
 26%|██▌       | 57471/225000 [119:10:05<324:15:30,  6.97s/it]
 26%|██▌       | 57472/225000 [119:10:10<292:28:43,  6.29s/it]
 26%|██▌       | 57473/225000 [119:10:15<277:04:34,  5.95s/it]
 26%|██▌       | 57474/225000 [119:10:20<264:08:08,  5.68s/it]
 26%|██▌       | 57475/225000 [119:10:26<259:23:10,  5.57s/it]
 26%|██▌       | 57476/225000 [119:10:31<256:51:10,  5.52s/it]
 26%|██▌       | 57477/225000 [119:10:37<260:46:20,  5.60s/it]
 26%|██▌       | 57478/225000 [119:10:47<318:43:22,  6.85s/it]
 26%|██▌       | 57479/225000 [119:10:57<370:19:30,  7.96s/it]
 26%|██▌       | 57480/225000 [119:11:04<361:24:23,  7.77s/it]
 26%|██▌       | 57481/225000 [119:11:08<308:57:43,  6.64s/it]
 26%|██▌       | 57482/225000 [119:11:17<342:26:07,  7.36s/it]
 26%|██▌       | 57483/225000 [119:11:22<305:27:37,  6.56s/it]
 26%|██▌       | 57484/225000 [119:11:26<267:18:55,  5.74s/it]
 26%|██▌       | 57485/225000 [119:11:35<314:46:59,  6.76s/it]
 26%|██▌       | 57486/225000 [119:11:40<284:27:14,  6.11s/it]
 26%|██▌       | 57487/225000 [119:11:45<266:44:43,  5.73s/it]
 26%|██▌       | 57488/225000 [119:11:54<314:31:35,  6.76s/it]
 26%|██▌       | 57489/225000 [119:12:02<335:14:33,  7.20s/it]
 26%|██▌       | 57490/225000 [119:12:11<357:10:28,  7.68s/it]
 26%|██▌       | 57491/225000 [119:12:15<310:41:35,  6.68s/it]
 26%|██▌       | 57492/225000 [119:12:23<322:35:56,  6.93s/it]
 26%|██▌       | 57493/225000 [119:12:27<292:08:16,  6.28s/it]
 26%|██▌       | 57494/225000 [119:12:36<326:01:37,  7.01s/it]
 26%|██▌       | 57495/225000 [119:12:40<288:32:45,  6.20s/it]
 26%|██▌       | 57496/225000 [119:12:50<333:20:13,  7.16s/it]
 26%|██▌       | 57497/225000 [119:12:54<297:27:10,  6.39s/it]
 26%|██▌       | 57498/225000 [119:12:59<275:10:41,  5.91s/it]
 26%|██▌       | 57499/225000 [119:13:12<368:19:32,  7.92s/it]
 26%|██▌       | 57500/225000 [119:13:16<319:23:15,  6.86s/it]
                                                              
{'loss': '0.7411', 'grad_norm': '0.831', 'learning_rate': '4.422e-05', 'epoch': '0.2556', 'num_input_tokens_seen': 198697320, 'train_runtime': '4.292e+05', 'train_tokens_per_second': '463'}

 26%|██▌       | 57500/225000 [119:13:16<319:23:15,  6.86s/it]
 26%|██▌       | 57501/225000 [119:13:21<286:23:18,  6.16s/it]
 26%|██▌       | 57502/225000 [119:13:30<333:31:04,  7.17s/it]
 26%|██▌       | 57503/225000 [119:13:35<303:40:34,  6.53s/it]
 26%|██▌       | 57504/225000 [119:13:45<341:51:18,  7.35s/it]
 26%|██▌       | 57505/225000 [119:13:49<299:34:28,  6.44s/it]
 26%|██▌       | 57506/225000 [119:13:55<293:22:30,  6.31s/it]
 26%|██▌       | 57507/225000 [119:14:03<317:36:09,  6.83s/it]
 26%|██▌       | 57508/225000 [119:14:09<309:01:27,  6.64s/it]
 26%|██▌       | 57509/225000 [119:14:13<275:41:52,  5.93s/it]
 26%|██▌       | 57510/225000 [119:14:25<351:50:36,  7.56s/it]
 26%|██▌       | 57511/225000 [119:14:29<303:10:45,  6.52s/it]
 26%|██▌       | 57512/225000 [119:14:32<262:17:06,  5.64s/it]
 26%|██▌       | 57513/225000 [119:14:39<270:16:49,  5.81s/it]
 26%|██▌       | 57514/225000 [119:14:50<351:27:27,  7.55s/it]
 26%|██▌       | 57515/225000 [119:14:56<322:49:04,  6.94s/it]
 26%|██▌       | 57516/225000 [119:15:02<307:04:33,  6.60s/it]
 26%|██▌       | 57517/225000 [119:15:09<313:41:26,  6.74s/it]
 26%|██▌       | 57518/225000 [119:15:13<287:28:07,  6.18s/it]
 26%|██▌       | 57519/225000 [119:15:19<277:28:17,  5.96s/it]
 26%|██▌       | 57520/225000 [119:15:24<271:22:52,  5.83s/it]
 26%|██▌       | 57521/225000 [119:15:30<263:55:34,  5.67s/it]
 26%|██▌       | 57522/225000 [119:15:35<258:50:03,  5.56s/it]
 26%|██▌       | 57523/225000 [119:15:41<257:38:37,  5.54s/it]
 26%|██▌       | 57524/225000 [119:15:46<252:40:52,  5.43s/it]
 26%|██▌       | 57525/225000 [119:15:59<356:29:19,  7.66s/it]
 26%|██▌       | 57526/225000 [119:16:06<353:56:12,  7.61s/it]
 26%|██▌       | 57527/225000 [119:16:13<350:55:03,  7.54s/it]
 26%|██▌       | 57528/225000 [119:16:17<300:51:02,  6.47s/it]
 26%|██▌       | 57529/225000 [119:16:22<270:08:38,  5.81s/it]
 26%|██▌       | 57530/225000 [119:16:34<354:32:42,  7.62s/it]
 26%|██▌       | 57531/225000 [119:16:37<301:49:26,  6.49s/it]
 26%|██▌       | 57532/225000 [119:16:46<332:54:53,  7.16s/it]
 26%|██▌       | 57533/225000 [119:16:50<286:58:15,  6.17s/it]
 26%|██▌       | 57534/225000 [119:16:54<258:58:16,  5.57s/it]
 26%|██▌       | 57535/225000 [119:17:06<345:58:28,  7.44s/it]
 26%|██▌       | 57536/225000 [119:17:10<296:41:15,  6.38s/it]
 26%|██▌       | 57537/225000 [119:17:15<276:15:35,  5.94s/it]
 26%|██▌       | 57538/225000 [119:17:23<305:22:02,  6.56s/it]
 26%|██▌       | 57539/225000 [119:17:31<325:19:42,  6.99s/it]
 26%|██▌       | 57540/225000 [119:17:39<339:04:19,  7.29s/it]
 26%|██▌       | 57541/225000 [119:17:43<291:48:30,  6.27s/it]
 26%|██▌       | 57542/225000 [119:17:48<278:51:49,  5.99s/it]
 26%|██▌       | 57543/225000 [119:17:54<278:32:11,  5.99s/it]
 26%|██▌       | 57544/225000 [119:18:00<280:32:50,  6.03s/it]
 26%|██▌       | 57545/225000 [119:18:06<277:23:05,  5.96s/it]
 26%|██▌       | 57546/225000 [119:18:12<282:14:44,  6.07s/it]
 26%|██▌       | 57547/225000 [119:18:18<272:18:51,  5.85s/it]
 26%|██▌       | 57548/225000 [119:18:27<317:25:33,  6.82s/it]
 26%|██▌       | 57549/225000 [119:18:38<380:33:15,  8.18s/it]
 26%|██▌       | 57550/225000 [119:18:42<320:00:31,  6.88s/it]
                                                              
{'loss': '0.744', 'grad_norm': '0.8729', 'learning_rate': '4.421e-05', 'epoch': '0.2558', 'num_input_tokens_seen': 198847120, 'train_runtime': '4.295e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57550/225000 [119:18:42<320:00:31,  6.88s/it]
 26%|██▌       | 57551/225000 [119:18:47<301:07:45,  6.47s/it]
 26%|██▌       | 57552/225000 [119:18:57<341:18:25,  7.34s/it]
 26%|██▌       | 57553/225000 [119:19:04<338:21:26,  7.27s/it]
 26%|██▌       | 57554/225000 [119:19:10<324:20:54,  6.97s/it]
 26%|██▌       | 57555/225000 [119:19:15<287:35:27,  6.18s/it]
 26%|██▌       | 57556/225000 [119:19:23<315:51:04,  6.79s/it]
 26%|██▌       | 57557/225000 [119:19:28<292:10:13,  6.28s/it]
 26%|██▌       | 57558/225000 [119:19:36<325:01:25,  6.99s/it]
 26%|██▌       | 57559/225000 [119:19:41<284:42:58,  6.12s/it]
 26%|██▌       | 57560/225000 [119:19:49<315:26:16,  6.78s/it]
 26%|██▌       | 57561/225000 [119:19:55<303:42:11,  6.53s/it]
 26%|██▌       | 57562/225000 [119:20:03<325:54:08,  7.01s/it]
 26%|██▌       | 57563/225000 [119:20:07<287:46:43,  6.19s/it]
 26%|██▌       | 57564/225000 [119:20:16<325:05:27,  6.99s/it]
 26%|██▌       | 57565/225000 [119:20:20<283:35:04,  6.10s/it]
 26%|██▌       | 57566/225000 [119:20:28<308:59:36,  6.64s/it]
 26%|██▌       | 57567/225000 [119:20:33<280:14:32,  6.03s/it]
 26%|██▌       | 57568/225000 [119:20:37<259:04:50,  5.57s/it]
 26%|██▌       | 57569/225000 [119:20:47<321:36:09,  6.91s/it]
 26%|██▌       | 57570/225000 [119:20:52<298:11:30,  6.41s/it]
 26%|██▌       | 57571/225000 [119:20:56<264:31:26,  5.69s/it]
 26%|██▌       | 57572/225000 [119:21:07<333:33:46,  7.17s/it]
 26%|██▌       | 57573/225000 [119:21:12<306:10:14,  6.58s/it]
 26%|██▌       | 57574/225000 [119:21:18<291:44:27,  6.27s/it]
 26%|██▌       | 57575/225000 [119:21:23<278:33:14,  5.99s/it]
 26%|██▌       | 57576/225000 [119:21:29<271:55:50,  5.85s/it]
 26%|██▌       | 57577/225000 [119:21:34<261:24:54,  5.62s/it]
 26%|██▌       | 57578/225000 [119:21:39<256:42:29,  5.52s/it]
 26%|██▌       | 57579/225000 [119:21:45<269:44:24,  5.80s/it]
 26%|██▌       | 57580/225000 [119:21:59<384:15:41,  8.26s/it]
 26%|██▌       | 57581/225000 [119:22:04<335:16:54,  7.21s/it]
 26%|██▌       | 57582/225000 [119:22:09<299:28:29,  6.44s/it]
 26%|██▌       | 57583/225000 [119:22:13<271:51:46,  5.85s/it]
 26%|██▌       | 57584/225000 [119:22:22<311:04:16,  6.69s/it]
 26%|██▌       | 57585/225000 [119:22:32<357:53:14,  7.70s/it]
 26%|██▌       | 57586/225000 [119:22:38<336:15:07,  7.23s/it]
 26%|██▌       | 57587/225000 [119:22:46<342:58:03,  7.38s/it]
 26%|██▌       | 57588/225000 [119:22:50<300:12:41,  6.46s/it]
 26%|██▌       | 57589/225000 [119:22:56<288:24:52,  6.20s/it]
 26%|██▌       | 57590/225000 [119:23:01<280:42:59,  6.04s/it]
 26%|██▌       | 57591/225000 [119:23:07<270:37:56,  5.82s/it]
 26%|██▌       | 57592/225000 [119:23:12<259:42:24,  5.58s/it]
 26%|██▌       | 57593/225000 [119:23:17<259:32:36,  5.58s/it]
 26%|██▌       | 57594/225000 [119:23:23<254:36:20,  5.48s/it]
 26%|██▌       | 57595/225000 [119:23:30<283:27:22,  6.10s/it]
 26%|██▌       | 57596/225000 [119:23:43<376:14:58,  8.09s/it]
 26%|██▌       | 57597/225000 [119:23:48<337:35:02,  7.26s/it]
 26%|██▌       | 57598/225000 [119:23:54<319:30:57,  6.87s/it]
 26%|██▌       | 57599/225000 [119:24:02<327:28:15,  7.04s/it]
 26%|██▌       | 57600/225000 [119:24:08<319:54:41,  6.88s/it]
                                                              
{'loss': '0.712', 'grad_norm': '0.7046', 'learning_rate': '4.42e-05', 'epoch': '0.256', 'num_input_tokens_seen': 198995072, 'train_runtime': '4.298e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57600/225000 [119:24:08<319:54:41,  6.88s/it]
 26%|██▌       | 57601/225000 [119:24:13<293:22:16,  6.31s/it]
 26%|██▌       | 57602/225000 [119:24:22<332:19:44,  7.15s/it]
 26%|██▌       | 57603/225000 [119:24:29<328:27:46,  7.06s/it]
 26%|██▌       | 57604/225000 [119:24:33<289:40:32,  6.23s/it]
 26%|██▌       | 57605/225000 [119:24:42<329:17:07,  7.08s/it]
 26%|██▌       | 57606/225000 [119:24:49<320:37:08,  6.90s/it]
 26%|██▌       | 57607/225000 [119:24:56<322:10:32,  6.93s/it]
 26%|██▌       | 57608/225000 [119:25:01<302:35:11,  6.51s/it]
 26%|██▌       | 57609/225000 [119:25:10<335:19:48,  7.21s/it]
 26%|██▌       | 57610/225000 [119:25:15<301:39:59,  6.49s/it]
 26%|██▌       | 57611/225000 [119:25:19<271:12:06,  5.83s/it]
 26%|██▌       | 57612/225000 [119:25:24<251:30:26,  5.41s/it]
 26%|██▌       | 57613/225000 [119:25:37<360:22:12,  7.75s/it]
 26%|██▌       | 57614/225000 [119:25:42<325:17:21,  7.00s/it]
 26%|██▌       | 57615/225000 [119:25:47<293:43:56,  6.32s/it]
 26%|██▌       | 57616/225000 [119:25:52<271:43:43,  5.84s/it]
 26%|██▌       | 57617/225000 [119:25:56<253:42:10,  5.46s/it]
 26%|██▌       | 57618/225000 [119:26:01<245:24:41,  5.28s/it]
 26%|██▌       | 57619/225000 [119:26:14<348:22:27,  7.49s/it]
 26%|██▌       | 57620/225000 [119:26:21<344:08:12,  7.40s/it]
 26%|██▌       | 57621/225000 [119:26:27<324:40:50,  6.98s/it]
 26%|██▌       | 57622/225000 [119:26:32<302:04:10,  6.50s/it]
 26%|██▌       | 57623/225000 [119:26:38<289:32:11,  6.23s/it]
 26%|██▌       | 57624/225000 [119:26:43<280:03:05,  6.02s/it]
 26%|██▌       | 57625/225000 [119:26:54<344:21:33,  7.41s/it]
 26%|██▌       | 57626/225000 [119:27:03<360:41:18,  7.76s/it]
 26%|██▌       | 57627/225000 [119:27:09<344:27:20,  7.41s/it]
 26%|██▌       | 57628/225000 [119:27:15<320:59:45,  6.90s/it]
 26%|██▌       | 57629/225000 [119:27:20<293:24:37,  6.31s/it]
 26%|██▌       | 57630/225000 [119:27:29<332:14:26,  7.15s/it]
 26%|██▌       | 57631/225000 [119:27:33<285:42:43,  6.15s/it]
 26%|██▌       | 57632/225000 [119:27:44<351:42:28,  7.57s/it]
 26%|██▌       | 57633/225000 [119:27:48<300:21:06,  6.46s/it]
 26%|██▌       | 57634/225000 [119:27:53<283:41:48,  6.10s/it]
 26%|██▌       | 57635/225000 [119:27:59<281:51:46,  6.06s/it]
 26%|██▌       | 57636/225000 [119:28:04<267:34:30,  5.76s/it]
 26%|██▌       | 57637/225000 [119:28:10<270:54:48,  5.83s/it]
 26%|██▌       | 57638/225000 [119:28:15<263:00:17,  5.66s/it]
 26%|██▌       | 57639/225000 [119:28:21<260:47:23,  5.61s/it]
 26%|██▌       | 57640/225000 [119:28:26<261:07:09,  5.62s/it]
 26%|██▌       | 57641/225000 [119:28:37<329:47:48,  7.09s/it]
 26%|██▌       | 57642/225000 [119:28:46<355:11:23,  7.64s/it]
 26%|██▌       | 57643/225000 [119:28:53<347:38:41,  7.48s/it]
 26%|██▌       | 57644/225000 [119:28:59<324:45:20,  6.99s/it]
 26%|██▌       | 57645/225000 [119:29:06<331:55:02,  7.14s/it]
 26%|██▌       | 57646/225000 [119:29:13<326:19:44,  7.02s/it]
 26%|██▌       | 57647/225000 [119:29:18<297:00:22,  6.39s/it]
 26%|██▌       | 57648/225000 [119:29:23<280:14:03,  6.03s/it]
 26%|██▌       | 57649/225000 [119:29:28<267:02:14,  5.74s/it]
 26%|██▌       | 57650/225000 [119:29:34<274:37:13,  5.91s/it]
                                                              
{'loss': '0.7257', 'grad_norm': '0.8217', 'learning_rate': '4.419e-05', 'epoch': '0.2562', 'num_input_tokens_seen': 199146080, 'train_runtime': '4.302e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57650/225000 [119:29:34<274:37:13,  5.91s/it]
 26%|██▌       | 57651/225000 [119:29:47<374:20:35,  8.05s/it]
 26%|██▌       | 57652/225000 [119:29:51<314:08:08,  6.76s/it]
 26%|██▌       | 57653/225000 [119:30:00<336:06:18,  7.23s/it]
 26%|██▌       | 57654/225000 [119:30:04<299:41:22,  6.45s/it]
 26%|██▌       | 57655/225000 [119:30:09<278:51:31,  6.00s/it]
 26%|██▌       | 57656/225000 [119:30:14<262:43:28,  5.65s/it]
 26%|██▌       | 57657/225000 [119:30:21<276:54:48,  5.96s/it]
 26%|██▌       | 57658/225000 [119:30:33<366:17:44,  7.88s/it]
 26%|██▌       | 57659/225000 [119:30:37<311:18:43,  6.70s/it]
 26%|██▌       | 57660/225000 [119:30:42<291:42:11,  6.28s/it]
 26%|██▌       | 57661/225000 [119:30:48<282:38:12,  6.08s/it]
 26%|██▌       | 57662/225000 [119:30:53<268:50:02,  5.78s/it]
 26%|██▌       | 57663/225000 [119:30:58<264:14:15,  5.68s/it]
 26%|██▌       | 57664/225000 [119:31:10<349:33:57,  7.52s/it]
 26%|██▌       | 57665/225000 [119:31:18<347:11:36,  7.47s/it]
 26%|██▌       | 57666/225000 [119:31:26<356:50:09,  7.68s/it]
 26%|██▌       | 57667/225000 [119:31:30<313:47:09,  6.75s/it]
 26%|██▌       | 57668/225000 [119:31:39<346:32:13,  7.46s/it]
 26%|██▌       | 57669/225000 [119:31:44<301:37:09,  6.49s/it]
 26%|██▌       | 57670/225000 [119:31:52<334:30:12,  7.20s/it]
 26%|██▌       | 57671/225000 [119:31:56<286:19:56,  6.16s/it]
 26%|██▌       | 57672/225000 [119:32:01<270:57:52,  5.83s/it]
 26%|██▌       | 57673/225000 [119:32:06<259:52:28,  5.59s/it]
 26%|██▌       | 57674/225000 [119:32:11<248:39:11,  5.35s/it]
 26%|██▌       | 57675/225000 [119:32:16<248:38:07,  5.35s/it]
 26%|██▌       | 57676/225000 [119:32:28<339:57:22,  7.31s/it]
 26%|██▌       | 57677/225000 [119:32:32<292:16:35,  6.29s/it]
 26%|██▌       | 57678/225000 [119:32:43<360:40:33,  7.76s/it]
 26%|██▌       | 57679/225000 [119:32:50<342:30:00,  7.37s/it]
 26%|██▌       | 57680/225000 [119:32:54<299:44:29,  6.45s/it]
 26%|██▌       | 57681/225000 [119:33:04<341:05:09,  7.34s/it]
 26%|██▌       | 57682/225000 [119:33:11<348:20:36,  7.49s/it]
 26%|██▌       | 57683/225000 [119:33:16<304:25:46,  6.55s/it]
 26%|██▌       | 57684/225000 [119:33:23<320:37:17,  6.90s/it]
 26%|██▌       | 57685/225000 [119:33:29<296:15:35,  6.37s/it]
 26%|██▌       | 57686/225000 [119:33:33<270:44:31,  5.83s/it]
 26%|██▌       | 57687/225000 [119:33:38<256:25:10,  5.52s/it]
 26%|██▌       | 57688/225000 [119:33:43<254:30:32,  5.48s/it]
 26%|██▌       | 57689/225000 [119:33:51<291:21:20,  6.27s/it]
 26%|██▌       | 57690/225000 [119:34:02<351:22:01,  7.56s/it]
 26%|██▌       | 57691/225000 [119:34:06<299:57:06,  6.45s/it]
 26%|██▌       | 57692/225000 [119:34:12<294:03:37,  6.33s/it]
 26%|██▌       | 57693/225000 [119:34:19<300:03:07,  6.46s/it]
 26%|██▌       | 57694/225000 [119:34:23<273:48:33,  5.89s/it]
 26%|██▌       | 57695/225000 [119:34:31<303:11:53,  6.52s/it]
 26%|██▌       | 57696/225000 [119:34:41<352:03:26,  7.58s/it]
 26%|██▌       | 57697/225000 [119:34:45<297:35:31,  6.40s/it]
 26%|██▌       | 57698/225000 [119:34:50<276:26:44,  5.95s/it]
 26%|██▌       | 57699/225000 [119:34:55<267:07:10,  5.75s/it]
 26%|██▌       | 57700/225000 [119:35:00<258:22:19,  5.56s/it]
                                                              
{'loss': '0.7426', 'grad_norm': '0.7913', 'learning_rate': '4.418e-05', 'epoch': '0.2564', 'num_input_tokens_seen': 199296688, 'train_runtime': '4.305e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57700/225000 [119:35:00<258:22:19,  5.56s/it]
 26%|██▌       | 57701/225000 [119:35:12<345:27:29,  7.43s/it]
 26%|██▌       | 57702/225000 [119:35:16<301:48:03,  6.49s/it]
 26%|██▌       | 57703/225000 [119:35:20<266:05:23,  5.73s/it]
 26%|██▌       | 57704/225000 [119:35:33<356:42:03,  7.68s/it]
 26%|██▌       | 57705/225000 [119:35:37<317:56:47,  6.84s/it]
 26%|██▌       | 57706/225000 [119:35:42<281:12:01,  6.05s/it]
 26%|██▌       | 57707/225000 [119:35:46<260:39:28,  5.61s/it]
 26%|██▌       | 57708/225000 [119:35:52<263:17:34,  5.67s/it]
 26%|██▌       | 57709/225000 [119:36:05<367:48:02,  7.91s/it]
 26%|██▌       | 57710/225000 [119:36:10<317:59:17,  6.84s/it]
 26%|██▌       | 57711/225000 [119:36:17<329:51:48,  7.10s/it]
 26%|██▌       | 57712/225000 [119:36:22<298:36:28,  6.43s/it]
 26%|██▌       | 57713/225000 [119:36:27<277:28:24,  5.97s/it]
 26%|██▌       | 57714/225000 [119:36:37<337:02:19,  7.25s/it]
 26%|██▌       | 57715/225000 [119:36:43<314:33:48,  6.77s/it]
 26%|██▌       | 57716/225000 [119:36:47<283:45:54,  6.11s/it]
 26%|██▌       | 57717/225000 [119:36:52<264:00:11,  5.68s/it]
 26%|██▌       | 57718/225000 [119:36:57<250:16:01,  5.39s/it]
 26%|██▌       | 57719/225000 [119:37:08<330:38:16,  7.12s/it]
 26%|██▌       | 57720/225000 [119:37:15<327:33:04,  7.05s/it]
 26%|██▌       | 57721/225000 [119:37:20<298:39:22,  6.43s/it]
 26%|██▌       | 57722/225000 [119:37:25<276:01:52,  5.94s/it]
 26%|██▌       | 57723/225000 [119:37:31<285:27:51,  6.14s/it]
 26%|██▌       | 57724/225000 [119:37:42<348:49:00,  7.51s/it]
 26%|██▌       | 57725/225000 [119:37:46<298:08:32,  6.42s/it]
 26%|██▌       | 57726/225000 [119:37:53<304:44:16,  6.56s/it]
 26%|██▌       | 57727/225000 [119:38:01<326:40:48,  7.03s/it]
 26%|██▌       | 57728/225000 [119:38:06<306:09:41,  6.59s/it]
 26%|██▌       | 57729/225000 [119:38:11<280:21:41,  6.03s/it]
 26%|██▌       | 57730/225000 [119:38:16<261:08:29,  5.62s/it]
 26%|██▌       | 57731/225000 [119:38:30<375:56:41,  8.09s/it]
 26%|██▌       | 57732/225000 [119:38:34<319:49:43,  6.88s/it]
 26%|██▌       | 57733/225000 [119:38:41<331:27:35,  7.13s/it]
 26%|██▌       | 57734/225000 [119:38:47<305:20:01,  6.57s/it]
 26%|██▌       | 57735/225000 [119:38:52<292:24:39,  6.29s/it]
 26%|██▌       | 57736/225000 [119:38:57<273:56:41,  5.90s/it]
 26%|██▌       | 57737/225000 [119:39:03<264:11:33,  5.69s/it]
 26%|██▌       | 57738/225000 [119:39:14<344:06:36,  7.41s/it]
 26%|██▌       | 57739/225000 [119:39:23<365:16:52,  7.86s/it]
 26%|██▌       | 57740/225000 [119:39:30<353:32:17,  7.61s/it]
 26%|██▌       | 57741/225000 [119:39:34<306:05:15,  6.59s/it]
 26%|██▌       | 57742/225000 [119:39:38<274:41:40,  5.91s/it]
 26%|██▌       | 57743/225000 [119:39:50<360:27:54,  7.76s/it]
 26%|██▌       | 57744/225000 [119:39:54<295:02:11,  6.35s/it]
 26%|██▌       | 57745/225000 [119:40:02<318:01:05,  6.85s/it]
 26%|██▌       | 57746/225000 [119:40:07<293:19:05,  6.31s/it]
 26%|██▌       | 57747/225000 [119:40:12<276:37:44,  5.95s/it]
 26%|██▌       | 57748/225000 [119:40:17<270:48:39,  5.83s/it]
 26%|██▌       | 57749/225000 [119:40:23<265:26:47,  5.71s/it]
 26%|██▌       | 57750/225000 [119:40:29<268:16:04,  5.77s/it]
                                                              
{'loss': '0.7322', 'grad_norm': '0.8021', 'learning_rate': '4.416e-05', 'epoch': '0.2567', 'num_input_tokens_seen': 199444472, 'train_runtime': '4.308e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57750/225000 [119:40:29<268:16:04,  5.77s/it]
 26%|██▌       | 57751/225000 [119:40:34<259:39:31,  5.59s/it]
 26%|██▌       | 57752/225000 [119:40:45<344:03:43,  7.41s/it]
 26%|██▌       | 57753/225000 [119:40:52<337:45:19,  7.27s/it]
 26%|██▌       | 57754/225000 [119:41:00<344:21:30,  7.41s/it]
 26%|██▌       | 57755/225000 [119:41:07<342:01:46,  7.36s/it]
 26%|██▌       | 57756/225000 [119:41:12<309:24:06,  6.66s/it]
 26%|██▌       | 57757/225000 [119:41:18<287:49:29,  6.20s/it]
 26%|██▌       | 57758/225000 [119:41:28<347:28:05,  7.48s/it]
 26%|██▌       | 57759/225000 [119:41:32<301:21:26,  6.49s/it]
 26%|██▌       | 57760/225000 [119:41:36<270:39:08,  5.83s/it]
 26%|██▌       | 57761/225000 [119:41:46<327:39:10,  7.05s/it]
 26%|██▌       | 57762/225000 [119:41:55<354:21:30,  7.63s/it]
 26%|██▌       | 57763/225000 [119:42:00<309:39:23,  6.67s/it]
 26%|██▌       | 57764/225000 [119:42:04<280:01:54,  6.03s/it]
 26%|██▌       | 57765/225000 [119:42:15<351:29:00,  7.57s/it]
 26%|██▌       | 57766/225000 [119:42:19<301:36:38,  6.49s/it]
 26%|██▌       | 57767/225000 [119:42:27<317:25:16,  6.83s/it]
 26%|██▌       | 57768/225000 [119:42:34<320:22:45,  6.90s/it]
 26%|██▌       | 57769/225000 [119:42:39<290:52:16,  6.26s/it]
 26%|██▌       | 57770/225000 [119:42:43<267:44:01,  5.76s/it]
 26%|██▌       | 57771/225000 [119:42:53<322:53:56,  6.95s/it]
 26%|██▌       | 57772/225000 [119:43:00<323:23:21,  6.96s/it]
 26%|██▌       | 57773/225000 [119:43:04<285:24:40,  6.14s/it]
 26%|██▌       | 57774/225000 [119:43:10<276:24:28,  5.95s/it]
 26%|██▌       | 57775/225000 [119:43:15<260:35:01,  5.61s/it]
 26%|██▌       | 57776/225000 [119:43:28<366:21:18,  7.89s/it]
 26%|██▌       | 57777/225000 [119:43:32<319:03:23,  6.87s/it]
 26%|██▌       | 57778/225000 [119:43:39<314:41:12,  6.77s/it]
 26%|██▌       | 57779/225000 [119:43:47<328:41:42,  7.08s/it]
 26%|██▌       | 57780/225000 [119:43:55<348:32:23,  7.50s/it]
 26%|██▌       | 57781/225000 [119:43:59<293:21:42,  6.32s/it]
 26%|██▌       | 57782/225000 [119:44:03<265:21:42,  5.71s/it]
 26%|██▌       | 57783/225000 [119:44:12<313:24:23,  6.75s/it]
 26%|██▌       | 57784/225000 [119:44:18<297:23:40,  6.40s/it]
 26%|██▌       | 57785/225000 [119:44:22<271:45:50,  5.85s/it]
 26%|██▌       | 57786/225000 [119:44:28<273:11:37,  5.88s/it]
 26%|██▌       | 57787/225000 [119:44:37<313:46:50,  6.76s/it]
 26%|██▌       | 57788/225000 [119:44:42<285:43:10,  6.15s/it]
 26%|██▌       | 57789/225000 [119:44:48<289:48:25,  6.24s/it]
 26%|██▌       | 57790/225000 [119:44:53<273:37:02,  5.89s/it]
 26%|██▌       | 57791/225000 [119:44:58<257:08:55,  5.54s/it]
 26%|██▌       | 57792/225000 [119:45:09<334:10:38,  7.19s/it]
 26%|██▌       | 57793/225000 [119:45:14<298:12:52,  6.42s/it]
 26%|██▌       | 57794/225000 [119:45:19<279:43:34,  6.02s/it]
 26%|██▌       | 57795/225000 [119:45:24<264:21:13,  5.69s/it]
 26%|██▌       | 57796/225000 [119:45:30<276:46:46,  5.96s/it]
 26%|██▌       | 57797/225000 [119:45:44<383:08:30,  8.25s/it]
 26%|██▌       | 57798/225000 [119:45:48<329:04:13,  7.09s/it]
 26%|██▌       | 57799/225000 [119:45:57<351:56:18,  7.58s/it]
 26%|██▌       | 57800/225000 [119:46:02<309:53:04,  6.67s/it]
                                                              
{'loss': '0.7366', 'grad_norm': '0.7527', 'learning_rate': '4.415e-05', 'epoch': '0.2569', 'num_input_tokens_seen': 199595000, 'train_runtime': '4.312e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57800/225000 [119:46:02<309:53:04,  6.67s/it]
 26%|██▌       | 57801/225000 [119:46:06<277:38:50,  5.98s/it]
 26%|██▌       | 57802/225000 [119:46:10<256:16:19,  5.52s/it]
 26%|██▌       | 57803/225000 [119:46:23<357:53:37,  7.71s/it]
 26%|██▌       | 57804/225000 [119:46:27<307:01:07,  6.61s/it]
 26%|██▌       | 57805/225000 [119:46:33<294:32:55,  6.34s/it]
 26%|██▌       | 57806/225000 [119:46:43<345:28:44,  7.44s/it]
 26%|██▌       | 57807/225000 [119:46:50<335:05:30,  7.22s/it]
 26%|██▌       | 57808/225000 [119:46:53<284:37:40,  6.13s/it]
 26%|██▌       | 57809/225000 [119:47:04<348:17:00,  7.50s/it]
 26%|██▌       | 57810/225000 [119:47:08<305:34:16,  6.58s/it]
 26%|██▌       | 57811/225000 [119:47:16<318:30:19,  6.86s/it]
 26%|██▌       | 57812/225000 [119:47:20<279:06:17,  6.01s/it]
 26%|██▌       | 57813/225000 [119:47:26<279:07:34,  6.01s/it]
 26%|██▌       | 57814/225000 [119:47:35<315:40:34,  6.80s/it]
 26%|██▌       | 57815/225000 [119:47:41<312:13:35,  6.72s/it]
 26%|██▌       | 57816/225000 [119:47:45<274:14:02,  5.91s/it]
 26%|██▌       | 57817/225000 [119:47:51<273:20:16,  5.89s/it]
 26%|██▌       | 57818/225000 [119:47:56<261:32:59,  5.63s/it]
 26%|██▌       | 57819/225000 [119:48:01<255:39:54,  5.51s/it]
 26%|██▌       | 57820/225000 [119:48:06<247:38:13,  5.33s/it]
 26%|██▌       | 57821/225000 [119:48:12<252:43:51,  5.44s/it]
 26%|██▌       | 57822/225000 [119:48:21<309:39:01,  6.67s/it]
 26%|██▌       | 57823/225000 [119:48:32<363:23:16,  7.83s/it]
 26%|██▌       | 57824/225000 [119:48:36<308:53:45,  6.65s/it]
 26%|██▌       | 57825/225000 [119:48:43<319:06:19,  6.87s/it]
 26%|██▌       | 57826/225000 [119:48:51<327:06:29,  7.04s/it]
 26%|██▌       | 57827/225000 [119:48:55<287:23:55,  6.19s/it]
 26%|██▌       | 57828/225000 [119:49:08<384:44:32,  8.29s/it]
 26%|██▌       | 57829/225000 [119:49:13<332:06:04,  7.15s/it]
 26%|██▌       | 57830/225000 [119:49:19<315:32:27,  6.80s/it]
 26%|██▌       | 57831/225000 [119:49:24<292:30:17,  6.30s/it]
 26%|██▌       | 57832/225000 [119:49:28<271:10:35,  5.84s/it]
 26%|██▌       | 57833/225000 [119:49:34<259:59:03,  5.60s/it]
 26%|██▌       | 57834/225000 [119:49:39<254:07:23,  5.47s/it]
 26%|██▌       | 57835/225000 [119:49:49<325:26:31,  7.01s/it]
 26%|██▌       | 57836/225000 [119:49:58<345:50:50,  7.45s/it]
 26%|██▌       | 57837/225000 [119:50:02<298:23:08,  6.43s/it]
 26%|██▌       | 57838/225000 [119:50:10<321:30:32,  6.92s/it]
 26%|██▌       | 57839/225000 [119:50:17<323:34:31,  6.97s/it]
 26%|██▌       | 57840/225000 [119:50:25<339:09:35,  7.30s/it]
 26%|██▌       | 57841/225000 [119:50:29<298:37:18,  6.43s/it]
 26%|██▌       | 57842/225000 [119:50:37<310:21:22,  6.68s/it]
 26%|██▌       | 57843/225000 [119:50:43<308:33:51,  6.65s/it]
 26%|██▌       | 57844/225000 [119:50:49<298:19:24,  6.42s/it]
 26%|██▌       | 57845/225000 [119:50:58<331:08:41,  7.13s/it]
 26%|██▌       | 57846/225000 [119:51:03<302:35:17,  6.52s/it]
 26%|██▌       | 57847/225000 [119:51:08<278:26:36,  6.00s/it]
 26%|██▌       | 57848/225000 [119:51:12<259:21:58,  5.59s/it]
 26%|██▌       | 57849/225000 [119:51:18<257:25:35,  5.54s/it]
 26%|██▌       | 57850/225000 [119:51:25<279:20:23,  6.02s/it]
                                                              
{'loss': '0.742', 'grad_norm': '0.7037', 'learning_rate': '4.414e-05', 'epoch': '0.2571', 'num_input_tokens_seen': 199743112, 'train_runtime': '4.315e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57850/225000 [119:51:25<279:20:23,  6.02s/it]
 26%|██▌       | 57851/225000 [119:51:39<385:25:22,  8.30s/it]
 26%|██▌       | 57852/225000 [119:51:43<334:44:44,  7.21s/it]
 26%|██▌       | 57853/225000 [119:51:50<324:47:11,  7.00s/it]
 26%|██▌       | 57854/225000 [119:51:54<291:04:52,  6.27s/it]
 26%|██▌       | 57855/225000 [119:51:59<266:14:43,  5.73s/it]
 26%|██▌       | 57856/225000 [119:52:10<340:14:11,  7.33s/it]
 26%|██▌       | 57857/225000 [119:52:16<319:17:26,  6.88s/it]
 26%|██▌       | 57858/225000 [119:52:22<312:44:19,  6.74s/it]
 26%|██▌       | 57859/225000 [119:52:29<319:38:07,  6.88s/it]
 26%|██▌       | 57860/225000 [119:52:35<300:18:38,  6.47s/it]
 26%|██▌       | 57861/225000 [119:52:40<286:01:31,  6.16s/it]
 26%|██▌       | 57862/225000 [119:52:46<276:27:18,  5.95s/it]
 26%|██▌       | 57863/225000 [119:52:53<299:36:31,  6.45s/it]
 26%|██▌       | 57864/225000 [119:53:02<328:47:52,  7.08s/it]
 26%|██▌       | 57865/225000 [119:53:07<296:50:05,  6.39s/it]
 26%|██▌       | 57866/225000 [119:53:15<329:13:38,  7.09s/it]
 26%|██▌       | 57867/225000 [119:53:22<325:14:48,  7.01s/it]
 26%|██▌       | 57868/225000 [119:53:29<318:32:31,  6.86s/it]
 26%|██▌       | 57869/225000 [119:53:33<286:47:24,  6.18s/it]
 26%|██▌       | 57870/225000 [119:53:43<331:03:48,  7.13s/it]
 26%|██▌       | 57871/225000 [119:53:48<309:53:33,  6.68s/it]
 26%|██▌       | 57872/225000 [119:53:57<332:37:14,  7.16s/it]
 26%|██▌       | 57873/225000 [119:54:04<330:09:34,  7.11s/it]
 26%|██▌       | 57874/225000 [119:54:11<328:02:41,  7.07s/it]
 26%|██▌       | 57875/225000 [119:54:15<297:36:13,  6.41s/it]
 26%|██▌       | 57876/225000 [119:54:20<267:51:36,  5.77s/it]
 26%|██▌       | 57877/225000 [119:54:27<289:46:41,  6.24s/it]
 26%|██▌       | 57878/225000 [119:54:37<338:32:08,  7.29s/it]
 26%|██▌       | 57879/225000 [119:54:41<295:58:37,  6.38s/it]
 26%|██▌       | 57880/225000 [119:54:49<318:33:18,  6.86s/it]
 26%|██▌       | 57881/225000 [119:54:54<286:28:58,  6.17s/it]
 26%|██▌       | 57882/225000 [119:54:58<261:36:02,  5.64s/it]
 26%|██▌       | 57883/225000 [119:55:07<308:53:16,  6.65s/it]
 26%|██▌       | 57884/225000 [119:55:13<296:11:16,  6.38s/it]
 26%|██▌       | 57885/225000 [119:55:22<335:47:40,  7.23s/it]
 26%|██▌       | 57886/225000 [119:55:25<278:16:10,  5.99s/it]
 26%|██▌       | 57887/225000 [119:55:34<324:02:38,  6.98s/it]
 26%|██▌       | 57888/225000 [119:55:39<296:02:34,  6.38s/it]
 26%|██▌       | 57889/225000 [119:55:47<310:12:01,  6.68s/it]
 26%|██▌       | 57890/225000 [119:55:55<334:34:03,  7.21s/it]
 26%|██▌       | 57891/225000 [119:56:01<317:47:10,  6.85s/it]
 26%|██▌       | 57892/225000 [119:56:07<301:56:48,  6.50s/it]
 26%|██▌       | 57893/225000 [119:56:11<270:43:00,  5.83s/it]
 26%|██▌       | 57894/225000 [119:56:22<335:49:41,  7.23s/it]
 26%|██▌       | 57895/225000 [119:56:26<292:12:51,  6.30s/it]
 26%|██▌       | 57896/225000 [119:56:33<302:53:13,  6.53s/it]
 26%|██▌       | 57897/225000 [119:56:40<317:39:19,  6.84s/it]
 26%|██▌       | 57898/225000 [119:56:45<280:20:09,  6.04s/it]
 26%|██▌       | 57899/225000 [119:56:55<340:18:59,  7.33s/it]
 26%|██▌       | 57900/225000 [119:56:59<288:07:02,  6.21s/it]
                                                              
{'loss': '0.7278', 'grad_norm': '0.7474', 'learning_rate': '4.413e-05', 'epoch': '0.2573', 'num_input_tokens_seen': 199892256, 'train_runtime': '4.318e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57900/225000 [119:56:59<288:07:02,  6.21s/it]
 26%|██▌       | 57901/225000 [119:57:07<325:36:04,  7.01s/it]
 26%|██▌       | 57902/225000 [119:57:12<291:22:36,  6.28s/it]
 26%|██▌       | 57903/225000 [119:57:21<325:22:59,  7.01s/it]
 26%|██▌       | 57904/225000 [119:57:25<284:12:47,  6.12s/it]
 26%|██▌       | 57905/225000 [119:57:30<275:36:34,  5.94s/it]
 26%|██▌       | 57906/225000 [119:57:38<296:46:50,  6.39s/it]
 26%|██▌       | 57907/225000 [119:57:45<312:24:21,  6.73s/it]
 26%|██▌       | 57908/225000 [119:57:52<313:11:28,  6.75s/it]
 26%|██▌       | 57909/225000 [119:57:56<278:34:21,  6.00s/it]
 26%|██▌       | 57910/225000 [119:58:03<288:06:26,  6.21s/it]
 26%|██▌       | 57911/225000 [119:58:12<325:10:30,  7.01s/it]
 26%|██▌       | 57912/225000 [119:58:18<306:20:51,  6.60s/it]
 26%|██▌       | 57913/225000 [119:58:23<294:54:00,  6.35s/it]
 26%|██▌       | 57914/225000 [119:58:28<277:59:08,  5.99s/it]
 26%|██▌       | 57915/225000 [119:58:33<265:11:38,  5.71s/it]
 26%|██▌       | 57916/225000 [119:58:38<255:10:31,  5.50s/it]
 26%|██▌       | 57917/225000 [119:58:49<318:03:12,  6.85s/it]
 26%|██▌       | 57918/225000 [119:58:58<351:10:26,  7.57s/it]
 26%|██▌       | 57919/225000 [119:59:05<342:14:08,  7.37s/it]
 26%|██▌       | 57920/225000 [119:59:08<288:32:08,  6.22s/it]
 26%|██▌       | 57921/225000 [119:59:17<328:16:06,  7.07s/it]
 26%|██▌       | 57922/225000 [119:59:24<324:51:22,  7.00s/it]
 26%|██▌       | 57923/225000 [119:59:30<310:14:49,  6.68s/it]
 26%|██▌       | 57924/225000 [119:59:36<304:57:47,  6.57s/it]
 26%|██▌       | 57925/225000 [119:59:41<275:56:59,  5.95s/it]
 26%|██▌       | 57926/225000 [119:59:49<312:44:54,  6.74s/it]
 26%|██▌       | 57927/225000 [119:59:54<277:13:46,  5.97s/it]
 26%|██▌       | 57928/225000 [120:00:05<347:22:19,  7.49s/it]
 26%|██▌       | 57929/225000 [120:00:09<300:21:34,  6.47s/it]
 26%|██▌       | 57930/225000 [120:00:14<286:14:36,  6.17s/it]
 26%|██▌       | 57931/225000 [120:00:20<275:11:52,  5.93s/it]
 26%|██▌       | 57932/225000 [120:00:25<268:37:06,  5.79s/it]
 26%|██▌       | 57933/225000 [120:00:31<268:18:27,  5.78s/it]
 26%|██▌       | 57934/225000 [120:00:42<346:54:35,  7.48s/it]
 26%|██▌       | 57935/225000 [120:00:47<305:46:35,  6.59s/it]
 26%|██▌       | 57936/225000 [120:00:51<275:48:48,  5.94s/it]
 26%|██▌       | 57937/225000 [120:00:57<273:03:42,  5.88s/it]
 26%|██▌       | 57938/225000 [120:01:09<356:40:54,  7.69s/it]
 26%|██▌       | 57939/225000 [120:01:12<296:47:35,  6.40s/it]
 26%|██▌       | 57940/225000 [120:01:21<323:59:03,  6.98s/it]
 26%|██▌       | 57941/225000 [120:01:26<308:51:03,  6.66s/it]
 26%|██▌       | 57942/225000 [120:01:34<315:53:35,  6.81s/it]
 26%|██▌       | 57943/225000 [120:01:39<296:28:06,  6.39s/it]
 26%|██▌       | 57944/225000 [120:01:46<305:27:31,  6.58s/it]
 26%|██▌       | 57945/225000 [120:01:53<312:05:26,  6.73s/it]
 26%|██▌       | 57946/225000 [120:01:59<302:59:41,  6.53s/it]
 26%|██▌       | 57947/225000 [120:02:03<267:03:17,  5.76s/it]
 26%|██▌       | 57948/225000 [120:02:10<281:40:20,  6.07s/it]
 26%|██▌       | 57949/225000 [120:02:18<308:20:09,  6.64s/it]
 26%|██▌       | 57950/225000 [120:02:22<268:17:07,  5.78s/it]
                                                              
{'loss': '0.7292', 'grad_norm': '0.7809', 'learning_rate': '4.412e-05', 'epoch': '0.2576', 'num_input_tokens_seen': 200039624, 'train_runtime': '4.321e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 57950/225000 [120:02:22<268:17:07,  5.78s/it]
 26%|██▌       | 57951/225000 [120:02:30<305:40:36,  6.59s/it]
 26%|██▌       | 57952/225000 [120:02:37<309:21:47,  6.67s/it]
 26%|██▌       | 57953/225000 [120:02:42<282:43:53,  6.09s/it]
 26%|██▌       | 57954/225000 [120:02:47<272:06:51,  5.86s/it]
 26%|██▌       | 57955/225000 [120:02:52<258:05:47,  5.56s/it]
 26%|██▌       | 57956/225000 [120:02:57<255:35:04,  5.51s/it]
 26%|██▌       | 57957/225000 [120:03:03<262:18:57,  5.65s/it]
 26%|██▌       | 57958/225000 [120:03:09<262:32:56,  5.66s/it]
 26%|██▌       | 57959/225000 [120:03:16<285:45:28,  6.16s/it]
 26%|██▌       | 57960/225000 [120:03:29<378:41:13,  8.16s/it]
 26%|██▌       | 57961/225000 [120:03:35<347:46:36,  7.50s/it]
 26%|██▌       | 57962/225000 [120:03:42<333:29:25,  7.19s/it]
 26%|██▌       | 57963/225000 [120:03:48<326:57:43,  7.05s/it]
 26%|██▌       | 57964/225000 [120:03:52<285:41:31,  6.16s/it]
 26%|██▌       | 57965/225000 [120:03:57<268:08:54,  5.78s/it]
 26%|██▌       | 57966/225000 [120:04:10<370:04:58,  7.98s/it]
 26%|██▌       | 57967/225000 [120:04:16<339:45:55,  7.32s/it]
 26%|██▌       | 57968/225000 [120:04:24<343:12:23,  7.40s/it]
 26%|██▌       | 57969/225000 [120:04:30<333:47:53,  7.19s/it]
 26%|██▌       | 57970/225000 [120:04:35<297:43:12,  6.42s/it]
 26%|██▌       | 57971/225000 [120:04:41<289:38:47,  6.24s/it]
 26%|██▌       | 57972/225000 [120:04:46<273:45:26,  5.90s/it]
 26%|██▌       | 57973/225000 [120:04:52<272:53:57,  5.88s/it]
 26%|██▌       | 57974/225000 [120:04:57<265:41:50,  5.73s/it]
 26%|██▌       | 57975/225000 [120:05:03<260:43:32,  5.62s/it]
 26%|██▌       | 57976/225000 [120:05:09<265:58:53,  5.73s/it]
 26%|██▌       | 57977/225000 [120:05:21<352:52:16,  7.61s/it]
 26%|██▌       | 57978/225000 [120:05:28<346:05:13,  7.46s/it]
 26%|██▌       | 57979/225000 [120:05:36<359:25:12,  7.75s/it]
 26%|██▌       | 57980/225000 [120:05:41<317:01:20,  6.83s/it]
 26%|██▌       | 57981/225000 [120:05:45<285:49:29,  6.16s/it]
 26%|██▌       | 57982/225000 [120:05:51<272:06:41,  5.87s/it]
 26%|██▌       | 57983/225000 [120:06:03<364:09:16,  7.85s/it]
 26%|██▌       | 57984/225000 [120:06:07<306:52:59,  6.61s/it]
 26%|██▌       | 57985/225000 [120:06:11<278:33:08,  6.00s/it]
 26%|██▌       | 57986/225000 [120:06:16<260:22:15,  5.61s/it]
 26%|██▌       | 57987/225000 [120:06:26<321:39:01,  6.93s/it]
 26%|██▌       | 57988/225000 [120:06:32<306:37:28,  6.61s/it]
 26%|██▌       | 57989/225000 [120:06:36<272:17:49,  5.87s/it]
 26%|██▌       | 57990/225000 [120:06:48<353:38:20,  7.62s/it]
 26%|██▌       | 57991/225000 [120:06:53<314:42:54,  6.78s/it]
 26%|██▌       | 57992/225000 [120:06:58<298:14:18,  6.43s/it]
 26%|██▌       | 57993/225000 [120:07:03<279:45:13,  6.03s/it]
 26%|██▌       | 57994/225000 [120:07:09<275:27:16,  5.94s/it]
 26%|██▌       | 57995/225000 [120:07:15<271:22:54,  5.85s/it]
 26%|██▌       | 57996/225000 [120:07:28<369:25:41,  7.96s/it]
 26%|██▌       | 57997/225000 [120:07:32<326:05:21,  7.03s/it]
 26%|██▌       | 57998/225000 [120:07:40<331:37:47,  7.15s/it]
 26%|██▌       | 57999/225000 [120:07:46<325:05:07,  7.01s/it]
 26%|██▌       | 58000/225000 [120:07:52<304:35:34,  6.57s/it]
                                                              
{'loss': '0.7415', 'grad_norm': '0.7268', 'learning_rate': '4.411e-05', 'epoch': '0.2578', 'num_input_tokens_seen': 200190352, 'train_runtime': '4.325e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58000/225000 [120:07:52<304:35:34,  6.57s/it]
 26%|██▌       | 58001/225000 [120:07:58<302:54:35,  6.53s/it]
 26%|██▌       | 58002/225000 [120:08:03<280:44:48,  6.05s/it]
 26%|██▌       | 58003/225000 [120:08:09<280:41:59,  6.05s/it]
 26%|██▌       | 58004/225000 [120:08:15<278:55:57,  6.01s/it]
 26%|██▌       | 58005/225000 [120:08:21<273:29:50,  5.90s/it]
 26%|██▌       | 58006/225000 [120:08:27<278:14:35,  6.00s/it]
 26%|██▌       | 58007/225000 [120:08:33<270:11:25,  5.82s/it]
 26%|██▌       | 58008/225000 [120:08:44<344:13:44,  7.42s/it]
 26%|██▌       | 58009/225000 [120:08:51<345:45:37,  7.45s/it]
 26%|██▌       | 58010/225000 [120:08:57<315:16:43,  6.80s/it]
 26%|██▌       | 58011/225000 [120:09:02<291:57:14,  6.29s/it]
 26%|██▌       | 58012/225000 [120:09:07<282:08:37,  6.08s/it]
 26%|██▌       | 58013/225000 [120:09:13<279:15:19,  6.02s/it]
 26%|██▌       | 58014/225000 [120:09:19<275:46:54,  5.95s/it]
 26%|██▌       | 58015/225000 [120:09:24<269:52:52,  5.82s/it]
 26%|██▌       | 58016/225000 [120:09:37<365:50:46,  7.89s/it]
 26%|██▌       | 58017/225000 [120:09:45<367:20:12,  7.92s/it]
 26%|██▌       | 58018/225000 [120:09:49<315:54:52,  6.81s/it]
 26%|██▌       | 58019/225000 [120:09:54<279:33:46,  6.03s/it]
 26%|██▌       | 58020/225000 [120:10:04<342:18:59,  7.38s/it]
 26%|██▌       | 58021/225000 [120:10:08<290:58:57,  6.27s/it]
 26%|██▌       | 58022/225000 [120:10:16<323:49:56,  6.98s/it]
 26%|██▌       | 58023/225000 [120:10:22<299:45:49,  6.46s/it]
 26%|██▌       | 58024/225000 [120:10:30<319:28:52,  6.89s/it]
 26%|██▌       | 58025/225000 [120:10:34<282:11:00,  6.08s/it]
 26%|██▌       | 58026/225000 [120:10:41<298:20:57,  6.43s/it]
 26%|██▌       | 58027/225000 [120:10:50<333:58:47,  7.20s/it]
 26%|██▌       | 58028/225000 [120:10:58<340:28:28,  7.34s/it]
 26%|██▌       | 58029/225000 [120:11:03<309:05:34,  6.66s/it]
 26%|██▌       | 58030/225000 [120:11:08<284:05:13,  6.13s/it]
 26%|██▌       | 58031/225000 [120:11:13<271:07:35,  5.85s/it]
 26%|██▌       | 58032/225000 [120:11:25<356:13:05,  7.68s/it]
 26%|██▌       | 58033/225000 [120:11:29<302:31:26,  6.52s/it]
 26%|██▌       | 58034/225000 [120:11:38<343:25:42,  7.40s/it]
 26%|██▌       | 58035/225000 [120:11:42<294:13:22,  6.34s/it]
 26%|██▌       | 58036/225000 [120:11:52<348:28:38,  7.51s/it]
 26%|██▌       | 58037/225000 [120:11:57<304:40:51,  6.57s/it]
 26%|██▌       | 58038/225000 [120:12:02<289:46:11,  6.25s/it]
 26%|██▌       | 58039/225000 [120:12:08<281:57:06,  6.08s/it]
 26%|██▌       | 58040/225000 [120:12:13<274:55:10,  5.93s/it]
 26%|██▌       | 58041/225000 [120:12:24<346:52:23,  7.48s/it]
 26%|██▌       | 58042/225000 [120:12:29<305:47:18,  6.59s/it]
 26%|██▌       | 58043/225000 [120:12:36<310:16:51,  6.69s/it]
 26%|██▌       | 58044/225000 [120:12:41<286:17:47,  6.17s/it]
 26%|██▌       | 58045/225000 [120:12:46<274:03:50,  5.91s/it]
 26%|██▌       | 58046/225000 [120:12:53<293:14:29,  6.32s/it]
 26%|██▌       | 58047/225000 [120:13:03<340:58:45,  7.35s/it]
 26%|██▌       | 58048/225000 [120:13:07<292:49:40,  6.31s/it]
 26%|██▌       | 58049/225000 [120:13:17<343:58:52,  7.42s/it]
 26%|██▌       | 58050/225000 [120:13:22<312:00:16,  6.73s/it]
                                                              
{'loss': '0.7415', 'grad_norm': '0.7531', 'learning_rate': '4.409e-05', 'epoch': '0.258', 'num_input_tokens_seen': 200340088, 'train_runtime': '4.328e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58050/225000 [120:13:22<312:00:16,  6.73s/it]
 26%|██▌       | 58051/225000 [120:13:27<281:21:52,  6.07s/it]
 26%|██▌       | 58052/225000 [120:13:32<266:38:04,  5.75s/it]
 26%|██▌       | 58053/225000 [120:13:43<348:17:22,  7.51s/it]
 26%|██▌       | 58054/225000 [120:13:48<306:53:34,  6.62s/it]
 26%|██▌       | 58055/225000 [120:13:52<278:25:19,  6.00s/it]
 26%|██▌       | 58056/225000 [120:14:03<344:19:44,  7.43s/it]
 26%|██▌       | 58057/225000 [120:14:10<339:20:37,  7.32s/it]
 26%|██▌       | 58058/225000 [120:14:15<305:56:03,  6.60s/it]
 26%|██▌       | 58059/225000 [120:14:21<291:25:39,  6.28s/it]
 26%|██▌       | 58060/225000 [120:14:26<277:37:57,  5.99s/it]
 26%|██▌       | 58061/225000 [120:14:31<269:09:06,  5.80s/it]
 26%|██▌       | 58062/225000 [120:14:37<259:48:29,  5.60s/it]
 26%|██▌       | 58063/225000 [120:14:48<342:16:57,  7.38s/it]
 26%|██▌       | 58064/225000 [120:14:55<336:00:09,  7.25s/it]
 26%|██▌       | 58065/225000 [120:15:00<301:38:58,  6.51s/it]
 26%|██▌       | 58066/225000 [120:15:04<276:46:40,  5.97s/it]
 26%|██▌       | 58067/225000 [120:15:14<331:32:18,  7.15s/it]
 26%|██▌       | 58068/225000 [120:15:22<332:40:13,  7.17s/it]
 26%|██▌       | 58069/225000 [120:15:28<321:00:45,  6.92s/it]
 26%|██▌       | 58070/225000 [120:15:32<281:26:13,  6.07s/it]
 26%|██▌       | 58071/225000 [120:15:37<259:37:27,  5.60s/it]
 26%|██▌       | 58072/225000 [120:15:49<355:48:52,  7.67s/it]
 26%|██▌       | 58073/225000 [120:15:53<300:13:14,  6.47s/it]
 26%|██▌       | 58074/225000 [120:16:02<334:04:37,  7.20s/it]
 26%|██▌       | 58075/225000 [120:16:05<287:09:11,  6.19s/it]
 26%|██▌       | 58076/225000 [120:16:13<305:01:46,  6.58s/it]
 26%|██▌       | 58077/225000 [120:16:18<289:46:41,  6.25s/it]
 26%|██▌       | 58078/225000 [120:16:27<326:08:10,  7.03s/it]
 26%|██▌       | 58079/225000 [120:16:31<277:49:46,  5.99s/it]
 26%|██▌       | 58080/225000 [120:16:42<350:11:59,  7.55s/it]
 26%|██▌       | 58081/225000 [120:16:46<304:44:31,  6.57s/it]
 26%|██▌       | 58082/225000 [120:16:53<306:52:48,  6.62s/it]
 26%|██▌       | 58083/225000 [120:17:00<312:36:56,  6.74s/it]
 26%|██▌       | 58084/225000 [120:17:05<281:37:46,  6.07s/it]
 26%|██▌       | 58085/225000 [120:17:10<274:27:24,  5.92s/it]
 26%|██▌       | 58086/225000 [120:17:15<254:52:51,  5.50s/it]
 26%|██▌       | 58087/225000 [120:17:20<249:32:52,  5.38s/it]
 26%|██▌       | 58088/225000 [120:17:26<255:39:55,  5.51s/it]
 26%|██▌       | 58089/225000 [120:17:36<320:06:54,  6.90s/it]
 26%|██▌       | 58090/225000 [120:17:47<375:40:26,  8.10s/it]
 26%|██▌       | 58091/225000 [120:17:51<319:09:15,  6.88s/it]
 26%|██▌       | 58092/225000 [120:17:57<311:22:17,  6.72s/it]
 26%|██▌       | 58093/225000 [120:18:07<353:23:56,  7.62s/it]
 26%|██▌       | 58094/225000 [120:18:11<304:02:35,  6.56s/it]
 26%|██▌       | 58095/225000 [120:18:20<346:20:47,  7.47s/it]
 26%|██▌       | 58096/225000 [120:18:26<319:01:39,  6.88s/it]
 26%|██▌       | 58097/225000 [120:18:30<277:11:30,  5.98s/it]
 26%|██▌       | 58098/225000 [120:18:40<329:32:32,  7.11s/it]
 26%|██▌       | 58099/225000 [120:18:47<331:38:39,  7.15s/it]
 26%|██▌       | 58100/225000 [120:18:51<290:05:27,  6.26s/it]
                                                              
{'loss': '0.7262', 'grad_norm': '0.7925', 'learning_rate': '4.408e-05', 'epoch': '0.2582', 'num_input_tokens_seen': 200489096, 'train_runtime': '4.331e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58100/225000 [120:18:51<290:05:27,  6.26s/it]
 26%|██▌       | 58101/225000 [120:18:58<303:20:12,  6.54s/it]
 26%|██▌       | 58102/225000 [120:19:06<315:50:27,  6.81s/it]
 26%|██▌       | 58103/225000 [120:19:13<322:03:32,  6.95s/it]
 26%|██▌       | 58104/225000 [120:19:17<287:35:08,  6.20s/it]
 26%|██▌       | 58105/225000 [120:19:26<317:06:38,  6.84s/it]
 26%|██▌       | 58106/225000 [120:19:30<281:34:21,  6.07s/it]
 26%|██▌       | 58107/225000 [120:19:42<361:29:14,  7.80s/it]
 26%|██▌       | 58108/225000 [120:19:45<302:36:02,  6.53s/it]
 26%|██▌       | 58109/225000 [120:19:51<290:21:49,  6.26s/it]
 26%|██▌       | 58110/225000 [120:19:57<283:11:09,  6.11s/it]
 26%|██▌       | 58111/225000 [120:20:02<277:39:40,  5.99s/it]
 26%|██▌       | 58112/225000 [120:20:13<342:48:36,  7.39s/it]
 26%|██▌       | 58113/225000 [120:20:21<345:12:01,  7.45s/it]
 26%|██▌       | 58114/225000 [120:20:24<292:04:16,  6.30s/it]
 26%|██▌       | 58115/225000 [120:20:32<314:18:46,  6.78s/it]
 26%|██▌       | 58116/225000 [120:20:39<308:13:59,  6.65s/it]
 26%|██▌       | 58117/225000 [120:20:44<292:07:53,  6.30s/it]
 26%|██▌       | 58118/225000 [120:20:52<313:49:03,  6.77s/it]
 26%|██▌       | 58119/225000 [120:20:56<274:06:16,  5.91s/it]
 26%|██▌       | 58120/225000 [120:21:01<262:55:33,  5.67s/it]
 26%|██▌       | 58121/225000 [120:21:12<343:54:35,  7.42s/it]
 26%|██▌       | 58122/225000 [120:21:16<291:07:40,  6.28s/it]
 26%|██▌       | 58123/225000 [120:21:20<255:42:37,  5.52s/it]
 26%|██▌       | 58124/225000 [120:21:29<307:37:19,  6.64s/it]
 26%|██▌       | 58125/225000 [120:21:34<281:35:09,  6.07s/it]
 26%|██▌       | 58126/225000 [120:21:43<326:58:52,  7.05s/it]
 26%|██▌       | 58127/225000 [120:21:47<281:59:57,  6.08s/it]
 26%|██▌       | 58128/225000 [120:21:56<328:47:48,  7.09s/it]
 26%|██▌       | 58129/225000 [120:22:02<311:25:02,  6.72s/it]
 26%|██▌       | 58130/225000 [120:22:09<317:53:09,  6.86s/it]
 26%|██▌       | 58131/225000 [120:22:14<286:45:40,  6.19s/it]
 26%|██▌       | 58132/225000 [120:22:19<274:39:54,  5.93s/it]
 26%|██▌       | 58133/225000 [120:22:30<334:37:45,  7.22s/it]
 26%|██▌       | 58134/225000 [120:22:35<306:40:44,  6.62s/it]
 26%|██▌       | 58135/225000 [120:22:40<286:19:08,  6.18s/it]
 26%|██▌       | 58136/225000 [120:22:46<277:16:15,  5.98s/it]
 26%|██▌       | 58137/225000 [120:22:51<264:12:56,  5.70s/it]
 26%|██▌       | 58138/225000 [120:22:56<259:51:34,  5.61s/it]
 26%|██▌       | 58139/225000 [120:23:10<371:16:58,  8.01s/it]
 26%|██▌       | 58140/225000 [120:23:15<341:59:10,  7.38s/it]
 26%|██▌       | 58141/225000 [120:23:21<311:58:43,  6.73s/it]
 26%|██▌       | 58142/225000 [120:23:29<327:57:11,  7.08s/it]
 26%|██▌       | 58143/225000 [120:23:32<282:12:31,  6.09s/it]
 26%|██▌       | 58144/225000 [120:23:41<324:35:10,  7.00s/it]
 26%|██▌       | 58145/225000 [120:23:48<318:09:45,  6.86s/it]
 26%|██▌       | 58146/225000 [120:23:52<275:24:16,  5.94s/it]
 26%|██▌       | 58147/225000 [120:23:59<295:38:42,  6.38s/it]
 26%|██▌       | 58148/225000 [120:24:03<258:47:53,  5.58s/it]
 26%|██▌       | 58149/225000 [120:24:14<331:07:37,  7.14s/it]
 26%|██▌       | 58150/225000 [120:24:18<290:49:03,  6.27s/it]
                                                              
{'loss': '0.7269', 'grad_norm': '0.8723', 'learning_rate': '4.407e-05', 'epoch': '0.2584', 'num_input_tokens_seen': 200638264, 'train_runtime': '4.335e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58150/225000 [120:24:18<290:49:03,  6.27s/it]
 26%|██▌       | 58151/225000 [120:24:24<293:18:52,  6.33s/it]
 26%|██▌       | 58152/225000 [120:24:36<362:29:44,  7.82s/it]
 26%|██▌       | 58153/225000 [120:24:39<298:10:27,  6.43s/it]
 26%|██▌       | 58154/225000 [120:24:45<286:35:16,  6.18s/it]
 26%|██▌       | 58155/225000 [120:24:53<311:41:27,  6.73s/it]
 26%|██▌       | 58156/225000 [120:25:01<331:02:06,  7.14s/it]
 26%|██▌       | 58157/225000 [120:25:06<300:36:23,  6.49s/it]
 26%|██▌       | 58158/225000 [120:25:11<286:59:35,  6.19s/it]
 26%|██▌       | 58159/225000 [120:25:17<276:40:40,  5.97s/it]
 26%|██▌       | 58160/225000 [120:25:28<352:52:23,  7.61s/it]
 26%|██▌       | 58161/225000 [120:25:35<339:20:02,  7.32s/it]
 26%|██▌       | 58162/225000 [120:25:39<299:25:56,  6.46s/it]
 26%|██▌       | 58163/225000 [120:25:44<274:33:55,  5.92s/it]
 26%|██▌       | 58164/225000 [120:25:53<315:13:05,  6.80s/it]
 26%|██▌       | 58165/225000 [120:26:00<329:53:10,  7.12s/it]
 26%|██▌       | 58166/225000 [120:26:05<291:05:10,  6.28s/it]
 26%|██▌       | 58167/225000 [120:26:09<265:27:15,  5.73s/it]
 26%|██▌       | 58168/225000 [120:26:22<367:04:03,  7.92s/it]
 26%|██▌       | 58169/225000 [120:26:26<308:01:53,  6.65s/it]
 26%|██▌       | 58170/225000 [120:26:31<283:15:17,  6.11s/it]
 26%|██▌       | 58171/225000 [120:26:35<260:19:07,  5.62s/it]
 26%|██▌       | 58172/225000 [120:26:48<353:15:30,  7.62s/it]
 26%|██▌       | 58173/225000 [120:26:52<304:12:02,  6.56s/it]
 26%|██▌       | 58174/225000 [120:26:57<288:55:39,  6.23s/it]
 26%|██▌       | 58175/225000 [120:27:03<283:00:59,  6.11s/it]
 26%|██▌       | 58176/225000 [120:27:11<314:06:15,  6.78s/it]
 26%|██▌       | 58177/225000 [120:27:20<346:46:38,  7.48s/it]
 26%|██▌       | 58178/225000 [120:27:26<318:12:52,  6.87s/it]
 26%|██▌       | 58179/225000 [120:27:30<280:19:25,  6.05s/it]
 26%|██▌       | 58180/225000 [120:27:37<298:03:17,  6.43s/it]
 26%|██▌       | 58181/225000 [120:27:44<299:40:06,  6.47s/it]
 26%|██▌       | 58182/225000 [120:27:49<286:01:15,  6.17s/it]
 26%|██▌       | 58183/225000 [120:27:55<272:33:36,  5.88s/it]
 26%|██▌       | 58184/225000 [120:28:00<266:10:03,  5.74s/it]
 26%|██▌       | 58185/225000 [120:28:13<367:17:51,  7.93s/it]
 26%|██▌       | 58186/225000 [120:28:20<359:14:06,  7.75s/it]
 26%|██▌       | 58187/225000 [120:28:27<346:09:16,  7.47s/it]
 26%|██▌       | 58188/225000 [120:28:34<334:49:56,  7.23s/it]
 26%|██▌       | 58189/225000 [120:28:39<311:24:19,  6.72s/it]
 26%|██▌       | 58190/225000 [120:28:44<280:20:29,  6.05s/it]
 26%|██▌       | 58191/225000 [120:28:49<265:07:31,  5.72s/it]
 26%|██▌       | 58192/225000 [120:28:59<322:08:02,  6.95s/it]
 26%|██▌       | 58193/225000 [120:29:04<299:18:04,  6.46s/it]
 26%|██▌       | 58194/225000 [120:29:09<284:08:48,  6.13s/it]
 26%|██▌       | 58195/225000 [120:29:15<273:04:48,  5.89s/it]
 26%|██▌       | 58196/225000 [120:29:20<266:49:48,  5.76s/it]
 26%|██▌       | 58197/225000 [120:29:30<321:47:31,  6.95s/it]
 26%|██▌       | 58198/225000 [120:29:37<331:59:43,  7.17s/it]
 26%|██▌       | 58199/225000 [120:29:43<309:33:11,  6.68s/it]
 26%|██▌       | 58200/225000 [120:29:48<288:42:44,  6.23s/it]
                                                              
{'loss': '0.7377', 'grad_norm': '0.7702', 'learning_rate': '4.406e-05', 'epoch': '0.2587', 'num_input_tokens_seen': 200789248, 'train_runtime': '4.338e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58200/225000 [120:29:48<288:42:44,  6.23s/it]
 26%|██▌       | 58201/225000 [120:29:54<279:02:03,  6.02s/it]
 26%|██▌       | 58202/225000 [120:29:59<264:42:34,  5.71s/it]
 26%|██▌       | 58203/225000 [120:30:04<260:14:45,  5.62s/it]
 26%|██▌       | 58204/225000 [120:30:10<261:57:00,  5.65s/it]
 26%|██▌       | 58205/225000 [120:30:15<259:45:25,  5.61s/it]
 26%|██▌       | 58206/225000 [120:30:22<270:46:27,  5.84s/it]
 26%|██▌       | 58207/225000 [120:30:34<353:58:21,  7.64s/it]
 26%|██▌       | 58208/225000 [120:30:45<407:32:25,  8.80s/it]
 26%|██▌       | 58209/225000 [120:30:50<347:47:38,  7.51s/it]
 26%|██▌       | 58210/225000 [120:30:55<319:47:18,  6.90s/it]
 26%|██▌       | 58211/225000 [120:31:03<337:19:17,  7.28s/it]
 26%|██▌       | 58212/225000 [120:31:09<311:28:23,  6.72s/it]
 26%|██▌       | 58213/225000 [120:31:17<337:13:25,  7.28s/it]
 26%|██▌       | 58214/225000 [120:31:21<292:33:23,  6.31s/it]
 26%|██▌       | 58215/225000 [120:31:30<326:25:09,  7.05s/it]
 26%|██▌       | 58216/225000 [120:31:34<282:38:26,  6.10s/it]
 26%|██▌       | 58217/225000 [120:31:38<251:57:18,  5.44s/it]
 26%|██▌       | 58218/225000 [120:31:49<331:02:32,  7.15s/it]
 26%|██▌       | 58219/225000 [120:31:53<291:53:43,  6.30s/it]
 26%|██▌       | 58220/225000 [120:32:00<291:04:17,  6.28s/it]
 26%|██▌       | 58221/225000 [120:32:07<312:35:28,  6.75s/it]
 26%|██▌       | 58222/225000 [120:32:12<279:33:19,  6.03s/it]
 26%|██▌       | 58223/225000 [120:32:16<251:21:11,  5.43s/it]
 26%|██▌       | 58224/225000 [120:32:28<350:47:26,  7.57s/it]
 26%|██▌       | 58225/225000 [120:32:32<301:19:25,  6.50s/it]
 26%|██▌       | 58226/225000 [120:32:37<281:35:33,  6.08s/it]
 26%|██▌       | 58227/225000 [120:32:47<328:58:08,  7.10s/it]
 26%|██▌       | 58228/225000 [120:32:52<295:52:47,  6.39s/it]
 26%|██▌       | 58229/225000 [120:33:00<316:39:42,  6.84s/it]
 26%|██▌       | 58230/225000 [120:33:04<286:55:10,  6.19s/it]
 26%|██▌       | 58231/225000 [120:33:09<268:44:05,  5.80s/it]
 26%|██▌       | 58232/225000 [120:33:21<347:36:06,  7.50s/it]
 26%|██▌       | 58233/225000 [120:33:25<310:16:21,  6.70s/it]
 26%|██▌       | 58234/225000 [120:33:35<351:51:14,  7.60s/it]
 26%|██▌       | 58235/225000 [120:33:39<302:45:48,  6.54s/it]
 26%|██▌       | 58236/225000 [120:33:48<336:20:16,  7.26s/it]
 26%|██▌       | 58237/225000 [120:33:52<286:09:06,  6.18s/it]
 26%|██▌       | 58238/225000 [120:33:59<306:35:19,  6.62s/it]
 26%|██▌       | 58239/225000 [120:34:05<292:27:04,  6.31s/it]
 26%|██▌       | 58240/225000 [120:34:11<288:16:34,  6.22s/it]
 26%|██▌       | 58241/225000 [120:34:16<268:04:52,  5.79s/it]
 26%|██▌       | 58242/225000 [120:34:22<270:18:01,  5.84s/it]
 26%|██▌       | 58243/225000 [120:34:27<261:56:48,  5.65s/it]
 26%|██▌       | 58244/225000 [120:34:32<255:26:06,  5.51s/it]
 26%|██▌       | 58245/225000 [120:34:37<249:25:12,  5.38s/it]
 26%|██▌       | 58246/225000 [120:34:48<326:33:19,  7.05s/it]
 26%|██▌       | 58247/225000 [120:34:58<358:51:33,  7.75s/it]
 26%|██▌       | 58248/225000 [120:35:01<301:00:04,  6.50s/it]
 26%|██▌       | 58249/225000 [120:35:06<276:49:20,  5.98s/it]
 26%|██▌       | 58250/225000 [120:35:18<363:04:39,  7.84s/it]
                                                              
{'loss': '0.7472', 'grad_norm': '0.9378', 'learning_rate': '4.405e-05', 'epoch': '0.2589', 'num_input_tokens_seen': 200938416, 'train_runtime': '4.341e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58250/225000 [120:35:18<363:04:39,  7.84s/it]
 26%|██▌       | 58251/225000 [120:35:22<313:25:12,  6.77s/it]
 26%|██▌       | 58252/225000 [120:35:27<283:50:49,  6.13s/it]
 26%|██▌       | 58253/225000 [120:35:31<259:23:11,  5.60s/it]
 26%|██▌       | 58254/225000 [120:35:41<315:35:48,  6.81s/it]
 26%|██▌       | 58255/225000 [120:35:49<334:07:42,  7.21s/it]
 26%|██▌       | 58256/225000 [120:35:54<294:42:50,  6.36s/it]
 26%|██▌       | 58257/225000 [120:36:05<360:15:48,  7.78s/it]
 26%|██▌       | 58258/225000 [120:36:09<310:32:50,  6.70s/it]
 26%|██▌       | 58259/225000 [120:36:13<278:07:17,  6.00s/it]
 26%|██▌       | 58260/225000 [120:36:25<355:22:17,  7.67s/it]
 26%|██▌       | 58261/225000 [120:36:28<300:55:46,  6.50s/it]
 26%|██▌       | 58262/225000 [120:36:35<298:20:05,  6.44s/it]
 26%|██▌       | 58263/225000 [120:36:39<265:18:32,  5.73s/it]
 26%|██▌       | 58264/225000 [120:36:49<322:40:42,  6.97s/it]
 26%|██▌       | 58265/225000 [120:36:53<286:18:48,  6.18s/it]
 26%|██▌       | 58266/225000 [120:36:57<260:09:15,  5.62s/it]
 26%|██▌       | 58267/225000 [120:37:08<331:53:35,  7.17s/it]
 26%|██▌       | 58268/225000 [120:37:14<310:39:26,  6.71s/it]
 26%|██▌       | 58269/225000 [120:37:22<326:13:28,  7.04s/it]
 26%|██▌       | 58270/225000 [120:37:27<299:15:48,  6.46s/it]
 26%|██▌       | 58271/225000 [120:37:32<280:27:00,  6.06s/it]
 26%|██▌       | 58272/225000 [120:37:37<264:38:33,  5.71s/it]
 26%|██▌       | 58273/225000 [120:37:47<329:31:07,  7.12s/it]
 26%|██▌       | 58274/225000 [120:37:54<328:51:51,  7.10s/it]
 26%|██▌       | 58275/225000 [120:37:58<286:57:23,  6.20s/it]
 26%|██▌       | 58276/225000 [120:38:02<253:36:53,  5.48s/it]
 26%|██▌       | 58277/225000 [120:38:07<241:51:36,  5.22s/it]
 26%|██▌       | 58278/225000 [120:38:16<300:01:39,  6.48s/it]
 26%|██▌       | 58279/225000 [120:38:27<355:20:56,  7.67s/it]
 26%|██▌       | 58280/225000 [120:38:30<302:53:02,  6.54s/it]
 26%|██▌       | 58281/225000 [120:38:40<338:35:19,  7.31s/it]
 26%|██▌       | 58282/225000 [120:38:43<285:18:51,  6.16s/it]
 26%|██▌       | 58283/225000 [120:38:51<307:29:05,  6.64s/it]
 26%|██▌       | 58284/225000 [120:38:56<281:40:28,  6.08s/it]
 26%|██▌       | 58285/225000 [120:39:02<289:44:36,  6.26s/it]
 26%|██▌       | 58286/225000 [120:39:07<273:43:24,  5.91s/it]
 26%|██▌       | 58287/225000 [120:39:13<273:11:05,  5.90s/it]
 26%|██▌       | 58288/225000 [120:39:19<271:59:56,  5.87s/it]
 26%|██▌       | 58289/225000 [120:39:25<274:47:47,  5.93s/it]
 26%|██▌       | 58290/225000 [120:39:32<290:44:37,  6.28s/it]
 26%|██▌       | 58291/225000 [120:39:39<292:53:35,  6.32s/it]
 26%|██▌       | 58292/225000 [120:39:45<288:31:39,  6.23s/it]
 26%|██▌       | 58293/225000 [120:39:50<282:03:34,  6.09s/it]
 26%|██▌       | 58294/225000 [120:40:02<355:40:00,  7.68s/it]
 26%|██▌       | 58295/225000 [120:40:10<361:12:52,  7.80s/it]
 26%|██▌       | 58296/225000 [120:40:18<363:32:32,  7.85s/it]
 26%|██▌       | 58297/225000 [120:40:24<339:33:06,  7.33s/it]
 26%|██▌       | 58298/225000 [120:40:31<331:06:04,  7.15s/it]
 26%|██▌       | 58299/225000 [120:40:36<303:54:54,  6.56s/it]
 26%|██▌       | 58300/225000 [120:40:41<281:51:50,  6.09s/it]
                                                              
{'loss': '0.7405', 'grad_norm': '0.8716', 'learning_rate': '4.403e-05', 'epoch': '0.2591', 'num_input_tokens_seen': 201086968, 'train_runtime': '4.344e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58300/225000 [120:40:41<281:51:50,  6.09s/it]
 26%|██▌       | 58301/225000 [120:40:46<273:34:44,  5.91s/it]
 26%|██▌       | 58302/225000 [120:40:52<268:28:13,  5.80s/it]
 26%|██▌       | 58303/225000 [120:40:58<272:26:09,  5.88s/it]
 26%|██▌       | 58304/225000 [120:41:10<362:53:25,  7.84s/it]
 26%|██▌       | 58305/225000 [120:41:15<324:11:56,  7.00s/it]
 26%|██▌       | 58306/225000 [120:41:20<288:54:01,  6.24s/it]
 26%|██▌       | 58307/225000 [120:41:30<346:03:42,  7.47s/it]
 26%|██▌       | 58308/225000 [120:41:36<324:16:52,  7.00s/it]
 26%|██▌       | 58309/225000 [120:41:41<298:01:26,  6.44s/it]
 26%|██▌       | 58310/225000 [120:41:46<278:24:21,  6.01s/it]
 26%|██▌       | 58311/225000 [120:41:55<316:27:05,  6.83s/it]
 26%|██▌       | 58312/225000 [120:41:59<279:34:20,  6.04s/it]
 26%|██▌       | 58313/225000 [120:42:04<263:05:50,  5.68s/it]
 26%|██▌       | 58314/225000 [120:42:13<305:25:04,  6.60s/it]
 26%|██▌       | 58315/225000 [120:42:21<333:13:28,  7.20s/it]
 26%|██▌       | 58316/225000 [120:42:25<289:54:37,  6.26s/it]
 26%|██▌       | 58317/225000 [120:42:30<264:16:41,  5.71s/it]
 26%|██▌       | 58318/225000 [120:42:42<358:55:56,  7.75s/it]
 26%|██▌       | 58319/225000 [120:42:47<312:19:47,  6.75s/it]
 26%|██▌       | 58320/225000 [120:42:52<297:34:45,  6.43s/it]
 26%|██▌       | 58321/225000 [120:43:02<338:17:24,  7.31s/it]
 26%|██▌       | 58322/225000 [120:43:07<302:02:21,  6.52s/it]
 26%|██▌       | 58323/225000 [120:43:12<282:40:17,  6.11s/it]
 26%|██▌       | 58324/225000 [120:43:17<265:30:13,  5.73s/it]
 26%|██▌       | 58325/225000 [120:43:21<252:32:49,  5.45s/it]
 26%|██▌       | 58326/225000 [120:43:34<355:16:01,  7.67s/it]
 26%|██▌       | 58327/225000 [120:43:40<324:32:00,  7.01s/it]
 26%|██▌       | 58328/225000 [120:43:48<338:41:39,  7.32s/it]
 26%|██▌       | 58329/225000 [120:43:53<310:01:08,  6.70s/it]
 26%|██▌       | 58330/225000 [120:43:59<294:19:34,  6.36s/it]
 26%|██▌       | 58331/225000 [120:44:04<276:14:47,  5.97s/it]
 26%|██▌       | 58332/225000 [120:44:09<272:04:41,  5.88s/it]
 26%|██▌       | 58333/225000 [120:44:15<264:04:11,  5.70s/it]
 26%|██▌       | 58334/225000 [120:44:26<343:09:27,  7.41s/it]
 26%|██▌       | 58335/225000 [120:44:33<344:31:12,  7.44s/it]
 26%|██▌       | 58336/225000 [120:44:37<292:40:23,  6.32s/it]
 26%|██▌       | 58337/225000 [120:44:48<359:22:06,  7.76s/it]
 26%|██▌       | 58338/225000 [120:44:53<310:39:39,  6.71s/it]
 26%|██▌       | 58339/225000 [120:44:57<282:13:56,  6.10s/it]
 26%|██▌       | 58340/225000 [120:45:02<258:33:27,  5.59s/it]
 26%|██▌       | 58341/225000 [120:45:07<251:11:26,  5.43s/it]
 26%|██▌       | 58342/225000 [120:45:18<333:52:46,  7.21s/it]
 26%|██▌       | 58343/225000 [120:45:22<291:04:06,  6.29s/it]
 26%|██▌       | 58344/225000 [120:45:31<326:14:03,  7.05s/it]
 26%|██▌       | 58345/225000 [120:45:36<300:36:24,  6.49s/it]
 26%|██▌       | 58346/225000 [120:45:41<284:02:54,  6.14s/it]
 26%|██▌       | 58347/225000 [120:45:46<266:31:58,  5.76s/it]
 26%|██▌       | 58348/225000 [120:45:52<260:36:29,  5.63s/it]
 26%|██▌       | 58349/225000 [120:45:57<256:17:24,  5.54s/it]
 26%|██▌       | 58350/225000 [120:46:06<306:49:56,  6.63s/it]
                                                              
{'loss': '0.7115', 'grad_norm': '0.7245', 'learning_rate': '4.402e-05', 'epoch': '0.2593', 'num_input_tokens_seen': 201236888, 'train_runtime': '4.348e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58350/225000 [120:46:06<306:49:56,  6.63s/it]
 26%|██▌       | 58351/225000 [120:46:17<368:07:11,  7.95s/it]
 26%|██▌       | 58352/225000 [120:46:23<335:52:08,  7.26s/it]
 26%|██▌       | 58353/225000 [120:46:29<322:23:49,  6.96s/it]
 26%|██▌       | 58354/225000 [120:46:35<310:07:43,  6.70s/it]
 26%|██▌       | 58355/225000 [120:46:40<277:15:21,  5.99s/it]
 26%|██▌       | 58356/225000 [120:46:50<332:45:10,  7.19s/it]
 26%|██▌       | 58357/225000 [120:46:56<321:20:09,  6.94s/it]
 26%|██▌       | 58358/225000 [120:47:01<290:58:51,  6.29s/it]
 26%|██▌       | 58359/225000 [120:47:06<276:09:49,  5.97s/it]
 26%|██▌       | 58360/225000 [120:47:11<264:32:02,  5.71s/it]
 26%|██▌       | 58361/225000 [120:47:22<338:47:38,  7.32s/it]
 26%|██▌       | 58362/225000 [120:47:30<343:22:18,  7.42s/it]
 26%|██▌       | 58363/225000 [120:47:34<300:05:28,  6.48s/it]
 26%|██▌       | 58364/225000 [120:47:43<337:22:47,  7.29s/it]
 26%|██▌       | 58365/225000 [120:47:49<317:35:54,  6.86s/it]
 26%|██▌       | 58366/225000 [120:47:54<294:55:16,  6.37s/it]
 26%|██▌       | 58367/225000 [120:47:59<273:07:02,  5.90s/it]
 26%|██▌       | 58368/225000 [120:48:11<354:25:59,  7.66s/it]
 26%|██▌       | 58369/225000 [120:48:16<316:10:46,  6.83s/it]
 26%|██▌       | 58370/225000 [120:48:21<290:17:58,  6.27s/it]
 26%|██▌       | 58371/225000 [120:48:26<277:27:55,  5.99s/it]
 26%|██▌       | 58372/225000 [120:48:32<270:29:37,  5.84s/it]
 26%|██▌       | 58373/225000 [120:48:37<266:16:02,  5.75s/it]
 26%|██▌       | 58374/225000 [120:48:43<262:47:33,  5.68s/it]
 26%|██▌       | 58375/225000 [120:48:48<258:47:05,  5.59s/it]
 26%|██▌       | 58376/225000 [120:49:02<377:49:05,  8.16s/it]
 26%|██▌       | 58377/225000 [120:49:08<349:52:19,  7.56s/it]
 26%|██▌       | 58378/225000 [120:49:16<349:57:13,  7.56s/it]
 26%|██▌       | 58379/225000 [120:49:20<306:51:24,  6.63s/it]
 26%|██▌       | 58380/225000 [120:49:24<272:47:54,  5.89s/it]
 26%|██▌       | 58381/225000 [120:49:36<346:10:51,  7.48s/it]
 26%|██▌       | 58382/225000 [120:49:41<321:37:34,  6.95s/it]
 26%|██▌       | 58383/225000 [120:49:47<296:28:10,  6.41s/it]
 26%|██▌       | 58384/225000 [120:49:52<278:05:42,  6.01s/it]
 26%|██▌       | 58385/225000 [120:49:57<266:20:09,  5.75s/it]
 26%|██▌       | 58386/225000 [120:50:02<261:18:52,  5.65s/it]
 26%|██▌       | 58387/225000 [120:50:10<291:23:40,  6.30s/it]
 26%|██▌       | 58388/225000 [120:50:22<373:13:34,  8.06s/it]
 26%|██▌       | 58389/225000 [120:50:26<317:27:10,  6.86s/it]
 26%|██▌       | 58390/225000 [120:50:36<357:02:29,  7.71s/it]
 26%|██▌       | 58391/225000 [120:50:40<310:29:43,  6.71s/it]
 26%|██▌       | 58392/225000 [120:50:45<282:20:45,  6.10s/it]
 26%|██▌       | 58393/225000 [120:50:54<316:38:27,  6.84s/it]
 26%|██▌       | 58394/225000 [120:51:00<305:22:28,  6.60s/it]
 26%|██▌       | 58395/225000 [120:51:05<295:05:19,  6.38s/it]
 26%|██▌       | 58396/225000 [120:51:13<305:57:38,  6.61s/it]
 26%|██▌       | 58397/225000 [120:51:20<319:46:45,  6.91s/it]
 26%|██▌       | 58398/225000 [120:51:24<279:32:33,  6.04s/it]
 26%|██▌       | 58399/225000 [120:51:31<296:55:18,  6.42s/it]
 26%|██▌       | 58400/225000 [120:51:40<323:12:47,  6.98s/it]
                                                              
{'loss': '0.7394', 'grad_norm': '0.7565', 'learning_rate': '4.401e-05', 'epoch': '0.2596', 'num_input_tokens_seen': 201386184, 'train_runtime': '4.351e+05', 'train_tokens_per_second': '462.9'}

 26%|██▌       | 58400/225000 [120:51:40<323:12:47,  6.98s/it]
 26%|██▌       | 58401/225000 [120:51:47<324:20:02,  7.01s/it]
 26%|██▌       | 58402/225000 [120:51:54<330:46:15,  7.15s/it]
 26%|██▌       | 58403/225000 [120:51:59<292:55:46,  6.33s/it]
 26%|██▌       | 58404/225000 [120:52:04<279:03:06,  6.03s/it]
 26%|██▌       | 58405/225000 [120:52:13<313:27:55,  6.77s/it]
 26%|██▌       | 58406/225000 [120:52:16<272:26:55,  5.89s/it]
 26%|██▌       | 58407/225000 [120:52:21<258:22:58,  5.58s/it]
 26%|██▌       | 58408/225000 [120:52:26<246:02:59,  5.32s/it]
 26%|██▌       | 58409/225000 [120:52:39<350:44:03,  7.58s/it]
 26%|██▌       | 58410/225000 [120:52:45<331:57:11,  7.17s/it]
 26%|██▌       | 58411/225000 [120:52:50<299:46:17,  6.48s/it]
 26%|██▌       | 58412/225000 [120:52:55<277:37:02,  6.00s/it]
 26%|██▌       | 58413/225000 [120:53:00<262:31:22,  5.67s/it]
 26%|██▌       | 58414/225000 [120:53:05<263:21:33,  5.69s/it]
 26%|██▌       | 58415/225000 [120:53:19<374:26:44,  8.09s/it]
 26%|██▌       | 58416/225000 [120:53:25<339:12:30,  7.33s/it]
 26%|██▌       | 58417/225000 [120:53:31<319:36:20,  6.91s/it]
 26%|██▌       | 58418/225000 [120:53:35<286:42:56,  6.20s/it]
 26%|██▌       | 58419/225000 [120:53:39<260:16:48,  5.62s/it]
 26%|██▌       | 58420/225000 [120:53:51<343:55:30,  7.43s/it]
 26%|██▌       | 58421/225000 [120:53:56<307:38:20,  6.65s/it]
 26%|██▌       | 58422/225000 [120:54:01<285:56:53,  6.18s/it]
 26%|██▌       | 58423/225000 [120:54:07<278:37:23,  6.02s/it]
 26%|██▌       | 58424/225000 [120:54:12<265:43:05,  5.74s/it]
 26%|██▌       | 58425/225000 [120:54:22<327:30:33,  7.08s/it]
 26%|██▌       | 58426/225000 [120:54:29<323:54:32,  7.00s/it]
 26%|██▌       | 58427/225000 [120:54:33<284:24:21,  6.15s/it]
 26%|██▌       | 58428/225000 [120:54:43<339:28:36,  7.34s/it]
 26%|██▌       | 58429/225000 [120:54:49<316:06:01,  6.83s/it]
 26%|██▌       | 58430/225000 [120:54:56<329:35:55,  7.12s/it]
 26%|██▌       | 58431/225000 [120:55:02<308:53:42,  6.68s/it]
 26%|██▌       | 58432/225000 [120:55:09<313:10:07,  6.77s/it]
 26%|██▌       | 58433/225000 [120:55:17<334:54:19,  7.24s/it]
 26%|██▌       | 58434/225000 [120:55:21<290:16:38,  6.27s/it]
 26%|██▌       | 58435/225000 [120:55:26<263:03:20,  5.69s/it]
 26%|██▌       | 58436/225000 [120:55:37<334:38:02,  7.23s/it]
 26%|██▌       | 58437/225000 [120:55:40<287:52:13,  6.22s/it]
 26%|██▌       | 58438/225000 [120:55:46<271:15:34,  5.86s/it]
 26%|██▌       | 58439/225000 [120:55:57<352:10:31,  7.61s/it]
 26%|██▌       | 58440/225000 [120:56:01<302:32:35,  6.54s/it]
 26%|██▌       | 58441/225000 [120:56:05<267:25:58,  5.78s/it]
 26%|██▌       | 58442/225000 [120:56:13<292:43:46,  6.33s/it]
 26%|██▌       | 58443/225000 [120:56:21<324:38:18,  7.02s/it]
 26%|██▌       | 58444/225000 [120:56:26<283:59:01,  6.14s/it]
 26%|██▌       | 58445/225000 [120:56:34<320:12:10,  6.92s/it]
 26%|██▌       | 58446/225000 [120:56:43<350:34:16,  7.58s/it]
 26%|██▌       | 58447/225000 [120:56:51<346:51:21,  7.50s/it]
 26%|██▌       | 58448/225000 [120:56:55<295:06:24,  6.38s/it]
 26%|██▌       | 58449/225000 [120:57:04<331:55:57,  7.17s/it]
 26%|██▌       | 58450/225000 [120:57:08<291:07:01,  6.29s/it]
                                                              
{'loss': '0.7559', 'grad_norm': '0.7689', 'learning_rate': '4.4e-05', 'epoch': '0.2598', 'num_input_tokens_seen': 201535080, 'train_runtime': '4.354e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58450/225000 [120:57:08<291:07:01,  6.29s/it]
 26%|██▌       | 58451/225000 [120:57:13<274:53:57,  5.94s/it]
 26%|██▌       | 58452/225000 [120:57:18<266:51:15,  5.77s/it]
 26%|██▌       | 58453/225000 [120:57:24<262:39:31,  5.68s/it]
 26%|██▌       | 58454/225000 [120:57:29<259:07:27,  5.60s/it]
 26%|██▌       | 58455/225000 [120:57:34<251:39:47,  5.44s/it]
 26%|██▌       | 58456/225000 [120:57:48<370:59:10,  8.02s/it]
 26%|██▌       | 58457/225000 [120:57:53<321:24:35,  6.95s/it]
 26%|██▌       | 58458/225000 [120:57:59<315:33:40,  6.82s/it]
 26%|██▌       | 58459/225000 [120:58:08<344:43:17,  7.45s/it]
 26%|██▌       | 58460/225000 [120:58:12<298:40:48,  6.46s/it]
 26%|██▌       | 58461/225000 [120:58:16<259:19:38,  5.61s/it]
 26%|██▌       | 58462/225000 [120:58:25<311:53:43,  6.74s/it]
 26%|██▌       | 58463/225000 [120:58:32<306:04:36,  6.62s/it]
 26%|██▌       | 58464/225000 [120:58:36<275:53:39,  5.96s/it]
 26%|██▌       | 58465/225000 [120:58:41<257:16:45,  5.56s/it]
 26%|██▌       | 58466/225000 [120:58:46<248:25:55,  5.37s/it]
 26%|██▌       | 58467/225000 [120:58:53<281:44:06,  6.09s/it]
 26%|██▌       | 58468/225000 [120:59:05<351:39:35,  7.60s/it]
 26%|██▌       | 58469/225000 [120:59:09<310:33:16,  6.71s/it]
 26%|██▌       | 58470/225000 [120:59:14<283:16:53,  6.12s/it]
 26%|██▌       | 58471/225000 [120:59:21<294:22:14,  6.36s/it]
 26%|██▌       | 58472/225000 [120:59:30<329:38:59,  7.13s/it]
 26%|██▌       | 58473/225000 [120:59:34<292:29:20,  6.32s/it]
 26%|██▌       | 58474/225000 [120:59:39<267:30:27,  5.78s/it]
 26%|██▌       | 58475/225000 [120:59:52<370:58:40,  8.02s/it]
 26%|██▌       | 58476/225000 [120:59:57<331:48:40,  7.17s/it]
 26%|██▌       | 58477/225000 [121:00:02<298:37:06,  6.46s/it]
 26%|██▌       | 58478/225000 [121:00:07<276:37:03,  5.98s/it]
 26%|██▌       | 58479/225000 [121:00:11<258:53:47,  5.60s/it]
 26%|██▌       | 58480/225000 [121:00:18<273:57:57,  5.92s/it]
 26%|██▌       | 58481/225000 [121:00:30<355:03:51,  7.68s/it]
 26%|██▌       | 58482/225000 [121:00:35<315:26:36,  6.82s/it]
 26%|██▌       | 58483/225000 [121:00:39<285:47:10,  6.18s/it]
 26%|██▌       | 58484/225000 [121:00:44<264:12:19,  5.71s/it]
 26%|██▌       | 58485/225000 [121:00:51<276:38:50,  5.98s/it]
 26%|██▌       | 58486/225000 [121:01:01<339:10:40,  7.33s/it]
 26%|██▌       | 58487/225000 [121:01:08<333:59:57,  7.22s/it]
 26%|██▌       | 58488/225000 [121:01:13<295:55:38,  6.40s/it]
 26%|██▌       | 58489/225000 [121:01:23<354:30:41,  7.66s/it]
 26%|██▌       | 58490/225000 [121:01:29<322:40:29,  6.98s/it]
 26%|██▌       | 58491/225000 [121:01:34<303:31:16,  6.56s/it]
 26%|██▌       | 58492/225000 [121:01:39<272:26:57,  5.89s/it]
 26%|██▌       | 58493/225000 [121:01:48<327:20:51,  7.08s/it]
 26%|██▌       | 58494/225000 [121:01:54<307:22:26,  6.65s/it]
 26%|██▌       | 58495/225000 [121:02:02<324:52:14,  7.02s/it]
 26%|██▌       | 58496/225000 [121:02:10<333:41:58,  7.21s/it]
 26%|██▌       | 58497/225000 [121:02:14<301:44:30,  6.52s/it]
 26%|██▌       | 58498/225000 [121:02:19<275:49:46,  5.96s/it]
 26%|██▌       | 58499/225000 [121:02:24<258:27:55,  5.59s/it]
 26%|██▌       | 58500/225000 [121:02:29<258:55:57,  5.60s/it]
                                                              
{'loss': '0.7293', 'grad_norm': '0.7534', 'learning_rate': '4.399e-05', 'epoch': '0.26', 'num_input_tokens_seen': 201683248, 'train_runtime': '4.357e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58500/225000 [121:02:29<258:55:57,  5.60s/it]
 26%|██▌       | 58501/225000 [121:02:41<347:40:06,  7.52s/it]
 26%|██▌       | 58502/225000 [121:02:45<298:28:26,  6.45s/it]
 26%|██▌       | 58503/225000 [121:02:54<321:40:35,  6.96s/it]
 26%|██▌       | 58504/225000 [121:03:00<318:20:27,  6.88s/it]
 26%|██▌       | 58505/225000 [121:03:04<280:17:33,  6.06s/it]
 26%|██▌       | 58506/225000 [121:03:15<340:41:38,  7.37s/it]
 26%|██▌       | 58507/225000 [121:03:20<308:59:36,  6.68s/it]
 26%|██▌       | 58508/225000 [121:03:25<289:27:48,  6.26s/it]
 26%|██▌       | 58509/225000 [121:03:30<267:14:45,  5.78s/it]
 26%|██▌       | 58510/225000 [121:03:42<350:04:47,  7.57s/it]
 26%|██▌       | 58511/225000 [121:03:45<298:01:18,  6.44s/it]
 26%|██▌       | 58512/225000 [121:03:54<327:41:23,  7.09s/it]
 26%|██▌       | 58513/225000 [121:04:00<310:39:46,  6.72s/it]
 26%|██▌       | 58514/225000 [121:04:06<296:25:17,  6.41s/it]
 26%|██▌       | 58515/225000 [121:04:12<303:06:32,  6.55s/it]
 26%|██▌       | 58516/225000 [121:04:17<274:56:04,  5.95s/it]
 26%|██▌       | 58517/225000 [121:04:21<253:05:54,  5.47s/it]
 26%|██▌       | 58518/225000 [121:04:26<246:22:55,  5.33s/it]
 26%|██▌       | 58519/225000 [121:04:38<341:14:51,  7.38s/it]
 26%|██▌       | 58520/225000 [121:04:44<310:11:16,  6.71s/it]
 26%|██▌       | 58521/225000 [121:04:49<287:05:05,  6.21s/it]
 26%|██▌       | 58522/225000 [121:04:54<271:41:16,  5.88s/it]
 26%|██▌       | 58523/225000 [121:04:59<269:03:41,  5.82s/it]
 26%|██▌       | 58524/225000 [121:05:09<324:07:43,  7.01s/it]
 26%|██▌       | 58525/225000 [121:05:19<364:22:22,  7.88s/it]
 26%|██▌       | 58526/225000 [121:05:24<322:05:48,  6.97s/it]
 26%|██▌       | 58527/225000 [121:05:29<297:45:48,  6.44s/it]
 26%|██▌       | 58528/225000 [121:05:34<281:43:45,  6.09s/it]
 26%|██▌       | 58529/225000 [121:05:40<268:41:05,  5.81s/it]
 26%|██▌       | 58530/225000 [121:05:50<325:40:55,  7.04s/it]
 26%|██▌       | 58531/225000 [121:05:58<346:12:20,  7.49s/it]
 26%|██▌       | 58532/225000 [121:06:03<306:27:07,  6.63s/it]
 26%|██▌       | 58533/225000 [121:06:08<289:47:43,  6.27s/it]
 26%|██▌       | 58534/225000 [121:06:14<281:35:56,  6.09s/it]
 26%|██▌       | 58535/225000 [121:06:20<277:14:18,  6.00s/it]
 26%|██▌       | 58536/225000 [121:06:25<273:18:18,  5.91s/it]
 26%|██▌       | 58537/225000 [121:06:38<365:54:03,  7.91s/it]
 26%|██▌       | 58538/225000 [121:06:46<364:38:12,  7.89s/it]
 26%|██▌       | 58539/225000 [121:06:50<310:35:10,  6.72s/it]
 26%|██▌       | 58540/225000 [121:06:57<321:12:43,  6.95s/it]
 26%|██▌       | 58541/225000 [121:07:02<298:35:09,  6.46s/it]
 26%|██▌       | 58542/225000 [121:07:08<280:58:48,  6.08s/it]
 26%|██▌       | 58543/225000 [121:07:13<264:49:36,  5.73s/it]
 26%|██▌       | 58544/225000 [121:07:17<250:12:55,  5.41s/it]
 26%|██▌       | 58545/225000 [121:07:25<287:16:44,  6.21s/it]
 26%|██▌       | 58546/225000 [121:07:38<371:19:43,  8.03s/it]
 26%|██▌       | 58547/225000 [121:07:42<314:15:48,  6.80s/it]
 26%|██▌       | 58548/225000 [121:07:51<349:04:21,  7.55s/it]
 26%|██▌       | 58549/225000 [121:07:55<308:44:41,  6.68s/it]
 26%|██▌       | 58550/225000 [121:08:00<282:06:05,  6.10s/it]
                                                              
{'loss': '0.7367', 'grad_norm': '0.6806', 'learning_rate': '4.398e-05', 'epoch': '0.2602', 'num_input_tokens_seen': 201833344, 'train_runtime': '4.361e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58550/225000 [121:08:00<282:06:05,  6.10s/it]
 26%|██▌       | 58551/225000 [121:08:05<264:53:46,  5.73s/it]
 26%|██▌       | 58552/225000 [121:08:10<253:40:56,  5.49s/it]
 26%|██▌       | 58553/225000 [121:08:22<343:52:06,  7.44s/it]
 26%|██▌       | 58554/225000 [121:08:30<350:46:17,  7.59s/it]
 26%|██▌       | 58555/225000 [121:08:34<299:01:57,  6.47s/it]
 26%|██▌       | 58556/225000 [121:08:43<335:09:21,  7.25s/it]
 26%|██▌       | 58557/225000 [121:08:48<300:22:48,  6.50s/it]
 26%|██▌       | 58558/225000 [121:08:53<279:25:13,  6.04s/it]
 26%|██▌       | 58559/225000 [121:08:58<271:43:20,  5.88s/it]
 26%|██▌       | 58560/225000 [121:09:03<260:45:15,  5.64s/it]
 26%|██▌       | 58561/225000 [121:09:09<257:52:25,  5.58s/it]
 26%|██▌       | 58562/225000 [121:09:22<360:10:29,  7.79s/it]
 26%|██▌       | 58563/225000 [121:09:26<318:50:17,  6.90s/it]
 26%|██▌       | 58564/225000 [121:09:34<335:30:12,  7.26s/it]
 26%|██▌       | 58565/225000 [121:09:39<292:37:10,  6.33s/it]
 26%|██▌       | 58566/225000 [121:09:43<265:13:03,  5.74s/it]
 26%|██▌       | 58567/225000 [121:09:54<338:27:39,  7.32s/it]
 26%|██▌       | 58568/225000 [121:10:02<347:10:47,  7.51s/it]
 26%|██▌       | 58569/225000 [121:10:07<307:02:18,  6.64s/it]
 26%|██▌       | 58570/225000 [121:10:13<297:23:53,  6.43s/it]
 26%|██▌       | 58571/225000 [121:10:23<350:45:39,  7.59s/it]
 26%|██▌       | 58572/225000 [121:10:27<301:49:25,  6.53s/it]
 26%|██▌       | 58573/225000 [121:10:35<328:11:00,  7.10s/it]
 26%|██▌       | 58574/225000 [121:10:40<297:08:20,  6.43s/it]
 26%|██▌       | 58575/225000 [121:10:45<280:07:13,  6.06s/it]
 26%|██▌       | 58576/225000 [121:10:51<268:50:19,  5.82s/it]
 26%|██▌       | 58577/225000 [121:11:01<325:51:59,  7.05s/it]
 26%|██▌       | 58578/225000 [121:11:10<357:06:39,  7.72s/it]
 26%|██▌       | 58579/225000 [121:11:16<340:12:26,  7.36s/it]
 26%|██▌       | 58580/225000 [121:11:21<307:19:03,  6.65s/it]
 26%|██▌       | 58581/225000 [121:11:26<277:56:01,  6.01s/it]
 26%|██▌       | 58582/225000 [121:11:31<260:09:52,  5.63s/it]
 26%|██▌       | 58583/225000 [121:11:36<254:15:49,  5.50s/it]
 26%|██▌       | 58584/225000 [121:11:49<360:24:29,  7.80s/it]
 26%|██▌       | 58585/225000 [121:11:53<315:12:02,  6.82s/it]
 26%|██▌       | 58586/225000 [121:12:02<344:30:48,  7.45s/it]
 26%|██▌       | 58587/225000 [121:12:07<300:39:43,  6.50s/it]
 26%|██▌       | 58588/225000 [121:12:11<273:52:48,  5.92s/it]
 26%|██▌       | 58589/225000 [121:12:16<262:17:21,  5.67s/it]
 26%|██▌       | 58590/225000 [121:12:26<313:44:52,  6.79s/it]
 26%|██▌       | 58591/225000 [121:12:36<356:06:44,  7.70s/it]
 26%|██▌       | 58592/225000 [121:12:39<301:20:44,  6.52s/it]
 26%|██▌       | 58593/225000 [121:12:49<338:24:59,  7.32s/it]
 26%|██▌       | 58594/225000 [121:12:53<294:16:50,  6.37s/it]
 26%|██▌       | 58595/225000 [121:12:57<264:41:24,  5.73s/it]
 26%|██▌       | 58596/225000 [121:13:07<323:48:44,  7.01s/it]
 26%|██▌       | 58597/225000 [121:13:14<319:36:31,  6.91s/it]
 26%|██▌       | 58598/225000 [121:13:19<295:29:26,  6.39s/it]
 26%|██▌       | 58599/225000 [121:13:28<339:20:07,  7.34s/it]
 26%|██▌       | 58600/225000 [121:13:34<312:12:56,  6.75s/it]
                                                              
{'loss': '0.7476', 'grad_norm': '0.7481', 'learning_rate': '4.396e-05', 'epoch': '0.2604', 'num_input_tokens_seen': 201982952, 'train_runtime': '4.364e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58600/225000 [121:13:34<312:12:56,  6.75s/it]
 26%|██▌       | 58601/225000 [121:13:39<298:37:46,  6.46s/it]
 26%|██▌       | 58602/225000 [121:13:45<287:31:47,  6.22s/it]
 26%|██▌       | 58603/225000 [121:13:51<284:25:43,  6.15s/it]
 26%|██▌       | 58604/225000 [121:13:57<283:45:50,  6.14s/it]
 26%|██▌       | 58605/225000 [121:14:02<269:05:55,  5.82s/it]
 26%|██▌       | 58606/225000 [121:14:08<267:54:01,  5.80s/it]
 26%|██▌       | 58607/225000 [121:14:21<360:53:25,  7.81s/it]
 26%|██▌       | 58608/225000 [121:14:29<371:41:15,  8.04s/it]
 26%|██▌       | 58609/225000 [121:14:34<330:44:14,  7.16s/it]
 26%|██▌       | 58610/225000 [121:14:44<368:12:11,  7.97s/it]
 26%|██▌       | 58611/225000 [121:14:48<311:47:49,  6.75s/it]
 26%|██▌       | 58612/225000 [121:14:58<354:30:42,  7.67s/it]
 26%|██▌       | 58613/225000 [121:15:02<302:54:05,  6.55s/it]
 26%|██▌       | 58614/225000 [121:15:09<305:42:18,  6.61s/it]
 26%|██▌       | 58615/225000 [121:15:18<340:42:56,  7.37s/it]
 26%|██▌       | 58616/225000 [121:15:22<302:08:46,  6.54s/it]
 26%|██▌       | 58617/225000 [121:15:27<283:54:37,  6.14s/it]
 26%|██▌       | 58618/225000 [121:15:33<269:47:26,  5.84s/it]
 26%|██▌       | 58619/225000 [121:15:38<261:34:16,  5.66s/it]
 26%|██▌       | 58620/225000 [121:15:51<365:31:40,  7.91s/it]
 26%|██▌       | 58621/225000 [121:15:55<317:42:06,  6.87s/it]
 26%|██▌       | 58622/225000 [121:16:00<284:46:04,  6.16s/it]
 26%|██▌       | 58623/225000 [121:16:10<334:14:07,  7.23s/it]
 26%|██▌       | 58624/225000 [121:16:15<303:35:58,  6.57s/it]
 26%|██▌       | 58625/225000 [121:16:20<288:08:16,  6.23s/it]
 26%|██▌       | 58626/225000 [121:16:28<309:46:24,  6.70s/it]
 26%|██▌       | 58627/225000 [121:16:37<340:23:25,  7.37s/it]
 26%|██▌       | 58628/225000 [121:16:42<305:27:52,  6.61s/it]
 26%|██▌       | 58629/225000 [121:16:52<357:13:45,  7.73s/it]
 26%|██▌       | 58630/225000 [121:16:56<309:21:15,  6.69s/it]
 26%|██▌       | 58631/225000 [121:17:01<275:58:30,  5.97s/it]
 26%|██▌       | 58632/225000 [121:17:07<287:17:01,  6.22s/it]
 26%|██▌       | 58633/225000 [121:17:18<347:51:58,  7.53s/it]
 26%|██▌       | 58634/225000 [121:17:22<298:50:57,  6.47s/it]
 26%|██▌       | 58635/225000 [121:17:31<334:20:25,  7.23s/it]
 26%|██▌       | 58636/225000 [121:17:36<304:28:48,  6.59s/it]
 26%|██▌       | 58637/225000 [121:17:41<283:33:54,  6.14s/it]
 26%|██▌       | 58638/225000 [121:17:46<267:40:52,  5.79s/it]
 26%|██▌       | 58639/225000 [121:17:54<294:14:48,  6.37s/it]
 26%|██▌       | 58640/225000 [121:18:05<360:44:28,  7.81s/it]
 26%|██▌       | 58641/225000 [121:18:10<316:49:29,  6.86s/it]
 26%|██▌       | 58642/225000 [121:18:15<289:11:33,  6.26s/it]
 26%|██▌       | 58643/225000 [121:18:19<265:29:39,  5.75s/it]
 26%|██▌       | 58644/225000 [121:18:25<273:31:42,  5.92s/it]
 26%|██▌       | 58645/225000 [121:18:38<360:20:28,  7.80s/it]
 26%|██▌       | 58646/225000 [121:18:43<320:22:48,  6.93s/it]
 26%|██▌       | 58647/225000 [121:18:48<296:27:45,  6.42s/it]
 26%|██▌       | 58648/225000 [121:18:53<279:38:49,  6.05s/it]
 26%|██▌       | 58649/225000 [121:18:58<267:09:02,  5.78s/it]
 26%|██▌       | 58650/225000 [121:19:04<269:10:42,  5.83s/it]
                                                              
{'loss': '0.7318', 'grad_norm': '1.31', 'learning_rate': '4.395e-05', 'epoch': '0.2607', 'num_input_tokens_seen': 202130736, 'train_runtime': '4.367e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58650/225000 [121:19:04<269:10:42,  5.83s/it]
 26%|██▌       | 58651/225000 [121:19:17<367:57:30,  7.96s/it]
 26%|██▌       | 58652/225000 [121:19:22<326:23:57,  7.06s/it]
 26%|██▌       | 58653/225000 [121:19:30<341:23:20,  7.39s/it]
 26%|██▌       | 58654/225000 [121:19:35<305:15:36,  6.61s/it]
 26%|██▌       | 58655/225000 [121:19:45<348:41:54,  7.55s/it]
 26%|██▌       | 58656/225000 [121:19:49<299:04:03,  6.47s/it]
 26%|██▌       | 58657/225000 [121:19:58<342:32:53,  7.41s/it]
 26%|██▌       | 58658/225000 [121:20:04<319:29:03,  6.91s/it]
 26%|██▌       | 58659/225000 [121:20:10<303:25:09,  6.57s/it]
 26%|██▌       | 58660/225000 [121:20:15<290:11:20,  6.28s/it]
 26%|██▌       | 58661/225000 [121:20:21<277:21:16,  6.00s/it]
 26%|██▌       | 58662/225000 [121:20:29<309:46:49,  6.70s/it]
 26%|██▌       | 58663/225000 [121:20:38<344:20:29,  7.45s/it]
 26%|██▌       | 58664/225000 [121:20:44<319:23:14,  6.91s/it]
 26%|██▌       | 58665/225000 [121:20:49<290:32:59,  6.29s/it]
 26%|██▌       | 58666/225000 [121:20:54<273:06:52,  5.91s/it]
 26%|██▌       | 58667/225000 [121:20:59<261:57:38,  5.67s/it]
 26%|██▌       | 58668/225000 [121:21:04<261:20:48,  5.66s/it]
 26%|██▌       | 58669/225000 [121:21:17<354:15:22,  7.67s/it]
 26%|██▌       | 58670/225000 [121:21:27<385:20:57,  8.34s/it]
 26%|██▌       | 58671/225000 [121:21:31<324:25:29,  7.02s/it]
 26%|██▌       | 58672/225000 [121:21:36<300:15:08,  6.50s/it]
 26%|██▌       | 58673/225000 [121:21:42<290:24:59,  6.29s/it]
 26%|██▌       | 58674/225000 [121:21:47<280:56:15,  6.08s/it]
 26%|██▌       | 58675/225000 [121:21:54<284:54:37,  6.17s/it]
 26%|██▌       | 58676/225000 [121:22:02<317:26:18,  6.87s/it]
 26%|██▌       | 58677/225000 [121:22:08<301:07:51,  6.52s/it]
 26%|██▌       | 58678/225000 [121:22:14<289:30:07,  6.27s/it]
 26%|██▌       | 58679/225000 [121:22:19<277:55:59,  6.02s/it]
 26%|██▌       | 58680/225000 [121:22:25<273:03:35,  5.91s/it]
 26%|██▌       | 58681/225000 [121:22:32<297:35:13,  6.44s/it]
 26%|██▌       | 58682/225000 [121:22:38<286:07:30,  6.19s/it]
 26%|██▌       | 58683/225000 [121:22:47<331:32:36,  7.18s/it]
 26%|██▌       | 58684/225000 [121:22:56<347:22:18,  7.52s/it]
 26%|██▌       | 58685/225000 [121:23:04<356:03:22,  7.71s/it]
 26%|██▌       | 58686/225000 [121:23:07<295:33:25,  6.40s/it]
 26%|██▌       | 58687/225000 [121:23:15<313:19:59,  6.78s/it]
 26%|██▌       | 58688/225000 [121:23:23<330:22:10,  7.15s/it]
 26%|██▌       | 58689/225000 [121:23:29<321:04:06,  6.95s/it]
 26%|██▌       | 58690/225000 [121:23:34<287:37:27,  6.23s/it]
 26%|██▌       | 58691/225000 [121:23:39<264:34:44,  5.73s/it]
 26%|██▌       | 58692/225000 [121:23:50<348:38:32,  7.55s/it]
 26%|██▌       | 58693/225000 [121:23:54<291:16:01,  6.30s/it]
 26%|██▌       | 58694/225000 [121:24:02<316:43:01,  6.86s/it]
 26%|██▌       | 58695/225000 [121:24:06<283:12:53,  6.13s/it]
 26%|██▌       | 58696/225000 [121:24:14<304:11:11,  6.58s/it]
 26%|██▌       | 58697/225000 [121:24:23<341:12:32,  7.39s/it]
 26%|██▌       | 58698/225000 [121:24:27<288:42:48,  6.25s/it]
 26%|██▌       | 58699/225000 [121:24:36<334:05:41,  7.23s/it]
 26%|██▌       | 58700/225000 [121:24:41<300:30:11,  6.51s/it]
                                                              
{'loss': '0.7272', 'grad_norm': '0.894', 'learning_rate': '4.394e-05', 'epoch': '0.2609', 'num_input_tokens_seen': 202279320, 'train_runtime': '4.371e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58700/225000 [121:24:41<300:30:11,  6.51s/it]
 26%|██▌       | 58701/225000 [121:24:46<280:21:06,  6.07s/it]
 26%|██▌       | 58702/225000 [121:24:56<331:00:55,  7.17s/it]
 26%|██▌       | 58703/225000 [121:25:04<337:53:22,  7.31s/it]
 26%|██▌       | 58704/225000 [121:25:11<345:48:50,  7.49s/it]
 26%|██▌       | 58705/225000 [121:25:17<312:40:10,  6.77s/it]
 26%|██▌       | 58706/225000 [121:25:21<281:01:49,  6.08s/it]
 26%|██▌       | 58707/225000 [121:25:26<263:20:10,  5.70s/it]
 26%|██▌       | 58708/225000 [121:25:33<279:23:17,  6.05s/it]
 26%|██▌       | 58709/225000 [121:25:44<349:28:51,  7.57s/it]
 26%|██▌       | 58710/225000 [121:25:48<296:19:03,  6.41s/it]
 26%|██▌       | 58711/225000 [121:25:57<336:55:36,  7.29s/it]
 26%|██▌       | 58712/225000 [121:26:02<300:21:16,  6.50s/it]
 26%|██▌       | 58713/225000 [121:26:07<284:12:40,  6.15s/it]
 26%|██▌       | 58714/225000 [121:26:16<320:21:39,  6.94s/it]
 26%|██▌       | 58715/225000 [121:26:23<321:38:10,  6.96s/it]
 26%|██▌       | 58716/225000 [121:26:31<346:20:56,  7.50s/it]
 26%|██▌       | 58717/225000 [121:26:38<339:19:13,  7.35s/it]
 26%|██▌       | 58718/225000 [121:26:42<291:01:45,  6.30s/it]
 26%|██▌       | 58719/225000 [121:26:52<342:31:58,  7.42s/it]
 26%|██▌       | 58720/225000 [121:26:57<301:53:29,  6.54s/it]
 26%|██▌       | 58721/225000 [121:27:01<270:44:29,  5.86s/it]
 26%|██▌       | 58722/225000 [121:27:12<343:26:21,  7.44s/it]
 26%|██▌       | 58723/225000 [121:27:18<314:41:07,  6.81s/it]
 26%|██▌       | 58724/225000 [121:27:24<307:49:25,  6.66s/it]
 26%|██▌       | 58725/225000 [121:27:33<349:30:13,  7.57s/it]
 26%|██▌       | 58726/225000 [121:27:37<297:37:44,  6.44s/it]
 26%|██▌       | 58727/225000 [121:27:42<278:51:54,  6.04s/it]
 26%|██▌       | 58728/225000 [121:27:52<322:41:10,  6.99s/it]
 26%|██▌       | 58729/225000 [121:27:56<291:12:44,  6.31s/it]
 26%|██▌       | 58730/225000 [121:28:06<338:09:04,  7.32s/it]
 26%|██▌       | 58731/225000 [121:28:11<303:56:25,  6.58s/it]
 26%|██▌       | 58732/225000 [121:28:15<275:01:07,  5.95s/it]
 26%|██▌       | 58733/225000 [121:28:20<253:07:33,  5.48s/it]
 26%|██▌       | 58734/225000 [121:28:25<252:48:06,  5.47s/it]
 26%|██▌       | 58735/225000 [121:28:39<363:31:47,  7.87s/it]
 26%|██▌       | 58736/225000 [121:28:44<329:22:11,  7.13s/it]
 26%|██▌       | 58737/225000 [121:28:48<290:47:46,  6.30s/it]
 26%|██▌       | 58738/225000 [121:28:56<305:55:11,  6.62s/it]
 26%|██▌       | 58739/225000 [121:29:04<327:21:20,  7.09s/it]
 26%|██▌       | 58740/225000 [121:29:08<286:19:01,  6.20s/it]
 26%|██▌       | 58741/225000 [121:29:19<352:23:28,  7.63s/it]
 26%|██▌       | 58742/225000 [121:29:26<349:21:17,  7.56s/it]
 26%|██▌       | 58743/225000 [121:29:31<309:32:40,  6.70s/it]
 26%|██▌       | 58744/225000 [121:29:37<301:10:45,  6.52s/it]
 26%|██▌       | 58745/225000 [121:29:43<283:14:20,  6.13s/it]
 26%|██▌       | 58746/225000 [121:29:48<269:14:51,  5.83s/it]
 26%|██▌       | 58747/225000 [121:29:53<263:22:42,  5.70s/it]
 26%|██▌       | 58748/225000 [121:30:04<337:05:34,  7.30s/it]
 26%|██▌       | 58749/225000 [121:30:14<379:24:13,  8.22s/it]
 26%|██▌       | 58750/225000 [121:30:19<326:59:59,  7.08s/it]
                                                              
{'loss': '0.7396', 'grad_norm': '0.8758', 'learning_rate': '4.393e-05', 'epoch': '0.2611', 'num_input_tokens_seen': 202429688, 'train_runtime': '4.374e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58750/225000 [121:30:19<326:59:59,  7.08s/it]
 26%|██▌       | 58751/225000 [121:30:24<306:54:48,  6.65s/it]
 26%|██▌       | 58752/225000 [121:30:32<319:16:45,  6.91s/it]
 26%|██▌       | 58753/225000 [121:30:37<294:45:55,  6.38s/it]
 26%|██▌       | 58754/225000 [121:30:42<270:54:20,  5.87s/it]
 26%|██▌       | 58755/225000 [121:30:51<318:52:38,  6.91s/it]
 26%|██▌       | 58756/225000 [121:30:59<332:22:23,  7.20s/it]
 26%|██▌       | 58757/225000 [121:31:04<307:12:14,  6.65s/it]
 26%|██▌       | 58758/225000 [121:31:09<277:55:51,  6.02s/it]
 26%|██▌       | 58759/225000 [121:31:16<296:58:38,  6.43s/it]
 26%|██▌       | 58760/225000 [121:31:25<327:51:54,  7.10s/it]
 26%|██▌       | 58761/225000 [121:31:30<293:12:45,  6.35s/it]
 26%|██▌       | 58762/225000 [121:31:34<272:18:26,  5.90s/it]
 26%|██▌       | 58763/225000 [121:31:44<320:51:42,  6.95s/it]
 26%|██▌       | 58764/225000 [121:31:54<361:53:29,  7.84s/it]
 26%|██▌       | 58765/225000 [121:31:58<308:54:03,  6.69s/it]
 26%|██▌       | 58766/225000 [121:32:02<271:12:03,  5.87s/it]
 26%|██▌       | 58767/225000 [121:32:09<294:57:17,  6.39s/it]
 26%|██▌       | 58768/225000 [121:32:20<350:24:19,  7.59s/it]
 26%|██▌       | 58769/225000 [121:32:23<294:00:18,  6.37s/it]
 26%|██▌       | 58770/225000 [121:32:33<337:59:49,  7.32s/it]
 26%|██▌       | 58771/225000 [121:32:38<305:37:42,  6.62s/it]
 26%|██▌       | 58772/225000 [121:32:43<282:19:34,  6.11s/it]
 26%|██▌       | 58773/225000 [121:32:53<342:55:46,  7.43s/it]
 26%|██▌       | 58774/225000 [121:33:01<342:07:59,  7.41s/it]
 26%|██▌       | 58775/225000 [121:33:06<308:58:31,  6.69s/it]
 26%|██▌       | 58776/225000 [121:33:11<295:23:31,  6.40s/it]
 26%|██▌       | 58777/225000 [121:33:16<278:01:20,  6.02s/it]
 26%|██▌       | 58778/225000 [121:33:22<269:05:06,  5.83s/it]
 26%|██▌       | 58779/225000 [121:33:27<261:43:01,  5.67s/it]
 26%|██▌       | 58780/225000 [121:33:33<265:13:41,  5.74s/it]
 26%|██▌       | 58781/225000 [121:33:39<261:38:26,  5.67s/it]
 26%|██▌       | 58782/225000 [121:33:51<360:43:10,  7.81s/it]
 26%|██▌       | 58783/225000 [121:33:59<362:29:08,  7.85s/it]
 26%|██▌       | 58784/225000 [121:34:04<320:01:09,  6.93s/it]
 26%|██▌       | 58785/225000 [121:34:12<340:22:31,  7.37s/it]
 26%|██▌       | 58786/225000 [121:34:20<343:04:36,  7.43s/it]
 26%|██▌       | 58787/225000 [121:34:25<306:39:08,  6.64s/it]
 26%|██▌       | 58788/225000 [121:34:32<310:55:30,  6.73s/it]
 26%|██▌       | 58789/225000 [121:34:41<347:58:41,  7.54s/it]
 26%|██▌       | 58790/225000 [121:34:48<341:37:12,  7.40s/it]
 26%|██▌       | 58791/225000 [121:34:53<305:54:52,  6.63s/it]
 26%|██▌       | 58792/225000 [121:34:58<282:00:04,  6.11s/it]
 26%|██▌       | 58793/225000 [121:35:03<272:50:02,  5.91s/it]
 26%|██▌       | 58794/225000 [121:35:13<317:24:41,  6.88s/it]
 26%|██▌       | 58795/225000 [121:35:23<369:44:41,  8.01s/it]
 26%|██▌       | 58796/225000 [121:35:28<318:31:41,  6.90s/it]
 26%|██▌       | 58797/225000 [121:35:38<363:19:46,  7.87s/it]
 26%|██▌       | 58798/225000 [121:35:41<301:42:30,  6.54s/it]
 26%|██▌       | 58799/225000 [121:35:49<327:41:39,  7.10s/it]
 26%|██▌       | 58800/225000 [121:35:54<288:40:32,  6.25s/it]
                                                              
{'loss': '0.7336', 'grad_norm': '0.7843', 'learning_rate': '4.392e-05', 'epoch': '0.2613', 'num_input_tokens_seen': 202577896, 'train_runtime': '4.378e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58800/225000 [121:35:54<288:40:32,  6.25s/it]
 26%|██▌       | 58801/225000 [121:35:58<255:48:42,  5.54s/it]
 26%|██▌       | 58802/225000 [121:36:11<358:13:30,  7.76s/it]
 26%|██▌       | 58803/225000 [121:36:14<301:41:52,  6.54s/it]
 26%|██▌       | 58804/225000 [121:36:23<334:28:31,  7.25s/it]
 26%|██▌       | 58805/225000 [121:36:27<284:09:17,  6.16s/it]
 26%|██▌       | 58806/225000 [121:36:31<257:35:15,  5.58s/it]
 26%|██▌       | 58807/225000 [121:36:37<262:05:55,  5.68s/it]
 26%|██▌       | 58808/225000 [121:36:43<264:35:54,  5.73s/it]
 26%|██▌       | 58809/225000 [121:36:56<367:47:09,  7.97s/it]
 26%|██▌       | 58810/225000 [121:37:01<322:16:47,  6.98s/it]
 26%|██▌       | 58811/225000 [121:37:09<336:21:05,  7.29s/it]
 26%|██▌       | 58812/225000 [121:37:17<345:52:55,  7.49s/it]
 26%|██▌       | 58813/225000 [121:37:23<337:31:03,  7.31s/it]
 26%|██▌       | 58814/225000 [121:37:29<309:45:48,  6.71s/it]
 26%|██▌       | 58815/225000 [121:37:34<289:35:14,  6.27s/it]
 26%|██▌       | 58816/225000 [121:37:39<277:16:01,  6.01s/it]
 26%|██▌       | 58817/225000 [121:37:44<263:32:00,  5.71s/it]
 26%|██▌       | 58818/225000 [121:37:56<349:04:39,  7.56s/it]
 26%|██▌       | 58819/225000 [121:38:02<326:58:06,  7.08s/it]
 26%|██▌       | 58820/225000 [121:38:11<343:30:20,  7.44s/it]
 26%|██▌       | 58821/225000 [121:38:19<352:47:45,  7.64s/it]
 26%|██▌       | 58822/225000 [121:38:23<299:49:07,  6.50s/it]
 26%|██▌       | 58823/225000 [121:38:27<266:43:09,  5.78s/it]
 26%|██▌       | 58824/225000 [121:38:37<332:35:36,  7.21s/it]
 26%|██▌       | 58825/225000 [121:38:42<298:32:56,  6.47s/it]
 26%|██▌       | 58826/225000 [121:38:51<332:46:12,  7.21s/it]
 26%|██▌       | 58827/225000 [121:38:58<331:15:51,  7.18s/it]
 26%|██▌       | 58828/225000 [121:39:04<320:23:32,  6.94s/it]
 26%|██▌       | 58829/225000 [121:39:13<340:01:52,  7.37s/it]
 26%|██▌       | 58830/225000 [121:39:21<359:48:04,  7.79s/it]
 26%|██▌       | 58831/225000 [121:39:26<309:36:56,  6.71s/it]
 26%|██▌       | 58832/225000 [121:39:33<315:07:17,  6.83s/it]
 26%|██▌       | 58833/225000 [121:39:38<286:30:08,  6.21s/it]
 26%|██▌       | 58834/225000 [121:39:42<260:39:31,  5.65s/it]
 26%|██▌       | 58835/225000 [121:39:47<249:50:42,  5.41s/it]
 26%|██▌       | 58836/225000 [121:40:00<353:06:43,  7.65s/it]
 26%|██▌       | 58837/225000 [121:40:04<305:18:30,  6.61s/it]
 26%|██▌       | 58838/225000 [121:40:09<284:05:03,  6.15s/it]
 26%|██▌       | 58839/225000 [121:40:19<340:22:11,  7.37s/it]
 26%|██▌       | 58840/225000 [121:40:24<302:48:52,  6.56s/it]
 26%|██▌       | 58841/225000 [121:40:29<284:14:05,  6.16s/it]
 26%|██▌       | 58842/225000 [121:40:34<268:42:13,  5.82s/it]
 26%|██▌       | 58843/225000 [121:40:39<255:22:26,  5.53s/it]
 26%|██▌       | 58844/225000 [121:40:51<346:51:35,  7.52s/it]
 26%|██▌       | 58845/225000 [121:40:58<334:09:22,  7.24s/it]
 26%|██▌       | 58846/225000 [121:41:03<308:56:32,  6.69s/it]
 26%|██▌       | 58847/225000 [121:41:09<292:00:51,  6.33s/it]
 26%|██▌       | 58848/225000 [121:41:14<275:36:51,  5.97s/it]
 26%|██▌       | 58849/225000 [121:41:20<279:53:29,  6.06s/it]
 26%|██▌       | 58850/225000 [121:41:32<363:22:50,  7.87s/it]
                                                              
{'loss': '0.7511', 'grad_norm': '0.8852', 'learning_rate': '4.39e-05', 'epoch': '0.2616', 'num_input_tokens_seen': 202728136, 'train_runtime': '4.381e+05', 'train_tokens_per_second': '462.8'}

 26%|██▌       | 58850/225000 [121:41:32<363:22:50,  7.87s/it]
 26%|██▌       | 58851/225000 [121:41:36<310:16:11,  6.72s/it]
 26%|██▌       | 58852/225000 [121:41:42<293:04:05,  6.35s/it]
 26%|██▌       | 58853/225000 [121:41:49<314:28:40,  6.81s/it]
 26%|██▌       | 58854/225000 [121:41:54<286:08:20,  6.20s/it]
 26%|██▌       | 58855/225000 [121:41:59<263:30:25,  5.71s/it]
 26%|██▌       | 58856/225000 [121:42:10<346:14:23,  7.50s/it]
 26%|██▌       | 58857/225000 [121:42:16<316:28:30,  6.86s/it]
 26%|██▌       | 58858/225000 [121:42:21<288:15:51,  6.25s/it]
 26%|██▌       | 58859/225000 [121:42:25<268:15:03,  5.81s/it]
 26%|██▌       | 58860/225000 [121:42:31<259:30:17,  5.62s/it]
 26%|██▌       | 58861/225000 [121:42:41<328:23:32,  7.12s/it]
 26%|██▌       | 58862/225000 [121:42:47<312:53:53,  6.78s/it]
 26%|██▌       | 58863/225000 [121:42:53<297:21:33,  6.44s/it]
 26%|██▌       | 58864/225000 [121:43:03<353:47:11,  7.67s/it]
 26%|██▌       | 58865/225000 [121:43:10<344:49:35,  7.47s/it]
 26%|██▌       | 58866/225000 [121:43:16<314:28:48,  6.81s/it]
 26%|██▌       | 58867/225000 [121:43:20<283:26:09,  6.14s/it]
 26%|██▌       | 58868/225000 [121:43:28<310:13:14,  6.72s/it]
 26%|██▌       | 58869/225000 [121:43:40<379:50:26,  8.23s/it]
 26%|██▌       | 58870/225000 [121:43:44<316:41:42,  6.86s/it]
 26%|██▌       | 58871/225000 [121:43:50<302:41:29,  6.56s/it]
 26%|██▌       | 58872/225000 [121:43:58<323:17:19,  7.01s/it]
 26%|██▌       | 58873/225000 [121:44:04<314:44:59,  6.82s/it]
 26%|██▌       | 58874/225000 [121:44:12<328:30:36,  7.12s/it]
 26%|██▌       | 58875/225000 [121:44:19<327:51:36,  7.10s/it]
 26%|██▌       | 58876/225000 [121:44:23<291:51:31,  6.32s/it]
 26%|██▌       | 58877/225000 [121:44:29<275:51:31,  5.98s/it]
 26%|██▌       | 58878/225000 [121:44:33<257:16:57,  5.58s/it]
 26%|██▌       | 58879/225000 [121:44:46<350:49:56,  7.60s/it]
 26%|██▌       | 58880/225000 [121:44:50<312:12:58,  6.77s/it]
 26%|██▌       | 58881/225000 [121:44:56<294:04:18,  6.37s/it]
 26%|██▌       | 58882/225000 [121:45:04<319:26:02,  6.92s/it]
 26%|██▌       | 58883/225000 [121:45:13<342:34:10,  7.42s/it]
 26%|██▌       | 58884/225000 [121:45:20<339:18:09,  7.35s/it]
 26%|██▌       | 58885/225000 [121:45:24<295:14:44,  6.40s/it]
 26%|██▌       | 58886/225000 [121:45:35<365:20:03,  7.92s/it]
 26%|██▌       | 58887/225000 [121:45:40<315:57:38,  6.85s/it]
 26%|██▌       | 58888/225000 [121:45:45<295:01:59,  6.39s/it]
 26%|██▌       | 58889/225000 [121:45:50<279:02:42,  6.05s/it]
 26%|██▌       | 58890/225000 [121:45:56<269:19:48,  5.84s/it]
 26%|██▌       | 58891/225000 [121:46:01<256:12:26,  5.55s/it]
 26%|██▌       | 58892/225000 [121:46:06<249:50:06,  5.41s/it]
 26%|██▌       | 58893/225000 [121:46:18<342:39:28,  7.43s/it]
 26%|██▌       | 58894/225000 [121:46:25<338:06:06,  7.33s/it]
 26%|██▌       | 58895/225000 [121:46:33<351:37:20,  7.62s/it]
 26%|██▌       | 58896/225000 [121:46:39<332:08:03,  7.20s/it]
 26%|██▌       | 58897/225000 [121:46:45<310:01:59,  6.72s/it]
 26%|██▌       | 58898/225000 [121:46:50<286:35:46,  6.21s/it]
 26%|██▌       | 58899/225000 [121:46:55<272:25:50,  5.90s/it]
 26%|██▌       | 58900/225000 [121:47:00<261:10:20,  5.66s/it]
                                                              
{'loss': '0.7311', 'grad_norm': '0.79', 'learning_rate': '4.389e-05', 'epoch': '0.2618', 'num_input_tokens_seen': 202878040, 'train_runtime': '4.384e+05', 'train_tokens_per_second': '462.7'}

 26%|██▌       | 58900/225000 [121:47:00<261:10:20,  5.66s/it]
 26%|██▌       | 58901/225000 [121:47:14<369:12:37,  8.00s/it]
 26%|██▌       | 58902/225000 [121:47:19<336:07:32,  7.29s/it]
 26%|██▌       | 58903/225000 [121:47:24<302:48:23,  6.56s/it]
 26%|██▌       | 58904/225000 [121:47:29<276:33:32,  5.99s/it]
 26%|██▌       | 58905/225000 [121:47:41<359:39:54,  7.80s/it]
 26%|██▌       | 58906/225000 [121:47:46<318:47:29,  6.91s/it]
 26%|██▌       | 58907/225000 [121:47:57<379:00:13,  8.21s/it]
 26%|██▌       | 58908/225000 [121:48:01<326:13:12,  7.07s/it]
 26%|██▌       | 58909/225000 [121:48:08<319:51:48,  6.93s/it]
 26%|██▌       | 58910/225000 [121:48:15<324:11:08,  7.03s/it]
 26%|██▌       | 58911/225000 [121:48:22<314:35:11,  6.82s/it]
 26%|██▌       | 58912/225000 [121:48:30<335:53:59,  7.28s/it]
 26%|██▌       | 58913/225000 [121:48:35<298:50:38,  6.48s/it]
 26%|██▌       | 58914/225000 [121:48:41<303:34:41,  6.58s/it]
 26%|██▌       | 58915/225000 [121:48:45<267:57:55,  5.81s/it]
 26%|██▌       | 58916/225000 [121:48:53<295:21:10,  6.40s/it]
 26%|██▌       | 58917/225000 [121:49:01<317:32:00,  6.88s/it]
 26%|██▌       | 58918/225000 [121:49:09<327:11:51,  7.09s/it]
 26%|██▌       | 58919/225000 [121:49:14<294:53:14,  6.39s/it]
 26%|██▌       | 58920/225000 [121:49:22<316:16:17,  6.86s/it]
 26%|██▌       | 58921/225000 [121:49:27<296:05:12,  6.42s/it]
 26%|██▌       | 58922/225000 [121:49:32<283:58:32,  6.16s/it]
 26%|██▌       | 58923/225000 [121:49:38<278:15:39,  6.03s/it]
 26%|██▌       | 58924/225000 [121:49:44<273:15:46,  5.92s/it]
 26%|██▌       | 58925/225000 [121:49:49<266:49:10,  5.78s/it]
 26%|██▌       | 58926/225000 [121:49:55<265:49:35,  5.76s/it]
 26%|██▌       | 58927/225000 [121:50:00<259:05:27,  5.62s/it]
 26%|██▌       | 58928/225000 [121:50:06<260:55:57,  5.66s/it]
 26%|██▌       | 58929/225000 [121:50:19<357:02:37,  7.74s/it]
 26%|██▌       | 58930/225000 [121:50:27<360:52:30,  7.82s/it]
 26%|██▌       | 58931/225000 [121:50:31<311:54:45,  6.76s/it]
 26%|██▌       | 58932/225000 [121:50:36<287:23:32,  6.23s/it]
 26%|██▌       | 58933/225000 [121:50:48<365:51:03,  7.93s/it]
 26%|██▌       | 58934/225000 [121:50:55<349:19:54,  7.57s/it]
 26%|██▌       | 58935/225000 [121:51:00<322:44:20,  7.00s/it]
 26%|██▌       | 58936/225000 [121:51:05<286:30:44,  6.21s/it]
 26%|██▌       | 58937/225000 [121:51:10<268:59:41,  5.83s/it]
 26%|██▌       | 58938/225000 [121:51:15<261:03:46,  5.66s/it]
 26%|██▌       | 58939/225000 [121:51:20<250:42:54,  5.44s/it]
 26%|██▌       | 58940/225000 [121:51:26<258:58:19,  5.61s/it]
 26%|██▌       | 58941/225000 [121:51:39<360:45:01,  7.82s/it]
 26%|██▌       | 58942/225000 [121:51:45<340:17:58,  7.38s/it]
 26%|██▌       | 58943/225000 [121:51:51<323:05:35,  7.00s/it]
 26%|██▌       | 58944/225000 [121:51:57<302:57:16,  6.57s/it]
 26%|██▌       | 58945/225000 [121:52:06<333:53:00,  7.24s/it]
 26%|██▌       | 58946/225000 [121:52:13<339:40:42,  7.36s/it]
 26%|██▌       | 58947/225000 [121:52:18<300:40:22,  6.52s/it]
 26%|██▌       | 58948/225000 [121:52:23<280:36:08,  6.08s/it]
 26%|██▌       | 58949/225000 [121:52:28<270:03:06,  5.85s/it]
 26%|██▌       | 58950/225000 [121:52:34<264:08:39,  5.73s/it]
                                                              
{'loss': '0.715', 'grad_norm': '0.6818', 'learning_rate': '4.388e-05', 'epoch': '0.262', 'num_input_tokens_seen': 203027952, 'train_runtime': '4.388e+05', 'train_tokens_per_second': '462.7'}

 26%|██▌       | 58950/225000 [121:52:34<264:08:39,  5.73s/it]
 26%|██▌       | 58951/225000 [121:52:39<259:59:22,  5.64s/it]
 26%|██▌       | 58952/225000 [121:52:53<369:23:32,  8.01s/it]
 26%|██▌       | 58953/225000 [121:53:00<365:28:02,  7.92s/it]
 26%|██▌       | 58954/225000 [121:53:09<372:15:21,  8.07s/it]
 26%|██▌       | 58955/225000 [121:53:13<317:36:56,  6.89s/it]
 26%|██▌       | 58956/225000 [121:53:23<365:16:07,  7.92s/it]
 26%|██▌       | 58957/225000 [121:53:28<316:17:34,  6.86s/it]
 26%|██▌       | 58958/225000 [121:53:32<276:25:31,  5.99s/it]
 26%|██▌       | 58959/225000 [121:53:43<347:16:04,  7.53s/it]
 26%|██▌       | 58960/225000 [121:53:50<342:16:57,  7.42s/it]
 26%|██▌       | 58961/225000 [121:53:53<286:11:56,  6.21s/it]
 26%|██▌       | 58962/225000 [121:53:58<271:53:34,  5.90s/it]
 26%|██▌       | 58963/225000 [121:54:04<272:30:21,  5.91s/it]
 26%|██▌       | 58964/225000 [121:54:10<263:23:46,  5.71s/it]
 26%|██▌       | 58965/225000 [121:54:15<257:34:43,  5.58s/it]
 26%|██▌       | 58966/225000 [121:54:20<252:57:47,  5.48s/it]
 26%|██▌       | 58967/225000 [121:54:25<248:04:00,  5.38s/it]
 26%|██▌       | 58968/225000 [121:54:37<330:14:33,  7.16s/it]
 26%|██▌       | 58969/225000 [121:54:44<334:10:00,  7.25s/it]
 26%|██▌       | 58970/225000 [121:54:49<301:14:51,  6.53s/it]
 26%|██▌       | 58971/225000 [121:54:54<287:08:57,  6.23s/it]
 26%|██▌       | 58972/225000 [121:55:00<273:06:52,  5.92s/it]
 26%|██▌       | 58973/225000 [121:55:05<266:20:16,  5.78s/it]
 26%|██▌       | 58974/225000 [121:55:15<322:47:52,  7.00s/it]
 26%|██▌       | 58975/225000 [121:55:26<375:21:08,  8.14s/it]
 26%|██▌       | 58976/225000 [121:55:31<335:38:38,  7.28s/it]
 26%|██▌       | 58977/225000 [121:55:39<347:44:23,  7.54s/it]
 26%|██▌       | 58978/225000 [121:55:44<311:06:32,  6.75s/it]
 26%|██▌       | 58979/225000 [121:55:49<285:19:27,  6.19s/it]
 26%|██▌       | 58980/225000 [121:55:54<272:05:29,  5.90s/it]
 26%|██▌       | 58981/225000 [121:55:59<263:28:27,  5.71s/it]
 26%|██▌       | 58982/225000 [121:56:05<258:18:19,  5.60s/it]
 26%|██▌       | 58983/225000 [121:56:16<340:35:00,  7.39s/it]
 26%|██▌       | 58984/225000 [121:56:23<330:29:51,  7.17s/it]
 26%|██▌       | 58985/225000 [121:56:31<339:12:55,  7.36s/it]
 26%|██▌       | 58986/225000 [121:56:35<301:04:56,  6.53s/it]
 26%|██▌       | 58987/225000 [121:56:40<274:55:58,  5.96s/it]
 26%|██▌       | 58988/225000 [121:56:45<259:18:40,  5.62s/it]
 26%|██▌       | 58989/225000 [121:56:50<257:55:20,  5.59s/it]
 26%|██▌       | 58990/225000 [121:57:04<375:48:22,  8.15s/it]
 26%|██▌       | 58991/225000 [121:57:09<328:19:59,  7.12s/it]
 26%|██▌       | 58992/225000 [121:57:14<297:30:35,  6.45s/it]
 26%|██▌       | 58993/225000 [121:57:19<271:41:50,  5.89s/it]
 26%|██▌       | 58994/225000 [121:57:31<357:39:21,  7.76s/it]
 26%|██▌       | 58995/225000 [121:57:35<303:23:07,  6.58s/it]
 26%|██▌       | 58996/225000 [121:57:43<333:26:09,  7.23s/it]
 26%|██▌       | 58997/225000 [121:57:50<322:54:27,  7.00s/it]
 26%|██▌       | 58998/225000 [121:57:56<306:39:43,  6.65s/it]
 26%|██▌       | 58999/225000 [121:58:00<280:28:54,  6.08s/it]
 26%|██▌       | 59000/225000 [121:58:05<264:05:15,  5.73s/it]
                                                              
{'loss': '0.7564', 'grad_norm': '0.8717', 'learning_rate': '4.387e-05', 'epoch': '0.2622', 'num_input_tokens_seen': 203177056, 'train_runtime': '4.391e+05', 'train_tokens_per_second': '462.7'}

 26%|██▌       | 59000/225000 [121:58:05<264:05:15,  5.73s/it]
 26%|██▌       | 59001/225000 [121:58:17<346:15:50,  7.51s/it]
 26%|██▌       | 59002/225000 [121:58:22<316:19:38,  6.86s/it]
 26%|██▌       | 59003/225000 [121:58:28<298:00:04,  6.46s/it]
 26%|██▌       | 59004/225000 [121:58:33<277:21:30,  6.02s/it]
 26%|██▌       | 59005/225000 [121:58:38<270:07:26,  5.86s/it]
 26%|██▌       | 59006/225000 [121:58:48<317:04:49,  6.88s/it]
 26%|██▌       | 59007/225000 [121:58:56<333:32:25,  7.23s/it]
 26%|██▌       | 59008/225000 [121:59:00<297:26:01,  6.45s/it]
 26%|██▌       | 59009/225000 [121:59:05<271:09:30,  5.88s/it]
 26%|██▌       | 59010/225000 [121:59:09<255:26:03,  5.54s/it]
 26%|██▌       | 59011/225000 [121:59:21<342:51:21,  7.44s/it]
 26%|██▌       | 59012/225000 [121:59:26<300:18:10,  6.51s/it]
 26%|██▌       | 59013/225000 [121:59:31<276:58:18,  6.01s/it]
 26%|██▌       | 59014/225000 [121:59:36<266:53:27,  5.79s/it]
 26%|██▌       | 59015/225000 [121:59:48<349:58:29,  7.59s/it]
 26%|██▌       | 59016/225000 [121:59:56<364:19:21,  7.90s/it]
 26%|██▌       | 59017/225000 [122:00:01<315:51:40,  6.85s/it]
 26%|██▌       | 59018/225000 [122:00:05<282:12:20,  6.12s/it]
 26%|██▌       | 59019/225000 [122:00:18<375:38:00,  8.15s/it]
 26%|██▌       | 59020/225000 [122:00:22<325:56:50,  7.07s/it]
 26%|██▌       | 59021/225000 [122:00:27<293:42:49,  6.37s/it]
 26%|██▌       | 59022/225000 [122:00:32<270:23:10,  5.86s/it]
 26%|██▌       | 59023/225000 [122:00:38<267:13:13,  5.80s/it]
 26%|██▌       | 59024/225000 [122:00:50<357:17:48,  7.75s/it]
 26%|██▌       | 59025/225000 [122:00:54<303:31:34,  6.58s/it]
 26%|██▌       | 59026/225000 [122:00:58<275:22:12,  5.97s/it]
 26%|██▌       | 59027/225000 [122:01:04<272:41:53,  5.91s/it]
 26%|██▌       | 59028/225000 [122:01:16<356:28:27,  7.73s/it]
 26%|██▌       | 59029/225000 [122:01:20<309:57:14,  6.72s/it]
 26%|██▌       | 59030/225000 [122:01:25<281:12:30,  6.10s/it]
 26%|██▌       | 59031/225000 [122:01:32<298:48:14,  6.48s/it]
 26%|██▌       | 59032/225000 [122:01:40<319:49:15,  6.94s/it]
 26%|██▌       | 59033/225000 [122:01:48<323:33:05,  7.02s/it]
 26%|██▌       | 59034/225000 [122:01:55<330:03:36,  7.16s/it]
 26%|██▌       | 59035/225000 [122:01:59<287:18:04,  6.23s/it]
 26%|██▌       | 59036/225000 [122:02:08<326:14:41,  7.08s/it]
 26%|██▌       | 59037/225000 [122:02:14<313:58:29,  6.81s/it]
 26%|██▌       | 59038/225000 [122:02:20<291:45:31,  6.33s/it]
 26%|██▌       | 59039/225000 [122:02:26<291:38:40,  6.33s/it]
 26%|██▌       | 59040/225000 [122:02:35<334:42:10,  7.26s/it]
 26%|██▌       | 59041/225000 [122:02:43<336:17:07,  7.29s/it]
 26%|██▌       | 59042/225000 [122:02:48<305:14:50,  6.62s/it]
 26%|██▌       | 59043/225000 [122:02:52<276:20:43,  5.99s/it]
 26%|██▌       | 59044/225000 [122:03:01<310:35:17,  6.74s/it]
 26%|██▌       | 59045/225000 [122:03:10<343:45:50,  7.46s/it]
 26%|██▌       | 59046/225000 [122:03:14<292:44:11,  6.35s/it]
 26%|██▌       | 59047/225000 [122:03:19<274:25:44,  5.95s/it]
 26%|██▌       | 59048/225000 [122:03:23<255:09:40,  5.54s/it]
 26%|██▌       | 59049/225000 [122:03:29<250:45:23,  5.44s/it]
 26%|██▌       | 59050/225000 [122:03:39<324:26:39,  7.04s/it]
                                                              
{'loss': '0.7275', 'grad_norm': '0.8467', 'learning_rate': '4.386e-05', 'epoch': '0.2624', 'num_input_tokens_seen': 203327288, 'train_runtime': '4.394e+05', 'train_tokens_per_second': '462.7'}

 26%|██▌       | 59050/225000 [122:03:39<324:26:39,  7.04s/it]
 26%|██▌       | 59051/225000 [122:03:49<358:50:35,  7.78s/it]
 26%|██▌       | 59052/225000 [122:03:54<326:47:55,  7.09s/it]
 26%|██▌       | 59053/225000 [122:04:00<308:15:55,  6.69s/it]
 26%|██▌       | 59054/225000 [122:04:08<322:19:08,  6.99s/it]
 26%|██▌       | 59055/225000 [122:04:12<284:01:12,  6.16s/it]
 26%|██▌       | 59056/225000 [122:04:21<327:58:23,  7.12s/it]
 26%|██▌       | 59057/225000 [122:04:27<311:33:48,  6.76s/it]
 26%|██▌       | 59058/225000 [122:04:32<278:55:33,  6.05s/it]
 26%|██▌       | 59059/225000 [122:04:36<260:30:23,  5.65s/it]
 26%|██▌       | 59060/225000 [122:04:47<327:33:55,  7.11s/it]
 26%|██▌       | 59061/225000 [122:04:53<320:53:03,  6.96s/it]
 26%|██▌       | 59062/225000 [122:05:03<351:42:23,  7.63s/it]
 26%|██▋       | 59063/225000 [122:05:06<298:18:38,  6.47s/it]
 26%|██▋       | 59064/225000 [122:05:13<301:33:40,  6.54s/it]
 26%|██▋       | 59065/225000 [122:05:21<316:25:25,  6.86s/it]
 26%|██▋       | 59066/225000 [122:05:28<319:23:15,  6.93s/it]
 26%|██▋       | 59067/225000 [122:05:35<328:55:14,  7.14s/it]
 26%|██▋       | 59068/225000 [122:05:41<305:48:29,  6.63s/it]
 26%|██▋       | 59069/225000 [122:05:47<294:48:17,  6.40s/it]
 26%|██▋       | 59070/225000 [122:05:52<284:55:08,  6.18s/it]
 26%|██▋       | 59071/225000 [122:05:58<276:21:36,  6.00s/it]
 26%|██▋       | 59072/225000 [122:06:04<273:53:49,  5.94s/it]
 26%|██▋       | 59073/225000 [122:06:10<274:29:37,  5.96s/it]
 26%|██▋       | 59074/225000 [122:06:15<269:29:50,  5.85s/it]
 26%|██▋       | 59075/225000 [122:06:20<258:11:49,  5.60s/it]
 26%|██▋       | 59076/225000 [122:06:28<287:10:45,  6.23s/it]
 26%|██▋       | 59077/225000 [122:06:42<391:58:27,  8.50s/it]
 26%|██▋       | 59078/225000 [122:06:48<353:28:30,  7.67s/it]
 26%|██▋       | 59079/225000 [122:06:54<338:22:11,  7.34s/it]
 26%|██▋       | 59080/225000 [122:07:02<349:28:16,  7.58s/it]
 26%|██▋       | 59081/225000 [122:07:10<343:39:00,  7.46s/it]
 26%|██▋       | 59082/225000 [122:07:13<293:55:57,  6.38s/it]
 26%|██▋       | 59083/225000 [122:07:18<263:13:40,  5.71s/it]
 26%|██▋       | 59084/225000 [122:07:29<340:34:17,  7.39s/it]
 26%|██▋       | 59085/225000 [122:07:33<290:40:27,  6.31s/it]
 26%|██▋       | 59086/225000 [122:07:37<259:45:46,  5.64s/it]
 26%|██▋       | 59087/225000 [122:07:49<349:19:32,  7.58s/it]
 26%|██▋       | 59088/225000 [122:07:55<334:12:21,  7.25s/it]
 26%|██▋       | 59089/225000 [122:08:00<292:29:04,  6.35s/it]
 26%|██▋       | 59090/225000 [122:08:10<351:22:37,  7.62s/it]
 26%|██▋       | 59091/225000 [122:08:14<297:27:35,  6.45s/it]
 26%|██▋       | 59092/225000 [122:08:23<329:51:13,  7.16s/it]
 26%|██▋       | 59093/225000 [122:08:27<287:04:25,  6.23s/it]
 26%|██▋       | 59094/225000 [122:08:36<326:22:14,  7.08s/it]
 26%|██▋       | 59095/225000 [122:08:40<282:49:20,  6.14s/it]
 26%|██▋       | 59096/225000 [122:08:47<294:48:22,  6.40s/it]
 26%|██▋       | 59097/225000 [122:08:52<278:39:52,  6.05s/it]
 26%|██▋       | 59098/225000 [122:09:02<328:46:33,  7.13s/it]
 26%|██▋       | 59099/225000 [122:09:06<294:09:21,  6.38s/it]
 26%|██▋       | 59100/225000 [122:09:16<333:33:11,  7.24s/it]
                                                              
{'loss': '0.7541', 'grad_norm': '0.8122', 'learning_rate': '4.384e-05', 'epoch': '0.2627', 'num_input_tokens_seen': 203476816, 'train_runtime': '4.398e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59100/225000 [122:09:16<333:33:11,  7.24s/it]
 26%|██▋       | 59101/225000 [122:09:20<298:25:44,  6.48s/it]
 26%|██▋       | 59102/225000 [122:09:25<276:31:11,  6.00s/it]
 26%|██▋       | 59103/225000 [122:09:30<259:40:30,  5.64s/it]
 26%|██▋       | 59104/225000 [122:09:34<245:25:20,  5.33s/it]
 26%|██▋       | 59105/225000 [122:09:41<263:56:28,  5.73s/it]
 26%|██▋       | 59106/225000 [122:09:53<346:58:50,  7.53s/it]
 26%|██▋       | 59107/225000 [122:09:57<297:30:14,  6.46s/it]
 26%|██▋       | 59108/225000 [122:10:08<369:33:12,  8.02s/it]
 26%|██▋       | 59109/225000 [122:10:13<320:00:47,  6.94s/it]
 26%|██▋       | 59110/225000 [122:10:17<282:16:45,  6.13s/it]
 26%|██▋       | 59111/225000 [122:10:26<326:18:39,  7.08s/it]
 26%|██▋       | 59112/225000 [122:10:35<352:17:00,  7.65s/it]
 26%|██▋       | 59113/225000 [122:10:40<315:05:12,  6.84s/it]
 26%|██▋       | 59114/225000 [122:10:45<287:58:13,  6.25s/it]
 26%|██▋       | 59115/225000 [122:10:50<264:47:53,  5.75s/it]
 26%|██▋       | 59116/225000 [122:10:59<306:10:49,  6.64s/it]
 26%|██▋       | 59117/225000 [122:11:05<310:05:55,  6.73s/it]
 26%|██▋       | 59118/225000 [122:11:14<336:46:29,  7.31s/it]
 26%|██▋       | 59119/225000 [122:11:20<310:30:57,  6.74s/it]
 26%|██▋       | 59120/225000 [122:11:27<323:27:13,  7.02s/it]
 26%|██▋       | 59121/225000 [122:11:32<291:45:56,  6.33s/it]
 26%|██▋       | 59122/225000 [122:11:41<326:38:07,  7.09s/it]
 26%|██▋       | 59123/225000 [122:11:49<341:22:29,  7.41s/it]
 26%|██▋       | 59124/225000 [122:11:53<292:08:44,  6.34s/it]
 26%|██▋       | 59125/225000 [122:12:02<333:03:37,  7.23s/it]
 26%|██▋       | 59126/225000 [122:12:07<294:01:56,  6.38s/it]
 26%|██▋       | 59127/225000 [122:12:15<317:38:42,  6.89s/it]
 26%|██▋       | 59128/225000 [122:12:19<276:36:27,  6.00s/it]
 26%|██▋       | 59129/225000 [122:12:28<326:24:49,  7.08s/it]
 26%|██▋       | 59130/225000 [122:12:33<289:47:58,  6.29s/it]
 26%|██▋       | 59131/225000 [122:12:37<269:14:03,  5.84s/it]
 26%|██▋       | 59132/225000 [122:12:43<263:00:29,  5.71s/it]
 26%|██▋       | 59133/225000 [122:12:48<250:47:09,  5.44s/it]
 26%|██▋       | 59134/225000 [122:12:52<239:00:56,  5.19s/it]
 26%|██▋       | 59135/225000 [122:13:05<343:35:12,  7.46s/it]
 26%|██▋       | 59136/225000 [122:13:12<332:58:00,  7.23s/it]
 26%|██▋       | 59137/225000 [122:13:16<296:53:45,  6.44s/it]
 26%|██▋       | 59138/225000 [122:13:22<292:13:16,  6.34s/it]
 26%|██▋       | 59139/225000 [122:13:27<275:05:38,  5.97s/it]
 26%|██▋       | 59140/225000 [122:13:33<265:55:59,  5.77s/it]
 26%|██▋       | 59141/225000 [122:13:39<267:56:31,  5.82s/it]
 26%|██▋       | 59142/225000 [122:13:47<304:01:12,  6.60s/it]
 26%|██▋       | 59143/225000 [122:13:59<382:00:52,  8.29s/it]
 26%|██▋       | 59144/225000 [122:14:04<326:14:14,  7.08s/it]
 26%|██▋       | 59145/225000 [122:14:09<303:35:15,  6.59s/it]
 26%|██▋       | 59146/225000 [122:14:15<289:23:22,  6.28s/it]
 26%|██▋       | 59147/225000 [122:14:20<272:03:38,  5.91s/it]
 26%|██▋       | 59148/225000 [122:14:25<261:38:08,  5.68s/it]
 26%|██▋       | 59149/225000 [122:14:31<270:05:10,  5.86s/it]
 26%|██▋       | 59150/225000 [122:14:36<261:49:06,  5.68s/it]
                                                              
{'loss': '0.7306', 'grad_norm': '0.8384', 'learning_rate': '4.383e-05', 'epoch': '0.2629', 'num_input_tokens_seen': 203625248, 'train_runtime': '4.401e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59150/225000 [122:14:36<261:49:06,  5.68s/it]
 26%|██▋       | 59151/225000 [122:14:47<334:08:30,  7.25s/it]
 26%|██▋       | 59152/225000 [122:14:58<384:44:18,  8.35s/it]
 26%|██▋       | 59153/225000 [122:15:03<334:24:53,  7.26s/it]
 26%|██▋       | 59154/225000 [122:15:07<293:31:40,  6.37s/it]
 26%|██▋       | 59155/225000 [122:15:18<355:53:59,  7.73s/it]
 26%|██▋       | 59156/225000 [122:15:24<325:44:01,  7.07s/it]
 26%|██▋       | 59157/225000 [122:15:29<306:48:21,  6.66s/it]
 26%|██▋       | 59158/225000 [122:15:35<294:44:19,  6.40s/it]
 26%|██▋       | 59159/225000 [122:15:41<289:27:36,  6.28s/it]
 26%|██▋       | 59160/225000 [122:15:47<282:19:03,  6.13s/it]
 26%|██▋       | 59161/225000 [122:15:53<283:11:50,  6.15s/it]
 26%|██▋       | 59162/225000 [122:16:05<357:35:27,  7.76s/it]
 26%|██▋       | 59163/225000 [122:16:10<327:09:18,  7.10s/it]
 26%|██▋       | 59164/225000 [122:16:16<308:46:36,  6.70s/it]
 26%|██▋       | 59165/225000 [122:16:24<330:42:46,  7.18s/it]
 26%|██▋       | 59166/225000 [122:16:30<313:16:37,  6.80s/it]
 26%|██▋       | 59167/225000 [122:16:35<288:01:25,  6.25s/it]
 26%|██▋       | 59168/225000 [122:16:41<284:32:02,  6.18s/it]
 26%|██▋       | 59169/225000 [122:16:47<277:56:59,  6.03s/it]
 26%|██▋       | 59170/225000 [122:16:59<357:21:42,  7.76s/it]
 26%|██▋       | 59171/225000 [122:17:05<337:52:26,  7.33s/it]
 26%|██▋       | 59172/225000 [122:17:12<329:40:56,  7.16s/it]
 26%|██▋       | 59173/225000 [122:17:17<310:02:16,  6.73s/it]
 26%|██▋       | 59174/225000 [122:17:22<282:40:20,  6.14s/it]
 26%|██▋       | 59175/225000 [122:17:28<277:51:26,  6.03s/it]
 26%|██▋       | 59176/225000 [122:17:40<359:19:46,  7.80s/it]
 26%|██▋       | 59177/225000 [122:17:44<305:02:03,  6.62s/it]
 26%|██▋       | 59178/225000 [122:17:49<288:56:58,  6.27s/it]
 26%|██▋       | 59179/225000 [122:17:54<273:53:45,  5.95s/it]
 26%|██▋       | 59180/225000 [122:18:00<263:21:40,  5.72s/it]
 26%|██▋       | 59181/225000 [122:18:05<257:46:18,  5.60s/it]
 26%|██▋       | 59182/225000 [122:18:14<299:33:06,  6.50s/it]
 26%|██▋       | 59183/225000 [122:18:24<355:38:55,  7.72s/it]
 26%|██▋       | 59184/225000 [122:18:28<302:53:56,  6.58s/it]
 26%|██▋       | 59185/225000 [122:18:37<329:36:49,  7.16s/it]
 26%|██▋       | 59186/225000 [122:18:40<281:38:15,  6.11s/it]
 26%|██▋       | 59187/225000 [122:18:49<320:46:18,  6.96s/it]
 26%|██▋       | 59188/225000 [122:18:57<331:52:44,  7.21s/it]
 26%|██▋       | 59189/225000 [122:19:02<299:14:48,  6.50s/it]
 26%|██▋       | 59190/225000 [122:19:06<270:11:13,  5.87s/it]
 26%|██▋       | 59191/225000 [122:19:17<334:39:49,  7.27s/it]
 26%|██▋       | 59192/225000 [122:19:24<332:40:37,  7.22s/it]
 26%|██▋       | 59193/225000 [122:19:31<334:12:19,  7.26s/it]
 26%|██▋       | 59194/225000 [122:19:35<292:15:48,  6.35s/it]
 26%|██▋       | 59195/225000 [122:19:40<265:18:44,  5.76s/it]
 26%|██▋       | 59196/225000 [122:19:47<285:00:07,  6.19s/it]
 26%|██▋       | 59197/225000 [122:19:57<333:01:46,  7.23s/it]
 26%|██▋       | 59198/225000 [122:20:02<309:10:18,  6.71s/it]
 26%|██▋       | 59199/225000 [122:20:07<282:12:23,  6.13s/it]
 26%|██▋       | 59200/225000 [122:20:12<266:47:37,  5.79s/it]
                                                              
{'loss': '0.7346', 'grad_norm': '0.6457', 'learning_rate': '4.382e-05', 'epoch': '0.2631', 'num_input_tokens_seen': 203772888, 'train_runtime': '4.404e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59200/225000 [122:20:12<266:47:37,  5.79s/it]
 26%|██▋       | 59201/225000 [122:20:16<248:03:00,  5.39s/it]
 26%|██▋       | 59202/225000 [122:20:27<325:44:41,  7.07s/it]
 26%|██▋       | 59203/225000 [122:20:34<326:13:04,  7.08s/it]
 26%|██▋       | 59204/225000 [122:20:40<302:17:22,  6.56s/it]
 26%|██▋       | 59205/225000 [122:20:48<325:21:12,  7.06s/it]
 26%|██▋       | 59206/225000 [122:20:52<288:21:39,  6.26s/it]
 26%|██▋       | 59207/225000 [122:21:01<314:44:39,  6.83s/it]
 26%|██▋       | 59208/225000 [122:21:05<283:06:00,  6.15s/it]
 26%|██▋       | 59209/225000 [122:21:11<273:09:18,  5.93s/it]
 26%|██▋       | 59210/225000 [122:21:16<269:51:52,  5.86s/it]
 26%|██▋       | 59211/225000 [122:21:22<266:52:56,  5.80s/it]
 26%|██▋       | 59212/225000 [122:21:27<256:54:46,  5.58s/it]
 26%|██▋       | 59213/225000 [122:21:34<281:16:17,  6.11s/it]
 26%|██▋       | 59214/225000 [122:21:50<407:11:17,  8.84s/it]
 26%|██▋       | 59215/225000 [122:21:54<353:36:03,  7.68s/it]
 26%|██▋       | 59216/225000 [122:21:59<306:59:48,  6.67s/it]
 26%|██▋       | 59217/225000 [122:22:04<279:55:31,  6.08s/it]
 26%|██▋       | 59218/225000 [122:22:13<323:00:04,  7.01s/it]
 26%|██▋       | 59219/225000 [122:22:21<344:18:39,  7.48s/it]
 26%|██▋       | 59220/225000 [122:22:30<358:08:31,  7.78s/it]
 26%|██▋       | 59221/225000 [122:22:34<306:33:21,  6.66s/it]
 26%|██▋       | 59222/225000 [122:22:38<275:46:23,  5.99s/it]
 26%|██▋       | 59223/225000 [122:22:49<336:02:34,  7.30s/it]
 26%|██▋       | 59224/225000 [122:22:54<304:20:26,  6.61s/it]
 26%|██▋       | 59225/225000 [122:22:59<292:03:36,  6.34s/it]
 26%|██▋       | 59226/225000 [122:23:05<276:50:31,  6.01s/it]
 26%|██▋       | 59227/225000 [122:23:10<271:20:01,  5.89s/it]
 26%|██▋       | 59228/225000 [122:23:18<298:28:44,  6.48s/it]
 26%|██▋       | 59229/225000 [122:23:29<355:55:55,  7.73s/it]
 26%|██▋       | 59230/225000 [122:23:33<304:03:47,  6.60s/it]
 26%|██▋       | 59231/225000 [122:23:41<334:32:33,  7.27s/it]
 26%|██▋       | 59232/225000 [122:23:46<296:58:00,  6.45s/it]
 26%|██▋       | 59233/225000 [122:23:51<274:44:02,  5.97s/it]
 26%|██▋       | 59234/225000 [122:23:56<260:05:47,  5.65s/it]
 26%|██▋       | 59235/225000 [122:24:00<245:08:06,  5.32s/it]
 26%|██▋       | 59236/225000 [122:24:14<362:15:07,  7.87s/it]
 26%|██▋       | 59237/225000 [122:24:19<318:33:33,  6.92s/it]
 26%|██▋       | 59238/225000 [122:24:23<279:23:19,  6.07s/it]
 26%|██▋       | 59239/225000 [122:24:28<267:05:37,  5.80s/it]
 26%|██▋       | 59240/225000 [122:24:40<357:19:48,  7.76s/it]
 26%|██▋       | 59241/225000 [122:24:46<332:52:24,  7.23s/it]
 26%|██▋       | 59242/225000 [122:24:51<303:01:46,  6.58s/it]
 26%|██▋       | 59243/225000 [122:24:59<318:45:19,  6.92s/it]
 26%|██▋       | 59244/225000 [122:25:08<345:35:55,  7.51s/it]
 26%|██▋       | 59245/225000 [122:25:12<303:27:09,  6.59s/it]
 26%|██▋       | 59246/225000 [122:25:21<331:37:32,  7.20s/it]
 26%|██▋       | 59247/225000 [122:25:26<306:08:33,  6.65s/it]
 26%|██▋       | 59248/225000 [122:25:32<286:19:41,  6.22s/it]
 26%|██▋       | 59249/225000 [122:25:37<272:52:44,  5.93s/it]
 26%|██▋       | 59250/225000 [122:25:43<279:41:40,  6.07s/it]
                                                              
{'loss': '0.7396', 'grad_norm': '0.9136', 'learning_rate': '4.381e-05', 'epoch': '0.2633', 'num_input_tokens_seen': 203921744, 'train_runtime': '4.407e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59250/225000 [122:25:43<279:41:40,  6.07s/it]
 26%|██▋       | 59251/225000 [122:25:57<379:19:13,  8.24s/it]
 26%|██▋       | 59252/225000 [122:26:00<313:10:54,  6.80s/it]
 26%|██▋       | 59253/225000 [122:26:10<349:59:50,  7.60s/it]
 26%|██▋       | 59254/225000 [122:26:14<310:57:36,  6.75s/it]
 26%|██▋       | 59255/225000 [122:26:19<278:18:15,  6.04s/it]
 26%|██▋       | 59256/225000 [122:26:28<322:00:14,  6.99s/it]
 26%|██▋       | 59257/225000 [122:26:34<314:14:52,  6.83s/it]
 26%|██▋       | 59258/225000 [122:26:39<280:00:55,  6.08s/it]
 26%|██▋       | 59259/225000 [122:26:50<353:10:01,  7.67s/it]
 26%|██▋       | 59260/225000 [122:26:57<345:15:04,  7.50s/it]
 26%|██▋       | 59261/225000 [122:27:03<319:45:13,  6.95s/it]
 26%|██▋       | 59262/225000 [122:27:08<290:14:05,  6.30s/it]
 26%|██▋       | 59263/225000 [122:27:12<268:47:27,  5.84s/it]
 26%|██▋       | 59264/225000 [122:27:18<259:26:12,  5.64s/it]
 26%|██▋       | 59265/225000 [122:27:23<253:27:42,  5.51s/it]
 26%|██▋       | 59266/225000 [122:27:28<253:38:55,  5.51s/it]
 26%|██▋       | 59267/225000 [122:27:39<331:16:03,  7.20s/it]
 26%|██▋       | 59268/225000 [122:27:46<329:47:38,  7.16s/it]
 26%|██▋       | 59269/225000 [122:27:54<333:16:16,  7.24s/it]
 26%|██▋       | 59270/225000 [122:27:58<295:53:19,  6.43s/it]
 26%|██▋       | 59271/225000 [122:28:03<263:51:05,  5.73s/it]
 26%|██▋       | 59272/225000 [122:28:11<296:51:29,  6.45s/it]
 26%|██▋       | 59273/225000 [122:28:18<310:37:51,  6.75s/it]
 26%|██▋       | 59274/225000 [122:28:25<310:18:51,  6.74s/it]
 26%|██▋       | 59275/225000 [122:28:32<316:18:56,  6.87s/it]
 26%|██▋       | 59276/225000 [122:28:37<291:18:08,  6.33s/it]
 26%|██▋       | 59277/225000 [122:28:43<281:56:21,  6.12s/it]
 26%|██▋       | 59278/225000 [122:28:48<270:49:44,  5.88s/it]
 26%|██▋       | 59279/225000 [122:28:53<263:14:55,  5.72s/it]
 26%|██▋       | 59280/225000 [122:29:02<298:18:11,  6.48s/it]
 26%|██▋       | 59281/225000 [122:29:12<348:21:07,  7.57s/it]
 26%|██▋       | 59282/225000 [122:29:16<296:39:00,  6.44s/it]
 26%|██▋       | 59283/225000 [122:29:24<323:44:35,  7.03s/it]
 26%|██▋       | 59284/225000 [122:29:30<307:31:05,  6.68s/it]
 26%|██▋       | 59285/225000 [122:29:37<310:40:24,  6.75s/it]
 26%|██▋       | 59286/225000 [122:29:40<269:16:05,  5.85s/it]
 26%|██▋       | 59287/225000 [122:29:50<319:06:33,  6.93s/it]
 26%|██▋       | 59288/225000 [122:29:54<275:14:49,  5.98s/it]
 26%|██▋       | 59289/225000 [122:29:58<256:53:57,  5.58s/it]
 26%|██▋       | 59290/225000 [122:30:09<322:19:29,  7.00s/it]
 26%|██▋       | 59291/225000 [122:30:13<280:54:04,  6.10s/it]
 26%|██▋       | 59292/225000 [122:30:21<317:56:16,  6.91s/it]
 26%|██▋       | 59293/225000 [122:30:26<282:05:05,  6.13s/it]
 26%|██▋       | 59294/225000 [122:30:30<259:18:02,  5.63s/it]
 26%|██▋       | 59295/225000 [122:30:39<298:56:06,  6.49s/it]
 26%|██▋       | 59296/225000 [122:30:49<351:47:51,  7.64s/it]
 26%|██▋       | 59297/225000 [122:30:53<299:51:20,  6.51s/it]
 26%|██▋       | 59298/225000 [122:31:01<320:28:43,  6.96s/it]
 26%|██▋       | 59299/225000 [122:31:05<279:26:40,  6.07s/it]
 26%|██▋       | 59300/225000 [122:31:09<254:44:02,  5.53s/it]
                                                              
{'loss': '0.7453', 'grad_norm': '0.7635', 'learning_rate': '4.38e-05', 'epoch': '0.2636', 'num_input_tokens_seen': 204069792, 'train_runtime': '4.411e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59300/225000 [122:31:09<254:44:02,  5.53s/it]
 26%|██▋       | 59301/225000 [122:31:23<369:46:20,  8.03s/it]
 26%|██▋       | 59302/225000 [122:31:28<319:26:21,  6.94s/it]
 26%|██▋       | 59303/225000 [122:31:35<333:56:22,  7.26s/it]
 26%|██▋       | 59304/225000 [122:31:39<285:22:58,  6.20s/it]
 26%|██▋       | 59305/225000 [122:31:48<326:47:03,  7.10s/it]
 26%|██▋       | 59306/225000 [122:31:52<282:57:02,  6.15s/it]
 26%|██▋       | 59307/225000 [122:32:01<316:52:58,  6.88s/it]
 26%|██▋       | 59308/225000 [122:32:06<295:54:11,  6.43s/it]
 26%|██▋       | 59309/225000 [122:32:12<279:32:34,  6.07s/it]
 26%|██▋       | 59310/225000 [122:32:17<273:25:02,  5.94s/it]
 26%|██▋       | 59311/225000 [122:32:23<269:33:17,  5.86s/it]
 26%|██▋       | 59312/225000 [122:32:28<266:12:20,  5.78s/it]
 26%|██▋       | 59313/225000 [122:32:42<375:59:03,  8.17s/it]
 26%|██▋       | 59314/225000 [122:32:49<350:01:47,  7.61s/it]
 26%|██▋       | 59315/225000 [122:32:53<313:29:26,  6.81s/it]
 26%|██▋       | 59316/225000 [122:32:58<281:46:01,  6.12s/it]
 26%|██▋       | 59317/225000 [122:33:03<268:54:17,  5.84s/it]
 26%|██▋       | 59318/225000 [122:33:11<292:11:39,  6.35s/it]
 26%|██▋       | 59319/225000 [122:33:24<384:54:36,  8.36s/it]
 26%|██▋       | 59320/225000 [122:33:28<323:40:41,  7.03s/it]
 26%|██▋       | 59321/225000 [122:33:33<295:49:00,  6.43s/it]
 26%|██▋       | 59322/225000 [122:33:38<278:47:38,  6.06s/it]
 26%|██▋       | 59323/225000 [122:33:43<268:21:17,  5.83s/it]
 26%|██▋       | 59324/225000 [122:33:48<260:30:40,  5.66s/it]
 26%|██▋       | 59325/225000 [122:33:58<316:36:27,  6.88s/it]
 26%|██▋       | 59326/225000 [122:34:10<382:41:34,  8.32s/it]
 26%|██▋       | 59327/225000 [122:34:16<347:32:21,  7.55s/it]
 26%|██▋       | 59328/225000 [122:34:21<321:41:21,  6.99s/it]
 26%|██▋       | 59329/225000 [122:34:26<283:47:06,  6.17s/it]
 26%|██▋       | 59330/225000 [122:34:35<334:46:17,  7.27s/it]
 26%|██▋       | 59331/225000 [122:34:39<289:42:09,  6.30s/it]
 26%|██▋       | 59332/225000 [122:34:43<257:58:55,  5.61s/it]
 26%|██▋       | 59333/225000 [122:34:49<253:40:35,  5.51s/it]
 26%|██▋       | 59334/225000 [122:34:55<258:18:50,  5.61s/it]
 26%|██▋       | 59335/225000 [122:35:00<255:26:37,  5.55s/it]
 26%|██▋       | 59336/225000 [122:35:13<360:09:24,  7.83s/it]
 26%|██▋       | 59337/225000 [122:35:19<331:09:52,  7.20s/it]
 26%|██▋       | 59338/225000 [122:35:24<301:27:11,  6.55s/it]
 26%|██▋       | 59339/225000 [122:35:28<274:36:10,  5.97s/it]
 26%|██▋       | 59340/225000 [122:35:35<279:51:02,  6.08s/it]
 26%|██▋       | 59341/225000 [122:35:43<308:56:34,  6.71s/it]
 26%|██▋       | 59342/225000 [122:35:48<290:20:05,  6.31s/it]
 26%|██▋       | 59343/225000 [122:35:54<285:03:17,  6.19s/it]
 26%|██▋       | 59344/225000 [122:36:00<278:28:22,  6.05s/it]
 26%|██▋       | 59345/225000 [122:36:06<278:39:20,  6.06s/it]
 26%|██▋       | 59346/225000 [122:36:11<267:33:24,  5.81s/it]
 26%|██▋       | 59347/225000 [122:36:19<295:12:04,  6.42s/it]
 26%|██▋       | 59348/225000 [122:36:33<395:48:23,  8.60s/it]
 26%|██▋       | 59349/225000 [122:36:40<370:05:17,  8.04s/it]
 26%|██▋       | 59350/225000 [122:36:44<319:44:38,  6.95s/it]
                                                              
{'loss': '0.7473', 'grad_norm': '0.9266', 'learning_rate': '4.378e-05', 'epoch': '0.2638', 'num_input_tokens_seen': 204218424, 'train_runtime': '4.414e+05', 'train_tokens_per_second': '462.7'}

 26%|██▋       | 59350/225000 [122:36:44<319:44:38,  6.95s/it]
 26%|██▋       | 59351/225000 [122:36:50<306:55:02,  6.67s/it]
 26%|██▋       | 59352/225000 [122:37:01<370:03:34,  8.04s/it]
 26%|██▋       | 59353/225000 [122:37:06<327:13:31,  7.11s/it]
 26%|██▋       | 59354/225000 [122:37:14<336:53:14,  7.32s/it]
 26%|██▋       | 59355/225000 [122:37:22<344:24:55,  7.49s/it]
 26%|██▋       | 59356/225000 [122:37:28<327:42:31,  7.12s/it]
 26%|██▋       | 59357/225000 [122:37:34<306:54:56,  6.67s/it]
 26%|██▋       | 59358/225000 [122:37:42<329:22:31,  7.16s/it]
 26%|██▋       | 59359/225000 [122:37:48<316:43:26,  6.88s/it]
 26%|██▋       | 59360/225000 [122:37:55<315:13:51,  6.85s/it]
 26%|██▋       | 59361/225000 [122:38:00<291:56:07,  6.34s/it]
 26%|██▋       | 59362/225000 [122:38:05<273:37:10,  5.95s/it]
 26%|██▋       | 59363/225000 [122:38:11<270:19:53,  5.88s/it]
 26%|██▋       | 59364/225000 [122:38:22<339:50:57,  7.39s/it]
 26%|██▋       | 59365/225000 [122:38:27<302:55:09,  6.58s/it]
 26%|██▋       | 59366/225000 [122:38:32<281:03:22,  6.11s/it]
 26%|██▋       | 59367/225000 [122:38:36<262:29:07,  5.71s/it]
 26%|██▋       | 59368/225000 [122:38:43<270:54:03,  5.89s/it]
 26%|██▋       | 59369/225000 [122:38:55<362:16:29,  7.87s/it]
 26%|██▋       | 59370/225000 [122:38:59<309:49:53,  6.73s/it]
 26%|██▋       | 59371/225000 [122:39:06<314:37:02,  6.84s/it]
 26%|██▋       | 59372/225000 [122:39:14<332:15:38,  7.22s/it]
 26%|██▋       | 59373/225000 [122:39:20<305:20:35,  6.64s/it]
 26%|██▋       | 59374/225000 [122:39:25<291:44:02,  6.34s/it]
 26%|██▋       | 59375/225000 [122:39:31<283:24:18,  6.16s/it]
 26%|██▋       | 59376/225000 [122:39:37<282:02:23,  6.13s/it]
 26%|██▋       | 59377/225000 [122:39:44<290:29:55,  6.31s/it]
 26%|██▋       | 59378/225000 [122:39:51<304:12:24,  6.61s/it]
 26%|██▋       | 59379/225000 [122:39:57<294:46:11,  6.41s/it]
 26%|██▋       | 59380/225000 [122:40:03<289:53:43,  6.30s/it]
 26%|██▋       | 59381/225000 [122:40:09<278:52:24,  6.06s/it]
 26%|██▋       | 59382/225000 [122:40:17<309:00:34,  6.72s/it]
 26%|██▋       | 59383/225000 [122:40:23<298:25:34,  6.49s/it]
 26%|██▋       | 59384/225000 [122:40:31<316:32:57,  6.88s/it]
 26%|██▋       | 59385/225000 [122:40:36<292:36:20,  6.36s/it]
 26%|██▋       | 59386/225000 [122:40:41<282:19:36,  6.14s/it]
 26%|██▋       | 59387/225000 [122:40:47<279:14:36,  6.07s/it]
 26%|██▋       | 59388/225000 [122:40:53<279:27:59,  6.07s/it]
 26%|██▋       | 59389/225000 [122:41:07<379:37:39,  8.25s/it]
 26%|██▋       | 59390/225000 [122:41:14<364:13:51,  7.92s/it]
 26%|██▋       | 59391/225000 [122:41:18<313:43:55,  6.82s/it]
 26%|██▋       | 59392/225000 [122:41:28<350:43:27,  7.62s/it]
 26%|██▋       | 59393/225000 [122:41:35<349:52:48,  7.61s/it]
 26%|██▋       | 59394/225000 [122:41:39<295:21:16,  6.42s/it]
 26%|██▋       | 59395/225000 [122:41:44<280:11:40,  6.09s/it]
 26%|██▋       | 59396/225000 [122:41:57<376:09:13,  8.18s/it]
 26%|██▋       | 59397/225000 [122:42:01<314:38:34,  6.84s/it]
 26%|██▋       | 59398/225000 [122:42:05<275:43:05,  5.99s/it]
 26%|██▋       | 59399/225000 [122:42:10<263:40:25,  5.73s/it]
 26%|██▋       | 59400/225000 [122:42:15<257:00:01,  5.59s/it]
                                                              
{'loss': '0.7394', 'grad_norm': '0.7718', 'learning_rate': '4.377e-05', 'epoch': '0.264', 'num_input_tokens_seen': 204367144, 'train_runtime': '4.417e+05', 'train_tokens_per_second': '462.6'}

 26%|██▋       | 59400/225000 [122:42:15<257:00:01,  5.59s/it]
 26%|██▋       | 59401/225000 [122:42:21<254:41:49,  5.54s/it]
 26%|██▋       | 59402/225000 [122:42:34<355:19:14,  7.72s/it]
 26%|██▋       | 59403/225000 [122:42:41<347:23:43,  7.55s/it]
 26%|██▋       | 59404/225000 [122:42:49<361:25:46,  7.86s/it]
 26%|██▋       | 59405/225000 [122:42:54<318:19:06,  6.92s/it]
 26%|██▋       | 59406/225000 [122:42:59<283:41:57,  6.17s/it]
 26%|██▋       | 59407/225000 [122:43:08<333:59:09,  7.26s/it]
 26%|██▋       | 59408/225000 [122:43:14<306:53:41,  6.67s/it]
 26%|██▋       | 59409/225000 [122:43:21<311:46:27,  6.78s/it]
 26%|██▋       | 59410/225000 [122:43:25<281:27:32,  6.12s/it]
 26%|██▋       | 59411/225000 [122:43:32<285:15:21,  6.20s/it]
 26%|██▋       | 59412/225000 [122:43:41<330:52:17,  7.19s/it]
 26%|██▋       | 59413/225000 [122:43:46<292:17:21,  6.35s/it]
 26%|██▋       | 59414/225000 [122:43:51<279:54:01,  6.09s/it]
 26%|██▋       | 59415/225000 [122:43:56<268:03:52,  5.83s/it]
 26%|██▋       | 59416/225000 [122:44:03<286:11:23,  6.22s/it]
 26%|██▋       | 59417/225000 [122:44:15<366:37:30,  7.97s/it]
 26%|██▋       | 59418/225000 [122:44:20<316:05:49,  6.87s/it]
 26%|██▋       | 59419/225000 [122:44:24<281:42:55,  6.12s/it]
 26%|██▋       | 59420/225000 [122:44:29<265:02:32,  5.76s/it]
 26%|██▋       | 59421/225000 [122:44:39<326:38:56,  7.10s/it]
 26%|██▋       | 59422/225000 [122:44:47<338:24:58,  7.36s/it]
 26%|██▋       | 59423/225000 [122:44:52<297:55:15,  6.48s/it]
 26%|██▋       | 59424/225000 [122:44:59<314:18:48,  6.83s/it]
 26%|██▋       | 59425/225000 [122:45:06<314:23:18,  6.84s/it]
 26%|██▋       | 59426/225000 [122:45:10<273:03:20,  5.94s/it]
 26%|██▋       | 59427/225000 [122:45:20<329:42:59,  7.17s/it]
 26%|██▋       | 59428/225000 [122:45:25<293:04:13,  6.37s/it]
 26%|██▋       | 59429/225000 [122:45:29<265:36:01,  5.77s/it]
 26%|██▋       | 59430/225000 [122:45:40<341:29:32,  7.43s/it]
 26%|██▋       | 59431/225000 [122:45:44<287:33:23,  6.25s/it]
 26%|██▋       | 59432/225000 [122:45:49<277:38:52,  6.04s/it]
 26%|██▋       | 59433/225000 [122:45:55<268:47:20,  5.84s/it]
 26%|██▋       | 59434/225000 [122:46:00<261:54:12,  5.69s/it]
 26%|██▋       | 59435/225000 [122:46:11<329:22:16,  7.16s/it]
 26%|██▋       | 59436/225000 [122:46:17<321:00:03,  6.98s/it]
 26%|██▋       | 59437/225000 [122:46:23<308:45:53,  6.71s/it]
 26%|██▋       | 59438/225000 [122:46:33<350:38:50,  7.62s/it]
 26%|██▋       | 59439/225000 [122:46:39<332:30:50,  7.23s/it]
 26%|██▋       | 59440/225000 [122:46:44<295:33:55,  6.43s/it]
 26%|██▋       | 59441/225000 [122:46:50<286:37:58,  6.23s/it]
 26%|██▋       | 59442/225000 [122:47:00<349:54:56,  7.61s/it]
 26%|██▋       | 59443/225000 [122:47:04<299:01:57,  6.50s/it]
 26%|██▋       | 59444/225000 [122:47:12<312:21:02,  6.79s/it]
 26%|██▋       | 59445/225000 [122:47:19<316:40:57,  6.89s/it]
 26%|██▋       | 59446/225000 [122:47:23<275:21:01,  5.99s/it]
 26%|██▋       | 59447/225000 [122:47:28<260:26:56,  5.66s/it]
 26%|██▋       | 59448/225000 [122:47:41<362:23:27,  7.88s/it]
 26%|██▋       | 59449/225000 [122:47:46<330:23:32,  7.18s/it]
 26%|██▋       | 59450/225000 [122:47:51<301:39:54,  6.56s/it]
                                                              
{'loss': '0.7377', 'grad_norm': '0.7832', 'learning_rate': '4.376e-05', 'epoch': '0.2642', 'num_input_tokens_seen': 204516312, 'train_runtime': '4.421e+05', 'train_tokens_per_second': '462.6'}

 26%|██▋       | 59450/225000 [122:47:51<301:39:54,  6.56s/it]
 26%|██▋       | 59451/225000 [122:47:57<286:06:39,  6.22s/it]
 26%|██▋       | 59452/225000 [122:48:02<275:30:13,  5.99s/it]
 26%|██▋       | 59453/225000 [122:48:08<265:49:32,  5.78s/it]
 26%|██▋       | 59454/225000 [122:48:13<260:18:22,  5.66s/it]
 26%|██▋       | 59455/225000 [122:48:25<352:59:54,  7.68s/it]
 26%|██▋       | 59456/225000 [122:48:31<325:28:51,  7.08s/it]
 26%|██▋       | 59457/225000 [122:48:36<292:12:56,  6.35s/it]
 26%|██▋       | 59458/225000 [122:48:47<358:41:03,  7.80s/it]
 26%|██▋       | 59459/225000 [122:48:51<313:54:55,  6.83s/it]
 26%|██▋       | 59460/225000 [122:48:57<296:36:47,  6.45s/it]
 26%|██▋       | 59461/225000 [122:49:06<329:27:59,  7.16s/it]
 26%|██▋       | 59462/225000 [122:49:13<329:56:13,  7.18s/it]
 26%|██▋       | 59463/225000 [122:49:18<296:53:43,  6.46s/it]
 26%|██▋       | 59464/225000 [122:49:25<300:47:05,  6.54s/it]
 26%|██▋       | 59465/225000 [122:49:32<314:21:10,  6.84s/it]
 26%|██▋       | 59466/225000 [122:49:37<288:26:22,  6.27s/it]
 26%|██▋       | 59467/225000 [122:49:42<276:54:27,  6.02s/it]
 26%|██▋       | 59468/225000 [122:49:53<342:16:55,  7.44s/it]
 26%|██▋       | 59469/225000 [122:49:59<314:35:36,  6.84s/it]
 26%|██▋       | 59470/225000 [122:50:04<294:09:52,  6.40s/it]
 26%|██▋       | 59471/225000 [122:50:10<289:06:11,  6.29s/it]
 26%|██▋       | 59472/225000 [122:50:20<339:03:07,  7.37s/it]
 26%|██▋       | 59473/225000 [122:50:28<348:08:46,  7.57s/it]
 26%|██▋       | 59474/225000 [122:50:33<311:07:16,  6.77s/it]
 26%|██▋       | 59475/225000 [122:50:38<283:16:48,  6.16s/it]
 26%|██▋       | 59476/225000 [122:50:42<264:25:00,  5.75s/it]
 26%|██▋       | 59477/225000 [122:50:48<264:00:11,  5.74s/it]
 26%|██▋       | 59478/225000 [122:50:58<316:25:04,  6.88s/it]
 26%|██▋       | 59479/225000 [122:51:07<348:12:10,  7.57s/it]
 26%|██▋       | 59480/225000 [122:51:14<336:15:54,  7.31s/it]
 26%|██▋       | 59481/225000 [122:51:19<309:06:55,  6.72s/it]
 26%|██▋       | 59482/225000 [122:51:24<292:53:54,  6.37s/it]
 26%|██▋       | 59483/225000 [122:51:30<285:32:15,  6.21s/it]
 26%|██▋       | 59484/225000 [122:51:36<271:56:49,  5.91s/it]
 26%|██▋       | 59485/225000 [122:51:41<262:48:34,  5.72s/it]
 26%|██▋       | 59486/225000 [122:51:47<274:03:19,  5.96s/it]
 26%|██▋       | 59487/225000 [122:52:01<376:06:33,  8.18s/it]
 26%|██▋       | 59488/225000 [122:52:07<350:28:25,  7.62s/it]
 26%|██▋       | 59489/225000 [122:52:13<321:29:52,  6.99s/it]
 26%|██▋       | 59490/225000 [122:52:18<293:50:32,  6.39s/it]
 26%|██▋       | 59491/225000 [122:52:29<370:43:21,  8.06s/it]
 26%|██▋       | 59492/225000 [122:52:33<306:24:30,  6.66s/it]
 26%|██▋       | 59493/225000 [122:52:41<332:03:17,  7.22s/it]
 26%|██▋       | 59494/225000 [122:52:45<283:11:24,  6.16s/it]
 26%|██▋       | 59495/225000 [122:52:49<252:51:51,  5.50s/it]
 26%|██▋       | 59496/225000 [122:52:54<251:56:10,  5.48s/it]
 26%|██▋       | 59497/225000 [122:53:06<333:15:30,  7.25s/it]
 26%|██▋       | 59498/225000 [122:53:10<291:49:02,  6.35s/it]
 26%|██▋       | 59499/225000 [122:53:17<294:23:54,  6.40s/it]
 26%|██▋       | 59500/225000 [122:53:26<341:07:11,  7.42s/it]
                                                              
{'loss': '0.727', 'grad_norm': '0.7453', 'learning_rate': '4.375e-05', 'epoch': '0.2644', 'num_input_tokens_seen': 204665848, 'train_runtime': '4.424e+05', 'train_tokens_per_second': '462.6'}

 26%|██▋       | 59500/225000 [122:53:26<341:07:11,  7.42s/it]
 26%|██▋       | 59501/225000 [122:53:31<306:09:51,  6.66s/it]
 26%|██▋       | 59502/225000 [122:53:36<278:22:31,  6.06s/it]
 26%|██▋       | 59503/225000 [122:53:41<262:05:28,  5.70s/it]
 26%|██▋       | 59504/225000 [122:53:46<253:32:53,  5.52s/it]
 26%|██▋       | 59505/225000 [122:54:00<367:24:27,  7.99s/it]
 26%|██▋       | 59506/225000 [122:54:05<330:53:11,  7.20s/it]
 26%|██▋       | 59507/225000 [122:54:12<325:56:10,  7.09s/it]
 26%|██▋       | 59508/225000 [122:54:19<320:53:46,  6.98s/it]
 26%|██▋       | 59509/225000 [122:54:26<327:06:30,  7.12s/it]
 26%|██▋       | 59510/225000 [122:54:31<293:42:19,  6.39s/it]
 26%|██▋       | 59511/225000 [122:54:39<317:45:55,  6.91s/it]
 26%|██▋       | 59512/225000 [122:54:43<285:40:16,  6.21s/it]
 26%|██▋       | 59513/225000 [122:54:48<257:49:14,  5.61s/it]
 26%|██▋       | 59514/225000 [122:54:55<275:02:10,  5.98s/it]
 26%|██▋       | 59515/225000 [122:55:06<349:22:16,  7.60s/it]
 26%|██▋       | 59516/225000 [122:55:10<305:37:22,  6.65s/it]
 26%|██▋       | 59517/225000 [122:55:18<320:57:08,  6.98s/it]
 26%|██▋       | 59518/225000 [122:55:23<288:07:21,  6.27s/it]
 26%|██▋       | 59519/225000 [122:55:27<266:44:14,  5.80s/it]
 26%|██▋       | 59520/225000 [122:55:37<319:50:17,  6.96s/it]
 26%|██▋       | 59521/225000 [122:55:42<289:57:51,  6.31s/it]
 26%|██▋       | 59522/225000 [122:55:48<285:45:20,  6.22s/it]
 26%|██▋       | 59523/225000 [122:55:59<352:19:21,  7.66s/it]
 26%|██▋       | 59524/225000 [122:56:03<299:04:38,  6.51s/it]
 26%|██▋       | 59525/225000 [122:56:12<342:03:22,  7.44s/it]
 26%|██▋       | 59526/225000 [122:56:16<296:50:42,  6.46s/it]
 26%|██▋       | 59527/225000 [122:56:21<271:14:54,  5.90s/it]
 26%|██▋       | 59528/225000 [122:56:33<349:42:32,  7.61s/it]
 26%|██▋       | 59529/225000 [122:56:39<333:56:30,  7.27s/it]
 26%|██▋       | 59530/225000 [122:56:45<310:30:46,  6.76s/it]
 26%|██▋       | 59531/225000 [122:56:53<330:28:30,  7.19s/it]
 26%|██▋       | 59532/225000 [122:56:57<287:55:39,  6.26s/it]
 26%|██▋       | 59533/225000 [122:57:05<308:18:23,  6.71s/it]
 26%|██▋       | 59534/225000 [122:57:12<316:13:44,  6.88s/it]
 26%|██▋       | 59535/225000 [122:57:19<312:21:45,  6.80s/it]
 26%|██▋       | 59536/225000 [122:57:28<349:39:06,  7.61s/it]
 26%|██▋       | 59537/225000 [122:57:32<304:18:42,  6.62s/it]
 26%|██▋       | 59538/225000 [122:57:37<279:42:22,  6.09s/it]
 26%|██▋       | 59539/225000 [122:57:46<320:04:50,  6.96s/it]
 26%|██▋       | 59540/225000 [122:57:51<286:37:24,  6.24s/it]
 26%|██▋       | 59541/225000 [122:57:59<315:29:16,  6.86s/it]
 26%|██▋       | 59542/225000 [122:58:05<301:57:57,  6.57s/it]
 26%|██▋       | 59543/225000 [122:58:12<313:21:33,  6.82s/it]
 26%|██▋       | 59544/225000 [122:58:23<359:58:27,  7.83s/it]
 26%|██▋       | 59545/225000 [122:58:28<322:27:14,  7.02s/it]
 26%|██▋       | 59546/225000 [122:58:35<323:05:19,  7.03s/it]
 26%|██▋       | 59547/225000 [122:58:39<286:46:16,  6.24s/it]
 26%|██▋       | 59548/225000 [122:58:44<261:53:11,  5.70s/it]
 26%|██▋       | 59549/225000 [122:58:56<347:10:01,  7.55s/it]
 26%|██▋       | 59550/225000 [122:59:01<314:13:00,  6.84s/it]
                                                              
{'loss': '0.754', 'grad_norm': '0.7788', 'learning_rate': '4.374e-05', 'epoch': '0.2647', 'num_input_tokens_seen': 204815296, 'train_runtime': '4.427e+05', 'train_tokens_per_second': '462.6'}

 26%|██▋       | 59550/225000 [122:59:01<314:13:00,  6.84s/it]
 26%|██▋       | 59551/225000 [122:59:10<344:18:36,  7.49s/it]
 26%|██▋       | 59552/225000 [122:59:14<300:43:47,  6.54s/it]
 26%|██▋       | 59553/225000 [122:59:18<270:42:07,  5.89s/it]
 26%|██▋       | 59554/225000 [122:59:27<314:23:32,  6.84s/it]
 26%|██▋       | 59555/225000 [122:59:34<313:28:29,  6.82s/it]
 26%|██▋       | 59556/225000 [122:59:42<324:27:39,  7.06s/it]
 26%|██▋       | 59557/225000 [122:59:47<296:20:20,  6.45s/it]
 26%|██▋       | 59558/225000 [122:59:52<274:08:48,  5.97s/it]
 26%|██▋       | 59559/225000 [122:59:57<259:19:57,  5.64s/it]
 26%|██▋       | 59560/225000 [123:00:05<300:03:24,  6.53s/it]
 26%|██▋       | 59561/225000 [123:00:13<321:06:47,  6.99s/it]
 26%|██▋       | 59562/225000 [123:00:19<298:27:29,  6.49s/it]
 26%|██▋       | 59563/225000 [123:00:24<287:01:10,  6.25s/it]
 26%|██▋       | 59564/225000 [123:00:29<269:10:19,  5.86s/it]
 26%|██▋       | 59565/225000 [123:00:34<256:49:22,  5.59s/it]
 26%|██▋       | 59566/225000 [123:00:46<336:37:28,  7.33s/it]
 26%|██▋       | 59567/225000 [123:00:54<351:57:53,  7.66s/it]
 26%|██▋       | 59568/225000 [123:00:58<304:55:39,  6.64s/it]
 26%|██▋       | 59569/225000 [123:01:03<272:37:17,  5.93s/it]
 26%|██▋       | 59570/225000 [123:01:16<375:07:15,  8.16s/it]
 26%|██▋       | 59571/225000 [123:01:20<325:31:25,  7.08s/it]
 26%|██▋       | 59572/225000 [123:01:25<286:22:44,  6.23s/it]
 26%|██▋       | 59573/225000 [123:01:34<330:25:21,  7.19s/it]
 26%|██▋       | 59574/225000 [123:01:38<288:41:57,  6.28s/it]
 26%|██▋       | 59575/225000 [123:01:43<261:49:15,  5.70s/it]
 26%|██▋       | 59576/225000 [123:01:53<326:00:17,  7.09s/it]
 26%|██▋       | 59577/225000 [123:01:56<275:31:13,  6.00s/it]
 26%|██▋       | 59578/225000 [123:02:04<299:27:09,  6.52s/it]
 26%|██▋       | 59579/225000 [123:02:12<319:00:30,  6.94s/it]
 26%|██▋       | 59580/225000 [123:02:16<283:16:42,  6.16s/it]
 26%|██▋       | 59581/225000 [123:02:28<353:31:56,  7.69s/it]
 26%|██▋       | 59582/225000 [123:02:33<322:29:37,  7.02s/it]
 26%|██▋       | 59583/225000 [123:02:39<302:34:48,  6.59s/it]
 26%|██▋       | 59584/225000 [123:02:47<330:36:58,  7.20s/it]
 26%|██▋       | 59585/225000 [123:02:52<295:09:44,  6.42s/it]
 26%|██▋       | 59586/225000 [123:03:01<326:14:30,  7.10s/it]
 26%|██▋       | 59587/225000 [123:03:06<308:03:32,  6.70s/it]
 26%|██▋       | 59588/225000 [123:03:15<336:37:33,  7.33s/it]
 26%|██▋       | 59589/225000 [123:03:23<342:36:26,  7.46s/it]
 26%|██▋       | 59590/225000 [123:03:29<320:06:46,  6.97s/it]
 26%|██▋       | 59591/225000 [123:03:37<343:19:41,  7.47s/it]
 26%|██▋       | 59592/225000 [123:03:45<350:02:48,  7.62s/it]
 26%|██▋       | 59593/225000 [123:03:51<315:33:55,  6.87s/it]
 26%|██▋       | 59594/225000 [123:03:56<293:01:04,  6.38s/it]
 26%|██▋       | 59595/225000 [123:04:01<283:04:11,  6.16s/it]
 26%|██▋       | 59596/225000 [123:04:07<281:00:07,  6.12s/it]
 26%|██▋       | 59597/225000 [123:04:13<276:35:09,  6.02s/it]
 26%|██▋       | 59598/225000 [123:04:19<268:47:16,  5.85s/it]
 26%|██▋       | 59599/225000 [123:04:24<260:51:30,  5.68s/it]
 26%|██▋       | 59600/225000 [123:04:30<259:29:26,  5.65s/it]
                                                              
{'loss': '0.7339', 'grad_norm': '0.8398', 'learning_rate': '4.372e-05', 'epoch': '0.2649', 'num_input_tokens_seen': 204966024, 'train_runtime': '4.431e+05', 'train_tokens_per_second': '462.6'}

 26%|██▋       | 59600/225000 [123:04:30<259:29:26,  5.65s/it]
 26%|██▋       | 59601/225000 [123:04:39<313:19:52,  6.82s/it]
 26%|██▋       | 59602/225000 [123:04:51<378:15:42,  8.23s/it]
 26%|██▋       | 59603/225000 [123:04:54<313:15:21,  6.82s/it]
 26%|██▋       | 59604/225000 [123:05:02<333:30:06,  7.26s/it]
 26%|██▋       | 59605/225000 [123:05:07<298:34:36,  6.50s/it]
 26%|██▋       | 59606/225000 [123:05:14<306:12:22,  6.66s/it]
 26%|██▋       | 59607/225000 [123:05:23<335:34:15,  7.30s/it]
 26%|██▋       | 59608/225000 [123:05:27<295:38:08,  6.43s/it]
 26%|██▋       | 59609/225000 [123:05:33<287:44:19,  6.26s/it]
 26%|██▋       | 59610/225000 [123:05:43<338:31:25,  7.37s/it]
 26%|██▋       | 59611/225000 [123:05:49<317:21:42,  6.91s/it]
 26%|██▋       | 59612/225000 [123:05:54<285:55:37,  6.22s/it]
 26%|██▋       | 59613/225000 [123:05:58<265:27:55,  5.78s/it]
 26%|██▋       | 59614/225000 [123:06:04<264:59:31,  5.77s/it]
 26%|██▋       | 59615/225000 [123:06:17<361:55:48,  7.88s/it]
 26%|██▋       | 59616/225000 [123:06:21<309:39:52,  6.74s/it]
 26%|██▋       | 59617/225000 [123:06:26<284:06:01,  6.18s/it]
 26%|██▋       | 59618/225000 [123:06:31<271:08:53,  5.90s/it]
 26%|██▋       | 59619/225000 [123:06:43<352:51:42,  7.68s/it]
 26%|██▋       | 59620/225000 [123:06:51<350:44:44,  7.64s/it]
 26%|██▋       | 59621/225000 [123:06:56<316:37:02,  6.89s/it]
 26%|██▋       | 59622/225000 [123:07:05<346:51:19,  7.55s/it]
 26%|██▋       | 59623/225000 [123:07:09<298:08:04,  6.49s/it]
 26%|██▋       | 59624/225000 [123:07:20<357:52:41,  7.79s/it]
 26%|██▋       | 59625/225000 [123:07:24<305:15:18,  6.65s/it]
 27%|██▋       | 59626/225000 [123:07:31<318:20:37,  6.93s/it]
 27%|██▋       | 59627/225000 [123:07:39<328:04:43,  7.14s/it]
 27%|██▋       | 59628/225000 [123:07:44<297:25:52,  6.47s/it]
 27%|██▋       | 59629/225000 [123:07:49<285:02:28,  6.21s/it]
 27%|██▋       | 59630/225000 [123:07:55<283:16:46,  6.17s/it]
 27%|██▋       | 59631/225000 [123:08:01<272:39:48,  5.94s/it]
 27%|██▋       | 59632/225000 [123:08:06<267:54:07,  5.83s/it]
 27%|██▋       | 59633/225000 [123:08:14<290:13:12,  6.32s/it]
 27%|██▋       | 59634/225000 [123:08:24<338:29:14,  7.37s/it]
 27%|██▋       | 59635/225000 [123:08:28<303:29:49,  6.61s/it]
 27%|██▋       | 59636/225000 [123:08:33<277:07:38,  6.03s/it]
 27%|██▋       | 59637/225000 [123:08:45<356:14:57,  7.76s/it]
 27%|██▋       | 59638/225000 [123:08:51<327:52:06,  7.14s/it]
 27%|██▋       | 59639/225000 [123:08:56<304:10:03,  6.62s/it]
 27%|██▋       | 59640/225000 [123:09:02<291:19:18,  6.34s/it]
 27%|██▋       | 59641/225000 [123:09:07<275:13:56,  5.99s/it]
 27%|██▋       | 59642/225000 [123:09:12<268:09:44,  5.84s/it]
 27%|██▋       | 59643/225000 [123:09:18<262:03:36,  5.71s/it]
 27%|██▋       | 59644/225000 [123:09:23<261:44:08,  5.70s/it]
 27%|██▋       | 59645/225000 [123:09:36<361:16:38,  7.87s/it]
 27%|██▋       | 59646/225000 [123:09:44<363:09:21,  7.91s/it]
 27%|██▋       | 59647/225000 [123:09:50<324:36:43,  7.07s/it]
 27%|██▋       | 59648/225000 [123:09:55<301:39:52,  6.57s/it]
 27%|██▋       | 59649/225000 [123:10:01<293:40:23,  6.39s/it]
 27%|██▋       | 59650/225000 [123:10:07<283:11:37,  6.17s/it]
                                                              
{'loss': '0.7349', 'grad_norm': '0.7993', 'learning_rate': '4.371e-05', 'epoch': '0.2651', 'num_input_tokens_seen': 205116032, 'train_runtime': '4.434e+05', 'train_tokens_per_second': '462.6'}

 27%|██▋       | 59650/225000 [123:10:07<283:11:37,  6.17s/it]
 27%|██▋       | 59651/225000 [123:10:12<270:37:49,  5.89s/it]
 27%|██▋       | 59652/225000 [123:10:18<269:01:26,  5.86s/it]
 27%|██▋       | 59653/225000 [123:10:23<264:28:31,  5.76s/it]
 27%|██▋       | 59654/225000 [123:10:29<264:23:23,  5.76s/it]
 27%|██▋       | 59655/225000 [123:10:42<359:55:37,  7.84s/it]
 27%|██▋       | 59656/225000 [123:10:51<385:21:28,  8.39s/it]
 27%|██▋       | 59657/225000 [123:10:55<328:32:47,  7.15s/it]
 27%|██▋       | 59658/225000 [123:11:06<369:42:46,  8.05s/it]
 27%|██▋       | 59659/225000 [123:11:09<311:29:54,  6.78s/it]
 27%|██▋       | 59660/225000 [123:11:19<347:51:13,  7.57s/it]
 27%|██▋       | 59661/225000 [123:11:23<301:18:41,  6.56s/it]
 27%|██▋       | 59662/225000 [123:11:32<336:21:19,  7.32s/it]
 27%|██▋       | 59663/225000 [123:11:36<292:37:06,  6.37s/it]
 27%|██▋       | 59664/225000 [123:11:46<339:01:45,  7.38s/it]
 27%|██▋       | 59665/225000 [123:11:52<314:41:37,  6.85s/it]
 27%|██▋       | 59666/225000 [123:11:56<282:37:45,  6.15s/it]
 27%|██▋       | 59667/225000 [123:12:05<321:50:39,  7.01s/it]
 27%|██▋       | 59668/225000 [123:12:11<311:26:55,  6.78s/it]
 27%|██▋       | 59669/225000 [123:12:17<289:36:01,  6.31s/it]
 27%|██▋       | 59670/225000 [123:12:25<312:26:12,  6.80s/it]
 27%|██▋       | 59671/225000 [123:12:33<332:18:29,  7.24s/it]
 27%|██▋       | 59672/225000 [123:12:38<305:42:46,  6.66s/it]
 27%|██▋       | 59673/225000 [123:12:44<289:49:30,  6.31s/it]
 27%|██▋       | 59674/225000 [123:12:49<279:25:49,  6.08s/it]
 27%|██▋       | 59675/225000 [123:12:54<264:57:30,  5.77s/it]
 27%|██▋       | 59676/225000 [123:13:02<292:33:43,  6.37s/it]
 27%|██▋       | 59677/225000 [123:13:14<376:18:55,  8.19s/it]
 27%|██▋       | 59678/225000 [123:13:20<338:55:56,  7.38s/it]
 27%|██▋       | 59679/225000 [123:13:25<307:02:52,  6.69s/it]
 27%|██▋       | 59680/225000 [123:13:32<316:21:23,  6.89s/it]
 27%|██▋       | 59681/225000 [123:13:37<280:55:41,  6.12s/it]
 27%|██▋       | 59682/225000 [123:13:41<259:45:26,  5.66s/it]
 27%|██▋       | 59683/225000 [123:13:47<257:10:31,  5.60s/it]
 27%|██▋       | 59684/225000 [123:13:53<266:30:10,  5.80s/it]
 27%|██▋       | 59685/225000 [123:13:59<267:19:48,  5.82s/it]
 27%|██▋       | 59686/225000 [123:14:05<273:49:28,  5.96s/it]
 27%|██▋       | 59687/225000 [123:14:11<275:19:13,  6.00s/it]
 27%|██▋       | 59688/225000 [123:14:17<276:57:55,  6.03s/it]
 27%|██▋       | 59689/225000 [123:14:30<362:14:27,  7.89s/it]
 27%|██▋       | 59690/225000 [123:14:39<376:20:55,  8.20s/it]
 27%|██▋       | 59691/225000 [123:14:44<335:20:36,  7.30s/it]
 27%|██▋       | 59692/225000 [123:14:49<311:27:05,  6.78s/it]
 27%|██▋       | 59693/225000 [123:14:57<319:14:23,  6.95s/it]
 27%|██▋       | 59694/225000 [123:15:02<302:48:17,  6.59s/it]
 27%|██▋       | 59695/225000 [123:15:08<283:59:31,  6.18s/it]
 27%|██▋       | 59696/225000 [123:15:13<271:34:38,  5.91s/it]
 27%|██▋       | 59697/225000 [123:15:18<262:03:43,  5.71s/it]
 27%|██▋       | 59698/225000 [123:15:24<261:32:48,  5.70s/it]
 27%|██▋       | 59699/225000 [123:15:32<289:37:40,  6.31s/it]
 27%|██▋       | 59700/225000 [123:15:45<391:22:38,  8.52s/it]
                                                              
{'loss': '0.7449', 'grad_norm': '0.6265', 'learning_rate': '4.37e-05', 'epoch': '0.2653', 'num_input_tokens_seen': 205265048, 'train_runtime': '4.437e+05', 'train_tokens_per_second': '462.6'}

 27%|██▋       | 59700/225000 [123:15:45<391:22:38,  8.52s/it]
 27%|██▋       | 59701/225000 [123:15:50<340:01:18,  7.41s/it]
 27%|██▋       | 59702/225000 [123:15:55<299:59:25,  6.53s/it]
 27%|██▋       | 59703/225000 [123:16:00<285:18:55,  6.21s/it]
 27%|██▋       | 59704/225000 [123:16:12<362:18:44,  7.89s/it]
 27%|██▋       | 59705/225000 [123:16:19<349:28:12,  7.61s/it]
 27%|██▋       | 59706/225000 [123:16:23<296:24:01,  6.46s/it]
 27%|██▋       | 59707/225000 [123:16:30<314:06:38,  6.84s/it]
 27%|██▋       | 59708/225000 [123:16:39<342:53:36,  7.47s/it]
 27%|██▋       | 59709/225000 [123:16:43<297:28:11,  6.48s/it]
 27%|██▋       | 59710/225000 [123:16:48<278:11:19,  6.06s/it]
 27%|██▋       | 59711/225000 [123:16:57<317:09:12,  6.91s/it]
 27%|██▋       | 59712/225000 [123:17:05<327:19:48,  7.13s/it]
 27%|██▋       | 59713/225000 [123:17:12<320:08:14,  6.97s/it]
 27%|██▋       | 59714/225000 [123:17:19<320:20:43,  6.98s/it]
 27%|██▋       | 59715/225000 [123:17:23<289:50:17,  6.31s/it]
 27%|██▋       | 59716/225000 [123:17:29<281:46:02,  6.14s/it]
 27%|██▋       | 59717/225000 [123:17:38<323:11:00,  7.04s/it]
 27%|██▋       | 59718/225000 [123:17:43<295:47:42,  6.44s/it]
 27%|██▋       | 59719/225000 [123:17:48<273:43:22,  5.96s/it]
 27%|██▋       | 59720/225000 [123:17:53<258:10:08,  5.62s/it]
 27%|██▋       | 59721/225000 [123:17:58<247:27:10,  5.39s/it]
 27%|██▋       | 59722/225000 [123:18:10<340:23:08,  7.41s/it]
 27%|██▋       | 59723/225000 [123:18:17<336:13:34,  7.32s/it]
 27%|██▋       | 59724/225000 [123:18:23<315:39:40,  6.88s/it]
 27%|██▋       | 59725/225000 [123:18:28<294:36:16,  6.42s/it]
 27%|██▋       | 59726/225000 [123:18:40<374:25:29,  8.16s/it]
 27%|██▋       | 59727/225000 [123:18:45<325:46:53,  7.10s/it]
 27%|██▋       | 59728/225000 [123:18:53<336:07:34,  7.32s/it]
 27%|██▋       | 59729/225000 [123:18:59<315:49:02,  6.88s/it]
 27%|██▋       | 59730/225000 [123:19:06<317:46:57,  6.92s/it]
 27%|██▋       | 59731/225000 [123:19:10<276:37:17,  6.03s/it]
 27%|██▋       | 59732/225000 [123:19:20<338:37:15,  7.38s/it]
 27%|██▋       | 59733/225000 [123:19:24<289:14:35,  6.30s/it]
 27%|██▋       | 59734/225000 [123:19:29<269:39:07,  5.87s/it]
 27%|██▋       | 59735/225000 [123:19:34<264:17:09,  5.76s/it]
 27%|██▋       | 59736/225000 [123:19:40<262:09:53,  5.71s/it]
 27%|██▋       | 59737/225000 [123:19:53<358:24:05,  7.81s/it]
 27%|██▋       | 59738/225000 [123:19:59<333:51:07,  7.27s/it]
 27%|██▋       | 59739/225000 [123:20:03<288:19:05,  6.28s/it]
 27%|██▋       | 59740/225000 [123:20:07<267:58:01,  5.84s/it]
 27%|██▋       | 59741/225000 [123:20:12<253:28:13,  5.52s/it]
 27%|██▋       | 59742/225000 [123:20:18<259:07:08,  5.64s/it]
 27%|██▋       | 59743/225000 [123:20:24<258:36:34,  5.63s/it]
 27%|██▋       | 59744/225000 [123:20:33<308:37:07,  6.72s/it]
 27%|██▋       | 59745/225000 [123:20:44<370:58:53,  8.08s/it]
 27%|██▋       | 59746/225000 [123:20:50<340:14:17,  7.41s/it]
 27%|██▋       | 59747/225000 [123:20:57<331:48:05,  7.23s/it]
 27%|██▋       | 59748/225000 [123:21:02<303:37:46,  6.61s/it]
 27%|██▋       | 59749/225000 [123:21:08<290:26:40,  6.33s/it]
 27%|██▋       | 59750/225000 [123:21:14<282:27:52,  6.15s/it]
                                                              
{'loss': '0.7286', 'grad_norm': '0.7963', 'learning_rate': '4.369e-05', 'epoch': '0.2656', 'num_input_tokens_seen': 205413968, 'train_runtime': '4.441e+05', 'train_tokens_per_second': '462.6'}

 27%|██▋       | 59750/225000 [123:21:14<282:27:52,  6.15s/it]
 27%|██▋       | 59751/225000 [123:21:19<272:39:09,  5.94s/it]
 27%|██▋       | 59752/225000 [123:21:24<261:05:28,  5.69s/it]
 27%|██▋       | 59753/225000 [123:21:37<360:07:49,  7.85s/it]
 27%|██▋       | 59754/225000 [123:21:43<341:46:35,  7.45s/it]
 27%|██▋       | 59755/225000 [123:21:48<296:05:49,  6.45s/it]
 27%|██▋       | 59756/225000 [123:21:54<293:46:11,  6.40s/it]
 27%|██▋       | 59757/225000 [123:22:06<372:36:32,  8.12s/it]
 27%|██▋       | 59758/225000 [123:22:10<314:09:13,  6.84s/it]
 27%|██▋       | 59759/225000 [123:22:18<335:46:33,  7.32s/it]
 27%|██▋       | 59760/225000 [123:22:24<306:47:39,  6.68s/it]
 27%|██▋       | 59761/225000 [123:22:29<295:11:46,  6.43s/it]
 27%|██▋       | 59762/225000 [123:22:34<271:21:17,  5.91s/it]
 27%|██▋       | 59763/225000 [123:22:39<253:04:26,  5.51s/it]
 27%|██▋       | 59764/225000 [123:22:52<366:49:43,  7.99s/it]
 27%|██▋       | 59765/225000 [123:22:56<308:34:05,  6.72s/it]
 27%|██▋       | 59766/225000 [123:23:05<341:40:24,  7.44s/it]
 27%|██▋       | 59767/225000 [123:23:09<295:41:47,  6.44s/it]
 27%|██▋       | 59768/225000 [123:23:14<273:03:42,  5.95s/it]
 27%|██▋       | 59769/225000 [123:23:20<274:59:19,  5.99s/it]
 27%|██▋       | 59770/225000 [123:23:30<323:40:04,  7.05s/it]
 27%|██▋       | 59771/225000 [123:23:35<302:47:20,  6.60s/it]
 27%|██▋       | 59772/225000 [123:23:43<311:37:10,  6.79s/it]
 27%|██▋       | 59773/225000 [123:23:49<305:51:30,  6.66s/it]
 27%|██▋       | 59774/225000 [123:23:56<308:50:51,  6.73s/it]
 27%|██▋       | 59775/225000 [123:24:03<313:40:10,  6.83s/it]
 27%|██▋       | 59776/225000 [123:24:09<299:35:01,  6.53s/it]
 27%|██▋       | 59777/225000 [123:24:14<278:41:56,  6.07s/it]
 27%|██▋       | 59778/225000 [123:24:19<264:52:25,  5.77s/it]
 27%|██▋       | 59779/225000 [123:24:24<251:52:55,  5.49s/it]
 27%|██▋       | 59780/225000 [123:24:29<246:58:48,  5.38s/it]
 27%|██▋       | 59781/225000 [123:24:35<253:18:13,  5.52s/it]
 27%|██▋       | 59782/225000 [123:24:48<365:25:16,  7.96s/it]
 27%|██▋       | 59783/225000 [123:24:54<333:05:41,  7.26s/it]
 27%|██▋       | 59784/225000 [123:25:02<337:56:09,  7.36s/it]
 27%|██▋       | 59785/225000 [123:25:10<352:26:50,  7.68s/it]
 27%|██▋       | 59786/225000 [123:25:14<306:29:08,  6.68s/it]
 27%|██▋       | 59787/225000 [123:25:24<348:20:50,  7.59s/it]
 27%|██▋       | 59788/225000 [123:25:27<290:14:13,  6.32s/it]
 27%|██▋       | 59789/225000 [123:25:36<317:07:15,  6.91s/it]
 27%|██▋       | 59790/225000 [123:25:40<284:11:21,  6.19s/it]
 27%|██▋       | 59791/225000 [123:25:45<264:50:32,  5.77s/it]
 27%|██▋       | 59792/225000 [123:25:57<347:27:37,  7.57s/it]
 27%|██▋       | 59793/225000 [123:26:00<289:37:52,  6.31s/it]
 27%|██▋       | 59794/225000 [123:26:05<270:50:58,  5.90s/it]
 27%|██▋       | 59795/225000 [123:26:10<252:49:19,  5.51s/it]
 27%|██▋       | 59796/225000 [123:26:17<282:06:15,  6.15s/it]
 27%|██▋       | 59797/225000 [123:26:30<372:25:45,  8.12s/it]
 27%|██▋       | 59798/225000 [123:26:34<312:50:07,  6.82s/it]
 27%|██▋       | 59799/225000 [123:26:39<286:29:52,  6.24s/it]
 27%|██▋       | 59800/225000 [123:26:44<278:27:04,  6.07s/it]
                                                              
{'loss': '0.7412', 'grad_norm': '0.839', 'learning_rate': '4.368e-05', 'epoch': '0.2658', 'num_input_tokens_seen': 205560576, 'train_runtime': '4.444e+05', 'train_tokens_per_second': '462.6'}

 27%|██▋       | 59800/225000 [123:26:44<278:27:04,  6.07s/it]
 27%|██▋       | 59801/225000 [123:26:53<320:28:14,  6.98s/it]
 27%|██▋       | 59802/225000 [123:27:04<363:13:54,  7.92s/it]
 27%|██▋       | 59803/225000 [123:27:08<309:57:21,  6.75s/it]
 27%|██▋       | 59804/225000 [123:27:13<297:02:59,  6.47s/it]
 27%|██▋       | 59805/225000 [123:27:18<264:37:15,  5.77s/it]
 27%|██▋       | 59806/225000 [123:27:23<258:44:39,  5.64s/it]
 27%|██▋       | 59807/225000 [123:27:33<326:27:44,  7.11s/it]
 27%|██▋       | 59808/225000 [123:27:38<296:04:52,  6.45s/it]
 27%|██▋       | 59809/225000 [123:27:43<271:48:56,  5.92s/it]
 27%|██▋       | 59810/225000 [123:27:49<269:54:02,  5.88s/it]
 27%|██▋       | 59811/225000 [123:28:01<362:07:15,  7.89s/it]
 27%|██▋       | 59812/225000 [123:28:06<311:16:44,  6.78s/it]
 27%|██▋       | 59813/225000 [123:28:10<276:52:09,  6.03s/it]
 27%|██▋       | 59814/225000 [123:28:22<360:17:02,  7.85s/it]
 27%|██▋       | 59815/225000 [123:28:26<312:57:22,  6.82s/it]
 27%|██▋       | 59816/225000 [123:28:31<276:36:37,  6.03s/it]
 27%|██▋       | 59817/225000 [123:28:36<265:28:02,  5.79s/it]
 27%|██▋       | 59818/225000 [123:28:41<260:51:18,  5.69s/it]
 27%|██▋       | 59819/225000 [123:28:55<372:04:10,  8.11s/it]
 27%|██▋       | 59820/225000 [123:29:02<360:11:27,  7.85s/it]
 27%|██▋       | 59821/225000 [123:29:10<353:51:37,  7.71s/it]
 27%|██▋       | 59822/225000 [123:29:14<304:25:52,  6.63s/it]
 27%|██▋       | 59823/225000 [123:29:23<333:55:17,  7.28s/it]
 27%|██▋       | 59824/225000 [123:29:28<303:48:47,  6.62s/it]
 27%|██▋       | 59825/225000 [123:29:33<282:07:14,  6.15s/it]
 27%|██▋       | 59826/225000 [123:29:37<259:45:28,  5.66s/it]
 27%|██▋       | 59827/225000 [123:29:50<354:53:43,  7.74s/it]
 27%|██▋       | 59828/225000 [123:29:54<300:49:03,  6.56s/it]
 27%|██▋       | 59829/225000 [123:30:04<348:38:17,  7.60s/it]
 27%|██▋       | 59830/225000 [123:30:08<297:48:31,  6.49s/it]
 27%|██▋       | 59831/225000 [123:30:11<263:11:50,  5.74s/it]
 27%|██▋       | 59832/225000 [123:30:19<291:52:01,  6.36s/it]
 27%|██▋       | 59833/225000 [123:30:27<314:27:03,  6.85s/it]
 27%|██▋       | 59834/225000 [123:30:32<283:29:57,  6.18s/it]
 27%|██▋       | 59835/225000 [123:30:37<264:37:45,  5.77s/it]
 27%|██▋       | 59836/225000 [123:30:51<375:45:23,  8.19s/it]
 27%|██▋       | 59837/225000 [123:30:54<315:09:31,  6.87s/it]
 27%|██▋       | 59838/225000 [123:31:04<350:04:43,  7.63s/it]
 27%|██▋       | 59839/225000 [123:31:09<310:33:32,  6.77s/it]
 27%|██▋       | 59840/225000 [123:31:13<277:34:16,  6.05s/it]
 27%|██▋       | 59841/225000 [123:31:18<265:54:04,  5.80s/it]
 27%|██▋       | 59842/225000 [123:31:23<257:55:02,  5.62s/it]
 27%|██▋       | 59843/225000 [123:31:35<341:30:27,  7.44s/it]
 27%|██▋       | 59844/225000 [123:31:40<314:28:43,  6.85s/it]
 27%|██▋       | 59845/225000 [123:31:46<290:00:00,  6.32s/it]
 27%|██▋       | 59846/225000 [123:31:57<357:45:23,  7.80s/it]
 27%|██▋       | 59847/225000 [123:32:01<305:48:29,  6.67s/it]
 27%|██▋       | 59848/225000 [123:32:05<268:13:17,  5.85s/it]
 27%|██▋       | 59849/225000 [123:32:15<328:15:59,  7.16s/it]
 27%|██▋       | 59850/225000 [123:32:20<296:31:46,  6.46s/it]
                                                              
{'loss': '0.7375', 'grad_norm': '0.7294', 'learning_rate': '4.366e-05', 'epoch': '0.266', 'num_input_tokens_seen': 205708904, 'train_runtime': '4.447e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 59850/225000 [123:32:20<296:31:46,  6.46s/it]
 27%|██▋       | 59851/225000 [123:32:25<279:20:44,  6.09s/it]
 27%|██▋       | 59852/225000 [123:32:30<264:01:07,  5.76s/it]
 27%|██▋       | 59853/225000 [123:32:38<289:53:29,  6.32s/it]
 27%|██▋       | 59854/225000 [123:32:49<355:25:08,  7.75s/it]
 27%|██▋       | 59855/225000 [123:32:53<308:41:41,  6.73s/it]
 27%|██▋       | 59856/225000 [123:32:57<273:03:44,  5.95s/it]
 27%|██▋       | 59857/225000 [123:33:09<353:09:07,  7.70s/it]
 27%|██▋       | 59858/225000 [123:33:15<324:38:31,  7.08s/it]
 27%|██▋       | 59859/225000 [123:33:19<293:01:42,  6.39s/it]
 27%|██▋       | 59860/225000 [123:33:29<335:22:24,  7.31s/it]
 27%|██▋       | 59861/225000 [123:33:35<314:01:42,  6.85s/it]
 27%|██▋       | 59862/225000 [123:33:40<297:33:48,  6.49s/it]
 27%|██▋       | 59863/225000 [123:33:45<274:08:21,  5.98s/it]
 27%|██▋       | 59864/225000 [123:33:51<272:27:23,  5.94s/it]
 27%|██▋       | 59865/225000 [123:34:01<323:03:22,  7.04s/it]
 27%|██▋       | 59866/225000 [123:34:07<315:33:14,  6.88s/it]
 27%|██▋       | 59867/225000 [123:34:12<290:14:11,  6.33s/it]
 27%|██▋       | 59868/225000 [123:34:18<283:08:30,  6.17s/it]
 27%|██▋       | 59869/225000 [123:34:23<271:21:03,  5.92s/it]
 27%|██▋       | 59870/225000 [123:34:28<260:56:01,  5.69s/it]
 27%|██▋       | 59871/225000 [123:34:42<367:39:48,  8.02s/it]
 27%|██▋       | 59872/225000 [123:34:46<319:55:42,  6.97s/it]
 27%|██▋       | 59873/225000 [123:34:55<344:14:25,  7.50s/it]
 27%|██▋       | 59874/225000 [123:35:02<332:07:50,  7.24s/it]
 27%|██▋       | 59875/225000 [123:35:08<317:13:26,  6.92s/it]
 27%|██▋       | 59876/225000 [123:35:18<357:04:04,  7.78s/it]
 27%|██▋       | 59877/225000 [123:35:22<303:50:56,  6.62s/it]
 27%|██▋       | 59878/225000 [123:35:26<279:14:15,  6.09s/it]
 27%|██▋       | 59879/225000 [123:35:32<275:18:05,  6.00s/it]
 27%|██▋       | 59880/225000 [123:35:38<272:22:15,  5.94s/it]
 27%|██▋       | 59881/225000 [123:35:48<325:49:03,  7.10s/it]
 27%|██▋       | 59882/225000 [123:35:54<317:24:10,  6.92s/it]
 27%|██▋       | 59883/225000 [123:36:01<319:35:05,  6.97s/it]
 27%|██▋       | 59884/225000 [123:36:07<301:53:20,  6.58s/it]
 27%|██▋       | 59885/225000 [123:36:11<271:39:57,  5.92s/it]
 27%|██▋       | 59886/225000 [123:36:18<283:24:57,  6.18s/it]
 27%|██▋       | 59887/225000 [123:36:28<338:10:53,  7.37s/it]
 27%|██▋       | 59888/225000 [123:36:36<338:52:59,  7.39s/it]
 27%|██▋       | 59889/225000 [123:36:40<299:07:12,  6.52s/it]
 27%|██▋       | 59890/225000 [123:36:48<319:50:10,  6.97s/it]
 27%|██▋       | 59891/225000 [123:36:54<301:43:51,  6.58s/it]
 27%|██▋       | 59892/225000 [123:36:59<279:08:34,  6.09s/it]
 27%|██▋       | 59893/225000 [123:37:09<331:46:12,  7.23s/it]
 27%|██▋       | 59894/225000 [123:37:16<336:20:10,  7.33s/it]
 27%|██▋       | 59895/225000 [123:37:23<324:44:10,  7.08s/it]
 27%|██▋       | 59896/225000 [123:37:30<330:36:38,  7.21s/it]
 27%|██▋       | 59897/225000 [123:37:37<316:12:04,  6.89s/it]
 27%|██▋       | 59898/225000 [123:37:42<290:18:36,  6.33s/it]
 27%|██▋       | 59899/225000 [123:37:47<272:23:59,  5.94s/it]
 27%|██▋       | 59900/225000 [123:37:52<263:09:57,  5.74s/it]
                                                              
{'loss': '0.7375', 'grad_norm': '0.647', 'learning_rate': '4.365e-05', 'epoch': '0.2662', 'num_input_tokens_seen': 205858520, 'train_runtime': '4.451e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 59900/225000 [123:37:52<263:09:57,  5.74s/it]
 27%|██▋       | 59901/225000 [123:37:57<251:43:52,  5.49s/it]
 27%|██▋       | 59902/225000 [123:38:07<315:40:27,  6.88s/it]
 27%|██▋       | 59903/225000 [123:38:15<325:31:05,  7.10s/it]
 27%|██▋       | 59904/225000 [123:38:19<292:19:38,  6.37s/it]
 27%|██▋       | 59905/225000 [123:38:24<267:42:26,  5.84s/it]
 27%|██▋       | 59906/225000 [123:38:32<303:43:19,  6.62s/it]
 27%|██▋       | 59907/225000 [123:38:41<328:17:32,  7.16s/it]
 27%|██▋       | 59908/225000 [123:38:46<304:48:08,  6.65s/it]
 27%|██▋       | 59909/225000 [123:38:54<320:40:28,  6.99s/it]
 27%|██▋       | 59910/225000 [123:39:02<331:53:18,  7.24s/it]
 27%|██▋       | 59911/225000 [123:39:09<332:45:37,  7.26s/it]
 27%|██▋       | 59912/225000 [123:39:15<313:41:49,  6.84s/it]
 27%|██▋       | 59913/225000 [123:39:19<282:24:38,  6.16s/it]
 27%|██▋       | 59914/225000 [123:39:31<359:34:55,  7.84s/it]
 27%|██▋       | 59915/225000 [123:39:35<301:56:52,  6.58s/it]
 27%|██▋       | 59916/225000 [123:39:40<274:43:40,  5.99s/it]
 27%|██▋       | 59917/225000 [123:39:47<295:52:09,  6.45s/it]
 27%|██▋       | 59918/225000 [123:39:56<327:54:57,  7.15s/it]
 27%|██▋       | 59919/225000 [123:40:03<324:30:32,  7.08s/it]
 27%|██▋       | 59920/225000 [123:40:09<308:46:34,  6.73s/it]
 27%|██▋       | 59921/225000 [123:40:12<266:51:55,  5.82s/it]
 27%|██▋       | 59922/225000 [123:40:22<317:20:42,  6.92s/it]
 27%|██▋       | 59923/225000 [123:40:29<314:34:57,  6.86s/it]
 27%|██▋       | 59924/225000 [123:40:33<276:13:37,  6.02s/it]
 27%|██▋       | 59925/225000 [123:40:42<315:55:38,  6.89s/it]
 27%|██▋       | 59926/225000 [123:40:45<272:07:38,  5.93s/it]
 27%|██▋       | 59927/225000 [123:40:55<328:27:17,  7.16s/it]
 27%|██▋       | 59928/225000 [123:41:00<289:29:37,  6.31s/it]
 27%|██▋       | 59929/225000 [123:41:05<269:55:40,  5.89s/it]
 27%|██▋       | 59930/225000 [123:41:15<329:02:05,  7.18s/it]
 27%|██▋       | 59931/225000 [123:41:19<295:26:24,  6.44s/it]
 27%|██▋       | 59932/225000 [123:41:28<322:27:21,  7.03s/it]
 27%|██▋       | 59933/225000 [123:41:31<276:10:47,  6.02s/it]
 27%|██▋       | 59934/225000 [123:41:40<316:55:39,  6.91s/it]
 27%|██▋       | 59935/225000 [123:41:45<288:37:20,  6.29s/it]
 27%|██▋       | 59936/225000 [123:41:50<271:10:28,  5.91s/it]
 27%|██▋       | 59937/225000 [123:41:55<257:58:06,  5.63s/it]
 27%|██▋       | 59938/225000 [123:42:01<256:55:40,  5.60s/it]
 27%|██▋       | 59939/225000 [123:42:16<382:26:55,  8.34s/it]
 27%|██▋       | 59940/225000 [123:42:20<330:51:35,  7.22s/it]
 27%|██▋       | 59941/225000 [123:42:28<342:39:56,  7.47s/it]
 27%|██▋       | 59942/225000 [123:42:32<297:36:06,  6.49s/it]
 27%|██▋       | 59943/225000 [123:42:37<275:00:13,  6.00s/it]
 27%|██▋       | 59944/225000 [123:42:43<267:28:26,  5.83s/it]
 27%|██▋       | 59945/225000 [123:42:55<361:55:51,  7.89s/it]
 27%|██▋       | 59946/225000 [123:43:02<345:41:48,  7.54s/it]
 27%|██▋       | 59947/225000 [123:43:07<307:52:16,  6.72s/it]
 27%|██▋       | 59948/225000 [123:43:12<282:55:09,  6.17s/it]
 27%|██▋       | 59949/225000 [123:43:16<261:37:02,  5.71s/it]
 27%|██▋       | 59950/225000 [123:43:22<252:34:24,  5.51s/it]
                                                              
{'loss': '0.7613', 'grad_norm': '0.7795', 'learning_rate': '4.364e-05', 'epoch': '0.2664', 'num_input_tokens_seen': 206007768, 'train_runtime': '4.454e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 59950/225000 [123:43:22<252:34:24,  5.51s/it]
 27%|██▋       | 59951/225000 [123:43:27<250:02:50,  5.45s/it]
 27%|██▋       | 59952/225000 [123:43:38<333:35:25,  7.28s/it]
 27%|██▋       | 59953/225000 [123:43:46<335:01:50,  7.31s/it]
 27%|██▋       | 59954/225000 [123:43:50<291:14:11,  6.35s/it]
 27%|██▋       | 59955/225000 [123:43:55<274:38:51,  5.99s/it]
 27%|██▋       | 59956/225000 [123:44:08<370:45:00,  8.09s/it]
 27%|██▋       | 59957/225000 [123:44:12<314:46:29,  6.87s/it]
 27%|██▋       | 59958/225000 [123:44:20<331:46:20,  7.24s/it]
 27%|██▋       | 59959/225000 [123:44:27<332:23:26,  7.25s/it]
 27%|██▋       | 59960/225000 [123:44:32<294:45:59,  6.43s/it]
 27%|██▋       | 59961/225000 [123:44:37<271:21:15,  5.92s/it]
 27%|██▋       | 59962/225000 [123:44:47<336:10:28,  7.33s/it]
 27%|██▋       | 59963/225000 [123:44:55<341:10:00,  7.44s/it]
 27%|██▋       | 59964/225000 [123:45:00<311:47:45,  6.80s/it]
 27%|██▋       | 59965/225000 [123:45:08<324:03:12,  7.07s/it]
 27%|██▋       | 59966/225000 [123:45:14<305:17:38,  6.66s/it]
 27%|██▋       | 59967/225000 [123:45:21<319:40:18,  6.97s/it]
 27%|██▋       | 59968/225000 [123:45:30<344:37:21,  7.52s/it]
 27%|██▋       | 59969/225000 [123:45:34<299:33:49,  6.53s/it]
 27%|██▋       | 59970/225000 [123:45:41<293:56:13,  6.41s/it]
 27%|██▋       | 59971/225000 [123:45:51<345:43:47,  7.54s/it]
 27%|██▋       | 59972/225000 [123:45:55<299:36:28,  6.54s/it]
 27%|██▋       | 59973/225000 [123:46:06<359:04:35,  7.83s/it]
 27%|██▋       | 59974/225000 [123:46:09<297:27:11,  6.49s/it]
 27%|██▋       | 59975/225000 [123:46:17<321:00:08,  7.00s/it]
 27%|██▋       | 59976/225000 [123:46:26<342:15:36,  7.47s/it]
 27%|██▋       | 59977/225000 [123:46:30<294:42:07,  6.43s/it]
 27%|██▋       | 59978/225000 [123:46:34<257:20:23,  5.61s/it]
 27%|██▋       | 59979/225000 [123:46:39<248:11:08,  5.41s/it]
 27%|██▋       | 59980/225000 [123:46:44<243:51:07,  5.32s/it]
 27%|██▋       | 59981/225000 [123:46:49<246:34:41,  5.38s/it]
 27%|██▋       | 59982/225000 [123:46:55<247:38:32,  5.40s/it]
 27%|██▋       | 59983/225000 [123:47:09<365:07:09,  7.97s/it]
 27%|██▋       | 59984/225000 [123:47:14<329:46:55,  7.19s/it]
 27%|██▋       | 59985/225000 [123:47:20<311:37:06,  6.80s/it]
 27%|██▋       | 59986/225000 [123:47:28<333:33:56,  7.28s/it]
 27%|██▋       | 59987/225000 [123:47:33<297:15:19,  6.49s/it]
 27%|██▋       | 59988/225000 [123:47:38<272:35:22,  5.95s/it]
 27%|██▋       | 59989/225000 [123:47:50<356:05:07,  7.77s/it]
 27%|██▋       | 59990/225000 [123:47:54<313:48:15,  6.85s/it]
 27%|██▋       | 59991/225000 [123:47:59<289:41:22,  6.32s/it]
 27%|██▋       | 59992/225000 [123:48:07<310:24:32,  6.77s/it]
 27%|██▋       | 59993/225000 [123:48:15<330:45:40,  7.22s/it]
 27%|██▋       | 59994/225000 [123:48:22<328:24:16,  7.16s/it]
 27%|██▋       | 59995/225000 [123:48:27<288:10:55,  6.29s/it]
 27%|██▋       | 59996/225000 [123:48:31<261:39:35,  5.71s/it]
 27%|██▋       | 59997/225000 [123:48:43<346:42:58,  7.56s/it]
 27%|██▋       | 59998/225000 [123:48:48<307:28:58,  6.71s/it]
 27%|██▋       | 59999/225000 [123:48:52<280:33:22,  6.12s/it]
 27%|██▋       | 60000/225000 [123:48:57<262:08:20,  5.72s/it]
                                                              
{'loss': '0.7458', 'grad_norm': '0.7661', 'learning_rate': '4.363e-05', 'epoch': '0.2667', 'num_input_tokens_seen': 206156992, 'train_runtime': '4.457e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 60000/225000 [123:48:57<262:08:20,  5.72s/it]
 27%|██▋       | 60001/225000 [123:49:09<350:34:33,  7.65s/it]
 27%|██▋       | 60002/225000 [123:49:13<300:50:44,  6.56s/it]
 27%|██▋       | 60003/225000 [123:49:20<298:38:15,  6.52s/it]
 27%|██▋       | 60004/225000 [123:49:27<313:20:09,  6.84s/it]
 27%|██▋       | 60005/225000 [123:49:33<298:22:55,  6.51s/it]
 27%|██▋       | 60006/225000 [123:49:43<341:48:38,  7.46s/it]
 27%|██▋       | 60007/225000 [123:49:47<290:45:38,  6.34s/it]
 27%|██▋       | 60008/225000 [123:49:54<304:48:57,  6.65s/it]
 27%|██▋       | 60009/225000 [123:50:02<318:37:45,  6.95s/it]
 27%|██▋       | 60010/225000 [123:50:06<284:20:55,  6.20s/it]
 27%|██▋       | 60011/225000 [123:50:17<355:10:59,  7.75s/it]
 27%|██▋       | 60012/225000 [123:50:24<342:54:29,  7.48s/it]
 27%|██▋       | 60013/225000 [123:50:28<294:46:02,  6.43s/it]
 27%|██▋       | 60014/225000 [123:50:37<321:16:44,  7.01s/it]
 27%|██▋       | 60015/225000 [123:50:42<293:36:08,  6.41s/it]
 27%|██▋       | 60016/225000 [123:50:49<302:28:42,  6.60s/it]
 27%|██▋       | 60017/225000 [123:50:55<296:21:19,  6.47s/it]
 27%|██▋       | 60018/225000 [123:50:59<267:58:14,  5.85s/it]
 27%|██▋       | 60019/225000 [123:51:05<260:37:47,  5.69s/it]
 27%|██▋       | 60020/225000 [123:51:09<250:38:28,  5.47s/it]
 27%|██▋       | 60021/225000 [123:51:15<247:48:31,  5.41s/it]
 27%|██▋       | 60022/225000 [123:51:27<340:11:03,  7.42s/it]
 27%|██▋       | 60023/225000 [123:51:35<344:18:01,  7.51s/it]
 27%|██▋       | 60024/225000 [123:51:41<328:13:44,  7.16s/it]
 27%|██▋       | 60025/225000 [123:51:46<304:15:01,  6.64s/it]
 27%|██▋       | 60026/225000 [123:51:52<290:57:39,  6.35s/it]
 27%|██▋       | 60027/225000 [123:51:58<281:01:45,  6.13s/it]
 27%|██▋       | 60028/225000 [123:52:03<273:52:01,  5.98s/it]
 27%|██▋       | 60029/225000 [123:52:09<266:31:55,  5.82s/it]
 27%|██▋       | 60030/225000 [123:52:14<265:12:58,  5.79s/it]
 27%|██▋       | 60031/225000 [123:52:19<254:52:40,  5.56s/it]
 27%|██▋       | 60032/225000 [123:52:33<360:48:15,  7.87s/it]
 27%|██▋       | 60033/225000 [123:52:42<377:46:38,  8.24s/it]
 27%|██▋       | 60034/225000 [123:52:46<328:12:52,  7.16s/it]
 27%|██▋       | 60035/225000 [123:52:51<292:41:32,  6.39s/it]
 27%|██▋       | 60036/225000 [123:52:56<275:10:03,  6.00s/it]
 27%|██▋       | 60037/225000 [123:53:08<348:43:26,  7.61s/it]
 27%|██▋       | 60038/225000 [123:53:14<326:51:53,  7.13s/it]
 27%|██▋       | 60039/225000 [123:53:23<363:49:09,  7.94s/it]
 27%|██▋       | 60040/225000 [123:53:31<358:16:53,  7.82s/it]
 27%|██▋       | 60041/225000 [123:53:35<303:57:43,  6.63s/it]
 27%|██▋       | 60042/225000 [123:53:44<335:30:32,  7.32s/it]
 27%|██▋       | 60043/225000 [123:53:49<304:25:57,  6.64s/it]
 27%|██▋       | 60044/225000 [123:53:56<311:57:45,  6.81s/it]
 27%|██▋       | 60045/225000 [123:54:01<286:08:53,  6.24s/it]
 27%|██▋       | 60046/225000 [123:54:07<279:34:41,  6.10s/it]
 27%|██▋       | 60047/225000 [123:54:17<338:52:21,  7.40s/it]
 27%|██▋       | 60048/225000 [123:54:21<296:28:30,  6.47s/it]
 27%|██▋       | 60049/225000 [123:54:30<324:41:34,  7.09s/it]
 27%|██▋       | 60050/225000 [123:54:35<302:54:01,  6.61s/it]
                                                              
{'loss': '0.7521', 'grad_norm': '0.8908', 'learning_rate': '4.361e-05', 'epoch': '0.2669', 'num_input_tokens_seen': 206306512, 'train_runtime': '4.461e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 60050/225000 [123:54:35<302:54:01,  6.61s/it]
 27%|██▋       | 60051/225000 [123:54:43<319:18:56,  6.97s/it]
 27%|██▋       | 60052/225000 [123:54:48<292:50:03,  6.39s/it]
 27%|██▋       | 60053/225000 [123:54:55<291:09:55,  6.35s/it]
 27%|██▋       | 60054/225000 [123:55:00<284:25:13,  6.21s/it]
 27%|██▋       | 60055/225000 [123:55:06<277:11:48,  6.05s/it]
 27%|██▋       | 60056/225000 [123:55:16<334:36:03,  7.30s/it]
 27%|██▋       | 60057/225000 [123:55:20<291:08:38,  6.35s/it]
 27%|██▋       | 60058/225000 [123:55:31<342:11:34,  7.47s/it]
 27%|██▋       | 60059/225000 [123:55:36<310:10:52,  6.77s/it]
 27%|██▋       | 60060/225000 [123:55:41<294:19:22,  6.42s/it]
 27%|██▋       | 60061/225000 [123:55:47<284:22:35,  6.21s/it]
 27%|██▋       | 60062/225000 [123:55:52<273:19:32,  5.97s/it]
 27%|██▋       | 60063/225000 [123:55:58<271:12:17,  5.92s/it]
 27%|██▋       | 60064/225000 [123:56:04<263:07:06,  5.74s/it]
 27%|██▋       | 60065/225000 [123:56:17<366:25:33,  8.00s/it]
 27%|██▋       | 60066/225000 [123:56:23<337:26:17,  7.37s/it]
 27%|██▋       | 60067/225000 [123:56:27<294:49:34,  6.44s/it]
 27%|██▋       | 60068/225000 [123:56:34<304:00:03,  6.64s/it]
 27%|██▋       | 60069/225000 [123:56:44<343:38:27,  7.50s/it]
 27%|██▋       | 60070/225000 [123:56:48<307:12:34,  6.71s/it]
 27%|██▋       | 60071/225000 [123:56:53<274:28:13,  5.99s/it]
 27%|██▋       | 60072/225000 [123:56:58<262:50:55,  5.74s/it]
 27%|██▋       | 60073/225000 [123:57:11<361:05:32,  7.88s/it]
 27%|██▋       | 60074/225000 [123:57:18<351:46:18,  7.68s/it]
 27%|██▋       | 60075/225000 [123:57:22<299:57:45,  6.55s/it]
 27%|██▋       | 60076/225000 [123:57:32<343:27:11,  7.50s/it]
 27%|██▋       | 60077/225000 [123:57:36<306:01:40,  6.68s/it]
 27%|██▋       | 60078/225000 [123:57:41<278:44:51,  6.08s/it]
 27%|██▋       | 60079/225000 [123:57:46<263:51:01,  5.76s/it]
 27%|██▋       | 60080/225000 [123:57:58<348:07:46,  7.60s/it]
 27%|██▋       | 60081/225000 [123:58:02<298:33:05,  6.52s/it]
 27%|██▋       | 60082/225000 [123:58:10<323:21:47,  7.06s/it]
 27%|██▋       | 60083/225000 [123:58:18<326:33:50,  7.13s/it]
 27%|██▋       | 60084/225000 [123:58:24<320:36:36,  7.00s/it]
 27%|██▋       | 60085/225000 [123:58:32<332:53:38,  7.27s/it]
 27%|██▋       | 60086/225000 [123:58:38<320:05:38,  6.99s/it]
 27%|██▋       | 60087/225000 [123:58:43<287:39:20,  6.28s/it]
 27%|██▋       | 60088/225000 [123:58:52<329:59:22,  7.20s/it]
 27%|██▋       | 60089/225000 [123:58:59<323:22:21,  7.06s/it]
 27%|██▋       | 60090/225000 [123:59:06<316:42:43,  6.91s/it]
 27%|██▋       | 60091/225000 [123:59:12<303:10:55,  6.62s/it]
 27%|██▋       | 60092/225000 [123:59:21<334:29:49,  7.30s/it]
 27%|██▋       | 60093/225000 [123:59:28<339:47:54,  7.42s/it]
 27%|██▋       | 60094/225000 [123:59:33<299:37:26,  6.54s/it]
 27%|██▋       | 60095/225000 [123:59:38<277:00:17,  6.05s/it]
 27%|██▋       | 60096/225000 [123:59:42<253:19:58,  5.53s/it]
 27%|██▋       | 60097/225000 [123:59:54<345:26:03,  7.54s/it]
 27%|██▋       | 60098/225000 [123:59:59<304:39:28,  6.65s/it]
 27%|██▋       | 60099/225000 [124:00:08<340:16:10,  7.43s/it]
 27%|██▋       | 60100/225000 [124:00:16<342:20:30,  7.47s/it]
                                                              
{'loss': '0.7482', 'grad_norm': '0.8819', 'learning_rate': '4.36e-05', 'epoch': '0.2671', 'num_input_tokens_seen': 206456648, 'train_runtime': '4.464e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 60100/225000 [124:00:16<342:20:30,  7.47s/it]
 27%|██▋       | 60101/225000 [124:00:19<292:23:46,  6.38s/it]
 27%|██▋       | 60102/225000 [124:00:29<330:27:56,  7.21s/it]
 27%|██▋       | 60103/225000 [124:00:33<292:59:00,  6.40s/it]
 27%|██▋       | 60104/225000 [124:00:38<269:50:56,  5.89s/it]
 27%|██▋       | 60105/225000 [124:00:43<260:34:34,  5.69s/it]
 27%|██▋       | 60106/225000 [124:00:48<254:22:28,  5.55s/it]
 27%|██▋       | 60107/225000 [124:00:53<249:41:50,  5.45s/it]
 27%|██▋       | 60108/225000 [124:00:59<253:59:06,  5.55s/it]
 27%|██▋       | 60109/225000 [124:01:05<252:14:57,  5.51s/it]
 27%|██▋       | 60110/225000 [124:01:17<343:59:31,  7.51s/it]
 27%|██▋       | 60111/225000 [124:01:24<334:40:44,  7.31s/it]
 27%|██▋       | 60112/225000 [124:01:35<389:13:38,  8.50s/it]
 27%|██▋       | 60113/225000 [124:01:38<319:07:08,  6.97s/it]
 27%|██▋       | 60114/225000 [124:01:46<333:35:38,  7.28s/it]
 27%|██▋       | 60115/225000 [124:01:50<286:47:46,  6.26s/it]
 27%|██▋       | 60116/225000 [124:02:01<342:12:20,  7.47s/it]
 27%|██▋       | 60117/225000 [124:02:06<310:33:10,  6.78s/it]
 27%|██▋       | 60118/225000 [124:02:12<309:07:44,  6.75s/it]
 27%|██▋       | 60119/225000 [124:02:17<281:35:21,  6.15s/it]
 27%|██▋       | 60120/225000 [124:02:21<257:03:57,  5.61s/it]
 27%|██▋       | 60121/225000 [124:02:26<243:04:51,  5.31s/it]
 27%|██▋       | 60122/225000 [124:02:40<359:04:44,  7.84s/it]
 27%|██▋       | 60123/225000 [124:02:44<302:07:58,  6.60s/it]
 27%|██▋       | 60124/225000 [124:02:51<309:16:29,  6.75s/it]
 27%|██▋       | 60125/225000 [124:02:57<297:27:19,  6.49s/it]
 27%|██▋       | 60126/225000 [124:03:06<340:26:34,  7.43s/it]
 27%|██▋       | 60127/225000 [124:03:11<308:06:09,  6.73s/it]
 27%|██▋       | 60128/225000 [124:03:16<285:10:20,  6.23s/it]
 27%|██▋       | 60129/225000 [124:03:27<345:05:38,  7.54s/it]
 27%|██▋       | 60130/225000 [124:03:31<294:18:42,  6.43s/it]
 27%|██▋       | 60131/225000 [124:03:37<286:53:11,  6.26s/it]
 27%|██▋       | 60132/225000 [124:03:47<345:13:46,  7.54s/it]
 27%|██▋       | 60133/225000 [124:03:52<303:23:05,  6.62s/it]
 27%|██▋       | 60134/225000 [124:03:56<277:45:06,  6.06s/it]
 27%|██▋       | 60135/225000 [124:04:05<317:07:08,  6.92s/it]
 27%|██▋       | 60136/225000 [124:04:12<308:05:17,  6.73s/it]
 27%|██▋       | 60137/225000 [124:04:21<350:32:32,  7.65s/it]
 27%|██▋       | 60138/225000 [124:04:26<303:15:31,  6.62s/it]
 27%|██▋       | 60139/225000 [124:04:35<339:47:02,  7.42s/it]
 27%|██▋       | 60140/225000 [124:04:39<296:12:29,  6.47s/it]
 27%|██▋       | 60141/225000 [124:04:47<317:15:21,  6.93s/it]
 27%|██▋       | 60142/225000 [124:04:51<275:54:39,  6.03s/it]
 27%|██▋       | 60143/225000 [124:05:02<342:37:17,  7.48s/it]
 27%|██▋       | 60144/225000 [124:05:06<299:35:41,  6.54s/it]
 27%|██▋       | 60145/225000 [124:05:15<334:09:02,  7.30s/it]
 27%|██▋       | 60146/225000 [124:05:20<301:13:55,  6.58s/it]
 27%|██▋       | 60147/225000 [124:05:28<322:25:16,  7.04s/it]
 27%|██▋       | 60148/225000 [124:05:32<275:52:27,  6.02s/it]
 27%|██▋       | 60149/225000 [124:05:37<261:11:58,  5.70s/it]
 27%|██▋       | 60150/225000 [124:05:42<257:41:26,  5.63s/it]
                                                              
{'loss': '0.743', 'grad_norm': '0.7545', 'learning_rate': '4.359e-05', 'epoch': '0.2673', 'num_input_tokens_seen': 206606088, 'train_runtime': '4.467e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 60150/225000 [124:05:42<257:41:26,  5.63s/it]
 27%|██▋       | 60151/225000 [124:05:48<260:16:37,  5.68s/it]
 27%|██▋       | 60152/225000 [124:06:02<371:43:42,  8.12s/it]
 27%|██▋       | 60153/225000 [124:06:10<364:44:39,  7.97s/it]
 27%|██▋       | 60154/225000 [124:06:14<309:18:20,  6.75s/it]
 27%|██▋       | 60155/225000 [124:06:23<349:39:56,  7.64s/it]
 27%|██▋       | 60156/225000 [124:06:27<301:06:10,  6.58s/it]
 27%|██▋       | 60157/225000 [124:06:37<336:44:35,  7.35s/it]
 27%|██▋       | 60158/225000 [124:06:43<322:13:38,  7.04s/it]
 27%|██▋       | 60159/225000 [124:06:51<339:44:53,  7.42s/it]
 27%|██▋       | 60160/225000 [124:06:55<291:50:13,  6.37s/it]
 27%|██▋       | 60161/225000 [124:07:04<330:08:33,  7.21s/it]
 27%|██▋       | 60162/225000 [124:07:09<297:03:46,  6.49s/it]
 27%|██▋       | 60163/225000 [124:07:15<285:15:50,  6.23s/it]
 27%|██▋       | 60164/225000 [124:07:27<364:35:01,  7.96s/it]
 27%|██▋       | 60165/225000 [124:07:31<311:29:59,  6.80s/it]
 27%|██▋       | 60166/225000 [124:07:41<353:46:27,  7.73s/it]
 27%|██▋       | 60167/225000 [124:07:45<303:07:58,  6.62s/it]
 27%|██▋       | 60168/225000 [124:07:52<316:25:12,  6.91s/it]
 27%|██▋       | 60169/225000 [124:07:59<319:42:05,  6.98s/it]
 27%|██▋       | 60170/225000 [124:08:05<298:42:10,  6.52s/it]
 27%|██▋       | 60171/225000 [124:08:12<302:43:57,  6.61s/it]
 27%|██▋       | 60172/225000 [124:08:16<273:22:32,  5.97s/it]
 27%|██▋       | 60173/225000 [124:08:23<288:58:28,  6.31s/it]
 27%|██▋       | 60174/225000 [124:08:33<339:27:14,  7.41s/it]
 27%|██▋       | 60175/225000 [124:08:37<287:35:21,  6.28s/it]
 27%|██▋       | 60176/225000 [124:08:46<325:13:27,  7.10s/it]
 27%|██▋       | 60177/225000 [124:08:50<289:52:57,  6.33s/it]
 27%|██▋       | 60178/225000 [124:08:55<269:40:12,  5.89s/it]
 27%|██▋       | 60179/225000 [124:09:00<255:21:47,  5.58s/it]
 27%|██▋       | 60180/225000 [124:09:05<247:53:34,  5.41s/it]
 27%|██▋       | 60181/225000 [124:09:11<248:50:30,  5.44s/it]
 27%|██▋       | 60182/225000 [124:09:23<337:11:06,  7.36s/it]
 27%|██▋       | 60183/225000 [124:09:30<334:02:07,  7.30s/it]
 27%|██▋       | 60184/225000 [124:09:34<297:57:52,  6.51s/it]
 27%|██▋       | 60185/225000 [124:09:43<324:33:00,  7.09s/it]
 27%|██▋       | 60186/225000 [124:09:49<308:11:38,  6.73s/it]
 27%|██▋       | 60187/225000 [124:09:54<285:26:10,  6.23s/it]
 27%|██▋       | 60188/225000 [124:10:01<293:29:15,  6.41s/it]
 27%|██▋       | 60189/225000 [124:10:07<290:05:59,  6.34s/it]
 27%|██▋       | 60190/225000 [124:10:12<275:26:50,  6.02s/it]
 27%|██▋       | 60191/225000 [124:10:24<354:53:57,  7.75s/it]
 27%|██▋       | 60192/225000 [124:10:31<345:17:08,  7.54s/it]
 27%|██▋       | 60193/225000 [124:10:35<300:04:56,  6.55s/it]
 27%|██▋       | 60194/225000 [124:10:45<351:19:40,  7.67s/it]
 27%|██▋       | 60195/225000 [124:10:52<337:42:48,  7.38s/it]
 27%|██▋       | 60196/225000 [124:10:59<330:45:29,  7.23s/it]
 27%|██▋       | 60197/225000 [124:11:06<324:56:56,  7.10s/it]
 27%|██▋       | 60198/225000 [124:11:11<301:09:10,  6.58s/it]
 27%|██▋       | 60199/225000 [124:11:18<300:12:10,  6.56s/it]
 27%|██▋       | 60200/225000 [124:11:22<274:36:40,  6.00s/it]
                                                              
{'loss': '0.7337', 'grad_norm': '0.7231', 'learning_rate': '4.358e-05', 'epoch': '0.2676', 'num_input_tokens_seen': 206756552, 'train_runtime': '4.471e+05', 'train_tokens_per_second': '462.5'}

 27%|██▋       | 60200/225000 [124:11:22<274:36:40,  6.00s/it]
 27%|██▋       | 60201/225000 [124:11:33<338:57:46,  7.40s/it]
 27%|██▋       | 60202/225000 [124:11:38<302:50:41,  6.62s/it]
 27%|██▋       | 60203/225000 [124:11:46<325:47:40,  7.12s/it]
 27%|██▋       | 60204/225000 [124:11:53<328:53:37,  7.18s/it]
 27%|██▋       | 60205/225000 [124:12:02<347:41:49,  7.60s/it]
 27%|██▋       | 60206/225000 [124:12:06<294:10:36,  6.43s/it]
 27%|██▋       | 60207/225000 [124:12:15<329:31:35,  7.20s/it]
 27%|██▋       | 60208/225000 [124:12:18<279:13:27,  6.10s/it]
 27%|██▋       | 60209/225000 [124:12:29<345:22:05,  7.54s/it]
 27%|██▋       | 60210/225000 [124:12:33<289:32:19,  6.33s/it]
 27%|██▋       | 60211/225000 [124:12:41<312:42:42,  6.83s/it]
 27%|██▋       | 60212/225000 [124:12:46<286:18:08,  6.25s/it]
 27%|██▋       | 60213/225000 [124:12:53<301:52:08,  6.59s/it]
 27%|██▋       | 60214/225000 [124:12:58<286:06:15,  6.25s/it]
 27%|██▋       | 60215/225000 [124:13:07<312:19:21,  6.82s/it]
 27%|██▋       | 60216/225000 [124:13:15<335:33:04,  7.33s/it]
 27%|██▋       | 60217/225000 [124:13:20<298:29:31,  6.52s/it]
 27%|██▋       | 60218/225000 [124:13:25<277:28:07,  6.06s/it]
 27%|██▋       | 60219/225000 [124:13:29<253:51:35,  5.55s/it]
 27%|██▋       | 60220/225000 [124:13:42<354:01:42,  7.73s/it]
 27%|██▋       | 60221/225000 [124:13:48<327:54:24,  7.16s/it]
 27%|██▋       | 60222/225000 [124:13:52<287:10:19,  6.27s/it]
 27%|██▋       | 60223/225000 [124:13:58<279:07:20,  6.10s/it]
 27%|██▋       | 60224/225000 [124:14:03<270:47:09,  5.92s/it]
 27%|██▋       | 60225/225000 [124:14:09<266:41:56,  5.83s/it]
 27%|██▋       | 60226/225000 [124:14:19<326:22:07,  7.13s/it]
 27%|██▋       | 60227/225000 [124:14:28<349:52:08,  7.64s/it]
 27%|██▋       | 60228/225000 [124:14:32<309:15:59,  6.76s/it]
 27%|██▋       | 60229/225000 [124:14:37<277:57:02,  6.07s/it]
 27%|██▋       | 60230/225000 [124:14:41<257:56:30,  5.64s/it]
 27%|██▋       | 60231/225000 [124:14:53<334:03:23,  7.30s/it]
 27%|██▋       | 60232/225000 [124:14:58<305:18:40,  6.67s/it]
 27%|██▋       | 60233/225000 [124:15:03<280:01:41,  6.12s/it]
 27%|██▋       | 60234/225000 [124:15:13<343:12:19,  7.50s/it]
 27%|██▋       | 60235/225000 [124:15:21<338:38:29,  7.40s/it]
 27%|██▋       | 60236/225000 [124:15:26<305:30:35,  6.68s/it]
 27%|██▋       | 60237/225000 [124:15:31<283:12:53,  6.19s/it]
 27%|██▋       | 60238/225000 [124:15:36<267:11:08,  5.84s/it]
 27%|██▋       | 60239/225000 [124:15:44<302:45:32,  6.62s/it]
 27%|██▋       | 60240/225000 [124:15:56<373:13:53,  8.16s/it]
 27%|██▋       | 60241/225000 [124:16:00<324:17:24,  7.09s/it]
 27%|██▋       | 60242/225000 [124:16:05<286:33:42,  6.26s/it]
 27%|██▋       | 60243/225000 [124:16:10<278:28:27,  6.08s/it]
 27%|██▋       | 60244/225000 [124:16:20<328:58:36,  7.19s/it]
 27%|██▋       | 60245/225000 [124:16:25<292:07:47,  6.38s/it]
 27%|██▋       | 60246/225000 [124:16:33<314:41:37,  6.88s/it]
 27%|██▋       | 60247/225000 [124:16:41<328:20:29,  7.17s/it]
 27%|██▋       | 60248/225000 [124:16:45<286:23:24,  6.26s/it]
 27%|██▋       | 60249/225000 [124:16:54<331:16:19,  7.24s/it]
 27%|██▋       | 60250/225000 [124:17:02<339:50:43,  7.43s/it]
                                                              
{'loss': '0.7353', 'grad_norm': '0.7496', 'learning_rate': '4.357e-05', 'epoch': '0.2678', 'num_input_tokens_seen': 206906088, 'train_runtime': '4.474e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60250/225000 [124:17:02<339:50:43,  7.43s/it]
 27%|██▋       | 60251/225000 [124:17:06<286:15:51,  6.26s/it]
 27%|██▋       | 60252/225000 [124:17:15<333:33:19,  7.29s/it]
 27%|██▋       | 60253/225000 [124:17:22<326:14:17,  7.13s/it]
 27%|██▋       | 60254/225000 [124:17:27<300:08:47,  6.56s/it]
 27%|██▋       | 60255/225000 [124:17:32<274:21:06,  6.00s/it]
 27%|██▋       | 60256/225000 [124:17:37<264:33:35,  5.78s/it]
 27%|██▋       | 60257/225000 [124:17:42<251:37:17,  5.50s/it]
 27%|██▋       | 60258/225000 [124:17:52<311:50:53,  6.81s/it]
 27%|██▋       | 60259/225000 [124:18:01<342:06:48,  7.48s/it]
 27%|██▋       | 60260/225000 [124:18:05<296:45:03,  6.48s/it]
 27%|██▋       | 60261/225000 [124:18:10<267:28:21,  5.85s/it]
 27%|██▋       | 60262/225000 [124:18:15<257:33:41,  5.63s/it]
 27%|██▋       | 60263/225000 [124:18:20<253:00:55,  5.53s/it]
 27%|██▋       | 60264/225000 [124:18:29<296:49:00,  6.49s/it]
 27%|██▋       | 60265/225000 [124:18:40<365:31:56,  7.99s/it]
 27%|██▋       | 60266/225000 [124:18:45<320:08:24,  7.00s/it]
 27%|██▋       | 60267/225000 [124:18:50<291:59:36,  6.38s/it]
 27%|██▋       | 60268/225000 [124:18:57<297:22:28,  6.50s/it]
 27%|██▋       | 60269/225000 [124:19:07<352:07:33,  7.70s/it]
 27%|██▋       | 60270/225000 [124:19:12<317:14:52,  6.93s/it]
 27%|██▋       | 60271/225000 [124:19:18<297:01:48,  6.49s/it]
 27%|██▋       | 60272/225000 [124:19:23<278:18:48,  6.08s/it]
 27%|██▋       | 60273/225000 [124:19:28<266:43:39,  5.83s/it]
 27%|██▋       | 60274/225000 [124:19:33<261:10:58,  5.71s/it]
 27%|██▋       | 60275/225000 [124:19:39<256:43:36,  5.61s/it]
 27%|██▋       | 60276/225000 [124:19:44<252:18:54,  5.51s/it]
 27%|██▋       | 60277/225000 [124:19:57<351:03:02,  7.67s/it]
 27%|██▋       | 60278/225000 [124:20:05<364:00:08,  7.96s/it]
 27%|██▋       | 60279/225000 [124:20:12<341:50:17,  7.47s/it]
 27%|██▋       | 60280/225000 [124:20:20<346:49:53,  7.58s/it]
 27%|██▋       | 60281/225000 [124:20:25<316:17:17,  6.91s/it]
 27%|██▋       | 60282/225000 [124:20:30<295:55:49,  6.47s/it]
 27%|██▋       | 60283/225000 [124:20:36<289:16:42,  6.32s/it]
 27%|██▋       | 60284/225000 [124:20:41<264:14:38,  5.78s/it]
 27%|██▋       | 60285/225000 [124:20:51<316:39:39,  6.92s/it]
 27%|██▋       | 60286/225000 [124:20:55<280:42:33,  6.14s/it]
 27%|██▋       | 60287/225000 [124:21:05<332:30:26,  7.27s/it]
 27%|██▋       | 60288/225000 [124:21:09<295:49:59,  6.47s/it]
 27%|██▋       | 60289/225000 [124:21:14<276:53:26,  6.05s/it]
 27%|██▋       | 60290/225000 [124:21:19<258:35:43,  5.65s/it]
 27%|██▋       | 60291/225000 [124:21:32<353:26:14,  7.72s/it]
 27%|██▋       | 60292/225000 [124:21:37<318:12:38,  6.96s/it]
 27%|██▋       | 60293/225000 [124:21:43<303:31:52,  6.63s/it]
 27%|██▋       | 60294/225000 [124:21:52<334:11:38,  7.30s/it]
 27%|██▋       | 60295/225000 [124:21:56<300:15:40,  6.56s/it]
 27%|██▋       | 60296/225000 [124:22:01<279:00:26,  6.10s/it]
 27%|██▋       | 60297/225000 [124:22:06<263:10:25,  5.75s/it]
 27%|██▋       | 60298/225000 [124:22:13<275:25:46,  6.02s/it]
 27%|██▋       | 60299/225000 [124:22:27<384:34:08,  8.41s/it]
 27%|██▋       | 60300/225000 [124:22:30<310:09:46,  6.78s/it]
                                                              
{'loss': '0.741', 'grad_norm': '0.8758', 'learning_rate': '4.355e-05', 'epoch': '0.268', 'num_input_tokens_seen': 207052880, 'train_runtime': '4.478e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60300/225000 [124:22:30<310:09:46,  6.78s/it]
 27%|██▋       | 60301/225000 [124:22:37<314:36:47,  6.88s/it]
 27%|██▋       | 60302/225000 [124:22:41<275:40:46,  6.03s/it]
 27%|██▋       | 60303/225000 [124:22:52<337:17:10,  7.37s/it]
 27%|██▋       | 60304/225000 [124:22:55<286:41:23,  6.27s/it]
 27%|██▋       | 60305/225000 [124:23:03<303:55:47,  6.64s/it]
 27%|██▋       | 60306/225000 [124:23:10<308:36:13,  6.75s/it]
 27%|██▋       | 60307/225000 [124:23:14<268:05:44,  5.86s/it]
 27%|██▋       | 60308/225000 [124:23:24<327:29:51,  7.16s/it]
 27%|██▋       | 60309/225000 [124:23:29<296:53:22,  6.49s/it]
 27%|██▋       | 60310/225000 [124:23:33<272:42:38,  5.96s/it]
 27%|██▋       | 60311/225000 [124:23:38<256:12:20,  5.60s/it]
 27%|██▋       | 60312/225000 [124:23:43<246:49:30,  5.40s/it]
 27%|██▋       | 60313/225000 [124:23:55<332:49:42,  7.28s/it]
 27%|██▋       | 60314/225000 [124:24:01<316:04:26,  6.91s/it]
 27%|██▋       | 60315/225000 [124:24:05<280:08:28,  6.12s/it]
 27%|██▋       | 60316/225000 [124:24:18<372:10:44,  8.14s/it]
 27%|██▋       | 60317/225000 [124:24:22<311:12:24,  6.80s/it]
 27%|██▋       | 60318/225000 [124:24:29<318:10:31,  6.96s/it]
 27%|██▋       | 60319/225000 [124:24:33<279:04:37,  6.10s/it]
 27%|██▋       | 60320/225000 [124:24:44<338:25:02,  7.40s/it]
 27%|██▋       | 60321/225000 [124:24:49<305:21:05,  6.68s/it]
 27%|██▋       | 60322/225000 [124:24:57<326:32:28,  7.14s/it]
 27%|██▋       | 60323/225000 [124:25:02<296:27:03,  6.48s/it]
 27%|██▋       | 60324/225000 [124:25:08<300:24:29,  6.57s/it]
 27%|██▋       | 60325/225000 [124:25:13<266:16:12,  5.82s/it]
 27%|██▋       | 60326/225000 [124:25:23<332:18:34,  7.26s/it]
 27%|██▋       | 60327/225000 [124:25:27<290:04:10,  6.34s/it]
 27%|██▋       | 60328/225000 [124:25:36<321:50:38,  7.04s/it]
 27%|██▋       | 60329/225000 [124:25:40<277:16:04,  6.06s/it]
 27%|██▋       | 60330/225000 [124:25:48<301:12:57,  6.59s/it]
 27%|██▋       | 60331/225000 [124:25:54<298:39:50,  6.53s/it]
 27%|██▋       | 60332/225000 [124:25:59<276:29:33,  6.04s/it]
 27%|██▋       | 60333/225000 [124:26:04<268:13:13,  5.86s/it]
 27%|██▋       | 60334/225000 [124:26:10<265:22:13,  5.80s/it]
 27%|██▋       | 60335/225000 [124:26:15<258:18:12,  5.65s/it]
 27%|██▋       | 60336/225000 [124:26:22<276:19:55,  6.04s/it]
 27%|██▋       | 60337/225000 [124:26:36<376:02:11,  8.22s/it]
 27%|██▋       | 60338/225000 [124:26:40<330:27:53,  7.22s/it]
 27%|██▋       | 60339/225000 [124:26:45<292:03:55,  6.39s/it]
 27%|██▋       | 60340/225000 [124:26:50<278:06:17,  6.08s/it]
 27%|██▋       | 60341/225000 [124:27:01<344:54:05,  7.54s/it]
 27%|██▋       | 60342/225000 [124:27:05<297:24:49,  6.50s/it]
 27%|██▋       | 60343/225000 [124:27:14<333:31:30,  7.29s/it]
 27%|██▋       | 60344/225000 [124:27:19<292:02:18,  6.39s/it]
 27%|██▋       | 60345/225000 [124:27:23<267:22:13,  5.85s/it]
 27%|██▋       | 60346/225000 [124:27:29<259:24:32,  5.67s/it]
 27%|██▋       | 60347/225000 [124:27:40<333:12:28,  7.29s/it]
 27%|██▋       | 60348/225000 [124:27:46<320:10:50,  7.00s/it]
 27%|██▋       | 60349/225000 [124:27:52<304:43:01,  6.66s/it]
 27%|██▋       | 60350/225000 [124:27:59<310:15:16,  6.78s/it]
                                                              
{'loss': '0.7396', 'grad_norm': '0.8927', 'learning_rate': '4.354e-05', 'epoch': '0.2682', 'num_input_tokens_seen': 207200952, 'train_runtime': '4.481e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60350/225000 [124:27:59<310:15:16,  6.78s/it]
 27%|██▋       | 60351/225000 [124:28:07<326:53:58,  7.15s/it]
 27%|██▋       | 60352/225000 [124:28:14<329:39:33,  7.21s/it]
 27%|██▋       | 60353/225000 [124:28:18<281:34:01,  6.16s/it]
 27%|██▋       | 60354/225000 [124:28:28<337:33:12,  7.38s/it]
 27%|██▋       | 60355/225000 [124:28:32<295:39:06,  6.46s/it]
 27%|██▋       | 60356/225000 [124:28:38<281:44:11,  6.16s/it]
 27%|██▋       | 60357/225000 [124:28:44<275:16:24,  6.02s/it]
 27%|██▋       | 60358/225000 [124:28:50<277:10:08,  6.06s/it]
 27%|██▋       | 60359/225000 [124:29:02<363:13:51,  7.94s/it]
 27%|██▋       | 60360/225000 [124:29:06<307:07:58,  6.72s/it]
 27%|██▋       | 60361/225000 [124:29:15<333:56:58,  7.30s/it]
 27%|██▋       | 60362/225000 [124:29:20<302:27:24,  6.61s/it]
 27%|██▋       | 60363/225000 [124:29:24<277:17:00,  6.06s/it]
 27%|██▋       | 60364/225000 [124:29:29<259:36:54,  5.68s/it]
 27%|██▋       | 60365/225000 [124:29:34<250:10:10,  5.47s/it]
 27%|██▋       | 60366/225000 [124:29:43<293:56:19,  6.43s/it]
 27%|██▋       | 60367/225000 [124:29:54<363:58:15,  7.96s/it]
 27%|██▋       | 60368/225000 [124:29:58<310:14:55,  6.78s/it]
 27%|██▋       | 60369/225000 [124:30:03<285:19:49,  6.24s/it]
 27%|██▋       | 60370/225000 [124:30:08<265:42:52,  5.81s/it]
 27%|██▋       | 60371/225000 [124:30:21<361:36:14,  7.91s/it]
 27%|██▋       | 60372/225000 [124:30:26<316:38:42,  6.92s/it]
 27%|██▋       | 60373/225000 [124:30:31<291:36:22,  6.38s/it]
 27%|██▋       | 60374/225000 [124:30:36<276:17:19,  6.04s/it]
 27%|██▋       | 60375/225000 [124:30:47<347:08:55,  7.59s/it]
 27%|██▋       | 60376/225000 [124:30:52<304:57:37,  6.67s/it]
 27%|██▋       | 60377/225000 [124:30:57<282:25:25,  6.18s/it]
 27%|██▋       | 60378/225000 [124:31:02<272:18:31,  5.95s/it]
 27%|██▋       | 60379/225000 [124:31:13<333:48:28,  7.30s/it]
 27%|██▋       | 60380/225000 [124:31:19<327:40:26,  7.17s/it]
 27%|██▋       | 60381/225000 [124:31:25<299:27:31,  6.55s/it]
 27%|██▋       | 60382/225000 [124:31:35<350:10:22,  7.66s/it]
 27%|██▋       | 60383/225000 [124:31:39<299:16:32,  6.54s/it]
 27%|██▋       | 60384/225000 [124:31:43<267:38:42,  5.85s/it]
 27%|██▋       | 60385/225000 [124:31:48<249:13:49,  5.45s/it]
 27%|██▋       | 60386/225000 [124:32:01<354:03:50,  7.74s/it]
 27%|██▋       | 60387/225000 [124:32:06<318:41:57,  6.97s/it]
 27%|██▋       | 60388/225000 [124:32:13<315:36:43,  6.90s/it]
 27%|██▋       | 60389/225000 [124:32:17<276:48:49,  6.05s/it]
 27%|██▋       | 60390/225000 [124:32:25<305:06:11,  6.67s/it]
 27%|██▋       | 60391/225000 [124:32:30<285:05:35,  6.23s/it]
 27%|██▋       | 60392/225000 [124:32:35<265:10:30,  5.80s/it]
 27%|██▋       | 60393/225000 [124:32:40<264:22:59,  5.78s/it]
 27%|██▋       | 60394/225000 [124:32:46<263:32:07,  5.76s/it]
 27%|██▋       | 60395/225000 [124:32:52<257:54:30,  5.64s/it]
 27%|██▋       | 60396/225000 [124:32:57<257:56:44,  5.64s/it]
 27%|██▋       | 60397/225000 [124:33:03<262:47:11,  5.75s/it]
 27%|██▋       | 60398/225000 [124:33:08<256:00:04,  5.60s/it]
 27%|██▋       | 60399/225000 [124:33:14<257:13:29,  5.63s/it]
 27%|██▋       | 60400/225000 [124:33:23<303:20:40,  6.63s/it]
                                                              
{'loss': '0.744', 'grad_norm': '0.8603', 'learning_rate': '4.353e-05', 'epoch': '0.2684', 'num_input_tokens_seen': 207350504, 'train_runtime': '4.484e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60400/225000 [124:33:23<303:20:40,  6.63s/it]
 27%|██▋       | 60401/225000 [124:33:36<388:53:50,  8.51s/it]
 27%|██▋       | 60402/225000 [124:33:40<334:05:17,  7.31s/it]
 27%|██▋       | 60403/225000 [124:33:50<364:27:51,  7.97s/it]
 27%|██▋       | 60404/225000 [124:33:54<310:42:20,  6.80s/it]
 27%|██▋       | 60405/225000 [124:34:00<302:34:02,  6.62s/it]
 27%|██▋       | 60406/225000 [124:34:07<309:09:56,  6.76s/it]
 27%|██▋       | 60407/225000 [124:34:14<313:32:39,  6.86s/it]
 27%|██▋       | 60408/225000 [124:34:19<284:57:44,  6.23s/it]
 27%|██▋       | 60409/225000 [124:34:24<265:35:44,  5.81s/it]
 27%|██▋       | 60410/225000 [124:34:30<261:46:34,  5.73s/it]
 27%|██▋       | 60411/225000 [124:34:41<337:08:18,  7.37s/it]
 27%|██▋       | 60412/225000 [124:34:44<286:33:23,  6.27s/it]
 27%|██▋       | 60413/225000 [124:34:53<317:43:37,  6.95s/it]
 27%|██▋       | 60414/225000 [124:35:00<321:09:22,  7.02s/it]
 27%|██▋       | 60415/225000 [124:35:05<290:52:39,  6.36s/it]
 27%|██▋       | 60416/225000 [124:35:10<268:27:04,  5.87s/it]
 27%|██▋       | 60417/225000 [124:35:14<252:10:15,  5.52s/it]
 27%|██▋       | 60418/225000 [124:35:20<248:54:28,  5.44s/it]
 27%|██▋       | 60419/225000 [124:35:27<274:36:24,  6.01s/it]
 27%|██▋       | 60420/225000 [124:35:40<376:20:24,  8.23s/it]
 27%|██▋       | 60421/225000 [124:35:46<344:00:48,  7.52s/it]
 27%|██▋       | 60422/225000 [124:35:50<293:54:42,  6.43s/it]
 27%|██▋       | 60423/225000 [124:35:59<330:48:44,  7.24s/it]
 27%|██▋       | 60424/225000 [124:36:03<276:57:13,  6.06s/it]
 27%|██▋       | 60425/225000 [124:36:11<311:30:13,  6.81s/it]
 27%|██▋       | 60426/225000 [124:36:15<274:13:36,  6.00s/it]
 27%|██▋       | 60427/225000 [124:36:20<260:56:01,  5.71s/it]
 27%|██▋       | 60428/225000 [124:36:25<248:07:42,  5.43s/it]
 27%|██▋       | 60429/225000 [124:36:36<317:18:34,  6.94s/it]
 27%|██▋       | 60430/225000 [124:36:46<371:08:27,  8.12s/it]
 27%|██▋       | 60431/225000 [124:36:51<318:05:38,  6.96s/it]
 27%|██▋       | 60432/225000 [124:37:00<350:05:56,  7.66s/it]
 27%|██▋       | 60433/225000 [124:37:05<314:28:59,  6.88s/it]
 27%|██▋       | 60434/225000 [124:37:10<286:48:53,  6.27s/it]
 27%|██▋       | 60435/225000 [124:37:15<268:13:08,  5.87s/it]
 27%|██▋       | 60436/225000 [124:37:22<282:16:37,  6.18s/it]
 27%|██▋       | 60437/225000 [124:37:32<331:56:51,  7.26s/it]
 27%|██▋       | 60438/225000 [124:37:36<298:54:22,  6.54s/it]
 27%|██▋       | 60439/225000 [124:37:42<280:20:03,  6.13s/it]
 27%|██▋       | 60440/225000 [124:37:46<262:34:26,  5.74s/it]
 27%|██▋       | 60441/225000 [124:37:52<255:30:36,  5.59s/it]
 27%|██▋       | 60442/225000 [124:38:05<362:12:48,  7.92s/it]
 27%|██▋       | 60443/225000 [124:38:11<331:53:30,  7.26s/it]
 27%|██▋       | 60444/225000 [124:38:18<326:44:56,  7.15s/it]
 27%|██▋       | 60445/225000 [124:38:22<293:38:52,  6.42s/it]
 27%|██▋       | 60446/225000 [124:38:28<279:42:32,  6.12s/it]
 27%|██▋       | 60447/225000 [124:38:32<259:54:40,  5.69s/it]
 27%|██▋       | 60448/225000 [124:38:45<360:32:43,  7.89s/it]
 27%|██▋       | 60449/225000 [124:38:50<316:46:29,  6.93s/it]
 27%|██▋       | 60450/225000 [124:38:58<330:01:18,  7.22s/it]
                                                              
{'loss': '0.7479', 'grad_norm': '0.8164', 'learning_rate': '4.352e-05', 'epoch': '0.2687', 'num_input_tokens_seen': 207499592, 'train_runtime': '4.487e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60450/225000 [124:38:58<330:01:18,  7.22s/it]
 27%|██▋       | 60451/225000 [124:39:05<330:26:14,  7.23s/it]
 27%|██▋       | 60452/225000 [124:39:12<327:33:01,  7.17s/it]
 27%|██▋       | 60453/225000 [124:39:20<336:44:42,  7.37s/it]
 27%|██▋       | 60454/225000 [124:39:25<296:44:14,  6.49s/it]
 27%|██▋       | 60455/225000 [124:39:29<266:55:35,  5.84s/it]
 27%|██▋       | 60456/225000 [124:39:34<251:02:12,  5.49s/it]
 27%|██▋       | 60457/225000 [124:39:43<310:48:51,  6.80s/it]
 27%|██▋       | 60458/225000 [124:39:49<296:26:10,  6.49s/it]
 27%|██▋       | 60459/225000 [124:39:53<265:15:04,  5.80s/it]
 27%|██▋       | 60460/225000 [124:40:00<272:44:50,  5.97s/it]
 27%|██▋       | 60461/225000 [124:40:09<313:42:24,  6.86s/it]
 27%|██▋       | 60462/225000 [124:40:16<312:56:47,  6.85s/it]
 27%|██▋       | 60463/225000 [124:40:19<273:21:42,  5.98s/it]
 27%|██▋       | 60464/225000 [124:40:28<314:11:38,  6.87s/it]
 27%|██▋       | 60465/225000 [124:40:35<303:53:32,  6.65s/it]
 27%|██▋       | 60466/225000 [124:40:39<272:55:01,  5.97s/it]
 27%|██▋       | 60467/225000 [124:40:44<266:31:10,  5.83s/it]
 27%|██▋       | 60468/225000 [124:40:50<260:42:36,  5.70s/it]
 27%|██▋       | 60469/225000 [124:41:02<353:01:05,  7.72s/it]
 27%|██▋       | 60470/225000 [124:41:08<324:21:17,  7.10s/it]
 27%|██▋       | 60471/225000 [124:41:13<295:43:57,  6.47s/it]
 27%|██▋       | 60472/225000 [124:41:21<317:41:26,  6.95s/it]
 27%|██▋       | 60473/225000 [124:41:25<283:39:19,  6.21s/it]
 27%|██▋       | 60474/225000 [124:41:30<261:56:25,  5.73s/it]
 27%|██▋       | 60475/225000 [124:41:39<304:13:56,  6.66s/it]
 27%|██▋       | 60476/225000 [124:41:49<355:03:10,  7.77s/it]
 27%|██▋       | 60477/225000 [124:41:56<342:08:16,  7.49s/it]
 27%|██▋       | 60478/225000 [124:42:01<305:03:36,  6.68s/it]
 27%|██▋       | 60479/225000 [124:42:06<278:18:35,  6.09s/it]
 27%|██▋       | 60480/225000 [124:42:11<261:55:39,  5.73s/it]
 27%|██▋       | 60481/225000 [124:42:15<247:32:54,  5.42s/it]
 27%|██▋       | 60482/225000 [124:42:20<243:57:31,  5.34s/it]
 27%|██▋       | 60483/225000 [124:42:29<289:04:32,  6.33s/it]
 27%|██▋       | 60484/225000 [124:42:40<354:09:09,  7.75s/it]
 27%|██▋       | 60485/225000 [124:42:44<301:57:23,  6.61s/it]
 27%|██▋       | 60486/225000 [124:42:53<334:41:07,  7.32s/it]
 27%|██▋       | 60487/225000 [124:42:58<305:09:30,  6.68s/it]
 27%|██▋       | 60488/225000 [124:43:03<281:33:33,  6.16s/it]
 27%|██▋       | 60489/225000 [124:43:08<268:08:21,  5.87s/it]
 27%|██▋       | 60490/225000 [124:43:13<256:24:10,  5.61s/it]
 27%|██▋       | 60491/225000 [124:43:23<314:16:17,  6.88s/it]
 27%|██▋       | 60492/225000 [124:43:31<324:24:48,  7.10s/it]
 27%|██▋       | 60493/225000 [124:43:36<303:56:47,  6.65s/it]
 27%|██▋       | 60494/225000 [124:43:42<290:02:43,  6.35s/it]
 27%|██▋       | 60495/225000 [124:43:48<279:34:40,  6.12s/it]
 27%|██▋       | 60496/225000 [124:43:53<269:59:02,  5.91s/it]
 27%|██▋       | 60497/225000 [124:43:59<265:44:13,  5.82s/it]
 27%|██▋       | 60498/225000 [124:44:04<258:08:49,  5.65s/it]
 27%|██▋       | 60499/225000 [124:44:10<262:31:12,  5.75s/it]
 27%|██▋       | 60500/225000 [124:44:21<341:51:48,  7.48s/it]
                                                              
{'loss': '0.7578', 'grad_norm': '1.005', 'learning_rate': '4.35e-05', 'epoch': '0.2689', 'num_input_tokens_seen': 207646616, 'train_runtime': '4.491e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60500/225000 [124:44:21<341:51:48,  7.48s/it]
 27%|██▋       | 60501/225000 [124:44:30<351:19:47,  7.69s/it]
 27%|██▋       | 60502/225000 [124:44:35<315:09:49,  6.90s/it]
 27%|██▋       | 60503/225000 [124:44:39<284:45:34,  6.23s/it]
 27%|██▋       | 60504/225000 [124:44:44<262:16:04,  5.74s/it]
 27%|██▋       | 60505/225000 [124:44:49<249:12:00,  5.45s/it]
 27%|██▋       | 60506/225000 [124:44:56<272:10:57,  5.96s/it]
 27%|██▋       | 60507/225000 [124:45:09<365:25:37,  8.00s/it]
 27%|██▋       | 60508/225000 [124:45:13<322:48:32,  7.06s/it]
 27%|██▋       | 60509/225000 [124:45:23<357:33:50,  7.83s/it]
 27%|██▋       | 60510/225000 [124:45:30<340:53:22,  7.46s/it]
 27%|██▋       | 60511/225000 [124:45:34<300:32:52,  6.58s/it]
 27%|██▋       | 60512/225000 [124:45:45<365:27:14,  8.00s/it]
 27%|██▋       | 60513/225000 [124:45:50<324:38:08,  7.11s/it]
 27%|██▋       | 60514/225000 [124:45:58<327:27:35,  7.17s/it]
 27%|██▋       | 60515/225000 [124:46:04<314:56:54,  6.89s/it]
 27%|██▋       | 60516/225000 [124:46:10<303:55:40,  6.65s/it]
 27%|██▋       | 60517/225000 [124:46:15<284:03:08,  6.22s/it]
 27%|██▋       | 60518/225000 [124:46:23<298:15:25,  6.53s/it]
 27%|██▋       | 60519/225000 [124:46:30<311:46:42,  6.82s/it]
 27%|██▋       | 60520/225000 [124:46:35<284:30:38,  6.23s/it]
 27%|██▋       | 60521/225000 [124:46:40<268:40:12,  5.88s/it]
 27%|██▋       | 60522/225000 [124:46:50<322:21:22,  7.06s/it]
 27%|██▋       | 60523/225000 [124:46:57<323:35:30,  7.08s/it]
 27%|██▋       | 60524/225000 [124:47:01<287:33:20,  6.29s/it]
 27%|██▋       | 60525/225000 [124:47:06<265:33:07,  5.81s/it]
 27%|██▋       | 60526/225000 [124:47:11<254:49:18,  5.58s/it]
 27%|██▋       | 60527/225000 [124:47:16<245:22:14,  5.37s/it]
 27%|██▋       | 60528/225000 [124:47:26<307:56:42,  6.74s/it]
 27%|██▋       | 60529/225000 [124:47:36<348:38:03,  7.63s/it]
 27%|██▋       | 60530/225000 [124:47:42<335:16:33,  7.34s/it]
 27%|██▋       | 60531/225000 [124:47:46<288:07:08,  6.31s/it]
 27%|██▋       | 60532/225000 [124:47:52<284:38:32,  6.23s/it]
 27%|██▋       | 60533/225000 [124:48:01<318:30:42,  6.97s/it]
 27%|██▋       | 60534/225000 [124:48:08<319:26:19,  6.99s/it]
 27%|██▋       | 60535/225000 [124:48:13<287:25:29,  6.29s/it]
 27%|██▋       | 60536/225000 [124:48:17<265:21:39,  5.81s/it]
 27%|██▋       | 60537/225000 [124:48:22<248:47:16,  5.45s/it]
 27%|██▋       | 60538/225000 [124:48:33<323:53:40,  7.09s/it]
 27%|██▋       | 60539/225000 [124:48:40<321:04:02,  7.03s/it]
 27%|██▋       | 60540/225000 [124:48:45<293:12:07,  6.42s/it]
 27%|██▋       | 60541/225000 [124:48:50<279:12:05,  6.11s/it]
 27%|██▋       | 60542/225000 [124:48:56<274:24:58,  6.01s/it]
 27%|██▋       | 60543/225000 [124:49:01<261:11:27,  5.72s/it]
 27%|██▋       | 60544/225000 [124:49:14<360:08:29,  7.88s/it]
 27%|██▋       | 60545/225000 [124:49:19<316:46:27,  6.93s/it]
 27%|██▋       | 60546/225000 [124:49:29<365:48:29,  8.01s/it]
 27%|██▋       | 60547/225000 [124:49:33<313:46:39,  6.87s/it]
 27%|██▋       | 60548/225000 [124:49:38<281:39:16,  6.17s/it]
 27%|██▋       | 60549/225000 [124:49:43<264:10:46,  5.78s/it]
 27%|██▋       | 60550/225000 [124:49:48<250:10:16,  5.48s/it]
                                                              
{'loss': '0.7512', 'grad_norm': '0.94', 'learning_rate': '4.349e-05', 'epoch': '0.2691', 'num_input_tokens_seen': 207794848, 'train_runtime': '4.494e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60550/225000 [124:49:48<250:10:16,  5.48s/it]
 27%|██▋       | 60551/225000 [124:49:58<320:05:51,  7.01s/it]
 27%|██▋       | 60552/225000 [124:50:05<323:36:17,  7.08s/it]
 27%|██▋       | 60553/225000 [124:50:14<338:44:49,  7.42s/it]
 27%|██▋       | 60554/225000 [124:50:19<318:06:45,  6.96s/it]
 27%|██▋       | 60555/225000 [124:50:24<281:39:08,  6.17s/it]
 27%|██▋       | 60556/225000 [124:50:29<264:31:17,  5.79s/it]
 27%|██▋       | 60557/225000 [124:50:33<243:21:01,  5.33s/it]
 27%|██▋       | 60558/225000 [124:50:46<350:16:19,  7.67s/it]
 27%|██▋       | 60559/225000 [124:50:52<323:14:32,  7.08s/it]
 27%|██▋       | 60560/225000 [124:50:57<294:04:39,  6.44s/it]
 27%|██▋       | 60561/225000 [124:51:02<277:40:52,  6.08s/it]
 27%|██▋       | 60562/225000 [124:51:07<265:30:15,  5.81s/it]
 27%|██▋       | 60563/225000 [124:51:13<262:24:06,  5.74s/it]
 27%|██▋       | 60564/225000 [124:51:18<250:02:37,  5.47s/it]
 27%|██▋       | 60565/225000 [124:51:30<350:28:03,  7.67s/it]
 27%|██▋       | 60566/225000 [124:51:35<311:25:47,  6.82s/it]
 27%|██▋       | 60567/225000 [124:51:40<288:37:55,  6.32s/it]
 27%|██▋       | 60568/225000 [124:51:46<276:45:52,  6.06s/it]
 27%|██▋       | 60569/225000 [124:51:51<267:20:16,  5.85s/it]
 27%|██▋       | 60570/225000 [124:51:56<258:18:51,  5.66s/it]
 27%|██▋       | 60571/225000 [124:52:03<269:27:12,  5.90s/it]
 27%|██▋       | 60572/225000 [124:52:16<374:40:37,  8.20s/it]
 27%|██▋       | 60573/225000 [124:52:22<344:26:58,  7.54s/it]
 27%|██▋       | 60574/225000 [124:52:30<351:38:13,  7.70s/it]
 27%|██▋       | 60575/225000 [124:52:37<337:00:22,  7.38s/it]
 27%|██▋       | 60576/225000 [124:52:44<329:08:58,  7.21s/it]
 27%|██▋       | 60577/225000 [124:52:50<313:49:03,  6.87s/it]
 27%|██▋       | 60578/225000 [124:52:56<307:09:54,  6.73s/it]
 27%|██▋       | 60579/225000 [124:53:00<268:04:18,  5.87s/it]
 27%|██▋       | 60580/225000 [124:53:04<244:06:34,  5.34s/it]
 27%|██▋       | 60581/225000 [124:53:16<332:55:41,  7.29s/it]
 27%|██▋       | 60582/225000 [124:53:20<290:36:24,  6.36s/it]
 27%|██▋       | 60583/225000 [124:53:28<313:45:15,  6.87s/it]
 27%|██▋       | 60584/225000 [124:53:32<265:03:27,  5.80s/it]
 27%|██▋       | 60585/225000 [124:53:41<310:23:04,  6.80s/it]
 27%|██▋       | 60586/225000 [124:53:46<282:27:33,  6.18s/it]
 27%|██▋       | 60587/225000 [124:53:56<335:00:11,  7.34s/it]
 27%|██▋       | 60588/225000 [124:54:00<288:37:06,  6.32s/it]
 27%|██▋       | 60589/225000 [124:54:08<318:25:33,  6.97s/it]
 27%|██▋       | 60590/225000 [124:54:15<323:23:53,  7.08s/it]
 27%|██▋       | 60591/225000 [124:54:22<314:01:50,  6.88s/it]
 27%|██▋       | 60592/225000 [124:54:27<294:19:23,  6.44s/it]
 27%|██▋       | 60593/225000 [124:54:35<317:51:25,  6.96s/it]
 27%|██▋       | 60594/225000 [124:54:40<281:14:08,  6.16s/it]
 27%|██▋       | 60595/225000 [124:54:51<344:55:57,  7.55s/it]
 27%|██▋       | 60596/225000 [124:54:55<300:09:18,  6.57s/it]
 27%|██▋       | 60597/225000 [124:54:59<264:20:06,  5.79s/it]
 27%|██▋       | 60598/225000 [124:55:08<310:45:00,  6.80s/it]
 27%|██▋       | 60599/225000 [124:55:12<273:35:56,  5.99s/it]
 27%|██▋       | 60600/225000 [124:55:21<311:33:29,  6.82s/it]
                                                              
{'loss': '0.7322', 'grad_norm': '0.8202', 'learning_rate': '4.348e-05', 'epoch': '0.2693', 'num_input_tokens_seen': 207941744, 'train_runtime': '4.497e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60600/225000 [124:55:21<311:33:29,  6.82s/it]
 27%|██▋       | 60601/225000 [124:55:25<271:14:30,  5.94s/it]
 27%|██▋       | 60602/225000 [124:55:34<315:31:49,  6.91s/it]
 27%|██▋       | 60603/225000 [124:55:38<271:13:50,  5.94s/it]
 27%|██▋       | 60604/225000 [124:55:47<321:18:42,  7.04s/it]
 27%|██▋       | 60605/225000 [124:55:53<300:05:46,  6.57s/it]
 27%|██▋       | 60606/225000 [124:55:58<281:17:57,  6.16s/it]
 27%|██▋       | 60607/225000 [124:56:10<365:24:28,  8.00s/it]
 27%|██▋       | 60608/225000 [124:56:14<308:01:08,  6.75s/it]
 27%|██▋       | 60609/225000 [124:56:21<309:50:38,  6.79s/it]
 27%|██▋       | 60610/225000 [124:56:27<303:18:51,  6.64s/it]
 27%|██▋       | 60611/225000 [124:56:32<274:36:50,  6.01s/it]
 27%|██▋       | 60612/225000 [124:56:37<267:50:01,  5.87s/it]
 27%|██▋       | 60613/225000 [124:56:49<350:13:38,  7.67s/it]
 27%|██▋       | 60614/225000 [124:56:53<292:01:31,  6.40s/it]
 27%|██▋       | 60615/225000 [124:57:00<313:51:46,  6.87s/it]
 27%|██▋       | 60616/225000 [124:57:09<335:30:34,  7.35s/it]
 27%|██▋       | 60617/225000 [124:57:13<292:00:55,  6.40s/it]
 27%|██▋       | 60618/225000 [124:57:18<267:20:46,  5.85s/it]
 27%|██▋       | 60619/225000 [124:57:23<261:28:18,  5.73s/it]
 27%|██▋       | 60620/225000 [124:57:28<248:33:44,  5.44s/it]
 27%|██▋       | 60621/225000 [124:57:41<346:22:21,  7.59s/it]
 27%|██▋       | 60622/225000 [124:57:47<329:58:25,  7.23s/it]
 27%|██▋       | 60623/225000 [124:57:54<335:04:13,  7.34s/it]
 27%|██▋       | 60624/225000 [124:57:59<294:08:11,  6.44s/it]
 27%|██▋       | 60625/225000 [124:58:04<273:39:44,  5.99s/it]
 27%|██▋       | 60626/225000 [124:58:08<255:04:57,  5.59s/it]
 27%|██▋       | 60627/225000 [124:58:13<237:24:32,  5.20s/it]
 27%|██▋       | 60628/225000 [124:58:24<323:19:12,  7.08s/it]
 27%|██▋       | 60629/225000 [124:58:32<329:03:56,  7.21s/it]
 27%|██▋       | 60630/225000 [124:58:36<283:32:40,  6.21s/it]
 27%|██▋       | 60631/225000 [124:58:46<340:25:21,  7.46s/it]
 27%|██▋       | 60632/225000 [124:58:49<284:26:42,  6.23s/it]
 27%|██▋       | 60633/225000 [124:58:57<297:38:17,  6.52s/it]
 27%|██▋       | 60634/225000 [124:59:02<282:38:22,  6.19s/it]
 27%|██▋       | 60635/225000 [124:59:10<309:15:23,  6.77s/it]
 27%|██▋       | 60636/225000 [124:59:15<284:03:10,  6.22s/it]
 27%|██▋       | 60637/225000 [124:59:23<309:02:18,  6.77s/it]
 27%|██▋       | 60638/225000 [124:59:30<311:50:50,  6.83s/it]
 27%|██▋       | 60639/225000 [124:59:34<278:01:21,  6.09s/it]
 27%|██▋       | 60640/225000 [124:59:39<256:10:36,  5.61s/it]
 27%|██▋       | 60641/225000 [124:59:45<266:17:48,  5.83s/it]
 27%|██▋       | 60642/225000 [124:59:56<337:22:18,  7.39s/it]
 27%|██▋       | 60643/225000 [125:00:00<283:56:30,  6.22s/it]
 27%|██▋       | 60644/225000 [125:00:08<309:42:06,  6.78s/it]
 27%|██▋       | 60645/225000 [125:00:12<269:55:51,  5.91s/it]
 27%|██▋       | 60646/225000 [125:00:17<255:41:20,  5.60s/it]
 27%|██▋       | 60647/225000 [125:00:22<247:27:54,  5.42s/it]
 27%|██▋       | 60648/225000 [125:00:35<350:14:50,  7.67s/it]
 27%|██▋       | 60649/225000 [125:00:41<336:14:30,  7.37s/it]
 27%|██▋       | 60650/225000 [125:00:49<348:04:02,  7.62s/it]
                                                              
{'loss': '0.7555', 'grad_norm': '0.7937', 'learning_rate': '4.347e-05', 'epoch': '0.2696', 'num_input_tokens_seen': 208089360, 'train_runtime': '4.5e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60650/225000 [125:00:49<348:04:02,  7.62s/it]
 27%|██▋       | 60651/225000 [125:00:54<302:42:18,  6.63s/it]
 27%|██▋       | 60652/225000 [125:00:58<271:06:03,  5.94s/it]
 27%|██▋       | 60653/225000 [125:01:05<279:23:04,  6.12s/it]
 27%|██▋       | 60654/225000 [125:01:15<339:41:34,  7.44s/it]
 27%|██▋       | 60655/225000 [125:01:21<320:38:11,  7.02s/it]
 27%|██▋       | 60656/225000 [125:01:27<302:38:53,  6.63s/it]
 27%|██▋       | 60657/225000 [125:01:31<274:25:07,  6.01s/it]
 27%|██▋       | 60658/225000 [125:01:36<256:21:14,  5.62s/it]
 27%|██▋       | 60659/225000 [125:01:41<243:55:50,  5.34s/it]
 27%|██▋       | 60660/225000 [125:01:47<253:43:14,  5.56s/it]
 27%|██▋       | 60661/225000 [125:02:00<363:10:48,  7.96s/it]
 27%|██▋       | 60662/225000 [125:02:04<307:30:11,  6.74s/it]
 27%|██▋       | 60663/225000 [125:02:08<267:11:36,  5.85s/it]
 27%|██▋       | 60664/225000 [125:02:20<345:17:31,  7.56s/it]
 27%|██▋       | 60665/225000 [125:02:25<314:37:29,  6.89s/it]
 27%|██▋       | 60666/225000 [125:02:30<289:06:15,  6.33s/it]
 27%|██▋       | 60667/225000 [125:02:36<277:37:46,  6.08s/it]
 27%|██▋       | 60668/225000 [125:02:41<265:47:02,  5.82s/it]
 27%|██▋       | 60669/225000 [125:02:47<265:31:30,  5.82s/it]
 27%|██▋       | 60670/225000 [125:02:52<262:17:01,  5.75s/it]
 27%|██▋       | 60671/225000 [125:03:03<336:46:48,  7.38s/it]
 27%|██▋       | 60672/225000 [125:03:10<333:19:12,  7.30s/it]
 27%|██▋       | 60673/225000 [125:03:15<302:34:27,  6.63s/it]
 27%|██▋       | 60674/225000 [125:03:20<276:02:47,  6.05s/it]
 27%|██▋       | 60675/225000 [125:03:26<267:24:24,  5.86s/it]
 27%|██▋       | 60676/225000 [125:03:32<279:19:04,  6.12s/it]
 27%|██▋       | 60677/225000 [125:03:45<373:09:15,  8.18s/it]
 27%|██▋       | 60678/225000 [125:03:49<309:28:16,  6.78s/it]
 27%|██▋       | 60679/225000 [125:03:59<354:03:58,  7.76s/it]
 27%|██▋       | 60680/225000 [125:04:03<299:36:47,  6.56s/it]
 27%|██▋       | 60681/225000 [125:04:11<328:32:35,  7.20s/it]
 27%|██▋       | 60682/225000 [125:04:16<292:16:55,  6.40s/it]
 27%|██▋       | 60683/225000 [125:04:22<289:45:05,  6.35s/it]
 27%|██▋       | 60684/225000 [125:04:26<259:52:11,  5.69s/it]
 27%|██▋       | 60685/225000 [125:04:36<321:08:43,  7.04s/it]
 27%|██▋       | 60686/225000 [125:04:42<307:36:23,  6.74s/it]
 27%|██▋       | 60687/225000 [125:04:48<297:44:50,  6.52s/it]
 27%|██▋       | 60688/225000 [125:04:53<270:53:39,  5.94s/it]
 27%|██▋       | 60689/225000 [125:05:01<298:20:38,  6.54s/it]
 27%|██▋       | 60690/225000 [125:05:06<270:46:39,  5.93s/it]
 27%|██▋       | 60691/225000 [125:05:10<253:46:58,  5.56s/it]
 27%|██▋       | 60692/225000 [125:05:19<294:48:00,  6.46s/it]
 27%|██▋       | 60693/225000 [125:05:26<305:39:02,  6.70s/it]
 27%|██▋       | 60694/225000 [125:05:33<307:13:47,  6.73s/it]
 27%|██▋       | 60695/225000 [125:05:37<274:01:41,  6.00s/it]
 27%|██▋       | 60696/225000 [125:05:42<259:21:08,  5.68s/it]
 27%|██▋       | 60697/225000 [125:05:47<249:27:24,  5.47s/it]
 27%|██▋       | 60698/225000 [125:06:00<357:26:45,  7.83s/it]
 27%|██▋       | 60699/225000 [125:06:04<297:48:24,  6.53s/it]
 27%|██▋       | 60700/225000 [125:06:10<290:08:51,  6.36s/it]
                                                              
{'loss': '0.7287', 'grad_norm': '0.7924', 'learning_rate': '4.345e-05', 'epoch': '0.2698', 'num_input_tokens_seen': 208237344, 'train_runtime': '4.504e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60700/225000 [125:06:10<290:08:51,  6.36s/it]
 27%|██▋       | 60701/225000 [125:06:17<295:58:06,  6.49s/it]
 27%|██▋       | 60702/225000 [125:06:23<289:53:32,  6.35s/it]
 27%|██▋       | 60703/225000 [125:06:29<284:06:11,  6.23s/it]
 27%|██▋       | 60704/225000 [125:06:34<269:03:11,  5.90s/it]
 27%|██▋       | 60705/225000 [125:06:40<268:44:24,  5.89s/it]
 27%|██▋       | 60706/225000 [125:06:47<284:36:05,  6.24s/it]
 27%|██▋       | 60707/225000 [125:06:57<336:40:50,  7.38s/it]
 27%|██▋       | 60708/225000 [125:07:02<311:35:08,  6.83s/it]
 27%|██▋       | 60709/225000 [125:07:07<288:47:07,  6.33s/it]
 27%|██▋       | 60710/225000 [125:07:13<283:58:21,  6.22s/it]
 27%|██▋       | 60711/225000 [125:07:19<274:50:05,  6.02s/it]
 27%|██▋       | 60712/225000 [125:07:25<271:50:39,  5.96s/it]
 27%|██▋       | 60713/225000 [125:07:30<268:49:42,  5.89s/it]
 27%|██▋       | 60714/225000 [125:07:39<298:49:49,  6.55s/it]
 27%|██▋       | 60715/225000 [125:07:52<388:59:47,  8.52s/it]
 27%|██▋       | 60716/225000 [125:07:56<329:08:40,  7.21s/it]
 27%|██▋       | 60717/225000 [125:08:01<304:48:14,  6.68s/it]
 27%|██▋       | 60718/225000 [125:08:06<275:18:42,  6.03s/it]
 27%|██▋       | 60719/225000 [125:08:11<267:48:10,  5.87s/it]
 27%|██▋       | 60720/225000 [125:08:17<261:20:04,  5.73s/it]
 27%|██▋       | 60721/225000 [125:08:29<355:12:44,  7.78s/it]
 27%|██▋       | 60722/225000 [125:08:35<328:23:58,  7.20s/it]
 27%|██▋       | 60723/225000 [125:08:43<343:59:03,  7.54s/it]
 27%|██▋       | 60724/225000 [125:08:47<293:00:07,  6.42s/it]
 27%|██▋       | 60725/225000 [125:08:51<262:01:32,  5.74s/it]
 27%|██▋       | 60726/225000 [125:08:57<257:05:34,  5.63s/it]
 27%|██▋       | 60727/225000 [125:09:09<349:46:19,  7.67s/it]
 27%|██▋       | 60728/225000 [125:09:14<310:43:21,  6.81s/it]
 27%|██▋       | 60729/225000 [125:09:19<292:20:53,  6.41s/it]
 27%|██▋       | 60730/225000 [125:09:30<347:57:37,  7.63s/it]
 27%|██▋       | 60731/225000 [125:09:36<332:19:25,  7.28s/it]
 27%|██▋       | 60732/225000 [125:09:40<285:10:26,  6.25s/it]
 27%|██▋       | 60733/225000 [125:09:45<261:38:35,  5.73s/it]
 27%|██▋       | 60734/225000 [125:09:55<316:47:50,  6.94s/it]
 27%|██▋       | 60735/225000 [125:10:01<311:26:36,  6.83s/it]
 27%|██▋       | 60736/225000 [125:10:07<293:40:28,  6.44s/it]
 27%|██▋       | 60737/225000 [125:10:17<345:15:28,  7.57s/it]
 27%|██▋       | 60738/225000 [125:10:21<301:16:24,  6.60s/it]
 27%|██▋       | 60739/225000 [125:10:26<271:41:25,  5.95s/it]
 27%|██▋       | 60740/225000 [125:10:30<248:42:16,  5.45s/it]
 27%|██▋       | 60741/225000 [125:10:41<324:14:41,  7.11s/it]
 27%|██▋       | 60742/225000 [125:10:48<329:26:35,  7.22s/it]
 27%|██▋       | 60743/225000 [125:10:53<292:34:25,  6.41s/it]
 27%|██▋       | 60744/225000 [125:10:58<270:13:42,  5.92s/it]
 27%|██▋       | 60745/225000 [125:11:03<257:30:14,  5.64s/it]
 27%|██▋       | 60746/225000 [125:11:08<259:59:01,  5.70s/it]
 27%|██▋       | 60747/225000 [125:11:22<363:44:10,  7.97s/it]
 27%|██▋       | 60748/225000 [125:11:26<312:14:43,  6.84s/it]
 27%|██▋       | 60749/225000 [125:11:34<325:18:11,  7.13s/it]
 27%|██▋       | 60750/225000 [125:11:41<332:09:31,  7.28s/it]
                                                              
{'loss': '0.731', 'grad_norm': '0.7304', 'learning_rate': '4.344e-05', 'epoch': '0.27', 'num_input_tokens_seen': 208386648, 'train_runtime': '4.507e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60750/225000 [125:11:41<332:09:31,  7.28s/it]
 27%|██▋       | 60751/225000 [125:11:45<285:37:13,  6.26s/it]
 27%|██▋       | 60752/225000 [125:11:53<305:48:52,  6.70s/it]
 27%|██▋       | 60753/225000 [125:11:57<275:49:01,  6.05s/it]
 27%|██▋       | 60754/225000 [125:12:08<332:02:34,  7.28s/it]
 27%|██▋       | 60755/225000 [125:12:14<318:22:55,  6.98s/it]
 27%|██▋       | 60756/225000 [125:12:19<292:08:26,  6.40s/it]
 27%|██▋       | 60757/225000 [125:12:23<254:24:51,  5.58s/it]
 27%|██▋       | 60758/225000 [125:12:33<324:39:58,  7.12s/it]
 27%|██▋       | 60759/225000 [125:12:37<278:22:00,  6.10s/it]
 27%|██▋       | 60760/225000 [125:12:45<300:18:58,  6.58s/it]
 27%|██▋       | 60761/225000 [125:12:51<294:00:58,  6.44s/it]
 27%|██▋       | 60762/225000 [125:12:56<279:20:42,  6.12s/it]
 27%|██▋       | 60763/225000 [125:13:02<267:18:07,  5.86s/it]
 27%|██▋       | 60764/225000 [125:13:07<261:24:23,  5.73s/it]
 27%|██▋       | 60765/225000 [125:13:13<263:31:21,  5.78s/it]
 27%|██▋       | 60766/225000 [125:13:18<260:34:57,  5.71s/it]
 27%|██▋       | 60767/225000 [125:13:24<256:32:17,  5.62s/it]
 27%|██▋       | 60768/225000 [125:13:29<257:10:35,  5.64s/it]
 27%|██▋       | 60769/225000 [125:13:38<295:28:00,  6.48s/it]
 27%|██▋       | 60770/225000 [125:13:50<378:55:25,  8.31s/it]
 27%|██▋       | 60771/225000 [125:13:55<320:00:58,  7.01s/it]
 27%|██▋       | 60772/225000 [125:14:01<313:30:49,  6.87s/it]
 27%|██▋       | 60773/225000 [125:14:09<330:46:27,  7.25s/it]
 27%|██▋       | 60774/225000 [125:14:13<286:33:28,  6.28s/it]
 27%|██▋       | 60775/225000 [125:14:18<263:29:50,  5.78s/it]
 27%|██▋       | 60776/225000 [125:14:24<275:50:34,  6.05s/it]
 27%|██▋       | 60777/225000 [125:14:37<359:04:03,  7.87s/it]
 27%|██▋       | 60778/225000 [125:14:41<316:56:00,  6.95s/it]
 27%|██▋       | 60779/225000 [125:14:46<279:01:19,  6.12s/it]
 27%|██▋       | 60780/225000 [125:14:56<332:48:58,  7.30s/it]
 27%|██▋       | 60781/225000 [125:15:00<293:38:21,  6.44s/it]
 27%|██▋       | 60782/225000 [125:15:10<339:14:50,  7.44s/it]
 27%|██▋       | 60783/225000 [125:15:14<287:58:08,  6.31s/it]
 27%|██▋       | 60784/225000 [125:15:23<329:27:30,  7.22s/it]
 27%|██▋       | 60785/225000 [125:15:29<320:32:55,  7.03s/it]
 27%|██▋       | 60786/225000 [125:15:36<312:18:09,  6.85s/it]
 27%|██▋       | 60787/225000 [125:15:42<308:54:56,  6.77s/it]
 27%|██▋       | 60788/225000 [125:15:49<312:06:49,  6.84s/it]
 27%|██▋       | 60789/225000 [125:15:54<282:09:30,  6.19s/it]
 27%|██▋       | 60790/225000 [125:15:59<258:04:34,  5.66s/it]
 27%|██▋       | 60791/225000 [125:16:03<247:30:33,  5.43s/it]
 27%|██▋       | 60792/225000 [125:16:15<325:48:56,  7.14s/it]
 27%|██▋       | 60793/225000 [125:16:18<280:31:22,  6.15s/it]
 27%|██▋       | 60794/225000 [125:16:28<321:26:00,  7.05s/it]
 27%|██▋       | 60795/225000 [125:16:32<286:39:42,  6.28s/it]
 27%|██▋       | 60796/225000 [125:16:39<301:53:09,  6.62s/it]
 27%|██▋       | 60797/225000 [125:16:44<275:54:27,  6.05s/it]
 27%|██▋       | 60798/225000 [125:16:53<319:57:35,  7.01s/it]
 27%|██▋       | 60799/225000 [125:17:00<317:13:44,  6.96s/it]
 27%|██▋       | 60800/225000 [125:17:05<283:16:19,  6.21s/it]
                                                              
{'loss': '0.7447', 'grad_norm': '0.9115', 'learning_rate': '4.343e-05', 'epoch': '0.2702', 'num_input_tokens_seen': 208534112, 'train_runtime': '4.51e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60800/225000 [125:17:05<283:16:19,  6.21s/it]
 27%|██▋       | 60801/225000 [125:17:09<263:22:55,  5.77s/it]
 27%|██▋       | 60802/225000 [125:17:15<255:05:41,  5.59s/it]
 27%|██▋       | 60803/225000 [125:17:21<270:22:20,  5.93s/it]
 27%|██▋       | 60804/225000 [125:17:32<337:28:08,  7.40s/it]
 27%|██▋       | 60805/225000 [125:17:37<302:07:45,  6.62s/it]
 27%|██▋       | 60806/225000 [125:17:42<273:24:42,  5.99s/it]
 27%|██▋       | 60807/225000 [125:17:46<248:27:55,  5.45s/it]
 27%|██▋       | 60808/225000 [125:17:57<333:35:02,  7.31s/it]
 27%|██▋       | 60809/225000 [125:18:04<327:24:22,  7.18s/it]
 27%|██▋       | 60810/225000 [125:18:11<318:02:32,  6.97s/it]
 27%|██▋       | 60811/225000 [125:18:15<286:31:47,  6.28s/it]
 27%|██▋       | 60812/225000 [125:18:25<327:39:47,  7.18s/it]
 27%|██▋       | 60813/225000 [125:18:29<294:21:51,  6.45s/it]
 27%|██▋       | 60814/225000 [125:18:35<285:56:11,  6.27s/it]
 27%|██▋       | 60815/225000 [125:18:45<330:18:31,  7.24s/it]
 27%|██▋       | 60816/225000 [125:18:49<294:12:00,  6.45s/it]
 27%|██▋       | 60817/225000 [125:18:54<274:00:10,  6.01s/it]
 27%|██▋       | 60818/225000 [125:18:59<260:12:48,  5.71s/it]
 27%|██▋       | 60819/225000 [125:19:05<260:26:42,  5.71s/it]
 27%|██▋       | 60820/225000 [125:19:10<247:08:07,  5.42s/it]
 27%|██▋       | 60821/225000 [125:19:19<303:51:03,  6.66s/it]
 27%|██▋       | 60822/225000 [125:19:29<347:00:21,  7.61s/it]
 27%|██▋       | 60823/225000 [125:19:33<300:20:35,  6.59s/it]
 27%|██▋       | 60824/225000 [125:19:39<283:01:14,  6.21s/it]
 27%|██▋       | 60825/225000 [125:19:47<304:38:14,  6.68s/it]
 27%|██▋       | 60826/225000 [125:19:51<278:17:23,  6.10s/it]
 27%|██▋       | 60827/225000 [125:20:01<328:05:51,  7.19s/it]
 27%|██▋       | 60828/225000 [125:20:08<331:40:08,  7.27s/it]
 27%|██▋       | 60829/225000 [125:20:13<299:11:55,  6.56s/it]
 27%|██▋       | 60830/225000 [125:20:19<288:26:30,  6.33s/it]
 27%|██▋       | 60831/225000 [125:20:24<272:14:20,  5.97s/it]
 27%|██▋       | 60832/225000 [125:20:30<268:06:53,  5.88s/it]
 27%|██▋       | 60833/225000 [125:20:35<252:49:35,  5.54s/it]
 27%|██▋       | 60834/225000 [125:20:40<252:31:52,  5.54s/it]
 27%|██▋       | 60835/225000 [125:20:46<250:38:00,  5.50s/it]
 27%|██▋       | 60836/225000 [125:20:57<326:28:18,  7.16s/it]
 27%|██▋       | 60837/225000 [125:21:06<362:32:32,  7.95s/it]
 27%|██▋       | 60838/225000 [125:21:11<320:44:18,  7.03s/it]
 27%|██▋       | 60839/225000 [125:21:20<346:23:45,  7.60s/it]
 27%|██▋       | 60840/225000 [125:21:27<331:39:10,  7.27s/it]
 27%|██▋       | 60841/225000 [125:21:31<283:46:37,  6.22s/it]
 27%|██▋       | 60842/225000 [125:21:40<325:39:38,  7.14s/it]
 27%|██▋       | 60843/225000 [125:21:45<293:02:26,  6.43s/it]
 27%|██▋       | 60844/225000 [125:21:49<265:55:14,  5.83s/it]
 27%|██▋       | 60845/225000 [125:21:55<271:09:14,  5.95s/it]
 27%|██▋       | 60846/225000 [125:22:08<360:24:45,  7.90s/it]
 27%|██▋       | 60847/225000 [125:22:11<303:07:38,  6.65s/it]
 27%|██▋       | 60848/225000 [125:22:19<312:17:21,  6.85s/it]
 27%|██▋       | 60849/225000 [125:22:24<285:13:40,  6.26s/it]
 27%|██▋       | 60850/225000 [125:22:29<266:59:47,  5.86s/it]
                                                              
{'loss': '0.7462', 'grad_norm': '0.7746', 'learning_rate': '4.342e-05', 'epoch': '0.2704', 'num_input_tokens_seen': 208682408, 'train_runtime': '4.513e+05', 'train_tokens_per_second': '462.4'}

 27%|██▋       | 60850/225000 [125:22:29<266:59:47,  5.86s/it]
 27%|██▋       | 60851/225000 [125:22:34<257:37:00,  5.65s/it]
 27%|██▋       | 60852/225000 [125:22:46<349:59:55,  7.68s/it]
 27%|██▋       | 60853/225000 [125:22:50<299:31:20,  6.57s/it]
 27%|██▋       | 60854/225000 [125:22:55<274:12:17,  6.01s/it]
 27%|██▋       | 60855/225000 [125:23:01<277:48:24,  6.09s/it]
 27%|██▋       | 60856/225000 [125:23:12<336:53:47,  7.39s/it]
 27%|██▋       | 60857/225000 [125:23:17<306:50:57,  6.73s/it]
 27%|██▋       | 60858/225000 [125:23:22<282:03:42,  6.19s/it]
 27%|██▋       | 60859/225000 [125:23:27<264:44:55,  5.81s/it]
 27%|██▋       | 60860/225000 [125:23:32<253:06:09,  5.55s/it]
 27%|██▋       | 60861/225000 [125:23:36<244:51:40,  5.37s/it]
 27%|██▋       | 60862/225000 [125:23:49<337:08:15,  7.39s/it]
 27%|██▋       | 60863/225000 [125:23:56<343:59:51,  7.54s/it]
 27%|██▋       | 60864/225000 [125:24:01<303:11:35,  6.65s/it]
 27%|██▋       | 60865/225000 [125:24:10<331:51:31,  7.28s/it]
 27%|██▋       | 60866/225000 [125:24:14<283:50:27,  6.23s/it]
 27%|██▋       | 60867/225000 [125:24:23<332:43:47,  7.30s/it]
 27%|██▋       | 60868/225000 [125:24:29<309:25:25,  6.79s/it]
 27%|██▋       | 60869/225000 [125:24:36<314:40:51,  6.90s/it]
 27%|██▋       | 60870/225000 [125:24:41<293:14:47,  6.43s/it]
 27%|██▋       | 60871/225000 [125:24:45<258:25:23,  5.67s/it]
 27%|██▋       | 60872/225000 [125:24:56<331:39:29,  7.27s/it]
 27%|██▋       | 60873/225000 [125:25:00<279:02:47,  6.12s/it]
 27%|██▋       | 60874/225000 [125:25:06<273:33:44,  6.00s/it]
 27%|██▋       | 60875/225000 [125:25:11<269:02:56,  5.90s/it]
 27%|██▋       | 60876/225000 [125:25:17<265:26:35,  5.82s/it]
 27%|██▋       | 60877/225000 [125:25:22<259:40:31,  5.70s/it]
 27%|██▋       | 60878/225000 [125:25:30<282:54:16,  6.21s/it]
 27%|██▋       | 60879/225000 [125:25:36<283:59:22,  6.23s/it]
 27%|██▋       | 60880/225000 [125:25:42<275:54:27,  6.05s/it]
 27%|██▋       | 60881/225000 [125:25:47<270:38:07,  5.94s/it]
 27%|██▋       | 60882/225000 [125:25:56<315:08:18,  6.91s/it]
 27%|██▋       | 60883/225000 [125:26:04<327:23:37,  7.18s/it]
 27%|██▋       | 60884/225000 [125:26:09<291:55:12,  6.40s/it]
 27%|██▋       | 60885/225000 [125:26:13<265:02:16,  5.81s/it]
 27%|██▋       | 60886/225000 [125:26:20<277:02:33,  6.08s/it]
 27%|██▋       | 60887/225000 [125:26:33<371:46:47,  8.16s/it]
 27%|██▋       | 60888/225000 [125:26:37<312:21:48,  6.85s/it]
 27%|██▋       | 60889/225000 [125:26:42<294:27:36,  6.46s/it]
 27%|██▋       | 60890/225000 [125:26:49<302:57:24,  6.65s/it]
 27%|██▋       | 60891/225000 [125:26:57<319:26:51,  7.01s/it]
 27%|██▋       | 60892/225000 [125:27:05<331:27:22,  7.27s/it]
 27%|██▋       | 60893/225000 [125:27:12<321:14:01,  7.05s/it]
 27%|██▋       | 60894/225000 [125:27:19<327:36:40,  7.19s/it]
 27%|██▋       | 60895/225000 [125:27:25<303:15:31,  6.65s/it]
 27%|██▋       | 60896/225000 [125:27:32<315:42:10,  6.93s/it]
 27%|██▋       | 60897/225000 [125:27:39<309:04:11,  6.78s/it]
 27%|██▋       | 60898/225000 [125:27:44<290:39:21,  6.38s/it]
 27%|██▋       | 60899/225000 [125:27:48<258:00:47,  5.66s/it]
 27%|██▋       | 60900/225000 [125:27:52<240:38:19,  5.28s/it]
                                                              
{'loss': '0.7458', 'grad_norm': '0.7408', 'learning_rate': '4.341e-05', 'epoch': '0.2707', 'num_input_tokens_seen': 208829848, 'train_runtime': '4.517e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 60900/225000 [125:27:52<240:38:19,  5.28s/it]
 27%|██▋       | 60901/225000 [125:28:01<280:38:00,  6.16s/it]
 27%|██▋       | 60902/225000 [125:28:09<313:04:22,  6.87s/it]
 27%|██▋       | 60903/225000 [125:28:14<280:37:21,  6.16s/it]
 27%|██▋       | 60904/225000 [125:28:19<272:22:24,  5.98s/it]
 27%|██▋       | 60905/225000 [125:28:31<353:45:51,  7.76s/it]
 27%|██▋       | 60906/225000 [125:28:35<297:03:36,  6.52s/it]
 27%|██▋       | 60907/225000 [125:28:40<286:00:20,  6.27s/it]
 27%|██▋       | 60908/225000 [125:28:46<271:52:24,  5.96s/it]
 27%|██▋       | 60909/225000 [125:28:51<264:28:13,  5.80s/it]
 27%|██▋       | 60910/225000 [125:28:58<280:01:03,  6.14s/it]
 27%|██▋       | 60911/225000 [125:29:11<378:06:55,  8.30s/it]
 27%|██▋       | 60912/225000 [125:29:16<330:07:49,  7.24s/it]
 27%|██▋       | 60913/225000 [125:29:24<339:01:10,  7.44s/it]
 27%|██▋       | 60914/225000 [125:29:29<301:51:32,  6.62s/it]
 27%|██▋       | 60915/225000 [125:29:34<285:05:23,  6.25s/it]
 27%|██▋       | 60916/225000 [125:29:45<344:05:14,  7.55s/it]
 27%|██▋       | 60917/225000 [125:29:49<299:00:17,  6.56s/it]
 27%|██▋       | 60918/225000 [125:29:53<269:36:01,  5.92s/it]
 27%|██▋       | 60919/225000 [125:30:01<294:52:06,  6.47s/it]
 27%|██▋       | 60920/225000 [125:30:09<315:31:22,  6.92s/it]
 27%|██▋       | 60921/225000 [125:30:15<303:59:02,  6.67s/it]
 27%|██▋       | 60922/225000 [125:30:21<285:49:43,  6.27s/it]
 27%|██▋       | 60923/225000 [125:30:26<276:27:41,  6.07s/it]
 27%|██▋       | 60924/225000 [125:30:34<299:23:54,  6.57s/it]
 27%|██▋       | 60925/225000 [125:30:40<291:47:33,  6.40s/it]
 27%|██▋       | 60926/225000 [125:30:45<279:45:12,  6.14s/it]
 27%|██▋       | 60927/225000 [125:30:51<273:59:44,  6.01s/it]
 27%|██▋       | 60928/225000 [125:30:57<266:44:52,  5.85s/it]
 27%|██▋       | 60929/225000 [125:31:02<264:31:11,  5.80s/it]
 27%|██▋       | 60930/225000 [125:31:08<263:17:17,  5.78s/it]
 27%|██▋       | 60931/225000 [125:31:18<320:52:26,  7.04s/it]
 27%|██▋       | 60932/225000 [125:31:25<320:51:55,  7.04s/it]
 27%|██▋       | 60933/225000 [125:31:34<351:27:42,  7.71s/it]
 27%|██▋       | 60934/225000 [125:31:41<336:02:35,  7.37s/it]
 27%|██▋       | 60935/225000 [125:31:49<340:32:45,  7.47s/it]
 27%|██▋       | 60936/225000 [125:31:53<303:33:51,  6.66s/it]
 27%|██▋       | 60937/225000 [125:32:02<325:10:02,  7.14s/it]
 27%|██▋       | 60938/225000 [125:32:08<314:18:33,  6.90s/it]
 27%|██▋       | 60939/225000 [125:32:12<271:50:13,  5.96s/it]
 27%|██▋       | 60940/225000 [125:32:17<261:27:05,  5.74s/it]
 27%|██▋       | 60941/225000 [125:32:22<249:45:15,  5.48s/it]
 27%|██▋       | 60942/225000 [125:32:27<246:29:13,  5.41s/it]
 27%|██▋       | 60943/225000 [125:32:40<347:36:22,  7.63s/it]
 27%|██▋       | 60944/225000 [125:32:47<341:17:42,  7.49s/it]
 27%|██▋       | 60945/225000 [125:32:55<341:52:08,  7.50s/it]
 27%|██▋       | 60946/225000 [125:33:00<309:53:23,  6.80s/it]
 27%|██▋       | 60947/225000 [125:33:05<294:12:01,  6.46s/it]
 27%|██▋       | 60948/225000 [125:33:10<272:16:50,  5.98s/it]
 27%|██▋       | 60949/225000 [125:33:15<259:00:24,  5.68s/it]
 27%|██▋       | 60950/225000 [125:33:21<253:53:24,  5.57s/it]
                                                              
{'loss': '0.7442', 'grad_norm': '0.901', 'learning_rate': '4.339e-05', 'epoch': '0.2709', 'num_input_tokens_seen': 208977856, 'train_runtime': '4.52e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 60950/225000 [125:33:21<253:53:24,  5.57s/it]
 27%|██▋       | 60951/225000 [125:33:32<328:29:57,  7.21s/it]
 27%|██▋       | 60952/225000 [125:33:39<332:06:58,  7.29s/it]
 27%|██▋       | 60953/225000 [125:33:45<315:34:24,  6.93s/it]
 27%|██▋       | 60954/225000 [125:33:55<360:35:35,  7.91s/it]
 27%|██▋       | 60955/225000 [125:34:00<316:21:53,  6.94s/it]
 27%|██▋       | 60956/225000 [125:34:06<304:15:31,  6.68s/it]
 27%|██▋       | 60957/225000 [125:34:11<283:35:27,  6.22s/it]
 27%|██▋       | 60958/225000 [125:34:17<270:32:50,  5.94s/it]
 27%|██▋       | 60959/225000 [125:34:22<266:03:26,  5.84s/it]
 27%|██▋       | 60960/225000 [125:34:32<323:05:29,  7.09s/it]
 27%|██▋       | 60961/225000 [125:34:42<356:21:13,  7.82s/it]
 27%|██▋       | 60962/225000 [125:34:48<333:37:02,  7.32s/it]
 27%|██▋       | 60963/225000 [125:34:53<302:00:47,  6.63s/it]
 27%|██▋       | 60964/225000 [125:34:59<294:44:00,  6.47s/it]
 27%|██▋       | 60965/225000 [125:35:06<303:37:54,  6.66s/it]
 27%|██▋       | 60966/225000 [125:35:13<314:39:04,  6.91s/it]
 27%|██▋       | 60967/225000 [125:35:22<329:42:42,  7.24s/it]
 27%|██▋       | 60968/225000 [125:35:26<297:24:23,  6.53s/it]
 27%|██▋       | 60969/225000 [125:35:31<276:01:28,  6.06s/it]
 27%|██▋       | 60970/225000 [125:35:39<295:56:21,  6.50s/it]
 27%|██▋       | 60971/225000 [125:35:44<280:14:14,  6.15s/it]
 27%|██▋       | 60972/225000 [125:35:49<262:12:40,  5.75s/it]
 27%|██▋       | 60973/225000 [125:35:55<262:20:23,  5.76s/it]
 27%|██▋       | 60974/225000 [125:36:01<262:51:11,  5.77s/it]
 27%|██▋       | 60975/225000 [125:36:06<254:55:51,  5.60s/it]
 27%|██▋       | 60976/225000 [125:36:11<253:38:01,  5.57s/it]
 27%|██▋       | 60977/225000 [125:36:25<367:36:07,  8.07s/it]
 27%|██▋       | 60978/225000 [125:36:31<334:16:12,  7.34s/it]
 27%|██▋       | 60979/225000 [125:36:38<330:34:46,  7.26s/it]
 27%|██▋       | 60980/225000 [125:36:47<358:14:24,  7.86s/it]
 27%|██▋       | 60981/225000 [125:36:53<331:30:59,  7.28s/it]
 27%|██▋       | 60982/225000 [125:36:59<307:41:48,  6.75s/it]
 27%|██▋       | 60983/225000 [125:37:06<310:31:43,  6.82s/it]
 27%|██▋       | 60984/225000 [125:37:10<278:53:56,  6.12s/it]
 27%|██▋       | 60985/225000 [125:37:17<290:52:27,  6.38s/it]
 27%|██▋       | 60986/225000 [125:37:26<319:47:38,  7.02s/it]
 27%|██▋       | 60987/225000 [125:37:29<274:51:13,  6.03s/it]
 27%|██▋       | 60988/225000 [125:37:38<317:32:59,  6.97s/it]
 27%|██▋       | 60989/225000 [125:37:42<276:02:13,  6.06s/it]
 27%|██▋       | 60990/225000 [125:37:53<344:43:04,  7.57s/it]
 27%|██▋       | 60991/225000 [125:37:58<305:50:37,  6.71s/it]
 27%|██▋       | 60992/225000 [125:38:03<277:11:00,  6.08s/it]
 27%|██▋       | 60993/225000 [125:38:08<264:45:57,  5.81s/it]
 27%|██▋       | 60994/225000 [125:38:19<338:43:44,  7.44s/it]
 27%|██▋       | 60995/225000 [125:38:27<347:06:06,  7.62s/it]
 27%|██▋       | 60996/225000 [125:38:31<299:50:39,  6.58s/it]
 27%|██▋       | 60997/225000 [125:38:37<282:07:04,  6.19s/it]
 27%|██▋       | 60998/225000 [125:38:42<267:47:55,  5.88s/it]
 27%|██▋       | 60999/225000 [125:38:47<254:54:53,  5.60s/it]
 27%|██▋       | 61000/225000 [125:39:00<352:27:27,  7.74s/it]
                                                              
{'loss': '0.7557', 'grad_norm': '0.6989', 'learning_rate': '4.338e-05', 'epoch': '0.2711', 'num_input_tokens_seen': 209125624, 'train_runtime': '4.523e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61000/225000 [125:39:00<352:27:27,  7.74s/it]
 27%|██▋       | 61001/225000 [125:39:06<331:32:34,  7.28s/it]
 27%|██▋       | 61002/225000 [125:39:11<307:11:44,  6.74s/it]
 27%|██▋       | 61003/225000 [125:39:16<277:18:50,  6.09s/it]
 27%|██▋       | 61004/225000 [125:39:26<338:25:34,  7.43s/it]
 27%|██▋       | 61005/225000 [125:39:32<318:06:51,  6.98s/it]
 27%|██▋       | 61006/225000 [125:39:39<320:35:10,  7.04s/it]
 27%|██▋       | 61007/225000 [125:39:44<287:17:38,  6.31s/it]
 27%|██▋       | 61008/225000 [125:39:49<268:47:50,  5.90s/it]
 27%|██▋       | 61009/225000 [125:39:54<256:21:12,  5.63s/it]
 27%|██▋       | 61010/225000 [125:40:00<260:57:29,  5.73s/it]
 27%|██▋       | 61011/225000 [125:40:13<357:47:20,  7.85s/it]
 27%|██▋       | 61012/225000 [125:40:18<328:30:45,  7.21s/it]
 27%|██▋       | 61013/225000 [125:40:23<293:33:08,  6.44s/it]
 27%|██▋       | 61014/225000 [125:40:28<267:49:25,  5.88s/it]
 27%|██▋       | 61015/225000 [125:40:40<358:29:21,  7.87s/it]
 27%|██▋       | 61016/225000 [125:40:44<307:45:37,  6.76s/it]
 27%|██▋       | 61017/225000 [125:40:53<335:53:56,  7.37s/it]
 27%|██▋       | 61018/225000 [125:40:58<302:20:40,  6.64s/it]
 27%|██▋       | 61019/225000 [125:41:04<291:04:47,  6.39s/it]
 27%|██▋       | 61020/225000 [125:41:11<306:38:18,  6.73s/it]
 27%|██▋       | 61021/225000 [125:41:17<285:50:06,  6.28s/it]
 27%|██▋       | 61022/225000 [125:41:22<271:16:29,  5.96s/it]
 27%|██▋       | 61023/225000 [125:41:27<262:09:01,  5.76s/it]
 27%|██▋       | 61024/225000 [125:41:32<256:24:20,  5.63s/it]
 27%|██▋       | 61025/225000 [125:41:39<271:50:07,  5.97s/it]
 27%|██▋       | 61026/225000 [125:41:53<374:45:21,  8.23s/it]
 27%|██▋       | 61027/225000 [125:41:59<345:04:19,  7.58s/it]
 27%|██▋       | 61028/225000 [125:42:04<307:35:16,  6.75s/it]
 27%|██▋       | 61029/225000 [125:42:08<279:51:20,  6.14s/it]
 27%|██▋       | 61030/225000 [125:42:13<262:01:59,  5.75s/it]
 27%|██▋       | 61031/225000 [125:42:21<289:08:45,  6.35s/it]
 27%|██▋       | 61032/225000 [125:42:32<356:25:54,  7.83s/it]
 27%|██▋       | 61033/225000 [125:42:36<303:25:01,  6.66s/it]
 27%|██▋       | 61034/225000 [125:42:44<316:09:19,  6.94s/it]
 27%|██▋       | 61035/225000 [125:42:50<304:51:48,  6.69s/it]
 27%|██▋       | 61036/225000 [125:42:58<331:17:34,  7.27s/it]
 27%|██▋       | 61037/225000 [125:43:04<309:45:00,  6.80s/it]
 27%|██▋       | 61038/225000 [125:43:08<271:47:52,  5.97s/it]
 27%|██▋       | 61039/225000 [125:43:13<256:34:25,  5.63s/it]
 27%|██▋       | 61040/225000 [125:43:18<243:53:21,  5.35s/it]
 27%|██▋       | 61041/225000 [125:43:29<322:47:02,  7.09s/it]
 27%|██▋       | 61042/225000 [125:43:37<332:12:52,  7.29s/it]
 27%|██▋       | 61043/225000 [125:43:41<293:25:01,  6.44s/it]
 27%|██▋       | 61044/225000 [125:43:46<265:43:49,  5.83s/it]
 27%|██▋       | 61045/225000 [125:43:51<264:29:16,  5.81s/it]
 27%|██▋       | 61046/225000 [125:44:02<332:37:29,  7.30s/it]
 27%|██▋       | 61047/225000 [125:44:08<307:03:49,  6.74s/it]
 27%|██▋       | 61048/225000 [125:44:15<317:48:23,  6.98s/it]
 27%|██▋       | 61049/225000 [125:44:22<317:09:43,  6.96s/it]
 27%|██▋       | 61050/225000 [125:44:26<283:22:11,  6.22s/it]
                                                              
{'loss': '0.738', 'grad_norm': '0.8084', 'learning_rate': '4.337e-05', 'epoch': '0.2713', 'num_input_tokens_seen': 209274016, 'train_runtime': '4.527e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61050/225000 [125:44:26<283:22:11,  6.22s/it]
 27%|██▋       | 61051/225000 [125:44:31<259:41:16,  5.70s/it]
 27%|██▋       | 61052/225000 [125:44:42<326:12:13,  7.16s/it]
 27%|██▋       | 61053/225000 [125:44:45<279:58:03,  6.15s/it]
 27%|██▋       | 61054/225000 [125:44:54<311:55:30,  6.85s/it]
 27%|██▋       | 61055/225000 [125:44:59<283:24:39,  6.22s/it]
 27%|██▋       | 61056/225000 [125:45:07<314:06:07,  6.90s/it]
 27%|██▋       | 61057/225000 [125:45:11<275:18:34,  6.05s/it]
 27%|██▋       | 61058/225000 [125:45:17<267:33:56,  5.88s/it]
 27%|██▋       | 61059/225000 [125:45:22<265:33:16,  5.83s/it]
 27%|██▋       | 61060/225000 [125:45:28<260:36:18,  5.72s/it]
 27%|██▋       | 61061/225000 [125:45:33<255:32:40,  5.61s/it]
 27%|██▋       | 61062/225000 [125:45:38<242:10:21,  5.32s/it]
 27%|██▋       | 61063/225000 [125:45:44<253:25:27,  5.57s/it]
 27%|██▋       | 61064/225000 [125:45:56<348:56:17,  7.66s/it]
 27%|██▋       | 61065/225000 [125:46:05<354:16:10,  7.78s/it]
 27%|██▋       | 61066/225000 [125:46:09<310:35:53,  6.82s/it]
 27%|██▋       | 61067/225000 [125:46:14<284:18:40,  6.24s/it]
 27%|██▋       | 61068/225000 [125:46:24<329:54:33,  7.24s/it]
 27%|██▋       | 61069/225000 [125:46:31<326:11:50,  7.16s/it]
 27%|██▋       | 61070/225000 [125:46:35<288:18:14,  6.33s/it]
 27%|██▋       | 61071/225000 [125:46:40<273:36:57,  6.01s/it]
 27%|██▋       | 61072/225000 [125:46:45<258:57:58,  5.69s/it]
 27%|██▋       | 61073/225000 [125:46:58<363:04:49,  7.97s/it]
 27%|██▋       | 61074/225000 [125:47:04<335:28:14,  7.37s/it]
 27%|██▋       | 61075/225000 [125:47:10<305:24:27,  6.71s/it]
 27%|██▋       | 61076/225000 [125:47:15<288:58:33,  6.35s/it]
 27%|██▋       | 61077/225000 [125:47:21<284:00:35,  6.24s/it]
 27%|██▋       | 61078/225000 [125:47:27<274:11:25,  6.02s/it]
 27%|██▋       | 61079/225000 [125:47:38<348:06:15,  7.64s/it]
 27%|██▋       | 61080/225000 [125:47:42<304:59:13,  6.70s/it]
 27%|██▋       | 61081/225000 [125:47:47<274:23:31,  6.03s/it]
 27%|██▋       | 61082/225000 [125:47:52<257:20:08,  5.65s/it]
 27%|██▋       | 61083/225000 [125:48:05<363:48:53,  7.99s/it]
 27%|██▋       | 61084/225000 [125:48:10<320:18:22,  7.03s/it]
 27%|██▋       | 61085/225000 [125:48:15<290:41:39,  6.38s/it]
 27%|██▋       | 61086/225000 [125:48:26<354:28:42,  7.79s/it]
 27%|██▋       | 61087/225000 [125:48:30<301:11:12,  6.61s/it]
 27%|██▋       | 61088/225000 [125:48:34<268:11:48,  5.89s/it]
 27%|██▋       | 61089/225000 [125:48:42<298:56:42,  6.57s/it]
 27%|██▋       | 61090/225000 [125:48:50<314:56:09,  6.92s/it]
 27%|██▋       | 61091/225000 [125:48:57<315:57:15,  6.94s/it]
 27%|██▋       | 61092/225000 [125:49:03<299:56:16,  6.59s/it]
 27%|██▋       | 61093/225000 [125:49:10<306:02:58,  6.72s/it]
 27%|██▋       | 61094/225000 [125:49:18<335:07:30,  7.36s/it]
 27%|██▋       | 61095/225000 [125:49:25<318:03:10,  6.99s/it]
 27%|██▋       | 61096/225000 [125:49:29<281:45:09,  6.19s/it]
 27%|██▋       | 61097/225000 [125:49:34<268:35:57,  5.90s/it]
 27%|██▋       | 61098/225000 [125:49:40<266:34:21,  5.86s/it]
 27%|██▋       | 61099/225000 [125:49:46<266:53:03,  5.86s/it]
 27%|██▋       | 61100/225000 [125:49:52<266:25:25,  5.85s/it]
                                                              
{'loss': '0.7508', 'grad_norm': '0.7114', 'learning_rate': '4.336e-05', 'epoch': '0.2716', 'num_input_tokens_seen': 209423616, 'train_runtime': '4.53e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61100/225000 [125:49:52<266:25:25,  5.85s/it]
 27%|██▋       | 61101/225000 [125:49:57<263:28:48,  5.79s/it]
 27%|██▋       | 61102/225000 [125:50:03<268:39:27,  5.90s/it]
 27%|██▋       | 61103/225000 [125:50:15<351:39:03,  7.72s/it]
 27%|██▋       | 61104/225000 [125:50:24<365:45:34,  8.03s/it]
 27%|██▋       | 61105/225000 [125:50:30<336:58:11,  7.40s/it]
 27%|██▋       | 61106/225000 [125:50:35<309:05:10,  6.79s/it]
 27%|██▋       | 61107/225000 [125:50:41<295:55:42,  6.50s/it]
 27%|██▋       | 61108/225000 [125:50:47<284:02:53,  6.24s/it]
 27%|██▋       | 61109/225000 [125:50:56<329:43:30,  7.24s/it]
 27%|██▋       | 61110/225000 [125:51:01<288:49:52,  6.34s/it]
 27%|██▋       | 61111/225000 [125:51:05<260:05:18,  5.71s/it]
 27%|██▋       | 61112/225000 [125:51:16<334:01:10,  7.34s/it]
 27%|██▋       | 61113/225000 [125:51:20<291:11:48,  6.40s/it]
 27%|██▋       | 61114/225000 [125:51:25<274:16:45,  6.02s/it]
 27%|██▋       | 61115/225000 [125:51:34<307:17:22,  6.75s/it]
 27%|██▋       | 61116/225000 [125:51:40<301:33:34,  6.62s/it]
 27%|██▋       | 61117/225000 [125:51:46<294:09:23,  6.46s/it]
 27%|██▋       | 61118/225000 [125:51:54<306:07:32,  6.72s/it]
 27%|██▋       | 61119/225000 [125:51:58<280:33:17,  6.16s/it]
 27%|██▋       | 61120/225000 [125:52:04<268:32:14,  5.90s/it]
 27%|██▋       | 61121/225000 [125:52:09<252:30:26,  5.55s/it]
 27%|██▋       | 61122/225000 [125:52:17<292:40:05,  6.43s/it]
 27%|██▋       | 61123/225000 [125:52:28<356:58:07,  7.84s/it]
 27%|██▋       | 61124/225000 [125:52:32<302:54:55,  6.65s/it]
 27%|██▋       | 61125/225000 [125:52:36<272:17:30,  5.98s/it]
 27%|██▋       | 61126/225000 [125:52:47<336:07:35,  7.38s/it]
 27%|██▋       | 61127/225000 [125:52:53<316:20:59,  6.95s/it]
 27%|██▋       | 61128/225000 [125:52:59<302:27:18,  6.64s/it]
 27%|██▋       | 61129/225000 [125:53:03<271:09:37,  5.96s/it]
 27%|██▋       | 61130/225000 [125:53:10<279:01:13,  6.13s/it]
 27%|██▋       | 61131/225000 [125:53:19<322:31:36,  7.09s/it]
 27%|██▋       | 61132/225000 [125:53:25<303:57:24,  6.68s/it]
 27%|██▋       | 61133/225000 [125:53:33<325:07:35,  7.14s/it]
 27%|██▋       | 61134/225000 [125:53:37<280:40:06,  6.17s/it]
 27%|██▋       | 61135/225000 [125:53:46<314:47:52,  6.92s/it]
 27%|██▋       | 61136/225000 [125:53:50<283:57:10,  6.24s/it]
 27%|██▋       | 61137/225000 [125:53:55<263:04:52,  5.78s/it]
 27%|██▋       | 61138/225000 [125:54:00<245:31:36,  5.39s/it]
 27%|██▋       | 61139/225000 [125:54:06<262:45:28,  5.77s/it]
 27%|██▋       | 61140/225000 [125:54:18<343:59:39,  7.56s/it]
 27%|██▋       | 61141/225000 [125:54:22<297:46:41,  6.54s/it]
 27%|██▋       | 61142/225000 [125:54:26<263:18:36,  5.78s/it]
 27%|██▋       | 61143/225000 [125:54:30<244:29:30,  5.37s/it]
 27%|██▋       | 61144/225000 [125:54:43<335:20:06,  7.37s/it]
 27%|██▋       | 61145/225000 [125:54:47<294:13:04,  6.46s/it]
 27%|██▋       | 61146/225000 [125:54:53<284:36:15,  6.25s/it]
 27%|██▋       | 61147/225000 [125:55:01<319:53:55,  7.03s/it]
 27%|██▋       | 61148/225000 [125:55:05<278:15:21,  6.11s/it]
 27%|██▋       | 61149/225000 [125:55:10<254:35:01,  5.59s/it]
 27%|██▋       | 61150/225000 [125:55:19<308:57:47,  6.79s/it]
                                                              
{'loss': '0.7368', 'grad_norm': '0.7398', 'learning_rate': '4.334e-05', 'epoch': '0.2718', 'num_input_tokens_seen': 209570312, 'train_runtime': '4.533e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61150/225000 [125:55:19<308:57:47,  6.79s/it]
 27%|██▋       | 61151/225000 [125:55:27<326:28:12,  7.17s/it]
 27%|██▋       | 61152/225000 [125:55:31<282:47:25,  6.21s/it]
 27%|██▋       | 61153/225000 [125:55:40<312:11:52,  6.86s/it]
 27%|██▋       | 61154/225000 [125:55:46<304:03:34,  6.68s/it]
 27%|██▋       | 61155/225000 [125:55:50<270:03:22,  5.93s/it]
 27%|██▋       | 61156/225000 [125:55:55<260:17:37,  5.72s/it]
 27%|██▋       | 61157/225000 [125:56:01<253:14:01,  5.56s/it]
 27%|██▋       | 61158/225000 [125:56:06<247:41:02,  5.44s/it]
 27%|██▋       | 61159/225000 [125:56:19<347:24:24,  7.63s/it]
 27%|██▋       | 61160/225000 [125:56:27<358:30:21,  7.88s/it]
 27%|██▋       | 61161/225000 [125:56:31<309:11:23,  6.79s/it]
 27%|██▋       | 61162/225000 [125:56:36<277:21:30,  6.09s/it]
 27%|██▋       | 61163/225000 [125:56:47<344:09:34,  7.56s/it]
 27%|██▋       | 61164/225000 [125:56:52<308:49:51,  6.79s/it]
 27%|██▋       | 61165/225000 [125:56:57<284:54:51,  6.26s/it]
 27%|██▋       | 61166/225000 [125:57:02<264:37:50,  5.81s/it]
 27%|██▋       | 61167/225000 [125:57:07<256:07:20,  5.63s/it]
 27%|██▋       | 61168/225000 [125:57:15<286:43:31,  6.30s/it]
 27%|██▋       | 61169/225000 [125:57:26<350:38:43,  7.71s/it]
 27%|██▋       | 61170/225000 [125:57:29<295:42:07,  6.50s/it]
 27%|██▋       | 61171/225000 [125:57:37<312:44:12,  6.87s/it]
 27%|██▋       | 61172/225000 [125:57:41<270:29:02,  5.94s/it]
 27%|██▋       | 61173/225000 [125:57:50<314:55:32,  6.92s/it]
 27%|██▋       | 61174/225000 [125:57:54<275:53:51,  6.06s/it]
 27%|██▋       | 61175/225000 [125:58:05<346:51:21,  7.62s/it]
 27%|██▋       | 61176/225000 [125:58:10<310:33:09,  6.82s/it]
 27%|██▋       | 61177/225000 [125:58:14<273:03:28,  6.00s/it]
 27%|██▋       | 61178/225000 [125:58:25<334:39:22,  7.35s/it]
 27%|██▋       | 61179/225000 [125:58:29<291:43:01,  6.41s/it]
 27%|██▋       | 61180/225000 [125:58:35<285:01:03,  6.26s/it]
 27%|██▋       | 61181/225000 [125:58:41<283:05:59,  6.22s/it]
 27%|██▋       | 61182/225000 [125:58:48<289:15:22,  6.36s/it]
 27%|██▋       | 61183/225000 [125:58:53<267:38:46,  5.88s/it]
 27%|██▋       | 61184/225000 [125:58:59<277:39:13,  6.10s/it]
 27%|██▋       | 61185/225000 [125:59:08<319:35:44,  7.02s/it]
 27%|██▋       | 61186/225000 [125:59:13<291:30:20,  6.41s/it]
 27%|██▋       | 61187/225000 [125:59:18<268:25:06,  5.90s/it]
 27%|██▋       | 61188/225000 [125:59:24<272:02:14,  5.98s/it]
 27%|██▋       | 61189/225000 [125:59:36<351:18:11,  7.72s/it]
 27%|██▋       | 61190/225000 [125:59:40<294:07:01,  6.46s/it]
 27%|██▋       | 61191/225000 [125:59:49<334:44:03,  7.36s/it]
 27%|██▋       | 61192/225000 [125:59:53<286:09:56,  6.29s/it]
 27%|██▋       | 61193/225000 [126:00:02<329:25:42,  7.24s/it]
 27%|██▋       | 61194/225000 [126:00:07<290:48:58,  6.39s/it]
 27%|██▋       | 61195/225000 [126:00:13<288:43:15,  6.35s/it]
 27%|██▋       | 61196/225000 [126:00:20<303:18:02,  6.67s/it]
 27%|██▋       | 61197/225000 [126:00:25<271:45:33,  5.97s/it]
 27%|██▋       | 61198/225000 [126:00:30<265:09:04,  5.83s/it]
 27%|██▋       | 61199/225000 [126:00:39<302:53:04,  6.66s/it]
 27%|██▋       | 61200/225000 [126:00:43<271:03:06,  5.96s/it]
                                                              
{'loss': '0.7442', 'grad_norm': '0.7907', 'learning_rate': '4.333e-05', 'epoch': '0.272', 'num_input_tokens_seen': 209716456, 'train_runtime': '4.536e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61200/225000 [126:00:43<271:03:06,  5.96s/it]
 27%|██▋       | 61201/225000 [126:00:54<339:53:18,  7.47s/it]
 27%|██▋       | 61202/225000 [126:00:58<285:45:07,  6.28s/it]
 27%|██▋       | 61203/225000 [126:01:06<311:54:50,  6.86s/it]
 27%|██▋       | 61204/225000 [126:01:10<281:10:03,  6.18s/it]
 27%|██▋       | 61205/225000 [126:01:15<256:42:11,  5.64s/it]
 27%|██▋       | 61206/225000 [126:01:23<290:24:32,  6.38s/it]
 27%|██▋       | 61207/225000 [126:01:30<303:24:47,  6.67s/it]
 27%|██▋       | 61208/225000 [126:01:36<292:03:24,  6.42s/it]
 27%|██▋       | 61209/225000 [126:01:40<259:48:43,  5.71s/it]
 27%|██▋       | 61210/225000 [126:01:50<314:33:31,  6.91s/it]
 27%|██▋       | 61211/225000 [126:01:54<271:24:16,  5.97s/it]
 27%|██▋       | 61212/225000 [126:01:59<264:46:35,  5.82s/it]
 27%|██▋       | 61213/225000 [126:02:08<304:25:10,  6.69s/it]
 27%|██▋       | 61214/225000 [126:02:12<277:45:13,  6.10s/it]
 27%|██▋       | 61215/225000 [126:02:17<259:24:13,  5.70s/it]
 27%|██▋       | 61216/225000 [126:02:23<256:17:28,  5.63s/it]
 27%|██▋       | 61217/225000 [126:02:37<369:24:50,  8.12s/it]
 27%|██▋       | 61218/225000 [126:02:40<307:39:58,  6.76s/it]
 27%|██▋       | 61219/225000 [126:02:47<301:41:13,  6.63s/it]
 27%|██▋       | 61220/225000 [126:02:55<321:40:47,  7.07s/it]
 27%|██▋       | 61221/225000 [126:03:01<305:28:47,  6.71s/it]
 27%|██▋       | 61222/225000 [126:03:09<327:51:33,  7.21s/it]
 27%|██▋       | 61223/225000 [126:03:17<343:55:49,  7.56s/it]
 27%|██▋       | 61224/225000 [126:03:21<288:20:27,  6.34s/it]
 27%|██▋       | 61225/225000 [126:03:29<313:38:43,  6.89s/it]
 27%|██▋       | 61226/225000 [126:03:35<295:15:20,  6.49s/it]
 27%|██▋       | 61227/225000 [126:03:40<278:44:06,  6.13s/it]
 27%|██▋       | 61228/225000 [126:03:45<265:42:12,  5.84s/it]
 27%|██▋       | 61229/225000 [126:03:50<257:51:15,  5.67s/it]
 27%|██▋       | 61230/225000 [126:03:56<253:11:22,  5.57s/it]
 27%|██▋       | 61231/225000 [126:04:01<247:27:55,  5.44s/it]
 27%|██▋       | 61232/225000 [126:04:06<242:54:48,  5.34s/it]
 27%|██▋       | 61233/225000 [126:04:15<289:06:45,  6.36s/it]
 27%|██▋       | 61234/225000 [126:04:27<373:14:18,  8.20s/it]
 27%|██▋       | 61235/225000 [126:04:31<321:04:42,  7.06s/it]
 27%|██▋       | 61236/225000 [126:04:36<291:18:26,  6.40s/it]
 27%|██▋       | 61237/225000 [126:04:42<276:46:53,  6.08s/it]
 27%|██▋       | 61238/225000 [126:04:48<274:09:03,  6.03s/it]
 27%|██▋       | 61239/225000 [126:05:00<365:12:10,  8.03s/it]
 27%|██▋       | 61240/225000 [126:05:04<313:40:53,  6.90s/it]
 27%|██▋       | 61241/225000 [126:05:09<278:48:04,  6.13s/it]
 27%|██▋       | 61242/225000 [126:05:20<341:26:52,  7.51s/it]
 27%|██▋       | 61243/225000 [126:05:24<301:54:08,  6.64s/it]
 27%|██▋       | 61244/225000 [126:05:28<268:54:44,  5.91s/it]
 27%|██▋       | 61245/225000 [126:05:41<359:49:28,  7.91s/it]
 27%|██▋       | 61246/225000 [126:05:46<317:19:58,  6.98s/it]
 27%|██▋       | 61247/225000 [126:05:53<315:06:58,  6.93s/it]
 27%|██▋       | 61248/225000 [126:05:59<315:05:34,  6.93s/it]
 27%|██▋       | 61249/225000 [126:06:05<295:24:26,  6.49s/it]
 27%|██▋       | 61250/225000 [126:06:11<292:03:53,  6.42s/it]
                                                              
{'loss': '0.7503', 'grad_norm': '0.8603', 'learning_rate': '4.332e-05', 'epoch': '0.2722', 'num_input_tokens_seen': 209864536, 'train_runtime': '4.54e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61250/225000 [126:06:11<292:03:53,  6.42s/it]
 27%|██▋       | 61251/225000 [126:06:19<312:45:05,  6.88s/it]
 27%|██▋       | 61252/225000 [126:06:25<301:56:58,  6.64s/it]
 27%|██▋       | 61253/225000 [126:06:33<312:45:59,  6.88s/it]
 27%|██▋       | 61254/225000 [126:06:36<269:25:34,  5.92s/it]
 27%|██▋       | 61255/225000 [126:06:45<300:06:40,  6.60s/it]
 27%|██▋       | 61256/225000 [126:06:52<317:40:07,  6.98s/it]
 27%|██▋       | 61257/225000 [126:06:58<299:24:22,  6.58s/it]
 27%|██▋       | 61258/225000 [126:07:02<260:29:29,  5.73s/it]
 27%|██▋       | 61259/225000 [126:07:11<313:01:04,  6.88s/it]
 27%|██▋       | 61260/225000 [126:07:16<276:01:25,  6.07s/it]
 27%|██▋       | 61261/225000 [126:07:23<297:46:19,  6.55s/it]
 27%|██▋       | 61262/225000 [126:07:31<311:33:07,  6.85s/it]
 27%|██▋       | 61263/225000 [126:07:39<328:52:32,  7.23s/it]
 27%|██▋       | 61264/225000 [126:07:43<286:58:47,  6.31s/it]
 27%|██▋       | 61265/225000 [126:07:47<259:16:56,  5.70s/it]
 27%|██▋       | 61266/225000 [126:07:56<299:26:33,  6.58s/it]
 27%|██▋       | 61267/225000 [126:08:06<345:45:06,  7.60s/it]
 27%|██▋       | 61268/225000 [126:08:10<302:12:37,  6.64s/it]
 27%|██▋       | 61269/225000 [126:08:16<288:47:05,  6.35s/it]
 27%|██▋       | 61270/225000 [126:08:25<322:03:06,  7.08s/it]
 27%|██▋       | 61271/225000 [126:08:30<293:06:42,  6.44s/it]
 27%|██▋       | 61272/225000 [126:08:34<265:48:10,  5.84s/it]
 27%|██▋       | 61273/225000 [126:08:39<246:25:10,  5.42s/it]
 27%|██▋       | 61274/225000 [126:08:49<316:59:00,  6.97s/it]
 27%|██▋       | 61275/225000 [126:08:55<296:44:29,  6.52s/it]
 27%|██▋       | 61276/225000 [126:09:02<312:45:22,  6.88s/it]
 27%|██▋       | 61277/225000 [126:09:06<268:09:52,  5.90s/it]
 27%|██▋       | 61278/225000 [126:09:12<266:52:41,  5.87s/it]
 27%|██▋       | 61279/225000 [126:09:17<254:34:23,  5.60s/it]
 27%|██▋       | 61280/225000 [126:09:22<248:22:00,  5.46s/it]
 27%|██▋       | 61281/225000 [126:09:35<350:13:51,  7.70s/it]
 27%|██▋       | 61282/225000 [126:09:40<317:58:29,  6.99s/it]
 27%|██▋       | 61283/225000 [126:09:45<284:23:43,  6.25s/it]
 27%|██▋       | 61284/225000 [126:09:49<258:25:25,  5.68s/it]
 27%|██▋       | 61285/225000 [126:09:59<312:04:27,  6.86s/it]
 27%|██▋       | 61286/225000 [126:10:07<336:03:13,  7.39s/it]
 27%|██▋       | 61287/225000 [126:10:12<299:39:15,  6.59s/it]
 27%|██▋       | 61288/225000 [126:10:21<335:18:31,  7.37s/it]
 27%|██▋       | 61289/225000 [126:10:26<293:22:20,  6.45s/it]
 27%|██▋       | 61290/225000 [126:10:33<309:22:32,  6.80s/it]
 27%|██▋       | 61291/225000 [126:10:37<266:06:28,  5.85s/it]
 27%|██▋       | 61292/225000 [126:10:46<317:36:47,  6.98s/it]
 27%|██▋       | 61293/225000 [126:10:50<270:27:15,  5.95s/it]
 27%|██▋       | 61294/225000 [126:10:54<245:04:59,  5.39s/it]
 27%|██▋       | 61295/225000 [126:11:04<304:09:10,  6.69s/it]
 27%|██▋       | 61296/225000 [126:11:09<283:24:51,  6.23s/it]
 27%|██▋       | 61297/225000 [126:11:18<317:21:10,  6.98s/it]
 27%|██▋       | 61298/225000 [126:11:23<288:46:58,  6.35s/it]
 27%|██▋       | 61299/225000 [126:11:28<278:07:09,  6.12s/it]
 27%|██▋       | 61300/225000 [126:11:34<268:14:09,  5.90s/it]
                                                              
{'loss': '0.7505', 'grad_norm': '0.7254', 'learning_rate': '4.331e-05', 'epoch': '0.2724', 'num_input_tokens_seen': 210011680, 'train_runtime': '4.543e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61300/225000 [126:11:34<268:14:09,  5.90s/it]
 27%|██▋       | 61301/225000 [126:11:39<259:52:09,  5.71s/it]
 27%|██▋       | 61302/225000 [126:11:44<251:15:21,  5.53s/it]
 27%|██▋       | 61303/225000 [126:11:49<250:07:40,  5.50s/it]
 27%|██▋       | 61304/225000 [126:12:02<353:07:25,  7.77s/it]
 27%|██▋       | 61305/225000 [126:12:10<355:47:10,  7.82s/it]
 27%|██▋       | 61306/225000 [126:12:15<313:51:21,  6.90s/it]
 27%|██▋       | 61307/225000 [126:12:20<287:52:24,  6.33s/it]
 27%|██▋       | 61308/225000 [126:12:25<266:05:42,  5.85s/it]
 27%|██▋       | 61309/225000 [126:12:37<358:22:38,  7.88s/it]
 27%|██▋       | 61310/225000 [126:12:41<302:00:56,  6.64s/it]
 27%|██▋       | 61311/225000 [126:12:47<287:03:55,  6.31s/it]
 27%|██▋       | 61312/225000 [126:12:52<279:38:47,  6.15s/it]
 27%|██▋       | 61313/225000 [126:12:58<274:35:19,  6.04s/it]
 27%|██▋       | 61314/225000 [126:13:05<286:57:17,  6.31s/it]
 27%|██▋       | 61315/225000 [126:13:16<352:51:54,  7.76s/it]
 27%|██▋       | 61316/225000 [126:13:24<345:50:03,  7.61s/it]
 27%|██▋       | 61317/225000 [126:13:28<295:08:31,  6.49s/it]
 27%|██▋       | 61318/225000 [126:13:33<276:42:42,  6.09s/it]
 27%|██▋       | 61319/225000 [126:13:41<311:49:35,  6.86s/it]
 27%|██▋       | 61320/225000 [126:13:47<292:13:30,  6.43s/it]
 27%|██▋       | 61321/225000 [126:13:56<336:11:08,  7.39s/it]
 27%|██▋       | 61322/225000 [126:14:02<311:53:24,  6.86s/it]
 27%|██▋       | 61323/225000 [126:14:07<292:03:57,  6.42s/it]
 27%|██▋       | 61324/225000 [126:14:13<280:35:53,  6.17s/it]
 27%|██▋       | 61325/225000 [126:14:18<269:50:53,  5.94s/it]
 27%|██▋       | 61326/225000 [126:14:24<269:08:48,  5.92s/it]
 27%|██▋       | 61327/225000 [126:14:30<261:18:35,  5.75s/it]
 27%|██▋       | 61328/225000 [126:14:35<257:59:53,  5.67s/it]
 27%|██▋       | 61329/225000 [126:14:41<258:00:20,  5.67s/it]
 27%|██▋       | 61330/225000 [126:14:53<350:01:45,  7.70s/it]
 27%|██▋       | 61331/225000 [126:15:03<373:27:55,  8.21s/it]
 27%|██▋       | 61332/225000 [126:15:09<342:05:05,  7.52s/it]
 27%|██▋       | 61333/225000 [126:15:13<297:18:42,  6.54s/it]
 27%|██▋       | 61334/225000 [126:15:17<268:43:59,  5.91s/it]
 27%|██▋       | 61335/225000 [126:15:29<349:21:22,  7.68s/it]
 27%|██▋       | 61336/225000 [126:15:37<349:49:41,  7.69s/it]
 27%|██▋       | 61337/225000 [126:15:42<311:26:05,  6.85s/it]
 27%|██▋       | 61338/225000 [126:15:47<287:15:47,  6.32s/it]
 27%|██▋       | 61339/225000 [126:15:51<265:20:27,  5.84s/it]
 27%|██▋       | 61340/225000 [126:15:56<249:36:39,  5.49s/it]
 27%|██▋       | 61341/225000 [126:16:01<246:52:35,  5.43s/it]
 27%|██▋       | 61342/225000 [126:16:07<243:01:55,  5.35s/it]
 27%|██▋       | 61343/225000 [126:16:17<316:51:29,  6.97s/it]
 27%|██▋       | 61344/225000 [126:16:26<333:51:58,  7.34s/it]
 27%|██▋       | 61345/225000 [126:16:30<295:16:00,  6.50s/it]
 27%|██▋       | 61346/225000 [126:16:38<311:12:51,  6.85s/it]
 27%|██▋       | 61347/225000 [126:16:46<327:48:45,  7.21s/it]
 27%|██▋       | 61348/225000 [126:16:50<281:19:33,  6.19s/it]
 27%|██▋       | 61349/225000 [126:16:55<265:00:10,  5.83s/it]
 27%|██▋       | 61350/225000 [126:16:59<252:32:00,  5.56s/it]
                                                              
{'loss': '0.7208', 'grad_norm': '0.7511', 'learning_rate': '4.329e-05', 'epoch': '0.2727', 'num_input_tokens_seen': 210159328, 'train_runtime': '4.546e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61350/225000 [126:16:59<252:32:00,  5.56s/it]
 27%|██▋       | 61351/225000 [126:17:04<245:07:55,  5.39s/it]
 27%|██▋       | 61352/225000 [126:17:16<325:47:54,  7.17s/it]
 27%|██▋       | 61353/225000 [126:17:23<324:46:10,  7.14s/it]
 27%|██▋       | 61354/225000 [126:17:29<313:55:26,  6.91s/it]
 27%|██▋       | 61355/225000 [126:17:37<321:01:43,  7.06s/it]
 27%|██▋       | 61356/225000 [126:17:42<297:45:41,  6.55s/it]
 27%|██▋       | 61357/225000 [126:17:48<288:23:35,  6.34s/it]
 27%|██▋       | 61358/225000 [126:17:53<274:13:13,  6.03s/it]
 27%|██▋       | 61359/225000 [126:17:59<271:33:28,  5.97s/it]
 27%|██▋       | 61360/225000 [126:18:07<302:10:59,  6.65s/it]
 27%|██▋       | 61361/225000 [126:18:14<303:57:47,  6.69s/it]
 27%|██▋       | 61362/225000 [126:18:19<286:33:48,  6.30s/it]
 27%|██▋       | 61363/225000 [126:18:24<268:29:43,  5.91s/it]
 27%|██▋       | 61364/225000 [126:18:30<266:56:26,  5.87s/it]
 27%|██▋       | 61365/225000 [126:18:36<260:22:10,  5.73s/it]
 27%|██▋       | 61366/225000 [126:18:42<263:46:46,  5.80s/it]
 27%|██▋       | 61367/225000 [126:18:54<359:17:49,  7.90s/it]
 27%|██▋       | 61368/225000 [126:19:02<361:28:22,  7.95s/it]
 27%|██▋       | 61369/225000 [126:19:07<312:16:38,  6.87s/it]
 27%|██▋       | 61370/225000 [126:19:12<293:23:26,  6.45s/it]
 27%|██▋       | 61371/225000 [126:19:18<280:49:05,  6.18s/it]
 27%|██▋       | 61372/225000 [126:19:24<277:37:21,  6.11s/it]
 27%|██▋       | 61373/225000 [126:19:29<268:55:34,  5.92s/it]
 27%|██▋       | 61374/225000 [126:19:35<261:31:45,  5.75s/it]
 27%|██▋       | 61375/225000 [126:19:42<287:09:52,  6.32s/it]
 27%|██▋       | 61376/225000 [126:19:56<382:25:18,  8.41s/it]
 27%|██▋       | 61377/225000 [126:20:01<347:34:18,  7.65s/it]
 27%|██▋       | 61378/225000 [126:20:08<328:39:42,  7.23s/it]
 27%|██▋       | 61379/225000 [126:20:12<284:27:39,  6.26s/it]
 27%|██▋       | 61380/225000 [126:20:22<339:06:29,  7.46s/it]
 27%|██▋       | 61381/225000 [126:20:27<303:06:14,  6.67s/it]
 27%|██▋       | 61382/225000 [126:20:35<321:27:08,  7.07s/it]
 27%|██▋       | 61383/225000 [126:20:43<340:13:44,  7.49s/it]
 27%|██▋       | 61384/225000 [126:20:48<303:10:52,  6.67s/it]
 27%|██▋       | 61385/225000 [126:20:57<331:22:47,  7.29s/it]
 27%|██▋       | 61386/225000 [126:21:03<318:41:42,  7.01s/it]
 27%|██▋       | 61387/225000 [126:21:09<306:21:59,  6.74s/it]
 27%|██▋       | 61388/225000 [126:21:16<310:46:20,  6.84s/it]
 27%|██▋       | 61389/225000 [126:21:24<318:52:21,  7.02s/it]
 27%|██▋       | 61390/225000 [126:21:31<321:10:20,  7.07s/it]
 27%|██▋       | 61391/225000 [126:21:35<279:11:01,  6.14s/it]
 27%|██▋       | 61392/225000 [126:21:39<258:00:06,  5.68s/it]
 27%|██▋       | 61393/225000 [126:21:49<314:37:39,  6.92s/it]
 27%|██▋       | 61394/225000 [126:21:56<317:18:35,  6.98s/it]
 27%|██▋       | 61395/225000 [126:22:02<296:21:29,  6.52s/it]
 27%|██▋       | 61396/225000 [126:22:07<278:01:19,  6.12s/it]
 27%|██▋       | 61397/225000 [126:22:13<277:07:43,  6.10s/it]
 27%|██▋       | 61398/225000 [126:22:19<276:07:36,  6.08s/it]
 27%|██▋       | 61399/225000 [126:22:25<273:24:43,  6.02s/it]
 27%|██▋       | 61400/225000 [126:22:30<263:13:39,  5.79s/it]
                                                              
{'loss': '0.7365', 'grad_norm': '1.006', 'learning_rate': '4.328e-05', 'epoch': '0.2729', 'num_input_tokens_seen': 210309168, 'train_runtime': '4.55e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61400/225000 [126:22:30<263:13:39,  5.79s/it]
 27%|██▋       | 61401/225000 [126:22:36<259:01:32,  5.70s/it]
 27%|██▋       | 61402/225000 [126:22:45<308:41:00,  6.79s/it]
 27%|██▋       | 61403/225000 [126:22:54<333:56:04,  7.35s/it]
 27%|██▋       | 61404/225000 [126:22:58<299:01:40,  6.58s/it]
 27%|██▋       | 61405/225000 [126:23:07<319:00:03,  7.02s/it]
 27%|██▋       | 61406/225000 [126:23:11<288:51:47,  6.36s/it]
 27%|██▋       | 61407/225000 [126:23:16<266:06:56,  5.86s/it]
 27%|██▋       | 61408/225000 [126:23:22<264:04:57,  5.81s/it]
 27%|██▋       | 61409/225000 [126:23:34<349:46:47,  7.70s/it]
 27%|██▋       | 61410/225000 [126:23:41<346:11:41,  7.62s/it]
 27%|██▋       | 61411/225000 [126:23:47<318:57:36,  7.02s/it]
 27%|██▋       | 61412/225000 [126:23:52<289:54:31,  6.38s/it]
 27%|██▋       | 61413/225000 [126:23:58<281:23:29,  6.19s/it]
 27%|██▋       | 61414/225000 [126:24:05<294:10:41,  6.47s/it]
 27%|██▋       | 61415/225000 [126:24:13<322:30:16,  7.10s/it]
 27%|██▋       | 61416/225000 [126:24:18<297:32:48,  6.55s/it]
 27%|██▋       | 61417/225000 [126:24:23<272:37:08,  6.00s/it]
 27%|██▋       | 61418/225000 [126:24:28<262:43:43,  5.78s/it]
 27%|██▋       | 61419/225000 [126:24:41<358:43:30,  7.89s/it]
 27%|██▋       | 61420/225000 [126:24:45<306:43:47,  6.75s/it]
 27%|██▋       | 61421/225000 [126:24:52<302:40:34,  6.66s/it]
 27%|██▋       | 61422/225000 [126:25:00<319:35:04,  7.03s/it]
 27%|██▋       | 61423/225000 [126:25:04<286:08:24,  6.30s/it]
 27%|██▋       | 61424/225000 [126:25:09<261:20:21,  5.75s/it]
 27%|██▋       | 61425/225000 [126:25:21<346:29:12,  7.63s/it]
 27%|██▋       | 61426/225000 [126:25:24<292:44:02,  6.44s/it]
 27%|██▋       | 61427/225000 [126:25:33<316:18:02,  6.96s/it]
 27%|██▋       | 61428/225000 [126:25:37<285:14:17,  6.28s/it]
 27%|██▋       | 61429/225000 [126:25:43<273:20:15,  6.02s/it]
 27%|██▋       | 61430/225000 [126:25:54<346:35:57,  7.63s/it]
 27%|██▋       | 61431/225000 [126:25:58<295:07:04,  6.50s/it]
 27%|██▋       | 61432/225000 [126:26:07<324:43:20,  7.15s/it]
 27%|██▋       | 61433/225000 [126:26:10<278:09:28,  6.12s/it]
 27%|██▋       | 61434/225000 [126:26:15<259:59:58,  5.72s/it]
 27%|██▋       | 61435/225000 [126:26:26<328:11:23,  7.22s/it]
 27%|██▋       | 61436/225000 [126:26:30<291:52:06,  6.42s/it]
 27%|██▋       | 61437/225000 [126:26:35<266:30:00,  5.87s/it]
 27%|██▋       | 61438/225000 [126:26:41<270:36:58,  5.96s/it]
 27%|██▋       | 61439/225000 [126:26:50<313:16:49,  6.90s/it]
 27%|██▋       | 61440/225000 [126:26:55<287:09:59,  6.32s/it]
 27%|██▋       | 61441/225000 [126:27:00<260:16:52,  5.73s/it]
 27%|██▋       | 61442/225000 [126:27:12<350:32:29,  7.72s/it]
 27%|██▋       | 61443/225000 [126:27:17<308:47:20,  6.80s/it]
 27%|██▋       | 61444/225000 [126:27:21<276:49:29,  6.09s/it]
 27%|██▋       | 61445/225000 [126:27:27<269:45:02,  5.94s/it]
 27%|██▋       | 61446/225000 [126:27:32<264:23:10,  5.82s/it]
 27%|██▋       | 61447/225000 [126:27:38<259:11:03,  5.70s/it]
 27%|██▋       | 61448/225000 [126:27:51<362:18:24,  7.97s/it]
 27%|██▋       | 61449/225000 [126:27:55<316:07:20,  6.96s/it]
 27%|██▋       | 61450/225000 [126:28:04<332:46:24,  7.32s/it]
                                                              
{'loss': '0.7345', 'grad_norm': '0.7704', 'learning_rate': '4.327e-05', 'epoch': '0.2731', 'num_input_tokens_seen': 210457040, 'train_runtime': '4.553e+05', 'train_tokens_per_second': '462.3'}

 27%|██▋       | 61450/225000 [126:28:04<332:46:24,  7.32s/it]
 27%|██▋       | 61451/225000 [126:28:08<297:22:11,  6.55s/it]
 27%|██▋       | 61452/225000 [126:28:13<271:22:23,  5.97s/it]
 27%|██▋       | 61453/225000 [126:28:17<251:03:34,  5.53s/it]
 27%|██▋       | 61454/225000 [126:28:23<246:40:26,  5.43s/it]
 27%|██▋       | 61455/225000 [126:28:35<338:32:36,  7.45s/it]
 27%|██▋       | 61456/225000 [126:28:41<323:17:05,  7.12s/it]
 27%|██▋       | 61457/225000 [126:28:46<287:08:49,  6.32s/it]
 27%|██▋       | 61458/225000 [126:28:56<347:16:59,  7.64s/it]
 27%|██▋       | 61459/225000 [126:29:03<332:14:53,  7.31s/it]
 27%|██▋       | 61460/225000 [126:29:08<297:05:38,  6.54s/it]
 27%|██▋       | 61461/225000 [126:29:12<273:25:18,  6.02s/it]
 27%|██▋       | 61462/225000 [126:29:18<263:27:12,  5.80s/it]
 27%|██▋       | 61463/225000 [126:29:23<261:29:46,  5.76s/it]
 27%|██▋       | 61464/225000 [126:29:36<357:36:23,  7.87s/it]
 27%|██▋       | 61465/225000 [126:29:40<306:35:48,  6.75s/it]
 27%|██▋       | 61466/225000 [126:29:50<346:53:01,  7.64s/it]
 27%|██▋       | 61467/225000 [126:29:55<314:35:38,  6.93s/it]
 27%|██▋       | 61468/225000 [126:30:03<323:20:33,  7.12s/it]
 27%|██▋       | 61469/225000 [126:30:09<308:44:05,  6.80s/it]
 27%|██▋       | 61470/225000 [126:30:14<290:46:59,  6.40s/it]
 27%|██▋       | 61471/225000 [126:30:20<275:29:40,  6.06s/it]
 27%|██▋       | 61472/225000 [126:30:25<262:32:40,  5.78s/it]
 27%|██▋       | 61473/225000 [126:30:32<281:26:20,  6.20s/it]
 27%|██▋       | 61474/225000 [126:30:43<346:31:43,  7.63s/it]
 27%|██▋       | 61475/225000 [126:30:48<309:40:44,  6.82s/it]
 27%|██▋       | 61476/225000 [126:30:53<288:09:01,  6.34s/it]
 27%|██▋       | 61477/225000 [126:30:59<275:32:24,  6.07s/it]
 27%|██▋       | 61478/225000 [126:31:04<263:06:37,  5.79s/it]
 27%|██▋       | 61479/225000 [126:31:09<258:24:31,  5.69s/it]
 27%|██▋       | 61480/225000 [126:31:22<356:20:58,  7.85s/it]
 27%|██▋       | 61481/225000 [126:31:28<333:53:00,  7.35s/it]
 27%|██▋       | 61482/225000 [126:31:33<304:21:19,  6.70s/it]
 27%|██▋       | 61483/225000 [126:31:39<288:19:13,  6.35s/it]
 27%|██▋       | 61484/225000 [126:31:44<275:12:40,  6.06s/it]
 27%|██▋       | 61485/225000 [126:31:50<266:31:49,  5.87s/it]
 27%|██▋       | 61486/225000 [126:31:56<265:43:56,  5.85s/it]
 27%|██▋       | 61487/225000 [126:32:05<319:27:29,  7.03s/it]
 27%|██▋       | 61488/225000 [126:32:16<369:14:02,  8.13s/it]
 27%|██▋       | 61489/225000 [126:32:23<354:05:01,  7.80s/it]
 27%|██▋       | 61490/225000 [126:32:27<305:43:52,  6.73s/it]
 27%|██▋       | 61491/225000 [126:32:31<270:10:12,  5.95s/it]
 27%|██▋       | 61492/225000 [126:32:40<311:22:56,  6.86s/it]
 27%|██▋       | 61493/225000 [126:32:48<319:58:26,  7.05s/it]
 27%|██▋       | 61494/225000 [126:32:56<329:38:43,  7.26s/it]
 27%|██▋       | 61495/225000 [126:33:00<295:42:52,  6.51s/it]
 27%|██▋       | 61496/225000 [126:33:06<278:34:04,  6.13s/it]
 27%|██▋       | 61497/225000 [126:33:11<273:26:11,  6.02s/it]
 27%|██▋       | 61498/225000 [126:33:17<269:52:50,  5.94s/it]
 27%|██▋       | 61499/225000 [126:33:22<255:52:32,  5.63s/it]
 27%|██▋       | 61500/225000 [126:33:28<262:17:17,  5.78s/it]
                                                              
{'loss': '0.7573', 'grad_norm': '0.7195', 'learning_rate': '4.326e-05', 'epoch': '0.2733', 'num_input_tokens_seen': 210604768, 'train_runtime': '4.556e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61500/225000 [126:33:28<262:17:17,  5.78s/it]
 27%|██▋       | 61501/225000 [126:33:34<261:25:21,  5.76s/it]
 27%|██▋       | 61502/225000 [126:33:45<332:53:25,  7.33s/it]
 27%|██▋       | 61503/225000 [126:33:50<305:12:45,  6.72s/it]
 27%|██▋       | 61504/225000 [126:33:57<302:46:01,  6.67s/it]
 27%|██▋       | 61505/225000 [126:34:02<287:22:21,  6.33s/it]
 27%|██▋       | 61506/225000 [126:34:08<275:59:53,  6.08s/it]
 27%|██▋       | 61507/225000 [126:34:15<293:39:51,  6.47s/it]
 27%|██▋       | 61508/225000 [126:34:28<376:01:16,  8.28s/it]
 27%|██▋       | 61509/225000 [126:34:31<314:16:47,  6.92s/it]
 27%|██▋       | 61510/225000 [126:34:41<355:58:28,  7.84s/it]
 27%|██▋       | 61511/225000 [126:34:45<300:29:40,  6.62s/it]
 27%|██▋       | 61512/225000 [126:34:55<341:02:19,  7.51s/it]
 27%|██▋       | 61513/225000 [126:34:58<289:16:14,  6.37s/it]
 27%|██▋       | 61514/225000 [126:35:06<302:54:43,  6.67s/it]
 27%|██▋       | 61515/225000 [126:35:11<287:29:54,  6.33s/it]
 27%|██▋       | 61516/225000 [126:35:21<331:49:19,  7.31s/it]
 27%|██▋       | 61517/225000 [126:35:25<289:15:06,  6.37s/it]
 27%|██▋       | 61518/225000 [126:35:34<322:48:11,  7.11s/it]
 27%|██▋       | 61519/225000 [126:35:39<292:47:12,  6.45s/it]
 27%|██▋       | 61520/225000 [126:35:46<299:44:06,  6.60s/it]
 27%|██▋       | 61521/225000 [126:35:51<274:21:56,  6.04s/it]
 27%|██▋       | 61522/225000 [126:35:56<268:15:03,  5.91s/it]
 27%|██▋       | 61523/225000 [126:36:02<263:06:23,  5.79s/it]
 27%|██▋       | 61524/225000 [126:36:07<259:22:17,  5.71s/it]
 27%|██▋       | 61525/225000 [126:36:13<257:20:19,  5.67s/it]
 27%|██▋       | 61526/225000 [126:36:19<268:34:25,  5.91s/it]
 27%|██▋       | 61527/225000 [126:36:32<359:18:05,  7.91s/it]
 27%|██▋       | 61528/225000 [126:36:39<349:12:53,  7.69s/it]
 27%|██▋       | 61529/225000 [126:36:46<339:19:47,  7.47s/it]
 27%|██▋       | 61530/225000 [126:36:49<285:06:00,  6.28s/it]
 27%|██▋       | 61531/225000 [126:36:59<328:40:40,  7.24s/it]
 27%|██▋       | 61532/225000 [126:37:03<280:04:09,  6.17s/it]
 27%|██▋       | 61533/225000 [126:37:11<304:17:47,  6.70s/it]
 27%|██▋       | 61534/225000 [126:37:15<270:21:49,  5.95s/it]
 27%|██▋       | 61535/225000 [126:37:25<331:28:03,  7.30s/it]
 27%|██▋       | 61536/225000 [126:37:29<285:25:10,  6.29s/it]
 27%|██▋       | 61537/225000 [126:37:37<301:06:01,  6.63s/it]
 27%|██▋       | 61538/225000 [126:37:44<317:01:02,  6.98s/it]
 27%|██▋       | 61539/225000 [126:37:51<315:00:42,  6.94s/it]
 27%|██▋       | 61540/225000 [126:37:57<302:20:11,  6.66s/it]
 27%|██▋       | 61541/225000 [126:38:03<289:39:54,  6.38s/it]
 27%|██▋       | 61542/225000 [126:38:10<300:24:26,  6.62s/it]
 27%|██▋       | 61543/225000 [126:38:18<315:56:24,  6.96s/it]
 27%|██▋       | 61544/225000 [126:38:23<297:12:08,  6.55s/it]
 27%|██▋       | 61545/225000 [126:38:29<286:09:52,  6.30s/it]
 27%|██▋       | 61546/225000 [126:38:35<275:28:37,  6.07s/it]
 27%|██▋       | 61547/225000 [126:38:41<272:41:35,  6.01s/it]
 27%|██▋       | 61548/225000 [126:38:46<263:34:44,  5.81s/it]
 27%|██▋       | 61549/225000 [126:38:54<290:43:02,  6.40s/it]
 27%|██▋       | 61550/225000 [126:38:59<281:07:35,  6.19s/it]
                                                              
{'loss': '0.7365', 'grad_norm': '0.8455', 'learning_rate': '4.324e-05', 'epoch': '0.2736', 'num_input_tokens_seen': 210753792, 'train_runtime': '4.559e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61550/225000 [126:38:59<281:07:35,  6.19s/it]
 27%|██▋       | 61551/225000 [126:39:05<277:20:07,  6.11s/it]
 27%|██▋       | 61552/225000 [126:39:11<268:53:07,  5.92s/it]
 27%|██▋       | 61553/225000 [126:39:16<252:37:14,  5.56s/it]
 27%|██▋       | 61554/225000 [126:39:28<340:43:49,  7.50s/it]
 27%|██▋       | 61555/225000 [126:39:35<339:24:39,  7.48s/it]
 27%|██▋       | 61556/225000 [126:39:38<282:16:56,  6.22s/it]
 27%|██▋       | 61557/225000 [126:39:47<310:37:35,  6.84s/it]
 27%|██▋       | 61558/225000 [126:39:52<286:43:02,  6.32s/it]
 27%|██▋       | 61559/225000 [126:39:57<270:08:41,  5.95s/it]
 27%|██▋       | 61560/225000 [126:40:09<349:57:16,  7.71s/it]
 27%|██▋       | 61561/225000 [126:40:14<323:59:17,  7.14s/it]
 27%|██▋       | 61562/225000 [126:40:19<291:09:10,  6.41s/it]
 27%|██▋       | 61563/225000 [126:40:24<274:26:28,  6.05s/it]
 27%|██▋       | 61564/225000 [126:40:32<290:52:40,  6.41s/it]
 27%|██▋       | 61565/225000 [126:40:42<343:28:53,  7.57s/it]
 27%|██▋       | 61566/225000 [126:40:45<288:15:44,  6.35s/it]
 27%|██▋       | 61567/225000 [126:40:54<316:33:19,  6.97s/it]
 27%|██▋       | 61568/225000 [126:41:01<325:39:49,  7.17s/it]
 27%|██▋       | 61569/225000 [126:41:06<289:42:49,  6.38s/it]
 27%|██▋       | 61570/225000 [126:41:14<306:46:31,  6.76s/it]
 27%|██▋       | 61571/225000 [126:41:22<332:23:55,  7.32s/it]
 27%|██▋       | 61572/225000 [126:41:26<289:09:32,  6.37s/it]
 27%|██▋       | 61573/225000 [126:41:31<266:17:51,  5.87s/it]
 27%|██▋       | 61574/225000 [126:41:36<257:08:48,  5.66s/it]
 27%|██▋       | 61575/225000 [126:41:41<250:02:04,  5.51s/it]
 27%|██▋       | 61576/225000 [126:41:55<355:36:38,  7.83s/it]
 27%|██▋       | 61577/225000 [126:42:01<333:16:00,  7.34s/it]
 27%|██▋       | 61578/225000 [126:42:06<303:34:49,  6.69s/it]
 27%|██▋       | 61579/225000 [126:42:12<293:26:13,  6.46s/it]
 27%|██▋       | 61580/225000 [126:42:18<282:52:45,  6.23s/it]
 27%|██▋       | 61581/225000 [126:42:23<271:00:38,  5.97s/it]
 27%|██▋       | 61582/225000 [126:42:28<257:32:44,  5.67s/it]
 27%|██▋       | 61583/225000 [126:42:34<269:48:24,  5.94s/it]
 27%|██▋       | 61584/225000 [126:42:47<365:09:53,  8.04s/it]
 27%|██▋       | 61585/225000 [126:42:54<348:17:00,  7.67s/it]
 27%|██▋       | 61586/225000 [126:43:00<316:29:14,  6.97s/it]
 27%|██▋       | 61587/225000 [126:43:07<322:10:29,  7.10s/it]
 27%|██▋       | 61588/225000 [126:43:12<300:29:42,  6.62s/it]
 27%|██▋       | 61589/225000 [126:43:18<278:59:14,  6.15s/it]
 27%|██▋       | 61590/225000 [126:43:23<273:55:09,  6.03s/it]
 27%|██▋       | 61591/225000 [126:43:29<262:38:03,  5.79s/it]
 27%|██▋       | 61592/225000 [126:43:34<258:34:34,  5.70s/it]
 27%|██▋       | 61593/225000 [126:43:39<250:12:26,  5.51s/it]
 27%|██▋       | 61594/225000 [126:43:51<337:27:00,  7.43s/it]
 27%|██▋       | 61595/225000 [126:44:00<362:38:05,  7.99s/it]
 27%|██▋       | 61596/225000 [126:44:05<313:23:18,  6.90s/it]
 27%|██▋       | 61597/225000 [126:44:09<277:28:25,  6.11s/it]
 27%|██▋       | 61598/225000 [126:44:21<360:53:47,  7.95s/it]
 27%|██▋       | 61599/225000 [126:44:26<322:52:12,  7.11s/it]
 27%|██▋       | 61600/225000 [126:44:32<297:47:56,  6.56s/it]
                                                              
{'loss': '0.7466', 'grad_norm': '0.8013', 'learning_rate': '4.323e-05', 'epoch': '0.2738', 'num_input_tokens_seen': 210901584, 'train_runtime': '4.563e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61600/225000 [126:44:32<297:47:56,  6.56s/it]
 27%|██▋       | 61601/225000 [126:44:37<279:28:17,  6.16s/it]
 27%|██▋       | 61602/225000 [126:44:42<265:00:58,  5.84s/it]
 27%|██▋       | 61603/225000 [126:44:47<257:33:22,  5.67s/it]
 27%|██▋       | 61604/225000 [126:44:58<324:00:48,  7.14s/it]
 27%|██▋       | 61605/225000 [126:45:08<365:29:10,  8.05s/it]
 27%|██▋       | 61606/225000 [126:45:12<312:29:49,  6.89s/it]
 27%|██▋       | 61607/225000 [126:45:22<354:48:48,  7.82s/it]
 27%|██▋       | 61608/225000 [126:45:26<302:33:10,  6.67s/it]
 27%|██▋       | 61609/225000 [126:45:32<287:00:53,  6.32s/it]
 27%|██▋       | 61610/225000 [126:45:37<270:07:11,  5.95s/it]
 27%|██▋       | 61611/225000 [126:45:46<318:53:05,  7.03s/it]
 27%|██▋       | 61612/225000 [126:45:58<383:01:01,  8.44s/it]
 27%|██▋       | 61613/225000 [126:46:02<316:43:21,  6.98s/it]
 27%|██▋       | 61614/225000 [126:46:11<353:37:05,  7.79s/it]
 27%|██▋       | 61615/225000 [126:46:16<310:36:33,  6.84s/it]
 27%|██▋       | 61616/225000 [126:46:22<301:49:31,  6.65s/it]
 27%|██▋       | 61617/225000 [126:46:28<286:22:25,  6.31s/it]
 27%|██▋       | 61618/225000 [126:46:36<316:42:03,  6.98s/it]
 27%|██▋       | 61619/225000 [126:46:45<338:56:08,  7.47s/it]
 27%|██▋       | 61620/225000 [126:46:49<302:27:00,  6.66s/it]
 27%|██▋       | 61621/225000 [126:46:55<290:28:38,  6.40s/it]
 27%|██▋       | 61622/225000 [126:47:01<282:24:24,  6.22s/it]
 27%|██▋       | 61623/225000 [126:47:07<272:14:42,  6.00s/it]
 27%|██▋       | 61624/225000 [126:47:13<276:02:57,  6.08s/it]
 27%|██▋       | 61625/225000 [126:47:25<360:50:21,  7.95s/it]
 27%|██▋       | 61626/225000 [126:47:30<321:23:29,  7.08s/it]
 27%|██▋       | 61627/225000 [126:47:39<340:55:23,  7.51s/it]
 27%|██▋       | 61628/225000 [126:47:43<297:23:24,  6.55s/it]
 27%|██▋       | 61629/225000 [126:47:52<329:21:42,  7.26s/it]
 27%|██▋       | 61630/225000 [126:47:56<292:43:27,  6.45s/it]
 27%|██▋       | 61631/225000 [126:48:01<268:19:08,  5.91s/it]
 27%|██▋       | 61632/225000 [126:48:06<254:25:23,  5.61s/it]
 27%|██▋       | 61633/225000 [126:48:11<239:04:43,  5.27s/it]
 27%|██▋       | 61634/225000 [126:48:16<241:01:11,  5.31s/it]
 27%|██▋       | 61635/225000 [126:48:25<292:13:53,  6.44s/it]
 27%|██▋       | 61636/225000 [126:48:36<360:16:14,  7.94s/it]
 27%|██▋       | 61637/225000 [126:48:40<302:13:58,  6.66s/it]
 27%|██▋       | 61638/225000 [126:48:50<351:50:33,  7.75s/it]
 27%|██▋       | 61639/225000 [126:48:55<309:03:13,  6.81s/it]
 27%|██▋       | 61640/225000 [126:49:00<277:18:00,  6.11s/it]
 27%|██▋       | 61641/225000 [126:49:08<312:22:05,  6.88s/it]
 27%|██▋       | 61642/225000 [126:49:16<322:20:55,  7.10s/it]
 27%|██▋       | 61643/225000 [126:49:21<299:36:47,  6.60s/it]
 27%|██▋       | 61644/225000 [126:49:27<281:29:08,  6.20s/it]
 27%|██▋       | 61645/225000 [126:49:32<272:44:04,  6.01s/it]
 27%|██▋       | 61646/225000 [126:49:37<264:10:13,  5.82s/it]
 27%|██▋       | 61647/225000 [126:49:43<257:52:16,  5.68s/it]
 27%|██▋       | 61648/225000 [126:49:49<258:05:04,  5.69s/it]
 27%|██▋       | 61649/225000 [126:49:54<251:55:44,  5.55s/it]
 27%|██▋       | 61650/225000 [126:50:01<275:06:19,  6.06s/it]
                                                              
{'loss': '0.74', 'grad_norm': '0.8391', 'learning_rate': '4.322e-05', 'epoch': '0.274', 'num_input_tokens_seen': 211051656, 'train_runtime': '4.566e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61650/225000 [126:50:01<275:06:19,  6.06s/it]
 27%|██▋       | 61651/225000 [126:50:15<377:17:07,  8.31s/it]
 27%|██▋       | 61652/225000 [126:50:21<347:22:20,  7.66s/it]
 27%|██▋       | 61653/225000 [126:50:24<294:00:51,  6.48s/it]
 27%|██▋       | 61654/225000 [126:50:32<309:35:30,  6.82s/it]
 27%|██▋       | 61655/225000 [126:50:41<341:47:52,  7.53s/it]
 27%|██▋       | 61656/225000 [126:50:46<306:58:46,  6.77s/it]
 27%|██▋       | 61657/225000 [126:50:51<279:30:05,  6.16s/it]
 27%|██▋       | 61658/225000 [126:50:56<262:54:30,  5.79s/it]
 27%|██▋       | 61659/225000 [126:51:05<305:01:29,  6.72s/it]
 27%|██▋       | 61660/225000 [126:51:15<357:50:55,  7.89s/it]
 27%|██▋       | 61661/225000 [126:51:23<351:20:11,  7.74s/it]
 27%|██▋       | 61662/225000 [126:51:26<292:12:26,  6.44s/it]
 27%|██▋       | 61663/225000 [126:51:33<300:46:38,  6.63s/it]
 27%|██▋       | 61664/225000 [126:51:38<273:18:34,  6.02s/it]
 27%|██▋       | 61665/225000 [126:51:42<249:15:59,  5.49s/it]
 27%|██▋       | 61666/225000 [126:51:47<241:41:35,  5.33s/it]
 27%|██▋       | 61667/225000 [126:52:00<340:09:51,  7.50s/it]
 27%|██▋       | 61668/225000 [126:52:05<308:13:56,  6.79s/it]
 27%|██▋       | 61669/225000 [126:52:09<269:47:30,  5.95s/it]
 27%|██▋       | 61670/225000 [126:52:13<248:13:01,  5.47s/it]
 27%|██▋       | 61671/225000 [126:52:24<328:19:14,  7.24s/it]
 27%|██▋       | 61672/225000 [126:52:31<322:02:59,  7.10s/it]
 27%|██▋       | 61673/225000 [126:52:36<292:33:19,  6.45s/it]
 27%|██▋       | 61674/225000 [126:52:42<289:38:38,  6.38s/it]
 27%|██▋       | 61675/225000 [126:52:50<310:20:15,  6.84s/it]
 27%|██▋       | 61676/225000 [126:53:00<344:57:33,  7.60s/it]
 27%|██▋       | 61677/225000 [126:53:07<340:11:04,  7.50s/it]
 27%|██▋       | 61678/225000 [126:53:14<334:26:26,  7.37s/it]
 27%|██▋       | 61679/225000 [126:53:20<318:11:36,  7.01s/it]
 27%|██▋       | 61680/225000 [126:53:25<292:26:40,  6.45s/it]
 27%|██▋       | 61681/225000 [126:53:34<320:55:06,  7.07s/it]
 27%|██▋       | 61682/225000 [126:53:40<304:30:17,  6.71s/it]
 27%|██▋       | 61683/225000 [126:53:44<266:59:37,  5.89s/it]
 27%|██▋       | 61684/225000 [126:53:49<262:39:28,  5.79s/it]
 27%|██▋       | 61685/225000 [126:53:59<314:07:24,  6.92s/it]
 27%|██▋       | 61686/225000 [126:54:06<322:01:23,  7.10s/it]
 27%|██▋       | 61687/225000 [126:54:13<309:51:08,  6.83s/it]
 27%|██▋       | 61688/225000 [126:54:18<292:21:27,  6.44s/it]
 27%|██▋       | 61689/225000 [126:54:24<280:46:33,  6.19s/it]
 27%|██▋       | 61690/225000 [126:54:30<280:32:32,  6.18s/it]
 27%|██▋       | 61691/225000 [126:54:35<265:46:47,  5.86s/it]
 27%|██▋       | 61692/225000 [126:54:42<280:20:21,  6.18s/it]
 27%|██▋       | 61693/225000 [126:54:55<380:25:09,  8.39s/it]
 27%|██▋       | 61694/225000 [126:55:02<355:17:59,  7.83s/it]
 27%|██▋       | 61695/225000 [126:55:10<356:46:50,  7.87s/it]
 27%|██▋       | 61696/225000 [126:55:14<299:51:30,  6.61s/it]
 27%|██▋       | 61697/225000 [126:55:21<317:18:00,  6.99s/it]
 27%|██▋       | 61698/225000 [126:55:28<310:16:31,  6.84s/it]
 27%|██▋       | 61699/225000 [126:55:33<283:35:21,  6.25s/it]
 27%|██▋       | 61700/225000 [126:55:41<302:55:16,  6.68s/it]
                                                              
{'loss': '0.7432', 'grad_norm': '0.75', 'learning_rate': '4.321e-05', 'epoch': '0.2742', 'num_input_tokens_seen': 211199576, 'train_runtime': '4.569e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61700/225000 [126:55:41<302:55:16,  6.68s/it]
 27%|██▋       | 61701/225000 [126:55:48<310:19:36,  6.84s/it]
 27%|██▋       | 61702/225000 [126:55:52<274:26:36,  6.05s/it]
 27%|██▋       | 61703/225000 [126:55:59<285:29:23,  6.29s/it]
 27%|██▋       | 61704/225000 [126:56:03<250:08:57,  5.51s/it]
 27%|██▋       | 61705/225000 [126:56:14<327:55:55,  7.23s/it]
 27%|██▋       | 61706/225000 [126:56:17<280:27:55,  6.18s/it]
 27%|██▋       | 61707/225000 [126:56:22<256:50:34,  5.66s/it]
 27%|██▋       | 61708/225000 [126:56:32<321:19:07,  7.08s/it]
 27%|██▋       | 61709/225000 [126:56:38<296:04:04,  6.53s/it]
 27%|██▋       | 61710/225000 [126:56:45<301:39:41,  6.65s/it]
 27%|██▋       | 61711/225000 [126:56:52<315:49:53,  6.96s/it]
 27%|██▋       | 61712/225000 [126:56:59<320:29:55,  7.07s/it]
 27%|██▋       | 61713/225000 [126:57:05<295:03:23,  6.51s/it]
 27%|██▋       | 61714/225000 [126:57:10<272:16:39,  6.00s/it]
 27%|██▋       | 61715/225000 [126:57:15<259:55:21,  5.73s/it]
 27%|██▋       | 61716/225000 [126:57:20<252:00:17,  5.56s/it]
 27%|██▋       | 61717/225000 [126:57:25<243:55:24,  5.38s/it]
 27%|██▋       | 61718/225000 [126:57:30<246:48:29,  5.44s/it]
 27%|██▋       | 61719/225000 [126:57:44<362:38:22,  8.00s/it]
 27%|██▋       | 61720/225000 [126:57:53<366:26:26,  8.08s/it]
 27%|██▋       | 61721/225000 [126:57:58<334:33:23,  7.38s/it]
 27%|██▋       | 61722/225000 [126:58:06<333:19:43,  7.35s/it]
 27%|██▋       | 61723/225000 [126:58:10<293:35:50,  6.47s/it]
 27%|██▋       | 61724/225000 [126:58:15<267:13:11,  5.89s/it]
 27%|██▋       | 61725/225000 [126:58:20<256:50:15,  5.66s/it]
 27%|██▋       | 61726/225000 [126:58:30<322:58:46,  7.12s/it]
 27%|██▋       | 61727/225000 [126:58:37<321:18:18,  7.08s/it]
 27%|██▋       | 61728/225000 [126:58:41<275:22:57,  6.07s/it]
 27%|██▋       | 61729/225000 [126:58:50<320:58:56,  7.08s/it]
 27%|██▋       | 61730/225000 [126:58:55<289:49:54,  6.39s/it]
 27%|██▋       | 61731/225000 [126:59:00<264:38:54,  5.84s/it]
 27%|██▋       | 61732/225000 [126:59:09<309:43:32,  6.83s/it]
 27%|██▋       | 61733/225000 [126:59:16<308:54:13,  6.81s/it]
 27%|██▋       | 61734/225000 [126:59:20<279:18:04,  6.16s/it]
 27%|██▋       | 61735/225000 [126:59:26<275:59:02,  6.09s/it]
 27%|██▋       | 61736/225000 [126:59:32<267:30:30,  5.90s/it]
 27%|██▋       | 61737/225000 [126:59:37<265:30:06,  5.85s/it]
 27%|██▋       | 61738/225000 [126:59:50<353:49:48,  7.80s/it]
 27%|██▋       | 61739/225000 [126:59:54<310:48:27,  6.85s/it]
 27%|██▋       | 61740/225000 [126:59:59<277:14:24,  6.11s/it]
 27%|██▋       | 61741/225000 [127:00:08<317:29:04,  7.00s/it]
 27%|██▋       | 61742/225000 [127:00:17<341:47:31,  7.54s/it]
 27%|██▋       | 61743/225000 [127:00:21<301:54:28,  6.66s/it]
 27%|██▋       | 61744/225000 [127:00:26<281:49:42,  6.21s/it]
 27%|██▋       | 61745/225000 [127:00:32<267:50:54,  5.91s/it]
 27%|██▋       | 61746/225000 [127:00:36<254:50:41,  5.62s/it]
 27%|██▋       | 61747/225000 [127:00:49<352:30:21,  7.77s/it]
 27%|██▋       | 61748/225000 [127:00:56<335:24:54,  7.40s/it]
 27%|██▋       | 61749/225000 [127:01:01<311:56:36,  6.88s/it]
 27%|██▋       | 61750/225000 [127:01:07<297:54:21,  6.57s/it]
                                                              
{'loss': '0.7439', 'grad_norm': '0.7582', 'learning_rate': '4.319e-05', 'epoch': '0.2744', 'num_input_tokens_seen': 211348376, 'train_runtime': '4.573e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61750/225000 [127:01:07<297:54:21,  6.57s/it]
 27%|██▋       | 61751/225000 [127:01:13<286:58:03,  6.33s/it]
 27%|██▋       | 61752/225000 [127:01:19<275:14:27,  6.07s/it]
 27%|██▋       | 61753/225000 [127:01:24<266:58:01,  5.89s/it]
 27%|██▋       | 61754/225000 [127:01:30<263:39:37,  5.81s/it]
 27%|██▋       | 61755/225000 [127:01:35<258:23:48,  5.70s/it]
 27%|██▋       | 61756/225000 [127:01:43<294:26:25,  6.49s/it]
 27%|██▋       | 61757/225000 [127:01:56<381:51:50,  8.42s/it]
 27%|██▋       | 61758/225000 [127:02:02<342:35:52,  7.56s/it]
 27%|██▋       | 61759/225000 [127:02:08<323:10:57,  7.13s/it]
 27%|██▋       | 61760/225000 [127:02:13<296:29:48,  6.54s/it]
 27%|██▋       | 61761/225000 [127:02:18<273:43:25,  6.04s/it]
 27%|██▋       | 61762/225000 [127:02:23<259:40:37,  5.73s/it]
 27%|██▋       | 61763/225000 [127:02:30<275:29:23,  6.08s/it]
 27%|██▋       | 61764/225000 [127:02:43<372:24:07,  8.21s/it]
 27%|██▋       | 61765/225000 [127:02:47<313:32:18,  6.91s/it]
 27%|██▋       | 61766/225000 [127:02:55<327:50:03,  7.23s/it]
 27%|██▋       | 61767/225000 [127:03:01<308:15:47,  6.80s/it]
 27%|██▋       | 61768/225000 [127:03:05<273:47:04,  6.04s/it]
 27%|██▋       | 61769/225000 [127:03:12<280:34:12,  6.19s/it]
 27%|██▋       | 61770/225000 [127:03:23<352:45:17,  7.78s/it]
 27%|██▋       | 61771/225000 [127:03:28<318:21:51,  7.02s/it]
 27%|██▋       | 61772/225000 [127:03:37<340:15:17,  7.50s/it]
 27%|██▋       | 61773/225000 [127:03:42<306:58:22,  6.77s/it]
 27%|██▋       | 61774/225000 [127:03:51<330:46:19,  7.30s/it]
 27%|██▋       | 61775/225000 [127:03:56<304:25:31,  6.71s/it]
 27%|██▋       | 61776/225000 [127:04:04<319:27:14,  7.05s/it]
 27%|██▋       | 61777/225000 [127:04:09<289:22:12,  6.38s/it]
 27%|██▋       | 61778/225000 [127:04:13<267:31:35,  5.90s/it]
 27%|██▋       | 61779/225000 [127:04:19<262:09:41,  5.78s/it]
 27%|██▋       | 61780/225000 [127:04:25<272:20:12,  6.01s/it]
 27%|██▋       | 61781/225000 [127:04:38<357:57:20,  7.90s/it]
 27%|██▋       | 61782/225000 [127:04:42<307:55:01,  6.79s/it]
 27%|██▋       | 61783/225000 [127:04:47<287:16:47,  6.34s/it]
 27%|██▋       | 61784/225000 [127:04:57<338:40:32,  7.47s/it]
 27%|██▋       | 61785/225000 [127:05:02<302:15:08,  6.67s/it]
 27%|██▋       | 61786/225000 [127:05:08<287:48:44,  6.35s/it]
 27%|██▋       | 61787/225000 [127:05:14<282:14:05,  6.23s/it]
 27%|██▋       | 61788/225000 [127:05:20<278:28:41,  6.14s/it]
 27%|██▋       | 61789/225000 [127:05:26<276:35:25,  6.10s/it]
 27%|██▋       | 61790/225000 [127:05:32<276:14:44,  6.09s/it]
 27%|██▋       | 61791/225000 [127:05:40<305:57:46,  6.75s/it]
 27%|██▋       | 61792/225000 [127:05:47<316:34:51,  6.98s/it]
 27%|██▋       | 61793/225000 [127:05:55<323:30:03,  7.14s/it]
 27%|██▋       | 61794/225000 [127:06:00<293:58:38,  6.48s/it]
 27%|██▋       | 61795/225000 [127:06:04<265:55:16,  5.87s/it]
 27%|██▋       | 61796/225000 [127:06:16<346:12:47,  7.64s/it]
 27%|██▋       | 61797/225000 [127:06:22<320:40:24,  7.07s/it]
 27%|██▋       | 61798/225000 [127:06:27<296:57:18,  6.55s/it]
 27%|██▋       | 61799/225000 [127:06:32<279:33:35,  6.17s/it]
 27%|██▋       | 61800/225000 [127:06:38<268:45:03,  5.93s/it]
                                                              
{'loss': '0.7572', 'grad_norm': '0.7845', 'learning_rate': '4.318e-05', 'epoch': '0.2747', 'num_input_tokens_seen': 211497360, 'train_runtime': '4.576e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61800/225000 [127:06:38<268:45:03,  5.93s/it]
 27%|██▋       | 61801/225000 [127:06:43<253:53:22,  5.60s/it]
 27%|██▋       | 61802/225000 [127:06:48<248:18:57,  5.48s/it]
 27%|██▋       | 61803/225000 [127:06:53<248:56:48,  5.49s/it]
 27%|██▋       | 61804/225000 [127:07:07<365:31:00,  8.06s/it]
 27%|██▋       | 61805/225000 [127:07:14<344:17:47,  7.60s/it]
 27%|██▋       | 61806/225000 [127:07:19<309:51:03,  6.84s/it]
 27%|██▋       | 61807/225000 [127:07:24<284:58:51,  6.29s/it]
 27%|██▋       | 61808/225000 [127:07:29<266:42:52,  5.88s/it]
 27%|██▋       | 61809/225000 [127:07:41<356:33:30,  7.87s/it]
 27%|██▋       | 61810/225000 [127:07:50<360:35:36,  7.95s/it]
 27%|██▋       | 61811/225000 [127:07:54<314:02:26,  6.93s/it]
 27%|██▋       | 61812/225000 [127:07:59<286:09:34,  6.31s/it]
 27%|██▋       | 61813/225000 [127:08:04<272:44:05,  6.02s/it]
 27%|██▋       | 61814/225000 [127:08:15<337:39:19,  7.45s/it]
 27%|██▋       | 61815/225000 [127:08:21<320:31:21,  7.07s/it]
 27%|██▋       | 61816/225000 [127:08:30<346:44:54,  7.65s/it]
 27%|██▋       | 61817/225000 [127:08:34<292:13:57,  6.45s/it]
 27%|██▋       | 61818/225000 [127:08:40<286:45:12,  6.33s/it]
 27%|██▋       | 61819/225000 [127:08:49<328:15:31,  7.24s/it]
 27%|██▋       | 61820/225000 [127:08:54<293:47:59,  6.48s/it]
 27%|██▋       | 61821/225000 [127:08:59<278:36:02,  6.15s/it]
 27%|██▋       | 61822/225000 [127:09:04<263:06:18,  5.80s/it]
 27%|██▋       | 61823/225000 [127:09:16<342:59:06,  7.57s/it]
 27%|██▋       | 61824/225000 [127:09:19<283:07:48,  6.25s/it]
 27%|██▋       | 61825/225000 [127:09:27<303:11:24,  6.69s/it]
 27%|██▋       | 61826/225000 [127:09:31<266:17:10,  5.87s/it]
 27%|██▋       | 61827/225000 [127:09:38<285:36:34,  6.30s/it]
 27%|██▋       | 61828/225000 [127:09:45<296:26:11,  6.54s/it]
 27%|██▋       | 61829/225000 [127:09:54<321:42:53,  7.10s/it]
 27%|██▋       | 61830/225000 [127:09:58<286:44:02,  6.33s/it]
 27%|██▋       | 61831/225000 [127:10:04<275:57:08,  6.09s/it]
 27%|██▋       | 61832/225000 [127:10:09<265:08:52,  5.85s/it]
 27%|██▋       | 61833/225000 [127:10:15<259:38:49,  5.73s/it]
 27%|██▋       | 61834/225000 [127:10:20<252:03:16,  5.56s/it]
 27%|██▋       | 61835/225000 [127:10:25<244:22:05,  5.39s/it]
 27%|██▋       | 61836/225000 [127:10:37<341:00:17,  7.52s/it]
 27%|██▋       | 61837/225000 [127:10:44<324:59:07,  7.17s/it]
 27%|██▋       | 61838/225000 [127:10:49<300:25:53,  6.63s/it]
 27%|██▋       | 61839/225000 [127:10:54<275:29:55,  6.08s/it]
 27%|██▋       | 61840/225000 [127:11:03<314:34:34,  6.94s/it]
 27%|██▋       | 61841/225000 [127:11:09<311:42:38,  6.88s/it]
 27%|██▋       | 61842/225000 [127:11:15<291:01:53,  6.42s/it]
 27%|██▋       | 61843/225000 [127:11:25<337:44:50,  7.45s/it]
 27%|██▋       | 61844/225000 [127:11:29<289:37:36,  6.39s/it]
 27%|██▋       | 61845/225000 [127:11:40<356:45:05,  7.87s/it]
 27%|██▋       | 61846/225000 [127:11:44<307:02:45,  6.77s/it]
 27%|██▋       | 61847/225000 [127:11:48<272:25:05,  6.01s/it]
 27%|██▋       | 61848/225000 [127:11:53<254:17:02,  5.61s/it]
 27%|██▋       | 61849/225000 [127:11:58<248:31:31,  5.48s/it]
 27%|██▋       | 61850/225000 [127:12:03<243:40:28,  5.38s/it]
                                                              
{'loss': '0.7442', 'grad_norm': '0.8379', 'learning_rate': '4.317e-05', 'epoch': '0.2749', 'num_input_tokens_seen': 211644536, 'train_runtime': '4.579e+05', 'train_tokens_per_second': '462.2'}

 27%|██▋       | 61850/225000 [127:12:03<243:40:28,  5.38s/it]
 27%|██▋       | 61851/225000 [127:12:15<324:44:09,  7.17s/it]
 27%|██▋       | 61852/225000 [127:12:22<332:05:51,  7.33s/it]
 27%|██▋       | 61853/225000 [127:12:29<319:41:51,  7.05s/it]
 27%|██▋       | 61854/225000 [127:12:36<322:16:23,  7.11s/it]
 27%|██▋       | 61855/225000 [127:12:42<301:19:01,  6.65s/it]
 27%|██▋       | 61856/225000 [127:12:47<277:53:35,  6.13s/it]
 27%|██▋       | 61857/225000 [127:12:52<265:23:29,  5.86s/it]
 27%|██▋       | 61858/225000 [127:12:57<259:17:12,  5.72s/it]
 27%|██▋       | 61859/225000 [127:13:03<256:03:12,  5.65s/it]
 27%|██▋       | 61860/225000 [127:13:08<254:09:24,  5.61s/it]
 27%|██▋       | 61861/225000 [127:13:17<301:27:13,  6.65s/it]
 27%|██▋       | 61862/225000 [127:13:29<374:09:13,  8.26s/it]
 27%|██▋       | 61863/225000 [127:13:35<336:08:23,  7.42s/it]
 27%|██▋       | 61864/225000 [127:13:40<306:54:18,  6.77s/it]
 27%|██▋       | 61865/225000 [127:13:45<279:24:33,  6.17s/it]
 27%|██▋       | 61866/225000 [127:13:51<277:11:03,  6.12s/it]
 27%|██▋       | 61867/225000 [127:14:03<365:43:07,  8.07s/it]
 27%|██▋       | 61868/225000 [127:14:09<338:00:54,  7.46s/it]
 27%|██▋       | 61869/225000 [127:14:17<345:27:47,  7.62s/it]
 27%|██▋       | 61870/225000 [127:14:23<319:57:57,  7.06s/it]
 27%|██▋       | 61871/225000 [127:14:29<306:36:09,  6.77s/it]
 27%|██▋       | 61872/225000 [127:14:36<310:44:28,  6.86s/it]
 27%|██▋       | 61873/225000 [127:14:43<307:31:43,  6.79s/it]
 27%|██▋       | 61874/225000 [127:14:48<281:21:48,  6.21s/it]
 28%|██▊       | 61875/225000 [127:14:57<324:00:17,  7.15s/it]
 28%|██▊       | 61876/225000 [127:15:03<309:42:45,  6.84s/it]
 28%|██▊       | 61877/225000 [127:15:09<293:57:32,  6.49s/it]
 28%|██▊       | 61878/225000 [127:15:14<280:44:12,  6.20s/it]
 28%|██▊       | 61879/225000 [127:15:24<322:35:42,  7.12s/it]
 28%|██▊       | 61880/225000 [127:15:32<334:12:27,  7.38s/it]
 28%|██▊       | 61881/225000 [127:15:36<287:58:14,  6.36s/it]
 28%|██▊       | 61882/225000 [127:15:40<262:06:45,  5.78s/it]
 28%|██▊       | 61883/225000 [127:15:52<346:00:49,  7.64s/it]
 28%|██▊       | 61884/225000 [127:15:56<295:37:55,  6.52s/it]
 28%|██▊       | 61885/225000 [127:16:01<279:40:34,  6.17s/it]
 28%|██▊       | 61886/225000 [127:16:07<274:14:22,  6.05s/it]
 28%|██▊       | 61887/225000 [127:16:13<266:46:13,  5.89s/it]
 28%|██▊       | 61888/225000 [127:16:18<260:20:48,  5.75s/it]
 28%|██▊       | 61889/225000 [127:16:31<357:44:44,  7.90s/it]
 28%|██▊       | 61890/225000 [127:16:38<339:36:49,  7.50s/it]
 28%|██▊       | 61891/225000 [127:16:41<290:17:36,  6.41s/it]
 28%|██▊       | 61892/225000 [127:16:51<339:53:47,  7.50s/it]
 28%|██▊       | 61893/225000 [127:16:56<296:47:14,  6.55s/it]
 28%|██▊       | 61894/225000 [127:17:01<272:50:15,  6.02s/it]
 28%|██▊       | 61895/225000 [127:17:09<308:29:25,  6.81s/it]
 28%|██▊       | 61896/225000 [127:17:13<266:31:16,  5.88s/it]
 28%|██▊       | 61897/225000 [127:17:24<340:45:15,  7.52s/it]
 28%|██▊       | 61898/225000 [127:17:28<290:34:01,  6.41s/it]
 28%|██▊       | 61899/225000 [127:17:33<264:00:09,  5.83s/it]
 28%|██▊       | 61900/225000 [127:17:37<251:49:40,  5.56s/it]
                                                              
{'loss': '0.7526', 'grad_norm': '0.8632', 'learning_rate': '4.316e-05', 'epoch': '0.2751', 'num_input_tokens_seen': 211793480, 'train_runtime': '4.583e+05', 'train_tokens_per_second': '462.2'}

 28%|██▊       | 61900/225000 [127:17:37<251:49:40,  5.56s/it]
 28%|██▊       | 61901/225000 [127:17:43<246:47:00,  5.45s/it]
 28%|██▊       | 61902/225000 [127:17:52<305:39:11,  6.75s/it]
 28%|██▊       | 61903/225000 [127:18:04<364:38:00,  8.05s/it]
 28%|██▊       | 61904/225000 [127:18:07<306:26:43,  6.76s/it]
 28%|██▊       | 61905/225000 [127:18:18<363:12:39,  8.02s/it]
 28%|██▊       | 61906/225000 [127:18:22<300:38:27,  6.64s/it]
 28%|██▊       | 61907/225000 [127:18:29<312:09:58,  6.89s/it]
 28%|██▊       | 61908/225000 [127:18:37<325:01:49,  7.17s/it]
 28%|██▊       | 61909/225000 [127:18:43<307:08:06,  6.78s/it]
 28%|██▊       | 61910/225000 [127:18:50<311:11:21,  6.87s/it]
 28%|██▊       | 61911/225000 [127:18:55<293:18:33,  6.47s/it]
 28%|██▊       | 61912/225000 [127:19:00<272:01:40,  6.00s/it]
 28%|██▊       | 61913/225000 [127:19:10<325:57:16,  7.20s/it]
 28%|██▊       | 61914/225000 [127:19:18<330:51:39,  7.30s/it]
 28%|██▊       | 61915/225000 [127:19:23<296:56:18,  6.55s/it]
 28%|██▊       | 61916/225000 [127:19:27<263:55:52,  5.83s/it]
 28%|██▊       | 61917/225000 [127:19:32<252:32:14,  5.57s/it]
 28%|██▊       | 61918/225000 [127:19:40<289:48:41,  6.40s/it]
 28%|██▊       | 61919/225000 [127:19:51<344:54:10,  7.61s/it]
 28%|██▊       | 61920/225000 [127:19:55<301:40:42,  6.66s/it]
 28%|██▊       | 61921/225000 [127:20:01<296:02:44,  6.54s/it]
 28%|██▊       | 61922/225000 [127:20:09<312:44:35,  6.90s/it]
 28%|██▊       | 61923/225000 [127:20:13<274:21:57,  6.06s/it]
 28%|██▊       | 61924/225000 [127:20:19<266:06:00,  5.87s/it]
 28%|██▊       | 61925/225000 [127:20:29<328:38:51,  7.26s/it]
 28%|██▊       | 61926/225000 [127:20:37<339:26:43,  7.49s/it]
 28%|██▊       | 61927/225000 [127:20:41<295:43:37,  6.53s/it]
 28%|██▊       | 61928/225000 [127:20:47<278:13:02,  6.14s/it]
 28%|██▊       | 61929/225000 [127:20:57<340:26:52,  7.52s/it]
 28%|██▊       | 61930/225000 [127:21:03<316:55:21,  7.00s/it]
 28%|██▊       | 61931/225000 [127:21:11<322:22:31,  7.12s/it]
 28%|██▊       | 61932/225000 [127:21:17<320:01:25,  7.07s/it]
 28%|██▊       | 61933/225000 [127:21:24<315:12:30,  6.96s/it]
 28%|██▊       | 61934/225000 [127:21:31<319:11:51,  7.05s/it]
 28%|██▊       | 61935/225000 [127:21:36<283:49:30,  6.27s/it]
 28%|██▊       | 61936/225000 [127:21:40<259:51:07,  5.74s/it]
 28%|██▊       | 61937/225000 [127:21:52<340:42:29,  7.52s/it]
 28%|██▊       | 61938/225000 [127:21:56<295:26:11,  6.52s/it]
 28%|██▊       | 61939/225000 [127:22:04<316:50:58,  7.00s/it]
 28%|██▊       | 61940/225000 [127:22:09<281:37:31,  6.22s/it]
 28%|██▊       | 61941/225000 [127:22:16<295:54:11,  6.53s/it]
 28%|██▊       | 61942/225000 [127:22:24<315:24:31,  6.96s/it]
 28%|██▊       | 61943/225000 [127:22:28<276:53:19,  6.11s/it]
 28%|██▊       | 61944/225000 [127:22:36<304:13:20,  6.72s/it]
 28%|██▊       | 61945/225000 [127:22:43<311:20:29,  6.87s/it]
 28%|██▊       | 61946/225000 [127:22:48<283:37:43,  6.26s/it]
 28%|██▊       | 61947/225000 [127:22:55<284:13:08,  6.28s/it]
 28%|██▊       | 61948/225000 [127:23:04<323:10:51,  7.14s/it]
 28%|██▊       | 61949/225000 [127:23:08<279:32:00,  6.17s/it]
 28%|██▊       | 61950/225000 [127:23:13<271:09:12,  5.99s/it]
                                                              
{'loss': '0.7547', 'grad_norm': '1.079', 'learning_rate': '4.314e-05', 'epoch': '0.2753', 'num_input_tokens_seen': 211944088, 'train_runtime': '4.586e+05', 'train_tokens_per_second': '462.2'}

 28%|██▊       | 61950/225000 [127:23:13<271:09:12,  5.99s/it]
 28%|██▊       | 61951/225000 [127:23:18<257:44:47,  5.69s/it]
 28%|██▊       | 61952/225000 [127:23:24<254:22:43,  5.62s/it]
 28%|██▊       | 61953/225000 [127:23:29<252:21:33,  5.57s/it]
 28%|██▊       | 61954/225000 [127:23:35<261:14:33,  5.77s/it]
 28%|██▊       | 61955/225000 [127:23:42<266:51:43,  5.89s/it]
 28%|██▊       | 61956/225000 [127:23:47<262:26:37,  5.79s/it]
 28%|██▊       | 61957/225000 [127:23:56<299:42:34,  6.62s/it]
 28%|██▊       | 61958/225000 [127:24:08<378:14:54,  8.35s/it]
 28%|██▊       | 61959/225000 [127:24:13<325:24:24,  7.19s/it]
 28%|██▊       | 61960/225000 [127:24:17<287:30:15,  6.35s/it]
 28%|██▊       | 61961/225000 [127:24:21<262:00:29,  5.79s/it]
 28%|██▊       | 61962/225000 [127:24:33<343:32:48,  7.59s/it]
 28%|██▊       | 61963/225000 [127:24:40<326:57:16,  7.22s/it]
 28%|██▊       | 61964/225000 [127:24:48<341:13:12,  7.53s/it]
 28%|██▊       | 61965/225000 [127:24:56<355:50:02,  7.86s/it]
 28%|██▊       | 61966/225000 [127:25:01<311:07:48,  6.87s/it]
 28%|██▊       | 61967/225000 [127:25:06<285:26:10,  6.30s/it]
 28%|██▊       | 61968/225000 [127:25:17<346:17:17,  7.65s/it]
 28%|██▊       | 61969/225000 [127:25:20<286:43:03,  6.33s/it]
 28%|██▊       | 61970/225000 [127:25:29<326:45:22,  7.22s/it]
 28%|██▊       | 61971/225000 [127:25:35<313:04:07,  6.91s/it]
 28%|██▊       | 61972/225000 [127:25:39<270:08:05,  5.97s/it]
 28%|██▊       | 61973/225000 [127:25:48<305:32:56,  6.75s/it]
 28%|██▊       | 61974/225000 [127:25:56<319:19:06,  7.05s/it]
 28%|██▊       | 61975/225000 [127:25:59<273:12:11,  6.03s/it]
 28%|██▊       | 61976/225000 [127:26:07<299:33:45,  6.62s/it]
 28%|██▊       | 61977/225000 [127:26:15<318:08:53,  7.03s/it]
 28%|██▊       | 61978/225000 [127:26:21<295:11:16,  6.52s/it]
 28%|██▊       | 61979/225000 [127:26:25<264:09:34,  5.83s/it]
 28%|██▊       | 61980/225000 [127:26:35<326:52:21,  7.22s/it]
 28%|██▊       | 61981/225000 [127:26:40<288:13:43,  6.37s/it]
 28%|██▊       | 61982/225000 [127:26:45<269:25:48,  5.95s/it]
 28%|██▊       | 61983/225000 [127:26:50<259:41:21,  5.73s/it]
 28%|██▊       | 61984/225000 [127:26:55<247:51:04,  5.47s/it]
 28%|██▊       | 61985/225000 [127:27:00<242:19:52,  5.35s/it]
 28%|██▊       | 61986/225000 [127:27:05<246:07:14,  5.44s/it]
 28%|██▊       | 61987/225000 [127:27:18<349:57:43,  7.73s/it]
 28%|██▊       | 61988/225000 [127:27:24<324:43:20,  7.17s/it]
 28%|██▊       | 61989/225000 [127:27:29<292:27:01,  6.46s/it]
 28%|██▊       | 61990/225000 [127:27:34<271:03:26,  5.99s/it]
 28%|██▊       | 61991/225000 [127:27:39<263:10:35,  5.81s/it]
 28%|██▊       | 61992/225000 [127:27:53<368:19:50,  8.13s/it]
 28%|██▊       | 61993/225000 [127:27:58<326:08:30,  7.20s/it]
 28%|██▊       | 61994/225000 [127:28:02<286:32:11,  6.33s/it]
 28%|██▊       | 61995/225000 [127:28:13<350:07:25,  7.73s/it]
 28%|██▊       | 61996/225000 [127:28:17<289:31:34,  6.39s/it]
 28%|██▊       | 61997/225000 [127:28:27<343:39:23,  7.59s/it]
 28%|██▊       | 61998/225000 [127:28:32<313:01:31,  6.91s/it]
 28%|██▊       | 61999/225000 [127:28:37<286:37:51,  6.33s/it]
 28%|██▊       | 62000/225000 [127:28:43<272:49:21,  6.03s/it]
                                                              
{'loss': '0.755', 'grad_norm': '0.8673', 'learning_rate': '4.313e-05', 'epoch': '0.2756', 'num_input_tokens_seen': 212090808, 'train_runtime': '4.589e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62000/225000 [127:28:43<272:49:21,  6.03s/it]
 28%|██▊       | 62001/225000 [127:28:48<262:02:28,  5.79s/it]
 28%|██▊       | 62002/225000 [127:28:53<251:23:13,  5.55s/it]
 28%|██▊       | 62003/225000 [127:28:57<239:56:06,  5.30s/it]
 28%|██▊       | 62004/225000 [127:29:02<235:54:35,  5.21s/it]
 28%|██▊       | 62005/225000 [127:29:15<333:37:29,  7.37s/it]
 28%|██▊       | 62006/225000 [127:29:21<321:49:25,  7.11s/it]
 28%|██▊       | 62007/225000 [127:29:29<326:41:11,  7.22s/it]
 28%|██▊       | 62008/225000 [127:29:36<329:26:49,  7.28s/it]
 28%|██▊       | 62009/225000 [127:29:42<312:23:50,  6.90s/it]
 28%|██▊       | 62010/225000 [127:29:46<266:38:13,  5.89s/it]
 28%|██▊       | 62011/225000 [127:29:50<241:57:25,  5.34s/it]
 28%|██▊       | 62012/225000 [127:30:00<308:39:02,  6.82s/it]
 28%|██▊       | 62013/225000 [127:30:08<319:05:45,  7.05s/it]
 28%|██▊       | 62014/225000 [127:30:12<280:34:00,  6.20s/it]
 28%|██▊       | 62015/225000 [127:30:17<268:46:02,  5.94s/it]
 28%|██▊       | 62016/225000 [127:30:23<269:39:48,  5.96s/it]
 28%|██▊       | 62017/225000 [127:30:29<262:54:55,  5.81s/it]
 28%|██▊       | 62018/225000 [127:30:34<257:37:13,  5.69s/it]
 28%|██▊       | 62019/225000 [127:30:40<258:08:18,  5.70s/it]
 28%|██▊       | 62020/225000 [127:30:48<295:43:41,  6.53s/it]
 28%|██▊       | 62021/225000 [127:31:02<387:12:10,  8.55s/it]
 28%|██▊       | 62022/225000 [127:31:08<356:16:20,  7.87s/it]
 28%|██▊       | 62023/225000 [127:31:12<305:23:16,  6.75s/it]
 28%|██▊       | 62024/225000 [127:31:24<382:25:01,  8.45s/it]
 28%|██▊       | 62025/225000 [127:31:31<360:14:59,  7.96s/it]
 28%|██▊       | 62026/225000 [127:31:35<304:47:11,  6.73s/it]
 28%|██▊       | 62027/225000 [127:31:43<313:20:55,  6.92s/it]
 28%|██▊       | 62028/225000 [127:31:47<277:24:28,  6.13s/it]
 28%|██▊       | 62029/225000 [127:31:53<275:37:20,  6.09s/it]
 28%|██▊       | 62030/225000 [127:32:04<339:45:42,  7.51s/it]
 28%|██▊       | 62031/225000 [127:32:07<286:26:02,  6.33s/it]
 28%|██▊       | 62032/225000 [127:32:14<297:22:16,  6.57s/it]
 28%|██▊       | 62033/225000 [127:32:22<306:29:19,  6.77s/it]
 28%|██▊       | 62034/225000 [127:32:28<305:50:50,  6.76s/it]
 28%|██▊       | 62035/225000 [127:32:36<322:03:09,  7.11s/it]
 28%|██▊       | 62036/225000 [127:32:42<301:06:26,  6.65s/it]
 28%|██▊       | 62037/225000 [127:32:47<280:05:23,  6.19s/it]
 28%|██▊       | 62038/225000 [127:32:52<264:16:06,  5.84s/it]
 28%|██▊       | 62039/225000 [127:32:58<262:17:06,  5.79s/it]
 28%|██▊       | 62040/225000 [127:33:04<265:14:53,  5.86s/it]
 28%|██▊       | 62041/225000 [127:33:17<366:45:12,  8.10s/it]
 28%|██▊       | 62042/225000 [127:33:23<332:15:37,  7.34s/it]
 28%|██▊       | 62043/225000 [127:33:28<313:16:27,  6.92s/it]
 28%|██▊       | 62044/225000 [127:33:34<291:09:56,  6.43s/it]
 28%|██▊       | 62045/225000 [127:33:39<277:19:35,  6.13s/it]
 28%|██▊       | 62046/225000 [127:33:45<268:39:25,  5.94s/it]
 28%|██▊       | 62047/225000 [127:33:51<268:01:47,  5.92s/it]
 28%|██▊       | 62048/225000 [127:34:04<364:11:55,  8.05s/it]
 28%|██▊       | 62049/225000 [127:34:10<337:41:06,  7.46s/it]
 28%|██▊       | 62050/225000 [127:34:15<313:00:00,  6.92s/it]
                                                              
{'loss': '0.7421', 'grad_norm': '0.6457', 'learning_rate': '4.312e-05', 'epoch': '0.2758', 'num_input_tokens_seen': 212238688, 'train_runtime': '4.593e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62050/225000 [127:34:15<313:00:00,  6.92s/it]
 28%|██▊       | 62051/225000 [127:34:21<294:20:16,  6.50s/it]
 28%|██▊       | 62052/225000 [127:34:28<301:44:46,  6.67s/it]
 28%|██▊       | 62053/225000 [127:34:36<327:18:15,  7.23s/it]
 28%|██▊       | 62054/225000 [127:34:42<303:46:10,  6.71s/it]
 28%|██▊       | 62055/225000 [127:34:49<311:22:20,  6.88s/it]
 28%|██▊       | 62056/225000 [127:34:57<318:28:23,  7.04s/it]
 28%|██▊       | 62057/225000 [127:35:01<279:32:49,  6.18s/it]
 28%|██▊       | 62058/225000 [127:35:10<327:54:35,  7.24s/it]
 28%|██▊       | 62059/225000 [127:35:15<290:25:51,  6.42s/it]
 28%|██▊       | 62060/225000 [127:35:19<262:05:42,  5.79s/it]
 28%|██▊       | 62061/225000 [127:35:23<238:35:06,  5.27s/it]
 28%|██▊       | 62062/225000 [127:35:36<332:02:58,  7.34s/it]
 28%|██▊       | 62063/225000 [127:35:40<294:10:57,  6.50s/it]
 28%|██▊       | 62064/225000 [127:35:48<315:57:36,  6.98s/it]
 28%|██▊       | 62065/225000 [127:35:55<316:24:32,  6.99s/it]
 28%|██▊       | 62066/225000 [127:36:01<296:16:55,  6.55s/it]
 28%|██▊       | 62067/225000 [127:36:05<267:36:28,  5.91s/it]
 28%|██▊       | 62068/225000 [127:36:14<304:44:40,  6.73s/it]
 28%|██▊       | 62069/225000 [127:36:19<290:52:13,  6.43s/it]
 28%|██▊       | 62070/225000 [127:36:24<264:23:58,  5.84s/it]
 28%|██▊       | 62071/225000 [127:36:36<342:27:17,  7.57s/it]
 28%|██▊       | 62072/225000 [127:36:42<327:53:14,  7.24s/it]
 28%|██▊       | 62073/225000 [127:36:49<320:50:12,  7.09s/it]
 28%|██▊       | 62074/225000 [127:36:55<311:34:01,  6.88s/it]
 28%|██▊       | 62075/225000 [127:37:04<339:11:41,  7.49s/it]
 28%|██▊       | 62076/225000 [127:37:10<311:56:38,  6.89s/it]
 28%|██▊       | 62077/225000 [127:37:16<300:15:13,  6.63s/it]
 28%|██▊       | 62078/225000 [127:37:20<272:47:42,  6.03s/it]
 28%|██▊       | 62079/225000 [127:37:26<262:32:26,  5.80s/it]
 28%|██▊       | 62080/225000 [127:37:31<255:24:49,  5.64s/it]
 28%|██▊       | 62081/225000 [127:37:36<245:48:18,  5.43s/it]
 28%|██▊       | 62082/225000 [127:37:42<251:28:39,  5.56s/it]
 28%|██▊       | 62083/225000 [127:37:55<357:28:43,  7.90s/it]
 28%|██▊       | 62084/225000 [127:38:00<320:18:18,  7.08s/it]
 28%|██▊       | 62085/225000 [127:38:06<301:40:30,  6.67s/it]
 28%|██▊       | 62086/225000 [127:38:10<269:50:01,  5.96s/it]
 28%|██▊       | 62087/225000 [127:38:21<337:24:00,  7.46s/it]
 28%|██▊       | 62088/225000 [127:38:26<305:05:07,  6.74s/it]
 28%|██▊       | 62089/225000 [127:38:32<290:16:30,  6.41s/it]
 28%|██▊       | 62090/225000 [127:38:41<333:56:46,  7.38s/it]
 28%|██▊       | 62091/225000 [127:38:47<309:29:51,  6.84s/it]
 28%|██▊       | 62092/225000 [127:38:52<290:22:48,  6.42s/it]
 28%|██▊       | 62093/225000 [127:38:57<268:32:11,  5.93s/it]
 28%|██▊       | 62094/225000 [127:39:03<260:01:45,  5.75s/it]
 28%|██▊       | 62095/225000 [127:39:08<259:31:13,  5.74s/it]
 28%|██▊       | 62096/225000 [127:39:14<257:53:08,  5.70s/it]
 28%|██▊       | 62097/225000 [127:39:19<251:32:30,  5.56s/it]
 28%|██▊       | 62098/225000 [127:39:32<353:01:39,  7.80s/it]
 28%|██▊       | 62099/225000 [127:39:37<307:14:20,  6.79s/it]
 28%|██▊       | 62100/225000 [127:39:46<336:43:58,  7.44s/it]
                                                              
{'loss': '0.7522', 'grad_norm': '0.8975', 'learning_rate': '4.311e-05', 'epoch': '0.276', 'num_input_tokens_seen': 212385816, 'train_runtime': '4.596e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62100/225000 [127:39:46<336:43:58,  7.44s/it]
 28%|██▊       | 62101/225000 [127:39:52<317:26:20,  7.02s/it]
 28%|██▊       | 62102/225000 [127:39:57<301:39:16,  6.67s/it]
 28%|██▊       | 62103/225000 [127:40:05<319:40:04,  7.06s/it]
 28%|██▊       | 62104/225000 [127:40:12<317:30:04,  7.02s/it]
 28%|██▊       | 62105/225000 [127:40:19<317:15:26,  7.01s/it]
 28%|██▊       | 62106/225000 [127:40:23<275:02:58,  6.08s/it]
 28%|██▊       | 62107/225000 [127:40:32<312:54:26,  6.92s/it]
 28%|██▊       | 62108/225000 [127:40:37<282:28:36,  6.24s/it]
 28%|██▊       | 62109/225000 [127:40:41<261:41:46,  5.78s/it]
 28%|██▊       | 62110/225000 [127:40:46<243:58:43,  5.39s/it]
 28%|██▊       | 62111/225000 [127:40:52<247:10:03,  5.46s/it]
 28%|██▊       | 62112/225000 [127:41:05<355:11:54,  7.85s/it]
 28%|██▊       | 62113/225000 [127:41:09<298:52:21,  6.61s/it]
 28%|██▊       | 62114/225000 [127:41:14<282:13:41,  6.24s/it]
 28%|██▊       | 62115/225000 [127:41:19<264:52:05,  5.85s/it]
 28%|██▊       | 62116/225000 [127:41:25<262:28:45,  5.80s/it]
 28%|██▊       | 62117/225000 [127:41:30<258:55:30,  5.72s/it]
 28%|██▊       | 62118/225000 [127:41:36<257:09:43,  5.68s/it]
 28%|██▊       | 62119/225000 [127:41:44<286:20:07,  6.33s/it]
 28%|██▊       | 62120/225000 [127:41:56<372:51:27,  8.24s/it]
 28%|██▊       | 62121/225000 [127:42:02<331:24:50,  7.33s/it]
 28%|██▊       | 62122/225000 [127:42:08<315:47:45,  6.98s/it]
 28%|██▊       | 62123/225000 [127:42:12<283:34:27,  6.27s/it]
 28%|██▊       | 62124/225000 [127:42:17<258:28:26,  5.71s/it]
 28%|██▊       | 62125/225000 [127:42:26<302:56:21,  6.70s/it]
 28%|██▊       | 62126/225000 [127:42:36<348:10:38,  7.70s/it]
 28%|██▊       | 62127/225000 [127:42:40<303:31:07,  6.71s/it]
 28%|██▊       | 62128/225000 [127:42:44<269:58:11,  5.97s/it]
 28%|██▊       | 62129/225000 [127:42:52<289:37:20,  6.40s/it]
 28%|██▊       | 62130/225000 [127:43:00<315:50:00,  6.98s/it]
 28%|██▊       | 62131/225000 [127:43:04<276:32:01,  6.11s/it]
 28%|██▊       | 62132/225000 [127:43:13<309:28:02,  6.84s/it]
 28%|██▊       | 62133/225000 [127:43:17<275:26:04,  6.09s/it]
 28%|██▊       | 62134/225000 [127:43:26<317:50:03,  7.03s/it]
 28%|██▊       | 62135/225000 [127:43:32<303:36:31,  6.71s/it]
 28%|██▊       | 62136/225000 [127:43:38<292:10:17,  6.46s/it]
 28%|██▊       | 62137/225000 [127:43:46<306:55:51,  6.78s/it]
 28%|██▊       | 62138/225000 [127:43:50<266:53:05,  5.90s/it]
 28%|██▊       | 62139/225000 [127:44:00<333:39:23,  7.38s/it]
 28%|██▊       | 62140/225000 [127:44:04<287:05:33,  6.35s/it]
 28%|██▊       | 62141/225000 [127:44:11<285:28:26,  6.31s/it]
 28%|██▊       | 62142/225000 [127:44:19<317:14:33,  7.01s/it]
 28%|██▊       | 62143/225000 [127:44:24<288:09:30,  6.37s/it]
 28%|██▊       | 62144/225000 [127:44:32<314:17:15,  6.95s/it]
 28%|██▊       | 62145/225000 [127:44:41<342:22:37,  7.57s/it]
 28%|██▊       | 62146/225000 [127:44:48<326:22:09,  7.21s/it]
 28%|██▊       | 62147/225000 [127:44:51<275:29:20,  6.09s/it]
 28%|██▊       | 62148/225000 [127:45:01<320:05:45,  7.08s/it]
 28%|██▊       | 62149/225000 [127:45:05<285:00:08,  6.30s/it]
 28%|██▊       | 62150/225000 [127:45:10<269:13:09,  5.95s/it]
                                                              
{'loss': '0.7255', 'grad_norm': '0.7499', 'learning_rate': '4.309e-05', 'epoch': '0.2762', 'num_input_tokens_seen': 212532280, 'train_runtime': '4.599e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62150/225000 [127:45:10<269:13:09,  5.95s/it]
 28%|██▊       | 62151/225000 [127:45:18<288:44:43,  6.38s/it]
 28%|██▊       | 62152/225000 [127:45:26<316:52:31,  7.01s/it]
 28%|██▊       | 62153/225000 [127:45:34<331:28:32,  7.33s/it]
 28%|██▊       | 62154/225000 [127:45:38<281:53:18,  6.23s/it]
 28%|██▊       | 62155/225000 [127:45:46<313:16:21,  6.93s/it]
 28%|██▊       | 62156/225000 [127:45:52<292:43:40,  6.47s/it]
 28%|██▊       | 62157/225000 [127:45:57<270:01:38,  5.97s/it]
 28%|██▊       | 62158/225000 [127:46:02<256:22:36,  5.67s/it]
 28%|██▊       | 62159/225000 [127:46:07<249:29:46,  5.52s/it]
 28%|██▊       | 62160/225000 [127:46:20<359:29:14,  7.95s/it]
 28%|██▊       | 62161/225000 [127:46:25<319:21:24,  7.06s/it]
 28%|██▊       | 62162/225000 [127:46:30<288:42:33,  6.38s/it]
 28%|██▊       | 62163/225000 [127:46:35<265:47:25,  5.88s/it]
 28%|██▊       | 62164/225000 [127:46:40<257:30:16,  5.69s/it]
 28%|██▊       | 62165/225000 [127:46:53<358:30:15,  7.93s/it]
 28%|██▊       | 62166/225000 [127:47:03<378:39:36,  8.37s/it]
 28%|██▊       | 62167/225000 [127:47:08<339:04:18,  7.50s/it]
 28%|██▊       | 62168/225000 [127:47:14<314:02:20,  6.94s/it]
 28%|██▊       | 62169/225000 [127:47:23<340:15:54,  7.52s/it]
 28%|██▊       | 62170/225000 [127:47:28<315:33:46,  6.98s/it]
 28%|██▊       | 62171/225000 [127:47:33<284:56:59,  6.30s/it]
 28%|██▊       | 62172/225000 [127:47:39<274:22:58,  6.07s/it]
 28%|██▊       | 62173/225000 [127:47:44<261:10:23,  5.77s/it]
 28%|██▊       | 62174/225000 [127:47:49<248:30:44,  5.49s/it]
 28%|██▊       | 62175/225000 [127:47:59<314:14:41,  6.95s/it]
 28%|██▊       | 62176/225000 [127:48:07<331:41:34,  7.33s/it]
 28%|██▊       | 62177/225000 [127:48:12<297:13:33,  6.57s/it]
 28%|██▊       | 62178/225000 [127:48:16<266:51:59,  5.90s/it]
 28%|██▊       | 62179/225000 [127:48:26<319:16:41,  7.06s/it]
 28%|██▊       | 62180/225000 [127:48:33<320:45:47,  7.09s/it]
 28%|██▊       | 62181/225000 [127:48:41<330:58:29,  7.32s/it]
 28%|██▊       | 62182/225000 [127:48:44<279:04:09,  6.17s/it]
 28%|██▊       | 62183/225000 [127:48:53<307:09:49,  6.79s/it]
 28%|██▊       | 62184/225000 [127:48:58<291:45:32,  6.45s/it]
 28%|██▊       | 62185/225000 [127:49:05<292:18:08,  6.46s/it]
 28%|██▊       | 62186/225000 [127:49:10<273:41:43,  6.05s/it]
 28%|██▊       | 62187/225000 [127:49:15<258:45:37,  5.72s/it]
 28%|██▊       | 62188/225000 [127:49:22<270:29:46,  5.98s/it]
 28%|██▊       | 62189/225000 [127:49:36<382:07:06,  8.45s/it]
 28%|██▊       | 62190/225000 [127:49:40<329:42:43,  7.29s/it]
 28%|██▊       | 62191/225000 [127:49:48<340:37:12,  7.53s/it]
 28%|██▊       | 62192/225000 [127:49:54<316:39:49,  7.00s/it]
 28%|██▊       | 62193/225000 [127:50:01<310:46:35,  6.87s/it]
 28%|██▊       | 62194/225000 [127:50:05<271:53:17,  6.01s/it]
 28%|██▊       | 62195/225000 [127:50:15<333:36:23,  7.38s/it]
 28%|██▊       | 62196/225000 [127:50:20<294:32:03,  6.51s/it]
 28%|██▊       | 62197/225000 [127:50:27<301:37:05,  6.67s/it]
 28%|██▊       | 62198/225000 [127:50:34<306:05:45,  6.77s/it]
 28%|██▊       | 62199/225000 [127:50:39<290:39:29,  6.43s/it]
 28%|██▊       | 62200/225000 [127:50:46<295:59:04,  6.55s/it]
                                                              
{'loss': '0.7517', 'grad_norm': '0.7624', 'learning_rate': '4.308e-05', 'epoch': '0.2764', 'num_input_tokens_seen': 212680808, 'train_runtime': '4.602e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62200/225000 [127:50:46<295:59:04,  6.55s/it]
 28%|██▊       | 62201/225000 [127:50:52<283:08:05,  6.26s/it]
 28%|██▊       | 62202/225000 [127:51:02<336:16:09,  7.44s/it]
 28%|██▊       | 62203/225000 [127:51:07<300:54:58,  6.65s/it]
 28%|██▊       | 62204/225000 [127:51:14<306:12:42,  6.77s/it]
 28%|██▊       | 62205/225000 [127:51:22<317:29:57,  7.02s/it]
 28%|██▊       | 62206/225000 [127:51:27<292:37:23,  6.47s/it]
 28%|██▊       | 62207/225000 [127:51:34<304:13:16,  6.73s/it]
 28%|██▊       | 62208/225000 [127:51:39<275:08:30,  6.08s/it]
 28%|██▊       | 62209/225000 [127:51:43<251:53:56,  5.57s/it]
 28%|██▊       | 62210/225000 [127:51:53<315:09:55,  6.97s/it]
 28%|██▊       | 62211/225000 [127:51:58<279:24:37,  6.18s/it]
 28%|██▊       | 62212/225000 [127:52:08<331:59:10,  7.34s/it]
 28%|██▊       | 62213/225000 [127:52:12<286:29:36,  6.34s/it]
 28%|██▊       | 62214/225000 [127:52:17<276:56:09,  6.12s/it]
 28%|██▊       | 62215/225000 [127:52:23<267:11:22,  5.91s/it]
 28%|██▊       | 62216/225000 [127:52:28<259:17:06,  5.73s/it]
 28%|██▊       | 62217/225000 [127:52:34<261:50:15,  5.79s/it]
 28%|██▊       | 62218/225000 [127:52:39<256:32:24,  5.67s/it]
 28%|██▊       | 62219/225000 [127:52:45<262:50:12,  5.81s/it]
 28%|██▊       | 62220/225000 [127:52:51<258:58:02,  5.73s/it]
 28%|██▊       | 62221/225000 [127:52:56<255:19:16,  5.65s/it]
 28%|██▊       | 62222/225000 [127:53:02<251:29:19,  5.56s/it]
 28%|██▊       | 62223/225000 [127:53:08<254:49:38,  5.64s/it]
 28%|██▊       | 62224/225000 [127:53:19<330:57:35,  7.32s/it]
 28%|██▊       | 62225/225000 [127:53:28<351:04:57,  7.76s/it]
 28%|██▊       | 62226/225000 [127:53:32<308:28:40,  6.82s/it]
 28%|██▊       | 62227/225000 [127:53:37<275:43:50,  6.10s/it]
 28%|██▊       | 62228/225000 [127:53:44<290:01:31,  6.41s/it]
 28%|██▊       | 62229/225000 [127:53:55<353:57:48,  7.83s/it]
 28%|██▊       | 62230/225000 [127:54:00<314:54:47,  6.96s/it]
 28%|██▊       | 62231/225000 [127:54:05<287:46:59,  6.36s/it]
 28%|██▊       | 62232/225000 [127:54:10<267:55:25,  5.93s/it]
 28%|██▊       | 62233/225000 [127:54:15<253:06:08,  5.60s/it]
 28%|██▊       | 62234/225000 [127:54:20<246:27:14,  5.45s/it]
 28%|██▊       | 62235/225000 [127:54:25<250:30:54,  5.54s/it]
 28%|██▊       | 62236/225000 [127:54:39<359:10:18,  7.94s/it]
 28%|██▊       | 62237/225000 [127:54:45<335:13:54,  7.41s/it]
 28%|██▊       | 62238/225000 [127:54:53<342:13:30,  7.57s/it]
 28%|██▊       | 62239/225000 [127:54:58<307:43:50,  6.81s/it]
 28%|██▊       | 62240/225000 [127:55:08<344:18:34,  7.62s/it]
 28%|██▊       | 62241/225000 [127:55:12<296:05:17,  6.55s/it]
 28%|██▊       | 62242/225000 [127:55:16<269:45:06,  5.97s/it]
 28%|██▊       | 62243/225000 [127:55:28<350:38:29,  7.76s/it]
 28%|██▊       | 62244/225000 [127:55:32<290:47:59,  6.43s/it]
 28%|██▊       | 62245/225000 [127:55:37<270:54:34,  5.99s/it]
 28%|██▊       | 62246/225000 [127:55:42<257:48:17,  5.70s/it]
 28%|██▊       | 62247/225000 [127:55:47<251:43:43,  5.57s/it]
 28%|██▊       | 62248/225000 [127:55:56<298:00:37,  6.59s/it]
 28%|██▊       | 62249/225000 [127:56:05<337:13:51,  7.46s/it]
 28%|██▊       | 62250/225000 [127:56:13<335:21:20,  7.42s/it]
                                                              
{'loss': '0.7525', 'grad_norm': '0.7098', 'learning_rate': '4.307e-05', 'epoch': '0.2767', 'num_input_tokens_seen': 212826104, 'train_runtime': '4.606e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62250/225000 [127:56:13<335:21:20,  7.42s/it]
 28%|██▊       | 62251/225000 [127:56:17<300:27:43,  6.65s/it]
 28%|██▊       | 62252/225000 [127:56:23<282:00:03,  6.24s/it]
 28%|██▊       | 62253/225000 [127:56:28<266:12:46,  5.89s/it]
 28%|██▊       | 62254/225000 [127:56:33<254:01:32,  5.62s/it]
 28%|██▊       | 62255/225000 [127:56:44<331:22:50,  7.33s/it]
 28%|██▊       | 62256/225000 [127:56:52<339:43:53,  7.52s/it]
 28%|██▊       | 62257/225000 [127:56:58<317:29:29,  7.02s/it]
 28%|██▊       | 62258/225000 [127:57:04<308:20:54,  6.82s/it]
 28%|██▊       | 62259/225000 [127:57:08<268:40:43,  5.94s/it]
 28%|██▊       | 62260/225000 [127:57:13<252:04:25,  5.58s/it]
 28%|██▊       | 62261/225000 [127:57:18<242:24:43,  5.36s/it]
 28%|██▊       | 62262/225000 [127:57:29<324:20:47,  7.18s/it]
 28%|██▊       | 62263/225000 [127:57:37<333:01:21,  7.37s/it]
 28%|██▊       | 62264/225000 [127:57:41<289:07:41,  6.40s/it]
 28%|██▊       | 62265/225000 [127:57:48<299:35:18,  6.63s/it]
 28%|██▊       | 62266/225000 [127:57:58<346:53:25,  7.67s/it]
 28%|██▊       | 62267/225000 [127:58:03<299:43:46,  6.63s/it]
 28%|██▊       | 62268/225000 [127:58:09<296:59:43,  6.57s/it]
 28%|██▊       | 62269/225000 [127:58:19<338:06:31,  7.48s/it]
 28%|██▊       | 62270/225000 [127:58:23<288:58:17,  6.39s/it]
 28%|██▊       | 62271/225000 [127:58:31<320:03:59,  7.08s/it]
 28%|██▊       | 62272/225000 [127:58:35<272:19:58,  6.02s/it]
 28%|██▊       | 62273/225000 [127:58:43<296:40:21,  6.56s/it]
 28%|██▊       | 62274/225000 [127:58:49<289:37:25,  6.41s/it]
 28%|██▊       | 62275/225000 [127:58:56<303:51:18,  6.72s/it]
 28%|██▊       | 62276/225000 [127:59:03<299:45:25,  6.63s/it]
 28%|██▊       | 62277/225000 [127:59:11<327:48:15,  7.25s/it]
 28%|██▊       | 62278/225000 [127:59:16<291:14:30,  6.44s/it]
 28%|██▊       | 62279/225000 [127:59:20<265:43:08,  5.88s/it]
 28%|██▊       | 62280/225000 [127:59:29<306:20:34,  6.78s/it]
 28%|██▊       | 62281/225000 [127:59:36<302:59:50,  6.70s/it]
 28%|██▊       | 62282/225000 [127:59:40<272:48:52,  6.04s/it]
 28%|██▊       | 62283/225000 [127:59:47<280:02:35,  6.20s/it]
 28%|██▊       | 62284/225000 [127:59:57<330:41:06,  7.32s/it]
 28%|██▊       | 62285/225000 [128:00:03<313:39:58,  6.94s/it]
 28%|██▊       | 62286/225000 [128:00:09<301:03:55,  6.66s/it]
 28%|██▊       | 62287/225000 [128:00:14<285:29:08,  6.32s/it]
 28%|██▊       | 62288/225000 [128:00:20<272:53:29,  6.04s/it]
 28%|██▊       | 62289/225000 [128:00:30<327:08:45,  7.24s/it]
 28%|██▊       | 62290/225000 [128:00:35<303:00:00,  6.70s/it]
 28%|██▊       | 62291/225000 [128:00:41<285:29:14,  6.32s/it]
 28%|██▊       | 62292/225000 [128:00:46<273:07:00,  6.04s/it]
 28%|██▊       | 62293/225000 [128:00:51<260:28:03,  5.76s/it]
 28%|██▊       | 62294/225000 [128:01:02<335:55:48,  7.43s/it]
 28%|██▊       | 62295/225000 [128:01:10<336:57:47,  7.46s/it]
 28%|██▊       | 62296/225000 [128:01:14<297:18:15,  6.58s/it]
 28%|██▊       | 62297/225000 [128:01:20<284:00:06,  6.28s/it]
 28%|██▊       | 62298/225000 [128:01:25<263:40:29,  5.83s/it]
 28%|██▊       | 62299/225000 [128:01:33<294:37:58,  6.52s/it]
 28%|██▊       | 62300/225000 [128:01:44<359:10:23,  7.95s/it]
                                                              
{'loss': '0.7541', 'grad_norm': '0.7423', 'learning_rate': '4.306e-05', 'epoch': '0.2769', 'num_input_tokens_seen': 212973696, 'train_runtime': '4.609e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62300/225000 [128:01:44<359:10:23,  7.95s/it]
 28%|██▊       | 62301/225000 [128:01:51<345:18:59,  7.64s/it]
 28%|██▊       | 62302/225000 [128:01:57<314:27:11,  6.96s/it]
 28%|██▊       | 62303/225000 [128:02:03<306:18:47,  6.78s/it]
 28%|██▊       | 62304/225000 [128:02:11<321:50:17,  7.12s/it]
 28%|██▊       | 62305/225000 [128:02:18<318:27:07,  7.05s/it]
 28%|██▊       | 62306/225000 [128:02:23<295:09:54,  6.53s/it]
 28%|██▊       | 62307/225000 [128:02:27<262:09:14,  5.80s/it]
 28%|██▊       | 62308/225000 [128:02:32<251:11:16,  5.56s/it]
 28%|██▊       | 62309/225000 [128:02:44<336:32:57,  7.45s/it]
 28%|██▊       | 62310/225000 [128:02:50<321:14:58,  7.11s/it]
 28%|██▊       | 62311/225000 [128:02:56<309:11:27,  6.84s/it]
 28%|██▊       | 62312/225000 [128:03:03<306:35:58,  6.78s/it]
 28%|██▊       | 62313/225000 [128:03:09<294:10:01,  6.51s/it]
 28%|██▊       | 62314/225000 [128:03:14<278:06:02,  6.15s/it]
 28%|██▊       | 62315/225000 [128:03:20<273:37:59,  6.06s/it]
 28%|██▊       | 62316/225000 [128:03:26<268:00:35,  5.93s/it]
 28%|██▊       | 62317/225000 [128:03:35<313:01:12,  6.93s/it]
 28%|██▊       | 62318/225000 [128:03:41<295:53:46,  6.55s/it]
 28%|██▊       | 62319/225000 [128:03:49<315:09:19,  6.97s/it]
 28%|██▊       | 62320/225000 [128:03:54<297:09:13,  6.58s/it]
 28%|██▊       | 62321/225000 [128:04:03<324:11:29,  7.17s/it]
 28%|██▊       | 62322/225000 [128:04:07<279:01:42,  6.17s/it]
 28%|██▊       | 62323/225000 [128:04:16<318:06:48,  7.04s/it]
 28%|██▊       | 62324/225000 [128:04:20<274:18:06,  6.07s/it]
 28%|██▊       | 62325/225000 [128:04:27<290:14:31,  6.42s/it]
 28%|██▊       | 62326/225000 [128:04:33<291:28:47,  6.45s/it]
 28%|██▊       | 62327/225000 [128:04:40<295:51:08,  6.55s/it]
 28%|██▊       | 62328/225000 [128:04:47<301:28:30,  6.67s/it]
 28%|██▊       | 62329/225000 [128:04:55<315:27:00,  6.98s/it]
 28%|██▊       | 62330/225000 [128:04:59<271:54:45,  6.02s/it]
 28%|██▊       | 62331/225000 [128:05:09<327:55:37,  7.26s/it]
 28%|██▊       | 62332/225000 [128:05:15<311:07:43,  6.89s/it]
 28%|██▊       | 62333/225000 [128:05:21<299:33:09,  6.63s/it]
 28%|██▊       | 62334/225000 [128:05:27<300:04:36,  6.64s/it]
 28%|██▊       | 62335/225000 [128:05:33<279:32:37,  6.19s/it]
 28%|██▊       | 62336/225000 [128:05:38<266:53:26,  5.91s/it]
 28%|██▊       | 62337/225000 [128:05:43<254:46:57,  5.64s/it]
 28%|██▊       | 62338/225000 [128:05:48<245:39:01,  5.44s/it]
 28%|██▊       | 62339/225000 [128:05:53<246:14:16,  5.45s/it]
 28%|██▊       | 62340/225000 [128:06:07<353:59:02,  7.83s/it]
 28%|██▊       | 62341/225000 [128:06:15<360:27:13,  7.98s/it]
 28%|██▊       | 62342/225000 [128:06:20<322:01:30,  7.13s/it]
 28%|██▊       | 62343/225000 [128:06:25<293:10:23,  6.49s/it]
 28%|██▊       | 62344/225000 [128:06:30<275:16:47,  6.09s/it]
 28%|██▊       | 62345/225000 [128:06:36<272:11:42,  6.02s/it]
 28%|██▊       | 62346/225000 [128:06:50<373:39:13,  8.27s/it]
 28%|██▊       | 62347/225000 [128:06:54<320:37:42,  7.10s/it]
 28%|██▊       | 62348/225000 [128:07:03<342:24:34,  7.58s/it]
 28%|██▊       | 62349/225000 [128:07:08<314:53:42,  6.97s/it]
 28%|██▊       | 62350/225000 [128:07:13<280:18:54,  6.20s/it]
                                                              
{'loss': '0.7538', 'grad_norm': '0.6756', 'learning_rate': '4.304e-05', 'epoch': '0.2771', 'num_input_tokens_seen': 213121000, 'train_runtime': '4.612e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62350/225000 [128:07:13<280:18:54,  6.20s/it]
 28%|██▊       | 62351/225000 [128:07:20<296:03:30,  6.55s/it]
 28%|██▊       | 62352/225000 [128:07:27<304:22:20,  6.74s/it]
 28%|██▊       | 62353/225000 [128:07:33<290:10:58,  6.42s/it]
 28%|██▊       | 62354/225000 [128:07:41<310:22:33,  6.87s/it]
 28%|██▊       | 62355/225000 [128:07:47<297:22:06,  6.58s/it]
 28%|██▊       | 62356/225000 [128:07:53<289:32:08,  6.41s/it]
 28%|██▊       | 62357/225000 [128:07:58<273:31:27,  6.05s/it]
 28%|██▊       | 62358/225000 [128:08:05<292:33:52,  6.48s/it]
 28%|██▊       | 62359/225000 [128:08:15<329:05:56,  7.28s/it]
 28%|██▊       | 62360/225000 [128:08:19<293:25:02,  6.49s/it]
 28%|██▊       | 62361/225000 [128:08:25<282:17:14,  6.25s/it]
 28%|██▊       | 62362/225000 [128:08:35<329:04:17,  7.28s/it]
 28%|██▊       | 62363/225000 [128:08:39<283:05:46,  6.27s/it]
 28%|██▊       | 62364/225000 [128:08:43<256:25:20,  5.68s/it]
 28%|██▊       | 62365/225000 [128:08:54<328:28:03,  7.27s/it]
 28%|██▊       | 62366/225000 [128:08:59<304:16:09,  6.74s/it]
 28%|██▊       | 62367/225000 [128:09:05<293:46:41,  6.50s/it]
 28%|██▊       | 62368/225000 [128:09:09<259:09:26,  5.74s/it]
 28%|██▊       | 62369/225000 [128:09:16<274:49:07,  6.08s/it]
 28%|██▊       | 62370/225000 [128:09:24<292:27:17,  6.47s/it]
 28%|██▊       | 62371/225000 [128:09:28<272:09:36,  6.02s/it]
 28%|██▊       | 62372/225000 [128:09:33<249:35:01,  5.52s/it]
 28%|██▊       | 62373/225000 [128:09:46<350:53:30,  7.77s/it]
 28%|██▊       | 62374/225000 [128:09:49<292:59:19,  6.49s/it]
 28%|██▊       | 62375/225000 [128:09:54<269:16:54,  5.96s/it]
 28%|██▊       | 62376/225000 [128:10:00<270:11:58,  5.98s/it]
 28%|██▊       | 62377/225000 [128:10:11<340:00:46,  7.53s/it]
 28%|██▊       | 62378/225000 [128:10:15<292:12:09,  6.47s/it]
 28%|██▊       | 62379/225000 [128:10:24<317:21:30,  7.03s/it]
 28%|██▊       | 62380/225000 [128:10:29<295:14:04,  6.54s/it]
 28%|██▊       | 62381/225000 [128:10:36<303:43:36,  6.72s/it]
 28%|██▊       | 62382/225000 [128:10:44<313:57:02,  6.95s/it]
 28%|██▊       | 62383/225000 [128:10:48<278:38:36,  6.17s/it]
 28%|██▊       | 62384/225000 [128:10:53<257:35:33,  5.70s/it]
 28%|██▊       | 62385/225000 [128:11:05<346:19:47,  7.67s/it]
 28%|██▊       | 62386/225000 [128:11:08<290:09:41,  6.42s/it]
 28%|██▊       | 62387/225000 [128:11:13<269:49:02,  5.97s/it]
 28%|██▊       | 62388/225000 [128:11:25<343:52:09,  7.61s/it]
 28%|██▊       | 62389/225000 [128:11:30<316:22:15,  7.00s/it]
 28%|██▊       | 62390/225000 [128:11:36<300:42:39,  6.66s/it]
 28%|██▊       | 62391/225000 [128:11:43<297:56:30,  6.60s/it]
 28%|██▊       | 62392/225000 [128:11:47<269:13:17,  5.96s/it]
 28%|██▊       | 62393/225000 [128:11:56<315:42:39,  6.99s/it]
 28%|██▊       | 62394/225000 [128:12:02<292:35:33,  6.48s/it]
 28%|██▊       | 62395/225000 [128:12:07<274:08:33,  6.07s/it]
 28%|██▊       | 62396/225000 [128:12:12<266:52:51,  5.91s/it]
 28%|██▊       | 62397/225000 [128:12:18<264:10:17,  5.85s/it]
 28%|██▊       | 62398/225000 [128:12:24<260:08:08,  5.76s/it]
 28%|██▊       | 62399/225000 [128:12:29<255:06:26,  5.65s/it]
 28%|██▊       | 62400/225000 [128:12:41<347:12:19,  7.69s/it]
                                                              
{'loss': '0.7429', 'grad_norm': '0.7845', 'learning_rate': '4.303e-05', 'epoch': '0.2773', 'num_input_tokens_seen': 213267760, 'train_runtime': '4.616e+05', 'train_tokens_per_second': '462.1'}

 28%|██▊       | 62400/225000 [128:12:41<347:12:19,  7.69s/it]
 28%|██▊       | 62401/225000 [128:12:48<333:15:35,  7.38s/it]
 28%|██▊       | 62402/225000 [128:12:56<335:42:55,  7.43s/it]
 28%|██▊       | 62403/225000 [128:13:03<331:47:15,  7.35s/it]
 28%|██▊       | 62404/225000 [128:13:09<314:20:07,  6.96s/it]
 28%|██▊       | 62405/225000 [128:13:16<314:54:49,  6.97s/it]
 28%|██▊       | 62406/225000 [128:13:20<278:28:18,  6.17s/it]
 28%|██▊       | 62407/225000 [128:13:26<270:13:16,  5.98s/it]
 28%|██▊       | 62408/225000 [128:13:39<363:18:16,  8.04s/it]
 28%|██▊       | 62409/225000 [128:13:43<313:50:43,  6.95s/it]
 28%|██▊       | 62410/225000 [128:13:50<317:22:47,  7.03s/it]
 28%|██▊       | 62411/225000 [128:13:55<284:40:54,  6.30s/it]
 28%|██▊       | 62412/225000 [128:14:00<273:12:54,  6.05s/it]
 28%|██▊       | 62413/225000 [128:14:09<313:04:12,  6.93s/it]
 28%|██▊       | 62414/225000 [128:14:19<350:17:18,  7.76s/it]
 28%|██▊       | 62415/225000 [128:14:24<309:29:26,  6.85s/it]
 28%|██▊       | 62416/225000 [128:14:29<290:23:35,  6.43s/it]
 28%|██▊       | 62417/225000 [128:14:37<313:10:49,  6.93s/it]
 28%|██▊       | 62418/225000 [128:14:41<273:28:05,  6.06s/it]
 28%|██▊       | 62419/225000 [128:14:52<332:14:48,  7.36s/it]
 28%|██▊       | 62420/225000 [128:14:59<333:13:40,  7.38s/it]
 28%|██▊       | 62421/225000 [128:15:05<319:48:46,  7.08s/it]
 28%|██▊       | 62422/225000 [128:15:10<287:06:13,  6.36s/it]
 28%|██▊       | 62423/225000 [128:15:15<271:38:15,  6.01s/it]
 28%|██▊       | 62424/225000 [128:15:24<309:47:40,  6.86s/it]
 28%|██▊       | 62425/225000 [128:15:28<267:07:23,  5.92s/it]
 28%|██▊       | 62426/225000 [128:15:37<311:04:08,  6.89s/it]
 28%|██▊       | 62427/225000 [128:15:41<270:23:16,  5.99s/it]
 28%|██▊       | 62428/225000 [128:15:47<276:27:19,  6.12s/it]
 28%|██▊       | 62429/225000 [128:15:55<302:15:19,  6.69s/it]
 28%|██▊       | 62430/225000 [128:16:02<299:11:29,  6.63s/it]
 28%|██▊       | 62431/225000 [128:16:06<265:23:38,  5.88s/it]
 28%|██▊       | 62432/225000 [128:16:16<323:24:26,  7.16s/it]
 28%|██▊       | 62433/225000 [128:16:22<301:04:22,  6.67s/it]
 28%|██▊       | 62434/225000 [128:16:27<284:32:46,  6.30s/it]
 28%|██▊       | 62435/225000 [128:16:36<323:53:46,  7.17s/it]
 28%|██▊       | 62436/225000 [128:16:41<285:36:41,  6.32s/it]
 28%|██▊       | 62437/225000 [128:16:51<336:09:07,  7.44s/it]
 28%|██▊       | 62438/225000 [128:16:57<314:22:39,  6.96s/it]
 28%|██▊       | 62439/225000 [128:17:03<309:18:43,  6.85s/it]
 28%|██▊       | 62440/225000 [128:17:10<312:30:05,  6.92s/it]
 28%|██▊       | 62441/225000 [128:17:16<298:32:13,  6.61s/it]
 28%|██▊       | 62442/225000 [128:17:21<272:48:43,  6.04s/it]
 28%|██▊       | 62443/225000 [128:17:26<258:08:30,  5.72s/it]
 28%|██▊       | 62444/225000 [128:17:31<250:41:04,  5.55s/it]
 28%|██▊       | 62445/225000 [128:17:36<248:18:27,  5.50s/it]
 28%|██▊       | 62446/225000 [128:17:49<350:37:21,  7.77s/it]
 28%|██▊       | 62447/225000 [128:17:54<304:34:51,  6.75s/it]
 28%|██▊       | 62448/225000 [128:18:02<325:55:30,  7.22s/it]
 28%|██▊       | 62449/225000 [128:18:09<323:02:51,  7.15s/it]
 28%|██▊       | 62450/225000 [128:18:14<290:53:36,  6.44s/it]
                                                              
{'loss': '0.7469', 'grad_norm': '0.7664', 'learning_rate': '4.302e-05', 'epoch': '0.2776', 'num_input_tokens_seen': 213416672, 'train_runtime': '4.619e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62450/225000 [128:18:14<290:53:36,  6.44s/it]
 28%|██▊       | 62451/225000 [128:18:22<310:23:20,  6.87s/it]
 28%|██▊       | 62452/225000 [128:18:27<287:49:05,  6.37s/it]
 28%|██▊       | 62453/225000 [128:18:32<265:46:18,  5.89s/it]
 28%|██▊       | 62454/225000 [128:18:38<276:00:26,  6.11s/it]
 28%|██▊       | 62455/225000 [128:18:50<346:37:12,  7.68s/it]
 28%|██▊       | 62456/225000 [128:18:54<301:44:45,  6.68s/it]
 28%|██▊       | 62457/225000 [128:18:58<269:54:04,  5.98s/it]
 28%|██▊       | 62458/225000 [128:19:05<280:25:00,  6.21s/it]
 28%|██▊       | 62459/225000 [128:19:16<338:40:01,  7.50s/it]
 28%|██▊       | 62460/225000 [128:19:23<336:12:13,  7.45s/it]
 28%|██▊       | 62461/225000 [128:19:27<290:06:18,  6.43s/it]
 28%|██▊       | 62462/225000 [128:19:34<303:34:04,  6.72s/it]
 28%|██▊       | 62463/225000 [128:19:38<265:35:27,  5.88s/it]
 28%|██▊       | 62464/225000 [128:19:44<269:06:52,  5.96s/it]
 28%|██▊       | 62465/225000 [128:19:54<323:11:20,  7.16s/it]
 28%|██▊       | 62466/225000 [128:19:58<276:05:28,  6.12s/it]
 28%|██▊       | 62467/225000 [128:20:03<255:56:59,  5.67s/it]
 28%|██▊       | 62468/225000 [128:20:12<309:18:21,  6.85s/it]
 28%|██▊       | 62469/225000 [128:20:20<316:53:05,  7.02s/it]
 28%|██▊       | 62470/225000 [128:20:25<296:12:01,  6.56s/it]
 28%|██▊       | 62471/225000 [128:20:30<271:24:11,  6.01s/it]
 28%|██▊       | 62472/225000 [128:20:35<259:56:08,  5.76s/it]
 28%|██▊       | 62473/225000 [128:20:41<258:00:08,  5.71s/it]
 28%|██▊       | 62474/225000 [128:20:46<249:14:26,  5.52s/it]
 28%|██▊       | 62475/225000 [128:21:00<364:02:53,  8.06s/it]
 28%|██▊       | 62476/225000 [128:21:04<311:43:09,  6.90s/it]
 28%|██▊       | 62477/225000 [128:21:12<320:39:36,  7.10s/it]
 28%|██▊       | 62478/225000 [128:21:16<279:21:10,  6.19s/it]
 28%|██▊       | 62479/225000 [128:21:27<345:07:43,  7.64s/it]
 28%|██▊       | 62480/225000 [128:21:33<326:11:25,  7.23s/it]
 28%|██▊       | 62481/225000 [128:21:38<293:29:29,  6.50s/it]
 28%|██▊       | 62482/225000 [128:21:46<320:56:00,  7.11s/it]
 28%|██▊       | 62483/225000 [128:21:51<286:54:29,  6.36s/it]
 28%|██▊       | 62484/225000 [128:21:55<258:35:14,  5.73s/it]
 28%|██▊       | 62485/225000 [128:22:00<241:10:43,  5.34s/it]
 28%|██▊       | 62486/225000 [128:22:04<229:08:37,  5.08s/it]
 28%|██▊       | 62487/225000 [128:22:12<267:32:37,  5.93s/it]
 28%|██▊       | 62488/225000 [128:22:23<337:46:54,  7.48s/it]
 28%|██▊       | 62489/225000 [128:22:28<298:39:40,  6.62s/it]
 28%|██▊       | 62490/225000 [128:22:35<304:44:37,  6.75s/it]
 28%|██▊       | 62491/225000 [128:22:42<305:41:50,  6.77s/it]
 28%|██▊       | 62492/225000 [128:22:48<301:31:22,  6.68s/it]
 28%|██▊       | 62493/225000 [128:22:53<282:43:09,  6.26s/it]
 28%|██▊       | 62494/225000 [128:22:58<260:21:48,  5.77s/it]
 28%|██▊       | 62495/225000 [128:23:03<257:56:05,  5.71s/it]
 28%|██▊       | 62496/225000 [128:23:14<321:54:43,  7.13s/it]
 28%|██▊       | 62497/225000 [128:23:23<346:33:39,  7.68s/it]
 28%|██▊       | 62498/225000 [128:23:28<308:59:56,  6.85s/it]
 28%|██▊       | 62499/225000 [128:23:33<286:37:22,  6.35s/it]
 28%|██▊       | 62500/225000 [128:23:38<271:12:53,  6.01s/it]
                                                              
{'loss': '0.7562', 'grad_norm': '0.7471', 'learning_rate': '4.301e-05', 'epoch': '0.2778', 'num_input_tokens_seen': 213562272, 'train_runtime': '4.622e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62500/225000 [128:23:38<271:12:53,  6.01s/it]
 28%|██▊       | 62501/225000 [128:23:42<247:09:48,  5.48s/it]
 28%|██▊       | 62502/225000 [128:23:48<251:54:55,  5.58s/it]
 28%|██▊       | 62503/225000 [128:24:01<352:24:36,  7.81s/it]
 28%|██▊       | 62504/225000 [128:24:05<291:06:30,  6.45s/it]
 28%|██▊       | 62505/225000 [128:24:10<278:28:51,  6.17s/it]
 28%|██▊       | 62506/225000 [128:24:15<264:29:39,  5.86s/it]
 28%|██▊       | 62507/225000 [128:24:21<269:14:19,  5.96s/it]
 28%|██▊       | 62508/225000 [128:24:26<253:45:05,  5.62s/it]
 28%|██▊       | 62509/225000 [128:24:32<254:14:41,  5.63s/it]
 28%|██▊       | 62510/225000 [128:24:41<303:51:00,  6.73s/it]
 28%|██▊       | 62511/225000 [128:24:52<357:55:20,  7.93s/it]
 28%|██▊       | 62512/225000 [128:24:56<301:23:07,  6.68s/it]
 28%|██▊       | 62513/225000 [128:25:06<349:20:11,  7.74s/it]
 28%|██▊       | 62514/225000 [128:25:12<326:57:53,  7.24s/it]
 28%|██▊       | 62515/225000 [128:25:19<330:47:51,  7.33s/it]
 28%|██▊       | 62516/225000 [128:25:28<340:50:36,  7.55s/it]
 28%|██▊       | 62517/225000 [128:25:33<312:06:18,  6.92s/it]
 28%|██▊       | 62518/225000 [128:25:38<281:01:16,  6.23s/it]
 28%|██▊       | 62519/225000 [128:25:48<333:58:05,  7.40s/it]
 28%|██▊       | 62520/225000 [128:25:51<284:52:08,  6.31s/it]
 28%|██▊       | 62521/225000 [128:26:00<309:14:52,  6.85s/it]
 28%|██▊       | 62522/225000 [128:26:05<285:42:15,  6.33s/it]
 28%|██▊       | 62523/225000 [128:26:09<260:30:10,  5.77s/it]
 28%|██▊       | 62524/225000 [128:26:20<326:37:27,  7.24s/it]
 28%|██▊       | 62525/225000 [128:26:25<298:00:28,  6.60s/it]
 28%|██▊       | 62526/225000 [128:26:30<272:30:07,  6.04s/it]
 28%|██▊       | 62527/225000 [128:26:41<349:11:50,  7.74s/it]
 28%|██▊       | 62528/225000 [128:26:47<318:16:28,  7.05s/it]
 28%|██▊       | 62529/225000 [128:26:53<312:29:14,  6.92s/it]
 28%|██▊       | 62530/225000 [128:26:57<270:39:37,  6.00s/it]
 28%|██▊       | 62531/225000 [128:27:08<330:23:44,  7.32s/it]
 28%|██▊       | 62532/225000 [128:27:11<279:26:13,  6.19s/it]
 28%|██▊       | 62533/225000 [128:27:20<317:31:23,  7.04s/it]
 28%|██▊       | 62534/225000 [128:27:25<290:26:31,  6.44s/it]
 28%|██▊       | 62535/225000 [128:27:30<266:10:10,  5.90s/it]
 28%|██▊       | 62536/225000 [128:27:35<254:11:28,  5.63s/it]
 28%|██▊       | 62537/225000 [128:27:44<293:29:21,  6.50s/it]
 28%|██▊       | 62538/225000 [128:27:53<333:35:58,  7.39s/it]
 28%|██▊       | 62539/225000 [128:27:59<312:10:08,  6.92s/it]
 28%|██▊       | 62540/225000 [128:28:04<283:24:02,  6.28s/it]
 28%|██▊       | 62541/225000 [128:28:12<313:14:16,  6.94s/it]
 28%|██▊       | 62542/225000 [128:28:16<277:16:17,  6.14s/it]
 28%|██▊       | 62543/225000 [128:28:21<253:41:14,  5.62s/it]
 28%|██▊       | 62544/225000 [128:28:32<331:55:09,  7.36s/it]
 28%|██▊       | 62545/225000 [128:28:37<293:22:57,  6.50s/it]
 28%|██▊       | 62546/225000 [128:28:45<320:00:41,  7.09s/it]
 28%|██▊       | 62547/225000 [128:28:49<278:16:52,  6.17s/it]
 28%|██▊       | 62548/225000 [128:29:00<336:01:59,  7.45s/it]
 28%|██▊       | 62549/225000 [128:29:04<301:28:06,  6.68s/it]
 28%|██▊       | 62550/225000 [128:29:09<269:34:39,  5.97s/it]
                                                              
{'loss': '0.7382', 'grad_norm': '0.8114', 'learning_rate': '4.299e-05', 'epoch': '0.278', 'num_input_tokens_seen': 213709632, 'train_runtime': '4.625e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62550/225000 [128:29:09<269:34:39,  5.97s/it]
 28%|██▊       | 62551/225000 [128:29:13<250:34:11,  5.55s/it]
 28%|██▊       | 62552/225000 [128:29:25<337:54:35,  7.49s/it]
 28%|██▊       | 62553/225000 [128:29:31<309:11:16,  6.85s/it]
 28%|██▊       | 62554/225000 [128:29:39<321:43:11,  7.13s/it]
 28%|██▊       | 62555/225000 [128:29:46<332:19:31,  7.36s/it]
 28%|██▊       | 62556/225000 [128:29:51<288:59:10,  6.40s/it]
 28%|██▊       | 62557/225000 [128:29:55<266:51:53,  5.91s/it]
 28%|██▊       | 62558/225000 [128:30:00<254:17:08,  5.64s/it]
 28%|██▊       | 62559/225000 [128:30:06<253:14:24,  5.61s/it]
 28%|██▊       | 62560/225000 [128:30:21<376:09:27,  8.34s/it]
 28%|██▊       | 62561/225000 [128:30:25<319:02:15,  7.07s/it]
 28%|██▊       | 62562/225000 [128:30:29<284:16:35,  6.30s/it]
 28%|██▊       | 62563/225000 [128:30:39<326:49:12,  7.24s/it]
 28%|██▊       | 62564/225000 [128:30:44<299:05:52,  6.63s/it]
 28%|██▊       | 62565/225000 [128:30:49<277:02:55,  6.14s/it]
 28%|██▊       | 62566/225000 [128:30:54<267:27:16,  5.93s/it]
 28%|██▊       | 62567/225000 [128:31:02<290:46:53,  6.44s/it]
 28%|██▊       | 62568/225000 [128:31:10<317:55:43,  7.05s/it]
 28%|██▊       | 62569/225000 [128:31:14<274:00:58,  6.07s/it]
 28%|██▊       | 62570/225000 [128:31:20<266:13:52,  5.90s/it]
 28%|██▊       | 62571/225000 [128:31:25<264:17:54,  5.86s/it]
 28%|██▊       | 62572/225000 [128:31:34<304:40:47,  6.75s/it]
 28%|██▊       | 62573/225000 [128:31:41<305:27:41,  6.77s/it]
 28%|██▊       | 62574/225000 [128:31:46<275:34:37,  6.11s/it]
 28%|██▊       | 62575/225000 [128:31:50<252:36:44,  5.60s/it]
 28%|██▊       | 62576/225000 [128:31:59<295:33:01,  6.55s/it]
 28%|██▊       | 62577/225000 [128:32:06<306:34:21,  6.79s/it]
 28%|██▊       | 62578/225000 [128:32:11<282:40:51,  6.27s/it]
 28%|██▊       | 62579/225000 [128:32:16<269:09:23,  5.97s/it]
 28%|██▊       | 62580/225000 [128:32:21<255:13:36,  5.66s/it]
 28%|██▊       | 62581/225000 [128:32:27<248:13:18,  5.50s/it]
 28%|██▊       | 62582/225000 [128:32:38<330:17:34,  7.32s/it]
 28%|██▊       | 62583/225000 [128:32:48<358:27:34,  7.95s/it]
 28%|██▊       | 62584/225000 [128:32:51<304:10:42,  6.74s/it]
 28%|██▊       | 62585/225000 [128:33:02<356:56:14,  7.91s/it]
 28%|██▊       | 62586/225000 [128:33:08<326:27:51,  7.24s/it]
 28%|██▊       | 62587/225000 [128:33:14<309:47:23,  6.87s/it]
 28%|██▊       | 62588/225000 [128:33:18<276:18:02,  6.12s/it]
 28%|██▊       | 62589/225000 [128:33:25<285:44:34,  6.33s/it]
 28%|██▊       | 62590/225000 [128:33:34<315:31:52,  6.99s/it]
 28%|██▊       | 62591/225000 [128:33:39<293:17:01,  6.50s/it]
 28%|██▊       | 62592/225000 [128:33:45<294:17:44,  6.52s/it]
 28%|██▊       | 62593/225000 [128:33:53<301:26:58,  6.68s/it]
 28%|██▊       | 62594/225000 [128:34:01<326:06:25,  7.23s/it]
 28%|██▊       | 62595/225000 [128:34:09<330:48:30,  7.33s/it]
 28%|██▊       | 62596/225000 [128:34:13<293:07:02,  6.50s/it]
 28%|██▊       | 62597/225000 [128:34:21<313:16:49,  6.94s/it]
 28%|██▊       | 62598/225000 [128:34:25<270:48:27,  6.00s/it]
 28%|██▊       | 62599/225000 [128:34:33<299:17:26,  6.63s/it]
 28%|██▊       | 62600/225000 [128:34:40<307:48:01,  6.82s/it]
                                                              
{'loss': '0.7723', 'grad_norm': '0.7253', 'learning_rate': '4.298e-05', 'epoch': '0.2782', 'num_input_tokens_seen': 213856080, 'train_runtime': '4.629e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62600/225000 [128:34:40<307:48:01,  6.82s/it]
 28%|██▊       | 62601/225000 [128:34:46<288:07:54,  6.39s/it]
 28%|██▊       | 62602/225000 [128:34:53<301:46:33,  6.69s/it]
 28%|██▊       | 62603/225000 [128:35:01<322:40:48,  7.15s/it]
 28%|██▊       | 62604/225000 [128:35:07<297:12:29,  6.59s/it]
 28%|██▊       | 62605/225000 [128:35:14<312:03:28,  6.92s/it]
 28%|██▊       | 62606/225000 [128:35:19<277:23:59,  6.15s/it]
 28%|██▊       | 62607/225000 [128:35:23<256:16:59,  5.68s/it]
 28%|██▊       | 62608/225000 [128:35:35<340:16:47,  7.54s/it]
 28%|██▊       | 62609/225000 [128:35:41<313:53:00,  6.96s/it]
 28%|██▊       | 62610/225000 [128:35:47<300:25:32,  6.66s/it]
 28%|██▊       | 62611/225000 [128:35:52<284:58:12,  6.32s/it]
 28%|██▊       | 62612/225000 [128:35:58<273:30:51,  6.06s/it]
 28%|██▊       | 62613/225000 [128:36:03<267:38:37,  5.93s/it]
 28%|██▊       | 62614/225000 [128:36:09<264:12:41,  5.86s/it]
 28%|██▊       | 62615/225000 [128:36:16<277:10:01,  6.14s/it]
 28%|██▊       | 62616/225000 [128:36:29<375:09:26,  8.32s/it]
 28%|██▊       | 62617/225000 [128:36:35<340:59:08,  7.56s/it]
 28%|██▊       | 62618/225000 [128:36:44<357:51:54,  7.93s/it]
 28%|██▊       | 62619/225000 [128:36:48<307:45:13,  6.82s/it]
 28%|██▊       | 62620/225000 [128:36:56<321:54:29,  7.14s/it]
 28%|██▊       | 62621/225000 [128:37:02<314:30:37,  6.97s/it]
 28%|██▊       | 62622/225000 [128:37:06<271:34:01,  6.02s/it]
 28%|██▊       | 62623/225000 [128:37:12<270:34:17,  6.00s/it]
 28%|██▊       | 62624/225000 [128:37:22<317:32:04,  7.04s/it]
 28%|██▊       | 62625/225000 [128:37:26<282:19:55,  6.26s/it]
 28%|██▊       | 62626/225000 [128:37:30<253:26:39,  5.62s/it]
 28%|██▊       | 62627/225000 [128:37:42<337:44:45,  7.49s/it]
 28%|██▊       | 62628/225000 [128:37:48<311:41:01,  6.91s/it]
 28%|██▊       | 62629/225000 [128:37:52<281:35:18,  6.24s/it]
 28%|██▊       | 62630/225000 [128:37:58<273:13:03,  6.06s/it]
 28%|██▊       | 62631/225000 [128:38:03<262:49:31,  5.83s/it]
 28%|██▊       | 62632/225000 [128:38:13<315:33:53,  7.00s/it]
 28%|██▊       | 62633/225000 [128:38:22<347:03:43,  7.70s/it]
 28%|██▊       | 62634/225000 [128:38:27<302:48:32,  6.71s/it]
 28%|██▊       | 62635/225000 [128:38:31<271:51:32,  6.03s/it]
 28%|██▊       | 62636/225000 [128:38:42<340:46:05,  7.56s/it]
 28%|██▊       | 62637/225000 [128:38:46<293:23:53,  6.51s/it]
 28%|██▊       | 62638/225000 [128:38:56<333:10:31,  7.39s/it]
 28%|██▊       | 62639/225000 [128:39:00<289:00:07,  6.41s/it]
 28%|██▊       | 62640/225000 [128:39:04<263:41:16,  5.85s/it]
 28%|██▊       | 62641/225000 [128:39:17<349:20:28,  7.75s/it]
 28%|██▊       | 62642/225000 [128:39:21<297:46:20,  6.60s/it]
 28%|██▊       | 62643/225000 [128:39:25<272:40:46,  6.05s/it]
 28%|██▊       | 62644/225000 [128:39:32<277:58:46,  6.16s/it]
 28%|██▊       | 62645/225000 [128:39:42<329:11:12,  7.30s/it]
 28%|██▊       | 62646/225000 [128:39:47<297:20:55,  6.59s/it]
 28%|██▊       | 62647/225000 [128:39:51<272:10:49,  6.04s/it]
 28%|██▊       | 62648/225000 [128:39:57<261:31:36,  5.80s/it]
 28%|██▊       | 62649/225000 [128:40:01<248:19:45,  5.51s/it]
 28%|██▊       | 62650/225000 [128:40:06<242:16:29,  5.37s/it]
                                                              
{'loss': '0.7647', 'grad_norm': '0.6831', 'learning_rate': '4.297e-05', 'epoch': '0.2784', 'num_input_tokens_seen': 214003472, 'train_runtime': '4.632e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62650/225000 [128:40:06<242:16:29,  5.37s/it]
 28%|██▊       | 62651/225000 [128:40:21<368:56:33,  8.18s/it]
 28%|██▊       | 62652/225000 [128:40:27<332:02:38,  7.36s/it]
 28%|██▊       | 62653/225000 [128:40:32<307:02:54,  6.81s/it]
 28%|██▊       | 62654/225000 [128:40:41<331:56:54,  7.36s/it]
 28%|██▊       | 62655/225000 [128:40:47<315:40:46,  7.00s/it]
 28%|██▊       | 62656/225000 [128:40:53<298:42:11,  6.62s/it]
 28%|██▊       | 62657/225000 [128:40:59<290:34:36,  6.44s/it]
 28%|██▊       | 62658/225000 [128:41:10<356:22:58,  7.90s/it]
 28%|██▊       | 62659/225000 [128:41:15<310:13:39,  6.88s/it]
 28%|██▊       | 62660/225000 [128:41:19<282:40:41,  6.27s/it]
 28%|██▊       | 62661/225000 [128:41:24<261:26:34,  5.80s/it]
 28%|██▊       | 62662/225000 [128:41:36<341:57:01,  7.58s/it]
 28%|██▊       | 62663/225000 [128:41:42<326:28:38,  7.24s/it]
 28%|██▊       | 62664/225000 [128:41:47<293:04:44,  6.50s/it]
 28%|██▊       | 62665/225000 [128:41:52<270:58:19,  6.01s/it]
 28%|██▊       | 62666/225000 [128:41:57<259:35:38,  5.76s/it]
 28%|██▊       | 62667/225000 [128:42:04<276:14:03,  6.13s/it]
 28%|██▊       | 62668/225000 [128:42:17<373:34:07,  8.28s/it]
 28%|██▊       | 62669/225000 [128:42:22<318:54:11,  7.07s/it]
 28%|██▊       | 62670/225000 [128:42:26<277:21:17,  6.15s/it]
 28%|██▊       | 62671/225000 [128:42:37<350:46:32,  7.78s/it]
 28%|██▊       | 62672/225000 [128:42:43<319:53:49,  7.09s/it]
 28%|██▊       | 62673/225000 [128:42:49<311:15:26,  6.90s/it]
 28%|██▊       | 62674/225000 [128:42:59<346:14:56,  7.68s/it]
 28%|██▊       | 62675/225000 [128:43:04<319:17:01,  7.08s/it]
 28%|██▊       | 62676/225000 [128:43:09<289:05:31,  6.41s/it]
 28%|██▊       | 62677/225000 [128:43:17<308:20:43,  6.84s/it]
 28%|██▊       | 62678/225000 [128:43:24<310:02:38,  6.88s/it]
 28%|██▊       | 62679/225000 [128:43:28<274:10:30,  6.08s/it]
 28%|██▊       | 62680/225000 [128:43:33<263:14:35,  5.84s/it]
 28%|██▊       | 62681/225000 [128:43:44<327:36:15,  7.27s/it]
 28%|██▊       | 62682/225000 [128:43:48<282:37:30,  6.27s/it]
 28%|██▊       | 62683/225000 [128:43:55<295:34:12,  6.56s/it]
 28%|██▊       | 62684/225000 [128:44:03<316:20:36,  7.02s/it]
 28%|██▊       | 62685/225000 [128:44:08<290:25:22,  6.44s/it]
 28%|██▊       | 62686/225000 [128:44:13<268:17:59,  5.95s/it]
 28%|██▊       | 62687/225000 [128:44:24<333:33:16,  7.40s/it]
 28%|██▊       | 62688/225000 [128:44:30<319:04:54,  7.08s/it]
 28%|██▊       | 62689/225000 [128:44:39<341:22:17,  7.57s/it]
 28%|██▊       | 62690/225000 [128:44:45<317:56:38,  7.05s/it]
 28%|██▊       | 62691/225000 [128:44:50<284:41:54,  6.31s/it]
 28%|██▊       | 62692/225000 [128:44:54<259:38:46,  5.76s/it]
 28%|██▊       | 62693/225000 [128:45:05<333:46:30,  7.40s/it]
 28%|██▊       | 62694/225000 [128:45:10<295:38:16,  6.56s/it]
 28%|██▊       | 62695/225000 [128:45:18<323:47:45,  7.18s/it]
 28%|██▊       | 62696/225000 [128:45:25<320:53:25,  7.12s/it]
 28%|██▊       | 62697/225000 [128:45:33<331:43:22,  7.36s/it]
 28%|██▊       | 62698/225000 [128:45:39<302:37:14,  6.71s/it]
 28%|██▊       | 62699/225000 [128:45:44<287:11:27,  6.37s/it]
 28%|██▊       | 62700/225000 [128:45:49<272:27:03,  6.04s/it]
                                                              
{'loss': '0.7525', 'grad_norm': '0.7687', 'learning_rate': '4.295e-05', 'epoch': '0.2787', 'num_input_tokens_seen': 214154448, 'train_runtime': '4.635e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62700/225000 [128:45:49<272:27:03,  6.04s/it]
 28%|██▊       | 62701/225000 [128:45:58<308:29:28,  6.84s/it]
 28%|██▊       | 62702/225000 [128:46:09<368:20:43,  8.17s/it]
 28%|██▊       | 62703/225000 [128:46:13<304:29:36,  6.75s/it]
 28%|██▊       | 62704/225000 [128:46:21<323:15:09,  7.17s/it]
 28%|██▊       | 62705/225000 [128:46:29<334:15:25,  7.41s/it]
 28%|██▊       | 62706/225000 [128:46:34<304:42:39,  6.76s/it]
 28%|██▊       | 62707/225000 [128:46:40<288:56:34,  6.41s/it]
 28%|██▊       | 62708/225000 [128:46:45<272:57:24,  6.05s/it]
 28%|██▊       | 62709/225000 [128:46:50<264:45:28,  5.87s/it]
 28%|██▊       | 62710/225000 [128:46:56<258:27:24,  5.73s/it]
 28%|██▊       | 62711/225000 [128:47:02<264:54:04,  5.88s/it]
 28%|██▊       | 62712/225000 [128:47:09<278:13:12,  6.17s/it]
 28%|██▊       | 62713/225000 [128:47:22<368:29:16,  8.17s/it]
 28%|██▊       | 62714/225000 [128:47:25<305:54:56,  6.79s/it]
 28%|██▊       | 62715/225000 [128:47:33<315:50:03,  7.01s/it]
 28%|██▊       | 62716/225000 [128:47:37<279:37:48,  6.20s/it]
 28%|██▊       | 62717/225000 [128:47:43<276:19:10,  6.13s/it]
 28%|██▊       | 62718/225000 [128:47:49<270:16:45,  6.00s/it]
 28%|██▊       | 62719/225000 [128:47:55<268:13:29,  5.95s/it]
 28%|██▊       | 62720/225000 [128:48:05<323:24:39,  7.17s/it]
 28%|██▊       | 62721/225000 [128:48:13<336:58:01,  7.48s/it]
 28%|██▊       | 62722/225000 [128:48:19<315:47:30,  7.01s/it]
 28%|██▊       | 62723/225000 [128:48:24<294:36:14,  6.54s/it]
 28%|██▊       | 62724/225000 [128:48:30<285:20:47,  6.33s/it]
 28%|██▊       | 62725/225000 [128:48:41<342:31:46,  7.60s/it]
 28%|██▊       | 62726/225000 [128:48:45<301:27:18,  6.69s/it]
 28%|██▊       | 62727/225000 [128:48:54<331:25:48,  7.35s/it]
 28%|██▊       | 62728/225000 [128:48:58<291:31:27,  6.47s/it]
 28%|██▊       | 62729/225000 [128:49:03<264:18:48,  5.86s/it]
 28%|██▊       | 62730/225000 [128:49:09<271:52:12,  6.03s/it]
 28%|██▊       | 62731/225000 [128:49:21<349:05:35,  7.74s/it]
 28%|██▊       | 62732/225000 [128:49:26<303:57:33,  6.74s/it]
 28%|██▊       | 62733/225000 [128:49:34<328:18:22,  7.28s/it]
 28%|██▊       | 62734/225000 [128:49:38<284:43:37,  6.32s/it]
 28%|██▊       | 62735/225000 [128:49:50<359:06:56,  7.97s/it]
 28%|██▊       | 62736/225000 [128:49:54<310:34:07,  6.89s/it]
 28%|██▊       | 62737/225000 [128:49:59<284:55:29,  6.32s/it]
 28%|██▊       | 62738/225000 [128:50:05<273:54:08,  6.08s/it]
 28%|██▊       | 62739/225000 [128:50:11<271:44:17,  6.03s/it]
 28%|██▊       | 62740/225000 [128:50:16<263:07:33,  5.84s/it]
 28%|██▊       | 62741/225000 [128:50:22<260:46:32,  5.79s/it]
 28%|██▊       | 62742/225000 [128:50:27<256:58:11,  5.70s/it]
 28%|██▊       | 62743/225000 [128:50:37<307:35:14,  6.82s/it]
 28%|██▊       | 62744/225000 [128:50:45<324:02:31,  7.19s/it]
 28%|██▊       | 62745/225000 [128:50:52<319:30:33,  7.09s/it]
 28%|██▊       | 62746/225000 [128:50:56<281:54:22,  6.25s/it]
 28%|██▊       | 62747/225000 [128:51:01<264:49:25,  5.88s/it]
 28%|██▊       | 62748/225000 [128:51:14<361:41:12,  8.03s/it]
 28%|██▊       | 62749/225000 [128:51:19<322:16:33,  7.15s/it]
 28%|██▊       | 62750/225000 [128:51:24<295:13:40,  6.55s/it]
                                                              
{'loss': '0.7412', 'grad_norm': '0.8073', 'learning_rate': '4.294e-05', 'epoch': '0.2789', 'num_input_tokens_seen': 214301416, 'train_runtime': '4.639e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62750/225000 [128:51:24<295:13:40,  6.55s/it]
 28%|██▊       | 62751/225000 [128:51:33<319:41:43,  7.09s/it]
 28%|██▊       | 62752/225000 [128:51:37<284:23:02,  6.31s/it]
 28%|██▊       | 62753/225000 [128:51:41<258:16:00,  5.73s/it]
 28%|██▊       | 62754/225000 [128:51:54<354:41:47,  7.87s/it]
 28%|██▊       | 62755/225000 [128:51:59<310:40:11,  6.89s/it]
 28%|██▊       | 62756/225000 [128:52:06<313:03:17,  6.95s/it]
 28%|██▊       | 62757/225000 [128:52:10<266:48:06,  5.92s/it]
 28%|██▊       | 62758/225000 [128:52:14<246:17:29,  5.46s/it]
 28%|██▊       | 62759/225000 [128:52:26<329:42:48,  7.32s/it]
 28%|██▊       | 62760/225000 [128:52:29<277:52:12,  6.17s/it]
 28%|██▊       | 62761/225000 [128:52:34<257:50:54,  5.72s/it]
 28%|██▊       | 62762/225000 [128:52:39<254:54:39,  5.66s/it]
 28%|██▊       | 62763/225000 [128:52:49<312:28:29,  6.93s/it]
 28%|██▊       | 62764/225000 [128:53:00<371:55:36,  8.25s/it]
 28%|██▊       | 62765/225000 [128:53:04<314:28:19,  6.98s/it]
 28%|██▊       | 62766/225000 [128:53:08<272:43:56,  6.05s/it]
 28%|██▊       | 62767/225000 [128:53:19<331:32:22,  7.36s/it]
 28%|██▊       | 62768/225000 [128:53:23<289:09:27,  6.42s/it]
 28%|██▊       | 62769/225000 [128:53:28<271:51:10,  6.03s/it]
 28%|██▊       | 62770/225000 [128:53:39<335:01:31,  7.43s/it]
 28%|██▊       | 62771/225000 [128:53:44<310:17:15,  6.89s/it]
 28%|██▊       | 62772/225000 [128:53:53<328:23:31,  7.29s/it]
 28%|██▊       | 62773/225000 [128:53:59<318:36:05,  7.07s/it]
 28%|██▊       | 62774/225000 [128:54:05<297:24:37,  6.60s/it]
 28%|██▊       | 62775/225000 [128:54:09<267:08:50,  5.93s/it]
 28%|██▊       | 62776/225000 [128:54:20<335:21:45,  7.44s/it]
 28%|██▊       | 62777/225000 [128:54:25<302:28:55,  6.71s/it]
 28%|██▊       | 62778/225000 [128:54:31<290:51:58,  6.45s/it]
 28%|██▊       | 62779/225000 [128:54:37<279:01:36,  6.19s/it]
 28%|██▊       | 62780/225000 [128:54:42<271:09:16,  6.02s/it]
 28%|██▊       | 62781/225000 [128:54:47<260:10:28,  5.77s/it]
 28%|██▊       | 62782/225000 [128:54:56<303:15:00,  6.73s/it]
 28%|██▊       | 62783/225000 [128:55:04<321:39:57,  7.14s/it]
 28%|██▊       | 62784/225000 [128:55:12<326:32:07,  7.25s/it]
 28%|██▊       | 62785/225000 [128:55:16<279:28:34,  6.20s/it]
 28%|██▊       | 62786/225000 [128:55:20<258:45:25,  5.74s/it]
 28%|██▊       | 62787/225000 [128:55:30<313:18:50,  6.95s/it]
 28%|██▊       | 62788/225000 [128:55:35<288:59:59,  6.41s/it]
 28%|██▊       | 62789/225000 [128:55:40<265:53:02,  5.90s/it]
 28%|██▊       | 62790/225000 [128:55:45<248:19:51,  5.51s/it]
 28%|██▊       | 62791/225000 [128:55:53<290:39:01,  6.45s/it]
 28%|██▊       | 62792/225000 [128:56:02<326:04:59,  7.24s/it]
 28%|██▊       | 62793/225000 [128:56:07<294:00:23,  6.53s/it]
 28%|██▊       | 62794/225000 [128:56:16<327:49:33,  7.28s/it]
 28%|██▊       | 62795/225000 [128:56:20<277:30:05,  6.16s/it]
 28%|██▊       | 62796/225000 [128:56:28<310:34:42,  6.89s/it]
 28%|██▊       | 62797/225000 [128:56:35<305:14:18,  6.77s/it]
 28%|██▊       | 62798/225000 [128:56:39<269:51:25,  5.99s/it]
 28%|██▊       | 62799/225000 [128:56:50<337:12:52,  7.48s/it]
 28%|██▊       | 62800/225000 [128:56:54<290:42:42,  6.45s/it]
                                                              
{'loss': '0.7572', 'grad_norm': '0.7202', 'learning_rate': '4.293e-05', 'epoch': '0.2791', 'num_input_tokens_seen': 214448816, 'train_runtime': '4.642e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62800/225000 [128:56:54<290:42:42,  6.45s/it]
 28%|██▊       | 62801/225000 [128:57:03<327:11:09,  7.26s/it]
 28%|██▊       | 62802/225000 [128:57:08<288:22:05,  6.40s/it]
 28%|██▊       | 62803/225000 [128:57:12<266:08:23,  5.91s/it]
 28%|██▊       | 62804/225000 [128:57:23<333:24:17,  7.40s/it]
 28%|██▊       | 62805/225000 [128:57:27<289:03:25,  6.42s/it]
 28%|██▊       | 62806/225000 [128:57:31<258:15:02,  5.73s/it]
 28%|██▊       | 62807/225000 [128:57:42<320:31:49,  7.11s/it]
 28%|██▊       | 62808/225000 [128:57:47<289:07:16,  6.42s/it]
 28%|██▊       | 62809/225000 [128:57:52<271:42:08,  6.03s/it]
 28%|██▊       | 62810/225000 [128:58:00<303:12:15,  6.73s/it]
 28%|██▊       | 62811/225000 [128:58:04<262:40:13,  5.83s/it]
 28%|██▊       | 62812/225000 [128:58:14<324:34:55,  7.20s/it]
 28%|██▊       | 62813/225000 [128:58:20<300:52:46,  6.68s/it]
 28%|██▊       | 62814/225000 [128:58:25<282:01:37,  6.26s/it]
 28%|██▊       | 62815/225000 [128:58:30<271:09:10,  6.02s/it]
 28%|██▊       | 62816/225000 [128:58:36<259:54:30,  5.77s/it]
 28%|██▊       | 62817/225000 [128:58:40<245:32:36,  5.45s/it]
 28%|██▊       | 62818/225000 [128:58:46<251:02:58,  5.57s/it]
 28%|██▊       | 62819/225000 [128:58:55<289:13:59,  6.42s/it]
 28%|██▊       | 62820/225000 [128:59:06<360:27:25,  8.00s/it]
 28%|██▊       | 62821/225000 [128:59:12<329:04:00,  7.30s/it]
 28%|██▊       | 62822/225000 [128:59:19<319:49:52,  7.10s/it]
 28%|██▊       | 62823/225000 [128:59:26<325:51:28,  7.23s/it]
 28%|██▊       | 62824/225000 [128:59:30<285:57:56,  6.35s/it]
 28%|██▊       | 62825/225000 [128:59:36<273:11:19,  6.06s/it]
 28%|██▊       | 62826/225000 [128:59:45<320:25:32,  7.11s/it]
 28%|██▊       | 62827/225000 [128:59:52<308:30:24,  6.85s/it]
 28%|██▊       | 62828/225000 [128:59:59<312:38:49,  6.94s/it]
 28%|██▊       | 62829/225000 [129:00:06<318:10:22,  7.06s/it]
 28%|██▊       | 62830/225000 [129:00:10<280:16:27,  6.22s/it]
 28%|██▊       | 62831/225000 [129:00:15<257:27:56,  5.72s/it]
 28%|██▊       | 62832/225000 [129:00:21<260:25:50,  5.78s/it]
 28%|██▊       | 62833/225000 [129:00:29<291:02:52,  6.46s/it]
 28%|██▊       | 62834/225000 [129:00:34<268:40:05,  5.96s/it]
 28%|██▊       | 62835/225000 [129:00:38<247:12:13,  5.49s/it]
 28%|██▊       | 62836/225000 [129:00:43<234:41:01,  5.21s/it]
 28%|██▊       | 62837/225000 [129:00:54<324:35:09,  7.21s/it]
 28%|██▊       | 62838/225000 [129:01:02<327:01:48,  7.26s/it]
 28%|██▊       | 62839/225000 [129:01:06<282:35:56,  6.27s/it]
 28%|██▊       | 62840/225000 [129:01:10<251:31:09,  5.58s/it]
 28%|██▊       | 62841/225000 [129:01:18<292:42:15,  6.50s/it]
 28%|██▊       | 62842/225000 [129:01:26<304:28:59,  6.76s/it]
 28%|██▊       | 62843/225000 [129:01:30<268:14:17,  5.96s/it]
 28%|██▊       | 62844/225000 [129:01:38<296:45:00,  6.59s/it]
 28%|██▊       | 62845/225000 [129:01:41<254:57:39,  5.66s/it]
 28%|██▊       | 62846/225000 [129:01:46<245:20:59,  5.45s/it]
 28%|██▊       | 62847/225000 [129:01:51<239:36:25,  5.32s/it]
 28%|██▊       | 62848/225000 [129:01:57<238:52:30,  5.30s/it]
 28%|██▊       | 62849/225000 [129:02:07<304:53:55,  6.77s/it]
 28%|██▊       | 62850/225000 [129:02:15<319:12:18,  7.09s/it]
                                                              
{'loss': '0.7672', 'grad_norm': '0.9148', 'learning_rate': '4.292e-05', 'epoch': '0.2793', 'num_input_tokens_seen': 214594648, 'train_runtime': '4.645e+05', 'train_tokens_per_second': '462'}

 28%|██▊       | 62850/225000 [129:02:15<319:12:18,  7.09s/it]
 28%|██▊       | 62851/225000 [129:02:23<332:43:27,  7.39s/it]
 28%|██▊       | 62852/225000 [129:02:28<299:08:09,  6.64s/it]
 28%|██▊       | 62853/225000 [129:02:32<269:31:22,  5.98s/it]
 28%|██▊       | 62854/225000 [129:02:36<246:40:40,  5.48s/it]
 28%|██▊       | 62855/225000 [129:02:50<357:12:21,  7.93s/it]
 28%|██▊       | 62856/225000 [129:02:55<312:44:19,  6.94s/it]
 28%|██▊       | 62857/225000 [129:03:03<334:18:05,  7.42s/it]
 28%|██▊       | 62858/225000 [129:03:08<304:00:23,  6.75s/it]
 28%|██▊       | 62859/225000 [129:03:15<298:35:16,  6.63s/it]
 28%|██▊       | 62860/225000 [129:03:19<268:51:07,  5.97s/it]
 28%|██▊       | 62861/225000 [129:03:24<247:20:23,  5.49s/it]
 28%|██▊       | 62862/225000 [129:03:36<342:16:08,  7.60s/it]
 28%|██▊       | 62863/225000 [129:03:41<307:49:38,  6.83s/it]
 28%|██▊       | 62864/225000 [129:03:47<288:17:22,  6.40s/it]
 28%|██▊       | 62865/225000 [129:03:51<267:35:37,  5.94s/it]
 28%|██▊       | 62866/225000 [129:03:57<258:49:20,  5.75s/it]
 28%|██▊       | 62867/225000 [129:04:02<250:02:04,  5.55s/it]
 28%|██▊       | 62868/225000 [129:04:07<250:59:31,  5.57s/it]
 28%|██▊       | 62869/225000 [129:04:21<354:47:53,  7.88s/it]
 28%|██▊       | 62870/225000 [129:04:29<356:11:13,  7.91s/it]
 28%|██▊       | 62871/225000 [129:04:33<311:17:45,  6.91s/it]
 28%|██▊       | 62872/225000 [129:04:39<290:57:58,  6.46s/it]
 28%|██▊       | 62873/225000 [129:04:49<339:30:12,  7.54s/it]
 28%|██▊       | 62874/225000 [129:04:53<291:19:51,  6.47s/it]
 28%|██▊       | 62875/225000 [129:05:00<309:43:20,  6.88s/it]
 28%|██▊       | 62876/225000 [129:05:05<282:01:51,  6.26s/it]
 28%|██▊       | 62877/225000 [129:05:14<317:42:18,  7.05s/it]
 28%|██▊       | 62878/225000 [129:05:18<279:21:06,  6.20s/it]
 28%|██▊       | 62879/225000 [129:05:26<294:15:54,  6.53s/it]
 28%|██▊       | 62880/225000 [129:05:34<320:31:56,  7.12s/it]
 28%|██▊       | 62881/225000 [129:05:40<297:25:12,  6.60s/it]
 28%|██▊       | 62882/225000 [129:05:46<289:11:54,  6.42s/it]
 28%|██▊       | 62883/225000 [129:05:53<308:25:22,  6.85s/it]
 28%|██▊       | 62884/225000 [129:05:58<278:57:25,  6.19s/it]
 28%|██▊       | 62885/225000 [129:06:04<269:23:35,  5.98s/it]
 28%|██▊       | 62886/225000 [129:06:09<267:47:17,  5.95s/it]
 28%|██▊       | 62887/225000 [129:06:15<256:45:14,  5.70s/it]
 28%|██▊       | 62888/225000 [129:06:20<257:21:12,  5.72s/it]
 28%|██▊       | 62889/225000 [129:06:26<255:02:28,  5.66s/it]
 28%|██▊       | 62890/225000 [129:06:31<253:09:36,  5.62s/it]
 28%|██▊       | 62891/225000 [129:06:38<259:32:32,  5.76s/it]
 28%|██▊       | 62892/225000 [129:06:51<361:29:38,  8.03s/it]
 28%|██▊       | 62893/225000 [129:06:59<363:50:12,  8.08s/it]
 28%|██▊       | 62894/225000 [129:07:04<317:48:49,  7.06s/it]
 28%|██▊       | 62895/225000 [129:07:10<307:25:36,  6.83s/it]
 28%|██▊       | 62896/225000 [129:07:18<320:32:06,  7.12s/it]
 28%|██▊       | 62897/225000 [129:07:26<329:11:24,  7.31s/it]
 28%|██▊       | 62898/225000 [129:07:30<295:45:04,  6.57s/it]
 28%|██▊       | 62899/225000 [129:07:39<317:17:45,  7.05s/it]
 28%|██▊       | 62900/225000 [129:07:42<272:00:07,  6.04s/it]
                                                              
{'loss': '0.7671', 'grad_norm': '0.7801', 'learning_rate': '4.29e-05', 'epoch': '0.2796', 'num_input_tokens_seen': 214742608, 'train_runtime': '4.649e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 62900/225000 [129:07:42<272:00:07,  6.04s/it]
 28%|██▊       | 62901/225000 [129:07:47<257:14:49,  5.71s/it]
 28%|██▊       | 62902/225000 [129:07:53<254:44:15,  5.66s/it]
 28%|██▊       | 62903/225000 [129:07:58<248:20:01,  5.52s/it]
 28%|██▊       | 62904/225000 [129:08:03<243:47:52,  5.41s/it]
 28%|██▊       | 62905/225000 [129:08:11<283:41:58,  6.30s/it]
 28%|██▊       | 62906/225000 [129:08:23<356:37:47,  7.92s/it]
 28%|██▊       | 62907/225000 [129:08:30<339:39:29,  7.54s/it]
 28%|██▊       | 62908/225000 [129:08:34<291:25:55,  6.47s/it]
 28%|██▊       | 62909/225000 [129:08:42<311:50:15,  6.93s/it]
 28%|██▊       | 62910/225000 [129:08:46<274:56:17,  6.11s/it]
 28%|██▊       | 62911/225000 [129:08:57<336:40:59,  7.48s/it]
 28%|██▊       | 62912/225000 [129:09:04<332:42:16,  7.39s/it]
 28%|██▊       | 62913/225000 [129:09:09<298:09:48,  6.62s/it]
 28%|██▊       | 62914/225000 [129:09:14<275:41:22,  6.12s/it]
 28%|██▊       | 62915/225000 [129:09:19<260:44:41,  5.79s/it]
 28%|██▊       | 62916/225000 [129:09:23<240:28:47,  5.34s/it]
 28%|██▊       | 62917/225000 [129:09:34<319:52:37,  7.10s/it]
 28%|██▊       | 62918/225000 [129:09:42<323:43:45,  7.19s/it]
 28%|██▊       | 62919/225000 [129:09:46<289:47:41,  6.44s/it]
 28%|██▊       | 62920/225000 [129:09:51<266:13:53,  5.91s/it]
 28%|██▊       | 62921/225000 [129:09:56<250:43:57,  5.57s/it]
 28%|██▊       | 62922/225000 [129:10:09<351:17:15,  7.80s/it]
 28%|██▊       | 62923/225000 [129:10:13<298:25:11,  6.63s/it]
 28%|██▊       | 62924/225000 [129:10:21<327:00:03,  7.26s/it]
 28%|██▊       | 62925/225000 [129:10:25<280:36:59,  6.23s/it]
 28%|██▊       | 62926/225000 [129:10:30<255:37:32,  5.68s/it]
 28%|██▊       | 62927/225000 [129:10:37<280:25:18,  6.23s/it]
 28%|██▊       | 62928/225000 [129:10:44<287:30:05,  6.39s/it]
 28%|██▊       | 62929/225000 [129:10:48<253:54:15,  5.64s/it]
 28%|██▊       | 62930/225000 [129:10:58<321:41:02,  7.15s/it]
 28%|██▊       | 62931/225000 [129:11:02<272:30:33,  6.05s/it]
 28%|██▊       | 62932/225000 [129:11:11<314:36:57,  6.99s/it]
 28%|██▊       | 62933/225000 [129:11:18<309:32:44,  6.88s/it]
 28%|██▊       | 62934/225000 [129:11:22<282:04:19,  6.27s/it]
 28%|██▊       | 62935/225000 [129:11:27<263:49:34,  5.86s/it]
 28%|██▊       | 62936/225000 [129:11:32<245:37:40,  5.46s/it]
 28%|██▊       | 62937/225000 [129:11:39<266:50:31,  5.93s/it]
 28%|██▊       | 62938/225000 [129:11:50<329:47:50,  7.33s/it]
 28%|██▊       | 62939/225000 [129:11:55<308:43:49,  6.86s/it]
 28%|██▊       | 62940/225000 [129:12:00<282:42:49,  6.28s/it]
 28%|██▊       | 62941/225000 [129:12:05<267:02:10,  5.93s/it]
 28%|██▊       | 62942/225000 [129:12:10<248:57:35,  5.53s/it]
 28%|██▊       | 62943/225000 [129:12:18<283:57:40,  6.31s/it]
 28%|██▊       | 62944/225000 [129:12:31<369:23:39,  8.21s/it]
 28%|██▊       | 62945/225000 [129:12:35<312:41:57,  6.95s/it]
 28%|██▊       | 62946/225000 [129:12:38<264:27:09,  5.87s/it]
 28%|██▊       | 62947/225000 [129:12:47<299:45:44,  6.66s/it]
 28%|██▊       | 62948/225000 [129:12:53<300:48:31,  6.68s/it]
 28%|██▊       | 62949/225000 [129:12:59<290:20:02,  6.45s/it]
 28%|██▊       | 62950/225000 [129:13:08<315:34:43,  7.01s/it]
                                                              
{'loss': '0.7657', 'grad_norm': '0.8154', 'learning_rate': '4.289e-05', 'epoch': '0.2798', 'num_input_tokens_seen': 214889032, 'train_runtime': '4.652e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 62950/225000 [129:13:08<315:34:43,  7.01s/it]
 28%|██▊       | 62951/225000 [129:13:14<306:20:18,  6.81s/it]
 28%|██▊       | 62952/225000 [129:13:20<290:40:07,  6.46s/it]
 28%|██▊       | 62953/225000 [129:13:26<286:32:16,  6.37s/it]
 28%|██▊       | 62954/225000 [129:13:34<309:19:03,  6.87s/it]
 28%|██▊       | 62955/225000 [129:13:37<266:39:21,  5.92s/it]
 28%|██▊       | 62956/225000 [129:13:46<304:32:31,  6.77s/it]
 28%|██▊       | 62957/225000 [129:13:50<261:16:11,  5.80s/it]
 28%|██▊       | 62958/225000 [129:13:55<252:06:59,  5.60s/it]
 28%|██▊       | 62959/225000 [129:14:00<248:25:41,  5.52s/it]
 28%|██▊       | 62960/225000 [129:14:06<249:08:11,  5.54s/it]
 28%|██▊       | 62961/225000 [129:14:11<250:36:08,  5.57s/it]
 28%|██▊       | 62962/225000 [129:14:17<251:06:06,  5.58s/it]
 28%|██▊       | 62963/225000 [129:14:23<256:04:39,  5.69s/it]
 28%|██▊       | 62964/225000 [129:14:31<294:00:01,  6.53s/it]
 28%|██▊       | 62965/225000 [129:14:44<375:03:49,  8.33s/it]
 28%|██▊       | 62966/225000 [129:14:48<320:16:04,  7.12s/it]
 28%|██▊       | 62967/225000 [129:14:55<312:28:57,  6.94s/it]
 28%|██▊       | 62968/225000 [129:14:59<273:01:27,  6.07s/it]
 28%|██▊       | 62969/225000 [129:15:04<255:14:37,  5.67s/it]
 28%|██▊       | 62970/225000 [129:15:16<346:54:52,  7.71s/it]
 28%|██▊       | 62971/225000 [129:15:19<288:33:19,  6.41s/it]
 28%|██▊       | 62972/225000 [129:15:24<264:40:01,  5.88s/it]
 28%|██▊       | 62973/225000 [129:15:34<316:58:41,  7.04s/it]
 28%|██▊       | 62974/225000 [129:15:38<272:20:51,  6.05s/it]
 28%|██▊       | 62975/225000 [129:15:44<283:28:09,  6.30s/it]
 28%|██▊       | 62976/225000 [129:15:53<312:31:28,  6.94s/it]
 28%|██▊       | 62977/225000 [129:15:57<269:37:24,  5.99s/it]
 28%|██▊       | 62978/225000 [129:16:02<258:51:56,  5.75s/it]
 28%|██▊       | 62979/225000 [129:16:07<254:01:21,  5.64s/it]
 28%|██▊       | 62980/225000 [129:16:13<250:19:48,  5.56s/it]
 28%|██▊       | 62981/225000 [129:16:21<292:52:37,  6.51s/it]
 28%|██▊       | 62982/225000 [129:16:26<268:28:41,  5.97s/it]
 28%|██▊       | 62983/225000 [129:16:31<258:13:13,  5.74s/it]
 28%|██▊       | 62984/225000 [129:16:44<349:49:32,  7.77s/it]
 28%|██▊       | 62985/225000 [129:16:49<321:29:21,  7.14s/it]
 28%|██▊       | 62986/225000 [129:16:54<293:11:47,  6.51s/it]
 28%|██▊       | 62987/225000 [129:17:03<319:53:07,  7.11s/it]
 28%|██▊       | 62988/225000 [129:17:07<275:34:28,  6.12s/it]
 28%|██▊       | 62989/225000 [129:17:15<307:11:51,  6.83s/it]
 28%|██▊       | 62990/225000 [129:17:22<303:37:41,  6.75s/it]
 28%|██▊       | 62991/225000 [129:17:27<277:56:46,  6.18s/it]
 28%|██▊       | 62992/225000 [129:17:31<258:37:03,  5.75s/it]
 28%|██▊       | 62993/225000 [129:17:36<247:16:38,  5.49s/it]
 28%|██▊       | 62994/225000 [129:17:47<310:55:31,  6.91s/it]
 28%|██▊       | 62995/225000 [129:17:55<331:12:53,  7.36s/it]
 28%|██▊       | 62996/225000 [129:17:59<287:48:00,  6.40s/it]
 28%|██▊       | 62997/225000 [129:18:10<347:30:56,  7.72s/it]
 28%|██▊       | 62998/225000 [129:18:14<293:04:34,  6.51s/it]
 28%|██▊       | 62999/225000 [129:18:22<312:42:52,  6.95s/it]
 28%|██▊       | 63000/225000 [129:18:25<270:06:37,  6.00s/it]
                                                              
{'loss': '0.7509', 'grad_norm': '0.8383', 'learning_rate': '4.288e-05', 'epoch': '0.28', 'num_input_tokens_seen': 215035008, 'train_runtime': '4.655e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63000/225000 [129:18:25<270:06:37,  6.00s/it]
 28%|██▊       | 63001/225000 [129:18:31<264:13:31,  5.87s/it]
 28%|██▊       | 63002/225000 [129:18:37<262:23:26,  5.83s/it]
 28%|██▊       | 63003/225000 [129:18:42<258:00:29,  5.73s/it]
 28%|██▊       | 63004/225000 [129:18:48<256:00:12,  5.69s/it]
 28%|██▊       | 63005/225000 [129:18:53<254:52:29,  5.66s/it]
 28%|██▊       | 63006/225000 [129:18:59<254:16:40,  5.65s/it]
 28%|██▊       | 63007/225000 [129:19:06<273:13:53,  6.07s/it]
 28%|██▊       | 63008/225000 [129:19:11<262:33:36,  5.83s/it]
 28%|██▊       | 63009/225000 [129:19:16<252:08:48,  5.60s/it]
 28%|██▊       | 63010/225000 [129:19:22<253:02:27,  5.62s/it]
 28%|██▊       | 63011/225000 [129:19:29<272:49:01,  6.06s/it]
 28%|██▊       | 63012/225000 [129:19:42<368:34:13,  8.19s/it]
 28%|██▊       | 63013/225000 [129:19:48<330:47:20,  7.35s/it]
 28%|██▊       | 63014/225000 [129:19:53<300:03:59,  6.67s/it]
 28%|██▊       | 63015/225000 [129:19:58<285:31:20,  6.35s/it]
 28%|██▊       | 63016/225000 [129:20:04<270:29:55,  6.01s/it]
 28%|██▊       | 63017/225000 [129:20:14<331:03:50,  7.36s/it]
 28%|██▊       | 63018/225000 [129:20:18<288:58:20,  6.42s/it]
 28%|██▊       | 63019/225000 [129:20:22<259:04:19,  5.76s/it]
 28%|██▊       | 63020/225000 [129:20:34<340:56:31,  7.58s/it]
 28%|██▊       | 63021/225000 [129:20:38<293:50:25,  6.53s/it]
 28%|██▊       | 63022/225000 [129:20:46<302:58:44,  6.73s/it]
 28%|██▊       | 63023/225000 [129:20:54<321:41:24,  7.15s/it]
 28%|██▊       | 63024/225000 [129:20:58<279:50:22,  6.22s/it]
 28%|██▊       | 63025/225000 [129:21:02<256:52:03,  5.71s/it]
 28%|██▊       | 63026/225000 [129:21:11<299:17:01,  6.65s/it]
 28%|██▊       | 63027/225000 [129:21:19<310:39:33,  6.90s/it]
 28%|██▊       | 63028/225000 [129:21:23<276:31:03,  6.15s/it]
 28%|██▊       | 63029/225000 [129:21:30<292:12:14,  6.49s/it]
 28%|██▊       | 63030/225000 [129:21:35<267:57:33,  5.96s/it]
 28%|██▊       | 63031/225000 [129:21:40<258:15:12,  5.74s/it]
 28%|██▊       | 63032/225000 [129:21:46<254:29:43,  5.66s/it]
 28%|██▊       | 63033/225000 [129:21:58<349:43:31,  7.77s/it]
 28%|██▊       | 63034/225000 [129:22:06<345:47:30,  7.69s/it]
 28%|██▊       | 63035/225000 [129:22:14<347:44:48,  7.73s/it]
 28%|██▊       | 63036/225000 [129:22:17<289:21:10,  6.43s/it]
 28%|██▊       | 63037/225000 [129:22:26<318:25:05,  7.08s/it]
 28%|██▊       | 63038/225000 [129:22:33<319:13:39,  7.10s/it]
 28%|██▊       | 63039/225000 [129:22:37<284:56:14,  6.33s/it]
 28%|██▊       | 63040/225000 [129:22:42<265:38:47,  5.90s/it]
 28%|██▊       | 63041/225000 [129:22:47<253:27:53,  5.63s/it]
 28%|██▊       | 63042/225000 [129:22:52<246:07:25,  5.47s/it]
 28%|██▊       | 63043/225000 [129:22:57<240:18:10,  5.34s/it]
 28%|██▊       | 63044/225000 [129:23:09<323:45:29,  7.20s/it]
 28%|██▊       | 63045/225000 [129:23:15<313:45:02,  6.97s/it]
 28%|██▊       | 63046/225000 [129:23:22<305:20:42,  6.79s/it]
 28%|██▊       | 63047/225000 [129:23:30<320:23:18,  7.12s/it]
 28%|██▊       | 63048/225000 [129:23:34<287:58:28,  6.40s/it]
 28%|██▊       | 63049/225000 [129:23:43<314:31:31,  6.99s/it]
 28%|██▊       | 63050/225000 [129:23:49<299:23:16,  6.66s/it]
                                                              
{'loss': '0.7664', 'grad_norm': '0.9026', 'learning_rate': '4.287e-05', 'epoch': '0.2802', 'num_input_tokens_seen': 215182416, 'train_runtime': '4.658e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63050/225000 [129:23:49<299:23:16,  6.66s/it]
 28%|██▊       | 63051/225000 [129:23:56<305:02:16,  6.78s/it]
 28%|██▊       | 63052/225000 [129:23:59<256:31:07,  5.70s/it]
 28%|██▊       | 63053/225000 [129:24:09<314:04:24,  6.98s/it]
 28%|██▊       | 63054/225000 [129:24:13<277:44:06,  6.17s/it]
 28%|██▊       | 63055/225000 [129:24:17<252:37:48,  5.62s/it]
 28%|██▊       | 63056/225000 [129:24:23<245:47:59,  5.46s/it]
 28%|██▊       | 63057/225000 [129:24:34<330:28:24,  7.35s/it]
 28%|██▊       | 63058/225000 [129:24:40<301:09:44,  6.69s/it]
 28%|██▊       | 63059/225000 [129:24:47<316:38:56,  7.04s/it]
 28%|██▊       | 63060/225000 [129:24:55<319:24:58,  7.10s/it]
 28%|██▊       | 63061/225000 [129:24:59<282:31:58,  6.28s/it]
 28%|██▊       | 63062/225000 [129:25:03<252:59:48,  5.62s/it]
 28%|██▊       | 63063/225000 [129:25:08<239:14:51,  5.32s/it]
 28%|██▊       | 63064/225000 [129:25:20<331:55:24,  7.38s/it]
 28%|██▊       | 63065/225000 [129:25:24<289:03:09,  6.43s/it]
 28%|██▊       | 63066/225000 [129:25:32<311:39:26,  6.93s/it]
 28%|██▊       | 63067/225000 [129:25:41<337:38:28,  7.51s/it]
 28%|██▊       | 63068/225000 [129:25:45<289:08:17,  6.43s/it]
 28%|██▊       | 63069/225000 [129:25:52<300:38:56,  6.68s/it]
 28%|██▊       | 63070/225000 [129:25:56<265:15:49,  5.90s/it]
 28%|██▊       | 63071/225000 [129:26:01<249:54:09,  5.56s/it]
 28%|██▊       | 63072/225000 [129:26:06<239:47:03,  5.33s/it]
 28%|██▊       | 63073/225000 [129:26:11<237:08:30,  5.27s/it]
 28%|██▊       | 63074/225000 [129:26:16<236:25:46,  5.26s/it]
 28%|██▊       | 63075/225000 [129:26:29<342:02:15,  7.60s/it]
 28%|██▊       | 63076/225000 [129:26:37<343:40:12,  7.64s/it]
 28%|██▊       | 63077/225000 [129:26:41<292:24:12,  6.50s/it]
 28%|██▊       | 63078/225000 [129:26:51<342:04:51,  7.61s/it]
 28%|██▊       | 63079/225000 [129:26:55<298:39:46,  6.64s/it]
 28%|██▊       | 63080/225000 [129:26:59<259:22:09,  5.77s/it]
 28%|██▊       | 63081/225000 [129:27:07<288:28:51,  6.41s/it]
 28%|██▊       | 63082/225000 [129:27:14<298:25:53,  6.64s/it]
 28%|██▊       | 63083/225000 [129:27:18<259:12:12,  5.76s/it]
 28%|██▊       | 63084/225000 [129:27:24<257:07:22,  5.72s/it]
 28%|██▊       | 63085/225000 [129:27:35<334:37:56,  7.44s/it]
 28%|██▊       | 63086/225000 [129:27:39<282:32:34,  6.28s/it]
 28%|██▊       | 63087/225000 [129:27:48<318:52:02,  7.09s/it]
 28%|██▊       | 63088/225000 [129:27:52<283:16:41,  6.30s/it]
 28%|██▊       | 63089/225000 [129:27:56<256:56:58,  5.71s/it]
 28%|██▊       | 63090/225000 [129:28:07<326:21:41,  7.26s/it]
 28%|██▊       | 63091/225000 [129:28:12<297:41:14,  6.62s/it]
 28%|██▊       | 63092/225000 [129:28:17<271:05:51,  6.03s/it]
 28%|██▊       | 63093/225000 [129:28:21<250:08:07,  5.56s/it]
 28%|██▊       | 63094/225000 [129:28:34<346:01:34,  7.69s/it]
 28%|██▊       | 63095/225000 [129:28:39<304:02:37,  6.76s/it]
 28%|██▊       | 63096/225000 [129:28:43<273:59:42,  6.09s/it]
 28%|██▊       | 63097/225000 [129:28:48<252:27:07,  5.61s/it]
 28%|██▊       | 63098/225000 [129:28:55<274:07:30,  6.10s/it]
 28%|██▊       | 63099/225000 [129:29:06<343:47:30,  7.64s/it]
 28%|██▊       | 63100/225000 [129:29:10<289:08:20,  6.43s/it]
                                                              
{'loss': '0.7605', 'grad_norm': '0.8225', 'learning_rate': '4.285e-05', 'epoch': '0.2804', 'num_input_tokens_seen': 215327080, 'train_runtime': '4.662e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63100/225000 [129:29:10<289:08:20,  6.43s/it]
 28%|██▊       | 63101/225000 [129:29:18<316:50:05,  7.05s/it]
 28%|██▊       | 63102/225000 [129:29:22<275:27:15,  6.13s/it]
 28%|██▊       | 63103/225000 [129:29:27<255:28:48,  5.68s/it]
 28%|██▊       | 63104/225000 [129:29:32<243:18:22,  5.41s/it]
 28%|██▊       | 63105/225000 [129:29:41<299:41:30,  6.66s/it]
 28%|██▊       | 63106/225000 [129:29:50<322:13:12,  7.17s/it]
 28%|██▊       | 63107/225000 [129:29:54<278:05:28,  6.18s/it]
 28%|██▊       | 63108/225000 [129:30:04<333:20:36,  7.41s/it]
 28%|██▊       | 63109/225000 [129:30:09<301:42:28,  6.71s/it]
 28%|██▊       | 63110/225000 [129:30:17<324:08:25,  7.21s/it]
 28%|██▊       | 63111/225000 [129:30:23<306:06:24,  6.81s/it]
 28%|██▊       | 63112/225000 [129:30:28<274:03:03,  6.09s/it]
 28%|██▊       | 63113/225000 [129:30:32<253:55:17,  5.65s/it]
 28%|██▊       | 63114/225000 [129:30:39<271:18:06,  6.03s/it]
 28%|██▊       | 63115/225000 [129:30:49<329:15:40,  7.32s/it]
 28%|██▊       | 63116/225000 [129:30:56<318:26:41,  7.08s/it]
 28%|██▊       | 63117/225000 [129:31:00<271:44:16,  6.04s/it]
 28%|██▊       | 63118/225000 [129:31:08<306:53:31,  6.82s/it]
 28%|██▊       | 63119/225000 [129:31:12<267:17:42,  5.94s/it]
 28%|██▊       | 63120/225000 [129:31:23<329:35:16,  7.33s/it]
 28%|██▊       | 63121/225000 [129:31:27<287:26:46,  6.39s/it]
 28%|██▊       | 63122/225000 [129:31:34<292:28:57,  6.50s/it]
 28%|██▊       | 63123/225000 [129:31:40<286:02:02,  6.36s/it]
 28%|██▊       | 63124/225000 [129:31:43<251:39:35,  5.60s/it]
 28%|██▊       | 63125/225000 [129:31:52<292:23:33,  6.50s/it]
 28%|██▊       | 63126/225000 [129:31:58<281:11:14,  6.25s/it]
 28%|██▊       | 63127/225000 [129:32:07<318:17:41,  7.08s/it]
 28%|██▊       | 63128/225000 [129:32:13<310:50:02,  6.91s/it]
 28%|██▊       | 63129/225000 [129:32:18<275:31:31,  6.13s/it]
 28%|██▊       | 63130/225000 [129:32:23<265:54:25,  5.91s/it]
 28%|██▊       | 63131/225000 [129:32:28<254:40:55,  5.66s/it]
 28%|██▊       | 63132/225000 [129:32:33<248:08:20,  5.52s/it]
 28%|██▊       | 63133/225000 [129:32:39<249:14:14,  5.54s/it]
 28%|██▊       | 63134/225000 [129:32:44<248:58:46,  5.54s/it]
 28%|██▊       | 63135/225000 [129:32:50<249:57:01,  5.56s/it]
 28%|██▊       | 63136/225000 [129:33:02<337:06:00,  7.50s/it]
 28%|██▊       | 63137/225000 [129:33:10<346:05:55,  7.70s/it]
 28%|██▊       | 63138/225000 [129:33:17<338:43:31,  7.53s/it]
 28%|██▊       | 63139/225000 [129:33:21<292:30:23,  6.51s/it]
 28%|██▊       | 63140/225000 [129:33:26<263:50:20,  5.87s/it]
 28%|██▊       | 63141/225000 [129:33:37<332:34:47,  7.40s/it]
 28%|██▊       | 63142/225000 [129:33:42<296:44:40,  6.60s/it]
 28%|██▊       | 63143/225000 [129:33:46<270:13:01,  6.01s/it]
 28%|██▊       | 63144/225000 [129:33:51<254:07:30,  5.65s/it]
 28%|██▊       | 63145/225000 [129:34:02<320:45:04,  7.13s/it]
 28%|██▊       | 63146/225000 [129:34:08<316:43:28,  7.04s/it]
 28%|██▊       | 63147/225000 [129:34:12<274:00:27,  6.09s/it]
 28%|██▊       | 63148/225000 [129:34:23<335:17:37,  7.46s/it]
 28%|██▊       | 63149/225000 [129:34:28<308:36:39,  6.86s/it]
 28%|██▊       | 63150/225000 [129:34:34<294:13:18,  6.54s/it]
                                                              
{'loss': '0.7428', 'grad_norm': '0.8042', 'learning_rate': '4.284e-05', 'epoch': '0.2807', 'num_input_tokens_seen': 215472784, 'train_runtime': '4.665e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63150/225000 [129:34:34<294:13:18,  6.54s/it]
 28%|██▊       | 63151/225000 [129:34:39<276:15:17,  6.14s/it]
 28%|██▊       | 63152/225000 [129:34:44<259:59:44,  5.78s/it]
 28%|██▊       | 63153/225000 [129:34:50<252:44:48,  5.62s/it]
 28%|██▊       | 63154/225000 [129:34:55<250:04:25,  5.56s/it]
 28%|██▊       | 63155/225000 [129:35:00<240:34:39,  5.35s/it]
 28%|██▊       | 63156/225000 [129:35:07<258:58:03,  5.76s/it]
 28%|██▊       | 63157/225000 [129:35:19<346:05:38,  7.70s/it]
 28%|██▊       | 63158/225000 [129:35:24<310:54:43,  6.92s/it]
 28%|██▊       | 63159/225000 [129:35:29<288:59:11,  6.43s/it]
 28%|██▊       | 63160/225000 [129:35:37<312:18:20,  6.95s/it]
 28%|██▊       | 63161/225000 [129:35:42<284:40:12,  6.33s/it]
 28%|██▊       | 63162/225000 [129:35:46<255:08:27,  5.68s/it]
 28%|██▊       | 63163/225000 [129:35:58<334:56:23,  7.45s/it]
 28%|██▊       | 63164/225000 [129:36:03<302:32:43,  6.73s/it]
 28%|██▊       | 63165/225000 [129:36:10<299:12:08,  6.66s/it]
 28%|██▊       | 63166/225000 [129:36:18<322:34:44,  7.18s/it]
 28%|██▊       | 63167/225000 [129:36:22<285:45:43,  6.36s/it]
 28%|██▊       | 63168/225000 [129:36:27<260:29:23,  5.79s/it]
 28%|██▊       | 63169/225000 [129:36:36<300:19:25,  6.68s/it]
 28%|██▊       | 63170/225000 [129:36:44<322:54:14,  7.18s/it]
 28%|██▊       | 63171/225000 [129:36:50<310:03:09,  6.90s/it]
 28%|██▊       | 63172/225000 [129:36:54<264:23:57,  5.88s/it]
 28%|██▊       | 63173/225000 [129:37:03<309:52:16,  6.89s/it]
 28%|██▊       | 63174/225000 [129:37:07<278:19:53,  6.19s/it]
 28%|██▊       | 63175/225000 [129:37:12<259:29:22,  5.77s/it]
 28%|██▊       | 63176/225000 [129:37:17<245:20:16,  5.46s/it]
 28%|██▊       | 63177/225000 [129:37:21<227:02:05,  5.05s/it]
 28%|██▊       | 63178/225000 [129:37:34<330:50:40,  7.36s/it]
 28%|██▊       | 63179/225000 [129:37:41<334:31:07,  7.44s/it]
 28%|██▊       | 63180/225000 [129:37:46<300:58:31,  6.70s/it]
 28%|██▊       | 63181/225000 [129:37:52<281:31:28,  6.26s/it]
 28%|██▊       | 63182/225000 [129:37:57<266:42:36,  5.93s/it]
 28%|██▊       | 63183/225000 [129:38:02<260:35:18,  5.80s/it]
 28%|██▊       | 63184/225000 [129:38:16<372:13:49,  8.28s/it]
 28%|██▊       | 63185/225000 [129:38:21<317:44:07,  7.07s/it]
 28%|██▊       | 63186/225000 [129:38:25<284:57:34,  6.34s/it]
 28%|██▊       | 63187/225000 [129:38:32<288:40:18,  6.42s/it]
 28%|██▊       | 63188/225000 [129:38:42<343:07:33,  7.63s/it]
 28%|██▊       | 63189/225000 [129:38:46<288:28:56,  6.42s/it]
 28%|██▊       | 63190/225000 [129:38:55<323:53:20,  7.21s/it]
 28%|██▊       | 63191/225000 [129:39:00<298:48:06,  6.65s/it]
 28%|██▊       | 63192/225000 [129:39:08<307:03:46,  6.83s/it]
 28%|██▊       | 63193/225000 [129:39:13<285:23:27,  6.35s/it]
 28%|██▊       | 63194/225000 [129:39:21<311:49:34,  6.94s/it]
 28%|██▊       | 63195/225000 [129:39:28<310:58:21,  6.92s/it]
 28%|██▊       | 63196/225000 [129:39:32<276:09:31,  6.14s/it]
 28%|██▊       | 63197/225000 [129:39:37<250:58:16,  5.58s/it]
 28%|██▊       | 63198/225000 [129:39:44<271:14:18,  6.03s/it]
 28%|██▊       | 63199/225000 [129:39:54<323:57:36,  7.21s/it]
 28%|██▊       | 63200/225000 [129:39:58<289:43:44,  6.45s/it]
                                                              
{'loss': '0.7736', 'grad_norm': '0.9074', 'learning_rate': '4.283e-05', 'epoch': '0.2809', 'num_input_tokens_seen': 215618912, 'train_runtime': '4.668e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63200/225000 [129:39:58<289:43:44,  6.45s/it]
 28%|██▊       | 63201/225000 [129:40:03<265:20:09,  5.90s/it]
 28%|██▊       | 63202/225000 [129:40:09<270:56:00,  6.03s/it]
 28%|██▊       | 63203/225000 [129:40:19<325:33:20,  7.24s/it]
 28%|██▊       | 63204/225000 [129:40:23<272:22:48,  6.06s/it]
 28%|██▊       | 63205/225000 [129:40:33<323:58:48,  7.21s/it]
 28%|██▊       | 63206/225000 [129:40:37<281:45:16,  6.27s/it]
 28%|██▊       | 63207/225000 [129:40:41<250:40:54,  5.58s/it]
 28%|██▊       | 63208/225000 [129:40:50<300:53:47,  6.70s/it]
 28%|██▊       | 63209/225000 [129:40:58<319:39:58,  7.11s/it]
 28%|██▊       | 63210/225000 [129:41:02<276:50:53,  6.16s/it]
 28%|██▊       | 63211/225000 [129:41:13<340:46:20,  7.58s/it]
 28%|██▊       | 63212/225000 [129:41:17<288:10:21,  6.41s/it]
 28%|██▊       | 63213/225000 [129:41:21<257:34:49,  5.73s/it]
 28%|██▊       | 63214/225000 [129:41:32<328:52:04,  7.32s/it]
 28%|██▊       | 63215/225000 [129:41:36<282:08:54,  6.28s/it]
 28%|██▊       | 63216/225000 [129:41:41<266:03:28,  5.92s/it]
 28%|██▊       | 63217/225000 [129:41:46<265:10:41,  5.90s/it]
 28%|██▊       | 63218/225000 [129:41:51<253:11:08,  5.63s/it]
 28%|██▊       | 63219/225000 [129:41:57<250:40:07,  5.58s/it]
 28%|██▊       | 63220/225000 [129:42:03<256:00:14,  5.70s/it]
 28%|██▊       | 63221/225000 [129:42:16<358:04:28,  7.97s/it]
 28%|██▊       | 63222/225000 [129:42:23<340:12:48,  7.57s/it]
 28%|██▊       | 63223/225000 [129:42:27<299:18:57,  6.66s/it]
 28%|██▊       | 63224/225000 [129:42:32<267:18:12,  5.95s/it]
 28%|██▊       | 63225/225000 [129:42:42<324:02:10,  7.21s/it]
 28%|██▊       | 63226/225000 [129:42:50<342:17:09,  7.62s/it]
 28%|██▊       | 63227/225000 [129:42:57<328:11:50,  7.30s/it]
 28%|██▊       | 63228/225000 [129:43:03<305:49:44,  6.81s/it]
 28%|██▊       | 63229/225000 [129:43:07<267:38:36,  5.96s/it]
 28%|██▊       | 63230/225000 [129:43:17<323:08:29,  7.19s/it]
 28%|██▊       | 63231/225000 [129:43:22<297:49:57,  6.63s/it]
 28%|██▊       | 63232/225000 [129:43:29<305:50:50,  6.81s/it]
 28%|██▊       | 63233/225000 [129:43:36<300:19:47,  6.68s/it]
 28%|██▊       | 63234/225000 [129:43:39<263:22:44,  5.86s/it]
 28%|██▊       | 63235/225000 [129:43:50<324:11:26,  7.21s/it]
 28%|██▊       | 63236/225000 [129:43:56<308:08:30,  6.86s/it]
 28%|██▊       | 63237/225000 [129:44:00<269:06:08,  5.99s/it]
 28%|██▊       | 63238/225000 [129:44:07<289:44:46,  6.45s/it]
 28%|██▊       | 63239/225000 [129:44:15<307:43:55,  6.85s/it]
 28%|██▊       | 63240/225000 [129:44:19<267:49:26,  5.96s/it]
 28%|██▊       | 63241/225000 [129:44:24<247:59:22,  5.52s/it]
 28%|██▊       | 63242/225000 [129:44:35<328:02:22,  7.30s/it]
 28%|██▊       | 63243/225000 [129:44:38<276:11:46,  6.15s/it]
 28%|██▊       | 63244/225000 [129:44:44<266:53:00,  5.94s/it]
 28%|██▊       | 63245/225000 [129:44:51<285:29:25,  6.35s/it]
 28%|██▊       | 63246/225000 [129:44:58<297:48:46,  6.63s/it]
 28%|██▊       | 63247/225000 [129:45:03<273:42:59,  6.09s/it]
 28%|██▊       | 63248/225000 [129:45:08<261:16:12,  5.81s/it]
 28%|██▊       | 63249/225000 [129:45:14<251:08:32,  5.59s/it]
 28%|██▊       | 63250/225000 [129:45:20<256:49:44,  5.72s/it]
                                                              
{'loss': '0.7522', 'grad_norm': '0.8851', 'learning_rate': '4.281e-05', 'epoch': '0.2811', 'num_input_tokens_seen': 215764200, 'train_runtime': '4.671e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63250/225000 [129:45:20<256:49:44,  5.72s/it]
 28%|██▊       | 63251/225000 [129:45:24<242:02:45,  5.39s/it]
 28%|██▊       | 63252/225000 [129:45:37<346:42:08,  7.72s/it]
 28%|██▊       | 63253/225000 [129:45:43<320:07:53,  7.13s/it]
 28%|██▊       | 63254/225000 [129:45:53<353:20:45,  7.86s/it]
 28%|██▊       | 63255/225000 [129:45:59<329:05:14,  7.32s/it]
 28%|██▊       | 63256/225000 [129:46:04<299:54:38,  6.68s/it]
 28%|██▊       | 63257/225000 [129:46:12<318:48:19,  7.10s/it]
 28%|██▊       | 63258/225000 [129:46:16<281:46:31,  6.27s/it]
 28%|██▊       | 63259/225000 [129:46:21<253:56:00,  5.65s/it]
 28%|██▊       | 63260/225000 [129:46:32<332:55:26,  7.41s/it]
 28%|██▊       | 63261/225000 [129:46:36<290:42:25,  6.47s/it]
 28%|██▊       | 63262/225000 [129:46:42<276:48:56,  6.16s/it]
 28%|██▊       | 63263/225000 [129:46:47<265:35:34,  5.91s/it]
 28%|██▊       | 63264/225000 [129:46:53<262:26:12,  5.84s/it]
 28%|██▊       | 63265/225000 [129:47:03<323:30:47,  7.20s/it]
 28%|██▊       | 63266/225000 [129:47:08<290:45:45,  6.47s/it]
 28%|██▊       | 63267/225000 [129:47:12<258:05:02,  5.74s/it]
 28%|██▊       | 63268/225000 [129:47:19<274:30:41,  6.11s/it]
 28%|██▊       | 63269/225000 [129:47:29<330:25:54,  7.36s/it]
 28%|██▊       | 63270/225000 [129:47:34<294:14:41,  6.55s/it]
 28%|██▊       | 63271/225000 [129:47:39<277:21:19,  6.17s/it]
 28%|██▊       | 63272/225000 [129:47:44<260:39:17,  5.80s/it]
 28%|██▊       | 63273/225000 [129:47:49<249:39:35,  5.56s/it]
 28%|██▊       | 63274/225000 [129:47:54<245:19:07,  5.46s/it]
 28%|██▊       | 63275/225000 [129:47:59<234:49:15,  5.23s/it]
 28%|██▊       | 63276/225000 [129:48:09<305:49:55,  6.81s/it]
 28%|██▊       | 63277/225000 [129:48:18<334:21:40,  7.44s/it]
 28%|██▊       | 63278/225000 [129:48:23<289:33:47,  6.45s/it]
 28%|██▊       | 63279/225000 [129:48:27<260:01:16,  5.79s/it]
 28%|██▊       | 63280/225000 [129:48:37<318:57:46,  7.10s/it]
 28%|██▊       | 63281/225000 [129:48:44<319:21:25,  7.11s/it]
 28%|██▊       | 63282/225000 [129:48:49<283:17:01,  6.31s/it]
 28%|██▊       | 63283/225000 [129:48:54<267:34:08,  5.96s/it]
 28%|██▊       | 63284/225000 [129:49:00<273:43:57,  6.09s/it]
 28%|██▊       | 63285/225000 [129:49:11<333:11:35,  7.42s/it]
 28%|██▊       | 63286/225000 [129:49:15<295:27:11,  6.58s/it]
 28%|██▊       | 63287/225000 [129:49:20<269:38:13,  6.00s/it]
 28%|██▊       | 63288/225000 [129:49:29<310:24:18,  6.91s/it]
 28%|██▊       | 63289/225000 [129:49:38<338:57:30,  7.55s/it]
 28%|██▊       | 63290/225000 [129:49:44<318:36:29,  7.09s/it]
 28%|██▊       | 63291/225000 [129:49:50<309:14:36,  6.88s/it]
 28%|██▊       | 63292/225000 [129:49:56<288:24:20,  6.42s/it]
 28%|██▊       | 63293/225000 [129:50:01<276:12:40,  6.15s/it]
 28%|██▊       | 63294/225000 [129:50:07<269:42:32,  6.00s/it]
 28%|██▊       | 63295/225000 [129:50:12<260:16:28,  5.79s/it]
 28%|██▊       | 63296/225000 [129:50:18<261:37:09,  5.82s/it]
 28%|██▊       | 63297/225000 [129:50:24<258:18:20,  5.75s/it]
 28%|██▊       | 63298/225000 [129:50:29<253:04:37,  5.63s/it]
 28%|██▊       | 63299/225000 [129:50:34<250:14:47,  5.57s/it]
 28%|██▊       | 63300/225000 [129:50:48<354:09:02,  7.88s/it]
                                                              
{'loss': '0.7688', 'grad_norm': '0.7691', 'learning_rate': '4.28e-05', 'epoch': '0.2813', 'num_input_tokens_seen': 215910200, 'train_runtime': '4.674e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63300/225000 [129:50:48<354:09:02,  7.88s/it]
 28%|██▊       | 63301/225000 [129:50:54<339:20:21,  7.55s/it]
 28%|██▊       | 63302/225000 [129:50:59<303:45:54,  6.76s/it]
 28%|██▊       | 63303/225000 [129:51:08<325:22:59,  7.24s/it]
 28%|██▊       | 63304/225000 [129:51:13<293:55:45,  6.54s/it]
 28%|██▊       | 63305/225000 [129:51:18<279:29:36,  6.22s/it]
 28%|██▊       | 63306/225000 [129:51:23<265:21:08,  5.91s/it]
 28%|██▊       | 63307/225000 [129:51:28<254:54:26,  5.68s/it]
 28%|██▊       | 63308/225000 [129:51:36<282:25:39,  6.29s/it]
 28%|██▊       | 63309/225000 [129:51:50<378:57:18,  8.44s/it]
 28%|██▊       | 63310/225000 [129:51:53<317:15:28,  7.06s/it]
 28%|██▊       | 63311/225000 [129:51:58<280:51:06,  6.25s/it]
 28%|██▊       | 63312/225000 [129:52:10<357:51:37,  7.97s/it]
 28%|██▊       | 63313/225000 [129:52:14<310:25:31,  6.91s/it]
 28%|██▊       | 63314/225000 [129:52:18<270:20:15,  6.02s/it]
 28%|██▊       | 63315/225000 [129:52:28<320:04:03,  7.13s/it]
 28%|██▊       | 63316/225000 [129:52:32<275:00:04,  6.12s/it]
 28%|██▊       | 63317/225000 [129:52:41<313:09:06,  6.97s/it]
 28%|██▊       | 63318/225000 [129:52:46<292:01:26,  6.50s/it]
 28%|██▊       | 63319/225000 [129:52:51<274:03:23,  6.10s/it]
 28%|██▊       | 63320/225000 [129:52:57<264:08:16,  5.88s/it]
 28%|██▊       | 63321/225000 [129:53:01<246:52:53,  5.50s/it]
 28%|██▊       | 63322/225000 [129:53:07<246:18:43,  5.48s/it]
 28%|██▊       | 63323/225000 [129:53:12<240:56:06,  5.36s/it]
 28%|██▊       | 63324/225000 [129:53:19<261:43:03,  5.83s/it]
 28%|██▊       | 63325/225000 [129:53:33<374:37:10,  8.34s/it]
 28%|██▊       | 63326/225000 [129:53:40<353:37:36,  7.87s/it]
 28%|██▊       | 63327/225000 [129:53:44<308:50:30,  6.88s/it]
 28%|██▊       | 63328/225000 [129:53:49<287:45:32,  6.41s/it]
 28%|██▊       | 63329/225000 [129:53:55<277:14:43,  6.17s/it]
 28%|██▊       | 63330/225000 [129:54:01<267:44:08,  5.96s/it]
 28%|██▊       | 63331/225000 [129:54:08<291:00:18,  6.48s/it]
 28%|██▊       | 63332/225000 [129:54:21<369:48:28,  8.23s/it]
 28%|██▊       | 63333/225000 [129:54:25<320:07:58,  7.13s/it]
 28%|██▊       | 63334/225000 [129:54:33<331:52:39,  7.39s/it]
 28%|██▊       | 63335/225000 [129:54:39<316:54:18,  7.06s/it]
 28%|██▊       | 63336/225000 [129:54:44<277:23:45,  6.18s/it]
 28%|██▊       | 63337/225000 [129:54:49<264:54:42,  5.90s/it]
 28%|██▊       | 63338/225000 [129:54:54<256:04:28,  5.70s/it]
 28%|██▊       | 63339/225000 [129:55:06<342:29:05,  7.63s/it]
 28%|██▊       | 63340/225000 [129:55:13<325:20:07,  7.24s/it]
 28%|██▊       | 63341/225000 [129:55:20<326:42:03,  7.28s/it]
 28%|██▊       | 63342/225000 [129:55:28<334:10:28,  7.44s/it]
 28%|██▊       | 63343/225000 [129:55:33<310:44:14,  6.92s/it]
 28%|██▊       | 63344/225000 [129:55:38<283:31:14,  6.31s/it]
 28%|██▊       | 63345/225000 [129:55:43<267:43:31,  5.96s/it]
 28%|██▊       | 63346/225000 [129:55:48<253:39:34,  5.65s/it]
 28%|██▊       | 63347/225000 [129:55:56<285:30:07,  6.36s/it]
 28%|██▊       | 63348/225000 [129:56:08<352:36:32,  7.85s/it]
 28%|██▊       | 63349/225000 [129:56:12<308:36:44,  6.87s/it]
 28%|██▊       | 63350/225000 [129:56:17<277:46:45,  6.19s/it]
                                                              
{'loss': '0.763', 'grad_norm': '0.7348', 'learning_rate': '4.279e-05', 'epoch': '0.2816', 'num_input_tokens_seen': 216057024, 'train_runtime': '4.678e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63350/225000 [129:56:17<277:46:45,  6.19s/it]
 28%|██▊       | 63351/225000 [129:56:22<260:28:20,  5.80s/it]
 28%|██▊       | 63352/225000 [129:56:29<285:42:11,  6.36s/it]
 28%|██▊       | 63353/225000 [129:56:40<347:18:23,  7.73s/it]
 28%|██▊       | 63354/225000 [129:56:45<299:09:29,  6.66s/it]
 28%|██▊       | 63355/225000 [129:56:53<316:55:03,  7.06s/it]
 28%|██▊       | 63356/225000 [129:57:01<339:19:46,  7.56s/it]
 28%|██▊       | 63357/225000 [129:57:06<295:45:07,  6.59s/it]
 28%|██▊       | 63358/225000 [129:57:10<261:06:04,  5.82s/it]
 28%|██▊       | 63359/225000 [129:57:21<336:04:46,  7.49s/it]
 28%|██▊       | 63360/225000 [129:57:24<282:19:58,  6.29s/it]
 28%|██▊       | 63361/225000 [129:57:33<310:45:12,  6.92s/it]
 28%|██▊       | 63362/225000 [129:57:38<282:42:06,  6.30s/it]
 28%|██▊       | 63363/225000 [129:57:43<271:46:40,  6.05s/it]
 28%|██▊       | 63364/225000 [129:57:48<254:07:41,  5.66s/it]
 28%|██▊       | 63365/225000 [129:57:53<251:38:09,  5.60s/it]
 28%|██▊       | 63366/225000 [129:57:59<253:54:00,  5.66s/it]
 28%|██▊       | 63367/225000 [129:58:05<252:20:44,  5.62s/it]
 28%|██▊       | 63368/225000 [129:58:10<249:31:04,  5.56s/it]
 28%|██▊       | 63369/225000 [129:58:23<349:38:05,  7.79s/it]
 28%|██▊       | 63370/225000 [129:58:29<317:39:01,  7.08s/it]
 28%|██▊       | 63371/225000 [129:58:36<329:17:40,  7.33s/it]
 28%|██▊       | 63372/225000 [129:58:44<327:10:13,  7.29s/it]
 28%|██▊       | 63373/225000 [129:58:48<281:20:25,  6.27s/it]
 28%|██▊       | 63374/225000 [129:58:53<264:22:02,  5.89s/it]
 28%|██▊       | 63375/225000 [129:59:04<340:48:54,  7.59s/it]
 28%|██▊       | 63376/225000 [129:59:08<287:25:42,  6.40s/it]
 28%|██▊       | 63377/225000 [129:59:14<283:09:43,  6.31s/it]
 28%|██▊       | 63378/225000 [129:59:19<271:08:13,  6.04s/it]
 28%|██▊       | 63379/225000 [129:59:24<260:12:45,  5.80s/it]
 28%|██▊       | 63380/225000 [129:59:36<333:23:23,  7.43s/it]
 28%|██▊       | 63381/225000 [129:59:40<294:09:09,  6.55s/it]
 28%|██▊       | 63382/225000 [129:59:45<273:41:17,  6.10s/it]
 28%|██▊       | 63383/225000 [129:59:52<278:53:54,  6.21s/it]
 28%|██▊       | 63384/225000 [130:00:02<333:12:23,  7.42s/it]
 28%|██▊       | 63385/225000 [130:00:06<290:02:06,  6.46s/it]
 28%|██▊       | 63386/225000 [130:00:11<271:48:48,  6.05s/it]
 28%|██▊       | 63387/225000 [130:00:20<312:51:07,  6.97s/it]
 28%|██▊       | 63388/225000 [130:00:25<280:51:09,  6.26s/it]
 28%|██▊       | 63389/225000 [130:00:33<300:06:39,  6.69s/it]
 28%|██▊       | 63390/225000 [130:00:40<303:08:03,  6.75s/it]
 28%|██▊       | 63391/225000 [130:00:46<299:47:45,  6.68s/it]
 28%|██▊       | 63392/225000 [130:00:51<275:51:35,  6.15s/it]
 28%|██▊       | 63393/225000 [130:00:56<254:00:48,  5.66s/it]
 28%|██▊       | 63394/225000 [130:01:05<307:02:21,  6.84s/it]
 28%|██▊       | 63395/225000 [130:01:13<320:27:04,  7.14s/it]
 28%|██▊       | 63396/225000 [130:01:18<286:10:44,  6.38s/it]
 28%|██▊       | 63397/225000 [130:01:22<266:22:35,  5.93s/it]
 28%|██▊       | 63398/225000 [130:01:29<279:14:40,  6.22s/it]
 28%|██▊       | 63399/225000 [130:01:40<341:15:24,  7.60s/it]
 28%|██▊       | 63400/225000 [130:01:44<291:26:36,  6.49s/it]
                                                              
{'loss': '0.7618', 'grad_norm': '0.7808', 'learning_rate': '4.278e-05', 'epoch': '0.2818', 'num_input_tokens_seen': 216203248, 'train_runtime': '4.681e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63400/225000 [130:01:44<291:26:36,  6.49s/it]
 28%|██▊       | 63401/225000 [130:01:52<304:21:28,  6.78s/it]
 28%|██▊       | 63402/225000 [130:01:59<314:08:22,  7.00s/it]
 28%|██▊       | 63403/225000 [130:02:03<273:20:33,  6.09s/it]
 28%|██▊       | 63404/225000 [130:02:08<252:00:36,  5.61s/it]
 28%|██▊       | 63405/225000 [130:02:14<268:47:34,  5.99s/it]
 28%|██▊       | 63406/225000 [130:02:26<344:57:30,  7.69s/it]
 28%|██▊       | 63407/225000 [130:02:30<296:49:26,  6.61s/it]
 28%|██▊       | 63408/225000 [130:02:35<267:00:44,  5.95s/it]
 28%|██▊       | 63409/225000 [130:02:44<313:40:25,  6.99s/it]
 28%|██▊       | 63410/225000 [130:02:50<296:17:39,  6.60s/it]
 28%|██▊       | 63411/225000 [130:02:56<297:39:41,  6.63s/it]
 28%|██▊       | 63412/225000 [130:03:00<263:01:31,  5.86s/it]
 28%|██▊       | 63413/225000 [130:03:06<253:34:25,  5.65s/it]
 28%|██▊       | 63414/225000 [130:03:15<304:30:33,  6.78s/it]
 28%|██▊       | 63415/225000 [130:03:23<316:29:48,  7.05s/it]
 28%|██▊       | 63416/225000 [130:03:31<339:44:51,  7.57s/it]
 28%|██▊       | 63417/225000 [130:03:35<287:30:58,  6.41s/it]
 28%|██▊       | 63418/225000 [130:03:43<307:11:51,  6.84s/it]
 28%|██▊       | 63419/225000 [130:03:47<270:58:55,  6.04s/it]
 28%|██▊       | 63420/225000 [130:03:51<246:52:19,  5.50s/it]
 28%|██▊       | 63421/225000 [130:04:00<293:56:45,  6.55s/it]
 28%|██▊       | 63422/225000 [130:04:08<310:32:31,  6.92s/it]
 28%|██▊       | 63423/225000 [130:04:13<275:43:28,  6.14s/it]
 28%|██▊       | 63424/225000 [130:04:20<299:35:46,  6.68s/it]
 28%|██▊       | 63425/225000 [130:04:25<270:15:26,  6.02s/it]
 28%|██▊       | 63426/225000 [130:04:29<249:27:19,  5.56s/it]
 28%|██▊       | 63427/225000 [130:04:37<276:18:22,  6.16s/it]
 28%|██▊       | 63428/225000 [130:04:47<332:26:10,  7.41s/it]
 28%|██▊       | 63429/225000 [130:04:53<304:12:48,  6.78s/it]
 28%|██▊       | 63430/225000 [130:04:58<289:40:00,  6.45s/it]
 28%|██▊       | 63431/225000 [130:05:04<284:14:15,  6.33s/it]
 28%|██▊       | 63432/225000 [130:05:10<273:43:12,  6.10s/it]
 28%|██▊       | 63433/225000 [130:05:16<272:49:21,  6.08s/it]
 28%|██▊       | 63434/225000 [130:05:22<268:22:08,  5.98s/it]
 28%|██▊       | 63435/225000 [130:05:27<261:50:08,  5.83s/it]
 28%|██▊       | 63436/225000 [130:05:32<253:36:13,  5.65s/it]
 28%|██▊       | 63437/225000 [130:05:40<278:51:58,  6.21s/it]
 28%|██▊       | 63438/225000 [130:05:53<370:33:36,  8.26s/it]
 28%|██▊       | 63439/225000 [130:05:56<304:15:46,  6.78s/it]
 28%|██▊       | 63440/225000 [130:06:05<330:01:06,  7.35s/it]
 28%|██▊       | 63441/225000 [130:06:10<292:09:04,  6.51s/it]
 28%|██▊       | 63442/225000 [130:06:16<292:52:13,  6.53s/it]
 28%|██▊       | 63443/225000 [130:06:20<257:39:18,  5.74s/it]
 28%|██▊       | 63444/225000 [130:06:25<248:46:27,  5.54s/it]
 28%|██▊       | 63445/225000 [130:06:31<249:19:23,  5.56s/it]
 28%|██▊       | 63446/225000 [130:06:44<349:31:00,  7.79s/it]
 28%|██▊       | 63447/225000 [130:06:47<294:44:34,  6.57s/it]
 28%|██▊       | 63448/225000 [130:06:51<260:51:06,  5.81s/it]
 28%|██▊       | 63449/225000 [130:07:02<324:38:34,  7.23s/it]
 28%|██▊       | 63450/225000 [130:07:07<298:02:46,  6.64s/it]
                                                              
{'loss': '0.7542', 'grad_norm': '0.7355', 'learning_rate': '4.276e-05', 'epoch': '0.282', 'num_input_tokens_seen': 216348704, 'train_runtime': '4.684e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63450/225000 [130:07:07<298:02:46,  6.64s/it]
 28%|██▊       | 63451/225000 [130:07:13<288:54:13,  6.44s/it]
 28%|██▊       | 63452/225000 [130:07:20<297:35:30,  6.63s/it]
 28%|██▊       | 63453/225000 [130:07:26<289:51:35,  6.46s/it]
 28%|██▊       | 63454/225000 [130:07:32<279:38:24,  6.23s/it]
 28%|██▊       | 63455/225000 [130:07:41<322:26:16,  7.19s/it]
 28%|██▊       | 63456/225000 [130:07:48<315:15:34,  7.03s/it]
 28%|██▊       | 63457/225000 [130:07:56<323:29:57,  7.21s/it]
 28%|██▊       | 63458/225000 [130:08:02<312:35:06,  6.97s/it]
 28%|██▊       | 63459/225000 [130:08:08<301:10:59,  6.71s/it]
 28%|██▊       | 63460/225000 [130:08:12<263:52:45,  5.88s/it]
 28%|██▊       | 63461/225000 [130:08:21<306:04:18,  6.82s/it]
 28%|██▊       | 63462/225000 [130:08:25<259:35:21,  5.79s/it]
 28%|██▊       | 63463/225000 [130:08:30<259:23:02,  5.78s/it]
 28%|██▊       | 63464/225000 [130:08:36<259:31:19,  5.78s/it]
 28%|██▊       | 63465/225000 [130:08:42<256:53:14,  5.73s/it]
 28%|██▊       | 63466/225000 [130:08:47<256:39:10,  5.72s/it]
 28%|██▊       | 63467/225000 [130:08:57<306:56:38,  6.84s/it]
 28%|██▊       | 63468/225000 [130:09:06<338:44:51,  7.55s/it]
 28%|██▊       | 63469/225000 [130:09:13<330:11:29,  7.36s/it]
 28%|██▊       | 63470/225000 [130:09:21<344:27:04,  7.68s/it]
 28%|██▊       | 63471/225000 [130:09:28<328:20:59,  7.32s/it]
 28%|██▊       | 63472/225000 [130:09:33<297:40:45,  6.63s/it]
 28%|██▊       | 63473/225000 [130:09:38<280:09:10,  6.24s/it]
 28%|██▊       | 63474/225000 [130:09:44<270:26:41,  6.03s/it]
 28%|██▊       | 63475/225000 [130:09:48<247:53:00,  5.52s/it]
 28%|██▊       | 63476/225000 [130:09:54<245:57:30,  5.48s/it]
 28%|██▊       | 63477/225000 [130:10:03<294:11:38,  6.56s/it]
 28%|██▊       | 63478/225000 [130:10:14<359:20:24,  8.01s/it]
 28%|██▊       | 63479/225000 [130:10:19<317:28:50,  7.08s/it]
 28%|██▊       | 63480/225000 [130:10:24<297:17:07,  6.63s/it]
 28%|██▊       | 63481/225000 [130:10:31<290:30:01,  6.47s/it]
 28%|██▊       | 63482/225000 [130:10:36<277:50:16,  6.19s/it]
 28%|██▊       | 63483/225000 [130:10:42<272:57:11,  6.08s/it]
 28%|██▊       | 63484/225000 [130:10:48<269:06:46,  6.00s/it]
 28%|██▊       | 63485/225000 [130:10:59<337:49:18,  7.53s/it]
 28%|██▊       | 63486/225000 [130:11:04<304:50:12,  6.79s/it]
 28%|██▊       | 63487/225000 [130:11:13<340:33:38,  7.59s/it]
 28%|██▊       | 63488/225000 [130:11:17<293:42:29,  6.55s/it]
 28%|██▊       | 63489/225000 [130:11:22<270:25:09,  6.03s/it]
 28%|██▊       | 63490/225000 [130:11:28<261:08:10,  5.82s/it]
 28%|██▊       | 63491/225000 [130:11:33<249:35:28,  5.56s/it]
 28%|██▊       | 63492/225000 [130:11:45<346:43:07,  7.73s/it]
 28%|██▊       | 63493/225000 [130:11:51<323:08:01,  7.20s/it]
 28%|██▊       | 63494/225000 [130:11:56<293:36:12,  6.54s/it]
 28%|██▊       | 63495/225000 [130:12:02<281:53:41,  6.28s/it]
 28%|██▊       | 63496/225000 [130:12:07<269:15:24,  6.00s/it]
 28%|██▊       | 63497/225000 [130:12:13<264:08:14,  5.89s/it]
 28%|██▊       | 63498/225000 [130:12:19<260:27:12,  5.81s/it]
 28%|██▊       | 63499/225000 [130:12:24<252:56:54,  5.64s/it]
 28%|██▊       | 63500/225000 [130:12:37<351:22:42,  7.83s/it]
                                                              
{'loss': '0.7432', 'grad_norm': '0.6824', 'learning_rate': '4.275e-05', 'epoch': '0.2822', 'num_input_tokens_seen': 216496520, 'train_runtime': '4.688e+05', 'train_tokens_per_second': '461.9'}

 28%|██▊       | 63500/225000 [130:12:37<351:22:42,  7.83s/it]
 28%|██▊       | 63501/225000 [130:12:42<316:44:59,  7.06s/it]
 28%|██▊       | 63502/225000 [130:12:49<315:48:53,  7.04s/it]
 28%|██▊       | 63503/225000 [130:12:56<315:02:07,  7.02s/it]
 28%|██▊       | 63504/225000 [130:13:03<316:47:36,  7.06s/it]
 28%|██▊       | 63505/225000 [130:13:08<285:06:25,  6.36s/it]
 28%|██▊       | 63506/225000 [130:13:17<321:14:15,  7.16s/it]
 28%|██▊       | 63507/225000 [130:13:21<273:52:06,  6.11s/it]
 28%|██▊       | 63508/225000 [130:13:27<282:37:48,  6.30s/it]
 28%|██▊       | 63509/225000 [130:13:32<254:58:37,  5.68s/it]
 28%|██▊       | 63510/225000 [130:13:41<298:30:32,  6.65s/it]
 28%|██▊       | 63511/225000 [130:13:47<291:34:29,  6.50s/it]
 28%|██▊       | 63512/225000 [130:13:53<291:08:11,  6.49s/it]
 28%|██▊       | 63513/225000 [130:13:57<261:24:41,  5.83s/it]
 28%|██▊       | 63514/225000 [130:14:02<246:37:47,  5.50s/it]
 28%|██▊       | 63515/225000 [130:14:08<248:36:41,  5.54s/it]
 28%|██▊       | 63516/225000 [130:14:21<357:19:41,  7.97s/it]
 28%|██▊       | 63517/225000 [130:14:25<303:50:10,  6.77s/it]
 28%|██▊       | 63518/225000 [130:14:29<260:43:31,  5.81s/it]
 28%|██▊       | 63519/225000 [130:14:38<300:01:31,  6.69s/it]
 28%|██▊       | 63520/225000 [130:14:42<274:01:58,  6.11s/it]
 28%|██▊       | 63521/225000 [130:14:49<279:06:44,  6.22s/it]
 28%|██▊       | 63522/225000 [130:14:57<302:58:36,  6.75s/it]
 28%|██▊       | 63523/225000 [130:15:04<303:46:13,  6.77s/it]
 28%|██▊       | 63524/225000 [130:15:08<268:07:17,  5.98s/it]
 28%|██▊       | 63525/225000 [130:15:18<325:31:29,  7.26s/it]
 28%|██▊       | 63526/225000 [130:15:23<296:28:48,  6.61s/it]
 28%|██▊       | 63527/225000 [130:15:27<265:01:32,  5.91s/it]
 28%|██▊       | 63528/225000 [130:15:33<264:41:36,  5.90s/it]
 28%|██▊       | 63529/225000 [130:15:38<251:31:06,  5.61s/it]
 28%|██▊       | 63530/225000 [130:15:46<282:42:10,  6.30s/it]
 28%|██▊       | 63531/225000 [130:15:57<340:39:42,  7.60s/it]
 28%|██▊       | 63532/225000 [130:16:05<341:40:03,  7.62s/it]
 28%|██▊       | 63533/225000 [130:16:08<289:42:56,  6.46s/it]
 28%|██▊       | 63534/225000 [130:16:13<272:54:20,  6.08s/it]
 28%|██▊       | 63535/225000 [130:16:19<269:46:05,  6.01s/it]
 28%|██▊       | 63536/225000 [130:16:29<313:14:09,  6.98s/it]
 28%|██▊       | 63537/225000 [130:16:36<318:58:46,  7.11s/it]
 28%|██▊       | 63538/225000 [130:16:44<336:13:09,  7.50s/it]
 28%|██▊       | 63539/225000 [130:16:49<292:21:26,  6.52s/it]
 28%|██▊       | 63540/225000 [130:16:54<277:35:37,  6.19s/it]
 28%|██▊       | 63541/225000 [130:17:00<272:19:35,  6.07s/it]
 28%|██▊       | 63542/225000 [130:17:05<266:07:07,  5.93s/it]
 28%|██▊       | 63543/225000 [130:17:10<249:11:56,  5.56s/it]
 28%|██▊       | 63544/225000 [130:17:16<251:53:43,  5.62s/it]
 28%|██▊       | 63545/225000 [130:17:22<257:02:31,  5.73s/it]
 28%|██▊       | 63546/225000 [130:17:35<354:34:33,  7.91s/it]
 28%|██▊       | 63547/225000 [130:17:42<342:23:43,  7.63s/it]
 28%|██▊       | 63548/225000 [130:17:46<294:47:20,  6.57s/it]
 28%|██▊       | 63549/225000 [130:17:50<266:46:41,  5.95s/it]
 28%|██▊       | 63550/225000 [130:18:03<353:19:04,  7.88s/it]
                                                              
{'loss': '0.7529', 'grad_norm': '0.8544', 'learning_rate': '4.274e-05', 'epoch': '0.2824', 'num_input_tokens_seen': 216642752, 'train_runtime': '4.691e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63550/225000 [130:18:03<353:19:04,  7.88s/it]
 28%|██▊       | 63551/225000 [130:18:07<300:54:56,  6.71s/it]
 28%|██▊       | 63552/225000 [130:18:12<279:34:30,  6.23s/it]
 28%|██▊       | 63553/225000 [130:18:23<338:14:40,  7.54s/it]
 28%|██▊       | 63554/225000 [130:18:27<293:56:40,  6.55s/it]
 28%|██▊       | 63555/225000 [130:18:31<265:21:57,  5.92s/it]
 28%|██▊       | 63556/225000 [130:18:39<286:42:19,  6.39s/it]
 28%|██▊       | 63557/225000 [130:18:50<352:05:48,  7.85s/it]
 28%|██▊       | 63558/225000 [130:18:54<298:55:48,  6.67s/it]
 28%|██▊       | 63559/225000 [130:18:58<268:44:33,  5.99s/it]
 28%|██▊       | 63560/225000 [130:19:04<258:37:23,  5.77s/it]
 28%|██▊       | 63561/225000 [130:19:15<332:19:07,  7.41s/it]
 28%|██▊       | 63562/225000 [130:19:20<303:43:13,  6.77s/it]
 28%|██▊       | 63563/225000 [130:19:25<278:33:35,  6.21s/it]
 28%|██▊       | 63564/225000 [130:19:31<270:17:13,  6.03s/it]
 28%|██▊       | 63565/225000 [130:19:36<264:15:16,  5.89s/it]
 28%|██▊       | 63566/225000 [130:19:42<260:14:59,  5.80s/it]
 28%|██▊       | 63567/225000 [130:19:47<256:07:38,  5.71s/it]
 28%|██▊       | 63568/225000 [130:19:53<252:37:58,  5.63s/it]
 28%|██▊       | 63569/225000 [130:20:05<348:27:20,  7.77s/it]
 28%|██▊       | 63570/225000 [130:20:13<348:35:18,  7.77s/it]
 28%|██▊       | 63571/225000 [130:20:17<293:27:14,  6.54s/it]
 28%|██▊       | 63572/225000 [130:20:26<331:08:18,  7.38s/it]
 28%|██▊       | 63573/225000 [130:20:31<290:46:41,  6.48s/it]
 28%|██▊       | 63574/225000 [130:20:38<296:14:00,  6.61s/it]
 28%|██▊       | 63575/225000 [130:20:43<274:36:53,  6.12s/it]
 28%|██▊       | 63576/225000 [130:20:48<265:58:45,  5.93s/it]
 28%|██▊       | 63577/225000 [130:20:53<257:10:35,  5.74s/it]
 28%|██▊       | 63578/225000 [130:20:59<251:10:03,  5.60s/it]
 28%|██▊       | 63579/225000 [130:21:04<245:20:29,  5.47s/it]
 28%|██▊       | 63580/225000 [130:21:08<230:31:26,  5.14s/it]
 28%|██▊       | 63581/225000 [130:21:14<235:51:11,  5.26s/it]
 28%|██▊       | 63582/225000 [130:21:19<242:37:28,  5.41s/it]
 28%|██▊       | 63583/225000 [130:21:29<300:49:48,  6.71s/it]
 28%|██▊       | 63584/225000 [130:21:41<376:08:51,  8.39s/it]
 28%|██▊       | 63585/225000 [130:21:47<332:47:22,  7.42s/it]
 28%|██▊       | 63586/225000 [130:21:52<301:39:19,  6.73s/it]
 28%|██▊       | 63587/225000 [130:21:57<283:33:25,  6.32s/it]
 28%|██▊       | 63588/225000 [130:22:02<263:49:09,  5.88s/it]
 28%|██▊       | 63589/225000 [130:22:07<257:58:49,  5.75s/it]
 28%|██▊       | 63590/225000 [130:22:19<335:09:05,  7.48s/it]
 28%|██▊       | 63591/225000 [130:22:26<332:05:33,  7.41s/it]
 28%|██▊       | 63592/225000 [130:22:32<309:32:37,  6.90s/it]
 28%|██▊       | 63593/225000 [130:22:39<305:58:57,  6.82s/it]
 28%|██▊       | 63594/225000 [130:22:43<273:51:49,  6.11s/it]
 28%|██▊       | 63595/225000 [130:22:48<256:08:31,  5.71s/it]
 28%|██▊       | 63596/225000 [130:22:52<239:24:36,  5.34s/it]
 28%|██▊       | 63597/225000 [130:23:04<324:05:05,  7.23s/it]
 28%|██▊       | 63598/225000 [130:23:13<344:14:45,  7.68s/it]
 28%|██▊       | 63599/225000 [130:23:17<304:19:15,  6.79s/it]
 28%|██▊       | 63600/225000 [130:23:25<319:16:51,  7.12s/it]
                                                              
{'loss': '0.7563', 'grad_norm': '0.7057', 'learning_rate': '4.272e-05', 'epoch': '0.2827', 'num_input_tokens_seen': 216789560, 'train_runtime': '4.694e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63600/225000 [130:23:25<319:16:51,  7.12s/it]
 28%|██▊       | 63601/225000 [130:23:32<321:17:46,  7.17s/it]
 28%|██▊       | 63602/225000 [130:23:37<289:13:20,  6.45s/it]
 28%|██▊       | 63603/225000 [130:23:44<299:02:50,  6.67s/it]
 28%|██▊       | 63604/225000 [130:23:51<302:33:36,  6.75s/it]
 28%|██▊       | 63605/225000 [130:23:58<299:48:11,  6.69s/it]
 28%|██▊       | 63606/225000 [130:24:02<267:44:46,  5.97s/it]
 28%|██▊       | 63607/225000 [130:24:06<244:06:45,  5.45s/it]
 28%|██▊       | 63608/225000 [130:24:16<301:24:06,  6.72s/it]
 28%|██▊       | 63609/225000 [130:24:21<270:15:04,  6.03s/it]
 28%|██▊       | 63610/225000 [130:24:25<251:34:40,  5.61s/it]
 28%|██▊       | 63611/225000 [130:24:30<238:01:35,  5.31s/it]
 28%|██▊       | 63612/225000 [130:24:34<226:24:52,  5.05s/it]
 28%|██▊       | 63613/225000 [130:24:47<334:43:09,  7.47s/it]
 28%|██▊       | 63614/225000 [130:24:53<316:56:10,  7.07s/it]
 28%|██▊       | 63615/225000 [130:25:03<345:48:25,  7.71s/it]
 28%|██▊       | 63616/225000 [130:25:07<301:55:39,  6.74s/it]
 28%|██▊       | 63617/225000 [130:25:16<336:51:17,  7.51s/it]
 28%|██▊       | 63618/225000 [130:25:21<300:26:34,  6.70s/it]
 28%|██▊       | 63619/225000 [130:25:26<273:55:15,  6.11s/it]
 28%|██▊       | 63620/225000 [130:25:31<254:54:00,  5.69s/it]
 28%|██▊       | 63621/225000 [130:25:41<313:02:32,  6.98s/it]
 28%|██▊       | 63622/225000 [130:25:49<326:30:13,  7.28s/it]
 28%|██▊       | 63623/225000 [130:25:57<344:12:07,  7.68s/it]
 28%|██▊       | 63624/225000 [130:26:03<321:55:57,  7.18s/it]
 28%|██▊       | 63625/225000 [130:26:08<281:43:02,  6.28s/it]
 28%|██▊       | 63626/225000 [130:26:12<255:20:06,  5.70s/it]
 28%|██▊       | 63627/225000 [130:26:17<245:17:04,  5.47s/it]
 28%|██▊       | 63628/225000 [130:26:25<283:10:25,  6.32s/it]
 28%|██▊       | 63629/225000 [130:26:37<354:37:35,  7.91s/it]
 28%|██▊       | 63630/225000 [130:26:40<298:30:08,  6.66s/it]
 28%|██▊       | 63631/225000 [130:26:46<277:10:24,  6.18s/it]
 28%|██▊       | 63632/225000 [130:26:51<263:36:14,  5.88s/it]
 28%|██▊       | 63633/225000 [130:27:03<343:25:38,  7.66s/it]
 28%|██▊       | 63634/225000 [130:27:08<314:52:17,  7.02s/it]
 28%|██▊       | 63635/225000 [130:27:13<284:03:43,  6.34s/it]
 28%|██▊       | 63636/225000 [130:27:19<280:56:55,  6.27s/it]
 28%|██▊       | 63637/225000 [130:27:30<352:42:51,  7.87s/it]
 28%|██▊       | 63638/225000 [130:27:35<307:42:13,  6.86s/it]
 28%|██▊       | 63639/225000 [130:27:39<270:56:03,  6.04s/it]
 28%|██▊       | 63640/225000 [130:27:48<310:09:51,  6.92s/it]
 28%|██▊       | 63641/225000 [130:27:54<292:47:26,  6.53s/it]
 28%|██▊       | 63642/225000 [130:27:59<275:20:54,  6.14s/it]
 28%|██▊       | 63643/225000 [130:28:05<271:50:09,  6.06s/it]
 28%|██▊       | 63644/225000 [130:28:11<266:47:22,  5.95s/it]
 28%|██▊       | 63645/225000 [130:28:16<260:05:30,  5.80s/it]
 28%|██▊       | 63646/225000 [130:28:22<259:36:37,  5.79s/it]
 28%|██▊       | 63647/225000 [130:28:27<254:17:50,  5.67s/it]
 28%|██▊       | 63648/225000 [130:28:40<351:09:00,  7.83s/it]
 28%|██▊       | 63649/225000 [130:28:47<338:11:41,  7.55s/it]
 28%|██▊       | 63650/225000 [130:28:52<303:37:15,  6.77s/it]
                                                              
{'loss': '0.7695', 'grad_norm': '0.9283', 'learning_rate': '4.271e-05', 'epoch': '0.2829', 'num_input_tokens_seen': 216936632, 'train_runtime': '4.697e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63650/225000 [130:28:52<303:37:15,  6.77s/it]
 28%|██▊       | 63651/225000 [130:28:57<277:07:27,  6.18s/it]
 28%|██▊       | 63652/225000 [130:29:01<255:29:56,  5.70s/it]
 28%|██▊       | 63653/225000 [130:29:12<322:02:17,  7.19s/it]
 28%|██▊       | 63654/225000 [130:29:18<300:42:44,  6.71s/it]
 28%|██▊       | 63655/225000 [130:29:25<309:34:33,  6.91s/it]
 28%|██▊       | 63656/225000 [130:29:29<277:08:44,  6.18s/it]
 28%|██▊       | 63657/225000 [130:29:37<302:24:11,  6.75s/it]
 28%|██▊       | 63658/225000 [130:29:42<266:36:03,  5.95s/it]
 28%|██▊       | 63659/225000 [130:29:51<315:14:35,  7.03s/it]
 28%|██▊       | 63660/225000 [130:29:58<315:09:35,  7.03s/it]
 28%|██▊       | 63661/225000 [130:30:05<311:31:33,  6.95s/it]
 28%|██▊       | 63662/225000 [130:30:11<295:06:48,  6.58s/it]
 28%|██▊       | 63663/225000 [130:30:15<264:39:09,  5.91s/it]
 28%|██▊       | 63664/225000 [130:30:25<319:27:59,  7.13s/it]
 28%|██▊       | 63665/225000 [130:30:29<277:51:39,  6.20s/it]
 28%|██▊       | 63666/225000 [130:30:37<297:35:44,  6.64s/it]
 28%|██▊       | 63667/225000 [130:30:40<258:24:03,  5.77s/it]
 28%|██▊       | 63668/225000 [130:30:51<324:36:23,  7.24s/it]
 28%|██▊       | 63669/225000 [130:30:55<278:56:56,  6.22s/it]
 28%|██▊       | 63670/225000 [130:31:00<259:02:15,  5.78s/it]
 28%|██▊       | 63671/225000 [130:31:09<307:56:47,  6.87s/it]
 28%|██▊       | 63672/225000 [130:31:14<284:03:51,  6.34s/it]
 28%|██▊       | 63673/225000 [130:31:19<260:17:16,  5.81s/it]
 28%|██▊       | 63674/225000 [130:31:32<354:55:31,  7.92s/it]
 28%|██▊       | 63675/225000 [130:31:36<303:35:56,  6.77s/it]
 28%|██▊       | 63676/225000 [130:31:40<270:40:08,  6.04s/it]
 28%|██▊       | 63677/225000 [130:31:50<327:37:17,  7.31s/it]
 28%|██▊       | 63678/225000 [130:31:55<295:16:44,  6.59s/it]
 28%|██▊       | 63679/225000 [130:32:01<279:09:08,  6.23s/it]
 28%|██▊       | 63680/225000 [130:32:06<269:25:11,  6.01s/it]
 28%|██▊       | 63681/225000 [130:32:11<257:44:27,  5.75s/it]
 28%|██▊       | 63682/225000 [130:32:17<253:42:05,  5.66s/it]
 28%|██▊       | 63683/225000 [130:32:22<252:33:35,  5.64s/it]
 28%|██▊       | 63684/225000 [130:32:29<269:52:25,  6.02s/it]
 28%|██▊       | 63685/225000 [130:32:43<370:21:13,  8.27s/it]
 28%|██▊       | 63686/225000 [130:32:47<314:34:43,  7.02s/it]
 28%|██▊       | 63687/225000 [130:32:55<335:22:47,  7.48s/it]
 28%|██▊       | 63688/225000 [130:33:00<297:29:33,  6.64s/it]
 28%|██▊       | 63689/225000 [130:33:04<267:16:36,  5.96s/it]
 28%|██▊       | 63690/225000 [130:33:09<244:52:43,  5.47s/it]
 28%|██▊       | 63691/225000 [130:33:18<301:13:46,  6.72s/it]
 28%|██▊       | 63692/225000 [130:33:27<321:35:55,  7.18s/it]
 28%|██▊       | 63693/225000 [130:33:31<286:15:06,  6.39s/it]
 28%|██▊       | 63694/225000 [130:33:39<300:22:14,  6.70s/it]
 28%|██▊       | 63695/225000 [130:33:45<293:56:17,  6.56s/it]
 28%|██▊       | 63696/225000 [130:33:51<288:41:36,  6.44s/it]
 28%|██▊       | 63697/225000 [130:34:01<339:12:40,  7.57s/it]
 28%|██▊       | 63698/225000 [130:34:05<285:43:33,  6.38s/it]
 28%|██▊       | 63699/225000 [130:34:10<272:36:19,  6.08s/it]
 28%|██▊       | 63700/225000 [130:34:18<291:24:49,  6.50s/it]
                                                              
{'loss': '0.753', 'grad_norm': '0.8496', 'learning_rate': '4.27e-05', 'epoch': '0.2831', 'num_input_tokens_seen': 217082864, 'train_runtime': '4.701e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63700/225000 [130:34:18<291:24:49,  6.50s/it]
 28%|██▊       | 63701/225000 [130:34:22<257:46:03,  5.75s/it]
 28%|██▊       | 63702/225000 [130:34:30<289:05:31,  6.45s/it]
 28%|██▊       | 63703/225000 [130:34:35<276:08:13,  6.16s/it]
 28%|██▊       | 63704/225000 [130:34:40<261:25:29,  5.83s/it]
 28%|██▊       | 63705/225000 [130:34:52<333:44:58,  7.45s/it]
 28%|██▊       | 63706/225000 [130:34:56<294:35:03,  6.57s/it]
 28%|██▊       | 63707/225000 [130:35:00<266:02:23,  5.94s/it]
 28%|██▊       | 63708/225000 [130:35:09<300:23:17,  6.70s/it]
 28%|██▊       | 63709/225000 [130:35:18<326:37:56,  7.29s/it]
 28%|██▊       | 63710/225000 [130:35:24<319:12:44,  7.12s/it]
 28%|██▊       | 63711/225000 [130:35:30<295:59:42,  6.61s/it]
 28%|██▊       | 63712/225000 [130:35:35<273:02:11,  6.09s/it]
 28%|██▊       | 63713/225000 [130:35:40<261:07:15,  5.83s/it]
 28%|██▊       | 63714/225000 [130:35:45<248:00:53,  5.54s/it]
 28%|██▊       | 63715/225000 [130:35:50<241:36:47,  5.39s/it]
 28%|██▊       | 63716/225000 [130:35:58<285:42:55,  6.38s/it]
 28%|██▊       | 63717/225000 [130:36:11<368:38:55,  8.23s/it]
 28%|██▊       | 63718/225000 [130:36:15<306:57:26,  6.85s/it]
 28%|██▊       | 63719/225000 [130:36:23<330:42:22,  7.38s/it]
 28%|██▊       | 63720/225000 [130:36:28<288:34:16,  6.44s/it]
 28%|██▊       | 63721/225000 [130:36:32<262:06:17,  5.85s/it]
 28%|██▊       | 63722/225000 [130:36:40<296:05:22,  6.61s/it]
 28%|██▊       | 63723/225000 [130:36:47<293:17:42,  6.55s/it]
 28%|██▊       | 63724/225000 [130:36:52<280:26:06,  6.26s/it]
 28%|██▊       | 63725/225000 [130:37:01<313:45:49,  7.00s/it]
 28%|██▊       | 63726/225000 [130:37:08<312:19:02,  6.97s/it]
 28%|██▊       | 63727/225000 [130:37:13<288:44:31,  6.45s/it]
 28%|██▊       | 63728/225000 [130:37:22<313:38:55,  7.00s/it]
 28%|██▊       | 63729/225000 [130:37:29<317:05:05,  7.08s/it]
 28%|██▊       | 63730/225000 [130:37:33<278:41:42,  6.22s/it]
 28%|██▊       | 63731/225000 [130:37:41<300:09:05,  6.70s/it]
 28%|██▊       | 63732/225000 [130:37:46<284:54:56,  6.36s/it]
 28%|██▊       | 63733/225000 [130:37:52<273:07:41,  6.10s/it]
 28%|██▊       | 63734/225000 [130:37:57<258:30:03,  5.77s/it]
 28%|██▊       | 63735/225000 [130:38:02<252:30:41,  5.64s/it]
 28%|██▊       | 63736/225000 [130:38:08<252:28:57,  5.64s/it]
 28%|██▊       | 63737/225000 [130:38:13<247:47:28,  5.53s/it]
 28%|██▊       | 63738/225000 [130:38:25<337:51:02,  7.54s/it]
 28%|██▊       | 63739/225000 [130:38:32<324:19:51,  7.24s/it]
 28%|██▊       | 63740/225000 [130:38:36<277:05:29,  6.19s/it]
 28%|██▊       | 63741/225000 [130:38:46<335:44:39,  7.50s/it]
 28%|██▊       | 63742/225000 [130:38:51<301:54:04,  6.74s/it]
 28%|██▊       | 63743/225000 [130:38:56<275:42:27,  6.16s/it]
 28%|██▊       | 63744/225000 [130:39:01<259:44:00,  5.80s/it]
 28%|██▊       | 63745/225000 [130:39:06<246:50:00,  5.51s/it]
 28%|██▊       | 63746/225000 [130:39:11<239:16:57,  5.34s/it]
 28%|██▊       | 63747/225000 [130:39:17<250:55:45,  5.60s/it]
 28%|██▊       | 63748/225000 [130:39:29<336:41:22,  7.52s/it]
 28%|██▊       | 63749/225000 [130:39:33<288:21:04,  6.44s/it]
 28%|██▊       | 63750/225000 [130:39:44<347:01:04,  7.75s/it]
                                                              
{'loss': '0.7637', 'grad_norm': '0.7528', 'learning_rate': '4.269e-05', 'epoch': '0.2833', 'num_input_tokens_seen': 217229360, 'train_runtime': '4.704e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63750/225000 [130:39:44<347:01:04,  7.75s/it]
 28%|██▊       | 63751/225000 [130:39:48<298:28:34,  6.66s/it]
 28%|██▊       | 63752/225000 [130:39:54<286:19:50,  6.39s/it]
 28%|██▊       | 63753/225000 [130:39:59<275:45:19,  6.16s/it]
 28%|██▊       | 63754/225000 [130:40:05<275:06:58,  6.14s/it]
 28%|██▊       | 63755/225000 [130:40:16<340:26:57,  7.60s/it]
 28%|██▊       | 63756/225000 [130:40:20<288:38:13,  6.44s/it]
 28%|██▊       | 63757/225000 [130:40:25<265:45:55,  5.93s/it]
 28%|██▊       | 63758/225000 [130:40:34<305:44:03,  6.83s/it]
 28%|██▊       | 63759/225000 [130:40:39<284:23:52,  6.35s/it]
 28%|██▊       | 63760/225000 [130:40:44<265:21:18,  5.92s/it]
 28%|██▊       | 63761/225000 [130:40:49<258:25:25,  5.77s/it]
 28%|██▊       | 63762/225000 [130:40:55<253:50:28,  5.67s/it]
 28%|██▊       | 63763/225000 [130:41:00<245:02:49,  5.47s/it]
 28%|██▊       | 63764/225000 [130:41:09<301:49:09,  6.74s/it]
 28%|██▊       | 63765/225000 [130:41:19<344:03:20,  7.68s/it]
 28%|██▊       | 63766/225000 [130:41:25<319:11:05,  7.13s/it]
 28%|██▊       | 63767/225000 [130:41:31<302:16:38,  6.75s/it]
 28%|██▊       | 63768/225000 [130:41:39<326:07:23,  7.28s/it]
 28%|██▊       | 63769/225000 [130:41:44<287:17:04,  6.41s/it]
 28%|██▊       | 63770/225000 [130:41:51<297:55:57,  6.65s/it]
 28%|██▊       | 63771/225000 [130:41:55<261:49:05,  5.85s/it]
 28%|██▊       | 63772/225000 [130:42:06<326:22:42,  7.29s/it]
 28%|██▊       | 63773/225000 [130:42:12<316:55:29,  7.08s/it]
 28%|██▊       | 63774/225000 [130:42:16<277:30:21,  6.20s/it]
 28%|██▊       | 63775/225000 [130:42:27<340:30:45,  7.60s/it]
 28%|██▊       | 63776/225000 [130:42:31<283:07:37,  6.32s/it]
 28%|██▊       | 63777/225000 [130:42:38<304:12:42,  6.79s/it]
 28%|██▊       | 63778/225000 [130:42:43<267:03:21,  5.96s/it]
 28%|██▊       | 63779/225000 [130:42:47<245:36:26,  5.48s/it]
 28%|██▊       | 63780/225000 [130:42:59<328:52:26,  7.34s/it]
 28%|██▊       | 63781/225000 [130:43:02<280:54:26,  6.27s/it]
 28%|██▊       | 63782/225000 [130:43:11<317:45:05,  7.10s/it]
 28%|██▊       | 63783/225000 [130:43:20<337:13:59,  7.53s/it]
 28%|██▊       | 63784/225000 [130:43:27<325:53:43,  7.28s/it]
 28%|██▊       | 63785/225000 [130:43:31<293:09:44,  6.55s/it]
 28%|██▊       | 63786/225000 [130:43:39<311:54:21,  6.97s/it]
 28%|██▊       | 63787/225000 [130:43:46<310:26:16,  6.93s/it]
 28%|██▊       | 63788/225000 [130:43:50<263:59:35,  5.90s/it]
 28%|██▊       | 63789/225000 [130:43:58<292:09:04,  6.52s/it]
 28%|██▊       | 63790/225000 [130:44:02<264:25:22,  5.90s/it]
 28%|██▊       | 63791/225000 [130:44:07<250:50:33,  5.60s/it]
 28%|██▊       | 63792/225000 [130:44:14<265:41:30,  5.93s/it]
 28%|██▊       | 63793/225000 [130:44:25<336:54:06,  7.52s/it]
 28%|██▊       | 63794/225000 [130:44:30<299:56:04,  6.70s/it]
 28%|██▊       | 63795/225000 [130:44:35<280:35:16,  6.27s/it]
 28%|██▊       | 63796/225000 [130:44:40<260:19:25,  5.81s/it]
 28%|██▊       | 63797/225000 [130:44:45<246:46:43,  5.51s/it]
 28%|██▊       | 63798/225000 [130:44:49<238:20:39,  5.32s/it]
 28%|██▊       | 63799/225000 [130:44:58<287:31:36,  6.42s/it]
 28%|██▊       | 63800/225000 [130:45:08<335:04:31,  7.48s/it]
                                                              
{'loss': '0.7528', 'grad_norm': '0.7068', 'learning_rate': '4.267e-05', 'epoch': '0.2836', 'num_input_tokens_seen': 217375384, 'train_runtime': '4.707e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63800/225000 [130:45:08<335:04:31,  7.48s/it]
 28%|██▊       | 63801/225000 [130:45:15<327:23:30,  7.31s/it]
 28%|██▊       | 63802/225000 [130:45:22<324:07:36,  7.24s/it]
 28%|██▊       | 63803/225000 [130:45:27<284:01:44,  6.34s/it]
 28%|██▊       | 63804/225000 [130:45:32<264:04:42,  5.90s/it]
 28%|██▊       | 63805/225000 [130:45:40<299:23:39,  6.69s/it]
 28%|██▊       | 63806/225000 [130:45:44<269:11:13,  6.01s/it]
 28%|██▊       | 63807/225000 [130:45:55<324:18:55,  7.24s/it]
 28%|██▊       | 63808/225000 [130:45:59<290:00:29,  6.48s/it]
 28%|██▊       | 63809/225000 [130:46:06<295:27:33,  6.60s/it]
 28%|██▊       | 63810/225000 [130:46:11<269:42:50,  6.02s/it]
 28%|██▊       | 63811/225000 [130:46:20<318:23:47,  7.11s/it]
 28%|██▊       | 63812/225000 [130:46:24<271:42:00,  6.07s/it]
 28%|██▊       | 63813/225000 [130:46:29<253:36:11,  5.66s/it]
 28%|██▊       | 63814/225000 [130:46:39<310:45:03,  6.94s/it]
 28%|██▊       | 63815/225000 [130:46:44<284:27:12,  6.35s/it]
 28%|██▊       | 63816/225000 [130:46:49<269:57:59,  6.03s/it]
 28%|██▊       | 63817/225000 [130:46:54<258:23:01,  5.77s/it]
 28%|██▊       | 63818/225000 [130:47:00<254:16:28,  5.68s/it]
 28%|██▊       | 63819/225000 [130:47:12<350:17:05,  7.82s/it]
 28%|██▊       | 63820/225000 [130:47:17<301:02:28,  6.72s/it]
 28%|██▊       | 63821/225000 [130:47:23<301:05:00,  6.72s/it]
 28%|██▊       | 63822/225000 [130:47:30<298:03:47,  6.66s/it]
 28%|██▊       | 63823/225000 [130:47:38<313:57:48,  7.01s/it]
 28%|██▊       | 63824/225000 [130:47:44<302:07:47,  6.75s/it]
 28%|██▊       | 63825/225000 [130:47:51<309:17:45,  6.91s/it]
 28%|██▊       | 63826/225000 [130:47:56<285:43:40,  6.38s/it]
 28%|██▊       | 63827/225000 [130:48:02<273:23:00,  6.11s/it]
 28%|██▊       | 63828/225000 [130:48:09<283:34:30,  6.33s/it]
 28%|██▊       | 63829/225000 [130:48:16<293:06:37,  6.55s/it]
 28%|██▊       | 63830/225000 [130:48:21<277:01:10,  6.19s/it]
 28%|██▊       | 63831/225000 [130:48:27<268:56:07,  6.01s/it]
 28%|██▊       | 63832/225000 [130:48:35<300:42:14,  6.72s/it]
 28%|██▊       | 63833/225000 [130:48:45<342:10:08,  7.64s/it]
 28%|██▊       | 63834/225000 [130:48:48<288:56:41,  6.45s/it]
 28%|██▊       | 63835/225000 [130:48:54<278:49:21,  6.23s/it]
 28%|██▊       | 63836/225000 [130:49:02<302:35:52,  6.76s/it]
 28%|██▊       | 63837/225000 [130:49:10<320:44:37,  7.16s/it]
 28%|██▊       | 63838/225000 [130:49:15<284:18:39,  6.35s/it]
 28%|██▊       | 63839/225000 [130:49:20<265:31:25,  5.93s/it]
 28%|██▊       | 63840/225000 [130:49:24<246:51:57,  5.51s/it]
 28%|██▊       | 63841/225000 [130:49:29<242:43:26,  5.42s/it]
 28%|██▊       | 63842/225000 [130:49:34<237:09:06,  5.30s/it]
 28%|██▊       | 63843/225000 [130:49:47<338:37:53,  7.56s/it]
 28%|██▊       | 63844/225000 [130:49:52<300:33:16,  6.71s/it]
 28%|██▊       | 63845/225000 [130:50:00<322:13:17,  7.20s/it]
 28%|██▊       | 63846/225000 [130:50:04<278:00:44,  6.21s/it]
 28%|██▊       | 63847/225000 [130:50:14<328:45:50,  7.34s/it]
 28%|██▊       | 63848/225000 [130:50:19<288:16:46,  6.44s/it]
 28%|██▊       | 63849/225000 [130:50:23<256:16:20,  5.72s/it]
 28%|██▊       | 63850/225000 [130:50:27<234:34:56,  5.24s/it]
                                                              
{'loss': '0.7581', 'grad_norm': '0.8136', 'learning_rate': '4.266e-05', 'epoch': '0.2838', 'num_input_tokens_seen': 217521600, 'train_runtime': '4.71e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63850/225000 [130:50:27<234:34:56,  5.24s/it]
 28%|██▊       | 63851/225000 [130:50:31<224:56:25,  5.03s/it]
 28%|██▊       | 63852/225000 [130:50:40<279:06:50,  6.24s/it]
 28%|██▊       | 63853/225000 [130:50:51<336:55:02,  7.53s/it]
 28%|██▊       | 63854/225000 [130:50:56<301:47:27,  6.74s/it]
 28%|██▊       | 63855/225000 [130:51:01<278:15:11,  6.22s/it]
 28%|██▊       | 63856/225000 [130:51:06<267:28:47,  5.98s/it]
 28%|██▊       | 63857/225000 [130:51:11<248:20:14,  5.55s/it]
 28%|██▊       | 63858/225000 [130:51:21<314:17:03,  7.02s/it]
 28%|██▊       | 63859/225000 [130:51:28<318:04:14,  7.11s/it]
 28%|██▊       | 63860/225000 [130:51:36<318:39:55,  7.12s/it]
 28%|██▊       | 63861/225000 [130:51:40<282:46:14,  6.32s/it]
 28%|██▊       | 63862/225000 [130:51:44<255:11:18,  5.70s/it]
 28%|██▊       | 63863/225000 [130:51:55<322:30:31,  7.21s/it]
 28%|██▊       | 63864/225000 [130:51:59<274:48:05,  6.14s/it]
 28%|██▊       | 63865/225000 [130:52:05<275:59:07,  6.17s/it]
 28%|██▊       | 63866/225000 [130:52:13<303:51:25,  6.79s/it]
 28%|██▊       | 63867/225000 [130:52:20<310:59:43,  6.95s/it]
 28%|██▊       | 63868/225000 [130:52:27<299:01:08,  6.68s/it]
 28%|██▊       | 63869/225000 [130:52:34<309:39:41,  6.92s/it]
 28%|██▊       | 63870/225000 [130:52:38<269:05:40,  6.01s/it]
 28%|██▊       | 63871/225000 [130:52:42<241:35:26,  5.40s/it]
 28%|██▊       | 63872/225000 [130:52:49<269:22:36,  6.02s/it]
 28%|██▊       | 63873/225000 [130:52:56<282:54:24,  6.32s/it]
 28%|██▊       | 63874/225000 [130:53:02<267:36:13,  5.98s/it]
 28%|██▊       | 63875/225000 [130:53:07<258:31:09,  5.78s/it]
 28%|██▊       | 63876/225000 [130:53:12<256:01:20,  5.72s/it]
 28%|██▊       | 63877/225000 [130:53:21<300:36:45,  6.72s/it]
 28%|██▊       | 63878/225000 [130:53:31<335:31:07,  7.50s/it]
 28%|██▊       | 63879/225000 [130:53:35<290:16:36,  6.49s/it]
 28%|██▊       | 63880/225000 [130:53:43<314:18:36,  7.02s/it]
 28%|██▊       | 63881/225000 [130:53:51<330:07:29,  7.38s/it]
 28%|██▊       | 63882/225000 [130:53:55<284:05:21,  6.35s/it]
 28%|██▊       | 63883/225000 [130:54:00<265:15:34,  5.93s/it]
 28%|██▊       | 63884/225000 [130:54:05<253:55:59,  5.67s/it]
 28%|██▊       | 63885/225000 [130:54:17<339:18:15,  7.58s/it]
 28%|██▊       | 63886/225000 [130:54:21<286:40:53,  6.41s/it]
 28%|██▊       | 63887/225000 [130:54:31<328:31:15,  7.34s/it]
 28%|██▊       | 63888/225000 [130:54:37<309:35:55,  6.92s/it]
 28%|██▊       | 63889/225000 [130:54:40<261:36:52,  5.85s/it]
 28%|██▊       | 63890/225000 [130:54:48<291:09:29,  6.51s/it]
 28%|██▊       | 63891/225000 [130:54:53<270:01:50,  6.03s/it]
 28%|██▊       | 63892/225000 [130:54:57<246:52:27,  5.52s/it]
 28%|██▊       | 63893/225000 [130:55:02<241:15:11,  5.39s/it]
 28%|██▊       | 63894/225000 [130:55:14<325:09:55,  7.27s/it]
 28%|██▊       | 63895/225000 [130:55:20<306:04:45,  6.84s/it]
 28%|██▊       | 63896/225000 [130:55:25<290:48:58,  6.50s/it]
 28%|██▊       | 63897/225000 [130:55:31<283:46:26,  6.34s/it]
 28%|██▊       | 63898/225000 [130:55:37<268:06:16,  5.99s/it]
 28%|██▊       | 63899/225000 [130:55:45<306:14:45,  6.84s/it]
 28%|██▊       | 63900/225000 [130:55:50<277:55:22,  6.21s/it]
                                                              
{'loss': '0.7695', 'grad_norm': '0.8297', 'learning_rate': '4.265e-05', 'epoch': '0.284', 'num_input_tokens_seen': 217665416, 'train_runtime': '4.714e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63900/225000 [130:55:50<277:55:22,  6.21s/it]
 28%|██▊       | 63901/225000 [130:55:56<272:52:25,  6.10s/it]
 28%|██▊       | 63902/225000 [130:56:02<270:12:01,  6.04s/it]
 28%|██▊       | 63903/225000 [130:56:08<267:53:10,  5.99s/it]
 28%|██▊       | 63904/225000 [130:56:13<262:20:46,  5.86s/it]
 28%|██▊       | 63905/225000 [130:56:22<298:38:54,  6.67s/it]
 28%|██▊       | 63906/225000 [130:56:32<345:27:23,  7.72s/it]
 28%|██▊       | 63907/225000 [130:56:37<308:18:40,  6.89s/it]
 28%|██▊       | 63908/225000 [130:56:45<322:14:59,  7.20s/it]
 28%|██▊       | 63909/225000 [130:56:49<281:42:28,  6.30s/it]
 28%|██▊       | 63910/225000 [130:56:57<305:55:36,  6.84s/it]
 28%|██▊       | 63911/225000 [130:57:03<286:51:52,  6.41s/it]
 28%|██▊       | 63912/225000 [130:57:09<282:31:25,  6.31s/it]
 28%|██▊       | 63913/225000 [130:57:15<279:16:44,  6.24s/it]
 28%|██▊       | 63914/225000 [130:57:19<254:17:04,  5.68s/it]
 28%|██▊       | 63915/225000 [130:57:29<312:12:22,  6.98s/it]
 28%|██▊       | 63916/225000 [130:57:35<298:02:03,  6.66s/it]
 28%|██▊       | 63917/225000 [130:57:42<295:07:59,  6.60s/it]
 28%|██▊       | 63918/225000 [130:57:48<290:11:11,  6.49s/it]
 28%|██▊       | 63919/225000 [130:57:54<287:05:09,  6.42s/it]
 28%|██▊       | 63920/225000 [130:58:00<277:14:52,  6.20s/it]
 28%|██▊       | 63921/225000 [130:58:05<269:38:59,  6.03s/it]
 28%|██▊       | 63922/225000 [130:58:10<254:06:24,  5.68s/it]
 28%|██▊       | 63923/225000 [130:58:16<252:23:08,  5.64s/it]
 28%|██▊       | 63924/225000 [130:58:21<242:00:49,  5.41s/it]
 28%|██▊       | 63925/225000 [130:58:26<240:37:22,  5.38s/it]
 28%|██▊       | 63926/225000 [130:58:33<263:20:21,  5.89s/it]
 28%|██▊       | 63927/225000 [130:58:46<358:09:22,  8.00s/it]
 28%|██▊       | 63928/225000 [130:58:50<300:28:04,  6.72s/it]
 28%|██▊       | 63929/225000 [130:58:55<278:26:25,  6.22s/it]
 28%|██▊       | 63930/225000 [130:59:00<263:02:24,  5.88s/it]
 28%|██▊       | 63931/225000 [130:59:05<256:14:31,  5.73s/it]
 28%|██▊       | 63932/225000 [130:59:18<348:54:42,  7.80s/it]
 28%|██▊       | 63933/225000 [130:59:23<315:17:32,  7.05s/it]
 28%|██▊       | 63934/225000 [130:59:31<327:31:30,  7.32s/it]
 28%|██▊       | 63935/225000 [130:59:37<309:03:42,  6.91s/it]
 28%|██▊       | 63936/225000 [130:59:42<276:39:17,  6.18s/it]
 28%|██▊       | 63937/225000 [130:59:51<319:07:06,  7.13s/it]
 28%|██▊       | 63938/225000 [130:59:56<290:50:28,  6.50s/it]
 28%|██▊       | 63939/225000 [131:00:01<276:23:59,  6.18s/it]
 28%|██▊       | 63940/225000 [131:00:07<272:34:03,  6.09s/it]
 28%|██▊       | 63941/225000 [131:00:13<266:20:14,  5.95s/it]
 28%|██▊       | 63942/225000 [131:00:21<298:41:40,  6.68s/it]
 28%|██▊       | 63943/225000 [131:00:29<317:03:24,  7.09s/it]
 28%|██▊       | 63944/225000 [131:00:37<332:28:12,  7.43s/it]
 28%|██▊       | 63945/225000 [131:00:41<281:53:39,  6.30s/it]
 28%|██▊       | 63946/225000 [131:00:50<316:43:34,  7.08s/it]
 28%|██▊       | 63947/225000 [131:00:56<300:32:22,  6.72s/it]
 28%|██▊       | 63948/225000 [131:01:00<264:38:27,  5.92s/it]
 28%|██▊       | 63949/225000 [131:01:05<257:29:17,  5.76s/it]
 28%|██▊       | 63950/225000 [131:01:11<255:36:41,  5.71s/it]
                                                              
{'loss': '0.7596', 'grad_norm': '0.764', 'learning_rate': '4.263e-05', 'epoch': '0.2842', 'num_input_tokens_seen': 217811360, 'train_runtime': '4.717e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 63950/225000 [131:01:11<255:36:41,  5.71s/it]
 28%|██▊       | 63951/225000 [131:01:23<334:26:55,  7.48s/it]
 28%|██▊       | 63952/225000 [131:01:26<282:29:37,  6.31s/it]
 28%|██▊       | 63953/225000 [131:01:30<252:47:54,  5.65s/it]
 28%|██▊       | 63954/225000 [131:01:41<323:27:12,  7.23s/it]
 28%|██▊       | 63955/225000 [131:01:45<277:29:47,  6.20s/it]
 28%|██▊       | 63956/225000 [131:01:52<282:21:19,  6.31s/it]
 28%|██▊       | 63957/225000 [131:01:56<254:34:55,  5.69s/it]
 28%|██▊       | 63958/225000 [131:02:05<305:53:02,  6.84s/it]
 28%|██▊       | 63959/225000 [131:02:10<271:33:33,  6.07s/it]
 28%|██▊       | 63960/225000 [131:02:18<306:26:10,  6.85s/it]
 28%|██▊       | 63961/225000 [131:02:23<280:41:43,  6.27s/it]
 28%|██▊       | 63962/225000 [131:02:28<257:39:24,  5.76s/it]
 28%|██▊       | 63963/225000 [131:02:33<245:11:27,  5.48s/it]
 28%|██▊       | 63964/225000 [131:02:46<351:49:23,  7.87s/it]
 28%|██▊       | 63965/225000 [131:02:50<296:10:57,  6.62s/it]
 28%|██▊       | 63966/225000 [131:02:54<262:48:03,  5.88s/it]
 28%|██▊       | 63967/225000 [131:02:58<240:48:39,  5.38s/it]
 28%|██▊       | 63968/225000 [131:03:11<339:52:41,  7.60s/it]
 28%|██▊       | 63969/225000 [131:03:14<284:41:38,  6.36s/it]
 28%|██▊       | 63970/225000 [131:03:19<260:59:57,  5.83s/it]
 28%|██▊       | 63971/225000 [131:03:23<242:35:15,  5.42s/it]
 28%|██▊       | 63972/225000 [131:03:32<282:28:34,  6.32s/it]
 28%|██▊       | 63973/225000 [131:03:41<320:37:28,  7.17s/it]
 28%|██▊       | 63974/225000 [131:03:46<286:46:54,  6.41s/it]
 28%|██▊       | 63975/225000 [131:03:54<316:40:57,  7.08s/it]
 28%|██▊       | 63976/225000 [131:04:02<328:14:36,  7.34s/it]
 28%|██▊       | 63977/225000 [131:04:07<292:42:53,  6.54s/it]
 28%|██▊       | 63978/225000 [131:04:11<265:36:09,  5.94s/it]
 28%|██▊       | 63979/225000 [131:04:16<249:52:19,  5.59s/it]
 28%|██▊       | 63980/225000 [131:04:22<254:26:34,  5.69s/it]
 28%|██▊       | 63981/225000 [131:04:33<326:08:29,  7.29s/it]
 28%|██▊       | 63982/225000 [131:04:37<284:41:26,  6.37s/it]
 28%|██▊       | 63983/225000 [131:04:45<298:46:20,  6.68s/it]
 28%|██▊       | 63984/225000 [131:04:52<300:29:24,  6.72s/it]
 28%|██▊       | 63985/225000 [131:05:00<322:21:44,  7.21s/it]
 28%|██▊       | 63986/225000 [131:05:04<277:31:26,  6.20s/it]
 28%|██▊       | 63987/225000 [131:05:12<300:42:14,  6.72s/it]
 28%|██▊       | 63988/225000 [131:05:16<271:12:29,  6.06s/it]
 28%|██▊       | 63989/225000 [131:05:21<252:31:21,  5.65s/it]
 28%|██▊       | 63990/225000 [131:05:30<300:43:04,  6.72s/it]
 28%|██▊       | 63991/225000 [131:05:37<300:28:57,  6.72s/it]
 28%|██▊       | 63992/225000 [131:05:42<286:13:42,  6.40s/it]
 28%|██▊       | 63993/225000 [131:05:48<278:25:48,  6.23s/it]
 28%|██▊       | 63994/225000 [131:05:54<266:26:28,  5.96s/it]
 28%|██▊       | 63995/225000 [131:05:59<258:41:19,  5.78s/it]
 28%|██▊       | 63996/225000 [131:06:04<248:22:08,  5.55s/it]
 28%|██▊       | 63997/225000 [131:06:10<253:57:40,  5.68s/it]
 28%|██▊       | 63998/225000 [131:06:16<264:12:53,  5.91s/it]
 28%|██▊       | 63999/225000 [131:06:24<282:55:05,  6.33s/it]
 28%|██▊       | 64000/225000 [131:06:30<276:39:04,  6.19s/it]
                                                              
{'loss': '0.7405', 'grad_norm': '0.6499', 'learning_rate': '4.262e-05', 'epoch': '0.2844', 'num_input_tokens_seen': 217957072, 'train_runtime': '4.72e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 64000/225000 [131:06:30<276:39:04,  6.19s/it]
 28%|██▊       | 64001/225000 [131:06:35<261:51:27,  5.86s/it]
 28%|██▊       | 64002/225000 [131:06:41<261:48:26,  5.85s/it]
 28%|██▊       | 64003/225000 [131:06:46<253:26:08,  5.67s/it]
 28%|██▊       | 64004/225000 [131:06:59<350:15:37,  7.83s/it]
 28%|██▊       | 64005/225000 [131:07:05<335:51:39,  7.51s/it]
 28%|██▊       | 64006/225000 [131:07:14<349:34:39,  7.82s/it]
 28%|██▊       | 64007/225000 [131:07:18<304:31:15,  6.81s/it]
 28%|██▊       | 64008/225000 [131:07:24<291:01:21,  6.51s/it]
 28%|██▊       | 64009/225000 [131:07:30<277:15:22,  6.20s/it]
 28%|██▊       | 64010/225000 [131:07:35<265:32:58,  5.94s/it]
 28%|██▊       | 64011/225000 [131:07:40<257:16:46,  5.75s/it]
 28%|██▊       | 64012/225000 [131:07:45<246:57:12,  5.52s/it]
 28%|██▊       | 64013/225000 [131:07:50<239:12:09,  5.35s/it]
 28%|██▊       | 64014/225000 [131:08:01<308:36:34,  6.90s/it]
 28%|██▊       | 64015/225000 [131:08:12<373:10:41,  8.35s/it]
 28%|██▊       | 64016/225000 [131:08:17<316:15:43,  7.07s/it]
 28%|██▊       | 64017/225000 [131:08:21<278:59:20,  6.24s/it]
 28%|██▊       | 64018/225000 [131:08:31<327:36:09,  7.33s/it]
 28%|██▊       | 64019/225000 [131:08:38<323:20:32,  7.23s/it]
 28%|██▊       | 64020/225000 [131:08:47<344:14:18,  7.70s/it]
 28%|██▊       | 64021/225000 [131:08:51<305:09:01,  6.82s/it]
 28%|██▊       | 64022/225000 [131:08:56<277:28:30,  6.21s/it]
 28%|██▊       | 64023/225000 [131:09:01<258:31:07,  5.78s/it]
 28%|██▊       | 64024/225000 [131:09:07<257:20:22,  5.76s/it]
 28%|██▊       | 64025/225000 [131:09:12<247:48:53,  5.54s/it]
 28%|██▊       | 64026/225000 [131:09:16<238:01:32,  5.32s/it]
 28%|██▊       | 64027/225000 [131:09:22<243:21:23,  5.44s/it]
 28%|██▊       | 64028/225000 [131:09:33<321:42:01,  7.19s/it]
 28%|██▊       | 64029/225000 [131:09:43<356:37:52,  7.98s/it]
 28%|██▊       | 64030/225000 [131:09:50<339:10:20,  7.59s/it]
 28%|██▊       | 64031/225000 [131:09:57<329:41:17,  7.37s/it]
 28%|██▊       | 64032/225000 [131:10:01<280:43:20,  6.28s/it]
 28%|██▊       | 64033/225000 [131:10:11<332:13:47,  7.43s/it]
 28%|██▊       | 64034/225000 [131:10:15<286:58:52,  6.42s/it]
 28%|██▊       | 64035/225000 [131:10:20<265:27:12,  5.94s/it]
 28%|██▊       | 64036/225000 [131:10:29<314:16:37,  7.03s/it]
 28%|██▊       | 64037/225000 [131:10:33<266:16:57,  5.96s/it]
 28%|██▊       | 64038/225000 [131:10:37<252:12:40,  5.64s/it]
 28%|██▊       | 64039/225000 [131:10:48<321:59:12,  7.20s/it]
 28%|██▊       | 64040/225000 [131:10:57<336:17:44,  7.52s/it]
 28%|██▊       | 64041/225000 [131:11:00<284:59:24,  6.37s/it]
 28%|██▊       | 64042/225000 [131:11:06<282:28:43,  6.32s/it]
 28%|██▊       | 64043/225000 [131:11:14<302:50:54,  6.77s/it]
 28%|██▊       | 64044/225000 [131:11:18<257:29:51,  5.76s/it]
 28%|██▊       | 64045/225000 [131:11:26<288:39:41,  6.46s/it]
 28%|██▊       | 64046/225000 [131:11:33<296:08:35,  6.62s/it]
 28%|██▊       | 64047/225000 [131:11:40<300:49:18,  6.73s/it]
 28%|██▊       | 64048/225000 [131:11:44<262:38:47,  5.87s/it]
 28%|██▊       | 64049/225000 [131:11:54<327:33:05,  7.33s/it]
 28%|██▊       | 64050/225000 [131:11:58<282:14:44,  6.31s/it]
                                                              
{'loss': '0.7544', 'grad_norm': '0.7835', 'learning_rate': '4.261e-05', 'epoch': '0.2847', 'num_input_tokens_seen': 218104776, 'train_runtime': '4.723e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 64050/225000 [131:11:58<282:14:44,  6.31s/it]
 28%|██▊       | 64051/225000 [131:12:03<258:28:33,  5.78s/it]
 28%|██▊       | 64052/225000 [131:12:09<265:21:58,  5.94s/it]
 28%|██▊       | 64053/225000 [131:12:14<249:58:51,  5.59s/it]
 28%|██▊       | 64054/225000 [131:12:19<242:53:34,  5.43s/it]
 28%|██▊       | 64055/225000 [131:12:32<345:19:34,  7.72s/it]
 28%|██▊       | 64056/225000 [131:12:39<331:54:06,  7.42s/it]
 28%|██▊       | 64057/225000 [131:12:44<302:25:23,  6.76s/it]
 28%|██▊       | 64058/225000 [131:12:49<274:48:12,  6.15s/it]
 28%|██▊       | 64059/225000 [131:12:53<253:09:51,  5.66s/it]
 28%|██▊       | 64060/225000 [131:12:58<239:03:12,  5.35s/it]
 28%|██▊       | 64061/225000 [131:13:11<342:43:52,  7.67s/it]
 28%|██▊       | 64062/225000 [131:13:17<326:33:56,  7.30s/it]
 28%|██▊       | 64063/225000 [131:13:22<286:56:59,  6.42s/it]
 28%|██▊       | 64064/225000 [131:13:32<335:20:24,  7.50s/it]
 28%|██▊       | 64065/225000 [131:13:35<284:23:12,  6.36s/it]
 28%|██▊       | 64066/225000 [131:13:44<316:56:27,  7.09s/it]
 28%|██▊       | 64067/225000 [131:13:50<294:05:15,  6.58s/it]
 28%|██▊       | 64068/225000 [131:13:55<281:15:56,  6.29s/it]
 28%|██▊       | 64069/225000 [131:14:01<269:48:39,  6.04s/it]
 28%|██▊       | 64070/225000 [131:14:06<261:42:40,  5.85s/it]
 28%|██▊       | 64071/225000 [131:14:11<250:57:10,  5.61s/it]
 28%|██▊       | 64072/225000 [131:14:17<247:35:02,  5.54s/it]
 28%|██▊       | 64073/225000 [131:14:29<343:38:34,  7.69s/it]
 28%|██▊       | 64074/225000 [131:14:36<333:43:38,  7.47s/it]
 28%|██▊       | 64075/225000 [131:14:43<321:20:48,  7.19s/it]
 28%|██▊       | 64076/225000 [131:14:48<289:39:28,  6.48s/it]
 28%|██▊       | 64077/225000 [131:14:52<267:31:47,  5.98s/it]
 28%|██▊       | 64078/225000 [131:14:57<255:10:05,  5.71s/it]
 28%|██▊       | 64079/225000 [131:15:03<253:15:21,  5.67s/it]
 28%|██▊       | 64080/225000 [131:15:15<338:20:02,  7.57s/it]
 28%|██▊       | 64081/225000 [131:15:21<318:41:28,  7.13s/it]
 28%|██▊       | 64082/225000 [131:15:31<354:56:01,  7.94s/it]
 28%|██▊       | 64083/225000 [131:15:35<300:05:11,  6.71s/it]
 28%|██▊       | 64084/225000 [131:15:39<267:20:59,  5.98s/it]
 28%|██▊       | 64085/225000 [131:15:49<315:39:58,  7.06s/it]
 28%|██▊       | 64086/225000 [131:15:53<284:10:14,  6.36s/it]
 28%|██▊       | 64087/225000 [131:15:59<269:10:42,  6.02s/it]
 28%|██▊       | 64088/225000 [131:16:04<256:05:10,  5.73s/it]
 28%|██▊       | 64089/225000 [131:16:10<268:06:55,  6.00s/it]
 28%|██▊       | 64090/225000 [131:16:23<352:27:48,  7.89s/it]
 28%|██▊       | 64091/225000 [131:16:27<308:00:04,  6.89s/it]
 28%|██▊       | 64092/225000 [131:16:34<307:07:58,  6.87s/it]
 28%|██▊       | 64093/225000 [131:16:42<327:03:30,  7.32s/it]
 28%|██▊       | 64094/225000 [131:16:47<289:31:04,  6.48s/it]
 28%|██▊       | 64095/225000 [131:16:51<262:41:19,  5.88s/it]
 28%|██▊       | 64096/225000 [131:17:02<321:09:54,  7.19s/it]
 28%|██▊       | 64097/225000 [131:17:08<314:41:19,  7.04s/it]
 28%|██▊       | 64098/225000 [131:17:13<277:26:15,  6.21s/it]
 28%|██▊       | 64099/225000 [131:17:21<312:04:34,  6.98s/it]
 28%|██▊       | 64100/225000 [131:17:28<311:49:56,  6.98s/it]
                                                              
{'loss': '0.7641', 'grad_norm': '0.9929', 'learning_rate': '4.26e-05', 'epoch': '0.2849', 'num_input_tokens_seen': 218252384, 'train_runtime': '4.726e+05', 'train_tokens_per_second': '461.8'}

 28%|██▊       | 64100/225000 [131:17:28<311:49:56,  6.98s/it]
 28%|██▊       | 64101/225000 [131:17:33<275:32:56,  6.17s/it]
 28%|██▊       | 64102/225000 [131:17:42<314:13:07,  7.03s/it]
 28%|██▊       | 64103/225000 [131:17:45<263:57:29,  5.91s/it]
 28%|██▊       | 64104/225000 [131:17:50<247:15:21,  5.53s/it]
 28%|██▊       | 64105/225000 [131:17:55<241:16:12,  5.40s/it]
 28%|██▊       | 64106/225000 [131:18:00<241:00:47,  5.39s/it]
 28%|██▊       | 64107/225000 [131:18:06<245:05:08,  5.48s/it]
 28%|██▊       | 64108/225000 [131:18:11<247:09:10,  5.53s/it]
 28%|██▊       | 64109/225000 [131:18:23<325:17:28,  7.28s/it]
 28%|██▊       | 64110/225000 [131:18:30<324:24:37,  7.26s/it]
 28%|██▊       | 64111/225000 [131:18:36<308:02:45,  6.89s/it]
 28%|██▊       | 64112/225000 [131:18:44<317:23:40,  7.10s/it]
 28%|██▊       | 64113/225000 [131:18:48<279:40:29,  6.26s/it]
 28%|██▊       | 64114/225000 [131:18:52<251:19:40,  5.62s/it]
 28%|██▊       | 64115/225000 [131:18:56<233:38:33,  5.23s/it]
 28%|██▊       | 64116/225000 [131:19:06<299:00:46,  6.69s/it]
 28%|██▊       | 64117/225000 [131:19:12<290:34:52,  6.50s/it]
 28%|██▊       | 64118/225000 [131:19:22<330:45:37,  7.40s/it]
 28%|██▊       | 64119/225000 [131:19:25<278:50:45,  6.24s/it]
 28%|██▊       | 64120/225000 [131:19:33<301:40:41,  6.75s/it]
 28%|██▊       | 64121/225000 [131:19:40<301:33:40,  6.75s/it]
 28%|██▊       | 64122/225000 [131:19:45<270:25:06,  6.05s/it]
 28%|██▊       | 64123/225000 [131:19:49<248:00:55,  5.55s/it]
 28%|██▊       | 64124/225000 [131:19:54<244:05:17,  5.46s/it]
 28%|██▊       | 64125/225000 [131:20:01<257:35:43,  5.76s/it]
 29%|██▊       | 64126/225000 [131:20:06<250:31:39,  5.61s/it]
 29%|██▊       | 64127/225000 [131:20:12<253:31:29,  5.67s/it]
 29%|██▊       | 64128/225000 [131:20:17<249:33:57,  5.58s/it]
 29%|██▊       | 64129/225000 [131:20:23<247:08:26,  5.53s/it]
 29%|██▊       | 64130/225000 [131:20:36<349:28:02,  7.82s/it]
 29%|██▊       | 64131/225000 [131:20:43<339:59:47,  7.61s/it]
 29%|██▊       | 64132/225000 [131:20:46<286:42:26,  6.42s/it]
 29%|██▊       | 64133/225000 [131:20:56<333:34:27,  7.46s/it]
 29%|██▊       | 64134/225000 [131:21:01<297:33:52,  6.66s/it]
 29%|██▊       | 64135/225000 [131:21:08<305:27:12,  6.84s/it]
 29%|██▊       | 64136/225000 [131:21:12<268:02:57,  6.00s/it]
 29%|██▊       | 64137/225000 [131:21:21<301:12:25,  6.74s/it]
 29%|██▊       | 64138/225000 [131:21:25<265:55:32,  5.95s/it]
 29%|██▊       | 64139/225000 [131:21:34<312:58:58,  7.00s/it]
 29%|██▊       | 64140/225000 [131:21:42<314:56:01,  7.05s/it]
 29%|██▊       | 64141/225000 [131:21:46<275:21:01,  6.16s/it]
 29%|██▊       | 64142/225000 [131:21:50<248:42:31,  5.57s/it]
 29%|██▊       | 64143/225000 [131:22:03<343:29:52,  7.69s/it]
 29%|██▊       | 64144/225000 [131:22:07<293:53:20,  6.58s/it]
 29%|██▊       | 64145/225000 [131:22:14<310:40:20,  6.95s/it]
 29%|██▊       | 64146/225000 [131:22:18<268:12:58,  6.00s/it]
 29%|██▊       | 64147/225000 [131:22:22<241:58:55,  5.42s/it]
 29%|██▊       | 64148/225000 [131:22:33<307:53:46,  6.89s/it]
 29%|██▊       | 64149/225000 [131:22:40<310:28:48,  6.95s/it]
 29%|██▊       | 64150/225000 [131:22:43<267:14:40,  5.98s/it]
                                                              
{'loss': '0.7652', 'grad_norm': '0.9313', 'learning_rate': '4.258e-05', 'epoch': '0.2851', 'num_input_tokens_seen': 218396712, 'train_runtime': '4.73e+05', 'train_tokens_per_second': '461.8'}

 29%|██▊       | 64150/225000 [131:22:43<267:14:40,  5.98s/it]
 29%|██▊       | 64151/225000 [131:22:48<245:14:42,  5.49s/it]
 29%|██▊       | 64152/225000 [131:22:59<321:03:52,  7.19s/it]
 29%|██▊       | 64153/225000 [131:23:03<276:43:20,  6.19s/it]
 29%|██▊       | 64154/225000 [131:23:07<255:37:48,  5.72s/it]
 29%|██▊       | 64155/225000 [131:23:16<296:42:23,  6.64s/it]
 29%|██▊       | 64156/225000 [131:23:26<336:18:20,  7.53s/it]
 29%|██▊       | 64157/225000 [131:23:31<302:32:27,  6.77s/it]
 29%|██▊       | 64158/225000 [131:23:36<277:22:04,  6.21s/it]
 29%|██▊       | 64159/225000 [131:23:41<261:42:26,  5.86s/it]
 29%|██▊       | 64160/225000 [131:23:45<243:56:50,  5.46s/it]
 29%|██▊       | 64161/225000 [131:23:51<245:48:23,  5.50s/it]
 29%|██▊       | 64162/225000 [131:23:56<242:32:11,  5.43s/it]
 29%|██▊       | 64163/225000 [131:24:01<241:33:33,  5.41s/it]
 29%|██▊       | 64164/225000 [131:24:07<238:27:05,  5.34s/it]
 29%|██▊       | 64165/225000 [131:24:20<347:39:47,  7.78s/it]
 29%|██▊       | 64166/225000 [131:24:26<319:02:29,  7.14s/it]
 29%|██▊       | 64167/225000 [131:24:34<333:52:44,  7.47s/it]
 29%|██▊       | 64168/225000 [131:24:38<283:16:03,  6.34s/it]
 29%|██▊       | 64169/225000 [131:24:47<321:15:06,  7.19s/it]
 29%|██▊       | 64170/225000 [131:24:53<305:15:03,  6.83s/it]
 29%|██▊       | 64171/225000 [131:24:58<281:44:28,  6.31s/it]
 29%|██▊       | 64172/225000 [131:25:02<254:43:55,  5.70s/it]
 29%|██▊       | 64173/225000 [131:25:13<317:06:40,  7.10s/it]
 29%|██▊       | 64174/225000 [131:25:17<281:20:53,  6.30s/it]
 29%|██▊       | 64175/225000 [131:25:22<261:03:55,  5.84s/it]
 29%|██▊       | 64176/225000 [131:25:26<239:13:24,  5.35s/it]
 29%|██▊       | 64177/225000 [131:25:38<331:10:45,  7.41s/it]
 29%|██▊       | 64178/225000 [131:25:43<291:30:55,  6.53s/it]
 29%|██▊       | 64179/225000 [131:25:50<307:08:15,  6.88s/it]
 29%|██▊       | 64180/225000 [131:25:55<270:53:09,  6.06s/it]
 29%|██▊       | 64181/225000 [131:26:01<274:23:23,  6.14s/it]
 29%|██▊       | 64182/225000 [131:26:11<323:15:19,  7.24s/it]
 29%|██▊       | 64183/225000 [131:26:15<283:31:09,  6.35s/it]
 29%|██▊       | 64184/225000 [131:26:20<271:35:14,  6.08s/it]
 29%|██▊       | 64185/225000 [131:26:31<335:36:30,  7.51s/it]
 29%|██▊       | 64186/225000 [131:26:35<288:29:52,  6.46s/it]
 29%|██▊       | 64187/225000 [131:26:40<263:23:47,  5.90s/it]
 29%|██▊       | 64188/225000 [131:26:45<249:31:23,  5.59s/it]
 29%|██▊       | 64189/225000 [131:26:49<238:42:11,  5.34s/it]
 29%|██▊       | 64190/225000 [131:26:54<233:54:56,  5.24s/it]
 29%|██▊       | 64191/225000 [131:27:00<232:25:40,  5.20s/it]
 29%|██▊       | 64192/225000 [131:27:04<227:43:35,  5.10s/it]
 29%|██▊       | 64193/225000 [131:27:17<325:41:04,  7.29s/it]
 29%|██▊       | 64194/225000 [131:27:22<303:05:08,  6.79s/it]
 29%|██▊       | 64195/225000 [131:27:31<321:30:48,  7.20s/it]
 29%|██▊       | 64196/225000 [131:27:36<296:42:52,  6.64s/it]
 29%|██▊       | 64197/225000 [131:27:42<282:36:22,  6.33s/it]
 29%|██▊       | 64198/225000 [131:27:47<270:34:38,  6.06s/it]
 29%|██▊       | 64199/225000 [131:27:52<260:00:02,  5.82s/it]
 29%|██▊       | 64200/225000 [131:27:58<254:35:31,  5.70s/it]
                                                              
{'loss': '0.7397', 'grad_norm': '0.7492', 'learning_rate': '4.257e-05', 'epoch': '0.2853', 'num_input_tokens_seen': 218541208, 'train_runtime': '4.733e+05', 'train_tokens_per_second': '461.8'}

 29%|██▊       | 64200/225000 [131:27:58<254:35:31,  5.70s/it]
 29%|██▊       | 64201/225000 [131:28:04<260:14:36,  5.83s/it]
 29%|██▊       | 64202/225000 [131:28:09<259:03:32,  5.80s/it]
 29%|██▊       | 64203/225000 [131:28:15<256:19:06,  5.74s/it]
 29%|██▊       | 64204/225000 [131:28:21<260:39:26,  5.84s/it]
 29%|██▊       | 64205/225000 [131:28:27<261:49:03,  5.86s/it]
 29%|██▊       | 64206/225000 [131:28:40<361:59:08,  8.10s/it]
 29%|██▊       | 64207/225000 [131:28:47<339:20:44,  7.60s/it]
 29%|██▊       | 64208/225000 [131:28:52<303:13:23,  6.79s/it]
 29%|██▊       | 64209/225000 [131:28:59<312:25:15,  6.99s/it]
 29%|██▊       | 64210/225000 [131:29:06<309:30:34,  6.93s/it]
 29%|██▊       | 64211/225000 [131:29:11<289:06:39,  6.47s/it]
 29%|██▊       | 64212/225000 [131:29:16<261:03:57,  5.85s/it]
 29%|██▊       | 64213/225000 [131:29:20<241:44:16,  5.41s/it]
 29%|██▊       | 64214/225000 [131:29:26<241:34:06,  5.41s/it]
 29%|██▊       | 64215/225000 [131:29:39<346:06:57,  7.75s/it]
 29%|██▊       | 64216/225000 [131:29:43<293:59:02,  6.58s/it]
 29%|██▊       | 64217/225000 [131:29:48<275:42:09,  6.17s/it]
 29%|██▊       | 64218/225000 [131:29:53<259:57:10,  5.82s/it]
 29%|██▊       | 64219/225000 [131:29:58<254:03:08,  5.69s/it]
 29%|██▊       | 64220/225000 [131:30:12<363:37:34,  8.14s/it]
 29%|██▊       | 64221/225000 [131:30:17<322:01:20,  7.21s/it]
 29%|██▊       | 64222/225000 [131:30:21<283:41:50,  6.35s/it]
 29%|██▊       | 64223/225000 [131:30:29<300:12:00,  6.72s/it]
 29%|██▊       | 64224/225000 [131:30:36<308:38:35,  6.91s/it]
 29%|██▊       | 64225/225000 [131:30:41<275:02:39,  6.16s/it]
 29%|██▊       | 64226/225000 [131:30:48<287:35:29,  6.44s/it]
 29%|██▊       | 64227/225000 [131:30:55<302:38:11,  6.78s/it]
 29%|██▊       | 64228/225000 [131:31:00<275:05:43,  6.16s/it]
 29%|██▊       | 64229/225000 [131:31:05<261:34:45,  5.86s/it]
 29%|██▊       | 64230/225000 [131:31:10<248:46:28,  5.57s/it]
 29%|██▊       | 64231/225000 [131:31:15<241:24:50,  5.41s/it]
 29%|██▊       | 64232/225000 [131:31:20<233:46:05,  5.23s/it]
 29%|██▊       | 64233/225000 [131:31:33<333:41:41,  7.47s/it]
 29%|██▊       | 64234/225000 [131:31:40<324:39:27,  7.27s/it]
 29%|██▊       | 64235/225000 [131:31:45<305:07:30,  6.83s/it]
 29%|██▊       | 64236/225000 [131:31:51<287:39:58,  6.44s/it]
 29%|██▊       | 64237/225000 [131:31:57<275:51:32,  6.18s/it]
 29%|██▊       | 64238/225000 [131:32:02<268:26:18,  6.01s/it]
 29%|██▊       | 64239/225000 [131:32:07<255:18:24,  5.72s/it]
 29%|██▊       | 64240/225000 [131:32:13<254:25:29,  5.70s/it]
 29%|██▊       | 64241/225000 [131:32:19<256:17:48,  5.74s/it]
 29%|██▊       | 64242/225000 [131:32:29<319:23:57,  7.15s/it]
 29%|██▊       | 64243/225000 [131:32:39<357:38:53,  8.01s/it]
 29%|██▊       | 64244/225000 [131:32:43<301:38:21,  6.75s/it]
 29%|██▊       | 64245/225000 [131:32:47<268:32:53,  6.01s/it]
 29%|██▊       | 64246/225000 [131:32:57<323:56:48,  7.25s/it]
 29%|██▊       | 64247/225000 [131:33:03<302:03:05,  6.76s/it]
 29%|██▊       | 64248/225000 [131:33:07<270:08:10,  6.05s/it]
 29%|██▊       | 64249/225000 [131:33:11<242:16:24,  5.43s/it]
 29%|██▊       | 64250/225000 [131:33:23<324:15:54,  7.26s/it]
                                                              
{'loss': '0.752', 'grad_norm': '0.8334', 'learning_rate': '4.256e-05', 'epoch': '0.2856', 'num_input_tokens_seen': 218687384, 'train_runtime': '4.736e+05', 'train_tokens_per_second': '461.8'}

 29%|██▊       | 64250/225000 [131:33:23<324:15:54,  7.26s/it]
 29%|██▊       | 64251/225000 [131:33:27<286:18:31,  6.41s/it]
 29%|██▊       | 64252/225000 [131:33:36<312:34:21,  7.00s/it]
 29%|██▊       | 64253/225000 [131:33:40<270:13:07,  6.05s/it]
 29%|██▊       | 64254/225000 [131:33:50<325:16:55,  7.28s/it]
 29%|██▊       | 64255/225000 [131:33:53<277:40:02,  6.22s/it]
 29%|██▊       | 64256/225000 [131:34:02<310:52:05,  6.96s/it]
 29%|██▊       | 64257/225000 [131:34:05<262:27:33,  5.88s/it]
 29%|██▊       | 64258/225000 [131:34:10<239:44:05,  5.37s/it]
 29%|██▊       | 64259/225000 [131:34:14<225:37:20,  5.05s/it]
 29%|██▊       | 64260/225000 [131:34:22<261:09:54,  5.85s/it]
 29%|██▊       | 64261/225000 [131:34:32<325:37:10,  7.29s/it]
 29%|██▊       | 64262/225000 [131:34:37<283:50:54,  6.36s/it]
 29%|██▊       | 64263/225000 [131:34:41<264:21:19,  5.92s/it]
 29%|██▊       | 64264/225000 [131:34:47<257:12:11,  5.76s/it]
 29%|██▊       | 64265/225000 [131:34:52<249:48:20,  5.59s/it]
 29%|██▊       | 64266/225000 [131:34:57<242:59:28,  5.44s/it]
 29%|██▊       | 64267/225000 [131:35:03<255:57:12,  5.73s/it]
 29%|██▊       | 64268/225000 [131:35:17<362:04:30,  8.11s/it]
 29%|██▊       | 64269/225000 [131:35:24<349:49:54,  7.84s/it]
 29%|██▊       | 64270/225000 [131:35:29<302:47:18,  6.78s/it]
 29%|██▊       | 64271/225000 [131:35:37<321:31:21,  7.20s/it]
 29%|██▊       | 64272/225000 [131:35:42<287:29:24,  6.44s/it]
 29%|██▊       | 64273/225000 [131:35:46<262:16:44,  5.87s/it]
 29%|██▊       | 64274/225000 [131:35:50<239:20:43,  5.36s/it]
 29%|██▊       | 64275/225000 [131:36:03<334:07:15,  7.48s/it]
 29%|██▊       | 64276/225000 [131:36:06<282:32:44,  6.33s/it]
 29%|██▊       | 64277/225000 [131:36:11<262:53:48,  5.89s/it]
 29%|██▊       | 64278/225000 [131:36:16<242:39:52,  5.44s/it]
 29%|██▊       | 64279/225000 [131:36:21<237:57:01,  5.33s/it]
 29%|██▊       | 64280/225000 [131:36:26<235:17:48,  5.27s/it]
 29%|██▊       | 64281/225000 [131:36:38<323:19:07,  7.24s/it]
 29%|██▊       | 64282/225000 [131:36:44<318:17:41,  7.13s/it]
 29%|██▊       | 64283/225000 [131:36:50<291:46:25,  6.54s/it]
 29%|██▊       | 64284/225000 [131:36:56<295:36:49,  6.62s/it]
 29%|██▊       | 64285/225000 [131:37:01<269:39:13,  6.04s/it]
 29%|██▊       | 64286/225000 [131:37:05<246:09:31,  5.51s/it]
 29%|██▊       | 64287/225000 [131:37:10<237:26:54,  5.32s/it]
 29%|██▊       | 64288/225000 [131:37:15<235:41:30,  5.28s/it]
 29%|██▊       | 64289/225000 [131:37:29<341:44:37,  7.66s/it]
 29%|██▊       | 64290/225000 [131:37:35<323:59:39,  7.26s/it]
 29%|██▊       | 64291/225000 [131:37:40<292:04:14,  6.54s/it]
 29%|██▊       | 64292/225000 [131:37:45<270:16:02,  6.05s/it]
 29%|██▊       | 64293/225000 [131:37:49<250:45:32,  5.62s/it]
 29%|██▊       | 64294/225000 [131:37:54<240:57:02,  5.40s/it]
 29%|██▊       | 64295/225000 [131:37:59<232:59:46,  5.22s/it]
 29%|██▊       | 64296/225000 [131:38:12<341:56:45,  7.66s/it]
 29%|██▊       | 64297/225000 [131:38:19<323:08:46,  7.24s/it]
 29%|██▊       | 64298/225000 [131:38:26<318:05:23,  7.13s/it]
 29%|██▊       | 64299/225000 [131:38:32<312:17:27,  7.00s/it]
 29%|██▊       | 64300/225000 [131:38:39<308:14:32,  6.91s/it]
                                                              
{'loss': '0.749', 'grad_norm': '0.7307', 'learning_rate': '4.254e-05', 'epoch': '0.2858', 'num_input_tokens_seen': 218832648, 'train_runtime': '4.739e+05', 'train_tokens_per_second': '461.8'}

 29%|██▊       | 64300/225000 [131:38:39<308:14:32,  6.91s/it]
 29%|██▊       | 64301/225000 [131:38:43<272:22:02,  6.10s/it]
 29%|██▊       | 64302/225000 [131:38:47<242:01:42,  5.42s/it]
 29%|██▊       | 64303/225000 [131:38:57<307:23:31,  6.89s/it]
 29%|██▊       | 64304/225000 [131:39:01<267:16:21,  5.99s/it]
 29%|██▊       | 64305/225000 [131:39:10<304:53:59,  6.83s/it]
 29%|██▊       | 64306/225000 [131:39:14<265:21:56,  5.94s/it]
 29%|██▊       | 64307/225000 [131:39:23<305:40:29,  6.85s/it]
 29%|██▊       | 64308/225000 [131:39:26<261:57:02,  5.87s/it]
 29%|██▊       | 64309/225000 [131:39:36<308:08:16,  6.90s/it]
 29%|██▊       | 64310/225000 [131:39:42<293:16:14,  6.57s/it]
 29%|██▊       | 64311/225000 [131:39:46<263:30:17,  5.90s/it]
 29%|██▊       | 64312/225000 [131:39:53<275:43:38,  6.18s/it]
 29%|██▊       | 64313/225000 [131:40:02<321:34:49,  7.20s/it]
 29%|██▊       | 64314/225000 [131:40:06<273:44:13,  6.13s/it]
 29%|██▊       | 64315/225000 [131:40:10<246:24:44,  5.52s/it]
 29%|██▊       | 64316/225000 [131:40:15<236:57:19,  5.31s/it]
 29%|██▊       | 64317/225000 [131:40:24<293:51:04,  6.58s/it]
 29%|██▊       | 64318/225000 [131:40:32<303:28:27,  6.80s/it]
 29%|██▊       | 64319/225000 [131:40:40<319:39:19,  7.16s/it]
 29%|██▊       | 64320/225000 [131:40:43<274:47:50,  6.16s/it]
 29%|██▊       | 64321/225000 [131:40:53<318:26:48,  7.13s/it]
 29%|██▊       | 64322/225000 [131:40:56<270:49:53,  6.07s/it]
 29%|██▊       | 64323/225000 [131:41:01<252:01:57,  5.65s/it]
 29%|██▊       | 64324/225000 [131:41:06<239:23:23,  5.36s/it]
 29%|██▊       | 64325/225000 [131:41:15<287:51:09,  6.45s/it]
 29%|██▊       | 64326/225000 [131:41:23<304:35:03,  6.82s/it]
 29%|██▊       | 64327/225000 [131:41:30<318:33:09,  7.14s/it]
 29%|██▊       | 64328/225000 [131:41:37<308:11:50,  6.91s/it]
 29%|██▊       | 64329/225000 [131:41:41<265:49:59,  5.96s/it]
 29%|██▊       | 64330/225000 [131:41:50<308:26:45,  6.91s/it]
 29%|██▊       | 64331/225000 [131:41:54<270:53:59,  6.07s/it]
 29%|██▊       | 64332/225000 [131:41:58<245:09:15,  5.49s/it]
 29%|██▊       | 64333/225000 [131:42:07<297:17:33,  6.66s/it]
 29%|██▊       | 64334/225000 [131:42:12<277:08:04,  6.21s/it]
 29%|██▊       | 64335/225000 [131:42:21<304:20:19,  6.82s/it]
 29%|██▊       | 64336/225000 [131:42:25<266:34:46,  5.97s/it]
 29%|██▊       | 64337/225000 [131:42:34<310:56:01,  6.97s/it]
 29%|██▊       | 64338/225000 [131:42:38<272:39:45,  6.11s/it]
 29%|██▊       | 64339/225000 [131:42:46<300:45:13,  6.74s/it]
 29%|██▊       | 64340/225000 [131:42:51<272:44:45,  6.11s/it]
 29%|██▊       | 64341/225000 [131:42:57<268:36:13,  6.02s/it]
 29%|██▊       | 64342/225000 [131:43:05<300:11:16,  6.73s/it]
 29%|██▊       | 64343/225000 [131:43:10<273:17:04,  6.12s/it]
 29%|██▊       | 64344/225000 [131:43:15<254:51:18,  5.71s/it]
 29%|██▊       | 64345/225000 [131:43:20<245:00:19,  5.49s/it]
 29%|██▊       | 64346/225000 [131:43:31<320:32:07,  7.18s/it]
 29%|██▊       | 64347/225000 [131:43:38<324:11:21,  7.26s/it]
 29%|██▊       | 64348/225000 [131:43:43<294:46:23,  6.61s/it]
 29%|██▊       | 64349/225000 [131:43:49<279:26:06,  6.26s/it]
 29%|██▊       | 64350/225000 [131:43:54<273:09:58,  6.12s/it]
                                                              
{'loss': '0.7522', 'grad_norm': '1.041', 'learning_rate': '4.253e-05', 'epoch': '0.286', 'num_input_tokens_seen': 218977664, 'train_runtime': '4.742e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64350/225000 [131:43:54<273:09:58,  6.12s/it]
 29%|██▊       | 64351/225000 [131:44:00<258:42:38,  5.80s/it]
 29%|██▊       | 64352/225000 [131:44:05<259:01:45,  5.80s/it]
 29%|██▊       | 64353/225000 [131:44:11<257:18:29,  5.77s/it]
 29%|██▊       | 64354/225000 [131:44:16<250:39:35,  5.62s/it]
 29%|██▊       | 64355/225000 [131:44:21<244:32:15,  5.48s/it]
 29%|██▊       | 64356/225000 [131:44:30<290:57:53,  6.52s/it]
 29%|██▊       | 64357/225000 [131:44:42<359:21:48,  8.05s/it]
 29%|██▊       | 64358/225000 [131:44:49<350:40:34,  7.86s/it]
 29%|██▊       | 64359/225000 [131:44:54<302:47:29,  6.79s/it]
 29%|██▊       | 64360/225000 [131:45:02<320:26:02,  7.18s/it]
 29%|██▊       | 64361/225000 [131:45:05<269:11:46,  6.03s/it]
 29%|██▊       | 64362/225000 [131:45:16<329:19:14,  7.38s/it]
 29%|██▊       | 64363/225000 [131:45:19<277:04:32,  6.21s/it]
 29%|██▊       | 64364/225000 [131:45:28<307:22:19,  6.89s/it]
 29%|██▊       | 64365/225000 [131:45:32<270:52:00,  6.07s/it]
 29%|██▊       | 64366/225000 [131:45:36<248:13:02,  5.56s/it]
 29%|██▊       | 64367/225000 [131:45:41<237:11:58,  5.32s/it]
 29%|██▊       | 64368/225000 [131:45:54<339:35:53,  7.61s/it]
 29%|██▊       | 64369/225000 [131:46:00<322:37:37,  7.23s/it]
 29%|██▊       | 64370/225000 [131:46:04<280:36:45,  6.29s/it]
 29%|██▊       | 64371/225000 [131:46:12<298:53:27,  6.70s/it]
 29%|██▊       | 64372/225000 [131:46:21<324:09:54,  7.27s/it]
 29%|██▊       | 64373/225000 [131:46:26<294:02:08,  6.59s/it]
 29%|██▊       | 64374/225000 [131:46:31<276:13:49,  6.19s/it]
 29%|██▊       | 64375/225000 [131:46:36<261:30:18,  5.86s/it]
 29%|██▊       | 64376/225000 [131:46:41<254:35:37,  5.71s/it]
 29%|██▊       | 64377/225000 [131:46:52<326:33:42,  7.32s/it]
 29%|██▊       | 64378/225000 [131:47:01<341:56:01,  7.66s/it]
 29%|██▊       | 64379/225000 [131:47:06<302:43:57,  6.79s/it]
 29%|██▊       | 64380/225000 [131:47:13<308:46:55,  6.92s/it]
 29%|██▊       | 64381/225000 [131:47:17<270:29:09,  6.06s/it]
 29%|██▊       | 64382/225000 [131:47:21<249:59:33,  5.60s/it]
 29%|██▊       | 64383/225000 [131:47:31<298:33:32,  6.69s/it]
 29%|██▊       | 64384/225000 [131:47:35<267:23:17,  5.99s/it]
 29%|██▊       | 64385/225000 [131:47:40<252:36:50,  5.66s/it]
 29%|██▊       | 64386/225000 [131:47:45<244:04:27,  5.47s/it]
 29%|██▊       | 64387/225000 [131:47:52<260:30:00,  5.84s/it]
 29%|██▊       | 64388/225000 [131:48:03<341:27:42,  7.65s/it]
 29%|██▊       | 64389/225000 [131:48:07<286:42:48,  6.43s/it]
 29%|██▊       | 64390/225000 [131:48:16<325:44:10,  7.30s/it]
 29%|██▊       | 64391/225000 [131:48:22<300:03:29,  6.73s/it]
 29%|██▊       | 64392/225000 [131:48:27<280:20:05,  6.28s/it]
 29%|██▊       | 64393/225000 [131:48:32<266:14:58,  5.97s/it]
 29%|██▊       | 64394/225000 [131:48:38<261:07:02,  5.85s/it]
 29%|██▊       | 64395/225000 [131:48:43<253:35:50,  5.68s/it]
 29%|██▊       | 64396/225000 [131:48:49<250:45:59,  5.62s/it]
 29%|██▊       | 64397/225000 [131:48:54<244:40:51,  5.48s/it]
 29%|██▊       | 64398/225000 [131:48:59<242:25:07,  5.43s/it]
 29%|██▊       | 64399/225000 [131:49:04<241:08:05,  5.41s/it]
 29%|██▊       | 64400/225000 [131:49:17<342:56:13,  7.69s/it]
                                                              
{'loss': '0.7442', 'grad_norm': '0.7125', 'learning_rate': '4.252e-05', 'epoch': '0.2862', 'num_input_tokens_seen': 219124816, 'train_runtime': '4.746e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64400/225000 [131:49:17<342:56:13,  7.69s/it]
 29%|██▊       | 64401/225000 [131:49:25<338:57:59,  7.60s/it]
 29%|██▊       | 64402/225000 [131:49:32<338:26:15,  7.59s/it]
 29%|██▊       | 64403/225000 [131:49:38<314:24:11,  7.05s/it]
 29%|██▊       | 64404/225000 [131:49:45<307:44:21,  6.90s/it]
 29%|██▊       | 64405/225000 [131:49:50<290:28:52,  6.51s/it]
 29%|██▊       | 64406/225000 [131:49:56<274:27:32,  6.15s/it]
 29%|██▊       | 64407/225000 [131:50:02<271:15:45,  6.08s/it]
 29%|██▊       | 64408/225000 [131:50:07<264:55:06,  5.94s/it]
 29%|██▊       | 64409/225000 [131:50:13<258:39:19,  5.80s/it]
 29%|██▊       | 64410/225000 [131:50:18<250:19:52,  5.61s/it]
 29%|██▊       | 64411/225000 [131:50:31<346:57:37,  7.78s/it]
 29%|██▊       | 64412/225000 [131:50:35<298:58:00,  6.70s/it]
 29%|██▊       | 64413/225000 [131:50:40<275:40:12,  6.18s/it]
 29%|██▊       | 64414/225000 [131:50:44<254:08:02,  5.70s/it]
 29%|██▊       | 64415/225000 [131:50:53<296:45:16,  6.65s/it]
 29%|██▊       | 64416/225000 [131:51:02<320:19:06,  7.18s/it]
 29%|██▊       | 64417/225000 [131:51:08<312:33:55,  7.01s/it]
 29%|██▊       | 64418/225000 [131:51:15<312:11:27,  7.00s/it]
 29%|██▊       | 64419/225000 [131:51:20<283:21:06,  6.35s/it]
 29%|██▊       | 64420/225000 [131:51:26<271:48:04,  6.09s/it]
 29%|██▊       | 64421/225000 [131:51:31<259:46:08,  5.82s/it]
 29%|██▊       | 64422/225000 [131:51:38<282:37:43,  6.34s/it]
 29%|██▊       | 64423/225000 [131:51:46<304:19:20,  6.82s/it]
 29%|██▊       | 64424/225000 [131:51:54<319:47:17,  7.17s/it]
 29%|██▊       | 64425/225000 [131:51:59<292:45:24,  6.56s/it]
 29%|██▊       | 64426/225000 [131:52:05<273:35:21,  6.13s/it]
 29%|██▊       | 64427/225000 [131:52:10<261:18:34,  5.86s/it]
 29%|██▊       | 64428/225000 [131:52:14<243:29:50,  5.46s/it]
 29%|██▊       | 64429/225000 [131:52:24<300:03:50,  6.73s/it]
 29%|██▊       | 64430/225000 [131:52:33<328:26:37,  7.36s/it]
 29%|██▊       | 64431/225000 [131:52:37<284:31:52,  6.38s/it]
 29%|██▊       | 64432/225000 [131:52:46<319:04:28,  7.15s/it]
 29%|██▊       | 64433/225000 [131:52:51<286:07:40,  6.42s/it]
 29%|██▊       | 64434/225000 [131:52:55<266:42:12,  5.98s/it]
 29%|██▊       | 64435/225000 [131:53:05<307:20:52,  6.89s/it]
 29%|██▊       | 64436/225000 [131:53:08<267:11:31,  5.99s/it]
 29%|██▊       | 64437/225000 [131:53:17<305:05:54,  6.84s/it]
 29%|██▊       | 64438/225000 [131:53:26<331:40:28,  7.44s/it]
 29%|██▊       | 64439/225000 [131:53:31<301:59:01,  6.77s/it]
 29%|██▊       | 64440/225000 [131:53:36<274:20:29,  6.15s/it]
 29%|██▊       | 64441/225000 [131:53:41<260:51:53,  5.85s/it]
 29%|██▊       | 64442/225000 [131:53:46<250:13:26,  5.61s/it]
 29%|██▊       | 64443/225000 [131:53:51<239:41:42,  5.37s/it]
 29%|██▊       | 64444/225000 [131:53:57<253:59:28,  5.70s/it]
 29%|██▊       | 64445/225000 [131:54:10<342:03:14,  7.67s/it]
 29%|██▊       | 64446/225000 [131:54:16<318:21:12,  7.14s/it]
 29%|██▊       | 64447/225000 [131:54:21<297:21:57,  6.67s/it]
 29%|██▊       | 64448/225000 [131:54:26<267:43:04,  6.00s/it]
 29%|██▊       | 64449/225000 [131:54:33<288:52:52,  6.48s/it]
 29%|██▊       | 64450/225000 [131:54:41<309:06:10,  6.93s/it]
                                                              
{'loss': '0.7536', 'grad_norm': '0.9252', 'learning_rate': '4.25e-05', 'epoch': '0.2864', 'num_input_tokens_seen': 219269888, 'train_runtime': '4.749e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64450/225000 [131:54:41<309:06:10,  6.93s/it]
 29%|██▊       | 64451/225000 [131:54:48<301:50:25,  6.77s/it]
 29%|██▊       | 64452/225000 [131:54:55<316:02:43,  7.09s/it]
 29%|██▊       | 64453/225000 [131:54:59<269:48:54,  6.05s/it]
 29%|██▊       | 64454/225000 [131:55:05<263:33:47,  5.91s/it]
 29%|██▊       | 64455/225000 [131:55:14<312:33:38,  7.01s/it]
 29%|██▊       | 64456/225000 [131:55:19<278:20:21,  6.24s/it]
 29%|██▊       | 64457/225000 [131:55:23<255:33:48,  5.73s/it]
 29%|██▊       | 64458/225000 [131:55:35<339:45:56,  7.62s/it]
 29%|██▊       | 64459/225000 [131:55:39<287:10:56,  6.44s/it]
 29%|██▊       | 64460/225000 [131:55:48<327:49:55,  7.35s/it]
 29%|██▊       | 64461/225000 [131:55:52<279:49:05,  6.27s/it]
 29%|██▊       | 64462/225000 [131:55:56<246:40:48,  5.53s/it]
 29%|██▊       | 64463/225000 [131:56:00<232:20:05,  5.21s/it]
 29%|██▊       | 64464/225000 [131:56:09<276:10:16,  6.19s/it]
 29%|██▊       | 64465/225000 [131:56:17<303:37:04,  6.81s/it]
 29%|██▊       | 64466/225000 [131:56:23<286:53:44,  6.43s/it]
 29%|██▊       | 64467/225000 [131:56:28<272:44:07,  6.12s/it]
 29%|██▊       | 64468/225000 [131:56:34<271:31:01,  6.09s/it]
 29%|██▊       | 64469/225000 [131:56:40<265:12:39,  5.95s/it]
 29%|██▊       | 64470/225000 [131:56:45<253:35:30,  5.69s/it]
 29%|██▊       | 64471/225000 [131:56:51<256:02:12,  5.74s/it]
 29%|██▊       | 64472/225000 [131:56:55<243:25:06,  5.46s/it]
 29%|██▊       | 64473/225000 [131:57:05<300:14:48,  6.73s/it]
 29%|██▊       | 64474/225000 [131:57:18<381:35:23,  8.56s/it]
 29%|██▊       | 64475/225000 [131:57:21<312:56:41,  7.02s/it]
 29%|██▊       | 64476/225000 [131:57:30<331:24:51,  7.43s/it]
 29%|██▊       | 64477/225000 [131:57:34<294:20:20,  6.60s/it]
 29%|██▊       | 64478/225000 [131:57:40<281:45:45,  6.32s/it]
 29%|██▊       | 64479/225000 [131:57:49<313:07:47,  7.02s/it]
 29%|██▊       | 64480/225000 [131:57:53<281:22:45,  6.31s/it]
 29%|██▊       | 64481/225000 [131:57:58<257:34:36,  5.78s/it]
 29%|██▊       | 64482/225000 [131:58:05<274:50:33,  6.16s/it]
 29%|██▊       | 64483/225000 [131:58:16<335:25:11,  7.52s/it]
 29%|██▊       | 64484/225000 [131:58:20<285:50:36,  6.41s/it]
 29%|██▊       | 64485/225000 [131:58:25<266:24:23,  5.97s/it]
 29%|██▊       | 64486/225000 [131:58:35<322:31:09,  7.23s/it]
 29%|██▊       | 64487/225000 [131:58:40<302:35:02,  6.79s/it]
 29%|██▊       | 64488/225000 [131:58:44<263:58:36,  5.92s/it]
 29%|██▊       | 64489/225000 [131:58:50<256:29:00,  5.75s/it]
 29%|██▊       | 64490/225000 [131:58:55<255:22:14,  5.73s/it]
 29%|██▊       | 64491/225000 [131:59:01<255:49:48,  5.74s/it]
 29%|██▊       | 64492/225000 [131:59:12<321:55:06,  7.22s/it]
 29%|██▊       | 64493/225000 [131:59:16<283:44:12,  6.36s/it]
 29%|██▊       | 64494/225000 [131:59:21<260:53:30,  5.85s/it]
 29%|██▊       | 64495/225000 [131:59:25<238:09:28,  5.34s/it]
 29%|██▊       | 64496/225000 [131:59:38<344:13:47,  7.72s/it]
 29%|██▊       | 64497/225000 [131:59:43<308:47:43,  6.93s/it]
 29%|██▊       | 64498/225000 [131:59:51<313:47:34,  7.04s/it]
 29%|██▊       | 64499/225000 [131:59:57<298:46:27,  6.70s/it]
 29%|██▊       | 64500/225000 [132:00:03<299:54:04,  6.73s/it]
                                                              
{'loss': '0.7297', 'grad_norm': '0.8013', 'learning_rate': '4.249e-05', 'epoch': '0.2867', 'num_input_tokens_seen': 219414792, 'train_runtime': '4.752e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64500/225000 [132:00:03<299:54:04,  6.73s/it]
 29%|██▊       | 64501/225000 [132:00:07<260:03:54,  5.83s/it]
 29%|██▊       | 64502/225000 [132:00:17<308:44:35,  6.93s/it]
 29%|██▊       | 64503/225000 [132:00:21<282:21:30,  6.33s/it]
 29%|██▊       | 64504/225000 [132:00:27<270:41:07,  6.07s/it]
 29%|██▊       | 64505/225000 [132:00:32<255:31:06,  5.73s/it]
 29%|██▊       | 64506/225000 [132:00:38<256:03:53,  5.74s/it]
 29%|██▊       | 64507/225000 [132:00:43<255:00:45,  5.72s/it]
 29%|██▊       | 64508/225000 [132:00:49<249:29:27,  5.60s/it]
 29%|██▊       | 64509/225000 [132:00:55<253:47:14,  5.69s/it]
 29%|██▊       | 64510/225000 [132:01:00<252:43:20,  5.67s/it]
 29%|██▊       | 64511/225000 [132:01:08<277:18:47,  6.22s/it]
 29%|██▊       | 64512/225000 [132:01:21<371:54:29,  8.34s/it]
 29%|██▊       | 64513/225000 [132:01:24<306:44:30,  6.88s/it]
 29%|██▊       | 64514/225000 [132:01:33<334:11:22,  7.50s/it]
 29%|██▊       | 64515/225000 [132:01:38<291:10:01,  6.53s/it]
 29%|██▊       | 64516/225000 [132:01:46<313:59:37,  7.04s/it]
 29%|██▊       | 64517/225000 [132:01:52<303:01:59,  6.80s/it]
 29%|██▊       | 64518/225000 [132:01:56<264:34:16,  5.93s/it]
 29%|██▊       | 64519/225000 [132:02:05<311:26:56,  6.99s/it]
 29%|██▊       | 64520/225000 [132:02:13<323:37:44,  7.26s/it]
 29%|██▊       | 64521/225000 [132:02:19<297:26:07,  6.67s/it]
 29%|██▊       | 64522/225000 [132:02:24<275:47:12,  6.19s/it]
 29%|██▊       | 64523/225000 [132:02:29<259:10:23,  5.81s/it]
 29%|██▊       | 64524/225000 [132:02:33<245:15:37,  5.50s/it]
 29%|██▊       | 64525/225000 [132:02:42<289:49:09,  6.50s/it]
 29%|██▊       | 64526/225000 [132:02:52<329:18:39,  7.39s/it]
 29%|██▊       | 64527/225000 [132:02:56<286:48:49,  6.43s/it]
 29%|██▊       | 64528/225000 [132:03:00<257:42:21,  5.78s/it]
 29%|██▊       | 64529/225000 [132:03:06<251:13:52,  5.64s/it]
 29%|██▊       | 64530/225000 [132:03:11<244:58:06,  5.50s/it]
 29%|██▊       | 64531/225000 [132:03:20<295:41:33,  6.63s/it]
 29%|██▊       | 64532/225000 [132:03:31<351:58:26,  7.90s/it]
 29%|██▊       | 64533/225000 [132:03:35<297:21:35,  6.67s/it]
 29%|██▊       | 64534/225000 [132:03:44<335:49:45,  7.53s/it]
 29%|██▊       | 64535/225000 [132:03:48<282:11:09,  6.33s/it]
 29%|██▊       | 64536/225000 [132:03:56<303:58:23,  6.82s/it]
 29%|██▊       | 64537/225000 [132:04:00<267:37:22,  6.00s/it]
 29%|██▊       | 64538/225000 [132:04:05<263:38:18,  5.91s/it]
 29%|██▊       | 64539/225000 [132:04:15<310:03:29,  6.96s/it]
 29%|██▊       | 64540/225000 [132:04:22<313:07:35,  7.03s/it]
 29%|██▊       | 64541/225000 [132:04:28<297:29:24,  6.67s/it]
 29%|██▊       | 64542/225000 [132:04:36<314:34:01,  7.06s/it]
 29%|██▊       | 64543/225000 [132:04:41<292:05:32,  6.55s/it]
 29%|██▊       | 64544/225000 [132:04:46<269:42:50,  6.05s/it]
 29%|██▊       | 64545/225000 [132:04:51<254:22:39,  5.71s/it]
 29%|██▊       | 64546/225000 [132:04:56<249:07:20,  5.59s/it]
 29%|██▊       | 64547/225000 [132:05:02<244:33:29,  5.49s/it]
 29%|██▊       | 64548/225000 [132:05:07<244:25:00,  5.48s/it]
 29%|██▊       | 64549/225000 [132:05:14<264:39:09,  5.94s/it]
 29%|██▊       | 64550/225000 [132:05:27<364:19:29,  8.17s/it]
                                                              
{'loss': '0.7389', 'grad_norm': '0.7677', 'learning_rate': '4.248e-05', 'epoch': '0.2869', 'num_input_tokens_seen': 219561640, 'train_runtime': '4.755e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64550/225000 [132:05:27<364:19:29,  8.17s/it]
 29%|██▊       | 64551/225000 [132:05:33<334:17:53,  7.50s/it]
 29%|██▊       | 64552/225000 [132:05:39<303:01:47,  6.80s/it]
 29%|██▊       | 64553/225000 [132:05:44<285:09:30,  6.40s/it]
 29%|██▊       | 64554/225000 [132:05:49<268:51:57,  6.03s/it]
 29%|██▊       | 64555/225000 [132:05:54<259:24:44,  5.82s/it]
 29%|██▊       | 64556/225000 [132:05:59<246:27:58,  5.53s/it]
 29%|██▊       | 64557/225000 [132:06:10<312:37:22,  7.01s/it]
 29%|██▊       | 64558/225000 [132:06:19<337:21:46,  7.57s/it]
 29%|██▊       | 64559/225000 [132:06:26<328:52:03,  7.38s/it]
 29%|██▊       | 64560/225000 [132:06:30<287:13:19,  6.44s/it]
 29%|██▊       | 64561/225000 [132:06:35<273:16:43,  6.13s/it]
 29%|██▊       | 64562/225000 [132:06:41<262:14:42,  5.88s/it]
 29%|██▊       | 64563/225000 [132:06:46<251:31:36,  5.64s/it]
 29%|██▊       | 64564/225000 [132:06:55<306:33:53,  6.88s/it]
 29%|██▊       | 64565/225000 [132:07:03<316:56:43,  7.11s/it]
 29%|██▊       | 64566/225000 [132:07:09<295:47:26,  6.64s/it]
 29%|██▊       | 64567/225000 [132:07:14<281:55:40,  6.33s/it]
 29%|██▊       | 64568/225000 [132:07:19<257:36:53,  5.78s/it]
 29%|██▊       | 64569/225000 [132:07:27<286:58:17,  6.44s/it]
 29%|██▊       | 64570/225000 [132:07:35<310:11:16,  6.96s/it]
 29%|██▊       | 64571/225000 [132:07:39<273:15:26,  6.13s/it]
 29%|██▊       | 64572/225000 [132:07:44<258:30:44,  5.80s/it]
 29%|██▊       | 64573/225000 [132:07:56<338:09:39,  7.59s/it]
 29%|██▊       | 64574/225000 [132:08:03<329:19:32,  7.39s/it]
 29%|██▊       | 64575/225000 [132:08:10<321:58:23,  7.23s/it]
 29%|██▊       | 64576/225000 [132:08:16<304:38:02,  6.84s/it]
 29%|██▊       | 64577/225000 [132:08:25<335:07:10,  7.52s/it]
 29%|██▊       | 64578/225000 [132:08:29<294:21:04,  6.61s/it]
 29%|██▊       | 64579/225000 [132:08:33<254:45:37,  5.72s/it]
 29%|██▊       | 64580/225000 [132:08:44<324:56:30,  7.29s/it]
 29%|██▊       | 64581/225000 [132:08:48<287:14:11,  6.45s/it]
 29%|██▊       | 64582/225000 [132:08:56<311:14:00,  6.98s/it]
 29%|██▊       | 64583/225000 [132:09:00<266:13:34,  5.97s/it]
 29%|██▊       | 64584/225000 [132:09:08<292:45:15,  6.57s/it]
 29%|██▊       | 64585/225000 [132:09:16<307:08:40,  6.89s/it]
 29%|██▊       | 64586/225000 [132:09:20<269:57:27,  6.06s/it]
 29%|██▊       | 64587/225000 [132:09:25<252:03:10,  5.66s/it]
 29%|██▊       | 64588/225000 [132:09:35<321:33:40,  7.22s/it]
 29%|██▊       | 64589/225000 [132:09:41<303:09:53,  6.80s/it]
 29%|██▊       | 64590/225000 [132:09:47<286:34:40,  6.43s/it]
 29%|██▊       | 64591/225000 [132:09:55<316:37:36,  7.11s/it]
 29%|██▊       | 64592/225000 [132:10:00<286:14:05,  6.42s/it]
 29%|██▊       | 64593/225000 [132:10:05<265:19:29,  5.95s/it]
 29%|██▊       | 64594/225000 [132:10:10<253:27:27,  5.69s/it]
 29%|██▊       | 64595/225000 [132:10:15<238:51:31,  5.36s/it]
 29%|██▊       | 64596/225000 [132:10:20<234:54:40,  5.27s/it]
 29%|██▊       | 64597/225000 [132:10:32<329:18:12,  7.39s/it]
 29%|██▊       | 64598/225000 [132:10:39<318:15:04,  7.14s/it]
 29%|██▊       | 64599/225000 [132:10:46<319:43:46,  7.18s/it]
 29%|██▊       | 64600/225000 [132:10:51<283:50:27,  6.37s/it]
                                                              
{'loss': '0.7595', 'grad_norm': '0.7456', 'learning_rate': '4.246e-05', 'epoch': '0.2871', 'num_input_tokens_seen': 219709432, 'train_runtime': '4.759e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64600/225000 [132:10:51<283:50:27,  6.37s/it]
 29%|██▊       | 64601/225000 [132:11:01<343:20:50,  7.71s/it]
 29%|██▊       | 64602/225000 [132:11:05<289:08:14,  6.49s/it]
 29%|██▊       | 64603/225000 [132:11:11<280:07:47,  6.29s/it]
 29%|██▊       | 64604/225000 [132:11:16<271:08:47,  6.09s/it]
 29%|██▊       | 64605/225000 [132:11:22<261:28:49,  5.87s/it]
 29%|██▊       | 64606/225000 [132:11:27<258:52:07,  5.81s/it]
 29%|██▊       | 64607/225000 [132:11:39<340:33:58,  7.64s/it]
 29%|██▊       | 64608/225000 [132:11:45<316:03:55,  7.09s/it]
 29%|██▊       | 64609/225000 [132:11:49<269:50:37,  6.06s/it]
 29%|██▊       | 64610/225000 [132:11:57<292:26:33,  6.56s/it]
 29%|██▊       | 64611/225000 [132:12:04<309:10:40,  6.94s/it]
 29%|██▊       | 64612/225000 [132:12:09<275:53:23,  6.19s/it]
 29%|██▊       | 64613/225000 [132:12:17<303:13:36,  6.81s/it]
 29%|██▊       | 64614/225000 [132:12:24<310:20:43,  6.97s/it]
 29%|██▊       | 64615/225000 [132:12:29<278:58:23,  6.26s/it]
 29%|██▊       | 64616/225000 [132:12:34<255:45:37,  5.74s/it]
 29%|██▊       | 64617/225000 [132:12:41<272:43:09,  6.12s/it]
 29%|██▊       | 64618/225000 [132:12:49<299:37:01,  6.73s/it]
 29%|██▊       | 64619/225000 [132:12:52<259:47:37,  5.83s/it]
 29%|██▊       | 64620/225000 [132:13:03<323:30:55,  7.26s/it]
 29%|██▊       | 64621/225000 [132:13:07<272:35:50,  6.12s/it]
 29%|██▊       | 64622/225000 [132:13:14<287:26:40,  6.45s/it]
 29%|██▊       | 64623/225000 [132:13:22<311:41:05,  7.00s/it]
 29%|██▊       | 64624/225000 [132:13:26<267:11:30,  6.00s/it]
 29%|██▊       | 64625/225000 [132:13:30<244:17:28,  5.48s/it]
 29%|██▊       | 64626/225000 [132:13:41<317:15:30,  7.12s/it]
 29%|██▊       | 64627/225000 [132:13:45<274:25:50,  6.16s/it]
 29%|██▊       | 64628/225000 [132:13:54<311:25:43,  6.99s/it]
 29%|██▊       | 64629/225000 [132:14:02<329:28:36,  7.40s/it]
 29%|██▊       | 64630/225000 [132:14:06<285:59:08,  6.42s/it]
 29%|██▊       | 64631/225000 [132:14:11<258:54:51,  5.81s/it]
 29%|██▊       | 64632/225000 [132:14:16<250:54:38,  5.63s/it]
 29%|██▊       | 64633/225000 [132:14:23<266:42:16,  5.99s/it]
 29%|██▊       | 64634/225000 [132:14:34<332:39:03,  7.47s/it]
 29%|██▊       | 64635/225000 [132:14:37<278:04:04,  6.24s/it]
 29%|██▊       | 64636/225000 [132:14:46<315:31:34,  7.08s/it]
 29%|██▊       | 64637/225000 [132:14:50<276:31:50,  6.21s/it]
 29%|██▊       | 64638/225000 [132:14:56<268:13:52,  6.02s/it]
 29%|██▊       | 64639/225000 [132:15:01<258:36:28,  5.81s/it]
 29%|██▊       | 64640/225000 [132:15:06<252:13:19,  5.66s/it]
 29%|██▊       | 64641/225000 [132:15:12<250:37:57,  5.63s/it]
 29%|██▊       | 64642/225000 [132:15:17<247:40:45,  5.56s/it]
 29%|██▊       | 64643/225000 [132:15:26<288:44:02,  6.48s/it]
 29%|██▊       | 64644/225000 [132:15:38<367:50:03,  8.26s/it]
 29%|██▊       | 64645/225000 [132:15:42<309:41:47,  6.95s/it]
 29%|██▊       | 64646/225000 [132:15:48<299:15:16,  6.72s/it]
 29%|██▊       | 64647/225000 [132:15:57<317:24:28,  7.13s/it]
 29%|██▊       | 64648/225000 [132:16:01<287:54:50,  6.46s/it]
 29%|██▊       | 64649/225000 [132:16:06<268:50:54,  6.04s/it]
 29%|██▊       | 64650/225000 [132:16:12<260:22:39,  5.85s/it]
                                                              
{'loss': '0.7504', 'grad_norm': '0.7476', 'learning_rate': '4.245e-05', 'epoch': '0.2873', 'num_input_tokens_seen': 219856848, 'train_runtime': '4.762e+05', 'train_tokens_per_second': '461.7'}

 29%|██▊       | 64650/225000 [132:16:12<260:22:39,  5.85s/it]
 29%|██▊       | 64651/225000 [132:16:17<248:21:08,  5.58s/it]
 29%|██▊       | 64652/225000 [132:16:22<247:27:12,  5.56s/it]
 29%|██▊       | 64653/225000 [132:16:32<299:31:29,  6.72s/it]
 29%|██▊       | 64654/225000 [132:16:43<359:10:14,  8.06s/it]
 29%|██▊       | 64655/225000 [132:16:47<305:34:39,  6.86s/it]
 29%|██▊       | 64656/225000 [132:16:51<271:57:05,  6.11s/it]
 29%|██▊       | 64657/225000 [132:16:56<253:06:16,  5.68s/it]
 29%|██▊       | 64658/225000 [132:17:09<343:46:28,  7.72s/it]
 29%|██▊       | 64659/225000 [132:17:14<320:00:07,  7.18s/it]
 29%|██▊       | 64660/225000 [132:17:19<290:46:03,  6.53s/it]
 29%|██▊       | 64661/225000 [132:17:25<272:32:22,  6.12s/it]
 29%|██▊       | 64662/225000 [132:17:29<254:28:38,  5.71s/it]
 29%|██▊       | 64663/225000 [132:17:38<298:02:33,  6.69s/it]
 29%|██▊       | 64664/225000 [132:17:48<342:38:01,  7.69s/it]
 29%|██▊       | 64665/225000 [132:17:55<326:23:23,  7.33s/it]
 29%|██▊       | 64666/225000 [132:18:01<310:00:50,  6.96s/it]
 29%|██▊       | 64667/225000 [132:18:09<322:05:22,  7.23s/it]
 29%|██▊       | 64668/225000 [132:18:14<300:05:55,  6.74s/it]
 29%|██▊       | 64669/225000 [132:18:19<275:39:59,  6.19s/it]
 29%|██▊       | 64670/225000 [132:18:24<253:12:39,  5.69s/it]
 29%|██▊       | 64671/225000 [132:18:29<248:57:17,  5.59s/it]
 29%|██▊       | 64672/225000 [132:18:42<350:19:11,  7.87s/it]
 29%|██▊       | 64673/225000 [132:18:47<305:38:50,  6.86s/it]
 29%|██▊       | 64674/225000 [132:18:53<289:40:14,  6.50s/it]
 29%|██▊       | 64675/225000 [132:19:02<323:01:00,  7.25s/it]
 29%|██▊       | 64676/225000 [132:19:07<298:26:16,  6.70s/it]
 29%|██▊       | 64677/225000 [132:19:12<270:53:29,  6.08s/it]
 29%|██▊       | 64678/225000 [132:19:16<252:03:13,  5.66s/it]
 29%|██▊       | 64679/225000 [132:19:22<247:37:45,  5.56s/it]
 29%|██▊       | 64680/225000 [132:19:33<324:26:50,  7.29s/it]
 29%|██▊       | 64681/225000 [132:19:39<311:23:15,  6.99s/it]
 29%|██▊       | 64682/225000 [132:19:45<290:06:23,  6.51s/it]
 29%|██▊       | 64683/225000 [132:19:49<262:34:12,  5.90s/it]
 29%|██▊       | 64684/225000 [132:20:00<331:23:22,  7.44s/it]
 29%|██▊       | 64685/225000 [132:20:04<289:20:58,  6.50s/it]
 29%|██▊       | 64686/225000 [132:20:09<257:59:34,  5.79s/it]
 29%|██▊       | 64687/225000 [132:20:16<276:50:54,  6.22s/it]
 29%|██▉       | 64688/225000 [132:20:25<320:16:49,  7.19s/it]
 29%|██▉       | 64689/225000 [132:20:31<302:42:24,  6.80s/it]
 29%|██▉       | 64690/225000 [132:20:37<284:50:31,  6.40s/it]
 29%|██▉       | 64691/225000 [132:20:42<271:35:42,  6.10s/it]
 29%|██▉       | 64692/225000 [132:20:51<312:12:31,  7.01s/it]
 29%|██▉       | 64693/225000 [132:20:58<306:19:13,  6.88s/it]
 29%|██▉       | 64694/225000 [132:21:01<261:31:36,  5.87s/it]
 29%|██▉       | 64695/225000 [132:21:10<293:22:19,  6.59s/it]
 29%|██▉       | 64696/225000 [132:21:17<300:38:44,  6.75s/it]
 29%|██▉       | 64697/225000 [132:21:23<292:56:32,  6.58s/it]
 29%|██▉       | 64698/225000 [132:21:31<312:59:43,  7.03s/it]
 29%|██▉       | 64699/225000 [132:21:35<274:17:02,  6.16s/it]
 29%|██▉       | 64700/225000 [132:21:42<291:21:19,  6.54s/it]
                                                              
{'loss': '0.7527', 'grad_norm': '0.8023', 'learning_rate': '4.244e-05', 'epoch': '0.2876', 'num_input_tokens_seen': 220004128, 'train_runtime': '4.765e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64700/225000 [132:21:42<291:21:19,  6.54s/it]
 29%|██▉       | 64701/225000 [132:21:48<276:44:29,  6.22s/it]
 29%|██▉       | 64702/225000 [132:21:54<268:45:23,  6.04s/it]
 29%|██▉       | 64703/225000 [132:21:59<262:33:41,  5.90s/it]
 29%|██▉       | 64704/225000 [132:22:04<255:19:37,  5.73s/it]
 29%|██▉       | 64705/225000 [132:22:10<258:17:43,  5.80s/it]
 29%|██▉       | 64706/225000 [132:22:19<289:40:25,  6.51s/it]
 29%|██▉       | 64707/225000 [132:22:28<333:25:11,  7.49s/it]
 29%|██▉       | 64708/225000 [132:22:34<312:08:50,  7.01s/it]
 29%|██▉       | 64709/225000 [132:22:39<278:20:48,  6.25s/it]
 29%|██▉       | 64710/225000 [132:22:47<310:54:42,  6.98s/it]
 29%|██▉       | 64711/225000 [132:22:51<270:03:01,  6.07s/it]
 29%|██▉       | 64712/225000 [132:22:57<259:51:12,  5.84s/it]
 29%|██▉       | 64713/225000 [132:23:02<246:40:29,  5.54s/it]
 29%|██▉       | 64714/225000 [132:23:06<238:25:21,  5.35s/it]
 29%|██▉       | 64715/225000 [132:23:13<249:10:35,  5.60s/it]
 29%|██▉       | 64716/225000 [132:23:25<344:04:11,  7.73s/it]
 29%|██▉       | 64717/225000 [132:23:30<304:51:27,  6.85s/it]
 29%|██▉       | 64718/225000 [132:23:35<273:16:28,  6.14s/it]
 29%|██▉       | 64719/225000 [132:23:45<323:59:46,  7.28s/it]
 29%|██▉       | 64720/225000 [132:23:50<301:33:50,  6.77s/it]
 29%|██▉       | 64721/225000 [132:23:55<272:42:24,  6.13s/it]
 29%|██▉       | 64722/225000 [132:23:59<247:29:52,  5.56s/it]
 29%|██▉       | 64723/225000 [132:24:04<246:41:47,  5.54s/it]
 29%|██▉       | 64724/225000 [132:24:17<337:16:29,  7.58s/it]
 29%|██▉       | 64725/225000 [132:24:22<299:21:21,  6.72s/it]
 29%|██▉       | 64726/225000 [132:24:26<269:39:35,  6.06s/it]
 29%|██▉       | 64727/225000 [132:24:30<246:36:44,  5.54s/it]
 29%|██▉       | 64728/225000 [132:24:35<236:38:27,  5.32s/it]
 29%|██▉       | 64729/225000 [132:24:46<311:22:48,  6.99s/it]
 29%|██▉       | 64730/225000 [132:24:54<323:33:01,  7.27s/it]
 29%|██▉       | 64731/225000 [132:24:58<284:07:43,  6.38s/it]
 29%|██▉       | 64732/225000 [132:25:04<279:53:23,  6.29s/it]
 29%|██▉       | 64733/225000 [132:25:14<319:33:08,  7.18s/it]
 29%|██▉       | 64734/225000 [132:25:22<337:28:42,  7.58s/it]
 29%|██▉       | 64735/225000 [132:25:26<289:40:23,  6.51s/it]
 29%|██▉       | 64736/225000 [132:25:31<262:12:27,  5.89s/it]
 29%|██▉       | 64737/225000 [132:25:41<317:59:52,  7.14s/it]
 29%|██▉       | 64738/225000 [132:25:44<271:02:28,  6.09s/it]
 29%|██▉       | 64739/225000 [132:25:53<310:57:57,  6.99s/it]
 29%|██▉       | 64740/225000 [132:25:57<270:46:52,  6.08s/it]
 29%|██▉       | 64741/225000 [132:26:01<244:09:43,  5.48s/it]
 29%|██▉       | 64742/225000 [132:26:12<310:08:55,  6.97s/it]
 29%|██▉       | 64743/225000 [132:26:17<281:00:33,  6.31s/it]
 29%|██▉       | 64744/225000 [132:26:25<306:32:29,  6.89s/it]
 29%|██▉       | 64745/225000 [132:26:30<287:04:37,  6.45s/it]
 29%|██▉       | 64746/225000 [132:26:36<284:07:58,  6.38s/it]
 29%|██▉       | 64747/225000 [132:26:42<270:15:52,  6.07s/it]
 29%|██▉       | 64748/225000 [132:26:47<264:09:26,  5.93s/it]
 29%|██▉       | 64749/225000 [132:26:53<256:07:52,  5.75s/it]
 29%|██▉       | 64750/225000 [132:26:58<246:44:26,  5.54s/it]
                                                              
{'loss': '0.7656', 'grad_norm': '0.7346', 'learning_rate': '4.243e-05', 'epoch': '0.2878', 'num_input_tokens_seen': 220150032, 'train_runtime': '4.768e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64750/225000 [132:26:58<246:44:26,  5.54s/it]
 29%|██▉       | 64751/225000 [132:27:03<246:10:43,  5.53s/it]
 29%|██▉       | 64752/225000 [132:27:10<262:16:19,  5.89s/it]
 29%|██▉       | 64753/225000 [132:27:23<351:40:54,  7.90s/it]
 29%|██▉       | 64754/225000 [132:27:28<314:06:46,  7.06s/it]
 29%|██▉       | 64755/225000 [132:27:32<280:39:40,  6.31s/it]
 29%|██▉       | 64756/225000 [132:27:41<313:45:02,  7.05s/it]
 29%|██▉       | 64757/225000 [132:27:50<337:15:30,  7.58s/it]
 29%|██▉       | 64758/225000 [132:27:54<293:53:47,  6.60s/it]
 29%|██▉       | 64759/225000 [132:27:59<263:30:05,  5.92s/it]
 29%|██▉       | 64760/225000 [132:28:08<309:27:04,  6.95s/it]
 29%|██▉       | 64761/225000 [132:28:12<265:59:45,  5.98s/it]
 29%|██▉       | 64762/225000 [132:28:17<262:44:59,  5.90s/it]
 29%|██▉       | 64763/225000 [132:28:23<253:03:05,  5.69s/it]
 29%|██▉       | 64764/225000 [132:28:29<269:07:38,  6.05s/it]
 29%|██▉       | 64765/225000 [132:28:35<265:45:01,  5.97s/it]
 29%|██▉       | 64766/225000 [132:28:41<261:27:02,  5.87s/it]
 29%|██▉       | 64767/225000 [132:28:46<256:23:30,  5.76s/it]
 29%|██▉       | 64768/225000 [132:28:52<259:49:51,  5.84s/it]
 29%|██▉       | 64769/225000 [132:29:06<359:39:35,  8.08s/it]
 29%|██▉       | 64770/225000 [132:29:11<323:16:38,  7.26s/it]
 29%|██▉       | 64771/225000 [132:29:15<279:43:00,  6.28s/it]
 29%|██▉       | 64772/225000 [132:29:25<326:24:00,  7.33s/it]
 29%|██▉       | 64773/225000 [132:29:30<302:30:46,  6.80s/it]
 29%|██▉       | 64774/225000 [132:29:36<293:14:05,  6.59s/it]
 29%|██▉       | 64775/225000 [132:29:45<319:06:35,  7.17s/it]
 29%|██▉       | 64776/225000 [132:29:49<277:40:15,  6.24s/it]
 29%|██▉       | 64777/225000 [132:29:54<257:19:18,  5.78s/it]
 29%|██▉       | 64778/225000 [132:29:59<254:21:11,  5.72s/it]
 29%|██▉       | 64779/225000 [132:30:11<336:02:05,  7.55s/it]
 29%|██▉       | 64780/225000 [132:30:15<282:22:11,  6.34s/it]
 29%|██▉       | 64781/225000 [132:30:24<316:04:26,  7.10s/it]
 29%|██▉       | 64782/225000 [132:30:29<287:49:02,  6.47s/it]
 29%|██▉       | 64783/225000 [132:30:34<272:48:35,  6.13s/it]
 29%|██▉       | 64784/225000 [132:30:39<258:13:48,  5.80s/it]
 29%|██▉       | 64785/225000 [132:30:45<255:05:28,  5.73s/it]
 29%|██▉       | 64786/225000 [132:30:50<250:52:47,  5.64s/it]
 29%|██▉       | 64787/225000 [132:30:56<258:41:48,  5.81s/it]
 29%|██▉       | 64788/225000 [132:31:10<361:40:22,  8.13s/it]
 29%|██▉       | 64789/225000 [132:31:14<312:20:15,  7.02s/it]
 29%|██▉       | 64790/225000 [132:31:19<286:09:05,  6.43s/it]
 29%|██▉       | 64791/225000 [132:31:24<266:22:51,  5.99s/it]
 29%|██▉       | 64792/225000 [132:31:30<264:09:19,  5.94s/it]
 29%|██▉       | 64793/225000 [132:31:36<259:24:49,  5.83s/it]
 29%|██▉       | 64794/225000 [132:31:42<261:46:37,  5.88s/it]
 29%|██▉       | 64795/225000 [132:31:47<254:01:35,  5.71s/it]
 29%|██▉       | 64796/225000 [132:31:56<299:11:36,  6.72s/it]
 29%|██▉       | 64797/225000 [132:32:06<342:20:15,  7.69s/it]
 29%|██▉       | 64798/225000 [132:32:12<325:16:26,  7.31s/it]
 29%|██▉       | 64799/225000 [132:32:20<325:54:25,  7.32s/it]
 29%|██▉       | 64800/225000 [132:32:25<303:54:20,  6.83s/it]
                                                              
{'loss': '0.7578', 'grad_norm': '1.027', 'learning_rate': '4.241e-05', 'epoch': '0.288', 'num_input_tokens_seen': 220298696, 'train_runtime': '4.771e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64800/225000 [132:32:25<303:54:20,  6.83s/it]
 29%|██▉       | 64801/225000 [132:32:33<317:54:16,  7.14s/it]
 29%|██▉       | 64802/225000 [132:32:39<305:39:39,  6.87s/it]
 29%|██▉       | 64803/225000 [132:32:45<292:27:03,  6.57s/it]
 29%|██▉       | 64804/225000 [132:32:51<286:05:57,  6.43s/it]
 29%|██▉       | 64805/225000 [132:32:57<276:14:39,  6.21s/it]
 29%|██▉       | 64806/225000 [132:33:08<335:33:42,  7.54s/it]
 29%|██▉       | 64807/225000 [132:33:12<285:32:50,  6.42s/it]
 29%|██▉       | 64808/225000 [132:33:16<255:02:26,  5.73s/it]
 29%|██▉       | 64809/225000 [132:33:23<278:21:29,  6.26s/it]
 29%|██▉       | 64810/225000 [132:33:32<319:20:04,  7.18s/it]
 29%|██▉       | 64811/225000 [132:33:37<280:00:57,  6.29s/it]
 29%|██▉       | 64812/225000 [132:33:46<321:23:53,  7.22s/it]
 29%|██▉       | 64813/225000 [132:33:51<290:51:11,  6.54s/it]
 29%|██▉       | 64814/225000 [132:33:56<271:05:20,  6.09s/it]
 29%|██▉       | 64815/225000 [132:34:01<251:00:46,  5.64s/it]
 29%|██▉       | 64816/225000 [132:34:05<238:36:38,  5.36s/it]
 29%|██▉       | 64817/225000 [132:34:18<332:18:04,  7.47s/it]
 29%|██▉       | 64818/225000 [132:34:22<289:05:52,  6.50s/it]
 29%|██▉       | 64819/225000 [132:34:26<256:27:53,  5.76s/it]
 29%|██▉       | 64820/225000 [132:34:36<318:53:42,  7.17s/it]
 29%|██▉       | 64821/225000 [132:34:43<315:55:41,  7.10s/it]
 29%|██▉       | 64822/225000 [132:34:49<299:32:41,  6.73s/it]
 29%|██▉       | 64823/225000 [132:34:55<283:41:12,  6.38s/it]
 29%|██▉       | 64824/225000 [132:35:01<277:30:57,  6.24s/it]
 29%|██▉       | 64825/225000 [132:35:10<321:21:09,  7.22s/it]
 29%|██▉       | 64826/225000 [132:35:14<277:55:55,  6.25s/it]
 29%|██▉       | 64827/225000 [132:35:18<243:07:43,  5.46s/it]
 29%|██▉       | 64828/225000 [132:35:26<276:06:55,  6.21s/it]
 29%|██▉       | 64829/225000 [132:35:33<283:33:25,  6.37s/it]
 29%|██▉       | 64830/225000 [132:35:37<261:03:52,  5.87s/it]
 29%|██▉       | 64831/225000 [132:35:49<340:15:39,  7.65s/it]
 29%|██▉       | 64832/225000 [132:35:54<299:04:32,  6.72s/it]
 29%|██▉       | 64833/225000 [132:35:58<271:13:34,  6.10s/it]
 29%|██▉       | 64834/225000 [132:36:04<260:25:45,  5.85s/it]
 29%|██▉       | 64835/225000 [132:36:08<244:57:45,  5.51s/it]
 29%|██▉       | 64836/225000 [132:36:14<254:10:37,  5.71s/it]
 29%|██▉       | 64837/225000 [132:36:27<346:07:45,  7.78s/it]
 29%|██▉       | 64838/225000 [132:36:33<318:16:29,  7.15s/it]
 29%|██▉       | 64839/225000 [132:36:38<297:04:10,  6.68s/it]
 29%|██▉       | 64840/225000 [132:36:44<281:52:48,  6.34s/it]
 29%|██▉       | 64841/225000 [132:36:50<273:50:10,  6.16s/it]
 29%|██▉       | 64842/225000 [132:36:55<263:46:55,  5.93s/it]
 29%|██▉       | 64843/225000 [132:37:01<262:26:03,  5.90s/it]
 29%|██▉       | 64844/225000 [132:37:06<256:16:21,  5.76s/it]
 29%|██▉       | 64845/225000 [132:37:12<254:45:32,  5.73s/it]
 29%|██▉       | 64846/225000 [132:37:17<249:48:36,  5.62s/it]
 29%|██▉       | 64847/225000 [132:37:30<349:38:47,  7.86s/it]
 29%|██▉       | 64848/225000 [132:37:35<306:46:37,  6.90s/it]
 29%|██▉       | 64849/225000 [132:37:40<276:47:45,  6.22s/it]
 29%|██▉       | 64850/225000 [132:37:51<351:30:45,  7.90s/it]
                                                              
{'loss': '0.7499', 'grad_norm': '0.8671', 'learning_rate': '4.24e-05', 'epoch': '0.2882', 'num_input_tokens_seen': 220445136, 'train_runtime': '4.775e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64850/225000 [132:37:51<351:30:45,  7.90s/it]
 29%|██▉       | 64851/225000 [132:37:56<300:45:38,  6.76s/it]
 29%|██▉       | 64852/225000 [132:38:03<304:43:42,  6.85s/it]
 29%|██▉       | 64853/225000 [132:38:07<266:54:30,  6.00s/it]
 29%|██▉       | 64854/225000 [132:38:12<259:31:51,  5.83s/it]
 29%|██▉       | 64855/225000 [132:38:25<348:36:28,  7.84s/it]
 29%|██▉       | 64856/225000 [132:38:28<292:49:25,  6.58s/it]
 29%|██▉       | 64857/225000 [132:38:36<311:49:48,  7.01s/it]
 29%|██▉       | 64858/225000 [132:38:41<276:40:29,  6.22s/it]
 29%|██▉       | 64859/225000 [132:38:45<251:50:25,  5.66s/it]
 29%|██▉       | 64860/225000 [132:38:56<326:34:49,  7.34s/it]
 29%|██▉       | 64861/225000 [132:39:02<299:58:39,  6.74s/it]
 29%|██▉       | 64862/225000 [132:39:07<279:38:53,  6.29s/it]
 29%|██▉       | 64863/225000 [132:39:12<269:50:12,  6.07s/it]
 29%|██▉       | 64864/225000 [132:39:17<255:06:34,  5.74s/it]
 29%|██▉       | 64865/225000 [132:39:22<246:22:47,  5.54s/it]
 29%|██▉       | 64866/225000 [132:39:33<319:17:19,  7.18s/it]
 29%|██▉       | 64867/225000 [132:39:41<330:30:29,  7.43s/it]
 29%|██▉       | 64868/225000 [132:39:46<297:54:09,  6.70s/it]
 29%|██▉       | 64869/225000 [132:39:54<312:35:34,  7.03s/it]
 29%|██▉       | 64870/225000 [132:39:59<276:56:09,  6.23s/it]
 29%|██▉       | 64871/225000 [132:40:03<254:16:46,  5.72s/it]
 29%|██▉       | 64872/225000 [132:40:08<238:25:27,  5.36s/it]
 29%|██▉       | 64873/225000 [132:40:18<305:30:55,  6.87s/it]
 29%|██▉       | 64874/225000 [132:40:22<267:07:47,  6.01s/it]
 29%|██▉       | 64875/225000 [132:40:30<297:06:55,  6.68s/it]
 29%|██▉       | 64876/225000 [132:40:38<315:57:04,  7.10s/it]
 29%|██▉       | 64877/225000 [132:40:43<289:12:19,  6.50s/it]
 29%|██▉       | 64878/225000 [132:40:48<267:20:43,  6.01s/it]
 29%|██▉       | 64879/225000 [132:40:53<252:49:26,  5.68s/it]
 29%|██▉       | 64880/225000 [132:40:58<234:08:56,  5.26s/it]
 29%|██▉       | 64881/225000 [132:41:04<250:52:13,  5.64s/it]
 29%|██▉       | 64882/225000 [132:41:17<351:21:50,  7.90s/it]
 29%|██▉       | 64883/225000 [132:41:21<301:08:19,  6.77s/it]
 29%|██▉       | 64884/225000 [132:41:27<289:43:02,  6.51s/it]
 29%|██▉       | 64885/225000 [132:41:34<290:08:11,  6.52s/it]
 29%|██▉       | 64886/225000 [132:41:40<291:44:02,  6.56s/it]
 29%|██▉       | 64887/225000 [132:41:45<262:34:20,  5.90s/it]
 29%|██▉       | 64888/225000 [132:41:49<242:29:47,  5.45s/it]
 29%|██▉       | 64889/225000 [132:42:01<330:40:05,  7.43s/it]
 29%|██▉       | 64890/225000 [132:42:07<300:58:04,  6.77s/it]
 29%|██▉       | 64891/225000 [132:42:11<275:29:54,  6.19s/it]
 29%|██▉       | 64892/225000 [132:42:16<259:16:50,  5.83s/it]
 29%|██▉       | 64893/225000 [132:42:21<249:49:28,  5.62s/it]
 29%|██▉       | 64894/225000 [132:42:32<321:51:51,  7.24s/it]
 29%|██▉       | 64895/225000 [132:42:38<301:22:50,  6.78s/it]
 29%|██▉       | 64896/225000 [132:42:44<285:04:51,  6.41s/it]
 29%|██▉       | 64897/225000 [132:42:49<274:06:29,  6.16s/it]
 29%|██▉       | 64898/225000 [132:42:55<263:56:50,  5.94s/it]
 29%|██▉       | 64899/225000 [132:43:00<251:40:55,  5.66s/it]
 29%|██▉       | 64900/225000 [132:43:05<241:21:34,  5.43s/it]
                                                              
{'loss': '0.7506', 'grad_norm': '0.8399', 'learning_rate': '4.239e-05', 'epoch': '0.2884', 'num_input_tokens_seen': 220591640, 'train_runtime': '4.778e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64900/225000 [132:43:05<241:21:34,  5.43s/it]
 29%|██▉       | 64901/225000 [132:43:11<250:10:41,  5.63s/it]
 29%|██▉       | 64902/225000 [132:43:16<250:39:45,  5.64s/it]
 29%|██▉       | 64903/225000 [132:43:29<341:46:49,  7.69s/it]
 29%|██▉       | 64904/225000 [132:43:36<335:32:40,  7.55s/it]
 29%|██▉       | 64905/225000 [132:43:43<328:18:03,  7.38s/it]
 29%|██▉       | 64906/225000 [132:43:49<305:05:55,  6.86s/it]
 29%|██▉       | 64907/225000 [132:43:55<296:16:39,  6.66s/it]
 29%|██▉       | 64908/225000 [132:44:03<313:30:56,  7.05s/it]
 29%|██▉       | 64909/225000 [132:44:08<286:20:47,  6.44s/it]
 29%|██▉       | 64910/225000 [132:44:14<280:08:15,  6.30s/it]
 29%|██▉       | 64911/225000 [132:44:19<270:43:38,  6.09s/it]
 29%|██▉       | 64912/225000 [132:44:27<288:24:27,  6.49s/it]
 29%|██▉       | 64913/225000 [132:44:33<280:52:51,  6.32s/it]
 29%|██▉       | 64914/225000 [132:44:38<265:42:15,  5.98s/it]
 29%|██▉       | 64915/225000 [132:44:46<297:10:20,  6.68s/it]
 29%|██▉       | 64916/225000 [132:44:53<293:47:19,  6.61s/it]
 29%|██▉       | 64917/225000 [132:44:59<284:12:07,  6.39s/it]
 29%|██▉       | 64918/225000 [132:45:04<273:44:03,  6.16s/it]
 29%|██▉       | 64919/225000 [132:45:10<266:54:28,  6.00s/it]
 29%|██▉       | 64920/225000 [132:45:16<263:32:24,  5.93s/it]
 29%|██▉       | 64921/225000 [132:45:21<262:26:50,  5.90s/it]
 29%|██▉       | 64922/225000 [132:45:27<262:36:06,  5.91s/it]
 29%|██▉       | 64923/225000 [132:45:33<262:11:27,  5.90s/it]
 29%|██▉       | 64924/225000 [132:45:44<327:35:19,  7.37s/it]
 29%|██▉       | 64925/225000 [132:45:50<303:26:26,  6.82s/it]
 29%|██▉       | 64926/225000 [132:45:57<308:01:12,  6.93s/it]
 29%|██▉       | 64927/225000 [132:46:02<282:11:05,  6.35s/it]
 29%|██▉       | 64928/225000 [132:46:07<272:18:51,  6.12s/it]
 29%|██▉       | 64929/225000 [132:46:13<267:05:10,  6.01s/it]
 29%|██▉       | 64930/225000 [132:46:19<264:44:33,  5.95s/it]
 29%|██▉       | 64931/225000 [132:46:25<259:32:32,  5.84s/it]
 29%|██▉       | 64932/225000 [132:46:34<313:39:13,  7.05s/it]
 29%|██▉       | 64933/225000 [132:46:40<294:48:53,  6.63s/it]
 29%|██▉       | 64934/225000 [132:46:45<276:19:38,  6.21s/it]
 29%|██▉       | 64935/225000 [132:46:51<274:14:56,  6.17s/it]
 29%|██▉       | 64936/225000 [132:46:57<267:27:37,  6.02s/it]
 29%|██▉       | 64937/225000 [132:47:04<281:49:44,  6.34s/it]
 29%|██▉       | 64938/225000 [132:47:17<363:24:50,  8.17s/it]
 29%|██▉       | 64939/225000 [132:47:24<351:47:36,  7.91s/it]
 29%|██▉       | 64940/225000 [132:47:29<314:20:37,  7.07s/it]
 29%|██▉       | 64941/225000 [132:47:34<290:11:16,  6.53s/it]
 29%|██▉       | 64942/225000 [132:47:39<269:42:54,  6.07s/it]
 29%|██▉       | 64943/225000 [132:47:44<258:44:24,  5.82s/it]
 29%|██▉       | 64944/225000 [132:47:56<337:44:15,  7.60s/it]
 29%|██▉       | 64945/225000 [132:48:02<314:02:09,  7.06s/it]
 29%|██▉       | 64946/225000 [132:48:06<271:14:41,  6.10s/it]
 29%|██▉       | 64947/225000 [132:48:12<266:05:51,  5.99s/it]
 29%|██▉       | 64948/225000 [132:48:17<260:27:39,  5.86s/it]
 29%|██▉       | 64949/225000 [132:48:23<256:36:44,  5.77s/it]
 29%|██▉       | 64950/225000 [132:48:28<256:25:30,  5.77s/it]
                                                              
{'loss': '0.751', 'grad_norm': '0.8494', 'learning_rate': '4.237e-05', 'epoch': '0.2887', 'num_input_tokens_seen': 220740280, 'train_runtime': '4.781e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 64950/225000 [132:48:28<256:25:30,  5.77s/it]
 29%|██▉       | 64951/225000 [132:48:35<269:05:33,  6.05s/it]
 29%|██▉       | 64952/225000 [132:48:41<265:13:40,  5.97s/it]
 29%|██▉       | 64953/225000 [132:48:47<262:11:28,  5.90s/it]
 29%|██▉       | 64954/225000 [132:48:53<264:22:50,  5.95s/it]
 29%|██▉       | 64955/225000 [132:49:05<348:03:46,  7.83s/it]
 29%|██▉       | 64956/225000 [132:49:09<296:43:56,  6.67s/it]
 29%|██▉       | 64957/225000 [132:49:14<274:48:55,  6.18s/it]
 29%|██▉       | 64958/225000 [132:49:22<305:46:04,  6.88s/it]
 29%|██▉       | 64959/225000 [132:49:29<307:09:33,  6.91s/it]
 29%|██▉       | 64960/225000 [132:49:33<268:24:08,  6.04s/it]
 29%|██▉       | 64961/225000 [132:49:45<343:36:49,  7.73s/it]
 29%|██▉       | 64962/225000 [132:49:52<331:30:40,  7.46s/it]
 29%|██▉       | 64963/225000 [132:49:57<297:49:05,  6.70s/it]
 29%|██▉       | 64964/225000 [132:50:02<274:48:58,  6.18s/it]
 29%|██▉       | 64965/225000 [132:50:07<258:20:23,  5.81s/it]
 29%|██▉       | 64966/225000 [132:50:12<248:01:05,  5.58s/it]
 29%|██▉       | 64967/225000 [132:50:24<330:55:31,  7.44s/it]
 29%|██▉       | 64968/225000 [132:50:31<322:59:37,  7.27s/it]
 29%|██▉       | 64969/225000 [132:50:36<304:37:35,  6.85s/it]
 29%|██▉       | 64970/225000 [132:50:42<283:09:00,  6.37s/it]
 29%|██▉       | 64971/225000 [132:50:51<319:57:07,  7.20s/it]
 29%|██▉       | 64972/225000 [132:50:55<280:03:17,  6.30s/it]
 29%|██▉       | 64973/225000 [132:51:04<322:31:53,  7.26s/it]
 29%|██▉       | 64974/225000 [132:51:09<280:22:16,  6.31s/it]
 29%|██▉       | 64975/225000 [132:51:13<254:07:38,  5.72s/it]
 29%|██▉       | 64976/225000 [132:51:17<235:29:30,  5.30s/it]
 29%|██▉       | 64977/225000 [132:51:23<248:19:39,  5.59s/it]
 29%|██▉       | 64978/225000 [132:51:33<296:45:31,  6.68s/it]
 29%|██▉       | 64979/225000 [132:51:38<278:14:12,  6.26s/it]
 29%|██▉       | 64980/225000 [132:51:43<259:30:24,  5.84s/it]
 29%|██▉       | 64981/225000 [132:51:48<246:42:37,  5.55s/it]
 29%|██▉       | 64982/225000 [132:51:52<233:38:45,  5.26s/it]
 29%|██▉       | 64983/225000 [132:52:05<337:03:30,  7.58s/it]
 29%|██▉       | 64984/225000 [132:52:11<313:49:56,  7.06s/it]
 29%|██▉       | 64985/225000 [132:52:16<289:46:20,  6.52s/it]
 29%|██▉       | 64986/225000 [132:52:23<292:45:17,  6.59s/it]
 29%|██▉       | 64987/225000 [132:52:29<284:20:20,  6.40s/it]
 29%|██▉       | 64988/225000 [132:52:35<274:39:18,  6.18s/it]
 29%|██▉       | 64989/225000 [132:52:40<265:46:24,  5.98s/it]
 29%|██▉       | 64990/225000 [132:52:46<261:58:39,  5.89s/it]
 29%|██▉       | 64991/225000 [132:52:52<257:02:12,  5.78s/it]
 29%|██▉       | 64992/225000 [132:52:57<253:01:22,  5.69s/it]
 29%|██▉       | 64993/225000 [132:53:03<258:26:43,  5.81s/it]
 29%|██▉       | 64994/225000 [132:53:16<352:57:46,  7.94s/it]
 29%|██▉       | 64995/225000 [132:53:21<312:08:24,  7.02s/it]
 29%|██▉       | 64996/225000 [132:53:29<325:55:25,  7.33s/it]
 29%|██▉       | 64997/225000 [132:53:36<321:55:11,  7.24s/it]
 29%|██▉       | 64998/225000 [132:53:41<294:18:15,  6.62s/it]
 29%|██▉       | 64999/225000 [132:53:46<266:17:34,  5.99s/it]
 29%|██▉       | 65000/225000 [132:53:51<255:41:19,  5.75s/it]
                                                              
{'loss': '0.7419', 'grad_norm': '0.7329', 'learning_rate': '4.236e-05', 'epoch': '0.2889', 'num_input_tokens_seen': 220887256, 'train_runtime': '4.784e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65000/225000 [132:53:51<255:41:19,  5.75s/it]
 29%|██▉       | 65001/225000 [132:54:01<313:50:49,  7.06s/it]
 29%|██▉       | 65002/225000 [132:54:09<323:17:52,  7.27s/it]
 29%|██▉       | 65003/225000 [132:54:13<284:58:39,  6.41s/it]
 29%|██▉       | 65004/225000 [132:54:17<256:33:23,  5.77s/it]
 29%|██▉       | 65005/225000 [132:54:29<328:30:33,  7.39s/it]
 29%|██▉       | 65006/225000 [132:54:34<301:56:37,  6.79s/it]
 29%|██▉       | 65007/225000 [132:54:39<273:59:53,  6.17s/it]
 29%|██▉       | 65008/225000 [132:54:43<255:32:23,  5.75s/it]
 29%|██▉       | 65009/225000 [132:54:49<248:34:09,  5.59s/it]
 29%|██▉       | 65010/225000 [132:55:02<346:32:50,  7.80s/it]
 29%|██▉       | 65011/225000 [132:55:06<306:17:29,  6.89s/it]
 29%|██▉       | 65012/225000 [132:55:11<274:04:23,  6.17s/it]
 29%|██▉       | 65013/225000 [132:55:16<254:02:49,  5.72s/it]
 29%|██▉       | 65014/225000 [132:55:21<247:52:29,  5.58s/it]
 29%|██▉       | 65015/225000 [132:55:26<240:52:24,  5.42s/it]
 29%|██▉       | 65016/225000 [132:55:32<248:44:38,  5.60s/it]
 29%|██▉       | 65017/225000 [132:55:45<349:03:22,  7.85s/it]
 29%|██▉       | 65018/225000 [132:55:51<329:26:48,  7.41s/it]
 29%|██▉       | 65019/225000 [132:55:56<287:43:07,  6.47s/it]
 29%|██▉       | 65020/225000 [132:56:00<265:14:14,  5.97s/it]
 29%|██▉       | 65021/225000 [132:56:05<248:01:58,  5.58s/it]
 29%|██▉       | 65022/225000 [132:56:10<240:21:26,  5.41s/it]
 29%|██▉       | 65023/225000 [132:56:15<236:44:20,  5.33s/it]
 29%|██▉       | 65024/225000 [132:56:21<240:36:16,  5.41s/it]
 29%|██▉       | 65025/225000 [132:56:34<337:10:13,  7.59s/it]
 29%|██▉       | 65026/225000 [132:56:39<305:53:23,  6.88s/it]
 29%|██▉       | 65027/225000 [132:56:45<300:07:38,  6.75s/it]
 29%|██▉       | 65028/225000 [132:56:53<313:29:08,  7.05s/it]
 29%|██▉       | 65029/225000 [132:56:57<274:55:17,  6.19s/it]
 29%|██▉       | 65030/225000 [132:57:01<248:00:08,  5.58s/it]
 29%|██▉       | 65031/225000 [132:57:12<321:01:46,  7.22s/it]
 29%|██▉       | 65032/225000 [132:57:18<298:51:34,  6.73s/it]
 29%|██▉       | 65033/225000 [132:57:23<282:38:52,  6.36s/it]
 29%|██▉       | 65034/225000 [132:57:32<307:58:18,  6.93s/it]
 29%|██▉       | 65035/225000 [132:57:38<296:54:25,  6.68s/it]
 29%|██▉       | 65036/225000 [132:57:43<277:57:55,  6.26s/it]
 29%|██▉       | 65037/225000 [132:57:51<303:18:39,  6.83s/it]
 29%|██▉       | 65038/225000 [132:57:58<300:37:39,  6.77s/it]
 29%|██▉       | 65039/225000 [132:58:01<256:26:36,  5.77s/it]
 29%|██▉       | 65040/225000 [132:58:08<263:17:52,  5.93s/it]
 29%|██▉       | 65041/225000 [132:58:15<277:16:33,  6.24s/it]
 29%|██▉       | 65042/225000 [132:58:19<250:02:41,  5.63s/it]
 29%|██▉       | 65043/225000 [132:58:29<307:08:15,  6.91s/it]
 29%|██▉       | 65044/225000 [132:58:33<275:51:03,  6.21s/it]
 29%|██▉       | 65045/225000 [132:58:38<255:49:09,  5.76s/it]
 29%|██▉       | 65046/225000 [132:58:43<241:26:29,  5.43s/it]
 29%|██▉       | 65047/225000 [132:58:47<229:34:42,  5.17s/it]
 29%|██▉       | 65048/225000 [132:58:58<305:26:21,  6.87s/it]
 29%|██▉       | 65049/225000 [132:59:07<339:31:57,  7.64s/it]
 29%|██▉       | 65050/225000 [132:59:15<332:39:01,  7.49s/it]
                                                              
{'loss': '0.742', 'grad_norm': '0.7769', 'learning_rate': '4.235e-05', 'epoch': '0.2891', 'num_input_tokens_seen': 221034416, 'train_runtime': '4.788e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65050/225000 [132:59:15<332:39:01,  7.49s/it]
 29%|██▉       | 65051/225000 [132:59:21<313:09:08,  7.05s/it]
 29%|██▉       | 65052/225000 [132:59:28<311:45:06,  7.02s/it]
 29%|██▉       | 65053/225000 [132:59:34<303:56:49,  6.84s/it]
 29%|██▉       | 65054/225000 [132:59:41<309:50:13,  6.97s/it]
 29%|██▉       | 65055/225000 [132:59:46<281:16:24,  6.33s/it]
 29%|██▉       | 65056/225000 [132:59:51<261:01:39,  5.88s/it]
 29%|██▉       | 65057/225000 [132:59:56<254:19:53,  5.72s/it]
 29%|██▉       | 65058/225000 [133:00:01<244:29:21,  5.50s/it]
 29%|██▉       | 65059/225000 [133:00:06<237:36:08,  5.35s/it]
 29%|██▉       | 65060/225000 [133:00:11<234:35:03,  5.28s/it]
 29%|██▉       | 65061/225000 [133:00:21<294:54:44,  6.64s/it]
 29%|██▉       | 65062/225000 [133:00:29<306:53:09,  6.91s/it]
 29%|██▉       | 65063/225000 [133:00:34<290:50:10,  6.55s/it]
 29%|██▉       | 65064/225000 [133:00:41<284:40:49,  6.41s/it]
 29%|██▉       | 65065/225000 [133:00:47<292:09:48,  6.58s/it]
 29%|██▉       | 65066/225000 [133:00:54<296:22:04,  6.67s/it]
 29%|██▉       | 65067/225000 [133:00:59<270:17:16,  6.08s/it]
 29%|██▉       | 65068/225000 [133:01:04<249:42:34,  5.62s/it]
 29%|██▉       | 65069/225000 [133:01:08<236:11:49,  5.32s/it]
 29%|██▉       | 65070/225000 [133:01:20<322:34:27,  7.26s/it]
 29%|██▉       | 65071/225000 [133:01:26<308:27:58,  6.94s/it]
 29%|██▉       | 65072/225000 [133:01:30<267:15:17,  6.02s/it]
 29%|██▉       | 65073/225000 [133:01:40<318:30:21,  7.17s/it]
 29%|██▉       | 65074/225000 [133:01:44<277:48:49,  6.25s/it]
 29%|██▉       | 65075/225000 [133:01:49<258:05:26,  5.81s/it]
 29%|██▉       | 65076/225000 [133:01:57<283:10:19,  6.37s/it]
 29%|██▉       | 65077/225000 [133:02:05<304:33:08,  6.86s/it]
 29%|██▉       | 65078/225000 [133:02:09<267:05:33,  6.01s/it]
 29%|██▉       | 65079/225000 [133:02:16<292:14:50,  6.58s/it]
 29%|██▉       | 65080/225000 [133:02:24<307:35:37,  6.92s/it]
 29%|██▉       | 65081/225000 [133:02:28<267:57:46,  6.03s/it]
 29%|██▉       | 65082/225000 [133:02:36<292:04:34,  6.58s/it]
 29%|██▉       | 65083/225000 [133:02:43<295:07:50,  6.64s/it]
 29%|██▉       | 65084/225000 [133:02:49<289:57:13,  6.53s/it]
 29%|██▉       | 65085/225000 [133:02:57<304:24:38,  6.85s/it]
 29%|██▉       | 65086/225000 [133:03:02<280:03:18,  6.30s/it]
 29%|██▉       | 65087/225000 [133:03:07<264:20:29,  5.95s/it]
 29%|██▉       | 65088/225000 [133:03:12<254:49:01,  5.74s/it]
 29%|██▉       | 65089/225000 [133:03:17<241:18:27,  5.43s/it]
 29%|██▉       | 65090/225000 [133:03:22<245:16:25,  5.52s/it]
 29%|██▉       | 65091/225000 [133:03:34<325:32:17,  7.33s/it]
 29%|██▉       | 65092/225000 [133:03:38<280:52:57,  6.32s/it]
 29%|██▉       | 65093/225000 [133:03:43<266:24:53,  6.00s/it]
 29%|██▉       | 65094/225000 [133:03:53<317:35:15,  7.15s/it]
 29%|██▉       | 65095/225000 [133:03:57<278:26:10,  6.27s/it]
 29%|██▉       | 65096/225000 [133:04:03<270:55:47,  6.10s/it]
 29%|██▉       | 65097/225000 [133:04:09<265:21:21,  5.97s/it]
 29%|██▉       | 65098/225000 [133:04:14<262:52:34,  5.92s/it]
 29%|██▉       | 65099/225000 [133:04:20<258:16:23,  5.81s/it]
 29%|██▉       | 65100/225000 [133:04:26<253:33:41,  5.71s/it]
                                                              
{'loss': '0.7482', 'grad_norm': '0.8499', 'learning_rate': '4.233e-05', 'epoch': '0.2893', 'num_input_tokens_seen': 221181440, 'train_runtime': '4.791e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65100/225000 [133:04:26<253:33:41,  5.71s/it]
 29%|██▉       | 65101/225000 [133:04:31<252:20:27,  5.68s/it]
 29%|██▉       | 65102/225000 [133:04:37<252:00:29,  5.67s/it]
 29%|██▉       | 65103/225000 [133:04:44<270:39:16,  6.09s/it]
 29%|██▉       | 65104/225000 [133:04:57<364:00:01,  8.20s/it]
 29%|██▉       | 65105/225000 [133:05:02<327:58:03,  7.38s/it]
 29%|██▉       | 65106/225000 [133:05:10<331:15:12,  7.46s/it]
 29%|██▉       | 65107/225000 [133:05:15<295:11:08,  6.65s/it]
 29%|██▉       | 65108/225000 [133:05:20<277:30:36,  6.25s/it]
 29%|██▉       | 65109/225000 [133:05:25<262:23:46,  5.91s/it]
 29%|██▉       | 65110/225000 [133:05:30<253:12:56,  5.70s/it]
 29%|██▉       | 65111/225000 [133:05:40<299:43:22,  6.75s/it]
 29%|██▉       | 65112/225000 [133:05:50<343:11:05,  7.73s/it]
 29%|██▉       | 65113/225000 [133:05:54<299:37:16,  6.75s/it]
 29%|██▉       | 65114/225000 [133:05:58<266:25:09,  6.00s/it]
 29%|██▉       | 65115/225000 [133:06:07<297:32:00,  6.70s/it]
 29%|██▉       | 65116/225000 [133:06:14<299:34:19,  6.75s/it]
 29%|██▉       | 65117/225000 [133:06:18<270:40:44,  6.09s/it]
 29%|██▉       | 65118/225000 [133:06:24<264:43:41,  5.96s/it]
 29%|██▉       | 65119/225000 [133:06:30<262:06:14,  5.90s/it]
 29%|██▉       | 65120/225000 [133:06:35<259:49:06,  5.85s/it]
 29%|██▉       | 65121/225000 [133:06:47<341:45:34,  7.70s/it]
 29%|██▉       | 65122/225000 [133:06:54<332:11:53,  7.48s/it]
 29%|██▉       | 65123/225000 [133:06:59<293:31:44,  6.61s/it]
 29%|██▉       | 65124/225000 [133:07:04<271:50:26,  6.12s/it]
 29%|██▉       | 65125/225000 [133:07:09<254:52:20,  5.74s/it]
 29%|██▉       | 65126/225000 [133:07:20<334:32:17,  7.53s/it]
 29%|██▉       | 65127/225000 [133:07:26<312:07:51,  7.03s/it]
 29%|██▉       | 65128/225000 [133:07:33<306:12:35,  6.90s/it]
 29%|██▉       | 65129/225000 [133:07:37<270:54:19,  6.10s/it]
 29%|██▉       | 65130/225000 [133:07:47<318:30:25,  7.17s/it]
 29%|██▉       | 65131/225000 [133:07:55<328:16:02,  7.39s/it]
 29%|██▉       | 65132/225000 [133:07:59<285:03:51,  6.42s/it]
 29%|██▉       | 65133/225000 [133:08:03<254:26:41,  5.73s/it]
 29%|██▉       | 65134/225000 [133:08:10<266:20:03,  6.00s/it]
 29%|██▉       | 65135/225000 [133:08:18<303:37:35,  6.84s/it]
 29%|██▉       | 65136/225000 [133:08:24<292:02:53,  6.58s/it]
 29%|██▉       | 65137/225000 [133:08:30<278:27:40,  6.27s/it]
 29%|██▉       | 65138/225000 [133:08:35<260:15:48,  5.86s/it]
 29%|██▉       | 65139/225000 [133:08:40<250:04:32,  5.63s/it]
 29%|██▉       | 65140/225000 [133:08:45<242:53:10,  5.47s/it]
 29%|██▉       | 65141/225000 [133:08:55<303:48:23,  6.84s/it]
 29%|██▉       | 65142/225000 [133:09:05<339:41:03,  7.65s/it]
 29%|██▉       | 65143/225000 [133:09:08<285:25:51,  6.43s/it]
 29%|██▉       | 65144/225000 [133:09:13<266:45:05,  6.01s/it]
 29%|██▉       | 65145/225000 [133:09:18<249:06:08,  5.61s/it]
 29%|██▉       | 65146/225000 [133:09:23<240:02:03,  5.41s/it]
 29%|██▉       | 65147/225000 [133:09:31<279:41:13,  6.30s/it]
 29%|██▉       | 65148/225000 [133:09:43<348:18:50,  7.84s/it]
 29%|██▉       | 65149/225000 [133:09:49<333:07:26,  7.50s/it]
 29%|██▉       | 65150/225000 [133:09:53<285:14:02,  6.42s/it]
                                                              
{'loss': '0.7382', 'grad_norm': '0.8614', 'learning_rate': '4.232e-05', 'epoch': '0.2896', 'num_input_tokens_seen': 221330392, 'train_runtime': '4.794e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65150/225000 [133:09:53<285:14:02,  6.42s/it]
 29%|██▉       | 65151/225000 [133:10:02<314:47:44,  7.09s/it]
 29%|██▉       | 65152/225000 [133:10:09<315:52:39,  7.11s/it]
 29%|██▉       | 65153/225000 [133:10:13<273:25:32,  6.16s/it]
 29%|██▉       | 65154/225000 [133:10:18<260:47:54,  5.87s/it]
 29%|██▉       | 65155/225000 [133:10:24<259:01:10,  5.83s/it]
 29%|██▉       | 65156/225000 [133:10:29<251:34:48,  5.67s/it]
 29%|██▉       | 65157/225000 [133:10:35<247:45:25,  5.58s/it]
 29%|██▉       | 65158/225000 [133:10:40<245:50:21,  5.54s/it]
 29%|██▉       | 65159/225000 [133:10:47<263:28:35,  5.93s/it]
 29%|██▉       | 65160/225000 [133:11:00<361:37:42,  8.14s/it]
 29%|██▉       | 65161/225000 [133:11:05<324:06:13,  7.30s/it]
 29%|██▉       | 65162/225000 [133:11:12<320:13:25,  7.21s/it]
 29%|██▉       | 65163/225000 [133:11:17<290:42:26,  6.55s/it]
 29%|██▉       | 65164/225000 [133:11:23<275:39:51,  6.21s/it]
 29%|██▉       | 65165/225000 [133:11:28<263:28:42,  5.93s/it]
 29%|██▉       | 65166/225000 [133:11:34<262:06:10,  5.90s/it]
 29%|██▉       | 65167/225000 [133:11:39<251:37:44,  5.67s/it]
 29%|██▉       | 65168/225000 [133:11:45<252:12:50,  5.68s/it]
 29%|██▉       | 65169/225000 [133:11:53<279:12:21,  6.29s/it]
 29%|██▉       | 65170/225000 [133:11:58<270:42:07,  6.10s/it]
 29%|██▉       | 65171/225000 [133:12:10<343:40:33,  7.74s/it]
 29%|██▉       | 65172/225000 [133:12:17<342:38:46,  7.72s/it]
 29%|██▉       | 65173/225000 [133:12:23<317:33:30,  7.15s/it]
 29%|██▉       | 65174/225000 [133:12:31<326:02:32,  7.34s/it]
 29%|██▉       | 65175/225000 [133:12:36<297:36:06,  6.70s/it]
 29%|██▉       | 65176/225000 [133:12:42<283:37:01,  6.39s/it]
 29%|██▉       | 65177/225000 [133:12:51<314:00:03,  7.07s/it]
 29%|██▉       | 65178/225000 [133:12:57<301:13:17,  6.79s/it]
 29%|██▉       | 65179/225000 [133:13:02<283:56:31,  6.40s/it]
 29%|██▉       | 65180/225000 [133:13:08<276:03:49,  6.22s/it]
 29%|██▉       | 65181/225000 [133:13:13<265:33:45,  5.98s/it]
 29%|██▉       | 65182/225000 [133:13:19<265:39:03,  5.98s/it]
 29%|██▉       | 65183/225000 [133:13:25<254:00:00,  5.72s/it]
 29%|██▉       | 65184/225000 [133:13:30<253:18:09,  5.71s/it]
 29%|██▉       | 65185/225000 [133:13:36<250:36:21,  5.65s/it]
 29%|██▉       | 65186/225000 [133:13:41<249:10:36,  5.61s/it]
 29%|██▉       | 65187/225000 [133:13:49<274:58:13,  6.19s/it]
 29%|██▉       | 65188/225000 [133:14:01<354:54:26,  7.99s/it]
 29%|██▉       | 65189/225000 [133:14:05<299:14:14,  6.74s/it]
 29%|██▉       | 65190/225000 [133:14:11<294:55:18,  6.64s/it]
 29%|██▉       | 65191/225000 [133:14:19<314:07:10,  7.08s/it]
 29%|██▉       | 65192/225000 [133:14:27<322:54:14,  7.27s/it]
 29%|██▉       | 65193/225000 [133:14:34<317:00:12,  7.14s/it]
 29%|██▉       | 65194/225000 [133:14:40<296:47:54,  6.69s/it]
 29%|██▉       | 65195/225000 [133:14:43<259:35:54,  5.85s/it]
 29%|██▉       | 65196/225000 [133:14:53<308:50:45,  6.96s/it]
 29%|██▉       | 65197/225000 [133:14:58<280:03:56,  6.31s/it]
 29%|██▉       | 65198/225000 [133:15:07<317:11:25,  7.15s/it]
 29%|██▉       | 65199/225000 [133:15:11<272:32:45,  6.14s/it]
 29%|██▉       | 65200/225000 [133:15:18<294:45:51,  6.64s/it]
                                                              
{'loss': '0.7438', 'grad_norm': '0.7407', 'learning_rate': '4.231e-05', 'epoch': '0.2898', 'num_input_tokens_seen': 221478560, 'train_runtime': '4.797e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65200/225000 [133:15:18<294:45:51,  6.64s/it]
 29%|██▉       | 65201/225000 [133:15:25<288:46:00,  6.51s/it]
 29%|██▉       | 65202/225000 [133:15:32<301:42:07,  6.80s/it]
 29%|██▉       | 65203/225000 [133:15:38<284:11:22,  6.40s/it]
 29%|██▉       | 65204/225000 [133:15:42<259:38:15,  5.85s/it]
 29%|██▉       | 65205/225000 [133:15:48<262:30:02,  5.91s/it]
 29%|██▉       | 65206/225000 [133:15:58<308:44:46,  6.96s/it]
 29%|██▉       | 65207/225000 [133:16:04<307:37:10,  6.93s/it]
 29%|██▉       | 65208/225000 [133:16:10<293:13:14,  6.61s/it]
 29%|██▉       | 65209/225000 [133:16:18<313:33:10,  7.06s/it]
 29%|██▉       | 65210/225000 [133:16:23<276:32:14,  6.23s/it]
 29%|██▉       | 65211/225000 [133:16:27<255:01:13,  5.75s/it]
 29%|██▉       | 65212/225000 [133:16:33<247:29:21,  5.58s/it]
 29%|██▉       | 65213/225000 [133:16:44<324:21:23,  7.31s/it]
 29%|██▉       | 65214/225000 [133:16:50<308:02:55,  6.94s/it]
 29%|██▉       | 65215/225000 [133:16:54<274:13:09,  6.18s/it]
 29%|██▉       | 65216/225000 [133:17:03<305:51:53,  6.89s/it]
 29%|██▉       | 65217/225000 [133:17:10<302:22:14,  6.81s/it]
 29%|██▉       | 65218/225000 [133:17:18<317:44:58,  7.16s/it]
 29%|██▉       | 65219/225000 [133:17:23<290:44:00,  6.55s/it]
 29%|██▉       | 65220/225000 [133:17:28<277:33:29,  6.25s/it]
 29%|██▉       | 65221/225000 [133:17:34<266:53:46,  6.01s/it]
 29%|██▉       | 65222/225000 [133:17:39<256:40:33,  5.78s/it]
 29%|██▉       | 65223/225000 [133:17:45<256:10:58,  5.77s/it]
 29%|██▉       | 65224/225000 [133:17:50<251:26:43,  5.67s/it]
 29%|██▉       | 65225/225000 [133:17:56<248:32:00,  5.60s/it]
 29%|██▉       | 65226/225000 [133:18:01<245:20:31,  5.53s/it]
 29%|██▉       | 65227/225000 [133:18:07<250:13:55,  5.64s/it]
 29%|██▉       | 65228/225000 [133:18:16<296:30:24,  6.68s/it]
 29%|██▉       | 65229/225000 [133:18:23<298:30:33,  6.73s/it]
 29%|██▉       | 65230/225000 [133:18:28<282:11:36,  6.36s/it]
 29%|██▉       | 65231/225000 [133:18:34<268:28:26,  6.05s/it]
 29%|██▉       | 65232/225000 [133:18:39<257:08:01,  5.79s/it]
 29%|██▉       | 65233/225000 [133:18:47<286:01:01,  6.44s/it]
 29%|██▉       | 65234/225000 [133:18:59<369:05:11,  8.32s/it]
 29%|██▉       | 65235/225000 [133:19:06<348:46:44,  7.86s/it]
 29%|██▉       | 65236/225000 [133:19:10<300:03:24,  6.76s/it]
 29%|██▉       | 65237/225000 [133:19:15<267:04:55,  6.02s/it]
 29%|██▉       | 65238/225000 [133:19:26<332:21:25,  7.49s/it]
 29%|██▉       | 65239/225000 [133:19:30<296:24:26,  6.68s/it]
 29%|██▉       | 65240/225000 [133:19:40<329:08:46,  7.42s/it]
 29%|██▉       | 65241/225000 [133:19:43<279:26:52,  6.30s/it]
 29%|██▉       | 65242/225000 [133:19:51<305:18:56,  6.88s/it]
 29%|██▉       | 65243/225000 [133:19:56<276:44:58,  6.24s/it]
 29%|██▉       | 65244/225000 [133:20:01<256:59:14,  5.79s/it]
 29%|██▉       | 65245/225000 [133:20:05<237:47:22,  5.36s/it]
 29%|██▉       | 65246/225000 [133:20:15<301:53:24,  6.80s/it]
 29%|██▉       | 65247/225000 [133:20:23<310:17:38,  6.99s/it]
 29%|██▉       | 65248/225000 [133:20:30<305:52:37,  6.89s/it]
 29%|██▉       | 65249/225000 [133:20:35<282:29:13,  6.37s/it]
 29%|██▉       | 65250/225000 [133:20:40<270:34:48,  6.10s/it]
                                                              
{'loss': '0.7532', 'grad_norm': '0.7662', 'learning_rate': '4.229e-05', 'epoch': '0.29', 'num_input_tokens_seen': 221627792, 'train_runtime': '4.8e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65250/225000 [133:20:40<270:34:48,  6.10s/it]
 29%|██▉       | 65251/225000 [133:20:45<255:38:54,  5.76s/it]
 29%|██▉       | 65252/225000 [133:20:50<248:15:06,  5.59s/it]
 29%|██▉       | 65253/225000 [133:20:55<238:57:14,  5.38s/it]
 29%|██▉       | 65254/225000 [133:21:01<239:13:19,  5.39s/it]
 29%|██▉       | 65255/225000 [133:21:08<269:12:15,  6.07s/it]
 29%|██▉       | 65256/225000 [133:21:21<360:12:50,  8.12s/it]
 29%|██▉       | 65257/225000 [133:21:25<299:06:56,  6.74s/it]
 29%|██▉       | 65258/225000 [133:21:34<330:56:46,  7.46s/it]
 29%|██▉       | 65259/225000 [133:21:38<290:38:52,  6.55s/it]
 29%|██▉       | 65260/225000 [133:21:43<266:44:35,  6.01s/it]
 29%|██▉       | 65261/225000 [133:21:48<252:13:41,  5.68s/it]
 29%|██▉       | 65262/225000 [133:21:53<238:47:37,  5.38s/it]
 29%|██▉       | 65263/225000 [133:22:01<273:28:46,  6.16s/it]
 29%|██▉       | 65264/225000 [133:22:09<306:34:40,  6.91s/it]
 29%|██▉       | 65265/225000 [133:22:13<263:42:21,  5.94s/it]
 29%|██▉       | 65266/225000 [133:22:22<307:27:12,  6.93s/it]
 29%|██▉       | 65267/225000 [133:22:26<268:21:51,  6.05s/it]
 29%|██▉       | 65268/225000 [133:22:36<316:07:40,  7.12s/it]
 29%|██▉       | 65269/225000 [133:22:42<303:40:32,  6.84s/it]
 29%|██▉       | 65270/225000 [133:22:50<322:51:31,  7.28s/it]
 29%|██▉       | 65271/225000 [133:22:54<273:48:04,  6.17s/it]
 29%|██▉       | 65272/225000 [133:23:02<299:13:01,  6.74s/it]
 29%|██▉       | 65273/225000 [133:23:06<259:41:48,  5.85s/it]
 29%|██▉       | 65274/225000 [133:23:15<308:23:19,  6.95s/it]
 29%|██▉       | 65275/225000 [133:23:20<273:06:19,  6.16s/it]
 29%|██▉       | 65276/225000 [133:23:26<275:48:20,  6.22s/it]
 29%|██▉       | 65277/225000 [133:23:33<292:11:37,  6.59s/it]
 29%|██▉       | 65278/225000 [133:23:42<315:30:51,  7.11s/it]
 29%|██▉       | 65279/225000 [133:23:45<265:43:38,  5.99s/it]
 29%|██▉       | 65280/225000 [133:23:53<288:22:53,  6.50s/it]
 29%|██▉       | 65281/225000 [133:23:57<260:08:37,  5.86s/it]
 29%|██▉       | 65282/225000 [133:24:03<258:53:40,  5.84s/it]
 29%|██▉       | 65283/225000 [133:24:10<273:22:48,  6.16s/it]
 29%|██▉       | 65284/225000 [133:24:17<285:26:27,  6.43s/it]
 29%|██▉       | 65285/225000 [133:24:24<300:41:36,  6.78s/it]
 29%|██▉       | 65286/225000 [133:24:28<261:30:18,  5.89s/it]
 29%|██▉       | 65287/225000 [133:24:33<241:45:46,  5.45s/it]
 29%|██▉       | 65288/225000 [133:24:44<312:43:26,  7.05s/it]
 29%|██▉       | 65289/225000 [133:24:50<309:02:54,  6.97s/it]
 29%|██▉       | 65290/225000 [133:24:55<278:15:26,  6.27s/it]
 29%|██▉       | 65291/225000 [133:25:00<260:44:44,  5.88s/it]
 29%|██▉       | 65292/225000 [133:25:05<247:25:28,  5.58s/it]
 29%|██▉       | 65293/225000 [133:25:10<245:47:53,  5.54s/it]
 29%|██▉       | 65294/225000 [133:25:15<241:24:58,  5.44s/it]
 29%|██▉       | 65295/225000 [133:25:21<239:43:24,  5.40s/it]
 29%|██▉       | 65296/225000 [133:25:27<246:54:18,  5.57s/it]
 29%|██▉       | 65297/225000 [133:25:40<352:21:42,  7.94s/it]
 29%|██▉       | 65298/225000 [133:25:45<307:29:16,  6.93s/it]
 29%|██▉       | 65299/225000 [133:25:49<272:11:04,  6.14s/it]
 29%|██▉       | 65300/225000 [133:25:53<248:51:46,  5.61s/it]
                                                              
{'loss': '0.7367', 'grad_norm': '0.8478', 'learning_rate': '4.228e-05', 'epoch': '0.2902', 'num_input_tokens_seen': 221773472, 'train_runtime': '4.804e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65300/225000 [133:25:53<248:51:46,  5.61s/it]
 29%|██▉       | 65301/225000 [133:26:02<281:52:09,  6.35s/it]
 29%|██▉       | 65302/225000 [133:26:11<327:00:57,  7.37s/it]
 29%|██▉       | 65303/225000 [133:26:16<290:22:50,  6.55s/it]
 29%|██▉       | 65304/225000 [133:26:21<268:51:57,  6.06s/it]
 29%|██▉       | 65305/225000 [133:26:25<250:15:12,  5.64s/it]
 29%|██▉       | 65306/225000 [133:26:33<279:02:29,  6.29s/it]
 29%|██▉       | 65307/225000 [133:26:42<310:56:12,  7.01s/it]
 29%|██▉       | 65308/225000 [133:26:50<324:58:46,  7.33s/it]
 29%|██▉       | 65309/225000 [133:26:55<297:48:52,  6.71s/it]
 29%|██▉       | 65310/225000 [133:27:00<275:38:47,  6.21s/it]
 29%|██▉       | 65311/225000 [133:27:06<271:17:46,  6.12s/it]
 29%|██▉       | 65312/225000 [133:27:12<267:32:26,  6.03s/it]
 29%|██▉       | 65313/225000 [133:27:18<262:09:24,  5.91s/it]
 29%|██▉       | 65314/225000 [133:27:24<260:52:48,  5.88s/it]
 29%|██▉       | 65315/225000 [133:27:31<280:40:17,  6.33s/it]
 29%|██▉       | 65316/225000 [133:27:39<309:54:04,  6.99s/it]
 29%|██▉       | 65317/225000 [133:27:47<321:37:47,  7.25s/it]
 29%|██▉       | 65318/225000 [133:27:51<272:49:21,  6.15s/it]
 29%|██▉       | 65319/225000 [133:28:01<327:24:47,  7.38s/it]
 29%|██▉       | 65320/225000 [133:28:05<286:51:04,  6.47s/it]
 29%|██▉       | 65321/225000 [133:28:11<272:46:53,  6.15s/it]
 29%|██▉       | 65322/225000 [133:28:16<259:36:42,  5.85s/it]
 29%|██▉       | 65323/225000 [133:28:21<254:11:19,  5.73s/it]
 29%|██▉       | 65324/225000 [133:28:27<249:45:23,  5.63s/it]
 29%|██▉       | 65325/225000 [133:28:32<247:18:37,  5.58s/it]
 29%|██▉       | 65326/225000 [133:28:38<243:20:58,  5.49s/it]
 29%|██▉       | 65327/225000 [133:28:43<246:34:53,  5.56s/it]
 29%|██▉       | 65328/225000 [133:28:55<321:36:03,  7.25s/it]
 29%|██▉       | 65329/225000 [133:29:04<347:15:23,  7.83s/it]
 29%|██▉       | 65330/225000 [133:29:08<296:51:43,  6.69s/it]
 29%|██▉       | 65331/225000 [133:29:18<344:24:53,  7.77s/it]
 29%|██▉       | 65332/225000 [133:29:22<288:25:14,  6.50s/it]
 29%|██▉       | 65333/225000 [133:29:30<320:48:55,  7.23s/it]
 29%|██▉       | 65334/225000 [133:29:34<274:50:11,  6.20s/it]
 29%|██▉       | 65335/225000 [133:29:42<296:15:46,  6.68s/it]
 29%|██▉       | 65336/225000 [133:29:50<316:43:25,  7.14s/it]
 29%|██▉       | 65337/225000 [133:29:55<287:46:21,  6.49s/it]
 29%|██▉       | 65338/225000 [133:30:01<274:56:01,  6.20s/it]
 29%|██▉       | 65339/225000 [133:30:06<256:54:41,  5.79s/it]
 29%|██▉       | 65340/225000 [133:30:11<244:54:02,  5.52s/it]
 29%|██▉       | 65341/225000 [133:30:16<238:43:47,  5.38s/it]
 29%|██▉       | 65342/225000 [133:30:23<269:26:33,  6.08s/it]
 29%|██▉       | 65343/225000 [133:30:36<352:28:00,  7.95s/it]
 29%|██▉       | 65344/225000 [133:30:40<299:33:44,  6.75s/it]
 29%|██▉       | 65345/225000 [133:30:46<294:51:17,  6.65s/it]
 29%|██▉       | 65346/225000 [133:30:50<254:53:24,  5.75s/it]
 29%|██▉       | 65347/225000 [133:30:56<266:42:42,  6.01s/it]
 29%|██▉       | 65348/225000 [133:31:05<298:52:08,  6.74s/it]
 29%|██▉       | 65349/225000 [133:31:09<272:33:36,  6.15s/it]
 29%|██▉       | 65350/225000 [133:31:15<258:51:43,  5.84s/it]
                                                              
{'loss': '0.7306', 'grad_norm': '0.8182', 'learning_rate': '4.227e-05', 'epoch': '0.2904', 'num_input_tokens_seen': 221920920, 'train_runtime': '4.807e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65350/225000 [133:31:15<258:51:43,  5.84s/it]
 29%|██▉       | 65351/225000 [133:31:19<246:10:09,  5.55s/it]
 29%|██▉       | 65352/225000 [133:31:25<241:39:06,  5.45s/it]
 29%|██▉       | 65353/225000 [133:31:29<232:57:26,  5.25s/it]
 29%|██▉       | 65354/225000 [133:31:43<348:30:14,  7.86s/it]
 29%|██▉       | 65355/225000 [133:31:48<299:31:13,  6.75s/it]
 29%|██▉       | 65356/225000 [133:31:52<267:45:40,  6.04s/it]
 29%|██▉       | 65357/225000 [133:32:03<332:34:45,  7.50s/it]
 29%|██▉       | 65358/225000 [133:32:07<284:00:55,  6.40s/it]
 29%|██▉       | 65359/225000 [133:32:11<261:50:48,  5.90s/it]
 29%|██▉       | 65360/225000 [133:32:21<314:18:48,  7.09s/it]
 29%|██▉       | 65361/225000 [133:32:29<322:51:19,  7.28s/it]
 29%|██▉       | 65362/225000 [133:32:32<270:46:46,  6.11s/it]
 29%|██▉       | 65363/225000 [133:32:37<255:37:59,  5.76s/it]
 29%|██▉       | 65364/225000 [133:32:42<246:45:02,  5.56s/it]
 29%|██▉       | 65365/225000 [133:32:48<243:31:19,  5.49s/it]
 29%|██▉       | 65366/225000 [133:32:53<246:01:34,  5.55s/it]
 29%|██▉       | 65367/225000 [133:32:58<239:01:26,  5.39s/it]
 29%|██▉       | 65368/225000 [133:33:10<326:39:09,  7.37s/it]
 29%|██▉       | 65369/225000 [133:33:17<319:01:22,  7.19s/it]
 29%|██▉       | 65370/225000 [133:33:21<275:13:07,  6.21s/it]
 29%|██▉       | 65371/225000 [133:33:26<257:37:27,  5.81s/it]
 29%|██▉       | 65372/225000 [133:33:31<249:07:17,  5.62s/it]
 29%|██▉       | 65373/225000 [133:33:40<298:06:53,  6.72s/it]
 29%|██▉       | 65374/225000 [133:33:49<321:42:51,  7.26s/it]
 29%|██▉       | 65375/225000 [133:33:55<311:14:53,  7.02s/it]
 29%|██▉       | 65376/225000 [133:34:03<315:16:07,  7.11s/it]
 29%|██▉       | 65377/225000 [133:34:10<315:59:07,  7.13s/it]
 29%|██▉       | 65378/225000 [133:34:16<300:22:06,  6.77s/it]
 29%|██▉       | 65379/225000 [133:34:20<262:21:41,  5.92s/it]
 29%|██▉       | 65380/225000 [133:34:25<259:01:10,  5.84s/it]
 29%|██▉       | 65381/225000 [133:34:32<268:35:01,  6.06s/it]
 29%|██▉       | 65382/225000 [133:34:42<315:46:51,  7.12s/it]
 29%|██▉       | 65383/225000 [133:34:45<268:55:50,  6.07s/it]
 29%|██▉       | 65384/225000 [133:34:51<265:59:47,  6.00s/it]
 29%|██▉       | 65385/225000 [133:34:57<262:34:34,  5.92s/it]
 29%|██▉       | 65386/225000 [133:35:02<255:14:20,  5.76s/it]
 29%|██▉       | 65387/225000 [133:35:11<300:46:15,  6.78s/it]
 29%|██▉       | 65388/225000 [133:35:17<284:53:51,  6.43s/it]
 29%|██▉       | 65389/225000 [133:35:22<266:10:58,  6.00s/it]
 29%|██▉       | 65390/225000 [133:35:27<253:22:48,  5.71s/it]
 29%|██▉       | 65391/225000 [133:35:39<336:43:10,  7.59s/it]
 29%|██▉       | 65392/225000 [133:35:43<288:11:55,  6.50s/it]
 29%|██▉       | 65393/225000 [133:35:50<297:21:41,  6.71s/it]
 29%|██▉       | 65394/225000 [133:35:58<308:50:07,  6.97s/it]
 29%|██▉       | 65395/225000 [133:36:02<268:20:48,  6.05s/it]
 29%|██▉       | 65396/225000 [133:36:12<319:03:49,  7.20s/it]
 29%|██▉       | 65397/225000 [133:36:16<286:04:11,  6.45s/it]
 29%|██▉       | 65398/225000 [133:36:21<265:43:07,  5.99s/it]
 29%|██▉       | 65399/225000 [133:36:26<251:41:51,  5.68s/it]
 29%|██▉       | 65400/225000 [133:36:35<293:29:31,  6.62s/it]
                                                              
{'loss': '0.7541', 'grad_norm': '0.8711', 'learning_rate': '4.225e-05', 'epoch': '0.2907', 'num_input_tokens_seen': 222067384, 'train_runtime': '4.81e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65400/225000 [133:36:35<293:29:31,  6.62s/it]
 29%|██▉       | 65401/225000 [133:36:42<300:57:34,  6.79s/it]
 29%|██▉       | 65402/225000 [133:36:51<325:36:34,  7.34s/it]
 29%|██▉       | 65403/225000 [133:36:55<280:34:41,  6.33s/it]
 29%|██▉       | 65404/225000 [133:37:04<322:09:33,  7.27s/it]
 29%|██▉       | 65405/225000 [133:37:08<277:02:32,  6.25s/it]
 29%|██▉       | 65406/225000 [133:37:13<259:01:38,  5.84s/it]
 29%|██▉       | 65407/225000 [133:37:18<251:06:38,  5.66s/it]
 29%|██▉       | 65408/225000 [133:37:23<245:34:58,  5.54s/it]
 29%|██▉       | 65409/225000 [133:37:32<286:29:45,  6.46s/it]
 29%|██▉       | 65410/225000 [133:37:41<326:22:56,  7.36s/it]
 29%|██▉       | 65411/225000 [133:37:46<282:15:48,  6.37s/it]
 29%|██▉       | 65412/225000 [133:37:54<312:49:22,  7.06s/it]
 29%|██▉       | 65413/225000 [133:37:58<269:46:48,  6.09s/it]
 29%|██▉       | 65414/225000 [133:38:07<307:26:16,  6.94s/it]
 29%|██▉       | 65415/225000 [133:38:13<301:46:46,  6.81s/it]
 29%|██▉       | 65416/225000 [133:38:18<268:20:47,  6.05s/it]
 29%|██▉       | 65417/225000 [133:38:23<260:32:44,  5.88s/it]
 29%|██▉       | 65418/225000 [133:38:29<258:55:32,  5.84s/it]
 29%|██▉       | 65419/225000 [133:38:34<248:24:27,  5.60s/it]
 29%|██▉       | 65420/225000 [133:38:42<283:55:42,  6.41s/it]
 29%|██▉       | 65421/225000 [133:38:51<310:41:26,  7.01s/it]
 29%|██▉       | 65422/225000 [133:38:56<293:28:48,  6.62s/it]
 29%|██▉       | 65423/225000 [133:39:04<310:38:51,  7.01s/it]
 29%|██▉       | 65424/225000 [133:39:10<298:12:24,  6.73s/it]
 29%|██▉       | 65425/225000 [133:39:17<301:14:00,  6.80s/it]
 29%|██▉       | 65426/225000 [133:39:22<266:43:50,  6.02s/it]
 29%|██▉       | 65427/225000 [133:39:27<256:16:44,  5.78s/it]
 29%|██▉       | 65428/225000 [133:39:32<247:27:26,  5.58s/it]
 29%|██▉       | 65429/225000 [133:39:37<244:51:31,  5.52s/it]
 29%|██▉       | 65430/225000 [133:39:44<264:16:50,  5.96s/it]
 29%|██▉       | 65431/225000 [133:39:56<342:55:39,  7.74s/it]
 29%|██▉       | 65432/225000 [133:40:02<316:21:09,  7.14s/it]
 29%|██▉       | 65433/225000 [133:40:07<289:35:05,  6.53s/it]
 29%|██▉       | 65434/225000 [133:40:12<269:57:09,  6.09s/it]
 29%|██▉       | 65435/225000 [133:40:17<252:18:58,  5.69s/it]
 29%|██▉       | 65436/225000 [133:40:29<338:34:18,  7.64s/it]
 29%|██▉       | 65437/225000 [133:40:34<303:10:51,  6.84s/it]
 29%|██▉       | 65438/225000 [133:40:39<278:33:27,  6.28s/it]
 29%|██▉       | 65439/225000 [133:40:44<259:57:08,  5.87s/it]
 29%|██▉       | 65440/225000 [133:40:55<331:33:10,  7.48s/it]
 29%|██▉       | 65441/225000 [133:41:00<296:44:10,  6.70s/it]
 29%|██▉       | 65442/225000 [133:41:05<279:56:54,  6.32s/it]
 29%|██▉       | 65443/225000 [133:41:11<272:24:35,  6.15s/it]
 29%|██▉       | 65444/225000 [133:41:17<263:05:13,  5.94s/it]
 29%|██▉       | 65445/225000 [133:41:22<258:54:10,  5.84s/it]
 29%|██▉       | 65446/225000 [133:41:28<253:24:42,  5.72s/it]
 29%|██▉       | 65447/225000 [133:41:34<258:10:47,  5.83s/it]
 29%|██▉       | 65448/225000 [133:41:47<357:28:32,  8.07s/it]
 29%|██▉       | 65449/225000 [133:41:52<315:43:10,  7.12s/it]
 29%|██▉       | 65450/225000 [133:41:56<273:49:45,  6.18s/it]
                                                              
{'loss': '0.7415', 'grad_norm': '0.8611', 'learning_rate': '4.224e-05', 'epoch': '0.2909', 'num_input_tokens_seen': 222215048, 'train_runtime': '4.813e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65450/225000 [133:41:56<273:49:45,  6.18s/it]
 29%|██▉       | 65451/225000 [133:42:05<310:23:22,  7.00s/it]
 29%|██▉       | 65452/225000 [133:42:11<299:33:41,  6.76s/it]
 29%|██▉       | 65453/225000 [133:42:19<319:27:22,  7.21s/it]
 29%|██▉       | 65454/225000 [133:42:24<281:51:43,  6.36s/it]
 29%|██▉       | 65455/225000 [133:42:32<302:58:32,  6.84s/it]
 29%|██▉       | 65456/225000 [133:42:37<284:39:41,  6.42s/it]
 29%|██▉       | 65457/225000 [133:42:42<268:44:07,  6.06s/it]
 29%|██▉       | 65458/225000 [133:42:48<261:21:20,  5.90s/it]
 29%|██▉       | 65459/225000 [133:42:53<247:21:39,  5.58s/it]
 29%|██▉       | 65460/225000 [133:42:58<238:35:33,  5.38s/it]
 29%|██▉       | 65461/225000 [133:43:06<272:43:40,  6.15s/it]
 29%|██▉       | 65462/225000 [133:43:16<328:27:40,  7.41s/it]
 29%|██▉       | 65463/225000 [133:43:23<323:44:41,  7.31s/it]
 29%|██▉       | 65464/225000 [133:43:27<277:00:55,  6.25s/it]
 29%|██▉       | 65465/225000 [133:43:35<308:34:30,  6.96s/it]
 29%|██▉       | 65466/225000 [133:43:40<273:25:29,  6.17s/it]
 29%|██▉       | 65467/225000 [133:43:44<253:48:11,  5.73s/it]
 29%|██▉       | 65468/225000 [133:43:55<313:24:09,  7.07s/it]
 29%|██▉       | 65469/225000 [133:43:59<280:41:45,  6.33s/it]
 29%|██▉       | 65470/225000 [133:44:08<314:28:45,  7.10s/it]
 29%|██▉       | 65471/225000 [133:44:15<308:03:27,  6.95s/it]
 29%|██▉       | 65472/225000 [133:44:19<277:27:56,  6.26s/it]
 29%|██▉       | 65473/225000 [133:44:24<256:00:35,  5.78s/it]
 29%|██▉       | 65474/225000 [133:44:33<299:22:56,  6.76s/it]
 29%|██▉       | 65475/225000 [133:44:38<271:46:09,  6.13s/it]
 29%|██▉       | 65476/225000 [133:44:43<261:16:36,  5.90s/it]
 29%|██▉       | 65477/225000 [133:44:48<253:14:15,  5.71s/it]
 29%|██▉       | 65478/225000 [133:44:55<270:30:25,  6.10s/it]
 29%|██▉       | 65479/225000 [133:45:06<328:50:05,  7.42s/it]
 29%|██▉       | 65480/225000 [133:45:12<309:27:06,  6.98s/it]
 29%|██▉       | 65481/225000 [133:45:16<267:27:02,  6.04s/it]
 29%|██▉       | 65482/225000 [133:45:21<257:15:55,  5.81s/it]
 29%|██▉       | 65483/225000 [133:45:26<242:08:06,  5.46s/it]
 29%|██▉       | 65484/225000 [133:45:30<231:51:16,  5.23s/it]
 29%|██▉       | 65485/225000 [133:45:43<337:20:10,  7.61s/it]
 29%|██▉       | 65486/225000 [133:45:49<312:25:42,  7.05s/it]
 29%|██▉       | 65487/225000 [133:45:53<269:55:25,  6.09s/it]
 29%|██▉       | 65488/225000 [133:46:03<318:36:38,  7.19s/it]
 29%|██▉       | 65489/225000 [133:46:07<281:30:04,  6.35s/it]
 29%|██▉       | 65490/225000 [133:46:11<252:45:14,  5.70s/it]
 29%|██▉       | 65491/225000 [133:46:20<295:50:14,  6.68s/it]
 29%|██▉       | 65492/225000 [133:46:26<285:35:52,  6.45s/it]
 29%|██▉       | 65493/225000 [133:46:35<316:56:32,  7.15s/it]
 29%|██▉       | 65494/225000 [133:46:39<272:15:19,  6.14s/it]
 29%|██▉       | 65495/225000 [133:46:47<304:35:51,  6.87s/it]
 29%|██▉       | 65496/225000 [133:46:54<298:59:56,  6.75s/it]
 29%|██▉       | 65497/225000 [133:46:59<280:45:20,  6.34s/it]
 29%|██▉       | 65498/225000 [133:47:06<292:51:00,  6.61s/it]
 29%|██▉       | 65499/225000 [133:47:10<251:58:50,  5.69s/it]
 29%|██▉       | 65500/225000 [133:47:18<287:23:35,  6.49s/it]
                                                              
{'loss': '0.7401', 'grad_norm': '0.9049', 'learning_rate': '4.223e-05', 'epoch': '0.2911', 'num_input_tokens_seen': 222363024, 'train_runtime': '4.816e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65500/225000 [133:47:18<287:23:35,  6.49s/it]
 29%|██▉       | 65501/225000 [133:47:23<262:56:51,  5.93s/it]
 29%|██▉       | 65502/225000 [133:47:27<243:08:44,  5.49s/it]
 29%|██▉       | 65503/225000 [133:47:35<271:05:33,  6.12s/it]
 29%|██▉       | 65504/225000 [133:47:44<315:03:05,  7.11s/it]
 29%|██▉       | 65505/225000 [133:47:49<285:23:51,  6.44s/it]
 29%|██▉       | 65506/225000 [133:47:54<260:19:39,  5.88s/it]
 29%|██▉       | 65507/225000 [133:47:59<248:57:46,  5.62s/it]
 29%|██▉       | 65508/225000 [133:48:04<244:55:19,  5.53s/it]
 29%|██▉       | 65509/225000 [133:48:17<339:31:15,  7.66s/it]
 29%|██▉       | 65510/225000 [133:48:21<289:16:54,  6.53s/it]
 29%|██▉       | 65511/225000 [133:48:25<259:42:31,  5.86s/it]
 29%|██▉       | 65512/225000 [133:48:29<239:22:06,  5.40s/it]
 29%|██▉       | 65513/225000 [133:48:35<237:38:04,  5.36s/it]
 29%|██▉       | 65514/225000 [133:48:48<339:01:23,  7.65s/it]
 29%|██▉       | 65515/225000 [133:48:52<292:29:21,  6.60s/it]
 29%|██▉       | 65516/225000 [133:48:58<285:10:46,  6.44s/it]
 29%|██▉       | 65517/225000 [133:49:07<318:58:49,  7.20s/it]
 29%|██▉       | 65518/225000 [133:49:11<278:05:22,  6.28s/it]
 29%|██▉       | 65519/225000 [133:49:16<255:07:32,  5.76s/it]
 29%|██▉       | 65520/225000 [133:49:20<240:32:11,  5.43s/it]
 29%|██▉       | 65521/225000 [133:49:25<238:43:20,  5.39s/it]
 29%|██▉       | 65522/225000 [133:49:38<337:36:25,  7.62s/it]
 29%|██▉       | 65523/225000 [133:49:42<284:21:24,  6.42s/it]
 29%|██▉       | 65524/225000 [133:49:50<312:14:09,  7.05s/it]
 29%|██▉       | 65525/225000 [133:49:57<310:17:12,  7.00s/it]
 29%|██▉       | 65526/225000 [133:50:02<276:12:45,  6.24s/it]
 29%|██▉       | 65527/225000 [133:50:08<269:49:16,  6.09s/it]
 29%|██▉       | 65528/225000 [133:50:15<294:22:53,  6.65s/it]
 29%|██▉       | 65529/225000 [133:50:20<270:42:12,  6.11s/it]
 29%|██▉       | 65530/225000 [133:50:25<252:26:21,  5.70s/it]
 29%|██▉       | 65531/225000 [133:50:30<240:56:07,  5.44s/it]
 29%|██▉       | 65532/225000 [133:50:41<316:56:55,  7.16s/it]
 29%|██▉       | 65533/225000 [133:50:49<321:57:54,  7.27s/it]
 29%|██▉       | 65534/225000 [133:50:53<285:19:38,  6.44s/it]
 29%|██▉       | 65535/225000 [133:51:03<324:47:14,  7.33s/it]
 29%|██▉       | 65536/225000 [133:51:07<287:28:03,  6.49s/it]
 29%|██▉       | 65537/225000 [133:51:12<265:27:46,  5.99s/it]
 29%|██▉       | 65538/225000 [133:51:21<306:31:26,  6.92s/it]
 29%|██▉       | 65539/225000 [133:51:30<329:19:59,  7.44s/it]
 29%|██▉       | 65540/225000 [133:51:35<307:58:04,  6.95s/it]
 29%|██▉       | 65541/225000 [133:51:42<299:55:57,  6.77s/it]
 29%|██▉       | 65542/225000 [133:51:46<263:29:47,  5.95s/it]
 29%|██▉       | 65543/225000 [133:51:54<291:27:39,  6.58s/it]
 29%|██▉       | 65544/225000 [133:52:00<291:24:49,  6.58s/it]
 29%|██▉       | 65545/225000 [133:52:09<315:21:56,  7.12s/it]
 29%|██▉       | 65546/225000 [133:52:13<271:24:05,  6.13s/it]
 29%|██▉       | 65547/225000 [133:52:20<293:25:31,  6.62s/it]
 29%|██▉       | 65548/225000 [133:52:26<279:10:47,  6.30s/it]
 29%|██▉       | 65549/225000 [133:52:32<276:40:51,  6.25s/it]
 29%|██▉       | 65550/225000 [133:52:37<260:07:49,  5.87s/it]
                                                              
{'loss': '0.7487', 'grad_norm': '0.736', 'learning_rate': '4.222e-05', 'epoch': '0.2913', 'num_input_tokens_seen': 222511712, 'train_runtime': '4.82e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65550/225000 [133:52:37<260:07:49,  5.87s/it]
 29%|██▉       | 65551/225000 [133:52:43<257:13:06,  5.81s/it]
 29%|██▉       | 65552/225000 [133:52:51<285:56:03,  6.46s/it]
 29%|██▉       | 65553/225000 [133:52:58<301:44:51,  6.81s/it]
 29%|██▉       | 65554/225000 [133:53:04<289:04:59,  6.53s/it]
 29%|██▉       | 65555/225000 [133:53:09<270:54:10,  6.12s/it]
 29%|██▉       | 65556/225000 [133:53:15<262:50:45,  5.93s/it]
 29%|██▉       | 65557/225000 [133:53:21<259:18:49,  5.85s/it]
 29%|██▉       | 65558/225000 [133:53:29<291:12:02,  6.57s/it]
 29%|██▉       | 65559/225000 [133:53:36<299:41:08,  6.77s/it]
 29%|██▉       | 65560/225000 [133:53:43<297:40:13,  6.72s/it]
 29%|██▉       | 65561/225000 [133:53:50<301:54:23,  6.82s/it]
 29%|██▉       | 65562/225000 [133:53:53<260:01:23,  5.87s/it]
 29%|██▉       | 65563/225000 [133:54:02<295:24:40,  6.67s/it]
 29%|██▉       | 65564/225000 [133:54:08<287:07:14,  6.48s/it]
 29%|██▉       | 65565/225000 [133:54:13<264:39:28,  5.98s/it]
 29%|██▉       | 65566/225000 [133:54:18<256:33:27,  5.79s/it]
 29%|██▉       | 65567/225000 [133:54:23<247:03:13,  5.58s/it]
 29%|██▉       | 65568/225000 [133:54:28<241:24:43,  5.45s/it]
 29%|██▉       | 65569/225000 [133:54:34<239:14:12,  5.40s/it]
 29%|██▉       | 65570/225000 [133:54:39<232:47:28,  5.26s/it]
 29%|██▉       | 65571/225000 [133:54:44<237:57:21,  5.37s/it]
 29%|██▉       | 65572/225000 [133:54:57<336:09:26,  7.59s/it]
 29%|██▉       | 65573/225000 [133:55:03<312:37:04,  7.06s/it]
 29%|██▉       | 65574/225000 [133:55:12<338:20:30,  7.64s/it]
 29%|██▉       | 65575/225000 [133:55:16<287:36:57,  6.49s/it]
 29%|██▉       | 65576/225000 [133:55:22<290:21:54,  6.56s/it]
 29%|██▉       | 65577/225000 [133:55:30<310:03:50,  7.00s/it]
 29%|██▉       | 65578/225000 [133:55:35<278:54:32,  6.30s/it]
 29%|██▉       | 65579/225000 [133:55:40<257:52:56,  5.82s/it]
 29%|██▉       | 65580/225000 [133:55:50<312:59:00,  7.07s/it]
 29%|██▉       | 65581/225000 [133:55:58<332:45:30,  7.51s/it]
 29%|██▉       | 65582/225000 [133:56:03<301:42:40,  6.81s/it]
 29%|██▉       | 65583/225000 [133:56:08<277:47:20,  6.27s/it]
 29%|██▉       | 65584/225000 [133:56:13<259:28:58,  5.86s/it]
 29%|██▉       | 65585/225000 [133:56:18<239:48:06,  5.42s/it]
 29%|██▉       | 65586/225000 [133:56:29<313:05:58,  7.07s/it]
 29%|██▉       | 65587/225000 [133:56:37<326:23:34,  7.37s/it]
 29%|██▉       | 65588/225000 [133:56:42<293:19:47,  6.62s/it]
 29%|██▉       | 65589/225000 [133:56:46<268:14:45,  6.06s/it]
 29%|██▉       | 65590/225000 [133:56:51<247:24:37,  5.59s/it]
 29%|██▉       | 65591/225000 [133:56:56<240:02:32,  5.42s/it]
 29%|██▉       | 65592/225000 [133:57:08<330:12:41,  7.46s/it]
 29%|██▉       | 65593/225000 [133:57:12<288:50:03,  6.52s/it]
 29%|██▉       | 65594/225000 [133:57:22<331:23:11,  7.48s/it]
 29%|██▉       | 65595/225000 [133:57:27<293:59:07,  6.64s/it]
 29%|██▉       | 65596/225000 [133:57:32<280:40:10,  6.34s/it]
 29%|██▉       | 65597/225000 [133:57:44<346:46:51,  7.83s/it]
 29%|██▉       | 65598/225000 [133:57:51<344:36:12,  7.78s/it]
 29%|██▉       | 65599/225000 [133:57:55<295:43:41,  6.68s/it]
 29%|██▉       | 65600/225000 [133:57:59<257:52:05,  5.82s/it]
                                                              
{'loss': '0.7458', 'grad_norm': '0.8312', 'learning_rate': '4.22e-05', 'epoch': '0.2916', 'num_input_tokens_seen': 222660552, 'train_runtime': '4.823e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65600/225000 [133:57:59<257:52:05,  5.82s/it]
 29%|██▉       | 65601/225000 [133:58:10<321:30:16,  7.26s/it]
 29%|██▉       | 65602/225000 [133:58:14<279:24:19,  6.31s/it]
 29%|██▉       | 65603/225000 [133:58:20<274:24:47,  6.20s/it]
 29%|██▉       | 65604/225000 [133:58:28<304:21:19,  6.87s/it]
 29%|██▉       | 65605/225000 [133:58:33<272:27:00,  6.15s/it]
 29%|██▉       | 65606/225000 [133:58:38<263:46:09,  5.96s/it]
 29%|██▉       | 65607/225000 [133:58:43<249:15:46,  5.63s/it]
 29%|██▉       | 65608/225000 [133:58:53<301:11:55,  6.80s/it]
 29%|██▉       | 65609/225000 [133:59:00<306:04:12,  6.91s/it]
 29%|██▉       | 65610/225000 [133:59:09<333:57:10,  7.54s/it]
 29%|██▉       | 65611/225000 [133:59:13<291:36:46,  6.59s/it]
 29%|██▉       | 65612/225000 [133:59:18<272:29:43,  6.15s/it]
 29%|██▉       | 65613/225000 [133:59:24<266:24:57,  6.02s/it]
 29%|██▉       | 65614/225000 [133:59:30<261:20:44,  5.90s/it]
 29%|██▉       | 65615/225000 [133:59:35<256:05:25,  5.78s/it]
 29%|██▉       | 65616/225000 [133:59:41<254:05:17,  5.74s/it]
 29%|██▉       | 65617/225000 [133:59:46<250:36:08,  5.66s/it]
 29%|██▉       | 65618/225000 [133:59:52<254:34:30,  5.75s/it]
 29%|██▉       | 65619/225000 [133:59:58<257:31:46,  5.82s/it]
 29%|██▉       | 65620/225000 [134:00:08<304:10:49,  6.87s/it]
 29%|██▉       | 65621/225000 [134:00:19<368:52:41,  8.33s/it]
 29%|██▉       | 65622/225000 [134:00:25<337:28:26,  7.62s/it]
 29%|██▉       | 65623/225000 [134:00:32<330:20:12,  7.46s/it]
 29%|██▉       | 65624/225000 [134:00:37<296:27:58,  6.70s/it]
 29%|██▉       | 65625/225000 [134:00:43<278:30:06,  6.29s/it]
 29%|██▉       | 65626/225000 [134:00:48<266:24:38,  6.02s/it]
 29%|██▉       | 65627/225000 [134:00:53<249:10:46,  5.63s/it]
 29%|██▉       | 65628/225000 [134:00:58<241:08:40,  5.45s/it]
 29%|██▉       | 65629/225000 [134:01:03<242:35:27,  5.48s/it]
 29%|██▉       | 65630/225000 [134:01:09<243:19:19,  5.50s/it]
 29%|██▉       | 65631/225000 [134:01:22<344:30:04,  7.78s/it]
 29%|██▉       | 65632/225000 [134:01:27<307:12:59,  6.94s/it]
 29%|██▉       | 65633/225000 [134:01:32<279:39:10,  6.32s/it]
 29%|██▉       | 65634/225000 [134:01:36<255:48:19,  5.78s/it]
 29%|██▉       | 65635/225000 [134:01:41<241:09:05,  5.45s/it]
 29%|██▉       | 65636/225000 [134:01:53<327:32:54,  7.40s/it]
 29%|██▉       | 65637/225000 [134:01:57<281:44:21,  6.36s/it]
 29%|██▉       | 65638/225000 [134:02:06<316:10:18,  7.14s/it]
 29%|██▉       | 65639/225000 [134:02:13<310:56:35,  7.02s/it]
 29%|██▉       | 65640/225000 [134:02:18<291:23:29,  6.58s/it]
 29%|██▉       | 65641/225000 [134:02:23<265:18:24,  5.99s/it]
 29%|██▉       | 65642/225000 [134:02:32<303:08:03,  6.85s/it]
 29%|██▉       | 65643/225000 [134:02:36<272:30:15,  6.16s/it]
 29%|██▉       | 65644/225000 [134:02:46<318:38:13,  7.20s/it]
 29%|██▉       | 65645/225000 [134:02:54<332:39:51,  7.52s/it]
 29%|██▉       | 65646/225000 [134:02:57<276:15:56,  6.24s/it]
 29%|██▉       | 65647/225000 [134:03:03<272:42:16,  6.16s/it]
 29%|██▉       | 65648/225000 [134:03:10<282:55:08,  6.39s/it]
 29%|██▉       | 65649/225000 [134:03:14<247:40:19,  5.60s/it]
 29%|██▉       | 65650/225000 [134:03:24<309:58:21,  7.00s/it]
                                                              
{'loss': '0.7537', 'grad_norm': '0.9205', 'learning_rate': '4.219e-05', 'epoch': '0.2918', 'num_input_tokens_seen': 222808800, 'train_runtime': '4.826e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65650/225000 [134:03:24<309:58:21,  7.00s/it]
 29%|██▉       | 65651/225000 [134:03:29<283:18:07,  6.40s/it]
 29%|██▉       | 65652/225000 [134:03:34<265:17:24,  5.99s/it]
 29%|██▉       | 65653/225000 [134:03:39<250:00:22,  5.65s/it]
 29%|██▉       | 65654/225000 [134:03:44<237:44:47,  5.37s/it]
 29%|██▉       | 65655/225000 [134:03:54<304:30:08,  6.88s/it]
 29%|██▉       | 65656/225000 [134:04:01<306:15:31,  6.92s/it]
 29%|██▉       | 65657/225000 [134:04:08<296:21:10,  6.70s/it]
 29%|██▉       | 65658/225000 [134:04:12<262:59:50,  5.94s/it]
 29%|██▉       | 65659/225000 [134:04:17<248:48:16,  5.62s/it]
 29%|██▉       | 65660/225000 [134:04:27<314:01:04,  7.09s/it]
 29%|██▉       | 65661/225000 [134:04:32<286:42:46,  6.48s/it]
 29%|██▉       | 65662/225000 [134:04:37<271:22:39,  6.13s/it]
 29%|██▉       | 65663/225000 [134:04:43<258:13:54,  5.83s/it]
 29%|██▉       | 65664/225000 [134:04:53<318:10:20,  7.19s/it]
 29%|██▉       | 65665/225000 [134:04:57<274:04:30,  6.19s/it]
 29%|██▉       | 65666/225000 [134:05:04<284:58:09,  6.44s/it]
 29%|██▉       | 65667/225000 [134:05:10<278:54:04,  6.30s/it]
 29%|██▉       | 65668/225000 [134:05:14<253:49:49,  5.74s/it]
 29%|██▉       | 65669/225000 [134:05:19<242:59:09,  5.49s/it]
 29%|██▉       | 65670/225000 [134:05:24<237:19:09,  5.36s/it]
 29%|██▉       | 65671/225000 [134:05:32<271:10:30,  6.13s/it]
 29%|██▉       | 65672/225000 [134:05:43<336:02:53,  7.59s/it]
 29%|██▉       | 65673/225000 [134:05:47<286:46:57,  6.48s/it]
 29%|██▉       | 65674/225000 [134:05:55<310:08:46,  7.01s/it]
 29%|██▉       | 65675/225000 [134:06:02<301:33:58,  6.81s/it]
 29%|██▉       | 65676/225000 [134:06:06<265:40:26,  6.00s/it]
 29%|██▉       | 65677/225000 [134:06:13<283:17:05,  6.40s/it]
 29%|██▉       | 65678/225000 [134:06:18<260:18:02,  5.88s/it]
 29%|██▉       | 65679/225000 [134:06:22<243:56:09,  5.51s/it]
 29%|██▉       | 65680/225000 [134:06:27<234:58:11,  5.31s/it]
 29%|██▉       | 65681/225000 [134:06:41<347:08:27,  7.84s/it]
 29%|██▉       | 65682/225000 [134:06:46<314:39:33,  7.11s/it]
 29%|██▉       | 65683/225000 [134:06:52<300:12:52,  6.78s/it]
 29%|██▉       | 65684/225000 [134:06:57<268:55:10,  6.08s/it]
 29%|██▉       | 65685/225000 [134:07:01<248:21:09,  5.61s/it]
 29%|██▉       | 65686/225000 [134:07:08<267:02:10,  6.03s/it]
 29%|██▉       | 65687/225000 [134:07:17<303:56:46,  6.87s/it]
 29%|██▉       | 65688/225000 [134:07:22<269:43:11,  6.09s/it]
 29%|██▉       | 65689/225000 [134:07:29<294:36:22,  6.66s/it]
 29%|██▉       | 65690/225000 [134:07:36<290:43:54,  6.57s/it]
 29%|██▉       | 65691/225000 [134:07:41<274:25:34,  6.20s/it]
 29%|██▉       | 65692/225000 [134:07:47<263:07:09,  5.95s/it]
 29%|██▉       | 65693/225000 [134:07:56<307:00:25,  6.94s/it]
 29%|██▉       | 65694/225000 [134:08:05<331:30:09,  7.49s/it]
 29%|██▉       | 65695/225000 [134:08:09<287:43:40,  6.50s/it]
 29%|██▉       | 65696/225000 [134:08:14<266:18:47,  6.02s/it]
 29%|██▉       | 65697/225000 [134:08:18<245:46:44,  5.55s/it]
 29%|██▉       | 65698/225000 [134:08:30<334:03:19,  7.55s/it]
 29%|██▉       | 65699/225000 [134:08:36<304:21:48,  6.88s/it]
 29%|██▉       | 65700/225000 [134:08:41<282:05:06,  6.37s/it]
                                                              
{'loss': '0.7351', 'grad_norm': '0.7256', 'learning_rate': '4.218e-05', 'epoch': '0.292', 'num_input_tokens_seen': 222957280, 'train_runtime': '4.829e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65700/225000 [134:08:41<282:05:06,  6.37s/it]
 29%|██▉       | 65701/225000 [134:08:46<265:52:21,  6.01s/it]
 29%|██▉       | 65702/225000 [134:08:51<252:00:46,  5.70s/it]
 29%|██▉       | 65703/225000 [134:08:57<251:57:44,  5.69s/it]
 29%|██▉       | 65704/225000 [134:09:02<248:25:41,  5.61s/it]
 29%|██▉       | 65705/225000 [134:09:12<306:43:59,  6.93s/it]
 29%|██▉       | 65706/225000 [134:09:21<334:16:38,  7.55s/it]
 29%|██▉       | 65707/225000 [134:09:25<288:47:52,  6.53s/it]
 29%|██▉       | 65708/225000 [134:09:30<261:09:05,  5.90s/it]
 29%|██▉       | 65709/225000 [134:09:35<257:07:12,  5.81s/it]
 29%|██▉       | 65710/225000 [134:09:46<316:18:11,  7.15s/it]
 29%|██▉       | 65711/225000 [134:09:51<293:58:10,  6.64s/it]
 29%|██▉       | 65712/225000 [134:09:56<270:19:59,  6.11s/it]
 29%|██▉       | 65713/225000 [134:10:01<255:03:19,  5.76s/it]
 29%|██▉       | 65714/225000 [134:10:06<244:35:58,  5.53s/it]
 29%|██▉       | 65715/225000 [134:10:11<240:56:47,  5.45s/it]
 29%|██▉       | 65716/225000 [134:10:22<317:50:39,  7.18s/it]
 29%|██▉       | 65717/225000 [134:10:29<311:21:17,  7.04s/it]
 29%|██▉       | 65718/225000 [134:10:37<319:54:16,  7.23s/it]
 29%|██▉       | 65719/225000 [134:10:40<270:57:09,  6.12s/it]
 29%|██▉       | 65720/225000 [134:10:48<295:33:37,  6.68s/it]
 29%|██▉       | 65721/225000 [134:10:57<318:04:24,  7.19s/it]
 29%|██▉       | 65722/225000 [134:11:01<279:03:23,  6.31s/it]
 29%|██▉       | 65723/225000 [134:11:08<295:02:14,  6.67s/it]
 29%|██▉       | 65724/225000 [134:11:12<258:04:44,  5.83s/it]
 29%|██▉       | 65725/225000 [134:11:21<295:30:09,  6.68s/it]
 29%|██▉       | 65726/225000 [134:11:29<309:24:59,  6.99s/it]
 29%|██▉       | 65727/225000 [134:11:33<270:15:02,  6.11s/it]
 29%|██▉       | 65728/225000 [134:11:37<250:36:12,  5.66s/it]
 29%|██▉       | 65729/225000 [134:11:42<234:28:23,  5.30s/it]
 29%|██▉       | 65730/225000 [134:11:47<234:01:19,  5.29s/it]
 29%|██▉       | 65731/225000 [134:11:57<294:23:35,  6.65s/it]
 29%|██▉       | 65732/225000 [134:12:04<307:16:46,  6.95s/it]
 29%|██▉       | 65733/225000 [134:12:12<314:13:38,  7.10s/it]
 29%|██▉       | 65734/225000 [134:12:16<269:41:19,  6.10s/it]
 29%|██▉       | 65735/225000 [134:12:24<302:24:24,  6.84s/it]
 29%|██▉       | 65736/225000 [134:12:28<265:19:16,  6.00s/it]
 29%|██▉       | 65737/225000 [134:12:34<259:34:01,  5.87s/it]
 29%|██▉       | 65738/225000 [134:12:39<256:04:22,  5.79s/it]
 29%|██▉       | 65739/225000 [134:12:45<257:47:02,  5.83s/it]
 29%|██▉       | 65740/225000 [134:12:51<251:47:16,  5.69s/it]
 29%|██▉       | 65741/225000 [134:12:56<242:31:56,  5.48s/it]
 29%|██▉       | 65742/225000 [134:13:01<245:02:58,  5.54s/it]
 29%|██▉       | 65743/225000 [134:13:15<349:15:05,  7.89s/it]
 29%|██▉       | 65744/225000 [134:13:22<339:11:29,  7.67s/it]
 29%|██▉       | 65745/225000 [134:13:26<294:04:36,  6.65s/it]
 29%|██▉       | 65746/225000 [134:13:31<274:34:28,  6.21s/it]
 29%|██▉       | 65747/225000 [134:13:40<311:42:13,  7.05s/it]
 29%|██▉       | 65748/225000 [134:13:52<369:26:00,  8.35s/it]
 29%|██▉       | 65749/225000 [134:13:56<315:11:45,  7.13s/it]
 29%|██▉       | 65750/225000 [134:14:01<288:37:24,  6.52s/it]
                                                              
{'loss': '0.734', 'grad_norm': '0.7442', 'learning_rate': '4.216e-05', 'epoch': '0.2922', 'num_input_tokens_seen': 223104288, 'train_runtime': '4.832e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65750/225000 [134:14:01<288:37:24,  6.52s/it]
 29%|██▉       | 65751/225000 [134:14:06<272:50:38,  6.17s/it]
 29%|██▉       | 65752/225000 [134:14:12<262:13:23,  5.93s/it]
 29%|██▉       | 65753/225000 [134:14:17<252:38:14,  5.71s/it]
 29%|██▉       | 65754/225000 [134:14:22<248:18:37,  5.61s/it]
 29%|██▉       | 65755/225000 [134:14:29<256:34:52,  5.80s/it]
 29%|██▉       | 65756/225000 [134:14:42<352:06:07,  7.96s/it]
 29%|██▉       | 65757/225000 [134:14:49<339:09:36,  7.67s/it]
 29%|██▉       | 65758/225000 [134:14:53<299:01:30,  6.76s/it]
 29%|██▉       | 65759/225000 [134:14:58<273:17:05,  6.18s/it]
 29%|██▉       | 65760/225000 [134:15:08<328:47:53,  7.43s/it]
 29%|██▉       | 65761/225000 [134:15:15<312:16:53,  7.06s/it]
 29%|██▉       | 65762/225000 [134:15:19<277:23:27,  6.27s/it]
 29%|██▉       | 65763/225000 [134:15:28<314:24:11,  7.11s/it]
 29%|██▉       | 65764/225000 [134:15:35<311:33:47,  7.04s/it]
 29%|██▉       | 65765/225000 [134:15:40<277:52:18,  6.28s/it]
 29%|██▉       | 65766/225000 [134:15:44<254:07:30,  5.75s/it]
 29%|██▉       | 65767/225000 [134:15:49<238:37:56,  5.40s/it]
 29%|██▉       | 65768/225000 [134:16:01<324:37:51,  7.34s/it]
 29%|██▉       | 65769/225000 [134:16:06<297:45:26,  6.73s/it]
 29%|██▉       | 65770/225000 [134:16:12<284:09:32,  6.42s/it]
 29%|██▉       | 65771/225000 [134:16:20<313:58:53,  7.10s/it]
 29%|██▉       | 65772/225000 [134:16:25<281:40:49,  6.37s/it]
 29%|██▉       | 65773/225000 [134:16:30<259:22:23,  5.86s/it]
 29%|██▉       | 65774/225000 [134:16:35<250:05:31,  5.65s/it]
 29%|██▉       | 65775/225000 [134:16:44<293:57:48,  6.65s/it]
 29%|██▉       | 65776/225000 [134:16:55<349:52:26,  7.91s/it]
 29%|██▉       | 65777/225000 [134:16:58<292:45:59,  6.62s/it]
 29%|██▉       | 65778/225000 [134:17:07<318:39:35,  7.20s/it]
 29%|██▉       | 65779/225000 [134:17:13<309:49:47,  7.01s/it]
 29%|██▉       | 65780/225000 [134:17:19<293:33:49,  6.64s/it]
 29%|██▉       | 65781/225000 [134:17:23<261:34:36,  5.91s/it]
 29%|██▉       | 65782/225000 [134:17:33<318:11:06,  7.19s/it]
 29%|██▉       | 65783/225000 [134:17:40<304:20:32,  6.88s/it]
 29%|██▉       | 65784/225000 [134:17:44<271:18:58,  6.13s/it]
 29%|██▉       | 65785/225000 [134:17:54<321:20:55,  7.27s/it]
 29%|██▉       | 65786/225000 [134:18:00<312:04:48,  7.06s/it]
 29%|██▉       | 65787/225000 [134:18:07<302:38:40,  6.84s/it]
 29%|██▉       | 65788/225000 [134:18:13<295:31:48,  6.68s/it]
 29%|██▉       | 65789/225000 [134:18:17<263:51:27,  5.97s/it]
 29%|██▉       | 65790/225000 [134:18:28<325:27:26,  7.36s/it]
 29%|██▉       | 65791/225000 [134:18:33<291:41:21,  6.60s/it]
 29%|██▉       | 65792/225000 [134:18:39<279:55:14,  6.33s/it]
 29%|██▉       | 65793/225000 [134:18:44<273:08:26,  6.18s/it]
 29%|██▉       | 65794/225000 [134:18:53<308:52:05,  6.98s/it]
 29%|██▉       | 65795/225000 [134:18:58<274:21:34,  6.20s/it]
 29%|██▉       | 65796/225000 [134:19:02<250:39:22,  5.67s/it]
 29%|██▉       | 65797/225000 [134:19:07<241:45:23,  5.47s/it]
 29%|██▉       | 65798/225000 [134:19:13<242:42:54,  5.49s/it]
 29%|██▉       | 65799/225000 [134:19:23<313:19:01,  7.09s/it]
 29%|██▉       | 65800/225000 [134:19:31<322:46:07,  7.30s/it]
                                                              
{'loss': '0.737', 'grad_norm': '0.7929', 'learning_rate': '4.215e-05', 'epoch': '0.2924', 'num_input_tokens_seen': 223252488, 'train_runtime': '4.836e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65800/225000 [134:19:31<322:46:07,  7.30s/it]
 29%|██▉       | 65801/225000 [134:19:35<279:13:28,  6.31s/it]
 29%|██▉       | 65802/225000 [134:19:40<261:58:43,  5.92s/it]
 29%|██▉       | 65803/225000 [134:19:45<253:24:34,  5.73s/it]
 29%|██▉       | 65804/225000 [134:19:51<248:42:39,  5.62s/it]
 29%|██▉       | 65805/225000 [134:19:56<245:11:28,  5.54s/it]
 29%|██▉       | 65806/225000 [134:20:02<243:22:41,  5.50s/it]
 29%|██▉       | 65807/225000 [134:20:07<235:29:34,  5.33s/it]
 29%|██▉       | 65808/225000 [134:20:19<334:36:51,  7.57s/it]
 29%|██▉       | 65809/225000 [134:20:26<322:47:12,  7.30s/it]
 29%|██▉       | 65810/225000 [134:20:30<276:13:35,  6.25s/it]
 29%|██▉       | 65811/225000 [134:20:41<347:55:22,  7.87s/it]
 29%|██▉       | 65812/225000 [134:20:45<291:44:21,  6.60s/it]
 29%|██▉       | 65813/225000 [134:20:55<332:55:23,  7.53s/it]
 29%|██▉       | 65814/225000 [134:20:59<289:44:36,  6.55s/it]
 29%|██▉       | 65815/225000 [134:21:04<262:02:11,  5.93s/it]
 29%|██▉       | 65816/225000 [134:21:09<256:54:49,  5.81s/it]
 29%|██▉       | 65817/225000 [134:21:20<329:34:57,  7.45s/it]
 29%|██▉       | 65818/225000 [134:21:24<278:42:48,  6.30s/it]
 29%|██▉       | 65819/225000 [134:21:34<323:54:11,  7.33s/it]
 29%|██▉       | 65820/225000 [134:21:38<286:44:54,  6.49s/it]
 29%|██▉       | 65821/225000 [134:21:43<261:08:11,  5.91s/it]
 29%|██▉       | 65822/225000 [134:21:47<240:15:36,  5.43s/it]
 29%|██▉       | 65823/225000 [134:22:00<335:48:29,  7.59s/it]
 29%|██▉       | 65824/225000 [134:22:04<294:41:02,  6.66s/it]
 29%|██▉       | 65825/225000 [134:22:08<259:41:37,  5.87s/it]
 29%|██▉       | 65826/225000 [134:22:19<325:17:11,  7.36s/it]
 29%|██▉       | 65827/225000 [134:22:25<300:16:33,  6.79s/it]
 29%|██▉       | 65828/225000 [134:22:32<312:36:11,  7.07s/it]
 29%|██▉       | 65829/225000 [134:22:37<278:57:54,  6.31s/it]
 29%|██▉       | 65830/225000 [134:22:42<258:05:17,  5.84s/it]
 29%|██▉       | 65831/225000 [134:22:47<252:53:31,  5.72s/it]
 29%|██▉       | 65832/225000 [134:22:58<323:10:15,  7.31s/it]
 29%|██▉       | 65833/225000 [134:23:03<286:14:33,  6.47s/it]
 29%|██▉       | 65834/225000 [134:23:07<260:44:06,  5.90s/it]
 29%|██▉       | 65835/225000 [134:23:12<247:24:56,  5.60s/it]
 29%|██▉       | 65836/225000 [134:23:20<279:12:43,  6.32s/it]
 29%|██▉       | 65837/225000 [134:23:30<325:55:16,  7.37s/it]
 29%|██▉       | 65838/225000 [134:23:34<284:15:19,  6.43s/it]
 29%|██▉       | 65839/225000 [134:23:39<259:54:35,  5.88s/it]
 29%|██▉       | 65840/225000 [134:23:49<319:18:11,  7.22s/it]
 29%|██▉       | 65841/225000 [134:23:57<326:10:56,  7.38s/it]
 29%|██▉       | 65842/225000 [134:24:02<292:11:38,  6.61s/it]
 29%|██▉       | 65843/225000 [134:24:07<276:32:20,  6.26s/it]
 29%|██▉       | 65844/225000 [134:24:13<271:59:21,  6.15s/it]
 29%|██▉       | 65845/225000 [134:24:19<266:55:34,  6.04s/it]
 29%|██▉       | 65846/225000 [134:24:24<255:27:24,  5.78s/it]
 29%|██▉       | 65847/225000 [134:24:29<251:12:56,  5.68s/it]
 29%|██▉       | 65848/225000 [134:24:41<336:02:22,  7.60s/it]
 29%|██▉       | 65849/225000 [134:24:49<336:36:02,  7.61s/it]
 29%|██▉       | 65850/225000 [134:24:53<291:50:09,  6.60s/it]
                                                              
{'loss': '0.7264', 'grad_norm': '0.8378', 'learning_rate': '4.214e-05', 'epoch': '0.2927', 'num_input_tokens_seen': 223399376, 'train_runtime': '4.839e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65850/225000 [134:24:53<291:50:09,  6.60s/it]
 29%|██▉       | 65851/225000 [134:25:03<334:28:10,  7.57s/it]
 29%|██▉       | 65852/225000 [134:25:08<295:31:32,  6.68s/it]
 29%|██▉       | 65853/225000 [134:25:12<262:35:28,  5.94s/it]
 29%|██▉       | 65854/225000 [134:25:24<349:15:27,  7.90s/it]
 29%|██▉       | 65855/225000 [134:25:30<320:12:41,  7.24s/it]
 29%|██▉       | 65856/225000 [134:25:38<332:55:33,  7.53s/it]
 29%|██▉       | 65857/225000 [134:25:46<335:24:41,  7.59s/it]
 29%|██▉       | 65858/225000 [134:25:49<276:42:37,  6.26s/it]
 29%|██▉       | 65859/225000 [134:25:58<309:45:29,  7.01s/it]
 29%|██▉       | 65860/225000 [134:26:02<265:46:03,  6.01s/it]
 29%|██▉       | 65861/225000 [134:26:06<242:58:22,  5.50s/it]
 29%|██▉       | 65862/225000 [134:26:17<314:22:48,  7.11s/it]
 29%|██▉       | 65863/225000 [134:26:21<271:49:07,  6.15s/it]
 29%|██▉       | 65864/225000 [134:26:26<259:00:04,  5.86s/it]
 29%|██▉       | 65865/225000 [134:26:31<254:44:26,  5.76s/it]
 29%|██▉       | 65866/225000 [134:26:36<244:40:02,  5.53s/it]
 29%|██▉       | 65867/225000 [134:26:42<240:28:11,  5.44s/it]
 29%|██▉       | 65868/225000 [134:26:47<233:28:42,  5.28s/it]
 29%|██▉       | 65869/225000 [134:26:55<278:31:15,  6.30s/it]
 29%|██▉       | 65870/225000 [134:27:07<347:38:18,  7.86s/it]
 29%|██▉       | 65871/225000 [134:27:11<300:54:35,  6.81s/it]
 29%|██▉       | 65872/225000 [134:27:15<269:01:15,  6.09s/it]
 29%|██▉       | 65873/225000 [134:27:28<349:18:36,  7.90s/it]
 29%|██▉       | 65874/225000 [134:27:31<294:28:20,  6.66s/it]
 29%|██▉       | 65875/225000 [134:27:39<312:17:41,  7.07s/it]
 29%|██▉       | 65876/225000 [134:27:45<288:10:10,  6.52s/it]
 29%|██▉       | 65877/225000 [134:27:50<272:50:03,  6.17s/it]
 29%|██▉       | 65878/225000 [134:27:55<258:43:37,  5.85s/it]
 29%|██▉       | 65879/225000 [134:28:00<249:16:01,  5.64s/it]
 29%|██▉       | 65880/225000 [134:28:12<325:13:35,  7.36s/it]
 29%|██▉       | 65881/225000 [134:28:20<341:00:59,  7.72s/it]
 29%|██▉       | 65882/225000 [134:28:24<293:20:53,  6.64s/it]
 29%|██▉       | 65883/225000 [134:28:31<294:00:50,  6.65s/it]
 29%|██▉       | 65884/225000 [134:28:39<308:05:40,  6.97s/it]
 29%|██▉       | 65885/225000 [134:28:44<288:41:55,  6.53s/it]
 29%|██▉       | 65886/225000 [134:28:52<301:43:21,  6.83s/it]
 29%|██▉       | 65887/225000 [134:28:56<268:26:36,  6.07s/it]
 29%|██▉       | 65888/225000 [134:29:00<241:33:25,  5.47s/it]
 29%|██▉       | 65889/225000 [134:29:13<335:41:53,  7.60s/it]
 29%|██▉       | 65890/225000 [134:29:17<295:24:57,  6.68s/it]
 29%|██▉       | 65891/225000 [134:29:22<271:24:50,  6.14s/it]
 29%|██▉       | 65892/225000 [134:29:27<253:21:51,  5.73s/it]
 29%|██▉       | 65893/225000 [134:29:32<245:53:58,  5.56s/it]
 29%|██▉       | 65894/225000 [134:29:45<349:56:32,  7.92s/it]
 29%|██▉       | 65895/225000 [134:29:50<304:40:58,  6.89s/it]
 29%|██▉       | 65896/225000 [134:29:54<273:17:41,  6.18s/it]
 29%|██▉       | 65897/225000 [134:29:59<248:20:36,  5.62s/it]
 29%|██▉       | 65898/225000 [134:30:11<334:10:15,  7.56s/it]
 29%|██▉       | 65899/225000 [134:30:16<307:50:47,  6.97s/it]
 29%|██▉       | 65900/225000 [134:30:22<286:57:52,  6.49s/it]
                                                              
{'loss': '0.7462', 'grad_norm': '0.938', 'learning_rate': '4.212e-05', 'epoch': '0.2929', 'num_input_tokens_seen': 223547584, 'train_runtime': '4.842e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65900/225000 [134:30:22<286:57:52,  6.49s/it]
 29%|██▉       | 65901/225000 [134:30:27<267:38:50,  6.06s/it]
 29%|██▉       | 65902/225000 [134:30:33<262:44:19,  5.95s/it]
 29%|██▉       | 65903/225000 [134:30:43<318:26:57,  7.21s/it]
 29%|██▉       | 65904/225000 [134:30:53<355:26:10,  8.04s/it]
 29%|██▉       | 65905/225000 [134:30:56<294:23:53,  6.66s/it]
 29%|██▉       | 65906/225000 [134:31:05<318:01:42,  7.20s/it]
 29%|██▉       | 65907/225000 [134:31:12<325:27:08,  7.36s/it]
 29%|██▉       | 65908/225000 [134:31:17<291:33:30,  6.60s/it]
 29%|██▉       | 65909/225000 [134:31:24<294:27:18,  6.66s/it]
 29%|██▉       | 65910/225000 [134:31:30<290:43:47,  6.58s/it]
 29%|██▉       | 65911/225000 [134:31:38<301:17:02,  6.82s/it]
 29%|██▉       | 65912/225000 [134:31:46<321:49:23,  7.28s/it]
 29%|██▉       | 65913/225000 [134:31:51<292:38:57,  6.62s/it]
 29%|██▉       | 65914/225000 [134:31:57<283:57:26,  6.43s/it]
 29%|██▉       | 65915/225000 [134:32:05<306:04:55,  6.93s/it]
 29%|██▉       | 65916/225000 [134:32:09<267:57:00,  6.06s/it]
 29%|██▉       | 65917/225000 [134:32:17<284:48:17,  6.45s/it]
 29%|██▉       | 65918/225000 [134:32:22<271:50:12,  6.15s/it]
 29%|██▉       | 65919/225000 [134:32:31<312:59:46,  7.08s/it]
 29%|██▉       | 65920/225000 [134:32:37<287:56:13,  6.52s/it]
 29%|██▉       | 65921/225000 [134:32:44<296:47:46,  6.72s/it]
 29%|██▉       | 65922/225000 [134:32:48<260:13:36,  5.89s/it]
 29%|██▉       | 65923/225000 [134:32:53<251:08:36,  5.68s/it]
 29%|██▉       | 65924/225000 [134:32:58<247:07:00,  5.59s/it]
 29%|██▉       | 65925/225000 [134:33:04<248:56:26,  5.63s/it]
 29%|██▉       | 65926/225000 [134:33:10<250:46:48,  5.68s/it]
 29%|██▉       | 65927/225000 [134:33:15<251:28:31,  5.69s/it]
 29%|██▉       | 65928/225000 [134:33:21<251:01:39,  5.68s/it]
 29%|██▉       | 65929/225000 [134:33:28<261:35:00,  5.92s/it]
 29%|██▉       | 65930/225000 [134:33:40<351:20:34,  7.95s/it]
 29%|██▉       | 65931/225000 [134:33:47<338:41:41,  7.67s/it]
 29%|██▉       | 65932/225000 [134:33:55<341:53:27,  7.74s/it]
 29%|██▉       | 65933/225000 [134:34:00<309:16:28,  7.00s/it]
 29%|██▉       | 65934/225000 [134:34:08<316:11:51,  7.16s/it]
 29%|██▉       | 65935/225000 [134:34:12<275:20:31,  6.23s/it]
 29%|██▉       | 65936/225000 [134:34:17<263:09:42,  5.96s/it]
 29%|██▉       | 65937/225000 [134:34:28<324:25:10,  7.34s/it]
 29%|██▉       | 65938/225000 [134:34:35<315:15:49,  7.14s/it]
 29%|██▉       | 65939/225000 [134:34:39<284:15:19,  6.43s/it]
 29%|██▉       | 65940/225000 [134:34:49<320:05:01,  7.24s/it]
 29%|██▉       | 65941/225000 [134:34:56<324:22:43,  7.34s/it]
 29%|██▉       | 65942/225000 [134:35:02<298:25:04,  6.75s/it]
 29%|██▉       | 65943/225000 [134:35:08<300:41:13,  6.81s/it]
 29%|██▉       | 65944/225000 [134:35:14<280:47:45,  6.36s/it]
 29%|██▉       | 65945/225000 [134:35:18<249:28:45,  5.65s/it]
 29%|██▉       | 65946/225000 [134:35:28<315:32:28,  7.14s/it]
 29%|██▉       | 65947/225000 [134:35:33<288:31:38,  6.53s/it]
 29%|██▉       | 65948/225000 [134:35:42<311:04:40,  7.04s/it]
 29%|██▉       | 65949/225000 [134:35:47<285:20:41,  6.46s/it]
 29%|██▉       | 65950/225000 [134:35:55<305:32:56,  6.92s/it]
                                                              
{'loss': '0.7333', 'grad_norm': '0.9335', 'learning_rate': '4.211e-05', 'epoch': '0.2931', 'num_input_tokens_seen': 223696792, 'train_runtime': '4.846e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 65950/225000 [134:35:55<305:32:56,  6.92s/it]
 29%|██▉       | 65951/225000 [134:36:00<278:27:13,  6.30s/it]
 29%|██▉       | 65952/225000 [134:36:05<260:47:25,  5.90s/it]
 29%|██▉       | 65953/225000 [134:36:10<254:56:07,  5.77s/it]
 29%|██▉       | 65954/225000 [134:36:15<243:03:05,  5.50s/it]
 29%|██▉       | 65955/225000 [134:36:24<284:50:48,  6.45s/it]
 29%|██▉       | 65956/225000 [134:36:35<347:03:44,  7.86s/it]
 29%|██▉       | 65957/225000 [134:36:39<299:54:19,  6.79s/it]
 29%|██▉       | 65958/225000 [134:36:44<273:23:19,  6.19s/it]
 29%|██▉       | 65959/225000 [134:36:49<260:27:38,  5.90s/it]
 29%|██▉       | 65960/225000 [134:37:02<350:55:22,  7.94s/it]
 29%|██▉       | 65961/225000 [134:37:05<291:50:40,  6.61s/it]
 29%|██▉       | 65962/225000 [134:37:14<315:01:41,  7.13s/it]
 29%|██▉       | 65963/225000 [134:37:17<266:09:46,  6.02s/it]
 29%|██▉       | 65964/225000 [134:37:26<307:51:32,  6.97s/it]
 29%|██▉       | 65965/225000 [134:37:34<312:55:45,  7.08s/it]
 29%|██▉       | 65966/225000 [134:37:39<287:41:44,  6.51s/it]
 29%|██▉       | 65967/225000 [134:37:44<271:54:40,  6.16s/it]
 29%|██▉       | 65968/225000 [134:37:49<261:21:46,  5.92s/it]
 29%|██▉       | 65969/225000 [134:37:55<253:42:31,  5.74s/it]
 29%|██▉       | 65970/225000 [134:38:00<245:58:04,  5.57s/it]
 29%|██▉       | 65971/225000 [134:38:05<237:44:27,  5.38s/it]
 29%|██▉       | 65972/225000 [134:38:13<272:33:07,  6.17s/it]
 29%|██▉       | 65973/225000 [134:38:23<318:08:50,  7.20s/it]
 29%|██▉       | 65974/225000 [134:38:27<285:07:46,  6.45s/it]
 29%|██▉       | 65975/225000 [134:38:33<270:39:17,  6.13s/it]
 29%|██▉       | 65976/225000 [134:38:38<260:49:06,  5.90s/it]
 29%|██▉       | 65977/225000 [134:38:43<252:51:12,  5.72s/it]
 29%|██▉       | 65978/225000 [134:38:51<276:10:18,  6.25s/it]
 29%|██▉       | 65979/225000 [134:39:03<357:44:13,  8.10s/it]
 29%|██▉       | 65980/225000 [134:39:07<305:42:59,  6.92s/it]
 29%|██▉       | 65981/225000 [134:39:18<355:24:30,  8.05s/it]
 29%|██▉       | 65982/225000 [134:39:22<300:40:48,  6.81s/it]
 29%|██▉       | 65983/225000 [134:39:27<272:52:36,  6.18s/it]
 29%|██▉       | 65984/225000 [134:39:37<325:04:14,  7.36s/it]
 29%|██▉       | 65985/225000 [134:39:41<279:50:11,  6.34s/it]
 29%|██▉       | 65986/225000 [134:39:50<322:22:19,  7.30s/it]
 29%|██▉       | 65987/225000 [134:39:54<273:47:40,  6.20s/it]
 29%|██▉       | 65988/225000 [134:40:00<270:46:41,  6.13s/it]
 29%|██▉       | 65989/225000 [134:40:06<266:36:09,  6.04s/it]
 29%|██▉       | 65990/225000 [134:40:11<260:12:16,  5.89s/it]
 29%|██▉       | 65991/225000 [134:40:17<256:42:24,  5.81s/it]
 29%|██▉       | 65992/225000 [134:40:22<251:23:29,  5.69s/it]
 29%|██▉       | 65993/225000 [134:40:33<324:20:29,  7.34s/it]
 29%|██▉       | 65994/225000 [134:40:44<361:48:15,  8.19s/it]
 29%|██▉       | 65995/225000 [134:40:51<346:48:41,  7.85s/it]
 29%|██▉       | 65996/225000 [134:40:55<304:20:16,  6.89s/it]
 29%|██▉       | 65997/225000 [134:41:00<278:19:54,  6.30s/it]
 29%|██▉       | 65998/225000 [134:41:06<269:13:33,  6.10s/it]
 29%|██▉       | 65999/225000 [134:41:11<261:24:30,  5.92s/it]
 29%|██▉       | 66000/225000 [134:41:17<260:36:02,  5.90s/it]
                                                              
{'loss': '0.743', 'grad_norm': '0.6731', 'learning_rate': '4.21e-05', 'epoch': '0.2933', 'num_input_tokens_seen': 223847072, 'train_runtime': '4.849e+05', 'train_tokens_per_second': '461.7'}

 29%|██▉       | 66000/225000 [134:41:17<260:36:02,  5.90s/it]
 29%|██▉       | 66001/225000 [134:41:29<334:25:01,  7.57s/it]
 29%|██▉       | 66002/225000 [134:41:37<347:30:11,  7.87s/it]
 29%|██▉       | 66003/225000 [134:41:42<304:00:11,  6.88s/it]
 29%|██▉       | 66004/225000 [134:41:46<274:12:33,  6.21s/it]
 29%|██▉       | 66005/225000 [134:41:52<263:22:07,  5.96s/it]
 29%|██▉       | 66006/225000 [134:41:57<258:40:40,  5.86s/it]
 29%|██▉       | 66007/225000 [134:42:03<255:26:04,  5.78s/it]
 29%|██▉       | 66008/225000 [134:42:11<280:25:07,  6.35s/it]
 29%|██▉       | 66009/225000 [134:42:24<367:23:44,  8.32s/it]
 29%|██▉       | 66010/225000 [134:42:29<329:49:02,  7.47s/it]
 29%|██▉       | 66011/225000 [134:42:34<299:05:42,  6.77s/it]
 29%|██▉       | 66012/225000 [134:42:40<287:31:15,  6.51s/it]
 29%|██▉       | 66013/225000 [134:42:47<291:21:53,  6.60s/it]
 29%|██▉       | 66014/225000 [134:42:57<333:05:26,  7.54s/it]
 29%|██▉       | 66015/225000 [134:43:01<290:39:53,  6.58s/it]
 29%|██▉       | 66016/225000 [134:43:06<262:33:50,  5.95s/it]
 29%|██▉       | 66017/225000 [134:43:11<253:27:49,  5.74s/it]
 29%|██▉       | 66018/225000 [134:43:17<265:23:21,  6.01s/it]
 29%|██▉       | 66019/225000 [134:43:30<354:54:29,  8.04s/it]
 29%|██▉       | 66020/225000 [134:43:36<322:18:10,  7.30s/it]
 29%|██▉       | 66021/225000 [134:43:44<328:56:49,  7.45s/it]
 29%|██▉       | 66022/225000 [134:43:49<305:10:21,  6.91s/it]
 29%|██▉       | 66023/225000 [134:43:57<313:53:10,  7.11s/it]
 29%|██▉       | 66024/225000 [134:44:04<310:24:04,  7.03s/it]
 29%|██▉       | 66025/225000 [134:44:10<299:09:22,  6.77s/it]
 29%|██▉       | 66026/225000 [134:44:17<307:50:27,  6.97s/it]
 29%|██▉       | 66027/225000 [134:44:24<306:54:54,  6.95s/it]
 29%|██▉       | 66028/225000 [134:44:29<277:28:54,  6.28s/it]
 29%|██▉       | 66029/225000 [134:44:34<264:24:33,  5.99s/it]
 29%|██▉       | 66030/225000 [134:44:39<251:00:08,  5.68s/it]
 29%|██▉       | 66031/225000 [134:44:45<253:14:50,  5.74s/it]
 29%|██▉       | 66032/225000 [134:44:58<353:45:05,  8.01s/it]
 29%|██▉       | 66033/225000 [134:45:03<308:15:14,  6.98s/it]
 29%|██▉       | 66034/225000 [134:45:08<281:12:14,  6.37s/it]
 29%|██▉       | 66035/225000 [134:45:12<253:52:15,  5.75s/it]
 29%|██▉       | 66036/225000 [134:45:20<285:05:55,  6.46s/it]
 29%|██▉       | 66037/225000 [134:45:30<327:22:15,  7.41s/it]
 29%|██▉       | 66038/225000 [134:45:34<283:21:01,  6.42s/it]
 29%|██▉       | 66039/225000 [134:45:45<346:47:14,  7.85s/it]
 29%|██▉       | 66040/225000 [134:45:50<299:39:25,  6.79s/it]
 29%|██▉       | 66041/225000 [134:45:57<307:54:13,  6.97s/it]
 29%|██▉       | 66042/225000 [134:46:04<309:58:27,  7.02s/it]
 29%|██▉       | 66043/225000 [134:46:09<281:56:58,  6.39s/it]
 29%|██▉       | 66044/225000 [134:46:18<319:39:14,  7.24s/it]
 29%|██▉       | 66045/225000 [134:46:23<287:39:12,  6.51s/it]
 29%|██▉       | 66046/225000 [134:46:28<265:01:04,  6.00s/it]
 29%|██▉       | 66047/225000 [134:46:33<254:46:23,  5.77s/it]
 29%|██▉       | 66048/225000 [134:46:38<242:35:51,  5.49s/it]
 29%|██▉       | 66049/225000 [134:46:51<346:48:45,  7.85s/it]
 29%|██▉       | 66050/225000 [134:46:58<330:11:23,  7.48s/it]
                                                              
{'loss': '0.7236', 'grad_norm': '0.9626', 'learning_rate': '4.208e-05', 'epoch': '0.2936', 'num_input_tokens_seen': 223996808, 'train_runtime': '4.852e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66050/225000 [134:46:58<330:11:23,  7.48s/it]
 29%|██▉       | 66051/225000 [134:47:03<300:21:02,  6.80s/it]
 29%|██▉       | 66052/225000 [134:47:09<284:25:24,  6.44s/it]
 29%|██▉       | 66053/225000 [134:47:15<277:33:03,  6.29s/it]
 29%|██▉       | 66054/225000 [134:47:21<274:38:46,  6.22s/it]
 29%|██▉       | 66055/225000 [134:47:27<271:00:08,  6.14s/it]
 29%|██▉       | 66056/225000 [134:47:36<316:40:16,  7.17s/it]
 29%|██▉       | 66057/225000 [134:47:46<355:39:46,  8.06s/it]
 29%|██▉       | 66058/225000 [134:47:51<308:13:56,  6.98s/it]
 29%|██▉       | 66059/225000 [134:47:57<304:19:05,  6.89s/it]
 29%|██▉       | 66060/225000 [134:48:05<316:57:54,  7.18s/it]
 29%|██▉       | 66061/225000 [134:48:11<297:13:38,  6.73s/it]
 29%|██▉       | 66062/225000 [134:48:20<324:06:15,  7.34s/it]
 29%|██▉       | 66063/225000 [134:48:26<308:25:51,  6.99s/it]
 29%|██▉       | 66064/225000 [134:48:32<293:07:59,  6.64s/it]
 29%|██▉       | 66065/225000 [134:48:37<276:13:16,  6.26s/it]
 29%|██▉       | 66066/225000 [134:48:47<323:49:22,  7.33s/it]
 29%|██▉       | 66067/225000 [134:48:53<304:17:23,  6.89s/it]
 29%|██▉       | 66068/225000 [134:48:58<286:59:45,  6.50s/it]
 29%|██▉       | 66069/225000 [134:49:03<265:57:26,  6.02s/it]
 29%|██▉       | 66070/225000 [134:49:09<255:08:49,  5.78s/it]
 29%|██▉       | 66071/225000 [134:49:14<251:17:36,  5.69s/it]
 29%|██▉       | 66072/225000 [134:49:20<257:45:02,  5.84s/it]
 29%|██▉       | 66073/225000 [134:49:26<253:14:04,  5.74s/it]
 29%|██▉       | 66074/225000 [134:49:31<251:04:04,  5.69s/it]
 29%|██▉       | 66075/225000 [134:49:39<284:14:53,  6.44s/it]
 29%|██▉       | 66076/225000 [134:49:54<386:04:00,  8.75s/it]
 29%|██▉       | 66077/225000 [134:49:58<328:17:07,  7.44s/it]
 29%|██▉       | 66078/225000 [134:50:02<287:48:45,  6.52s/it]
 29%|██▉       | 66079/225000 [134:50:10<298:01:46,  6.75s/it]
 29%|██▉       | 66080/225000 [134:50:20<341:59:14,  7.75s/it]
 29%|██▉       | 66081/225000 [134:50:27<332:04:47,  7.52s/it]
 29%|██▉       | 66082/225000 [134:50:31<294:52:54,  6.68s/it]
 29%|██▉       | 66083/225000 [134:50:36<268:55:05,  6.09s/it]
 29%|██▉       | 66084/225000 [134:50:42<260:06:37,  5.89s/it]
 29%|██▉       | 66085/225000 [134:50:47<252:34:41,  5.72s/it]
 29%|██▉       | 66086/225000 [134:50:52<244:00:37,  5.53s/it]
 29%|██▉       | 66087/225000 [134:51:00<270:41:17,  6.13s/it]
 29%|██▉       | 66088/225000 [134:51:12<359:45:40,  8.15s/it]
 29%|██▉       | 66089/225000 [134:51:17<312:10:20,  7.07s/it]
 29%|██▉       | 66090/225000 [134:51:25<323:41:41,  7.33s/it]
 29%|██▉       | 66091/225000 [134:51:30<293:13:51,  6.64s/it]
 29%|██▉       | 66092/225000 [134:51:35<269:40:41,  6.11s/it]
 29%|██▉       | 66093/225000 [134:51:42<278:02:24,  6.30s/it]
 29%|██▉       | 66094/225000 [134:51:53<348:24:59,  7.89s/it]
 29%|██▉       | 66095/225000 [134:51:57<292:34:09,  6.63s/it]
 29%|██▉       | 66096/225000 [134:52:06<324:00:53,  7.34s/it]
 29%|██▉       | 66097/225000 [134:52:10<284:47:31,  6.45s/it]
 29%|██▉       | 66098/225000 [134:52:15<265:23:21,  6.01s/it]
 29%|██▉       | 66099/225000 [134:52:24<307:27:32,  6.97s/it]
 29%|██▉       | 66100/225000 [134:52:30<287:23:48,  6.51s/it]
                                                              
{'loss': '0.7506', 'grad_norm': '0.7631', 'learning_rate': '4.207e-05', 'epoch': '0.2938', 'num_input_tokens_seen': 224146384, 'train_runtime': '4.856e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66100/225000 [134:52:30<287:23:48,  6.51s/it]
 29%|██▉       | 66101/225000 [134:52:35<273:19:49,  6.19s/it]
 29%|██▉       | 66102/225000 [134:52:41<260:33:49,  5.90s/it]
 29%|██▉       | 66103/225000 [134:52:45<245:46:22,  5.57s/it]
 29%|██▉       | 66104/225000 [134:52:53<276:37:13,  6.27s/it]
 29%|██▉       | 66105/225000 [134:53:03<329:09:02,  7.46s/it]
 29%|██▉       | 66106/225000 [134:53:08<285:32:49,  6.47s/it]
 29%|██▉       | 66107/225000 [134:53:17<326:16:00,  7.39s/it]
 29%|██▉       | 66108/225000 [134:53:22<294:41:51,  6.68s/it]
 29%|██▉       | 66109/225000 [134:53:27<273:14:00,  6.19s/it]
 29%|██▉       | 66110/225000 [134:53:32<260:20:09,  5.90s/it]
 29%|██▉       | 66111/225000 [134:53:38<252:35:52,  5.72s/it]
 29%|██▉       | 66112/225000 [134:53:50<345:03:22,  7.82s/it]
 29%|██▉       | 66113/225000 [134:53:57<334:05:30,  7.57s/it]
 29%|██▉       | 66114/225000 [134:54:03<302:36:44,  6.86s/it]
 29%|██▉       | 66115/225000 [134:54:11<322:49:12,  7.31s/it]
 29%|██▉       | 66116/225000 [134:54:16<294:42:35,  6.68s/it]
 29%|██▉       | 66117/225000 [134:54:20<262:56:39,  5.96s/it]
 29%|██▉       | 66118/225000 [134:54:28<281:57:57,  6.39s/it]
 29%|██▉       | 66119/225000 [134:54:37<318:04:39,  7.21s/it]
 29%|██▉       | 66120/225000 [134:54:42<283:44:09,  6.43s/it]
 29%|██▉       | 66121/225000 [134:54:47<264:35:15,  6.00s/it]
 29%|██▉       | 66122/225000 [134:54:56<308:32:00,  6.99s/it]
 29%|██▉       | 66123/225000 [134:55:05<333:15:09,  7.55s/it]
 29%|██▉       | 66124/225000 [134:55:09<287:55:47,  6.52s/it]
 29%|██▉       | 66125/225000 [134:55:13<258:00:23,  5.85s/it]
 29%|██▉       | 66126/225000 [134:55:23<309:03:15,  7.00s/it]
 29%|██▉       | 66127/225000 [134:55:30<305:15:27,  6.92s/it]
 29%|██▉       | 66128/225000 [134:55:35<279:14:10,  6.33s/it]
 29%|██▉       | 66129/225000 [134:55:40<264:10:27,  5.99s/it]
 29%|██▉       | 66130/225000 [134:55:45<254:22:09,  5.76s/it]
 29%|██▉       | 66131/225000 [134:55:50<248:42:08,  5.64s/it]
 29%|██▉       | 66132/225000 [134:55:56<254:59:05,  5.78s/it]
 29%|██▉       | 66133/225000 [134:56:10<358:27:09,  8.12s/it]
 29%|██▉       | 66134/225000 [134:56:15<313:08:23,  7.10s/it]
 29%|██▉       | 66135/225000 [134:56:19<279:34:14,  6.34s/it]
 29%|██▉       | 66136/225000 [134:56:24<262:05:50,  5.94s/it]
 29%|██▉       | 66137/225000 [134:56:31<277:57:47,  6.30s/it]
 29%|██▉       | 66138/225000 [134:56:37<268:16:30,  6.08s/it]
 29%|██▉       | 66139/225000 [134:56:43<263:55:58,  5.98s/it]
 29%|██▉       | 66140/225000 [134:56:54<332:45:26,  7.54s/it]
 29%|██▉       | 66141/225000 [134:57:03<354:06:33,  8.02s/it]
 29%|██▉       | 66142/225000 [134:57:08<318:23:20,  7.22s/it]
 29%|██▉       | 66143/225000 [134:57:13<290:15:34,  6.58s/it]
 29%|██▉       | 66144/225000 [134:57:18<267:12:45,  6.06s/it]
 29%|██▉       | 66145/225000 [134:57:23<253:42:10,  5.75s/it]
 29%|██▉       | 66146/225000 [134:57:29<250:26:18,  5.68s/it]
 29%|██▉       | 66147/225000 [134:57:37<282:22:47,  6.40s/it]
 29%|██▉       | 66148/225000 [134:57:50<364:49:05,  8.27s/it]
 29%|██▉       | 66149/225000 [134:57:54<310:33:57,  7.04s/it]
 29%|██▉       | 66150/225000 [134:58:02<327:38:42,  7.43s/it]
                                                              
{'loss': '0.7331', 'grad_norm': '0.9656', 'learning_rate': '4.206e-05', 'epoch': '0.294', 'num_input_tokens_seen': 224295312, 'train_runtime': '4.859e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66150/225000 [134:58:02<327:38:42,  7.43s/it]
 29%|██▉       | 66151/225000 [134:58:07<300:17:14,  6.81s/it]
 29%|██▉       | 66152/225000 [134:58:15<313:28:44,  7.10s/it]
 29%|██▉       | 66153/225000 [134:58:20<283:26:28,  6.42s/it]
 29%|██▉       | 66154/225000 [134:58:25<266:05:04,  6.03s/it]
 29%|██▉       | 66155/225000 [134:58:30<253:30:50,  5.75s/it]
 29%|██▉       | 66156/225000 [134:58:39<292:54:37,  6.64s/it]
 29%|██▉       | 66157/225000 [134:58:49<344:02:38,  7.80s/it]
 29%|██▉       | 66158/225000 [134:58:54<294:56:53,  6.68s/it]
 29%|██▉       | 66159/225000 [134:59:03<332:07:13,  7.53s/it]
 29%|██▉       | 66160/225000 [134:59:07<290:38:02,  6.59s/it]
 29%|██▉       | 66161/225000 [134:59:17<324:11:21,  7.35s/it]
 29%|██▉       | 66162/225000 [134:59:23<308:21:51,  6.99s/it]
 29%|██▉       | 66163/225000 [134:59:28<286:12:24,  6.49s/it]
 29%|██▉       | 66164/225000 [134:59:34<275:16:43,  6.24s/it]
 29%|██▉       | 66165/225000 [134:59:39<267:09:55,  6.06s/it]
 29%|██▉       | 66166/225000 [134:59:45<261:30:38,  5.93s/it]
 29%|██▉       | 66167/225000 [134:59:50<252:53:10,  5.73s/it]
 29%|██▉       | 66168/225000 [134:59:55<246:29:55,  5.59s/it]
 29%|██▉       | 66169/225000 [135:00:06<315:17:13,  7.15s/it]
 29%|██▉       | 66170/225000 [135:00:15<336:48:05,  7.63s/it]
 29%|██▉       | 66171/225000 [135:00:20<302:54:53,  6.87s/it]
 29%|██▉       | 66172/225000 [135:00:25<281:49:46,  6.39s/it]
 29%|██▉       | 66173/225000 [135:00:30<260:58:24,  5.92s/it]
 29%|██▉       | 66174/225000 [135:00:41<330:19:40,  7.49s/it]
 29%|██▉       | 66175/225000 [135:00:49<329:36:24,  7.47s/it]
 29%|██▉       | 66176/225000 [135:00:53<283:39:01,  6.43s/it]
 29%|██▉       | 66177/225000 [135:01:00<290:11:02,  6.58s/it]
 29%|██▉       | 66178/225000 [135:01:08<309:36:48,  7.02s/it]
 29%|██▉       | 66179/225000 [135:01:12<278:09:56,  6.31s/it]
 29%|██▉       | 66180/225000 [135:01:20<291:29:23,  6.61s/it]
 29%|██▉       | 66181/225000 [135:01:25<278:32:31,  6.31s/it]
 29%|██▉       | 66182/225000 [135:01:35<319:38:01,  7.25s/it]
 29%|██▉       | 66183/225000 [135:01:41<301:11:01,  6.83s/it]
 29%|██▉       | 66184/225000 [135:01:46<279:55:31,  6.35s/it]
 29%|██▉       | 66185/225000 [135:01:52<273:09:43,  6.19s/it]
 29%|██▉       | 66186/225000 [135:02:03<345:07:19,  7.82s/it]
 29%|██▉       | 66187/225000 [135:02:07<290:34:53,  6.59s/it]
 29%|██▉       | 66188/225000 [135:02:11<260:32:04,  5.91s/it]
 29%|██▉       | 66189/225000 [135:02:21<317:02:30,  7.19s/it]
 29%|██▉       | 66190/225000 [135:02:26<277:45:19,  6.30s/it]
 29%|██▉       | 66191/225000 [135:02:35<320:15:31,  7.26s/it]
 29%|██▉       | 66192/225000 [135:02:41<306:31:21,  6.95s/it]
 29%|██▉       | 66193/225000 [135:02:48<303:20:38,  6.88s/it]
 29%|██▉       | 66194/225000 [135:02:57<333:56:44,  7.57s/it]
 29%|██▉       | 66195/225000 [135:03:01<285:11:47,  6.47s/it]
 29%|██▉       | 66196/225000 [135:03:08<294:18:13,  6.67s/it]
 29%|██▉       | 66197/225000 [135:03:15<287:43:22,  6.52s/it]
 29%|██▉       | 66198/225000 [135:03:23<315:25:39,  7.15s/it]
 29%|██▉       | 66199/225000 [135:03:29<296:34:28,  6.72s/it]
 29%|██▉       | 66200/225000 [135:03:34<278:06:04,  6.30s/it]
                                                              
{'loss': '0.7362', 'grad_norm': '0.8686', 'learning_rate': '4.204e-05', 'epoch': '0.2942', 'num_input_tokens_seen': 224443112, 'train_runtime': '4.862e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66200/225000 [135:03:34<278:06:04,  6.30s/it]
 29%|██▉       | 66201/225000 [135:03:40<266:23:37,  6.04s/it]
 29%|██▉       | 66202/225000 [135:03:45<261:00:45,  5.92s/it]
 29%|██▉       | 66203/225000 [135:03:50<251:29:53,  5.70s/it]
 29%|██▉       | 66204/225000 [135:03:56<250:08:08,  5.67s/it]
 29%|██▉       | 66205/225000 [135:04:06<309:58:34,  7.03s/it]
 29%|██▉       | 66206/225000 [135:04:16<345:30:17,  7.83s/it]
 29%|██▉       | 66207/225000 [135:04:20<299:03:27,  6.78s/it]
 29%|██▉       | 66208/225000 [135:04:26<291:14:02,  6.60s/it]
 29%|██▉       | 66209/225000 [135:04:39<366:15:03,  8.30s/it]
 29%|██▉       | 66210/225000 [135:04:43<307:54:35,  6.98s/it]
 29%|██▉       | 66211/225000 [135:04:49<292:57:22,  6.64s/it]
 29%|██▉       | 66212/225000 [135:04:54<276:46:11,  6.27s/it]
 29%|██▉       | 66213/225000 [135:05:00<268:06:29,  6.08s/it]
 29%|██▉       | 66214/225000 [135:05:05<256:50:25,  5.82s/it]
 29%|██▉       | 66215/225000 [135:05:11<255:45:23,  5.80s/it]
 29%|██▉       | 66216/225000 [135:05:16<256:37:42,  5.82s/it]
 29%|██▉       | 66217/225000 [135:05:22<258:02:24,  5.85s/it]
 29%|██▉       | 66218/225000 [135:05:34<330:04:40,  7.48s/it]
 29%|██▉       | 66219/225000 [135:05:42<343:51:43,  7.80s/it]
 29%|██▉       | 66220/225000 [135:05:48<316:08:12,  7.17s/it]
 29%|██▉       | 66221/225000 [135:05:54<296:41:32,  6.73s/it]
 29%|██▉       | 66222/225000 [135:06:04<348:23:35,  7.90s/it]
 29%|██▉       | 66223/225000 [135:06:09<301:29:40,  6.84s/it]
 29%|██▉       | 66224/225000 [135:06:17<318:00:25,  7.21s/it]
 29%|██▉       | 66225/225000 [135:06:21<279:08:41,  6.33s/it]
 29%|██▉       | 66226/225000 [135:06:28<295:40:09,  6.70s/it]
 29%|██▉       | 66227/225000 [135:06:38<330:11:29,  7.49s/it]
 29%|██▉       | 66228/225000 [135:06:45<326:20:39,  7.40s/it]
 29%|██▉       | 66229/225000 [135:06:50<291:18:00,  6.60s/it]
 29%|██▉       | 66230/225000 [135:06:59<324:40:27,  7.36s/it]
 29%|██▉       | 66231/225000 [135:07:03<278:13:50,  6.31s/it]
 29%|██▉       | 66232/225000 [135:07:10<289:43:58,  6.57s/it]
 29%|██▉       | 66233/225000 [135:07:17<292:54:55,  6.64s/it]
 29%|██▉       | 66234/225000 [135:07:24<308:25:33,  6.99s/it]
 29%|██▉       | 66235/225000 [135:07:30<283:44:50,  6.43s/it]
 29%|██▉       | 66236/225000 [135:07:34<257:26:46,  5.84s/it]
 29%|██▉       | 66237/225000 [135:07:43<303:09:56,  6.87s/it]
 29%|██▉       | 66238/225000 [135:07:51<312:03:48,  7.08s/it]
 29%|██▉       | 66239/225000 [135:07:55<273:23:38,  6.20s/it]
 29%|██▉       | 66240/225000 [135:08:02<284:06:17,  6.44s/it]
 29%|██▉       | 66241/225000 [135:08:11<318:12:56,  7.22s/it]
 29%|██▉       | 66242/225000 [135:08:17<298:23:46,  6.77s/it]
 29%|██▉       | 66243/225000 [135:08:25<311:27:11,  7.06s/it]
 29%|██▉       | 66244/225000 [135:08:30<283:41:41,  6.43s/it]
 29%|██▉       | 66245/225000 [135:08:35<266:38:31,  6.05s/it]
 29%|██▉       | 66246/225000 [135:08:40<254:59:55,  5.78s/it]
 29%|██▉       | 66247/225000 [135:08:44<239:19:14,  5.43s/it]
 29%|██▉       | 66248/225000 [135:08:49<231:37:13,  5.25s/it]
 29%|██▉       | 66249/225000 [135:09:03<337:24:49,  7.65s/it]
 29%|██▉       | 66250/225000 [135:09:10<338:55:33,  7.69s/it]
                                                              
{'loss': '0.7348', 'grad_norm': '0.8035', 'learning_rate': '4.203e-05', 'epoch': '0.2944', 'num_input_tokens_seen': 224591568, 'train_runtime': '4.866e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66250/225000 [135:09:10<338:55:33,  7.69s/it]
 29%|██▉       | 66251/225000 [135:09:14<292:04:58,  6.62s/it]
 29%|██▉       | 66252/225000 [135:09:20<275:33:01,  6.25s/it]
 29%|██▉       | 66253/225000 [135:09:31<345:29:09,  7.83s/it]
 29%|██▉       | 66254/225000 [135:09:35<292:57:48,  6.64s/it]
 29%|██▉       | 66255/225000 [135:09:42<296:00:50,  6.71s/it]
 29%|██▉       | 66256/225000 [135:09:46<260:04:50,  5.90s/it]
 29%|██▉       | 66257/225000 [135:09:56<316:09:48,  7.17s/it]
 29%|██▉       | 66258/225000 [135:10:02<298:38:12,  6.77s/it]
 29%|██▉       | 66259/225000 [135:10:12<334:03:12,  7.58s/it]
 29%|██▉       | 66260/225000 [135:10:18<320:08:24,  7.26s/it]
 29%|██▉       | 66261/225000 [135:10:23<286:46:57,  6.50s/it]
 29%|██▉       | 66262/225000 [135:10:27<262:22:45,  5.95s/it]
 29%|██▉       | 66263/225000 [135:10:32<240:36:04,  5.46s/it]
 29%|██▉       | 66264/225000 [135:10:37<236:19:33,  5.36s/it]
 29%|██▉       | 66265/225000 [135:10:50<341:57:39,  7.76s/it]
 29%|██▉       | 66266/225000 [135:10:54<293:44:39,  6.66s/it]
 29%|██▉       | 66267/225000 [135:11:03<318:11:36,  7.22s/it]
 29%|██▉       | 66268/225000 [135:11:07<283:47:46,  6.44s/it]
 29%|██▉       | 66269/225000 [135:11:13<265:26:30,  6.02s/it]
 29%|██▉       | 66270/225000 [135:11:18<259:48:01,  5.89s/it]
 29%|██▉       | 66271/225000 [135:11:23<248:17:44,  5.63s/it]
 29%|██▉       | 66272/225000 [135:11:33<306:33:52,  6.95s/it]
 29%|██▉       | 66273/225000 [135:11:43<351:11:00,  7.96s/it]
 29%|██▉       | 66274/225000 [135:11:48<311:24:28,  7.06s/it]
 29%|██▉       | 66275/225000 [135:11:53<279:37:58,  6.34s/it]
 29%|██▉       | 66276/225000 [135:11:58<264:33:19,  6.00s/it]
 29%|██▉       | 66277/225000 [135:12:06<282:14:09,  6.40s/it]
 29%|██▉       | 66278/225000 [135:12:18<365:33:43,  8.29s/it]
 29%|██▉       | 66279/225000 [135:12:24<325:58:21,  7.39s/it]
 29%|██▉       | 66280/225000 [135:12:28<284:59:27,  6.46s/it]
 29%|██▉       | 66281/225000 [135:12:33<270:04:07,  6.13s/it]
 29%|██▉       | 66282/225000 [135:12:45<338:23:45,  7.68s/it]
 29%|██▉       | 66283/225000 [135:12:53<344:52:14,  7.82s/it]
 29%|██▉       | 66284/225000 [135:13:01<346:06:39,  7.85s/it]
 29%|██▉       | 66285/225000 [135:13:05<303:45:54,  6.89s/it]
 29%|██▉       | 66286/225000 [135:13:13<309:55:34,  7.03s/it]
 29%|██▉       | 66287/225000 [135:13:18<284:34:12,  6.45s/it]
 29%|██▉       | 66288/225000 [135:13:23<271:52:46,  6.17s/it]
 29%|██▉       | 66289/225000 [135:13:29<266:09:15,  6.04s/it]
 29%|██▉       | 66290/225000 [135:13:35<259:31:02,  5.89s/it]
 29%|██▉       | 66291/225000 [135:13:40<257:57:53,  5.85s/it]
 29%|██▉       | 66292/225000 [135:13:53<352:43:36,  8.00s/it]
 29%|██▉       | 66293/225000 [135:13:59<327:10:01,  7.42s/it]
 29%|██▉       | 66294/225000 [135:14:04<286:30:56,  6.50s/it]
 29%|██▉       | 66295/225000 [135:14:13<328:56:27,  7.46s/it]
 29%|██▉       | 66296/225000 [135:14:20<312:01:17,  7.08s/it]
 29%|██▉       | 66297/225000 [135:14:24<282:29:39,  6.41s/it]
 29%|██▉       | 66298/225000 [135:14:29<258:19:57,  5.86s/it]
 29%|██▉       | 66299/225000 [135:14:39<311:19:38,  7.06s/it]
 29%|██▉       | 66300/225000 [135:14:46<316:18:31,  7.18s/it]
                                                              
{'loss': '0.7318', 'grad_norm': '0.8417', 'learning_rate': '4.202e-05', 'epoch': '0.2947', 'num_input_tokens_seen': 224741120, 'train_runtime': '4.869e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66300/225000 [135:14:46<316:18:31,  7.18s/it]
 29%|██▉       | 66301/225000 [135:14:56<342:21:30,  7.77s/it]
 29%|██▉       | 66302/225000 [135:15:01<309:28:06,  7.02s/it]
 29%|██▉       | 66303/225000 [135:15:07<296:14:07,  6.72s/it]
 29%|██▉       | 66304/225000 [135:15:14<304:12:07,  6.90s/it]
 29%|██▉       | 66305/225000 [135:15:21<305:44:40,  6.94s/it]
 29%|██▉       | 66306/225000 [135:15:28<306:45:17,  6.96s/it]
 29%|██▉       | 66307/225000 [135:15:32<268:52:46,  6.10s/it]
 29%|██▉       | 66308/225000 [135:15:37<254:49:50,  5.78s/it]
 29%|██▉       | 66309/225000 [135:15:43<249:57:53,  5.67s/it]
 29%|██▉       | 66310/225000 [135:15:47<237:49:36,  5.40s/it]
 29%|██▉       | 66311/225000 [135:16:00<334:00:39,  7.58s/it]
 29%|██▉       | 66312/225000 [135:16:07<326:34:39,  7.41s/it]
 29%|██▉       | 66313/225000 [135:16:11<285:02:24,  6.47s/it]
 29%|██▉       | 66314/225000 [135:16:19<296:03:02,  6.72s/it]
 29%|██▉       | 66315/225000 [135:16:27<317:12:33,  7.20s/it]
 29%|██▉       | 66316/225000 [135:16:31<279:18:51,  6.34s/it]
 29%|██▉       | 66317/225000 [135:16:41<322:59:05,  7.33s/it]
 29%|██▉       | 66318/225000 [135:16:45<281:00:40,  6.38s/it]
 29%|██▉       | 66319/225000 [135:16:53<305:57:33,  6.94s/it]
 29%|██▉       | 66320/225000 [135:16:58<281:06:31,  6.38s/it]
 29%|██▉       | 66321/225000 [135:17:07<308:45:44,  7.00s/it]
 29%|██▉       | 66322/225000 [135:17:12<280:28:13,  6.36s/it]
 29%|██▉       | 66323/225000 [135:17:20<300:33:31,  6.82s/it]
 29%|██▉       | 66324/225000 [135:17:24<265:11:00,  6.02s/it]
 29%|██▉       | 66325/225000 [135:17:34<324:36:25,  7.36s/it]
 29%|██▉       | 66326/225000 [135:17:38<277:32:12,  6.30s/it]
 29%|██▉       | 66327/225000 [135:17:46<302:37:55,  6.87s/it]
 29%|██▉       | 66328/225000 [135:17:52<280:32:57,  6.37s/it]
 29%|██▉       | 66329/225000 [135:18:00<306:19:27,  6.95s/it]
 29%|██▉       | 66330/225000 [135:18:06<297:34:07,  6.75s/it]
 29%|██▉       | 66331/225000 [135:18:11<271:02:45,  6.15s/it]
 29%|██▉       | 66332/225000 [135:18:16<252:06:08,  5.72s/it]
 29%|██▉       | 66333/225000 [135:18:23<278:16:58,  6.31s/it]
 29%|██▉       | 66334/225000 [135:18:33<329:03:45,  7.47s/it]
 29%|██▉       | 66335/225000 [135:18:38<291:46:21,  6.62s/it]
 29%|██▉       | 66336/225000 [135:18:47<320:38:02,  7.28s/it]
 29%|██▉       | 66337/225000 [135:18:53<307:59:05,  6.99s/it]
 29%|██▉       | 66338/225000 [135:18:57<267:38:11,  6.07s/it]
 29%|██▉       | 66339/225000 [135:19:02<253:40:05,  5.76s/it]
 29%|██▉       | 66340/225000 [135:19:15<343:47:32,  7.80s/it]
 29%|██▉       | 66341/225000 [135:19:20<303:42:04,  6.89s/it]
 29%|██▉       | 66342/225000 [135:19:26<299:30:25,  6.80s/it]
 29%|██▉       | 66343/225000 [135:19:34<313:33:50,  7.11s/it]
 29%|██▉       | 66344/225000 [135:19:38<266:52:09,  6.06s/it]
 29%|██▉       | 66345/225000 [135:19:48<321:46:40,  7.30s/it]
 29%|██▉       | 66346/225000 [135:19:53<288:26:02,  6.54s/it]
 29%|██▉       | 66347/225000 [135:20:01<319:30:07,  7.25s/it]
 29%|██▉       | 66348/225000 [135:20:05<274:41:55,  6.23s/it]
 29%|██▉       | 66349/225000 [135:20:15<325:56:30,  7.40s/it]
 29%|██▉       | 66350/225000 [135:20:24<336:54:56,  7.65s/it]
                                                              
{'loss': '0.7283', 'grad_norm': '0.8903', 'learning_rate': '4.2e-05', 'epoch': '0.2949', 'num_input_tokens_seen': 224891216, 'train_runtime': '4.872e+05', 'train_tokens_per_second': '461.6'}

 29%|██▉       | 66350/225000 [135:20:24<336:54:56,  7.65s/it]
 29%|██▉       | 66351/225000 [135:20:28<293:28:35,  6.66s/it]
 29%|██▉       | 66352/225000 [135:20:35<303:55:50,  6.90s/it]
 29%|██▉       | 66353/225000 [135:20:40<268:49:34,  6.10s/it]
 29%|██▉       | 66354/225000 [135:20:44<249:18:29,  5.66s/it]
 29%|██▉       | 66355/225000 [135:20:49<241:14:10,  5.47s/it]
 29%|██▉       | 66356/225000 [135:20:56<256:53:03,  5.83s/it]
 29%|██▉       | 66357/225000 [135:21:08<344:38:30,  7.82s/it]
 29%|██▉       | 66358/225000 [135:21:13<301:36:36,  6.84s/it]
 29%|██▉       | 66359/225000 [135:21:22<332:24:23,  7.54s/it]
 29%|██▉       | 66360/225000 [135:21:28<311:22:38,  7.07s/it]
 29%|██▉       | 66361/225000 [135:21:34<296:17:06,  6.72s/it]
 29%|██▉       | 66362/225000 [135:21:39<269:44:22,  6.12s/it]
 29%|██▉       | 66363/225000 [135:21:44<257:40:18,  5.85s/it]
 29%|██▉       | 66364/225000 [135:21:50<262:55:09,  5.97s/it]
 29%|██▉       | 66365/225000 [135:22:02<344:22:34,  7.82s/it]
 29%|██▉       | 66366/225000 [135:22:08<314:53:04,  7.15s/it]
 29%|██▉       | 66367/225000 [135:22:13<284:18:44,  6.45s/it]
 29%|██▉       | 66368/225000 [135:22:21<306:19:33,  6.95s/it]
 29%|██▉       | 66369/225000 [135:22:26<288:02:50,  6.54s/it]
 29%|██▉       | 66370/225000 [135:22:33<292:54:23,  6.65s/it]
 29%|██▉       | 66371/225000 [135:22:38<259:40:26,  5.89s/it]
 29%|██▉       | 66372/225000 [135:22:47<312:26:17,  7.09s/it]
 29%|██▉       | 66373/225000 [135:22:54<312:05:48,  7.08s/it]
 29%|██▉       | 66374/225000 [135:23:00<285:33:08,  6.48s/it]
 30%|██▉       | 66375/225000 [135:23:05<275:02:09,  6.24s/it]
 30%|██▉       | 66376/225000 [135:23:13<289:51:52,  6.58s/it]
 30%|██▉       | 66377/225000 [135:23:18<274:46:20,  6.24s/it]
 30%|██▉       | 66378/225000 [135:23:24<264:34:05,  6.00s/it]
 30%|██▉       | 66379/225000 [135:23:30<274:42:57,  6.23s/it]
 30%|██▉       | 66380/225000 [135:23:38<291:45:34,  6.62s/it]
 30%|██▉       | 66381/225000 [135:23:44<285:00:19,  6.47s/it]
 30%|██▉       | 66382/225000 [135:23:52<308:43:00,  7.01s/it]
 30%|██▉       | 66383/225000 [135:24:03<354:00:25,  8.03s/it]
 30%|██▉       | 66384/225000 [135:24:10<344:22:10,  7.82s/it]
 30%|██▉       | 66385/225000 [135:24:14<293:20:54,  6.66s/it]
 30%|██▉       | 66386/225000 [135:24:20<284:32:33,  6.46s/it]
 30%|██▉       | 66387/225000 [135:24:29<318:39:35,  7.23s/it]
 30%|██▉       | 66388/225000 [135:24:34<287:15:53,  6.52s/it]
 30%|██▉       | 66389/225000 [135:24:38<255:53:51,  5.81s/it]
 30%|██▉       | 66390/225000 [135:24:43<244:49:44,  5.56s/it]
 30%|██▉       | 66391/225000 [135:24:49<258:20:56,  5.86s/it]
 30%|██▉       | 66392/225000 [135:25:02<353:00:12,  8.01s/it]
 30%|██▉       | 66393/225000 [135:25:08<325:14:24,  7.38s/it]
 30%|██▉       | 66394/225000 [135:25:14<308:10:39,  6.99s/it]
 30%|██▉       | 66395/225000 [135:25:19<279:29:09,  6.34s/it]
 30%|██▉       | 66396/225000 [135:25:27<303:43:59,  6.89s/it]
 30%|██▉       | 66397/225000 [135:25:32<274:49:01,  6.24s/it]
 30%|██▉       | 66398/225000 [135:25:37<254:11:42,  5.77s/it]
 30%|██▉       | 66399/225000 [135:25:49<336:17:57,  7.63s/it]
 30%|██▉       | 66400/225000 [135:25:53<289:23:56,  6.57s/it]
                                                              
{'loss': '0.7453', 'grad_norm': '0.8089', 'learning_rate': '4.199e-05', 'epoch': '0.2951', 'num_input_tokens_seen': 225040104, 'train_runtime': '4.876e+05', 'train_tokens_per_second': '461.6'}

 30%|██▉       | 66400/225000 [135:25:53<289:23:56,  6.57s/it]
 30%|██▉       | 66401/225000 [135:25:57<256:16:27,  5.82s/it]
 30%|██▉       | 66402/225000 [135:26:05<285:26:27,  6.48s/it]
 30%|██▉       | 66403/225000 [135:26:15<330:09:37,  7.49s/it]
 30%|██▉       | 66404/225000 [135:26:22<324:20:28,  7.36s/it]
 30%|██▉       | 66405/225000 [135:26:27<289:56:53,  6.58s/it]
 30%|██▉       | 66406/225000 [135:26:35<317:47:50,  7.21s/it]
 30%|██▉       | 66407/225000 [135:26:40<284:57:50,  6.47s/it]
 30%|██▉       | 66408/225000 [135:26:49<315:01:18,  7.15s/it]
 30%|██▉       | 66409/225000 [135:26:55<301:50:54,  6.85s/it]
 30%|██▉       | 66410/225000 [135:27:02<298:08:46,  6.77s/it]
 30%|██▉       | 66411/225000 [135:27:11<336:52:10,  7.65s/it]
 30%|██▉       | 66412/225000 [135:27:16<294:41:26,  6.69s/it]
 30%|██▉       | 66413/225000 [135:27:21<271:45:57,  6.17s/it]
 30%|██▉       | 66414/225000 [135:27:26<258:16:15,  5.86s/it]
 30%|██▉       | 66415/225000 [135:27:39<351:24:01,  7.98s/it]
 30%|██▉       | 66416/225000 [135:27:43<300:22:15,  6.82s/it]
 30%|██▉       | 66417/225000 [135:27:50<300:35:14,  6.82s/it]
 30%|██▉       | 66418/225000 [135:27:55<286:49:39,  6.51s/it]
 30%|██▉       | 66419/225000 [135:28:03<303:30:59,  6.89s/it]
 30%|██▉       | 66420/225000 [135:28:08<272:33:10,  6.19s/it]
 30%|██▉       | 66421/225000 [135:28:13<265:48:55,  6.03s/it]
 30%|██▉       | 66422/225000 [135:28:19<257:17:04,  5.84s/it]
 30%|██▉       | 66423/225000 [135:28:25<256:02:42,  5.81s/it]
 30%|██▉       | 66424/225000 [135:28:31<261:11:20,  5.93s/it]
 30%|██▉       | 66425/225000 [135:28:37<258:21:13,  5.87s/it]
 30%|██▉       | 66426/225000 [135:28:48<331:24:18,  7.52s/it]
 30%|██▉       | 66427/225000 [135:28:59<375:14:42,  8.52s/it]
 30%|██▉       | 66428/225000 [135:29:02<307:23:48,  6.98s/it]
 30%|██▉       | 66429/225000 [135:29:11<334:01:26,  7.58s/it]
 30%|██▉       | 66430/225000 [135:29:17<308:44:21,  7.01s/it]
 30%|██▉       | 66431/225000 [135:29:25<322:59:29,  7.33s/it]
 30%|██▉       | 66432/225000 [135:29:30<297:35:21,  6.76s/it]
 30%|██▉       | 66433/225000 [135:29:40<342:00:00,  7.76s/it]
 30%|██▉       | 66434/225000 [135:29:44<288:09:51,  6.54s/it]
 30%|██▉       | 66435/225000 [135:29:49<263:29:09,  5.98s/it]
 30%|██▉       | 66436/225000 [135:29:54<250:46:55,  5.69s/it]
 30%|██▉       | 66437/225000 [135:29:59<245:39:21,  5.58s/it]
 30%|██▉       | 66438/225000 [135:30:12<345:42:39,  7.85s/it]
 30%|██▉       | 66439/225000 [135:30:17<301:33:15,  6.85s/it]
 30%|██▉       | 66440/225000 [135:30:21<270:19:50,  6.14s/it]
 30%|██▉       | 66441/225000 [135:30:32<327:44:55,  7.44s/it]
 30%|██▉       | 66442/225000 [135:30:38<312:09:39,  7.09s/it]
 30%|██▉       | 66443/225000 [135:30:46<328:32:01,  7.46s/it]
 30%|██▉       | 66444/225000 [135:30:51<293:46:51,  6.67s/it]
 30%|██▉       | 66445/225000 [135:30:56<273:12:58,  6.20s/it]
 30%|██▉       | 66446/225000 [135:31:02<268:43:27,  6.10s/it]
 30%|██▉       | 66447/225000 [135:31:08<264:43:45,  6.01s/it]
 30%|██▉       | 66448/225000 [135:31:14<266:04:23,  6.04s/it]
 30%|██▉       | 66449/225000 [135:31:26<339:04:48,  7.70s/it]
 30%|██▉       | 66450/225000 [135:31:29<286:53:16,  6.51s/it]
                                                              
{'loss': '0.7331', 'grad_norm': '0.8046', 'learning_rate': '4.198e-05', 'epoch': '0.2953', 'num_input_tokens_seen': 225189296, 'train_runtime': '4.879e+05', 'train_tokens_per_second': '461.6'}

 30%|██▉       | 66450/225000 [135:31:29<286:53:16,  6.51s/it]
 30%|██▉       | 66451/225000 [135:31:40<341:56:30,  7.76s/it]
 30%|██▉       | 66452/225000 [135:31:45<307:15:23,  6.98s/it]
 30%|██▉       | 66453/225000 [135:31:50<281:24:51,  6.39s/it]
 30%|██▉       | 66454/225000 [135:31:55<261:18:16,  5.93s/it]
 30%|██▉       | 66455/225000 [135:32:00<253:14:06,  5.75s/it]
 30%|██▉       | 66456/225000 [135:32:10<297:39:08,  6.76s/it]
 30%|██▉       | 66457/225000 [135:32:19<331:25:00,  7.53s/it]
 30%|██▉       | 66458/225000 [135:32:27<335:47:18,  7.62s/it]
 30%|██▉       | 66459/225000 [135:32:31<290:18:07,  6.59s/it]
 30%|██▉       | 66460/225000 [135:32:40<318:18:03,  7.23s/it]
 30%|██▉       | 66461/225000 [135:32:47<318:21:17,  7.23s/it]
 30%|██▉       | 66462/225000 [135:32:51<282:07:25,  6.41s/it]
 30%|██▉       | 66463/225000 [135:33:02<337:34:19,  7.67s/it]
 30%|██▉       | 66464/225000 [135:33:06<291:42:42,  6.62s/it]
 30%|██▉       | 66465/225000 [135:33:14<302:37:37,  6.87s/it]
 30%|██▉       | 66466/225000 [135:33:19<289:22:27,  6.57s/it]
 30%|██▉       | 66467/225000 [135:33:27<303:45:22,  6.90s/it]
 30%|██▉       | 66468/225000 [135:33:31<265:33:24,  6.03s/it]
 30%|██▉       | 66469/225000 [135:33:41<315:18:40,  7.16s/it]
 30%|██▉       | 66470/225000 [135:33:45<272:51:45,  6.20s/it]
 30%|██▉       | 66471/225000 [135:33:53<299:41:19,  6.81s/it]
 30%|██▉       | 66472/225000 [135:33:57<268:21:25,  6.09s/it]
 30%|██▉       | 66473/225000 [135:34:03<255:16:22,  5.80s/it]
 30%|██▉       | 66474/225000 [135:34:08<248:23:21,  5.64s/it]
 30%|██▉       | 66475/225000 [135:34:13<242:05:31,  5.50s/it]
 30%|██▉       | 66476/225000 [135:34:19<243:48:17,  5.54s/it]
 30%|██▉       | 66477/225000 [135:34:24<241:58:14,  5.50s/it]
 30%|██▉       | 66478/225000 [135:34:30<244:06:00,  5.54s/it]
 30%|██▉       | 66479/225000 [135:34:42<327:01:49,  7.43s/it]
 30%|██▉       | 66480/225000 [135:34:53<382:44:29,  8.69s/it]
 30%|██▉       | 66481/225000 [135:34:59<340:37:13,  7.74s/it]
 30%|██▉       | 66482/225000 [135:35:04<313:53:58,  7.13s/it]
 30%|██▉       | 66483/225000 [135:35:13<332:38:46,  7.55s/it]
 30%|██▉       | 66484/225000 [135:35:17<285:22:12,  6.48s/it]
 30%|██▉       | 66485/225000 [135:35:22<269:54:10,  6.13s/it]
 30%|██▉       | 66486/225000 [135:35:27<251:55:56,  5.72s/it]
 30%|██▉       | 66487/225000 [135:35:32<244:39:00,  5.56s/it]
 30%|██▉       | 66488/225000 [135:35:40<268:18:57,  6.09s/it]
 30%|██▉       | 66489/225000 [135:35:51<339:13:25,  7.70s/it]
 30%|██▉       | 66490/225000 [135:35:55<285:54:48,  6.49s/it]
 30%|██▉       | 66491/225000 [135:36:03<316:49:38,  7.20s/it]
 30%|██▉       | 66492/225000 [135:36:08<274:49:56,  6.24s/it]
 30%|██▉       | 66493/225000 [135:36:17<322:08:23,  7.32s/it]
 30%|██▉       | 66494/225000 [135:36:22<283:26:41,  6.44s/it]
 30%|██▉       | 66495/225000 [135:36:27<262:43:37,  5.97s/it]
 30%|██▉       | 66496/225000 [135:36:37<316:16:16,  7.18s/it]
 30%|██▉       | 66497/225000 [135:36:41<280:22:41,  6.37s/it]
 30%|██▉       | 66498/225000 [135:36:46<256:55:14,  5.84s/it]
 30%|██▉       | 66499/225000 [135:36:51<250:37:35,  5.69s/it]
 30%|██▉       | 66500/225000 [135:37:05<354:48:30,  8.06s/it]
                                                              
{'loss': '0.7232', 'grad_norm': '0.663', 'learning_rate': '4.196e-05', 'epoch': '0.2956', 'num_input_tokens_seen': 225338704, 'train_runtime': '4.882e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66500/225000 [135:37:05<354:48:30,  8.06s/it]
 30%|██▉       | 66501/225000 [135:37:10<326:07:28,  7.41s/it]
 30%|██▉       | 66502/225000 [135:37:15<287:54:43,  6.54s/it]
 30%|██▉       | 66503/225000 [135:37:20<264:28:25,  6.01s/it]
 30%|██▉       | 66504/225000 [135:37:24<246:39:09,  5.60s/it]
 30%|██▉       | 66505/225000 [135:37:33<284:28:25,  6.46s/it]
 30%|██▉       | 66506/225000 [135:37:43<335:12:55,  7.61s/it]
 30%|██▉       | 66507/225000 [135:37:49<305:03:36,  6.93s/it]
 30%|██▉       | 66508/225000 [135:37:54<280:18:42,  6.37s/it]
 30%|██▉       | 66509/225000 [135:37:59<266:54:44,  6.06s/it]
 30%|██▉       | 66510/225000 [135:38:05<260:42:13,  5.92s/it]
 30%|██▉       | 66511/225000 [135:38:15<321:43:46,  7.31s/it]
 30%|██▉       | 66512/225000 [135:38:21<308:20:31,  7.00s/it]
 30%|██▉       | 66513/225000 [135:38:29<321:56:16,  7.31s/it]
 30%|██▉       | 66514/225000 [135:38:35<292:48:46,  6.65s/it]
 30%|██▉       | 66515/225000 [135:38:40<278:58:21,  6.34s/it]
 30%|██▉       | 66516/225000 [135:38:45<256:54:49,  5.84s/it]
 30%|██▉       | 66517/225000 [135:38:56<334:19:58,  7.59s/it]
 30%|██▉       | 66518/225000 [135:39:01<288:17:17,  6.55s/it]
 30%|██▉       | 66519/225000 [135:39:11<342:01:54,  7.77s/it]
 30%|██▉       | 66520/225000 [135:39:16<296:31:31,  6.74s/it]
 30%|██▉       | 66521/225000 [135:39:20<266:07:58,  6.05s/it]
 30%|██▉       | 66522/225000 [135:39:31<334:19:37,  7.59s/it]
 30%|██▉       | 66523/225000 [135:39:37<310:37:50,  7.06s/it]
 30%|██▉       | 66524/225000 [135:39:42<278:34:15,  6.33s/it]
 30%|██▉       | 66525/225000 [135:39:49<294:33:25,  6.69s/it]
 30%|██▉       | 66526/225000 [135:39:53<262:30:06,  5.96s/it]
 30%|██▉       | 66527/225000 [135:40:04<321:10:11,  7.30s/it]
 30%|██▉       | 66528/225000 [135:40:09<293:27:01,  6.67s/it]
 30%|██▉       | 66529/225000 [135:40:14<270:46:44,  6.15s/it]
 30%|██▉       | 66530/225000 [135:40:18<248:27:34,  5.64s/it]
 30%|██▉       | 66531/225000 [135:40:23<238:42:29,  5.42s/it]
 30%|██▉       | 66532/225000 [135:40:33<300:22:09,  6.82s/it]
 30%|██▉       | 66533/225000 [135:40:42<318:42:38,  7.24s/it]
 30%|██▉       | 66534/225000 [135:40:46<280:42:03,  6.38s/it]
 30%|██▉       | 66535/225000 [135:40:51<258:28:00,  5.87s/it]
 30%|██▉       | 66536/225000 [135:40:55<237:50:22,  5.40s/it]
 30%|██▉       | 66537/225000 [135:41:04<282:56:57,  6.43s/it]
 30%|██▉       | 66538/225000 [135:41:13<322:03:02,  7.32s/it]
 30%|██▉       | 66539/225000 [135:41:21<331:00:36,  7.52s/it]
 30%|██▉       | 66540/225000 [135:41:25<282:16:33,  6.41s/it]
 30%|██▉       | 66541/225000 [135:41:34<321:41:37,  7.31s/it]
 30%|██▉       | 66542/225000 [135:41:39<289:42:06,  6.58s/it]
 30%|██▉       | 66543/225000 [135:41:44<261:17:24,  5.94s/it]
 30%|██▉       | 66544/225000 [135:41:49<257:48:35,  5.86s/it]
 30%|██▉       | 66545/225000 [135:41:54<247:16:36,  5.62s/it]
 30%|██▉       | 66546/225000 [135:42:03<292:13:39,  6.64s/it]
 30%|██▉       | 66547/225000 [135:42:15<355:32:36,  8.08s/it]
 30%|██▉       | 66548/225000 [135:42:18<294:57:28,  6.70s/it]
 30%|██▉       | 66549/225000 [135:42:27<313:36:37,  7.13s/it]
 30%|██▉       | 66550/225000 [135:42:31<274:24:15,  6.23s/it]
                                                              
{'loss': '0.7227', 'grad_norm': '0.8168', 'learning_rate': '4.195e-05', 'epoch': '0.2958', 'num_input_tokens_seen': 225487560, 'train_runtime': '4.886e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66550/225000 [135:42:31<274:24:15,  6.23s/it]
 30%|██▉       | 66551/225000 [135:42:36<258:21:31,  5.87s/it]
 30%|██▉       | 66552/225000 [135:42:40<241:09:42,  5.48s/it]
 30%|██▉       | 66553/225000 [135:42:46<238:18:26,  5.41s/it]
 30%|██▉       | 66554/225000 [135:42:56<298:51:50,  6.79s/it]
 30%|██▉       | 66555/225000 [135:43:06<343:18:25,  7.80s/it]
 30%|██▉       | 66556/225000 [135:43:10<300:27:08,  6.83s/it]
 30%|██▉       | 66557/225000 [135:43:15<272:39:21,  6.20s/it]
 30%|██▉       | 66558/225000 [135:43:20<255:44:31,  5.81s/it]
 30%|██▉       | 66559/225000 [135:43:25<245:02:06,  5.57s/it]
 30%|██▉       | 66560/225000 [135:43:37<336:49:20,  7.65s/it]
 30%|██▉       | 66561/225000 [135:43:44<318:43:22,  7.24s/it]
 30%|██▉       | 66562/225000 [135:43:49<286:39:44,  6.51s/it]
 30%|██▉       | 66563/225000 [135:43:53<265:11:31,  6.03s/it]
 30%|██▉       | 66564/225000 [135:44:05<333:51:36,  7.59s/it]
 30%|██▉       | 66565/225000 [135:44:12<325:07:44,  7.39s/it]
 30%|██▉       | 66566/225000 [135:44:16<287:30:53,  6.53s/it]
 30%|██▉       | 66567/225000 [135:44:21<260:17:15,  5.91s/it]
 30%|██▉       | 66568/225000 [135:44:28<277:40:29,  6.31s/it]
 30%|██▉       | 66569/225000 [135:44:37<313:03:08,  7.11s/it]
 30%|██▉       | 66570/225000 [135:44:42<283:09:13,  6.43s/it]
 30%|██▉       | 66571/225000 [135:44:47<263:43:10,  5.99s/it]
 30%|██▉       | 66572/225000 [135:44:52<252:38:34,  5.74s/it]
 30%|██▉       | 66573/225000 [135:44:57<249:12:31,  5.66s/it]
 30%|██▉       | 66574/225000 [135:45:10<341:09:53,  7.75s/it]
 30%|██▉       | 66575/225000 [135:45:15<308:34:53,  7.01s/it]
 30%|██▉       | 66576/225000 [135:45:20<285:12:26,  6.48s/it]
 30%|██▉       | 66577/225000 [135:45:29<312:58:50,  7.11s/it]
 30%|██▉       | 66578/225000 [135:45:37<322:34:42,  7.33s/it]
 30%|██▉       | 66579/225000 [135:45:44<320:55:57,  7.29s/it]
 30%|██▉       | 66580/225000 [135:45:49<288:13:07,  6.55s/it]
 30%|██▉       | 66581/225000 [135:45:56<294:29:57,  6.69s/it]
 30%|██▉       | 66582/225000 [135:46:03<301:10:11,  6.84s/it]
 30%|██▉       | 66583/225000 [135:46:08<275:59:07,  6.27s/it]
 30%|██▉       | 66584/225000 [135:46:13<257:45:42,  5.86s/it]
 30%|██▉       | 66585/225000 [135:46:18<245:36:56,  5.58s/it]
 30%|██▉       | 66586/225000 [135:46:23<240:42:07,  5.47s/it]
 30%|██▉       | 66587/225000 [135:46:34<308:56:25,  7.02s/it]
 30%|██▉       | 66588/225000 [135:46:40<299:39:58,  6.81s/it]
 30%|██▉       | 66589/225000 [135:46:44<263:59:54,  6.00s/it]
 30%|██▉       | 66590/225000 [135:46:54<309:11:37,  7.03s/it]
 30%|██▉       | 66591/225000 [135:46:59<290:41:00,  6.61s/it]
 30%|██▉       | 66592/225000 [135:47:08<321:48:32,  7.31s/it]
 30%|██▉       | 66593/225000 [135:47:16<325:26:34,  7.40s/it]
 30%|██▉       | 66594/225000 [135:47:20<279:29:52,  6.35s/it]
 30%|██▉       | 66595/225000 [135:47:29<317:29:09,  7.22s/it]
 30%|██▉       | 66596/225000 [135:47:34<289:27:00,  6.58s/it]
 30%|██▉       | 66597/225000 [135:47:41<296:06:16,  6.73s/it]
 30%|██▉       | 66598/225000 [135:47:47<286:01:27,  6.50s/it]
 30%|██▉       | 66599/225000 [135:47:52<266:27:41,  6.06s/it]
 30%|██▉       | 66600/225000 [135:47:57<257:49:13,  5.86s/it]
                                                              
{'loss': '0.7258', 'grad_norm': '0.8296', 'learning_rate': '4.193e-05', 'epoch': '0.296', 'num_input_tokens_seen': 225636792, 'train_runtime': '4.889e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66600/225000 [135:47:57<257:49:13,  5.86s/it]
 30%|██▉       | 66601/225000 [135:48:04<274:07:43,  6.23s/it]
 30%|██▉       | 66602/225000 [135:48:15<330:15:45,  7.51s/it]
 30%|██▉       | 66603/225000 [135:48:20<299:54:20,  6.82s/it]
 30%|██▉       | 66604/225000 [135:48:26<284:12:49,  6.46s/it]
 30%|██▉       | 66605/225000 [135:48:31<269:59:11,  6.14s/it]
 30%|██▉       | 66606/225000 [135:48:36<258:46:01,  5.88s/it]
 30%|██▉       | 66607/225000 [135:48:42<248:01:12,  5.64s/it]
 30%|██▉       | 66608/225000 [135:48:47<247:50:45,  5.63s/it]
 30%|██▉       | 66609/225000 [135:48:52<238:15:52,  5.42s/it]
 30%|██▉       | 66610/225000 [135:49:05<336:46:19,  7.65s/it]
 30%|██▉       | 66611/225000 [135:49:12<328:25:50,  7.46s/it]
 30%|██▉       | 66612/225000 [135:49:19<317:12:55,  7.21s/it]
 30%|██▉       | 66613/225000 [135:49:26<322:38:22,  7.33s/it]
 30%|██▉       | 66614/225000 [135:49:31<286:30:35,  6.51s/it]
 30%|██▉       | 66615/225000 [135:49:40<318:33:48,  7.24s/it]
 30%|██▉       | 66616/225000 [135:49:47<324:26:02,  7.37s/it]
 30%|██▉       | 66617/225000 [135:49:51<279:24:59,  6.35s/it]
 30%|██▉       | 66618/225000 [135:49:59<296:47:02,  6.75s/it]
 30%|██▉       | 66619/225000 [135:50:09<344:35:51,  7.83s/it]
 30%|██▉       | 66620/225000 [135:50:14<295:57:29,  6.73s/it]
 30%|██▉       | 66621/225000 [135:50:18<268:27:42,  6.10s/it]
 30%|██▉       | 66622/225000 [135:50:23<251:28:05,  5.72s/it]
 30%|██▉       | 66623/225000 [135:50:28<248:06:24,  5.64s/it]
 30%|██▉       | 66624/225000 [135:50:34<242:25:10,  5.51s/it]
 30%|██▉       | 66625/225000 [135:50:47<347:59:38,  7.91s/it]
 30%|██▉       | 66626/225000 [135:50:56<363:43:30,  8.27s/it]
 30%|██▉       | 66627/225000 [135:51:00<303:52:45,  6.91s/it]
 30%|██▉       | 66628/225000 [135:51:08<315:58:35,  7.18s/it]
 30%|██▉       | 66629/225000 [135:51:12<271:51:52,  6.18s/it]
 30%|██▉       | 66630/225000 [135:51:22<326:11:49,  7.41s/it]
 30%|██▉       | 66631/225000 [135:51:29<316:01:46,  7.18s/it]
 30%|██▉       | 66632/225000 [135:51:32<268:28:54,  6.10s/it]
 30%|██▉       | 66633/225000 [135:51:37<247:38:51,  5.63s/it]
 30%|██▉       | 66634/225000 [135:51:42<245:33:30,  5.58s/it]
 30%|██▉       | 66635/225000 [135:51:48<245:24:32,  5.58s/it]
 30%|██▉       | 66636/225000 [135:51:53<245:37:26,  5.58s/it]
 30%|██▉       | 66637/225000 [135:52:02<281:56:55,  6.41s/it]
 30%|██▉       | 66638/225000 [135:52:12<331:11:12,  7.53s/it]
 30%|██▉       | 66639/225000 [135:52:17<302:47:15,  6.88s/it]
 30%|██▉       | 66640/225000 [135:52:23<282:04:37,  6.41s/it]
 30%|██▉       | 66641/225000 [135:52:28<262:45:13,  5.97s/it]
 30%|██▉       | 66642/225000 [135:52:33<252:35:14,  5.74s/it]
 30%|██▉       | 66643/225000 [135:52:38<240:53:40,  5.48s/it]
 30%|██▉       | 66644/225000 [135:52:45<266:29:34,  6.06s/it]
 30%|██▉       | 66645/225000 [135:52:56<334:30:51,  7.60s/it]
 30%|██▉       | 66646/225000 [135:53:00<287:57:08,  6.55s/it]
 30%|██▉       | 66647/225000 [135:53:11<344:06:54,  7.82s/it]
 30%|██▉       | 66648/225000 [135:53:14<285:50:40,  6.50s/it]
 30%|██▉       | 66649/225000 [135:53:23<317:59:14,  7.23s/it]
 30%|██▉       | 66650/225000 [135:53:28<288:17:14,  6.55s/it]
                                                              
{'loss': '0.7234', 'grad_norm': '0.8111', 'learning_rate': '4.192e-05', 'epoch': '0.2962', 'num_input_tokens_seen': 225786496, 'train_runtime': '4.892e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66650/225000 [135:53:28<288:17:14,  6.55s/it]
 30%|██▉       | 66651/225000 [135:53:33<257:29:17,  5.85s/it]
 30%|██▉       | 66652/225000 [135:53:39<269:09:38,  6.12s/it]
 30%|██▉       | 66653/225000 [135:53:51<342:27:45,  7.79s/it]
 30%|██▉       | 66654/225000 [135:53:55<295:28:22,  6.72s/it]
 30%|██▉       | 66655/225000 [135:54:04<323:10:43,  7.35s/it]
 30%|██▉       | 66656/225000 [135:54:09<294:18:40,  6.69s/it]
 30%|██▉       | 66657/225000 [135:54:15<278:24:58,  6.33s/it]
 30%|██▉       | 66658/225000 [135:54:20<263:09:54,  5.98s/it]
 30%|██▉       | 66659/225000 [135:54:25<247:04:05,  5.62s/it]
 30%|██▉       | 66660/225000 [135:54:38<351:06:28,  7.98s/it]
 30%|██▉       | 66661/225000 [135:54:43<314:54:35,  7.16s/it]
 30%|██▉       | 66662/225000 [135:54:53<350:27:15,  7.97s/it]
 30%|██▉       | 66663/225000 [135:54:57<292:26:45,  6.65s/it]
 30%|██▉       | 66664/225000 [135:55:01<265:37:17,  6.04s/it]
 30%|██▉       | 66665/225000 [135:55:10<294:30:24,  6.70s/it]
 30%|██▉       | 66666/225000 [135:55:18<314:27:21,  7.15s/it]
 30%|██▉       | 66667/225000 [135:55:22<276:00:00,  6.28s/it]
 30%|██▉       | 66668/225000 [135:55:27<255:18:19,  5.80s/it]
 30%|██▉       | 66669/225000 [135:55:35<280:01:01,  6.37s/it]
 30%|██▉       | 66670/225000 [135:55:42<292:48:06,  6.66s/it]
 30%|██▉       | 66671/225000 [135:55:50<314:43:29,  7.16s/it]
 30%|██▉       | 66672/225000 [135:55:55<278:53:18,  6.34s/it]
 30%|██▉       | 66673/225000 [135:55:59<253:24:59,  5.76s/it]
 30%|██▉       | 66674/225000 [135:56:04<242:15:28,  5.51s/it]
 30%|██▉       | 66675/225000 [135:56:16<333:40:21,  7.59s/it]
 30%|██▉       | 66676/225000 [135:56:22<305:29:11,  6.95s/it]
 30%|██▉       | 66677/225000 [135:56:28<300:31:46,  6.83s/it]
 30%|██▉       | 66678/225000 [135:56:34<278:19:10,  6.33s/it]
 30%|██▉       | 66679/225000 [135:56:41<294:23:21,  6.69s/it]
 30%|██▉       | 66680/225000 [135:56:50<319:13:54,  7.26s/it]
 30%|██▉       | 66681/225000 [135:56:54<279:02:09,  6.34s/it]
 30%|██▉       | 66682/225000 [135:56:58<252:38:49,  5.74s/it]
 30%|██▉       | 66683/225000 [135:57:05<271:49:22,  6.18s/it]
 30%|██▉       | 66684/225000 [135:57:16<326:48:43,  7.43s/it]
 30%|██▉       | 66685/225000 [135:57:20<281:27:25,  6.40s/it]
 30%|██▉       | 66686/225000 [135:57:26<276:37:45,  6.29s/it]
 30%|██▉       | 66687/225000 [135:57:35<309:48:32,  7.04s/it]
 30%|██▉       | 66688/225000 [135:57:40<281:43:43,  6.41s/it]
 30%|██▉       | 66689/225000 [135:57:44<255:40:29,  5.81s/it]
 30%|██▉       | 66690/225000 [135:57:55<322:59:47,  7.35s/it]
 30%|██▉       | 66691/225000 [135:58:00<297:37:40,  6.77s/it]
 30%|██▉       | 66692/225000 [135:58:08<310:55:45,  7.07s/it]
 30%|██▉       | 66693/225000 [135:58:15<311:36:40,  7.09s/it]
 30%|██▉       | 66694/225000 [135:58:20<285:24:06,  6.49s/it]
 30%|██▉       | 66695/225000 [135:58:26<276:50:13,  6.30s/it]
 30%|██▉       | 66696/225000 [135:58:32<272:32:04,  6.20s/it]
 30%|██▉       | 66697/225000 [135:58:37<249:00:16,  5.66s/it]
 30%|██▉       | 66698/225000 [135:58:47<311:04:49,  7.07s/it]
 30%|██▉       | 66699/225000 [135:58:51<277:24:08,  6.31s/it]
 30%|██▉       | 66700/225000 [135:58:59<300:15:56,  6.83s/it]
                                                              
{'loss': '0.7423', 'grad_norm': '0.7991', 'learning_rate': '4.191e-05', 'epoch': '0.2964', 'num_input_tokens_seen': 225932520, 'train_runtime': '4.895e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66700/225000 [135:58:59<300:15:56,  6.83s/it]
 30%|██▉       | 66701/225000 [135:59:03<259:26:27,  5.90s/it]
 30%|██▉       | 66702/225000 [135:59:14<327:33:59,  7.45s/it]
 30%|██▉       | 66703/225000 [135:59:19<287:39:39,  6.54s/it]
 30%|██▉       | 66704/225000 [135:59:23<257:10:04,  5.85s/it]
 30%|██▉       | 66705/225000 [135:59:30<276:07:22,  6.28s/it]
 30%|██▉       | 66706/225000 [135:59:39<304:56:11,  6.94s/it]
 30%|██▉       | 66707/225000 [135:59:42<261:23:53,  5.94s/it]
 30%|██▉       | 66708/225000 [135:59:53<321:01:52,  7.30s/it]
 30%|██▉       | 66709/225000 [136:00:00<318:03:25,  7.23s/it]
 30%|██▉       | 66710/225000 [136:00:05<287:51:44,  6.55s/it]
 30%|██▉       | 66711/225000 [136:00:10<273:52:33,  6.23s/it]
 30%|██▉       | 66712/225000 [136:00:16<262:55:17,  5.98s/it]
 30%|██▉       | 66713/225000 [136:00:21<253:25:14,  5.76s/it]
 30%|██▉       | 66714/225000 [136:00:26<242:29:00,  5.51s/it]
 30%|██▉       | 66715/225000 [136:00:34<275:51:40,  6.27s/it]
 30%|██▉       | 66716/225000 [136:00:44<330:25:36,  7.52s/it]
 30%|██▉       | 66717/225000 [136:00:49<290:14:00,  6.60s/it]
 30%|██▉       | 66718/225000 [136:00:54<272:37:37,  6.20s/it]
 30%|██▉       | 66719/225000 [136:00:59<254:32:15,  5.79s/it]
 30%|██▉       | 66720/225000 [136:01:09<312:47:21,  7.11s/it]
 30%|██▉       | 66721/225000 [136:01:20<358:21:13,  8.15s/it]
 30%|██▉       | 66722/225000 [136:01:26<333:07:51,  7.58s/it]
 30%|██▉       | 66723/225000 [136:01:31<302:27:42,  6.88s/it]
 30%|██▉       | 66724/225000 [136:01:36<276:54:29,  6.30s/it]
 30%|██▉       | 66725/225000 [136:01:41<253:57:58,  5.78s/it]
 30%|██▉       | 66726/225000 [136:01:45<240:51:30,  5.48s/it]
 30%|██▉       | 66727/225000 [136:01:58<328:41:42,  7.48s/it]
 30%|██▉       | 66728/225000 [136:02:04<318:30:05,  7.24s/it]
 30%|██▉       | 66729/225000 [136:02:13<335:41:21,  7.64s/it]
 30%|██▉       | 66730/225000 [136:02:17<289:57:26,  6.60s/it]
 30%|██▉       | 66731/225000 [136:02:27<340:15:30,  7.74s/it]
 30%|██▉       | 66732/225000 [136:02:32<293:04:47,  6.67s/it]
 30%|██▉       | 66733/225000 [136:02:40<320:15:16,  7.28s/it]
 30%|██▉       | 66734/225000 [136:02:48<321:12:40,  7.31s/it]
 30%|██▉       | 66735/225000 [136:02:52<279:31:27,  6.36s/it]
 30%|██▉       | 66736/225000 [136:03:00<308:00:54,  7.01s/it]
 30%|██▉       | 66737/225000 [136:03:05<278:39:49,  6.34s/it]
 30%|██▉       | 66738/225000 [136:03:10<258:48:26,  5.89s/it]
 30%|██▉       | 66739/225000 [136:03:15<250:22:55,  5.70s/it]
 30%|██▉       | 66740/225000 [136:03:28<338:04:37,  7.69s/it]
 30%|██▉       | 66741/225000 [136:03:32<292:40:26,  6.66s/it]
 30%|██▉       | 66742/225000 [136:03:36<262:36:46,  5.97s/it]
 30%|██▉       | 66743/225000 [136:03:46<310:08:14,  7.05s/it]
 30%|██▉       | 66744/225000 [136:03:54<330:17:43,  7.51s/it]
 30%|██▉       | 66745/225000 [136:03:59<293:33:56,  6.68s/it]
 30%|██▉       | 66746/225000 [136:04:08<322:44:08,  7.34s/it]
 30%|██▉       | 66747/225000 [136:04:15<324:31:18,  7.38s/it]
 30%|██▉       | 66748/225000 [136:04:19<276:46:11,  6.30s/it]
 30%|██▉       | 66749/225000 [136:04:23<248:05:25,  5.64s/it]
 30%|██▉       | 66750/225000 [136:04:33<296:27:28,  6.74s/it]
                                                              
{'loss': '0.7298', 'grad_norm': '0.6233', 'learning_rate': '4.189e-05', 'epoch': '0.2967', 'num_input_tokens_seen': 226082440, 'train_runtime': '4.899e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66750/225000 [136:04:33<296:27:28,  6.74s/it]
 30%|██▉       | 66751/225000 [136:04:40<302:31:43,  6.88s/it]
 30%|██▉       | 66752/225000 [136:04:47<310:22:57,  7.06s/it]
 30%|██▉       | 66753/225000 [136:04:56<333:57:36,  7.60s/it]
 30%|██▉       | 66754/225000 [136:05:00<281:25:09,  6.40s/it]
 30%|██▉       | 66755/225000 [136:05:08<303:25:59,  6.90s/it]
 30%|██▉       | 66756/225000 [136:05:13<284:44:10,  6.48s/it]
 30%|██▉       | 66757/225000 [136:05:17<254:14:25,  5.78s/it]
 30%|██▉       | 66758/225000 [136:05:29<323:45:49,  7.37s/it]
 30%|██▉       | 66759/225000 [136:05:32<268:35:24,  6.11s/it]
 30%|██▉       | 66760/225000 [136:05:39<281:52:21,  6.41s/it]
 30%|██▉       | 66761/225000 [136:05:43<257:21:18,  5.85s/it]
 30%|██▉       | 66762/225000 [136:05:52<298:55:48,  6.80s/it]
 30%|██▉       | 66763/225000 [136:05:56<257:42:28,  5.86s/it]
 30%|██▉       | 66764/225000 [136:06:06<317:20:43,  7.22s/it]
 30%|██▉       | 66765/225000 [136:06:11<288:14:48,  6.56s/it]
 30%|██▉       | 66766/225000 [136:06:16<264:27:07,  6.02s/it]
 30%|██▉       | 66767/225000 [136:06:21<254:31:30,  5.79s/it]
 30%|██▉       | 66768/225000 [136:06:31<308:06:19,  7.01s/it]
 30%|██▉       | 66769/225000 [136:06:39<321:15:27,  7.31s/it]
 30%|██▉       | 66770/225000 [136:06:44<289:07:01,  6.58s/it]
 30%|██▉       | 66771/225000 [136:06:50<282:56:48,  6.44s/it]
 30%|██▉       | 66772/225000 [136:06:57<287:38:26,  6.54s/it]
 30%|██▉       | 66773/225000 [136:07:11<378:25:41,  8.61s/it]
 30%|██▉       | 66774/225000 [136:07:14<309:51:26,  7.05s/it]
 30%|██▉       | 66775/225000 [136:07:24<345:44:43,  7.87s/it]
 30%|██▉       | 66776/225000 [136:07:29<310:19:30,  7.06s/it]
 30%|██▉       | 66777/225000 [136:07:35<295:22:07,  6.72s/it]
 30%|██▉       | 66778/225000 [136:07:39<263:39:34,  6.00s/it]
 30%|██▉       | 66779/225000 [136:07:50<323:39:20,  7.36s/it]
 30%|██▉       | 66780/225000 [136:07:53<267:40:36,  6.09s/it]
 30%|██▉       | 66781/225000 [136:08:02<307:35:57,  7.00s/it]
 30%|██▉       | 66782/225000 [136:08:08<297:25:05,  6.77s/it]
 30%|██▉       | 66783/225000 [136:08:12<264:49:09,  6.03s/it]
 30%|██▉       | 66784/225000 [136:08:19<272:52:27,  6.21s/it]
 30%|██▉       | 66785/225000 [136:08:27<301:54:01,  6.87s/it]
 30%|██▉       | 66786/225000 [136:08:32<274:59:17,  6.26s/it]
 30%|██▉       | 66787/225000 [136:08:37<260:22:16,  5.92s/it]
 30%|██▉       | 66788/225000 [136:08:42<244:05:47,  5.55s/it]
 30%|██▉       | 66789/225000 [136:08:48<241:53:08,  5.50s/it]
 30%|██▉       | 66790/225000 [136:09:00<335:06:10,  7.63s/it]
 30%|██▉       | 66791/225000 [136:09:07<322:34:18,  7.34s/it]
 30%|██▉       | 66792/225000 [136:09:15<330:28:30,  7.52s/it]
 30%|██▉       | 66793/225000 [136:09:22<331:07:12,  7.53s/it]
 30%|██▉       | 66794/225000 [136:09:26<284:37:22,  6.48s/it]
 30%|██▉       | 66795/225000 [136:09:30<252:35:18,  5.75s/it]
 30%|██▉       | 66796/225000 [136:09:40<302:06:01,  6.87s/it]
 30%|██▉       | 66797/225000 [136:09:45<273:26:56,  6.22s/it]
 30%|██▉       | 66798/225000 [136:09:49<251:03:39,  5.71s/it]
 30%|██▉       | 66799/225000 [136:10:02<347:18:29,  7.90s/it]
 30%|██▉       | 66800/225000 [136:10:08<320:50:30,  7.30s/it]
                                                              
{'loss': '0.7409', 'grad_norm': '0.7633', 'learning_rate': '4.188e-05', 'epoch': '0.2969', 'num_input_tokens_seen': 226231024, 'train_runtime': '4.902e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66800/225000 [136:10:08<320:50:30,  7.30s/it]
 30%|██▉       | 66801/225000 [136:10:12<280:10:30,  6.38s/it]
 30%|██▉       | 66802/225000 [136:10:18<267:06:03,  6.08s/it]
 30%|██▉       | 66803/225000 [136:10:26<292:20:55,  6.65s/it]
 30%|██▉       | 66804/225000 [136:10:34<320:07:44,  7.29s/it]
 30%|██▉       | 66805/225000 [136:10:38<277:12:49,  6.31s/it]
 30%|██▉       | 66806/225000 [136:10:49<330:38:01,  7.52s/it]
 30%|██▉       | 66807/225000 [136:10:53<287:27:30,  6.54s/it]
 30%|██▉       | 66808/225000 [136:11:00<294:56:32,  6.71s/it]
 30%|██▉       | 66809/225000 [136:11:08<306:11:43,  6.97s/it]
 30%|██▉       | 66810/225000 [136:11:11<264:09:28,  6.01s/it]
 30%|██▉       | 66811/225000 [136:11:22<324:39:58,  7.39s/it]
 30%|██▉       | 66812/225000 [136:11:26<279:20:54,  6.36s/it]
 30%|██▉       | 66813/225000 [136:11:30<250:46:44,  5.71s/it]
 30%|██▉       | 66814/225000 [136:11:39<298:19:33,  6.79s/it]
 30%|██▉       | 66815/225000 [136:11:46<298:56:37,  6.80s/it]
 30%|██▉       | 66816/225000 [136:11:51<268:25:40,  6.11s/it]
 30%|██▉       | 66817/225000 [136:11:59<294:06:56,  6.69s/it]
 30%|██▉       | 66818/225000 [136:12:08<322:28:38,  7.34s/it]
 30%|██▉       | 66819/225000 [136:12:11<271:54:53,  6.19s/it]
 30%|██▉       | 66820/225000 [136:12:17<263:25:24,  6.00s/it]
 30%|██▉       | 66821/225000 [136:12:22<255:51:26,  5.82s/it]
 30%|██▉       | 66822/225000 [136:12:28<255:30:39,  5.82s/it]
 30%|██▉       | 66823/225000 [136:12:34<255:23:03,  5.81s/it]
 30%|██▉       | 66824/225000 [136:12:42<283:55:28,  6.46s/it]
 30%|██▉       | 66825/225000 [136:12:55<374:54:31,  8.53s/it]
 30%|██▉       | 66826/225000 [136:13:00<321:34:31,  7.32s/it]
 30%|██▉       | 66827/225000 [136:13:08<332:57:50,  7.58s/it]
 30%|██▉       | 66828/225000 [136:13:15<325:01:39,  7.40s/it]
 30%|██▉       | 66829/225000 [136:13:22<321:39:55,  7.32s/it]
 30%|██▉       | 66830/225000 [136:13:29<311:53:00,  7.10s/it]
 30%|██▉       | 66831/225000 [136:13:34<295:07:24,  6.72s/it]
 30%|██▉       | 66832/225000 [136:13:42<312:17:34,  7.11s/it]
 30%|██▉       | 66833/225000 [136:13:47<274:26:14,  6.25s/it]
 30%|██▉       | 66834/225000 [136:13:51<253:16:42,  5.76s/it]
 30%|██▉       | 66835/225000 [136:13:56<244:35:56,  5.57s/it]
 30%|██▉       | 66836/225000 [136:14:08<327:49:12,  7.46s/it]
 30%|██▉       | 66837/225000 [136:14:16<329:05:46,  7.49s/it]
 30%|██▉       | 66838/225000 [136:14:21<296:32:20,  6.75s/it]
 30%|██▉       | 66839/225000 [136:14:26<273:50:43,  6.23s/it]
 30%|██▉       | 66840/225000 [136:14:31<253:55:38,  5.78s/it]
 30%|██▉       | 66841/225000 [136:14:36<248:29:01,  5.66s/it]
 30%|██▉       | 66842/225000 [136:14:50<363:59:01,  8.29s/it]
 30%|██▉       | 66843/225000 [136:14:55<311:22:41,  7.09s/it]
 30%|██▉       | 66844/225000 [136:14:59<280:43:10,  6.39s/it]
 30%|██▉       | 66845/225000 [136:15:04<253:19:00,  5.77s/it]
 30%|██▉       | 66846/225000 [136:15:16<332:51:38,  7.58s/it]
 30%|██▉       | 66847/225000 [136:15:21<302:50:54,  6.89s/it]
 30%|██▉       | 66848/225000 [136:15:27<288:10:33,  6.56s/it]
 30%|██▉       | 66849/225000 [136:15:31<263:56:09,  6.01s/it]
 30%|██▉       | 66850/225000 [136:15:37<260:01:25,  5.92s/it]
                                                              
{'loss': '0.7474', 'grad_norm': '0.8005', 'learning_rate': '4.187e-05', 'epoch': '0.2971', 'num_input_tokens_seen': 226379328, 'train_runtime': '4.905e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66850/225000 [136:15:37<260:01:25,  5.92s/it]
 30%|██▉       | 66851/225000 [136:15:42<247:23:09,  5.63s/it]
 30%|██▉       | 66852/225000 [136:15:50<274:06:02,  6.24s/it]
 30%|██▉       | 66853/225000 [136:16:02<351:51:40,  8.01s/it]
 30%|██▉       | 66854/225000 [136:16:06<306:04:01,  6.97s/it]
 30%|██▉       | 66855/225000 [136:16:17<355:19:52,  8.09s/it]
 30%|██▉       | 66856/225000 [136:16:21<300:34:34,  6.84s/it]
 30%|██▉       | 66857/225000 [136:16:25<264:55:39,  6.03s/it]
 30%|██▉       | 66858/225000 [136:16:36<328:55:48,  7.49s/it]
 30%|██▉       | 66859/225000 [136:16:41<292:40:37,  6.66s/it]
 30%|██▉       | 66860/225000 [136:16:46<269:22:38,  6.13s/it]
 30%|██▉       | 66861/225000 [136:16:51<256:02:21,  5.83s/it]
 30%|██▉       | 66862/225000 [136:16:56<250:07:48,  5.69s/it]
 30%|██▉       | 66863/225000 [136:17:01<238:46:27,  5.44s/it]
 30%|██▉       | 66864/225000 [136:17:08<259:19:59,  5.90s/it]
 30%|██▉       | 66865/225000 [136:17:21<357:37:28,  8.14s/it]
 30%|██▉       | 66866/225000 [136:17:26<306:46:44,  6.98s/it]
 30%|██▉       | 66867/225000 [136:17:31<280:27:43,  6.38s/it]
 30%|██▉       | 66868/225000 [136:17:42<343:02:13,  7.81s/it]
 30%|██▉       | 66869/225000 [136:17:46<292:11:18,  6.65s/it]
 30%|██▉       | 66870/225000 [136:17:54<319:44:13,  7.28s/it]
 30%|██▉       | 66871/225000 [136:17:58<269:26:38,  6.13s/it]
 30%|██▉       | 66872/225000 [136:18:06<294:19:15,  6.70s/it]
 30%|██▉       | 66873/225000 [136:18:10<254:32:13,  5.79s/it]
 30%|██▉       | 66874/225000 [136:18:19<301:25:26,  6.86s/it]
 30%|██▉       | 66875/225000 [136:18:25<290:05:16,  6.60s/it]
 30%|██▉       | 66876/225000 [136:18:31<280:20:00,  6.38s/it]
 30%|██▉       | 66877/225000 [136:18:36<270:22:44,  6.16s/it]
 30%|██▉       | 66878/225000 [136:18:44<294:54:53,  6.71s/it]
 30%|██▉       | 66879/225000 [136:18:52<303:13:34,  6.90s/it]
 30%|██▉       | 66880/225000 [136:18:56<271:15:12,  6.18s/it]
 30%|██▉       | 66881/225000 [136:19:03<281:36:45,  6.41s/it]
 30%|██▉       | 66882/225000 [136:19:15<346:44:26,  7.89s/it]
 30%|██▉       | 66883/225000 [136:19:18<292:08:51,  6.65s/it]
 30%|██▉       | 66884/225000 [136:19:28<327:17:26,  7.45s/it]
 30%|██▉       | 66885/225000 [136:19:31<274:16:29,  6.24s/it]
 30%|██▉       | 66886/225000 [136:19:39<301:42:28,  6.87s/it]
 30%|██▉       | 66887/225000 [136:19:43<262:15:00,  5.97s/it]
 30%|██▉       | 66888/225000 [136:19:54<325:23:05,  7.41s/it]
 30%|██▉       | 66889/225000 [136:19:58<279:29:19,  6.36s/it]
 30%|██▉       | 66890/225000 [136:20:04<279:16:50,  6.36s/it]
 30%|██▉       | 66891/225000 [136:20:10<271:01:55,  6.17s/it]
 30%|██▉       | 66892/225000 [136:20:19<312:28:44,  7.11s/it]
 30%|██▉       | 66893/225000 [136:20:26<302:06:40,  6.88s/it]
 30%|██▉       | 66894/225000 [136:20:31<281:17:23,  6.40s/it]
 30%|██▉       | 66895/225000 [136:20:36<265:39:28,  6.05s/it]
 30%|██▉       | 66896/225000 [136:20:42<260:16:05,  5.93s/it]
 30%|██▉       | 66897/225000 [136:20:48<265:56:08,  6.06s/it]
 30%|██▉       | 66898/225000 [136:20:54<259:51:45,  5.92s/it]
 30%|██▉       | 66899/225000 [136:21:00<259:05:26,  5.90s/it]
 30%|██▉       | 66900/225000 [136:21:10<313:25:52,  7.14s/it]
                                                              
{'loss': '0.7405', 'grad_norm': '0.7854', 'learning_rate': '4.185e-05', 'epoch': '0.2973', 'num_input_tokens_seen': 226527448, 'train_runtime': '4.909e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66900/225000 [136:21:10<313:25:52,  7.14s/it]
 30%|██▉       | 66901/225000 [136:21:20<354:19:30,  8.07s/it]
 30%|██▉       | 66902/225000 [136:21:24<297:59:31,  6.79s/it]
 30%|██▉       | 66903/225000 [136:21:28<265:00:45,  6.03s/it]
 30%|██▉       | 66904/225000 [136:21:38<319:24:10,  7.27s/it]
 30%|██▉       | 66905/225000 [136:21:43<293:06:37,  6.67s/it]
 30%|██▉       | 66906/225000 [136:21:49<275:42:44,  6.28s/it]
 30%|██▉       | 66907/225000 [136:21:58<320:15:36,  7.29s/it]
 30%|██▉       | 66908/225000 [136:22:03<278:24:48,  6.34s/it]
 30%|██▉       | 66909/225000 [136:22:07<257:52:38,  5.87s/it]
 30%|██▉       | 66910/225000 [136:22:20<341:36:13,  7.78s/it]
 30%|██▉       | 66911/225000 [136:22:25<311:22:27,  7.09s/it]
 30%|██▉       | 66912/225000 [136:22:31<294:35:04,  6.71s/it]
 30%|██▉       | 66913/225000 [136:22:36<276:04:37,  6.29s/it]
 30%|██▉       | 66914/225000 [136:22:42<264:00:41,  6.01s/it]
 30%|██▉       | 66915/225000 [136:22:48<266:08:56,  6.06s/it]
 30%|██▉       | 66916/225000 [136:22:53<258:57:16,  5.90s/it]
 30%|██▉       | 66917/225000 [136:22:59<252:13:54,  5.74s/it]
 30%|██▉       | 66918/225000 [136:23:04<249:07:12,  5.67s/it]
 30%|██▉       | 66919/225000 [136:23:15<319:47:06,  7.28s/it]
 30%|██▉       | 66920/225000 [136:23:23<331:38:29,  7.55s/it]
 30%|██▉       | 66921/225000 [136:23:28<288:57:43,  6.58s/it]
 30%|██▉       | 66922/225000 [136:23:35<304:58:24,  6.95s/it]
 30%|██▉       | 66923/225000 [136:23:44<325:16:20,  7.41s/it]
 30%|██▉       | 66924/225000 [136:23:48<283:23:50,  6.45s/it]
 30%|██▉       | 66925/225000 [136:23:53<260:04:56,  5.92s/it]
 30%|██▉       | 66926/225000 [136:24:01<285:14:12,  6.50s/it]
 30%|██▉       | 66927/225000 [136:24:10<327:20:01,  7.45s/it]
 30%|██▉       | 66928/225000 [136:24:18<333:17:38,  7.59s/it]
 30%|██▉       | 66929/225000 [136:24:24<308:17:56,  7.02s/it]
 30%|██▉       | 66930/225000 [136:24:29<279:30:15,  6.37s/it]
 30%|██▉       | 66931/225000 [136:24:35<274:34:49,  6.25s/it]
 30%|██▉       | 66932/225000 [136:24:40<261:46:27,  5.96s/it]
 30%|██▉       | 66933/225000 [136:24:46<260:36:53,  5.94s/it]
 30%|██▉       | 66934/225000 [136:24:51<249:52:40,  5.69s/it]
 30%|██▉       | 66935/225000 [136:25:04<343:32:40,  7.82s/it]
 30%|██▉       | 66936/225000 [136:25:11<335:57:02,  7.65s/it]
 30%|██▉       | 66937/225000 [136:25:16<305:24:55,  6.96s/it]
 30%|██▉       | 66938/225000 [136:25:22<287:05:50,  6.54s/it]
 30%|██▉       | 66939/225000 [136:25:33<349:49:59,  7.97s/it]
 30%|██▉       | 66940/225000 [136:25:40<328:18:03,  7.48s/it]
 30%|██▉       | 66941/225000 [136:25:44<290:22:57,  6.61s/it]
 30%|██▉       | 66942/225000 [136:25:49<264:29:49,  6.02s/it]
 30%|██▉       | 66943/225000 [136:26:01<348:16:08,  7.93s/it]
 30%|██▉       | 66944/225000 [136:26:05<297:45:41,  6.78s/it]
 30%|██▉       | 66945/225000 [136:26:15<329:19:42,  7.50s/it]
 30%|██▉       | 66946/225000 [136:26:19<285:17:09,  6.50s/it]
 30%|██▉       | 66947/225000 [136:26:23<260:17:25,  5.93s/it]
 30%|██▉       | 66948/225000 [136:26:34<318:41:48,  7.26s/it]
 30%|██▉       | 66949/225000 [136:26:39<298:51:54,  6.81s/it]
 30%|██▉       | 66950/225000 [136:26:45<284:10:39,  6.47s/it]
                                                              
{'loss': '0.7136', 'grad_norm': '0.8237', 'learning_rate': '4.184e-05', 'epoch': '0.2976', 'num_input_tokens_seen': 226676920, 'train_runtime': '4.912e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 66950/225000 [136:26:45<284:10:39,  6.47s/it]
 30%|██▉       | 66951/225000 [136:26:54<309:25:14,  7.05s/it]
 30%|██▉       | 66952/225000 [136:26:58<278:33:58,  6.35s/it]
 30%|██▉       | 66953/225000 [136:27:03<259:40:12,  5.91s/it]
 30%|██▉       | 66954/225000 [136:27:09<256:45:07,  5.85s/it]
 30%|██▉       | 66955/225000 [136:27:19<308:43:14,  7.03s/it]
 30%|██▉       | 66956/225000 [136:27:25<295:33:49,  6.73s/it]
 30%|██▉       | 66957/225000 [136:27:34<327:46:18,  7.47s/it]
 30%|██▉       | 66958/225000 [136:27:38<281:44:45,  6.42s/it]
 30%|██▉       | 66959/225000 [136:27:43<271:08:52,  6.18s/it]
 30%|██▉       | 66960/225000 [136:27:53<316:30:25,  7.21s/it]
 30%|██▉       | 66961/225000 [136:27:57<270:47:30,  6.17s/it]
 30%|██▉       | 66962/225000 [136:28:02<263:03:31,  5.99s/it]
 30%|██▉       | 66963/225000 [136:28:10<283:48:12,  6.46s/it]
 30%|██▉       | 66964/225000 [136:28:14<246:05:27,  5.61s/it]
 30%|██▉       | 66965/225000 [136:28:25<325:19:26,  7.41s/it]
 30%|██▉       | 66966/225000 [136:28:29<278:05:36,  6.33s/it]
 30%|██▉       | 66967/225000 [136:28:38<316:30:32,  7.21s/it]
 30%|██▉       | 66968/225000 [136:28:43<279:03:34,  6.36s/it]
 30%|██▉       | 66969/225000 [136:28:51<301:02:38,  6.86s/it]
 30%|██▉       | 66970/225000 [136:28:55<270:55:26,  6.17s/it]
 30%|██▉       | 66971/225000 [136:29:00<248:26:42,  5.66s/it]
 30%|██▉       | 66972/225000 [136:29:12<340:31:41,  7.76s/it]
 30%|██▉       | 66973/225000 [136:29:18<306:45:38,  6.99s/it]
 30%|██▉       | 66974/225000 [136:29:23<282:35:51,  6.44s/it]
 30%|██▉       | 66975/225000 [136:29:28<267:58:14,  6.10s/it]
 30%|██▉       | 66976/225000 [136:29:33<253:16:45,  5.77s/it]
 30%|██▉       | 66977/225000 [136:29:38<240:59:34,  5.49s/it]
 30%|██▉       | 66978/225000 [136:29:43<241:01:14,  5.49s/it]
 30%|██▉       | 66979/225000 [136:29:56<339:43:41,  7.74s/it]
 30%|██▉       | 66980/225000 [136:30:00<288:19:08,  6.57s/it]
 30%|██▉       | 66981/225000 [136:30:08<302:59:12,  6.90s/it]
 30%|██▉       | 66982/225000 [136:30:15<312:42:23,  7.12s/it]
 30%|██▉       | 66983/225000 [136:30:20<274:28:19,  6.25s/it]
 30%|██▉       | 66984/225000 [136:30:31<337:56:59,  7.70s/it]
 30%|██▉       | 66985/225000 [136:30:36<304:39:30,  6.94s/it]
 30%|██▉       | 66986/225000 [136:30:40<268:40:10,  6.12s/it]
 30%|██▉       | 66987/225000 [136:30:49<309:11:47,  7.04s/it]
 30%|██▉       | 66988/225000 [136:30:57<311:00:00,  7.09s/it]
 30%|██▉       | 66989/225000 [136:31:01<281:37:36,  6.42s/it]
 30%|██▉       | 66990/225000 [136:31:06<259:05:38,  5.90s/it]
 30%|██▉       | 66991/225000 [136:31:15<294:42:38,  6.71s/it]
 30%|██▉       | 66992/225000 [136:31:22<308:09:49,  7.02s/it]
 30%|██▉       | 66993/225000 [136:31:29<307:52:56,  7.01s/it]
 30%|██▉       | 66994/225000 [136:31:36<305:18:16,  6.96s/it]
 30%|██▉       | 66995/225000 [136:31:41<281:44:33,  6.42s/it]
 30%|██▉       | 66996/225000 [136:31:47<266:09:47,  6.06s/it]
 30%|██▉       | 66997/225000 [136:31:52<259:06:43,  5.90s/it]
 30%|██▉       | 66998/225000 [136:31:58<252:46:30,  5.76s/it]
 30%|██▉       | 66999/225000 [136:32:05<268:37:23,  6.12s/it]
 30%|██▉       | 67000/225000 [136:32:17<349:52:57,  7.97s/it]
                                                              
{'loss': '0.7148', 'grad_norm': '0.796', 'learning_rate': '4.183e-05', 'epoch': '0.2978', 'num_input_tokens_seen': 226827976, 'train_runtime': '4.915e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 67000/225000 [136:32:17<349:52:57,  7.97s/it]
 30%|██▉       | 67001/225000 [136:32:21<304:29:04,  6.94s/it]
 30%|██▉       | 67002/225000 [136:32:26<274:00:33,  6.24s/it]
 30%|██▉       | 67003/225000 [136:32:37<330:10:37,  7.52s/it]
 30%|██▉       | 67004/225000 [136:32:41<283:56:25,  6.47s/it]
 30%|██▉       | 67005/225000 [136:32:45<252:15:53,  5.75s/it]
 30%|██▉       | 67006/225000 [136:32:57<342:06:49,  7.80s/it]
 30%|██▉       | 67007/225000 [136:33:01<294:07:41,  6.70s/it]
 30%|██▉       | 67008/225000 [136:33:07<278:43:17,  6.35s/it]
 30%|██▉       | 67009/225000 [136:33:12<261:17:04,  5.95s/it]
 30%|██▉       | 67010/225000 [136:33:18<264:41:02,  6.03s/it]
 30%|██▉       | 67011/225000 [136:33:24<260:18:13,  5.93s/it]
 30%|██▉       | 67012/225000 [136:33:30<260:00:44,  5.92s/it]
 30%|██▉       | 67013/225000 [136:33:38<289:11:30,  6.59s/it]
 30%|██▉       | 67014/225000 [136:33:51<374:33:09,  8.53s/it]
 30%|██▉       | 67015/225000 [136:33:57<344:05:18,  7.84s/it]
 30%|██▉       | 67016/225000 [136:34:04<327:55:05,  7.47s/it]
 30%|██▉       | 67017/225000 [136:34:08<289:18:40,  6.59s/it]
 30%|██▉       | 67018/225000 [136:34:16<310:33:21,  7.08s/it]
 30%|██▉       | 67019/225000 [136:34:21<283:05:09,  6.45s/it]
 30%|██▉       | 67020/225000 [136:34:26<263:11:35,  6.00s/it]
 30%|██▉       | 67021/225000 [136:34:32<251:54:14,  5.74s/it]
 30%|██▉       | 67022/225000 [136:34:44<342:02:04,  7.79s/it]
 30%|██▉       | 67023/225000 [136:34:49<308:49:19,  7.04s/it]
 30%|██▉       | 67024/225000 [136:34:54<272:33:40,  6.21s/it]
 30%|██▉       | 67025/225000 [136:34:59<257:57:47,  5.88s/it]
 30%|██▉       | 67026/225000 [136:35:04<249:45:07,  5.69s/it]
 30%|██▉       | 67027/225000 [136:35:09<234:56:58,  5.35s/it]
 30%|██▉       | 67028/225000 [136:35:22<337:47:39,  7.70s/it]
 30%|██▉       | 67029/225000 [136:35:27<305:44:24,  6.97s/it]
 30%|██▉       | 67030/225000 [136:35:36<328:27:14,  7.49s/it]
 30%|██▉       | 67031/225000 [136:35:41<300:08:05,  6.84s/it]
 30%|██▉       | 67032/225000 [136:35:50<333:15:29,  7.59s/it]
 30%|██▉       | 67033/225000 [136:35:56<300:41:10,  6.85s/it]
 30%|██▉       | 67034/225000 [136:36:03<314:14:18,  7.16s/it]
 30%|██▉       | 67035/225000 [136:36:07<272:03:25,  6.20s/it]
 30%|██▉       | 67036/225000 [136:36:12<255:54:42,  5.83s/it]
 30%|██▉       | 67037/225000 [136:36:18<247:25:41,  5.64s/it]
 30%|██▉       | 67038/225000 [136:36:23<241:52:21,  5.51s/it]
 30%|██▉       | 67039/225000 [136:36:33<309:18:58,  7.05s/it]
 30%|██▉       | 67040/225000 [136:36:44<350:08:02,  7.98s/it]
 30%|██▉       | 67041/225000 [136:36:50<335:25:06,  7.64s/it]
 30%|██▉       | 67042/225000 [136:36:55<292:31:28,  6.67s/it]
 30%|██▉       | 67043/225000 [136:36:59<257:28:04,  5.87s/it]
 30%|██▉       | 67044/225000 [136:37:09<318:51:41,  7.27s/it]
 30%|██▉       | 67045/225000 [136:37:14<288:44:05,  6.58s/it]
 30%|██▉       | 67046/225000 [136:37:20<272:19:13,  6.21s/it]
 30%|██▉       | 67047/225000 [136:37:25<263:53:18,  6.01s/it]
 30%|██▉       | 67048/225000 [136:37:30<249:44:53,  5.69s/it]
 30%|██▉       | 67049/225000 [136:37:36<248:39:28,  5.67s/it]
 30%|██▉       | 67050/225000 [136:37:41<245:07:05,  5.59s/it]
                                                              
{'loss': '0.7239', 'grad_norm': '0.8355', 'learning_rate': '4.181e-05', 'epoch': '0.298', 'num_input_tokens_seen': 226977752, 'train_runtime': '4.919e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 67050/225000 [136:37:41<245:07:05,  5.59s/it]
 30%|██▉       | 67051/225000 [136:37:47<247:54:39,  5.65s/it]
 30%|██▉       | 67052/225000 [136:38:00<338:50:15,  7.72s/it]
 30%|██▉       | 67053/225000 [136:38:07<333:52:32,  7.61s/it]
 30%|██▉       | 67054/225000 [136:38:11<294:21:51,  6.71s/it]
 30%|██▉       | 67055/225000 [136:38:20<313:05:09,  7.14s/it]
 30%|██▉       | 67056/225000 [136:38:26<301:25:03,  6.87s/it]
 30%|██▉       | 67057/225000 [136:38:30<263:30:28,  6.01s/it]
 30%|██▉       | 67058/225000 [136:38:39<298:30:09,  6.80s/it]
 30%|██▉       | 67059/225000 [136:38:46<303:40:47,  6.92s/it]
 30%|██▉       | 67060/225000 [136:38:55<336:17:11,  7.67s/it]
 30%|██▉       | 67061/225000 [136:39:00<306:15:06,  6.98s/it]
 30%|██▉       | 67062/225000 [136:39:04<266:15:45,  6.07s/it]
 30%|██▉       | 67063/225000 [136:39:14<306:09:31,  6.98s/it]
 30%|██▉       | 67064/225000 [136:39:17<265:05:16,  6.04s/it]
 30%|██▉       | 67065/225000 [136:39:28<329:35:07,  7.51s/it]
 30%|██▉       | 67066/225000 [136:39:33<293:25:12,  6.69s/it]
 30%|██▉       | 67067/225000 [136:39:38<273:19:38,  6.23s/it]
 30%|██▉       | 67068/225000 [136:39:43<258:54:55,  5.90s/it]
 30%|██▉       | 67069/225000 [136:39:49<251:23:45,  5.73s/it]
 30%|██▉       | 67070/225000 [136:39:54<248:54:49,  5.67s/it]
 30%|██▉       | 67071/225000 [136:40:09<361:59:20,  8.25s/it]
 30%|██▉       | 67072/225000 [136:40:12<303:50:56,  6.93s/it]
 30%|██▉       | 67073/225000 [136:40:17<270:13:42,  6.16s/it]
 30%|██▉       | 67074/225000 [136:40:29<352:22:24,  8.03s/it]
 30%|██▉       | 67075/225000 [136:40:33<294:40:08,  6.72s/it]
 30%|██▉       | 67076/225000 [136:40:39<291:18:30,  6.64s/it]
 30%|██▉       | 67077/225000 [136:40:49<332:27:27,  7.58s/it]
 30%|██▉       | 67078/225000 [136:40:54<294:47:37,  6.72s/it]
 30%|██▉       | 67079/225000 [136:40:58<268:13:13,  6.11s/it]
 30%|██▉       | 67080/225000 [136:41:03<244:35:57,  5.58s/it]
 30%|██▉       | 67081/225000 [136:41:07<229:21:40,  5.23s/it]
 30%|██▉       | 67082/225000 [136:41:20<327:26:58,  7.46s/it]
 30%|██▉       | 67083/225000 [136:41:24<281:00:07,  6.41s/it]
 30%|██▉       | 67084/225000 [136:41:33<311:33:59,  7.10s/it]
 30%|██▉       | 67085/225000 [136:41:37<283:32:55,  6.46s/it]
 30%|██▉       | 67086/225000 [136:41:42<256:18:36,  5.84s/it]
 30%|██▉       | 67087/225000 [136:41:47<249:39:00,  5.69s/it]
 30%|██▉       | 67088/225000 [136:41:54<258:41:20,  5.90s/it]
 30%|██▉       | 67089/225000 [136:42:06<341:34:54,  7.79s/it]
 30%|██▉       | 67090/225000 [136:42:10<295:37:44,  6.74s/it]
 30%|██▉       | 67091/225000 [136:42:16<282:20:32,  6.44s/it]
 30%|██▉       | 67092/225000 [136:42:24<306:34:02,  6.99s/it]
 30%|██▉       | 67093/225000 [136:42:30<285:47:03,  6.52s/it]
 30%|██▉       | 67094/225000 [136:42:34<265:09:32,  6.05s/it]
 30%|██▉       | 67095/225000 [136:42:40<260:02:01,  5.93s/it]
 30%|██▉       | 67096/225000 [136:42:47<274:48:12,  6.27s/it]
 30%|██▉       | 67097/225000 [136:43:00<365:15:28,  8.33s/it]
 30%|██▉       | 67098/225000 [136:43:05<314:09:24,  7.16s/it]
 30%|██▉       | 67099/225000 [136:43:13<327:42:06,  7.47s/it]
 30%|██▉       | 67100/225000 [136:43:17<285:25:53,  6.51s/it]
                                                              
{'loss': '0.7262', 'grad_norm': '0.8638', 'learning_rate': '4.18e-05', 'epoch': '0.2982', 'num_input_tokens_seen': 227125784, 'train_runtime': '4.922e+05', 'train_tokens_per_second': '461.5'}

 30%|██▉       | 67100/225000 [136:43:17<285:25:53,  6.51s/it]
 30%|██▉       | 67101/225000 [136:43:22<260:42:26,  5.94s/it]
 30%|██▉       | 67102/225000 [136:43:27<246:18:56,  5.62s/it]
 30%|██▉       | 67103/225000 [136:43:34<268:33:40,  6.12s/it]
 30%|██▉       | 67104/225000 [136:43:45<333:30:39,  7.60s/it]
 30%|██▉       | 67105/225000 [136:43:53<334:32:26,  7.63s/it]
 30%|██▉       | 67106/225000 [136:43:57<292:36:58,  6.67s/it]
 30%|██▉       | 67107/225000 [136:44:02<265:08:08,  6.05s/it]
 30%|██▉       | 67108/225000 [136:44:08<264:18:35,  6.03s/it]
 30%|██▉       | 67109/225000 [136:44:20<340:31:33,  7.76s/it]
 30%|██▉       | 67110/225000 [136:44:24<293:36:31,  6.69s/it]
 30%|██▉       | 67111/225000 [136:44:33<326:48:40,  7.45s/it]
 30%|██▉       | 67112/225000 [136:44:39<309:26:21,  7.06s/it]
 30%|██▉       | 67113/225000 [136:44:44<279:14:46,  6.37s/it]
 30%|██▉       | 67114/225000 [136:44:49<264:09:54,  6.02s/it]
 30%|██▉       | 67115/225000 [136:44:54<248:40:28,  5.67s/it]
 30%|██▉       | 67116/225000 [136:45:05<321:48:53,  7.34s/it]
 30%|██▉       | 67117/225000 [136:45:11<300:33:09,  6.85s/it]
 30%|██▉       | 67118/225000 [136:45:18<297:40:38,  6.79s/it]
 30%|██▉       | 67119/225000 [136:45:22<263:57:40,  6.02s/it]
 30%|██▉       | 67120/225000 [136:45:29<283:04:59,  6.45s/it]
 30%|██▉       | 67121/225000 [136:45:38<307:56:20,  7.02s/it]
 30%|██▉       | 67122/225000 [136:45:44<296:58:39,  6.77s/it]
 30%|██▉       | 67123/225000 [136:45:51<303:51:33,  6.93s/it]
 30%|██▉       | 67124/225000 [136:45:58<301:05:34,  6.87s/it]
 30%|██▉       | 67125/225000 [136:46:02<264:40:32,  6.04s/it]
 30%|██▉       | 67126/225000 [136:46:12<312:47:16,  7.13s/it]
 30%|██▉       | 67127/225000 [136:46:16<280:53:26,  6.41s/it]
 30%|██▉       | 67128/225000 [136:46:21<258:54:11,  5.90s/it]
 30%|██▉       | 67129/225000 [136:46:26<247:11:09,  5.64s/it]
 30%|██▉       | 67130/225000 [136:46:39<347:00:45,  7.91s/it]
 30%|██▉       | 67131/225000 [136:46:44<308:29:52,  7.03s/it]
 30%|██▉       | 67132/225000 [136:46:52<311:58:33,  7.11s/it]
 30%|██▉       | 67133/225000 [136:46:57<284:16:34,  6.48s/it]
 30%|██▉       | 67134/225000 [136:47:01<262:00:00,  5.97s/it]
 30%|██▉       | 67135/225000 [136:47:10<297:05:39,  6.78s/it]
 30%|██▉       | 67136/225000 [136:47:19<328:11:55,  7.48s/it]
 30%|██▉       | 67137/225000 [136:47:23<278:04:04,  6.34s/it]
 30%|██▉       | 67138/225000 [136:47:32<310:23:20,  7.08s/it]
 30%|██▉       | 67139/225000 [136:47:35<267:30:40,  6.10s/it]
 30%|██▉       | 67140/225000 [136:47:41<265:25:37,  6.05s/it]
 30%|██▉       | 67141/225000 [136:47:51<314:35:17,  7.17s/it]
 30%|██▉       | 67142/225000 [136:47:57<299:50:26,  6.84s/it]
 30%|██▉       | 67143/225000 [136:48:03<281:24:11,  6.42s/it]
 30%|██▉       | 67144/225000 [136:48:07<257:35:17,  5.87s/it]
 30%|██▉       | 67145/225000 [136:48:16<292:45:05,  6.68s/it]
 30%|██▉       | 67146/225000 [136:48:24<315:41:28,  7.20s/it]
 30%|██▉       | 67147/225000 [136:48:32<325:21:25,  7.42s/it]
 30%|██▉       | 67148/225000 [136:48:37<297:53:58,  6.79s/it]
 30%|██▉       | 67149/225000 [136:48:43<280:16:37,  6.39s/it]
 30%|██▉       | 67150/225000 [136:48:49<278:17:44,  6.35s/it]
                                                              
{'loss': '0.7324', 'grad_norm': '0.652', 'learning_rate': '4.179e-05', 'epoch': '0.2984', 'num_input_tokens_seen': 227275800, 'train_runtime': '4.925e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67150/225000 [136:48:49<278:17:44,  6.35s/it]
 30%|██▉       | 67151/225000 [136:48:55<270:43:02,  6.17s/it]
 30%|██▉       | 67152/225000 [136:49:00<262:15:10,  5.98s/it]
 30%|██▉       | 67153/225000 [136:49:10<311:44:23,  7.11s/it]
 30%|██▉       | 67154/225000 [136:49:19<337:35:35,  7.70s/it]
 30%|██▉       | 67155/225000 [136:49:24<295:11:42,  6.73s/it]
 30%|██▉       | 67156/225000 [136:49:29<275:34:12,  6.29s/it]
 30%|██▉       | 67157/225000 [136:49:41<354:07:29,  8.08s/it]
 30%|██▉       | 67158/225000 [136:49:45<295:24:05,  6.74s/it]
 30%|██▉       | 67159/225000 [136:49:51<285:50:37,  6.52s/it]
 30%|██▉       | 67160/225000 [136:49:58<299:46:08,  6.84s/it]
 30%|██▉       | 67161/225000 [136:50:05<289:36:24,  6.61s/it]
 30%|██▉       | 67162/225000 [136:50:09<262:36:43,  5.99s/it]
 30%|██▉       | 67163/225000 [136:50:14<254:24:35,  5.80s/it]
 30%|██▉       | 67164/225000 [136:50:26<323:43:24,  7.38s/it]
 30%|██▉       | 67165/225000 [136:50:30<279:32:23,  6.38s/it]
 30%|██▉       | 67166/225000 [136:50:35<267:01:04,  6.09s/it]
 30%|██▉       | 67167/225000 [136:50:45<314:23:10,  7.17s/it]
 30%|██▉       | 67168/225000 [136:50:49<274:56:04,  6.27s/it]
 30%|██▉       | 67169/225000 [136:50:57<297:08:08,  6.78s/it]
 30%|██▉       | 67170/225000 [136:51:04<308:04:26,  7.03s/it]
 30%|██▉       | 67171/225000 [136:51:09<281:25:07,  6.42s/it]
 30%|██▉       | 67172/225000 [136:51:14<257:06:45,  5.86s/it]
 30%|██▉       | 67173/225000 [136:51:19<240:50:25,  5.49s/it]
 30%|██▉       | 67174/225000 [136:51:24<238:01:04,  5.43s/it]
 30%|██▉       | 67175/225000 [136:51:36<324:08:37,  7.39s/it]
 30%|██▉       | 67176/225000 [136:51:40<285:25:43,  6.51s/it]
 30%|██▉       | 67177/225000 [136:51:45<256:39:25,  5.85s/it]
 30%|██▉       | 67178/225000 [136:51:55<321:39:25,  7.34s/it]
 30%|██▉       | 67179/225000 [136:52:02<314:29:15,  7.17s/it]
 30%|██▉       | 67180/225000 [136:52:06<273:51:03,  6.25s/it]
 30%|██▉       | 67181/225000 [136:52:11<251:27:51,  5.74s/it]
 30%|██▉       | 67182/225000 [136:52:15<235:14:31,  5.37s/it]
 30%|██▉       | 67183/225000 [136:52:28<327:29:18,  7.47s/it]
 30%|██▉       | 67184/225000 [136:52:35<325:46:06,  7.43s/it]
 30%|██▉       | 67185/225000 [136:52:39<283:22:33,  6.46s/it]
 30%|██▉       | 67186/225000 [136:52:50<335:14:16,  7.65s/it]
 30%|██▉       | 67187/225000 [136:52:53<284:50:14,  6.50s/it]
 30%|██▉       | 67188/225000 [136:53:04<335:59:15,  7.66s/it]
 30%|██▉       | 67189/225000 [136:53:08<290:55:36,  6.64s/it]
 30%|██▉       | 67190/225000 [136:53:19<346:15:40,  7.90s/it]
 30%|██▉       | 67191/225000 [136:53:23<296:49:08,  6.77s/it]
 30%|██▉       | 67192/225000 [136:53:29<281:49:42,  6.43s/it]
 30%|██▉       | 67193/225000 [136:53:36<286:27:00,  6.53s/it]
 30%|██▉       | 67194/225000 [136:53:45<328:17:23,  7.49s/it]
 30%|██▉       | 67195/225000 [136:53:50<298:19:31,  6.81s/it]
 30%|██▉       | 67196/225000 [136:53:54<259:12:19,  5.91s/it]
 30%|██▉       | 67197/225000 [136:54:03<294:38:51,  6.72s/it]
 30%|██▉       | 67198/225000 [136:54:09<287:36:51,  6.56s/it]
 30%|██▉       | 67199/225000 [136:54:14<263:54:45,  6.02s/it]
 30%|██▉       | 67200/225000 [136:54:18<244:55:01,  5.59s/it]
                                                              
{'loss': '0.7254', 'grad_norm': '0.7178', 'learning_rate': '4.177e-05', 'epoch': '0.2987', 'num_input_tokens_seen': 227423752, 'train_runtime': '4.929e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67200/225000 [136:54:18<244:55:01,  5.59s/it]
 30%|██▉       | 67201/225000 [136:54:24<241:20:58,  5.51s/it]
 30%|██▉       | 67202/225000 [136:54:29<240:40:49,  5.49s/it]
 30%|██▉       | 67203/225000 [136:54:36<261:08:54,  5.96s/it]
 30%|██▉       | 67204/225000 [136:54:48<343:14:11,  7.83s/it]
 30%|██▉       | 67205/225000 [136:54:53<299:04:54,  6.82s/it]
 30%|██▉       | 67206/225000 [136:54:59<287:10:16,  6.55s/it]
 30%|██▉       | 67207/225000 [136:55:08<319:46:09,  7.30s/it]
 30%|██▉       | 67208/225000 [136:55:12<274:35:05,  6.26s/it]
 30%|██▉       | 67209/225000 [136:55:16<244:43:14,  5.58s/it]
 30%|██▉       | 67210/225000 [136:55:23<269:17:31,  6.14s/it]
 30%|██▉       | 67211/225000 [136:55:34<331:26:01,  7.56s/it]
 30%|██▉       | 67212/225000 [136:55:38<279:38:34,  6.38s/it]
 30%|██▉       | 67213/225000 [136:55:47<315:13:26,  7.19s/it]
 30%|██▉       | 67214/225000 [136:55:51<276:10:31,  6.30s/it]
 30%|██▉       | 67215/225000 [136:55:59<301:06:49,  6.87s/it]
 30%|██▉       | 67216/225000 [136:56:04<272:53:22,  6.23s/it]
 30%|██▉       | 67217/225000 [136:56:10<276:21:33,  6.31s/it]
 30%|██▉       | 67218/225000 [136:56:21<334:32:35,  7.63s/it]
 30%|██▉       | 67219/225000 [136:56:25<291:42:21,  6.66s/it]
 30%|██▉       | 67220/225000 [136:56:32<289:27:59,  6.60s/it]
 30%|██▉       | 67221/225000 [136:56:42<332:03:40,  7.58s/it]
 30%|██▉       | 67222/225000 [136:56:47<299:01:14,  6.82s/it]
 30%|██▉       | 67223/225000 [136:56:52<274:17:33,  6.26s/it]
 30%|██▉       | 67224/225000 [136:56:57<260:11:58,  5.94s/it]
 30%|██▉       | 67225/225000 [136:57:03<255:57:44,  5.84s/it]
 30%|██▉       | 67226/225000 [136:57:08<253:40:51,  5.79s/it]
 30%|██▉       | 67227/225000 [136:57:14<256:40:49,  5.86s/it]
 30%|██▉       | 67228/225000 [136:57:24<306:48:53,  7.00s/it]
 30%|██▉       | 67229/225000 [136:57:34<347:28:06,  7.93s/it]
 30%|██▉       | 67230/225000 [136:57:40<315:40:25,  7.20s/it]
 30%|██▉       | 67231/225000 [136:57:45<293:21:01,  6.69s/it]
 30%|██▉       | 67232/225000 [136:57:51<278:15:19,  6.35s/it]
 30%|██▉       | 67233/225000 [136:57:57<272:17:48,  6.21s/it]
 30%|██▉       | 67234/225000 [136:58:02<258:35:14,  5.90s/it]
 30%|██▉       | 67235/225000 [136:58:09<273:40:03,  6.24s/it]
 30%|██▉       | 67236/225000 [136:58:22<359:50:09,  8.21s/it]
 30%|██▉       | 67237/225000 [136:58:28<330:47:46,  7.55s/it]
 30%|██▉       | 67238/225000 [136:58:31<283:18:55,  6.47s/it]
 30%|██▉       | 67239/225000 [136:58:42<341:00:57,  7.78s/it]
 30%|██▉       | 67240/225000 [136:58:47<297:09:41,  6.78s/it]
 30%|██▉       | 67241/225000 [136:58:51<265:06:47,  6.05s/it]
 30%|██▉       | 67242/225000 [136:59:01<320:49:04,  7.32s/it]
 30%|██▉       | 67243/225000 [136:59:06<279:33:22,  6.38s/it]
 30%|██▉       | 67244/225000 [136:59:13<292:53:01,  6.68s/it]
 30%|██▉       | 67245/225000 [136:59:21<307:49:30,  7.02s/it]
 30%|██▉       | 67246/225000 [136:59:27<301:26:33,  6.88s/it]
 30%|██▉       | 67247/225000 [136:59:31<259:19:27,  5.92s/it]
 30%|██▉       | 67248/225000 [136:59:42<325:55:42,  7.44s/it]
 30%|██▉       | 67249/225000 [136:59:47<287:48:27,  6.57s/it]
 30%|██▉       | 67250/225000 [136:59:55<307:23:20,  7.01s/it]
                                                              
{'loss': '0.7322', 'grad_norm': '0.7784', 'learning_rate': '4.176e-05', 'epoch': '0.2989', 'num_input_tokens_seen': 227573616, 'train_runtime': '4.932e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67250/225000 [136:59:55<307:23:20,  7.01s/it]
 30%|██▉       | 67251/225000 [136:59:59<279:10:14,  6.37s/it]
 30%|██▉       | 67252/225000 [137:00:08<301:10:51,  6.87s/it]
 30%|██▉       | 67253/225000 [137:00:14<301:57:14,  6.89s/it]
 30%|██▉       | 67254/225000 [137:00:18<260:39:27,  5.95s/it]
 30%|██▉       | 67255/225000 [137:00:23<250:28:58,  5.72s/it]
 30%|██▉       | 67256/225000 [137:00:35<331:24:09,  7.56s/it]
 30%|██▉       | 67257/225000 [137:00:42<323:04:20,  7.37s/it]
 30%|██▉       | 67258/225000 [137:00:47<290:17:29,  6.63s/it]
 30%|██▉       | 67259/225000 [137:00:52<272:51:51,  6.23s/it]
 30%|██▉       | 67260/225000 [137:00:58<262:41:44,  6.00s/it]
 30%|██▉       | 67261/225000 [137:01:03<246:10:32,  5.62s/it]
 30%|██▉       | 67262/225000 [137:01:15<334:42:53,  7.64s/it]
 30%|██▉       | 67263/225000 [137:01:19<292:50:02,  6.68s/it]
 30%|██▉       | 67264/225000 [137:01:28<319:36:38,  7.29s/it]
 30%|██▉       | 67265/225000 [137:01:35<312:00:13,  7.12s/it]
 30%|██▉       | 67266/225000 [137:01:43<331:04:46,  7.56s/it]
 30%|██▉       | 67267/225000 [137:01:48<293:44:08,  6.70s/it]
 30%|██▉       | 67268/225000 [137:01:53<268:14:10,  6.12s/it]
 30%|██▉       | 67269/225000 [137:01:59<271:47:41,  6.20s/it]
 30%|██▉       | 67270/225000 [137:02:09<314:18:16,  7.17s/it]
 30%|██▉       | 67271/225000 [137:02:13<279:21:24,  6.38s/it]
 30%|██▉       | 67272/225000 [137:02:23<318:56:44,  7.28s/it]
 30%|██▉       | 67273/225000 [137:02:28<295:33:22,  6.75s/it]
 30%|██▉       | 67274/225000 [137:02:34<283:05:41,  6.46s/it]
 30%|██▉       | 67275/225000 [137:02:40<274:11:32,  6.26s/it]
 30%|██▉       | 67276/225000 [137:02:45<257:29:57,  5.88s/it]
 30%|██▉       | 67277/225000 [137:02:50<253:11:30,  5.78s/it]
 30%|██▉       | 67278/225000 [137:03:03<345:26:43,  7.88s/it]
 30%|██▉       | 67279/225000 [137:03:08<305:16:39,  6.97s/it]
 30%|██▉       | 67280/225000 [137:03:17<330:43:17,  7.55s/it]
 30%|██▉       | 67281/225000 [137:03:22<301:24:47,  6.88s/it]
 30%|██▉       | 67282/225000 [137:03:27<273:17:06,  6.24s/it]
 30%|██▉       | 67283/225000 [137:03:37<328:09:59,  7.49s/it]
 30%|██▉       | 67284/225000 [137:03:44<322:30:29,  7.36s/it]
 30%|██▉       | 67285/225000 [137:03:50<296:33:45,  6.77s/it]
 30%|██▉       | 67286/225000 [137:03:55<275:26:44,  6.29s/it]
 30%|██▉       | 67287/225000 [137:04:00<255:00:33,  5.82s/it]
 30%|██▉       | 67288/225000 [137:04:06<262:23:49,  5.99s/it]
 30%|██▉       | 67289/225000 [137:04:19<361:58:45,  8.26s/it]
 30%|██▉       | 67290/225000 [137:04:24<314:09:57,  7.17s/it]
 30%|██▉       | 67291/225000 [137:04:30<291:25:58,  6.65s/it]
 30%|██▉       | 67292/225000 [137:04:35<270:42:23,  6.18s/it]
 30%|██▉       | 67293/225000 [137:04:40<263:37:14,  6.02s/it]
 30%|██▉       | 67294/225000 [137:04:46<257:07:13,  5.87s/it]
 30%|██▉       | 67295/225000 [137:04:52<259:44:47,  5.93s/it]
 30%|██▉       | 67296/225000 [137:05:05<355:30:22,  8.12s/it]
 30%|██▉       | 67297/225000 [137:05:11<329:36:04,  7.52s/it]
 30%|██▉       | 67298/225000 [137:05:16<299:44:02,  6.84s/it]
 30%|██▉       | 67299/225000 [137:05:22<278:00:25,  6.35s/it]
 30%|██▉       | 67300/225000 [137:05:27<264:25:09,  6.04s/it]
                                                              
{'loss': '0.7296', 'grad_norm': '0.829', 'learning_rate': '4.175e-05', 'epoch': '0.2991', 'num_input_tokens_seen': 227724488, 'train_runtime': '4.935e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67300/225000 [137:05:27<264:25:09,  6.04s/it]
 30%|██▉       | 67301/225000 [137:05:32<253:37:37,  5.79s/it]
 30%|██▉       | 67302/225000 [137:05:38<255:56:30,  5.84s/it]
 30%|██▉       | 67303/225000 [137:05:44<252:47:15,  5.77s/it]
 30%|██▉       | 67304/225000 [137:05:49<248:28:19,  5.67s/it]
 30%|██▉       | 67305/225000 [137:06:01<325:30:14,  7.43s/it]
 30%|██▉       | 67306/225000 [137:06:12<377:24:06,  8.62s/it]
 30%|██▉       | 67307/225000 [137:06:16<311:44:21,  7.12s/it]
 30%|██▉       | 67308/225000 [137:06:25<338:14:19,  7.72s/it]
 30%|██▉       | 67309/225000 [137:06:29<297:38:31,  6.80s/it]
 30%|██▉       | 67310/225000 [137:06:35<282:00:02,  6.44s/it]
 30%|██▉       | 67311/225000 [137:06:41<271:41:05,  6.20s/it]
 30%|██▉       | 67312/225000 [137:06:46<260:27:30,  5.95s/it]
 30%|██▉       | 67313/225000 [137:06:52<257:06:40,  5.87s/it]
 30%|██▉       | 67314/225000 [137:06:58<264:02:29,  6.03s/it]
 30%|██▉       | 67315/225000 [137:07:12<359:48:00,  8.21s/it]
 30%|██▉       | 67316/225000 [137:07:17<325:55:15,  7.44s/it]
 30%|██▉       | 67317/225000 [137:07:21<282:55:59,  6.46s/it]
 30%|██▉       | 67318/225000 [137:07:31<330:38:31,  7.55s/it]
 30%|██▉       | 67319/225000 [137:07:37<308:28:52,  7.04s/it]
 30%|██▉       | 67320/225000 [137:07:42<279:13:39,  6.38s/it]
 30%|██▉       | 67321/225000 [137:07:52<331:19:31,  7.56s/it]
 30%|██▉       | 67322/225000 [137:07:57<286:35:45,  6.54s/it]
 30%|██▉       | 67323/225000 [137:08:04<294:24:03,  6.72s/it]
 30%|██▉       | 67324/225000 [137:08:11<297:20:11,  6.79s/it]
 30%|██▉       | 67325/225000 [137:08:19<311:31:40,  7.11s/it]
 30%|██▉       | 67326/225000 [137:08:24<294:33:44,  6.73s/it]
 30%|██▉       | 67327/225000 [137:08:30<277:42:52,  6.34s/it]
 30%|██▉       | 67328/225000 [137:08:35<261:18:27,  5.97s/it]
 30%|██▉       | 67329/225000 [137:08:40<245:27:38,  5.60s/it]
 30%|██▉       | 67330/225000 [137:08:45<239:18:21,  5.46s/it]
 30%|██▉       | 67331/225000 [137:08:58<340:07:28,  7.77s/it]
 30%|██▉       | 67332/225000 [137:09:04<324:20:18,  7.41s/it]
 30%|██▉       | 67333/225000 [137:09:09<289:43:32,  6.62s/it]
 30%|██▉       | 67334/225000 [137:09:14<266:33:18,  6.09s/it]
 30%|██▉       | 67335/225000 [137:09:22<295:35:55,  6.75s/it]
 30%|██▉       | 67336/225000 [137:09:32<330:36:44,  7.55s/it]
 30%|██▉       | 67337/225000 [137:09:36<288:46:50,  6.59s/it]
 30%|██▉       | 67338/225000 [137:09:46<327:17:42,  7.47s/it]
 30%|██▉       | 67339/225000 [137:09:51<297:19:16,  6.79s/it]
 30%|██▉       | 67340/225000 [137:09:56<279:36:04,  6.38s/it]
 30%|██▉       | 67341/225000 [137:10:08<345:56:54,  7.90s/it]
 30%|██▉       | 67342/225000 [137:10:12<293:15:55,  6.70s/it]
 30%|██▉       | 67343/225000 [137:10:21<322:34:57,  7.37s/it]
 30%|██▉       | 67344/225000 [137:10:26<294:30:29,  6.72s/it]
 30%|██▉       | 67345/225000 [137:10:32<281:50:52,  6.44s/it]
 30%|██▉       | 67346/225000 [137:10:37<268:24:45,  6.13s/it]
 30%|██▉       | 67347/225000 [137:10:42<255:47:26,  5.84s/it]
 30%|██▉       | 67348/225000 [137:10:47<247:27:46,  5.65s/it]
 30%|██▉       | 67349/225000 [137:10:58<318:21:07,  7.27s/it]
 30%|██▉       | 67350/225000 [137:11:06<323:29:22,  7.39s/it]
                                                              
{'loss': '0.7197', 'grad_norm': '0.7896', 'learning_rate': '4.173e-05', 'epoch': '0.2993', 'num_input_tokens_seen': 227876712, 'train_runtime': '4.939e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67350/225000 [137:11:06<323:29:22,  7.39s/it]
 30%|██▉       | 67351/225000 [137:11:10<281:06:37,  6.42s/it]
 30%|██▉       | 67352/225000 [137:11:14<247:44:53,  5.66s/it]
 30%|██▉       | 67353/225000 [137:11:27<342:47:36,  7.83s/it]
 30%|██▉       | 67354/225000 [137:11:31<286:35:21,  6.54s/it]
 30%|██▉       | 67355/225000 [137:11:38<296:53:02,  6.78s/it]
 30%|██▉       | 67356/225000 [137:11:43<276:01:47,  6.30s/it]
 30%|██▉       | 67357/225000 [137:11:49<269:35:41,  6.16s/it]
 30%|██▉       | 67358/225000 [137:11:54<260:38:07,  5.95s/it]
 30%|██▉       | 67359/225000 [137:12:00<256:02:57,  5.85s/it]
 30%|██▉       | 67360/225000 [137:12:06<261:52:46,  5.98s/it]
 30%|██▉       | 67361/225000 [137:12:19<350:36:55,  8.01s/it]
 30%|██▉       | 67362/225000 [137:12:25<320:56:44,  7.33s/it]
 30%|██▉       | 67363/225000 [137:12:31<302:09:28,  6.90s/it]
 30%|██▉       | 67364/225000 [137:12:36<283:24:50,  6.47s/it]
 30%|██▉       | 67365/225000 [137:12:42<274:01:19,  6.26s/it]
 30%|██▉       | 67366/225000 [137:12:48<266:44:05,  6.09s/it]
 30%|██▉       | 67367/225000 [137:12:59<332:14:52,  7.59s/it]
 30%|██▉       | 67368/225000 [137:13:04<301:53:30,  6.89s/it]
 30%|██▉       | 67369/225000 [137:13:09<282:39:31,  6.46s/it]
 30%|██▉       | 67370/225000 [137:13:15<268:22:42,  6.13s/it]
 30%|██▉       | 67371/225000 [137:13:21<263:30:39,  6.02s/it]
 30%|██▉       | 67372/225000 [137:13:26<256:06:09,  5.85s/it]
 30%|██▉       | 67373/225000 [137:13:32<264:27:04,  6.04s/it]
 30%|██▉       | 67374/225000 [137:13:46<359:55:24,  8.22s/it]
 30%|██▉       | 67375/225000 [137:13:51<315:36:38,  7.21s/it]
 30%|██▉       | 67376/225000 [137:14:00<341:50:59,  7.81s/it]
 30%|██▉       | 67377/225000 [137:14:05<308:31:46,  7.05s/it]
 30%|██▉       | 67378/225000 [137:14:10<279:11:31,  6.38s/it]
 30%|██▉       | 67379/225000 [137:14:20<326:15:32,  7.45s/it]
 30%|██▉       | 67380/225000 [137:14:23<275:30:07,  6.29s/it]
 30%|██▉       | 67381/225000 [137:14:32<306:04:10,  6.99s/it]
 30%|██▉       | 67382/225000 [137:14:37<276:31:40,  6.32s/it]
 30%|██▉       | 67383/225000 [137:14:42<259:34:43,  5.93s/it]
 30%|██▉       | 67384/225000 [137:14:47<244:25:13,  5.58s/it]
 30%|██▉       | 67385/225000 [137:14:58<317:03:04,  7.24s/it]
 30%|██▉       | 67386/225000 [137:15:04<305:09:37,  6.97s/it]
 30%|██▉       | 67387/225000 [137:15:11<299:32:32,  6.84s/it]
 30%|██▉       | 67388/225000 [137:15:17<299:48:21,  6.85s/it]
 30%|██▉       | 67389/225000 [137:15:25<304:40:33,  6.96s/it]
 30%|██▉       | 67390/225000 [137:15:29<264:47:26,  6.05s/it]
 30%|██▉       | 67391/225000 [137:15:33<245:27:07,  5.61s/it]
 30%|██▉       | 67392/225000 [137:15:47<350:47:11,  8.01s/it]
 30%|██▉       | 67393/225000 [137:15:51<305:08:53,  6.97s/it]
 30%|██▉       | 67394/225000 [137:15:58<298:27:12,  6.82s/it]
 30%|██▉       | 67395/225000 [137:16:06<313:21:22,  7.16s/it]
 30%|██▉       | 67396/225000 [137:16:13<309:00:37,  7.06s/it]
 30%|██▉       | 67397/225000 [137:16:17<276:36:27,  6.32s/it]
 30%|██▉       | 67398/225000 [137:16:22<257:43:03,  5.89s/it]
 30%|██▉       | 67399/225000 [137:16:27<239:58:29,  5.48s/it]
 30%|██▉       | 67400/225000 [137:16:38<321:54:09,  7.35s/it]
                                                              
{'loss': '0.7308', 'grad_norm': '0.8632', 'learning_rate': '4.172e-05', 'epoch': '0.2996', 'num_input_tokens_seen': 228026896, 'train_runtime': '4.942e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67400/225000 [137:16:38<321:54:09,  7.35s/it]
 30%|██▉       | 67401/225000 [137:16:45<316:10:57,  7.22s/it]
 30%|██▉       | 67402/225000 [137:16:50<288:32:33,  6.59s/it]
 30%|██▉       | 67403/225000 [137:16:56<270:37:38,  6.18s/it]
 30%|██▉       | 67404/225000 [137:17:01<254:15:26,  5.81s/it]
 30%|██▉       | 67405/225000 [137:17:13<344:53:54,  7.88s/it]
 30%|██▉       | 67406/225000 [137:17:19<314:16:57,  7.18s/it]
 30%|██▉       | 67407/225000 [137:17:24<287:13:24,  6.56s/it]
 30%|██▉       | 67408/225000 [137:17:29<268:18:09,  6.13s/it]
 30%|██▉       | 67409/225000 [137:17:35<261:15:00,  5.97s/it]
 30%|██▉       | 67410/225000 [137:17:40<253:48:44,  5.80s/it]
 30%|██▉       | 67411/225000 [137:17:53<344:49:20,  7.88s/it]
 30%|██▉       | 67412/225000 [137:17:59<317:32:57,  7.25s/it]
 30%|██▉       | 67413/225000 [137:18:09<355:32:30,  8.12s/it]
 30%|██▉       | 67414/225000 [137:18:14<319:55:17,  7.31s/it]
 30%|██▉       | 67415/225000 [137:18:19<292:24:07,  6.68s/it]
 30%|██▉       | 67416/225000 [137:18:24<271:15:29,  6.20s/it]
 30%|██▉       | 67417/225000 [137:18:30<266:05:17,  6.08s/it]
 30%|██▉       | 67418/225000 [137:18:35<254:18:57,  5.81s/it]
 30%|██▉       | 67419/225000 [137:18:41<249:41:28,  5.70s/it]
 30%|██▉       | 67420/225000 [137:18:54<349:23:16,  7.98s/it]
 30%|██▉       | 67421/225000 [137:19:01<340:16:18,  7.77s/it]
 30%|██▉       | 67422/225000 [137:19:06<292:11:53,  6.68s/it]
 30%|██▉       | 67423/225000 [137:19:13<299:55:16,  6.85s/it]
 30%|██▉       | 67424/225000 [137:19:19<290:43:33,  6.64s/it]
 30%|██▉       | 67425/225000 [137:19:27<305:37:50,  6.98s/it]
 30%|██▉       | 67426/225000 [137:19:33<294:44:57,  6.73s/it]
 30%|██▉       | 67427/225000 [137:19:38<269:13:16,  6.15s/it]
 30%|██▉       | 67428/225000 [137:19:43<253:11:33,  5.78s/it]
 30%|██▉       | 67429/225000 [137:19:48<252:56:22,  5.78s/it]
 30%|██▉       | 67430/225000 [137:19:54<249:05:02,  5.69s/it]
 30%|██▉       | 67431/225000 [137:19:59<243:29:50,  5.56s/it]
 30%|██▉       | 67432/225000 [137:20:04<239:31:55,  5.47s/it]
 30%|██▉       | 67433/225000 [137:20:18<341:37:27,  7.81s/it]
 30%|██▉       | 67434/225000 [137:20:25<337:52:50,  7.72s/it]
 30%|██▉       | 67435/225000 [137:20:30<299:45:03,  6.85s/it]
 30%|██▉       | 67436/225000 [137:20:35<270:24:37,  6.18s/it]
 30%|██▉       | 67437/225000 [137:20:46<335:57:48,  7.68s/it]
 30%|██▉       | 67438/225000 [137:20:55<356:39:21,  8.15s/it]
 30%|██▉       | 67439/225000 [137:21:02<348:19:31,  7.96s/it]
 30%|██▉       | 67440/225000 [137:21:06<295:16:35,  6.75s/it]
 30%|██▉       | 67441/225000 [137:21:12<281:59:02,  6.44s/it]
 30%|██▉       | 67442/225000 [137:21:21<316:54:14,  7.24s/it]
 30%|██▉       | 67443/225000 [137:21:26<278:26:22,  6.36s/it]
 30%|██▉       | 67444/225000 [137:21:30<254:21:55,  5.81s/it]
 30%|██▉       | 67445/225000 [137:21:41<320:51:03,  7.33s/it]
 30%|██▉       | 67446/225000 [137:21:49<334:10:13,  7.64s/it]
 30%|██▉       | 67447/225000 [137:21:55<313:06:12,  7.15s/it]
 30%|██▉       | 67448/225000 [137:22:01<292:03:16,  6.67s/it]
 30%|██▉       | 67449/225000 [137:22:05<252:59:00,  5.78s/it]
 30%|██▉       | 67450/225000 [137:22:14<306:52:32,  7.01s/it]
                                                              
{'loss': '0.7294', 'grad_norm': '0.8007', 'learning_rate': '4.171e-05', 'epoch': '0.2998', 'num_input_tokens_seen': 228176056, 'train_runtime': '4.945e+05', 'train_tokens_per_second': '461.4'}

 30%|██▉       | 67450/225000 [137:22:14<306:52:32,  7.01s/it]
 30%|██▉       | 67451/225000 [137:22:19<273:43:18,  6.25s/it]
 30%|██▉       | 67452/225000 [137:22:23<248:01:55,  5.67s/it]
 30%|██▉       | 67453/225000 [137:22:36<334:53:05,  7.65s/it]
 30%|██▉       | 67454/225000 [137:22:40<293:30:05,  6.71s/it]
 30%|██▉       | 67455/225000 [137:22:49<328:44:17,  7.51s/it]
 30%|██▉       | 67456/225000 [137:22:54<290:28:34,  6.64s/it]
 30%|██▉       | 67457/225000 [137:22:59<265:54:16,  6.08s/it]
 30%|██▉       | 67458/225000 [137:23:03<244:32:18,  5.59s/it]
 30%|██▉       | 67459/225000 [137:23:15<329:42:05,  7.53s/it]
 30%|██▉       | 67460/225000 [137:23:21<307:42:16,  7.03s/it]
 30%|██▉       | 67461/225000 [137:23:30<325:06:51,  7.43s/it]
 30%|██▉       | 67462/225000 [137:23:36<311:02:17,  7.11s/it]
 30%|██▉       | 67463/225000 [137:23:43<310:05:18,  7.09s/it]
 30%|██▉       | 67464/225000 [137:23:48<289:06:15,  6.61s/it]
 30%|██▉       | 67465/225000 [137:23:53<259:49:25,  5.94s/it]
 30%|██▉       | 67466/225000 [137:24:03<315:21:51,  7.21s/it]
 30%|██▉       | 67467/225000 [137:24:07<279:50:51,  6.40s/it]
 30%|██▉       | 67468/225000 [137:24:16<313:21:02,  7.16s/it]
 30%|██▉       | 67469/225000 [137:24:20<267:57:07,  6.12s/it]
 30%|██▉       | 67470/225000 [137:24:29<309:11:53,  7.07s/it]
 30%|██▉       | 67471/225000 [137:24:35<292:19:35,  6.68s/it]
 30%|██▉       | 67472/225000 [137:24:42<298:01:01,  6.81s/it]
 30%|██▉       | 67473/225000 [137:24:47<273:52:05,  6.26s/it]
 30%|██▉       | 67474/225000 [137:24:52<257:20:47,  5.88s/it]
 30%|██▉       | 67475/225000 [137:24:57<243:24:52,  5.56s/it]
 30%|██▉       | 67476/225000 [137:25:03<242:46:39,  5.55s/it]
 30%|██▉       | 67477/225000 [137:25:16<340:08:35,  7.77s/it]
 30%|██▉       | 67478/225000 [137:25:22<317:44:45,  7.26s/it]
 30%|██▉       | 67479/225000 [137:25:26<280:13:28,  6.40s/it]
 30%|██▉       | 67480/225000 [137:25:31<257:44:05,  5.89s/it]
 30%|██▉       | 67481/225000 [137:25:40<306:12:20,  7.00s/it]
 30%|██▉       | 67482/225000 [137:25:47<306:51:19,  7.01s/it]
 30%|██▉       | 67483/225000 [137:25:54<308:43:18,  7.06s/it]
 30%|██▉       | 67484/225000 [137:26:00<292:05:32,  6.68s/it]
 30%|██▉       | 67485/225000 [137:26:10<330:07:04,  7.54s/it]
 30%|██▉       | 67486/225000 [137:26:16<308:21:12,  7.05s/it]
 30%|██▉       | 67487/225000 [137:26:20<272:46:21,  6.23s/it]
 30%|██▉       | 67488/225000 [137:26:25<257:16:52,  5.88s/it]
 30%|██▉       | 67489/225000 [137:26:30<244:20:04,  5.58s/it]
 30%|██▉       | 67490/225000 [137:26:35<238:26:06,  5.45s/it]
 30%|██▉       | 67491/225000 [137:26:49<343:58:07,  7.86s/it]
 30%|██▉       | 67492/225000 [137:26:54<307:05:56,  7.02s/it]
 30%|██▉       | 67493/225000 [137:26:58<272:39:05,  6.23s/it]
 30%|██▉       | 67494/225000 [137:27:03<254:05:47,  5.81s/it]
 30%|██▉       | 67495/225000 [137:27:08<242:17:35,  5.54s/it]
 30%|██▉       | 67496/225000 [137:27:20<326:26:09,  7.46s/it]
 30%|██▉       | 67497/225000 [137:27:29<353:42:39,  8.08s/it]
 30%|██▉       | 67498/225000 [137:27:34<313:46:30,  7.17s/it]
 30%|██▉       | 67499/225000 [137:27:42<315:59:14,  7.22s/it]
 30%|███       | 67500/225000 [137:27:46<282:16:13,  6.45s/it]
                                                              
{'loss': '0.7275', 'grad_norm': '0.8938', 'learning_rate': '4.169e-05', 'epoch': '0.3', 'num_input_tokens_seen': 228327048, 'train_runtime': '4.949e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67500/225000 [137:27:46<282:16:13,  6.45s/it]
 30%|███       | 67501/225000 [137:27:56<323:29:17,  7.39s/it]
 30%|███       | 67502/225000 [137:28:01<298:27:28,  6.82s/it]
 30%|███       | 67503/225000 [137:28:11<334:33:46,  7.65s/it]
 30%|███       | 67504/225000 [137:28:15<292:15:38,  6.68s/it]
 30%|███       | 67505/225000 [137:28:26<339:17:56,  7.76s/it]
 30%|███       | 67506/225000 [137:28:30<300:29:24,  6.87s/it]
 30%|███       | 67507/225000 [137:28:38<302:59:05,  6.93s/it]
 30%|███       | 67508/225000 [137:28:42<265:14:13,  6.06s/it]
 30%|███       | 67509/225000 [137:28:52<322:08:32,  7.36s/it]
 30%|███       | 67510/225000 [137:28:58<303:47:28,  6.94s/it]
 30%|███       | 67511/225000 [137:29:04<298:05:30,  6.81s/it]
 30%|███       | 67512/225000 [137:29:09<262:16:26,  6.00s/it]
 30%|███       | 67513/225000 [137:29:20<332:36:31,  7.60s/it]
 30%|███       | 67514/225000 [137:29:25<304:16:19,  6.96s/it]
 30%|███       | 67515/225000 [137:29:31<282:14:12,  6.45s/it]
 30%|███       | 67516/225000 [137:29:36<262:38:02,  6.00s/it]
 30%|███       | 67517/225000 [137:29:49<359:08:52,  8.21s/it]
 30%|███       | 67518/225000 [137:29:53<305:29:32,  6.98s/it]
 30%|███       | 67519/225000 [137:30:01<316:21:41,  7.23s/it]
 30%|███       | 67520/225000 [137:30:06<286:34:18,  6.55s/it]
 30%|███       | 67521/225000 [137:30:11<262:19:29,  6.00s/it]
 30%|███       | 67522/225000 [137:30:16<249:44:52,  5.71s/it]
 30%|███       | 67523/225000 [137:30:29<353:48:59,  8.09s/it]
 30%|███       | 67524/225000 [137:30:35<316:56:39,  7.25s/it]
 30%|███       | 67525/225000 [137:30:40<295:20:26,  6.75s/it]
 30%|███       | 67526/225000 [137:30:45<270:25:06,  6.18s/it]
 30%|███       | 67527/225000 [137:30:50<257:23:39,  5.88s/it]
 30%|███       | 67528/225000 [137:30:56<254:47:01,  5.82s/it]
 30%|███       | 67529/225000 [137:31:08<335:01:29,  7.66s/it]
 30%|███       | 67530/225000 [137:31:15<333:22:01,  7.62s/it]
 30%|███       | 67531/225000 [137:31:19<285:11:55,  6.52s/it]
 30%|███       | 67532/225000 [137:31:25<270:23:04,  6.18s/it]
 30%|███       | 67533/225000 [137:31:29<249:50:31,  5.71s/it]
 30%|███       | 67534/225000 [137:31:42<338:58:18,  7.75s/it]
 30%|███       | 67535/225000 [137:31:46<296:09:08,  6.77s/it]
 30%|███       | 67536/225000 [137:31:51<268:28:30,  6.14s/it]
 30%|███       | 67537/225000 [137:31:59<300:12:06,  6.86s/it]
 30%|███       | 67538/225000 [137:32:09<337:32:26,  7.72s/it]
 30%|███       | 67539/225000 [137:32:13<285:59:07,  6.54s/it]
 30%|███       | 67540/225000 [137:32:21<305:40:01,  6.99s/it]
 30%|███       | 67541/225000 [137:32:28<305:26:23,  6.98s/it]
 30%|███       | 67542/225000 [137:32:34<294:37:58,  6.74s/it]
 30%|███       | 67543/225000 [137:32:40<280:29:10,  6.41s/it]
 30%|███       | 67544/225000 [137:32:49<322:25:16,  7.37s/it]
 30%|███       | 67545/225000 [137:32:53<271:53:17,  6.22s/it]
 30%|███       | 67546/225000 [137:33:02<304:30:17,  6.96s/it]
 30%|███       | 67547/225000 [137:33:07<277:35:19,  6.35s/it]
 30%|███       | 67548/225000 [137:33:15<311:17:36,  7.12s/it]
 30%|███       | 67549/225000 [137:33:21<290:00:29,  6.63s/it]
 30%|███       | 67550/225000 [137:33:30<315:32:04,  7.21s/it]
                                                              
{'loss': '0.7267', 'grad_norm': '0.7761', 'learning_rate': '4.168e-05', 'epoch': '0.3002', 'num_input_tokens_seen': 228478768, 'train_runtime': '4.952e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67550/225000 [137:33:30<315:32:04,  7.21s/it]
 30%|███       | 67551/225000 [137:33:34<280:50:55,  6.42s/it]
 30%|███       | 67552/225000 [137:33:39<263:46:18,  6.03s/it]
 30%|███       | 67553/225000 [137:33:44<251:53:58,  5.76s/it]
 30%|███       | 67554/225000 [137:33:49<241:32:15,  5.52s/it]
 30%|███       | 67555/225000 [137:34:02<336:12:23,  7.69s/it]
 30%|███       | 67556/225000 [137:34:08<307:02:57,  7.02s/it]
 30%|███       | 67557/225000 [137:34:15<310:42:46,  7.10s/it]
 30%|███       | 67558/225000 [137:34:23<319:08:35,  7.30s/it]
 30%|███       | 67559/225000 [137:34:27<285:58:28,  6.54s/it]
 30%|███       | 67560/225000 [137:34:37<332:29:34,  7.60s/it]
 30%|███       | 67561/225000 [137:34:41<286:08:44,  6.54s/it]
 30%|███       | 67562/225000 [137:34:47<273:48:30,  6.26s/it]
 30%|███       | 67563/225000 [137:34:55<294:39:04,  6.74s/it]
 30%|███       | 67564/225000 [137:35:01<281:01:49,  6.43s/it]
 30%|███       | 67565/225000 [137:35:06<270:23:23,  6.18s/it]
 30%|███       | 67566/225000 [137:35:11<257:47:24,  5.89s/it]
 30%|███       | 67567/225000 [137:35:20<287:20:40,  6.57s/it]
 30%|███       | 67568/225000 [137:35:29<319:21:11,  7.30s/it]
 30%|███       | 67569/225000 [137:35:33<281:50:29,  6.44s/it]
 30%|███       | 67570/225000 [137:35:43<330:34:56,  7.56s/it]
 30%|███       | 67571/225000 [137:35:51<328:05:19,  7.50s/it]
 30%|███       | 67572/225000 [137:35:55<287:42:09,  6.58s/it]
 30%|███       | 67573/225000 [137:36:00<264:39:32,  6.05s/it]
 30%|███       | 67574/225000 [137:36:05<247:22:41,  5.66s/it]
 30%|███       | 67575/225000 [137:36:12<272:33:19,  6.23s/it]
 30%|███       | 67576/225000 [137:36:23<335:26:58,  7.67s/it]
 30%|███       | 67577/225000 [137:36:28<294:31:16,  6.74s/it]
 30%|███       | 67578/225000 [137:36:32<267:14:03,  6.11s/it]
 30%|███       | 67579/225000 [137:36:43<324:03:35,  7.41s/it]
 30%|███       | 67580/225000 [137:36:47<285:53:21,  6.54s/it]
 30%|███       | 67581/225000 [137:36:52<267:40:41,  6.12s/it]
 30%|███       | 67582/225000 [137:36:57<251:59:47,  5.76s/it]
 30%|███       | 67583/225000 [137:37:03<243:47:23,  5.58s/it]
 30%|███       | 67584/225000 [137:37:15<330:00:15,  7.55s/it]
 30%|███       | 67585/225000 [137:37:19<282:00:26,  6.45s/it]
 30%|███       | 67586/225000 [137:37:28<316:16:46,  7.23s/it]
 30%|███       | 67587/225000 [137:37:32<276:31:37,  6.32s/it]
 30%|███       | 67588/225000 [137:37:37<261:04:19,  5.97s/it]
 30%|███       | 67589/225000 [137:37:49<345:45:52,  7.91s/it]
 30%|███       | 67590/225000 [137:37:54<300:10:17,  6.86s/it]
 30%|███       | 67591/225000 [137:37:59<272:05:19,  6.22s/it]
 30%|███       | 67592/225000 [137:38:03<246:18:21,  5.63s/it]
 30%|███       | 67593/225000 [137:38:15<328:08:54,  7.50s/it]
 30%|███       | 67594/225000 [137:38:20<302:35:45,  6.92s/it]
 30%|███       | 67595/225000 [137:38:24<262:27:09,  6.00s/it]
 30%|███       | 67596/225000 [137:38:32<292:44:16,  6.70s/it]
 30%|███       | 67597/225000 [137:38:37<270:33:21,  6.19s/it]
 30%|███       | 67598/225000 [137:38:43<256:40:14,  5.87s/it]
 30%|███       | 67599/225000 [137:38:48<252:47:21,  5.78s/it]
 30%|███       | 67600/225000 [137:39:00<332:01:01,  7.59s/it]
                                                              
{'loss': '0.7282', 'grad_norm': '0.801', 'learning_rate': '4.166e-05', 'epoch': '0.3004', 'num_input_tokens_seen': 228628448, 'train_runtime': '4.955e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67600/225000 [137:39:00<332:01:01,  7.59s/it]
 30%|███       | 67601/225000 [137:39:07<331:01:34,  7.57s/it]
 30%|███       | 67602/225000 [137:39:15<335:15:36,  7.67s/it]
 30%|███       | 67603/225000 [137:39:20<293:24:29,  6.71s/it]
 30%|███       | 67604/225000 [137:39:27<305:11:57,  6.98s/it]
 30%|███       | 67605/225000 [137:39:36<322:29:55,  7.38s/it]
 30%|███       | 67606/225000 [137:39:40<278:39:19,  6.37s/it]
 30%|███       | 67607/225000 [137:39:46<274:21:48,  6.28s/it]
 30%|███       | 67608/225000 [137:39:55<312:42:31,  7.15s/it]
 30%|███       | 67609/225000 [137:39:59<275:15:41,  6.30s/it]
 30%|███       | 67610/225000 [137:40:04<253:10:47,  5.79s/it]
 30%|███       | 67611/225000 [137:40:09<242:39:19,  5.55s/it]
 30%|███       | 67612/225000 [137:40:19<299:01:15,  6.84s/it]
 30%|███       | 67613/225000 [137:40:30<359:21:42,  8.22s/it]
 30%|███       | 67614/225000 [137:40:37<341:07:32,  7.80s/it]
 30%|███       | 67615/225000 [137:40:42<303:56:12,  6.95s/it]
 30%|███       | 67616/225000 [137:40:47<279:58:40,  6.40s/it]
 30%|███       | 67617/225000 [137:40:53<269:49:00,  6.17s/it]
 30%|███       | 67618/225000 [137:40:59<265:31:29,  6.07s/it]
 30%|███       | 67619/225000 [137:41:04<258:10:39,  5.91s/it]
 30%|███       | 67620/225000 [137:41:09<248:49:31,  5.69s/it]
 30%|███       | 67621/225000 [137:41:19<301:48:59,  6.90s/it]
 30%|███       | 67622/225000 [137:41:24<281:18:30,  6.43s/it]
 30%|███       | 67623/225000 [137:41:35<333:41:13,  7.63s/it]
 30%|███       | 67624/225000 [137:41:39<290:23:10,  6.64s/it]
 30%|███       | 67625/225000 [137:41:45<273:26:38,  6.26s/it]
 30%|███       | 67626/225000 [137:41:51<282:27:46,  6.46s/it]
 30%|███       | 67627/225000 [137:42:03<351:36:23,  8.04s/it]
 30%|███       | 67628/225000 [137:42:10<337:27:52,  7.72s/it]
 30%|███       | 67629/225000 [137:42:17<329:35:10,  7.54s/it]
 30%|███       | 67630/225000 [137:42:21<278:18:18,  6.37s/it]
 30%|███       | 67631/225000 [137:42:26<258:14:38,  5.91s/it]
 30%|███       | 67632/225000 [137:42:31<248:09:33,  5.68s/it]
 30%|███       | 67633/225000 [137:42:36<236:45:08,  5.42s/it]
 30%|███       | 67634/225000 [137:42:42<244:49:23,  5.60s/it]
 30%|███       | 67635/225000 [137:42:54<337:48:49,  7.73s/it]
 30%|███       | 67636/225000 [137:43:01<326:51:40,  7.48s/it]
 30%|███       | 67637/225000 [137:43:06<284:54:15,  6.52s/it]
 30%|███       | 67638/225000 [137:43:11<263:44:53,  6.03s/it]
 30%|███       | 67639/225000 [137:43:20<312:01:31,  7.14s/it]
 30%|███       | 67640/225000 [137:43:27<309:06:32,  7.07s/it]
 30%|███       | 67641/225000 [137:43:31<271:05:55,  6.20s/it]
 30%|███       | 67642/225000 [137:43:42<329:15:25,  7.53s/it]
 30%|███       | 67643/225000 [137:43:46<282:26:39,  6.46s/it]
 30%|███       | 67644/225000 [137:43:55<318:12:44,  7.28s/it]
 30%|███       | 67645/225000 [137:43:59<272:50:37,  6.24s/it]
 30%|███       | 67646/225000 [137:44:07<297:25:32,  6.80s/it]
 30%|███       | 67647/225000 [137:44:15<315:23:26,  7.22s/it]
 30%|███       | 67648/225000 [137:44:20<280:47:41,  6.42s/it]
 30%|███       | 67649/225000 [137:44:26<281:17:23,  6.44s/it]
 30%|███       | 67650/225000 [137:44:32<267:18:49,  6.12s/it]
                                                              
{'loss': '0.7258', 'grad_norm': '0.7965', 'learning_rate': '4.165e-05', 'epoch': '0.3007', 'num_input_tokens_seen': 228779040, 'train_runtime': '4.959e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67650/225000 [137:44:32<267:18:49,  6.12s/it]
 30%|███       | 67651/225000 [137:44:39<285:38:14,  6.54s/it]
 30%|███       | 67652/225000 [137:44:47<302:19:49,  6.92s/it]
 30%|███       | 67653/225000 [137:44:56<334:24:04,  7.65s/it]
 30%|███       | 67654/225000 [137:45:04<339:28:06,  7.77s/it]
 30%|███       | 67655/225000 [137:45:09<301:22:56,  6.90s/it]
 30%|███       | 67656/225000 [137:45:19<343:21:37,  7.86s/it]
 30%|███       | 67657/225000 [137:45:23<294:09:09,  6.73s/it]
 30%|███       | 67658/225000 [137:45:28<268:49:03,  6.15s/it]
 30%|███       | 67659/225000 [137:45:33<250:21:42,  5.73s/it]
 30%|███       | 67660/225000 [137:45:46<342:36:45,  7.84s/it]
 30%|███       | 67661/225000 [137:45:52<321:55:26,  7.37s/it]
 30%|███       | 67662/225000 [137:45:57<289:41:34,  6.63s/it]
 30%|███       | 67663/225000 [137:46:02<273:38:51,  6.26s/it]
 30%|███       | 67664/225000 [137:46:07<256:19:50,  5.87s/it]
 30%|███       | 67665/225000 [137:46:16<293:23:19,  6.71s/it]
 30%|███       | 67666/225000 [137:46:27<344:07:20,  7.87s/it]
 30%|███       | 67667/225000 [137:46:31<298:49:59,  6.84s/it]
 30%|███       | 67668/225000 [137:46:40<325:00:24,  7.44s/it]
 30%|███       | 67669/225000 [137:46:49<354:37:34,  8.11s/it]
 30%|███       | 67670/225000 [137:46:53<288:31:46,  6.60s/it]
 30%|███       | 67671/225000 [137:47:01<313:37:46,  7.18s/it]
 30%|███       | 67672/225000 [137:47:05<271:10:52,  6.21s/it]
 30%|███       | 67673/225000 [137:47:13<298:17:40,  6.83s/it]
 30%|███       | 67674/225000 [137:47:22<319:01:48,  7.30s/it]
 30%|███       | 67675/225000 [137:47:26<275:02:28,  6.29s/it]
 30%|███       | 67676/225000 [137:47:35<314:33:22,  7.20s/it]
 30%|███       | 67677/225000 [137:47:39<268:12:28,  6.14s/it]
 30%|███       | 67678/225000 [137:47:44<253:18:16,  5.80s/it]
 30%|███       | 67679/225000 [137:47:49<246:01:51,  5.63s/it]
 30%|███       | 67680/225000 [137:47:54<240:39:08,  5.51s/it]
 30%|███       | 67681/225000 [137:47:59<238:22:48,  5.45s/it]
 30%|███       | 67682/225000 [137:48:06<253:03:11,  5.79s/it]
 30%|███       | 67683/225000 [137:48:19<353:52:56,  8.10s/it]
 30%|███       | 67684/225000 [137:48:24<303:25:00,  6.94s/it]
 30%|███       | 67685/225000 [137:48:33<338:17:28,  7.74s/it]
 30%|███       | 67686/225000 [137:48:38<303:24:02,  6.94s/it]
 30%|███       | 67687/225000 [137:48:43<277:52:43,  6.36s/it]
 30%|███       | 67688/225000 [137:48:49<262:43:12,  6.01s/it]
 30%|███       | 67689/225000 [137:48:53<246:25:40,  5.64s/it]
 30%|███       | 67690/225000 [137:49:03<299:02:43,  6.84s/it]
 30%|███       | 67691/225000 [137:49:12<321:31:57,  7.36s/it]
 30%|███       | 67692/225000 [137:49:18<303:56:55,  6.96s/it]
 30%|███       | 67693/225000 [137:49:26<319:48:14,  7.32s/it]
 30%|███       | 67694/225000 [137:49:34<331:34:08,  7.59s/it]
 30%|███       | 67695/225000 [137:49:38<288:57:55,  6.61s/it]
 30%|███       | 67696/225000 [137:49:43<260:22:22,  5.96s/it]
 30%|███       | 67697/225000 [137:49:51<288:21:15,  6.60s/it]
 30%|███       | 67698/225000 [137:49:59<306:22:26,  7.01s/it]
 30%|███       | 67699/225000 [137:50:03<275:29:16,  6.30s/it]
 30%|███       | 67700/225000 [137:50:11<295:57:24,  6.77s/it]
                                                              
{'loss': '0.7201', 'grad_norm': '0.731', 'learning_rate': '4.164e-05', 'epoch': '0.3009', 'num_input_tokens_seen': 228929504, 'train_runtime': '4.962e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67700/225000 [137:50:11<295:57:24,  6.77s/it]
 30%|███       | 67701/225000 [137:50:17<280:08:06,  6.41s/it]
 30%|███       | 67702/225000 [137:50:23<276:16:06,  6.32s/it]
 30%|███       | 67703/225000 [137:50:31<302:45:31,  6.93s/it]
 30%|███       | 67704/225000 [137:50:36<276:54:29,  6.34s/it]
 30%|███       | 67705/225000 [137:50:42<266:24:40,  6.10s/it]
 30%|███       | 67706/225000 [137:50:47<259:58:25,  5.95s/it]
 30%|███       | 67707/225000 [137:51:00<349:00:49,  7.99s/it]
 30%|███       | 67708/225000 [137:51:05<305:28:10,  6.99s/it]
 30%|███       | 67709/225000 [137:51:09<274:10:30,  6.28s/it]
 30%|███       | 67710/225000 [137:51:21<345:03:56,  7.90s/it]
 30%|███       | 67711/225000 [137:51:29<340:22:23,  7.79s/it]
 30%|███       | 67712/225000 [137:51:33<292:03:15,  6.68s/it]
 30%|███       | 67713/225000 [137:51:37<253:44:37,  5.81s/it]
 30%|███       | 67714/225000 [137:51:47<311:32:23,  7.13s/it]
 30%|███       | 67715/225000 [137:51:54<312:39:56,  7.16s/it]
 30%|███       | 67716/225000 [137:51:59<283:13:40,  6.48s/it]
 30%|███       | 67717/225000 [137:52:03<253:02:12,  5.79s/it]
 30%|███       | 67718/225000 [137:52:13<313:10:20,  7.17s/it]
 30%|███       | 67719/225000 [137:52:18<278:45:25,  6.38s/it]
 30%|███       | 67720/225000 [137:52:23<255:36:13,  5.85s/it]
 30%|███       | 67721/225000 [137:52:27<242:58:46,  5.56s/it]
 30%|███       | 67722/225000 [137:52:35<271:43:06,  6.22s/it]
 30%|███       | 67723/225000 [137:52:47<339:06:23,  7.76s/it]
 30%|███       | 67724/225000 [137:52:51<299:02:25,  6.84s/it]
 30%|███       | 67725/225000 [137:52:59<308:22:54,  7.06s/it]
 30%|███       | 67726/225000 [137:53:04<287:22:57,  6.58s/it]
 30%|███       | 67727/225000 [137:53:10<272:46:13,  6.24s/it]
 30%|███       | 67728/225000 [137:53:15<256:16:42,  5.87s/it]
 30%|███       | 67729/225000 [137:53:24<297:50:07,  6.82s/it]
 30%|███       | 67730/225000 [137:53:34<344:15:48,  7.88s/it]
 30%|███       | 67731/225000 [137:53:38<290:29:24,  6.65s/it]
 30%|███       | 67732/225000 [137:53:50<357:49:58,  8.19s/it]
 30%|███       | 67733/225000 [137:53:54<301:39:36,  6.91s/it]
 30%|███       | 67734/225000 [137:54:01<309:04:03,  7.07s/it]
 30%|███       | 67735/225000 [137:54:08<304:46:46,  6.98s/it]
 30%|███       | 67736/225000 [137:54:16<319:17:06,  7.31s/it]
 30%|███       | 67737/225000 [137:54:20<280:28:00,  6.42s/it]
 30%|███       | 67738/225000 [137:54:25<256:06:32,  5.86s/it]
 30%|███       | 67739/225000 [137:54:33<288:19:13,  6.60s/it]
 30%|███       | 67740/225000 [137:54:41<306:26:28,  7.02s/it]
 30%|███       | 67741/225000 [137:54:45<269:05:32,  6.16s/it]
 30%|███       | 67742/225000 [137:54:55<320:46:31,  7.34s/it]
 30%|███       | 67743/225000 [137:55:02<311:45:21,  7.14s/it]
 30%|███       | 67744/225000 [137:55:06<274:10:26,  6.28s/it]
 30%|███       | 67745/225000 [137:55:16<318:11:12,  7.28s/it]
 30%|███       | 67746/225000 [137:55:22<307:09:16,  7.03s/it]
 30%|███       | 67747/225000 [137:55:26<266:34:50,  6.10s/it]
 30%|███       | 67748/225000 [137:55:31<244:39:49,  5.60s/it]
 30%|███       | 67749/225000 [137:55:43<327:00:40,  7.49s/it]
 30%|███       | 67750/225000 [137:55:48<298:17:46,  6.83s/it]
                                                              
{'loss': '0.7231', 'grad_norm': '0.7841', 'learning_rate': '4.162e-05', 'epoch': '0.3011', 'num_input_tokens_seen': 229081864, 'train_runtime': '4.965e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 67750/225000 [137:55:48<298:17:46,  6.83s/it]
 30%|███       | 67751/225000 [137:55:54<286:51:41,  6.57s/it]
 30%|███       | 67752/225000 [137:55:59<264:11:49,  6.05s/it]
 30%|███       | 67753/225000 [137:56:04<248:19:05,  5.68s/it]
 30%|███       | 67754/225000 [137:56:11<269:38:31,  6.17s/it]
 30%|███       | 67755/225000 [137:56:21<324:22:12,  7.43s/it]
 30%|███       | 67756/225000 [137:56:27<296:10:00,  6.78s/it]
 30%|███       | 67757/225000 [137:56:33<296:34:10,  6.79s/it]
 30%|███       | 67758/225000 [137:56:43<328:25:50,  7.52s/it]
 30%|███       | 67759/225000 [137:56:49<311:55:37,  7.14s/it]
 30%|███       | 67760/225000 [137:56:53<268:29:59,  6.15s/it]
 30%|███       | 67761/225000 [137:57:03<329:55:57,  7.55s/it]
 30%|███       | 67762/225000 [137:57:08<286:42:19,  6.56s/it]
 30%|███       | 67763/225000 [137:57:13<273:39:30,  6.27s/it]
 30%|███       | 67764/225000 [137:57:23<318:48:03,  7.30s/it]
 30%|███       | 67765/225000 [137:57:28<293:04:32,  6.71s/it]
 30%|███       | 67766/225000 [137:57:32<257:52:59,  5.90s/it]
 30%|███       | 67767/225000 [137:57:37<244:24:13,  5.60s/it]
 30%|███       | 67768/225000 [137:57:49<324:23:30,  7.43s/it]
 30%|███       | 67769/225000 [137:57:54<297:33:41,  6.81s/it]
 30%|███       | 67770/225000 [137:57:59<265:37:12,  6.08s/it]
 30%|███       | 67771/225000 [137:58:04<260:14:01,  5.96s/it]
 30%|███       | 67772/225000 [137:58:16<336:50:15,  7.71s/it]
 30%|███       | 67773/225000 [137:58:20<282:46:50,  6.47s/it]
 30%|███       | 67774/225000 [137:58:28<306:28:59,  7.02s/it]
 30%|███       | 67775/225000 [137:58:37<328:23:35,  7.52s/it]
 30%|███       | 67776/225000 [137:58:40<278:02:47,  6.37s/it]
 30%|███       | 67777/225000 [137:58:49<301:04:11,  6.89s/it]
 30%|███       | 67778/225000 [137:58:53<270:14:39,  6.19s/it]
 30%|███       | 67779/225000 [137:58:59<260:27:58,  5.96s/it]
 30%|███       | 67780/225000 [137:59:10<327:02:30,  7.49s/it]
 30%|███       | 67781/225000 [137:59:14<282:03:16,  6.46s/it]
 30%|███       | 67782/225000 [137:59:23<317:26:23,  7.27s/it]
 30%|███       | 67783/225000 [137:59:27<278:25:31,  6.38s/it]
 30%|███       | 67784/225000 [137:59:31<250:31:19,  5.74s/it]
 30%|███       | 67785/225000 [137:59:41<296:54:56,  6.80s/it]
 30%|███       | 67786/225000 [137:59:47<287:18:47,  6.58s/it]
 30%|███       | 67787/225000 [137:59:53<280:43:27,  6.43s/it]
 30%|███       | 67788/225000 [138:00:02<317:19:58,  7.27s/it]
 30%|███       | 67789/225000 [138:00:07<287:10:07,  6.58s/it]
 30%|███       | 67790/225000 [138:00:12<263:37:19,  6.04s/it]
 30%|███       | 67791/225000 [138:00:16<246:08:51,  5.64s/it]
 30%|███       | 67792/225000 [138:00:22<246:12:48,  5.64s/it]
 30%|███       | 67793/225000 [138:00:27<234:14:30,  5.36s/it]
 30%|███       | 67794/225000 [138:00:39<321:39:54,  7.37s/it]
 30%|███       | 67795/225000 [138:00:46<318:01:32,  7.28s/it]
 30%|███       | 67796/225000 [138:00:51<287:43:14,  6.59s/it]
 30%|███       | 67797/225000 [138:00:59<310:19:39,  7.11s/it]
 30%|███       | 67798/225000 [138:01:04<278:57:59,  6.39s/it]
 30%|███       | 67799/225000 [138:01:09<263:27:09,  6.03s/it]
 30%|███       | 67800/225000 [138:01:14<247:55:20,  5.68s/it]
                                                              
{'loss': '0.7388', 'grad_norm': '0.7193', 'learning_rate': '4.161e-05', 'epoch': '0.3013', 'num_input_tokens_seen': 229233088, 'train_runtime': '4.969e+05', 'train_tokens_per_second': '461.4'}

 30%|███       | 67800/225000 [138:01:14<247:55:20,  5.68s/it]
 30%|███       | 67801/225000 [138:01:19<234:53:56,  5.38s/it]
 30%|███       | 67802/225000 [138:01:32<335:06:26,  7.67s/it]
 30%|███       | 67803/225000 [138:01:38<320:57:43,  7.35s/it]
 30%|███       | 67804/225000 [138:01:42<279:24:59,  6.40s/it]
 30%|███       | 67805/225000 [138:01:53<335:48:13,  7.69s/it]
 30%|███       | 67806/225000 [138:01:58<303:18:41,  6.95s/it]
 30%|███       | 67807/225000 [138:02:02<265:45:20,  6.09s/it]
 30%|███       | 67808/225000 [138:02:11<295:55:18,  6.78s/it]
 30%|███       | 67809/225000 [138:02:19<310:03:56,  7.10s/it]
 30%|███       | 67810/225000 [138:02:23<273:23:58,  6.26s/it]
 30%|███       | 67811/225000 [138:02:34<335:43:20,  7.69s/it]
 30%|███       | 67812/225000 [138:02:38<282:55:16,  6.48s/it]
 30%|███       | 67813/225000 [138:02:47<319:42:45,  7.32s/it]
 30%|███       | 67814/225000 [138:02:52<294:37:39,  6.75s/it]
 30%|███       | 67815/225000 [138:02:58<278:24:09,  6.38s/it]
 30%|███       | 67816/225000 [138:03:03<256:37:06,  5.88s/it]
 30%|███       | 67817/225000 [138:03:08<250:20:20,  5.73s/it]
 30%|███       | 67818/225000 [138:03:21<348:09:02,  7.97s/it]
 30%|███       | 67819/225000 [138:03:27<318:10:38,  7.29s/it]
 30%|███       | 67820/225000 [138:03:35<334:54:30,  7.67s/it]
 30%|███       | 67821/225000 [138:03:39<284:38:19,  6.52s/it]
 30%|███       | 67822/225000 [138:03:49<320:26:18,  7.34s/it]
 30%|███       | 67823/225000 [138:03:53<282:09:20,  6.46s/it]
 30%|███       | 67824/225000 [138:03:57<254:08:52,  5.82s/it]
 30%|███       | 67825/225000 [138:04:07<308:45:53,  7.07s/it]
 30%|███       | 67826/225000 [138:04:13<286:58:43,  6.57s/it]
 30%|███       | 67827/225000 [138:04:17<253:57:55,  5.82s/it]
 30%|███       | 67828/225000 [138:04:27<314:29:50,  7.20s/it]
 30%|███       | 67829/225000 [138:04:34<309:09:06,  7.08s/it]
 30%|███       | 67830/225000 [138:04:38<275:33:26,  6.31s/it]
 30%|███       | 67831/225000 [138:04:48<320:53:14,  7.35s/it]
 30%|███       | 67832/225000 [138:04:53<287:34:12,  6.59s/it]
 30%|███       | 67833/225000 [138:04:58<261:36:28,  5.99s/it]
 30%|███       | 67834/225000 [138:05:10<347:34:42,  7.96s/it]
 30%|███       | 67835/225000 [138:05:14<287:47:46,  6.59s/it]
 30%|███       | 67836/225000 [138:05:24<333:53:40,  7.65s/it]
 30%|███       | 67837/225000 [138:05:30<315:13:32,  7.22s/it]
 30%|███       | 67838/225000 [138:05:34<274:54:12,  6.30s/it]
 30%|███       | 67839/225000 [138:05:40<270:47:52,  6.20s/it]
 30%|███       | 67840/225000 [138:05:49<312:27:49,  7.16s/it]
 30%|███       | 67841/225000 [138:05:57<314:27:41,  7.20s/it]
 30%|███       | 67842/225000 [138:06:01<270:09:37,  6.19s/it]
 30%|███       | 67843/225000 [138:06:06<264:08:06,  6.05s/it]
 30%|███       | 67844/225000 [138:06:12<255:04:37,  5.84s/it]
 30%|███       | 67845/225000 [138:06:17<253:48:50,  5.81s/it]
 30%|███       | 67846/225000 [138:06:23<249:43:13,  5.72s/it]
 30%|███       | 67847/225000 [138:06:30<266:39:01,  6.11s/it]
 30%|███       | 67848/225000 [138:06:43<363:25:00,  8.33s/it]
 30%|███       | 67849/225000 [138:06:47<302:58:56,  6.94s/it]
 30%|███       | 67850/225000 [138:06:51<265:33:18,  6.08s/it]
                                                              
{'loss': '0.7236', 'grad_norm': '0.7438', 'learning_rate': '4.16e-05', 'epoch': '0.3016', 'num_input_tokens_seen': 229384256, 'train_runtime': '4.972e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 67850/225000 [138:06:51<265:33:18,  6.08s/it]
 30%|███       | 67851/225000 [138:07:01<314:36:35,  7.21s/it]
 30%|███       | 67852/225000 [138:07:08<311:28:41,  7.14s/it]
 30%|███       | 67853/225000 [138:07:16<317:05:20,  7.26s/it]
 30%|███       | 67854/225000 [138:07:21<294:03:00,  6.74s/it]
 30%|███       | 67855/225000 [138:07:25<261:59:35,  6.00s/it]
 30%|███       | 67856/225000 [138:07:31<253:20:19,  5.80s/it]
 30%|███       | 67857/225000 [138:07:35<238:05:31,  5.45s/it]
 30%|███       | 67858/225000 [138:07:41<234:30:40,  5.37s/it]
 30%|███       | 67859/225000 [138:07:49<278:48:12,  6.39s/it]
 30%|███       | 67860/225000 [138:08:02<367:20:22,  8.42s/it]
 30%|███       | 67861/225000 [138:08:06<305:08:51,  6.99s/it]
 30%|███       | 67862/225000 [138:08:13<300:41:16,  6.89s/it]
 30%|███       | 67863/225000 [138:08:19<298:19:41,  6.83s/it]
 30%|███       | 67864/225000 [138:08:26<288:14:14,  6.60s/it]
 30%|███       | 67865/225000 [138:08:37<352:53:13,  8.08s/it]
 30%|███       | 67866/225000 [138:08:41<294:29:43,  6.75s/it]
 30%|███       | 67867/225000 [138:08:48<295:30:59,  6.77s/it]
 30%|███       | 67868/225000 [138:08:55<298:54:07,  6.85s/it]
 30%|███       | 67869/225000 [138:09:00<284:20:03,  6.51s/it]
 30%|███       | 67870/225000 [138:09:07<281:01:37,  6.44s/it]
 30%|███       | 67871/225000 [138:09:11<249:34:02,  5.72s/it]
 30%|███       | 67872/225000 [138:09:21<309:26:05,  7.09s/it]
 30%|███       | 67873/225000 [138:09:26<286:32:58,  6.57s/it]
 30%|███       | 67874/225000 [138:09:31<268:23:34,  6.15s/it]
 30%|███       | 67875/225000 [138:09:38<268:20:39,  6.15s/it]
 30%|███       | 67876/225000 [138:09:44<269:16:06,  6.17s/it]
 30%|███       | 67877/225000 [138:09:49<254:28:26,  5.83s/it]
 30%|███       | 67878/225000 [138:09:55<259:02:24,  5.94s/it]
 30%|███       | 67879/225000 [138:10:00<252:13:10,  5.78s/it]
 30%|███       | 67880/225000 [138:10:06<251:54:53,  5.77s/it]
 30%|███       | 67881/225000 [138:10:14<278:33:49,  6.38s/it]
 30%|███       | 67882/225000 [138:10:27<367:24:59,  8.42s/it]
 30%|███       | 67883/225000 [138:10:34<341:40:41,  7.83s/it]
 30%|███       | 67884/225000 [138:10:40<322:33:25,  7.39s/it]
 30%|███       | 67885/225000 [138:10:44<282:13:45,  6.47s/it]
 30%|███       | 67886/225000 [138:10:56<354:51:07,  8.13s/it]
 30%|███       | 67887/225000 [138:11:02<322:53:32,  7.40s/it]
 30%|███       | 67888/225000 [138:11:06<284:57:00,  6.53s/it]
 30%|███       | 67889/225000 [138:11:16<320:28:14,  7.34s/it]
 30%|███       | 67890/225000 [138:11:20<276:29:27,  6.34s/it]
 30%|███       | 67891/225000 [138:11:29<315:36:41,  7.23s/it]
 30%|███       | 67892/225000 [138:11:35<296:52:39,  6.80s/it]
 30%|███       | 67893/225000 [138:11:41<293:32:39,  6.73s/it]
 30%|███       | 67894/225000 [138:11:45<250:35:26,  5.74s/it]
 30%|███       | 67895/225000 [138:11:56<320:29:17,  7.34s/it]
 30%|███       | 67896/225000 [138:12:01<288:18:36,  6.61s/it]
 30%|███       | 67897/225000 [138:12:08<299:32:37,  6.86s/it]
 30%|███       | 67898/225000 [138:12:12<264:23:15,  6.06s/it]
 30%|███       | 67899/225000 [138:12:24<334:27:43,  7.66s/it]
 30%|███       | 67900/225000 [138:12:28<286:06:35,  6.56s/it]
                                                              
{'loss': '0.7041', 'grad_norm': '0.8956', 'learning_rate': '4.158e-05', 'epoch': '0.3018', 'num_input_tokens_seen': 229535448, 'train_runtime': '4.975e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 67900/225000 [138:12:28<286:06:35,  6.56s/it]
 30%|███       | 67901/225000 [138:12:37<324:00:36,  7.42s/it]
 30%|███       | 67902/225000 [138:12:41<271:08:32,  6.21s/it]
 30%|███       | 67903/225000 [138:12:45<246:16:17,  5.64s/it]
 30%|███       | 67904/225000 [138:12:51<251:02:11,  5.75s/it]
 30%|███       | 67905/225000 [138:12:57<253:14:59,  5.80s/it]
 30%|███       | 67906/225000 [138:13:10<345:00:14,  7.91s/it]
 30%|███       | 67907/225000 [138:13:16<321:29:01,  7.37s/it]
 30%|███       | 67908/225000 [138:13:21<299:25:12,  6.86s/it]
 30%|███       | 67909/225000 [138:13:26<271:03:06,  6.21s/it]
 30%|███       | 67910/225000 [138:13:31<259:19:39,  5.94s/it]
 30%|███       | 67911/225000 [138:13:37<248:09:30,  5.69s/it]
 30%|███       | 67912/225000 [138:13:42<245:25:03,  5.62s/it]
 30%|███       | 67913/225000 [138:13:49<262:06:08,  6.01s/it]
 30%|███       | 67914/225000 [138:14:02<355:48:10,  8.15s/it]
 30%|███       | 67915/225000 [138:14:08<321:43:22,  7.37s/it]
 30%|███       | 67916/225000 [138:14:13<290:16:31,  6.65s/it]
 30%|███       | 67917/225000 [138:14:17<265:40:40,  6.09s/it]
 30%|███       | 67918/225000 [138:14:29<341:05:25,  7.82s/it]
 30%|███       | 67919/225000 [138:14:34<296:19:44,  6.79s/it]
 30%|███       | 67920/225000 [138:14:38<267:14:39,  6.12s/it]
 30%|███       | 67921/225000 [138:14:50<339:00:47,  7.77s/it]
 30%|███       | 67922/225000 [138:14:54<292:51:09,  6.71s/it]
 30%|███       | 67923/225000 [138:14:59<269:52:29,  6.19s/it]
 30%|███       | 67924/225000 [138:15:04<248:28:01,  5.69s/it]
 30%|███       | 67925/225000 [138:15:09<239:33:18,  5.49s/it]
 30%|███       | 67926/225000 [138:15:21<326:12:56,  7.48s/it]
 30%|███       | 67927/225000 [138:15:25<285:10:57,  6.54s/it]
 30%|███       | 67928/225000 [138:15:29<254:57:10,  5.84s/it]
 30%|███       | 67929/225000 [138:15:40<316:51:43,  7.26s/it]
 30%|███       | 67930/225000 [138:15:45<288:16:57,  6.61s/it]
 30%|███       | 67931/225000 [138:15:50<266:06:47,  6.10s/it]
 30%|███       | 67932/225000 [138:15:56<260:09:52,  5.96s/it]
 30%|███       | 67933/225000 [138:16:08<339:12:52,  7.77s/it]
 30%|███       | 67934/225000 [138:16:11<288:23:19,  6.61s/it]
 30%|███       | 67935/225000 [138:16:21<325:03:05,  7.45s/it]
 30%|███       | 67936/225000 [138:16:25<288:52:06,  6.62s/it]
 30%|███       | 67937/225000 [138:16:34<314:58:29,  7.22s/it]
 30%|███       | 67938/225000 [138:16:40<300:38:43,  6.89s/it]
 30%|███       | 67939/225000 [138:16:46<281:49:40,  6.46s/it]
 30%|███       | 67940/225000 [138:16:54<312:02:18,  7.15s/it]
 30%|███       | 67941/225000 [138:16:59<271:57:52,  6.23s/it]
 30%|███       | 67942/225000 [138:17:10<343:32:22,  7.87s/it]
 30%|███       | 67943/225000 [138:17:15<297:16:17,  6.81s/it]
 30%|███       | 67944/225000 [138:17:24<334:09:47,  7.66s/it]
 30%|███       | 67945/225000 [138:17:28<284:49:25,  6.53s/it]
 30%|███       | 67946/225000 [138:17:37<309:50:44,  7.10s/it]
 30%|███       | 67947/225000 [138:17:41<279:02:04,  6.40s/it]
 30%|███       | 67948/225000 [138:17:46<250:31:33,  5.74s/it]
 30%|███       | 67949/225000 [138:17:58<337:53:46,  7.75s/it]
 30%|███       | 67950/225000 [138:18:02<291:03:03,  6.67s/it]
                                                              
{'loss': '0.7309', 'grad_norm': '1.91', 'learning_rate': '4.157e-05', 'epoch': '0.302', 'num_input_tokens_seen': 229685360, 'train_runtime': '4.979e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 67950/225000 [138:18:02<291:03:03,  6.67s/it]
 30%|███       | 67951/225000 [138:18:06<257:43:23,  5.91s/it]
 30%|███       | 67952/225000 [138:18:14<284:50:51,  6.53s/it]
 30%|███       | 67953/225000 [138:18:23<308:48:17,  7.08s/it]
 30%|███       | 67954/225000 [138:18:27<277:13:41,  6.35s/it]
 30%|███       | 67955/225000 [138:18:33<269:11:21,  6.17s/it]
 30%|███       | 67956/225000 [138:18:39<263:35:32,  6.04s/it]
 30%|███       | 67957/225000 [138:18:44<253:09:24,  5.80s/it]
 30%|███       | 67958/225000 [138:18:49<242:54:24,  5.57s/it]
 30%|███       | 67959/225000 [138:19:01<325:34:56,  7.46s/it]
 30%|███       | 67960/225000 [138:19:09<331:03:04,  7.59s/it]
 30%|███       | 67961/225000 [138:19:13<290:25:50,  6.66s/it]
 30%|███       | 67962/225000 [138:19:18<267:36:08,  6.13s/it]
 30%|███       | 67963/225000 [138:19:27<305:13:30,  7.00s/it]
 30%|███       | 67964/225000 [138:19:35<321:04:32,  7.36s/it]
 30%|███       | 67965/225000 [138:19:43<321:40:11,  7.37s/it]
 30%|███       | 67966/225000 [138:19:48<288:26:18,  6.61s/it]
 30%|███       | 67967/225000 [138:19:55<296:43:22,  6.80s/it]
 30%|███       | 67968/225000 [138:20:01<284:12:06,  6.52s/it]
 30%|███       | 67969/225000 [138:20:05<258:15:58,  5.92s/it]
 30%|███       | 67970/225000 [138:20:11<259:05:15,  5.94s/it]
 30%|███       | 67971/225000 [138:20:17<255:51:51,  5.87s/it]
 30%|███       | 67972/225000 [138:20:23<254:38:12,  5.84s/it]
 30%|███       | 67973/225000 [138:20:28<252:12:45,  5.78s/it]
 30%|███       | 67974/225000 [138:20:34<246:33:07,  5.65s/it]
 30%|███       | 67975/225000 [138:20:39<246:26:07,  5.65s/it]
 30%|███       | 67976/225000 [138:20:50<312:19:25,  7.16s/it]
 30%|███       | 67977/225000 [138:21:01<367:32:56,  8.43s/it]
 30%|███       | 67978/225000 [138:21:05<309:57:41,  7.11s/it]
 30%|███       | 67979/225000 [138:21:10<272:34:35,  6.25s/it]
 30%|███       | 67980/225000 [138:21:21<334:34:50,  7.67s/it]
 30%|███       | 67981/225000 [138:21:24<283:22:24,  6.50s/it]
 30%|███       | 67982/225000 [138:21:34<324:27:59,  7.44s/it]
 30%|███       | 67983/225000 [138:21:38<280:01:15,  6.42s/it]
 30%|███       | 67984/225000 [138:21:43<264:47:59,  6.07s/it]
 30%|███       | 67985/225000 [138:21:49<255:36:24,  5.86s/it]
 30%|███       | 67986/225000 [138:21:55<266:33:17,  6.11s/it]
 30%|███       | 67987/225000 [138:22:09<359:33:48,  8.24s/it]
 30%|███       | 67988/225000 [138:22:15<330:10:12,  7.57s/it]
 30%|███       | 67989/225000 [138:22:20<304:55:11,  6.99s/it]
 30%|███       | 67990/225000 [138:22:27<306:59:59,  7.04s/it]
 30%|███       | 67991/225000 [138:22:32<278:10:28,  6.38s/it]
 30%|███       | 67992/225000 [138:22:37<252:52:26,  5.80s/it]
 30%|███       | 67993/225000 [138:22:44<274:20:45,  6.29s/it]
 30%|███       | 67994/225000 [138:22:53<308:00:38,  7.06s/it]
 30%|███       | 67995/225000 [138:22:58<276:26:29,  6.34s/it]
 30%|███       | 67996/225000 [138:23:06<297:16:06,  6.82s/it]
 30%|███       | 67997/225000 [138:23:10<260:49:10,  5.98s/it]
 30%|███       | 67998/225000 [138:23:20<315:00:57,  7.22s/it]
 30%|███       | 67999/225000 [138:23:26<304:32:58,  6.98s/it]
 30%|███       | 68000/225000 [138:23:33<297:01:55,  6.81s/it]
                                                              
{'loss': '0.712', 'grad_norm': '0.8741', 'learning_rate': '4.156e-05', 'epoch': '0.3022', 'num_input_tokens_seen': 229835408, 'train_runtime': '4.982e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68000/225000 [138:23:33<297:01:55,  6.81s/it]
 30%|███       | 68001/225000 [138:23:37<263:34:43,  6.04s/it]
 30%|███       | 68002/225000 [138:23:49<343:53:34,  7.89s/it]
 30%|███       | 68003/225000 [138:23:53<294:14:18,  6.75s/it]
 30%|███       | 68004/225000 [138:23:57<260:47:10,  5.98s/it]
 30%|███       | 68005/225000 [138:24:09<331:43:07,  7.61s/it]
 30%|███       | 68006/225000 [138:24:12<279:32:41,  6.41s/it]
 30%|███       | 68007/225000 [138:24:19<279:14:27,  6.40s/it]
 30%|███       | 68008/225000 [138:24:27<297:31:42,  6.82s/it]
 30%|███       | 68009/225000 [138:24:34<304:45:46,  6.99s/it]
 30%|███       | 68010/225000 [138:24:38<264:51:33,  6.07s/it]
 30%|███       | 68011/225000 [138:24:48<321:41:12,  7.38s/it]
 30%|███       | 68012/225000 [138:24:54<298:09:11,  6.84s/it]
 30%|███       | 68013/225000 [138:24:58<260:42:56,  5.98s/it]
 30%|███       | 68014/225000 [138:25:07<307:24:10,  7.05s/it]
 30%|███       | 68015/225000 [138:25:14<300:59:02,  6.90s/it]
 30%|███       | 68016/225000 [138:25:21<297:10:22,  6.81s/it]
 30%|███       | 68017/225000 [138:25:25<263:19:20,  6.04s/it]
 30%|███       | 68018/225000 [138:25:37<342:49:32,  7.86s/it]
 30%|███       | 68019/225000 [138:25:41<294:56:43,  6.76s/it]
 30%|███       | 68020/225000 [138:25:46<267:44:14,  6.14s/it]
 30%|███       | 68021/225000 [138:25:55<305:17:12,  7.00s/it]
 30%|███       | 68022/225000 [138:25:59<269:53:06,  6.19s/it]
 30%|███       | 68023/225000 [138:26:03<240:34:04,  5.52s/it]
 30%|███       | 68024/225000 [138:26:14<318:12:59,  7.30s/it]
 30%|███       | 68025/225000 [138:26:21<313:01:45,  7.18s/it]
 30%|███       | 68026/225000 [138:26:26<282:25:02,  6.48s/it]
 30%|███       | 68027/225000 [138:26:32<267:59:55,  6.15s/it]
 30%|███       | 68028/225000 [138:26:37<257:44:23,  5.91s/it]
 30%|███       | 68029/225000 [138:26:42<244:38:32,  5.61s/it]
 30%|███       | 68030/225000 [138:26:50<279:46:42,  6.42s/it]
 30%|███       | 68031/225000 [138:27:02<348:47:06,  8.00s/it]
 30%|███       | 68032/225000 [138:27:06<294:23:09,  6.75s/it]
 30%|███       | 68033/225000 [138:27:14<309:46:51,  7.10s/it]
 30%|███       | 68034/225000 [138:27:19<281:54:13,  6.47s/it]
 30%|███       | 68035/225000 [138:27:28<326:41:45,  7.49s/it]
 30%|███       | 68036/225000 [138:27:36<325:30:45,  7.47s/it]
 30%|███       | 68037/225000 [138:27:41<291:00:18,  6.67s/it]
 30%|███       | 68038/225000 [138:27:45<265:33:01,  6.09s/it]
 30%|███       | 68039/225000 [138:27:51<255:20:29,  5.86s/it]
 30%|███       | 68040/225000 [138:27:56<252:42:20,  5.80s/it]
 30%|███       | 68041/225000 [138:28:02<255:50:40,  5.87s/it]
 30%|███       | 68042/225000 [138:28:08<251:57:08,  5.78s/it]
 30%|███       | 68043/225000 [138:28:13<246:10:00,  5.65s/it]
 30%|███       | 68044/225000 [138:28:19<245:44:56,  5.64s/it]
 30%|███       | 68045/225000 [138:28:32<344:50:59,  7.91s/it]
 30%|███       | 68046/225000 [138:28:42<370:49:06,  8.51s/it]
 30%|███       | 68047/225000 [138:28:49<350:44:11,  8.04s/it]
 30%|███       | 68048/225000 [138:28:55<318:10:07,  7.30s/it]
 30%|███       | 68049/225000 [138:29:00<293:06:09,  6.72s/it]
 30%|███       | 68050/225000 [138:29:05<277:31:59,  6.37s/it]
                                                              
{'loss': '0.6984', 'grad_norm': '0.8233', 'learning_rate': '4.154e-05', 'epoch': '0.3024', 'num_input_tokens_seen': 229986824, 'train_runtime': '4.985e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68050/225000 [138:29:05<277:31:59,  6.37s/it]
 30%|███       | 68051/225000 [138:29:11<263:34:53,  6.05s/it]
 30%|███       | 68052/225000 [138:29:21<313:30:22,  7.19s/it]
 30%|███       | 68053/225000 [138:29:31<351:04:04,  8.05s/it]
 30%|███       | 68054/225000 [138:29:34<291:55:20,  6.70s/it]
 30%|███       | 68055/225000 [138:29:44<328:06:25,  7.53s/it]
 30%|███       | 68056/225000 [138:29:50<311:49:05,  7.15s/it]
 30%|███       | 68057/225000 [138:29:56<297:47:40,  6.83s/it]
 30%|███       | 68058/225000 [138:30:02<279:41:14,  6.42s/it]
 30%|███       | 68059/225000 [138:30:09<288:36:15,  6.62s/it]
 30%|███       | 68060/225000 [138:30:13<262:56:11,  6.03s/it]
 30%|███       | 68061/225000 [138:30:18<244:34:24,  5.61s/it]
 30%|███       | 68062/225000 [138:30:30<335:55:33,  7.71s/it]
 30%|███       | 68063/225000 [138:30:35<300:32:34,  6.89s/it]
 30%|███       | 68064/225000 [138:30:41<283:35:36,  6.51s/it]
 30%|███       | 68065/225000 [138:30:47<277:10:10,  6.36s/it]
 30%|███       | 68066/225000 [138:30:58<331:04:49,  7.59s/it]
 30%|███       | 68067/225000 [138:31:01<280:08:48,  6.43s/it]
 30%|███       | 68068/225000 [138:31:07<271:41:20,  6.23s/it]
 30%|███       | 68069/225000 [138:31:18<330:08:51,  7.57s/it]
 30%|███       | 68070/225000 [138:31:23<295:05:08,  6.77s/it]
 30%|███       | 68071/225000 [138:31:28<279:58:53,  6.42s/it]
 30%|███       | 68072/225000 [138:31:34<266:51:44,  6.12s/it]
 30%|███       | 68073/225000 [138:31:47<362:50:43,  8.32s/it]
 30%|███       | 68074/225000 [138:31:52<323:26:10,  7.42s/it]
 30%|███       | 68075/225000 [138:31:59<310:05:53,  7.11s/it]
 30%|███       | 68076/225000 [138:32:07<321:58:41,  7.39s/it]
 30%|███       | 68077/225000 [138:32:12<294:15:55,  6.75s/it]
 30%|███       | 68078/225000 [138:32:22<330:41:40,  7.59s/it]
 30%|███       | 68079/225000 [138:32:30<340:42:15,  7.82s/it]
 30%|███       | 68080/225000 [138:32:34<290:46:49,  6.67s/it]
 30%|███       | 68081/225000 [138:32:40<277:32:16,  6.37s/it]
 30%|███       | 68082/225000 [138:32:48<308:47:07,  7.08s/it]
 30%|███       | 68083/225000 [138:32:54<288:37:47,  6.62s/it]
 30%|███       | 68084/225000 [138:33:00<277:31:47,  6.37s/it]
 30%|███       | 68085/225000 [138:33:05<265:22:17,  6.09s/it]
 30%|███       | 68086/225000 [138:33:11<259:52:26,  5.96s/it]
 30%|███       | 68087/225000 [138:33:17<261:38:32,  6.00s/it]
 30%|███       | 68088/225000 [138:33:24<279:39:32,  6.42s/it]
 30%|███       | 68089/225000 [138:33:36<352:34:31,  8.09s/it]
 30%|███       | 68090/225000 [138:33:40<292:58:40,  6.72s/it]
 30%|███       | 68091/225000 [138:33:49<328:35:38,  7.54s/it]
 30%|███       | 68092/225000 [138:33:57<328:28:15,  7.54s/it]
 30%|███       | 68093/225000 [138:34:02<293:17:54,  6.73s/it]
 30%|███       | 68094/225000 [138:34:07<270:56:53,  6.22s/it]
 30%|███       | 68095/225000 [138:34:11<250:17:42,  5.74s/it]
 30%|███       | 68096/225000 [138:34:21<300:51:21,  6.90s/it]
 30%|███       | 68097/225000 [138:34:29<318:24:19,  7.31s/it]
 30%|███       | 68098/225000 [138:34:34<284:52:41,  6.54s/it]
 30%|███       | 68099/225000 [138:34:39<266:54:26,  6.12s/it]
 30%|███       | 68100/225000 [138:34:44<248:56:14,  5.71s/it]
                                                              
{'loss': '0.7313', 'grad_norm': '0.7862', 'learning_rate': '4.153e-05', 'epoch': '0.3027', 'num_input_tokens_seen': 230138920, 'train_runtime': '4.989e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68100/225000 [138:34:44<248:56:14,  5.71s/it]
 30%|███       | 68101/225000 [138:34:49<241:11:35,  5.53s/it]
 30%|███       | 68102/225000 [138:34:57<272:57:25,  6.26s/it]
 30%|███       | 68103/225000 [138:35:09<354:59:58,  8.15s/it]
 30%|███       | 68104/225000 [138:35:16<336:00:34,  7.71s/it]
 30%|███       | 68105/225000 [138:35:24<331:05:27,  7.60s/it]
 30%|███       | 68106/225000 [138:35:28<290:55:40,  6.68s/it]
 30%|███       | 68107/225000 [138:35:32<258:40:27,  5.94s/it]
 30%|███       | 68108/225000 [138:35:37<242:31:40,  5.56s/it]
 30%|███       | 68109/225000 [138:35:43<242:49:33,  5.57s/it]
 30%|███       | 68110/225000 [138:35:49<250:19:03,  5.74s/it]
 30%|███       | 68111/225000 [138:36:03<357:05:38,  8.19s/it]
 30%|███       | 68112/225000 [138:36:08<321:55:05,  7.39s/it]
 30%|███       | 68113/225000 [138:36:17<341:52:27,  7.84s/it]
 30%|███       | 68114/225000 [138:36:22<301:46:21,  6.92s/it]
 30%|███       | 68115/225000 [138:36:26<271:35:22,  6.23s/it]
 30%|███       | 68116/225000 [138:36:37<328:54:21,  7.55s/it]
 30%|███       | 68117/225000 [138:36:44<318:27:22,  7.31s/it]
 30%|███       | 68118/225000 [138:36:52<331:12:12,  7.60s/it]
 30%|███       | 68119/225000 [138:36:58<303:27:46,  6.96s/it]
 30%|███       | 68120/225000 [138:37:03<285:11:34,  6.54s/it]
 30%|███       | 68121/225000 [138:37:09<270:52:38,  6.22s/it]
 30%|███       | 68122/225000 [138:37:21<355:46:46,  8.16s/it]
 30%|███       | 68123/225000 [138:37:29<348:48:25,  8.00s/it]
 30%|███       | 68124/225000 [138:37:34<309:16:49,  7.10s/it]
 30%|███       | 68125/225000 [138:37:40<291:07:11,  6.68s/it]
 30%|███       | 68126/225000 [138:37:45<272:32:56,  6.25s/it]
 30%|███       | 68127/225000 [138:37:50<253:01:30,  5.81s/it]
 30%|███       | 68128/225000 [138:37:54<240:09:18,  5.51s/it]
 30%|███       | 68129/225000 [138:38:07<327:50:07,  7.52s/it]
 30%|███       | 68130/225000 [138:38:12<294:22:48,  6.76s/it]
 30%|███       | 68131/225000 [138:38:18<283:08:20,  6.50s/it]
 30%|███       | 68132/225000 [138:38:23<273:15:00,  6.27s/it]
 30%|███       | 68133/225000 [138:38:29<268:09:48,  6.15s/it]
 30%|███       | 68134/225000 [138:38:41<342:03:10,  7.85s/it]
 30%|███       | 68135/225000 [138:38:45<288:00:56,  6.61s/it]
 30%|███       | 68136/225000 [138:38:49<262:02:11,  6.01s/it]
 30%|███       | 68137/225000 [138:39:00<324:16:57,  7.44s/it]
 30%|███       | 68138/225000 [138:39:05<297:25:21,  6.83s/it]
 30%|███       | 68139/225000 [138:39:11<282:47:16,  6.49s/it]
 30%|███       | 68140/225000 [138:39:16<266:53:28,  6.13s/it]
 30%|███       | 68141/225000 [138:39:22<255:54:12,  5.87s/it]
 30%|███       | 68142/225000 [138:39:28<255:47:01,  5.87s/it]
 30%|███       | 68143/225000 [138:39:41<350:44:45,  8.05s/it]
 30%|███       | 68144/225000 [138:39:46<308:57:16,  7.09s/it]
 30%|███       | 68145/225000 [138:39:54<322:58:20,  7.41s/it]
 30%|███       | 68146/225000 [138:39:58<287:51:05,  6.61s/it]
 30%|███       | 68147/225000 [138:40:03<264:00:19,  6.06s/it]
 30%|███       | 68148/225000 [138:40:08<246:35:14,  5.66s/it]
 30%|███       | 68149/225000 [138:40:13<237:49:21,  5.46s/it]
 30%|███       | 68150/225000 [138:40:22<282:08:06,  6.48s/it]
                                                              
{'loss': '0.7156', 'grad_norm': '0.8792', 'learning_rate': '4.151e-05', 'epoch': '0.3029', 'num_input_tokens_seen': 230290080, 'train_runtime': '4.992e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68150/225000 [138:40:22<282:08:06,  6.48s/it]
 30%|███       | 68151/225000 [138:40:31<322:53:40,  7.41s/it]
 30%|███       | 68152/225000 [138:40:38<311:08:21,  7.14s/it]
 30%|███       | 68153/225000 [138:40:44<298:31:49,  6.85s/it]
 30%|███       | 68154/225000 [138:40:48<266:35:25,  6.12s/it]
 30%|███       | 68155/225000 [138:40:54<252:15:52,  5.79s/it]
 30%|███       | 68156/225000 [138:41:01<275:36:18,  6.33s/it]
 30%|███       | 68157/225000 [138:41:12<339:21:33,  7.79s/it]
 30%|███       | 68158/225000 [138:41:17<298:19:17,  6.85s/it]
 30%|███       | 68159/225000 [138:41:22<275:01:23,  6.31s/it]
 30%|███       | 68160/225000 [138:41:27<257:47:23,  5.92s/it]
 30%|███       | 68161/225000 [138:41:41<368:01:04,  8.45s/it]
 30%|███       | 68162/225000 [138:41:46<324:00:07,  7.44s/it]
 30%|███       | 68163/225000 [138:41:52<299:02:49,  6.86s/it]
 30%|███       | 68164/225000 [138:42:00<314:31:14,  7.22s/it]
 30%|███       | 68165/225000 [138:42:07<308:31:20,  7.08s/it]
 30%|███       | 68166/225000 [138:42:14<307:36:26,  7.06s/it]
 30%|███       | 68167/225000 [138:42:19<279:15:37,  6.41s/it]
 30%|███       | 68168/225000 [138:42:27<304:30:01,  6.99s/it]
 30%|███       | 68169/225000 [138:42:32<274:48:55,  6.31s/it]
 30%|███       | 68170/225000 [138:42:37<256:14:46,  5.88s/it]
 30%|███       | 68171/225000 [138:42:44<274:42:46,  6.31s/it]
 30%|███       | 68172/225000 [138:42:54<323:15:49,  7.42s/it]
 30%|███       | 68173/225000 [138:42:58<278:45:04,  6.40s/it]
 30%|███       | 68174/225000 [138:43:06<294:16:12,  6.76s/it]
 30%|███       | 68175/225000 [138:43:12<290:10:11,  6.66s/it]
 30%|███       | 68176/225000 [138:43:18<276:11:35,  6.34s/it]
 30%|███       | 68177/225000 [138:43:24<271:36:01,  6.23s/it]
 30%|███       | 68178/225000 [138:43:29<262:09:41,  6.02s/it]
 30%|███       | 68179/225000 [138:43:35<256:56:53,  5.90s/it]
 30%|███       | 68180/225000 [138:43:40<246:21:59,  5.66s/it]
 30%|███       | 68181/225000 [138:43:45<244:32:31,  5.61s/it]
 30%|███       | 68182/225000 [138:43:51<245:34:53,  5.64s/it]
 30%|███       | 68183/225000 [138:44:03<333:57:08,  7.67s/it]
 30%|███       | 68184/225000 [138:44:10<322:44:37,  7.41s/it]
 30%|███       | 68185/225000 [138:44:15<286:20:16,  6.57s/it]
 30%|███       | 68186/225000 [138:44:24<319:28:15,  7.33s/it]
 30%|███       | 68187/225000 [138:44:34<361:49:40,  8.31s/it]
 30%|███       | 68188/225000 [138:44:39<314:09:34,  7.21s/it]
 30%|███       | 68189/225000 [138:44:44<279:08:06,  6.41s/it]
 30%|███       | 68190/225000 [138:44:49<263:34:30,  6.05s/it]
 30%|███       | 68191/225000 [138:45:01<336:22:07,  7.72s/it]
 30%|███       | 68192/225000 [138:45:05<291:09:56,  6.68s/it]
 30%|███       | 68193/225000 [138:45:12<303:55:25,  6.98s/it]
 30%|███       | 68194/225000 [138:45:17<274:32:50,  6.30s/it]
 30%|███       | 68195/225000 [138:45:22<256:41:48,  5.89s/it]
 30%|███       | 68196/225000 [138:45:27<248:04:49,  5.70s/it]
 30%|███       | 68197/225000 [138:45:35<273:28:30,  6.28s/it]
 30%|███       | 68198/225000 [138:45:47<350:23:26,  8.04s/it]
 30%|███       | 68199/225000 [138:45:52<302:40:26,  6.95s/it]
 30%|███       | 68200/225000 [138:45:56<272:51:18,  6.26s/it]
                                                              
{'loss': '0.7182', 'grad_norm': '0.7845', 'learning_rate': '4.15e-05', 'epoch': '0.3031', 'num_input_tokens_seen': 230439728, 'train_runtime': '4.996e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68200/225000 [138:45:56<272:51:18,  6.26s/it]
 30%|███       | 68201/225000 [138:46:06<317:48:47,  7.30s/it]
 30%|███       | 68202/225000 [138:46:14<326:42:18,  7.50s/it]
 30%|███       | 68203/225000 [138:46:18<283:50:28,  6.52s/it]
 30%|███       | 68204/225000 [138:46:23<265:58:25,  6.11s/it]
 30%|███       | 68205/225000 [138:46:28<248:57:17,  5.72s/it]
 30%|███       | 68206/225000 [138:46:40<334:44:21,  7.69s/it]
 30%|███       | 68207/225000 [138:46:46<310:07:37,  7.12s/it]
 30%|███       | 68208/225000 [138:46:51<279:46:46,  6.42s/it]
 30%|███       | 68209/225000 [138:47:02<335:58:19,  7.71s/it]
 30%|███       | 68210/225000 [138:47:06<290:52:56,  6.68s/it]
 30%|███       | 68211/225000 [138:47:11<264:20:26,  6.07s/it]
 30%|███       | 68212/225000 [138:47:21<322:19:16,  7.40s/it]
 30%|███       | 68213/225000 [138:47:29<326:19:09,  7.49s/it]
 30%|███       | 68214/225000 [138:47:36<325:10:58,  7.47s/it]
 30%|███       | 68215/225000 [138:47:40<280:55:10,  6.45s/it]
 30%|███       | 68216/225000 [138:47:49<310:44:13,  7.13s/it]
 30%|███       | 68217/225000 [138:47:55<293:40:13,  6.74s/it]
 30%|███       | 68218/225000 [138:48:01<289:25:08,  6.65s/it]
 30%|███       | 68219/225000 [138:48:07<273:38:32,  6.28s/it]
 30%|███       | 68220/225000 [138:48:12<260:26:37,  5.98s/it]
 30%|███       | 68221/225000 [138:48:18<258:24:10,  5.93s/it]
 30%|███       | 68222/225000 [138:48:23<253:11:18,  5.81s/it]
 30%|███       | 68223/225000 [138:48:29<251:17:41,  5.77s/it]
 30%|███       | 68224/225000 [138:48:35<254:02:49,  5.83s/it]
 30%|███       | 68225/225000 [138:48:44<296:58:04,  6.82s/it]
 30%|███       | 68226/225000 [138:48:55<352:07:32,  8.09s/it]
 30%|███       | 68227/225000 [138:49:00<306:52:19,  7.05s/it]
 30%|███       | 68228/225000 [138:49:04<269:50:33,  6.20s/it]
 30%|███       | 68229/225000 [138:49:13<305:44:31,  7.02s/it]
 30%|███       | 68230/225000 [138:49:23<343:11:50,  7.88s/it]
 30%|███       | 68231/225000 [138:49:30<339:18:26,  7.79s/it]
 30%|███       | 68232/225000 [138:49:35<302:21:19,  6.94s/it]
 30%|███       | 68233/225000 [138:49:40<276:44:12,  6.35s/it]
 30%|███       | 68234/225000 [138:49:46<262:57:28,  6.04s/it]
 30%|███       | 68235/225000 [138:49:51<248:01:59,  5.70s/it]
 30%|███       | 68236/225000 [138:49:57<263:01:37,  6.04s/it]
 30%|███       | 68237/225000 [138:50:09<340:24:07,  7.82s/it]
 30%|███       | 68238/225000 [138:50:14<297:40:25,  6.84s/it]
 30%|███       | 68239/225000 [138:50:21<304:59:53,  7.00s/it]
 30%|███       | 68240/225000 [138:50:29<318:20:44,  7.31s/it]
 30%|███       | 68241/225000 [138:50:33<276:53:25,  6.36s/it]
 30%|███       | 68242/225000 [138:50:38<256:22:42,  5.89s/it]
 30%|███       | 68243/225000 [138:50:49<320:09:00,  7.35s/it]
 30%|███       | 68244/225000 [138:50:53<272:10:18,  6.25s/it]
 30%|███       | 68245/225000 [138:51:02<317:44:57,  7.30s/it]
 30%|███       | 68246/225000 [138:51:06<273:36:55,  6.28s/it]
 30%|███       | 68247/225000 [138:51:17<333:11:15,  7.65s/it]
 30%|███       | 68248/225000 [138:51:22<297:05:13,  6.82s/it]
 30%|███       | 68249/225000 [138:51:27<275:27:05,  6.33s/it]
 30%|███       | 68250/225000 [138:51:33<264:40:50,  6.08s/it]
                                                              
{'loss': '0.7334', 'grad_norm': '0.8131', 'learning_rate': '4.149e-05', 'epoch': '0.3033', 'num_input_tokens_seen': 230589944, 'train_runtime': '4.999e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68250/225000 [138:51:33<264:40:50,  6.08s/it]
 30%|███       | 68251/225000 [138:51:38<256:58:43,  5.90s/it]
 30%|███       | 68252/225000 [138:51:43<247:32:59,  5.69s/it]
 30%|███       | 68253/225000 [138:51:50<260:58:30,  5.99s/it]
 30%|███       | 68254/225000 [138:52:01<328:26:37,  7.54s/it]
 30%|███       | 68255/225000 [138:52:05<282:24:19,  6.49s/it]
 30%|███       | 68256/225000 [138:52:14<310:04:07,  7.12s/it]
 30%|███       | 68257/225000 [138:52:20<296:03:26,  6.80s/it]
 30%|███       | 68258/225000 [138:52:28<316:27:19,  7.27s/it]
 30%|███       | 68259/225000 [138:52:32<270:53:02,  6.22s/it]
 30%|███       | 68260/225000 [138:52:41<300:41:10,  6.91s/it]
 30%|███       | 68261/225000 [138:52:49<321:07:35,  7.38s/it]
 30%|███       | 68262/225000 [138:52:56<310:51:29,  7.14s/it]
 30%|███       | 68263/225000 [138:53:01<287:20:38,  6.60s/it]
 30%|███       | 68264/225000 [138:53:06<272:25:43,  6.26s/it]
 30%|███       | 68265/225000 [138:53:11<254:12:37,  5.84s/it]
 30%|███       | 68266/225000 [138:53:24<339:33:39,  7.80s/it]
 30%|███       | 68267/225000 [138:53:30<318:30:59,  7.32s/it]
 30%|███       | 68268/225000 [138:53:34<272:48:19,  6.27s/it]
 30%|███       | 68269/225000 [138:53:42<303:35:30,  6.97s/it]
 30%|███       | 68270/225000 [138:53:47<271:23:17,  6.23s/it]
 30%|███       | 68271/225000 [138:53:56<315:10:53,  7.24s/it]
 30%|███       | 68272/225000 [138:54:04<313:22:31,  7.20s/it]
 30%|███       | 68273/225000 [138:54:09<286:58:07,  6.59s/it]
 30%|███       | 68274/225000 [138:54:15<284:31:57,  6.54s/it]
 30%|███       | 68275/225000 [138:54:21<279:51:56,  6.43s/it]
 30%|███       | 68276/225000 [138:54:28<283:17:10,  6.51s/it]
 30%|███       | 68277/225000 [138:54:33<268:16:17,  6.16s/it]
 30%|███       | 68278/225000 [138:54:43<309:51:37,  7.12s/it]
 30%|███       | 68279/225000 [138:54:48<291:34:46,  6.70s/it]
 30%|███       | 68280/225000 [138:54:54<274:10:21,  6.30s/it]
 30%|███       | 68281/225000 [138:54:59<263:58:20,  6.06s/it]
 30%|███       | 68282/225000 [138:55:05<258:26:17,  5.94s/it]
 30%|███       | 68283/225000 [138:55:11<263:31:24,  6.05s/it]
 30%|███       | 68284/225000 [138:55:23<332:56:24,  7.65s/it]
 30%|███       | 68285/225000 [138:55:27<289:18:16,  6.65s/it]
 30%|███       | 68286/225000 [138:55:31<252:47:31,  5.81s/it]
 30%|███       | 68287/225000 [138:55:36<250:54:48,  5.76s/it]
 30%|███       | 68288/225000 [138:55:42<246:15:51,  5.66s/it]
 30%|███       | 68289/225000 [138:55:53<316:39:25,  7.27s/it]
 30%|███       | 68290/225000 [138:56:00<312:31:28,  7.18s/it]
 30%|███       | 68291/225000 [138:56:04<267:49:53,  6.15s/it]
 30%|███       | 68292/225000 [138:56:15<335:32:01,  7.71s/it]
 30%|███       | 68293/225000 [138:56:21<315:36:13,  7.25s/it]
 30%|███       | 68294/225000 [138:56:26<285:25:17,  6.56s/it]
 30%|███       | 68295/225000 [138:56:31<263:49:34,  6.06s/it]
 30%|███       | 68296/225000 [138:56:36<253:39:39,  5.83s/it]
 30%|███       | 68297/225000 [138:56:49<349:23:43,  8.03s/it]
 30%|███       | 68298/225000 [138:56:54<305:27:11,  7.02s/it]
 30%|███       | 68299/225000 [138:57:02<316:38:16,  7.27s/it]
 30%|███       | 68300/225000 [138:57:10<320:35:40,  7.37s/it]
                                                              
{'loss': '0.7168', 'grad_norm': '0.908', 'learning_rate': '4.147e-05', 'epoch': '0.3036', 'num_input_tokens_seen': 230739904, 'train_runtime': '5.002e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68300/225000 [138:57:10<320:35:40,  7.37s/it]
 30%|███       | 68301/225000 [138:57:15<289:16:11,  6.65s/it]
 30%|███       | 68302/225000 [138:57:20<272:25:47,  6.26s/it]
 30%|███       | 68303/225000 [138:57:25<260:15:46,  5.98s/it]
 30%|███       | 68304/225000 [138:57:30<245:19:43,  5.64s/it]
 30%|███       | 68305/225000 [138:57:35<241:46:23,  5.55s/it]
 30%|███       | 68306/225000 [138:57:46<312:26:19,  7.18s/it]
 30%|███       | 68307/225000 [138:57:55<331:28:09,  7.62s/it]
 30%|███       | 68308/225000 [138:57:59<282:22:00,  6.49s/it]
 30%|███       | 68309/225000 [138:58:04<263:45:00,  6.06s/it]
 30%|███       | 68310/225000 [138:58:09<253:12:22,  5.82s/it]
 30%|███       | 68311/225000 [138:58:14<245:13:15,  5.63s/it]
 30%|███       | 68312/225000 [138:58:26<326:58:03,  7.51s/it]
 30%|███       | 68313/225000 [138:58:32<302:18:37,  6.95s/it]
 30%|███       | 68314/225000 [138:58:36<269:35:08,  6.19s/it]
 30%|███       | 68315/225000 [138:58:48<335:12:02,  7.70s/it]
 30%|███       | 68316/225000 [138:58:54<316:41:15,  7.28s/it]
 30%|███       | 68317/225000 [138:58:59<291:21:18,  6.69s/it]
 30%|███       | 68318/225000 [138:59:05<274:39:05,  6.31s/it]
 30%|███       | 68319/225000 [138:59:14<315:21:21,  7.25s/it]
 30%|███       | 68320/225000 [138:59:20<303:13:55,  6.97s/it]
 30%|███       | 68321/225000 [138:59:28<318:24:32,  7.32s/it]
 30%|███       | 68322/225000 [138:59:35<312:02:20,  7.17s/it]
 30%|███       | 68323/225000 [138:59:39<264:40:39,  6.08s/it]
 30%|███       | 68324/225000 [138:59:43<238:18:23,  5.48s/it]
 30%|███       | 68325/225000 [138:59:55<326:46:17,  7.51s/it]
 30%|███       | 68326/225000 [139:00:01<300:13:28,  6.90s/it]
 30%|███       | 68327/225000 [139:00:04<256:46:22,  5.90s/it]
 30%|███       | 68328/225000 [139:00:13<298:15:44,  6.85s/it]
 30%|███       | 68329/225000 [139:00:20<298:54:37,  6.87s/it]
 30%|███       | 68330/225000 [139:00:24<259:51:11,  5.97s/it]
 30%|███       | 68331/225000 [139:00:29<240:19:10,  5.52s/it]
 30%|███       | 68332/225000 [139:00:33<228:20:55,  5.25s/it]
 30%|███       | 68333/225000 [139:00:46<331:28:07,  7.62s/it]
 30%|███       | 68334/225000 [139:00:53<319:06:35,  7.33s/it]
 30%|███       | 68335/225000 [139:00:59<296:32:29,  6.81s/it]
 30%|███       | 68336/225000 [139:01:04<272:28:12,  6.26s/it]
 30%|███       | 68337/225000 [139:01:09<259:08:27,  5.95s/it]
 30%|███       | 68338/225000 [139:01:18<301:17:40,  6.92s/it]
 30%|███       | 68339/225000 [139:01:27<328:21:52,  7.55s/it]
 30%|███       | 68340/225000 [139:01:31<288:23:25,  6.63s/it]
 30%|███       | 68341/225000 [139:01:36<262:53:39,  6.04s/it]
 30%|███       | 68342/225000 [139:01:45<304:39:46,  7.00s/it]
 30%|███       | 68343/225000 [139:01:53<312:28:25,  7.18s/it]
 30%|███       | 68344/225000 [139:01:58<282:09:12,  6.48s/it]
 30%|███       | 68345/225000 [139:02:02<257:06:24,  5.91s/it]
 30%|███       | 68346/225000 [139:02:07<245:10:08,  5.63s/it]
 30%|███       | 68347/225000 [139:02:18<311:57:19,  7.17s/it]
 30%|███       | 68348/225000 [139:02:26<317:18:13,  7.29s/it]
 30%|███       | 68349/225000 [139:02:33<314:18:23,  7.22s/it]
 30%|███       | 68350/225000 [139:02:38<287:40:48,  6.61s/it]
                                                              
{'loss': '0.7218', 'grad_norm': '0.7426', 'learning_rate': '4.146e-05', 'epoch': '0.3038', 'num_input_tokens_seen': 230888392, 'train_runtime': '5.006e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68350/225000 [139:02:38<287:40:48,  6.61s/it]
 30%|███       | 68351/225000 [139:02:43<263:01:15,  6.04s/it]
 30%|███       | 68352/225000 [139:02:48<250:45:40,  5.76s/it]
 30%|███       | 68353/225000 [139:02:52<237:04:54,  5.45s/it]
 30%|███       | 68354/225000 [139:03:00<266:29:26,  6.12s/it]
 30%|███       | 68355/225000 [139:03:12<344:49:34,  7.92s/it]
 30%|███       | 68356/225000 [139:03:16<293:43:26,  6.75s/it]
 30%|███       | 68357/225000 [139:03:25<313:12:38,  7.20s/it]
 30%|███       | 68358/225000 [139:03:28<268:55:31,  6.18s/it]
 30%|███       | 68359/225000 [139:03:38<320:33:34,  7.37s/it]
 30%|███       | 68360/225000 [139:03:43<277:28:03,  6.38s/it]
 30%|███       | 68361/225000 [139:03:48<270:30:43,  6.22s/it]
 30%|███       | 68362/225000 [139:03:59<326:22:49,  7.50s/it]
 30%|███       | 68363/225000 [139:04:02<272:06:48,  6.25s/it]
 30%|███       | 68364/225000 [139:04:11<304:15:27,  6.99s/it]
 30%|███       | 68365/225000 [139:04:15<265:47:32,  6.11s/it]
 30%|███       | 68366/225000 [139:04:26<327:09:46,  7.52s/it]
 30%|███       | 68367/225000 [139:04:30<282:29:10,  6.49s/it]
 30%|███       | 68368/225000 [139:04:35<260:20:45,  5.98s/it]
 30%|███       | 68369/225000 [139:04:44<301:30:37,  6.93s/it]
 30%|███       | 68370/225000 [139:04:51<306:13:45,  7.04s/it]
 30%|███       | 68371/225000 [139:04:56<283:37:07,  6.52s/it]
 30%|███       | 68372/225000 [139:05:03<286:58:00,  6.60s/it]
 30%|███       | 68373/225000 [139:05:11<296:27:51,  6.81s/it]
 30%|███       | 68374/225000 [139:05:19<316:53:55,  7.28s/it]
 30%|███       | 68375/225000 [139:05:23<273:59:10,  6.30s/it]
 30%|███       | 68376/225000 [139:05:30<278:26:37,  6.40s/it]
 30%|███       | 68377/225000 [139:05:36<281:59:18,  6.48s/it]
 30%|███       | 68378/225000 [139:05:42<273:43:31,  6.29s/it]
 30%|███       | 68379/225000 [139:05:49<280:55:32,  6.46s/it]
 30%|███       | 68380/225000 [139:05:57<307:53:12,  7.08s/it]
 30%|███       | 68381/225000 [139:06:01<265:03:17,  6.09s/it]
 30%|███       | 68382/225000 [139:06:06<242:49:26,  5.58s/it]
 30%|███       | 68383/225000 [139:06:10<229:19:29,  5.27s/it]
 30%|███       | 68384/225000 [139:06:15<229:16:19,  5.27s/it]
 30%|███       | 68385/225000 [139:06:25<288:07:03,  6.62s/it]
 30%|███       | 68386/225000 [139:06:35<330:58:28,  7.61s/it]
 30%|███       | 68387/225000 [139:06:40<295:27:54,  6.79s/it]
 30%|███       | 68388/225000 [139:06:45<266:24:56,  6.12s/it]
 30%|███       | 68389/225000 [139:06:50<254:22:34,  5.85s/it]
 30%|███       | 68390/225000 [139:06:55<248:53:58,  5.72s/it]
 30%|███       | 68391/225000 [139:07:03<273:42:42,  6.29s/it]
 30%|███       | 68392/225000 [139:07:14<331:04:17,  7.61s/it]
 30%|███       | 68393/225000 [139:07:20<316:20:41,  7.27s/it]
 30%|███       | 68394/225000 [139:07:25<283:55:29,  6.53s/it]
 30%|███       | 68395/225000 [139:07:35<334:12:41,  7.68s/it]
 30%|███       | 68396/225000 [139:07:42<326:58:59,  7.52s/it]
 30%|███       | 68397/225000 [139:07:48<297:44:21,  6.84s/it]
 30%|███       | 68398/225000 [139:07:55<302:18:02,  6.95s/it]
 30%|███       | 68399/225000 [139:08:00<281:34:07,  6.47s/it]
 30%|███       | 68400/225000 [139:08:05<265:23:33,  6.10s/it]
                                                              
{'loss': '0.7194', 'grad_norm': '0.7825', 'learning_rate': '4.145e-05', 'epoch': '0.304', 'num_input_tokens_seen': 231035392, 'train_runtime': '5.009e+05', 'train_tokens_per_second': '461.3'}

 30%|███       | 68400/225000 [139:08:05<265:23:33,  6.10s/it]
 30%|███       | 68401/225000 [139:08:11<263:39:14,  6.06s/it]
 30%|███       | 68402/225000 [139:08:19<289:07:10,  6.65s/it]
 30%|███       | 68403/225000 [139:08:25<270:12:34,  6.21s/it]
 30%|███       | 68404/225000 [139:08:31<267:59:50,  6.16s/it]
 30%|███       | 68405/225000 [139:08:36<257:54:40,  5.93s/it]
 30%|███       | 68406/225000 [139:08:42<256:12:21,  5.89s/it]
 30%|███       | 68407/225000 [139:08:47<245:28:51,  5.64s/it]
 30%|███       | 68408/225000 [139:09:01<353:50:57,  8.13s/it]
 30%|███       | 68409/225000 [139:09:07<325:25:46,  7.48s/it]
 30%|███       | 68410/225000 [139:09:13<303:30:57,  6.98s/it]
 30%|███       | 68411/225000 [139:09:19<297:33:27,  6.84s/it]
 30%|███       | 68412/225000 [139:09:24<266:58:54,  6.14s/it]
 30%|███       | 68413/225000 [139:09:32<292:17:20,  6.72s/it]
 30%|███       | 68414/225000 [139:09:37<268:43:58,  6.18s/it]
 30%|███       | 68415/225000 [139:09:44<291:16:45,  6.70s/it]
 30%|███       | 68416/225000 [139:09:53<319:25:03,  7.34s/it]
 30%|███       | 68417/225000 [139:10:03<345:35:18,  7.95s/it]
 30%|███       | 68418/225000 [139:10:07<294:47:09,  6.78s/it]
 30%|███       | 68419/225000 [139:10:12<272:02:32,  6.25s/it]
 30%|███       | 68420/225000 [139:10:17<254:45:29,  5.86s/it]
 30%|███       | 68421/225000 [139:10:21<240:11:09,  5.52s/it]
 30%|███       | 68422/225000 [139:10:33<314:34:27,  7.23s/it]
 30%|███       | 68423/225000 [139:10:40<311:22:50,  7.16s/it]
 30%|███       | 68424/225000 [139:10:46<298:07:47,  6.85s/it]
 30%|███       | 68425/225000 [139:10:50<266:36:02,  6.13s/it]
 30%|███       | 68426/225000 [139:11:01<324:11:09,  7.45s/it]
 30%|███       | 68427/225000 [139:11:06<289:41:02,  6.66s/it]
 30%|███       | 68428/225000 [139:11:10<266:11:49,  6.12s/it]
 30%|███       | 68429/225000 [139:11:16<254:45:29,  5.86s/it]
 30%|███       | 68430/225000 [139:11:20<238:54:21,  5.49s/it]
 30%|███       | 68431/225000 [139:11:25<231:49:23,  5.33s/it]
 30%|███       | 68432/225000 [139:11:35<295:05:54,  6.79s/it]
 30%|███       | 68433/225000 [139:11:44<322:02:56,  7.40s/it]
 30%|███       | 68434/225000 [139:11:48<275:36:54,  6.34s/it]
 30%|███       | 68435/225000 [139:11:57<303:22:52,  6.98s/it]
 30%|███       | 68436/225000 [139:12:03<291:18:02,  6.70s/it]
 30%|███       | 68437/225000 [139:12:10<303:42:41,  6.98s/it]
 30%|███       | 68438/225000 [139:12:18<308:44:38,  7.10s/it]
 30%|███       | 68439/225000 [139:12:22<274:32:00,  6.31s/it]
 30%|███       | 68440/225000 [139:12:27<257:08:48,  5.91s/it]
 30%|███       | 68441/225000 [139:12:33<256:20:27,  5.89s/it]
 30%|███       | 68442/225000 [139:12:44<323:12:16,  7.43s/it]
 30%|███       | 68443/225000 [139:12:51<313:26:19,  7.21s/it]
 30%|███       | 68444/225000 [139:12:55<279:26:43,  6.43s/it]
 30%|███       | 68445/225000 [139:12:59<245:01:20,  5.63s/it]
 30%|███       | 68446/225000 [139:13:11<321:32:50,  7.39s/it]
 30%|███       | 68447/225000 [139:13:15<288:04:12,  6.62s/it]
 30%|███       | 68448/225000 [139:13:19<253:17:21,  5.82s/it]
 30%|███       | 68449/225000 [139:13:29<298:39:43,  6.87s/it]
 30%|███       | 68450/225000 [139:13:33<259:53:55,  5.98s/it]
                                                              
{'loss': '0.7356', 'grad_norm': '0.8255', 'learning_rate': '4.143e-05', 'epoch': '0.3042', 'num_input_tokens_seen': 231182408, 'train_runtime': '5.012e+05', 'train_tokens_per_second': '461.2'}

 30%|███       | 68450/225000 [139:13:33<259:53:55,  5.98s/it]
 30%|███       | 68451/225000 [139:13:39<261:27:52,  6.01s/it]
 30%|███       | 68452/225000 [139:13:48<308:04:50,  7.08s/it]
 30%|███       | 68453/225000 [139:13:52<268:04:23,  6.16s/it]
 30%|███       | 68454/225000 [139:14:00<294:33:05,  6.77s/it]
 30%|███       | 68455/225000 [139:14:08<300:17:42,  6.91s/it]
 30%|███       | 68456/225000 [139:14:16<320:58:04,  7.38s/it]
 30%|███       | 68457/225000 [139:14:20<277:32:19,  6.38s/it]
 30%|███       | 68458/225000 [139:14:24<244:18:27,  5.62s/it]
 30%|███       | 68459/225000 [139:14:34<295:09:56,  6.79s/it]
 30%|███       | 68460/225000 [139:14:38<265:53:51,  6.11s/it]
 30%|███       | 68461/225000 [139:14:42<241:28:37,  5.55s/it]
 30%|███       | 68462/225000 [139:14:49<253:52:57,  5.84s/it]
 30%|███       | 68463/225000 [139:14:59<307:24:26,  7.07s/it]
 30%|███       | 68464/225000 [139:15:03<263:45:51,  6.07s/it]
 30%|███       | 68465/225000 [139:15:14<337:22:36,  7.76s/it]
 30%|███       | 68466/225000 [139:15:18<284:59:11,  6.55s/it]
 30%|███       | 68467/225000 [139:15:25<292:03:03,  6.72s/it]
 30%|███       | 68468/225000 [139:15:32<292:21:12,  6.72s/it]
 30%|███       | 68469/225000 [139:15:37<275:47:46,  6.34s/it]
 30%|███       | 68470/225000 [139:15:46<303:56:09,  6.99s/it]
 30%|███       | 68471/225000 [139:15:50<261:53:40,  6.02s/it]
 30%|███       | 68472/225000 [139:15:58<289:07:06,  6.65s/it]
 30%|███       | 68473/225000 [139:16:04<290:46:04,  6.69s/it]
 30%|███       | 68474/225000 [139:16:12<301:13:37,  6.93s/it]
 30%|███       | 68475/225000 [139:16:16<258:27:30,  5.94s/it]
 30%|███       | 68476/225000 [139:16:23<280:26:16,  6.45s/it]
 30%|███       | 68477/225000 [139:16:27<246:47:16,  5.68s/it]
 30%|███       | 68478/225000 [139:16:35<282:27:36,  6.50s/it]
 30%|███       | 68479/225000 [139:16:39<247:19:02,  5.69s/it]
 30%|███       | 68480/225000 [139:16:44<230:26:39,  5.30s/it]
 30%|███       | 68481/225000 [139:16:57<330:11:50,  7.59s/it]
 30%|███       | 68482/225000 [139:17:00<281:18:53,  6.47s/it]
 30%|███       | 68483/225000 [139:17:05<249:36:59,  5.74s/it]
 30%|███       | 68484/225000 [139:17:14<304:44:35,  7.01s/it]
 30%|███       | 68485/225000 [139:17:18<260:34:48,  5.99s/it]
 30%|███       | 68486/225000 [139:17:27<298:57:12,  6.88s/it]
 30%|███       | 68487/225000 [139:17:31<259:20:15,  5.97s/it]
 30%|███       | 68488/225000 [139:17:41<312:21:11,  7.18s/it]
 30%|███       | 68489/225000 [139:17:45<272:21:30,  6.26s/it]
 30%|███       | 68490/225000 [139:17:55<315:41:09,  7.26s/it]
 30%|███       | 68491/225000 [139:17:59<273:06:50,  6.28s/it]
 30%|███       | 68492/225000 [139:18:03<250:56:00,  5.77s/it]
 30%|███       | 68493/225000 [139:18:15<332:32:40,  7.65s/it]
 30%|███       | 68494/225000 [139:18:21<307:34:48,  7.08s/it]
 30%|███       | 68495/225000 [139:18:27<293:34:35,  6.75s/it]
 30%|███       | 68496/225000 [139:18:32<267:56:46,  6.16s/it]
 30%|███       | 68497/225000 [139:18:36<249:20:10,  5.74s/it]
 30%|███       | 68498/225000 [139:18:48<328:31:48,  7.56s/it]
 30%|███       | 68499/225000 [139:18:52<281:08:52,  6.47s/it]
 30%|███       | 68500/225000 [139:19:01<310:57:05,  7.15s/it]
                                                              
{'loss': '0.7383', 'grad_norm': '0.8252', 'learning_rate': '4.142e-05', 'epoch': '0.3044', 'num_input_tokens_seen': 231327512, 'train_runtime': '5.015e+05', 'train_tokens_per_second': '461.2'}

 30%|███       | 68500/225000 [139:19:01<310:57:05,  7.15s/it]
 30%|███       | 68501/225000 [139:19:07<291:02:58,  6.70s/it]
 30%|███       | 68502/225000 [139:19:12<272:52:36,  6.28s/it]
 30%|███       | 68503/225000 [139:19:20<302:38:20,  6.96s/it]
 30%|███       | 68504/225000 [139:19:25<266:22:36,  6.13s/it]
 30%|███       | 68505/225000 [139:19:33<299:01:42,  6.88s/it]
 30%|███       | 68506/225000 [139:19:41<310:22:34,  7.14s/it]
 30%|███       | 68507/225000 [139:19:45<268:12:29,  6.17s/it]
 30%|███       | 68508/225000 [139:19:52<277:21:58,  6.38s/it]
 30%|███       | 68509/225000 [139:19:59<288:08:07,  6.63s/it]
 30%|███       | 68510/225000 [139:20:03<248:19:01,  5.71s/it]
 30%|███       | 68511/225000 [139:20:12<303:00:07,  6.97s/it]
 30%|███       | 68512/225000 [139:20:19<296:08:50,  6.81s/it]
 30%|███       | 68513/225000 [139:20:23<260:37:37,  6.00s/it]
 30%|███       | 68514/225000 [139:20:29<266:18:38,  6.13s/it]
 30%|███       | 68515/225000 [139:20:38<294:57:29,  6.79s/it]
 30%|███       | 68516/225000 [139:20:41<253:34:39,  5.83s/it]
 30%|███       | 68517/225000 [139:20:49<278:45:54,  6.41s/it]
 30%|███       | 68518/225000 [139:20:56<279:23:14,  6.43s/it]
 30%|███       | 68519/225000 [139:21:02<282:27:50,  6.50s/it]
 30%|███       | 68520/225000 [139:21:06<252:16:31,  5.80s/it]
 30%|███       | 68521/225000 [139:21:11<239:54:33,  5.52s/it]
 30%|███       | 68522/225000 [139:21:16<229:48:44,  5.29s/it]
 30%|███       | 68523/225000 [139:21:21<220:14:56,  5.07s/it]
 30%|███       | 68524/225000 [139:21:34<326:22:21,  7.51s/it]
 30%|███       | 68525/225000 [139:21:38<282:49:56,  6.51s/it]
 30%|███       | 68526/225000 [139:21:46<303:33:31,  6.98s/it]
 30%|███       | 68527/225000 [139:21:51<272:49:03,  6.28s/it]
 30%|███       | 68528/225000 [139:21:56<254:01:29,  5.84s/it]
 30%|███       | 68529/225000 [139:22:01<254:07:26,  5.85s/it]
 30%|███       | 68530/225000 [139:22:14<345:21:17,  7.95s/it]
 30%|███       | 68531/225000 [139:22:18<292:44:24,  6.74s/it]
 30%|███       | 68532/225000 [139:22:22<250:55:42,  5.77s/it]
 30%|███       | 68533/225000 [139:22:30<278:56:15,  6.42s/it]
 30%|███       | 68534/225000 [139:22:35<269:00:34,  6.19s/it]
 30%|███       | 68535/225000 [139:22:40<253:03:33,  5.82s/it]
 30%|███       | 68536/225000 [139:22:50<308:58:35,  7.11s/it]
 30%|███       | 68537/225000 [139:22:55<280:48:21,  6.46s/it]
 30%|███       | 68538/225000 [139:23:00<255:12:16,  5.87s/it]
 30%|███       | 68539/225000 [139:23:05<241:50:41,  5.56s/it]
 30%|███       | 68540/225000 [139:23:17<328:45:39,  7.56s/it]
 30%|███       | 68541/225000 [139:23:21<287:11:09,  6.61s/it]
 30%|███       | 68542/225000 [139:23:27<270:34:44,  6.23s/it]
 30%|███       | 68543/225000 [139:23:32<263:08:36,  6.05s/it]
 30%|███       | 68544/225000 [139:23:38<257:14:41,  5.92s/it]
 30%|███       | 68545/225000 [139:23:44<256:26:27,  5.90s/it]
 30%|███       | 68546/225000 [139:23:49<253:28:50,  5.83s/it]
 30%|███       | 68547/225000 [139:23:55<244:56:51,  5.64s/it]
 30%|███       | 68548/225000 [139:24:01<252:45:27,  5.82s/it]
 30%|███       | 68549/225000 [139:24:06<248:58:33,  5.73s/it]
 30%|███       | 68550/225000 [139:24:15<289:37:35,  6.66s/it]
                                                              
{'loss': '0.7231', 'grad_norm': '0.8965', 'learning_rate': '4.14e-05', 'epoch': '0.3047', 'num_input_tokens_seen': 231472152, 'train_runtime': '5.019e+05', 'train_tokens_per_second': '461.2'}

 30%|███       | 68550/225000 [139:24:15<289:37:35,  6.66s/it]
 30%|███       | 68551/225000 [139:24:28<366:47:48,  8.44s/it]
 30%|███       | 68552/225000 [139:24:32<318:09:49,  7.32s/it]
 30%|███       | 68553/225000 [139:24:37<283:24:36,  6.52s/it]
 30%|███       | 68554/225000 [139:24:43<274:05:30,  6.31s/it]
 30%|███       | 68555/225000 [139:24:48<253:56:07,  5.84s/it]
 30%|███       | 68556/225000 [139:25:01<352:49:38,  8.12s/it]
 30%|███       | 68557/225000 [139:25:06<311:09:56,  7.16s/it]
 30%|███       | 68558/225000 [139:25:13<303:49:00,  6.99s/it]
 30%|███       | 68559/225000 [139:25:17<273:45:43,  6.30s/it]
 30%|███       | 68560/225000 [139:25:22<251:22:28,  5.78s/it]
 30%|███       | 68561/225000 [139:25:27<236:26:17,  5.44s/it]
 30%|███       | 68562/225000 [139:25:39<329:32:29,  7.58s/it]
 30%|███       | 68563/225000 [139:25:44<290:13:48,  6.68s/it]
 30%|███       | 68564/225000 [139:25:48<265:01:05,  6.10s/it]
 30%|███       | 68565/225000 [139:25:53<248:12:18,  5.71s/it]
 30%|███       | 68566/225000 [139:26:02<282:43:31,  6.51s/it]
 30%|███       | 68567/225000 [139:26:11<323:33:28,  7.45s/it]
 30%|███       | 68568/225000 [139:26:15<277:31:30,  6.39s/it]
 30%|███       | 68569/225000 [139:26:19<245:52:43,  5.66s/it]
 30%|███       | 68570/225000 [139:26:24<233:32:20,  5.37s/it]
 30%|███       | 68571/225000 [139:26:36<326:14:33,  7.51s/it]
 30%|███       | 68572/225000 [139:26:44<327:21:01,  7.53s/it]
 30%|███       | 68573/225000 [139:26:48<283:20:42,  6.52s/it]
 30%|███       | 68574/225000 [139:26:52<252:03:09,  5.80s/it]
 30%|███       | 68575/225000 [139:27:05<346:08:32,  7.97s/it]
 30%|███       | 68576/225000 [139:27:09<290:44:38,  6.69s/it]
 30%|███       | 68577/225000 [139:27:13<255:13:50,  5.87s/it]
 30%|███       | 68578/225000 [139:27:23<312:43:31,  7.20s/it]
 30%|███       | 68579/225000 [139:27:27<273:59:28,  6.31s/it]
 30%|███       | 68580/225000 [139:27:32<247:36:54,  5.70s/it]
 30%|███       | 68581/225000 [139:27:37<238:35:13,  5.49s/it]
 30%|███       | 68582/225000 [139:27:43<244:54:29,  5.64s/it]
 30%|███       | 68583/225000 [139:27:50<260:55:58,  6.01s/it]
 30%|███       | 68584/225000 [139:28:02<344:02:22,  7.92s/it]
 30%|███       | 68585/225000 [139:28:06<294:54:14,  6.79s/it]
 30%|███       | 68586/225000 [139:28:11<276:02:34,  6.35s/it]
 30%|███       | 68587/225000 [139:28:18<280:56:13,  6.47s/it]
 30%|███       | 68588/225000 [139:28:27<309:56:05,  7.13s/it]
 30%|███       | 68589/225000 [139:28:32<279:08:21,  6.42s/it]
 30%|███       | 68590/225000 [139:28:40<302:47:10,  6.97s/it]
 30%|███       | 68591/225000 [139:28:47<302:28:34,  6.96s/it]
 30%|███       | 68592/225000 [139:28:51<272:08:32,  6.26s/it]
 30%|███       | 68593/225000 [139:28:56<249:37:42,  5.75s/it]
 30%|███       | 68594/225000 [139:29:01<236:26:20,  5.44s/it]
 30%|███       | 68595/225000 [139:29:11<296:55:34,  6.83s/it]
 30%|███       | 68596/225000 [139:29:16<278:33:03,  6.41s/it]
 30%|███       | 68597/225000 [139:29:21<261:54:10,  6.03s/it]
 30%|███       | 68598/225000 [139:29:26<244:06:40,  5.62s/it]
 30%|███       | 68599/225000 [139:29:31<230:33:24,  5.31s/it]
 30%|███       | 68600/225000 [139:29:43<317:49:22,  7.32s/it]
                                                              
{'loss': '0.7222', 'grad_norm': '0.8269', 'learning_rate': '4.139e-05', 'epoch': '0.3049', 'num_input_tokens_seen': 231614040, 'train_runtime': '5.022e+05', 'train_tokens_per_second': '461.2'}

 30%|███       | 68600/225000 [139:29:43<317:49:22,  7.32s/it]
 30%|███       | 68601/225000 [139:29:47<283:55:40,  6.54s/it]
 30%|███       | 68602/225000 [139:29:55<305:13:32,  7.03s/it]
 30%|███       | 68603/225000 [139:30:00<277:57:19,  6.40s/it]
 30%|███       | 68604/225000 [139:30:10<314:43:29,  7.24s/it]
 30%|███       | 68605/225000 [139:30:15<293:44:01,  6.76s/it]
 30%|███       | 68606/225000 [139:30:20<269:26:03,  6.20s/it]
 30%|███       | 68607/225000 [139:30:28<291:47:18,  6.72s/it]
 30%|███       | 68608/225000 [139:30:33<264:35:47,  6.09s/it]
 30%|███       | 68609/225000 [139:30:37<244:03:17,  5.62s/it]
 30%|███       | 68610/225000 [139:30:50<331:37:18,  7.63s/it]
 30%|███       | 68611/225000 [139:30:54<288:26:10,  6.64s/it]
 30%|███       | 68612/225000 [139:30:58<256:14:53,  5.90s/it]
 30%|███       | 68613/225000 [139:31:08<310:36:26,  7.15s/it]
 30%|███       | 68614/225000 [139:31:14<299:22:12,  6.89s/it]
 30%|███       | 68615/225000 [139:31:20<283:29:57,  6.53s/it]
 30%|███       | 68616/225000 [139:31:28<299:58:16,  6.91s/it]
 30%|███       | 68617/225000 [139:31:32<264:30:27,  6.09s/it]
 30%|███       | 68618/225000 [139:31:36<242:10:50,  5.58s/it]
 30%|███       | 68619/225000 [139:31:47<308:23:41,  7.10s/it]
 30%|███       | 68620/225000 [139:31:51<262:45:53,  6.05s/it]
 30%|███       | 68621/225000 [139:31:57<263:56:03,  6.08s/it]
 30%|███       | 68622/225000 [139:32:04<283:32:09,  6.53s/it]
 30%|███       | 68623/225000 [139:32:08<250:42:16,  5.77s/it]
 30%|███       | 68624/225000 [139:32:17<283:01:28,  6.52s/it]
 30%|███       | 68625/225000 [139:32:23<286:11:19,  6.59s/it]
 31%|███       | 68626/225000 [139:32:28<254:41:40,  5.86s/it]
 31%|███       | 68627/225000 [139:32:32<236:58:47,  5.46s/it]
 31%|███       | 68628/225000 [139:32:44<319:25:20,  7.35s/it]
 31%|███       | 68629/225000 [139:32:48<283:01:51,  6.52s/it]
 31%|███       | 68630/225000 [139:32:53<253:27:44,  5.84s/it]
 31%|███       | 68631/225000 [139:32:57<239:41:18,  5.52s/it]
 31%|███       | 68632/225000 [139:33:10<325:48:36,  7.50s/it]
 31%|███       | 68633/225000 [139:33:15<302:45:15,  6.97s/it]
 31%|███       | 68634/225000 [139:33:24<322:06:13,  7.42s/it]
 31%|███       | 68635/225000 [139:33:27<273:48:56,  6.30s/it]
 31%|███       | 68636/225000 [139:33:32<252:01:25,  5.80s/it]
 31%|███       | 68637/225000 [139:33:37<241:46:27,  5.57s/it]
 31%|███       | 68638/225000 [139:33:41<226:18:09,  5.21s/it]
 31%|███       | 68639/225000 [139:33:47<224:13:13,  5.16s/it]
 31%|███       | 68640/225000 [139:34:00<328:07:42,  7.55s/it]
 31%|███       | 68641/225000 [139:34:06<309:30:27,  7.13s/it]
 31%|███       | 68642/225000 [139:34:12<301:12:28,  6.94s/it]
 31%|███       | 68643/225000 [139:34:19<296:05:29,  6.82s/it]
 31%|███       | 68644/225000 [139:34:22<253:10:38,  5.83s/it]
 31%|███       | 68645/225000 [139:34:32<302:06:21,  6.96s/it]
 31%|███       | 68646/225000 [139:34:37<270:52:24,  6.24s/it]
 31%|███       | 68647/225000 [139:34:41<250:19:07,  5.76s/it]
 31%|███       | 68648/225000 [139:34:46<242:42:59,  5.59s/it]
 31%|███       | 68649/225000 [139:34:55<286:54:34,  6.61s/it]
 31%|███       | 68650/225000 [139:35:02<293:10:38,  6.75s/it]
                                                              
{'loss': '0.7494', 'grad_norm': '0.9578', 'learning_rate': '4.138e-05', 'epoch': '0.3051', 'num_input_tokens_seen': 231757080, 'train_runtime': '5.025e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68650/225000 [139:35:02<293:10:38,  6.75s/it]
 31%|███       | 68651/225000 [139:35:06<256:06:54,  5.90s/it]
 31%|███       | 68652/225000 [139:35:16<306:48:19,  7.06s/it]
 31%|███       | 68653/225000 [139:35:20<266:35:32,  6.14s/it]
 31%|███       | 68654/225000 [139:35:29<296:37:36,  6.83s/it]
 31%|███       | 68655/225000 [139:35:33<264:54:15,  6.10s/it]
 31%|███       | 68656/225000 [139:35:37<233:20:35,  5.37s/it]
 31%|███       | 68657/225000 [139:35:42<230:11:47,  5.30s/it]
 31%|███       | 68658/225000 [139:35:53<313:15:25,  7.21s/it]
 31%|███       | 68659/225000 [139:36:00<304:45:57,  7.02s/it]
 31%|███       | 68660/225000 [139:36:04<262:29:17,  6.04s/it]
 31%|███       | 68661/225000 [139:36:08<244:42:15,  5.63s/it]
 31%|███       | 68662/225000 [139:36:14<243:48:56,  5.61s/it]
 31%|███       | 68663/225000 [139:36:20<242:49:28,  5.59s/it]
 31%|███       | 68664/225000 [139:36:26<250:03:38,  5.76s/it]
 31%|███       | 68665/225000 [139:36:38<333:10:11,  7.67s/it]
 31%|███       | 68666/225000 [139:36:42<284:36:51,  6.55s/it]
 31%|███       | 68667/225000 [139:36:49<299:20:12,  6.89s/it]
 31%|███       | 68668/225000 [139:36:55<277:37:10,  6.39s/it]
 31%|███       | 68669/225000 [139:37:01<270:26:07,  6.23s/it]
 31%|███       | 68670/225000 [139:37:06<266:45:27,  6.14s/it]
 31%|███       | 68671/225000 [139:37:13<273:38:10,  6.30s/it]
 31%|███       | 68672/225000 [139:37:19<269:41:24,  6.21s/it]
 31%|███       | 68673/225000 [139:37:25<264:55:13,  6.10s/it]
 31%|███       | 68674/225000 [139:37:31<260:47:33,  6.01s/it]
 31%|███       | 68675/225000 [139:37:42<332:49:31,  7.66s/it]
 31%|███       | 68676/225000 [139:37:46<276:08:41,  6.36s/it]
 31%|███       | 68677/225000 [139:37:50<251:32:43,  5.79s/it]
 31%|███       | 68678/225000 [139:37:55<238:29:06,  5.49s/it]
 31%|███       | 68679/225000 [139:38:00<235:39:38,  5.43s/it]
 31%|███       | 68680/225000 [139:38:07<252:59:21,  5.83s/it]
 31%|███       | 68681/225000 [139:38:17<311:41:09,  7.18s/it]
 31%|███       | 68682/225000 [139:38:25<320:39:11,  7.38s/it]
 31%|███       | 68683/225000 [139:38:29<275:56:09,  6.35s/it]
 31%|███       | 68684/225000 [139:38:34<252:17:09,  5.81s/it]
 31%|███       | 68685/225000 [139:38:43<299:57:50,  6.91s/it]
 31%|███       | 68686/225000 [139:38:49<285:21:56,  6.57s/it]
 31%|███       | 68687/225000 [139:38:54<263:52:11,  6.08s/it]
 31%|███       | 68688/225000 [139:38:59<251:06:52,  5.78s/it]
 31%|███       | 68689/225000 [139:39:04<236:47:27,  5.45s/it]
 31%|███       | 68690/225000 [139:39:13<289:34:08,  6.67s/it]
 31%|███       | 68691/225000 [139:39:20<296:28:55,  6.83s/it]
 31%|███       | 68692/225000 [139:39:25<273:14:45,  6.29s/it]
 31%|███       | 68693/225000 [139:39:30<250:38:16,  5.77s/it]
 31%|███       | 68694/225000 [139:39:35<236:10:21,  5.44s/it]
 31%|███       | 68695/225000 [139:39:39<229:16:34,  5.28s/it]
 31%|███       | 68696/225000 [139:39:53<334:25:47,  7.70s/it]
 31%|███       | 68697/225000 [139:39:59<319:42:06,  7.36s/it]
 31%|███       | 68698/225000 [139:40:04<283:28:53,  6.53s/it]
 31%|███       | 68699/225000 [139:40:08<255:08:12,  5.88s/it]
 31%|███       | 68700/225000 [139:40:20<332:10:14,  7.65s/it]
                                                              
{'loss': '0.7338', 'grad_norm': '0.7816', 'learning_rate': '4.136e-05', 'epoch': '0.3053', 'num_input_tokens_seen': 231899232, 'train_runtime': '5.028e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68700/225000 [139:40:20<332:10:14,  7.65s/it]
 31%|███       | 68701/225000 [139:40:23<276:49:42,  6.38s/it]
 31%|███       | 68702/225000 [139:40:31<288:52:47,  6.65s/it]
 31%|███       | 68703/225000 [139:40:35<250:44:21,  5.78s/it]
 31%|███       | 68704/225000 [139:40:42<269:02:09,  6.20s/it]
 31%|███       | 68705/225000 [139:40:46<243:32:21,  5.61s/it]
 31%|███       | 68706/225000 [139:40:51<239:32:38,  5.52s/it]
 31%|███       | 68707/225000 [139:40:56<224:30:51,  5.17s/it]
 31%|███       | 68708/225000 [139:41:07<304:21:57,  7.01s/it]
 31%|███       | 68709/225000 [139:41:16<336:02:37,  7.74s/it]
 31%|███       | 68710/225000 [139:41:21<295:13:17,  6.80s/it]
 31%|███       | 68711/225000 [139:41:25<265:41:52,  6.12s/it]
 31%|███       | 68712/225000 [139:41:37<337:19:16,  7.77s/it]
 31%|███       | 68713/225000 [139:41:42<295:09:35,  6.80s/it]
 31%|███       | 68714/225000 [139:41:47<270:33:40,  6.23s/it]
 31%|███       | 68715/225000 [139:41:50<240:53:35,  5.55s/it]
 31%|███       | 68716/225000 [139:42:00<298:10:36,  6.87s/it]
 31%|███       | 68717/225000 [139:42:05<267:12:41,  6.16s/it]
 31%|███       | 68718/225000 [139:42:09<243:54:31,  5.62s/it]
 31%|███       | 68719/225000 [139:42:15<247:13:55,  5.70s/it]
 31%|███       | 68720/225000 [139:42:20<235:43:49,  5.43s/it]
 31%|███       | 68721/225000 [139:42:35<358:27:56,  8.26s/it]
 31%|███       | 68722/225000 [139:42:39<306:51:55,  7.07s/it]
 31%|███       | 68723/225000 [139:42:45<290:41:01,  6.70s/it]
 31%|███       | 68724/225000 [139:42:50<269:43:22,  6.21s/it]
 31%|███       | 68725/225000 [139:42:55<256:20:26,  5.91s/it]
 31%|███       | 68726/225000 [139:43:01<254:54:16,  5.87s/it]
 31%|███       | 68727/225000 [139:43:06<239:03:32,  5.51s/it]
 31%|███       | 68728/225000 [139:43:11<234:02:31,  5.39s/it]
 31%|███       | 68729/225000 [139:43:17<237:59:10,  5.48s/it]
 31%|███       | 68730/225000 [139:43:24<268:02:00,  6.17s/it]
 31%|███       | 68731/225000 [139:43:37<354:03:20,  8.16s/it]
 31%|███       | 68732/225000 [139:43:42<308:24:41,  7.10s/it]
 31%|███       | 68733/225000 [139:43:46<276:24:27,  6.37s/it]
 31%|███       | 68734/225000 [139:43:52<267:03:18,  6.15s/it]
 31%|███       | 68735/225000 [139:43:57<253:34:15,  5.84s/it]
 31%|███       | 68736/225000 [139:44:04<268:45:35,  6.19s/it]
 31%|███       | 68737/225000 [139:44:09<253:14:31,  5.83s/it]
 31%|███       | 68738/225000 [139:44:18<297:30:14,  6.85s/it]
 31%|███       | 68739/225000 [139:44:29<341:46:25,  7.87s/it]
 31%|███       | 68740/225000 [139:44:33<296:46:34,  6.84s/it]
 31%|███       | 68741/225000 [139:44:37<260:00:26,  5.99s/it]
 31%|███       | 68742/225000 [139:44:43<263:29:15,  6.07s/it]
 31%|███       | 68743/225000 [139:44:55<331:44:18,  7.64s/it]
 31%|███       | 68744/225000 [139:44:58<281:28:28,  6.48s/it]
 31%|███       | 68745/225000 [139:45:02<242:08:14,  5.58s/it]
 31%|███       | 68746/225000 [139:45:08<254:54:24,  5.87s/it]
 31%|███       | 68747/225000 [139:45:17<293:23:22,  6.76s/it]
 31%|███       | 68748/225000 [139:45:23<275:55:18,  6.36s/it]
 31%|███       | 68749/225000 [139:45:30<286:44:10,  6.61s/it]
 31%|███       | 68750/225000 [139:45:36<283:18:35,  6.53s/it]
                                                              
{'loss': '0.7441', 'grad_norm': '0.6773', 'learning_rate': '4.135e-05', 'epoch': '0.3056', 'num_input_tokens_seen': 232041536, 'train_runtime': '5.031e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68750/225000 [139:45:36<283:18:35,  6.53s/it]
 31%|███       | 68751/225000 [139:45:47<340:49:00,  7.85s/it]
 31%|███       | 68752/225000 [139:45:52<298:22:48,  6.87s/it]
 31%|███       | 68753/225000 [139:45:57<278:30:57,  6.42s/it]
 31%|███       | 68754/225000 [139:46:07<317:39:48,  7.32s/it]
 31%|███       | 68755/225000 [139:46:12<291:54:33,  6.73s/it]
 31%|███       | 68756/225000 [139:46:16<262:26:45,  6.05s/it]
 31%|███       | 68757/225000 [139:46:22<250:59:42,  5.78s/it]
 31%|███       | 68758/225000 [139:46:27<249:02:53,  5.74s/it]
 31%|███       | 68759/225000 [139:46:32<231:23:50,  5.33s/it]
 31%|███       | 68760/225000 [139:46:40<271:13:13,  6.25s/it]
 31%|███       | 68761/225000 [139:46:51<333:38:22,  7.69s/it]
 31%|███       | 68762/225000 [139:46:55<285:19:32,  6.57s/it]
 31%|███       | 68763/225000 [139:47:02<293:20:33,  6.76s/it]
 31%|███       | 68764/225000 [139:47:08<280:36:10,  6.47s/it]
 31%|███       | 68765/225000 [139:47:15<294:53:45,  6.80s/it]
 31%|███       | 68766/225000 [139:47:20<266:10:50,  6.13s/it]
 31%|███       | 68767/225000 [139:47:24<241:52:12,  5.57s/it]
 31%|███       | 68768/225000 [139:47:30<242:11:59,  5.58s/it]
 31%|███       | 68769/225000 [139:47:38<279:51:54,  6.45s/it]
 31%|███       | 68770/225000 [139:47:49<331:36:56,  7.64s/it]
 31%|███       | 68771/225000 [139:47:53<280:36:12,  6.47s/it]
 31%|███       | 68772/225000 [139:48:02<314:45:44,  7.25s/it]
 31%|███       | 68773/225000 [139:48:07<284:37:40,  6.56s/it]
 31%|███       | 68774/225000 [139:48:14<298:04:40,  6.87s/it]
 31%|███       | 68775/225000 [139:48:18<264:47:44,  6.10s/it]
 31%|███       | 68776/225000 [139:48:24<251:10:32,  5.79s/it]
 31%|███       | 68777/225000 [139:48:28<237:00:19,  5.46s/it]
 31%|███       | 68778/225000 [139:48:34<236:05:06,  5.44s/it]
 31%|███       | 68779/225000 [139:48:38<221:32:19,  5.11s/it]
 31%|███       | 68780/225000 [139:48:43<222:04:02,  5.12s/it]
 31%|███       | 68781/225000 [139:48:48<225:07:46,  5.19s/it]
 31%|███       | 68782/225000 [139:48:59<290:54:16,  6.70s/it]
 31%|███       | 68783/225000 [139:49:07<318:23:42,  7.34s/it]
 31%|███       | 68784/225000 [139:49:13<295:35:08,  6.81s/it]
 31%|███       | 68785/225000 [139:49:20<294:44:52,  6.79s/it]
 31%|███       | 68786/225000 [139:49:24<255:13:20,  5.88s/it]
 31%|███       | 68787/225000 [139:49:28<232:26:42,  5.36s/it]
 31%|███       | 68788/225000 [139:49:34<242:15:39,  5.58s/it]
 31%|███       | 68789/225000 [139:49:45<316:47:31,  7.30s/it]
 31%|███       | 68790/225000 [139:49:48<263:02:57,  6.06s/it]
 31%|███       | 68791/225000 [139:49:56<286:07:09,  6.59s/it]
 31%|███       | 68792/225000 [139:50:01<256:58:46,  5.92s/it]
 31%|███       | 68793/225000 [139:50:05<241:20:42,  5.56s/it]
 31%|███       | 68794/225000 [139:50:10<229:24:00,  5.29s/it]
 31%|███       | 68795/225000 [139:50:22<313:30:10,  7.23s/it]
 31%|███       | 68796/225000 [139:50:29<310:07:51,  7.15s/it]
 31%|███       | 68797/225000 [139:50:33<268:49:27,  6.20s/it]
 31%|███       | 68798/225000 [139:50:43<319:15:54,  7.36s/it]
 31%|███       | 68799/225000 [139:50:46<273:23:34,  6.30s/it]
 31%|███       | 68800/225000 [139:50:53<271:58:44,  6.27s/it]
                                                              
{'loss': '0.7518', 'grad_norm': '1.122', 'learning_rate': '4.134e-05', 'epoch': '0.3058', 'num_input_tokens_seen': 232181992, 'train_runtime': '5.035e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68800/225000 [139:50:53<271:58:44,  6.27s/it]
 31%|███       | 68801/225000 [139:50:56<234:22:50,  5.40s/it]
 31%|███       | 68802/225000 [139:51:04<268:24:42,  6.19s/it]
 31%|███       | 68803/225000 [139:51:08<233:25:22,  5.38s/it]
 31%|███       | 68804/225000 [139:51:17<292:36:22,  6.74s/it]
 31%|███       | 68805/225000 [139:51:24<295:38:26,  6.81s/it]
 31%|███       | 68806/225000 [139:51:30<281:17:40,  6.48s/it]
 31%|███       | 68807/225000 [139:51:36<268:43:21,  6.19s/it]
 31%|███       | 68808/225000 [139:51:41<256:22:29,  5.91s/it]
 31%|███       | 68809/225000 [139:51:53<331:14:28,  7.63s/it]
 31%|███       | 68810/225000 [139:51:56<272:25:35,  6.28s/it]
 31%|███       | 68811/225000 [139:52:01<254:39:23,  5.87s/it]
 31%|███       | 68812/225000 [139:52:06<245:02:23,  5.65s/it]
 31%|███       | 68813/225000 [139:52:11<245:26:27,  5.66s/it]
 31%|███       | 68814/225000 [139:52:17<240:11:05,  5.54s/it]
 31%|███       | 68815/225000 [139:52:23<254:09:28,  5.86s/it]
 31%|███       | 68816/225000 [139:52:38<373:31:29,  8.61s/it]
 31%|███       | 68817/225000 [139:52:42<303:23:53,  6.99s/it]
 31%|███       | 68818/225000 [139:52:46<265:40:15,  6.12s/it]
 31%|███       | 68819/225000 [139:52:54<299:19:21,  6.90s/it]
 31%|███       | 68820/225000 [139:53:01<293:13:44,  6.76s/it]
 31%|███       | 68821/225000 [139:53:07<292:21:33,  6.74s/it]
 31%|███       | 68822/225000 [139:53:12<261:16:45,  6.02s/it]
 31%|███       | 68823/225000 [139:53:18<260:37:52,  6.01s/it]
 31%|███       | 68824/225000 [139:53:26<293:44:29,  6.77s/it]
 31%|███       | 68825/225000 [139:53:31<267:30:35,  6.17s/it]
 31%|███       | 68826/225000 [139:53:36<247:02:42,  5.69s/it]
 31%|███       | 68827/225000 [139:53:44<275:11:47,  6.34s/it]
 31%|███       | 68828/225000 [139:53:52<301:13:30,  6.94s/it]
 31%|███       | 68829/225000 [139:53:59<298:41:44,  6.89s/it]
 31%|███       | 68830/225000 [139:54:03<262:28:34,  6.05s/it]
 31%|███       | 68831/225000 [139:54:09<266:29:19,  6.14s/it]
 31%|███       | 68832/225000 [139:54:18<300:14:40,  6.92s/it]
 31%|███       | 68833/225000 [139:54:22<261:17:23,  6.02s/it]
 31%|███       | 68834/225000 [139:54:30<284:18:24,  6.55s/it]
 31%|███       | 68835/225000 [139:54:35<273:19:50,  6.30s/it]
 31%|███       | 68836/225000 [139:54:39<235:34:55,  5.43s/it]
 31%|███       | 68837/225000 [139:54:49<294:29:06,  6.79s/it]
 31%|███       | 68838/225000 [139:54:53<259:41:05,  5.99s/it]
 31%|███       | 68839/225000 [139:55:01<285:47:15,  6.59s/it]
 31%|███       | 68840/225000 [139:55:05<251:22:42,  5.80s/it]
 31%|███       | 68841/225000 [139:55:12<277:07:54,  6.39s/it]
 31%|███       | 68842/225000 [139:55:19<280:03:10,  6.46s/it]
 31%|███       | 68843/225000 [139:55:24<254:00:22,  5.86s/it]
 31%|███       | 68844/225000 [139:55:29<245:37:45,  5.66s/it]
 31%|███       | 68845/225000 [139:55:38<292:06:50,  6.73s/it]
 31%|███       | 68846/225000 [139:55:47<315:54:50,  7.28s/it]
 31%|███       | 68847/225000 [139:55:50<270:24:38,  6.23s/it]
 31%|███       | 68848/225000 [139:56:00<319:43:59,  7.37s/it]
 31%|███       | 68849/225000 [139:56:04<272:20:54,  6.28s/it]
 31%|███       | 68850/225000 [139:56:10<272:14:01,  6.28s/it]
                                                              
{'loss': '0.7315', 'grad_norm': '0.7473', 'learning_rate': '4.132e-05', 'epoch': '0.306', 'num_input_tokens_seen': 232322432, 'train_runtime': '5.038e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68850/225000 [139:56:10<272:14:01,  6.28s/it]
 31%|███       | 68851/225000 [139:56:15<257:32:08,  5.94s/it]
 31%|███       | 68852/225000 [139:56:21<246:15:05,  5.68s/it]
 31%|███       | 68853/225000 [139:56:30<288:46:13,  6.66s/it]
 31%|███       | 68854/225000 [139:56:35<268:30:45,  6.19s/it]
 31%|███       | 68855/225000 [139:56:39<250:07:50,  5.77s/it]
 31%|███       | 68856/225000 [139:56:45<246:06:02,  5.67s/it]
 31%|███       | 68857/225000 [139:56:51<248:52:43,  5.74s/it]
 31%|███       | 68858/225000 [139:56:56<245:37:30,  5.66s/it]
 31%|███       | 68859/225000 [139:57:01<237:17:36,  5.47s/it]
 31%|███       | 68860/225000 [139:57:07<239:47:51,  5.53s/it]
 31%|███       | 68861/225000 [139:57:20<340:59:39,  7.86s/it]
 31%|███       | 68862/225000 [139:57:28<337:12:46,  7.77s/it]
 31%|███       | 68863/225000 [139:57:33<304:35:46,  7.02s/it]
 31%|███       | 68864/225000 [139:57:38<275:25:45,  6.35s/it]
 31%|███       | 68865/225000 [139:57:42<248:20:29,  5.73s/it]
 31%|███       | 68866/225000 [139:57:53<321:29:19,  7.41s/it]
 31%|███       | 68867/225000 [139:57:57<271:35:57,  6.26s/it]
 31%|███       | 68868/225000 [139:58:01<244:39:38,  5.64s/it]
 31%|███       | 68869/225000 [139:58:09<271:37:28,  6.26s/it]
 31%|███       | 68870/225000 [139:58:16<280:00:25,  6.46s/it]
 31%|███       | 68871/225000 [139:58:20<247:44:27,  5.71s/it]
 31%|███       | 68872/225000 [139:58:30<301:04:51,  6.94s/it]
 31%|███       | 68873/225000 [139:58:34<263:31:16,  6.08s/it]
 31%|███       | 68874/225000 [139:58:38<243:53:51,  5.62s/it]
 31%|███       | 68875/225000 [139:58:43<233:09:09,  5.38s/it]
 31%|███       | 68876/225000 [139:58:48<221:43:23,  5.11s/it]
 31%|███       | 68877/225000 [139:58:55<251:00:53,  5.79s/it]
 31%|███       | 68878/225000 [139:59:05<305:17:17,  7.04s/it]
 31%|███       | 68879/225000 [139:59:09<264:06:01,  6.09s/it]
 31%|███       | 68880/225000 [139:59:13<239:24:58,  5.52s/it]
 31%|███       | 68881/225000 [139:59:22<285:25:56,  6.58s/it]
 31%|███       | 68882/225000 [139:59:29<290:17:41,  6.69s/it]
 31%|███       | 68883/225000 [139:59:33<252:08:20,  5.81s/it]
 31%|███       | 68884/225000 [139:59:38<244:23:06,  5.64s/it]
 31%|███       | 68885/225000 [139:59:47<293:50:34,  6.78s/it]
 31%|███       | 68886/225000 [139:59:52<269:53:18,  6.22s/it]
 31%|███       | 68887/225000 [139:59:56<232:10:05,  5.35s/it]
 31%|███       | 68888/225000 [140:00:07<314:41:37,  7.26s/it]
 31%|███       | 68889/225000 [140:00:13<290:39:37,  6.70s/it]
 31%|███       | 68890/225000 [140:00:17<259:52:12,  5.99s/it]
 31%|███       | 68891/225000 [140:00:21<235:23:28,  5.43s/it]
 31%|███       | 68892/225000 [140:00:25<218:57:58,  5.05s/it]
 31%|███       | 68893/225000 [140:00:36<291:45:47,  6.73s/it]
 31%|███       | 68894/225000 [140:00:43<298:12:25,  6.88s/it]
 31%|███       | 68895/225000 [140:00:48<276:47:54,  6.38s/it]
 31%|███       | 68896/225000 [140:00:56<288:34:29,  6.65s/it]
 31%|███       | 68897/225000 [140:01:01<268:37:52,  6.20s/it]
 31%|███       | 68898/225000 [140:01:05<245:59:22,  5.67s/it]
 31%|███       | 68899/225000 [140:01:11<247:14:32,  5.70s/it]
 31%|███       | 68900/225000 [140:01:15<224:22:58,  5.17s/it]
                                                              
{'loss': '0.7606', 'grad_norm': '0.7912', 'learning_rate': '4.131e-05', 'epoch': '0.3062', 'num_input_tokens_seen': 232460320, 'train_runtime': '5.041e+05', 'train_tokens_per_second': '461.2'}

 31%|███       | 68900/225000 [140:01:15<224:22:58,  5.17s/it]
 31%|███       | 68901/225000 [140:01:22<241:36:22,  5.57s/it]
 31%|███       | 68902/225000 [140:01:35<340:03:14,  7.84s/it]
 31%|███       | 68903/225000 [140:01:39<289:37:27,  6.68s/it]
 31%|███       | 68904/225000 [140:01:44<267:46:53,  6.18s/it]
 31%|███       | 68905/225000 [140:01:52<298:46:13,  6.89s/it]
 31%|███       | 68906/225000 [140:01:56<259:50:28,  5.99s/it]
 31%|███       | 68907/225000 [140:02:04<278:52:52,  6.43s/it]
 31%|███       | 68908/225000 [140:02:11<289:34:21,  6.68s/it]
 31%|███       | 68909/225000 [140:02:15<251:56:44,  5.81s/it]
 31%|███       | 68910/225000 [140:02:19<239:22:47,  5.52s/it]
 31%|███       | 68911/225000 [140:02:31<320:16:04,  7.39s/it]
 31%|███       | 68912/225000 [140:02:36<291:10:11,  6.72s/it]
 31%|███       | 68913/225000 [140:02:42<278:28:50,  6.42s/it]
 31%|███       | 68914/225000 [140:02:47<259:41:58,  5.99s/it]
 31%|███       | 68915/225000 [140:02:53<256:57:47,  5.93s/it]
 31%|███       | 68916/225000 [140:02:58<245:32:43,  5.66s/it]
 31%|███       | 68917/225000 [140:03:03<239:04:47,  5.51s/it]
 31%|███       | 68918/225000 [140:03:08<235:41:18,  5.44s/it]
 31%|███       | 68919/225000 [140:03:18<284:44:26,  6.57s/it]
 31%|███       | 68920/225000 [140:03:22<255:55:13,  5.90s/it]
 31%|███       | 68921/225000 [140:03:26<232:52:32,  5.37s/it]
 31%|███       | 68922/225000 [140:03:34<267:02:05,  6.16s/it]
 31%|███       | 68923/225000 [140:03:42<292:28:07,  6.75s/it]
 31%|███       | 68924/225000 [140:03:46<252:18:21,  5.82s/it]
 31%|███       | 68925/225000 [140:03:56<314:19:25,  7.25s/it]
 31%|███       | 68926/225000 [140:04:00<264:57:50,  6.11s/it]
 31%|███       | 68927/225000 [140:04:09<303:26:07,  7.00s/it]
 31%|███       | 68928/225000 [140:04:13<260:12:13,  6.00s/it]
 31%|███       | 68929/225000 [140:04:17<237:53:27,  5.49s/it]
 31%|███       | 68930/225000 [140:04:26<287:03:19,  6.62s/it]
 31%|███       | 68931/225000 [140:04:31<262:30:59,  6.06s/it]
 31%|███       | 68932/225000 [140:04:34<231:09:43,  5.33s/it]
 31%|███       | 68933/225000 [140:04:39<225:20:31,  5.20s/it]
 31%|███       | 68934/225000 [140:04:44<223:51:01,  5.16s/it]
 31%|███       | 68935/225000 [140:04:53<271:06:58,  6.25s/it]
 31%|███       | 68936/225000 [140:05:02<297:51:49,  6.87s/it]
 31%|███       | 68937/225000 [140:05:10<317:26:25,  7.32s/it]
 31%|███       | 68938/225000 [140:05:14<273:04:02,  6.30s/it]
 31%|███       | 68939/225000 [140:05:18<247:07:40,  5.70s/it]
 31%|███       | 68940/225000 [140:05:23<237:15:44,  5.47s/it]
 31%|███       | 68941/225000 [140:05:29<243:11:41,  5.61s/it]
 31%|███       | 68942/225000 [140:05:39<295:47:50,  6.82s/it]
 31%|███       | 68943/225000 [140:05:42<252:40:45,  5.83s/it]
 31%|███       | 68944/225000 [140:05:52<302:53:05,  6.99s/it]
 31%|███       | 68945/225000 [140:05:56<263:19:54,  6.07s/it]
 31%|███       | 68946/225000 [140:06:00<233:29:02,  5.39s/it]
 31%|███       | 68947/225000 [140:06:09<280:17:09,  6.47s/it]
 31%|███       | 68948/225000 [140:06:12<243:42:42,  5.62s/it]
 31%|███       | 68949/225000 [140:06:23<311:26:36,  7.18s/it]
 31%|███       | 68950/225000 [140:06:27<273:40:26,  6.31s/it]
                                                              
{'loss': '0.7603', 'grad_norm': '1.083', 'learning_rate': '4.129e-05', 'epoch': '0.3064', 'num_input_tokens_seen': 232596384, 'train_runtime': '5.044e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 68950/225000 [140:06:27<273:40:26,  6.31s/it]
 31%|███       | 68951/225000 [140:06:31<240:40:49,  5.55s/it]
 31%|███       | 68952/225000 [140:06:35<222:24:54,  5.13s/it]
 31%|███       | 68953/225000 [140:06:46<294:46:15,  6.80s/it]
 31%|███       | 68954/225000 [140:06:50<257:17:55,  5.94s/it]
 31%|███       | 68955/225000 [140:06:55<241:37:59,  5.57s/it]
 31%|███       | 68956/225000 [140:07:04<288:47:10,  6.66s/it]
 31%|███       | 68957/225000 [140:07:09<267:13:38,  6.17s/it]
 31%|███       | 68958/225000 [140:07:18<300:07:55,  6.92s/it]
 31%|███       | 68959/225000 [140:07:24<289:49:55,  6.69s/it]
 31%|███       | 68960/225000 [140:07:28<264:03:36,  6.09s/it]
 31%|███       | 68961/225000 [140:07:34<252:16:29,  5.82s/it]
 31%|███       | 68962/225000 [140:07:39<250:40:23,  5.78s/it]
 31%|███       | 68963/225000 [140:07:46<258:54:45,  5.97s/it]
 31%|███       | 68964/225000 [140:07:51<248:08:19,  5.72s/it]
 31%|███       | 68965/225000 [140:07:56<235:30:53,  5.43s/it]
 31%|███       | 68966/225000 [140:08:01<234:42:00,  5.41s/it]
 31%|███       | 68967/225000 [140:08:06<229:53:45,  5.30s/it]
 31%|███       | 68968/225000 [140:08:12<237:18:19,  5.48s/it]
 31%|███       | 68969/225000 [140:08:18<248:17:21,  5.73s/it]
 31%|███       | 68970/225000 [140:08:31<340:32:16,  7.86s/it]
 31%|███       | 68971/225000 [140:08:38<324:33:22,  7.49s/it]
 31%|███       | 68972/225000 [140:08:43<300:49:26,  6.94s/it]
 31%|███       | 68973/225000 [140:08:49<280:25:50,  6.47s/it]
 31%|███       | 68974/225000 [140:08:53<246:18:05,  5.68s/it]
 31%|███       | 68975/225000 [140:08:57<225:52:58,  5.21s/it]
 31%|███       | 68976/225000 [140:09:05<267:33:43,  6.17s/it]
 31%|███       | 68977/225000 [140:09:13<287:25:29,  6.63s/it]
 31%|███       | 68978/225000 [140:09:17<254:16:03,  5.87s/it]
 31%|███       | 68979/225000 [140:09:21<229:03:36,  5.29s/it]
 31%|███       | 68980/225000 [140:09:31<299:17:52,  6.91s/it]
 31%|███       | 68981/225000 [140:09:35<259:36:42,  5.99s/it]
 31%|███       | 68982/225000 [140:09:40<236:39:50,  5.46s/it]
 31%|███       | 68983/225000 [140:09:48<274:42:52,  6.34s/it]
 31%|███       | 68984/225000 [140:09:51<234:35:39,  5.41s/it]
 31%|███       | 68985/225000 [140:10:02<305:07:52,  7.04s/it]
 31%|███       | 68986/225000 [140:10:07<275:01:59,  6.35s/it]
 31%|███       | 68987/225000 [140:10:12<255:00:02,  5.88s/it]
 31%|███       | 68988/225000 [140:10:16<235:34:24,  5.44s/it]
 31%|███       | 68989/225000 [140:10:21<224:59:29,  5.19s/it]
 31%|███       | 68990/225000 [140:10:29<265:53:18,  6.14s/it]
 31%|███       | 68991/225000 [140:10:37<290:59:25,  6.71s/it]
 31%|███       | 68992/225000 [140:10:42<266:04:11,  6.14s/it]
 31%|███       | 68993/225000 [140:10:47<259:11:14,  5.98s/it]
 31%|███       | 68994/225000 [140:10:58<322:57:36,  7.45s/it]
 31%|███       | 68995/225000 [140:11:02<275:20:55,  6.35s/it]
 31%|███       | 68996/225000 [140:11:09<285:08:00,  6.58s/it]
 31%|███       | 68997/225000 [140:11:17<298:45:44,  6.89s/it]
 31%|███       | 68998/225000 [140:11:21<270:46:05,  6.25s/it]
 31%|███       | 68999/225000 [140:11:29<283:39:11,  6.55s/it]
 31%|███       | 69000/225000 [140:11:32<246:22:08,  5.69s/it]
                                                              
{'loss': '0.7458', 'grad_norm': '0.8695', 'learning_rate': '4.128e-05', 'epoch': '0.3067', 'num_input_tokens_seen': 232733672, 'train_runtime': '5.047e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69000/225000 [140:11:32<246:22:08,  5.69s/it]
 31%|███       | 69001/225000 [140:11:37<232:48:36,  5.37s/it]
 31%|███       | 69002/225000 [140:11:47<296:22:30,  6.84s/it]
 31%|███       | 69003/225000 [140:11:52<266:10:20,  6.14s/it]
 31%|███       | 69004/225000 [140:11:56<244:27:51,  5.64s/it]
 31%|███       | 69005/225000 [140:12:01<236:00:34,  5.45s/it]
 31%|███       | 69006/225000 [140:12:06<228:25:09,  5.27s/it]
 31%|███       | 69007/225000 [140:12:11<228:51:07,  5.28s/it]
 31%|███       | 69008/225000 [140:12:17<231:25:09,  5.34s/it]
 31%|███       | 69009/225000 [140:12:22<232:01:38,  5.35s/it]
 31%|███       | 69010/225000 [140:12:34<315:07:16,  7.27s/it]
 31%|███       | 69011/225000 [140:12:41<305:45:45,  7.06s/it]
 31%|███       | 69012/225000 [140:12:48<306:24:39,  7.07s/it]
 31%|███       | 69013/225000 [140:12:54<297:21:56,  6.86s/it]
 31%|███       | 69014/225000 [140:12:58<252:33:53,  5.83s/it]
 31%|███       | 69015/225000 [140:13:04<257:20:05,  5.94s/it]
 31%|███       | 69016/225000 [140:13:11<270:16:39,  6.24s/it]
 31%|███       | 69017/225000 [140:13:15<239:24:35,  5.53s/it]
 31%|███       | 69018/225000 [140:13:20<239:26:15,  5.53s/it]
 31%|███       | 69019/225000 [140:13:26<246:08:35,  5.68s/it]
 31%|███       | 69020/225000 [140:13:32<247:12:02,  5.71s/it]
 31%|███       | 69021/225000 [140:13:39<271:45:14,  6.27s/it]
 31%|███       | 69022/225000 [140:13:44<247:06:15,  5.70s/it]
 31%|███       | 69023/225000 [140:13:50<248:11:20,  5.73s/it]
 31%|███       | 69024/225000 [140:13:54<229:52:00,  5.31s/it]
 31%|███       | 69025/225000 [140:13:58<210:27:48,  4.86s/it]
 31%|███       | 69026/225000 [140:14:07<272:51:51,  6.30s/it]
 31%|███       | 69027/225000 [140:14:11<240:49:03,  5.56s/it]
 31%|███       | 69028/225000 [140:14:15<220:54:09,  5.10s/it]
 31%|███       | 69029/225000 [140:14:23<257:32:56,  5.94s/it]
 31%|███       | 69030/225000 [140:14:28<241:14:29,  5.57s/it]
 31%|███       | 69031/225000 [140:14:33<235:19:13,  5.43s/it]
 31%|███       | 69032/225000 [140:14:38<226:03:33,  5.22s/it]
 31%|███       | 69033/225000 [140:14:42<218:37:24,  5.05s/it]
 31%|███       | 69034/225000 [140:14:47<215:34:06,  4.98s/it]
 31%|███       | 69035/225000 [140:14:57<281:26:53,  6.50s/it]
 31%|███       | 69036/225000 [140:15:06<311:29:02,  7.19s/it]
 31%|███       | 69037/225000 [140:15:12<301:22:22,  6.96s/it]
 31%|███       | 69038/225000 [140:15:17<268:28:21,  6.20s/it]
 31%|███       | 69039/225000 [140:15:21<241:54:27,  5.58s/it]
 31%|███       | 69040/225000 [140:15:32<307:28:07,  7.10s/it]
 31%|███       | 69041/225000 [140:15:37<278:54:10,  6.44s/it]
 31%|███       | 69042/225000 [140:15:43<277:59:24,  6.42s/it]
 31%|███       | 69043/225000 [140:15:47<251:02:37,  5.79s/it]
 31%|███       | 69044/225000 [140:15:52<232:22:40,  5.36s/it]
 31%|███       | 69045/225000 [140:16:02<291:56:47,  6.74s/it]
 31%|███       | 69046/225000 [140:16:08<287:49:24,  6.64s/it]
 31%|███       | 69047/225000 [140:16:12<250:53:48,  5.79s/it]
 31%|███       | 69048/225000 [140:16:19<263:37:25,  6.09s/it]
 31%|███       | 69049/225000 [140:16:27<292:20:59,  6.75s/it]
 31%|███       | 69050/225000 [140:16:32<268:55:48,  6.21s/it]
                                                              
{'loss': '0.7501', 'grad_norm': '0.9255', 'learning_rate': '4.127e-05', 'epoch': '0.3069', 'num_input_tokens_seen': 232869432, 'train_runtime': '5.05e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69050/225000 [140:16:32<268:55:48,  6.21s/it]
 31%|███       | 69051/225000 [140:16:38<268:38:57,  6.20s/it]
 31%|███       | 69052/225000 [140:16:44<264:58:52,  6.12s/it]
 31%|███       | 69053/225000 [140:16:51<273:07:22,  6.30s/it]
 31%|███       | 69054/225000 [140:16:55<244:42:27,  5.65s/it]
 31%|███       | 69055/225000 [140:16:59<223:29:33,  5.16s/it]
 31%|███       | 69056/225000 [140:17:05<239:10:29,  5.52s/it]
 31%|███       | 69057/225000 [140:17:17<316:38:46,  7.31s/it]
 31%|███       | 69058/225000 [140:17:21<279:50:51,  6.46s/it]
 31%|███       | 69059/225000 [140:17:26<262:52:52,  6.07s/it]
 31%|███       | 69060/225000 [140:17:33<273:13:40,  6.31s/it]
 31%|███       | 69061/225000 [140:17:37<238:01:02,  5.49s/it]
 31%|███       | 69062/225000 [140:17:41<215:51:12,  4.98s/it]
 31%|███       | 69063/225000 [140:17:48<249:53:09,  5.77s/it]
 31%|███       | 69064/225000 [140:17:59<309:53:17,  7.15s/it]
 31%|███       | 69065/225000 [140:18:02<257:54:20,  5.95s/it]
 31%|███       | 69066/225000 [140:18:09<280:35:31,  6.48s/it]
 31%|███       | 69067/225000 [140:18:14<256:33:39,  5.92s/it]
 31%|███       | 69068/225000 [140:18:20<261:56:22,  6.05s/it]
 31%|███       | 69069/225000 [140:18:25<240:46:41,  5.56s/it]
 31%|███       | 69070/225000 [140:18:29<222:54:41,  5.15s/it]
 31%|███       | 69071/225000 [140:18:34<218:53:13,  5.05s/it]
 31%|███       | 69072/225000 [140:18:46<314:20:36,  7.26s/it]
 31%|███       | 69073/225000 [140:18:51<282:50:34,  6.53s/it]
 31%|███       | 69074/225000 [140:18:56<259:06:04,  5.98s/it]
 31%|███       | 69075/225000 [140:19:00<234:21:01,  5.41s/it]
 31%|███       | 69076/225000 [140:19:09<277:49:32,  6.41s/it]
 31%|███       | 69077/225000 [140:19:18<311:26:56,  7.19s/it]
 31%|███       | 69078/225000 [140:19:21<265:48:39,  6.14s/it]
 31%|███       | 69079/225000 [140:19:30<298:11:48,  6.88s/it]
 31%|███       | 69080/225000 [140:19:35<279:40:37,  6.46s/it]
 31%|███       | 69081/225000 [140:19:38<236:23:28,  5.46s/it]
 31%|███       | 69082/225000 [140:19:47<282:47:57,  6.53s/it]
 31%|███       | 69083/225000 [140:19:51<242:16:44,  5.59s/it]
 31%|███       | 69084/225000 [140:20:01<294:47:22,  6.81s/it]
 31%|███       | 69085/225000 [140:20:06<278:03:55,  6.42s/it]
 31%|███       | 69086/225000 [140:20:11<253:52:30,  5.86s/it]
 31%|███       | 69087/225000 [140:20:15<235:42:55,  5.44s/it]
 31%|███       | 69088/225000 [140:20:20<223:33:43,  5.16s/it]
 31%|███       | 69089/225000 [140:20:25<224:06:52,  5.17s/it]
 31%|███       | 69090/225000 [140:20:36<303:53:49,  7.02s/it]
 31%|███       | 69091/225000 [140:20:42<287:18:12,  6.63s/it]
 31%|███       | 69092/225000 [140:20:47<263:49:01,  6.09s/it]
 31%|███       | 69093/225000 [140:20:52<249:12:58,  5.75s/it]
 31%|███       | 69094/225000 [140:20:56<233:31:26,  5.39s/it]
 31%|███       | 69095/225000 [140:21:01<228:22:47,  5.27s/it]
 31%|███       | 69096/225000 [140:21:06<225:16:53,  5.20s/it]
 31%|███       | 69097/225000 [140:21:11<218:52:00,  5.05s/it]
 31%|███       | 69098/225000 [140:21:17<228:38:12,  5.28s/it]
 31%|███       | 69099/225000 [140:21:28<304:22:21,  7.03s/it]
 31%|███       | 69100/225000 [140:21:36<318:25:18,  7.35s/it]
                                                              
{'loss': '0.7423', 'grad_norm': '0.7352', 'learning_rate': '4.125e-05', 'epoch': '0.3071', 'num_input_tokens_seen': 233002696, 'train_runtime': '5.053e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69100/225000 [140:21:36<318:25:18,  7.35s/it]
 31%|███       | 69101/225000 [140:21:40<269:30:58,  6.22s/it]
 31%|███       | 69102/225000 [140:21:49<315:26:46,  7.28s/it]
 31%|███       | 69103/225000 [140:21:55<288:53:58,  6.67s/it]
 31%|███       | 69104/225000 [140:22:00<271:43:54,  6.27s/it]
 31%|███       | 69105/225000 [140:22:05<258:20:04,  5.97s/it]
 31%|███       | 69106/225000 [140:22:10<243:41:25,  5.63s/it]
 31%|███       | 69107/225000 [140:22:15<236:29:10,  5.46s/it]
 31%|███       | 69108/225000 [140:22:24<283:33:18,  6.55s/it]
 31%|███       | 69109/225000 [140:22:33<319:06:15,  7.37s/it]
 31%|███       | 69110/225000 [140:22:37<271:11:00,  6.26s/it]
 31%|███       | 69111/225000 [140:22:45<293:13:01,  6.77s/it]
 31%|███       | 69112/225000 [140:22:51<284:03:34,  6.56s/it]
 31%|███       | 69113/225000 [140:22:58<282:28:37,  6.52s/it]
 31%|███       | 69114/225000 [140:23:05<299:48:36,  6.92s/it]
 31%|███       | 69115/225000 [140:23:10<271:16:19,  6.26s/it]
 31%|███       | 69116/225000 [140:23:17<282:51:10,  6.53s/it]
 31%|███       | 69117/225000 [140:23:22<265:03:12,  6.12s/it]
 31%|███       | 69118/225000 [140:23:29<265:43:34,  6.14s/it]
 31%|███       | 69119/225000 [140:23:36<276:47:38,  6.39s/it]
 31%|███       | 69120/225000 [140:23:39<241:32:30,  5.58s/it]
 31%|███       | 69121/225000 [140:23:44<230:24:05,  5.32s/it]
 31%|███       | 69122/225000 [140:23:48<216:53:26,  5.01s/it]
 31%|███       | 69123/225000 [140:24:01<311:31:28,  7.19s/it]
 31%|███       | 69124/225000 [140:24:05<272:19:36,  6.29s/it]
 31%|███       | 69125/225000 [140:24:12<278:28:18,  6.43s/it]
 31%|███       | 69126/225000 [140:24:17<268:47:14,  6.21s/it]
 31%|███       | 69127/225000 [140:24:22<245:13:14,  5.66s/it]
 31%|███       | 69128/225000 [140:24:27<237:49:39,  5.49s/it]
 31%|███       | 69129/225000 [140:24:38<309:12:41,  7.14s/it]
 31%|███       | 69130/225000 [140:24:43<289:43:12,  6.69s/it]
 31%|███       | 69131/225000 [140:24:50<295:33:49,  6.83s/it]
 31%|███       | 69132/225000 [140:24:57<291:33:30,  6.73s/it]
 31%|███       | 69133/225000 [140:25:01<261:13:56,  6.03s/it]
 31%|███       | 69134/225000 [140:25:08<268:02:00,  6.19s/it]
 31%|███       | 69135/225000 [140:25:12<243:54:38,  5.63s/it]
 31%|███       | 69136/225000 [140:25:22<298:57:45,  6.91s/it]
 31%|███       | 69137/225000 [140:25:26<264:45:26,  6.12s/it]
 31%|███       | 69138/225000 [140:25:31<240:56:42,  5.57s/it]
 31%|███       | 69139/225000 [140:25:35<229:16:02,  5.30s/it]
 31%|███       | 69140/225000 [140:25:40<221:25:07,  5.11s/it]
 31%|███       | 69141/225000 [140:25:45<216:31:55,  5.00s/it]
 31%|███       | 69142/225000 [140:25:50<214:34:24,  4.96s/it]
 31%|███       | 69143/225000 [140:26:01<291:37:02,  6.74s/it]
 31%|███       | 69144/225000 [140:26:10<320:47:00,  7.41s/it]
 31%|███       | 69145/225000 [140:26:12<261:14:26,  6.03s/it]
 31%|███       | 69146/225000 [140:26:22<305:14:38,  7.05s/it]
 31%|███       | 69147/225000 [140:26:26<270:54:53,  6.26s/it]
 31%|███       | 69148/225000 [140:26:31<251:50:51,  5.82s/it]
 31%|███       | 69149/225000 [140:26:35<229:03:32,  5.29s/it]
 31%|███       | 69150/225000 [140:26:39<211:22:40,  4.88s/it]
                                                              
{'loss': '0.7429', 'grad_norm': '1.346', 'learning_rate': '4.124e-05', 'epoch': '0.3073', 'num_input_tokens_seen': 233137280, 'train_runtime': '5.056e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69150/225000 [140:26:39<211:22:40,  4.88s/it]
 31%|███       | 69151/225000 [140:26:44<208:53:55,  4.83s/it]
 31%|███       | 69152/225000 [140:26:48<206:36:35,  4.77s/it]
 31%|███       | 69153/225000 [140:26:59<284:41:51,  6.58s/it]
 31%|███       | 69154/225000 [140:27:04<261:26:44,  6.04s/it]
 31%|███       | 69155/225000 [140:27:08<232:57:32,  5.38s/it]
 31%|███       | 69156/225000 [140:27:14<249:27:40,  5.76s/it]
 31%|███       | 69157/225000 [140:27:22<267:28:06,  6.18s/it]
 31%|███       | 69158/225000 [140:27:27<254:46:31,  5.89s/it]
 31%|███       | 69159/225000 [140:27:31<238:17:17,  5.50s/it]
 31%|███       | 69160/225000 [140:27:39<271:15:22,  6.27s/it]
 31%|███       | 69161/225000 [140:27:47<284:01:02,  6.56s/it]
 31%|███       | 69162/225000 [140:27:50<246:56:32,  5.70s/it]
 31%|███       | 69163/225000 [140:27:55<238:49:27,  5.52s/it]
 31%|███       | 69164/225000 [140:28:05<287:38:43,  6.64s/it]
 31%|███       | 69165/225000 [140:28:13<313:46:50,  7.25s/it]
 31%|███       | 69166/225000 [140:28:17<264:12:09,  6.10s/it]
 31%|███       | 69167/225000 [140:28:24<281:22:27,  6.50s/it]
 31%|███       | 69168/225000 [140:28:27<239:19:08,  5.53s/it]
 31%|███       | 69169/225000 [140:28:32<224:25:06,  5.18s/it]
 31%|███       | 69170/225000 [140:28:37<229:25:17,  5.30s/it]
 31%|███       | 69171/225000 [140:28:42<220:30:46,  5.09s/it]
 31%|███       | 69172/225000 [140:28:53<303:07:41,  7.00s/it]
 31%|███       | 69173/225000 [140:28:59<278:24:32,  6.43s/it]
 31%|███       | 69174/225000 [140:29:03<257:51:00,  5.96s/it]
 31%|███       | 69175/225000 [140:29:10<264:28:00,  6.11s/it]
 31%|███       | 69176/225000 [140:29:14<244:48:24,  5.66s/it]
 31%|███       | 69177/225000 [140:29:19<225:10:43,  5.20s/it]
 31%|███       | 69178/225000 [140:29:23<211:08:08,  4.88s/it]
 31%|███       | 69179/225000 [140:29:27<207:29:29,  4.79s/it]
 31%|███       | 69180/225000 [140:29:32<209:42:17,  4.84s/it]
 31%|███       | 69181/225000 [140:29:42<272:12:49,  6.29s/it]
 31%|███       | 69182/225000 [140:29:54<342:12:11,  7.91s/it]
 31%|███       | 69183/225000 [140:29:57<286:21:52,  6.62s/it]
 31%|███       | 69184/225000 [140:30:04<282:10:17,  6.52s/it]
 31%|███       | 69185/225000 [140:30:10<283:35:51,  6.55s/it]
 31%|███       | 69186/225000 [140:30:15<259:12:02,  5.99s/it]
 31%|███       | 69187/225000 [140:30:19<237:27:30,  5.49s/it]
 31%|███       | 69188/225000 [140:30:24<227:19:01,  5.25s/it]
 31%|███       | 69189/225000 [140:30:36<321:03:05,  7.42s/it]
 31%|███       | 69190/225000 [140:30:40<266:36:07,  6.16s/it]
 31%|███       | 69191/225000 [140:30:44<244:52:02,  5.66s/it]
 31%|███       | 69192/225000 [140:30:53<284:21:20,  6.57s/it]
 31%|███       | 69193/225000 [140:30:57<253:00:50,  5.85s/it]
 31%|███       | 69194/225000 [140:31:05<277:29:36,  6.41s/it]
 31%|███       | 69195/225000 [140:31:08<238:41:12,  5.52s/it]
 31%|███       | 69196/225000 [140:31:14<250:16:56,  5.78s/it]
 31%|███       | 69197/225000 [140:31:21<257:27:25,  5.95s/it]
 31%|███       | 69198/225000 [140:31:25<233:39:59,  5.40s/it]
 31%|███       | 69199/225000 [140:31:30<235:11:40,  5.43s/it]
 31%|███       | 69200/225000 [140:31:38<267:40:22,  6.18s/it]
                                                              
{'loss': '0.7465', 'grad_norm': '0.8873', 'learning_rate': '4.122e-05', 'epoch': '0.3076', 'num_input_tokens_seen': 233267960, 'train_runtime': '5.059e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69200/225000 [140:31:38<267:40:22,  6.18s/it]
 31%|███       | 69201/225000 [140:31:42<235:13:53,  5.44s/it]
 31%|███       | 69202/225000 [140:31:46<218:12:59,  5.04s/it]
 31%|███       | 69203/225000 [140:31:51<211:02:39,  4.88s/it]
 31%|███       | 69204/225000 [140:32:04<325:15:54,  7.52s/it]
 31%|███       | 69205/225000 [140:32:08<272:18:17,  6.29s/it]
 31%|███       | 69206/225000 [140:32:12<242:15:49,  5.60s/it]
 31%|███       | 69207/225000 [140:32:15<212:03:35,  4.90s/it]
 31%|███       | 69208/225000 [140:32:24<262:48:10,  6.07s/it]
 31%|███       | 69209/225000 [140:32:30<260:16:56,  6.01s/it]
 31%|███       | 69210/225000 [140:32:33<227:55:06,  5.27s/it]
 31%|███       | 69211/225000 [140:32:38<216:38:01,  5.01s/it]
 31%|███       | 69212/225000 [140:32:41<199:26:40,  4.61s/it]
 31%|███       | 69213/225000 [140:32:50<256:18:24,  5.92s/it]
 31%|███       | 69214/225000 [140:32:56<250:19:06,  5.78s/it]
 31%|███       | 69215/225000 [140:33:01<239:36:55,  5.54s/it]
 31%|███       | 69216/225000 [140:33:05<229:10:02,  5.30s/it]
 31%|███       | 69217/225000 [140:33:10<224:10:48,  5.18s/it]
 31%|███       | 69218/225000 [140:33:16<230:50:49,  5.33s/it]
 31%|███       | 69219/225000 [140:33:21<230:37:26,  5.33s/it]
 31%|███       | 69220/225000 [140:33:26<225:21:08,  5.21s/it]
 31%|███       | 69221/225000 [140:33:32<225:47:54,  5.22s/it]
 31%|███       | 69222/225000 [140:33:43<307:55:46,  7.12s/it]
 31%|███       | 69223/225000 [140:33:53<343:23:06,  7.94s/it]
 31%|███       | 69224/225000 [140:33:57<298:48:14,  6.91s/it]
 31%|███       | 69225/225000 [140:34:04<291:02:16,  6.73s/it]
 31%|███       | 69226/225000 [140:34:10<281:03:41,  6.50s/it]
 31%|███       | 69227/225000 [140:34:13<244:06:59,  5.64s/it]
 31%|███       | 69228/225000 [140:34:18<230:06:39,  5.32s/it]
 31%|███       | 69229/225000 [140:34:22<217:51:58,  5.04s/it]
 31%|███       | 69230/225000 [140:34:26<203:28:37,  4.70s/it]
 31%|███       | 69231/225000 [140:34:31<202:31:33,  4.68s/it]
 31%|███       | 69232/225000 [140:34:36<202:04:00,  4.67s/it]
 31%|███       | 69233/225000 [140:34:48<307:30:10,  7.11s/it]
 31%|███       | 69234/225000 [140:34:53<270:17:14,  6.25s/it]
 31%|███       | 69235/225000 [140:35:01<302:59:50,  7.00s/it]
 31%|███       | 69236/225000 [140:35:05<257:21:43,  5.95s/it]
 31%|███       | 69237/225000 [140:35:15<313:42:23,  7.25s/it]
 31%|███       | 69238/225000 [140:35:20<284:22:36,  6.57s/it]
 31%|███       | 69239/225000 [140:35:24<252:38:11,  5.84s/it]
 31%|███       | 69240/225000 [140:35:28<232:05:43,  5.36s/it]
 31%|███       | 69241/225000 [140:35:33<217:33:35,  5.03s/it]
 31%|███       | 69242/225000 [140:35:37<213:55:36,  4.94s/it]
 31%|███       | 69243/225000 [140:35:44<237:20:33,  5.49s/it]
 31%|███       | 69244/225000 [140:35:57<327:23:53,  7.57s/it]
 31%|███       | 69245/225000 [140:36:00<268:26:48,  6.20s/it]
 31%|███       | 69246/225000 [140:36:03<235:28:17,  5.44s/it]
 31%|███       | 69247/225000 [140:36:11<262:50:00,  6.08s/it]
 31%|███       | 69248/225000 [140:36:16<254:28:48,  5.88s/it]
 31%|███       | 69249/225000 [140:36:20<221:06:20,  5.11s/it]
 31%|███       | 69250/225000 [140:36:24<210:10:59,  4.86s/it]
                                                              
{'loss': '0.7705', 'grad_norm': '0.913', 'learning_rate': '4.121e-05', 'epoch': '0.3078', 'num_input_tokens_seen': 233397080, 'train_runtime': '5.062e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69250/225000 [140:36:24<210:10:59,  4.86s/it]
 31%|███       | 69251/225000 [140:36:35<295:58:26,  6.84s/it]
 31%|███       | 69252/225000 [140:36:40<272:26:54,  6.30s/it]
 31%|███       | 69253/225000 [140:36:44<243:24:32,  5.63s/it]
 31%|███       | 69254/225000 [140:36:49<224:00:00,  5.18s/it]
 31%|███       | 69255/225000 [140:36:54<221:11:39,  5.11s/it]
 31%|███       | 69256/225000 [140:37:04<294:52:50,  6.82s/it]
 31%|███       | 69257/225000 [140:37:07<245:48:48,  5.68s/it]
 31%|███       | 69258/225000 [140:37:12<237:26:44,  5.49s/it]
 31%|███       | 69259/225000 [140:37:18<240:22:55,  5.56s/it]
 31%|███       | 69260/225000 [140:37:24<238:32:43,  5.51s/it]
 31%|███       | 69261/225000 [140:37:29<233:07:06,  5.39s/it]
 31%|███       | 69262/225000 [140:37:38<278:43:10,  6.44s/it]
 31%|███       | 69263/225000 [140:37:48<331:10:15,  7.66s/it]
 31%|███       | 69264/225000 [140:37:51<274:39:47,  6.35s/it]
 31%|███       | 69265/225000 [140:37:59<293:51:32,  6.79s/it]
 31%|███       | 69266/225000 [140:38:03<259:05:19,  5.99s/it]
 31%|███       | 69267/225000 [140:38:10<268:54:33,  6.22s/it]
 31%|███       | 69268/225000 [140:38:18<288:10:04,  6.66s/it]
 31%|███       | 69269/225000 [140:38:22<253:03:50,  5.85s/it]
 31%|███       | 69270/225000 [140:38:26<232:40:05,  5.38s/it]
 31%|███       | 69271/225000 [140:38:31<222:11:46,  5.14s/it]
 31%|███       | 69272/225000 [140:38:40<282:37:41,  6.53s/it]
 31%|███       | 69273/225000 [140:38:45<260:35:36,  6.02s/it]
 31%|███       | 69274/225000 [140:38:49<234:46:28,  5.43s/it]
 31%|███       | 69275/225000 [140:38:53<211:52:55,  4.90s/it]
 31%|███       | 69276/225000 [140:39:00<240:45:21,  5.57s/it]
 31%|███       | 69277/225000 [140:39:07<265:58:20,  6.15s/it]
 31%|███       | 69278/225000 [140:39:13<260:55:31,  6.03s/it]
 31%|███       | 69279/225000 [140:39:17<234:23:27,  5.42s/it]
 31%|███       | 69280/225000 [140:39:22<219:47:28,  5.08s/it]
 31%|███       | 69281/225000 [140:39:33<307:20:03,  7.11s/it]
 31%|███       | 69282/225000 [140:39:38<276:36:13,  6.39s/it]
 31%|███       | 69283/225000 [140:39:43<256:22:15,  5.93s/it]
 31%|███       | 69284/225000 [140:39:48<239:23:41,  5.53s/it]
 31%|███       | 69285/225000 [140:39:52<228:33:14,  5.28s/it]
 31%|███       | 69286/225000 [140:40:01<268:14:44,  6.20s/it]
 31%|███       | 69287/225000 [140:40:08<288:13:23,  6.66s/it]
 31%|███       | 69288/225000 [140:40:15<282:57:30,  6.54s/it]
 31%|███       | 69289/225000 [140:40:19<252:40:56,  5.84s/it]
 31%|███       | 69290/225000 [140:40:24<241:34:53,  5.59s/it]
 31%|███       | 69291/225000 [140:40:28<219:50:38,  5.08s/it]
 31%|███       | 69292/225000 [140:40:39<299:56:29,  6.93s/it]
 31%|███       | 69293/225000 [140:40:43<266:54:47,  6.17s/it]
 31%|███       | 69294/225000 [140:40:47<238:09:46,  5.51s/it]
 31%|███       | 69295/225000 [140:40:58<307:09:13,  7.10s/it]
 31%|███       | 69296/225000 [140:41:01<255:39:46,  5.91s/it]
 31%|███       | 69297/225000 [140:41:05<232:52:56,  5.38s/it]
 31%|███       | 69298/225000 [140:41:10<229:05:27,  5.30s/it]
 31%|███       | 69299/225000 [140:41:15<224:06:16,  5.18s/it]
 31%|███       | 69300/225000 [140:41:19<207:02:13,  4.79s/it]
                                                              
{'loss': '0.7446', 'grad_norm': '0.9939', 'learning_rate': '4.12e-05', 'epoch': '0.308', 'num_input_tokens_seen': 233527512, 'train_runtime': '5.065e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69300/225000 [140:41:19<207:02:13,  4.79s/it]
 31%|███       | 69301/225000 [140:41:30<281:37:09,  6.51s/it]
 31%|███       | 69302/225000 [140:41:36<279:57:53,  6.47s/it]
 31%|███       | 69303/225000 [140:41:40<246:10:19,  5.69s/it]
 31%|███       | 69304/225000 [140:41:49<289:15:10,  6.69s/it]
 31%|███       | 69305/225000 [140:41:53<255:51:18,  5.92s/it]
 31%|███       | 69306/225000 [140:42:00<262:24:07,  6.07s/it]
 31%|███       | 69307/225000 [140:42:04<246:55:01,  5.71s/it]
 31%|███       | 69308/225000 [140:42:10<246:05:48,  5.69s/it]
 31%|███       | 69309/225000 [140:42:13<215:45:43,  4.99s/it]
 31%|███       | 69310/225000 [140:42:22<266:07:04,  6.15s/it]
 31%|███       | 69311/225000 [140:42:26<229:37:24,  5.31s/it]
 31%|███       | 69312/225000 [140:42:29<209:58:42,  4.86s/it]
 31%|███       | 69313/225000 [140:42:33<196:23:32,  4.54s/it]
 31%|███       | 69314/225000 [140:42:42<255:07:41,  5.90s/it]
 31%|███       | 69315/225000 [140:42:50<279:50:18,  6.47s/it]
 31%|███       | 69316/225000 [140:42:53<237:19:52,  5.49s/it]
 31%|███       | 69317/225000 [140:43:01<263:42:01,  6.10s/it]
 31%|███       | 69318/225000 [140:43:06<252:48:31,  5.85s/it]
 31%|███       | 69319/225000 [140:43:15<293:50:25,  6.79s/it]
 31%|███       | 69320/225000 [140:43:19<261:14:57,  6.04s/it]
 31%|███       | 69321/225000 [140:43:23<225:02:09,  5.20s/it]
 31%|███       | 69322/225000 [140:43:31<259:27:04,  6.00s/it]
 31%|███       | 69323/225000 [140:43:36<253:32:16,  5.86s/it]
 31%|███       | 69324/225000 [140:43:40<222:57:09,  5.16s/it]
 31%|███       | 69325/225000 [140:43:43<204:32:54,  4.73s/it]
 31%|███       | 69326/225000 [140:43:49<221:15:05,  5.12s/it]
 31%|███       | 69327/225000 [140:44:01<300:18:57,  6.94s/it]
 31%|███       | 69328/225000 [140:44:05<261:42:16,  6.05s/it]
 31%|███       | 69329/225000 [140:44:08<232:17:38,  5.37s/it]
 31%|███       | 69330/225000 [140:44:20<308:47:30,  7.14s/it]
 31%|███       | 69331/225000 [140:44:25<281:04:41,  6.50s/it]
 31%|███       | 69332/225000 [140:44:29<250:27:16,  5.79s/it]
 31%|███       | 69333/225000 [140:44:33<227:58:09,  5.27s/it]
 31%|███       | 69334/225000 [140:44:38<224:05:52,  5.18s/it]
 31%|███       | 69335/225000 [140:44:47<279:27:03,  6.46s/it]
 31%|███       | 69336/225000 [140:44:51<249:30:53,  5.77s/it]
 31%|███       | 69337/225000 [140:44:55<225:40:48,  5.22s/it]
 31%|███       | 69338/225000 [140:44:59<204:31:43,  4.73s/it]
 31%|███       | 69339/225000 [140:45:12<311:50:45,  7.21s/it]
 31%|███       | 69340/225000 [140:45:15<261:38:18,  6.05s/it]
 31%|███       | 69341/225000 [140:45:23<289:58:16,  6.71s/it]
 31%|███       | 69342/225000 [140:45:29<281:20:37,  6.51s/it]
 31%|███       | 69343/225000 [140:45:34<257:47:50,  5.96s/it]
 31%|███       | 69344/225000 [140:45:39<240:47:20,  5.57s/it]
 31%|███       | 69345/225000 [140:45:43<225:55:10,  5.23s/it]
 31%|███       | 69346/225000 [140:45:48<217:59:02,  5.04s/it]
 31%|███       | 69347/225000 [140:45:52<205:21:14,  4.75s/it]
 31%|███       | 69348/225000 [140:45:57<214:38:20,  4.96s/it]
 31%|███       | 69349/225000 [140:46:09<299:13:07,  6.92s/it]
 31%|███       | 69350/225000 [140:46:14<278:59:57,  6.45s/it]
                                                              
{'loss': '0.765', 'grad_norm': '0.9338', 'learning_rate': '4.118e-05', 'epoch': '0.3082', 'num_input_tokens_seen': 233655456, 'train_runtime': '5.068e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69350/225000 [140:46:14<278:59:57,  6.45s/it]
 31%|███       | 69351/225000 [140:46:19<250:41:10,  5.80s/it]
 31%|███       | 69352/225000 [140:46:23<238:51:56,  5.52s/it]
 31%|███       | 69353/225000 [140:46:27<219:34:20,  5.08s/it]
 31%|███       | 69354/225000 [140:46:32<213:24:03,  4.94s/it]
 31%|███       | 69355/225000 [140:46:43<297:02:23,  6.87s/it]
 31%|███       | 69356/225000 [140:46:47<252:28:45,  5.84s/it]
 31%|███       | 69357/225000 [140:46:51<228:01:16,  5.27s/it]
 31%|███       | 69358/225000 [140:46:54<205:30:46,  4.75s/it]
 31%|███       | 69359/225000 [140:47:01<233:03:56,  5.39s/it]
 31%|███       | 69360/225000 [140:47:12<300:21:35,  6.95s/it]
 31%|███       | 69361/225000 [140:47:16<262:58:24,  6.08s/it]
 31%|███       | 69362/225000 [140:47:21<245:28:17,  5.68s/it]
 31%|███       | 69363/225000 [140:47:31<306:11:53,  7.08s/it]
 31%|███       | 69364/225000 [140:47:35<261:38:32,  6.05s/it]
 31%|███       | 69365/225000 [140:47:39<241:26:16,  5.58s/it]
 31%|███       | 69366/225000 [140:47:43<219:38:28,  5.08s/it]
 31%|███       | 69367/225000 [140:47:51<250:57:58,  5.81s/it]
 31%|███       | 69368/225000 [140:47:58<267:07:03,  6.18s/it]
 31%|███       | 69369/225000 [140:48:03<259:22:48,  6.00s/it]
 31%|███       | 69370/225000 [140:48:09<261:57:00,  6.06s/it]
 31%|███       | 69371/225000 [140:48:13<235:15:15,  5.44s/it]
 31%|███       | 69372/225000 [140:48:17<215:39:49,  4.99s/it]
 31%|███       | 69373/225000 [140:48:22<213:44:12,  4.94s/it]
 31%|███       | 69374/225000 [140:48:28<222:20:54,  5.14s/it]
 31%|███       | 69375/225000 [140:48:33<220:23:57,  5.10s/it]
 31%|███       | 69376/225000 [140:48:42<277:51:06,  6.43s/it]
 31%|███       | 69377/225000 [140:48:52<316:20:04,  7.32s/it]
 31%|███       | 69378/225000 [140:48:55<267:53:01,  6.20s/it]
 31%|███       | 69379/225000 [140:48:59<233:54:38,  5.41s/it]
 31%|███       | 69380/225000 [140:49:03<212:47:31,  4.92s/it]
 31%|███       | 69381/225000 [140:49:08<216:31:39,  5.01s/it]
 31%|███       | 69382/225000 [140:49:16<262:22:43,  6.07s/it]
 31%|███       | 69383/225000 [140:49:26<303:27:37,  7.02s/it]
 31%|███       | 69384/225000 [140:49:29<254:40:30,  5.89s/it]
 31%|███       | 69385/225000 [140:49:36<265:22:11,  6.14s/it]
 31%|███       | 69386/225000 [140:49:42<268:18:17,  6.21s/it]
 31%|███       | 69387/225000 [140:49:46<234:40:35,  5.43s/it]
 31%|███       | 69388/225000 [140:49:49<213:47:37,  4.95s/it]
 31%|███       | 69389/225000 [140:49:57<249:52:22,  5.78s/it]
 31%|███       | 69390/225000 [140:50:04<265:27:24,  6.14s/it]
 31%|███       | 69391/225000 [140:50:11<281:26:40,  6.51s/it]
 31%|███       | 69392/225000 [140:50:20<303:18:32,  7.02s/it]
 31%|███       | 69393/225000 [140:50:23<250:39:37,  5.80s/it]
 31%|███       | 69394/225000 [140:50:30<271:57:20,  6.29s/it]
 31%|███       | 69395/225000 [140:50:34<241:18:46,  5.58s/it]
 31%|███       | 69396/225000 [140:50:42<267:22:10,  6.19s/it]
 31%|███       | 69397/225000 [140:50:48<267:52:46,  6.20s/it]
 31%|███       | 69398/225000 [140:50:53<253:20:11,  5.86s/it]
 31%|███       | 69399/225000 [140:50:58<243:42:39,  5.64s/it]
 31%|███       | 69400/225000 [140:51:03<238:10:28,  5.51s/it]
                                                              
{'loss': '0.7494', 'grad_norm': '0.79', 'learning_rate': '4.117e-05', 'epoch': '0.3084', 'num_input_tokens_seen': 233785912, 'train_runtime': '5.071e+05', 'train_tokens_per_second': '461.1'}

 31%|███       | 69400/225000 [140:51:03<238:10:28,  5.51s/it]
 31%|███       | 69401/225000 [140:51:11<273:28:08,  6.33s/it]
 31%|███       | 69402/225000 [140:51:15<242:53:40,  5.62s/it]
 31%|███       | 69403/225000 [140:51:20<224:27:18,  5.19s/it]
 31%|███       | 69404/225000 [140:51:24<213:49:00,  4.95s/it]
 31%|███       | 69405/225000 [140:51:32<249:59:29,  5.78s/it]
 31%|███       | 69406/225000 [140:51:43<323:12:10,  7.48s/it]
 31%|███       | 69407/225000 [140:51:50<311:40:14,  7.21s/it]
 31%|███       | 69408/225000 [140:51:55<283:05:11,  6.55s/it]
 31%|███       | 69409/225000 [140:52:00<264:17:09,  6.11s/it]
 31%|███       | 69410/225000 [140:52:07<274:13:52,  6.35s/it]
 31%|███       | 69411/225000 [140:52:11<253:12:50,  5.86s/it]
 31%|███       | 69412/225000 [140:52:17<251:55:55,  5.83s/it]
 31%|███       | 69413/225000 [140:52:25<281:14:26,  6.51s/it]
 31%|███       | 69414/225000 [140:52:28<237:28:05,  5.49s/it]
 31%|███       | 69415/225000 [140:52:36<266:46:08,  6.17s/it]
 31%|███       | 69416/225000 [140:52:40<241:40:33,  5.59s/it]
 31%|███       | 69417/225000 [140:52:45<233:21:24,  5.40s/it]
 31%|███       | 69418/225000 [140:52:50<221:58:40,  5.14s/it]
 31%|███       | 69419/225000 [140:52:54<212:40:53,  4.92s/it]
 31%|███       | 69420/225000 [140:52:59<205:50:30,  4.76s/it]
 31%|███       | 69421/225000 [140:53:04<211:04:07,  4.88s/it]
 31%|███       | 69422/225000 [140:53:12<258:03:07,  5.97s/it]
 31%|███       | 69423/225000 [140:53:22<311:11:46,  7.20s/it]
 31%|███       | 69424/225000 [140:53:26<265:14:03,  6.14s/it]
 31%|███       | 69425/225000 [140:53:32<256:27:48,  5.93s/it]
 31%|███       | 69426/225000 [140:53:38<266:36:02,  6.17s/it]
 31%|███       | 69427/225000 [140:53:44<254:55:55,  5.90s/it]
 31%|███       | 69428/225000 [140:53:50<258:14:23,  5.98s/it]
 31%|███       | 69429/225000 [140:53:56<263:16:23,  6.09s/it]
 31%|███       | 69430/225000 [140:54:01<253:44:55,  5.87s/it]
 31%|███       | 69431/225000 [140:54:05<221:47:47,  5.13s/it]
 31%|███       | 69432/225000 [140:54:14<280:15:56,  6.49s/it]
 31%|███       | 69433/225000 [140:54:18<238:29:11,  5.52s/it]
 31%|███       | 69434/225000 [140:54:23<234:51:23,  5.43s/it]
 31%|███       | 69435/225000 [140:54:27<217:14:42,  5.03s/it]
 31%|███       | 69436/225000 [140:54:32<212:41:13,  4.92s/it]
 31%|███       | 69437/225000 [140:54:41<268:51:21,  6.22s/it]
 31%|███       | 69438/225000 [140:54:47<263:44:36,  6.10s/it]
 31%|███       | 69439/225000 [140:54:54<280:12:41,  6.48s/it]
 31%|███       | 69440/225000 [140:54:58<240:44:23,  5.57s/it]
 31%|███       | 69441/225000 [140:55:02<223:03:04,  5.16s/it]
 31%|███       | 69442/225000 [140:55:10<256:36:08,  5.94s/it]
 31%|███       | 69443/225000 [140:55:14<234:36:56,  5.43s/it]
 31%|███       | 69444/225000 [140:55:24<292:17:52,  6.76s/it]
 31%|███       | 69445/225000 [140:55:27<252:44:37,  5.85s/it]
 31%|███       | 69446/225000 [140:55:32<234:16:41,  5.42s/it]
 31%|███       | 69447/225000 [140:55:36<221:28:23,  5.13s/it]
 31%|███       | 69448/225000 [140:55:41<213:27:04,  4.94s/it]
 31%|███       | 69449/225000 [140:55:45<205:48:30,  4.76s/it]
 31%|███       | 69450/225000 [140:55:54<263:08:01,  6.09s/it]
                                                              
{'loss': '0.7538', 'grad_norm': '0.8884', 'learning_rate': '4.116e-05', 'epoch': '0.3087', 'num_input_tokens_seen': 233915160, 'train_runtime': '5.074e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69450/225000 [140:55:54<263:08:01,  6.09s/it]
 31%|███       | 69451/225000 [140:56:02<278:57:14,  6.46s/it]
 31%|███       | 69452/225000 [140:56:06<246:20:25,  5.70s/it]
 31%|███       | 69453/225000 [140:56:09<216:44:33,  5.02s/it]
 31%|███       | 69454/225000 [140:56:21<309:21:34,  7.16s/it]
 31%|███       | 69455/225000 [140:56:24<259:36:39,  6.01s/it]
 31%|███       | 69456/225000 [140:56:33<295:28:07,  6.84s/it]
 31%|███       | 69457/225000 [140:56:37<260:58:38,  6.04s/it]
 31%|███       | 69458/225000 [140:56:46<299:41:04,  6.94s/it]
 31%|███       | 69459/225000 [140:56:52<286:54:10,  6.64s/it]
 31%|███       | 69460/225000 [140:56:57<261:24:58,  6.05s/it]
 31%|███       | 69461/225000 [140:57:02<246:00:39,  5.69s/it]
 31%|███       | 69462/225000 [140:57:07<237:21:45,  5.49s/it]
 31%|███       | 69463/225000 [140:57:11<217:06:09,  5.02s/it]
 31%|███       | 69464/225000 [140:57:15<209:43:27,  4.85s/it]
 31%|███       | 69465/225000 [140:57:20<208:47:47,  4.83s/it]
 31%|███       | 69466/225000 [140:57:24<199:58:00,  4.63s/it]
 31%|███       | 69467/225000 [140:57:31<232:17:13,  5.38s/it]
 31%|███       | 69468/225000 [140:57:39<265:20:08,  6.14s/it]
 31%|███       | 69469/225000 [140:57:43<237:22:28,  5.49s/it]
 31%|███       | 69470/225000 [140:57:52<272:58:50,  6.32s/it]
 31%|███       | 69471/225000 [140:57:56<244:05:45,  5.65s/it]
 31%|███       | 69472/225000 [140:58:00<229:55:53,  5.32s/it]
 31%|███       | 69473/225000 [140:58:04<207:14:08,  4.80s/it]
 31%|███       | 69474/225000 [140:58:08<200:46:49,  4.65s/it]
 31%|███       | 69475/225000 [140:58:14<213:17:17,  4.94s/it]
 31%|███       | 69476/225000 [140:58:19<216:07:53,  5.00s/it]
 31%|███       | 69477/225000 [140:58:29<282:47:59,  6.55s/it]
 31%|███       | 69478/225000 [140:58:37<303:34:52,  7.03s/it]
 31%|███       | 69479/225000 [140:58:43<283:44:53,  6.57s/it]
 31%|███       | 69480/225000 [140:58:49<286:52:32,  6.64s/it]
 31%|███       | 69481/225000 [140:58:55<274:50:57,  6.36s/it]
 31%|███       | 69482/225000 [140:59:00<249:18:53,  5.77s/it]
 31%|███       | 69483/225000 [140:59:03<225:24:17,  5.22s/it]
 31%|███       | 69484/225000 [140:59:14<296:19:00,  6.86s/it]
 31%|███       | 69485/225000 [140:59:19<266:26:59,  6.17s/it]
 31%|███       | 69486/225000 [140:59:22<230:52:56,  5.34s/it]
 31%|███       | 69487/225000 [140:59:26<209:30:00,  4.85s/it]
 31%|███       | 69488/225000 [140:59:30<205:26:24,  4.76s/it]
 31%|███       | 69489/225000 [140:59:35<197:49:13,  4.58s/it]
 31%|███       | 69490/225000 [140:59:39<195:17:15,  4.52s/it]
 31%|███       | 69491/225000 [140:59:48<258:27:12,  5.98s/it]
 31%|███       | 69492/225000 [140:59:54<258:11:35,  5.98s/it]
 31%|███       | 69493/225000 [140:59:58<229:36:30,  5.32s/it]
 31%|███       | 69494/225000 [141:00:08<285:14:54,  6.60s/it]
 31%|███       | 69495/225000 [141:00:12<262:06:47,  6.07s/it]
 31%|███       | 69496/225000 [141:00:17<239:19:49,  5.54s/it]
 31%|███       | 69497/225000 [141:00:20<208:03:58,  4.82s/it]
 31%|███       | 69498/225000 [141:00:24<203:56:02,  4.72s/it]
 31%|███       | 69499/225000 [141:00:29<206:14:00,  4.77s/it]
 31%|███       | 69500/225000 [141:00:42<302:38:36,  7.01s/it]
                                                              
{'loss': '0.7563', 'grad_norm': '1.026', 'learning_rate': '4.114e-05', 'epoch': '0.3089', 'num_input_tokens_seen': 234041032, 'train_runtime': '5.076e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69500/225000 [141:00:42<302:38:36,  7.01s/it]
 31%|███       | 69501/225000 [141:00:47<286:12:17,  6.63s/it]
 31%|███       | 69502/225000 [141:00:53<276:09:21,  6.39s/it]
 31%|███       | 69503/225000 [141:00:58<254:08:22,  5.88s/it]
 31%|███       | 69504/225000 [141:01:02<230:59:26,  5.35s/it]
 31%|███       | 69505/225000 [141:01:07<221:43:48,  5.13s/it]
 31%|███       | 69506/225000 [141:01:12<225:30:31,  5.22s/it]
 31%|███       | 69507/225000 [141:01:17<220:40:28,  5.11s/it]
 31%|███       | 69508/225000 [141:01:22<218:03:07,  5.05s/it]
 31%|███       | 69509/225000 [141:01:32<291:09:24,  6.74s/it]
 31%|███       | 69510/225000 [141:01:41<311:24:57,  7.21s/it]
 31%|███       | 69511/225000 [141:01:44<254:10:14,  5.88s/it]
 31%|███       | 69512/225000 [141:01:50<267:00:40,  6.18s/it]
 31%|███       | 69513/225000 [141:01:55<252:12:59,  5.84s/it]
 31%|███       | 69514/225000 [141:02:00<234:07:55,  5.42s/it]
 31%|███       | 69515/225000 [141:02:04<222:48:25,  5.16s/it]
 31%|███       | 69516/225000 [141:02:09<211:33:44,  4.90s/it]
 31%|███       | 69517/225000 [141:02:13<202:06:13,  4.68s/it]
 31%|███       | 69518/225000 [141:02:19<226:11:54,  5.24s/it]
 31%|███       | 69519/225000 [141:02:31<308:05:31,  7.13s/it]
 31%|███       | 69520/225000 [141:02:34<257:14:49,  5.96s/it]
 31%|███       | 69521/225000 [141:02:38<223:45:32,  5.18s/it]
 31%|███       | 69522/225000 [141:02:47<280:25:03,  6.49s/it]
 31%|███       | 69523/225000 [141:02:52<262:59:43,  6.09s/it]
 31%|███       | 69524/225000 [141:02:56<226:35:03,  5.25s/it]
 31%|███       | 69525/225000 [141:03:04<274:38:29,  6.36s/it]
 31%|███       | 69526/225000 [141:03:10<266:49:26,  6.18s/it]
 31%|███       | 69527/225000 [141:03:15<247:54:38,  5.74s/it]
 31%|███       | 69528/225000 [141:03:18<217:18:34,  5.03s/it]
 31%|███       | 69529/225000 [141:03:25<244:47:07,  5.67s/it]
 31%|███       | 69530/225000 [141:03:35<290:37:11,  6.73s/it]
 31%|███       | 69531/225000 [141:03:39<260:34:20,  6.03s/it]
 31%|███       | 69532/225000 [141:03:46<269:38:54,  6.24s/it]
 31%|███       | 69533/225000 [141:03:50<237:45:05,  5.51s/it]
 31%|███       | 69534/225000 [141:03:54<228:47:55,  5.30s/it]
 31%|███       | 69535/225000 [141:04:01<246:24:13,  5.71s/it]
 31%|███       | 69536/225000 [141:04:05<221:29:23,  5.13s/it]
 31%|███       | 69537/225000 [141:04:13<257:27:26,  5.96s/it]
 31%|███       | 69538/225000 [141:04:16<226:56:03,  5.26s/it]
 31%|███       | 69539/225000 [141:04:20<208:32:11,  4.83s/it]
 31%|███       | 69540/225000 [141:04:30<278:08:35,  6.44s/it]
 31%|███       | 69541/225000 [141:04:33<228:39:34,  5.30s/it]
 31%|███       | 69542/225000 [141:04:36<197:49:17,  4.58s/it]
 31%|███       | 69543/225000 [141:04:40<192:34:11,  4.46s/it]
 31%|███       | 69544/225000 [141:04:51<273:16:00,  6.33s/it]
 31%|███       | 69545/225000 [141:04:54<231:11:26,  5.35s/it]
 31%|███       | 69546/225000 [141:05:03<273:37:14,  6.34s/it]
 31%|███       | 69547/225000 [141:05:06<239:11:42,  5.54s/it]
 31%|███       | 69548/225000 [141:05:11<226:46:21,  5.25s/it]
 31%|███       | 69549/225000 [141:05:19<266:55:04,  6.18s/it]
 31%|███       | 69550/225000 [141:05:22<228:40:19,  5.30s/it]
                                                              
{'loss': '0.7412', 'grad_norm': '1.019', 'learning_rate': '4.113e-05', 'epoch': '0.3091', 'num_input_tokens_seen': 234162928, 'train_runtime': '5.079e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69550/225000 [141:05:22<228:40:19,  5.30s/it]
 31%|███       | 69551/225000 [141:05:27<224:21:16,  5.20s/it]
 31%|███       | 69552/225000 [141:05:36<273:30:20,  6.33s/it]
 31%|███       | 69553/225000 [141:05:39<231:39:45,  5.37s/it]
 31%|███       | 69554/225000 [141:05:47<266:02:01,  6.16s/it]
 31%|███       | 69555/225000 [141:05:53<261:45:52,  6.06s/it]
 31%|███       | 69556/225000 [141:05:58<244:52:35,  5.67s/it]
 31%|███       | 69557/225000 [141:06:03<229:33:33,  5.32s/it]
 31%|███       | 69558/225000 [141:06:06<206:11:49,  4.78s/it]
 31%|███       | 69559/225000 [141:06:11<205:05:19,  4.75s/it]
 31%|███       | 69560/225000 [141:06:16<207:00:49,  4.79s/it]
 31%|███       | 69561/225000 [141:06:28<305:28:10,  7.07s/it]
 31%|███       | 69562/225000 [141:06:34<290:40:14,  6.73s/it]
 31%|███       | 69563/225000 [141:06:41<290:43:12,  6.73s/it]
 31%|███       | 69564/225000 [141:06:48<293:35:31,  6.80s/it]
 31%|███       | 69565/225000 [141:06:54<293:44:10,  6.80s/it]
 31%|███       | 69566/225000 [141:06:58<257:41:47,  5.97s/it]
 31%|███       | 69567/225000 [141:07:02<228:36:52,  5.29s/it]
 31%|███       | 69568/225000 [141:07:06<207:57:15,  4.82s/it]
 31%|███       | 69569/225000 [141:07:11<208:25:50,  4.83s/it]
 31%|███       | 69570/225000 [141:07:19<249:11:05,  5.77s/it]
 31%|███       | 69571/225000 [141:07:23<234:19:14,  5.43s/it]
 31%|███       | 69572/225000 [141:07:28<222:59:14,  5.16s/it]
 31%|███       | 69573/225000 [141:07:32<214:57:19,  4.98s/it]
 31%|███       | 69574/225000 [141:07:36<196:33:16,  4.55s/it]
 31%|███       | 69575/225000 [141:07:43<223:38:50,  5.18s/it]
 31%|███       | 69576/225000 [141:07:48<231:59:39,  5.37s/it]
 31%|███       | 69577/225000 [141:07:54<232:30:10,  5.39s/it]
 31%|███       | 69578/225000 [141:08:00<242:34:47,  5.62s/it]
 31%|███       | 69579/225000 [141:08:10<292:19:05,  6.77s/it]
 31%|███       | 69580/225000 [141:08:13<248:25:44,  5.75s/it]
 31%|███       | 69581/225000 [141:08:16<217:45:33,  5.04s/it]
 31%|███       | 69582/225000 [141:08:20<200:46:14,  4.65s/it]
 31%|███       | 69583/225000 [141:08:31<285:30:06,  6.61s/it]
 31%|███       | 69584/225000 [141:08:35<254:59:28,  5.91s/it]
 31%|███       | 69585/225000 [141:08:39<220:18:32,  5.10s/it]
 31%|███       | 69586/225000 [141:08:47<267:00:19,  6.18s/it]
 31%|███       | 69587/225000 [141:08:51<236:22:37,  5.48s/it]
 31%|███       | 69588/225000 [141:08:56<222:43:10,  5.16s/it]
 31%|███       | 69589/225000 [141:09:02<238:34:14,  5.53s/it]
 31%|███       | 69590/225000 [141:09:12<289:50:43,  6.71s/it]
 31%|███       | 69591/225000 [141:09:16<257:33:09,  5.97s/it]
 31%|███       | 69592/225000 [141:09:26<307:04:07,  7.11s/it]
 31%|███       | 69593/225000 [141:09:31<290:32:39,  6.73s/it]
 31%|███       | 69594/225000 [141:09:35<252:11:39,  5.84s/it]
 31%|███       | 69595/225000 [141:09:42<261:27:09,  6.06s/it]
 31%|███       | 69596/225000 [141:09:49<276:43:44,  6.41s/it]
 31%|███       | 69597/225000 [141:09:53<240:48:12,  5.58s/it]
 31%|███       | 69598/225000 [141:09:57<225:27:51,  5.22s/it]
 31%|███       | 69599/225000 [141:10:05<257:44:05,  5.97s/it]
 31%|███       | 69600/225000 [141:10:10<253:26:05,  5.87s/it]
                                                              
{'loss': '0.7386', 'grad_norm': '0.8402', 'learning_rate': '4.111e-05', 'epoch': '0.3093', 'num_input_tokens_seen': 234287096, 'train_runtime': '5.082e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69600/225000 [141:10:10<253:26:05,  5.87s/it]
 31%|███       | 69601/225000 [141:10:17<261:27:25,  6.06s/it]
 31%|███       | 69602/225000 [141:10:21<234:56:48,  5.44s/it]
 31%|███       | 69603/225000 [141:10:25<215:24:48,  4.99s/it]
 31%|███       | 69604/225000 [141:10:28<189:34:25,  4.39s/it]
 31%|███       | 69605/225000 [141:10:38<264:34:25,  6.13s/it]
 31%|███       | 69606/225000 [141:10:42<243:19:35,  5.64s/it]
 31%|███       | 69607/225000 [141:10:46<217:59:44,  5.05s/it]
 31%|███       | 69608/225000 [141:10:50<208:56:24,  4.84s/it]
 31%|███       | 69609/225000 [141:10:55<211:00:19,  4.89s/it]
 31%|███       | 69610/225000 [141:10:59<189:56:10,  4.40s/it]
 31%|███       | 69611/225000 [141:11:02<179:50:12,  4.17s/it]
 31%|███       | 69612/225000 [141:11:13<263:12:58,  6.10s/it]
 31%|███       | 69613/225000 [141:11:20<271:49:21,  6.30s/it]
 31%|███       | 69614/225000 [141:11:23<237:26:10,  5.50s/it]
 31%|███       | 69615/225000 [141:11:27<216:59:37,  5.03s/it]
 31%|███       | 69616/225000 [141:11:32<214:29:33,  4.97s/it]
 31%|███       | 69617/225000 [141:11:36<204:14:43,  4.73s/it]
 31%|███       | 69618/225000 [141:11:49<301:43:56,  6.99s/it]
 31%|███       | 69619/225000 [141:11:52<256:20:32,  5.94s/it]
 31%|███       | 69620/225000 [141:11:58<262:04:48,  6.07s/it]
 31%|███       | 69621/225000 [141:12:04<256:45:03,  5.95s/it]
 31%|███       | 69622/225000 [141:12:08<236:07:39,  5.47s/it]
 31%|███       | 69623/225000 [141:12:13<224:25:41,  5.20s/it]
 31%|███       | 69624/225000 [141:12:21<259:04:12,  6.00s/it]
 31%|███       | 69625/225000 [141:12:30<299:37:41,  6.94s/it]
 31%|███       | 69626/225000 [141:12:34<257:19:26,  5.96s/it]
 31%|███       | 69627/225000 [141:12:41<276:41:53,  6.41s/it]
 31%|███       | 69628/225000 [141:12:45<240:09:07,  5.56s/it]
 31%|███       | 69629/225000 [141:12:54<282:45:13,  6.55s/it]
 31%|███       | 69630/225000 [141:12:58<255:28:59,  5.92s/it]
 31%|███       | 69631/225000 [141:13:01<215:33:35,  4.99s/it]
 31%|███       | 69632/225000 [141:13:09<252:40:04,  5.85s/it]
 31%|███       | 69633/225000 [141:13:12<224:11:02,  5.19s/it]
 31%|███       | 69634/225000 [141:13:20<250:31:22,  5.80s/it]
 31%|███       | 69635/225000 [141:13:23<218:03:48,  5.05s/it]
 31%|███       | 69636/225000 [141:13:29<231:55:21,  5.37s/it]
 31%|███       | 69637/225000 [141:13:35<239:21:11,  5.55s/it]
 31%|███       | 69638/225000 [141:13:40<235:30:39,  5.46s/it]
 31%|███       | 69639/225000 [141:13:45<228:50:25,  5.30s/it]
 31%|███       | 69640/225000 [141:13:49<206:21:34,  4.78s/it]
 31%|███       | 69641/225000 [141:13:52<189:54:20,  4.40s/it]
 31%|███       | 69642/225000 [141:14:04<284:06:15,  6.58s/it]
 31%|███       | 69643/225000 [141:14:08<247:40:36,  5.74s/it]
 31%|███       | 69644/225000 [141:14:12<230:29:41,  5.34s/it]
 31%|███       | 69645/225000 [141:14:16<208:39:09,  4.84s/it]
 31%|███       | 69646/225000 [141:14:20<196:48:30,  4.56s/it]
 31%|███       | 69647/225000 [141:14:24<194:26:50,  4.51s/it]
 31%|███       | 69648/225000 [141:14:33<252:07:02,  5.84s/it]
 31%|███       | 69649/225000 [141:14:42<296:16:39,  6.87s/it]
 31%|███       | 69650/225000 [141:14:48<283:12:30,  6.56s/it]
                                                              
{'loss': '0.7528', 'grad_norm': '1.136', 'learning_rate': '4.11e-05', 'epoch': '0.3096', 'num_input_tokens_seen': 234408920, 'train_runtime': '5.085e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69650/225000 [141:14:48<283:12:30,  6.56s/it]
 31%|███       | 69651/225000 [141:14:53<257:31:00,  5.97s/it]
 31%|███       | 69652/225000 [141:14:59<258:31:28,  5.99s/it]
 31%|███       | 69653/225000 [141:15:02<227:19:02,  5.27s/it]
 31%|███       | 69654/225000 [141:15:05<195:04:20,  4.52s/it]
 31%|███       | 69655/225000 [141:15:13<239:35:03,  5.55s/it]
 31%|███       | 69656/225000 [141:15:22<280:26:50,  6.50s/it]
 31%|███       | 69657/225000 [141:15:25<235:26:56,  5.46s/it]
 31%|███       | 69658/225000 [141:15:28<206:44:44,  4.79s/it]
 31%|███       | 69659/225000 [141:15:35<232:50:38,  5.40s/it]
 31%|███       | 69660/225000 [141:15:40<223:33:25,  5.18s/it]
 31%|███       | 69661/225000 [141:15:44<212:19:19,  4.92s/it]
 31%|███       | 69662/225000 [141:15:48<200:11:01,  4.64s/it]
 31%|███       | 69663/225000 [141:15:56<250:50:23,  5.81s/it]
 31%|███       | 69664/225000 [141:16:04<272:34:53,  6.32s/it]
 31%|███       | 69665/225000 [141:16:11<278:27:02,  6.45s/it]
 31%|███       | 69666/225000 [141:16:15<251:06:08,  5.82s/it]
 31%|███       | 69667/225000 [141:16:20<235:40:59,  5.46s/it]
 31%|███       | 69668/225000 [141:16:23<208:43:05,  4.84s/it]
 31%|███       | 69669/225000 [141:16:27<197:47:00,  4.58s/it]
 31%|███       | 69670/225000 [141:16:32<199:26:38,  4.62s/it]
 31%|███       | 69671/225000 [141:16:36<198:01:21,  4.59s/it]
 31%|███       | 69672/225000 [141:16:40<193:43:18,  4.49s/it]
 31%|███       | 69673/225000 [141:16:46<208:13:21,  4.83s/it]
 31%|███       | 69674/225000 [141:16:56<268:04:32,  6.21s/it]
 31%|███       | 69675/225000 [141:17:03<286:23:19,  6.64s/it]
 31%|███       | 69676/225000 [141:17:07<248:09:16,  5.75s/it]
 31%|███       | 69677/225000 [141:17:10<217:04:19,  5.03s/it]
 31%|███       | 69678/225000 [141:17:21<294:40:14,  6.83s/it]
 31%|███       | 69679/225000 [141:17:26<264:40:21,  6.13s/it]
 31%|███       | 69680/225000 [141:17:31<258:20:05,  5.99s/it]
 31%|███       | 69681/225000 [141:17:35<231:41:33,  5.37s/it]
 31%|███       | 69682/225000 [141:17:39<215:46:10,  5.00s/it]
 31%|███       | 69683/225000 [141:17:44<207:32:34,  4.81s/it]
 31%|███       | 69684/225000 [141:17:49<210:34:10,  4.88s/it]
 31%|███       | 69685/225000 [141:17:58<259:42:37,  6.02s/it]
 31%|███       | 69686/225000 [141:18:04<271:04:22,  6.28s/it]
 31%|███       | 69687/225000 [141:18:08<239:22:48,  5.55s/it]
 31%|███       | 69688/225000 [141:18:13<229:30:33,  5.32s/it]
 31%|███       | 69689/225000 [141:18:25<314:47:34,  7.30s/it]
 31%|███       | 69690/225000 [141:18:30<290:06:08,  6.72s/it]
 31%|███       | 69691/225000 [141:18:36<269:50:20,  6.25s/it]
 31%|███       | 69692/225000 [141:18:39<235:07:04,  5.45s/it]
 31%|███       | 69693/225000 [141:18:43<216:33:14,  5.02s/it]
 31%|███       | 69694/225000 [141:18:46<193:32:58,  4.49s/it]
 31%|███       | 69695/225000 [141:18:53<217:19:51,  5.04s/it]
 31%|███       | 69696/225000 [141:19:02<270:28:56,  6.27s/it]
 31%|███       | 69697/225000 [141:19:06<240:08:43,  5.57s/it]
 31%|███       | 69698/225000 [141:19:10<219:15:48,  5.08s/it]
 31%|███       | 69699/225000 [141:19:14<202:58:45,  4.71s/it]
 31%|███       | 69700/225000 [141:19:17<193:08:19,  4.48s/it]
                                                              
{'loss': '0.7477', 'grad_norm': '1.199', 'learning_rate': '4.109e-05', 'epoch': '0.3098', 'num_input_tokens_seen': 234529408, 'train_runtime': '5.088e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69700/225000 [141:19:17<193:08:19,  4.48s/it]
 31%|███       | 69701/225000 [141:19:21<184:36:07,  4.28s/it]
 31%|███       | 69702/225000 [141:19:26<187:30:32,  4.35s/it]
 31%|███       | 69703/225000 [141:19:30<183:18:49,  4.25s/it]
 31%|███       | 69704/225000 [141:19:41<275:25:04,  6.38s/it]
 31%|███       | 69705/225000 [141:19:46<260:24:19,  6.04s/it]
 31%|███       | 69706/225000 [141:19:51<240:38:46,  5.58s/it]
 31%|███       | 69707/225000 [141:19:56<232:44:55,  5.40s/it]
 31%|███       | 69708/225000 [141:20:00<213:24:02,  4.95s/it]
 31%|███       | 69709/225000 [141:20:05<215:53:35,  5.00s/it]
 31%|███       | 69710/225000 [141:20:10<215:58:21,  5.01s/it]
 31%|███       | 69711/225000 [141:20:14<209:33:11,  4.86s/it]
 31%|███       | 69712/225000 [141:20:27<306:46:37,  7.11s/it]
 31%|███       | 69713/225000 [141:20:34<304:57:26,  7.07s/it]
 31%|███       | 69714/225000 [141:20:37<258:23:03,  5.99s/it]
 31%|███       | 69715/225000 [141:20:41<231:07:03,  5.36s/it]
 31%|███       | 69716/225000 [141:20:48<250:44:27,  5.81s/it]
 31%|███       | 69717/225000 [141:20:59<319:47:44,  7.41s/it]
 31%|███       | 69718/225000 [141:21:02<255:53:17,  5.93s/it]
 31%|███       | 69719/225000 [141:21:08<255:43:57,  5.93s/it]
 31%|███       | 69720/225000 [141:21:12<239:40:24,  5.56s/it]
 31%|███       | 69721/225000 [141:21:16<221:42:22,  5.14s/it]
 31%|███       | 69722/225000 [141:21:21<210:30:54,  4.88s/it]
 31%|███       | 69723/225000 [141:21:29<249:16:58,  5.78s/it]
 31%|███       | 69724/225000 [141:21:39<311:01:49,  7.21s/it]
 31%|███       | 69725/225000 [141:21:43<262:21:51,  6.08s/it]
 31%|███       | 69726/225000 [141:21:51<292:13:45,  6.78s/it]
 31%|███       | 69727/225000 [141:21:54<247:25:30,  5.74s/it]
 31%|███       | 69728/225000 [141:22:03<287:08:02,  6.66s/it]
 31%|███       | 69729/225000 [141:22:07<251:06:05,  5.82s/it]
 31%|███       | 69730/225000 [141:22:10<219:28:59,  5.09s/it]
 31%|███       | 69731/225000 [141:22:15<210:44:11,  4.89s/it]
 31%|███       | 69732/225000 [141:22:18<186:57:26,  4.33s/it]
 31%|███       | 69733/225000 [141:22:30<282:37:13,  6.55s/it]
 31%|███       | 69734/225000 [141:22:34<250:12:04,  5.80s/it]
 31%|███       | 69735/225000 [141:22:38<234:20:00,  5.43s/it]
 31%|███       | 69736/225000 [141:22:42<208:27:56,  4.83s/it]
 31%|███       | 69737/225000 [141:22:47<220:39:51,  5.12s/it]
 31%|███       | 69738/225000 [141:22:56<261:31:10,  6.06s/it]
 31%|███       | 69739/225000 [141:22:59<220:28:59,  5.11s/it]
 31%|███       | 69740/225000 [141:23:02<193:24:39,  4.48s/it]
 31%|███       | 69741/225000 [141:23:10<250:13:11,  5.80s/it]
 31%|███       | 69742/225000 [141:23:15<233:55:09,  5.42s/it]
 31%|███       | 69743/225000 [141:23:19<211:21:02,  4.90s/it]
 31%|███       | 69744/225000 [141:23:27<261:34:24,  6.07s/it]
 31%|███       | 69745/225000 [141:23:33<249:17:53,  5.78s/it]
 31%|███       | 69746/225000 [141:23:37<237:15:08,  5.50s/it]
 31%|███       | 69747/225000 [141:23:42<224:52:09,  5.21s/it]
 31%|███       | 69748/225000 [141:23:47<220:22:38,  5.11s/it]
 31%|███       | 69749/225000 [141:23:57<284:45:19,  6.60s/it]
 31%|███       | 69750/225000 [141:24:00<240:27:12,  5.58s/it]
                                                              
{'loss': '0.7786', 'grad_norm': '1.347', 'learning_rate': '4.107e-05', 'epoch': '0.31', 'num_input_tokens_seen': 234650992, 'train_runtime': '5.09e+05', 'train_tokens_per_second': '461'}

 31%|███       | 69750/225000 [141:24:00<240:27:12,  5.58s/it]
 31%|███       | 69751/225000 [141:24:03<212:11:23,  4.92s/it]
 31%|███       | 69752/225000 [141:24:13<267:15:53,  6.20s/it]
 31%|███       | 69753/225000 [141:24:17<242:03:27,  5.61s/it]
 31%|███       | 69754/225000 [141:24:20<208:58:20,  4.85s/it]
 31%|███       | 69755/225000 [141:24:28<244:43:41,  5.68s/it]
 31%|███       | 69756/225000 [141:24:33<238:51:38,  5.54s/it]
 31%|███       | 69757/225000 [141:24:39<245:24:25,  5.69s/it]
 31%|███       | 69758/225000 [141:24:43<222:40:55,  5.16s/it]
 31%|███       | 69759/225000 [141:24:47<211:13:10,  4.90s/it]
 31%|███       | 69760/225000 [141:24:56<262:07:03,  6.08s/it]
 31%|███       | 69761/225000 [141:24:59<229:15:03,  5.32s/it]
 31%|███       | 69762/225000 [141:25:03<211:23:33,  4.90s/it]
 31%|███       | 69763/225000 [141:25:08<214:24:01,  4.97s/it]
 31%|███       | 69764/225000 [141:25:20<302:57:51,  7.03s/it]
 31%|███       | 69765/225000 [141:25:24<256:30:33,  5.95s/it]
 31%|███       | 69766/225000 [141:25:32<282:39:50,  6.56s/it]
 31%|███       | 69767/225000 [141:25:34<230:01:53,  5.33s/it]
 31%|███       | 69768/225000 [141:25:42<268:14:58,  6.22s/it]
 31%|███       | 69769/225000 [141:25:47<249:48:31,  5.79s/it]
 31%|███       | 69770/225000 [141:25:51<217:29:57,  5.04s/it]
 31%|███       | 69771/225000 [141:25:59<262:17:50,  6.08s/it]
 31%|███       | 69772/225000 [141:26:03<239:17:58,  5.55s/it]
 31%|███       | 69773/225000 [141:26:11<262:19:09,  6.08s/it]
 31%|███       | 69774/225000 [141:26:15<240:39:41,  5.58s/it]
 31%|███       | 69775/225000 [141:26:23<267:39:02,  6.21s/it]
 31%|███       | 69776/225000 [141:26:30<281:30:06,  6.53s/it]
 31%|███       | 69777/225000 [141:26:34<251:41:25,  5.84s/it]
 31%|███       | 69778/225000 [141:26:39<231:22:33,  5.37s/it]
 31%|███       | 69779/225000 [141:26:42<213:00:34,  4.94s/it]
 31%|███       | 69780/225000 [141:26:47<203:17:58,  4.72s/it]
 31%|███       | 69781/225000 [141:26:52<211:38:42,  4.91s/it]
 31%|███       | 69782/225000 [141:27:03<294:42:54,  6.84s/it]
 31%|███       | 69783/225000 [141:27:08<270:59:20,  6.29s/it]
 31%|███       | 69784/225000 [141:27:12<235:59:44,  5.47s/it]
 31%|███       | 69785/225000 [141:27:15<209:47:35,  4.87s/it]
 31%|███       | 69786/225000 [141:27:24<254:15:55,  5.90s/it]
 31%|███       | 69787/225000 [141:27:29<243:18:08,  5.64s/it]
 31%|███       | 69788/225000 [141:27:35<257:15:43,  5.97s/it]
 31%|███       | 69789/225000 [141:27:40<241:03:46,  5.59s/it]
 31%|███       | 69790/225000 [141:27:44<218:21:14,  5.06s/it]
 31%|███       | 69791/225000 [141:27:48<204:28:54,  4.74s/it]
 31%|███       | 69792/225000 [141:27:52<190:25:01,  4.42s/it]
 31%|███       | 69793/225000 [141:27:57<200:47:37,  4.66s/it]
 31%|███       | 69794/225000 [141:28:01<196:07:43,  4.55s/it]
 31%|███       | 69795/225000 [141:28:06<204:19:19,  4.74s/it]
 31%|███       | 69796/225000 [141:28:16<264:26:52,  6.13s/it]
 31%|███       | 69797/225000 [141:28:22<265:09:59,  6.15s/it]
 31%|███       | 69798/225000 [141:28:26<234:08:08,  5.43s/it]
 31%|███       | 69799/225000 [141:28:35<282:17:05,  6.55s/it]
 31%|███       | 69800/225000 [141:28:42<287:52:23,  6.68s/it]
                                                              
{'loss': '0.7411', 'grad_norm': '0.9563', 'learning_rate': '4.106e-05', 'epoch': '0.3102', 'num_input_tokens_seen': 234771288, 'train_runtime': '5.093e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 69800/225000 [141:28:42<287:52:23,  6.68s/it]
 31%|███       | 69801/225000 [141:28:47<262:34:27,  6.09s/it]
 31%|███       | 69802/225000 [141:28:50<234:08:17,  5.43s/it]
 31%|███       | 69803/225000 [141:28:54<207:51:19,  4.82s/it]
 31%|███       | 69804/225000 [141:28:59<212:31:32,  4.93s/it]
 31%|███       | 69805/225000 [141:29:07<252:36:22,  5.86s/it]
 31%|███       | 69806/225000 [141:29:11<224:27:59,  5.21s/it]
 31%|███       | 69807/225000 [141:29:14<198:43:25,  4.61s/it]
 31%|███       | 69808/225000 [141:29:18<189:25:19,  4.39s/it]
 31%|███       | 69809/225000 [141:29:28<261:26:20,  6.06s/it]
 31%|███       | 69810/225000 [141:29:35<270:33:42,  6.28s/it]
 31%|███       | 69811/225000 [141:29:39<244:09:53,  5.66s/it]
 31%|███       | 69812/225000 [141:29:43<225:25:36,  5.23s/it]
 31%|███       | 69813/225000 [141:29:47<214:31:10,  4.98s/it]
 31%|███       | 69814/225000 [141:29:52<204:51:11,  4.75s/it]
 31%|███       | 69815/225000 [141:30:02<275:58:12,  6.40s/it]
 31%|███       | 69816/225000 [141:30:08<274:43:48,  6.37s/it]
 31%|███       | 69817/225000 [141:30:12<236:56:02,  5.50s/it]
 31%|███       | 69818/225000 [141:30:16<216:03:13,  5.01s/it]
 31%|███       | 69819/225000 [141:30:19<199:34:06,  4.63s/it]
 31%|███       | 69820/225000 [141:30:30<279:36:43,  6.49s/it]
 31%|███       | 69821/225000 [141:30:33<238:12:05,  5.53s/it]
 31%|███       | 69822/225000 [141:30:36<204:07:52,  4.74s/it]
 31%|███       | 69823/225000 [141:30:44<245:04:32,  5.69s/it]
 31%|███       | 69824/225000 [141:30:49<239:19:30,  5.55s/it]
 31%|███       | 69825/225000 [141:30:57<260:16:35,  6.04s/it]
 31%|███       | 69826/225000 [141:31:03<259:59:46,  6.03s/it]
 31%|███       | 69827/225000 [141:31:09<263:58:24,  6.12s/it]
 31%|███       | 69828/225000 [141:31:14<244:24:20,  5.67s/it]
 31%|███       | 69829/225000 [141:31:18<225:05:24,  5.22s/it]
 31%|███       | 69830/225000 [141:31:23<223:01:09,  5.17s/it]
 31%|███       | 69831/225000 [141:31:28<225:54:21,  5.24s/it]
 31%|███       | 69832/225000 [141:31:33<220:40:37,  5.12s/it]
 31%|███       | 69833/225000 [141:31:38<220:11:59,  5.11s/it]
 31%|███       | 69834/225000 [141:31:48<276:00:09,  6.40s/it]
 31%|███       | 69835/225000 [141:31:57<317:41:58,  7.37s/it]
 31%|███       | 69836/225000 [141:32:00<259:57:30,  6.03s/it]
 31%|███       | 69837/225000 [141:32:04<237:25:44,  5.51s/it]
 31%|███       | 69838/225000 [141:32:08<213:57:34,  4.96s/it]
 31%|███       | 69839/225000 [141:32:13<211:31:01,  4.91s/it]
 31%|███       | 69840/225000 [141:32:23<278:31:58,  6.46s/it]
 31%|███       | 69841/225000 [141:32:26<238:45:46,  5.54s/it]
 31%|███       | 69842/225000 [141:32:35<284:46:17,  6.61s/it]
 31%|███       | 69843/225000 [141:32:40<252:38:28,  5.86s/it]
 31%|███       | 69844/225000 [141:32:43<219:14:34,  5.09s/it]
 31%|███       | 69845/225000 [141:32:49<228:48:03,  5.31s/it]
 31%|███       | 69846/225000 [141:32:54<231:52:42,  5.38s/it]
 31%|███       | 69847/225000 [141:32:58<207:36:56,  4.82s/it]
 31%|███       | 69848/225000 [141:33:05<237:08:34,  5.50s/it]
 31%|███       | 69849/225000 [141:33:10<233:36:49,  5.42s/it]
 31%|███       | 69850/225000 [141:33:15<226:59:41,  5.27s/it]
                                                              
{'loss': '0.7822', 'grad_norm': '1.009', 'learning_rate': '4.104e-05', 'epoch': '0.3104', 'num_input_tokens_seen': 234890400, 'train_runtime': '5.096e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 69850/225000 [141:33:15<226:59:41,  5.27s/it]
 31%|███       | 69851/225000 [141:33:19<217:36:57,  5.05s/it]
 31%|███       | 69852/225000 [141:33:24<210:52:53,  4.89s/it]
 31%|███       | 69853/225000 [141:33:29<206:54:24,  4.80s/it]
 31%|███       | 69854/225000 [141:33:34<211:45:49,  4.91s/it]
 31%|███       | 69855/225000 [141:33:45<294:40:15,  6.84s/it]
 31%|███       | 69856/225000 [141:33:50<264:00:02,  6.13s/it]
 31%|███       | 69857/225000 [141:33:54<236:17:08,  5.48s/it]
 31%|███       | 69858/225000 [141:34:02<280:00:02,  6.50s/it]
 31%|███       | 69859/225000 [141:34:08<266:51:34,  6.19s/it]
 31%|███       | 69860/225000 [141:34:12<240:30:51,  5.58s/it]
 31%|███       | 69861/225000 [141:34:21<285:51:04,  6.63s/it]
 31%|███       | 69862/225000 [141:34:24<237:09:54,  5.50s/it]
 31%|███       | 69863/225000 [141:34:30<245:09:26,  5.69s/it]
 31%|███       | 69864/225000 [141:34:34<220:01:20,  5.11s/it]
 31%|███       | 69865/225000 [141:34:38<213:41:25,  4.96s/it]
 31%|███       | 69866/225000 [141:34:47<262:17:52,  6.09s/it]
 31%|███       | 69867/225000 [141:34:56<294:15:00,  6.83s/it]
 31%|███       | 69868/225000 [141:35:00<255:21:19,  5.93s/it]
 31%|███       | 69869/225000 [141:35:03<224:32:43,  5.21s/it]
 31%|███       | 69870/225000 [141:35:07<208:51:50,  4.85s/it]
 31%|███       | 69871/225000 [141:35:12<209:16:17,  4.86s/it]
 31%|███       | 69872/225000 [141:35:17<209:53:59,  4.87s/it]
 31%|███       | 69873/225000 [141:35:21<205:19:12,  4.76s/it]
 31%|███       | 69874/225000 [141:35:30<259:27:17,  6.02s/it]
 31%|███       | 69875/225000 [141:35:34<234:14:46,  5.44s/it]
 31%|███       | 69876/225000 [141:35:38<208:51:59,  4.85s/it]
 31%|███       | 69877/225000 [141:35:47<267:49:51,  6.22s/it]
 31%|███       | 69878/225000 [141:35:53<262:48:15,  6.10s/it]
 31%|███       | 69879/225000 [141:35:56<222:19:09,  5.16s/it]
 31%|███       | 69880/225000 [141:36:03<249:38:13,  5.79s/it]
 31%|███       | 69881/225000 [141:36:08<237:33:37,  5.51s/it]
 31%|███       | 69882/225000 [141:36:12<212:23:53,  4.93s/it]
 31%|███       | 69883/225000 [141:36:16<201:07:42,  4.67s/it]
 31%|███       | 69884/225000 [141:36:20<189:40:45,  4.40s/it]
 31%|███       | 69885/225000 [141:36:25<195:57:17,  4.55s/it]
 31%|███       | 69886/225000 [141:36:34<256:28:54,  5.95s/it]
 31%|███       | 69887/225000 [141:36:41<271:31:27,  6.30s/it]
 31%|███       | 69888/225000 [141:36:45<241:13:41,  5.60s/it]
 31%|███       | 69889/225000 [141:36:48<212:50:56,  4.94s/it]
 31%|███       | 69890/225000 [141:36:52<200:59:58,  4.67s/it]
 31%|███       | 69891/225000 [141:37:03<277:12:07,  6.43s/it]
 31%|███       | 69892/225000 [141:37:06<241:18:04,  5.60s/it]
 31%|███       | 69893/225000 [141:37:10<215:22:00,  5.00s/it]
 31%|███       | 69894/225000 [141:37:14<202:06:56,  4.69s/it]
 31%|███       | 69895/225000 [141:37:18<193:49:22,  4.50s/it]
 31%|███       | 69896/225000 [141:37:22<189:14:17,  4.39s/it]
 31%|███       | 69897/225000 [141:37:26<186:59:02,  4.34s/it]
 31%|███       | 69898/225000 [141:37:35<239:23:52,  5.56s/it]
 31%|███       | 69899/225000 [141:37:45<295:53:04,  6.87s/it]
 31%|███       | 69900/225000 [141:37:48<248:56:53,  5.78s/it]
                                                              
{'loss': '0.7552', 'grad_norm': '0.9647', 'learning_rate': '4.103e-05', 'epoch': '0.3107', 'num_input_tokens_seen': 235009288, 'train_runtime': '5.099e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 69900/225000 [141:37:48<248:56:53,  5.78s/it]
 31%|███       | 69901/225000 [141:37:57<294:47:17,  6.84s/it]
 31%|███       | 69902/225000 [141:38:02<269:11:30,  6.25s/it]
 31%|███       | 69903/225000 [141:38:07<249:04:38,  5.78s/it]
 31%|███       | 69904/225000 [141:38:12<235:17:16,  5.46s/it]
 31%|███       | 69905/225000 [141:38:15<213:29:59,  4.96s/it]
 31%|███       | 69906/225000 [141:38:19<194:15:24,  4.51s/it]
 31%|███       | 69907/225000 [141:38:28<258:37:13,  6.00s/it]
 31%|███       | 69908/225000 [141:38:33<240:43:07,  5.59s/it]
 31%|███       | 69909/225000 [141:38:38<234:12:49,  5.44s/it]
 31%|███       | 69910/225000 [141:38:43<225:44:21,  5.24s/it]
 31%|███       | 69911/225000 [141:38:52<278:07:23,  6.46s/it]
 31%|███       | 69912/225000 [141:38:55<231:24:00,  5.37s/it]
 31%|███       | 69913/225000 [141:38:58<197:58:07,  4.60s/it]
 31%|███       | 69914/225000 [141:39:02<195:47:34,  4.54s/it]
 31%|███       | 69915/225000 [141:39:06<191:52:21,  4.45s/it]
 31%|███       | 69916/225000 [141:39:11<196:05:38,  4.55s/it]
 31%|███       | 69917/225000 [141:39:21<266:17:50,  6.18s/it]
 31%|███       | 69918/225000 [141:39:25<231:59:27,  5.39s/it]
 31%|███       | 69919/225000 [141:39:32<250:46:30,  5.82s/it]
 31%|███       | 69920/225000 [141:39:36<228:48:41,  5.31s/it]
 31%|███       | 69921/225000 [141:39:39<208:00:24,  4.83s/it]
 31%|███       | 69922/225000 [141:39:44<203:30:28,  4.72s/it]
 31%|███       | 69923/225000 [141:39:48<194:27:43,  4.51s/it]
 31%|███       | 69924/225000 [141:39:53<198:00:18,  4.60s/it]
 31%|███       | 69925/225000 [141:40:03<272:43:53,  6.33s/it]
 31%|███       | 69926/225000 [141:40:13<317:54:20,  7.38s/it]
 31%|███       | 69927/225000 [141:40:17<270:38:31,  6.28s/it]
 31%|███       | 69928/225000 [141:40:21<246:06:29,  5.71s/it]
 31%|███       | 69929/225000 [141:40:24<217:20:17,  5.05s/it]
 31%|███       | 69930/225000 [141:40:29<213:08:17,  4.95s/it]
 31%|███       | 69931/225000 [141:40:34<209:26:24,  4.86s/it]
 31%|███       | 69932/225000 [141:40:40<224:04:50,  5.20s/it]
 31%|███       | 69933/225000 [141:40:51<305:50:32,  7.10s/it]
 31%|███       | 69934/225000 [141:40:57<281:25:37,  6.53s/it]
 31%|███       | 69935/225000 [141:41:00<237:17:07,  5.51s/it]
 31%|███       | 69936/225000 [141:41:07<260:16:24,  6.04s/it]
 31%|███       | 69937/225000 [141:41:12<243:17:16,  5.65s/it]
 31%|███       | 69938/225000 [141:41:16<222:30:40,  5.17s/it]
 31%|███       | 69939/225000 [141:41:20<211:15:28,  4.90s/it]
 31%|███       | 69940/225000 [141:41:27<243:22:13,  5.65s/it]
 31%|███       | 69941/225000 [141:41:33<239:43:02,  5.57s/it]
 31%|███       | 69942/225000 [141:41:38<235:00:19,  5.46s/it]
 31%|███       | 69943/225000 [141:41:42<217:09:12,  5.04s/it]
 31%|███       | 69944/225000 [141:41:48<225:37:40,  5.24s/it]
 31%|███       | 69945/225000 [141:41:55<252:20:54,  5.86s/it]
 31%|███       | 69946/225000 [141:41:59<222:46:26,  5.17s/it]
 31%|███       | 69947/225000 [141:42:02<204:12:40,  4.74s/it]
 31%|███       | 69948/225000 [141:42:07<198:27:20,  4.61s/it]
 31%|███       | 69949/225000 [141:42:12<207:10:05,  4.81s/it]
 31%|███       | 69950/225000 [141:42:22<280:05:38,  6.50s/it]
                                                              
{'loss': '0.7525', 'grad_norm': '0.7574', 'learning_rate': '4.102e-05', 'epoch': '0.3109', 'num_input_tokens_seen': 235127648, 'train_runtime': '5.101e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 69950/225000 [141:42:22<280:05:38,  6.50s/it]
 31%|███       | 69951/225000 [141:42:26<240:06:55,  5.58s/it]
 31%|███       | 69952/225000 [141:42:31<230:11:57,  5.34s/it]
 31%|███       | 69953/225000 [141:42:36<236:09:39,  5.48s/it]
 31%|███       | 69954/225000 [141:42:40<208:06:40,  4.83s/it]
 31%|███       | 69955/225000 [141:42:48<249:55:21,  5.80s/it]
 31%|███       | 69956/225000 [141:42:53<242:29:49,  5.63s/it]
 31%|███       | 69957/225000 [141:42:57<216:42:07,  5.03s/it]
 31%|███       | 69958/225000 [141:43:05<265:01:00,  6.15s/it]
 31%|███       | 69959/225000 [141:43:09<236:32:29,  5.49s/it]
 31%|███       | 69960/225000 [141:43:13<217:33:26,  5.05s/it]
 31%|███       | 69961/225000 [141:43:21<252:06:05,  5.85s/it]
 31%|███       | 69962/225000 [141:43:28<270:55:35,  6.29s/it]
 31%|███       | 69963/225000 [141:43:32<240:23:34,  5.58s/it]
 31%|███       | 69964/225000 [141:43:36<220:29:46,  5.12s/it]
 31%|███       | 69965/225000 [141:43:41<218:06:23,  5.06s/it]
 31%|███       | 69966/225000 [141:43:52<287:42:27,  6.68s/it]
 31%|███       | 69967/225000 [141:43:56<253:15:13,  5.88s/it]
 31%|███       | 69968/225000 [141:44:01<238:51:37,  5.55s/it]
 31%|███       | 69969/225000 [141:44:05<219:45:47,  5.10s/it]
 31%|███       | 69970/225000 [141:44:09<212:15:10,  4.93s/it]
 31%|███       | 69971/225000 [141:44:14<206:17:29,  4.79s/it]
 31%|███       | 69972/225000 [141:44:23<268:04:01,  6.22s/it]
 31%|███       | 69973/225000 [141:44:32<298:59:02,  6.94s/it]
 31%|███       | 69974/225000 [141:44:35<249:10:52,  5.79s/it]
 31%|███       | 69975/225000 [141:44:39<232:21:31,  5.40s/it]
 31%|███       | 69976/225000 [141:44:45<228:38:42,  5.31s/it]
 31%|███       | 69977/225000 [141:44:51<242:08:00,  5.62s/it]
 31%|███       | 69978/225000 [141:44:56<230:28:44,  5.35s/it]
 31%|███       | 69979/225000 [141:44:59<210:11:04,  4.88s/it]
 31%|███       | 69980/225000 [141:45:03<191:46:55,  4.45s/it]
 31%|███       | 69981/225000 [141:45:07<188:19:20,  4.37s/it]
 31%|███       | 69982/225000 [141:45:11<183:36:23,  4.26s/it]
 31%|███       | 69983/225000 [141:45:20<246:36:10,  5.73s/it]
 31%|███       | 69984/225000 [141:45:28<272:33:40,  6.33s/it]
 31%|███       | 69985/225000 [141:45:32<249:56:59,  5.80s/it]
 31%|███       | 69986/225000 [141:45:36<225:34:52,  5.24s/it]
 31%|███       | 69987/225000 [141:45:41<215:18:00,  5.00s/it]
 31%|███       | 69988/225000 [141:45:46<220:41:22,  5.13s/it]
 31%|███       | 69989/225000 [141:45:51<217:19:05,  5.05s/it]
 31%|███       | 69990/225000 [141:45:58<241:08:03,  5.60s/it]
 31%|███       | 69991/225000 [141:46:05<256:38:18,  5.96s/it]
 31%|███       | 69992/225000 [141:46:08<223:29:49,  5.19s/it]
 31%|███       | 69993/225000 [141:46:11<191:18:19,  4.44s/it]
 31%|███       | 69994/225000 [141:46:21<259:38:01,  6.03s/it]
 31%|███       | 69995/225000 [141:46:27<266:49:39,  6.20s/it]
 31%|███       | 69996/225000 [141:46:31<233:04:24,  5.41s/it]
 31%|███       | 69997/225000 [141:46:36<231:02:00,  5.37s/it]
 31%|███       | 69998/225000 [141:46:44<265:12:44,  6.16s/it]
 31%|███       | 69999/225000 [141:46:47<220:09:06,  5.11s/it]
 31%|███       | 70000/225000 [141:46:51<202:31:27,  4.70s/it]
                                                              
{'loss': '0.7546', 'grad_norm': '1.074', 'learning_rate': '4.1e-05', 'epoch': '0.3111', 'num_input_tokens_seen': 235245744, 'train_runtime': '5.104e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 70000/225000 [141:46:51<202:31:27,  4.70s/it]
 31%|███       | 70001/225000 [141:46:58<238:52:15,  5.55s/it]
 31%|███       | 70002/225000 [141:47:03<228:57:11,  5.32s/it]
 31%|███       | 70003/225000 [141:47:07<210:36:02,  4.89s/it]
 31%|███       | 70004/225000 [141:47:12<210:30:49,  4.89s/it]
 31%|███       | 70005/225000 [141:47:16<206:06:18,  4.79s/it]
 31%|███       | 70006/225000 [141:47:20<196:22:18,  4.56s/it]
 31%|███       | 70007/225000 [141:47:25<199:50:59,  4.64s/it]
 31%|███       | 70008/225000 [141:47:37<292:00:36,  6.78s/it]
 31%|███       | 70009/225000 [141:47:42<265:51:38,  6.18s/it]
 31%|███       | 70010/225000 [141:47:45<228:49:33,  5.32s/it]
 31%|███       | 70011/225000 [141:47:52<251:25:08,  5.84s/it]
 31%|███       | 70012/225000 [141:47:59<262:36:25,  6.10s/it]
 31%|███       | 70013/225000 [141:48:02<229:54:13,  5.34s/it]
 31%|███       | 70014/225000 [141:48:09<250:41:11,  5.82s/it]
 31%|███       | 70015/225000 [141:48:14<232:16:53,  5.40s/it]
 31%|███       | 70016/225000 [141:48:18<220:44:13,  5.13s/it]
 31%|███       | 70017/225000 [141:48:23<212:30:05,  4.94s/it]
 31%|███       | 70018/225000 [141:48:26<197:27:41,  4.59s/it]
 31%|███       | 70019/225000 [141:48:37<269:59:25,  6.27s/it]
 31%|███       | 70020/225000 [141:48:45<293:00:54,  6.81s/it]
 31%|███       | 70021/225000 [141:48:48<254:10:17,  5.90s/it]
 31%|███       | 70022/225000 [141:48:52<224:28:43,  5.21s/it]
 31%|███       | 70023/225000 [141:48:56<209:44:09,  4.87s/it]
 31%|███       | 70024/225000 [141:49:02<226:00:47,  5.25s/it]
 31%|███       | 70025/225000 [141:49:09<246:23:42,  5.72s/it]
 31%|███       | 70026/225000 [141:49:13<219:47:32,  5.11s/it]
 31%|███       | 70027/225000 [141:49:16<195:04:36,  4.53s/it]
 31%|███       | 70028/225000 [141:49:20<185:19:25,  4.31s/it]
 31%|███       | 70029/225000 [141:49:23<178:28:11,  4.15s/it]
 31%|███       | 70030/225000 [141:49:32<239:05:16,  5.55s/it]
 31%|███       | 70031/225000 [141:49:38<240:23:47,  5.58s/it]
 31%|███       | 70032/225000 [141:49:42<221:10:46,  5.14s/it]
 31%|███       | 70033/225000 [141:49:46<200:59:50,  4.67s/it]
 31%|███       | 70034/225000 [141:49:54<253:25:31,  5.89s/it]
 31%|███       | 70035/225000 [141:49:58<224:44:58,  5.22s/it]
 31%|███       | 70036/225000 [141:50:03<216:06:19,  5.02s/it]
 31%|███       | 70037/225000 [141:50:06<193:37:49,  4.50s/it]
 31%|███       | 70038/225000 [141:50:16<273:22:41,  6.35s/it]
 31%|███       | 70039/225000 [141:50:20<232:39:26,  5.41s/it]
 31%|███       | 70040/225000 [141:50:23<206:17:24,  4.79s/it]
 31%|███       | 70041/225000 [141:50:28<202:54:39,  4.71s/it]
 31%|███       | 70042/225000 [141:50:32<200:15:12,  4.65s/it]
 31%|███       | 70043/225000 [141:50:37<201:54:58,  4.69s/it]
 31%|███       | 70044/225000 [141:50:42<206:10:53,  4.79s/it]
 31%|███       | 70045/225000 [141:50:52<270:43:25,  6.29s/it]
 31%|███       | 70046/225000 [141:50:56<246:09:51,  5.72s/it]
 31%|███       | 70047/225000 [141:51:00<220:31:59,  5.12s/it]
 31%|███       | 70048/225000 [141:51:09<279:28:01,  6.49s/it]
 31%|███       | 70049/225000 [141:51:17<290:01:52,  6.74s/it]
 31%|███       | 70050/225000 [141:51:21<255:33:20,  5.94s/it]
                                                              
{'loss': '0.7396', 'grad_norm': '1.349', 'learning_rate': '4.099e-05', 'epoch': '0.3113', 'num_input_tokens_seen': 235363528, 'train_runtime': '5.107e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 70050/225000 [141:51:21<255:33:20,  5.94s/it]
 31%|███       | 70051/225000 [141:51:23<211:37:03,  4.92s/it]
 31%|███       | 70052/225000 [141:51:27<193:16:33,  4.49s/it]
 31%|███       | 70053/225000 [141:51:36<247:01:00,  5.74s/it]
 31%|███       | 70054/225000 [141:51:40<233:22:07,  5.42s/it]
 31%|███       | 70055/225000 [141:51:44<213:03:00,  4.95s/it]
 31%|███       | 70056/225000 [141:51:47<192:20:54,  4.47s/it]
 31%|███       | 70057/225000 [141:51:56<248:49:35,  5.78s/it]
 31%|███       | 70058/225000 [141:52:00<227:48:40,  5.29s/it]
 31%|███       | 70059/225000 [141:52:04<203:49:08,  4.74s/it]
 31%|███       | 70060/225000 [141:52:13<256:18:01,  5.96s/it]
 31%|███       | 70061/225000 [141:52:16<225:19:51,  5.24s/it]
 31%|███       | 70062/225000 [141:52:21<213:49:27,  4.97s/it]
 31%|███       | 70063/225000 [141:52:26<214:07:05,  4.98s/it]
 31%|███       | 70064/225000 [141:52:29<189:11:23,  4.40s/it]
 31%|███       | 70065/225000 [141:52:35<216:26:08,  5.03s/it]
 31%|███       | 70066/225000 [141:52:40<214:21:06,  4.98s/it]
 31%|███       | 70067/225000 [141:52:44<199:34:42,  4.64s/it]
 31%|███       | 70068/225000 [141:52:47<186:15:20,  4.33s/it]
 31%|███       | 70069/225000 [141:52:52<188:10:31,  4.37s/it]
 31%|███       | 70070/225000 [141:52:58<211:52:11,  4.92s/it]
 31%|███       | 70071/225000 [141:53:07<259:15:25,  6.02s/it]
 31%|███       | 70072/225000 [141:53:10<223:08:11,  5.18s/it]
 31%|███       | 70073/225000 [141:53:14<212:49:28,  4.95s/it]
 31%|███       | 70074/225000 [141:53:18<199:13:52,  4.63s/it]
 31%|███       | 70075/225000 [141:53:23<195:43:18,  4.55s/it]
 31%|███       | 70076/225000 [141:53:27<195:17:45,  4.54s/it]
 31%|███       | 70077/225000 [141:53:38<276:06:53,  6.42s/it]
 31%|███       | 70078/225000 [141:53:44<271:32:13,  6.31s/it]
 31%|███       | 70079/225000 [141:53:47<234:34:35,  5.45s/it]
 31%|███       | 70080/225000 [141:53:54<247:43:45,  5.76s/it]
 31%|███       | 70081/225000 [141:54:02<282:06:25,  6.56s/it]
 31%|███       | 70082/225000 [141:54:05<236:10:18,  5.49s/it]
 31%|███       | 70083/225000 [141:54:09<215:13:02,  5.00s/it]
 31%|███       | 70084/225000 [141:54:18<264:59:08,  6.16s/it]
 31%|███       | 70085/225000 [141:54:26<284:17:32,  6.61s/it]
 31%|███       | 70086/225000 [141:54:29<242:16:09,  5.63s/it]
 31%|███       | 70087/225000 [141:54:37<275:33:12,  6.40s/it]
 31%|███       | 70088/225000 [141:54:43<270:32:03,  6.29s/it]
 31%|███       | 70089/225000 [141:54:49<261:57:44,  6.09s/it]
 31%|███       | 70090/225000 [141:54:53<232:28:15,  5.40s/it]
 31%|███       | 70091/225000 [141:54:59<239:25:10,  5.56s/it]
 31%|███       | 70092/225000 [141:55:03<219:39:40,  5.10s/it]
 31%|███       | 70093/225000 [141:55:11<260:06:31,  6.04s/it]
 31%|███       | 70094/225000 [141:55:15<233:31:56,  5.43s/it]
 31%|███       | 70095/225000 [141:55:18<208:04:54,  4.84s/it]
 31%|███       | 70096/225000 [141:55:29<278:50:19,  6.48s/it]
 31%|███       | 70097/225000 [141:55:33<246:56:59,  5.74s/it]
 31%|███       | 70098/225000 [141:55:37<230:24:04,  5.35s/it]
 31%|███       | 70099/225000 [141:55:45<266:13:12,  6.19s/it]
 31%|███       | 70100/225000 [141:55:52<270:54:50,  6.30s/it]
                                                              
{'loss': '0.7783', 'grad_norm': '1.008', 'learning_rate': '4.097e-05', 'epoch': '0.3116', 'num_input_tokens_seen': 235480520, 'train_runtime': '5.11e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 70100/225000 [141:55:52<270:54:50,  6.30s/it]
 31%|███       | 70101/225000 [141:55:58<275:54:07,  6.41s/it]
 31%|███       | 70102/225000 [141:56:04<270:02:45,  6.28s/it]
 31%|███       | 70103/225000 [141:56:10<257:39:54,  5.99s/it]
 31%|███       | 70104/225000 [141:56:14<229:23:00,  5.33s/it]
 31%|███       | 70105/225000 [141:56:17<210:13:05,  4.89s/it]
 31%|███       | 70106/225000 [141:56:23<222:20:34,  5.17s/it]
 31%|███       | 70107/225000 [141:56:33<286:54:06,  6.67s/it]
 31%|███       | 70108/225000 [141:56:37<248:31:31,  5.78s/it]
 31%|███       | 70109/225000 [141:56:40<216:54:32,  5.04s/it]
 31%|███       | 70110/225000 [141:56:45<208:18:51,  4.84s/it]
 31%|███       | 70111/225000 [141:56:49<203:53:39,  4.74s/it]
 31%|███       | 70112/225000 [141:56:59<269:18:24,  6.26s/it]
 31%|███       | 70113/225000 [141:57:02<226:35:46,  5.27s/it]
 31%|███       | 70114/225000 [141:57:07<229:09:53,  5.33s/it]
 31%|███       | 70115/225000 [141:57:13<226:32:44,  5.27s/it]
 31%|███       | 70116/225000 [141:57:17<213:24:28,  4.96s/it]
 31%|███       | 70117/225000 [141:57:20<195:29:18,  4.54s/it]
 31%|███       | 70118/225000 [141:57:27<219:38:06,  5.11s/it]
 31%|███       | 70119/225000 [141:57:31<209:36:26,  4.87s/it]
 31%|███       | 70120/225000 [141:57:35<200:22:05,  4.66s/it]
 31%|███       | 70121/225000 [141:57:39<189:54:59,  4.41s/it]
 31%|███       | 70122/225000 [141:57:45<211:43:03,  4.92s/it]
 31%|███       | 70123/225000 [141:57:53<253:01:09,  5.88s/it]
 31%|███       | 70124/225000 [141:58:00<259:01:51,  6.02s/it]
 31%|███       | 70125/225000 [141:58:04<232:11:26,  5.40s/it]
 31%|███       | 70126/225000 [141:58:08<216:59:33,  5.04s/it]
 31%|███       | 70127/225000 [141:58:12<209:59:36,  4.88s/it]
 31%|███       | 70128/225000 [141:58:16<197:17:28,  4.59s/it]
 31%|███       | 70129/225000 [141:58:21<197:12:25,  4.58s/it]
 31%|███       | 70130/225000 [141:58:25<191:03:22,  4.44s/it]
 31%|███       | 70131/225000 [141:58:30<192:33:45,  4.48s/it]
 31%|███       | 70132/225000 [141:58:38<246:05:58,  5.72s/it]
 31%|███       | 70133/225000 [141:58:48<294:47:50,  6.85s/it]
 31%|███       | 70134/225000 [141:58:51<248:33:07,  5.78s/it]
 31%|███       | 70135/225000 [141:58:54<212:39:10,  4.94s/it]
 31%|███       | 70136/225000 [141:58:58<203:01:00,  4.72s/it]
 31%|███       | 70137/225000 [141:59:03<199:47:08,  4.64s/it]
 31%|███       | 70138/225000 [141:59:13<271:07:56,  6.30s/it]
 31%|███       | 70139/225000 [141:59:18<254:52:43,  5.93s/it]
 31%|███       | 70140/225000 [141:59:24<257:36:30,  5.99s/it]
 31%|███       | 70141/225000 [141:59:28<230:41:43,  5.36s/it]
 31%|███       | 70142/225000 [141:59:36<271:01:02,  6.30s/it]
 31%|███       | 70143/225000 [141:59:39<228:37:18,  5.31s/it]
 31%|███       | 70144/225000 [141:59:45<230:11:50,  5.35s/it]
 31%|███       | 70145/225000 [141:59:52<258:58:20,  6.02s/it]
 31%|███       | 70146/225000 [141:59:55<221:18:18,  5.14s/it]
 31%|███       | 70147/225000 [142:00:03<247:02:57,  5.74s/it]
 31%|███       | 70148/225000 [142:00:06<215:40:12,  5.01s/it]
 31%|███       | 70149/225000 [142:00:10<208:09:25,  4.84s/it]
 31%|███       | 70150/225000 [142:00:14<187:03:40,  4.35s/it]
                                                              
{'loss': '0.7578', 'grad_norm': '1.566', 'learning_rate': '4.096e-05', 'epoch': '0.3118', 'num_input_tokens_seen': 235594368, 'train_runtime': '5.112e+05', 'train_tokens_per_second': '460.9'}

 31%|███       | 70150/225000 [142:00:14<187:03:40,  4.35s/it]
 31%|███       | 70151/225000 [142:00:18<190:32:35,  4.43s/it]
 31%|███       | 70152/225000 [142:00:29<269:29:05,  6.27s/it]
 31%|███       | 70153/225000 [142:00:33<244:54:11,  5.69s/it]
 31%|███       | 70154/225000 [142:00:37<221:05:27,  5.14s/it]
 31%|███       | 70155/225000 [142:00:45<260:49:56,  6.06s/it]
 31%|███       | 70156/225000 [142:00:51<263:48:15,  6.13s/it]
 31%|███       | 70157/225000 [142:00:55<228:39:02,  5.32s/it]
 31%|███       | 70158/225000 [142:01:00<221:12:03,  5.14s/it]
 31%|███       | 70159/225000 [142:01:04<207:21:58,  4.82s/it]
 31%|███       | 70160/225000 [142:01:10<229:55:46,  5.35s/it]
 31%|███       | 70161/225000 [142:01:22<316:01:53,  7.35s/it]
 31%|███       | 70162/225000 [142:01:26<273:03:05,  6.35s/it]
 31%|███       | 70163/225000 [142:01:29<226:03:10,  5.26s/it]
 31%|███       | 70164/225000 [142:01:36<243:35:55,  5.66s/it]
 31%|███       | 70165/225000 [142:01:41<246:33:59,  5.73s/it]
 31%|███       | 70166/225000 [142:01:47<241:32:40,  5.62s/it]
 31%|███       | 70167/225000 [142:01:53<252:21:29,  5.87s/it]
 31%|███       | 70168/225000 [142:01:57<220:08:37,  5.12s/it]
 31%|███       | 70169/225000 [142:02:01<214:34:54,  4.99s/it]
 31%|███       | 70170/225000 [142:02:06<211:17:14,  4.91s/it]
 31%|███       | 70171/225000 [142:02:11<213:28:26,  4.96s/it]
 31%|███       | 70172/225000 [142:02:15<193:23:11,  4.50s/it]
 31%|███       | 70173/225000 [142:02:19<189:44:14,  4.41s/it]
 31%|███       | 70174/225000 [142:02:23<186:59:24,  4.35s/it]
 31%|███       | 70175/225000 [142:02:31<228:06:37,  5.30s/it]
 31%|███       | 70176/225000 [142:02:40<286:53:58,  6.67s/it]
 31%|███       | 70177/225000 [142:02:44<245:10:43,  5.70s/it]
 31%|███       | 70178/225000 [142:02:47<208:46:26,  4.85s/it]
 31%|███       | 70179/225000 [142:02:52<212:14:43,  4.94s/it]
 31%|███       | 70180/225000 [142:02:56<198:53:43,  4.62s/it]
 31%|███       | 70181/225000 [142:03:01<202:38:32,  4.71s/it]
 31%|███       | 70182/225000 [142:03:06<208:03:29,  4.84s/it]
 31%|███       | 70183/225000 [142:03:10<201:14:18,  4.68s/it]
 31%|███       | 70184/225000 [142:03:15<208:33:02,  4.85s/it]
 31%|███       | 70185/225000 [142:03:20<202:29:45,  4.71s/it]
 31%|███       | 70186/225000 [142:03:27<233:58:25,  5.44s/it]
 31%|███       | 70187/225000 [142:03:33<243:40:49,  5.67s/it]
 31%|███       | 70188/225000 [142:03:38<230:53:32,  5.37s/it]
 31%|███       | 70189/225000 [142:03:42<216:22:02,  5.03s/it]
 31%|███       | 70190/225000 [142:03:46<208:30:28,  4.85s/it]
 31%|███       | 70191/225000 [142:03:52<212:53:51,  4.95s/it]
 31%|███       | 70192/225000 [142:04:02<289:30:27,  6.73s/it]
 31%|███       | 70193/225000 [142:04:06<252:24:30,  5.87s/it]
 31%|███       | 70194/225000 [142:04:10<221:27:12,  5.15s/it]
 31%|███       | 70195/225000 [142:04:20<283:16:27,  6.59s/it]
 31%|███       | 70196/225000 [142:04:25<260:36:36,  6.06s/it]
 31%|███       | 70197/225000 [142:04:29<237:56:26,  5.53s/it]
 31%|███       | 70198/225000 [142:04:33<220:50:32,  5.14s/it]
 31%|███       | 70199/225000 [142:04:38<220:04:40,  5.12s/it]
 31%|███       | 70200/225000 [142:04:43<210:37:53,  4.90s/it]
                                                              
{'loss': '0.7652', 'grad_norm': '1.147', 'learning_rate': '4.095e-05', 'epoch': '0.312', 'num_input_tokens_seen': 235710080, 'train_runtime': '5.115e+05', 'train_tokens_per_second': '460.8'}

 31%|███       | 70200/225000 [142:04:43<210:37:53,  4.90s/it]
 31%|███       | 70201/225000 [142:04:46<198:03:12,  4.61s/it]
 31%|███       | 70202/225000 [142:04:58<283:57:34,  6.60s/it]
 31%|███       | 70203/225000 [142:05:01<247:17:29,  5.75s/it]
 31%|███       | 70204/225000 [142:05:05<223:43:17,  5.20s/it]
 31%|███       | 70205/225000 [142:05:09<208:59:02,  4.86s/it]
 31%|███       | 70206/225000 [142:05:18<254:03:16,  5.91s/it]
 31%|███       | 70207/225000 [142:05:26<280:41:15,  6.53s/it]
 31%|███       | 70208/225000 [142:05:32<278:07:04,  6.47s/it]
 31%|███       | 70209/225000 [142:05:35<231:52:19,  5.39s/it]
 31%|███       | 70210/225000 [142:05:38<198:25:58,  4.62s/it]
 31%|███       | 70211/225000 [142:05:46<246:11:04,  5.73s/it]
 31%|███       | 70212/225000 [142:05:54<276:08:56,  6.42s/it]
 31%|███       | 70213/225000 [142:05:58<240:05:20,  5.58s/it]
 31%|███       | 70214/225000 [142:06:02<226:10:25,  5.26s/it]
 31%|███       | 70215/225000 [142:06:06<210:44:37,  4.90s/it]
 31%|███       | 70216/225000 [142:06:10<197:53:55,  4.60s/it]
 31%|███       | 70217/225000 [142:06:14<185:04:04,  4.30s/it]
 31%|███       | 70218/225000 [142:06:24<264:21:51,  6.15s/it]
 31%|███       | 70219/225000 [142:06:28<235:38:46,  5.48s/it]
 31%|███       | 70220/225000 [142:06:32<213:51:01,  4.97s/it]
 31%|███       | 70221/225000 [142:06:36<194:37:23,  4.53s/it]
 31%|███       | 70222/225000 [142:06:46<276:38:15,  6.43s/it]
 31%|███       | 70223/225000 [142:06:49<224:28:34,  5.22s/it]
 31%|███       | 70224/225000 [142:06:55<232:50:49,  5.42s/it]
 31%|███       | 70225/225000 [142:07:03<276:44:20,  6.44s/it]
 31%|███       | 70226/225000 [142:07:07<239:17:37,  5.57s/it]
 31%|███       | 70227/225000 [142:07:10<208:10:16,  4.84s/it]
 31%|███       | 70228/225000 [142:07:15<205:51:18,  4.79s/it]
 31%|███       | 70229/225000 [142:07:19<199:43:00,  4.65s/it]
 31%|███       | 70230/225000 [142:07:24<200:10:41,  4.66s/it]
 31%|███       | 70231/225000 [142:07:31<233:45:18,  5.44s/it]
 31%|███       | 70232/225000 [142:07:44<324:40:12,  7.55s/it]
 31%|███       | 70233/225000 [142:07:47<268:10:49,  6.24s/it]
 31%|███       | 70234/225000 [142:07:53<272:44:07,  6.34s/it]
 31%|███       | 70235/225000 [142:07:58<244:36:44,  5.69s/it]
 31%|███       | 70236/225000 [142:08:04<257:10:01,  5.98s/it]
 31%|███       | 70237/225000 [142:08:10<254:47:53,  5.93s/it]
 31%|███       | 70238/225000 [142:08:15<248:52:44,  5.79s/it]
 31%|███       | 70239/225000 [142:08:20<226:43:33,  5.27s/it]
 31%|███       | 70240/225000 [142:08:23<205:27:12,  4.78s/it]
 31%|███       | 70241/225000 [142:08:27<194:40:53,  4.53s/it]
 31%|███       | 70242/225000 [142:08:35<238:51:40,  5.56s/it]
 31%|███       | 70243/225000 [142:08:41<246:24:25,  5.73s/it]
 31%|███       | 70244/225000 [142:08:46<240:41:00,  5.60s/it]
 31%|███       | 70245/225000 [142:08:53<250:06:53,  5.82s/it]
 31%|███       | 70246/225000 [142:08:56<215:43:31,  5.02s/it]
 31%|███       | 70247/225000 [142:09:04<260:17:01,  6.05s/it]
 31%|███       | 70248/225000 [142:09:08<229:53:05,  5.35s/it]
 31%|███       | 70249/225000 [142:09:12<206:10:15,  4.80s/it]
 31%|███       | 70250/225000 [142:09:15<184:40:23,  4.30s/it]
                                                              
{'loss': '0.7373', 'grad_norm': '1.378', 'learning_rate': '4.093e-05', 'epoch': '0.3122', 'num_input_tokens_seen': 235825560, 'train_runtime': '5.118e+05', 'train_tokens_per_second': '460.8'}

 31%|███       | 70250/225000 [142:09:15<184:40:23,  4.30s/it]
 31%|███       | 70251/225000 [142:09:20<199:46:50,  4.65s/it]
 31%|███       | 70252/225000 [142:09:28<240:01:16,  5.58s/it]
 31%|███       | 70253/225000 [142:09:33<238:31:54,  5.55s/it]
 31%|███       | 70254/225000 [142:09:37<212:34:04,  4.95s/it]
 31%|███       | 70255/225000 [142:09:40<186:58:29,  4.35s/it]
 31%|███       | 70256/225000 [142:09:48<236:16:54,  5.50s/it]
 31%|███       | 70257/225000 [142:09:52<216:52:16,  5.05s/it]
 31%|███       | 70258/225000 [142:09:59<239:12:37,  5.57s/it]
 31%|███       | 70259/225000 [142:10:06<257:47:39,  6.00s/it]
 31%|███       | 70260/225000 [142:10:11<252:03:09,  5.86s/it]
 31%|███       | 70261/225000 [142:10:16<233:19:40,  5.43s/it]
 31%|███       | 70262/225000 [142:10:20<219:46:09,  5.11s/it]
 31%|███       | 70263/225000 [142:10:25<217:31:28,  5.06s/it]
 31%|███       | 70264/225000 [142:10:30<208:22:09,  4.85s/it]
 31%|███       | 70265/225000 [142:10:34<198:07:53,  4.61s/it]
 31%|███       | 70266/225000 [142:10:45<282:34:25,  6.57s/it]
 31%|███       | 70267/225000 [142:10:49<258:48:11,  6.02s/it]
 31%|███       | 70268/225000 [142:10:52<219:15:40,  5.10s/it]
 31%|███       | 70269/225000 [142:10:55<187:20:36,  4.36s/it]
 31%|███       | 70270/225000 [142:11:03<238:41:14,  5.55s/it]
 31%|███       | 70271/225000 [142:11:07<219:12:19,  5.10s/it]
 31%|███       | 70272/225000 [142:11:11<201:17:17,  4.68s/it]
 31%|███       | 70273/225000 [142:11:15<189:31:05,  4.41s/it]
 31%|███       | 70274/225000 [142:11:25<258:58:05,  6.03s/it]
 31%|███       | 70275/225000 [142:11:29<240:46:04,  5.60s/it]
 31%|███       | 70276/225000 [142:11:33<214:02:25,  4.98s/it]
 31%|███       | 70277/225000 [142:11:37<198:04:27,  4.61s/it]
 31%|███       | 70278/225000 [142:11:41<194:18:36,  4.52s/it]
 31%|███       | 70279/225000 [142:11:53<296:14:18,  6.89s/it]
 31%|███       | 70280/225000 [142:11:59<275:54:03,  6.42s/it]
 31%|███       | 70281/225000 [142:12:02<239:09:09,  5.56s/it]
 31%|███       | 70282/225000 [142:12:06<217:17:28,  5.06s/it]
 31%|███       | 70283/225000 [142:12:10<202:48:36,  4.72s/it]
 31%|███       | 70284/225000 [142:12:14<194:33:22,  4.53s/it]
 31%|███       | 70285/225000 [142:12:24<266:07:53,  6.19s/it]
 31%|███       | 70286/225000 [142:12:31<272:36:14,  6.34s/it]
 31%|███       | 70287/225000 [142:12:36<255:33:06,  5.95s/it]
 31%|███       | 70288/225000 [142:12:44<278:47:39,  6.49s/it]
 31%|███       | 70289/225000 [142:12:47<237:14:02,  5.52s/it]
 31%|███       | 70290/225000 [142:12:51<213:33:29,  4.97s/it]
 31%|███       | 70291/225000 [142:12:55<210:00:58,  4.89s/it]
 31%|███       | 70292/225000 [142:13:03<244:13:38,  5.68s/it]
 31%|███       | 70293/225000 [142:13:06<217:09:51,  5.05s/it]
 31%|███       | 70294/225000 [142:13:11<206:53:03,  4.81s/it]
 31%|███       | 70295/225000 [142:13:14<184:48:40,  4.30s/it]
 31%|███       | 70296/225000 [142:13:18<184:44:11,  4.30s/it]
 31%|███       | 70297/225000 [142:13:27<245:14:58,  5.71s/it]
 31%|███       | 70298/225000 [142:13:33<251:13:46,  5.85s/it]
 31%|███       | 70299/225000 [142:13:37<222:45:01,  5.18s/it]
 31%|███       | 70300/225000 [142:13:46<271:00:50,  6.31s/it]
                                                              
{'loss': '0.7524', 'grad_norm': '0.9833', 'learning_rate': '4.092e-05', 'epoch': '0.3124', 'num_input_tokens_seen': 235939512, 'train_runtime': '5.12e+05', 'train_tokens_per_second': '460.8'}

 31%|███       | 70300/225000 [142:13:46<271:00:50,  6.31s/it]
 31%|███       | 70301/225000 [142:13:52<272:47:32,  6.35s/it]
 31%|███       | 70302/225000 [142:13:56<234:35:35,  5.46s/it]
 31%|███       | 70303/225000 [142:13:59<207:51:36,  4.84s/it]
 31%|███       | 70304/225000 [142:14:07<249:18:09,  5.80s/it]
 31%|███       | 70305/225000 [142:14:11<228:47:54,  5.32s/it]
 31%|███       | 70306/225000 [142:14:16<225:30:11,  5.25s/it]
 31%|███       | 70307/225000 [142:14:21<223:46:38,  5.21s/it]
 31%|███       | 70308/225000 [142:14:25<205:19:03,  4.78s/it]
 31%|███       | 70309/225000 [142:14:33<238:30:32,  5.55s/it]
 31%|███       | 70310/225000 [142:14:35<198:44:10,  4.63s/it]
 31%|███       | 70311/225000 [142:14:38<181:58:09,  4.23s/it]
 31%|███       | 70312/225000 [142:14:44<202:31:49,  4.71s/it]
 31%|███▏      | 70313/225000 [142:14:50<220:37:33,  5.13s/it]
 31%|███▏      | 70314/225000 [142:14:54<202:06:29,  4.70s/it]
 31%|███▏      | 70315/225000 [142:14:57<178:25:17,  4.15s/it]
 31%|███▏      | 70316/225000 [142:15:03<209:54:06,  4.89s/it]
 31%|███▏      | 70317/225000 [142:15:09<214:58:09,  5.00s/it]
 31%|███▏      | 70318/225000 [142:15:12<192:03:58,  4.47s/it]
 31%|███▏      | 70319/225000 [142:15:16<188:22:53,  4.38s/it]
 31%|███▏      | 70320/225000 [142:15:27<270:25:28,  6.29s/it]
 31%|███▏      | 70321/225000 [142:15:31<237:44:51,  5.53s/it]
 31%|███▏      | 70322/225000 [142:15:34<207:41:49,  4.83s/it]
 31%|███▏      | 70323/225000 [142:15:38<193:57:30,  4.51s/it]
 31%|███▏      | 70324/225000 [142:15:42<188:59:22,  4.40s/it]
 31%|███▏      | 70325/225000 [142:15:46<191:03:04,  4.45s/it]
 31%|███▏      | 70326/225000 [142:15:52<204:14:48,  4.75s/it]
 31%|███▏      | 70327/225000 [142:15:57<207:40:34,  4.83s/it]
 31%|███▏      | 70328/225000 [142:16:06<261:25:46,  6.08s/it]
 31%|███▏      | 70329/225000 [142:16:15<298:43:06,  6.95s/it]
 31%|███▏      | 70330/225000 [142:16:18<255:14:19,  5.94s/it]
 31%|███▏      | 70331/225000 [142:16:22<219:19:45,  5.11s/it]
 31%|███▏      | 70332/225000 [142:16:30<258:20:55,  6.01s/it]
 31%|███▏      | 70333/225000 [142:16:33<219:34:38,  5.11s/it]
 31%|███▏      | 70334/225000 [142:16:36<190:43:58,  4.44s/it]
 31%|███▏      | 70335/225000 [142:16:39<178:06:00,  4.15s/it]
 31%|███▏      | 70336/225000 [142:16:48<241:47:47,  5.63s/it]
 31%|███▏      | 70337/225000 [142:16:54<242:33:14,  5.65s/it]
 31%|███▏      | 70338/225000 [142:16:58<219:12:42,  5.10s/it]
 31%|███▏      | 70339/225000 [142:17:02<203:56:37,  4.75s/it]
 31%|███▏      | 70340/225000 [142:17:12<280:19:12,  6.52s/it]
 31%|███▏      | 70341/225000 [142:17:17<259:46:34,  6.05s/it]
 31%|███▏      | 70342/225000 [142:17:22<244:17:52,  5.69s/it]
 31%|███▏      | 70343/225000 [142:17:27<229:58:02,  5.35s/it]
 31%|███▏      | 70344/225000 [142:17:32<226:29:48,  5.27s/it]
 31%|███▏      | 70345/225000 [142:17:36<216:02:40,  5.03s/it]
 31%|███▏      | 70346/225000 [142:17:41<210:21:11,  4.90s/it]
 31%|███▏      | 70347/225000 [142:17:45<204:10:26,  4.75s/it]
 31%|███▏      | 70348/225000 [142:17:55<265:00:00,  6.17s/it]
 31%|███▏      | 70349/225000 [142:17:59<236:02:57,  5.49s/it]
 31%|███▏      | 70350/225000 [142:18:07<268:56:25,  6.26s/it]
                                                              
{'loss': '0.7486', 'grad_norm': '1.146', 'learning_rate': '4.09e-05', 'epoch': '0.3127', 'num_input_tokens_seen': 236054544, 'train_runtime': '5.123e+05', 'train_tokens_per_second': '460.8'}

 31%|███▏      | 70350/225000 [142:18:07<268:56:25,  6.26s/it]
 31%|███▏      | 70351/225000 [142:18:10<227:27:53,  5.30s/it]
 31%|███▏      | 70352/225000 [142:18:13<201:34:01,  4.69s/it]
 31%|███▏      | 70353/225000 [142:18:17<187:56:11,  4.37s/it]
 31%|███▏      | 70354/225000 [142:18:20<171:56:46,  4.00s/it]
 31%|███▏      | 70355/225000 [142:18:24<180:53:52,  4.21s/it]
 31%|███▏      | 70356/225000 [142:18:28<177:20:11,  4.13s/it]
 31%|███▏      | 70357/225000 [142:18:37<236:02:45,  5.50s/it]
 31%|███▏      | 70358/225000 [142:18:46<281:48:08,  6.56s/it]
 31%|███▏      | 70359/225000 [142:18:49<241:50:12,  5.63s/it]
 31%|███▏      | 70360/225000 [142:18:53<217:34:55,  5.07s/it]
 31%|███▏      | 70361/225000 [142:19:03<273:59:13,  6.38s/it]
 31%|███▏      | 70362/225000 [142:19:06<238:13:59,  5.55s/it]
 31%|███▏      | 70363/225000 [142:19:10<209:03:07,  4.87s/it]
 31%|███▏      | 70364/225000 [142:19:14<200:28:34,  4.67s/it]
 31%|███▏      | 70365/225000 [142:19:18<190:09:58,  4.43s/it]
 31%|███▏      | 70366/225000 [142:19:27<258:20:11,  6.01s/it]
 31%|███▏      | 70367/225000 [142:19:32<245:50:12,  5.72s/it]
 31%|███▏      | 70368/225000 [142:19:39<257:24:30,  5.99s/it]
 31%|███▏      | 70369/225000 [142:19:45<251:46:21,  5.86s/it]
 31%|███▏      | 70370/225000 [142:19:48<214:34:06,  5.00s/it]
 31%|███▏      | 70371/225000 [142:19:51<197:24:02,  4.60s/it]
 31%|███▏      | 70372/225000 [142:20:00<246:02:04,  5.73s/it]
 31%|███▏      | 70373/225000 [142:20:06<250:56:05,  5.84s/it]
 31%|███▏      | 70374/225000 [142:20:08<211:33:50,  4.93s/it]
 31%|███▏      | 70375/225000 [142:20:12<192:12:47,  4.48s/it]
 31%|███▏      | 70376/225000 [142:20:16<186:30:26,  4.34s/it]
 31%|███▏      | 70377/225000 [142:20:21<197:35:51,  4.60s/it]
 31%|███▏      | 70378/225000 [142:20:25<182:54:22,  4.26s/it]
 31%|███▏      | 70379/225000 [142:20:29<189:33:39,  4.41s/it]
 31%|███▏      | 70380/225000 [142:20:37<230:45:22,  5.37s/it]
 31%|███▏      | 70381/225000 [142:20:46<281:58:32,  6.57s/it]
 31%|███▏      | 70382/225000 [142:20:50<249:26:11,  5.81s/it]
 31%|███▏      | 70383/225000 [142:20:55<227:53:54,  5.31s/it]
 31%|███▏      | 70384/225000 [142:20:59<221:41:51,  5.16s/it]
 31%|███▏      | 70385/225000 [142:21:06<239:08:39,  5.57s/it]
 31%|███▏      | 70386/225000 [142:21:13<253:34:28,  5.90s/it]
 31%|███▏      | 70387/225000 [142:21:17<229:29:54,  5.34s/it]
 31%|███▏      | 70388/225000 [142:21:23<239:33:22,  5.58s/it]
 31%|███▏      | 70389/225000 [142:21:30<264:36:56,  6.16s/it]
 31%|███▏      | 70390/225000 [142:21:33<219:07:12,  5.10s/it]
 31%|███▏      | 70391/225000 [142:21:37<204:24:23,  4.76s/it]
 31%|███▏      | 70392/225000 [142:21:42<214:43:20,  5.00s/it]
 31%|███▏      | 70393/225000 [142:21:49<241:28:21,  5.62s/it]
 31%|███▏      | 70394/225000 [142:21:54<222:45:50,  5.19s/it]
 31%|███▏      | 70395/225000 [142:21:58<208:53:31,  4.86s/it]
 31%|███▏      | 70396/225000 [142:22:01<193:11:28,  4.50s/it]
 31%|███▏      | 70397/225000 [142:22:05<182:24:58,  4.25s/it]
 31%|███▏      | 70398/225000 [142:22:08<171:50:31,  4.00s/it]
 31%|███▏      | 70399/225000 [142:22:12<161:49:21,  3.77s/it]
 31%|███▏      | 70400/225000 [142:22:16<172:05:47,  4.01s/it]
                                                              
{'loss': '0.7582', 'grad_norm': '1.309', 'learning_rate': '4.089e-05', 'epoch': '0.3129', 'num_input_tokens_seen': 236164192, 'train_runtime': '5.125e+05', 'train_tokens_per_second': '460.8'}

 31%|███▏      | 70400/225000 [142:22:16<172:05:47,  4.01s/it]
 31%|███▏      | 70401/225000 [142:22:20<171:01:30,  3.98s/it]
 31%|███▏      | 70402/225000 [142:22:32<266:37:31,  6.21s/it]
 31%|███▏      | 70403/225000 [142:22:39<279:01:28,  6.50s/it]
 31%|███▏      | 70404/225000 [142:22:42<242:27:08,  5.65s/it]
 31%|███▏      | 70405/225000 [142:22:47<223:12:19,  5.20s/it]
 31%|███▏      | 70406/225000 [142:22:56<283:59:03,  6.61s/it]
 31%|███▏      | 70407/225000 [142:23:00<239:15:49,  5.57s/it]
 31%|███▏      | 70408/225000 [142:23:06<252:08:40,  5.87s/it]
 31%|███▏      | 70409/225000 [142:23:11<235:41:36,  5.49s/it]
 31%|███▏      | 70410/225000 [142:23:15<217:51:40,  5.07s/it]
 31%|███▏      | 70411/225000 [142:23:19<202:48:26,  4.72s/it]
 31%|███▏      | 70412/225000 [142:23:23<195:04:38,  4.54s/it]
 31%|███▏      | 70413/225000 [142:23:28<202:43:41,  4.72s/it]
 31%|███▏      | 70414/225000 [142:23:32<193:07:07,  4.50s/it]
 31%|███▏      | 70415/225000 [142:23:36<188:24:26,  4.39s/it]
 31%|███▏      | 70416/225000 [142:23:40<180:19:44,  4.20s/it]
 31%|███▏      | 70417/225000 [142:23:44<174:40:48,  4.07s/it]
 31%|███▏      | 70418/225000 [142:23:49<193:33:19,  4.51s/it]
 31%|███▏      | 70419/225000 [142:23:54<200:07:37,  4.66s/it]
 31%|███▏      | 70420/225000 [142:24:00<212:13:39,  4.94s/it]
 31%|███▏      | 70421/225000 [142:24:05<217:20:55,  5.06s/it]
 31%|███▏      | 70422/225000 [142:24:09<196:16:32,  4.57s/it]
 31%|███▏      | 70423/225000 [142:24:13<195:25:47,  4.55s/it]
 31%|███▏      | 70424/225000 [142:24:18<198:27:51,  4.62s/it]
 31%|███▏      | 70425/225000 [142:24:28<269:56:16,  6.29s/it]
 31%|███▏      | 70426/225000 [142:24:38<311:53:19,  7.26s/it]
 31%|███▏      | 70427/225000 [142:24:41<255:35:13,  5.95s/it]
 31%|███▏      | 70428/225000 [142:24:49<291:28:20,  6.79s/it]
 31%|███▏      | 70429/225000 [142:24:53<253:25:52,  5.90s/it]
 31%|███▏      | 70430/225000 [142:24:58<234:52:54,  5.47s/it]
 31%|███▏      | 70431/225000 [142:25:02<228:03:36,  5.31s/it]
 31%|███▏      | 70432/225000 [142:25:06<205:29:30,  4.79s/it]
 31%|███▏      | 70433/225000 [142:25:10<197:44:53,  4.61s/it]
 31%|███▏      | 70434/225000 [142:25:14<191:28:03,  4.46s/it]
 31%|███▏      | 70435/225000 [142:25:19<188:25:47,  4.39s/it]
 31%|███▏      | 70436/225000 [142:25:23<190:39:00,  4.44s/it]
 31%|███▏      | 70437/225000 [142:25:26<174:36:47,  4.07s/it]
 31%|███▏      | 70438/225000 [142:25:35<234:15:30,  5.46s/it]
 31%|███▏      | 70439/225000 [142:25:43<264:49:21,  6.17s/it]
 31%|███▏      | 70440/225000 [142:25:48<253:38:41,  5.91s/it]
 31%|███▏      | 70441/225000 [142:25:52<232:42:48,  5.42s/it]
 31%|███▏      | 70442/225000 [142:25:56<209:59:54,  4.89s/it]
 31%|███▏      | 70443/225000 [142:26:00<197:09:46,  4.59s/it]
 31%|███▏      | 70444/225000 [142:26:05<207:24:58,  4.83s/it]
 31%|███▏      | 70445/225000 [142:26:17<295:13:27,  6.88s/it]
 31%|███▏      | 70446/225000 [142:26:22<269:29:32,  6.28s/it]
 31%|███▏      | 70447/225000 [142:26:26<241:06:35,  5.62s/it]
 31%|███▏      | 70448/225000 [142:26:32<241:41:43,  5.63s/it]
 31%|███▏      | 70449/225000 [142:26:36<222:51:03,  5.19s/it]
 31%|███▏      | 70450/225000 [142:26:40<205:44:28,  4.79s/it]
                                                              
{'loss': '0.7307', 'grad_norm': '1.24', 'learning_rate': '4.087e-05', 'epoch': '0.3131', 'num_input_tokens_seen': 236276960, 'train_runtime': '5.128e+05', 'train_tokens_per_second': '460.8'}

 31%|███▏      | 70450/225000 [142:26:40<205:44:28,  4.79s/it]
 31%|███▏      | 70451/225000 [142:26:44<205:52:17,  4.80s/it]
 31%|███▏      | 70452/225000 [142:26:49<197:23:28,  4.60s/it]
 31%|███▏      | 70453/225000 [142:26:53<195:41:01,  4.56s/it]
 31%|███▏      | 70454/225000 [142:26:59<209:50:42,  4.89s/it]
 31%|███▏      | 70455/225000 [142:27:11<299:28:05,  6.98s/it]
 31%|███▏      | 70456/225000 [142:27:14<251:14:59,  5.85s/it]
 31%|███▏      | 70457/225000 [142:27:17<216:55:13,  5.05s/it]
 31%|███▏      | 70458/225000 [142:27:21<199:38:47,  4.65s/it]
 31%|███▏      | 70459/225000 [142:27:25<193:21:51,  4.50s/it]
 31%|███▏      | 70460/225000 [142:27:36<273:41:36,  6.38s/it]
 31%|███▏      | 70461/225000 [142:27:42<271:36:59,  6.33s/it]
 31%|███▏      | 70462/225000 [142:27:45<235:18:03,  5.48s/it]
 31%|███▏      | 70463/225000 [142:27:53<263:47:09,  6.14s/it]
 31%|███▏      | 70464/225000 [142:27:57<240:30:13,  5.60s/it]
 31%|███▏      | 70465/225000 [142:28:01<211:40:17,  4.93s/it]
 31%|███▏      | 70466/225000 [142:28:05<203:56:48,  4.75s/it]
 31%|███▏      | 70467/225000 [142:28:10<207:09:47,  4.83s/it]
 31%|███▏      | 70468/225000 [142:28:21<285:58:29,  6.66s/it]
 31%|███▏      | 70469/225000 [142:28:25<250:06:24,  5.83s/it]
 31%|███▏      | 70470/225000 [142:28:29<226:27:32,  5.28s/it]
 31%|███▏      | 70471/225000 [142:28:35<237:14:13,  5.53s/it]
 31%|███▏      | 70472/225000 [142:28:46<308:39:43,  7.19s/it]
 31%|███▏      | 70473/225000 [142:28:49<259:58:03,  6.06s/it]
 31%|███▏      | 70474/225000 [142:28:53<223:04:12,  5.20s/it]
 31%|███▏      | 70475/225000 [142:28:56<199:08:57,  4.64s/it]
 31%|███▏      | 70476/225000 [142:29:00<191:40:44,  4.47s/it]
 31%|███▏      | 70477/225000 [142:29:07<229:00:22,  5.34s/it]
 31%|███▏      | 70478/225000 [142:29:13<233:13:22,  5.43s/it]
 31%|███▏      | 70479/225000 [142:29:16<201:55:51,  4.70s/it]
 31%|███▏      | 70480/225000 [142:29:26<264:09:37,  6.15s/it]
 31%|███▏      | 70481/225000 [142:29:29<225:54:13,  5.26s/it]
 31%|███▏      | 70482/225000 [142:29:33<213:32:53,  4.98s/it]
 31%|███▏      | 70483/225000 [142:29:37<202:24:01,  4.72s/it]
 31%|███▏      | 70484/225000 [142:29:41<190:13:11,  4.43s/it]
 31%|███▏      | 70485/225000 [142:29:46<194:23:02,  4.53s/it]
 31%|███▏      | 70486/225000 [142:29:55<248:58:58,  5.80s/it]
 31%|███▏      | 70487/225000 [142:29:58<224:18:21,  5.23s/it]
 31%|███▏      | 70488/225000 [142:30:02<205:40:04,  4.79s/it]
 31%|███▏      | 70489/225000 [142:30:07<204:06:10,  4.76s/it]
 31%|███▏      | 70490/225000 [142:30:10<187:24:09,  4.37s/it]
 31%|███▏      | 70491/225000 [142:30:15<187:32:03,  4.37s/it]
 31%|███▏      | 70492/225000 [142:30:18<177:42:40,  4.14s/it]
 31%|███▏      | 70493/225000 [142:30:23<186:18:30,  4.34s/it]
 31%|███▏      | 70494/225000 [142:30:32<248:36:09,  5.79s/it]
 31%|███▏      | 70495/225000 [142:30:37<234:30:12,  5.46s/it]
 31%|███▏      | 70496/225000 [142:30:41<209:50:13,  4.89s/it]
 31%|███▏      | 70497/225000 [142:30:45<204:30:03,  4.76s/it]
 31%|███▏      | 70498/225000 [142:30:49<190:26:48,  4.44s/it]
 31%|███▏      | 70499/225000 [142:30:55<220:37:46,  5.14s/it]
 31%|███▏      | 70500/225000 [142:31:02<244:17:11,  5.69s/it]
                                                              
{'loss': '0.7567', 'grad_norm': '1.233', 'learning_rate': '4.086e-05', 'epoch': '0.3133', 'num_input_tokens_seen': 236389488, 'train_runtime': '5.131e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70500/225000 [142:31:02<244:17:11,  5.69s/it]
 31%|███▏      | 70501/225000 [142:31:06<213:02:25,  4.96s/it]
 31%|███▏      | 70502/225000 [142:31:09<190:48:50,  4.45s/it]
 31%|███▏      | 70503/225000 [142:31:14<197:48:08,  4.61s/it]
 31%|███▏      | 70504/225000 [142:31:17<182:04:13,  4.24s/it]
 31%|███▏      | 70505/225000 [142:31:22<185:47:19,  4.33s/it]
 31%|███▏      | 70506/225000 [142:31:25<172:34:37,  4.02s/it]
 31%|███▏      | 70507/225000 [142:31:29<171:45:05,  4.00s/it]
 31%|███▏      | 70508/225000 [142:31:35<201:25:43,  4.69s/it]
 31%|███▏      | 70509/225000 [142:31:45<265:05:52,  6.18s/it]
 31%|███▏      | 70510/225000 [142:31:50<246:41:02,  5.75s/it]
 31%|███▏      | 70511/225000 [142:31:53<213:06:23,  4.97s/it]
 31%|███▏      | 70512/225000 [142:31:56<190:38:01,  4.44s/it]
 31%|███▏      | 70513/225000 [142:32:03<218:02:33,  5.08s/it]
 31%|███▏      | 70514/225000 [142:32:13<279:03:03,  6.50s/it]
 31%|███▏      | 70515/225000 [142:32:16<235:58:15,  5.50s/it]
 31%|███▏      | 70516/225000 [142:32:23<258:13:15,  6.02s/it]
 31%|███▏      | 70517/225000 [142:32:26<224:41:26,  5.24s/it]
 31%|███▏      | 70518/225000 [142:32:33<241:07:06,  5.62s/it]
 31%|███▏      | 70519/225000 [142:32:39<242:31:27,  5.65s/it]
 31%|███▏      | 70520/225000 [142:32:42<219:09:55,  5.11s/it]
 31%|███▏      | 70521/225000 [142:32:46<196:04:19,  4.57s/it]
 31%|███▏      | 70522/225000 [142:32:52<223:10:14,  5.20s/it]
 31%|███▏      | 70523/225000 [142:32:59<240:11:50,  5.60s/it]
 31%|███▏      | 70524/225000 [142:33:02<202:49:22,  4.73s/it]
 31%|███▏      | 70525/225000 [142:33:06<196:02:03,  4.57s/it]
 31%|███▏      | 70526/225000 [142:33:10<185:54:51,  4.33s/it]
 31%|███▏      | 70527/225000 [142:33:13<172:45:29,  4.03s/it]
 31%|███▏      | 70528/225000 [142:33:23<255:53:22,  5.96s/it]
 31%|███▏      | 70529/225000 [142:33:27<228:35:25,  5.33s/it]
 31%|███▏      | 70530/225000 [142:33:31<202:41:30,  4.72s/it]
 31%|███▏      | 70531/225000 [142:33:34<190:28:53,  4.44s/it]
 31%|███▏      | 70532/225000 [142:33:38<182:40:00,  4.26s/it]
 31%|███▏      | 70533/225000 [142:33:42<171:26:51,  4.00s/it]
 31%|███▏      | 70534/225000 [142:33:48<207:21:58,  4.83s/it]
 31%|███▏      | 70535/225000 [142:33:56<246:20:12,  5.74s/it]
 31%|███▏      | 70536/225000 [142:33:59<210:01:23,  4.89s/it]
 31%|███▏      | 70537/225000 [142:34:06<231:03:18,  5.39s/it]
 31%|███▏      | 70538/225000 [142:34:13<252:05:31,  5.88s/it]
 31%|███▏      | 70539/225000 [142:34:18<243:34:22,  5.68s/it]
 31%|███▏      | 70540/225000 [142:34:22<223:54:18,  5.22s/it]
 31%|███▏      | 70541/225000 [142:34:26<203:24:16,  4.74s/it]
 31%|███▏      | 70542/225000 [142:34:35<266:36:54,  6.21s/it]
 31%|███▏      | 70543/225000 [142:34:39<235:48:23,  5.50s/it]
 31%|███▏      | 70544/225000 [142:34:45<235:10:10,  5.48s/it]
 31%|███▏      | 70545/225000 [142:34:48<208:02:06,  4.85s/it]
 31%|███▏      | 70546/225000 [142:34:52<203:07:20,  4.73s/it]
 31%|███▏      | 70547/225000 [142:34:57<205:02:31,  4.78s/it]
 31%|███▏      | 70548/225000 [142:35:02<199:41:49,  4.65s/it]
 31%|███▏      | 70549/225000 [142:35:05<183:30:30,  4.28s/it]
 31%|███▏      | 70550/225000 [142:35:09<185:19:44,  4.32s/it]
                                                              
{'loss': '0.7984', 'grad_norm': '1.234', 'learning_rate': '4.085e-05', 'epoch': '0.3136', 'num_input_tokens_seen': 236497184, 'train_runtime': '5.133e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70550/225000 [142:35:09<185:19:44,  4.32s/it]
 31%|███▏      | 70551/225000 [142:35:12<165:58:16,  3.87s/it]
 31%|███▏      | 70552/225000 [142:35:17<180:50:00,  4.22s/it]
 31%|███▏      | 70553/225000 [142:35:22<189:40:38,  4.42s/it]
 31%|███▏      | 70554/225000 [142:35:32<261:38:23,  6.10s/it]
 31%|███▏      | 70555/225000 [142:35:38<259:55:22,  6.06s/it]
 31%|███▏      | 70556/225000 [142:35:43<248:32:59,  5.79s/it]
 31%|███▏      | 70557/225000 [142:35:48<229:43:34,  5.35s/it]
 31%|███▏      | 70558/225000 [142:35:50<194:22:12,  4.53s/it]
 31%|███▏      | 70559/225000 [142:35:53<176:13:29,  4.11s/it]
 31%|███▏      | 70560/225000 [142:35:58<176:40:38,  4.12s/it]
 31%|███▏      | 70561/225000 [142:36:08<256:27:52,  5.98s/it]
 31%|███▏      | 70562/225000 [142:36:13<244:57:13,  5.71s/it]
 31%|███▏      | 70563/225000 [142:36:17<223:02:40,  5.20s/it]
 31%|███▏      | 70564/225000 [142:36:20<192:59:52,  4.50s/it]
 31%|███▏      | 70565/225000 [142:36:23<179:02:18,  4.17s/it]
 31%|███▏      | 70566/225000 [142:36:33<249:22:00,  5.81s/it]
 31%|███▏      | 70567/225000 [142:36:37<222:03:33,  5.18s/it]
 31%|███▏      | 70568/225000 [142:36:40<200:43:42,  4.68s/it]
 31%|███▏      | 70569/225000 [142:36:44<187:25:17,  4.37s/it]
 31%|███▏      | 70570/225000 [142:36:48<184:01:53,  4.29s/it]
 31%|███▏      | 70571/225000 [142:36:52<186:19:30,  4.34s/it]
 31%|███▏      | 70572/225000 [142:37:02<258:51:22,  6.03s/it]
 31%|███▏      | 70573/225000 [142:37:08<257:15:13,  6.00s/it]
 31%|███▏      | 70574/225000 [142:37:13<244:53:50,  5.71s/it]
 31%|███▏      | 70575/225000 [142:37:18<231:03:53,  5.39s/it]
 31%|███▏      | 70576/225000 [142:37:22<210:13:20,  4.90s/it]
 31%|███▏      | 70577/225000 [142:37:31<266:16:21,  6.21s/it]
 31%|███▏      | 70578/225000 [142:37:34<230:27:44,  5.37s/it]
 31%|███▏      | 70579/225000 [142:37:38<203:28:21,  4.74s/it]
 31%|███▏      | 70580/225000 [142:37:41<184:19:59,  4.30s/it]
 31%|███▏      | 70581/225000 [142:37:45<175:32:58,  4.09s/it]
 31%|███▏      | 70582/225000 [142:37:53<232:21:52,  5.42s/it]
 31%|███▏      | 70583/225000 [142:38:00<254:08:12,  5.92s/it]
 31%|███▏      | 70584/225000 [142:38:07<263:00:44,  6.13s/it]
 31%|███▏      | 70585/225000 [142:38:10<229:52:16,  5.36s/it]
 31%|███▏      | 70586/225000 [142:38:14<207:05:04,  4.83s/it]
 31%|███▏      | 70587/225000 [142:38:23<266:49:54,  6.22s/it]
 31%|███▏      | 70588/225000 [142:38:28<242:24:23,  5.65s/it]
 31%|███▏      | 70589/225000 [142:38:31<206:30:16,  4.81s/it]
 31%|███▏      | 70590/225000 [142:38:38<234:34:51,  5.47s/it]
 31%|███▏      | 70591/225000 [142:38:41<206:30:49,  4.81s/it]
 31%|███▏      | 70592/225000 [142:38:44<183:37:34,  4.28s/it]
 31%|███▏      | 70593/225000 [142:38:48<175:37:44,  4.09s/it]
 31%|███▏      | 70594/225000 [142:38:51<174:01:12,  4.06s/it]
 31%|███▏      | 70595/225000 [142:38:59<218:17:33,  5.09s/it]
 31%|███▏      | 70596/225000 [142:39:06<246:56:26,  5.76s/it]
 31%|███▏      | 70597/225000 [142:39:10<222:52:10,  5.20s/it]
 31%|███▏      | 70598/225000 [142:39:14<205:09:09,  4.78s/it]
 31%|███▏      | 70599/225000 [142:39:18<189:50:26,  4.43s/it]
 31%|███▏      | 70600/225000 [142:39:21<171:15:41,  3.99s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.108', 'learning_rate': '4.083e-05', 'epoch': '0.3138', 'num_input_tokens_seen': 236605952, 'train_runtime': '5.136e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70600/225000 [142:39:21<171:15:41,  3.99s/it]
 31%|███▏      | 70601/225000 [142:39:30<239:44:04,  5.59s/it]
 31%|███▏      | 70602/225000 [142:39:36<251:05:19,  5.85s/it]
 31%|███▏      | 70603/225000 [142:39:42<246:05:25,  5.74s/it]
 31%|███▏      | 70604/225000 [142:39:48<252:09:12,  5.88s/it]
 31%|███▏      | 70605/225000 [142:39:54<246:56:50,  5.76s/it]
 31%|███▏      | 70606/225000 [142:39:57<216:59:50,  5.06s/it]
 31%|███▏      | 70607/225000 [142:40:03<235:03:18,  5.48s/it]
 31%|███▏      | 70608/225000 [142:40:10<243:42:37,  5.68s/it]
 31%|███▏      | 70609/225000 [142:40:13<210:25:30,  4.91s/it]
 31%|███▏      | 70610/225000 [142:40:17<198:09:35,  4.62s/it]
 31%|███▏      | 70611/225000 [142:40:21<194:15:06,  4.53s/it]
 31%|███▏      | 70612/225000 [142:40:24<180:58:54,  4.22s/it]
 31%|███▏      | 70613/225000 [142:40:35<260:45:43,  6.08s/it]
 31%|███▏      | 70614/225000 [142:40:37<213:21:31,  4.98s/it]
 31%|███▏      | 70615/225000 [142:40:43<218:01:14,  5.08s/it]
 31%|███▏      | 70616/225000 [142:40:46<191:19:26,  4.46s/it]
 31%|███▏      | 70617/225000 [142:40:51<200:28:59,  4.67s/it]
 31%|███▏      | 70618/225000 [142:40:54<180:12:57,  4.20s/it]
 31%|███▏      | 70619/225000 [142:40:57<167:46:09,  3.91s/it]
 31%|███▏      | 70620/225000 [142:41:02<176:02:56,  4.11s/it]
 31%|███▏      | 70621/225000 [142:41:08<206:23:49,  4.81s/it]
 31%|███▏      | 70622/225000 [142:41:16<244:47:08,  5.71s/it]
 31%|███▏      | 70623/225000 [142:41:20<219:56:41,  5.13s/it]
 31%|███▏      | 70624/225000 [142:41:22<189:12:17,  4.41s/it]
 31%|███▏      | 70625/225000 [142:41:31<241:11:29,  5.62s/it]
 31%|███▏      | 70626/225000 [142:41:34<206:54:20,  4.83s/it]
 31%|███▏      | 70627/225000 [142:41:38<193:38:56,  4.52s/it]
 31%|███▏      | 70628/225000 [142:41:41<176:59:43,  4.13s/it]
 31%|███▏      | 70629/225000 [142:41:45<177:00:49,  4.13s/it]
 31%|███▏      | 70630/225000 [142:41:49<181:39:48,  4.24s/it]
 31%|███▏      | 70631/225000 [142:41:54<181:51:03,  4.24s/it]
 31%|███▏      | 70632/225000 [142:42:02<234:20:33,  5.47s/it]
 31%|███▏      | 70633/225000 [142:42:12<287:05:43,  6.70s/it]
 31%|███▏      | 70634/225000 [142:42:15<243:37:23,  5.68s/it]
 31%|███▏      | 70635/225000 [142:42:19<226:48:25,  5.29s/it]
 31%|███▏      | 70636/225000 [142:42:22<198:20:39,  4.63s/it]
 31%|███▏      | 70637/225000 [142:42:26<188:04:53,  4.39s/it]
 31%|███▏      | 70638/225000 [142:42:31<187:59:52,  4.38s/it]
 31%|███▏      | 70639/225000 [142:42:35<192:24:36,  4.49s/it]
 31%|███▏      | 70640/225000 [142:42:47<281:47:40,  6.57s/it]
 31%|███▏      | 70641/225000 [142:42:51<253:05:38,  5.90s/it]
 31%|███▏      | 70642/225000 [142:42:54<220:35:25,  5.14s/it]
 31%|███▏      | 70643/225000 [142:43:04<274:26:59,  6.40s/it]
 31%|███▏      | 70644/225000 [142:43:07<227:34:54,  5.31s/it]
 31%|███▏      | 70645/225000 [142:43:13<242:23:27,  5.65s/it]
 31%|███▏      | 70646/225000 [142:43:19<244:53:38,  5.71s/it]
 31%|███▏      | 70647/225000 [142:43:22<212:56:06,  4.97s/it]
 31%|███▏      | 70648/225000 [142:43:26<200:33:28,  4.68s/it]
 31%|███▏      | 70649/225000 [142:43:30<191:40:05,  4.47s/it]
 31%|███▏      | 70650/225000 [142:43:39<247:40:12,  5.78s/it]
                                                              
{'loss': '0.7632', 'grad_norm': '1.333', 'learning_rate': '4.082e-05', 'epoch': '0.314', 'num_input_tokens_seen': 236714176, 'train_runtime': '5.138e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70650/225000 [142:43:39<247:40:12,  5.78s/it]
 31%|███▏      | 70651/225000 [142:43:43<223:41:58,  5.22s/it]
 31%|███▏      | 70652/225000 [142:43:46<202:25:06,  4.72s/it]
 31%|███▏      | 70653/225000 [142:43:52<215:56:28,  5.04s/it]
 31%|███▏      | 70654/225000 [142:43:59<244:04:51,  5.69s/it]
 31%|███▏      | 70655/225000 [142:44:02<210:03:59,  4.90s/it]
 31%|███▏      | 70656/225000 [142:44:06<189:41:23,  4.42s/it]
 31%|███▏      | 70657/225000 [142:44:10<182:42:34,  4.26s/it]
 31%|███▏      | 70658/225000 [142:44:13<172:42:04,  4.03s/it]
 31%|███▏      | 70659/225000 [142:44:17<171:02:09,  3.99s/it]
 31%|███▏      | 70660/225000 [142:44:24<208:14:01,  4.86s/it]
 31%|███▏      | 70661/225000 [142:44:35<282:31:53,  6.59s/it]
 31%|███▏      | 70662/225000 [142:44:38<245:35:40,  5.73s/it]
 31%|███▏      | 70663/225000 [142:44:43<233:51:04,  5.45s/it]
 31%|███▏      | 70664/225000 [142:44:47<218:36:02,  5.10s/it]
 31%|███▏      | 70665/225000 [142:44:51<201:38:06,  4.70s/it]
 31%|███▏      | 70666/225000 [142:44:55<189:15:07,  4.41s/it]
 31%|███▏      | 70667/225000 [142:44:58<176:04:36,  4.11s/it]
 31%|███▏      | 70668/225000 [142:45:02<175:24:50,  4.09s/it]
 31%|███▏      | 70669/225000 [142:45:08<190:34:08,  4.45s/it]
 31%|███▏      | 70670/225000 [142:45:18<266:41:50,  6.22s/it]
 31%|███▏      | 70671/225000 [142:45:22<234:22:19,  5.47s/it]
 31%|███▏      | 70672/225000 [142:45:26<215:57:09,  5.04s/it]
 31%|███▏      | 70673/225000 [142:45:29<195:19:13,  4.56s/it]
 31%|███▏      | 70674/225000 [142:45:33<189:40:45,  4.42s/it]
 31%|███▏      | 70675/225000 [142:45:44<267:35:27,  6.24s/it]
 31%|███▏      | 70676/225000 [142:45:48<247:08:12,  5.77s/it]
 31%|███▏      | 70677/225000 [142:45:52<218:35:15,  5.10s/it]
 31%|███▏      | 70678/225000 [142:45:56<202:24:55,  4.72s/it]
 31%|███▏      | 70679/225000 [142:45:59<182:19:15,  4.25s/it]
 31%|███▏      | 70680/225000 [142:46:09<256:17:07,  5.98s/it]
 31%|███▏      | 70681/225000 [142:46:14<240:40:01,  5.61s/it]
 31%|███▏      | 70682/225000 [142:46:17<212:11:58,  4.95s/it]
 31%|███▏      | 70683/225000 [142:46:21<197:20:43,  4.60s/it]
 31%|███▏      | 70684/225000 [142:46:24<183:08:55,  4.27s/it]
 31%|███▏      | 70685/225000 [142:46:33<235:34:15,  5.50s/it]
 31%|███▏      | 70686/225000 [142:46:35<198:55:32,  4.64s/it]
 31%|███▏      | 70687/225000 [142:46:44<247:24:51,  5.77s/it]
 31%|███▏      | 70688/225000 [142:46:47<217:09:40,  5.07s/it]
 31%|███▏      | 70689/225000 [142:46:53<220:41:41,  5.15s/it]
 31%|███▏      | 70690/225000 [142:46:57<212:22:28,  4.95s/it]
 31%|███▏      | 70691/225000 [142:47:04<232:09:18,  5.42s/it]
 31%|███▏      | 70692/225000 [142:47:08<215:56:49,  5.04s/it]
 31%|███▏      | 70693/225000 [142:47:10<186:51:22,  4.36s/it]
 31%|███▏      | 70694/225000 [142:47:14<171:27:06,  4.00s/it]
 31%|███▏      | 70695/225000 [142:47:17<161:59:53,  3.78s/it]
 31%|███▏      | 70696/225000 [142:47:21<164:57:14,  3.85s/it]
 31%|███▏      | 70697/225000 [142:47:32<256:34:07,  5.99s/it]
 31%|███▏      | 70698/225000 [142:47:34<213:01:34,  4.97s/it]
 31%|███▏      | 70699/225000 [142:47:38<191:32:09,  4.47s/it]
 31%|███▏      | 70700/225000 [142:47:46<237:06:14,  5.53s/it]
                                                              
{'loss': '0.7547', 'grad_norm': '1.159', 'learning_rate': '4.08e-05', 'epoch': '0.3142', 'num_input_tokens_seen': 236822352, 'train_runtime': '5.141e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70700/225000 [142:47:46<237:06:14,  5.53s/it]
 31%|███▏      | 70701/225000 [142:47:50<215:01:19,  5.02s/it]
 31%|███▏      | 70702/225000 [142:47:53<199:03:34,  4.64s/it]
 31%|███▏      | 70703/225000 [142:47:57<190:54:07,  4.45s/it]
 31%|███▏      | 70704/225000 [142:48:01<179:44:38,  4.19s/it]
 31%|███▏      | 70705/225000 [142:48:05<177:29:09,  4.14s/it]
 31%|███▏      | 70706/225000 [142:48:09<179:06:44,  4.18s/it]
 31%|███▏      | 70707/225000 [142:48:19<254:18:34,  5.93s/it]
 31%|███▏      | 70708/225000 [142:48:25<254:37:45,  5.94s/it]
 31%|███▏      | 70709/225000 [142:48:30<233:43:48,  5.45s/it]
 31%|███▏      | 70710/225000 [142:48:33<207:02:07,  4.83s/it]
 31%|███▏      | 70711/225000 [142:48:38<207:26:33,  4.84s/it]
 31%|███▏      | 70712/225000 [142:48:41<186:27:52,  4.35s/it]
 31%|███▏      | 70713/225000 [142:48:45<187:08:00,  4.37s/it]
 31%|███▏      | 70714/225000 [142:48:49<178:17:01,  4.16s/it]
 31%|███▏      | 70715/225000 [142:48:53<173:57:40,  4.06s/it]
 31%|███▏      | 70716/225000 [142:49:01<220:39:21,  5.15s/it]
 31%|███▏      | 70717/225000 [142:49:07<236:20:23,  5.51s/it]
 31%|███▏      | 70718/225000 [142:49:12<232:45:25,  5.43s/it]
 31%|███▏      | 70719/225000 [142:49:19<250:03:14,  5.83s/it]
 31%|███▏      | 70720/225000 [142:49:25<252:44:49,  5.90s/it]
 31%|███▏      | 70721/225000 [142:49:30<245:22:08,  5.73s/it]
 31%|███▏      | 70722/225000 [142:49:36<248:44:30,  5.80s/it]
 31%|███▏      | 70723/225000 [142:49:39<214:31:58,  5.01s/it]
 31%|███▏      | 70724/225000 [142:49:43<199:53:54,  4.66s/it]
 31%|███▏      | 70725/225000 [142:49:49<215:01:05,  5.02s/it]
 31%|███▏      | 70726/225000 [142:50:00<292:25:46,  6.82s/it]
 31%|███▏      | 70727/225000 [142:50:04<253:31:18,  5.92s/it]
 31%|███▏      | 70728/225000 [142:50:07<217:07:22,  5.07s/it]
 31%|███▏      | 70729/225000 [142:50:11<200:09:56,  4.67s/it]
 31%|███▏      | 70730/225000 [142:50:15<192:30:00,  4.49s/it]
 31%|███▏      | 70731/225000 [142:50:24<249:01:04,  5.81s/it]
 31%|███▏      | 70732/225000 [142:50:30<254:33:07,  5.94s/it]
 31%|███▏      | 70733/225000 [142:50:34<222:22:39,  5.19s/it]
 31%|███▏      | 70734/225000 [142:50:38<210:12:47,  4.91s/it]
 31%|███▏      | 70735/225000 [142:50:41<193:11:18,  4.51s/it]
 31%|███▏      | 70736/225000 [142:50:46<195:23:43,  4.56s/it]
 31%|███▏      | 70737/225000 [142:50:50<188:21:45,  4.40s/it]
 31%|███▏      | 70738/225000 [142:50:54<186:50:11,  4.36s/it]
 31%|███▏      | 70739/225000 [142:51:04<257:53:48,  6.02s/it]
 31%|███▏      | 70740/225000 [142:51:09<248:29:35,  5.80s/it]
 31%|███▏      | 70741/225000 [142:51:14<234:10:04,  5.46s/it]
 31%|███▏      | 70742/225000 [142:51:22<262:58:21,  6.14s/it]
 31%|███▏      | 70743/225000 [142:51:26<236:57:24,  5.53s/it]
 31%|███▏      | 70744/225000 [142:51:35<278:49:27,  6.51s/it]
 31%|███▏      | 70745/225000 [142:51:39<246:53:35,  5.76s/it]
 31%|███▏      | 70746/225000 [142:51:43<231:16:44,  5.40s/it]
 31%|███▏      | 70747/225000 [142:51:47<209:43:17,  4.89s/it]
 31%|███▏      | 70748/225000 [142:51:51<191:04:09,  4.46s/it]
 31%|███▏      | 70749/225000 [142:51:55<186:30:23,  4.35s/it]
 31%|███▏      | 70750/225000 [142:51:59<182:08:02,  4.25s/it]
                                                              
{'loss': '0.7181', 'grad_norm': '1.065', 'learning_rate': '4.079e-05', 'epoch': '0.3144', 'num_input_tokens_seen': 236933736, 'train_runtime': '5.143e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70750/225000 [142:51:59<182:08:02,  4.25s/it]
 31%|███▏      | 70751/225000 [142:52:09<258:14:59,  6.03s/it]
 31%|███▏      | 70752/225000 [142:52:14<251:30:06,  5.87s/it]
 31%|███▏      | 70753/225000 [142:52:17<213:51:13,  4.99s/it]
 31%|███▏      | 70754/225000 [142:52:21<199:35:30,  4.66s/it]
 31%|███▏      | 70755/225000 [142:52:26<203:29:53,  4.75s/it]
 31%|███▏      | 70756/225000 [142:52:30<189:37:10,  4.43s/it]
 31%|███▏      | 70757/225000 [142:52:35<198:30:53,  4.63s/it]
 31%|███▏      | 70758/225000 [142:52:41<217:13:08,  5.07s/it]
 31%|███▏      | 70759/225000 [142:52:47<230:02:34,  5.37s/it]
 31%|███▏      | 70760/225000 [142:52:52<223:58:38,  5.23s/it]
 31%|███▏      | 70761/225000 [142:52:56<205:06:50,  4.79s/it]
 31%|███▏      | 70762/225000 [142:53:00<193:26:51,  4.52s/it]
 31%|███▏      | 70763/225000 [142:53:06<214:53:36,  5.02s/it]
 31%|███▏      | 70764/225000 [142:53:17<297:36:58,  6.95s/it]
 31%|███▏      | 70765/225000 [142:53:21<253:29:28,  5.92s/it]
 31%|███▏      | 70766/225000 [142:53:24<223:42:44,  5.22s/it]
 31%|███▏      | 70767/225000 [142:53:28<201:45:05,  4.71s/it]
 31%|███▏      | 70768/225000 [142:53:36<242:38:54,  5.66s/it]
 31%|███▏      | 70769/225000 [142:53:39<210:56:33,  4.92s/it]
 31%|███▏      | 70770/225000 [142:53:43<196:19:55,  4.58s/it]
 31%|███▏      | 70771/225000 [142:53:52<262:06:11,  6.12s/it]
 31%|███▏      | 70772/225000 [142:53:55<217:30:50,  5.08s/it]
 31%|███▏      | 70773/225000 [142:53:58<186:48:35,  4.36s/it]
 31%|███▏      | 70774/225000 [142:54:01<169:06:14,  3.95s/it]
 31%|███▏      | 70775/225000 [142:54:08<211:14:29,  4.93s/it]
 31%|███▏      | 70776/225000 [142:54:14<229:08:02,  5.35s/it]
 31%|███▏      | 70777/225000 [142:54:18<209:32:13,  4.89s/it]
 31%|███▏      | 70778/225000 [142:54:21<188:02:47,  4.39s/it]
 31%|███▏      | 70779/225000 [142:54:28<212:33:54,  4.96s/it]
 31%|███▏      | 70780/225000 [142:54:37<266:37:26,  6.22s/it]
 31%|███▏      | 70781/225000 [142:54:40<229:16:59,  5.35s/it]
 31%|███▏      | 70782/225000 [142:54:43<195:45:54,  4.57s/it]
 31%|███▏      | 70783/225000 [142:54:51<239:23:56,  5.59s/it]
 31%|███▏      | 70784/225000 [142:54:56<234:25:10,  5.47s/it]
 31%|███▏      | 70785/225000 [142:55:01<225:50:29,  5.27s/it]
 31%|███▏      | 70786/225000 [142:55:05<208:02:52,  4.86s/it]
 31%|███▏      | 70787/225000 [142:55:09<205:24:18,  4.80s/it]
 31%|███▏      | 70788/225000 [142:55:14<198:23:04,  4.63s/it]
 31%|███▏      | 70789/225000 [142:55:18<201:35:07,  4.71s/it]
 31%|███▏      | 70790/225000 [142:55:23<193:10:50,  4.51s/it]
 31%|███▏      | 70791/225000 [142:55:27<189:12:40,  4.42s/it]
 31%|███▏      | 70792/225000 [142:55:33<209:38:59,  4.89s/it]
 31%|███▏      | 70793/225000 [142:55:41<250:38:37,  5.85s/it]
 31%|███▏      | 70794/225000 [142:55:44<216:52:28,  5.06s/it]
 31%|███▏      | 70795/225000 [142:55:52<250:10:27,  5.84s/it]
 31%|███▏      | 70796/225000 [142:55:55<217:00:38,  5.07s/it]
 31%|███▏      | 70797/225000 [142:55:58<188:53:25,  4.41s/it]
 31%|███▏      | 70798/225000 [142:56:07<245:02:55,  5.72s/it]
 31%|███▏      | 70799/225000 [142:56:10<213:56:23,  4.99s/it]
 31%|███▏      | 70800/225000 [142:56:14<197:23:44,  4.61s/it]
                                                              
{'loss': '0.776', 'grad_norm': '0.9209', 'learning_rate': '4.078e-05', 'epoch': '0.3147', 'num_input_tokens_seen': 237042528, 'train_runtime': '5.146e+05', 'train_tokens_per_second': '460.7'}

 31%|███▏      | 70800/225000 [142:56:14<197:23:44,  4.61s/it]
 31%|███▏      | 70801/225000 [142:56:22<240:33:18,  5.62s/it]
 31%|███▏      | 70802/225000 [142:56:27<240:06:31,  5.61s/it]
 31%|███▏      | 70803/225000 [142:56:31<213:37:43,  4.99s/it]
 31%|███▏      | 70804/225000 [142:56:35<203:29:33,  4.75s/it]
 31%|███▏      | 70805/225000 [142:56:39<194:52:59,  4.55s/it]
 31%|███▏      | 70806/225000 [142:56:43<191:53:57,  4.48s/it]
 31%|███▏      | 70807/225000 [142:56:47<186:33:34,  4.36s/it]
 31%|███▏      | 70808/225000 [142:56:52<187:26:51,  4.38s/it]
 31%|███▏      | 70809/225000 [142:57:03<278:41:20,  6.51s/it]
 31%|███▏      | 70810/225000 [142:57:09<267:05:56,  6.24s/it]
 31%|███▏      | 70811/225000 [142:57:13<237:02:31,  5.53s/it]
 31%|███▏      | 70812/225000 [142:57:17<222:42:14,  5.20s/it]
 31%|███▏      | 70813/225000 [142:57:21<208:56:04,  4.88s/it]
 31%|███▏      | 70814/225000 [142:57:26<200:44:07,  4.69s/it]
 31%|███▏      | 70815/225000 [142:57:29<188:53:33,  4.41s/it]
 31%|███▏      | 70816/225000 [142:57:33<180:12:02,  4.21s/it]
 31%|███▏      | 70817/225000 [142:57:42<240:57:34,  5.63s/it]
 31%|███▏      | 70818/225000 [142:57:47<236:52:57,  5.53s/it]
 31%|███▏      | 70819/225000 [142:57:51<214:53:07,  5.02s/it]
 31%|███▏      | 70820/225000 [142:57:55<199:38:48,  4.66s/it]
 31%|███▏      | 70821/225000 [142:58:00<201:20:10,  4.70s/it]
 31%|███▏      | 70822/225000 [142:58:09<257:42:55,  6.02s/it]
 31%|███▏      | 70823/225000 [142:58:13<227:35:49,  5.31s/it]
 31%|███▏      | 70824/225000 [142:58:16<203:56:31,  4.76s/it]
 31%|███▏      | 70825/225000 [142:58:18<174:20:27,  4.07s/it]
 31%|███▏      | 70826/225000 [142:58:24<189:59:46,  4.44s/it]
 31%|███▏      | 70827/225000 [142:58:35<272:56:39,  6.37s/it]
 31%|███▏      | 70828/225000 [142:58:39<245:46:23,  5.74s/it]
 31%|███▏      | 70829/225000 [142:58:43<219:34:17,  5.13s/it]
 31%|███▏      | 70830/225000 [142:58:47<209:08:13,  4.88s/it]
 31%|███▏      | 70831/225000 [142:58:50<184:32:51,  4.31s/it]
 31%|███▏      | 70832/225000 [142:58:54<182:14:50,  4.26s/it]
 31%|███▏      | 70833/225000 [142:58:58<176:45:45,  4.13s/it]
 31%|███▏      | 70834/225000 [142:59:01<170:20:53,  3.98s/it]
 31%|███▏      | 70835/225000 [142:59:05<161:36:59,  3.77s/it]
 31%|███▏      | 70836/225000 [142:59:17<266:21:11,  6.22s/it]
 31%|███▏      | 70837/225000 [142:59:22<248:12:46,  5.80s/it]
 31%|███▏      | 70838/225000 [142:59:25<215:50:33,  5.04s/it]
 31%|███▏      | 70839/225000 [142:59:31<231:42:20,  5.41s/it]
 31%|███▏      | 70840/225000 [142:59:37<238:30:30,  5.57s/it]
 31%|███▏      | 70841/225000 [142:59:40<209:55:38,  4.90s/it]
 31%|███▏      | 70842/225000 [142:59:45<203:29:26,  4.75s/it]
 31%|███▏      | 70843/225000 [142:59:49<195:00:28,  4.55s/it]
 31%|███▏      | 70844/225000 [142:59:58<256:06:51,  5.98s/it]
 31%|███▏      | 70845/225000 [143:00:01<216:53:57,  5.07s/it]
 31%|███▏      | 70846/225000 [143:00:04<195:38:00,  4.57s/it]
 31%|███▏      | 70847/225000 [143:00:08<185:19:57,  4.33s/it]
 31%|███▏      | 70848/225000 [143:00:15<217:34:54,  5.08s/it]
 31%|███▏      | 70849/225000 [143:00:24<269:42:28,  6.30s/it]
 31%|███▏      | 70850/225000 [143:00:27<222:58:34,  5.21s/it]
                                                              
{'loss': '0.7476', 'grad_norm': '1.161', 'learning_rate': '4.076e-05', 'epoch': '0.3149', 'num_input_tokens_seen': 237150240, 'train_runtime': '5.148e+05', 'train_tokens_per_second': '460.6'}

 31%|███▏      | 70850/225000 [143:00:27<222:58:34,  5.21s/it]
 31%|███▏      | 70851/225000 [143:00:32<222:59:43,  5.21s/it]
 31%|███▏      | 70852/225000 [143:00:40<255:28:00,  5.97s/it]
 31%|███▏      | 70853/225000 [143:00:43<217:09:02,  5.07s/it]
 31%|███▏      | 70854/225000 [143:00:46<191:17:06,  4.47s/it]
 31%|███▏      | 70855/225000 [143:00:50<182:35:47,  4.26s/it]
 31%|███▏      | 70856/225000 [143:00:56<209:34:00,  4.89s/it]
 31%|███▏      | 70857/225000 [143:01:04<247:50:42,  5.79s/it]
 31%|███▏      | 70858/225000 [143:01:09<233:24:26,  5.45s/it]
 31%|███▏      | 70859/225000 [143:01:13<222:03:39,  5.19s/it]
 31%|███▏      | 70860/225000 [143:01:17<211:06:15,  4.93s/it]
 31%|███▏      | 70861/225000 [143:01:21<190:54:48,  4.46s/it]
 31%|███▏      | 70862/225000 [143:01:24<175:53:55,  4.11s/it]
 31%|███▏      | 70863/225000 [143:01:28<170:45:54,  3.99s/it]
 31%|███▏      | 70864/225000 [143:01:37<235:53:38,  5.51s/it]
 31%|███▏      | 70865/225000 [143:01:42<235:58:28,  5.51s/it]
 31%|███▏      | 70866/225000 [143:01:46<216:21:52,  5.05s/it]
 31%|███▏      | 70867/225000 [143:01:53<234:48:39,  5.48s/it]
 31%|███▏      | 70868/225000 [143:01:59<240:10:03,  5.61s/it]
 31%|███▏      | 70869/225000 [143:02:05<246:20:50,  5.75s/it]
 31%|███▏      | 70870/225000 [143:02:08<217:12:16,  5.07s/it]
 31%|███▏      | 70871/225000 [143:02:13<207:12:33,  4.84s/it]
 31%|███▏      | 70872/225000 [143:02:16<186:12:34,  4.35s/it]
 31%|███▏      | 70873/225000 [143:02:20<186:26:20,  4.35s/it]
 31%|███▏      | 70874/225000 [143:02:23<171:51:54,  4.01s/it]
 32%|███▏      | 70875/225000 [143:02:27<167:39:08,  3.92s/it]
 32%|███▏      | 70876/225000 [143:02:31<163:05:21,  3.81s/it]
 32%|███▏      | 70877/225000 [143:02:36<177:40:00,  4.15s/it]
 32%|███▏      | 70878/225000 [143:02:46<259:57:04,  6.07s/it]
 32%|███▏      | 70879/225000 [143:02:52<256:20:01,  5.99s/it]
 32%|███▏      | 70880/225000 [143:02:55<219:47:24,  5.13s/it]
 32%|███▏      | 70881/225000 [143:03:00<211:38:18,  4.94s/it]
 32%|███▏      | 70882/225000 [143:03:03<195:16:45,  4.56s/it]
 32%|███▏      | 70883/225000 [143:03:08<191:33:29,  4.47s/it]
 32%|███▏      | 70884/225000 [143:03:11<181:40:45,  4.24s/it]
 32%|███▏      | 70885/225000 [143:03:18<212:36:51,  4.97s/it]
 32%|███▏      | 70886/225000 [143:03:30<306:01:51,  7.15s/it]
 32%|███▏      | 70887/225000 [143:03:34<267:52:42,  6.26s/it]
 32%|███▏      | 70888/225000 [143:03:39<245:54:14,  5.74s/it]
 32%|███▏      | 70889/225000 [143:03:42<212:29:06,  4.96s/it]
 32%|███▏      | 70890/225000 [143:03:46<202:21:40,  4.73s/it]
 32%|███▏      | 70891/225000 [143:03:50<192:21:29,  4.49s/it]
 32%|███▏      | 70892/225000 [143:03:54<184:39:59,  4.31s/it]
 32%|███▏      | 70893/225000 [143:04:02<226:34:28,  5.29s/it]
 32%|███▏      | 70894/225000 [143:04:08<238:55:45,  5.58s/it]
 32%|███▏      | 70895/225000 [143:04:12<214:21:11,  5.01s/it]
 32%|███▏      | 70896/225000 [143:04:14<184:54:21,  4.32s/it]
 32%|███▏      | 70897/225000 [143:04:20<197:15:17,  4.61s/it]
 32%|███▏      | 70898/225000 [143:04:30<267:13:36,  6.24s/it]
 32%|███▏      | 70899/225000 [143:04:32<223:21:03,  5.22s/it]
 32%|███▏      | 70900/225000 [143:04:35<189:37:31,  4.43s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.981', 'learning_rate': '4.075e-05', 'epoch': '0.3151', 'num_input_tokens_seen': 237255016, 'train_runtime': '5.151e+05', 'train_tokens_per_second': '460.6'}

 32%|███▏      | 70900/225000 [143:04:35<189:37:31,  4.43s/it]
 32%|███▏      | 70901/225000 [143:04:42<225:13:49,  5.26s/it]
 32%|███▏      | 70902/225000 [143:04:47<216:54:57,  5.07s/it]
 32%|███▏      | 70903/225000 [143:04:51<198:47:34,  4.64s/it]
 32%|███▏      | 70904/225000 [143:04:54<189:58:38,  4.44s/it]
 32%|███▏      | 70905/225000 [143:05:05<265:54:38,  6.21s/it]
 32%|███▏      | 70906/225000 [143:05:09<236:49:54,  5.53s/it]
 32%|███▏      | 70907/225000 [143:05:16<255:21:51,  5.97s/it]
 32%|███▏      | 70908/225000 [143:05:19<223:32:39,  5.22s/it]
 32%|███▏      | 70909/225000 [143:05:22<196:09:00,  4.58s/it]
 32%|███▏      | 70910/225000 [143:05:31<246:55:37,  5.77s/it]
 32%|███▏      | 70911/225000 [143:05:35<221:02:30,  5.16s/it]
 32%|███▏      | 70912/225000 [143:05:38<192:05:39,  4.49s/it]
 32%|███▏      | 70913/225000 [143:05:41<181:09:20,  4.23s/it]
 32%|███▏      | 70914/225000 [143:05:44<165:22:37,  3.86s/it]
 32%|███▏      | 70915/225000 [143:05:48<166:39:37,  3.89s/it]
 32%|███▏      | 70916/225000 [143:05:56<217:08:52,  5.07s/it]
 32%|███▏      | 70917/225000 [143:06:04<258:44:36,  6.05s/it]
 32%|███▏      | 70918/225000 [143:06:08<224:39:42,  5.25s/it]
 32%|███▏      | 70919/225000 [143:06:11<195:59:48,  4.58s/it]
 32%|███▏      | 70920/225000 [143:06:14<182:12:56,  4.26s/it]
 32%|███▏      | 70921/225000 [143:06:18<173:00:08,  4.04s/it]
 32%|███▏      | 70922/225000 [143:06:22<170:36:18,  3.99s/it]
 32%|███▏      | 70923/225000 [143:06:25<167:00:52,  3.90s/it]
 32%|███▏      | 70924/225000 [143:06:29<165:00:27,  3.86s/it]
 32%|███▏      | 70925/225000 [143:06:33<165:07:20,  3.86s/it]
 32%|███▏      | 70926/225000 [143:06:41<216:55:25,  5.07s/it]
 32%|███▏      | 70927/225000 [143:06:51<281:37:03,  6.58s/it]
 32%|███▏      | 70928/225000 [143:06:55<248:21:31,  5.80s/it]
 32%|███▏      | 70929/225000 [143:06:58<217:32:56,  5.08s/it]
 32%|███▏      | 70930/225000 [143:07:02<203:30:52,  4.76s/it]
 32%|███▏      | 70931/225000 [143:07:12<269:38:06,  6.30s/it]
 32%|███▏      | 70932/225000 [143:07:17<244:22:52,  5.71s/it]
 32%|███▏      | 70933/225000 [143:07:22<239:09:43,  5.59s/it]
 32%|███▏      | 70934/225000 [143:07:25<204:57:30,  4.79s/it]
 32%|███▏      | 70935/225000 [143:07:28<184:54:17,  4.32s/it]
 32%|███▏      | 70936/225000 [143:07:38<254:35:31,  5.95s/it]
 32%|███▏      | 70937/225000 [143:07:42<238:03:03,  5.56s/it]
 32%|███▏      | 70938/225000 [143:07:46<214:19:49,  5.01s/it]
 32%|███▏      | 70939/225000 [143:07:50<200:12:54,  4.68s/it]
 32%|███▏      | 70940/225000 [143:07:54<187:56:54,  4.39s/it]
 32%|███▏      | 70941/225000 [143:08:05<270:34:04,  6.32s/it]
 32%|███▏      | 70942/225000 [143:08:08<228:26:19,  5.34s/it]
 32%|███▏      | 70943/225000 [143:08:12<211:18:25,  4.94s/it]
 32%|███▏      | 70944/225000 [143:08:19<247:24:43,  5.78s/it]
 32%|███▏      | 70945/225000 [143:08:24<237:31:15,  5.55s/it]
 32%|███▏      | 70946/225000 [143:08:28<215:23:20,  5.03s/it]
 32%|███▏      | 70947/225000 [143:08:35<241:56:43,  5.65s/it]
 32%|███▏      | 70948/225000 [143:08:38<206:19:23,  4.82s/it]
 32%|███▏      | 70949/225000 [143:08:47<254:11:26,  5.94s/it]
 32%|███▏      | 70950/225000 [143:08:50<214:26:26,  5.01s/it]
                                                              
{'loss': '0.778', 'grad_norm': '0.9364', 'learning_rate': '4.073e-05', 'epoch': '0.3153', 'num_input_tokens_seen': 237363072, 'train_runtime': '5.153e+05', 'train_tokens_per_second': '460.6'}

 32%|███▏      | 70950/225000 [143:08:50<214:26:26,  5.01s/it]
 32%|███▏      | 70951/225000 [143:08:53<198:28:46,  4.64s/it]
 32%|███▏      | 70952/225000 [143:08:58<200:19:27,  4.68s/it]
 32%|███▏      | 70953/225000 [143:09:02<192:16:28,  4.49s/it]
 32%|███▏      | 70954/225000 [143:09:06<185:58:07,  4.35s/it]
 32%|███▏      | 70955/225000 [143:09:11<186:13:44,  4.35s/it]
 32%|███▏      | 70956/225000 [143:09:14<173:52:30,  4.06s/it]
 32%|███▏      | 70957/225000 [143:09:18<172:36:35,  4.03s/it]
 32%|███▏      | 70958/225000 [143:09:22<173:50:28,  4.06s/it]
 32%|███▏      | 70959/225000 [143:09:26<178:00:38,  4.16s/it]
 32%|███▏      | 70960/225000 [143:09:31<179:56:35,  4.21s/it]
 32%|███▏      | 70961/225000 [143:09:35<183:12:12,  4.28s/it]
 32%|███▏      | 70962/225000 [143:09:41<208:58:18,  4.88s/it]
 32%|███▏      | 70963/225000 [143:09:51<266:53:09,  6.24s/it]
 32%|███▏      | 70964/225000 [143:09:55<240:39:39,  5.62s/it]
 32%|███▏      | 70965/225000 [143:09:59<221:46:16,  5.18s/it]
 32%|███▏      | 70966/225000 [143:10:03<206:24:39,  4.82s/it]
 32%|███▏      | 70967/225000 [143:10:08<204:38:26,  4.78s/it]
 32%|███▏      | 70968/225000 [143:10:12<198:38:24,  4.64s/it]
 32%|███▏      | 70969/225000 [143:10:16<190:22:57,  4.45s/it]
 32%|███▏      | 70970/225000 [143:10:20<187:37:07,  4.39s/it]
 32%|███▏      | 70971/225000 [143:10:24<179:34:24,  4.20s/it]
 32%|███▏      | 70972/225000 [143:10:28<175:47:29,  4.11s/it]
 32%|███▏      | 70973/225000 [143:10:33<183:51:30,  4.30s/it]
 32%|███▏      | 70974/225000 [143:10:41<231:13:38,  5.40s/it]
 32%|███▏      | 70975/225000 [143:10:47<236:24:19,  5.53s/it]
 32%|███▏      | 70976/225000 [143:10:56<287:10:05,  6.71s/it]
 32%|███▏      | 70977/225000 [143:10:59<237:25:23,  5.55s/it]
 32%|███▏      | 70978/225000 [143:11:07<272:09:19,  6.36s/it]
 32%|███▏      | 70979/225000 [143:11:10<224:51:21,  5.26s/it]
 32%|███▏      | 70980/225000 [143:11:13<194:25:37,  4.54s/it]
 32%|███▏      | 70981/225000 [143:11:16<175:31:50,  4.10s/it]
 32%|███▏      | 70982/225000 [143:11:25<244:51:43,  5.72s/it]
 32%|███▏      | 70983/225000 [143:11:30<225:55:36,  5.28s/it]
 32%|███▏      | 70984/225000 [143:11:33<196:26:55,  4.59s/it]
 32%|███▏      | 70985/225000 [143:11:36<177:46:07,  4.16s/it]
 32%|███▏      | 70986/225000 [143:11:45<245:20:02,  5.73s/it]
 32%|███▏      | 70987/225000 [143:11:48<214:03:04,  5.00s/it]
 32%|███▏      | 70988/225000 [143:11:51<186:16:01,  4.35s/it]
 32%|███▏      | 70989/225000 [143:11:54<168:34:55,  3.94s/it]
 32%|███▏      | 70990/225000 [143:11:59<181:15:40,  4.24s/it]
 32%|███▏      | 70991/225000 [143:12:05<201:50:54,  4.72s/it]
 32%|███▏      | 70992/225000 [143:12:09<195:13:57,  4.56s/it]
 32%|███▏      | 70993/225000 [143:12:16<227:34:02,  5.32s/it]
 32%|███▏      | 70994/225000 [143:12:22<231:33:06,  5.41s/it]
 32%|███▏      | 70995/225000 [143:12:24<192:24:29,  4.50s/it]
 32%|███▏      | 70996/225000 [143:12:32<229:20:41,  5.36s/it]
 32%|███▏      | 70997/225000 [143:12:36<212:35:59,  4.97s/it]
 32%|███▏      | 70998/225000 [143:12:40<197:29:13,  4.62s/it]
 32%|███▏      | 70999/225000 [143:12:44<192:21:00,  4.50s/it]
 32%|███▏      | 71000/225000 [143:12:48<194:25:23,  4.54s/it]
                                                              
{'loss': '0.8008', 'grad_norm': '1.304', 'learning_rate': '4.072e-05', 'epoch': '0.3156', 'num_input_tokens_seen': 237466544, 'train_runtime': '5.156e+05', 'train_tokens_per_second': '460.6'}

 32%|███▏      | 71000/225000 [143:12:48<194:25:23,  4.54s/it]
 32%|███▏      | 71001/225000 [143:12:57<252:41:10,  5.91s/it]
 32%|███▏      | 71002/225000 [143:13:01<227:14:33,  5.31s/it]
 32%|███▏      | 71003/225000 [143:13:05<206:35:11,  4.83s/it]
 32%|███▏      | 71004/225000 [143:13:09<193:34:58,  4.53s/it]
 32%|███▏      | 71005/225000 [143:13:19<265:01:50,  6.20s/it]
 32%|███▏      | 71006/225000 [143:13:23<232:40:29,  5.44s/it]
 32%|███▏      | 71007/225000 [143:13:27<213:30:46,  4.99s/it]
 32%|███▏      | 71008/225000 [143:13:30<198:15:02,  4.63s/it]
 32%|███▏      | 71009/225000 [143:13:35<191:24:18,  4.47s/it]
 32%|███▏      | 71010/225000 [143:13:46<279:09:41,  6.53s/it]
 32%|███▏      | 71011/225000 [143:13:49<233:03:39,  5.45s/it]
 32%|███▏      | 71012/225000 [143:13:53<212:56:40,  4.98s/it]
 32%|███▏      | 71013/225000 [143:14:00<242:43:48,  5.67s/it]
 32%|███▏      | 71014/225000 [143:14:03<206:22:23,  4.82s/it]
 32%|███▏      | 71015/225000 [143:14:06<182:56:55,  4.28s/it]
 32%|███▏      | 71016/225000 [143:14:09<164:21:25,  3.84s/it]
 32%|███▏      | 71017/225000 [143:14:16<213:11:35,  4.98s/it]
 32%|███▏      | 71018/225000 [143:14:21<206:02:05,  4.82s/it]
 32%|███▏      | 71019/225000 [143:14:23<179:17:40,  4.19s/it]
 32%|███▏      | 71020/225000 [143:14:28<178:54:19,  4.18s/it]
 32%|███▏      | 71021/225000 [143:14:36<234:42:20,  5.49s/it]
 32%|███▏      | 71022/225000 [143:14:39<206:39:10,  4.83s/it]
 32%|███▏      | 71023/225000 [143:14:42<176:53:46,  4.14s/it]
 32%|███▏      | 71024/225000 [143:14:45<160:17:50,  3.75s/it]
 32%|███▏      | 71025/225000 [143:14:48<151:34:18,  3.54s/it]
 32%|███▏      | 71026/225000 [143:14:56<215:43:22,  5.04s/it]
 32%|███▏      | 71027/225000 [143:15:01<216:25:35,  5.06s/it]
 32%|███▏      | 71028/225000 [143:15:09<247:49:13,  5.79s/it]
 32%|███▏      | 71029/225000 [143:15:13<226:23:55,  5.29s/it]
 32%|███▏      | 71030/225000 [143:15:16<192:10:43,  4.49s/it]
 32%|███▏      | 71031/225000 [143:15:24<240:45:01,  5.63s/it]
 32%|███▏      | 71032/225000 [143:15:27<210:00:55,  4.91s/it]
 32%|███▏      | 71033/225000 [143:15:32<209:15:18,  4.89s/it]
 32%|███▏      | 71034/225000 [143:15:40<243:54:35,  5.70s/it]
 32%|███▏      | 71035/225000 [143:15:43<215:17:10,  5.03s/it]
 32%|███▏      | 71036/225000 [143:15:47<193:23:02,  4.52s/it]
 32%|███▏      | 71037/225000 [143:15:51<191:33:32,  4.48s/it]
 32%|███▏      | 71038/225000 [143:15:55<184:48:03,  4.32s/it]
 32%|███▏      | 71039/225000 [143:15:59<180:12:57,  4.21s/it]
 32%|███▏      | 71040/225000 [143:16:03<182:06:48,  4.26s/it]
 32%|███▏      | 71041/225000 [143:16:09<199:59:37,  4.68s/it]
 32%|███▏      | 71042/225000 [143:16:16<236:22:51,  5.53s/it]
 32%|███▏      | 71043/225000 [143:16:24<264:06:19,  6.18s/it]
 32%|███▏      | 71044/225000 [143:16:27<224:25:07,  5.25s/it]
 32%|███▏      | 71045/225000 [143:16:31<207:10:07,  4.84s/it]
 32%|███▏      | 71046/225000 [143:16:35<194:40:14,  4.55s/it]
 32%|███▏      | 71047/225000 [143:16:40<196:21:59,  4.59s/it]
 32%|███▏      | 71048/225000 [143:16:44<190:53:46,  4.46s/it]
 32%|███▏      | 71049/225000 [143:16:53<250:08:06,  5.85s/it]
 32%|███▏      | 71050/225000 [143:17:00<265:34:26,  6.21s/it]
                                                              
{'loss': '0.7725', 'grad_norm': '1.036', 'learning_rate': '4.071e-05', 'epoch': '0.3158', 'num_input_tokens_seen': 237573088, 'train_runtime': '5.158e+05', 'train_tokens_per_second': '460.6'}

 32%|███▏      | 71050/225000 [143:17:00<265:34:26,  6.21s/it]
 32%|███▏      | 71051/225000 [143:17:03<228:21:57,  5.34s/it]
 32%|███▏      | 71052/225000 [143:17:11<264:06:24,  6.18s/it]
 32%|███▏      | 71053/225000 [143:17:15<228:54:28,  5.35s/it]
 32%|███▏      | 71054/225000 [143:17:19<208:44:42,  4.88s/it]
 32%|███▏      | 71055/225000 [143:17:22<192:00:01,  4.49s/it]
 32%|███▏      | 71056/225000 [143:17:30<232:32:39,  5.44s/it]
 32%|███▏      | 71057/225000 [143:17:34<221:08:30,  5.17s/it]
 32%|███▏      | 71058/225000 [143:17:37<191:44:15,  4.48s/it]
 32%|███▏      | 71059/225000 [143:17:41<178:48:09,  4.18s/it]
 32%|███▏      | 71060/225000 [143:17:48<224:49:58,  5.26s/it]
 32%|███▏      | 71061/225000 [143:17:55<235:37:15,  5.51s/it]
 32%|███▏      | 71062/225000 [143:17:57<196:10:24,  4.59s/it]
 32%|███▏      | 71063/225000 [143:18:00<182:28:02,  4.27s/it]
 32%|███▏      | 71064/225000 [143:18:04<176:28:55,  4.13s/it]
 32%|███▏      | 71065/225000 [143:18:13<233:35:53,  5.46s/it]
 32%|███▏      | 71066/225000 [143:18:18<227:38:31,  5.32s/it]
 32%|███▏      | 71067/225000 [143:18:22<215:23:08,  5.04s/it]
 32%|███▏      | 71068/225000 [143:18:27<208:24:26,  4.87s/it]
 32%|███▏      | 71069/225000 [143:18:31<198:12:39,  4.64s/it]
 32%|███▏      | 71070/225000 [143:18:35<189:35:13,  4.43s/it]
 32%|███▏      | 71071/225000 [143:18:39<187:14:33,  4.38s/it]
 32%|███▏      | 71072/225000 [143:18:47<232:56:59,  5.45s/it]
 32%|███▏      | 71073/225000 [143:18:52<231:19:56,  5.41s/it]
 32%|███▏      | 71074/225000 [143:18:55<198:01:53,  4.63s/it]
 32%|███▏      | 71075/225000 [143:18:58<179:40:32,  4.20s/it]
 32%|███▏      | 71076/225000 [143:19:01<163:59:27,  3.84s/it]
 32%|███▏      | 71077/225000 [143:19:06<172:25:39,  4.03s/it]
 32%|███▏      | 71078/225000 [143:19:15<238:08:05,  5.57s/it]
 32%|███▏      | 71079/225000 [143:19:21<249:18:42,  5.83s/it]
 32%|███▏      | 71080/225000 [143:19:24<210:27:37,  4.92s/it]
 32%|███▏      | 71081/225000 [143:19:31<229:26:11,  5.37s/it]
 32%|███▏      | 71082/225000 [143:19:34<199:05:11,  4.66s/it]
 32%|███▏      | 71083/225000 [143:19:36<173:02:59,  4.05s/it]
 32%|███▏      | 71084/225000 [143:19:43<208:47:06,  4.88s/it]
 32%|███▏      | 71085/225000 [143:19:49<222:55:55,  5.21s/it]
 32%|███▏      | 71086/225000 [143:19:52<193:15:31,  4.52s/it]
 32%|███▏      | 71087/225000 [143:19:54<167:55:46,  3.93s/it]
 32%|███▏      | 71088/225000 [143:19:58<163:32:36,  3.83s/it]
 32%|███▏      | 71089/225000 [143:20:02<168:08:07,  3.93s/it]
 32%|███▏      | 71090/225000 [143:20:10<213:01:55,  4.98s/it]
 32%|███▏      | 71091/225000 [143:20:20<280:33:07,  6.56s/it]
 32%|███▏      | 71092/225000 [143:20:24<245:47:16,  5.75s/it]
 32%|███▏      | 71093/225000 [143:20:29<240:36:53,  5.63s/it]
 32%|███▏      | 71094/225000 [143:20:32<211:00:33,  4.94s/it]
 32%|███▏      | 71095/225000 [143:20:37<204:57:48,  4.79s/it]
 32%|███▏      | 71096/225000 [143:20:41<200:35:31,  4.69s/it]
 32%|███▏      | 71097/225000 [143:20:45<189:28:22,  4.43s/it]
 32%|███▏      | 71098/225000 [143:20:49<175:46:09,  4.11s/it]
 32%|███▏      | 71099/225000 [143:20:58<248:57:44,  5.82s/it]
 32%|███▏      | 71100/225000 [143:21:04<242:29:35,  5.67s/it]
                                                              
{'loss': '0.809', 'grad_norm': '1.696', 'learning_rate': '4.069e-05', 'epoch': '0.316', 'num_input_tokens_seen': 237675832, 'train_runtime': '5.161e+05', 'train_tokens_per_second': '460.6'}

 32%|███▏      | 71100/225000 [143:21:04<242:29:35,  5.67s/it]
 32%|███▏      | 71101/225000 [143:21:07<217:15:05,  5.08s/it]
 32%|███▏      | 71102/225000 [143:21:11<196:17:43,  4.59s/it]
 32%|███▏      | 71103/225000 [143:21:14<180:03:47,  4.21s/it]
 32%|███▏      | 71104/225000 [143:21:17<167:53:06,  3.93s/it]
 32%|███▏      | 71105/225000 [143:21:22<173:05:57,  4.05s/it]
 32%|███▏      | 71106/225000 [143:21:32<259:07:25,  6.06s/it]
 32%|███▏      | 71107/225000 [143:21:37<240:39:27,  5.63s/it]
 32%|███▏      | 71108/225000 [143:21:40<203:16:19,  4.76s/it]
 32%|███▏      | 71109/225000 [143:21:47<232:57:46,  5.45s/it]
 32%|███▏      | 71110/225000 [143:21:51<216:31:11,  5.07s/it]
 32%|███▏      | 71111/225000 [143:21:56<208:53:09,  4.89s/it]
 32%|███▏      | 71112/225000 [143:21:58<183:13:09,  4.29s/it]
 32%|███▏      | 71113/225000 [143:22:02<168:13:35,  3.94s/it]
 32%|███▏      | 71114/225000 [143:22:10<226:09:00,  5.29s/it]
 32%|███▏      | 71115/225000 [143:22:14<205:32:09,  4.81s/it]
 32%|███▏      | 71116/225000 [143:22:17<183:53:47,  4.30s/it]
 32%|███▏      | 71117/225000 [143:22:26<248:12:57,  5.81s/it]
 32%|███▏      | 71118/225000 [143:22:29<207:33:33,  4.86s/it]
 32%|███▏      | 71119/225000 [143:22:32<183:22:12,  4.29s/it]
 32%|███▏      | 71120/225000 [143:22:35<164:41:56,  3.85s/it]
 32%|███▏      | 71121/225000 [143:22:38<158:44:33,  3.71s/it]
 32%|███▏      | 71122/225000 [143:22:43<169:48:20,  3.97s/it]
 32%|███▏      | 71123/225000 [143:22:47<170:16:05,  3.98s/it]
 32%|███▏      | 71124/225000 [143:22:51<172:04:51,  4.03s/it]
 32%|███▏      | 71125/225000 [143:22:54<164:45:48,  3.85s/it]
 32%|███▏      | 71126/225000 [143:22:58<162:08:31,  3.79s/it]
 32%|███▏      | 71127/225000 [143:23:03<185:14:58,  4.33s/it]
 32%|███▏      | 71128/225000 [143:23:15<278:23:31,  6.51s/it]
 32%|███▏      | 71129/225000 [143:23:19<244:21:10,  5.72s/it]
 32%|███▏      | 71130/225000 [143:23:22<208:29:08,  4.88s/it]
 32%|███▏      | 71131/225000 [143:23:25<183:32:24,  4.29s/it]
 32%|███▏      | 71132/225000 [143:23:33<237:54:54,  5.57s/it]
 32%|███▏      | 71133/225000 [143:23:41<263:46:07,  6.17s/it]
 32%|███▏      | 71134/225000 [143:23:45<234:03:02,  5.48s/it]
 32%|███▏      | 71135/225000 [143:23:48<201:42:35,  4.72s/it]
 32%|███▏      | 71136/225000 [143:23:50<176:19:59,  4.13s/it]
 32%|███▏      | 71137/225000 [143:23:54<166:02:19,  3.88s/it]
 32%|███▏      | 71138/225000 [143:24:07<282:47:49,  6.62s/it]
 32%|███▏      | 71139/225000 [143:24:10<236:17:26,  5.53s/it]
 32%|███▏      | 71140/225000 [143:24:13<208:45:59,  4.88s/it]
 32%|███▏      | 71141/225000 [143:24:16<181:06:54,  4.24s/it]
 32%|███▏      | 71142/225000 [143:24:20<175:33:25,  4.11s/it]
 32%|███▏      | 71143/225000 [143:24:26<208:33:52,  4.88s/it]
 32%|███▏      | 71144/225000 [143:24:34<244:20:22,  5.72s/it]
 32%|███▏      | 71145/225000 [143:24:37<214:14:11,  5.01s/it]
 32%|███▏      | 71146/225000 [143:24:40<186:12:00,  4.36s/it]
 32%|███▏      | 71147/225000 [143:24:44<175:17:05,  4.10s/it]
 32%|███▏      | 71148/225000 [143:24:48<179:25:25,  4.20s/it]
 32%|███▏      | 71149/225000 [143:24:51<163:49:20,  3.83s/it]
 32%|███▏      | 71150/225000 [143:24:55<170:38:41,  3.99s/it]
                                                              
{'loss': '0.768', 'grad_norm': '1.2', 'learning_rate': '4.068e-05', 'epoch': '0.3162', 'num_input_tokens_seen': 237778392, 'train_runtime': '5.163e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71150/225000 [143:24:55<170:38:41,  3.99s/it]
 32%|███▏      | 71151/225000 [143:24:59<168:04:23,  3.93s/it]
 32%|███▏      | 71152/225000 [143:25:10<254:00:57,  5.94s/it]
 32%|███▏      | 71153/225000 [143:25:17<268:12:13,  6.28s/it]
 32%|███▏      | 71154/225000 [143:25:20<229:57:37,  5.38s/it]
 32%|███▏      | 71155/225000 [143:25:24<208:56:37,  4.89s/it]
 32%|███▏      | 71156/225000 [143:25:28<197:12:09,  4.61s/it]
 32%|███▏      | 71157/225000 [143:25:32<194:17:20,  4.55s/it]
 32%|███▏      | 71158/225000 [143:25:37<193:44:21,  4.53s/it]
 32%|███▏      | 71159/225000 [143:25:42<199:03:04,  4.66s/it]
 32%|███▏      | 71160/225000 [143:25:45<185:52:42,  4.35s/it]
 32%|███▏      | 71161/225000 [143:25:49<177:21:28,  4.15s/it]
 32%|███▏      | 71162/225000 [143:25:55<206:01:59,  4.82s/it]
 32%|███▏      | 71163/225000 [143:26:07<290:56:36,  6.81s/it]
 32%|███▏      | 71164/225000 [143:26:10<244:22:54,  5.72s/it]
 32%|███▏      | 71165/225000 [143:26:14<227:08:48,  5.32s/it]
 32%|███▏      | 71166/225000 [143:26:21<247:32:41,  5.79s/it]
 32%|███▏      | 71167/225000 [143:26:29<267:42:24,  6.26s/it]
 32%|███▏      | 71168/225000 [143:26:32<233:59:23,  5.48s/it]
 32%|███▏      | 71169/225000 [143:26:36<215:37:22,  5.05s/it]
 32%|███▏      | 71170/225000 [143:26:40<197:46:04,  4.63s/it]
 32%|███▏      | 71171/225000 [143:26:44<185:34:33,  4.34s/it]
 32%|███▏      | 71172/225000 [143:26:47<174:12:24,  4.08s/it]
 32%|███▏      | 71173/225000 [143:26:51<170:28:21,  3.99s/it]
 32%|███▏      | 71174/225000 [143:26:56<185:41:40,  4.35s/it]
 32%|███▏      | 71175/225000 [143:27:07<265:26:04,  6.21s/it]
 32%|███▏      | 71176/225000 [143:27:12<249:11:57,  5.83s/it]
 32%|███▏      | 71177/225000 [143:27:14<210:51:20,  4.93s/it]
 32%|███▏      | 71178/225000 [143:27:19<201:06:15,  4.71s/it]
 32%|███▏      | 71179/225000 [143:27:26<234:21:22,  5.48s/it]
 32%|███▏      | 71180/225000 [143:27:32<246:45:39,  5.78s/it]
 32%|███▏      | 71181/225000 [143:27:37<232:28:26,  5.44s/it]
 32%|███▏      | 71182/225000 [143:27:40<201:35:52,  4.72s/it]
 32%|███▏      | 71183/225000 [143:27:47<232:53:42,  5.45s/it]
 32%|███▏      | 71184/225000 [143:27:51<216:37:17,  5.07s/it]
 32%|███▏      | 71185/225000 [143:27:55<197:28:41,  4.62s/it]
 32%|███▏      | 71186/225000 [143:27:58<182:32:46,  4.27s/it]
 32%|███▏      | 71187/225000 [143:28:02<176:09:56,  4.12s/it]
 32%|███▏      | 71188/225000 [143:28:09<216:37:35,  5.07s/it]
 32%|███▏      | 71189/225000 [143:28:13<194:49:55,  4.56s/it]
 32%|███▏      | 71190/225000 [143:28:19<209:47:55,  4.91s/it]
 32%|███▏      | 71191/225000 [143:28:25<233:16:33,  5.46s/it]
 32%|███▏      | 71192/225000 [143:28:29<213:59:32,  5.01s/it]
 32%|███▏      | 71193/225000 [143:28:39<270:55:59,  6.34s/it]
 32%|███▏      | 71194/225000 [143:28:43<240:09:00,  5.62s/it]
 32%|███▏      | 71195/225000 [143:28:47<221:30:31,  5.18s/it]
 32%|███▏      | 71196/225000 [143:28:50<201:03:44,  4.71s/it]
 32%|███▏      | 71197/225000 [143:28:58<237:47:41,  5.57s/it]
 32%|███▏      | 71198/225000 [143:29:04<247:27:52,  5.79s/it]
 32%|███▏      | 71199/225000 [143:29:09<233:03:07,  5.46s/it]
 32%|███▏      | 71200/225000 [143:29:13<218:21:46,  5.11s/it]
                                                              
{'loss': '0.7613', 'grad_norm': '1.243', 'learning_rate': '4.066e-05', 'epoch': '0.3164', 'num_input_tokens_seen': 237883688, 'train_runtime': '5.166e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71200/225000 [143:29:13<218:21:46,  5.11s/it]
 32%|███▏      | 71201/225000 [143:29:16<192:35:22,  4.51s/it]
 32%|███▏      | 71202/225000 [143:29:24<225:53:09,  5.29s/it]
 32%|███▏      | 71203/225000 [143:29:26<195:32:38,  4.58s/it]
 32%|███▏      | 71204/225000 [143:29:29<169:57:58,  3.98s/it]
 32%|███▏      | 71205/225000 [143:29:38<231:51:39,  5.43s/it]
 32%|███▏      | 71206/225000 [143:29:43<226:21:45,  5.30s/it]
 32%|███▏      | 71207/225000 [143:29:46<204:57:03,  4.80s/it]
 32%|███▏      | 71208/225000 [143:29:49<182:12:53,  4.27s/it]
 32%|███▏      | 71209/225000 [143:29:53<173:35:32,  4.06s/it]
 32%|███▏      | 71210/225000 [143:29:57<169:19:42,  3.96s/it]
 32%|███▏      | 71211/225000 [143:30:01<168:28:31,  3.94s/it]
 32%|███▏      | 71212/225000 [143:30:11<247:24:46,  5.79s/it]
 32%|███▏      | 71213/225000 [143:30:17<250:37:38,  5.87s/it]
 32%|███▏      | 71214/225000 [143:30:20<220:46:54,  5.17s/it]
 32%|███▏      | 71215/225000 [143:30:23<194:25:10,  4.55s/it]
 32%|███▏      | 71216/225000 [143:30:28<197:31:18,  4.62s/it]
 32%|███▏      | 71217/225000 [143:30:37<253:53:41,  5.94s/it]
 32%|███▏      | 71218/225000 [143:30:42<238:41:18,  5.59s/it]
 32%|███▏      | 71219/225000 [143:30:45<209:40:13,  4.91s/it]
 32%|███▏      | 71220/225000 [143:30:49<195:33:39,  4.58s/it]
 32%|███▏      | 71221/225000 [143:30:52<172:11:28,  4.03s/it]
 32%|███▏      | 71222/225000 [143:30:57<186:50:43,  4.37s/it]
 32%|███▏      | 71223/225000 [143:31:06<248:15:17,  5.81s/it]
 32%|███▏      | 71224/225000 [143:31:12<241:24:17,  5.65s/it]
 32%|███▏      | 71225/225000 [143:31:15<211:05:06,  4.94s/it]
 32%|███▏      | 71226/225000 [143:31:23<246:02:17,  5.76s/it]
 32%|███▏      | 71227/225000 [143:31:27<227:54:46,  5.34s/it]
 32%|███▏      | 71228/225000 [143:31:31<208:38:40,  4.88s/it]
 32%|███▏      | 71229/225000 [143:31:34<190:49:42,  4.47s/it]
 32%|███▏      | 71230/225000 [143:31:38<178:15:48,  4.17s/it]
 32%|███▏      | 71231/225000 [143:31:42<178:40:33,  4.18s/it]
 32%|███▏      | 71232/225000 [143:31:49<216:46:20,  5.08s/it]
 32%|███▏      | 71233/225000 [143:31:56<247:10:35,  5.79s/it]
 32%|███▏      | 71234/225000 [143:32:00<217:31:14,  5.09s/it]
 32%|███▏      | 71235/225000 [143:32:03<189:02:23,  4.43s/it]
 32%|███▏      | 71236/225000 [143:32:11<236:01:49,  5.53s/it]
 32%|███▏      | 71237/225000 [143:32:15<213:38:33,  5.00s/it]
 32%|███▏      | 71238/225000 [143:32:17<185:14:02,  4.34s/it]
 32%|███▏      | 71239/225000 [143:32:22<183:34:41,  4.30s/it]
 32%|███▏      | 71240/225000 [143:32:31<244:17:27,  5.72s/it]
 32%|███▏      | 71241/225000 [143:32:36<242:25:35,  5.68s/it]
 32%|███▏      | 71242/225000 [143:32:40<216:26:55,  5.07s/it]
 32%|███▏      | 71243/225000 [143:32:43<195:18:55,  4.57s/it]
 32%|███▏      | 71244/225000 [143:32:47<184:15:47,  4.31s/it]
 32%|███▏      | 71245/225000 [143:32:51<173:11:21,  4.06s/it]
 32%|███▏      | 71246/225000 [143:32:55<174:28:14,  4.09s/it]
 32%|███▏      | 71247/225000 [143:32:59<176:21:28,  4.13s/it]
 32%|███▏      | 71248/225000 [143:33:04<194:28:58,  4.55s/it]
 32%|███▏      | 71249/225000 [143:33:11<214:56:00,  5.03s/it]
 32%|███▏      | 71250/225000 [143:33:14<196:28:38,  4.60s/it]
                                                              
{'loss': '0.7657', 'grad_norm': '1.59', 'learning_rate': '4.065e-05', 'epoch': '0.3167', 'num_input_tokens_seen': 237987432, 'train_runtime': '5.168e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71250/225000 [143:33:14<196:28:38,  4.60s/it]
 32%|███▏      | 71251/225000 [143:33:20<216:55:59,  5.08s/it]
 32%|███▏      | 71252/225000 [143:33:26<227:02:59,  5.32s/it]
 32%|███▏      | 71253/225000 [143:33:30<212:53:34,  4.98s/it]
 32%|███▏      | 71254/225000 [143:33:34<196:12:39,  4.59s/it]
 32%|███▏      | 71255/225000 [143:33:38<182:54:59,  4.28s/it]
 32%|███▏      | 71256/225000 [143:33:47<241:33:00,  5.66s/it]
 32%|███▏      | 71257/225000 [143:33:51<220:26:03,  5.16s/it]
 32%|███▏      | 71258/225000 [143:33:54<197:47:12,  4.63s/it]
 32%|███▏      | 71259/225000 [143:33:57<174:34:56,  4.09s/it]
 32%|███▏      | 71260/225000 [143:34:05<223:03:43,  5.22s/it]
 32%|███▏      | 71261/225000 [143:34:10<230:01:23,  5.39s/it]
 32%|███▏      | 71262/225000 [143:34:15<221:38:42,  5.19s/it]
 32%|███▏      | 71263/225000 [143:34:19<202:11:25,  4.73s/it]
 32%|███▏      | 71264/225000 [143:34:23<200:00:46,  4.68s/it]
 32%|███▏      | 71265/225000 [143:34:27<189:13:15,  4.43s/it]
 32%|███▏      | 71266/225000 [143:34:30<173:41:35,  4.07s/it]
 32%|███▏      | 71267/225000 [143:34:42<264:01:52,  6.18s/it]
 32%|███▏      | 71268/225000 [143:34:50<288:23:54,  6.75s/it]
 32%|███▏      | 71269/225000 [143:34:54<253:28:19,  5.94s/it]
 32%|███▏      | 71270/225000 [143:34:58<233:34:24,  5.47s/it]
 32%|███▏      | 71271/225000 [143:35:04<241:18:26,  5.65s/it]
 32%|███▏      | 71272/225000 [143:35:12<265:47:09,  6.22s/it]
 32%|███▏      | 71273/225000 [143:35:15<226:25:59,  5.30s/it]
 32%|███▏      | 71274/225000 [143:35:22<244:57:59,  5.74s/it]
 32%|███▏      | 71275/225000 [143:35:26<232:27:33,  5.44s/it]
 32%|███▏      | 71276/225000 [143:35:30<212:48:17,  4.98s/it]
 32%|███▏      | 71277/225000 [143:35:34<197:02:23,  4.61s/it]
 32%|███▏      | 71278/225000 [143:35:38<189:55:08,  4.45s/it]
 32%|███▏      | 71279/225000 [143:35:42<176:30:09,  4.13s/it]
 32%|███▏      | 71280/225000 [143:35:45<171:46:11,  4.02s/it]
 32%|███▏      | 71281/225000 [143:35:49<165:24:44,  3.87s/it]
 32%|███▏      | 71282/225000 [143:35:53<172:47:27,  4.05s/it]
 32%|███▏      | 71283/225000 [143:35:57<165:12:22,  3.87s/it]
 32%|███▏      | 71284/225000 [143:36:01<171:29:16,  4.02s/it]
 32%|███▏      | 71285/225000 [143:36:07<196:25:33,  4.60s/it]
 32%|███▏      | 71286/225000 [143:36:18<278:18:58,  6.52s/it]
 32%|███▏      | 71287/225000 [143:36:22<245:05:04,  5.74s/it]
 32%|███▏      | 71288/225000 [143:36:26<217:25:47,  5.09s/it]
 32%|███▏      | 71289/225000 [143:36:31<224:11:30,  5.25s/it]
 32%|███▏      | 71290/225000 [143:36:38<247:11:04,  5.79s/it]
 32%|███▏      | 71291/225000 [143:36:42<222:41:47,  5.22s/it]
 32%|███▏      | 71292/225000 [143:36:46<206:20:24,  4.83s/it]
 32%|███▏      | 71293/225000 [143:36:49<187:05:09,  4.38s/it]
 32%|███▏      | 71294/225000 [143:36:53<177:21:14,  4.15s/it]
 32%|███▏      | 71295/225000 [143:36:57<176:24:20,  4.13s/it]
 32%|███▏      | 71296/225000 [143:37:01<169:17:14,  3.96s/it]
 32%|███▏      | 71297/225000 [143:37:07<198:49:51,  4.66s/it]
 32%|███▏      | 71298/225000 [143:37:15<245:05:52,  5.74s/it]
 32%|███▏      | 71299/225000 [143:37:18<209:42:17,  4.91s/it]
 32%|███▏      | 71300/225000 [143:37:27<256:09:13,  6.00s/it]
                                                              
{'loss': '0.7941', 'grad_norm': '1.237', 'learning_rate': '4.063e-05', 'epoch': '0.3169', 'num_input_tokens_seen': 238092872, 'train_runtime': '5.17e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71300/225000 [143:37:27<256:09:13,  6.00s/it]
 32%|███▏      | 71301/225000 [143:37:30<218:04:59,  5.11s/it]
 32%|███▏      | 71302/225000 [143:37:32<187:53:48,  4.40s/it]
 32%|███▏      | 71303/225000 [143:37:40<221:52:52,  5.20s/it]
 32%|███▏      | 71304/225000 [143:37:43<202:33:07,  4.74s/it]
 32%|███▏      | 71305/225000 [143:37:48<200:38:10,  4.70s/it]
 32%|███▏      | 71306/225000 [143:37:52<195:54:46,  4.59s/it]
 32%|███▏      | 71307/225000 [143:37:56<188:11:33,  4.41s/it]
 32%|███▏      | 71308/225000 [143:38:00<185:57:03,  4.36s/it]
 32%|███▏      | 71309/225000 [143:38:05<186:05:02,  4.36s/it]
 32%|███▏      | 71310/225000 [143:38:11<214:02:35,  5.01s/it]
 32%|███▏      | 71311/225000 [143:38:22<285:34:40,  6.69s/it]
 32%|███▏      | 71312/225000 [143:38:25<246:04:50,  5.76s/it]
 32%|███▏      | 71313/225000 [143:38:32<250:35:44,  5.87s/it]
 32%|███▏      | 71314/225000 [143:38:35<213:10:47,  4.99s/it]
 32%|███▏      | 71315/225000 [143:38:41<232:38:37,  5.45s/it]
 32%|███▏      | 71316/225000 [143:38:45<219:53:16,  5.15s/it]
 32%|███▏      | 71317/225000 [143:38:49<201:51:18,  4.73s/it]
 32%|███▏      | 71318/225000 [143:38:52<175:01:30,  4.10s/it]
 32%|███▏      | 71319/225000 [143:38:56<170:58:58,  4.01s/it]
 32%|███▏      | 71320/225000 [143:38:59<168:50:42,  3.96s/it]
 32%|███▏      | 71321/225000 [143:39:11<265:43:13,  6.22s/it]
 32%|███▏      | 71322/225000 [143:39:14<222:17:25,  5.21s/it]
 32%|███▏      | 71323/225000 [143:39:21<246:35:46,  5.78s/it]
 32%|███▏      | 71324/225000 [143:39:24<209:35:22,  4.91s/it]
 32%|███▏      | 71325/225000 [143:39:27<185:54:00,  4.35s/it]
 32%|███▏      | 71326/225000 [143:39:30<163:54:30,  3.84s/it]
 32%|███▏      | 71327/225000 [143:39:37<213:51:01,  5.01s/it]
 32%|███▏      | 71328/225000 [143:39:41<203:25:34,  4.77s/it]
 32%|███▏      | 71329/225000 [143:39:45<190:29:22,  4.46s/it]
 32%|███▏      | 71330/225000 [143:39:53<231:29:34,  5.42s/it]
 32%|███▏      | 71331/225000 [143:39:56<202:02:38,  4.73s/it]
 32%|███▏      | 71332/225000 [143:40:02<214:33:32,  5.03s/it]
 32%|███▏      | 71333/225000 [143:40:05<196:06:24,  4.59s/it]
 32%|███▏      | 71334/225000 [143:40:09<189:47:40,  4.45s/it]
 32%|███▏      | 71335/225000 [143:40:13<177:53:06,  4.17s/it]
 32%|███▏      | 71336/225000 [143:40:16<162:21:31,  3.80s/it]
 32%|███▏      | 71337/225000 [143:40:24<223:20:16,  5.23s/it]
 32%|███▏      | 71338/225000 [143:40:31<241:47:15,  5.66s/it]
 32%|███▏      | 71339/225000 [143:40:35<217:16:23,  5.09s/it]
 32%|███▏      | 71340/225000 [143:40:44<269:15:23,  6.31s/it]
 32%|███▏      | 71341/225000 [143:40:48<235:46:35,  5.52s/it]
 32%|███▏      | 71342/225000 [143:40:53<231:27:55,  5.42s/it]
 32%|███▏      | 71343/225000 [143:40:56<195:36:51,  4.58s/it]
 32%|███▏      | 71344/225000 [143:41:01<204:58:17,  4.80s/it]
 32%|███▏      | 71345/225000 [143:41:04<179:27:36,  4.20s/it]
 32%|███▏      | 71346/225000 [143:41:07<164:23:22,  3.85s/it]
 32%|███▏      | 71347/225000 [143:41:11<169:54:31,  3.98s/it]
 32%|███▏      | 71348/225000 [143:41:15<166:16:50,  3.90s/it]
 32%|███▏      | 71349/225000 [143:41:24<232:47:08,  5.45s/it]
 32%|███▏      | 71350/225000 [143:41:27<209:01:03,  4.90s/it]
                                                              
{'loss': '0.7847', 'grad_norm': '1.598', 'learning_rate': '4.062e-05', 'epoch': '0.3171', 'num_input_tokens_seen': 238194888, 'train_runtime': '5.173e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71350/225000 [143:41:27<209:01:03,  4.90s/it]
 32%|███▏      | 71351/225000 [143:41:32<203:42:58,  4.77s/it]
 32%|███▏      | 71352/225000 [143:41:35<188:34:28,  4.42s/it]
 32%|███▏      | 71353/225000 [143:41:39<174:00:00,  4.08s/it]
 32%|███▏      | 71354/225000 [143:41:43<175:30:06,  4.11s/it]
 32%|███▏      | 71355/225000 [143:41:46<168:43:30,  3.95s/it]
 32%|███▏      | 71356/225000 [143:41:50<169:10:49,  3.96s/it]
 32%|███▏      | 71357/225000 [143:42:00<235:05:55,  5.51s/it]
 32%|███▏      | 71358/225000 [143:42:05<231:58:23,  5.44s/it]
 32%|███▏      | 71359/225000 [143:42:08<197:52:14,  4.64s/it]
 32%|███▏      | 71360/225000 [143:42:11<185:32:10,  4.35s/it]
 32%|███▏      | 71361/225000 [143:42:15<171:33:46,  4.02s/it]
 32%|███▏      | 71362/225000 [143:42:18<160:02:57,  3.75s/it]
 32%|███▏      | 71363/225000 [143:42:21<158:34:58,  3.72s/it]
 32%|███▏      | 71364/225000 [143:42:26<167:27:00,  3.92s/it]
 32%|███▏      | 71365/225000 [143:42:34<223:26:58,  5.24s/it]
 32%|███▏      | 71366/225000 [143:42:44<280:45:56,  6.58s/it]
 32%|███▏      | 71367/225000 [143:42:47<238:43:35,  5.59s/it]
 32%|███▏      | 71368/225000 [143:42:51<216:29:59,  5.07s/it]
 32%|███▏      | 71369/225000 [143:42:55<201:21:37,  4.72s/it]
 32%|███▏      | 71370/225000 [143:43:00<201:45:05,  4.73s/it]
 32%|███▏      | 71371/225000 [143:43:03<190:20:35,  4.46s/it]
 32%|███▏      | 71372/225000 [143:43:07<184:59:22,  4.33s/it]
 32%|███▏      | 71373/225000 [143:43:12<185:36:05,  4.35s/it]
 32%|███▏      | 71374/225000 [143:43:16<185:14:00,  4.34s/it]
 32%|███▏      | 71375/225000 [143:43:22<207:50:59,  4.87s/it]
 32%|███▏      | 71376/225000 [143:43:27<200:51:10,  4.71s/it]
 32%|███▏      | 71377/225000 [143:43:36<256:16:50,  6.01s/it]
 32%|███▏      | 71378/225000 [143:43:42<265:13:09,  6.22s/it]
 32%|███▏      | 71379/225000 [143:43:45<221:59:46,  5.20s/it]
 32%|███▏      | 71380/225000 [143:43:50<218:08:10,  5.11s/it]
 32%|███▏      | 71381/225000 [143:43:53<192:40:37,  4.52s/it]
 32%|███▏      | 71382/225000 [143:44:00<219:57:15,  5.15s/it]
 32%|███▏      | 71383/225000 [143:44:04<203:45:22,  4.78s/it]
 32%|███▏      | 71384/225000 [143:44:08<201:55:11,  4.73s/it]
 32%|███▏      | 71385/225000 [143:44:11<180:55:48,  4.24s/it]
 32%|███▏      | 71386/225000 [143:44:15<170:43:54,  4.00s/it]
 32%|███▏      | 71387/225000 [143:44:19<174:49:27,  4.10s/it]
 32%|███▏      | 71388/225000 [143:44:22<163:32:19,  3.83s/it]
 32%|███▏      | 71389/225000 [143:44:26<164:46:38,  3.86s/it]
 32%|███▏      | 71390/225000 [143:44:30<159:59:11,  3.75s/it]
 32%|███▏      | 71391/225000 [143:44:34<161:11:35,  3.78s/it]
 32%|███▏      | 71392/225000 [143:44:38<171:47:41,  4.03s/it]
 32%|███▏      | 71393/225000 [143:44:47<227:11:22,  5.32s/it]
 32%|███▏      | 71394/225000 [143:44:56<285:17:58,  6.69s/it]
 32%|███▏      | 71395/225000 [143:45:00<250:35:40,  5.87s/it]
 32%|███▏      | 71396/225000 [143:45:05<228:45:50,  5.36s/it]
 32%|███▏      | 71397/225000 [143:45:09<211:22:04,  4.95s/it]
 32%|███▏      | 71398/225000 [143:45:12<196:05:48,  4.60s/it]
 32%|███▏      | 71399/225000 [143:45:17<192:51:23,  4.52s/it]
 32%|███▏      | 71400/225000 [143:45:20<182:59:27,  4.29s/it]
                                                              
{'loss': '0.7995', 'grad_norm': '1.361', 'learning_rate': '4.061e-05', 'epoch': '0.3173', 'num_input_tokens_seen': 238295720, 'train_runtime': '5.175e+05', 'train_tokens_per_second': '460.5'}

 32%|███▏      | 71400/225000 [143:45:20<182:59:27,  4.29s/it]
 32%|███▏      | 71401/225000 [143:45:24<169:43:49,  3.98s/it]
 32%|███▏      | 71402/225000 [143:45:34<253:41:42,  5.95s/it]
 32%|███▏      | 71403/225000 [143:45:40<246:46:24,  5.78s/it]
 32%|███▏      | 71404/225000 [143:45:42<205:01:50,  4.81s/it]
 32%|███▏      | 71405/225000 [143:45:46<188:22:16,  4.42s/it]
 32%|███▏      | 71406/225000 [143:45:50<193:22:20,  4.53s/it]
 32%|███▏      | 71407/225000 [143:46:01<272:53:48,  6.40s/it]
 32%|███▏      | 71408/225000 [143:46:04<220:48:02,  5.18s/it]
 32%|███▏      | 71409/225000 [143:46:07<196:51:47,  4.61s/it]
 32%|███▏      | 71410/225000 [143:46:12<205:29:43,  4.82s/it]
 32%|███▏      | 71411/225000 [143:46:17<205:46:49,  4.82s/it]
 32%|███▏      | 71412/225000 [143:46:20<186:48:32,  4.38s/it]
 32%|███▏      | 71413/225000 [143:46:23<165:08:07,  3.87s/it]
 32%|███▏      | 71414/225000 [143:46:26<153:08:04,  3.59s/it]
 32%|███▏      | 71415/225000 [143:46:34<205:12:25,  4.81s/it]
 32%|███▏      | 71416/225000 [143:46:39<211:35:41,  4.96s/it]
 32%|███▏      | 71417/225000 [143:46:46<234:19:38,  5.49s/it]
 32%|███▏      | 71418/225000 [143:46:50<220:51:32,  5.18s/it]
 32%|███▏      | 71419/225000 [143:46:53<189:45:37,  4.45s/it]
 32%|███▏      | 71420/225000 [143:47:01<237:25:13,  5.57s/it]
 32%|███▏      | 71421/225000 [143:47:04<202:41:02,  4.75s/it]
 32%|███▏      | 71422/225000 [143:47:07<183:31:56,  4.30s/it]
 32%|███▏      | 71423/225000 [143:47:11<177:46:13,  4.17s/it]
 32%|███▏      | 71424/225000 [143:47:15<179:01:18,  4.20s/it]
 32%|███▏      | 71425/225000 [143:47:20<180:53:21,  4.24s/it]
 32%|███▏      | 71426/225000 [143:47:24<179:46:51,  4.21s/it]
 32%|███▏      | 71427/225000 [143:47:35<276:21:27,  6.48s/it]
 32%|███▏      | 71428/225000 [143:47:42<272:05:05,  6.38s/it]
 32%|███▏      | 71429/225000 [143:47:44<225:22:31,  5.28s/it]
 32%|███▏      | 71430/225000 [143:47:52<259:15:34,  6.08s/it]
 32%|███▏      | 71431/225000 [143:47:56<229:28:50,  5.38s/it]
 32%|███▏      | 71432/225000 [143:47:59<192:49:45,  4.52s/it]
 32%|███▏      | 71433/225000 [143:48:04<198:53:38,  4.66s/it]
 32%|███▏      | 71434/225000 [143:48:08<196:51:46,  4.61s/it]
 32%|███▏      | 71435/225000 [143:48:12<181:54:56,  4.26s/it]
 32%|███▏      | 71436/225000 [143:48:16<189:57:18,  4.45s/it]
 32%|███▏      | 71437/225000 [143:48:20<181:44:53,  4.26s/it]
 32%|███▏      | 71438/225000 [143:48:25<187:34:22,  4.40s/it]
 32%|███▏      | 71439/225000 [143:48:34<244:55:33,  5.74s/it]
 32%|███▏      | 71440/225000 [143:48:39<232:00:52,  5.44s/it]
 32%|███▏      | 71441/225000 [143:48:41<200:07:02,  4.69s/it]
 32%|███▏      | 71442/225000 [143:48:45<181:54:30,  4.26s/it]
 32%|███▏      | 71443/225000 [143:48:49<175:48:35,  4.12s/it]
 32%|███▏      | 71444/225000 [143:48:52<173:32:38,  4.07s/it]
 32%|███▏      | 71445/225000 [143:48:59<201:52:18,  4.73s/it]
 32%|███▏      | 71446/225000 [143:49:03<190:29:54,  4.47s/it]
 32%|███▏      | 71447/225000 [143:49:06<180:08:03,  4.22s/it]
 32%|███▏      | 71448/225000 [143:49:10<176:16:59,  4.13s/it]
 32%|███▏      | 71449/225000 [143:49:14<174:39:18,  4.09s/it]
 32%|███▏      | 71450/225000 [143:49:18<177:08:40,  4.15s/it]
                                                              
{'loss': '0.7875', 'grad_norm': '1.118', 'learning_rate': '4.059e-05', 'epoch': '0.3176', 'num_input_tokens_seen': 238397552, 'train_runtime': '5.178e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71450/225000 [143:49:18<177:08:40,  4.15s/it]
 32%|███▏      | 71451/225000 [143:49:22<166:39:39,  3.91s/it]
 32%|███▏      | 71452/225000 [143:49:26<168:03:15,  3.94s/it]
 32%|███▏      | 71453/225000 [143:49:32<200:12:49,  4.69s/it]
 32%|███▏      | 71454/225000 [143:49:42<266:06:48,  6.24s/it]
 32%|███▏      | 71455/225000 [143:49:49<275:36:53,  6.46s/it]
 32%|███▏      | 71456/225000 [143:49:51<223:13:48,  5.23s/it]
 32%|███▏      | 71457/225000 [143:49:58<240:22:28,  5.64s/it]
 32%|███▏      | 71458/225000 [143:50:02<212:59:38,  4.99s/it]
 32%|███▏      | 71459/225000 [143:50:05<196:46:06,  4.61s/it]
 32%|███▏      | 71460/225000 [143:50:09<183:47:17,  4.31s/it]
 32%|███▏      | 71461/225000 [143:50:13<186:30:00,  4.37s/it]
 32%|███▏      | 71462/225000 [143:50:18<194:49:15,  4.57s/it]
 32%|███▏      | 71463/225000 [143:50:29<266:08:16,  6.24s/it]
 32%|███▏      | 71464/225000 [143:50:33<243:18:41,  5.70s/it]
 32%|███▏      | 71465/225000 [143:50:36<211:32:40,  4.96s/it]
 32%|███▏      | 71466/225000 [143:50:40<197:12:43,  4.62s/it]
 32%|███▏      | 71467/225000 [143:50:44<183:58:28,  4.31s/it]
 32%|███▏      | 71468/225000 [143:50:50<213:50:56,  5.01s/it]
 32%|███▏      | 71469/225000 [143:51:00<267:51:21,  6.28s/it]
 32%|███▏      | 71470/225000 [143:51:03<236:10:06,  5.54s/it]
 32%|███▏      | 71471/225000 [143:51:08<223:15:02,  5.23s/it]
 32%|███▏      | 71472/225000 [143:51:12<206:59:02,  4.85s/it]
 32%|███▏      | 71473/225000 [143:51:16<191:28:14,  4.49s/it]
 32%|███▏      | 71474/225000 [143:51:20<193:45:54,  4.54s/it]
 32%|███▏      | 71475/225000 [143:51:24<180:19:20,  4.23s/it]
 32%|███▏      | 71476/225000 [143:51:28<185:18:32,  4.35s/it]
 32%|███▏      | 71477/225000 [143:51:32<182:12:21,  4.27s/it]
 32%|███▏      | 71478/225000 [143:51:37<184:15:42,  4.32s/it]
 32%|███▏      | 71479/225000 [143:51:42<195:39:53,  4.59s/it]
 32%|███▏      | 71480/225000 [143:51:46<188:50:43,  4.43s/it]
 32%|███▏      | 71481/225000 [143:51:50<176:18:12,  4.13s/it]
 32%|███▏      | 71482/225000 [143:52:00<253:17:36,  5.94s/it]
 32%|███▏      | 71483/225000 [143:52:06<253:29:40,  5.94s/it]
 32%|███▏      | 71484/225000 [143:52:10<237:13:32,  5.56s/it]
 32%|███▏      | 71485/225000 [143:52:14<209:34:41,  4.91s/it]
 32%|███▏      | 71486/225000 [143:52:18<203:10:13,  4.76s/it]
 32%|███▏      | 71487/225000 [143:52:23<201:44:43,  4.73s/it]
 32%|███▏      | 71488/225000 [143:52:27<190:37:15,  4.47s/it]
 32%|███▏      | 71489/225000 [143:52:31<186:29:52,  4.37s/it]
 32%|███▏      | 71490/225000 [143:52:40<253:22:33,  5.94s/it]
 32%|███▏      | 71491/225000 [143:52:49<285:07:46,  6.69s/it]
 32%|███▏      | 71492/225000 [143:52:51<232:00:50,  5.44s/it]
 32%|███▏      | 71493/225000 [143:52:55<206:44:31,  4.85s/it]
 32%|███▏      | 71494/225000 [143:52:58<190:50:11,  4.48s/it]
 32%|███▏      | 71495/225000 [143:53:02<177:30:02,  4.16s/it]
 32%|███▏      | 71496/225000 [143:53:06<172:43:09,  4.05s/it]
 32%|███▏      | 71497/225000 [143:53:10<170:49:27,  4.01s/it]
 32%|███▏      | 71498/225000 [143:53:14<176:17:33,  4.13s/it]
 32%|███▏      | 71499/225000 [143:53:25<258:35:21,  6.06s/it]
 32%|███▏      | 71500/225000 [143:53:31<257:43:24,  6.04s/it]
                                                              
{'loss': '0.7581', 'grad_norm': '1.282', 'learning_rate': '4.058e-05', 'epoch': '0.3178', 'num_input_tokens_seen': 238501344, 'train_runtime': '5.18e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71500/225000 [143:53:31<257:43:24,  6.04s/it]
 32%|███▏      | 71501/225000 [143:53:35<233:19:30,  5.47s/it]
 32%|███▏      | 71502/225000 [143:53:38<206:55:42,  4.85s/it]
 32%|███▏      | 71503/225000 [143:53:41<184:22:41,  4.32s/it]
 32%|███▏      | 71504/225000 [143:53:44<168:51:15,  3.96s/it]
 32%|███▏      | 71505/225000 [143:53:54<242:33:20,  5.69s/it]
 32%|███▏      | 71506/225000 [143:53:57<210:27:49,  4.94s/it]
 32%|███▏      | 71507/225000 [143:54:00<188:32:24,  4.42s/it]
 32%|███▏      | 71508/225000 [143:54:05<191:47:02,  4.50s/it]
 32%|███▏      | 71509/225000 [143:54:08<174:40:30,  4.10s/it]
 32%|███▏      | 71510/225000 [143:54:12<164:24:22,  3.86s/it]
 32%|███▏      | 71511/225000 [143:54:15<160:12:08,  3.76s/it]
 32%|███▏      | 71512/225000 [143:54:20<169:18:03,  3.97s/it]
 32%|███▏      | 71513/225000 [143:54:26<206:46:16,  4.85s/it]
 32%|███▏      | 71514/225000 [143:54:35<252:22:16,  5.92s/it]
 32%|███▏      | 71515/225000 [143:54:38<222:54:07,  5.23s/it]
 32%|███▏      | 71516/225000 [143:54:41<190:35:24,  4.47s/it]
 32%|███▏      | 71517/225000 [143:54:44<173:14:39,  4.06s/it]
 32%|███▏      | 71518/225000 [143:54:48<171:44:16,  4.03s/it]
 32%|███▏      | 71519/225000 [143:54:53<175:06:54,  4.11s/it]
 32%|███▏      | 71520/225000 [143:55:00<222:01:47,  5.21s/it]
 32%|███▏      | 71521/225000 [143:55:06<224:00:39,  5.25s/it]
 32%|███▏      | 71522/225000 [143:55:10<211:20:40,  4.96s/it]
 32%|███▏      | 71523/225000 [143:55:19<259:04:06,  6.08s/it]
 32%|███▏      | 71524/225000 [143:55:21<214:53:11,  5.04s/it]
 32%|███▏      | 71525/225000 [143:55:28<239:27:21,  5.62s/it]
 32%|███▏      | 71526/225000 [143:55:31<206:01:59,  4.83s/it]
 32%|███▏      | 71527/225000 [143:55:34<177:17:17,  4.16s/it]
 32%|███▏      | 71528/225000 [143:55:40<202:58:47,  4.76s/it]
 32%|███▏      | 71529/225000 [143:55:48<242:29:05,  5.69s/it]
 32%|███▏      | 71530/225000 [143:55:51<209:56:44,  4.92s/it]
 32%|███▏      | 71531/225000 [143:55:55<198:06:48,  4.65s/it]
 32%|███▏      | 71532/225000 [143:55:58<177:05:48,  4.15s/it]
 32%|███▏      | 71533/225000 [143:56:02<171:12:04,  4.02s/it]
 32%|███▏      | 71534/225000 [143:56:06<177:27:54,  4.16s/it]
 32%|███▏      | 71535/225000 [143:56:11<181:11:02,  4.25s/it]
 32%|███▏      | 71536/225000 [143:56:14<166:34:43,  3.91s/it]
 32%|███▏      | 71537/225000 [143:56:17<157:28:21,  3.69s/it]
 32%|███▏      | 71538/225000 [143:56:21<157:32:25,  3.70s/it]
 32%|███▏      | 71539/225000 [143:56:31<246:46:16,  5.79s/it]
 32%|███▏      | 71540/225000 [143:56:38<260:17:23,  6.11s/it]
 32%|███▏      | 71541/225000 [143:56:41<218:32:22,  5.13s/it]
 32%|███▏      | 71542/225000 [143:56:45<204:05:29,  4.79s/it]
 32%|███▏      | 71543/225000 [143:56:48<185:41:42,  4.36s/it]
 32%|███▏      | 71544/225000 [143:56:52<171:57:12,  4.03s/it]
 32%|███▏      | 71545/225000 [143:57:02<249:26:05,  5.85s/it]
 32%|███▏      | 71546/225000 [143:57:05<217:27:07,  5.10s/it]
 32%|███▏      | 71547/225000 [143:57:08<195:02:22,  4.58s/it]
 32%|███▏      | 71548/225000 [143:57:17<244:23:27,  5.73s/it]
 32%|███▏      | 71549/225000 [143:57:20<208:25:01,  4.89s/it]
 32%|███▏      | 71550/225000 [143:57:23<188:34:20,  4.42s/it]
                                                              
{'loss': '0.8106', 'grad_norm': '1.252', 'learning_rate': '4.056e-05', 'epoch': '0.318', 'num_input_tokens_seen': 238602224, 'train_runtime': '5.182e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71550/225000 [143:57:23<188:34:20,  4.42s/it]
 32%|███▏      | 71551/225000 [143:57:26<173:03:19,  4.06s/it]
 32%|███▏      | 71552/225000 [143:57:37<253:28:06,  5.95s/it]
 32%|███▏      | 71553/225000 [143:57:41<235:37:24,  5.53s/it]
 32%|███▏      | 71554/225000 [143:57:48<246:34:08,  5.78s/it]
 32%|███▏      | 71555/225000 [143:57:51<213:47:32,  5.02s/it]
 32%|███▏      | 71556/225000 [143:57:54<189:18:16,  4.44s/it]
 32%|███▏      | 71557/225000 [143:57:57<171:40:22,  4.03s/it]
 32%|███▏      | 71558/225000 [143:58:06<239:22:45,  5.62s/it]
 32%|███▏      | 71559/225000 [143:58:11<233:32:41,  5.48s/it]
 32%|███▏      | 71560/225000 [143:58:15<203:50:31,  4.78s/it]
 32%|███▏      | 71561/225000 [143:58:19<192:48:58,  4.52s/it]
 32%|███▏      | 71562/225000 [143:58:23<187:07:42,  4.39s/it]
 32%|███▏      | 71563/225000 [143:58:28<200:16:24,  4.70s/it]
 32%|███▏      | 71564/225000 [143:58:31<178:28:50,  4.19s/it]
 32%|███▏      | 71565/225000 [143:58:35<175:47:03,  4.12s/it]
 32%|███▏      | 71566/225000 [143:58:44<236:26:51,  5.55s/it]
 32%|███▏      | 71567/225000 [143:58:51<255:02:07,  5.98s/it]
 32%|███▏      | 71568/225000 [143:58:55<231:06:48,  5.42s/it]
 32%|███▏      | 71569/225000 [143:59:02<245:31:45,  5.76s/it]
 32%|███▏      | 71570/225000 [143:59:09<264:05:26,  6.20s/it]
 32%|███▏      | 71571/225000 [143:59:13<233:27:38,  5.48s/it]
 32%|███▏      | 71572/225000 [143:59:16<204:04:06,  4.79s/it]
 32%|███▏      | 71573/225000 [143:59:20<200:18:36,  4.70s/it]
 32%|███▏      | 71574/225000 [143:59:23<181:02:10,  4.25s/it]
 32%|███▏      | 71575/225000 [143:59:27<173:20:24,  4.07s/it]
 32%|███▏      | 71576/225000 [143:59:36<233:59:37,  5.49s/it]
 32%|███▏      | 71577/225000 [143:59:41<232:29:39,  5.46s/it]
 32%|███▏      | 71578/225000 [143:59:44<192:37:10,  4.52s/it]
 32%|███▏      | 71579/225000 [143:59:47<181:02:08,  4.25s/it]
 32%|███▏      | 71580/225000 [143:59:51<179:15:07,  4.21s/it]
 32%|███▏      | 71581/225000 [143:59:55<177:48:36,  4.17s/it]
 32%|███▏      | 71582/225000 [144:00:00<177:17:12,  4.16s/it]
 32%|███▏      | 71583/225000 [144:00:03<171:09:44,  4.02s/it]
 32%|███▏      | 71584/225000 [144:00:07<167:09:55,  3.92s/it]
 32%|███▏      | 71585/225000 [144:00:16<233:12:39,  5.47s/it]
 32%|███▏      | 71586/225000 [144:00:19<206:29:44,  4.85s/it]
 32%|███▏      | 71587/225000 [144:00:23<189:43:29,  4.45s/it]
 32%|███▏      | 71588/225000 [144:00:30<224:43:28,  5.27s/it]
 32%|███▏      | 71589/225000 [144:00:33<198:29:27,  4.66s/it]
 32%|███▏      | 71590/225000 [144:00:37<186:50:42,  4.38s/it]
 32%|███▏      | 71591/225000 [144:00:40<171:22:11,  4.02s/it]
 32%|███▏      | 71592/225000 [144:00:48<216:57:36,  5.09s/it]
 32%|███▏      | 71593/225000 [144:00:56<252:54:15,  5.93s/it]
 32%|███▏      | 71594/225000 [144:00:59<213:12:51,  5.00s/it]
 32%|███▏      | 71595/225000 [144:01:02<197:26:43,  4.63s/it]
 32%|███▏      | 71596/225000 [144:01:06<182:45:53,  4.29s/it]
 32%|███▏      | 71597/225000 [144:01:09<169:05:54,  3.97s/it]
 32%|███▏      | 71598/225000 [144:01:17<224:28:22,  5.27s/it]
 32%|███▏      | 71599/225000 [144:01:24<237:02:46,  5.56s/it]
 32%|███▏      | 71600/225000 [144:01:27<207:11:34,  4.86s/it]
                                                              
{'loss': '0.7739', 'grad_norm': '1.353', 'learning_rate': '4.055e-05', 'epoch': '0.3182', 'num_input_tokens_seen': 238702848, 'train_runtime': '5.185e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71600/225000 [144:01:27<207:11:34,  4.86s/it]
 32%|███▏      | 71601/225000 [144:01:30<189:46:07,  4.45s/it]
 32%|███▏      | 71602/225000 [144:01:35<198:33:20,  4.66s/it]
 32%|███▏      | 71603/225000 [144:01:46<268:41:49,  6.31s/it]
 32%|███▏      | 71604/225000 [144:01:49<230:04:28,  5.40s/it]
 32%|███▏      | 71605/225000 [144:01:53<212:07:52,  4.98s/it]
 32%|███▏      | 71606/225000 [144:01:59<220:47:39,  5.18s/it]
 32%|███▏      | 71607/225000 [144:02:05<235:05:41,  5.52s/it]
 32%|███▏      | 71608/225000 [144:02:08<206:11:11,  4.84s/it]
 32%|███▏      | 71609/225000 [144:02:12<192:13:29,  4.51s/it]
 32%|███▏      | 71610/225000 [144:02:15<171:48:55,  4.03s/it]
 32%|███▏      | 71611/225000 [144:02:18<161:58:53,  3.80s/it]
 32%|███▏      | 71612/225000 [144:02:29<251:53:22,  5.91s/it]
 32%|███▏      | 71613/225000 [144:02:31<206:24:28,  4.84s/it]
 32%|███▏      | 71614/225000 [144:02:34<177:33:10,  4.17s/it]
 32%|███▏      | 71615/225000 [144:02:37<165:06:29,  3.88s/it]
 32%|███▏      | 71616/225000 [144:02:41<169:23:52,  3.98s/it]
 32%|███▏      | 71617/225000 [144:02:49<213:07:44,  5.00s/it]
 32%|███▏      | 71618/225000 [144:02:56<244:19:00,  5.73s/it]
 32%|███▏      | 71619/225000 [144:03:03<254:58:17,  5.98s/it]
 32%|███▏      | 71620/225000 [144:03:08<251:09:44,  5.90s/it]
 32%|███▏      | 71621/225000 [144:03:11<214:34:51,  5.04s/it]
 32%|███▏      | 71622/225000 [144:03:15<191:13:27,  4.49s/it]
 32%|███▏      | 71623/225000 [144:03:18<173:43:31,  4.08s/it]
 32%|███▏      | 71624/225000 [144:03:22<172:13:24,  4.04s/it]
 32%|███▏      | 71625/225000 [144:03:25<168:37:26,  3.96s/it]
 32%|███▏      | 71626/225000 [144:03:29<166:55:43,  3.92s/it]
 32%|███▏      | 71627/225000 [144:03:38<223:08:43,  5.24s/it]
 32%|███▏      | 71628/225000 [144:03:44<232:27:13,  5.46s/it]
 32%|███▏      | 71629/225000 [144:03:47<205:39:40,  4.83s/it]
 32%|███▏      | 71630/225000 [144:03:51<191:16:39,  4.49s/it]
 32%|███▏      | 71631/225000 [144:03:59<247:28:12,  5.81s/it]
 32%|███▏      | 71632/225000 [144:04:03<217:57:08,  5.12s/it]
 32%|███▏      | 71633/225000 [144:04:06<189:58:52,  4.46s/it]
 32%|███▏      | 71634/225000 [144:04:11<197:34:39,  4.64s/it]
 32%|███▏      | 71635/225000 [144:04:17<213:14:51,  5.01s/it]
 32%|███▏      | 71636/225000 [144:04:20<187:35:01,  4.40s/it]
 32%|███▏      | 71637/225000 [144:04:23<174:49:01,  4.10s/it]
 32%|███▏      | 71638/225000 [144:04:26<164:09:54,  3.85s/it]
 32%|███▏      | 71639/225000 [144:04:33<195:05:16,  4.58s/it]
 32%|███▏      | 71640/225000 [144:04:44<277:58:38,  6.53s/it]
 32%|███▏      | 71641/225000 [144:04:47<237:33:52,  5.58s/it]
 32%|███▏      | 71642/225000 [144:04:51<214:55:04,  5.05s/it]
 32%|███▏      | 71643/225000 [144:04:56<209:10:39,  4.91s/it]
 32%|███▏      | 71644/225000 [144:04:59<195:58:57,  4.60s/it]
 32%|███▏      | 71645/225000 [144:05:03<187:43:40,  4.41s/it]
 32%|███▏      | 71646/225000 [144:05:07<174:38:39,  4.10s/it]
 32%|███▏      | 71647/225000 [144:05:11<169:58:40,  3.99s/it]
 32%|███▏      | 71648/225000 [144:05:15<171:47:08,  4.03s/it]
 32%|███▏      | 71649/225000 [144:05:20<191:23:15,  4.49s/it]
 32%|███▏      | 71650/225000 [144:05:32<286:56:43,  6.74s/it]
                                                              
{'loss': '0.8047', 'grad_norm': '1.125', 'learning_rate': '4.053e-05', 'epoch': '0.3184', 'num_input_tokens_seen': 238804672, 'train_runtime': '5.187e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71650/225000 [144:05:32<286:56:43,  6.74s/it]
 32%|███▏      | 71651/225000 [144:05:36<251:01:38,  5.89s/it]
 32%|███▏      | 71652/225000 [144:05:40<222:05:54,  5.21s/it]
 32%|███▏      | 71653/225000 [144:05:43<202:31:26,  4.75s/it]
 32%|███▏      | 71654/225000 [144:05:47<183:49:06,  4.32s/it]
 32%|███▏      | 71655/225000 [144:05:51<188:50:10,  4.43s/it]
 32%|███▏      | 71656/225000 [144:05:55<175:40:18,  4.12s/it]
 32%|███▏      | 71657/225000 [144:05:59<177:49:20,  4.17s/it]
 32%|███▏      | 71658/225000 [144:06:03<170:11:55,  4.00s/it]
 32%|███▏      | 71659/225000 [144:06:07<178:07:00,  4.18s/it]
 32%|███▏      | 71660/225000 [144:06:12<178:28:35,  4.19s/it]
 32%|███▏      | 71661/225000 [144:06:21<244:36:09,  5.74s/it]
 32%|███▏      | 71662/225000 [144:06:28<260:10:58,  6.11s/it]
 32%|███▏      | 71663/225000 [144:06:32<230:44:28,  5.42s/it]
 32%|███▏      | 71664/225000 [144:06:35<198:30:39,  4.66s/it]
 32%|███▏      | 71665/225000 [144:06:38<177:19:22,  4.16s/it]
 32%|███▏      | 71666/225000 [144:06:41<173:06:37,  4.06s/it]
 32%|███▏      | 71667/225000 [144:06:45<164:18:04,  3.86s/it]
 32%|███▏      | 71668/225000 [144:06:51<189:10:08,  4.44s/it]
 32%|███▏      | 71669/225000 [144:07:00<256:59:47,  6.03s/it]
 32%|███▏      | 71670/225000 [144:07:04<227:21:08,  5.34s/it]
 32%|███▏      | 71671/225000 [144:07:08<205:48:45,  4.83s/it]
 32%|███▏      | 71672/225000 [144:07:11<190:10:38,  4.47s/it]
 32%|███▏      | 71673/225000 [144:07:15<177:13:28,  4.16s/it]
 32%|███▏      | 71674/225000 [144:07:19<174:48:41,  4.10s/it]
 32%|███▏      | 71675/225000 [144:07:22<164:03:46,  3.85s/it]
 32%|███▏      | 71676/225000 [144:07:29<210:50:12,  4.95s/it]
 32%|███▏      | 71677/225000 [144:07:37<238:59:01,  5.61s/it]
 32%|███▏      | 71678/225000 [144:07:40<205:27:19,  4.82s/it]
 32%|███▏      | 71679/225000 [144:07:43<180:55:03,  4.25s/it]
 32%|███▏      | 71680/225000 [144:07:48<197:10:18,  4.63s/it]
 32%|███▏      | 71681/225000 [144:07:57<249:15:20,  5.85s/it]
 32%|███▏      | 71682/225000 [144:08:00<211:24:08,  4.96s/it]
 32%|███▏      | 71683/225000 [144:08:06<227:37:20,  5.34s/it]
 32%|███▏      | 71684/225000 [144:08:09<196:00:36,  4.60s/it]
 32%|███▏      | 71685/225000 [144:08:13<186:15:05,  4.37s/it]
 32%|███▏      | 71686/225000 [144:08:21<232:37:47,  5.46s/it]
 32%|███▏      | 71687/225000 [144:08:23<193:13:53,  4.54s/it]
 32%|███▏      | 71688/225000 [144:08:29<214:12:22,  5.03s/it]
 32%|███▏      | 71689/225000 [144:08:36<232:41:39,  5.46s/it]
 32%|███▏      | 71690/225000 [144:08:38<197:48:01,  4.64s/it]
 32%|███▏      | 71691/225000 [144:08:46<233:23:51,  5.48s/it]
 32%|███▏      | 71692/225000 [144:08:49<198:54:44,  4.67s/it]
 32%|███▏      | 71693/225000 [144:08:52<188:48:25,  4.43s/it]
 32%|███▏      | 71694/225000 [144:09:00<227:53:51,  5.35s/it]
 32%|███▏      | 71695/225000 [144:09:03<202:55:44,  4.77s/it]
 32%|███▏      | 71696/225000 [144:09:10<228:01:14,  5.35s/it]
 32%|███▏      | 71697/225000 [144:09:14<211:28:17,  4.97s/it]
 32%|███▏      | 71698/225000 [144:09:18<193:30:15,  4.54s/it]
 32%|███▏      | 71699/225000 [144:09:24<217:50:51,  5.12s/it]
 32%|███▏      | 71700/225000 [144:09:27<192:09:13,  4.51s/it]
                                                              
{'loss': '0.7797', 'grad_norm': '1.362', 'learning_rate': '4.052e-05', 'epoch': '0.3187', 'num_input_tokens_seen': 238906776, 'train_runtime': '5.19e+05', 'train_tokens_per_second': '460.4'}

 32%|███▏      | 71700/225000 [144:09:27<192:09:13,  4.51s/it]
 32%|███▏      | 71701/225000 [144:09:31<183:22:11,  4.31s/it]
 32%|███▏      | 71702/225000 [144:09:35<180:44:29,  4.24s/it]
 32%|███▏      | 71703/225000 [144:09:41<200:01:37,  4.70s/it]
 32%|███▏      | 71704/225000 [144:09:47<215:31:09,  5.06s/it]
 32%|███▏      | 71705/225000 [144:09:50<195:41:31,  4.60s/it]
 32%|███▏      | 71706/225000 [144:10:00<255:39:19,  6.00s/it]
 32%|███▏      | 71707/225000 [144:10:03<224:11:28,  5.27s/it]
 32%|███▏      | 71708/225000 [144:10:06<198:43:18,  4.67s/it]
 32%|███▏      | 71709/225000 [144:10:10<178:51:07,  4.20s/it]
 32%|███▏      | 71710/225000 [144:10:16<210:10:04,  4.94s/it]
 32%|███▏      | 71711/225000 [144:10:24<248:36:20,  5.84s/it]
 32%|███▏      | 71712/225000 [144:10:27<209:01:53,  4.91s/it]
 32%|███▏      | 71713/225000 [144:10:30<188:58:18,  4.44s/it]
 32%|███▏      | 71714/225000 [144:10:37<220:29:45,  5.18s/it]
 32%|███▏      | 71715/225000 [144:10:41<199:32:33,  4.69s/it]
 32%|███▏      | 71716/225000 [144:10:47<220:38:11,  5.18s/it]
 32%|███▏      | 71717/225000 [144:10:51<203:30:16,  4.78s/it]
 32%|███▏      | 71718/225000 [144:10:54<185:08:35,  4.35s/it]
 32%|███▏      | 71719/225000 [144:10:57<166:27:45,  3.91s/it]
 32%|███▏      | 71720/225000 [144:11:08<250:37:16,  5.89s/it]
 32%|███▏      | 71721/225000 [144:11:12<230:12:02,  5.41s/it]
 32%|███▏      | 71722/225000 [144:11:16<209:24:09,  4.92s/it]
 32%|███▏      | 71723/225000 [144:11:20<201:47:12,  4.74s/it]
 32%|███▏      | 71724/225000 [144:11:26<216:44:08,  5.09s/it]
 32%|███▏      | 71725/225000 [144:11:33<240:22:19,  5.65s/it]
 32%|███▏      | 71726/225000 [144:11:38<230:46:57,  5.42s/it]
 32%|███▏      | 71727/225000 [144:11:40<194:26:46,  4.57s/it]
 32%|███▏      | 71728/225000 [144:11:47<220:41:21,  5.18s/it]
 32%|███▏      | 71729/225000 [144:11:50<188:54:15,  4.44s/it]
 32%|███▏      | 71730/225000 [144:11:53<170:45:39,  4.01s/it]
 32%|███▏      | 71731/225000 [144:12:01<225:18:59,  5.29s/it]
 32%|███▏      | 71732/225000 [144:12:03<185:35:29,  4.36s/it]
 32%|███▏      | 71733/225000 [144:12:10<220:46:17,  5.19s/it]
 32%|███▏      | 71734/225000 [144:12:13<195:55:41,  4.60s/it]
 32%|███▏      | 71735/225000 [144:12:17<178:05:38,  4.18s/it]
 32%|███▏      | 71736/225000 [144:12:21<185:21:42,  4.35s/it]
 32%|███▏      | 71737/225000 [144:12:27<204:37:55,  4.81s/it]
 32%|███▏      | 71738/225000 [144:12:31<189:45:16,  4.46s/it]
 32%|███▏      | 71739/225000 [144:12:34<177:59:20,  4.18s/it]
 32%|███▏      | 71740/225000 [144:12:38<165:09:25,  3.88s/it]
 32%|███▏      | 71741/225000 [144:12:41<155:38:11,  3.66s/it]
 32%|███▏      | 71742/225000 [144:12:44<156:03:00,  3.67s/it]
 32%|███▏      | 71743/225000 [144:12:53<220:29:18,  5.18s/it]
 32%|███▏      | 71744/225000 [144:13:02<265:48:53,  6.24s/it]
 32%|███▏      | 71745/225000 [144:13:05<221:27:12,  5.20s/it]
 32%|███▏      | 71746/225000 [144:13:07<189:35:31,  4.45s/it]
 32%|███▏      | 71747/225000 [144:13:11<185:20:31,  4.35s/it]
 32%|███▏      | 71748/225000 [144:13:17<204:49:03,  4.81s/it]
 32%|███▏      | 71749/225000 [144:13:20<181:43:32,  4.27s/it]
 32%|███▏      | 71750/225000 [144:13:24<173:02:00,  4.06s/it]
                                                              
{'loss': '0.7903', 'grad_norm': '1.348', 'learning_rate': '4.051e-05', 'epoch': '0.3189', 'num_input_tokens_seen': 239006696, 'train_runtime': '5.192e+05', 'train_tokens_per_second': '460.3'}

 32%|███▏      | 71750/225000 [144:13:24<173:02:00,  4.06s/it]
 32%|███▏      | 71751/225000 [144:13:28<169:36:18,  3.98s/it]
 32%|███▏      | 71752/225000 [144:13:37<236:10:45,  5.55s/it]
 32%|███▏      | 71753/225000 [144:13:42<226:26:47,  5.32s/it]
 32%|███▏      | 71754/225000 [144:13:45<199:49:44,  4.69s/it]
 32%|███▏      | 71755/225000 [144:13:49<189:50:19,  4.46s/it]
 32%|███▏      | 71756/225000 [144:13:52<173:18:24,  4.07s/it]
 32%|███▏      | 71757/225000 [144:13:55<160:36:29,  3.77s/it]
 32%|███▏      | 71758/225000 [144:13:59<165:13:09,  3.88s/it]
 32%|███▏      | 71759/225000 [144:14:07<219:58:49,  5.17s/it]
 32%|███▏      | 71760/225000 [144:14:14<240:23:56,  5.65s/it]
 32%|███▏      | 71761/225000 [144:14:18<211:51:42,  4.98s/it]
 32%|███▏      | 71762/225000 [144:14:21<190:40:27,  4.48s/it]
 32%|███▏      | 71763/225000 [144:14:24<170:54:23,  4.02s/it]
 32%|███▏      | 71764/225000 [144:14:32<225:56:17,  5.31s/it]
 32%|███▏      | 71765/225000 [144:14:37<224:48:57,  5.28s/it]
 32%|███▏      | 71766/225000 [144:14:42<218:44:26,  5.14s/it]
 32%|███▏      | 71767/225000 [144:14:46<195:28:34,  4.59s/it]
 32%|███▏      | 71768/225000 [144:14:48<169:08:44,  3.97s/it]
 32%|███▏      | 71769/225000 [144:14:53<182:11:22,  4.28s/it]
 32%|███▏      | 71770/225000 [144:15:00<219:27:36,  5.16s/it]
 32%|███▏      | 71771/225000 [144:15:03<193:34:28,  4.55s/it]
 32%|███▏      | 71772/225000 [144:15:06<174:28:13,  4.10s/it]
 32%|███▏      | 71773/225000 [144:15:10<173:01:24,  4.07s/it]
 32%|███▏      | 71774/225000 [144:15:14<167:20:06,  3.93s/it]
 32%|███▏      | 71775/225000 [144:15:20<196:32:20,  4.62s/it]
 32%|███▏      | 71776/225000 [144:15:29<251:28:48,  5.91s/it]
 32%|███▏      | 71777/225000 [144:15:34<233:10:19,  5.48s/it]
 32%|███▏      | 71778/225000 [144:15:37<207:01:46,  4.86s/it]
 32%|███▏      | 71779/225000 [144:15:41<192:30:28,  4.52s/it]
 32%|███▏      | 71780/225000 [144:15:48<230:09:59,  5.41s/it]
 32%|███▏      | 71781/225000 [144:15:57<268:31:46,  6.31s/it]
 32%|███▏      | 71782/225000 [144:16:00<224:09:57,  5.27s/it]
 32%|███▏      | 71783/225000 [144:16:03<204:33:19,  4.81s/it]
 32%|███▏      | 71784/225000 [144:16:11<245:37:30,  5.77s/it]
 32%|███▏      | 71785/225000 [144:16:15<221:27:07,  5.20s/it]
 32%|███▏      | 71786/225000 [144:16:19<202:38:18,  4.76s/it]
 32%|███▏      | 71787/225000 [144:16:23<197:25:51,  4.64s/it]
 32%|███▏      | 71788/225000 [144:16:26<178:53:03,  4.20s/it]
 32%|███▏      | 71789/225000 [144:16:34<225:14:47,  5.29s/it]
 32%|███▏      | 71790/225000 [144:16:41<245:32:52,  5.77s/it]
 32%|███▏      | 71791/225000 [144:16:45<220:25:59,  5.18s/it]
 32%|███▏      | 71792/225000 [144:16:48<195:27:38,  4.59s/it]
 32%|███▏      | 71793/225000 [144:16:52<180:15:15,  4.24s/it]
 32%|███▏      | 71794/225000 [144:16:56<178:52:18,  4.20s/it]
 32%|███▏      | 71795/225000 [144:17:06<255:49:41,  6.01s/it]
 32%|███▏      | 71796/225000 [144:17:11<249:38:45,  5.87s/it]
 32%|███▏      | 71797/225000 [144:17:14<212:00:42,  4.98s/it]
 32%|███▏      | 71798/225000 [144:17:17<184:32:22,  4.34s/it]
 32%|███▏      | 71799/225000 [144:17:24<218:26:43,  5.13s/it]
 32%|███▏      | 71800/225000 [144:17:31<233:13:25,  5.48s/it]
                                                              
{'loss': '0.7906', 'grad_norm': '1.507', 'learning_rate': '4.049e-05', 'epoch': '0.3191', 'num_input_tokens_seen': 239107064, 'train_runtime': '5.195e+05', 'train_tokens_per_second': '460.3'}

 32%|███▏      | 71800/225000 [144:17:31<233:13:25,  5.48s/it]
 32%|███▏      | 71801/225000 [144:17:34<211:40:48,  4.97s/it]
 32%|███▏      | 71802/225000 [144:17:38<194:57:24,  4.58s/it]
 32%|███▏      | 71803/225000 [144:17:43<196:58:27,  4.63s/it]
 32%|███▏      | 71804/225000 [144:17:46<180:18:34,  4.24s/it]
 32%|███▏      | 71805/225000 [144:17:50<175:28:51,  4.12s/it]
 32%|███▏      | 71806/225000 [144:17:58<221:02:20,  5.19s/it]
 32%|███▏      | 71807/225000 [144:18:04<241:01:18,  5.66s/it]
 32%|███▏      | 71808/225000 [144:18:08<210:53:54,  4.96s/it]
 32%|███▏      | 71809/225000 [144:18:15<237:11:44,  5.57s/it]
 32%|███▏      | 71810/225000 [144:18:18<205:53:11,  4.84s/it]
 32%|███▏      | 71811/225000 [144:18:21<187:31:08,  4.41s/it]
 32%|███▏      | 71812/225000 [144:18:24<173:41:57,  4.08s/it]
 32%|███▏      | 71813/225000 [144:18:28<168:20:50,  3.96s/it]
 32%|███▏      | 71814/225000 [144:18:32<166:40:24,  3.92s/it]
 32%|███▏      | 71815/225000 [144:18:41<235:59:10,  5.55s/it]
 32%|███▏      | 71816/225000 [144:18:45<209:28:24,  4.92s/it]
 32%|███▏      | 71817/225000 [144:18:48<189:56:39,  4.46s/it]
 32%|███▏      | 71818/225000 [144:18:55<225:46:09,  5.31s/it]
 32%|███▏      | 71819/225000 [144:18:59<200:28:05,  4.71s/it]
 32%|███▏      | 71820/225000 [144:19:03<190:13:51,  4.47s/it]
 32%|███▏      | 71821/225000 [144:19:08<195:17:08,  4.59s/it]
 32%|███▏      | 71822/225000 [144:19:13<210:17:24,  4.94s/it]
 32%|███▏      | 71823/225000 [144:19:22<256:31:21,  6.03s/it]
 32%|███▏      | 71824/225000 [144:19:26<226:32:28,  5.32s/it]
 32%|███▏      | 71825/225000 [144:19:29<197:51:59,  4.65s/it]
 32%|███▏      | 71826/225000 [144:19:37<240:16:43,  5.65s/it]
 32%|███▏      | 71827/225000 [144:19:40<212:46:39,  5.00s/it]
 32%|███▏      | 71828/225000 [144:19:43<190:40:05,  4.48s/it]
 32%|███▏      | 71829/225000 [144:19:47<183:36:56,  4.32s/it]
 32%|███▏      | 71830/225000 [144:19:51<181:36:06,  4.27s/it]
 32%|███▏      | 71831/225000 [144:19:57<202:48:46,  4.77s/it]
 32%|███▏      | 71832/225000 [144:20:04<230:06:54,  5.41s/it]
 32%|███▏      | 71833/225000 [144:20:08<203:19:41,  4.78s/it]
 32%|███▏      | 71834/225000 [144:20:12<202:02:09,  4.75s/it]
 32%|███▏      | 71835/225000 [144:20:15<182:02:51,  4.28s/it]
 32%|███▏      | 71836/225000 [144:20:27<273:05:43,  6.42s/it]
 32%|███▏      | 71837/225000 [144:20:31<247:34:42,  5.82s/it]
 32%|███▏      | 71838/225000 [144:20:34<204:24:00,  4.80s/it]
 32%|███▏      | 71839/225000 [144:20:42<251:24:04,  5.91s/it]
 32%|███▏      | 71840/225000 [144:20:45<214:24:41,  5.04s/it]
 32%|███▏      | 71841/225000 [144:20:48<191:20:44,  4.50s/it]
 32%|███▏      | 71842/225000 [144:20:55<214:16:55,  5.04s/it]
 32%|███▏      | 71843/225000 [144:20:58<193:06:12,  4.54s/it]
 32%|███▏      | 71844/225000 [144:21:01<176:23:31,  4.15s/it]
 32%|███▏      | 71845/225000 [144:21:05<171:36:54,  4.03s/it]
 32%|███▏      | 71846/225000 [144:21:08<160:59:33,  3.78s/it]
 32%|███▏      | 71847/225000 [144:21:18<240:57:56,  5.66s/it]
 32%|███▏      | 71848/225000 [144:21:21<203:31:33,  4.78s/it]
 32%|███▏      | 71849/225000 [144:21:23<172:06:06,  4.05s/it]
 32%|███▏      | 71850/225000 [144:21:27<162:29:26,  3.82s/it]
                                                              
{'loss': '0.7759', 'grad_norm': '1.059', 'learning_rate': '4.048e-05', 'epoch': '0.3193', 'num_input_tokens_seen': 239208448, 'train_runtime': '5.197e+05', 'train_tokens_per_second': '460.3'}

 32%|███▏      | 71850/225000 [144:21:27<162:29:26,  3.82s/it]
 32%|███▏      | 71851/225000 [144:21:35<218:37:33,  5.14s/it]
 32%|███▏      | 71852/225000 [144:21:40<222:00:11,  5.22s/it]
 32%|███▏      | 71853/225000 [144:21:43<193:47:20,  4.56s/it]
 32%|███▏      | 71854/225000 [144:21:47<175:27:28,  4.12s/it]
 32%|███▏      | 71855/225000 [144:21:51<184:54:48,  4.35s/it]
 32%|███▏      | 71856/225000 [144:21:59<228:27:22,  5.37s/it]
 32%|███▏      | 71857/225000 [144:22:03<206:59:15,  4.87s/it]
 32%|███▏      | 71858/225000 [144:22:07<193:47:45,  4.56s/it]
 32%|███▏      | 71859/225000 [144:22:10<181:43:25,  4.27s/it]
 32%|███▏      | 71860/225000 [144:22:14<172:58:00,  4.07s/it]
 32%|███▏      | 71861/225000 [144:22:18<169:23:30,  3.98s/it]
 32%|███▏      | 71862/225000 [144:22:21<155:16:25,  3.65s/it]
 32%|███▏      | 71863/225000 [144:22:25<171:29:24,  4.03s/it]
 32%|███▏      | 71864/225000 [144:22:34<234:50:55,  5.52s/it]
 32%|███▏      | 71865/225000 [144:22:40<239:07:08,  5.62s/it]
 32%|███▏      | 71866/225000 [144:22:44<210:59:33,  4.96s/it]
 32%|███▏      | 71867/225000 [144:22:47<184:58:28,  4.35s/it]
 32%|███▏      | 71868/225000 [144:22:55<238:29:41,  5.61s/it]
 32%|███▏      | 71869/225000 [144:22:59<211:06:57,  4.96s/it]
 32%|███▏      | 71870/225000 [144:23:02<195:44:54,  4.60s/it]
 32%|███▏      | 71871/225000 [144:23:05<175:20:02,  4.12s/it]
 32%|███▏      | 71872/225000 [144:23:09<163:37:30,  3.85s/it]
 32%|███▏      | 71873/225000 [144:23:16<211:38:03,  4.98s/it]
 32%|███▏      | 71874/225000 [144:23:22<215:54:24,  5.08s/it]
 32%|███▏      | 71875/225000 [144:23:26<202:40:07,  4.76s/it]
 32%|███▏      | 71876/225000 [144:23:29<191:44:29,  4.51s/it]
 32%|███▏      | 71877/225000 [144:23:40<269:53:26,  6.35s/it]
 32%|███▏      | 71878/225000 [144:23:43<220:44:30,  5.19s/it]
 32%|███▏      | 71879/225000 [144:23:46<200:30:40,  4.71s/it]
 32%|███▏      | 71880/225000 [144:23:54<234:43:47,  5.52s/it]
 32%|███▏      | 71881/225000 [144:23:58<218:44:58,  5.14s/it]
 32%|███▏      | 71882/225000 [144:24:02<208:05:08,  4.89s/it]
 32%|███▏      | 71883/225000 [144:24:06<195:04:04,  4.59s/it]
 32%|███▏      | 71884/225000 [144:24:10<181:03:58,  4.26s/it]
 32%|███▏      | 71885/225000 [144:24:13<167:24:21,  3.94s/it]
 32%|███▏      | 71886/225000 [144:24:23<248:26:45,  5.84s/it]
 32%|███▏      | 71887/225000 [144:24:27<219:03:31,  5.15s/it]
 32%|███▏      | 71888/225000 [144:24:30<192:17:28,  4.52s/it]
 32%|███▏      | 71889/225000 [144:24:37<224:40:11,  5.28s/it]
 32%|███▏      | 71890/225000 [144:24:39<193:17:57,  4.54s/it]
 32%|███▏      | 71891/225000 [144:24:42<171:38:07,  4.04s/it]
 32%|███▏      | 71892/225000 [144:24:46<160:51:59,  3.78s/it]
 32%|███▏      | 71893/225000 [144:24:48<149:56:41,  3.53s/it]
 32%|███▏      | 71894/225000 [144:24:59<239:46:28,  5.64s/it]
 32%|███▏      | 71895/225000 [144:25:02<209:00:31,  4.91s/it]
 32%|███▏      | 71896/225000 [144:25:05<180:03:29,  4.23s/it]
 32%|███▏      | 71897/225000 [144:25:13<230:06:13,  5.41s/it]
 32%|███▏      | 71898/225000 [144:25:18<224:52:39,  5.29s/it]
 32%|███▏      | 71899/225000 [144:25:21<196:36:10,  4.62s/it]
 32%|███▏      | 71900/225000 [144:25:25<184:01:40,  4.33s/it]
                                                              
{'loss': '0.8206', 'grad_norm': '1.55', 'learning_rate': '4.046e-05', 'epoch': '0.3196', 'num_input_tokens_seen': 239306648, 'train_runtime': '5.199e+05', 'train_tokens_per_second': '460.3'}

 32%|███▏      | 71900/225000 [144:25:25<184:01:40,  4.33s/it]
 32%|███▏      | 71901/225000 [144:25:28<169:39:01,  3.99s/it]
 32%|███▏      | 71902/225000 [144:25:32<172:56:03,  4.07s/it]
 32%|███▏      | 71903/225000 [144:25:36<171:07:14,  4.02s/it]
 32%|███▏      | 71904/225000 [144:25:40<172:24:44,  4.05s/it]
 32%|███▏      | 71905/225000 [144:25:43<161:37:08,  3.80s/it]
 32%|███▏      | 71906/225000 [144:25:47<152:23:39,  3.58s/it]
 32%|███▏      | 71907/225000 [144:25:52<177:13:25,  4.17s/it]
 32%|███▏      | 71908/225000 [144:26:01<240:39:34,  5.66s/it]
 32%|███▏      | 71909/225000 [144:26:09<263:47:50,  6.20s/it]
 32%|███▏      | 71910/225000 [144:26:12<223:31:57,  5.26s/it]
 32%|███▏      | 71911/225000 [144:26:14<189:41:57,  4.46s/it]
 32%|███▏      | 71912/225000 [144:26:17<171:43:13,  4.04s/it]
 32%|███▏      | 71913/225000 [144:26:26<231:53:37,  5.45s/it]
 32%|███▏      | 71914/225000 [144:26:32<233:46:32,  5.50s/it]
 32%|███▏      | 71915/225000 [144:26:35<200:10:11,  4.71s/it]
 32%|███▏      | 71916/225000 [144:26:43<244:03:47,  5.74s/it]
 32%|███▏      | 71917/225000 [144:26:46<207:18:23,  4.88s/it]
 32%|███▏      | 71918/225000 [144:26:49<185:15:06,  4.36s/it]
 32%|███▏      | 71919/225000 [144:26:52<175:25:38,  4.13s/it]
 32%|███▏      | 71920/225000 [144:26:55<159:26:12,  3.75s/it]
 32%|███▏      | 71921/225000 [144:26:59<157:00:09,  3.69s/it]
 32%|███▏      | 71922/225000 [144:27:03<164:30:16,  3.87s/it]
 32%|███▏      | 71923/225000 [144:27:13<247:06:00,  5.81s/it]
 32%|███▏      | 71924/225000 [144:27:18<232:56:58,  5.48s/it]
 32%|███▏      | 71925/225000 [144:27:21<197:31:29,  4.65s/it]
 32%|███▏      | 71926/225000 [144:27:24<184:31:22,  4.34s/it]
 32%|███▏      | 71927/225000 [144:27:34<249:30:08,  5.87s/it]
 32%|███▏      | 71928/225000 [144:27:37<218:48:00,  5.15s/it]
 32%|███▏      | 71929/225000 [144:27:41<204:27:40,  4.81s/it]
 32%|███▏      | 71930/225000 [144:27:45<187:13:01,  4.40s/it]
 32%|███▏      | 71931/225000 [144:27:48<169:01:44,  3.98s/it]
 32%|███▏      | 71932/225000 [144:27:52<178:42:04,  4.20s/it]
 32%|███▏      | 71933/225000 [144:27:56<174:18:36,  4.10s/it]
 32%|███▏      | 71934/225000 [144:27:59<161:13:10,  3.79s/it]
 32%|███▏      | 71935/225000 [144:28:06<200:13:06,  4.71s/it]
 32%|███▏      | 71936/225000 [144:28:15<253:55:30,  5.97s/it]
 32%|███▏      | 71937/225000 [144:28:18<219:10:17,  5.15s/it]
 32%|███▏      | 71938/225000 [144:28:23<208:12:51,  4.90s/it]
 32%|███▏      | 71939/225000 [144:28:31<257:00:50,  6.04s/it]
 32%|███▏      | 71940/225000 [144:28:34<212:59:00,  5.01s/it]
 32%|███▏      | 71941/225000 [144:28:40<222:22:40,  5.23s/it]
 32%|███▏      | 71942/225000 [144:28:43<194:59:36,  4.59s/it]
 32%|███▏      | 71943/225000 [144:28:46<181:12:45,  4.26s/it]
 32%|███▏      | 71944/225000 [144:28:54<219:29:49,  5.16s/it]
 32%|███▏      | 71945/225000 [144:28:57<194:02:18,  4.56s/it]
 32%|███▏      | 71946/225000 [144:29:00<173:37:53,  4.08s/it]
 32%|███▏      | 71947/225000 [144:29:06<198:14:33,  4.66s/it]
 32%|███▏      | 71948/225000 [144:29:13<230:48:54,  5.43s/it]
 32%|███▏      | 71949/225000 [144:29:16<203:29:13,  4.79s/it]
 32%|███▏      | 71950/225000 [144:29:19<179:33:58,  4.22s/it]
                                                              
{'loss': '0.7784', 'grad_norm': '1.654', 'learning_rate': '4.045e-05', 'epoch': '0.3198', 'num_input_tokens_seen': 239406008, 'train_runtime': '5.202e+05', 'train_tokens_per_second': '460.3'}

 32%|███▏      | 71950/225000 [144:29:19<179:33:58,  4.22s/it]
 32%|███▏      | 71951/225000 [144:29:22<163:55:49,  3.86s/it]
 32%|███▏      | 71952/225000 [144:29:30<213:13:02,  5.02s/it]
 32%|███▏      | 71953/225000 [144:29:36<222:15:47,  5.23s/it]
 32%|███▏      | 71954/225000 [144:29:44<256:50:57,  6.04s/it]
 32%|███▏      | 71955/225000 [144:29:48<230:06:59,  5.41s/it]
 32%|███▏      | 71956/225000 [144:29:51<205:56:28,  4.84s/it]
 32%|███▏      | 71957/225000 [144:29:54<183:18:36,  4.31s/it]
 32%|███▏      | 71958/225000 [144:30:01<221:24:42,  5.21s/it]
 32%|███▏      | 71959/225000 [144:30:08<241:58:26,  5.69s/it]
 32%|███▏      | 71960/225000 [144:30:12<222:50:24,  5.24s/it]
 32%|███▏      | 71961/225000 [144:30:16<202:06:25,  4.75s/it]
 32%|███▏      | 71962/225000 [144:30:20<189:14:32,  4.45s/it]
 32%|███▏      | 71963/225000 [144:30:23<179:02:52,  4.21s/it]
 32%|███▏      | 71964/225000 [144:30:28<181:52:49,  4.28s/it]
 32%|███▏      | 71965/225000 [144:30:31<161:13:03,  3.79s/it]
 32%|███▏      | 71966/225000 [144:30:34<159:15:50,  3.75s/it]
 32%|███▏      | 71967/225000 [144:30:40<190:16:56,  4.48s/it]
 32%|███▏      | 71968/225000 [144:30:52<279:57:50,  6.59s/it]
 32%|███▏      | 71969/225000 [144:30:55<241:12:47,  5.67s/it]
 32%|███▏      | 71970/225000 [144:31:00<221:07:22,  5.20s/it]
 32%|███▏      | 71971/225000 [144:31:05<225:13:36,  5.30s/it]
 32%|███▏      | 71972/225000 [144:31:09<201:47:00,  4.75s/it]
 32%|███▏      | 71973/225000 [144:31:11<176:29:10,  4.15s/it]
 32%|███▏      | 71974/225000 [144:31:18<211:24:35,  4.97s/it]
 32%|███▏      | 71975/225000 [144:31:22<197:00:14,  4.63s/it]
 32%|███▏      | 71976/225000 [144:31:24<167:57:49,  3.95s/it]
 32%|███▏      | 71977/225000 [144:31:27<156:02:54,  3.67s/it]
 32%|███▏      | 71978/225000 [144:31:30<146:58:30,  3.46s/it]
 32%|███▏      | 71979/225000 [144:31:37<185:25:23,  4.36s/it]
 32%|███▏      | 71980/225000 [144:31:44<217:06:40,  5.11s/it]
 32%|███▏      | 71981/225000 [144:31:47<197:56:36,  4.66s/it]
 32%|███▏      | 71982/225000 [144:31:51<184:23:27,  4.34s/it]
 32%|███▏      | 71983/225000 [144:31:55<178:25:49,  4.20s/it]
 32%|███▏      | 71984/225000 [144:32:00<193:21:30,  4.55s/it]
 32%|███▏      | 71985/225000 [144:32:07<228:48:12,  5.38s/it]
 32%|███▏      | 71986/225000 [144:32:11<203:58:37,  4.80s/it]
 32%|███▏      | 71987/225000 [144:32:15<189:23:48,  4.46s/it]
 32%|███▏      | 71988/225000 [144:32:19<185:20:05,  4.36s/it]
 32%|███▏      | 71989/225000 [144:32:23<181:23:08,  4.27s/it]
 32%|███▏      | 71990/225000 [144:32:27<178:06:14,  4.19s/it]
 32%|███▏      | 71991/225000 [144:32:31<179:02:45,  4.21s/it]
 32%|███▏      | 71992/225000 [144:32:43<272:36:53,  6.41s/it]
 32%|███▏      | 71993/225000 [144:32:47<247:18:29,  5.82s/it]
 32%|███▏      | 71994/225000 [144:32:53<252:20:06,  5.94s/it]
 32%|███▏      | 71995/225000 [144:32:57<229:04:37,  5.39s/it]
 32%|███▏      | 71996/225000 [144:33:00<193:03:44,  4.54s/it]
 32%|███▏      | 71997/225000 [144:33:03<178:16:36,  4.19s/it]
 32%|███▏      | 71998/225000 [144:33:07<174:32:39,  4.11s/it]
 32%|███▏      | 71999/225000 [144:33:10<160:16:30,  3.77s/it]
 32%|███▏      | 72000/225000 [144:33:13<153:28:09,  3.61s/it]
                                                              
{'loss': '0.782', 'grad_norm': '2.501', 'learning_rate': '4.043e-05', 'epoch': '0.32', 'num_input_tokens_seen': 239504856, 'train_runtime': '5.204e+05', 'train_tokens_per_second': '460.2'}

 32%|███▏      | 72000/225000 [144:33:13<153:28:09,  3.61s/it]
 32%|███▏      | 72001/225000 [144:33:22<218:47:47,  5.15s/it]
 32%|███▏      | 72002/225000 [144:33:28<232:45:01,  5.48s/it]
 32%|███▏      | 72003/225000 [144:33:31<199:37:46,  4.70s/it]
 32%|███▏      | 72004/225000 [144:33:39<235:55:36,  5.55s/it]
 32%|███▏      | 72005/225000 [144:33:41<196:55:15,  4.63s/it]
 32%|███▏      | 72006/225000 [144:33:47<207:36:32,  4.89s/it]
 32%|███▏      | 72007/225000 [144:33:52<211:11:18,  4.97s/it]
 32%|███▏      | 72008/225000 [144:33:56<202:04:45,  4.76s/it]
 32%|███▏      | 72009/225000 [144:34:00<188:44:53,  4.44s/it]
 32%|███▏      | 72010/225000 [144:34:03<176:55:51,  4.16s/it]
 32%|███▏      | 72011/225000 [144:34:07<166:54:04,  3.93s/it]
 32%|███▏      | 72012/225000 [144:34:10<160:21:03,  3.77s/it]
 32%|███▏      | 72013/225000 [144:34:16<182:25:30,  4.29s/it]
 32%|███▏      | 72014/225000 [144:34:26<263:01:25,  6.19s/it]
 32%|███▏      | 72015/225000 [144:34:30<231:00:52,  5.44s/it]
 32%|███▏      | 72016/225000 [144:34:34<212:08:12,  4.99s/it]
 32%|███▏      | 72017/225000 [144:34:38<197:03:34,  4.64s/it]
 32%|███▏      | 72018/225000 [144:34:45<233:20:35,  5.49s/it]
 32%|███▏      | 72019/225000 [144:34:48<205:04:53,  4.83s/it]
 32%|███▏      | 72020/225000 [144:34:51<177:44:02,  4.18s/it]
 32%|███▏      | 72021/225000 [144:34:54<163:42:37,  3.85s/it]
 32%|███▏      | 72022/225000 [144:34:58<158:19:13,  3.73s/it]
 32%|███▏      | 72023/225000 [144:35:01<152:39:44,  3.59s/it]
 32%|███▏      | 72024/225000 [144:35:08<195:45:18,  4.61s/it]
 32%|███▏      | 72025/225000 [144:35:15<232:51:07,  5.48s/it]
 32%|███▏      | 72026/225000 [144:35:19<208:48:05,  4.91s/it]
 32%|███▏      | 72027/225000 [144:35:22<189:13:39,  4.45s/it]
 32%|███▏      | 72028/225000 [144:35:26<179:39:37,  4.23s/it]
 32%|███▏      | 72029/225000 [144:35:30<174:54:16,  4.12s/it]
 32%|███▏      | 72030/225000 [144:35:34<168:06:01,  3.96s/it]
 32%|███▏      | 72031/225000 [144:35:43<233:14:16,  5.49s/it]
 32%|███▏      | 72032/225000 [144:35:50<252:11:08,  5.94s/it]
 32%|███▏      | 72033/225000 [144:35:53<221:57:06,  5.22s/it]
 32%|███▏      | 72034/225000 [144:35:59<228:24:45,  5.38s/it]
 32%|███▏      | 72035/225000 [144:36:03<218:23:31,  5.14s/it]
 32%|███▏      | 72036/225000 [144:36:08<211:24:18,  4.98s/it]
 32%|███▏      | 72037/225000 [144:36:11<186:38:57,  4.39s/it]
 32%|███▏      | 72038/225000 [144:36:15<184:58:20,  4.35s/it]
 32%|███▏      | 72039/225000 [144:36:19<177:31:52,  4.18s/it]
 32%|███▏      | 72040/225000 [144:36:23<169:01:29,  3.98s/it]
 32%|███▏      | 72041/225000 [144:36:27<171:31:28,  4.04s/it]
 32%|███▏      | 72042/225000 [144:36:30<165:14:59,  3.89s/it]
 32%|███▏      | 72043/225000 [144:36:37<203:02:15,  4.78s/it]
 32%|███▏      | 72044/225000 [144:36:45<243:45:01,  5.74s/it]
 32%|███▏      | 72045/225000 [144:36:49<219:04:59,  5.16s/it]
 32%|███▏      | 72046/225000 [144:36:52<185:34:47,  4.37s/it]
 32%|███▏      | 72047/225000 [144:36:55<172:33:25,  4.06s/it]
 32%|███▏      | 72048/225000 [144:37:03<226:40:00,  5.34s/it]
 32%|███▏      | 72049/225000 [144:37:08<224:23:19,  5.28s/it]
 32%|███▏      | 72050/225000 [144:37:11<191:36:32,  4.51s/it]
                                                              
{'loss': '0.8153', 'grad_norm': '2.451', 'learning_rate': '4.042e-05', 'epoch': '0.3202', 'num_input_tokens_seen': 239603664, 'train_runtime': '5.206e+05', 'train_tokens_per_second': '460.2'}

 32%|███▏      | 72050/225000 [144:37:11<191:36:32,  4.51s/it]
 32%|███▏      | 72051/225000 [144:37:14<177:50:09,  4.19s/it]
 32%|███▏      | 72052/225000 [144:37:22<216:40:31,  5.10s/it]
 32%|███▏      | 72053/225000 [144:37:25<192:58:58,  4.54s/it]
 32%|███▏      | 72054/225000 [144:37:29<188:48:04,  4.44s/it]
 32%|███▏      | 72055/225000 [144:37:32<167:33:15,  3.94s/it]
 32%|███▏      | 72056/225000 [144:37:36<166:00:45,  3.91s/it]
 32%|███▏      | 72057/225000 [144:37:43<205:57:57,  4.85s/it]
 32%|███▏      | 72058/225000 [144:37:46<187:07:24,  4.40s/it]
 32%|███▏      | 72059/225000 [144:37:50<173:54:04,  4.09s/it]
 32%|███▏      | 72060/225000 [144:37:53<162:44:42,  3.83s/it]
 32%|███▏      | 72061/225000 [144:37:57<165:10:30,  3.89s/it]
 32%|███▏      | 72062/225000 [144:38:01<163:45:24,  3.85s/it]
 32%|███▏      | 72063/225000 [144:38:07<197:42:47,  4.65s/it]
 32%|███▏      | 72064/225000 [144:38:16<252:16:11,  5.94s/it]
 32%|███▏      | 72065/225000 [144:38:22<247:57:33,  5.84s/it]
 32%|███▏      | 72066/225000 [144:38:26<224:28:53,  5.28s/it]
 32%|███▏      | 72067/225000 [144:38:30<214:18:42,  5.04s/it]
 32%|███▏      | 72068/225000 [144:38:34<193:47:33,  4.56s/it]
 32%|███▏      | 72069/225000 [144:38:37<177:01:57,  4.17s/it]
 32%|███▏      | 72070/225000 [144:38:42<189:55:50,  4.47s/it]
 32%|███▏      | 72071/225000 [144:38:45<175:43:38,  4.14s/it]
 32%|███▏      | 72072/225000 [144:38:48<162:40:13,  3.83s/it]
 32%|███▏      | 72073/225000 [144:38:53<166:20:26,  3.92s/it]
 32%|███▏      | 72074/225000 [144:38:56<159:18:15,  3.75s/it]
 32%|███▏      | 72075/225000 [144:38:59<152:28:29,  3.59s/it]
 32%|███▏      | 72076/225000 [144:39:07<207:10:33,  4.88s/it]
 32%|███▏      | 72077/225000 [144:39:13<221:56:22,  5.22s/it]
 32%|███▏      | 72078/225000 [144:39:18<213:18:17,  5.02s/it]
 32%|███▏      | 72079/225000 [144:39:21<194:41:24,  4.58s/it]
 32%|███▏      | 72080/225000 [144:39:24<174:29:34,  4.11s/it]
 32%|███▏      | 72081/225000 [144:39:28<167:44:44,  3.95s/it]
 32%|███▏      | 72082/225000 [144:39:31<164:27:59,  3.87s/it]
 32%|███▏      | 72083/225000 [144:39:41<232:01:15,  5.46s/it]
 32%|███▏      | 72084/225000 [144:39:46<237:21:59,  5.59s/it]
 32%|███▏      | 72085/225000 [144:39:52<242:27:56,  5.71s/it]
 32%|███▏      | 72086/225000 [144:39:56<213:44:19,  5.03s/it]
 32%|███▏      | 72087/225000 [144:39:59<190:55:28,  4.49s/it]
 32%|███▏      | 72088/225000 [144:40:02<173:47:12,  4.09s/it]
 32%|███▏      | 72089/225000 [144:40:05<160:56:56,  3.79s/it]
 32%|███▏      | 72090/225000 [144:40:16<243:57:37,  5.74s/it]
 32%|███▏      | 72091/225000 [144:40:19<209:27:09,  4.93s/it]
 32%|███▏      | 72092/225000 [144:40:22<189:22:29,  4.46s/it]
 32%|███▏      | 72093/225000 [144:40:25<176:04:37,  4.15s/it]
 32%|███▏      | 72094/225000 [144:40:33<220:56:45,  5.20s/it]
 32%|███▏      | 72095/225000 [144:40:37<197:32:35,  4.65s/it]
 32%|███▏      | 72096/225000 [144:40:40<185:56:15,  4.38s/it]
 32%|███▏      | 72097/225000 [144:40:44<182:17:53,  4.29s/it]
 32%|███▏      | 72098/225000 [144:40:48<173:42:11,  4.09s/it]
 32%|███▏      | 72099/225000 [144:40:57<237:30:49,  5.59s/it]
 32%|███▏      | 72100/225000 [144:41:01<209:53:20,  4.94s/it]
                                                              
{'loss': '0.7986', 'grad_norm': '2.31', 'learning_rate': '4.04e-05', 'epoch': '0.3204', 'num_input_tokens_seen': 239700032, 'train_runtime': '5.209e+05', 'train_tokens_per_second': '460.2'}

 32%|███▏      | 72100/225000 [144:41:01<209:53:20,  4.94s/it]
 32%|███▏      | 72101/225000 [144:41:04<187:13:24,  4.41s/it]
 32%|███▏      | 72102/225000 [144:41:07<170:14:47,  4.01s/it]
 32%|███▏      | 72103/225000 [144:41:10<157:35:36,  3.71s/it]
 32%|███▏      | 72104/225000 [144:41:14<158:02:56,  3.72s/it]
 32%|███▏      | 72105/225000 [144:41:22<221:06:58,  5.21s/it]
 32%|███▏      | 72106/225000 [144:41:30<249:02:03,  5.86s/it]
 32%|███▏      | 72107/225000 [144:41:32<205:31:56,  4.84s/it]
 32%|███▏      | 72108/225000 [144:41:41<254:36:26,  5.99s/it]
 32%|███▏      | 72109/225000 [144:41:43<210:22:05,  4.95s/it]
 32%|███▏      | 72110/225000 [144:41:46<178:52:21,  4.21s/it]
 32%|███▏      | 72111/225000 [144:41:53<211:46:25,  4.99s/it]
 32%|███▏      | 72112/225000 [144:41:56<191:28:45,  4.51s/it]
 32%|███▏      | 72113/225000 [144:41:59<173:27:09,  4.08s/it]
 32%|███▏      | 72114/225000 [144:42:02<160:40:03,  3.78s/it]
 32%|███▏      | 72115/225000 [144:42:06<168:20:32,  3.96s/it]
 32%|███▏      | 72116/225000 [144:42:10<162:00:33,  3.81s/it]
 32%|███▏      | 72117/225000 [144:42:13<158:37:53,  3.74s/it]
 32%|███▏      | 72118/225000 [144:42:17<151:23:01,  3.56s/it]
 32%|███▏      | 72119/225000 [144:42:27<234:35:59,  5.52s/it]
 32%|███▏      | 72120/225000 [144:42:34<261:16:31,  6.15s/it]
 32%|███▏      | 72121/225000 [144:42:39<235:38:04,  5.55s/it]
 32%|███▏      | 72122/225000 [144:42:42<208:36:05,  4.91s/it]
 32%|███▏      | 72123/225000 [144:42:45<188:46:38,  4.45s/it]
 32%|███▏      | 72124/225000 [144:42:49<181:00:30,  4.26s/it]
 32%|███▏      | 72125/225000 [144:42:53<172:58:59,  4.07s/it]
 32%|███▏      | 72126/225000 [144:42:56<159:34:30,  3.76s/it]
 32%|███▏      | 72127/225000 [144:42:59<153:48:14,  3.62s/it]
 32%|███▏      | 72128/225000 [144:43:04<165:38:06,  3.90s/it]
 32%|███▏      | 72129/225000 [144:43:13<235:11:19,  5.54s/it]
 32%|███▏      | 72130/225000 [144:43:16<207:18:45,  4.88s/it]
 32%|███▏      | 72131/225000 [144:43:20<193:05:33,  4.55s/it]
 32%|███▏      | 72132/225000 [144:43:24<181:42:14,  4.28s/it]
 32%|███▏      | 72133/225000 [144:43:27<173:00:18,  4.07s/it]
 32%|███▏      | 72134/225000 [144:43:31<168:36:40,  3.97s/it]
 32%|███▏      | 72135/225000 [144:43:36<177:32:19,  4.18s/it]
 32%|███▏      | 72136/225000 [144:43:39<163:44:21,  3.86s/it]
 32%|███▏      | 72137/225000 [144:43:48<231:00:47,  5.44s/it]
 32%|███▏      | 72138/225000 [144:43:51<201:32:57,  4.75s/it]
 32%|███▏      | 72139/225000 [144:43:55<184:07:51,  4.34s/it]
 32%|███▏      | 72140/225000 [144:43:58<178:53:34,  4.21s/it]
 32%|███▏      | 72141/225000 [144:44:01<163:38:03,  3.85s/it]
 32%|███▏      | 72142/225000 [144:44:12<253:54:35,  5.98s/it]
 32%|███▏      | 72143/225000 [144:44:15<210:24:37,  4.96s/it]
 32%|███▏      | 72144/225000 [144:44:18<183:50:32,  4.33s/it]
 32%|███▏      | 72145/225000 [144:44:20<162:14:57,  3.82s/it]
 32%|███▏      | 72146/225000 [144:44:24<159:19:34,  3.75s/it]
 32%|███▏      | 72147/225000 [144:44:27<148:33:56,  3.50s/it]
 32%|███▏      | 72148/225000 [144:44:36<219:16:52,  5.16s/it]
 32%|███▏      | 72149/225000 [144:44:43<242:48:33,  5.72s/it]
 32%|███▏      | 72150/225000 [144:44:47<225:04:02,  5.30s/it]
                                                              
{'loss': '0.8318', 'grad_norm': '1.365', 'learning_rate': '4.039e-05', 'epoch': '0.3207', 'num_input_tokens_seen': 239794568, 'train_runtime': '5.211e+05', 'train_tokens_per_second': '460.2'}

 32%|███▏      | 72150/225000 [144:44:47<225:04:02,  5.30s/it]
 32%|███▏      | 72151/225000 [144:44:53<230:04:05,  5.42s/it]
 32%|███▏      | 72152/225000 [144:44:56<193:13:23,  4.55s/it]
 32%|███▏      | 72153/225000 [144:44:58<169:38:14,  4.00s/it]
 32%|███▏      | 72154/225000 [144:45:06<214:36:14,  5.05s/it]
 32%|███▏      | 72155/225000 [144:45:09<187:36:18,  4.42s/it]
 32%|███▏      | 72156/225000 [144:45:12<171:59:02,  4.05s/it]
 32%|███▏      | 72157/225000 [144:45:15<159:47:15,  3.76s/it]
 32%|███▏      | 72158/225000 [144:45:18<154:51:27,  3.65s/it]
 32%|███▏      | 72159/225000 [144:45:22<153:42:04,  3.62s/it]
 32%|███▏      | 72160/225000 [144:45:25<150:44:22,  3.55s/it]
 32%|███▏      | 72161/225000 [144:45:34<210:40:20,  4.96s/it]
 32%|███▏      | 72162/225000 [144:45:41<237:28:59,  5.59s/it]
 32%|███▏      | 72163/225000 [144:45:45<223:35:17,  5.27s/it]
 32%|███▏      | 72164/225000 [144:45:51<232:19:07,  5.47s/it]
 32%|███▏      | 72165/225000 [144:45:55<209:42:52,  4.94s/it]
 32%|███▏      | 72166/225000 [144:45:58<186:17:43,  4.39s/it]
 32%|███▏      | 72167/225000 [144:46:01<174:08:17,  4.10s/it]
 32%|███▏      | 72168/225000 [144:46:04<162:01:17,  3.82s/it]
 32%|███▏      | 72169/225000 [144:46:08<157:07:04,  3.70s/it]
 32%|███▏      | 72170/225000 [144:46:12<166:05:07,  3.91s/it]
 32%|███▏      | 72171/225000 [144:46:16<159:20:22,  3.75s/it]
 32%|███▏      | 72172/225000 [144:46:19<159:33:22,  3.76s/it]
 32%|███▏      | 72173/225000 [144:46:23<161:04:22,  3.79s/it]
 32%|███▏      | 72174/225000 [144:46:27<160:26:56,  3.78s/it]
 32%|███▏      | 72175/225000 [144:46:34<200:48:45,  4.73s/it]
 32%|███▏      | 72176/225000 [144:46:37<182:09:43,  4.29s/it]
 32%|███▏      | 72177/225000 [144:46:41<172:57:31,  4.07s/it]
 32%|███▏      | 72178/225000 [144:46:45<167:53:46,  3.96s/it]
 32%|███▏      | 72179/225000 [144:46:49<178:08:53,  4.20s/it]
 32%|███▏      | 72180/225000 [144:46:56<216:00:04,  5.09s/it]
 32%|███▏      | 72181/225000 [144:47:00<198:02:49,  4.67s/it]
 32%|███▏      | 72182/225000 [144:47:03<177:22:06,  4.18s/it]
 32%|███▏      | 72183/225000 [144:47:07<170:03:42,  4.01s/it]
 32%|███▏      | 72184/225000 [144:47:10<161:19:28,  3.80s/it]
 32%|███▏      | 72185/225000 [144:47:14<162:22:38,  3.83s/it]
 32%|███▏      | 72186/225000 [144:47:24<240:36:46,  5.67s/it]
 32%|███▏      | 72187/225000 [144:47:30<250:43:22,  5.91s/it]
 32%|███▏      | 72188/225000 [144:47:34<221:52:15,  5.23s/it]
 32%|███▏      | 72189/225000 [144:47:37<196:02:40,  4.62s/it]
 32%|███▏      | 72190/225000 [144:47:41<179:12:11,  4.22s/it]
 32%|███▏      | 72191/225000 [144:47:44<174:16:14,  4.11s/it]
 32%|███▏      | 72192/225000 [144:47:48<162:03:49,  3.82s/it]
 32%|███▏      | 72193/225000 [144:47:54<195:22:18,  4.60s/it]
 32%|███▏      | 72194/225000 [144:48:03<256:27:28,  6.04s/it]
 32%|███▏      | 72195/225000 [144:48:08<238:30:02,  5.62s/it]
 32%|███▏      | 72196/225000 [144:48:11<206:55:45,  4.88s/it]
 32%|███▏      | 72197/225000 [144:48:14<178:24:03,  4.20s/it]
 32%|███▏      | 72198/225000 [144:48:17<167:32:01,  3.95s/it]
 32%|███▏      | 72199/225000 [144:48:24<207:48:47,  4.90s/it]
 32%|███▏      | 72200/225000 [144:48:31<231:10:04,  5.45s/it]
                                                              
{'loss': '0.8129', 'grad_norm': '2.02', 'learning_rate': '4.038e-05', 'epoch': '0.3209', 'num_input_tokens_seen': 239888592, 'train_runtime': '5.213e+05', 'train_tokens_per_second': '460.2'}

 32%|███▏      | 72200/225000 [144:48:31<231:10:04,  5.45s/it]
 32%|███▏      | 72201/225000 [144:48:34<194:34:36,  4.58s/it]
 32%|███▏      | 72202/225000 [144:48:37<177:52:33,  4.19s/it]
 32%|███▏      | 72203/225000 [144:48:40<159:14:21,  3.75s/it]
 32%|███▏      | 72204/225000 [144:48:48<213:58:28,  5.04s/it]
 32%|███▏      | 72205/225000 [144:48:51<198:50:21,  4.68s/it]
 32%|███▏      | 72206/225000 [144:48:55<187:13:58,  4.41s/it]
 32%|███▏      | 72207/225000 [144:48:59<183:07:38,  4.31s/it]
 32%|███▏      | 72208/225000 [144:49:03<170:08:40,  4.01s/it]
 32%|███▏      | 72209/225000 [144:49:07<169:36:20,  4.00s/it]
 32%|███▏      | 72210/225000 [144:49:11<168:54:14,  3.98s/it]
 32%|███▏      | 72211/225000 [144:49:19<230:33:45,  5.43s/it]
 32%|███▏      | 72212/225000 [144:49:25<237:17:25,  5.59s/it]
 32%|███▏      | 72213/225000 [144:49:29<210:46:49,  4.97s/it]
 32%|███▏      | 72214/225000 [144:49:32<184:54:56,  4.36s/it]
 32%|███▏      | 72215/225000 [144:49:36<179:21:22,  4.23s/it]
 32%|███▏      | 72216/225000 [144:49:39<167:09:03,  3.94s/it]
 32%|███▏      | 72217/225000 [144:49:42<157:44:50,  3.72s/it]
 32%|███▏      | 72218/225000 [144:49:51<221:52:45,  5.23s/it]
 32%|███▏      | 72219/225000 [144:49:59<261:59:30,  6.17s/it]
 32%|███▏      | 72220/225000 [144:50:02<217:07:22,  5.12s/it]
 32%|███▏      | 72221/225000 [144:50:06<198:46:17,  4.68s/it]
 32%|███▏      | 72222/225000 [144:50:12<225:00:16,  5.30s/it]
 32%|███▏      | 72223/225000 [144:50:16<197:37:05,  4.66s/it]
 32%|███▏      | 72224/225000 [144:50:19<183:21:57,  4.32s/it]
 32%|███▏      | 72225/225000 [144:50:22<166:09:01,  3.92s/it]
 32%|███▏      | 72226/225000 [144:50:30<213:34:23,  5.03s/it]
 32%|███▏      | 72227/225000 [144:50:35<211:55:46,  4.99s/it]
 32%|███▏      | 72228/225000 [144:50:38<187:12:14,  4.41s/it]
 32%|███▏      | 72229/225000 [144:50:43<193:24:16,  4.56s/it]
 32%|███▏      | 72230/225000 [144:50:47<190:44:01,  4.49s/it]
 32%|███▏      | 72231/225000 [144:50:55<232:09:22,  5.47s/it]
 32%|███▏      | 72232/225000 [144:50:58<206:42:46,  4.87s/it]
 32%|███▏      | 72233/225000 [144:51:02<198:55:04,  4.69s/it]
 32%|███▏      | 72234/225000 [144:51:09<229:55:46,  5.42s/it]
 32%|███▏      | 72235/225000 [144:51:14<222:01:42,  5.23s/it]
 32%|███▏      | 72236/225000 [144:51:17<195:03:53,  4.60s/it]
 32%|███▏      | 72237/225000 [144:51:21<179:05:32,  4.22s/it]
 32%|███▏      | 72238/225000 [144:51:25<182:24:57,  4.30s/it]
 32%|███▏      | 72239/225000 [144:51:31<204:14:37,  4.81s/it]
 32%|███▏      | 72240/225000 [144:51:34<181:48:03,  4.28s/it]
 32%|███▏      | 72241/225000 [144:51:37<168:15:14,  3.97s/it]
 32%|███▏      | 72242/225000 [144:51:40<155:22:11,  3.66s/it]
 32%|███▏      | 72243/225000 [144:51:43<145:37:12,  3.43s/it]
 32%|███▏      | 72244/225000 [144:51:49<172:02:46,  4.05s/it]
 32%|███▏      | 72245/225000 [144:51:52<166:01:56,  3.91s/it]
 32%|███▏      | 72246/225000 [144:51:56<160:36:56,  3.79s/it]
 32%|███▏      | 72247/225000 [144:52:00<158:21:44,  3.73s/it]
 32%|███▏      | 72248/225000 [144:52:03<152:04:46,  3.58s/it]
 32%|███▏      | 72249/225000 [144:52:13<232:34:01,  5.48s/it]
 32%|███▏      | 72250/225000 [144:52:17<223:26:57,  5.27s/it]
                                                              
{'loss': '0.8047', 'grad_norm': '1.439', 'learning_rate': '4.036e-05', 'epoch': '0.3211', 'num_input_tokens_seen': 239984208, 'train_runtime': '5.215e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72250/225000 [144:52:17<223:26:57,  5.27s/it]
 32%|███▏      | 72251/225000 [144:52:21<199:21:11,  4.70s/it]
 32%|███▏      | 72252/225000 [144:52:24<182:04:59,  4.29s/it]
 32%|███▏      | 72253/225000 [144:52:29<184:25:06,  4.35s/it]
 32%|███▏      | 72254/225000 [144:52:36<220:17:13,  5.19s/it]
 32%|███▏      | 72255/225000 [144:52:43<250:18:12,  5.90s/it]
 32%|███▏      | 72256/225000 [144:52:46<205:45:56,  4.85s/it]
 32%|███▏      | 72257/225000 [144:52:49<184:32:57,  4.35s/it]
 32%|███▏      | 72258/225000 [144:52:53<175:52:19,  4.15s/it]
 32%|███▏      | 72259/225000 [144:52:56<163:30:14,  3.85s/it]
 32%|███▏      | 72260/225000 [144:53:02<193:54:47,  4.57s/it]
 32%|███▏      | 72261/225000 [144:53:11<247:43:15,  5.84s/it]
 32%|███▏      | 72262/225000 [144:53:13<207:08:48,  4.88s/it]
 32%|███▏      | 72263/225000 [144:53:17<185:49:22,  4.38s/it]
 32%|███▏      | 72264/225000 [144:53:21<179:08:10,  4.22s/it]
 32%|███▏      | 72265/225000 [144:53:24<173:26:11,  4.09s/it]
 32%|███▏      | 72266/225000 [144:53:31<205:08:55,  4.84s/it]
 32%|███▏      | 72267/225000 [144:53:33<176:32:24,  4.16s/it]
 32%|███▏      | 72268/225000 [144:53:37<164:07:20,  3.87s/it]
 32%|███▏      | 72269/225000 [144:53:44<203:07:51,  4.79s/it]
 32%|███▏      | 72270/225000 [144:53:49<207:39:55,  4.89s/it]
 32%|███▏      | 72271/225000 [144:53:53<197:46:49,  4.66s/it]
 32%|███▏      | 72272/225000 [144:53:57<186:27:05,  4.39s/it]
 32%|███▏      | 72273/225000 [144:54:01<182:23:47,  4.30s/it]
 32%|███▏      | 72274/225000 [144:54:04<169:20:11,  3.99s/it]
 32%|███▏      | 72275/225000 [144:54:08<166:56:17,  3.94s/it]
 32%|███▏      | 72276/225000 [144:54:11<160:54:14,  3.79s/it]
 32%|███▏      | 72277/225000 [144:54:14<153:57:11,  3.63s/it]
 32%|███▏      | 72278/225000 [144:54:18<152:49:52,  3.60s/it]
 32%|███▏      | 72279/225000 [144:54:22<153:36:31,  3.62s/it]
 32%|███▏      | 72280/225000 [144:54:25<147:40:17,  3.48s/it]
 32%|███▏      | 72281/225000 [144:54:29<155:09:26,  3.66s/it]
 32%|███▏      | 72282/225000 [144:54:34<179:26:51,  4.23s/it]
 32%|███▏      | 72283/225000 [144:54:43<235:35:56,  5.55s/it]
 32%|███▏      | 72284/225000 [144:54:52<276:38:08,  6.52s/it]
 32%|███▏      | 72285/225000 [144:54:55<229:41:23,  5.41s/it]
 32%|███▏      | 72286/225000 [144:54:58<201:34:55,  4.75s/it]
 32%|███▏      | 72287/225000 [144:55:01<181:50:58,  4.29s/it]
 32%|███▏      | 72288/225000 [144:55:10<242:07:06,  5.71s/it]
 32%|███▏      | 72289/225000 [144:55:15<227:16:55,  5.36s/it]
 32%|███▏      | 72290/225000 [144:55:19<212:15:08,  5.00s/it]
 32%|███▏      | 72291/225000 [144:55:23<196:10:27,  4.62s/it]
 32%|███▏      | 72292/225000 [144:55:26<182:47:26,  4.31s/it]
 32%|███▏      | 72293/225000 [144:55:30<174:29:15,  4.11s/it]
 32%|███▏      | 72294/225000 [144:55:34<169:26:12,  3.99s/it]
 32%|███▏      | 72295/225000 [144:55:37<160:26:22,  3.78s/it]
 32%|███▏      | 72296/225000 [144:55:40<157:44:50,  3.72s/it]
 32%|███▏      | 72297/225000 [144:55:46<177:38:39,  4.19s/it]
 32%|███▏      | 72298/225000 [144:55:52<200:58:19,  4.74s/it]
 32%|███▏      | 72299/225000 [144:56:03<281:31:24,  6.64s/it]
 32%|███▏      | 72300/225000 [144:56:09<274:21:57,  6.47s/it]
                                                              
{'loss': '0.8087', 'grad_norm': '1.504', 'learning_rate': '4.035e-05', 'epoch': '0.3213', 'num_input_tokens_seen': 240079808, 'train_runtime': '5.218e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72300/225000 [144:56:09<274:21:57,  6.47s/it]
 32%|███▏      | 72301/225000 [144:56:12<232:17:35,  5.48s/it]
 32%|███▏      | 72302/225000 [144:56:15<204:48:54,  4.83s/it]
 32%|███▏      | 72303/225000 [144:56:18<181:43:37,  4.28s/it]
 32%|███▏      | 72304/225000 [144:56:22<169:38:09,  4.00s/it]
 32%|███▏      | 72305/225000 [144:56:25<163:07:21,  3.85s/it]
 32%|███▏      | 72306/225000 [144:56:35<243:40:55,  5.75s/it]
 32%|███▏      | 72307/225000 [144:56:40<230:57:40,  5.45s/it]
 32%|███▏      | 72308/225000 [144:56:43<195:52:28,  4.62s/it]
 32%|███▏      | 72309/225000 [144:56:46<173:57:01,  4.10s/it]
 32%|███▏      | 72310/225000 [144:56:50<171:45:39,  4.05s/it]
 32%|███▏      | 72311/225000 [144:56:58<227:47:23,  5.37s/it]
 32%|███▏      | 72312/225000 [144:57:02<203:19:01,  4.79s/it]
 32%|███▏      | 72313/225000 [144:57:05<186:17:24,  4.39s/it]
 32%|███▏      | 72314/225000 [144:57:14<250:30:57,  5.91s/it]
 32%|███▏      | 72315/225000 [144:57:18<222:43:19,  5.25s/it]
 32%|███▏      | 72316/225000 [144:57:23<213:29:20,  5.03s/it]
 32%|███▏      | 72317/225000 [144:57:26<195:37:17,  4.61s/it]
 32%|███▏      | 72318/225000 [144:57:31<193:47:01,  4.57s/it]
 32%|███▏      | 72319/225000 [144:57:38<223:04:57,  5.26s/it]
 32%|███▏      | 72320/225000 [144:57:41<198:09:31,  4.67s/it]
 32%|███▏      | 72321/225000 [144:57:44<180:07:22,  4.25s/it]
 32%|███▏      | 72322/225000 [144:57:47<158:14:35,  3.73s/it]
 32%|███▏      | 72323/225000 [144:57:49<142:53:07,  3.37s/it]
 32%|███▏      | 72324/225000 [144:57:53<145:19:14,  3.43s/it]
 32%|███▏      | 72325/225000 [144:57:56<137:03:38,  3.23s/it]
 32%|███▏      | 72326/225000 [144:57:59<134:57:51,  3.18s/it]
 32%|███▏      | 72327/225000 [144:58:05<179:36:04,  4.23s/it]
 32%|███▏      | 72328/225000 [144:58:16<262:51:27,  6.20s/it]
 32%|███▏      | 72329/225000 [144:58:19<221:40:06,  5.23s/it]
 32%|███▏      | 72330/225000 [144:58:24<214:00:26,  5.05s/it]
 32%|███▏      | 72331/225000 [144:58:28<206:06:51,  4.86s/it]
 32%|███▏      | 72332/225000 [144:58:31<179:06:46,  4.22s/it]
 32%|███▏      | 72333/225000 [144:58:35<172:21:29,  4.06s/it]
 32%|███▏      | 72334/225000 [144:58:41<198:08:48,  4.67s/it]
 32%|███▏      | 72335/225000 [144:58:44<186:06:07,  4.39s/it]
 32%|███▏      | 72336/225000 [144:58:48<179:54:33,  4.24s/it]
 32%|███▏      | 72337/225000 [144:58:51<161:58:35,  3.82s/it]
 32%|███▏      | 72338/225000 [144:58:55<163:16:42,  3.85s/it]
 32%|███▏      | 72339/225000 [144:58:58<154:59:14,  3.65s/it]
 32%|███▏      | 72340/225000 [144:59:03<163:45:14,  3.86s/it]
 32%|███▏      | 72341/225000 [144:59:07<165:06:40,  3.89s/it]
 32%|███▏      | 72342/225000 [144:59:11<167:56:59,  3.96s/it]
 32%|███▏      | 72343/225000 [144:59:19<222:06:14,  5.24s/it]
 32%|███▏      | 72344/225000 [144:59:27<256:27:25,  6.05s/it]
 32%|███▏      | 72345/225000 [144:59:30<215:53:24,  5.09s/it]
 32%|███▏      | 72346/225000 [144:59:35<214:59:49,  5.07s/it]
 32%|███▏      | 72347/225000 [144:59:39<199:23:59,  4.70s/it]
 32%|███▏      | 72348/225000 [144:59:42<180:05:56,  4.25s/it]
 32%|███▏      | 72349/225000 [144:59:46<183:27:39,  4.33s/it]
 32%|███▏      | 72350/225000 [144:59:50<173:39:56,  4.10s/it]
                                                              
{'loss': '0.8202', 'grad_norm': '1.586', 'learning_rate': '4.033e-05', 'epoch': '0.3216', 'num_input_tokens_seen': 240175104, 'train_runtime': '5.22e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72350/225000 [144:59:50<173:39:56,  4.10s/it]
 32%|███▏      | 72351/225000 [144:59:53<161:33:00,  3.81s/it]
 32%|███▏      | 72352/225000 [144:59:59<189:19:02,  4.46s/it]
 32%|███▏      | 72353/225000 [145:00:07<239:42:38,  5.65s/it]
 32%|███▏      | 72354/225000 [145:00:11<209:47:34,  4.95s/it]
 32%|███▏      | 72355/225000 [145:00:16<210:07:29,  4.96s/it]
 32%|███▏      | 72356/225000 [145:00:22<230:11:32,  5.43s/it]
 32%|███▏      | 72357/225000 [145:00:26<204:05:11,  4.81s/it]
 32%|███▏      | 72358/225000 [145:00:33<234:16:29,  5.53s/it]
 32%|███▏      | 72359/225000 [145:00:35<196:26:41,  4.63s/it]
 32%|███▏      | 72360/225000 [145:00:42<227:19:58,  5.36s/it]
 32%|███▏      | 72361/225000 [145:00:45<188:22:29,  4.44s/it]
 32%|███▏      | 72362/225000 [145:00:47<168:15:14,  3.97s/it]
 32%|███▏      | 72363/225000 [145:00:52<169:39:30,  4.00s/it]
 32%|███▏      | 72364/225000 [145:00:57<190:52:58,  4.50s/it]
 32%|███▏      | 72365/225000 [145:01:01<176:19:12,  4.16s/it]
 32%|███▏      | 72366/225000 [145:01:04<169:58:16,  4.01s/it]
 32%|███▏      | 72367/225000 [145:01:08<168:54:14,  3.98s/it]
 32%|███▏      | 72368/225000 [145:01:12<164:01:40,  3.87s/it]
 32%|███▏      | 72369/225000 [145:01:19<206:52:45,  4.88s/it]
 32%|███▏      | 72370/225000 [145:01:24<211:35:29,  4.99s/it]
 32%|███▏      | 72371/225000 [145:01:28<194:27:25,  4.59s/it]
 32%|███▏      | 72372/225000 [145:01:32<190:37:42,  4.50s/it]
 32%|███▏      | 72373/225000 [145:01:36<186:31:54,  4.40s/it]
 32%|███▏      | 72374/225000 [145:01:40<173:30:39,  4.09s/it]
 32%|███▏      | 72375/225000 [145:01:44<172:45:07,  4.07s/it]
 32%|███▏      | 72376/225000 [145:01:49<187:36:59,  4.43s/it]
 32%|███▏      | 72377/225000 [145:01:53<188:09:22,  4.44s/it]
 32%|███▏      | 72378/225000 [145:02:02<235:17:28,  5.55s/it]
 32%|███▏      | 72379/225000 [145:02:06<216:02:47,  5.10s/it]
 32%|███▏      | 72380/225000 [145:02:09<197:31:37,  4.66s/it]
 32%|███▏      | 72381/225000 [145:02:12<176:47:38,  4.17s/it]
 32%|███▏      | 72382/225000 [145:02:16<166:52:57,  3.94s/it]
 32%|███▏      | 72383/225000 [145:02:19<154:52:47,  3.65s/it]
 32%|███▏      | 72384/225000 [145:02:24<173:50:24,  4.10s/it]
 32%|███▏      | 72385/225000 [145:02:33<232:10:03,  5.48s/it]
 32%|███▏      | 72386/225000 [145:02:36<212:23:40,  5.01s/it]
 32%|███▏      | 72387/225000 [145:02:42<218:20:52,  5.15s/it]
 32%|███▏      | 72388/225000 [145:02:46<209:17:29,  4.94s/it]
 32%|███▏      | 72389/225000 [145:02:50<195:34:05,  4.61s/it]
 32%|███▏      | 72390/225000 [145:02:55<202:25:49,  4.78s/it]
 32%|███▏      | 72391/225000 [145:03:02<226:31:17,  5.34s/it]
 32%|███▏      | 72392/225000 [145:03:07<225:50:28,  5.33s/it]
 32%|███▏      | 72393/225000 [145:03:11<199:06:08,  4.70s/it]
 32%|███▏      | 72394/225000 [145:03:17<225:00:31,  5.31s/it]
 32%|███▏      | 72395/225000 [145:03:20<196:33:05,  4.64s/it]
 32%|███▏      | 72396/225000 [145:03:25<201:29:44,  4.75s/it]
 32%|███▏      | 72397/225000 [145:03:30<204:20:10,  4.82s/it]
 32%|███▏      | 72398/225000 [145:03:34<186:36:58,  4.40s/it]
 32%|███▏      | 72399/225000 [145:03:38<189:40:27,  4.47s/it]
 32%|███▏      | 72400/225000 [145:03:43<184:39:09,  4.36s/it]
                                                              
{'loss': '0.8184', 'grad_norm': '1.308', 'learning_rate': '4.032e-05', 'epoch': '0.3218', 'num_input_tokens_seen': 240270592, 'train_runtime': '5.222e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72400/225000 [145:03:43<184:39:09,  4.36s/it]
 32%|███▏      | 72401/225000 [145:03:47<189:06:33,  4.46s/it]
 32%|███▏      | 72402/225000 [145:03:51<181:19:53,  4.28s/it]
 32%|███▏      | 72403/225000 [145:03:55<182:01:31,  4.29s/it]
 32%|███▏      | 72404/225000 [145:03:59<177:11:52,  4.18s/it]
 32%|███▏      | 72405/225000 [145:04:03<171:20:13,  4.04s/it]
 32%|███▏      | 72406/225000 [145:04:11<214:55:57,  5.07s/it]
 32%|███▏      | 72407/225000 [145:04:19<256:18:47,  6.05s/it]
 32%|███▏      | 72408/225000 [145:04:22<216:43:32,  5.11s/it]
 32%|███▏      | 72409/225000 [145:04:25<193:01:50,  4.55s/it]
 32%|███▏      | 72410/225000 [145:04:34<246:00:14,  5.80s/it]
 32%|███▏      | 72411/225000 [145:04:39<235:43:09,  5.56s/it]
 32%|███▏      | 72412/225000 [145:04:43<216:31:56,  5.11s/it]
 32%|███▏      | 72413/225000 [145:04:47<200:20:21,  4.73s/it]
 32%|███▏      | 72414/225000 [145:04:52<211:03:51,  4.98s/it]
 32%|███▏      | 72415/225000 [145:04:58<221:17:54,  5.22s/it]
 32%|███▏      | 72416/225000 [145:05:01<199:15:34,  4.70s/it]
 32%|███▏      | 72417/225000 [145:05:05<181:39:01,  4.29s/it]
 32%|███▏      | 72418/225000 [145:05:09<176:59:25,  4.18s/it]
 32%|███▏      | 72419/225000 [145:05:13<172:28:59,  4.07s/it]
 32%|███▏      | 72420/225000 [145:05:23<250:03:32,  5.90s/it]
 32%|███▏      | 72421/225000 [145:05:26<218:34:38,  5.16s/it]
 32%|███▏      | 72422/225000 [145:05:30<199:23:10,  4.70s/it]
 32%|███▏      | 72423/225000 [145:05:34<198:40:31,  4.69s/it]
 32%|███▏      | 72424/225000 [145:05:38<179:15:10,  4.23s/it]
 32%|███▏      | 72425/225000 [145:05:41<170:22:56,  4.02s/it]
 32%|███▏      | 72426/225000 [145:05:45<163:33:42,  3.86s/it]
 32%|███▏      | 72427/225000 [145:05:48<156:08:14,  3.68s/it]
 32%|███▏      | 72428/225000 [145:05:58<234:04:58,  5.52s/it]
 32%|███▏      | 72429/225000 [145:06:01<211:33:33,  4.99s/it]
 32%|███▏      | 72430/225000 [145:06:04<183:22:26,  4.33s/it]
 32%|███▏      | 72431/225000 [145:06:10<198:31:28,  4.68s/it]
 32%|███▏      | 72432/225000 [145:06:13<180:39:11,  4.26s/it]
 32%|███▏      | 72433/225000 [145:06:17<177:51:48,  4.20s/it]
 32%|███▏      | 72434/225000 [145:06:20<164:31:59,  3.88s/it]
 32%|███▏      | 72435/225000 [145:06:24<158:28:39,  3.74s/it]
 32%|███▏      | 72436/225000 [145:06:32<217:02:35,  5.12s/it]
 32%|███▏      | 72437/225000 [145:06:35<191:03:31,  4.51s/it]
 32%|███▏      | 72438/225000 [145:06:39<180:32:33,  4.26s/it]
 32%|███▏      | 72439/225000 [145:06:42<167:07:49,  3.94s/it]
 32%|███▏      | 72440/225000 [145:06:45<161:00:26,  3.80s/it]
 32%|███▏      | 72441/225000 [145:06:56<245:10:32,  5.79s/it]
 32%|███▏      | 72442/225000 [145:07:00<219:48:42,  5.19s/it]
 32%|███▏      | 72443/225000 [145:07:03<193:04:47,  4.56s/it]
 32%|███▏      | 72444/225000 [145:07:06<177:59:50,  4.20s/it]
 32%|███▏      | 72445/225000 [145:07:10<172:44:34,  4.08s/it]
 32%|███▏      | 72446/225000 [145:07:14<174:06:27,  4.11s/it]
 32%|███▏      | 72447/225000 [145:07:18<169:01:23,  3.99s/it]
 32%|███▏      | 72448/225000 [145:07:22<167:54:46,  3.96s/it]
 32%|███▏      | 72449/225000 [145:07:26<174:43:32,  4.12s/it]
 32%|███▏      | 72450/225000 [145:07:29<163:37:43,  3.86s/it]
                                                              
{'loss': '0.8187', 'grad_norm': '3.118', 'learning_rate': '4.03e-05', 'epoch': '0.322', 'num_input_tokens_seen': 240366200, 'train_runtime': '5.224e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72450/225000 [145:07:29<163:37:43,  3.86s/it]
 32%|███▏      | 72451/225000 [145:07:33<160:38:47,  3.79s/it]
 32%|███▏      | 72452/225000 [145:07:36<148:55:20,  3.51s/it]
 32%|███▏      | 72453/225000 [145:07:39<146:06:50,  3.45s/it]
 32%|███▏      | 72454/225000 [145:07:46<188:20:42,  4.44s/it]
 32%|███▏      | 72455/225000 [145:07:56<256:37:02,  6.06s/it]
 32%|███▏      | 72456/225000 [145:07:59<219:08:54,  5.17s/it]
 32%|███▏      | 72457/225000 [145:08:01<186:14:42,  4.40s/it]
 32%|███▏      | 72458/225000 [145:08:05<169:47:28,  4.01s/it]
 32%|███▏      | 72459/225000 [145:08:11<196:41:48,  4.64s/it]
 32%|███▏      | 72460/225000 [145:08:14<184:33:12,  4.36s/it]
 32%|███▏      | 72461/225000 [145:08:20<196:02:26,  4.63s/it]
 32%|███▏      | 72462/225000 [145:08:22<171:45:48,  4.05s/it]
 32%|███▏      | 72463/225000 [145:08:25<158:33:39,  3.74s/it]
 32%|███▏      | 72464/225000 [145:08:28<147:30:32,  3.48s/it]
 32%|███▏      | 72465/225000 [145:08:38<230:06:21,  5.43s/it]
 32%|███▏      | 72466/225000 [145:08:42<208:28:28,  4.92s/it]
 32%|███▏      | 72467/225000 [145:08:47<212:52:46,  5.02s/it]
 32%|███▏      | 72468/225000 [145:08:52<205:21:03,  4.85s/it]
 32%|███▏      | 72469/225000 [145:08:55<186:56:15,  4.41s/it]
 32%|███▏      | 72470/225000 [145:08:58<167:28:32,  3.95s/it]
 32%|███▏      | 72471/225000 [145:09:01<159:59:07,  3.78s/it]
 32%|███▏      | 72472/225000 [145:09:04<149:47:27,  3.54s/it]
 32%|███▏      | 72473/225000 [145:09:08<147:08:54,  3.47s/it]
 32%|███▏      | 72474/225000 [145:09:11<140:24:23,  3.31s/it]
 32%|███▏      | 72475/225000 [145:09:14<138:04:46,  3.26s/it]
 32%|███▏      | 72476/225000 [145:09:21<193:25:24,  4.57s/it]
 32%|███▏      | 72477/225000 [145:09:31<258:51:02,  6.11s/it]
 32%|███▏      | 72478/225000 [145:09:33<209:53:19,  4.95s/it]
 32%|███▏      | 72479/225000 [145:09:36<178:37:54,  4.22s/it]
 32%|███▏      | 72480/225000 [145:09:42<204:36:24,  4.83s/it]
 32%|███▏      | 72481/225000 [145:09:47<203:52:46,  4.81s/it]
 32%|███▏      | 72482/225000 [145:09:50<179:18:12,  4.23s/it]
 32%|███▏      | 72483/225000 [145:09:52<157:37:35,  3.72s/it]
 32%|███▏      | 72484/225000 [145:09:55<149:27:27,  3.53s/it]
 32%|███▏      | 72485/225000 [145:10:01<181:51:28,  4.29s/it]
 32%|███▏      | 72486/225000 [145:10:10<242:21:24,  5.72s/it]
 32%|███▏      | 72487/225000 [145:10:13<206:22:00,  4.87s/it]
 32%|███▏      | 72488/225000 [145:10:17<186:36:46,  4.40s/it]
 32%|███▏      | 72489/225000 [145:10:19<160:20:01,  3.78s/it]
 32%|███▏      | 72490/225000 [145:10:27<214:00:03,  5.05s/it]
 32%|███▏      | 72491/225000 [145:10:31<195:39:10,  4.62s/it]
 32%|███▏      | 72492/225000 [145:10:36<206:36:00,  4.88s/it]
 32%|███▏      | 72493/225000 [145:10:40<199:22:56,  4.71s/it]
 32%|███▏      | 72494/225000 [145:10:43<176:40:35,  4.17s/it]
 32%|███▏      | 72495/225000 [145:10:51<226:57:07,  5.36s/it]
 32%|███▏      | 72496/225000 [145:10:56<213:58:14,  5.05s/it]
 32%|███▏      | 72497/225000 [145:10:59<195:06:12,  4.61s/it]
 32%|███▏      | 72498/225000 [145:11:03<184:35:22,  4.36s/it]
 32%|███▏      | 72499/225000 [145:11:07<173:08:51,  4.09s/it]
 32%|███▏      | 72500/225000 [145:11:16<243:00:30,  5.74s/it]
                                                              
{'loss': '0.7998', 'grad_norm': '1.065', 'learning_rate': '4.029e-05', 'epoch': '0.3222', 'num_input_tokens_seen': 240457952, 'train_runtime': '5.227e+05', 'train_tokens_per_second': '460.1'}

 32%|███▏      | 72500/225000 [145:11:16<243:00:30,  5.74s/it]
 32%|███▏      | 72501/225000 [145:11:19<202:37:38,  4.78s/it]
 32%|███▏      | 72502/225000 [145:11:21<174:43:42,  4.12s/it]
 32%|███▏      | 72503/225000 [145:11:25<172:44:32,  4.08s/it]
 32%|███▏      | 72504/225000 [145:11:34<229:22:28,  5.41s/it]
 32%|███▏      | 72505/225000 [145:11:36<191:00:27,  4.51s/it]
 32%|███▏      | 72506/225000 [145:11:42<203:44:20,  4.81s/it]
 32%|███▏      | 72507/225000 [145:11:47<206:52:31,  4.88s/it]
 32%|███▏      | 72508/225000 [145:11:50<184:50:48,  4.36s/it]
 32%|███▏      | 72509/225000 [145:11:53<169:52:50,  4.01s/it]
 32%|███▏      | 72510/225000 [145:11:56<160:18:17,  3.78s/it]
 32%|███▏      | 72511/225000 [145:12:00<153:04:19,  3.61s/it]
 32%|███▏      | 72512/225000 [145:12:04<159:02:06,  3.75s/it]
 32%|███▏      | 72513/225000 [145:12:11<200:52:59,  4.74s/it]
 32%|███▏      | 72514/225000 [145:12:19<245:04:30,  5.79s/it]
 32%|███▏      | 72515/225000 [145:12:22<205:44:49,  4.86s/it]
 32%|███▏      | 72516/225000 [145:12:25<184:37:54,  4.36s/it]
 32%|███▏      | 72517/225000 [145:12:28<171:06:22,  4.04s/it]
 32%|███▏      | 72518/225000 [145:12:32<166:27:32,  3.93s/it]
 32%|███▏      | 72519/225000 [145:12:40<226:25:23,  5.35s/it]
 32%|███▏      | 72520/225000 [145:12:44<204:49:26,  4.84s/it]
 32%|███▏      | 72521/225000 [145:12:47<179:37:48,  4.24s/it]
 32%|███▏      | 72522/225000 [145:12:51<176:16:03,  4.16s/it]
 32%|███▏      | 72523/225000 [145:12:54<167:17:37,  3.95s/it]
 32%|███▏      | 72524/225000 [145:13:02<209:53:15,  4.96s/it]
 32%|███▏      | 72525/225000 [145:13:09<236:37:52,  5.59s/it]
 32%|███▏      | 72526/225000 [145:13:12<208:19:26,  4.92s/it]
 32%|███▏      | 72527/225000 [145:13:16<195:40:32,  4.62s/it]
 32%|███▏      | 72528/225000 [145:13:20<182:04:10,  4.30s/it]
 32%|███▏      | 72529/225000 [145:13:23<172:01:47,  4.06s/it]
 32%|███▏      | 72530/225000 [145:13:26<157:07:44,  3.71s/it]
 32%|███▏      | 72531/225000 [145:13:36<236:11:36,  5.58s/it]
 32%|███▏      | 72532/225000 [145:13:41<228:03:36,  5.38s/it]
 32%|███▏      | 72533/225000 [145:13:44<204:41:20,  4.83s/it]
 32%|███▏      | 72534/225000 [145:13:47<182:50:55,  4.32s/it]
 32%|███▏      | 72535/225000 [145:13:50<162:45:23,  3.84s/it]
 32%|███▏      | 72536/225000 [145:13:54<159:58:04,  3.78s/it]
 32%|███▏      | 72537/225000 [145:13:57<157:02:54,  3.71s/it]
 32%|███▏      | 72538/225000 [145:14:07<226:10:52,  5.34s/it]
 32%|███▏      | 72539/225000 [145:14:11<219:00:59,  5.17s/it]
 32%|███▏      | 72540/225000 [145:14:14<192:42:29,  4.55s/it]
 32%|███▏      | 72541/225000 [145:14:18<175:37:32,  4.15s/it]
 32%|███▏      | 72542/225000 [145:14:21<163:39:10,  3.86s/it]
 32%|███▏      | 72543/225000 [145:14:25<166:18:21,  3.93s/it]
 32%|███▏      | 72544/225000 [145:14:35<241:02:04,  5.69s/it]
 32%|███▏      | 72545/225000 [145:14:38<213:04:08,  5.03s/it]
 32%|███▏      | 72546/225000 [145:14:41<185:14:35,  4.37s/it]
 32%|███▏      | 72547/225000 [145:14:44<171:07:49,  4.04s/it]
 32%|███▏      | 72548/225000 [145:14:48<166:11:08,  3.92s/it]
 32%|███▏      | 72549/225000 [145:14:51<159:58:56,  3.78s/it]
 32%|███▏      | 72550/225000 [145:15:00<223:45:18,  5.28s/it]
                                                              
{'loss': '0.8383', 'grad_norm': '1.391', 'learning_rate': '4.028e-05', 'epoch': '0.3224', 'num_input_tokens_seen': 240551976, 'train_runtime': '5.229e+05', 'train_tokens_per_second': '460'}

 32%|███▏      | 72550/225000 [145:15:00<223:45:18,  5.28s/it]
 32%|███▏      | 72551/225000 [145:15:04<200:18:27,  4.73s/it]
 32%|███▏      | 72552/225000 [145:15:07<185:22:20,  4.38s/it]
 32%|███▏      | 72553/225000 [145:15:11<173:26:16,  4.10s/it]
 32%|███▏      | 72554/225000 [145:15:14<159:03:10,  3.76s/it]
 32%|███▏      | 72555/225000 [145:15:21<204:55:19,  4.84s/it]
 32%|███▏      | 72556/225000 [145:15:27<222:27:32,  5.25s/it]
 32%|███▏      | 72557/225000 [145:15:29<184:17:26,  4.35s/it]
 32%|███▏      | 72558/225000 [145:15:36<213:40:44,  5.05s/it]
 32%|███▏      | 72559/225000 [145:15:39<191:31:00,  4.52s/it]
 32%|███▏      | 72560/225000 [145:15:43<178:58:13,  4.23s/it]
 32%|███▏      | 72561/225000 [145:15:47<172:27:24,  4.07s/it]
 32%|███▏      | 72562/225000 [145:15:50<162:21:03,  3.83s/it]
 32%|███▏      | 72563/225000 [145:15:53<158:47:45,  3.75s/it]
 32%|███▏      | 72564/225000 [145:15:58<163:31:14,  3.86s/it]
 32%|███▏      | 72565/225000 [145:16:02<168:29:56,  3.98s/it]
 32%|███▏      | 72566/225000 [145:16:06<173:06:14,  4.09s/it]
 32%|███▏      | 72567/225000 [145:16:10<172:37:13,  4.08s/it]
 32%|███▏      | 72568/225000 [145:16:14<172:51:04,  4.08s/it]
 32%|███▏      | 72569/225000 [145:16:24<241:38:30,  5.71s/it]
 32%|███▏      | 72570/225000 [145:16:27<209:09:33,  4.94s/it]
 32%|███▏      | 72571/225000 [145:16:29<174:28:20,  4.12s/it]
 32%|███▏      | 72572/225000 [145:16:35<201:42:17,  4.76s/it]
 32%|███▏      | 72573/225000 [145:16:41<211:41:55,  5.00s/it]
 32%|███▏      | 72574/225000 [145:16:48<235:30:51,  5.56s/it]
 32%|███▏      | 72575/225000 [145:16:51<202:44:20,  4.79s/it]
 32%|███▏      | 72576/225000 [145:16:54<178:51:35,  4.22s/it]
 32%|███▏      | 72577/225000 [145:16:56<156:44:24,  3.70s/it]
 32%|███▏      | 72578/225000 [145:17:00<157:48:13,  3.73s/it]
 32%|███▏      | 72579/225000 [145:17:08<206:45:04,  4.88s/it]
 32%|███▏      | 72580/225000 [145:17:10<178:47:31,  4.22s/it]
 32%|███▏      | 72581/225000 [145:17:13<161:01:10,  3.80s/it]
 32%|███▏      | 72582/225000 [145:17:21<210:40:55,  4.98s/it]
 32%|███▏      | 72583/225000 [145:17:25<199:45:24,  4.72s/it]
 32%|███▏      | 72584/225000 [145:17:28<175:35:45,  4.15s/it]
 32%|███▏      | 72585/225000 [145:17:30<157:37:48,  3.72s/it]
 32%|███▏      | 72586/225000 [145:17:38<206:32:01,  4.88s/it]
 32%|███▏      | 72587/225000 [145:17:43<209:35:44,  4.95s/it]
 32%|███▏      | 72588/225000 [145:17:46<181:30:01,  4.29s/it]
 32%|███▏      | 72589/225000 [145:17:54<232:08:25,  5.48s/it]
 32%|███▏      | 72590/225000 [145:17:57<201:04:16,  4.75s/it]
 32%|███▏      | 72591/225000 [145:18:01<194:27:16,  4.59s/it]
 32%|███▏      | 72592/225000 [145:18:04<169:02:20,  3.99s/it]
 32%|███▏      | 72593/225000 [145:18:07<154:48:23,  3.66s/it]
 32%|███▏      | 72594/225000 [145:18:11<161:48:53,  3.82s/it]
 32%|███▏      | 72595/225000 [145:18:19<216:00:08,  5.10s/it]
 32%|███▏      | 72596/225000 [145:18:25<227:39:42,  5.38s/it]
 32%|███▏      | 72597/225000 [145:18:29<205:08:09,  4.85s/it]
 32%|███▏      | 72598/225000 [145:18:32<183:47:18,  4.34s/it]
 32%|███▏      | 72599/225000 [145:18:35<167:30:23,  3.96s/it]
 32%|███▏      | 72600/225000 [145:18:38<158:36:57,  3.75s/it]
                                                              
{'loss': '0.7956', 'grad_norm': '1.721', 'learning_rate': '4.026e-05', 'epoch': '0.3227', 'num_input_tokens_seen': 240644840, 'train_runtime': '5.231e+05', 'train_tokens_per_second': '460'}

 32%|███▏      | 72600/225000 [145:18:38<158:36:57,  3.75s/it]
 32%|███▏      | 72601/225000 [145:18:42<155:20:47,  3.67s/it]
 32%|███▏      | 72602/225000 [145:18:45<154:20:03,  3.65s/it]
 32%|███▏      | 72603/225000 [145:18:49<153:20:48,  3.62s/it]
 32%|███▏      | 72604/225000 [145:18:55<182:10:30,  4.30s/it]
 32%|███▏      | 72605/225000 [145:19:03<233:46:57,  5.52s/it]
 32%|███▏      | 72606/225000 [145:19:07<208:12:17,  4.92s/it]
 32%|███▏      | 72607/225000 [145:19:12<208:22:01,  4.92s/it]
 32%|███▏      | 72608/225000 [145:19:18<220:17:24,  5.20s/it]
 32%|███▏      | 72609/225000 [145:19:21<200:39:24,  4.74s/it]
 32%|███▏      | 72610/225000 [145:19:25<183:19:38,  4.33s/it]
 32%|███▏      | 72611/225000 [145:19:27<164:54:11,  3.90s/it]
 32%|███▏      | 72612/225000 [145:19:30<150:48:46,  3.56s/it]
 32%|███▏      | 72613/225000 [145:19:33<144:57:46,  3.42s/it]
 32%|███▏      | 72614/225000 [145:19:38<156:24:34,  3.70s/it]
 32%|███▏      | 72615/225000 [145:19:45<202:02:48,  4.77s/it]
 32%|███▏      | 72616/225000 [145:19:53<239:12:02,  5.65s/it]
 32%|███▏      | 72617/225000 [145:19:56<206:22:53,  4.88s/it]
 32%|███▏      | 72618/225000 [145:19:59<189:57:58,  4.49s/it]
 32%|███▏      | 72619/225000 [145:20:02<171:20:08,  4.05s/it]
 32%|███▏      | 72620/225000 [145:20:05<159:53:45,  3.78s/it]
 32%|███▏      | 72621/225000 [145:20:09<158:03:36,  3.73s/it]
 32%|███▏      | 72622/225000 [145:20:12<151:42:19,  3.58s/it]
 32%|███▏      | 72623/225000 [145:20:16<149:56:46,  3.54s/it]
 32%|███▏      | 72624/225000 [145:20:21<175:57:26,  4.16s/it]
 32%|███▏      | 72625/225000 [145:20:29<224:14:25,  5.30s/it]
 32%|███▏      | 72626/225000 [145:20:32<197:06:51,  4.66s/it]
 32%|███▏      | 72627/225000 [145:20:35<173:26:25,  4.10s/it]
 32%|███▏      | 72628/225000 [145:20:42<209:54:24,  4.96s/it]
 32%|███▏      | 72629/225000 [145:20:46<195:59:51,  4.63s/it]
 32%|███▏      | 72630/225000 [145:20:50<187:15:15,  4.42s/it]
 32%|███▏      | 72631/225000 [145:20:53<168:20:54,  3.98s/it]
 32%|███▏      | 72632/225000 [145:20:57<163:26:55,  3.86s/it]
 32%|███▏      | 72633/225000 [145:21:01<164:25:43,  3.88s/it]
 32%|███▏      | 72634/225000 [145:21:04<159:08:57,  3.76s/it]
 32%|███▏      | 72635/225000 [145:21:11<197:53:14,  4.68s/it]
 32%|███▏      | 72636/225000 [145:21:14<180:42:04,  4.27s/it]
 32%|███▏      | 72637/225000 [145:21:20<195:54:38,  4.63s/it]
 32%|███▏      | 72638/225000 [145:21:24<190:51:19,  4.51s/it]
 32%|███▏      | 72639/225000 [145:21:27<179:04:12,  4.23s/it]
 32%|███▏      | 72640/225000 [145:21:32<187:36:22,  4.43s/it]
 32%|███▏      | 72641/225000 [145:21:39<215:09:09,  5.08s/it]
 32%|███▏      | 72642/225000 [145:21:43<199:46:53,  4.72s/it]
 32%|███▏      | 72643/225000 [145:21:47<195:40:25,  4.62s/it]
 32%|███▏      | 72644/225000 [145:21:56<250:18:45,  5.91s/it]
 32%|███▏      | 72645/225000 [145:21:59<210:03:28,  4.96s/it]
 32%|███▏      | 72646/225000 [145:22:05<220:45:56,  5.22s/it]
 32%|███▏      | 72647/225000 [145:22:09<203:57:11,  4.82s/it]
 32%|███▏      | 72648/225000 [145:22:13<195:53:34,  4.63s/it]
 32%|███▏      | 72649/225000 [145:22:16<181:48:02,  4.30s/it]
 32%|███▏      | 72650/225000 [145:22:19<164:11:26,  3.88s/it]
                                                              
{'loss': '0.7968', 'grad_norm': '1.537', 'learning_rate': '4.025e-05', 'epoch': '0.3229', 'num_input_tokens_seen': 240738288, 'train_runtime': '5.233e+05', 'train_tokens_per_second': '460'}

 32%|███▏      | 72650/225000 [145:22:19<164:11:26,  3.88s/it]
 32%|███▏      | 72651/225000 [145:22:22<156:51:39,  3.71s/it]
 32%|███▏      | 72652/225000 [145:22:32<230:45:46,  5.45s/it]
 32%|███▏      | 72653/225000 [145:22:35<198:28:04,  4.69s/it]
 32%|███▏      | 72654/225000 [145:22:38<174:00:49,  4.11s/it]
 32%|███▏      | 72655/225000 [145:22:41<162:36:12,  3.84s/it]
 32%|███▏      | 72656/225000 [145:22:44<159:38:07,  3.77s/it]
 32%|███▏      | 72657/225000 [145:22:48<153:23:27,  3.62s/it]
 32%|███▏      | 72658/225000 [145:22:56<209:40:09,  4.95s/it]
 32%|███▏      | 72659/225000 [145:23:05<264:11:02,  6.24s/it]
 32%|███▏      | 72660/225000 [145:23:08<223:51:08,  5.29s/it]
 32%|███▏      | 72661/225000 [145:23:11<191:54:53,  4.54s/it]
 32%|███▏      | 72662/225000 [145:23:21<259:34:02,  6.13s/it]
 32%|███▏      | 72663/225000 [145:23:23<214:32:48,  5.07s/it]
 32%|███▏      | 72664/225000 [145:23:27<192:45:25,  4.56s/it]
 32%|███▏      | 72665/225000 [145:23:33<219:58:06,  5.20s/it]
 32%|███▏      | 72666/225000 [145:23:36<183:15:18,  4.33s/it]
 32%|███▏      | 72667/225000 [145:23:42<213:06:59,  5.04s/it]
 32%|███▏      | 72668/225000 [145:23:45<184:34:12,  4.36s/it]
 32%|███▏      | 72669/225000 [145:23:48<165:40:34,  3.92s/it]
 32%|███▏      | 72670/225000 [145:23:51<156:11:31,  3.69s/it]
 32%|███▏      | 72671/225000 [145:24:00<214:59:44,  5.08s/it]
 32%|███▏      | 72672/225000 [145:24:05<214:11:25,  5.06s/it]
 32%|███▏      | 72673/225000 [145:24:08<189:29:51,  4.48s/it]
 32%|███▏      | 72674/225000 [145:24:11<171:54:38,  4.06s/it]
 32%|███▏      | 72675/225000 [145:24:14<163:30:09,  3.86s/it]
 32%|███▏      | 72676/225000 [145:24:22<217:42:34,  5.15s/it]
 32%|███▏      | 72677/225000 [145:24:27<211:19:54,  4.99s/it]
 32%|███▏      | 72678/225000 [145:24:30<186:58:28,  4.42s/it]
 32%|███▏      | 72679/225000 [145:24:33<171:06:50,  4.04s/it]
 32%|███▏      | 72680/225000 [145:24:37<169:40:04,  4.01s/it]
 32%|███▏      | 72681/225000 [145:24:44<209:47:17,  4.96s/it]
 32%|███▏      | 72682/225000 [145:24:52<242:11:15,  5.72s/it]
 32%|███▏      | 72683/225000 [145:24:54<201:07:33,  4.75s/it]
 32%|███▏      | 72684/225000 [145:24:57<174:53:28,  4.13s/it]
 32%|███▏      | 72685/225000 [145:24:59<152:43:41,  3.61s/it]
 32%|███▏      | 72686/225000 [145:25:02<138:45:00,  3.28s/it]
 32%|███▏      | 72687/225000 [145:25:11<214:01:11,  5.06s/it]
 32%|███▏      | 72688/225000 [145:25:14<183:14:06,  4.33s/it]
 32%|███▏      | 72689/225000 [145:25:17<166:29:01,  3.93s/it]
 32%|███▏      | 72690/225000 [145:25:22<181:36:33,  4.29s/it]
 32%|███▏      | 72691/225000 [145:25:25<169:24:48,  4.00s/it]
 32%|███▏      | 72692/225000 [145:25:29<161:21:42,  3.81s/it]
 32%|███▏      | 72693/225000 [145:25:33<174:17:40,  4.12s/it]
 32%|███▏      | 72694/225000 [145:25:38<176:11:52,  4.16s/it]
 32%|███▏      | 72695/225000 [145:25:41<167:02:22,  3.95s/it]
 32%|███▏      | 72696/225000 [145:25:50<229:27:34,  5.42s/it]
 32%|███▏      | 72697/225000 [145:25:55<223:39:42,  5.29s/it]
 32%|███▏      | 72698/225000 [145:25:58<199:38:55,  4.72s/it]
 32%|███▏      | 72699/225000 [145:26:01<174:09:49,  4.12s/it]
 32%|███▏      | 72700/225000 [145:26:04<159:18:50,  3.77s/it]
                                                              
{'loss': '0.8336', 'grad_norm': '1.494', 'learning_rate': '4.023e-05', 'epoch': '0.3231', 'num_input_tokens_seen': 240833152, 'train_runtime': '5.236e+05', 'train_tokens_per_second': '460'}

 32%|███▏      | 72700/225000 [145:26:04<159:18:50,  3.77s/it]
 32%|███▏      | 72701/225000 [145:26:08<156:18:46,  3.69s/it]
 32%|███▏      | 72702/225000 [145:26:11<153:00:52,  3.62s/it]
 32%|███▏      | 72703/225000 [145:26:14<147:45:38,  3.49s/it]
 32%|███▏      | 72704/225000 [145:26:21<189:34:18,  4.48s/it]
 32%|███▏      | 72705/225000 [145:26:30<252:49:44,  5.98s/it]
 32%|███▏      | 72706/225000 [145:26:34<222:36:35,  5.26s/it]
 32%|███▏      | 72707/225000 [145:26:38<200:51:02,  4.75s/it]
 32%|███▏      | 72708/225000 [145:26:42<201:30:41,  4.76s/it]
 32%|███▏      | 72709/225000 [145:26:46<182:28:30,  4.31s/it]
 32%|███▏      | 72710/225000 [145:26:49<167:53:44,  3.97s/it]
 32%|███▏      | 72711/225000 [145:26:52<159:07:40,  3.76s/it]
 32%|███▏      | 72712/225000 [145:26:58<182:26:08,  4.31s/it]
 32%|███▏      | 72713/225000 [145:27:04<202:01:27,  4.78s/it]
 32%|███▏      | 72714/225000 [145:27:07<185:00:36,  4.37s/it]
 32%|███▏      | 72715/225000 [145:27:10<172:58:37,  4.09s/it]
 32%|███▏      | 72716/225000 [145:27:15<178:36:04,  4.22s/it]
 32%|███▏      | 72717/225000 [145:27:19<179:13:47,  4.24s/it]
 32%|███▏      | 72718/225000 [145:27:23<173:34:54,  4.10s/it]
 32%|███▏      | 72719/225000 [145:27:26<161:17:54,  3.81s/it]
 32%|███▏      | 72720/225000 [145:27:31<173:56:32,  4.11s/it]
 32%|███▏      | 72721/225000 [145:27:40<241:15:28,  5.70s/it]
 32%|███▏      | 72722/225000 [145:27:46<236:04:25,  5.58s/it]
 32%|███▏      | 72723/225000 [145:27:50<216:06:48,  5.11s/it]
 32%|███▏      | 72724/225000 [145:27:57<246:23:09,  5.82s/it]
 32%|███▏      | 72725/225000 [145:28:00<206:07:20,  4.87s/it]
 32%|███▏      | 72726/225000 [145:28:06<224:53:02,  5.32s/it]
 32%|███▏      | 72727/225000 [145:28:09<189:10:09,  4.47s/it]
 32%|███▏      | 72728/225000 [145:28:11<165:22:20,  3.91s/it]
 32%|███▏      | 72729/225000 [145:28:15<157:48:43,  3.73s/it]
 32%|███▏      | 72730/225000 [145:28:22<199:55:46,  4.73s/it]
 32%|███▏      | 72731/225000 [145:28:25<181:44:42,  4.30s/it]
 32%|███▏      | 72732/225000 [145:28:29<174:38:50,  4.13s/it]
 32%|███▏      | 72733/225000 [145:28:36<219:38:12,  5.19s/it]
 32%|███▏      | 72734/225000 [145:28:40<195:33:58,  4.62s/it]
 32%|███▏      | 72735/225000 [145:28:43<175:29:58,  4.15s/it]
 32%|███▏      | 72736/225000 [145:28:46<161:10:46,  3.81s/it]
 32%|███▏      | 72737/225000 [145:28:48<148:16:01,  3.51s/it]
 32%|███▏      | 72738/225000 [145:28:52<147:00:21,  3.48s/it]
 32%|███▏      | 72739/225000 [145:28:55<137:21:30,  3.25s/it]
 32%|███▏      | 72740/225000 [145:28:59<146:21:04,  3.46s/it]
 32%|███▏      | 72741/225000 [145:29:08<217:49:14,  5.15s/it]
 32%|███▏      | 72742/225000 [145:29:11<198:20:08,  4.69s/it]
 32%|███▏      | 72743/225000 [145:29:14<175:30:09,  4.15s/it]
 32%|███▏      | 72744/225000 [145:29:17<162:51:20,  3.85s/it]
 32%|███▏      | 72745/225000 [145:29:20<151:02:46,  3.57s/it]
 32%|███▏      | 72746/225000 [145:29:23<140:26:48,  3.32s/it]
 32%|███▏      | 72747/225000 [145:29:29<168:55:17,  3.99s/it]
 32%|███▏      | 72748/225000 [145:29:38<240:21:09,  5.68s/it]
 32%|███▏      | 72749/225000 [145:29:42<213:36:03,  5.05s/it]
 32%|███▏      | 72750/225000 [145:29:46<197:59:02,  4.68s/it]
                                                              
{'loss': '0.8397', 'grad_norm': '1.268', 'learning_rate': '4.022e-05', 'epoch': '0.3233', 'num_input_tokens_seen': 240924552, 'train_runtime': '5.238e+05', 'train_tokens_per_second': '460'}

 32%|███▏      | 72750/225000 [145:29:46<197:59:02,  4.68s/it]
 32%|███▏      | 72751/225000 [145:29:49<177:33:10,  4.20s/it]
 32%|███▏      | 72752/225000 [145:29:57<227:17:24,  5.37s/it]
 32%|███▏      | 72753/225000 [145:30:00<196:31:42,  4.65s/it]
 32%|███▏      | 72754/225000 [145:30:03<178:07:52,  4.21s/it]
 32%|███▏      | 72755/225000 [145:30:07<173:57:04,  4.11s/it]
 32%|███▏      | 72756/225000 [145:30:09<154:26:35,  3.65s/it]
 32%|███▏      | 72757/225000 [145:30:18<217:37:51,  5.15s/it]
 32%|███▏      | 72758/225000 [145:30:21<195:39:21,  4.63s/it]
 32%|███▏      | 72759/225000 [145:30:24<168:19:54,  3.98s/it]
 32%|███▏      | 72760/225000 [145:30:31<209:31:30,  4.95s/it]
 32%|███▏      | 72761/225000 [145:30:34<179:24:45,  4.24s/it]
 32%|███▏      | 72762/225000 [145:30:38<180:20:09,  4.26s/it]
 32%|███▏      | 72763/225000 [145:30:43<193:23:58,  4.57s/it]
 32%|███▏      | 72764/225000 [145:30:47<181:50:17,  4.30s/it]
 32%|███▏      | 72765/225000 [145:30:51<176:39:17,  4.18s/it]
 32%|███▏      | 72766/225000 [145:30:54<170:04:51,  4.02s/it]
 32%|███▏      | 72767/225000 [145:30:58<169:42:50,  4.01s/it]
 32%|███▏      | 72768/225000 [145:31:03<170:35:37,  4.03s/it]
 32%|███▏      | 72769/225000 [145:31:06<168:18:44,  3.98s/it]
 32%|███▏      | 72770/225000 [145:31:13<207:32:34,  4.91s/it]
 32%|███▏      | 72771/225000 [145:31:21<246:27:44,  5.83s/it]
 32%|███▏      | 72772/225000 [145:31:24<202:43:08,  4.79s/it]
 32%|███▏      | 72773/225000 [145:31:27<182:44:59,  4.32s/it]
 32%|███▏      | 72774/225000 [145:31:31<175:57:38,  4.16s/it]
 32%|███▏      | 72775/225000 [145:31:39<226:25:53,  5.35s/it]
 32%|███▏      | 72776/225000 [145:31:43<206:30:00,  4.88s/it]
 32%|███▏      | 72777/225000 [145:31:45<172:05:25,  4.07s/it]
 32%|███▏      | 72778/225000 [145:31:53<223:10:27,  5.28s/it]
 32%|███▏      | 72779/225000 [145:31:56<190:13:07,  4.50s/it]
 32%|███▏      | 72780/225000 [145:31:59<171:42:37,  4.06s/it]
 32%|███▏      | 72781/225000 [145:32:01<153:57:20,  3.64s/it]
 32%|███▏      | 72782/225000 [145:32:04<141:29:09,  3.35s/it]
 32%|███▏      | 72783/225000 [145:32:12<205:35:43,  4.86s/it]
 32%|███▏      | 72784/225000 [145:32:18<207:46:42,  4.91s/it]
 32%|███▏      | 72785/225000 [145:32:21<186:21:43,  4.41s/it]
 32%|███▏      | 72786/225000 [145:32:24<167:25:32,  3.96s/it]
 32%|███▏      | 72787/225000 [145:32:28<176:44:41,  4.18s/it]
 32%|███▏      | 72788/225000 [145:32:37<232:48:29,  5.51s/it]
 32%|███▏      | 72789/225000 [145:32:39<192:22:56,  4.55s/it]
 32%|███▏      | 72790/225000 [145:32:42<174:18:07,  4.12s/it]
 32%|███▏      | 72791/225000 [145:32:45<157:54:24,  3.73s/it]
 32%|███▏      | 72792/225000 [145:32:50<169:57:02,  4.02s/it]
 32%|███▏      | 72793/225000 [145:32:57<203:21:59,  4.81s/it]
 32%|███▏      | 72794/225000 [145:33:00<192:04:51,  4.54s/it]
 32%|███▏      | 72795/225000 [145:33:04<178:55:24,  4.23s/it]
 32%|███▏      | 72796/225000 [145:33:10<204:05:47,  4.83s/it]
 32%|███▏      | 72797/225000 [145:33:13<178:40:51,  4.23s/it]
 32%|███▏      | 72798/225000 [145:33:16<161:36:14,  3.82s/it]
 32%|███▏      | 72799/225000 [145:33:21<175:38:25,  4.15s/it]
 32%|███▏      | 72800/225000 [145:33:28<215:58:36,  5.11s/it]
                                                              
{'loss': '0.8517', 'grad_norm': '2.096', 'learning_rate': '4.02e-05', 'epoch': '0.3236', 'num_input_tokens_seen': 241017256, 'train_runtime': '5.24e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 72800/225000 [145:33:28<215:58:36,  5.11s/it]
 32%|███▏      | 72801/225000 [145:33:31<186:56:02,  4.42s/it]
 32%|███▏      | 72802/225000 [145:33:38<214:45:32,  5.08s/it]
 32%|███▏      | 72803/225000 [145:33:40<184:11:02,  4.36s/it]
 32%|███▏      | 72804/225000 [145:33:44<180:04:31,  4.26s/it]
 32%|███▏      | 72805/225000 [145:33:48<168:26:45,  3.98s/it]
 32%|███▏      | 72806/225000 [145:33:52<167:33:42,  3.96s/it]
 32%|███▏      | 72807/225000 [145:33:56<169:04:58,  4.00s/it]
 32%|███▏      | 72808/225000 [145:34:00<174:27:38,  4.13s/it]
 32%|███▏      | 72809/225000 [145:34:09<230:52:50,  5.46s/it]
 32%|███▏      | 72810/225000 [145:34:12<208:21:57,  4.93s/it]
 32%|███▏      | 72811/225000 [145:34:15<175:23:53,  4.15s/it]
 32%|███▏      | 72812/225000 [145:34:18<159:52:48,  3.78s/it]
 32%|███▏      | 72813/225000 [145:34:21<158:28:47,  3.75s/it]
 32%|███▏      | 72814/225000 [145:34:25<152:24:31,  3.61s/it]
 32%|███▏      | 72815/225000 [145:34:28<147:03:45,  3.48s/it]
 32%|███▏      | 72816/225000 [145:34:31<146:15:49,  3.46s/it]
 32%|███▏      | 72817/225000 [145:34:35<151:57:14,  3.59s/it]
 32%|███▏      | 72818/225000 [145:34:40<173:45:36,  4.11s/it]
 32%|███▏      | 72819/225000 [145:34:45<185:39:37,  4.39s/it]
 32%|███▏      | 72820/225000 [145:34:54<241:26:02,  5.71s/it]
 32%|███▏      | 72821/225000 [145:34:58<212:29:44,  5.03s/it]
 32%|███▏      | 72822/225000 [145:35:01<187:22:16,  4.43s/it]
 32%|███▏      | 72823/225000 [145:35:04<175:22:40,  4.15s/it]
 32%|███▏      | 72824/225000 [145:35:07<164:51:40,  3.90s/it]
 32%|███▏      | 72825/225000 [145:35:11<160:11:53,  3.79s/it]
 32%|███▏      | 72826/225000 [145:35:15<167:57:02,  3.97s/it]
 32%|███▏      | 72827/225000 [145:35:24<231:39:33,  5.48s/it]
 32%|███▏      | 72828/225000 [145:35:27<196:41:38,  4.65s/it]
 32%|███▏      | 72829/225000 [145:35:30<174:52:11,  4.14s/it]
 32%|███▏      | 72830/225000 [145:35:38<228:55:52,  5.42s/it]
 32%|███▏      | 72831/225000 [145:35:42<204:14:41,  4.83s/it]
 32%|███▏      | 72832/225000 [145:35:47<211:54:23,  5.01s/it]
 32%|███▏      | 72833/225000 [145:35:50<182:52:20,  4.33s/it]
 32%|███▏      | 72834/225000 [145:35:53<165:47:26,  3.92s/it]
 32%|███▏      | 72835/225000 [145:35:56<154:34:15,  3.66s/it]
 32%|███▏      | 72836/225000 [145:35:59<143:27:32,  3.39s/it]
 32%|███▏      | 72837/225000 [145:36:02<142:12:07,  3.36s/it]
 32%|███▏      | 72838/225000 [145:36:06<149:40:23,  3.54s/it]
 32%|███▏      | 72839/225000 [145:36:14<209:22:46,  4.95s/it]
 32%|███▏      | 72840/225000 [145:36:18<188:47:35,  4.47s/it]
 32%|███▏      | 72841/225000 [145:36:22<180:18:46,  4.27s/it]
 32%|███▏      | 72842/225000 [145:36:25<167:42:58,  3.97s/it]
 32%|███▏      | 72843/225000 [145:36:28<163:00:09,  3.86s/it]
 32%|███▏      | 72844/225000 [145:36:34<181:56:16,  4.30s/it]
 32%|███▏      | 72845/225000 [145:36:44<256:16:33,  6.06s/it]
 32%|███▏      | 72846/225000 [145:36:49<245:55:31,  5.82s/it]
 32%|███▏      | 72847/225000 [145:36:51<200:49:37,  4.75s/it]
 32%|███▏      | 72848/225000 [145:36:54<171:35:49,  4.06s/it]
 32%|███▏      | 72849/225000 [145:36:57<156:24:57,  3.70s/it]
 32%|███▏      | 72850/225000 [145:37:00<152:47:28,  3.62s/it]
                                                              
{'loss': '0.84', 'grad_norm': '1.436', 'learning_rate': '4.019e-05', 'epoch': '0.3238', 'num_input_tokens_seen': 241106680, 'train_runtime': '5.242e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 72850/225000 [145:37:00<152:47:28,  3.62s/it]
 32%|███▏      | 72851/225000 [145:37:04<153:18:20,  3.63s/it]
 32%|███▏      | 72852/225000 [145:37:07<147:58:00,  3.50s/it]
 32%|███▏      | 72853/225000 [145:37:10<142:47:13,  3.38s/it]
 32%|███▏      | 72854/225000 [145:37:16<177:28:48,  4.20s/it]
 32%|███▏      | 72855/225000 [145:37:24<226:01:59,  5.35s/it]
 32%|███▏      | 72856/225000 [145:37:29<223:40:12,  5.29s/it]
 32%|███▏      | 72857/225000 [145:37:33<198:13:56,  4.69s/it]
 32%|███▏      | 72858/225000 [145:37:37<193:45:08,  4.58s/it]
 32%|███▏      | 72859/225000 [145:37:41<190:57:26,  4.52s/it]
 32%|███▏      | 72860/225000 [145:37:45<182:25:59,  4.32s/it]
 32%|███▏      | 72861/225000 [145:37:55<245:20:35,  5.81s/it]
 32%|███▏      | 72862/225000 [145:38:00<243:05:50,  5.75s/it]
 32%|███▏      | 72863/225000 [145:38:03<208:43:26,  4.94s/it]
 32%|███▏      | 72864/225000 [145:38:10<238:36:05,  5.65s/it]
 32%|███▏      | 72865/225000 [145:38:13<200:06:00,  4.74s/it]
 32%|███▏      | 72866/225000 [145:38:16<181:27:24,  4.29s/it]
 32%|███▏      | 72867/225000 [145:38:22<203:42:51,  4.82s/it]
 32%|███▏      | 72868/225000 [145:38:28<214:26:38,  5.07s/it]
 32%|███▏      | 72869/225000 [145:38:31<185:40:21,  4.39s/it]
 32%|███▏      | 72870/225000 [145:38:35<176:49:28,  4.18s/it]
 32%|███▏      | 72871/225000 [145:38:37<160:49:33,  3.81s/it]
 32%|███▏      | 72872/225000 [145:38:41<151:13:51,  3.58s/it]
 32%|███▏      | 72873/225000 [145:38:49<213:08:43,  5.04s/it]
 32%|███▏      | 72874/225000 [145:38:52<181:53:27,  4.30s/it]
 32%|███▏      | 72875/225000 [145:38:55<172:56:18,  4.09s/it]
 32%|███▏      | 72876/225000 [145:38:59<170:56:38,  4.05s/it]
 32%|███▏      | 72877/225000 [145:39:03<166:09:14,  3.93s/it]
 32%|███▏      | 72878/225000 [145:39:08<182:02:18,  4.31s/it]
 32%|███▏      | 72879/225000 [145:39:16<225:57:49,  5.35s/it]
 32%|███▏      | 72880/225000 [145:39:18<187:46:56,  4.44s/it]
 32%|███▏      | 72881/225000 [145:39:21<164:28:00,  3.89s/it]
 32%|███▏      | 72882/225000 [145:39:24<154:08:08,  3.65s/it]
 32%|███▏      | 72883/225000 [145:39:32<207:40:49,  4.91s/it]
 32%|███▏      | 72884/225000 [145:39:34<176:27:51,  4.18s/it]
 32%|███▏      | 72885/225000 [145:39:40<192:07:39,  4.55s/it]
 32%|███▏      | 72886/225000 [145:39:43<177:48:50,  4.21s/it]
 32%|███▏      | 72887/225000 [145:39:46<165:22:24,  3.91s/it]
 32%|███▏      | 72888/225000 [145:39:49<155:21:55,  3.68s/it]
 32%|███▏      | 72889/225000 [145:39:53<155:30:10,  3.68s/it]
 32%|███▏      | 72890/225000 [145:39:56<153:29:00,  3.63s/it]
 32%|███▏      | 72891/225000 [145:40:05<221:46:38,  5.25s/it]
 32%|███▏      | 72892/225000 [145:40:10<208:41:42,  4.94s/it]
 32%|███▏      | 72893/225000 [145:40:16<225:01:21,  5.33s/it]
 32%|███▏      | 72894/225000 [145:40:21<215:56:52,  5.11s/it]
 32%|███▏      | 72895/225000 [145:40:24<190:52:52,  4.52s/it]
 32%|███▏      | 72896/225000 [145:40:31<232:09:25,  5.49s/it]
 32%|███▏      | 72897/225000 [145:40:34<193:21:22,  4.58s/it]
 32%|███▏      | 72898/225000 [145:40:37<177:59:34,  4.21s/it]
 32%|███▏      | 72899/225000 [145:40:40<154:39:36,  3.66s/it]
 32%|███▏      | 72900/225000 [145:40:43<150:50:29,  3.57s/it]
                                                              
{'loss': '0.8518', 'grad_norm': '1.348', 'learning_rate': '4.017e-05', 'epoch': '0.324', 'num_input_tokens_seen': 241199272, 'train_runtime': '5.244e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 72900/225000 [145:40:43<150:50:29,  3.57s/it]
 32%|███▏      | 72901/225000 [145:40:51<211:01:51,  4.99s/it]
 32%|███▏      | 72902/225000 [145:40:57<219:53:43,  5.20s/it]
 32%|███▏      | 72903/225000 [145:40:59<183:34:07,  4.34s/it]
 32%|███▏      | 72904/225000 [145:41:02<165:22:27,  3.91s/it]
 32%|███▏      | 72905/225000 [145:41:10<211:08:11,  5.00s/it]
 32%|███▏      | 72906/225000 [145:41:14<196:34:01,  4.65s/it]
 32%|███▏      | 72907/225000 [145:41:17<181:07:43,  4.29s/it]
 32%|███▏      | 72908/225000 [145:41:21<177:26:14,  4.20s/it]
 32%|███▏      | 72909/225000 [145:41:24<167:30:55,  3.97s/it]
 32%|███▏      | 72910/225000 [145:41:28<156:50:57,  3.71s/it]
 32%|███▏      | 72911/225000 [145:41:32<168:09:46,  3.98s/it]
 32%|███▏      | 72912/225000 [145:41:35<154:59:54,  3.67s/it]
 32%|███▏      | 72913/225000 [145:41:39<155:59:45,  3.69s/it]
 32%|███▏      | 72914/225000 [145:41:43<163:01:05,  3.86s/it]
 32%|███▏      | 72915/225000 [145:41:46<148:32:46,  3.52s/it]
 32%|███▏      | 72916/225000 [145:41:49<140:26:23,  3.32s/it]
 32%|███▏      | 72917/225000 [145:41:57<201:38:00,  4.77s/it]
 32%|███▏      | 72918/225000 [145:42:04<230:00:28,  5.44s/it]
 32%|███▏      | 72919/225000 [145:42:07<202:12:08,  4.79s/it]
 32%|███▏      | 72920/225000 [145:42:15<243:37:47,  5.77s/it]
 32%|███▏      | 72921/225000 [145:42:18<202:15:50,  4.79s/it]
 32%|███▏      | 72922/225000 [145:42:20<171:00:33,  4.05s/it]
 32%|███▏      | 72923/225000 [145:42:24<168:20:03,  3.98s/it]
 32%|███▏      | 72924/225000 [145:42:28<167:42:44,  3.97s/it]
 32%|███▏      | 72925/225000 [145:42:35<213:37:48,  5.06s/it]
 32%|███▏      | 72926/225000 [145:42:39<194:11:45,  4.60s/it]
 32%|███▏      | 72927/225000 [145:42:42<180:41:00,  4.28s/it]
 32%|███▏      | 72928/225000 [145:42:45<164:02:59,  3.88s/it]
 32%|███▏      | 72929/225000 [145:42:49<157:23:58,  3.73s/it]
 32%|███▏      | 72930/225000 [145:42:52<148:27:42,  3.51s/it]
 32%|███▏      | 72931/225000 [145:42:59<195:57:10,  4.64s/it]
 32%|███▏      | 72932/225000 [145:43:05<212:15:11,  5.02s/it]
 32%|███▏      | 72933/225000 [145:43:08<181:50:34,  4.30s/it]
 32%|███▏      | 72934/225000 [145:43:13<200:44:04,  4.75s/it]
 32%|███▏      | 72935/225000 [145:43:21<239:15:01,  5.66s/it]
 32%|███▏      | 72936/225000 [145:43:24<204:03:26,  4.83s/it]
 32%|███▏      | 72937/225000 [145:43:31<230:31:55,  5.46s/it]
 32%|███▏      | 72938/225000 [145:43:34<195:54:34,  4.64s/it]
 32%|███▏      | 72939/225000 [145:43:37<175:15:05,  4.15s/it]
 32%|███▏      | 72940/225000 [145:43:40<159:51:42,  3.78s/it]
 32%|███▏      | 72941/225000 [145:43:49<229:20:08,  5.43s/it]
 32%|███▏      | 72942/225000 [145:43:52<197:30:55,  4.68s/it]
 32%|███▏      | 72943/225000 [145:43:59<231:01:34,  5.47s/it]
 32%|███▏      | 72944/225000 [145:44:03<207:59:09,  4.92s/it]
 32%|███▏      | 72945/225000 [145:44:08<207:38:15,  4.92s/it]
 32%|███▏      | 72946/225000 [145:44:11<192:10:29,  4.55s/it]
 32%|███▏      | 72947/225000 [145:44:15<177:32:28,  4.20s/it]
 32%|███▏      | 72948/225000 [145:44:18<162:55:44,  3.86s/it]
 32%|███▏      | 72949/225000 [145:44:21<151:33:34,  3.59s/it]
 32%|███▏      | 72950/225000 [145:44:24<147:32:56,  3.49s/it]
                                                              
{'loss': '0.8314', 'grad_norm': '1.978', 'learning_rate': '4.016e-05', 'epoch': '0.3242', 'num_input_tokens_seen': 241291352, 'train_runtime': '5.247e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 72950/225000 [145:44:24<147:32:56,  3.49s/it]
 32%|███▏      | 72951/225000 [145:44:34<234:07:44,  5.54s/it]
 32%|███▏      | 72952/225000 [145:44:38<204:22:27,  4.84s/it]
 32%|███▏      | 72953/225000 [145:44:42<193:38:08,  4.58s/it]
 32%|███▏      | 72954/225000 [145:44:45<178:59:39,  4.24s/it]
 32%|███▏      | 72955/225000 [145:44:48<163:34:35,  3.87s/it]
 32%|███▏      | 72956/225000 [145:44:52<167:10:10,  3.96s/it]
 32%|███▏      | 72957/225000 [145:44:55<149:08:27,  3.53s/it]
 32%|███▏      | 72958/225000 [145:44:59<158:42:59,  3.76s/it]
 32%|███▏      | 72959/225000 [145:45:03<157:27:18,  3.73s/it]
 32%|███▏      | 72960/225000 [145:45:09<184:51:51,  4.38s/it]
 32%|███▏      | 72961/225000 [145:45:17<239:23:19,  5.67s/it]
 32%|███▏      | 72962/225000 [145:45:21<215:02:30,  5.09s/it]
 32%|███▏      | 72963/225000 [145:45:24<188:49:50,  4.47s/it]
 32%|███▏      | 72964/225000 [145:45:27<166:24:45,  3.94s/it]
 32%|███▏      | 72965/225000 [145:45:34<210:03:34,  4.97s/it]
 32%|███▏      | 72966/225000 [145:45:39<210:51:09,  4.99s/it]
 32%|███▏      | 72967/225000 [145:45:42<180:21:18,  4.27s/it]
 32%|███▏      | 72968/225000 [145:45:49<219:07:38,  5.19s/it]
 32%|███▏      | 72969/225000 [145:45:52<189:02:04,  4.48s/it]
 32%|███▏      | 72970/225000 [145:45:57<192:27:11,  4.56s/it]
 32%|███▏      | 72971/225000 [145:46:02<199:52:10,  4.73s/it]
 32%|███▏      | 72972/225000 [145:46:05<176:54:13,  4.19s/it]
 32%|███▏      | 72973/225000 [145:46:10<196:21:53,  4.65s/it]
 32%|███▏      | 72974/225000 [145:46:18<230:21:20,  5.45s/it]
 32%|███▏      | 72975/225000 [145:46:21<202:54:15,  4.80s/it]
 32%|███▏      | 72976/225000 [145:46:25<187:56:42,  4.45s/it]
 32%|███▏      | 72977/225000 [145:46:28<168:17:06,  3.99s/it]
 32%|███▏      | 72978/225000 [145:46:36<219:14:28,  5.19s/it]
 32%|███▏      | 72979/225000 [145:46:39<194:49:42,  4.61s/it]
 32%|███▏      | 72980/225000 [145:46:42<173:10:03,  4.10s/it]
 32%|███▏      | 72981/225000 [145:46:47<185:07:10,  4.38s/it]
 32%|███▏      | 72982/225000 [145:46:50<173:53:21,  4.12s/it]
 32%|███▏      | 72983/225000 [145:46:53<157:54:33,  3.74s/it]
 32%|███▏      | 72984/225000 [145:47:00<200:55:54,  4.76s/it]
 32%|███▏      | 72985/225000 [145:47:03<172:44:50,  4.09s/it]
 32%|███▏      | 72986/225000 [145:47:06<160:09:56,  3.79s/it]
 32%|███▏      | 72987/225000 [145:47:15<223:19:01,  5.29s/it]
 32%|███▏      | 72988/225000 [145:47:18<202:10:00,  4.79s/it]
 32%|███▏      | 72989/225000 [145:47:21<181:24:13,  4.30s/it]
 32%|███▏      | 72990/225000 [145:47:24<164:59:49,  3.91s/it]
 32%|███▏      | 72991/225000 [145:47:28<154:43:12,  3.66s/it]
 32%|███▏      | 72992/225000 [145:47:35<201:18:54,  4.77s/it]
 32%|███▏      | 72993/225000 [145:47:42<235:43:21,  5.58s/it]
 32%|███▏      | 72994/225000 [145:47:45<199:40:53,  4.73s/it]
 32%|███▏      | 72995/225000 [145:47:48<173:07:06,  4.10s/it]
 32%|███▏      | 72996/225000 [145:47:51<164:02:00,  3.88s/it]
 32%|███▏      | 72997/225000 [145:47:54<150:18:49,  3.56s/it]
 32%|███▏      | 72998/225000 [145:47:57<148:43:21,  3.52s/it]
 32%|███▏      | 72999/225000 [145:48:03<170:13:11,  4.03s/it]
 32%|███▏      | 73000/225000 [145:48:10<206:55:07,  4.90s/it]
                                                              
{'loss': '0.8288', 'grad_norm': '2.208', 'learning_rate': '4.014e-05', 'epoch': '0.3244', 'num_input_tokens_seen': 241382872, 'train_runtime': '5.249e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 73000/225000 [145:48:10<206:55:07,  4.90s/it]
 32%|███▏      | 73001/225000 [145:48:15<208:41:12,  4.94s/it]
 32%|███▏      | 73002/225000 [145:48:18<185:41:04,  4.40s/it]
 32%|███▏      | 73003/225000 [145:48:21<168:55:49,  4.00s/it]
 32%|███▏      | 73004/225000 [145:48:24<162:32:36,  3.85s/it]
 32%|███▏      | 73005/225000 [145:48:28<155:53:25,  3.69s/it]
 32%|███▏      | 73006/225000 [145:48:31<153:50:39,  3.64s/it]
 32%|███▏      | 73007/225000 [145:48:34<143:25:06,  3.40s/it]
 32%|███▏      | 73008/225000 [145:48:37<136:00:12,  3.22s/it]
 32%|███▏      | 73009/225000 [145:48:46<207:08:47,  4.91s/it]
 32%|███▏      | 73010/225000 [145:48:51<215:28:19,  5.10s/it]
 32%|███▏      | 73011/225000 [145:48:54<188:51:31,  4.47s/it]
 32%|███▏      | 73012/225000 [145:48:57<171:17:39,  4.06s/it]
 32%|███▏      | 73013/225000 [145:49:01<165:33:50,  3.92s/it]
 32%|███▏      | 73014/225000 [145:49:04<154:09:43,  3.65s/it]
 32%|███▏      | 73015/225000 [145:49:09<170:29:24,  4.04s/it]
 32%|███▏      | 73016/225000 [145:49:14<184:12:55,  4.36s/it]
 32%|███▏      | 73017/225000 [145:49:24<253:09:20,  6.00s/it]
 32%|███▏      | 73018/225000 [145:49:29<242:33:43,  5.75s/it]
 32%|███▏      | 73019/225000 [145:49:31<201:27:39,  4.77s/it]
 32%|███▏      | 73020/225000 [145:49:35<191:37:48,  4.54s/it]
 32%|███▏      | 73021/225000 [145:49:39<178:39:20,  4.23s/it]
 32%|███▏      | 73022/225000 [145:49:42<161:11:00,  3.82s/it]
 32%|███▏      | 73023/225000 [145:49:44<147:22:57,  3.49s/it]
 32%|███▏      | 73024/225000 [145:49:48<144:51:47,  3.43s/it]
 32%|███▏      | 73025/225000 [145:49:53<162:38:09,  3.85s/it]
 32%|███▏      | 73026/225000 [145:50:02<235:43:31,  5.58s/it]
 32%|███▏      | 73027/225000 [145:50:06<217:47:52,  5.16s/it]
 32%|███▏      | 73028/225000 [145:50:10<192:22:47,  4.56s/it]
 32%|███▏      | 73029/225000 [145:50:13<174:21:48,  4.13s/it]
 32%|███▏      | 73030/225000 [145:50:17<172:51:36,  4.09s/it]
 32%|███▏      | 73031/225000 [145:50:22<194:18:38,  4.60s/it]
 32%|███▏      | 73032/225000 [145:50:26<177:08:46,  4.20s/it]
 32%|███▏      | 73033/225000 [145:50:29<161:01:10,  3.81s/it]
 32%|███▏      | 73034/225000 [145:50:32<153:14:25,  3.63s/it]
 32%|███▏      | 73035/225000 [145:50:35<143:10:18,  3.39s/it]
 32%|███▏      | 73036/225000 [145:50:38<148:02:56,  3.51s/it]
 32%|███▏      | 73037/225000 [145:50:44<177:39:49,  4.21s/it]
 32%|███▏      | 73038/225000 [145:50:54<246:36:48,  5.84s/it]
 32%|███▏      | 73039/225000 [145:50:58<221:37:16,  5.25s/it]
 32%|███▏      | 73040/225000 [145:51:01<191:15:20,  4.53s/it]
 32%|███▏      | 73041/225000 [145:51:03<165:56:41,  3.93s/it]
 32%|███▏      | 73042/225000 [145:51:06<154:26:40,  3.66s/it]
 32%|███▏      | 73043/225000 [145:51:09<141:38:39,  3.36s/it]
 32%|███▏      | 73044/225000 [145:51:17<205:03:57,  4.86s/it]
 32%|███▏      | 73045/225000 [145:51:24<224:35:13,  5.32s/it]
 32%|███▏      | 73046/225000 [145:51:26<189:06:15,  4.48s/it]
 32%|███▏      | 73047/225000 [145:51:29<170:47:07,  4.05s/it]
 32%|███▏      | 73048/225000 [145:51:32<155:23:43,  3.68s/it]
 32%|███▏      | 73049/225000 [145:51:35<146:30:06,  3.47s/it]
 32%|███▏      | 73050/225000 [145:51:43<202:14:58,  4.79s/it]
                                                              
{'loss': '0.8235', 'grad_norm': '1.387', 'learning_rate': '4.013e-05', 'epoch': '0.3247', 'num_input_tokens_seen': 241473032, 'train_runtime': '5.251e+05', 'train_tokens_per_second': '459.9'}

 32%|███▏      | 73050/225000 [145:51:43<202:14:58,  4.79s/it]
 32%|███▏      | 73051/225000 [145:51:47<190:04:04,  4.50s/it]
 32%|███▏      | 73052/225000 [145:51:50<178:28:23,  4.23s/it]
 32%|███▏      | 73053/225000 [145:51:54<175:52:01,  4.17s/it]
 32%|███▏      | 73054/225000 [145:51:57<159:05:00,  3.77s/it]
 32%|███▏      | 73055/225000 [145:52:05<207:58:51,  4.93s/it]
 32%|███▏      | 73056/225000 [145:52:08<188:27:19,  4.47s/it]
 32%|███▏      | 73057/225000 [145:52:13<193:20:59,  4.58s/it]
 32%|███▏      | 73058/225000 [145:52:17<182:44:17,  4.33s/it]
 32%|███▏      | 73059/225000 [145:52:22<190:01:00,  4.50s/it]
 32%|███▏      | 73060/225000 [145:52:26<183:11:03,  4.34s/it]
 32%|███▏      | 73061/225000 [145:52:34<233:18:39,  5.53s/it]
 32%|███▏      | 73062/225000 [145:52:37<195:58:45,  4.64s/it]
 32%|███▏      | 73063/225000 [145:52:40<175:35:05,  4.16s/it]
 32%|███▏      | 73064/225000 [145:52:48<227:35:50,  5.39s/it]
 32%|███▏      | 73065/225000 [145:52:51<199:59:50,  4.74s/it]
 32%|███▏      | 73066/225000 [145:52:54<172:39:31,  4.09s/it]
 32%|███▏      | 73067/225000 [145:53:01<215:29:24,  5.11s/it]
 32%|███▏      | 73068/225000 [145:53:05<197:31:22,  4.68s/it]
 32%|███▏      | 73069/225000 [145:53:09<185:48:24,  4.40s/it]
 32%|███▏      | 73070/225000 [145:53:12<172:05:39,  4.08s/it]
 32%|███▏      | 73071/225000 [145:53:15<157:38:25,  3.74s/it]
 32%|███▏      | 73072/225000 [145:53:18<150:26:40,  3.56s/it]
 32%|███▏      | 73073/225000 [145:53:22<151:46:48,  3.60s/it]
 32%|███▏      | 73074/225000 [145:53:25<150:05:15,  3.56s/it]
 32%|███▏      | 73075/225000 [145:53:29<153:23:53,  3.63s/it]
 32%|███▏      | 73076/225000 [145:53:37<213:12:29,  5.05s/it]
 32%|███▏      | 73077/225000 [145:53:46<255:11:52,  6.05s/it]
 32%|███▏      | 73078/225000 [145:53:48<214:33:08,  5.08s/it]
 32%|███▏      | 73079/225000 [145:53:52<199:05:05,  4.72s/it]
 32%|███▏      | 73080/225000 [145:53:58<213:36:01,  5.06s/it]
 32%|███▏      | 73081/225000 [145:54:01<186:33:00,  4.42s/it]
 32%|███▏      | 73082/225000 [145:54:04<163:25:11,  3.87s/it]
 32%|███▏      | 73083/225000 [145:54:07<153:51:51,  3.65s/it]
 32%|███▏      | 73084/225000 [145:54:14<193:13:53,  4.58s/it]
 32%|███▏      | 73085/225000 [145:54:21<224:14:02,  5.31s/it]
 32%|███▏      | 73086/225000 [145:54:24<198:49:05,  4.71s/it]
 32%|███▏      | 73087/225000 [145:54:27<182:10:04,  4.32s/it]
 32%|███▏      | 73088/225000 [145:54:30<165:59:23,  3.93s/it]
 32%|███▏      | 73089/225000 [145:54:34<164:51:30,  3.91s/it]
 32%|███▏      | 73090/225000 [145:54:37<150:55:52,  3.58s/it]
 32%|███▏      | 73091/225000 [145:54:40<145:10:41,  3.44s/it]
 32%|███▏      | 73092/225000 [145:54:43<141:36:55,  3.36s/it]
 32%|███▏      | 73093/225000 [145:54:47<139:26:47,  3.30s/it]
 32%|███▏      | 73094/225000 [145:54:51<160:00:39,  3.79s/it]
 32%|███▏      | 73095/225000 [145:54:55<162:39:46,  3.85s/it]
 32%|███▏      | 73096/225000 [145:54:59<157:20:58,  3.73s/it]
 32%|███▏      | 73097/225000 [145:55:03<160:31:45,  3.80s/it]
 32%|███▏      | 73098/225000 [145:55:09<187:19:57,  4.44s/it]
 32%|███▏      | 73099/225000 [145:55:18<247:39:02,  5.87s/it]
 32%|███▏      | 73100/225000 [145:55:24<247:52:21,  5.87s/it]
                                                              
{'loss': '0.8054', 'grad_norm': '1.539', 'learning_rate': '4.012e-05', 'epoch': '0.3249', 'num_input_tokens_seen': 241564544, 'train_runtime': '5.253e+05', 'train_tokens_per_second': '459.8'}

 32%|███▏      | 73100/225000 [145:55:24<247:52:21,  5.87s/it]
 32%|███▏      | 73101/225000 [145:55:26<205:19:15,  4.87s/it]
 32%|███▏      | 73102/225000 [145:55:29<175:51:04,  4.17s/it]
 32%|███▏      | 73103/225000 [145:55:32<160:38:55,  3.81s/it]
 32%|███▏      | 73104/225000 [145:55:40<212:51:39,  5.04s/it]
 32%|███▏      | 73105/225000 [145:55:42<177:02:56,  4.20s/it]
 32%|███▏      | 73106/225000 [145:55:47<185:39:44,  4.40s/it]
 32%|███▏      | 73107/225000 [145:55:53<202:49:42,  4.81s/it]
 32%|███▏      | 73108/225000 [145:55:56<180:36:32,  4.28s/it]
 32%|███▏      | 73109/225000 [145:56:03<220:53:35,  5.24s/it]
 32%|███▏      | 73110/225000 [145:56:07<200:08:37,  4.74s/it]
 32%|███▏      | 73111/225000 [145:56:10<175:35:12,  4.16s/it]
 32%|███▏      | 73112/225000 [145:56:15<189:44:07,  4.50s/it]
 32%|███▏      | 73113/225000 [145:56:20<195:26:22,  4.63s/it]
 32%|███▏      | 73114/225000 [145:56:23<180:36:50,  4.28s/it]
 32%|███▏      | 73115/225000 [145:56:26<164:14:29,  3.89s/it]
 32%|███▏      | 73116/225000 [145:56:30<156:49:04,  3.72s/it]
 32%|███▏      | 73117/225000 [145:56:33<158:33:44,  3.76s/it]
 32%|███▏      | 73118/225000 [145:56:37<150:40:43,  3.57s/it]
 32%|███▏      | 73119/225000 [145:56:40<146:48:48,  3.48s/it]
 32%|███▏      | 73120/225000 [145:56:44<151:04:33,  3.58s/it]
 32%|███▏      | 73121/225000 [145:56:47<148:46:25,  3.53s/it]
 32%|███▏      | 73122/225000 [145:56:56<214:16:13,  5.08s/it]
 32%|███▏      | 73123/225000 [145:57:02<226:29:55,  5.37s/it]
 32%|███▏      | 73124/225000 [145:57:04<192:37:05,  4.57s/it]
 32%|███▎      | 73125/225000 [145:57:08<173:59:56,  4.12s/it]
 33%|███▎      | 73126/225000 [145:57:10<158:28:23,  3.76s/it]
 33%|███▎      | 73127/225000 [145:57:20<226:41:50,  5.37s/it]
 33%|███▎      | 73128/225000 [145:57:23<200:41:00,  4.76s/it]
 33%|███▎      | 73129/225000 [145:57:27<189:12:42,  4.49s/it]
 33%|███▎      | 73130/225000 [145:57:30<174:48:16,  4.14s/it]
 33%|███▎      | 73131/225000 [145:57:33<159:56:56,  3.79s/it]
 33%|███▎      | 73132/225000 [145:57:37<157:13:11,  3.73s/it]
 33%|███▎      | 73133/225000 [145:57:42<173:23:23,  4.11s/it]
 33%|███▎      | 73134/225000 [145:57:48<199:07:08,  4.72s/it]
 33%|███▎      | 73135/225000 [145:57:51<179:28:39,  4.25s/it]
 33%|███▎      | 73136/225000 [145:57:54<162:46:29,  3.86s/it]
 33%|███▎      | 73137/225000 [145:57:58<166:15:54,  3.94s/it]
 33%|███▎      | 73138/225000 [145:58:01<155:12:26,  3.68s/it]
 33%|███▎      | 73139/225000 [145:58:06<169:44:30,  4.02s/it]
 33%|███▎      | 73140/225000 [145:58:13<203:49:40,  4.83s/it]
 33%|███▎      | 73141/225000 [145:58:15<177:31:25,  4.21s/it]
 33%|███▎      | 73142/225000 [145:58:18<159:06:16,  3.77s/it]
 33%|███▎      | 73143/225000 [145:58:23<169:05:09,  4.01s/it]
 33%|███▎      | 73144/225000 [145:58:30<204:17:05,  4.84s/it]
 33%|███▎      | 73145/225000 [145:58:32<173:37:52,  4.12s/it]
 33%|███▎      | 73146/225000 [145:58:35<163:43:36,  3.88s/it]
 33%|███▎      | 73147/225000 [145:58:44<219:59:31,  5.22s/it]
 33%|███▎      | 73148/225000 [145:58:47<201:12:28,  4.77s/it]
 33%|███▎      | 73149/225000 [145:58:50<173:29:48,  4.11s/it]
 33%|███▎      | 73150/225000 [145:58:53<161:10:42,  3.82s/it]
                                                              
{'loss': '0.8214', 'grad_norm': '2.304', 'learning_rate': '4.01e-05', 'epoch': '0.3251', 'num_input_tokens_seen': 241653736, 'train_runtime': '5.255e+05', 'train_tokens_per_second': '459.8'}

 33%|███▎      | 73150/225000 [145:58:53<161:10:42,  3.82s/it]
 33%|███▎      | 73151/225000 [145:58:56<151:48:59,  3.60s/it]
 33%|███▎      | 73152/225000 [145:58:59<148:34:27,  3.52s/it]
 33%|███▎      | 73153/225000 [145:59:03<151:57:22,  3.60s/it]
 33%|███▎      | 73154/225000 [145:59:12<217:35:54,  5.16s/it]
 33%|███▎      | 73155/225000 [145:59:18<233:14:35,  5.53s/it]
 33%|███▎      | 73156/225000 [145:59:21<199:31:59,  4.73s/it]
 33%|███▎      | 73157/225000 [145:59:24<176:54:42,  4.19s/it]
 33%|███▎      | 73158/225000 [145:59:28<171:22:12,  4.06s/it]
 33%|███▎      | 73159/225000 [145:59:37<231:08:06,  5.48s/it]
 33%|███▎      | 73160/225000 [145:59:40<204:02:08,  4.84s/it]
 33%|███▎      | 73161/225000 [145:59:43<181:19:35,  4.30s/it]
 33%|███▎      | 73162/225000 [145:59:49<203:22:16,  4.82s/it]
 33%|███▎      | 73163/225000 [145:59:54<206:58:12,  4.91s/it]
 33%|███▎      | 73164/225000 [145:59:58<189:36:40,  4.50s/it]
 33%|███▎      | 73165/225000 [146:00:02<180:30:16,  4.28s/it]
 33%|███▎      | 73166/225000 [146:00:05<166:45:05,  3.95s/it]
 33%|███▎      | 73167/225000 [146:00:09<163:34:33,  3.88s/it]
 33%|███▎      | 73168/225000 [146:00:12<153:35:28,  3.64s/it]
 33%|███▎      | 73169/225000 [146:00:15<151:27:02,  3.59s/it]
 33%|███▎      | 73170/225000 [146:00:18<146:02:05,  3.46s/it]
 33%|███▎      | 73171/225000 [146:00:22<152:16:00,  3.61s/it]
 33%|███▎      | 73172/225000 [146:00:31<221:39:50,  5.26s/it]
 33%|███▎      | 73173/225000 [146:00:35<206:42:19,  4.90s/it]
 33%|███▎      | 73174/225000 [146:00:41<214:13:18,  5.08s/it]
 33%|███▎      | 73175/225000 [146:00:46<209:44:20,  4.97s/it]
 33%|███▎      | 73176/225000 [146:00:48<182:51:03,  4.34s/it]
 33%|███▎      | 73177/225000 [146:00:52<169:12:30,  4.01s/it]
 33%|███▎      | 73178/225000 [146:01:00<216:58:21,  5.14s/it]
 33%|███▎      | 73179/225000 [146:01:03<196:40:48,  4.66s/it]
 33%|███▎      | 73180/225000 [146:01:06<172:45:10,  4.10s/it]
 33%|███▎      | 73181/225000 [146:01:08<154:41:26,  3.67s/it]
 33%|███▎      | 73182/225000 [146:01:12<149:46:34,  3.55s/it]
 33%|███▎      | 73183/225000 [146:01:21<226:30:34,  5.37s/it]
 33%|███▎      | 73184/225000 [146:01:24<191:18:59,  4.54s/it]
 33%|███▎      | 73185/225000 [146:01:26<163:21:55,  3.87s/it]
 33%|███▎      | 73186/225000 [146:01:32<191:45:59,  4.55s/it]
 33%|███▎      | 73187/225000 [146:01:35<165:29:56,  3.92s/it]
 33%|███▎      | 73188/225000 [146:01:38<150:44:40,  3.57s/it]
 33%|███▎      | 73189/225000 [146:01:46<206:46:54,  4.90s/it]
 33%|███▎      | 73190/225000 [146:01:49<182:01:54,  4.32s/it]
 33%|███▎      | 73191/225000 [146:01:51<158:50:19,  3.77s/it]
 33%|███▎      | 73192/225000 [146:01:54<148:26:15,  3.52s/it]
 33%|███▎      | 73193/225000 [146:01:58<152:18:07,  3.61s/it]
 33%|███▎      | 73194/225000 [146:02:01<147:40:25,  3.50s/it]
 33%|███▎      | 73195/225000 [146:02:05<147:58:00,  3.51s/it]
 33%|███▎      | 73196/225000 [146:02:10<174:01:30,  4.13s/it]
 33%|███▎      | 73197/225000 [146:02:18<215:46:17,  5.12s/it]
 33%|███▎      | 73198/225000 [146:02:22<207:21:30,  4.92s/it]
 33%|███▎      | 73199/225000 [146:02:25<183:12:51,  4.34s/it]
 33%|███▎      | 73200/225000 [146:02:29<175:12:07,  4.15s/it]
                                                              
{'loss': '0.8409', 'grad_norm': '1.859', 'learning_rate': '4.009e-05', 'epoch': '0.3253', 'num_input_tokens_seen': 241741696, 'train_runtime': '5.257e+05', 'train_tokens_per_second': '459.8'}

 33%|███▎      | 73200/225000 [146:02:29<175:12:07,  4.15s/it]
 33%|███▎      | 73201/225000 [146:02:32<164:30:07,  3.90s/it]
 33%|███▎      | 73202/225000 [146:02:39<199:50:26,  4.74s/it]
 33%|███▎      | 73203/225000 [146:02:46<235:47:17,  5.59s/it]
 33%|███▎      | 73204/225000 [146:02:49<197:48:24,  4.69s/it]
 33%|███▎      | 73205/225000 [146:02:52<171:31:00,  4.07s/it]
 33%|███▎      | 73206/225000 [146:02:55<158:01:49,  3.75s/it]
 33%|███▎      | 73207/225000 [146:02:59<170:50:20,  4.05s/it]
 33%|███▎      | 73208/225000 [146:03:07<214:36:25,  5.09s/it]
 33%|███▎      | 73209/225000 [146:03:11<202:06:41,  4.79s/it]
 33%|███▎      | 73210/225000 [146:03:15<188:01:30,  4.46s/it]
 33%|███▎      | 73211/225000 [146:03:19<185:14:42,  4.39s/it]
 33%|███▎      | 73212/225000 [146:03:22<170:13:52,  4.04s/it]
 33%|███▎      | 73213/225000 [146:03:26<162:58:37,  3.87s/it]
 33%|███▎      | 73214/225000 [146:03:28<150:27:53,  3.57s/it]
 33%|███▎      | 73215/225000 [146:03:33<161:18:18,  3.83s/it]
 33%|███▎      | 73216/225000 [146:03:36<152:09:07,  3.61s/it]
 33%|███▎      | 73217/225000 [146:03:45<224:02:35,  5.31s/it]
 33%|███▎      | 73218/225000 [146:03:53<253:13:22,  6.01s/it]
 33%|███▎      | 73219/225000 [146:03:55<209:39:06,  4.97s/it]
 33%|███▎      | 73220/225000 [146:03:58<182:44:55,  4.33s/it]
 33%|███▎      | 73221/225000 [146:04:01<162:43:12,  3.86s/it]
 33%|███▎      | 73222/225000 [146:04:10<225:03:42,  5.34s/it]
 33%|███▎      | 73223/225000 [146:04:15<220:13:00,  5.22s/it]
 33%|███▎      | 73224/225000 [146:04:18<188:38:30,  4.47s/it]
 33%|███▎      | 73225/225000 [146:04:20<163:38:16,  3.88s/it]
 33%|███▎      | 73226/225000 [146:04:23<149:17:45,  3.54s/it]
 33%|███▎      | 73227/225000 [146:04:32<220:15:42,  5.22s/it]
 33%|███▎      | 73228/225000 [146:04:35<190:43:51,  4.52s/it]
 33%|███▎      | 73229/225000 [146:04:38<170:46:37,  4.05s/it]
 33%|███▎      | 73230/225000 [146:04:46<220:43:30,  5.24s/it]
 33%|███▎      | 73231/225000 [146:04:48<188:16:34,  4.47s/it]
 33%|███▎      | 73232/225000 [146:04:51<168:54:16,  4.01s/it]
 33%|███▎      | 73233/225000 [146:04:59<213:09:42,  5.06s/it]
 33%|███▎      | 73234/225000 [146:05:03<203:47:52,  4.83s/it]
 33%|███▎      | 73235/225000 [146:05:06<176:49:24,  4.19s/it]
 33%|███▎      | 73236/225000 [146:05:08<154:55:49,  3.68s/it]
 33%|███▎      | 73237/225000 [146:05:11<142:07:45,  3.37s/it]
 33%|███▎      | 73238/225000 [146:05:14<134:34:18,  3.19s/it]
 33%|███▎      | 73239/225000 [146:05:17<135:23:38,  3.21s/it]
 33%|███▎      | 73240/225000 [146:05:26<205:22:07,  4.87s/it]
 33%|███▎      | 73241/225000 [146:05:29<190:01:23,  4.51s/it]
 33%|███▎      | 73242/225000 [146:05:32<160:41:36,  3.81s/it]
 33%|███▎      | 73243/225000 [146:05:34<147:05:02,  3.49s/it]
 33%|███▎      | 73244/225000 [146:05:41<185:01:27,  4.39s/it]
 33%|███▎      | 73245/225000 [146:05:47<210:05:17,  4.98s/it]
 33%|███▎      | 73246/225000 [146:05:50<187:35:21,  4.45s/it]
 33%|███▎      | 73247/225000 [146:05:54<170:52:40,  4.05s/it]
 33%|███▎      | 73248/225000 [146:05:57<160:05:33,  3.80s/it]
 33%|███▎      | 73249/225000 [146:05:59<145:17:50,  3.45s/it]
 33%|███▎      | 73250/225000 [146:06:03<148:53:41,  3.53s/it]
                                                              
{'loss': '0.8477', 'grad_norm': '1.771', 'learning_rate': '4.007e-05', 'epoch': '0.3256', 'num_input_tokens_seen': 241830584, 'train_runtime': '5.26e+05', 'train_tokens_per_second': '459.8'}

 33%|███▎      | 73250/225000 [146:06:03<148:53:41,  3.53s/it]
 33%|███▎      | 73251/225000 [146:06:10<196:19:36,  4.66s/it]
 33%|███▎      | 73252/225000 [146:06:15<201:36:47,  4.78s/it]
 33%|███▎      | 73253/225000 [146:06:19<184:35:58,  4.38s/it]
 33%|███▎      | 73254/225000 [146:06:23<176:38:14,  4.19s/it]
 33%|███▎      | 73255/225000 [146:06:26<169:27:21,  4.02s/it]
 33%|███▎      | 73256/225000 [146:06:29<153:12:01,  3.63s/it]
 33%|███▎      | 73257/225000 [146:06:32<147:13:26,  3.49s/it]
 33%|███▎      | 73258/225000 [146:06:35<144:27:15,  3.43s/it]
 33%|███▎      | 73259/225000 [146:06:39<141:04:08,  3.35s/it]
 33%|███▎      | 73260/225000 [146:06:42<137:33:40,  3.26s/it]
 33%|███▎      | 73261/225000 [146:06:46<150:34:07,  3.57s/it]
 33%|███▎      | 73262/225000 [146:06:54<203:45:17,  4.83s/it]
 33%|███▎      | 73263/225000 [146:07:00<218:55:10,  5.19s/it]
 33%|███▎      | 73264/225000 [146:07:05<225:06:08,  5.34s/it]
 33%|███▎      | 73265/225000 [146:07:09<198:06:48,  4.70s/it]
 33%|███▎      | 73266/225000 [146:07:12<175:42:03,  4.17s/it]
 33%|███▎      | 73267/225000 [146:07:14<155:35:56,  3.69s/it]
 33%|███▎      | 73268/225000 [146:07:23<216:28:48,  5.14s/it]
 33%|███▎      | 73269/225000 [146:07:27<201:00:59,  4.77s/it]
 33%|███▎      | 73270/225000 [146:07:29<174:26:14,  4.14s/it]
 33%|███▎      | 73271/225000 [146:07:32<155:28:49,  3.69s/it]
 33%|███▎      | 73272/225000 [146:07:42<240:16:25,  5.70s/it]
 33%|███▎      | 73273/225000 [146:07:45<206:27:06,  4.90s/it]
 33%|███▎      | 73274/225000 [146:07:49<195:59:30,  4.65s/it]
 33%|███▎      | 73275/225000 [146:07:52<175:08:05,  4.16s/it]
 33%|███▎      | 73276/225000 [146:07:56<171:31:28,  4.07s/it]
 33%|███▎      | 73277/225000 [146:08:00<161:34:14,  3.83s/it]
 33%|███▎      | 73278/225000 [146:08:04<163:47:53,  3.89s/it]
 33%|███▎      | 73279/225000 [146:08:07<161:47:55,  3.84s/it]
 33%|███▎      | 73280/225000 [146:08:11<163:02:08,  3.87s/it]
 33%|███▎      | 73281/225000 [146:08:19<215:18:06,  5.11s/it]
 33%|███▎      | 73282/225000 [146:08:25<229:53:59,  5.46s/it]
 33%|███▎      | 73283/225000 [146:08:28<197:51:00,  4.69s/it]
 33%|███▎      | 73284/225000 [146:08:35<223:38:48,  5.31s/it]
 33%|███▎      | 73285/225000 [146:08:39<200:58:33,  4.77s/it]
 33%|███▎      | 73286/225000 [146:08:42<177:48:31,  4.22s/it]
 33%|███▎      | 73287/225000 [146:08:46<185:40:02,  4.41s/it]
 33%|███▎      | 73288/225000 [146:08:51<190:10:51,  4.51s/it]
 33%|███▎      | 73289/225000 [146:08:54<168:02:56,  3.99s/it]
 33%|███▎      | 73290/225000 [146:08:57<151:36:42,  3.60s/it]
 33%|███▎      | 73291/225000 [146:09:00<148:30:48,  3.52s/it]
 33%|███▎      | 73292/225000 [146:09:03<138:55:20,  3.30s/it]
 33%|███▎      | 73293/225000 [146:09:06<134:39:01,  3.20s/it]
 33%|███▎      | 73294/225000 [146:09:13<187:53:28,  4.46s/it]
 33%|███▎      | 73295/225000 [146:09:20<218:08:44,  5.18s/it]
 33%|███▎      | 73296/225000 [146:09:23<190:21:22,  4.52s/it]
 33%|███▎      | 73297/225000 [146:09:26<171:15:05,  4.06s/it]
 33%|███▎      | 73298/225000 [146:09:33<214:30:24,  5.09s/it]
 33%|███▎      | 73299/225000 [146:09:38<212:50:57,  5.05s/it]
 33%|███▎      | 73300/225000 [146:09:41<181:22:18,  4.30s/it]
                                                              
{'loss': '0.8486', 'grad_norm': '1.595', 'learning_rate': '4.006e-05', 'epoch': '0.3258', 'num_input_tokens_seen': 241918448, 'train_runtime': '5.262e+05', 'train_tokens_per_second': '459.8'}

 33%|███▎      | 73300/225000 [146:09:41<181:22:18,  4.30s/it]
 33%|███▎      | 73301/225000 [146:09:44<163:05:55,  3.87s/it]
 33%|███▎      | 73302/225000 [146:09:51<201:44:59,  4.79s/it]
 33%|███▎      | 73303/225000 [146:09:55<200:24:04,  4.76s/it]
 33%|███▎      | 73304/225000 [146:09:58<172:28:59,  4.09s/it]
 33%|███▎      | 73305/225000 [146:10:03<179:38:34,  4.26s/it]
 33%|███▎      | 73306/225000 [146:10:08<192:42:52,  4.57s/it]
 33%|███▎      | 73307/225000 [146:10:10<163:45:58,  3.89s/it]
 33%|███▎      | 73308/225000 [146:10:13<152:24:25,  3.62s/it]
 33%|███▎      | 73309/225000 [146:10:17<149:36:18,  3.55s/it]
 33%|███▎      | 73310/225000 [146:10:20<148:15:32,  3.52s/it]
 33%|███▎      | 73311/225000 [146:10:23<144:37:52,  3.43s/it]
 33%|███▎      | 73312/225000 [146:10:27<143:16:31,  3.40s/it]
 33%|███▎      | 73313/225000 [146:10:31<152:47:53,  3.63s/it]
 33%|███▎      | 73314/225000 [146:10:35<157:45:58,  3.74s/it]
 33%|███▎      | 73315/225000 [146:10:42<204:16:56,  4.85s/it]
 33%|███▎      | 73316/225000 [146:10:51<258:18:30,  6.13s/it]
 33%|███▎      | 73317/225000 [146:10:55<221:46:30,  5.26s/it]
 33%|███▎      | 73318/225000 [146:10:58<201:20:19,  4.78s/it]
 33%|███▎      | 73319/225000 [146:11:01<178:53:50,  4.25s/it]
 33%|███▎      | 73320/225000 [146:11:07<200:26:20,  4.76s/it]
 33%|███▎      | 73321/225000 [146:11:11<192:53:53,  4.58s/it]
 33%|███▎      | 73322/225000 [146:11:14<169:03:48,  4.01s/it]
 33%|███▎      | 73323/225000 [146:11:17<155:03:29,  3.68s/it]
 33%|███▎      | 73324/225000 [146:11:20<141:23:28,  3.36s/it]
 33%|███▎      | 73325/225000 [146:11:27<195:01:50,  4.63s/it]
 33%|███▎      | 73326/225000 [146:11:31<191:06:18,  4.54s/it]
 33%|███▎      | 73327/225000 [146:11:34<166:01:18,  3.94s/it]
 33%|███▎      | 73328/225000 [146:11:37<151:39:47,  3.60s/it]
 33%|███▎      | 73329/225000 [146:11:42<171:46:46,  4.08s/it]
 33%|███▎      | 73330/225000 [146:11:48<198:45:45,  4.72s/it]
 33%|███▎      | 73331/225000 [146:11:52<190:26:05,  4.52s/it]
 33%|███▎      | 73332/225000 [146:11:58<207:30:27,  4.93s/it]
 33%|███▎      | 73333/225000 [146:12:02<187:42:47,  4.46s/it]
 33%|███▎      | 73334/225000 [146:12:05<171:55:13,  4.08s/it]
 33%|███▎      | 73335/225000 [146:12:08<157:20:46,  3.73s/it]
 33%|███▎      | 73336/225000 [146:12:11<150:10:15,  3.56s/it]
 33%|███▎      | 73337/225000 [146:12:13<139:10:53,  3.30s/it]
 33%|███▎      | 73338/225000 [146:12:16<130:32:09,  3.10s/it]
 33%|███▎      | 73339/225000 [146:12:19<129:39:10,  3.08s/it]
 33%|███▎      | 73340/225000 [146:12:23<134:30:22,  3.19s/it]
 33%|███▎      | 73341/225000 [146:12:30<189:35:10,  4.50s/it]
 33%|███▎      | 73342/225000 [146:12:39<245:50:08,  5.84s/it]
 33%|███▎      | 73343/225000 [146:12:41<201:27:12,  4.78s/it]
 33%|███▎      | 73344/225000 [146:12:45<182:52:58,  4.34s/it]
 33%|███▎      | 73345/225000 [146:12:47<157:57:43,  3.75s/it]
 33%|███▎      | 73346/225000 [146:12:55<211:24:10,  5.02s/it]
 33%|███▎      | 73347/225000 [146:12:59<191:32:15,  4.55s/it]
 33%|███▎      | 73348/225000 [146:13:03<188:02:50,  4.46s/it]
 33%|███▎      | 73349/225000 [146:13:10<217:40:50,  5.17s/it]
 33%|███▎      | 73350/225000 [146:13:12<182:45:53,  4.34s/it]
                                                              
{'loss': '0.8216', 'grad_norm': '1.745', 'learning_rate': '4.004e-05', 'epoch': '0.326', 'num_input_tokens_seen': 242004880, 'train_runtime': '5.264e+05', 'train_tokens_per_second': '459.7'}

 33%|███▎      | 73350/225000 [146:13:12<182:45:53,  4.34s/it]
 33%|███▎      | 73351/225000 [146:13:18<201:12:49,  4.78s/it]
 33%|███▎      | 73352/225000 [146:13:20<171:35:22,  4.07s/it]
 33%|███▎      | 73353/225000 [146:13:26<189:55:56,  4.51s/it]
 33%|███▎      | 73354/225000 [146:13:29<168:43:07,  4.01s/it]
 33%|███▎      | 73355/225000 [146:13:32<163:07:50,  3.87s/it]
 33%|███▎      | 73356/225000 [146:13:36<167:16:07,  3.97s/it]
 33%|███▎      | 73357/225000 [146:13:40<158:35:39,  3.77s/it]
 33%|███▎      | 73358/225000 [146:13:44<161:30:37,  3.83s/it]
 33%|███▎      | 73359/225000 [146:13:47<153:07:17,  3.64s/it]
 33%|███▎      | 73360/225000 [146:13:50<153:24:49,  3.64s/it]
 33%|███▎      | 73361/225000 [146:13:53<142:29:33,  3.38s/it]
 33%|███▎      | 73362/225000 [146:13:56<138:09:42,  3.28s/it]
 33%|███▎      | 73363/225000 [146:14:04<198:53:10,  4.72s/it]
 33%|███▎      | 73364/225000 [146:14:11<225:01:47,  5.34s/it]
 33%|███▎      | 73365/225000 [146:14:14<194:25:28,  4.62s/it]
 33%|███▎      | 73366/225000 [146:14:21<217:12:52,  5.16s/it]
 33%|███▎      | 73367/225000 [146:14:24<198:46:18,  4.72s/it]
 33%|███▎      | 73368/225000 [146:14:27<178:14:27,  4.23s/it]
 33%|███▎      | 73369/225000 [146:14:31<170:27:09,  4.05s/it]
 33%|███▎      | 73370/225000 [146:14:34<160:37:15,  3.81s/it]
 33%|███▎      | 73371/225000 [146:14:38<159:00:41,  3.78s/it]
 33%|███▎      | 73372/225000 [146:14:41<146:48:56,  3.49s/it]
 33%|███▎      | 73373/225000 [146:14:47<187:51:55,  4.46s/it]
 33%|███▎      | 73374/225000 [146:14:56<244:51:51,  5.81s/it]
 33%|███▎      | 73375/225000 [146:14:59<206:14:48,  4.90s/it]
 33%|███▎      | 73376/225000 [146:15:02<181:44:38,  4.32s/it]
 33%|███▎      | 73377/225000 [146:15:05<165:42:33,  3.93s/it]
 33%|███▎      | 73378/225000 [146:15:10<175:51:06,  4.18s/it]
 33%|███▎      | 73379/225000 [146:15:13<167:47:39,  3.98s/it]
 33%|███▎      | 73380/225000 [146:15:17<163:17:19,  3.88s/it]
 33%|███▎      | 73381/225000 [146:15:24<202:39:28,  4.81s/it]
 33%|███▎      | 73382/225000 [146:15:27<176:49:29,  4.20s/it]
 33%|███▎      | 73383/225000 [146:15:33<201:59:36,  4.80s/it]
 33%|███▎      | 73384/225000 [146:15:36<177:04:16,  4.20s/it]
 33%|███▎      | 73385/225000 [146:15:40<171:27:14,  4.07s/it]
 33%|███▎      | 73386/225000 [146:15:43<158:50:39,  3.77s/it]
 33%|███▎      | 73387/225000 [146:15:46<147:03:37,  3.49s/it]
 33%|███▎      | 73388/225000 [146:15:49<145:00:50,  3.44s/it]
 33%|███▎      | 73389/225000 [146:15:52<143:22:11,  3.40s/it]
 33%|███▎      | 73390/225000 [146:15:59<183:46:22,  4.36s/it]
 33%|███▎      | 73391/225000 [146:16:07<235:29:38,  5.59s/it]
 33%|███▎      | 73392/225000 [146:16:10<194:05:43,  4.61s/it]
 33%|███▎      | 73393/225000 [146:16:12<172:21:49,  4.09s/it]
 33%|███▎      | 73394/225000 [146:16:20<216:24:45,  5.14s/it]
 33%|███▎      | 73395/225000 [146:16:23<184:05:12,  4.37s/it]
 33%|███▎      | 73396/225000 [146:16:25<161:17:22,  3.83s/it]
 33%|███▎      | 73397/225000 [146:16:28<147:48:35,  3.51s/it]
 33%|███▎      | 73398/225000 [146:16:35<193:45:39,  4.60s/it]
 33%|███▎      | 73399/225000 [146:16:39<180:36:29,  4.29s/it]
 33%|███▎      | 73400/225000 [146:16:41<158:34:10,  3.77s/it]
                                                              
{'loss': '0.8395', 'grad_norm': '1.925', 'learning_rate': '4.003e-05', 'epoch': '0.3262', 'num_input_tokens_seen': 242091752, 'train_runtime': '5.266e+05', 'train_tokens_per_second': '459.7'}

 33%|███▎      | 73400/225000 [146:16:41<158:34:10,  3.77s/it]
 33%|███▎      | 73401/225000 [146:16:44<144:14:10,  3.43s/it]
 33%|███▎      | 73402/225000 [146:16:47<137:26:09,  3.26s/it]
 33%|███▎      | 73403/225000 [146:16:53<178:42:23,  4.24s/it]
 33%|███▎      | 73404/225000 [146:17:00<212:21:37,  5.04s/it]
 33%|███▎      | 73405/225000 [146:17:04<199:33:38,  4.74s/it]
 33%|███▎      | 73406/225000 [146:17:08<184:14:32,  4.38s/it]
 33%|███▎      | 73407/225000 [146:17:11<165:11:21,  3.92s/it]
 33%|███▎      | 73408/225000 [146:17:14<156:45:10,  3.72s/it]
 33%|███▎      | 73409/225000 [146:17:17<154:45:09,  3.68s/it]
 33%|███▎      | 73410/225000 [146:17:21<154:25:51,  3.67s/it]
 33%|███▎      | 73411/225000 [146:17:25<163:19:38,  3.88s/it]
 33%|███▎      | 73412/225000 [146:17:36<246:46:42,  5.86s/it]
 33%|███▎      | 73413/225000 [146:17:40<229:44:33,  5.46s/it]
 33%|███▎      | 73414/225000 [146:17:43<195:52:14,  4.65s/it]
 33%|███▎      | 73415/225000 [146:17:48<195:36:17,  4.65s/it]
 33%|███▎      | 73416/225000 [146:17:54<210:25:11,  5.00s/it]
 33%|███▎      | 73417/225000 [146:17:57<186:07:34,  4.42s/it]
 33%|███▎      | 73418/225000 [146:17:59<165:24:31,  3.93s/it]
 33%|███▎      | 73419/225000 [146:18:02<147:04:07,  3.49s/it]
 33%|███▎      | 73420/225000 [146:18:11<220:04:33,  5.23s/it]
 33%|███▎      | 73421/225000 [146:18:16<215:03:40,  5.11s/it]
 33%|███▎      | 73422/225000 [146:18:19<184:57:36,  4.39s/it]
 33%|███▎      | 73423/225000 [146:18:21<163:32:28,  3.88s/it]
 33%|███▎      | 73424/225000 [146:18:28<201:45:37,  4.79s/it]
 33%|███▎      | 73425/225000 [146:18:33<200:38:23,  4.77s/it]
 33%|███▎      | 73426/225000 [146:18:36<171:21:24,  4.07s/it]
 33%|███▎      | 73427/225000 [146:18:39<169:20:05,  4.02s/it]
 33%|███▎      | 73428/225000 [146:18:45<183:44:02,  4.36s/it]
 33%|███▎      | 73429/225000 [146:18:48<168:43:34,  4.01s/it]
 33%|███▎      | 73430/225000 [146:18:51<156:10:39,  3.71s/it]
 33%|███▎      | 73431/225000 [146:18:56<170:26:25,  4.05s/it]
 33%|███▎      | 73432/225000 [146:18:59<156:55:15,  3.73s/it]
 33%|███▎      | 73433/225000 [146:19:02<155:01:26,  3.68s/it]
 33%|███▎      | 73434/225000 [146:19:05<138:49:46,  3.30s/it]
 33%|███▎      | 73435/225000 [146:19:11<182:00:21,  4.32s/it]
 33%|███▎      | 73436/225000 [146:19:16<188:22:41,  4.47s/it]
 33%|███▎      | 73437/225000 [146:19:19<165:51:27,  3.94s/it]
 33%|███▎      | 73438/225000 [146:19:22<158:48:32,  3.77s/it]
 33%|███▎      | 73439/225000 [146:19:25<145:42:11,  3.46s/it]
 33%|███▎      | 73440/225000 [146:19:33<205:23:18,  4.88s/it]
 33%|███▎      | 73441/225000 [146:19:37<196:32:06,  4.67s/it]
 33%|███▎      | 73442/225000 [146:19:40<178:03:05,  4.23s/it]
 33%|███▎      | 73443/225000 [146:19:43<162:23:04,  3.86s/it]
 33%|███▎      | 73444/225000 [146:19:47<155:33:59,  3.70s/it]
 33%|███▎      | 73445/225000 [146:19:50<145:09:05,  3.45s/it]
 33%|███▎      | 73446/225000 [146:19:52<134:48:14,  3.20s/it]
 33%|███▎      | 73447/225000 [146:19:55<133:14:38,  3.17s/it]
 33%|███▎      | 73448/225000 [146:20:01<163:52:35,  3.89s/it]
 33%|███▎      | 73449/225000 [146:20:10<230:59:11,  5.49s/it]
 33%|███▎      | 73450/225000 [146:20:13<198:44:52,  4.72s/it]
                                                              
{'loss': '0.8233', 'grad_norm': '1.066', 'learning_rate': '4.001e-05', 'epoch': '0.3264', 'num_input_tokens_seen': 242178576, 'train_runtime': '5.268e+05', 'train_tokens_per_second': '459.7'}

 33%|███▎      | 73450/225000 [146:20:13<198:44:52,  4.72s/it]
 33%|███▎      | 73451/225000 [146:20:16<176:12:05,  4.19s/it]
 33%|███▎      | 73452/225000 [146:20:19<166:30:44,  3.96s/it]
 33%|███▎      | 73453/225000 [146:20:22<150:01:07,  3.56s/it]
 33%|███▎      | 73454/225000 [146:20:25<145:02:44,  3.45s/it]
 33%|███▎      | 73455/225000 [146:20:34<211:29:25,  5.02s/it]
 33%|███▎      | 73456/225000 [146:20:38<194:06:44,  4.61s/it]
 33%|███▎      | 73457/225000 [146:20:41<172:35:35,  4.10s/it]
 33%|███▎      | 73458/225000 [146:20:46<187:53:11,  4.46s/it]
 33%|███▎      | 73459/225000 [146:20:52<210:23:10,  5.00s/it]
 33%|███▎      | 73460/225000 [146:20:55<182:59:04,  4.35s/it]
 33%|███▎      | 73461/225000 [146:20:57<159:16:28,  3.78s/it]
 33%|███▎      | 73462/225000 [146:21:01<152:13:14,  3.62s/it]
 33%|███▎      | 73463/225000 [146:21:04<146:05:06,  3.47s/it]
 33%|███▎      | 73464/225000 [146:21:09<165:31:54,  3.93s/it]
 33%|███▎      | 73465/225000 [146:21:16<211:34:41,  5.03s/it]
 33%|███▎      | 73466/225000 [146:21:20<189:49:12,  4.51s/it]
 33%|███▎      | 73467/225000 [146:21:23<170:36:45,  4.05s/it]
 33%|███▎      | 73468/225000 [146:21:26<156:56:22,  3.73s/it]
 33%|███▎      | 73469/225000 [146:21:28<144:26:43,  3.43s/it]
 33%|███▎      | 73470/225000 [146:21:31<136:24:11,  3.24s/it]
 33%|███▎      | 73471/225000 [146:21:35<140:02:12,  3.33s/it]
 33%|███▎      | 73472/225000 [146:21:38<145:16:15,  3.45s/it]
 33%|███▎      | 73473/225000 [146:21:42<144:04:52,  3.42s/it]
 33%|███▎      | 73474/225000 [146:21:45<135:53:05,  3.23s/it]
 33%|███▎      | 73475/225000 [146:21:49<152:06:18,  3.61s/it]
 33%|███▎      | 73476/225000 [146:21:59<232:22:58,  5.52s/it]
 33%|███▎      | 73477/225000 [146:22:02<195:03:20,  4.63s/it]
 33%|███▎      | 73478/225000 [146:22:05<177:54:58,  4.23s/it]
 33%|███▎      | 73479/225000 [146:22:08<166:15:21,  3.95s/it]
 33%|███▎      | 73480/225000 [146:22:13<181:12:13,  4.31s/it]
 33%|███▎      | 73481/225000 [146:22:22<235:26:36,  5.59s/it]
 33%|███▎      | 73482/225000 [146:22:25<199:34:35,  4.74s/it]
 33%|███▎      | 73483/225000 [146:22:28<182:10:52,  4.33s/it]
 33%|███▎      | 73484/225000 [146:22:35<209:55:23,  4.99s/it]
 33%|███▎      | 73485/225000 [146:22:39<196:50:00,  4.68s/it]
 33%|███▎      | 73486/225000 [146:22:41<172:17:40,  4.09s/it]
 33%|███▎      | 73487/225000 [146:22:48<208:35:29,  4.96s/it]
 33%|███▎      | 73488/225000 [146:22:52<188:53:45,  4.49s/it]
 33%|███▎      | 73489/225000 [146:22:55<169:13:21,  4.02s/it]
 33%|███▎      | 73490/225000 [146:22:58<155:59:24,  3.71s/it]
 33%|███▎      | 73491/225000 [146:23:01<152:52:09,  3.63s/it]
 33%|███▎      | 73492/225000 [146:23:04<145:35:35,  3.46s/it]
 33%|███▎      | 73493/225000 [146:23:10<172:35:32,  4.10s/it]
 33%|███▎      | 73494/225000 [146:23:18<224:13:28,  5.33s/it]
 33%|███▎      | 73495/225000 [146:23:21<194:52:25,  4.63s/it]
 33%|███▎      | 73496/225000 [146:23:24<172:57:45,  4.11s/it]
 33%|███▎      | 73497/225000 [146:23:27<161:26:09,  3.84s/it]
 33%|███▎      | 73498/225000 [146:23:31<159:08:09,  3.78s/it]
 33%|███▎      | 73499/225000 [146:23:34<157:16:35,  3.74s/it]
 33%|███▎      | 73500/225000 [146:23:43<225:26:26,  5.36s/it]
                                                              
{'loss': '0.833', 'grad_norm': '1.707', 'learning_rate': '4e-05', 'epoch': '0.3267', 'num_input_tokens_seen': 242265424, 'train_runtime': '5.27e+05', 'train_tokens_per_second': '459.7'}

 33%|███▎      | 73500/225000 [146:23:43<225:26:26,  5.36s/it]
 33%|███▎      | 73501/225000 [146:23:47<199:49:11,  4.75s/it]
 33%|███▎      | 73502/225000 [146:23:50<177:28:29,  4.22s/it]
 33%|███▎      | 73503/225000 [146:23:58<228:34:54,  5.43s/it]
 33%|███▎      | 73504/225000 [146:24:00<191:36:34,  4.55s/it]
 33%|███▎      | 73505/225000 [146:24:03<168:34:34,  4.01s/it]
 33%|███▎      | 73506/225000 [146:24:09<191:57:44,  4.56s/it]
 33%|███▎      | 73507/225000 [146:24:13<182:21:22,  4.33s/it]
 33%|███▎      | 73508/225000 [146:24:15<160:31:28,  3.81s/it]
 33%|███▎      | 73509/225000 [146:24:18<146:02:58,  3.47s/it]
 33%|███▎      | 73510/225000 [146:24:21<143:25:30,  3.41s/it]
 33%|███▎      | 73511/225000 [146:24:24<133:32:34,  3.17s/it]
 33%|███▎      | 73512/225000 [146:24:27<136:40:38,  3.25s/it]
 33%|███▎      | 73513/225000 [146:24:31<137:55:50,  3.28s/it]
 33%|███▎      | 73514/225000 [146:24:34<135:42:23,  3.23s/it]
 33%|███▎      | 73515/225000 [146:24:37<137:23:38,  3.27s/it]
 33%|███▎      | 73516/225000 [146:24:40<135:55:38,  3.23s/it]
 33%|███▎      | 73517/225000 [146:24:44<145:41:50,  3.46s/it]
 33%|███▎      | 73518/225000 [146:24:53<206:15:44,  4.90s/it]
 33%|███▎      | 73519/225000 [146:25:00<237:12:58,  5.64s/it]
 33%|███▎      | 73520/225000 [146:25:02<196:06:39,  4.66s/it]
 33%|███▎      | 73521/225000 [146:25:05<172:15:40,  4.09s/it]
 33%|███▎      | 73522/225000 [146:25:13<219:43:14,  5.22s/it]
 33%|███▎      | 73523/225000 [146:25:16<192:34:36,  4.58s/it]
 33%|███▎      | 73524/225000 [146:25:19<169:00:06,  4.02s/it]
 33%|███▎      | 73525/225000 [146:25:22<158:20:35,  3.76s/it]
 33%|███▎      | 73526/225000 [146:25:28<183:25:20,  4.36s/it]
 33%|███▎      | 73527/225000 [146:25:33<196:09:52,  4.66s/it]
 33%|███▎      | 73528/225000 [146:25:36<174:27:33,  4.15s/it]
 33%|███▎      | 73529/225000 [146:25:38<151:07:55,  3.59s/it]
 33%|███▎      | 73530/225000 [146:25:42<149:18:50,  3.55s/it]
 33%|███▎      | 73531/225000 [146:25:45<144:24:18,  3.43s/it]
 33%|███▎      | 73532/225000 [146:25:48<138:44:05,  3.30s/it]
 33%|███▎      | 73533/225000 [146:25:51<142:14:00,  3.38s/it]
 33%|███▎      | 73534/225000 [146:25:55<146:47:48,  3.49s/it]
 33%|███▎      | 73535/225000 [146:26:04<220:07:11,  5.23s/it]
 33%|███▎      | 73536/225000 [146:26:11<236:34:18,  5.62s/it]
 33%|███▎      | 73537/225000 [146:26:14<204:36:59,  4.86s/it]
 33%|███▎      | 73538/225000 [146:26:17<185:05:06,  4.40s/it]
 33%|███▎      | 73539/225000 [146:26:21<170:12:29,  4.05s/it]
 33%|███▎      | 73540/225000 [146:26:23<152:11:04,  3.62s/it]
 33%|███▎      | 73541/225000 [146:26:28<169:43:18,  4.03s/it]
 33%|███▎      | 73542/225000 [146:26:39<248:35:33,  5.91s/it]
 33%|███▎      | 73543/225000 [146:26:42<215:00:35,  5.11s/it]
 33%|███▎      | 73544/225000 [146:26:46<203:16:47,  4.83s/it]
 33%|███▎      | 73545/225000 [146:26:49<183:12:54,  4.35s/it]
 33%|███▎      | 73546/225000 [146:26:52<168:33:52,  4.01s/it]
 33%|███▎      | 73547/225000 [146:26:56<158:25:43,  3.77s/it]
 33%|███▎      | 73548/225000 [146:26:59<148:09:05,  3.52s/it]
 33%|███▎      | 73549/225000 [146:27:02<151:17:11,  3.60s/it]
 33%|███▎      | 73550/225000 [146:27:12<232:52:53,  5.54s/it]
                                                              
{'loss': '0.8411', 'grad_norm': '1.598', 'learning_rate': '3.998e-05', 'epoch': '0.3269', 'num_input_tokens_seen': 242351816, 'train_runtime': '5.272e+05', 'train_tokens_per_second': '459.7'}

 33%|███▎      | 73550/225000 [146:27:12<232:52:53,  5.54s/it]
 33%|███▎      | 73551/225000 [146:27:16<211:17:43,  5.02s/it]
 33%|███▎      | 73552/225000 [146:27:19<179:36:36,  4.27s/it]
 33%|███▎      | 73553/225000 [146:27:21<154:29:37,  3.67s/it]
 33%|███▎      | 73554/225000 [146:27:26<173:19:01,  4.12s/it]
 33%|███▎      | 73555/225000 [146:27:32<196:59:34,  4.68s/it]
 33%|███▎      | 73556/225000 [146:27:35<167:49:56,  3.99s/it]
 33%|███▎      | 73557/225000 [146:27:41<198:54:05,  4.73s/it]
 33%|███▎      | 73558/225000 [146:27:44<177:13:19,  4.21s/it]
 33%|███▎      | 73559/225000 [146:27:47<165:49:22,  3.94s/it]
 33%|███▎      | 73560/225000 [146:27:50<147:10:13,  3.50s/it]
 33%|███▎      | 73561/225000 [146:27:53<146:50:10,  3.49s/it]
 33%|███▎      | 73562/225000 [146:27:57<144:23:31,  3.43s/it]
 33%|███▎      | 73563/225000 [146:28:00<147:19:55,  3.50s/it]
 33%|███▎      | 73564/225000 [146:28:03<141:46:55,  3.37s/it]
 33%|███▎      | 73565/225000 [146:28:07<144:18:47,  3.43s/it]
 33%|███▎      | 73566/225000 [146:28:12<164:33:35,  3.91s/it]
 33%|███▎      | 73567/225000 [146:28:20<220:05:32,  5.23s/it]
 33%|███▎      | 73568/225000 [146:28:26<223:24:29,  5.31s/it]
 33%|███▎      | 73569/225000 [146:28:28<191:27:35,  4.55s/it]
 33%|███▎      | 73570/225000 [146:28:36<223:15:08,  5.31s/it]
 33%|███▎      | 73571/225000 [146:28:39<199:01:48,  4.73s/it]
 33%|███▎      | 73572/225000 [146:28:42<181:17:50,  4.31s/it]
 33%|███▎      | 73573/225000 [146:28:45<160:11:37,  3.81s/it]
 33%|███▎      | 73574/225000 [146:28:48<145:21:13,  3.46s/it]
 33%|███▎      | 73575/225000 [146:28:50<139:12:29,  3.31s/it]
 33%|███▎      | 73576/225000 [146:28:58<190:04:30,  4.52s/it]
 33%|███▎      | 73577/225000 [146:29:05<223:28:22,  5.31s/it]
 33%|███▎      | 73578/225000 [146:29:08<197:57:50,  4.71s/it]
 33%|███▎      | 73579/225000 [146:29:12<181:57:34,  4.33s/it]
 33%|███▎      | 73580/225000 [146:29:15<168:48:32,  4.01s/it]
 33%|███▎      | 73581/225000 [146:29:25<244:17:11,  5.81s/it]
 33%|███▎      | 73582/225000 [146:29:28<203:23:10,  4.84s/it]
 33%|███▎      | 73583/225000 [146:29:30<177:28:19,  4.22s/it]
 33%|███▎      | 73584/225000 [146:29:33<153:32:58,  3.65s/it]
 33%|███▎      | 73585/225000 [146:29:39<188:38:54,  4.49s/it]
 33%|███▎      | 73586/225000 [146:29:42<170:07:10,  4.04s/it]
 33%|███▎      | 73587/225000 [146:29:45<149:04:54,  3.54s/it]
 33%|███▎      | 73588/225000 [146:29:48<153:07:35,  3.64s/it]
 33%|███▎      | 73589/225000 [146:29:56<200:43:59,  4.77s/it]
 33%|███▎      | 73590/225000 [146:29:58<170:03:05,  4.04s/it]
 33%|███▎      | 73591/225000 [146:30:01<159:57:46,  3.80s/it]
 33%|███▎      | 73592/225000 [146:30:05<159:26:47,  3.79s/it]
 33%|███▎      | 73593/225000 [146:30:09<154:25:36,  3.67s/it]
 33%|███▎      | 73594/225000 [146:30:12<145:39:36,  3.46s/it]
 33%|███▎      | 73595/225000 [146:30:15<147:05:03,  3.50s/it]
 33%|███▎      | 73596/225000 [146:30:18<142:40:14,  3.39s/it]
 33%|███▎      | 73597/225000 [146:30:22<150:28:47,  3.58s/it]
 33%|███▎      | 73598/225000 [146:30:30<206:12:03,  4.90s/it]
 33%|███▎      | 73599/225000 [146:30:36<217:50:07,  5.18s/it]
 33%|███▎      | 73600/225000 [146:30:39<192:46:21,  4.58s/it]
                                                              
{'loss': '0.8542', 'grad_norm': '1.348', 'learning_rate': '3.997e-05', 'epoch': '0.3271', 'num_input_tokens_seen': 242437832, 'train_runtime': '5.274e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73600/225000 [146:30:39<192:46:21,  4.58s/it]
 33%|███▎      | 73601/225000 [146:30:42<174:15:24,  4.14s/it]
 33%|███▎      | 73602/225000 [146:30:45<157:00:58,  3.73s/it]
 33%|███▎      | 73603/225000 [146:30:49<156:35:07,  3.72s/it]
 33%|███▎      | 73604/225000 [146:30:53<159:05:53,  3.78s/it]
 33%|███▎      | 73605/225000 [146:30:56<157:13:00,  3.74s/it]
 33%|███▎      | 73606/225000 [146:31:05<215:54:41,  5.13s/it]
 33%|███▎      | 73607/225000 [146:31:09<207:25:10,  4.93s/it]
 33%|███▎      | 73608/225000 [146:31:12<179:19:22,  4.26s/it]
 33%|███▎      | 73609/225000 [146:31:15<163:38:24,  3.89s/it]
 33%|███▎      | 73610/225000 [146:31:21<188:04:59,  4.47s/it]
 33%|███▎      | 73611/225000 [146:31:26<201:45:44,  4.80s/it]
 33%|███▎      | 73612/225000 [146:31:30<184:18:50,  4.38s/it]
 33%|███▎      | 73613/225000 [146:31:34<178:33:07,  4.25s/it]
 33%|███▎      | 73614/225000 [146:31:40<199:29:41,  4.74s/it]
 33%|███▎      | 73615/225000 [146:31:43<184:31:24,  4.39s/it]
 33%|███▎      | 73616/225000 [146:31:46<168:47:17,  4.01s/it]
 33%|███▎      | 73617/225000 [146:31:50<161:35:16,  3.84s/it]
 33%|███▎      | 73618/225000 [146:31:53<154:05:42,  3.66s/it]
 33%|███▎      | 73619/225000 [146:31:57<161:42:55,  3.85s/it]
 33%|███▎      | 73620/225000 [146:32:03<182:26:51,  4.34s/it]
 33%|███▎      | 73621/225000 [146:32:12<243:52:43,  5.80s/it]
 33%|███▎      | 73622/225000 [146:32:15<215:07:54,  5.12s/it]
 33%|███▎      | 73623/225000 [146:32:18<181:42:44,  4.32s/it]
 33%|███▎      | 73624/225000 [146:32:21<162:08:01,  3.86s/it]
 33%|███▎      | 73625/225000 [146:32:24<159:00:08,  3.78s/it]
 33%|███▎      | 73626/225000 [146:32:27<150:22:12,  3.58s/it]
 33%|███▎      | 73627/225000 [146:32:31<145:21:59,  3.46s/it]
 33%|███▎      | 73628/225000 [146:32:34<141:32:18,  3.37s/it]
 33%|███▎      | 73629/225000 [146:32:37<137:20:53,  3.27s/it]
 33%|███▎      | 73630/225000 [146:32:40<140:14:56,  3.34s/it]
 33%|███▎      | 73631/225000 [146:32:48<196:05:35,  4.66s/it]
 33%|███▎      | 73632/225000 [146:32:56<241:33:03,  5.74s/it]
 33%|███▎      | 73633/225000 [146:32:59<207:40:36,  4.94s/it]
 33%|███▎      | 73634/225000 [146:33:02<182:13:22,  4.33s/it]
 33%|███▎      | 73635/225000 [146:33:05<165:34:24,  3.94s/it]
 33%|███▎      | 73636/225000 [146:33:08<147:19:26,  3.50s/it]
 33%|███▎      | 73637/225000 [146:33:12<152:11:09,  3.62s/it]
 33%|███▎      | 73638/225000 [146:33:17<179:26:46,  4.27s/it]
 33%|███▎      | 73639/225000 [146:33:26<229:11:10,  5.45s/it]
 33%|███▎      | 73640/225000 [146:33:30<209:29:14,  4.98s/it]
 33%|███▎      | 73641/225000 [146:33:32<182:56:00,  4.35s/it]
 33%|███▎      | 73642/225000 [146:33:35<161:00:44,  3.83s/it]
 33%|███▎      | 73643/225000 [146:33:44<226:48:06,  5.39s/it]
 33%|███▎      | 73644/225000 [146:33:47<195:05:02,  4.64s/it]
 33%|███▎      | 73645/225000 [146:33:50<177:37:07,  4.22s/it]
 33%|███▎      | 73646/225000 [146:33:53<158:26:56,  3.77s/it]
 33%|███▎      | 73647/225000 [146:34:01<211:37:51,  5.03s/it]
 33%|███▎      | 73648/225000 [146:34:07<226:35:38,  5.39s/it]
 33%|███▎      | 73649/225000 [146:34:10<188:51:19,  4.49s/it]
 33%|███▎      | 73650/225000 [146:34:13<169:38:44,  4.04s/it]
                                                              
{'loss': '0.8596', 'grad_norm': '2.826', 'learning_rate': '3.996e-05', 'epoch': '0.3273', 'num_input_tokens_seen': 242524312, 'train_runtime': '5.277e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73650/225000 [146:34:13<169:38:44,  4.04s/it]
 33%|███▎      | 73651/225000 [146:34:15<156:00:25,  3.71s/it]
 33%|███▎      | 73652/225000 [146:34:19<153:39:39,  3.66s/it]
 33%|███▎      | 73653/225000 [146:34:22<144:47:13,  3.44s/it]
 33%|███▎      | 73654/225000 [146:34:25<140:33:23,  3.34s/it]
 33%|███▎      | 73655/225000 [146:34:28<138:21:03,  3.29s/it]
 33%|███▎      | 73656/225000 [146:34:32<147:44:27,  3.51s/it]
 33%|███▎      | 73657/225000 [146:34:36<144:27:34,  3.44s/it]
 33%|███▎      | 73658/225000 [146:34:39<141:09:13,  3.36s/it]
 33%|███▎      | 73659/225000 [146:34:42<140:31:06,  3.34s/it]
 33%|███▎      | 73660/225000 [146:34:45<138:46:43,  3.30s/it]
 33%|███▎      | 73661/225000 [146:34:48<134:54:15,  3.21s/it]
 33%|███▎      | 73662/225000 [146:34:51<131:02:56,  3.12s/it]
 33%|███▎      | 73663/225000 [146:34:54<133:32:57,  3.18s/it]
 33%|███▎      | 73664/225000 [146:35:00<166:40:04,  3.96s/it]
 33%|███▎      | 73665/225000 [146:35:10<235:20:38,  5.60s/it]
 33%|███▎      | 73666/225000 [146:35:13<204:42:42,  4.87s/it]
 33%|███▎      | 73667/225000 [146:35:15<173:03:18,  4.12s/it]
 33%|███▎      | 73668/225000 [146:35:18<157:11:08,  3.74s/it]
 33%|███▎      | 73669/225000 [146:35:21<152:41:48,  3.63s/it]
 33%|███▎      | 73670/225000 [146:35:25<146:42:28,  3.49s/it]
 33%|███▎      | 73671/225000 [146:35:31<188:12:07,  4.48s/it]
 33%|███▎      | 73672/225000 [146:35:40<244:17:15,  5.81s/it]
 33%|███▎      | 73673/225000 [146:35:44<213:17:56,  5.07s/it]
 33%|███▎      | 73674/225000 [146:35:47<189:17:19,  4.50s/it]
 33%|███▎      | 73675/225000 [146:35:50<169:13:55,  4.03s/it]
 33%|███▎      | 73676/225000 [146:35:52<152:49:46,  3.64s/it]
 33%|███▎      | 73677/225000 [146:35:55<141:21:42,  3.36s/it]
 33%|███▎      | 73678/225000 [146:35:58<138:45:29,  3.30s/it]
 33%|███▎      | 73679/225000 [146:36:01<136:15:07,  3.24s/it]
 33%|███▎      | 73680/225000 [146:36:05<138:25:08,  3.29s/it]
 33%|███▎      | 73681/225000 [146:36:10<165:47:11,  3.94s/it]
 33%|███▎      | 73682/225000 [146:36:21<246:13:12,  5.86s/it]
 33%|███▎      | 73683/225000 [146:36:25<225:10:10,  5.36s/it]
 33%|███▎      | 73684/225000 [146:36:28<195:31:47,  4.65s/it]
 33%|███▎      | 73685/225000 [146:36:30<168:14:50,  4.00s/it]
 33%|███▎      | 73686/225000 [146:36:33<153:08:06,  3.64s/it]
 33%|███▎      | 73687/225000 [146:36:41<204:34:29,  4.87s/it]
 33%|███▎      | 73688/225000 [146:36:44<185:26:15,  4.41s/it]
 33%|███▎      | 73689/225000 [146:36:51<211:35:14,  5.03s/it]
 33%|███▎      | 73690/225000 [146:36:53<180:53:20,  4.30s/it]
 33%|███▎      | 73691/225000 [146:36:56<157:10:48,  3.74s/it]
 33%|███▎      | 73692/225000 [146:37:00<167:13:23,  3.98s/it]
 33%|███▎      | 73693/225000 [146:37:05<175:36:04,  4.18s/it]
 33%|███▎      | 73694/225000 [146:37:08<159:47:22,  3.80s/it]
 33%|███▎      | 73695/225000 [146:37:11<152:04:28,  3.62s/it]
 33%|███▎      | 73696/225000 [146:37:14<144:31:08,  3.44s/it]
 33%|███▎      | 73697/225000 [146:37:17<137:00:24,  3.26s/it]
 33%|███▎      | 73698/225000 [146:37:19<128:20:38,  3.05s/it]
 33%|███▎      | 73699/225000 [146:37:27<188:46:06,  4.49s/it]
 33%|███▎      | 73700/225000 [146:37:33<209:16:11,  4.98s/it]
                                                              
{'loss': '0.9214', 'grad_norm': '1.837', 'learning_rate': '3.994e-05', 'epoch': '0.3276', 'num_input_tokens_seen': 242608792, 'train_runtime': '5.279e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73700/225000 [146:37:33<209:16:11,  4.98s/it]
 33%|███▎      | 73701/225000 [146:37:36<184:32:29,  4.39s/it]
 33%|███▎      | 73702/225000 [146:37:40<174:54:52,  4.16s/it]
 33%|███▎      | 73703/225000 [146:37:43<160:13:32,  3.81s/it]
 33%|███▎      | 73704/225000 [146:37:46<146:52:30,  3.49s/it]
 33%|███▎      | 73705/225000 [146:37:54<210:41:10,  5.01s/it]
 33%|███▎      | 73706/225000 [146:37:58<193:09:39,  4.60s/it]
 33%|███▎      | 73707/225000 [146:38:00<162:51:38,  3.88s/it]
 33%|███▎      | 73708/225000 [146:38:03<154:56:26,  3.69s/it]
 33%|███▎      | 73709/225000 [146:38:11<202:53:49,  4.83s/it]
 33%|███▎      | 73710/225000 [146:38:14<177:29:55,  4.22s/it]
 33%|███▎      | 73711/225000 [146:38:17<161:17:30,  3.84s/it]
 33%|███▎      | 73712/225000 [146:38:20<154:12:18,  3.67s/it]
 33%|███▎      | 73713/225000 [146:38:24<158:27:26,  3.77s/it]
 33%|███▎      | 73714/225000 [146:38:28<158:09:58,  3.76s/it]
 33%|███▎      | 73715/225000 [146:38:30<145:57:04,  3.47s/it]
 33%|███▎      | 73716/225000 [146:38:40<216:10:22,  5.14s/it]
 33%|███▎      | 73717/225000 [146:38:46<228:25:53,  5.44s/it]
 33%|███▎      | 73718/225000 [146:38:48<194:49:50,  4.64s/it]
 33%|███▎      | 73719/225000 [146:38:52<182:19:21,  4.34s/it]
 33%|███▎      | 73720/225000 [146:38:57<189:20:07,  4.51s/it]
 33%|███▎      | 73721/225000 [146:39:04<220:41:33,  5.25s/it]
 33%|███▎      | 73722/225000 [146:39:06<184:52:47,  4.40s/it]
 33%|███▎      | 73723/225000 [146:39:11<182:03:32,  4.33s/it]
 33%|███▎      | 73724/225000 [146:39:18<219:43:08,  5.23s/it]
 33%|███▎      | 73725/225000 [146:39:21<190:09:44,  4.53s/it]
 33%|███▎      | 73726/225000 [146:39:24<169:21:10,  4.03s/it]
 33%|███▎      | 73727/225000 [146:39:27<157:24:30,  3.75s/it]
 33%|███▎      | 73728/225000 [146:39:38<254:06:10,  6.05s/it]
 33%|███▎      | 73729/225000 [146:39:41<217:24:22,  5.17s/it]
 33%|███▎      | 73730/225000 [146:39:46<213:56:36,  5.09s/it]
 33%|███▎      | 73731/225000 [146:39:49<185:51:50,  4.42s/it]
 33%|███▎      | 73732/225000 [146:39:52<162:02:01,  3.86s/it]
 33%|███▎      | 73733/225000 [146:39:54<142:29:32,  3.39s/it]
 33%|███▎      | 73734/225000 [146:39:57<143:03:10,  3.40s/it]
 33%|███▎      | 73735/225000 [146:40:01<145:32:15,  3.46s/it]
 33%|███▎      | 73736/225000 [146:40:04<143:17:47,  3.41s/it]
 33%|███▎      | 73737/225000 [146:40:08<142:37:18,  3.39s/it]
 33%|███▎      | 73738/225000 [146:40:11<139:32:41,  3.32s/it]
 33%|███▎      | 73739/225000 [146:40:14<133:22:13,  3.17s/it]
 33%|███▎      | 73740/225000 [146:40:21<186:33:21,  4.44s/it]
 33%|███▎      | 73741/225000 [146:40:27<211:44:39,  5.04s/it]
 33%|███▎      | 73742/225000 [146:40:30<184:32:12,  4.39s/it]
 33%|███▎      | 73743/225000 [146:40:33<167:57:12,  4.00s/it]
 33%|███▎      | 73744/225000 [146:40:36<150:02:32,  3.57s/it]
 33%|███▎      | 73745/225000 [146:40:39<140:40:31,  3.35s/it]
 33%|███▎      | 73746/225000 [146:40:47<197:50:47,  4.71s/it]
 33%|███▎      | 73747/225000 [146:40:51<197:57:58,  4.71s/it]
 33%|███▎      | 73748/225000 [146:40:54<178:13:49,  4.24s/it]
 33%|███▎      | 73749/225000 [146:40:57<155:24:41,  3.70s/it]
 33%|███▎      | 73750/225000 [146:41:00<146:24:20,  3.48s/it]
                                                              
{'loss': '0.8682', 'grad_norm': '2.173', 'learning_rate': '3.993e-05', 'epoch': '0.3278', 'num_input_tokens_seen': 242695368, 'train_runtime': '5.281e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73750/225000 [146:41:00<146:24:20,  3.48s/it]
 33%|███▎      | 73751/225000 [146:41:02<134:54:25,  3.21s/it]
 33%|███▎      | 73752/225000 [146:41:05<128:14:26,  3.05s/it]
 33%|███▎      | 73753/225000 [146:41:08<123:30:02,  2.94s/it]
 33%|███▎      | 73754/225000 [146:41:11<130:02:41,  3.10s/it]
 33%|███▎      | 73755/225000 [146:41:22<222:07:25,  5.29s/it]
 33%|███▎      | 73756/225000 [146:41:29<249:59:42,  5.95s/it]
 33%|███▎      | 73757/225000 [146:41:32<207:24:45,  4.94s/it]
 33%|███▎      | 73758/225000 [146:41:34<173:14:31,  4.12s/it]
 33%|███▎      | 73759/225000 [146:41:37<164:45:35,  3.92s/it]
 33%|███▎      | 73760/225000 [146:41:40<150:39:56,  3.59s/it]
 33%|███▎      | 73761/225000 [146:41:43<140:07:40,  3.34s/it]
 33%|███▎      | 73762/225000 [146:41:46<132:06:04,  3.14s/it]
 33%|███▎      | 73763/225000 [146:41:49<134:17:11,  3.20s/it]
 33%|███▎      | 73764/225000 [146:41:54<156:12:12,  3.72s/it]
 33%|███▎      | 73765/225000 [146:42:05<243:47:58,  5.80s/it]
 33%|███▎      | 73766/225000 [146:42:07<200:47:35,  4.78s/it]
 33%|███▎      | 73767/225000 [146:42:10<175:38:41,  4.18s/it]
 33%|███▎      | 73768/225000 [146:42:13<160:05:12,  3.81s/it]
 33%|███▎      | 73769/225000 [146:42:16<151:08:18,  3.60s/it]
 33%|███▎      | 73770/225000 [146:42:19<140:54:52,  3.35s/it]
 33%|███▎      | 73771/225000 [146:42:22<141:39:48,  3.37s/it]
 33%|███▎      | 73772/225000 [146:42:25<131:41:55,  3.14s/it]
 33%|███▎      | 73773/225000 [146:42:33<198:52:33,  4.73s/it]
 33%|███▎      | 73774/225000 [146:42:41<241:01:34,  5.74s/it]
 33%|███▎      | 73775/225000 [146:42:46<223:59:59,  5.33s/it]
 33%|███▎      | 73776/225000 [146:42:49<202:46:17,  4.83s/it]
 33%|███▎      | 73777/225000 [146:42:52<180:30:45,  4.30s/it]
 33%|███▎      | 73778/225000 [146:42:55<167:11:10,  3.98s/it]
 33%|███▎      | 73779/225000 [146:42:59<160:57:37,  3.83s/it]
 33%|███▎      | 73780/225000 [146:43:03<158:53:43,  3.78s/it]
 33%|███▎      | 73781/225000 [146:43:07<166:58:58,  3.98s/it]
 33%|███▎      | 73782/225000 [146:43:11<162:58:22,  3.88s/it]
 33%|███▎      | 73783/225000 [146:43:14<155:13:37,  3.70s/it]
 33%|███▎      | 73784/225000 [146:43:23<216:25:52,  5.15s/it]
 33%|███▎      | 73785/225000 [146:43:27<214:02:54,  5.10s/it]
 33%|███▎      | 73786/225000 [146:43:31<194:43:49,  4.64s/it]
 33%|███▎      | 73787/225000 [146:43:34<173:48:09,  4.14s/it]
 33%|███▎      | 73788/225000 [146:43:37<155:44:56,  3.71s/it]
 33%|███▎      | 73789/225000 [146:43:42<172:11:03,  4.10s/it]
 33%|███▎      | 73790/225000 [146:43:49<208:12:27,  4.96s/it]
 33%|███▎      | 73791/225000 [146:43:51<177:56:04,  4.24s/it]
 33%|███▎      | 73792/225000 [146:43:54<163:35:04,  3.89s/it]
 33%|███▎      | 73793/225000 [146:43:57<150:27:18,  3.58s/it]
 33%|███▎      | 73794/225000 [146:44:00<140:12:37,  3.34s/it]
 33%|███▎      | 73795/225000 [146:44:03<137:32:07,  3.27s/it]
 33%|███▎      | 73796/225000 [146:44:07<139:22:09,  3.32s/it]
 33%|███▎      | 73797/225000 [146:44:10<135:21:48,  3.22s/it]
 33%|███▎      | 73798/225000 [146:44:18<195:59:05,  4.67s/it]
 33%|███▎      | 73799/225000 [146:44:23<200:45:44,  4.78s/it]
 33%|███▎      | 73800/225000 [146:44:26<179:00:53,  4.26s/it]
                                                              
{'loss': '0.8815', 'grad_norm': '1.273', 'learning_rate': '3.991e-05', 'epoch': '0.328', 'num_input_tokens_seen': 242781136, 'train_runtime': '5.283e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73800/225000 [146:44:26<179:00:53,  4.26s/it]
 33%|███▎      | 73801/225000 [146:44:34<228:35:00,  5.44s/it]
 33%|███▎      | 73802/225000 [146:44:37<197:42:42,  4.71s/it]
 33%|███▎      | 73803/225000 [146:44:39<169:14:15,  4.03s/it]
 33%|███▎      | 73804/225000 [146:44:44<180:09:34,  4.29s/it]
 33%|███▎      | 73805/225000 [146:44:48<180:30:50,  4.30s/it]
 33%|███▎      | 73806/225000 [146:44:51<155:39:19,  3.71s/it]
 33%|███▎      | 73807/225000 [146:44:54<152:51:04,  3.64s/it]
 33%|███▎      | 73808/225000 [146:45:00<173:56:29,  4.14s/it]
 33%|███▎      | 73809/225000 [146:45:05<192:39:57,  4.59s/it]
 33%|███▎      | 73810/225000 [146:45:08<167:27:08,  3.99s/it]
 33%|███▎      | 73811/225000 [146:45:10<150:26:39,  3.58s/it]
 33%|███▎      | 73812/225000 [146:45:13<137:52:59,  3.28s/it]
 33%|███▎      | 73813/225000 [146:45:20<188:39:23,  4.49s/it]
 33%|███▎      | 73814/225000 [146:45:25<186:05:30,  4.43s/it]
 33%|███▎      | 73815/225000 [146:45:28<176:28:58,  4.20s/it]
 33%|███▎      | 73816/225000 [146:45:32<164:58:46,  3.93s/it]
 33%|███▎      | 73817/225000 [146:45:35<157:59:36,  3.76s/it]
 33%|███▎      | 73818/225000 [146:45:39<155:26:07,  3.70s/it]
 33%|███▎      | 73819/225000 [146:45:42<147:47:52,  3.52s/it]
 33%|███▎      | 73820/225000 [146:45:44<138:53:26,  3.31s/it]
 33%|███▎      | 73821/225000 [146:45:48<142:10:30,  3.39s/it]
 33%|███▎      | 73822/225000 [146:45:52<153:51:47,  3.66s/it]
 33%|███▎      | 73823/225000 [146:45:56<152:07:54,  3.62s/it]
 33%|███▎      | 73824/225000 [146:46:04<205:28:02,  4.89s/it]
 33%|███▎      | 73825/225000 [146:46:13<255:06:16,  6.07s/it]
 33%|███▎      | 73826/225000 [146:46:15<210:07:02,  5.00s/it]
 33%|███▎      | 73827/225000 [146:46:18<187:03:37,  4.45s/it]
 33%|███▎      | 73828/225000 [146:46:21<167:02:55,  3.98s/it]
 33%|███▎      | 73829/225000 [146:46:31<235:41:45,  5.61s/it]
 33%|███▎      | 73830/225000 [146:46:33<200:28:40,  4.77s/it]
 33%|███▎      | 73831/225000 [146:46:36<171:46:37,  4.09s/it]
 33%|███▎      | 73832/225000 [146:46:42<193:27:57,  4.61s/it]
 33%|███▎      | 73833/225000 [146:46:44<165:19:45,  3.94s/it]
 33%|███▎      | 73834/225000 [146:46:47<148:36:14,  3.54s/it]
 33%|███▎      | 73835/225000 [146:46:50<142:08:30,  3.39s/it]
 33%|███▎      | 73836/225000 [146:47:00<226:53:32,  5.40s/it]
 33%|███▎      | 73837/225000 [146:47:03<198:11:29,  4.72s/it]
 33%|███▎      | 73838/225000 [146:47:06<183:03:14,  4.36s/it]
 33%|███▎      | 73839/225000 [146:47:13<213:49:25,  5.09s/it]
 33%|███▎      | 73840/225000 [146:47:16<178:46:36,  4.26s/it]
 33%|███▎      | 73841/225000 [146:47:23<221:52:44,  5.28s/it]
 33%|███▎      | 73842/225000 [146:47:25<183:40:15,  4.37s/it]
 33%|███▎      | 73843/225000 [146:47:28<164:52:35,  3.93s/it]
 33%|███▎      | 73844/225000 [146:47:35<193:00:35,  4.60s/it]
 33%|███▎      | 73845/225000 [146:47:39<194:29:51,  4.63s/it]
 33%|███▎      | 73846/225000 [146:47:43<184:56:24,  4.40s/it]
 33%|███▎      | 73847/225000 [146:47:48<192:45:08,  4.59s/it]
 33%|███▎      | 73848/225000 [146:47:50<163:45:00,  3.90s/it]
 33%|███▎      | 73849/225000 [146:47:53<151:53:06,  3.62s/it]
 33%|███▎      | 73850/225000 [146:47:56<140:48:32,  3.35s/it]
                                                              
{'loss': '0.8991', 'grad_norm': '4.732', 'learning_rate': '3.99e-05', 'epoch': '0.3282', 'num_input_tokens_seen': 242867240, 'train_runtime': '5.285e+05', 'train_tokens_per_second': '459.6'}

 33%|███▎      | 73850/225000 [146:47:56<140:48:32,  3.35s/it]
 33%|███▎      | 73851/225000 [146:48:00<143:23:42,  3.42s/it]
 33%|███▎      | 73852/225000 [146:48:03<136:17:35,  3.25s/it]
 33%|███▎      | 73853/225000 [146:48:06<139:12:11,  3.32s/it]
 33%|███▎      | 73854/225000 [146:48:09<134:31:45,  3.20s/it]
 33%|███▎      | 73855/225000 [146:48:12<134:45:09,  3.21s/it]
 33%|███▎      | 73856/225000 [146:48:15<136:18:24,  3.25s/it]
 33%|███▎      | 73857/225000 [146:48:20<147:08:35,  3.50s/it]
 33%|███▎      | 73858/225000 [146:48:26<188:35:32,  4.49s/it]
 33%|███▎      | 73859/225000 [146:48:35<246:15:19,  5.87s/it]
 33%|███▎      | 73860/225000 [146:48:38<208:10:08,  4.96s/it]
 33%|███▎      | 73861/225000 [146:48:44<211:40:11,  5.04s/it]
 33%|███▎      | 73862/225000 [146:48:49<215:52:05,  5.14s/it]
 33%|███▎      | 73863/225000 [146:48:52<194:39:34,  4.64s/it]
 33%|███▎      | 73864/225000 [146:48:55<170:54:52,  4.07s/it]
 33%|███▎      | 73865/225000 [146:48:58<157:17:04,  3.75s/it]
 33%|███▎      | 73866/225000 [146:49:01<151:18:00,  3.60s/it]
 33%|███▎      | 73867/225000 [146:49:05<152:28:42,  3.63s/it]
 33%|███▎      | 73868/225000 [146:49:08<140:04:27,  3.34s/it]
 33%|███▎      | 73869/225000 [146:49:11<142:52:04,  3.40s/it]
 33%|███▎      | 73870/225000 [146:49:18<178:13:59,  4.25s/it]
 33%|███▎      | 73871/225000 [146:49:26<235:28:45,  5.61s/it]
 33%|███▎      | 73872/225000 [146:49:30<207:00:02,  4.93s/it]
 33%|███▎      | 73873/225000 [146:49:32<178:32:36,  4.25s/it]
 33%|███▎      | 73874/225000 [146:49:39<210:09:30,  5.01s/it]
 33%|███▎      | 73875/225000 [146:49:42<184:10:13,  4.39s/it]
 33%|███▎      | 73876/225000 [146:49:45<166:17:35,  3.96s/it]
 33%|███▎      | 73877/225000 [146:49:48<151:19:51,  3.60s/it]
 33%|███▎      | 73878/225000 [146:49:51<145:25:26,  3.46s/it]
 33%|███▎      | 73879/225000 [146:49:55<148:28:09,  3.54s/it]
 33%|███▎      | 73880/225000 [146:50:03<205:29:24,  4.90s/it]
 33%|███▎      | 73881/225000 [146:50:05<179:22:56,  4.27s/it]
 33%|███▎      | 73882/225000 [146:50:08<156:06:01,  3.72s/it]
 33%|███▎      | 73883/225000 [146:50:12<160:14:54,  3.82s/it]
 33%|███▎      | 73884/225000 [146:50:19<196:18:06,  4.68s/it]
 33%|███▎      | 73885/225000 [146:50:22<182:44:59,  4.35s/it]
 33%|███▎      | 73886/225000 [146:50:27<188:06:39,  4.48s/it]
 33%|███▎      | 73887/225000 [146:50:29<159:49:54,  3.81s/it]
 33%|███▎      | 73888/225000 [146:50:32<152:33:22,  3.63s/it]
 33%|███▎      | 73889/225000 [146:50:36<154:04:07,  3.67s/it]
 33%|███▎      | 73890/225000 [146:50:39<145:12:49,  3.46s/it]
 33%|███▎      | 73891/225000 [146:50:44<165:18:46,  3.94s/it]
 33%|███▎      | 73892/225000 [146:50:47<151:41:39,  3.61s/it]
 33%|███▎      | 73893/225000 [146:50:51<152:02:26,  3.62s/it]
 33%|███▎      | 73894/225000 [146:50:54<147:51:46,  3.52s/it]
 33%|███▎      | 73895/225000 [146:50:57<145:04:31,  3.46s/it]
 33%|███▎      | 73896/225000 [146:51:06<209:30:41,  4.99s/it]
 33%|███▎      | 73897/225000 [146:51:09<186:03:08,  4.43s/it]
 33%|███▎      | 73898/225000 [146:51:13<178:53:40,  4.26s/it]
 33%|███▎      | 73899/225000 [146:51:18<192:01:37,  4.58s/it]
 33%|███▎      | 73900/225000 [146:51:22<181:31:06,  4.32s/it]
                                                              
{'loss': '0.8096', 'grad_norm': '3.452', 'learning_rate': '3.988e-05', 'epoch': '0.3284', 'num_input_tokens_seen': 242951824, 'train_runtime': '5.287e+05', 'train_tokens_per_second': '459.5'}

 33%|███▎      | 73900/225000 [146:51:22<181:31:06,  4.32s/it]
 33%|███▎      | 73901/225000 [146:51:27<193:54:54,  4.62s/it]
 33%|███▎      | 73902/225000 [146:51:31<179:07:24,  4.27s/it]
 33%|███▎      | 73903/225000 [146:51:33<156:25:15,  3.73s/it]
 33%|███▎      | 73904/225000 [146:51:39<181:27:04,  4.32s/it]
 33%|███▎      | 73905/225000 [146:51:42<166:00:51,  3.96s/it]
 33%|███▎      | 73906/225000 [146:51:45<154:02:35,  3.67s/it]
 33%|███▎      | 73907/225000 [146:51:48<146:04:16,  3.48s/it]
 33%|███▎      | 73908/225000 [146:51:51<135:47:18,  3.24s/it]
 33%|███▎      | 73909/225000 [146:51:54<140:19:27,  3.34s/it]
 33%|███▎      | 73910/225000 [146:51:58<140:28:08,  3.35s/it]
 33%|███▎      | 73911/225000 [146:52:08<230:08:43,  5.48s/it]
 33%|███▎      | 73912/225000 [146:52:15<241:42:03,  5.76s/it]
 33%|███▎      | 73913/225000 [146:52:17<201:28:35,  4.80s/it]
 33%|███▎      | 73914/225000 [146:52:20<177:46:04,  4.24s/it]
 33%|███▎      | 73915/225000 [146:52:22<153:08:42,  3.65s/it]
 33%|███▎      | 73916/225000 [146:52:25<145:45:09,  3.47s/it]
 33%|███▎      | 73917/225000 [146:52:29<144:39:05,  3.45s/it]
 33%|███▎      | 73918/225000 [146:52:32<145:41:32,  3.47s/it]
 33%|███▎      | 73919/225000 [146:52:37<158:29:23,  3.78s/it]
 33%|███▎      | 73920/225000 [146:52:44<207:15:06,  4.94s/it]
 33%|███▎      | 73921/225000 [146:52:47<181:55:02,  4.33s/it]
 33%|███▎      | 73922/225000 [146:52:52<191:55:50,  4.57s/it]
 33%|███▎      | 73923/225000 [146:52:58<198:08:27,  4.72s/it]
 33%|███▎      | 73924/225000 [146:53:01<180:36:39,  4.30s/it]
 33%|███▎      | 73925/225000 [146:53:04<166:10:05,  3.96s/it]
 33%|███▎      | 73926/225000 [146:53:07<152:54:15,  3.64s/it]
 33%|███▎      | 73927/225000 [146:53:09<139:08:48,  3.32s/it]
 33%|███▎      | 73928/225000 [146:53:13<135:48:13,  3.24s/it]
 33%|███▎      | 73929/225000 [146:53:15<131:50:59,  3.14s/it]
 33%|███▎      | 73930/225000 [146:53:18<125:31:30,  2.99s/it]
 33%|███▎      | 73931/225000 [146:53:22<132:09:37,  3.15s/it]
 33%|███▎      | 73932/225000 [146:53:31<204:49:18,  4.88s/it]
 33%|███▎      | 73933/225000 [146:53:37<219:31:09,  5.23s/it]
 33%|███▎      | 73934/225000 [146:53:40<194:28:29,  4.63s/it]
 33%|███▎      | 73935/225000 [146:53:42<167:52:19,  4.00s/it]
 33%|███▎      | 73936/225000 [146:53:45<152:27:50,  3.63s/it]
 33%|███▎      | 73937/225000 [146:53:52<197:33:18,  4.71s/it]
 33%|███▎      | 73938/225000 [146:53:56<178:46:52,  4.26s/it]
 33%|███▎      | 73939/225000 [146:53:58<161:07:13,  3.84s/it]
 33%|███▎      | 73940/225000 [146:54:01<147:28:56,  3.51s/it]
 33%|███▎      | 73941/225000 [146:54:04<134:52:20,  3.21s/it]
 33%|███▎      | 73942/225000 [146:54:07<132:42:27,  3.16s/it]
 33%|███▎      | 73943/225000 [146:54:14<188:58:30,  4.50s/it]
 33%|███▎      | 73944/225000 [146:54:22<223:44:12,  5.33s/it]
 33%|███▎      | 73945/225000 [146:54:26<216:46:47,  5.17s/it]
 33%|███▎      | 73946/225000 [146:54:29<181:37:45,  4.33s/it]
 33%|███▎      | 73947/225000 [146:54:31<156:48:11,  3.74s/it]
 33%|███▎      | 73948/225000 [146:54:36<165:53:21,  3.95s/it]
 33%|███▎      | 73949/225000 [146:54:43<204:44:40,  4.88s/it]
 33%|███▎      | 73950/225000 [146:54:48<206:52:03,  4.93s/it]
                                                              
{'loss': '0.8629', 'grad_norm': '1.65', 'learning_rate': '3.987e-05', 'epoch': '0.3287', 'num_input_tokens_seen': 243036224, 'train_runtime': '5.289e+05', 'train_tokens_per_second': '459.5'}

 33%|███▎      | 73950/225000 [146:54:48<206:52:03,  4.93s/it]
 33%|███▎      | 73951/225000 [146:54:50<171:17:28,  4.08s/it]
 33%|███▎      | 73952/225000 [146:54:52<149:37:32,  3.57s/it]
 33%|███▎      | 73953/225000 [146:54:56<150:00:52,  3.58s/it]
 33%|███▎      | 73954/225000 [146:54:59<143:56:33,  3.43s/it]
 33%|███▎      | 73955/225000 [146:55:01<132:57:17,  3.17s/it]
 33%|███▎      | 73956/225000 [146:55:08<170:28:31,  4.06s/it]
 33%|███▎      | 73957/225000 [146:55:14<202:55:26,  4.84s/it]
 33%|███▎      | 73958/225000 [146:55:18<184:38:32,  4.40s/it]
 33%|███▎      | 73959/225000 [146:55:21<176:21:23,  4.20s/it]
 33%|███▎      | 73960/225000 [146:55:25<163:42:42,  3.90s/it]
 33%|███▎      | 73961/225000 [146:55:28<154:30:22,  3.68s/it]
 33%|███▎      | 73962/225000 [146:55:32<167:34:03,  3.99s/it]
 33%|███▎      | 73963/225000 [146:55:37<176:47:14,  4.21s/it]
 33%|███▎      | 73964/225000 [146:55:40<163:54:49,  3.91s/it]
 33%|███▎      | 73965/225000 [146:55:44<164:20:51,  3.92s/it]
 33%|███▎      | 73966/225000 [146:55:49<169:42:10,  4.04s/it]
 33%|███▎      | 73967/225000 [146:55:54<192:41:19,  4.59s/it]
 33%|███▎      | 73968/225000 [146:55:57<165:33:28,  3.95s/it]
 33%|███▎      | 73969/225000 [146:56:00<151:50:59,  3.62s/it]
 33%|███▎      | 73970/225000 [146:56:02<140:17:12,  3.34s/it]
 33%|███▎      | 73971/225000 [146:56:11<206:26:42,  4.92s/it]
 33%|███▎      | 73972/225000 [146:56:15<190:03:09,  4.53s/it]
 33%|███▎      | 73973/225000 [146:56:17<166:23:03,  3.97s/it]
 33%|███▎      | 73974/225000 [146:56:21<168:19:39,  4.01s/it]
 33%|███▎      | 73975/225000 [146:56:24<150:57:56,  3.60s/it]
 33%|███▎      | 73976/225000 [146:56:27<144:00:17,  3.43s/it]
 33%|███▎      | 73977/225000 [146:56:30<137:42:49,  3.28s/it]
 33%|███▎      | 73978/225000 [146:56:33<138:17:07,  3.30s/it]
 33%|███▎      | 73979/225000 [146:56:36<129:26:57,  3.09s/it]
 33%|███▎      | 73980/225000 [146:56:43<175:48:38,  4.19s/it]
 33%|███▎      | 73981/225000 [146:56:51<223:37:02,  5.33s/it]
 33%|███▎      | 73982/225000 [146:56:55<214:47:10,  5.12s/it]
 33%|███▎      | 73983/225000 [146:56:58<181:24:55,  4.32s/it]
 33%|███▎      | 73984/225000 [146:57:00<157:39:37,  3.76s/it]
 33%|███▎      | 73985/225000 [146:57:04<151:22:57,  3.61s/it]
 33%|███▎      | 73986/225000 [146:57:12<209:27:17,  4.99s/it]
 33%|███▎      | 73987/225000 [146:57:14<175:22:40,  4.18s/it]
 33%|███▎      | 73988/225000 [146:57:17<156:20:54,  3.73s/it]
 33%|███▎      | 73989/225000 [146:57:19<144:05:47,  3.44s/it]
 33%|███▎      | 73990/225000 [146:57:22<132:19:00,  3.15s/it]
 33%|███▎      | 73991/225000 [146:57:25<128:37:51,  3.07s/it]
 33%|███▎      | 73992/225000 [146:57:32<175:36:40,  4.19s/it]
 33%|███▎      | 73993/225000 [146:57:39<211:55:32,  5.05s/it]
 33%|███▎      | 73994/225000 [146:57:42<185:07:27,  4.41s/it]
 33%|███▎      | 73995/225000 [146:57:44<164:40:27,  3.93s/it]
 33%|███▎      | 73996/225000 [146:57:48<156:31:44,  3.73s/it]
 33%|███▎      | 73997/225000 [146:57:53<175:07:05,  4.17s/it]
 33%|███▎      | 73998/225000 [146:57:59<198:53:17,  4.74s/it]
 33%|███▎      | 73999/225000 [146:58:03<190:49:29,  4.55s/it]
 33%|███▎      | 74000/225000 [146:58:06<173:22:47,  4.13s/it]
                                                              
{'loss': '0.8757', 'grad_norm': '2.465', 'learning_rate': '3.985e-05', 'epoch': '0.3289', 'num_input_tokens_seen': 243117920, 'train_runtime': '5.291e+05', 'train_tokens_per_second': '459.5'}

 33%|███▎      | 74000/225000 [146:58:06<173:22:47,  4.13s/it]
 33%|███▎      | 74001/225000 [146:58:09<161:16:13,  3.84s/it]
 33%|███▎      | 74002/225000 [146:58:12<149:05:44,  3.55s/it]
 33%|███▎      | 74003/225000 [146:58:20<197:17:32,  4.70s/it]
 33%|███▎      | 74004/225000 [146:58:23<183:57:42,  4.39s/it]
 33%|███▎      | 74005/225000 [146:58:26<164:41:16,  3.93s/it]
 33%|███▎      | 74006/225000 [146:58:29<156:23:35,  3.73s/it]
 33%|███▎      | 74007/225000 [146:58:32<146:50:04,  3.50s/it]
 33%|███▎      | 74008/225000 [146:58:42<219:00:50,  5.22s/it]
 33%|███▎      | 74009/225000 [146:58:46<202:54:11,  4.84s/it]
 33%|███▎      | 74010/225000 [146:58:49<181:14:09,  4.32s/it]
 33%|███▎      | 74011/225000 [146:58:52<162:36:54,  3.88s/it]
 33%|███▎      | 74012/225000 [146:58:54<149:13:12,  3.56s/it]
 33%|███▎      | 74013/225000 [146:58:57<139:52:44,  3.34s/it]
 33%|███▎      | 74014/225000 [146:59:01<140:28:35,  3.35s/it]
 33%|███▎      | 74015/225000 [146:59:04<140:59:58,  3.36s/it]
 33%|███▎      | 74016/225000 [146:59:08<147:34:59,  3.52s/it]
 33%|███▎      | 74017/225000 [146:59:14<175:15:38,  4.18s/it]
 33%|███▎      | 74018/225000 [146:59:22<224:39:53,  5.36s/it]
 33%|███▎      | 74019/225000 [146:59:24<189:41:55,  4.52s/it]
 33%|███▎      | 74020/225000 [146:59:27<166:29:48,  3.97s/it]
 33%|███▎      | 74021/225000 [146:59:35<221:31:10,  5.28s/it]
 33%|███▎      | 74022/225000 [146:59:39<204:25:21,  4.87s/it]
 33%|███▎      | 74023/225000 [146:59:42<184:23:10,  4.40s/it]
 33%|███▎      | 74024/225000 [146:59:45<165:14:24,  3.94s/it]
 33%|███▎      | 74025/225000 [146:59:48<155:11:51,  3.70s/it]
 33%|███▎      | 74026/225000 [146:59:51<141:19:15,  3.37s/it]
 33%|███▎      | 74027/225000 [146:59:58<185:01:32,  4.41s/it]
 33%|███▎      | 74028/225000 [147:00:04<201:42:43,  4.81s/it]
 33%|███▎      | 74029/225000 [147:00:07<181:54:59,  4.34s/it]
 33%|███▎      | 74030/225000 [147:00:10<162:02:39,  3.86s/it]
 33%|███▎      | 74031/225000 [147:00:12<146:37:36,  3.50s/it]
 33%|███▎      | 74032/225000 [147:00:15<135:28:10,  3.23s/it]
 33%|███▎      | 74033/225000 [147:00:18<128:00:38,  3.05s/it]
 33%|███▎      | 74034/225000 [147:00:26<194:28:36,  4.64s/it]
 33%|███▎      | 74035/225000 [147:00:31<200:45:32,  4.79s/it]
 33%|███▎      | 74036/225000 [147:00:37<211:08:50,  5.04s/it]
 33%|███▎      | 74037/225000 [147:00:39<182:36:24,  4.35s/it]
 33%|███▎      | 74038/225000 [147:00:43<169:18:15,  4.04s/it]
 33%|███▎      | 74039/225000 [147:00:50<215:15:43,  5.13s/it]
 33%|███▎      | 74040/225000 [147:00:53<178:36:27,  4.26s/it]
 33%|███▎      | 74041/225000 [147:00:55<157:52:08,  3.76s/it]
 33%|███▎      | 74042/225000 [147:01:00<166:32:45,  3.97s/it]
 33%|███▎      | 74043/225000 [147:01:04<175:58:57,  4.20s/it]
 33%|███▎      | 74044/225000 [147:01:07<152:57:37,  3.65s/it]
 33%|███▎      | 74045/225000 [147:01:09<140:11:00,  3.34s/it]
 33%|███▎      | 74046/225000 [147:01:13<147:13:43,  3.51s/it]
 33%|███▎      | 74047/225000 [147:01:16<140:46:40,  3.36s/it]
 33%|███▎      | 74048/225000 [147:01:19<134:33:03,  3.21s/it]
 33%|███▎      | 74049/225000 [147:01:22<130:17:58,  3.11s/it]
 33%|███▎      | 74050/225000 [147:01:25<130:04:03,  3.10s/it]
                                                              
{'loss': '0.8602', 'grad_norm': '2.402', 'learning_rate': '3.984e-05', 'epoch': '0.3291', 'num_input_tokens_seen': 243199472, 'train_runtime': '5.293e+05', 'train_tokens_per_second': '459.5'}

 33%|███▎      | 74050/225000 [147:01:25<130:04:03,  3.10s/it]
 33%|███▎      | 74051/225000 [147:01:30<150:47:49,  3.60s/it]
 33%|███▎      | 74052/225000 [147:01:38<213:56:23,  5.10s/it]
 33%|███▎      | 74053/225000 [147:01:44<219:22:30,  5.23s/it]
 33%|███▎      | 74054/225000 [147:01:47<191:40:51,  4.57s/it]
 33%|███▎      | 74055/225000 [147:01:51<180:50:53,  4.31s/it]
 33%|███▎      | 74056/225000 [147:01:58<218:07:18,  5.20s/it]
 33%|███▎      | 74057/225000 [147:02:01<190:45:58,  4.55s/it]
 33%|███▎      | 74058/225000 [147:02:04<170:21:40,  4.06s/it]
 33%|███▎      | 74059/225000 [147:02:07<152:54:09,  3.65s/it]
 33%|███▎      | 74060/225000 [147:02:09<138:56:28,  3.31s/it]
 33%|███▎      | 74061/225000 [147:02:17<197:16:12,  4.71s/it]
 33%|███▎      | 74062/225000 [147:02:22<196:22:57,  4.68s/it]
 33%|███▎      | 74063/225000 [147:02:25<181:05:38,  4.32s/it]
 33%|███▎      | 74064/225000 [147:02:28<159:43:23,  3.81s/it]
 33%|███▎      | 74065/225000 [147:02:30<144:02:16,  3.44s/it]
 33%|███▎      | 74066/225000 [147:02:33<133:53:00,  3.19s/it]
 33%|███▎      | 74067/225000 [147:02:41<197:39:37,  4.71s/it]
 33%|███▎      | 74068/225000 [147:02:44<178:05:42,  4.25s/it]
 33%|███▎      | 74069/225000 [147:02:51<200:43:08,  4.79s/it]
 33%|███▎      | 74070/225000 [147:02:54<179:21:17,  4.28s/it]
 33%|███▎      | 74071/225000 [147:02:57<168:44:28,  4.02s/it]
 33%|███▎      | 74072/225000 [147:03:02<175:36:52,  4.19s/it]
 33%|███▎      | 74073/225000 [147:03:08<208:59:01,  4.98s/it]
 33%|███▎      | 74074/225000 [147:03:13<205:47:23,  4.91s/it]
 33%|███▎      | 74075/225000 [147:03:17<188:57:08,  4.51s/it]
 33%|███▎      | 74076/225000 [147:03:20<172:04:47,  4.10s/it]
 33%|███▎      | 74077/225000 [147:03:23<158:24:10,  3.78s/it]
 33%|███▎      | 74078/225000 [147:03:26<152:51:37,  3.65s/it]
 33%|███▎      | 74079/225000 [147:03:30<155:38:02,  3.71s/it]
 33%|███▎      | 74080/225000 [147:03:33<148:58:00,  3.55s/it]
 33%|███▎      | 74081/225000 [147:03:37<144:35:47,  3.45s/it]
 33%|███▎      | 74082/225000 [147:03:40<145:57:56,  3.48s/it]
 33%|███▎      | 74083/225000 [147:03:44<146:33:29,  3.50s/it]
 33%|███▎      | 74084/225000 [147:03:47<144:13:04,  3.44s/it]
 33%|███▎      | 74085/225000 [147:03:50<142:11:43,  3.39s/it]
 33%|███▎      | 74086/225000 [147:03:53<134:35:58,  3.21s/it]
 33%|███▎      | 74087/225000 [147:04:01<194:07:45,  4.63s/it]
 33%|███▎      | 74088/225000 [147:04:08<230:36:07,  5.50s/it]
 33%|███▎      | 74089/225000 [147:04:11<196:54:18,  4.70s/it]
 33%|███▎      | 74090/225000 [147:04:14<177:21:32,  4.23s/it]
 33%|███▎      | 74091/225000 [147:04:20<197:32:44,  4.71s/it]
 33%|███▎      | 74092/225000 [147:04:25<198:03:25,  4.72s/it]
 33%|███▎      | 74093/225000 [147:04:28<174:37:58,  4.17s/it]
 33%|███▎      | 74094/225000 [147:04:31<162:19:17,  3.87s/it]
 33%|███▎      | 74095/225000 [147:04:34<149:31:37,  3.57s/it]
 33%|███▎      | 74096/225000 [147:04:37<145:04:00,  3.46s/it]
 33%|███▎      | 74097/225000 [147:04:40<139:16:48,  3.32s/it]
 33%|███▎      | 74098/225000 [147:04:44<150:33:34,  3.59s/it]
 33%|███▎      | 74099/225000 [147:04:54<220:30:23,  5.26s/it]
 33%|███▎      | 74100/225000 [147:04:57<197:00:06,  4.70s/it]
                                                              
{'loss': '0.8647', 'grad_norm': '1.671', 'learning_rate': '3.982e-05', 'epoch': '0.3293', 'num_input_tokens_seen': 243283008, 'train_runtime': '5.295e+05', 'train_tokens_per_second': '459.5'}

 33%|███▎      | 74100/225000 [147:04:57<197:00:06,  4.70s/it]
 33%|███▎      | 74101/225000 [147:05:01<193:00:44,  4.60s/it]
 33%|███▎      | 74102/225000 [147:05:06<195:25:14,  4.66s/it]
 33%|███▎      | 74103/225000 [147:05:09<169:55:44,  4.05s/it]
 33%|███▎      | 74104/225000 [147:05:12<155:59:48,  3.72s/it]
 33%|███▎      | 74105/225000 [147:05:15<149:22:51,  3.56s/it]
 33%|███▎      | 74106/225000 [147:05:24<217:25:34,  5.19s/it]
 33%|███▎      | 74107/225000 [147:05:27<191:16:40,  4.56s/it]
 33%|███▎      | 74108/225000 [147:05:30<170:29:16,  4.07s/it]
 33%|███▎      | 74109/225000 [147:05:33<156:32:11,  3.73s/it]
 33%|███▎      | 74110/225000 [147:05:40<197:38:45,  4.72s/it]
 33%|███▎      | 74111/225000 [147:05:44<189:39:19,  4.52s/it]
 33%|███▎      | 74112/225000 [147:05:46<160:38:42,  3.83s/it]
 33%|███▎      | 74113/225000 [147:05:49<149:49:02,  3.57s/it]
 33%|███▎      | 74114/225000 [147:05:52<142:44:12,  3.41s/it]
 33%|███▎      | 74115/225000 [147:05:55<137:07:14,  3.27s/it]
 33%|███▎      | 74116/225000 [147:05:59<144:16:35,  3.44s/it]
 33%|███▎      | 74117/225000 [147:06:04<168:19:21,  4.02s/it]
 33%|███▎      | 74118/225000 [147:06:14<239:49:04,  5.72s/it]
 33%|███▎      | 74119/225000 [147:06:18<215:23:53,  5.14s/it]
 33%|███▎      | 74120/225000 [147:06:21<190:56:10,  4.56s/it]
 33%|███▎      | 74121/225000 [147:06:24<172:57:52,  4.13s/it]
 33%|███▎      | 74122/225000 [147:06:27<159:28:00,  3.80s/it]
 33%|███▎      | 74123/225000 [147:06:30<154:04:45,  3.68s/it]
 33%|███▎      | 74124/225000 [147:06:33<141:22:22,  3.37s/it]
 33%|███▎      | 74125/225000 [147:06:42<215:45:33,  5.15s/it]
 33%|███▎      | 74126/225000 [147:06:45<181:45:13,  4.34s/it]
 33%|███▎      | 74127/225000 [147:06:48<160:49:40,  3.84s/it]
 33%|███▎      | 74128/225000 [147:06:50<148:50:42,  3.55s/it]
 33%|███▎      | 74129/225000 [147:06:53<133:55:59,  3.20s/it]
 33%|███▎      | 74130/225000 [147:06:55<125:38:51,  3.00s/it]
 33%|███▎      | 74131/225000 [147:07:01<156:11:57,  3.73s/it]
 33%|███▎      | 74132/225000 [147:07:08<196:53:06,  4.70s/it]
 33%|███▎      | 74133/225000 [147:07:10<168:04:05,  4.01s/it]
 33%|███▎      | 74134/225000 [147:07:13<152:49:52,  3.65s/it]
 33%|███▎      | 74135/225000 [147:07:15<136:50:48,  3.27s/it]
 33%|███▎      | 74136/225000 [147:07:19<136:45:13,  3.26s/it]
 33%|███▎      | 74137/225000 [147:07:25<175:21:59,  4.18s/it]
 33%|███▎      | 74138/225000 [147:07:30<187:57:58,  4.49s/it]
 33%|███▎      | 74139/225000 [147:07:33<167:58:48,  4.01s/it]
 33%|███▎      | 74140/225000 [147:07:36<158:46:05,  3.79s/it]
 33%|███▎      | 74141/225000 [147:07:40<152:40:25,  3.64s/it]
 33%|███▎      | 74142/225000 [147:07:43<150:01:50,  3.58s/it]
 33%|███▎      | 74143/225000 [147:07:46<148:35:29,  3.55s/it]
 33%|███▎      | 74144/225000 [147:07:49<139:41:20,  3.33s/it]
 33%|███▎      | 74145/225000 [147:07:52<132:15:35,  3.16s/it]
 33%|███▎      | 74146/225000 [147:07:55<127:58:08,  3.05s/it]
 33%|███▎      | 74147/225000 [147:07:58<134:17:01,  3.20s/it]
 33%|███▎      | 74148/225000 [147:08:04<157:57:29,  3.77s/it]
 33%|███▎      | 74149/225000 [147:08:12<216:17:52,  5.16s/it]
 33%|███▎      | 74150/225000 [147:08:19<243:17:00,  5.81s/it]
                                                              
{'loss': '0.887', 'grad_norm': '1.982', 'learning_rate': '3.981e-05', 'epoch': '0.3296', 'num_input_tokens_seen': 243366272, 'train_runtime': '5.297e+05', 'train_tokens_per_second': '459.4'}

 33%|███▎      | 74150/225000 [147:08:19<243:17:00,  5.81s/it]
 33%|███▎      | 74151/225000 [147:08:21<197:57:33,  4.72s/it]
 33%|███▎      | 74152/225000 [147:08:28<224:00:59,  5.35s/it]
 33%|███▎      | 74153/225000 [147:08:31<192:43:25,  4.60s/it]
 33%|███▎      | 74154/225000 [147:08:34<166:15:05,  3.97s/it]
 33%|███▎      | 74155/225000 [147:08:36<148:55:39,  3.55s/it]
 33%|███▎      | 74156/225000 [147:08:39<134:53:15,  3.22s/it]
 33%|███▎      | 74157/225000 [147:08:42<131:27:23,  3.14s/it]
 33%|███▎      | 74158/225000 [147:08:51<214:37:28,  5.12s/it]
 33%|███▎      | 74159/225000 [147:08:55<193:01:23,  4.61s/it]
 33%|███▎      | 74160/225000 [147:08:58<175:10:18,  4.18s/it]
 33%|███▎      | 74161/225000 [147:09:02<170:42:21,  4.07s/it]
 33%|███▎      | 74162/225000 [147:09:04<152:00:05,  3.63s/it]
 33%|███▎      | 74163/225000 [147:09:07<138:21:42,  3.30s/it]
 33%|███▎      | 74164/225000 [147:09:12<165:31:16,  3.95s/it]
 33%|███▎      | 74165/225000 [147:09:19<202:37:31,  4.84s/it]
 33%|███▎      | 74166/225000 [147:09:22<179:23:12,  4.28s/it]
 33%|███▎      | 74167/225000 [147:09:28<200:25:59,  4.78s/it]
 33%|███▎      | 74168/225000 [147:09:31<173:22:43,  4.14s/it]
 33%|███▎      | 74169/225000 [147:09:34<156:21:56,  3.73s/it]
 33%|███▎      | 74170/225000 [147:09:42<212:53:21,  5.08s/it]
 33%|███▎      | 74171/225000 [147:09:45<186:05:19,  4.44s/it]
 33%|███▎      | 74172/225000 [147:09:48<167:12:14,  3.99s/it]
 33%|███▎      | 74173/225000 [147:09:50<146:19:53,  3.49s/it]
 33%|███▎      | 74174/225000 [147:09:52<133:05:25,  3.18s/it]
 33%|███▎      | 74175/225000 [147:09:55<126:12:24,  3.01s/it]
 33%|███▎      | 74176/225000 [147:09:58<127:55:00,  3.05s/it]
 33%|███▎      | 74177/225000 [147:10:06<190:58:01,  4.56s/it]
 33%|███▎      | 74178/225000 [147:10:12<199:10:30,  4.75s/it]
 33%|███▎      | 74179/225000 [147:10:14<173:31:07,  4.14s/it]
 33%|███▎      | 74180/225000 [147:10:17<156:21:46,  3.73s/it]
 33%|███▎      | 74181/225000 [147:10:20<147:00:32,  3.51s/it]
 33%|███▎      | 74182/225000 [147:10:23<134:20:10,  3.21s/it]
 33%|███▎      | 74183/225000 [147:10:25<123:56:35,  2.96s/it]
 33%|███▎      | 74184/225000 [147:10:28<124:43:31,  2.98s/it]
 33%|███▎      | 74185/225000 [147:10:31<124:27:56,  2.97s/it]
 33%|███▎      | 74186/225000 [147:10:39<195:29:55,  4.67s/it]
 33%|███▎      | 74187/225000 [147:10:47<229:14:48,  5.47s/it]
 33%|███▎      | 74188/225000 [147:10:50<198:44:34,  4.74s/it]
 33%|███▎      | 74189/225000 [147:10:53<173:35:15,  4.14s/it]
 33%|███▎      | 74190/225000 [147:10:55<153:53:48,  3.67s/it]
 33%|███▎      | 74191/225000 [147:10:58<147:55:43,  3.53s/it]
 33%|███▎      | 74192/225000 [147:11:03<157:35:56,  3.76s/it]
 33%|███▎      | 74193/225000 [147:11:06<155:28:19,  3.71s/it]
 33%|███▎      | 74194/225000 [147:11:10<150:20:26,  3.59s/it]
 33%|███▎      | 74195/225000 [147:11:15<170:17:33,  4.07s/it]
 33%|███▎      | 74196/225000 [147:11:22<206:55:54,  4.94s/it]
 33%|███▎      | 74197/225000 [147:11:25<184:47:00,  4.41s/it]
 33%|███▎      | 74198/225000 [147:11:29<180:37:47,  4.31s/it]
 33%|███▎      | 74199/225000 [147:11:36<208:04:05,  4.97s/it]
 33%|███▎      | 74200/225000 [147:11:38<173:00:41,  4.13s/it]
                                                              
{'loss': '0.8985', 'grad_norm': '3.734', 'learning_rate': '3.979e-05', 'epoch': '0.3298', 'num_input_tokens_seen': 243448888, 'train_runtime': '5.299e+05', 'train_tokens_per_second': '459.4'}

 33%|███▎      | 74200/225000 [147:11:38<173:00:41,  4.13s/it]
 33%|███▎      | 74201/225000 [147:11:44<204:15:46,  4.88s/it]
 33%|███▎      | 74202/225000 [147:11:47<175:34:53,  4.19s/it]
 33%|███▎      | 74203/225000 [147:11:50<158:05:44,  3.77s/it]
 33%|███▎      | 74204/225000 [147:11:53<150:18:35,  3.59s/it]
 33%|███▎      | 74205/225000 [147:11:57<153:30:04,  3.66s/it]
 33%|███▎      | 74206/225000 [147:12:01<155:28:44,  3.71s/it]
 33%|███▎      | 74207/225000 [147:12:04<152:13:58,  3.63s/it]
 33%|███▎      | 74208/225000 [147:12:09<175:57:37,  4.20s/it]
 33%|███▎      | 74209/225000 [147:12:18<229:37:18,  5.48s/it]
 33%|███▎      | 74210/225000 [147:12:23<227:55:27,  5.44s/it]
 33%|███▎      | 74211/225000 [147:12:26<188:17:30,  4.50s/it]
 33%|███▎      | 74212/225000 [147:12:29<172:36:56,  4.12s/it]
 33%|███▎      | 74213/225000 [147:12:34<186:32:20,  4.45s/it]
 33%|███▎      | 74214/225000 [147:12:37<163:44:59,  3.91s/it]
 33%|███▎      | 74215/225000 [147:12:39<142:52:17,  3.41s/it]
 33%|███▎      | 74216/225000 [147:12:42<136:11:46,  3.25s/it]
 33%|███▎      | 74217/225000 [147:12:45<134:40:53,  3.22s/it]
 33%|███▎      | 74218/225000 [147:12:53<189:02:07,  4.51s/it]
 33%|███▎      | 74219/225000 [147:12:55<167:21:41,  4.00s/it]
 33%|███▎      | 74220/225000 [147:12:58<156:49:11,  3.74s/it]
 33%|███▎      | 74221/225000 [147:13:02<151:41:10,  3.62s/it]
 33%|███▎      | 74222/225000 [147:13:05<147:34:16,  3.52s/it]
 33%|███▎      | 74223/225000 [147:13:12<185:24:14,  4.43s/it]
 33%|███▎      | 74224/225000 [147:13:19<226:18:44,  5.40s/it]
 33%|███▎      | 74225/225000 [147:13:22<191:49:11,  4.58s/it]
 33%|███▎      | 74226/225000 [147:13:25<168:01:03,  4.01s/it]
 33%|███▎      | 74227/225000 [147:13:27<149:47:04,  3.58s/it]
 33%|███▎      | 74228/225000 [147:13:30<143:42:42,  3.43s/it]
 33%|███▎      | 74229/225000 [147:13:40<217:01:45,  5.18s/it]
 33%|███▎      | 74230/225000 [147:13:44<209:58:46,  5.01s/it]
 33%|███▎      | 74231/225000 [147:13:46<175:41:13,  4.19s/it]
 33%|███▎      | 74232/225000 [147:13:49<159:58:40,  3.82s/it]
 33%|███▎      | 74233/225000 [147:13:53<152:38:46,  3.64s/it]
 33%|███▎      | 74234/225000 [147:13:55<142:03:29,  3.39s/it]
 33%|███▎      | 74235/225000 [147:13:58<137:02:11,  3.27s/it]
 33%|███▎      | 74236/225000 [147:14:01<132:25:05,  3.16s/it]
 33%|███▎      | 74237/225000 [147:14:04<130:51:49,  3.12s/it]
 33%|███▎      | 74238/225000 [147:14:09<143:40:00,  3.43s/it]
 33%|███▎      | 74239/225000 [147:14:12<143:29:47,  3.43s/it]
 33%|███▎      | 74240/225000 [147:14:18<181:48:51,  4.34s/it]
 33%|███▎      | 74241/225000 [147:14:27<232:10:36,  5.54s/it]
 33%|███▎      | 74242/225000 [147:14:30<196:58:46,  4.70s/it]
 33%|███▎      | 74243/225000 [147:14:32<168:19:31,  4.02s/it]
 33%|███▎      | 74244/225000 [147:14:41<230:05:10,  5.49s/it]
 33%|███▎      | 74245/225000 [147:14:43<189:47:46,  4.53s/it]
 33%|███▎      | 74246/225000 [147:14:45<161:13:54,  3.85s/it]
 33%|███▎      | 74247/225000 [147:14:48<146:53:34,  3.51s/it]
 33%|███▎      | 74248/225000 [147:14:52<145:23:46,  3.47s/it]
 33%|███▎      | 74249/225000 [147:14:54<137:52:30,  3.29s/it]
 33%|███▎      | 74250/225000 [147:14:58<143:54:15,  3.44s/it]
                                                              
{'loss': '0.8756', 'grad_norm': '1.465', 'learning_rate': '3.978e-05', 'epoch': '0.33', 'num_input_tokens_seen': 243531616, 'train_runtime': '5.301e+05', 'train_tokens_per_second': '459.4'}

 33%|███▎      | 74250/225000 [147:14:58<143:54:15,  3.44s/it]
 33%|███▎      | 74251/225000 [147:15:06<199:35:20,  4.77s/it]
 33%|███▎      | 74252/225000 [147:15:10<194:47:50,  4.65s/it]
 33%|███▎      | 74253/225000 [147:15:14<178:37:12,  4.27s/it]
 33%|███▎      | 74254/225000 [147:15:17<165:36:13,  3.95s/it]
 33%|███▎      | 74255/225000 [147:15:20<151:33:50,  3.62s/it]
 33%|███▎      | 74256/225000 [147:15:23<140:56:11,  3.37s/it]
 33%|███▎      | 74257/225000 [147:15:25<133:46:21,  3.19s/it]
 33%|███▎      | 74258/225000 [147:15:28<126:52:13,  3.03s/it]
 33%|███▎      | 74259/225000 [147:15:34<168:20:06,  4.02s/it]
 33%|███▎      | 74260/225000 [147:15:41<200:01:49,  4.78s/it]
 33%|███▎      | 74261/225000 [147:15:46<199:50:25,  4.77s/it]
 33%|███▎      | 74262/225000 [147:15:51<211:53:59,  5.06s/it]
 33%|███▎      | 74263/225000 [147:15:54<180:08:15,  4.30s/it]
 33%|███▎      | 74264/225000 [147:15:58<177:30:12,  4.24s/it]
 33%|███▎      | 74265/225000 [147:16:03<180:18:56,  4.31s/it]
 33%|███▎      | 74266/225000 [147:16:05<158:10:53,  3.78s/it]
 33%|███▎      | 74267/225000 [147:16:08<146:21:30,  3.50s/it]
 33%|███▎      | 74268/225000 [147:16:17<220:00:34,  5.25s/it]
 33%|███▎      | 74269/225000 [147:16:20<184:08:07,  4.40s/it]
 33%|███▎      | 74270/225000 [147:16:25<192:39:05,  4.60s/it]
 33%|███▎      | 74271/225000 [147:16:31<211:48:34,  5.06s/it]
 33%|███▎      | 74272/225000 [147:16:33<175:50:32,  4.20s/it]
 33%|███▎      | 74273/225000 [147:16:36<155:48:55,  3.72s/it]
 33%|███▎      | 74274/225000 [147:16:39<145:49:54,  3.48s/it]
 33%|███▎      | 74275/225000 [147:16:42<139:55:25,  3.34s/it]
 33%|███▎      | 74276/225000 [147:16:45<134:39:06,  3.22s/it]
 33%|███▎      | 74277/225000 [147:16:47<125:57:02,  3.01s/it]
 33%|███▎      | 74278/225000 [147:16:50<120:11:24,  2.87s/it]
 33%|███▎      | 74279/225000 [147:16:53<129:48:12,  3.10s/it]
 33%|███▎      | 74280/225000 [147:17:00<180:50:18,  4.32s/it]
 33%|███▎      | 74281/225000 [147:17:08<227:18:50,  5.43s/it]
 33%|███▎      | 74282/225000 [147:17:11<190:48:11,  4.56s/it]
 33%|███▎      | 74283/225000 [147:17:13<165:17:11,  3.95s/it]
 33%|███▎      | 74284/225000 [147:17:16<148:32:08,  3.55s/it]
 33%|███▎      | 74285/225000 [147:17:19<134:20:18,  3.21s/it]
 33%|███▎      | 74286/225000 [147:17:27<200:32:06,  4.79s/it]
 33%|███▎      | 74287/225000 [147:17:31<188:03:16,  4.49s/it]
 33%|███▎      | 74288/225000 [147:17:34<175:24:14,  4.19s/it]
 33%|███▎      | 74289/225000 [147:17:37<162:00:39,  3.87s/it]
 33%|███▎      | 74290/225000 [147:17:41<156:32:00,  3.74s/it]
 33%|███▎      | 74291/225000 [147:17:45<164:09:43,  3.92s/it]
 33%|███▎      | 74292/225000 [147:17:48<148:59:13,  3.56s/it]
 33%|███▎      | 74293/225000 [147:17:51<143:19:06,  3.42s/it]
 33%|███▎      | 74294/225000 [147:17:54<143:07:13,  3.42s/it]
 33%|███▎      | 74295/225000 [147:17:58<140:16:44,  3.35s/it]
 33%|███▎      | 74296/225000 [147:18:00<132:35:52,  3.17s/it]
 33%|███▎      | 74297/225000 [147:18:08<182:55:48,  4.37s/it]
 33%|███▎      | 74298/225000 [147:18:14<210:56:25,  5.04s/it]
 33%|███▎      | 74299/225000 [147:18:17<179:56:33,  4.30s/it]
 33%|███▎      | 74300/225000 [147:18:20<169:30:32,  4.05s/it]
                                                              
{'loss': '0.8428', 'grad_norm': '1.778', 'learning_rate': '3.976e-05', 'epoch': '0.3302', 'num_input_tokens_seen': 243614208, 'train_runtime': '5.303e+05', 'train_tokens_per_second': '459.4'}

 33%|███▎      | 74300/225000 [147:18:20<169:30:32,  4.05s/it]
 33%|███▎      | 74301/225000 [147:18:24<170:31:16,  4.07s/it]
 33%|███▎      | 74302/225000 [147:18:32<217:11:27,  5.19s/it]
 33%|███▎      | 74303/225000 [147:18:35<183:23:48,  4.38s/it]
 33%|███▎      | 74304/225000 [147:18:38<170:27:05,  4.07s/it]
 33%|███▎      | 74305/225000 [147:18:41<158:39:42,  3.79s/it]
 33%|███▎      | 74306/225000 [147:18:44<148:06:55,  3.54s/it]
 33%|███▎      | 74307/225000 [147:18:47<139:26:28,  3.33s/it]
 33%|███▎      | 74308/225000 [147:18:49<129:59:47,  3.11s/it]
 33%|███▎      | 74309/225000 [147:18:57<189:47:00,  4.53s/it]
 33%|███▎      | 74310/225000 [147:19:03<207:26:59,  4.96s/it]
 33%|███▎      | 74311/225000 [147:19:06<183:05:51,  4.37s/it]
 33%|███▎      | 74312/225000 [147:19:09<162:49:26,  3.89s/it]
 33%|███▎      | 74313/225000 [147:19:17<219:53:04,  5.25s/it]
 33%|███▎      | 74314/225000 [147:19:20<186:15:33,  4.45s/it]
 33%|███▎      | 74315/225000 [147:19:24<186:00:31,  4.44s/it]
 33%|███▎      | 74316/225000 [147:19:28<168:35:41,  4.03s/it]
 33%|███▎      | 74317/225000 [147:19:31<163:43:33,  3.91s/it]
 33%|███▎      | 74318/225000 [147:19:35<156:57:14,  3.75s/it]
 33%|███▎      | 74319/225000 [147:19:38<155:19:15,  3.71s/it]
 33%|███▎      | 74320/225000 [147:19:41<145:07:55,  3.47s/it]
 33%|███▎      | 74321/225000 [147:19:50<208:12:44,  4.97s/it]
 33%|███▎      | 74322/225000 [147:19:52<180:09:09,  4.30s/it]
 33%|███▎      | 74323/225000 [147:19:55<161:26:28,  3.86s/it]
 33%|███▎      | 74324/225000 [147:19:59<159:20:16,  3.81s/it]
 33%|███▎      | 74325/225000 [147:20:02<154:56:12,  3.70s/it]
 33%|███▎      | 74326/225000 [147:20:05<149:13:48,  3.57s/it]
 33%|███▎      | 74327/225000 [147:20:11<175:13:51,  4.19s/it]
 33%|███▎      | 74328/225000 [147:20:14<161:30:08,  3.86s/it]
 33%|███▎      | 74329/225000 [147:20:18<158:29:14,  3.79s/it]
 33%|███▎      | 74330/225000 [147:20:23<171:38:12,  4.10s/it]
 33%|███▎      | 74331/225000 [147:20:26<161:47:56,  3.87s/it]
 33%|███▎      | 74332/225000 [147:20:29<150:06:21,  3.59s/it]
 33%|███▎      | 74333/225000 [147:20:32<144:55:50,  3.46s/it]
 33%|███▎      | 74334/225000 [147:20:35<141:30:30,  3.38s/it]
 33%|███▎      | 74335/225000 [147:20:39<142:46:41,  3.41s/it]
 33%|███▎      | 74336/225000 [147:20:42<146:14:16,  3.49s/it]
 33%|███▎      | 74337/225000 [147:20:45<139:08:42,  3.32s/it]
 33%|███▎      | 74338/225000 [147:20:49<144:57:55,  3.46s/it]
 33%|███▎      | 74339/225000 [147:20:53<154:59:43,  3.70s/it]
 33%|███▎      | 74340/225000 [147:20:58<164:25:37,  3.93s/it]
 33%|███▎      | 74341/225000 [147:21:03<181:45:22,  4.34s/it]
 33%|███▎      | 74342/225000 [147:21:12<234:29:09,  5.60s/it]
 33%|███▎      | 74343/225000 [147:21:16<217:10:37,  5.19s/it]
 33%|███▎      | 74344/225000 [147:21:19<189:30:47,  4.53s/it]
 33%|███▎      | 74345/225000 [147:21:22<168:35:39,  4.03s/it]
 33%|███▎      | 74346/225000 [147:21:25<157:12:38,  3.76s/it]
 33%|███▎      | 74347/225000 [147:21:30<172:15:03,  4.12s/it]
 33%|███▎      | 74348/225000 [147:21:37<203:57:19,  4.87s/it]
 33%|███▎      | 74349/225000 [147:21:40<180:28:40,  4.31s/it]
 33%|███▎      | 74350/225000 [147:21:43<165:13:25,  3.95s/it]
                                                              
{'loss': '0.8789', 'grad_norm': '1.394', 'learning_rate': '3.975e-05', 'epoch': '0.3304', 'num_input_tokens_seen': 243695720, 'train_runtime': '5.305e+05', 'train_tokens_per_second': '459.4'}

 33%|███▎      | 74350/225000 [147:21:43<165:13:25,  3.95s/it]
 33%|███▎      | 74351/225000 [147:21:46<163:08:47,  3.90s/it]
 33%|███▎      | 74352/225000 [147:21:50<161:06:08,  3.85s/it]
 33%|███▎      | 74353/225000 [147:21:54<156:21:58,  3.74s/it]
 33%|███▎      | 74354/225000 [147:22:02<217:08:50,  5.19s/it]
 33%|███▎      | 74355/225000 [147:22:04<180:45:03,  4.32s/it]
 33%|███▎      | 74356/225000 [147:22:07<158:21:27,  3.78s/it]
 33%|███▎      | 74357/225000 [147:22:13<185:18:07,  4.43s/it]
 33%|███▎      | 74358/225000 [147:22:18<187:58:20,  4.49s/it]
 33%|███▎      | 74359/225000 [147:22:20<162:28:14,  3.88s/it]
 33%|███▎      | 74360/225000 [147:22:23<145:49:10,  3.48s/it]
 33%|███▎      | 74361/225000 [147:22:25<133:41:54,  3.20s/it]
 33%|███▎      | 74362/225000 [147:22:28<124:03:04,  2.96s/it]
 33%|███▎      | 74363/225000 [147:22:30<120:18:17,  2.88s/it]
 33%|███▎      | 74364/225000 [147:22:39<191:34:24,  4.58s/it]
 33%|███▎      | 74365/225000 [147:22:42<176:56:39,  4.23s/it]
 33%|███▎      | 74366/225000 [147:22:47<187:24:03,  4.48s/it]
 33%|███▎      | 74367/225000 [147:22:52<184:38:11,  4.41s/it]
 33%|███▎      | 74368/225000 [147:22:54<159:22:57,  3.81s/it]
 33%|███▎      | 74369/225000 [147:22:57<147:39:16,  3.53s/it]
 33%|███▎      | 74370/225000 [147:22:59<133:27:03,  3.19s/it]
 33%|███▎      | 74371/225000 [147:23:02<124:12:24,  2.97s/it]
 33%|███▎      | 74372/225000 [147:23:04<122:43:49,  2.93s/it]
 33%|███▎      | 74373/225000 [147:23:07<122:04:50,  2.92s/it]
 33%|███▎      | 74374/225000 [147:23:11<130:23:32,  3.12s/it]
 33%|███▎      | 74375/225000 [147:23:14<128:08:00,  3.06s/it]
 33%|███▎      | 74376/225000 [147:23:17<125:09:43,  2.99s/it]
 33%|███▎      | 74377/225000 [147:23:20<127:41:20,  3.05s/it]
 33%|███▎      | 74378/225000 [147:23:29<198:07:37,  4.74s/it]
 33%|███▎      | 74379/225000 [147:23:37<238:14:45,  5.69s/it]
 33%|███▎      | 74380/225000 [147:23:40<214:42:05,  5.13s/it]
 33%|███▎      | 74381/225000 [147:23:43<181:35:21,  4.34s/it]
 33%|███▎      | 74382/225000 [147:23:46<167:09:27,  4.00s/it]
 33%|███▎      | 74383/225000 [147:23:49<154:57:35,  3.70s/it]
 33%|███▎      | 74384/225000 [147:23:55<181:51:03,  4.35s/it]
 33%|███▎      | 74385/225000 [147:24:00<190:05:58,  4.54s/it]
 33%|███▎      | 74386/225000 [147:24:02<163:46:10,  3.91s/it]
 33%|███▎      | 74387/225000 [147:24:05<146:40:16,  3.51s/it]
 33%|███▎      | 74388/225000 [147:24:08<143:24:37,  3.43s/it]
 33%|███▎      | 74389/225000 [147:24:15<187:20:08,  4.48s/it]
 33%|███▎      | 74390/225000 [147:24:20<190:27:42,  4.55s/it]
 33%|███▎      | 74391/225000 [147:24:23<172:38:42,  4.13s/it]
 33%|███▎      | 74392/225000 [147:24:25<151:28:45,  3.62s/it]
 33%|███▎      | 74393/225000 [147:24:28<141:48:55,  3.39s/it]
 33%|███▎      | 74394/225000 [147:24:31<138:45:25,  3.32s/it]
 33%|███▎      | 74395/225000 [147:24:40<204:53:23,  4.90s/it]
 33%|███▎      | 74396/225000 [147:24:43<185:01:30,  4.42s/it]
 33%|███▎      | 74397/225000 [147:24:46<168:36:08,  4.03s/it]
 33%|███▎      | 74398/225000 [147:24:50<158:59:05,  3.80s/it]
 33%|███▎      | 74399/225000 [147:24:53<152:32:25,  3.65s/it]
 33%|███▎      | 74400/225000 [147:25:02<222:40:57,  5.32s/it]
                                                              
{'loss': '0.9083', 'grad_norm': '1.733', 'learning_rate': '3.973e-05', 'epoch': '0.3307', 'num_input_tokens_seen': 243776800, 'train_runtime': '5.307e+05', 'train_tokens_per_second': '459.3'}

 33%|███▎      | 74400/225000 [147:25:02<222:40:57,  5.32s/it]
 33%|███▎      | 74401/225000 [147:25:07<219:55:03,  5.26s/it]
 33%|███▎      | 74402/225000 [147:25:11<207:02:27,  4.95s/it]
 33%|███▎      | 74403/225000 [147:25:14<178:50:19,  4.28s/it]
 33%|███▎      | 74404/225000 [147:25:17<162:29:06,  3.88s/it]
 33%|███▎      | 74405/225000 [147:25:25<210:11:49,  5.02s/it]
 33%|███▎      | 74406/225000 [147:25:28<185:48:34,  4.44s/it]
 33%|███▎      | 74407/225000 [147:25:30<159:32:00,  3.81s/it]
 33%|███▎      | 74408/225000 [147:25:33<150:48:47,  3.61s/it]
 33%|███▎      | 74409/225000 [147:25:37<146:19:49,  3.50s/it]
 33%|███▎      | 74410/225000 [147:25:40<143:19:10,  3.43s/it]
 33%|███▎      | 74411/225000 [147:25:43<141:26:28,  3.38s/it]
 33%|███▎      | 74412/225000 [147:25:47<148:50:26,  3.56s/it]
 33%|███▎      | 74413/225000 [147:25:56<212:38:51,  5.08s/it]
 33%|███▎      | 74414/225000 [147:26:00<207:46:30,  4.97s/it]
 33%|███▎      | 74415/225000 [147:26:03<181:09:59,  4.33s/it]
 33%|███▎      | 74416/225000 [147:26:06<165:49:37,  3.96s/it]
 33%|███▎      | 74417/225000 [147:26:10<156:48:12,  3.75s/it]
 33%|███▎      | 74418/225000 [147:26:17<198:24:19,  4.74s/it]
 33%|███▎      | 74419/225000 [147:26:22<205:12:41,  4.91s/it]
 33%|███▎      | 74420/225000 [147:26:25<179:47:14,  4.30s/it]
 33%|███▎      | 74421/225000 [147:26:27<154:23:54,  3.69s/it]
 33%|███▎      | 74422/225000 [147:26:35<204:16:10,  4.88s/it]
 33%|███▎      | 74423/225000 [147:26:38<181:29:49,  4.34s/it]
 33%|███▎      | 74424/225000 [147:26:42<175:02:51,  4.19s/it]
 33%|███▎      | 74425/225000 [147:26:45<163:25:59,  3.91s/it]
 33%|███▎      | 74426/225000 [147:26:48<157:26:52,  3.76s/it]
 33%|███▎      | 74427/225000 [147:26:51<146:25:44,  3.50s/it]
 33%|███▎      | 74428/225000 [147:26:54<139:21:59,  3.33s/it]
 33%|███▎      | 74429/225000 [147:26:57<137:16:53,  3.28s/it]
 33%|███▎      | 74430/225000 [147:27:03<161:17:27,  3.86s/it]
 33%|███▎      | 74431/225000 [147:27:09<196:11:08,  4.69s/it]
 33%|███▎      | 74432/225000 [147:27:12<172:42:09,  4.13s/it]
 33%|███▎      | 74433/225000 [147:27:15<155:37:11,  3.72s/it]
 33%|███▎      | 74434/225000 [147:27:18<143:02:04,  3.42s/it]
 33%|███▎      | 74435/225000 [147:27:26<200:24:16,  4.79s/it]
 33%|███▎      | 74436/225000 [147:27:28<171:39:45,  4.10s/it]
 33%|███▎      | 74437/225000 [147:27:31<152:21:50,  3.64s/it]
 33%|███▎      | 74438/225000 [147:27:37<183:31:09,  4.39s/it]
 33%|███▎      | 74439/225000 [147:27:42<198:44:20,  4.75s/it]
 33%|███▎      | 74440/225000 [147:27:45<172:32:54,  4.13s/it]
 33%|███▎      | 74441/225000 [147:27:48<158:08:22,  3.78s/it]
 33%|███▎      | 74442/225000 [147:27:51<146:17:08,  3.50s/it]
 33%|███▎      | 74443/225000 [147:27:59<204:05:59,  4.88s/it]
 33%|███▎      | 74444/225000 [147:28:01<173:15:10,  4.14s/it]
 33%|███▎      | 74445/225000 [147:28:05<161:29:16,  3.86s/it]
 33%|███▎      | 74446/225000 [147:28:07<146:05:04,  3.49s/it]
 33%|███▎      | 74447/225000 [147:28:10<141:07:41,  3.37s/it]
 33%|███▎      | 74448/225000 [147:28:13<136:10:20,  3.26s/it]
 33%|███▎      | 74449/225000 [147:28:16<134:59:24,  3.23s/it]
 33%|███▎      | 74450/225000 [147:28:24<185:54:41,  4.45s/it]
                                                              
{'loss': '0.8652', 'grad_norm': '1.622', 'learning_rate': '3.972e-05', 'epoch': '0.3309', 'num_input_tokens_seen': 243859448, 'train_runtime': '5.309e+05', 'train_tokens_per_second': '459.3'}

 33%|███▎      | 74450/225000 [147:28:24<185:54:41,  4.45s/it]
 33%|███▎      | 74451/225000 [147:28:28<187:21:54,  4.48s/it]
 33%|███▎      | 74452/225000 [147:28:32<183:41:26,  4.39s/it]
 33%|███▎      | 74453/225000 [147:28:37<183:32:18,  4.39s/it]
 33%|███▎      | 74454/225000 [147:28:39<156:40:56,  3.75s/it]
 33%|███▎      | 74455/225000 [147:28:42<148:59:55,  3.56s/it]
 33%|███▎      | 74456/225000 [147:28:50<206:48:33,  4.95s/it]
 33%|███▎      | 74457/225000 [147:28:53<182:28:19,  4.36s/it]
 33%|███▎      | 74458/225000 [147:28:56<161:46:39,  3.87s/it]
 33%|███▎      | 74459/225000 [147:28:59<143:16:06,  3.43s/it]
 33%|███▎      | 74460/225000 [147:29:01<131:58:43,  3.16s/it]
 33%|███▎      | 74461/225000 [147:29:04<127:43:36,  3.05s/it]
 33%|███▎      | 74462/225000 [147:29:06<121:12:17,  2.90s/it]
 33%|███▎      | 74463/225000 [147:29:10<128:22:22,  3.07s/it]
 33%|███▎      | 74464/225000 [147:29:13<133:58:13,  3.20s/it]
 33%|███▎      | 74465/225000 [147:29:17<134:12:51,  3.21s/it]
 33%|███▎      | 74466/225000 [147:29:22<165:25:41,  3.96s/it]
 33%|███▎      | 74467/225000 [147:29:31<220:31:55,  5.27s/it]
 33%|███▎      | 74468/225000 [147:29:35<207:39:02,  4.97s/it]
 33%|███▎      | 74469/225000 [147:29:38<183:17:36,  4.38s/it]
 33%|███▎      | 74470/225000 [147:29:42<177:51:06,  4.25s/it]
 33%|███▎      | 74471/225000 [147:29:45<167:30:16,  4.01s/it]
 33%|███▎      | 74472/225000 [147:29:48<146:48:35,  3.51s/it]
 33%|███▎      | 74473/225000 [147:29:51<138:14:31,  3.31s/it]
 33%|███▎      | 74474/225000 [147:29:57<172:03:39,  4.12s/it]
 33%|███▎      | 74475/225000 [147:30:03<201:01:30,  4.81s/it]
 33%|███▎      | 74476/225000 [147:30:06<177:42:47,  4.25s/it]
 33%|███▎      | 74477/225000 [147:30:09<160:22:34,  3.84s/it]
 33%|███▎      | 74478/225000 [147:30:12<153:02:38,  3.66s/it]
 33%|███▎      | 74479/225000 [147:30:15<141:13:14,  3.38s/it]
 33%|███▎      | 74480/225000 [147:30:17<132:39:44,  3.17s/it]
 33%|███▎      | 74481/225000 [147:30:21<132:32:43,  3.17s/it]
 33%|███▎      | 74482/225000 [147:30:23<126:48:22,  3.03s/it]
 33%|███▎      | 74483/225000 [147:30:28<142:57:14,  3.42s/it]
 33%|███▎      | 74484/225000 [147:30:37<222:47:49,  5.33s/it]
 33%|███▎      | 74485/225000 [147:30:41<199:31:43,  4.77s/it]
 33%|███▎      | 74486/225000 [147:30:43<170:23:38,  4.08s/it]
 33%|███▎      | 74487/225000 [147:30:46<156:51:15,  3.75s/it]
 33%|███▎      | 74488/225000 [147:30:49<143:23:19,  3.43s/it]
 33%|███▎      | 74489/225000 [147:30:53<147:43:37,  3.53s/it]
 33%|███▎      | 74490/225000 [147:31:01<210:04:50,  5.02s/it]
 33%|███▎      | 74491/225000 [147:31:04<185:24:45,  4.43s/it]
 33%|███▎      | 74492/225000 [147:31:07<168:01:04,  4.02s/it]
 33%|███▎      | 74493/225000 [147:31:10<150:39:17,  3.60s/it]
 33%|███▎      | 74494/225000 [147:31:13<144:14:30,  3.45s/it]
 33%|███▎      | 74495/225000 [147:31:16<137:20:16,  3.29s/it]
 33%|███▎      | 74496/225000 [147:31:19<136:24:22,  3.26s/it]
 33%|███▎      | 74497/225000 [147:31:22<135:02:27,  3.23s/it]
 33%|███▎      | 74498/225000 [147:31:30<191:04:25,  4.57s/it]
 33%|███▎      | 74499/225000 [147:31:37<223:43:58,  5.35s/it]
 33%|███▎      | 74500/225000 [147:31:40<194:09:48,  4.64s/it]
                                                              
{'loss': '0.9133', 'grad_norm': '2.078', 'learning_rate': '3.971e-05', 'epoch': '0.3311', 'num_input_tokens_seen': 243940176, 'train_runtime': '5.311e+05', 'train_tokens_per_second': '459.3'}

 33%|███▎      | 74500/225000 [147:31:40<194:09:48,  4.64s/it]
 33%|███▎      | 74501/225000 [147:31:43<167:39:01,  4.01s/it]
 33%|███▎      | 74502/225000 [147:31:46<155:40:10,  3.72s/it]
 33%|███▎      | 74503/225000 [147:31:50<164:40:57,  3.94s/it]
 33%|███▎      | 74504/225000 [147:31:57<203:57:06,  4.88s/it]
 33%|███▎      | 74505/225000 [147:32:00<172:31:03,  4.13s/it]
 33%|███▎      | 74506/225000 [147:32:02<150:39:46,  3.60s/it]
 33%|███▎      | 74507/225000 [147:32:10<204:45:09,  4.90s/it]
 33%|███▎      | 74508/225000 [147:32:14<192:29:39,  4.60s/it]
 33%|███▎      | 74509/225000 [147:32:17<169:56:48,  4.07s/it]
 33%|███▎      | 74510/225000 [147:32:20<164:08:37,  3.93s/it]
 33%|███▎      | 74511/225000 [147:32:24<162:19:00,  3.88s/it]
 33%|███▎      | 74512/225000 [147:32:27<150:38:20,  3.60s/it]
 33%|███▎      | 74513/225000 [147:32:30<139:35:29,  3.34s/it]
 33%|███▎      | 74514/225000 [147:32:33<135:00:39,  3.23s/it]
 33%|███▎      | 74515/225000 [147:32:42<208:27:04,  4.99s/it]
 33%|███▎      | 74516/225000 [147:32:47<207:40:16,  4.97s/it]
 33%|███▎      | 74517/225000 [147:32:50<181:35:33,  4.34s/it]
 33%|███▎      | 74518/225000 [147:32:53<164:58:55,  3.95s/it]
 33%|███▎      | 74519/225000 [147:32:55<148:00:00,  3.54s/it]
 33%|███▎      | 74520/225000 [147:32:59<146:50:12,  3.51s/it]
 33%|███▎      | 74521/225000 [147:33:05<178:06:54,  4.26s/it]
 33%|███▎      | 74522/225000 [147:33:10<188:12:41,  4.50s/it]
 33%|███▎      | 74523/225000 [147:33:12<163:18:11,  3.91s/it]
 33%|███▎      | 74524/225000 [147:33:16<158:45:01,  3.80s/it]
 33%|███▎      | 74525/225000 [147:33:19<148:19:55,  3.55s/it]
 33%|███▎      | 74526/225000 [147:33:24<165:01:32,  3.95s/it]
 33%|███▎      | 74527/225000 [147:33:29<179:46:04,  4.30s/it]
 33%|███▎      | 74528/225000 [147:33:33<174:20:34,  4.17s/it]
 33%|███▎      | 74529/225000 [147:33:36<166:54:50,  3.99s/it]
 33%|███▎      | 74530/225000 [147:33:39<152:42:14,  3.65s/it]
 33%|███▎      | 74531/225000 [147:33:42<137:53:44,  3.30s/it]
 33%|███▎      | 74532/225000 [147:33:45<134:21:59,  3.21s/it]
 33%|███▎      | 74533/225000 [147:33:47<129:42:36,  3.10s/it]
 33%|███▎      | 74534/225000 [147:33:51<129:57:55,  3.11s/it]
 33%|███▎      | 74535/225000 [147:33:57<175:28:11,  4.20s/it]
 33%|███▎      | 74536/225000 [147:34:05<218:30:45,  5.23s/it]
 33%|███▎      | 74537/225000 [147:34:07<184:16:50,  4.41s/it]
 33%|███▎      | 74538/225000 [147:34:10<156:23:03,  3.74s/it]
 33%|███▎      | 74539/225000 [147:34:12<138:56:31,  3.32s/it]
 33%|███▎      | 74540/225000 [147:34:20<194:34:10,  4.66s/it]
 33%|███▎      | 74541/225000 [147:34:23<175:09:09,  4.19s/it]
 33%|███▎      | 74542/225000 [147:34:25<154:36:58,  3.70s/it]
 33%|███▎      | 74543/225000 [147:34:28<140:01:26,  3.35s/it]
 33%|███▎      | 74544/225000 [147:34:37<213:41:59,  5.11s/it]
 33%|███▎      | 74545/225000 [147:34:40<184:44:41,  4.42s/it]
 33%|███▎      | 74546/225000 [147:34:44<180:42:36,  4.32s/it]
 33%|███▎      | 74547/225000 [147:34:47<165:10:15,  3.95s/it]
 33%|███▎      | 74548/225000 [147:34:50<153:12:47,  3.67s/it]
 33%|███▎      | 74549/225000 [147:34:53<148:26:16,  3.55s/it]
 33%|███▎      | 74550/225000 [147:34:57<151:25:01,  3.62s/it]
                                                              
{'loss': '0.8936', 'grad_norm': '1.376', 'learning_rate': '3.969e-05', 'epoch': '0.3313', 'num_input_tokens_seen': 244021080, 'train_runtime': '5.313e+05', 'train_tokens_per_second': '459.3'}

 33%|███▎      | 74550/225000 [147:34:57<151:25:01,  3.62s/it]
 33%|███▎      | 74551/225000 [147:35:03<183:37:52,  4.39s/it]
 33%|███▎      | 74552/225000 [147:35:12<236:48:02,  5.67s/it]
 33%|███▎      | 74553/225000 [147:35:14<195:47:06,  4.68s/it]
 33%|███▎      | 74554/225000 [147:35:18<175:48:29,  4.21s/it]
 33%|███▎      | 74555/225000 [147:35:20<157:46:26,  3.78s/it]
 33%|███▎      | 74556/225000 [147:35:23<147:37:02,  3.53s/it]
 33%|███▎      | 74557/225000 [147:35:32<209:28:22,  5.01s/it]
 33%|███▎      | 74558/225000 [147:35:36<199:09:43,  4.77s/it]
 33%|███▎      | 74559/225000 [147:35:39<173:25:33,  4.15s/it]
 33%|███▎      | 74560/225000 [147:35:41<151:01:58,  3.61s/it]
 33%|███▎      | 74561/225000 [147:35:45<152:59:04,  3.66s/it]
 33%|███▎      | 74562/225000 [147:35:50<177:20:23,  4.24s/it]
 33%|███▎      | 74563/225000 [147:35:54<169:23:00,  4.05s/it]
 33%|███▎      | 74564/225000 [147:35:57<155:28:57,  3.72s/it]
 33%|███▎      | 74565/225000 [147:36:00<144:21:29,  3.45s/it]
 33%|███▎      | 74566/225000 [147:36:03<136:51:21,  3.28s/it]
 33%|███▎      | 74567/225000 [147:36:06<135:47:38,  3.25s/it]
 33%|███▎      | 74568/225000 [147:36:09<140:20:23,  3.36s/it]
 33%|███▎      | 74569/225000 [147:36:12<136:00:35,  3.25s/it]
 33%|███▎      | 74570/225000 [147:36:21<207:02:02,  4.95s/it]
 33%|███▎      | 74571/225000 [147:36:27<211:23:52,  5.06s/it]
 33%|███▎      | 74572/225000 [147:36:29<179:24:00,  4.29s/it]
 33%|███▎      | 74573/225000 [147:36:32<158:32:51,  3.79s/it]
 33%|███▎      | 74574/225000 [147:36:34<140:45:18,  3.37s/it]
 33%|███▎      | 74575/225000 [147:36:44<224:39:32,  5.38s/it]
 33%|███▎      | 74576/225000 [147:36:47<197:39:39,  4.73s/it]
 33%|███▎      | 74577/225000 [147:36:50<174:03:04,  4.17s/it]
 33%|███▎      | 74578/225000 [147:36:53<159:03:19,  3.81s/it]
 33%|███▎      | 74579/225000 [147:36:58<171:42:47,  4.11s/it]
 33%|███▎      | 74580/225000 [147:37:01<155:17:42,  3.72s/it]
 33%|███▎      | 74581/225000 [147:37:04<145:31:47,  3.48s/it]
 33%|███▎      | 74582/225000 [147:37:07<140:00:15,  3.35s/it]
 33%|███▎      | 74583/225000 [147:37:10<134:48:41,  3.23s/it]
 33%|███▎      | 74584/225000 [147:37:13<132:26:15,  3.17s/it]
 33%|███▎      | 74585/225000 [147:37:21<196:56:45,  4.71s/it]
 33%|███▎      | 74586/225000 [147:37:28<223:35:42,  5.35s/it]
 33%|███▎      | 74587/225000 [147:37:31<189:34:00,  4.54s/it]
 33%|███▎      | 74588/225000 [147:37:33<164:50:02,  3.95s/it]
 33%|███▎      | 74589/225000 [147:37:36<145:48:20,  3.49s/it]
 33%|███▎      | 74590/225000 [147:37:44<201:49:02,  4.83s/it]
 33%|███▎      | 74591/225000 [147:37:47<189:31:29,  4.54s/it]
 33%|███▎      | 74592/225000 [147:37:51<171:31:50,  4.11s/it]
 33%|███▎      | 74593/225000 [147:37:53<155:00:50,  3.71s/it]
 33%|███▎      | 74594/225000 [147:38:01<206:56:34,  4.95s/it]
 33%|███▎      | 74595/225000 [147:38:05<187:40:14,  4.49s/it]
 33%|███▎      | 74596/225000 [147:38:07<162:24:23,  3.89s/it]
 33%|███▎      | 74597/225000 [147:38:11<158:36:30,  3.80s/it]
 33%|███▎      | 74598/225000 [147:38:17<186:03:32,  4.45s/it]
 33%|███▎      | 74599/225000 [147:38:20<173:50:11,  4.16s/it]
 33%|███▎      | 74600/225000 [147:38:23<157:05:28,  3.76s/it]
                                                              
{'loss': '0.8886', 'grad_norm': '2.65', 'learning_rate': '3.968e-05', 'epoch': '0.3316', 'num_input_tokens_seen': 244102976, 'train_runtime': '5.315e+05', 'train_tokens_per_second': '459.3'}

 33%|███▎      | 74600/225000 [147:38:23<157:05:28,  3.76s/it]
 33%|███▎      | 74601/225000 [147:38:26<148:23:07,  3.55s/it]
 33%|███▎      | 74602/225000 [147:38:33<196:29:31,  4.70s/it]
 33%|███▎      | 74603/225000 [147:38:39<202:09:56,  4.84s/it]
 33%|███▎      | 74604/225000 [147:38:42<185:52:54,  4.45s/it]
 33%|███▎      | 74605/225000 [147:38:45<169:30:37,  4.06s/it]
 33%|███▎      | 74606/225000 [147:38:48<151:59:19,  3.64s/it]
 33%|███▎      | 74607/225000 [147:38:53<166:45:26,  3.99s/it]
 33%|███▎      | 74608/225000 [147:38:58<188:02:46,  4.50s/it]
 33%|███▎      | 74609/225000 [147:39:01<163:31:36,  3.91s/it]
 33%|███▎      | 74610/225000 [147:39:04<149:25:43,  3.58s/it]
 33%|███▎      | 74611/225000 [147:39:09<174:17:05,  4.17s/it]
 33%|███▎      | 74612/225000 [147:39:15<188:08:41,  4.50s/it]
 33%|███▎      | 74613/225000 [147:39:19<181:55:54,  4.36s/it]
 33%|███▎      | 74614/225000 [147:39:22<164:27:30,  3.94s/it]
 33%|███▎      | 74615/225000 [147:39:25<153:05:15,  3.66s/it]
 33%|███▎      | 74616/225000 [147:39:27<139:37:06,  3.34s/it]
 33%|███▎      | 74617/225000 [147:39:31<145:38:03,  3.49s/it]
 33%|███▎      | 74618/225000 [147:39:41<222:21:59,  5.32s/it]
 33%|███▎      | 74619/225000 [147:39:44<202:49:11,  4.86s/it]
 33%|███▎      | 74620/225000 [147:39:47<169:21:31,  4.05s/it]
 33%|███▎      | 74621/225000 [147:39:50<160:18:44,  3.84s/it]
 33%|███▎      | 74622/225000 [147:39:53<148:11:03,  3.55s/it]
 33%|███▎      | 74623/225000 [147:39:56<138:25:22,  3.31s/it]
 33%|███▎      | 74624/225000 [147:39:58<133:35:11,  3.20s/it]
 33%|███▎      | 74625/225000 [147:40:06<188:39:44,  4.52s/it]
 33%|███▎      | 74626/225000 [147:40:12<204:27:41,  4.89s/it]
 33%|███▎      | 74627/225000 [147:40:14<176:08:04,  4.22s/it]
 33%|███▎      | 74628/225000 [147:40:17<157:02:05,  3.76s/it]
 33%|███▎      | 74629/225000 [147:40:20<145:24:29,  3.48s/it]
 33%|███▎      | 74630/225000 [147:40:23<138:41:39,  3.32s/it]
 33%|███▎      | 74631/225000 [147:40:26<130:51:43,  3.13s/it]
 33%|███▎      | 74632/225000 [147:40:31<158:12:39,  3.79s/it]
 33%|███▎      | 74633/225000 [147:40:37<186:42:46,  4.47s/it]
 33%|███▎      | 74634/225000 [147:40:40<169:10:35,  4.05s/it]
 33%|███▎      | 74635/225000 [147:40:43<154:51:59,  3.71s/it]
 33%|███▎      | 74636/225000 [147:40:46<148:10:28,  3.55s/it]
 33%|███▎      | 74637/225000 [147:40:49<144:35:56,  3.46s/it]
 33%|███▎      | 74638/225000 [147:40:53<145:38:04,  3.49s/it]
 33%|███▎      | 74639/225000 [147:40:56<135:22:15,  3.24s/it]
 33%|███▎      | 74640/225000 [147:40:58<128:52:46,  3.09s/it]
 33%|███▎      | 74641/225000 [147:41:01<125:38:34,  3.01s/it]
 33%|███▎      | 74642/225000 [147:41:04<127:23:34,  3.05s/it]
 33%|███▎      | 74643/225000 [147:41:11<173:40:54,  4.16s/it]
 33%|███▎      | 74644/225000 [147:41:18<205:35:42,  4.92s/it]
 33%|███▎      | 74645/225000 [147:41:23<203:13:21,  4.87s/it]
 33%|███▎      | 74646/225000 [147:41:25<176:49:25,  4.23s/it]
 33%|███▎      | 74647/225000 [147:41:28<156:15:15,  3.74s/it]
 33%|███▎      | 74648/225000 [147:41:31<143:48:52,  3.44s/it]
 33%|███▎      | 74649/225000 [147:41:36<164:14:44,  3.93s/it]
 33%|███▎      | 74650/225000 [147:41:43<202:04:44,  4.84s/it]
                                                              
{'loss': '0.9233', 'grad_norm': '1.413', 'learning_rate': '3.966e-05', 'epoch': '0.3318', 'num_input_tokens_seen': 244183112, 'train_runtime': '5.317e+05', 'train_tokens_per_second': '459.2'}

 33%|███▎      | 74650/225000 [147:41:43<202:04:44,  4.84s/it]
 33%|███▎      | 74651/225000 [147:41:45<175:32:49,  4.20s/it]
 33%|███▎      | 74652/225000 [147:41:52<205:42:36,  4.93s/it]
 33%|███▎      | 74653/225000 [147:41:54<171:26:51,  4.11s/it]
 33%|███▎      | 74654/225000 [147:41:57<149:30:07,  3.58s/it]
 33%|███▎      | 74655/225000 [147:41:59<139:38:19,  3.34s/it]
 33%|███▎      | 74656/225000 [147:42:02<133:49:23,  3.20s/it]
 33%|███▎      | 74657/225000 [147:42:06<135:17:37,  3.24s/it]
 33%|███▎      | 74658/225000 [147:42:12<180:57:38,  4.33s/it]
 33%|███▎      | 74659/225000 [147:42:16<171:07:44,  4.10s/it]
 33%|███▎      | 74660/225000 [147:42:21<177:34:56,  4.25s/it]
 33%|███▎      | 74661/225000 [147:42:23<158:43:07,  3.80s/it]
 33%|███▎      | 74662/225000 [147:42:27<155:11:43,  3.72s/it]
 33%|███▎      | 74663/225000 [147:42:30<143:59:09,  3.45s/it]
 33%|███▎      | 74664/225000 [147:42:34<149:28:18,  3.58s/it]
 33%|███▎      | 74665/225000 [147:42:41<202:05:53,  4.84s/it]
 33%|███▎      | 74666/225000 [147:42:46<203:20:01,  4.87s/it]
 33%|███▎      | 74667/225000 [147:42:49<175:47:24,  4.21s/it]
 33%|███▎      | 74668/225000 [147:42:52<157:21:18,  3.77s/it]
 33%|███▎      | 74669/225000 [147:42:55<146:21:12,  3.50s/it]
 33%|███▎      | 74670/225000 [147:42:58<142:40:16,  3.42s/it]
 33%|███▎      | 74671/225000 [147:43:01<138:17:19,  3.31s/it]
 33%|███▎      | 74672/225000 [147:43:05<146:51:19,  3.52s/it]
 33%|███▎      | 74673/225000 [147:43:08<140:35:45,  3.37s/it]
 33%|███▎      | 74674/225000 [147:43:17<213:40:17,  5.12s/it]
 33%|███▎      | 74675/225000 [147:43:23<221:53:44,  5.31s/it]
 33%|███▎      | 74676/225000 [147:43:26<189:20:00,  4.53s/it]
 33%|███▎      | 74677/225000 [147:43:28<160:14:30,  3.84s/it]
 33%|███▎      | 74678/225000 [147:43:32<170:51:02,  4.09s/it]
 33%|███▎      | 74679/225000 [147:43:38<191:19:52,  4.58s/it]
 33%|███▎      | 74680/225000 [147:43:41<169:45:28,  4.07s/it]
 33%|███▎      | 74681/225000 [147:43:44<158:54:23,  3.81s/it]
 33%|███▎      | 74682/225000 [147:43:47<140:53:06,  3.37s/it]
 33%|███▎      | 74683/225000 [147:43:50<142:06:09,  3.40s/it]
 33%|███▎      | 74684/225000 [147:43:59<208:55:15,  5.00s/it]
 33%|███▎      | 74685/225000 [147:44:03<197:15:48,  4.72s/it]
 33%|███▎      | 74686/225000 [147:44:05<168:00:23,  4.02s/it]
 33%|███▎      | 74687/225000 [147:44:08<153:24:46,  3.67s/it]
 33%|███▎      | 74688/225000 [147:44:13<170:40:00,  4.09s/it]
 33%|███▎      | 74689/225000 [147:44:16<161:02:31,  3.86s/it]
 33%|███▎      | 74690/225000 [147:44:19<149:06:19,  3.57s/it]
 33%|███▎      | 74691/225000 [147:44:22<136:39:27,  3.27s/it]
 33%|███▎      | 74692/225000 [147:44:25<137:31:59,  3.29s/it]
 33%|███▎      | 74693/225000 [147:44:34<201:02:00,  4.81s/it]
 33%|███▎      | 74694/225000 [147:44:38<197:25:44,  4.73s/it]
 33%|███▎      | 74695/225000 [147:44:41<175:44:44,  4.21s/it]
 33%|███▎      | 74696/225000 [147:44:44<155:05:49,  3.71s/it]
 33%|███▎      | 74697/225000 [147:44:48<164:00:43,  3.93s/it]
 33%|███▎      | 74698/225000 [147:44:55<205:00:50,  4.91s/it]
 33%|███▎      | 74699/225000 [147:44:58<172:42:53,  4.14s/it]
 33%|███▎      | 74700/225000 [147:45:00<155:00:03,  3.71s/it]
                                                              
{'loss': '0.9076', 'grad_norm': '2.097', 'learning_rate': '3.965e-05', 'epoch': '0.332', 'num_input_tokens_seen': 244264768, 'train_runtime': '5.319e+05', 'train_tokens_per_second': '459.2'}

 33%|███▎      | 74700/225000 [147:45:00<155:00:03,  3.71s/it]
 33%|███▎      | 74701/225000 [147:45:04<154:41:50,  3.71s/it]
 33%|███▎      | 74702/225000 [147:45:13<215:48:26,  5.17s/it]
 33%|███▎      | 74703/225000 [147:45:16<186:11:42,  4.46s/it]
 33%|███▎      | 74704/225000 [147:45:19<169:37:27,  4.06s/it]
 33%|███▎      | 74705/225000 [147:45:22<157:35:54,  3.77s/it]
 33%|███▎      | 74706/225000 [147:45:26<161:35:31,  3.87s/it]
 33%|███▎      | 74707/225000 [147:45:29<155:14:52,  3.72s/it]
 33%|███▎      | 74708/225000 [147:45:32<143:14:28,  3.43s/it]
 33%|███▎      | 74709/225000 [147:45:35<141:13:42,  3.38s/it]
 33%|███▎      | 74710/225000 [147:45:38<131:14:05,  3.14s/it]
 33%|███▎      | 74711/225000 [147:45:41<130:05:56,  3.12s/it]
 33%|███▎      | 74712/225000 [147:45:50<201:31:03,  4.83s/it]
 33%|███▎      | 74713/225000 [147:45:57<237:45:45,  5.70s/it]
 33%|███▎      | 74714/225000 [147:46:00<202:11:05,  4.84s/it]
 33%|███▎      | 74715/225000 [147:46:03<181:18:43,  4.34s/it]
 33%|███▎      | 74716/225000 [147:46:07<174:59:01,  4.19s/it]
 33%|███▎      | 74717/225000 [147:46:10<162:23:48,  3.89s/it]
 33%|███▎      | 74718/225000 [147:46:14<153:58:33,  3.69s/it]
 33%|███▎      | 74719/225000 [147:46:19<173:23:23,  4.15s/it]
 33%|███▎      | 74720/225000 [147:46:27<224:30:06,  5.38s/it]
 33%|███▎      | 74721/225000 [147:46:30<191:25:08,  4.59s/it]
 33%|███▎      | 74722/225000 [147:46:33<169:11:59,  4.05s/it]
 33%|███▎      | 74723/225000 [147:46:41<219:52:58,  5.27s/it]
 33%|███▎      | 74724/225000 [147:46:43<181:51:57,  4.36s/it]
 33%|███▎      | 74725/225000 [147:46:45<157:40:10,  3.78s/it]
 33%|███▎      | 74726/225000 [147:46:52<192:52:49,  4.62s/it]
 33%|███▎      | 74727/225000 [147:46:55<167:14:59,  4.01s/it]
 33%|███▎      | 74728/225000 [147:46:57<150:44:26,  3.61s/it]
 33%|███▎      | 74729/225000 [147:47:00<134:06:17,  3.21s/it]
 33%|███▎      | 74730/225000 [147:47:07<181:10:09,  4.34s/it]
 33%|███▎      | 74731/225000 [147:47:11<187:58:01,  4.50s/it]
 33%|███▎      | 74732/225000 [147:47:14<159:19:57,  3.82s/it]
 33%|███▎      | 74733/225000 [147:47:16<139:24:16,  3.34s/it]
 33%|███▎      | 74734/225000 [147:47:19<133:30:49,  3.20s/it]
 33%|███▎      | 74735/225000 [147:47:24<157:58:39,  3.78s/it]
 33%|███▎      | 74736/225000 [147:47:28<165:42:22,  3.97s/it]
 33%|███▎      | 74737/225000 [147:47:33<172:51:51,  4.14s/it]
 33%|███▎      | 74738/225000 [147:47:36<156:13:54,  3.74s/it]
 33%|███▎      | 74739/225000 [147:47:38<139:21:47,  3.34s/it]
 33%|███▎      | 74740/225000 [147:47:41<136:23:27,  3.27s/it]
 33%|███▎      | 74741/225000 [147:47:49<196:35:04,  4.71s/it]
 33%|███▎      | 74742/225000 [147:47:55<205:32:04,  4.92s/it]
 33%|███▎      | 74743/225000 [147:47:57<175:34:10,  4.21s/it]
 33%|███▎      | 74744/225000 [147:48:00<158:24:34,  3.80s/it]
 33%|███▎      | 74745/225000 [147:48:05<175:15:09,  4.20s/it]
 33%|███▎      | 74746/225000 [147:48:12<211:19:03,  5.06s/it]
 33%|███▎      | 74747/225000 [147:48:15<179:15:54,  4.30s/it]
 33%|███▎      | 74748/225000 [147:48:18<163:08:59,  3.91s/it]
 33%|███▎      | 74749/225000 [147:48:21<153:32:50,  3.68s/it]
 33%|███▎      | 74750/225000 [147:48:25<153:26:32,  3.68s/it]
                                                              
{'loss': '0.902', 'grad_norm': '2.857', 'learning_rate': '3.963e-05', 'epoch': '0.3322', 'num_input_tokens_seen': 244346208, 'train_runtime': '5.321e+05', 'train_tokens_per_second': '459.2'}

 33%|███▎      | 74750/225000 [147:48:25<153:26:32,  3.68s/it]
 33%|███▎      | 74751/225000 [147:48:28<150:09:49,  3.60s/it]
 33%|███▎      | 74752/225000 [147:48:31<147:06:22,  3.52s/it]
 33%|███▎      | 74753/225000 [147:48:37<176:51:16,  4.24s/it]
 33%|███▎      | 74754/225000 [147:48:41<167:42:05,  4.02s/it]
 33%|███▎      | 74755/225000 [147:48:44<158:50:58,  3.81s/it]
 33%|███▎      | 74756/225000 [147:48:48<158:52:35,  3.81s/it]
 33%|███▎      | 74757/225000 [147:48:52<156:28:14,  3.75s/it]
 33%|███▎      | 74758/225000 [147:48:57<172:28:20,  4.13s/it]
 33%|███▎      | 74759/225000 [147:49:04<217:12:43,  5.20s/it]
 33%|███▎      | 74760/225000 [147:49:07<187:22:53,  4.49s/it]
 33%|███▎      | 74761/225000 [147:49:10<163:04:37,  3.91s/it]
 33%|███▎      | 74762/225000 [147:49:13<158:50:55,  3.81s/it]
 33%|███▎      | 74763/225000 [147:49:16<146:17:30,  3.51s/it]
 33%|███▎      | 74764/225000 [147:49:19<141:11:20,  3.38s/it]
 33%|███▎      | 74765/225000 [147:49:25<172:24:12,  4.13s/it]
 33%|███▎      | 74766/225000 [147:49:34<233:42:48,  5.60s/it]
 33%|███▎      | 74767/225000 [147:49:37<199:01:52,  4.77s/it]
 33%|███▎      | 74768/225000 [147:49:41<191:34:18,  4.59s/it]
 33%|███▎      | 74769/225000 [147:49:44<175:27:36,  4.20s/it]
 33%|███▎      | 74770/225000 [147:49:47<154:47:35,  3.71s/it]
 33%|███▎      | 74771/225000 [147:49:50<147:33:24,  3.54s/it]
 33%|███▎      | 74772/225000 [147:49:53<146:18:17,  3.51s/it]
 33%|███▎      | 74773/225000 [147:49:56<140:08:24,  3.36s/it]
 33%|███▎      | 74774/225000 [147:50:03<179:21:39,  4.30s/it]
 33%|███▎      | 74775/225000 [147:50:09<205:00:18,  4.91s/it]
 33%|███▎      | 74776/225000 [147:50:12<179:45:39,  4.31s/it]
 33%|███▎      | 74777/225000 [147:50:14<154:30:29,  3.70s/it]
 33%|███▎      | 74778/225000 [147:50:17<145:45:24,  3.49s/it]
 33%|███▎      | 74779/225000 [147:50:24<185:35:27,  4.45s/it]
 33%|███▎      | 74780/225000 [147:50:28<174:24:52,  4.18s/it]
 33%|███▎      | 74781/225000 [147:50:30<156:45:05,  3.76s/it]
 33%|███▎      | 74782/225000 [147:50:34<153:40:37,  3.68s/it]
 33%|███▎      | 74783/225000 [147:50:38<153:44:16,  3.68s/it]
 33%|███▎      | 74784/225000 [147:50:41<149:23:52,  3.58s/it]
 33%|███▎      | 74785/225000 [147:50:47<177:18:36,  4.25s/it]
 33%|███▎      | 74786/225000 [147:50:50<169:40:59,  4.07s/it]
 33%|███▎      | 74787/225000 [147:50:53<153:38:47,  3.68s/it]
 33%|███▎      | 74788/225000 [147:50:57<150:14:18,  3.60s/it]
 33%|███▎      | 74789/225000 [147:51:00<143:53:08,  3.45s/it]
 33%|███▎      | 74790/225000 [147:51:03<139:46:37,  3.35s/it]
 33%|███▎      | 74791/225000 [147:51:06<137:54:10,  3.31s/it]
 33%|███▎      | 74792/225000 [147:51:09<132:36:12,  3.18s/it]
 33%|███▎      | 74793/225000 [147:51:12<134:51:35,  3.23s/it]
 33%|███▎      | 74794/225000 [147:51:16<140:10:11,  3.36s/it]
 33%|███▎      | 74795/225000 [147:51:19<133:55:21,  3.21s/it]
 33%|███▎      | 74796/225000 [147:51:27<194:55:25,  4.67s/it]
 33%|███▎      | 74797/225000 [147:51:32<197:54:08,  4.74s/it]
 33%|███▎      | 74798/225000 [147:51:35<175:00:40,  4.19s/it]
 33%|███▎      | 74799/225000 [147:51:37<153:40:12,  3.68s/it]
 33%|███▎      | 74800/225000 [147:51:40<141:36:38,  3.39s/it]
                                                              
{'loss': '0.8661', 'grad_norm': 'nan', 'learning_rate': '3.962e-05', 'epoch': '0.3324', 'num_input_tokens_seen': 244428528, 'train_runtime': '5.323e+05', 'train_tokens_per_second': '459.2'}

 33%|███▎      | 74800/225000 [147:51:40<141:36:38,  3.39s/it]
 33%|███▎      | 74801/225000 [147:51:48<198:55:49,  4.77s/it]
 33%|███▎      | 74802/225000 [147:51:53<198:15:35,  4.75s/it]
 33%|███▎      | 74803/225000 [147:51:56<179:56:03,  4.31s/it]
 33%|███▎      | 74804/225000 [147:52:00<173:24:26,  4.16s/it]
 33%|███▎      | 74805/225000 [147:52:03<162:00:41,  3.88s/it]
 33%|███▎      | 74806/225000 [147:52:06<146:14:56,  3.51s/it]
 33%|███▎      | 74807/225000 [147:52:09<146:07:08,  3.50s/it]
 33%|███▎      | 74808/225000 [147:52:13<151:07:13,  3.62s/it]
 33%|███▎      | 74809/225000 [147:52:22<213:43:37,  5.12s/it]
 33%|███▎      | 74810/225000 [147:52:27<219:49:25,  5.27s/it]
 33%|███▎      | 74811/225000 [147:52:30<182:45:53,  4.38s/it]
 33%|███▎      | 74812/225000 [147:52:33<165:42:14,  3.97s/it]
 33%|███▎      | 74813/225000 [147:52:36<154:49:59,  3.71s/it]
 33%|███▎      | 74814/225000 [147:52:39<148:20:13,  3.56s/it]
 33%|███▎      | 74815/225000 [147:52:42<142:36:08,  3.42s/it]
 33%|███▎      | 74816/225000 [147:52:46<156:45:47,  3.76s/it]
 33%|███▎      | 74817/225000 [147:52:55<211:46:34,  5.08s/it]
 33%|███▎      | 74818/225000 [147:52:57<180:23:22,  4.32s/it]
 33%|███▎      | 74819/225000 [147:53:00<157:08:38,  3.77s/it]
 33%|███▎      | 74820/225000 [147:53:02<144:27:17,  3.46s/it]
 33%|███▎      | 74821/225000 [147:53:05<134:53:20,  3.23s/it]
 33%|███▎      | 74822/225000 [147:53:14<203:21:41,  4.87s/it]
 33%|███▎      | 74823/225000 [147:53:18<199:54:18,  4.79s/it]
 33%|███▎      | 74824/225000 [147:53:21<169:04:52,  4.05s/it]
 33%|███▎      | 74825/225000 [147:53:26<177:58:43,  4.27s/it]
 33%|███▎      | 74826/225000 [147:53:31<190:19:06,  4.56s/it]
 33%|███▎      | 74827/225000 [147:53:33<166:21:01,  3.99s/it]
 33%|███▎      | 74828/225000 [147:53:37<166:24:59,  3.99s/it]
 33%|███▎      | 74829/225000 [147:53:40<153:18:55,  3.68s/it]
 33%|███▎      | 74830/225000 [147:53:44<152:43:49,  3.66s/it]
 33%|███▎      | 74831/225000 [147:53:47<146:24:19,  3.51s/it]
 33%|███▎      | 74832/225000 [147:53:51<149:01:02,  3.57s/it]
 33%|███▎      | 74833/225000 [147:53:54<144:59:35,  3.48s/it]
 33%|███▎      | 74834/225000 [147:53:58<144:52:40,  3.47s/it]
 33%|███▎      | 74835/225000 [147:54:00<136:17:30,  3.27s/it]
 33%|███▎      | 74836/225000 [147:54:04<135:52:07,  3.26s/it]
 33%|███▎      | 74837/225000 [147:54:10<175:47:23,  4.21s/it]
 33%|███▎      | 74838/225000 [147:54:19<233:15:24,  5.59s/it]
 33%|███▎      | 74839/225000 [147:54:22<199:28:05,  4.78s/it]
 33%|███▎      | 74840/225000 [147:54:24<168:52:17,  4.05s/it]
 33%|███▎      | 74841/225000 [147:54:31<205:38:24,  4.93s/it]
 33%|███▎      | 74842/225000 [147:54:35<189:28:41,  4.54s/it]
 33%|███▎      | 74843/225000 [147:54:37<165:12:34,  3.96s/it]
 33%|███▎      | 74844/225000 [147:54:40<148:14:59,  3.55s/it]
 33%|███▎      | 74845/225000 [147:54:43<143:37:17,  3.44s/it]
 33%|███▎      | 74846/225000 [147:54:46<141:22:21,  3.39s/it]
 33%|███▎      | 74847/225000 [147:54:49<136:58:13,  3.28s/it]
 33%|███▎      | 74848/225000 [147:54:53<139:28:40,  3.34s/it]
 33%|███▎      | 74849/225000 [147:54:56<135:48:33,  3.26s/it]
 33%|███▎      | 74850/225000 [147:54:58<126:33:01,  3.03s/it]
                                                              
{'loss': '0.8765', 'grad_norm': '2.097', 'learning_rate': '3.96e-05', 'epoch': '0.3327', 'num_input_tokens_seen': 244508688, 'train_runtime': '5.325e+05', 'train_tokens_per_second': '459.2'}

 33%|███▎      | 74850/225000 [147:54:58<126:33:01,  3.03s/it]
 33%|███▎      | 74851/225000 [147:55:05<167:25:06,  4.01s/it]
 33%|███▎      | 74852/225000 [147:55:13<222:46:35,  5.34s/it]
 33%|███▎      | 74853/225000 [147:55:16<185:04:53,  4.44s/it]
 33%|███▎      | 74854/225000 [147:55:18<159:24:32,  3.82s/it]
 33%|███▎      | 74855/225000 [147:55:21<147:55:35,  3.55s/it]
 33%|███▎      | 74856/225000 [147:55:26<166:04:26,  3.98s/it]
 33%|███▎      | 74857/225000 [147:55:31<181:32:08,  4.35s/it]
 33%|███▎      | 74858/225000 [147:55:34<160:44:51,  3.85s/it]
 33%|███▎      | 74859/225000 [147:55:37<149:40:52,  3.59s/it]
 33%|███▎      | 74860/225000 [147:55:40<143:41:03,  3.45s/it]
 33%|███▎      | 74861/225000 [147:55:43<135:45:37,  3.26s/it]
 33%|███▎      | 74862/225000 [147:55:49<179:51:51,  4.31s/it]
 33%|███▎      | 74863/225000 [147:55:54<184:53:27,  4.43s/it]
 33%|███▎      | 74864/225000 [147:55:58<179:17:36,  4.30s/it]
 33%|███▎      | 74865/225000 [147:56:01<164:17:53,  3.94s/it]
 33%|███▎      | 74866/225000 [147:56:07<182:17:55,  4.37s/it]
 33%|███▎      | 74867/225000 [147:56:11<185:02:11,  4.44s/it]
 33%|███▎      | 74868/225000 [147:56:14<167:05:38,  4.01s/it]
 33%|███▎      | 74869/225000 [147:56:18<159:13:16,  3.82s/it]
 33%|███▎      | 74870/225000 [147:56:20<145:59:03,  3.50s/it]
 33%|███▎      | 74871/225000 [147:56:24<143:47:57,  3.45s/it]
 33%|███▎      | 74872/225000 [147:56:27<141:14:23,  3.39s/it]
 33%|███▎      | 74873/225000 [147:56:30<134:26:42,  3.22s/it]
 33%|███▎      | 74874/225000 [147:56:33<131:54:20,  3.16s/it]
 33%|███▎      | 74875/225000 [147:56:36<137:05:43,  3.29s/it]
 33%|███▎      | 74876/225000 [147:56:40<140:24:07,  3.37s/it]
 33%|███▎      | 74877/225000 [147:56:45<164:48:05,  3.95s/it]
 33%|███▎      | 74878/225000 [147:56:54<220:02:45,  5.28s/it]
 33%|███▎      | 74879/225000 [147:56:59<218:14:19,  5.23s/it]
 33%|███▎      | 74880/225000 [147:57:01<180:18:39,  4.32s/it]
 33%|███▎      | 74881/225000 [147:57:05<174:34:39,  4.19s/it]
 33%|███▎      | 74882/225000 [147:57:11<199:11:08,  4.78s/it]
 33%|███▎      | 74883/225000 [147:57:14<176:55:40,  4.24s/it]
 33%|███▎      | 74884/225000 [147:57:17<160:36:19,  3.85s/it]
 33%|███▎      | 74885/225000 [147:57:20<150:21:54,  3.61s/it]
 33%|███▎      | 74886/225000 [147:57:23<147:42:23,  3.54s/it]
 33%|███▎      | 74887/225000 [147:57:27<147:46:59,  3.54s/it]
 33%|███▎      | 74888/225000 [147:57:30<144:38:01,  3.47s/it]
 33%|███▎      | 74889/225000 [147:57:39<217:19:09,  5.21s/it]
 33%|███▎      | 74890/225000 [147:57:43<198:44:28,  4.77s/it]
 33%|███▎      | 74891/225000 [147:57:46<180:13:55,  4.32s/it]
 33%|███▎      | 74892/225000 [147:57:49<159:39:57,  3.83s/it]
 33%|███▎      | 74893/225000 [147:57:52<153:20:44,  3.68s/it]
 33%|███▎      | 74894/225000 [147:57:56<149:53:01,  3.59s/it]
 33%|███▎      | 74895/225000 [147:57:59<141:34:48,  3.40s/it]
 33%|███▎      | 74896/225000 [147:58:02<135:28:47,  3.25s/it]
 33%|███▎      | 74897/225000 [147:58:05<137:27:23,  3.30s/it]
 33%|███▎      | 74898/225000 [147:58:12<185:19:57,  4.44s/it]
 33%|███▎      | 74899/225000 [147:58:21<236:25:06,  5.67s/it]
 33%|███▎      | 74900/225000 [147:58:23<194:22:43,  4.66s/it]
                                                              
{'loss': '0.9302', 'grad_norm': '4.082', 'learning_rate': '3.959e-05', 'epoch': '0.3329', 'num_input_tokens_seen': 244588520, 'train_runtime': '5.327e+05', 'train_tokens_per_second': '459.1'}

 33%|███▎      | 74900/225000 [147:58:23<194:22:43,  4.66s/it]
 33%|███▎      | 74901/225000 [147:58:26<168:44:17,  4.05s/it]
 33%|███▎      | 74902/225000 [147:58:28<153:52:00,  3.69s/it]
 33%|███▎      | 74903/225000 [147:58:32<157:59:36,  3.79s/it]
 33%|███▎      | 74904/225000 [147:58:36<150:47:29,  3.62s/it]
 33%|███▎      | 74905/225000 [147:58:39<142:03:53,  3.41s/it]
 33%|███▎      | 74906/225000 [147:58:42<137:16:11,  3.29s/it]
 33%|███▎      | 74907/225000 [147:58:45<141:30:47,  3.39s/it]
 33%|███▎      | 74908/225000 [147:58:49<142:44:08,  3.42s/it]
 33%|███▎      | 74909/225000 [147:58:53<147:41:16,  3.54s/it]
 33%|███▎      | 74910/225000 [147:59:00<196:50:26,  4.72s/it]
 33%|███▎      | 74911/225000 [147:59:08<240:16:10,  5.76s/it]
 33%|███▎      | 74912/225000 [147:59:11<200:36:43,  4.81s/it]
 33%|███▎      | 74913/225000 [147:59:13<172:14:07,  4.13s/it]
 33%|███▎      | 74914/225000 [147:59:20<205:39:50,  4.93s/it]
 33%|███▎      | 74915/225000 [147:59:24<194:27:20,  4.66s/it]
 33%|███▎      | 74916/225000 [147:59:28<181:45:58,  4.36s/it]
 33%|███▎      | 74917/225000 [147:59:31<162:09:04,  3.89s/it]
 33%|███▎      | 74918/225000 [147:59:33<141:57:38,  3.41s/it]
 33%|███▎      | 74919/225000 [147:59:39<170:14:06,  4.08s/it]
 33%|███▎      | 74920/225000 [147:59:41<153:12:09,  3.67s/it]
 33%|███▎      | 74921/225000 [147:59:44<143:37:12,  3.45s/it]
 33%|███▎      | 74922/225000 [147:59:47<137:57:55,  3.31s/it]
 33%|███▎      | 74923/225000 [147:59:50<130:30:46,  3.13s/it]
 33%|███▎      | 74924/225000 [147:59:54<140:13:05,  3.36s/it]
 33%|███▎      | 74925/225000 [148:00:00<171:17:23,  4.11s/it]
 33%|███▎      | 74926/225000 [148:00:03<156:02:57,  3.74s/it]
 33%|███▎      | 74927/225000 [148:00:05<141:44:04,  3.40s/it]
 33%|███▎      | 74928/225000 [148:00:08<139:57:50,  3.36s/it]
 33%|███▎      | 74929/225000 [148:00:12<141:27:31,  3.39s/it]
 33%|███▎      | 74930/225000 [148:00:15<137:42:32,  3.30s/it]
 33%|███▎      | 74931/225000 [148:00:20<155:42:25,  3.74s/it]
 33%|███▎      | 74932/225000 [148:00:30<231:11:45,  5.55s/it]
 33%|███▎      | 74933/225000 [148:00:34<215:38:41,  5.17s/it]
 33%|███▎      | 74934/225000 [148:00:36<178:21:48,  4.28s/it]
 33%|███▎      | 74935/225000 [148:00:41<183:44:05,  4.41s/it]
 33%|███▎      | 74936/225000 [148:00:46<198:42:37,  4.77s/it]
 33%|███▎      | 74937/225000 [148:00:52<213:32:03,  5.12s/it]
 33%|███▎      | 74938/225000 [148:00:55<187:23:50,  4.50s/it]
 33%|███▎      | 74939/225000 [148:01:01<196:07:35,  4.71s/it]
 33%|███▎      | 74940/225000 [148:01:05<192:13:49,  4.61s/it]
 33%|███▎      | 74941/225000 [148:01:08<167:07:09,  4.01s/it]
 33%|███▎      | 74942/225000 [148:01:10<150:36:36,  3.61s/it]
 33%|███▎      | 74943/225000 [148:01:13<136:42:21,  3.28s/it]
 33%|███▎      | 74944/225000 [148:01:15<127:34:38,  3.06s/it]
 33%|███▎      | 74945/225000 [148:01:18<127:25:31,  3.06s/it]
 33%|███▎      | 74946/225000 [148:01:21<126:29:39,  3.03s/it]
 33%|███▎      | 74947/225000 [148:01:24<119:19:49,  2.86s/it]
 33%|███▎      | 74948/225000 [148:01:31<171:24:22,  4.11s/it]
 33%|███▎      | 74949/225000 [148:01:35<167:34:08,  4.02s/it]
 33%|███▎      | 74950/225000 [148:01:37<147:30:54,  3.54s/it]
                                                              
{'loss': '0.9349', 'grad_norm': '2.119', 'learning_rate': '3.957e-05', 'epoch': '0.3331', 'num_input_tokens_seen': 244668376, 'train_runtime': '5.329e+05', 'train_tokens_per_second': '459.1'}

 33%|███▎      | 74950/225000 [148:01:37<147:30:54,  3.54s/it]
 33%|███▎      | 74951/225000 [148:01:40<143:10:14,  3.43s/it]
 33%|███▎      | 74952/225000 [148:01:44<142:35:33,  3.42s/it]
 33%|███▎      | 74953/225000 [148:01:47<141:58:16,  3.41s/it]
 33%|███▎      | 74954/225000 [148:01:53<175:21:35,  4.21s/it]
 33%|███▎      | 74955/225000 [148:02:02<235:21:22,  5.65s/it]
 33%|███▎      | 74956/225000 [148:02:05<201:53:40,  4.84s/it]
 33%|███▎      | 74957/225000 [148:02:08<172:35:40,  4.14s/it]
 33%|███▎      | 74958/225000 [148:02:15<212:07:40,  5.09s/it]
 33%|███▎      | 74959/225000 [148:02:18<187:39:18,  4.50s/it]
 33%|███▎      | 74960/225000 [148:02:21<168:01:04,  4.03s/it]
 33%|███▎      | 74961/225000 [148:02:24<151:09:39,  3.63s/it]
 33%|███▎      | 74962/225000 [148:02:26<140:42:04,  3.38s/it]
 33%|███▎      | 74963/225000 [148:02:32<165:34:37,  3.97s/it]
 33%|███▎      | 74964/225000 [148:02:38<198:22:47,  4.76s/it]
 33%|███▎      | 74965/225000 [148:02:41<168:19:53,  4.04s/it]
 33%|███▎      | 74966/225000 [148:02:43<149:30:15,  3.59s/it]
 33%|███▎      | 74967/225000 [148:02:46<136:08:59,  3.27s/it]
 33%|███▎      | 74968/225000 [148:02:55<211:32:42,  5.08s/it]
 33%|███▎      | 74969/225000 [148:02:58<189:36:59,  4.55s/it]
 33%|███▎      | 74970/225000 [148:03:01<172:00:43,  4.13s/it]
 33%|███▎      | 74971/225000 [148:03:06<181:03:39,  4.34s/it]
 33%|███▎      | 74972/225000 [148:03:10<167:33:13,  4.02s/it]
 33%|███▎      | 74973/225000 [148:03:12<148:39:25,  3.57s/it]
 33%|███▎      | 74974/225000 [148:03:20<198:19:58,  4.76s/it]
 33%|███▎      | 74975/225000 [148:03:22<167:18:33,  4.01s/it]
 33%|███▎      | 74976/225000 [148:03:25<150:38:59,  3.62s/it]
 33%|███▎      | 74977/225000 [148:03:27<137:57:15,  3.31s/it]
 33%|███▎      | 74978/225000 [148:03:34<177:38:52,  4.26s/it]
 33%|███▎      | 74979/225000 [148:03:39<195:26:53,  4.69s/it]
 33%|███▎      | 74980/225000 [148:03:42<165:23:57,  3.97s/it]
 33%|███▎      | 74981/225000 [148:03:44<145:24:20,  3.49s/it]
 33%|███▎      | 74982/225000 [148:03:50<172:23:13,  4.14s/it]
 33%|███▎      | 74983/225000 [148:03:54<169:48:39,  4.07s/it]
 33%|███▎      | 74984/225000 [148:03:56<154:28:04,  3.71s/it]
 33%|███▎      | 74985/225000 [148:03:59<138:16:09,  3.32s/it]
 33%|███▎      | 74986/225000 [148:04:01<127:28:57,  3.06s/it]
 33%|███▎      | 74987/225000 [148:04:08<177:06:13,  4.25s/it]
 33%|███▎      | 74988/225000 [148:04:14<193:16:12,  4.64s/it]
 33%|███▎      | 74989/225000 [148:04:17<173:42:18,  4.17s/it]
 33%|███▎      | 74990/225000 [148:04:20<154:53:56,  3.72s/it]
 33%|███▎      | 74991/225000 [148:04:24<161:57:44,  3.89s/it]
 33%|███▎      | 74992/225000 [148:04:28<164:13:39,  3.94s/it]
 33%|███▎      | 74993/225000 [148:04:35<203:14:29,  4.88s/it]
 33%|███▎      | 74994/225000 [148:04:41<211:51:26,  5.08s/it]
 33%|███▎      | 74995/225000 [148:04:43<182:02:33,  4.37s/it]
 33%|███▎      | 74996/225000 [148:04:46<166:42:41,  4.00s/it]
 33%|███▎      | 74997/225000 [148:04:49<150:39:03,  3.62s/it]
 33%|███▎      | 74998/225000 [148:04:52<143:53:36,  3.45s/it]
 33%|███▎      | 74999/225000 [148:04:56<152:58:20,  3.67s/it]
 33%|███▎      | 75000/225000 [148:05:00<147:21:32,  3.54s/it]
                                                              
{'loss': '0.8568', 'grad_norm': '1.826', 'learning_rate': '3.956e-05', 'epoch': '0.3333', 'num_input_tokens_seen': 244751432, 'train_runtime': '5.331e+05', 'train_tokens_per_second': '459.1'}

 33%|███▎      | 75000/225000 [148:05:00<147:21:32,  3.54s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                              
{'eval_loss': '1.284', 'eval_runtime': '459.8', 'eval_samples_per_second': '1.087', 'eval_steps_per_second': '0.544', 'epoch': '0.3333', 'num_input_tokens_seen': 244751432}

 33%|███▎      | 75000/225000 [148:12:40<147:21:32,  3.54s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-75000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 33%|███▎      | 75001/225000 [148:12:44<5906:06:24, 141.75s/it]
 33%|███▎      | 75002/225000 [148:12:46<4165:44:00, 99.98s/it] 
 33%|███▎      | 75003/225000 [148:12:50<2955:55:10, 70.94s/it]
 33%|███▎      | 75004/225000 [148:12:53<2106:38:59, 50.56s/it]
 33%|███▎      | 75005/225000 [148:12:55<1509:32:22, 36.23s/it]
 33%|███▎      | 75006/225000 [148:12:58<1087:36:36, 26.10s/it]
 33%|███▎      | 75007/225000 [148:13:00<791:47:02, 19.00s/it] 
 33%|███▎      | 75008/225000 [148:13:03<586:19:20, 14.07s/it]
 33%|███▎      | 75009/225000 [148:13:05<440:32:50, 10.57s/it]
 33%|███▎      | 75010/225000 [148:13:08<338:23:28,  8.12s/it]
 33%|███▎      | 75011/225000 [148:13:11<277:32:47,  6.66s/it]
 33%|███▎      | 75012/225000 [148:13:19<291:05:44,  6.99s/it]
 33%|███▎      | 75013/225000 [148:13:24<268:39:37,  6.45s/it]
 33%|███▎      | 75014/225000 [148:13:26<218:26:33,  5.24s/it]
 33%|███▎      | 75015/225000 [148:13:34<251:33:12,  6.04s/it]
 33%|███▎      | 75016/225000 [148:13:37<211:14:23,  5.07s/it]
 33%|███▎      | 75017/225000 [148:13:40<182:52:24,  4.39s/it]
 33%|███▎      | 75018/225000 [148:13:43<173:04:37,  4.15s/it]
 33%|███▎      | 75019/225000 [148:13:50<202:41:44,  4.87s/it]
 33%|███▎      | 75020/225000 [148:13:54<193:46:33,  4.65s/it]
 33%|███▎      | 75021/225000 [148:13:58<178:50:52,  4.29s/it]
 33%|███▎      | 75022/225000 [148:14:01<165:27:56,  3.97s/it]
 33%|███▎      | 75023/225000 [148:14:04<154:17:20,  3.70s/it]
 33%|███▎      | 75024/225000 [148:14:07<151:14:46,  3.63s/it]
 33%|███▎      | 75025/225000 [148:14:11<147:12:37,  3.53s/it]
 33%|███▎      | 75026/225000 [148:14:14<147:45:47,  3.55s/it]
 33%|███▎      | 75027/225000 [148:14:20<174:05:24,  4.18s/it]
 33%|███▎      | 75028/225000 [148:14:23<158:05:57,  3.80s/it]
 33%|███▎      | 75029/225000 [148:14:26<146:33:49,  3.52s/it]
 33%|███▎      | 75030/225000 [148:14:30<151:07:54,  3.63s/it]
 33%|███▎      | 75031/225000 [148:14:32<142:00:06,  3.41s/it]
 33%|███▎      | 75032/225000 [148:14:35<136:33:57,  3.28s/it]
 33%|███▎      | 75033/225000 [148:14:39<138:28:22,  3.32s/it]
 33%|███▎      | 75034/225000 [148:14:48<210:01:53,  5.04s/it]
 33%|███▎      | 75035/225000 [148:14:53<214:16:58,  5.14s/it]
 33%|███▎      | 75036/225000 [148:14:56<188:08:45,  4.52s/it]
 33%|███▎      | 75037/225000 [148:15:01<196:23:58,  4.71s/it]
 33%|███▎      | 75038/225000 [148:15:05<180:06:24,  4.32s/it]
 33%|███▎      | 75039/225000 [148:15:10<187:53:45,  4.51s/it]
 33%|███▎      | 75040/225000 [148:15:13<171:31:10,  4.12s/it]
 33%|███▎      | 75041/225000 [148:15:16<156:05:16,  3.75s/it]
 33%|███▎      | 75042/225000 [148:15:19<144:29:17,  3.47s/it]
 33%|███▎      | 75043/225000 [148:15:22<138:50:08,  3.33s/it]
 33%|███▎      | 75044/225000 [148:15:25<141:01:49,  3.39s/it]
 33%|███▎      | 75045/225000 [148:15:28<132:16:54,  3.18s/it]
 33%|███▎      | 75046/225000 [148:15:37<204:19:16,  4.91s/it]
 33%|███▎      | 75047/225000 [148:15:40<184:01:18,  4.42s/it]
 33%|███▎      | 75048/225000 [148:15:43<162:48:12,  3.91s/it]
 33%|███▎      | 75049/225000 [148:15:48<180:19:03,  4.33s/it]
 33%|███▎      | 75050/225000 [148:15:51<164:36:58,  3.95s/it]
                                                              
{'loss': '0.8813', 'grad_norm': '2.322', 'learning_rate': '3.954e-05', 'epoch': '0.3336', 'num_input_tokens_seen': 244832520, 'train_runtime': '5.338e+05', 'train_tokens_per_second': '458.7'}

 33%|███▎      | 75050/225000 [148:15:51<164:36:58,  3.95s/it]
 33%|███▎      | 75051/225000 [148:15:54<149:09:58,  3.58s/it]
 33%|███▎      | 75052/225000 [148:15:57<139:07:23,  3.34s/it]
 33%|███▎      | 75053/225000 [148:15:59<128:24:36,  3.08s/it]
 33%|███▎      | 75054/225000 [148:16:03<139:05:01,  3.34s/it]
 33%|███▎      | 75055/225000 [148:16:14<228:22:12,  5.48s/it]
 33%|███▎      | 75056/225000 [148:16:16<193:53:04,  4.65s/it]
 33%|███▎      | 75057/225000 [148:16:19<166:24:05,  4.00s/it]
 33%|███▎      | 75058/225000 [148:16:22<149:31:15,  3.59s/it]
 33%|███▎      | 75059/225000 [148:16:30<206:25:23,  4.96s/it]
 33%|███▎      | 75060/225000 [148:16:34<201:26:33,  4.84s/it]
 33%|███▎      | 75061/225000 [148:16:37<172:07:26,  4.13s/it]
 33%|███▎      | 75062/225000 [148:16:40<155:39:54,  3.74s/it]
 33%|███▎      | 75063/225000 [148:16:42<141:08:31,  3.39s/it]
 33%|███▎      | 75064/225000 [148:16:45<131:43:59,  3.16s/it]
 33%|███▎      | 75065/225000 [148:16:48<137:10:39,  3.29s/it]
 33%|███▎      | 75066/225000 [148:16:52<137:59:46,  3.31s/it]
 33%|███▎      | 75067/225000 [148:16:55<134:59:40,  3.24s/it]
 33%|███▎      | 75068/225000 [148:16:58<129:19:23,  3.11s/it]
 33%|███▎      | 75069/225000 [148:17:01<131:30:09,  3.16s/it]
 33%|███▎      | 75070/225000 [148:17:04<131:36:38,  3.16s/it]
 33%|███▎      | 75071/225000 [148:17:14<216:04:50,  5.19s/it]
 33%|███▎      | 75072/225000 [148:17:19<217:28:47,  5.22s/it]
 33%|███▎      | 75073/225000 [148:17:22<182:40:16,  4.39s/it]
 33%|███▎      | 75074/225000 [148:17:24<161:12:13,  3.87s/it]
 33%|███▎      | 75075/225000 [148:17:30<182:32:11,  4.38s/it]
 33%|███▎      | 75076/225000 [148:17:36<208:30:12,  5.01s/it]
 33%|███▎      | 75077/225000 [148:17:39<182:23:54,  4.38s/it]
 33%|███▎      | 75078/225000 [148:17:43<169:04:23,  4.06s/it]
 33%|███▎      | 75079/225000 [148:17:46<163:00:06,  3.91s/it]
 33%|███▎      | 75080/225000 [148:17:49<155:34:09,  3.74s/it]
 33%|███▎      | 75081/225000 [148:17:53<150:08:37,  3.61s/it]
 33%|███▎      | 75082/225000 [148:17:57<151:58:33,  3.65s/it]
 33%|███▎      | 75083/225000 [148:18:02<176:04:02,  4.23s/it]
 33%|███▎      | 75084/225000 [148:18:05<163:59:29,  3.94s/it]
 33%|███▎      | 75085/225000 [148:18:08<150:31:12,  3.61s/it]
 33%|███▎      | 75086/225000 [148:18:11<142:04:05,  3.41s/it]
 33%|███▎      | 75087/225000 [148:18:16<161:20:47,  3.87s/it]
 33%|███▎      | 75088/225000 [148:18:20<158:05:03,  3.80s/it]
 33%|███▎      | 75089/225000 [148:18:23<153:01:11,  3.67s/it]
 33%|███▎      | 75090/225000 [148:18:26<148:23:22,  3.56s/it]
 33%|███▎      | 75091/225000 [148:18:30<145:58:42,  3.51s/it]
 33%|███▎      | 75092/225000 [148:18:33<140:36:55,  3.38s/it]
 33%|███▎      | 75093/225000 [148:18:37<150:48:04,  3.62s/it]
 33%|███▎      | 75094/225000 [148:18:40<147:57:15,  3.55s/it]
 33%|███▎      | 75095/225000 [148:18:48<197:16:13,  4.74s/it]
 33%|███▎      | 75096/225000 [148:18:55<230:18:00,  5.53s/it]
 33%|███▎      | 75097/225000 [148:18:58<198:39:11,  4.77s/it]
 33%|███▎      | 75098/225000 [148:19:01<176:58:43,  4.25s/it]
 33%|███▎      | 75099/225000 [148:19:05<162:53:45,  3.91s/it]
 33%|███▎      | 75100/225000 [148:19:08<155:12:57,  3.73s/it]
                                                              
{'loss': '0.8711', 'grad_norm': '1.774', 'learning_rate': '3.953e-05', 'epoch': '0.3338', 'num_input_tokens_seen': 244913824, 'train_runtime': '5.339e+05', 'train_tokens_per_second': '458.7'}

 33%|███▎      | 75100/225000 [148:19:08<155:12:57,  3.73s/it]
 33%|███▎      | 75101/225000 [148:19:11<149:00:50,  3.58s/it]
 33%|███▎      | 75102/225000 [148:19:14<146:17:54,  3.51s/it]
 33%|███▎      | 75103/225000 [148:19:18<142:47:04,  3.43s/it]
 33%|███▎      | 75104/225000 [148:19:21<138:50:17,  3.33s/it]
 33%|███▎      | 75105/225000 [148:19:24<132:30:36,  3.18s/it]
 33%|███▎      | 75106/225000 [148:19:29<166:30:08,  4.00s/it]
 33%|███▎      | 75107/225000 [148:19:38<222:37:36,  5.35s/it]
 33%|███▎      | 75108/225000 [148:19:40<187:06:36,  4.49s/it]
 33%|███▎      | 75109/225000 [148:19:45<188:26:49,  4.53s/it]
 33%|███▎      | 75110/225000 [148:19:49<178:40:35,  4.29s/it]
 33%|███▎      | 75111/225000 [148:19:52<160:33:04,  3.86s/it]
 33%|███▎      | 75112/225000 [148:19:54<147:49:01,  3.55s/it]
 33%|███▎      | 75113/225000 [148:19:57<138:28:55,  3.33s/it]
 33%|███▎      | 75114/225000 [148:20:00<132:24:35,  3.18s/it]
 33%|███▎      | 75115/225000 [148:20:09<200:11:08,  4.81s/it]
 33%|███▎      | 75116/225000 [148:20:13<193:49:05,  4.66s/it]
 33%|███▎      | 75117/225000 [148:20:16<167:37:58,  4.03s/it]
 33%|███▎      | 75118/225000 [148:20:18<152:09:37,  3.65s/it]
 33%|███▎      | 75119/225000 [148:20:21<144:14:10,  3.46s/it]
 33%|███▎      | 75120/225000 [148:20:25<146:55:46,  3.53s/it]
 33%|███▎      | 75121/225000 [148:20:28<142:47:11,  3.43s/it]
 33%|███▎      | 75122/225000 [148:20:37<212:38:00,  5.11s/it]
 33%|███▎      | 75123/225000 [148:20:41<193:32:17,  4.65s/it]
 33%|███▎      | 75124/225000 [148:20:46<197:26:13,  4.74s/it]
 33%|███▎      | 75125/225000 [148:20:49<171:43:10,  4.12s/it]
 33%|███▎      | 75126/225000 [148:20:51<151:41:24,  3.64s/it]
 33%|███▎      | 75127/225000 [148:20:54<141:44:30,  3.40s/it]
 33%|███▎      | 75128/225000 [148:20:57<140:22:52,  3.37s/it]
 33%|███▎      | 75129/225000 [148:21:00<129:01:48,  3.10s/it]
 33%|███▎      | 75130/225000 [148:21:07<183:55:41,  4.42s/it]
 33%|███▎      | 75131/225000 [148:21:14<215:28:36,  5.18s/it]
 33%|███▎      | 75132/225000 [148:21:17<184:23:02,  4.43s/it]
 33%|███▎      | 75133/225000 [148:21:19<161:55:56,  3.89s/it]
 33%|███▎      | 75134/225000 [148:21:23<156:58:38,  3.77s/it]
 33%|███▎      | 75135/225000 [148:21:25<141:18:42,  3.39s/it]
 33%|███▎      | 75136/225000 [148:21:29<137:30:17,  3.30s/it]
 33%|███▎      | 75137/225000 [148:21:38<211:22:31,  5.08s/it]
 33%|███▎      | 75138/225000 [148:21:42<199:14:43,  4.79s/it]
 33%|███▎      | 75139/225000 [148:21:45<172:43:32,  4.15s/it]
 33%|███▎      | 75140/225000 [148:21:48<160:26:51,  3.85s/it]
 33%|███▎      | 75141/225000 [148:21:51<148:47:04,  3.57s/it]
 33%|███▎      | 75142/225000 [148:21:53<138:47:51,  3.33s/it]
 33%|███▎      | 75143/225000 [148:21:59<171:43:38,  4.13s/it]
 33%|███▎      | 75144/225000 [148:22:05<190:15:27,  4.57s/it]
 33%|███▎      | 75145/225000 [148:22:08<171:29:52,  4.12s/it]
 33%|███▎      | 75146/225000 [148:22:11<157:46:23,  3.79s/it]
 33%|███▎      | 75147/225000 [148:22:14<145:27:46,  3.49s/it]
 33%|███▎      | 75148/225000 [148:22:17<139:51:29,  3.36s/it]
 33%|███▎      | 75149/225000 [148:22:20<132:31:40,  3.18s/it]
 33%|███▎      | 75150/225000 [148:22:22<126:08:25,  3.03s/it]
                                                              
{'loss': '0.8887', 'grad_norm': '4.2', 'learning_rate': '3.951e-05', 'epoch': '0.334', 'num_input_tokens_seen': 244993992, 'train_runtime': '5.341e+05', 'train_tokens_per_second': '458.7'}

 33%|███▎      | 75150/225000 [148:22:22<126:08:25,  3.03s/it]
 33%|███▎      | 75151/225000 [148:22:25<125:21:31,  3.01s/it]
 33%|███▎      | 75152/225000 [148:22:30<150:51:43,  3.62s/it]
 33%|███▎      | 75153/225000 [148:22:39<216:16:59,  5.20s/it]
 33%|███▎      | 75154/225000 [148:22:42<189:58:24,  4.56s/it]
 33%|███▎      | 75155/225000 [148:22:45<162:20:53,  3.90s/it]
 33%|███▎      | 75156/225000 [148:22:47<147:55:09,  3.55s/it]
 33%|███▎      | 75157/225000 [148:22:50<141:00:00,  3.39s/it]
 33%|███▎      | 75158/225000 [148:22:56<173:10:27,  4.16s/it]
 33%|███▎      | 75159/225000 [148:23:02<195:37:51,  4.70s/it]
 33%|███▎      | 75160/225000 [148:23:05<170:36:51,  4.10s/it]
 33%|███▎      | 75161/225000 [148:23:08<152:06:24,  3.65s/it]
 33%|███▎      | 75162/225000 [148:23:14<185:57:22,  4.47s/it]
 33%|███▎      | 75163/225000 [148:23:18<175:30:48,  4.22s/it]
 33%|███▎      | 75164/225000 [148:23:21<162:05:32,  3.89s/it]
 33%|███▎      | 75165/225000 [148:23:24<147:58:46,  3.56s/it]
 33%|███▎      | 75166/225000 [148:23:29<176:47:21,  4.25s/it]
 33%|███▎      | 75167/225000 [148:23:32<161:59:28,  3.89s/it]
 33%|███▎      | 75168/225000 [148:23:36<153:29:11,  3.69s/it]
 33%|███▎      | 75169/225000 [148:23:39<145:52:12,  3.50s/it]
 33%|███▎      | 75170/225000 [148:23:43<152:24:39,  3.66s/it]
 33%|███▎      | 75171/225000 [148:23:46<147:27:51,  3.54s/it]
 33%|███▎      | 75172/225000 [148:23:54<197:46:35,  4.75s/it]
 33%|███▎      | 75173/225000 [148:24:00<220:49:42,  5.31s/it]
 33%|███▎      | 75174/225000 [148:24:03<184:39:43,  4.44s/it]
 33%|███▎      | 75175/225000 [148:24:05<161:40:47,  3.88s/it]
 33%|███▎      | 75176/225000 [148:24:08<149:23:12,  3.59s/it]
 33%|███▎      | 75177/225000 [148:24:16<197:11:34,  4.74s/it]
 33%|███▎      | 75178/225000 [148:24:18<172:21:18,  4.14s/it]
 33%|███▎      | 75179/225000 [148:24:21<158:51:53,  3.82s/it]
 33%|███▎      | 75180/225000 [148:24:24<144:44:05,  3.48s/it]
 33%|███▎      | 75181/225000 [148:24:27<135:44:16,  3.26s/it]
 33%|███▎      | 75182/225000 [148:24:29<127:03:49,  3.05s/it]
 33%|███▎      | 75183/225000 [148:24:32<125:47:51,  3.02s/it]
 33%|███▎      | 75184/225000 [148:24:35<120:30:37,  2.90s/it]
 33%|███▎      | 75185/225000 [148:24:38<125:54:12,  3.03s/it]
 33%|███▎      | 75186/225000 [148:24:42<136:23:56,  3.28s/it]
 33%|███▎      | 75187/225000 [148:24:51<210:58:55,  5.07s/it]
 33%|███▎      | 75188/225000 [148:24:55<198:51:37,  4.78s/it]
 33%|███▎      | 75189/225000 [148:24:58<169:54:57,  4.08s/it]
 33%|███▎      | 75190/225000 [148:25:01<151:17:02,  3.64s/it]
 33%|███▎      | 75191/225000 [148:25:04<144:04:29,  3.46s/it]
 33%|███▎      | 75192/225000 [148:25:06<134:58:11,  3.24s/it]
 33%|███▎      | 75193/225000 [148:25:14<194:24:45,  4.67s/it]
 33%|███▎      | 75194/225000 [148:25:20<207:45:33,  4.99s/it]
 33%|███▎      | 75195/225000 [148:25:23<176:18:12,  4.24s/it]
 33%|███▎      | 75196/225000 [148:25:28<187:09:32,  4.50s/it]
 33%|███▎      | 75197/225000 [148:25:30<159:22:48,  3.83s/it]
 33%|███▎      | 75198/225000 [148:25:33<146:36:22,  3.52s/it]
 33%|███▎      | 75199/225000 [148:25:36<142:28:52,  3.42s/it]
 33%|███▎      | 75200/225000 [148:25:40<146:13:47,  3.51s/it]
                                                              
{'loss': '0.8579', 'grad_norm': '3.204', 'learning_rate': '3.95e-05', 'epoch': '0.3342', 'num_input_tokens_seen': 245074912, 'train_runtime': '5.343e+05', 'train_tokens_per_second': '458.6'}

 33%|███▎      | 75200/225000 [148:25:40<146:13:47,  3.51s/it]
 33%|███▎      | 75201/225000 [148:25:43<139:49:41,  3.36s/it]
 33%|███▎      | 75202/225000 [148:25:46<139:37:56,  3.36s/it]
 33%|███▎      | 75203/225000 [148:25:49<134:21:32,  3.23s/it]
 33%|███▎      | 75204/225000 [148:25:52<130:04:24,  3.13s/it]
 33%|███▎      | 75205/225000 [148:25:59<182:36:10,  4.39s/it]
 33%|███▎      | 75206/225000 [148:26:03<175:25:42,  4.22s/it]
 33%|███▎      | 75207/225000 [148:26:06<158:06:12,  3.80s/it]
 33%|███▎      | 75208/225000 [148:26:09<146:27:36,  3.52s/it]
 33%|███▎      | 75209/225000 [148:26:12<147:30:44,  3.55s/it]
 33%|███▎      | 75210/225000 [148:26:15<139:18:18,  3.35s/it]
 33%|███▎      | 75211/225000 [148:26:18<133:59:33,  3.22s/it]
 33%|███▎      | 75212/225000 [148:26:21<135:36:46,  3.26s/it]
 33%|███▎      | 75213/225000 [148:26:24<132:57:57,  3.20s/it]
 33%|███▎      | 75214/225000 [148:26:28<139:10:04,  3.34s/it]
 33%|███▎      | 75215/225000 [148:26:36<198:52:32,  4.78s/it]
 33%|███▎      | 75216/225000 [148:26:42<215:27:17,  5.18s/it]
 33%|███▎      | 75217/225000 [148:26:45<188:49:31,  4.54s/it]
 33%|███▎      | 75218/225000 [148:26:50<192:17:23,  4.62s/it]
 33%|███▎      | 75219/225000 [148:26:56<202:10:58,  4.86s/it]
 33%|███▎      | 75220/225000 [148:26:59<180:01:03,  4.33s/it]
 33%|███▎      | 75221/225000 [148:27:03<179:32:32,  4.32s/it]
 33%|███▎      | 75222/225000 [148:27:06<167:38:18,  4.03s/it]
 33%|███▎      | 75223/225000 [148:27:10<165:23:46,  3.98s/it]
 33%|███▎      | 75224/225000 [148:27:13<151:28:39,  3.64s/it]
 33%|███▎      | 75225/225000 [148:27:18<162:20:02,  3.90s/it]
 33%|███▎      | 75226/225000 [148:27:25<210:25:37,  5.06s/it]
 33%|███▎      | 75227/225000 [148:27:29<191:49:11,  4.61s/it]
 33%|███▎      | 75228/225000 [148:27:32<170:15:25,  4.09s/it]
 33%|███▎      | 75229/225000 [148:27:35<157:20:28,  3.78s/it]
 33%|███▎      | 75230/225000 [148:27:38<143:23:24,  3.45s/it]
 33%|███▎      | 75231/225000 [148:27:40<134:57:50,  3.24s/it]
 33%|███▎      | 75232/225000 [148:27:43<133:35:13,  3.21s/it]
 33%|███▎      | 75233/225000 [148:27:48<149:12:29,  3.59s/it]
 33%|███▎      | 75234/225000 [148:27:56<208:45:55,  5.02s/it]
 33%|███▎      | 75235/225000 [148:28:00<190:49:09,  4.59s/it]
 33%|███▎      | 75236/225000 [148:28:03<169:29:27,  4.07s/it]
 33%|███▎      | 75237/225000 [148:28:06<159:49:14,  3.84s/it]
 33%|███▎      | 75238/225000 [148:28:09<152:34:02,  3.67s/it]
 33%|███▎      | 75239/225000 [148:28:13<150:25:39,  3.62s/it]
 33%|███▎      | 75240/225000 [148:28:16<144:19:19,  3.47s/it]
 33%|███▎      | 75241/225000 [148:28:19<144:21:17,  3.47s/it]
 33%|███▎      | 75242/225000 [148:28:23<140:43:32,  3.38s/it]
 33%|███▎      | 75243/225000 [148:28:30<192:05:52,  4.62s/it]
 33%|███▎      | 75244/225000 [148:28:35<200:12:23,  4.81s/it]
 33%|███▎      | 75245/225000 [148:28:39<182:16:21,  4.38s/it]
 33%|███▎      | 75246/225000 [148:28:44<195:02:41,  4.69s/it]
 33%|███▎      | 75247/225000 [148:28:47<178:14:32,  4.28s/it]
 33%|███▎      | 75248/225000 [148:28:50<160:23:50,  3.86s/it]
 33%|███▎      | 75249/225000 [148:28:54<155:21:32,  3.73s/it]
 33%|███▎      | 75250/225000 [148:28:56<142:48:40,  3.43s/it]
                                                              
{'loss': '0.8524', 'grad_norm': '3.151', 'learning_rate': '3.948e-05', 'epoch': '0.3344', 'num_input_tokens_seen': 245154360, 'train_runtime': '5.345e+05', 'train_tokens_per_second': '458.6'}

 33%|███▎      | 75250/225000 [148:28:56<142:48:40,  3.43s/it]
 33%|███▎      | 75251/225000 [148:28:59<129:42:53,  3.12s/it]
 33%|███▎      | 75252/225000 [148:29:04<157:49:21,  3.79s/it]
 33%|███▎      | 75253/225000 [148:29:12<210:45:35,  5.07s/it]
 33%|███▎      | 75254/225000 [148:29:15<186:34:21,  4.49s/it]
 33%|███▎      | 75255/225000 [148:29:18<167:16:40,  4.02s/it]
 33%|███▎      | 75256/225000 [148:29:22<161:56:41,  3.89s/it]
 33%|███▎      | 75257/225000 [148:29:25<152:15:44,  3.66s/it]
 33%|███▎      | 75258/225000 [148:29:28<149:04:27,  3.58s/it]
 33%|███▎      | 75259/225000 [148:29:32<143:22:49,  3.45s/it]
 33%|███▎      | 75260/225000 [148:29:34<136:16:26,  3.28s/it]
 33%|███▎      | 75261/225000 [148:29:40<165:11:09,  3.97s/it]
 33%|███▎      | 75262/225000 [148:29:49<225:46:16,  5.43s/it]
 33%|███▎      | 75263/225000 [148:29:54<218:41:16,  5.26s/it]
 33%|███▎      | 75264/225000 [148:29:57<189:19:31,  4.55s/it]
 33%|███▎      | 75265/225000 [148:30:00<168:40:02,  4.06s/it]
 33%|███▎      | 75266/225000 [148:30:02<150:02:52,  3.61s/it]
 33%|███▎      | 75267/225000 [148:30:05<144:39:12,  3.48s/it]
 33%|███▎      | 75268/225000 [148:30:11<167:29:07,  4.03s/it]
 33%|███▎      | 75269/225000 [148:30:20<234:58:42,  5.65s/it]
 33%|███▎      | 75270/225000 [148:30:23<195:21:21,  4.70s/it]
 33%|███▎      | 75271/225000 [148:30:25<163:50:01,  3.94s/it]
 33%|███▎      | 75272/225000 [148:30:31<197:14:35,  4.74s/it]
 33%|███▎      | 75273/225000 [148:30:34<171:45:55,  4.13s/it]
 33%|███▎      | 75274/225000 [148:30:39<185:16:08,  4.45s/it]
 33%|███▎      | 75275/225000 [148:30:43<171:44:07,  4.13s/it]
 33%|███▎      | 75276/225000 [148:30:46<158:14:50,  3.80s/it]
 33%|███▎      | 75277/225000 [148:30:48<141:51:36,  3.41s/it]
 33%|███▎      | 75278/225000 [148:30:51<130:41:03,  3.14s/it]
 33%|███▎      | 75279/225000 [148:30:53<121:54:28,  2.93s/it]
 33%|███▎      | 75280/225000 [148:30:56<118:57:48,  2.86s/it]
 33%|███▎      | 75281/225000 [148:30:58<113:45:54,  2.74s/it]
 33%|███▎      | 75282/225000 [148:31:06<175:19:26,  4.22s/it]
 33%|███▎      | 75283/225000 [148:31:13<217:20:43,  5.23s/it]
 33%|███▎      | 75284/225000 [148:31:16<189:59:39,  4.57s/it]
 33%|███▎      | 75285/225000 [148:31:19<166:12:03,  4.00s/it]
 33%|███▎      | 75286/225000 [148:31:23<164:08:51,  3.95s/it]
 33%|███▎      | 75287/225000 [148:31:27<160:31:42,  3.86s/it]
 33%|███▎      | 75288/225000 [148:31:31<162:38:53,  3.91s/it]
 33%|███▎      | 75289/225000 [148:31:34<155:55:43,  3.75s/it]
 33%|███▎      | 75290/225000 [148:31:37<142:57:33,  3.44s/it]
 33%|███▎      | 75291/225000 [148:31:40<136:33:27,  3.28s/it]
 33%|███▎      | 75292/225000 [148:31:43<139:44:01,  3.36s/it]
 33%|███▎      | 75293/225000 [148:31:47<150:06:18,  3.61s/it]
 33%|███▎      | 75294/225000 [148:31:50<138:40:18,  3.33s/it]
 33%|███▎      | 75295/225000 [148:31:57<187:12:10,  4.50s/it]
 33%|███▎      | 75296/225000 [148:32:05<232:41:58,  5.60s/it]
 33%|███▎      | 75297/225000 [148:32:09<202:36:04,  4.87s/it]
 33%|███▎      | 75298/225000 [148:32:11<175:20:42,  4.22s/it]
 33%|███▎      | 75299/225000 [148:32:14<152:06:16,  3.66s/it]
 33%|███▎      | 75300/225000 [148:32:16<135:15:52,  3.25s/it]
                                                              
{'loss': '0.8956', 'grad_norm': '2.238', 'learning_rate': '3.947e-05', 'epoch': '0.3347', 'num_input_tokens_seen': 245235504, 'train_runtime': '5.347e+05', 'train_tokens_per_second': '458.6'}

 33%|███▎      | 75300/225000 [148:32:16<135:15:52,  3.25s/it]
 33%|███▎      | 75301/225000 [148:32:24<198:52:26,  4.78s/it]
 33%|███▎      | 75302/225000 [148:32:27<172:12:09,  4.14s/it]
 33%|███▎      | 75303/225000 [148:32:30<152:05:11,  3.66s/it]
 33%|███▎      | 75304/225000 [148:32:32<138:58:55,  3.34s/it]
 33%|███▎      | 75305/225000 [148:32:41<202:03:06,  4.86s/it]
 33%|███▎      | 75306/225000 [148:32:45<199:13:20,  4.79s/it]
 33%|███▎      | 75307/225000 [148:32:47<167:25:37,  4.03s/it]
 33%|███▎      | 75308/225000 [148:32:50<152:51:04,  3.68s/it]
 33%|███▎      | 75309/225000 [148:32:53<134:49:37,  3.24s/it]
 33%|███▎      | 75310/225000 [148:33:01<195:28:59,  4.70s/it]
 33%|███▎      | 75311/225000 [148:33:04<181:00:21,  4.35s/it]
 33%|███▎      | 75312/225000 [148:33:07<161:18:42,  3.88s/it]
 33%|███▎      | 75313/225000 [148:33:10<147:47:38,  3.55s/it]
 33%|███▎      | 75314/225000 [148:33:13<140:47:38,  3.39s/it]
 33%|███▎      | 75315/225000 [148:33:15<130:45:28,  3.14s/it]
 33%|███▎      | 75316/225000 [148:33:23<183:41:05,  4.42s/it]
 33%|███▎      | 75317/225000 [148:33:28<196:02:35,  4.72s/it]
 33%|███▎      | 75318/225000 [148:33:31<171:21:04,  4.12s/it]
 33%|███▎      | 75319/225000 [148:33:34<164:44:22,  3.96s/it]
 33%|███▎      | 75320/225000 [148:33:42<210:20:41,  5.06s/it]
 33%|███▎      | 75321/225000 [148:33:45<187:06:13,  4.50s/it]
 33%|███▎      | 75322/225000 [148:33:49<173:14:54,  4.17s/it]
 33%|███▎      | 75323/225000 [148:33:52<161:28:30,  3.88s/it]
 33%|███▎      | 75324/225000 [148:33:55<153:28:40,  3.69s/it]
 33%|███▎      | 75325/225000 [148:33:58<145:44:30,  3.51s/it]
 33%|███▎      | 75326/225000 [148:34:01<138:41:52,  3.34s/it]
 33%|███▎      | 75327/225000 [148:34:05<141:59:08,  3.42s/it]
 33%|███▎      | 75328/225000 [148:34:08<145:25:17,  3.50s/it]
 33%|███▎      | 75329/225000 [148:34:12<144:06:48,  3.47s/it]
 33%|███▎      | 75330/225000 [148:34:17<170:40:21,  4.11s/it]
 33%|███▎      | 75331/225000 [148:34:22<172:58:59,  4.16s/it]
 33%|███▎      | 75332/225000 [148:34:25<166:57:24,  4.02s/it]
 33%|███▎      | 75333/225000 [148:34:28<155:44:18,  3.75s/it]
 33%|███▎      | 75334/225000 [148:34:31<144:06:50,  3.47s/it]
 33%|███▎      | 75335/225000 [148:34:35<144:08:34,  3.47s/it]
 33%|███▎      | 75336/225000 [148:34:38<139:36:11,  3.36s/it]
 33%|███▎      | 75337/225000 [148:34:41<138:12:32,  3.32s/it]
 33%|███▎      | 75338/225000 [148:34:47<170:06:47,  4.09s/it]
 33%|███▎      | 75339/225000 [148:34:56<237:12:46,  5.71s/it]
 33%|███▎      | 75340/225000 [148:35:00<211:11:10,  5.08s/it]
 33%|███▎      | 75341/225000 [148:35:03<188:59:05,  4.55s/it]
 33%|███▎      | 75342/225000 [148:35:07<171:22:17,  4.12s/it]
 33%|███▎      | 75343/225000 [148:35:10<157:23:17,  3.79s/it]
 33%|███▎      | 75344/225000 [148:35:12<146:18:48,  3.52s/it]
 33%|███▎      | 75345/225000 [148:35:15<138:14:08,  3.33s/it]
 33%|███▎      | 75346/225000 [148:35:19<144:15:30,  3.47s/it]
 33%|███▎      | 75347/225000 [148:35:23<151:22:34,  3.64s/it]
 33%|███▎      | 75348/225000 [148:35:34<247:20:16,  5.95s/it]
 33%|███▎      | 75349/225000 [148:35:38<215:32:06,  5.18s/it]
 33%|███▎      | 75350/225000 [148:35:40<183:32:27,  4.42s/it]
                                                              
{'loss': '0.8389', 'grad_norm': '3.656', 'learning_rate': '3.945e-05', 'epoch': '0.3349', 'num_input_tokens_seen': 245317080, 'train_runtime': '5.349e+05', 'train_tokens_per_second': '458.6'}

 33%|███▎      | 75350/225000 [148:35:40<183:32:27,  4.42s/it]
 33%|███▎      | 75351/225000 [148:35:43<164:55:32,  3.97s/it]
 33%|███▎      | 75352/225000 [148:35:46<151:53:15,  3.65s/it]
 33%|███▎      | 75353/225000 [148:35:52<175:10:27,  4.21s/it]
 33%|███▎      | 75354/225000 [148:35:59<209:55:43,  5.05s/it]
 33%|███▎      | 75355/225000 [148:36:01<179:01:43,  4.31s/it]
 33%|███▎      | 75356/225000 [148:36:04<155:56:32,  3.75s/it]
 33%|███▎      | 75357/225000 [148:36:06<139:59:51,  3.37s/it]
 33%|███▎      | 75358/225000 [148:36:13<181:05:29,  4.36s/it]
 33%|███▎      | 75359/225000 [148:36:16<164:10:14,  3.95s/it]
 33%|███▎      | 75360/225000 [148:36:19<155:11:36,  3.73s/it]
 33%|███▎      | 75361/225000 [148:36:22<142:12:53,  3.42s/it]
 33%|███▎      | 75362/225000 [148:36:25<133:46:55,  3.22s/it]
 33%|███▎      | 75363/225000 [148:36:33<192:23:06,  4.63s/it]
 33%|███▎      | 75364/225000 [148:36:38<201:15:18,  4.84s/it]
 33%|███▎      | 75365/225000 [148:36:41<180:02:43,  4.33s/it]
 33%|███▎      | 75366/225000 [148:36:45<169:44:12,  4.08s/it]
 33%|███▎      | 75367/225000 [148:36:47<154:39:38,  3.72s/it]
 33%|███▎      | 75368/225000 [148:36:50<144:30:04,  3.48s/it]
 33%|███▎      | 75369/225000 [148:36:54<144:40:11,  3.48s/it]
 33%|███▎      | 75370/225000 [148:36:57<140:24:08,  3.38s/it]
 33%|███▎      | 75371/225000 [148:37:04<186:48:27,  4.49s/it]
 33%|███▎      | 75372/225000 [148:37:08<175:11:45,  4.22s/it]
 33%|███▎      | 75373/225000 [148:37:11<165:29:04,  3.98s/it]
 33%|███▎      | 75374/225000 [148:37:15<161:03:28,  3.88s/it]
 34%|███▎      | 75375/225000 [148:37:18<156:14:41,  3.76s/it]
 34%|███▎      | 75376/225000 [148:37:21<149:15:10,  3.59s/it]
 34%|███▎      | 75377/225000 [148:37:25<146:46:47,  3.53s/it]
 34%|███▎      | 75378/225000 [148:37:34<217:02:01,  5.22s/it]
 34%|███▎      | 75379/225000 [148:37:38<207:24:43,  4.99s/it]
 34%|███▎      | 75380/225000 [148:37:41<174:55:19,  4.21s/it]
 34%|███▎      | 75381/225000 [148:37:43<151:00:04,  3.63s/it]
 34%|███▎      | 75382/225000 [148:37:47<159:14:01,  3.83s/it]
 34%|███▎      | 75383/225000 [148:37:53<181:16:25,  4.36s/it]
 34%|███▎      | 75384/225000 [148:37:56<158:25:41,  3.81s/it]
 34%|███▎      | 75385/225000 [148:38:00<162:44:06,  3.92s/it]
 34%|███▎      | 75386/225000 [148:38:05<181:03:08,  4.36s/it]
 34%|███▎      | 75387/225000 [148:38:08<159:21:14,  3.83s/it]
 34%|███▎      | 75388/225000 [148:38:10<141:02:59,  3.39s/it]
 34%|███▎      | 75389/225000 [148:38:13<130:31:41,  3.14s/it]
 34%|███▎      | 75390/225000 [148:38:22<202:53:26,  4.88s/it]
 34%|███▎      | 75391/225000 [148:38:24<171:29:03,  4.13s/it]
 34%|███▎      | 75392/225000 [148:38:27<152:38:49,  3.67s/it]
 34%|███▎      | 75393/225000 [148:38:29<137:16:50,  3.30s/it]
 34%|███▎      | 75394/225000 [148:38:31<126:30:33,  3.04s/it]
 34%|███▎      | 75395/225000 [148:38:37<154:28:24,  3.72s/it]
 34%|███▎      | 75396/225000 [148:38:43<185:01:39,  4.45s/it]
 34%|███▎      | 75397/225000 [148:38:49<208:19:23,  5.01s/it]
 34%|███▎      | 75398/225000 [148:38:53<196:03:57,  4.72s/it]
 34%|███▎      | 75399/225000 [148:38:56<170:17:33,  4.10s/it]
 34%|███▎      | 75400/225000 [148:38:58<149:32:34,  3.60s/it]
                                                              
{'loss': '0.8932', 'grad_norm': '3.543', 'learning_rate': '3.944e-05', 'epoch': '0.3351', 'num_input_tokens_seen': 245396920, 'train_runtime': '5.351e+05', 'train_tokens_per_second': '458.6'}

 34%|███▎      | 75400/225000 [148:38:58<149:32:34,  3.60s/it]
 34%|███▎      | 75401/225000 [148:39:01<136:25:34,  3.28s/it]
 34%|███▎      | 75402/225000 [148:39:08<186:59:56,  4.50s/it]
 34%|███▎      | 75403/225000 [148:39:12<175:08:08,  4.21s/it]
 34%|███▎      | 75404/225000 [148:39:15<161:49:05,  3.89s/it]
 34%|███▎      | 75405/225000 [148:39:17<144:53:20,  3.49s/it]
 34%|███▎      | 75406/225000 [148:39:21<140:41:41,  3.39s/it]
 34%|███▎      | 75407/225000 [148:39:23<132:31:15,  3.19s/it]
 34%|███▎      | 75408/225000 [148:39:31<187:47:35,  4.52s/it]
 34%|███▎      | 75409/225000 [148:39:36<197:52:16,  4.76s/it]
 34%|███▎      | 75410/225000 [148:39:38<165:42:25,  3.99s/it]
 34%|███▎      | 75411/225000 [148:39:41<149:18:54,  3.59s/it]
 34%|███▎      | 75412/225000 [148:39:44<137:34:30,  3.31s/it]
 34%|███▎      | 75413/225000 [148:39:48<154:13:18,  3.71s/it]
 34%|███▎      | 75414/225000 [148:39:55<195:35:08,  4.71s/it]
 34%|███▎      | 75415/225000 [148:39:58<172:38:50,  4.16s/it]
 34%|███▎      | 75416/225000 [148:40:05<201:19:14,  4.85s/it]
 34%|███▎      | 75417/225000 [148:40:07<171:47:28,  4.13s/it]
 34%|███▎      | 75418/225000 [148:40:10<154:05:34,  3.71s/it]
 34%|███▎      | 75419/225000 [148:40:13<147:47:10,  3.56s/it]
 34%|███▎      | 75420/225000 [148:40:20<193:17:00,  4.65s/it]
 34%|███▎      | 75421/225000 [148:40:24<181:03:42,  4.36s/it]
 34%|███▎      | 75422/225000 [148:40:27<169:29:16,  4.08s/it]
 34%|███▎      | 75423/225000 [148:40:33<186:20:02,  4.48s/it]
 34%|███▎      | 75424/225000 [148:40:35<162:11:12,  3.90s/it]
 34%|███▎      | 75425/225000 [148:40:38<144:39:22,  3.48s/it]
 34%|███▎      | 75426/225000 [148:40:41<140:57:02,  3.39s/it]
 34%|███▎      | 75427/225000 [148:40:45<142:35:26,  3.43s/it]
 34%|███▎      | 75428/225000 [148:40:52<192:09:21,  4.62s/it]
 34%|███▎      | 75429/225000 [148:40:55<169:53:44,  4.09s/it]
 34%|███▎      | 75430/225000 [148:40:58<157:10:10,  3.78s/it]
 34%|███▎      | 75431/225000 [148:41:01<144:39:11,  3.48s/it]
 34%|███▎      | 75432/225000 [148:41:07<174:17:10,  4.19s/it]
 34%|███▎      | 75433/225000 [148:41:13<198:13:38,  4.77s/it]
 34%|███▎      | 75434/225000 [148:41:15<166:55:03,  4.02s/it]
 34%|███▎      | 75435/225000 [148:41:18<150:55:57,  3.63s/it]
 34%|███▎      | 75436/225000 [148:41:25<196:11:31,  4.72s/it]
 34%|███▎      | 75437/225000 [148:41:28<171:01:10,  4.12s/it]
 34%|███▎      | 75438/225000 [148:41:31<161:21:42,  3.88s/it]
 34%|███▎      | 75439/225000 [148:41:35<165:00:09,  3.97s/it]
 34%|███▎      | 75440/225000 [148:41:39<158:25:16,  3.81s/it]
 34%|███▎      | 75441/225000 [148:41:41<146:00:30,  3.51s/it]
 34%|███▎      | 75442/225000 [148:41:44<139:17:26,  3.35s/it]
 34%|███▎      | 75443/225000 [148:41:48<137:52:31,  3.32s/it]
 34%|███▎      | 75444/225000 [148:41:50<131:24:20,  3.16s/it]
 34%|███▎      | 75445/225000 [148:41:54<141:21:14,  3.40s/it]
 34%|███▎      | 75446/225000 [148:42:03<205:32:46,  4.95s/it]
 34%|███▎      | 75447/225000 [148:42:08<208:39:49,  5.02s/it]
 34%|███▎      | 75448/225000 [148:42:11<175:15:31,  4.22s/it]
 34%|███▎      | 75449/225000 [148:42:13<152:55:08,  3.68s/it]
 34%|███▎      | 75450/225000 [148:42:15<138:30:08,  3.33s/it]
                                                              
{'loss': '0.8365', 'grad_norm': '4.102', 'learning_rate': '3.942e-05', 'epoch': '0.3353', 'num_input_tokens_seen': 245476272, 'train_runtime': '5.353e+05', 'train_tokens_per_second': '458.5'}

 34%|███▎      | 75450/225000 [148:42:15<138:30:08,  3.33s/it]
 34%|███▎      | 75451/225000 [148:42:19<143:38:22,  3.46s/it]
 34%|███▎      | 75452/225000 [148:42:27<203:20:31,  4.89s/it]
 34%|███▎      | 75453/225000 [148:42:31<185:27:14,  4.46s/it]
 34%|███▎      | 75454/225000 [148:42:34<164:28:30,  3.96s/it]
 34%|███▎      | 75455/225000 [148:42:36<149:52:40,  3.61s/it]
 34%|███▎      | 75456/225000 [148:42:39<139:40:54,  3.36s/it]
 34%|███▎      | 75457/225000 [148:42:48<202:24:21,  4.87s/it]
 34%|███▎      | 75458/225000 [148:42:51<187:29:34,  4.51s/it]
 34%|███▎      | 75459/225000 [148:42:54<166:03:22,  4.00s/it]
 34%|███▎      | 75460/225000 [148:42:57<152:55:33,  3.68s/it]
 34%|███▎      | 75461/225000 [148:43:00<148:33:28,  3.58s/it]
 34%|███▎      | 75462/225000 [148:43:03<140:39:17,  3.39s/it]
 34%|███▎      | 75463/225000 [148:43:06<136:39:22,  3.29s/it]
 34%|███▎      | 75464/225000 [148:43:09<131:35:32,  3.17s/it]
 34%|███▎      | 75465/225000 [148:43:13<140:21:09,  3.38s/it]
 34%|███▎      | 75466/225000 [148:43:17<141:46:22,  3.41s/it]
 34%|███▎      | 75467/225000 [148:43:21<149:48:18,  3.61s/it]
 34%|███▎      | 75468/225000 [148:43:31<226:53:45,  5.46s/it]
 34%|███▎      | 75469/225000 [148:43:36<228:10:09,  5.49s/it]
 34%|███▎      | 75470/225000 [148:43:39<199:10:51,  4.80s/it]
 34%|███▎      | 75471/225000 [148:43:42<175:12:18,  4.22s/it]
 34%|███▎      | 75472/225000 [148:43:45<158:06:00,  3.81s/it]
 34%|███▎      | 75473/225000 [148:43:48<148:03:19,  3.56s/it]
 34%|███▎      | 75474/225000 [148:43:51<143:21:16,  3.45s/it]
 34%|███▎      | 75475/225000 [148:43:54<137:36:50,  3.31s/it]
 34%|███▎      | 75476/225000 [148:43:57<135:06:00,  3.25s/it]
 34%|███▎      | 75477/225000 [148:44:05<188:54:43,  4.55s/it]
 34%|███▎      | 75478/225000 [148:44:13<234:12:33,  5.64s/it]
 34%|███▎      | 75479/225000 [148:44:17<209:00:59,  5.03s/it]
 34%|███▎      | 75480/225000 [148:44:19<178:40:47,  4.30s/it]
 34%|███▎      | 75481/225000 [148:44:22<155:41:46,  3.75s/it]
 34%|███▎      | 75482/225000 [148:44:28<187:31:59,  4.52s/it]
 34%|███▎      | 75483/225000 [148:44:32<182:30:21,  4.39s/it]
 34%|███▎      | 75484/225000 [148:44:35<158:41:37,  3.82s/it]
 34%|███▎      | 75485/225000 [148:44:41<190:47:23,  4.59s/it]
 34%|███▎      | 75486/225000 [148:44:46<191:27:21,  4.61s/it]
 34%|███▎      | 75487/225000 [148:44:49<174:25:25,  4.20s/it]
 34%|███▎      | 75488/225000 [148:44:51<153:28:44,  3.70s/it]
 34%|███▎      | 75489/225000 [148:44:54<144:30:39,  3.48s/it]
 34%|███▎      | 75490/225000 [148:44:57<136:31:59,  3.29s/it]
 34%|███▎      | 75491/225000 [148:45:06<209:32:29,  5.05s/it]
 34%|███▎      | 75492/225000 [148:45:09<185:55:02,  4.48s/it]
 34%|███▎      | 75493/225000 [148:45:12<166:44:55,  4.02s/it]
 34%|███▎      | 75494/225000 [148:45:15<153:34:44,  3.70s/it]
 34%|███▎      | 75495/225000 [148:45:18<142:52:10,  3.44s/it]
 34%|███▎      | 75496/225000 [148:45:21<140:30:22,  3.38s/it]
 34%|███▎      | 75497/225000 [148:45:26<148:29:12,  3.58s/it]
 34%|███▎      | 75498/225000 [148:45:28<139:30:36,  3.36s/it]
 34%|███▎      | 75499/225000 [148:45:32<138:55:15,  3.35s/it]
 34%|███▎      | 75500/225000 [148:45:35<136:57:48,  3.30s/it]
                                                              
{'loss': '0.8517', 'grad_norm': '2.552', 'learning_rate': '3.941e-05', 'epoch': '0.3356', 'num_input_tokens_seen': 245557128, 'train_runtime': '5.355e+05', 'train_tokens_per_second': '458.5'}

 34%|███▎      | 75500/225000 [148:45:35<136:57:48,  3.30s/it]
 34%|███▎      | 75501/225000 [148:45:38<135:13:57,  3.26s/it]
 34%|███▎      | 75502/225000 [148:45:42<137:59:35,  3.32s/it]
 34%|███▎      | 75503/225000 [148:45:46<158:36:20,  3.82s/it]
 34%|███▎      | 75504/225000 [148:45:51<170:07:45,  4.10s/it]
 34%|███▎      | 75505/225000 [148:45:54<159:29:04,  3.84s/it]
 34%|███▎      | 75506/225000 [148:45:58<157:42:44,  3.80s/it]
 34%|███▎      | 75507/225000 [148:46:04<179:06:23,  4.31s/it]
 34%|███▎      | 75508/225000 [148:46:14<253:02:23,  6.09s/it]
 34%|███▎      | 75509/225000 [148:46:16<208:42:08,  5.03s/it]
 34%|███▎      | 75510/225000 [148:46:19<176:04:00,  4.24s/it]
 34%|███▎      | 75511/225000 [148:46:22<157:14:28,  3.79s/it]
 34%|███▎      | 75512/225000 [148:46:29<196:09:18,  4.72s/it]
 34%|███▎      | 75513/225000 [148:46:31<173:20:00,  4.17s/it]
 34%|███▎      | 75514/225000 [148:46:34<157:51:24,  3.80s/it]
 34%|███▎      | 75515/225000 [148:46:37<145:52:16,  3.51s/it]
 34%|███▎      | 75516/225000 [148:46:40<140:54:01,  3.39s/it]
 34%|███▎      | 75517/225000 [148:46:43<133:31:23,  3.22s/it]
 34%|███▎      | 75518/225000 [148:46:46<126:46:30,  3.05s/it]
 34%|███▎      | 75519/225000 [148:46:53<182:58:05,  4.41s/it]
 34%|███▎      | 75520/225000 [148:47:02<229:59:45,  5.54s/it]
 34%|███▎      | 75521/225000 [148:47:04<188:49:06,  4.55s/it]
 34%|███▎      | 75522/225000 [148:47:06<163:56:21,  3.95s/it]
 34%|███▎      | 75523/225000 [148:47:09<146:12:03,  3.52s/it]
 34%|███▎      | 75524/225000 [148:47:12<140:40:37,  3.39s/it]
 34%|███▎      | 75525/225000 [148:47:15<138:07:13,  3.33s/it]
 34%|███▎      | 75526/225000 [148:47:19<141:25:02,  3.41s/it]
 34%|███▎      | 75527/225000 [148:47:23<153:33:17,  3.70s/it]
 34%|███▎      | 75528/225000 [148:47:31<207:59:58,  5.01s/it]
 34%|███▎      | 75529/225000 [148:47:34<184:40:47,  4.45s/it]
 34%|███▎      | 75530/225000 [148:47:38<174:05:05,  4.19s/it]
 34%|███▎      | 75531/225000 [148:47:41<156:31:12,  3.77s/it]
 34%|███▎      | 75532/225000 [148:47:44<150:42:21,  3.63s/it]
 34%|███▎      | 75533/225000 [148:47:47<145:20:16,  3.50s/it]
 34%|███▎      | 75534/225000 [148:47:54<185:08:06,  4.46s/it]
 34%|███▎      | 75535/225000 [148:47:57<166:03:57,  4.00s/it]
 34%|███▎      | 75536/225000 [148:48:01<164:18:34,  3.96s/it]
 34%|███▎      | 75537/225000 [148:48:03<147:19:23,  3.55s/it]
 34%|███▎      | 75538/225000 [148:48:10<186:07:25,  4.48s/it]
 34%|███▎      | 75539/225000 [148:48:19<248:25:10,  5.98s/it]
 34%|███▎      | 75540/225000 [148:48:22<207:48:00,  5.01s/it]
 34%|███▎      | 75541/225000 [148:48:25<179:55:44,  4.33s/it]
 34%|███▎      | 75542/225000 [148:48:31<204:25:16,  4.92s/it]
 34%|███▎      | 75543/225000 [148:48:34<177:03:59,  4.27s/it]
 34%|███▎      | 75544/225000 [148:48:37<157:54:33,  3.80s/it]
 34%|███▎      | 75545/225000 [148:48:40<152:47:40,  3.68s/it]
 34%|███▎      | 75546/225000 [148:48:43<140:52:08,  3.39s/it]
 34%|███▎      | 75547/225000 [148:48:46<135:58:34,  3.28s/it]
 34%|███▎      | 75548/225000 [148:48:49<134:43:28,  3.25s/it]
 34%|███▎      | 75549/225000 [148:48:52<128:52:38,  3.10s/it]
 34%|███▎      | 75550/225000 [148:48:55<126:53:10,  3.06s/it]
                                                              
{'loss': '0.9148', 'grad_norm': '2.75', 'learning_rate': '3.939e-05', 'epoch': '0.3358', 'num_input_tokens_seen': 245639224, 'train_runtime': '5.357e+05', 'train_tokens_per_second': '458.5'}

 34%|███▎      | 75550/225000 [148:48:55<126:53:10,  3.06s/it]
 34%|███▎      | 75551/225000 [148:48:59<145:13:24,  3.50s/it]
 34%|███▎      | 75552/225000 [148:49:08<208:52:45,  5.03s/it]
 34%|███▎      | 75553/225000 [148:49:14<220:32:35,  5.31s/it]
 34%|███▎      | 75554/225000 [148:49:16<188:37:11,  4.54s/it]
 34%|███▎      | 75555/225000 [148:49:19<160:47:59,  3.87s/it]
 34%|███▎      | 75556/225000 [148:49:21<142:08:22,  3.42s/it]
 34%|███▎      | 75557/225000 [148:49:26<164:24:26,  3.96s/it]
 34%|███▎      | 75558/225000 [148:49:33<196:25:25,  4.73s/it]
 34%|███▎      | 75559/225000 [148:49:35<167:06:03,  4.03s/it]
 34%|███▎      | 75560/225000 [148:49:43<208:18:42,  5.02s/it]
 34%|███▎      | 75561/225000 [148:49:45<180:49:12,  4.36s/it]
 34%|███▎      | 75562/225000 [148:49:50<179:48:35,  4.33s/it]
 34%|███▎      | 75563/225000 [148:49:53<171:10:36,  4.12s/it]
 34%|███▎      | 75564/225000 [148:49:57<161:20:23,  3.89s/it]
 34%|███▎      | 75565/225000 [148:50:00<156:33:45,  3.77s/it]
 34%|███▎      | 75566/225000 [148:50:05<175:51:25,  4.24s/it]
 34%|███▎      | 75567/225000 [148:50:09<166:31:38,  4.01s/it]
 34%|███▎      | 75568/225000 [148:50:15<193:59:05,  4.67s/it]
 34%|███▎      | 75569/225000 [148:50:19<183:40:14,  4.42s/it]
 34%|███▎      | 75570/225000 [148:50:22<165:04:36,  3.98s/it]
 34%|███▎      | 75571/225000 [148:50:26<160:19:19,  3.86s/it]
 34%|███▎      | 75572/225000 [148:50:29<155:02:04,  3.74s/it]
 34%|███▎      | 75573/225000 [148:50:32<148:25:06,  3.58s/it]
 34%|███▎      | 75574/225000 [148:50:35<141:08:13,  3.40s/it]
 34%|███▎      | 75575/225000 [148:50:38<135:14:53,  3.26s/it]
 34%|███▎      | 75576/225000 [148:50:42<137:40:04,  3.32s/it]
 34%|███▎      | 75577/225000 [148:50:44<132:25:40,  3.19s/it]
 34%|███▎      | 75578/225000 [148:50:48<133:51:50,  3.23s/it]
 34%|███▎      | 75579/225000 [148:50:51<128:39:43,  3.10s/it]
 34%|███▎      | 75580/225000 [148:50:55<139:03:38,  3.35s/it]
 34%|███▎      | 75581/225000 [148:50:58<134:38:47,  3.24s/it]
 34%|███▎      | 75582/225000 [148:51:01<135:20:38,  3.26s/it]
 34%|███▎      | 75583/225000 [148:51:04<134:34:56,  3.24s/it]
 34%|███▎      | 75584/225000 [148:51:07<137:17:45,  3.31s/it]
 34%|███▎      | 75585/225000 [148:51:13<159:23:48,  3.84s/it]
 34%|███▎      | 75586/225000 [148:51:21<215:07:57,  5.18s/it]
 34%|███▎      | 75587/225000 [148:51:26<215:25:08,  5.19s/it]
 34%|███▎      | 75588/225000 [148:51:29<190:41:51,  4.59s/it]
 34%|███▎      | 75589/225000 [148:51:33<181:15:20,  4.37s/it]
 34%|███▎      | 75590/225000 [148:51:39<194:40:38,  4.69s/it]
 34%|███▎      | 75591/225000 [148:51:41<168:43:26,  4.07s/it]
 34%|███▎      | 75592/225000 [148:51:44<152:52:19,  3.68s/it]
 34%|███▎      | 75593/225000 [148:51:49<165:44:48,  3.99s/it]
 34%|███▎      | 75594/225000 [148:51:54<184:59:51,  4.46s/it]
 34%|███▎      | 75595/225000 [148:51:57<166:02:33,  4.00s/it]
 34%|███▎      | 75596/225000 [148:52:00<151:51:59,  3.66s/it]
 34%|███▎      | 75597/225000 [148:52:03<142:34:20,  3.44s/it]
 34%|███▎      | 75598/225000 [148:52:06<135:50:04,  3.27s/it]
 34%|███▎      | 75599/225000 [148:52:09<131:03:25,  3.16s/it]
 34%|███▎      | 75600/225000 [148:52:12<128:35:05,  3.10s/it]
                                                              
{'loss': '0.8316', 'grad_norm': '4.177', 'learning_rate': '3.938e-05', 'epoch': '0.336', 'num_input_tokens_seen': 245719168, 'train_runtime': '5.359e+05', 'train_tokens_per_second': '458.5'}

 34%|███▎      | 75600/225000 [148:52:12<128:35:05,  3.10s/it]
 34%|███▎      | 75601/225000 [148:52:18<170:18:10,  4.10s/it]
 34%|███▎      | 75602/225000 [148:52:26<211:21:56,  5.09s/it]
 34%|███▎      | 75603/225000 [148:52:29<192:32:19,  4.64s/it]
 34%|███▎      | 75604/225000 [148:52:32<167:07:18,  4.03s/it]
 34%|███▎      | 75605/225000 [148:52:34<147:13:28,  3.55s/it]
 34%|███▎      | 75606/225000 [148:52:40<175:25:22,  4.23s/it]
 34%|███▎      | 75607/225000 [148:52:44<171:59:36,  4.14s/it]
 34%|███▎      | 75608/225000 [148:52:48<169:16:20,  4.08s/it]
 34%|███▎      | 75609/225000 [148:52:51<158:05:08,  3.81s/it]
 34%|███▎      | 75610/225000 [148:52:54<146:46:00,  3.54s/it]
 34%|███▎      | 75611/225000 [148:52:58<148:41:44,  3.58s/it]
 34%|███▎      | 75612/225000 [148:53:03<166:31:50,  4.01s/it]
 34%|███▎      | 75613/225000 [148:53:12<237:12:45,  5.72s/it]
 34%|███▎      | 75614/225000 [148:53:16<212:03:43,  5.11s/it]
 34%|███▎      | 75615/225000 [148:53:19<183:06:36,  4.41s/it]
 34%|███▎      | 75616/225000 [148:53:23<181:42:28,  4.38s/it]
 34%|███▎      | 75617/225000 [148:53:29<202:22:01,  4.88s/it]
 34%|███▎      | 75618/225000 [148:53:32<174:08:07,  4.20s/it]
 34%|███▎      | 75619/225000 [148:53:34<155:03:24,  3.74s/it]
 34%|███▎      | 75620/225000 [148:53:39<163:10:33,  3.93s/it]
 34%|███▎      | 75621/225000 [148:53:45<190:04:04,  4.58s/it]
 34%|███▎      | 75622/225000 [148:53:47<160:04:01,  3.86s/it]
 34%|███▎      | 75623/225000 [148:53:52<167:42:57,  4.04s/it]
 34%|███▎      | 75624/225000 [148:53:57<187:23:04,  4.52s/it]
 34%|███▎      | 75625/225000 [148:54:00<162:07:11,  3.91s/it]
 34%|███▎      | 75626/225000 [148:54:02<145:59:35,  3.52s/it]
 34%|███▎      | 75627/225000 [148:54:05<139:37:59,  3.37s/it]
 34%|███▎      | 75628/225000 [148:54:08<136:20:16,  3.29s/it]
 34%|███▎      | 75629/225000 [148:54:11<131:02:23,  3.16s/it]
 34%|███▎      | 75630/225000 [148:54:14<126:43:12,  3.05s/it]
 34%|███▎      | 75631/225000 [148:54:23<200:11:56,  4.83s/it]
 34%|███▎      | 75632/225000 [148:54:25<170:48:06,  4.12s/it]
 34%|███▎      | 75633/225000 [148:54:29<167:56:36,  4.05s/it]
 34%|███▎      | 75634/225000 [148:54:36<195:47:29,  4.72s/it]
 34%|███▎      | 75635/225000 [148:54:38<165:56:05,  4.00s/it]
 34%|███▎      | 75636/225000 [148:54:41<150:38:35,  3.63s/it]
 34%|███▎      | 75637/225000 [148:54:48<198:28:18,  4.78s/it]
 34%|███▎      | 75638/225000 [148:54:51<175:23:11,  4.23s/it]
 34%|███▎      | 75639/225000 [148:54:54<159:36:15,  3.85s/it]
 34%|███▎      | 75640/225000 [148:54:57<151:23:04,  3.65s/it]
 34%|███▎      | 75641/225000 [148:55:00<145:26:38,  3.51s/it]
 34%|███▎      | 75642/225000 [148:55:04<140:15:16,  3.38s/it]
 34%|███▎      | 75643/225000 [148:55:07<135:47:36,  3.27s/it]
 34%|███▎      | 75644/225000 [148:55:15<193:59:00,  4.68s/it]
 34%|███▎      | 75645/225000 [148:55:18<173:37:38,  4.19s/it]
 34%|███▎      | 75646/225000 [148:55:24<199:11:11,  4.80s/it]
 34%|███▎      | 75647/225000 [148:55:27<180:59:42,  4.36s/it]
 34%|███▎      | 75648/225000 [148:55:30<164:24:53,  3.96s/it]
 34%|███▎      | 75649/225000 [148:55:34<162:27:06,  3.92s/it]
 34%|███▎      | 75650/225000 [148:55:37<154:14:16,  3.72s/it]
                                                              
{'loss': '0.8683', 'grad_norm': '2.24', 'learning_rate': '3.936e-05', 'epoch': '0.3362', 'num_input_tokens_seen': 245799200, 'train_runtime': '5.361e+05', 'train_tokens_per_second': '458.5'}

 34%|███▎      | 75650/225000 [148:55:37<154:14:16,  3.72s/it]
 34%|███▎      | 75651/225000 [148:55:40<142:52:04,  3.44s/it]
 34%|███▎      | 75652/225000 [148:55:43<133:29:50,  3.22s/it]
 34%|███▎      | 75653/225000 [148:55:48<160:08:19,  3.86s/it]
 34%|███▎      | 75654/225000 [148:55:56<213:11:15,  5.14s/it]
 34%|███▎      | 75655/225000 [148:55:59<179:25:41,  4.33s/it]
 34%|███▎      | 75656/225000 [148:56:02<166:52:16,  4.02s/it]
 34%|███▎      | 75657/225000 [148:56:09<198:39:59,  4.79s/it]
 34%|███▎      | 75658/225000 [148:56:11<168:52:56,  4.07s/it]
 34%|███▎      | 75659/225000 [148:56:13<146:57:07,  3.54s/it]
 34%|███▎      | 75660/225000 [148:56:16<140:54:00,  3.40s/it]
 34%|███▎      | 75661/225000 [148:56:22<168:22:56,  4.06s/it]
 34%|███▎      | 75662/225000 [148:56:28<188:13:37,  4.54s/it]
 34%|███▎      | 75663/225000 [148:56:30<159:50:17,  3.85s/it]
 34%|███▎      | 75664/225000 [148:56:36<188:02:37,  4.53s/it]
 34%|███▎      | 75665/225000 [148:56:40<185:54:35,  4.48s/it]
 34%|███▎      | 75666/225000 [148:56:43<160:44:47,  3.88s/it]
 34%|███▎      | 75667/225000 [148:56:45<144:53:13,  3.49s/it]
 34%|███▎      | 75668/225000 [148:56:48<139:31:51,  3.36s/it]
 34%|███▎      | 75669/225000 [148:56:51<135:19:08,  3.26s/it]
 34%|███▎      | 75670/225000 [148:56:54<128:02:22,  3.09s/it]
 34%|███▎      | 75671/225000 [148:56:57<124:36:11,  3.00s/it]
 34%|███▎      | 75672/225000 [148:57:00<125:46:35,  3.03s/it]
 34%|███▎      | 75673/225000 [148:57:08<185:40:02,  4.48s/it]
 34%|███▎      | 75674/225000 [148:57:15<214:29:26,  5.17s/it]
 34%|███▎      | 75675/225000 [148:57:17<184:49:05,  4.46s/it]
 34%|███▎      | 75676/225000 [148:57:20<157:40:37,  3.80s/it]
 34%|███▎      | 75677/225000 [148:57:25<174:05:02,  4.20s/it]
 34%|███▎      | 75678/225000 [148:57:30<182:25:42,  4.40s/it]
 34%|███▎      | 75679/225000 [148:57:32<158:33:41,  3.82s/it]
 34%|███▎      | 75680/225000 [148:57:35<141:26:33,  3.41s/it]
 34%|███▎      | 75681/225000 [148:57:38<136:11:32,  3.28s/it]
 34%|███▎      | 75682/225000 [148:57:45<191:44:50,  4.62s/it]
 34%|███▎      | 75683/225000 [148:57:49<179:54:00,  4.34s/it]
 34%|███▎      | 75684/225000 [148:57:53<169:57:42,  4.10s/it]
 34%|███▎      | 75685/225000 [148:57:56<156:21:30,  3.77s/it]
 34%|███▎      | 75686/225000 [148:57:59<149:57:32,  3.62s/it]
 34%|███▎      | 75687/225000 [148:58:02<141:34:02,  3.41s/it]
 34%|███▎      | 75688/225000 [148:58:06<149:55:25,  3.61s/it]
 34%|███▎      | 75689/225000 [148:58:10<152:08:44,  3.67s/it]
 34%|███▎      | 75690/225000 [148:58:13<144:24:11,  3.48s/it]
 34%|███▎      | 75691/225000 [148:58:16<141:02:46,  3.40s/it]
 34%|███▎      | 75692/225000 [148:58:24<202:04:06,  4.87s/it]
 34%|███▎      | 75693/225000 [148:58:30<208:04:05,  5.02s/it]
 34%|███▎      | 75694/225000 [148:58:32<179:12:00,  4.32s/it]
 34%|███▎      | 75695/225000 [148:58:35<155:20:08,  3.75s/it]
 34%|███▎      | 75696/225000 [148:58:38<147:45:19,  3.56s/it]
 34%|███▎      | 75697/225000 [148:58:41<139:44:31,  3.37s/it]
 34%|███▎      | 75698/225000 [148:58:44<134:29:18,  3.24s/it]
 34%|███▎      | 75699/225000 [148:58:47<134:53:44,  3.25s/it]
 34%|███▎      | 75700/225000 [148:58:50<138:10:48,  3.33s/it]
                                                              
{'loss': '0.8433', 'grad_norm': '2.131', 'learning_rate': '3.935e-05', 'epoch': '0.3364', 'num_input_tokens_seen': 245880136, 'train_runtime': '5.363e+05', 'train_tokens_per_second': '458.4'}

 34%|███▎      | 75700/225000 [148:58:50<138:10:48,  3.33s/it]
 34%|███▎      | 75701/225000 [148:58:57<176:59:14,  4.27s/it]
 34%|███▎      | 75702/225000 [148:59:06<235:05:15,  5.67s/it]
 34%|███▎      | 75703/225000 [148:59:08<194:00:59,  4.68s/it]
 34%|███▎      | 75704/225000 [148:59:11<164:51:02,  3.98s/it]
 34%|███▎      | 75705/225000 [148:59:13<145:12:22,  3.50s/it]
 34%|███▎      | 75706/225000 [148:59:20<185:16:29,  4.47s/it]
 34%|███▎      | 75707/225000 [148:59:25<191:26:27,  4.62s/it]
 34%|███▎      | 75708/225000 [148:59:28<172:10:07,  4.15s/it]
 34%|███▎      | 75709/225000 [148:59:31<156:33:02,  3.78s/it]
 34%|███▎      | 75710/225000 [148:59:34<148:24:49,  3.58s/it]
 34%|███▎      | 75711/225000 [148:59:39<166:38:47,  4.02s/it]
 34%|███▎      | 75712/225000 [148:59:46<210:13:47,  5.07s/it]
 34%|███▎      | 75713/225000 [148:59:49<184:45:20,  4.46s/it]
 34%|███▎      | 75714/225000 [148:59:53<172:19:33,  4.16s/it]
 34%|███▎      | 75715/225000 [148:59:56<162:19:15,  3.91s/it]
 34%|███▎      | 75716/225000 [148:59:59<148:40:05,  3.59s/it]
 34%|███▎      | 75717/225000 [149:00:03<150:25:16,  3.63s/it]
 34%|███▎      | 75718/225000 [149:00:06<145:17:46,  3.50s/it]
 34%|███▎      | 75719/225000 [149:00:09<136:55:27,  3.30s/it]
 34%|███▎      | 75720/225000 [149:00:12<132:12:09,  3.19s/it]
 34%|███▎      | 75721/225000 [149:00:15<133:43:08,  3.22s/it]
 34%|███▎      | 75722/225000 [149:00:21<174:02:46,  4.20s/it]
 34%|███▎      | 75723/225000 [149:00:33<270:34:38,  6.53s/it]
 34%|███▎      | 75724/225000 [149:00:36<219:49:45,  5.30s/it]
 34%|███▎      | 75725/225000 [149:00:38<185:00:38,  4.46s/it]
 34%|███▎      | 75726/225000 [149:00:41<161:03:44,  3.88s/it]
 34%|███▎      | 75727/225000 [149:00:43<145:33:56,  3.51s/it]
 34%|███▎      | 75728/225000 [149:00:51<190:16:30,  4.59s/it]
 34%|███▎      | 75729/225000 [149:00:56<202:56:13,  4.89s/it]
 34%|███▎      | 75730/225000 [149:00:59<175:18:48,  4.23s/it]
 34%|███▎      | 75731/225000 [149:01:02<160:58:51,  3.88s/it]
 34%|███▎      | 75732/225000 [149:01:05<150:57:38,  3.64s/it]
 34%|███▎      | 75733/225000 [149:01:11<177:41:19,  4.29s/it]
 34%|███▎      | 75734/225000 [149:01:19<229:11:41,  5.53s/it]
 34%|███▎      | 75735/225000 [149:01:21<187:59:07,  4.53s/it]
 34%|███▎      | 75736/225000 [149:01:24<159:27:51,  3.85s/it]
 34%|███▎      | 75737/225000 [149:01:26<143:45:22,  3.47s/it]
 34%|███▎      | 75738/225000 [149:01:29<128:21:42,  3.10s/it]
 34%|███▎      | 75739/225000 [149:01:34<158:39:46,  3.83s/it]
 34%|███▎      | 75740/225000 [149:01:38<159:34:33,  3.85s/it]
 34%|███▎      | 75741/225000 [149:01:40<140:26:01,  3.39s/it]
 34%|███▎      | 75742/225000 [149:01:43<128:30:37,  3.10s/it]
 34%|███▎      | 75743/225000 [149:01:48<156:45:08,  3.78s/it]
 34%|███▎      | 75744/225000 [149:01:56<214:54:03,  5.18s/it]
 34%|███▎      | 75745/225000 [149:01:59<185:12:22,  4.47s/it]
 34%|███▎      | 75746/225000 [149:02:03<170:24:38,  4.11s/it]
 34%|███▎      | 75747/225000 [149:02:06<163:40:31,  3.95s/it]
 34%|███▎      | 75748/225000 [149:02:10<158:03:22,  3.81s/it]
 34%|███▎      | 75749/225000 [149:02:13<153:37:56,  3.71s/it]
 34%|███▎      | 75750/225000 [149:02:16<147:14:38,  3.55s/it]
                                                              
{'loss': '0.8838', 'grad_norm': '2.72', 'learning_rate': '3.933e-05', 'epoch': '0.3367', 'num_input_tokens_seen': 245962256, 'train_runtime': '5.365e+05', 'train_tokens_per_second': '458.4'}

 34%|███▎      | 75750/225000 [149:02:16<147:14:38,  3.55s/it]
 34%|███▎      | 75751/225000 [149:02:19<138:58:35,  3.35s/it]
 34%|███▎      | 75752/225000 [149:02:22<132:37:20,  3.20s/it]
 34%|███▎      | 75753/225000 [149:02:26<137:08:10,  3.31s/it]
 34%|███▎      | 75754/225000 [149:02:29<135:22:04,  3.27s/it]
 34%|███▎      | 75755/225000 [149:02:34<165:38:38,  4.00s/it]
 34%|███▎      | 75756/225000 [149:02:43<217:00:24,  5.23s/it]
 34%|███▎      | 75757/225000 [149:02:50<242:42:14,  5.85s/it]
 34%|███▎      | 75758/225000 [149:02:53<212:13:53,  5.12s/it]
 34%|███▎      | 75759/225000 [149:02:56<184:01:20,  4.44s/it]
 34%|███▎      | 75760/225000 [149:02:59<160:14:04,  3.87s/it]
 34%|███▎      | 75761/225000 [149:03:01<141:08:22,  3.40s/it]
 34%|███▎      | 75762/225000 [149:03:06<164:49:09,  3.98s/it]
 34%|███▎      | 75763/225000 [149:03:12<191:46:33,  4.63s/it]
 34%|███▎      | 75764/225000 [149:03:15<171:26:06,  4.14s/it]
 34%|███▎      | 75765/225000 [149:03:18<151:22:31,  3.65s/it]
 34%|███▎      | 75766/225000 [149:03:20<135:21:52,  3.27s/it]
 34%|███▎      | 75767/225000 [149:03:23<133:32:03,  3.22s/it]
 34%|███▎      | 75768/225000 [149:03:26<129:50:59,  3.13s/it]
 34%|███▎      | 75769/225000 [149:03:34<182:31:20,  4.40s/it]
 34%|███▎      | 75770/225000 [149:03:38<182:01:12,  4.39s/it]
 34%|███▎      | 75771/225000 [149:03:40<156:06:39,  3.77s/it]
 34%|███▎      | 75772/225000 [149:03:43<137:57:20,  3.33s/it]
 34%|███▎      | 75773/225000 [149:03:47<152:42:01,  3.68s/it]
 34%|███▎      | 75774/225000 [149:03:51<149:10:32,  3.60s/it]
 34%|███▎      | 75775/225000 [149:03:53<139:57:56,  3.38s/it]
 34%|███▎      | 75776/225000 [149:03:56<129:18:41,  3.12s/it]
 34%|███▎      | 75777/225000 [149:04:01<150:54:55,  3.64s/it]
 34%|███▎      | 75778/225000 [149:04:08<198:37:52,  4.79s/it]
 34%|███▎      | 75779/225000 [149:04:11<173:40:10,  4.19s/it]
 34%|███▎      | 75780/225000 [149:04:14<157:00:35,  3.79s/it]
 34%|███▎      | 75781/225000 [149:04:16<140:55:18,  3.40s/it]
 34%|███▎      | 75782/225000 [149:04:19<129:45:20,  3.13s/it]
 34%|███▎      | 75783/225000 [149:04:27<186:24:59,  4.50s/it]
 34%|███▎      | 75784/225000 [149:04:33<207:04:51,  5.00s/it]
 34%|███▎      | 75785/225000 [149:04:36<178:29:51,  4.31s/it]
 34%|███▎      | 75786/225000 [149:04:38<158:47:41,  3.83s/it]
 34%|███▎      | 75787/225000 [149:04:42<152:14:58,  3.67s/it]
 34%|███▎      | 75788/225000 [149:04:44<141:56:37,  3.42s/it]
 34%|███▎      | 75789/225000 [149:04:47<130:32:08,  3.15s/it]
 34%|███▎      | 75790/225000 [149:04:53<164:06:32,  3.96s/it]
 34%|███▎      | 75791/225000 [149:04:56<160:00:50,  3.86s/it]
 34%|███▎      | 75792/225000 [149:05:00<162:44:41,  3.93s/it]
 34%|███▎      | 75793/225000 [149:05:04<153:13:22,  3.70s/it]
 34%|███▎      | 75794/225000 [149:05:07<147:04:12,  3.55s/it]
 34%|███▎      | 75795/225000 [149:05:10<145:45:49,  3.52s/it]
 34%|███▎      | 75796/225000 [149:05:13<140:22:07,  3.39s/it]
 34%|███▎      | 75797/225000 [149:05:16<137:15:46,  3.31s/it]
 34%|███▎      | 75798/225000 [149:05:20<138:02:22,  3.33s/it]
 34%|███▎      | 75799/225000 [149:05:28<197:11:03,  4.76s/it]
 34%|███▎      | 75800/225000 [149:05:34<214:18:46,  5.17s/it]
                                                              
{'loss': '0.8698', 'grad_norm': '2.665', 'learning_rate': '3.932e-05', 'epoch': '0.3369', 'num_input_tokens_seen': 246043752, 'train_runtime': '5.367e+05', 'train_tokens_per_second': '458.4'}

 34%|███▎      | 75800/225000 [149:05:34<214:18:46,  5.17s/it]
 34%|███▎      | 75801/225000 [149:05:37<184:01:32,  4.44s/it]
 34%|███▎      | 75802/225000 [149:05:39<161:05:17,  3.89s/it]
 34%|███▎      | 75803/225000 [149:05:42<147:24:52,  3.56s/it]
 34%|███▎      | 75804/225000 [149:05:46<146:28:56,  3.53s/it]
 34%|███▎      | 75805/225000 [149:05:49<142:28:56,  3.44s/it]
 34%|███▎      | 75806/225000 [149:05:53<153:58:30,  3.72s/it]
 34%|███▎      | 75807/225000 [149:05:57<151:35:44,  3.66s/it]
 34%|███▎      | 75808/225000 [149:06:00<144:01:09,  3.48s/it]
 34%|███▎      | 75809/225000 [149:06:07<189:07:46,  4.56s/it]
 34%|███▎      | 75810/225000 [149:06:15<236:01:27,  5.70s/it]
 34%|███▎      | 75811/225000 [149:06:18<196:55:24,  4.75s/it]
 34%|███▎      | 75812/225000 [149:06:21<174:28:06,  4.21s/it]
 34%|███▎      | 75813/225000 [149:06:24<158:13:31,  3.82s/it]
 34%|███▎      | 75814/225000 [149:06:28<159:12:52,  3.84s/it]
 34%|███▎      | 75815/225000 [149:06:30<146:07:33,  3.53s/it]
 34%|███▎      | 75816/225000 [149:06:33<139:50:13,  3.37s/it]
 34%|███▎      | 75817/225000 [149:06:36<133:07:01,  3.21s/it]
 34%|███▎      | 75818/225000 [149:06:39<127:38:35,  3.08s/it]
 34%|███▎      | 75819/225000 [149:06:42<127:00:59,  3.07s/it]
 34%|███▎      | 75820/225000 [149:06:47<154:43:42,  3.73s/it]
 34%|███▎      | 75821/225000 [149:06:56<221:59:38,  5.36s/it]
 34%|███▎      | 75822/225000 [149:07:00<202:26:28,  4.89s/it]
 34%|███▎      | 75823/225000 [149:07:03<175:19:50,  4.23s/it]
 34%|███▎      | 75824/225000 [149:07:07<173:29:32,  4.19s/it]
 34%|███▎      | 75825/225000 [149:07:13<198:57:30,  4.80s/it]
 34%|███▎      | 75826/225000 [149:07:16<172:03:12,  4.15s/it]
 34%|███▎      | 75827/225000 [149:07:20<172:29:35,  4.16s/it]
 34%|███▎      | 75828/225000 [149:07:26<190:48:49,  4.60s/it]
 34%|███▎      | 75829/225000 [149:07:28<161:09:41,  3.89s/it]
 34%|███▎      | 75830/225000 [149:07:34<184:32:27,  4.45s/it]
 34%|███▎      | 75831/225000 [149:07:37<176:08:55,  4.25s/it]
 34%|███▎      | 75832/225000 [149:07:41<171:47:23,  4.15s/it]
 34%|███▎      | 75833/225000 [149:07:47<196:21:18,  4.74s/it]
 34%|███▎      | 75834/225000 [149:07:51<183:34:47,  4.43s/it]
 34%|███▎      | 75835/225000 [149:07:54<165:20:17,  3.99s/it]
 34%|███▎      | 75836/225000 [149:07:57<156:51:26,  3.79s/it]
 34%|███▎      | 75837/225000 [149:08:00<145:11:01,  3.50s/it]
 34%|███▎      | 75838/225000 [149:08:03<139:19:49,  3.36s/it]
 34%|███▎      | 75839/225000 [149:08:06<135:17:25,  3.27s/it]
 34%|███▎      | 75840/225000 [149:08:09<131:11:44,  3.17s/it]
 34%|███▎      | 75841/225000 [149:08:12<130:08:47,  3.14s/it]
 34%|███▎      | 75842/225000 [149:08:15<129:19:22,  3.12s/it]
 34%|███▎      | 75843/225000 [149:08:19<128:10:18,  3.09s/it]
 34%|███▎      | 75844/225000 [149:08:26<188:21:42,  4.55s/it]
 34%|███▎      | 75845/225000 [149:08:30<179:34:08,  4.33s/it]
 34%|███▎      | 75846/225000 [149:08:34<171:12:40,  4.13s/it]
 34%|███▎      | 75847/225000 [149:08:40<189:28:08,  4.57s/it]
 34%|███▎      | 75848/225000 [149:08:42<165:59:04,  4.01s/it]
 34%|███▎      | 75849/225000 [149:08:47<176:52:37,  4.27s/it]
 34%|███▎      | 75850/225000 [149:08:50<155:38:26,  3.76s/it]
                                                              
{'loss': '0.8635', 'grad_norm': '1.477', 'learning_rate': '3.93e-05', 'epoch': '0.3371', 'num_input_tokens_seen': 246123960, 'train_runtime': '5.369e+05', 'train_tokens_per_second': '458.4'}

 34%|███▎      | 75850/225000 [149:08:50<155:38:26,  3.76s/it]
 34%|███▎      | 75851/225000 [149:08:52<142:13:57,  3.43s/it]
 34%|███▎      | 75852/225000 [149:08:55<131:57:37,  3.19s/it]
 34%|███▎      | 75853/225000 [149:08:59<136:21:39,  3.29s/it]
 34%|███▎      | 75854/225000 [149:09:02<139:58:57,  3.38s/it]
 34%|███▎      | 75855/225000 [149:09:06<143:44:40,  3.47s/it]
 34%|███▎      | 75856/225000 [149:09:15<210:11:55,  5.07s/it]
 34%|███▎      | 75857/225000 [149:09:21<225:42:05,  5.45s/it]
 34%|███▎      | 75858/225000 [149:09:23<185:57:48,  4.49s/it]
 34%|███▎      | 75859/225000 [149:09:26<163:01:33,  3.94s/it]
 34%|███▎      | 75860/225000 [149:09:32<193:12:15,  4.66s/it]
 34%|███▎      | 75861/225000 [149:09:35<166:00:39,  4.01s/it]
 34%|███▎      | 75862/225000 [149:09:38<154:02:24,  3.72s/it]
 34%|███▎      | 75863/225000 [149:09:41<145:14:07,  3.51s/it]
 34%|███▎      | 75864/225000 [149:09:43<133:30:43,  3.22s/it]
 34%|███▎      | 75865/225000 [149:09:47<139:31:11,  3.37s/it]
 34%|███▎      | 75866/225000 [149:09:53<166:39:54,  4.02s/it]
 34%|███▎      | 75867/225000 [149:10:00<211:28:55,  5.11s/it]
 34%|███▎      | 75868/225000 [149:10:04<199:59:37,  4.83s/it]
 34%|███▎      | 75869/225000 [149:10:07<178:35:44,  4.31s/it]
 34%|███▎      | 75870/225000 [149:10:11<163:42:56,  3.95s/it]
 34%|███▎      | 75871/225000 [149:10:13<149:21:24,  3.61s/it]
 34%|███▎      | 75872/225000 [149:10:16<138:15:29,  3.34s/it]
 34%|███▎      | 75873/225000 [149:10:19<129:33:38,  3.13s/it]
 34%|███▎      | 75874/225000 [149:10:23<141:50:35,  3.42s/it]
 34%|███▎      | 75875/225000 [149:10:30<193:48:51,  4.68s/it]
 34%|███▎      | 75876/225000 [149:10:36<207:59:24,  5.02s/it]
 34%|███▎      | 75877/225000 [149:10:39<184:04:19,  4.44s/it]
 34%|███▎      | 75878/225000 [149:10:42<161:48:06,  3.91s/it]
 34%|███▎      | 75879/225000 [149:10:45<147:30:11,  3.56s/it]
 34%|███▎      | 75880/225000 [149:10:47<135:50:25,  3.28s/it]
 34%|███▎      | 75881/225000 [149:10:51<135:59:53,  3.28s/it]
 34%|███▎      | 75882/225000 [149:10:54<135:42:50,  3.28s/it]
 34%|███▎      | 75883/225000 [149:10:58<144:50:13,  3.50s/it]
 34%|███▎      | 75884/225000 [149:11:08<224:13:47,  5.41s/it]
 34%|███▎      | 75885/225000 [149:11:10<187:50:23,  4.53s/it]
 34%|███▎      | 75886/225000 [149:11:13<169:10:53,  4.08s/it]
 34%|███▎      | 75887/225000 [149:11:16<155:24:58,  3.75s/it]
 34%|███▎      | 75888/225000 [149:11:19<145:48:11,  3.52s/it]
 34%|███▎      | 75889/225000 [149:11:22<138:28:28,  3.34s/it]
 34%|███▎      | 75890/225000 [149:11:29<176:44:55,  4.27s/it]
 34%|███▎      | 75891/225000 [149:11:35<203:19:44,  4.91s/it]
 34%|███▎      | 75892/225000 [149:11:38<174:48:46,  4.22s/it]
 34%|███▎      | 75893/225000 [149:11:40<156:28:58,  3.78s/it]
 34%|███▎      | 75894/225000 [149:11:43<143:24:01,  3.46s/it]
 34%|███▎      | 75895/225000 [149:11:47<142:16:30,  3.44s/it]
 34%|███▎      | 75896/225000 [149:11:49<133:22:08,  3.22s/it]
 34%|███▎      | 75897/225000 [149:11:53<140:09:40,  3.38s/it]
 34%|███▎      | 75898/225000 [149:11:56<135:12:47,  3.26s/it]
 34%|███▎      | 75899/225000 [149:12:04<193:28:59,  4.67s/it]
 34%|███▎      | 75900/225000 [149:12:10<213:36:36,  5.16s/it]
                                                              
{'loss': '0.801', 'grad_norm': '2.396', 'learning_rate': '3.929e-05', 'epoch': '0.3373', 'num_input_tokens_seen': 246205816, 'train_runtime': '5.371e+05', 'train_tokens_per_second': '458.4'}

 34%|███▎      | 75900/225000 [149:12:10<213:36:36,  5.16s/it]
 34%|███▎      | 75901/225000 [149:12:13<184:05:28,  4.44s/it]
 34%|███▎      | 75902/225000 [149:12:16<169:15:46,  4.09s/it]
 34%|███▎      | 75903/225000 [149:12:20<166:02:44,  4.01s/it]
 34%|███▎      | 75904/225000 [149:12:23<151:25:05,  3.66s/it]
 34%|███▎      | 75905/225000 [149:12:25<137:41:57,  3.32s/it]
 34%|███▎      | 75906/225000 [149:12:28<131:49:06,  3.18s/it]
 34%|███▎      | 75907/225000 [149:12:33<151:16:30,  3.65s/it]
 34%|███▎      | 75908/225000 [149:12:43<234:59:55,  5.67s/it]
 34%|███▎      | 75909/225000 [149:12:46<201:20:24,  4.86s/it]
 34%|███▎      | 75910/225000 [149:12:50<182:50:28,  4.41s/it]
 34%|███▎      | 75911/225000 [149:12:53<169:23:52,  4.09s/it]
 34%|███▎      | 75912/225000 [149:12:56<153:19:00,  3.70s/it]
 34%|███▎      | 75913/225000 [149:12:59<140:00:10,  3.38s/it]
 34%|███▎      | 75914/225000 [149:13:04<159:53:40,  3.86s/it]
 34%|███▎      | 75915/225000 [149:13:12<221:06:21,  5.34s/it]
 34%|███▎      | 75916/225000 [149:13:15<184:06:48,  4.45s/it]
 34%|███▎      | 75917/225000 [149:13:18<168:09:51,  4.06s/it]
 34%|███▎      | 75918/225000 [149:13:20<147:04:51,  3.55s/it]
 34%|███▎      | 75919/225000 [149:13:23<137:24:15,  3.32s/it]
 34%|███▎      | 75920/225000 [149:13:30<185:14:39,  4.47s/it]
 34%|███▎      | 75921/225000 [149:13:36<204:27:11,  4.94s/it]
 34%|███▎      | 75922/225000 [149:13:39<172:45:33,  4.17s/it]
 34%|███▎      | 75923/225000 [149:13:42<157:32:53,  3.80s/it]
 34%|███▎      | 75924/225000 [149:13:48<186:22:47,  4.50s/it]
 34%|███▎      | 75925/225000 [149:13:50<164:31:28,  3.97s/it]
 34%|███▎      | 75926/225000 [149:13:53<151:29:28,  3.66s/it]
 34%|███▎      | 75927/225000 [149:13:56<142:32:41,  3.44s/it]
 34%|███▎      | 75928/225000 [149:14:02<175:09:38,  4.23s/it]
 34%|███▎      | 75929/225000 [149:14:06<167:07:16,  4.04s/it]
 34%|███▎      | 75930/225000 [149:14:09<158:25:10,  3.83s/it]
 34%|███▎      | 75931/225000 [149:14:13<151:51:26,  3.67s/it]
 34%|███▎      | 75932/225000 [149:14:16<149:16:27,  3.60s/it]
 34%|███▎      | 75933/225000 [149:14:19<141:19:25,  3.41s/it]
 34%|███▎      | 75934/225000 [149:14:22<142:26:01,  3.44s/it]
 34%|███▎      | 75935/225000 [149:14:26<143:14:32,  3.46s/it]
 34%|███▎      | 75936/225000 [149:14:30<146:50:08,  3.55s/it]
 34%|███▎      | 75937/225000 [149:14:37<194:18:21,  4.69s/it]
 34%|███▍      | 75938/225000 [149:14:40<169:24:38,  4.09s/it]
 34%|███▍      | 75939/225000 [149:14:42<149:23:19,  3.61s/it]
 34%|███▍      | 75940/225000 [149:14:49<193:52:59,  4.68s/it]
 34%|███▍      | 75941/225000 [149:14:52<164:07:32,  3.96s/it]
 34%|███▍      | 75942/225000 [149:14:54<145:14:31,  3.51s/it]
 34%|███▍      | 75943/225000 [149:14:58<155:10:50,  3.75s/it]
 34%|███▍      | 75944/225000 [149:15:06<204:48:31,  4.95s/it]
 34%|███▍      | 75945/225000 [149:15:09<182:15:47,  4.40s/it]
 34%|███▍      | 75946/225000 [149:15:12<164:09:13,  3.96s/it]
 34%|███▍      | 75947/225000 [149:15:16<155:59:19,  3.77s/it]
 34%|███▍      | 75948/225000 [149:15:21<182:07:08,  4.40s/it]
 34%|███▍      | 75949/225000 [149:15:25<168:18:31,  4.07s/it]
 34%|███▍      | 75950/225000 [149:15:28<153:11:43,  3.70s/it]
                                                              
{'loss': '0.8605', 'grad_norm': '3.142', 'learning_rate': '3.927e-05', 'epoch': '0.3376', 'num_input_tokens_seen': 246286944, 'train_runtime': '5.373e+05', 'train_tokens_per_second': '458.4'}

 34%|███▍      | 75950/225000 [149:15:28<153:11:43,  3.70s/it]
 34%|███▍      | 75951/225000 [149:15:31<154:05:46,  3.72s/it]
 34%|███▍      | 75952/225000 [149:15:35<151:13:59,  3.65s/it]
 34%|███▍      | 75953/225000 [149:15:38<143:45:13,  3.47s/it]
 34%|███▍      | 75954/225000 [149:15:44<175:28:02,  4.24s/it]
 34%|███▍      | 75955/225000 [149:15:49<191:05:46,  4.62s/it]
 34%|███▍      | 75956/225000 [149:15:53<173:55:05,  4.20s/it]
 34%|███▍      | 75957/225000 [149:15:56<159:15:08,  3.85s/it]
 34%|███▍      | 75958/225000 [149:15:58<142:52:39,  3.45s/it]
 34%|███▍      | 75959/225000 [149:16:01<135:41:32,  3.28s/it]
 34%|███▍      | 75960/225000 [149:16:04<124:57:04,  3.02s/it]
 34%|███▍      | 75961/225000 [149:16:07<129:10:42,  3.12s/it]
 34%|███▍      | 75962/225000 [149:16:10<132:13:47,  3.19s/it]
 34%|███▍      | 75963/225000 [149:16:17<181:04:02,  4.37s/it]
 34%|███▍      | 75964/225000 [149:16:21<172:38:08,  4.17s/it]
 34%|███▍      | 75965/225000 [149:16:25<165:55:41,  4.01s/it]
 34%|███▍      | 75966/225000 [149:16:28<157:43:52,  3.81s/it]
 34%|███▍      | 75967/225000 [149:16:36<203:25:13,  4.91s/it]
 34%|███▍      | 75968/225000 [149:16:43<235:02:03,  5.68s/it]
 34%|███▍      | 75969/225000 [149:16:45<195:35:07,  4.72s/it]
 34%|███▍      | 75970/225000 [149:16:49<175:12:03,  4.23s/it]
 34%|███▍      | 75971/225000 [149:16:52<161:24:06,  3.90s/it]
 34%|███▍      | 75972/225000 [149:16:54<144:31:00,  3.49s/it]
 34%|███▍      | 75973/225000 [149:17:03<208:32:32,  5.04s/it]
 34%|███▍      | 75974/225000 [149:17:06<189:25:10,  4.58s/it]
 34%|███▍      | 75975/225000 [149:17:09<169:42:25,  4.10s/it]
 34%|███▍      | 75976/225000 [149:17:13<159:38:21,  3.86s/it]
 34%|███▍      | 75977/225000 [149:17:16<155:33:02,  3.76s/it]
 34%|███▍      | 75978/225000 [149:17:21<162:55:09,  3.94s/it]
 34%|███▍      | 75979/225000 [149:17:25<175:10:07,  4.23s/it]
 34%|███▍      | 75980/225000 [149:17:30<182:25:50,  4.41s/it]
 34%|███▍      | 75981/225000 [149:17:33<156:38:08,  3.78s/it]
 34%|███▍      | 75982/225000 [149:17:35<141:44:19,  3.42s/it]
 34%|███▍      | 75983/225000 [149:17:38<130:08:55,  3.14s/it]
 34%|███▍      | 75984/225000 [149:17:42<143:48:27,  3.47s/it]
 34%|███▍      | 75985/225000 [149:17:48<181:31:54,  4.39s/it]
 34%|███▍      | 75986/225000 [149:17:51<160:23:42,  3.87s/it]
 34%|███▍      | 75987/225000 [149:17:56<170:15:07,  4.11s/it]
 34%|███▍      | 75988/225000 [149:18:01<183:37:18,  4.44s/it]
 34%|███▍      | 75989/225000 [149:18:04<168:38:21,  4.07s/it]
 34%|███▍      | 75990/225000 [149:18:07<150:46:49,  3.64s/it]
 34%|███▍      | 75991/225000 [149:18:10<145:53:00,  3.52s/it]
 34%|███▍      | 75992/225000 [149:18:13<138:07:18,  3.34s/it]
 34%|███▍      | 75993/225000 [149:18:16<131:50:06,  3.19s/it]
 34%|███▍      | 75994/225000 [149:18:19<133:26:55,  3.22s/it]
 34%|███▍      | 75995/225000 [149:18:23<137:14:31,  3.32s/it]
 34%|███▍      | 75996/225000 [149:18:31<193:44:54,  4.68s/it]
 34%|███▍      | 75997/225000 [149:18:38<225:51:59,  5.46s/it]
 34%|███▍      | 75998/225000 [149:18:41<191:58:26,  4.64s/it]
 34%|███▍      | 75999/225000 [149:18:43<171:01:58,  4.13s/it]
 34%|███▍      | 76000/225000 [149:18:46<156:52:29,  3.79s/it]
                                                              
{'loss': '0.8677', 'grad_norm': '3.877', 'learning_rate': '3.926e-05', 'epoch': '0.3378', 'num_input_tokens_seen': 246368048, 'train_runtime': '5.375e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76000/225000 [149:18:46<156:52:29,  3.79s/it]
 34%|███▍      | 76001/225000 [149:18:50<152:30:48,  3.68s/it]
 34%|███▍      | 76002/225000 [149:18:54<153:44:50,  3.71s/it]
 34%|███▍      | 76003/225000 [149:18:56<140:39:39,  3.40s/it]
 34%|███▍      | 76004/225000 [149:18:59<131:35:45,  3.18s/it]
 34%|███▍      | 76005/225000 [149:19:02<129:28:30,  3.13s/it]
 34%|███▍      | 76006/225000 [149:19:08<168:11:38,  4.06s/it]
 34%|███▍      | 76007/225000 [149:19:15<199:31:11,  4.82s/it]
 34%|███▍      | 76008/225000 [149:19:17<171:09:29,  4.14s/it]
 34%|███▍      | 76009/225000 [149:19:20<154:23:35,  3.73s/it]
 34%|███▍      | 76010/225000 [149:19:23<142:33:52,  3.44s/it]
 34%|███▍      | 76011/225000 [149:19:26<136:26:55,  3.30s/it]
 34%|███▍      | 76012/225000 [149:19:30<148:13:26,  3.58s/it]
 34%|███▍      | 76013/225000 [149:19:38<195:34:53,  4.73s/it]
 34%|███▍      | 76014/225000 [149:19:40<166:54:18,  4.03s/it]
 34%|███▍      | 76015/225000 [149:19:46<195:59:51,  4.74s/it]
 34%|███▍      | 76016/225000 [149:19:49<170:40:10,  4.12s/it]
 34%|███▍      | 76017/225000 [149:19:52<153:33:01,  3.71s/it]
 34%|███▍      | 76018/225000 [149:19:55<146:12:17,  3.53s/it]
 34%|███▍      | 76019/225000 [149:19:59<147:48:38,  3.57s/it]
 34%|███▍      | 76020/225000 [149:20:01<137:56:47,  3.33s/it]
 34%|███▍      | 76021/225000 [149:20:08<178:40:19,  4.32s/it]
 34%|███▍      | 76022/225000 [149:20:14<206:03:13,  4.98s/it]
 34%|███▍      | 76023/225000 [149:20:17<175:52:09,  4.25s/it]
 34%|███▍      | 76024/225000 [149:20:19<152:56:46,  3.70s/it]
 34%|███▍      | 76025/225000 [149:20:22<136:47:45,  3.31s/it]
 34%|███▍      | 76026/225000 [149:20:27<162:52:53,  3.94s/it]
 34%|███▍      | 76027/225000 [149:20:34<197:28:45,  4.77s/it]
 34%|███▍      | 76028/225000 [149:20:38<192:16:52,  4.65s/it]
 34%|███▍      | 76029/225000 [149:20:41<168:24:13,  4.07s/it]
 34%|███▍      | 76030/225000 [149:20:44<152:20:20,  3.68s/it]
 34%|███▍      | 76031/225000 [149:20:47<147:00:23,  3.55s/it]
 34%|███▍      | 76032/225000 [149:20:50<143:57:14,  3.48s/it]
 34%|███▍      | 76033/225000 [149:20:54<143:16:50,  3.46s/it]
 34%|███▍      | 76034/225000 [149:21:01<189:51:29,  4.59s/it]
 34%|███▍      | 76035/225000 [149:21:08<223:44:25,  5.41s/it]
 34%|███▍      | 76036/225000 [149:21:11<187:40:31,  4.54s/it]
 34%|███▍      | 76037/225000 [149:21:14<166:27:53,  4.02s/it]
 34%|███▍      | 76038/225000 [149:21:16<151:27:42,  3.66s/it]
 34%|███▍      | 76039/225000 [149:21:19<142:18:52,  3.44s/it]
 34%|███▍      | 76040/225000 [149:21:22<136:49:33,  3.31s/it]
 34%|███▍      | 76041/225000 [149:21:31<206:54:58,  5.00s/it]
 34%|███▍      | 76042/225000 [149:21:34<183:24:35,  4.43s/it]
 34%|███▍      | 76043/225000 [149:21:37<161:37:38,  3.91s/it]
 34%|███▍      | 76044/225000 [149:21:40<143:57:47,  3.48s/it]
 34%|███▍      | 76045/225000 [149:21:43<137:31:04,  3.32s/it]
 34%|███▍      | 76046/225000 [149:21:46<136:32:06,  3.30s/it]
 34%|███▍      | 76047/225000 [149:21:50<142:23:44,  3.44s/it]
 34%|███▍      | 76048/225000 [149:21:53<143:52:03,  3.48s/it]
 34%|███▍      | 76049/225000 [149:22:01<195:45:03,  4.73s/it]
 34%|███▍      | 76050/225000 [149:22:05<185:14:11,  4.48s/it]
                                                              
{'loss': '0.8363', 'grad_norm': '2.479', 'learning_rate': '3.924e-05', 'epoch': '0.338', 'num_input_tokens_seen': 246449640, 'train_runtime': '5.377e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76050/225000 [149:22:05<185:14:11,  4.48s/it]
 34%|███▍      | 76051/225000 [149:22:07<164:28:54,  3.98s/it]
 34%|███▍      | 76052/225000 [149:22:15<204:16:11,  4.94s/it]
 34%|███▍      | 76053/225000 [149:22:18<179:04:14,  4.33s/it]
 34%|███▍      | 76054/225000 [149:22:20<155:11:47,  3.75s/it]
 34%|███▍      | 76055/225000 [149:22:23<144:40:05,  3.50s/it]
 34%|███▍      | 76056/225000 [149:22:26<144:11:51,  3.49s/it]
 34%|███▍      | 76057/225000 [149:22:30<142:10:08,  3.44s/it]
 34%|███▍      | 76058/225000 [149:22:33<139:34:23,  3.37s/it]
 34%|███▍      | 76059/225000 [149:22:37<149:45:10,  3.62s/it]
 34%|███▍      | 76060/225000 [149:22:41<158:53:52,  3.84s/it]
 34%|███▍      | 76061/225000 [149:22:46<164:19:54,  3.97s/it]
 34%|███▍      | 76062/225000 [149:22:49<153:30:23,  3.71s/it]
 34%|███▍      | 76063/225000 [149:22:53<154:28:34,  3.73s/it]
 34%|███▍      | 76064/225000 [149:22:56<148:19:26,  3.59s/it]
 34%|███▍      | 76065/225000 [149:22:59<147:38:58,  3.57s/it]
 34%|███▍      | 76066/225000 [149:23:08<214:20:51,  5.18s/it]
 34%|███▍      | 76067/225000 [149:23:13<206:01:58,  4.98s/it]
 34%|███▍      | 76068/225000 [149:23:16<188:07:14,  4.55s/it]
 34%|███▍      | 76069/225000 [149:23:19<163:37:09,  3.96s/it]
 34%|███▍      | 76070/225000 [149:23:22<152:47:39,  3.69s/it]
 34%|███▍      | 76071/225000 [149:23:25<141:50:27,  3.43s/it]
 34%|███▍      | 76072/225000 [149:23:31<172:26:20,  4.17s/it]
 34%|███▍      | 76073/225000 [149:23:38<212:31:06,  5.14s/it]
 34%|███▍      | 76074/225000 [149:23:41<181:26:47,  4.39s/it]
 34%|███▍      | 76075/225000 [149:23:44<166:03:17,  4.01s/it]
 34%|███▍      | 76076/225000 [149:23:47<154:12:57,  3.73s/it]
 34%|███▍      | 76077/225000 [149:23:50<144:00:39,  3.48s/it]
 34%|███▍      | 76078/225000 [149:23:53<140:30:50,  3.40s/it]
 34%|███▍      | 76079/225000 [149:23:56<133:17:03,  3.22s/it]
 34%|███▍      | 76080/225000 [149:23:59<130:58:59,  3.17s/it]
 34%|███▍      | 76081/225000 [149:24:02<126:28:29,  3.06s/it]
 34%|███▍      | 76082/225000 [149:24:05<125:20:30,  3.03s/it]
 34%|███▍      | 76083/225000 [149:24:08<131:10:58,  3.17s/it]
 34%|███▍      | 76084/225000 [149:24:12<141:22:25,  3.42s/it]
 34%|███▍      | 76085/225000 [149:24:21<210:17:41,  5.08s/it]
 34%|███▍      | 76086/225000 [149:24:28<231:20:29,  5.59s/it]
 34%|███▍      | 76087/225000 [149:24:30<190:53:16,  4.61s/it]
 34%|███▍      | 76088/225000 [149:24:33<167:58:58,  4.06s/it]
 34%|███▍      | 76089/225000 [149:24:41<210:23:48,  5.09s/it]
 34%|███▍      | 76090/225000 [149:24:43<176:45:10,  4.27s/it]
 34%|███▍      | 76091/225000 [149:24:46<158:34:55,  3.83s/it]
 34%|███▍      | 76092/225000 [149:24:48<141:49:38,  3.43s/it]
 34%|███▍      | 76093/225000 [149:24:51<134:45:03,  3.26s/it]
 34%|███▍      | 76094/225000 [149:24:53<124:39:24,  3.01s/it]
 34%|███▍      | 76095/225000 [149:25:00<172:07:29,  4.16s/it]
 34%|███▍      | 76096/225000 [149:25:06<189:47:00,  4.59s/it]
 34%|███▍      | 76097/225000 [149:25:09<174:30:30,  4.22s/it]
 34%|███▍      | 76098/225000 [149:25:12<158:36:19,  3.83s/it]
 34%|███▍      | 76099/225000 [149:25:19<196:07:36,  4.74s/it]
 34%|███▍      | 76100/225000 [149:25:22<175:25:28,  4.24s/it]
                                                              
{'loss': '0.8809', 'grad_norm': '2.211', 'learning_rate': '3.923e-05', 'epoch': '0.3382', 'num_input_tokens_seen': 246530064, 'train_runtime': '5.379e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76100/225000 [149:25:22<175:25:28,  4.24s/it]
 34%|███▍      | 76101/225000 [149:25:25<155:34:33,  3.76s/it]
 34%|███▍      | 76102/225000 [149:25:28<152:38:35,  3.69s/it]
 34%|███▍      | 76103/225000 [149:25:35<184:05:28,  4.45s/it]
 34%|███▍      | 76104/225000 [149:25:37<155:42:46,  3.76s/it]
 34%|███▍      | 76105/225000 [149:25:42<179:36:07,  4.34s/it]
 34%|███▍      | 76106/225000 [149:25:47<180:31:03,  4.36s/it]
 34%|███▍      | 76107/225000 [149:25:50<166:30:39,  4.03s/it]
 34%|███▍      | 76108/225000 [149:25:54<166:07:22,  4.02s/it]
 34%|███▍      | 76109/225000 [149:25:57<152:04:53,  3.68s/it]
 34%|███▍      | 76110/225000 [149:26:00<145:24:51,  3.52s/it]
 34%|███▍      | 76111/225000 [149:26:04<144:36:13,  3.50s/it]
 34%|███▍      | 76112/225000 [149:26:07<146:56:53,  3.55s/it]
 34%|███▍      | 76113/225000 [149:26:10<136:34:28,  3.30s/it]
 34%|███▍      | 76114/225000 [149:26:13<134:14:40,  3.25s/it]
 34%|███▍      | 76115/225000 [149:26:17<138:14:08,  3.34s/it]
 34%|███▍      | 76116/225000 [149:26:25<196:33:53,  4.75s/it]
 34%|███▍      | 76117/225000 [149:26:29<197:44:01,  4.78s/it]
 34%|███▍      | 76118/225000 [149:26:37<235:39:07,  5.70s/it]
 34%|███▍      | 76119/225000 [149:26:40<200:29:27,  4.85s/it]
 34%|███▍      | 76120/225000 [149:26:43<169:37:00,  4.10s/it]
 34%|███▍      | 76121/225000 [149:26:48<191:23:23,  4.63s/it]
 34%|███▍      | 76122/225000 [149:26:53<186:03:19,  4.50s/it]
 34%|███▍      | 76123/225000 [149:26:55<164:16:36,  3.97s/it]
 34%|███▍      | 76124/225000 [149:27:00<167:16:23,  4.04s/it]
 34%|███▍      | 76125/225000 [149:27:05<186:39:54,  4.51s/it]
 34%|███▍      | 76126/225000 [149:27:08<161:18:20,  3.90s/it]
 34%|███▍      | 76127/225000 [149:27:10<147:13:13,  3.56s/it]
 34%|███▍      | 76128/225000 [149:27:13<137:04:01,  3.31s/it]
 34%|███▍      | 76129/225000 [149:27:16<130:40:27,  3.16s/it]
 34%|███▍      | 76130/225000 [149:27:19<132:21:24,  3.20s/it]
 34%|███▍      | 76131/225000 [149:27:25<169:05:23,  4.09s/it]
 34%|███▍      | 76132/225000 [149:27:33<210:33:00,  5.09s/it]
 34%|███▍      | 76133/225000 [149:27:35<178:54:47,  4.33s/it]
 34%|███▍      | 76134/225000 [149:27:38<159:27:59,  3.86s/it]
 34%|███▍      | 76135/225000 [149:27:41<151:03:53,  3.65s/it]
 34%|███▍      | 76136/225000 [149:27:44<139:30:51,  3.37s/it]
 34%|███▍      | 76137/225000 [149:27:47<129:08:22,  3.12s/it]
 34%|███▍      | 76138/225000 [149:27:49<124:40:40,  3.02s/it]
 34%|███▍      | 76139/225000 [149:27:57<180:38:42,  4.37s/it]
 34%|███▍      | 76140/225000 [149:28:02<191:37:34,  4.63s/it]
 34%|███▍      | 76141/225000 [149:28:06<185:53:49,  4.50s/it]
 34%|███▍      | 76142/225000 [149:28:10<182:13:06,  4.41s/it]
 34%|███▍      | 76143/225000 [149:28:13<162:16:40,  3.92s/it]
 34%|███▍      | 76144/225000 [149:28:16<144:00:58,  3.48s/it]
 34%|███▍      | 76145/225000 [149:28:21<160:31:04,  3.88s/it]
 34%|███▍      | 76146/225000 [149:28:29<213:56:48,  5.17s/it]
 34%|███▍      | 76147/225000 [149:28:31<178:35:56,  4.32s/it]
 34%|███▍      | 76148/225000 [149:28:34<155:08:59,  3.75s/it]
 34%|███▍      | 76149/225000 [149:28:36<144:14:55,  3.49s/it]
 34%|███▍      | 76150/225000 [149:28:39<135:05:47,  3.27s/it]
                                                              
{'loss': '0.8596', 'grad_norm': '3.754', 'learning_rate': '3.921e-05', 'epoch': '0.3384', 'num_input_tokens_seen': 246611424, 'train_runtime': '5.381e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76150/225000 [149:28:39<135:05:47,  3.27s/it]
 34%|███▍      | 76151/225000 [149:28:42<133:54:42,  3.24s/it]
 34%|███▍      | 76152/225000 [149:28:45<126:42:15,  3.06s/it]
 34%|███▍      | 76153/225000 [149:28:55<215:46:40,  5.22s/it]
 34%|███▍      | 76154/225000 [149:28:59<195:52:39,  4.74s/it]
 34%|███▍      | 76155/225000 [149:29:04<199:05:45,  4.82s/it]
 34%|███▍      | 76156/225000 [149:29:08<186:18:11,  4.51s/it]
 34%|███▍      | 76157/225000 [149:29:10<164:53:21,  3.99s/it]
 34%|███▍      | 76158/225000 [149:29:13<146:40:51,  3.55s/it]
 34%|███▍      | 76159/225000 [149:29:16<138:08:56,  3.34s/it]
 34%|███▍      | 76160/225000 [149:29:18<128:09:06,  3.10s/it]
 34%|███▍      | 76161/225000 [149:29:21<125:15:30,  3.03s/it]
 34%|███▍      | 76162/225000 [149:29:28<168:30:38,  4.08s/it]
 34%|███▍      | 76163/225000 [149:29:32<170:58:50,  4.14s/it]
 34%|███▍      | 76164/225000 [149:29:36<172:42:10,  4.18s/it]
 34%|███▍      | 76165/225000 [149:29:40<165:03:36,  3.99s/it]
 34%|███▍      | 76166/225000 [149:29:43<149:43:35,  3.62s/it]
 34%|███▍      | 76167/225000 [149:29:45<141:05:53,  3.41s/it]
 34%|███▍      | 76168/225000 [149:29:48<132:48:09,  3.21s/it]
 34%|███▍      | 76169/225000 [149:29:51<126:43:46,  3.07s/it]
 34%|███▍      | 76170/225000 [149:29:55<134:00:57,  3.24s/it]
 34%|███▍      | 76171/225000 [149:30:00<158:52:05,  3.84s/it]
 34%|███▍      | 76172/225000 [149:30:07<205:18:23,  4.97s/it]
 34%|███▍      | 76173/225000 [149:30:10<180:20:48,  4.36s/it]
 34%|███▍      | 76174/225000 [149:30:13<162:46:07,  3.94s/it]
 34%|███▍      | 76175/225000 [149:30:16<152:53:38,  3.70s/it]
 34%|███▍      | 76176/225000 [149:30:23<191:49:36,  4.64s/it]
 34%|███▍      | 76177/225000 [149:30:26<171:42:31,  4.15s/it]
 34%|███▍      | 76178/225000 [149:30:29<155:05:23,  3.75s/it]
 34%|███▍      | 76179/225000 [149:30:34<174:31:18,  4.22s/it]
 34%|███▍      | 76180/225000 [149:30:41<206:46:33,  5.00s/it]
 34%|███▍      | 76181/225000 [149:30:44<174:40:37,  4.23s/it]
 34%|███▍      | 76182/225000 [149:30:46<150:28:06,  3.64s/it]
 34%|███▍      | 76183/225000 [149:30:50<150:13:28,  3.63s/it]
 34%|███▍      | 76184/225000 [149:30:53<142:45:32,  3.45s/it]
 34%|███▍      | 76185/225000 [149:30:56<147:57:20,  3.58s/it]
 34%|███▍      | 76186/225000 [149:31:02<176:05:19,  4.26s/it]
 34%|███▍      | 76187/225000 [149:31:09<210:15:42,  5.09s/it]
 34%|███▍      | 76188/225000 [149:31:12<179:54:23,  4.35s/it]
 34%|███▍      | 76189/225000 [149:31:14<155:13:37,  3.76s/it]
 34%|███▍      | 76190/225000 [149:31:17<139:47:14,  3.38s/it]
 34%|███▍      | 76191/225000 [149:31:20<134:24:52,  3.25s/it]
 34%|███▍      | 76192/225000 [149:31:23<136:55:16,  3.31s/it]
 34%|███▍      | 76193/225000 [149:31:26<135:05:08,  3.27s/it]
 34%|███▍      | 76194/225000 [149:31:33<175:33:33,  4.25s/it]
 34%|███▍      | 76195/225000 [149:31:39<192:54:18,  4.67s/it]
 34%|███▍      | 76196/225000 [149:31:42<174:34:38,  4.22s/it]
 34%|███▍      | 76197/225000 [149:31:45<157:23:26,  3.81s/it]
 34%|███▍      | 76198/225000 [149:31:48<147:11:48,  3.56s/it]
 34%|███▍      | 76199/225000 [149:31:50<137:40:00,  3.33s/it]
 34%|███▍      | 76200/225000 [149:31:53<132:16:58,  3.20s/it]
                                                              
{'loss': '0.8426', 'grad_norm': '3.868', 'learning_rate': '3.92e-05', 'epoch': '0.3387', 'num_input_tokens_seen': 246693560, 'train_runtime': '5.383e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76200/225000 [149:31:53<132:16:58,  3.20s/it]
 34%|███▍      | 76201/225000 [149:31:58<151:41:25,  3.67s/it]
 34%|███▍      | 76202/225000 [149:32:08<230:03:18,  5.57s/it]
 34%|███▍      | 76203/225000 [149:32:11<196:15:18,  4.75s/it]
 34%|███▍      | 76204/225000 [149:32:13<165:14:23,  4.00s/it]
 34%|███▍      | 76205/225000 [149:32:19<191:03:05,  4.62s/it]
 34%|███▍      | 76206/225000 [149:32:24<196:04:23,  4.74s/it]
 34%|███▍      | 76207/225000 [149:32:27<174:56:29,  4.23s/it]
 34%|███▍      | 76208/225000 [149:32:30<155:50:01,  3.77s/it]
 34%|███▍      | 76209/225000 [149:32:33<140:54:19,  3.41s/it]
 34%|███▍      | 76210/225000 [149:32:35<130:17:16,  3.15s/it]
 34%|███▍      | 76211/225000 [149:32:38<129:37:53,  3.14s/it]
 34%|███▍      | 76212/225000 [149:32:41<128:15:54,  3.10s/it]
 34%|███▍      | 76213/225000 [149:32:47<163:18:27,  3.95s/it]
 34%|███▍      | 76214/225000 [149:32:50<153:54:04,  3.72s/it]
 34%|███▍      | 76215/225000 [149:32:53<143:41:51,  3.48s/it]
 34%|███▍      | 76216/225000 [149:32:59<171:58:43,  4.16s/it]
 34%|███▍      | 76217/225000 [149:33:02<161:46:38,  3.91s/it]
 34%|███▍      | 76218/225000 [149:33:05<149:42:37,  3.62s/it]
 34%|███▍      | 76219/225000 [149:33:08<141:50:00,  3.43s/it]
 34%|███▍      | 76220/225000 [149:33:12<141:17:08,  3.42s/it]
 34%|███▍      | 76221/225000 [149:33:15<134:03:53,  3.24s/it]
 34%|███▍      | 76222/225000 [149:33:17<130:08:25,  3.15s/it]
 34%|███▍      | 76223/225000 [149:33:20<127:13:06,  3.08s/it]
 34%|███▍      | 76224/225000 [149:33:24<138:17:51,  3.35s/it]
 34%|███▍      | 76225/225000 [149:33:34<211:22:23,  5.11s/it]
 34%|███▍      | 76226/225000 [149:33:39<214:25:59,  5.19s/it]
 34%|███▍      | 76227/225000 [149:33:41<177:46:24,  4.30s/it]
 34%|███▍      | 76228/225000 [149:33:44<161:49:24,  3.92s/it]
 34%|███▍      | 76229/225000 [149:33:47<144:02:42,  3.49s/it]
 34%|███▍      | 76230/225000 [149:33:50<139:30:01,  3.38s/it]
 34%|███▍      | 76231/225000 [149:33:57<192:27:24,  4.66s/it]
 34%|███▍      | 76232/225000 [149:34:04<211:02:51,  5.11s/it]
 34%|███▍      | 76233/225000 [149:34:07<185:29:53,  4.49s/it]
 34%|███▍      | 76234/225000 [149:34:09<161:10:22,  3.90s/it]
 34%|███▍      | 76235/225000 [149:34:12<145:15:52,  3.52s/it]
 34%|███▍      | 76236/225000 [149:34:14<133:38:11,  3.23s/it]
 34%|███▍      | 76237/225000 [149:34:18<138:56:09,  3.36s/it]
 34%|███▍      | 76238/225000 [149:34:21<133:25:53,  3.23s/it]
 34%|███▍      | 76239/225000 [149:34:26<156:55:33,  3.80s/it]
 34%|███▍      | 76240/225000 [149:34:34<209:37:38,  5.07s/it]
 34%|███▍      | 76241/225000 [149:34:37<183:30:03,  4.44s/it]
 34%|███▍      | 76242/225000 [149:34:39<156:57:20,  3.80s/it]
 34%|███▍      | 76243/225000 [149:34:43<151:32:49,  3.67s/it]
 34%|███▍      | 76244/225000 [149:34:46<141:43:56,  3.43s/it]
 34%|███▍      | 76245/225000 [149:34:49<142:55:14,  3.46s/it]
 34%|███▍      | 76246/225000 [149:34:53<152:59:18,  3.70s/it]
 34%|███▍      | 76247/225000 [149:34:59<176:04:55,  4.26s/it]
 34%|███▍      | 76248/225000 [149:35:09<241:41:15,  5.85s/it]
 34%|███▍      | 76249/225000 [149:35:12<206:16:03,  4.99s/it]
 34%|███▍      | 76250/225000 [149:35:15<186:59:16,  4.53s/it]
                                                              
{'loss': '0.8823', 'grad_norm': '2.276', 'learning_rate': '3.918e-05', 'epoch': '0.3389', 'num_input_tokens_seen': 246775312, 'train_runtime': '5.385e+05', 'train_tokens_per_second': '458.3'}

 34%|███▍      | 76250/225000 [149:35:15<186:59:16,  4.53s/it]
 34%|███▍      | 76251/225000 [149:35:19<183:45:14,  4.45s/it]
 34%|███▍      | 76252/225000 [149:35:22<165:41:50,  4.01s/it]
 34%|███▍      | 76253/225000 [149:35:25<155:30:18,  3.76s/it]
 34%|███▍      | 76254/225000 [149:35:28<147:19:24,  3.57s/it]
 34%|███▍      | 76255/225000 [149:35:32<143:00:00,  3.46s/it]
 34%|███▍      | 76256/225000 [149:35:35<137:14:09,  3.32s/it]
 34%|███▍      | 76257/225000 [149:35:37<128:47:30,  3.12s/it]
 34%|███▍      | 76258/225000 [149:35:41<139:18:57,  3.37s/it]
 34%|███▍      | 76259/225000 [149:35:50<207:25:50,  5.02s/it]
 34%|███▍      | 76260/225000 [149:35:56<212:03:48,  5.13s/it]
 34%|███▍      | 76261/225000 [149:35:59<186:38:42,  4.52s/it]
 34%|███▍      | 76262/225000 [149:36:01<159:00:47,  3.85s/it]
 34%|███▍      | 76263/225000 [149:36:04<149:34:06,  3.62s/it]
 34%|███▍      | 76264/225000 [149:36:09<162:48:50,  3.94s/it]
 34%|███▍      | 76265/225000 [149:36:16<204:51:51,  4.96s/it]
 34%|███▍      | 76266/225000 [149:36:21<203:36:03,  4.93s/it]
 34%|███▍      | 76267/225000 [149:36:24<174:43:47,  4.23s/it]
 34%|███▍      | 76268/225000 [149:36:26<152:55:35,  3.70s/it]
 34%|███▍      | 76269/225000 [149:36:29<144:53:28,  3.51s/it]
 34%|███▍      | 76270/225000 [149:36:33<152:33:14,  3.69s/it]
 34%|███▍      | 76271/225000 [149:36:36<144:53:35,  3.51s/it]
 34%|███▍      | 76272/225000 [149:36:40<143:01:06,  3.46s/it]
 34%|███▍      | 76273/225000 [149:36:43<139:47:59,  3.38s/it]
 34%|███▍      | 76274/225000 [149:36:52<206:22:58,  5.00s/it]
 34%|███▍      | 76275/225000 [149:36:56<196:41:58,  4.76s/it]
 34%|███▍      | 76276/225000 [149:36:58<165:29:47,  4.01s/it]
 34%|███▍      | 76277/225000 [149:37:00<144:20:49,  3.49s/it]
 34%|███▍      | 76278/225000 [149:37:06<171:00:27,  4.14s/it]
 34%|███▍      | 76279/225000 [149:37:13<202:25:32,  4.90s/it]
 34%|███▍      | 76280/225000 [149:37:15<174:31:38,  4.22s/it]
 34%|███▍      | 76281/225000 [149:37:19<165:15:18,  4.00s/it]
 34%|███▍      | 76282/225000 [149:37:22<157:12:43,  3.81s/it]
 34%|███▍      | 76283/225000 [149:37:25<146:24:34,  3.54s/it]
 34%|███▍      | 76284/225000 [149:37:29<154:55:54,  3.75s/it]
 34%|███▍      | 76285/225000 [149:37:37<208:47:06,  5.05s/it]
 34%|███▍      | 76286/225000 [149:37:40<182:57:42,  4.43s/it]
 34%|███▍      | 76287/225000 [149:37:43<164:44:14,  3.99s/it]
 34%|███▍      | 76288/225000 [149:37:46<148:53:36,  3.60s/it]
 34%|███▍      | 76289/225000 [149:37:49<137:15:25,  3.32s/it]
 34%|███▍      | 76290/225000 [149:37:57<197:50:29,  4.79s/it]
 34%|███▍      | 76291/225000 [149:38:01<186:45:29,  4.52s/it]
 34%|███▍      | 76292/225000 [149:38:05<186:16:26,  4.51s/it]
 34%|███▍      | 76293/225000 [149:38:08<167:00:11,  4.04s/it]
 34%|███▍      | 76294/225000 [149:38:10<144:35:08,  3.50s/it]
 34%|███▍      | 76295/225000 [149:38:17<185:37:18,  4.49s/it]
 34%|███▍      | 76296/225000 [149:38:21<171:31:41,  4.15s/it]
 34%|███▍      | 76297/225000 [149:38:23<152:02:48,  3.68s/it]
 34%|███▍      | 76298/225000 [149:38:26<142:16:02,  3.44s/it]
 34%|███▍      | 76299/225000 [149:38:29<131:10:35,  3.18s/it]
 34%|███▍      | 76300/225000 [149:38:31<126:19:09,  3.06s/it]
                                                              
{'loss': '0.8154', 'grad_norm': '1.991', 'learning_rate': '3.917e-05', 'epoch': '0.3391', 'num_input_tokens_seen': 246857984, 'train_runtime': '5.387e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76300/225000 [149:38:31<126:19:09,  3.06s/it]
 34%|███▍      | 76301/225000 [149:38:38<170:28:39,  4.13s/it]
 34%|███▍      | 76302/225000 [149:38:43<186:43:44,  4.52s/it]
 34%|███▍      | 76303/225000 [149:38:46<164:31:41,  3.98s/it]
 34%|███▍      | 76304/225000 [149:38:50<159:44:00,  3.87s/it]
 34%|███▍      | 76305/225000 [149:38:53<153:04:07,  3.71s/it]
 34%|███▍      | 76306/225000 [149:38:56<146:40:08,  3.55s/it]
 34%|███▍      | 76307/225000 [149:38:59<139:50:28,  3.39s/it]
 34%|███▍      | 76308/225000 [149:39:02<132:21:43,  3.20s/it]
 34%|███▍      | 76309/225000 [149:39:08<160:51:36,  3.89s/it]
 34%|███▍      | 76310/225000 [149:39:16<213:52:21,  5.18s/it]
 34%|███▍      | 76311/225000 [149:39:19<195:19:22,  4.73s/it]
 34%|███▍      | 76312/225000 [149:39:22<174:12:29,  4.22s/it]
 34%|███▍      | 76313/225000 [149:39:26<170:59:25,  4.14s/it]
 34%|███▍      | 76314/225000 [149:39:29<154:32:05,  3.74s/it]
 34%|███▍      | 76315/225000 [149:39:32<140:58:53,  3.41s/it]
 34%|███▍      | 76316/225000 [149:39:35<137:11:45,  3.32s/it]
 34%|███▍      | 76317/225000 [149:39:38<126:56:46,  3.07s/it]
 34%|███▍      | 76318/225000 [149:39:46<192:15:00,  4.65s/it]
 34%|███▍      | 76319/225000 [149:39:51<200:54:52,  4.86s/it]
 34%|███▍      | 76320/225000 [149:39:54<175:58:16,  4.26s/it]
 34%|███▍      | 76321/225000 [149:39:57<157:01:59,  3.80s/it]
 34%|███▍      | 76322/225000 [149:39:59<142:06:28,  3.44s/it]
 34%|███▍      | 76323/225000 [149:40:02<136:35:56,  3.31s/it]
 34%|███▍      | 76324/225000 [149:40:05<133:27:08,  3.23s/it]
 34%|███▍      | 76325/225000 [149:40:09<131:35:05,  3.19s/it]
 34%|███▍      | 76326/225000 [149:40:16<190:58:56,  4.62s/it]
 34%|███▍      | 76327/225000 [149:40:23<214:45:44,  5.20s/it]
 34%|███▍      | 76328/225000 [149:40:26<185:53:23,  4.50s/it]
 34%|███▍      | 76329/225000 [149:40:28<160:11:10,  3.88s/it]
 34%|███▍      | 76330/225000 [149:40:31<145:24:34,  3.52s/it]
 34%|███▍      | 76331/225000 [149:40:35<152:21:03,  3.69s/it]
 34%|███▍      | 76332/225000 [149:40:44<212:10:42,  5.14s/it]
 34%|███▍      | 76333/225000 [149:40:47<186:10:34,  4.51s/it]
 34%|███▍      | 76334/225000 [149:40:49<160:36:50,  3.89s/it]
 34%|███▍      | 76335/225000 [149:40:52<150:23:11,  3.64s/it]
 34%|███▍      | 76336/225000 [149:40:59<190:15:02,  4.61s/it]
 34%|███▍      | 76337/225000 [149:41:02<174:51:27,  4.23s/it]
 34%|███▍      | 76338/225000 [149:41:05<151:07:52,  3.66s/it]
 34%|███▍      | 76339/225000 [149:41:13<203:35:01,  4.93s/it]
 34%|███▍      | 76340/225000 [149:41:16<180:18:44,  4.37s/it]
 34%|███▍      | 76341/225000 [149:41:18<158:20:34,  3.83s/it]
 34%|███▍      | 76342/225000 [149:41:21<142:20:40,  3.45s/it]
 34%|███▍      | 76343/225000 [149:41:27<181:39:57,  4.40s/it]
 34%|███▍      | 76344/225000 [149:41:33<193:14:35,  4.68s/it]
 34%|███▍      | 76345/225000 [149:41:35<163:46:53,  3.97s/it]
 34%|███▍      | 76346/225000 [149:41:40<170:08:29,  4.12s/it]
 34%|███▍      | 76347/225000 [149:41:46<200:57:12,  4.87s/it]
 34%|███▍      | 76348/225000 [149:41:49<170:02:44,  4.12s/it]
 34%|███▍      | 76349/225000 [149:41:54<181:14:53,  4.39s/it]
 34%|███▍      | 76350/225000 [149:41:56<160:39:45,  3.89s/it]
                                                              
{'loss': '0.8645', 'grad_norm': '6.121', 'learning_rate': '3.915e-05', 'epoch': '0.3393', 'num_input_tokens_seen': 246940728, 'train_runtime': '5.389e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76350/225000 [149:41:56<160:39:45,  3.89s/it]
 34%|███▍      | 76351/225000 [149:41:59<144:28:38,  3.50s/it]
 34%|███▍      | 76352/225000 [149:42:02<142:42:22,  3.46s/it]
 34%|███▍      | 76353/225000 [149:42:11<207:46:54,  5.03s/it]
 34%|███▍      | 76354/225000 [149:42:14<189:13:35,  4.58s/it]
 34%|███▍      | 76355/225000 [149:42:17<168:42:06,  4.09s/it]
 34%|███▍      | 76356/225000 [149:42:21<157:50:41,  3.82s/it]
 34%|███▍      | 76357/225000 [149:42:24<150:58:47,  3.66s/it]
 34%|███▍      | 76358/225000 [149:42:28<151:22:44,  3.67s/it]
 34%|███▍      | 76359/225000 [149:42:31<143:14:51,  3.47s/it]
 34%|███▍      | 76360/225000 [149:42:35<153:17:30,  3.71s/it]
 34%|███▍      | 76361/225000 [149:42:44<218:23:18,  5.29s/it]
 34%|███▍      | 76362/225000 [149:42:48<200:05:04,  4.85s/it]
 34%|███▍      | 76363/225000 [149:42:50<172:46:47,  4.18s/it]
 34%|███▍      | 76364/225000 [149:42:53<153:49:20,  3.73s/it]
 34%|███▍      | 76365/225000 [149:42:56<145:15:48,  3.52s/it]
 34%|███▍      | 76366/225000 [149:42:58<131:49:26,  3.19s/it]
 34%|███▍      | 76367/225000 [149:43:07<198:40:48,  4.81s/it]
 34%|███▍      | 76368/225000 [149:43:11<194:53:38,  4.72s/it]
 34%|███▍      | 76369/225000 [149:43:14<173:10:15,  4.19s/it]
 34%|███▍      | 76370/225000 [149:43:17<153:53:51,  3.73s/it]
 34%|███▍      | 76371/225000 [149:43:24<188:09:02,  4.56s/it]
 34%|███▍      | 76372/225000 [149:43:28<185:52:39,  4.50s/it]
 34%|███▍      | 76373/225000 [149:43:30<157:10:52,  3.81s/it]
 34%|███▍      | 76374/225000 [149:43:33<139:59:03,  3.39s/it]
 34%|███▍      | 76375/225000 [149:43:40<195:58:10,  4.75s/it]
 34%|███▍      | 76376/225000 [149:43:43<171:37:02,  4.16s/it]
 34%|███▍      | 76377/225000 [149:43:48<176:30:07,  4.28s/it]
 34%|███▍      | 76378/225000 [149:43:51<159:39:20,  3.87s/it]
 34%|███▍      | 76379/225000 [149:43:54<150:14:52,  3.64s/it]
 34%|███▍      | 76380/225000 [149:43:57<139:36:31,  3.38s/it]
 34%|███▍      | 76381/225000 [149:43:59<128:59:37,  3.12s/it]
 34%|███▍      | 76382/225000 [149:44:02<126:06:14,  3.05s/it]
 34%|███▍      | 76383/225000 [149:44:05<121:25:18,  2.94s/it]
 34%|███▍      | 76384/225000 [149:44:12<171:36:27,  4.16s/it]
 34%|███▍      | 76385/225000 [149:44:16<179:40:24,  4.35s/it]
 34%|███▍      | 76386/225000 [149:44:19<160:07:23,  3.88s/it]
 34%|███▍      | 76387/225000 [149:44:22<149:09:24,  3.61s/it]
 34%|███▍      | 76388/225000 [149:44:25<139:54:35,  3.39s/it]
 34%|███▍      | 76389/225000 [149:44:28<129:18:39,  3.13s/it]
 34%|███▍      | 76390/225000 [149:44:36<188:47:50,  4.57s/it]
 34%|███▍      | 76391/225000 [149:44:41<194:44:49,  4.72s/it]
 34%|███▍      | 76392/225000 [149:44:43<166:15:54,  4.03s/it]
 34%|███▍      | 76393/225000 [149:44:45<146:09:46,  3.54s/it]
 34%|███▍      | 76394/225000 [149:44:49<143:15:00,  3.47s/it]
 34%|███▍      | 76395/225000 [149:44:54<163:18:01,  3.96s/it]
 34%|███▍      | 76396/225000 [149:44:59<183:06:14,  4.44s/it]
 34%|███▍      | 76397/225000 [149:45:02<157:32:03,  3.82s/it]
 34%|███▍      | 76398/225000 [149:45:05<147:14:09,  3.57s/it]
 34%|███▍      | 76399/225000 [149:45:08<144:33:03,  3.50s/it]
 34%|███▍      | 76400/225000 [149:45:11<142:46:59,  3.46s/it]
                                                              
{'loss': '0.8364', 'grad_norm': '2.4', 'learning_rate': '3.914e-05', 'epoch': '0.3396', 'num_input_tokens_seen': 247023128, 'train_runtime': '5.391e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76400/225000 [149:45:11<142:46:59,  3.46s/it]
 34%|███▍      | 76401/225000 [149:45:15<138:13:01,  3.35s/it]
 34%|███▍      | 76402/225000 [149:45:18<134:51:53,  3.27s/it]
 34%|███▍      | 76403/225000 [149:45:21<140:26:03,  3.40s/it]
 34%|███▍      | 76404/225000 [149:45:25<139:53:22,  3.39s/it]
 34%|███▍      | 76405/225000 [149:45:28<133:25:25,  3.23s/it]
 34%|███▍      | 76406/225000 [149:45:31<135:13:20,  3.28s/it]
 34%|███▍      | 76407/225000 [149:45:35<140:20:06,  3.40s/it]
 34%|███▍      | 76408/225000 [149:45:43<197:58:15,  4.80s/it]
 34%|███▍      | 76409/225000 [149:45:52<254:05:30,  6.16s/it]
 34%|███▍      | 76410/225000 [149:45:55<209:15:42,  5.07s/it]
 34%|███▍      | 76411/225000 [149:45:57<182:30:07,  4.42s/it]
 34%|███▍      | 76412/225000 [149:46:00<158:43:56,  3.85s/it]
 34%|███▍      | 76413/225000 [149:46:03<149:50:13,  3.63s/it]
 34%|███▍      | 76414/225000 [149:46:06<137:43:00,  3.34s/it]
 34%|███▍      | 76415/225000 [149:46:08<128:09:40,  3.11s/it]
 34%|███▍      | 76416/225000 [149:46:11<119:14:43,  2.89s/it]
 34%|███▍      | 76417/225000 [149:46:19<185:04:35,  4.48s/it]
 34%|███▍      | 76418/225000 [149:46:25<201:13:49,  4.88s/it]
 34%|███▍      | 76419/225000 [149:46:27<169:59:14,  4.12s/it]
 34%|███▍      | 76420/225000 [149:46:29<148:53:41,  3.61s/it]
 34%|███▍      | 76421/225000 [149:46:36<188:35:36,  4.57s/it]
 34%|███▍      | 76422/225000 [149:46:39<166:14:32,  4.03s/it]
 34%|███▍      | 76423/225000 [149:46:42<156:32:57,  3.79s/it]
 34%|███▍      | 76424/225000 [149:46:45<145:00:14,  3.51s/it]
 34%|███▍      | 76425/225000 [149:46:48<140:00:38,  3.39s/it]
 34%|███▍      | 76426/225000 [149:46:51<137:47:50,  3.34s/it]
 34%|███▍      | 76427/225000 [149:46:55<137:07:01,  3.32s/it]
 34%|███▍      | 76428/225000 [149:46:58<140:46:26,  3.41s/it]
 34%|███▍      | 76429/225000 [149:47:02<148:27:49,  3.60s/it]
 34%|███▍      | 76430/225000 [149:47:06<144:31:29,  3.50s/it]
 34%|███▍      | 76431/225000 [149:47:09<137:15:13,  3.33s/it]
 34%|███▍      | 76432/225000 [149:47:12<138:05:20,  3.35s/it]
 34%|███▍      | 76433/225000 [149:47:15<134:24:06,  3.26s/it]
 34%|███▍      | 76434/225000 [149:47:18<136:16:43,  3.30s/it]
 34%|███▍      | 76435/225000 [149:47:27<195:24:43,  4.74s/it]
 34%|███▍      | 76436/225000 [149:47:33<220:38:07,  5.35s/it]
 34%|███▍      | 76437/225000 [149:47:36<186:34:19,  4.52s/it]
 34%|███▍      | 76438/225000 [149:47:39<172:45:20,  4.19s/it]
 34%|███▍      | 76439/225000 [149:47:42<157:45:14,  3.82s/it]
 34%|███▍      | 76440/225000 [149:47:45<142:28:24,  3.45s/it]
 34%|███▍      | 76441/225000 [149:47:48<143:33:01,  3.48s/it]
 34%|███▍      | 76442/225000 [149:47:51<136:30:08,  3.31s/it]
 34%|███▍      | 76443/225000 [149:47:55<135:20:26,  3.28s/it]
 34%|███▍      | 76444/225000 [149:48:00<166:01:28,  4.02s/it]
 34%|███▍      | 76445/225000 [149:48:08<216:52:13,  5.26s/it]
 34%|███▍      | 76446/225000 [149:48:14<217:01:45,  5.26s/it]
 34%|███▍      | 76447/225000 [149:48:16<183:29:53,  4.45s/it]
 34%|███▍      | 76448/225000 [149:48:19<164:00:37,  3.97s/it]
 34%|███▍      | 76449/225000 [149:48:22<144:17:01,  3.50s/it]
 34%|███▍      | 76450/225000 [149:48:31<218:51:22,  5.30s/it]
                                                              
{'loss': '0.8545', 'grad_norm': '1.949', 'learning_rate': '3.912e-05', 'epoch': '0.3398', 'num_input_tokens_seen': 247102616, 'train_runtime': '5.393e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76450/225000 [149:48:31<218:51:22,  5.30s/it]
 34%|███▍      | 76451/225000 [149:48:34<184:55:54,  4.48s/it]
 34%|███▍      | 76452/225000 [149:48:38<187:28:47,  4.54s/it]
 34%|███▍      | 76453/225000 [149:48:42<172:33:11,  4.18s/it]
 34%|███▍      | 76454/225000 [149:48:45<156:24:57,  3.79s/it]
 34%|███▍      | 76455/225000 [149:48:48<146:56:56,  3.56s/it]
 34%|███▍      | 76456/225000 [149:48:51<141:58:22,  3.44s/it]
 34%|███▍      | 76457/225000 [149:48:53<133:28:16,  3.23s/it]
 34%|███▍      | 76458/225000 [149:48:57<132:12:23,  3.20s/it]
 34%|███▍      | 76459/225000 [149:49:00<133:04:57,  3.23s/it]
 34%|███▍      | 76460/225000 [149:49:02<125:37:46,  3.04s/it]
 34%|███▍      | 76461/225000 [149:49:10<176:49:09,  4.29s/it]
 34%|███▍      | 76462/225000 [149:49:16<203:20:33,  4.93s/it]
 34%|███▍      | 76463/225000 [149:49:20<190:05:44,  4.61s/it]
 34%|███▍      | 76464/225000 [149:49:23<175:05:11,  4.24s/it]
 34%|███▍      | 76465/225000 [149:49:26<161:42:26,  3.92s/it]
 34%|███▍      | 76466/225000 [149:49:30<155:41:06,  3.77s/it]
 34%|███▍      | 76467/225000 [149:49:33<146:05:16,  3.54s/it]
 34%|███▍      | 76468/225000 [149:49:36<140:55:20,  3.42s/it]
 34%|███▍      | 76469/225000 [149:49:39<141:00:21,  3.42s/it]
 34%|███▍      | 76470/225000 [149:49:43<139:16:30,  3.38s/it]
 34%|███▍      | 76471/225000 [149:49:46<133:51:35,  3.24s/it]
 34%|███▍      | 76472/225000 [149:49:49<128:41:14,  3.12s/it]
 34%|███▍      | 76473/225000 [149:49:55<171:12:15,  4.15s/it]
 34%|███▍      | 76474/225000 [149:50:03<224:02:13,  5.43s/it]
 34%|███▍      | 76475/225000 [149:50:07<205:04:49,  4.97s/it]
 34%|███▍      | 76476/225000 [149:50:10<175:37:02,  4.26s/it]
 34%|███▍      | 76477/225000 [149:50:13<161:23:44,  3.91s/it]
 34%|███▍      | 76478/225000 [149:50:18<172:37:44,  4.18s/it]
 34%|███▍      | 76479/225000 [149:50:26<223:31:26,  5.42s/it]
 34%|███▍      | 76480/225000 [149:50:28<184:24:42,  4.47s/it]
 34%|███▍      | 76481/225000 [149:50:31<159:29:16,  3.87s/it]
 34%|███▍      | 76482/225000 [149:50:33<142:20:42,  3.45s/it]
 34%|███▍      | 76483/225000 [149:50:36<134:48:57,  3.27s/it]
 34%|███▍      | 76484/225000 [149:50:46<209:05:35,  5.07s/it]
 34%|███▍      | 76485/225000 [149:50:50<202:11:09,  4.90s/it]
 34%|███▍      | 76486/225000 [149:50:53<172:55:03,  4.19s/it]
 34%|███▍      | 76487/225000 [149:50:55<154:46:14,  3.75s/it]
 34%|███▍      | 76488/225000 [149:50:58<145:40:36,  3.53s/it]
 34%|███▍      | 76489/225000 [149:51:01<140:09:35,  3.40s/it]
 34%|███▍      | 76490/225000 [149:51:04<133:32:05,  3.24s/it]
 34%|███▍      | 76491/225000 [149:51:11<179:10:24,  4.34s/it]
 34%|███▍      | 76492/225000 [149:51:16<185:50:12,  4.50s/it]
 34%|███▍      | 76493/225000 [149:51:19<172:05:11,  4.17s/it]
 34%|███▍      | 76494/225000 [149:51:23<158:46:41,  3.85s/it]
 34%|███▍      | 76495/225000 [149:51:26<149:56:21,  3.63s/it]
 34%|███▍      | 76496/225000 [149:51:28<139:08:55,  3.37s/it]
 34%|███▍      | 76497/225000 [149:51:31<135:14:16,  3.28s/it]
 34%|███▍      | 76498/225000 [149:51:35<133:28:18,  3.24s/it]
 34%|███▍      | 76499/225000 [149:51:38<130:31:58,  3.16s/it]
 34%|███▍      | 76500/225000 [149:51:42<139:33:39,  3.38s/it]
                                                              
{'loss': '0.8359', 'grad_norm': '3.662', 'learning_rate': '3.911e-05', 'epoch': '0.34', 'num_input_tokens_seen': 247185240, 'train_runtime': '5.395e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76500/225000 [149:51:42<139:33:39,  3.38s/it]
 34%|███▍      | 76501/225000 [149:51:49<185:57:15,  4.51s/it]
 34%|███▍      | 76502/225000 [149:51:57<237:35:29,  5.76s/it]
 34%|███▍      | 76503/225000 [149:52:00<197:31:07,  4.79s/it]
 34%|███▍      | 76504/225000 [149:52:03<173:26:00,  4.20s/it]
 34%|███▍      | 76505/225000 [149:52:06<160:59:45,  3.90s/it]
 34%|███▍      | 76506/225000 [149:52:09<149:57:05,  3.64s/it]
 34%|███▍      | 76507/225000 [149:52:12<139:10:50,  3.37s/it]
 34%|███▍      | 76508/225000 [149:52:15<141:38:29,  3.43s/it]
 34%|███▍      | 76509/225000 [149:52:25<218:42:27,  5.30s/it]
 34%|███▍      | 76510/225000 [149:52:29<208:36:06,  5.06s/it]
 34%|███▍      | 76511/225000 [149:52:32<175:17:03,  4.25s/it]
 34%|███▍      | 76512/225000 [149:52:38<195:46:34,  4.75s/it]
 34%|███▍      | 76513/225000 [149:52:43<197:45:07,  4.79s/it]
 34%|███▍      | 76514/225000 [149:52:45<166:21:34,  4.03s/it]
 34%|███▍      | 76515/225000 [149:52:47<147:31:48,  3.58s/it]
 34%|███▍      | 76516/225000 [149:52:53<171:55:38,  4.17s/it]
 34%|███▍      | 76517/225000 [149:52:56<161:35:31,  3.92s/it]
 34%|███▍      | 76518/225000 [149:52:59<148:14:25,  3.59s/it]
 34%|███▍      | 76519/225000 [149:53:02<137:24:44,  3.33s/it]
 34%|███▍      | 76520/225000 [149:53:05<137:14:23,  3.33s/it]
 34%|███▍      | 76521/225000 [149:53:08<131:28:07,  3.19s/it]
 34%|███▍      | 76522/225000 [149:53:11<126:26:49,  3.07s/it]
 34%|███▍      | 76523/225000 [149:53:14<128:06:21,  3.11s/it]
 34%|███▍      | 76524/225000 [149:53:23<199:19:52,  4.83s/it]
 34%|███▍      | 76525/225000 [149:53:29<211:20:06,  5.12s/it]
 34%|███▍      | 76526/225000 [149:53:31<178:44:45,  4.33s/it]
 34%|███▍      | 76527/225000 [149:53:33<154:20:00,  3.74s/it]
 34%|███▍      | 76528/225000 [149:53:36<140:32:20,  3.41s/it]
 34%|███▍      | 76529/225000 [149:53:39<132:59:47,  3.22s/it]
 34%|███▍      | 76530/225000 [149:53:42<130:37:59,  3.17s/it]
 34%|███▍      | 76531/225000 [149:53:50<196:25:53,  4.76s/it]
 34%|███▍      | 76532/225000 [149:53:57<218:01:59,  5.29s/it]
 34%|███▍      | 76533/225000 [149:53:59<179:57:19,  4.36s/it]
 34%|███▍      | 76534/225000 [149:54:02<162:01:11,  3.93s/it]
 34%|███▍      | 76535/225000 [149:54:05<144:47:44,  3.51s/it]
 34%|███▍      | 76536/225000 [149:54:09<157:09:16,  3.81s/it]
 34%|███▍      | 76537/225000 [149:54:15<179:31:14,  4.35s/it]
 34%|███▍      | 76538/225000 [149:54:19<173:31:22,  4.21s/it]
 34%|███▍      | 76539/225000 [149:54:25<204:03:45,  4.95s/it]
 34%|███▍      | 76540/225000 [149:54:28<173:43:26,  4.21s/it]
 34%|███▍      | 76541/225000 [149:54:30<155:03:50,  3.76s/it]
 34%|███▍      | 76542/225000 [149:54:34<151:24:35,  3.67s/it]
 34%|███▍      | 76543/225000 [149:54:39<171:27:16,  4.16s/it]
 34%|███▍      | 76544/225000 [149:54:45<194:56:55,  4.73s/it]
 34%|███▍      | 76545/225000 [149:54:48<167:07:50,  4.05s/it]
 34%|███▍      | 76546/225000 [149:54:54<191:10:16,  4.64s/it]
 34%|███▍      | 76547/225000 [149:54:58<182:16:48,  4.42s/it]
 34%|███▍      | 76548/225000 [149:55:01<165:21:31,  4.01s/it]
 34%|███▍      | 76549/225000 [149:55:05<165:44:04,  4.02s/it]
 34%|███▍      | 76550/225000 [149:55:08<159:34:01,  3.87s/it]
                                                              
{'loss': '0.8563', 'grad_norm': '2.945', 'learning_rate': '3.909e-05', 'epoch': '0.3402', 'num_input_tokens_seen': 247268168, 'train_runtime': '5.397e+05', 'train_tokens_per_second': '458.2'}

 34%|███▍      | 76550/225000 [149:55:08<159:34:01,  3.87s/it]
 34%|███▍      | 76551/225000 [149:55:12<159:43:14,  3.87s/it]
 34%|███▍      | 76552/225000 [149:55:16<154:28:42,  3.75s/it]
 34%|███▍      | 76553/225000 [149:55:24<212:20:31,  5.15s/it]
 34%|███▍      | 76554/225000 [149:55:26<176:48:44,  4.29s/it]
 34%|███▍      | 76555/225000 [149:55:29<151:49:38,  3.68s/it]
 34%|███▍      | 76556/225000 [149:55:32<146:56:37,  3.56s/it]
 34%|███▍      | 76557/225000 [149:55:35<141:16:45,  3.43s/it]
 34%|███▍      | 76558/225000 [149:55:38<135:24:56,  3.28s/it]
 34%|███▍      | 76559/225000 [149:55:41<138:31:28,  3.36s/it]
 34%|███▍      | 76560/225000 [149:55:44<133:00:53,  3.23s/it]
 34%|███▍      | 76561/225000 [149:55:47<130:50:48,  3.17s/it]
 34%|███▍      | 76562/225000 [149:55:50<127:02:15,  3.08s/it]
 34%|███▍      | 76563/225000 [149:55:54<130:02:46,  3.15s/it]
 34%|███▍      | 76564/225000 [149:55:58<139:55:27,  3.39s/it]
 34%|███▍      | 76565/225000 [149:56:08<228:11:52,  5.53s/it]
 34%|███▍      | 76566/225000 [149:56:12<206:51:44,  5.02s/it]
 34%|███▍      | 76567/225000 [149:56:14<176:52:56,  4.29s/it]
 34%|███▍      | 76568/225000 [149:56:22<211:13:43,  5.12s/it]
 34%|███▍      | 76569/225000 [149:56:24<175:25:25,  4.25s/it]
 34%|███▍      | 76570/225000 [149:56:26<153:54:33,  3.73s/it]
 34%|███▍      | 76571/225000 [149:56:29<145:01:44,  3.52s/it]
 34%|███▍      | 76572/225000 [149:56:35<170:12:38,  4.13s/it]
 34%|███▍      | 76573/225000 [149:56:39<166:16:54,  4.03s/it]
 34%|███▍      | 76574/225000 [149:56:42<159:12:59,  3.86s/it]
 34%|███▍      | 76575/225000 [149:56:45<151:32:44,  3.68s/it]
 34%|███▍      | 76576/225000 [149:56:49<146:23:25,  3.55s/it]
 34%|███▍      | 76577/225000 [149:56:56<187:30:04,  4.55s/it]
 34%|███▍      | 76578/225000 [149:56:58<166:46:59,  4.05s/it]
 34%|███▍      | 76579/225000 [149:57:02<155:56:15,  3.78s/it]
 34%|███▍      | 76580/225000 [149:57:04<140:27:36,  3.41s/it]
 34%|███▍      | 76581/225000 [149:57:07<133:28:06,  3.24s/it]
 34%|███▍      | 76582/225000 [149:57:10<131:53:26,  3.20s/it]
 34%|███▍      | 76583/225000 [149:57:14<135:07:49,  3.28s/it]
 34%|███▍      | 76584/225000 [149:57:21<192:19:54,  4.67s/it]
 34%|███▍      | 76585/225000 [149:57:26<193:33:53,  4.70s/it]
 34%|███▍      | 76586/225000 [149:57:29<170:58:47,  4.15s/it]
 34%|███▍      | 76587/225000 [149:57:32<156:42:48,  3.80s/it]
 34%|███▍      | 76588/225000 [149:57:35<140:36:12,  3.41s/it]
 34%|███▍      | 76589/225000 [149:57:37<130:02:17,  3.15s/it]
 34%|███▍      | 76590/225000 [149:57:40<127:10:37,  3.08s/it]
 34%|███▍      | 76591/225000 [149:57:49<202:43:59,  4.92s/it]
 34%|███▍      | 76592/225000 [149:57:54<205:20:00,  4.98s/it]
 34%|███▍      | 76593/225000 [149:57:57<178:35:56,  4.33s/it]
 34%|███▍      | 76594/225000 [149:58:00<160:46:35,  3.90s/it]
 34%|███▍      | 76595/225000 [149:58:03<143:22:12,  3.48s/it]
 34%|███▍      | 76596/225000 [149:58:05<133:48:13,  3.25s/it]
 34%|███▍      | 76597/225000 [149:58:11<162:35:15,  3.94s/it]
 34%|███▍      | 76598/225000 [149:58:19<210:10:05,  5.10s/it]
 34%|███▍      | 76599/225000 [149:58:22<184:33:41,  4.48s/it]
 34%|███▍      | 76600/225000 [149:58:24<160:29:52,  3.89s/it]
                                                              
{'loss': '0.8467', 'grad_norm': '3.549', 'learning_rate': '3.908e-05', 'epoch': '0.3404', 'num_input_tokens_seen': 247350864, 'train_runtime': '5.399e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76600/225000 [149:58:24<160:29:52,  3.89s/it]
 34%|███▍      | 76601/225000 [149:58:27<142:18:25,  3.45s/it]
 34%|███▍      | 76602/225000 [149:58:33<173:07:59,  4.20s/it]
 34%|███▍      | 76603/225000 [149:58:37<173:09:00,  4.20s/it]
 34%|███▍      | 76604/225000 [149:58:39<149:22:40,  3.62s/it]
 34%|███▍      | 76605/225000 [149:58:42<137:54:19,  3.35s/it]
 34%|███▍      | 76606/225000 [149:58:44<129:51:19,  3.15s/it]
 34%|███▍      | 76607/225000 [149:58:47<127:09:46,  3.08s/it]
 34%|███▍      | 76608/225000 [149:58:50<122:19:07,  2.97s/it]
 34%|███▍      | 76609/225000 [149:58:59<202:25:24,  4.91s/it]
 34%|███▍      | 76610/225000 [149:59:03<180:43:11,  4.38s/it]
 34%|███▍      | 76611/225000 [149:59:05<159:57:29,  3.88s/it]
 34%|███▍      | 76612/225000 [149:59:08<148:52:06,  3.61s/it]
 34%|███▍      | 76613/225000 [149:59:12<150:10:25,  3.64s/it]
 34%|███▍      | 76614/225000 [149:59:15<145:50:11,  3.54s/it]
 34%|███▍      | 76615/225000 [149:59:19<142:19:30,  3.45s/it]
 34%|███▍      | 76616/225000 [149:59:22<143:44:18,  3.49s/it]
 34%|███▍      | 76617/225000 [149:59:25<139:38:41,  3.39s/it]
 34%|███▍      | 76618/225000 [149:59:30<154:11:01,  3.74s/it]
 34%|███▍      | 76619/225000 [149:59:41<242:01:18,  5.87s/it]
 34%|███▍      | 76620/225000 [149:59:44<210:34:13,  5.11s/it]
 34%|███▍      | 76621/225000 [149:59:50<219:22:50,  5.32s/it]
 34%|███▍      | 76622/225000 [149:59:55<220:25:24,  5.35s/it]
 34%|███▍      | 76623/225000 [149:59:58<183:03:22,  4.44s/it]
 34%|███▍      | 76624/225000 [150:00:00<158:43:39,  3.85s/it]
 34%|███▍      | 76625/225000 [150:00:06<189:16:09,  4.59s/it]
 34%|███▍      | 76626/225000 [150:00:11<192:22:02,  4.67s/it]
 34%|███▍      | 76627/225000 [150:00:14<165:13:57,  4.01s/it]
 34%|███▍      | 76628/225000 [150:00:17<152:45:09,  3.71s/it]
 34%|███▍      | 76629/225000 [150:00:23<178:55:39,  4.34s/it]
 34%|███▍      | 76630/225000 [150:00:25<158:06:44,  3.84s/it]
 34%|███▍      | 76631/225000 [150:00:28<146:37:57,  3.56s/it]
 34%|███▍      | 76632/225000 [150:00:31<142:35:40,  3.46s/it]
 34%|███▍      | 76633/225000 [150:00:36<159:32:26,  3.87s/it]
 34%|███▍      | 76634/225000 [150:00:43<201:23:34,  4.89s/it]
 34%|███▍      | 76635/225000 [150:00:46<172:49:20,  4.19s/it]
 34%|███▍      | 76636/225000 [150:00:49<156:35:34,  3.80s/it]
 34%|███▍      | 76637/225000 [150:00:52<154:15:37,  3.74s/it]
 34%|███▍      | 76638/225000 [150:00:56<154:02:45,  3.74s/it]
 34%|███▍      | 76639/225000 [150:01:05<222:07:52,  5.39s/it]
 34%|███▍      | 76640/225000 [150:01:08<187:28:16,  4.55s/it]
 34%|███▍      | 76641/225000 [150:01:11<163:38:40,  3.97s/it]
 34%|███▍      | 76642/225000 [150:01:14<149:34:05,  3.63s/it]
 34%|███▍      | 76643/225000 [150:01:16<134:50:45,  3.27s/it]
 34%|███▍      | 76644/225000 [150:01:19<132:34:06,  3.22s/it]
 34%|███▍      | 76645/225000 [150:01:29<210:41:35,  5.11s/it]
 34%|███▍      | 76646/225000 [150:01:31<179:53:40,  4.37s/it]
 34%|███▍      | 76647/225000 [150:01:34<155:41:15,  3.78s/it]
 34%|███▍      | 76648/225000 [150:01:36<142:28:38,  3.46s/it]
 34%|███▍      | 76649/225000 [150:01:39<134:16:21,  3.26s/it]
 34%|███▍      | 76650/225000 [150:01:43<147:08:31,  3.57s/it]
                                                              
{'loss': '0.8529', 'grad_norm': '2.44', 'learning_rate': '3.906e-05', 'epoch': '0.3407', 'num_input_tokens_seen': 247434328, 'train_runtime': '5.401e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76650/225000 [150:01:43<147:08:31,  3.57s/it]
 34%|███▍      | 76651/225000 [150:01:54<229:48:25,  5.58s/it]
 34%|███▍      | 76652/225000 [150:01:56<193:24:58,  4.69s/it]
 34%|███▍      | 76653/225000 [150:01:59<167:40:38,  4.07s/it]
 34%|███▍      | 76654/225000 [150:02:02<155:46:34,  3.78s/it]
 34%|███▍      | 76655/225000 [150:02:05<143:06:54,  3.47s/it]
 34%|███▍      | 76656/225000 [150:02:13<205:47:47,  4.99s/it]
 34%|███▍      | 76657/225000 [150:02:16<180:48:04,  4.39s/it]
 34%|███▍      | 76658/225000 [150:02:20<173:06:49,  4.20s/it]
 34%|███▍      | 76659/225000 [150:02:27<211:07:44,  5.12s/it]
 34%|███▍      | 76660/225000 [150:02:30<177:26:35,  4.31s/it]
 34%|███▍      | 76661/225000 [150:02:32<153:10:25,  3.72s/it]
 34%|███▍      | 76662/225000 [150:02:35<145:36:15,  3.53s/it]
 34%|███▍      | 76663/225000 [150:02:38<140:07:59,  3.40s/it]
 34%|███▍      | 76664/225000 [150:02:41<138:01:22,  3.35s/it]
 34%|███▍      | 76665/225000 [150:02:45<145:55:11,  3.54s/it]
 34%|███▍      | 76666/225000 [150:02:49<141:24:04,  3.43s/it]
 34%|███▍      | 76667/225000 [150:02:52<139:00:44,  3.37s/it]
 34%|███▍      | 76668/225000 [150:02:56<147:24:04,  3.58s/it]
 34%|███▍      | 76669/225000 [150:03:04<200:11:40,  4.86s/it]
 34%|███▍      | 76670/225000 [150:03:10<220:18:59,  5.35s/it]
 34%|███▍      | 76671/225000 [150:03:13<193:16:42,  4.69s/it]
 34%|███▍      | 76672/225000 [150:03:16<171:13:35,  4.16s/it]
 34%|███▍      | 76673/225000 [150:03:19<153:54:22,  3.74s/it]
 34%|███▍      | 76674/225000 [150:03:22<144:59:43,  3.52s/it]
 34%|███▍      | 76675/225000 [150:03:25<142:58:14,  3.47s/it]
 34%|███▍      | 76676/225000 [150:03:29<140:05:28,  3.40s/it]
 34%|███▍      | 76677/225000 [150:03:32<138:36:01,  3.36s/it]
 34%|███▍      | 76678/225000 [150:03:36<143:34:23,  3.48s/it]
 34%|███▍      | 76679/225000 [150:03:44<201:44:47,  4.90s/it]
 34%|███▍      | 76680/225000 [150:03:50<214:36:03,  5.21s/it]
 34%|███▍      | 76681/225000 [150:03:53<186:02:24,  4.52s/it]
 34%|███▍      | 76682/225000 [150:03:56<167:36:41,  4.07s/it]
 34%|███▍      | 76683/225000 [150:03:59<160:27:50,  3.89s/it]
 34%|███▍      | 76684/225000 [150:04:04<168:15:31,  4.08s/it]
 34%|███▍      | 76685/225000 [150:04:09<187:21:10,  4.55s/it]
 34%|███▍      | 76686/225000 [150:04:13<170:53:06,  4.15s/it]
 34%|███▍      | 76687/225000 [150:04:15<151:24:21,  3.68s/it]
 34%|███▍      | 76688/225000 [150:04:19<150:08:32,  3.64s/it]
 34%|███▍      | 76689/225000 [150:04:22<140:18:41,  3.41s/it]
 34%|███▍      | 76690/225000 [150:04:25<135:05:03,  3.28s/it]
 34%|███▍      | 76691/225000 [150:04:34<209:40:25,  5.09s/it]
 34%|███▍      | 76692/225000 [150:04:39<208:01:41,  5.05s/it]
 34%|███▍      | 76693/225000 [150:04:41<173:34:41,  4.21s/it]
 34%|███▍      | 76694/225000 [150:04:47<193:14:50,  4.69s/it]
 34%|███▍      | 76695/225000 [150:04:50<169:15:15,  4.11s/it]
 34%|███▍      | 76696/225000 [150:04:53<153:05:50,  3.72s/it]
 34%|███▍      | 76697/225000 [150:04:55<141:17:21,  3.43s/it]
 34%|███▍      | 76698/225000 [150:05:00<154:14:16,  3.74s/it]
 34%|███▍      | 76699/225000 [150:05:03<144:51:10,  3.52s/it]
 34%|███▍      | 76700/225000 [150:05:06<138:13:46,  3.36s/it]
                                                              
{'loss': '0.838', 'grad_norm': '3.622', 'learning_rate': '3.905e-05', 'epoch': '0.3409', 'num_input_tokens_seen': 247519120, 'train_runtime': '5.403e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76700/225000 [150:05:06<138:13:46,  3.36s/it]
 34%|███▍      | 76701/225000 [150:05:13<190:47:35,  4.63s/it]
 34%|███▍      | 76702/225000 [150:05:20<215:30:58,  5.23s/it]
 34%|███▍      | 76703/225000 [150:05:22<180:53:45,  4.39s/it]
 34%|███▍      | 76704/225000 [150:05:25<155:27:38,  3.77s/it]
 34%|███▍      | 76705/225000 [150:05:28<145:57:27,  3.54s/it]
 34%|███▍      | 76706/225000 [150:05:31<140:17:04,  3.41s/it]
 34%|███▍      | 76707/225000 [150:05:33<129:15:56,  3.14s/it]
 34%|███▍      | 76708/225000 [150:05:37<130:23:55,  3.17s/it]
 34%|███▍      | 76709/225000 [150:05:40<127:55:09,  3.11s/it]
 34%|███▍      | 76710/225000 [150:05:48<195:46:38,  4.75s/it]
 34%|███▍      | 76711/225000 [150:05:52<185:42:43,  4.51s/it]
 34%|███▍      | 76712/225000 [150:05:55<163:17:54,  3.96s/it]
 34%|███▍      | 76713/225000 [150:05:58<151:17:49,  3.67s/it]
 34%|███▍      | 76714/225000 [150:06:01<149:04:09,  3.62s/it]
 34%|███▍      | 76715/225000 [150:06:04<142:03:02,  3.45s/it]
 34%|███▍      | 76716/225000 [150:06:07<138:40:28,  3.37s/it]
 34%|███▍      | 76717/225000 [150:06:17<212:17:44,  5.15s/it]
 34%|███▍      | 76718/225000 [150:06:22<216:36:38,  5.26s/it]
 34%|███▍      | 76719/225000 [150:06:25<183:52:34,  4.46s/it]
 34%|███▍      | 76720/225000 [150:06:28<164:38:19,  4.00s/it]
 34%|███▍      | 76721/225000 [150:06:31<156:23:21,  3.80s/it]
 34%|███▍      | 76722/225000 [150:06:35<152:48:44,  3.71s/it]
 34%|███▍      | 76723/225000 [150:06:38<143:12:26,  3.48s/it]
 34%|███▍      | 76724/225000 [150:06:43<171:30:36,  4.16s/it]
 34%|███▍      | 76725/225000 [150:06:49<185:52:58,  4.51s/it]
 34%|███▍      | 76726/225000 [150:06:52<176:04:05,  4.27s/it]
 34%|███▍      | 76727/225000 [150:06:55<159:07:24,  3.86s/it]
 34%|███▍      | 76728/225000 [150:06:58<147:29:54,  3.58s/it]
 34%|███▍      | 76729/225000 [150:07:01<134:52:59,  3.27s/it]
 34%|███▍      | 76730/225000 [150:07:09<192:46:27,  4.68s/it]
 34%|███▍      | 76731/225000 [150:07:17<239:59:51,  5.83s/it]
 34%|███▍      | 76732/225000 [150:07:20<207:25:32,  5.04s/it]
 34%|███▍      | 76733/225000 [150:07:23<176:35:44,  4.29s/it]
 34%|███▍      | 76734/225000 [150:07:26<157:34:31,  3.83s/it]
 34%|███▍      | 76735/225000 [150:07:29<154:10:37,  3.74s/it]
 34%|███▍      | 76736/225000 [150:07:32<145:59:23,  3.54s/it]
 34%|███▍      | 76737/225000 [150:07:35<140:27:22,  3.41s/it]
 34%|███▍      | 76738/225000 [150:07:43<192:03:26,  4.66s/it]
 34%|███▍      | 76739/225000 [150:07:46<176:29:29,  4.29s/it]
 34%|███▍      | 76740/225000 [150:07:50<172:33:28,  4.19s/it]
 34%|███▍      | 76741/225000 [150:07:56<186:27:25,  4.53s/it]
 34%|███▍      | 76742/225000 [150:07:59<164:55:12,  4.00s/it]
 34%|███▍      | 76743/225000 [150:08:01<148:49:46,  3.61s/it]
 34%|███▍      | 76744/225000 [150:08:04<139:10:43,  3.38s/it]
 34%|███▍      | 76745/225000 [150:08:07<132:20:02,  3.21s/it]
 34%|███▍      | 76746/225000 [150:08:11<142:22:53,  3.46s/it]
 34%|███▍      | 76747/225000 [150:08:14<142:57:46,  3.47s/it]
 34%|███▍      | 76748/225000 [150:08:22<188:12:56,  4.57s/it]
 34%|███▍      | 76749/225000 [150:08:29<222:41:09,  5.41s/it]
 34%|███▍      | 76750/225000 [150:08:32<194:44:11,  4.73s/it]
                                                              
{'loss': '0.8059', 'grad_norm': '2.199', 'learning_rate': '3.903e-05', 'epoch': '0.3411', 'num_input_tokens_seen': 247602112, 'train_runtime': '5.405e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76750/225000 [150:08:32<194:44:11,  4.73s/it]
 34%|███▍      | 76751/225000 [150:08:35<166:50:19,  4.05s/it]
 34%|███▍      | 76752/225000 [150:08:37<150:56:25,  3.67s/it]
 34%|███▍      | 76753/225000 [150:08:42<168:28:08,  4.09s/it]
 34%|███▍      | 76754/225000 [150:08:49<200:34:38,  4.87s/it]
 34%|███▍      | 76755/225000 [150:08:51<168:19:02,  4.09s/it]
 34%|███▍      | 76756/225000 [150:08:54<151:31:29,  3.68s/it]
 34%|███▍      | 76757/225000 [150:09:01<196:05:19,  4.76s/it]
 34%|███▍      | 76758/225000 [150:09:04<174:33:01,  4.24s/it]
 34%|███▍      | 76759/225000 [150:09:07<159:35:54,  3.88s/it]
 34%|███▍      | 76760/225000 [150:09:10<146:19:49,  3.55s/it]
 34%|███▍      | 76761/225000 [150:09:13<142:07:07,  3.45s/it]
 34%|███▍      | 76762/225000 [150:09:17<139:21:11,  3.38s/it]
 34%|███▍      | 76763/225000 [150:09:20<139:05:37,  3.38s/it]
 34%|███▍      | 76764/225000 [150:09:23<131:00:45,  3.18s/it]
 34%|███▍      | 76765/225000 [150:09:25<125:42:45,  3.05s/it]
 34%|███▍      | 76766/225000 [150:09:28<122:13:30,  2.97s/it]
 34%|███▍      | 76767/225000 [150:09:37<196:38:48,  4.78s/it]
 34%|███▍      | 76768/225000 [150:09:42<193:53:41,  4.71s/it]
 34%|███▍      | 76769/225000 [150:09:44<164:27:26,  3.99s/it]
 34%|███▍      | 76770/225000 [150:09:47<146:24:03,  3.56s/it]
 34%|███▍      | 76771/225000 [150:09:49<136:08:09,  3.31s/it]
 34%|███▍      | 76772/225000 [150:09:52<128:02:04,  3.11s/it]
 34%|███▍      | 76773/225000 [150:09:55<128:49:15,  3.13s/it]
 34%|███▍      | 76774/225000 [150:10:02<173:28:08,  4.21s/it]
 34%|███▍      | 76775/225000 [150:10:08<197:33:42,  4.80s/it]
 34%|███▍      | 76776/225000 [150:10:11<170:55:51,  4.15s/it]
 34%|███▍      | 76777/225000 [150:10:17<199:09:29,  4.84s/it]
 34%|███▍      | 76778/225000 [150:10:20<178:32:28,  4.34s/it]
 34%|███▍      | 76779/225000 [150:10:23<161:16:56,  3.92s/it]
 34%|███▍      | 76780/225000 [150:10:30<191:16:21,  4.65s/it]
 34%|███▍      | 76781/225000 [150:10:32<166:48:10,  4.05s/it]
 34%|███▍      | 76782/225000 [150:10:35<147:49:00,  3.59s/it]
 34%|███▍      | 76783/225000 [150:10:38<140:31:30,  3.41s/it]
 34%|███▍      | 76784/225000 [150:10:47<214:20:17,  5.21s/it]
 34%|███▍      | 76785/225000 [150:10:51<200:28:50,  4.87s/it]
 34%|███▍      | 76786/225000 [150:10:56<201:23:04,  4.89s/it]
 34%|███▍      | 76787/225000 [150:10:59<177:42:27,  4.32s/it]
 34%|███▍      | 76788/225000 [150:11:05<197:29:04,  4.80s/it]
 34%|███▍      | 76789/225000 [150:11:08<169:04:13,  4.11s/it]
 34%|███▍      | 76790/225000 [150:11:11<155:33:26,  3.78s/it]
 34%|███▍      | 76791/225000 [150:11:14<149:09:08,  3.62s/it]
 34%|███▍      | 76792/225000 [150:11:17<141:16:21,  3.43s/it]
 34%|███▍      | 76793/225000 [150:11:20<136:43:55,  3.32s/it]
 34%|███▍      | 76794/225000 [150:11:23<136:01:49,  3.30s/it]
 34%|███▍      | 76795/225000 [150:11:27<146:24:14,  3.56s/it]
 34%|███▍      | 76796/225000 [150:11:30<139:48:28,  3.40s/it]
 34%|███▍      | 76797/225000 [150:11:34<140:42:00,  3.42s/it]
 34%|███▍      | 76798/225000 [150:11:41<189:02:00,  4.59s/it]
 34%|███▍      | 76799/225000 [150:11:47<202:54:36,  4.93s/it]
 34%|███▍      | 76800/225000 [150:11:49<173:18:48,  4.21s/it]
                                                              
{'loss': '0.8726', 'grad_norm': '2.643', 'learning_rate': '3.902e-05', 'epoch': '0.3413', 'num_input_tokens_seen': 247685240, 'train_runtime': '5.407e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76800/225000 [150:11:49<173:18:48,  4.21s/it]
 34%|███▍      | 76801/225000 [150:11:52<150:51:52,  3.66s/it]
 34%|███▍      | 76802/225000 [150:11:55<142:11:12,  3.45s/it]
 34%|███▍      | 76803/225000 [150:11:58<137:17:15,  3.33s/it]
 34%|███▍      | 76804/225000 [150:12:01<132:19:12,  3.21s/it]
 34%|███▍      | 76805/225000 [150:12:04<134:03:12,  3.26s/it]
 34%|███▍      | 76806/225000 [150:12:08<140:49:14,  3.42s/it]
 34%|███▍      | 76807/225000 [150:12:11<135:08:24,  3.28s/it]
 34%|███▍      | 76808/225000 [150:12:14<137:42:35,  3.35s/it]
 34%|███▍      | 76809/225000 [150:12:23<198:46:39,  4.83s/it]
 34%|███▍      | 76810/225000 [150:12:30<232:46:52,  5.65s/it]
 34%|███▍      | 76811/225000 [150:12:33<194:26:12,  4.72s/it]
 34%|███▍      | 76812/225000 [150:12:36<176:19:06,  4.28s/it]
 34%|███▍      | 76813/225000 [150:12:40<176:53:54,  4.30s/it]
 34%|███▍      | 76814/225000 [150:12:48<218:30:46,  5.31s/it]
 34%|███▍      | 76815/225000 [150:12:51<195:21:56,  4.75s/it]
 34%|███▍      | 76816/225000 [150:12:54<173:09:31,  4.21s/it]
 34%|███▍      | 76817/225000 [150:12:57<155:40:21,  3.78s/it]
 34%|███▍      | 76818/225000 [150:13:00<147:29:18,  3.58s/it]
 34%|███▍      | 76819/225000 [150:13:03<140:29:52,  3.41s/it]
 34%|███▍      | 76820/225000 [150:13:06<134:49:26,  3.28s/it]
 34%|███▍      | 76821/225000 [150:13:09<133:13:19,  3.24s/it]
 34%|███▍      | 76822/225000 [150:13:13<134:27:50,  3.27s/it]
 34%|███▍      | 76823/225000 [150:13:18<159:51:37,  3.88s/it]
 34%|███▍      | 76824/225000 [150:13:28<238:47:35,  5.80s/it]
 34%|███▍      | 76825/225000 [150:13:32<217:07:35,  5.28s/it]
 34%|███▍      | 76826/225000 [150:13:36<191:24:36,  4.65s/it]
 34%|███▍      | 76827/225000 [150:13:39<175:16:02,  4.26s/it]
 34%|███▍      | 76828/225000 [150:13:42<155:09:09,  3.77s/it]
 34%|███▍      | 76829/225000 [150:13:45<147:31:18,  3.58s/it]
 34%|███▍      | 76830/225000 [150:13:48<144:05:08,  3.50s/it]
 34%|███▍      | 76831/225000 [150:13:51<141:07:35,  3.43s/it]
 34%|███▍      | 76832/225000 [150:13:55<141:09:30,  3.43s/it]
 34%|███▍      | 76833/225000 [150:13:58<141:49:13,  3.45s/it]
 34%|███▍      | 76834/225000 [150:14:01<135:12:34,  3.29s/it]
 34%|███▍      | 76835/225000 [150:14:04<134:46:27,  3.27s/it]
 34%|███▍      | 76836/225000 [150:14:08<140:25:47,  3.41s/it]
 34%|███▍      | 76837/225000 [150:14:13<153:09:20,  3.72s/it]
 34%|███▍      | 76838/225000 [150:14:22<226:22:21,  5.50s/it]
 34%|███▍      | 76839/225000 [150:14:26<210:09:03,  5.11s/it]
 34%|███▍      | 76840/225000 [150:14:29<180:39:50,  4.39s/it]
 34%|███▍      | 76841/225000 [150:14:32<160:41:14,  3.90s/it]
 34%|███▍      | 76842/225000 [150:14:34<141:36:03,  3.44s/it]
 34%|███▍      | 76843/225000 [150:14:40<174:26:56,  4.24s/it]
 34%|███▍      | 76844/225000 [150:14:46<186:13:14,  4.52s/it]
 34%|███▍      | 76845/225000 [150:14:48<158:17:28,  3.85s/it]
 34%|███▍      | 76846/225000 [150:14:51<145:50:28,  3.54s/it]
 34%|███▍      | 76847/225000 [150:14:54<143:12:20,  3.48s/it]
 34%|███▍      | 76848/225000 [150:14:57<142:00:32,  3.45s/it]
 34%|███▍      | 76849/225000 [150:15:00<137:02:29,  3.33s/it]
 34%|███▍      | 76850/225000 [150:15:04<138:36:00,  3.37s/it]
                                                              
{'loss': '0.849', 'grad_norm': '2.605', 'learning_rate': '3.9e-05', 'epoch': '0.3416', 'num_input_tokens_seen': 247768000, 'train_runtime': '5.409e+05', 'train_tokens_per_second': '458.1'}

 34%|███▍      | 76850/225000 [150:15:04<138:36:00,  3.37s/it]
 34%|███▍      | 76851/225000 [150:15:11<187:52:47,  4.57s/it]
 34%|███▍      | 76852/225000 [150:15:15<178:48:43,  4.35s/it]
 34%|███▍      | 76853/225000 [150:15:18<161:48:15,  3.93s/it]
 34%|███▍      | 76854/225000 [150:15:22<158:16:50,  3.85s/it]
 34%|███▍      | 76855/225000 [150:15:25<152:27:11,  3.70s/it]
 34%|███▍      | 76856/225000 [150:15:28<146:25:56,  3.56s/it]
 34%|███▍      | 76857/225000 [150:15:32<143:03:00,  3.48s/it]
 34%|███▍      | 76858/225000 [150:15:39<187:29:23,  4.56s/it]
 34%|███▍      | 76859/225000 [150:15:45<211:05:54,  5.13s/it]
 34%|███▍      | 76860/225000 [150:15:48<186:20:07,  4.53s/it]
 34%|███▍      | 76861/225000 [150:15:51<169:46:54,  4.13s/it]
 34%|███▍      | 76862/225000 [150:15:55<158:38:58,  3.86s/it]
 34%|███▍      | 76863/225000 [150:15:58<147:56:12,  3.60s/it]
 34%|███▍      | 76864/225000 [150:16:01<147:47:29,  3.59s/it]
 34%|███▍      | 76865/225000 [150:16:05<145:34:34,  3.54s/it]
 34%|███▍      | 76866/225000 [150:16:11<180:28:03,  4.39s/it]
 34%|███▍      | 76867/225000 [150:16:15<181:43:24,  4.42s/it]
 34%|███▍      | 76868/225000 [150:16:18<159:55:25,  3.89s/it]
 34%|███▍      | 76869/225000 [150:16:21<151:06:56,  3.67s/it]
 34%|███▍      | 76870/225000 [150:16:24<138:54:53,  3.38s/it]
 34%|███▍      | 76871/225000 [150:16:27<130:01:27,  3.16s/it]
 34%|███▍      | 76872/225000 [150:16:32<154:31:52,  3.76s/it]
 34%|███▍      | 76873/225000 [150:16:41<216:49:51,  5.27s/it]
 34%|███▍      | 76874/225000 [150:16:43<185:39:12,  4.51s/it]
 34%|███▍      | 76875/225000 [150:16:46<162:11:59,  3.94s/it]
 34%|███▍      | 76876/225000 [150:16:50<159:27:13,  3.88s/it]
 34%|███▍      | 76877/225000 [150:16:55<175:58:10,  4.28s/it]
 34%|███▍      | 76878/225000 [150:16:59<170:35:03,  4.15s/it]
 34%|███▍      | 76879/225000 [150:17:02<162:54:51,  3.96s/it]
 34%|███▍      | 76880/225000 [150:17:05<151:53:41,  3.69s/it]
 34%|███▍      | 76881/225000 [150:17:08<141:18:20,  3.43s/it]
 34%|███▍      | 76882/225000 [150:17:12<143:00:20,  3.48s/it]
 34%|███▍      | 76883/225000 [150:17:15<143:23:15,  3.49s/it]
 34%|███▍      | 76884/225000 [150:17:18<140:39:57,  3.42s/it]
 34%|███▍      | 76885/225000 [150:17:25<182:10:28,  4.43s/it]
 34%|███▍      | 76886/225000 [150:17:33<227:29:24,  5.53s/it]
 34%|███▍      | 76887/225000 [150:17:38<221:53:08,  5.39s/it]
 34%|███▍      | 76888/225000 [150:17:42<196:35:19,  4.78s/it]
 34%|███▍      | 76889/225000 [150:17:45<173:39:45,  4.22s/it]
 34%|███▍      | 76890/225000 [150:17:48<157:51:08,  3.84s/it]
 34%|███▍      | 76891/225000 [150:17:51<156:31:18,  3.80s/it]
 34%|███▍      | 76892/225000 [150:17:54<146:08:50,  3.55s/it]
 34%|███▍      | 76893/225000 [150:17:58<142:59:18,  3.48s/it]
 34%|███▍      | 76894/225000 [150:18:02<158:43:22,  3.86s/it]
 34%|███▍      | 76895/225000 [150:18:11<216:51:56,  5.27s/it]
 34%|███▍      | 76896/225000 [150:18:14<192:51:07,  4.69s/it]
 34%|███▍      | 76897/225000 [150:18:17<170:31:11,  4.14s/it]
 34%|███▍      | 76898/225000 [150:18:21<164:44:48,  4.00s/it]
 34%|███▍      | 76899/225000 [150:18:24<156:56:25,  3.81s/it]
 34%|███▍      | 76900/225000 [150:18:27<147:50:26,  3.59s/it]
                                                              
{'loss': '0.8302', 'grad_norm': '2.887', 'learning_rate': '3.899e-05', 'epoch': '0.3418', 'num_input_tokens_seen': 247851784, 'train_runtime': '5.411e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 76900/225000 [150:18:27<147:50:26,  3.59s/it]
 34%|███▍      | 76901/225000 [150:18:30<142:15:29,  3.46s/it]
 34%|███▍      | 76902/225000 [150:18:34<141:24:41,  3.44s/it]
 34%|███▍      | 76903/225000 [150:18:37<141:11:29,  3.43s/it]
 34%|███▍      | 76904/225000 [150:18:41<141:14:50,  3.43s/it]
 34%|███▍      | 76905/225000 [150:18:44<135:38:44,  3.30s/it]
 34%|███▍      | 76906/225000 [150:18:47<131:06:50,  3.19s/it]
 34%|███▍      | 76907/225000 [150:18:50<128:44:13,  3.13s/it]
 34%|███▍      | 76908/225000 [150:18:59<204:17:05,  4.97s/it]
 34%|███▍      | 76909/225000 [150:19:05<219:02:30,  5.32s/it]
 34%|███▍      | 76910/225000 [150:19:07<183:03:51,  4.45s/it]
 34%|███▍      | 76911/225000 [150:19:11<176:44:51,  4.30s/it]
 34%|███▍      | 76912/225000 [150:19:19<213:09:53,  5.18s/it]
 34%|███▍      | 76913/225000 [150:19:22<189:07:51,  4.60s/it]
 34%|███▍      | 76914/225000 [150:19:25<166:17:02,  4.04s/it]
 34%|███▍      | 76915/225000 [150:19:27<151:02:52,  3.67s/it]
 34%|███▍      | 76916/225000 [150:19:30<144:07:37,  3.50s/it]
 34%|███▍      | 76917/225000 [150:19:33<135:57:01,  3.31s/it]
 34%|███▍      | 76918/225000 [150:19:36<133:17:13,  3.24s/it]
 34%|███▍      | 76919/225000 [150:19:40<138:00:24,  3.36s/it]
 34%|███▍      | 76920/225000 [150:19:43<127:45:39,  3.11s/it]
 34%|███▍      | 76921/225000 [150:19:46<129:32:24,  3.15s/it]
 34%|███▍      | 76922/225000 [150:19:53<176:16:07,  4.29s/it]
 34%|███▍      | 76923/225000 [150:20:00<207:23:49,  5.04s/it]
 34%|███▍      | 76924/225000 [150:20:02<174:43:11,  4.25s/it]
 34%|███▍      | 76925/225000 [150:20:04<149:58:13,  3.65s/it]
 34%|███▍      | 76926/225000 [150:20:07<137:38:33,  3.35s/it]
 34%|███▍      | 76927/225000 [150:20:12<157:53:04,  3.84s/it]
 34%|███▍      | 76928/225000 [150:20:20<209:21:19,  5.09s/it]
 34%|███▍      | 76929/225000 [150:20:23<183:56:46,  4.47s/it]
 34%|███▍      | 76930/225000 [150:20:26<163:29:16,  3.97s/it]
 34%|███▍      | 76931/225000 [150:20:28<145:53:42,  3.55s/it]
 34%|███▍      | 76932/225000 [150:20:31<136:55:53,  3.33s/it]
 34%|███▍      | 76933/225000 [150:20:36<155:28:27,  3.78s/it]
 34%|███▍      | 76934/225000 [150:20:44<204:34:27,  4.97s/it]
 34%|███▍      | 76935/225000 [150:20:46<171:16:33,  4.16s/it]
 34%|███▍      | 76936/225000 [150:20:49<151:44:20,  3.69s/it]
 34%|███▍      | 76937/225000 [150:20:51<140:17:26,  3.41s/it]
 34%|███▍      | 76938/225000 [150:20:54<130:44:59,  3.18s/it]
 34%|███▍      | 76939/225000 [150:20:57<127:37:59,  3.10s/it]
 34%|███▍      | 76940/225000 [150:21:00<125:19:08,  3.05s/it]
 34%|███▍      | 76941/225000 [150:21:03<133:14:53,  3.24s/it]
 34%|███▍      | 76942/225000 [150:21:08<143:33:46,  3.49s/it]
 34%|███▍      | 76943/225000 [150:21:15<187:03:59,  4.55s/it]
 34%|███▍      | 76944/225000 [150:21:21<209:01:36,  5.08s/it]
 34%|███▍      | 76945/225000 [150:21:24<188:32:06,  4.58s/it]
 34%|███▍      | 76946/225000 [150:21:27<168:01:37,  4.09s/it]
 34%|███▍      | 76947/225000 [150:21:30<157:36:46,  3.83s/it]
 34%|███▍      | 76948/225000 [150:21:34<158:10:09,  3.85s/it]
 34%|███▍      | 76949/225000 [150:21:42<204:07:51,  4.96s/it]
 34%|███▍      | 76950/225000 [150:21:49<230:06:51,  5.60s/it]
                                                              
{'loss': '0.8612', 'grad_norm': '2.756', 'learning_rate': '3.897e-05', 'epoch': '0.342', 'num_input_tokens_seen': 247934264, 'train_runtime': '5.413e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 76950/225000 [150:21:49<230:06:51,  5.60s/it]
 34%|███▍      | 76951/225000 [150:21:52<197:06:02,  4.79s/it]
 34%|███▍      | 76952/225000 [150:21:55<177:43:30,  4.32s/it]
 34%|███▍      | 76953/225000 [150:21:59<166:48:54,  4.06s/it]
 34%|███▍      | 76954/225000 [150:22:02<155:33:11,  3.78s/it]
 34%|███▍      | 76955/225000 [150:22:05<152:46:33,  3.72s/it]
 34%|███▍      | 76956/225000 [150:22:08<144:12:25,  3.51s/it]
 34%|███▍      | 76957/225000 [150:22:12<144:38:13,  3.52s/it]
 34%|███▍      | 76958/225000 [150:22:15<142:29:16,  3.46s/it]
 34%|███▍      | 76959/225000 [150:22:18<138:06:39,  3.36s/it]
 34%|███▍      | 76960/225000 [150:22:27<204:50:33,  4.98s/it]
 34%|███▍      | 76961/225000 [150:22:31<195:30:54,  4.75s/it]
 34%|███▍      | 76962/225000 [150:22:34<170:54:53,  4.16s/it]
 34%|███▍      | 76963/225000 [150:22:36<150:07:36,  3.65s/it]
 34%|███▍      | 76964/225000 [150:22:39<140:16:18,  3.41s/it]
 34%|███▍      | 76965/225000 [150:22:42<128:58:06,  3.14s/it]
 34%|███▍      | 76966/225000 [150:22:46<140:54:22,  3.43s/it]
 34%|███▍      | 76967/225000 [150:22:51<156:49:22,  3.81s/it]
 34%|███▍      | 76968/225000 [150:22:54<147:20:31,  3.58s/it]
 34%|███▍      | 76969/225000 [150:22:58<152:41:56,  3.71s/it]
 34%|███▍      | 76970/225000 [150:23:05<192:04:02,  4.67s/it]
 34%|███▍      | 76971/225000 [150:23:13<236:37:15,  5.75s/it]
 34%|███▍      | 76972/225000 [150:23:16<199:05:33,  4.84s/it]
 34%|███▍      | 76973/225000 [150:23:18<172:03:32,  4.18s/it]
 34%|███▍      | 76974/225000 [150:23:21<154:45:52,  3.76s/it]
 34%|███▍      | 76975/225000 [150:23:24<141:38:41,  3.44s/it]
 34%|███▍      | 76976/225000 [150:23:31<191:49:58,  4.67s/it]
 34%|███▍      | 76977/225000 [150:23:34<172:37:44,  4.20s/it]
 34%|███▍      | 76978/225000 [150:23:38<163:26:51,  3.98s/it]
 34%|███▍      | 76979/225000 [150:23:42<159:43:41,  3.88s/it]
 34%|███▍      | 76980/225000 [150:23:45<155:24:40,  3.78s/it]
 34%|███▍      | 76981/225000 [150:23:48<151:18:43,  3.68s/it]
 34%|███▍      | 76982/225000 [150:23:54<167:59:00,  4.09s/it]
 34%|███▍      | 76983/225000 [150:23:57<156:03:38,  3.80s/it]
 34%|███▍      | 76984/225000 [150:24:00<150:53:52,  3.67s/it]
 34%|███▍      | 76985/225000 [150:24:04<148:56:34,  3.62s/it]
 34%|███▍      | 76986/225000 [150:24:10<182:01:33,  4.43s/it]
 34%|███▍      | 76987/225000 [150:24:14<183:30:20,  4.46s/it]
 34%|███▍      | 76988/225000 [150:24:17<156:39:38,  3.81s/it]
 34%|███▍      | 76989/225000 [150:24:19<139:24:21,  3.39s/it]
 34%|███▍      | 76990/225000 [150:24:27<201:05:34,  4.89s/it]
 34%|███▍      | 76991/225000 [150:24:30<175:28:02,  4.27s/it]
 34%|███▍      | 76992/225000 [150:24:33<157:55:35,  3.84s/it]
 34%|███▍      | 76993/225000 [150:24:36<142:53:44,  3.48s/it]
 34%|███▍      | 76994/225000 [150:24:38<132:27:58,  3.22s/it]
 34%|███▍      | 76995/225000 [150:24:41<127:29:21,  3.10s/it]
 34%|███▍      | 76996/225000 [150:24:47<166:39:01,  4.05s/it]
 34%|███▍      | 76997/225000 [150:24:54<193:20:19,  4.70s/it]
 34%|███▍      | 76998/225000 [150:24:56<167:13:56,  4.07s/it]
 34%|███▍      | 76999/225000 [150:24:59<150:42:05,  3.67s/it]
 34%|███▍      | 77000/225000 [150:25:03<157:06:49,  3.82s/it]
                                                              
{'loss': '0.8192', 'grad_norm': '1.771', 'learning_rate': '3.896e-05', 'epoch': '0.3422', 'num_input_tokens_seen': 248018128, 'train_runtime': '5.415e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 77000/225000 [150:25:03<157:06:49,  3.82s/it]
 34%|███▍      | 77001/225000 [150:25:06<150:26:10,  3.66s/it]
 34%|███▍      | 77002/225000 [150:25:11<155:10:48,  3.77s/it]
 34%|███▍      | 77003/225000 [150:25:14<154:35:36,  3.76s/it]
 34%|███▍      | 77004/225000 [150:25:19<162:45:16,  3.96s/it]
 34%|███▍      | 77005/225000 [150:25:26<200:46:28,  4.88s/it]
 34%|███▍      | 77006/225000 [150:25:34<244:16:19,  5.94s/it]
 34%|███▍      | 77007/225000 [150:25:37<209:15:41,  5.09s/it]
 34%|███▍      | 77008/225000 [150:25:39<174:13:11,  4.24s/it]
 34%|███▍      | 77009/225000 [150:25:42<157:00:12,  3.82s/it]
 34%|███▍      | 77010/225000 [150:25:45<148:00:21,  3.60s/it]
 34%|███▍      | 77011/225000 [150:25:51<167:14:23,  4.07s/it]
 34%|███▍      | 77012/225000 [150:25:57<193:31:32,  4.71s/it]
 34%|███▍      | 77013/225000 [150:25:59<163:11:40,  3.97s/it]
 34%|███▍      | 77014/225000 [150:26:04<181:38:53,  4.42s/it]
 34%|███▍      | 77015/225000 [150:26:07<162:24:54,  3.95s/it]
 34%|███▍      | 77016/225000 [150:26:10<150:40:27,  3.67s/it]
 34%|███▍      | 77017/225000 [150:26:14<147:05:28,  3.58s/it]
 34%|███▍      | 77018/225000 [150:26:17<142:36:12,  3.47s/it]
 34%|███▍      | 77019/225000 [150:26:20<140:36:41,  3.42s/it]
 34%|███▍      | 77020/225000 [150:26:26<169:38:14,  4.13s/it]
 34%|███▍      | 77021/225000 [150:26:32<190:14:16,  4.63s/it]
 34%|███▍      | 77022/225000 [150:26:35<170:33:15,  4.15s/it]
 34%|███▍      | 77023/225000 [150:26:38<156:14:11,  3.80s/it]
 34%|███▍      | 77024/225000 [150:26:41<144:36:53,  3.52s/it]
 34%|███▍      | 77025/225000 [150:26:44<137:41:23,  3.35s/it]
 34%|███▍      | 77026/225000 [150:26:47<134:59:14,  3.28s/it]
 34%|███▍      | 77027/225000 [150:26:56<205:21:53,  5.00s/it]
 34%|███▍      | 77028/225000 [150:27:00<199:24:33,  4.85s/it]
 34%|███▍      | 77029/225000 [150:27:04<180:18:49,  4.39s/it]
 34%|███▍      | 77030/225000 [150:27:07<162:40:15,  3.96s/it]
 34%|███▍      | 77031/225000 [150:27:09<149:25:40,  3.64s/it]
 34%|███▍      | 77032/225000 [150:27:14<163:11:20,  3.97s/it]
 34%|███▍      | 77033/225000 [150:27:23<217:14:23,  5.29s/it]
 34%|███▍      | 77034/225000 [150:27:26<199:02:11,  4.84s/it]
 34%|███▍      | 77035/225000 [150:27:29<169:18:32,  4.12s/it]
 34%|███▍      | 77036/225000 [150:27:31<151:02:10,  3.67s/it]
 34%|███▍      | 77037/225000 [150:27:35<145:38:01,  3.54s/it]
 34%|███▍      | 77038/225000 [150:27:38<145:47:25,  3.55s/it]
 34%|███▍      | 77039/225000 [150:27:42<145:48:15,  3.55s/it]
 34%|███▍      | 77040/225000 [150:27:45<139:16:43,  3.39s/it]
 34%|███▍      | 77041/225000 [150:27:48<134:51:44,  3.28s/it]
 34%|███▍      | 77042/225000 [150:27:53<164:05:57,  3.99s/it]
 34%|███▍      | 77043/225000 [150:27:56<150:16:33,  3.66s/it]
 34%|███▍      | 77044/225000 [150:28:00<145:11:36,  3.53s/it]
 34%|███▍      | 77045/225000 [150:28:03<147:50:52,  3.60s/it]
 34%|███▍      | 77046/225000 [150:28:06<136:13:37,  3.31s/it]
 34%|███▍      | 77047/225000 [150:28:10<139:49:42,  3.40s/it]
 34%|███▍      | 77048/225000 [150:28:13<136:59:17,  3.33s/it]
 34%|███▍      | 77049/225000 [150:28:16<139:18:34,  3.39s/it]
 34%|███▍      | 77050/225000 [150:28:22<164:40:35,  4.01s/it]
                                                              
{'loss': '0.8552', 'grad_norm': '3.638', 'learning_rate': '3.894e-05', 'epoch': '0.3424', 'num_input_tokens_seen': 248100848, 'train_runtime': '5.417e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 77050/225000 [150:28:22<164:40:35,  4.01s/it]
 34%|███▍      | 77051/225000 [150:28:31<226:09:28,  5.50s/it]
 34%|███▍      | 77052/225000 [150:28:34<194:41:48,  4.74s/it]
 34%|███▍      | 77053/225000 [150:28:37<175:02:33,  4.26s/it]
 34%|███▍      | 77054/225000 [150:28:41<168:28:56,  4.10s/it]
 34%|███▍      | 77055/225000 [150:28:44<156:11:13,  3.80s/it]
 34%|███▍      | 77056/225000 [150:28:49<174:26:18,  4.24s/it]
 34%|███▍      | 77057/225000 [150:28:52<158:29:05,  3.86s/it]
 34%|███▍      | 77058/225000 [150:28:54<141:42:30,  3.45s/it]
 34%|███▍      | 77059/225000 [150:28:57<131:48:04,  3.21s/it]
 34%|███▍      | 77060/225000 [150:29:03<170:01:10,  4.14s/it]
 34%|███▍      | 77061/225000 [150:29:10<202:23:24,  4.93s/it]
 34%|███▍      | 77062/225000 [150:29:13<173:31:38,  4.22s/it]
 34%|███▍      | 77063/225000 [150:29:16<159:45:31,  3.89s/it]
 34%|███▍      | 77064/225000 [150:29:19<148:39:02,  3.62s/it]
 34%|███▍      | 77065/225000 [150:29:21<135:23:26,  3.29s/it]
 34%|███▍      | 77066/225000 [150:29:25<140:31:11,  3.42s/it]
 34%|███▍      | 77067/225000 [150:29:35<216:06:03,  5.26s/it]
 34%|███▍      | 77068/225000 [150:29:37<185:08:10,  4.51s/it]
 34%|███▍      | 77069/225000 [150:29:40<162:11:23,  3.95s/it]
 34%|███▍      | 77070/225000 [150:29:44<159:42:47,  3.89s/it]
 34%|███▍      | 77071/225000 [150:29:47<148:00:21,  3.60s/it]
 34%|███▍      | 77072/225000 [150:29:51<152:24:31,  3.71s/it]
 34%|███▍      | 77073/225000 [150:29:54<152:10:14,  3.70s/it]
 34%|███▍      | 77074/225000 [150:29:57<145:53:26,  3.55s/it]
 34%|███▍      | 77075/225000 [150:30:00<136:41:32,  3.33s/it]
 34%|███▍      | 77076/225000 [150:30:04<140:08:17,  3.41s/it]
 34%|███▍      | 77077/225000 [150:30:07<134:52:17,  3.28s/it]
 34%|███▍      | 77078/225000 [150:30:16<201:00:36,  4.89s/it]
 34%|███▍      | 77079/225000 [150:30:23<232:10:11,  5.65s/it]
 34%|███▍      | 77080/225000 [150:30:25<190:35:45,  4.64s/it]
 34%|███▍      | 77081/225000 [150:30:28<166:43:41,  4.06s/it]
 34%|███▍      | 77082/225000 [150:30:35<207:21:22,  5.05s/it]
 34%|███▍      | 77083/225000 [150:30:38<178:32:06,  4.35s/it]
 34%|███▍      | 77084/225000 [150:30:40<155:43:44,  3.79s/it]
 34%|███▍      | 77085/225000 [150:30:44<147:11:15,  3.58s/it]
 34%|███▍      | 77086/225000 [150:30:46<136:57:51,  3.33s/it]
 34%|███▍      | 77087/225000 [150:30:51<153:58:31,  3.75s/it]
 34%|███▍      | 77088/225000 [150:30:57<185:14:31,  4.51s/it]
 34%|███▍      | 77089/225000 [150:31:00<162:46:05,  3.96s/it]
 34%|███▍      | 77090/225000 [150:31:03<151:25:28,  3.69s/it]
 34%|███▍      | 77091/225000 [150:31:07<154:19:05,  3.76s/it]
 34%|███▍      | 77092/225000 [150:31:10<148:05:03,  3.60s/it]
 34%|███▍      | 77093/225000 [150:31:16<171:06:30,  4.16s/it]
 34%|███▍      | 77094/225000 [150:31:23<210:38:54,  5.13s/it]
 34%|███▍      | 77095/225000 [150:31:26<181:27:15,  4.42s/it]
 34%|███▍      | 77096/225000 [150:31:29<165:40:15,  4.03s/it]
 34%|███▍      | 77097/225000 [150:31:32<149:53:41,  3.65s/it]
 34%|███▍      | 77098/225000 [150:31:35<140:06:41,  3.41s/it]
 34%|███▍      | 77099/225000 [150:31:37<133:14:11,  3.24s/it]
 34%|███▍      | 77100/225000 [150:31:40<127:51:10,  3.11s/it]
                                                              
{'loss': '0.8145', 'grad_norm': '2.54', 'learning_rate': '3.893e-05', 'epoch': '0.3427', 'num_input_tokens_seen': 248182800, 'train_runtime': '5.419e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 77100/225000 [150:31:40<127:51:10,  3.11s/it]
 34%|███▍      | 77101/225000 [150:31:43<126:34:56,  3.08s/it]
 34%|███▍      | 77102/225000 [150:31:47<140:16:27,  3.41s/it]
 34%|███▍      | 77103/225000 [150:31:51<137:42:54,  3.35s/it]
 34%|███▍      | 77104/225000 [150:31:54<134:19:02,  3.27s/it]
 34%|███▍      | 77105/225000 [150:32:00<165:53:35,  4.04s/it]
 34%|███▍      | 77106/225000 [150:32:08<225:02:31,  5.48s/it]
 34%|███▍      | 77107/225000 [150:32:11<196:01:49,  4.77s/it]
 34%|███▍      | 77108/225000 [150:32:14<166:59:53,  4.07s/it]
 34%|███▍      | 77109/225000 [150:32:21<206:43:31,  5.03s/it]
 34%|███▍      | 77110/225000 [150:32:24<173:45:22,  4.23s/it]
 34%|███▍      | 77111/225000 [150:32:27<158:25:18,  3.86s/it]
 34%|███▍      | 77112/225000 [150:32:29<144:50:34,  3.53s/it]
 34%|███▍      | 77113/225000 [150:32:32<138:02:33,  3.36s/it]
 34%|███▍      | 77114/225000 [150:32:35<132:37:09,  3.23s/it]
 34%|███▍      | 77115/225000 [150:32:41<169:50:57,  4.13s/it]
 34%|███▍      | 77116/225000 [150:32:50<219:49:06,  5.35s/it]
 34%|███▍      | 77117/225000 [150:32:53<192:46:16,  4.69s/it]
 34%|███▍      | 77118/225000 [150:32:56<176:04:30,  4.29s/it]
 34%|███▍      | 77119/225000 [150:32:59<164:35:54,  4.01s/it]
 34%|███▍      | 77120/225000 [150:33:03<155:33:54,  3.79s/it]
 34%|███▍      | 77121/225000 [150:33:06<144:16:04,  3.51s/it]
 34%|███▍      | 77122/225000 [150:33:09<142:12:43,  3.46s/it]
 34%|███▍      | 77123/225000 [150:33:12<133:23:52,  3.25s/it]
 34%|███▍      | 77124/225000 [150:33:16<141:29:55,  3.44s/it]
 34%|███▍      | 77125/225000 [150:33:26<221:44:47,  5.40s/it]
 34%|███▍      | 77126/225000 [150:33:29<201:42:44,  4.91s/it]
 34%|███▍      | 77127/225000 [150:33:32<175:59:49,  4.28s/it]
 34%|███▍      | 77128/225000 [150:33:35<154:07:40,  3.75s/it]
 34%|███▍      | 77129/225000 [150:33:37<141:48:32,  3.45s/it]
 34%|███▍      | 77130/225000 [150:33:42<154:35:31,  3.76s/it]
 34%|███▍      | 77131/225000 [150:33:48<188:07:33,  4.58s/it]
 34%|███▍      | 77132/225000 [150:33:53<193:01:40,  4.70s/it]
 34%|███▍      | 77133/225000 [150:33:59<208:16:59,  5.07s/it]
 34%|███▍      | 77134/225000 [150:34:02<173:25:21,  4.22s/it]
 34%|███▍      | 77135/225000 [150:34:04<149:33:50,  3.64s/it]
 34%|███▍      | 77136/225000 [150:34:11<194:06:09,  4.73s/it]
 34%|███▍      | 77137/225000 [150:34:15<178:50:52,  4.35s/it]
 34%|███▍      | 77138/225000 [150:34:18<160:54:55,  3.92s/it]
 34%|███▍      | 77139/225000 [150:34:21<153:33:10,  3.74s/it]
 34%|███▍      | 77140/225000 [150:34:24<147:57:22,  3.60s/it]
 34%|███▍      | 77141/225000 [150:34:27<140:42:00,  3.43s/it]
 34%|███▍      | 77142/225000 [150:34:30<139:02:42,  3.39s/it]
 34%|███▍      | 77143/225000 [150:34:34<143:15:56,  3.49s/it]
 34%|███▍      | 77144/225000 [150:34:38<141:53:17,  3.45s/it]
 34%|███▍      | 77145/225000 [150:34:40<135:37:01,  3.30s/it]
 34%|███▍      | 77146/225000 [150:34:44<141:36:56,  3.45s/it]
 34%|███▍      | 77147/225000 [150:34:51<184:17:11,  4.49s/it]
 34%|███▍      | 77148/225000 [150:34:59<229:52:18,  5.60s/it]
 34%|███▍      | 77149/225000 [150:35:02<192:59:45,  4.70s/it]
 34%|███▍      | 77150/225000 [150:35:05<166:59:44,  4.07s/it]
                                                              
{'loss': '0.8143', 'grad_norm': '3.126', 'learning_rate': '3.891e-05', 'epoch': '0.3429', 'num_input_tokens_seen': 248266032, 'train_runtime': '5.421e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 77150/225000 [150:35:05<166:59:44,  4.07s/it]
 34%|███▍      | 77151/225000 [150:35:13<217:18:04,  5.29s/it]
 34%|███▍      | 77152/225000 [150:35:15<180:45:20,  4.40s/it]
 34%|███▍      | 77153/225000 [150:35:18<158:20:08,  3.86s/it]
 34%|███▍      | 77154/225000 [150:35:24<186:49:37,  4.55s/it]
 34%|███▍      | 77155/225000 [150:35:28<187:12:18,  4.56s/it]
 34%|███▍      | 77156/225000 [150:35:31<167:26:20,  4.08s/it]
 34%|███▍      | 77157/225000 [150:35:35<159:35:27,  3.89s/it]
 34%|███▍      | 77158/225000 [150:35:44<222:45:33,  5.42s/it]
 34%|███▍      | 77159/225000 [150:35:48<207:21:10,  5.05s/it]
 34%|███▍      | 77160/225000 [150:35:51<188:27:00,  4.59s/it]
 34%|███▍      | 77161/225000 [150:35:55<176:06:19,  4.29s/it]
 34%|███▍      | 77162/225000 [150:35:58<158:58:34,  3.87s/it]
 34%|███▍      | 77163/225000 [150:36:01<151:34:22,  3.69s/it]
 34%|███▍      | 77164/225000 [150:36:04<141:56:24,  3.46s/it]
 34%|███▍      | 77165/225000 [150:36:07<139:34:42,  3.40s/it]
 34%|███▍      | 77166/225000 [150:36:10<135:04:51,  3.29s/it]
 34%|███▍      | 77167/225000 [150:36:13<130:27:03,  3.18s/it]
 34%|███▍      | 77168/225000 [150:36:16<130:16:04,  3.17s/it]
 34%|███▍      | 77169/225000 [150:36:20<134:58:42,  3.29s/it]
 34%|███▍      | 77170/225000 [150:36:23<130:00:08,  3.17s/it]
 34%|███▍      | 77171/225000 [150:36:27<138:45:37,  3.38s/it]
 34%|███▍      | 77172/225000 [150:36:30<139:58:22,  3.41s/it]
 34%|███▍      | 77173/225000 [150:36:35<152:44:22,  3.72s/it]
 34%|███▍      | 77174/225000 [150:36:44<216:29:01,  5.27s/it]
 34%|███▍      | 77175/225000 [150:36:47<190:35:24,  4.64s/it]
 34%|███▍      | 77176/225000 [150:36:50<172:47:03,  4.21s/it]
 34%|███▍      | 77177/225000 [150:36:54<166:41:13,  4.06s/it]
 34%|███▍      | 77178/225000 [150:37:01<203:54:06,  4.97s/it]
 34%|███▍      | 77179/225000 [150:37:03<176:08:14,  4.29s/it]
 34%|███▍      | 77180/225000 [150:37:07<162:47:00,  3.96s/it]
 34%|███▍      | 77181/225000 [150:37:10<148:42:53,  3.62s/it]
 34%|███▍      | 77182/225000 [150:37:12<136:44:23,  3.33s/it]
 34%|███▍      | 77183/225000 [150:37:22<215:02:59,  5.24s/it]
 34%|███▍      | 77184/225000 [150:37:26<202:41:57,  4.94s/it]
 34%|███▍      | 77185/225000 [150:37:29<182:12:57,  4.44s/it]
 34%|███▍      | 77186/225000 [150:37:32<160:46:13,  3.92s/it]
 34%|███▍      | 77187/225000 [150:37:35<152:03:35,  3.70s/it]
 34%|███▍      | 77188/225000 [150:37:38<143:58:53,  3.51s/it]
 34%|███▍      | 77189/225000 [150:37:43<161:31:01,  3.93s/it]
 34%|███▍      | 77190/225000 [150:37:51<212:11:42,  5.17s/it]
 34%|███▍      | 77191/225000 [150:37:54<178:36:14,  4.35s/it]
 34%|███▍      | 77192/225000 [150:37:56<158:19:07,  3.86s/it]
 34%|███▍      | 77193/225000 [150:37:59<143:11:02,  3.49s/it]
 34%|███▍      | 77194/225000 [150:38:02<135:54:07,  3.31s/it]
 34%|███▍      | 77195/225000 [150:38:07<160:10:48,  3.90s/it]
 34%|███▍      | 77196/225000 [150:38:13<184:22:07,  4.49s/it]
 34%|███▍      | 77197/225000 [150:38:18<189:20:44,  4.61s/it]
 34%|███▍      | 77198/225000 [150:38:20<159:54:21,  3.89s/it]
 34%|███▍      | 77199/225000 [150:38:23<140:27:05,  3.42s/it]
 34%|███▍      | 77200/225000 [150:38:26<140:47:13,  3.43s/it]
                                                              
{'loss': '0.8437', 'grad_norm': '3.76', 'learning_rate': '3.89e-05', 'epoch': '0.3431', 'num_input_tokens_seen': 248350176, 'train_runtime': '5.423e+05', 'train_tokens_per_second': '458'}

 34%|███▍      | 77200/225000 [150:38:26<140:47:13,  3.43s/it]
 34%|███▍      | 77201/225000 [150:38:30<145:21:46,  3.54s/it]
 34%|███▍      | 77202/225000 [150:38:34<150:38:44,  3.67s/it]
 34%|███▍      | 77203/225000 [150:38:39<170:36:40,  4.16s/it]
 34%|███▍      | 77204/225000 [150:38:48<226:43:00,  5.52s/it]
 34%|███▍      | 77205/225000 [150:38:50<189:14:10,  4.61s/it]
 34%|███▍      | 77206/225000 [150:38:53<163:21:40,  3.98s/it]
 34%|███▍      | 77207/225000 [150:38:58<179:23:34,  4.37s/it]
 34%|███▍      | 77208/225000 [150:39:03<183:49:56,  4.48s/it]
 34%|███▍      | 77209/225000 [150:39:05<161:16:24,  3.93s/it]
 34%|███▍      | 77210/225000 [150:39:08<148:57:22,  3.63s/it]
 34%|███▍      | 77211/225000 [150:39:14<172:32:56,  4.20s/it]
 34%|███▍      | 77212/225000 [150:39:18<170:19:32,  4.15s/it]
 34%|███▍      | 77213/225000 [150:39:21<156:10:14,  3.80s/it]
 34%|███▍      | 77214/225000 [150:39:27<178:33:33,  4.35s/it]
 34%|███▍      | 77215/225000 [150:39:30<167:27:15,  4.08s/it]
 34%|███▍      | 77216/225000 [150:39:33<156:18:40,  3.81s/it]
 34%|███▍      | 77217/225000 [150:39:37<151:25:30,  3.69s/it]
 34%|███▍      | 77218/225000 [150:39:40<147:38:38,  3.60s/it]
 34%|███▍      | 77219/225000 [150:39:43<146:02:49,  3.56s/it]
 34%|███▍      | 77220/225000 [150:39:47<144:00:45,  3.51s/it]
 34%|███▍      | 77221/225000 [150:39:54<190:00:06,  4.63s/it]
 34%|███▍      | 77222/225000 [150:40:02<225:57:30,  5.50s/it]
 34%|███▍      | 77223/225000 [150:40:05<198:20:00,  4.83s/it]
 34%|███▍      | 77224/225000 [150:40:07<171:01:46,  4.17s/it]
 34%|███▍      | 77225/225000 [150:40:11<158:05:50,  3.85s/it]
 34%|███▍      | 77226/225000 [150:40:13<141:26:00,  3.45s/it]
 34%|███▍      | 77227/225000 [150:40:16<129:04:43,  3.14s/it]
 34%|███▍      | 77228/225000 [150:40:19<138:23:27,  3.37s/it]
 34%|███▍      | 77229/225000 [150:40:28<202:57:05,  4.94s/it]
 34%|███▍      | 77230/225000 [150:40:32<190:43:37,  4.65s/it]
 34%|███▍      | 77231/225000 [150:40:35<172:37:59,  4.21s/it]
 34%|███▍      | 77232/225000 [150:40:38<155:21:47,  3.79s/it]
 34%|███▍      | 77233/225000 [150:40:42<153:18:05,  3.73s/it]
 34%|███▍      | 77234/225000 [150:40:45<144:58:47,  3.53s/it]
 34%|███▍      | 77235/225000 [150:40:48<140:24:52,  3.42s/it]
 34%|███▍      | 77236/225000 [150:40:51<137:32:44,  3.35s/it]
 34%|███▍      | 77237/225000 [150:40:54<137:35:36,  3.35s/it]
 34%|███▍      | 77238/225000 [150:40:57<132:48:45,  3.24s/it]
 34%|███▍      | 77239/225000 [150:41:01<138:27:10,  3.37s/it]
 34%|███▍      | 77240/225000 [150:41:04<139:01:51,  3.39s/it]
 34%|███▍      | 77241/225000 [150:41:13<205:12:17,  5.00s/it]
 34%|███▍      | 77242/225000 [150:41:21<236:27:18,  5.76s/it]
 34%|███▍      | 77243/225000 [150:41:23<197:36:14,  4.81s/it]
 34%|███▍      | 77244/225000 [150:41:26<167:23:33,  4.08s/it]
 34%|███▍      | 77245/225000 [150:41:29<152:43:12,  3.72s/it]
 34%|███▍      | 77246/225000 [150:41:37<211:52:25,  5.16s/it]
 34%|███▍      | 77247/225000 [150:41:41<193:57:24,  4.73s/it]
 34%|███▍      | 77248/225000 [150:41:44<170:48:08,  4.16s/it]
 34%|███▍      | 77249/225000 [150:41:46<151:14:16,  3.68s/it]
 34%|███▍      | 77250/225000 [150:41:49<137:07:11,  3.34s/it]
                                                              
{'loss': '0.8153', 'grad_norm': '3.866', 'learning_rate': '3.888e-05', 'epoch': '0.3433', 'num_input_tokens_seen': 248435144, 'train_runtime': '5.425e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77250/225000 [150:41:49<137:07:11,  3.34s/it]
 34%|███▍      | 77251/225000 [150:41:53<152:03:28,  3.70s/it]
 34%|███▍      | 77252/225000 [150:42:01<202:31:57,  4.93s/it]
 34%|███▍      | 77253/225000 [150:42:04<175:05:53,  4.27s/it]
 34%|███▍      | 77254/225000 [150:42:07<157:54:49,  3.85s/it]
 34%|███▍      | 77255/225000 [150:42:13<187:37:54,  4.57s/it]
 34%|███▍      | 77256/225000 [150:42:16<163:35:02,  3.99s/it]
 34%|███▍      | 77257/225000 [150:42:18<147:31:00,  3.59s/it]
 34%|███▍      | 77258/225000 [150:42:23<162:08:59,  3.95s/it]
 34%|███▍      | 77259/225000 [150:42:29<192:04:53,  4.68s/it]
 34%|███▍      | 77260/225000 [150:42:33<182:17:01,  4.44s/it]
 34%|███▍      | 77261/225000 [150:42:36<165:18:50,  4.03s/it]
 34%|███▍      | 77262/225000 [150:42:43<192:25:18,  4.69s/it]
 34%|███▍      | 77263/225000 [150:42:45<168:45:45,  4.11s/it]
 34%|███▍      | 77264/225000 [150:42:48<150:17:19,  3.66s/it]
 34%|███▍      | 77265/225000 [150:42:51<138:49:26,  3.38s/it]
 34%|███▍      | 77266/225000 [150:42:54<141:33:17,  3.45s/it]
 34%|███▍      | 77267/225000 [150:43:02<189:21:30,  4.61s/it]
 34%|███▍      | 77268/225000 [150:43:04<167:00:13,  4.07s/it]
 34%|███▍      | 77269/225000 [150:43:08<158:35:23,  3.86s/it]
 34%|███▍      | 77270/225000 [150:43:11<145:46:31,  3.55s/it]
 34%|███▍      | 77271/225000 [150:43:14<138:29:50,  3.38s/it]
 34%|███▍      | 77272/225000 [150:43:18<144:38:32,  3.52s/it]
 34%|███▍      | 77273/225000 [150:43:20<135:23:35,  3.30s/it]
 34%|███▍      | 77274/225000 [150:43:24<134:53:03,  3.29s/it]
 34%|███▍      | 77275/225000 [150:43:34<219:29:15,  5.35s/it]
 34%|███▍      | 77276/225000 [150:43:38<204:24:09,  4.98s/it]
 34%|███▍      | 77277/225000 [150:43:41<178:55:33,  4.36s/it]
 34%|███▍      | 77278/225000 [150:43:43<153:47:16,  3.75s/it]
 34%|███▍      | 77279/225000 [150:43:52<211:42:16,  5.16s/it]
 34%|███▍      | 77280/225000 [150:43:56<198:05:30,  4.83s/it]
 34%|███▍      | 77281/225000 [150:43:58<165:41:17,  4.04s/it]
 34%|███▍      | 77282/225000 [150:44:03<183:31:47,  4.47s/it]
 34%|███▍      | 77283/225000 [150:44:08<181:09:59,  4.42s/it]
 34%|███▍      | 77284/225000 [150:44:13<194:05:48,  4.73s/it]
 34%|███▍      | 77285/225000 [150:44:16<174:29:22,  4.25s/it]
 34%|███▍      | 77286/225000 [150:44:19<159:16:18,  3.88s/it]
 34%|███▍      | 77287/225000 [150:44:22<143:12:21,  3.49s/it]
 34%|███▍      | 77288/225000 [150:44:24<133:25:09,  3.25s/it]
 34%|███▍      | 77289/225000 [150:44:33<202:10:37,  4.93s/it]
 34%|███▍      | 77290/225000 [150:44:36<175:54:12,  4.29s/it]
 34%|███▍      | 77291/225000 [150:44:41<185:25:05,  4.52s/it]
 34%|███▍      | 77292/225000 [150:44:47<202:01:56,  4.92s/it]
 34%|███▍      | 77293/225000 [150:44:50<172:51:26,  4.21s/it]
 34%|███▍      | 77294/225000 [150:44:52<149:01:23,  3.63s/it]
 34%|███▍      | 77295/225000 [150:44:57<167:04:00,  4.07s/it]
 34%|███▍      | 77296/225000 [150:45:03<191:42:39,  4.67s/it]
 34%|███▍      | 77297/225000 [150:45:06<168:36:48,  4.11s/it]
 34%|███▍      | 77298/225000 [150:45:09<158:16:23,  3.86s/it]
 34%|███▍      | 77299/225000 [150:45:12<144:14:03,  3.52s/it]
 34%|███▍      | 77300/225000 [150:45:14<132:16:40,  3.22s/it]
                                                              
{'loss': '0.8493', 'grad_norm': '3.662', 'learning_rate': '3.887e-05', 'epoch': '0.3436', 'num_input_tokens_seen': 248519968, 'train_runtime': '5.427e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77300/225000 [150:45:14<132:16:40,  3.22s/it]
 34%|███▍      | 77301/225000 [150:45:17<128:32:23,  3.13s/it]
 34%|███▍      | 77302/225000 [150:45:23<162:07:13,  3.95s/it]
 34%|███▍      | 77303/225000 [150:45:32<218:16:47,  5.32s/it]
 34%|███▍      | 77304/225000 [150:45:34<186:52:30,  4.55s/it]
 34%|███▍      | 77305/225000 [150:45:37<163:04:09,  3.97s/it]
 34%|███▍      | 77306/225000 [150:45:40<148:09:51,  3.61s/it]
 34%|███▍      | 77307/225000 [150:45:45<170:24:47,  4.15s/it]
 34%|███▍      | 77308/225000 [150:45:50<179:55:21,  4.39s/it]
 34%|███▍      | 77309/225000 [150:45:54<170:12:54,  4.15s/it]
 34%|███▍      | 77310/225000 [150:45:57<157:54:52,  3.85s/it]
 34%|███▍      | 77311/225000 [150:46:00<147:02:39,  3.58s/it]
 34%|███▍      | 77312/225000 [150:46:03<136:19:01,  3.32s/it]
 34%|███▍      | 77313/225000 [150:46:05<132:00:46,  3.22s/it]
 34%|███▍      | 77314/225000 [150:46:09<131:19:30,  3.20s/it]
 34%|███▍      | 77315/225000 [150:46:12<133:21:03,  3.25s/it]
 34%|███▍      | 77316/225000 [150:46:18<164:09:53,  4.00s/it]
 34%|███▍      | 77317/225000 [150:46:28<240:32:02,  5.86s/it]
 34%|███▍      | 77318/225000 [150:46:30<197:30:23,  4.81s/it]
 34%|███▍      | 77319/225000 [150:46:33<167:24:34,  4.08s/it]
 34%|███▍      | 77320/225000 [150:46:36<160:53:09,  3.92s/it]
 34%|███▍      | 77321/225000 [150:46:43<199:03:32,  4.85s/it]
 34%|███▍      | 77322/225000 [150:46:46<174:49:03,  4.26s/it]
 34%|███▍      | 77323/225000 [150:46:50<163:45:17,  3.99s/it]
 34%|███▍      | 77324/225000 [150:46:53<157:10:54,  3.83s/it]
 34%|███▍      | 77325/225000 [150:46:56<144:41:45,  3.53s/it]
 34%|███▍      | 77326/225000 [150:46:59<141:26:42,  3.45s/it]
 34%|███▍      | 77327/225000 [150:47:02<136:20:05,  3.32s/it]
 34%|███▍      | 77328/225000 [150:47:05<134:11:58,  3.27s/it]
 34%|███▍      | 77329/225000 [150:47:09<134:19:10,  3.27s/it]
 34%|███▍      | 77330/225000 [150:47:11<130:13:13,  3.17s/it]
 34%|███▍      | 77331/225000 [150:47:17<164:24:22,  4.01s/it]
 34%|███▍      | 77332/225000 [150:47:26<219:12:57,  5.34s/it]
 34%|███▍      | 77333/225000 [150:47:31<210:09:40,  5.12s/it]
 34%|███▍      | 77334/225000 [150:47:34<184:56:16,  4.51s/it]
 34%|███▍      | 77335/225000 [150:47:37<172:09:37,  4.20s/it]
 34%|███▍      | 77336/225000 [150:47:40<162:45:39,  3.97s/it]
 34%|███▍      | 77337/225000 [150:47:44<155:14:05,  3.78s/it]
 34%|███▍      | 77338/225000 [150:47:47<147:56:57,  3.61s/it]
 34%|███▍      | 77339/225000 [150:47:50<138:18:09,  3.37s/it]
 34%|███▍      | 77340/225000 [150:47:58<196:16:57,  4.79s/it]
 34%|███▍      | 77341/225000 [150:48:05<218:22:40,  5.32s/it]
 34%|███▍      | 77342/225000 [150:48:07<182:39:02,  4.45s/it]
 34%|███▍      | 77343/225000 [150:48:10<166:20:41,  4.06s/it]
 34%|███▍      | 77344/225000 [150:48:13<156:32:12,  3.82s/it]
 34%|███▍      | 77345/225000 [150:48:16<143:21:33,  3.50s/it]
 34%|███▍      | 77346/225000 [150:48:22<171:46:47,  4.19s/it]
 34%|███▍      | 77347/225000 [150:48:28<191:41:20,  4.67s/it]
 34%|███▍      | 77348/225000 [150:48:33<193:30:10,  4.72s/it]
 34%|███▍      | 77349/225000 [150:48:35<168:34:47,  4.11s/it]
 34%|███▍      | 77350/225000 [150:48:37<145:38:09,  3.55s/it]
                                                              
{'loss': '0.838', 'grad_norm': '2.574', 'learning_rate': '3.885e-05', 'epoch': '0.3438', 'num_input_tokens_seen': 248605184, 'train_runtime': '5.429e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77350/225000 [150:48:37<145:38:09,  3.55s/it]
 34%|███▍      | 77351/225000 [150:48:41<149:27:07,  3.64s/it]
 34%|███▍      | 77352/225000 [150:48:45<154:04:27,  3.76s/it]
 34%|███▍      | 77353/225000 [150:48:51<183:14:28,  4.47s/it]
 34%|███▍      | 77354/225000 [150:48:54<164:17:13,  4.01s/it]
 34%|███▍      | 77355/225000 [150:48:57<152:37:33,  3.72s/it]
 34%|███▍      | 77356/225000 [150:49:00<142:43:05,  3.48s/it]
 34%|███▍      | 77357/225000 [150:49:04<140:57:56,  3.44s/it]
 34%|███▍      | 77358/225000 [150:49:07<138:57:18,  3.39s/it]
 34%|███▍      | 77359/225000 [150:49:10<137:33:27,  3.35s/it]
 34%|███▍      | 77360/225000 [150:49:13<129:53:22,  3.17s/it]
 34%|███▍      | 77361/225000 [150:49:20<178:35:29,  4.35s/it]
 34%|███▍      | 77362/225000 [150:49:28<220:07:24,  5.37s/it]
 34%|███▍      | 77363/225000 [150:49:31<189:07:19,  4.61s/it]
 34%|███▍      | 77364/225000 [150:49:34<171:32:56,  4.18s/it]
 34%|███▍      | 77365/225000 [150:49:36<149:54:15,  3.66s/it]
 34%|███▍      | 77366/225000 [150:49:44<203:38:46,  4.97s/it]
 34%|███▍      | 77367/225000 [150:49:47<172:39:04,  4.21s/it]
 34%|███▍      | 77368/225000 [150:49:50<162:27:53,  3.96s/it]
 34%|███▍      | 77369/225000 [150:49:53<151:12:36,  3.69s/it]
 34%|███▍      | 77370/225000 [150:49:56<137:40:46,  3.36s/it]
 34%|███▍      | 77371/225000 [150:49:59<133:40:08,  3.26s/it]
 34%|███▍      | 77372/225000 [150:50:02<134:09:45,  3.27s/it]
 34%|███▍      | 77373/225000 [150:50:05<132:20:45,  3.23s/it]
 34%|███▍      | 77374/225000 [150:50:12<171:02:25,  4.17s/it]
 34%|███▍      | 77375/225000 [150:50:19<207:10:21,  5.05s/it]
 34%|███▍      | 77376/225000 [150:50:22<180:05:00,  4.39s/it]
 34%|███▍      | 77377/225000 [150:50:24<161:20:03,  3.93s/it]
 34%|███▍      | 77378/225000 [150:50:27<145:55:47,  3.56s/it]
 34%|███▍      | 77379/225000 [150:50:30<136:46:06,  3.34s/it]
 34%|███▍      | 77380/225000 [150:50:33<130:44:15,  3.19s/it]
 34%|███▍      | 77381/225000 [150:50:39<170:26:11,  4.16s/it]
 34%|███▍      | 77382/225000 [150:50:47<210:38:52,  5.14s/it]
 34%|███▍      | 77383/225000 [150:50:49<178:42:58,  4.36s/it]
 34%|███▍      | 77384/225000 [150:50:53<168:25:24,  4.11s/it]
 34%|███▍      | 77385/225000 [150:50:58<188:45:48,  4.60s/it]
 34%|███▍      | 77386/225000 [150:51:01<164:48:53,  4.02s/it]
 34%|███▍      | 77387/225000 [150:51:04<149:45:14,  3.65s/it]
 34%|███▍      | 77388/225000 [150:51:10<177:20:15,  4.32s/it]
 34%|███▍      | 77389/225000 [150:51:14<176:39:32,  4.31s/it]
 34%|███▍      | 77390/225000 [150:51:17<162:34:26,  3.96s/it]
 34%|███▍      | 77391/225000 [150:51:20<153:03:56,  3.73s/it]
 34%|███▍      | 77392/225000 [150:51:24<145:26:24,  3.55s/it]
 34%|███▍      | 77393/225000 [150:51:29<173:09:18,  4.22s/it]
 34%|███▍      | 77394/225000 [150:51:32<149:03:16,  3.64s/it]
 34%|███▍      | 77395/225000 [150:51:34<135:37:17,  3.31s/it]
 34%|███▍      | 77396/225000 [150:51:38<144:54:50,  3.53s/it]
 34%|███▍      | 77397/225000 [150:51:46<200:29:13,  4.89s/it]
 34%|███▍      | 77398/225000 [150:51:49<174:40:48,  4.26s/it]
 34%|███▍      | 77399/225000 [150:51:52<162:21:32,  3.96s/it]
 34%|███▍      | 77400/225000 [150:51:55<151:02:58,  3.68s/it]
                                                              
{'loss': '0.8204', 'grad_norm': '3.447', 'learning_rate': '3.884e-05', 'epoch': '0.344', 'num_input_tokens_seen': 248687496, 'train_runtime': '5.431e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77400/225000 [150:51:55<151:02:58,  3.68s/it]
 34%|███▍      | 77401/225000 [150:51:59<145:30:41,  3.55s/it]
 34%|███▍      | 77402/225000 [150:52:02<148:10:46,  3.61s/it]
 34%|███▍      | 77403/225000 [150:52:05<137:11:19,  3.35s/it]
 34%|███▍      | 77404/225000 [150:52:08<131:07:55,  3.20s/it]
 34%|███▍      | 77405/225000 [150:52:11<127:24:56,  3.11s/it]
 34%|███▍      | 77406/225000 [150:52:14<128:45:17,  3.14s/it]
 34%|███▍      | 77407/225000 [150:52:18<139:59:56,  3.41s/it]
 34%|███▍      | 77408/225000 [150:52:29<230:50:25,  5.63s/it]
 34%|███▍      | 77409/225000 [150:52:36<252:41:23,  6.16s/it]
 34%|███▍      | 77410/225000 [150:52:38<203:03:35,  4.95s/it]
 34%|███▍      | 77411/225000 [150:52:41<171:34:33,  4.19s/it]
 34%|███▍      | 77412/225000 [150:52:44<155:19:40,  3.79s/it]
 34%|███▍      | 77413/225000 [150:52:47<149:24:05,  3.64s/it]
 34%|███▍      | 77414/225000 [150:52:50<144:48:02,  3.53s/it]
 34%|███▍      | 77415/225000 [150:52:54<141:48:21,  3.46s/it]
 34%|███▍      | 77416/225000 [150:52:57<138:13:26,  3.37s/it]
 34%|███▍      | 77417/225000 [150:53:00<135:25:46,  3.30s/it]
 34%|███▍      | 77418/225000 [150:53:03<135:03:00,  3.29s/it]
 34%|███▍      | 77419/225000 [150:53:06<130:08:54,  3.17s/it]
 34%|███▍      | 77420/225000 [150:53:13<173:28:52,  4.23s/it]
 34%|███▍      | 77421/225000 [150:53:21<225:40:50,  5.51s/it]
 34%|███▍      | 77422/225000 [150:53:24<195:55:51,  4.78s/it]
 34%|███▍      | 77423/225000 [150:53:27<166:22:38,  4.06s/it]
 34%|███▍      | 77424/225000 [150:53:30<163:10:27,  3.98s/it]
 34%|███▍      | 77425/225000 [150:53:38<203:51:44,  4.97s/it]
 34%|███▍      | 77426/225000 [150:53:40<170:18:29,  4.15s/it]
 34%|███▍      | 77427/225000 [150:53:43<157:18:33,  3.84s/it]
 34%|███▍      | 77428/225000 [150:53:51<207:47:35,  5.07s/it]
 34%|███▍      | 77429/225000 [150:53:53<172:26:58,  4.21s/it]
 34%|███▍      | 77430/225000 [150:53:55<148:08:05,  3.61s/it]
 34%|███▍      | 77431/225000 [150:53:58<133:16:48,  3.25s/it]
 34%|███▍      | 77432/225000 [150:54:01<130:28:58,  3.18s/it]
 34%|███▍      | 77433/225000 [150:54:04<129:43:42,  3.16s/it]
 34%|███▍      | 77434/225000 [150:54:07<128:19:45,  3.13s/it]
 34%|███▍      | 77435/225000 [150:54:11<133:33:34,  3.26s/it]
 34%|███▍      | 77436/225000 [150:54:14<134:45:09,  3.29s/it]
 34%|███▍      | 77437/225000 [150:54:17<131:40:54,  3.21s/it]
 34%|███▍      | 77438/225000 [150:54:26<197:34:35,  4.82s/it]
 34%|███▍      | 77439/225000 [150:54:32<218:06:01,  5.32s/it]
 34%|███▍      | 77440/225000 [150:54:37<210:14:16,  5.13s/it]
 34%|███▍      | 77441/225000 [150:54:40<183:28:04,  4.48s/it]
 34%|███▍      | 77442/225000 [150:54:42<161:38:19,  3.94s/it]
 34%|███▍      | 77443/225000 [150:54:46<157:39:05,  3.85s/it]
 34%|███▍      | 77444/225000 [150:54:49<149:39:02,  3.65s/it]
 34%|███▍      | 77445/225000 [150:54:53<145:21:04,  3.55s/it]
 34%|███▍      | 77446/225000 [150:54:56<140:37:29,  3.43s/it]
 34%|███▍      | 77447/225000 [150:54:59<137:32:45,  3.36s/it]
 34%|███▍      | 77448/225000 [150:55:04<163:05:26,  3.98s/it]
 34%|███▍      | 77449/225000 [150:55:08<154:40:05,  3.77s/it]
 34%|███▍      | 77450/225000 [150:55:17<223:40:09,  5.46s/it]
                                                              
{'loss': '0.8519', 'grad_norm': '1.709', 'learning_rate': '3.882e-05', 'epoch': '0.3442', 'num_input_tokens_seen': 248771416, 'train_runtime': '5.433e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77450/225000 [150:55:17<223:40:09,  5.46s/it]
 34%|███▍      | 77451/225000 [150:55:20<192:43:35,  4.70s/it]
 34%|███▍      | 77452/225000 [150:55:23<171:00:17,  4.17s/it]
 34%|███▍      | 77453/225000 [150:55:26<153:30:43,  3.75s/it]
 34%|███▍      | 77454/225000 [150:55:29<144:32:00,  3.53s/it]
 34%|███▍      | 77455/225000 [150:55:33<155:31:29,  3.79s/it]
 34%|███▍      | 77456/225000 [150:55:41<200:40:24,  4.90s/it]
 34%|███▍      | 77457/225000 [150:55:43<177:16:40,  4.33s/it]
 34%|███▍      | 77458/225000 [150:55:46<158:15:13,  3.86s/it]
 34%|███▍      | 77459/225000 [150:55:49<143:46:44,  3.51s/it]
 34%|███▍      | 77460/225000 [150:55:52<133:06:43,  3.25s/it]
 34%|███▍      | 77461/225000 [150:55:55<133:47:08,  3.26s/it]
 34%|███▍      | 77462/225000 [150:55:58<134:30:01,  3.28s/it]
 34%|███▍      | 77463/225000 [150:56:07<200:52:26,  4.90s/it]
 34%|███▍      | 77464/225000 [150:56:12<202:50:27,  4.95s/it]
 34%|███▍      | 77465/225000 [150:56:15<177:34:09,  4.33s/it]
 34%|███▍      | 77466/225000 [150:56:18<165:59:08,  4.05s/it]
 34%|███▍      | 77467/225000 [150:56:22<157:52:16,  3.85s/it]
 34%|███▍      | 77468/225000 [150:56:25<149:06:32,  3.64s/it]
 34%|███▍      | 77469/225000 [150:56:28<140:26:59,  3.43s/it]
 34%|███▍      | 77470/225000 [150:56:31<138:47:40,  3.39s/it]
 34%|███▍      | 77471/225000 [150:56:35<151:39:22,  3.70s/it]
 34%|███▍      | 77472/225000 [150:56:42<182:21:39,  4.45s/it]
 34%|███▍      | 77473/225000 [150:56:45<169:41:16,  4.14s/it]
 34%|███▍      | 77474/225000 [150:56:52<201:09:58,  4.91s/it]
 34%|███▍      | 77475/225000 [150:56:54<173:51:09,  4.24s/it]
 34%|███▍      | 77476/225000 [150:56:58<160:47:35,  3.92s/it]
 34%|███▍      | 77477/225000 [150:57:00<145:29:59,  3.55s/it]
 34%|███▍      | 77478/225000 [150:57:03<135:06:25,  3.30s/it]
 34%|███▍      | 77479/225000 [150:57:06<131:11:51,  3.20s/it]
 34%|███▍      | 77480/225000 [150:57:09<134:53:49,  3.29s/it]
 34%|███▍      | 77481/225000 [150:57:12<130:45:59,  3.19s/it]
 34%|███▍      | 77482/225000 [150:57:16<132:08:25,  3.22s/it]
 34%|███▍      | 77483/225000 [150:57:19<134:48:12,  3.29s/it]
 34%|███▍      | 77484/225000 [150:57:22<132:47:33,  3.24s/it]
 34%|███▍      | 77485/225000 [150:57:30<193:09:37,  4.71s/it]
 34%|███▍      | 77486/225000 [150:57:36<198:01:21,  4.83s/it]
 34%|███▍      | 77487/225000 [150:57:39<179:07:06,  4.37s/it]
 34%|███▍      | 77488/225000 [150:57:43<171:31:47,  4.19s/it]
 34%|███▍      | 77489/225000 [150:57:46<165:18:05,  4.03s/it]
 34%|███▍      | 77490/225000 [150:57:50<160:48:54,  3.92s/it]
 34%|███▍      | 77491/225000 [150:57:56<183:37:41,  4.48s/it]
 34%|███▍      | 77492/225000 [150:57:59<168:41:11,  4.12s/it]
 34%|███▍      | 77493/225000 [150:58:02<158:26:22,  3.87s/it]
 34%|███▍      | 77494/225000 [150:58:06<158:10:16,  3.86s/it]
 34%|███▍      | 77495/225000 [150:58:09<151:03:33,  3.69s/it]
 34%|███▍      | 77496/225000 [150:58:17<194:06:09,  4.74s/it]
 34%|███▍      | 77497/225000 [150:58:20<173:40:23,  4.24s/it]
 34%|███▍      | 77498/225000 [150:58:23<158:31:24,  3.87s/it]
 34%|███▍      | 77499/225000 [150:58:25<144:24:27,  3.52s/it]
 34%|███▍      | 77500/225000 [150:58:29<144:34:23,  3.53s/it]
                                                              
{'loss': '0.7616', 'grad_norm': '3.634', 'learning_rate': '3.881e-05', 'epoch': '0.3444', 'num_input_tokens_seen': 248855488, 'train_runtime': '5.435e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77500/225000 [150:58:29<144:34:23,  3.53s/it]
 34%|███▍      | 77501/225000 [150:58:33<148:50:10,  3.63s/it]
 34%|███▍      | 77502/225000 [150:58:39<181:18:44,  4.43s/it]
 34%|███▍      | 77503/225000 [150:58:48<231:43:26,  5.66s/it]
 34%|███▍      | 77504/225000 [150:58:50<194:25:54,  4.75s/it]
 34%|███▍      | 77505/225000 [150:58:53<166:10:50,  4.06s/it]
 34%|███▍      | 77506/225000 [150:58:56<153:24:16,  3.74s/it]
 34%|███▍      | 77507/225000 [150:58:59<150:42:30,  3.68s/it]
 34%|███▍      | 77508/225000 [150:59:03<149:39:20,  3.65s/it]
 34%|███▍      | 77509/225000 [150:59:06<141:26:53,  3.45s/it]
 34%|███▍      | 77510/225000 [150:59:09<138:41:00,  3.39s/it]
 34%|███▍      | 77511/225000 [150:59:12<136:33:32,  3.33s/it]
 34%|███▍      | 77512/225000 [150:59:16<138:26:12,  3.38s/it]
 34%|███▍      | 77513/225000 [150:59:19<131:41:23,  3.21s/it]
 34%|███▍      | 77514/225000 [150:59:27<190:14:10,  4.64s/it]
 34%|███▍      | 77515/225000 [150:59:37<264:05:03,  6.45s/it]
 34%|███▍      | 77516/225000 [150:59:40<216:31:05,  5.29s/it]
 34%|███▍      | 77517/225000 [150:59:42<183:01:57,  4.47s/it]
 34%|███▍      | 77518/225000 [150:59:49<213:59:55,  5.22s/it]
 34%|███▍      | 77519/225000 [150:59:53<190:06:48,  4.64s/it]
 34%|███▍      | 77520/225000 [150:59:56<174:10:10,  4.25s/it]
 34%|███▍      | 77521/225000 [150:59:59<161:11:51,  3.93s/it]
 34%|███▍      | 77522/225000 [151:00:02<149:29:31,  3.65s/it]
 34%|███▍      | 77523/225000 [151:00:05<140:08:11,  3.42s/it]
 34%|███▍      | 77524/225000 [151:00:08<139:04:52,  3.40s/it]
 34%|███▍      | 77525/225000 [151:00:12<138:53:47,  3.39s/it]
 34%|███▍      | 77526/225000 [151:00:15<134:12:53,  3.28s/it]
 34%|███▍      | 77527/225000 [151:00:18<136:01:10,  3.32s/it]
 34%|███▍      | 77528/225000 [151:00:21<131:02:26,  3.20s/it]
 34%|███▍      | 77529/225000 [151:00:24<133:21:16,  3.26s/it]
 34%|███▍      | 77530/225000 [151:00:28<134:44:57,  3.29s/it]
 34%|███▍      | 77531/225000 [151:00:32<150:09:06,  3.67s/it]
 34%|███▍      | 77532/225000 [151:00:42<226:28:20,  5.53s/it]
 34%|███▍      | 77533/225000 [151:00:48<225:52:33,  5.51s/it]
 34%|███▍      | 77534/225000 [151:00:51<199:04:01,  4.86s/it]
 34%|███▍      | 77535/225000 [151:00:54<172:19:44,  4.21s/it]
 34%|███▍      | 77536/225000 [151:00:56<153:19:41,  3.74s/it]
 34%|███▍      | 77537/225000 [151:00:59<144:34:53,  3.53s/it]
 34%|███▍      | 77538/225000 [151:01:02<133:58:57,  3.27s/it]
 34%|███▍      | 77539/225000 [151:01:09<181:42:24,  4.44s/it]
 34%|███▍      | 77540/225000 [151:01:14<190:58:46,  4.66s/it]
 34%|███▍      | 77541/225000 [151:01:17<163:00:54,  3.98s/it]
 34%|███▍      | 77542/225000 [151:01:20<147:16:11,  3.60s/it]
 34%|███▍      | 77543/225000 [151:01:22<137:57:42,  3.37s/it]
 34%|███▍      | 77544/225000 [151:01:29<182:49:52,  4.46s/it]
 34%|███▍      | 77545/225000 [151:01:35<195:25:29,  4.77s/it]
 34%|███▍      | 77546/225000 [151:01:38<170:40:29,  4.17s/it]
 34%|███▍      | 77547/225000 [151:01:40<153:01:29,  3.74s/it]
 34%|███▍      | 77548/225000 [151:01:43<138:46:23,  3.39s/it]
 34%|███▍      | 77549/225000 [151:01:51<198:26:07,  4.84s/it]
 34%|███▍      | 77550/225000 [151:01:55<181:08:25,  4.42s/it]
                                                              
{'loss': '0.8241', 'grad_norm': '2.066', 'learning_rate': '3.879e-05', 'epoch': '0.3447', 'num_input_tokens_seen': 248939800, 'train_runtime': '5.437e+05', 'train_tokens_per_second': '457.9'}

 34%|███▍      | 77550/225000 [151:01:55<181:08:25,  4.42s/it]
 34%|███▍      | 77551/225000 [151:01:58<164:43:30,  4.02s/it]
 34%|███▍      | 77552/225000 [151:02:01<151:25:10,  3.70s/it]
 34%|███▍      | 77553/225000 [151:02:03<140:20:12,  3.43s/it]
 34%|███▍      | 77554/225000 [151:02:06<135:01:09,  3.30s/it]
 34%|███▍      | 77555/225000 [151:02:09<128:30:30,  3.14s/it]
 34%|███▍      | 77556/225000 [151:02:13<136:39:28,  3.34s/it]
 34%|███▍      | 77557/225000 [151:02:21<199:09:51,  4.86s/it]
 34%|███▍      | 77558/225000 [151:02:26<199:46:41,  4.88s/it]
 34%|███▍      | 77559/225000 [151:02:29<175:32:20,  4.29s/it]
 34%|███▍      | 77560/225000 [151:02:32<157:53:54,  3.86s/it]
 34%|███▍      | 77561/225000 [151:02:35<141:34:31,  3.46s/it]
 34%|███▍      | 77562/225000 [151:02:44<218:02:04,  5.32s/it]
 34%|███▍      | 77563/225000 [151:02:47<184:54:13,  4.51s/it]
 34%|███▍      | 77564/225000 [151:02:50<161:30:15,  3.94s/it]
 34%|███▍      | 77565/225000 [151:02:53<149:42:10,  3.66s/it]
 34%|███▍      | 77566/225000 [151:02:55<137:46:31,  3.36s/it]
 34%|███▍      | 77567/225000 [151:03:05<210:49:25,  5.15s/it]
 34%|███▍      | 77568/225000 [151:03:08<192:29:34,  4.70s/it]
 34%|███▍      | 77569/225000 [151:03:11<165:16:36,  4.04s/it]
 34%|███▍      | 77570/225000 [151:03:13<146:06:43,  3.57s/it]
 34%|███▍      | 77571/225000 [151:03:16<138:49:25,  3.39s/it]
 34%|███▍      | 77572/225000 [151:03:19<133:34:38,  3.26s/it]
 34%|███▍      | 77573/225000 [151:03:27<190:50:04,  4.66s/it]
 34%|███▍      | 77574/225000 [151:03:32<190:08:52,  4.64s/it]
 34%|███▍      | 77575/225000 [151:03:35<176:07:39,  4.30s/it]
 34%|███▍      | 77576/225000 [151:03:38<162:25:41,  3.97s/it]
 34%|███▍      | 77577/225000 [151:03:42<154:10:43,  3.76s/it]
 34%|███▍      | 77578/225000 [151:03:45<151:24:04,  3.70s/it]
 34%|███▍      | 77579/225000 [151:03:49<148:10:54,  3.62s/it]
 34%|███▍      | 77580/225000 [151:03:52<146:12:08,  3.57s/it]
 34%|███▍      | 77581/225000 [151:03:56<148:17:51,  3.62s/it]
 34%|███▍      | 77582/225000 [151:03:59<143:17:16,  3.50s/it]
 34%|███▍      | 77583/225000 [151:04:02<141:36:21,  3.46s/it]
 34%|███▍      | 77584/225000 [151:04:06<142:20:23,  3.48s/it]
 34%|███▍      | 77585/225000 [151:04:10<153:57:05,  3.76s/it]
 34%|███▍      | 77586/225000 [151:04:19<209:21:58,  5.11s/it]
 34%|███▍      | 77587/225000 [151:04:22<190:21:38,  4.65s/it]
 34%|███▍      | 77588/225000 [151:04:25<174:01:43,  4.25s/it]
 34%|███▍      | 77589/225000 [151:04:29<170:53:36,  4.17s/it]
 34%|███▍      | 77590/225000 [151:04:34<170:25:54,  4.16s/it]
 34%|███▍      | 77591/225000 [151:04:38<168:44:32,  4.12s/it]
 34%|███▍      | 77592/225000 [151:04:41<159:04:09,  3.88s/it]
 34%|███▍      | 77593/225000 [151:04:45<158:12:02,  3.86s/it]
 34%|███▍      | 77594/225000 [151:04:49<158:27:22,  3.87s/it]
 34%|███▍      | 77595/225000 [151:04:52<149:25:35,  3.65s/it]
 34%|███▍      | 77596/225000 [151:04:55<148:38:06,  3.63s/it]
 34%|███▍      | 77597/225000 [151:04:59<143:14:08,  3.50s/it]
 34%|███▍      | 77598/225000 [151:05:02<145:14:54,  3.55s/it]
 34%|███▍      | 77599/225000 [151:05:06<146:51:18,  3.59s/it]
 34%|███▍      | 77600/225000 [151:05:09<141:32:06,  3.46s/it]
                                                              
{'loss': '0.8451', 'grad_norm': '3.69', 'learning_rate': '3.878e-05', 'epoch': '0.3449', 'num_input_tokens_seen': 249025296, 'train_runtime': '5.439e+05', 'train_tokens_per_second': '457.8'}

 34%|███▍      | 77600/225000 [151:05:09<141:32:06,  3.46s/it]
 34%|███▍      | 77601/225000 [151:05:17<191:16:57,  4.67s/it]
 34%|███▍      | 77602/225000 [151:05:27<262:27:43,  6.41s/it]
 34%|███▍      | 77603/225000 [151:05:30<223:43:19,  5.46s/it]
 34%|███▍      | 77604/225000 [151:05:33<184:17:30,  4.50s/it]
 34%|███▍      | 77605/225000 [151:05:38<199:05:27,  4.86s/it]
 34%|███▍      | 77606/225000 [151:05:41<168:13:41,  4.11s/it]
 34%|███▍      | 77607/225000 [151:05:48<208:39:33,  5.10s/it]
 34%|███▍      | 77608/225000 [151:05:53<202:40:50,  4.95s/it]
 34%|███▍      | 77609/225000 [151:05:56<182:53:33,  4.47s/it]
 34%|███▍      | 77610/225000 [151:05:59<166:16:50,  4.06s/it]
 34%|███▍      | 77611/225000 [151:06:03<160:00:25,  3.91s/it]
 34%|███▍      | 77612/225000 [151:06:05<147:20:06,  3.60s/it]
 34%|███▍      | 77613/225000 [151:06:08<137:41:39,  3.36s/it]
 34%|███▍      | 77614/225000 [151:06:11<134:04:29,  3.27s/it]
 34%|███▍      | 77615/225000 [151:06:15<135:27:43,  3.31s/it]
 34%|███▍      | 77616/225000 [151:06:18<136:34:11,  3.34s/it]
 34%|███▍      | 77617/225000 [151:06:21<136:09:30,  3.33s/it]
 34%|███▍      | 77618/225000 [151:06:25<137:37:19,  3.36s/it]
 34%|███▍      | 77619/225000 [151:06:28<133:28:57,  3.26s/it]
 34%|███▍      | 77620/225000 [151:06:31<129:56:49,  3.17s/it]
 34%|███▍      | 77621/225000 [151:06:39<188:32:28,  4.61s/it]
 34%|███▍      | 77622/225000 [151:06:45<206:22:19,  5.04s/it]
 34%|███▍      | 77623/225000 [151:06:47<172:13:20,  4.21s/it]
 34%|███▍      | 77624/225000 [151:06:50<151:09:57,  3.69s/it]
 34%|███▍      | 77625/225000 [151:06:55<170:28:13,  4.16s/it]
 35%|███▍      | 77626/225000 [151:07:00<175:38:36,  4.29s/it]
 35%|███▍      | 77627/225000 [151:07:03<162:26:21,  3.97s/it]
 35%|███▍      | 77628/225000 [151:07:07<161:06:30,  3.94s/it]
 35%|███▍      | 77629/225000 [151:07:10<149:10:54,  3.64s/it]
 35%|███▍      | 77630/225000 [151:07:13<143:54:33,  3.52s/it]
 35%|███▍      | 77631/225000 [151:07:16<144:46:13,  3.54s/it]
 35%|███▍      | 77632/225000 [151:07:19<137:24:54,  3.36s/it]
 35%|███▍      | 77633/225000 [151:07:22<134:23:17,  3.28s/it]
 35%|███▍      | 77634/225000 [151:07:32<208:23:59,  5.09s/it]
 35%|███▍      | 77635/225000 [151:07:36<204:24:55,  4.99s/it]
 35%|███▍      | 77636/225000 [151:07:39<174:51:09,  4.27s/it]
 35%|███▍      | 77637/225000 [151:07:41<151:51:42,  3.71s/it]
 35%|███▍      | 77638/225000 [151:07:44<135:31:47,  3.31s/it]
 35%|███▍      | 77639/225000 [151:07:50<176:40:59,  4.32s/it]
 35%|███▍      | 77640/225000 [151:07:57<204:30:54,  5.00s/it]
 35%|███▍      | 77641/225000 [151:08:00<172:56:28,  4.22s/it]
 35%|███▍      | 77642/225000 [151:08:02<149:52:08,  3.66s/it]
 35%|███▍      | 77643/225000 [151:08:04<132:54:10,  3.25s/it]
 35%|███▍      | 77644/225000 [151:08:12<188:00:39,  4.59s/it]
 35%|███▍      | 77645/225000 [151:08:15<172:30:57,  4.21s/it]
 35%|███▍      | 77646/225000 [151:08:17<148:37:23,  3.63s/it]
 35%|███▍      | 77647/225000 [151:08:20<136:48:52,  3.34s/it]
 35%|███▍      | 77648/225000 [151:08:27<185:50:06,  4.54s/it]
 35%|███▍      | 77649/225000 [151:08:30<162:59:16,  3.98s/it]
 35%|███▍      | 77650/225000 [151:08:33<150:21:23,  3.67s/it]
                                                              
{'loss': '0.858', 'grad_norm': '4.251', 'learning_rate': '3.876e-05', 'epoch': '0.3451', 'num_input_tokens_seen': 249108320, 'train_runtime': '5.441e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77650/225000 [151:08:33<150:21:23,  3.67s/it]
 35%|███▍      | 77651/225000 [151:08:36<138:40:21,  3.39s/it]
 35%|███▍      | 77652/225000 [151:08:39<136:20:09,  3.33s/it]
 35%|███▍      | 77653/225000 [151:08:42<137:14:52,  3.35s/it]
 35%|███▍      | 77654/225000 [151:08:45<130:32:48,  3.19s/it]
 35%|███▍      | 77655/225000 [151:08:51<159:58:52,  3.91s/it]
 35%|███▍      | 77656/225000 [151:08:58<198:26:51,  4.85s/it]
 35%|███▍      | 77657/225000 [151:09:01<176:11:45,  4.30s/it]
 35%|███▍      | 77658/225000 [151:09:03<153:09:29,  3.74s/it]
 35%|███▍      | 77659/225000 [151:09:13<219:53:23,  5.37s/it]
 35%|███▍      | 77660/225000 [151:09:17<205:33:11,  5.02s/it]
 35%|███▍      | 77661/225000 [151:09:19<176:51:20,  4.32s/it]
 35%|███▍      | 77662/225000 [151:09:22<155:23:37,  3.80s/it]
 35%|███▍      | 77663/225000 [151:09:32<230:57:35,  5.64s/it]
 35%|███▍      | 77664/225000 [151:09:34<192:02:34,  4.69s/it]
 35%|███▍      | 77665/225000 [151:09:37<164:40:05,  4.02s/it]
 35%|███▍      | 77666/225000 [151:09:39<145:51:58,  3.56s/it]
 35%|███▍      | 77667/225000 [151:09:46<188:24:27,  4.60s/it]
 35%|███▍      | 77668/225000 [151:09:50<180:09:31,  4.40s/it]
 35%|███▍      | 77669/225000 [151:09:54<176:28:34,  4.31s/it]
 35%|███▍      | 77670/225000 [151:09:57<161:13:18,  3.94s/it]
 35%|███▍      | 77671/225000 [151:10:00<147:28:20,  3.60s/it]
 35%|███▍      | 77672/225000 [151:10:03<137:34:38,  3.36s/it]
 35%|███▍      | 77673/225000 [151:10:06<131:24:26,  3.21s/it]
 35%|███▍      | 77674/225000 [151:10:09<129:55:04,  3.17s/it]
 35%|███▍      | 77675/225000 [151:10:12<125:50:27,  3.08s/it]
 35%|███▍      | 77676/225000 [151:10:15<123:50:14,  3.03s/it]
 35%|███▍      | 77677/225000 [151:10:21<157:26:08,  3.85s/it]
 35%|███▍      | 77678/225000 [151:10:29<215:46:34,  5.27s/it]
 35%|███▍      | 77679/225000 [151:10:32<184:35:02,  4.51s/it]
 35%|███▍      | 77680/225000 [151:10:35<161:14:06,  3.94s/it]
 35%|███▍      | 77681/225000 [151:10:38<157:05:36,  3.84s/it]
 35%|███▍      | 77682/225000 [151:10:41<146:28:22,  3.58s/it]
 35%|███▍      | 77683/225000 [151:10:44<135:49:11,  3.32s/it]
 35%|███▍      | 77684/225000 [151:10:46<127:31:21,  3.12s/it]
 35%|███▍      | 77685/225000 [151:10:53<173:31:51,  4.24s/it]
 35%|███▍      | 77686/225000 [151:11:01<213:09:42,  5.21s/it]
 35%|███▍      | 77687/225000 [151:11:03<177:09:44,  4.33s/it]
 35%|███▍      | 77688/225000 [151:11:07<166:59:43,  4.08s/it]
 35%|███▍      | 77689/225000 [151:11:09<149:26:23,  3.65s/it]
 35%|███▍      | 77690/225000 [151:11:19<221:32:20,  5.41s/it]
 35%|███▍      | 77691/225000 [151:11:23<206:36:42,  5.05s/it]
 35%|███▍      | 77692/225000 [151:11:26<178:28:46,  4.36s/it]
 35%|███▍      | 77693/225000 [151:11:28<157:31:51,  3.85s/it]
 35%|███▍      | 77694/225000 [151:11:36<204:55:43,  5.01s/it]
 35%|███▍      | 77695/225000 [151:11:40<187:06:08,  4.57s/it]
 35%|███▍      | 77696/225000 [151:11:42<162:42:11,  3.98s/it]
 35%|███▍      | 77697/225000 [151:11:50<207:23:19,  5.07s/it]
 35%|███▍      | 77698/225000 [151:11:53<186:35:07,  4.56s/it]
 35%|███▍      | 77699/225000 [151:11:59<197:17:44,  4.82s/it]
 35%|███▍      | 77700/225000 [151:12:01<170:17:30,  4.16s/it]
                                                              
{'loss': '0.8573', 'grad_norm': '3.912', 'learning_rate': '3.874e-05', 'epoch': '0.3453', 'num_input_tokens_seen': 249194632, 'train_runtime': '5.443e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77700/225000 [151:12:01<170:17:30,  4.16s/it]
 35%|███▍      | 77701/225000 [151:12:04<149:34:42,  3.66s/it]
 35%|███▍      | 77702/225000 [151:12:06<138:10:56,  3.38s/it]
 35%|███▍      | 77703/225000 [151:12:09<129:39:43,  3.17s/it]
 35%|███▍      | 77704/225000 [151:12:12<125:59:59,  3.08s/it]
 35%|███▍      | 77705/225000 [151:12:21<200:22:32,  4.90s/it]
 35%|███▍      | 77706/225000 [151:12:24<169:49:00,  4.15s/it]
 35%|███▍      | 77707/225000 [151:12:27<156:51:34,  3.83s/it]
 35%|███▍      | 77708/225000 [151:12:29<142:31:39,  3.48s/it]
 35%|███▍      | 77709/225000 [151:12:37<197:37:33,  4.83s/it]
 35%|███▍      | 77710/225000 [151:12:42<192:53:50,  4.71s/it]
 35%|███▍      | 77711/225000 [151:12:44<167:04:11,  4.08s/it]
 35%|███▍      | 77712/225000 [151:12:47<150:46:40,  3.69s/it]
 35%|███▍      | 77713/225000 [151:12:50<141:10:54,  3.45s/it]
 35%|███▍      | 77714/225000 [151:12:54<144:50:00,  3.54s/it]
 35%|███▍      | 77715/225000 [151:12:57<136:49:27,  3.34s/it]
 35%|███▍      | 77716/225000 [151:13:00<131:28:03,  3.21s/it]
 35%|███▍      | 77717/225000 [151:13:08<192:07:30,  4.70s/it]
 35%|███▍      | 77718/225000 [151:13:13<201:33:42,  4.93s/it]
 35%|███▍      | 77719/225000 [151:13:16<178:09:03,  4.35s/it]
 35%|███▍      | 77720/225000 [151:13:19<158:11:42,  3.87s/it]
 35%|███▍      | 77721/225000 [151:13:21<140:55:04,  3.44s/it]
 35%|███▍      | 77722/225000 [151:13:24<131:03:45,  3.20s/it]
 35%|███▍      | 77723/225000 [151:13:27<132:10:17,  3.23s/it]
 35%|███▍      | 77724/225000 [151:13:37<210:22:53,  5.14s/it]
 35%|███▍      | 77725/225000 [151:13:40<183:44:37,  4.49s/it]
 35%|███▍      | 77726/225000 [151:13:42<157:18:32,  3.85s/it]
 35%|███▍      | 77727/225000 [151:13:45<142:06:44,  3.47s/it]
 35%|███▍      | 77728/225000 [151:13:48<133:00:10,  3.25s/it]
 35%|███▍      | 77729/225000 [151:13:56<202:44:04,  4.96s/it]
 35%|███▍      | 77730/225000 [151:13:59<171:17:17,  4.19s/it]
 35%|███▍      | 77731/225000 [151:14:02<156:55:43,  3.84s/it]
 35%|███▍      | 77732/225000 [151:14:07<166:47:04,  4.08s/it]
 35%|███▍      | 77733/225000 [151:14:13<193:26:38,  4.73s/it]
 35%|███▍      | 77734/225000 [151:14:16<169:26:58,  4.14s/it]
 35%|███▍      | 77735/225000 [151:14:18<150:16:50,  3.67s/it]
 35%|███▍      | 77736/225000 [151:14:21<143:53:47,  3.52s/it]
 35%|███▍      | 77737/225000 [151:14:29<189:34:21,  4.63s/it]
 35%|███▍      | 77738/225000 [151:14:32<178:48:57,  4.37s/it]
 35%|███▍      | 77739/225000 [151:14:35<162:05:00,  3.96s/it]
 35%|███▍      | 77740/225000 [151:14:41<177:33:57,  4.34s/it]
 35%|███▍      | 77741/225000 [151:14:43<159:14:14,  3.89s/it]
 35%|███▍      | 77742/225000 [151:14:46<148:30:59,  3.63s/it]
 35%|███▍      | 77743/225000 [151:14:50<147:53:33,  3.62s/it]
 35%|███▍      | 77744/225000 [151:14:53<142:00:03,  3.47s/it]
 35%|███▍      | 77745/225000 [151:14:57<142:20:38,  3.48s/it]
 35%|███▍      | 77746/225000 [151:15:00<136:46:02,  3.34s/it]
 35%|███▍      | 77747/225000 [151:15:03<134:36:24,  3.29s/it]
 35%|███▍      | 77748/225000 [151:15:06<131:39:02,  3.22s/it]
 35%|███▍      | 77749/225000 [151:15:09<135:01:09,  3.30s/it]
 35%|███▍      | 77750/225000 [151:15:17<192:32:44,  4.71s/it]
                                                              
{'loss': '0.8588', 'grad_norm': '1.743', 'learning_rate': '3.873e-05', 'epoch': '0.3456', 'num_input_tokens_seen': 249277712, 'train_runtime': '5.445e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77750/225000 [151:15:17<192:32:44,  4.71s/it]
 35%|███▍      | 77751/225000 [151:15:24<218:30:58,  5.34s/it]
 35%|███▍      | 77752/225000 [151:15:27<183:08:15,  4.48s/it]
 35%|███▍      | 77753/225000 [151:15:29<158:52:47,  3.88s/it]
 35%|███▍      | 77754/225000 [151:15:32<144:41:28,  3.54s/it]
 35%|███▍      | 77755/225000 [151:15:34<133:13:53,  3.26s/it]
 35%|███▍      | 77756/225000 [151:15:37<125:55:04,  3.08s/it]
 35%|███▍      | 77757/225000 [151:15:41<134:40:45,  3.29s/it]
 35%|███▍      | 77758/225000 [151:15:49<193:29:42,  4.73s/it]
 35%|███▍      | 77759/225000 [151:15:53<184:44:25,  4.52s/it]
 35%|███▍      | 77760/225000 [151:15:57<172:32:48,  4.22s/it]
 35%|███▍      | 77761/225000 [151:16:02<185:20:00,  4.53s/it]
 35%|███▍      | 77762/225000 [151:16:06<187:19:41,  4.58s/it]
 35%|███▍      | 77763/225000 [151:16:10<176:56:07,  4.33s/it]
 35%|███▍      | 77764/225000 [151:16:14<171:29:52,  4.19s/it]
 35%|███▍      | 77765/225000 [151:16:19<178:11:14,  4.36s/it]
 35%|███▍      | 77766/225000 [151:16:22<166:03:01,  4.06s/it]
 35%|███▍      | 77767/225000 [151:16:25<147:51:06,  3.62s/it]
 35%|███▍      | 77768/225000 [151:16:28<138:48:47,  3.39s/it]
 35%|███▍      | 77769/225000 [151:16:31<137:08:59,  3.35s/it]
 35%|███▍      | 77770/225000 [151:16:34<131:03:28,  3.20s/it]
 35%|███▍      | 77771/225000 [151:16:38<143:12:10,  3.50s/it]
 35%|███▍      | 77772/225000 [151:16:47<209:56:26,  5.13s/it]
 35%|███▍      | 77773/225000 [151:16:51<195:19:22,  4.78s/it]
 35%|███▍      | 77774/225000 [151:16:53<167:26:06,  4.09s/it]
 35%|███▍      | 77775/225000 [151:16:56<146:05:50,  3.57s/it]
 35%|███▍      | 77776/225000 [151:17:03<195:50:23,  4.79s/it]
 35%|███▍      | 77777/225000 [151:17:08<188:59:01,  4.62s/it]
 35%|███▍      | 77778/225000 [151:17:10<166:50:38,  4.08s/it]
 35%|███▍      | 77779/225000 [151:17:13<149:33:15,  3.66s/it]
 35%|███▍      | 77780/225000 [151:17:20<192:49:52,  4.72s/it]
 35%|███▍      | 77781/225000 [151:17:23<167:10:06,  4.09s/it]
 35%|███▍      | 77782/225000 [151:17:26<152:58:46,  3.74s/it]
 35%|███▍      | 77783/225000 [151:17:32<177:37:37,  4.34s/it]
 35%|███▍      | 77784/225000 [151:17:34<154:40:05,  3.78s/it]
 35%|███▍      | 77785/225000 [151:17:40<177:36:09,  4.34s/it]
 35%|███▍      | 77786/225000 [151:17:44<181:58:13,  4.45s/it]
 35%|███▍      | 77787/225000 [151:17:47<159:30:05,  3.90s/it]
 35%|███▍      | 77788/225000 [151:17:49<142:00:08,  3.47s/it]
 35%|███▍      | 77789/225000 [151:17:52<128:36:23,  3.15s/it]
 35%|███▍      | 77790/225000 [151:17:58<164:49:15,  4.03s/it]
 35%|███▍      | 77791/225000 [151:18:04<195:00:53,  4.77s/it]
 35%|███▍      | 77792/225000 [151:18:07<165:20:22,  4.04s/it]
 35%|███▍      | 77793/225000 [151:18:10<149:12:50,  3.65s/it]
 35%|███▍      | 77794/225000 [151:18:16<180:45:29,  4.42s/it]
 35%|███▍      | 77795/225000 [151:18:19<172:33:13,  4.22s/it]
 35%|███▍      | 77796/225000 [151:18:23<162:48:19,  3.98s/it]
 35%|███▍      | 77797/225000 [151:18:26<155:39:41,  3.81s/it]
 35%|███▍      | 77798/225000 [151:18:29<145:57:34,  3.57s/it]
 35%|███▍      | 77799/225000 [151:18:33<143:34:07,  3.51s/it]
 35%|███▍      | 77800/225000 [151:18:36<138:07:47,  3.38s/it]
                                                              
{'loss': '0.8127', 'grad_norm': '3.49', 'learning_rate': '3.871e-05', 'epoch': '0.3458', 'num_input_tokens_seen': 249361544, 'train_runtime': '5.447e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77800/225000 [151:18:36<138:07:47,  3.38s/it]
 35%|███▍      | 77801/225000 [151:18:39<131:51:46,  3.22s/it]
 35%|███▍      | 77802/225000 [151:18:42<133:06:09,  3.26s/it]
 35%|███▍      | 77803/225000 [151:18:48<161:42:48,  3.96s/it]
 35%|███▍      | 77804/225000 [151:18:57<229:48:04,  5.62s/it]
 35%|███▍      | 77805/225000 [151:19:01<205:11:02,  5.02s/it]
 35%|███▍      | 77806/225000 [151:19:03<174:24:37,  4.27s/it]
 35%|███▍      | 77807/225000 [151:19:05<149:56:04,  3.67s/it]
 35%|███▍      | 77808/225000 [151:19:08<137:49:18,  3.37s/it]
 35%|███▍      | 77809/225000 [151:19:11<133:25:29,  3.26s/it]
 35%|███▍      | 77810/225000 [151:19:14<133:42:34,  3.27s/it]
 35%|███▍      | 77811/225000 [151:19:22<189:00:10,  4.62s/it]
 35%|███▍      | 77812/225000 [151:19:27<193:39:46,  4.74s/it]
 35%|███▍      | 77813/225000 [151:19:30<169:30:16,  4.15s/it]
 35%|███▍      | 77814/225000 [151:19:37<206:10:15,  5.04s/it]
 35%|███▍      | 77815/225000 [151:19:39<172:19:44,  4.21s/it]
 35%|███▍      | 77816/225000 [151:19:42<149:47:11,  3.66s/it]
 35%|███▍      | 77817/225000 [151:19:44<138:12:25,  3.38s/it]
 35%|███▍      | 77818/225000 [151:19:51<181:22:26,  4.44s/it]
 35%|███▍      | 77819/225000 [151:19:56<180:55:24,  4.43s/it]
 35%|███▍      | 77820/225000 [151:19:59<161:07:43,  3.94s/it]
 35%|███▍      | 77821/225000 [151:20:02<149:05:01,  3.65s/it]
 35%|███▍      | 77822/225000 [151:20:04<139:23:21,  3.41s/it]
 35%|███▍      | 77823/225000 [151:20:07<132:46:04,  3.25s/it]
 35%|███▍      | 77824/225000 [151:20:10<124:25:52,  3.04s/it]
 35%|███▍      | 77825/225000 [151:20:17<173:31:04,  4.24s/it]
 35%|███▍      | 77826/225000 [151:20:25<215:22:08,  5.27s/it]
 35%|███▍      | 77827/225000 [151:20:27<177:41:24,  4.35s/it]
 35%|███▍      | 77828/225000 [151:20:29<152:44:59,  3.74s/it]
 35%|███▍      | 77829/225000 [151:20:32<138:22:45,  3.38s/it]
 35%|███▍      | 77830/225000 [151:20:38<176:51:43,  4.33s/it]
 35%|███▍      | 77831/225000 [151:20:44<190:40:07,  4.66s/it]
 35%|███▍      | 77832/225000 [151:20:46<163:05:28,  3.99s/it]
 35%|███▍      | 77833/225000 [151:20:48<144:17:13,  3.53s/it]
 35%|███▍      | 77834/225000 [151:20:54<171:14:25,  4.19s/it]
 35%|███▍      | 77835/225000 [151:20:58<168:51:58,  4.13s/it]
 35%|███▍      | 77836/225000 [151:21:01<157:22:09,  3.85s/it]
 35%|███▍      | 77837/225000 [151:21:05<150:49:15,  3.69s/it]
 35%|███▍      | 77838/225000 [151:21:08<143:38:54,  3.51s/it]
 35%|███▍      | 77839/225000 [151:21:11<136:05:03,  3.33s/it]
 35%|███▍      | 77840/225000 [151:21:14<131:35:26,  3.22s/it]
 35%|███▍      | 77841/225000 [151:21:17<132:42:35,  3.25s/it]
 35%|███▍      | 77842/225000 [151:21:20<135:38:36,  3.32s/it]
 35%|███▍      | 77843/225000 [151:21:28<189:34:49,  4.64s/it]
 35%|███▍      | 77844/225000 [151:21:37<240:11:04,  5.88s/it]
 35%|███▍      | 77845/225000 [151:21:40<206:24:15,  5.05s/it]
 35%|███▍      | 77846/225000 [151:21:43<180:34:30,  4.42s/it]
 35%|███▍      | 77847/225000 [151:21:46<163:25:30,  4.00s/it]
 35%|███▍      | 77848/225000 [151:21:49<145:08:57,  3.55s/it]
 35%|███▍      | 77849/225000 [151:21:51<137:36:51,  3.37s/it]
 35%|███▍      | 77850/225000 [151:21:54<131:55:40,  3.23s/it]
                                                              
{'loss': '0.818', 'grad_norm': '3.512', 'learning_rate': '3.87e-05', 'epoch': '0.346', 'num_input_tokens_seen': 249444304, 'train_runtime': '5.449e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77850/225000 [151:21:54<131:55:40,  3.23s/it]
 35%|███▍      | 77851/225000 [151:21:58<131:28:12,  3.22s/it]
 35%|███▍      | 77852/225000 [151:22:04<167:30:32,  4.10s/it]
 35%|███▍      | 77853/225000 [151:22:12<220:50:25,  5.40s/it]
 35%|███▍      | 77854/225000 [151:22:16<196:32:00,  4.81s/it]
 35%|███▍      | 77855/225000 [151:22:18<170:47:01,  4.18s/it]
 35%|███▍      | 77856/225000 [151:22:21<156:08:44,  3.82s/it]
 35%|███▍      | 77857/225000 [151:22:24<143:17:21,  3.51s/it]
 35%|███▍      | 77858/225000 [151:22:28<145:19:54,  3.56s/it]
 35%|███▍      | 77859/225000 [151:22:31<139:37:39,  3.42s/it]
 35%|███▍      | 77860/225000 [151:22:34<139:14:19,  3.41s/it]
 35%|███▍      | 77861/225000 [151:22:40<166:51:41,  4.08s/it]
 35%|███▍      | 77862/225000 [151:22:49<226:23:36,  5.54s/it]
 35%|███▍      | 77863/225000 [151:22:51<188:37:22,  4.62s/it]
 35%|███▍      | 77864/225000 [151:22:54<164:39:44,  4.03s/it]
 35%|███▍      | 77865/225000 [151:22:57<147:06:28,  3.60s/it]
 35%|███▍      | 77866/225000 [151:22:59<134:43:00,  3.30s/it]
 35%|███▍      | 77867/225000 [151:23:03<142:48:34,  3.49s/it]
 35%|███▍      | 77868/225000 [151:23:10<190:38:23,  4.66s/it]
 35%|███▍      | 77869/225000 [151:23:13<162:22:56,  3.97s/it]
 35%|███▍      | 77870/225000 [151:23:15<144:08:23,  3.53s/it]
 35%|███▍      | 77871/225000 [151:23:18<135:38:58,  3.32s/it]
 35%|███▍      | 77872/225000 [151:23:21<129:56:53,  3.18s/it]
 35%|███▍      | 77873/225000 [151:23:30<200:11:11,  4.90s/it]
 35%|███▍      | 77874/225000 [151:23:34<194:05:49,  4.75s/it]
 35%|███▍      | 77875/225000 [151:23:38<176:23:21,  4.32s/it]
 35%|███▍      | 77876/225000 [151:23:42<171:18:58,  4.19s/it]
 35%|███▍      | 77877/225000 [151:23:48<204:59:17,  5.02s/it]
 35%|███▍      | 77878/225000 [151:23:51<177:48:17,  4.35s/it]
 35%|███▍      | 77879/225000 [151:23:54<161:45:06,  3.96s/it]
 35%|███▍      | 77880/225000 [151:23:57<147:41:27,  3.61s/it]
 35%|███▍      | 77881/225000 [151:24:00<142:12:30,  3.48s/it]
 35%|███▍      | 77882/225000 [151:24:03<132:35:34,  3.24s/it]
 35%|███▍      | 77883/225000 [151:24:06<131:59:31,  3.23s/it]
 35%|███▍      | 77884/225000 [151:24:13<182:09:21,  4.46s/it]
 35%|███▍      | 77885/225000 [151:24:19<190:40:52,  4.67s/it]
 35%|███▍      | 77886/225000 [151:24:22<170:58:00,  4.18s/it]
 35%|███▍      | 77887/225000 [151:24:27<179:41:56,  4.40s/it]
 35%|███▍      | 77888/225000 [151:24:31<174:50:39,  4.28s/it]
 35%|███▍      | 77889/225000 [151:24:33<153:27:41,  3.76s/it]
 35%|███▍      | 77890/225000 [151:24:40<193:11:04,  4.73s/it]
 35%|███▍      | 77891/225000 [151:24:42<162:59:14,  3.99s/it]
 35%|███▍      | 77892/225000 [151:24:45<145:34:03,  3.56s/it]
 35%|███▍      | 77893/225000 [151:24:47<131:43:33,  3.22s/it]
 35%|███▍      | 77894/225000 [151:24:53<161:10:19,  3.94s/it]
 35%|███▍      | 77895/225000 [151:24:57<164:19:28,  4.02s/it]
 35%|███▍      | 77896/225000 [151:25:00<150:10:57,  3.68s/it]
 35%|███▍      | 77897/225000 [151:25:03<141:08:23,  3.45s/it]
 35%|███▍      | 77898/225000 [151:25:06<140:15:46,  3.43s/it]
 35%|███▍      | 77899/225000 [151:25:10<137:52:40,  3.37s/it]
 35%|███▍      | 77900/225000 [151:25:15<157:13:20,  3.85s/it]
                                                              
{'loss': '0.85', 'grad_norm': '5.095', 'learning_rate': '3.868e-05', 'epoch': '0.3462', 'num_input_tokens_seen': 249528328, 'train_runtime': '5.451e+05', 'train_tokens_per_second': '457.8'}

 35%|███▍      | 77900/225000 [151:25:15<157:13:20,  3.85s/it]
 35%|███▍      | 77901/225000 [151:25:22<206:22:24,  5.05s/it]
 35%|███▍      | 77902/225000 [151:25:25<178:07:44,  4.36s/it]
 35%|███▍      | 77903/225000 [151:25:28<163:56:20,  4.01s/it]
 35%|███▍      | 77904/225000 [151:25:31<149:21:22,  3.66s/it]
 35%|███▍      | 77905/225000 [151:25:35<145:17:21,  3.56s/it]
 35%|███▍      | 77906/225000 [151:25:38<144:42:45,  3.54s/it]
 35%|███▍      | 77907/225000 [151:25:41<135:42:50,  3.32s/it]
 35%|███▍      | 77908/225000 [151:25:44<138:57:37,  3.40s/it]
 35%|███▍      | 77909/225000 [151:25:48<141:34:56,  3.47s/it]
 35%|███▍      | 77910/225000 [151:25:54<166:31:21,  4.08s/it]
 35%|███▍      | 77911/225000 [151:26:02<223:21:24,  5.47s/it]
 35%|███▍      | 77912/225000 [151:26:05<195:34:26,  4.79s/it]
 35%|███▍      | 77913/225000 [151:26:08<169:29:55,  4.15s/it]
 35%|███▍      | 77914/225000 [151:26:14<192:00:50,  4.70s/it]
 35%|███▍      | 77915/225000 [151:26:17<165:34:02,  4.05s/it]
 35%|███▍      | 77916/225000 [151:26:19<150:12:51,  3.68s/it]
 35%|███▍      | 77917/225000 [151:26:26<183:25:49,  4.49s/it]
 35%|███▍      | 77918/225000 [151:26:29<172:35:32,  4.22s/it]
 35%|███▍      | 77919/225000 [151:26:33<164:44:08,  4.03s/it]
 35%|███▍      | 77920/225000 [151:26:37<158:46:55,  3.89s/it]
 35%|███▍      | 77921/225000 [151:26:40<148:45:34,  3.64s/it]
 35%|███▍      | 77922/225000 [151:26:43<148:08:24,  3.63s/it]
 35%|███▍      | 77923/225000 [151:26:47<143:50:37,  3.52s/it]
 35%|███▍      | 77924/225000 [151:26:50<148:21:22,  3.63s/it]
 35%|███▍      | 77925/225000 [151:26:53<139:54:34,  3.42s/it]
 35%|███▍      | 77926/225000 [151:26:56<133:59:05,  3.28s/it]
 35%|███▍      | 77927/225000 [151:27:00<134:00:57,  3.28s/it]
 35%|███▍      | 77928/225000 [151:27:03<141:46:22,  3.47s/it]
 35%|███▍      | 77929/225000 [151:27:12<207:13:06,  5.07s/it]
 35%|███▍      | 77930/225000 [151:27:18<213:18:59,  5.22s/it]
 35%|███▍      | 77931/225000 [151:27:21<182:53:43,  4.48s/it]
 35%|███▍      | 77932/225000 [151:27:23<162:14:53,  3.97s/it]
 35%|███▍      | 77933/225000 [151:27:26<146:39:00,  3.59s/it]
 35%|███▍      | 77934/225000 [151:27:29<134:18:07,  3.29s/it]
 35%|███▍      | 77935/225000 [151:27:34<158:01:50,  3.87s/it]
 35%|███▍      | 77936/225000 [151:27:43<216:05:10,  5.29s/it]
 35%|███▍      | 77937/225000 [151:27:45<182:12:59,  4.46s/it]
 35%|███▍      | 77938/225000 [151:27:48<169:16:38,  4.14s/it]
 35%|███▍      | 77939/225000 [151:27:52<160:40:49,  3.93s/it]
 35%|███▍      | 77940/225000 [151:27:55<151:42:37,  3.71s/it]
 35%|███▍      | 77941/225000 [151:27:58<144:51:29,  3.55s/it]
 35%|███▍      | 77942/225000 [151:28:02<142:11:39,  3.48s/it]
 35%|███▍      | 77943/225000 [151:28:04<135:17:06,  3.31s/it]
 35%|███▍      | 77944/225000 [151:28:10<160:58:26,  3.94s/it]
 35%|███▍      | 77945/225000 [151:28:18<217:38:41,  5.33s/it]
 35%|███▍      | 77946/225000 [151:28:23<211:14:41,  5.17s/it]
 35%|███▍      | 77947/225000 [151:28:26<178:45:09,  4.38s/it]
 35%|███▍      | 77948/225000 [151:28:33<214:36:59,  5.25s/it]
 35%|███▍      | 77949/225000 [151:28:37<198:41:53,  4.86s/it]
 35%|███▍      | 77950/225000 [151:28:40<175:31:40,  4.30s/it]
                                                              
{'loss': '0.8117', 'grad_norm': '3.107', 'learning_rate': '3.867e-05', 'epoch': '0.3464', 'num_input_tokens_seen': 249612768, 'train_runtime': '5.453e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 77950/225000 [151:28:40<175:31:40,  4.30s/it]
 35%|███▍      | 77951/225000 [151:28:43<161:33:50,  3.96s/it]
 35%|███▍      | 77952/225000 [151:28:46<147:36:08,  3.61s/it]
 35%|███▍      | 77953/225000 [151:28:49<136:49:44,  3.35s/it]
 35%|███▍      | 77954/225000 [151:28:52<134:49:40,  3.30s/it]
 35%|███▍      | 77955/225000 [151:29:01<201:09:26,  4.92s/it]
 35%|███▍      | 77956/225000 [151:29:05<191:37:42,  4.69s/it]
 35%|███▍      | 77957/225000 [151:29:08<173:00:08,  4.24s/it]
 35%|███▍      | 77958/225000 [151:29:11<156:35:52,  3.83s/it]
 35%|███▍      | 77959/225000 [151:29:13<141:28:13,  3.46s/it]
 35%|███▍      | 77960/225000 [151:29:16<131:30:33,  3.22s/it]
 35%|███▍      | 77961/225000 [151:29:19<129:55:23,  3.18s/it]
 35%|███▍      | 77962/225000 [151:29:25<157:06:36,  3.85s/it]
 35%|███▍      | 77963/225000 [151:29:32<205:29:45,  5.03s/it]
 35%|███▍      | 77964/225000 [151:29:36<193:13:18,  4.73s/it]
 35%|███▍      | 77965/225000 [151:29:39<169:21:32,  4.15s/it]
 35%|███▍      | 77966/225000 [151:29:42<149:26:46,  3.66s/it]
 35%|███▍      | 77967/225000 [151:29:46<157:10:29,  3.85s/it]
 35%|███▍      | 77968/225000 [151:29:52<177:50:13,  4.35s/it]
 35%|███▍      | 77969/225000 [151:29:54<156:35:39,  3.83s/it]
 35%|███▍      | 77970/225000 [151:29:57<143:40:50,  3.52s/it]
 35%|███▍      | 77971/225000 [151:30:00<140:36:51,  3.44s/it]
 35%|███▍      | 77972/225000 [151:30:03<132:38:01,  3.25s/it]
 35%|███▍      | 77973/225000 [151:30:06<132:21:08,  3.24s/it]
 35%|███▍      | 77974/225000 [151:30:11<150:06:36,  3.68s/it]
 35%|███▍      | 77975/225000 [151:30:21<230:55:31,  5.65s/it]
 35%|███▍      | 77976/225000 [151:30:24<200:16:59,  4.90s/it]
 35%|███▍      | 77977/225000 [151:30:28<179:06:38,  4.39s/it]
 35%|███▍      | 77978/225000 [151:30:30<161:43:43,  3.96s/it]
 35%|███▍      | 77979/225000 [151:30:33<144:58:35,  3.55s/it]
 35%|███▍      | 77980/225000 [151:30:37<152:00:45,  3.72s/it]
 35%|███▍      | 77981/225000 [151:30:45<200:37:01,  4.91s/it]
 35%|███▍      | 77982/225000 [151:30:48<175:33:52,  4.30s/it]
 35%|███▍      | 77983/225000 [151:30:51<158:17:03,  3.88s/it]
 35%|███▍      | 77984/225000 [151:30:58<203:06:41,  4.97s/it]
 35%|███▍      | 77985/225000 [151:31:01<171:10:30,  4.19s/it]
 35%|███▍      | 77986/225000 [151:31:03<148:07:23,  3.63s/it]
 35%|███▍      | 77987/225000 [151:31:06<145:33:39,  3.56s/it]
 35%|███▍      | 77988/225000 [151:31:10<142:56:11,  3.50s/it]
 35%|███▍      | 77989/225000 [151:31:13<138:11:12,  3.38s/it]
 35%|███▍      | 77990/225000 [151:31:16<135:21:11,  3.31s/it]
 35%|███▍      | 77991/225000 [151:31:24<193:49:23,  4.75s/it]
 35%|███▍      | 77992/225000 [151:31:30<210:32:50,  5.16s/it]
 35%|███▍      | 77993/225000 [151:31:33<180:52:56,  4.43s/it]
 35%|███▍      | 77994/225000 [151:31:35<159:25:18,  3.90s/it]
 35%|███▍      | 77995/225000 [151:31:39<152:04:52,  3.72s/it]
 35%|███▍      | 77996/225000 [151:31:42<144:12:08,  3.53s/it]
 35%|███▍      | 77997/225000 [151:31:46<149:49:27,  3.67s/it]
 35%|███▍      | 77998/225000 [151:31:50<161:00:51,  3.94s/it]
 35%|███▍      | 77999/225000 [151:32:00<223:55:56,  5.48s/it]
 35%|███▍      | 78000/225000 [151:32:02<190:37:32,  4.67s/it]
                                                              
{'loss': '0.8107', 'grad_norm': '3.391', 'learning_rate': '3.865e-05', 'epoch': '0.3467', 'num_input_tokens_seen': 249696856, 'train_runtime': '5.455e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 78000/225000 [151:32:02<190:37:32,  4.67s/it]
 35%|███▍      | 78001/225000 [151:32:06<172:47:11,  4.23s/it]
 35%|███▍      | 78002/225000 [151:32:11<186:22:10,  4.56s/it]
 35%|███▍      | 78003/225000 [151:32:13<157:34:37,  3.86s/it]
 35%|███▍      | 78004/225000 [151:32:16<140:34:54,  3.44s/it]
 35%|███▍      | 78005/225000 [151:32:18<129:04:40,  3.16s/it]
 35%|███▍      | 78006/225000 [151:32:26<186:12:15,  4.56s/it]
 35%|███▍      | 78007/225000 [151:32:29<171:10:25,  4.19s/it]
 35%|███▍      | 78008/225000 [151:32:32<156:13:41,  3.83s/it]
 35%|███▍      | 78009/225000 [151:32:35<144:42:49,  3.54s/it]
 35%|███▍      | 78010/225000 [151:32:38<132:54:14,  3.26s/it]
 35%|███▍      | 78011/225000 [151:32:41<129:30:28,  3.17s/it]
 35%|███▍      | 78012/225000 [151:32:49<190:15:09,  4.66s/it]
 35%|███▍      | 78013/225000 [151:32:53<184:07:46,  4.51s/it]
 35%|███▍      | 78014/225000 [151:32:56<165:19:03,  4.05s/it]
 35%|███▍      | 78015/225000 [151:32:59<153:00:06,  3.75s/it]
 35%|███▍      | 78016/225000 [151:33:02<143:15:21,  3.51s/it]
 35%|███▍      | 78017/225000 [151:33:09<184:57:52,  4.53s/it]
 35%|███▍      | 78018/225000 [151:33:14<196:23:10,  4.81s/it]
 35%|███▍      | 78019/225000 [151:33:17<168:54:05,  4.14s/it]
 35%|███▍      | 78020/225000 [151:33:20<161:29:11,  3.96s/it]
 35%|███▍      | 78021/225000 [151:33:24<157:53:59,  3.87s/it]
 35%|███▍      | 78022/225000 [151:33:30<184:19:55,  4.51s/it]
 35%|███▍      | 78023/225000 [151:33:34<172:22:18,  4.22s/it]
 35%|███▍      | 78024/225000 [151:33:36<153:00:56,  3.75s/it]
 35%|███▍      | 78025/225000 [151:33:39<141:08:31,  3.46s/it]
 35%|███▍      | 78026/225000 [151:33:42<139:27:56,  3.42s/it]
 35%|███▍      | 78027/225000 [151:33:45<134:28:32,  3.29s/it]
 35%|███▍      | 78028/225000 [151:33:49<133:15:55,  3.26s/it]
 35%|███▍      | 78029/225000 [151:33:53<150:11:10,  3.68s/it]
 35%|███▍      | 78030/225000 [151:34:01<206:38:18,  5.06s/it]
 35%|███▍      | 78031/225000 [151:34:05<188:28:55,  4.62s/it]
 35%|███▍      | 78032/225000 [151:34:08<172:48:55,  4.23s/it]
 35%|███▍      | 78033/225000 [151:34:12<159:32:04,  3.91s/it]
 35%|███▍      | 78034/225000 [151:34:17<174:33:43,  4.28s/it]
 35%|███▍      | 78035/225000 [151:34:20<161:12:02,  3.95s/it]
 35%|███▍      | 78036/225000 [151:34:23<145:35:13,  3.57s/it]
 35%|███▍      | 78037/225000 [151:34:25<137:57:49,  3.38s/it]
 35%|███▍      | 78038/225000 [151:34:28<130:20:22,  3.19s/it]
 35%|███▍      | 78039/225000 [151:34:31<122:53:26,  3.01s/it]
 35%|███▍      | 78040/225000 [151:34:39<183:32:45,  4.50s/it]
 35%|███▍      | 78041/225000 [151:34:43<184:36:01,  4.52s/it]
 35%|███▍      | 78042/225000 [151:34:47<170:01:08,  4.16s/it]
 35%|███▍      | 78043/225000 [151:34:51<168:08:10,  4.12s/it]
 35%|███▍      | 78044/225000 [151:34:53<151:19:46,  3.71s/it]
 35%|███▍      | 78045/225000 [151:34:57<144:41:16,  3.54s/it]
 35%|███▍      | 78046/225000 [151:35:00<146:46:27,  3.60s/it]
 35%|███▍      | 78047/225000 [151:35:06<176:26:54,  4.32s/it]
 35%|███▍      | 78048/225000 [151:35:14<221:27:26,  5.43s/it]
 35%|███▍      | 78049/225000 [151:35:17<192:24:44,  4.71s/it]
 35%|███▍      | 78050/225000 [151:35:21<177:29:29,  4.35s/it]
                                                              
{'loss': '0.8864', 'grad_norm': '2.691', 'learning_rate': '3.864e-05', 'epoch': '0.3469', 'num_input_tokens_seen': 249780024, 'train_runtime': '5.457e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 78050/225000 [151:35:21<177:29:29,  4.35s/it]
 35%|███▍      | 78051/225000 [151:35:24<162:49:55,  3.99s/it]
 35%|███▍      | 78052/225000 [151:35:27<151:43:06,  3.72s/it]
 35%|███▍      | 78053/225000 [151:35:30<146:09:51,  3.58s/it]
 35%|███▍      | 78054/225000 [151:35:33<139:32:42,  3.42s/it]
 35%|███▍      | 78055/225000 [151:35:37<146:00:11,  3.58s/it]
 35%|███▍      | 78056/225000 [151:35:41<145:57:31,  3.58s/it]
 35%|███▍      | 78057/225000 [151:35:44<140:19:24,  3.44s/it]
 35%|███▍      | 78058/225000 [151:35:48<142:05:43,  3.48s/it]
 35%|███▍      | 78059/225000 [151:35:54<180:57:33,  4.43s/it]
 35%|███▍      | 78060/225000 [151:36:02<217:32:20,  5.33s/it]
 35%|███▍      | 78061/225000 [151:36:05<193:57:42,  4.75s/it]
 35%|███▍      | 78062/225000 [151:36:08<174:24:38,  4.27s/it]
 35%|███▍      | 78063/225000 [151:36:12<169:36:07,  4.16s/it]
 35%|███▍      | 78064/225000 [151:36:16<161:29:53,  3.96s/it]
 35%|███▍      | 78065/225000 [151:36:19<152:39:06,  3.74s/it]
 35%|███▍      | 78066/225000 [151:36:22<149:31:44,  3.66s/it]
 35%|███▍      | 78067/225000 [151:36:32<217:48:53,  5.34s/it]
 35%|███▍      | 78068/225000 [151:36:36<203:37:50,  4.99s/it]
 35%|███▍      | 78069/225000 [151:36:39<183:22:40,  4.49s/it]
 35%|███▍      | 78070/225000 [151:36:42<163:44:28,  4.01s/it]
 35%|███▍      | 78071/225000 [151:36:45<147:36:17,  3.62s/it]
 35%|███▍      | 78072/225000 [151:36:48<147:04:47,  3.60s/it]
 35%|███▍      | 78073/225000 [151:36:52<142:44:30,  3.50s/it]
 35%|███▍      | 78074/225000 [151:36:55<141:51:42,  3.48s/it]
 35%|███▍      | 78075/225000 [151:36:58<134:47:09,  3.30s/it]
 35%|███▍      | 78076/225000 [151:37:03<162:27:51,  3.98s/it]
 35%|███▍      | 78077/225000 [151:37:12<217:10:50,  5.32s/it]
 35%|███▍      | 78078/225000 [151:37:15<196:26:35,  4.81s/it]
 35%|███▍      | 78079/225000 [151:37:18<165:58:59,  4.07s/it]
 35%|███▍      | 78080/225000 [151:37:21<150:44:14,  3.69s/it]
 35%|███▍      | 78081/225000 [151:37:28<196:52:41,  4.82s/it]
 35%|███▍      | 78082/225000 [151:37:31<169:41:55,  4.16s/it]
 35%|███▍      | 78083/225000 [151:37:34<155:36:32,  3.81s/it]
 35%|███▍      | 78084/225000 [151:37:36<139:15:01,  3.41s/it]
 35%|███▍      | 78085/225000 [151:37:39<130:47:42,  3.20s/it]
 35%|███▍      | 78086/225000 [151:37:46<179:48:24,  4.41s/it]
 35%|███▍      | 78087/225000 [151:37:53<205:12:32,  5.03s/it]
 35%|███▍      | 78088/225000 [151:37:55<174:06:52,  4.27s/it]
 35%|███▍      | 78089/225000 [151:37:58<158:43:36,  3.89s/it]
 35%|███▍      | 78090/225000 [151:38:01<142:30:50,  3.49s/it]
 35%|███▍      | 78091/225000 [151:38:03<132:18:12,  3.24s/it]
 35%|███▍      | 78092/225000 [151:38:11<181:29:35,  4.45s/it]
 35%|███▍      | 78093/225000 [151:38:15<185:02:22,  4.53s/it]
 35%|███▍      | 78094/225000 [151:38:18<158:55:22,  3.89s/it]
 35%|███▍      | 78095/225000 [151:38:20<143:34:47,  3.52s/it]
 35%|███▍      | 78096/225000 [151:38:23<138:48:45,  3.40s/it]
 35%|███▍      | 78097/225000 [151:38:31<183:40:29,  4.50s/it]
 35%|███▍      | 78098/225000 [151:38:35<187:49:39,  4.60s/it]
 35%|███▍      | 78099/225000 [151:38:38<163:13:51,  4.00s/it]
 35%|███▍      | 78100/225000 [151:38:41<153:47:41,  3.77s/it]
                                                              
{'loss': '0.8096', 'grad_norm': '2.373', 'learning_rate': '3.862e-05', 'epoch': '0.3471', 'num_input_tokens_seen': 249864536, 'train_runtime': '5.459e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 78100/225000 [151:38:41<153:47:41,  3.77s/it]
 35%|███▍      | 78101/225000 [151:38:44<141:39:35,  3.47s/it]
 35%|███▍      | 78102/225000 [151:38:49<158:48:06,  3.89s/it]
 35%|███▍      | 78103/225000 [151:38:57<206:40:28,  5.06s/it]
 35%|███▍      | 78104/225000 [151:39:01<191:40:59,  4.70s/it]
 35%|███▍      | 78105/225000 [151:39:04<171:21:19,  4.20s/it]
 35%|███▍      | 78106/225000 [151:39:07<163:59:46,  4.02s/it]
 35%|███▍      | 78107/225000 [151:39:10<151:57:26,  3.72s/it]
 35%|███▍      | 78108/225000 [151:39:13<143:35:45,  3.52s/it]
 35%|███▍      | 78109/225000 [151:39:17<144:58:09,  3.55s/it]
 35%|███▍      | 78110/225000 [151:39:20<144:39:23,  3.55s/it]
 35%|███▍      | 78111/225000 [151:39:23<138:48:29,  3.40s/it]
 35%|███▍      | 78112/225000 [151:39:27<136:08:06,  3.34s/it]
 35%|███▍      | 78113/225000 [151:39:30<135:12:11,  3.31s/it]
 35%|███▍      | 78114/225000 [151:39:33<134:22:38,  3.29s/it]
 35%|███▍      | 78115/225000 [151:39:37<136:03:52,  3.33s/it]
 35%|███▍      | 78116/225000 [151:39:41<143:47:04,  3.52s/it]
 35%|███▍      | 78117/225000 [151:39:44<146:51:55,  3.60s/it]
 35%|███▍      | 78118/225000 [151:39:47<137:32:16,  3.37s/it]
 35%|███▍      | 78119/225000 [151:39:55<195:45:14,  4.80s/it]
 35%|███▍      | 78120/225000 [151:40:01<210:23:15,  5.16s/it]
 35%|███▍      | 78121/225000 [151:40:04<177:11:27,  4.34s/it]
 35%|███▍      | 78122/225000 [151:40:06<155:14:04,  3.80s/it]
 35%|███▍      | 78123/225000 [151:40:13<192:25:29,  4.72s/it]
 35%|███▍      | 78124/225000 [151:40:19<202:44:39,  4.97s/it]
 35%|███▍      | 78125/225000 [151:40:21<175:52:33,  4.31s/it]
 35%|███▍      | 78126/225000 [151:40:28<197:22:35,  4.84s/it]
 35%|███▍      | 78127/225000 [151:40:30<172:09:22,  4.22s/it]
 35%|███▍      | 78128/225000 [151:40:34<160:00:33,  3.92s/it]
 35%|███▍      | 78129/225000 [151:40:37<158:31:08,  3.89s/it]
 35%|███▍      | 78130/225000 [151:40:41<154:37:11,  3.79s/it]
 35%|███▍      | 78131/225000 [151:40:44<150:32:57,  3.69s/it]
 35%|███▍      | 78132/225000 [151:40:47<141:09:44,  3.46s/it]
 35%|███▍      | 78133/225000 [151:40:50<136:05:17,  3.34s/it]
 35%|███▍      | 78134/225000 [151:40:53<133:00:34,  3.26s/it]
 35%|███▍      | 78135/225000 [151:40:59<161:22:52,  3.96s/it]
 35%|███▍      | 78136/225000 [151:41:07<213:21:41,  5.23s/it]
 35%|███▍      | 78137/225000 [151:41:11<195:10:03,  4.78s/it]
 35%|███▍      | 78138/225000 [151:41:13<164:25:15,  4.03s/it]
 35%|███▍      | 78139/225000 [151:41:16<145:48:55,  3.57s/it]
 35%|███▍      | 78140/225000 [151:41:18<132:45:30,  3.25s/it]
 35%|███▍      | 78141/225000 [151:41:21<130:04:55,  3.19s/it]
 35%|███▍      | 78142/225000 [151:41:30<193:20:13,  4.74s/it]
 35%|███▍      | 78143/225000 [151:41:34<188:18:27,  4.62s/it]
 35%|███▍      | 78144/225000 [151:41:36<161:29:29,  3.96s/it]
 35%|███▍      | 78145/225000 [151:41:44<201:23:20,  4.94s/it]
 35%|███▍      | 78146/225000 [151:41:47<178:32:43,  4.38s/it]
 35%|███▍      | 78147/225000 [151:41:50<166:55:49,  4.09s/it]
 35%|███▍      | 78148/225000 [151:41:54<159:31:48,  3.91s/it]
 35%|███▍      | 78149/225000 [151:41:57<150:41:32,  3.69s/it]
 35%|███▍      | 78150/225000 [151:42:00<143:38:49,  3.52s/it]
                                                              
{'loss': '0.8509', 'grad_norm': '2.558', 'learning_rate': '3.861e-05', 'epoch': '0.3473', 'num_input_tokens_seen': 249949512, 'train_runtime': '5.461e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 78150/225000 [151:42:00<143:38:49,  3.52s/it]
 35%|███▍      | 78151/225000 [151:42:03<144:08:36,  3.53s/it]
 35%|███▍      | 78152/225000 [151:42:07<140:21:39,  3.44s/it]
 35%|███▍      | 78153/225000 [151:42:10<138:37:40,  3.40s/it]
 35%|███▍      | 78154/225000 [151:42:13<137:17:32,  3.37s/it]
 35%|███▍      | 78155/225000 [151:42:22<206:16:05,  5.06s/it]
 35%|███▍      | 78156/225000 [151:42:27<208:13:31,  5.10s/it]
 35%|███▍      | 78157/225000 [151:42:30<173:08:34,  4.24s/it]
 35%|███▍      | 78158/225000 [151:42:32<150:35:18,  3.69s/it]
 35%|███▍      | 78159/225000 [151:42:38<177:13:18,  4.34s/it]
 35%|███▍      | 78160/225000 [151:42:43<186:11:53,  4.56s/it]
 35%|███▍      | 78161/225000 [151:42:46<161:21:12,  3.96s/it]
 35%|███▍      | 78162/225000 [151:42:48<145:02:41,  3.56s/it]
 35%|███▍      | 78163/225000 [151:42:52<153:08:12,  3.75s/it]
 35%|███▍      | 78164/225000 [151:42:59<189:25:19,  4.64s/it]
 35%|███▍      | 78165/225000 [151:43:04<191:50:51,  4.70s/it]
 35%|███▍      | 78166/225000 [151:43:06<162:30:57,  3.98s/it]
 35%|███▍      | 78167/225000 [151:43:13<201:05:34,  4.93s/it]
 35%|███▍      | 78168/225000 [151:43:16<177:58:09,  4.36s/it]
 35%|███▍      | 78169/225000 [151:43:20<163:18:09,  4.00s/it]
 35%|███▍      | 78170/225000 [151:43:22<147:28:11,  3.62s/it]
 35%|███▍      | 78171/225000 [151:43:25<137:05:48,  3.36s/it]
 35%|███▍      | 78172/225000 [151:43:31<166:56:48,  4.09s/it]
 35%|███▍      | 78173/225000 [151:43:38<200:55:01,  4.93s/it]
 35%|███▍      | 78174/225000 [151:43:42<186:48:03,  4.58s/it]
 35%|███▍      | 78175/225000 [151:43:45<169:16:40,  4.15s/it]
 35%|███▍      | 78176/225000 [151:43:48<164:42:20,  4.04s/it]
 35%|███▍      | 78177/225000 [151:43:54<184:38:32,  4.53s/it]
 35%|███▍      | 78178/225000 [151:43:57<167:44:49,  4.11s/it]
 35%|███▍      | 78179/225000 [151:44:03<186:33:16,  4.57s/it]
 35%|███▍      | 78180/225000 [151:44:09<199:48:28,  4.90s/it]
 35%|███▍      | 78181/225000 [151:44:11<171:28:00,  4.20s/it]
 35%|███▍      | 78182/225000 [151:44:17<194:00:12,  4.76s/it]
 35%|███▍      | 78183/225000 [151:44:22<199:27:31,  4.89s/it]
 35%|███▍      | 78184/225000 [151:44:25<173:47:43,  4.26s/it]
 35%|███▍      | 78185/225000 [151:44:28<152:32:47,  3.74s/it]
 35%|███▍      | 78186/225000 [151:44:31<150:29:19,  3.69s/it]
 35%|███▍      | 78187/225000 [151:44:40<213:40:38,  5.24s/it]
 35%|███▍      | 78188/225000 [151:44:43<183:14:49,  4.49s/it]
 35%|███▍      | 78189/225000 [151:44:46<159:45:40,  3.92s/it]
 35%|███▍      | 78190/225000 [151:44:52<188:18:02,  4.62s/it]
 35%|███▍      | 78191/225000 [151:44:54<159:11:53,  3.90s/it]
 35%|███▍      | 78192/225000 [151:44:57<146:33:34,  3.59s/it]
 35%|███▍      | 78193/225000 [151:45:00<143:10:58,  3.51s/it]
 35%|███▍      | 78194/225000 [151:45:03<138:26:52,  3.40s/it]
 35%|███▍      | 78195/225000 [151:45:07<144:28:19,  3.54s/it]
 35%|███▍      | 78196/225000 [151:45:11<147:59:03,  3.63s/it]
 35%|███▍      | 78197/225000 [151:45:15<147:06:46,  3.61s/it]
 35%|███▍      | 78198/225000 [151:45:18<141:45:39,  3.48s/it]
 35%|███▍      | 78199/225000 [151:45:26<204:17:08,  5.01s/it]
 35%|███▍      | 78200/225000 [151:45:29<174:45:51,  4.29s/it]
                                                              
{'loss': '0.7908', 'grad_norm': '3.363', 'learning_rate': '3.859e-05', 'epoch': '0.3476', 'num_input_tokens_seen': 250035520, 'train_runtime': '5.463e+05', 'train_tokens_per_second': '457.7'}

 35%|███▍      | 78200/225000 [151:45:29<174:45:51,  4.29s/it]
 35%|███▍      | 78201/225000 [151:45:32<157:04:39,  3.85s/it]
 35%|███▍      | 78202/225000 [151:45:34<143:09:29,  3.51s/it]
 35%|███▍      | 78203/225000 [151:45:39<152:36:10,  3.74s/it]
 35%|███▍      | 78204/225000 [151:45:45<184:39:35,  4.53s/it]
 35%|███▍      | 78205/225000 [151:45:48<159:22:31,  3.91s/it]
 35%|███▍      | 78206/225000 [151:45:50<144:24:45,  3.54s/it]
 35%|███▍      | 78207/225000 [151:45:53<136:56:38,  3.36s/it]
 35%|███▍      | 78208/225000 [151:45:56<129:41:04,  3.18s/it]
 35%|███▍      | 78209/225000 [151:46:04<189:16:59,  4.64s/it]
 35%|███▍      | 78210/225000 [151:46:08<179:29:13,  4.40s/it]
 35%|███▍      | 78211/225000 [151:46:10<154:55:27,  3.80s/it]
 35%|███▍      | 78212/225000 [151:46:13<140:47:04,  3.45s/it]
 35%|███▍      | 78213/225000 [151:46:19<172:26:54,  4.23s/it]
 35%|███▍      | 78214/225000 [151:46:24<178:18:49,  4.37s/it]
 35%|███▍      | 78215/225000 [151:46:26<153:42:57,  3.77s/it]
 35%|███▍      | 78216/225000 [151:46:29<139:35:07,  3.42s/it]
 35%|███▍      | 78217/225000 [151:46:31<129:23:54,  3.17s/it]
 35%|███▍      | 78218/225000 [151:46:40<198:36:46,  4.87s/it]
 35%|███▍      | 78219/225000 [151:46:44<188:41:39,  4.63s/it]
 35%|███▍      | 78220/225000 [151:46:48<173:48:08,  4.26s/it]
 35%|███▍      | 78221/225000 [151:46:51<163:22:34,  4.01s/it]
 35%|███▍      | 78222/225000 [151:46:54<151:06:49,  3.71s/it]
 35%|███▍      | 78223/225000 [151:46:57<143:01:58,  3.51s/it]
 35%|███▍      | 78224/225000 [151:47:00<137:15:25,  3.37s/it]
 35%|███▍      | 78225/225000 [151:47:03<131:23:39,  3.22s/it]
 35%|███▍      | 78226/225000 [151:47:06<129:05:37,  3.17s/it]
 35%|███▍      | 78227/225000 [151:47:12<168:51:04,  4.14s/it]
 35%|███▍      | 78228/225000 [151:47:22<230:14:38,  5.65s/it]
 35%|███▍      | 78229/225000 [151:47:26<216:34:37,  5.31s/it]
 35%|███▍      | 78230/225000 [151:47:29<187:25:35,  4.60s/it]
 35%|███▍      | 78231/225000 [151:47:32<169:03:26,  4.15s/it]
 35%|███▍      | 78232/225000 [151:47:35<151:10:53,  3.71s/it]
 35%|███▍      | 78233/225000 [151:47:38<141:12:45,  3.46s/it]
 35%|███▍      | 78234/225000 [151:47:41<135:27:09,  3.32s/it]
 35%|███▍      | 78235/225000 [151:47:43<126:53:43,  3.11s/it]
 35%|███▍      | 78236/225000 [151:47:46<124:02:44,  3.04s/it]
 35%|███▍      | 78237/225000 [151:47:54<177:14:27,  4.35s/it]
 35%|███▍      | 78238/225000 [151:48:01<210:02:35,  5.15s/it]
 35%|███▍      | 78239/225000 [151:48:03<174:37:18,  4.28s/it]
 35%|███▍      | 78240/225000 [151:48:06<155:02:49,  3.80s/it]
 35%|███▍      | 78241/225000 [151:48:09<146:27:07,  3.59s/it]
 35%|███▍      | 78242/225000 [151:48:12<139:34:10,  3.42s/it]
 35%|███▍      | 78243/225000 [151:48:15<137:36:01,  3.38s/it]
 35%|███▍      | 78244/225000 [151:48:21<170:12:54,  4.18s/it]
 35%|███▍      | 78245/225000 [151:48:29<221:51:33,  5.44s/it]
 35%|███▍      | 78246/225000 [151:48:32<183:40:08,  4.51s/it]
 35%|███▍      | 78247/225000 [151:48:34<162:51:31,  4.00s/it]
 35%|███▍      | 78248/225000 [151:48:37<143:16:56,  3.51s/it]
 35%|███▍      | 78249/225000 [151:48:43<174:52:04,  4.29s/it]
 35%|███▍      | 78250/225000 [151:48:49<195:10:07,  4.79s/it]
                                                              
{'loss': '0.8539', 'grad_norm': '2.792', 'learning_rate': '3.858e-05', 'epoch': '0.3478', 'num_input_tokens_seen': 250119024, 'train_runtime': '5.465e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78250/225000 [151:48:49<195:10:07,  4.79s/it]
 35%|███▍      | 78251/225000 [151:48:52<171:07:01,  4.20s/it]
 35%|███▍      | 78252/225000 [151:48:55<154:15:31,  3.78s/it]
 35%|███▍      | 78253/225000 [151:48:57<140:45:10,  3.45s/it]
 35%|███▍      | 78254/225000 [151:49:00<130:55:04,  3.21s/it]
 35%|███▍      | 78255/225000 [151:49:05<153:03:28,  3.75s/it]
 35%|███▍      | 78256/225000 [151:49:12<194:39:07,  4.78s/it]
 35%|███▍      | 78257/225000 [151:49:15<173:15:07,  4.25s/it]
 35%|███▍      | 78258/225000 [151:49:19<166:10:26,  4.08s/it]
 35%|███▍      | 78259/225000 [151:49:23<164:33:31,  4.04s/it]
 35%|███▍      | 78260/225000 [151:49:26<156:41:47,  3.84s/it]
 35%|███▍      | 78261/225000 [151:49:29<147:37:20,  3.62s/it]
 35%|███▍      | 78262/225000 [151:49:32<139:34:42,  3.42s/it]
 35%|███▍      | 78263/225000 [151:49:39<186:09:30,  4.57s/it]
 35%|███▍      | 78264/225000 [151:49:44<187:59:08,  4.61s/it]
 35%|███▍      | 78265/225000 [151:49:47<162:50:22,  4.00s/it]
 35%|███▍      | 78266/225000 [151:49:50<150:07:15,  3.68s/it]
 35%|███▍      | 78267/225000 [151:49:53<147:57:37,  3.63s/it]
 35%|███▍      | 78268/225000 [151:49:56<142:06:47,  3.49s/it]
 35%|███▍      | 78269/225000 [151:49:59<134:47:12,  3.31s/it]
 35%|███▍      | 78270/225000 [151:50:02<131:21:57,  3.22s/it]
 35%|███▍      | 78271/225000 [151:50:11<194:00:40,  4.76s/it]
 35%|███▍      | 78272/225000 [151:50:20<246:07:26,  6.04s/it]
 35%|███▍      | 78273/225000 [151:50:22<204:48:55,  5.03s/it]
 35%|███▍      | 78274/225000 [151:50:28<218:40:27,  5.37s/it]
 35%|███▍      | 78275/225000 [151:50:31<180:40:21,  4.43s/it]
 35%|███▍      | 78276/225000 [151:50:33<159:15:04,  3.91s/it]
 35%|███▍      | 78277/225000 [151:50:36<147:53:52,  3.63s/it]
 35%|███▍      | 78278/225000 [151:50:39<138:32:18,  3.40s/it]
 35%|███▍      | 78279/225000 [151:50:42<130:32:54,  3.20s/it]
 35%|███▍      | 78280/225000 [151:50:51<197:40:24,  4.85s/it]
 35%|███▍      | 78281/225000 [151:50:54<179:19:51,  4.40s/it]
 35%|███▍      | 78282/225000 [151:50:56<155:00:46,  3.80s/it]
 35%|███▍      | 78283/225000 [151:51:00<156:01:31,  3.83s/it]
 35%|███▍      | 78284/225000 [151:51:10<222:57:10,  5.47s/it]
 35%|███▍      | 78285/225000 [151:51:13<195:13:31,  4.79s/it]
 35%|███▍      | 78286/225000 [151:51:15<165:38:42,  4.06s/it]
 35%|███▍      | 78287/225000 [151:51:18<151:15:47,  3.71s/it]
 35%|███▍      | 78288/225000 [151:51:22<152:23:25,  3.74s/it]
 35%|███▍      | 78289/225000 [151:51:25<145:17:09,  3.57s/it]
 35%|███▍      | 78290/225000 [151:51:28<142:28:44,  3.50s/it]
 35%|███▍      | 78291/225000 [151:51:31<136:22:49,  3.35s/it]
 35%|███▍      | 78292/225000 [151:51:35<142:21:04,  3.49s/it]
 35%|███▍      | 78293/225000 [151:51:38<137:24:43,  3.37s/it]
 35%|███▍      | 78294/225000 [151:51:45<182:21:04,  4.47s/it]
 35%|███▍      | 78295/225000 [151:51:54<229:50:05,  5.64s/it]
 35%|███▍      | 78296/225000 [151:51:57<196:33:52,  4.82s/it]
 35%|███▍      | 78297/225000 [151:51:59<170:30:05,  4.18s/it]
 35%|███▍      | 78298/225000 [151:52:02<153:00:26,  3.75s/it]
 35%|███▍      | 78299/225000 [151:52:09<189:18:17,  4.65s/it]
 35%|███▍      | 78300/225000 [151:52:12<166:13:32,  4.08s/it]
                                                              
{'loss': '0.8354', 'grad_norm': '3.821', 'learning_rate': '3.856e-05', 'epoch': '0.348', 'num_input_tokens_seen': 250204304, 'train_runtime': '5.467e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78300/225000 [151:52:12<166:13:32,  4.08s/it]
 35%|███▍      | 78301/225000 [151:52:15<156:34:28,  3.84s/it]
 35%|███▍      | 78302/225000 [151:52:18<144:03:53,  3.54s/it]
 35%|███▍      | 78303/225000 [151:52:21<141:28:33,  3.47s/it]
 35%|███▍      | 78304/225000 [151:52:24<139:27:38,  3.42s/it]
 35%|███▍      | 78305/225000 [151:52:27<136:48:35,  3.36s/it]
 35%|███▍      | 78306/225000 [151:52:35<182:04:29,  4.47s/it]
 35%|███▍      | 78307/225000 [151:52:42<219:07:18,  5.38s/it]
 35%|███▍      | 78308/225000 [151:52:45<191:20:23,  4.70s/it]
 35%|███▍      | 78309/225000 [151:52:48<167:50:29,  4.12s/it]
 35%|███▍      | 78310/225000 [151:52:51<152:56:20,  3.75s/it]
 35%|███▍      | 78311/225000 [151:52:54<140:39:30,  3.45s/it]
 35%|███▍      | 78312/225000 [151:52:57<139:28:46,  3.42s/it]
 35%|███▍      | 78313/225000 [151:53:00<134:36:55,  3.30s/it]
 35%|███▍      | 78314/225000 [151:53:10<222:03:46,  5.45s/it]
 35%|███▍      | 78315/225000 [151:53:14<198:24:27,  4.87s/it]
 35%|███▍      | 78316/225000 [151:53:17<170:53:19,  4.19s/it]
 35%|███▍      | 78317/225000 [151:53:20<158:40:37,  3.89s/it]
 35%|███▍      | 78318/225000 [151:53:23<150:21:12,  3.69s/it]
 35%|███▍      | 78319/225000 [151:53:26<138:39:39,  3.40s/it]
 35%|███▍      | 78320/225000 [151:53:29<133:21:39,  3.27s/it]
 35%|███▍      | 78321/225000 [151:53:33<143:32:30,  3.52s/it]
 35%|███▍      | 78322/225000 [151:53:43<225:44:54,  5.54s/it]
 35%|███▍      | 78323/225000 [151:53:47<208:26:08,  5.12s/it]
 35%|███▍      | 78324/225000 [151:53:49<174:30:16,  4.28s/it]
 35%|███▍      | 78325/225000 [151:53:53<161:17:41,  3.96s/it]
 35%|███▍      | 78326/225000 [151:53:56<152:12:34,  3.74s/it]
 35%|███▍      | 78327/225000 [151:54:03<195:22:09,  4.80s/it]
 35%|███▍      | 78328/225000 [151:54:06<167:11:57,  4.10s/it]
 35%|███▍      | 78329/225000 [151:54:08<151:07:17,  3.71s/it]
 35%|███▍      | 78330/225000 [151:54:12<144:52:54,  3.56s/it]
 35%|███▍      | 78331/225000 [151:54:20<203:02:34,  4.98s/it]
 35%|███▍      | 78332/225000 [151:54:23<180:06:20,  4.42s/it]
 35%|███▍      | 78333/225000 [151:54:26<164:19:23,  4.03s/it]
 35%|███▍      | 78334/225000 [151:54:29<150:37:52,  3.70s/it]
 35%|███▍      | 78335/225000 [151:54:32<145:03:14,  3.56s/it]
 35%|███▍      | 78336/225000 [151:54:36<141:57:08,  3.48s/it]
 35%|███▍      | 78337/225000 [151:54:39<137:27:06,  3.37s/it]
 35%|███▍      | 78338/225000 [151:54:41<128:39:47,  3.16s/it]
 35%|███▍      | 78339/225000 [151:54:45<128:39:58,  3.16s/it]
 35%|███▍      | 78340/225000 [151:54:50<154:50:29,  3.80s/it]
 35%|███▍      | 78341/225000 [151:54:59<214:46:00,  5.27s/it]
 35%|███▍      | 78342/225000 [151:55:04<219:12:43,  5.38s/it]
 35%|███▍      | 78343/225000 [151:55:07<193:28:04,  4.75s/it]
 35%|███▍      | 78344/225000 [151:55:11<176:19:34,  4.33s/it]
 35%|███▍      | 78345/225000 [151:55:14<160:18:19,  3.94s/it]
 35%|███▍      | 78346/225000 [151:55:17<148:48:33,  3.65s/it]
 35%|███▍      | 78347/225000 [151:55:20<139:44:56,  3.43s/it]
 35%|███▍      | 78348/225000 [151:55:22<131:31:25,  3.23s/it]
 35%|███▍      | 78349/225000 [151:55:32<202:57:05,  4.98s/it]
 35%|███▍      | 78350/225000 [151:55:35<186:43:16,  4.58s/it]
                                                              
{'loss': '0.7994', 'grad_norm': '3.972', 'learning_rate': '3.855e-05', 'epoch': '0.3482', 'num_input_tokens_seen': 250289552, 'train_runtime': '5.469e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78350/225000 [151:55:35<186:43:16,  4.58s/it]
 35%|███▍      | 78351/225000 [151:55:38<168:32:13,  4.14s/it]
 35%|███▍      | 78352/225000 [151:55:41<151:31:30,  3.72s/it]
 35%|███▍      | 78353/225000 [151:55:44<138:56:46,  3.41s/it]
 35%|███▍      | 78354/225000 [151:55:47<132:51:54,  3.26s/it]
 35%|███▍      | 78355/225000 [151:55:50<129:49:18,  3.19s/it]
 35%|███▍      | 78356/225000 [151:55:58<196:27:00,  4.82s/it]
 35%|███▍      | 78357/225000 [151:56:05<220:54:14,  5.42s/it]
 35%|███▍      | 78358/225000 [151:56:08<190:34:50,  4.68s/it]
 35%|███▍      | 78359/225000 [151:56:11<165:45:14,  4.07s/it]
 35%|███▍      | 78360/225000 [151:56:14<153:58:31,  3.78s/it]
 35%|███▍      | 78361/225000 [151:56:17<142:19:42,  3.49s/it]
 35%|███▍      | 78362/225000 [151:56:19<130:05:22,  3.19s/it]
 35%|███▍      | 78363/225000 [151:56:27<183:40:06,  4.51s/it]
 35%|███▍      | 78364/225000 [151:56:32<188:21:39,  4.62s/it]
 35%|███▍      | 78365/225000 [151:56:34<164:38:57,  4.04s/it]
 35%|███▍      | 78366/225000 [151:56:37<149:37:54,  3.67s/it]
 35%|███▍      | 78367/225000 [151:56:40<139:58:40,  3.44s/it]
 35%|███▍      | 78368/225000 [151:56:46<176:17:25,  4.33s/it]
 35%|███▍      | 78369/225000 [151:56:54<213:49:17,  5.25s/it]
 35%|███▍      | 78370/225000 [151:56:57<187:11:08,  4.60s/it]
 35%|███▍      | 78371/225000 [151:57:00<165:33:52,  4.06s/it]
 35%|███▍      | 78372/225000 [151:57:02<146:28:56,  3.60s/it]
 35%|███▍      | 78373/225000 [151:57:07<161:57:46,  3.98s/it]
 35%|███▍      | 78374/225000 [151:57:15<214:45:40,  5.27s/it]
 35%|███▍      | 78375/225000 [151:57:18<184:16:18,  4.52s/it]
 35%|███▍      | 78376/225000 [151:57:21<159:58:11,  3.93s/it]
 35%|███▍      | 78377/225000 [151:57:23<145:15:52,  3.57s/it]
 35%|███▍      | 78378/225000 [151:57:26<134:35:21,  3.30s/it]
 35%|███▍      | 78379/225000 [151:57:29<131:05:33,  3.22s/it]
 35%|███▍      | 78380/225000 [151:57:36<172:19:36,  4.23s/it]
 35%|███▍      | 78381/225000 [151:57:43<214:36:05,  5.27s/it]
 35%|███▍      | 78382/225000 [151:57:46<177:17:25,  4.35s/it]
 35%|███▍      | 78383/225000 [151:57:48<155:38:32,  3.82s/it]
 35%|███▍      | 78384/225000 [151:57:51<140:40:38,  3.45s/it]
 35%|███▍      | 78385/225000 [151:57:54<140:23:37,  3.45s/it]
 35%|███▍      | 78386/225000 [151:57:58<138:47:58,  3.41s/it]
 35%|███▍      | 78387/225000 [151:58:01<139:35:41,  3.43s/it]
 35%|███▍      | 78388/225000 [151:58:04<133:45:17,  3.28s/it]
 35%|███▍      | 78389/225000 [151:58:09<149:21:36,  3.67s/it]
 35%|███▍      | 78390/225000 [151:58:16<199:31:51,  4.90s/it]
 35%|███▍      | 78391/225000 [151:58:20<182:19:27,  4.48s/it]
 35%|███▍      | 78392/225000 [151:58:23<163:17:25,  4.01s/it]
 35%|███▍      | 78393/225000 [151:58:26<152:31:24,  3.75s/it]
 35%|███▍      | 78394/225000 [151:58:28<138:08:59,  3.39s/it]
 35%|███▍      | 78395/225000 [151:58:36<194:36:41,  4.78s/it]
 35%|███▍      | 78396/225000 [151:58:41<196:08:55,  4.82s/it]
 35%|███▍      | 78397/225000 [151:58:44<172:00:19,  4.22s/it]
 35%|███▍      | 78398/225000 [151:58:47<153:35:19,  3.77s/it]
 35%|███▍      | 78399/225000 [151:58:52<175:46:12,  4.32s/it]
 35%|███▍      | 78400/225000 [151:58:57<184:10:24,  4.52s/it]
                                                              
{'loss': '0.8553', 'grad_norm': '2.359', 'learning_rate': '3.853e-05', 'epoch': '0.3484', 'num_input_tokens_seen': 250373616, 'train_runtime': '5.471e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78400/225000 [151:58:57<184:10:24,  4.52s/it]
 35%|███▍      | 78401/225000 [151:59:01<169:57:52,  4.17s/it]
 35%|███▍      | 78402/225000 [151:59:04<157:16:03,  3.86s/it]
 35%|███▍      | 78403/225000 [151:59:07<152:00:16,  3.73s/it]
 35%|███▍      | 78404/225000 [151:59:10<143:44:23,  3.53s/it]
 35%|███▍      | 78405/225000 [151:59:13<136:39:56,  3.36s/it]
 35%|███▍      | 78406/225000 [151:59:19<158:50:15,  3.90s/it]
 35%|███▍      | 78407/225000 [151:59:27<214:00:42,  5.26s/it]
 35%|███▍      | 78408/225000 [151:59:30<189:07:29,  4.64s/it]
 35%|███▍      | 78409/225000 [151:59:33<164:32:41,  4.04s/it]
 35%|███▍      | 78410/225000 [151:59:37<168:03:59,  4.13s/it]
 35%|███▍      | 78411/225000 [151:59:47<235:56:28,  5.79s/it]
 35%|███▍      | 78412/225000 [151:59:49<192:34:18,  4.73s/it]
 35%|███▍      | 78413/225000 [151:59:52<173:16:05,  4.26s/it]
 35%|███▍      | 78414/225000 [151:59:59<202:55:46,  4.98s/it]
 35%|███▍      | 78415/225000 [152:00:02<177:27:05,  4.36s/it]
 35%|███▍      | 78416/225000 [152:00:05<157:07:11,  3.86s/it]
 35%|███▍      | 78417/225000 [152:00:07<146:12:17,  3.59s/it]
 35%|███▍      | 78418/225000 [152:00:10<138:21:48,  3.40s/it]
 35%|███▍      | 78419/225000 [152:00:13<129:37:29,  3.18s/it]
 35%|███▍      | 78420/225000 [152:00:21<188:36:16,  4.63s/it]
 35%|███▍      | 78421/225000 [152:00:28<211:50:23,  5.20s/it]
 35%|███▍      | 78422/225000 [152:00:30<181:40:16,  4.46s/it]
 35%|███▍      | 78423/225000 [152:00:36<191:16:56,  4.70s/it]
 35%|███▍      | 78424/225000 [152:00:38<167:56:34,  4.12s/it]
 35%|███▍      | 78425/225000 [152:00:41<151:50:52,  3.73s/it]
 35%|███▍      | 78426/225000 [152:00:45<150:26:09,  3.69s/it]
 35%|███▍      | 78427/225000 [152:00:48<137:39:00,  3.38s/it]
 35%|███▍      | 78428/225000 [152:00:51<140:14:32,  3.44s/it]
 35%|███▍      | 78429/225000 [152:00:54<137:17:59,  3.37s/it]
 35%|███▍      | 78430/225000 [152:00:57<133:55:28,  3.29s/it]
 35%|███▍      | 78431/225000 [152:01:01<133:32:06,  3.28s/it]
 35%|███▍      | 78432/225000 [152:01:04<132:43:11,  3.26s/it]
 35%|███▍      | 78433/225000 [152:01:09<159:32:29,  3.92s/it]
 35%|███▍      | 78434/225000 [152:01:18<214:48:27,  5.28s/it]
 35%|███▍      | 78435/225000 [152:01:20<182:57:56,  4.49s/it]
 35%|███▍      | 78436/225000 [152:01:24<166:49:54,  4.10s/it]
 35%|███▍      | 78437/225000 [152:01:31<211:02:05,  5.18s/it]
 35%|███▍      | 78438/225000 [152:01:36<206:54:14,  5.08s/it]
 35%|███▍      | 78439/225000 [152:01:38<172:59:05,  4.25s/it]
 35%|███▍      | 78440/225000 [152:01:42<168:07:10,  4.13s/it]
 35%|███▍      | 78441/225000 [152:01:45<153:46:45,  3.78s/it]
 35%|███▍      | 78442/225000 [152:01:48<143:50:48,  3.53s/it]
 35%|███▍      | 78443/225000 [152:01:51<137:29:54,  3.38s/it]
 35%|███▍      | 78444/225000 [152:01:55<140:46:48,  3.46s/it]
 35%|███▍      | 78445/225000 [152:01:58<140:49:58,  3.46s/it]
 35%|███▍      | 78446/225000 [152:02:01<134:25:32,  3.30s/it]
 35%|███▍      | 78447/225000 [152:02:05<139:29:17,  3.43s/it]
 35%|███▍      | 78448/225000 [152:02:13<192:44:04,  4.73s/it]
 35%|███▍      | 78449/225000 [152:02:20<224:35:37,  5.52s/it]
 35%|███▍      | 78450/225000 [152:02:22<183:59:07,  4.52s/it]
                                                              
{'loss': '0.8333', 'grad_norm': '4.194', 'learning_rate': '3.852e-05', 'epoch': '0.3487', 'num_input_tokens_seen': 250459600, 'train_runtime': '5.473e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78450/225000 [152:02:22<183:59:07,  4.52s/it]
 35%|███▍      | 78451/225000 [152:02:30<221:22:08,  5.44s/it]
 35%|███▍      | 78452/225000 [152:02:33<187:24:09,  4.60s/it]
 35%|███▍      | 78453/225000 [152:02:37<180:00:18,  4.42s/it]
 35%|███▍      | 78454/225000 [152:02:43<201:27:27,  4.95s/it]
 35%|███▍      | 78455/225000 [152:02:45<169:18:04,  4.16s/it]
 35%|███▍      | 78456/225000 [152:02:48<152:15:24,  3.74s/it]
 35%|███▍      | 78457/225000 [152:02:51<140:38:25,  3.45s/it]
 35%|███▍      | 78458/225000 [152:02:54<144:46:27,  3.56s/it]
 35%|███▍      | 78459/225000 [152:03:00<170:56:10,  4.20s/it]
 35%|███▍      | 78460/225000 [152:03:03<152:24:49,  3.74s/it]
 35%|███▍      | 78461/225000 [152:03:06<143:03:32,  3.51s/it]
 35%|███▍      | 78462/225000 [152:03:08<132:53:44,  3.26s/it]
 35%|███▍      | 78463/225000 [152:03:16<186:43:04,  4.59s/it]
 35%|███▍      | 78464/225000 [152:03:22<200:16:26,  4.92s/it]
 35%|███▍      | 78465/225000 [152:03:24<168:56:41,  4.15s/it]
 35%|███▍      | 78466/225000 [152:03:27<153:52:55,  3.78s/it]
 35%|███▍      | 78467/225000 [152:03:30<147:26:00,  3.62s/it]
 35%|███▍      | 78468/225000 [152:03:34<146:01:14,  3.59s/it]
 35%|███▍      | 78469/225000 [152:03:37<141:54:56,  3.49s/it]
 35%|███▍      | 78470/225000 [152:03:40<137:55:10,  3.39s/it]
 35%|███▍      | 78471/225000 [152:03:44<144:55:08,  3.56s/it]
 35%|███▍      | 78472/225000 [152:03:53<207:49:42,  5.11s/it]
 35%|███▍      | 78473/225000 [152:03:59<221:17:36,  5.44s/it]
 35%|███▍      | 78474/225000 [152:04:02<191:26:27,  4.70s/it]
 35%|███▍      | 78475/225000 [152:04:05<165:32:57,  4.07s/it]
 35%|███▍      | 78476/225000 [152:04:08<155:55:53,  3.83s/it]
 35%|███▍      | 78477/225000 [152:04:11<145:37:50,  3.58s/it]
 35%|███▍      | 78478/225000 [152:04:14<140:55:06,  3.46s/it]
 35%|███▍      | 78479/225000 [152:04:18<140:21:19,  3.45s/it]
 35%|███▍      | 78480/225000 [152:04:27<218:25:09,  5.37s/it]
 35%|███▍      | 78481/225000 [152:04:32<208:43:58,  5.13s/it]
 35%|███▍      | 78482/225000 [152:04:38<219:22:40,  5.39s/it]
 35%|███▍      | 78483/225000 [152:04:41<186:35:25,  4.58s/it]
 35%|███▍      | 78484/225000 [152:04:44<165:40:26,  4.07s/it]
 35%|███▍      | 78485/225000 [152:04:49<182:40:17,  4.49s/it]
 35%|███▍      | 78486/225000 [152:04:53<175:09:25,  4.30s/it]
 35%|███▍      | 78487/225000 [152:04:55<152:53:24,  3.76s/it]
 35%|███▍      | 78488/225000 [152:04:58<141:18:49,  3.47s/it]
 35%|███▍      | 78489/225000 [152:05:06<199:09:56,  4.89s/it]
 35%|███▍      | 78490/225000 [152:05:11<198:06:29,  4.87s/it]
 35%|███▍      | 78491/225000 [152:05:14<173:48:08,  4.27s/it]
 35%|███▍      | 78492/225000 [152:05:17<154:55:46,  3.81s/it]
 35%|███▍      | 78493/225000 [152:05:19<140:07:56,  3.44s/it]
 35%|███▍      | 78494/225000 [152:05:22<130:04:12,  3.20s/it]
 35%|███▍      | 78495/225000 [152:05:25<127:41:13,  3.14s/it]
 35%|███▍      | 78496/225000 [152:05:32<168:01:53,  4.13s/it]
 35%|███▍      | 78497/225000 [152:05:38<196:18:57,  4.82s/it]
 35%|███▍      | 78498/225000 [152:05:40<164:55:32,  4.05s/it]
 35%|███▍      | 78499/225000 [152:05:46<180:07:27,  4.43s/it]
 35%|███▍      | 78500/225000 [152:05:48<160:17:54,  3.94s/it]
                                                              
{'loss': '0.7977', 'grad_norm': '3.457', 'learning_rate': '3.85e-05', 'epoch': '0.3489', 'num_input_tokens_seen': 250544232, 'train_runtime': '5.475e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78500/225000 [152:05:48<160:17:54,  3.94s/it]
 35%|███▍      | 78501/225000 [152:05:51<143:07:51,  3.52s/it]
 35%|███▍      | 78502/225000 [152:05:54<141:40:56,  3.48s/it]
 35%|███▍      | 78503/225000 [152:05:57<134:15:09,  3.30s/it]
 35%|███▍      | 78504/225000 [152:06:03<167:55:43,  4.13s/it]
 35%|███▍      | 78505/225000 [152:06:12<220:18:12,  5.41s/it]
 35%|███▍      | 78506/225000 [152:06:14<185:35:29,  4.56s/it]
 35%|███▍      | 78507/225000 [152:06:17<160:32:18,  3.95s/it]
 35%|███▍      | 78508/225000 [152:06:19<143:01:21,  3.51s/it]
 35%|███▍      | 78509/225000 [152:06:23<143:58:15,  3.54s/it]
 35%|███▍      | 78510/225000 [152:06:26<142:33:54,  3.50s/it]
 35%|███▍      | 78511/225000 [152:06:30<146:07:45,  3.59s/it]
 35%|███▍      | 78512/225000 [152:06:34<145:50:37,  3.58s/it]
 35%|███▍      | 78513/225000 [152:06:40<179:29:43,  4.41s/it]
 35%|███▍      | 78514/225000 [152:06:47<209:06:13,  5.14s/it]
 35%|███▍      | 78515/225000 [152:06:49<175:01:49,  4.30s/it]
 35%|███▍      | 78516/225000 [152:06:52<160:30:13,  3.94s/it]
 35%|███▍      | 78517/225000 [152:06:58<185:34:42,  4.56s/it]
 35%|███▍      | 78518/225000 [152:07:03<194:28:04,  4.78s/it]
 35%|███▍      | 78519/225000 [152:07:06<171:02:32,  4.20s/it]
 35%|███▍      | 78520/225000 [152:07:09<157:37:01,  3.87s/it]
 35%|███▍      | 78521/225000 [152:07:17<204:09:56,  5.02s/it]
 35%|███▍      | 78522/225000 [152:07:20<179:40:21,  4.42s/it]
 35%|███▍      | 78523/225000 [152:07:23<162:48:21,  4.00s/it]
 35%|███▍      | 78524/225000 [152:07:26<145:42:48,  3.58s/it]
 35%|███▍      | 78525/225000 [152:07:29<135:17:53,  3.33s/it]
 35%|███▍      | 78526/225000 [152:07:31<128:28:25,  3.16s/it]
 35%|███▍      | 78527/225000 [152:07:35<129:27:52,  3.18s/it]
 35%|███▍      | 78528/225000 [152:07:37<124:56:59,  3.07s/it]
 35%|███▍      | 78529/225000 [152:07:40<117:57:18,  2.90s/it]
 35%|███▍      | 78530/225000 [152:07:43<121:04:33,  2.98s/it]
 35%|███▍      | 78531/225000 [152:07:51<180:57:02,  4.45s/it]
 35%|███▍      | 78532/225000 [152:07:58<210:35:21,  5.18s/it]
 35%|███▍      | 78533/225000 [152:08:00<177:52:47,  4.37s/it]
 35%|███▍      | 78534/225000 [152:08:03<160:28:50,  3.94s/it]
 35%|███▍      | 78535/225000 [152:08:09<187:00:58,  4.60s/it]
 35%|███▍      | 78536/225000 [152:08:15<197:44:19,  4.86s/it]
 35%|███▍      | 78537/225000 [152:08:18<172:14:37,  4.23s/it]
 35%|███▍      | 78538/225000 [152:08:20<151:26:22,  3.72s/it]
 35%|███▍      | 78539/225000 [152:08:23<147:25:52,  3.62s/it]
 35%|███▍      | 78540/225000 [152:08:26<137:34:33,  3.38s/it]
 35%|███▍      | 78541/225000 [152:08:30<140:38:36,  3.46s/it]
 35%|███▍      | 78542/225000 [152:08:39<206:44:30,  5.08s/it]
 35%|███▍      | 78543/225000 [152:08:41<173:53:51,  4.27s/it]
 35%|███▍      | 78544/225000 [152:08:43<149:38:44,  3.68s/it]
 35%|███▍      | 78545/225000 [152:08:46<138:19:34,  3.40s/it]
 35%|███▍      | 78546/225000 [152:08:50<137:18:44,  3.38s/it]
 35%|███▍      | 78547/225000 [152:08:52<130:56:00,  3.22s/it]
 35%|███▍      | 78548/225000 [152:08:56<132:02:02,  3.25s/it]
 35%|███▍      | 78549/225000 [152:09:05<201:53:16,  4.96s/it]
 35%|███▍      | 78550/225000 [152:09:07<174:37:50,  4.29s/it]
                                                              
{'loss': '0.8594', 'grad_norm': '4.358', 'learning_rate': '3.848e-05', 'epoch': '0.3491', 'num_input_tokens_seen': 250627384, 'train_runtime': '5.477e+05', 'train_tokens_per_second': '457.6'}

 35%|███▍      | 78550/225000 [152:09:07<174:37:50,  4.29s/it]
 35%|███▍      | 78551/225000 [152:09:11<160:50:02,  3.95s/it]
 35%|███▍      | 78552/225000 [152:09:13<147:20:55,  3.62s/it]
 35%|███▍      | 78553/225000 [152:09:17<142:36:21,  3.51s/it]
 35%|███▍      | 78554/225000 [152:09:23<178:35:48,  4.39s/it]
 35%|███▍      | 78555/225000 [152:09:28<189:08:31,  4.65s/it]
 35%|███▍      | 78556/225000 [152:09:31<163:34:03,  4.02s/it]
 35%|███▍      | 78557/225000 [152:09:34<150:09:20,  3.69s/it]
 35%|███▍      | 78558/225000 [152:09:41<198:12:32,  4.87s/it]
 35%|███▍      | 78559/225000 [152:09:44<173:09:10,  4.26s/it]
 35%|███▍      | 78560/225000 [152:09:47<156:15:03,  3.84s/it]
 35%|███▍      | 78561/225000 [152:09:53<181:44:30,  4.47s/it]
 35%|███▍      | 78562/225000 [152:09:58<188:07:45,  4.62s/it]
 35%|███▍      | 78563/225000 [152:10:01<163:09:37,  4.01s/it]
 35%|███▍      | 78564/225000 [152:10:03<144:55:35,  3.56s/it]
 35%|███▍      | 78565/225000 [152:10:06<134:15:35,  3.30s/it]
 35%|███▍      | 78566/225000 [152:10:14<195:12:49,  4.80s/it]
 35%|███▍      | 78567/225000 [152:10:17<173:29:38,  4.27s/it]
 35%|███▍      | 78568/225000 [152:10:21<162:05:43,  3.99s/it]
 35%|███▍      | 78569/225000 [152:10:24<150:26:05,  3.70s/it]
 35%|███▍      | 78570/225000 [152:10:26<140:41:32,  3.46s/it]
 35%|███▍      | 78571/225000 [152:10:29<133:06:09,  3.27s/it]
 35%|███▍      | 78572/225000 [152:10:32<130:08:51,  3.20s/it]
 35%|███▍      | 78573/225000 [152:10:35<126:25:11,  3.11s/it]
 35%|███▍      | 78574/225000 [152:10:38<123:45:13,  3.04s/it]
 35%|███▍      | 78575/225000 [152:10:45<175:02:19,  4.30s/it]
 35%|███▍      | 78576/225000 [152:10:54<230:06:58,  5.66s/it]
 35%|███▍      | 78577/225000 [152:10:57<190:46:52,  4.69s/it]
 35%|███▍      | 78578/225000 [152:11:00<170:52:13,  4.20s/it]
 35%|███▍      | 78579/225000 [152:11:03<162:39:37,  4.00s/it]
 35%|███▍      | 78580/225000 [152:11:06<148:53:00,  3.66s/it]
 35%|███▍      | 78581/225000 [152:11:09<140:49:47,  3.46s/it]
 35%|███▍      | 78582/225000 [152:11:12<132:41:42,  3.26s/it]
 35%|███▍      | 78583/225000 [152:11:15<133:26:09,  3.28s/it]
 35%|███▍      | 78584/225000 [152:11:24<198:43:32,  4.89s/it]
 35%|███▍      | 78585/225000 [152:11:30<211:58:23,  5.21s/it]
 35%|███▍      | 78586/225000 [152:11:34<201:05:51,  4.94s/it]
 35%|███▍      | 78587/225000 [152:11:37<172:12:03,  4.23s/it]
 35%|███▍      | 78588/225000 [152:11:40<164:21:40,  4.04s/it]
 35%|███▍      | 78589/225000 [152:11:47<197:12:36,  4.85s/it]
 35%|███▍      | 78590/225000 [152:11:50<175:33:05,  4.32s/it]
 35%|███▍      | 78591/225000 [152:11:53<155:41:35,  3.83s/it]
 35%|███▍      | 78592/225000 [152:11:56<149:29:01,  3.68s/it]
 35%|███▍      | 78593/225000 [152:11:59<137:56:01,  3.39s/it]
 35%|███▍      | 78594/225000 [152:12:02<140:41:03,  3.46s/it]
 35%|███▍      | 78595/225000 [152:12:06<141:44:02,  3.49s/it]
 35%|███▍      | 78596/225000 [152:12:09<131:55:15,  3.24s/it]
 35%|███▍      | 78597/225000 [152:12:12<127:44:45,  3.14s/it]
 35%|███▍      | 78598/225000 [152:12:20<194:51:25,  4.79s/it]
 35%|███▍      | 78599/225000 [152:12:27<219:00:33,  5.39s/it]
 35%|███▍      | 78600/225000 [152:12:29<182:43:19,  4.49s/it]
                                                              
{'loss': '0.8167', 'grad_norm': '3.72', 'learning_rate': '3.847e-05', 'epoch': '0.3493', 'num_input_tokens_seen': 250711104, 'train_runtime': '5.479e+05', 'train_tokens_per_second': '457.5'}

 35%|███▍      | 78600/225000 [152:12:29<182:43:19,  4.49s/it]
 35%|███▍      | 78601/225000 [152:12:35<194:36:57,  4.79s/it]
 35%|███▍      | 78602/225000 [152:12:39<186:34:07,  4.59s/it]
 35%|███▍      | 78603/225000 [152:12:45<205:27:52,  5.05s/it]
 35%|███▍      | 78604/225000 [152:12:48<179:51:39,  4.42s/it]
 35%|███▍      | 78605/225000 [152:12:51<167:11:55,  4.11s/it]
 35%|███▍      | 78606/225000 [152:12:55<157:56:34,  3.88s/it]
 35%|███▍      | 78607/225000 [152:12:58<144:04:49,  3.54s/it]
 35%|███▍      | 78608/225000 [152:13:00<135:42:45,  3.34s/it]
 35%|███▍      | 78609/225000 [152:13:04<135:01:22,  3.32s/it]
 35%|███▍      | 78610/225000 [152:13:06<128:36:35,  3.16s/it]
 35%|███▍      | 78611/225000 [152:13:10<134:14:13,  3.30s/it]
 35%|███▍      | 78612/225000 [152:13:19<200:14:39,  4.92s/it]
 35%|███▍      | 78613/225000 [152:13:23<190:27:53,  4.68s/it]
 35%|███▍      | 78614/225000 [152:13:26<169:15:32,  4.16s/it]
 35%|███▍      | 78615/225000 [152:13:29<160:40:43,  3.95s/it]
 35%|███▍      | 78616/225000 [152:13:32<150:08:27,  3.69s/it]
 35%|███▍      | 78617/225000 [152:13:36<143:03:00,  3.52s/it]
 35%|███▍      | 78618/225000 [152:13:41<168:06:46,  4.13s/it]
 35%|███▍      | 78619/225000 [152:13:49<217:37:05,  5.35s/it]
 35%|███▍      | 78620/225000 [152:13:52<184:01:23,  4.53s/it]
 35%|███▍      | 78621/225000 [152:13:55<160:36:51,  3.95s/it]
 35%|███▍      | 78622/225000 [152:13:57<147:13:58,  3.62s/it]
 35%|███▍      | 78623/225000 [152:14:05<199:49:34,  4.91s/it]
 35%|███▍      | 78624/225000 [152:14:09<188:19:06,  4.63s/it]
 35%|███▍      | 78625/225000 [152:14:12<170:45:44,  4.20s/it]
 35%|███▍      | 78626/225000 [152:14:15<151:43:48,  3.73s/it]
 35%|███▍      | 78627/225000 [152:14:19<153:39:54,  3.78s/it]
 35%|███▍      | 78628/225000 [152:14:27<199:39:58,  4.91s/it]
 35%|███▍      | 78629/225000 [152:14:29<174:11:13,  4.28s/it]
 35%|███▍      | 78630/225000 [152:14:32<156:14:08,  3.84s/it]
 35%|███▍      | 78631/225000 [152:14:36<155:00:46,  3.81s/it]
 35%|███▍      | 78632/225000 [152:14:39<151:20:34,  3.72s/it]
 35%|███▍      | 78633/225000 [152:14:43<145:36:43,  3.58s/it]
 35%|███▍      | 78634/225000 [152:14:52<211:18:49,  5.20s/it]
 35%|███▍      | 78635/225000 [152:14:55<193:40:52,  4.76s/it]
 35%|███▍      | 78636/225000 [152:14:58<169:13:20,  4.16s/it]
 35%|███▍      | 78637/225000 [152:15:01<152:21:30,  3.75s/it]
 35%|███▍      | 78638/225000 [152:15:04<140:07:14,  3.45s/it]
 35%|███▍      | 78639/225000 [152:15:07<136:39:26,  3.36s/it]
 35%|███▍      | 78640/225000 [152:15:11<151:36:38,  3.73s/it]
 35%|███▍      | 78641/225000 [152:15:19<195:47:51,  4.82s/it]
 35%|███▍      | 78642/225000 [152:15:22<172:04:32,  4.23s/it]
 35%|███▍      | 78643/225000 [152:15:24<151:25:38,  3.72s/it]
 35%|███▍      | 78644/225000 [152:15:27<141:56:55,  3.49s/it]
 35%|███▍      | 78645/225000 [152:15:30<134:13:26,  3.30s/it]
 35%|███▍      | 78646/225000 [152:15:38<193:51:33,  4.77s/it]
 35%|███▍      | 78647/225000 [152:15:42<187:57:11,  4.62s/it]
 35%|███▍      | 78648/225000 [152:15:45<163:34:44,  4.02s/it]
 35%|███▍      | 78649/225000 [152:15:48<148:50:06,  3.66s/it]
 35%|███▍      | 78650/225000 [152:15:50<134:51:26,  3.32s/it]
                                                              
{'loss': '0.7944', 'grad_norm': '8.358', 'learning_rate': '3.845e-05', 'epoch': '0.3496', 'num_input_tokens_seen': 250796128, 'train_runtime': '5.482e+05', 'train_tokens_per_second': '457.5'}

 35%|███▍      | 78650/225000 [152:15:50<134:51:26,  3.32s/it]
 35%|███▍      | 78651/225000 [152:15:53<130:49:56,  3.22s/it]
 35%|███▍      | 78652/225000 [152:15:56<129:07:07,  3.18s/it]
 35%|███▍      | 78653/225000 [152:16:00<134:26:08,  3.31s/it]
 35%|███▍      | 78654/225000 [152:16:05<153:32:16,  3.78s/it]
 35%|███▍      | 78655/225000 [152:16:08<148:36:41,  3.66s/it]
 35%|███▍      | 78656/225000 [152:16:12<145:40:32,  3.58s/it]
 35%|███▍      | 78657/225000 [152:16:15<140:47:08,  3.46s/it]
 35%|███▍      | 78658/225000 [152:16:19<145:29:07,  3.58s/it]
 35%|███▍      | 78659/225000 [152:16:23<155:35:31,  3.83s/it]
 35%|███▍      | 78660/225000 [152:16:30<194:21:15,  4.78s/it]
 35%|███▍      | 78661/225000 [152:16:33<168:16:24,  4.14s/it]
 35%|███▍      | 78662/225000 [152:16:36<152:14:00,  3.75s/it]
 35%|███▍      | 78663/225000 [152:16:39<152:07:50,  3.74s/it]
 35%|███▍      | 78664/225000 [152:16:42<140:21:01,  3.45s/it]
 35%|███▍      | 78665/225000 [152:16:49<183:37:07,  4.52s/it]
 35%|███▍      | 78666/225000 [152:16:56<209:45:55,  5.16s/it]
 35%|███▍      | 78667/225000 [152:16:58<176:23:28,  4.34s/it]
 35%|███▍      | 78668/225000 [152:17:01<157:12:42,  3.87s/it]
 35%|███▍      | 78669/225000 [152:17:04<146:20:44,  3.60s/it]
 35%|███▍      | 78670/225000 [152:17:07<136:28:25,  3.36s/it]
 35%|███▍      | 78671/225000 [152:17:12<153:50:30,  3.78s/it]
 35%|███▍      | 78672/225000 [152:17:18<185:01:04,  4.55s/it]
 35%|███▍      | 78673/225000 [152:17:21<169:11:02,  4.16s/it]
 35%|███▍      | 78674/225000 [152:17:24<157:16:24,  3.87s/it]
 35%|███▍      | 78675/225000 [152:17:27<144:28:05,  3.55s/it]
 35%|███▍      | 78676/225000 [152:17:30<135:07:37,  3.32s/it]
 35%|███▍      | 78677/225000 [152:17:33<136:49:24,  3.37s/it]
 35%|███▍      | 78678/225000 [152:17:36<132:56:25,  3.27s/it]
 35%|███▍      | 78679/225000 [152:17:40<130:57:48,  3.22s/it]
 35%|███▍      | 78680/225000 [152:17:48<193:04:42,  4.75s/it]
 35%|███▍      | 78681/225000 [152:17:54<210:58:10,  5.19s/it]
 35%|███▍      | 78682/225000 [152:17:57<178:42:48,  4.40s/it]
 35%|███▍      | 78683/225000 [152:17:59<157:47:37,  3.88s/it]
 35%|███▍      | 78684/225000 [152:18:02<140:28:01,  3.46s/it]
 35%|███▍      | 78685/225000 [152:18:07<157:09:01,  3.87s/it]
 35%|███▍      | 78686/225000 [152:18:14<198:27:57,  4.88s/it]
 35%|███▍      | 78687/225000 [152:18:16<168:28:35,  4.15s/it]
 35%|███▍      | 78688/225000 [152:18:19<149:50:18,  3.69s/it]
 35%|███▍      | 78689/225000 [152:18:22<146:35:30,  3.61s/it]
 35%|███▍      | 78690/225000 [152:18:29<180:41:00,  4.45s/it]
 35%|███▍      | 78691/225000 [152:18:33<179:15:18,  4.41s/it]
 35%|███▍      | 78692/225000 [152:18:36<156:36:12,  3.85s/it]
 35%|███▍      | 78693/225000 [152:18:38<138:08:08,  3.40s/it]
 35%|███▍      | 78694/225000 [152:18:45<180:30:39,  4.44s/it]
 35%|███▍      | 78695/225000 [152:18:50<193:40:53,  4.77s/it]
 35%|███▍      | 78696/225000 [152:18:53<162:08:35,  3.99s/it]
 35%|███▍      | 78697/225000 [152:18:55<141:19:03,  3.48s/it]
 35%|███▍      | 78698/225000 [152:18:58<132:11:20,  3.25s/it]
 35%|███▍      | 78699/225000 [152:19:06<193:02:54,  4.75s/it]
 35%|███▍      | 78700/225000 [152:19:09<176:45:46,  4.35s/it]
                                                              
{'loss': '0.8494', 'grad_norm': '2.964', 'learning_rate': '3.844e-05', 'epoch': '0.3498', 'num_input_tokens_seen': 250879048, 'train_runtime': '5.483e+05', 'train_tokens_per_second': '457.5'}

 35%|███▍      | 78700/225000 [152:19:09<176:45:46,  4.35s/it]
 35%|███▍      | 78701/225000 [152:19:12<151:58:31,  3.74s/it]
 35%|███▍      | 78702/225000 [152:19:14<138:40:46,  3.41s/it]
 35%|███▍      | 78703/225000 [152:19:18<139:19:57,  3.43s/it]
 35%|███▍      | 78704/225000 [152:19:21<136:04:35,  3.35s/it]
 35%|███▍      | 78705/225000 [152:19:24<136:52:35,  3.37s/it]
 35%|███▍      | 78706/225000 [152:19:27<132:50:56,  3.27s/it]
 35%|███▍      | 78707/225000 [152:19:31<132:14:09,  3.25s/it]
 35%|███▍      | 78708/225000 [152:19:34<134:01:25,  3.30s/it]
 35%|███▍      | 78709/225000 [152:19:37<133:00:26,  3.27s/it]
 35%|███▍      | 78710/225000 [152:19:40<130:48:53,  3.22s/it]
 35%|███▍      | 78711/225000 [152:19:45<147:12:42,  3.62s/it]
 35%|███▍      | 78712/225000 [152:19:49<149:05:01,  3.67s/it]
 35%|███▍      | 78713/225000 [152:19:53<159:16:00,  3.92s/it]
 35%|███▍      | 78714/225000 [152:19:57<153:37:31,  3.78s/it]
 35%|███▍      | 78715/225000 [152:20:03<181:13:08,  4.46s/it]
 35%|███▍      | 78716/225000 [152:20:11<230:19:45,  5.67s/it]
 35%|███▍      | 78717/225000 [152:20:16<217:47:13,  5.36s/it]
 35%|███▍      | 78718/225000 [152:20:18<184:26:59,  4.54s/it]
 35%|███▍      | 78719/225000 [152:20:21<161:28:15,  3.97s/it]
 35%|███▍      | 78720/225000 [152:20:24<147:54:44,  3.64s/it]
 35%|███▍      | 78721/225000 [152:20:32<198:48:57,  4.89s/it]
 35%|███▍      | 78722/225000 [152:20:34<170:38:29,  4.20s/it]
 35%|███▍      | 78723/225000 [152:20:37<151:44:24,  3.73s/it]
 35%|███▍      | 78724/225000 [152:20:44<196:00:12,  4.82s/it]
 35%|███▍      | 78725/225000 [152:20:48<182:56:14,  4.50s/it]
 35%|███▍      | 78726/225000 [152:20:52<174:41:56,  4.30s/it]
 35%|███▍      | 78727/225000 [152:20:55<158:38:12,  3.90s/it]
 35%|███▍      | 78728/225000 [152:20:58<150:58:22,  3.72s/it]
 35%|███▍      | 78729/225000 [152:21:01<141:45:57,  3.49s/it]
 35%|███▍      | 78730/225000 [152:21:04<136:16:26,  3.35s/it]
 35%|███▍      | 78731/225000 [152:21:07<135:51:01,  3.34s/it]
 35%|███▍      | 78732/225000 [152:21:11<134:46:32,  3.32s/it]
 35%|███▍      | 78733/225000 [152:21:15<152:44:49,  3.76s/it]
 35%|███▍      | 78734/225000 [152:21:20<168:32:33,  4.15s/it]
 35%|███▍      | 78735/225000 [152:21:24<158:00:44,  3.89s/it]
 35%|███▍      | 78736/225000 [152:21:27<153:09:25,  3.77s/it]
 35%|███▍      | 78737/225000 [152:21:31<146:37:48,  3.61s/it]
 35%|███▍      | 78738/225000 [152:21:34<144:13:00,  3.55s/it]
 35%|███▍      | 78739/225000 [152:21:40<174:51:10,  4.30s/it]
 35%|███▍      | 78740/225000 [152:21:49<230:26:56,  5.67s/it]
 35%|███▍      | 78741/225000 [152:21:52<194:16:53,  4.78s/it]
 35%|███▍      | 78742/225000 [152:21:54<169:37:20,  4.18s/it]
 35%|███▍      | 78743/225000 [152:21:57<151:18:30,  3.72s/it]
 35%|███▍      | 78744/225000 [152:21:59<135:11:31,  3.33s/it]
 35%|███▍      | 78745/225000 [152:22:02<125:48:24,  3.10s/it]
 35%|███▍      | 78746/225000 [152:22:06<137:29:52,  3.38s/it]
 35%|███▍      | 78747/225000 [152:22:15<206:10:39,  5.08s/it]
 35%|███▍      | 78748/225000 [152:22:18<182:17:45,  4.49s/it]
 35%|███▍      | 78749/225000 [152:22:21<157:54:24,  3.89s/it]
 35%|███▌      | 78750/225000 [152:22:24<153:05:27,  3.77s/it]
                                                              
{'loss': '0.8502', 'grad_norm': '3.673', 'learning_rate': '3.842e-05', 'epoch': '0.35', 'num_input_tokens_seen': 250962664, 'train_runtime': '5.485e+05', 'train_tokens_per_second': '457.5'}

 35%|███▌      | 78750/225000 [152:22:24<153:05:27,  3.77s/it]
 35%|███▌      | 78751/225000 [152:22:28<148:38:20,  3.66s/it]
 35%|███▌      | 78752/225000 [152:22:31<144:18:20,  3.55s/it]
 35%|███▌      | 78753/225000 [152:22:34<142:13:41,  3.50s/it]
 35%|███▌      | 78754/225000 [152:22:37<138:47:56,  3.42s/it]
 35%|███▌      | 78755/225000 [152:22:40<133:23:24,  3.28s/it]
 35%|███▌      | 78756/225000 [152:22:48<184:37:57,  4.54s/it]
 35%|███▌      | 78757/225000 [152:22:55<219:47:05,  5.41s/it]
 35%|███▌      | 78758/225000 [152:22:58<187:37:36,  4.62s/it]
 35%|███▌      | 78759/225000 [152:23:01<162:43:39,  4.01s/it]
 35%|███▌      | 78760/225000 [152:23:08<201:14:00,  4.95s/it]
 35%|███▌      | 78761/225000 [152:23:11<175:06:24,  4.31s/it]
 35%|███▌      | 78762/225000 [152:23:14<159:10:03,  3.92s/it]
 35%|███▌      | 78763/225000 [152:23:16<144:59:55,  3.57s/it]
 35%|███▌      | 78764/225000 [152:23:19<133:49:02,  3.29s/it]
 35%|███▌      | 78765/225000 [152:23:25<163:27:45,  4.02s/it]
 35%|███▌      | 78766/225000 [152:23:34<230:52:25,  5.68s/it]
 35%|███▌      | 78767/225000 [152:23:37<189:15:41,  4.66s/it]
 35%|███▌      | 78768/225000 [152:23:39<164:15:34,  4.04s/it]
 35%|███▌      | 78769/225000 [152:23:42<146:44:58,  3.61s/it]
 35%|███▌      | 78770/225000 [152:23:44<134:13:18,  3.30s/it]
 35%|███▌      | 78771/225000 [152:23:51<169:34:46,  4.17s/it]
 35%|███▌      | 78772/225000 [152:23:56<183:10:38,  4.51s/it]
 35%|███▌      | 78773/225000 [152:23:59<163:07:50,  4.02s/it]
 35%|███▌      | 78774/225000 [152:24:02<149:27:55,  3.68s/it]
 35%|███▌      | 78775/225000 [152:24:04<134:29:02,  3.31s/it]
 35%|███▌      | 78776/225000 [152:24:10<168:05:56,  4.14s/it]
 35%|███▌      | 78777/225000 [152:24:15<176:53:29,  4.36s/it]
 35%|███▌      | 78778/225000 [152:24:18<159:00:23,  3.91s/it]
 35%|███▌      | 78779/225000 [152:24:21<149:38:09,  3.68s/it]
 35%|███▌      | 78780/225000 [152:24:24<142:49:28,  3.52s/it]
 35%|███▌      | 78781/225000 [152:24:27<135:44:51,  3.34s/it]
 35%|███▌      | 78782/225000 [152:24:30<133:20:28,  3.28s/it]
 35%|███▌      | 78783/225000 [152:24:34<134:24:44,  3.31s/it]
 35%|███▌      | 78784/225000 [152:24:37<134:10:29,  3.30s/it]
 35%|███▌      | 78785/225000 [152:24:40<127:01:05,  3.13s/it]
 35%|███▌      | 78786/225000 [152:24:44<141:20:29,  3.48s/it]
 35%|███▌      | 78787/225000 [152:24:53<208:00:39,  5.12s/it]
 35%|███▌      | 78788/225000 [152:24:58<205:08:49,  5.05s/it]
 35%|███▌      | 78789/225000 [152:25:01<179:49:41,  4.43s/it]
 35%|███▌      | 78790/225000 [152:25:04<161:49:06,  3.98s/it]
 35%|███▌      | 78791/225000 [152:25:07<149:02:07,  3.67s/it]
 35%|███▌      | 78792/225000 [152:25:09<137:56:41,  3.40s/it]
 35%|███▌      | 78793/225000 [152:25:19<209:35:59,  5.16s/it]
 35%|███▌      | 78794/225000 [152:25:24<207:31:17,  5.11s/it]
 35%|███▌      | 78795/225000 [152:25:30<217:08:52,  5.35s/it]
 35%|███▌      | 78796/225000 [152:25:33<191:42:51,  4.72s/it]
 35%|███▌      | 78797/225000 [152:25:35<162:02:50,  3.99s/it]
 35%|███▌      | 78798/225000 [152:25:38<144:46:32,  3.56s/it]
 35%|███▌      | 78799/225000 [152:25:41<136:37:13,  3.36s/it]
 35%|███▌      | 78800/225000 [152:25:46<162:49:11,  4.01s/it]
                                                              
{'loss': '0.8427', 'grad_norm': '4.6', 'learning_rate': '3.841e-05', 'epoch': '0.3502', 'num_input_tokens_seen': 251046864, 'train_runtime': '5.487e+05', 'train_tokens_per_second': '457.5'}

 35%|███▌      | 78800/225000 [152:25:46<162:49:11,  4.01s/it]
 35%|███▌      | 78801/225000 [152:25:50<161:14:41,  3.97s/it]
 35%|███▌      | 78802/225000 [152:25:55<179:58:08,  4.43s/it]
 35%|███▌      | 78803/225000 [152:25:59<168:49:34,  4.16s/it]
 35%|███▌      | 78804/225000 [152:26:02<155:04:49,  3.82s/it]
 35%|███▌      | 78805/225000 [152:26:05<139:19:39,  3.43s/it]
 35%|███▌      | 78806/225000 [152:26:07<131:51:21,  3.25s/it]
 35%|███▌      | 78807/225000 [152:26:10<128:59:38,  3.18s/it]
 35%|███▌      | 78808/225000 [152:26:13<124:03:48,  3.06s/it]
 35%|███▌      | 78809/225000 [152:26:16<124:37:23,  3.07s/it]
 35%|███▌      | 78810/225000 [152:26:19<123:58:20,  3.05s/it]
 35%|███▌      | 78811/225000 [152:26:27<175:44:00,  4.33s/it]
 35%|███▌      | 78812/225000 [152:26:34<214:29:05,  5.28s/it]
 35%|███▌      | 78813/225000 [152:26:37<183:52:33,  4.53s/it]
 35%|███▌      | 78814/225000 [152:26:39<160:48:24,  3.96s/it]
 35%|███▌      | 78815/225000 [152:26:42<147:15:34,  3.63s/it]
 35%|███▌      | 78816/225000 [152:26:45<132:20:41,  3.26s/it]
 35%|███▌      | 78817/225000 [152:26:50<156:12:09,  3.85s/it]
 35%|███▌      | 78818/225000 [152:26:58<204:23:35,  5.03s/it]
 35%|███▌      | 78819/225000 [152:27:00<171:14:28,  4.22s/it]
 35%|███▌      | 78820/225000 [152:27:04<162:46:16,  4.01s/it]
 35%|███▌      | 78821/225000 [152:27:11<200:11:33,  4.93s/it]
 35%|███▌      | 78822/225000 [152:27:14<179:28:40,  4.42s/it]
 35%|███▌      | 78823/225000 [152:27:17<161:47:22,  3.98s/it]
 35%|███▌      | 78824/225000 [152:27:20<154:23:28,  3.80s/it]
 35%|███▌      | 78825/225000 [152:27:23<141:55:28,  3.50s/it]
 35%|███▌      | 78826/225000 [152:27:26<132:59:24,  3.28s/it]
 35%|███▌      | 78827/225000 [152:27:29<127:00:42,  3.13s/it]
 35%|███▌      | 78828/225000 [152:27:32<126:20:04,  3.11s/it]
 35%|███▌      | 78829/225000 [152:27:38<163:08:49,  4.02s/it]
 35%|███▌      | 78830/225000 [152:27:46<212:38:29,  5.24s/it]
 35%|███▌      | 78831/225000 [152:27:48<176:27:13,  4.35s/it]
 35%|███▌      | 78832/225000 [152:27:51<153:07:09,  3.77s/it]
 35%|███▌      | 78833/225000 [152:27:58<203:20:05,  5.01s/it]
 35%|███▌      | 78834/225000 [152:28:02<182:44:21,  4.50s/it]
 35%|███▌      | 78835/225000 [152:28:04<160:14:34,  3.95s/it]
 35%|███▌      | 78836/225000 [152:28:10<174:32:53,  4.30s/it]
 35%|███▌      | 78837/225000 [152:28:15<188:48:45,  4.65s/it]
 35%|███▌      | 78838/225000 [152:28:17<160:59:50,  3.97s/it]
 35%|███▌      | 78839/225000 [152:28:22<169:45:34,  4.18s/it]
 35%|███▌      | 78840/225000 [152:28:27<180:36:54,  4.45s/it]
 35%|███▌      | 78841/225000 [152:28:29<153:53:41,  3.79s/it]
 35%|███▌      | 78842/225000 [152:28:37<195:55:20,  4.83s/it]
 35%|███▌      | 78843/225000 [152:28:39<169:58:31,  4.19s/it]
 35%|███▌      | 78844/225000 [152:28:42<149:33:01,  3.68s/it]
 35%|███▌      | 78845/225000 [152:28:44<135:03:56,  3.33s/it]
 35%|███▌      | 78846/225000 [152:28:48<138:51:16,  3.42s/it]
 35%|███▌      | 78847/225000 [152:28:51<132:27:48,  3.26s/it]
 35%|███▌      | 78848/225000 [152:28:54<131:28:15,  3.24s/it]
 35%|███▌      | 78849/225000 [152:28:58<137:28:06,  3.39s/it]
 35%|███▌      | 78850/225000 [152:29:01<136:00:58,  3.35s/it]
                                                              
{'loss': '0.8583', 'grad_norm': '2.886', 'learning_rate': '3.839e-05', 'epoch': '0.3504', 'num_input_tokens_seen': 251128080, 'train_runtime': '5.489e+05', 'train_tokens_per_second': '457.5'}

 35%|███▌      | 78850/225000 [152:29:01<136:00:58,  3.35s/it]
 35%|███▌      | 78851/225000 [152:29:04<135:16:09,  3.33s/it]
 35%|███▌      | 78852/225000 [152:29:13<200:44:59,  4.94s/it]
 35%|███▌      | 78853/225000 [152:29:18<204:46:42,  5.04s/it]
 35%|███▌      | 78854/225000 [152:29:21<172:51:22,  4.26s/it]
 35%|███▌      | 78855/225000 [152:29:24<161:48:09,  3.99s/it]
 35%|███▌      | 78856/225000 [152:29:30<182:56:29,  4.51s/it]
 35%|███▌      | 78857/225000 [152:29:36<198:55:58,  4.90s/it]
 35%|███▌      | 78858/225000 [152:29:38<168:22:06,  4.15s/it]
 35%|███▌      | 78859/225000 [152:29:40<146:40:34,  3.61s/it]
 35%|███▌      | 78860/225000 [152:29:43<136:05:04,  3.35s/it]
 35%|███▌      | 78861/225000 [152:29:46<125:17:34,  3.09s/it]
 35%|███▌      | 78862/225000 [152:29:53<180:28:39,  4.45s/it]
 35%|███▌      | 78863/225000 [152:29:58<188:06:46,  4.63s/it]
 35%|███▌      | 78864/225000 [152:30:01<169:33:16,  4.18s/it]
 35%|███▌      | 78865/225000 [152:30:05<160:13:14,  3.95s/it]
 35%|███▌      | 78866/225000 [152:30:08<151:30:57,  3.73s/it]
 35%|███▌      | 78867/225000 [152:30:15<184:34:22,  4.55s/it]
 35%|███▌      | 78868/225000 [152:30:17<165:20:41,  4.07s/it]
 35%|███▌      | 78869/225000 [152:30:20<144:32:48,  3.56s/it]
 35%|███▌      | 78870/225000 [152:30:25<169:30:40,  4.18s/it]
 35%|███▌      | 78871/225000 [152:30:29<160:55:58,  3.96s/it]
 35%|███▌      | 78872/225000 [152:30:34<172:51:43,  4.26s/it]
 35%|███▌      | 78873/225000 [152:30:37<161:47:13,  3.99s/it]
 35%|███▌      | 78874/225000 [152:30:43<184:34:00,  4.55s/it]
 35%|███▌      | 78875/225000 [152:30:46<168:30:21,  4.15s/it]
 35%|███▌      | 78876/225000 [152:30:49<149:34:10,  3.68s/it]
 35%|███▌      | 78877/225000 [152:30:52<140:27:25,  3.46s/it]
 35%|███▌      | 78878/225000 [152:30:55<136:09:25,  3.35s/it]
 35%|███▌      | 78879/225000 [152:30:58<127:34:54,  3.14s/it]
 35%|███▌      | 78880/225000 [152:31:00<124:10:24,  3.06s/it]
 35%|███▌      | 78881/225000 [152:31:08<183:38:08,  4.52s/it]
 35%|███▌      | 78882/225000 [152:31:12<175:11:06,  4.32s/it]
 35%|███▌      | 78883/225000 [152:31:15<160:25:45,  3.95s/it]
 35%|███▌      | 78884/225000 [152:31:18<146:27:45,  3.61s/it]
 35%|███▌      | 78885/225000 [152:31:21<143:25:23,  3.53s/it]
 35%|███▌      | 78886/225000 [152:31:27<166:26:02,  4.10s/it]
 35%|███▌      | 78887/225000 [152:31:32<176:57:46,  4.36s/it]
 35%|███▌      | 78888/225000 [152:31:37<186:32:56,  4.60s/it]
 35%|███▌      | 78889/225000 [152:31:40<168:15:16,  4.15s/it]
 35%|███▌      | 78890/225000 [152:31:44<162:25:53,  4.00s/it]
 35%|███▌      | 78891/225000 [152:31:47<149:48:43,  3.69s/it]
 35%|███▌      | 78892/225000 [152:31:50<143:54:23,  3.55s/it]
 35%|███▌      | 78893/225000 [152:31:53<141:16:26,  3.48s/it]
 35%|███▌      | 78894/225000 [152:31:56<137:56:15,  3.40s/it]
 35%|███▌      | 78895/225000 [152:32:00<137:26:25,  3.39s/it]
 35%|███▌      | 78896/225000 [152:32:03<133:32:20,  3.29s/it]
 35%|███▌      | 78897/225000 [152:32:06<131:53:15,  3.25s/it]
 35%|███▌      | 78898/225000 [152:32:09<133:26:10,  3.29s/it]
 35%|███▌      | 78899/225000 [152:32:13<134:58:07,  3.33s/it]
 35%|███▌      | 78900/225000 [152:32:22<201:37:27,  4.97s/it]
                                                              
{'loss': '0.8154', 'grad_norm': '3.811', 'learning_rate': '3.838e-05', 'epoch': '0.3507', 'num_input_tokens_seen': 251211064, 'train_runtime': '5.491e+05', 'train_tokens_per_second': '457.5'}

 35%|███▌      | 78900/225000 [152:32:22<201:37:27,  4.97s/it]
 35%|███▌      | 78901/225000 [152:32:28<216:33:45,  5.34s/it]
 35%|███▌      | 78902/225000 [152:32:30<179:53:05,  4.43s/it]
 35%|███▌      | 78903/225000 [152:32:33<159:47:06,  3.94s/it]
 35%|███▌      | 78904/225000 [152:32:40<199:46:27,  4.92s/it]
 35%|███▌      | 78905/225000 [152:32:43<171:37:07,  4.23s/it]
 35%|███▌      | 78906/225000 [152:32:46<164:13:24,  4.05s/it]
 35%|███▌      | 78907/225000 [152:32:50<153:14:45,  3.78s/it]
 35%|███▌      | 78908/225000 [152:32:53<147:54:40,  3.64s/it]
 35%|███▌      | 78909/225000 [152:32:56<142:02:09,  3.50s/it]
 35%|███▌      | 78910/225000 [152:32:59<141:05:35,  3.48s/it]
 35%|███▌      | 78911/225000 [152:33:06<174:49:05,  4.31s/it]
 35%|███▌      | 78912/225000 [152:33:12<196:52:22,  4.85s/it]
 35%|███▌      | 78913/225000 [152:33:15<176:08:14,  4.34s/it]
 35%|███▌      | 78914/225000 [152:33:18<163:51:50,  4.04s/it]
 35%|███▌      | 78915/225000 [152:33:22<156:07:10,  3.85s/it]
 35%|███▌      | 78916/225000 [152:33:25<151:46:44,  3.74s/it]
 35%|███▌      | 78917/225000 [152:33:29<148:14:05,  3.65s/it]
 35%|███▌      | 78918/225000 [152:33:32<144:54:47,  3.57s/it]
 35%|███▌      | 78919/225000 [152:33:35<143:07:31,  3.53s/it]
 35%|███▌      | 78920/225000 [152:33:39<137:19:57,  3.38s/it]
 35%|███▌      | 78921/225000 [152:33:42<133:30:30,  3.29s/it]
 35%|███▌      | 78922/225000 [152:33:45<135:28:26,  3.34s/it]
 35%|███▌      | 78923/225000 [152:33:49<140:29:46,  3.46s/it]
 35%|███▌      | 78924/225000 [152:33:52<139:27:44,  3.44s/it]
 35%|███▌      | 78925/225000 [152:33:55<136:05:31,  3.35s/it]
 35%|███▌      | 78926/225000 [152:33:59<134:58:37,  3.33s/it]
 35%|███▌      | 78927/225000 [152:34:06<182:32:50,  4.50s/it]
 35%|███▌      | 78928/225000 [152:34:13<218:01:50,  5.37s/it]
 35%|███▌      | 78929/225000 [152:34:16<183:32:21,  4.52s/it]
 35%|███▌      | 78930/225000 [152:34:19<163:33:20,  4.03s/it]
 35%|███▌      | 78931/225000 [152:34:22<152:54:04,  3.77s/it]
 35%|███▌      | 78932/225000 [152:34:25<141:38:51,  3.49s/it]
 35%|███▌      | 78933/225000 [152:34:27<129:47:02,  3.20s/it]
 35%|███▌      | 78934/225000 [152:34:30<125:35:42,  3.10s/it]
 35%|███▌      | 78935/225000 [152:34:37<173:20:43,  4.27s/it]
 35%|███▌      | 78936/225000 [152:34:46<228:39:44,  5.64s/it]
 35%|███▌      | 78937/225000 [152:34:48<187:37:17,  4.62s/it]
 35%|███▌      | 78938/225000 [152:34:53<194:46:06,  4.80s/it]
 35%|███▌      | 78939/225000 [152:34:56<173:36:20,  4.28s/it]
 35%|███▌      | 78940/225000 [152:34:59<155:27:17,  3.83s/it]
 35%|███▌      | 78941/225000 [152:35:02<147:59:31,  3.65s/it]
 35%|███▌      | 78942/225000 [152:35:05<135:36:42,  3.34s/it]
 35%|███▌      | 78943/225000 [152:35:13<194:31:08,  4.79s/it]
 35%|███▌      | 78944/225000 [152:35:16<172:27:11,  4.25s/it]
 35%|███▌      | 78945/225000 [152:35:20<165:16:10,  4.07s/it]
 35%|███▌      | 78946/225000 [152:35:23<149:31:37,  3.69s/it]
 35%|███▌      | 78947/225000 [152:35:26<146:42:53,  3.62s/it]
 35%|███▌      | 78948/225000 [152:35:29<139:47:07,  3.45s/it]
 35%|███▌      | 78949/225000 [152:35:32<134:40:16,  3.32s/it]
 35%|███▌      | 78950/225000 [152:35:35<133:53:38,  3.30s/it]
                                                              
{'loss': '0.8086', 'grad_norm': '3.873', 'learning_rate': '3.836e-05', 'epoch': '0.3509', 'num_input_tokens_seen': 251293464, 'train_runtime': '5.493e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 78950/225000 [152:35:35<133:53:38,  3.30s/it]
 35%|███▌      | 78951/225000 [152:35:41<166:11:23,  4.10s/it]
 35%|███▌      | 78952/225000 [152:35:50<225:07:13,  5.55s/it]
 35%|███▌      | 78953/225000 [152:35:53<187:12:47,  4.61s/it]
 35%|███▌      | 78954/225000 [152:35:56<164:39:46,  4.06s/it]
 35%|███▌      | 78955/225000 [152:35:58<146:34:19,  3.61s/it]
 35%|███▌      | 78956/225000 [152:36:04<172:22:21,  4.25s/it]
 35%|███▌      | 78957/225000 [152:36:09<179:27:29,  4.42s/it]
 35%|███▌      | 78958/225000 [152:36:11<154:38:45,  3.81s/it]
 35%|███▌      | 78959/225000 [152:36:14<141:27:39,  3.49s/it]
 35%|███▌      | 78960/225000 [152:36:16<130:12:25,  3.21s/it]
 35%|███▌      | 78961/225000 [152:36:20<129:56:01,  3.20s/it]
 35%|███▌      | 78962/225000 [152:36:27<181:19:13,  4.47s/it]
 35%|███▌      | 78963/225000 [152:36:33<197:06:38,  4.86s/it]
 35%|███▌      | 78964/225000 [152:36:35<166:07:01,  4.10s/it]
 35%|███▌      | 78965/225000 [152:36:38<150:13:23,  3.70s/it]
 35%|███▌      | 78966/225000 [152:36:40<137:32:11,  3.39s/it]
 35%|███▌      | 78967/225000 [152:36:43<128:33:07,  3.17s/it]
 35%|███▌      | 78968/225000 [152:36:46<122:49:00,  3.03s/it]
 35%|███▌      | 78969/225000 [152:36:53<167:11:26,  4.12s/it]
 35%|███▌      | 78970/225000 [152:36:59<198:05:30,  4.88s/it]
 35%|███▌      | 78971/225000 [152:37:02<170:42:30,  4.21s/it]
 35%|███▌      | 78972/225000 [152:37:04<146:31:55,  3.61s/it]
 35%|███▌      | 78973/225000 [152:37:10<173:09:32,  4.27s/it]
 35%|███▌      | 78974/225000 [152:37:13<153:46:38,  3.79s/it]
 35%|███▌      | 78975/225000 [152:37:15<141:04:51,  3.48s/it]
 35%|███▌      | 78976/225000 [152:37:18<131:00:37,  3.23s/it]
 35%|███▌      | 78977/225000 [152:37:21<133:49:25,  3.30s/it]
 35%|███▌      | 78978/225000 [152:37:24<129:21:04,  3.19s/it]
 35%|███▌      | 78979/225000 [152:37:30<160:37:47,  3.96s/it]
 35%|███▌      | 78980/225000 [152:37:38<209:43:00,  5.17s/it]
 35%|███▌      | 78981/225000 [152:37:41<178:35:05,  4.40s/it]
 35%|███▌      | 78982/225000 [152:37:44<163:29:06,  4.03s/it]
 35%|███▌      | 78983/225000 [152:37:50<192:59:44,  4.76s/it]
 35%|███▌      | 78984/225000 [152:37:56<199:55:01,  4.93s/it]
 35%|███▌      | 78985/225000 [152:37:58<171:39:19,  4.23s/it]
 35%|███▌      | 78986/225000 [152:38:02<169:41:53,  4.18s/it]
 35%|███▌      | 78987/225000 [152:38:08<187:39:08,  4.63s/it]
 35%|███▌      | 78988/225000 [152:38:11<164:44:03,  4.06s/it]
 35%|███▌      | 78989/225000 [152:38:13<146:06:38,  3.60s/it]
 35%|███▌      | 78990/225000 [152:38:16<133:34:18,  3.29s/it]
 35%|███▌      | 78991/225000 [152:38:25<202:38:50,  5.00s/it]
 35%|███▌      | 78992/225000 [152:38:28<178:09:01,  4.39s/it]
 35%|███▌      | 78993/225000 [152:38:30<151:43:09,  3.74s/it]
 35%|███▌      | 78994/225000 [152:38:33<140:20:32,  3.46s/it]
 35%|███▌      | 78995/225000 [152:38:36<134:34:39,  3.32s/it]
 35%|███▌      | 78996/225000 [152:38:39<131:45:49,  3.25s/it]
 35%|███▌      | 78997/225000 [152:38:42<132:10:52,  3.26s/it]
 35%|███▌      | 78998/225000 [152:38:46<137:30:02,  3.39s/it]
 35%|███▌      | 78999/225000 [152:38:49<139:21:01,  3.44s/it]
 35%|███▌      | 79000/225000 [152:38:53<140:58:59,  3.48s/it]
                                                              
{'loss': '0.8531', 'grad_norm': '3.395', 'learning_rate': '3.835e-05', 'epoch': '0.3511', 'num_input_tokens_seen': 251376072, 'train_runtime': '5.495e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79000/225000 [152:38:53<140:58:59,  3.48s/it]
 35%|███▌      | 79001/225000 [152:38:59<175:56:29,  4.34s/it]
 35%|███▌      | 79002/225000 [152:39:03<171:02:54,  4.22s/it]
 35%|███▌      | 79003/225000 [152:39:07<160:33:39,  3.96s/it]
 35%|███▌      | 79004/225000 [152:39:10<154:49:41,  3.82s/it]
 35%|███▌      | 79005/225000 [152:39:14<158:31:13,  3.91s/it]
 35%|███▌      | 79006/225000 [152:39:21<197:51:22,  4.88s/it]
 35%|███▌      | 79007/225000 [152:39:29<226:10:26,  5.58s/it]
 35%|███▌      | 79008/225000 [152:39:31<193:53:16,  4.78s/it]
 35%|███▌      | 79009/225000 [152:39:34<168:04:12,  4.14s/it]
 35%|███▌      | 79010/225000 [152:39:37<151:52:20,  3.75s/it]
 35%|███▌      | 79011/225000 [152:39:40<139:56:48,  3.45s/it]
 35%|███▌      | 79012/225000 [152:39:43<134:14:03,  3.31s/it]
 35%|███▌      | 79013/225000 [152:39:53<216:38:09,  5.34s/it]
 35%|███▌      | 79014/225000 [152:39:56<185:33:15,  4.58s/it]
 35%|███▌      | 79015/225000 [152:39:58<158:45:07,  3.91s/it]
 35%|███▌      | 79016/225000 [152:40:00<141:33:38,  3.49s/it]
 35%|███▌      | 79017/225000 [152:40:09<206:09:57,  5.08s/it]
 35%|███▌      | 79018/225000 [152:40:12<172:57:09,  4.27s/it]
 35%|███▌      | 79019/225000 [152:40:14<156:08:10,  3.85s/it]
 35%|███▌      | 79020/225000 [152:40:21<185:06:36,  4.56s/it]
 35%|███▌      | 79021/225000 [152:40:24<172:23:10,  4.25s/it]
 35%|███▌      | 79022/225000 [152:40:27<156:34:29,  3.86s/it]
 35%|███▌      | 79023/225000 [152:40:30<143:10:58,  3.53s/it]
 35%|███▌      | 79024/225000 [152:40:33<139:02:27,  3.43s/it]
 35%|███▌      | 79025/225000 [152:40:36<133:19:58,  3.29s/it]
 35%|███▌      | 79026/225000 [152:40:39<132:18:31,  3.26s/it]
 35%|███▌      | 79027/225000 [152:40:42<125:17:06,  3.09s/it]
 35%|███▌      | 79028/225000 [152:40:45<121:27:42,  3.00s/it]
 35%|███▌      | 79029/225000 [152:40:53<181:56:15,  4.49s/it]
 35%|███▌      | 79030/225000 [152:40:58<197:30:26,  4.87s/it]
 35%|███▌      | 79031/225000 [152:41:01<171:27:06,  4.23s/it]
 35%|███▌      | 79032/225000 [152:41:04<156:31:27,  3.86s/it]
 35%|███▌      | 79033/225000 [152:41:07<143:01:00,  3.53s/it]
 35%|███▌      | 79034/225000 [152:41:10<131:58:36,  3.25s/it]
 35%|███▌      | 79035/225000 [152:41:12<127:11:10,  3.14s/it]
 35%|███▌      | 79036/225000 [152:41:15<120:07:36,  2.96s/it]
 35%|███▌      | 79037/225000 [152:41:22<172:11:17,  4.25s/it]
 35%|███▌      | 79038/225000 [152:41:28<196:23:08,  4.84s/it]
 35%|███▌      | 79039/225000 [152:41:31<169:13:19,  4.17s/it]
 35%|███▌      | 79040/225000 [152:41:33<147:00:16,  3.63s/it]
 35%|███▌      | 79041/225000 [152:41:37<141:43:34,  3.50s/it]
 35%|███▌      | 79042/225000 [152:41:39<133:06:29,  3.28s/it]
 35%|███▌      | 79043/225000 [152:41:43<132:01:19,  3.26s/it]
 35%|███▌      | 79044/225000 [152:41:49<169:15:24,  4.17s/it]
 35%|███▌      | 79045/225000 [152:41:54<174:18:22,  4.30s/it]
 35%|███▌      | 79046/225000 [152:41:57<163:49:29,  4.04s/it]
 35%|███▌      | 79047/225000 [152:42:00<156:59:02,  3.87s/it]
 35%|███▌      | 79048/225000 [152:42:04<153:57:33,  3.80s/it]
 35%|███▌      | 79049/225000 [152:42:08<151:26:31,  3.74s/it]
 35%|███▌      | 79050/225000 [152:42:12<152:51:07,  3.77s/it]
                                                              
{'loss': '0.8383', 'grad_norm': '3.106', 'learning_rate': '3.833e-05', 'epoch': '0.3513', 'num_input_tokens_seen': 251458976, 'train_runtime': '5.497e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79050/225000 [152:42:12<152:51:07,  3.77s/it]
 35%|███▌      | 79051/225000 [152:42:15<146:26:20,  3.61s/it]
 35%|███▌      | 79052/225000 [152:42:18<145:39:01,  3.59s/it]
 35%|███▌      | 79053/225000 [152:42:22<143:42:31,  3.54s/it]
 35%|███▌      | 79054/225000 [152:42:25<142:31:50,  3.52s/it]
 35%|███▌      | 79055/225000 [152:42:28<136:57:49,  3.38s/it]
 35%|███▌      | 79056/225000 [152:42:32<139:50:04,  3.45s/it]
 35%|███▌      | 79057/225000 [152:42:35<137:00:48,  3.38s/it]
 35%|███▌      | 79058/225000 [152:42:38<135:04:29,  3.33s/it]
 35%|███▌      | 79059/225000 [152:42:47<197:25:28,  4.87s/it]
 35%|███▌      | 79060/225000 [152:42:56<246:27:28,  6.08s/it]
 35%|███▌      | 79061/225000 [152:42:59<207:58:51,  5.13s/it]
 35%|███▌      | 79062/225000 [152:43:01<177:10:24,  4.37s/it]
 35%|███▌      | 79063/225000 [152:43:04<161:12:39,  3.98s/it]
 35%|███▌      | 79064/225000 [152:43:07<147:55:18,  3.65s/it]
 35%|███▌      | 79065/225000 [152:43:15<194:46:06,  4.80s/it]
 35%|███▌      | 79066/225000 [152:43:17<165:36:56,  4.09s/it]
 35%|███▌      | 79067/225000 [152:43:19<145:14:11,  3.58s/it]
 35%|███▌      | 79068/225000 [152:43:22<130:32:44,  3.22s/it]
 35%|███▌      | 79069/225000 [152:43:27<151:02:17,  3.73s/it]
 35%|███▌      | 79070/225000 [152:43:35<200:47:22,  4.95s/it]
 35%|███▌      | 79071/225000 [152:43:37<169:16:25,  4.18s/it]
 35%|███▌      | 79072/225000 [152:43:41<172:19:44,  4.25s/it]
 35%|███▌      | 79073/225000 [152:43:45<167:36:23,  4.13s/it]
 35%|███▌      | 79074/225000 [152:43:48<155:09:44,  3.83s/it]
 35%|███▌      | 79075/225000 [152:43:52<147:47:50,  3.65s/it]
 35%|███▌      | 79076/225000 [152:43:54<138:48:36,  3.42s/it]
 35%|███▌      | 79077/225000 [152:43:57<132:05:30,  3.26s/it]
 35%|███▌      | 79078/225000 [152:44:00<130:46:41,  3.23s/it]
 35%|███▌      | 79079/225000 [152:44:04<139:39:37,  3.45s/it]
 35%|███▌      | 79080/225000 [152:44:07<132:40:12,  3.27s/it]
 35%|███▌      | 79081/225000 [152:44:16<199:41:28,  4.93s/it]
 35%|███▌      | 79082/225000 [152:44:24<232:18:57,  5.73s/it]
 35%|███▌      | 79083/225000 [152:44:26<194:21:54,  4.80s/it]
 35%|███▌      | 79084/225000 [152:44:29<170:23:52,  4.20s/it]
 35%|███▌      | 79085/225000 [152:44:32<157:01:30,  3.87s/it]
 35%|███▌      | 79086/225000 [152:44:35<146:59:16,  3.63s/it]
 35%|███▌      | 79087/225000 [152:44:38<140:42:10,  3.47s/it]
 35%|███▌      | 79088/225000 [152:44:41<131:32:00,  3.25s/it]
 35%|███▌      | 79089/225000 [152:44:45<139:13:18,  3.43s/it]
 35%|███▌      | 79090/225000 [152:44:54<206:55:00,  5.11s/it]
 35%|███▌      | 79091/225000 [152:44:59<204:51:15,  5.05s/it]
 35%|███▌      | 79092/225000 [152:45:01<173:15:50,  4.27s/it]
 35%|███▌      | 79093/225000 [152:45:04<152:20:44,  3.76s/it]
 35%|███▌      | 79094/225000 [152:45:12<200:27:06,  4.95s/it]
 35%|███▌      | 79095/225000 [152:45:14<174:18:34,  4.30s/it]
 35%|███▌      | 79096/225000 [152:45:17<148:59:19,  3.68s/it]
 35%|███▌      | 79097/225000 [152:45:20<143:21:06,  3.54s/it]
 35%|███▌      | 79098/225000 [152:45:26<174:27:24,  4.30s/it]
 35%|███▌      | 79099/225000 [152:45:30<167:05:36,  4.12s/it]
 35%|███▌      | 79100/225000 [152:45:33<157:16:16,  3.88s/it]
                                                              
{'loss': '0.828', 'grad_norm': '3.837', 'learning_rate': '3.832e-05', 'epoch': '0.3516', 'num_input_tokens_seen': 251540872, 'train_runtime': '5.499e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79100/225000 [152:45:33<157:16:16,  3.88s/it]
 35%|███▌      | 79101/225000 [152:45:37<158:13:42,  3.90s/it]
 35%|███▌      | 79102/225000 [152:45:43<181:44:37,  4.48s/it]
 35%|███▌      | 79103/225000 [152:45:46<164:20:48,  4.06s/it]
 35%|███▌      | 79104/225000 [152:45:48<143:56:07,  3.55s/it]
 35%|███▌      | 79105/225000 [152:45:51<131:10:39,  3.24s/it]
 35%|███▌      | 79106/225000 [152:45:54<133:28:52,  3.29s/it]
 35%|███▌      | 79107/225000 [152:46:01<182:30:05,  4.50s/it]
 35%|███▌      | 79108/225000 [152:46:04<159:55:21,  3.95s/it]
 35%|███▌      | 79109/225000 [152:46:07<141:44:01,  3.50s/it]
 35%|███▌      | 79110/225000 [152:46:09<128:21:53,  3.17s/it]
 35%|███▌      | 79111/225000 [152:46:13<144:58:18,  3.58s/it]
 35%|███▌      | 79112/225000 [152:46:20<185:51:58,  4.59s/it]
 35%|███▌      | 79113/225000 [152:46:23<163:17:09,  4.03s/it]
 35%|███▌      | 79114/225000 [152:46:26<143:50:08,  3.55s/it]
 35%|███▌      | 79115/225000 [152:46:32<175:39:08,  4.33s/it]
 35%|███▌      | 79116/225000 [152:46:35<162:53:39,  4.02s/it]
 35%|███▌      | 79117/225000 [152:46:39<157:30:48,  3.89s/it]
 35%|███▌      | 79118/225000 [152:46:42<150:00:21,  3.70s/it]
 35%|███▌      | 79119/225000 [152:46:45<146:36:13,  3.62s/it]
 35%|███▌      | 79120/225000 [152:46:48<140:31:03,  3.47s/it]
 35%|███▌      | 79121/225000 [152:46:51<134:23:11,  3.32s/it]
 35%|███▌      | 79122/225000 [152:46:54<128:24:32,  3.17s/it]
 35%|███▌      | 79123/225000 [152:46:57<127:21:44,  3.14s/it]
 35%|███▌      | 79124/225000 [152:47:05<188:49:20,  4.66s/it]
 35%|███▌      | 79125/225000 [152:47:11<204:12:14,  5.04s/it]
 35%|███▌      | 79126/225000 [152:47:14<180:28:19,  4.45s/it]
 35%|███▌      | 79127/225000 [152:47:17<158:36:37,  3.91s/it]
 35%|███▌      | 79128/225000 [152:47:20<146:36:02,  3.62s/it]
 35%|███▌      | 79129/225000 [152:47:28<201:22:23,  4.97s/it]
 35%|███▌      | 79130/225000 [152:47:32<186:20:03,  4.60s/it]
 35%|███▌      | 79131/225000 [152:47:34<159:49:58,  3.94s/it]
 35%|███▌      | 79132/225000 [152:47:37<142:01:04,  3.50s/it]
 35%|███▌      | 79133/225000 [152:47:43<179:31:45,  4.43s/it]
 35%|███▌      | 79134/225000 [152:47:50<205:53:45,  5.08s/it]
 35%|███▌      | 79135/225000 [152:47:53<178:18:17,  4.40s/it]
 35%|███▌      | 79136/225000 [152:47:56<159:59:28,  3.95s/it]
 35%|███▌      | 79137/225000 [152:47:58<145:50:35,  3.60s/it]
 35%|███▌      | 79138/225000 [152:48:01<134:54:23,  3.33s/it]
 35%|███▌      | 79139/225000 [152:48:04<130:13:16,  3.21s/it]
 35%|███▌      | 79140/225000 [152:48:08<132:43:09,  3.28s/it]
 35%|███▌      | 79141/225000 [152:48:11<134:02:18,  3.31s/it]
 35%|███▌      | 79142/225000 [152:48:14<132:20:49,  3.27s/it]
 35%|███▌      | 79143/225000 [152:48:21<179:39:48,  4.43s/it]
 35%|███▌      | 79144/225000 [152:48:29<219:49:45,  5.43s/it]
 35%|███▌      | 79145/225000 [152:48:32<185:32:28,  4.58s/it]
 35%|███▌      | 79146/225000 [152:48:34<163:46:34,  4.04s/it]
 35%|███▌      | 79147/225000 [152:48:38<157:38:23,  3.89s/it]
 35%|███▌      | 79148/225000 [152:48:45<190:33:22,  4.70s/it]
 35%|███▌      | 79149/225000 [152:48:47<166:48:14,  4.12s/it]
 35%|███▌      | 79150/225000 [152:48:50<148:17:07,  3.66s/it]
                                                              
{'loss': '0.8299', 'grad_norm': '2.344', 'learning_rate': '3.83e-05', 'epoch': '0.3518', 'num_input_tokens_seen': 251623664, 'train_runtime': '5.501e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79150/225000 [152:48:50<148:17:07,  3.66s/it]
 35%|███▌      | 79151/225000 [152:48:53<135:49:05,  3.35s/it]
 35%|███▌      | 79152/225000 [152:48:55<129:44:50,  3.20s/it]
 35%|███▌      | 79153/225000 [152:48:58<121:38:07,  3.00s/it]
 35%|███▌      | 79154/225000 [152:49:02<136:00:50,  3.36s/it]
 35%|███▌      | 79155/225000 [152:49:11<198:07:38,  4.89s/it]
 35%|███▌      | 79156/225000 [152:49:13<169:43:23,  4.19s/it]
 35%|███▌      | 79157/225000 [152:49:16<149:22:41,  3.69s/it]
 35%|███▌      | 79158/225000 [152:49:18<138:14:05,  3.41s/it]
 35%|███▌      | 79159/225000 [152:49:21<133:22:35,  3.29s/it]
 35%|███▌      | 79160/225000 [152:49:24<128:10:28,  3.16s/it]
 35%|███▌      | 79161/225000 [152:49:32<182:48:26,  4.51s/it]
 35%|███▌      | 79162/225000 [152:49:38<198:27:32,  4.90s/it]
 35%|███▌      | 79163/225000 [152:49:44<216:08:41,  5.34s/it]
 35%|███▌      | 79164/225000 [152:49:47<182:44:14,  4.51s/it]
 35%|███▌      | 79165/225000 [152:49:49<157:12:59,  3.88s/it]
 35%|███▌      | 79166/225000 [152:49:51<139:14:53,  3.44s/it]
 35%|███▌      | 79167/225000 [152:49:54<127:07:37,  3.14s/it]
 35%|███▌      | 79168/225000 [152:49:57<126:33:44,  3.12s/it]
 35%|███▌      | 79169/225000 [152:50:00<123:02:53,  3.04s/it]
 35%|███▌      | 79170/225000 [152:50:06<160:11:24,  3.95s/it]
 35%|███▌      | 79171/225000 [152:50:15<225:15:25,  5.56s/it]
 35%|███▌      | 79172/225000 [152:50:18<186:05:34,  4.59s/it]
 35%|███▌      | 79173/225000 [152:50:20<161:52:53,  4.00s/it]
 35%|███▌      | 79174/225000 [152:50:22<141:06:18,  3.48s/it]
 35%|███▌      | 79175/225000 [152:50:30<193:11:10,  4.77s/it]
 35%|███▌      | 79176/225000 [152:50:33<173:53:51,  4.29s/it]
 35%|███▌      | 79177/225000 [152:50:36<154:08:36,  3.81s/it]
 35%|███▌      | 79178/225000 [152:50:39<143:26:33,  3.54s/it]
 35%|███▌      | 79179/225000 [152:50:42<135:06:38,  3.34s/it]
 35%|███▌      | 79180/225000 [152:50:45<132:06:29,  3.26s/it]
 35%|███▌      | 79181/225000 [152:50:48<127:13:53,  3.14s/it]
 35%|███▌      | 79182/225000 [152:50:53<154:47:44,  3.82s/it]
 35%|███▌      | 79183/225000 [152:51:02<210:53:44,  5.21s/it]
 35%|███▌      | 79184/225000 [152:51:05<190:55:39,  4.71s/it]
 35%|███▌      | 79185/225000 [152:51:08<163:07:48,  4.03s/it]
 35%|███▌      | 79186/225000 [152:51:11<155:59:25,  3.85s/it]
 35%|███▌      | 79187/225000 [152:51:14<147:28:31,  3.64s/it]
 35%|███▌      | 79188/225000 [152:51:18<145:03:08,  3.58s/it]
 35%|███▌      | 79189/225000 [152:51:21<135:47:20,  3.35s/it]
 35%|███▌      | 79190/225000 [152:51:24<137:03:57,  3.38s/it]
 35%|███▌      | 79191/225000 [152:51:27<136:09:22,  3.36s/it]
 35%|███▌      | 79192/225000 [152:51:32<148:46:11,  3.67s/it]
 35%|███▌      | 79193/225000 [152:51:40<207:23:22,  5.12s/it]
 35%|███▌      | 79194/225000 [152:51:45<199:08:02,  4.92s/it]
 35%|███▌      | 79195/225000 [152:51:47<166:13:50,  4.10s/it]
 35%|███▌      | 79196/225000 [152:51:53<196:01:49,  4.84s/it]
 35%|███▌      | 79197/225000 [152:51:56<171:51:58,  4.24s/it]
 35%|███▌      | 79198/225000 [152:51:59<156:02:50,  3.85s/it]
 35%|███▌      | 79199/225000 [152:52:03<151:23:59,  3.74s/it]
 35%|███▌      | 79200/225000 [152:52:06<144:24:36,  3.57s/it]
                                                              
{'loss': '0.808', 'grad_norm': '2.982', 'learning_rate': '3.828e-05', 'epoch': '0.352', 'num_input_tokens_seen': 251706472, 'train_runtime': '5.503e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79200/225000 [152:52:06<144:24:36,  3.57s/it]
 35%|███▌      | 79201/225000 [152:52:09<141:58:13,  3.51s/it]
 35%|███▌      | 79202/225000 [152:52:13<148:32:21,  3.67s/it]
 35%|███▌      | 79203/225000 [152:52:17<150:46:04,  3.72s/it]
 35%|███▌      | 79204/225000 [152:52:21<147:46:11,  3.65s/it]
 35%|███▌      | 79205/225000 [152:52:24<149:07:37,  3.68s/it]
 35%|███▌      | 79206/225000 [152:52:28<148:48:57,  3.67s/it]
 35%|███▌      | 79207/225000 [152:52:36<195:52:28,  4.84s/it]
 35%|███▌      | 79208/225000 [152:52:42<218:59:38,  5.41s/it]
 35%|███▌      | 79209/225000 [152:52:45<182:07:51,  4.50s/it]
 35%|███▌      | 79210/225000 [152:52:47<156:28:53,  3.86s/it]
 35%|███▌      | 79211/225000 [152:52:55<209:53:38,  5.18s/it]
 35%|███▌      | 79212/225000 [152:52:58<181:22:14,  4.48s/it]
 35%|███▌      | 79213/225000 [152:53:01<156:37:21,  3.87s/it]
 35%|███▌      | 79214/225000 [152:53:03<139:46:44,  3.45s/it]
 35%|███▌      | 79215/225000 [152:53:11<190:37:33,  4.71s/it]
 35%|███▌      | 79216/225000 [152:53:13<164:22:53,  4.06s/it]
 35%|███▌      | 79217/225000 [152:53:16<154:03:50,  3.80s/it]
 35%|███▌      | 79218/225000 [152:53:19<143:46:49,  3.55s/it]
 35%|███▌      | 79219/225000 [152:53:22<131:49:06,  3.26s/it]
 35%|███▌      | 79220/225000 [152:53:25<127:29:09,  3.15s/it]
 35%|███▌      | 79221/225000 [152:53:33<187:54:01,  4.64s/it]
 35%|███▌      | 79222/225000 [152:53:37<183:41:55,  4.54s/it]
 35%|███▌      | 79223/225000 [152:53:40<159:05:02,  3.93s/it]
 35%|███▌      | 79224/225000 [152:53:42<141:56:52,  3.51s/it]
 35%|███▌      | 79225/225000 [152:53:45<129:55:32,  3.21s/it]
 35%|███▌      | 79226/225000 [152:53:48<125:01:16,  3.09s/it]
 35%|███▌      | 79227/225000 [152:53:54<168:28:45,  4.16s/it]
 35%|███▌      | 79228/225000 [152:53:59<173:07:08,  4.28s/it]
 35%|███▌      | 79229/225000 [152:54:01<152:19:55,  3.76s/it]
 35%|███▌      | 79230/225000 [152:54:06<166:47:22,  4.12s/it]
 35%|███▌      | 79231/225000 [152:54:10<164:35:35,  4.06s/it]
 35%|███▌      | 79232/225000 [152:54:13<153:28:38,  3.79s/it]
 35%|███▌      | 79233/225000 [152:54:19<169:08:07,  4.18s/it]
 35%|███▌      | 79234/225000 [152:54:23<177:54:26,  4.39s/it]
 35%|███▌      | 79235/225000 [152:54:26<157:09:21,  3.88s/it]
 35%|███▌      | 79236/225000 [152:54:29<143:57:01,  3.56s/it]
 35%|███▌      | 79237/225000 [152:54:32<139:06:06,  3.44s/it]
 35%|███▌      | 79238/225000 [152:54:35<131:08:10,  3.24s/it]
 35%|███▌      | 79239/225000 [152:54:38<132:31:09,  3.27s/it]
 35%|███▌      | 79240/225000 [152:54:47<198:37:08,  4.91s/it]
 35%|███▌      | 79241/225000 [152:54:50<181:22:51,  4.48s/it]
 35%|███▌      | 79242/225000 [152:54:53<163:09:03,  4.03s/it]
 35%|███▌      | 79243/225000 [152:54:58<173:11:28,  4.28s/it]
 35%|███▌      | 79244/225000 [152:55:03<177:59:36,  4.40s/it]
 35%|███▌      | 79245/225000 [152:55:06<159:10:47,  3.93s/it]
 35%|███▌      | 79246/225000 [152:55:08<143:43:07,  3.55s/it]
 35%|███▌      | 79247/225000 [152:55:14<164:49:27,  4.07s/it]
 35%|███▌      | 79248/225000 [152:55:21<201:30:29,  4.98s/it]
 35%|███▌      | 79249/225000 [152:55:23<169:01:38,  4.17s/it]
 35%|███▌      | 79250/225000 [152:55:26<149:53:19,  3.70s/it]
                                                              
{'loss': '0.8086', 'grad_norm': '5.041', 'learning_rate': '3.827e-05', 'epoch': '0.3522', 'num_input_tokens_seen': 251789856, 'train_runtime': '5.505e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79250/225000 [152:55:26<149:53:19,  3.70s/it]
 35%|███▌      | 79251/225000 [152:55:28<137:46:56,  3.40s/it]
 35%|███▌      | 79252/225000 [152:55:31<132:40:28,  3.28s/it]
 35%|███▌      | 79253/225000 [152:55:36<150:05:53,  3.71s/it]
 35%|███▌      | 79254/225000 [152:55:43<183:50:11,  4.54s/it]
 35%|███▌      | 79255/225000 [152:55:45<155:24:09,  3.84s/it]
 35%|███▌      | 79256/225000 [152:55:49<164:25:07,  4.06s/it]
 35%|███▌      | 79257/225000 [152:55:54<169:30:37,  4.19s/it]
 35%|███▌      | 79258/225000 [152:55:56<147:48:44,  3.65s/it]
 35%|███▌      | 79259/225000 [152:55:59<131:58:01,  3.26s/it]
 35%|███▌      | 79260/225000 [152:56:04<156:03:02,  3.85s/it]
 35%|███▌      | 79261/225000 [152:56:11<198:14:43,  4.90s/it]
 35%|███▌      | 79262/225000 [152:56:14<170:30:37,  4.21s/it]
 35%|███▌      | 79263/225000 [152:56:16<151:19:49,  3.74s/it]
 35%|███▌      | 79264/225000 [152:56:19<139:25:50,  3.44s/it]
 35%|███▌      | 79265/225000 [152:56:22<132:25:44,  3.27s/it]
 35%|███▌      | 79266/225000 [152:56:25<125:16:13,  3.09s/it]
 35%|███▌      | 79267/225000 [152:56:32<180:18:03,  4.45s/it]
 35%|███▌      | 79268/225000 [152:56:35<160:10:01,  3.96s/it]
 35%|███▌      | 79269/225000 [152:56:38<147:03:13,  3.63s/it]
 35%|███▌      | 79270/225000 [152:56:41<140:27:27,  3.47s/it]
 35%|███▌      | 79271/225000 [152:56:44<133:20:56,  3.29s/it]
 35%|███▌      | 79272/225000 [152:56:47<131:19:42,  3.24s/it]
 35%|███▌      | 79273/225000 [152:56:50<127:02:36,  3.14s/it]
 35%|███▌      | 79274/225000 [152:56:54<139:21:27,  3.44s/it]
 35%|███▌      | 79275/225000 [152:57:03<199:33:06,  4.93s/it]
 35%|███▌      | 79276/225000 [152:57:07<188:48:45,  4.66s/it]
 35%|███▌      | 79277/225000 [152:57:09<165:37:37,  4.09s/it]
 35%|███▌      | 79278/225000 [152:57:12<151:25:05,  3.74s/it]
 35%|███▌      | 79279/225000 [152:57:16<148:29:49,  3.67s/it]
 35%|███▌      | 79280/225000 [152:57:23<190:26:19,  4.70s/it]
 35%|███▌      | 79281/225000 [152:57:26<169:30:53,  4.19s/it]
 35%|███▌      | 79282/225000 [152:57:29<155:06:18,  3.83s/it]
 35%|███▌      | 79283/225000 [152:57:32<149:59:16,  3.71s/it]
 35%|███▌      | 79284/225000 [152:57:36<149:15:20,  3.69s/it]
 35%|███▌      | 79285/225000 [152:57:39<146:11:01,  3.61s/it]
 35%|███▌      | 79286/225000 [152:57:43<140:34:29,  3.47s/it]
 35%|███▌      | 79287/225000 [152:57:46<137:30:57,  3.40s/it]
 35%|███▌      | 79288/225000 [152:57:50<142:00:50,  3.51s/it]
 35%|███▌      | 79289/225000 [152:57:56<179:00:51,  4.42s/it]
 35%|███▌      | 79290/225000 [152:58:02<195:52:41,  4.84s/it]
 35%|███▌      | 79291/225000 [152:58:05<175:26:40,  4.33s/it]
 35%|███▌      | 79292/225000 [152:58:08<163:23:02,  4.04s/it]
 35%|███▌      | 79293/225000 [152:58:12<156:06:12,  3.86s/it]
 35%|███▌      | 79294/225000 [152:58:15<149:00:10,  3.68s/it]
 35%|███▌      | 79295/225000 [152:58:18<140:58:21,  3.48s/it]
 35%|███▌      | 79296/225000 [152:58:22<144:20:49,  3.57s/it]
 35%|███▌      | 79297/225000 [152:58:26<146:03:52,  3.61s/it]
 35%|███▌      | 79298/225000 [152:58:29<144:30:42,  3.57s/it]
 35%|███▌      | 79299/225000 [152:58:32<139:58:46,  3.46s/it]
 35%|███▌      | 79300/225000 [152:58:36<137:32:04,  3.40s/it]
                                                              
{'loss': '0.867', 'grad_norm': '5.172', 'learning_rate': '3.825e-05', 'epoch': '0.3524', 'num_input_tokens_seen': 251871136, 'train_runtime': '5.507e+05', 'train_tokens_per_second': '457.4'}

 35%|███▌      | 79300/225000 [152:58:36<137:32:04,  3.40s/it]
 35%|███▌      | 79301/225000 [152:58:39<136:22:52,  3.37s/it]
 35%|███▌      | 79302/225000 [152:58:42<137:59:01,  3.41s/it]
 35%|███▌      | 79303/225000 [152:58:46<144:15:11,  3.56s/it]
 35%|███▌      | 79304/225000 [152:58:50<143:55:52,  3.56s/it]
 35%|███▌      | 79305/225000 [152:58:59<209:51:17,  5.19s/it]
 35%|███▌      | 79306/225000 [152:59:04<214:49:14,  5.31s/it]
 35%|███▌      | 79307/225000 [152:59:10<217:02:54,  5.36s/it]
 35%|███▌      | 79308/225000 [152:59:13<193:08:35,  4.77s/it]
 35%|███▌      | 79309/225000 [152:59:17<175:34:56,  4.34s/it]
 35%|███▌      | 79310/225000 [152:59:20<159:58:32,  3.95s/it]
 35%|███▌      | 79311/225000 [152:59:23<148:50:04,  3.68s/it]
 35%|███▌      | 79312/225000 [152:59:26<145:24:56,  3.59s/it]
 35%|███▌      | 79313/225000 [152:59:30<146:44:52,  3.63s/it]
 35%|███▌      | 79314/225000 [152:59:33<141:38:09,  3.50s/it]
 35%|███▌      | 79315/225000 [152:59:37<144:26:04,  3.57s/it]
 35%|███▌      | 79316/225000 [152:59:40<142:52:51,  3.53s/it]
 35%|███▌      | 79317/225000 [152:59:43<138:40:15,  3.43s/it]
 35%|███▌      | 79318/225000 [152:59:47<142:24:26,  3.52s/it]
 35%|███▌      | 79319/225000 [152:59:51<144:23:42,  3.57s/it]
 35%|███▌      | 79320/225000 [152:59:54<144:31:34,  3.57s/it]
 35%|███▌      | 79321/225000 [152:59:57<139:22:27,  3.44s/it]
 35%|███▌      | 79322/225000 [153:00:01<140:14:19,  3.47s/it]
 35%|███▌      | 79323/225000 [153:00:04<137:59:26,  3.41s/it]
 35%|███▌      | 79324/225000 [153:00:12<196:31:33,  4.86s/it]
 35%|███▌      | 79325/225000 [153:00:20<227:58:20,  5.63s/it]
 35%|███▌      | 79326/225000 [153:00:23<193:05:53,  4.77s/it]
 35%|███▌      | 79327/225000 [153:00:26<176:05:01,  4.35s/it]
 35%|███▌      | 79328/225000 [153:00:29<159:16:21,  3.94s/it]
 35%|███▌      | 79329/225000 [153:00:32<150:45:50,  3.73s/it]
 35%|███▌      | 79330/225000 [153:00:35<140:05:40,  3.46s/it]
 35%|███▌      | 79331/225000 [153:00:38<131:37:09,  3.25s/it]
 35%|███▌      | 79332/225000 [153:00:41<135:36:35,  3.35s/it]
 35%|███▌      | 79333/225000 [153:00:48<176:05:03,  4.35s/it]
 35%|███▌      | 79334/225000 [153:00:57<234:05:53,  5.79s/it]
 35%|███▌      | 79335/225000 [153:01:00<193:37:56,  4.79s/it]
 35%|███▌      | 79336/225000 [153:01:03<169:50:06,  4.20s/it]
 35%|███▌      | 79337/225000 [153:01:05<148:18:29,  3.67s/it]
 35%|███▌      | 79338/225000 [153:01:07<133:01:34,  3.29s/it]
 35%|███▌      | 79339/225000 [153:01:16<197:57:33,  4.89s/it]
 35%|███▌      | 79340/225000 [153:01:20<184:34:56,  4.56s/it]
 35%|███▌      | 79341/225000 [153:01:22<158:25:56,  3.92s/it]
 35%|███▌      | 79342/225000 [153:01:25<144:38:29,  3.57s/it]
 35%|███▌      | 79343/225000 [153:01:28<138:43:16,  3.43s/it]
 35%|███▌      | 79344/225000 [153:01:32<144:07:40,  3.56s/it]
 35%|███▌      | 79345/225000 [153:01:36<145:21:47,  3.59s/it]
 35%|███▌      | 79346/225000 [153:01:39<143:33:07,  3.55s/it]
 35%|███▌      | 79347/225000 [153:01:42<138:01:28,  3.41s/it]
 35%|███▌      | 79348/225000 [153:01:45<136:49:43,  3.38s/it]
 35%|███▌      | 79349/225000 [153:01:49<139:01:34,  3.44s/it]
 35%|███▌      | 79350/225000 [153:01:52<138:13:16,  3.42s/it]
                                                              
{'loss': '0.8572', 'grad_norm': '2.804', 'learning_rate': '3.824e-05', 'epoch': '0.3527', 'num_input_tokens_seen': 251953864, 'train_runtime': '5.509e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79350/225000 [153:01:52<138:13:16,  3.42s/it]
 35%|███▌      | 79351/225000 [153:01:56<135:31:12,  3.35s/it]
 35%|███▌      | 79352/225000 [153:02:00<145:57:11,  3.61s/it]
 35%|███▌      | 79353/225000 [153:02:03<140:31:59,  3.47s/it]
 35%|███▌      | 79354/225000 [153:02:07<150:35:15,  3.72s/it]
 35%|███▌      | 79355/225000 [153:02:16<213:52:52,  5.29s/it]
 35%|███▌      | 79356/225000 [153:02:22<217:16:06,  5.37s/it]
 35%|███▌      | 79357/225000 [153:02:24<181:27:11,  4.49s/it]
 35%|███▌      | 79358/225000 [153:02:27<158:28:55,  3.92s/it]
 35%|███▌      | 79359/225000 [153:02:30<146:11:23,  3.61s/it]
 35%|███▌      | 79360/225000 [153:02:37<195:31:29,  4.83s/it]
 35%|███▌      | 79361/225000 [153:02:40<168:10:47,  4.16s/it]
 35%|███▌      | 79362/225000 [153:02:43<148:50:57,  3.68s/it]
 35%|███▌      | 79363/225000 [153:02:45<138:12:15,  3.42s/it]
 35%|███▌      | 79364/225000 [153:02:53<191:53:34,  4.74s/it]
 35%|███▌      | 79365/225000 [153:02:56<166:43:10,  4.12s/it]
 35%|███▌      | 79366/225000 [153:02:59<150:48:12,  3.73s/it]
 35%|███▌      | 79367/225000 [153:03:02<144:00:01,  3.56s/it]
 35%|███▌      | 79368/225000 [153:03:05<143:37:51,  3.55s/it]
 35%|███▌      | 79369/225000 [153:03:09<143:49:30,  3.56s/it]
 35%|███▌      | 79370/225000 [153:03:12<139:19:51,  3.44s/it]
 35%|███▌      | 79371/225000 [153:03:15<136:24:53,  3.37s/it]
 35%|███▌      | 79372/225000 [153:03:19<144:07:31,  3.56s/it]
 35%|███▌      | 79373/225000 [153:03:25<174:56:29,  4.32s/it]
 35%|███▌      | 79374/225000 [153:03:32<201:51:11,  4.99s/it]
 35%|███▌      | 79375/225000 [153:03:35<179:18:43,  4.43s/it]
 35%|███▌      | 79376/225000 [153:03:38<157:42:34,  3.90s/it]
 35%|███▌      | 79377/225000 [153:03:40<142:59:18,  3.53s/it]
 35%|███▌      | 79378/225000 [153:03:47<185:02:50,  4.57s/it]
 35%|███▌      | 79379/225000 [153:03:53<195:55:25,  4.84s/it]
 35%|███▌      | 79380/225000 [153:03:55<166:51:24,  4.13s/it]
 35%|███▌      | 79381/225000 [153:04:00<178:36:40,  4.42s/it]
 35%|███▌      | 79382/225000 [153:04:03<155:06:14,  3.83s/it]
 35%|███▌      | 79383/225000 [153:04:05<138:21:16,  3.42s/it]
 35%|███▌      | 79384/225000 [153:04:10<154:33:34,  3.82s/it]
 35%|███▌      | 79385/225000 [153:04:15<164:39:04,  4.07s/it]
 35%|███▌      | 79386/225000 [153:04:17<143:35:35,  3.55s/it]
 35%|███▌      | 79387/225000 [153:04:20<133:58:21,  3.31s/it]
 35%|███▌      | 79388/225000 [153:04:23<129:05:21,  3.19s/it]
 35%|███▌      | 79389/225000 [153:04:26<129:27:12,  3.20s/it]
 35%|███▌      | 79390/225000 [153:04:29<127:11:27,  3.14s/it]
 35%|███▌      | 79391/225000 [153:04:32<129:49:16,  3.21s/it]
 35%|███▌      | 79392/225000 [153:04:36<130:23:40,  3.22s/it]
 35%|███▌      | 79393/225000 [153:04:39<135:30:41,  3.35s/it]
 35%|███▌      | 79394/225000 [153:04:43<137:36:03,  3.40s/it]
 35%|███▌      | 79395/225000 [153:04:46<133:20:45,  3.30s/it]
 35%|███▌      | 79396/225000 [153:04:54<188:52:35,  4.67s/it]
 35%|███▌      | 79397/225000 [153:05:02<231:20:58,  5.72s/it]
 35%|███▌      | 79398/225000 [153:05:05<193:58:44,  4.80s/it]
 35%|███▌      | 79399/225000 [153:05:08<173:30:15,  4.29s/it]
 35%|███▌      | 79400/225000 [153:05:15<205:43:29,  5.09s/it]
                                                              
{'loss': '0.8301', 'grad_norm': '5.282', 'learning_rate': '3.822e-05', 'epoch': '0.3529', 'num_input_tokens_seen': 252036664, 'train_runtime': '5.511e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79400/225000 [153:05:15<205:43:29,  5.09s/it]
 35%|███▌      | 79401/225000 [153:05:17<175:58:07,  4.35s/it]
 35%|███▌      | 79402/225000 [153:05:21<164:56:54,  4.08s/it]
 35%|███▌      | 79403/225000 [153:05:24<151:02:50,  3.73s/it]
 35%|███▌      | 79404/225000 [153:05:28<161:06:44,  3.98s/it]
 35%|███▌      | 79405/225000 [153:05:36<208:08:58,  5.15s/it]
 35%|███▌      | 79406/225000 [153:05:39<181:36:32,  4.49s/it]
 35%|███▌      | 79407/225000 [153:05:42<161:29:01,  3.99s/it]
 35%|███▌      | 79408/225000 [153:05:45<154:36:11,  3.82s/it]
 35%|███▌      | 79409/225000 [153:05:48<144:19:48,  3.57s/it]
 35%|███▌      | 79410/225000 [153:05:53<157:28:38,  3.89s/it]
 35%|███▌      | 79411/225000 [153:06:02<220:22:05,  5.45s/it]
 35%|███▌      | 79412/225000 [153:06:05<185:48:26,  4.59s/it]
 35%|███▌      | 79413/225000 [153:06:07<158:20:13,  3.92s/it]
 35%|███▌      | 79414/225000 [153:06:14<200:48:26,  4.97s/it]
 35%|███▌      | 79415/225000 [153:06:17<170:50:37,  4.22s/it]
 35%|███▌      | 79416/225000 [153:06:19<151:11:00,  3.74s/it]
 35%|███▌      | 79417/225000 [153:06:22<135:46:00,  3.36s/it]
 35%|███▌      | 79418/225000 [153:06:25<129:17:08,  3.20s/it]
 35%|███▌      | 79419/225000 [153:06:28<125:41:37,  3.11s/it]
 35%|███▌      | 79420/225000 [153:06:35<175:20:00,  4.34s/it]
 35%|███▌      | 79421/225000 [153:06:41<195:17:42,  4.83s/it]
 35%|███▌      | 79422/225000 [153:06:43<167:41:58,  4.15s/it]
 35%|███▌      | 79423/225000 [153:06:47<162:24:33,  4.02s/it]
 35%|███▌      | 79424/225000 [153:06:51<156:17:44,  3.87s/it]
 35%|███▌      | 79425/225000 [153:06:53<144:02:50,  3.56s/it]
 35%|███▌      | 79426/225000 [153:06:57<138:31:41,  3.43s/it]
 35%|███▌      | 79427/225000 [153:07:00<135:00:22,  3.34s/it]
 35%|███▌      | 79428/225000 [153:07:03<137:01:48,  3.39s/it]
 35%|███▌      | 79429/225000 [153:07:11<192:38:11,  4.76s/it]
 35%|███▌      | 79430/225000 [153:07:16<196:05:36,  4.85s/it]
 35%|███▌      | 79431/225000 [153:07:19<175:47:06,  4.35s/it]
 35%|███▌      | 79432/225000 [153:07:22<156:40:31,  3.87s/it]
 35%|███▌      | 79433/225000 [153:07:25<147:03:34,  3.64s/it]
 35%|███▌      | 79434/225000 [153:07:29<147:32:52,  3.65s/it]
 35%|███▌      | 79435/225000 [153:07:32<144:30:23,  3.57s/it]
 35%|███▌      | 79436/225000 [153:07:35<136:58:52,  3.39s/it]
 35%|███▌      | 79437/225000 [153:07:39<135:33:00,  3.35s/it]
 35%|███▌      | 79438/225000 [153:07:42<132:25:42,  3.28s/it]
 35%|███▌      | 79439/225000 [153:07:47<159:16:21,  3.94s/it]
 35%|███▌      | 79440/225000 [153:07:55<205:06:02,  5.07s/it]
 35%|███▌      | 79441/225000 [153:07:57<173:07:34,  4.28s/it]
 35%|███▌      | 79442/225000 [153:08:00<154:09:22,  3.81s/it]
 35%|███▌      | 79443/225000 [153:08:03<140:00:16,  3.46s/it]
 35%|███▌      | 79444/225000 [153:08:07<148:02:32,  3.66s/it]
 35%|███▌      | 79445/225000 [153:08:16<214:54:13,  5.32s/it]
 35%|███▌      | 79446/225000 [153:08:19<188:36:53,  4.67s/it]
 35%|███▌      | 79447/225000 [153:08:22<165:54:59,  4.10s/it]
 35%|███▌      | 79448/225000 [153:08:25<149:09:44,  3.69s/it]
 35%|███▌      | 79449/225000 [153:08:31<177:01:13,  4.38s/it]
 35%|███▌      | 79450/225000 [153:08:35<177:05:28,  4.38s/it]
                                                              
{'loss': '0.7833', 'grad_norm': '2.092', 'learning_rate': '3.821e-05', 'epoch': '0.3531', 'num_input_tokens_seen': 252120816, 'train_runtime': '5.513e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79450/225000 [153:08:35<177:05:28,  4.38s/it]
 35%|███▌      | 79451/225000 [153:08:38<160:22:13,  3.97s/it]
 35%|███▌      | 79452/225000 [153:08:41<145:01:42,  3.59s/it]
 35%|███▌      | 79453/225000 [153:08:44<140:44:04,  3.48s/it]
 35%|███▌      | 79454/225000 [153:08:46<129:47:09,  3.21s/it]
 35%|███▌      | 79455/225000 [153:08:50<134:34:54,  3.33s/it]
 35%|███▌      | 79456/225000 [153:08:59<202:53:23,  5.02s/it]
 35%|███▌      | 79457/225000 [153:09:02<173:22:38,  4.29s/it]
 35%|███▌      | 79458/225000 [153:09:04<150:32:56,  3.72s/it]
 35%|███▌      | 79459/225000 [153:09:11<186:01:40,  4.60s/it]
 35%|███▌      | 79460/225000 [153:09:15<180:38:41,  4.47s/it]
 35%|███▌      | 79461/225000 [153:09:17<158:44:24,  3.93s/it]
 35%|███▌      | 79462/225000 [153:09:20<142:17:40,  3.52s/it]
 35%|███▌      | 79463/225000 [153:09:23<130:25:57,  3.23s/it]
 35%|███▌      | 79464/225000 [153:09:25<124:37:25,  3.08s/it]
 35%|███▌      | 79465/225000 [153:09:32<168:03:27,  4.16s/it]
 35%|███▌      | 79466/225000 [153:09:36<166:16:58,  4.11s/it]
 35%|███▌      | 79467/225000 [153:09:39<158:22:18,  3.92s/it]
 35%|███▌      | 79468/225000 [153:09:42<144:20:31,  3.57s/it]
 35%|███▌      | 79469/225000 [153:09:45<138:48:56,  3.43s/it]
 35%|███▌      | 79470/225000 [153:09:49<135:48:30,  3.36s/it]
 35%|███▌      | 79471/225000 [153:09:57<198:07:39,  4.90s/it]
 35%|███▌      | 79472/225000 [153:10:03<213:15:13,  5.28s/it]
 35%|███▌      | 79473/225000 [153:10:06<187:08:21,  4.63s/it]
 35%|███▌      | 79474/225000 [153:10:10<170:18:38,  4.21s/it]
 35%|███▌      | 79475/225000 [153:10:13<155:13:31,  3.84s/it]
 35%|███▌      | 79476/225000 [153:10:17<164:51:40,  4.08s/it]
 35%|███▌      | 79477/225000 [153:10:23<184:21:26,  4.56s/it]
 35%|███▌      | 79478/225000 [153:10:26<167:18:51,  4.14s/it]
 35%|███▌      | 79479/225000 [153:10:29<151:59:57,  3.76s/it]
 35%|███▌      | 79480/225000 [153:10:32<140:42:31,  3.48s/it]
 35%|███▌      | 79481/225000 [153:10:35<134:56:27,  3.34s/it]
 35%|███▌      | 79482/225000 [153:10:44<202:16:42,  5.00s/it]
 35%|███▌      | 79483/225000 [153:10:46<175:56:13,  4.35s/it]
 35%|███▌      | 79484/225000 [153:10:50<165:14:50,  4.09s/it]
 35%|███▌      | 79485/225000 [153:10:53<152:37:13,  3.78s/it]
 35%|███▌      | 79486/225000 [153:10:56<141:04:32,  3.49s/it]
 35%|███▌      | 79487/225000 [153:10:59<133:10:30,  3.29s/it]
 35%|███▌      | 79488/225000 [153:11:02<130:08:03,  3.22s/it]
 35%|███▌      | 79489/225000 [153:11:05<128:56:36,  3.19s/it]
 35%|███▌      | 79490/225000 [153:11:08<132:53:27,  3.29s/it]
 35%|███▌      | 79491/225000 [153:11:17<201:10:22,  4.98s/it]
 35%|███▌      | 79492/225000 [153:11:24<226:01:33,  5.59s/it]
 35%|███▌      | 79493/225000 [153:11:27<191:54:53,  4.75s/it]
 35%|███▌      | 79494/225000 [153:11:30<173:13:54,  4.29s/it]
 35%|███▌      | 79495/225000 [153:11:33<151:00:40,  3.74s/it]
 35%|███▌      | 79496/225000 [153:11:35<138:45:25,  3.43s/it]
 35%|███▌      | 79497/225000 [153:11:42<181:32:04,  4.49s/it]
 35%|███▌      | 79498/225000 [153:11:47<181:53:33,  4.50s/it]
 35%|███▌      | 79499/225000 [153:11:49<154:00:23,  3.81s/it]
 35%|███▌      | 79500/225000 [153:11:54<164:18:08,  4.07s/it]
                                                              
{'loss': '0.7827', 'grad_norm': '3.306', 'learning_rate': '3.819e-05', 'epoch': '0.3533', 'num_input_tokens_seen': 252203144, 'train_runtime': '5.515e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79500/225000 [153:11:54<164:18:08,  4.07s/it]
 35%|███▌      | 79501/225000 [153:11:58<165:15:08,  4.09s/it]
 35%|███▌      | 79502/225000 [153:12:02<164:17:36,  4.07s/it]
 35%|███▌      | 79503/225000 [153:12:05<158:18:41,  3.92s/it]
 35%|███▌      | 79504/225000 [153:12:12<188:28:21,  4.66s/it]
 35%|███▌      | 79505/225000 [153:12:14<160:43:42,  3.98s/it]
 35%|███▌      | 79506/225000 [153:12:18<153:39:52,  3.80s/it]
 35%|███▌      | 79507/225000 [153:12:20<141:53:44,  3.51s/it]
 35%|███▌      | 79508/225000 [153:12:23<133:18:54,  3.30s/it]
 35%|███▌      | 79509/225000 [153:12:26<129:51:17,  3.21s/it]
 35%|███▌      | 79510/225000 [153:12:29<127:29:41,  3.15s/it]
 35%|███▌      | 79511/225000 [153:12:37<188:08:34,  4.66s/it]
 35%|███▌      | 79512/225000 [153:12:41<174:42:21,  4.32s/it]
 35%|███▌      | 79513/225000 [153:12:44<154:52:37,  3.83s/it]
 35%|███▌      | 79514/225000 [153:12:46<141:02:18,  3.49s/it]
 35%|███▌      | 79515/225000 [153:12:49<131:31:23,  3.25s/it]
 35%|███▌      | 79516/225000 [153:12:52<132:56:50,  3.29s/it]
 35%|███▌      | 79517/225000 [153:12:57<150:56:21,  3.74s/it]
 35%|███▌      | 79518/225000 [153:13:06<216:57:38,  5.37s/it]
 35%|███▌      | 79519/225000 [153:13:10<193:31:45,  4.79s/it]
 35%|███▌      | 79520/225000 [153:13:13<170:02:51,  4.21s/it]
 35%|███▌      | 79521/225000 [153:13:16<155:23:25,  3.85s/it]
 35%|███▌      | 79522/225000 [153:13:19<144:53:09,  3.59s/it]
 35%|███▌      | 79523/225000 [153:13:26<193:38:43,  4.79s/it]
 35%|███▌      | 79524/225000 [153:13:29<170:23:39,  4.22s/it]
 35%|███▌      | 79525/225000 [153:13:32<147:53:53,  3.66s/it]
 35%|███▌      | 79526/225000 [153:13:37<168:45:19,  4.18s/it]
 35%|███▌      | 79527/225000 [153:13:41<173:18:36,  4.29s/it]
 35%|███▌      | 79528/225000 [153:13:44<149:47:56,  3.71s/it]
 35%|███▌      | 79529/225000 [153:13:47<139:24:15,  3.45s/it]
 35%|███▌      | 79530/225000 [153:13:54<188:49:09,  4.67s/it]
 35%|███▌      | 79531/225000 [153:13:57<160:55:15,  3.98s/it]
 35%|███▌      | 79532/225000 [153:13:59<141:04:50,  3.49s/it]
 35%|███▌      | 79533/225000 [153:14:02<135:56:29,  3.36s/it]
 35%|███▌      | 79534/225000 [153:14:09<178:29:45,  4.42s/it]
 35%|███▌      | 79535/225000 [153:14:13<172:48:12,  4.28s/it]
 35%|███▌      | 79536/225000 [153:14:16<158:54:52,  3.93s/it]
 35%|███▌      | 79537/225000 [153:14:19<149:22:15,  3.70s/it]
 35%|███▌      | 79538/225000 [153:14:23<146:02:55,  3.61s/it]
 35%|███▌      | 79539/225000 [153:14:26<143:59:23,  3.56s/it]
 35%|███▌      | 79540/225000 [153:14:29<139:04:54,  3.44s/it]
 35%|███▌      | 79541/225000 [153:14:33<139:26:14,  3.45s/it]
 35%|███▌      | 79542/225000 [153:14:36<134:22:18,  3.33s/it]
 35%|███▌      | 79543/225000 [153:14:39<134:20:35,  3.32s/it]
 35%|███▌      | 79544/225000 [153:14:42<130:21:03,  3.23s/it]
 35%|███▌      | 79545/225000 [153:14:46<140:02:12,  3.47s/it]
 35%|███▌      | 79546/225000 [153:14:51<155:36:18,  3.85s/it]
 35%|███▌      | 79547/225000 [153:14:59<212:07:31,  5.25s/it]
 35%|███▌      | 79548/225000 [153:15:04<205:36:11,  5.09s/it]
 35%|███▌      | 79549/225000 [153:15:06<173:04:56,  4.28s/it]
 35%|███▌      | 79550/225000 [153:15:09<150:23:26,  3.72s/it]
                                                              
{'loss': '0.8182', 'grad_norm': '4.378', 'learning_rate': '3.818e-05', 'epoch': '0.3536', 'num_input_tokens_seen': 252285400, 'train_runtime': '5.517e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79550/225000 [153:15:09<150:23:26,  3.72s/it]
 35%|███▌      | 79551/225000 [153:15:16<190:45:12,  4.72s/it]
 35%|███▌      | 79552/225000 [153:15:21<197:07:46,  4.88s/it]
 35%|███▌      | 79553/225000 [153:15:24<169:32:37,  4.20s/it]
 35%|███▌      | 79554/225000 [153:15:26<152:55:42,  3.79s/it]
 35%|███▌      | 79555/225000 [153:15:29<138:17:22,  3.42s/it]
 35%|███▌      | 79556/225000 [153:15:32<127:45:13,  3.16s/it]
 35%|███▌      | 79557/225000 [153:15:34<124:20:41,  3.08s/it]
 35%|███▌      | 79558/225000 [153:15:40<154:22:30,  3.82s/it]
 35%|███▌      | 79559/225000 [153:15:48<203:51:17,  5.05s/it]
 35%|███▌      | 79560/225000 [153:15:51<182:18:32,  4.51s/it]
 35%|███▌      | 79561/225000 [153:15:54<161:35:04,  4.00s/it]
 35%|███▌      | 79562/225000 [153:15:57<150:53:16,  3.73s/it]
 35%|███▌      | 79563/225000 [153:16:00<144:40:03,  3.58s/it]
 35%|███▌      | 79564/225000 [153:16:04<140:09:57,  3.47s/it]
 35%|███▌      | 79565/225000 [153:16:07<135:11:26,  3.35s/it]
 35%|███▌      | 79566/225000 [153:16:10<132:27:15,  3.28s/it]
 35%|███▌      | 79567/225000 [153:16:13<133:16:49,  3.30s/it]
 35%|███▌      | 79568/225000 [153:16:22<195:16:09,  4.83s/it]
 35%|███▌      | 79569/225000 [153:16:27<198:21:37,  4.91s/it]
 35%|███▌      | 79570/225000 [153:16:29<171:36:45,  4.25s/it]
 35%|███▌      | 79571/225000 [153:16:32<153:55:23,  3.81s/it]
 35%|███▌      | 79572/225000 [153:16:36<149:07:53,  3.69s/it]
 35%|███▌      | 79573/225000 [153:16:38<139:36:54,  3.46s/it]
 35%|███▌      | 79574/225000 [153:16:42<136:55:37,  3.39s/it]
 35%|███▌      | 79575/225000 [153:16:50<198:11:24,  4.91s/it]
 35%|███▌      | 79576/225000 [153:16:55<200:49:27,  4.97s/it]
 35%|███▌      | 79577/225000 [153:16:58<173:03:41,  4.28s/it]
 35%|███▌      | 79578/225000 [153:17:01<155:43:42,  3.86s/it]
 35%|███▌      | 79579/225000 [153:17:04<146:05:37,  3.62s/it]
 35%|███▌      | 79580/225000 [153:17:06<134:31:47,  3.33s/it]
 35%|███▌      | 79581/225000 [153:17:10<140:23:07,  3.48s/it]
 35%|███▌      | 79582/225000 [153:17:19<204:42:55,  5.07s/it]
 35%|███▌      | 79583/225000 [153:17:22<183:18:02,  4.54s/it]
 35%|███▌      | 79584/225000 [153:17:26<170:06:36,  4.21s/it]
 35%|███▌      | 79585/225000 [153:17:29<158:18:16,  3.92s/it]
 35%|███▌      | 79586/225000 [153:17:32<148:26:49,  3.68s/it]
 35%|███▌      | 79587/225000 [153:17:35<142:37:08,  3.53s/it]
 35%|███▌      | 79588/225000 [153:17:39<138:17:02,  3.42s/it]
 35%|███▌      | 79589/225000 [153:17:41<130:37:28,  3.23s/it]
 35%|███▌      | 79590/225000 [153:17:44<124:36:25,  3.08s/it]
 35%|███▌      | 79591/225000 [153:17:53<199:26:50,  4.94s/it]
 35%|███▌      | 79592/225000 [153:17:59<205:41:24,  5.09s/it]
 35%|███▌      | 79593/225000 [153:18:01<173:15:48,  4.29s/it]
 35%|███▌      | 79594/225000 [153:18:04<153:20:42,  3.80s/it]
 35%|███▌      | 79595/225000 [153:18:06<138:31:14,  3.43s/it]
 35%|███▌      | 79596/225000 [153:18:10<134:45:00,  3.34s/it]
 35%|███▌      | 79597/225000 [153:18:13<135:44:32,  3.36s/it]
 35%|███▌      | 79598/225000 [153:18:22<206:19:09,  5.11s/it]
 35%|███▌      | 79599/225000 [153:18:25<184:01:29,  4.56s/it]
 35%|███▌      | 79600/225000 [153:18:29<167:23:44,  4.14s/it]
                                                              
{'loss': '0.8305', 'grad_norm': '3.261', 'learning_rate': '3.816e-05', 'epoch': '0.3538', 'num_input_tokens_seen': 252367856, 'train_runtime': '5.519e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79600/225000 [153:18:29<167:23:44,  4.14s/it]
 35%|███▌      | 79601/225000 [153:18:32<153:53:37,  3.81s/it]
 35%|███▌      | 79602/225000 [153:18:34<140:27:03,  3.48s/it]
 35%|███▌      | 79603/225000 [153:18:38<139:19:49,  3.45s/it]
 35%|███▌      | 79604/225000 [153:18:41<138:58:57,  3.44s/it]
 35%|███▌      | 79605/225000 [153:18:44<131:26:32,  3.25s/it]
 35%|███▌      | 79606/225000 [153:18:47<126:48:44,  3.14s/it]
 35%|███▌      | 79607/225000 [153:18:55<188:07:38,  4.66s/it]
 35%|███▌      | 79608/225000 [153:19:02<216:38:21,  5.36s/it]
 35%|███▌      | 79609/225000 [153:19:04<181:37:45,  4.50s/it]
 35%|███▌      | 79610/225000 [153:19:07<158:07:40,  3.92s/it]
 35%|███▌      | 79611/225000 [153:19:10<145:09:27,  3.59s/it]
 35%|███▌      | 79612/225000 [153:19:13<139:07:03,  3.44s/it]
 35%|███▌      | 79613/225000 [153:19:16<138:54:58,  3.44s/it]
 35%|███▌      | 79614/225000 [153:19:20<140:37:44,  3.48s/it]
 35%|███▌      | 79615/225000 [153:19:24<145:09:59,  3.59s/it]
 35%|███▌      | 79616/225000 [153:19:27<143:55:36,  3.56s/it]
 35%|███▌      | 79617/225000 [153:19:32<156:22:51,  3.87s/it]
 35%|███▌      | 79618/225000 [153:19:41<225:06:30,  5.57s/it]
 35%|███▌      | 79619/225000 [153:19:46<206:34:19,  5.12s/it]
 35%|███▌      | 79620/225000 [153:19:48<174:51:02,  4.33s/it]
 35%|███▌      | 79621/225000 [153:19:51<157:34:29,  3.90s/it]
 35%|███▌      | 79622/225000 [153:19:54<143:39:24,  3.56s/it]
 35%|███▌      | 79623/225000 [153:20:00<182:38:05,  4.52s/it]
 35%|███▌      | 79624/225000 [153:20:05<176:50:42,  4.38s/it]
 35%|███▌      | 79625/225000 [153:20:08<160:28:03,  3.97s/it]
 35%|███▌      | 79626/225000 [153:20:10<143:04:20,  3.54s/it]
 35%|███▌      | 79627/225000 [153:20:13<133:07:05,  3.30s/it]
 35%|███▌      | 79628/225000 [153:20:16<126:21:51,  3.13s/it]
 35%|███▌      | 79629/225000 [153:20:25<199:09:01,  4.93s/it]
 35%|███▌      | 79630/225000 [153:20:28<178:47:32,  4.43s/it]
 35%|███▌      | 79631/225000 [153:20:31<156:35:33,  3.88s/it]
 35%|███▌      | 79632/225000 [153:20:33<139:22:52,  3.45s/it]
 35%|███▌      | 79633/225000 [153:20:36<130:18:14,  3.23s/it]
 35%|███▌      | 79634/225000 [153:20:42<171:36:54,  4.25s/it]
 35%|███▌      | 79635/225000 [153:20:45<157:54:10,  3.91s/it]
 35%|███▌      | 79636/225000 [153:20:48<147:07:21,  3.64s/it]
 35%|███▌      | 79637/225000 [153:20:52<142:21:29,  3.53s/it]
 35%|███▌      | 79638/225000 [153:20:55<141:26:45,  3.50s/it]
 35%|███▌      | 79639/225000 [153:20:58<136:17:02,  3.38s/it]
 35%|███▌      | 79640/225000 [153:21:02<137:32:22,  3.41s/it]
 35%|███▌      | 79641/225000 [153:21:05<135:20:28,  3.35s/it]
 35%|███▌      | 79642/225000 [153:21:09<138:44:33,  3.44s/it]
 35%|███▌      | 79643/225000 [153:21:12<134:19:07,  3.33s/it]
 35%|███▌      | 79644/225000 [153:21:15<129:45:47,  3.21s/it]
 35%|███▌      | 79645/225000 [153:21:18<128:42:56,  3.19s/it]
 35%|███▌      | 79646/225000 [153:21:21<129:26:50,  3.21s/it]
 35%|███▌      | 79647/225000 [153:21:29<184:44:17,  4.58s/it]
 35%|███▌      | 79648/225000 [153:21:36<214:23:55,  5.31s/it]
 35%|███▌      | 79649/225000 [153:21:39<184:24:57,  4.57s/it]
 35%|███▌      | 79650/225000 [153:21:41<164:01:16,  4.06s/it]
                                                              
{'loss': '0.809', 'grad_norm': '5.092', 'learning_rate': '3.814e-05', 'epoch': '0.354', 'num_input_tokens_seen': 252449504, 'train_runtime': '5.521e+05', 'train_tokens_per_second': '457.3'}

 35%|███▌      | 79650/225000 [153:21:41<164:01:16,  4.06s/it]
 35%|███▌      | 79651/225000 [153:21:44<146:49:39,  3.64s/it]
 35%|███▌      | 79652/225000 [153:21:47<136:38:41,  3.38s/it]
 35%|███▌      | 79653/225000 [153:21:50<136:55:38,  3.39s/it]
 35%|███▌      | 79654/225000 [153:21:54<136:17:22,  3.38s/it]
 35%|███▌      | 79655/225000 [153:21:58<152:09:13,  3.77s/it]
 35%|███▌      | 79656/225000 [153:22:07<209:32:24,  5.19s/it]
 35%|███▌      | 79657/225000 [153:22:10<183:56:29,  4.56s/it]
 35%|███▌      | 79658/225000 [153:22:13<163:04:04,  4.04s/it]
 35%|███▌      | 79659/225000 [153:22:16<155:50:50,  3.86s/it]
 35%|███▌      | 79660/225000 [153:22:20<148:59:33,  3.69s/it]
 35%|███▌      | 79661/225000 [153:22:26<178:52:10,  4.43s/it]
 35%|███▌      | 79662/225000 [153:22:34<227:46:52,  5.64s/it]
 35%|███▌      | 79663/225000 [153:22:37<195:49:36,  4.85s/it]
 35%|███▌      | 79664/225000 [153:22:40<167:04:41,  4.14s/it]
 35%|███▌      | 79665/225000 [153:22:42<149:47:05,  3.71s/it]
 35%|███▌      | 79666/225000 [153:22:45<140:53:36,  3.49s/it]
 35%|███▌      | 79667/225000 [153:22:48<133:05:03,  3.30s/it]
 35%|███▌      | 79668/225000 [153:22:52<143:47:47,  3.56s/it]
 35%|███▌      | 79669/225000 [153:23:01<210:11:13,  5.21s/it]
 35%|███▌      | 79670/225000 [153:23:04<178:58:50,  4.43s/it]
 35%|███▌      | 79671/225000 [153:23:07<156:44:50,  3.88s/it]
 35%|███▌      | 79672/225000 [153:23:09<139:10:32,  3.45s/it]
 35%|███▌      | 79673/225000 [153:23:12<130:53:29,  3.24s/it]
 35%|███▌      | 79674/225000 [153:23:19<181:34:46,  4.50s/it]
 35%|███▌      | 79675/225000 [153:23:24<180:10:59,  4.46s/it]
 35%|███▌      | 79676/225000 [153:23:26<154:34:29,  3.83s/it]
 35%|███▌      | 79677/225000 [153:23:29<140:20:51,  3.48s/it]
 35%|███▌      | 79678/225000 [153:23:34<159:35:54,  3.95s/it]
 35%|███▌      | 79679/225000 [153:23:41<196:03:55,  4.86s/it]
 35%|███▌      | 79680/225000 [153:23:44<173:01:55,  4.29s/it]
 35%|███▌      | 79681/225000 [153:23:46<155:13:37,  3.85s/it]
 35%|███▌      | 79682/225000 [153:23:52<182:00:59,  4.51s/it]
 35%|███▌      | 79683/225000 [153:23:56<165:15:52,  4.09s/it]
 35%|███▌      | 79684/225000 [153:23:59<154:14:16,  3.82s/it]
 35%|███▌      | 79685/225000 [153:24:02<147:05:01,  3.64s/it]
 35%|███▌      | 79686/225000 [153:24:05<137:56:22,  3.42s/it]
 35%|███▌      | 79687/225000 [153:24:08<139:41:37,  3.46s/it]
 35%|███▌      | 79688/225000 [153:24:14<159:29:50,  3.95s/it]
 35%|███▌      | 79689/225000 [153:24:20<187:42:43,  4.65s/it]
 35%|███▌      | 79690/225000 [153:24:23<164:25:38,  4.07s/it]
 35%|███▌      | 79691/225000 [153:24:25<147:33:10,  3.66s/it]
 35%|███▌      | 79692/225000 [153:24:29<144:08:07,  3.57s/it]
 35%|███▌      | 79693/225000 [153:24:31<135:16:25,  3.35s/it]
 35%|███▌      | 79694/225000 [153:24:40<194:51:27,  4.83s/it]
 35%|███▌      | 79695/225000 [153:24:45<196:34:40,  4.87s/it]
 35%|███▌      | 79696/225000 [153:24:47<169:13:59,  4.19s/it]
 35%|███▌      | 79697/225000 [153:24:50<148:35:41,  3.68s/it]
 35%|███▌      | 79698/225000 [153:24:52<132:02:05,  3.27s/it]
 35%|███▌      | 79699/225000 [153:25:00<191:16:42,  4.74s/it]
 35%|███▌      | 79700/225000 [153:25:03<170:41:38,  4.23s/it]
                                                              
{'loss': '0.7646', 'grad_norm': '3.318', 'learning_rate': '3.813e-05', 'epoch': '0.3542', 'num_input_tokens_seen': 252531984, 'train_runtime': '5.523e+05', 'train_tokens_per_second': '457.2'}

 35%|███▌      | 79700/225000 [153:25:03<170:41:38,  4.23s/it]
 35%|███▌      | 79701/225000 [153:25:06<153:46:57,  3.81s/it]
 35%|███▌      | 79702/225000 [153:25:09<145:35:10,  3.61s/it]
 35%|███▌      | 79703/225000 [153:25:13<144:21:23,  3.58s/it]
 35%|███▌      | 79704/225000 [153:25:16<136:02:35,  3.37s/it]
 35%|███▌      | 79705/225000 [153:25:19<140:14:23,  3.47s/it]
 35%|███▌      | 79706/225000 [153:25:24<158:13:23,  3.92s/it]
 35%|███▌      | 79707/225000 [153:25:33<219:26:56,  5.44s/it]
 35%|███▌      | 79708/225000 [153:25:37<194:02:36,  4.81s/it]
 35%|███▌      | 79709/225000 [153:25:39<167:45:45,  4.16s/it]
 35%|███▌      | 79710/225000 [153:25:46<197:47:17,  4.90s/it]
 35%|███▌      | 79711/225000 [153:25:48<165:50:32,  4.11s/it]
 35%|███▌      | 79712/225000 [153:25:50<143:23:11,  3.55s/it]
 35%|███▌      | 79713/225000 [153:25:54<141:16:19,  3.50s/it]
 35%|███▌      | 79714/225000 [153:25:57<136:24:35,  3.38s/it]
 35%|███▌      | 79715/225000 [153:26:00<130:12:42,  3.23s/it]
 35%|███▌      | 79716/225000 [153:26:03<125:54:14,  3.12s/it]
 35%|███▌      | 79717/225000 [153:26:06<123:20:48,  3.06s/it]
 35%|███▌      | 79718/225000 [153:26:11<154:42:43,  3.83s/it]
 35%|███▌      | 79719/225000 [153:26:20<208:45:54,  5.17s/it]
 35%|███▌      | 79720/225000 [153:26:23<184:22:04,  4.57s/it]
 35%|███▌      | 79721/225000 [153:26:25<161:41:40,  4.01s/it]
 35%|███▌      | 79722/225000 [153:26:28<149:19:52,  3.70s/it]
 35%|███▌      | 79723/225000 [153:26:31<137:12:24,  3.40s/it]
 35%|███▌      | 79724/225000 [153:26:34<128:49:13,  3.19s/it]
 35%|███▌      | 79725/225000 [153:26:39<151:10:17,  3.75s/it]
 35%|███▌      | 79726/225000 [153:26:47<207:41:27,  5.15s/it]
 35%|███▌      | 79727/225000 [153:26:50<183:14:26,  4.54s/it]
 35%|███▌      | 79728/225000 [153:26:53<157:19:51,  3.90s/it]
 35%|███▌      | 79729/225000 [153:26:59<186:59:31,  4.63s/it]
 35%|███▌      | 79730/225000 [153:27:03<171:58:25,  4.26s/it]
 35%|███▌      | 79731/225000 [153:27:06<158:49:41,  3.94s/it]
 35%|███▌      | 79732/225000 [153:27:11<173:11:48,  4.29s/it]
 35%|███▌      | 79733/225000 [153:27:14<158:22:51,  3.92s/it]
 35%|███▌      | 79734/225000 [153:27:17<152:47:30,  3.79s/it]
 35%|███▌      | 79735/225000 [153:27:24<185:14:24,  4.59s/it]
 35%|███▌      | 79736/225000 [153:27:26<159:52:17,  3.96s/it]
 35%|███▌      | 79737/225000 [153:27:29<147:06:17,  3.65s/it]
 35%|███▌      | 79738/225000 [153:27:35<169:51:20,  4.21s/it]
 35%|███▌      | 79739/225000 [153:27:39<165:14:35,  4.10s/it]
 35%|███▌      | 79740/225000 [153:27:41<145:46:10,  3.61s/it]
 35%|███▌      | 79741/225000 [153:27:44<137:16:37,  3.40s/it]
 35%|███▌      | 79742/225000 [153:27:47<135:07:06,  3.35s/it]
 35%|███▌      | 79743/225000 [153:27:50<133:31:38,  3.31s/it]
 35%|███▌      | 79744/225000 [153:27:53<128:37:46,  3.19s/it]
 35%|███▌      | 79745/225000 [153:27:57<132:39:17,  3.29s/it]
 35%|███▌      | 79746/225000 [153:28:05<194:57:06,  4.83s/it]
 35%|███▌      | 79747/225000 [153:28:10<196:52:27,  4.88s/it]
 35%|███▌      | 79748/225000 [153:28:13<171:35:17,  4.25s/it]
 35%|███▌      | 79749/225000 [153:28:16<151:23:48,  3.75s/it]
 35%|███▌      | 79750/225000 [153:28:19<144:13:49,  3.57s/it]
                                                              
{'loss': '0.7796', 'grad_norm': '4.059', 'learning_rate': '3.811e-05', 'epoch': '0.3544', 'num_input_tokens_seen': 252615112, 'train_runtime': '5.525e+05', 'train_tokens_per_second': '457.2'}

 35%|███▌      | 79750/225000 [153:28:19<144:13:49,  3.57s/it]
 35%|███▌      | 79751/225000 [153:28:24<168:22:41,  4.17s/it]
 35%|███▌      | 79752/225000 [153:28:32<215:46:20,  5.35s/it]
 35%|███▌      | 79753/225000 [153:28:35<185:49:15,  4.61s/it]
 35%|███▌      | 79754/225000 [153:28:38<162:23:56,  4.03s/it]
 35%|███▌      | 79755/225000 [153:28:41<147:14:30,  3.65s/it]
 35%|███▌      | 79756/225000 [153:28:44<138:55:52,  3.44s/it]
 35%|███▌      | 79757/225000 [153:28:46<129:47:36,  3.22s/it]
 35%|███▌      | 79758/225000 [153:28:49<127:39:06,  3.16s/it]
 35%|███▌      | 79759/225000 [153:28:58<194:59:05,  4.83s/it]
 35%|███▌      | 79760/225000 [153:29:02<181:03:45,  4.49s/it]
 35%|███▌      | 79761/225000 [153:29:05<160:29:23,  3.98s/it]
 35%|███▌      | 79762/225000 [153:29:07<141:29:01,  3.51s/it]
 35%|███▌      | 79763/225000 [153:29:15<193:40:31,  4.80s/it]
 35%|███▌      | 79764/225000 [153:29:17<163:26:23,  4.05s/it]
 35%|███▌      | 79765/225000 [153:29:23<182:35:28,  4.53s/it]
 35%|███▌      | 79766/225000 [153:29:26<170:58:24,  4.24s/it]
 35%|███▌      | 79767/225000 [153:29:29<152:31:32,  3.78s/it]
 35%|███▌      | 79768/225000 [153:29:32<140:23:21,  3.48s/it]
 35%|███▌      | 79769/225000 [153:29:34<129:19:28,  3.21s/it]
 35%|███▌      | 79770/225000 [153:29:37<125:29:47,  3.11s/it]
 35%|███▌      | 79771/225000 [153:29:41<130:58:50,  3.25s/it]
 35%|███▌      | 79772/225000 [153:29:44<129:11:36,  3.20s/it]
 35%|███▌      | 79773/225000 [153:29:47<130:23:24,  3.23s/it]
 35%|███▌      | 79774/225000 [153:29:51<131:09:45,  3.25s/it]
 35%|███▌      | 79775/225000 [153:29:55<144:45:35,  3.59s/it]
 35%|███▌      | 79776/225000 [153:30:02<182:35:59,  4.53s/it]
 35%|███▌      | 79777/225000 [153:30:07<192:13:19,  4.77s/it]
 35%|███▌      | 79778/225000 [153:30:15<233:45:44,  5.79s/it]
 35%|███▌      | 79779/225000 [153:30:18<191:36:10,  4.75s/it]
 35%|███▌      | 79780/225000 [153:30:20<162:47:16,  4.04s/it]
 35%|███▌      | 79781/225000 [153:30:23<153:09:25,  3.80s/it]
 35%|███▌      | 79782/225000 [153:30:30<188:04:48,  4.66s/it]
 35%|███▌      | 79783/225000 [153:30:33<166:10:57,  4.12s/it]
 35%|███▌      | 79784/225000 [153:30:35<146:59:52,  3.64s/it]
 35%|███▌      | 79785/225000 [153:30:38<133:45:16,  3.32s/it]
 35%|███▌      | 79786/225000 [153:30:41<130:02:02,  3.22s/it]
 35%|███▌      | 79787/225000 [153:30:45<139:16:10,  3.45s/it]
 35%|███▌      | 79788/225000 [153:30:53<197:27:44,  4.90s/it]
 35%|███▌      | 79789/225000 [153:30:57<186:08:09,  4.61s/it]
 35%|███▌      | 79790/225000 [153:31:00<161:40:54,  4.01s/it]
 35%|███▌      | 79791/225000 [153:31:02<147:55:00,  3.67s/it]
 35%|███▌      | 79792/225000 [153:31:05<134:12:48,  3.33s/it]
 35%|███▌      | 79793/225000 [153:31:08<129:47:59,  3.22s/it]
 35%|███▌      | 79794/225000 [153:31:11<130:48:46,  3.24s/it]
 35%|███▌      | 79795/225000 [153:31:15<134:00:41,  3.32s/it]
 35%|███▌      | 79796/225000 [153:31:18<133:11:57,  3.30s/it]
 35%|███▌      | 79797/225000 [153:31:21<131:14:14,  3.25s/it]
 35%|███▌      | 79798/225000 [153:31:25<138:26:39,  3.43s/it]
 35%|███▌      | 79799/225000 [153:31:28<137:01:55,  3.40s/it]
 35%|███▌      | 79800/225000 [153:31:32<135:45:22,  3.37s/it]
                                                              
{'loss': '0.7884', 'grad_norm': '3.738', 'learning_rate': '3.81e-05', 'epoch': '0.3547', 'num_input_tokens_seen': 252697936, 'train_runtime': '5.527e+05', 'train_tokens_per_second': '457.2'}

 35%|███▌      | 79800/225000 [153:31:32<135:45:22,  3.37s/it]
 35%|███▌      | 79801/225000 [153:31:35<138:00:14,  3.42s/it]
 35%|███▌      | 79802/225000 [153:31:43<186:03:44,  4.61s/it]
 35%|███▌      | 79803/225000 [153:31:51<227:51:52,  5.65s/it]
 35%|███▌      | 79804/225000 [153:31:54<198:55:48,  4.93s/it]
 35%|███▌      | 79805/225000 [153:31:56<168:26:15,  4.18s/it]
 35%|███▌      | 79806/225000 [153:32:02<185:19:36,  4.60s/it]
 35%|███▌      | 79807/225000 [153:32:07<196:14:00,  4.87s/it]
 35%|███▌      | 79808/225000 [153:32:10<166:09:08,  4.12s/it]
 35%|███▌      | 79809/225000 [153:32:13<157:08:18,  3.90s/it]
 35%|███▌      | 79810/225000 [153:32:18<165:23:39,  4.10s/it]
 35%|███▌      | 79811/225000 [153:32:22<162:23:05,  4.03s/it]
 35%|███▌      | 79812/225000 [153:32:28<187:46:36,  4.66s/it]
 35%|███▌      | 79813/225000 [153:32:31<177:33:39,  4.40s/it]
 35%|███▌      | 79814/225000 [153:32:35<167:15:45,  4.15s/it]
 35%|███▌      | 79815/225000 [153:32:38<149:53:27,  3.72s/it]
 35%|███▌      | 79816/225000 [153:32:41<139:04:39,  3.45s/it]
 35%|███▌      | 79817/225000 [153:32:44<134:09:20,  3.33s/it]
 35%|███▌      | 79818/225000 [153:32:49<162:52:41,  4.04s/it]
 35%|███▌      | 79819/225000 [153:32:58<213:44:21,  5.30s/it]
 35%|███▌      | 79820/225000 [153:33:00<178:19:29,  4.42s/it]
 35%|███▌      | 79821/225000 [153:33:02<154:24:29,  3.83s/it]
 35%|███▌      | 79822/225000 [153:33:05<144:22:38,  3.58s/it]
 35%|███▌      | 79823/225000 [153:33:08<137:08:50,  3.40s/it]
 35%|███▌      | 79824/225000 [153:33:12<136:53:05,  3.39s/it]
 35%|███▌      | 79825/225000 [153:33:16<147:25:30,  3.66s/it]
 35%|███▌      | 79826/225000 [153:33:19<138:56:31,  3.45s/it]
 35%|███▌      | 79827/225000 [153:33:22<138:41:40,  3.44s/it]
 35%|███▌      | 79828/225000 [153:33:26<141:14:27,  3.50s/it]
 35%|███▌      | 79829/225000 [153:33:32<174:13:50,  4.32s/it]
 35%|███▌      | 79830/225000 [153:33:41<224:07:10,  5.56s/it]
 35%|███▌      | 79831/225000 [153:33:44<197:14:52,  4.89s/it]
 35%|███▌      | 79832/225000 [153:33:47<173:35:47,  4.30s/it]
 35%|███▌      | 79833/225000 [153:33:50<152:48:26,  3.79s/it]
 35%|███▌      | 79834/225000 [153:33:52<136:55:51,  3.40s/it]
 35%|███▌      | 79835/225000 [153:33:57<155:07:22,  3.85s/it]
 35%|███▌      | 79836/225000 [153:34:03<186:00:45,  4.61s/it]
 35%|███▌      | 79837/225000 [153:34:06<159:23:24,  3.95s/it]
 35%|███▌      | 79838/225000 [153:34:08<140:13:13,  3.48s/it]
 35%|███▌      | 79839/225000 [153:34:14<165:57:13,  4.12s/it]
 35%|███▌      | 79840/225000 [153:34:22<210:45:38,  5.23s/it]
 35%|███▌      | 79841/225000 [153:34:24<176:15:27,  4.37s/it]
 35%|███▌      | 79842/225000 [153:34:27<158:46:17,  3.94s/it]
 35%|███▌      | 79843/225000 [153:34:32<175:07:10,  4.34s/it]
 35%|███▌      | 79844/225000 [153:34:36<175:15:48,  4.35s/it]
 35%|███▌      | 79845/225000 [153:34:39<152:58:18,  3.79s/it]
 35%|███▌      | 79846/225000 [153:34:43<156:10:17,  3.87s/it]
 35%|███▌      | 79847/225000 [153:34:46<148:07:41,  3.67s/it]
 35%|███▌      | 79848/225000 [153:34:50<147:45:18,  3.66s/it]
 35%|███▌      | 79849/225000 [153:34:53<141:12:15,  3.50s/it]
 35%|███▌      | 79850/225000 [153:34:56<134:49:55,  3.34s/it]
                                                              
{'loss': '0.8084', 'grad_norm': '4.863', 'learning_rate': '3.808e-05', 'epoch': '0.3549', 'num_input_tokens_seen': 252781024, 'train_runtime': '5.529e+05', 'train_tokens_per_second': '457.2'}

 35%|███▌      | 79850/225000 [153:34:56<134:49:55,  3.34s/it]
 35%|███▌      | 79851/225000 [153:34:59<136:19:52,  3.38s/it]
 35%|███▌      | 79852/225000 [153:35:03<135:46:56,  3.37s/it]
 35%|███▌      | 79853/225000 [153:35:12<203:20:08,  5.04s/it]
 35%|███▌      | 79854/225000 [153:35:16<199:44:49,  4.95s/it]
 35%|███▌      | 79855/225000 [153:35:19<173:12:13,  4.30s/it]
 35%|███▌      | 79856/225000 [153:35:23<170:46:23,  4.24s/it]
 35%|███▌      | 79857/225000 [153:35:29<183:03:47,  4.54s/it]
 35%|███▌      | 79858/225000 [153:35:31<159:39:02,  3.96s/it]
 35%|███▌      | 79859/225000 [153:35:34<142:06:25,  3.52s/it]
 35%|███▌      | 79860/225000 [153:35:37<134:46:50,  3.34s/it]
 35%|███▌      | 79861/225000 [153:35:44<183:44:19,  4.56s/it]
 35%|███▌      | 79862/225000 [153:35:47<169:35:40,  4.21s/it]
 35%|███▌      | 79863/225000 [153:35:50<149:06:20,  3.70s/it]
 35%|███▌      | 79864/225000 [153:35:53<147:37:51,  3.66s/it]
 35%|███▌      | 79865/225000 [153:35:59<175:26:40,  4.35s/it]
 35%|███▌      | 79866/225000 [153:36:02<154:12:15,  3.82s/it]
 35%|███▌      | 79867/225000 [153:36:05<141:54:46,  3.52s/it]
 35%|███▌      | 79868/225000 [153:36:08<131:24:23,  3.26s/it]
 35%|███▌      | 79869/225000 [153:36:11<130:58:07,  3.25s/it]
 35%|███▌      | 79870/225000 [153:36:14<126:35:49,  3.14s/it]
 35%|███▌      | 79871/225000 [153:36:22<186:39:52,  4.63s/it]
 35%|███▌      | 79872/225000 [153:36:26<178:24:34,  4.43s/it]
 35%|███▌      | 79873/225000 [153:36:28<158:54:33,  3.94s/it]
 35%|███▌      | 79874/225000 [153:36:32<149:41:11,  3.71s/it]
 36%|███▌      | 79875/225000 [153:36:35<147:26:28,  3.66s/it]
 36%|███▌      | 79876/225000 [153:36:39<149:53:05,  3.72s/it]
 36%|███▌      | 79877/225000 [153:36:47<195:12:59,  4.84s/it]
 36%|███▌      | 79878/225000 [153:36:49<167:08:25,  4.15s/it]
 36%|███▌      | 79879/225000 [153:36:51<146:36:03,  3.64s/it]
 36%|███▌      | 79880/225000 [153:36:54<137:24:35,  3.41s/it]
 36%|███▌      | 79881/225000 [153:36:57<128:36:48,  3.19s/it]
 36%|███▌      | 79882/225000 [153:37:00<122:52:40,  3.05s/it]
 36%|███▌      | 79883/225000 [153:37:06<161:39:45,  4.01s/it]
 36%|███▌      | 79884/225000 [153:37:12<181:21:27,  4.50s/it]
 36%|███▌      | 79885/225000 [153:37:14<160:09:20,  3.97s/it]
 36%|███▌      | 79886/225000 [153:37:18<151:31:31,  3.76s/it]
 36%|███▌      | 79887/225000 [153:37:21<140:24:51,  3.48s/it]
 36%|███▌      | 79888/225000 [153:37:24<136:11:40,  3.38s/it]
 36%|███▌      | 79889/225000 [153:37:27<130:14:49,  3.23s/it]
 36%|███▌      | 79890/225000 [153:37:29<127:02:31,  3.15s/it]
 36%|███▌      | 79891/225000 [153:37:38<186:21:35,  4.62s/it]
 36%|███▌      | 79892/225000 [153:37:42<187:50:48,  4.66s/it]
 36%|███▌      | 79893/225000 [153:37:45<160:49:01,  3.99s/it]
 36%|███▌      | 79894/225000 [153:37:47<144:23:51,  3.58s/it]
 36%|███▌      | 79895/225000 [153:37:50<129:49:43,  3.22s/it]
 36%|███▌      | 79896/225000 [153:37:58<186:24:03,  4.62s/it]
 36%|███▌      | 79897/225000 [153:38:01<174:56:06,  4.34s/it]
 36%|███▌      | 79898/225000 [153:38:04<156:51:51,  3.89s/it]
 36%|███▌      | 79899/225000 [153:38:07<145:02:32,  3.60s/it]
 36%|███▌      | 79900/225000 [153:38:10<140:35:48,  3.49s/it]
                                                              
{'loss': '0.7862', 'grad_norm': '3.585', 'learning_rate': '3.807e-05', 'epoch': '0.3551', 'num_input_tokens_seen': 252862184, 'train_runtime': '5.531e+05', 'train_tokens_per_second': '457.2'}

 36%|███▌      | 79900/225000 [153:38:10<140:35:48,  3.49s/it]
 36%|███▌      | 79901/225000 [153:38:13<136:07:43,  3.38s/it]
 36%|███▌      | 79902/225000 [153:38:16<132:29:25,  3.29s/it]
 36%|███▌      | 79903/225000 [153:38:20<131:11:01,  3.25s/it]
 36%|███▌      | 79904/225000 [153:38:23<130:49:26,  3.25s/it]
 36%|███▌      | 79905/225000 [153:38:26<132:01:09,  3.28s/it]
 36%|███▌      | 79906/225000 [153:38:36<210:16:02,  5.22s/it]
 36%|███▌      | 79907/225000 [153:38:41<203:22:13,  5.05s/it]
 36%|███▌      | 79908/225000 [153:38:44<187:00:32,  4.64s/it]
 36%|███▌      | 79909/225000 [153:38:50<198:58:14,  4.94s/it]
 36%|███▌      | 79910/225000 [153:38:52<165:50:04,  4.11s/it]
 36%|███▌      | 79911/225000 [153:38:57<174:10:24,  4.32s/it]
 36%|███▌      | 79912/225000 [153:39:00<153:00:59,  3.80s/it]
 36%|███▌      | 79913/225000 [153:39:02<137:51:29,  3.42s/it]
 36%|███▌      | 79914/225000 [153:39:05<136:47:46,  3.39s/it]
 36%|███▌      | 79915/225000 [153:39:08<131:25:03,  3.26s/it]
 36%|███▌      | 79916/225000 [153:39:11<124:02:26,  3.08s/it]
 36%|███▌      | 79917/225000 [153:39:14<122:00:11,  3.03s/it]
 36%|███▌      | 79918/225000 [153:39:17<128:01:50,  3.18s/it]
 36%|███▌      | 79919/225000 [153:39:23<160:15:19,  3.98s/it]
 36%|███▌      | 79920/225000 [153:39:30<197:58:29,  4.91s/it]
 36%|███▌      | 79921/225000 [153:39:33<172:55:59,  4.29s/it]
 36%|███▌      | 79922/225000 [153:39:36<158:38:08,  3.94s/it]
 36%|███▌      | 79923/225000 [153:39:40<150:15:44,  3.73s/it]
 36%|███▌      | 79924/225000 [153:39:43<142:43:26,  3.54s/it]
 36%|███▌      | 79925/225000 [153:39:46<141:06:04,  3.50s/it]
 36%|███▌      | 79926/225000 [153:39:49<134:53:27,  3.35s/it]
 36%|███▌      | 79927/225000 [153:39:53<136:57:44,  3.40s/it]
 36%|███▌      | 79928/225000 [153:40:01<194:17:52,  4.82s/it]
 36%|███▌      | 79929/225000 [153:40:05<190:56:27,  4.74s/it]
 36%|███▌      | 79930/225000 [153:40:08<169:39:40,  4.21s/it]
 36%|███▌      | 79931/225000 [153:40:11<156:10:49,  3.88s/it]
 36%|███▌      | 79932/225000 [153:40:14<141:35:28,  3.51s/it]
 36%|███▌      | 79933/225000 [153:40:17<135:31:39,  3.36s/it]
 36%|███▌      | 79934/225000 [153:40:20<131:37:31,  3.27s/it]
 36%|███▌      | 79935/225000 [153:40:23<128:46:21,  3.20s/it]
 36%|███▌      | 79936/225000 [153:40:29<164:23:26,  4.08s/it]
 36%|███▌      | 79937/225000 [153:40:38<225:48:56,  5.60s/it]
 36%|███▌      | 79938/225000 [153:40:41<194:22:05,  4.82s/it]
 36%|███▌      | 79939/225000 [153:40:44<165:00:59,  4.10s/it]
 36%|███▌      | 79940/225000 [153:40:46<144:55:52,  3.60s/it]
 36%|███▌      | 79941/225000 [153:40:49<140:39:49,  3.49s/it]
 36%|███▌      | 79942/225000 [153:40:57<195:20:14,  4.85s/it]
 36%|███▌      | 79943/225000 [153:41:01<175:09:15,  4.35s/it]
 36%|███▌      | 79944/225000 [153:41:03<151:38:14,  3.76s/it]
 36%|███▌      | 79945/225000 [153:41:09<174:46:11,  4.34s/it]
 36%|███▌      | 79946/225000 [153:41:14<182:31:03,  4.53s/it]
 36%|███▌      | 79947/225000 [153:41:17<163:59:44,  4.07s/it]
 36%|███▌      | 79948/225000 [153:41:19<144:48:57,  3.59s/it]
 36%|███▌      | 79949/225000 [153:41:22<133:05:32,  3.30s/it]
 36%|███▌      | 79950/225000 [153:41:25<125:24:15,  3.11s/it]
                                                              
{'loss': '0.7885', 'grad_norm': '4.642', 'learning_rate': '3.805e-05', 'epoch': '0.3553', 'num_input_tokens_seen': 252943000, 'train_runtime': '5.533e+05', 'train_tokens_per_second': '457.2'}

 36%|███▌      | 79950/225000 [153:41:25<125:24:15,  3.11s/it]
 36%|███▌      | 79951/225000 [153:41:28<128:52:23,  3.20s/it]
 36%|███▌      | 79952/225000 [153:41:37<197:37:30,  4.90s/it]
 36%|███▌      | 79953/225000 [153:41:41<184:06:13,  4.57s/it]
 36%|███▌      | 79954/225000 [153:41:43<161:30:20,  4.01s/it]
 36%|███▌      | 79955/225000 [153:41:46<144:53:01,  3.60s/it]
 36%|███▌      | 79956/225000 [153:41:49<137:04:06,  3.40s/it]
 36%|███▌      | 79957/225000 [153:41:52<130:32:49,  3.24s/it]
 36%|███▌      | 79958/225000 [153:41:54<124:27:50,  3.09s/it]
 36%|███▌      | 79959/225000 [153:41:57<123:09:23,  3.06s/it]
 36%|███▌      | 79960/225000 [153:42:05<177:39:53,  4.41s/it]
 36%|███▌      | 79961/225000 [153:42:10<184:35:32,  4.58s/it]
 36%|███▌      | 79962/225000 [153:42:13<162:39:05,  4.04s/it]
 36%|███▌      | 79963/225000 [153:42:16<152:26:53,  3.78s/it]
 36%|███▌      | 79964/225000 [153:42:19<141:41:58,  3.52s/it]
 36%|███▌      | 79965/225000 [153:42:22<136:07:02,  3.38s/it]
 36%|███▌      | 79966/225000 [153:42:25<126:46:16,  3.15s/it]
 36%|███▌      | 79967/225000 [153:42:29<144:37:27,  3.59s/it]
 36%|███▌      | 79968/225000 [153:42:37<201:39:14,  5.01s/it]
 36%|███▌      | 79969/225000 [153:42:40<177:32:01,  4.41s/it]
 36%|███▌      | 79970/225000 [153:42:43<156:17:28,  3.88s/it]
 36%|███▌      | 79971/225000 [153:42:46<142:57:38,  3.55s/it]
 36%|███▌      | 79972/225000 [153:42:49<141:50:17,  3.52s/it]
 36%|███▌      | 79973/225000 [153:42:52<136:13:20,  3.38s/it]
 36%|███▌      | 79974/225000 [153:42:59<175:30:25,  4.36s/it]
 36%|███▌      | 79975/225000 [153:43:05<189:38:20,  4.71s/it]
 36%|███▌      | 79976/225000 [153:43:07<167:31:16,  4.16s/it]
 36%|███▌      | 79977/225000 [153:43:10<153:03:00,  3.80s/it]
 36%|███▌      | 79978/225000 [153:43:17<184:09:54,  4.57s/it]
 36%|███▌      | 79979/225000 [153:43:21<184:19:36,  4.58s/it]
 36%|███▌      | 79980/225000 [153:43:25<171:58:55,  4.27s/it]
 36%|███▌      | 79981/225000 [153:43:30<185:16:11,  4.60s/it]
 36%|███▌      | 79982/225000 [153:43:33<163:37:04,  4.06s/it]
 36%|███▌      | 79983/225000 [153:43:36<145:20:00,  3.61s/it]
 36%|███▌      | 79984/225000 [153:43:38<131:58:17,  3.28s/it]
 36%|███▌      | 79985/225000 [153:43:41<126:33:07,  3.14s/it]
 36%|███▌      | 79986/225000 [153:43:45<137:28:38,  3.41s/it]
 36%|███▌      | 79987/225000 [153:43:53<197:09:37,  4.89s/it]
 36%|███▌      | 79988/225000 [153:43:57<187:10:52,  4.65s/it]
 36%|███▌      | 79989/225000 [153:44:00<165:37:15,  4.11s/it]
 36%|███▌      | 79990/225000 [153:44:03<153:50:08,  3.82s/it]
 36%|███▌      | 79991/225000 [153:44:06<142:01:36,  3.53s/it]
 36%|███▌      | 79992/225000 [153:44:09<134:45:29,  3.35s/it]
 36%|███▌      | 79993/225000 [153:44:12<131:18:38,  3.26s/it]
 36%|███▌      | 79994/225000 [153:44:15<128:36:24,  3.19s/it]
 36%|███▌      | 79995/225000 [153:44:19<129:19:59,  3.21s/it]
 36%|███▌      | 79996/225000 [153:44:22<127:51:02,  3.17s/it]
 36%|███▌      | 79997/225000 [153:44:25<127:22:32,  3.16s/it]
 36%|███▌      | 79998/225000 [153:44:28<133:12:09,  3.31s/it]
 36%|███▌      | 79999/225000 [153:44:36<187:30:52,  4.66s/it]
 36%|███▌      | 80000/225000 [153:44:45<242:59:14,  6.03s/it]
                                                              
{'loss': '0.7815', 'grad_norm': '3.015', 'learning_rate': '3.804e-05', 'epoch': '0.3556', 'num_input_tokens_seen': 253024768, 'train_runtime': '5.535e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80000/225000 [153:44:45<242:59:14,  6.03s/it]
 36%|███▌      | 80001/225000 [153:44:48<206:54:44,  5.14s/it]
 36%|███▌      | 80002/225000 [153:44:52<185:50:55,  4.61s/it]
 36%|███▌      | 80003/225000 [153:44:55<172:30:16,  4.28s/it]
 36%|███▌      | 80004/225000 [153:44:58<157:32:24,  3.91s/it]
 36%|███▌      | 80005/225000 [153:45:02<148:41:17,  3.69s/it]
 36%|███▌      | 80006/225000 [153:45:05<143:41:21,  3.57s/it]
 36%|███▌      | 80007/225000 [153:45:08<139:59:27,  3.48s/it]
 36%|███▌      | 80008/225000 [153:45:11<138:14:16,  3.43s/it]
 36%|███▌      | 80009/225000 [153:45:16<155:00:58,  3.85s/it]
 36%|███▌      | 80010/225000 [153:45:25<210:20:51,  5.22s/it]
 36%|███▌      | 80011/225000 [153:45:27<174:38:18,  4.34s/it]
 36%|███▌      | 80012/225000 [153:45:30<159:55:34,  3.97s/it]
 36%|███▌      | 80013/225000 [153:45:34<155:52:57,  3.87s/it]
 36%|███▌      | 80014/225000 [153:45:37<149:27:09,  3.71s/it]
 36%|███▌      | 80015/225000 [153:45:42<162:35:50,  4.04s/it]
 36%|███▌      | 80016/225000 [153:45:46<163:13:52,  4.05s/it]
 36%|███▌      | 80017/225000 [153:45:49<152:05:23,  3.78s/it]
 36%|███▌      | 80018/225000 [153:45:52<146:54:26,  3.65s/it]
 36%|███▌      | 80019/225000 [153:45:56<139:27:01,  3.46s/it]
 36%|███▌      | 80020/225000 [153:45:59<142:00:46,  3.53s/it]
 36%|███▌      | 80021/225000 [153:46:02<136:58:53,  3.40s/it]
 36%|███▌      | 80022/225000 [153:46:05<133:57:35,  3.33s/it]
 36%|███▌      | 80023/225000 [153:46:09<135:54:20,  3.37s/it]
 36%|███▌      | 80024/225000 [153:46:12<133:59:51,  3.33s/it]
 36%|███▌      | 80025/225000 [153:46:16<137:18:38,  3.41s/it]
 36%|███▌      | 80026/225000 [153:46:19<135:57:18,  3.38s/it]
 36%|███▌      | 80027/225000 [153:46:23<139:18:10,  3.46s/it]
 36%|███▌      | 80028/225000 [153:46:31<203:21:48,  5.05s/it]
 36%|███▌      | 80029/225000 [153:46:40<250:31:27,  6.22s/it]
 36%|███▌      | 80030/225000 [153:46:43<206:33:47,  5.13s/it]
 36%|███▌      | 80031/225000 [153:46:45<172:48:45,  4.29s/it]
 36%|███▌      | 80032/225000 [153:46:48<151:11:43,  3.75s/it]
 36%|███▌      | 80033/225000 [153:46:56<200:01:53,  4.97s/it]
 36%|███▌      | 80034/225000 [153:47:00<194:08:59,  4.82s/it]
 36%|███▌      | 80035/225000 [153:47:02<163:07:57,  4.05s/it]
 36%|███▌      | 80036/225000 [153:47:05<142:34:23,  3.54s/it]
 36%|███▌      | 80037/225000 [153:47:12<182:08:42,  4.52s/it]
 36%|███▌      | 80038/225000 [153:47:14<154:26:56,  3.84s/it]
 36%|███▌      | 80039/225000 [153:47:18<157:53:10,  3.92s/it]
 36%|███▌      | 80040/225000 [153:47:23<169:22:16,  4.21s/it]
 36%|███▌      | 80041/225000 [153:47:26<158:35:34,  3.94s/it]
 36%|███▌      | 80042/225000 [153:47:29<149:29:49,  3.71s/it]
 36%|███▌      | 80043/225000 [153:47:32<140:03:50,  3.48s/it]
 36%|███▌      | 80044/225000 [153:47:35<133:18:56,  3.31s/it]
 36%|███▌      | 80045/225000 [153:47:38<128:59:54,  3.20s/it]
 36%|███▌      | 80046/225000 [153:47:41<124:05:18,  3.08s/it]
 36%|███▌      | 80047/225000 [153:47:44<120:37:19,  3.00s/it]
 36%|███▌      | 80048/225000 [153:47:47<121:36:15,  3.02s/it]
 36%|███▌      | 80049/225000 [153:47:50<120:17:52,  2.99s/it]
 36%|███▌      | 80050/225000 [153:47:53<129:21:25,  3.21s/it]
                                                              
{'loss': '0.7918', 'grad_norm': '4.162', 'learning_rate': '3.802e-05', 'epoch': '0.3558', 'num_input_tokens_seen': 253105960, 'train_runtime': '5.537e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80050/225000 [153:47:53<129:21:25,  3.21s/it]
 36%|███▌      | 80051/225000 [153:48:03<201:47:52,  5.01s/it]
 36%|███▌      | 80052/225000 [153:48:09<215:05:57,  5.34s/it]
 36%|███▌      | 80053/225000 [153:48:12<185:28:57,  4.61s/it]
 36%|███▌      | 80054/225000 [153:48:14<163:58:45,  4.07s/it]
 36%|███▌      | 80055/225000 [153:48:17<148:37:04,  3.69s/it]
 36%|███▌      | 80056/225000 [153:48:20<135:19:38,  3.36s/it]
 36%|███▌      | 80057/225000 [153:48:26<165:04:22,  4.10s/it]
 36%|███▌      | 80058/225000 [153:48:31<183:40:17,  4.56s/it]
 36%|███▌      | 80059/225000 [153:48:33<155:01:59,  3.85s/it]
 36%|███▌      | 80060/225000 [153:48:36<135:22:06,  3.36s/it]
 36%|███▌      | 80061/225000 [153:48:39<136:55:07,  3.40s/it]
 36%|███▌      | 80062/225000 [153:48:46<182:33:45,  4.53s/it]
 36%|███▌      | 80063/225000 [153:48:49<160:19:33,  3.98s/it]
 36%|███▌      | 80064/225000 [153:48:52<142:45:02,  3.55s/it]
 36%|███▌      | 80065/225000 [153:48:55<138:47:52,  3.45s/it]
 36%|███▌      | 80066/225000 [153:49:03<195:36:01,  4.86s/it]
 36%|███▌      | 80067/225000 [153:49:05<167:09:10,  4.15s/it]
 36%|███▌      | 80068/225000 [153:49:08<147:41:47,  3.67s/it]
 36%|███▌      | 80069/225000 [153:49:11<141:48:26,  3.52s/it]
 36%|███▌      | 80070/225000 [153:49:14<130:59:08,  3.25s/it]
 36%|███▌      | 80071/225000 [153:49:21<183:49:56,  4.57s/it]
 36%|███▌      | 80072/225000 [153:49:26<181:36:22,  4.51s/it]
 36%|███▌      | 80073/225000 [153:49:29<162:35:01,  4.04s/it]
 36%|███▌      | 80074/225000 [153:49:32<153:26:12,  3.81s/it]
 36%|███▌      | 80075/225000 [153:49:36<150:54:49,  3.75s/it]
 36%|███▌      | 80076/225000 [153:49:39<145:19:07,  3.61s/it]
 36%|███▌      | 80077/225000 [153:49:42<137:52:00,  3.42s/it]
 36%|███▌      | 80078/225000 [153:49:45<136:32:39,  3.39s/it]
 36%|███▌      | 80079/225000 [153:49:49<137:58:35,  3.43s/it]
 36%|███▌      | 80080/225000 [153:49:52<132:48:13,  3.30s/it]
 36%|███▌      | 80081/225000 [153:49:55<135:16:25,  3.36s/it]
 36%|███▌      | 80082/225000 [153:50:04<196:45:22,  4.89s/it]
 36%|███▌      | 80083/225000 [153:50:09<205:27:09,  5.10s/it]
 36%|███▌      | 80084/225000 [153:50:12<175:44:17,  4.37s/it]
 36%|███▌      | 80085/225000 [153:50:15<160:24:19,  3.98s/it]
 36%|███▌      | 80086/225000 [153:50:18<148:49:20,  3.70s/it]
 36%|███▌      | 80087/225000 [153:50:21<135:32:20,  3.37s/it]
 36%|███▌      | 80088/225000 [153:50:25<141:58:04,  3.53s/it]
 36%|███▌      | 80089/225000 [153:50:33<201:20:19,  5.00s/it]
 36%|███▌      | 80090/225000 [153:50:36<181:55:57,  4.52s/it]
 36%|███▌      | 80091/225000 [153:50:39<161:08:57,  4.00s/it]
 36%|███▌      | 80092/225000 [153:50:42<148:17:30,  3.68s/it]
 36%|███▌      | 80093/225000 [153:50:45<137:36:05,  3.42s/it]
 36%|███▌      | 80094/225000 [153:50:48<130:23:44,  3.24s/it]
 36%|███▌      | 80095/225000 [153:50:57<197:49:35,  4.91s/it]
 36%|███▌      | 80096/225000 [153:51:00<176:08:36,  4.38s/it]
 36%|███▌      | 80097/225000 [153:51:02<152:59:05,  3.80s/it]
 36%|███▌      | 80098/225000 [153:51:08<172:31:30,  4.29s/it]
 36%|███▌      | 80099/225000 [153:51:13<183:15:43,  4.55s/it]
 36%|███▌      | 80100/225000 [153:51:15<158:18:25,  3.93s/it]
                                                              
{'loss': '0.7889', 'grad_norm': '3.813', 'learning_rate': '3.8e-05', 'epoch': '0.356', 'num_input_tokens_seen': 253188112, 'train_runtime': '5.539e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80100/225000 [153:51:15<158:18:25,  3.93s/it]
 36%|███▌      | 80101/225000 [153:51:19<154:57:01,  3.85s/it]
 36%|███▌      | 80102/225000 [153:51:25<184:47:24,  4.59s/it]
 36%|███▌      | 80103/225000 [153:51:27<156:28:29,  3.89s/it]
 36%|███▌      | 80104/225000 [153:51:31<153:56:12,  3.82s/it]
 36%|███▌      | 80105/225000 [153:51:37<184:16:42,  4.58s/it]
 36%|███▌      | 80106/225000 [153:51:40<159:59:49,  3.98s/it]
 36%|███▌      | 80107/225000 [153:51:43<144:59:59,  3.60s/it]
 36%|███▌      | 80108/225000 [153:51:46<136:08:24,  3.38s/it]
 36%|███▌      | 80109/225000 [153:51:49<131:02:55,  3.26s/it]
 36%|███▌      | 80110/225000 [153:51:52<127:44:24,  3.17s/it]
 36%|███▌      | 80111/225000 [153:51:55<130:00:19,  3.23s/it]
 36%|███▌      | 80112/225000 [153:51:58<125:38:49,  3.12s/it]
 36%|███▌      | 80113/225000 [153:52:01<124:16:36,  3.09s/it]
 36%|███▌      | 80114/225000 [153:52:04<123:17:36,  3.06s/it]
 36%|███▌      | 80115/225000 [153:52:11<169:44:44,  4.22s/it]
 36%|███▌      | 80116/225000 [153:52:19<215:36:08,  5.36s/it]
 36%|███▌      | 80117/225000 [153:52:21<177:46:00,  4.42s/it]
 36%|███▌      | 80118/225000 [153:52:24<155:32:42,  3.86s/it]
 36%|███▌      | 80119/225000 [153:52:31<201:39:52,  5.01s/it]
 36%|███▌      | 80120/225000 [153:52:34<170:51:02,  4.25s/it]
 36%|███▌      | 80121/225000 [153:52:36<152:45:19,  3.80s/it]
 36%|███▌      | 80122/225000 [153:52:39<140:40:57,  3.50s/it]
 36%|███▌      | 80123/225000 [153:52:42<129:53:30,  3.23s/it]
 36%|███▌      | 80124/225000 [153:52:49<171:55:42,  4.27s/it]
 36%|███▌      | 80125/225000 [153:52:54<185:06:14,  4.60s/it]
 36%|███▌      | 80126/225000 [153:52:57<168:16:57,  4.18s/it]
 36%|███▌      | 80127/225000 [153:53:01<163:33:19,  4.06s/it]
 36%|███▌      | 80128/225000 [153:53:04<155:44:41,  3.87s/it]
 36%|███▌      | 80129/225000 [153:53:08<148:27:56,  3.69s/it]
 36%|███▌      | 80130/225000 [153:53:12<150:57:39,  3.75s/it]
 36%|███▌      | 80131/225000 [153:53:15<146:34:14,  3.64s/it]
 36%|███▌      | 80132/225000 [153:53:21<173:37:44,  4.31s/it]
 36%|███▌      | 80133/225000 [153:53:24<159:35:46,  3.97s/it]
 36%|███▌      | 80134/225000 [153:53:27<150:16:14,  3.73s/it]
 36%|███▌      | 80135/225000 [153:53:32<169:48:27,  4.22s/it]
 36%|███▌      | 80136/225000 [153:53:36<158:29:22,  3.94s/it]
 36%|███▌      | 80137/225000 [153:53:39<150:17:22,  3.73s/it]
 36%|███▌      | 80138/225000 [153:53:42<144:02:02,  3.58s/it]
 36%|███▌      | 80139/225000 [153:53:46<141:51:13,  3.53s/it]
 36%|███▌      | 80140/225000 [153:53:49<140:03:57,  3.48s/it]
 36%|███▌      | 80141/225000 [153:53:52<139:45:14,  3.47s/it]
 36%|███▌      | 80142/225000 [153:53:56<143:32:28,  3.57s/it]
 36%|███▌      | 80143/225000 [153:54:00<140:00:13,  3.48s/it]
 36%|███▌      | 80144/225000 [153:54:03<145:10:33,  3.61s/it]
 36%|███▌      | 80145/225000 [153:54:07<147:49:12,  3.67s/it]
 36%|███▌      | 80146/225000 [153:54:11<150:02:09,  3.73s/it]
 36%|███▌      | 80147/225000 [153:54:19<203:40:38,  5.06s/it]
 36%|███▌      | 80148/225000 [153:54:26<226:24:16,  5.63s/it]
 36%|███▌      | 80149/225000 [153:54:29<189:51:39,  4.72s/it]
 36%|███▌      | 80150/225000 [153:54:31<161:47:07,  4.02s/it]
                                                              
{'loss': '0.7791', 'grad_norm': '4.562', 'learning_rate': '3.799e-05', 'epoch': '0.3562', 'num_input_tokens_seen': 253269112, 'train_runtime': '5.541e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80150/225000 [153:54:31<161:47:07,  4.02s/it]
 36%|███▌      | 80151/225000 [153:54:34<142:24:15,  3.54s/it]
 36%|███▌      | 80152/225000 [153:54:37<142:45:26,  3.55s/it]
 36%|███▌      | 80153/225000 [153:54:40<132:40:04,  3.30s/it]
 36%|███▌      | 80154/225000 [153:54:43<130:51:43,  3.25s/it]
 36%|███▌      | 80155/225000 [153:54:48<146:02:27,  3.63s/it]
 36%|███▌      | 80156/225000 [153:54:56<207:36:28,  5.16s/it]
 36%|███▌      | 80157/225000 [153:55:01<205:17:58,  5.10s/it]
 36%|███▌      | 80158/225000 [153:55:05<184:19:42,  4.58s/it]
 36%|███▌      | 80159/225000 [153:55:08<166:28:48,  4.14s/it]
 36%|███▌      | 80160/225000 [153:55:11<155:15:39,  3.86s/it]
 36%|███▌      | 80161/225000 [153:55:14<149:23:08,  3.71s/it]
 36%|███▌      | 80162/225000 [153:55:17<142:47:27,  3.55s/it]
 36%|███▌      | 80163/225000 [153:55:21<138:56:44,  3.45s/it]
 36%|███▌      | 80164/225000 [153:55:24<139:48:19,  3.47s/it]
 36%|███▌      | 80165/225000 [153:55:28<139:38:15,  3.47s/it]
 36%|███▌      | 80166/225000 [153:55:31<135:40:05,  3.37s/it]
 36%|███▌      | 80167/225000 [153:55:34<134:11:22,  3.34s/it]
 36%|███▌      | 80168/225000 [153:55:38<136:42:27,  3.40s/it]
 36%|███▌      | 80169/225000 [153:55:41<131:39:47,  3.27s/it]
 36%|███▌      | 80170/225000 [153:55:44<131:01:18,  3.26s/it]
 36%|███▌      | 80171/225000 [153:55:49<157:20:30,  3.91s/it]
 36%|███▌      | 80172/225000 [153:55:53<151:56:32,  3.78s/it]
 36%|███▌      | 80173/225000 [153:55:56<147:21:09,  3.66s/it]
 36%|███▌      | 80174/225000 [153:55:59<141:27:31,  3.52s/it]
 36%|███▌      | 80175/225000 [153:56:02<135:09:48,  3.36s/it]
 36%|███▌      | 80176/225000 [153:56:06<138:12:55,  3.44s/it]
 36%|███▌      | 80177/225000 [153:56:09<134:26:48,  3.34s/it]
 36%|███▌      | 80178/225000 [153:56:14<152:46:34,  3.80s/it]
 36%|███▌      | 80179/225000 [153:56:23<215:16:40,  5.35s/it]
 36%|███▌      | 80180/225000 [153:56:26<186:00:59,  4.62s/it]
 36%|███▌      | 80181/225000 [153:56:29<166:43:01,  4.14s/it]
 36%|███▌      | 80182/225000 [153:56:32<154:16:00,  3.83s/it]
 36%|███▌      | 80183/225000 [153:56:35<147:03:16,  3.66s/it]
 36%|███▌      | 80184/225000 [153:56:38<140:51:30,  3.50s/it]
 36%|███▌      | 80185/225000 [153:56:41<135:15:17,  3.36s/it]
 36%|███▌      | 80186/225000 [153:56:45<139:05:31,  3.46s/it]
 36%|███▌      | 80187/225000 [153:56:55<214:24:48,  5.33s/it]
 36%|███▌      | 80188/225000 [153:56:58<190:42:20,  4.74s/it]
 36%|███▌      | 80189/225000 [153:57:01<168:50:22,  4.20s/it]
 36%|███▌      | 80190/225000 [153:57:04<154:06:58,  3.83s/it]
 36%|███▌      | 80191/225000 [153:57:07<143:24:03,  3.56s/it]
 36%|███▌      | 80192/225000 [153:57:10<136:18:29,  3.39s/it]
 36%|███▌      | 80193/225000 [153:57:13<133:42:44,  3.32s/it]
 36%|███▌      | 80194/225000 [153:57:16<134:03:50,  3.33s/it]
 36%|███▌      | 80195/225000 [153:57:20<133:25:24,  3.32s/it]
 36%|███▌      | 80196/225000 [153:57:23<132:52:23,  3.30s/it]
 36%|███▌      | 80197/225000 [153:57:27<140:16:31,  3.49s/it]
 36%|███▌      | 80198/225000 [153:57:34<185:40:44,  4.62s/it]
 36%|███▌      | 80199/225000 [153:57:43<238:35:03,  5.93s/it]
 36%|███▌      | 80200/225000 [153:57:46<200:25:24,  4.98s/it]
                                                              
{'loss': '0.7807', 'grad_norm': '2.389', 'learning_rate': '3.797e-05', 'epoch': '0.3564', 'num_input_tokens_seen': 253350456, 'train_runtime': '5.543e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80200/225000 [153:57:46<200:25:24,  4.98s/it]
 36%|███▌      | 80201/225000 [153:57:49<173:41:33,  4.32s/it]
 36%|███▌      | 80202/225000 [153:57:51<154:04:05,  3.83s/it]
 36%|███▌      | 80203/225000 [153:57:54<139:39:17,  3.47s/it]
 36%|███▌      | 80204/225000 [153:57:57<130:40:35,  3.25s/it]
 36%|███▌      | 80205/225000 [153:58:05<194:51:58,  4.84s/it]
 36%|███▌      | 80206/225000 [153:58:08<172:16:42,  4.28s/it]
 36%|███▌      | 80207/225000 [153:58:11<149:44:32,  3.72s/it]
 36%|███▌      | 80208/225000 [153:58:14<144:19:04,  3.59s/it]
 36%|███▌      | 80209/225000 [153:58:17<143:03:55,  3.56s/it]
 36%|███▌      | 80210/225000 [153:58:21<139:50:22,  3.48s/it]
 36%|███▌      | 80211/225000 [153:58:24<132:23:44,  3.29s/it]
 36%|███▌      | 80212/225000 [153:58:27<128:41:45,  3.20s/it]
 36%|███▌      | 80213/225000 [153:58:30<130:38:10,  3.25s/it]
 36%|███▌      | 80214/225000 [153:58:33<129:22:09,  3.22s/it]
 36%|███▌      | 80215/225000 [153:58:36<129:49:36,  3.23s/it]
 36%|███▌      | 80216/225000 [153:58:40<138:12:00,  3.44s/it]
 36%|███▌      | 80217/225000 [153:58:49<205:20:39,  5.11s/it]
 36%|███▌      | 80218/225000 [153:58:56<226:08:43,  5.62s/it]
 36%|███▌      | 80219/225000 [153:58:59<191:19:24,  4.76s/it]
 36%|███▌      | 80220/225000 [153:59:01<162:47:46,  4.05s/it]
 36%|███▌      | 80221/225000 [153:59:04<147:32:13,  3.67s/it]
 36%|███▌      | 80222/225000 [153:59:07<136:13:43,  3.39s/it]
 36%|███▌      | 80223/225000 [153:59:10<131:25:26,  3.27s/it]
 36%|███▌      | 80224/225000 [153:59:13<125:22:48,  3.12s/it]
 36%|███▌      | 80225/225000 [153:59:16<123:56:39,  3.08s/it]
 36%|███▌      | 80226/225000 [153:59:19<124:19:59,  3.09s/it]
 36%|███▌      | 80227/225000 [153:59:27<181:56:56,  4.52s/it]
 36%|███▌      | 80228/225000 [153:59:33<206:47:31,  5.14s/it]
 36%|███▌      | 80229/225000 [153:59:36<174:08:01,  4.33s/it]
 36%|███▌      | 80230/225000 [153:59:38<152:06:13,  3.78s/it]
 36%|███▌      | 80231/225000 [153:59:44<173:05:49,  4.30s/it]
 36%|███▌      | 80232/225000 [153:59:50<196:57:14,  4.90s/it]
 36%|███▌      | 80233/225000 [153:59:52<169:23:09,  4.21s/it]
 36%|███▌      | 80234/225000 [153:59:55<150:26:47,  3.74s/it]
 36%|███▌      | 80235/225000 [153:59:58<144:12:34,  3.59s/it]
 36%|███▌      | 80236/225000 [154:00:02<139:19:11,  3.46s/it]
 36%|███▌      | 80237/225000 [154:00:05<137:23:09,  3.42s/it]
 36%|███▌      | 80238/225000 [154:00:12<182:41:38,  4.54s/it]
 36%|███▌      | 80239/225000 [154:00:17<194:13:46,  4.83s/it]
 36%|███▌      | 80240/225000 [154:00:21<174:53:38,  4.35s/it]
 36%|███▌      | 80241/225000 [154:00:24<161:53:17,  4.03s/it]
 36%|███▌      | 80242/225000 [154:00:27<145:05:16,  3.61s/it]
 36%|███▌      | 80243/225000 [154:00:30<139:09:41,  3.46s/it]
 36%|███▌      | 80244/225000 [154:00:33<132:48:44,  3.30s/it]
 36%|███▌      | 80245/225000 [154:00:36<133:18:42,  3.32s/it]
 36%|███▌      | 80246/225000 [154:00:41<157:15:26,  3.91s/it]
 36%|███▌      | 80247/225000 [154:00:50<217:55:10,  5.42s/it]
 36%|███▌      | 80248/225000 [154:00:53<189:00:43,  4.70s/it]
 36%|███▌      | 80249/225000 [154:00:56<165:26:14,  4.11s/it]
 36%|███▌      | 80250/225000 [154:00:59<146:56:47,  3.65s/it]
                                                              
{'loss': '0.8539', 'grad_norm': '4.159', 'learning_rate': '3.796e-05', 'epoch': '0.3567', 'num_input_tokens_seen': 253432456, 'train_runtime': '5.545e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80250/225000 [154:00:59<146:56:47,  3.65s/it]
 36%|███▌      | 80251/225000 [154:01:01<135:39:51,  3.37s/it]
 36%|███▌      | 80252/225000 [154:01:05<133:31:41,  3.32s/it]
 36%|███▌      | 80253/225000 [154:01:08<135:17:53,  3.36s/it]
 36%|███▌      | 80254/225000 [154:01:11<135:35:38,  3.37s/it]
 36%|███▌      | 80255/225000 [154:01:14<132:20:20,  3.29s/it]
 36%|███▌      | 80256/225000 [154:01:18<131:03:53,  3.26s/it]
 36%|███▌      | 80257/225000 [154:01:21<133:17:42,  3.32s/it]
 36%|███▌      | 80258/225000 [154:01:25<134:41:47,  3.35s/it]
 36%|███▌      | 80259/225000 [154:01:31<176:57:31,  4.40s/it]
 36%|███▌      | 80260/225000 [154:01:37<190:09:09,  4.73s/it]
 36%|███▌      | 80261/225000 [154:01:41<179:25:53,  4.46s/it]
 36%|███▌      | 80262/225000 [154:01:44<165:52:21,  4.13s/it]
 36%|███▌      | 80263/225000 [154:01:48<162:30:17,  4.04s/it]
 36%|███▌      | 80264/225000 [154:01:51<155:19:33,  3.86s/it]
 36%|███▌      | 80265/225000 [154:01:54<145:39:38,  3.62s/it]
 36%|███▌      | 80266/225000 [154:02:02<188:49:45,  4.70s/it]
 36%|███▌      | 80267/225000 [154:02:09<216:35:28,  5.39s/it]
 36%|███▌      | 80268/225000 [154:02:11<182:01:44,  4.53s/it]
 36%|███▌      | 80269/225000 [154:02:17<202:56:10,  5.05s/it]
 36%|███▌      | 80270/225000 [154:02:20<176:59:34,  4.40s/it]
 36%|███▌      | 80271/225000 [154:02:23<155:22:51,  3.86s/it]
 36%|███▌      | 80272/225000 [154:02:26<141:14:52,  3.51s/it]
 36%|███▌      | 80273/225000 [154:02:28<131:37:34,  3.27s/it]
 36%|███▌      | 80274/225000 [154:02:31<125:36:29,  3.12s/it]
 36%|███▌      | 80275/225000 [154:02:35<129:31:16,  3.22s/it]
 36%|███▌      | 80276/225000 [154:02:38<129:22:19,  3.22s/it]
 36%|███▌      | 80277/225000 [154:02:46<191:55:40,  4.77s/it]
 36%|███▌      | 80278/225000 [154:02:51<192:20:15,  4.78s/it]
 36%|███▌      | 80279/225000 [154:02:54<174:37:02,  4.34s/it]
 36%|███▌      | 80280/225000 [154:02:57<157:09:39,  3.91s/it]
 36%|███▌      | 80281/225000 [154:03:00<144:33:27,  3.60s/it]
 36%|███▌      | 80282/225000 [154:03:03<137:44:44,  3.43s/it]
 36%|███▌      | 80283/225000 [154:03:06<130:04:09,  3.24s/it]
 36%|███▌      | 80284/225000 [154:03:09<133:25:41,  3.32s/it]
 36%|███▌      | 80285/225000 [154:03:13<131:31:10,  3.27s/it]
 36%|███▌      | 80286/225000 [154:03:16<135:47:05,  3.38s/it]
 36%|███▌      | 80287/225000 [154:03:19<131:52:00,  3.28s/it]
 36%|███▌      | 80288/225000 [154:03:27<180:02:54,  4.48s/it]
 36%|███▌      | 80289/225000 [154:03:35<229:20:38,  5.71s/it]
 36%|███▌      | 80290/225000 [154:03:38<194:58:29,  4.85s/it]
 36%|███▌      | 80291/225000 [154:03:42<179:40:56,  4.47s/it]
 36%|███▌      | 80292/225000 [154:03:47<195:22:14,  4.86s/it]
 36%|███▌      | 80293/225000 [154:03:50<174:44:19,  4.35s/it]
 36%|███▌      | 80294/225000 [154:03:53<158:28:11,  3.94s/it]
 36%|███▌      | 80295/225000 [154:03:56<145:51:59,  3.63s/it]
 36%|███▌      | 80296/225000 [154:04:00<142:10:06,  3.54s/it]
 36%|███▌      | 80297/225000 [154:04:03<135:11:23,  3.36s/it]
 36%|███▌      | 80298/225000 [154:04:06<138:41:48,  3.45s/it]
 36%|███▌      | 80299/225000 [154:04:09<131:31:50,  3.27s/it]
 36%|███▌      | 80300/225000 [154:04:12<127:47:23,  3.18s/it]
                                                              
{'loss': '0.8435', 'grad_norm': '4.714', 'learning_rate': '3.794e-05', 'epoch': '0.3569', 'num_input_tokens_seen': 253514112, 'train_runtime': '5.547e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80300/225000 [154:04:12<127:47:23,  3.18s/it]
 36%|███▌      | 80301/225000 [154:04:16<132:29:10,  3.30s/it]
 36%|███▌      | 80302/225000 [154:04:19<135:18:44,  3.37s/it]
 36%|███▌      | 80303/225000 [154:04:23<141:49:46,  3.53s/it]
 36%|███▌      | 80304/225000 [154:04:32<200:43:55,  4.99s/it]
 36%|███▌      | 80305/225000 [154:04:39<225:26:20,  5.61s/it]
 36%|███▌      | 80306/225000 [154:04:41<189:49:24,  4.72s/it]
 36%|███▌      | 80307/225000 [154:04:44<164:34:44,  4.09s/it]
 36%|███▌      | 80308/225000 [154:04:46<144:17:55,  3.59s/it]
 36%|███▌      | 80309/225000 [154:04:49<130:16:48,  3.24s/it]
 36%|███▌      | 80310/225000 [154:04:52<125:39:36,  3.13s/it]
 36%|███▌      | 80311/225000 [154:04:54<122:48:04,  3.06s/it]
 36%|███▌      | 80312/225000 [154:04:58<124:56:07,  3.11s/it]
 36%|███▌      | 80313/225000 [154:05:06<192:15:00,  4.78s/it]
 36%|███▌      | 80314/225000 [154:05:12<196:47:55,  4.90s/it]
 36%|███▌      | 80315/225000 [154:05:15<175:16:53,  4.36s/it]
 36%|███▌      | 80316/225000 [154:05:17<157:00:39,  3.91s/it]
 36%|███▌      | 80317/225000 [154:05:20<141:52:24,  3.53s/it]
 36%|███▌      | 80318/225000 [154:05:23<131:01:34,  3.26s/it]
 36%|███▌      | 80319/225000 [154:05:25<123:34:26,  3.07s/it]
 36%|███▌      | 80320/225000 [154:05:33<172:28:04,  4.29s/it]
 36%|███▌      | 80321/225000 [154:05:39<192:57:59,  4.80s/it]
 36%|███▌      | 80322/225000 [154:05:41<169:17:53,  4.21s/it]
 36%|███▌      | 80323/225000 [154:05:44<146:35:54,  3.65s/it]
 36%|███▌      | 80324/225000 [154:05:46<133:17:16,  3.32s/it]
 36%|███▌      | 80325/225000 [154:05:54<181:59:25,  4.53s/it]
 36%|███▌      | 80326/225000 [154:05:59<188:37:28,  4.69s/it]
 36%|███▌      | 80327/225000 [154:06:01<161:43:47,  4.02s/it]
 36%|███▌      | 80328/225000 [154:06:07<181:39:25,  4.52s/it]
 36%|███▌      | 80329/225000 [154:06:09<156:06:35,  3.88s/it]
 36%|███▌      | 80330/225000 [154:06:12<141:26:38,  3.52s/it]
 36%|███▌      | 80331/225000 [154:06:15<137:12:43,  3.41s/it]
 36%|███▌      | 80332/225000 [154:06:22<184:22:40,  4.59s/it]
 36%|███▌      | 80333/225000 [154:06:25<160:59:50,  4.01s/it]
 36%|███▌      | 80334/225000 [154:06:27<142:26:57,  3.54s/it]
 36%|███▌      | 80335/225000 [154:06:30<130:29:24,  3.25s/it]
 36%|███▌      | 80336/225000 [154:06:34<139:40:30,  3.48s/it]
 36%|███▌      | 80337/225000 [154:06:37<135:55:51,  3.38s/it]
 36%|███▌      | 80338/225000 [154:06:41<135:35:03,  3.37s/it]
 36%|███▌      | 80339/225000 [154:06:44<136:03:14,  3.39s/it]
 36%|███▌      | 80340/225000 [154:06:47<135:03:40,  3.36s/it]
 36%|███▌      | 80341/225000 [154:06:51<136:34:40,  3.40s/it]
 36%|███▌      | 80342/225000 [154:06:54<134:08:00,  3.34s/it]
 36%|███▌      | 80343/225000 [154:06:58<136:58:28,  3.41s/it]
 36%|███▌      | 80344/225000 [154:07:01<139:33:49,  3.47s/it]
 36%|███▌      | 80345/225000 [154:07:05<139:14:56,  3.47s/it]
 36%|███▌      | 80346/225000 [154:07:08<137:45:40,  3.43s/it]
 36%|███▌      | 80347/225000 [154:07:11<137:38:37,  3.43s/it]
 36%|███▌      | 80348/225000 [154:07:20<196:01:00,  4.88s/it]
 36%|███▌      | 80349/225000 [154:07:29<245:41:35,  6.11s/it]
 36%|███▌      | 80350/225000 [154:07:32<207:49:42,  5.17s/it]
                                                              
{'loss': '0.7891', 'grad_norm': '2.837', 'learning_rate': '3.793e-05', 'epoch': '0.3571', 'num_input_tokens_seen': 253596480, 'train_runtime': '5.549e+05', 'train_tokens_per_second': '457.1'}

 36%|███▌      | 80350/225000 [154:07:32<207:49:42,  5.17s/it]
 36%|███▌      | 80351/225000 [154:07:34<173:53:52,  4.33s/it]
 36%|███▌      | 80352/225000 [154:07:39<178:26:34,  4.44s/it]
 36%|███▌      | 80353/225000 [154:07:43<179:16:42,  4.46s/it]
 36%|███▌      | 80354/225000 [154:07:46<157:08:31,  3.91s/it]
 36%|███▌      | 80355/225000 [154:07:48<140:15:13,  3.49s/it]
 36%|███▌      | 80356/225000 [154:07:55<180:33:32,  4.49s/it]
 36%|███▌      | 80357/225000 [154:07:58<162:40:54,  4.05s/it]
 36%|███▌      | 80358/225000 [154:08:01<148:47:22,  3.70s/it]
 36%|███▌      | 80359/225000 [154:08:07<172:31:27,  4.29s/it]
 36%|███▌      | 80360/225000 [154:08:10<157:42:08,  3.93s/it]
 36%|███▌      | 80361/225000 [154:08:13<151:56:36,  3.78s/it]
 36%|███▌      | 80362/225000 [154:08:16<144:12:10,  3.59s/it]
 36%|███▌      | 80363/225000 [154:08:19<136:23:57,  3.39s/it]
 36%|███▌      | 80364/225000 [154:08:22<132:45:37,  3.30s/it]
 36%|███▌      | 80365/225000 [154:08:26<135:17:19,  3.37s/it]
 36%|███▌      | 80366/225000 [154:08:29<135:59:56,  3.39s/it]
 36%|███▌      | 80367/225000 [154:08:32<132:16:44,  3.29s/it]
 36%|███▌      | 80368/225000 [154:08:36<132:19:57,  3.29s/it]
 36%|███▌      | 80369/225000 [154:08:40<138:33:37,  3.45s/it]
 36%|███▌      | 80370/225000 [154:08:48<198:54:29,  4.95s/it]
 36%|███▌      | 80371/225000 [154:08:56<232:59:04,  5.80s/it]
 36%|███▌      | 80372/225000 [154:08:59<201:08:09,  5.01s/it]
 36%|███▌      | 80373/225000 [154:09:02<172:46:15,  4.30s/it]
 36%|███▌      | 80374/225000 [154:09:04<150:40:26,  3.75s/it]
 36%|███▌      | 80375/225000 [154:09:07<138:23:58,  3.45s/it]
 36%|███▌      | 80376/225000 [154:09:09<127:35:34,  3.18s/it]
 36%|███▌      | 80377/225000 [154:09:19<200:43:02,  5.00s/it]
 36%|███▌      | 80378/225000 [154:09:22<178:17:25,  4.44s/it]
 36%|███▌      | 80379/225000 [154:09:24<156:20:30,  3.89s/it]
 36%|███▌      | 80380/225000 [154:09:27<143:27:42,  3.57s/it]
 36%|███▌      | 80381/225000 [154:09:30<140:07:50,  3.49s/it]
 36%|███▌      | 80382/225000 [154:09:34<136:03:12,  3.39s/it]
 36%|███▌      | 80383/225000 [154:09:42<201:25:37,  5.01s/it]
 36%|███▌      | 80384/225000 [154:09:46<188:22:58,  4.69s/it]
 36%|███▌      | 80385/225000 [154:09:49<162:10:40,  4.04s/it]
 36%|███▌      | 80386/225000 [154:09:56<193:34:38,  4.82s/it]
 36%|███▌      | 80387/225000 [154:09:58<163:35:22,  4.07s/it]
 36%|███▌      | 80388/225000 [154:10:00<146:11:13,  3.64s/it]
 36%|███▌      | 80389/225000 [154:10:05<156:57:16,  3.91s/it]
 36%|███▌      | 80390/225000 [154:10:11<183:52:07,  4.58s/it]
 36%|███▌      | 80391/225000 [154:10:14<163:16:43,  4.06s/it]
 36%|███▌      | 80392/225000 [154:10:17<149:41:17,  3.73s/it]
 36%|███▌      | 80393/225000 [154:10:20<138:26:18,  3.45s/it]
 36%|███▌      | 80394/225000 [154:10:23<131:40:38,  3.28s/it]
 36%|███▌      | 80395/225000 [154:10:26<128:08:28,  3.19s/it]
 36%|███▌      | 80396/225000 [154:10:29<135:49:34,  3.38s/it]
 36%|███▌      | 80397/225000 [154:10:38<197:49:30,  4.93s/it]
 36%|███▌      | 80398/225000 [154:10:42<182:14:51,  4.54s/it]
 36%|███▌      | 80399/225000 [154:10:44<155:58:42,  3.88s/it]
 36%|███▌      | 80400/225000 [154:10:47<144:06:24,  3.59s/it]
                                                              
{'loss': '0.7828', 'grad_norm': '5.813', 'learning_rate': '3.791e-05', 'epoch': '0.3573', 'num_input_tokens_seen': 253679384, 'train_runtime': '5.55e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80400/225000 [154:10:47<144:06:24,  3.59s/it]
 36%|███▌      | 80401/225000 [154:10:50<142:17:48,  3.54s/it]
 36%|███▌      | 80402/225000 [154:10:53<137:46:24,  3.43s/it]
 36%|███▌      | 80403/225000 [154:10:57<135:19:59,  3.37s/it]
 36%|███▌      | 80404/225000 [154:11:00<135:39:53,  3.38s/it]
 36%|███▌      | 80405/225000 [154:11:03<135:18:54,  3.37s/it]
 36%|███▌      | 80406/225000 [154:11:07<134:07:48,  3.34s/it]
 36%|███▌      | 80407/225000 [154:11:10<133:04:17,  3.31s/it]
 36%|███▌      | 80408/225000 [154:11:19<201:13:38,  5.01s/it]
 36%|███▌      | 80409/225000 [154:11:27<239:16:03,  5.96s/it]
 36%|███▌      | 80410/225000 [154:11:30<197:21:07,  4.91s/it]
 36%|███▌      | 80411/225000 [154:11:32<168:05:42,  4.19s/it]
 36%|███▌      | 80412/225000 [154:11:35<151:08:12,  3.76s/it]
 36%|███▌      | 80413/225000 [154:11:38<140:23:00,  3.50s/it]
 36%|███▌      | 80414/225000 [154:11:40<131:50:53,  3.28s/it]
 36%|███▌      | 80415/225000 [154:11:45<150:47:20,  3.75s/it]
 36%|███▌      | 80416/225000 [154:11:54<213:30:44,  5.32s/it]
 36%|███▌      | 80417/225000 [154:11:57<186:44:34,  4.65s/it]
 36%|███▌      | 80418/225000 [154:12:00<158:03:03,  3.94s/it]
 36%|███▌      | 80419/225000 [154:12:02<141:02:59,  3.51s/it]
 36%|███▌      | 80420/225000 [154:12:05<134:48:50,  3.36s/it]
 36%|███▌      | 80421/225000 [154:12:08<133:29:42,  3.32s/it]
 36%|███▌      | 80422/225000 [154:12:16<183:24:30,  4.57s/it]
 36%|███▌      | 80423/225000 [154:12:20<180:39:45,  4.50s/it]
 36%|███▌      | 80424/225000 [154:12:24<168:58:34,  4.21s/it]
 36%|███▌      | 80425/225000 [154:12:28<174:30:26,  4.35s/it]
 36%|███▌      | 80426/225000 [154:12:31<159:00:38,  3.96s/it]
 36%|███▌      | 80427/225000 [154:12:34<145:08:35,  3.61s/it]
 36%|███▌      | 80428/225000 [154:12:37<136:11:08,  3.39s/it]
 36%|███▌      | 80429/225000 [154:12:40<132:48:11,  3.31s/it]
 36%|███▌      | 80430/225000 [154:12:43<130:02:33,  3.24s/it]
 36%|███▌      | 80431/225000 [154:12:46<125:04:05,  3.11s/it]
 36%|███▌      | 80432/225000 [154:12:50<133:05:06,  3.31s/it]
 36%|███▌      | 80433/225000 [154:12:58<186:24:39,  4.64s/it]
 36%|███▌      | 80434/225000 [154:13:04<209:22:43,  5.21s/it]
 36%|███▌      | 80435/225000 [154:13:07<180:18:04,  4.49s/it]
 36%|███▌      | 80436/225000 [154:13:10<162:23:48,  4.04s/it]
 36%|███▌      | 80437/225000 [154:13:13<146:46:16,  3.65s/it]
 36%|███▌      | 80438/225000 [154:13:16<136:52:54,  3.41s/it]
 36%|███▌      | 80439/225000 [154:13:19<136:57:54,  3.41s/it]
 36%|███▌      | 80440/225000 [154:13:29<211:03:43,  5.26s/it]
 36%|███▌      | 80441/225000 [154:13:32<186:24:16,  4.64s/it]
 36%|███▌      | 80442/225000 [154:13:34<159:32:35,  3.97s/it]
 36%|███▌      | 80443/225000 [154:13:41<189:11:40,  4.71s/it]
 36%|███▌      | 80444/225000 [154:13:44<177:01:13,  4.41s/it]
 36%|███▌      | 80445/225000 [154:13:47<151:23:19,  3.77s/it]
 36%|███▌      | 80446/225000 [154:13:49<135:46:30,  3.38s/it]
 36%|███▌      | 80447/225000 [154:13:52<130:38:47,  3.25s/it]
 36%|███▌      | 80448/225000 [154:13:55<128:07:51,  3.19s/it]
 36%|███▌      | 80449/225000 [154:13:58<129:55:21,  3.24s/it]
 36%|███▌      | 80450/225000 [154:14:01<127:18:50,  3.17s/it]
                                                              
{'loss': '0.7483', 'grad_norm': '3.422', 'learning_rate': '3.79e-05', 'epoch': '0.3576', 'num_input_tokens_seen': 253761080, 'train_runtime': '5.552e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80450/225000 [154:14:01<127:18:50,  3.17s/it]
 36%|███▌      | 80451/225000 [154:14:05<130:54:51,  3.26s/it]
 36%|███▌      | 80452/225000 [154:14:08<127:59:31,  3.19s/it]
 36%|███▌      | 80453/225000 [154:14:14<159:44:53,  3.98s/it]
 36%|███▌      | 80454/225000 [154:14:21<202:04:23,  5.03s/it]
 36%|███▌      | 80455/225000 [154:14:25<187:44:46,  4.68s/it]
 36%|███▌      | 80456/225000 [154:14:28<167:19:11,  4.17s/it]
 36%|███▌      | 80457/225000 [154:14:31<152:18:03,  3.79s/it]
 36%|███▌      | 80458/225000 [154:14:34<147:07:32,  3.66s/it]
 36%|███▌      | 80459/225000 [154:14:38<145:35:53,  3.63s/it]
 36%|███▌      | 80460/225000 [154:14:41<139:11:03,  3.47s/it]
 36%|███▌      | 80461/225000 [154:14:44<137:26:28,  3.42s/it]
 36%|███▌      | 80462/225000 [154:14:48<135:04:24,  3.36s/it]
 36%|███▌      | 80463/225000 [154:14:55<183:51:52,  4.58s/it]
 36%|███▌      | 80464/225000 [154:15:02<215:10:28,  5.36s/it]
 36%|███▌      | 80465/225000 [154:15:04<178:28:24,  4.45s/it]
 36%|███▌      | 80466/225000 [154:15:07<154:43:56,  3.85s/it]
 36%|███▌      | 80467/225000 [154:15:10<146:24:41,  3.65s/it]
 36%|███▌      | 80468/225000 [154:15:13<141:23:26,  3.52s/it]
 36%|███▌      | 80469/225000 [154:15:17<137:46:25,  3.43s/it]
 36%|███▌      | 80470/225000 [154:15:20<136:14:34,  3.39s/it]
 36%|███▌      | 80471/225000 [154:15:23<135:07:51,  3.37s/it]
 36%|███▌      | 80472/225000 [154:15:27<139:14:41,  3.47s/it]
 36%|███▌      | 80473/225000 [154:15:31<144:14:05,  3.59s/it]
 36%|███▌      | 80474/225000 [154:15:34<140:39:50,  3.50s/it]
 36%|███▌      | 80475/225000 [154:15:37<139:25:18,  3.47s/it]
 36%|███▌      | 80476/225000 [154:15:41<138:53:20,  3.46s/it]
 36%|███▌      | 80477/225000 [154:15:46<156:27:23,  3.90s/it]
 36%|███▌      | 80478/225000 [154:15:52<180:22:47,  4.49s/it]
 36%|███▌      | 80479/225000 [154:15:55<167:11:25,  4.16s/it]
 36%|███▌      | 80480/225000 [154:15:59<162:40:26,  4.05s/it]
 36%|███▌      | 80481/225000 [154:16:03<161:01:15,  4.01s/it]
 36%|███▌      | 80482/225000 [154:16:09<190:43:21,  4.75s/it]
 36%|███▌      | 80483/225000 [154:16:15<201:22:10,  5.02s/it]
 36%|███▌      | 80484/225000 [154:16:18<172:13:01,  4.29s/it]
 36%|███▌      | 80485/225000 [154:16:20<153:29:30,  3.82s/it]
 36%|███▌      | 80486/225000 [154:16:23<143:45:53,  3.58s/it]
 36%|███▌      | 80487/225000 [154:16:26<134:44:59,  3.36s/it]
 36%|███▌      | 80488/225000 [154:16:29<131:48:30,  3.28s/it]
 36%|███▌      | 80489/225000 [154:16:34<144:46:37,  3.61s/it]
 36%|███▌      | 80490/225000 [154:16:42<202:31:45,  5.05s/it]
 36%|███▌      | 80491/225000 [154:16:46<185:47:08,  4.63s/it]
 36%|███▌      | 80492/225000 [154:16:49<166:22:19,  4.14s/it]
 36%|███▌      | 80493/225000 [154:16:51<146:45:07,  3.66s/it]
 36%|███▌      | 80494/225000 [154:16:54<135:19:25,  3.37s/it]
 36%|███▌      | 80495/225000 [154:17:03<199:26:08,  4.97s/it]
 36%|███▌      | 80496/225000 [154:17:06<179:11:59,  4.46s/it]
 36%|███▌      | 80497/225000 [154:17:09<157:52:50,  3.93s/it]
 36%|███▌      | 80498/225000 [154:17:11<143:51:56,  3.58s/it]
 36%|███▌      | 80499/225000 [154:17:14<130:14:06,  3.24s/it]
 36%|███▌      | 80500/225000 [154:17:21<182:36:38,  4.55s/it]
                                                              
{'loss': '0.7512', 'grad_norm': '2.612', 'learning_rate': '3.788e-05', 'epoch': '0.3578', 'num_input_tokens_seen': 253843680, 'train_runtime': '5.554e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80500/225000 [154:17:21<182:36:38,  4.55s/it]
 36%|███▌      | 80501/225000 [154:17:26<189:44:50,  4.73s/it]
 36%|███▌      | 80502/225000 [154:17:30<170:16:21,  4.24s/it]
 36%|███▌      | 80503/225000 [154:17:33<163:37:11,  4.08s/it]
 36%|███▌      | 80504/225000 [154:17:37<154:26:15,  3.85s/it]
 36%|███▌      | 80505/225000 [154:17:40<145:24:35,  3.62s/it]
 36%|███▌      | 80506/225000 [154:17:43<139:21:02,  3.47s/it]
 36%|███▌      | 80507/225000 [154:17:46<137:51:29,  3.43s/it]
 36%|███▌      | 80508/225000 [154:17:49<133:07:48,  3.32s/it]
 36%|███▌      | 80509/225000 [154:17:53<133:26:38,  3.32s/it]
 36%|███▌      | 80510/225000 [154:17:56<134:24:53,  3.35s/it]
 36%|███▌      | 80511/225000 [154:17:59<134:53:25,  3.36s/it]
 36%|███▌      | 80512/225000 [154:18:03<137:14:57,  3.42s/it]
 36%|███▌      | 80513/225000 [154:18:06<137:27:10,  3.42s/it]
 36%|███▌      | 80514/225000 [154:18:10<135:04:40,  3.37s/it]
 36%|███▌      | 80515/225000 [154:18:13<130:41:14,  3.26s/it]
 36%|███▌      | 80516/225000 [154:18:16<131:51:51,  3.29s/it]
 36%|███▌      | 80517/225000 [154:18:20<141:25:44,  3.52s/it]
 36%|███▌      | 80518/225000 [154:18:29<203:45:17,  5.08s/it]
 36%|███▌      | 80519/225000 [154:18:36<233:04:48,  5.81s/it]
 36%|███▌      | 80520/225000 [154:18:39<193:54:57,  4.83s/it]
 36%|███▌      | 80521/225000 [154:18:41<164:23:51,  4.10s/it]
 36%|███▌      | 80522/225000 [154:18:44<146:59:30,  3.66s/it]
 36%|███▌      | 80523/225000 [154:18:49<167:04:59,  4.16s/it]
 36%|███▌      | 80524/225000 [154:18:55<183:56:48,  4.58s/it]
 36%|███▌      | 80525/225000 [154:18:57<159:49:21,  3.98s/it]
 36%|███▌      | 80526/225000 [154:19:00<144:15:09,  3.59s/it]
 36%|███▌      | 80527/225000 [154:19:03<132:11:10,  3.29s/it]
 36%|███▌      | 80528/225000 [154:19:10<177:06:50,  4.41s/it]
 36%|███▌      | 80529/225000 [154:19:14<178:46:14,  4.45s/it]
 36%|███▌      | 80530/225000 [154:19:17<156:43:57,  3.91s/it]
 36%|███▌      | 80531/225000 [154:19:20<143:04:21,  3.57s/it]
 36%|███▌      | 80532/225000 [154:19:22<133:03:10,  3.32s/it]
 36%|███▌      | 80533/225000 [154:19:26<142:03:46,  3.54s/it]
 36%|███▌      | 80534/225000 [154:19:35<202:43:39,  5.05s/it]
 36%|███▌      | 80535/225000 [154:19:38<173:25:46,  4.32s/it]
 36%|███▌      | 80536/225000 [154:19:40<155:04:14,  3.86s/it]
 36%|███▌      | 80537/225000 [154:19:43<142:39:15,  3.55s/it]
 36%|███▌      | 80538/225000 [154:19:46<135:22:54,  3.37s/it]
 36%|███▌      | 80539/225000 [154:19:51<156:14:37,  3.89s/it]
 36%|███▌      | 80540/225000 [154:19:57<175:50:26,  4.38s/it]
 36%|███▌      | 80541/225000 [154:20:00<157:32:48,  3.93s/it]
 36%|███▌      | 80542/225000 [154:20:02<144:13:51,  3.59s/it]
 36%|███▌      | 80543/225000 [154:20:05<135:38:07,  3.38s/it]
 36%|███▌      | 80544/225000 [154:20:08<132:27:12,  3.30s/it]
 36%|███▌      | 80545/225000 [154:20:11<127:20:24,  3.17s/it]
 36%|███▌      | 80546/225000 [154:20:15<129:44:40,  3.23s/it]
 36%|███▌      | 80547/225000 [154:20:20<153:01:22,  3.81s/it]
 36%|███▌      | 80548/225000 [154:20:29<213:24:17,  5.32s/it]
 36%|███▌      | 80549/225000 [154:20:32<184:44:34,  4.60s/it]
 36%|███▌      | 80550/225000 [154:20:34<161:54:48,  4.04s/it]
                                                              
{'loss': '0.7622', 'grad_norm': '3.253', 'learning_rate': '3.786e-05', 'epoch': '0.358', 'num_input_tokens_seen': 253926248, 'train_runtime': '5.556e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80550/225000 [154:20:34<161:54:48,  4.04s/it]
 36%|███▌      | 80551/225000 [154:20:37<145:40:33,  3.63s/it]
 36%|███▌      | 80552/225000 [154:20:40<139:53:48,  3.49s/it]
 36%|███▌      | 80553/225000 [154:20:43<134:09:06,  3.34s/it]
 36%|███▌      | 80554/225000 [154:20:50<177:35:36,  4.43s/it]
 36%|███▌      | 80555/225000 [154:20:57<212:43:04,  5.30s/it]
 36%|███▌      | 80556/225000 [154:21:00<178:48:19,  4.46s/it]
 36%|███▌      | 80557/225000 [154:21:03<156:57:10,  3.91s/it]
 36%|███▌      | 80558/225000 [154:21:06<154:25:03,  3.85s/it]
 36%|███▌      | 80559/225000 [154:21:13<193:05:05,  4.81s/it]
 36%|███▌      | 80560/225000 [154:21:16<166:36:49,  4.15s/it]
 36%|███▌      | 80561/225000 [154:21:19<149:51:37,  3.74s/it]
 36%|███▌      | 80562/225000 [154:21:22<138:56:53,  3.46s/it]
 36%|███▌      | 80563/225000 [154:21:24<131:18:32,  3.27s/it]
 36%|███▌      | 80564/225000 [154:21:30<154:38:27,  3.85s/it]
 36%|███▌      | 80565/225000 [154:21:38<203:39:24,  5.08s/it]
 36%|███▌      | 80566/225000 [154:21:40<172:23:16,  4.30s/it]
 36%|███▌      | 80567/225000 [154:21:45<176:34:59,  4.40s/it]
 36%|███▌      | 80568/225000 [154:21:50<183:52:05,  4.58s/it]
 36%|███▌      | 80569/225000 [154:21:52<162:07:20,  4.04s/it]
 36%|███▌      | 80570/225000 [154:21:56<157:46:58,  3.93s/it]
 36%|███▌      | 80571/225000 [154:21:59<142:45:05,  3.56s/it]
 36%|███▌      | 80572/225000 [154:22:02<134:46:07,  3.36s/it]
 36%|███▌      | 80573/225000 [154:22:04<124:04:30,  3.09s/it]
 36%|███▌      | 80574/225000 [154:22:07<116:58:28,  2.92s/it]
 36%|███▌      | 80575/225000 [154:22:10<127:36:06,  3.18s/it]
 36%|███▌      | 80576/225000 [154:22:19<190:48:41,  4.76s/it]
 36%|███▌      | 80577/225000 [154:22:23<178:47:10,  4.46s/it]
 36%|███▌      | 80578/225000 [154:22:26<160:28:40,  4.00s/it]
 36%|███▌      | 80579/225000 [154:22:28<145:18:59,  3.62s/it]
 36%|███▌      | 80580/225000 [154:22:31<138:09:56,  3.44s/it]
 36%|███▌      | 80581/225000 [154:22:34<131:31:46,  3.28s/it]
 36%|███▌      | 80582/225000 [154:22:38<132:58:34,  3.31s/it]
 36%|███▌      | 80583/225000 [154:22:43<157:48:47,  3.93s/it]
 36%|███▌      | 80584/225000 [154:22:50<197:19:36,  4.92s/it]
 36%|███▌      | 80585/225000 [154:22:53<171:24:22,  4.27s/it]
 36%|███▌      | 80586/225000 [154:22:59<193:53:07,  4.83s/it]
 36%|███▌      | 80587/225000 [154:23:04<189:51:47,  4.73s/it]
 36%|███▌      | 80588/225000 [154:23:07<172:25:06,  4.30s/it]
 36%|███▌      | 80589/225000 [154:23:10<161:12:43,  4.02s/it]
 36%|███▌      | 80590/225000 [154:23:14<152:44:37,  3.81s/it]
 36%|███▌      | 80591/225000 [154:23:17<142:58:11,  3.56s/it]
 36%|███▌      | 80592/225000 [154:23:20<143:30:14,  3.58s/it]
 36%|███▌      | 80593/225000 [154:23:24<146:20:27,  3.65s/it]
 36%|███▌      | 80594/225000 [154:23:27<143:15:50,  3.57s/it]
 36%|███▌      | 80595/225000 [154:23:31<139:34:52,  3.48s/it]
 36%|███▌      | 80596/225000 [154:23:34<135:22:49,  3.38s/it]
 36%|███▌      | 80597/225000 [154:23:39<158:13:38,  3.94s/it]
 36%|███▌      | 80598/225000 [154:23:43<152:26:25,  3.80s/it]
 36%|███▌      | 80599/225000 [154:23:46<144:14:33,  3.60s/it]
 36%|███▌      | 80600/225000 [154:23:49<138:35:07,  3.46s/it]
                                                              
{'loss': '0.7977', 'grad_norm': '4.348', 'learning_rate': '3.785e-05', 'epoch': '0.3582', 'num_input_tokens_seen': 254009216, 'train_runtime': '5.558e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80600/225000 [154:23:49<138:35:07,  3.46s/it]
 36%|███▌      | 80601/225000 [154:23:53<141:51:37,  3.54s/it]
 36%|███▌      | 80602/225000 [154:23:56<136:56:47,  3.41s/it]
 36%|███▌      | 80603/225000 [154:23:59<136:36:52,  3.41s/it]
 36%|███▌      | 80604/225000 [154:24:02<132:03:40,  3.29s/it]
 36%|███▌      | 80605/225000 [154:24:11<198:00:03,  4.94s/it]
 36%|███▌      | 80606/225000 [154:24:17<218:28:20,  5.45s/it]
 36%|███▌      | 80607/225000 [154:24:20<179:51:39,  4.48s/it]
 36%|███▌      | 80608/225000 [154:24:23<167:37:43,  4.18s/it]
 36%|███▌      | 80609/225000 [154:24:27<167:32:47,  4.18s/it]
 36%|███▌      | 80610/225000 [154:24:33<189:00:28,  4.71s/it]
 36%|███▌      | 80611/225000 [154:24:36<163:59:05,  4.09s/it]
 36%|███▌      | 80612/225000 [154:24:40<158:57:09,  3.96s/it]
 36%|███▌      | 80613/225000 [154:24:43<156:09:35,  3.89s/it]
 36%|███▌      | 80614/225000 [154:24:47<152:06:03,  3.79s/it]
 36%|███▌      | 80615/225000 [154:24:50<144:54:32,  3.61s/it]
 36%|███▌      | 80616/225000 [154:24:54<142:33:02,  3.55s/it]
 36%|███▌      | 80617/225000 [154:24:57<140:10:19,  3.50s/it]
 36%|███▌      | 80618/225000 [154:25:00<135:28:17,  3.38s/it]
 36%|███▌      | 80619/225000 [154:25:04<139:48:18,  3.49s/it]
 36%|███▌      | 80620/225000 [154:25:07<142:14:23,  3.55s/it]
 36%|███▌      | 80621/225000 [154:25:13<162:40:22,  4.06s/it]
 36%|███▌      | 80622/225000 [154:25:16<157:00:07,  3.91s/it]
 36%|███▌      | 80623/225000 [154:25:20<152:00:05,  3.79s/it]
 36%|███▌      | 80624/225000 [154:25:23<145:39:32,  3.63s/it]
 36%|███▌      | 80625/225000 [154:25:29<178:15:04,  4.44s/it]
 36%|███▌      | 80626/225000 [154:25:32<160:37:54,  4.01s/it]
 36%|███▌      | 80627/225000 [154:25:36<150:50:47,  3.76s/it]
 36%|███▌      | 80628/225000 [154:25:43<191:25:07,  4.77s/it]
 36%|███▌      | 80629/225000 [154:25:47<180:43:26,  4.51s/it]
 36%|███▌      | 80630/225000 [154:25:50<164:01:07,  4.09s/it]
 36%|███▌      | 80631/225000 [154:25:53<158:37:22,  3.96s/it]
 36%|███▌      | 80632/225000 [154:25:56<148:00:58,  3.69s/it]
 36%|███▌      | 80633/225000 [154:25:59<139:08:22,  3.47s/it]
 36%|███▌      | 80634/225000 [154:26:08<196:24:48,  4.90s/it]
 36%|███▌      | 80635/225000 [154:26:12<193:11:17,  4.82s/it]
 36%|███▌      | 80636/225000 [154:26:15<164:24:06,  4.10s/it]
 36%|███▌      | 80637/225000 [154:26:17<147:21:12,  3.67s/it]
 36%|███▌      | 80638/225000 [154:26:21<142:17:58,  3.55s/it]
 36%|███▌      | 80639/225000 [154:26:29<199:09:17,  4.97s/it]
 36%|███▌      | 80640/225000 [154:26:34<196:28:58,  4.90s/it]
 36%|███▌      | 80641/225000 [154:26:36<168:17:31,  4.20s/it]
 36%|███▌      | 80642/225000 [154:26:40<158:36:17,  3.96s/it]
 36%|███▌      | 80643/225000 [154:26:42<143:08:16,  3.57s/it]
 36%|███▌      | 80644/225000 [154:26:46<147:18:18,  3.67s/it]
 36%|███▌      | 80645/225000 [154:26:55<211:48:29,  5.28s/it]
 36%|███▌      | 80646/225000 [154:26:58<183:03:20,  4.57s/it]
 36%|███▌      | 80647/225000 [154:27:01<158:54:56,  3.96s/it]
 36%|███▌      | 80648/225000 [154:27:03<141:36:46,  3.53s/it]
 36%|███▌      | 80649/225000 [154:27:06<134:54:47,  3.36s/it]
 36%|███▌      | 80650/225000 [154:27:09<133:42:15,  3.33s/it]
                                                              
{'loss': '0.781', 'grad_norm': '2.792', 'learning_rate': '3.783e-05', 'epoch': '0.3584', 'num_input_tokens_seen': 254092984, 'train_runtime': '5.56e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80650/225000 [154:27:09<133:42:15,  3.33s/it]
 36%|███▌      | 80651/225000 [154:27:12<126:29:10,  3.15s/it]
 36%|███▌      | 80652/225000 [154:27:21<193:06:43,  4.82s/it]
 36%|███▌      | 80653/225000 [154:27:28<217:44:10,  5.43s/it]
 36%|███▌      | 80654/225000 [154:27:30<182:10:35,  4.54s/it]
 36%|███▌      | 80655/225000 [154:27:34<171:23:45,  4.27s/it]
 36%|███▌      | 80656/225000 [154:27:40<189:49:05,  4.73s/it]
 36%|███▌      | 80657/225000 [154:27:42<162:53:14,  4.06s/it]
 36%|███▌      | 80658/225000 [154:27:48<185:19:55,  4.62s/it]
 36%|███▌      | 80659/225000 [154:27:50<158:20:08,  3.95s/it]
 36%|███▌      | 80660/225000 [154:27:53<142:27:58,  3.55s/it]
 36%|███▌      | 80661/225000 [154:27:55<128:31:37,  3.21s/it]
 36%|███▌      | 80662/225000 [154:28:04<199:21:32,  4.97s/it]
 36%|███▌      | 80663/225000 [154:28:07<169:36:12,  4.23s/it]
 36%|███▌      | 80664/225000 [154:28:09<147:13:12,  3.67s/it]
 36%|███▌      | 80665/225000 [154:28:16<177:26:58,  4.43s/it]
 36%|███▌      | 80666/225000 [154:28:19<170:53:34,  4.26s/it]
 36%|███▌      | 80667/225000 [154:28:23<157:02:46,  3.92s/it]
 36%|███▌      | 80668/225000 [154:28:26<148:39:18,  3.71s/it]
 36%|███▌      | 80669/225000 [154:28:29<140:40:36,  3.51s/it]
 36%|███▌      | 80670/225000 [154:28:32<134:18:47,  3.35s/it]
 36%|███▌      | 80671/225000 [154:28:35<135:04:38,  3.37s/it]
 36%|███▌      | 80672/225000 [154:28:38<133:00:16,  3.32s/it]
 36%|███▌      | 80673/225000 [154:28:41<127:45:23,  3.19s/it]
 36%|███▌      | 80674/225000 [154:28:45<132:18:04,  3.30s/it]
 36%|███▌      | 80675/225000 [154:28:48<132:12:00,  3.30s/it]
 36%|███▌      | 80676/225000 [154:28:51<131:23:35,  3.28s/it]
 36%|███▌      | 80677/225000 [154:28:55<132:45:26,  3.31s/it]
 36%|███▌      | 80678/225000 [154:28:58<136:21:12,  3.40s/it]
 36%|███▌      | 80679/225000 [154:29:08<208:58:29,  5.21s/it]
 36%|███▌      | 80680/225000 [154:29:14<224:25:17,  5.60s/it]
 36%|███▌      | 80681/225000 [154:29:17<188:15:07,  4.70s/it]
 36%|███▌      | 80682/225000 [154:29:20<164:07:11,  4.09s/it]
 36%|███▌      | 80683/225000 [154:29:22<146:17:14,  3.65s/it]
 36%|███▌      | 80684/225000 [154:29:28<169:57:46,  4.24s/it]
 36%|███▌      | 80685/225000 [154:29:34<191:57:15,  4.79s/it]
 36%|███▌      | 80686/225000 [154:29:36<164:48:52,  4.11s/it]
 36%|███▌      | 80687/225000 [154:29:39<149:27:52,  3.73s/it]
 36%|███▌      | 80688/225000 [154:29:42<140:07:07,  3.50s/it]
 36%|███▌      | 80689/225000 [154:29:45<131:38:30,  3.28s/it]
 36%|███▌      | 80690/225000 [154:29:53<186:47:23,  4.66s/it]
 36%|███▌      | 80691/225000 [154:29:57<174:40:19,  4.36s/it]
 36%|███▌      | 80692/225000 [154:29:59<157:01:35,  3.92s/it]
 36%|███▌      | 80693/225000 [154:30:02<142:31:32,  3.56s/it]
 36%|███▌      | 80694/225000 [154:30:05<133:37:29,  3.33s/it]
 36%|███▌      | 80695/225000 [154:30:08<126:56:21,  3.17s/it]
 36%|███▌      | 80696/225000 [154:30:11<122:40:43,  3.06s/it]
 36%|███▌      | 80697/225000 [154:30:14<125:28:46,  3.13s/it]
 36%|███▌      | 80698/225000 [154:30:24<204:58:37,  5.11s/it]
 36%|███▌      | 80699/225000 [154:30:28<199:08:19,  4.97s/it]
 36%|███▌      | 80700/225000 [154:30:31<170:44:59,  4.26s/it]
                                                              
{'loss': '0.7411', 'grad_norm': '3.903', 'learning_rate': '3.782e-05', 'epoch': '0.3587', 'num_input_tokens_seen': 254177048, 'train_runtime': '5.562e+05', 'train_tokens_per_second': '457'}

 36%|███▌      | 80700/225000 [154:30:31<170:44:59,  4.26s/it]
 36%|███▌      | 80701/225000 [154:30:37<192:16:32,  4.80s/it]
 36%|███▌      | 80702/225000 [154:30:40<168:43:24,  4.21s/it]
 36%|███▌      | 80703/225000 [154:30:42<150:04:30,  3.74s/it]
 36%|███▌      | 80704/225000 [154:30:46<145:17:41,  3.62s/it]
 36%|███▌      | 80705/225000 [154:30:49<139:53:45,  3.49s/it]
 36%|███▌      | 80706/225000 [154:30:52<138:02:39,  3.44s/it]
 36%|███▌      | 80707/225000 [154:30:55<135:52:10,  3.39s/it]
 36%|███▌      | 80708/225000 [154:31:00<152:31:59,  3.81s/it]
 36%|███▌      | 80709/225000 [154:31:09<214:29:35,  5.35s/it]
 36%|███▌      | 80710/225000 [154:31:12<179:36:23,  4.48s/it]
 36%|███▌      | 80711/225000 [154:31:14<157:06:25,  3.92s/it]
 36%|███▌      | 80712/225000 [154:31:17<138:56:13,  3.47s/it]
 36%|███▌      | 80713/225000 [154:31:25<200:57:21,  5.01s/it]
 36%|███▌      | 80714/225000 [154:31:30<201:43:33,  5.03s/it]
 36%|███▌      | 80715/225000 [154:31:34<179:36:31,  4.48s/it]
 36%|███▌      | 80716/225000 [154:31:37<163:14:38,  4.07s/it]
 36%|███▌      | 80717/225000 [154:31:40<149:19:09,  3.73s/it]
 36%|███▌      | 80718/225000 [154:31:43<142:42:36,  3.56s/it]
 36%|███▌      | 80719/225000 [154:31:45<131:56:39,  3.29s/it]
 36%|███▌      | 80720/225000 [154:31:50<143:59:30,  3.59s/it]
 36%|███▌      | 80721/225000 [154:31:58<199:09:21,  4.97s/it]
 36%|███▌      | 80722/225000 [154:32:01<173:18:45,  4.32s/it]
 36%|███▌      | 80723/225000 [154:32:03<149:53:39,  3.74s/it]
 36%|███▌      | 80724/225000 [154:32:07<153:48:09,  3.84s/it]
 36%|███▌      | 80725/225000 [154:32:15<203:30:56,  5.08s/it]
 36%|███▌      | 80726/225000 [154:32:18<180:02:51,  4.49s/it]
 36%|███▌      | 80727/225000 [154:32:21<158:54:02,  3.96s/it]
 36%|███▌      | 80728/225000 [154:32:24<147:25:35,  3.68s/it]
 36%|███▌      | 80729/225000 [154:32:27<136:58:57,  3.42s/it]
 36%|███▌      | 80730/225000 [154:32:32<153:03:27,  3.82s/it]
 36%|███▌      | 80731/225000 [154:32:41<215:23:33,  5.37s/it]
 36%|███▌      | 80732/225000 [154:32:43<177:13:38,  4.42s/it]
 36%|███▌      | 80733/225000 [154:32:45<150:48:40,  3.76s/it]
 36%|███▌      | 80734/225000 [154:32:52<186:34:33,  4.66s/it]
 36%|███▌      | 80735/225000 [154:32:54<157:25:48,  3.93s/it]
 36%|███▌      | 80736/225000 [154:32:56<138:24:00,  3.45s/it]
 36%|███▌      | 80737/225000 [154:32:59<126:44:18,  3.16s/it]
 36%|███▌      | 80738/225000 [154:33:02<124:13:28,  3.10s/it]
 36%|███▌      | 80739/225000 [154:33:06<137:50:58,  3.44s/it]
 36%|███▌      | 80740/225000 [154:33:13<177:03:43,  4.42s/it]
 36%|███▌      | 80741/225000 [154:33:15<152:47:27,  3.81s/it]
 36%|███▌      | 80742/225000 [154:33:18<140:02:23,  3.49s/it]
 36%|███▌      | 80743/225000 [154:33:20<128:36:11,  3.21s/it]
 36%|███▌      | 80744/225000 [154:33:23<124:18:33,  3.10s/it]
 36%|███▌      | 80745/225000 [154:33:26<124:44:51,  3.11s/it]
 36%|███▌      | 80746/225000 [154:33:35<191:56:01,  4.79s/it]
 36%|███▌      | 80747/225000 [154:33:40<198:48:28,  4.96s/it]
 36%|███▌      | 80748/225000 [154:33:43<165:30:48,  4.13s/it]
 36%|███▌      | 80749/225000 [154:33:45<142:03:17,  3.55s/it]
 36%|███▌      | 80750/225000 [154:33:52<181:11:15,  4.52s/it]
                                                              
{'loss': '0.7822', 'grad_norm': '3.504', 'learning_rate': '3.78e-05', 'epoch': '0.3589', 'num_input_tokens_seen': 254260064, 'train_runtime': '5.564e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 80750/225000 [154:33:52<181:11:15,  4.52s/it]
 36%|███▌      | 80751/225000 [154:33:54<159:02:08,  3.97s/it]
 36%|███▌      | 80752/225000 [154:33:57<144:41:10,  3.61s/it]
 36%|███▌      | 80753/225000 [154:34:00<134:17:17,  3.35s/it]
 36%|███▌      | 80754/225000 [154:34:03<127:33:56,  3.18s/it]
 36%|███▌      | 80755/225000 [154:34:05<123:34:59,  3.08s/it]
 36%|███▌      | 80756/225000 [154:34:09<127:42:01,  3.19s/it]
 36%|███▌      | 80757/225000 [154:34:16<180:14:24,  4.50s/it]
 36%|███▌      | 80758/225000 [154:34:19<156:28:29,  3.91s/it]
 36%|███▌      | 80759/225000 [154:34:21<139:51:23,  3.49s/it]
 36%|███▌      | 80760/225000 [154:34:24<130:03:02,  3.25s/it]
 36%|███▌      | 80761/225000 [154:34:29<151:33:21,  3.78s/it]
 36%|███▌      | 80762/225000 [154:34:38<207:40:00,  5.18s/it]
 36%|███▌      | 80763/225000 [154:34:42<193:11:37,  4.82s/it]
 36%|███▌      | 80764/225000 [154:34:44<162:51:27,  4.06s/it]
 36%|███▌      | 80765/225000 [154:34:47<151:22:31,  3.78s/it]
 36%|███▌      | 80766/225000 [154:34:50<144:22:32,  3.60s/it]
 36%|███▌      | 80767/225000 [154:34:53<132:03:32,  3.30s/it]
 36%|███▌      | 80768/225000 [154:34:56<128:16:39,  3.20s/it]
 36%|███▌      | 80769/225000 [154:35:04<183:14:48,  4.57s/it]
 36%|███▌      | 80770/225000 [154:35:10<210:31:54,  5.25s/it]
 36%|███▌      | 80771/225000 [154:35:13<175:57:40,  4.39s/it]
 36%|███▌      | 80772/225000 [154:35:15<150:47:37,  3.76s/it]
 36%|███▌      | 80773/225000 [154:35:22<194:26:57,  4.85s/it]
 36%|███▌      | 80774/225000 [154:35:25<166:35:00,  4.16s/it]
 36%|███▌      | 80775/225000 [154:35:28<150:04:24,  3.75s/it]
 36%|███▌      | 80776/225000 [154:35:31<139:04:24,  3.47s/it]
 36%|███▌      | 80777/225000 [154:35:34<138:57:50,  3.47s/it]
 36%|███▌      | 80778/225000 [154:35:42<190:26:55,  4.75s/it]
 36%|███▌      | 80779/225000 [154:35:45<166:42:19,  4.16s/it]
 36%|███▌      | 80780/225000 [154:35:48<158:10:33,  3.95s/it]
 36%|███▌      | 80781/225000 [154:35:51<146:04:57,  3.65s/it]
 36%|███▌      | 80782/225000 [154:35:54<137:51:40,  3.44s/it]
 36%|███▌      | 80783/225000 [154:35:57<134:23:45,  3.35s/it]
 36%|███▌      | 80784/225000 [154:36:00<131:42:40,  3.29s/it]
 36%|███▌      | 80785/225000 [154:36:03<128:50:50,  3.22s/it]
 36%|███▌      | 80786/225000 [154:36:06<124:00:26,  3.10s/it]
 36%|███▌      | 80787/225000 [154:36:10<130:05:19,  3.25s/it]
 36%|███▌      | 80788/225000 [154:36:13<126:26:14,  3.16s/it]
 36%|███▌      | 80789/225000 [154:36:16<123:05:20,  3.07s/it]
 36%|███▌      | 80790/225000 [154:36:22<165:41:40,  4.14s/it]
 36%|███▌      | 80791/225000 [154:36:30<213:32:28,  5.33s/it]
 36%|███▌      | 80792/225000 [154:36:34<195:50:11,  4.89s/it]
 36%|███▌      | 80793/225000 [154:36:37<166:52:14,  4.17s/it]
 36%|███▌      | 80794/225000 [154:36:40<157:23:09,  3.93s/it]
 36%|███▌      | 80795/225000 [154:36:43<149:56:21,  3.74s/it]
 36%|███▌      | 80796/225000 [154:36:46<143:07:36,  3.57s/it]
 36%|███▌      | 80797/225000 [154:36:50<136:37:06,  3.41s/it]
 36%|███▌      | 80798/225000 [154:36:53<141:11:02,  3.52s/it]
 36%|███▌      | 80799/225000 [154:36:58<156:52:57,  3.92s/it]
 36%|███▌      | 80800/225000 [154:37:07<220:53:39,  5.51s/it]
                                                              
{'loss': '0.731', 'grad_norm': '6.32', 'learning_rate': '3.779e-05', 'epoch': '0.3591', 'num_input_tokens_seen': 254342048, 'train_runtime': '5.566e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 80800/225000 [154:37:07<220:53:39,  5.51s/it]
 36%|███▌      | 80801/225000 [154:37:11<201:56:13,  5.04s/it]
 36%|███▌      | 80802/225000 [154:37:14<176:41:31,  4.41s/it]
 36%|███▌      | 80803/225000 [154:37:17<158:48:53,  3.96s/it]
 36%|███▌      | 80804/225000 [154:37:20<145:12:15,  3.63s/it]
 36%|███▌      | 80805/225000 [154:37:23<134:04:11,  3.35s/it]
 36%|███▌      | 80806/225000 [154:37:25<125:41:54,  3.14s/it]
 36%|███▌      | 80807/225000 [154:37:32<162:30:05,  4.06s/it]
 36%|███▌      | 80808/225000 [154:37:38<192:50:50,  4.81s/it]
 36%|███▌      | 80809/225000 [154:37:41<166:09:41,  4.15s/it]
 36%|███▌      | 80810/225000 [154:37:44<156:06:53,  3.90s/it]
 36%|███▌      | 80811/225000 [154:37:47<144:26:23,  3.61s/it]
 36%|███▌      | 80812/225000 [154:37:50<133:54:57,  3.34s/it]
 36%|███▌      | 80813/225000 [154:37:57<177:54:47,  4.44s/it]
 36%|███▌      | 80814/225000 [154:38:05<224:20:15,  5.60s/it]
 36%|███▌      | 80815/225000 [154:38:07<186:41:03,  4.66s/it]
 36%|███▌      | 80816/225000 [154:38:10<161:57:08,  4.04s/it]
 36%|███▌      | 80817/225000 [154:38:12<142:05:25,  3.55s/it]
 36%|███▌      | 80818/225000 [154:38:15<127:26:23,  3.18s/it]
 36%|███▌      | 80819/225000 [154:38:23<185:10:11,  4.62s/it]
 36%|███▌      | 80820/225000 [154:38:27<179:48:16,  4.49s/it]
 36%|███▌      | 80821/225000 [154:38:30<159:51:36,  3.99s/it]
 36%|███▌      | 80822/225000 [154:38:33<150:33:23,  3.76s/it]
 36%|███▌      | 80823/225000 [154:38:36<139:24:46,  3.48s/it]
 36%|███▌      | 80824/225000 [154:38:39<135:38:48,  3.39s/it]
 36%|███▌      | 80825/225000 [154:38:42<131:10:37,  3.28s/it]
 36%|███▌      | 80826/225000 [154:38:45<130:43:15,  3.26s/it]
 36%|███▌      | 80827/225000 [154:38:48<126:51:41,  3.17s/it]
 36%|███▌      | 80828/225000 [154:38:51<126:44:25,  3.16s/it]
 36%|███▌      | 80829/225000 [154:38:54<123:45:36,  3.09s/it]
 36%|███▌      | 80830/225000 [154:38:58<131:13:12,  3.28s/it]
 36%|███▌      | 80831/225000 [154:39:02<133:58:56,  3.35s/it]
 36%|███▌      | 80832/225000 [154:39:04<129:30:34,  3.23s/it]
 36%|███▌      | 80833/225000 [154:39:12<184:11:26,  4.60s/it]
 36%|███▌      | 80834/225000 [154:39:20<221:27:11,  5.53s/it]
 36%|███▌      | 80835/225000 [154:39:22<184:21:10,  4.60s/it]
 36%|███▌      | 80836/225000 [154:39:25<157:41:50,  3.94s/it]
 36%|███▌      | 80837/225000 [154:39:27<141:38:16,  3.54s/it]
 36%|███▌      | 80838/225000 [154:39:35<188:59:57,  4.72s/it]
 36%|███▌      | 80839/225000 [154:39:39<184:59:23,  4.62s/it]
 36%|███▌      | 80840/225000 [154:39:42<164:18:29,  4.10s/it]
 36%|███▌      | 80841/225000 [154:39:45<148:47:34,  3.72s/it]
 36%|███▌      | 80842/225000 [154:39:48<135:40:54,  3.39s/it]
 36%|███▌      | 80843/225000 [154:39:50<125:52:53,  3.14s/it]
 36%|███▌      | 80844/225000 [154:39:53<119:02:08,  2.97s/it]
 36%|███▌      | 80845/225000 [154:39:58<150:21:48,  3.76s/it]
 36%|███▌      | 80846/225000 [154:40:04<175:30:08,  4.38s/it]
 36%|███▌      | 80847/225000 [154:40:07<150:51:18,  3.77s/it]
 36%|███▌      | 80848/225000 [154:40:12<171:11:27,  4.28s/it]
 36%|███▌      | 80849/225000 [154:40:14<148:20:31,  3.70s/it]
 36%|███▌      | 80850/225000 [154:40:17<132:15:32,  3.30s/it]
                                                              
{'loss': '0.7456', 'grad_norm': '4.3', 'learning_rate': '3.777e-05', 'epoch': '0.3593', 'num_input_tokens_seen': 254424200, 'train_runtime': '5.568e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 80850/225000 [154:40:17<132:15:32,  3.30s/it]
 36%|███▌      | 80851/225000 [154:40:24<180:26:22,  4.51s/it]
 36%|███▌      | 80852/225000 [154:40:29<182:50:15,  4.57s/it]
 36%|███▌      | 80853/225000 [154:40:31<160:35:51,  4.01s/it]
 36%|███▌      | 80854/225000 [154:40:34<141:36:41,  3.54s/it]
 36%|███▌      | 80855/225000 [154:40:36<128:08:35,  3.20s/it]
 36%|███▌      | 80856/225000 [154:40:39<128:01:09,  3.20s/it]
 36%|███▌      | 80857/225000 [154:40:43<134:22:32,  3.36s/it]
 36%|███▌      | 80858/225000 [154:40:47<134:47:55,  3.37s/it]
 36%|███▌      | 80859/225000 [154:40:50<138:00:41,  3.45s/it]
 36%|███▌      | 80860/225000 [154:40:54<138:27:16,  3.46s/it]
 36%|███▌      | 80861/225000 [154:40:57<132:59:02,  3.32s/it]
 36%|███▌      | 80862/225000 [154:41:00<136:59:56,  3.42s/it]
 36%|███▌      | 80863/225000 [154:41:04<138:02:20,  3.45s/it]
 36%|███▌      | 80864/225000 [154:41:10<174:36:32,  4.36s/it]
 36%|███▌      | 80865/225000 [154:41:19<225:40:09,  5.64s/it]
 36%|███▌      | 80866/225000 [154:41:24<217:33:00,  5.43s/it]
 36%|███▌      | 80867/225000 [154:41:27<185:33:39,  4.63s/it]
 36%|███▌      | 80868/225000 [154:41:29<163:05:51,  4.07s/it]
 36%|███▌      | 80869/225000 [154:41:32<149:42:23,  3.74s/it]
 36%|███▌      | 80870/225000 [154:41:36<142:41:18,  3.56s/it]
 36%|███▌      | 80871/225000 [154:41:39<138:45:27,  3.47s/it]
 36%|███▌      | 80872/225000 [154:41:43<152:49:46,  3.82s/it]
 36%|███▌      | 80873/225000 [154:41:51<196:50:40,  4.92s/it]
 36%|███▌      | 80874/225000 [154:41:53<165:26:56,  4.13s/it]
 36%|███▌      | 80875/225000 [154:41:56<146:14:36,  3.65s/it]
 36%|███▌      | 80876/225000 [154:41:58<134:33:43,  3.36s/it]
 36%|███▌      | 80877/225000 [154:42:05<176:49:13,  4.42s/it]
 36%|███▌      | 80878/225000 [154:42:09<173:29:47,  4.33s/it]
 36%|███▌      | 80879/225000 [154:42:12<154:15:24,  3.85s/it]
 36%|███▌      | 80880/225000 [154:42:16<152:22:21,  3.81s/it]
 36%|███▌      | 80881/225000 [154:42:19<142:32:49,  3.56s/it]
 36%|███▌      | 80882/225000 [154:42:22<140:08:41,  3.50s/it]
 36%|███▌      | 80883/225000 [154:42:25<134:17:16,  3.35s/it]
 36%|███▌      | 80884/225000 [154:42:28<130:21:10,  3.26s/it]
 36%|███▌      | 80885/225000 [154:42:31<127:51:52,  3.19s/it]
 36%|███▌      | 80886/225000 [154:42:34<125:56:38,  3.15s/it]
 36%|███▌      | 80887/225000 [154:42:38<131:57:28,  3.30s/it]
 36%|███▌      | 80888/225000 [154:42:42<135:04:53,  3.37s/it]
 36%|███▌      | 80889/225000 [154:42:45<132:58:08,  3.32s/it]
 36%|███▌      | 80890/225000 [154:42:48<128:19:00,  3.21s/it]
 36%|███▌      | 80891/225000 [154:42:51<126:41:26,  3.16s/it]
 36%|███▌      | 80892/225000 [154:42:59<185:56:15,  4.64s/it]
 36%|███▌      | 80893/225000 [154:43:06<220:45:25,  5.51s/it]
 36%|███▌      | 80894/225000 [154:43:09<185:08:26,  4.63s/it]
 36%|███▌      | 80895/225000 [154:43:11<159:40:30,  3.99s/it]
 36%|███▌      | 80896/225000 [154:43:14<145:59:53,  3.65s/it]
 36%|███▌      | 80897/225000 [154:43:17<134:38:28,  3.36s/it]
 36%|███▌      | 80898/225000 [154:43:20<128:25:00,  3.21s/it]
 36%|███▌      | 80899/225000 [154:43:26<167:07:42,  4.18s/it]
 36%|███▌      | 80900/225000 [154:43:34<208:43:39,  5.21s/it]
                                                              
{'loss': '0.6601', 'grad_norm': '4.638', 'learning_rate': '3.775e-05', 'epoch': '0.3596', 'num_input_tokens_seen': 254506272, 'train_runtime': '5.57e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 80900/225000 [154:43:34<208:43:39,  5.21s/it]
 36%|███▌      | 80901/225000 [154:43:37<181:23:18,  4.53s/it]
 36%|███▌      | 80902/225000 [154:43:40<168:26:10,  4.21s/it]
 36%|███▌      | 80903/225000 [154:43:44<157:22:02,  3.93s/it]
 36%|███▌      | 80904/225000 [154:43:47<145:23:46,  3.63s/it]
 36%|███▌      | 80905/225000 [154:43:50<139:01:42,  3.47s/it]
 36%|███▌      | 80906/225000 [154:43:53<132:40:15,  3.31s/it]
 36%|███▌      | 80907/225000 [154:43:56<128:34:24,  3.21s/it]
 36%|███▌      | 80908/225000 [154:43:58<123:39:09,  3.09s/it]
 36%|███▌      | 80909/225000 [154:44:02<124:41:57,  3.12s/it]
 36%|███▌      | 80910/225000 [154:44:05<125:12:03,  3.13s/it]
 36%|███▌      | 80911/225000 [154:44:10<152:29:51,  3.81s/it]
 36%|███▌      | 80912/225000 [154:44:19<207:16:04,  5.18s/it]
 36%|███▌      | 80913/225000 [154:44:22<181:57:48,  4.55s/it]
 36%|███▌      | 80914/225000 [154:44:24<154:30:53,  3.86s/it]
 36%|███▌      | 80915/225000 [154:44:26<136:18:57,  3.41s/it]
 36%|███▌      | 80916/225000 [154:44:34<187:17:35,  4.68s/it]
 36%|███▌      | 80917/225000 [154:44:38<179:52:50,  4.49s/it]
 36%|███▌      | 80918/225000 [154:44:40<153:52:42,  3.84s/it]
 36%|███▌      | 80919/225000 [154:44:44<147:42:04,  3.69s/it]
 36%|███▌      | 80920/225000 [154:44:47<139:27:30,  3.48s/it]
 36%|███▌      | 80921/225000 [154:44:50<138:03:43,  3.45s/it]
 36%|███▌      | 80922/225000 [154:44:53<133:14:38,  3.33s/it]
 36%|███▌      | 80923/225000 [154:44:57<137:28:03,  3.43s/it]
 36%|███▌      | 80924/225000 [154:45:00<136:59:14,  3.42s/it]
 36%|███▌      | 80925/225000 [154:45:04<138:58:28,  3.47s/it]
 36%|███▌      | 80926/225000 [154:45:10<174:26:40,  4.36s/it]
 36%|███▌      | 80927/225000 [154:45:13<158:47:22,  3.97s/it]
 36%|███▌      | 80928/225000 [154:45:16<151:17:16,  3.78s/it]
 36%|███▌      | 80929/225000 [154:45:20<145:13:22,  3.63s/it]
 36%|███▌      | 80930/225000 [154:45:23<139:16:18,  3.48s/it]
 36%|███▌      | 80931/225000 [154:45:27<141:12:17,  3.53s/it]
 36%|███▌      | 80932/225000 [154:45:30<138:01:32,  3.45s/it]
 36%|███▌      | 80933/225000 [154:45:34<144:00:25,  3.60s/it]
 36%|███▌      | 80934/225000 [154:45:37<139:41:45,  3.49s/it]
 36%|███▌      | 80935/225000 [154:45:42<156:36:11,  3.91s/it]
 36%|███▌      | 80936/225000 [154:45:50<208:32:50,  5.21s/it]
 36%|███▌      | 80937/225000 [154:45:55<205:55:14,  5.15s/it]
 36%|███▌      | 80938/225000 [154:45:58<183:11:41,  4.58s/it]
 36%|███▌      | 80939/225000 [154:46:01<157:01:32,  3.92s/it]
 36%|███▌      | 80940/225000 [154:46:07<188:11:49,  4.70s/it]
 36%|███▌      | 80941/225000 [154:46:11<174:49:43,  4.37s/it]
 36%|███▌      | 80942/225000 [154:46:14<163:06:49,  4.08s/it]
 36%|███▌      | 80943/225000 [154:46:17<151:05:45,  3.78s/it]
 36%|███▌      | 80944/225000 [154:46:20<139:36:03,  3.49s/it]
 36%|███▌      | 80945/225000 [154:46:24<140:16:23,  3.51s/it]
 36%|███▌      | 80946/225000 [154:46:27<137:26:57,  3.43s/it]
 36%|███▌      | 80947/225000 [154:46:30<130:16:28,  3.26s/it]
 36%|███▌      | 80948/225000 [154:46:37<179:43:43,  4.49s/it]
 36%|███▌      | 80949/225000 [154:46:44<203:58:37,  5.10s/it]
 36%|███▌      | 80950/225000 [154:46:46<175:23:09,  4.38s/it]
                                                              
{'loss': '0.7161', 'grad_norm': '6.503', 'learning_rate': '3.774e-05', 'epoch': '0.3598', 'num_input_tokens_seen': 254588672, 'train_runtime': '5.572e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 80950/225000 [154:46:46<175:23:09,  4.38s/it]
 36%|███▌      | 80951/225000 [154:46:49<155:17:06,  3.88s/it]
 36%|███▌      | 80952/225000 [154:46:53<154:14:31,  3.85s/it]
 36%|███▌      | 80953/225000 [154:47:01<199:57:46,  5.00s/it]
 36%|███▌      | 80954/225000 [154:47:03<170:26:10,  4.26s/it]
 36%|███▌      | 80955/225000 [154:47:10<198:40:54,  4.97s/it]
 36%|███▌      | 80956/225000 [154:47:12<170:23:20,  4.26s/it]
 36%|███▌      | 80957/225000 [154:47:15<152:27:52,  3.81s/it]
 36%|███▌      | 80958/225000 [154:47:18<139:33:54,  3.49s/it]
 36%|███▌      | 80959/225000 [154:47:20<129:27:40,  3.24s/it]
 36%|███▌      | 80960/225000 [154:47:23<122:22:04,  3.06s/it]
 36%|███▌      | 80961/225000 [154:47:29<153:26:54,  3.84s/it]
 36%|███▌      | 80962/225000 [154:47:37<207:31:39,  5.19s/it]
 36%|███▌      | 80963/225000 [154:47:40<176:36:50,  4.41s/it]
 36%|███▌      | 80964/225000 [154:47:42<151:14:35,  3.78s/it]
 36%|███▌      | 80965/225000 [154:47:45<136:26:17,  3.41s/it]
 36%|███▌      | 80966/225000 [154:47:52<185:19:14,  4.63s/it]
 36%|███▌      | 80967/225000 [154:47:57<185:02:09,  4.62s/it]
 36%|███▌      | 80968/225000 [154:47:59<158:31:50,  3.96s/it]
 36%|███▌      | 80969/225000 [154:48:03<159:51:30,  4.00s/it]
 36%|███▌      | 80970/225000 [154:48:09<176:22:09,  4.41s/it]
 36%|███▌      | 80971/225000 [154:48:11<156:11:40,  3.90s/it]
 36%|███▌      | 80972/225000 [154:48:14<141:39:48,  3.54s/it]
 36%|███▌      | 80973/225000 [154:48:17<132:06:24,  3.30s/it]
 36%|███▌      | 80974/225000 [154:48:19<121:32:26,  3.04s/it]
 36%|███▌      | 80975/225000 [154:48:28<188:34:13,  4.71s/it]
 36%|███▌      | 80976/225000 [154:48:32<185:24:32,  4.63s/it]
 36%|███▌      | 80977/225000 [154:48:35<166:04:24,  4.15s/it]
 36%|███▌      | 80978/225000 [154:48:40<175:33:18,  4.39s/it]
 36%|███▌      | 80979/225000 [154:48:45<184:34:23,  4.61s/it]
 36%|███▌      | 80980/225000 [154:48:48<162:53:57,  4.07s/it]
 36%|███▌      | 80981/225000 [154:48:51<150:40:20,  3.77s/it]
 36%|███▌      | 80982/225000 [154:48:54<143:04:40,  3.58s/it]
 36%|███▌      | 80983/225000 [154:48:58<140:51:58,  3.52s/it]
 36%|███▌      | 80984/225000 [154:49:01<133:58:25,  3.35s/it]
 36%|███▌      | 80985/225000 [154:49:04<133:59:22,  3.35s/it]
 36%|███▌      | 80986/225000 [154:49:07<133:27:03,  3.34s/it]
 36%|███▌      | 80987/225000 [154:49:11<134:39:14,  3.37s/it]
 36%|███▌      | 80988/225000 [154:49:18<184:10:37,  4.60s/it]
 36%|███▌      | 80989/225000 [154:49:25<207:43:31,  5.19s/it]
 36%|███▌      | 80990/225000 [154:49:28<178:47:57,  4.47s/it]
 36%|███▌      | 80991/225000 [154:49:30<157:54:00,  3.95s/it]
 36%|███▌      | 80992/225000 [154:49:36<175:49:15,  4.40s/it]
 36%|███▌      | 80993/225000 [154:49:41<182:00:44,  4.55s/it]
 36%|███▌      | 80994/225000 [154:49:44<165:13:30,  4.13s/it]
 36%|███▌      | 80995/225000 [154:49:47<150:00:44,  3.75s/it]
 36%|███▌      | 80996/225000 [154:49:49<136:15:22,  3.41s/it]
 36%|███▌      | 80997/225000 [154:49:52<127:22:24,  3.18s/it]
 36%|███▌      | 80998/225000 [154:49:55<123:15:43,  3.08s/it]
 36%|███▌      | 80999/225000 [154:50:02<173:05:30,  4.33s/it]
 36%|███▌      | 81000/225000 [154:50:05<160:24:32,  4.01s/it]
                                                              
{'loss': '0.6841', 'grad_norm': '6.048', 'learning_rate': '3.772e-05', 'epoch': '0.36', 'num_input_tokens_seen': 254673136, 'train_runtime': '5.574e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 81000/225000 [154:50:05<160:24:32,  4.01s/it]
 36%|███▌      | 81001/225000 [154:50:09<156:48:19,  3.92s/it]
 36%|███▌      | 81002/225000 [154:50:13<154:32:42,  3.86s/it]
 36%|███▌      | 81003/225000 [154:50:16<143:19:39,  3.58s/it]
 36%|███▌      | 81004/225000 [154:50:19<143:49:39,  3.60s/it]
 36%|███▌      | 81005/225000 [154:50:23<141:06:54,  3.53s/it]
 36%|███▌      | 81006/225000 [154:50:29<170:09:26,  4.25s/it]
 36%|███▌      | 81007/225000 [154:50:33<167:55:46,  4.20s/it]
 36%|███▌      | 81008/225000 [154:50:37<174:10:25,  4.35s/it]
 36%|███▌      | 81009/225000 [154:50:41<164:56:18,  4.12s/it]
 36%|███▌      | 81010/225000 [154:50:44<155:42:34,  3.89s/it]
 36%|███▌      | 81011/225000 [154:50:48<149:12:06,  3.73s/it]
 36%|███▌      | 81012/225000 [154:50:51<141:30:18,  3.54s/it]
 36%|███▌      | 81013/225000 [154:50:54<135:03:58,  3.38s/it]
 36%|███▌      | 81014/225000 [154:50:57<137:42:23,  3.44s/it]
 36%|███▌      | 81015/225000 [154:51:01<136:47:16,  3.42s/it]
 36%|███▌      | 81016/225000 [154:51:04<138:31:00,  3.46s/it]
 36%|███▌      | 81017/225000 [154:51:07<133:52:15,  3.35s/it]
 36%|███▌      | 81018/225000 [154:51:10<130:27:15,  3.26s/it]
 36%|███▌      | 81019/225000 [154:51:14<139:32:45,  3.49s/it]
 36%|███▌      | 81020/225000 [154:51:23<199:52:38,  5.00s/it]
 36%|███▌      | 81021/225000 [154:51:31<238:38:57,  5.97s/it]
 36%|███▌      | 81022/225000 [154:51:34<197:10:04,  4.93s/it]
 36%|███▌      | 81023/225000 [154:51:38<192:44:13,  4.82s/it]
 36%|███▌      | 81024/225000 [154:51:43<193:01:42,  4.83s/it]
 36%|███▌      | 81025/225000 [154:51:45<162:51:20,  4.07s/it]
 36%|███▌      | 81026/225000 [154:51:48<141:16:41,  3.53s/it]
 36%|███▌      | 81027/225000 [154:51:53<159:25:17,  3.99s/it]
 36%|███▌      | 81028/225000 [154:51:59<184:26:01,  4.61s/it]
 36%|███▌      | 81029/225000 [154:52:02<166:52:04,  4.17s/it]
 36%|███▌      | 81030/225000 [154:52:05<150:48:17,  3.77s/it]
 36%|███▌      | 81031/225000 [154:52:08<142:01:19,  3.55s/it]
 36%|███▌      | 81032/225000 [154:52:11<132:32:01,  3.31s/it]
 36%|███▌      | 81033/225000 [154:52:13<126:23:39,  3.16s/it]
 36%|███▌      | 81034/225000 [154:52:16<124:57:17,  3.12s/it]
 36%|███▌      | 81035/225000 [154:52:19<123:53:54,  3.10s/it]
 36%|███▌      | 81036/225000 [154:52:28<191:18:34,  4.78s/it]
 36%|███▌      | 81037/225000 [154:52:32<182:20:53,  4.56s/it]
 36%|███▌      | 81038/225000 [154:52:35<159:23:38,  3.99s/it]
 36%|███▌      | 81039/225000 [154:52:37<141:29:03,  3.54s/it]
 36%|███▌      | 81040/225000 [154:52:44<176:48:57,  4.42s/it]
 36%|███▌      | 81041/225000 [154:52:49<184:01:01,  4.60s/it]
 36%|███▌      | 81042/225000 [154:52:53<173:47:45,  4.35s/it]
 36%|███▌      | 81043/225000 [154:52:55<156:07:27,  3.90s/it]
 36%|███▌      | 81044/225000 [154:52:59<148:30:30,  3.71s/it]
 36%|███▌      | 81045/225000 [154:53:02<141:41:25,  3.54s/it]
 36%|███▌      | 81046/225000 [154:53:05<135:42:08,  3.39s/it]
 36%|███▌      | 81047/225000 [154:53:08<130:58:27,  3.28s/it]
 36%|███▌      | 81048/225000 [154:53:11<128:34:21,  3.22s/it]
 36%|███▌      | 81049/225000 [154:53:18<179:41:31,  4.49s/it]
 36%|███▌      | 81050/225000 [154:53:25<206:11:38,  5.16s/it]
                                                              
{'loss': '0.7327', 'grad_norm': '3.01', 'learning_rate': '3.771e-05', 'epoch': '0.3602', 'num_input_tokens_seen': 254756528, 'train_runtime': '5.576e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 81050/225000 [154:53:25<206:11:38,  5.16s/it]
 36%|███▌      | 81051/225000 [154:53:27<170:59:34,  4.28s/it]
 36%|███▌      | 81052/225000 [154:53:30<149:05:22,  3.73s/it]
 36%|███▌      | 81053/225000 [154:53:33<139:48:37,  3.50s/it]
 36%|███▌      | 81054/225000 [154:53:36<133:24:54,  3.34s/it]
 36%|███▌      | 81055/225000 [154:53:39<126:09:35,  3.16s/it]
 36%|███▌      | 81056/225000 [154:53:45<161:36:36,  4.04s/it]
 36%|███▌      | 81057/225000 [154:53:52<200:50:14,  5.02s/it]
 36%|███▌      | 81058/225000 [154:53:54<168:59:08,  4.23s/it]
 36%|███▌      | 81059/225000 [154:53:57<151:22:07,  3.79s/it]
 36%|███▌      | 81060/225000 [154:54:00<138:23:12,  3.46s/it]
 36%|███▌      | 81061/225000 [154:54:03<133:23:03,  3.34s/it]
 36%|███▌      | 81062/225000 [154:54:06<133:41:14,  3.34s/it]
 36%|███▌      | 81063/225000 [154:54:10<137:57:36,  3.45s/it]
 36%|███▌      | 81064/225000 [154:54:18<198:08:48,  4.96s/it]
 36%|███▌      | 81065/225000 [154:54:22<187:42:37,  4.69s/it]
 36%|███▌      | 81066/225000 [154:54:25<162:24:45,  4.06s/it]
 36%|███▌      | 81067/225000 [154:54:28<151:11:24,  3.78s/it]
 36%|███▌      | 81068/225000 [154:54:31<144:20:59,  3.61s/it]
 36%|███▌      | 81069/225000 [154:54:34<138:23:14,  3.46s/it]
 36%|███▌      | 81070/225000 [154:54:38<133:38:09,  3.34s/it]
 36%|███▌      | 81071/225000 [154:54:41<129:30:36,  3.24s/it]
 36%|███▌      | 81072/225000 [154:54:44<132:38:10,  3.32s/it]
 36%|███▌      | 81073/225000 [154:54:47<128:05:30,  3.20s/it]
 36%|███▌      | 81074/225000 [154:54:51<132:20:01,  3.31s/it]
 36%|███▌      | 81075/225000 [154:54:58<183:36:59,  4.59s/it]
 36%|███▌      | 81076/225000 [154:55:03<190:38:23,  4.77s/it]
 36%|███▌      | 81077/225000 [154:55:07<179:48:09,  4.50s/it]
 36%|███▌      | 81078/225000 [154:55:11<172:16:45,  4.31s/it]
 36%|███▌      | 81079/225000 [154:55:14<161:15:15,  4.03s/it]
 36%|███▌      | 81080/225000 [154:55:18<150:30:48,  3.76s/it]
 36%|███▌      | 81081/225000 [154:55:21<151:11:31,  3.78s/it]
 36%|███▌      | 81082/225000 [154:55:25<147:48:38,  3.70s/it]
 36%|███▌      | 81083/225000 [154:55:28<145:28:37,  3.64s/it]
 36%|███▌      | 81084/225000 [154:55:33<161:27:49,  4.04s/it]
 36%|███▌      | 81085/225000 [154:55:37<151:38:30,  3.79s/it]
 36%|███▌      | 81086/225000 [154:55:44<195:04:11,  4.88s/it]
 36%|███▌      | 81087/225000 [154:55:52<231:46:50,  5.80s/it]
 36%|███▌      | 81088/225000 [154:55:54<191:46:17,  4.80s/it]
 36%|███▌      | 81089/225000 [154:55:57<164:24:24,  4.11s/it]
 36%|███▌      | 81090/225000 [154:56:00<146:32:51,  3.67s/it]
 36%|███▌      | 81091/225000 [154:56:02<136:07:33,  3.41s/it]
 36%|███▌      | 81092/225000 [154:56:11<194:40:10,  4.87s/it]
 36%|███▌      | 81093/225000 [154:56:14<179:09:42,  4.48s/it]
 36%|███▌      | 81094/225000 [154:56:17<160:17:36,  4.01s/it]
 36%|███▌      | 81095/225000 [154:56:20<146:29:45,  3.66s/it]
 36%|███▌      | 81096/225000 [154:56:23<137:55:37,  3.45s/it]
 36%|███▌      | 81097/225000 [154:56:26<136:33:38,  3.42s/it]
 36%|███▌      | 81098/225000 [154:56:33<175:47:59,  4.40s/it]
 36%|███▌      | 81099/225000 [154:56:37<175:19:24,  4.39s/it]
 36%|███▌      | 81100/225000 [154:56:41<168:53:49,  4.23s/it]
                                                              
{'loss': '0.7143', 'grad_norm': '4.73', 'learning_rate': '3.769e-05', 'epoch': '0.3604', 'num_input_tokens_seen': 254840712, 'train_runtime': '5.578e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 81100/225000 [154:56:41<168:53:49,  4.23s/it]
 36%|███▌      | 81101/225000 [154:56:47<185:32:59,  4.64s/it]
 36%|███▌      | 81102/225000 [154:56:50<169:36:06,  4.24s/it]
 36%|███▌      | 81103/225000 [154:56:53<152:41:21,  3.82s/it]
 36%|███▌      | 81104/225000 [154:56:56<145:04:36,  3.63s/it]
 36%|███▌      | 81105/225000 [154:56:59<136:01:51,  3.40s/it]
 36%|███▌      | 81106/225000 [154:57:02<136:05:53,  3.40s/it]
 36%|███▌      | 81107/225000 [154:57:05<131:33:41,  3.29s/it]
 36%|███▌      | 81108/225000 [154:57:08<127:55:40,  3.20s/it]
 36%|███▌      | 81109/225000 [154:57:14<150:59:16,  3.78s/it]
 36%|███▌      | 81110/225000 [154:57:23<218:30:48,  5.47s/it]
 36%|███▌      | 81111/225000 [154:57:25<183:56:04,  4.60s/it]
 36%|███▌      | 81112/225000 [154:57:28<158:42:08,  3.97s/it]
 36%|███▌      | 81113/225000 [154:57:31<147:44:11,  3.70s/it]
 36%|███▌      | 81114/225000 [154:57:39<201:16:52,  5.04s/it]
 36%|███▌      | 81115/225000 [154:57:43<184:37:54,  4.62s/it]
 36%|███▌      | 81116/225000 [154:57:45<157:22:24,  3.94s/it]
 36%|███▌      | 81117/225000 [154:57:48<139:13:25,  3.48s/it]
 36%|███▌      | 81118/225000 [154:57:51<134:08:44,  3.36s/it]
 36%|███▌      | 81119/225000 [154:57:54<131:27:29,  3.29s/it]
 36%|███▌      | 81120/225000 [154:57:58<145:37:06,  3.64s/it]
 36%|███▌      | 81121/225000 [154:58:03<154:42:04,  3.87s/it]
 36%|███▌      | 81122/225000 [154:58:05<141:01:01,  3.53s/it]
 36%|███▌      | 81123/225000 [154:58:08<131:38:21,  3.29s/it]
 36%|███▌      | 81124/225000 [154:58:11<125:59:50,  3.15s/it]
 36%|███▌      | 81125/225000 [154:58:18<172:52:44,  4.33s/it]
 36%|███▌      | 81126/225000 [154:58:23<182:34:22,  4.57s/it]
 36%|███▌      | 81127/225000 [154:58:26<161:11:00,  4.03s/it]
 36%|███▌      | 81128/225000 [154:58:29<149:16:00,  3.73s/it]
 36%|███▌      | 81129/225000 [154:58:32<141:37:33,  3.54s/it]
 36%|███▌      | 81130/225000 [154:58:35<130:58:11,  3.28s/it]
 36%|███▌      | 81131/225000 [154:58:38<125:39:37,  3.14s/it]
 36%|███▌      | 81132/225000 [154:58:43<154:16:07,  3.86s/it]
 36%|███▌      | 81133/225000 [154:58:51<198:39:05,  4.97s/it]
 36%|███▌      | 81134/225000 [154:58:53<168:22:48,  4.21s/it]
 36%|███▌      | 81135/225000 [154:58:56<149:18:47,  3.74s/it]
 36%|███▌      | 81136/225000 [154:58:59<139:59:51,  3.50s/it]
 36%|███▌      | 81137/225000 [154:59:01<131:18:27,  3.29s/it]
 36%|███▌      | 81138/225000 [154:59:05<137:48:16,  3.45s/it]
 36%|███▌      | 81139/225000 [154:59:09<139:37:29,  3.49s/it]
 36%|███▌      | 81140/225000 [154:59:14<164:01:03,  4.10s/it]
 36%|███▌      | 81141/225000 [154:59:18<160:06:34,  4.01s/it]
 36%|███▌      | 81142/225000 [154:59:22<154:17:22,  3.86s/it]
 36%|███▌      | 81143/225000 [154:59:25<146:25:19,  3.66s/it]
 36%|███▌      | 81144/225000 [154:59:28<141:08:20,  3.53s/it]
 36%|███▌      | 81145/225000 [154:59:31<138:21:59,  3.46s/it]
 36%|███▌      | 81146/225000 [154:59:35<136:04:11,  3.41s/it]
 36%|███▌      | 81147/225000 [154:59:38<138:28:57,  3.47s/it]
 36%|███▌      | 81148/225000 [154:59:42<136:46:37,  3.42s/it]
 36%|███▌      | 81149/225000 [154:59:45<135:39:51,  3.40s/it]
 36%|███▌      | 81150/225000 [154:59:49<146:39:41,  3.67s/it]
                                                              
{'loss': '0.7139', 'grad_norm': '2.562', 'learning_rate': '3.768e-05', 'epoch': '0.3607', 'num_input_tokens_seen': 254923864, 'train_runtime': '5.58e+05', 'train_tokens_per_second': '456.9'}

 36%|███▌      | 81150/225000 [154:59:49<146:39:41,  3.67s/it]
 36%|███▌      | 81151/225000 [154:59:58<210:25:42,  5.27s/it]
 36%|███▌      | 81152/225000 [155:00:05<221:23:29,  5.54s/it]
 36%|███▌      | 81153/225000 [155:00:07<186:42:25,  4.67s/it]
 36%|███▌      | 81154/225000 [155:00:10<169:07:10,  4.23s/it]
 36%|███▌      | 81155/225000 [155:00:13<149:19:13,  3.74s/it]
 36%|███▌      | 81156/225000 [155:00:16<140:39:26,  3.52s/it]
 36%|███▌      | 81157/225000 [155:00:19<131:07:53,  3.28s/it]
 36%|███▌      | 81158/225000 [155:00:28<203:21:29,  5.09s/it]
 36%|███▌      | 81159/225000 [155:00:33<199:53:45,  5.00s/it]
 36%|███▌      | 81160/225000 [155:00:35<166:57:39,  4.18s/it]
 36%|███▌      | 81161/225000 [155:00:38<151:08:56,  3.78s/it]
 36%|███▌      | 81162/225000 [155:00:41<142:48:11,  3.57s/it]
 36%|███▌      | 81163/225000 [155:00:44<135:49:56,  3.40s/it]
 36%|███▌      | 81164/225000 [155:00:47<131:00:27,  3.28s/it]
 36%|███▌      | 81165/225000 [155:00:50<132:31:17,  3.32s/it]
 36%|███▌      | 81166/225000 [155:00:53<128:29:32,  3.22s/it]
 36%|███▌      | 81167/225000 [155:00:56<126:27:21,  3.17s/it]
 36%|███▌      | 81168/225000 [155:00:59<123:41:49,  3.10s/it]
 36%|███▌      | 81169/225000 [155:01:03<130:58:11,  3.28s/it]
 36%|███▌      | 81170/225000 [155:01:09<167:22:15,  4.19s/it]
 36%|███▌      | 81171/225000 [155:01:18<224:40:27,  5.62s/it]
 36%|███▌      | 81172/225000 [155:01:22<196:17:16,  4.91s/it]
 36%|███▌      | 81173/225000 [155:01:24<168:42:44,  4.22s/it]
 36%|███▌      | 81174/225000 [155:01:27<152:06:08,  3.81s/it]
 36%|███▌      | 81175/225000 [155:01:30<138:19:15,  3.46s/it]
 36%|███▌      | 81176/225000 [155:01:32<128:37:05,  3.22s/it]
 36%|███▌      | 81177/225000 [155:01:35<122:52:50,  3.08s/it]
 36%|███▌      | 81178/225000 [155:01:42<169:33:06,  4.24s/it]
 36%|███▌      | 81179/225000 [155:01:49<198:22:28,  4.97s/it]
 36%|███▌      | 81180/225000 [155:01:52<172:24:24,  4.32s/it]
 36%|███▌      | 81181/225000 [155:01:55<156:52:33,  3.93s/it]
 36%|███▌      | 81182/225000 [155:01:57<141:06:50,  3.53s/it]
 36%|███▌      | 81183/225000 [155:02:02<161:17:24,  4.04s/it]
 36%|███▌      | 81184/225000 [155:02:11<212:57:12,  5.33s/it]
 36%|███▌      | 81185/225000 [155:02:13<179:46:23,  4.50s/it]
 36%|███▌      | 81186/225000 [155:02:16<161:50:11,  4.05s/it]
 36%|███▌      | 81187/225000 [155:02:19<150:04:09,  3.76s/it]
 36%|███▌      | 81188/225000 [155:02:22<139:43:38,  3.50s/it]
 36%|███▌      | 81189/225000 [155:02:25<129:23:05,  3.24s/it]
 36%|███▌      | 81190/225000 [155:02:32<171:41:38,  4.30s/it]
 36%|███▌      | 81191/225000 [155:02:39<203:46:13,  5.10s/it]
 36%|███▌      | 81192/225000 [155:02:41<173:38:01,  4.35s/it]
 36%|███▌      | 81193/225000 [155:02:48<202:08:46,  5.06s/it]
 36%|███▌      | 81194/225000 [155:02:51<176:00:58,  4.41s/it]
 36%|███▌      | 81195/225000 [155:02:53<152:46:07,  3.82s/it]
 36%|███▌      | 81196/225000 [155:02:56<138:55:21,  3.48s/it]
 36%|███▌      | 81197/225000 [155:02:59<132:10:22,  3.31s/it]
 36%|███▌      | 81198/225000 [155:03:02<132:17:17,  3.31s/it]
 36%|███▌      | 81199/225000 [155:03:10<181:51:34,  4.55s/it]
 36%|███▌      | 81200/225000 [155:03:16<201:27:35,  5.04s/it]
                                                              
{'loss': '0.7436', 'grad_norm': '3.746', 'learning_rate': '3.766e-05', 'epoch': '0.3609', 'num_input_tokens_seen': 255007448, 'train_runtime': '5.582e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81200/225000 [155:03:16<201:27:35,  5.04s/it]
 36%|███▌      | 81201/225000 [155:03:18<168:31:47,  4.22s/it]
 36%|███▌      | 81202/225000 [155:03:20<145:06:41,  3.63s/it]
 36%|███▌      | 81203/225000 [155:03:23<135:57:44,  3.40s/it]
 36%|███▌      | 81204/225000 [155:03:30<177:36:17,  4.45s/it]
 36%|███▌      | 81205/225000 [155:03:33<153:59:45,  3.86s/it]
 36%|███▌      | 81206/225000 [155:03:35<137:29:58,  3.44s/it]
 36%|███▌      | 81207/225000 [155:03:40<155:37:33,  3.90s/it]
 36%|███▌      | 81208/225000 [155:03:44<162:20:24,  4.06s/it]
 36%|███▌      | 81209/225000 [155:03:47<140:44:21,  3.52s/it]
 36%|███▌      | 81210/225000 [155:03:51<155:06:08,  3.88s/it]
 36%|███▌      | 81211/225000 [155:03:57<172:41:24,  4.32s/it]
 36%|███▌      | 81212/225000 [155:03:59<150:48:15,  3.78s/it]
 36%|███▌      | 81213/225000 [155:04:02<135:37:00,  3.40s/it]
 36%|███▌      | 81214/225000 [155:04:05<130:44:42,  3.27s/it]
 36%|███▌      | 81215/225000 [155:04:08<131:19:12,  3.29s/it]
 36%|███▌      | 81216/225000 [155:04:12<134:18:44,  3.36s/it]
 36%|███▌      | 81217/225000 [155:04:15<137:18:30,  3.44s/it]
 36%|███▌      | 81218/225000 [155:04:20<148:35:18,  3.72s/it]
 36%|███▌      | 81219/225000 [155:04:23<140:22:00,  3.51s/it]
 36%|███▌      | 81220/225000 [155:04:29<176:01:26,  4.41s/it]
 36%|███▌      | 81221/225000 [155:04:37<222:06:25,  5.56s/it]
 36%|███▌      | 81222/225000 [155:04:40<185:39:00,  4.65s/it]
 36%|███▌      | 81223/225000 [155:04:42<157:43:40,  3.95s/it]
 36%|███▌      | 81224/225000 [155:04:45<145:00:48,  3.63s/it]
 36%|███▌      | 81225/225000 [155:04:48<139:56:23,  3.50s/it]
 36%|███▌      | 81226/225000 [155:04:52<144:32:54,  3.62s/it]
 36%|███▌      | 81227/225000 [155:04:56<142:06:40,  3.56s/it]
 36%|███▌      | 81228/225000 [155:04:59<136:30:30,  3.42s/it]
 36%|███▌      | 81229/225000 [155:05:02<138:38:57,  3.47s/it]
 36%|███▌      | 81230/225000 [155:05:06<134:49:01,  3.38s/it]
 36%|███▌      | 81231/225000 [155:05:12<169:00:10,  4.23s/it]
 36%|███▌      | 81232/225000 [155:05:20<219:50:53,  5.51s/it]
 36%|███▌      | 81233/225000 [155:05:23<190:53:33,  4.78s/it]
 36%|███▌      | 81234/225000 [155:05:27<178:38:09,  4.47s/it]
 36%|███▌      | 81235/225000 [155:05:33<194:22:54,  4.87s/it]
 36%|███▌      | 81236/225000 [155:05:36<170:49:49,  4.28s/it]
 36%|███▌      | 81237/225000 [155:05:38<150:15:37,  3.76s/it]
 36%|███▌      | 81238/225000 [155:05:41<139:31:17,  3.49s/it]
 36%|███▌      | 81239/225000 [155:05:44<132:14:32,  3.31s/it]
 36%|███▌      | 81240/225000 [155:05:52<190:50:54,  4.78s/it]
 36%|███▌      | 81241/225000 [155:05:57<194:43:42,  4.88s/it]
 36%|███▌      | 81242/225000 [155:06:00<164:40:45,  4.12s/it]
 36%|███▌      | 81243/225000 [155:06:02<146:12:00,  3.66s/it]
 36%|███▌      | 81244/225000 [155:06:05<131:44:06,  3.30s/it]
 36%|███▌      | 81245/225000 [155:06:08<130:29:41,  3.27s/it]
 36%|███▌      | 81246/225000 [155:06:11<126:54:13,  3.18s/it]
 36%|███▌      | 81247/225000 [155:06:14<122:06:51,  3.06s/it]
 36%|███▌      | 81248/225000 [155:06:22<181:04:42,  4.53s/it]
 36%|███▌      | 81249/225000 [155:06:28<197:35:16,  4.95s/it]
 36%|███▌      | 81250/225000 [155:06:30<172:34:00,  4.32s/it]
                                                              
{'loss': '0.7242', 'grad_norm': '4.333', 'learning_rate': '3.764e-05', 'epoch': '0.3611', 'num_input_tokens_seen': 255090888, 'train_runtime': '5.584e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81250/225000 [155:06:30<172:34:00,  4.32s/it]
 36%|███▌      | 81251/225000 [155:06:33<153:31:49,  3.84s/it]
 36%|███▌      | 81252/225000 [155:06:37<149:34:06,  3.75s/it]
 36%|███▌      | 81253/225000 [155:06:45<207:05:54,  5.19s/it]
 36%|███▌      | 81254/225000 [155:06:48<178:16:47,  4.46s/it]
 36%|███▌      | 81255/225000 [155:06:50<151:26:08,  3.79s/it]
 36%|███▌      | 81256/225000 [155:06:54<148:14:24,  3.71s/it]
 36%|███▌      | 81257/225000 [155:06:57<137:40:35,  3.45s/it]
 36%|███▌      | 81258/225000 [155:06:59<128:51:16,  3.23s/it]
 36%|███▌      | 81259/225000 [155:07:09<202:29:24,  5.07s/it]
 36%|███▌      | 81260/225000 [155:07:12<179:27:19,  4.49s/it]
 36%|███▌      | 81261/225000 [155:07:14<154:06:58,  3.86s/it]
 36%|███▌      | 81262/225000 [155:07:17<138:18:12,  3.46s/it]
 36%|███▌      | 81263/225000 [155:07:20<133:20:33,  3.34s/it]
 36%|███▌      | 81264/225000 [155:07:23<129:12:56,  3.24s/it]
 36%|███▌      | 81265/225000 [155:07:26<131:28:30,  3.29s/it]
 36%|███▌      | 81266/225000 [155:07:30<134:08:41,  3.36s/it]
 36%|███▌      | 81267/225000 [155:07:33<132:11:26,  3.31s/it]
 36%|███▌      | 81268/225000 [155:07:36<129:46:52,  3.25s/it]
 36%|███▌      | 81269/225000 [155:07:39<130:26:21,  3.27s/it]
 36%|███▌      | 81270/225000 [155:07:42<128:26:49,  3.22s/it]
 36%|███▌      | 81271/225000 [155:07:46<126:16:44,  3.16s/it]
 36%|███▌      | 81272/225000 [155:07:49<131:19:47,  3.29s/it]
 36%|███▌      | 81273/225000 [155:07:56<170:13:17,  4.26s/it]
 36%|███▌      | 81274/225000 [155:08:05<227:19:15,  5.69s/it]
 36%|███▌      | 81275/225000 [155:08:11<230:13:35,  5.77s/it]
 36%|███▌      | 81276/225000 [155:08:13<189:53:49,  4.76s/it]
 36%|███▌      | 81277/225000 [155:08:16<168:14:02,  4.21s/it]
 36%|███▌      | 81278/225000 [155:08:19<151:56:24,  3.81s/it]
 36%|███▌      | 81279/225000 [155:08:22<141:07:12,  3.53s/it]
 36%|███▌      | 81280/225000 [155:08:25<141:41:02,  3.55s/it]
 36%|███▌      | 81281/225000 [155:08:29<137:48:28,  3.45s/it]
 36%|███▌      | 81282/225000 [155:08:35<177:28:28,  4.45s/it]
 36%|███▌      | 81283/225000 [155:08:42<210:05:07,  5.26s/it]
 36%|███▌      | 81284/225000 [155:08:46<186:35:09,  4.67s/it]
 36%|███▌      | 81285/225000 [155:08:49<173:40:45,  4.35s/it]
 36%|███▌      | 81286/225000 [155:08:52<158:34:04,  3.97s/it]
 36%|███▌      | 81287/225000 [155:08:55<146:40:29,  3.67s/it]
 36%|███▌      | 81288/225000 [155:08:58<137:28:33,  3.44s/it]
 36%|███▌      | 81289/225000 [155:09:02<138:37:30,  3.47s/it]
 36%|███▌      | 81290/225000 [155:09:09<179:06:25,  4.49s/it]
 36%|███▌      | 81291/225000 [155:09:15<205:07:58,  5.14s/it]
 36%|███▌      | 81292/225000 [155:09:18<170:06:27,  4.26s/it]
 36%|███▌      | 81293/225000 [155:09:21<158:53:49,  3.98s/it]
 36%|███▌      | 81294/225000 [155:09:27<180:24:42,  4.52s/it]
 36%|███▌      | 81295/225000 [155:09:32<190:21:43,  4.77s/it]
 36%|███▌      | 81296/225000 [155:09:35<163:51:57,  4.11s/it]
 36%|███▌      | 81297/225000 [155:09:38<149:53:19,  3.75s/it]
 36%|███▌      | 81298/225000 [155:09:41<143:52:45,  3.60s/it]
 36%|███▌      | 81299/225000 [155:09:48<191:12:45,  4.79s/it]
 36%|███▌      | 81300/225000 [155:09:51<161:37:11,  4.05s/it]
                                                              
{'loss': '0.6921', 'grad_norm': '3.95', 'learning_rate': '3.763e-05', 'epoch': '0.3613', 'num_input_tokens_seen': 255175776, 'train_runtime': '5.586e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81300/225000 [155:09:51<161:37:11,  4.05s/it]
 36%|███▌      | 81301/225000 [155:09:54<152:06:20,  3.81s/it]
 36%|███▌      | 81302/225000 [155:09:57<148:38:51,  3.72s/it]
 36%|███▌      | 81303/225000 [155:10:01<148:38:44,  3.72s/it]
 36%|███▌      | 81304/225000 [155:10:07<173:53:46,  4.36s/it]
 36%|███▌      | 81305/225000 [155:10:11<166:32:28,  4.17s/it]
 36%|███▌      | 81306/225000 [155:10:14<158:34:54,  3.97s/it]
 36%|███▌      | 81307/225000 [155:10:18<151:17:35,  3.79s/it]
 36%|███▌      | 81308/225000 [155:10:21<151:27:06,  3.79s/it]
 36%|███▌      | 81309/225000 [155:10:25<149:16:05,  3.74s/it]
 36%|███▌      | 81310/225000 [155:10:31<174:08:40,  4.36s/it]
 36%|███▌      | 81311/225000 [155:10:37<199:53:25,  5.01s/it]
 36%|███▌      | 81312/225000 [155:10:41<180:07:27,  4.51s/it]
 36%|███▌      | 81313/225000 [155:10:44<170:28:57,  4.27s/it]
 36%|███▌      | 81314/225000 [155:10:47<151:49:58,  3.80s/it]
 36%|███▌      | 81315/225000 [155:10:50<141:15:59,  3.54s/it]
 36%|███▌      | 81316/225000 [155:10:53<132:56:09,  3.33s/it]
 36%|███▌      | 81317/225000 [155:11:03<209:14:44,  5.24s/it]
 36%|███▌      | 81318/225000 [155:11:06<188:18:28,  4.72s/it]
 36%|███▌      | 81319/225000 [155:11:09<160:31:43,  4.02s/it]
 36%|███▌      | 81320/225000 [155:11:11<143:28:05,  3.59s/it]
 36%|███▌      | 81321/225000 [155:11:15<147:59:20,  3.71s/it]
 36%|███▌      | 81322/225000 [155:11:19<144:50:01,  3.63s/it]
 36%|███▌      | 81323/225000 [155:11:22<145:30:13,  3.65s/it]
 36%|███▌      | 81324/225000 [155:11:27<154:46:47,  3.88s/it]
 36%|███▌      | 81325/225000 [155:11:34<201:16:29,  5.04s/it]
 36%|███▌      | 81326/225000 [155:11:37<172:56:57,  4.33s/it]
 36%|███▌      | 81327/225000 [155:11:40<153:27:43,  3.85s/it]
 36%|███▌      | 81328/225000 [155:11:44<155:37:42,  3.90s/it]
 36%|███▌      | 81329/225000 [155:11:47<141:19:31,  3.54s/it]
 36%|███▌      | 81330/225000 [155:11:50<139:50:25,  3.50s/it]
 36%|███▌      | 81331/225000 [155:12:00<221:56:51,  5.56s/it]
 36%|███▌      | 81332/225000 [155:12:04<198:27:52,  4.97s/it]
 36%|███▌      | 81333/225000 [155:12:06<169:09:16,  4.24s/it]
 36%|███▌      | 81334/225000 [155:12:09<155:15:52,  3.89s/it]
 36%|███▌      | 81335/225000 [155:12:13<150:16:54,  3.77s/it]
 36%|███▌      | 81336/225000 [155:12:16<141:09:36,  3.54s/it]
 36%|███▌      | 81337/225000 [155:12:19<133:59:06,  3.36s/it]
 36%|███▌      | 81338/225000 [155:12:22<134:26:26,  3.37s/it]
 36%|███▌      | 81339/225000 [155:12:30<184:45:49,  4.63s/it]
 36%|███▌      | 81340/225000 [155:12:36<201:12:13,  5.04s/it]
 36%|███▌      | 81341/225000 [155:12:39<173:58:42,  4.36s/it]
 36%|███▌      | 81342/225000 [155:12:45<197:57:31,  4.96s/it]
 36%|███▌      | 81343/225000 [155:12:48<171:09:34,  4.29s/it]
 36%|███▌      | 81344/225000 [155:12:51<154:35:25,  3.87s/it]
 36%|███▌      | 81345/225000 [155:12:54<145:12:08,  3.64s/it]
 36%|███▌      | 81346/225000 [155:13:00<170:56:06,  4.28s/it]
 36%|███▌      | 81347/225000 [155:13:06<191:41:59,  4.80s/it]
 36%|███▌      | 81348/225000 [155:13:08<161:36:58,  4.05s/it]
 36%|███▌      | 81349/225000 [155:13:11<145:26:15,  3.64s/it]
 36%|███▌      | 81350/225000 [155:13:14<139:20:10,  3.49s/it]
                                                              
{'loss': '0.6941', 'grad_norm': '3.269', 'learning_rate': '3.761e-05', 'epoch': '0.3616', 'num_input_tokens_seen': 255260576, 'train_runtime': '5.588e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81350/225000 [155:13:14<139:20:10,  3.49s/it]
 36%|███▌      | 81351/225000 [155:13:17<138:27:04,  3.47s/it]
 36%|███▌      | 81352/225000 [155:13:20<132:37:19,  3.32s/it]
 36%|███▌      | 81353/225000 [155:13:25<155:15:41,  3.89s/it]
 36%|███▌      | 81354/225000 [155:13:32<192:10:45,  4.82s/it]
 36%|███▌      | 81355/225000 [155:13:35<168:06:56,  4.21s/it]
 36%|███▌      | 81356/225000 [155:13:38<151:29:52,  3.80s/it]
 36%|███▌      | 81357/225000 [155:13:41<139:29:25,  3.50s/it]
 36%|███▌      | 81358/225000 [155:13:43<131:04:23,  3.28s/it]
 36%|███▌      | 81359/225000 [155:13:47<128:54:09,  3.23s/it]
 36%|███▌      | 81360/225000 [155:13:49<123:06:18,  3.09s/it]
 36%|███▌      | 81361/225000 [155:13:52<123:28:16,  3.09s/it]
 36%|███▌      | 81362/225000 [155:13:58<147:56:29,  3.71s/it]
 36%|███▌      | 81363/225000 [155:14:06<205:07:37,  5.14s/it]
 36%|███▌      | 81364/225000 [155:14:09<184:26:06,  4.62s/it]
 36%|███▌      | 81365/225000 [155:14:12<162:34:46,  4.07s/it]
 36%|███▌      | 81366/225000 [155:14:16<154:45:53,  3.88s/it]
 36%|███▌      | 81367/225000 [155:14:19<147:10:16,  3.69s/it]
 36%|███▌      | 81368/225000 [155:14:23<147:30:20,  3.70s/it]
 36%|███▌      | 81369/225000 [155:14:26<144:46:28,  3.63s/it]
 36%|███▌      | 81370/225000 [155:14:32<173:41:31,  4.35s/it]
 36%|███▌      | 81371/225000 [155:14:39<203:37:26,  5.10s/it]
 36%|███▌      | 81372/225000 [155:14:41<169:05:41,  4.24s/it]
 36%|███▌      | 81373/225000 [155:14:44<154:35:09,  3.87s/it]
 36%|███▌      | 81374/225000 [155:14:47<145:54:35,  3.66s/it]
 36%|███▌      | 81375/225000 [155:14:51<150:08:26,  3.76s/it]
 36%|███▌      | 81376/225000 [155:14:55<147:17:57,  3.69s/it]
 36%|███▌      | 81377/225000 [155:14:58<141:39:55,  3.55s/it]
 36%|███▌      | 81378/225000 [155:15:01<138:27:04,  3.47s/it]
 36%|███▌      | 81379/225000 [155:15:05<135:41:19,  3.40s/it]
 36%|███▌      | 81380/225000 [155:15:08<136:54:54,  3.43s/it]
 36%|███▌      | 81381/225000 [155:15:11<133:18:29,  3.34s/it]
 36%|███▌      | 81382/225000 [155:15:15<135:36:43,  3.40s/it]
 36%|███▌      | 81383/225000 [155:15:18<132:16:26,  3.32s/it]
 36%|███▌      | 81384/225000 [155:15:22<140:55:30,  3.53s/it]
 36%|███▌      | 81385/225000 [155:15:25<137:16:01,  3.44s/it]
 36%|███▌      | 81386/225000 [155:15:29<138:20:37,  3.47s/it]
 36%|███▌      | 81387/225000 [155:15:35<169:22:37,  4.25s/it]
 36%|███▌      | 81388/225000 [155:15:39<165:44:43,  4.15s/it]
 36%|███▌      | 81389/225000 [155:15:42<154:21:03,  3.87s/it]
 36%|███▌      | 81390/225000 [155:15:45<145:19:02,  3.64s/it]
 36%|███▌      | 81391/225000 [155:15:48<142:18:56,  3.57s/it]
 36%|███▌      | 81392/225000 [155:15:52<142:01:20,  3.56s/it]
 36%|███▌      | 81393/225000 [155:16:00<193:59:47,  4.86s/it]
 36%|███▌      | 81394/225000 [155:16:05<191:42:26,  4.81s/it]
 36%|███▌      | 81395/225000 [155:16:09<189:49:18,  4.76s/it]
 36%|███▌      | 81396/225000 [155:16:15<205:36:31,  5.15s/it]
 36%|███▌      | 81397/225000 [155:16:18<173:04:26,  4.34s/it]
 36%|███▌      | 81398/225000 [155:16:20<152:56:23,  3.83s/it]
 36%|███▌      | 81399/225000 [155:16:23<141:53:02,  3.56s/it]
 36%|███▌      | 81400/225000 [155:16:27<137:44:04,  3.45s/it]
                                                              
{'loss': '0.6598', 'grad_norm': '6.433', 'learning_rate': '3.76e-05', 'epoch': '0.3618', 'num_input_tokens_seen': 255343760, 'train_runtime': '5.59e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81400/225000 [155:16:27<137:44:04,  3.45s/it]
 36%|███▌      | 81401/225000 [155:16:30<140:08:12,  3.51s/it]
 36%|███▌      | 81402/225000 [155:16:34<140:26:18,  3.52s/it]
 36%|███▌      | 81403/225000 [155:16:37<141:20:46,  3.54s/it]
 36%|███▌      | 81404/225000 [155:16:41<141:46:55,  3.55s/it]
 36%|███▌      | 81405/225000 [155:16:44<139:13:52,  3.49s/it]
 36%|███▌      | 81406/225000 [155:16:48<141:28:19,  3.55s/it]
 36%|███▌      | 81407/225000 [155:16:52<141:49:37,  3.56s/it]
 36%|███▌      | 81408/225000 [155:16:55<140:34:25,  3.52s/it]
 36%|███▌      | 81409/225000 [155:16:58<139:43:45,  3.50s/it]
 36%|███▌      | 81410/225000 [155:17:02<137:44:38,  3.45s/it]
 36%|███▌      | 81411/225000 [155:17:06<144:21:21,  3.62s/it]
 36%|███▌      | 81412/225000 [155:17:11<168:41:29,  4.23s/it]
 36%|███▌      | 81413/225000 [155:17:21<233:05:29,  5.84s/it]
 36%|███▌      | 81414/225000 [155:17:25<211:35:09,  5.30s/it]
 36%|███▌      | 81415/225000 [155:17:28<182:07:28,  4.57s/it]
 36%|███▌      | 81416/225000 [155:17:31<164:57:00,  4.14s/it]
 36%|███▌      | 81417/225000 [155:17:34<152:41:10,  3.83s/it]
 36%|███▌      | 81418/225000 [155:17:37<144:16:53,  3.62s/it]
 36%|███▌      | 81419/225000 [155:17:44<179:57:14,  4.51s/it]
 36%|███▌      | 81420/225000 [155:17:51<207:46:45,  5.21s/it]
 36%|███▌      | 81421/225000 [155:17:53<176:55:03,  4.44s/it]
 36%|███▌      | 81422/225000 [155:17:56<154:27:52,  3.87s/it]
 36%|███▌      | 81423/225000 [155:17:59<142:25:52,  3.57s/it]
 36%|███▌      | 81424/225000 [155:18:02<138:34:19,  3.47s/it]
 36%|███▌      | 81425/225000 [155:18:10<187:38:40,  4.71s/it]
 36%|███▌      | 81426/225000 [155:18:15<193:31:13,  4.85s/it]
 36%|███▌      | 81427/225000 [155:18:17<165:23:21,  4.15s/it]
 36%|███▌      | 81428/225000 [155:18:20<149:35:20,  3.75s/it]
 36%|███▌      | 81429/225000 [155:18:27<189:21:43,  4.75s/it]
 36%|███▌      | 81430/225000 [155:18:30<170:00:07,  4.26s/it]
 36%|███▌      | 81431/225000 [155:18:33<152:34:16,  3.83s/it]
 36%|███▌      | 81432/225000 [155:18:36<145:22:27,  3.65s/it]
 36%|███▌      | 81433/225000 [155:18:39<137:06:22,  3.44s/it]
 36%|███▌      | 81434/225000 [155:18:43<137:03:20,  3.44s/it]
 36%|███▌      | 81435/225000 [155:18:47<143:54:09,  3.61s/it]
 36%|███▌      | 81436/225000 [155:18:56<212:52:58,  5.34s/it]
 36%|███▌      | 81437/225000 [155:18:58<174:58:23,  4.39s/it]
 36%|███▌      | 81438/225000 [155:19:02<169:15:59,  4.24s/it]
 36%|███▌      | 81439/225000 [155:19:06<169:17:20,  4.25s/it]
 36%|███▌      | 81440/225000 [155:19:09<151:23:37,  3.80s/it]
 36%|███▌      | 81441/225000 [155:19:12<136:14:59,  3.42s/it]
 36%|███▌      | 81442/225000 [155:19:14<126:43:42,  3.18s/it]
 36%|███▌      | 81443/225000 [155:19:18<129:56:52,  3.26s/it]
 36%|███▌      | 81444/225000 [155:19:26<184:13:50,  4.62s/it]
 36%|███▌      | 81445/225000 [155:19:31<198:13:51,  4.97s/it]
 36%|███▌      | 81446/225000 [155:19:37<204:22:16,  5.13s/it]
 36%|███▌      | 81447/225000 [155:19:40<177:27:10,  4.45s/it]
 36%|███▌      | 81448/225000 [155:19:42<154:44:38,  3.88s/it]
 36%|███▌      | 81449/225000 [155:19:45<136:28:51,  3.42s/it]
 36%|███▌      | 81450/225000 [155:19:47<123:56:42,  3.11s/it]
                                                              
{'loss': '0.6537', 'grad_norm': '4.667', 'learning_rate': '3.758e-05', 'epoch': '0.362', 'num_input_tokens_seen': 255428560, 'train_runtime': '5.592e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81450/225000 [155:19:47<123:56:42,  3.11s/it]
 36%|███▌      | 81451/225000 [155:19:54<169:04:35,  4.24s/it]
 36%|███▌      | 81452/225000 [155:19:58<171:28:05,  4.30s/it]
 36%|███▌      | 81453/225000 [155:20:01<154:49:12,  3.88s/it]
 36%|███▌      | 81454/225000 [155:20:04<146:42:09,  3.68s/it]
 36%|███▌      | 81455/225000 [155:20:07<136:27:38,  3.42s/it]
 36%|███▌      | 81456/225000 [155:20:10<129:38:01,  3.25s/it]
 36%|███▌      | 81457/225000 [155:20:13<126:23:11,  3.17s/it]
 36%|███▌      | 81458/225000 [155:20:18<148:16:18,  3.72s/it]
 36%|███▌      | 81459/225000 [155:20:28<219:22:16,  5.50s/it]
 36%|███▌      | 81460/225000 [155:20:31<190:12:48,  4.77s/it]
 36%|███▌      | 81461/225000 [155:20:33<164:49:48,  4.13s/it]
 36%|███▌      | 81462/225000 [155:20:36<150:31:20,  3.78s/it]
 36%|███▌      | 81463/225000 [155:20:40<146:23:29,  3.67s/it]
 36%|███▌      | 81464/225000 [155:20:43<134:13:19,  3.37s/it]
 36%|███▌      | 81465/225000 [155:20:46<134:59:17,  3.39s/it]
 36%|███▌      | 81466/225000 [155:20:49<133:22:10,  3.35s/it]
 36%|███▌      | 81467/225000 [155:20:58<200:25:45,  5.03s/it]
 36%|███▌      | 81468/225000 [155:21:02<183:03:28,  4.59s/it]
 36%|███▌      | 81469/225000 [155:21:04<158:58:08,  3.99s/it]
 36%|███▌      | 81470/225000 [155:21:07<148:05:12,  3.71s/it]
 36%|███▌      | 81471/225000 [155:21:14<179:36:22,  4.50s/it]
 36%|███▌      | 81472/225000 [155:21:16<157:20:23,  3.95s/it]
 36%|███▌      | 81473/225000 [155:21:19<146:46:31,  3.68s/it]
 36%|███▌      | 81474/225000 [155:21:27<189:27:47,  4.75s/it]
 36%|███▌      | 81475/225000 [155:21:29<166:04:21,  4.17s/it]
 36%|███▌      | 81476/225000 [155:21:32<148:28:00,  3.72s/it]
 36%|███▌      | 81477/225000 [155:21:35<140:04:24,  3.51s/it]
 36%|███▌      | 81478/225000 [155:21:39<137:43:49,  3.45s/it]
 36%|███▌      | 81479/225000 [155:21:42<140:13:13,  3.52s/it]
 36%|███▌      | 81480/225000 [155:21:46<146:57:57,  3.69s/it]
 36%|███▌      | 81481/225000 [155:21:50<143:58:05,  3.61s/it]
 36%|███▌      | 81482/225000 [155:21:53<139:09:00,  3.49s/it]
 36%|███▌      | 81483/225000 [155:21:58<152:34:28,  3.83s/it]
 36%|███▌      | 81484/225000 [155:22:05<198:31:52,  4.98s/it]
 36%|███▌      | 81485/225000 [155:22:08<169:02:59,  4.24s/it]
 36%|███▌      | 81486/225000 [155:22:10<148:54:29,  3.74s/it]
 36%|███▌      | 81487/225000 [155:22:13<140:32:48,  3.53s/it]
 36%|███▌      | 81488/225000 [155:22:17<140:56:28,  3.54s/it]
 36%|███▌      | 81489/225000 [155:22:20<140:39:24,  3.53s/it]
 36%|███▌      | 81490/225000 [155:22:29<202:13:26,  5.07s/it]
 36%|███▌      | 81491/225000 [155:22:32<177:02:07,  4.44s/it]
 36%|███▌      | 81492/225000 [155:22:35<164:17:57,  4.12s/it]
 36%|███▌      | 81493/225000 [155:22:38<148:08:27,  3.72s/it]
 36%|███▌      | 81494/225000 [155:22:41<136:58:05,  3.44s/it]
 36%|███▌      | 81495/225000 [155:22:44<135:58:57,  3.41s/it]
 36%|███▌      | 81496/225000 [155:22:50<157:58:48,  3.96s/it]
 36%|███▌      | 81497/225000 [155:22:58<213:25:56,  5.35s/it]
 36%|███▌      | 81498/225000 [155:23:02<198:09:36,  4.97s/it]
 36%|███▌      | 81499/225000 [155:23:06<179:36:45,  4.51s/it]
 36%|███▌      | 81500/225000 [155:23:08<159:51:18,  4.01s/it]
                                                              
{'loss': '0.6505', 'grad_norm': '4.579', 'learning_rate': '3.757e-05', 'epoch': '0.3622', 'num_input_tokens_seen': 255512408, 'train_runtime': '5.594e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81500/225000 [155:23:09<159:51:18,  4.01s/it]
 36%|███▌      | 81501/225000 [155:23:12<152:24:50,  3.82s/it]
 36%|███▌      | 81502/225000 [155:23:17<172:42:03,  4.33s/it]
 36%|███▌      | 81503/225000 [155:23:24<201:29:27,  5.05s/it]
 36%|███▌      | 81504/225000 [155:23:27<172:33:06,  4.33s/it]
 36%|███▌      | 81505/225000 [155:23:29<150:51:34,  3.78s/it]
 36%|███▌      | 81506/225000 [155:23:32<138:54:12,  3.48s/it]
 36%|███▌      | 81507/225000 [155:23:35<135:17:13,  3.39s/it]
 36%|███▌      | 81508/225000 [155:23:38<133:09:20,  3.34s/it]
 36%|███▌      | 81509/225000 [155:23:42<129:48:50,  3.26s/it]
 36%|███▌      | 81510/225000 [155:23:51<203:54:46,  5.12s/it]
 36%|███▌      | 81511/225000 [155:23:55<185:25:31,  4.65s/it]
 36%|███▌      | 81512/225000 [155:23:58<166:46:49,  4.18s/it]
 36%|███▌      | 81513/225000 [155:24:01<158:58:39,  3.99s/it]
 36%|███▌      | 81514/225000 [155:24:05<152:10:42,  3.82s/it]
 36%|███▌      | 81515/225000 [155:24:08<141:57:01,  3.56s/it]
 36%|███▌      | 81516/225000 [155:24:11<137:10:24,  3.44s/it]
 36%|███▌      | 81517/225000 [155:24:14<138:11:06,  3.47s/it]
 36%|███▌      | 81518/225000 [155:24:17<132:53:43,  3.33s/it]
 36%|███▌      | 81519/225000 [155:24:20<129:54:21,  3.26s/it]
 36%|███▌      | 81520/225000 [155:24:28<184:23:25,  4.63s/it]
 36%|███▌      | 81521/225000 [155:24:36<228:07:48,  5.72s/it]
 36%|███▌      | 81522/225000 [155:24:39<186:42:51,  4.68s/it]
 36%|███▌      | 81523/225000 [155:24:43<186:34:37,  4.68s/it]
 36%|███▌      | 81524/225000 [155:24:47<176:22:32,  4.43s/it]
 36%|███▌      | 81525/225000 [155:24:50<155:41:00,  3.91s/it]
 36%|███▌      | 81526/225000 [155:24:53<142:41:26,  3.58s/it]
 36%|███▌      | 81527/225000 [155:24:56<133:55:17,  3.36s/it]
 36%|███▌      | 81528/225000 [155:24:59<130:31:16,  3.28s/it]
 36%|███▌      | 81529/225000 [155:25:01<122:14:46,  3.07s/it]
 36%|███▌      | 81530/225000 [155:25:09<174:20:02,  4.37s/it]
 36%|███▌      | 81531/225000 [155:25:16<206:48:06,  5.19s/it]
 36%|███▌      | 81532/225000 [155:25:19<178:18:11,  4.47s/it]
 36%|███▌      | 81533/225000 [155:25:21<159:37:36,  4.01s/it]
 36%|███▌      | 81534/225000 [155:25:24<141:16:31,  3.55s/it]
 36%|███▌      | 81535/225000 [155:25:27<131:43:12,  3.31s/it]
 36%|███▌      | 81536/225000 [155:25:29<124:50:38,  3.13s/it]
 36%|███▌      | 81537/225000 [155:25:36<170:00:00,  4.27s/it]
 36%|███▌      | 81538/225000 [155:25:43<202:02:01,  5.07s/it]
 36%|███▌      | 81539/225000 [155:25:46<168:27:26,  4.23s/it]
 36%|███▌      | 81540/225000 [155:25:51<186:12:43,  4.67s/it]
 36%|███▌      | 81541/225000 [155:25:55<169:52:35,  4.26s/it]
 36%|███▌      | 81542/225000 [155:25:58<157:42:55,  3.96s/it]
 36%|███▌      | 81543/225000 [155:26:01<152:44:44,  3.83s/it]
 36%|███▌      | 81544/225000 [155:26:05<149:16:35,  3.75s/it]
 36%|███▌      | 81545/225000 [155:26:09<149:51:16,  3.76s/it]
 36%|███▌      | 81546/225000 [155:26:12<142:48:02,  3.58s/it]
 36%|███▌      | 81547/225000 [155:26:15<143:12:39,  3.59s/it]
 36%|███▌      | 81548/225000 [155:26:19<138:46:15,  3.48s/it]
 36%|███▌      | 81549/225000 [155:26:22<139:23:32,  3.50s/it]
 36%|███▌      | 81550/225000 [155:26:26<137:40:52,  3.46s/it]
                                                              
{'loss': '0.7183', 'grad_norm': '4.722', 'learning_rate': '3.755e-05', 'epoch': '0.3624', 'num_input_tokens_seen': 255597240, 'train_runtime': '5.596e+05', 'train_tokens_per_second': '456.8'}

 36%|███▌      | 81550/225000 [155:26:26<137:40:52,  3.46s/it]
 36%|███▌      | 81551/225000 [155:26:31<159:45:30,  4.01s/it]
 36%|███▌      | 81552/225000 [155:26:36<173:06:32,  4.34s/it]
 36%|███▌      | 81553/225000 [155:26:39<151:04:31,  3.79s/it]
 36%|███▌      | 81554/225000 [155:26:47<207:23:12,  5.20s/it]
 36%|███▌      | 81555/225000 [155:26:50<176:39:38,  4.43s/it]
 36%|███▌      | 81556/225000 [155:26:52<153:31:42,  3.85s/it]
 36%|███▌      | 81557/225000 [155:26:55<141:35:39,  3.55s/it]
 36%|███▌      | 81558/225000 [155:27:00<160:26:31,  4.03s/it]
 36%|███▌      | 81559/225000 [155:27:05<173:18:38,  4.35s/it]
 36%|███▌      | 81560/225000 [155:27:08<153:38:09,  3.86s/it]
 36%|███▌      | 81561/225000 [155:27:12<153:13:21,  3.85s/it]
 36%|███▌      | 81562/225000 [155:27:15<148:33:10,  3.73s/it]
 36%|███▋      | 81563/225000 [155:27:19<144:26:00,  3.63s/it]
 36%|███▋      | 81564/225000 [155:27:22<142:34:57,  3.58s/it]
 36%|███▋      | 81565/225000 [155:27:25<140:25:23,  3.52s/it]
 36%|███▋      | 81566/225000 [155:27:29<137:11:25,  3.44s/it]
 36%|███▋      | 81567/225000 [155:27:32<132:40:00,  3.33s/it]
 36%|███▋      | 81568/225000 [155:27:40<191:02:42,  4.80s/it]
 36%|███▋      | 81569/225000 [155:27:44<183:29:46,  4.61s/it]
 36%|███▋      | 81570/225000 [155:27:47<156:18:04,  3.92s/it]
 36%|███▋      | 81571/225000 [155:27:50<152:57:29,  3.84s/it]
 36%|███▋      | 81572/225000 [155:27:53<146:11:16,  3.67s/it]
 36%|███▋      | 81573/225000 [155:27:57<141:57:00,  3.56s/it]
 36%|███▋      | 81574/225000 [155:28:00<138:48:29,  3.48s/it]
 36%|███▋      | 81575/225000 [155:28:03<134:23:53,  3.37s/it]
 36%|███▋      | 81576/225000 [155:28:07<141:29:55,  3.55s/it]
 36%|███▋      | 81577/225000 [155:28:16<203:24:25,  5.11s/it]
 36%|███▋      | 81578/225000 [155:28:20<186:42:08,  4.69s/it]
 36%|███▋      | 81579/225000 [155:28:22<161:56:58,  4.07s/it]
 36%|███▋      | 81580/225000 [155:28:24<141:03:19,  3.54s/it]
 36%|███▋      | 81581/225000 [155:28:27<130:23:59,  3.27s/it]
 36%|███▋      | 81582/225000 [155:28:30<128:30:25,  3.23s/it]
 36%|███▋      | 81583/225000 [155:28:33<123:49:24,  3.11s/it]
 36%|███▋      | 81584/225000 [155:28:36<123:10:27,  3.09s/it]
 36%|███▋      | 81585/225000 [155:28:43<170:15:02,  4.27s/it]
 36%|███▋      | 81586/225000 [155:28:53<234:40:51,  5.89s/it]
 36%|███▋      | 81587/225000 [155:28:56<197:56:53,  4.97s/it]
 36%|███▋      | 81588/225000 [155:28:58<171:00:10,  4.29s/it]
 36%|███▋      | 81589/225000 [155:29:01<153:31:19,  3.85s/it]
 36%|███▋      | 81590/225000 [155:29:04<139:07:56,  3.49s/it]
 36%|███▋      | 81591/225000 [155:29:06<128:05:15,  3.22s/it]
 36%|███▋      | 81592/225000 [155:29:11<146:46:49,  3.68s/it]
 36%|███▋      | 81593/225000 [155:29:20<204:34:25,  5.14s/it]
 36%|███▋      | 81594/225000 [155:29:23<176:41:00,  4.44s/it]
 36%|███▋      | 81595/225000 [155:29:26<166:52:03,  4.19s/it]
 36%|███▋      | 81596/225000 [155:29:32<188:00:06,  4.72s/it]
 36%|███▋      | 81597/225000 [155:29:34<158:52:28,  3.99s/it]
 36%|███▋      | 81598/225000 [155:29:39<165:20:36,  4.15s/it]
 36%|███▋      | 81599/225000 [155:29:44<181:42:42,  4.56s/it]
 36%|███▋      | 81600/225000 [155:29:47<156:19:31,  3.92s/it]
                                                              
{'loss': '0.6685', 'grad_norm': '4.288', 'learning_rate': '3.753e-05', 'epoch': '0.3627', 'num_input_tokens_seen': 255680904, 'train_runtime': '5.598e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81600/225000 [155:29:47<156:19:31,  3.92s/it]
 36%|███▋      | 81601/225000 [155:29:50<144:02:16,  3.62s/it]
 36%|███▋      | 81602/225000 [155:29:52<130:10:35,  3.27s/it]
 36%|███▋      | 81603/225000 [155:29:55<122:35:03,  3.08s/it]
 36%|███▋      | 81604/225000 [155:29:57<116:25:18,  2.92s/it]
 36%|███▋      | 81605/225000 [155:30:03<146:12:59,  3.67s/it]
 36%|███▋      | 81606/225000 [155:30:06<142:45:26,  3.58s/it]
 36%|███▋      | 81607/225000 [155:30:13<182:53:34,  4.59s/it]
 36%|███▋      | 81608/225000 [155:30:18<190:22:37,  4.78s/it]
 36%|███▋      | 81609/225000 [155:30:21<169:41:58,  4.26s/it]
 36%|███▋      | 81610/225000 [155:30:24<154:59:56,  3.89s/it]
 36%|███▋      | 81611/225000 [155:30:27<141:54:48,  3.56s/it]
 36%|███▋      | 81612/225000 [155:30:30<137:48:38,  3.46s/it]
 36%|███▋      | 81613/225000 [155:30:33<131:48:37,  3.31s/it]
 36%|███▋      | 81614/225000 [155:30:36<128:55:14,  3.24s/it]
 36%|███▋      | 81615/225000 [155:30:45<188:11:07,  4.72s/it]
 36%|███▋      | 81616/225000 [155:30:53<226:16:21,  5.68s/it]
 36%|███▋      | 81617/225000 [155:30:55<187:53:58,  4.72s/it]
 36%|███▋      | 81618/225000 [155:30:58<168:55:23,  4.24s/it]
 36%|███▋      | 81619/225000 [155:31:01<150:13:31,  3.77s/it]
 36%|███▋      | 81620/225000 [155:31:04<137:33:55,  3.45s/it]
 36%|███▋      | 81621/225000 [155:31:07<138:33:28,  3.48s/it]
 36%|███▋      | 81622/225000 [155:31:12<153:20:04,  3.85s/it]
 36%|███▋      | 81623/225000 [155:31:19<197:14:05,  4.95s/it]
 36%|███▋      | 81624/225000 [155:31:22<170:14:46,  4.27s/it]
 36%|███▋      | 81625/225000 [155:31:25<156:30:24,  3.93s/it]
 36%|███▋      | 81626/225000 [155:31:28<141:43:58,  3.56s/it]
 36%|███▋      | 81627/225000 [155:31:31<132:42:31,  3.33s/it]
 36%|███▋      | 81628/225000 [155:31:34<135:17:12,  3.40s/it]
 36%|███▋      | 81629/225000 [155:31:38<139:41:20,  3.51s/it]
 36%|███▋      | 81630/225000 [155:31:44<165:10:49,  4.15s/it]
 36%|███▋      | 81631/225000 [155:31:47<156:53:57,  3.94s/it]
 36%|███▋      | 81632/225000 [155:31:51<151:08:35,  3.80s/it]
 36%|███▋      | 81633/225000 [155:31:54<149:19:00,  3.75s/it]
 36%|███▋      | 81634/225000 [155:31:58<149:50:19,  3.76s/it]
 36%|███▋      | 81635/225000 [155:32:02<153:03:16,  3.84s/it]
 36%|███▋      | 81636/225000 [155:32:05<144:50:00,  3.64s/it]
 36%|███▋      | 81637/225000 [155:32:09<141:56:25,  3.56s/it]
 36%|███▋      | 81638/225000 [155:32:12<144:05:52,  3.62s/it]
 36%|███▋      | 81639/225000 [155:32:16<145:48:53,  3.66s/it]
 36%|███▋      | 81640/225000 [155:32:20<153:32:53,  3.86s/it]
 36%|███▋      | 81641/225000 [155:32:29<207:59:13,  5.22s/it]
 36%|███▋      | 81642/225000 [155:32:36<231:09:52,  5.80s/it]
 36%|███▋      | 81643/225000 [155:32:39<195:22:37,  4.91s/it]
 36%|███▋      | 81644/225000 [155:32:41<168:16:46,  4.23s/it]
 36%|███▋      | 81645/225000 [155:32:44<148:33:45,  3.73s/it]
 36%|███▋      | 81646/225000 [155:32:47<138:43:36,  3.48s/it]
 36%|███▋      | 81647/225000 [155:32:50<135:18:14,  3.40s/it]
 36%|███▋      | 81648/225000 [155:32:58<183:58:17,  4.62s/it]
 36%|███▋      | 81649/225000 [155:33:01<169:37:14,  4.26s/it]
 36%|███▋      | 81650/225000 [155:33:04<156:24:28,  3.93s/it]
                                                              
{'loss': '0.667', 'grad_norm': '4.152', 'learning_rate': '3.752e-05', 'epoch': '0.3629', 'num_input_tokens_seen': 255764848, 'train_runtime': '5.6e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81650/225000 [155:33:04<156:24:28,  3.93s/it]
 36%|███▋      | 81651/225000 [155:33:07<144:37:34,  3.63s/it]
 36%|███▋      | 81652/225000 [155:33:10<135:05:43,  3.39s/it]
 36%|███▋      | 81653/225000 [155:33:13<125:35:03,  3.15s/it]
 36%|███▋      | 81654/225000 [155:33:16<126:02:44,  3.17s/it]
 36%|███▋      | 81655/225000 [155:33:22<168:46:09,  4.24s/it]
 36%|███▋      | 81656/225000 [155:33:29<199:52:02,  5.02s/it]
 36%|███▋      | 81657/225000 [155:33:32<170:49:40,  4.29s/it]
 36%|███▋      | 81658/225000 [155:33:35<161:37:29,  4.06s/it]
 36%|███▋      | 81659/225000 [155:33:42<193:27:31,  4.86s/it]
 36%|███▋      | 81660/225000 [155:33:45<174:31:10,  4.38s/it]
 36%|███▋      | 81661/225000 [155:33:48<156:38:03,  3.93s/it]
 36%|███▋      | 81662/225000 [155:33:51<144:46:38,  3.64s/it]
 36%|███▋      | 81663/225000 [155:33:54<135:49:36,  3.41s/it]
 36%|███▋      | 81664/225000 [155:33:57<133:49:45,  3.36s/it]
 36%|███▋      | 81665/225000 [155:34:00<129:57:52,  3.26s/it]
 36%|███▋      | 81666/225000 [155:34:03<125:51:19,  3.16s/it]
 36%|███▋      | 81667/225000 [155:34:07<128:43:43,  3.23s/it]
 36%|███▋      | 81668/225000 [155:34:10<128:47:24,  3.23s/it]
 36%|███▋      | 81669/225000 [155:34:14<139:21:01,  3.50s/it]
 36%|███▋      | 81670/225000 [155:34:23<207:08:42,  5.20s/it]
 36%|███▋      | 81671/225000 [155:34:26<182:26:38,  4.58s/it]
 36%|███▋      | 81672/225000 [155:34:29<155:41:16,  3.91s/it]
 36%|███▋      | 81673/225000 [155:34:31<136:25:54,  3.43s/it]
 36%|███▋      | 81674/225000 [155:34:35<148:44:43,  3.74s/it]
 36%|███▋      | 81675/225000 [155:34:41<173:16:04,  4.35s/it]
 36%|███▋      | 81676/225000 [155:34:45<161:30:31,  4.06s/it]
 36%|███▋      | 81677/225000 [155:34:48<148:37:45,  3.73s/it]
 36%|███▋      | 81678/225000 [155:34:51<143:42:34,  3.61s/it]
 36%|███▋      | 81679/225000 [155:34:55<143:26:56,  3.60s/it]
 36%|███▋      | 81680/225000 [155:34:58<141:46:18,  3.56s/it]
 36%|███▋      | 81681/225000 [155:35:01<138:42:01,  3.48s/it]
 36%|███▋      | 81682/225000 [155:35:05<137:15:28,  3.45s/it]
 36%|███▋      | 81683/225000 [155:35:13<192:50:54,  4.84s/it]
 36%|███▋      | 81684/225000 [155:35:17<184:23:35,  4.63s/it]
 36%|███▋      | 81685/225000 [155:35:20<166:09:25,  4.17s/it]
 36%|███▋      | 81686/225000 [155:35:23<152:31:43,  3.83s/it]
 36%|███▋      | 81687/225000 [155:35:26<143:49:20,  3.61s/it]
 36%|███▋      | 81688/225000 [155:35:29<139:02:42,  3.49s/it]
 36%|███▋      | 81689/225000 [155:35:33<139:07:53,  3.50s/it]
 36%|███▋      | 81690/225000 [155:35:36<133:52:31,  3.36s/it]
 36%|███▋      | 81691/225000 [155:35:45<200:44:22,  5.04s/it]
 36%|███▋      | 81692/225000 [155:35:50<204:17:54,  5.13s/it]
 36%|███▋      | 81693/225000 [155:35:53<172:44:12,  4.34s/it]
 36%|███▋      | 81694/225000 [155:35:56<164:35:35,  4.13s/it]
 36%|███▋      | 81695/225000 [155:36:03<195:56:56,  4.92s/it]
 36%|███▋      | 81696/225000 [155:36:06<169:12:05,  4.25s/it]
 36%|███▋      | 81697/225000 [155:36:09<155:03:58,  3.90s/it]
 36%|███▋      | 81698/225000 [155:36:12<142:46:48,  3.59s/it]
 36%|███▋      | 81699/225000 [155:36:15<141:14:10,  3.55s/it]
 36%|███▋      | 81700/225000 [155:36:18<137:34:17,  3.46s/it]
                                                              
{'loss': '0.686', 'grad_norm': '5.562', 'learning_rate': '3.75e-05', 'epoch': '0.3631', 'num_input_tokens_seen': 255846616, 'train_runtime': '5.602e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81700/225000 [155:36:18<137:34:17,  3.46s/it]
 36%|███▋      | 81701/225000 [155:36:22<133:12:37,  3.35s/it]
 36%|███▋      | 81702/225000 [155:36:24<128:28:55,  3.23s/it]
 36%|███▋      | 81703/225000 [155:36:28<129:49:12,  3.26s/it]
 36%|███▋      | 81704/225000 [155:36:32<145:55:55,  3.67s/it]
 36%|███▋      | 81705/225000 [155:36:41<208:57:25,  5.25s/it]
 36%|███▋      | 81706/225000 [155:36:44<179:05:26,  4.50s/it]
 36%|███▋      | 81707/225000 [155:36:47<162:06:00,  4.07s/it]
 36%|███▋      | 81708/225000 [155:36:51<159:42:25,  4.01s/it]
 36%|███▋      | 81709/225000 [155:36:54<145:28:30,  3.65s/it]
 36%|███▋      | 81710/225000 [155:36:57<138:47:42,  3.49s/it]
 36%|███▋      | 81711/225000 [155:37:04<184:58:06,  4.65s/it]
 36%|███▋      | 81712/225000 [155:37:10<197:44:35,  4.97s/it]
 36%|███▋      | 81713/225000 [155:37:12<166:10:59,  4.18s/it]
 36%|███▋      | 81714/225000 [155:37:16<155:58:00,  3.92s/it]
 36%|███▋      | 81715/225000 [155:37:19<147:28:10,  3.71s/it]
 36%|███▋      | 81716/225000 [155:37:23<148:43:27,  3.74s/it]
 36%|███▋      | 81717/225000 [155:37:26<142:29:43,  3.58s/it]
 36%|███▋      | 81718/225000 [155:37:29<138:17:01,  3.47s/it]
 36%|███▋      | 81719/225000 [155:37:33<141:11:55,  3.55s/it]
 36%|███▋      | 81720/225000 [155:37:37<143:47:02,  3.61s/it]
 36%|███▋      | 81721/225000 [155:37:40<143:01:08,  3.59s/it]
 36%|███▋      | 81722/225000 [155:37:44<143:49:54,  3.61s/it]
 36%|███▋      | 81723/225000 [155:37:48<145:11:49,  3.65s/it]
 36%|███▋      | 81724/225000 [155:37:51<144:11:22,  3.62s/it]
 36%|███▋      | 81725/225000 [155:37:55<146:58:28,  3.69s/it]
 36%|███▋      | 81726/225000 [155:37:58<140:04:05,  3.52s/it]
 36%|███▋      | 81727/225000 [155:38:01<138:16:19,  3.47s/it]
 36%|███▋      | 81728/225000 [155:38:05<140:48:03,  3.54s/it]
 36%|███▋      | 81729/225000 [155:38:14<203:21:29,  5.11s/it]
 36%|███▋      | 81730/225000 [155:38:22<234:37:32,  5.90s/it]
 36%|███▋      | 81731/225000 [155:38:24<197:27:02,  4.96s/it]
 36%|███▋      | 81732/225000 [155:38:27<170:44:02,  4.29s/it]
 36%|███▋      | 81733/225000 [155:38:30<155:52:07,  3.92s/it]
 36%|███▋      | 81734/225000 [155:38:35<162:19:29,  4.08s/it]
 36%|███▋      | 81735/225000 [155:38:42<205:29:38,  5.16s/it]
 36%|███▋      | 81736/225000 [155:38:45<172:20:33,  4.33s/it]
 36%|███▋      | 81737/225000 [155:38:48<154:13:51,  3.88s/it]
 36%|███▋      | 81738/225000 [155:38:50<141:23:25,  3.55s/it]
 36%|███▋      | 81739/225000 [155:38:54<142:53:24,  3.59s/it]
 36%|███▋      | 81740/225000 [155:38:57<137:11:51,  3.45s/it]
 36%|███▋      | 81741/225000 [155:39:00<131:36:43,  3.31s/it]
 36%|███▋      | 81742/225000 [155:39:04<132:43:41,  3.34s/it]
 36%|███▋      | 81743/225000 [155:39:12<193:54:49,  4.87s/it]
 36%|███▋      | 81744/225000 [155:39:17<191:38:06,  4.82s/it]
 36%|███▋      | 81745/225000 [155:39:19<163:42:58,  4.11s/it]
 36%|███▋      | 81746/225000 [155:39:22<145:49:22,  3.66s/it]
 36%|███▋      | 81747/225000 [155:39:24<133:33:19,  3.36s/it]
 36%|███▋      | 81748/225000 [155:39:28<132:51:13,  3.34s/it]
 36%|███▋      | 81749/225000 [155:39:31<128:48:04,  3.24s/it]
 36%|███▋      | 81750/225000 [155:39:34<129:30:59,  3.25s/it]
                                                              
{'loss': '0.6818', 'grad_norm': '4.263', 'learning_rate': '3.749e-05', 'epoch': '0.3633', 'num_input_tokens_seen': 255930808, 'train_runtime': '5.604e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81750/225000 [155:39:34<129:30:59,  3.25s/it]
 36%|███▋      | 81751/225000 [155:39:37<125:07:12,  3.14s/it]
 36%|███▋      | 81752/225000 [155:39:40<128:29:36,  3.23s/it]
 36%|███▋      | 81753/225000 [155:39:44<133:24:27,  3.35s/it]
 36%|███▋      | 81754/225000 [155:39:48<143:09:25,  3.60s/it]
 36%|███▋      | 81755/225000 [155:39:57<200:17:41,  5.03s/it]
 36%|███▋      | 81756/225000 [155:40:02<202:31:58,  5.09s/it]
 36%|███▋      | 81757/225000 [155:40:04<169:29:28,  4.26s/it]
 36%|███▋      | 81758/225000 [155:40:06<147:01:40,  3.70s/it]
 36%|███▋      | 81759/225000 [155:40:10<143:47:58,  3.61s/it]
 36%|███▋      | 81760/225000 [155:40:13<143:26:42,  3.61s/it]
 36%|███▋      | 81761/225000 [155:40:17<140:41:57,  3.54s/it]
 36%|███▋      | 81762/225000 [155:40:20<141:10:55,  3.55s/it]
 36%|███▋      | 81763/225000 [155:40:24<139:53:31,  3.52s/it]
 36%|███▋      | 81764/225000 [155:40:27<140:06:07,  3.52s/it]
 36%|███▋      | 81765/225000 [155:40:31<140:45:11,  3.54s/it]
 36%|███▋      | 81766/225000 [155:40:36<161:28:06,  4.06s/it]
 36%|███▋      | 81767/225000 [155:40:45<220:32:22,  5.54s/it]
 36%|███▋      | 81768/225000 [155:40:51<224:24:45,  5.64s/it]
 36%|███▋      | 81769/225000 [155:40:53<184:02:14,  4.63s/it]
 36%|███▋      | 81770/225000 [155:40:56<158:35:45,  3.99s/it]
 36%|███▋      | 81771/225000 [155:40:59<144:57:14,  3.64s/it]
 36%|███▋      | 81772/225000 [155:41:01<133:13:58,  3.35s/it]
 36%|███▋      | 81773/225000 [155:41:10<198:22:05,  4.99s/it]
 36%|███▋      | 81774/225000 [155:41:16<203:53:40,  5.12s/it]
 36%|███▋      | 81775/225000 [155:41:18<174:14:03,  4.38s/it]
 36%|███▋      | 81776/225000 [155:41:23<181:48:38,  4.57s/it]
 36%|███▋      | 81777/225000 [155:41:26<157:32:40,  3.96s/it]
 36%|███▋      | 81778/225000 [155:41:29<143:08:28,  3.60s/it]
 36%|███▋      | 81779/225000 [155:41:31<130:52:47,  3.29s/it]
 36%|███▋      | 81780/225000 [155:41:40<197:25:51,  4.96s/it]
 36%|███▋      | 81781/225000 [155:41:43<171:44:31,  4.32s/it]
 36%|███▋      | 81782/225000 [155:41:46<153:32:40,  3.86s/it]
 36%|███▋      | 81783/225000 [155:41:49<142:33:50,  3.58s/it]
 36%|███▋      | 81784/225000 [155:41:51<131:16:40,  3.30s/it]
 36%|███▋      | 81785/225000 [155:41:54<123:13:01,  3.10s/it]
 36%|███▋      | 81786/225000 [155:41:57<118:38:36,  2.98s/it]
 36%|███▋      | 81787/225000 [155:42:02<151:33:37,  3.81s/it]
 36%|███▋      | 81788/225000 [155:42:11<209:40:27,  5.27s/it]
 36%|███▋      | 81789/225000 [155:42:14<182:39:28,  4.59s/it]
 36%|███▋      | 81790/225000 [155:42:16<154:34:30,  3.89s/it]
 36%|███▋      | 81791/225000 [155:42:21<169:21:22,  4.26s/it]
 36%|███▋      | 81792/225000 [155:42:26<177:07:19,  4.45s/it]
 36%|███▋      | 81793/225000 [155:42:29<157:29:23,  3.96s/it]
 36%|███▋      | 81794/225000 [155:42:32<147:14:49,  3.70s/it]
 36%|███▋      | 81795/225000 [155:42:35<138:13:51,  3.47s/it]
 36%|███▋      | 81796/225000 [155:42:38<129:27:25,  3.25s/it]
 36%|███▋      | 81797/225000 [155:42:41<123:23:25,  3.10s/it]
 36%|███▋      | 81798/225000 [155:42:44<124:09:44,  3.12s/it]
 36%|███▋      | 81799/225000 [155:42:46<118:40:39,  2.98s/it]
 36%|███▋      | 81800/225000 [155:42:53<167:11:14,  4.20s/it]
                                                              
{'loss': '0.6577', 'grad_norm': '4.78', 'learning_rate': '3.747e-05', 'epoch': '0.3636', 'num_input_tokens_seen': 256013800, 'train_runtime': '5.606e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81800/225000 [155:42:53<167:11:14,  4.20s/it]
 36%|███▋      | 81801/225000 [155:43:02<216:36:45,  5.45s/it]
 36%|███▋      | 81802/225000 [155:43:04<179:15:48,  4.51s/it]
 36%|███▋      | 81803/225000 [155:43:07<154:22:30,  3.88s/it]
 36%|███▋      | 81804/225000 [155:43:09<138:44:45,  3.49s/it]
 36%|███▋      | 81805/225000 [155:43:12<131:36:17,  3.31s/it]
 36%|███▋      | 81806/225000 [155:43:16<134:40:46,  3.39s/it]
 36%|███▋      | 81807/225000 [155:43:19<137:45:13,  3.46s/it]
 36%|███▋      | 81808/225000 [155:43:23<137:44:28,  3.46s/it]
 36%|███▋      | 81809/225000 [155:43:26<134:24:06,  3.38s/it]
 36%|███▋      | 81810/225000 [155:43:32<165:52:42,  4.17s/it]
 36%|███▋      | 81811/225000 [155:43:36<162:58:39,  4.10s/it]
 36%|███▋      | 81812/225000 [155:43:39<157:49:06,  3.97s/it]
 36%|███▋      | 81813/225000 [155:43:43<149:05:09,  3.75s/it]
 36%|███▋      | 81814/225000 [155:43:51<208:27:17,  5.24s/it]
 36%|███▋      | 81815/225000 [155:43:55<189:37:48,  4.77s/it]
 36%|███▋      | 81816/225000 [155:43:58<164:35:24,  4.14s/it]
 36%|███▋      | 81817/225000 [155:44:00<145:36:37,  3.66s/it]
 36%|███▋      | 81818/225000 [155:44:03<134:14:29,  3.38s/it]
 36%|███▋      | 81819/225000 [155:44:11<190:41:21,  4.79s/it]
 36%|███▋      | 81820/225000 [155:44:14<168:26:13,  4.24s/it]
 36%|███▋      | 81821/225000 [155:44:17<154:31:38,  3.89s/it]
 36%|███▋      | 81822/225000 [155:44:20<143:35:22,  3.61s/it]
 36%|███▋      | 81823/225000 [155:44:23<132:18:58,  3.33s/it]
 36%|███▋      | 81824/225000 [155:44:28<155:31:34,  3.91s/it]
 36%|███▋      | 81825/225000 [155:44:36<199:04:52,  5.01s/it]
 36%|███▋      | 81826/225000 [155:44:38<168:36:56,  4.24s/it]
 36%|███▋      | 81827/225000 [155:44:40<147:28:32,  3.71s/it]
 36%|███▋      | 81828/225000 [155:44:43<132:03:42,  3.32s/it]
 36%|███▋      | 81829/225000 [155:44:49<170:37:55,  4.29s/it]
 36%|███▋      | 81830/225000 [155:44:52<155:33:01,  3.91s/it]
 36%|███▋      | 81831/225000 [155:44:55<144:13:43,  3.63s/it]
 36%|███▋      | 81832/225000 [155:44:58<135:06:48,  3.40s/it]
 36%|███▋      | 81833/225000 [155:45:01<132:27:51,  3.33s/it]
 36%|███▋      | 81834/225000 [155:45:04<127:56:17,  3.22s/it]
 36%|███▋      | 81835/225000 [155:45:08<129:59:09,  3.27s/it]
 36%|███▋      | 81836/225000 [155:45:11<133:51:20,  3.37s/it]
 36%|███▋      | 81837/225000 [155:45:15<132:29:49,  3.33s/it]
 36%|███▋      | 81838/225000 [155:45:18<138:31:55,  3.48s/it]
 36%|███▋      | 81839/225000 [155:45:24<159:37:21,  4.01s/it]
 36%|███▋      | 81840/225000 [155:45:32<213:08:15,  5.36s/it]
 36%|███▋      | 81841/225000 [155:45:38<217:44:38,  5.48s/it]
 36%|███▋      | 81842/225000 [155:45:41<188:24:56,  4.74s/it]
 36%|███▋      | 81843/225000 [155:45:44<164:48:38,  4.14s/it]
 36%|███▋      | 81844/225000 [155:45:47<149:17:22,  3.75s/it]
 36%|███▋      | 81845/225000 [155:45:50<140:24:15,  3.53s/it]
 36%|███▋      | 81846/225000 [155:45:52<132:00:46,  3.32s/it]
 36%|███▋      | 81847/225000 [155:46:01<189:15:09,  4.76s/it]
 36%|███▋      | 81848/225000 [155:46:05<181:44:35,  4.57s/it]
 36%|███▋      | 81849/225000 [155:46:07<160:17:03,  4.03s/it]
 36%|███▋      | 81850/225000 [155:46:10<142:20:27,  3.58s/it]
                                                              
{'loss': '0.6497', 'grad_norm': '3.465', 'learning_rate': '3.745e-05', 'epoch': '0.3638', 'num_input_tokens_seen': 256097736, 'train_runtime': '5.608e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81850/225000 [155:46:10<142:20:27,  3.58s/it]
 36%|███▋      | 81851/225000 [155:46:15<158:57:40,  4.00s/it]
 36%|███▋      | 81852/225000 [155:46:22<196:30:03,  4.94s/it]
 36%|███▋      | 81853/225000 [155:46:25<166:00:23,  4.17s/it]
 36%|███▋      | 81854/225000 [155:46:27<143:56:10,  3.62s/it]
 36%|███▋      | 81855/225000 [155:46:29<129:12:44,  3.25s/it]
 36%|███▋      | 81856/225000 [155:46:37<184:44:38,  4.65s/it]
 36%|███▋      | 81857/225000 [155:46:40<169:25:26,  4.26s/it]
 36%|███▋      | 81858/225000 [155:46:43<150:00:24,  3.77s/it]
 36%|███▋      | 81859/225000 [155:46:46<144:33:00,  3.64s/it]
 36%|███▋      | 81860/225000 [155:46:50<141:01:11,  3.55s/it]
 36%|███▋      | 81861/225000 [155:46:53<139:56:51,  3.52s/it]
 36%|███▋      | 81862/225000 [155:46:59<163:06:31,  4.10s/it]
 36%|███▋      | 81863/225000 [155:47:07<215:20:54,  5.42s/it]
 36%|███▋      | 81864/225000 [155:47:11<199:42:56,  5.02s/it]
 36%|███▋      | 81865/225000 [155:47:14<169:38:43,  4.27s/it]
 36%|███▋      | 81866/225000 [155:47:16<149:58:49,  3.77s/it]
 36%|███▋      | 81867/225000 [155:47:23<187:00:55,  4.70s/it]
 36%|███▋      | 81868/225000 [155:47:27<180:53:03,  4.55s/it]
 36%|███▋      | 81869/225000 [155:47:30<158:52:38,  4.00s/it]
 36%|███▋      | 81870/225000 [155:47:33<143:54:10,  3.62s/it]
 36%|███▋      | 81871/225000 [155:47:39<174:43:10,  4.39s/it]
 36%|███▋      | 81872/225000 [155:47:42<157:22:56,  3.96s/it]
 36%|███▋      | 81873/225000 [155:47:44<138:09:50,  3.48s/it]
 36%|███▋      | 81874/225000 [155:47:48<145:12:56,  3.65s/it]
 36%|███▋      | 81875/225000 [155:47:54<167:42:39,  4.22s/it]
 36%|███▋      | 81876/225000 [155:47:57<147:27:12,  3.71s/it]
 36%|███▋      | 81877/225000 [155:48:02<166:12:47,  4.18s/it]
 36%|███▋      | 81878/225000 [155:48:07<178:47:58,  4.50s/it]
 36%|███▋      | 81879/225000 [155:48:09<153:44:47,  3.87s/it]
 36%|███▋      | 81880/225000 [155:48:12<136:39:52,  3.44s/it]
 36%|███▋      | 81881/225000 [155:48:14<125:39:51,  3.16s/it]
 36%|███▋      | 81882/225000 [155:48:18<130:48:06,  3.29s/it]
 36%|███▋      | 81883/225000 [155:48:21<132:49:45,  3.34s/it]
 36%|███▋      | 81884/225000 [155:48:30<191:57:43,  4.83s/it]
 36%|███▋      | 81885/225000 [155:48:34<189:56:17,  4.78s/it]
 36%|███▋      | 81886/225000 [155:48:37<163:51:23,  4.12s/it]
 36%|███▋      | 81887/225000 [155:48:39<143:51:29,  3.62s/it]
 36%|███▋      | 81888/225000 [155:48:46<174:44:17,  4.40s/it]
 36%|███▋      | 81889/225000 [155:48:51<189:45:04,  4.77s/it]
 36%|███▋      | 81890/225000 [155:48:54<161:58:12,  4.07s/it]
 36%|███▋      | 81891/225000 [155:48:56<144:14:14,  3.63s/it]
 36%|███▋      | 81892/225000 [155:48:59<133:16:59,  3.35s/it]
 36%|███▋      | 81893/225000 [155:49:05<159:28:15,  4.01s/it]
 36%|███▋      | 81894/225000 [155:49:10<179:11:53,  4.51s/it]
 36%|███▋      | 81895/225000 [155:49:13<155:46:09,  3.92s/it]
 36%|███▋      | 81896/225000 [155:49:16<142:13:53,  3.58s/it]
 36%|███▋      | 81897/225000 [155:49:19<137:21:29,  3.46s/it]
 36%|███▋      | 81898/225000 [155:49:22<132:20:34,  3.33s/it]
 36%|███▋      | 81899/225000 [155:49:25<134:13:13,  3.38s/it]
 36%|███▋      | 81900/225000 [155:49:32<170:45:57,  4.30s/it]
                                                              
{'loss': '0.6585', 'grad_norm': '4.899', 'learning_rate': '3.744e-05', 'epoch': '0.364', 'num_input_tokens_seen': 256183800, 'train_runtime': '5.61e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81900/225000 [155:49:32<170:45:57,  4.30s/it]
 36%|███▋      | 81901/225000 [155:49:37<187:15:12,  4.71s/it]
 36%|███▋      | 81902/225000 [155:49:40<165:43:28,  4.17s/it]
 36%|███▋      | 81903/225000 [155:49:44<157:26:21,  3.96s/it]
 36%|███▋      | 81904/225000 [155:49:47<146:01:04,  3.67s/it]
 36%|███▋      | 81905/225000 [155:49:50<141:22:07,  3.56s/it]
 36%|███▋      | 81906/225000 [155:49:53<137:41:30,  3.46s/it]
 36%|███▋      | 81907/225000 [155:49:56<134:05:48,  3.37s/it]
 36%|███▋      | 81908/225000 [155:50:00<131:47:33,  3.32s/it]
 36%|███▋      | 81909/225000 [155:50:03<127:17:37,  3.20s/it]
 36%|███▋      | 81910/225000 [155:50:06<127:33:32,  3.21s/it]
 36%|███▋      | 81911/225000 [155:50:09<127:45:07,  3.21s/it]
 36%|███▋      | 81912/225000 [155:50:12<130:22:15,  3.28s/it]
 36%|███▋      | 81913/225000 [155:50:21<188:05:27,  4.73s/it]
 36%|███▋      | 81914/225000 [155:50:27<206:18:17,  5.19s/it]
 36%|███▋      | 81915/225000 [155:50:30<181:54:30,  4.58s/it]
 36%|███▋      | 81916/225000 [155:50:33<167:00:55,  4.20s/it]
 36%|███▋      | 81917/225000 [155:50:39<186:23:05,  4.69s/it]
 36%|███▋      | 81918/225000 [155:50:47<222:05:10,  5.59s/it]
 36%|███▋      | 81919/225000 [155:50:49<185:50:09,  4.68s/it]
 36%|███▋      | 81920/225000 [155:50:52<163:49:59,  4.12s/it]
 36%|███▋      | 81921/225000 [155:50:55<144:06:52,  3.63s/it]
 36%|███▋      | 81922/225000 [155:50:57<131:38:55,  3.31s/it]
 36%|███▋      | 81923/225000 [155:51:05<189:55:23,  4.78s/it]
 36%|███▋      | 81924/225000 [155:51:11<194:48:52,  4.90s/it]
 36%|███▋      | 81925/225000 [155:51:14<172:03:42,  4.33s/it]
 36%|███▋      | 81926/225000 [155:51:17<157:54:51,  3.97s/it]
 36%|███▋      | 81927/225000 [155:51:20<148:29:44,  3.74s/it]
 36%|███▋      | 81928/225000 [155:51:23<136:53:57,  3.44s/it]
 36%|███▋      | 81929/225000 [155:51:26<131:35:37,  3.31s/it]
 36%|███▋      | 81930/225000 [155:51:33<173:15:35,  4.36s/it]
 36%|███▋      | 81931/225000 [155:51:41<222:51:38,  5.61s/it]
 36%|███▋      | 81932/225000 [155:51:44<187:28:54,  4.72s/it]
 36%|███▋      | 81933/225000 [155:51:47<164:51:04,  4.15s/it]
 36%|███▋      | 81934/225000 [155:51:49<148:34:48,  3.74s/it]
 36%|███▋      | 81935/225000 [155:51:52<139:41:24,  3.52s/it]
 36%|███▋      | 81936/225000 [155:51:56<135:58:16,  3.42s/it]
 36%|███▋      | 81937/225000 [155:51:59<132:31:46,  3.33s/it]
 36%|███▋      | 81938/225000 [155:52:03<138:57:05,  3.50s/it]
 36%|███▋      | 81939/225000 [155:52:07<146:15:48,  3.68s/it]
 36%|███▋      | 81940/225000 [155:52:11<149:52:57,  3.77s/it]
 36%|███▋      | 81941/225000 [155:52:14<148:15:38,  3.73s/it]
 36%|███▋      | 81942/225000 [155:52:17<142:05:10,  3.58s/it]
 36%|███▋      | 81943/225000 [155:52:21<136:24:19,  3.43s/it]
 36%|███▋      | 81944/225000 [155:52:24<136:03:27,  3.42s/it]
 36%|███▋      | 81945/225000 [155:52:28<137:36:34,  3.46s/it]
 36%|███▋      | 81946/225000 [155:52:31<134:14:43,  3.38s/it]
 36%|███▋      | 81947/225000 [155:52:35<147:10:48,  3.70s/it]
 36%|███▋      | 81948/225000 [155:52:45<216:13:32,  5.44s/it]
 36%|███▋      | 81949/225000 [155:52:50<218:13:16,  5.49s/it]
 36%|███▋      | 81950/225000 [155:52:53<184:54:18,  4.65s/it]
                                                              
{'loss': '0.6098', 'grad_norm': '3.761', 'learning_rate': '3.742e-05', 'epoch': '0.3642', 'num_input_tokens_seen': 256268792, 'train_runtime': '5.612e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 81950/225000 [155:52:53<184:54:18,  4.65s/it]
 36%|███▋      | 81951/225000 [155:52:55<157:38:05,  3.97s/it]
 36%|███▋      | 81952/225000 [155:53:02<185:51:11,  4.68s/it]
 36%|███▋      | 81953/225000 [155:53:07<196:15:54,  4.94s/it]
 36%|███▋      | 81954/225000 [155:53:10<169:12:48,  4.26s/it]
 36%|███▋      | 81955/225000 [155:53:13<150:12:33,  3.78s/it]
 36%|███▋      | 81956/225000 [155:53:15<137:39:45,  3.46s/it]
 36%|███▋      | 81957/225000 [155:53:18<130:34:24,  3.29s/it]
 36%|███▋      | 81958/225000 [155:53:21<127:53:01,  3.22s/it]
 36%|███▋      | 81959/225000 [155:53:24<125:35:10,  3.16s/it]
 36%|███▋      | 81960/225000 [155:53:31<171:41:57,  4.32s/it]
 36%|███▋      | 81961/225000 [155:53:38<201:27:19,  5.07s/it]
 36%|███▋      | 81962/225000 [155:53:40<168:32:34,  4.24s/it]
 36%|███▋      | 81963/225000 [155:53:43<148:08:24,  3.73s/it]
 36%|███▋      | 81964/225000 [155:53:50<185:49:25,  4.68s/it]
 36%|███▋      | 81965/225000 [155:53:54<177:22:05,  4.46s/it]
 36%|███▋      | 81966/225000 [155:53:56<153:48:17,  3.87s/it]
 36%|███▋      | 81967/225000 [155:54:03<188:42:31,  4.75s/it]
 36%|███▋      | 81968/225000 [155:54:06<168:29:26,  4.24s/it]
 36%|███▋      | 81969/225000 [155:54:09<153:39:03,  3.87s/it]
 36%|███▋      | 81970/225000 [155:54:12<147:20:45,  3.71s/it]
 36%|███▋      | 81971/225000 [155:54:17<158:57:53,  4.00s/it]
 36%|███▋      | 81972/225000 [155:54:24<193:35:55,  4.87s/it]
 36%|███▋      | 81973/225000 [155:54:26<163:20:26,  4.11s/it]
 36%|███▋      | 81974/225000 [155:54:29<142:11:35,  3.58s/it]
 36%|███▋      | 81975/225000 [155:54:32<138:34:28,  3.49s/it]
 36%|███▋      | 81976/225000 [155:54:39<176:18:47,  4.44s/it]
 36%|███▋      | 81977/225000 [155:54:41<152:37:20,  3.84s/it]
 36%|███▋      | 81978/225000 [155:54:44<141:13:55,  3.55s/it]
 36%|███▋      | 81979/225000 [155:54:47<136:07:07,  3.43s/it]
 36%|███▋      | 81980/225000 [155:54:50<132:01:20,  3.32s/it]
 36%|███▋      | 81981/225000 [155:54:53<126:22:21,  3.18s/it]
 36%|███▋      | 81982/225000 [155:54:56<122:44:32,  3.09s/it]
 36%|███▋      | 81983/225000 [155:55:00<132:29:25,  3.34s/it]
 36%|███▋      | 81984/225000 [155:55:04<142:11:13,  3.58s/it]
 36%|███▋      | 81985/225000 [155:55:07<140:46:04,  3.54s/it]
 36%|███▋      | 81986/225000 [155:55:15<188:53:32,  4.75s/it]
 36%|███▋      | 81987/225000 [155:55:22<221:12:09,  5.57s/it]
 36%|███▋      | 81988/225000 [155:55:25<188:54:36,  4.76s/it]
 36%|███▋      | 81989/225000 [155:55:28<169:01:07,  4.25s/it]
 36%|███▋      | 81990/225000 [155:55:31<151:36:41,  3.82s/it]
 36%|███▋      | 81991/225000 [155:55:34<144:54:30,  3.65s/it]
 36%|███▋      | 81992/225000 [155:55:38<145:15:08,  3.66s/it]
 36%|███▋      | 81993/225000 [155:55:41<139:39:51,  3.52s/it]
 36%|███▋      | 81994/225000 [155:55:44<132:22:49,  3.33s/it]
 36%|███▋      | 81995/225000 [155:55:47<127:28:04,  3.21s/it]
 36%|███▋      | 81996/225000 [155:55:51<138:58:21,  3.50s/it]
 36%|███▋      | 81997/225000 [155:56:00<197:30:02,  4.97s/it]
 36%|███▋      | 81998/225000 [155:56:04<186:34:49,  4.70s/it]
 36%|███▋      | 81999/225000 [155:56:07<171:58:55,  4.33s/it]
 36%|███▋      | 82000/225000 [155:56:11<164:20:49,  4.14s/it]
                                                              
{'loss': '0.6776', 'grad_norm': '5.178', 'learning_rate': '3.741e-05', 'epoch': '0.3644', 'num_input_tokens_seen': 256352416, 'train_runtime': '5.614e+05', 'train_tokens_per_second': '456.7'}

 36%|███▋      | 82000/225000 [155:56:11<164:20:49,  4.14s/it]
 36%|███▋      | 82001/225000 [155:56:17<184:15:33,  4.64s/it]
 36%|███▋      | 82002/225000 [155:56:19<156:32:56,  3.94s/it]
 36%|███▋      | 82003/225000 [155:56:22<141:39:21,  3.57s/it]
 36%|███▋      | 82004/225000 [155:56:24<128:01:50,  3.22s/it]
 36%|███▋      | 82005/225000 [155:56:27<120:26:36,  3.03s/it]
 36%|███▋      | 82006/225000 [155:56:30<128:18:15,  3.23s/it]
 36%|███▋      | 82007/225000 [155:56:34<136:28:40,  3.44s/it]
 36%|███▋      | 82008/225000 [155:56:38<138:59:39,  3.50s/it]
 36%|███▋      | 82009/225000 [155:56:42<140:46:44,  3.54s/it]
 36%|███▋      | 82010/225000 [155:56:45<143:47:17,  3.62s/it]
 36%|███▋      | 82011/225000 [155:56:49<145:39:44,  3.67s/it]
 36%|███▋      | 82012/225000 [155:56:53<146:15:15,  3.68s/it]
 36%|███▋      | 82013/225000 [155:56:57<145:18:45,  3.66s/it]
 36%|███▋      | 82014/225000 [155:57:01<151:37:20,  3.82s/it]
 36%|███▋      | 82015/225000 [155:57:05<151:39:41,  3.82s/it]
 36%|███▋      | 82016/225000 [155:57:08<151:54:34,  3.82s/it]
 36%|███▋      | 82017/225000 [155:57:12<146:18:41,  3.68s/it]
 36%|███▋      | 82018/225000 [155:57:17<168:25:25,  4.24s/it]
 36%|███▋      | 82019/225000 [155:57:26<221:32:38,  5.58s/it]
 36%|███▋      | 82020/225000 [155:57:33<236:16:55,  5.95s/it]
 36%|███▋      | 82021/225000 [155:57:35<193:47:02,  4.88s/it]
 36%|███▋      | 82022/225000 [155:57:39<177:03:31,  4.46s/it]
 36%|███▋      | 82023/225000 [155:57:42<167:13:32,  4.21s/it]
 36%|███▋      | 82024/225000 [155:57:46<155:36:53,  3.92s/it]
 36%|███▋      | 82025/225000 [155:57:49<148:14:23,  3.73s/it]
 36%|███▋      | 82026/225000 [155:57:55<175:00:23,  4.41s/it]
 36%|███▋      | 82027/225000 [155:58:01<199:49:06,  5.03s/it]
 36%|███▋      | 82028/225000 [155:58:07<204:11:03,  5.14s/it]
 36%|███▋      | 82029/225000 [155:58:09<170:06:58,  4.28s/it]
 36%|███▋      | 82030/225000 [155:58:12<156:34:24,  3.94s/it]
 36%|███▋      | 82031/225000 [155:58:15<145:44:22,  3.67s/it]
 36%|███▋      | 82032/225000 [155:58:18<138:05:07,  3.48s/it]
 36%|███▋      | 82033/225000 [155:58:21<134:41:03,  3.39s/it]
 36%|███▋      | 82034/225000 [155:58:24<129:27:33,  3.26s/it]
 36%|███▋      | 82035/225000 [155:58:28<128:14:07,  3.23s/it]
 36%|███▋      | 82036/225000 [155:58:32<143:42:54,  3.62s/it]
 36%|███▋      | 82037/225000 [155:58:42<215:05:53,  5.42s/it]
 36%|███▋      | 82038/225000 [155:58:45<184:52:19,  4.66s/it]
 36%|███▋      | 82039/225000 [155:58:50<195:14:34,  4.92s/it]
 36%|███▋      | 82040/225000 [155:58:53<172:37:46,  4.35s/it]
 36%|███▋      | 82041/225000 [155:58:56<151:37:15,  3.82s/it]
 36%|███▋      | 82042/225000 [155:58:59<147:31:53,  3.72s/it]
 36%|███▋      | 82043/225000 [155:59:02<142:17:32,  3.58s/it]
 36%|███▋      | 82044/225000 [155:59:06<142:12:27,  3.58s/it]
 36%|███▋      | 82045/225000 [155:59:10<141:38:42,  3.57s/it]
 36%|███▋      | 82046/225000 [155:59:13<138:06:56,  3.48s/it]
 36%|███▋      | 82047/225000 [155:59:16<138:52:35,  3.50s/it]
 36%|███▋      | 82048/225000 [155:59:20<136:49:49,  3.45s/it]
 36%|███▋      | 82049/225000 [155:59:23<134:51:22,  3.40s/it]
 36%|███▋      | 82050/225000 [155:59:26<134:00:14,  3.37s/it]
                                                              
{'loss': '0.6448', 'grad_norm': '3.3', 'learning_rate': '3.739e-05', 'epoch': '0.3647', 'num_input_tokens_seen': 256437072, 'train_runtime': '5.616e+05', 'train_tokens_per_second': '456.6'}

 36%|███▋      | 82050/225000 [155:59:26<134:00:14,  3.37s/it]
 36%|███▋      | 82051/225000 [155:59:31<153:52:01,  3.87s/it]
 36%|███▋      | 82052/225000 [155:59:35<149:29:27,  3.76s/it]
 36%|███▋      | 82053/225000 [155:59:38<146:07:56,  3.68s/it]
 36%|███▋      | 82054/225000 [155:59:42<147:26:09,  3.71s/it]
 36%|███▋      | 82055/225000 [155:59:46<148:43:39,  3.75s/it]
 36%|███▋      | 82056/225000 [155:59:49<142:39:19,  3.59s/it]
 36%|███▋      | 82057/225000 [155:59:54<163:04:57,  4.11s/it]
 36%|███▋      | 82058/225000 [155:59:58<154:40:40,  3.90s/it]
 36%|███▋      | 82059/225000 [156:00:02<154:08:13,  3.88s/it]
 36%|███▋      | 82060/225000 [156:00:05<151:26:01,  3.81s/it]
 36%|███▋      | 82061/225000 [156:00:09<150:45:33,  3.80s/it]
 36%|███▋      | 82062/225000 [156:00:14<167:21:20,  4.21s/it]
 36%|███▋      | 82063/225000 [156:00:19<168:56:03,  4.25s/it]
 36%|███▋      | 82064/225000 [156:00:22<162:53:02,  4.10s/it]
 36%|███▋      | 82065/225000 [156:00:27<166:12:13,  4.19s/it]
 36%|███▋      | 82066/225000 [156:00:35<216:30:34,  5.45s/it]
 36%|███▋      | 82067/225000 [156:00:40<209:05:22,  5.27s/it]
 36%|███▋      | 82068/225000 [156:00:43<180:43:48,  4.55s/it]
 36%|███▋      | 82069/225000 [156:00:45<154:57:27,  3.90s/it]
 36%|███▋      | 82070/225000 [156:00:48<138:56:23,  3.50s/it]
 36%|███▋      | 82071/225000 [156:00:51<133:07:30,  3.35s/it]
 36%|███▋      | 82072/225000 [156:00:59<190:32:12,  4.80s/it]
 36%|███▋      | 82073/225000 [156:01:04<197:29:16,  4.97s/it]
 36%|███▋      | 82074/225000 [156:01:07<166:10:05,  4.19s/it]
 36%|███▋      | 82075/225000 [156:01:09<145:28:25,  3.66s/it]
 36%|███▋      | 82076/225000 [156:01:12<137:42:10,  3.47s/it]
 36%|███▋      | 82077/225000 [156:01:16<143:52:48,  3.62s/it]
 36%|███▋      | 82078/225000 [156:01:20<147:40:20,  3.72s/it]
 36%|███▋      | 82079/225000 [156:01:23<142:20:48,  3.59s/it]
 36%|███▋      | 82080/225000 [156:01:32<201:13:47,  5.07s/it]
 36%|███▋      | 82081/225000 [156:01:35<173:07:18,  4.36s/it]
 36%|███▋      | 82082/225000 [156:01:38<156:33:59,  3.94s/it]
 36%|███▋      | 82083/225000 [156:01:41<150:36:34,  3.79s/it]
 36%|███▋      | 82084/225000 [156:01:50<206:07:10,  5.19s/it]
 36%|███▋      | 82085/225000 [156:01:52<175:02:24,  4.41s/it]
 36%|███▋      | 82086/225000 [156:01:54<149:56:45,  3.78s/it]
 36%|███▋      | 82087/225000 [156:01:57<135:31:42,  3.41s/it]
 36%|███▋      | 82088/225000 [156:02:00<128:31:34,  3.24s/it]
 36%|███▋      | 82089/225000 [156:02:03<129:20:25,  3.26s/it]
 36%|███▋      | 82090/225000 [156:02:07<130:58:25,  3.30s/it]
 36%|███▋      | 82091/225000 [156:02:13<170:58:45,  4.31s/it]
 36%|███▋      | 82092/225000 [156:02:19<183:03:26,  4.61s/it]
 36%|███▋      | 82093/225000 [156:02:22<164:38:11,  4.15s/it]
 36%|███▋      | 82094/225000 [156:02:25<150:38:01,  3.79s/it]
 36%|███▋      | 82095/225000 [156:02:28<142:33:27,  3.59s/it]
 36%|███▋      | 82096/225000 [156:02:31<136:10:42,  3.43s/it]
 36%|███▋      | 82097/225000 [156:02:39<199:34:29,  5.03s/it]
 36%|███▋      | 82098/225000 [156:02:44<189:31:33,  4.77s/it]
 36%|███▋      | 82099/225000 [156:02:46<163:04:47,  4.11s/it]
 36%|███▋      | 82100/225000 [156:02:50<160:46:45,  4.05s/it]
                                                              
{'loss': '0.6699', 'grad_norm': '2.448', 'learning_rate': '3.738e-05', 'epoch': '0.3649', 'num_input_tokens_seen': 256521808, 'train_runtime': '5.618e+05', 'train_tokens_per_second': '456.6'}

 36%|███▋      | 82100/225000 [156:02:50<160:46:45,  4.05s/it]
 36%|███▋      | 82101/225000 [156:02:56<187:46:22,  4.73s/it]
 36%|███▋      | 82102/225000 [156:03:00<169:21:52,  4.27s/it]
 36%|███▋      | 82103/225000 [156:03:04<165:13:39,  4.16s/it]
 36%|███▋      | 82104/225000 [156:03:07<158:55:13,  4.00s/it]
 36%|███▋      | 82105/225000 [156:03:11<151:30:45,  3.82s/it]
 36%|███▋      | 82106/225000 [156:03:15<153:14:45,  3.86s/it]
 36%|███▋      | 82107/225000 [156:03:20<175:41:10,  4.43s/it]
 36%|███▋      | 82108/225000 [156:03:26<193:51:48,  4.88s/it]
 36%|███▋      | 82109/225000 [156:03:29<172:39:20,  4.35s/it]
 36%|███▋      | 82110/225000 [156:03:32<157:09:48,  3.96s/it]
 36%|███▋      | 82111/225000 [156:03:35<139:24:15,  3.51s/it]
 36%|███▋      | 82112/225000 [156:03:43<189:37:17,  4.78s/it]
 36%|███▋      | 82113/225000 [156:03:48<196:37:22,  4.95s/it]
 36%|███▋      | 82114/225000 [156:03:51<173:19:05,  4.37s/it]
 36%|███▋      | 82115/225000 [156:03:54<152:48:07,  3.85s/it]
 36%|███▋      | 82116/225000 [156:03:56<137:14:31,  3.46s/it]
 36%|███▋      | 82117/225000 [156:04:00<137:29:15,  3.46s/it]
 36%|███▋      | 82118/225000 [156:04:03<137:00:12,  3.45s/it]
 36%|███▋      | 82119/225000 [156:04:07<144:00:53,  3.63s/it]
 36%|███▋      | 82120/225000 [156:04:10<141:07:44,  3.56s/it]
 36%|███▋      | 82121/225000 [156:04:19<199:08:10,  5.02s/it]
 36%|███▋      | 82122/225000 [156:04:22<177:05:25,  4.46s/it]
 36%|███▋      | 82123/225000 [156:04:25<163:21:06,  4.12s/it]
 36%|███▋      | 82124/225000 [156:04:28<150:36:52,  3.79s/it]
 36%|███▋      | 82125/225000 [156:04:31<140:17:03,  3.53s/it]
 37%|███▋      | 82126/225000 [156:04:34<131:44:41,  3.32s/it]
 37%|███▋      | 82127/225000 [156:04:37<127:46:34,  3.22s/it]
 37%|███▋      | 82128/225000 [156:04:40<123:53:08,  3.12s/it]
 37%|███▋      | 82129/225000 [156:04:44<132:51:34,  3.35s/it]
 37%|███▋      | 82130/225000 [156:04:52<193:33:09,  4.88s/it]
 37%|███▋      | 82131/225000 [156:04:57<185:18:58,  4.67s/it]
 37%|███▋      | 82132/225000 [156:04:59<156:56:01,  3.95s/it]
 37%|███▋      | 82133/225000 [156:05:02<149:49:38,  3.78s/it]
 37%|███▋      | 82134/225000 [156:05:05<139:37:26,  3.52s/it]
 37%|███▋      | 82135/225000 [156:05:09<143:23:20,  3.61s/it]
 37%|███▋      | 82136/225000 [156:05:18<207:35:57,  5.23s/it]
 37%|███▋      | 82137/225000 [156:05:23<205:09:48,  5.17s/it]
 37%|███▋      | 82138/225000 [156:05:25<170:24:54,  4.29s/it]
 37%|███▋      | 82139/225000 [156:05:28<151:54:21,  3.83s/it]
 37%|███▋      | 82140/225000 [156:05:31<143:37:41,  3.62s/it]
 37%|███▋      | 82141/225000 [156:05:34<138:13:30,  3.48s/it]
 37%|███▋      | 82142/225000 [156:05:37<133:43:50,  3.37s/it]
 37%|███▋      | 82143/225000 [156:05:40<130:55:07,  3.30s/it]
 37%|███▋      | 82144/225000 [156:05:44<129:20:55,  3.26s/it]
 37%|███▋      | 82145/225000 [156:05:47<124:54:06,  3.15s/it]
 37%|███▋      | 82146/225000 [156:05:50<131:55:48,  3.32s/it]
 37%|███▋      | 82147/225000 [156:05:55<147:51:02,  3.73s/it]
 37%|███▋      | 82148/225000 [156:06:03<200:25:52,  5.05s/it]
 37%|███▋      | 82149/225000 [156:06:10<226:58:38,  5.72s/it]
 37%|███▋      | 82150/225000 [156:06:13<186:27:54,  4.70s/it]
                                                              
{'loss': '0.6775', 'grad_norm': '5.534', 'learning_rate': '3.736e-05', 'epoch': '0.3651', 'num_input_tokens_seen': 256607368, 'train_runtime': '5.62e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82150/225000 [156:06:13<186:27:54,  4.70s/it]
 37%|███▋      | 82151/225000 [156:06:15<157:52:18,  3.98s/it]
 37%|███▋      | 82152/225000 [156:06:18<144:47:14,  3.65s/it]
 37%|███▋      | 82153/225000 [156:06:24<174:41:28,  4.40s/it]
 37%|███▋      | 82154/225000 [156:06:29<184:38:27,  4.65s/it]
 37%|███▋      | 82155/225000 [156:06:32<157:15:07,  3.96s/it]
 37%|███▋      | 82156/225000 [156:06:34<136:43:21,  3.45s/it]
 37%|███▋      | 82157/225000 [156:06:38<146:07:58,  3.68s/it]
 37%|███▋      | 82158/225000 [156:06:47<213:01:02,  5.37s/it]
 37%|███▋      | 82159/225000 [156:06:50<179:14:39,  4.52s/it]
 37%|███▋      | 82160/225000 [156:06:53<157:20:16,  3.97s/it]
 37%|███▋      | 82161/225000 [156:06:55<139:21:59,  3.51s/it]
 37%|███▋      | 82162/225000 [156:06:58<127:20:56,  3.21s/it]
 37%|███▋      | 82163/225000 [156:07:04<163:19:22,  4.12s/it]
 37%|███▋      | 82164/225000 [156:07:07<155:44:55,  3.93s/it]
 37%|███▋      | 82165/225000 [156:07:11<153:54:22,  3.88s/it]
 37%|███▋      | 82166/225000 [156:07:15<150:22:23,  3.79s/it]
 37%|███▋      | 82167/225000 [156:07:18<146:33:23,  3.69s/it]
 37%|███▋      | 82168/225000 [156:07:22<143:53:00,  3.63s/it]
 37%|███▋      | 82169/225000 [156:07:25<136:06:23,  3.43s/it]
 37%|███▋      | 82170/225000 [156:07:28<132:31:14,  3.34s/it]
 37%|███▋      | 82171/225000 [156:07:31<133:11:25,  3.36s/it]
 37%|███▋      | 82172/225000 [156:07:34<130:13:23,  3.28s/it]
 37%|███▋      | 82173/225000 [156:07:38<141:59:42,  3.58s/it]
 37%|███▋      | 82174/225000 [156:07:48<208:13:29,  5.25s/it]
 37%|███▋      | 82175/225000 [156:07:54<216:43:42,  5.46s/it]
 37%|███▋      | 82176/225000 [156:07:56<184:58:02,  4.66s/it]
 37%|███▋      | 82177/225000 [156:07:59<162:01:30,  4.08s/it]
 37%|███▋      | 82178/225000 [156:08:05<183:40:15,  4.63s/it]
 37%|███▋      | 82179/225000 [156:08:12<206:56:41,  5.22s/it]
 37%|███▋      | 82180/225000 [156:08:14<173:45:13,  4.38s/it]
 37%|███▋      | 82181/225000 [156:08:17<155:16:10,  3.91s/it]
 37%|███▋      | 82182/225000 [156:08:19<139:55:00,  3.53s/it]
 37%|███▋      | 82183/225000 [156:08:22<132:08:39,  3.33s/it]
 37%|███▋      | 82184/225000 [156:08:25<129:18:38,  3.26s/it]
 37%|███▋      | 82185/225000 [156:08:28<126:18:07,  3.18s/it]
 37%|███▋      | 82186/225000 [156:08:31<123:40:50,  3.12s/it]
 37%|███▋      | 82187/225000 [156:08:37<149:24:26,  3.77s/it]
 37%|███▋      | 82188/225000 [156:08:47<229:15:49,  5.78s/it]
 37%|███▋      | 82189/225000 [156:08:52<218:57:59,  5.52s/it]
 37%|███▋      | 82190/225000 [156:08:55<191:12:29,  4.82s/it]
 37%|███▋      | 82191/225000 [156:08:58<168:40:05,  4.25s/it]
 37%|███▋      | 82192/225000 [156:09:01<150:13:32,  3.79s/it]
 37%|███▋      | 82193/225000 [156:09:04<140:33:15,  3.54s/it]
 37%|███▋      | 82194/225000 [156:09:07<137:30:55,  3.47s/it]
 37%|███▋      | 82195/225000 [156:09:10<129:55:14,  3.28s/it]
 37%|███▋      | 82196/225000 [156:09:13<123:03:27,  3.10s/it]
 37%|███▋      | 82197/225000 [156:09:18<146:59:21,  3.71s/it]
 37%|███▋      | 82198/225000 [156:09:26<203:30:15,  5.13s/it]
 37%|███▋      | 82199/225000 [156:09:29<177:49:16,  4.48s/it]
 37%|███▋      | 82200/225000 [156:09:32<157:55:40,  3.98s/it]
                                                              
{'loss': '0.6509', 'grad_norm': '7.205', 'learning_rate': '3.734e-05', 'epoch': '0.3653', 'num_input_tokens_seen': 256692424, 'train_runtime': '5.622e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82200/225000 [156:09:32<157:55:40,  3.98s/it]
 37%|███▋      | 82201/225000 [156:09:35<147:49:31,  3.73s/it]
 37%|███▋      | 82202/225000 [156:09:38<141:21:25,  3.56s/it]
 37%|███▋      | 82203/225000 [156:09:41<132:49:31,  3.35s/it]
 37%|███▋      | 82204/225000 [156:09:44<123:35:54,  3.12s/it]
 37%|███▋      | 82205/225000 [156:09:47<125:54:20,  3.17s/it]
 37%|███▋      | 82206/225000 [156:09:54<167:36:41,  4.23s/it]
 37%|███▋      | 82207/225000 [156:10:02<221:08:47,  5.58s/it]
 37%|███▋      | 82208/225000 [156:10:05<186:00:31,  4.69s/it]
 37%|███▋      | 82209/225000 [156:10:08<166:15:32,  4.19s/it]
 37%|███▋      | 82210/225000 [156:10:11<151:47:39,  3.83s/it]
 37%|███▋      | 82211/225000 [156:10:14<144:14:28,  3.64s/it]
 37%|███▋      | 82212/225000 [156:10:17<139:20:34,  3.51s/it]
 37%|███▋      | 82213/225000 [156:10:26<194:02:11,  4.89s/it]
 37%|███▋      | 82214/225000 [156:10:31<198:04:45,  4.99s/it]
 37%|███▋      | 82215/225000 [156:10:34<176:38:46,  4.45s/it]
 37%|███▋      | 82216/225000 [156:10:37<164:44:11,  4.15s/it]
 37%|███▋      | 82217/225000 [156:10:40<149:42:15,  3.77s/it]
 37%|███▋      | 82218/225000 [156:10:44<145:25:15,  3.67s/it]
 37%|███▋      | 82219/225000 [156:10:47<136:47:11,  3.45s/it]
 37%|███▋      | 82220/225000 [156:10:51<151:36:15,  3.82s/it]
 37%|███▋      | 82221/225000 [156:10:55<144:08:18,  3.63s/it]
 37%|███▋      | 82222/225000 [156:10:59<148:56:10,  3.76s/it]
 37%|███▋      | 82223/225000 [156:11:07<205:42:27,  5.19s/it]
 37%|███▋      | 82224/225000 [156:11:11<190:36:42,  4.81s/it]
 37%|███▋      | 82225/225000 [156:11:13<161:17:30,  4.07s/it]
 37%|███▋      | 82226/225000 [156:11:17<154:02:33,  3.88s/it]
 37%|███▋      | 82227/225000 [156:11:20<143:46:26,  3.63s/it]
 37%|███▋      | 82228/225000 [156:11:23<140:20:11,  3.54s/it]
 37%|███▋      | 82229/225000 [156:11:27<140:04:50,  3.53s/it]
 37%|███▋      | 82230/225000 [156:11:30<137:10:11,  3.46s/it]
 37%|███▋      | 82231/225000 [156:11:33<135:28:14,  3.42s/it]
 37%|███▋      | 82232/225000 [156:11:39<159:13:45,  4.02s/it]
 37%|███▋      | 82233/225000 [156:11:46<200:43:10,  5.06s/it]
 37%|███▋      | 82234/225000 [156:11:49<171:07:15,  4.32s/it]
 37%|███▋      | 82235/225000 [156:11:54<180:05:43,  4.54s/it]
 37%|███▋      | 82236/225000 [156:12:00<194:52:05,  4.91s/it]
 37%|███▋      | 82237/225000 [156:12:03<170:49:23,  4.31s/it]
 37%|███▋      | 82238/225000 [156:12:05<150:05:04,  3.78s/it]
 37%|███▋      | 82239/225000 [156:12:09<145:19:43,  3.66s/it]
 37%|███▋      | 82240/225000 [156:12:12<142:30:28,  3.59s/it]
 37%|███▋      | 82241/225000 [156:12:15<140:41:24,  3.55s/it]
 37%|███▋      | 82242/225000 [156:12:19<137:01:34,  3.46s/it]
 37%|███▋      | 82243/225000 [156:12:22<132:26:08,  3.34s/it]
 37%|███▋      | 82244/225000 [156:12:25<132:26:07,  3.34s/it]
 37%|███▋      | 82245/225000 [156:12:29<136:24:37,  3.44s/it]
 37%|███▋      | 82246/225000 [156:12:35<170:08:55,  4.29s/it]
 37%|███▋      | 82247/225000 [156:12:42<197:05:31,  4.97s/it]
 37%|███▋      | 82248/225000 [156:12:45<173:58:37,  4.39s/it]
 37%|███▋      | 82249/225000 [156:12:47<154:43:03,  3.90s/it]
 37%|███▋      | 82250/225000 [156:12:51<151:28:45,  3.82s/it]
                                                              
{'loss': '0.6503', 'grad_norm': '3.775', 'learning_rate': '3.733e-05', 'epoch': '0.3656', 'num_input_tokens_seen': 256777872, 'train_runtime': '5.624e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82250/225000 [156:12:51<151:28:45,  3.82s/it]
 37%|███▋      | 82251/225000 [156:12:54<145:52:55,  3.68s/it]
 37%|███▋      | 82252/225000 [156:12:57<138:00:10,  3.48s/it]
 37%|███▋      | 82253/225000 [156:13:07<206:37:42,  5.21s/it]
 37%|███▋      | 82254/225000 [156:13:11<201:54:25,  5.09s/it]
 37%|███▋      | 82255/225000 [156:13:14<176:30:55,  4.45s/it]
 37%|███▋      | 82256/225000 [156:13:18<166:17:07,  4.19s/it]
 37%|███▋      | 82257/225000 [156:13:21<156:19:42,  3.94s/it]
 37%|███▋      | 82258/225000 [156:13:25<152:13:08,  3.84s/it]
 37%|███▋      | 82259/225000 [156:13:28<142:15:52,  3.59s/it]
 37%|███▋      | 82260/225000 [156:13:31<134:29:47,  3.39s/it]
 37%|███▋      | 82261/225000 [156:13:35<143:26:41,  3.62s/it]
 37%|███▋      | 82262/225000 [156:13:38<138:26:49,  3.49s/it]
 37%|███▋      | 82263/225000 [156:13:42<138:15:54,  3.49s/it]
 37%|███▋      | 82264/225000 [156:13:50<200:08:55,  5.05s/it]
 37%|███▋      | 82265/225000 [156:13:56<208:27:37,  5.26s/it]
 37%|███▋      | 82266/225000 [156:13:59<183:05:13,  4.62s/it]
 37%|███▋      | 82267/225000 [156:14:02<162:51:08,  4.11s/it]
 37%|███▋      | 82268/225000 [156:14:05<153:04:58,  3.86s/it]
 37%|███▋      | 82269/225000 [156:14:09<145:12:20,  3.66s/it]
 37%|███▋      | 82270/225000 [156:14:12<137:57:55,  3.48s/it]
 37%|███▋      | 82271/225000 [156:14:15<138:50:04,  3.50s/it]
 37%|███▋      | 82272/225000 [156:14:24<204:03:06,  5.15s/it]
 37%|███▋      | 82273/225000 [156:14:30<206:26:57,  5.21s/it]
 37%|███▋      | 82274/225000 [156:14:32<176:44:57,  4.46s/it]
 37%|███▋      | 82275/225000 [156:14:35<152:52:27,  3.86s/it]
 37%|███▋      | 82276/225000 [156:14:42<198:58:18,  5.02s/it]
 37%|███▋      | 82277/225000 [156:14:45<168:28:16,  4.25s/it]
 37%|███▋      | 82278/225000 [156:14:48<148:35:52,  3.75s/it]
 37%|███▋      | 82279/225000 [156:14:52<157:33:25,  3.97s/it]
 37%|███▋      | 82280/225000 [156:14:57<174:30:51,  4.40s/it]
 37%|███▋      | 82281/225000 [156:15:00<151:37:37,  3.82s/it]
 37%|███▋      | 82282/225000 [156:15:03<139:17:10,  3.51s/it]
 37%|███▋      | 82283/225000 [156:15:08<165:47:42,  4.18s/it]
 37%|███▋      | 82284/225000 [156:15:13<176:07:50,  4.44s/it]
 37%|███▋      | 82285/225000 [156:15:16<156:25:56,  3.95s/it]
 37%|███▋      | 82286/225000 [156:15:19<144:53:04,  3.65s/it]
 37%|███▋      | 82287/225000 [156:15:22<136:02:45,  3.43s/it]
 37%|███▋      | 82288/225000 [156:15:25<125:28:50,  3.17s/it]
 37%|███▋      | 82289/225000 [156:15:28<121:42:15,  3.07s/it]
 37%|███▋      | 82290/225000 [156:15:32<142:12:13,  3.59s/it]
 37%|███▋      | 82291/225000 [156:15:37<155:54:58,  3.93s/it]
 37%|███▋      | 82292/225000 [156:15:46<213:03:01,  5.37s/it]
 37%|███▋      | 82293/225000 [156:15:48<178:55:53,  4.51s/it]
 37%|███▋      | 82294/225000 [156:15:51<155:25:22,  3.92s/it]
 37%|███▋      | 82295/225000 [156:15:54<140:24:14,  3.54s/it]
 37%|███▋      | 82296/225000 [156:15:58<154:11:02,  3.89s/it]
 37%|███▋      | 82297/225000 [156:16:07<216:23:44,  5.46s/it]
 37%|███▋      | 82298/225000 [156:16:10<180:07:46,  4.54s/it]
 37%|███▋      | 82299/225000 [156:16:12<153:40:15,  3.88s/it]
 37%|███▋      | 82300/225000 [156:16:15<142:49:42,  3.60s/it]
                                                              
{'loss': '0.6301', 'grad_norm': '4.24', 'learning_rate': '3.731e-05', 'epoch': '0.3658', 'num_input_tokens_seen': 256863336, 'train_runtime': '5.626e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82300/225000 [156:16:15<142:49:42,  3.60s/it]
 37%|███▋      | 82301/225000 [156:16:21<175:08:21,  4.42s/it]
 37%|███▋      | 82302/225000 [156:16:25<161:45:17,  4.08s/it]
 37%|███▋      | 82303/225000 [156:16:29<162:57:23,  4.11s/it]
 37%|███▋      | 82304/225000 [156:16:32<156:51:48,  3.96s/it]
 37%|███▋      | 82305/225000 [156:16:36<147:33:33,  3.72s/it]
 37%|███▋      | 82306/225000 [156:16:39<144:32:32,  3.65s/it]
 37%|███▋      | 82307/225000 [156:16:43<142:09:34,  3.59s/it]
 37%|███▋      | 82308/225000 [156:16:46<141:20:37,  3.57s/it]
 37%|███▋      | 82309/225000 [156:16:50<141:21:23,  3.57s/it]
 37%|███▋      | 82310/225000 [156:16:53<138:06:19,  3.48s/it]
 37%|███▋      | 82311/225000 [156:16:57<142:53:50,  3.61s/it]
 37%|███▋      | 82312/225000 [156:17:05<198:08:30,  5.00s/it]
 37%|███▋      | 82313/225000 [156:17:10<202:32:13,  5.11s/it]
 37%|███▋      | 82314/225000 [156:17:13<177:05:15,  4.47s/it]
 37%|███▋      | 82315/225000 [156:17:17<162:23:34,  4.10s/it]
 37%|███▋      | 82316/225000 [156:17:21<161:40:36,  4.08s/it]
 37%|███▋      | 82317/225000 [156:17:24<148:09:33,  3.74s/it]
 37%|███▋      | 82318/225000 [156:17:27<146:18:48,  3.69s/it]
 37%|███▋      | 82319/225000 [156:17:30<139:14:20,  3.51s/it]
 37%|███▋      | 82320/225000 [156:17:34<136:36:11,  3.45s/it]
 37%|███▋      | 82321/225000 [156:17:37<134:46:07,  3.40s/it]
 37%|███▋      | 82322/225000 [156:17:40<132:33:53,  3.34s/it]
 37%|███▋      | 82323/225000 [156:17:43<129:55:14,  3.28s/it]
 37%|███▋      | 82324/225000 [156:17:46<128:50:05,  3.25s/it]
 37%|███▋      | 82325/225000 [156:17:50<129:53:49,  3.28s/it]
 37%|███▋      | 82326/225000 [156:17:58<187:31:59,  4.73s/it]
 37%|███▋      | 82327/225000 [156:18:06<227:42:15,  5.75s/it]
 37%|███▋      | 82328/225000 [156:18:08<186:40:40,  4.71s/it]
 37%|███▋      | 82329/225000 [156:18:11<157:49:28,  3.98s/it]
 37%|███▋      | 82330/225000 [156:18:14<152:46:59,  3.86s/it]
 37%|███▋      | 82331/225000 [156:18:23<214:07:16,  5.40s/it]
 37%|███▋      | 82332/225000 [156:18:25<177:56:47,  4.49s/it]
 37%|███▋      | 82333/225000 [156:18:28<154:46:59,  3.91s/it]
 37%|███▋      | 82334/225000 [156:18:30<136:24:47,  3.44s/it]
 37%|███▋      | 82335/225000 [156:18:33<132:16:18,  3.34s/it]
 37%|███▋      | 82336/225000 [156:18:36<123:58:32,  3.13s/it]
 37%|███▋      | 82337/225000 [156:18:45<196:16:36,  4.95s/it]
 37%|███▋      | 82338/225000 [156:18:49<182:40:48,  4.61s/it]
 37%|███▋      | 82339/225000 [156:18:51<155:42:23,  3.93s/it]
 37%|███▋      | 82340/225000 [156:18:54<143:19:59,  3.62s/it]
 37%|███▋      | 82341/225000 [156:18:57<136:25:13,  3.44s/it]
 37%|███▋      | 82342/225000 [156:19:06<196:05:44,  4.95s/it]
 37%|███▋      | 82343/225000 [156:19:10<183:17:36,  4.63s/it]
 37%|███▋      | 82344/225000 [156:19:12<156:06:31,  3.94s/it]
 37%|███▋      | 82345/225000 [156:19:14<137:36:43,  3.47s/it]
 37%|███▋      | 82346/225000 [156:19:17<128:53:54,  3.25s/it]
 37%|███▋      | 82347/225000 [156:19:21<140:18:55,  3.54s/it]
 37%|███▋      | 82348/225000 [156:19:27<170:01:24,  4.29s/it]
 37%|███▋      | 82349/225000 [156:19:32<176:09:22,  4.45s/it]
 37%|███▋      | 82350/225000 [156:19:36<169:16:43,  4.27s/it]
                                                              
{'loss': '0.6312', 'grad_norm': '6.036', 'learning_rate': '3.73e-05', 'epoch': '0.366', 'num_input_tokens_seen': 256948512, 'train_runtime': '5.628e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82350/225000 [156:19:36<169:16:43,  4.27s/it]
 37%|███▋      | 82351/225000 [156:19:39<148:15:44,  3.74s/it]
 37%|███▋      | 82352/225000 [156:19:41<136:30:52,  3.45s/it]
 37%|███▋      | 82353/225000 [156:19:48<173:31:10,  4.38s/it]
 37%|███▋      | 82354/225000 [156:19:53<186:49:12,  4.71s/it]
 37%|███▋      | 82355/225000 [156:19:56<160:48:31,  4.06s/it]
 37%|███▋      | 82356/225000 [156:19:58<140:12:50,  3.54s/it]
 37%|███▋      | 82357/225000 [156:20:04<163:11:33,  4.12s/it]
 37%|███▋      | 82358/225000 [156:20:09<181:12:55,  4.57s/it]
 37%|███▋      | 82359/225000 [156:20:13<166:12:46,  4.19s/it]
 37%|███▋      | 82360/225000 [156:20:15<147:53:20,  3.73s/it]
 37%|███▋      | 82361/225000 [156:20:18<138:35:33,  3.50s/it]
 37%|███▋      | 82362/225000 [156:20:27<206:10:00,  5.20s/it]
 37%|███▋      | 82363/225000 [156:20:30<174:22:50,  4.40s/it]
 37%|███▋      | 82364/225000 [156:20:33<152:26:18,  3.85s/it]
 37%|███▋      | 82365/225000 [156:20:35<140:18:58,  3.54s/it]
 37%|███▋      | 82366/225000 [156:20:41<164:04:41,  4.14s/it]
 37%|███▋      | 82367/225000 [156:20:48<198:32:22,  5.01s/it]
 37%|███▋      | 82368/225000 [156:20:50<166:34:32,  4.20s/it]
 37%|███▋      | 82369/225000 [156:20:53<145:57:19,  3.68s/it]
 37%|███▋      | 82370/225000 [156:20:58<158:37:36,  4.00s/it]
 37%|███▋      | 82371/225000 [156:21:04<189:18:20,  4.78s/it]
 37%|███▋      | 82372/225000 [156:21:07<165:05:13,  4.17s/it]
 37%|███▋      | 82373/225000 [156:21:09<146:02:48,  3.69s/it]
 37%|███▋      | 82374/225000 [156:21:18<203:27:35,  5.14s/it]
 37%|███▋      | 82375/225000 [156:21:21<178:27:14,  4.50s/it]
 37%|███▋      | 82376/225000 [156:21:24<156:51:45,  3.96s/it]
 37%|███▋      | 82377/225000 [156:21:26<140:37:35,  3.55s/it]
 37%|███▋      | 82378/225000 [156:21:29<130:12:01,  3.29s/it]
 37%|███▋      | 82379/225000 [156:21:32<124:13:49,  3.14s/it]
 37%|███▋      | 82380/225000 [156:21:37<146:34:09,  3.70s/it]
 37%|███▋      | 82381/225000 [156:21:43<176:51:43,  4.46s/it]
 37%|███▋      | 82382/225000 [156:21:46<157:39:44,  3.98s/it]
 37%|███▋      | 82383/225000 [156:21:48<141:46:14,  3.58s/it]
 37%|███▋      | 82384/225000 [156:21:56<183:15:18,  4.63s/it]
 37%|███▋      | 82385/225000 [156:22:00<176:08:30,  4.45s/it]
 37%|███▋      | 82386/225000 [156:22:02<157:21:00,  3.97s/it]
 37%|███▋      | 82387/225000 [156:22:06<147:31:21,  3.72s/it]
 37%|███▋      | 82388/225000 [156:22:08<135:35:11,  3.42s/it]
 37%|███▋      | 82389/225000 [156:22:11<129:57:22,  3.28s/it]
 37%|███▋      | 82390/225000 [156:22:14<129:26:12,  3.27s/it]
 37%|███▋      | 82391/225000 [156:22:20<156:58:08,  3.96s/it]
 37%|███▋      | 82392/225000 [156:22:28<201:06:13,  5.08s/it]
 37%|███▋      | 82393/225000 [156:22:30<170:58:36,  4.32s/it]
 37%|███▋      | 82394/225000 [156:22:33<147:30:20,  3.72s/it]
 37%|███▋      | 82395/225000 [156:22:35<131:58:36,  3.33s/it]
 37%|███▋      | 82396/225000 [156:22:38<123:02:46,  3.11s/it]
 37%|███▋      | 82397/225000 [156:22:41<126:35:17,  3.20s/it]
 37%|███▋      | 82398/225000 [156:22:46<143:10:40,  3.61s/it]
 37%|███▋      | 82399/225000 [156:22:54<201:16:37,  5.08s/it]
 37%|███▋      | 82400/225000 [156:22:57<179:52:16,  4.54s/it]
                                                              
{'loss': '0.6393', 'grad_norm': '3.709', 'learning_rate': '3.728e-05', 'epoch': '0.3662', 'num_input_tokens_seen': 257034696, 'train_runtime': '5.63e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82400/225000 [156:22:57<179:52:16,  4.54s/it]
 37%|███▋      | 82401/225000 [156:23:01<166:16:33,  4.20s/it]
 37%|███▋      | 82402/225000 [156:23:04<149:58:20,  3.79s/it]
 37%|███▋      | 82403/225000 [156:23:07<144:33:37,  3.65s/it]
 37%|███▋      | 82404/225000 [156:23:11<154:26:47,  3.90s/it]
 37%|███▋      | 82405/225000 [156:23:20<206:59:50,  5.23s/it]
 37%|███▋      | 82406/225000 [156:23:23<177:59:50,  4.49s/it]
 37%|███▋      | 82407/225000 [156:23:26<161:41:32,  4.08s/it]
 37%|███▋      | 82408/225000 [156:23:28<145:54:14,  3.68s/it]
 37%|███▋      | 82409/225000 [156:23:32<139:54:10,  3.53s/it]
 37%|███▋      | 82410/225000 [156:23:34<130:18:43,  3.29s/it]
 37%|███▋      | 82411/225000 [156:23:37<128:10:35,  3.24s/it]
 37%|███▋      | 82412/225000 [156:23:42<145:19:03,  3.67s/it]
 37%|███▋      | 82413/225000 [156:23:51<210:13:27,  5.31s/it]
 37%|███▋      | 82414/225000 [156:23:54<179:45:29,  4.54s/it]
 37%|███▋      | 82415/225000 [156:23:57<161:29:37,  4.08s/it]
 37%|███▋      | 82416/225000 [156:24:01<163:47:33,  4.14s/it]
 37%|███▋      | 82417/225000 [156:24:08<199:42:38,  5.04s/it]
 37%|███▋      | 82418/225000 [156:24:11<168:36:26,  4.26s/it]
 37%|███▋      | 82419/225000 [156:24:16<182:46:00,  4.61s/it]
 37%|███▋      | 82420/225000 [156:24:19<164:02:15,  4.14s/it]
 37%|███▋      | 82421/225000 [156:24:23<154:20:52,  3.90s/it]
 37%|███▋      | 82422/225000 [156:24:26<144:46:12,  3.66s/it]
 37%|███▋      | 82423/225000 [156:24:29<139:01:10,  3.51s/it]
 37%|███▋      | 82424/225000 [156:24:32<133:23:17,  3.37s/it]
 37%|███▋      | 82425/225000 [156:24:35<132:00:38,  3.33s/it]
 37%|███▋      | 82426/225000 [156:24:39<131:49:52,  3.33s/it]
 37%|███▋      | 82427/225000 [156:24:42<135:15:31,  3.42s/it]
 37%|███▋      | 82428/225000 [156:24:45<131:16:41,  3.31s/it]
 37%|███▋      | 82429/225000 [156:24:49<131:15:55,  3.31s/it]
 37%|███▋      | 82430/225000 [156:24:52<130:18:30,  3.29s/it]
 37%|███▋      | 82431/225000 [156:24:56<135:49:43,  3.43s/it]
 37%|███▋      | 82432/225000 [156:24:59<133:20:28,  3.37s/it]
 37%|███▋      | 82433/225000 [156:25:02<132:28:25,  3.35s/it]
 37%|███▋      | 82434/225000 [156:25:10<188:43:52,  4.77s/it]
 37%|███▋      | 82435/225000 [156:25:19<232:02:03,  5.86s/it]
 37%|███▋      | 82436/225000 [156:25:21<197:29:37,  4.99s/it]
 37%|███▋      | 82437/225000 [156:25:24<168:22:25,  4.25s/it]
 37%|███▋      | 82438/225000 [156:25:27<151:11:15,  3.82s/it]
 37%|███▋      | 82439/225000 [156:25:31<158:13:29,  4.00s/it]
 37%|███▋      | 82440/225000 [156:25:38<188:28:03,  4.76s/it]
 37%|███▋      | 82441/225000 [156:25:40<161:05:44,  4.07s/it]
 37%|███▋      | 82442/225000 [156:25:43<142:13:59,  3.59s/it]
 37%|███▋      | 82443/225000 [156:25:49<178:12:43,  4.50s/it]
 37%|███▋      | 82444/225000 [156:25:52<160:13:20,  4.05s/it]
 37%|███▋      | 82445/225000 [156:25:55<144:37:37,  3.65s/it]
 37%|███▋      | 82446/225000 [156:25:58<140:19:05,  3.54s/it]
 37%|███▋      | 82447/225000 [156:26:01<134:23:27,  3.39s/it]
 37%|███▋      | 82448/225000 [156:26:04<129:51:32,  3.28s/it]
 37%|███▋      | 82449/225000 [156:26:08<131:48:51,  3.33s/it]
 37%|███▋      | 82450/225000 [156:26:11<125:30:31,  3.17s/it]
                                                              
{'loss': '0.6473', 'grad_norm': '4.459', 'learning_rate': '3.726e-05', 'epoch': '0.3664', 'num_input_tokens_seen': 257119216, 'train_runtime': '5.632e+05', 'train_tokens_per_second': '456.6'}

 37%|███▋      | 82450/225000 [156:26:11<125:30:31,  3.17s/it]
 37%|███▋      | 82451/225000 [156:26:18<171:44:12,  4.34s/it]
 37%|███▋      | 82452/225000 [156:26:25<210:08:20,  5.31s/it]
 37%|███▋      | 82453/225000 [156:26:29<195:51:58,  4.95s/it]
 37%|███▋      | 82454/225000 [156:26:35<208:10:06,  5.26s/it]
 37%|███▋      | 82455/225000 [156:26:38<172:35:31,  4.36s/it]
 37%|███▋      | 82456/225000 [156:26:40<154:49:25,  3.91s/it]
 37%|███▋      | 82457/225000 [156:26:48<194:23:53,  4.91s/it]
 37%|███▋      | 82458/225000 [156:26:50<166:25:10,  4.20s/it]
 37%|███▋      | 82459/225000 [156:26:53<148:28:32,  3.75s/it]
 37%|███▋      | 82460/225000 [156:26:57<151:32:11,  3.83s/it]
 37%|███▋      | 82461/225000 [156:27:04<191:25:01,  4.83s/it]
 37%|███▋      | 82462/225000 [156:27:07<166:58:05,  4.22s/it]
 37%|███▋      | 82463/225000 [156:27:09<147:22:04,  3.72s/it]
 37%|███▋      | 82464/225000 [156:27:12<134:52:32,  3.41s/it]
 37%|███▋      | 82465/225000 [156:27:15<124:15:28,  3.14s/it]
 37%|███▋      | 82466/225000 [156:27:18<120:59:10,  3.06s/it]
 37%|███▋      | 82467/225000 [156:27:24<158:47:33,  4.01s/it]
 37%|███▋      | 82468/225000 [156:27:31<192:04:27,  4.85s/it]
 37%|███▋      | 82469/225000 [156:27:33<164:49:48,  4.16s/it]
 37%|███▋      | 82470/225000 [156:27:36<145:17:15,  3.67s/it]
 37%|███▋      | 82471/225000 [156:27:39<138:19:58,  3.49s/it]
 37%|███▋      | 82472/225000 [156:27:45<173:52:23,  4.39s/it]
 37%|███▋      | 82473/225000 [156:27:51<195:10:55,  4.93s/it]
 37%|███▋      | 82474/225000 [156:27:55<174:07:00,  4.40s/it]
 37%|███▋      | 82475/225000 [156:27:57<155:51:35,  3.94s/it]
 37%|███▋      | 82476/225000 [156:28:00<139:40:46,  3.53s/it]
 37%|███▋      | 82477/225000 [156:28:03<128:52:17,  3.26s/it]
 37%|███▋      | 82478/225000 [156:28:08<157:31:52,  3.98s/it]
 37%|███▋      | 82479/225000 [156:28:14<172:45:03,  4.36s/it]
 37%|███▋      | 82480/225000 [156:28:17<161:44:22,  4.09s/it]
 37%|███▋      | 82481/225000 [156:28:20<148:01:32,  3.74s/it]
 37%|███▋      | 82482/225000 [156:28:23<138:59:25,  3.51s/it]
 37%|███▋      | 82483/225000 [156:28:26<135:38:16,  3.43s/it]
 37%|███▋      | 82484/225000 [156:28:30<135:36:26,  3.43s/it]
 37%|███▋      | 82485/225000 [156:28:33<131:59:22,  3.33s/it]
 37%|███▋      | 82486/225000 [156:28:36<130:42:17,  3.30s/it]
 37%|███▋      | 82487/225000 [156:28:39<126:08:44,  3.19s/it]
 37%|███▋      | 82488/225000 [156:28:48<197:05:51,  4.98s/it]
 37%|███▋      | 82489/225000 [156:28:55<225:39:45,  5.70s/it]
 37%|███▋      | 82490/225000 [156:28:58<186:13:48,  4.70s/it]
 37%|███▋      | 82491/225000 [156:29:00<162:47:26,  4.11s/it]
 37%|███▋      | 82492/225000 [156:29:04<153:17:43,  3.87s/it]
 37%|███▋      | 82493/225000 [156:29:11<192:06:42,  4.85s/it]
 37%|███▋      | 82494/225000 [156:29:14<169:32:23,  4.28s/it]
 37%|███▋      | 82495/225000 [156:29:17<159:38:57,  4.03s/it]
 37%|███▋      | 82496/225000 [156:29:21<149:22:54,  3.77s/it]
 37%|███▋      | 82497/225000 [156:29:24<143:28:10,  3.62s/it]
 37%|███▋      | 82498/225000 [156:29:28<144:38:09,  3.65s/it]
 37%|███▋      | 82499/225000 [156:29:30<136:11:28,  3.44s/it]
 37%|███▋      | 82500/225000 [156:29:34<141:45:34,  3.58s/it]
                                                              
{'loss': '0.5957', 'grad_norm': '4.46', 'learning_rate': '3.725e-05', 'epoch': '0.3667', 'num_input_tokens_seen': 257205400, 'train_runtime': '5.634e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82500/225000 [156:29:34<141:45:34,  3.58s/it]
 37%|███▋      | 82501/225000 [156:29:38<138:12:59,  3.49s/it]
 37%|███▋      | 82502/225000 [156:29:46<200:36:18,  5.07s/it]
 37%|███▋      | 82503/225000 [156:29:52<204:59:51,  5.18s/it]
 37%|███▋      | 82504/225000 [156:29:55<177:13:08,  4.48s/it]
 37%|███▋      | 82505/225000 [156:29:58<158:31:20,  4.00s/it]
 37%|███▋      | 82506/225000 [156:30:00<142:33:23,  3.60s/it]
 37%|███▋      | 82507/225000 [156:30:03<131:55:56,  3.33s/it]
 37%|███▋      | 82508/225000 [156:30:06<124:20:52,  3.14s/it]
 37%|███▋      | 82509/225000 [156:30:15<197:11:40,  4.98s/it]
 37%|███▋      | 82510/225000 [156:30:18<179:38:49,  4.54s/it]
 37%|███▋      | 82511/225000 [156:30:22<164:26:18,  4.15s/it]
 37%|███▋      | 82512/225000 [156:30:25<157:25:39,  3.98s/it]
 37%|███▋      | 82513/225000 [156:30:29<151:31:58,  3.83s/it]
 37%|███▋      | 82514/225000 [156:30:32<141:23:55,  3.57s/it]
 37%|███▋      | 82515/225000 [156:30:35<134:00:40,  3.39s/it]
 37%|███▋      | 82516/225000 [156:30:42<179:50:15,  4.54s/it]
 37%|███▋      | 82517/225000 [156:30:49<205:07:13,  5.18s/it]
 37%|███▋      | 82518/225000 [156:30:52<185:23:24,  4.68s/it]
 37%|███▋      | 82519/225000 [156:30:55<164:53:55,  4.17s/it]
 37%|███▋      | 82520/225000 [156:30:58<156:10:35,  3.95s/it]
 37%|███▋      | 82521/225000 [156:31:02<148:45:09,  3.76s/it]
 37%|███▋      | 82522/225000 [156:31:05<144:08:01,  3.64s/it]
 37%|███▋      | 82523/225000 [156:31:09<144:37:19,  3.65s/it]
 37%|███▋      | 82524/225000 [156:31:12<140:23:53,  3.55s/it]
 37%|███▋      | 82525/225000 [156:31:16<139:24:28,  3.52s/it]
 37%|███▋      | 82526/225000 [156:31:19<141:04:44,  3.56s/it]
 37%|███▋      | 82527/225000 [156:31:22<136:42:14,  3.45s/it]
 37%|███▋      | 82528/225000 [156:31:31<192:47:34,  4.87s/it]
 37%|███▋      | 82529/225000 [156:31:36<200:09:27,  5.06s/it]
 37%|███▋      | 82530/225000 [156:31:39<176:04:26,  4.45s/it]
 37%|███▋      | 82531/225000 [156:31:42<159:40:38,  4.03s/it]
 37%|███▋      | 82532/225000 [156:31:47<167:26:21,  4.23s/it]
 37%|███▋      | 82533/225000 [156:31:55<216:48:41,  5.48s/it]
 37%|███▋      | 82534/225000 [156:31:58<178:59:48,  4.52s/it]
 37%|███▋      | 82535/225000 [156:32:00<153:19:47,  3.87s/it]
 37%|███▋      | 82536/225000 [156:32:03<144:37:57,  3.65s/it]
 37%|███▋      | 82537/225000 [156:32:06<133:26:12,  3.37s/it]
 37%|███▋      | 82538/225000 [156:32:10<148:23:12,  3.75s/it]
 37%|███▋      | 82539/225000 [156:32:19<204:40:13,  5.17s/it]
 37%|███▋      | 82540/225000 [156:32:22<179:41:10,  4.54s/it]
 37%|███▋      | 82541/225000 [156:32:25<155:56:15,  3.94s/it]
 37%|███▋      | 82542/225000 [156:32:28<144:52:47,  3.66s/it]
 37%|███▋      | 82543/225000 [156:32:31<141:30:44,  3.58s/it]
 37%|███▋      | 82544/225000 [156:32:35<142:56:37,  3.61s/it]
 37%|███▋      | 82545/225000 [156:32:38<144:48:42,  3.66s/it]
 37%|███▋      | 82546/225000 [156:32:42<143:26:50,  3.63s/it]
 37%|███▋      | 82547/225000 [156:32:46<143:36:15,  3.63s/it]
 37%|███▋      | 82548/225000 [156:32:49<140:21:21,  3.55s/it]
 37%|███▋      | 82549/225000 [156:32:53<141:20:03,  3.57s/it]
 37%|███▋      | 82550/225000 [156:32:56<137:30:12,  3.47s/it]
                                                              
{'loss': '0.6198', 'grad_norm': '6.062', 'learning_rate': '3.723e-05', 'epoch': '0.3669', 'num_input_tokens_seen': 257291784, 'train_runtime': '5.636e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82550/225000 [156:32:56<137:30:12,  3.47s/it]
 37%|███▋      | 82551/225000 [156:32:59<137:04:48,  3.46s/it]
 37%|███▋      | 82552/225000 [156:33:03<141:25:33,  3.57s/it]
 37%|███▋      | 82553/225000 [156:33:08<158:19:36,  4.00s/it]
 37%|███▋      | 82554/225000 [156:33:18<224:37:55,  5.68s/it]
 37%|███▋      | 82555/225000 [156:33:23<216:40:25,  5.48s/it]
 37%|███▋      | 82556/225000 [156:33:25<184:13:34,  4.66s/it]
 37%|███▋      | 82557/225000 [156:33:29<167:37:15,  4.24s/it]
 37%|███▋      | 82558/225000 [156:33:33<163:47:14,  4.14s/it]
 37%|███▋      | 82559/225000 [156:33:41<215:03:20,  5.44s/it]
 37%|███▋      | 82560/225000 [156:33:44<184:24:07,  4.66s/it]
 37%|███▋      | 82561/225000 [156:33:46<156:44:22,  3.96s/it]
 37%|███▋      | 82562/225000 [156:33:53<192:44:33,  4.87s/it]
 37%|███▋      | 82563/225000 [156:33:57<173:39:01,  4.39s/it]
 37%|███▋      | 82564/225000 [156:34:00<161:04:09,  4.07s/it]
 37%|███▋      | 82565/225000 [156:34:02<144:15:39,  3.65s/it]
 37%|███▋      | 82566/225000 [156:34:10<188:06:29,  4.75s/it]
 37%|███▋      | 82567/225000 [156:34:14<182:10:43,  4.60s/it]
 37%|███▋      | 82568/225000 [156:34:17<163:33:01,  4.13s/it]
 37%|███▋      | 82569/225000 [156:34:20<145:50:33,  3.69s/it]
 37%|███▋      | 82570/225000 [156:34:29<212:26:35,  5.37s/it]
 37%|███▋      | 82571/225000 [156:34:32<187:52:13,  4.75s/it]
 37%|███▋      | 82572/225000 [156:34:35<167:10:10,  4.23s/it]
 37%|███▋      | 82573/225000 [156:34:39<158:16:24,  4.00s/it]
 37%|███▋      | 82574/225000 [156:34:42<147:59:36,  3.74s/it]
 37%|███▋      | 82575/225000 [156:34:45<140:22:07,  3.55s/it]
 37%|███▋      | 82576/225000 [156:34:49<141:42:39,  3.58s/it]
 37%|███▋      | 82577/225000 [156:34:55<174:02:26,  4.40s/it]
 37%|███▋      | 82578/225000 [156:34:59<168:48:24,  4.27s/it]
 37%|███▋      | 82579/225000 [156:35:02<154:31:49,  3.91s/it]
 37%|███▋      | 82580/225000 [156:35:05<146:57:10,  3.71s/it]
 37%|███▋      | 82581/225000 [156:35:09<140:57:55,  3.56s/it]
 37%|███▋      | 82582/225000 [156:35:11<131:49:04,  3.33s/it]
 37%|███▋      | 82583/225000 [156:35:15<130:34:00,  3.30s/it]
 37%|███▋      | 82584/225000 [156:35:23<190:09:04,  4.81s/it]
 37%|███▋      | 82585/225000 [156:35:27<179:56:56,  4.55s/it]
 37%|███▋      | 82586/225000 [156:35:30<164:59:12,  4.17s/it]
 37%|███▋      | 82587/225000 [156:35:34<156:01:04,  3.94s/it]
 37%|███▋      | 82588/225000 [156:35:37<151:42:31,  3.84s/it]
 37%|███▋      | 82589/225000 [156:35:40<141:40:16,  3.58s/it]
 37%|███▋      | 82590/225000 [156:35:43<135:46:36,  3.43s/it]
 37%|███▋      | 82591/225000 [156:35:47<138:16:44,  3.50s/it]
 37%|███▋      | 82592/225000 [156:35:50<134:55:41,  3.41s/it]
 37%|███▋      | 82593/225000 [156:35:53<131:47:27,  3.33s/it]
 37%|███▋      | 82594/225000 [156:35:57<133:02:38,  3.36s/it]
 37%|███▋      | 82595/225000 [156:36:01<142:49:11,  3.61s/it]
 37%|███▋      | 82596/225000 [156:36:05<146:19:06,  3.70s/it]
 37%|███▋      | 82597/225000 [156:36:13<195:45:03,  4.95s/it]
 37%|███▋      | 82598/225000 [156:36:22<244:29:43,  6.18s/it]
 37%|███▋      | 82599/225000 [156:36:25<207:35:17,  5.25s/it]
 37%|███▋      | 82600/225000 [156:36:28<181:22:41,  4.59s/it]
                                                              
{'loss': '0.6088', 'grad_norm': '4.571', 'learning_rate': '3.722e-05', 'epoch': '0.3671', 'num_input_tokens_seen': 257378664, 'train_runtime': '5.638e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82600/225000 [156:36:28<181:22:41,  4.59s/it]
 37%|███▋      | 82601/225000 [156:36:31<162:54:07,  4.12s/it]
 37%|███▋      | 82602/225000 [156:36:34<153:46:20,  3.89s/it]
 37%|███▋      | 82603/225000 [156:36:37<142:07:08,  3.59s/it]
 37%|███▋      | 82604/225000 [156:36:40<138:47:39,  3.51s/it]
 37%|███▋      | 82605/225000 [156:36:43<133:17:37,  3.37s/it]
 37%|███▋      | 82606/225000 [156:36:46<127:08:10,  3.21s/it]
 37%|███▋      | 82607/225000 [156:36:55<189:41:36,  4.80s/it]
 37%|███▋      | 82608/225000 [156:37:02<213:58:17,  5.41s/it]
 37%|███▋      | 82609/225000 [156:37:04<180:59:41,  4.58s/it]
 37%|███▋      | 82610/225000 [156:37:07<158:14:02,  4.00s/it]
 37%|███▋      | 82611/225000 [156:37:12<174:28:34,  4.41s/it]
 37%|███▋      | 82612/225000 [156:37:17<183:43:41,  4.65s/it]
 37%|███▋      | 82613/225000 [156:37:20<162:06:32,  4.10s/it]
 37%|███▋      | 82614/225000 [156:37:23<142:44:08,  3.61s/it]
 37%|███▋      | 82615/225000 [156:37:26<135:11:26,  3.42s/it]
 37%|███▋      | 82616/225000 [156:37:28<126:43:38,  3.20s/it]
 37%|███▋      | 82617/225000 [156:37:35<173:00:04,  4.37s/it]
 37%|███▋      | 82618/225000 [156:37:40<173:06:39,  4.38s/it]
 37%|███▋      | 82619/225000 [156:37:43<163:08:51,  4.13s/it]
 37%|███▋      | 82620/225000 [156:37:47<152:48:35,  3.86s/it]
 37%|███▋      | 82621/225000 [156:37:50<145:16:07,  3.67s/it]
 37%|███▋      | 82622/225000 [156:37:53<139:04:52,  3.52s/it]
 37%|███▋      | 82623/225000 [156:37:56<135:18:01,  3.42s/it]
 37%|███▋      | 82624/225000 [156:38:00<133:57:43,  3.39s/it]
 37%|███▋      | 82625/225000 [156:38:07<186:04:25,  4.70s/it]
 37%|███▋      | 82626/225000 [156:38:13<196:12:44,  4.96s/it]
 37%|███▋      | 82627/225000 [156:38:16<170:40:26,  4.32s/it]
 37%|███▋      | 82628/225000 [156:38:19<157:16:59,  3.98s/it]
 37%|███▋      | 82629/225000 [156:38:22<148:02:55,  3.74s/it]
 37%|███▋      | 82630/225000 [156:38:26<146:39:01,  3.71s/it]
 37%|███▋      | 82631/225000 [156:38:29<141:33:16,  3.58s/it]
 37%|███▋      | 82632/225000 [156:38:34<158:13:22,  4.00s/it]
 37%|███▋      | 82633/225000 [156:38:43<212:21:48,  5.37s/it]
 37%|███▋      | 82634/225000 [156:38:45<179:39:35,  4.54s/it]
 37%|███▋      | 82635/225000 [156:38:49<176:50:50,  4.47s/it]
 37%|███▋      | 82636/225000 [156:38:55<191:21:47,  4.84s/it]
 37%|███▋      | 82637/225000 [156:38:58<166:52:07,  4.22s/it]
 37%|███▋      | 82638/225000 [156:39:01<154:09:15,  3.90s/it]
 37%|███▋      | 82639/225000 [156:39:05<152:04:49,  3.85s/it]
 37%|███▋      | 82640/225000 [156:39:08<147:07:26,  3.72s/it]
 37%|███▋      | 82641/225000 [156:39:11<139:38:37,  3.53s/it]
 37%|███▋      | 82642/225000 [156:39:15<136:04:02,  3.44s/it]
 37%|███▋      | 82643/225000 [156:39:19<150:28:58,  3.81s/it]
 37%|███▋      | 82644/225000 [156:39:28<204:33:19,  5.17s/it]
 37%|███▋      | 82645/225000 [156:39:30<171:53:26,  4.35s/it]
 37%|███▋      | 82646/225000 [156:39:35<177:29:21,  4.49s/it]
 37%|███▋      | 82647/225000 [156:39:39<170:54:03,  4.32s/it]
 37%|███▋      | 82648/225000 [156:39:42<161:01:20,  4.07s/it]
 37%|███▋      | 82649/225000 [156:39:46<155:05:32,  3.92s/it]
 37%|███▋      | 82650/225000 [156:39:49<150:02:04,  3.79s/it]
                                                              
{'loss': '0.613', 'grad_norm': '2.911', 'learning_rate': '3.72e-05', 'epoch': '0.3673', 'num_input_tokens_seen': 257465872, 'train_runtime': '5.64e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82650/225000 [156:39:49<150:02:04,  3.79s/it]
 37%|███▋      | 82651/225000 [156:39:53<143:19:20,  3.62s/it]
 37%|███▋      | 82652/225000 [156:39:56<140:07:22,  3.54s/it]
 37%|███▋      | 82653/225000 [156:39:59<140:48:10,  3.56s/it]
 37%|███▋      | 82654/225000 [156:40:03<140:38:25,  3.56s/it]
 37%|███▋      | 82655/225000 [156:40:06<138:06:49,  3.49s/it]
 37%|███▋      | 82656/225000 [156:40:14<185:00:33,  4.68s/it]
 37%|███▋      | 82657/225000 [156:40:17<168:17:02,  4.26s/it]
 37%|███▋      | 82658/225000 [156:40:20<155:19:44,  3.93s/it]
 37%|███▋      | 82659/225000 [156:40:24<152:10:14,  3.85s/it]
 37%|███▋      | 82660/225000 [156:40:28<152:12:52,  3.85s/it]
 37%|███▋      | 82661/225000 [156:40:31<145:52:25,  3.69s/it]
 37%|███▋      | 82662/225000 [156:40:38<184:29:51,  4.67s/it]
 37%|███▋      | 82663/225000 [156:40:43<192:32:58,  4.87s/it]
 37%|███▋      | 82664/225000 [156:40:46<171:48:54,  4.35s/it]
 37%|███▋      | 82665/225000 [156:40:50<161:18:38,  4.08s/it]
 37%|███▋      | 82666/225000 [156:40:53<151:17:00,  3.83s/it]
 37%|███▋      | 82667/225000 [156:40:57<148:15:13,  3.75s/it]
 37%|███▋      | 82668/225000 [156:41:00<145:08:34,  3.67s/it]
 37%|███▋      | 82669/225000 [156:41:04<144:22:41,  3.65s/it]
 37%|███▋      | 82670/225000 [156:41:07<143:41:39,  3.63s/it]
 37%|███▋      | 82671/225000 [156:41:11<137:12:34,  3.47s/it]
 37%|███▋      | 82672/225000 [156:41:14<141:06:07,  3.57s/it]
 37%|███▋      | 82673/225000 [156:41:18<137:40:55,  3.48s/it]
 37%|███▋      | 82674/225000 [156:41:21<139:07:59,  3.52s/it]
 37%|███▋      | 82675/225000 [156:41:25<144:47:17,  3.66s/it]
 37%|███▋      | 82676/225000 [156:41:29<149:16:01,  3.78s/it]
 37%|███▋      | 82677/225000 [156:41:33<145:04:29,  3.67s/it]
 37%|███▋      | 82678/225000 [156:41:36<143:27:35,  3.63s/it]
 37%|███▋      | 82679/225000 [156:41:40<140:36:06,  3.56s/it]
 37%|███▋      | 82680/225000 [156:41:48<198:25:51,  5.02s/it]
 37%|███▋      | 82681/225000 [156:41:55<222:52:26,  5.64s/it]
 37%|███▋      | 82682/225000 [156:41:58<190:30:48,  4.82s/it]
 37%|███▋      | 82683/225000 [156:42:01<171:04:11,  4.33s/it]
 37%|███▋      | 82684/225000 [156:42:04<158:21:22,  4.01s/it]
 37%|███▋      | 82685/225000 [156:42:08<147:02:39,  3.72s/it]
 37%|███▋      | 82686/225000 [156:42:10<135:40:33,  3.43s/it]
 37%|███▋      | 82687/225000 [156:42:16<162:03:33,  4.10s/it]
 37%|███▋      | 82688/225000 [156:42:22<181:56:50,  4.60s/it]
 37%|███▋      | 82689/225000 [156:42:26<181:58:55,  4.60s/it]
 37%|███▋      | 82690/225000 [156:42:29<157:51:16,  3.99s/it]
 37%|███▋      | 82691/225000 [156:42:32<144:14:51,  3.65s/it]
 37%|███▋      | 82692/225000 [156:42:34<133:41:54,  3.38s/it]
 37%|███▋      | 82693/225000 [156:42:37<129:20:14,  3.27s/it]
 37%|███▋      | 82694/225000 [156:42:46<189:24:50,  4.79s/it]
 37%|███▋      | 82695/225000 [156:42:52<203:02:23,  5.14s/it]
 37%|███▋      | 82696/225000 [156:42:55<176:31:15,  4.47s/it]
 37%|███▋      | 82697/225000 [156:42:58<157:45:31,  3.99s/it]
 37%|███▋      | 82698/225000 [156:43:01<150:36:41,  3.81s/it]
 37%|███▋      | 82699/225000 [156:43:05<150:11:25,  3.80s/it]
 37%|███▋      | 82700/225000 [156:43:08<141:56:33,  3.59s/it]
                                                              
{'loss': '0.6762', 'grad_norm': '4.996', 'learning_rate': '3.718e-05', 'epoch': '0.3676', 'num_input_tokens_seen': 257551944, 'train_runtime': '5.642e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82700/225000 [156:43:08<141:56:33,  3.59s/it]
 37%|███▋      | 82701/225000 [156:43:11<132:19:19,  3.35s/it]
 37%|███▋      | 82702/225000 [156:43:14<129:44:03,  3.28s/it]
 37%|███▋      | 82703/225000 [156:43:21<176:42:31,  4.47s/it]
 37%|███▋      | 82704/225000 [156:43:30<229:00:40,  5.79s/it]
 37%|███▋      | 82705/225000 [156:43:32<189:16:14,  4.79s/it]
 37%|███▋      | 82706/225000 [156:43:35<162:19:42,  4.11s/it]
 37%|███▋      | 82707/225000 [156:43:38<151:17:09,  3.83s/it]
 37%|███▋      | 82708/225000 [156:43:41<143:39:42,  3.63s/it]
 37%|███▋      | 82709/225000 [156:43:45<143:25:20,  3.63s/it]
 37%|███▋      | 82710/225000 [156:43:48<139:34:21,  3.53s/it]
 37%|███▋      | 82711/225000 [156:43:51<135:25:10,  3.43s/it]
 37%|███▋      | 82712/225000 [156:43:55<134:02:38,  3.39s/it]
 37%|███▋      | 82713/225000 [156:43:58<131:03:04,  3.32s/it]
 37%|███▋      | 82714/225000 [156:44:01<131:19:38,  3.32s/it]
 37%|███▋      | 82715/225000 [156:44:04<128:25:36,  3.25s/it]
 37%|███▋      | 82716/225000 [156:44:07<129:07:37,  3.27s/it]
 37%|███▋      | 82717/225000 [156:44:16<188:08:30,  4.76s/it]
 37%|███▋      | 82718/225000 [156:44:22<205:29:05,  5.20s/it]
 37%|███▋      | 82719/225000 [156:44:25<174:34:53,  4.42s/it]
 37%|███▋      | 82720/225000 [156:44:28<160:52:57,  4.07s/it]
 37%|███▋      | 82721/225000 [156:44:34<186:21:51,  4.72s/it]
 37%|███▋      | 82722/225000 [156:44:36<159:22:38,  4.03s/it]
 37%|███▋      | 82723/225000 [156:44:39<143:14:11,  3.62s/it]
 37%|███▋      | 82724/225000 [156:44:42<130:08:39,  3.29s/it]
 37%|███▋      | 82725/225000 [156:44:45<135:06:26,  3.42s/it]
 37%|███▋      | 82726/225000 [156:44:53<188:41:43,  4.77s/it]
 37%|███▋      | 82727/225000 [156:44:58<185:09:15,  4.69s/it]
 37%|███▋      | 82728/225000 [156:45:01<162:33:06,  4.11s/it]
 37%|███▋      | 82729/225000 [156:45:04<151:02:37,  3.82s/it]
 37%|███▋      | 82730/225000 [156:45:07<141:59:36,  3.59s/it]
 37%|███▋      | 82731/225000 [156:45:09<130:58:47,  3.31s/it]
 37%|███▋      | 82732/225000 [156:45:17<178:18:00,  4.51s/it]
 37%|███▋      | 82733/225000 [156:45:23<194:26:54,  4.92s/it]
 37%|███▋      | 82734/225000 [156:45:25<166:41:27,  4.22s/it]
 37%|███▋      | 82735/225000 [156:45:28<150:13:44,  3.80s/it]
 37%|███▋      | 82736/225000 [156:45:36<205:34:42,  5.20s/it]
 37%|███▋      | 82737/225000 [156:45:40<185:40:59,  4.70s/it]
 37%|███▋      | 82738/225000 [156:45:43<165:50:52,  4.20s/it]
 37%|███▋      | 82739/225000 [156:45:46<145:37:39,  3.69s/it]
 37%|███▋      | 82740/225000 [156:45:48<133:37:06,  3.38s/it]
 37%|███▋      | 82741/225000 [156:45:56<181:59:55,  4.61s/it]
 37%|███▋      | 82742/225000 [156:45:59<165:08:27,  4.18s/it]
 37%|███▋      | 82743/225000 [156:46:02<149:33:18,  3.78s/it]
 37%|███▋      | 82744/225000 [156:46:04<136:11:27,  3.45s/it]
 37%|███▋      | 82745/225000 [156:46:07<126:31:51,  3.20s/it]
 37%|███▋      | 82746/225000 [156:46:10<123:27:53,  3.12s/it]
 37%|███▋      | 82747/225000 [156:46:14<134:43:25,  3.41s/it]
 37%|███▋      | 82748/225000 [156:46:23<196:36:50,  4.98s/it]
 37%|███▋      | 82749/225000 [156:46:26<173:43:16,  4.40s/it]
 37%|███▋      | 82750/225000 [156:46:28<152:05:49,  3.85s/it]
                                                              
{'loss': '0.5722', 'grad_norm': '6.467', 'learning_rate': '3.717e-05', 'epoch': '0.3678', 'num_input_tokens_seen': 257638832, 'train_runtime': '5.644e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82750/225000 [156:46:28<152:05:49,  3.85s/it]
 37%|███▋      | 82751/225000 [156:46:33<166:04:37,  4.20s/it]
 37%|███▋      | 82752/225000 [156:46:38<175:53:38,  4.45s/it]
 37%|███▋      | 82753/225000 [156:46:42<161:57:12,  4.10s/it]
 37%|███▋      | 82754/225000 [156:46:45<149:38:59,  3.79s/it]
 37%|███▋      | 82755/225000 [156:46:48<138:46:49,  3.51s/it]
 37%|███▋      | 82756/225000 [156:46:51<143:35:56,  3.63s/it]
 37%|███▋      | 82757/225000 [156:46:54<136:25:19,  3.45s/it]
 37%|███▋      | 82758/225000 [156:46:58<131:57:39,  3.34s/it]
 37%|███▋      | 82759/225000 [156:47:02<144:48:11,  3.66s/it]
 37%|███▋      | 82760/225000 [156:47:11<206:57:31,  5.24s/it]
 37%|███▋      | 82761/225000 [156:47:14<181:14:09,  4.59s/it]
 37%|███▋      | 82762/225000 [156:47:17<159:54:58,  4.05s/it]
 37%|███▋      | 82763/225000 [156:47:20<151:13:24,  3.83s/it]
 37%|███▋      | 82764/225000 [156:47:23<142:31:10,  3.61s/it]
 37%|███▋      | 82765/225000 [156:47:26<136:29:53,  3.45s/it]
 37%|███▋      | 82766/225000 [156:47:30<136:23:58,  3.45s/it]
 37%|███▋      | 82767/225000 [156:47:39<200:38:45,  5.08s/it]
 37%|███▋      | 82768/225000 [156:47:45<220:02:00,  5.57s/it]
 37%|███▋      | 82769/225000 [156:47:48<181:58:48,  4.61s/it]
 37%|███▋      | 82770/225000 [156:47:50<156:31:49,  3.96s/it]
 37%|███▋      | 82771/225000 [156:47:55<164:43:47,  4.17s/it]
 37%|███▋      | 82772/225000 [156:47:59<171:34:34,  4.34s/it]
 37%|███▋      | 82773/225000 [156:48:02<151:02:29,  3.82s/it]
 37%|███▋      | 82774/225000 [156:48:10<197:49:29,  5.01s/it]
 37%|███▋      | 82775/225000 [156:48:13<180:07:53,  4.56s/it]
 37%|███▋      | 82776/225000 [156:48:16<162:57:57,  4.13s/it]
 37%|███▋      | 82777/225000 [156:48:19<148:01:49,  3.75s/it]
 37%|███▋      | 82778/225000 [156:48:22<134:55:45,  3.42s/it]
 37%|███▋      | 82779/225000 [156:48:25<132:00:19,  3.34s/it]
 37%|███▋      | 82780/225000 [156:48:28<128:58:51,  3.26s/it]
 37%|███▋      | 82781/225000 [156:48:32<134:41:01,  3.41s/it]
 37%|███▋      | 82782/225000 [156:48:36<143:07:41,  3.62s/it]
 37%|███▋      | 82783/225000 [156:48:47<227:10:49,  5.75s/it]
 37%|███▋      | 82784/225000 [156:48:50<190:48:13,  4.83s/it]
 37%|███▋      | 82785/225000 [156:48:53<171:39:11,  4.35s/it]
 37%|███▋      | 82786/225000 [156:48:56<162:27:12,  4.11s/it]
 37%|███▋      | 82787/225000 [156:48:59<149:06:00,  3.77s/it]
 37%|███▋      | 82788/225000 [156:49:07<194:02:08,  4.91s/it]
 37%|███▋      | 82789/225000 [156:49:12<195:33:04,  4.95s/it]
 37%|███▋      | 82790/225000 [156:49:14<165:48:46,  4.20s/it]
 37%|███▋      | 82791/225000 [156:49:21<193:02:48,  4.89s/it]
 37%|███▋      | 82792/225000 [156:49:24<174:08:17,  4.41s/it]
 37%|███▋      | 82793/225000 [156:49:27<155:53:45,  3.95s/it]
 37%|███▋      | 82794/225000 [156:49:30<143:16:52,  3.63s/it]
 37%|███▋      | 82795/225000 [156:49:38<200:20:51,  5.07s/it]
 37%|███▋      | 82796/225000 [156:49:41<176:11:46,  4.46s/it]
 37%|███▋      | 82797/225000 [156:49:44<157:00:01,  3.97s/it]
 37%|███▋      | 82798/225000 [156:49:51<194:28:30,  4.92s/it]
 37%|███▋      | 82799/225000 [156:49:55<176:52:57,  4.48s/it]
 37%|███▋      | 82800/225000 [156:49:57<155:32:05,  3.94s/it]
                                                              
{'loss': '0.596', 'grad_norm': '4.848', 'learning_rate': '3.715e-05', 'epoch': '0.368', 'num_input_tokens_seen': 257726984, 'train_runtime': '5.646e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82800/225000 [156:49:57<155:32:05,  3.94s/it]
 37%|███▋      | 82801/225000 [156:50:00<137:44:21,  3.49s/it]
 37%|███▋      | 82802/225000 [156:50:02<124:50:15,  3.16s/it]
 37%|███▋      | 82803/225000 [156:50:08<160:19:23,  4.06s/it]
 37%|███▋      | 82804/225000 [156:50:15<186:51:57,  4.73s/it]
 37%|███▋      | 82805/225000 [156:50:17<161:51:58,  4.10s/it]
 37%|███▋      | 82806/225000 [156:50:20<145:43:06,  3.69s/it]
 37%|███▋      | 82807/225000 [156:50:23<132:40:16,  3.36s/it]
 37%|███▋      | 82808/225000 [156:50:30<182:46:30,  4.63s/it]
 37%|███▋      | 82809/225000 [156:50:35<179:01:27,  4.53s/it]
 37%|███▋      | 82810/225000 [156:50:37<158:02:47,  4.00s/it]
 37%|███▋      | 82811/225000 [156:50:40<140:27:14,  3.56s/it]
 37%|███▋      | 82812/225000 [156:50:43<137:40:14,  3.49s/it]
 37%|███▋      | 82813/225000 [156:50:51<191:33:16,  4.85s/it]
 37%|███▋      | 82814/225000 [156:50:54<161:35:28,  4.09s/it]
 37%|███▋      | 82815/225000 [156:50:56<140:44:27,  3.56s/it]
 37%|███▋      | 82816/225000 [156:51:01<153:41:31,  3.89s/it]
 37%|███▋      | 82817/225000 [156:51:05<159:39:26,  4.04s/it]
 37%|███▋      | 82818/225000 [156:51:08<143:44:05,  3.64s/it]
 37%|███▋      | 82819/225000 [156:51:10<128:13:30,  3.25s/it]
 37%|███▋      | 82820/225000 [156:51:17<168:17:36,  4.26s/it]
 37%|███▋      | 82821/225000 [156:51:21<172:47:46,  4.38s/it]
 37%|███▋      | 82822/225000 [156:51:24<150:44:47,  3.82s/it]
 37%|███▋      | 82823/225000 [156:51:29<169:48:26,  4.30s/it]
 37%|███▋      | 82824/225000 [156:51:33<165:18:32,  4.19s/it]
 37%|███▋      | 82825/225000 [156:51:36<147:17:37,  3.73s/it]
 37%|███▋      | 82826/225000 [156:51:39<138:42:26,  3.51s/it]
 37%|███▋      | 82827/225000 [156:51:41<128:45:35,  3.26s/it]
 37%|███▋      | 82828/225000 [156:51:44<126:32:31,  3.20s/it]
 37%|███▋      | 82829/225000 [156:51:48<131:12:31,  3.32s/it]
 37%|███▋      | 82830/225000 [156:51:54<164:33:10,  4.17s/it]
 37%|███▋      | 82831/225000 [156:52:03<216:29:00,  5.48s/it]
 37%|███▋      | 82832/225000 [156:52:06<188:36:22,  4.78s/it]
 37%|███▋      | 82833/225000 [156:52:09<171:46:00,  4.35s/it]
 37%|███▋      | 82834/225000 [156:52:13<163:27:02,  4.14s/it]
 37%|███▋      | 82835/225000 [156:52:16<149:37:39,  3.79s/it]
 37%|███▋      | 82836/225000 [156:52:22<180:45:17,  4.58s/it]
 37%|███▋      | 82837/225000 [156:52:25<162:45:47,  4.12s/it]
 37%|███▋      | 82838/225000 [156:52:28<146:05:03,  3.70s/it]
 37%|███▋      | 82839/225000 [156:52:31<136:18:38,  3.45s/it]
 37%|███▋      | 82840/225000 [156:52:34<133:56:32,  3.39s/it]
 37%|███▋      | 82841/225000 [156:52:37<124:11:44,  3.15s/it]
 37%|███▋      | 82842/225000 [156:52:45<187:55:52,  4.76s/it]
 37%|███▋      | 82843/225000 [156:52:50<190:50:55,  4.83s/it]
 37%|███▋      | 82844/225000 [156:52:54<173:18:09,  4.39s/it]
 37%|███▋      | 82845/225000 [156:52:57<159:23:13,  4.04s/it]
 37%|███▋      | 82846/225000 [156:53:00<151:22:21,  3.83s/it]
 37%|███▋      | 82847/225000 [156:53:03<142:28:32,  3.61s/it]
 37%|███▋      | 82848/225000 [156:53:07<139:58:47,  3.54s/it]
 37%|███▋      | 82849/225000 [156:53:10<134:03:36,  3.40s/it]
 37%|███▋      | 82850/225000 [156:53:14<142:25:42,  3.61s/it]
                                                              
{'loss': '0.6442', 'grad_norm': '3.18', 'learning_rate': '3.714e-05', 'epoch': '0.3682', 'num_input_tokens_seen': 257813520, 'train_runtime': '5.648e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82850/225000 [156:53:14<142:25:42,  3.61s/it]
 37%|███▋      | 82851/225000 [156:53:17<140:13:44,  3.55s/it]
 37%|███▋      | 82852/225000 [156:53:21<143:44:52,  3.64s/it]
 37%|███▋      | 82853/225000 [156:53:28<186:08:43,  4.71s/it]
 37%|███▋      | 82854/225000 [156:53:37<238:48:08,  6.05s/it]
 37%|███▋      | 82855/225000 [156:53:40<196:16:43,  4.97s/it]
 37%|███▋      | 82856/225000 [156:53:42<167:38:22,  4.25s/it]
 37%|███▋      | 82857/225000 [156:53:46<163:52:02,  4.15s/it]
 37%|███▋      | 82858/225000 [156:53:52<183:44:06,  4.65s/it]
 37%|███▋      | 82859/225000 [156:53:57<189:02:17,  4.79s/it]
 37%|███▋      | 82860/225000 [156:54:00<166:16:15,  4.21s/it]
 37%|███▋      | 82861/225000 [156:54:03<151:26:15,  3.84s/it]
 37%|███▋      | 82862/225000 [156:54:07<149:18:19,  3.78s/it]
 37%|███▋      | 82863/225000 [156:54:10<138:12:24,  3.50s/it]
 37%|███▋      | 82864/225000 [156:54:13<134:00:23,  3.39s/it]
 37%|███▋      | 82865/225000 [156:54:16<127:43:01,  3.23s/it]
 37%|███▋      | 82866/225000 [156:54:19<126:14:34,  3.20s/it]
 37%|███▋      | 82867/225000 [156:54:27<185:31:14,  4.70s/it]
 37%|███▋      | 82868/225000 [156:54:34<213:22:39,  5.40s/it]
 37%|███▋      | 82869/225000 [156:54:37<190:10:30,  4.82s/it]
 37%|███▋      | 82870/225000 [156:54:41<170:08:01,  4.31s/it]
 37%|███▋      | 82871/225000 [156:54:44<155:38:54,  3.94s/it]
 37%|███▋      | 82872/225000 [156:54:47<142:34:00,  3.61s/it]
 37%|███▋      | 82873/225000 [156:54:49<134:14:47,  3.40s/it]
 37%|███▋      | 82874/225000 [156:54:53<130:36:05,  3.31s/it]
 37%|███▋      | 82875/225000 [156:54:56<132:40:42,  3.36s/it]
 37%|███▋      | 82876/225000 [156:54:59<126:54:31,  3.21s/it]
 37%|███▋      | 82877/225000 [156:55:02<131:12:17,  3.32s/it]
 37%|███▋      | 82878/225000 [156:55:11<192:53:02,  4.89s/it]
 37%|███▋      | 82879/225000 [156:55:17<202:45:56,  5.14s/it]
 37%|███▋      | 82880/225000 [156:55:19<172:57:55,  4.38s/it]
 37%|███▋      | 82881/225000 [156:55:24<176:35:04,  4.47s/it]
 37%|███▋      | 82882/225000 [156:55:30<198:32:16,  5.03s/it]
 37%|███▋      | 82883/225000 [156:55:33<169:14:40,  4.29s/it]
 37%|███▋      | 82884/225000 [156:55:36<156:38:27,  3.97s/it]
 37%|███▋      | 82885/225000 [156:55:40<150:01:35,  3.80s/it]
 37%|███▋      | 82886/225000 [156:55:45<164:37:31,  4.17s/it]
 37%|███▋      | 82887/225000 [156:55:52<199:58:08,  5.07s/it]
 37%|███▋      | 82888/225000 [156:55:54<172:02:41,  4.36s/it]
 37%|███▋      | 82889/225000 [156:55:57<149:49:20,  3.80s/it]
 37%|███▋      | 82890/225000 [156:56:04<185:39:20,  4.70s/it]
 37%|███▋      | 82891/225000 [156:56:06<161:46:49,  4.10s/it]
 37%|███▋      | 82892/225000 [156:56:09<144:00:30,  3.65s/it]
 37%|███▋      | 82893/225000 [156:56:12<134:16:50,  3.40s/it]
 37%|███▋      | 82894/225000 [156:56:15<129:07:33,  3.27s/it]
 37%|███▋      | 82895/225000 [156:56:25<205:44:59,  5.21s/it]
 37%|███▋      | 82896/225000 [156:56:27<173:03:38,  4.38s/it]
 37%|███▋      | 82897/225000 [156:56:30<151:49:43,  3.85s/it]
 37%|███▋      | 82898/225000 [156:56:33<142:45:51,  3.62s/it]
 37%|███▋      | 82899/225000 [156:56:37<146:08:42,  3.70s/it]
 37%|███▋      | 82900/225000 [156:56:40<142:13:50,  3.60s/it]
                                                              
{'loss': '0.6249', 'grad_norm': '3.715', 'learning_rate': '3.712e-05', 'epoch': '0.3684', 'num_input_tokens_seen': 257901168, 'train_runtime': '5.65e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82900/225000 [156:56:40<142:13:50,  3.60s/it]
 37%|███▋      | 82901/225000 [156:56:47<178:32:51,  4.52s/it]
 37%|███▋      | 82902/225000 [156:56:51<170:51:55,  4.33s/it]
 37%|███▋      | 82903/225000 [156:56:54<160:47:01,  4.07s/it]
 37%|███▋      | 82904/225000 [156:56:58<156:17:39,  3.96s/it]
 37%|███▋      | 82905/225000 [156:57:01<147:32:39,  3.74s/it]
 37%|███▋      | 82906/225000 [156:57:04<144:35:31,  3.66s/it]
 37%|███▋      | 82907/225000 [156:57:08<142:58:45,  3.62s/it]
 37%|███▋      | 82908/225000 [156:57:12<143:13:05,  3.63s/it]
 37%|███▋      | 82909/225000 [156:57:15<145:19:45,  3.68s/it]
 37%|███▋      | 82910/225000 [156:57:19<140:26:14,  3.56s/it]
 37%|███▋      | 82911/225000 [156:57:22<135:22:01,  3.43s/it]
 37%|███▋      | 82912/225000 [156:57:25<132:07:42,  3.35s/it]
 37%|███▋      | 82913/225000 [156:57:29<138:19:16,  3.50s/it]
 37%|███▋      | 82914/225000 [156:57:32<138:47:25,  3.52s/it]
 37%|███▋      | 82915/225000 [156:57:38<162:44:44,  4.12s/it]
 37%|███▋      | 82916/225000 [156:57:42<164:37:01,  4.17s/it]
 37%|███▋      | 82917/225000 [156:57:46<157:20:44,  3.99s/it]
 37%|███▋      | 82918/225000 [156:57:49<149:00:27,  3.78s/it]
 37%|███▋      | 82919/225000 [156:57:52<143:04:47,  3.63s/it]
 37%|███▋      | 82920/225000 [156:57:56<140:00:24,  3.55s/it]
 37%|███▋      | 82921/225000 [156:58:00<146:04:43,  3.70s/it]
 37%|███▋      | 82922/225000 [156:58:04<149:07:04,  3.78s/it]
 37%|███▋      | 82923/225000 [156:58:09<172:11:32,  4.36s/it]
 37%|███▋      | 82924/225000 [156:58:17<212:50:59,  5.39s/it]
 37%|███▋      | 82925/225000 [156:58:21<189:01:51,  4.79s/it]
 37%|███▋      | 82926/225000 [156:58:24<174:26:47,  4.42s/it]
 37%|███▋      | 82927/225000 [156:58:28<167:25:13,  4.24s/it]
 37%|███▋      | 82928/225000 [156:58:32<162:51:27,  4.13s/it]
 37%|███▋      | 82929/225000 [156:58:35<150:36:23,  3.82s/it]
 37%|███▋      | 82930/225000 [156:58:39<148:25:26,  3.76s/it]
 37%|███▋      | 82931/225000 [156:58:42<144:08:57,  3.65s/it]
 37%|███▋      | 82932/225000 [156:58:45<139:23:07,  3.53s/it]
 37%|███▋      | 82933/225000 [156:58:48<135:04:34,  3.42s/it]
 37%|███▋      | 82934/225000 [156:58:53<145:09:49,  3.68s/it]
 37%|███▋      | 82935/225000 [156:58:58<166:26:57,  4.22s/it]
 37%|███▋      | 82936/225000 [156:59:01<155:47:12,  3.95s/it]
 37%|███▋      | 82937/225000 [156:59:05<151:46:51,  3.85s/it]
 37%|███▋      | 82938/225000 [156:59:09<147:34:12,  3.74s/it]
 37%|███▋      | 82939/225000 [156:59:12<148:47:20,  3.77s/it]
 37%|███▋      | 82940/225000 [156:59:16<145:18:24,  3.68s/it]
 37%|███▋      | 82941/225000 [156:59:24<196:29:28,  4.98s/it]
 37%|███▋      | 82942/225000 [156:59:27<176:26:27,  4.47s/it]
 37%|███▋      | 82943/225000 [156:59:31<165:04:56,  4.18s/it]
 37%|███▋      | 82944/225000 [156:59:34<153:17:27,  3.88s/it]
 37%|███▋      | 82945/225000 [156:59:37<146:31:41,  3.71s/it]
 37%|███▋      | 82946/225000 [156:59:41<144:30:02,  3.66s/it]
 37%|███▋      | 82947/225000 [156:59:49<199:55:49,  5.07s/it]
 37%|███▋      | 82948/225000 [156:59:52<179:42:54,  4.55s/it]
 37%|███▋      | 82949/225000 [156:59:56<170:30:46,  4.32s/it]
 37%|███▋      | 82950/225000 [156:59:59<157:19:47,  3.99s/it]
                                                              
{'loss': '0.6378', 'grad_norm': '4.762', 'learning_rate': '3.71e-05', 'epoch': '0.3687', 'num_input_tokens_seen': 257988544, 'train_runtime': '5.652e+05', 'train_tokens_per_second': '456.5'}

 37%|███▋      | 82950/225000 [156:59:59<157:19:47,  3.99s/it]
 37%|███▋      | 82951/225000 [157:00:03<149:36:58,  3.79s/it]
 37%|███▋      | 82952/225000 [157:00:06<142:13:45,  3.60s/it]
 37%|███▋      | 82953/225000 [157:00:10<142:43:45,  3.62s/it]
 37%|███▋      | 82954/225000 [157:00:16<182:01:23,  4.61s/it]
 37%|███▋      | 82955/225000 [157:00:20<165:13:34,  4.19s/it]
 37%|███▋      | 82956/225000 [157:00:23<152:49:58,  3.87s/it]
 37%|███▋      | 82957/225000 [157:00:31<209:22:41,  5.31s/it]
 37%|███▋      | 82958/225000 [157:00:35<184:31:46,  4.68s/it]
 37%|███▋      | 82959/225000 [157:00:38<163:19:16,  4.14s/it]
 37%|███▋      | 82960/225000 [157:00:40<145:35:41,  3.69s/it]
 37%|███▋      | 82961/225000 [157:00:43<133:39:58,  3.39s/it]
 37%|███▋      | 82962/225000 [157:00:46<132:09:17,  3.35s/it]
 37%|███▋      | 82963/225000 [157:00:49<128:46:09,  3.26s/it]
 37%|███▋      | 82964/225000 [157:00:56<170:40:05,  4.33s/it]
 37%|███▋      | 82965/225000 [157:01:05<226:22:49,  5.74s/it]
 37%|███▋      | 82966/225000 [157:01:08<194:27:53,  4.93s/it]
 37%|███▋      | 82967/225000 [157:01:11<169:45:40,  4.30s/it]
 37%|███▋      | 82968/225000 [157:01:14<156:40:48,  3.97s/it]
 37%|███▋      | 82969/225000 [157:01:17<144:06:08,  3.65s/it]
 37%|███▋      | 82970/225000 [157:01:21<147:13:03,  3.73s/it]
 37%|███▋      | 82971/225000 [157:01:30<209:45:03,  5.32s/it]
 37%|███▋      | 82972/225000 [157:01:34<194:47:22,  4.94s/it]
 37%|███▋      | 82973/225000 [157:01:36<165:25:09,  4.19s/it]
 37%|███▋      | 82974/225000 [157:01:39<148:34:52,  3.77s/it]
 37%|███▋      | 82975/225000 [157:01:42<135:23:52,  3.43s/it]
 37%|███▋      | 82976/225000 [157:01:48<162:10:24,  4.11s/it]
 37%|███▋      | 82977/225000 [157:01:51<152:22:22,  3.86s/it]
 37%|███▋      | 82978/225000 [157:01:54<144:47:32,  3.67s/it]
 37%|███▋      | 82979/225000 [157:01:58<147:16:05,  3.73s/it]
 37%|███▋      | 82980/225000 [157:02:05<183:22:58,  4.65s/it]
 37%|███▋      | 82981/225000 [157:02:08<162:53:00,  4.13s/it]
 37%|███▋      | 82982/225000 [157:02:11<153:06:11,  3.88s/it]
 37%|███▋      | 82983/225000 [157:02:14<145:01:20,  3.68s/it]
 37%|███▋      | 82984/225000 [157:02:20<166:40:31,  4.23s/it]
 37%|███▋      | 82985/225000 [157:02:29<222:58:34,  5.65s/it]
 37%|███▋      | 82986/225000 [157:02:31<183:30:52,  4.65s/it]
 37%|███▋      | 82987/225000 [157:02:34<160:12:05,  4.06s/it]
 37%|███▋      | 82988/225000 [157:02:36<143:30:02,  3.64s/it]
 37%|███▋      | 82989/225000 [157:02:39<136:16:47,  3.45s/it]
 37%|███▋      | 82990/225000 [157:02:43<132:58:24,  3.37s/it]
 37%|███▋      | 82991/225000 [157:02:45<125:18:25,  3.18s/it]
 37%|███▋      | 82992/225000 [157:02:52<170:45:45,  4.33s/it]
 37%|███▋      | 82993/225000 [157:02:59<199:59:51,  5.07s/it]
 37%|███▋      | 82994/225000 [157:03:02<171:36:07,  4.35s/it]
 37%|███▋      | 82995/225000 [157:03:04<151:23:52,  3.84s/it]
 37%|███▋      | 82996/225000 [157:03:07<138:38:14,  3.51s/it]
 37%|███▋      | 82997/225000 [157:03:10<129:45:00,  3.29s/it]
 37%|███▋      | 82998/225000 [157:03:13<131:29:01,  3.33s/it]
 37%|███▋      | 82999/225000 [157:03:20<167:11:02,  4.24s/it]
 37%|███▋      | 83000/225000 [157:03:27<207:54:08,  5.27s/it]
                                                              
{'loss': '0.6141', 'grad_norm': '4.672', 'learning_rate': '3.709e-05', 'epoch': '0.3689', 'num_input_tokens_seen': 258075432, 'train_runtime': '5.654e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83000/225000 [157:03:27<207:54:08,  5.27s/it]
 37%|███▋      | 83001/225000 [157:03:30<174:12:04,  4.42s/it]
 37%|███▋      | 83002/225000 [157:03:33<155:04:39,  3.93s/it]
 37%|███▋      | 83003/225000 [157:03:36<145:55:33,  3.70s/it]
 37%|███▋      | 83004/225000 [157:03:39<135:28:48,  3.43s/it]
 37%|███▋      | 83005/225000 [157:03:43<152:08:08,  3.86s/it]
 37%|███▋      | 83006/225000 [157:03:52<203:58:15,  5.17s/it]
 37%|███▋      | 83007/225000 [157:03:54<171:39:15,  4.35s/it]
 37%|███▋      | 83008/225000 [157:03:57<149:15:23,  3.78s/it]
 37%|███▋      | 83009/225000 [157:04:01<162:37:32,  4.12s/it]
 37%|███▋      | 83010/225000 [157:04:06<164:33:27,  4.17s/it]
 37%|███▋      | 83011/225000 [157:04:09<150:01:03,  3.80s/it]
 37%|███▋      | 83012/225000 [157:04:12<144:38:34,  3.67s/it]
 37%|███▋      | 83013/225000 [157:04:15<141:59:12,  3.60s/it]
 37%|███▋      | 83014/225000 [157:04:19<143:52:13,  3.65s/it]
 37%|███▋      | 83015/225000 [157:04:22<138:31:08,  3.51s/it]
 37%|███▋      | 83016/225000 [157:04:27<148:26:33,  3.76s/it]
 37%|███▋      | 83017/225000 [157:04:36<210:23:46,  5.33s/it]
 37%|███▋      | 83018/225000 [157:04:40<194:49:38,  4.94s/it]
 37%|███▋      | 83019/225000 [157:04:43<172:08:29,  4.36s/it]
 37%|███▋      | 83020/225000 [157:04:46<157:16:28,  3.99s/it]
 37%|███▋      | 83021/225000 [157:04:50<152:45:49,  3.87s/it]
 37%|███▋      | 83022/225000 [157:04:53<147:15:08,  3.73s/it]
 37%|███▋      | 83023/225000 [157:04:57<146:27:41,  3.71s/it]
 37%|███▋      | 83024/225000 [157:05:00<139:35:51,  3.54s/it]
 37%|███▋      | 83025/225000 [157:05:04<146:51:46,  3.72s/it]
 37%|███▋      | 83026/225000 [157:05:07<141:15:05,  3.58s/it]
 37%|███▋      | 83027/225000 [157:05:11<148:46:23,  3.77s/it]
 37%|███▋      | 83028/225000 [157:05:15<141:22:00,  3.58s/it]
 37%|███▋      | 83029/225000 [157:05:18<144:20:20,  3.66s/it]
 37%|███▋      | 83030/225000 [157:05:22<142:37:22,  3.62s/it]
 37%|███▋      | 83031/225000 [157:05:26<144:14:12,  3.66s/it]
 37%|███▋      | 83032/225000 [157:05:33<189:06:54,  4.80s/it]
 37%|███▋      | 83033/225000 [157:05:39<205:54:56,  5.22s/it]
 37%|███▋      | 83034/225000 [157:05:42<172:23:38,  4.37s/it]
 37%|███▋      | 83035/225000 [157:05:48<192:21:51,  4.88s/it]
 37%|███▋      | 83036/225000 [157:05:51<177:11:15,  4.49s/it]
 37%|███▋      | 83037/225000 [157:05:54<157:17:13,  3.99s/it]
 37%|███▋      | 83038/225000 [157:05:57<143:16:32,  3.63s/it]
 37%|███▋      | 83039/225000 [157:06:05<199:08:16,  5.05s/it]
 37%|███▋      | 83040/225000 [157:06:08<174:26:50,  4.42s/it]
 37%|███▋      | 83041/225000 [157:06:11<153:37:01,  3.90s/it]
 37%|███▋      | 83042/225000 [157:06:14<147:58:33,  3.75s/it]
 37%|███▋      | 83043/225000 [157:06:22<191:46:43,  4.86s/it]
 37%|███▋      | 83044/225000 [157:06:24<162:00:57,  4.11s/it]
 37%|███▋      | 83045/225000 [157:06:27<144:47:02,  3.67s/it]
 37%|███▋      | 83046/225000 [157:06:30<135:34:47,  3.44s/it]
 37%|███▋      | 83047/225000 [157:06:34<147:39:50,  3.74s/it]
 37%|███▋      | 83048/225000 [157:06:40<169:00:42,  4.29s/it]
 37%|███▋      | 83049/225000 [157:06:43<160:30:52,  4.07s/it]
 37%|███▋      | 83050/225000 [157:06:47<161:41:03,  4.10s/it]
                                                              
{'loss': '0.6037', 'grad_norm': '4.367', 'learning_rate': '3.707e-05', 'epoch': '0.3691', 'num_input_tokens_seen': 258163128, 'train_runtime': '5.656e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83050/225000 [157:06:47<161:41:03,  4.10s/it]
 37%|███▋      | 83051/225000 [157:06:51<152:04:04,  3.86s/it]
 37%|███▋      | 83052/225000 [157:06:55<152:21:24,  3.86s/it]
 37%|███▋      | 83053/225000 [157:06:58<143:21:45,  3.64s/it]
 37%|███▋      | 83054/225000 [157:07:01<141:43:00,  3.59s/it]
 37%|███▋      | 83055/225000 [157:07:05<138:30:02,  3.51s/it]
 37%|███▋      | 83056/225000 [157:07:08<136:59:35,  3.47s/it]
 37%|███▋      | 83057/225000 [157:07:11<135:09:37,  3.43s/it]
 37%|███▋      | 83058/225000 [157:07:18<177:11:28,  4.49s/it]
 37%|███▋      | 83059/225000 [157:07:24<197:56:34,  5.02s/it]
 37%|███▋      | 83060/225000 [157:07:27<170:28:07,  4.32s/it]
 37%|███▋      | 83061/225000 [157:07:30<155:46:41,  3.95s/it]
 37%|███▋      | 83062/225000 [157:07:34<148:13:55,  3.76s/it]
 37%|███▋      | 83063/225000 [157:07:40<179:59:38,  4.57s/it]
 37%|███▋      | 83064/225000 [157:07:46<202:36:25,  5.14s/it]
 37%|███▋      | 83065/225000 [157:07:49<169:13:37,  4.29s/it]
 37%|███▋      | 83066/225000 [157:07:51<150:12:29,  3.81s/it]
 37%|███▋      | 83067/225000 [157:07:54<139:00:54,  3.53s/it]
 37%|███▋      | 83068/225000 [157:07:57<132:47:00,  3.37s/it]
 37%|███▋      | 83069/225000 [157:08:06<189:45:27,  4.81s/it]
 37%|███▋      | 83070/225000 [157:08:09<169:53:55,  4.31s/it]
 37%|███▋      | 83071/225000 [157:08:11<150:02:07,  3.81s/it]
 37%|███▋      | 83072/225000 [157:08:14<140:21:51,  3.56s/it]
 37%|███▋      | 83073/225000 [157:08:18<138:46:02,  3.52s/it]
 37%|███▋      | 83074/225000 [157:08:21<138:47:24,  3.52s/it]
 37%|███▋      | 83075/225000 [157:08:24<133:16:09,  3.38s/it]
 37%|███▋      | 83076/225000 [157:08:27<130:21:23,  3.31s/it]
 37%|███▋      | 83077/225000 [157:08:31<135:30:58,  3.44s/it]
 37%|███▋      | 83078/225000 [157:08:34<134:21:26,  3.41s/it]
 37%|███▋      | 83079/225000 [157:08:39<141:55:47,  3.60s/it]
 37%|███▋      | 83080/225000 [157:08:42<137:12:20,  3.48s/it]
 37%|███▋      | 83081/225000 [157:08:45<136:55:40,  3.47s/it]
 37%|███▋      | 83082/225000 [157:08:48<134:18:29,  3.41s/it]
 37%|███▋      | 83083/225000 [157:08:52<133:25:09,  3.38s/it]
 37%|███▋      | 83084/225000 [157:08:59<183:38:03,  4.66s/it]
 37%|███▋      | 83085/225000 [157:09:08<227:51:25,  5.78s/it]
 37%|███▋      | 83086/225000 [157:09:11<196:09:24,  4.98s/it]
 37%|███▋      | 83087/225000 [157:09:14<174:15:17,  4.42s/it]
 37%|███▋      | 83088/225000 [157:09:17<158:18:23,  4.02s/it]
 37%|███▋      | 83089/225000 [157:09:20<147:32:18,  3.74s/it]
 37%|███▋      | 83090/225000 [157:09:23<142:04:37,  3.60s/it]
 37%|███▋      | 83091/225000 [157:09:27<136:09:01,  3.45s/it]
 37%|███▋      | 83092/225000 [157:09:30<138:41:12,  3.52s/it]
 37%|███▋      | 83093/225000 [157:09:38<193:29:44,  4.91s/it]
 37%|███▋      | 83094/225000 [157:09:44<201:00:50,  5.10s/it]
 37%|███▋      | 83095/225000 [157:09:47<173:08:14,  4.39s/it]
 37%|███▋      | 83096/225000 [157:09:54<211:03:56,  5.35s/it]
 37%|███▋      | 83097/225000 [157:09:57<175:55:42,  4.46s/it]
 37%|███▋      | 83098/225000 [157:10:03<192:06:26,  4.87s/it]
 37%|███▋      | 83099/225000 [157:10:05<168:59:36,  4.29s/it]
 37%|███▋      | 83100/225000 [157:10:08<149:27:27,  3.79s/it]
                                                              
{'loss': '0.5987', 'grad_norm': '7.818', 'learning_rate': '3.706e-05', 'epoch': '0.3693', 'num_input_tokens_seen': 258249872, 'train_runtime': '5.658e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83100/225000 [157:10:08<149:27:27,  3.79s/it]
 37%|███▋      | 83101/225000 [157:10:11<134:53:02,  3.42s/it]
 37%|███▋      | 83102/225000 [157:10:18<184:20:14,  4.68s/it]
 37%|███▋      | 83103/225000 [157:10:22<169:03:19,  4.29s/it]
 37%|███▋      | 83104/225000 [157:10:28<188:20:08,  4.78s/it]
 37%|███▋      | 83105/225000 [157:10:32<189:28:30,  4.81s/it]
 37%|███▋      | 83106/225000 [157:10:36<170:55:19,  4.34s/it]
 37%|███▋      | 83107/225000 [157:10:39<156:42:15,  3.98s/it]
 37%|███▋      | 83108/225000 [157:10:42<147:50:33,  3.75s/it]
 37%|███▋      | 83109/225000 [157:10:45<140:36:42,  3.57s/it]
 37%|███▋      | 83110/225000 [157:10:49<141:14:25,  3.58s/it]
 37%|███▋      | 83111/225000 [157:10:51<130:37:40,  3.31s/it]
 37%|███▋      | 83112/225000 [157:10:54<122:48:05,  3.12s/it]
 37%|███▋      | 83113/225000 [157:10:58<136:30:55,  3.46s/it]
 37%|███▋      | 83114/225000 [157:11:08<211:35:49,  5.37s/it]
 37%|███▋      | 83115/225000 [157:11:11<184:28:44,  4.68s/it]
 37%|███▋      | 83116/225000 [157:11:14<162:38:13,  4.13s/it]
 37%|███▋      | 83117/225000 [157:11:17<149:50:41,  3.80s/it]
 37%|███▋      | 83118/225000 [157:11:20<142:18:08,  3.61s/it]
 37%|███▋      | 83119/225000 [157:11:23<130:53:15,  3.32s/it]
 37%|███▋      | 83120/225000 [157:11:26<123:56:27,  3.14s/it]
 37%|███▋      | 83121/225000 [157:11:33<172:29:58,  4.38s/it]
 37%|███▋      | 83122/225000 [157:11:39<197:53:22,  5.02s/it]
 37%|███▋      | 83123/225000 [157:11:43<174:24:28,  4.43s/it]
 37%|███▋      | 83124/225000 [157:11:45<155:35:16,  3.95s/it]
 37%|███▋      | 83125/225000 [157:11:48<142:54:36,  3.63s/it]
 37%|███▋      | 83126/225000 [157:11:51<133:02:45,  3.38s/it]
 37%|███▋      | 83127/225000 [157:11:54<129:36:21,  3.29s/it]
 37%|███▋      | 83128/225000 [157:11:57<130:09:57,  3.30s/it]
 37%|███▋      | 83129/225000 [157:12:06<198:04:48,  5.03s/it]
 37%|███▋      | 83130/225000 [157:12:10<184:37:15,  4.68s/it]
 37%|███▋      | 83131/225000 [157:12:13<165:28:48,  4.20s/it]
 37%|███▋      | 83132/225000 [157:12:17<153:24:17,  3.89s/it]
 37%|███▋      | 83133/225000 [157:12:20<148:10:27,  3.76s/it]
 37%|███▋      | 83134/225000 [157:12:24<144:24:50,  3.66s/it]
 37%|███▋      | 83135/225000 [157:12:27<142:14:39,  3.61s/it]
 37%|███▋      | 83136/225000 [157:12:30<135:24:32,  3.44s/it]
 37%|███▋      | 83137/225000 [157:12:33<132:22:33,  3.36s/it]
 37%|███▋      | 83138/225000 [157:12:37<138:04:45,  3.50s/it]
 37%|███▋      | 83139/225000 [157:12:45<196:30:43,  4.99s/it]
 37%|███▋      | 83140/225000 [157:12:51<198:42:39,  5.04s/it]
 37%|███▋      | 83141/225000 [157:12:53<169:06:34,  4.29s/it]
 37%|███▋      | 83142/225000 [157:12:56<147:31:31,  3.74s/it]
 37%|███▋      | 83143/225000 [157:12:59<136:55:12,  3.47s/it]
 37%|███▋      | 83144/225000 [157:13:04<157:10:01,  3.99s/it]
 37%|███▋      | 83145/225000 [157:13:10<183:04:33,  4.65s/it]
 37%|███▋      | 83146/225000 [157:13:13<160:08:11,  4.06s/it]
 37%|███▋      | 83147/225000 [157:13:16<147:44:22,  3.75s/it]
 37%|███▋      | 83148/225000 [157:13:19<141:30:28,  3.59s/it]
 37%|███▋      | 83149/225000 [157:13:23<142:56:05,  3.63s/it]
 37%|███▋      | 83150/225000 [157:13:26<143:35:11,  3.64s/it]
                                                              
{'loss': '0.578', 'grad_norm': '4.458', 'learning_rate': '3.704e-05', 'epoch': '0.3696', 'num_input_tokens_seen': 258336336, 'train_runtime': '5.66e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83150/225000 [157:13:26<143:35:11,  3.64s/it]
 37%|███▋      | 83151/225000 [157:13:30<139:28:36,  3.54s/it]
 37%|███▋      | 83152/225000 [157:13:33<136:04:33,  3.45s/it]
 37%|███▋      | 83153/225000 [157:13:36<134:33:47,  3.42s/it]
 37%|███▋      | 83154/225000 [157:13:46<207:02:55,  5.25s/it]
 37%|███▋      | 83155/225000 [157:13:49<188:26:38,  4.78s/it]
 37%|███▋      | 83156/225000 [157:13:53<178:21:57,  4.53s/it]
 37%|███▋      | 83157/225000 [157:13:57<166:52:14,  4.24s/it]
 37%|███▋      | 83158/225000 [157:14:00<158:21:30,  4.02s/it]
 37%|███▋      | 83159/225000 [157:14:04<153:22:20,  3.89s/it]
 37%|███▋      | 83160/225000 [157:14:09<165:59:24,  4.21s/it]
 37%|███▋      | 83161/225000 [157:14:13<169:16:06,  4.30s/it]
 37%|███▋      | 83162/225000 [157:14:17<158:00:36,  4.01s/it]
 37%|███▋      | 83163/225000 [157:14:22<170:38:24,  4.33s/it]
 37%|███▋      | 83164/225000 [157:14:28<197:03:30,  5.00s/it]
 37%|███▋      | 83165/225000 [157:14:31<165:55:42,  4.21s/it]
 37%|███▋      | 83166/225000 [157:14:36<177:42:18,  4.51s/it]
 37%|███▋      | 83167/225000 [157:14:39<157:19:00,  3.99s/it]
 37%|███▋      | 83168/225000 [157:14:42<142:59:47,  3.63s/it]
 37%|███▋      | 83169/225000 [157:14:44<133:44:33,  3.39s/it]
 37%|███▋      | 83170/225000 [157:14:48<131:40:34,  3.34s/it]
 37%|███▋      | 83171/225000 [157:14:56<191:11:30,  4.85s/it]
 37%|███▋      | 83172/225000 [157:15:01<192:17:45,  4.88s/it]
 37%|███▋      | 83173/225000 [157:15:04<171:56:03,  4.36s/it]
 37%|███▋      | 83174/225000 [157:15:07<153:04:23,  3.89s/it]
 37%|███▋      | 83175/225000 [157:15:10<141:08:46,  3.58s/it]
 37%|███▋      | 83176/225000 [157:15:15<165:27:54,  4.20s/it]
 37%|███▋      | 83177/225000 [157:15:18<152:13:55,  3.86s/it]
 37%|███▋      | 83178/225000 [157:15:21<138:00:27,  3.50s/it]
 37%|███▋      | 83179/225000 [157:15:24<130:24:32,  3.31s/it]
 37%|███▋      | 83180/225000 [157:15:28<134:37:58,  3.42s/it]
 37%|███▋      | 83181/225000 [157:15:33<159:11:44,  4.04s/it]
 37%|███▋      | 83182/225000 [157:15:38<174:23:43,  4.43s/it]
 37%|███▋      | 83183/225000 [157:15:42<162:04:45,  4.11s/it]
 37%|███▋      | 83184/225000 [157:15:49<194:37:51,  4.94s/it]
 37%|███▋      | 83185/225000 [157:15:51<163:38:39,  4.15s/it]
 37%|███▋      | 83186/225000 [157:15:54<150:36:34,  3.82s/it]
 37%|███▋      | 83187/225000 [157:15:58<146:14:29,  3.71s/it]
 37%|███▋      | 83188/225000 [157:16:01<146:31:29,  3.72s/it]
 37%|███▋      | 83189/225000 [157:16:05<145:16:12,  3.69s/it]
 37%|███▋      | 83190/225000 [157:16:09<146:06:10,  3.71s/it]
 37%|███▋      | 83191/225000 [157:16:12<143:24:11,  3.64s/it]
 37%|███▋      | 83192/225000 [157:16:20<191:46:08,  4.87s/it]
 37%|███▋      | 83193/225000 [157:16:24<186:19:59,  4.73s/it]
 37%|███▋      | 83194/225000 [157:16:27<158:51:28,  4.03s/it]
 37%|███▋      | 83195/225000 [157:16:29<139:37:45,  3.54s/it]
 37%|███▋      | 83196/225000 [157:16:32<133:42:01,  3.39s/it]
 37%|███▋      | 83197/225000 [157:16:36<136:58:57,  3.48s/it]
 37%|███▋      | 83198/225000 [157:16:43<177:44:07,  4.51s/it]
 37%|███▋      | 83199/225000 [157:16:48<191:02:06,  4.85s/it]
 37%|███▋      | 83200/225000 [157:16:51<165:59:41,  4.21s/it]
                                                              
{'loss': '0.5294', 'grad_norm': '4.84', 'learning_rate': '3.702e-05', 'epoch': '0.3698', 'num_input_tokens_seen': 258423200, 'train_runtime': '5.662e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83200/225000 [157:16:51<165:59:41,  4.21s/it]
 37%|███▋      | 83201/225000 [157:16:54<147:33:39,  3.75s/it]
 37%|███▋      | 83202/225000 [157:16:57<137:59:50,  3.50s/it]
 37%|███▋      | 83203/225000 [157:16:59<129:26:10,  3.29s/it]
 37%|███▋      | 83204/225000 [157:17:03<127:02:03,  3.23s/it]
 37%|███▋      | 83205/225000 [157:17:05<121:41:33,  3.09s/it]
 37%|███▋      | 83206/225000 [157:17:10<145:03:28,  3.68s/it]
 37%|███▋      | 83207/225000 [157:17:19<200:57:44,  5.10s/it]
 37%|███▋      | 83208/225000 [157:17:22<175:02:23,  4.44s/it]
 37%|███▋      | 83209/225000 [157:17:24<152:48:01,  3.88s/it]
 37%|███▋      | 83210/225000 [157:17:32<195:12:28,  4.96s/it]
 37%|███▋      | 83211/225000 [157:17:34<168:12:35,  4.27s/it]
 37%|███▋      | 83212/225000 [157:17:39<166:43:41,  4.23s/it]
 37%|███▋      | 83213/225000 [157:17:44<186:38:03,  4.74s/it]
 37%|███▋      | 83214/225000 [157:17:47<161:16:22,  4.09s/it]
 37%|███▋      | 83215/225000 [157:17:50<142:31:11,  3.62s/it]
 37%|███▋      | 83216/225000 [157:17:57<184:47:19,  4.69s/it]
 37%|███▋      | 83217/225000 [157:18:00<172:20:16,  4.38s/it]
 37%|███▋      | 83218/225000 [157:18:03<151:02:24,  3.84s/it]
 37%|███▋      | 83219/225000 [157:18:05<134:50:04,  3.42s/it]
 37%|███▋      | 83220/225000 [157:18:08<124:46:51,  3.17s/it]
 37%|███▋      | 83221/225000 [157:18:12<131:44:13,  3.35s/it]
 37%|███▋      | 83222/225000 [157:18:15<134:36:41,  3.42s/it]
 37%|███▋      | 83223/225000 [157:18:18<130:45:51,  3.32s/it]
 37%|███▋      | 83224/225000 [157:18:22<128:34:37,  3.26s/it]
 37%|███▋      | 83225/225000 [157:18:25<124:47:53,  3.17s/it]
 37%|███▋      | 83226/225000 [157:18:28<127:45:10,  3.24s/it]
 37%|███▋      | 83227/225000 [157:18:33<153:58:23,  3.91s/it]
 37%|███▋      | 83228/225000 [157:18:42<215:10:02,  5.46s/it]
 37%|███▋      | 83229/225000 [157:18:47<202:50:56,  5.15s/it]
 37%|███▋      | 83230/225000 [157:18:50<173:35:59,  4.41s/it]
 37%|███▋      | 83231/225000 [157:18:58<221:34:18,  5.63s/it]
 37%|███▋      | 83232/225000 [157:19:01<190:11:29,  4.83s/it]
 37%|███▋      | 83233/225000 [157:19:04<166:12:20,  4.22s/it]
 37%|███▋      | 83234/225000 [157:19:06<144:57:45,  3.68s/it]
 37%|███▋      | 83235/225000 [157:19:12<174:19:45,  4.43s/it]
 37%|███▋      | 83236/225000 [157:19:16<159:09:25,  4.04s/it]
 37%|███▋      | 83237/225000 [157:19:20<168:05:58,  4.27s/it]
 37%|███▋      | 83238/225000 [157:19:27<190:53:47,  4.85s/it]
 37%|███▋      | 83239/225000 [157:19:30<174:07:19,  4.42s/it]
 37%|███▋      | 83240/225000 [157:19:35<183:42:44,  4.67s/it]
 37%|███▋      | 83241/225000 [157:19:41<193:01:04,  4.90s/it]
 37%|███▋      | 83242/225000 [157:19:43<162:56:52,  4.14s/it]
 37%|███▋      | 83243/225000 [157:19:45<140:39:40,  3.57s/it]
 37%|███▋      | 83244/225000 [157:19:49<142:12:23,  3.61s/it]
 37%|███▋      | 83245/225000 [157:19:52<135:50:15,  3.45s/it]
 37%|███▋      | 83246/225000 [157:19:56<137:00:14,  3.48s/it]
 37%|███▋      | 83247/225000 [157:19:59<134:43:34,  3.42s/it]
 37%|███▋      | 83248/225000 [157:20:07<190:19:13,  4.83s/it]
 37%|███▋      | 83249/225000 [157:20:13<202:08:15,  5.13s/it]
 37%|███▋      | 83250/225000 [157:20:17<191:04:58,  4.85s/it]
                                                              
{'loss': '0.5728', 'grad_norm': '6.931', 'learning_rate': '3.701e-05', 'epoch': '0.37', 'num_input_tokens_seen': 258510072, 'train_runtime': '5.664e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83250/225000 [157:20:17<191:04:58,  4.85s/it]
 37%|███▋      | 83251/225000 [157:20:20<163:28:33,  4.15s/it]
 37%|███▋      | 83252/225000 [157:20:22<141:46:05,  3.60s/it]
 37%|███▋      | 83253/225000 [157:20:27<164:45:04,  4.18s/it]
 37%|███▋      | 83254/225000 [157:20:30<146:13:50,  3.71s/it]
 37%|███▋      | 83255/225000 [157:20:33<138:46:47,  3.52s/it]
 37%|███▋      | 83256/225000 [157:20:36<133:09:15,  3.38s/it]
 37%|███▋      | 83257/225000 [157:20:39<130:21:43,  3.31s/it]
 37%|███▋      | 83258/225000 [157:20:42<128:17:54,  3.26s/it]
 37%|███▋      | 83259/225000 [157:20:45<123:50:56,  3.15s/it]
 37%|███▋      | 83260/225000 [157:20:48<124:11:51,  3.15s/it]
 37%|███▋      | 83261/225000 [157:20:54<154:18:20,  3.92s/it]
 37%|███▋      | 83262/225000 [157:21:04<229:15:10,  5.82s/it]
 37%|███▋      | 83263/225000 [157:21:07<195:16:39,  4.96s/it]
 37%|███▋      | 83264/225000 [157:21:10<171:46:36,  4.36s/it]
 37%|███▋      | 83265/225000 [157:21:14<168:07:02,  4.27s/it]
 37%|███▋      | 83266/225000 [157:21:18<160:33:58,  4.08s/it]
 37%|███▋      | 83267/225000 [157:21:21<152:34:53,  3.88s/it]
 37%|███▋      | 83268/225000 [157:21:25<151:20:17,  3.84s/it]
 37%|███▋      | 83269/225000 [157:21:29<146:23:45,  3.72s/it]
 37%|███▋      | 83270/225000 [157:21:32<143:39:32,  3.65s/it]
 37%|███▋      | 83271/225000 [157:21:35<136:22:32,  3.46s/it]
 37%|███▋      | 83272/225000 [157:21:39<144:34:04,  3.67s/it]
 37%|███▋      | 83273/225000 [157:21:46<183:42:21,  4.67s/it]
 37%|███▋      | 83274/225000 [157:21:55<227:04:01,  5.77s/it]
 37%|███▋      | 83275/225000 [157:21:57<187:01:16,  4.75s/it]
 37%|███▋      | 83276/225000 [157:22:00<164:09:03,  4.17s/it]
 37%|███▋      | 83277/225000 [157:22:03<149:34:07,  3.80s/it]
 37%|███▋      | 83278/225000 [157:22:06<139:25:21,  3.54s/it]
 37%|███▋      | 83279/225000 [157:22:09<134:44:18,  3.42s/it]
 37%|███▋      | 83280/225000 [157:22:17<186:52:27,  4.75s/it]
 37%|███▋      | 83281/225000 [157:22:23<201:04:31,  5.11s/it]
 37%|███▋      | 83282/225000 [157:22:25<174:14:10,  4.43s/it]
 37%|███▋      | 83283/225000 [157:22:31<187:29:46,  4.76s/it]
 37%|███▋      | 83284/225000 [157:22:34<169:21:16,  4.30s/it]
 37%|███▋      | 83285/225000 [157:22:37<156:25:18,  3.97s/it]
 37%|███▋      | 83286/225000 [157:22:41<148:37:34,  3.78s/it]
 37%|███▋      | 83287/225000 [157:22:44<138:27:50,  3.52s/it]
 37%|███▋      | 83288/225000 [157:22:47<138:42:23,  3.52s/it]
 37%|███▋      | 83289/225000 [157:22:51<139:33:57,  3.55s/it]
 37%|███▋      | 83290/225000 [157:23:00<200:32:45,  5.09s/it]
 37%|███▋      | 83291/225000 [157:23:05<200:13:14,  5.09s/it]
 37%|███▋      | 83292/225000 [157:23:08<178:26:06,  4.53s/it]
 37%|███▋      | 83293/225000 [157:23:11<164:13:12,  4.17s/it]
 37%|███▋      | 83294/225000 [157:23:15<159:11:41,  4.04s/it]
 37%|███▋      | 83295/225000 [157:23:18<153:05:54,  3.89s/it]
 37%|███▋      | 83296/225000 [157:23:22<149:49:51,  3.81s/it]
 37%|███▋      | 83297/225000 [157:23:26<147:00:51,  3.73s/it]
 37%|███▋      | 83298/225000 [157:23:29<148:16:07,  3.77s/it]
 37%|███▋      | 83299/225000 [157:23:33<139:24:14,  3.54s/it]
 37%|███▋      | 83300/225000 [157:23:36<134:06:39,  3.41s/it]
                                                              
{'loss': '0.5885', 'grad_norm': '3.598', 'learning_rate': '3.699e-05', 'epoch': '0.3702', 'num_input_tokens_seen': 258597752, 'train_runtime': '5.666e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83300/225000 [157:23:36<134:06:39,  3.41s/it]
 37%|███▋      | 83301/225000 [157:23:39<137:35:43,  3.50s/it]
 37%|███▋      | 83302/225000 [157:23:46<173:21:55,  4.40s/it]
 37%|███▋      | 83303/225000 [157:23:54<221:42:49,  5.63s/it]
 37%|███▋      | 83304/225000 [157:23:58<201:01:10,  5.11s/it]
 37%|███▋      | 83305/225000 [157:24:01<169:23:05,  4.30s/it]
 37%|███▋      | 83306/225000 [157:24:04<158:41:58,  4.03s/it]
 37%|███▋      | 83307/225000 [157:24:07<150:19:01,  3.82s/it]
 37%|███▋      | 83308/225000 [157:24:11<142:35:29,  3.62s/it]
 37%|███▋      | 83309/225000 [157:24:14<136:17:28,  3.46s/it]
 37%|███▋      | 83310/225000 [157:24:17<134:09:14,  3.41s/it]
 37%|███▋      | 83311/225000 [157:24:21<137:00:18,  3.48s/it]
 37%|███▋      | 83312/225000 [157:24:24<138:50:42,  3.53s/it]
 37%|███▋      | 83313/225000 [157:24:28<141:34:11,  3.60s/it]
 37%|███▋      | 83314/225000 [157:24:32<141:52:35,  3.60s/it]
 37%|███▋      | 83315/225000 [157:24:35<144:17:24,  3.67s/it]
 37%|███▋      | 83316/225000 [157:24:39<140:21:43,  3.57s/it]
 37%|███▋      | 83317/225000 [157:24:42<136:54:40,  3.48s/it]
 37%|███▋      | 83318/225000 [157:24:46<143:28:11,  3.65s/it]
 37%|███▋      | 83319/225000 [157:24:51<158:08:37,  4.02s/it]
 37%|███▋      | 83320/225000 [157:25:00<217:28:07,  5.53s/it]
 37%|███▋      | 83321/225000 [157:25:07<238:45:09,  6.07s/it]
 37%|███▋      | 83322/225000 [157:25:10<196:07:58,  4.98s/it]
 37%|███▋      | 83323/225000 [157:25:18<232:40:34,  5.91s/it]
 37%|███▋      | 83324/225000 [157:25:21<200:07:13,  5.09s/it]
 37%|███▋      | 83325/225000 [157:25:24<170:44:32,  4.34s/it]
 37%|███▋      | 83326/225000 [157:25:26<154:06:54,  3.92s/it]
 37%|███▋      | 83327/225000 [157:25:29<141:23:55,  3.59s/it]
 37%|███▋      | 83328/225000 [157:25:32<132:16:56,  3.36s/it]
 37%|███▋      | 83329/225000 [157:25:37<145:53:10,  3.71s/it]
 37%|███▋      | 83330/225000 [157:25:46<210:59:22,  5.36s/it]
 37%|███▋      | 83331/225000 [157:25:49<190:16:12,  4.84s/it]
 37%|███▋      | 83332/225000 [157:25:52<161:53:18,  4.11s/it]
 37%|███▋      | 83333/225000 [157:25:55<145:29:50,  3.70s/it]
 37%|███▋      | 83334/225000 [157:25:58<147:03:57,  3.74s/it]
 37%|███▋      | 83335/225000 [157:26:07<198:22:52,  5.04s/it]
 37%|███▋      | 83336/225000 [157:26:09<168:14:54,  4.28s/it]
 37%|███▋      | 83337/225000 [157:26:11<146:24:03,  3.72s/it]
 37%|███▋      | 83338/225000 [157:26:17<165:05:53,  4.20s/it]
 37%|███▋      | 83339/225000 [157:26:22<182:05:45,  4.63s/it]
 37%|███▋      | 83340/225000 [157:26:25<158:14:56,  4.02s/it]
 37%|███▋      | 83341/225000 [157:26:28<142:37:00,  3.62s/it]
 37%|███▋      | 83342/225000 [157:26:31<132:54:31,  3.38s/it]
 37%|███▋      | 83343/225000 [157:26:40<201:25:49,  5.12s/it]
 37%|███▋      | 83344/225000 [157:26:45<201:52:29,  5.13s/it]
 37%|███▋      | 83345/225000 [157:26:47<169:07:58,  4.30s/it]
 37%|███▋      | 83346/225000 [157:26:50<146:28:50,  3.72s/it]
 37%|███▋      | 83347/225000 [157:26:52<132:15:28,  3.36s/it]
 37%|███▋      | 83348/225000 [157:26:59<173:23:39,  4.41s/it]
 37%|███▋      | 83349/225000 [157:27:04<176:47:40,  4.49s/it]
 37%|███▋      | 83350/225000 [157:27:06<153:32:12,  3.90s/it]
                                                              
{'loss': '0.5438', 'grad_norm': '2.603', 'learning_rate': '3.698e-05', 'epoch': '0.3704', 'num_input_tokens_seen': 258685088, 'train_runtime': '5.668e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83350/225000 [157:27:06<153:32:12,  3.90s/it]
 37%|███▋      | 83351/225000 [157:27:09<138:29:39,  3.52s/it]
 37%|███▋      | 83352/225000 [157:27:16<185:44:00,  4.72s/it]
 37%|███▋      | 83353/225000 [157:27:21<182:41:57,  4.64s/it]
 37%|███▋      | 83354/225000 [157:27:24<165:48:46,  4.21s/it]
 37%|███▋      | 83355/225000 [157:27:30<184:19:31,  4.68s/it]
 37%|███▋      | 83356/225000 [157:27:33<163:35:23,  4.16s/it]
 37%|███▋      | 83357/225000 [157:27:36<150:15:10,  3.82s/it]
 37%|███▋      | 83358/225000 [157:27:39<138:20:57,  3.52s/it]
 37%|███▋      | 83359/225000 [157:27:42<137:09:08,  3.49s/it]
 37%|███▋      | 83360/225000 [157:27:45<130:09:50,  3.31s/it]
 37%|███▋      | 83361/225000 [157:27:48<133:51:00,  3.40s/it]
 37%|███▋      | 83362/225000 [157:27:52<130:17:42,  3.31s/it]
 37%|███▋      | 83363/225000 [157:27:56<138:27:28,  3.52s/it]
 37%|███▋      | 83364/225000 [157:28:03<189:20:45,  4.81s/it]
 37%|███▋      | 83365/225000 [157:28:09<197:36:05,  5.02s/it]
 37%|███▋      | 83366/225000 [157:28:12<180:13:11,  4.58s/it]
 37%|███▋      | 83367/225000 [157:28:20<209:47:32,  5.33s/it]
 37%|███▋      | 83368/225000 [157:28:22<174:58:00,  4.45s/it]
 37%|███▋      | 83369/225000 [157:28:25<154:50:28,  3.94s/it]
 37%|███▋      | 83370/225000 [157:28:28<146:07:37,  3.71s/it]
 37%|███▋      | 83371/225000 [157:28:31<137:36:59,  3.50s/it]
 37%|███▋      | 83372/225000 [157:28:34<133:15:49,  3.39s/it]
 37%|███▋      | 83373/225000 [157:28:37<131:30:25,  3.34s/it]
 37%|███▋      | 83374/225000 [157:28:46<192:31:37,  4.89s/it]
 37%|███▋      | 83375/225000 [157:28:51<191:18:26,  4.86s/it]
 37%|███▋      | 83376/225000 [157:28:53<167:35:33,  4.26s/it]
 37%|███▋      | 83377/225000 [157:28:56<151:37:23,  3.85s/it]
 37%|███▋      | 83378/225000 [157:28:59<141:45:09,  3.60s/it]
 37%|███▋      | 83379/225000 [157:29:06<182:30:12,  4.64s/it]
 37%|███▋      | 83380/225000 [157:29:12<188:34:11,  4.79s/it]
 37%|███▋      | 83381/225000 [157:29:14<164:00:03,  4.17s/it]
 37%|███▋      | 83382/225000 [157:29:17<146:05:38,  3.71s/it]
 37%|███▋      | 83383/225000 [157:29:24<187:44:31,  4.77s/it]
 37%|███▋      | 83384/225000 [157:29:29<184:22:32,  4.69s/it]
 37%|███▋      | 83385/225000 [157:29:31<158:23:24,  4.03s/it]
 37%|███▋      | 83386/225000 [157:29:38<193:55:00,  4.93s/it]
 37%|███▋      | 83387/225000 [157:29:44<200:22:36,  5.09s/it]
 37%|███▋      | 83388/225000 [157:29:46<169:35:37,  4.31s/it]
 37%|███▋      | 83389/225000 [157:29:49<158:22:29,  4.03s/it]
 37%|███▋      | 83390/225000 [157:29:53<146:55:19,  3.74s/it]
 37%|███▋      | 83391/225000 [157:29:56<146:28:48,  3.72s/it]
 37%|███▋      | 83392/225000 [157:30:00<145:13:01,  3.69s/it]
 37%|███▋      | 83393/225000 [157:30:03<140:35:12,  3.57s/it]
 37%|███▋      | 83394/225000 [157:30:07<147:38:37,  3.75s/it]
 37%|███▋      | 83395/225000 [157:30:10<137:48:10,  3.50s/it]
 37%|███▋      | 83396/225000 [157:30:13<134:48:56,  3.43s/it]
 37%|███▋      | 83397/225000 [157:30:17<131:07:48,  3.33s/it]
 37%|███▋      | 83398/225000 [157:30:20<135:40:11,  3.45s/it]
 37%|███▋      | 83399/225000 [157:30:29<191:27:16,  4.87s/it]
 37%|███▋      | 83400/225000 [157:30:36<217:14:12,  5.52s/it]
                                                              
{'loss': '0.5877', 'grad_norm': '5.999', 'learning_rate': '3.696e-05', 'epoch': '0.3707', 'num_input_tokens_seen': 258772808, 'train_runtime': '5.67e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83400/225000 [157:30:36<217:14:12,  5.52s/it]
 37%|███▋      | 83401/225000 [157:30:38<178:54:44,  4.55s/it]
 37%|███▋      | 83402/225000 [157:30:40<152:42:43,  3.88s/it]
 37%|███▋      | 83403/225000 [157:30:43<135:50:10,  3.45s/it]
 37%|███▋      | 83404/225000 [157:30:45<124:31:16,  3.17s/it]
 37%|███▋      | 83405/225000 [157:30:51<151:11:03,  3.84s/it]
 37%|███▋      | 83406/225000 [157:30:57<185:14:03,  4.71s/it]
 37%|███▋      | 83407/225000 [157:31:00<162:38:45,  4.14s/it]
 37%|███▋      | 83408/225000 [157:31:05<169:50:07,  4.32s/it]
 37%|███▋      | 83409/225000 [157:31:07<147:31:47,  3.75s/it]
 37%|███▋      | 83410/225000 [157:31:10<134:57:10,  3.43s/it]
 37%|███▋      | 83411/225000 [157:31:18<191:37:56,  4.87s/it]
 37%|███▋      | 83412/225000 [157:31:22<181:14:06,  4.61s/it]
 37%|███▋      | 83413/225000 [157:31:28<191:39:00,  4.87s/it]
 37%|███▋      | 83414/225000 [157:31:31<174:11:17,  4.43s/it]
 37%|███▋      | 83415/225000 [157:31:34<156:48:32,  3.99s/it]
 37%|███▋      | 83416/225000 [157:31:37<143:35:42,  3.65s/it]
 37%|███▋      | 83417/225000 [157:31:40<142:54:59,  3.63s/it]
 37%|███▋      | 83418/225000 [157:31:43<134:42:37,  3.43s/it]
 37%|███▋      | 83419/225000 [157:31:47<134:09:56,  3.41s/it]
 37%|███▋      | 83420/225000 [157:31:50<135:35:25,  3.45s/it]
 37%|███▋      | 83421/225000 [157:31:58<187:55:27,  4.78s/it]
 37%|███▋      | 83422/225000 [157:32:04<204:16:41,  5.19s/it]
 37%|███▋      | 83423/225000 [157:32:07<179:01:33,  4.55s/it]
 37%|███▋      | 83424/225000 [157:32:11<162:03:41,  4.12s/it]
 37%|███▋      | 83425/225000 [157:32:18<196:25:59,  4.99s/it]
 37%|███▋      | 83426/225000 [157:32:23<200:23:54,  5.10s/it]
 37%|███▋      | 83427/225000 [157:32:26<174:02:20,  4.43s/it]
 37%|███▋      | 83428/225000 [157:32:29<154:44:39,  3.93s/it]
 37%|███▋      | 83429/225000 [157:32:32<144:09:15,  3.67s/it]
 37%|███▋      | 83430/225000 [157:32:39<187:36:16,  4.77s/it]
 37%|███▋      | 83431/225000 [157:32:45<199:47:19,  5.08s/it]
 37%|███▋      | 83432/225000 [157:32:48<174:46:15,  4.44s/it]
 37%|███▋      | 83433/225000 [157:32:51<159:57:18,  4.07s/it]
 37%|███▋      | 83434/225000 [157:32:54<147:12:57,  3.74s/it]
 37%|███▋      | 83435/225000 [157:32:57<139:04:15,  3.54s/it]
 37%|███▋      | 83436/225000 [157:33:00<132:04:39,  3.36s/it]
 37%|███▋      | 83437/225000 [157:33:03<129:17:53,  3.29s/it]
 37%|███▋      | 83438/225000 [157:33:11<190:33:27,  4.85s/it]
 37%|███▋      | 83439/225000 [157:33:16<184:30:01,  4.69s/it]
 37%|███▋      | 83440/225000 [157:33:18<160:02:43,  4.07s/it]
 37%|███▋      | 83441/225000 [157:33:21<141:45:16,  3.60s/it]
 37%|███▋      | 83442/225000 [157:33:24<132:02:45,  3.36s/it]
 37%|███▋      | 83443/225000 [157:33:26<124:02:02,  3.15s/it]
 37%|███▋      | 83444/225000 [157:33:31<139:28:24,  3.55s/it]
 37%|███▋      | 83445/225000 [157:33:35<143:58:25,  3.66s/it]
 37%|███▋      | 83446/225000 [157:33:40<159:47:19,  4.06s/it]
 37%|███▋      | 83447/225000 [157:33:43<150:06:52,  3.82s/it]
 37%|███▋      | 83448/225000 [157:33:46<143:32:34,  3.65s/it]
 37%|███▋      | 83449/225000 [157:33:51<158:18:28,  4.03s/it]
 37%|███▋      | 83450/225000 [157:33:57<178:20:43,  4.54s/it]
                                                              
{'loss': '0.5714', 'grad_norm': '4.235', 'learning_rate': '3.694e-05', 'epoch': '0.3709', 'num_input_tokens_seen': 258860000, 'train_runtime': '5.672e+05', 'train_tokens_per_second': '456.4'}

 37%|███▋      | 83450/225000 [157:33:57<178:20:43,  4.54s/it]
 37%|███▋      | 83451/225000 [157:34:02<186:01:04,  4.73s/it]
 37%|███▋      | 83452/225000 [157:34:08<194:45:59,  4.95s/it]
 37%|███▋      | 83453/225000 [157:34:10<165:35:31,  4.21s/it]
 37%|███▋      | 83454/225000 [157:34:13<150:11:06,  3.82s/it]
 37%|███▋      | 83455/225000 [157:34:16<137:09:03,  3.49s/it]
 37%|███▋      | 83456/225000 [157:34:19<138:40:01,  3.53s/it]
 37%|███▋      | 83457/225000 [157:34:28<204:23:34,  5.20s/it]
 37%|███▋      | 83458/225000 [157:34:31<172:04:09,  4.38s/it]
 37%|███▋      | 83459/225000 [157:34:33<150:34:05,  3.83s/it]
 37%|███▋      | 83460/225000 [157:34:37<142:47:19,  3.63s/it]
 37%|███▋      | 83461/225000 [157:34:40<138:19:33,  3.52s/it]
 37%|███▋      | 83462/225000 [157:34:43<138:36:05,  3.53s/it]
 37%|███▋      | 83463/225000 [157:34:47<141:22:03,  3.60s/it]
 37%|███▋      | 83464/225000 [157:34:54<174:57:41,  4.45s/it]
 37%|███▋      | 83465/225000 [157:35:00<197:12:00,  5.02s/it]
 37%|███▋      | 83466/225000 [157:35:03<179:15:45,  4.56s/it]
 37%|███▋      | 83467/225000 [157:35:07<163:58:47,  4.17s/it]
 37%|███▋      | 83468/225000 [157:35:10<150:42:04,  3.83s/it]
 37%|███▋      | 83469/225000 [157:35:13<143:01:45,  3.64s/it]
 37%|███▋      | 83470/225000 [157:35:17<143:19:16,  3.65s/it]
 37%|███▋      | 83471/225000 [157:35:20<140:15:40,  3.57s/it]
 37%|███▋      | 83472/225000 [157:35:23<138:45:19,  3.53s/it]
 37%|███▋      | 83473/225000 [157:35:27<134:42:22,  3.43s/it]
 37%|███▋      | 83474/225000 [157:35:30<135:20:02,  3.44s/it]
 37%|███▋      | 83475/225000 [157:35:34<138:47:20,  3.53s/it]
 37%|███▋      | 83476/225000 [157:35:37<132:33:08,  3.37s/it]
 37%|███▋      | 83477/225000 [157:35:40<131:24:17,  3.34s/it]
 37%|███▋      | 83478/225000 [157:35:43<132:27:10,  3.37s/it]
 37%|███▋      | 83479/225000 [157:35:50<166:31:35,  4.24s/it]
 37%|███▋      | 83480/225000 [157:35:56<192:34:31,  4.90s/it]
 37%|███▋      | 83481/225000 [157:35:59<171:00:42,  4.35s/it]
 37%|███▋      | 83482/225000 [157:36:03<160:37:11,  4.09s/it]
 37%|███▋      | 83483/225000 [157:36:06<154:47:52,  3.94s/it]
 37%|███▋      | 83484/225000 [157:36:10<148:23:06,  3.77s/it]
 37%|███▋      | 83485/225000 [157:36:13<141:08:01,  3.59s/it]
 37%|███▋      | 83486/225000 [157:36:17<141:56:27,  3.61s/it]
 37%|███▋      | 83487/225000 [157:36:20<145:27:14,  3.70s/it]
 37%|███▋      | 83488/225000 [157:36:28<194:56:05,  4.96s/it]
 37%|███▋      | 83489/225000 [157:36:37<233:38:02,  5.94s/it]
 37%|███▋      | 83490/225000 [157:36:39<190:58:59,  4.86s/it]
 37%|███▋      | 83491/225000 [157:36:42<167:53:30,  4.27s/it]
 37%|███▋      | 83492/225000 [157:36:48<194:06:58,  4.94s/it]
 37%|███▋      | 83493/225000 [157:36:51<167:39:55,  4.27s/it]
 37%|███▋      | 83494/225000 [157:36:54<153:06:13,  3.90s/it]
 37%|███▋      | 83495/225000 [157:36:57<141:58:14,  3.61s/it]
 37%|███▋      | 83496/225000 [157:37:01<141:59:55,  3.61s/it]
 37%|███▋      | 83497/225000 [157:37:04<139:27:23,  3.55s/it]
 37%|███▋      | 83498/225000 [157:37:11<176:44:21,  4.50s/it]
 37%|███▋      | 83499/225000 [157:37:17<202:37:44,  5.16s/it]
 37%|███▋      | 83500/225000 [157:37:20<172:49:22,  4.40s/it]
                                                              
{'loss': '0.5502', 'grad_norm': '4.461', 'learning_rate': '3.693e-05', 'epoch': '0.3711', 'num_input_tokens_seen': 258946808, 'train_runtime': '5.674e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83500/225000 [157:37:20<172:49:22,  4.40s/it]
 37%|███▋      | 83501/225000 [157:37:26<193:41:09,  4.93s/it]
 37%|███▋      | 83502/225000 [157:37:29<167:05:14,  4.25s/it]
 37%|███▋      | 83503/225000 [157:37:35<191:04:46,  4.86s/it]
 37%|███▋      | 83504/225000 [157:37:38<170:12:17,  4.33s/it]
 37%|███▋      | 83505/225000 [157:37:41<151:26:27,  3.85s/it]
 37%|███▋      | 83506/225000 [157:37:44<137:38:19,  3.50s/it]
 37%|███▋      | 83507/225000 [157:37:46<129:11:06,  3.29s/it]
 37%|███▋      | 83508/225000 [157:37:50<130:37:01,  3.32s/it]
 37%|███▋      | 83509/225000 [157:37:55<152:41:35,  3.89s/it]
 37%|███▋      | 83510/225000 [157:38:04<206:39:13,  5.26s/it]
 37%|███▋      | 83511/225000 [157:38:06<174:11:14,  4.43s/it]
 37%|███▋      | 83512/225000 [157:38:09<155:08:44,  3.95s/it]
 37%|███▋      | 83513/225000 [157:38:11<139:22:36,  3.55s/it]
 37%|███▋      | 83514/225000 [157:38:19<184:24:57,  4.69s/it]
 37%|███▋      | 83515/225000 [157:38:23<175:28:04,  4.46s/it]
 37%|███▋      | 83516/225000 [157:38:25<149:56:23,  3.82s/it]
 37%|███▋      | 83517/225000 [157:38:29<148:36:36,  3.78s/it]
 37%|███▋      | 83518/225000 [157:38:34<162:04:56,  4.12s/it]
 37%|███▋      | 83519/225000 [157:38:39<176:46:55,  4.50s/it]
 37%|███▋      | 83520/225000 [157:38:42<160:51:29,  4.09s/it]
 37%|███▋      | 83521/225000 [157:38:45<149:53:39,  3.81s/it]
 37%|███▋      | 83522/225000 [157:38:49<142:21:29,  3.62s/it]
 37%|███▋      | 83523/225000 [157:38:55<170:21:34,  4.33s/it]
 37%|███▋      | 83524/225000 [157:38:58<160:22:07,  4.08s/it]
 37%|███▋      | 83525/225000 [157:39:01<149:20:43,  3.80s/it]
 37%|███▋      | 83526/225000 [157:39:05<149:17:27,  3.80s/it]
 37%|███▋      | 83527/225000 [157:39:08<144:21:07,  3.67s/it]
 37%|███▋      | 83528/225000 [157:39:12<145:14:33,  3.70s/it]
 37%|███▋      | 83529/225000 [157:39:15<139:13:09,  3.54s/it]
 37%|███▋      | 83530/225000 [157:39:19<136:11:29,  3.47s/it]
 37%|███▋      | 83531/225000 [157:39:22<131:33:42,  3.35s/it]
 37%|███▋      | 83532/225000 [157:39:25<129:46:25,  3.30s/it]
 37%|███▋      | 83533/225000 [157:39:28<131:45:12,  3.35s/it]
 37%|███▋      | 83534/225000 [157:39:32<140:31:44,  3.58s/it]
 37%|███▋      | 83535/225000 [157:39:36<139:23:05,  3.55s/it]
 37%|███▋      | 83536/225000 [157:39:40<144:58:07,  3.69s/it]
 37%|███▋      | 83537/225000 [157:39:43<141:11:39,  3.59s/it]
 37%|███▋      | 83538/225000 [157:39:49<169:36:47,  4.32s/it]
 37%|███▋      | 83539/225000 [157:39:58<220:21:54,  5.61s/it]
 37%|███▋      | 83540/225000 [157:40:04<222:46:11,  5.67s/it]
 37%|███▋      | 83541/225000 [157:40:07<189:11:38,  4.81s/it]
 37%|███▋      | 83542/225000 [157:40:13<211:31:26,  5.38s/it]
 37%|███▋      | 83543/225000 [157:40:16<180:51:55,  4.60s/it]
 37%|███▋      | 83544/225000 [157:40:18<153:32:16,  3.91s/it]
 37%|███▋      | 83545/225000 [157:40:26<193:41:57,  4.93s/it]
 37%|███▋      | 83546/225000 [157:40:29<174:15:00,  4.43s/it]
 37%|███▋      | 83547/225000 [157:40:32<158:59:17,  4.05s/it]
 37%|███▋      | 83548/225000 [157:40:35<142:38:23,  3.63s/it]
 37%|███▋      | 83549/225000 [157:40:43<199:34:46,  5.08s/it]
 37%|███▋      | 83550/225000 [157:40:46<176:41:15,  4.50s/it]
                                                              
{'loss': '0.5811', 'grad_norm': '4.064', 'learning_rate': '3.691e-05', 'epoch': '0.3713', 'num_input_tokens_seen': 259033912, 'train_runtime': '5.676e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83550/225000 [157:40:46<176:41:15,  4.50s/it]
 37%|███▋      | 83551/225000 [157:40:49<157:31:20,  4.01s/it]
 37%|███▋      | 83552/225000 [157:40:53<153:41:28,  3.91s/it]
 37%|███▋      | 83553/225000 [157:40:56<148:10:13,  3.77s/it]
 37%|███▋      | 83554/225000 [157:41:00<143:15:42,  3.65s/it]
 37%|███▋      | 83555/225000 [157:41:07<184:28:11,  4.70s/it]
 37%|███▋      | 83556/225000 [157:41:09<157:46:49,  4.02s/it]
 37%|███▋      | 83557/225000 [157:41:13<156:26:30,  3.98s/it]
 37%|███▋      | 83558/225000 [157:41:20<193:55:26,  4.94s/it]
 37%|███▋      | 83559/225000 [157:41:23<164:24:13,  4.18s/it]
 37%|███▋      | 83560/225000 [157:41:25<145:18:16,  3.70s/it]
 37%|███▋      | 83561/225000 [157:41:28<135:18:18,  3.44s/it]
 37%|███▋      | 83562/225000 [157:41:34<168:24:05,  4.29s/it]
 37%|███▋      | 83563/225000 [157:41:40<182:52:36,  4.65s/it]
 37%|███▋      | 83564/225000 [157:41:43<168:37:51,  4.29s/it]
 37%|███▋      | 83565/225000 [157:41:46<154:47:18,  3.94s/it]
 37%|███▋      | 83566/225000 [157:41:49<143:01:53,  3.64s/it]
 37%|███▋      | 83567/225000 [157:41:52<133:46:47,  3.41s/it]
 37%|███▋      | 83568/225000 [157:41:55<130:18:49,  3.32s/it]
 37%|███▋      | 83569/225000 [157:41:59<131:28:12,  3.35s/it]
 37%|███▋      | 83570/225000 [157:42:02<134:29:54,  3.42s/it]
 37%|███▋      | 83571/225000 [157:42:06<131:48:05,  3.35s/it]
 37%|███▋      | 83572/225000 [157:42:09<138:21:38,  3.52s/it]
 37%|███▋      | 83573/225000 [157:42:18<202:13:33,  5.15s/it]
 37%|███▋      | 83574/225000 [157:42:22<186:04:42,  4.74s/it]
 37%|███▋      | 83575/225000 [157:42:25<159:29:03,  4.06s/it]
 37%|███▋      | 83576/225000 [157:42:30<170:34:24,  4.34s/it]
 37%|███▋      | 83577/225000 [157:42:35<177:34:36,  4.52s/it]
 37%|███▋      | 83578/225000 [157:42:38<162:23:01,  4.13s/it]
 37%|███▋      | 83579/225000 [157:42:42<159:37:50,  4.06s/it]
 37%|███▋      | 83580/225000 [157:42:47<170:39:12,  4.34s/it]
 37%|███▋      | 83581/225000 [157:42:52<178:58:50,  4.56s/it]
 37%|███▋      | 83582/225000 [157:42:55<158:44:28,  4.04s/it]
 37%|███▋      | 83583/225000 [157:42:57<142:02:24,  3.62s/it]
 37%|███▋      | 83584/225000 [157:43:00<135:27:28,  3.45s/it]
 37%|███▋      | 83585/225000 [157:43:03<129:50:25,  3.31s/it]
 37%|███▋      | 83586/225000 [157:43:06<123:13:41,  3.14s/it]
 37%|███▋      | 83587/225000 [157:43:11<145:50:26,  3.71s/it]
 37%|███▋      | 83588/225000 [157:43:19<200:14:19,  5.10s/it]
 37%|███▋      | 83589/225000 [157:43:23<179:10:39,  4.56s/it]
 37%|███▋      | 83590/225000 [157:43:26<160:17:19,  4.08s/it]
 37%|███▋      | 83591/225000 [157:43:28<142:14:38,  3.62s/it]
 37%|███▋      | 83592/225000 [157:43:31<134:34:37,  3.43s/it]
 37%|███▋      | 83593/225000 [157:43:40<202:14:24,  5.15s/it]
 37%|███▋      | 83594/225000 [157:43:44<183:15:25,  4.67s/it]
 37%|███▋      | 83595/225000 [157:43:47<158:58:27,  4.05s/it]
 37%|███▋      | 83596/225000 [157:43:50<149:20:08,  3.80s/it]
 37%|███▋      | 83597/225000 [157:43:53<137:48:09,  3.51s/it]
 37%|███▋      | 83598/225000 [157:43:56<134:53:50,  3.43s/it]
 37%|███▋      | 83599/225000 [157:43:59<134:32:42,  3.43s/it]
 37%|███▋      | 83600/225000 [157:44:06<169:35:26,  4.32s/it]
                                                              
{'loss': '0.5804', 'grad_norm': '5.257', 'learning_rate': '3.69e-05', 'epoch': '0.3716', 'num_input_tokens_seen': 259120560, 'train_runtime': '5.678e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83600/225000 [157:44:06<169:35:26,  4.32s/it]
 37%|███▋      | 83601/225000 [157:44:14<216:27:38,  5.51s/it]
 37%|███▋      | 83602/225000 [157:44:17<181:44:55,  4.63s/it]
 37%|███▋      | 83603/225000 [157:44:19<158:24:02,  4.03s/it]
 37%|███▋      | 83604/225000 [157:44:22<140:04:05,  3.57s/it]
 37%|███▋      | 83605/225000 [157:44:26<145:16:00,  3.70s/it]
 37%|███▋      | 83606/225000 [157:44:33<185:21:24,  4.72s/it]
 37%|███▋      | 83607/225000 [157:44:35<161:58:40,  4.12s/it]
 37%|███▋      | 83608/225000 [157:44:38<146:32:04,  3.73s/it]
 37%|███▋      | 83609/225000 [157:44:41<137:46:56,  3.51s/it]
 37%|███▋      | 83610/225000 [157:44:44<133:26:39,  3.40s/it]
 37%|███▋      | 83611/225000 [157:44:47<126:02:22,  3.21s/it]
 37%|███▋      | 83612/225000 [157:44:50<122:10:00,  3.11s/it]
 37%|███▋      | 83613/225000 [157:44:56<156:41:00,  3.99s/it]
 37%|███▋      | 83614/225000 [157:45:04<208:16:17,  5.30s/it]
 37%|███▋      | 83615/225000 [157:45:07<177:24:34,  4.52s/it]
 37%|███▋      | 83616/225000 [157:45:11<166:47:44,  4.25s/it]
 37%|███▋      | 83617/225000 [157:45:14<155:26:14,  3.96s/it]
 37%|███▋      | 83618/225000 [157:45:17<144:06:53,  3.67s/it]
 37%|███▋      | 83619/225000 [157:45:20<135:26:23,  3.45s/it]
 37%|███▋      | 83620/225000 [157:45:24<139:51:35,  3.56s/it]
 37%|███▋      | 83621/225000 [157:45:27<132:52:53,  3.38s/it]
 37%|███▋      | 83622/225000 [157:45:30<131:42:44,  3.35s/it]
 37%|███▋      | 83623/225000 [157:45:33<128:52:18,  3.28s/it]
 37%|███▋      | 83624/225000 [157:45:37<129:32:02,  3.30s/it]
 37%|███▋      | 83625/225000 [157:45:40<129:18:14,  3.29s/it]
 37%|███▋      | 83626/225000 [157:45:45<156:59:22,  4.00s/it]
 37%|███▋      | 83627/225000 [157:45:54<208:59:33,  5.32s/it]
 37%|███▋      | 83628/225000 [157:45:58<195:29:23,  4.98s/it]
 37%|███▋      | 83629/225000 [157:46:00<165:36:40,  4.22s/it]
 37%|███▋      | 83630/225000 [157:46:04<158:36:06,  4.04s/it]
 37%|███▋      | 83631/225000 [157:46:08<151:41:43,  3.86s/it]
 37%|███▋      | 83632/225000 [157:46:11<149:56:14,  3.82s/it]
 37%|███▋      | 83633/225000 [157:46:15<148:14:16,  3.77s/it]
 37%|███▋      | 83634/225000 [157:46:19<148:31:06,  3.78s/it]
 37%|███▋      | 83635/225000 [157:46:22<139:56:06,  3.56s/it]
 37%|███▋      | 83636/225000 [157:46:29<179:37:35,  4.57s/it]
 37%|███▋      | 83637/225000 [157:46:32<167:58:57,  4.28s/it]
 37%|███▋      | 83638/225000 [157:46:36<158:12:33,  4.03s/it]
 37%|███▋      | 83639/225000 [157:46:39<152:06:42,  3.87s/it]
 37%|███▋      | 83640/225000 [157:46:43<145:12:27,  3.70s/it]
 37%|███▋      | 83641/225000 [157:46:46<141:41:36,  3.61s/it]
 37%|███▋      | 83642/225000 [157:46:49<139:32:37,  3.55s/it]
 37%|███▋      | 83643/225000 [157:46:57<192:54:25,  4.91s/it]
 37%|███▋      | 83644/225000 [157:47:04<216:48:27,  5.52s/it]
 37%|███▋      | 83645/225000 [157:47:07<179:17:04,  4.57s/it]
 37%|███▋      | 83646/225000 [157:47:09<157:58:21,  4.02s/it]
 37%|███▋      | 83647/225000 [157:47:14<169:10:43,  4.31s/it]
 37%|███▋      | 83648/225000 [157:47:21<197:52:29,  5.04s/it]
 37%|███▋      | 83649/225000 [157:47:24<167:25:47,  4.26s/it]
 37%|███▋      | 83650/225000 [157:47:27<152:20:27,  3.88s/it]
                                                              
{'loss': '0.579', 'grad_norm': '5.704', 'learning_rate': '3.688e-05', 'epoch': '0.3718', 'num_input_tokens_seen': 259207432, 'train_runtime': '5.68e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83650/225000 [157:47:27<152:20:27,  3.88s/it]
 37%|███▋      | 83651/225000 [157:47:30<144:37:22,  3.68s/it]
 37%|███▋      | 83652/225000 [157:47:33<133:40:13,  3.40s/it]
 37%|███▋      | 83653/225000 [157:47:36<131:26:32,  3.35s/it]
 37%|███▋      | 83654/225000 [157:47:42<163:55:40,  4.18s/it]
 37%|███▋      | 83655/225000 [157:47:50<215:04:53,  5.48s/it]
 37%|███▋      | 83656/225000 [157:47:53<176:47:29,  4.50s/it]
 37%|███▋      | 83657/225000 [157:47:55<153:36:47,  3.91s/it]
 37%|███▋      | 83658/225000 [157:48:01<178:44:23,  4.55s/it]
 37%|███▋      | 83659/225000 [157:48:04<158:18:34,  4.03s/it]
 37%|███▋      | 83660/225000 [157:48:07<145:28:20,  3.71s/it]
 37%|███▋      | 83661/225000 [157:48:10<139:17:41,  3.55s/it]
 37%|███▋      | 83662/225000 [157:48:13<133:45:29,  3.41s/it]
 37%|███▋      | 83663/225000 [157:48:17<133:18:27,  3.40s/it]
 37%|███▋      | 83664/225000 [157:48:20<133:00:58,  3.39s/it]
 37%|███▋      | 83665/225000 [157:48:23<127:37:07,  3.25s/it]
 37%|███▋      | 83666/225000 [157:48:31<184:15:26,  4.69s/it]
 37%|███▋      | 83667/225000 [157:48:37<204:55:20,  5.22s/it]
 37%|███▋      | 83668/225000 [157:48:41<179:05:36,  4.56s/it]
 37%|███▋      | 83669/225000 [157:48:44<163:40:01,  4.17s/it]
 37%|███▋      | 83670/225000 [157:48:47<154:29:23,  3.94s/it]
 37%|███▋      | 83671/225000 [157:48:50<144:08:39,  3.67s/it]
 37%|███▋      | 83672/225000 [157:48:53<137:53:29,  3.51s/it]
 37%|███▋      | 83673/225000 [157:48:58<153:14:40,  3.90s/it]
 37%|███▋      | 83674/225000 [157:49:06<204:38:37,  5.21s/it]
 37%|███▋      | 83675/225000 [157:49:09<169:50:55,  4.33s/it]
 37%|███▋      | 83676/225000 [157:49:11<150:59:37,  3.85s/it]
 37%|███▋      | 83677/225000 [157:49:14<139:20:06,  3.55s/it]
 37%|███▋      | 83678/225000 [157:49:17<130:07:39,  3.31s/it]
 37%|███▋      | 83679/225000 [157:49:26<200:12:41,  5.10s/it]
 37%|███▋      | 83680/225000 [157:49:32<202:50:42,  5.17s/it]
 37%|███▋      | 83681/225000 [157:49:35<179:44:17,  4.58s/it]
 37%|███▋      | 83682/225000 [157:49:39<170:17:36,  4.34s/it]
 37%|███▋      | 83683/225000 [157:49:42<156:45:30,  3.99s/it]
 37%|███▋      | 83684/225000 [157:49:45<148:25:12,  3.78s/it]
 37%|███▋      | 83685/225000 [157:49:48<139:48:32,  3.56s/it]
 37%|███▋      | 83686/225000 [157:49:51<136:14:28,  3.47s/it]
 37%|███▋      | 83687/225000 [157:49:55<134:49:21,  3.43s/it]
 37%|███▋      | 83688/225000 [157:49:58<137:14:26,  3.50s/it]
 37%|███▋      | 83689/225000 [157:50:07<201:12:39,  5.13s/it]
 37%|███▋      | 83690/225000 [157:50:12<194:05:39,  4.94s/it]
 37%|███▋      | 83691/225000 [157:50:14<163:17:20,  4.16s/it]
 37%|███▋      | 83692/225000 [157:50:18<157:29:02,  4.01s/it]
 37%|███▋      | 83693/225000 [157:50:21<150:17:20,  3.83s/it]
 37%|███▋      | 83694/225000 [157:50:25<143:56:26,  3.67s/it]
 37%|███▋      | 83695/225000 [157:50:28<139:28:46,  3.55s/it]
 37%|███▋      | 83696/225000 [157:50:31<132:41:45,  3.38s/it]
 37%|███▋      | 83697/225000 [157:50:34<131:46:57,  3.36s/it]
 37%|███▋      | 83698/225000 [157:50:37<128:56:10,  3.28s/it]
 37%|███▋      | 83699/225000 [157:50:45<177:39:13,  4.53s/it]
 37%|███▋      | 83700/225000 [157:50:51<202:42:29,  5.16s/it]
                                                              
{'loss': '0.5995', 'grad_norm': '3.96', 'learning_rate': '3.686e-05', 'epoch': '0.372', 'num_input_tokens_seen': 259294272, 'train_runtime': '5.683e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83700/225000 [157:50:51<202:42:29,  5.16s/it]
 37%|███▋      | 83701/225000 [157:50:54<172:54:29,  4.41s/it]
 37%|███▋      | 83702/225000 [157:50:59<185:19:28,  4.72s/it]
 37%|███▋      | 83703/225000 [157:51:02<163:07:26,  4.16s/it]
 37%|███▋      | 83704/225000 [157:51:05<148:44:19,  3.79s/it]
 37%|███▋      | 83705/225000 [157:51:13<195:47:42,  4.99s/it]
 37%|███▋      | 83706/225000 [157:51:17<182:53:23,  4.66s/it]
 37%|███▋      | 83707/225000 [157:51:19<158:06:35,  4.03s/it]
 37%|███▋      | 83708/225000 [157:51:23<148:11:04,  3.78s/it]
 37%|███▋      | 83709/225000 [157:51:25<137:05:15,  3.49s/it]
 37%|███▋      | 83710/225000 [157:51:29<133:52:53,  3.41s/it]
 37%|███▋      | 83711/225000 [157:51:32<137:17:31,  3.50s/it]
 37%|███▋      | 83712/225000 [157:51:41<202:23:16,  5.16s/it]
 37%|███▋      | 83713/225000 [157:51:45<185:16:51,  4.72s/it]
 37%|███▋      | 83714/225000 [157:51:48<158:48:07,  4.05s/it]
 37%|███▋      | 83715/225000 [157:51:50<141:36:41,  3.61s/it]
 37%|███▋      | 83716/225000 [157:51:59<207:29:28,  5.29s/it]
 37%|███▋      | 83717/225000 [157:52:02<172:24:43,  4.39s/it]
 37%|███▋      | 83718/225000 [157:52:06<174:16:21,  4.44s/it]
 37%|███▋      | 83719/225000 [157:52:09<156:01:35,  3.98s/it]
 37%|███▋      | 83720/225000 [157:52:12<142:49:58,  3.64s/it]
 37%|███▋      | 83721/225000 [157:52:15<136:30:10,  3.48s/it]
 37%|███▋      | 83722/225000 [157:52:18<131:52:57,  3.36s/it]
 37%|███▋      | 83723/225000 [157:52:24<162:11:13,  4.13s/it]
 37%|███▋      | 83724/225000 [157:52:31<193:18:05,  4.93s/it]
 37%|███▋      | 83725/225000 [157:52:34<177:51:54,  4.53s/it]
 37%|███▋      | 83726/225000 [157:52:38<167:12:31,  4.26s/it]
 37%|███▋      | 83727/225000 [157:52:41<157:26:48,  4.01s/it]
 37%|███▋      | 83728/225000 [157:52:44<145:31:48,  3.71s/it]
 37%|███▋      | 83729/225000 [157:52:48<144:44:29,  3.69s/it]
 37%|███▋      | 83730/225000 [157:52:51<138:06:00,  3.52s/it]
 37%|███▋      | 83731/225000 [157:52:55<136:13:58,  3.47s/it]
 37%|███▋      | 83732/225000 [157:52:58<133:16:31,  3.40s/it]
 37%|███▋      | 83733/225000 [157:53:01<131:13:04,  3.34s/it]
 37%|███▋      | 83734/225000 [157:53:05<133:24:27,  3.40s/it]
 37%|███▋      | 83735/225000 [157:53:08<134:21:51,  3.42s/it]
 37%|███▋      | 83736/225000 [157:53:11<132:59:47,  3.39s/it]
 37%|███▋      | 83737/225000 [157:53:15<131:15:51,  3.35s/it]
 37%|███▋      | 83738/225000 [157:53:20<158:35:01,  4.04s/it]
 37%|███▋      | 83739/225000 [157:53:29<211:12:23,  5.38s/it]
 37%|███▋      | 83740/225000 [157:53:32<180:41:52,  4.61s/it]
 37%|███▋      | 83741/225000 [157:53:35<160:44:37,  4.10s/it]
 37%|███▋      | 83742/225000 [157:53:37<145:39:30,  3.71s/it]
 37%|███▋      | 83743/225000 [157:53:41<140:52:34,  3.59s/it]
 37%|███▋      | 83744/225000 [157:53:44<140:33:55,  3.58s/it]
 37%|███▋      | 83745/225000 [157:53:47<136:54:54,  3.49s/it]
 37%|███▋      | 83746/225000 [157:53:53<166:41:16,  4.25s/it]
 37%|███▋      | 83747/225000 [157:54:01<201:29:18,  5.14s/it]
 37%|███▋      | 83748/225000 [157:54:04<178:44:47,  4.56s/it]
 37%|███▋      | 83749/225000 [157:54:07<167:15:35,  4.26s/it]
 37%|███▋      | 83750/225000 [157:54:12<167:15:20,  4.26s/it]
                                                              
{'loss': '0.6188', 'grad_norm': '4.191', 'learning_rate': '3.685e-05', 'epoch': '0.3722', 'num_input_tokens_seen': 259381424, 'train_runtime': '5.685e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83750/225000 [157:54:12<167:15:20,  4.26s/it]
 37%|███▋      | 83751/225000 [157:54:16<162:11:24,  4.13s/it]
 37%|███▋      | 83752/225000 [157:54:19<151:02:49,  3.85s/it]
 37%|███▋      | 83753/225000 [157:54:23<151:10:20,  3.85s/it]
 37%|███▋      | 83754/225000 [157:54:26<147:38:39,  3.76s/it]
 37%|███▋      | 83755/225000 [157:54:30<144:12:40,  3.68s/it]
 37%|███▋      | 83756/225000 [157:54:34<151:09:26,  3.85s/it]
 37%|███▋      | 83757/225000 [157:54:37<145:54:21,  3.72s/it]
 37%|███▋      | 83758/225000 [157:54:41<145:51:48,  3.72s/it]
 37%|███▋      | 83759/225000 [157:54:44<139:51:59,  3.56s/it]
 37%|███▋      | 83760/225000 [157:54:48<143:42:34,  3.66s/it]
 37%|███▋      | 83761/225000 [157:54:51<139:56:33,  3.57s/it]
 37%|███▋      | 83762/225000 [157:54:55<138:19:13,  3.53s/it]
 37%|███▋      | 83763/225000 [157:54:59<139:18:07,  3.55s/it]
 37%|███▋      | 83764/225000 [157:55:02<139:27:50,  3.55s/it]
 37%|███▋      | 83765/225000 [157:55:05<137:43:31,  3.51s/it]
 37%|███▋      | 83766/225000 [157:55:12<175:43:06,  4.48s/it]
 37%|███▋      | 83767/225000 [157:55:22<244:01:42,  6.22s/it]
 37%|███▋      | 83768/225000 [157:55:26<211:47:07,  5.40s/it]
 37%|███▋      | 83769/225000 [157:55:29<184:39:55,  4.71s/it]
 37%|███▋      | 83770/225000 [157:55:32<167:40:23,  4.27s/it]
 37%|███▋      | 83771/225000 [157:55:36<157:38:51,  4.02s/it]
 37%|███▋      | 83772/225000 [157:55:39<145:45:06,  3.72s/it]
 37%|███▋      | 83773/225000 [157:55:42<139:42:03,  3.56s/it]
 37%|███▋      | 83774/225000 [157:55:45<137:54:21,  3.52s/it]
 37%|███▋      | 83775/225000 [157:55:55<205:35:22,  5.24s/it]
 37%|███▋      | 83776/225000 [157:56:00<212:34:29,  5.42s/it]
 37%|███▋      | 83777/225000 [157:56:04<184:37:03,  4.71s/it]
 37%|███▋      | 83778/225000 [157:56:07<165:20:33,  4.21s/it]
 37%|███▋      | 83779/225000 [157:56:10<153:50:01,  3.92s/it]
 37%|███▋      | 83780/225000 [157:56:18<198:58:01,  5.07s/it]
 37%|███▋      | 83781/225000 [157:56:22<188:35:38,  4.81s/it]
 37%|███▋      | 83782/225000 [157:56:25<166:10:04,  4.24s/it]
 37%|███▋      | 83783/225000 [157:56:27<147:53:17,  3.77s/it]
 37%|███▋      | 83784/225000 [157:56:31<141:12:53,  3.60s/it]
 37%|███▋      | 83785/225000 [157:56:34<136:34:25,  3.48s/it]
 37%|███▋      | 83786/225000 [157:56:37<133:02:35,  3.39s/it]
 37%|███▋      | 83787/225000 [157:56:41<137:49:10,  3.51s/it]
 37%|███▋      | 83788/225000 [157:56:44<131:22:34,  3.35s/it]
 37%|███▋      | 83789/225000 [157:56:47<130:56:03,  3.34s/it]
 37%|███▋      | 83790/225000 [157:56:51<133:33:34,  3.40s/it]
 37%|███▋      | 83791/225000 [157:56:57<167:45:42,  4.28s/it]
 37%|███▋      | 83792/225000 [157:57:07<235:56:40,  6.02s/it]
 37%|███▋      | 83793/225000 [157:57:10<201:56:03,  5.15s/it]
 37%|███▋      | 83794/225000 [157:57:13<169:50:50,  4.33s/it]
 37%|███▋      | 83795/225000 [157:57:18<182:05:50,  4.64s/it]
 37%|███▋      | 83796/225000 [157:57:22<180:11:20,  4.59s/it]
 37%|███▋      | 83797/225000 [157:57:25<155:14:46,  3.96s/it]
 37%|███▋      | 83798/225000 [157:57:28<142:32:43,  3.63s/it]
 37%|███▋      | 83799/225000 [157:57:31<141:24:45,  3.61s/it]
 37%|███▋      | 83800/225000 [157:57:38<179:34:30,  4.58s/it]
                                                              
{'loss': '0.602', 'grad_norm': '5.931', 'learning_rate': '3.683e-05', 'epoch': '0.3724', 'num_input_tokens_seen': 259468232, 'train_runtime': '5.687e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83800/225000 [157:57:38<179:34:30,  4.58s/it]
 37%|███▋      | 83801/225000 [157:57:43<178:28:26,  4.55s/it]
 37%|███▋      | 83802/225000 [157:57:46<163:22:00,  4.17s/it]
 37%|███▋      | 83803/225000 [157:57:49<150:09:29,  3.83s/it]
 37%|███▋      | 83804/225000 [157:57:52<144:12:06,  3.68s/it]
 37%|███▋      | 83805/225000 [157:57:56<143:55:46,  3.67s/it]
 37%|███▋      | 83806/225000 [157:57:59<138:53:45,  3.54s/it]
 37%|███▋      | 83807/225000 [157:58:03<137:04:23,  3.49s/it]
 37%|███▋      | 83808/225000 [157:58:05<129:13:44,  3.29s/it]
 37%|███▋      | 83809/225000 [157:58:08<126:44:26,  3.23s/it]
 37%|███▋      | 83810/225000 [157:58:15<169:47:49,  4.33s/it]
 37%|███▋      | 83811/225000 [157:58:23<205:47:13,  5.25s/it]
 37%|███▋      | 83812/225000 [157:58:25<172:56:37,  4.41s/it]
 37%|███▋      | 83813/225000 [157:58:28<154:26:30,  3.94s/it]
 37%|███▋      | 83814/225000 [157:58:31<138:10:39,  3.52s/it]
 37%|███▋      | 83815/225000 [157:58:34<134:29:52,  3.43s/it]
 37%|███▋      | 83816/225000 [157:58:42<194:15:17,  4.95s/it]
 37%|███▋      | 83817/225000 [157:58:48<209:00:47,  5.33s/it]
 37%|███▋      | 83818/225000 [157:58:51<174:52:14,  4.46s/it]
 37%|███▋      | 83819/225000 [157:58:54<158:16:09,  4.04s/it]
 37%|███▋      | 83820/225000 [157:58:57<146:59:14,  3.75s/it]
 37%|███▋      | 83821/225000 [157:59:01<144:27:07,  3.68s/it]
 37%|███▋      | 83822/225000 [157:59:04<141:13:55,  3.60s/it]
 37%|███▋      | 83823/225000 [157:59:07<137:29:20,  3.51s/it]
 37%|███▋      | 83824/225000 [157:59:11<134:39:20,  3.43s/it]
 37%|███▋      | 83825/225000 [157:59:14<134:22:36,  3.43s/it]
 37%|███▋      | 83826/225000 [157:59:17<128:00:43,  3.26s/it]
 37%|███▋      | 83827/225000 [157:59:23<165:02:41,  4.21s/it]
 37%|███▋      | 83828/225000 [157:59:31<205:45:14,  5.25s/it]
 37%|███▋      | 83829/225000 [157:59:34<175:41:51,  4.48s/it]
 37%|███▋      | 83830/225000 [157:59:40<192:54:47,  4.92s/it]
 37%|███▋      | 83831/225000 [157:59:44<190:08:44,  4.85s/it]
 37%|███▋      | 83832/225000 [157:59:47<162:31:41,  4.14s/it]
 37%|███▋      | 83833/225000 [157:59:53<182:11:41,  4.65s/it]
 37%|███▋      | 83834/225000 [157:59:56<171:36:57,  4.38s/it]
 37%|███▋      | 83835/225000 [158:00:00<162:59:02,  4.16s/it]
 37%|███▋      | 83836/225000 [158:00:03<150:37:01,  3.84s/it]
 37%|███▋      | 83837/225000 [158:00:06<143:58:47,  3.67s/it]
 37%|███▋      | 83838/225000 [158:00:10<139:44:28,  3.56s/it]
 37%|███▋      | 83839/225000 [158:00:13<137:40:51,  3.51s/it]
 37%|███▋      | 83840/225000 [158:00:16<135:20:57,  3.45s/it]
 37%|███▋      | 83841/225000 [158:00:20<133:53:52,  3.41s/it]
 37%|███▋      | 83842/225000 [158:00:23<133:59:31,  3.42s/it]
 37%|███▋      | 83843/225000 [158:00:27<135:58:32,  3.47s/it]
 37%|███▋      | 83844/225000 [158:00:30<139:19:58,  3.55s/it]
 37%|███▋      | 83845/225000 [158:00:34<138:17:59,  3.53s/it]
 37%|███▋      | 83846/225000 [158:00:38<141:23:41,  3.61s/it]
 37%|███▋      | 83847/225000 [158:00:41<137:40:46,  3.51s/it]
 37%|███▋      | 83848/225000 [158:00:45<139:44:46,  3.56s/it]
 37%|███▋      | 83849/225000 [158:00:53<200:35:02,  5.12s/it]
 37%|███▋      | 83850/225000 [158:01:02<246:31:41,  6.29s/it]
                                                              
{'loss': '0.5852', 'grad_norm': '6.604', 'learning_rate': '3.682e-05', 'epoch': '0.3727', 'num_input_tokens_seen': 259555640, 'train_runtime': '5.689e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83850/225000 [158:01:02<246:31:41,  6.29s/it]
 37%|███▋      | 83851/225000 [158:01:07<225:18:54,  5.75s/it]
 37%|███▋      | 83852/225000 [158:01:09<188:25:57,  4.81s/it]
 37%|███▋      | 83853/225000 [158:01:12<160:53:17,  4.10s/it]
 37%|███▋      | 83854/225000 [158:01:18<185:08:13,  4.72s/it]
 37%|███▋      | 83855/225000 [158:01:21<167:38:54,  4.28s/it]
 37%|███▋      | 83856/225000 [158:01:24<146:01:06,  3.72s/it]
 37%|███▋      | 83857/225000 [158:01:26<132:17:13,  3.37s/it]
 37%|███▋      | 83858/225000 [158:01:32<157:27:39,  4.02s/it]
 37%|███▋      | 83859/225000 [158:01:38<179:27:50,  4.58s/it]
 37%|███▋      | 83860/225000 [158:01:41<160:49:27,  4.10s/it]
 37%|███▋      | 83861/225000 [158:01:44<149:48:15,  3.82s/it]
 37%|███▋      | 83862/225000 [158:01:47<145:16:19,  3.71s/it]
 37%|███▋      | 83863/225000 [158:01:50<138:09:12,  3.52s/it]
 37%|███▋      | 83864/225000 [158:01:53<131:13:54,  3.35s/it]
 37%|███▋      | 83865/225000 [158:01:57<129:12:13,  3.30s/it]
 37%|███▋      | 83866/225000 [158:02:00<132:48:33,  3.39s/it]
 37%|███▋      | 83867/225000 [158:02:03<128:36:11,  3.28s/it]
 37%|███▋      | 83868/225000 [158:02:09<153:18:24,  3.91s/it]
 37%|███▋      | 83869/225000 [158:02:19<230:10:07,  5.87s/it]
 37%|███▋      | 83870/225000 [158:02:23<207:15:38,  5.29s/it]
 37%|███▋      | 83871/225000 [158:02:26<184:21:23,  4.70s/it]
 37%|███▋      | 83872/225000 [158:02:29<164:22:55,  4.19s/it]
 37%|███▋      | 83873/225000 [158:02:32<151:09:49,  3.86s/it]
 37%|███▋      | 83874/225000 [158:02:39<188:53:38,  4.82s/it]
 37%|███▋      | 83875/225000 [158:02:42<165:52:20,  4.23s/it]
 37%|███▋      | 83876/225000 [158:02:48<185:19:35,  4.73s/it]
 37%|███▋      | 83877/225000 [158:02:53<191:33:51,  4.89s/it]
 37%|███▋      | 83878/225000 [158:02:57<170:45:15,  4.36s/it]
 37%|███▋      | 83879/225000 [158:03:00<156:12:25,  3.98s/it]
 37%|███▋      | 83880/225000 [158:03:03<148:07:14,  3.78s/it]
 37%|███▋      | 83881/225000 [158:03:07<145:20:30,  3.71s/it]
 37%|███▋      | 83882/225000 [158:03:10<145:07:06,  3.70s/it]
 37%|███▋      | 83883/225000 [158:03:14<141:36:07,  3.61s/it]
 37%|███▋      | 83884/225000 [158:03:17<135:40:15,  3.46s/it]
 37%|███▋      | 83885/225000 [158:03:20<129:02:36,  3.29s/it]
 37%|███▋      | 83886/225000 [158:03:23<130:59:33,  3.34s/it]
 37%|███▋      | 83887/225000 [158:03:26<128:59:55,  3.29s/it]
 37%|███▋      | 83888/225000 [158:03:34<177:09:16,  4.52s/it]
 37%|███▋      | 83889/225000 [158:03:41<206:54:17,  5.28s/it]
 37%|███▋      | 83890/225000 [158:03:43<176:41:40,  4.51s/it]
 37%|███▋      | 83891/225000 [158:03:46<158:10:24,  4.04s/it]
 37%|███▋      | 83892/225000 [158:03:49<145:14:02,  3.71s/it]
 37%|███▋      | 83893/225000 [158:03:52<133:22:38,  3.40s/it]
 37%|███▋      | 83894/225000 [158:03:55<130:32:51,  3.33s/it]
 37%|███▋      | 83895/225000 [158:04:04<190:23:58,  4.86s/it]
 37%|███▋      | 83896/225000 [158:04:08<183:48:54,  4.69s/it]
 37%|███▋      | 83897/225000 [158:04:11<163:05:57,  4.16s/it]
 37%|███▋      | 83898/225000 [158:04:14<150:26:58,  3.84s/it]
 37%|███▋      | 83899/225000 [158:04:17<140:00:47,  3.57s/it]
 37%|███▋      | 83900/225000 [158:04:20<131:12:50,  3.35s/it]
                                                              
{'loss': '0.5813', 'grad_norm': '7.462', 'learning_rate': '3.68e-05', 'epoch': '0.3729', 'num_input_tokens_seen': 259642456, 'train_runtime': '5.691e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83900/225000 [158:04:20<131:12:50,  3.35s/it]
 37%|███▋      | 83901/225000 [158:04:23<132:43:26,  3.39s/it]
 37%|███▋      | 83902/225000 [158:04:26<130:08:56,  3.32s/it]
 37%|███▋      | 83903/225000 [158:04:30<139:17:13,  3.55s/it]
 37%|███▋      | 83904/225000 [158:04:40<210:22:29,  5.37s/it]
 37%|███▋      | 83905/225000 [158:04:43<183:43:11,  4.69s/it]
 37%|███▋      | 83906/225000 [158:04:46<165:16:30,  4.22s/it]
 37%|███▋      | 83907/225000 [158:04:49<150:55:04,  3.85s/it]
 37%|███▋      | 83908/225000 [158:04:52<140:29:59,  3.58s/it]
 37%|███▋      | 83909/225000 [158:04:58<173:13:35,  4.42s/it]
 37%|███▋      | 83910/225000 [158:05:06<204:42:18,  5.22s/it]
 37%|███▋      | 83911/225000 [158:05:08<174:44:56,  4.46s/it]
 37%|███▋      | 83912/225000 [158:05:15<199:21:05,  5.09s/it]
 37%|███▋      | 83913/225000 [158:05:17<169:15:59,  4.32s/it]
 37%|███▋      | 83914/225000 [158:05:20<146:52:23,  3.75s/it]
 37%|███▋      | 83915/225000 [158:05:23<135:16:45,  3.45s/it]
 37%|███▋      | 83916/225000 [158:05:30<181:11:46,  4.62s/it]
 37%|███▋      | 83917/225000 [158:05:33<164:55:33,  4.21s/it]
 37%|███▋      | 83918/225000 [158:05:36<145:36:35,  3.72s/it]
 37%|███▋      | 83919/225000 [158:05:39<136:40:42,  3.49s/it]
 37%|███▋      | 83920/225000 [158:05:44<156:32:05,  3.99s/it]
 37%|███▋      | 83921/225000 [158:05:50<186:09:50,  4.75s/it]
 37%|███▋      | 83922/225000 [158:05:54<167:49:41,  4.28s/it]
 37%|███▋      | 83923/225000 [158:05:57<157:12:31,  4.01s/it]
 37%|███▋      | 83924/225000 [158:06:00<145:03:18,  3.70s/it]
 37%|███▋      | 83925/225000 [158:06:03<142:02:22,  3.62s/it]
 37%|███▋      | 83926/225000 [158:06:07<137:17:23,  3.50s/it]
 37%|███▋      | 83927/225000 [158:06:10<135:04:01,  3.45s/it]
 37%|███▋      | 83928/225000 [158:06:14<141:56:52,  3.62s/it]
 37%|███▋      | 83929/225000 [158:06:17<136:57:12,  3.49s/it]
 37%|███▋      | 83930/225000 [158:06:24<179:42:33,  4.59s/it]
 37%|███▋      | 83931/225000 [158:06:30<192:16:29,  4.91s/it]
 37%|███▋      | 83932/225000 [158:06:33<174:29:32,  4.45s/it]
 37%|███▋      | 83933/225000 [158:06:36<158:56:25,  4.06s/it]
 37%|███▋      | 83934/225000 [158:06:40<151:05:58,  3.86s/it]
 37%|███▋      | 83935/225000 [158:06:44<154:00:01,  3.93s/it]
 37%|███▋      | 83936/225000 [158:06:48<151:36:44,  3.87s/it]
 37%|███▋      | 83937/225000 [158:06:51<146:46:38,  3.75s/it]
 37%|███▋      | 83938/225000 [158:06:54<142:16:43,  3.63s/it]
 37%|███▋      | 83939/225000 [158:06:58<136:34:34,  3.49s/it]
 37%|███▋      | 83940/225000 [158:07:01<137:33:58,  3.51s/it]
 37%|███▋      | 83941/225000 [158:07:05<136:40:53,  3.49s/it]
 37%|███▋      | 83942/225000 [158:07:08<134:37:13,  3.44s/it]
 37%|███▋      | 83943/225000 [158:07:12<141:37:10,  3.61s/it]
 37%|███▋      | 83944/225000 [158:07:18<167:04:36,  4.26s/it]
 37%|███▋      | 83945/225000 [158:07:22<164:27:04,  4.20s/it]
 37%|███▋      | 83946/225000 [158:07:25<151:33:12,  3.87s/it]
 37%|███▋      | 83947/225000 [158:07:29<152:24:07,  3.89s/it]
 37%|███▋      | 83948/225000 [158:07:32<147:05:57,  3.75s/it]
 37%|███▋      | 83949/225000 [158:07:35<140:23:34,  3.58s/it]
 37%|███▋      | 83950/225000 [158:07:39<140:33:51,  3.59s/it]
                                                              
{'loss': '0.5889', 'grad_norm': '4.288', 'learning_rate': '3.678e-05', 'epoch': '0.3731', 'num_input_tokens_seen': 259729912, 'train_runtime': '5.693e+05', 'train_tokens_per_second': '456.3'}

 37%|███▋      | 83950/225000 [158:07:39<140:33:51,  3.59s/it]
 37%|███▋      | 83951/225000 [158:07:44<159:34:18,  4.07s/it]
 37%|███▋      | 83952/225000 [158:07:54<225:09:26,  5.75s/it]
 37%|███▋      | 83953/225000 [158:07:58<200:21:57,  5.11s/it]
 37%|███▋      | 83954/225000 [158:08:05<224:32:03,  5.73s/it]
 37%|███▋      | 83955/225000 [158:08:07<184:21:17,  4.71s/it]
 37%|███▋      | 83956/225000 [158:08:10<163:16:18,  4.17s/it]
 37%|███▋      | 83957/225000 [158:08:14<163:04:02,  4.16s/it]
 37%|███▋      | 83958/225000 [158:08:17<151:33:29,  3.87s/it]
 37%|███▋      | 83959/225000 [158:08:20<143:45:22,  3.67s/it]
 37%|███▋      | 83960/225000 [158:08:24<137:06:55,  3.50s/it]
 37%|███▋      | 83961/225000 [158:08:27<141:29:09,  3.61s/it]
 37%|███▋      | 83962/225000 [158:08:31<138:11:57,  3.53s/it]
 37%|███▋      | 83963/225000 [158:08:34<134:58:13,  3.45s/it]
 37%|███▋      | 83964/225000 [158:08:40<159:57:41,  4.08s/it]
 37%|███▋      | 83965/225000 [158:08:46<190:13:53,  4.86s/it]
 37%|███▋      | 83966/225000 [158:08:50<175:24:21,  4.48s/it]
 37%|███▋      | 83967/225000 [158:08:53<160:40:56,  4.10s/it]
 37%|███▋      | 83968/225000 [158:08:57<156:56:01,  4.01s/it]
 37%|███▋      | 83969/225000 [158:09:00<147:58:55,  3.78s/it]
 37%|███▋      | 83970/225000 [158:09:04<148:35:06,  3.79s/it]
 37%|███▋      | 83971/225000 [158:09:07<143:33:29,  3.66s/it]
 37%|███▋      | 83972/225000 [158:09:11<141:42:53,  3.62s/it]
 37%|███▋      | 83973/225000 [158:09:14<141:14:25,  3.61s/it]
 37%|███▋      | 83974/225000 [158:09:19<148:07:50,  3.78s/it]
 37%|███▋      | 83975/225000 [158:09:22<144:45:16,  3.70s/it]
 37%|███▋      | 83976/225000 [158:09:26<145:59:34,  3.73s/it]
 37%|███▋      | 83977/225000 [158:09:33<187:08:21,  4.78s/it]
 37%|███▋      | 83978/225000 [158:09:38<192:17:07,  4.91s/it]
 37%|███▋      | 83979/225000 [158:09:42<175:25:22,  4.48s/it]
 37%|███▋      | 83980/225000 [158:09:45<163:40:49,  4.18s/it]
 37%|███▋      | 83981/225000 [158:09:49<155:14:26,  3.96s/it]
 37%|███▋      | 83982/225000 [158:09:52<151:48:50,  3.88s/it]
 37%|███▋      | 83983/225000 [158:09:56<154:35:50,  3.95s/it]
 37%|███▋      | 83984/225000 [158:10:00<150:56:49,  3.85s/it]
 37%|███▋      | 83985/225000 [158:10:04<155:13:05,  3.96s/it]
 37%|███▋      | 83986/225000 [158:10:14<221:22:55,  5.65s/it]
 37%|███▋      | 83987/225000 [158:10:19<209:29:59,  5.35s/it]
 37%|███▋      | 83988/225000 [158:10:21<179:41:57,  4.59s/it]
 37%|███▋      | 83989/225000 [158:10:24<155:21:34,  3.97s/it]
 37%|███▋      | 83990/225000 [158:10:27<142:06:37,  3.63s/it]
 37%|███▋      | 83991/225000 [158:10:30<139:40:21,  3.57s/it]
 37%|███▋      | 83992/225000 [158:10:33<133:07:46,  3.40s/it]
 37%|███▋      | 83993/225000 [158:10:36<128:24:16,  3.28s/it]
 37%|███▋      | 83994/225000 [158:10:40<134:03:17,  3.42s/it]
 37%|███▋      | 83995/225000 [158:10:43<129:35:24,  3.31s/it]
 37%|███▋      | 83996/225000 [158:10:47<138:47:15,  3.54s/it]
 37%|███▋      | 83997/225000 [158:10:55<188:30:05,  4.81s/it]
 37%|███▋      | 83998/225000 [158:11:03<227:47:31,  5.82s/it]
 37%|███▋      | 83999/225000 [158:11:07<208:51:34,  5.33s/it]
 37%|███▋      | 84000/225000 [158:11:10<183:06:53,  4.68s/it]
                                                              
{'loss': '0.6302', 'grad_norm': '5.09', 'learning_rate': '3.677e-05', 'epoch': '0.3733', 'num_input_tokens_seen': 259818016, 'train_runtime': '5.695e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84000/225000 [158:11:10<183:06:53,  4.68s/it]
 37%|███▋      | 84001/225000 [158:11:14<165:35:21,  4.23s/it]
 37%|███▋      | 84002/225000 [158:11:17<156:03:04,  3.98s/it]
 37%|███▋      | 84003/225000 [158:11:20<143:38:58,  3.67s/it]
 37%|███▋      | 84004/225000 [158:11:23<137:27:49,  3.51s/it]
 37%|███▋      | 84005/225000 [158:11:26<128:58:35,  3.29s/it]
 37%|███▋      | 84006/225000 [158:11:34<186:14:01,  4.76s/it]
 37%|███▋      | 84007/225000 [158:11:40<203:09:49,  5.19s/it]
 37%|███▋      | 84008/225000 [158:11:43<177:28:16,  4.53s/it]
 37%|███▋      | 84009/225000 [158:11:46<155:05:13,  3.96s/it]
 37%|███▋      | 84010/225000 [158:11:49<144:17:45,  3.68s/it]
 37%|███▋      | 84011/225000 [158:11:54<157:01:25,  4.01s/it]
 37%|███▋      | 84012/225000 [158:12:03<216:04:30,  5.52s/it]
 37%|███▋      | 84013/225000 [158:12:06<185:43:30,  4.74s/it]
 37%|███▋      | 84014/225000 [158:12:08<163:49:49,  4.18s/it]
 37%|███▋      | 84015/225000 [158:12:12<152:11:30,  3.89s/it]
 37%|███▋      | 84016/225000 [158:12:14<139:49:24,  3.57s/it]
 37%|███▋      | 84017/225000 [158:12:17<130:03:18,  3.32s/it]
 37%|███▋      | 84018/225000 [158:12:24<166:23:52,  4.25s/it]
 37%|███▋      | 84019/225000 [158:12:30<191:38:08,  4.89s/it]
 37%|███▋      | 84020/225000 [158:12:32<161:32:44,  4.13s/it]
 37%|███▋      | 84021/225000 [158:12:35<140:29:47,  3.59s/it]
 37%|███▋      | 84022/225000 [158:12:42<180:48:58,  4.62s/it]
 37%|███▋      | 84023/225000 [158:12:46<174:35:08,  4.46s/it]
 37%|███▋      | 84024/225000 [158:12:49<155:01:46,  3.96s/it]
 37%|███▋      | 84025/225000 [158:12:51<139:31:48,  3.56s/it]
 37%|███▋      | 84026/225000 [158:12:54<128:07:07,  3.27s/it]
 37%|███▋      | 84027/225000 [158:13:03<196:26:15,  5.02s/it]
 37%|███▋      | 84028/225000 [158:13:05<166:38:58,  4.26s/it]
 37%|███▋      | 84029/225000 [158:13:08<146:04:26,  3.73s/it]
 37%|███▋      | 84030/225000 [158:13:10<132:42:09,  3.39s/it]
 37%|███▋      | 84031/225000 [158:13:18<180:22:55,  4.61s/it]
 37%|███▋      | 84032/225000 [158:13:21<165:48:09,  4.23s/it]
 37%|███▋      | 84033/225000 [158:13:24<146:16:46,  3.74s/it]
 37%|███▋      | 84034/225000 [158:13:26<132:54:22,  3.39s/it]
 37%|███▋      | 84035/225000 [158:13:29<126:14:01,  3.22s/it]
 37%|███▋      | 84036/225000 [158:13:32<122:21:54,  3.13s/it]
 37%|███▋      | 84037/225000 [158:13:40<172:15:08,  4.40s/it]
 37%|███▋      | 84038/225000 [158:13:43<162:39:35,  4.15s/it]
 37%|███▋      | 84039/225000 [158:13:46<145:30:35,  3.72s/it]
 37%|███▋      | 84040/225000 [158:13:49<138:29:04,  3.54s/it]
 37%|███▋      | 84041/225000 [158:13:52<131:53:26,  3.37s/it]
 37%|███▋      | 84042/225000 [158:13:55<129:27:05,  3.31s/it]
 37%|███▋      | 84043/225000 [158:13:58<127:34:00,  3.26s/it]
 37%|███▋      | 84044/225000 [158:14:06<178:32:04,  4.56s/it]
 37%|███▋      | 84045/225000 [158:14:12<192:21:08,  4.91s/it]
 37%|███▋      | 84046/225000 [158:14:14<164:51:45,  4.21s/it]
 37%|███▋      | 84047/225000 [158:14:17<146:06:24,  3.73s/it]
 37%|███▋      | 84048/225000 [158:14:23<181:05:14,  4.63s/it]
 37%|███▋      | 84049/225000 [158:14:26<154:48:39,  3.95s/it]
 37%|███▋      | 84050/225000 [158:14:28<138:52:28,  3.55s/it]
                                                              
{'loss': '0.5678', 'grad_norm': '3.161', 'learning_rate': '3.675e-05', 'epoch': '0.3736', 'num_input_tokens_seen': 259904872, 'train_runtime': '5.697e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84050/225000 [158:14:28<138:52:28,  3.55s/it]
 37%|███▋      | 84051/225000 [158:14:32<140:54:50,  3.60s/it]
 37%|███▋      | 84052/225000 [158:14:40<191:40:25,  4.90s/it]
 37%|███▋      | 84053/225000 [158:14:43<169:22:43,  4.33s/it]
 37%|███▋      | 84054/225000 [158:14:46<157:02:20,  4.01s/it]
 37%|███▋      | 84055/225000 [158:14:49<142:59:49,  3.65s/it]
 37%|███▋      | 84056/225000 [158:14:53<143:30:10,  3.67s/it]
 37%|███▋      | 84057/225000 [158:14:56<135:38:57,  3.46s/it]
 37%|███▋      | 84058/225000 [158:14:59<134:32:54,  3.44s/it]
 37%|███▋      | 84059/225000 [158:15:08<196:02:37,  5.01s/it]
 37%|███▋      | 84060/225000 [158:15:13<191:54:45,  4.90s/it]
 37%|███▋      | 84061/225000 [158:15:16<170:13:04,  4.35s/it]
 37%|███▋      | 84062/225000 [158:15:19<153:01:02,  3.91s/it]
 37%|███▋      | 84063/225000 [158:15:22<149:39:20,  3.82s/it]
 37%|███▋      | 84064/225000 [158:15:26<151:39:26,  3.87s/it]
 37%|███▋      | 84065/225000 [158:15:30<146:06:28,  3.73s/it]
 37%|███▋      | 84066/225000 [158:15:33<141:17:19,  3.61s/it]
 37%|███▋      | 84067/225000 [158:15:37<146:32:03,  3.74s/it]
 37%|███▋      | 84068/225000 [158:15:45<198:35:05,  5.07s/it]
 37%|███▋      | 84069/225000 [158:15:51<210:32:19,  5.38s/it]
 37%|███▋      | 84070/225000 [158:15:56<199:33:41,  5.10s/it]
 37%|███▋      | 84071/225000 [158:16:00<190:44:24,  4.87s/it]
 37%|███▋      | 84072/225000 [158:16:02<160:33:12,  4.10s/it]
 37%|███▋      | 84073/225000 [158:16:05<140:10:59,  3.58s/it]
 37%|███▋      | 84074/225000 [158:16:08<132:28:52,  3.38s/it]
 37%|███▋      | 84075/225000 [158:16:11<133:21:19,  3.41s/it]
 37%|███▋      | 84076/225000 [158:16:15<134:43:01,  3.44s/it]
 37%|███▋      | 84077/225000 [158:16:18<138:13:19,  3.53s/it]
 37%|███▋      | 84078/225000 [158:16:22<142:13:40,  3.63s/it]
 37%|███▋      | 84079/225000 [158:16:25<135:09:18,  3.45s/it]
 37%|███▋      | 84080/225000 [158:16:33<191:12:25,  4.88s/it]
 37%|███▋      | 84081/225000 [158:16:40<213:50:53,  5.46s/it]
 37%|███▋      | 84082/225000 [158:16:43<179:05:35,  4.58s/it]
 37%|███▋      | 84083/225000 [158:16:45<157:22:15,  4.02s/it]
 37%|███▋      | 84084/225000 [158:16:48<142:28:32,  3.64s/it]
 37%|███▋      | 84085/225000 [158:16:51<135:55:43,  3.47s/it]
 37%|███▋      | 84086/225000 [158:16:54<127:01:23,  3.25s/it]
 37%|███▋      | 84087/225000 [158:17:03<197:42:36,  5.05s/it]
 37%|███▋      | 84088/225000 [158:17:07<181:05:44,  4.63s/it]
 37%|███▋      | 84089/225000 [158:17:11<174:30:29,  4.46s/it]
 37%|███▋      | 84090/225000 [158:17:15<169:20:14,  4.33s/it]
 37%|███▋      | 84091/225000 [158:17:21<194:49:51,  4.98s/it]
 37%|███▋      | 84092/225000 [158:17:24<166:20:21,  4.25s/it]
 37%|███▋      | 84093/225000 [158:17:27<152:11:55,  3.89s/it]
 37%|███▋      | 84094/225000 [158:17:33<177:34:12,  4.54s/it]
 37%|███▋      | 84095/225000 [158:17:37<164:23:49,  4.20s/it]
 37%|███▋      | 84096/225000 [158:17:40<154:22:56,  3.94s/it]
 37%|███▋      | 84097/225000 [158:17:44<155:34:17,  3.97s/it]
 37%|███▋      | 84098/225000 [158:17:47<150:14:47,  3.84s/it]
 37%|███▋      | 84099/225000 [158:17:53<167:55:55,  4.29s/it]
 37%|███▋      | 84100/225000 [158:17:57<164:05:56,  4.19s/it]
                                                              
{'loss': '0.5771', 'grad_norm': '4.925', 'learning_rate': '3.674e-05', 'epoch': '0.3738', 'num_input_tokens_seen': 259991592, 'train_runtime': '5.699e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84100/225000 [158:17:57<164:05:56,  4.19s/it]
 37%|███▋      | 84101/225000 [158:18:01<161:30:33,  4.13s/it]
 37%|███▋      | 84102/225000 [158:18:05<157:42:04,  4.03s/it]
 37%|███▋      | 84103/225000 [158:18:12<192:09:10,  4.91s/it]
 37%|███▋      | 84104/225000 [158:18:15<176:22:59,  4.51s/it]
 37%|███▋      | 84105/225000 [158:18:19<169:08:41,  4.32s/it]
 37%|███▋      | 84106/225000 [158:18:23<160:32:48,  4.10s/it]
 37%|███▋      | 84107/225000 [158:18:26<150:04:35,  3.83s/it]
 37%|███▋      | 84108/225000 [158:18:31<166:01:52,  4.24s/it]
 37%|███▋      | 84109/225000 [158:18:35<160:18:59,  4.10s/it]
 37%|███▋      | 84110/225000 [158:18:38<151:34:30,  3.87s/it]
 37%|███▋      | 84111/225000 [158:18:41<144:50:46,  3.70s/it]
 37%|███▋      | 84112/225000 [158:18:47<172:04:52,  4.40s/it]
 37%|███▋      | 84113/225000 [158:18:51<159:30:32,  4.08s/it]
 37%|███▋      | 84114/225000 [158:18:54<150:09:05,  3.84s/it]
 37%|███▋      | 84115/225000 [158:18:58<147:06:13,  3.76s/it]
 37%|███▋      | 84116/225000 [158:19:01<148:09:59,  3.79s/it]
 37%|███▋      | 84117/225000 [158:19:05<142:29:25,  3.64s/it]
 37%|███▋      | 84118/225000 [158:19:08<136:47:45,  3.50s/it]
 37%|███▋      | 84119/225000 [158:19:11<136:59:14,  3.50s/it]
 37%|███▋      | 84120/225000 [158:19:19<180:50:34,  4.62s/it]
 37%|███▋      | 84121/225000 [158:19:21<154:48:15,  3.96s/it]
 37%|███▋      | 84122/225000 [158:19:23<136:38:48,  3.49s/it]
 37%|███▋      | 84123/225000 [158:19:26<125:15:33,  3.20s/it]
 37%|███▋      | 84124/225000 [158:19:28<117:09:27,  2.99s/it]
 37%|███▋      | 84125/225000 [158:19:32<119:44:22,  3.06s/it]
 37%|███▋      | 84126/225000 [158:19:39<163:53:43,  4.19s/it]
 37%|███▋      | 84127/225000 [158:19:45<190:39:13,  4.87s/it]
 37%|███▋      | 84128/225000 [158:19:48<170:46:03,  4.36s/it]
 37%|███▋      | 84129/225000 [158:19:52<162:28:34,  4.15s/it]
 37%|███▋      | 84130/225000 [158:19:55<147:19:17,  3.76s/it]
 37%|███▋      | 84131/225000 [158:19:58<139:41:37,  3.57s/it]
 37%|███▋      | 84132/225000 [158:20:01<131:26:16,  3.36s/it]
 37%|███▋      | 84133/225000 [158:20:05<145:45:06,  3.72s/it]
 37%|███▋      | 84134/225000 [158:20:13<198:07:15,  5.06s/it]
 37%|███▋      | 84135/225000 [158:20:17<179:10:42,  4.58s/it]
 37%|███▋      | 84136/225000 [158:20:20<163:57:10,  4.19s/it]
 37%|███▋      | 84137/225000 [158:20:24<162:06:42,  4.14s/it]
 37%|███▋      | 84138/225000 [158:20:27<147:53:58,  3.78s/it]
 37%|███▋      | 84139/225000 [158:20:31<151:21:37,  3.87s/it]
 37%|███▋      | 84140/225000 [158:20:34<141:47:13,  3.62s/it]
 37%|███▋      | 84141/225000 [158:20:38<137:57:54,  3.53s/it]
 37%|███▋      | 84142/225000 [158:20:41<135:49:00,  3.47s/it]
 37%|███▋      | 84143/225000 [158:20:46<153:19:06,  3.92s/it]
 37%|███▋      | 84144/225000 [158:20:54<206:55:02,  5.29s/it]
 37%|███▋      | 84145/225000 [158:20:58<189:35:49,  4.85s/it]
 37%|███▋      | 84146/225000 [158:21:01<162:32:58,  4.15s/it]
 37%|███▋      | 84147/225000 [158:21:04<147:10:04,  3.76s/it]
 37%|███▋      | 84148/225000 [158:21:07<143:21:09,  3.66s/it]
 37%|███▋      | 84149/225000 [158:21:11<149:36:33,  3.82s/it]
 37%|███▋      | 84150/225000 [158:21:14<142:08:53,  3.63s/it]
                                                              
{'loss': '0.6079', 'grad_norm': '6.494', 'learning_rate': '3.672e-05', 'epoch': '0.374', 'num_input_tokens_seen': 260077200, 'train_runtime': '5.701e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84150/225000 [158:21:14<142:08:53,  3.63s/it]
 37%|███▋      | 84151/225000 [158:21:18<139:43:48,  3.57s/it]
 37%|███▋      | 84152/225000 [158:21:22<143:15:37,  3.66s/it]
 37%|███▋      | 84153/225000 [158:21:25<143:24:54,  3.67s/it]
 37%|███▋      | 84154/225000 [158:21:29<138:51:48,  3.55s/it]
 37%|███▋      | 84155/225000 [158:21:32<134:59:01,  3.45s/it]
 37%|███▋      | 84156/225000 [158:21:35<133:40:20,  3.42s/it]
 37%|███▋      | 84157/225000 [158:21:39<137:23:33,  3.51s/it]
 37%|███▋      | 84158/225000 [158:21:47<195:10:13,  4.99s/it]
 37%|███▋      | 84159/225000 [158:21:55<232:12:02,  5.94s/it]
 37%|███▋      | 84160/225000 [158:22:00<210:18:32,  5.38s/it]
 37%|███▋      | 84161/225000 [158:22:02<179:25:41,  4.59s/it]
 37%|███▋      | 84162/225000 [158:22:05<155:53:45,  3.98s/it]
 37%|███▋      | 84163/225000 [158:22:07<138:38:05,  3.54s/it]
 37%|███▋      | 84164/225000 [158:22:15<188:04:08,  4.81s/it]
 37%|███▋      | 84165/225000 [158:22:18<165:29:01,  4.23s/it]
 37%|███▋      | 84166/225000 [158:22:21<153:22:24,  3.92s/it]
 37%|███▋      | 84167/225000 [158:22:25<149:36:15,  3.82s/it]
 37%|███▋      | 84168/225000 [158:22:29<152:13:49,  3.89s/it]
 37%|███▋      | 84169/225000 [158:22:32<145:49:38,  3.73s/it]
 37%|███▋      | 84170/225000 [158:22:37<160:37:57,  4.11s/it]
 37%|███▋      | 84171/225000 [158:22:46<218:44:36,  5.59s/it]
 37%|███▋      | 84172/225000 [158:22:50<199:08:20,  5.09s/it]
 37%|███▋      | 84173/225000 [158:22:53<170:46:30,  4.37s/it]
 37%|███▋      | 84174/225000 [158:22:58<177:11:48,  4.53s/it]
 37%|███▋      | 84175/225000 [158:23:03<180:08:30,  4.61s/it]
 37%|███▋      | 84176/225000 [158:23:06<163:43:32,  4.19s/it]
 37%|███▋      | 84177/225000 [158:23:09<150:16:45,  3.84s/it]
 37%|███▋      | 84178/225000 [158:23:12<142:48:58,  3.65s/it]
 37%|███▋      | 84179/225000 [158:23:15<138:04:36,  3.53s/it]
 37%|███▋      | 84180/225000 [158:23:23<184:04:39,  4.71s/it]
 37%|███▋      | 84181/225000 [158:23:27<183:52:24,  4.70s/it]
 37%|███▋      | 84182/225000 [158:23:31<165:08:11,  4.22s/it]
 37%|███▋      | 84183/225000 [158:23:34<151:20:11,  3.87s/it]
 37%|███▋      | 84184/225000 [158:23:36<138:45:49,  3.55s/it]
 37%|███▋      | 84185/225000 [158:23:39<130:27:55,  3.34s/it]
 37%|███▋      | 84186/225000 [158:23:42<128:42:35,  3.29s/it]
 37%|███▋      | 84187/225000 [158:23:46<128:41:10,  3.29s/it]
 37%|███▋      | 84188/225000 [158:23:53<181:54:44,  4.65s/it]
 37%|███▋      | 84189/225000 [158:23:58<179:44:28,  4.60s/it]
 37%|███▋      | 84190/225000 [158:24:00<155:19:39,  3.97s/it]
 37%|███▋      | 84191/225000 [158:24:03<142:49:09,  3.65s/it]
 37%|███▋      | 84192/225000 [158:24:06<131:32:00,  3.36s/it]
 37%|███▋      | 84193/225000 [158:24:14<185:31:42,  4.74s/it]
 37%|███▋      | 84194/225000 [158:24:18<170:43:02,  4.36s/it]
 37%|███▋      | 84195/225000 [158:24:20<152:02:35,  3.89s/it]
 37%|███▋      | 84196/225000 [158:24:23<143:45:11,  3.68s/it]
 37%|███▋      | 84197/225000 [158:24:26<132:39:39,  3.39s/it]
 37%|███▋      | 84198/225000 [158:24:29<129:34:13,  3.31s/it]
 37%|███▋      | 84199/225000 [158:24:33<129:31:05,  3.31s/it]
 37%|███▋      | 84200/225000 [158:24:38<157:20:34,  4.02s/it]
                                                              
{'loss': '0.575', 'grad_norm': '3.723', 'learning_rate': '3.67e-05', 'epoch': '0.3742', 'num_input_tokens_seen': 260163888, 'train_runtime': '5.703e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84200/225000 [158:24:38<157:20:34,  4.02s/it]
 37%|███▋      | 84201/225000 [158:24:47<209:40:38,  5.36s/it]
 37%|███▋      | 84202/225000 [158:24:49<173:38:52,  4.44s/it]
 37%|███▋      | 84203/225000 [158:24:52<150:09:29,  3.84s/it]
 37%|███▋      | 84204/225000 [158:24:55<149:43:47,  3.83s/it]
 37%|███▋      | 84205/225000 [158:25:02<188:42:29,  4.83s/it]
 37%|███▋      | 84206/225000 [158:25:05<166:29:01,  4.26s/it]
 37%|███▋      | 84207/225000 [158:25:08<150:26:42,  3.85s/it]
 37%|███▋      | 84208/225000 [158:25:11<136:53:50,  3.50s/it]
 37%|███▋      | 84209/225000 [158:25:14<132:46:55,  3.40s/it]
 37%|███▋      | 84210/225000 [158:25:17<128:41:42,  3.29s/it]
 37%|███▋      | 84211/225000 [158:25:20<127:01:32,  3.25s/it]
 37%|███▋      | 84212/225000 [158:25:24<127:27:36,  3.26s/it]
 37%|███▋      | 84213/225000 [158:25:27<126:03:52,  3.22s/it]
 37%|███▋      | 84214/225000 [158:25:35<186:10:24,  4.76s/it]
 37%|███▋      | 84215/225000 [158:25:41<200:37:08,  5.13s/it]
 37%|███▋      | 84216/225000 [158:25:44<173:15:00,  4.43s/it]
 37%|███▋      | 84217/225000 [158:25:46<150:36:43,  3.85s/it]
 37%|███▋      | 84218/225000 [158:25:49<139:06:29,  3.56s/it]
 37%|███▋      | 84219/225000 [158:25:57<186:37:25,  4.77s/it]
 37%|███▋      | 84220/225000 [158:26:01<180:39:37,  4.62s/it]
 37%|███▋      | 84221/225000 [158:26:04<161:23:52,  4.13s/it]
 37%|███▋      | 84222/225000 [158:26:07<147:41:57,  3.78s/it]
 37%|███▋      | 84223/225000 [158:26:10<136:57:43,  3.50s/it]
 37%|███▋      | 84224/225000 [158:26:13<129:36:51,  3.31s/it]
 37%|███▋      | 84225/225000 [158:26:16<128:30:23,  3.29s/it]
 37%|███▋      | 84226/225000 [158:26:21<147:32:02,  3.77s/it]
 37%|███▋      | 84227/225000 [158:26:28<187:59:00,  4.81s/it]
 37%|███▋      | 84228/225000 [158:26:32<171:38:20,  4.39s/it]
 37%|███▋      | 84229/225000 [158:26:35<157:32:42,  4.03s/it]
 37%|███▋      | 84230/225000 [158:26:38<147:35:33,  3.77s/it]
 37%|███▋      | 84231/225000 [158:26:42<155:32:58,  3.98s/it]
 37%|███▋      | 84232/225000 [158:26:48<170:42:35,  4.37s/it]
 37%|███▋      | 84233/225000 [158:26:51<159:52:56,  4.09s/it]
 37%|███▋      | 84234/225000 [158:26:54<149:20:21,  3.82s/it]
 37%|███▋      | 84235/225000 [158:26:57<141:40:53,  3.62s/it]
 37%|███▋      | 84236/225000 [158:27:01<136:46:23,  3.50s/it]
 37%|███▋      | 84237/225000 [158:27:04<136:29:04,  3.49s/it]
 37%|███▋      | 84238/225000 [158:27:11<173:57:32,  4.45s/it]
 37%|███▋      | 84239/225000 [158:27:14<159:19:02,  4.07s/it]
 37%|███▋      | 84240/225000 [158:27:17<148:24:38,  3.80s/it]
 37%|███▋      | 84241/225000 [158:27:20<140:49:21,  3.60s/it]
 37%|███▋      | 84242/225000 [158:27:24<142:16:14,  3.64s/it]
 37%|███▋      | 84243/225000 [158:27:27<139:53:17,  3.58s/it]
 37%|███▋      | 84244/225000 [158:27:31<139:28:03,  3.57s/it]
 37%|███▋      | 84245/225000 [158:27:35<140:43:06,  3.60s/it]
 37%|███▋      | 84246/225000 [158:27:38<142:45:55,  3.65s/it]
 37%|███▋      | 84247/225000 [158:27:42<142:09:13,  3.64s/it]
 37%|███▋      | 84248/225000 [158:27:47<161:49:20,  4.14s/it]
 37%|███▋      | 84249/225000 [158:27:51<159:41:45,  4.08s/it]
 37%|███▋      | 84250/225000 [158:27:55<155:37:45,  3.98s/it]
                                                              
{'loss': '0.6091', 'grad_norm': '4.743', 'learning_rate': '3.669e-05', 'epoch': '0.3744', 'num_input_tokens_seen': 260250640, 'train_runtime': '5.705e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84250/225000 [158:27:55<155:37:45,  3.98s/it]
 37%|███▋      | 84251/225000 [158:28:00<161:42:21,  4.14s/it]
 37%|███▋      | 84252/225000 [158:28:03<150:55:05,  3.86s/it]
 37%|███▋      | 84253/225000 [158:28:08<161:02:16,  4.12s/it]
 37%|███▋      | 84254/225000 [158:28:17<225:03:08,  5.76s/it]
 37%|███▋      | 84255/225000 [158:28:21<203:16:18,  5.20s/it]
 37%|███▋      | 84256/225000 [158:28:24<176:18:43,  4.51s/it]
 37%|███▋      | 84257/225000 [158:28:27<161:36:10,  4.13s/it]
 37%|███▋      | 84258/225000 [158:28:31<154:58:27,  3.96s/it]
 37%|███▋      | 84259/225000 [158:28:34<149:37:58,  3.83s/it]
 37%|███▋      | 84260/225000 [158:28:38<146:12:51,  3.74s/it]
 37%|███▋      | 84261/225000 [158:28:41<140:27:54,  3.59s/it]
 37%|███▋      | 84262/225000 [158:28:47<167:58:57,  4.30s/it]
 37%|███▋      | 84263/225000 [158:28:55<216:21:57,  5.53s/it]
 37%|███▋      | 84264/225000 [158:28:59<196:25:54,  5.02s/it]
 37%|███▋      | 84265/225000 [158:29:02<164:26:06,  4.21s/it]
 37%|███▋      | 84266/225000 [158:29:04<145:54:18,  3.73s/it]
 37%|███▋      | 84267/225000 [158:29:08<142:01:33,  3.63s/it]
 37%|███▋      | 84268/225000 [158:29:11<142:24:48,  3.64s/it]
 37%|███▋      | 84269/225000 [158:29:17<163:56:19,  4.19s/it]
 37%|███▋      | 84270/225000 [158:29:23<190:10:26,  4.86s/it]
 37%|███▋      | 84271/225000 [158:29:26<165:44:59,  4.24s/it]
 37%|███▋      | 84272/225000 [158:29:34<209:01:25,  5.35s/it]
 37%|███▋      | 84273/225000 [158:29:36<172:15:42,  4.41s/it]
 37%|███▋      | 84274/225000 [158:29:41<176:40:40,  4.52s/it]
 37%|███▋      | 84275/225000 [158:29:44<155:07:04,  3.97s/it]
 37%|███▋      | 84276/225000 [158:29:47<143:58:44,  3.68s/it]
 37%|███▋      | 84277/225000 [158:29:50<137:42:21,  3.52s/it]
 37%|███▋      | 84278/225000 [158:29:53<132:45:58,  3.40s/it]
 37%|███▋      | 84279/225000 [158:29:56<127:26:12,  3.26s/it]
 37%|███▋      | 84280/225000 [158:29:59<123:00:05,  3.15s/it]
 37%|███▋      | 84281/225000 [158:30:02<120:52:24,  3.09s/it]
 37%|███▋      | 84282/225000 [158:30:08<164:14:08,  4.20s/it]
 37%|███▋      | 84283/225000 [158:30:16<199:22:15,  5.10s/it]
 37%|███▋      | 84284/225000 [158:30:18<171:48:16,  4.40s/it]
 37%|███▋      | 84285/225000 [158:30:22<158:18:03,  4.05s/it]
 37%|███▋      | 84286/225000 [158:30:24<143:31:52,  3.67s/it]
 37%|███▋      | 84287/225000 [158:30:27<133:14:54,  3.41s/it]
 37%|███▋      | 84288/225000 [158:30:30<130:40:24,  3.34s/it]
 37%|███▋      | 84289/225000 [158:30:33<126:03:10,  3.22s/it]
 37%|███▋      | 84290/225000 [158:30:36<124:35:46,  3.19s/it]
 37%|███▋      | 84291/225000 [158:30:44<171:17:27,  4.38s/it]
 37%|███▋      | 84292/225000 [158:30:52<216:00:00,  5.53s/it]
 37%|███▋      | 84293/225000 [158:30:54<179:41:55,  4.60s/it]
 37%|███▋      | 84294/225000 [158:30:56<152:26:00,  3.90s/it]
 37%|███▋      | 84295/225000 [158:30:59<134:49:25,  3.45s/it]
 37%|███▋      | 84296/225000 [158:31:02<129:56:24,  3.32s/it]
 37%|███▋      | 84297/225000 [158:31:11<195:21:24,  5.00s/it]
 37%|███▋      | 84298/225000 [158:31:14<171:18:15,  4.38s/it]
 37%|███▋      | 84299/225000 [158:31:16<150:02:54,  3.84s/it]
 37%|███▋      | 84300/225000 [158:31:19<132:56:52,  3.40s/it]
                                                              
{'loss': '0.6008', 'grad_norm': '5.442', 'learning_rate': '3.667e-05', 'epoch': '0.3747', 'num_input_tokens_seen': 260335784, 'train_runtime': '5.707e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84300/225000 [158:31:19<132:56:52,  3.40s/it]
 37%|███▋      | 84301/225000 [158:31:22<132:22:15,  3.39s/it]
 37%|███▋      | 84302/225000 [158:31:30<184:49:24,  4.73s/it]
 37%|███▋      | 84303/225000 [158:31:33<167:18:16,  4.28s/it]
 37%|███▋      | 84304/225000 [158:31:36<148:02:57,  3.79s/it]
 37%|███▋      | 84305/225000 [158:31:38<134:53:54,  3.45s/it]
 37%|███▋      | 84306/225000 [158:31:41<124:33:38,  3.19s/it]
 37%|███▋      | 84307/225000 [158:31:44<116:47:02,  2.99s/it]
 37%|███▋      | 84308/225000 [158:31:53<193:10:26,  4.94s/it]
 37%|███▋      | 84309/225000 [158:31:58<197:03:26,  5.04s/it]
 37%|███▋      | 84310/225000 [158:32:01<172:00:46,  4.40s/it]
 37%|███▋      | 84311/225000 [158:32:04<150:21:11,  3.85s/it]
 37%|███▋      | 84312/225000 [158:32:07<137:35:41,  3.52s/it]
 37%|███▋      | 84313/225000 [158:32:10<131:21:25,  3.36s/it]
 37%|███▋      | 84314/225000 [158:32:17<177:23:47,  4.54s/it]
 37%|███▋      | 84315/225000 [158:32:22<181:13:18,  4.64s/it]
 37%|███▋      | 84316/225000 [158:32:24<159:54:07,  4.09s/it]
 37%|███▋      | 84317/225000 [158:32:27<144:46:52,  3.70s/it]
 37%|███▋      | 84318/225000 [158:32:35<189:57:58,  4.86s/it]
 37%|███▋      | 84319/225000 [158:32:37<161:07:51,  4.12s/it]
 37%|███▋      | 84320/225000 [158:32:40<144:36:59,  3.70s/it]
 37%|███▋      | 84321/225000 [158:32:45<158:12:21,  4.05s/it]
 37%|███▋      | 84322/225000 [158:32:52<195:39:05,  5.01s/it]
 37%|███▋      | 84323/225000 [158:32:55<167:42:53,  4.29s/it]
 37%|███▋      | 84324/225000 [158:32:57<148:42:40,  3.81s/it]
 37%|███▋      | 84325/225000 [158:33:00<136:39:52,  3.50s/it]
 37%|███▋      | 84326/225000 [158:33:04<140:22:20,  3.59s/it]
 37%|███▋      | 84327/225000 [158:33:13<202:57:07,  5.19s/it]
 37%|███▋      | 84328/225000 [158:33:17<191:06:10,  4.89s/it]
 37%|███▋      | 84329/225000 [158:33:20<163:20:54,  4.18s/it]
 37%|███▋      | 84330/225000 [158:33:22<144:06:18,  3.69s/it]
 37%|███▋      | 84331/225000 [158:33:26<150:14:30,  3.84s/it]
 37%|███▋      | 84332/225000 [158:33:34<190:48:23,  4.88s/it]
 37%|███▋      | 84333/225000 [158:33:36<162:24:58,  4.16s/it]
 37%|███▋      | 84334/225000 [158:33:38<141:23:26,  3.62s/it]
 37%|███▋      | 84335/225000 [158:33:42<141:58:02,  3.63s/it]
 37%|███▋      | 84336/225000 [158:33:49<178:01:16,  4.56s/it]
 37%|███▋      | 84337/225000 [158:33:51<152:56:02,  3.91s/it]
 37%|███▋      | 84338/225000 [158:33:56<163:39:56,  4.19s/it]
 37%|███▋      | 84339/225000 [158:33:59<143:04:37,  3.66s/it]
 37%|███▋      | 84340/225000 [158:34:01<133:55:28,  3.43s/it]
 37%|███▋      | 84341/225000 [158:34:04<124:53:04,  3.20s/it]
 37%|███▋      | 84342/225000 [158:34:10<158:04:38,  4.05s/it]
 37%|███▋      | 84343/225000 [158:34:17<187:09:30,  4.79s/it]
 37%|███▋      | 84344/225000 [158:34:19<164:14:29,  4.20s/it]
 37%|███▋      | 84345/225000 [158:34:22<145:31:40,  3.72s/it]
 37%|███▋      | 84346/225000 [158:34:25<136:05:05,  3.48s/it]
 37%|███▋      | 84347/225000 [158:34:28<129:51:27,  3.32s/it]
 37%|███▋      | 84348/225000 [158:34:37<192:18:37,  4.92s/it]
 37%|███▋      | 84349/225000 [158:34:40<169:06:22,  4.33s/it]
 37%|███▋      | 84350/225000 [158:34:43<155:25:45,  3.98s/it]
                                                              
{'loss': '0.5772', 'grad_norm': '5.748', 'learning_rate': '3.665e-05', 'epoch': '0.3749', 'num_input_tokens_seen': 260422312, 'train_runtime': '5.709e+05', 'train_tokens_per_second': '456.2'}

 37%|███▋      | 84350/225000 [158:34:43<155:25:45,  3.98s/it]
 37%|███▋      | 84351/225000 [158:34:46<148:37:13,  3.80s/it]
 37%|███▋      | 84352/225000 [158:34:49<136:20:11,  3.49s/it]
 37%|███▋      | 84353/225000 [158:34:52<127:07:35,  3.25s/it]
 37%|███▋      | 84354/225000 [158:34:55<124:02:10,  3.17s/it]
 37%|███▋      | 84355/225000 [158:35:03<182:12:08,  4.66s/it]
 37%|███▋      | 84356/225000 [158:35:06<166:32:11,  4.26s/it]
 37%|███▋      | 84357/225000 [158:35:09<152:45:23,  3.91s/it]
 37%|███▋      | 84358/225000 [158:35:12<143:56:49,  3.68s/it]
 37%|███▋      | 84359/225000 [158:35:16<141:22:23,  3.62s/it]
 37%|███▋      | 84360/225000 [158:35:19<133:15:38,  3.41s/it]
 37%|███▋      | 84361/225000 [158:35:22<128:17:19,  3.28s/it]
 37%|███▋      | 84362/225000 [158:35:26<145:50:54,  3.73s/it]
 37%|███▋      | 84363/225000 [158:35:35<202:40:28,  5.19s/it]
 37%|███▋      | 84364/225000 [158:35:37<170:31:09,  4.36s/it]
 37%|███▋      | 84365/225000 [158:35:40<147:13:35,  3.77s/it]
 37%|███▋      | 84366/225000 [158:35:43<137:12:13,  3.51s/it]
 37%|███▋      | 84367/225000 [158:35:49<171:59:55,  4.40s/it]
 37%|███▋      | 84368/225000 [158:35:53<169:44:21,  4.35s/it]
 37%|███▋      | 84369/225000 [158:35:57<155:57:22,  3.99s/it]
 37%|███▋      | 84370/225000 [158:36:00<150:05:27,  3.84s/it]
 37%|███▋      | 84371/225000 [158:36:04<145:46:04,  3.73s/it]
 37%|███▋      | 84372/225000 [158:36:07<142:02:34,  3.64s/it]
 37%|███▋      | 84373/225000 [158:36:10<139:27:02,  3.57s/it]
 37%|███▋      | 84374/225000 [158:36:14<139:12:35,  3.56s/it]
 38%|███▊      | 84375/225000 [158:36:17<137:38:43,  3.52s/it]
 38%|███▊      | 84376/225000 [158:36:21<133:48:04,  3.43s/it]
 38%|███▊      | 84377/225000 [158:36:24<130:09:20,  3.33s/it]
 38%|███▊      | 84378/225000 [158:36:27<129:11:05,  3.31s/it]
 38%|███▊      | 84379/225000 [158:36:34<167:46:55,  4.30s/it]
 38%|███▊      | 84380/225000 [158:36:41<207:02:45,  5.30s/it]
 38%|███▊      | 84381/225000 [158:36:44<174:22:30,  4.46s/it]
 38%|███▊      | 84382/225000 [158:36:47<156:20:05,  4.00s/it]
 38%|███▊      | 84383/225000 [158:36:54<194:12:03,  4.97s/it]
 38%|███▊      | 84384/225000 [158:36:57<173:18:04,  4.44s/it]
 38%|███▊      | 84385/225000 [158:36:59<149:25:19,  3.83s/it]
 38%|███▊      | 84386/225000 [158:37:05<172:43:55,  4.42s/it]
 38%|███▊      | 84387/225000 [158:37:09<167:40:10,  4.29s/it]
 38%|███▊      | 84388/225000 [158:37:12<147:18:42,  3.77s/it]
 38%|███▊      | 84389/225000 [158:37:15<144:06:38,  3.69s/it]
 38%|███▊      | 84390/225000 [158:37:22<181:39:59,  4.65s/it]
 38%|███▊      | 84391/225000 [158:37:25<162:41:46,  4.17s/it]
 38%|███▊      | 84392/225000 [158:37:29<154:02:29,  3.94s/it]
 38%|███▊      | 84393/225000 [158:37:31<139:34:19,  3.57s/it]
 38%|███▊      | 84394/225000 [158:37:34<128:22:39,  3.29s/it]
 38%|███▊      | 84395/225000 [158:37:37<129:31:03,  3.32s/it]
 38%|███▊      | 84396/225000 [158:37:45<186:04:35,  4.76s/it]
 38%|███▊      | 84397/225000 [158:37:51<193:50:54,  4.96s/it]
 38%|███▊      | 84398/225000 [158:37:53<162:18:54,  4.16s/it]
 38%|███▊      | 84399/225000 [158:38:00<187:55:29,  4.81s/it]
 38%|███▊      | 84400/225000 [158:38:03<169:45:53,  4.35s/it]
                                                              
{'loss': '0.5722', 'grad_norm': '4.712', 'learning_rate': '3.664e-05', 'epoch': '0.3751', 'num_input_tokens_seen': 260507976, 'train_runtime': '5.711e+05', 'train_tokens_per_second': '456.2'}

 38%|███▊      | 84400/225000 [158:38:03<169:45:53,  4.35s/it]
 38%|███▊      | 84401/225000 [158:38:06<156:16:53,  4.00s/it]
 38%|███▊      | 84402/225000 [158:38:09<138:51:41,  3.56s/it]
 38%|███▊      | 84403/225000 [158:38:11<128:33:21,  3.29s/it]
 38%|███▊      | 84404/225000 [158:38:14<121:45:39,  3.12s/it]
 38%|███▊      | 84405/225000 [158:38:19<139:44:35,  3.58s/it]
 38%|███▊      | 84406/225000 [158:38:27<196:38:09,  5.03s/it]
 38%|███▊      | 84407/225000 [158:38:30<167:54:54,  4.30s/it]
 38%|███▊      | 84408/225000 [158:38:32<145:25:31,  3.72s/it]
 38%|███▊      | 84409/225000 [158:38:36<147:16:00,  3.77s/it]
 38%|███▊      | 84410/225000 [158:38:43<192:15:56,  4.92s/it]
 38%|███▊      | 84411/225000 [158:38:46<165:55:20,  4.25s/it]
 38%|███▊      | 84412/225000 [158:38:49<145:37:46,  3.73s/it]
 38%|███▊      | 84413/225000 [158:38:56<187:29:21,  4.80s/it]
 38%|███▊      | 84414/225000 [158:39:00<180:32:48,  4.62s/it]
 38%|███▊      | 84415/225000 [158:39:04<166:11:42,  4.26s/it]
 38%|███▊      | 84416/225000 [158:39:06<150:41:19,  3.86s/it]
 38%|███▊      | 84417/225000 [158:39:09<138:07:32,  3.54s/it]
 38%|███▊      | 84418/225000 [158:39:12<133:59:11,  3.43s/it]
 38%|███▊      | 84419/225000 [158:39:16<132:10:52,  3.38s/it]
 38%|███▊      | 84420/225000 [158:39:20<141:26:58,  3.62s/it]
 38%|███▊      | 84421/225000 [158:39:29<205:26:00,  5.26s/it]
 38%|███▊      | 84422/225000 [158:39:34<199:03:25,  5.10s/it]
 38%|███▊      | 84423/225000 [158:39:36<169:39:35,  4.34s/it]
 38%|███▊      | 84424/225000 [158:39:39<148:31:43,  3.80s/it]
 38%|███▊      | 84425/225000 [158:39:43<158:15:42,  4.05s/it]
 38%|███▊      | 84426/225000 [158:39:51<201:00:24,  5.15s/it]
 38%|███▊      | 84427/225000 [158:39:53<167:09:09,  4.28s/it]
 38%|███▊      | 84428/225000 [158:39:56<147:11:31,  3.77s/it]
 38%|███▊      | 84429/225000 [158:40:02<178:26:35,  4.57s/it]
 38%|███▊      | 84430/225000 [158:40:06<164:19:27,  4.21s/it]
 38%|███▊      | 84431/225000 [158:40:11<175:10:59,  4.49s/it]
 38%|███▊      | 84432/225000 [158:40:14<160:07:36,  4.10s/it]
 38%|███▊      | 84433/225000 [158:40:18<152:56:52,  3.92s/it]
 38%|███▊      | 84434/225000 [158:40:20<140:43:22,  3.60s/it]
 38%|███▊      | 84435/225000 [158:40:24<141:17:43,  3.62s/it]
 38%|███▊      | 84436/225000 [158:40:28<142:05:25,  3.64s/it]
 38%|███▊      | 84437/225000 [158:40:31<138:37:53,  3.55s/it]
 38%|███▊      | 84438/225000 [158:40:35<138:24:03,  3.54s/it]
 38%|███▊      | 84439/225000 [158:40:38<132:06:50,  3.38s/it]
 38%|███▊      | 84440/225000 [158:40:42<141:01:15,  3.61s/it]
 38%|███▊      | 84441/225000 [158:40:51<204:47:45,  5.25s/it]
 38%|███▊      | 84442/225000 [158:40:57<215:11:45,  5.51s/it]
 38%|███▊      | 84443/225000 [158:41:00<185:29:26,  4.75s/it]
 38%|███▊      | 84444/225000 [158:41:03<168:16:27,  4.31s/it]
 38%|███▊      | 84445/225000 [158:41:07<155:50:11,  3.99s/it]
 38%|███▊      | 84446/225000 [158:41:10<144:02:06,  3.69s/it]
 38%|███▊      | 84447/225000 [158:41:13<138:11:18,  3.54s/it]
 38%|███▊      | 84448/225000 [158:41:16<132:51:03,  3.40s/it]
 38%|███▊      | 84449/225000 [158:41:19<131:46:22,  3.38s/it]
 38%|███▊      | 84450/225000 [158:41:22<131:14:33,  3.36s/it]
                                                              
{'loss': '0.5843', 'grad_norm': '5.573', 'learning_rate': '3.662e-05', 'epoch': '0.3753', 'num_input_tokens_seen': 260594304, 'train_runtime': '5.713e+05', 'train_tokens_per_second': '456.2'}

 38%|███▊      | 84450/225000 [158:41:22<131:14:33,  3.36s/it]
 38%|███▊      | 84451/225000 [158:41:26<130:27:16,  3.34s/it]
 38%|███▊      | 84452/225000 [158:41:29<133:32:11,  3.42s/it]
 38%|███▊      | 84453/225000 [158:41:39<201:30:32,  5.16s/it]
 38%|███▊      | 84454/225000 [158:41:44<203:32:04,  5.21s/it]
 38%|███▊      | 84455/225000 [158:41:46<170:21:21,  4.36s/it]
 38%|███▊      | 84456/225000 [158:41:49<153:07:53,  3.92s/it]
 38%|███▊      | 84457/225000 [158:41:52<140:06:26,  3.59s/it]
 38%|███▊      | 84458/225000 [158:41:55<135:09:08,  3.46s/it]
 38%|███▊      | 84459/225000 [158:41:58<132:03:54,  3.38s/it]
 38%|███▊      | 84460/225000 [158:42:07<198:06:16,  5.07s/it]
 38%|███▊      | 84461/225000 [158:42:10<173:24:11,  4.44s/it]
 38%|███▊      | 84462/225000 [158:42:13<152:18:31,  3.90s/it]
 38%|███▊      | 84463/225000 [158:42:16<140:24:10,  3.60s/it]
 38%|███▊      | 84464/225000 [158:42:19<132:00:06,  3.38s/it]
 38%|███▊      | 84465/225000 [158:42:22<133:06:55,  3.41s/it]
 38%|███▊      | 84466/225000 [158:42:30<184:03:09,  4.71s/it]
 38%|███▊      | 84467/225000 [158:42:34<177:44:55,  4.55s/it]
 38%|███▊      | 84468/225000 [158:42:37<157:57:23,  4.05s/it]
 38%|███▊      | 84469/225000 [158:42:40<147:39:37,  3.78s/it]
 38%|███▊      | 84470/225000 [158:42:44<142:45:29,  3.66s/it]
 38%|███▊      | 84471/225000 [158:42:47<139:25:20,  3.57s/it]
 38%|███▊      | 84472/225000 [158:42:50<133:52:04,  3.43s/it]
 38%|███▊      | 84473/225000 [158:42:54<135:19:59,  3.47s/it]
 38%|███▊      | 84474/225000 [158:43:02<190:51:35,  4.89s/it]
 38%|███▊      | 84475/225000 [158:43:09<215:25:44,  5.52s/it]
 38%|███▊      | 84476/225000 [158:43:12<183:17:21,  4.70s/it]
 38%|███▊      | 84477/225000 [158:43:14<160:54:50,  4.12s/it]
 38%|███▊      | 84478/225000 [158:43:18<151:55:55,  3.89s/it]
 38%|███▊      | 84479/225000 [158:43:21<139:41:28,  3.58s/it]
 38%|███▊      | 84480/225000 [158:43:24<132:40:41,  3.40s/it]
 38%|███▊      | 84481/225000 [158:43:27<130:30:53,  3.34s/it]
 38%|███▊      | 84482/225000 [158:43:35<193:48:38,  4.97s/it]
 38%|███▊      | 84483/225000 [158:43:41<195:25:46,  5.01s/it]
 38%|███▊      | 84484/225000 [158:43:43<170:27:47,  4.37s/it]
 38%|███▊      | 84485/225000 [158:43:46<147:40:41,  3.78s/it]
 38%|███▊      | 84486/225000 [158:43:49<140:07:08,  3.59s/it]
 38%|███▊      | 84487/225000 [158:43:57<195:56:22,  5.02s/it]
 38%|███▊      | 84488/225000 [158:44:00<167:10:54,  4.28s/it]
 38%|███▊      | 84489/225000 [158:44:03<151:28:07,  3.88s/it]
 38%|███▊      | 84490/225000 [158:44:06<142:58:25,  3.66s/it]
 38%|███▊      | 84491/225000 [158:44:09<136:52:32,  3.51s/it]
 38%|███▊      | 84492/225000 [158:44:13<137:35:05,  3.53s/it]
 38%|███▊      | 84493/225000 [158:44:16<134:24:07,  3.44s/it]
 38%|███▊      | 84494/225000 [158:44:19<133:01:54,  3.41s/it]
 38%|███▊      | 84495/225000 [158:44:23<131:15:36,  3.36s/it]
 38%|███▊      | 84496/225000 [158:44:26<128:52:47,  3.30s/it]
 38%|███▊      | 84497/225000 [158:44:31<152:28:10,  3.91s/it]
 38%|███▊      | 84498/225000 [158:44:39<201:34:46,  5.16s/it]
 38%|███▊      | 84499/225000 [158:44:45<205:31:05,  5.27s/it]
 38%|███▊      | 84500/225000 [158:44:47<175:22:10,  4.49s/it]
                                                              
{'loss': '0.5708', 'grad_norm': '4.814', 'learning_rate': '3.661e-05', 'epoch': '0.3756', 'num_input_tokens_seen': 260680848, 'train_runtime': '5.715e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84500/225000 [158:44:47<175:22:10,  4.49s/it]
 38%|███▊      | 84501/225000 [158:44:50<155:10:07,  3.98s/it]
 38%|███▊      | 84502/225000 [158:44:57<190:39:40,  4.89s/it]
 38%|███▊      | 84503/225000 [158:45:00<167:16:05,  4.29s/it]
 38%|███▊      | 84504/225000 [158:45:03<152:10:36,  3.90s/it]
 38%|███▊      | 84505/225000 [158:45:06<137:49:20,  3.53s/it]
 38%|███▊      | 84506/225000 [158:45:09<129:43:34,  3.32s/it]
 38%|███▊      | 84507/225000 [158:45:12<136:03:24,  3.49s/it]
 38%|███▊      | 84508/225000 [158:45:16<134:31:50,  3.45s/it]
 38%|███▊      | 84509/225000 [158:45:20<141:58:01,  3.64s/it]
 38%|███▊      | 84510/225000 [158:45:24<144:16:54,  3.70s/it]
 38%|███▊      | 84511/225000 [158:45:27<140:03:41,  3.59s/it]
 38%|███▊      | 84512/225000 [158:45:34<179:31:17,  4.60s/it]
 38%|███▊      | 84513/225000 [158:45:43<227:36:23,  5.83s/it]
 38%|███▊      | 84514/225000 [158:45:45<190:15:35,  4.88s/it]
 38%|███▊      | 84515/225000 [158:45:48<162:40:24,  4.17s/it]
 38%|███▊      | 84516/225000 [158:45:55<192:17:43,  4.93s/it]
 38%|███▊      | 84517/225000 [158:45:57<165:07:51,  4.23s/it]
 38%|███▊      | 84518/225000 [158:46:00<153:44:32,  3.94s/it]
 38%|███▊      | 84519/225000 [158:46:04<149:51:42,  3.84s/it]
 38%|███▊      | 84520/225000 [158:46:07<144:11:27,  3.70s/it]
 38%|███▊      | 84521/225000 [158:46:11<144:08:07,  3.69s/it]
 38%|███▊      | 84522/225000 [158:46:15<148:15:27,  3.80s/it]
 38%|███▊      | 84523/225000 [158:46:19<143:36:13,  3.68s/it]
 38%|███▊      | 84524/225000 [158:46:22<144:52:43,  3.71s/it]
 38%|███▊      | 84525/225000 [158:46:26<145:00:34,  3.72s/it]
 38%|███▊      | 84526/225000 [158:46:29<137:58:19,  3.54s/it]
 38%|███▊      | 84527/225000 [158:46:32<135:16:18,  3.47s/it]
 38%|███▊      | 84528/225000 [158:46:36<131:16:19,  3.36s/it]
 38%|███▊      | 84529/225000 [158:46:44<191:10:27,  4.90s/it]
 38%|███▊      | 84530/225000 [158:46:50<205:58:22,  5.28s/it]
 38%|███▊      | 84531/225000 [158:46:53<171:03:04,  4.38s/it]
 38%|███▊      | 84532/225000 [158:46:55<151:52:05,  3.89s/it]
 38%|███▊      | 84533/225000 [158:46:58<135:15:35,  3.47s/it]
 38%|███▊      | 84534/225000 [158:47:05<184:15:27,  4.72s/it]
 38%|███▊      | 84535/225000 [158:47:10<181:02:26,  4.64s/it]
 38%|███▊      | 84536/225000 [158:47:12<153:45:26,  3.94s/it]
 38%|███▊      | 84537/225000 [158:47:15<141:33:31,  3.63s/it]
 38%|███▊      | 84538/225000 [158:47:21<174:45:26,  4.48s/it]
 38%|███▊      | 84539/225000 [158:47:27<185:24:44,  4.75s/it]
 38%|███▊      | 84540/225000 [158:47:29<156:31:20,  4.01s/it]
 38%|███▊      | 84541/225000 [158:47:35<176:16:24,  4.52s/it]
 38%|███▊      | 84542/225000 [158:47:37<151:04:33,  3.87s/it]
 38%|███▊      | 84543/225000 [158:47:40<140:15:51,  3.60s/it]
 38%|███▊      | 84544/225000 [158:47:43<130:33:32,  3.35s/it]
 38%|███▊      | 84545/225000 [158:47:46<125:55:08,  3.23s/it]
 38%|███▊      | 84546/225000 [158:47:49<122:48:59,  3.15s/it]
 38%|███▊      | 84547/225000 [158:47:52<123:51:51,  3.17s/it]
 38%|███▊      | 84548/225000 [158:47:59<167:14:41,  4.29s/it]
 38%|███▊      | 84549/225000 [158:48:07<209:50:12,  5.38s/it]
 38%|███▊      | 84550/225000 [158:48:10<181:34:44,  4.65s/it]
                                                              
{'loss': '0.5505', 'grad_norm': '5.113', 'learning_rate': '3.659e-05', 'epoch': '0.3758', 'num_input_tokens_seen': 260767552, 'train_runtime': '5.717e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84550/225000 [158:48:10<181:34:44,  4.65s/it]
 38%|███▊      | 84551/225000 [158:48:13<164:42:36,  4.22s/it]
 38%|███▊      | 84552/225000 [158:48:16<150:16:37,  3.85s/it]
 38%|███▊      | 84553/225000 [158:48:19<139:13:04,  3.57s/it]
 38%|███▊      | 84554/225000 [158:48:22<132:39:43,  3.40s/it]
 38%|███▊      | 84555/225000 [158:48:25<127:40:39,  3.27s/it]
 38%|███▊      | 84556/225000 [158:48:29<130:49:27,  3.35s/it]
 38%|███▊      | 84557/225000 [158:48:32<134:17:31,  3.44s/it]
 38%|███▊      | 84558/225000 [158:48:37<153:29:41,  3.93s/it]
 38%|███▊      | 84559/225000 [158:48:42<163:05:40,  4.18s/it]
 38%|███▊      | 84560/225000 [158:48:45<151:32:38,  3.88s/it]
 38%|███▊      | 84561/225000 [158:48:49<153:37:16,  3.94s/it]
 38%|███▊      | 84562/225000 [158:48:53<148:57:18,  3.82s/it]
 38%|███▊      | 84563/225000 [158:48:56<146:50:24,  3.76s/it]
 38%|███▊      | 84564/225000 [158:49:00<145:28:16,  3.73s/it]
 38%|███▊      | 84565/225000 [158:49:04<148:58:42,  3.82s/it]
 38%|███▊      | 84566/225000 [158:49:07<141:32:19,  3.63s/it]
 38%|███▊      | 84567/225000 [158:49:11<136:53:26,  3.51s/it]
 38%|███▊      | 84568/225000 [158:49:14<140:38:41,  3.61s/it]
 38%|███▊      | 84569/225000 [158:49:23<198:08:27,  5.08s/it]
 38%|███▊      | 84570/225000 [158:49:28<200:39:43,  5.14s/it]
 38%|███▊      | 84571/225000 [158:49:31<171:01:51,  4.38s/it]
 38%|███▊      | 84572/225000 [158:49:33<150:10:17,  3.85s/it]
 38%|███▊      | 84573/225000 [158:49:39<170:20:58,  4.37s/it]
 38%|███▊      | 84574/225000 [158:49:46<199:39:37,  5.12s/it]
 38%|███▊      | 84575/225000 [158:49:48<166:49:14,  4.28s/it]
 38%|███▊      | 84576/225000 [158:49:51<146:08:44,  3.75s/it]
 38%|███▊      | 84577/225000 [158:49:58<184:52:28,  4.74s/it]
 38%|███▊      | 84578/225000 [158:50:02<181:04:44,  4.64s/it]
 38%|███▊      | 84579/225000 [158:50:06<168:06:51,  4.31s/it]
 38%|███▊      | 84580/225000 [158:50:11<183:01:22,  4.69s/it]
 38%|███▊      | 84581/225000 [158:50:14<155:45:25,  3.99s/it]
 38%|███▊      | 84582/225000 [158:50:16<138:47:04,  3.56s/it]
 38%|███▊      | 84583/225000 [158:50:20<138:57:14,  3.56s/it]
 38%|███▊      | 84584/225000 [158:50:23<138:11:25,  3.54s/it]
 38%|███▊      | 84585/225000 [158:50:27<135:34:07,  3.48s/it]
 38%|███▊      | 84586/225000 [158:50:30<132:23:29,  3.39s/it]
 38%|███▊      | 84587/225000 [158:50:35<157:20:01,  4.03s/it]
 38%|███▊      | 84588/225000 [158:50:39<157:32:01,  4.04s/it]
 38%|███▊      | 84589/225000 [158:50:43<147:53:26,  3.79s/it]
 38%|███▊      | 84590/225000 [158:50:46<147:28:41,  3.78s/it]
 38%|███▊      | 84591/225000 [158:50:50<144:32:27,  3.71s/it]
 38%|███▊      | 84592/225000 [158:50:53<138:40:58,  3.56s/it]
 38%|███▊      | 84593/225000 [158:50:56<136:32:44,  3.50s/it]
 38%|███▊      | 84594/225000 [158:51:00<133:33:21,  3.42s/it]
 38%|███▊      | 84595/225000 [158:51:03<135:04:34,  3.46s/it]
 38%|███▊      | 84596/225000 [158:51:07<134:51:34,  3.46s/it]
 38%|███▊      | 84597/225000 [158:51:14<175:03:40,  4.49s/it]
 38%|███▊      | 84598/225000 [158:51:22<221:06:43,  5.67s/it]
 38%|███▊      | 84599/225000 [158:51:25<192:05:19,  4.93s/it]
 38%|███▊      | 84600/225000 [158:51:29<176:39:13,  4.53s/it]
                                                              
{'loss': '0.5628', 'grad_norm': '3.377', 'learning_rate': '3.657e-05', 'epoch': '0.376', 'num_input_tokens_seen': 260854376, 'train_runtime': '5.719e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84600/225000 [158:51:29<176:39:13,  4.53s/it]
 38%|███▊      | 84601/225000 [158:51:32<158:23:33,  4.06s/it]
 38%|███▊      | 84602/225000 [158:51:35<145:20:39,  3.73s/it]
 38%|███▊      | 84603/225000 [158:51:38<137:27:06,  3.52s/it]
 38%|███▊      | 84604/225000 [158:51:41<130:06:35,  3.34s/it]
 38%|███▊      | 84605/225000 [158:51:44<132:07:49,  3.39s/it]
 38%|███▊      | 84606/225000 [158:51:47<130:01:56,  3.33s/it]
 38%|███▊      | 84607/225000 [158:51:55<180:18:37,  4.62s/it]
 38%|███▊      | 84608/225000 [158:52:04<230:31:44,  5.91s/it]
 38%|███▊      | 84609/225000 [158:52:06<191:11:30,  4.90s/it]
 38%|███▊      | 84610/225000 [158:52:09<164:33:52,  4.22s/it]
 38%|███▊      | 84611/225000 [158:52:12<145:52:43,  3.74s/it]
 38%|███▊      | 84612/225000 [158:52:15<137:09:12,  3.52s/it]
 38%|███▊      | 84613/225000 [158:52:17<127:32:19,  3.27s/it]
 38%|███▊      | 84614/225000 [158:52:21<136:24:48,  3.50s/it]
 38%|███▊      | 84615/225000 [158:52:30<194:45:35,  4.99s/it]
 38%|███▊      | 84616/225000 [158:52:33<174:48:44,  4.48s/it]
 38%|███▊      | 84617/225000 [158:52:36<151:53:04,  3.89s/it]
 38%|███▊      | 84618/225000 [158:52:38<135:13:13,  3.47s/it]
 38%|███▊      | 84619/225000 [158:52:41<126:02:06,  3.23s/it]
 38%|███▊      | 84620/225000 [158:52:43<118:11:36,  3.03s/it]
 38%|███▊      | 84621/225000 [158:52:51<173:56:13,  4.46s/it]
 38%|███▊      | 84622/225000 [158:52:55<171:07:11,  4.39s/it]
 38%|███▊      | 84623/225000 [158:52:59<157:29:45,  4.04s/it]
 38%|███▊      | 84624/225000 [158:53:05<188:30:45,  4.83s/it]
 38%|███▊      | 84625/225000 [158:53:09<175:00:21,  4.49s/it]
 38%|███▊      | 84626/225000 [158:53:16<206:18:37,  5.29s/it]
 38%|███▊      | 84627/225000 [158:53:18<171:26:33,  4.40s/it]
 38%|███▊      | 84628/225000 [158:53:21<146:34:39,  3.76s/it]
 38%|███▊      | 84629/225000 [158:53:24<141:32:56,  3.63s/it]
 38%|███▊      | 84630/225000 [158:53:30<171:18:29,  4.39s/it]
 38%|███▊      | 84631/225000 [158:53:33<146:13:59,  3.75s/it]
 38%|███▊      | 84632/225000 [158:53:40<186:12:14,  4.78s/it]
 38%|███▊      | 84633/225000 [158:53:42<157:51:03,  4.05s/it]
 38%|███▊      | 84634/225000 [158:53:45<141:17:24,  3.62s/it]
 38%|███▊      | 84635/225000 [158:53:48<134:17:34,  3.44s/it]
 38%|███▊      | 84636/225000 [158:53:52<142:54:34,  3.67s/it]
 38%|███▊      | 84637/225000 [158:53:58<173:57:55,  4.46s/it]
 38%|███▊      | 84638/225000 [158:54:01<154:23:22,  3.96s/it]
 38%|███▊      | 84639/225000 [158:54:04<145:35:27,  3.73s/it]
 38%|███▊      | 84640/225000 [158:54:07<134:27:44,  3.45s/it]
 38%|███▊      | 84641/225000 [158:54:11<137:26:46,  3.53s/it]
 38%|███▊      | 84642/225000 [158:54:14<133:40:59,  3.43s/it]
 38%|███▊      | 84643/225000 [158:54:22<191:56:16,  4.92s/it]
 38%|███▊      | 84644/225000 [158:54:27<186:07:52,  4.77s/it]
 38%|███▊      | 84645/225000 [158:54:30<168:01:29,  4.31s/it]
 38%|███▊      | 84646/225000 [158:54:33<152:38:11,  3.92s/it]
 38%|███▊      | 84647/225000 [158:54:36<142:02:55,  3.64s/it]
 38%|███▊      | 84648/225000 [158:54:39<138:23:37,  3.55s/it]
 38%|███▊      | 84649/225000 [158:54:43<135:48:21,  3.48s/it]
 38%|███▊      | 84650/225000 [158:54:46<138:31:33,  3.55s/it]
                                                              
{'loss': '0.5891', 'grad_norm': '6.089', 'learning_rate': '3.656e-05', 'epoch': '0.3762', 'num_input_tokens_seen': 260938672, 'train_runtime': '5.721e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84650/225000 [158:54:46<138:31:33,  3.55s/it]
 38%|███▊      | 84651/225000 [158:54:50<138:28:11,  3.55s/it]
 38%|███▊      | 84652/225000 [158:54:53<133:12:32,  3.42s/it]
 38%|███▊      | 84653/225000 [158:54:56<129:05:19,  3.31s/it]
 38%|███▊      | 84654/225000 [158:55:00<131:36:59,  3.38s/it]
 38%|███▊      | 84655/225000 [158:55:07<181:38:58,  4.66s/it]
 38%|███▊      | 84656/225000 [158:55:15<215:29:42,  5.53s/it]
 38%|███▊      | 84657/225000 [158:55:17<178:53:33,  4.59s/it]
 38%|███▊      | 84658/225000 [158:55:20<156:42:59,  4.02s/it]
 38%|███▊      | 84659/225000 [158:55:24<155:23:42,  3.99s/it]
 38%|███▊      | 84660/225000 [158:55:27<143:13:03,  3.67s/it]
 38%|███▊      | 84661/225000 [158:55:30<134:44:57,  3.46s/it]
 38%|███▊      | 84662/225000 [158:55:33<133:09:19,  3.42s/it]
 38%|███▊      | 84663/225000 [158:55:36<133:18:10,  3.42s/it]
 38%|███▊      | 84664/225000 [158:55:40<136:42:46,  3.51s/it]
 38%|███▊      | 84665/225000 [158:55:43<133:38:37,  3.43s/it]
 38%|███▊      | 84666/225000 [158:55:47<136:26:10,  3.50s/it]
 38%|███▊      | 84667/225000 [158:55:54<176:05:50,  4.52s/it]
 38%|███▊      | 84668/225000 [158:55:59<179:00:42,  4.59s/it]
 38%|███▊      | 84669/225000 [158:56:04<183:07:02,  4.70s/it]
 38%|███▊      | 84670/225000 [158:56:09<189:30:53,  4.86s/it]
 38%|███▊      | 84671/225000 [158:56:16<217:18:51,  5.57s/it]
 38%|███▊      | 84672/225000 [158:56:19<184:39:21,  4.74s/it]
 38%|███▊      | 84673/225000 [158:56:22<162:21:24,  4.17s/it]
 38%|███▊      | 84674/225000 [158:56:25<146:43:07,  3.76s/it]
 38%|███▊      | 84675/225000 [158:56:27<136:34:12,  3.50s/it]
 38%|███▊      | 84676/225000 [158:56:30<130:21:38,  3.34s/it]
 38%|███▊      | 84677/225000 [158:56:38<178:44:48,  4.59s/it]
 38%|███▊      | 84678/225000 [158:56:43<182:57:43,  4.69s/it]
 38%|███▊      | 84679/225000 [158:56:46<162:17:12,  4.16s/it]
 38%|███▊      | 84680/225000 [158:56:49<155:00:04,  3.98s/it]
 38%|███▊      | 84681/225000 [158:56:53<146:22:41,  3.76s/it]
 38%|███▊      | 84682/225000 [158:56:56<145:27:58,  3.73s/it]
 38%|███▊      | 84683/225000 [158:56:59<135:20:57,  3.47s/it]
 38%|███▊      | 84684/225000 [158:57:03<135:04:32,  3.47s/it]
 38%|███▊      | 84685/225000 [158:57:06<131:01:05,  3.36s/it]
 38%|███▊      | 84686/225000 [158:57:11<158:41:55,  4.07s/it]
 38%|███▊      | 84687/225000 [158:57:20<215:43:28,  5.53s/it]
 38%|███▊      | 84688/225000 [158:57:23<186:06:08,  4.77s/it]
 38%|███▊      | 84689/225000 [158:57:26<161:49:10,  4.15s/it]
 38%|███▊      | 84690/225000 [158:57:29<145:16:28,  3.73s/it]
 38%|███▊      | 84691/225000 [158:57:32<134:59:10,  3.46s/it]
 38%|███▊      | 84692/225000 [158:57:34<126:53:43,  3.26s/it]
 38%|███▊      | 84693/225000 [158:57:37<123:36:04,  3.17s/it]
 38%|███▊      | 84694/225000 [158:57:40<121:08:40,  3.11s/it]
 38%|███▊      | 84695/225000 [158:57:43<120:40:23,  3.10s/it]
 38%|███▊      | 84696/225000 [158:57:53<195:43:20,  5.02s/it]
 38%|███▊      | 84697/225000 [158:57:57<188:09:58,  4.83s/it]
 38%|███▊      | 84698/225000 [158:58:00<168:49:02,  4.33s/it]
 38%|███▊      | 84699/225000 [158:58:03<150:39:32,  3.87s/it]
 38%|███▊      | 84700/225000 [158:58:06<140:42:21,  3.61s/it]
                                                              
{'loss': '0.5628', 'grad_norm': '4.034', 'learning_rate': '3.654e-05', 'epoch': '0.3764', 'num_input_tokens_seen': 261025576, 'train_runtime': '5.723e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84700/225000 [158:58:06<140:42:21,  3.61s/it]
 38%|███▊      | 84701/225000 [158:58:09<134:09:08,  3.44s/it]
 38%|███▊      | 84702/225000 [158:58:12<129:33:21,  3.32s/it]
 38%|███▊      | 84703/225000 [158:58:19<173:08:14,  4.44s/it]
 38%|███▊      | 84704/225000 [158:58:29<236:38:51,  6.07s/it]
 38%|███▊      | 84705/225000 [158:58:33<203:06:33,  5.21s/it]
 38%|███▊      | 84706/225000 [158:58:36<178:03:15,  4.57s/it]
 38%|███▊      | 84707/225000 [158:58:38<155:04:02,  3.98s/it]
 38%|███▊      | 84708/225000 [158:58:41<143:42:08,  3.69s/it]
 38%|███▊      | 84709/225000 [158:58:45<139:27:45,  3.58s/it]
 38%|███▊      | 84710/225000 [158:58:50<157:52:03,  4.05s/it]
 38%|███▊      | 84711/225000 [158:58:55<167:18:02,  4.29s/it]
 38%|███▊      | 84712/225000 [158:58:57<147:33:42,  3.79s/it]
 38%|███▊      | 84713/225000 [158:59:00<133:42:49,  3.43s/it]
 38%|███▊      | 84714/225000 [158:59:03<128:17:06,  3.29s/it]
 38%|███▊      | 84715/225000 [158:59:05<122:22:09,  3.14s/it]
 38%|███▊      | 84716/225000 [158:59:09<122:07:53,  3.13s/it]
 38%|███▊      | 84717/225000 [158:59:12<120:36:45,  3.10s/it]
 38%|███▊      | 84718/225000 [158:59:20<181:03:43,  4.65s/it]
 38%|███▊      | 84719/225000 [158:59:27<208:51:18,  5.36s/it]
 38%|███▊      | 84720/225000 [158:59:30<180:08:53,  4.62s/it]
 38%|███▊      | 84721/225000 [158:59:33<161:46:01,  4.15s/it]
 38%|███▊      | 84722/225000 [158:59:36<144:38:15,  3.71s/it]
 38%|███▊      | 84723/225000 [158:59:40<151:52:07,  3.90s/it]
 38%|███▊      | 84724/225000 [158:59:49<212:47:43,  5.46s/it]
 38%|███▊      | 84725/225000 [158:59:51<175:04:10,  4.49s/it]
 38%|███▊      | 84726/225000 [158:59:54<149:48:10,  3.84s/it]
 38%|███▊      | 84727/225000 [158:59:56<138:16:36,  3.55s/it]
 38%|███▊      | 84728/225000 [159:00:00<136:55:31,  3.51s/it]
 38%|███▊      | 84729/225000 [159:00:04<148:27:01,  3.81s/it]
 38%|███▊      | 84730/225000 [159:00:08<143:18:31,  3.68s/it]
 38%|███▊      | 84731/225000 [159:00:12<146:13:49,  3.75s/it]
 38%|███▊      | 84732/225000 [159:00:15<141:44:52,  3.64s/it]
 38%|███▊      | 84733/225000 [159:00:19<146:34:39,  3.76s/it]
 38%|███▊      | 84734/225000 [159:00:23<146:31:35,  3.76s/it]
 38%|███▊      | 84735/225000 [159:00:26<144:54:02,  3.72s/it]
 38%|███▊      | 84736/225000 [159:00:30<145:24:55,  3.73s/it]
 38%|███▊      | 84737/225000 [159:00:34<141:41:17,  3.64s/it]
 38%|███▊      | 84738/225000 [159:00:38<149:38:19,  3.84s/it]
 38%|███▊      | 84739/225000 [159:00:42<148:44:31,  3.82s/it]
 38%|███▊      | 84740/225000 [159:00:46<156:14:18,  4.01s/it]
 38%|███▊      | 84741/225000 [159:00:55<207:51:26,  5.34s/it]
 38%|███▊      | 84742/225000 [159:00:59<197:38:28,  5.07s/it]
 38%|███▊      | 84743/225000 [159:01:02<171:56:30,  4.41s/it]
 38%|███▊      | 84744/225000 [159:01:05<151:58:29,  3.90s/it]
 38%|███▊      | 84745/225000 [159:01:08<141:36:27,  3.63s/it]
 38%|███▊      | 84746/225000 [159:01:15<186:03:19,  4.78s/it]
 38%|███▊      | 84747/225000 [159:01:23<224:06:03,  5.75s/it]
 38%|███▊      | 84748/225000 [159:01:26<194:38:23,  5.00s/it]
 38%|███▊      | 84749/225000 [159:01:29<172:12:24,  4.42s/it]
 38%|███▊      | 84750/225000 [159:01:32<155:24:35,  3.99s/it]
                                                              
{'loss': '0.5436', 'grad_norm': '3.01', 'learning_rate': '3.653e-05', 'epoch': '0.3767', 'num_input_tokens_seen': 261111776, 'train_runtime': '5.725e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84750/225000 [159:01:32<155:24:35,  3.99s/it]
 38%|███▊      | 84751/225000 [159:01:36<145:16:24,  3.73s/it]
 38%|███▊      | 84752/225000 [159:01:39<141:33:47,  3.63s/it]
 38%|███▊      | 84753/225000 [159:01:42<136:40:10,  3.51s/it]
 38%|███▊      | 84754/225000 [159:01:47<157:38:27,  4.05s/it]
 38%|███▊      | 84755/225000 [159:01:55<201:58:04,  5.18s/it]
 38%|███▊      | 84756/225000 [159:01:59<183:07:12,  4.70s/it]
 38%|███▊      | 84757/225000 [159:02:02<166:21:30,  4.27s/it]
 38%|███▊      | 84758/225000 [159:02:05<151:23:07,  3.89s/it]
 38%|███▊      | 84759/225000 [159:02:08<145:15:35,  3.73s/it]
 38%|███▊      | 84760/225000 [159:02:11<135:58:44,  3.49s/it]
 38%|███▊      | 84761/225000 [159:02:15<132:18:51,  3.40s/it]
 38%|███▊      | 84762/225000 [159:02:18<134:28:55,  3.45s/it]
 38%|███▊      | 84763/225000 [159:02:21<132:34:25,  3.40s/it]
 38%|███▊      | 84764/225000 [159:02:25<131:53:52,  3.39s/it]
 38%|███▊      | 84765/225000 [159:02:28<133:21:54,  3.42s/it]
 38%|███▊      | 84766/225000 [159:02:32<130:36:33,  3.35s/it]
 38%|███▊      | 84767/225000 [159:02:37<157:13:05,  4.04s/it]
 38%|███▊      | 84768/225000 [159:02:46<215:56:32,  5.54s/it]
 38%|███▊      | 84769/225000 [159:02:51<204:33:24,  5.25s/it]
 38%|███▊      | 84770/225000 [159:02:54<176:18:16,  4.53s/it]
 38%|███▊      | 84771/225000 [159:02:57<157:24:29,  4.04s/it]
 38%|███▊      | 84772/225000 [159:02:59<143:03:51,  3.67s/it]
 38%|███▊      | 84773/225000 [159:03:03<139:47:10,  3.59s/it]
 38%|███▊      | 84774/225000 [159:03:11<195:20:53,  5.02s/it]
 38%|███▊      | 84775/225000 [159:03:14<166:05:44,  4.26s/it]
 38%|███▊      | 84776/225000 [159:03:16<147:36:45,  3.79s/it]
 38%|███▊      | 84777/225000 [159:03:19<135:45:47,  3.49s/it]
 38%|███▊      | 84778/225000 [159:03:22<124:25:44,  3.19s/it]
 38%|███▊      | 84779/225000 [159:03:28<166:01:16,  4.26s/it]
 38%|███▊      | 84780/225000 [159:03:34<187:23:13,  4.81s/it]
 38%|███▊      | 84781/225000 [159:03:37<162:43:00,  4.18s/it]
 38%|███▊      | 84782/225000 [159:03:40<144:41:52,  3.72s/it]
 38%|███▊      | 84783/225000 [159:03:42<129:50:39,  3.33s/it]
 38%|███▊      | 84784/225000 [159:03:48<156:39:02,  4.02s/it]
 38%|███▊      | 84785/225000 [159:03:53<171:13:47,  4.40s/it]
 38%|███▊      | 84786/225000 [159:03:56<158:49:55,  4.08s/it]
 38%|███▊      | 84787/225000 [159:04:00<156:05:08,  4.01s/it]
 38%|███▊      | 84788/225000 [159:04:03<143:18:37,  3.68s/it]
 38%|███▊      | 84789/225000 [159:04:06<137:53:36,  3.54s/it]
 38%|███▊      | 84790/225000 [159:04:10<135:46:49,  3.49s/it]
 38%|███▊      | 84791/225000 [159:04:13<130:28:45,  3.35s/it]
 38%|███▊      | 84792/225000 [159:04:16<129:49:55,  3.33s/it]
 38%|███▊      | 84793/225000 [159:04:20<134:13:19,  3.45s/it]
 38%|███▊      | 84794/225000 [159:04:23<133:40:47,  3.43s/it]
 38%|███▊      | 84795/225000 [159:04:33<203:41:06,  5.23s/it]
 38%|███▊      | 84796/225000 [159:04:37<199:19:16,  5.12s/it]
 38%|███▊      | 84797/225000 [159:04:40<172:32:46,  4.43s/it]
 38%|███▊      | 84798/225000 [159:04:48<205:13:58,  5.27s/it]
 38%|███▊      | 84799/225000 [159:04:51<188:11:33,  4.83s/it]
 38%|███▊      | 84800/225000 [159:04:55<169:37:22,  4.36s/it]
                                                              
{'loss': '0.5606', 'grad_norm': '4.395', 'learning_rate': '3.651e-05', 'epoch': '0.3769', 'num_input_tokens_seen': 261198376, 'train_runtime': '5.727e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84800/225000 [159:04:55<169:37:22,  4.36s/it]
 38%|███▊      | 84801/225000 [159:04:58<156:15:33,  4.01s/it]
 38%|███▊      | 84802/225000 [159:05:01<150:13:25,  3.86s/it]
 38%|███▊      | 84803/225000 [159:05:04<140:17:27,  3.60s/it]
 38%|███▊      | 84804/225000 [159:05:08<139:21:42,  3.58s/it]
 38%|███▊      | 84805/225000 [159:05:11<133:18:38,  3.42s/it]
 38%|███▊      | 84806/225000 [159:05:15<139:12:20,  3.57s/it]
 38%|███▊      | 84807/225000 [159:05:18<134:35:29,  3.46s/it]
 38%|███▊      | 84808/225000 [159:05:21<129:52:35,  3.34s/it]
 38%|███▊      | 84809/225000 [159:05:25<134:00:03,  3.44s/it]
 38%|███▊      | 84810/225000 [159:05:28<131:30:49,  3.38s/it]
 38%|███▊      | 84811/225000 [159:05:36<182:32:45,  4.69s/it]
 38%|███▊      | 84812/225000 [159:05:42<199:33:06,  5.12s/it]
 38%|███▊      | 84813/225000 [159:05:45<176:27:01,  4.53s/it]
 38%|███▊      | 84814/225000 [159:05:48<155:14:43,  3.99s/it]
 38%|███▊      | 84815/225000 [159:05:51<142:45:23,  3.67s/it]
 38%|███▊      | 84816/225000 [159:05:54<134:01:44,  3.44s/it]
 38%|███▊      | 84817/225000 [159:05:59<158:33:34,  4.07s/it]
 38%|███▊      | 84818/225000 [159:06:06<194:58:23,  5.01s/it]
 38%|███▊      | 84819/225000 [159:06:09<172:10:18,  4.42s/it]
 38%|███▊      | 84820/225000 [159:06:12<155:02:44,  3.98s/it]
 38%|███▊      | 84821/225000 [159:06:16<146:44:19,  3.77s/it]
 38%|███▊      | 84822/225000 [159:06:19<137:55:57,  3.54s/it]
 38%|███▊      | 84823/225000 [159:06:22<138:10:54,  3.55s/it]
 38%|███▊      | 84824/225000 [159:06:26<137:19:02,  3.53s/it]
 38%|███▊      | 84825/225000 [159:06:29<130:35:49,  3.35s/it]
 38%|███▊      | 84826/225000 [159:06:32<126:26:15,  3.25s/it]
 38%|███▊      | 84827/225000 [159:06:35<123:40:39,  3.18s/it]
 38%|███▊      | 84828/225000 [159:06:39<141:05:58,  3.62s/it]
 38%|███▊      | 84829/225000 [159:06:48<197:38:10,  5.08s/it]
 38%|███▊      | 84830/225000 [159:06:52<192:25:38,  4.94s/it]
 38%|███▊      | 84831/225000 [159:06:56<174:01:46,  4.47s/it]
 38%|███▊      | 84832/225000 [159:06:59<155:15:19,  3.99s/it]
 38%|███▊      | 84833/225000 [159:07:02<147:00:48,  3.78s/it]
 38%|███▊      | 84834/225000 [159:07:05<139:39:37,  3.59s/it]
 38%|███▊      | 84835/225000 [159:07:08<135:29:13,  3.48s/it]
 38%|███▊      | 84836/225000 [159:07:12<133:30:16,  3.43s/it]
 38%|███▊      | 84837/225000 [159:07:15<129:10:40,  3.32s/it]
 38%|███▊      | 84838/225000 [159:07:19<147:43:23,  3.79s/it]
 38%|███▊      | 84839/225000 [159:07:29<209:29:33,  5.38s/it]
 38%|███▊      | 84840/225000 [159:07:32<185:28:02,  4.76s/it]
 38%|███▊      | 84841/225000 [159:07:35<160:32:50,  4.12s/it]
 38%|███▊      | 84842/225000 [159:07:37<144:37:11,  3.71s/it]
 38%|███▊      | 84843/225000 [159:07:40<133:59:49,  3.44s/it]
 38%|███▊      | 84844/225000 [159:07:47<169:52:03,  4.36s/it]
 38%|███▊      | 84845/225000 [159:07:52<186:28:53,  4.79s/it]
 38%|███▊      | 84846/225000 [159:07:55<158:39:23,  4.08s/it]
 38%|███▊      | 84847/225000 [159:07:58<145:45:11,  3.74s/it]
 38%|███▊      | 84848/225000 [159:08:01<144:20:10,  3.71s/it]
 38%|███▊      | 84849/225000 [159:08:05<138:05:30,  3.55s/it]
 38%|███▊      | 84850/225000 [159:08:08<140:26:31,  3.61s/it]
                                                              
{'loss': '0.5583', 'grad_norm': '7.285', 'learning_rate': '3.649e-05', 'epoch': '0.3771', 'num_input_tokens_seen': 261284120, 'train_runtime': '5.729e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84850/225000 [159:08:08<140:26:31,  3.61s/it]
 38%|███▊      | 84851/225000 [159:08:12<141:01:36,  3.62s/it]
 38%|███▊      | 84852/225000 [159:08:15<138:50:07,  3.57s/it]
 38%|███▊      | 84853/225000 [159:08:19<138:40:36,  3.56s/it]
 38%|███▊      | 84854/225000 [159:08:22<137:33:09,  3.53s/it]
 38%|███▊      | 84855/225000 [159:08:26<133:45:07,  3.44s/it]
 38%|███▊      | 84856/225000 [159:08:31<151:43:16,  3.90s/it]
 38%|███▊      | 84857/225000 [159:08:39<203:58:04,  5.24s/it]
 38%|███▊      | 84858/225000 [159:08:45<217:14:44,  5.58s/it]
 38%|███▊      | 84859/225000 [159:08:49<190:48:32,  4.90s/it]
 38%|███▊      | 84860/225000 [159:08:52<174:20:55,  4.48s/it]
 38%|███▊      | 84861/225000 [159:08:56<161:35:55,  4.15s/it]
 38%|███▊      | 84862/225000 [159:08:59<148:20:49,  3.81s/it]
 38%|███▊      | 84863/225000 [159:09:02<145:19:03,  3.73s/it]
 38%|███▊      | 84864/225000 [159:09:05<140:48:09,  3.62s/it]
 38%|███▊      | 84865/225000 [159:09:10<149:23:33,  3.84s/it]
 38%|███▊      | 84866/225000 [159:09:13<140:23:15,  3.61s/it]
 38%|███▊      | 84867/225000 [159:09:16<134:54:21,  3.47s/it]
 38%|███▊      | 84868/225000 [159:09:19<130:09:43,  3.34s/it]
 38%|███▊      | 84869/225000 [159:09:23<141:46:31,  3.64s/it]
 38%|███▊      | 84870/225000 [159:09:32<199:29:11,  5.12s/it]
 38%|███▊      | 84871/225000 [159:09:36<190:33:43,  4.90s/it]
 38%|███▊      | 84872/225000 [159:09:39<166:40:32,  4.28s/it]
 38%|███▊      | 84873/225000 [159:09:42<153:36:04,  3.95s/it]
 38%|███▊      | 84874/225000 [159:09:46<145:43:28,  3.74s/it]
 38%|███▊      | 84875/225000 [159:09:49<144:31:58,  3.71s/it]
 38%|███▊      | 84876/225000 [159:09:53<142:04:55,  3.65s/it]
 38%|███▊      | 84877/225000 [159:09:56<133:23:31,  3.43s/it]
 38%|███▊      | 84878/225000 [159:09:59<129:09:17,  3.32s/it]
 38%|███▊      | 84879/225000 [159:10:06<172:18:20,  4.43s/it]
 38%|███▊      | 84880/225000 [159:10:14<215:20:35,  5.53s/it]
 38%|███▊      | 84881/225000 [159:10:18<193:22:26,  4.97s/it]
 38%|███▊      | 84882/225000 [159:10:21<176:00:10,  4.52s/it]
 38%|███▊      | 84883/225000 [159:10:24<163:31:54,  4.20s/it]
 38%|███▊      | 84884/225000 [159:10:28<154:03:27,  3.96s/it]
 38%|███▊      | 84885/225000 [159:10:31<143:39:50,  3.69s/it]
 38%|███▊      | 84886/225000 [159:10:34<138:37:10,  3.56s/it]
 38%|███▊      | 84887/225000 [159:10:38<142:04:46,  3.65s/it]
 38%|███▊      | 84888/225000 [159:10:42<140:03:41,  3.60s/it]
 38%|███▊      | 84889/225000 [159:10:45<138:33:07,  3.56s/it]
 38%|███▊      | 84890/225000 [159:10:48<134:52:52,  3.47s/it]
 38%|███▊      | 84891/225000 [159:10:52<133:24:29,  3.43s/it]
 38%|███▊      | 84892/225000 [159:10:55<129:40:02,  3.33s/it]
 38%|███▊      | 84893/225000 [159:10:58<133:03:29,  3.42s/it]
 38%|███▊      | 84894/225000 [159:11:02<130:51:27,  3.36s/it]
 38%|███▊      | 84895/225000 [159:11:05<131:12:58,  3.37s/it]
 38%|███▊      | 84896/225000 [159:11:13<184:28:10,  4.74s/it]
 38%|███▊      | 84897/225000 [159:11:16<166:47:55,  4.29s/it]
 38%|███▊      | 84898/225000 [159:11:19<156:13:01,  4.01s/it]
 38%|███▊      | 84899/225000 [159:11:23<146:05:23,  3.75s/it]
 38%|███▊      | 84900/225000 [159:11:26<139:47:27,  3.59s/it]
                                                              
{'loss': '0.5797', 'grad_norm': '4.998', 'learning_rate': '3.648e-05', 'epoch': '0.3773', 'num_input_tokens_seen': 261369528, 'train_runtime': '5.731e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84900/225000 [159:11:26<139:47:27,  3.59s/it]
 38%|███▊      | 84901/225000 [159:11:29<135:48:18,  3.49s/it]
 38%|███▊      | 84902/225000 [159:11:33<136:54:17,  3.52s/it]
 38%|███▊      | 84903/225000 [159:11:36<138:15:39,  3.55s/it]
 38%|███▊      | 84904/225000 [159:11:40<138:58:44,  3.57s/it]
 38%|███▊      | 84905/225000 [159:11:44<140:44:25,  3.62s/it]
 38%|███▊      | 84906/225000 [159:11:48<145:19:45,  3.73s/it]
 38%|███▊      | 84907/225000 [159:11:54<177:37:23,  4.56s/it]
 38%|███▊      | 84908/225000 [159:12:03<223:21:32,  5.74s/it]
 38%|███▊      | 84909/225000 [159:12:08<223:37:06,  5.75s/it]
 38%|███▊      | 84910/225000 [159:12:11<190:37:00,  4.90s/it]
 38%|███▊      | 84911/225000 [159:12:14<169:05:06,  4.35s/it]
 38%|███▊      | 84912/225000 [159:12:18<160:09:09,  4.12s/it]
 38%|███▊      | 84913/225000 [159:12:21<153:28:22,  3.94s/it]
 38%|███▊      | 84914/225000 [159:12:24<141:57:37,  3.65s/it]
 38%|███▊      | 84915/225000 [159:12:28<137:53:19,  3.54s/it]
 38%|███▊      | 84916/225000 [159:12:31<131:52:19,  3.39s/it]
 38%|███▊      | 84917/225000 [159:12:34<130:02:06,  3.34s/it]
 38%|███▊      | 84918/225000 [159:12:37<131:06:26,  3.37s/it]
 38%|███▊      | 84919/225000 [159:12:41<128:19:57,  3.30s/it]
 38%|███▊      | 84920/225000 [159:12:47<161:10:09,  4.14s/it]
 38%|███▊      | 84921/225000 [159:12:55<211:24:31,  5.43s/it]
 38%|███▊      | 84922/225000 [159:13:02<222:47:08,  5.73s/it]
 38%|███▊      | 84923/225000 [159:13:04<189:35:34,  4.87s/it]
 38%|███▊      | 84924/225000 [159:13:07<166:36:12,  4.28s/it]
 38%|███▊      | 84925/225000 [159:13:10<153:24:46,  3.94s/it]
 38%|███▊      | 84926/225000 [159:13:13<141:20:03,  3.63s/it]
 38%|███▊      | 84927/225000 [159:13:21<183:12:43,  4.71s/it]
 38%|███▊      | 84928/225000 [159:13:24<172:19:26,  4.43s/it]
 38%|███▊      | 84929/225000 [159:13:28<157:54:39,  4.06s/it]
 38%|███▊      | 84930/225000 [159:13:30<144:23:53,  3.71s/it]
 38%|███▊      | 84931/225000 [159:13:34<146:38:24,  3.77s/it]
 38%|███▊      | 84932/225000 [159:13:37<136:05:41,  3.50s/it]
 38%|███▊      | 84933/225000 [159:13:40<130:42:49,  3.36s/it]
 38%|███▊      | 84934/225000 [159:13:47<164:56:51,  4.24s/it]
 38%|███▊      | 84935/225000 [159:13:55<210:27:43,  5.41s/it]
 38%|███▊      | 84936/225000 [159:13:58<181:19:39,  4.66s/it]
 38%|███▊      | 84937/225000 [159:14:00<154:06:14,  3.96s/it]
 38%|███▊      | 84938/225000 [159:14:03<141:14:36,  3.63s/it]
 38%|███▊      | 84939/225000 [159:14:05<130:01:52,  3.34s/it]
 38%|███▊      | 84940/225000 [159:14:10<138:32:14,  3.56s/it]
 38%|███▊      | 84941/225000 [159:14:18<191:26:05,  4.92s/it]
 38%|███▊      | 84942/225000 [159:14:23<192:06:16,  4.94s/it]
 38%|███▊      | 84943/225000 [159:14:26<170:10:00,  4.37s/it]
 38%|███▊      | 84944/225000 [159:14:29<156:14:00,  4.02s/it]
 38%|███▊      | 84945/225000 [159:14:32<147:20:47,  3.79s/it]
 38%|███▊      | 84946/225000 [159:14:35<138:43:14,  3.57s/it]
 38%|███▊      | 84947/225000 [159:14:39<136:47:00,  3.52s/it]
 38%|███▊      | 84948/225000 [159:14:41<128:59:27,  3.32s/it]
 38%|███▊      | 84949/225000 [159:14:50<191:20:47,  4.92s/it]
 38%|███▊      | 84950/225000 [159:14:56<198:23:29,  5.10s/it]
                                                              
{'loss': '0.5724', 'grad_norm': '4.467', 'learning_rate': '3.646e-05', 'epoch': '0.3776', 'num_input_tokens_seen': 261456392, 'train_runtime': '5.733e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 84950/225000 [159:14:56<198:23:29,  5.10s/it]
 38%|███▊      | 84951/225000 [159:14:58<172:20:09,  4.43s/it]
 38%|███▊      | 84952/225000 [159:15:01<150:46:14,  3.88s/it]
 38%|███▊      | 84953/225000 [159:15:04<142:30:58,  3.66s/it]
 38%|███▊      | 84954/225000 [159:15:12<194:38:19,  5.00s/it]
 38%|███▊      | 84955/225000 [159:15:15<171:40:34,  4.41s/it]
 38%|███▊      | 84956/225000 [159:15:18<146:27:10,  3.76s/it]
 38%|███▊      | 84957/225000 [159:15:20<131:15:32,  3.37s/it]
 38%|███▊      | 84958/225000 [159:15:28<180:03:03,  4.63s/it]
 38%|███▊      | 84959/225000 [159:15:31<159:29:10,  4.10s/it]
 38%|███▊      | 84960/225000 [159:15:34<147:17:58,  3.79s/it]
 38%|███▊      | 84961/225000 [159:15:37<137:29:19,  3.53s/it]
 38%|███▊      | 84962/225000 [159:15:40<131:13:04,  3.37s/it]
 38%|███▊      | 84963/225000 [159:15:42<125:14:00,  3.22s/it]
 38%|███▊      | 84964/225000 [159:15:46<125:30:58,  3.23s/it]
 38%|███▊      | 84965/225000 [159:15:49<125:29:56,  3.23s/it]
 38%|███▊      | 84966/225000 [159:15:52<120:09:43,  3.09s/it]
 38%|███▊      | 84967/225000 [159:15:58<161:56:24,  4.16s/it]
 38%|███▊      | 84968/225000 [159:16:07<216:36:37,  5.57s/it]
 38%|███▊      | 84969/225000 [159:16:10<181:02:33,  4.65s/it]
 38%|███▊      | 84970/225000 [159:16:12<155:40:25,  4.00s/it]
 38%|███▊      | 84971/225000 [159:16:15<137:57:38,  3.55s/it]
 38%|███▊      | 84972/225000 [159:16:17<128:27:13,  3.30s/it]
 38%|███▊      | 84973/225000 [159:16:24<165:21:00,  4.25s/it]
 38%|███▊      | 84974/225000 [159:16:30<183:00:47,  4.71s/it]
 38%|███▊      | 84975/225000 [159:16:32<156:50:05,  4.03s/it]
 38%|███▊      | 84976/225000 [159:16:35<140:12:05,  3.60s/it]
 38%|███▊      | 84977/225000 [159:16:38<134:17:54,  3.45s/it]
 38%|███▊      | 84978/225000 [159:16:41<126:23:18,  3.25s/it]
 38%|███▊      | 84979/225000 [159:16:49<192:29:23,  4.95s/it]
 38%|███▊      | 84980/225000 [159:16:53<170:52:37,  4.39s/it]
 38%|███▊      | 84981/225000 [159:16:55<153:43:03,  3.95s/it]
 38%|███▊      | 84982/225000 [159:16:59<144:06:24,  3.71s/it]
 38%|███▊      | 84983/225000 [159:17:01<133:29:02,  3.43s/it]
 38%|███▊      | 84984/225000 [159:17:04<126:45:23,  3.26s/it]
 38%|███▊      | 84985/225000 [159:17:08<128:42:03,  3.31s/it]
 38%|███▊      | 84986/225000 [159:17:11<127:00:37,  3.27s/it]
 38%|███▊      | 84987/225000 [159:17:16<152:18:51,  3.92s/it]
 38%|███▊      | 84988/225000 [159:17:25<205:30:47,  5.28s/it]
 38%|███▊      | 84989/225000 [159:17:29<199:25:12,  5.13s/it]
 38%|███▊      | 84990/225000 [159:17:33<178:35:26,  4.59s/it]
 38%|███▊      | 84991/225000 [159:17:36<163:26:32,  4.20s/it]
 38%|███▊      | 84992/225000 [159:17:39<150:07:45,  3.86s/it]
 38%|███▊      | 84993/225000 [159:17:43<146:28:45,  3.77s/it]
 38%|███▊      | 84994/225000 [159:17:46<142:34:23,  3.67s/it]
 38%|███▊      | 84995/225000 [159:17:49<134:56:17,  3.47s/it]
 38%|███▊      | 84996/225000 [159:17:53<135:02:32,  3.47s/it]
 38%|███▊      | 84997/225000 [159:17:56<131:08:39,  3.37s/it]
 38%|███▊      | 84998/225000 [159:17:59<132:12:13,  3.40s/it]
 38%|███▊      | 84999/225000 [159:18:03<139:30:36,  3.59s/it]
 38%|███▊      | 85000/225000 [159:18:13<206:12:57,  5.30s/it]
                                                              
{'loss': '0.5291', 'grad_norm': '6.56', 'learning_rate': '3.644e-05', 'epoch': '0.3778', 'num_input_tokens_seen': 261542240, 'train_runtime': '5.735e+05', 'train_tokens_per_second': '456.1'}

 38%|███▊      | 85000/225000 [159:18:13<206:12:57,  5.30s/it]
 38%|███▊      | 85001/225000 [159:18:18<203:49:10,  5.24s/it]
 38%|███▊      | 85002/225000 [159:18:20<172:11:07,  4.43s/it]
 38%|███▊      | 85003/225000 [159:18:26<182:52:04,  4.70s/it]
 38%|███▊      | 85004/225000 [159:18:29<168:09:57,  4.32s/it]
 38%|███▊      | 85005/225000 [159:18:31<145:53:27,  3.75s/it]
 38%|███▊      | 85006/225000 [159:18:34<131:54:38,  3.39s/it]
 38%|███▊      | 85007/225000 [159:18:37<127:27:50,  3.28s/it]
 38%|███▊      | 85008/225000 [159:18:40<128:06:20,  3.29s/it]
 38%|███▊      | 85009/225000 [159:18:44<133:21:36,  3.43s/it]
 38%|███▊      | 85010/225000 [159:18:48<136:56:47,  3.52s/it]
 38%|███▊      | 85011/225000 [159:18:54<166:25:20,  4.28s/it]
 38%|███▊      | 85012/225000 [159:19:01<202:20:11,  5.20s/it]
 38%|███▊      | 85013/225000 [159:19:04<175:28:01,  4.51s/it]
 38%|███▊      | 85014/225000 [159:19:07<156:26:05,  4.02s/it]
 38%|███▊      | 85015/225000 [159:19:10<144:56:31,  3.73s/it]
 38%|███▊      | 85016/225000 [159:19:13<136:56:01,  3.52s/it]
 38%|███▊      | 85017/225000 [159:19:16<128:18:02,  3.30s/it]
 38%|███▊      | 85018/225000 [159:19:23<168:47:55,  4.34s/it]
 38%|███▊      | 85019/225000 [159:19:29<196:24:28,  5.05s/it]
 38%|███▊      | 85020/225000 [159:19:32<166:57:33,  4.29s/it]
 38%|███▊      | 85021/225000 [159:19:37<176:29:49,  4.54s/it]
 38%|███▊      | 85022/225000 [159:19:40<160:13:44,  4.12s/it]
 38%|███▊      | 85023/225000 [159:19:43<146:06:12,  3.76s/it]
 38%|███▊      | 85024/225000 [159:19:49<167:22:54,  4.30s/it]
 38%|███▊      | 85025/225000 [159:19:52<160:23:12,  4.12s/it]
 38%|███▊      | 85026/225000 [159:19:55<149:31:43,  3.85s/it]
 38%|███▊      | 85027/225000 [159:19:59<149:14:13,  3.84s/it]
 38%|███▊      | 85028/225000 [159:20:02<139:14:32,  3.58s/it]
 38%|███▊      | 85029/225000 [159:20:06<134:50:52,  3.47s/it]
 38%|███▊      | 85030/225000 [159:20:09<129:28:07,  3.33s/it]
 38%|███▊      | 85031/225000 [159:20:12<128:57:15,  3.32s/it]
 38%|███▊      | 85032/225000 [159:20:15<124:13:40,  3.20s/it]
 38%|███▊      | 85033/225000 [159:20:18<123:19:31,  3.17s/it]
 38%|███▊      | 85034/225000 [159:20:21<123:49:32,  3.18s/it]
 38%|███▊      | 85035/225000 [159:20:24<124:50:19,  3.21s/it]
 38%|███▊      | 85036/225000 [159:20:34<199:28:23,  5.13s/it]
 38%|███▊      | 85037/225000 [159:20:39<204:31:43,  5.26s/it]
 38%|███▊      | 85038/225000 [159:20:42<172:26:30,  4.44s/it]
 38%|███▊      | 85039/225000 [159:20:45<154:34:33,  3.98s/it]
 38%|███▊      | 85040/225000 [159:20:47<137:35:53,  3.54s/it]
 38%|███▊      | 85041/225000 [159:20:50<127:46:59,  3.29s/it]
 38%|███▊      | 85042/225000 [159:20:58<185:08:51,  4.76s/it]
 38%|███▊      | 85043/225000 [159:21:04<199:11:51,  5.12s/it]
 38%|███▊      | 85044/225000 [159:21:07<167:50:25,  4.32s/it]
 38%|███▊      | 85045/225000 [159:21:10<152:02:31,  3.91s/it]
 38%|███▊      | 85046/225000 [159:21:12<136:43:12,  3.52s/it]
 38%|███▊      | 85047/225000 [159:21:18<168:05:52,  4.32s/it]
 38%|███▊      | 85048/225000 [159:21:24<180:19:59,  4.64s/it]
 38%|███▊      | 85049/225000 [159:21:26<154:49:33,  3.98s/it]
 38%|███▊      | 85050/225000 [159:21:32<171:04:39,  4.40s/it]
                                                              
{'loss': '0.5612', 'grad_norm': '4.842', 'learning_rate': '3.643e-05', 'epoch': '0.378', 'num_input_tokens_seen': 261627072, 'train_runtime': '5.737e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85050/225000 [159:21:32<171:04:39,  4.40s/it]
 38%|███▊      | 85051/225000 [159:21:36<165:25:53,  4.26s/it]
 38%|███▊      | 85052/225000 [159:21:38<144:59:34,  3.73s/it]
 38%|███▊      | 85053/225000 [159:21:40<128:52:15,  3.32s/it]
 38%|███▊      | 85054/225000 [159:21:43<118:56:39,  3.06s/it]
 38%|███▊      | 85055/225000 [159:21:46<119:24:41,  3.07s/it]
 38%|███▊      | 85056/225000 [159:21:50<134:42:04,  3.47s/it]
 38%|███▊      | 85057/225000 [159:21:59<193:38:06,  4.98s/it]
 38%|███▊      | 85058/225000 [159:22:02<174:10:20,  4.48s/it]
 38%|███▊      | 85059/225000 [159:22:05<151:09:56,  3.89s/it]
 38%|███▊      | 85060/225000 [159:22:07<135:22:54,  3.48s/it]
 38%|███▊      | 85061/225000 [159:22:10<126:38:35,  3.26s/it]
 38%|███▊      | 85062/225000 [159:22:13<121:02:18,  3.11s/it]
 38%|███▊      | 85063/225000 [159:22:22<196:12:13,  5.05s/it]
 38%|███▊      | 85064/225000 [159:22:26<176:06:20,  4.53s/it]
 38%|███▊      | 85065/225000 [159:22:28<152:57:18,  3.93s/it]
 38%|███▊      | 85066/225000 [159:22:31<135:54:30,  3.50s/it]
 38%|███▊      | 85067/225000 [159:22:33<126:08:44,  3.25s/it]
 38%|███▊      | 85068/225000 [159:22:36<119:46:36,  3.08s/it]
 38%|███▊      | 85069/225000 [159:22:41<146:44:10,  3.78s/it]
 38%|███▊      | 85070/225000 [159:22:49<190:57:41,  4.91s/it]
 38%|███▊      | 85071/225000 [159:22:52<165:38:09,  4.26s/it]
 38%|███▊      | 85072/225000 [159:22:55<148:21:00,  3.82s/it]
 38%|███▊      | 85073/225000 [159:22:57<133:25:04,  3.43s/it]
 38%|███▊      | 85074/225000 [159:23:00<125:15:28,  3.22s/it]
 38%|███▊      | 85075/225000 [159:23:09<194:53:50,  5.01s/it]
 38%|███▊      | 85076/225000 [159:23:13<183:49:43,  4.73s/it]
 38%|███▊      | 85077/225000 [159:23:16<160:57:38,  4.14s/it]
 38%|███▊      | 85078/225000 [159:23:20<165:10:14,  4.25s/it]
 38%|███▊      | 85079/225000 [159:23:24<162:33:25,  4.18s/it]
 38%|███▊      | 85080/225000 [159:23:27<142:38:39,  3.67s/it]
 38%|███▊      | 85081/225000 [159:23:30<133:01:17,  3.42s/it]
 38%|███▊      | 85082/225000 [159:23:33<127:27:24,  3.28s/it]
 38%|███▊      | 85083/225000 [159:23:40<170:31:18,  4.39s/it]
 38%|███▊      | 85084/225000 [159:23:45<183:04:37,  4.71s/it]
 38%|███▊      | 85085/225000 [159:23:47<156:25:27,  4.02s/it]
 38%|███▊      | 85086/225000 [159:23:50<141:02:10,  3.63s/it]
 38%|███▊      | 85087/225000 [159:24:01<219:27:46,  5.65s/it]
 38%|███▊      | 85088/225000 [159:24:03<180:29:53,  4.64s/it]
 38%|███▊      | 85089/225000 [159:24:05<152:54:06,  3.93s/it]
 38%|███▊      | 85090/225000 [159:24:07<133:11:49,  3.43s/it]
 38%|███▊      | 85091/225000 [159:24:10<127:00:36,  3.27s/it]
 38%|███▊      | 85092/225000 [159:24:14<135:53:02,  3.50s/it]
 38%|███▊      | 85093/225000 [159:24:23<201:56:30,  5.20s/it]
 38%|███▊      | 85094/225000 [159:24:26<167:46:49,  4.32s/it]
 38%|███▊      | 85095/225000 [159:24:29<150:37:26,  3.88s/it]
 38%|███▊      | 85096/225000 [159:24:31<137:47:38,  3.55s/it]
 38%|███▊      | 85097/225000 [159:24:34<131:18:17,  3.38s/it]
 38%|███▊      | 85098/225000 [159:24:37<126:32:44,  3.26s/it]
 38%|███▊      | 85099/225000 [159:24:40<124:07:47,  3.19s/it]
 38%|███▊      | 85100/225000 [159:24:44<126:07:54,  3.25s/it]
                                                              
{'loss': '0.5476', 'grad_norm': '5.947', 'learning_rate': '3.641e-05', 'epoch': '0.3782', 'num_input_tokens_seen': 261712336, 'train_runtime': '5.739e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85100/225000 [159:24:44<126:07:54,  3.25s/it]
 38%|███▊      | 85101/225000 [159:24:53<194:17:02,  5.00s/it]
 38%|███▊      | 85102/225000 [159:24:56<170:34:43,  4.39s/it]
 38%|███▊      | 85103/225000 [159:24:59<152:33:16,  3.93s/it]
 38%|███▊      | 85104/225000 [159:25:02<143:43:02,  3.70s/it]
 38%|███▊      | 85105/225000 [159:25:05<135:27:50,  3.49s/it]
 38%|███▊      | 85106/225000 [159:25:10<159:21:01,  4.10s/it]
 38%|███▊      | 85107/225000 [159:25:19<216:20:29,  5.57s/it]
 38%|███▊      | 85108/225000 [159:25:23<189:46:44,  4.88s/it]
 38%|███▊      | 85109/225000 [159:25:25<162:19:17,  4.18s/it]
 38%|███▊      | 85110/225000 [159:25:28<144:17:33,  3.71s/it]
 38%|███▊      | 85111/225000 [159:25:31<140:13:43,  3.61s/it]
 38%|███▊      | 85112/225000 [159:25:35<140:42:03,  3.62s/it]
 38%|███▊      | 85113/225000 [159:25:39<144:58:46,  3.73s/it]
 38%|███▊      | 85114/225000 [159:25:42<140:32:01,  3.62s/it]
 38%|███▊      | 85115/225000 [159:25:46<140:28:29,  3.62s/it]
 38%|███▊      | 85116/225000 [159:25:50<149:16:36,  3.84s/it]
 38%|███▊      | 85117/225000 [159:25:53<142:30:14,  3.67s/it]
 38%|███▊      | 85118/225000 [159:25:58<148:07:17,  3.81s/it]
 38%|███▊      | 85119/225000 [159:26:04<175:46:10,  4.52s/it]
 38%|███▊      | 85120/225000 [159:26:10<195:50:21,  5.04s/it]
 38%|███▊      | 85121/225000 [159:26:13<169:39:15,  4.37s/it]
 38%|███▊      | 85122/225000 [159:26:16<156:15:05,  4.02s/it]
 38%|███▊      | 85123/225000 [159:26:19<145:09:04,  3.74s/it]
 38%|███▊      | 85124/225000 [159:26:22<140:00:36,  3.60s/it]
 38%|███▊      | 85125/225000 [159:26:30<185:22:29,  4.77s/it]
 38%|███▊      | 85126/225000 [159:26:39<236:37:06,  6.09s/it]
 38%|███▊      | 85127/225000 [159:26:41<192:12:38,  4.95s/it]
 38%|███▊      | 85128/225000 [159:26:44<163:09:59,  4.20s/it]
 38%|███▊      | 85129/225000 [159:26:46<142:38:46,  3.67s/it]
 38%|███▊      | 85130/225000 [159:26:51<158:17:31,  4.07s/it]
 38%|███▊      | 85131/225000 [159:26:58<193:14:38,  4.97s/it]
 38%|███▊      | 85132/225000 [159:27:00<161:32:39,  4.16s/it]
 38%|███▊      | 85133/225000 [159:27:06<181:01:02,  4.66s/it]
 38%|███▊      | 85134/225000 [159:27:09<154:52:54,  3.99s/it]
 38%|███▊      | 85135/225000 [159:27:11<136:49:24,  3.52s/it]
 38%|███▊      | 85136/225000 [159:27:14<132:11:42,  3.40s/it]
 38%|███▊      | 85137/225000 [159:27:20<159:06:50,  4.10s/it]
 38%|███▊      | 85138/225000 [159:27:23<150:36:19,  3.88s/it]
 38%|███▊      | 85139/225000 [159:27:27<143:20:14,  3.69s/it]
 38%|███▊      | 85140/225000 [159:27:30<141:52:15,  3.65s/it]
 38%|███▊      | 85141/225000 [159:27:34<138:48:11,  3.57s/it]
 38%|███▊      | 85142/225000 [159:27:37<138:37:30,  3.57s/it]
 38%|███▊      | 85143/225000 [159:27:41<137:50:10,  3.55s/it]
 38%|███▊      | 85144/225000 [159:27:44<139:52:22,  3.60s/it]
 38%|███▊      | 85145/225000 [159:27:48<142:08:41,  3.66s/it]
 38%|███▊      | 85146/225000 [159:27:52<144:50:34,  3.73s/it]
 38%|███▊      | 85147/225000 [159:27:56<142:43:53,  3.67s/it]
 38%|███▊      | 85148/225000 [159:27:59<142:39:50,  3.67s/it]
 38%|███▊      | 85149/225000 [159:28:03<140:11:54,  3.61s/it]
 38%|███▊      | 85150/225000 [159:28:06<137:42:02,  3.54s/it]
                                                              
{'loss': '0.4986', 'grad_norm': '6.637', 'learning_rate': '3.64e-05', 'epoch': '0.3784', 'num_input_tokens_seen': 261799416, 'train_runtime': '5.741e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85150/225000 [159:28:06<137:42:02,  3.54s/it]
 38%|███▊      | 85151/225000 [159:28:14<190:51:36,  4.91s/it]
 38%|███▊      | 85152/225000 [159:28:23<238:50:15,  6.15s/it]
 38%|███▊      | 85153/225000 [159:28:26<200:58:17,  5.17s/it]
 38%|███▊      | 85154/225000 [159:28:29<178:12:08,  4.59s/it]
 38%|███▊      | 85155/225000 [159:28:32<159:08:24,  4.10s/it]
 38%|███▊      | 85156/225000 [159:28:36<148:26:57,  3.82s/it]
 38%|███▊      | 85157/225000 [159:28:38<138:43:37,  3.57s/it]
 38%|███▊      | 85158/225000 [159:28:42<140:26:11,  3.62s/it]
 38%|███▊      | 85159/225000 [159:28:45<136:28:08,  3.51s/it]
 38%|███▊      | 85160/225000 [159:28:51<156:32:19,  4.03s/it]
 38%|███▊      | 85161/225000 [159:28:59<202:54:11,  5.22s/it]
 38%|███▊      | 85162/225000 [159:29:01<172:16:54,  4.44s/it]
 38%|███▊      | 85163/225000 [159:29:04<157:03:29,  4.04s/it]
 38%|███▊      | 85164/225000 [159:29:07<143:44:15,  3.70s/it]
 38%|███▊      | 85165/225000 [159:29:10<135:52:41,  3.50s/it]
 38%|███▊      | 85166/225000 [159:29:16<162:42:12,  4.19s/it]
 38%|███▊      | 85167/225000 [159:29:23<196:14:39,  5.05s/it]
 38%|███▊      | 85168/225000 [159:29:26<171:20:06,  4.41s/it]
 38%|███▊      | 85169/225000 [159:29:29<157:42:40,  4.06s/it]
 38%|███▊      | 85170/225000 [159:29:33<147:41:30,  3.80s/it]
 38%|███▊      | 85171/225000 [159:29:35<133:51:30,  3.45s/it]
 38%|███▊      | 85172/225000 [159:29:42<176:41:27,  4.55s/it]
 38%|███▊      | 85173/225000 [159:29:47<181:59:15,  4.69s/it]
 38%|███▊      | 85174/225000 [159:29:50<159:17:33,  4.10s/it]
 38%|███▊      | 85175/225000 [159:29:53<140:21:28,  3.61s/it]
 38%|███▊      | 85176/225000 [159:29:56<133:28:29,  3.44s/it]
 38%|███▊      | 85177/225000 [159:30:01<156:39:58,  4.03s/it]
 38%|███▊      | 85178/225000 [159:30:10<208:42:04,  5.37s/it]
 38%|███▊      | 85179/225000 [159:30:12<177:32:07,  4.57s/it]
 38%|███▊      | 85180/225000 [159:30:15<153:05:20,  3.94s/it]
 38%|███▊      | 85181/225000 [159:30:17<133:02:26,  3.43s/it]
 38%|███▊      | 85182/225000 [159:30:25<188:29:37,  4.85s/it]
 38%|███▊      | 85183/225000 [159:30:27<159:44:49,  4.11s/it]
 38%|███▊      | 85184/225000 [159:30:30<138:04:47,  3.56s/it]
 38%|███▊      | 85185/225000 [159:30:37<177:44:23,  4.58s/it]
 38%|███▊      | 85186/225000 [159:30:39<153:36:56,  3.96s/it]
 38%|███▊      | 85187/225000 [159:30:43<147:03:52,  3.79s/it]
 38%|███▊      | 85188/225000 [159:30:46<138:05:40,  3.56s/it]
 38%|███▊      | 85189/225000 [159:30:49<131:07:42,  3.38s/it]
 38%|███▊      | 85190/225000 [159:30:52<131:22:38,  3.38s/it]
 38%|███▊      | 85191/225000 [159:30:55<123:52:11,  3.19s/it]
 38%|███▊      | 85192/225000 [159:30:58<128:09:34,  3.30s/it]
 38%|███▊      | 85193/225000 [159:31:03<145:22:42,  3.74s/it]
 38%|███▊      | 85194/225000 [159:31:11<199:40:58,  5.14s/it]
 38%|███▊      | 85195/225000 [159:31:15<182:27:09,  4.70s/it]
 38%|███▊      | 85196/225000 [159:31:19<167:42:29,  4.32s/it]
 38%|███▊      | 85197/225000 [159:31:22<155:27:09,  4.00s/it]
 38%|███▊      | 85198/225000 [159:31:25<145:30:37,  3.75s/it]
 38%|███▊      | 85199/225000 [159:31:28<142:11:10,  3.66s/it]
 38%|███▊      | 85200/225000 [159:31:32<139:48:54,  3.60s/it]
                                                              
{'loss': '0.5499', 'grad_norm': '6.439', 'learning_rate': '3.638e-05', 'epoch': '0.3787', 'num_input_tokens_seen': 261885696, 'train_runtime': '5.743e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85200/225000 [159:31:32<139:48:54,  3.60s/it]
 38%|███▊      | 85201/225000 [159:31:36<142:54:55,  3.68s/it]
 38%|███▊      | 85202/225000 [159:31:39<135:51:08,  3.50s/it]
 38%|███▊      | 85203/225000 [159:31:42<132:25:00,  3.41s/it]
 38%|███▊      | 85204/225000 [159:31:45<129:36:09,  3.34s/it]
 38%|███▊      | 85205/225000 [159:31:49<136:44:08,  3.52s/it]
 38%|███▊      | 85206/225000 [159:31:52<133:55:26,  3.45s/it]
 38%|███▊      | 85207/225000 [159:32:01<196:43:32,  5.07s/it]
 38%|███▊      | 85208/225000 [159:32:08<217:32:22,  5.60s/it]
 38%|███▊      | 85209/225000 [159:32:10<179:05:53,  4.61s/it]
 38%|███▊      | 85210/225000 [159:32:13<154:47:25,  3.99s/it]
 38%|███▊      | 85211/225000 [159:32:16<143:41:38,  3.70s/it]
 38%|███▊      | 85212/225000 [159:32:20<142:08:46,  3.66s/it]
 38%|███▊      | 85213/225000 [159:32:23<136:58:04,  3.53s/it]
 38%|███▊      | 85214/225000 [159:32:26<132:45:31,  3.42s/it]
 38%|███▊      | 85215/225000 [159:32:32<163:54:01,  4.22s/it]
 38%|███▊      | 85216/225000 [159:32:39<191:30:36,  4.93s/it]
 38%|███▊      | 85217/225000 [159:32:42<171:18:42,  4.41s/it]
 38%|███▊      | 85218/225000 [159:32:46<167:37:04,  4.32s/it]
 38%|███▊      | 85219/225000 [159:32:49<153:47:04,  3.96s/it]
 38%|███▊      | 85220/225000 [159:32:52<147:52:33,  3.81s/it]
 38%|███▊      | 85221/225000 [159:32:56<145:46:45,  3.75s/it]
 38%|███▊      | 85222/225000 [159:33:01<160:14:44,  4.13s/it]
 38%|███▊      | 85223/225000 [159:33:06<174:39:49,  4.50s/it]
 38%|███▊      | 85224/225000 [159:33:10<161:40:50,  4.16s/it]
 38%|███▊      | 85225/225000 [159:33:13<149:55:05,  3.86s/it]
 38%|███▊      | 85226/225000 [159:33:16<141:56:55,  3.66s/it]
 38%|███▊      | 85227/225000 [159:33:19<135:13:05,  3.48s/it]
 38%|███▊      | 85228/225000 [159:33:23<133:36:48,  3.44s/it]
 38%|███▊      | 85229/225000 [159:33:26<131:38:56,  3.39s/it]
 38%|███▊      | 85230/225000 [159:33:29<131:03:09,  3.38s/it]
 38%|███▊      | 85231/225000 [159:33:33<139:33:43,  3.59s/it]
 38%|███▊      | 85232/225000 [159:33:42<202:46:30,  5.22s/it]
 38%|███▊      | 85233/225000 [159:33:47<195:43:09,  5.04s/it]
 38%|███▊      | 85234/225000 [159:33:50<171:05:26,  4.41s/it]
 38%|███▊      | 85235/225000 [159:33:53<151:40:03,  3.91s/it]
 38%|███▊      | 85236/225000 [159:33:55<137:46:36,  3.55s/it]
 38%|███▊      | 85237/225000 [159:33:58<130:24:25,  3.36s/it]
 38%|███▊      | 85238/225000 [159:34:02<132:09:54,  3.40s/it]
 38%|███▊      | 85239/225000 [159:34:05<129:59:33,  3.35s/it]
 38%|███▊      | 85240/225000 [159:34:10<145:40:31,  3.75s/it]
 38%|███▊      | 85241/225000 [159:34:18<200:28:37,  5.16s/it]
 38%|███▊      | 85242/225000 [159:34:21<176:21:21,  4.54s/it]
 38%|███▊      | 85243/225000 [159:34:24<153:21:08,  3.95s/it]
 38%|███▊      | 85244/225000 [159:34:30<178:51:22,  4.61s/it]
 38%|███▊      | 85245/225000 [159:34:34<169:06:02,  4.36s/it]
 38%|███▊      | 85246/225000 [159:34:36<147:25:02,  3.80s/it]
 38%|███▊      | 85247/225000 [159:34:39<134:34:59,  3.47s/it]
 38%|███▊      | 85248/225000 [159:34:42<130:50:34,  3.37s/it]
 38%|███▊      | 85249/225000 [159:34:45<129:54:46,  3.35s/it]
 38%|███▊      | 85250/225000 [159:34:49<132:29:12,  3.41s/it]
                                                              
{'loss': '0.5244', 'grad_norm': '3.668', 'learning_rate': '3.636e-05', 'epoch': '0.3789', 'num_input_tokens_seen': 261971960, 'train_runtime': '5.745e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85250/225000 [159:34:49<132:29:12,  3.41s/it]
 38%|███▊      | 85251/225000 [159:34:59<215:53:45,  5.56s/it]
 38%|███▊      | 85252/225000 [159:35:03<196:34:21,  5.06s/it]
 38%|███▊      | 85253/225000 [159:35:06<171:13:02,  4.41s/it]
 38%|███▊      | 85254/225000 [159:35:09<156:41:16,  4.04s/it]
 38%|███▊      | 85255/225000 [159:35:13<150:59:29,  3.89s/it]
 38%|███▊      | 85256/225000 [159:35:16<138:11:57,  3.56s/it]
 38%|███▊      | 85257/225000 [159:35:19<128:40:24,  3.31s/it]
 38%|███▊      | 85258/225000 [159:35:27<187:09:37,  4.82s/it]
 38%|███▊      | 85259/225000 [159:35:30<173:03:20,  4.46s/it]
 38%|███▊      | 85260/225000 [159:35:33<155:09:28,  4.00s/it]
 38%|███▊      | 85261/225000 [159:35:37<146:39:54,  3.78s/it]
 38%|███▊      | 85262/225000 [159:35:40<141:15:31,  3.64s/it]
 38%|███▊      | 85263/225000 [159:35:43<134:12:58,  3.46s/it]
 38%|███▊      | 85264/225000 [159:35:46<132:09:20,  3.40s/it]
 38%|███▊      | 85265/225000 [159:35:49<127:17:16,  3.28s/it]
 38%|███▊      | 85266/225000 [159:35:52<123:25:43,  3.18s/it]
 38%|███▊      | 85267/225000 [159:35:56<125:24:26,  3.23s/it]
 38%|███▊      | 85268/225000 [159:36:01<149:12:47,  3.84s/it]
 38%|███▊      | 85269/225000 [159:36:10<206:04:33,  5.31s/it]
 38%|███▊      | 85270/225000 [159:36:15<202:55:23,  5.23s/it]
 38%|███▊      | 85271/225000 [159:36:17<172:12:32,  4.44s/it]
 38%|███▊      | 85272/225000 [159:36:20<149:48:42,  3.86s/it]
 38%|███▊      | 85273/225000 [159:36:27<185:49:57,  4.79s/it]
 38%|███▊      | 85274/225000 [159:36:30<167:38:56,  4.32s/it]
 38%|███▊      | 85275/225000 [159:36:33<151:47:58,  3.91s/it]
 38%|███▊      | 85276/225000 [159:36:36<138:04:28,  3.56s/it]
 38%|███▊      | 85277/225000 [159:36:38<130:22:40,  3.36s/it]
 38%|███▊      | 85278/225000 [159:36:41<126:14:14,  3.25s/it]
 38%|███▊      | 85279/225000 [159:36:46<145:14:59,  3.74s/it]
 38%|███▊      | 85280/225000 [159:36:55<199:14:41,  5.13s/it]
 38%|███▊      | 85281/225000 [159:36:57<166:30:54,  4.29s/it]
 38%|███▊      | 85282/225000 [159:37:00<145:10:10,  3.74s/it]
 38%|███▊      | 85283/225000 [159:37:02<128:40:42,  3.32s/it]
 38%|███▊      | 85284/225000 [159:37:05<125:31:38,  3.23s/it]
 38%|███▊      | 85285/225000 [159:37:08<127:37:52,  3.29s/it]
 38%|███▊      | 85286/225000 [159:37:11<124:23:13,  3.21s/it]
 38%|███▊      | 85287/225000 [159:37:15<124:25:30,  3.21s/it]
 38%|███▊      | 85288/225000 [159:37:18<130:25:13,  3.36s/it]
 38%|███▊      | 85289/225000 [159:37:22<134:51:02,  3.47s/it]
 38%|███▊      | 85290/225000 [159:37:25<134:25:04,  3.46s/it]
 38%|███▊      | 85291/225000 [159:37:29<131:04:01,  3.38s/it]
 38%|███▊      | 85292/225000 [159:37:32<129:28:48,  3.34s/it]
 38%|███▊      | 85293/225000 [159:37:35<129:23:09,  3.33s/it]
 38%|███▊      | 85294/225000 [159:37:39<131:06:48,  3.38s/it]
 38%|███▊      | 85295/225000 [159:37:42<132:40:19,  3.42s/it]
 38%|███▊      | 85296/225000 [159:37:46<142:30:48,  3.67s/it]
 38%|███▊      | 85297/225000 [159:37:55<198:35:07,  5.12s/it]
 38%|███▊      | 85298/225000 [159:38:04<240:23:05,  6.19s/it]
 38%|███▊      | 85299/225000 [159:38:06<196:30:20,  5.06s/it]
 38%|███▊      | 85300/225000 [159:38:08<165:06:27,  4.25s/it]
                                                              
{'loss': '0.5379', 'grad_norm': '3.84', 'learning_rate': '3.635e-05', 'epoch': '0.3791', 'num_input_tokens_seen': 262057056, 'train_runtime': '5.747e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85300/225000 [159:38:08<165:06:27,  4.25s/it]
 38%|███▊      | 85301/225000 [159:38:12<153:40:24,  3.96s/it]
 38%|███▊      | 85302/225000 [159:38:15<144:40:53,  3.73s/it]
 38%|███▊      | 85303/225000 [159:38:18<140:09:04,  3.61s/it]
 38%|███▊      | 85304/225000 [159:38:22<137:54:48,  3.55s/it]
 38%|███▊      | 85305/225000 [159:38:25<136:40:11,  3.52s/it]
 38%|███▊      | 85306/225000 [159:38:29<136:36:34,  3.52s/it]
 38%|███▊      | 85307/225000 [159:38:34<161:53:34,  4.17s/it]
 38%|███▊      | 85308/225000 [159:38:37<150:20:23,  3.87s/it]
 38%|███▊      | 85309/225000 [159:38:41<142:54:17,  3.68s/it]
 38%|███▊      | 85310/225000 [159:38:44<139:26:40,  3.59s/it]
 38%|███▊      | 85311/225000 [159:38:50<164:19:04,  4.23s/it]
 38%|███▊      | 85312/225000 [159:38:53<152:49:22,  3.94s/it]
 38%|███▊      | 85313/225000 [159:38:57<151:06:37,  3.89s/it]
 38%|███▊      | 85314/225000 [159:39:01<150:11:40,  3.87s/it]
 38%|███▊      | 85315/225000 [159:39:04<142:57:40,  3.68s/it]
 38%|███▊      | 85316/225000 [159:39:07<141:33:38,  3.65s/it]
 38%|███▊      | 85317/225000 [159:39:11<137:25:20,  3.54s/it]
 38%|███▊      | 85318/225000 [159:39:14<137:30:21,  3.54s/it]
 38%|███▊      | 85319/225000 [159:39:18<133:51:47,  3.45s/it]
 38%|███▊      | 85320/225000 [159:39:21<134:46:14,  3.47s/it]
 38%|███▊      | 85321/225000 [159:39:25<135:35:35,  3.49s/it]
 38%|███▊      | 85322/225000 [159:39:32<184:35:28,  4.76s/it]
 38%|███▊      | 85323/225000 [159:39:41<234:17:23,  6.04s/it]
 38%|███▊      | 85324/225000 [159:39:44<190:30:27,  4.91s/it]
 38%|███▊      | 85325/225000 [159:39:46<162:17:04,  4.18s/it]
 38%|███▊      | 85326/225000 [159:39:49<144:12:13,  3.72s/it]
 38%|███▊      | 85327/225000 [159:39:51<131:23:09,  3.39s/it]
 38%|███▊      | 85328/225000 [159:39:54<125:32:05,  3.24s/it]
 38%|███▊      | 85329/225000 [159:40:05<207:38:25,  5.35s/it]
 38%|███▊      | 85330/225000 [159:40:09<192:06:25,  4.95s/it]
 38%|███▊      | 85331/225000 [159:40:11<162:27:40,  4.19s/it]
 38%|███▊      | 85332/225000 [159:40:15<155:48:33,  4.02s/it]
 38%|███▊      | 85333/225000 [159:40:18<150:49:59,  3.89s/it]
 38%|███▊      | 85334/225000 [159:40:22<149:16:21,  3.85s/it]
 38%|███▊      | 85335/225000 [159:40:26<146:21:18,  3.77s/it]
 38%|███▊      | 85336/225000 [159:40:29<143:49:36,  3.71s/it]
 38%|███▊      | 85337/225000 [159:40:33<146:02:58,  3.76s/it]
 38%|███▊      | 85338/225000 [159:40:37<144:09:14,  3.72s/it]
 38%|███▊      | 85339/225000 [159:40:40<142:30:12,  3.67s/it]
 38%|███▊      | 85340/225000 [159:40:44<140:07:05,  3.61s/it]
 38%|███▊      | 85341/225000 [159:40:47<140:12:59,  3.61s/it]
 38%|███▊      | 85342/225000 [159:40:51<137:01:44,  3.53s/it]
 38%|███▊      | 85343/225000 [159:40:54<137:44:43,  3.55s/it]
 38%|███▊      | 85344/225000 [159:40:58<138:25:59,  3.57s/it]
 38%|███▊      | 85345/225000 [159:41:01<139:57:33,  3.61s/it]
 38%|███▊      | 85346/225000 [159:41:05<139:41:14,  3.60s/it]
 38%|███▊      | 85347/225000 [159:41:09<145:20:47,  3.75s/it]
 38%|███▊      | 85348/225000 [159:41:13<143:34:30,  3.70s/it]
 38%|███▊      | 85349/225000 [159:41:16<140:16:33,  3.62s/it]
 38%|███▊      | 85350/225000 [159:41:20<137:12:38,  3.54s/it]
                                                              
{'loss': '0.5', 'grad_norm': '4.802', 'learning_rate': '3.633e-05', 'epoch': '0.3793', 'num_input_tokens_seen': 262142072, 'train_runtime': '5.749e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85350/225000 [159:41:20<137:12:38,  3.54s/it]
 38%|███▊      | 85351/225000 [159:41:23<135:12:47,  3.49s/it]
 38%|███▊      | 85352/225000 [159:41:26<133:47:04,  3.45s/it]
 38%|███▊      | 85353/225000 [159:41:35<190:20:48,  4.91s/it]
 38%|███▊      | 85354/225000 [159:41:44<237:57:36,  6.13s/it]
 38%|███▊      | 85355/225000 [159:41:46<192:31:07,  4.96s/it]
 38%|███▊      | 85356/225000 [159:41:52<211:26:16,  5.45s/it]
 38%|███▊      | 85357/225000 [159:41:56<186:09:02,  4.80s/it]
 38%|███▊      | 85358/225000 [159:41:59<168:02:39,  4.33s/it]
 38%|███▊      | 85359/225000 [159:42:02<152:39:14,  3.94s/it]
 38%|███▊      | 85360/225000 [159:42:05<142:26:02,  3.67s/it]
 38%|███▊      | 85361/225000 [159:42:09<143:05:14,  3.69s/it]
 38%|███▊      | 85362/225000 [159:42:12<139:41:09,  3.60s/it]
 38%|███▊      | 85363/225000 [159:42:18<162:02:43,  4.18s/it]
 38%|███▊      | 85364/225000 [159:42:27<224:19:35,  5.78s/it]
 38%|███▊      | 85365/225000 [159:42:31<206:26:17,  5.32s/it]
 38%|███▊      | 85366/225000 [159:42:34<179:13:30,  4.62s/it]
 38%|███▊      | 85367/225000 [159:42:37<157:57:49,  4.07s/it]
 38%|███▊      | 85368/225000 [159:42:40<143:30:16,  3.70s/it]
 38%|███▊      | 85369/225000 [159:42:43<135:26:00,  3.49s/it]
 38%|███▊      | 85370/225000 [159:42:46<127:44:04,  3.29s/it]
 38%|███▊      | 85371/225000 [159:42:49<121:57:19,  3.14s/it]
 38%|███▊      | 85372/225000 [159:42:52<122:31:38,  3.16s/it]
 38%|███▊      | 85373/225000 [159:42:55<122:17:32,  3.15s/it]
 38%|███▊      | 85374/225000 [159:42:58<124:16:20,  3.20s/it]
 38%|███▊      | 85375/225000 [159:43:04<157:29:16,  4.06s/it]
 38%|███▊      | 85376/225000 [159:43:14<221:08:16,  5.70s/it]
 38%|███▊      | 85377/225000 [159:43:18<206:38:22,  5.33s/it]
 38%|███▊      | 85378/225000 [159:43:21<171:46:39,  4.43s/it]
 38%|███▊      | 85379/225000 [159:43:23<150:07:04,  3.87s/it]
 38%|███▊      | 85380/225000 [159:43:28<165:44:10,  4.27s/it]
 38%|███▊      | 85381/225000 [159:43:34<179:41:29,  4.63s/it]
 38%|███▊      | 85382/225000 [159:43:37<161:07:25,  4.15s/it]
 38%|███▊      | 85383/225000 [159:43:40<145:33:56,  3.75s/it]
 38%|███▊      | 85384/225000 [159:43:42<131:53:50,  3.40s/it]
 38%|███▊      | 85385/225000 [159:43:47<143:34:19,  3.70s/it]
 38%|███▊      | 85386/225000 [159:43:54<183:01:12,  4.72s/it]
 38%|███▊      | 85387/225000 [159:43:57<161:46:43,  4.17s/it]
 38%|███▊      | 85388/225000 [159:44:00<147:31:45,  3.80s/it]
 38%|███▊      | 85389/225000 [159:44:03<141:08:03,  3.64s/it]
 38%|███▊      | 85390/225000 [159:44:06<132:50:57,  3.43s/it]
 38%|███▊      | 85391/225000 [159:44:09<128:32:40,  3.31s/it]
 38%|███▊      | 85392/225000 [159:44:12<125:42:30,  3.24s/it]
 38%|███▊      | 85393/225000 [159:44:15<123:08:50,  3.18s/it]
 38%|███▊      | 85394/225000 [159:44:24<187:05:01,  4.82s/it]
 38%|███▊      | 85395/225000 [159:44:30<203:31:21,  5.25s/it]
 38%|███▊      | 85396/225000 [159:44:32<169:02:13,  4.36s/it]
 38%|███▊      | 85397/225000 [159:44:35<151:21:12,  3.90s/it]
 38%|███▊      | 85398/225000 [159:44:43<202:59:21,  5.23s/it]
 38%|███▊      | 85399/225000 [159:44:46<174:20:36,  4.50s/it]
 38%|███▊      | 85400/225000 [159:44:49<155:21:18,  4.01s/it]
                                                              
{'loss': '0.5282', 'grad_norm': '5.14', 'learning_rate': '3.631e-05', 'epoch': '0.3796', 'num_input_tokens_seen': 262227808, 'train_runtime': '5.751e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85400/225000 [159:44:49<155:21:18,  4.01s/it]
 38%|███▊      | 85401/225000 [159:44:53<153:38:26,  3.96s/it]
 38%|███▊      | 85402/225000 [159:44:56<146:38:44,  3.78s/it]
 38%|███▊      | 85403/225000 [159:44:59<140:10:44,  3.62s/it]
 38%|███▊      | 85404/225000 [159:45:03<144:11:27,  3.72s/it]
 38%|███▊      | 85405/225000 [159:45:07<137:24:08,  3.54s/it]
 38%|███▊      | 85406/225000 [159:45:10<135:04:03,  3.48s/it]
 38%|███▊      | 85407/225000 [159:45:13<130:34:28,  3.37s/it]
 38%|███▊      | 85408/225000 [159:45:18<147:37:14,  3.81s/it]
 38%|███▊      | 85409/225000 [159:45:28<222:08:46,  5.73s/it]
 38%|███▊      | 85410/225000 [159:45:30<182:39:57,  4.71s/it]
 38%|███▊      | 85411/225000 [159:45:33<156:17:00,  4.03s/it]
 38%|███▊      | 85412/225000 [159:45:35<140:14:47,  3.62s/it]
 38%|███▊      | 85413/225000 [159:45:39<138:13:33,  3.56s/it]
 38%|███▊      | 85414/225000 [159:45:42<134:28:33,  3.47s/it]
 38%|███▊      | 85415/225000 [159:45:45<130:47:03,  3.37s/it]
 38%|███▊      | 85416/225000 [159:45:49<131:06:28,  3.38s/it]
 38%|███▊      | 85417/225000 [159:45:52<128:18:15,  3.31s/it]
 38%|███▊      | 85418/225000 [159:45:55<125:48:40,  3.24s/it]
 38%|███▊      | 85419/225000 [159:46:04<190:36:33,  4.92s/it]
 38%|███▊      | 85420/225000 [159:46:09<197:24:20,  5.09s/it]
 38%|███▊      | 85421/225000 [159:46:12<171:54:53,  4.43s/it]
 38%|███▊      | 85422/225000 [159:46:18<185:44:20,  4.79s/it]
 38%|███▊      | 85423/225000 [159:46:21<168:48:21,  4.35s/it]
 38%|███▊      | 85424/225000 [159:46:24<148:01:34,  3.82s/it]
 38%|███▊      | 85425/225000 [159:46:26<133:21:13,  3.44s/it]
 38%|███▊      | 85426/225000 [159:46:29<125:33:07,  3.24s/it]
 38%|███▊      | 85427/225000 [159:46:32<124:46:43,  3.22s/it]
 38%|███▊      | 85428/225000 [159:46:39<170:34:07,  4.40s/it]
 38%|███▊      | 85429/225000 [159:46:43<159:31:27,  4.11s/it]
 38%|███▊      | 85430/225000 [159:46:46<149:58:39,  3.87s/it]
 38%|███▊      | 85431/225000 [159:46:49<142:58:12,  3.69s/it]
 38%|███▊      | 85432/225000 [159:46:53<141:04:13,  3.64s/it]
 38%|███▊      | 85433/225000 [159:46:56<137:35:21,  3.55s/it]
 38%|███▊      | 85434/225000 [159:47:00<138:43:48,  3.58s/it]
 38%|███▊      | 85435/225000 [159:47:04<139:51:49,  3.61s/it]
 38%|███▊      | 85436/225000 [159:47:08<144:53:26,  3.74s/it]
 38%|███▊      | 85437/225000 [159:47:11<145:16:49,  3.75s/it]
 38%|███▊      | 85438/225000 [159:47:15<147:54:28,  3.82s/it]
 38%|███▊      | 85439/225000 [159:47:20<161:50:31,  4.17s/it]
 38%|███▊      | 85440/225000 [159:47:29<212:35:28,  5.48s/it]
 38%|███▊      | 85441/225000 [159:47:34<204:18:27,  5.27s/it]
 38%|███▊      | 85442/225000 [159:47:36<171:56:50,  4.44s/it]
 38%|███▊      | 85443/225000 [159:47:39<153:33:27,  3.96s/it]
 38%|███▊      | 85444/225000 [159:47:42<138:59:32,  3.59s/it]
 38%|███▊      | 85445/225000 [159:47:45<136:05:10,  3.51s/it]
 38%|███▊      | 85446/225000 [159:47:53<192:42:58,  4.97s/it]
 38%|███▊      | 85447/225000 [159:47:56<165:48:32,  4.28s/it]
 38%|███▊      | 85448/225000 [159:47:59<152:54:58,  3.94s/it]
 38%|███▊      | 85449/225000 [159:48:02<140:47:48,  3.63s/it]
 38%|███▊      | 85450/225000 [159:48:05<131:06:42,  3.38s/it]
                                                              
{'loss': '0.5556', 'grad_norm': '5.058', 'learning_rate': '3.63e-05', 'epoch': '0.3798', 'num_input_tokens_seen': 262313544, 'train_runtime': '5.753e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85450/225000 [159:48:05<131:06:42,  3.38s/it]
 38%|███▊      | 85451/225000 [159:48:08<124:50:36,  3.22s/it]
 38%|███▊      | 85452/225000 [159:48:15<172:55:05,  4.46s/it]
 38%|███▊      | 85453/225000 [159:48:21<192:27:09,  4.96s/it]
 38%|███▊      | 85454/225000 [159:48:24<167:21:47,  4.32s/it]
 38%|███▊      | 85455/225000 [159:48:27<150:01:14,  3.87s/it]
 38%|███▊      | 85456/225000 [159:48:30<142:24:17,  3.67s/it]
 38%|███▊      | 85457/225000 [159:48:33<130:20:10,  3.36s/it]
 38%|███▊      | 85458/225000 [159:48:38<155:22:55,  4.01s/it]
 38%|███▊      | 85459/225000 [159:48:46<202:36:09,  5.23s/it]
 38%|███▊      | 85460/225000 [159:48:49<173:12:15,  4.47s/it]
 38%|███▊      | 85461/225000 [159:48:52<151:42:15,  3.91s/it]
 38%|███▊      | 85462/225000 [159:48:54<136:44:06,  3.53s/it]
 38%|███▊      | 85463/225000 [159:48:57<129:21:26,  3.34s/it]
 38%|███▊      | 85464/225000 [159:49:01<129:52:50,  3.35s/it]
 38%|███▊      | 85465/225000 [159:49:09<189:07:24,  4.88s/it]
 38%|███▊      | 85466/225000 [159:49:12<167:43:24,  4.33s/it]
 38%|███▊      | 85467/225000 [159:49:14<144:34:11,  3.73s/it]
 38%|███▊      | 85468/225000 [159:49:18<139:14:56,  3.59s/it]
 38%|███▊      | 85469/225000 [159:49:21<135:23:59,  3.49s/it]
 38%|███▊      | 85470/225000 [159:49:24<132:43:16,  3.42s/it]
 38%|███▊      | 85471/225000 [159:49:28<132:15:42,  3.41s/it]
 38%|███▊      | 85472/225000 [159:49:33<159:39:29,  4.12s/it]
 38%|███▊      | 85473/225000 [159:49:37<157:27:27,  4.06s/it]
 38%|███▊      | 85474/225000 [159:49:40<147:14:20,  3.80s/it]
 38%|███▊      | 85475/225000 [159:49:44<139:05:08,  3.59s/it]
 38%|███▊      | 85476/225000 [159:49:47<138:26:33,  3.57s/it]
 38%|███▊      | 85477/225000 [159:49:50<133:42:03,  3.45s/it]
 38%|███▊      | 85478/225000 [159:49:54<136:16:08,  3.52s/it]
 38%|███▊      | 85479/225000 [159:49:57<136:25:33,  3.52s/it]
 38%|███▊      | 85480/225000 [159:50:01<134:20:26,  3.47s/it]
 38%|███▊      | 85481/225000 [159:50:05<141:37:13,  3.65s/it]
 38%|███▊      | 85482/225000 [159:50:09<141:38:43,  3.65s/it]
 38%|███▊      | 85483/225000 [159:50:12<141:16:50,  3.65s/it]
 38%|███▊      | 85484/225000 [159:50:19<172:34:56,  4.45s/it]
 38%|███▊      | 85485/225000 [159:50:27<219:42:44,  5.67s/it]
 38%|███▊      | 85486/225000 [159:50:32<212:21:22,  5.48s/it]
 38%|███▊      | 85487/225000 [159:50:35<181:46:08,  4.69s/it]
 38%|███▊      | 85488/225000 [159:50:38<163:05:17,  4.21s/it]
 38%|███▊      | 85489/225000 [159:50:41<151:41:24,  3.91s/it]
 38%|███▊      | 85490/225000 [159:50:44<141:06:18,  3.64s/it]
 38%|███▊      | 85491/225000 [159:50:47<135:07:44,  3.49s/it]
 38%|███▊      | 85492/225000 [159:50:50<130:01:32,  3.36s/it]
 38%|███▊      | 85493/225000 [159:50:54<128:21:16,  3.31s/it]
 38%|███▊      | 85494/225000 [159:50:58<139:51:32,  3.61s/it]
 38%|███▊      | 85495/225000 [159:51:07<204:03:35,  5.27s/it]
 38%|███▊      | 85496/225000 [159:51:14<219:05:38,  5.65s/it]
 38%|███▊      | 85497/225000 [159:51:16<180:03:22,  4.65s/it]
 38%|███▊      | 85498/225000 [159:51:19<158:33:01,  4.09s/it]
 38%|███▊      | 85499/225000 [159:51:23<155:59:40,  4.03s/it]
 38%|███▊      | 85500/225000 [159:51:26<146:10:24,  3.77s/it]
                                                              
{'loss': '0.5505', 'grad_norm': '5.423', 'learning_rate': '3.628e-05', 'epoch': '0.38', 'num_input_tokens_seen': 262398968, 'train_runtime': '5.755e+05', 'train_tokens_per_second': '456'}

 38%|███▊      | 85500/225000 [159:51:26<146:10:24,  3.77s/it]
 38%|███▊      | 85501/225000 [159:51:29<141:49:50,  3.66s/it]
 38%|███▊      | 85502/225000 [159:51:32<136:36:56,  3.53s/it]
 38%|███▊      | 85503/225000 [159:51:35<131:42:59,  3.40s/it]
 38%|███▊      | 85504/225000 [159:51:42<164:50:42,  4.25s/it]
 38%|███▊      | 85505/225000 [159:51:50<209:12:58,  5.40s/it]
 38%|███▊      | 85506/225000 [159:51:52<172:35:13,  4.45s/it]
 38%|███▊      | 85507/225000 [159:51:55<154:34:11,  3.99s/it]
 38%|███▊      | 85508/225000 [159:52:01<179:57:08,  4.64s/it]
 38%|███▊      | 85509/225000 [159:52:04<154:19:00,  3.98s/it]
 38%|███▊      | 85510/225000 [159:52:09<174:35:47,  4.51s/it]
 38%|███▊      | 85511/225000 [159:52:13<164:05:06,  4.23s/it]
 38%|███▊      | 85512/225000 [159:52:15<143:40:40,  3.71s/it]
 38%|███▊      | 85513/225000 [159:52:18<133:26:08,  3.44s/it]
 38%|███▊      | 85514/225000 [159:52:22<142:06:37,  3.67s/it]
 38%|███▊      | 85515/225000 [159:52:29<173:15:54,  4.47s/it]
 38%|███▊      | 85516/225000 [159:52:33<171:23:33,  4.42s/it]
 38%|███▊      | 85517/225000 [159:52:37<167:07:28,  4.31s/it]
 38%|███▊      | 85518/225000 [159:52:42<172:02:48,  4.44s/it]
 38%|███▊      | 85519/225000 [159:52:45<152:54:55,  3.95s/it]
 38%|███▊      | 85520/225000 [159:52:47<138:41:43,  3.58s/it]
 38%|███▊      | 85521/225000 [159:52:51<140:30:54,  3.63s/it]
 38%|███▊      | 85522/225000 [159:53:00<201:58:38,  5.21s/it]
 38%|███▊      | 85523/225000 [159:53:03<174:50:06,  4.51s/it]
 38%|███▊      | 85524/225000 [159:53:06<153:06:40,  3.95s/it]
 38%|███▊      | 85525/225000 [159:53:08<138:11:39,  3.57s/it]
 38%|███▊      | 85526/225000 [159:53:11<129:05:30,  3.33s/it]
 38%|███▊      | 85527/225000 [159:53:14<122:41:10,  3.17s/it]
 38%|███▊      | 85528/225000 [159:53:21<164:36:23,  4.25s/it]
 38%|███▊      | 85529/225000 [159:53:26<176:06:14,  4.55s/it]
 38%|███▊      | 85530/225000 [159:53:28<154:27:25,  3.99s/it]
 38%|███▊      | 85531/225000 [159:53:31<140:22:40,  3.62s/it]
 38%|███▊      | 85532/225000 [159:53:34<132:32:11,  3.42s/it]
 38%|███▊      | 85533/225000 [159:53:40<164:11:44,  4.24s/it]
 38%|███▊      | 85534/225000 [159:53:47<192:17:18,  4.96s/it]
 38%|███▊      | 85535/225000 [159:53:50<172:07:53,  4.44s/it]
 38%|███▊      | 85536/225000 [159:53:53<154:09:27,  3.98s/it]
 38%|███▊      | 85537/225000 [159:53:56<145:14:45,  3.75s/it]
 38%|███▊      | 85538/225000 [159:53:59<136:20:09,  3.52s/it]
 38%|███▊      | 85539/225000 [159:54:03<139:24:49,  3.60s/it]
 38%|███▊      | 85540/225000 [159:54:06<132:31:45,  3.42s/it]
 38%|███▊      | 85541/225000 [159:54:09<129:13:30,  3.34s/it]
 38%|███▊      | 85542/225000 [159:54:12<126:00:24,  3.25s/it]
 38%|███▊      | 85543/225000 [159:54:16<128:07:20,  3.31s/it]
 38%|███▊      | 85544/225000 [159:54:19<127:02:20,  3.28s/it]
 38%|███▊      | 85545/225000 [159:54:22<123:22:29,  3.18s/it]
 38%|███▊      | 85546/225000 [159:54:27<143:05:20,  3.69s/it]
 38%|███▊      | 85547/225000 [159:54:35<199:13:52,  5.14s/it]
 38%|███▊      | 85548/225000 [159:54:42<222:26:46,  5.74s/it]
 38%|███▊      | 85549/225000 [159:54:45<183:26:06,  4.74s/it]
 38%|███▊      | 85550/225000 [159:54:47<157:12:58,  4.06s/it]
                                                              
{'loss': '0.5518', 'grad_norm': '4.935', 'learning_rate': '3.627e-05', 'epoch': '0.3802', 'num_input_tokens_seen': 262485040, 'train_runtime': '5.757e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85550/225000 [159:54:47<157:12:58,  4.06s/it]
 38%|███▊      | 85551/225000 [159:54:50<142:06:33,  3.67s/it]
 38%|███▊      | 85552/225000 [159:54:57<184:43:54,  4.77s/it]
 38%|███▊      | 85553/225000 [159:55:01<175:14:26,  4.52s/it]
 38%|███▊      | 85554/225000 [159:55:04<150:32:15,  3.89s/it]
 38%|███▊      | 85555/225000 [159:55:06<132:19:15,  3.42s/it]
 38%|███▊      | 85556/225000 [159:55:10<134:19:45,  3.47s/it]
 38%|███▊      | 85557/225000 [159:55:15<155:25:27,  4.01s/it]
 38%|███▊      | 85558/225000 [159:55:21<179:31:28,  4.63s/it]
 38%|███▊      | 85559/225000 [159:55:23<153:42:57,  3.97s/it]
 38%|███▊      | 85560/225000 [159:55:26<141:13:04,  3.65s/it]
 38%|███▊      | 85561/225000 [159:55:29<126:39:18,  3.27s/it]
 38%|███▊      | 85562/225000 [159:55:32<122:23:43,  3.16s/it]
 38%|███▊      | 85563/225000 [159:55:35<120:15:31,  3.10s/it]
 38%|███▊      | 85564/225000 [159:55:39<132:52:17,  3.43s/it]
 38%|███▊      | 85565/225000 [159:55:47<190:56:53,  4.93s/it]
 38%|███▊      | 85566/225000 [159:55:53<195:54:16,  5.06s/it]
 38%|███▊      | 85567/225000 [159:55:55<163:38:58,  4.23s/it]
 38%|███▊      | 85568/225000 [159:55:57<144:32:22,  3.73s/it]
 38%|███▊      | 85569/225000 [159:56:02<150:12:26,  3.88s/it]
 38%|███▊      | 85570/225000 [159:56:08<178:49:19,  4.62s/it]
 38%|███▊      | 85571/225000 [159:56:11<156:00:23,  4.03s/it]
 38%|███▊      | 85572/225000 [159:56:18<191:55:50,  4.96s/it]
 38%|███▊      | 85573/225000 [159:56:20<163:14:32,  4.21s/it]
 38%|███▊      | 85574/225000 [159:56:23<142:27:15,  3.68s/it]
 38%|███▊      | 85575/225000 [159:56:27<147:08:32,  3.80s/it]
 38%|███▊      | 85576/225000 [159:56:32<160:29:37,  4.14s/it]
 38%|███▊      | 85577/225000 [159:56:35<148:16:43,  3.83s/it]
 38%|███▊      | 85578/225000 [159:56:38<135:59:52,  3.51s/it]
 38%|███▊      | 85579/225000 [159:56:41<130:23:14,  3.37s/it]
 38%|███▊      | 85580/225000 [159:56:44<126:21:48,  3.26s/it]
 38%|███▊      | 85581/225000 [159:56:46<121:19:10,  3.13s/it]
 38%|███▊      | 85582/225000 [159:56:50<122:03:16,  3.15s/it]
 38%|███▊      | 85583/225000 [159:56:52<117:13:45,  3.03s/it]
 38%|███▊      | 85584/225000 [159:56:58<145:05:19,  3.75s/it]
 38%|███▊      | 85585/225000 [159:57:07<206:52:08,  5.34s/it]
 38%|███▊      | 85586/225000 [159:57:10<178:39:04,  4.61s/it]
 38%|███▊      | 85587/225000 [159:57:13<161:04:15,  4.16s/it]
 38%|███▊      | 85588/225000 [159:57:16<145:06:38,  3.75s/it]
 38%|███▊      | 85589/225000 [159:57:18<133:32:25,  3.45s/it]
 38%|███▊      | 85590/225000 [159:57:26<185:05:32,  4.78s/it]
 38%|███▊      | 85591/225000 [159:57:30<172:05:56,  4.44s/it]
 38%|███▊      | 85592/225000 [159:57:33<156:07:22,  4.03s/it]
 38%|███▊      | 85593/225000 [159:57:36<142:59:39,  3.69s/it]
 38%|███▊      | 85594/225000 [159:57:39<134:13:03,  3.47s/it]
 38%|███▊      | 85595/225000 [159:57:42<129:18:01,  3.34s/it]
 38%|███▊      | 85596/225000 [159:57:49<173:28:07,  4.48s/it]
 38%|███▊      | 85597/225000 [159:57:52<159:21:55,  4.12s/it]
 38%|███▊      | 85598/225000 [159:57:56<149:25:39,  3.86s/it]
 38%|███▊      | 85599/225000 [159:57:59<140:14:12,  3.62s/it]
 38%|███▊      | 85600/225000 [159:58:02<132:46:45,  3.43s/it]
                                                              
{'loss': '0.5045', 'grad_norm': '4.758', 'learning_rate': '3.625e-05', 'epoch': '0.3804', 'num_input_tokens_seen': 262569192, 'train_runtime': '5.759e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85600/225000 [159:58:02<132:46:45,  3.43s/it]
 38%|███▊      | 85601/225000 [159:58:05<132:30:58,  3.42s/it]
 38%|███▊      | 85602/225000 [159:58:08<132:16:38,  3.42s/it]
 38%|███▊      | 85603/225000 [159:58:12<128:59:31,  3.33s/it]
 38%|███▊      | 85604/225000 [159:58:16<140:59:25,  3.64s/it]
 38%|███▊      | 85605/225000 [159:58:19<139:32:37,  3.60s/it]
 38%|███▊      | 85606/225000 [159:58:28<192:46:22,  4.98s/it]
 38%|███▊      | 85607/225000 [159:58:35<222:43:32,  5.75s/it]
 38%|███▊      | 85608/225000 [159:58:38<183:42:33,  4.74s/it]
 38%|███▊      | 85609/225000 [159:58:40<159:25:03,  4.12s/it]
 38%|███▊      | 85610/225000 [159:58:43<142:53:04,  3.69s/it]
 38%|███▊      | 85611/225000 [159:58:46<130:57:14,  3.38s/it]
 38%|███▊      | 85612/225000 [159:58:49<126:01:29,  3.25s/it]
 38%|███▊      | 85613/225000 [159:58:51<121:05:43,  3.13s/it]
 38%|███▊      | 85614/225000 [159:58:59<173:54:29,  4.49s/it]
 38%|███▊      | 85615/225000 [159:59:06<201:01:08,  5.19s/it]
 38%|███▊      | 85616/225000 [159:59:09<173:40:27,  4.49s/it]
 38%|███▊      | 85617/225000 [159:59:11<149:05:26,  3.85s/it]
 38%|███▊      | 85618/225000 [159:59:14<135:03:03,  3.49s/it]
 38%|███▊      | 85619/225000 [159:59:17<128:22:44,  3.32s/it]
 38%|███▊      | 85620/225000 [159:59:20<131:17:02,  3.39s/it]
 38%|███▊      | 85621/225000 [159:59:23<127:49:05,  3.30s/it]
 38%|███▊      | 85622/225000 [159:59:28<145:03:45,  3.75s/it]
 38%|███▊      | 85623/225000 [159:59:37<199:38:13,  5.16s/it]
 38%|███▊      | 85624/225000 [159:59:40<177:36:39,  4.59s/it]
 38%|███▊      | 85625/225000 [159:59:43<159:11:24,  4.11s/it]
 38%|███▊      | 85626/225000 [159:59:46<144:20:24,  3.73s/it]
 38%|███▊      | 85627/225000 [159:59:49<137:36:59,  3.55s/it]
 38%|███▊      | 85628/225000 [159:59:52<135:36:48,  3.50s/it]
 38%|███▊      | 85629/225000 [159:59:56<139:20:40,  3.60s/it]
 38%|███▊      | 85630/225000 [159:59:59<133:42:14,  3.45s/it]
 38%|███▊      | 85631/225000 [160:00:03<135:04:02,  3.49s/it]
 38%|███▊      | 85632/225000 [160:00:07<147:33:34,  3.81s/it]
 38%|███▊      | 85633/225000 [160:00:16<207:52:07,  5.37s/it]
 38%|███▊      | 85634/225000 [160:00:21<198:19:09,  5.12s/it]
 38%|███▊      | 85635/225000 [160:00:23<165:26:28,  4.27s/it]
 38%|███▊      | 85636/225000 [160:00:29<179:29:05,  4.64s/it]
 38%|███▊      | 85637/225000 [160:00:32<160:00:01,  4.13s/it]
 38%|███▊      | 85638/225000 [160:00:35<148:01:03,  3.82s/it]
 38%|███▊      | 85639/225000 [160:00:38<137:22:36,  3.55s/it]
 38%|███▊      | 85640/225000 [160:00:41<130:32:46,  3.37s/it]
 38%|███▊      | 85641/225000 [160:00:44<129:22:54,  3.34s/it]
 38%|███▊      | 85642/225000 [160:00:47<123:46:40,  3.20s/it]
 38%|███▊      | 85643/225000 [160:00:54<166:18:45,  4.30s/it]
 38%|███▊      | 85644/225000 [160:01:00<187:48:43,  4.85s/it]
 38%|███▊      | 85645/225000 [160:01:02<159:00:04,  4.11s/it]
 38%|███▊      | 85646/225000 [160:01:05<140:35:54,  3.63s/it]
 38%|███▊      | 85647/225000 [160:01:08<137:10:21,  3.54s/it]
 38%|███▊      | 85648/225000 [160:01:15<182:24:06,  4.71s/it]
 38%|███▊      | 85649/225000 [160:01:18<162:18:09,  4.19s/it]
 38%|███▊      | 85650/225000 [160:01:22<151:08:11,  3.90s/it]
                                                              
{'loss': '0.5396', 'grad_norm': '3.778', 'learning_rate': '3.623e-05', 'epoch': '0.3807', 'num_input_tokens_seen': 262655792, 'train_runtime': '5.761e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85650/225000 [160:01:22<151:08:11,  3.90s/it]
 38%|███▊      | 85651/225000 [160:01:24<139:16:20,  3.60s/it]
 38%|███▊      | 85652/225000 [160:01:27<129:53:37,  3.36s/it]
 38%|███▊      | 85653/225000 [160:01:30<121:58:36,  3.15s/it]
 38%|███▊      | 85654/225000 [160:01:33<121:01:42,  3.13s/it]
 38%|███▊      | 85655/225000 [160:01:39<159:50:45,  4.13s/it]
 38%|███▊      | 85656/225000 [160:01:46<183:47:15,  4.75s/it]
 38%|███▊      | 85657/225000 [160:01:48<155:51:35,  4.03s/it]
 38%|███▊      | 85658/225000 [160:01:55<187:10:17,  4.84s/it]
 38%|███▊      | 85659/225000 [160:01:58<174:23:43,  4.51s/it]
 38%|███▊      | 85660/225000 [160:02:02<157:41:21,  4.07s/it]
 38%|███▊      | 85661/225000 [160:02:04<141:05:42,  3.65s/it]
 38%|███▊      | 85662/225000 [160:02:07<131:29:21,  3.40s/it]
 38%|███▊      | 85663/225000 [160:02:10<128:29:06,  3.32s/it]
 38%|███▊      | 85664/225000 [160:02:15<143:26:07,  3.71s/it]
 38%|███▊      | 85665/225000 [160:02:24<204:17:59,  5.28s/it]
 38%|███▊      | 85666/225000 [160:02:27<182:53:04,  4.73s/it]
 38%|███▊      | 85667/225000 [160:02:30<161:43:32,  4.18s/it]
 38%|███▊      | 85668/225000 [160:02:33<150:53:32,  3.90s/it]
 38%|███▊      | 85669/225000 [160:02:37<144:59:40,  3.75s/it]
 38%|███▊      | 85670/225000 [160:02:40<143:35:19,  3.71s/it]
 38%|███▊      | 85671/225000 [160:02:48<186:24:41,  4.82s/it]
 38%|███▊      | 85672/225000 [160:02:50<156:57:35,  4.06s/it]
 38%|███▊      | 85673/225000 [160:02:56<174:54:46,  4.52s/it]
 38%|███▊      | 85674/225000 [160:03:01<188:35:55,  4.87s/it]
 38%|███▊      | 85675/225000 [160:03:04<158:18:04,  4.09s/it]
 38%|███▊      | 85676/225000 [160:03:07<145:40:25,  3.76s/it]
 38%|███▊      | 85677/225000 [160:03:12<166:10:03,  4.29s/it]
 38%|███▊      | 85678/225000 [160:03:15<147:35:42,  3.81s/it]
 38%|███▊      | 85679/225000 [160:03:17<132:14:04,  3.42s/it]
 38%|███▊      | 85680/225000 [160:03:20<121:42:52,  3.15s/it]
 38%|███▊      | 85681/225000 [160:03:23<122:10:09,  3.16s/it]
 38%|███▊      | 85682/225000 [160:03:27<138:01:59,  3.57s/it]
 38%|███▊      | 85683/225000 [160:03:34<177:28:25,  4.59s/it]
 38%|███▊      | 85684/225000 [160:03:37<158:06:04,  4.09s/it]
 38%|███▊      | 85685/225000 [160:03:40<146:48:26,  3.79s/it]
 38%|███▊      | 85686/225000 [160:03:44<145:07:44,  3.75s/it]
 38%|███▊      | 85687/225000 [160:03:47<138:47:17,  3.59s/it]
 38%|███▊      | 85688/225000 [160:03:51<142:33:12,  3.68s/it]
 38%|███▊      | 85689/225000 [160:03:55<140:14:07,  3.62s/it]
 38%|███▊      | 85690/225000 [160:03:59<143:03:23,  3.70s/it]
 38%|███▊      | 85691/225000 [160:04:02<142:23:11,  3.68s/it]
 38%|███▊      | 85692/225000 [160:04:06<142:17:06,  3.68s/it]
 38%|███▊      | 85693/225000 [160:04:10<143:27:31,  3.71s/it]
 38%|███▊      | 85694/225000 [160:04:13<144:00:55,  3.72s/it]
 38%|███▊      | 85695/225000 [160:04:17<140:56:19,  3.64s/it]
 38%|███▊      | 85696/225000 [160:04:21<146:36:44,  3.79s/it]
 38%|███▊      | 85697/225000 [160:04:31<214:42:31,  5.55s/it]
 38%|███▊      | 85698/225000 [160:04:35<204:07:43,  5.28s/it]
 38%|███▊      | 85699/225000 [160:04:38<171:42:03,  4.44s/it]
 38%|███▊      | 85700/225000 [160:04:40<149:22:02,  3.86s/it]
                                                              
{'loss': '0.492', 'grad_norm': '5.1', 'learning_rate': '3.622e-05', 'epoch': '0.3809', 'num_input_tokens_seen': 262741016, 'train_runtime': '5.763e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85700/225000 [160:04:40<149:22:02,  3.86s/it]
 38%|███▊      | 85701/225000 [160:04:49<201:54:34,  5.22s/it]
 38%|███▊      | 85702/225000 [160:04:52<174:47:23,  4.52s/it]
 38%|███▊      | 85703/225000 [160:04:54<154:02:44,  3.98s/it]
 38%|███▊      | 85704/225000 [160:04:57<137:14:29,  3.55s/it]
 38%|███▊      | 85705/225000 [160:04:59<126:12:23,  3.26s/it]
 38%|███▊      | 85706/225000 [160:05:03<124:58:31,  3.23s/it]
 38%|███▊      | 85707/225000 [160:05:06<125:34:32,  3.25s/it]
 38%|███▊      | 85708/225000 [160:05:14<178:33:56,  4.62s/it]
 38%|███▊      | 85709/225000 [160:05:17<167:50:59,  4.34s/it]
 38%|███▊      | 85710/225000 [160:05:21<158:40:51,  4.10s/it]
 38%|███▊      | 85711/225000 [160:05:24<150:11:59,  3.88s/it]
 38%|███▊      | 85712/225000 [160:05:28<145:24:26,  3.76s/it]
 38%|███▊      | 85713/225000 [160:05:31<140:22:28,  3.63s/it]
 38%|███▊      | 85714/225000 [160:05:35<147:45:24,  3.82s/it]
 38%|███▊      | 85715/225000 [160:05:39<147:47:01,  3.82s/it]
 38%|███▊      | 85716/225000 [160:05:43<148:56:57,  3.85s/it]
 38%|███▊      | 85717/225000 [160:05:46<142:37:20,  3.69s/it]
 38%|███▊      | 85718/225000 [160:05:50<141:55:17,  3.67s/it]
 38%|███▊      | 85719/225000 [160:05:54<141:28:18,  3.66s/it]
 38%|███▊      | 85720/225000 [160:05:57<141:03:05,  3.65s/it]
 38%|███▊      | 85721/225000 [160:06:01<142:42:14,  3.69s/it]
 38%|███▊      | 85722/225000 [160:06:04<135:00:01,  3.49s/it]
 38%|███▊      | 85723/225000 [160:06:07<131:32:02,  3.40s/it]
 38%|███▊      | 85724/225000 [160:06:11<130:32:10,  3.37s/it]
 38%|███▊      | 85725/225000 [160:06:14<134:39:02,  3.48s/it]
 38%|███▊      | 85726/225000 [160:06:23<190:16:12,  4.92s/it]
 38%|███▊      | 85727/225000 [160:06:30<215:05:49,  5.56s/it]
 38%|███▊      | 85728/225000 [160:06:32<178:46:47,  4.62s/it]
 38%|███▊      | 85729/225000 [160:06:34<152:40:32,  3.95s/it]
 38%|███▊      | 85730/225000 [160:06:37<139:33:49,  3.61s/it]
 38%|███▊      | 85731/225000 [160:06:43<166:07:55,  4.29s/it]
 38%|███▊      | 85732/225000 [160:06:49<189:42:28,  4.90s/it]
 38%|███▊      | 85733/225000 [160:06:52<164:16:26,  4.25s/it]
 38%|███▊      | 85734/225000 [160:06:55<145:22:13,  3.76s/it]
 38%|███▊      | 85735/225000 [160:06:58<137:51:31,  3.56s/it]
 38%|███▊      | 85736/225000 [160:07:06<185:39:38,  4.80s/it]
 38%|███▊      | 85737/225000 [160:07:08<157:18:42,  4.07s/it]
 38%|███▊      | 85738/225000 [160:07:14<176:04:36,  4.55s/it]
 38%|███▊      | 85739/225000 [160:07:16<151:45:38,  3.92s/it]
 38%|███▊      | 85740/225000 [160:07:21<162:08:58,  4.19s/it]
 38%|███▊      | 85741/225000 [160:07:26<168:21:09,  4.35s/it]
 38%|███▊      | 85742/225000 [160:07:28<145:04:21,  3.75s/it]
 38%|███▊      | 85743/225000 [160:07:31<141:03:27,  3.65s/it]
 38%|███▊      | 85744/225000 [160:07:35<140:44:42,  3.64s/it]
 38%|███▊      | 85745/225000 [160:07:38<137:55:42,  3.57s/it]
 38%|███▊      | 85746/225000 [160:07:42<135:49:33,  3.51s/it]
 38%|███▊      | 85747/225000 [160:07:45<137:33:59,  3.56s/it]
 38%|███▊      | 85748/225000 [160:07:51<158:44:12,  4.10s/it]
 38%|███▊      | 85749/225000 [160:07:56<167:23:33,  4.33s/it]
 38%|███▊      | 85750/225000 [160:08:00<163:02:35,  4.22s/it]
                                                              
{'loss': '0.5043', 'grad_norm': '4.612', 'learning_rate': '3.62e-05', 'epoch': '0.3811', 'num_input_tokens_seen': 262826704, 'train_runtime': '5.765e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85750/225000 [160:08:00<163:02:35,  4.22s/it]
 38%|███▊      | 85751/225000 [160:08:05<173:30:58,  4.49s/it]
 38%|███▊      | 85752/225000 [160:08:11<190:18:16,  4.92s/it]
 38%|███▊      | 85753/225000 [160:08:13<165:07:36,  4.27s/it]
 38%|███▊      | 85754/225000 [160:08:16<145:55:59,  3.77s/it]
 38%|███▊      | 85755/225000 [160:08:19<137:27:50,  3.55s/it]
 38%|███▊      | 85756/225000 [160:08:23<140:51:04,  3.64s/it]
 38%|███▊      | 85757/225000 [160:08:27<140:19:28,  3.63s/it]
 38%|███▊      | 85758/225000 [160:08:30<141:03:42,  3.65s/it]
 38%|███▊      | 85759/225000 [160:08:34<137:08:15,  3.55s/it]
 38%|███▊      | 85760/225000 [160:08:37<138:15:42,  3.57s/it]
 38%|███▊      | 85761/225000 [160:08:41<138:37:47,  3.58s/it]
 38%|███▊      | 85762/225000 [160:08:44<139:19:01,  3.60s/it]
 38%|███▊      | 85763/225000 [160:08:48<139:44:14,  3.61s/it]
 38%|███▊      | 85764/225000 [160:08:52<138:57:06,  3.59s/it]
 38%|███▊      | 85765/225000 [160:08:55<140:14:37,  3.63s/it]
 38%|███▊      | 85766/225000 [160:08:59<137:09:12,  3.55s/it]
 38%|███▊      | 85767/225000 [160:09:02<134:25:43,  3.48s/it]
 38%|███▊      | 85768/225000 [160:09:10<189:51:27,  4.91s/it]
 38%|███▊      | 85769/225000 [160:09:14<175:43:54,  4.54s/it]
 38%|███▊      | 85770/225000 [160:09:17<161:39:02,  4.18s/it]
 38%|███▊      | 85771/225000 [160:09:21<151:49:49,  3.93s/it]
 38%|███▊      | 85772/225000 [160:09:24<148:39:22,  3.84s/it]
 38%|███▊      | 85773/225000 [160:09:28<142:59:59,  3.70s/it]
 38%|███▊      | 85774/225000 [160:09:31<143:28:14,  3.71s/it]
 38%|███▊      | 85775/225000 [160:09:35<142:02:48,  3.67s/it]
 38%|███▊      | 85776/225000 [160:09:38<137:59:49,  3.57s/it]
 38%|███▊      | 85777/225000 [160:09:42<142:22:28,  3.68s/it]
 38%|███▊      | 85778/225000 [160:09:46<141:36:00,  3.66s/it]
 38%|███▊      | 85779/225000 [160:09:52<169:28:18,  4.38s/it]
 38%|███▊      | 85780/225000 [160:09:59<198:15:02,  5.13s/it]
 38%|███▊      | 85781/225000 [160:10:03<191:00:16,  4.94s/it]
 38%|███▊      | 85782/225000 [160:10:07<175:02:03,  4.53s/it]
 38%|███▊      | 85783/225000 [160:10:11<168:37:09,  4.36s/it]
 38%|███▊      | 85784/225000 [160:10:14<156:28:38,  4.05s/it]
 38%|███▊      | 85785/225000 [160:10:18<150:25:03,  3.89s/it]
 38%|███▊      | 85786/225000 [160:10:21<143:35:24,  3.71s/it]
 38%|███▊      | 85787/225000 [160:10:25<142:32:02,  3.69s/it]
 38%|███▊      | 85788/225000 [160:10:28<145:14:14,  3.76s/it]
 38%|███▊      | 85789/225000 [160:10:32<141:27:39,  3.66s/it]
 38%|███▊      | 85790/225000 [160:10:36<141:25:53,  3.66s/it]
 38%|███▊      | 85791/225000 [160:10:40<151:08:50,  3.91s/it]
 38%|███▊      | 85792/225000 [160:10:43<144:49:21,  3.75s/it]
 38%|███▊      | 85793/225000 [160:10:47<140:21:10,  3.63s/it]
 38%|███▊      | 85794/225000 [160:10:50<137:32:51,  3.56s/it]
 38%|███▊      | 85795/225000 [160:10:54<139:12:57,  3.60s/it]
 38%|███▊      | 85796/225000 [160:10:57<134:54:33,  3.49s/it]
 38%|███▊      | 85797/225000 [160:11:02<151:40:14,  3.92s/it]
 38%|███▊      | 85798/225000 [160:11:05<144:55:49,  3.75s/it]
 38%|███▊      | 85799/225000 [160:11:09<143:58:25,  3.72s/it]
 38%|███▊      | 85800/225000 [160:11:13<146:06:17,  3.78s/it]
                                                              
{'loss': '0.552', 'grad_norm': '3.873', 'learning_rate': '3.618e-05', 'epoch': '0.3813', 'num_input_tokens_seen': 262911856, 'train_runtime': '5.767e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85800/225000 [160:11:13<146:06:17,  3.78s/it]
 38%|███▊      | 85801/225000 [160:11:17<148:42:54,  3.85s/it]
 38%|███▊      | 85802/225000 [160:11:20<143:04:59,  3.70s/it]
 38%|███▊      | 85803/225000 [160:11:25<149:23:12,  3.86s/it]
 38%|███▊      | 85804/225000 [160:11:28<143:40:39,  3.72s/it]
 38%|███▊      | 85805/225000 [160:11:35<181:32:44,  4.70s/it]
 38%|███▊      | 85806/225000 [160:11:43<225:54:42,  5.84s/it]
 38%|███▊      | 85807/225000 [160:11:46<188:25:41,  4.87s/it]
 38%|███▊      | 85808/225000 [160:11:49<169:42:33,  4.39s/it]
 38%|███▊      | 85809/225000 [160:11:52<149:52:42,  3.88s/it]
 38%|███▊      | 85810/225000 [160:11:55<136:36:31,  3.53s/it]
 38%|███▊      | 85811/225000 [160:11:58<129:11:08,  3.34s/it]
 38%|███▊      | 85812/225000 [160:12:02<140:20:57,  3.63s/it]
 38%|███▊      | 85813/225000 [160:12:06<141:51:15,  3.67s/it]
 38%|███▊      | 85814/225000 [160:12:09<140:46:45,  3.64s/it]
 38%|███▊      | 85815/225000 [160:12:13<142:01:19,  3.67s/it]
 38%|███▊      | 85816/225000 [160:12:16<138:03:57,  3.57s/it]
 38%|███▊      | 85817/225000 [160:12:20<137:19:37,  3.55s/it]
 38%|███▊      | 85818/225000 [160:12:24<142:16:07,  3.68s/it]
 38%|███▊      | 85819/225000 [160:12:27<139:23:02,  3.61s/it]
 38%|███▊      | 85820/225000 [160:12:36<204:36:00,  5.29s/it]
 38%|███▊      | 85821/225000 [160:12:41<197:18:01,  5.10s/it]
 38%|███▊      | 85822/225000 [160:12:44<167:25:50,  4.33s/it]
 38%|███▊      | 85823/225000 [160:12:52<215:51:36,  5.58s/it]
 38%|███▊      | 85824/225000 [160:12:54<177:16:21,  4.59s/it]
 38%|███▊      | 85825/225000 [160:12:57<153:33:58,  3.97s/it]
 38%|███▊      | 85826/225000 [160:13:03<176:59:45,  4.58s/it]
 38%|███▊      | 85827/225000 [160:13:05<152:19:35,  3.94s/it]
 38%|███▊      | 85828/225000 [160:13:08<140:03:53,  3.62s/it]
 38%|███▊      | 85829/225000 [160:13:11<130:40:40,  3.38s/it]
 38%|███▊      | 85830/225000 [160:13:19<184:21:57,  4.77s/it]
 38%|███▊      | 85831/225000 [160:13:23<169:44:58,  4.39s/it]
 38%|███▊      | 85832/225000 [160:13:26<152:51:42,  3.95s/it]
 38%|███▊      | 85833/225000 [160:13:29<150:46:30,  3.90s/it]
 38%|███▊      | 85834/225000 [160:13:32<142:22:15,  3.68s/it]
 38%|███▊      | 85835/225000 [160:13:36<138:31:17,  3.58s/it]
 38%|███▊      | 85836/225000 [160:13:39<135:43:15,  3.51s/it]
 38%|███▊      | 85837/225000 [160:13:42<131:23:52,  3.40s/it]
 38%|███▊      | 85838/225000 [160:13:45<128:09:29,  3.32s/it]
 38%|███▊      | 85839/225000 [160:13:48<124:25:59,  3.22s/it]
 38%|███▊      | 85840/225000 [160:13:56<169:44:37,  4.39s/it]
 38%|███▊      | 85841/225000 [160:14:03<203:23:31,  5.26s/it]
 38%|███▊      | 85842/225000 [160:14:05<168:41:43,  4.36s/it]
 38%|███▊      | 85843/225000 [160:14:08<146:07:51,  3.78s/it]
 38%|███▊      | 85844/225000 [160:14:10<132:23:39,  3.43s/it]
 38%|███▊      | 85845/225000 [160:14:14<136:00:25,  3.52s/it]
 38%|███▊      | 85846/225000 [160:14:22<189:44:56,  4.91s/it]
 38%|███▊      | 85847/225000 [160:14:25<164:16:14,  4.25s/it]
 38%|███▊      | 85848/225000 [160:14:27<146:23:35,  3.79s/it]
 38%|███▊      | 85849/225000 [160:14:30<134:02:23,  3.47s/it]
 38%|███▊      | 85850/225000 [160:14:33<128:26:09,  3.32s/it]
                                                              
{'loss': '0.4955', 'grad_norm': '5.286', 'learning_rate': '3.617e-05', 'epoch': '0.3816', 'num_input_tokens_seen': 262996488, 'train_runtime': '5.769e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85850/225000 [160:14:33<128:26:09,  3.32s/it]
 38%|███▊      | 85851/225000 [160:14:36<123:46:27,  3.20s/it]
 38%|███▊      | 85852/225000 [160:14:40<136:36:27,  3.53s/it]
 38%|███▊      | 85853/225000 [160:14:50<208:09:49,  5.39s/it]
 38%|███▊      | 85854/225000 [160:14:54<186:18:39,  4.82s/it]
 38%|███▊      | 85855/225000 [160:14:56<160:33:44,  4.15s/it]
 38%|███▊      | 85856/225000 [160:14:59<144:17:37,  3.73s/it]
 38%|███▊      | 85857/225000 [160:15:02<131:24:04,  3.40s/it]
 38%|███▊      | 85858/225000 [160:15:04<124:25:45,  3.22s/it]
 38%|███▊      | 85859/225000 [160:15:11<166:49:58,  4.32s/it]
 38%|███▊      | 85860/225000 [160:15:15<158:47:24,  4.11s/it]
 38%|███▊      | 85861/225000 [160:15:19<157:50:07,  4.08s/it]
 38%|███▊      | 85862/225000 [160:15:22<148:46:59,  3.85s/it]
 38%|███▊      | 85863/225000 [160:15:26<144:40:22,  3.74s/it]
 38%|███▊      | 85864/225000 [160:15:29<144:53:35,  3.75s/it]
 38%|███▊      | 85865/225000 [160:15:33<145:47:41,  3.77s/it]
 38%|███▊      | 85866/225000 [160:15:36<138:09:13,  3.57s/it]
 38%|███▊      | 85867/225000 [160:15:40<137:43:11,  3.56s/it]
 38%|███▊      | 85868/225000 [160:15:43<132:43:11,  3.43s/it]
 38%|███▊      | 85869/225000 [160:15:53<202:45:47,  5.25s/it]
 38%|███▊      | 85870/225000 [160:15:57<194:20:16,  5.03s/it]
 38%|███▊      | 85871/225000 [160:16:00<173:43:00,  4.49s/it]
 38%|███▊      | 85872/225000 [160:16:03<151:01:18,  3.91s/it]
 38%|███▊      | 85873/225000 [160:16:06<139:09:08,  3.60s/it]
 38%|███▊      | 85874/225000 [160:16:08<129:03:51,  3.34s/it]
 38%|███▊      | 85875/225000 [160:16:12<129:50:41,  3.36s/it]
 38%|███▊      | 85876/225000 [160:16:15<132:08:24,  3.42s/it]
 38%|███▊      | 85877/225000 [160:16:18<128:05:09,  3.31s/it]
 38%|███▊      | 85878/225000 [160:16:25<164:41:41,  4.26s/it]
 38%|███▊      | 85879/225000 [160:16:34<215:09:08,  5.57s/it]
 38%|███▊      | 85880/225000 [160:16:36<177:30:40,  4.59s/it]
 38%|███▊      | 85881/225000 [160:16:38<151:01:30,  3.91s/it]
 38%|███▊      | 85882/225000 [160:16:41<142:26:12,  3.69s/it]
 38%|███▊      | 85883/225000 [160:16:49<188:10:16,  4.87s/it]
 38%|███▊      | 85884/225000 [160:16:52<166:55:40,  4.32s/it]
 38%|███▊      | 85885/225000 [160:16:55<150:46:06,  3.90s/it]
 38%|███▊      | 85886/225000 [160:16:58<142:20:27,  3.68s/it]
 38%|███▊      | 85887/225000 [160:17:01<132:42:14,  3.43s/it]
 38%|███▊      | 85888/225000 [160:17:04<131:46:56,  3.41s/it]
 38%|███▊      | 85889/225000 [160:17:08<128:55:34,  3.34s/it]
 38%|███▊      | 85890/225000 [160:17:10<123:09:26,  3.19s/it]
 38%|███▊      | 85891/225000 [160:17:13<120:10:11,  3.11s/it]
 38%|███▊      | 85892/225000 [160:17:17<127:49:02,  3.31s/it]
 38%|███▊      | 85893/225000 [160:17:24<168:17:54,  4.36s/it]
 38%|███▊      | 85894/225000 [160:17:33<221:59:54,  5.75s/it]
 38%|███▊      | 85895/225000 [160:17:37<200:11:12,  5.18s/it]
 38%|███▊      | 85896/225000 [160:17:39<167:59:04,  4.35s/it]
 38%|███▊      | 85897/225000 [160:17:42<146:05:19,  3.78s/it]
 38%|███▊      | 85898/225000 [160:17:48<181:04:34,  4.69s/it]
 38%|███▊      | 85899/225000 [160:17:53<186:05:38,  4.82s/it]
 38%|███▊      | 85900/225000 [160:17:56<157:36:17,  4.08s/it]
                                                              
{'loss': '0.5264', 'grad_norm': '5.426', 'learning_rate': '3.615e-05', 'epoch': '0.3818', 'num_input_tokens_seen': 263081360, 'train_runtime': '5.771e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85900/225000 [160:17:56<157:36:17,  4.08s/it]
 38%|███▊      | 85901/225000 [160:17:58<140:09:16,  3.63s/it]
 38%|███▊      | 85902/225000 [160:18:02<134:58:57,  3.49s/it]
 38%|███▊      | 85903/225000 [160:18:05<129:46:28,  3.36s/it]
 38%|███▊      | 85904/225000 [160:18:08<128:09:16,  3.32s/it]
 38%|███▊      | 85905/225000 [160:18:14<160:12:00,  4.15s/it]
 38%|███▊      | 85906/225000 [160:18:21<199:34:54,  5.17s/it]
 38%|███▊      | 85907/225000 [160:18:24<167:36:36,  4.34s/it]
 38%|███▊      | 85908/225000 [160:18:27<148:31:10,  3.84s/it]
 38%|███▊      | 85909/225000 [160:18:29<133:54:30,  3.47s/it]
 38%|███▊      | 85910/225000 [160:18:33<132:43:36,  3.44s/it]
 38%|███▊      | 85911/225000 [160:18:36<134:02:42,  3.47s/it]
 38%|███▊      | 85912/225000 [160:18:41<150:27:13,  3.89s/it]
 38%|███▊      | 85913/225000 [160:18:46<166:28:42,  4.31s/it]
 38%|███▊      | 85914/225000 [160:18:50<158:58:58,  4.11s/it]
 38%|███▊      | 85915/225000 [160:18:57<195:33:28,  5.06s/it]
 38%|███▊      | 85916/225000 [160:19:00<170:56:12,  4.42s/it]
 38%|███▊      | 85917/225000 [160:19:03<157:22:34,  4.07s/it]
 38%|███▊      | 85918/225000 [160:19:06<146:08:06,  3.78s/it]
 38%|███▊      | 85919/225000 [160:19:09<136:16:14,  3.53s/it]
 38%|███▊      | 85920/225000 [160:19:12<126:22:52,  3.27s/it]
 38%|███▊      | 85921/225000 [160:19:15<127:31:15,  3.30s/it]
 38%|███▊      | 85922/225000 [160:19:23<174:23:11,  4.51s/it]
 38%|███▊      | 85923/225000 [160:19:28<181:45:43,  4.70s/it]
 38%|███▊      | 85924/225000 [160:19:34<196:46:23,  5.09s/it]
 38%|███▊      | 85925/225000 [160:19:37<174:20:58,  4.51s/it]
 38%|███▊      | 85926/225000 [160:19:40<150:03:15,  3.88s/it]
 38%|███▊      | 85927/225000 [160:19:46<181:41:25,  4.70s/it]
 38%|███▊      | 85928/225000 [160:19:50<166:31:09,  4.31s/it]
 38%|███▊      | 85929/225000 [160:19:52<146:52:06,  3.80s/it]
 38%|███▊      | 85930/225000 [160:19:55<135:37:06,  3.51s/it]
 38%|███▊      | 85931/225000 [160:19:58<127:38:20,  3.30s/it]
 38%|███▊      | 85932/225000 [160:20:01<122:28:51,  3.17s/it]
 38%|███▊      | 85933/225000 [160:20:05<138:00:13,  3.57s/it]
 38%|███▊      | 85934/225000 [160:20:13<193:20:46,  5.01s/it]
 38%|███▊      | 85935/225000 [160:20:16<161:51:32,  4.19s/it]
 38%|███▊      | 85936/225000 [160:20:18<139:08:30,  3.60s/it]
 38%|███▊      | 85937/225000 [160:20:20<125:21:01,  3.25s/it]
 38%|███▊      | 85938/225000 [160:20:26<154:48:55,  4.01s/it]
 38%|███▊      | 85939/225000 [160:20:29<142:58:42,  3.70s/it]
 38%|███▊      | 85940/225000 [160:20:32<135:22:53,  3.50s/it]
 38%|███▊      | 85941/225000 [160:20:36<134:35:17,  3.48s/it]
 38%|███▊      | 85942/225000 [160:20:39<127:56:48,  3.31s/it]
 38%|███▊      | 85943/225000 [160:20:42<128:14:49,  3.32s/it]
 38%|███▊      | 85944/225000 [160:20:45<123:59:40,  3.21s/it]
 38%|███▊      | 85945/225000 [160:20:48<126:34:52,  3.28s/it]
 38%|███▊      | 85946/225000 [160:20:52<127:19:30,  3.30s/it]
 38%|███▊      | 85947/225000 [160:20:55<124:42:53,  3.23s/it]
 38%|███▊      | 85948/225000 [160:20:58<128:47:21,  3.33s/it]
 38%|███▊      | 85949/225000 [160:21:04<160:37:54,  4.16s/it]
 38%|███▊      | 85950/225000 [160:21:13<210:36:34,  5.45s/it]
                                                              
{'loss': '0.4895', 'grad_norm': '5.507', 'learning_rate': '3.613e-05', 'epoch': '0.382', 'num_input_tokens_seen': 263166048, 'train_runtime': '5.773e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 85950/225000 [160:21:13<210:36:34,  5.45s/it]
 38%|███▊      | 85951/225000 [160:21:17<194:50:12,  5.04s/it]
 38%|███▊      | 85952/225000 [160:21:20<173:03:13,  4.48s/it]
 38%|███▊      | 85953/225000 [160:21:23<158:47:06,  4.11s/it]
 38%|███▊      | 85954/225000 [160:21:27<151:21:26,  3.92s/it]
 38%|███▊      | 85955/225000 [160:21:30<146:31:33,  3.79s/it]
 38%|███▊      | 85956/225000 [160:21:34<143:07:44,  3.71s/it]
 38%|███▊      | 85957/225000 [160:21:37<136:56:08,  3.55s/it]
 38%|███▊      | 85958/225000 [160:21:40<131:41:51,  3.41s/it]
 38%|███▊      | 85959/225000 [160:21:44<134:51:18,  3.49s/it]
 38%|███▊      | 85960/225000 [160:21:47<133:02:33,  3.44s/it]
 38%|███▊      | 85961/225000 [160:21:51<132:26:22,  3.43s/it]
 38%|███▊      | 85962/225000 [160:21:54<131:14:49,  3.40s/it]
 38%|███▊      | 85963/225000 [160:21:57<131:13:30,  3.40s/it]
 38%|███▊      | 85964/225000 [160:22:01<132:24:47,  3.43s/it]
 38%|███▊      | 85965/225000 [160:22:05<137:25:59,  3.56s/it]
 38%|███▊      | 85966/225000 [160:22:13<190:33:20,  4.93s/it]
 38%|███▊      | 85967/225000 [160:22:20<222:48:37,  5.77s/it]
 38%|███▊      | 85968/225000 [160:22:23<182:03:39,  4.71s/it]
 38%|███▊      | 85969/225000 [160:22:26<171:05:02,  4.43s/it]
 38%|███▊      | 85970/225000 [160:22:33<190:46:21,  4.94s/it]
 38%|███▊      | 85971/225000 [160:22:36<178:20:04,  4.62s/it]
 38%|███▊      | 85972/225000 [160:22:40<165:19:05,  4.28s/it]
 38%|███▊      | 85973/225000 [160:22:43<155:09:19,  4.02s/it]
 38%|███▊      | 85974/225000 [160:22:47<147:06:41,  3.81s/it]
 38%|███▊      | 85975/225000 [160:22:50<142:14:01,  3.68s/it]
 38%|███▊      | 85976/225000 [160:22:53<138:10:32,  3.58s/it]
 38%|███▊      | 85977/225000 [160:22:57<138:07:49,  3.58s/it]
 38%|███▊      | 85978/225000 [160:23:01<141:59:42,  3.68s/it]
 38%|███▊      | 85979/225000 [160:23:05<141:54:06,  3.67s/it]
 38%|███▊      | 85980/225000 [160:23:08<140:14:47,  3.63s/it]
 38%|███▊      | 85981/225000 [160:23:11<135:42:22,  3.51s/it]
 38%|███▊      | 85982/225000 [160:23:15<133:40:04,  3.46s/it]
 38%|███▊      | 85983/225000 [160:23:18<136:20:23,  3.53s/it]
 38%|███▊      | 85984/225000 [160:23:22<138:13:38,  3.58s/it]
 38%|███▊      | 85985/225000 [160:23:25<135:45:55,  3.52s/it]
 38%|███▊      | 85986/225000 [160:23:29<140:36:26,  3.64s/it]
 38%|███▊      | 85987/225000 [160:23:38<194:14:32,  5.03s/it]
 38%|███▊      | 85988/225000 [160:23:43<201:52:48,  5.23s/it]
 38%|███▊      | 85989/225000 [160:23:46<175:02:33,  4.53s/it]
 38%|███▊      | 85990/225000 [160:23:50<160:43:34,  4.16s/it]
 38%|███▊      | 85991/225000 [160:23:52<145:11:12,  3.76s/it]
 38%|███▊      | 85992/225000 [160:23:55<136:16:50,  3.53s/it]
 38%|███▊      | 85993/225000 [160:23:58<130:18:59,  3.37s/it]
 38%|███▊      | 85994/225000 [160:24:06<183:51:07,  4.76s/it]
 38%|███▊      | 85995/225000 [160:24:11<182:17:00,  4.72s/it]
 38%|███▊      | 85996/225000 [160:24:14<162:06:15,  4.20s/it]
 38%|███▊      | 85997/225000 [160:24:17<151:29:29,  3.92s/it]
 38%|███▊      | 85998/225000 [160:24:21<145:31:59,  3.77s/it]
 38%|███▊      | 85999/225000 [160:24:24<144:28:25,  3.74s/it]
 38%|███▊      | 86000/225000 [160:24:28<140:14:57,  3.63s/it]
                                                              
{'loss': '0.5219', 'grad_norm': '3.674', 'learning_rate': '3.612e-05', 'epoch': '0.3822', 'num_input_tokens_seen': 263251008, 'train_runtime': '5.775e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 86000/225000 [160:24:28<140:14:57,  3.63s/it]
 38%|███▊      | 86001/225000 [160:24:31<136:18:19,  3.53s/it]
 38%|███▊      | 86002/225000 [160:24:35<138:11:07,  3.58s/it]
 38%|███▊      | 86003/225000 [160:24:38<140:35:56,  3.64s/it]
 38%|███▊      | 86004/225000 [160:24:42<138:02:19,  3.58s/it]
 38%|███▊      | 86005/225000 [160:24:46<140:54:09,  3.65s/it]
 38%|███▊      | 86006/225000 [160:24:49<139:32:13,  3.61s/it]
 38%|███▊      | 86007/225000 [160:24:53<138:52:46,  3.60s/it]
 38%|███▊      | 86008/225000 [160:24:56<138:18:59,  3.58s/it]
 38%|███▊      | 86009/225000 [160:25:00<141:40:26,  3.67s/it]
 38%|███▊      | 86010/225000 [160:25:04<140:40:05,  3.64s/it]
 38%|███▊      | 86011/225000 [160:25:08<141:43:52,  3.67s/it]
 38%|███▊      | 86012/225000 [160:25:11<144:29:47,  3.74s/it]
 38%|███▊      | 86013/225000 [160:25:15<139:51:00,  3.62s/it]
 38%|███▊      | 86014/225000 [160:25:20<157:24:39,  4.08s/it]
 38%|███▊      | 86015/225000 [160:25:28<204:59:12,  5.31s/it]
 38%|███▊      | 86016/225000 [160:25:34<213:56:32,  5.54s/it]
 38%|███▊      | 86017/225000 [160:25:38<189:50:00,  4.92s/it]
 38%|███▊      | 86018/225000 [160:25:41<169:58:27,  4.40s/it]
 38%|███▊      | 86019/225000 [160:25:44<158:41:31,  4.11s/it]
 38%|███▊      | 86020/225000 [160:25:52<196:53:05,  5.10s/it]
 38%|███▊      | 86021/225000 [160:25:54<169:25:37,  4.39s/it]
 38%|███▊      | 86022/225000 [160:25:57<152:53:14,  3.96s/it]
 38%|███▊      | 86023/225000 [160:26:00<141:36:29,  3.67s/it]
 38%|███▊      | 86024/225000 [160:26:03<134:15:07,  3.48s/it]
 38%|███▊      | 86025/225000 [160:26:06<127:56:25,  3.31s/it]
 38%|███▊      | 86026/225000 [160:26:09<124:29:21,  3.22s/it]
 38%|███▊      | 86027/225000 [160:26:12<119:48:50,  3.10s/it]
 38%|███▊      | 86028/225000 [160:26:18<147:17:34,  3.82s/it]
 38%|███▊      | 86029/225000 [160:26:27<207:51:18,  5.38s/it]
 38%|███▊      | 86030/225000 [160:26:30<180:20:30,  4.67s/it]
 38%|███▊      | 86031/225000 [160:26:33<158:25:08,  4.10s/it]
 38%|███▊      | 86032/225000 [160:26:36<150:16:40,  3.89s/it]
 38%|███▊      | 86033/225000 [160:26:41<163:42:52,  4.24s/it]
 38%|███▊      | 86034/225000 [160:26:45<164:49:21,  4.27s/it]
 38%|███▊      | 86035/225000 [160:26:48<151:57:38,  3.94s/it]
 38%|███▊      | 86036/225000 [160:26:52<142:15:11,  3.69s/it]
 38%|███▊      | 86037/225000 [160:26:54<132:37:40,  3.44s/it]
 38%|███▊      | 86038/225000 [160:26:58<134:05:54,  3.47s/it]
 38%|███▊      | 86039/225000 [160:27:01<131:25:08,  3.40s/it]
 38%|███▊      | 86040/225000 [160:27:04<125:06:42,  3.24s/it]
 38%|███▊      | 86041/225000 [160:27:12<181:16:15,  4.70s/it]
 38%|███▊      | 86042/225000 [160:27:18<198:57:45,  5.15s/it]
 38%|███▊      | 86043/225000 [160:27:21<168:53:38,  4.38s/it]
 38%|███▊      | 86044/225000 [160:27:24<149:52:47,  3.88s/it]
 38%|███▊      | 86045/225000 [160:27:26<137:06:39,  3.55s/it]
 38%|███▊      | 86046/225000 [160:27:29<129:27:10,  3.35s/it]
 38%|███▊      | 86047/225000 [160:27:32<126:56:01,  3.29s/it]
 38%|███▊      | 86048/225000 [160:27:36<129:53:11,  3.37s/it]
 38%|███▊      | 86049/225000 [160:27:40<141:50:11,  3.67s/it]
 38%|███▊      | 86050/225000 [160:27:49<198:41:39,  5.15s/it]
                                                              
{'loss': '0.4839', 'grad_norm': '4.63', 'learning_rate': '3.61e-05', 'epoch': '0.3824', 'num_input_tokens_seen': 263336584, 'train_runtime': '5.777e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 86050/225000 [160:27:49<198:41:39,  5.15s/it]
 38%|███▊      | 86051/225000 [160:27:53<182:40:33,  4.73s/it]
 38%|███▊      | 86052/225000 [160:27:56<162:21:26,  4.21s/it]
 38%|███▊      | 86053/225000 [160:28:00<157:45:45,  4.09s/it]
 38%|███▊      | 86054/225000 [160:28:03<147:39:56,  3.83s/it]
 38%|███▊      | 86055/225000 [160:28:06<136:45:00,  3.54s/it]
 38%|███▊      | 86056/225000 [160:28:09<130:48:38,  3.39s/it]
 38%|███▊      | 86057/225000 [160:28:12<128:50:56,  3.34s/it]
 38%|███▊      | 86058/225000 [160:28:17<147:31:53,  3.82s/it]
 38%|███▊      | 86059/225000 [160:28:25<202:00:04,  5.23s/it]
 38%|███▊      | 86060/225000 [160:28:30<195:09:42,  5.06s/it]
 38%|███▊      | 86061/225000 [160:28:32<164:23:27,  4.26s/it]
 38%|███▊      | 86062/225000 [160:28:37<172:19:04,  4.46s/it]
 38%|███▊      | 86063/225000 [160:28:43<183:25:48,  4.75s/it]
 38%|███▊      | 86064/225000 [160:28:45<155:47:20,  4.04s/it]
 38%|███▊      | 86065/225000 [160:28:48<138:23:05,  3.59s/it]
 38%|███▊      | 86066/225000 [160:28:51<129:08:29,  3.35s/it]
 38%|███▊      | 86067/225000 [160:28:53<123:29:54,  3.20s/it]
 38%|███▊      | 86068/225000 [160:28:56<118:28:36,  3.07s/it]
 38%|███▊      | 86069/225000 [160:29:04<171:49:30,  4.45s/it]
 38%|███▊      | 86070/225000 [160:29:11<200:09:55,  5.19s/it]
 38%|███▊      | 86071/225000 [160:29:13<167:08:33,  4.33s/it]
 38%|███▊      | 86072/225000 [160:29:16<146:58:52,  3.81s/it]
 38%|███▊      | 86073/225000 [160:29:22<175:01:01,  4.54s/it]
 38%|███▊      | 86074/225000 [160:29:25<159:42:09,  4.14s/it]
 38%|███▊      | 86075/225000 [160:29:28<145:25:36,  3.77s/it]
 38%|███▊      | 86076/225000 [160:29:31<138:41:42,  3.59s/it]
 38%|███▊      | 86077/225000 [160:29:35<136:03:22,  3.53s/it]
 38%|███▊      | 86078/225000 [160:29:38<132:26:46,  3.43s/it]
 38%|███▊      | 86079/225000 [160:29:41<126:54:05,  3.29s/it]
 38%|███▊      | 86080/225000 [160:29:47<164:36:09,  4.27s/it]
 38%|███▊      | 86081/225000 [160:29:56<219:30:19,  5.69s/it]
 38%|███▊      | 86082/225000 [160:29:59<182:12:29,  4.72s/it]
 38%|███▊      | 86083/225000 [160:30:02<160:44:06,  4.17s/it]
 38%|███▊      | 86084/225000 [160:30:04<145:51:22,  3.78s/it]
 38%|███▊      | 86085/225000 [160:30:07<133:23:32,  3.46s/it]
 38%|███▊      | 86086/225000 [160:30:11<137:48:13,  3.57s/it]
 38%|███▊      | 86087/225000 [160:30:18<182:33:38,  4.73s/it]
 38%|███▊      | 86088/225000 [160:30:21<157:26:01,  4.08s/it]
 38%|███▊      | 86089/225000 [160:30:23<137:20:04,  3.56s/it]
 38%|███▊      | 86090/225000 [160:30:26<126:10:21,  3.27s/it]
 38%|███▊      | 86091/225000 [160:30:34<178:51:33,  4.64s/it]
 38%|███▊      | 86092/225000 [160:30:39<180:29:55,  4.68s/it]
 38%|███▊      | 86093/225000 [160:30:42<167:04:50,  4.33s/it]
 38%|███▊      | 86094/225000 [160:30:45<155:53:14,  4.04s/it]
 38%|███▊      | 86095/225000 [160:30:49<146:02:59,  3.79s/it]
 38%|███▊      | 86096/225000 [160:30:52<142:52:39,  3.70s/it]
 38%|███▊      | 86097/225000 [160:30:55<138:45:33,  3.60s/it]
 38%|███▊      | 86098/225000 [160:30:59<138:03:34,  3.58s/it]
 38%|███▊      | 86099/225000 [160:31:05<161:13:23,  4.18s/it]
 38%|███▊      | 86100/225000 [160:31:08<149:49:50,  3.88s/it]
                                                              
{'loss': '0.4927', 'grad_norm': '6.883', 'learning_rate': '3.609e-05', 'epoch': '0.3827', 'num_input_tokens_seen': 263422024, 'train_runtime': '5.779e+05', 'train_tokens_per_second': '455.9'}

 38%|███▊      | 86100/225000 [160:31:08<149:49:50,  3.88s/it]
 38%|███▊      | 86101/225000 [160:31:11<146:16:20,  3.79s/it]
 38%|███▊      | 86102/225000 [160:31:15<139:40:21,  3.62s/it]
 38%|███▊      | 86103/225000 [160:31:18<135:51:50,  3.52s/it]
 38%|███▊      | 86104/225000 [160:31:23<151:37:23,  3.93s/it]
 38%|███▊      | 86105/225000 [160:31:26<143:40:03,  3.72s/it]
 38%|███▊      | 86106/225000 [160:31:29<140:45:38,  3.65s/it]
 38%|███▊      | 86107/225000 [160:31:33<142:19:53,  3.69s/it]
 38%|███▊      | 86108/225000 [160:31:37<139:01:53,  3.60s/it]
 38%|███▊      | 86109/225000 [160:31:40<134:18:56,  3.48s/it]
 38%|███▊      | 86110/225000 [160:31:43<131:45:03,  3.41s/it]
 38%|███▊      | 86111/225000 [160:31:48<145:52:23,  3.78s/it]
 38%|███▊      | 86112/225000 [160:31:57<207:04:44,  5.37s/it]
 38%|███▊      | 86113/225000 [160:32:03<212:48:13,  5.52s/it]
 38%|███▊      | 86114/225000 [160:32:05<178:44:43,  4.63s/it]
 38%|███▊      | 86115/225000 [160:32:08<158:26:13,  4.11s/it]
 38%|███▊      | 86116/225000 [160:32:12<151:13:13,  3.92s/it]
 38%|███▊      | 86117/225000 [160:32:15<143:12:56,  3.71s/it]
 38%|███▊      | 86118/225000 [160:32:21<167:13:35,  4.33s/it]
 38%|███▊      | 86119/225000 [160:32:24<155:57:18,  4.04s/it]
 38%|███▊      | 86120/225000 [160:32:27<144:48:55,  3.75s/it]
 38%|███▊      | 86121/225000 [160:32:30<136:56:42,  3.55s/it]
 38%|███▊      | 86122/225000 [160:32:33<130:29:02,  3.38s/it]
 38%|███▊      | 86123/225000 [160:32:37<131:49:39,  3.42s/it]
 38%|███▊      | 86124/225000 [160:32:40<132:18:04,  3.43s/it]
 38%|███▊      | 86125/225000 [160:32:48<189:43:37,  4.92s/it]
 38%|███▊      | 86126/225000 [160:32:54<195:57:35,  5.08s/it]
 38%|███▊      | 86127/225000 [160:32:56<166:13:44,  4.31s/it]
 38%|███▊      | 86128/225000 [160:33:02<185:07:47,  4.80s/it]
 38%|███▊      | 86129/225000 [160:33:08<192:02:05,  4.98s/it]
 38%|███▊      | 86130/225000 [160:33:11<165:47:54,  4.30s/it]
 38%|███▊      | 86131/225000 [160:33:13<149:54:31,  3.89s/it]
 38%|███▊      | 86132/225000 [160:33:16<135:38:03,  3.52s/it]
 38%|███▊      | 86133/225000 [160:33:19<124:52:25,  3.24s/it]
 38%|███▊      | 86134/225000 [160:33:26<174:43:59,  4.53s/it]
 38%|███▊      | 86135/225000 [160:33:30<163:06:24,  4.23s/it]
 38%|███▊      | 86136/225000 [160:33:33<146:41:31,  3.80s/it]
 38%|███▊      | 86137/225000 [160:33:35<130:39:23,  3.39s/it]
 38%|███▊      | 86138/225000 [160:33:39<137:02:19,  3.55s/it]
 38%|███▊      | 86139/225000 [160:33:42<135:02:06,  3.50s/it]
 38%|███▊      | 86140/225000 [160:33:45<131:41:26,  3.41s/it]
 38%|███▊      | 86141/225000 [160:33:49<129:42:11,  3.36s/it]
 38%|███▊      | 86142/225000 [160:33:54<156:32:29,  4.06s/it]
 38%|███▊      | 86143/225000 [160:33:58<154:08:42,  4.00s/it]
 38%|███▊      | 86144/225000 [160:34:02<145:31:25,  3.77s/it]
 38%|███▊      | 86145/225000 [160:34:05<142:45:31,  3.70s/it]
 38%|███▊      | 86146/225000 [160:34:13<194:09:45,  5.03s/it]
 38%|███▊      | 86147/225000 [160:34:17<176:40:57,  4.58s/it]
 38%|███▊      | 86148/225000 [160:34:20<160:24:33,  4.16s/it]
 38%|███▊      | 86149/225000 [160:34:23<147:59:29,  3.84s/it]
 38%|███▊      | 86150/225000 [160:34:26<143:23:59,  3.72s/it]
                                                              
{'loss': '0.4771', 'grad_norm': '4.396', 'learning_rate': '3.607e-05', 'epoch': '0.3829', 'num_input_tokens_seen': 263508544, 'train_runtime': '5.781e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86150/225000 [160:34:26<143:23:59,  3.72s/it]
 38%|███▊      | 86151/225000 [160:34:30<136:59:26,  3.55s/it]
 38%|███▊      | 86152/225000 [160:34:33<139:41:51,  3.62s/it]
 38%|███▊      | 86153/225000 [160:34:37<135:09:19,  3.50s/it]
 38%|███▊      | 86154/225000 [160:34:40<132:21:03,  3.43s/it]
 38%|███▊      | 86155/225000 [160:34:44<136:13:09,  3.53s/it]
 38%|███▊      | 86156/225000 [160:34:47<131:43:18,  3.42s/it]
 38%|███▊      | 86157/225000 [160:34:50<130:18:14,  3.38s/it]
 38%|███▊      | 86158/225000 [160:34:54<131:00:15,  3.40s/it]
 38%|███▊      | 86159/225000 [160:35:01<174:34:30,  4.53s/it]
 38%|███▊      | 86160/225000 [160:35:10<226:37:52,  5.88s/it]
 38%|███▊      | 86161/225000 [160:35:12<187:14:03,  4.85s/it]
 38%|███▊      | 86162/225000 [160:35:15<158:53:05,  4.12s/it]
 38%|███▊      | 86163/225000 [160:35:17<140:40:39,  3.65s/it]
 38%|███▊      | 86164/225000 [160:35:21<146:57:25,  3.81s/it]
 38%|███▊      | 86165/225000 [160:35:30<200:47:36,  5.21s/it]
 38%|███▊      | 86166/225000 [160:35:33<172:08:40,  4.46s/it]
 38%|███▊      | 86167/225000 [160:35:35<149:16:22,  3.87s/it]
 38%|███▊      | 86168/225000 [160:35:41<174:44:18,  4.53s/it]
 38%|███▊      | 86169/225000 [160:35:44<150:45:19,  3.91s/it]
 38%|███▊      | 86170/225000 [160:35:47<145:21:25,  3.77s/it]
 38%|███▊      | 86171/225000 [160:35:51<143:22:13,  3.72s/it]
 38%|███▊      | 86172/225000 [160:35:54<141:39:46,  3.67s/it]
 38%|███▊      | 86173/225000 [160:36:03<197:11:02,  5.11s/it]
 38%|███▊      | 86174/225000 [160:36:06<175:20:57,  4.55s/it]
 38%|███▊      | 86175/225000 [160:36:09<155:05:09,  4.02s/it]
 38%|███▊      | 86176/225000 [160:36:11<141:21:27,  3.67s/it]
 38%|███▊      | 86177/225000 [160:36:15<134:08:44,  3.48s/it]
 38%|███▊      | 86178/225000 [160:36:17<125:53:05,  3.26s/it]
 38%|███▊      | 86179/225000 [160:36:20<120:43:19,  3.13s/it]
 38%|███▊      | 86180/225000 [160:36:26<152:45:13,  3.96s/it]
 38%|███▊      | 86181/225000 [160:36:34<194:07:23,  5.03s/it]
 38%|███▊      | 86182/225000 [160:36:36<162:28:07,  4.21s/it]
 38%|███▊      | 86183/225000 [160:36:38<142:15:10,  3.69s/it]
 38%|███▊      | 86184/225000 [160:36:41<129:23:37,  3.36s/it]
 38%|███▊      | 86185/225000 [160:36:48<178:19:16,  4.62s/it]
 38%|███▊      | 86186/225000 [160:36:53<179:09:43,  4.65s/it]
 38%|███▊      | 86187/225000 [160:36:56<153:02:36,  3.97s/it]
 38%|███▊      | 86188/225000 [160:36:58<137:01:12,  3.55s/it]
 38%|███▊      | 86189/225000 [160:37:02<136:50:54,  3.55s/it]
 38%|███▊      | 86190/225000 [160:37:09<182:11:51,  4.73s/it]
 38%|███▊      | 86191/225000 [160:37:12<159:39:37,  4.14s/it]
 38%|███▊      | 86192/225000 [160:37:15<148:59:15,  3.86s/it]
 38%|███▊      | 86193/225000 [160:37:18<136:49:05,  3.55s/it]
 38%|███▊      | 86194/225000 [160:37:21<126:35:17,  3.28s/it]
 38%|███▊      | 86195/225000 [160:37:23<119:23:09,  3.10s/it]
 38%|███▊      | 86196/225000 [160:37:26<116:42:16,  3.03s/it]
 38%|███▊      | 86197/225000 [160:37:35<180:20:31,  4.68s/it]
 38%|███▊      | 86198/225000 [160:37:38<170:38:08,  4.43s/it]
 38%|███▊      | 86199/225000 [160:37:41<150:48:41,  3.91s/it]
 38%|███▊      | 86200/225000 [160:37:49<196:15:01,  5.09s/it]
                                                              
{'loss': '0.5003', 'grad_norm': '4.742', 'learning_rate': '3.605e-05', 'epoch': '0.3831', 'num_input_tokens_seen': 263593944, 'train_runtime': '5.783e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86200/225000 [160:37:49<196:15:01,  5.09s/it]
 38%|███▊      | 86201/225000 [160:37:52<170:16:16,  4.42s/it]
 38%|███▊      | 86202/225000 [160:37:55<153:11:56,  3.97s/it]
 38%|███▊      | 86203/225000 [160:37:58<139:49:34,  3.63s/it]
 38%|███▊      | 86204/225000 [160:38:00<129:22:54,  3.36s/it]
 38%|███▊      | 86205/225000 [160:38:04<128:51:23,  3.34s/it]
 38%|███▊      | 86206/225000 [160:38:07<130:07:58,  3.38s/it]
 38%|███▊      | 86207/225000 [160:38:11<140:31:04,  3.64s/it]
 38%|███▊      | 86208/225000 [160:38:15<145:34:27,  3.78s/it]
 38%|███▊      | 86209/225000 [160:38:19<142:54:39,  3.71s/it]
 38%|███▊      | 86210/225000 [160:38:23<140:16:48,  3.64s/it]
 38%|███▊      | 86211/225000 [160:38:29<178:55:07,  4.64s/it]
 38%|███▊      | 86212/225000 [160:38:33<168:49:25,  4.38s/it]
 38%|███▊      | 86213/225000 [160:38:37<158:56:57,  4.12s/it]
 38%|███▊      | 86214/225000 [160:38:41<155:15:10,  4.03s/it]
 38%|███▊      | 86215/225000 [160:38:47<186:17:10,  4.83s/it]
 38%|███▊      | 86216/225000 [160:38:52<185:48:07,  4.82s/it]
 38%|███▊      | 86217/225000 [160:38:55<163:51:55,  4.25s/it]
 38%|███▊      | 86218/225000 [160:38:58<150:28:40,  3.90s/it]
 38%|███▊      | 86219/225000 [160:39:01<137:47:46,  3.57s/it]
 38%|███▊      | 86220/225000 [160:39:04<132:02:51,  3.43s/it]
 38%|███▊      | 86221/225000 [160:39:12<180:00:39,  4.67s/it]
 38%|███▊      | 86222/225000 [160:39:18<201:33:14,  5.23s/it]
 38%|███▊      | 86223/225000 [160:39:20<168:20:50,  4.37s/it]
 38%|███▊      | 86224/225000 [160:39:24<155:34:20,  4.04s/it]
 38%|███▊      | 86225/225000 [160:39:27<149:23:46,  3.88s/it]
 38%|███▊      | 86226/225000 [160:39:30<141:33:33,  3.67s/it]
 38%|███▊      | 86227/225000 [160:39:33<134:11:39,  3.48s/it]
 38%|███▊      | 86228/225000 [160:39:37<133:44:46,  3.47s/it]
 38%|███▊      | 86229/225000 [160:39:40<134:30:13,  3.49s/it]
 38%|███▊      | 86230/225000 [160:39:44<136:38:23,  3.54s/it]
 38%|███▊      | 86231/225000 [160:39:47<134:20:02,  3.48s/it]
 38%|███▊      | 86232/225000 [160:39:51<131:23:38,  3.41s/it]
 38%|███▊      | 86233/225000 [160:39:54<131:53:14,  3.42s/it]
 38%|███▊      | 86234/225000 [160:39:58<136:19:52,  3.54s/it]
 38%|███▊      | 86235/225000 [160:40:02<136:52:42,  3.55s/it]
 38%|███▊      | 86236/225000 [160:40:05<138:58:07,  3.61s/it]
 38%|███▊      | 86237/225000 [160:40:11<164:25:16,  4.27s/it]
 38%|███▊      | 86238/225000 [160:40:20<220:27:55,  5.72s/it]
 38%|███▊      | 86239/225000 [160:40:24<197:12:10,  5.12s/it]
 38%|███▊      | 86240/225000 [160:40:27<171:44:01,  4.46s/it]
 38%|███▊      | 86241/225000 [160:40:30<157:13:26,  4.08s/it]
 38%|███▊      | 86242/225000 [160:40:33<148:53:35,  3.86s/it]
 38%|███▊      | 86243/225000 [160:40:36<136:45:58,  3.55s/it]
 38%|███▊      | 86244/225000 [160:40:39<133:10:42,  3.46s/it]
 38%|███▊      | 86245/225000 [160:40:43<135:00:07,  3.50s/it]
 38%|███▊      | 86246/225000 [160:40:51<188:21:03,  4.89s/it]
 38%|███▊      | 86247/225000 [160:40:56<184:50:20,  4.80s/it]
 38%|███▊      | 86248/225000 [160:40:58<155:42:08,  4.04s/it]
 38%|███▊      | 86249/225000 [160:41:00<137:35:59,  3.57s/it]
 38%|███▊      | 86250/225000 [160:41:04<131:31:31,  3.41s/it]
                                                              
{'loss': '0.4837', 'grad_norm': '3.523', 'learning_rate': '3.604e-05', 'epoch': '0.3833', 'num_input_tokens_seen': 263679264, 'train_runtime': '5.785e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86250/225000 [160:41:04<131:31:31,  3.41s/it]
 38%|███▊      | 86251/225000 [160:41:07<131:59:37,  3.42s/it]
 38%|███▊      | 86252/225000 [160:41:11<133:35:14,  3.47s/it]
 38%|███▊      | 86253/225000 [160:41:14<131:58:24,  3.42s/it]
 38%|███▊      | 86254/225000 [160:41:17<129:53:01,  3.37s/it]
 38%|███▊      | 86255/225000 [160:41:21<135:18:21,  3.51s/it]
 38%|███▊      | 86256/225000 [160:41:25<139:06:05,  3.61s/it]
 38%|███▊      | 86257/225000 [160:41:29<145:53:29,  3.79s/it]
 38%|███▊      | 86258/225000 [160:41:33<143:15:41,  3.72s/it]
 38%|███▊      | 86259/225000 [160:41:36<137:06:21,  3.56s/it]
 38%|███▊      | 86260/225000 [160:41:39<136:52:39,  3.55s/it]
 38%|███▊      | 86261/225000 [160:41:44<153:10:08,  3.97s/it]
 38%|███▊      | 86262/225000 [160:41:53<210:06:46,  5.45s/it]
 38%|███▊      | 86263/225000 [160:41:59<212:01:01,  5.50s/it]
 38%|███▊      | 86264/225000 [160:42:01<175:15:25,  4.55s/it]
 38%|███▊      | 86265/225000 [160:42:04<151:19:29,  3.93s/it]
 38%|███▊      | 86266/225000 [160:42:11<188:09:40,  4.88s/it]
 38%|███▊      | 86267/225000 [160:42:13<159:21:06,  4.14s/it]
 38%|███▊      | 86268/225000 [160:42:15<137:18:14,  3.56s/it]
 38%|███▊      | 86269/225000 [160:42:23<184:34:27,  4.79s/it]
 38%|███▊      | 86270/225000 [160:42:26<164:00:30,  4.26s/it]
 38%|███▊      | 86271/225000 [160:42:29<148:16:58,  3.85s/it]
 38%|███▊      | 86272/225000 [160:42:32<136:58:07,  3.55s/it]
 38%|███▊      | 86273/225000 [160:42:35<131:34:28,  3.41s/it]
 38%|███▊      | 86274/225000 [160:42:38<125:07:33,  3.25s/it]
 38%|███▊      | 86275/225000 [160:42:41<124:33:43,  3.23s/it]
 38%|███▊      | 86276/225000 [160:42:44<119:56:51,  3.11s/it]
 38%|███▊      | 86277/225000 [160:42:51<172:35:37,  4.48s/it]
 38%|███▊      | 86278/225000 [160:42:58<198:57:58,  5.16s/it]
 38%|███▊      | 86279/225000 [160:43:00<166:42:25,  4.33s/it]
 38%|███▊      | 86280/225000 [160:43:03<147:29:01,  3.83s/it]
 38%|███▊      | 86281/225000 [160:43:07<143:18:14,  3.72s/it]
 38%|███▊      | 86282/225000 [160:43:10<140:16:48,  3.64s/it]
 38%|███▊      | 86283/225000 [160:43:19<198:48:24,  5.16s/it]
 38%|███▊      | 86284/225000 [160:43:23<192:34:14,  5.00s/it]
 38%|███▊      | 86285/225000 [160:43:26<169:09:27,  4.39s/it]
 38%|███▊      | 86286/225000 [160:43:29<147:05:52,  3.82s/it]
 38%|███▊      | 86287/225000 [160:43:33<151:42:38,  3.94s/it]
 38%|███▊      | 86288/225000 [160:43:40<190:41:26,  4.95s/it]
 38%|███▊      | 86289/225000 [160:43:43<165:12:57,  4.29s/it]
 38%|███▊      | 86290/225000 [160:43:45<142:58:55,  3.71s/it]
 38%|███▊      | 86291/225000 [160:43:49<137:36:27,  3.57s/it]
 38%|███▊      | 86292/225000 [160:43:52<134:36:14,  3.49s/it]
 38%|███▊      | 86293/225000 [160:43:55<129:19:54,  3.36s/it]
 38%|███▊      | 86294/225000 [160:44:00<146:46:52,  3.81s/it]
 38%|███▊      | 86295/225000 [160:44:09<207:31:06,  5.39s/it]
 38%|███▊      | 86296/225000 [160:44:12<183:52:25,  4.77s/it]
 38%|███▊      | 86297/225000 [160:44:15<158:51:37,  4.12s/it]
 38%|███▊      | 86298/225000 [160:44:18<142:23:31,  3.70s/it]
 38%|███▊      | 86299/225000 [160:44:24<174:19:26,  4.52s/it]
 38%|███▊      | 86300/225000 [160:44:27<152:01:25,  3.95s/it]
                                                              
{'loss': '0.4756', 'grad_norm': '4.639', 'learning_rate': '3.602e-05', 'epoch': '0.3836', 'num_input_tokens_seen': 263763976, 'train_runtime': '5.787e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86300/225000 [160:44:27<152:01:25,  3.95s/it]
 38%|███▊      | 86301/225000 [160:44:30<142:44:19,  3.70s/it]
 38%|███▊      | 86302/225000 [160:44:33<139:17:41,  3.62s/it]
 38%|███▊      | 86303/225000 [160:44:36<133:57:39,  3.48s/it]
 38%|███▊      | 86304/225000 [160:44:40<131:24:28,  3.41s/it]
 38%|███▊      | 86305/225000 [160:44:43<128:08:12,  3.33s/it]
 38%|███▊      | 86306/225000 [160:44:46<127:12:58,  3.30s/it]
 38%|███▊      | 86307/225000 [160:44:49<125:17:25,  3.25s/it]
 38%|███▊      | 86308/225000 [160:44:53<127:32:31,  3.31s/it]
 38%|███▊      | 86309/225000 [160:44:56<126:25:52,  3.28s/it]
 38%|███▊      | 86310/225000 [160:44:59<124:24:29,  3.23s/it]
 38%|███▊      | 86311/225000 [160:45:03<128:37:19,  3.34s/it]
 38%|███▊      | 86312/225000 [160:45:11<190:25:45,  4.94s/it]
 38%|███▊      | 86313/225000 [160:45:18<213:06:57,  5.53s/it]
 38%|███▊      | 86314/225000 [160:45:21<177:52:03,  4.62s/it]
 38%|███▊      | 86315/225000 [160:45:23<154:33:13,  4.01s/it]
 38%|███▊      | 86316/225000 [160:45:27<149:02:57,  3.87s/it]
 38%|███▊      | 86317/225000 [160:45:34<184:09:15,  4.78s/it]
 38%|███▊      | 86318/225000 [160:45:36<160:53:37,  4.18s/it]
 38%|███▊      | 86319/225000 [160:45:39<148:05:56,  3.84s/it]
 38%|███▊      | 86320/225000 [160:45:43<141:37:25,  3.68s/it]
 38%|███▊      | 86321/225000 [160:45:45<130:29:29,  3.39s/it]
 38%|███▊      | 86322/225000 [160:45:48<122:48:53,  3.19s/it]
 38%|███▊      | 86323/225000 [160:45:52<127:03:45,  3.30s/it]
 38%|███▊      | 86324/225000 [160:46:01<192:12:48,  4.99s/it]
 38%|███▊      | 86325/225000 [160:46:05<180:41:28,  4.69s/it]
 38%|███▊      | 86326/225000 [160:46:07<156:38:54,  4.07s/it]
 38%|███▊      | 86327/225000 [160:46:12<159:01:53,  4.13s/it]
 38%|███▊      | 86328/225000 [160:46:18<180:40:48,  4.69s/it]
 38%|███▊      | 86329/225000 [160:46:20<152:27:17,  3.96s/it]
 38%|███▊      | 86330/225000 [160:46:22<134:44:01,  3.50s/it]
 38%|███▊      | 86331/225000 [160:46:25<126:44:25,  3.29s/it]
 38%|███▊      | 86332/225000 [160:46:28<122:37:26,  3.18s/it]
 38%|███▊      | 86333/225000 [160:46:31<124:41:01,  3.24s/it]
 38%|███▊      | 86334/225000 [160:46:36<136:38:22,  3.55s/it]
 38%|███▊      | 86335/225000 [160:46:39<135:26:56,  3.52s/it]
 38%|███▊      | 86336/225000 [160:46:43<135:06:31,  3.51s/it]
 38%|███▊      | 86337/225000 [160:46:47<140:46:27,  3.65s/it]
 38%|███▊      | 86338/225000 [160:46:54<185:50:05,  4.82s/it]
 38%|███▊      | 86339/225000 [160:46:59<187:49:27,  4.88s/it]
 38%|███▊      | 86340/225000 [160:47:06<210:33:32,  5.47s/it]
 38%|███▊      | 86341/225000 [160:47:10<193:03:05,  5.01s/it]
 38%|███▊      | 86342/225000 [160:47:14<181:26:05,  4.71s/it]
 38%|███▊      | 86343/225000 [160:47:17<166:15:54,  4.32s/it]
 38%|███▊      | 86344/225000 [160:47:21<155:50:25,  4.05s/it]
 38%|███▊      | 86345/225000 [160:47:24<147:38:15,  3.83s/it]
 38%|███▊      | 86346/225000 [160:47:28<145:59:44,  3.79s/it]
 38%|███▊      | 86347/225000 [160:47:32<147:53:13,  3.84s/it]
 38%|███▊      | 86348/225000 [160:47:37<169:24:53,  4.40s/it]
 38%|███▊      | 86349/225000 [160:47:43<180:12:31,  4.68s/it]
 38%|███▊      | 86350/225000 [160:47:46<164:54:22,  4.28s/it]
                                                              
{'loss': '0.4764', 'grad_norm': '4.777', 'learning_rate': '3.6e-05', 'epoch': '0.3838', 'num_input_tokens_seen': 263849408, 'train_runtime': '5.789e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86350/225000 [160:47:46<164:54:22,  4.28s/it]
 38%|███▊      | 86351/225000 [160:47:49<147:14:07,  3.82s/it]
 38%|███▊      | 86352/225000 [160:47:52<136:56:37,  3.56s/it]
 38%|███▊      | 86353/225000 [160:47:59<180:50:23,  4.70s/it]
 38%|███▊      | 86354/225000 [160:48:02<160:08:11,  4.16s/it]
 38%|███▊      | 86355/225000 [160:48:05<145:21:11,  3.77s/it]
 38%|███▊      | 86356/225000 [160:48:08<137:15:49,  3.56s/it]
 38%|███▊      | 86357/225000 [160:48:11<129:14:32,  3.36s/it]
 38%|███▊      | 86358/225000 [160:48:14<130:17:25,  3.38s/it]
 38%|███▊      | 86359/225000 [160:48:17<124:12:00,  3.23s/it]
 38%|███▊      | 86360/225000 [160:48:20<124:47:56,  3.24s/it]
 38%|███▊      | 86361/225000 [160:48:28<176:30:38,  4.58s/it]
 38%|███▊      | 86362/225000 [160:48:34<193:05:53,  5.01s/it]
 38%|███▊      | 86363/225000 [160:48:37<172:51:53,  4.49s/it]
 38%|███▊      | 86364/225000 [160:48:41<161:08:32,  4.18s/it]
 38%|███▊      | 86365/225000 [160:48:44<148:27:27,  3.86s/it]
 38%|███▊      | 86366/225000 [160:48:47<139:06:56,  3.61s/it]
 38%|███▊      | 86367/225000 [160:48:50<133:38:01,  3.47s/it]
 38%|███▊      | 86368/225000 [160:48:54<133:14:45,  3.46s/it]
 38%|███▊      | 86369/225000 [160:48:57<128:30:34,  3.34s/it]
 38%|███▊      | 86370/225000 [160:49:00<124:37:05,  3.24s/it]
 38%|███▊      | 86371/225000 [160:49:03<123:23:18,  3.20s/it]
 38%|███▊      | 86372/225000 [160:49:06<124:56:37,  3.24s/it]
 38%|███▊      | 86373/225000 [160:49:11<148:25:01,  3.85s/it]
 38%|███▊      | 86374/225000 [160:49:19<195:24:15,  5.07s/it]
 38%|███▊      | 86375/225000 [160:49:25<199:25:02,  5.18s/it]
 38%|███▊      | 86376/225000 [160:49:29<189:22:57,  4.92s/it]
 38%|███▊      | 86377/225000 [160:49:32<170:00:51,  4.42s/it]
 38%|███▊      | 86378/225000 [160:49:36<158:39:21,  4.12s/it]
 38%|███▊      | 86379/225000 [160:49:39<145:57:44,  3.79s/it]
 38%|███▊      | 86380/225000 [160:49:42<139:19:29,  3.62s/it]
 38%|███▊      | 86381/225000 [160:49:45<132:35:05,  3.44s/it]
 38%|███▊      | 86382/225000 [160:49:49<138:58:44,  3.61s/it]
 38%|███▊      | 86383/225000 [160:49:52<135:02:45,  3.51s/it]
 38%|███▊      | 86384/225000 [160:49:57<151:51:15,  3.94s/it]
 38%|███▊      | 86385/225000 [160:50:07<217:14:49,  5.64s/it]
 38%|███▊      | 86386/225000 [160:50:10<187:12:43,  4.86s/it]
 38%|███▊      | 86387/225000 [160:50:13<161:15:31,  4.19s/it]
 38%|███▊      | 86388/225000 [160:50:18<175:07:24,  4.55s/it]
 38%|███▊      | 86389/225000 [160:50:22<164:26:42,  4.27s/it]
 38%|███▊      | 86390/225000 [160:50:25<150:44:06,  3.91s/it]
 38%|███▊      | 86391/225000 [160:50:28<141:29:02,  3.67s/it]
 38%|███▊      | 86392/225000 [160:50:30<131:10:29,  3.41s/it]
 38%|███▊      | 86393/225000 [160:50:33<124:17:43,  3.23s/it]
 38%|███▊      | 86394/225000 [160:50:36<123:36:25,  3.21s/it]
 38%|███▊      | 86395/225000 [160:50:40<122:21:41,  3.18s/it]
 38%|███▊      | 86396/225000 [160:50:43<121:33:45,  3.16s/it]
 38%|███▊      | 86397/225000 [160:50:46<119:23:21,  3.10s/it]
 38%|███▊      | 86398/225000 [160:50:52<154:50:35,  4.02s/it]
 38%|███▊      | 86399/225000 [160:51:00<197:57:04,  5.14s/it]
 38%|███▊      | 86400/225000 [160:51:02<165:56:20,  4.31s/it]
                                                              
{'loss': '0.4575', 'grad_norm': '6.344', 'learning_rate': '3.599e-05', 'epoch': '0.384', 'num_input_tokens_seen': 263933416, 'train_runtime': '5.791e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86400/225000 [160:51:02<165:56:20,  4.31s/it]
 38%|███▊      | 86401/225000 [160:51:04<143:30:33,  3.73s/it]
 38%|███▊      | 86402/225000 [160:51:08<146:04:00,  3.79s/it]
 38%|███▊      | 86403/225000 [160:51:14<170:29:39,  4.43s/it]
 38%|███▊      | 86404/225000 [160:51:17<152:50:58,  3.97s/it]
 38%|███▊      | 86405/225000 [160:51:21<146:30:17,  3.81s/it]
 38%|███▊      | 86406/225000 [160:51:23<136:49:24,  3.55s/it]
 38%|███▊      | 86407/225000 [160:51:27<132:15:10,  3.44s/it]
 38%|███▊      | 86408/225000 [160:51:30<128:40:19,  3.34s/it]
 38%|███▊      | 86409/225000 [160:51:33<130:28:09,  3.39s/it]
 38%|███▊      | 86410/225000 [160:51:36<125:15:10,  3.25s/it]
 38%|███▊      | 86411/225000 [160:51:39<123:42:10,  3.21s/it]
 38%|███▊      | 86412/225000 [160:51:42<121:05:04,  3.15s/it]
 38%|███▊      | 86413/225000 [160:51:45<118:37:31,  3.08s/it]
 38%|███▊      | 86414/225000 [160:51:52<156:25:48,  4.06s/it]
 38%|███▊      | 86415/225000 [160:52:00<205:45:40,  5.35s/it]
 38%|███▊      | 86416/225000 [160:52:02<170:30:03,  4.43s/it]
 38%|███▊      | 86417/225000 [160:52:05<148:17:16,  3.85s/it]
 38%|███▊      | 86418/225000 [160:52:07<132:14:41,  3.44s/it]
 38%|███▊      | 86419/225000 [160:52:10<127:21:20,  3.31s/it]
 38%|███▊      | 86420/225000 [160:52:18<183:20:20,  4.76s/it]
 38%|███▊      | 86421/225000 [160:52:21<159:58:23,  4.16s/it]
 38%|███▊      | 86422/225000 [160:52:24<144:25:16,  3.75s/it]
 38%|███▊      | 86423/225000 [160:52:27<132:20:25,  3.44s/it]
 38%|███▊      | 86424/225000 [160:52:30<126:56:47,  3.30s/it]
 38%|███▊      | 86425/225000 [160:52:32<120:43:55,  3.14s/it]
 38%|███▊      | 86426/225000 [160:52:35<116:49:58,  3.04s/it]
 38%|███▊      | 86427/225000 [160:52:38<118:27:20,  3.08s/it]
 38%|███▊      | 86428/225000 [160:52:43<133:41:16,  3.47s/it]
 38%|███▊      | 86429/225000 [160:52:52<199:40:37,  5.19s/it]
 38%|███▊      | 86430/225000 [160:52:56<183:30:50,  4.77s/it]
 38%|███▊      | 86431/225000 [160:52:58<160:01:33,  4.16s/it]
 38%|███▊      | 86432/225000 [160:53:01<143:57:50,  3.74s/it]
 38%|███▊      | 86433/225000 [160:53:09<187:20:47,  4.87s/it]
 38%|███▊      | 86434/225000 [160:53:11<158:25:11,  4.12s/it]
 38%|███▊      | 86435/225000 [160:53:13<138:40:31,  3.60s/it]
 38%|███▊      | 86436/225000 [160:53:21<183:54:45,  4.78s/it]
 38%|███▊      | 86437/225000 [160:53:24<160:02:36,  4.16s/it]
 38%|███▊      | 86438/225000 [160:53:27<147:43:42,  3.84s/it]
 38%|███▊      | 86439/225000 [160:53:30<137:44:47,  3.58s/it]
 38%|███▊      | 86440/225000 [160:53:33<129:21:28,  3.36s/it]
 38%|███▊      | 86441/225000 [160:53:37<139:11:51,  3.62s/it]
 38%|███▊      | 86442/225000 [160:53:45<187:06:53,  4.86s/it]
 38%|███▊      | 86443/225000 [160:53:48<170:41:21,  4.43s/it]
 38%|███▊      | 86444/225000 [160:53:54<187:44:20,  4.88s/it]
 38%|███▊      | 86445/225000 [160:53:57<161:55:47,  4.21s/it]
 38%|███▊      | 86446/225000 [160:53:59<142:22:01,  3.70s/it]
 38%|███▊      | 86447/225000 [160:54:06<182:21:06,  4.74s/it]
 38%|███▊      | 86448/225000 [160:54:09<155:18:35,  4.04s/it]
 38%|███▊      | 86449/225000 [160:54:14<169:36:03,  4.41s/it]
 38%|███▊      | 86450/225000 [160:54:16<144:57:46,  3.77s/it]
                                                              
{'loss': '0.4674', 'grad_norm': '6.606', 'learning_rate': '3.597e-05', 'epoch': '0.3842', 'num_input_tokens_seen': 264018944, 'train_runtime': '5.793e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86450/225000 [160:54:16<144:57:46,  3.77s/it]
 38%|███▊      | 86451/225000 [160:54:19<128:17:21,  3.33s/it]
 38%|███▊      | 86452/225000 [160:54:24<151:29:08,  3.94s/it]
 38%|███▊      | 86453/225000 [160:54:28<157:58:05,  4.10s/it]
 38%|███▊      | 86454/225000 [160:54:31<142:33:49,  3.70s/it]
 38%|███▊      | 86455/225000 [160:54:34<132:14:39,  3.44s/it]
 38%|███▊      | 86456/225000 [160:54:37<124:29:41,  3.23s/it]
 38%|███▊      | 86457/225000 [160:54:39<119:06:29,  3.09s/it]
 38%|███▊      | 86458/225000 [160:54:43<120:15:43,  3.12s/it]
 38%|███▊      | 86459/225000 [160:54:51<180:46:50,  4.70s/it]
 38%|███▊      | 86460/225000 [160:54:55<176:02:13,  4.57s/it]
 38%|███▊      | 86461/225000 [160:54:59<161:17:20,  4.19s/it]
 38%|███▊      | 86462/225000 [160:55:02<149:22:08,  3.88s/it]
 38%|███▊      | 86463/225000 [160:55:05<140:09:41,  3.64s/it]
 38%|███▊      | 86464/225000 [160:55:08<131:30:12,  3.42s/it]
 38%|███▊      | 86465/225000 [160:55:11<124:19:58,  3.23s/it]
 38%|███▊      | 86466/225000 [160:55:14<124:34:08,  3.24s/it]
 38%|███▊      | 86467/225000 [160:55:17<126:28:13,  3.29s/it]
 38%|███▊      | 86468/225000 [160:55:25<180:58:22,  4.70s/it]
 38%|███▊      | 86469/225000 [160:55:31<190:29:27,  4.95s/it]
 38%|███▊      | 86470/225000 [160:55:33<164:31:28,  4.28s/it]
 38%|███▊      | 86471/225000 [160:55:36<142:40:11,  3.71s/it]
 38%|███▊      | 86472/225000 [160:55:39<132:53:34,  3.45s/it]
 38%|███▊      | 86473/225000 [160:55:42<129:06:48,  3.36s/it]
 38%|███▊      | 86474/225000 [160:55:45<126:31:29,  3.29s/it]
 38%|███▊      | 86475/225000 [160:55:48<123:17:30,  3.20s/it]
 38%|███▊      | 86476/225000 [160:55:51<123:46:08,  3.22s/it]
 38%|███▊      | 86477/225000 [160:55:54<122:39:20,  3.19s/it]
 38%|███▊      | 86478/225000 [160:55:57<121:20:04,  3.15s/it]
 38%|███▊      | 86479/225000 [160:56:02<133:38:35,  3.47s/it]
 38%|███▊      | 86480/225000 [160:56:11<204:01:16,  5.30s/it]
 38%|███▊      | 86481/225000 [160:56:15<188:39:57,  4.90s/it]
 38%|███▊      | 86482/225000 [160:56:18<160:07:43,  4.16s/it]
 38%|███▊      | 86483/225000 [160:56:20<144:33:57,  3.76s/it]
 38%|███▊      | 86484/225000 [160:56:24<139:07:03,  3.62s/it]
 38%|███▊      | 86485/225000 [160:56:31<177:02:26,  4.60s/it]
 38%|███▊      | 86486/225000 [160:56:37<192:45:15,  5.01s/it]
 38%|███▊      | 86487/225000 [160:56:39<164:40:25,  4.28s/it]
 38%|███▊      | 86488/225000 [160:56:42<149:35:36,  3.89s/it]
 38%|███▊      | 86489/225000 [160:56:45<140:22:31,  3.65s/it]
 38%|███▊      | 86490/225000 [160:56:55<207:24:15,  5.39s/it]
 38%|███▊      | 86491/225000 [160:56:57<173:48:17,  4.52s/it]
 38%|███▊      | 86492/225000 [160:57:00<155:53:13,  4.05s/it]
 38%|███▊      | 86493/225000 [160:57:03<138:04:10,  3.59s/it]
 38%|███▊      | 86494/225000 [160:57:06<132:54:19,  3.45s/it]
 38%|███▊      | 86495/225000 [160:57:08<124:50:26,  3.24s/it]
 38%|███▊      | 86496/225000 [160:57:12<128:52:42,  3.35s/it]
 38%|███▊      | 86497/225000 [160:57:16<131:04:02,  3.41s/it]
 38%|███▊      | 86498/225000 [160:57:19<127:59:40,  3.33s/it]
 38%|███▊      | 86499/225000 [160:57:22<131:20:00,  3.41s/it]
 38%|███▊      | 86500/225000 [160:57:30<178:38:15,  4.64s/it]
                                                              
{'loss': '0.4619', 'grad_norm': '3.357', 'learning_rate': '3.595e-05', 'epoch': '0.3844', 'num_input_tokens_seen': 264104072, 'train_runtime': '5.795e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86500/225000 [160:57:30<178:38:15,  4.64s/it]
 38%|███▊      | 86501/225000 [160:57:36<196:00:18,  5.09s/it]
 38%|███▊      | 86502/225000 [160:57:38<163:09:28,  4.24s/it]
 38%|███▊      | 86503/225000 [160:57:44<185:18:22,  4.82s/it]
 38%|███▊      | 86504/225000 [160:57:48<166:04:18,  4.32s/it]
 38%|███▊      | 86505/225000 [160:57:51<150:41:19,  3.92s/it]
 38%|███▊      | 86506/225000 [160:57:54<146:41:56,  3.81s/it]
 38%|███▊      | 86507/225000 [160:57:57<137:06:55,  3.56s/it]
 38%|███▊      | 86508/225000 [160:58:00<133:54:17,  3.48s/it]
 38%|███▊      | 86509/225000 [160:58:04<130:57:21,  3.40s/it]
 38%|███▊      | 86510/225000 [160:58:07<128:54:20,  3.35s/it]
 38%|███▊      | 86511/225000 [160:58:11<133:40:14,  3.47s/it]
 38%|███▊      | 86512/225000 [160:58:14<130:19:32,  3.39s/it]
 38%|███▊      | 86513/225000 [160:58:17<129:12:56,  3.36s/it]
 38%|███▊      | 86514/225000 [160:58:21<131:19:03,  3.41s/it]
 38%|███▊      | 86515/225000 [160:58:28<171:36:59,  4.46s/it]
 38%|███▊      | 86516/225000 [160:58:35<208:00:03,  5.41s/it]
 38%|███▊      | 86517/225000 [160:58:38<176:58:32,  4.60s/it]
 38%|███▊      | 86518/225000 [160:58:41<157:36:48,  4.10s/it]
 38%|███▊      | 86519/225000 [160:58:43<139:17:28,  3.62s/it]
 38%|███▊      | 86520/225000 [160:58:47<135:19:50,  3.52s/it]
 38%|███▊      | 86521/225000 [160:58:55<192:27:26,  5.00s/it]
 38%|███▊      | 86522/225000 [160:58:59<175:04:40,  4.55s/it]
 38%|███▊      | 86523/225000 [160:59:01<150:47:45,  3.92s/it]
 38%|███▊      | 86524/225000 [160:59:04<140:20:51,  3.65s/it]
 38%|███▊      | 86525/225000 [160:59:07<132:34:03,  3.45s/it]
 38%|███▊      | 86526/225000 [160:59:10<129:10:48,  3.36s/it]
 38%|███▊      | 86527/225000 [160:59:13<128:48:34,  3.35s/it]
 38%|███▊      | 86528/225000 [160:59:19<154:52:34,  4.03s/it]
 38%|███▊      | 86529/225000 [160:59:27<194:03:22,  5.05s/it]
 38%|███▊      | 86530/225000 [160:59:29<163:08:42,  4.24s/it]
 38%|███▊      | 86531/225000 [160:59:31<141:41:59,  3.68s/it]
 38%|███▊      | 86532/225000 [160:59:34<132:40:14,  3.45s/it]
 38%|███▊      | 86533/225000 [160:59:37<128:34:59,  3.34s/it]
 38%|███▊      | 86534/225000 [160:59:41<136:43:38,  3.55s/it]
 38%|███▊      | 86535/225000 [160:59:50<199:49:56,  5.20s/it]
 38%|███▊      | 86536/225000 [160:59:54<177:14:51,  4.61s/it]
 38%|███▊      | 86537/225000 [160:59:56<151:45:15,  3.95s/it]
 38%|███▊      | 86538/225000 [160:59:59<135:59:36,  3.54s/it]
 38%|███▊      | 86539/225000 [161:00:06<180:46:16,  4.70s/it]
 38%|███▊      | 86540/225000 [161:00:10<170:58:30,  4.45s/it]
 38%|███▊      | 86541/225000 [161:00:13<152:09:57,  3.96s/it]
 38%|███▊      | 86542/225000 [161:00:15<137:31:00,  3.58s/it]
 38%|███▊      | 86543/225000 [161:00:18<127:30:07,  3.32s/it]
 38%|███▊      | 86544/225000 [161:00:21<128:56:52,  3.35s/it]
 38%|███▊      | 86545/225000 [161:00:24<120:31:34,  3.13s/it]
 38%|███▊      | 86546/225000 [161:00:27<115:04:45,  2.99s/it]
 38%|███▊      | 86547/225000 [161:00:30<121:24:00,  3.16s/it]
 38%|███▊      | 86548/225000 [161:00:34<128:29:05,  3.34s/it]
 38%|███▊      | 86549/225000 [161:00:40<152:48:38,  3.97s/it]
 38%|███▊      | 86550/225000 [161:00:48<203:22:50,  5.29s/it]
                                                              
{'loss': '0.4763', 'grad_norm': '3.777', 'learning_rate': '3.594e-05', 'epoch': '0.3847', 'num_input_tokens_seen': 264188632, 'train_runtime': '5.796e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86550/225000 [161:00:48<203:22:50,  5.29s/it]
 38%|███▊      | 86551/225000 [161:00:51<176:31:12,  4.59s/it]
 38%|███▊      | 86552/225000 [161:00:53<151:45:00,  3.95s/it]
 38%|███▊      | 86553/225000 [161:00:56<137:05:48,  3.56s/it]
 38%|███▊      | 86554/225000 [161:01:04<188:34:05,  4.90s/it]
 38%|███▊      | 86555/225000 [161:01:08<181:15:40,  4.71s/it]
 38%|███▊      | 86556/225000 [161:01:11<156:34:29,  4.07s/it]
 38%|███▊      | 86557/225000 [161:01:14<147:25:06,  3.83s/it]
 38%|███▊      | 86558/225000 [161:01:18<150:44:36,  3.92s/it]
 38%|███▊      | 86559/225000 [161:01:21<140:29:19,  3.65s/it]
 38%|███▊      | 86560/225000 [161:01:24<135:42:25,  3.53s/it]
 38%|███▊      | 86561/225000 [161:01:28<130:11:39,  3.39s/it]
 38%|███▊      | 86562/225000 [161:01:31<129:29:01,  3.37s/it]
 38%|███▊      | 86563/225000 [161:01:34<128:48:30,  3.35s/it]
 38%|███▊      | 86564/225000 [161:01:42<184:53:59,  4.81s/it]
 38%|███▊      | 86565/225000 [161:01:49<203:18:41,  5.29s/it]
 38%|███▊      | 86566/225000 [161:01:51<170:33:26,  4.44s/it]
 38%|███▊      | 86567/225000 [161:01:54<148:37:19,  3.86s/it]
 38%|███▊      | 86568/225000 [161:01:57<142:50:24,  3.71s/it]
 38%|███▊      | 86569/225000 [161:02:06<203:56:16,  5.30s/it]
 38%|███▊      | 86570/225000 [161:02:09<177:43:28,  4.62s/it]
 38%|███▊      | 86571/225000 [161:02:12<154:26:59,  4.02s/it]
 38%|███▊      | 86572/225000 [161:02:15<142:58:23,  3.72s/it]
 38%|███▊      | 86573/225000 [161:02:18<134:47:12,  3.51s/it]
 38%|███▊      | 86574/225000 [161:02:21<129:32:09,  3.37s/it]
 38%|███▊      | 86575/225000 [161:02:26<147:55:39,  3.85s/it]
 38%|███▊      | 86576/225000 [161:02:34<200:16:22,  5.21s/it]
 38%|███▊      | 86577/225000 [161:02:37<176:36:00,  4.59s/it]
 38%|███▊      | 86578/225000 [161:02:40<157:16:31,  4.09s/it]
 38%|███▊      | 86579/225000 [161:02:44<148:34:10,  3.86s/it]
 38%|███▊      | 86580/225000 [161:02:47<140:40:59,  3.66s/it]
 38%|███▊      | 86581/225000 [161:02:50<134:42:39,  3.50s/it]
 38%|███▊      | 86582/225000 [161:02:53<130:27:24,  3.39s/it]
 38%|███▊      | 86583/225000 [161:02:56<129:39:29,  3.37s/it]
 38%|███▊      | 86584/225000 [161:03:01<149:04:24,  3.88s/it]
 38%|███▊      | 86585/225000 [161:03:10<202:03:19,  5.26s/it]
 38%|███▊      | 86586/225000 [161:03:15<200:30:13,  5.21s/it]
 38%|███▊      | 86587/225000 [161:03:18<171:02:32,  4.45s/it]
 38%|███▊      | 86588/225000 [161:03:20<150:04:32,  3.90s/it]
 38%|███▊      | 86589/225000 [161:03:23<136:16:26,  3.54s/it]
 38%|███▊      | 86590/225000 [161:03:26<126:17:57,  3.29s/it]
 38%|███▊      | 86591/225000 [161:03:31<143:53:33,  3.74s/it]
 38%|███▊      | 86592/225000 [161:03:39<203:30:06,  5.29s/it]
 38%|███▊      | 86593/225000 [161:03:42<174:07:20,  4.53s/it]
 38%|███▊      | 86594/225000 [161:03:45<153:55:29,  4.00s/it]
 38%|███▊      | 86595/225000 [161:03:48<141:15:31,  3.67s/it]
 38%|███▊      | 86596/225000 [161:03:55<177:29:05,  4.62s/it]
 38%|███▊      | 86597/225000 [161:03:58<158:49:41,  4.13s/it]
 38%|███▊      | 86598/225000 [161:04:00<142:04:09,  3.70s/it]
 38%|███▊      | 86599/225000 [161:04:07<177:35:44,  4.62s/it]
 38%|███▊      | 86600/225000 [161:04:11<165:37:01,  4.31s/it]
                                                              
{'loss': '0.4751', 'grad_norm': '4.255', 'learning_rate': '3.592e-05', 'epoch': '0.3849', 'num_input_tokens_seen': 264274568, 'train_runtime': '5.799e+05', 'train_tokens_per_second': '455.8'}

 38%|███▊      | 86600/225000 [161:04:11<165:37:01,  4.31s/it]
 38%|███▊      | 86601/225000 [161:04:14<152:38:35,  3.97s/it]
 38%|███▊      | 86602/225000 [161:04:17<144:40:46,  3.76s/it]
 38%|███▊      | 86603/225000 [161:04:20<134:51:56,  3.51s/it]
 38%|███▊      | 86604/225000 [161:04:28<179:48:50,  4.68s/it]
 38%|███▊      | 86605/225000 [161:04:31<165:35:41,  4.31s/it]
 38%|███▊      | 86606/225000 [161:04:34<150:21:43,  3.91s/it]
 38%|███▊      | 86607/225000 [161:04:37<140:16:11,  3.65s/it]
 38%|███▊      | 86608/225000 [161:04:40<131:36:35,  3.42s/it]
 38%|███▊      | 86609/225000 [161:04:43<133:11:17,  3.46s/it]
 38%|███▊      | 86610/225000 [161:04:46<127:44:41,  3.32s/it]
 38%|███▊      | 86611/225000 [161:04:53<160:50:34,  4.18s/it]
 38%|███▊      | 86612/225000 [161:05:00<201:55:14,  5.25s/it]
 38%|███▊      | 86613/225000 [161:05:03<172:17:37,  4.48s/it]
 38%|███▊      | 86614/225000 [161:05:06<151:20:05,  3.94s/it]
 38%|███▊      | 86615/225000 [161:05:08<132:19:03,  3.44s/it]
 38%|███▊      | 86616/225000 [161:05:16<184:39:50,  4.80s/it]
 38%|███▊      | 86617/225000 [161:05:18<155:34:50,  4.05s/it]
 38%|███▊      | 86618/225000 [161:05:21<135:32:16,  3.53s/it]
 38%|███▊      | 86619/225000 [161:05:24<129:06:37,  3.36s/it]
 38%|███▊      | 86620/225000 [161:05:26<124:14:45,  3.23s/it]
 38%|███▊      | 86621/225000 [161:05:30<127:14:30,  3.31s/it]
 38%|███▊      | 86622/225000 [161:05:33<129:01:52,  3.36s/it]
 38%|███▊      | 86623/225000 [161:05:36<123:42:31,  3.22s/it]
 38%|███▊      | 86624/225000 [161:05:39<122:04:15,  3.18s/it]
 38%|███▊      | 86625/225000 [161:05:48<187:57:43,  4.89s/it]
 39%|███▊      | 86626/225000 [161:05:55<207:52:57,  5.41s/it]
 39%|███▊      | 86627/225000 [161:05:58<178:03:01,  4.63s/it]
 39%|███▊      | 86628/225000 [161:06:01<159:47:21,  4.16s/it]
 39%|███▊      | 86629/225000 [161:06:04<143:57:31,  3.75s/it]
 39%|███▊      | 86630/225000 [161:06:06<132:28:03,  3.45s/it]
 39%|███▊      | 86631/225000 [161:06:14<178:47:06,  4.65s/it]
 39%|███▊      | 86632/225000 [161:06:18<178:01:37,  4.63s/it]
 39%|███▊      | 86633/225000 [161:06:23<180:29:31,  4.70s/it]
 39%|███▊      | 86634/225000 [161:06:26<162:16:02,  4.22s/it]
 39%|███▊      | 86635/225000 [161:06:29<146:57:22,  3.82s/it]
 39%|███▊      | 86636/225000 [161:06:32<135:32:27,  3.53s/it]
 39%|███▊      | 86637/225000 [161:06:35<126:57:18,  3.30s/it]
 39%|███▊      | 86638/225000 [161:06:38<124:53:35,  3.25s/it]
 39%|███▊      | 86639/225000 [161:06:41<127:05:06,  3.31s/it]
 39%|███▊      | 86640/225000 [161:06:45<124:51:28,  3.25s/it]
 39%|███▊      | 86641/225000 [161:06:48<125:06:28,  3.26s/it]
 39%|███▊      | 86642/225000 [161:06:51<120:20:39,  3.13s/it]
 39%|███▊      | 86643/225000 [161:06:54<121:17:13,  3.16s/it]
 39%|███▊      | 86644/225000 [161:07:01<170:51:52,  4.45s/it]
 39%|███▊      | 86645/225000 [161:07:10<220:47:59,  5.75s/it]
 39%|███▊      | 86646/225000 [161:07:13<183:28:13,  4.77s/it]
 39%|███▊      | 86647/225000 [161:07:15<155:41:40,  4.05s/it]
 39%|███▊      | 86648/225000 [161:07:21<173:35:48,  4.52s/it]
 39%|███▊      | 86649/225000 [161:07:26<187:04:32,  4.87s/it]
 39%|███▊      | 86650/225000 [161:07:29<164:35:00,  4.28s/it]
                                                              
{'loss': '0.442', 'grad_norm': '4.087', 'learning_rate': '3.591e-05', 'epoch': '0.3851', 'num_input_tokens_seen': 264360752, 'train_runtime': '5.8e+05', 'train_tokens_per_second': '455.8'}

 39%|███▊      | 86650/225000 [161:07:29<164:35:00,  4.28s/it]
 39%|███▊      | 86651/225000 [161:07:32<152:07:06,  3.96s/it]
 39%|███▊      | 86652/225000 [161:07:35<142:21:54,  3.70s/it]
 39%|███▊      | 86653/225000 [161:07:38<133:06:10,  3.46s/it]
 39%|███▊      | 86654/225000 [161:07:41<125:50:59,  3.27s/it]
 39%|███▊      | 86655/225000 [161:07:44<122:08:03,  3.18s/it]
 39%|███▊      | 86656/225000 [161:07:51<163:12:18,  4.25s/it]
 39%|███▊      | 86657/225000 [161:07:58<197:50:12,  5.15s/it]
 39%|███▊      | 86658/225000 [161:08:01<168:38:56,  4.39s/it]
 39%|███▊      | 86659/225000 [161:08:04<155:13:22,  4.04s/it]
 39%|███▊      | 86660/225000 [161:08:09<169:14:59,  4.40s/it]
 39%|███▊      | 86661/225000 [161:08:15<181:14:58,  4.72s/it]
 39%|███▊      | 86662/225000 [161:08:18<164:42:45,  4.29s/it]
 39%|███▊      | 86663/225000 [161:08:22<161:09:46,  4.19s/it]
 39%|███▊      | 86664/225000 [161:08:26<154:04:38,  4.01s/it]
 39%|███▊      | 86665/225000 [161:08:29<146:19:41,  3.81s/it]
 39%|███▊      | 86666/225000 [161:08:32<143:24:39,  3.73s/it]
 39%|███▊      | 86667/225000 [161:08:38<165:52:09,  4.32s/it]
 39%|███▊      | 86668/225000 [161:08:42<165:25:32,  4.31s/it]
 39%|███▊      | 86669/225000 [161:08:45<143:31:37,  3.74s/it]
 39%|███▊      | 86670/225000 [161:08:48<139:20:09,  3.63s/it]
 39%|███▊      | 86671/225000 [161:08:55<181:52:10,  4.73s/it]
 39%|███▊      | 86672/225000 [161:08:58<157:33:10,  4.10s/it]
 39%|███▊      | 86673/225000 [161:09:01<143:31:26,  3.74s/it]
 39%|███▊      | 86674/225000 [161:09:04<133:17:31,  3.47s/it]
 39%|███▊      | 86675/225000 [161:09:07<125:35:34,  3.27s/it]
 39%|███▊      | 86676/225000 [161:09:09<120:17:28,  3.13s/it]
 39%|███▊      | 86677/225000 [161:09:12<114:14:36,  2.97s/it]
 39%|███▊      | 86678/225000 [161:09:20<176:53:24,  4.60s/it]
 39%|███▊      | 86679/225000 [161:09:26<184:12:37,  4.79s/it]
 39%|███▊      | 86680/225000 [161:09:28<158:27:16,  4.12s/it]
 39%|███▊      | 86681/225000 [161:09:31<140:55:00,  3.67s/it]
 39%|███▊      | 86682/225000 [161:09:34<135:03:24,  3.52s/it]
 39%|███▊      | 86683/225000 [161:09:37<131:14:00,  3.42s/it]
 39%|███▊      | 86684/225000 [161:09:41<132:03:24,  3.44s/it]
 39%|███▊      | 86685/225000 [161:09:44<133:56:07,  3.49s/it]
 39%|███▊      | 86686/225000 [161:09:48<133:22:30,  3.47s/it]
 39%|███▊      | 86687/225000 [161:09:51<129:02:03,  3.36s/it]
 39%|███▊      | 86688/225000 [161:09:57<166:09:09,  4.32s/it]
 39%|███▊      | 86689/225000 [161:10:06<221:04:56,  5.75s/it]
 39%|███▊      | 86690/225000 [161:10:09<184:31:23,  4.80s/it]
 39%|███▊      | 86691/225000 [161:10:11<155:03:26,  4.04s/it]
 39%|███▊      | 86692/225000 [161:10:17<170:45:04,  4.44s/it]
 39%|███▊      | 86693/225000 [161:10:20<151:40:03,  3.95s/it]
 39%|███▊      | 86694/225000 [161:10:22<139:51:54,  3.64s/it]
 39%|███▊      | 86695/225000 [161:10:26<133:45:57,  3.48s/it]
 39%|███▊      | 86696/225000 [161:10:29<133:09:46,  3.47s/it]
 39%|███▊      | 86697/225000 [161:10:32<129:19:29,  3.37s/it]
 39%|███▊      | 86698/225000 [161:10:40<179:50:15,  4.68s/it]
 39%|███▊      | 86699/225000 [161:10:48<216:46:49,  5.64s/it]
 39%|███▊      | 86700/225000 [161:10:51<186:30:38,  4.85s/it]
                                                              
{'loss': '0.4274', 'grad_norm': '5.982', 'learning_rate': '3.589e-05', 'epoch': '0.3853', 'num_input_tokens_seen': 264447672, 'train_runtime': '5.803e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86700/225000 [161:10:51<186:30:38,  4.85s/it]
 39%|███▊      | 86701/225000 [161:10:54<170:13:48,  4.43s/it]
 39%|███▊      | 86702/225000 [161:10:57<154:17:23,  4.02s/it]
 39%|███▊      | 86703/225000 [161:11:00<144:08:44,  3.75s/it]
 39%|███▊      | 86704/225000 [161:11:03<136:17:14,  3.55s/it]
 39%|███▊      | 86705/225000 [161:11:06<128:07:40,  3.34s/it]
 39%|███▊      | 86706/225000 [161:11:09<124:50:40,  3.25s/it]
 39%|███▊      | 86707/225000 [161:11:12<122:49:52,  3.20s/it]
 39%|███▊      | 86708/225000 [161:11:17<137:43:21,  3.59s/it]
 39%|███▊      | 86709/225000 [161:11:25<193:52:26,  5.05s/it]
 39%|███▊      | 86710/225000 [161:11:32<216:29:27,  5.64s/it]
 39%|███▊      | 86711/225000 [161:11:35<180:31:04,  4.70s/it]
 39%|███▊      | 86712/225000 [161:11:37<152:06:33,  3.96s/it]
 39%|███▊      | 86713/225000 [161:11:40<138:22:39,  3.60s/it]
 39%|███▊      | 86714/225000 [161:11:45<150:00:55,  3.91s/it]
 39%|███▊      | 86715/225000 [161:11:52<193:17:59,  5.03s/it]
 39%|███▊      | 86716/225000 [161:11:55<166:56:18,  4.35s/it]
 39%|███▊      | 86717/225000 [161:11:58<150:42:16,  3.92s/it]
 39%|███▊      | 86718/225000 [161:12:01<141:47:24,  3.69s/it]
 39%|███▊      | 86719/225000 [161:12:04<137:08:19,  3.57s/it]
 39%|███▊      | 86720/225000 [161:12:07<128:31:08,  3.35s/it]
 39%|███▊      | 86721/225000 [161:12:10<122:25:35,  3.19s/it]
 39%|███▊      | 86722/225000 [161:12:13<117:38:13,  3.06s/it]
 39%|███▊      | 86723/225000 [161:12:19<153:13:25,  3.99s/it]
 39%|███▊      | 86724/225000 [161:12:27<206:01:14,  5.36s/it]
 39%|███▊      | 86725/225000 [161:12:31<180:03:10,  4.69s/it]
 39%|███▊      | 86726/225000 [161:12:33<152:32:52,  3.97s/it]
 39%|███▊      | 86727/225000 [161:12:35<133:43:50,  3.48s/it]
 39%|███▊      | 86728/225000 [161:12:43<184:05:59,  4.79s/it]
 39%|███▊      | 86729/225000 [161:12:46<163:08:58,  4.25s/it]
 39%|███▊      | 86730/225000 [161:12:49<147:43:52,  3.85s/it]
 39%|███▊      | 86731/225000 [161:12:52<134:39:25,  3.51s/it]
 39%|███▊      | 86732/225000 [161:12:54<125:07:08,  3.26s/it]
 39%|███▊      | 86733/225000 [161:12:57<119:00:00,  3.10s/it]
 39%|███▊      | 86734/225000 [161:13:00<118:14:03,  3.08s/it]
 39%|███▊      | 86735/225000 [161:13:06<153:43:26,  4.00s/it]
 39%|███▊      | 86736/225000 [161:13:14<199:53:10,  5.20s/it]
 39%|███▊      | 86737/225000 [161:13:17<172:00:21,  4.48s/it]
 39%|███▊      | 86738/225000 [161:13:21<162:16:34,  4.23s/it]
 39%|███▊      | 86739/225000 [161:13:25<159:06:04,  4.14s/it]
 39%|███▊      | 86740/225000 [161:13:28<151:51:15,  3.95s/it]
 39%|███▊      | 86741/225000 [161:13:32<146:25:06,  3.81s/it]
 39%|███▊      | 86742/225000 [161:13:35<144:40:58,  3.77s/it]
 39%|███▊      | 86743/225000 [161:13:39<143:26:15,  3.73s/it]
 39%|███▊      | 86744/225000 [161:13:42<138:31:44,  3.61s/it]
 39%|███▊      | 86745/225000 [161:13:46<139:16:03,  3.63s/it]
 39%|███▊      | 86746/225000 [161:13:50<139:15:36,  3.63s/it]
 39%|███▊      | 86747/225000 [161:13:54<146:03:35,  3.80s/it]
 39%|███▊      | 86748/225000 [161:14:03<206:19:43,  5.37s/it]
 39%|███▊      | 86749/225000 [161:14:06<181:17:57,  4.72s/it]
 39%|███▊      | 86750/225000 [161:14:09<159:54:34,  4.16s/it]
                                                              
{'loss': '0.4405', 'grad_norm': '5.666', 'learning_rate': '3.587e-05', 'epoch': '0.3856', 'num_input_tokens_seen': 264533664, 'train_runtime': '5.804e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86750/225000 [161:14:09<159:54:34,  4.16s/it]
 39%|███▊      | 86751/225000 [161:14:12<150:20:00,  3.91s/it]
 39%|███▊      | 86752/225000 [161:14:16<150:05:53,  3.91s/it]
 39%|███▊      | 86753/225000 [161:14:20<145:23:07,  3.79s/it]
 39%|███▊      | 86754/225000 [161:14:25<167:00:52,  4.35s/it]
 39%|███▊      | 86755/225000 [161:14:28<149:45:27,  3.90s/it]
 39%|███▊      | 86756/225000 [161:14:31<143:37:19,  3.74s/it]
 39%|███▊      | 86757/225000 [161:14:35<137:13:01,  3.57s/it]
 39%|███▊      | 86758/225000 [161:14:38<130:44:06,  3.40s/it]
 39%|███▊      | 86759/225000 [161:14:41<127:58:55,  3.33s/it]
 39%|███▊      | 86760/225000 [161:14:44<126:01:32,  3.28s/it]
 39%|███▊      | 86761/225000 [161:14:47<122:29:49,  3.19s/it]
 39%|███▊      | 86762/225000 [161:14:50<123:44:11,  3.22s/it]
 39%|███▊      | 86763/225000 [161:14:53<122:24:33,  3.19s/it]
 39%|███▊      | 86764/225000 [161:15:00<166:08:05,  4.33s/it]
 39%|███▊      | 86765/225000 [161:15:10<233:00:08,  6.07s/it]
 39%|███▊      | 86766/225000 [161:15:13<195:53:42,  5.10s/it]
 39%|███▊      | 86767/225000 [161:15:16<165:27:13,  4.31s/it]
 39%|███▊      | 86768/225000 [161:15:18<147:06:00,  3.83s/it]
 39%|███▊      | 86769/225000 [161:15:21<137:38:02,  3.58s/it]
 39%|███▊      | 86770/225000 [161:15:25<132:28:17,  3.45s/it]
 39%|███▊      | 86771/225000 [161:15:28<133:02:23,  3.46s/it]
 39%|███▊      | 86772/225000 [161:15:35<172:20:31,  4.49s/it]
 39%|███▊      | 86773/225000 [161:15:43<208:43:28,  5.44s/it]
 39%|███▊      | 86774/225000 [161:15:45<172:02:31,  4.48s/it]
 39%|███▊      | 86775/225000 [161:15:49<166:16:20,  4.33s/it]
 39%|███▊      | 86776/225000 [161:15:53<163:37:59,  4.26s/it]
 39%|███▊      | 86777/225000 [161:15:56<144:39:18,  3.77s/it]
 39%|███▊      | 86778/225000 [161:15:59<135:39:15,  3.53s/it]
 39%|███▊      | 86779/225000 [161:16:01<127:03:06,  3.31s/it]
 39%|███▊      | 86780/225000 [161:16:04<123:19:33,  3.21s/it]
 39%|███▊      | 86781/225000 [161:16:10<154:03:51,  4.01s/it]
 39%|███▊      | 86782/225000 [161:16:18<202:24:42,  5.27s/it]
 39%|███▊      | 86783/225000 [161:16:21<168:55:39,  4.40s/it]
 39%|███▊      | 86784/225000 [161:16:23<147:16:15,  3.84s/it]
 39%|███▊      | 86785/225000 [161:16:26<130:07:20,  3.39s/it]
 39%|███▊      | 86786/225000 [161:16:28<122:03:43,  3.18s/it]
 39%|███▊      | 86787/225000 [161:16:32<122:27:56,  3.19s/it]
 39%|███▊      | 86788/225000 [161:16:35<121:48:54,  3.17s/it]
 39%|███▊      | 86789/225000 [161:16:38<124:30:08,  3.24s/it]
 39%|███▊      | 86790/225000 [161:16:42<128:27:26,  3.35s/it]
 39%|███▊      | 86791/225000 [161:16:45<129:04:21,  3.36s/it]
 39%|███▊      | 86792/225000 [161:16:48<128:48:59,  3.36s/it]
 39%|███▊      | 86793/225000 [161:16:52<127:51:52,  3.33s/it]
 39%|███▊      | 86794/225000 [161:16:55<131:06:08,  3.41s/it]
 39%|███▊      | 86795/225000 [161:16:59<132:25:15,  3.45s/it]
 39%|███▊      | 86796/225000 [161:17:04<152:05:23,  3.96s/it]
 39%|███▊      | 86797/225000 [161:17:13<204:43:32,  5.33s/it]
 39%|███▊      | 86798/225000 [161:17:18<208:55:10,  5.44s/it]
 39%|███▊      | 86799/225000 [161:17:22<184:01:25,  4.79s/it]
 39%|███▊      | 86800/225000 [161:17:24<161:05:58,  4.20s/it]
                                                              
{'loss': '0.4477', 'grad_norm': '4.779', 'learning_rate': '3.586e-05', 'epoch': '0.3858', 'num_input_tokens_seen': 264618296, 'train_runtime': '5.806e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86800/225000 [161:17:24<161:05:58,  4.20s/it]
 39%|███▊      | 86801/225000 [161:17:27<149:13:59,  3.89s/it]
 39%|███▊      | 86802/225000 [161:17:30<137:50:55,  3.59s/it]
 39%|███▊      | 86803/225000 [161:17:39<190:20:41,  4.96s/it]
 39%|███▊      | 86804/225000 [161:17:45<208:10:13,  5.42s/it]
 39%|███▊      | 86805/225000 [161:17:48<184:11:37,  4.80s/it]
 39%|███▊      | 86806/225000 [161:17:51<164:11:02,  4.28s/it]
 39%|███▊      | 86807/225000 [161:17:55<153:02:32,  3.99s/it]
 39%|███▊      | 86808/225000 [161:17:58<147:54:34,  3.85s/it]
 39%|███▊      | 86809/225000 [161:18:02<144:57:30,  3.78s/it]
 39%|███▊      | 86810/225000 [161:18:05<137:34:55,  3.58s/it]
 39%|███▊      | 86811/225000 [161:18:09<138:24:27,  3.61s/it]
 39%|███▊      | 86812/225000 [161:18:12<132:27:29,  3.45s/it]
 39%|███▊      | 86813/225000 [161:18:15<129:49:40,  3.38s/it]
 39%|███▊      | 86814/225000 [161:18:19<139:21:51,  3.63s/it]
 39%|███▊      | 86815/225000 [161:18:23<140:07:06,  3.65s/it]
 39%|███▊      | 86816/225000 [161:18:27<139:59:26,  3.65s/it]
 39%|███▊      | 86817/225000 [161:18:35<195:04:54,  5.08s/it]
 39%|███▊      | 86818/225000 [161:18:40<193:36:08,  5.04s/it]
 39%|███▊      | 86819/225000 [161:18:42<161:39:16,  4.21s/it]
 39%|███▊      | 86820/225000 [161:18:45<147:21:48,  3.84s/it]
 39%|███▊      | 86821/225000 [161:18:49<142:19:04,  3.71s/it]
 39%|███▊      | 86822/225000 [161:18:53<152:19:04,  3.97s/it]
 39%|███▊      | 86823/225000 [161:19:01<195:57:14,  5.11s/it]
 39%|███▊      | 86824/225000 [161:19:04<167:05:24,  4.35s/it]
 39%|███▊      | 86825/225000 [161:19:06<146:00:25,  3.80s/it]
 39%|███▊      | 86826/225000 [161:19:13<179:30:33,  4.68s/it]
 39%|███▊      | 86827/225000 [161:19:16<167:22:40,  4.36s/it]
 39%|███▊      | 86828/225000 [161:19:19<144:37:40,  3.77s/it]
 39%|███▊      | 86829/225000 [161:19:21<128:45:32,  3.35s/it]
 39%|███▊      | 86830/225000 [161:19:24<122:20:15,  3.19s/it]
 39%|███▊      | 86831/225000 [161:19:27<116:30:07,  3.04s/it]
 39%|███▊      | 86832/225000 [161:19:29<114:09:51,  2.97s/it]
 39%|███▊      | 86833/225000 [161:19:36<158:44:35,  4.14s/it]
 39%|███▊      | 86834/225000 [161:19:43<187:54:11,  4.90s/it]
 39%|███▊      | 86835/225000 [161:19:47<173:14:43,  4.51s/it]
 39%|███▊      | 86836/225000 [161:19:50<156:13:38,  4.07s/it]
 39%|███▊      | 86837/225000 [161:19:53<145:07:59,  3.78s/it]
 39%|███▊      | 86838/225000 [161:19:56<135:11:43,  3.52s/it]
 39%|███▊      | 86839/225000 [161:19:59<131:31:18,  3.43s/it]
 39%|███▊      | 86840/225000 [161:20:02<129:49:03,  3.38s/it]
 39%|███▊      | 86841/225000 [161:20:05<123:49:55,  3.23s/it]
 39%|███▊      | 86842/225000 [161:20:10<139:08:11,  3.63s/it]
 39%|███▊      | 86843/225000 [161:20:19<207:07:15,  5.40s/it]
 39%|███▊      | 86844/225000 [161:20:22<178:24:04,  4.65s/it]
 39%|███▊      | 86845/225000 [161:20:25<163:32:19,  4.26s/it]
 39%|███▊      | 86846/225000 [161:20:28<148:54:56,  3.88s/it]
 39%|███▊      | 86847/225000 [161:20:31<138:37:37,  3.61s/it]
 39%|███▊      | 86848/225000 [161:20:34<129:37:20,  3.38s/it]
 39%|███▊      | 86849/225000 [161:20:37<126:29:34,  3.30s/it]
 39%|███▊      | 86850/225000 [161:20:40<121:44:22,  3.17s/it]
                                                              
{'loss': '0.4283', 'grad_norm': '4.9', 'learning_rate': '3.584e-05', 'epoch': '0.386', 'num_input_tokens_seen': 264704648, 'train_runtime': '5.808e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86850/225000 [161:20:40<121:44:22,  3.17s/it]
 39%|███▊      | 86851/225000 [161:20:49<186:46:54,  4.87s/it]
 39%|███▊      | 86852/225000 [161:20:54<189:17:12,  4.93s/it]
 39%|███▊      | 86853/225000 [161:20:56<158:55:59,  4.14s/it]
 39%|███▊      | 86854/225000 [161:20:59<145:01:07,  3.78s/it]
 39%|███▊      | 86855/225000 [161:21:02<133:14:37,  3.47s/it]
 39%|███▊      | 86856/225000 [161:21:05<125:07:01,  3.26s/it]
 39%|███▊      | 86857/225000 [161:21:08<120:17:20,  3.13s/it]
 39%|███▊      | 86858/225000 [161:21:15<174:01:43,  4.54s/it]
 39%|███▊      | 86859/225000 [161:21:22<191:40:57,  5.00s/it]
 39%|███▊      | 86860/225000 [161:21:24<163:20:43,  4.26s/it]
 39%|███▊      | 86861/225000 [161:21:27<145:09:09,  3.78s/it]
 39%|███▊      | 86862/225000 [161:21:29<131:14:58,  3.42s/it]
 39%|███▊      | 86863/225000 [161:21:33<129:03:52,  3.36s/it]
 39%|███▊      | 86864/225000 [161:21:38<154:06:14,  4.02s/it]
 39%|███▊      | 86865/225000 [161:21:47<213:12:48,  5.56s/it]
 39%|███▊      | 86866/225000 [161:21:50<182:05:07,  4.75s/it]
 39%|███▊      | 86867/225000 [161:21:52<152:49:36,  3.98s/it]
 39%|███▊      | 86868/225000 [161:21:55<141:21:41,  3.68s/it]
 39%|███▊      | 86869/225000 [161:21:58<130:45:10,  3.41s/it]
 39%|███▊      | 86870/225000 [161:22:01<125:35:17,  3.27s/it]
 39%|███▊      | 86871/225000 [161:22:04<123:18:54,  3.21s/it]
 39%|███▊      | 86872/225000 [161:22:11<169:41:35,  4.42s/it]
 39%|███▊      | 86873/225000 [161:22:17<183:17:51,  4.78s/it]
 39%|███▊      | 86874/225000 [161:22:20<164:25:16,  4.29s/it]
 39%|███▊      | 86875/225000 [161:22:23<154:45:24,  4.03s/it]
 39%|███▊      | 86876/225000 [161:22:26<142:48:49,  3.72s/it]
 39%|███▊      | 86877/225000 [161:22:29<134:23:02,  3.50s/it]
 39%|███▊      | 86878/225000 [161:22:32<127:02:18,  3.31s/it]
 39%|███▊      | 86879/225000 [161:22:35<124:33:05,  3.25s/it]
 39%|███▊      | 86880/225000 [161:22:43<177:22:31,  4.62s/it]
 39%|███▊      | 86881/225000 [161:22:48<174:18:53,  4.54s/it]
 39%|███▊      | 86882/225000 [161:22:50<152:10:03,  3.97s/it]
 39%|███▊      | 86883/225000 [161:22:57<188:44:34,  4.92s/it]
 39%|███▊      | 86884/225000 [161:23:03<191:19:24,  4.99s/it]
 39%|███▊      | 86885/225000 [161:23:05<161:04:22,  4.20s/it]
 39%|███▊      | 86886/225000 [161:23:08<145:36:38,  3.80s/it]
 39%|███▊      | 86887/225000 [161:23:11<135:30:37,  3.53s/it]
 39%|███▊      | 86888/225000 [161:23:13<125:52:31,  3.28s/it]
 39%|███▊      | 86889/225000 [161:23:16<121:59:09,  3.18s/it]
 39%|███▊      | 86890/225000 [161:23:23<158:12:42,  4.12s/it]
 39%|███▊      | 86891/225000 [161:23:30<200:11:52,  5.22s/it]
 39%|███▊      | 86892/225000 [161:23:33<170:24:41,  4.44s/it]
 39%|███▊      | 86893/225000 [161:23:36<148:58:48,  3.88s/it]
 39%|███▊      | 86894/225000 [161:23:38<137:03:47,  3.57s/it]
 39%|███▊      | 86895/225000 [161:23:45<174:15:24,  4.54s/it]
 39%|███▊      | 86896/225000 [161:23:49<161:52:03,  4.22s/it]
 39%|███▊      | 86897/225000 [161:23:52<150:40:58,  3.93s/it]
 39%|███▊      | 86898/225000 [161:23:56<147:07:06,  3.84s/it]
 39%|███▊      | 86899/225000 [161:23:59<141:15:13,  3.68s/it]
 39%|███▊      | 86900/225000 [161:24:03<141:35:01,  3.69s/it]
                                                              
{'loss': '0.4329', 'grad_norm': '4.301', 'learning_rate': '3.582e-05', 'epoch': '0.3862', 'num_input_tokens_seen': 264790856, 'train_runtime': '5.81e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86900/225000 [161:24:03<141:35:01,  3.69s/it]
 39%|███▊      | 86901/225000 [161:24:06<142:17:31,  3.71s/it]
 39%|███▊      | 86902/225000 [161:24:10<137:44:57,  3.59s/it]
 39%|███▊      | 86903/225000 [161:24:13<138:02:22,  3.60s/it]
 39%|███▊      | 86904/225000 [161:24:17<137:40:57,  3.59s/it]
 39%|███▊      | 86905/225000 [161:24:20<136:12:14,  3.55s/it]
 39%|███▊      | 86906/225000 [161:24:24<132:18:33,  3.45s/it]
 39%|███▊      | 86907/225000 [161:24:27<129:38:46,  3.38s/it]
 39%|███▊      | 86908/225000 [161:24:31<134:49:51,  3.51s/it]
 39%|███▊      | 86909/225000 [161:24:34<133:50:01,  3.49s/it]
 39%|███▊      | 86910/225000 [161:24:42<188:25:56,  4.91s/it]
 39%|███▊      | 86911/225000 [161:24:51<226:53:22,  5.92s/it]
 39%|███▊      | 86912/225000 [161:24:54<196:29:00,  5.12s/it]
 39%|███▊      | 86913/225000 [161:24:59<194:36:17,  5.07s/it]
 39%|███▊      | 86914/225000 [161:25:01<163:44:25,  4.27s/it]
 39%|███▊      | 86915/225000 [161:25:09<205:21:59,  5.35s/it]
 39%|███▊      | 86916/225000 [161:25:12<175:27:33,  4.57s/it]
 39%|███▊      | 86917/225000 [161:25:14<149:19:19,  3.89s/it]
 39%|███▊      | 86918/225000 [161:25:19<160:44:01,  4.19s/it]
 39%|███▊      | 86919/225000 [161:25:24<170:35:17,  4.45s/it]
 39%|███▊      | 86920/225000 [161:25:27<155:27:44,  4.05s/it]
 39%|███▊      | 86921/225000 [161:25:30<142:29:50,  3.72s/it]
 39%|███▊      | 86922/225000 [161:25:33<133:51:56,  3.49s/it]
 39%|███▊      | 86923/225000 [161:25:36<130:10:26,  3.39s/it]
 39%|███▊      | 86924/225000 [161:25:39<128:00:46,  3.34s/it]
 39%|███▊      | 86925/225000 [161:25:43<127:58:58,  3.34s/it]
 39%|███▊      | 86926/225000 [161:25:46<130:23:16,  3.40s/it]
 39%|███▊      | 86927/225000 [161:25:50<129:21:57,  3.37s/it]
 39%|███▊      | 86928/225000 [161:25:59<193:38:29,  5.05s/it]
 39%|███▊      | 86929/225000 [161:26:03<188:08:34,  4.91s/it]
 39%|███▊      | 86930/225000 [161:26:07<171:10:41,  4.46s/it]
 39%|███▊      | 86931/225000 [161:26:10<160:43:33,  4.19s/it]
 39%|███▊      | 86932/225000 [161:26:14<153:32:37,  4.00s/it]
 39%|███▊      | 86933/225000 [161:26:18<152:14:52,  3.97s/it]
 39%|███▊      | 86934/225000 [161:26:22<154:36:23,  4.03s/it]
 39%|███▊      | 86935/225000 [161:26:25<146:03:18,  3.81s/it]
 39%|███▊      | 86936/225000 [161:26:28<139:13:37,  3.63s/it]
 39%|███▊      | 86937/225000 [161:26:32<135:37:24,  3.54s/it]
 39%|███▊      | 86938/225000 [161:26:35<136:56:15,  3.57s/it]
 39%|███▊      | 86939/225000 [161:26:39<133:50:43,  3.49s/it]
 39%|███▊      | 86940/225000 [161:26:42<131:04:17,  3.42s/it]
 39%|███▊      | 86941/225000 [161:26:45<130:44:23,  3.41s/it]
 39%|███▊      | 86942/225000 [161:26:49<130:12:11,  3.40s/it]
 39%|███▊      | 86943/225000 [161:26:55<164:32:33,  4.29s/it]
 39%|███▊      | 86944/225000 [161:27:02<201:50:59,  5.26s/it]
 39%|███▊      | 86945/225000 [161:27:05<173:48:08,  4.53s/it]
 39%|███▊      | 86946/225000 [161:27:08<152:50:36,  3.99s/it]
 39%|███▊      | 86947/225000 [161:27:11<141:37:18,  3.69s/it]
 39%|███▊      | 86948/225000 [161:27:14<130:19:45,  3.40s/it]
 39%|███▊      | 86949/225000 [161:27:17<130:26:44,  3.40s/it]
 39%|███▊      | 86950/225000 [161:27:21<135:02:48,  3.52s/it]
                                                              
{'loss': '0.4106', 'grad_norm': '4.753', 'learning_rate': '3.581e-05', 'epoch': '0.3864', 'num_input_tokens_seen': 264875928, 'train_runtime': '5.812e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 86950/225000 [161:27:21<135:02:48,  3.52s/it]
 39%|███▊      | 86951/225000 [161:27:30<197:13:53,  5.14s/it]
 39%|███▊      | 86952/225000 [161:27:33<178:22:37,  4.65s/it]
 39%|███▊      | 86953/225000 [161:27:37<162:35:59,  4.24s/it]
 39%|███▊      | 86954/225000 [161:27:40<151:15:41,  3.94s/it]
 39%|███▊      | 86955/225000 [161:27:43<139:49:59,  3.65s/it]
 39%|███▊      | 86956/225000 [161:27:46<139:18:37,  3.63s/it]
 39%|███▊      | 86957/225000 [161:27:49<131:22:10,  3.43s/it]
 39%|███▊      | 86958/225000 [161:27:52<126:09:52,  3.29s/it]
 39%|███▊      | 86959/225000 [161:27:56<124:53:17,  3.26s/it]
 39%|███▊      | 86960/225000 [161:28:04<189:21:03,  4.94s/it]
 39%|███▊      | 86961/225000 [161:28:10<192:08:42,  5.01s/it]
 39%|███▊      | 86962/225000 [161:28:12<163:53:36,  4.27s/it]
 39%|███▊      | 86963/225000 [161:28:16<155:45:45,  4.06s/it]
 39%|███▊      | 86964/225000 [161:28:19<151:32:27,  3.95s/it]
 39%|███▊      | 86965/225000 [161:28:23<151:51:48,  3.96s/it]
 39%|███▊      | 86966/225000 [161:28:27<148:56:47,  3.88s/it]
 39%|███▊      | 86967/225000 [161:28:31<143:39:18,  3.75s/it]
 39%|███▊      | 86968/225000 [161:28:39<197:03:36,  5.14s/it]
 39%|███▊      | 86969/225000 [161:28:41<165:30:51,  4.32s/it]
 39%|███▊      | 86970/225000 [161:28:44<145:17:59,  3.79s/it]
 39%|███▊      | 86971/225000 [161:28:47<132:52:54,  3.47s/it]
 39%|███▊      | 86972/225000 [161:28:50<131:26:51,  3.43s/it]
 39%|███▊      | 86973/225000 [161:28:54<134:57:47,  3.52s/it]
 39%|███▊      | 86974/225000 [161:28:57<129:31:43,  3.38s/it]
 39%|███▊      | 86975/225000 [161:29:00<131:52:42,  3.44s/it]
 39%|███▊      | 86976/225000 [161:29:04<131:06:27,  3.42s/it]
 39%|███▊      | 86977/225000 [161:29:10<163:41:24,  4.27s/it]
 39%|███▊      | 86978/225000 [161:29:18<212:30:49,  5.54s/it]
 39%|███▊      | 86979/225000 [161:29:22<194:04:36,  5.06s/it]
 39%|███▊      | 86980/225000 [161:29:25<165:09:03,  4.31s/it]
 39%|███▊      | 86981/225000 [161:29:27<144:37:17,  3.77s/it]
 39%|███▊      | 86982/225000 [161:29:30<131:21:37,  3.43s/it]
 39%|███▊      | 86983/225000 [161:29:39<189:35:50,  4.95s/it]
 39%|███▊      | 86984/225000 [161:29:42<166:58:52,  4.36s/it]
 39%|███▊      | 86985/225000 [161:29:44<148:05:24,  3.86s/it]
 39%|███▊      | 86986/225000 [161:29:47<136:07:16,  3.55s/it]
 39%|███▊      | 86987/225000 [161:29:50<127:27:16,  3.32s/it]
 39%|███▊      | 86988/225000 [161:29:52<119:38:57,  3.12s/it]
 39%|███▊      | 86989/225000 [161:30:00<171:22:26,  4.47s/it]
 39%|███▊      | 86990/225000 [161:30:04<161:31:52,  4.21s/it]
 39%|███▊      | 86991/225000 [161:30:06<144:37:19,  3.77s/it]
 39%|███▊      | 86992/225000 [161:30:09<132:16:27,  3.45s/it]
 39%|███▊      | 86993/225000 [161:30:12<126:46:47,  3.31s/it]
 39%|███▊      | 86994/225000 [161:30:20<182:22:16,  4.76s/it]
 39%|███▊      | 86995/225000 [161:30:24<173:56:19,  4.54s/it]
 39%|███▊      | 86996/225000 [161:30:27<150:10:06,  3.92s/it]
 39%|███▊      | 86997/225000 [161:30:29<135:09:12,  3.53s/it]
 39%|███▊      | 86998/225000 [161:30:36<165:00:34,  4.30s/it]
 39%|███▊      | 86999/225000 [161:30:39<160:20:27,  4.18s/it]
 39%|███▊      | 87000/225000 [161:30:42<143:56:14,  3.75s/it]
                                                              
{'loss': '0.4397', 'grad_norm': '6.641', 'learning_rate': '3.579e-05', 'epoch': '0.3867', 'num_input_tokens_seen': 264962104, 'train_runtime': '5.814e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 87000/225000 [161:30:42<143:56:14,  3.75s/it]
 39%|███▊      | 87001/225000 [161:30:45<131:51:14,  3.44s/it]
 39%|███▊      | 87002/225000 [161:30:49<139:50:29,  3.65s/it]
 39%|███▊      | 87003/225000 [161:30:56<172:50:45,  4.51s/it]
 39%|███▊      | 87004/225000 [161:30:58<149:50:15,  3.91s/it]
 39%|███▊      | 87005/225000 [161:31:01<140:31:40,  3.67s/it]
 39%|███▊      | 87006/225000 [161:31:05<137:04:25,  3.58s/it]
 39%|███▊      | 87007/225000 [161:31:07<130:01:05,  3.39s/it]
 39%|███▊      | 87008/225000 [161:31:10<124:59:12,  3.26s/it]
 39%|███▊      | 87009/225000 [161:31:13<122:41:14,  3.20s/it]
 39%|███▊      | 87010/225000 [161:31:17<124:12:19,  3.24s/it]
 39%|███▊      | 87011/225000 [161:31:20<122:57:58,  3.21s/it]
 39%|███▊      | 87012/225000 [161:31:29<185:57:25,  4.85s/it]
 39%|███▊      | 87013/225000 [161:31:33<181:18:37,  4.73s/it]
 39%|███▊      | 87014/225000 [161:31:36<156:47:47,  4.09s/it]
 39%|███▊      | 87015/225000 [161:31:39<145:44:45,  3.80s/it]
 39%|███▊      | 87016/225000 [161:31:42<141:09:24,  3.68s/it]
 39%|███▊      | 87017/225000 [161:31:45<133:04:00,  3.47s/it]
 39%|███▊      | 87018/225000 [161:31:48<130:52:43,  3.41s/it]
 39%|███▊      | 87019/225000 [161:31:54<155:15:32,  4.05s/it]
 39%|███▊      | 87020/225000 [161:32:04<218:00:23,  5.69s/it]
 39%|███▊      | 87021/225000 [161:32:06<185:39:55,  4.84s/it]
 39%|███▊      | 87022/225000 [161:32:09<158:41:47,  4.14s/it]
 39%|███▊      | 87023/225000 [161:32:12<149:51:32,  3.91s/it]
 39%|███▊      | 87024/225000 [161:32:15<138:45:49,  3.62s/it]
 39%|███▊      | 87025/225000 [161:32:19<135:32:17,  3.54s/it]
 39%|███▊      | 87026/225000 [161:32:22<132:51:03,  3.47s/it]
 39%|███▊      | 87027/225000 [161:32:25<131:53:44,  3.44s/it]
 39%|███▊      | 87028/225000 [161:32:29<134:18:59,  3.50s/it]
 39%|███▊      | 87029/225000 [161:32:38<193:39:17,  5.05s/it]
 39%|███▊      | 87030/225000 [161:32:43<198:34:41,  5.18s/it]
 39%|███▊      | 87031/225000 [161:32:46<169:02:56,  4.41s/it]
 39%|███▊      | 87032/225000 [161:32:48<150:59:36,  3.94s/it]
 39%|███▊      | 87033/225000 [161:32:55<177:06:02,  4.62s/it]
 39%|███▊      | 87034/225000 [161:32:59<177:52:21,  4.64s/it]
 39%|███▊      | 87035/225000 [161:33:03<160:41:56,  4.19s/it]
 39%|███▊      | 87036/225000 [161:33:06<149:54:30,  3.91s/it]
 39%|███▊      | 87037/225000 [161:33:10<149:00:19,  3.89s/it]
 39%|███▊      | 87038/225000 [161:33:13<142:52:54,  3.73s/it]
 39%|███▊      | 87039/225000 [161:33:16<136:31:04,  3.56s/it]
 39%|███▊      | 87040/225000 [161:33:19<130:24:02,  3.40s/it]
 39%|███▊      | 87041/225000 [161:33:29<201:38:23,  5.26s/it]
 39%|███▊      | 87042/225000 [161:33:35<208:47:33,  5.45s/it]
 39%|███▊      | 87043/225000 [161:33:37<176:16:14,  4.60s/it]
 39%|███▊      | 87044/225000 [161:33:41<163:32:08,  4.27s/it]
 39%|███▊      | 87045/225000 [161:33:47<186:49:04,  4.88s/it]
 39%|███▊      | 87046/225000 [161:33:50<161:44:37,  4.22s/it]
 39%|███▊      | 87047/225000 [161:33:53<146:44:30,  3.83s/it]
 39%|███▊      | 87048/225000 [161:34:00<188:17:01,  4.91s/it]
 39%|███▊      | 87049/225000 [161:34:04<177:05:57,  4.62s/it]
 39%|███▊      | 87050/225000 [161:34:07<152:52:11,  3.99s/it]
                                                              
{'loss': '0.4306', 'grad_norm': '5.842', 'learning_rate': '3.577e-05', 'epoch': '0.3869', 'num_input_tokens_seen': 265049224, 'train_runtime': '5.816e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 87050/225000 [161:34:07<152:52:11,  3.99s/it]
 39%|███▊      | 87051/225000 [161:34:09<134:35:13,  3.51s/it]
 39%|███▊      | 87052/225000 [161:34:12<126:37:46,  3.30s/it]
 39%|███▊      | 87053/225000 [161:34:15<129:56:32,  3.39s/it]
 39%|███▊      | 87054/225000 [161:34:18<125:48:15,  3.28s/it]
 39%|███▊      | 87055/225000 [161:34:22<126:21:37,  3.30s/it]
 39%|███▊      | 87056/225000 [161:34:25<123:15:30,  3.22s/it]
 39%|███▊      | 87057/225000 [161:34:28<123:10:16,  3.21s/it]
 39%|███▊      | 87058/225000 [161:34:36<183:45:20,  4.80s/it]
 39%|███▊      | 87059/225000 [161:34:43<206:02:54,  5.38s/it]
 39%|███▊      | 87060/225000 [161:34:46<177:44:42,  4.64s/it]
 39%|███▊      | 87061/225000 [161:34:49<154:38:00,  4.04s/it]
 39%|███▊      | 87062/225000 [161:34:51<138:58:01,  3.63s/it]
 39%|███▊      | 87063/225000 [161:34:59<179:10:23,  4.68s/it]
 39%|███▊      | 87064/225000 [161:35:02<163:26:22,  4.27s/it]
 39%|███▊      | 87065/225000 [161:35:05<148:07:23,  3.87s/it]
 39%|███▊      | 87066/225000 [161:35:08<141:38:50,  3.70s/it]
 39%|███▊      | 87067/225000 [161:35:11<133:42:24,  3.49s/it]
 39%|███▊      | 87068/225000 [161:35:14<132:40:07,  3.46s/it]
 39%|███▊      | 87069/225000 [161:35:18<134:26:03,  3.51s/it]
 39%|███▊      | 87070/225000 [161:35:21<128:04:59,  3.34s/it]
 39%|███▊      | 87071/225000 [161:35:29<182:31:20,  4.76s/it]
 39%|███▊      | 87072/225000 [161:35:34<186:34:58,  4.87s/it]
 39%|███▊      | 87073/225000 [161:35:37<161:23:52,  4.21s/it]
 39%|███▊      | 87074/225000 [161:35:40<153:48:09,  4.01s/it]
 39%|███▊      | 87075/225000 [161:35:43<137:33:37,  3.59s/it]
 39%|███▊      | 87076/225000 [161:35:47<136:18:10,  3.56s/it]
 39%|███▊      | 87077/225000 [161:35:55<189:40:06,  4.95s/it]
 39%|███▊      | 87078/225000 [161:35:59<179:19:04,  4.68s/it]
 39%|███▊      | 87079/225000 [161:36:02<157:55:12,  4.12s/it]
 39%|███▊      | 87080/225000 [161:36:04<141:25:41,  3.69s/it]
 39%|███▊      | 87081/225000 [161:36:07<133:10:47,  3.48s/it]
 39%|███▊      | 87082/225000 [161:36:12<142:28:13,  3.72s/it]
 39%|███▊      | 87083/225000 [161:36:18<176:39:20,  4.61s/it]
 39%|███▊      | 87084/225000 [161:36:22<162:31:36,  4.24s/it]
 39%|███▊      | 87085/225000 [161:36:25<148:23:29,  3.87s/it]
 39%|███▊      | 87086/225000 [161:36:28<138:37:49,  3.62s/it]
 39%|███▊      | 87087/225000 [161:36:31<132:24:41,  3.46s/it]
 39%|███▊      | 87088/225000 [161:36:34<129:50:49,  3.39s/it]
 39%|███▊      | 87089/225000 [161:36:37<127:13:09,  3.32s/it]
 39%|███▊      | 87090/225000 [161:36:40<124:26:32,  3.25s/it]
 39%|███▊      | 87091/225000 [161:36:48<177:23:29,  4.63s/it]
 39%|███▊      | 87092/225000 [161:36:57<226:47:27,  5.92s/it]
 39%|███▊      | 87093/225000 [161:36:59<186:51:23,  4.88s/it]
 39%|███▊      | 87094/225000 [161:37:02<159:48:15,  4.17s/it]
 39%|███▊      | 87095/225000 [161:37:08<179:41:58,  4.69s/it]
 39%|███▊      | 87096/225000 [161:37:11<162:00:26,  4.23s/it]
 39%|███▊      | 87097/225000 [161:37:14<147:59:41,  3.86s/it]
 39%|███▊      | 87098/225000 [161:37:17<136:24:02,  3.56s/it]
 39%|███▊      | 87099/225000 [161:37:20<128:45:34,  3.36s/it]
 39%|███▊      | 87100/225000 [161:37:23<121:58:37,  3.18s/it]
                                                              
{'loss': '0.4185', 'grad_norm': '5.206', 'learning_rate': '3.576e-05', 'epoch': '0.3871', 'num_input_tokens_seen': 265135000, 'train_runtime': '5.818e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 87100/225000 [161:37:23<121:58:37,  3.18s/it]
 39%|███▊      | 87101/225000 [161:37:26<124:11:13,  3.24s/it]
 39%|███▊      | 87102/225000 [161:37:33<172:08:14,  4.49s/it]
 39%|███▊      | 87103/225000 [161:37:40<200:36:20,  5.24s/it]
 39%|███▊      | 87104/225000 [161:37:43<169:05:44,  4.41s/it]
 39%|███▊      | 87105/225000 [161:37:46<152:22:48,  3.98s/it]
 39%|███▊      | 87106/225000 [161:37:49<147:49:45,  3.86s/it]
 39%|███▊      | 87107/225000 [161:37:53<145:36:46,  3.80s/it]
 39%|███▊      | 87108/225000 [161:37:57<141:40:12,  3.70s/it]
 39%|███▊      | 87109/225000 [161:38:01<147:03:11,  3.84s/it]
 39%|███▊      | 87110/225000 [161:38:04<139:45:46,  3.65s/it]
 39%|███▊      | 87111/225000 [161:38:07<134:20:08,  3.51s/it]
 39%|███▊      | 87112/225000 [161:38:11<135:11:01,  3.53s/it]
 39%|███▊      | 87113/225000 [161:38:15<141:51:46,  3.70s/it]
 39%|███▊      | 87114/225000 [161:38:18<137:06:13,  3.58s/it]
 39%|███▊      | 87115/225000 [161:38:22<136:16:14,  3.56s/it]
 39%|███▊      | 87116/225000 [161:38:25<140:08:21,  3.66s/it]
 39%|███▊      | 87117/225000 [161:38:29<141:52:59,  3.70s/it]
 39%|███▊      | 87118/225000 [161:38:36<179:23:40,  4.68s/it]
 39%|███▊      | 87119/225000 [161:38:42<197:09:41,  5.15s/it]
 39%|███▊      | 87120/225000 [161:38:46<177:15:15,  4.63s/it]
 39%|███▊      | 87121/225000 [161:38:49<160:52:48,  4.20s/it]
 39%|███▊      | 87122/225000 [161:38:53<159:18:48,  4.16s/it]
 39%|███▊      | 87123/225000 [161:38:58<167:26:48,  4.37s/it]
 39%|███▊      | 87124/225000 [161:39:03<171:29:44,  4.48s/it]
 39%|███▊      | 87125/225000 [161:39:07<163:53:14,  4.28s/it]
 39%|███▊      | 87126/225000 [161:39:10<151:19:34,  3.95s/it]
 39%|███▊      | 87127/225000 [161:39:13<142:59:19,  3.73s/it]
 39%|███▊      | 87128/225000 [161:39:18<160:31:21,  4.19s/it]
 39%|███▊      | 87129/225000 [161:39:23<167:10:52,  4.37s/it]
 39%|███▊      | 87130/225000 [161:39:26<150:44:30,  3.94s/it]
 39%|███▊      | 87131/225000 [161:39:29<141:20:54,  3.69s/it]
 39%|███▊      | 87132/225000 [161:39:32<137:39:11,  3.59s/it]
 39%|███▊      | 87133/225000 [161:39:35<131:07:19,  3.42s/it]
 39%|███▊      | 87134/225000 [161:39:43<178:24:15,  4.66s/it]
 39%|███▊      | 87135/225000 [161:39:51<212:31:57,  5.55s/it]
 39%|███▊      | 87136/225000 [161:39:53<174:36:43,  4.56s/it]
 39%|███▊      | 87137/225000 [161:39:55<150:27:25,  3.93s/it]
 39%|███▊      | 87138/225000 [161:39:58<140:39:30,  3.67s/it]
 39%|███▊      | 87139/225000 [161:40:06<189:27:38,  4.95s/it]
 39%|███▊      | 87140/225000 [161:40:09<165:33:05,  4.32s/it]
 39%|███▊      | 87141/225000 [161:40:12<144:06:35,  3.76s/it]
 39%|███▊      | 87142/225000 [161:40:14<130:33:29,  3.41s/it]
 39%|███▊      | 87143/225000 [161:40:17<123:44:26,  3.23s/it]
 39%|███▊      | 87144/225000 [161:40:26<190:05:46,  4.96s/it]
 39%|███▊      | 87145/225000 [161:40:29<165:44:35,  4.33s/it]
 39%|███▊      | 87146/225000 [161:40:32<147:22:09,  3.85s/it]
 39%|███▊      | 87147/225000 [161:40:34<131:12:32,  3.43s/it]
 39%|███▊      | 87148/225000 [161:40:37<121:41:33,  3.18s/it]
 39%|███▊      | 87149/225000 [161:40:41<136:33:30,  3.57s/it]
 39%|███▊      | 87150/225000 [161:40:49<182:11:57,  4.76s/it]
                                                              
{'loss': '0.4309', 'grad_norm': '6.608', 'learning_rate': '3.574e-05', 'epoch': '0.3873', 'num_input_tokens_seen': 265220144, 'train_runtime': '5.82e+05', 'train_tokens_per_second': '455.7'}

 39%|███▊      | 87150/225000 [161:40:49<182:11:57,  4.76s/it]
 39%|███▊      | 87151/225000 [161:40:51<157:52:07,  4.12s/it]
 39%|███▊      | 87152/225000 [161:40:54<143:54:07,  3.76s/it]
 39%|███▊      | 87153/225000 [161:40:57<127:58:20,  3.34s/it]
 39%|███▊      | 87154/225000 [161:41:04<170:14:28,  4.45s/it]
 39%|███▊      | 87155/225000 [161:41:06<149:10:36,  3.90s/it]
 39%|███▊      | 87156/225000 [161:41:09<135:32:48,  3.54s/it]
 39%|███▊      | 87157/225000 [161:41:12<125:53:27,  3.29s/it]
 39%|███▊      | 87158/225000 [161:41:15<124:01:06,  3.24s/it]
 39%|███▊      | 87159/225000 [161:41:17<118:08:41,  3.09s/it]
 39%|███▊      | 87160/225000 [161:41:21<124:05:25,  3.24s/it]
 39%|███▊      | 87161/225000 [161:41:31<196:21:48,  5.13s/it]
 39%|███▊      | 87162/225000 [161:41:35<182:56:06,  4.78s/it]
 39%|███▊      | 87163/225000 [161:41:37<157:03:30,  4.10s/it]
 39%|███▊      | 87164/225000 [161:41:40<145:22:27,  3.80s/it]
 39%|███▊      | 87165/225000 [161:41:43<138:23:25,  3.61s/it]
 39%|███▊      | 87166/225000 [161:41:51<185:56:21,  4.86s/it]
 39%|███▊      | 87167/225000 [161:41:54<162:46:15,  4.25s/it]
 39%|███▊      | 87168/225000 [161:41:57<147:44:19,  3.86s/it]
 39%|███▊      | 87169/225000 [161:42:00<134:48:05,  3.52s/it]
 39%|███▊      | 87170/225000 [161:42:03<127:40:48,  3.33s/it]
 39%|███▊      | 87171/225000 [161:42:05<123:06:12,  3.22s/it]
 39%|███▊      | 87172/225000 [161:42:08<120:39:43,  3.15s/it]
 39%|███▊      | 87173/225000 [161:42:12<121:59:21,  3.19s/it]
 39%|███▊      | 87174/225000 [161:42:15<119:20:48,  3.12s/it]
 39%|███▊      | 87175/225000 [161:42:21<155:52:19,  4.07s/it]
 39%|███▊      | 87176/225000 [161:42:30<209:09:40,  5.46s/it]
 39%|███▊      | 87177/225000 [161:42:33<181:51:34,  4.75s/it]
 39%|███▊      | 87178/225000 [161:42:35<153:55:30,  4.02s/it]
 39%|███▊      | 87179/225000 [161:42:38<135:19:50,  3.53s/it]
 39%|███▊      | 87180/225000 [161:42:40<127:48:53,  3.34s/it]
 39%|███▊      | 87181/225000 [161:42:44<127:46:37,  3.34s/it]
 39%|███▊      | 87182/225000 [161:42:47<126:32:53,  3.31s/it]
 39%|███▊      | 87183/225000 [161:42:50<127:46:01,  3.34s/it]
 39%|███▊      | 87184/225000 [161:42:54<130:50:46,  3.42s/it]
 39%|███▊      | 87185/225000 [161:42:57<129:37:29,  3.39s/it]
 39%|███▊      | 87186/225000 [161:43:06<192:31:18,  5.03s/it]
 39%|███▊      | 87187/225000 [161:43:12<200:23:30,  5.23s/it]
 39%|███▉      | 87188/225000 [161:43:14<168:57:54,  4.41s/it]
 39%|███▉      | 87189/225000 [161:43:17<145:22:43,  3.80s/it]
 39%|███▉      | 87190/225000 [161:43:22<167:29:57,  4.38s/it]
 39%|███▉      | 87191/225000 [161:43:25<148:28:15,  3.88s/it]
 39%|███▉      | 87192/225000 [161:43:28<133:01:35,  3.48s/it]
 39%|███▉      | 87193/225000 [161:43:36<184:03:01,  4.81s/it]
 39%|███▉      | 87194/225000 [161:43:39<168:11:50,  4.39s/it]
 39%|███▉      | 87195/225000 [161:43:43<158:33:47,  4.14s/it]
 39%|███▉      | 87196/225000 [161:43:47<156:31:49,  4.09s/it]
 39%|███▉      | 87197/225000 [161:43:53<185:55:03,  4.86s/it]
 39%|███▉      | 87198/225000 [161:43:56<164:07:17,  4.29s/it]
 39%|███▉      | 87199/225000 [161:43:59<149:28:26,  3.90s/it]
 39%|███▉      | 87200/225000 [161:44:03<143:00:03,  3.74s/it]
                                                              
{'loss': '0.4268', 'grad_norm': '4.169', 'learning_rate': '3.572e-05', 'epoch': '0.3876', 'num_input_tokens_seen': 265305992, 'train_runtime': '5.822e+05', 'train_tokens_per_second': '455.7'}

 39%|███▉      | 87200/225000 [161:44:03<143:00:03,  3.74s/it]
 39%|███▉      | 87201/225000 [161:44:06<134:46:56,  3.52s/it]
 39%|███▉      | 87202/225000 [161:44:08<125:27:24,  3.28s/it]
 39%|███▉      | 87203/225000 [161:44:14<150:16:34,  3.93s/it]
 39%|███▉      | 87204/225000 [161:44:22<204:38:26,  5.35s/it]
 39%|███▉      | 87205/225000 [161:44:25<173:49:16,  4.54s/it]
 39%|███▉      | 87206/225000 [161:44:28<151:17:49,  3.95s/it]
 39%|███▉      | 87207/225000 [161:44:33<173:06:28,  4.52s/it]
 39%|███▉      | 87208/225000 [161:44:37<157:55:13,  4.13s/it]
 39%|███▉      | 87209/225000 [161:44:40<145:25:30,  3.80s/it]
 39%|███▉      | 87210/225000 [161:44:43<141:31:09,  3.70s/it]
 39%|███▉      | 87211/225000 [161:44:46<132:13:28,  3.45s/it]
 39%|███▉      | 87212/225000 [161:44:49<125:57:04,  3.29s/it]
 39%|███▉      | 87213/225000 [161:44:52<124:32:43,  3.25s/it]
 39%|███▉      | 87214/225000 [161:44:55<121:00:30,  3.16s/it]
 39%|███▉      | 87215/225000 [161:44:58<121:38:52,  3.18s/it]
 39%|███▉      | 87216/225000 [161:45:02<126:16:52,  3.30s/it]
 39%|███▉      | 87217/225000 [161:45:05<122:53:30,  3.21s/it]
 39%|███▉      | 87218/225000 [161:45:13<174:04:54,  4.55s/it]
 39%|███▉      | 87219/225000 [161:45:21<219:48:19,  5.74s/it]
 39%|███▉      | 87220/225000 [161:45:24<186:17:27,  4.87s/it]
 39%|███▉      | 87221/225000 [161:45:27<162:47:15,  4.25s/it]
 39%|███▉      | 87222/225000 [161:45:33<185:28:26,  4.85s/it]
 39%|███▉      | 87223/225000 [161:45:37<171:13:47,  4.47s/it]
 39%|███▉      | 87224/225000 [161:45:40<156:32:57,  4.09s/it]
 39%|███▉      | 87225/225000 [161:45:43<142:47:02,  3.73s/it]
 39%|███▉      | 87226/225000 [161:45:45<130:16:50,  3.40s/it]
 39%|███▉      | 87227/225000 [161:45:53<180:31:15,  4.72s/it]
 39%|███▉      | 87228/225000 [161:45:58<178:37:22,  4.67s/it]
 39%|███▉      | 87229/225000 [161:46:00<152:16:53,  3.98s/it]
 39%|███▉      | 87230/225000 [161:46:03<135:44:42,  3.55s/it]
 39%|███▉      | 87231/225000 [161:46:05<127:50:55,  3.34s/it]
 39%|███▉      | 87232/225000 [161:46:08<121:50:38,  3.18s/it]
 39%|███▉      | 87233/225000 [161:46:11<116:28:53,  3.04s/it]
 39%|███▉      | 87234/225000 [161:46:20<186:07:34,  4.86s/it]
 39%|███▉      | 87235/225000 [161:46:25<181:57:37,  4.75s/it]
 39%|███▉      | 87236/225000 [161:46:27<156:44:42,  4.10s/it]
 39%|███▉      | 87237/225000 [161:46:30<145:51:16,  3.81s/it]
 39%|███▉      | 87238/225000 [161:46:33<133:48:15,  3.50s/it]
 39%|███▉      | 87239/225000 [161:46:37<138:46:36,  3.63s/it]
 39%|███▉      | 87240/225000 [161:46:45<188:43:05,  4.93s/it]
 39%|███▉      | 87241/225000 [161:46:48<162:21:13,  4.24s/it]
 39%|███▉      | 87242/225000 [161:46:51<149:29:35,  3.91s/it]
 39%|███▉      | 87243/225000 [161:46:53<134:04:53,  3.50s/it]
 39%|███▉      | 87244/225000 [161:46:56<125:52:24,  3.29s/it]
 39%|███▉      | 87245/225000 [161:46:59<120:36:32,  3.15s/it]
 39%|███▉      | 87246/225000 [161:47:02<115:20:30,  3.01s/it]
 39%|███▉      | 87247/225000 [161:47:07<145:49:03,  3.81s/it]
 39%|███▉      | 87248/225000 [161:47:17<215:32:25,  5.63s/it]
 39%|███▉      | 87249/225000 [161:47:19<177:00:45,  4.63s/it]
 39%|███▉      | 87250/225000 [161:47:22<154:16:07,  4.03s/it]
                                                              
{'loss': '0.4324', 'grad_norm': '4.39', 'learning_rate': '3.571e-05', 'epoch': '0.3878', 'num_input_tokens_seen': 265390864, 'train_runtime': '5.824e+05', 'train_tokens_per_second': '455.7'}

 39%|███▉      | 87250/225000 [161:47:22<154:16:07,  4.03s/it]
 39%|███▉      | 87251/225000 [161:47:27<162:03:56,  4.24s/it]
 39%|███▉      | 87252/225000 [161:47:30<151:43:39,  3.97s/it]
 39%|███▉      | 87253/225000 [161:47:34<146:03:17,  3.82s/it]
 39%|███▉      | 87254/225000 [161:47:37<138:42:25,  3.63s/it]
 39%|███▉      | 87255/225000 [161:47:40<134:15:50,  3.51s/it]
 39%|███▉      | 87256/225000 [161:47:43<131:57:44,  3.45s/it]
 39%|███▉      | 87257/225000 [161:47:46<128:50:43,  3.37s/it]
 39%|███▉      | 87258/225000 [161:47:53<162:55:41,  4.26s/it]
 39%|███▉      | 87259/225000 [161:48:01<202:49:51,  5.30s/it]
 39%|███▉      | 87260/225000 [161:48:04<179:22:20,  4.69s/it]
 39%|███▉      | 87261/225000 [161:48:08<170:28:38,  4.46s/it]
 39%|███▉      | 87262/225000 [161:48:11<160:32:23,  4.20s/it]
 39%|███▉      | 87263/225000 [161:48:14<145:39:08,  3.81s/it]
 39%|███▉      | 87264/225000 [161:48:17<138:48:43,  3.63s/it]
 39%|███▉      | 87265/225000 [161:48:21<134:25:02,  3.51s/it]
 39%|███▉      | 87266/225000 [161:48:24<132:40:49,  3.47s/it]
 39%|███▉      | 87267/225000 [161:48:28<133:29:18,  3.49s/it]
 39%|███▉      | 87268/225000 [161:48:31<131:57:45,  3.45s/it]
 39%|███▉      | 87269/225000 [161:48:34<131:36:01,  3.44s/it]
 39%|███▉      | 87270/225000 [161:48:41<163:54:23,  4.28s/it]
 39%|███▉      | 87271/225000 [161:48:49<214:40:23,  5.61s/it]
 39%|███▉      | 87272/225000 [161:48:53<198:37:55,  5.19s/it]
 39%|███▉      | 87273/225000 [161:48:57<174:20:47,  4.56s/it]
 39%|███▉      | 87274/225000 [161:49:00<156:39:23,  4.09s/it]
 39%|███▉      | 87275/225000 [161:49:02<140:36:44,  3.68s/it]
 39%|███▉      | 87276/225000 [161:49:05<129:34:32,  3.39s/it]
 39%|███▉      | 87277/225000 [161:49:08<123:09:44,  3.22s/it]
 39%|███▉      | 87278/225000 [161:49:11<123:59:07,  3.24s/it]
 39%|███▉      | 87279/225000 [161:49:20<189:59:47,  4.97s/it]
 39%|███▉      | 87280/225000 [161:49:23<169:58:39,  4.44s/it]
 39%|███▉      | 87281/225000 [161:49:26<144:34:08,  3.78s/it]
 39%|███▉      | 87282/225000 [161:49:29<135:32:30,  3.54s/it]
 39%|███▉      | 87283/225000 [161:49:33<143:31:53,  3.75s/it]
 39%|███▉      | 87284/225000 [161:49:37<143:32:46,  3.75s/it]
 39%|███▉      | 87285/225000 [161:49:40<141:10:40,  3.69s/it]
 39%|███▉      | 87286/225000 [161:49:47<172:49:34,  4.52s/it]
 39%|███▉      | 87287/225000 [161:49:53<190:51:12,  4.99s/it]
 39%|███▉      | 87288/225000 [161:49:55<160:01:39,  4.18s/it]
 39%|███▉      | 87289/225000 [161:49:59<163:26:10,  4.27s/it]
 39%|███▉      | 87290/225000 [161:50:04<171:07:52,  4.47s/it]
 39%|███▉      | 87291/225000 [161:50:07<148:12:49,  3.87s/it]
 39%|███▉      | 87292/225000 [161:50:10<134:48:16,  3.52s/it]
 39%|███▉      | 87293/225000 [161:50:12<126:02:18,  3.29s/it]
 39%|███▉      | 87294/225000 [161:50:15<118:38:54,  3.10s/it]
 39%|███▉      | 87295/225000 [161:50:18<116:32:09,  3.05s/it]
 39%|███▉      | 87296/225000 [161:50:21<114:51:44,  3.00s/it]
 39%|███▉      | 87297/225000 [161:50:24<114:08:56,  2.98s/it]
 39%|███▉      | 87298/225000 [161:50:27<118:38:17,  3.10s/it]
 39%|███▉      | 87299/225000 [161:50:35<170:20:16,  4.45s/it]
 39%|███▉      | 87300/225000 [161:50:43<213:06:45,  5.57s/it]
                                                              
{'loss': '0.4082', 'grad_norm': '5.477', 'learning_rate': '3.569e-05', 'epoch': '0.388', 'num_input_tokens_seen': 265475664, 'train_runtime': '5.826e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87300/225000 [161:50:43<213:06:45,  5.57s/it]
 39%|███▉      | 87301/225000 [161:50:46<179:44:51,  4.70s/it]
 39%|███▉      | 87302/225000 [161:50:48<158:03:25,  4.13s/it]
 39%|███▉      | 87303/225000 [161:50:52<151:28:04,  3.96s/it]
 39%|███▉      | 87304/225000 [161:50:55<142:57:34,  3.74s/it]
 39%|███▉      | 87305/225000 [161:50:58<137:25:15,  3.59s/it]
 39%|███▉      | 87306/225000 [161:51:02<133:09:45,  3.48s/it]
 39%|███▉      | 87307/225000 [161:51:06<145:28:33,  3.80s/it]
 39%|███▉      | 87308/225000 [161:51:16<213:30:44,  5.58s/it]
 39%|███▉      | 87309/225000 [161:51:19<184:17:13,  4.82s/it]
 39%|███▉      | 87310/225000 [161:51:21<157:27:57,  4.12s/it]
 39%|███▉      | 87311/225000 [161:51:26<157:51:13,  4.13s/it]
 39%|███▉      | 87312/225000 [161:51:32<189:35:31,  4.96s/it]
 39%|███▉      | 87313/225000 [161:51:35<163:41:38,  4.28s/it]
 39%|███▉      | 87314/225000 [161:51:38<143:16:19,  3.75s/it]
 39%|███▉      | 87315/225000 [161:51:44<174:19:07,  4.56s/it]
 39%|███▉      | 87316/225000 [161:51:47<158:37:04,  4.15s/it]
 39%|███▉      | 87317/225000 [161:51:52<167:05:45,  4.37s/it]
 39%|███▉      | 87318/225000 [161:51:56<166:25:44,  4.35s/it]
 39%|███▉      | 87319/225000 [161:52:00<151:41:15,  3.97s/it]
 39%|███▉      | 87320/225000 [161:52:03<141:30:32,  3.70s/it]
 39%|███▉      | 87321/225000 [161:52:06<141:39:46,  3.70s/it]
 39%|███▉      | 87322/225000 [161:52:09<135:09:25,  3.53s/it]
 39%|███▉      | 87323/225000 [161:52:13<134:55:08,  3.53s/it]
 39%|███▉      | 87324/225000 [161:52:16<133:57:57,  3.50s/it]
 39%|███▉      | 87325/225000 [161:52:20<135:32:07,  3.54s/it]
 39%|███▉      | 87326/225000 [161:52:23<131:24:35,  3.44s/it]
 39%|███▉      | 87327/225000 [161:52:29<158:12:53,  4.14s/it]
 39%|███▉      | 87328/225000 [161:52:32<147:44:11,  3.86s/it]
 39%|███▉      | 87329/225000 [161:52:36<142:27:37,  3.73s/it]
 39%|███▉      | 87330/225000 [161:52:39<135:30:10,  3.54s/it]
 39%|███▉      | 87331/225000 [161:52:42<130:51:57,  3.42s/it]
 39%|███▉      | 87332/225000 [161:52:45<125:56:21,  3.29s/it]
 39%|███▉      | 87333/225000 [161:52:48<125:16:57,  3.28s/it]
 39%|███▉      | 87334/225000 [161:52:52<126:33:18,  3.31s/it]
 39%|███▉      | 87335/225000 [161:52:55<132:20:17,  3.46s/it]
 39%|███▉      | 87336/225000 [161:53:02<165:20:28,  4.32s/it]
 39%|███▉      | 87337/225000 [161:53:10<213:18:13,  5.58s/it]
 39%|███▉      | 87338/225000 [161:53:14<187:30:16,  4.90s/it]
 39%|███▉      | 87339/225000 [161:53:16<160:24:20,  4.19s/it]
 39%|███▉      | 87340/225000 [161:53:19<141:00:00,  3.69s/it]
 39%|███▉      | 87341/225000 [161:53:21<127:36:44,  3.34s/it]
 39%|███▉      | 87342/225000 [161:53:24<120:43:09,  3.16s/it]
 39%|███▉      | 87343/225000 [161:53:30<160:51:59,  4.21s/it]
 39%|███▉      | 87344/225000 [161:53:36<179:35:08,  4.70s/it]
 39%|███▉      | 87345/225000 [161:53:40<165:22:25,  4.32s/it]
 39%|███▉      | 87346/225000 [161:53:43<150:34:58,  3.94s/it]
 39%|███▉      | 87347/225000 [161:53:46<146:38:22,  3.84s/it]
 39%|███▉      | 87348/225000 [161:53:50<139:04:10,  3.64s/it]
 39%|███▉      | 87349/225000 [161:53:53<132:42:48,  3.47s/it]
 39%|███▉      | 87350/225000 [161:53:56<132:53:11,  3.48s/it]
                                                              
{'loss': '0.4145', 'grad_norm': '6.66', 'learning_rate': '3.568e-05', 'epoch': '0.3882', 'num_input_tokens_seen': 265561200, 'train_runtime': '5.828e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87350/225000 [161:53:56<132:53:11,  3.48s/it]
 39%|███▉      | 87351/225000 [161:53:59<130:08:18,  3.40s/it]
 39%|███▉      | 87352/225000 [161:54:03<135:42:05,  3.55s/it]
 39%|███▉      | 87353/225000 [161:54:07<135:06:42,  3.53s/it]
 39%|███▉      | 87354/225000 [161:54:15<189:25:25,  4.95s/it]
 39%|███▉      | 87355/225000 [161:54:22<209:34:36,  5.48s/it]
 39%|███▉      | 87356/225000 [161:54:24<175:09:57,  4.58s/it]
 39%|███▉      | 87357/225000 [161:54:27<149:13:34,  3.90s/it]
 39%|███▉      | 87358/225000 [161:54:32<161:33:27,  4.23s/it]
 39%|███▉      | 87359/225000 [161:54:37<181:25:03,  4.74s/it]
 39%|███▉      | 87360/225000 [161:54:40<155:39:54,  4.07s/it]
 39%|███▉      | 87361/225000 [161:54:43<139:13:03,  3.64s/it]
 39%|███▉      | 87362/225000 [161:54:45<128:29:12,  3.36s/it]
 39%|███▉      | 87363/225000 [161:54:48<122:17:55,  3.20s/it]
 39%|███▉      | 87364/225000 [161:54:51<122:06:53,  3.19s/it]
 39%|███▉      | 87365/225000 [161:54:55<125:49:06,  3.29s/it]
 39%|███▉      | 87366/225000 [161:54:59<140:24:58,  3.67s/it]
 39%|███▉      | 87367/225000 [161:55:08<201:27:47,  5.27s/it]
 39%|███▉      | 87368/225000 [161:55:13<189:42:55,  4.96s/it]
 39%|███▉      | 87369/225000 [161:55:16<173:11:28,  4.53s/it]
 39%|███▉      | 87370/225000 [161:55:22<184:22:20,  4.82s/it]
 39%|███▉      | 87371/225000 [161:55:24<159:17:26,  4.17s/it]
 39%|███▉      | 87372/225000 [161:55:27<141:47:59,  3.71s/it]
 39%|███▉      | 87373/225000 [161:55:30<128:42:21,  3.37s/it]
 39%|███▉      | 87374/225000 [161:55:32<122:49:34,  3.21s/it]
 39%|███▉      | 87375/225000 [161:55:40<177:54:32,  4.65s/it]
 39%|███▉      | 87376/225000 [161:55:44<169:15:53,  4.43s/it]
 39%|███▉      | 87377/225000 [161:55:47<152:44:50,  4.00s/it]
 39%|███▉      | 87378/225000 [161:55:50<134:44:00,  3.52s/it]
 39%|███▉      | 87379/225000 [161:55:57<173:36:16,  4.54s/it]
 39%|███▉      | 87380/225000 [161:56:01<168:25:20,  4.41s/it]
 39%|███▉      | 87381/225000 [161:56:03<147:00:50,  3.85s/it]
 39%|███▉      | 87382/225000 [161:56:06<136:09:15,  3.56s/it]
 39%|███▉      | 87383/225000 [161:56:09<125:33:10,  3.28s/it]
 39%|███▉      | 87384/225000 [161:56:11<117:52:58,  3.08s/it]
 39%|███▉      | 87385/225000 [161:56:17<142:28:18,  3.73s/it]
 39%|███▉      | 87386/225000 [161:56:26<207:27:35,  5.43s/it]
 39%|███▉      | 87387/225000 [161:56:30<190:11:33,  4.98s/it]
 39%|███▉      | 87388/225000 [161:56:34<178:35:33,  4.67s/it]
 39%|███▉      | 87389/225000 [161:56:37<160:51:50,  4.21s/it]
 39%|███▉      | 87390/225000 [161:56:40<151:32:29,  3.96s/it]
 39%|███▉      | 87391/225000 [161:56:44<145:48:03,  3.81s/it]
 39%|███▉      | 87392/225000 [161:56:47<139:23:28,  3.65s/it]
 39%|███▉      | 87393/225000 [161:56:51<141:22:34,  3.70s/it]
 39%|███▉      | 87394/225000 [161:56:55<142:59:42,  3.74s/it]
 39%|███▉      | 87395/225000 [161:56:58<139:56:55,  3.66s/it]
 39%|███▉      | 87396/225000 [161:57:02<138:43:19,  3.63s/it]
 39%|███▉      | 87397/225000 [161:57:05<136:37:29,  3.57s/it]
 39%|███▉      | 87398/225000 [161:57:09<132:46:56,  3.47s/it]
 39%|███▉      | 87399/225000 [161:57:13<142:12:46,  3.72s/it]
 39%|███▉      | 87400/225000 [161:57:17<148:04:37,  3.87s/it]
                                                              
{'loss': '0.4011', 'grad_norm': '4.948', 'learning_rate': '3.566e-05', 'epoch': '0.3884', 'num_input_tokens_seen': 265647552, 'train_runtime': '5.83e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87400/225000 [161:57:17<148:04:37,  3.87s/it]
 39%|███▉      | 87401/225000 [161:57:20<141:08:40,  3.69s/it]
 39%|███▉      | 87402/225000 [161:57:24<139:17:25,  3.64s/it]
 39%|███▉      | 87403/225000 [161:57:29<154:26:26,  4.04s/it]
 39%|███▉      | 87404/225000 [161:57:32<143:30:38,  3.75s/it]
 39%|███▉      | 87405/225000 [161:57:35<139:10:49,  3.64s/it]
 39%|███▉      | 87406/225000 [161:57:39<139:25:21,  3.65s/it]
 39%|███▉      | 87407/225000 [161:57:47<185:53:00,  4.86s/it]
 39%|███▉      | 87408/225000 [161:57:52<191:37:26,  5.01s/it]
 39%|███▉      | 87409/225000 [161:57:55<170:03:26,  4.45s/it]
 39%|███▉      | 87410/225000 [161:57:59<157:50:58,  4.13s/it]
 39%|███▉      | 87411/225000 [161:58:02<150:55:17,  3.95s/it]
 39%|███▉      | 87412/225000 [161:58:06<156:16:51,  4.09s/it]
 39%|███▉      | 87413/225000 [161:58:10<152:12:55,  3.98s/it]
 39%|███▉      | 87414/225000 [161:58:14<145:42:38,  3.81s/it]
 39%|███▉      | 87415/225000 [161:58:17<136:45:25,  3.58s/it]
 39%|███▉      | 87416/225000 [161:58:20<139:11:53,  3.64s/it]
 39%|███▉      | 87417/225000 [161:58:24<132:50:52,  3.48s/it]
 39%|███▉      | 87418/225000 [161:58:26<126:30:31,  3.31s/it]
 39%|███▉      | 87419/225000 [161:58:30<127:00:23,  3.32s/it]
 39%|███▉      | 87420/225000 [161:58:33<129:41:24,  3.39s/it]
 39%|███▉      | 87421/225000 [161:58:40<170:47:48,  4.47s/it]
 39%|███▉      | 87422/225000 [161:58:50<228:43:57,  5.99s/it]
 39%|███▉      | 87423/225000 [161:58:53<198:46:32,  5.20s/it]
 39%|███▉      | 87424/225000 [161:58:56<171:08:29,  4.48s/it]
 39%|███▉      | 87425/225000 [161:58:59<153:27:27,  4.02s/it]
 39%|███▉      | 87426/225000 [161:59:02<139:03:51,  3.64s/it]
 39%|███▉      | 87427/225000 [161:59:04<128:30:04,  3.36s/it]
 39%|███▉      | 87428/225000 [161:59:08<124:55:36,  3.27s/it]
 39%|███▉      | 87429/225000 [161:59:15<174:15:49,  4.56s/it]
 39%|███▉      | 87430/225000 [161:59:22<201:19:37,  5.27s/it]
 39%|███▉      | 87431/225000 [161:59:25<171:52:23,  4.50s/it]
 39%|███▉      | 87432/225000 [161:59:27<150:16:45,  3.93s/it]
 39%|███▉      | 87433/225000 [161:59:30<135:30:21,  3.55s/it]
 39%|███▉      | 87434/225000 [161:59:33<126:42:54,  3.32s/it]
 39%|███▉      | 87435/225000 [161:59:36<122:35:05,  3.21s/it]
 39%|███▉      | 87436/225000 [161:59:44<185:32:16,  4.86s/it]
 39%|███▉      | 87437/225000 [161:59:48<169:36:03,  4.44s/it]
 39%|███▉      | 87438/225000 [161:59:52<165:03:16,  4.32s/it]
 39%|███▉      | 87439/225000 [161:59:59<196:10:14,  5.13s/it]
 39%|███▉      | 87440/225000 [162:00:02<168:28:08,  4.41s/it]
 39%|███▉      | 87441/225000 [162:00:04<144:16:03,  3.78s/it]
 39%|███▉      | 87442/225000 [162:00:07<130:41:54,  3.42s/it]
 39%|███▉      | 87443/225000 [162:00:10<128:23:20,  3.36s/it]
 39%|███▉      | 87444/225000 [162:00:13<125:06:10,  3.27s/it]
 39%|███▉      | 87445/225000 [162:00:17<131:21:58,  3.44s/it]
 39%|███▉      | 87446/225000 [162:00:20<133:43:13,  3.50s/it]
 39%|███▉      | 87447/225000 [162:00:25<148:50:26,  3.90s/it]
 39%|███▉      | 87448/225000 [162:00:35<212:52:08,  5.57s/it]
 39%|███▉      | 87449/225000 [162:00:37<175:04:38,  4.58s/it]
 39%|███▉      | 87450/225000 [162:00:42<183:41:22,  4.81s/it]
                                                              
{'loss': '0.405', 'grad_norm': '5.729', 'learning_rate': '3.564e-05', 'epoch': '0.3887', 'num_input_tokens_seen': 265733744, 'train_runtime': '5.832e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87450/225000 [162:00:42<183:41:22,  4.81s/it]
 39%|███▉      | 87451/225000 [162:00:44<154:27:52,  4.04s/it]
 39%|███▉      | 87452/225000 [162:00:47<138:11:26,  3.62s/it]
 39%|███▉      | 87453/225000 [162:00:55<186:03:09,  4.87s/it]
 39%|███▉      | 87454/225000 [162:00:57<158:48:50,  4.16s/it]
 39%|███▉      | 87455/225000 [162:01:00<140:31:15,  3.68s/it]
 39%|███▉      | 87456/225000 [162:01:07<175:33:32,  4.59s/it]
 39%|███▉      | 87457/225000 [162:01:11<172:14:20,  4.51s/it]
 39%|███▉      | 87458/225000 [162:01:13<149:25:25,  3.91s/it]
 39%|███▉      | 87459/225000 [162:01:17<140:21:04,  3.67s/it]
 39%|███▉      | 87460/225000 [162:01:19<128:22:09,  3.36s/it]
 39%|███▉      | 87461/225000 [162:01:23<132:42:57,  3.47s/it]
 39%|███▉      | 87462/225000 [162:01:31<187:46:01,  4.91s/it]
 39%|███▉      | 87463/225000 [162:01:34<163:26:13,  4.28s/it]
 39%|███▉      | 87464/225000 [162:01:37<145:09:58,  3.80s/it]
 39%|███▉      | 87465/225000 [162:01:40<139:15:12,  3.64s/it]
 39%|███▉      | 87466/225000 [162:01:43<129:00:40,  3.38s/it]
 39%|███▉      | 87467/225000 [162:01:46<124:36:02,  3.26s/it]
 39%|███▉      | 87468/225000 [162:01:49<121:06:00,  3.17s/it]
 39%|███▉      | 87469/225000 [162:01:53<137:13:14,  3.59s/it]
 39%|███▉      | 87470/225000 [162:02:02<199:45:05,  5.23s/it]
 39%|███▉      | 87471/225000 [162:02:08<199:08:31,  5.21s/it]
 39%|███▉      | 87472/225000 [162:02:10<165:26:21,  4.33s/it]
 39%|███▉      | 87473/225000 [162:02:16<187:17:38,  4.90s/it]
 39%|███▉      | 87474/225000 [162:02:20<174:07:56,  4.56s/it]
 39%|███▉      | 87475/225000 [162:02:25<182:16:14,  4.77s/it]
 39%|███▉      | 87476/225000 [162:02:27<155:15:44,  4.06s/it]
 39%|███▉      | 87477/225000 [162:02:30<143:01:42,  3.74s/it]
 39%|███▉      | 87478/225000 [162:02:34<137:12:18,  3.59s/it]
 39%|███▉      | 87479/225000 [162:02:37<133:57:54,  3.51s/it]
 39%|███▉      | 87480/225000 [162:02:40<124:50:34,  3.27s/it]
 39%|███▉      | 87481/225000 [162:02:47<171:03:37,  4.48s/it]
 39%|███▉      | 87482/225000 [162:02:52<182:32:54,  4.78s/it]
 39%|███▉      | 87483/225000 [162:02:56<168:01:55,  4.40s/it]
 39%|███▉      | 87484/225000 [162:03:02<188:10:31,  4.93s/it]
 39%|███▉      | 87485/225000 [162:03:05<161:36:55,  4.23s/it]
 39%|███▉      | 87486/225000 [162:03:08<145:11:33,  3.80s/it]
 39%|███▉      | 87487/225000 [162:03:10<131:52:56,  3.45s/it]
 39%|███▉      | 87488/225000 [162:03:13<123:11:34,  3.23s/it]
 39%|███▉      | 87489/225000 [162:03:16<119:45:48,  3.14s/it]
 39%|███▉      | 87490/225000 [162:03:24<177:00:22,  4.63s/it]
 39%|███▉      | 87491/225000 [162:03:31<203:46:19,  5.33s/it]
 39%|███▉      | 87492/225000 [162:03:34<172:36:29,  4.52s/it]
 39%|███▉      | 87493/225000 [162:03:36<150:15:27,  3.93s/it]
 39%|███▉      | 87494/225000 [162:03:39<134:43:43,  3.53s/it]
 39%|███▉      | 87495/225000 [162:03:46<172:20:22,  4.51s/it]
 39%|███▉      | 87496/225000 [162:03:49<163:24:12,  4.28s/it]
 39%|███▉      | 87497/225000 [162:03:55<183:11:28,  4.80s/it]
 39%|███▉      | 87498/225000 [162:03:58<163:50:10,  4.29s/it]
 39%|███▉      | 87499/225000 [162:04:02<151:28:43,  3.97s/it]
 39%|███▉      | 87500/225000 [162:04:05<145:34:15,  3.81s/it]
                                                              
{'loss': '0.4148', 'grad_norm': '4.212', 'learning_rate': '3.563e-05', 'epoch': '0.3889', 'num_input_tokens_seen': 265820736, 'train_runtime': '5.834e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87500/225000 [162:04:05<145:34:15,  3.81s/it]
 39%|███▉      | 87501/225000 [162:04:08<139:42:54,  3.66s/it]
 39%|███▉      | 87502/225000 [162:04:12<140:50:30,  3.69s/it]
 39%|███▉      | 87503/225000 [162:04:15<135:50:00,  3.56s/it]
 39%|███▉      | 87504/225000 [162:04:19<133:07:00,  3.49s/it]
 39%|███▉      | 87505/225000 [162:04:22<135:51:54,  3.56s/it]
 39%|███▉      | 87506/225000 [162:04:28<159:02:17,  4.16s/it]
 39%|███▉      | 87507/225000 [162:04:36<205:30:52,  5.38s/it]
 39%|███▉      | 87508/225000 [162:04:39<171:56:37,  4.50s/it]
 39%|███▉      | 87509/225000 [162:04:41<146:55:53,  3.85s/it]
 39%|███▉      | 87510/225000 [162:04:43<130:09:25,  3.41s/it]
 39%|███▉      | 87511/225000 [162:04:47<127:45:15,  3.35s/it]
 39%|███▉      | 87512/225000 [162:04:55<186:10:58,  4.88s/it]
 39%|███▉      | 87513/225000 [162:04:57<157:42:36,  4.13s/it]
 39%|███▉      | 87514/225000 [162:05:00<139:15:54,  3.65s/it]
 39%|███▉      | 87515/225000 [162:05:07<182:30:15,  4.78s/it]
 39%|███▉      | 87516/225000 [162:05:10<155:35:59,  4.07s/it]
 39%|███▉      | 87517/225000 [162:05:12<136:37:13,  3.58s/it]
 39%|███▉      | 87518/225000 [162:05:15<126:41:18,  3.32s/it]
 39%|███▉      | 87519/225000 [162:05:19<131:51:32,  3.45s/it]
 39%|███▉      | 87520/225000 [162:05:24<155:28:17,  4.07s/it]
 39%|███▉      | 87521/225000 [162:05:27<139:04:29,  3.64s/it]
 39%|███▉      | 87522/225000 [162:05:29<128:34:50,  3.37s/it]
 39%|███▉      | 87523/225000 [162:05:32<118:35:01,  3.11s/it]
 39%|███▉      | 87524/225000 [162:05:36<125:02:58,  3.27s/it]
 39%|███▉      | 87525/225000 [162:05:44<182:25:05,  4.78s/it]
 39%|███▉      | 87526/225000 [162:05:46<156:19:36,  4.09s/it]
 39%|███▉      | 87527/225000 [162:05:49<138:28:27,  3.63s/it]
 39%|███▉      | 87528/225000 [162:05:52<127:03:26,  3.33s/it]
 39%|███▉      | 87529/225000 [162:05:55<125:36:24,  3.29s/it]
 39%|███▉      | 87530/225000 [162:05:58<124:01:49,  3.25s/it]
 39%|███▉      | 87531/225000 [162:06:01<126:34:07,  3.31s/it]
 39%|███▉      | 87532/225000 [162:06:08<166:58:51,  4.37s/it]
 39%|███▉      | 87533/225000 [162:06:17<214:54:48,  5.63s/it]
 39%|███▉      | 87534/225000 [162:06:19<180:43:43,  4.73s/it]
 39%|███▉      | 87535/225000 [162:06:22<156:33:40,  4.10s/it]
 39%|███▉      | 87536/225000 [162:06:25<141:36:17,  3.71s/it]
 39%|███▉      | 87537/225000 [162:06:27<128:53:08,  3.38s/it]
 39%|███▉      | 87538/225000 [162:06:33<157:48:33,  4.13s/it]
 39%|███▉      | 87539/225000 [162:06:41<201:11:37,  5.27s/it]
 39%|███▉      | 87540/225000 [162:06:44<169:44:28,  4.45s/it]
 39%|███▉      | 87541/225000 [162:06:46<144:56:01,  3.80s/it]
 39%|███▉      | 87542/225000 [162:06:50<146:30:25,  3.84s/it]
 39%|███▉      | 87543/225000 [162:06:58<189:04:37,  4.95s/it]
 39%|███▉      | 87544/225000 [162:07:00<165:14:36,  4.33s/it]
 39%|███▉      | 87545/225000 [162:07:03<146:46:08,  3.84s/it]
 39%|███▉      | 87546/225000 [162:07:06<132:26:17,  3.47s/it]
 39%|███▉      | 87547/225000 [162:07:08<122:50:06,  3.22s/it]
 39%|███▉      | 87548/225000 [162:07:11<118:45:33,  3.11s/it]
 39%|███▉      | 87549/225000 [162:07:21<193:34:18,  5.07s/it]
 39%|███▉      | 87550/225000 [162:07:25<177:47:18,  4.66s/it]
                                                              
{'loss': '0.4193', 'grad_norm': '4.36', 'learning_rate': '3.561e-05', 'epoch': '0.3891', 'num_input_tokens_seen': 265907240, 'train_runtime': '5.836e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87550/225000 [162:07:25<177:47:18,  4.66s/it]
 39%|███▉      | 87551/225000 [162:07:27<155:16:36,  4.07s/it]
 39%|███▉      | 87552/225000 [162:07:30<137:23:16,  3.60s/it]
 39%|███▉      | 87553/225000 [162:07:37<179:09:44,  4.69s/it]
 39%|███▉      | 87554/225000 [162:07:40<161:53:13,  4.24s/it]
 39%|███▉      | 87555/225000 [162:07:43<142:46:44,  3.74s/it]
 39%|███▉      | 87556/225000 [162:07:45<128:33:25,  3.37s/it]
 39%|███▉      | 87557/225000 [162:07:52<162:08:48,  4.25s/it]
 39%|███▉      | 87558/225000 [162:07:57<178:35:53,  4.68s/it]
 39%|███▉      | 87559/225000 [162:08:01<164:45:14,  4.32s/it]
 39%|███▉      | 87560/225000 [162:08:04<157:09:16,  4.12s/it]
 39%|███▉      | 87561/225000 [162:08:08<150:52:37,  3.95s/it]
 39%|███▉      | 87562/225000 [162:08:11<140:16:04,  3.67s/it]
 39%|███▉      | 87563/225000 [162:08:14<137:50:17,  3.61s/it]
 39%|███▉      | 87564/225000 [162:08:17<130:42:37,  3.42s/it]
 39%|███▉      | 87565/225000 [162:08:21<126:51:34,  3.32s/it]
 39%|███▉      | 87566/225000 [162:08:23<121:09:54,  3.17s/it]
 39%|███▉      | 87567/225000 [162:08:26<119:59:01,  3.14s/it]
 39%|███▉      | 87568/225000 [162:08:34<166:04:29,  4.35s/it]
 39%|███▉      | 87569/225000 [162:08:42<217:17:41,  5.69s/it]
 39%|███▉      | 87570/225000 [162:08:45<183:58:34,  4.82s/it]
 39%|███▉      | 87571/225000 [162:08:48<161:49:56,  4.24s/it]
 39%|███▉      | 87572/225000 [162:08:54<179:59:22,  4.71s/it]
 39%|███▉      | 87573/225000 [162:08:57<166:36:02,  4.36s/it]
 39%|███▉      | 87574/225000 [162:09:00<147:41:06,  3.87s/it]
 39%|███▉      | 87575/225000 [162:09:03<136:50:06,  3.58s/it]
 39%|███▉      | 87576/225000 [162:09:06<129:14:43,  3.39s/it]
 39%|███▉      | 87577/225000 [162:09:09<127:36:15,  3.34s/it]
 39%|███▉      | 87578/225000 [162:09:17<174:05:27,  4.56s/it]
 39%|███▉      | 87579/225000 [162:09:23<196:11:24,  5.14s/it]
 39%|███▉      | 87580/225000 [162:09:26<167:03:58,  4.38s/it]
 39%|███▉      | 87581/225000 [162:09:28<145:38:17,  3.82s/it]
 39%|███▉      | 87582/225000 [162:09:31<129:30:44,  3.39s/it]
 39%|███▉      | 87583/225000 [162:09:39<187:31:24,  4.91s/it]
 39%|███▉      | 87584/225000 [162:09:42<160:09:31,  4.20s/it]
 39%|███▉      | 87585/225000 [162:09:44<140:53:09,  3.69s/it]
 39%|███▉      | 87586/225000 [162:09:49<158:47:08,  4.16s/it]
 39%|███▉      | 87587/225000 [162:09:55<173:33:17,  4.55s/it]
 39%|███▉      | 87588/225000 [162:09:57<149:26:13,  3.92s/it]
 39%|███▉      | 87589/225000 [162:10:00<134:51:48,  3.53s/it]
 39%|███▉      | 87590/225000 [162:10:03<131:41:53,  3.45s/it]
 39%|███▉      | 87591/225000 [162:10:07<134:30:59,  3.52s/it]
 39%|███▉      | 87592/225000 [162:10:15<191:56:44,  5.03s/it]
 39%|███▉      | 87593/225000 [162:10:19<178:54:23,  4.69s/it]
 39%|███▉      | 87594/225000 [162:10:24<179:27:07,  4.70s/it]
 39%|███▉      | 87595/225000 [162:10:26<153:01:32,  4.01s/it]
 39%|███▉      | 87596/225000 [162:10:29<137:40:38,  3.61s/it]
 39%|███▉      | 87597/225000 [162:10:32<125:39:19,  3.29s/it]
 39%|███▉      | 87598/225000 [162:10:35<126:17:48,  3.31s/it]
 39%|███▉      | 87599/225000 [162:10:38<123:09:28,  3.23s/it]
 39%|███▉      | 87600/225000 [162:10:42<135:52:07,  3.56s/it]
                                                              
{'loss': '0.3707', 'grad_norm': '4.747', 'learning_rate': '3.559e-05', 'epoch': '0.3893', 'num_input_tokens_seen': 265994000, 'train_runtime': '5.838e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87600/225000 [162:10:42<135:52:07,  3.56s/it]
 39%|███▉      | 87601/225000 [162:10:51<192:21:44,  5.04s/it]
 39%|███▉      | 87602/225000 [162:10:54<169:28:04,  4.44s/it]
 39%|███▉      | 87603/225000 [162:10:57<152:21:42,  3.99s/it]
 39%|███▉      | 87604/225000 [162:11:00<145:24:56,  3.81s/it]
 39%|███▉      | 87605/225000 [162:11:03<134:33:51,  3.53s/it]
 39%|███▉      | 87606/225000 [162:11:06<131:07:16,  3.44s/it]
 39%|███▉      | 87607/225000 [162:11:10<129:27:36,  3.39s/it]
 39%|███▉      | 87608/225000 [162:11:13<124:52:15,  3.27s/it]
 39%|███▉      | 87609/225000 [162:11:16<120:41:01,  3.16s/it]
 39%|███▉      | 87610/225000 [162:11:23<166:51:19,  4.37s/it]
 39%|███▉      | 87611/225000 [162:11:28<176:57:37,  4.64s/it]
 39%|███▉      | 87612/225000 [162:11:31<160:36:51,  4.21s/it]
 39%|███▉      | 87613/225000 [162:11:34<145:39:06,  3.82s/it]
 39%|███▉      | 87614/225000 [162:11:37<133:37:24,  3.50s/it]
 39%|███▉      | 87615/225000 [162:11:40<128:40:10,  3.37s/it]
 39%|███▉      | 87616/225000 [162:11:43<130:01:21,  3.41s/it]
 39%|███▉      | 87617/225000 [162:11:47<131:12:47,  3.44s/it]
 39%|███▉      | 87618/225000 [162:11:50<129:21:38,  3.39s/it]
 39%|███▉      | 87619/225000 [162:11:55<141:40:51,  3.71s/it]
 39%|███▉      | 87620/225000 [162:12:03<195:45:08,  5.13s/it]
 39%|███▉      | 87621/225000 [162:12:09<204:08:42,  5.35s/it]
 39%|███▉      | 87622/225000 [162:12:11<168:14:27,  4.41s/it]
 39%|███▉      | 87623/225000 [162:12:15<161:02:48,  4.22s/it]
 39%|███▉      | 87624/225000 [162:12:19<155:09:20,  4.07s/it]
 39%|███▉      | 87625/225000 [162:12:25<178:21:15,  4.67s/it]
 39%|███▉      | 87626/225000 [162:12:30<179:18:51,  4.70s/it]
 39%|███▉      | 87627/225000 [162:12:32<154:59:49,  4.06s/it]
 39%|███▉      | 87628/225000 [162:12:35<140:49:50,  3.69s/it]
 39%|███▉      | 87629/225000 [162:12:37<126:18:42,  3.31s/it]
 39%|███▉      | 87630/225000 [162:12:45<180:51:37,  4.74s/it]
 39%|███▉      | 87631/225000 [162:12:49<171:33:09,  4.50s/it]
 39%|███▉      | 87632/225000 [162:12:52<151:11:42,  3.96s/it]
 39%|███▉      | 87633/225000 [162:12:55<140:15:07,  3.68s/it]
 39%|███▉      | 87634/225000 [162:12:58<131:44:46,  3.45s/it]
 39%|███▉      | 87635/225000 [162:13:01<125:17:13,  3.28s/it]
 39%|███▉      | 87636/225000 [162:13:08<167:24:08,  4.39s/it]
 39%|███▉      | 87637/225000 [162:13:14<183:38:50,  4.81s/it]
 39%|███▉      | 87638/225000 [162:13:16<155:49:57,  4.08s/it]
 39%|███▉      | 87639/225000 [162:13:19<138:17:18,  3.62s/it]
 39%|███▉      | 87640/225000 [162:13:21<127:59:08,  3.35s/it]
 39%|███▉      | 87641/225000 [162:13:24<124:39:39,  3.27s/it]
 39%|███▉      | 87642/225000 [162:13:28<124:24:07,  3.26s/it]
 39%|███▉      | 87643/225000 [162:13:32<140:27:26,  3.68s/it]
 39%|███▉      | 87644/225000 [162:13:41<201:22:19,  5.28s/it]
 39%|███▉      | 87645/225000 [162:13:44<169:31:41,  4.44s/it]
 39%|███▉      | 87646/225000 [162:13:47<154:35:15,  4.05s/it]
 39%|███▉      | 87647/225000 [162:13:53<175:26:13,  4.60s/it]
 39%|███▉      | 87648/225000 [162:13:59<193:49:10,  5.08s/it]
 39%|███▉      | 87649/225000 [162:14:02<171:34:17,  4.50s/it]
 39%|███▉      | 87650/225000 [162:14:05<156:32:22,  4.10s/it]
                                                              
{'loss': '0.3755', 'grad_norm': '6.353', 'learning_rate': '3.558e-05', 'epoch': '0.3896', 'num_input_tokens_seen': 266079416, 'train_runtime': '5.84e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87650/225000 [162:14:05<156:32:22,  4.10s/it]
 39%|███▉      | 87651/225000 [162:14:09<148:23:02,  3.89s/it]
 39%|███▉      | 87652/225000 [162:14:12<140:57:30,  3.69s/it]
 39%|███▉      | 87653/225000 [162:14:15<132:09:10,  3.46s/it]
 39%|███▉      | 87654/225000 [162:14:18<130:20:58,  3.42s/it]
 39%|███▉      | 87655/225000 [162:14:21<124:00:55,  3.25s/it]
 39%|███▉      | 87656/225000 [162:14:24<120:30:47,  3.16s/it]
 39%|███▉      | 87657/225000 [162:14:27<120:17:31,  3.15s/it]
 39%|███▉      | 87658/225000 [162:14:35<172:20:42,  4.52s/it]
 39%|███▉      | 87659/225000 [162:14:43<211:35:13,  5.55s/it]
 39%|███▉      | 87660/225000 [162:14:46<180:37:02,  4.73s/it]
 39%|███▉      | 87661/225000 [162:14:49<166:24:04,  4.36s/it]
 39%|███▉      | 87662/225000 [162:14:53<155:36:18,  4.08s/it]
 39%|███▉      | 87663/225000 [162:14:56<152:51:25,  4.01s/it]
 39%|███▉      | 87664/225000 [162:15:01<155:55:55,  4.09s/it]
 39%|███▉      | 87665/225000 [162:15:07<179:31:47,  4.71s/it]
 39%|███▉      | 87666/225000 [162:15:10<161:35:53,  4.24s/it]
 39%|███▉      | 87667/225000 [162:15:13<144:43:52,  3.79s/it]
 39%|███▉      | 87668/225000 [162:15:15<132:43:10,  3.48s/it]
 39%|███▉      | 87669/225000 [162:15:18<127:34:15,  3.34s/it]
 39%|███▉      | 87670/225000 [162:15:22<127:26:12,  3.34s/it]
 39%|███▉      | 87671/225000 [162:15:25<121:10:59,  3.18s/it]
 39%|███▉      | 87672/225000 [162:15:28<119:45:42,  3.14s/it]
 39%|███▉      | 87673/225000 [162:15:36<178:08:57,  4.67s/it]
 39%|███▉      | 87674/225000 [162:15:42<194:25:37,  5.10s/it]
 39%|███▉      | 87675/225000 [162:15:45<171:49:04,  4.50s/it]
 39%|███▉      | 87676/225000 [162:15:48<156:24:29,  4.10s/it]
 39%|███▉      | 87677/225000 [162:15:52<147:17:40,  3.86s/it]
 39%|███▉      | 87678/225000 [162:15:54<135:37:52,  3.56s/it]
 39%|███▉      | 87679/225000 [162:15:58<132:15:38,  3.47s/it]
 39%|███▉      | 87680/225000 [162:16:01<133:10:24,  3.49s/it]
 39%|███▉      | 87681/225000 [162:16:05<132:19:13,  3.47s/it]
 39%|███▉      | 87682/225000 [162:16:08<126:02:39,  3.30s/it]
 39%|███▉      | 87683/225000 [162:16:12<143:40:36,  3.77s/it]
 39%|███▉      | 87684/225000 [162:16:21<197:06:32,  5.17s/it]
 39%|███▉      | 87685/225000 [162:16:25<185:00:25,  4.85s/it]
 39%|███▉      | 87686/225000 [162:16:28<164:08:54,  4.30s/it]
 39%|███▉      | 87687/225000 [162:16:32<156:56:39,  4.11s/it]
 39%|███▉      | 87688/225000 [162:16:35<149:00:32,  3.91s/it]
 39%|███▉      | 87689/225000 [162:16:40<163:14:16,  4.28s/it]
 39%|███▉      | 87690/225000 [162:16:46<175:34:59,  4.60s/it]
 39%|███▉      | 87691/225000 [162:16:48<154:05:52,  4.04s/it]
 39%|███▉      | 87692/225000 [162:16:51<141:59:43,  3.72s/it]
 39%|███▉      | 87693/225000 [162:16:54<132:05:01,  3.46s/it]
 39%|███▉      | 87694/225000 [162:16:57<124:39:06,  3.27s/it]
 39%|███▉      | 87695/225000 [162:17:01<131:26:04,  3.45s/it]
 39%|███▉      | 87696/225000 [162:17:09<187:47:45,  4.92s/it]
 39%|███▉      | 87697/225000 [162:17:12<163:38:20,  4.29s/it]
 39%|███▉      | 87698/225000 [162:17:15<143:39:29,  3.77s/it]
 39%|███▉      | 87699/225000 [162:17:22<180:11:49,  4.72s/it]
 39%|███▉      | 87700/225000 [162:17:25<168:33:32,  4.42s/it]
                                                              
{'loss': '0.3987', 'grad_norm': '4.208', 'learning_rate': '3.556e-05', 'epoch': '0.3898', 'num_input_tokens_seen': 266166000, 'train_runtime': '5.842e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87700/225000 [162:17:25<168:33:32,  4.42s/it]
 39%|███▉      | 87701/225000 [162:17:28<151:05:09,  3.96s/it]
 39%|███▉      | 87702/225000 [162:17:31<142:43:41,  3.74s/it]
 39%|███▉      | 87703/225000 [162:17:34<132:54:35,  3.48s/it]
 39%|███▉      | 87704/225000 [162:17:38<135:18:29,  3.55s/it]
 39%|███▉      | 87705/225000 [162:17:41<132:19:05,  3.47s/it]
 39%|███▉      | 87706/225000 [162:17:44<126:08:25,  3.31s/it]
 39%|███▉      | 87707/225000 [162:17:49<148:11:02,  3.89s/it]
 39%|███▉      | 87708/225000 [162:17:59<213:13:07,  5.59s/it]
 39%|███▉      | 87709/225000 [162:18:02<178:37:43,  4.68s/it]
 39%|███▉      | 87710/225000 [162:18:04<155:55:36,  4.09s/it]
 39%|███▉      | 87711/225000 [162:18:11<185:29:15,  4.86s/it]
 39%|███▉      | 87712/225000 [162:18:13<156:01:10,  4.09s/it]
 39%|███▉      | 87713/225000 [162:18:16<144:33:45,  3.79s/it]
 39%|███▉      | 87714/225000 [162:18:19<134:42:45,  3.53s/it]
 39%|███▉      | 87715/225000 [162:18:22<131:50:26,  3.46s/it]
 39%|███▉      | 87716/225000 [162:18:25<125:45:01,  3.30s/it]
 39%|███▉      | 87717/225000 [162:18:28<122:59:09,  3.23s/it]
 39%|███▉      | 87718/225000 [162:18:33<138:41:24,  3.64s/it]
 39%|███▉      | 87719/225000 [162:18:42<199:13:04,  5.22s/it]
 39%|███▉      | 87720/225000 [162:18:46<179:42:00,  4.71s/it]
 39%|███▉      | 87721/225000 [162:18:48<155:19:20,  4.07s/it]
 39%|███▉      | 87722/225000 [162:18:51<137:02:55,  3.59s/it]
 39%|███▉      | 87723/225000 [162:18:54<130:20:59,  3.42s/it]
 39%|███▉      | 87724/225000 [162:18:57<125:29:43,  3.29s/it]
 39%|███▉      | 87725/225000 [162:19:00<122:57:01,  3.22s/it]
 39%|███▉      | 87726/225000 [162:19:03<120:55:48,  3.17s/it]
 39%|███▉      | 87727/225000 [162:19:11<184:11:08,  4.83s/it]
 39%|███▉      | 87728/225000 [162:19:18<200:01:39,  5.25s/it]
 39%|███▉      | 87729/225000 [162:19:20<167:35:34,  4.40s/it]
 39%|███▉      | 87730/225000 [162:19:23<148:23:06,  3.89s/it]
 39%|███▉      | 87731/225000 [162:19:26<144:04:32,  3.78s/it]
 39%|███▉      | 87732/225000 [162:19:30<141:25:36,  3.71s/it]
 39%|███▉      | 87733/225000 [162:19:33<140:25:06,  3.68s/it]
 39%|███▉      | 87734/225000 [162:19:37<134:02:57,  3.52s/it]
 39%|███▉      | 87735/225000 [162:19:43<165:17:17,  4.33s/it]
 39%|███▉      | 87736/225000 [162:19:51<208:52:35,  5.48s/it]
 39%|███▉      | 87737/225000 [162:19:54<177:57:07,  4.67s/it]
 39%|███▉      | 87738/225000 [162:19:56<154:59:07,  4.06s/it]
 39%|███▉      | 87739/225000 [162:19:59<139:14:40,  3.65s/it]
 39%|███▉      | 87740/225000 [162:20:02<125:43:59,  3.30s/it]
 39%|███▉      | 87741/225000 [162:20:09<172:24:09,  4.52s/it]
 39%|███▉      | 87742/225000 [162:20:15<195:22:17,  5.12s/it]
 39%|███▉      | 87743/225000 [162:20:18<165:17:29,  4.34s/it]
 39%|███▉      | 87744/225000 [162:20:21<146:48:23,  3.85s/it]
 39%|███▉      | 87745/225000 [162:20:29<193:37:48,  5.08s/it]
 39%|███▉      | 87746/225000 [162:20:31<168:27:26,  4.42s/it]
 39%|███▉      | 87747/225000 [162:20:34<148:36:21,  3.90s/it]
 39%|███▉      | 87748/225000 [162:20:38<143:46:01,  3.77s/it]
 39%|███▉      | 87749/225000 [162:20:41<136:51:44,  3.59s/it]
 39%|███▉      | 87750/225000 [162:20:44<136:47:44,  3.59s/it]
                                                              
{'loss': '0.4034', 'grad_norm': '4.224', 'learning_rate': '3.554e-05', 'epoch': '0.39', 'num_input_tokens_seen': 266253464, 'train_runtime': '5.844e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87750/225000 [162:20:44<136:47:44,  3.59s/it]
 39%|███▉      | 87751/225000 [162:20:51<166:28:14,  4.37s/it]
 39%|███▉      | 87752/225000 [162:20:57<189:51:09,  4.98s/it]
 39%|███▉      | 87753/225000 [162:21:00<167:47:50,  4.40s/it]
 39%|███▉      | 87754/225000 [162:21:03<148:52:17,  3.90s/it]
 39%|███▉      | 87755/225000 [162:21:06<136:38:56,  3.58s/it]
 39%|███▉      | 87756/225000 [162:21:09<130:06:05,  3.41s/it]
 39%|███▉      | 87757/225000 [162:21:12<125:52:24,  3.30s/it]
 39%|███▉      | 87758/225000 [162:21:14<120:24:37,  3.16s/it]
 39%|███▉      | 87759/225000 [162:21:22<166:04:20,  4.36s/it]
 39%|███▉      | 87760/225000 [162:21:31<223:54:41,  5.87s/it]
 39%|███▉      | 87761/225000 [162:21:33<183:29:32,  4.81s/it]
 39%|███▉      | 87762/225000 [162:21:36<162:20:44,  4.26s/it]
 39%|███▉      | 87763/225000 [162:21:39<146:21:20,  3.84s/it]
 39%|███▉      | 87764/225000 [162:21:43<141:18:56,  3.71s/it]
 39%|███▉      | 87765/225000 [162:21:45<131:51:42,  3.46s/it]
 39%|███▉      | 87766/225000 [162:21:49<127:53:18,  3.35s/it]
 39%|███▉      | 87767/225000 [162:21:52<127:54:05,  3.36s/it]
 39%|███▉      | 87768/225000 [162:21:55<125:59:19,  3.31s/it]
 39%|███▉      | 87769/225000 [162:22:05<195:30:14,  5.13s/it]
 39%|███▉      | 87770/225000 [162:22:10<200:42:57,  5.27s/it]
 39%|███▉      | 87771/225000 [162:22:14<180:38:51,  4.74s/it]
 39%|███▉      | 87772/225000 [162:22:17<162:27:47,  4.26s/it]
 39%|███▉      | 87773/225000 [162:22:21<161:09:07,  4.23s/it]
 39%|███▉      | 87774/225000 [162:22:24<146:51:17,  3.85s/it]
 39%|███▉      | 87775/225000 [162:22:27<136:26:14,  3.58s/it]
 39%|███▉      | 87776/225000 [162:22:30<128:39:53,  3.38s/it]
 39%|███▉      | 87777/225000 [162:22:33<126:21:16,  3.31s/it]
 39%|███▉      | 87778/225000 [162:22:36<121:57:12,  3.20s/it]
 39%|███▉      | 87779/225000 [162:22:41<140:23:08,  3.68s/it]
 39%|███▉      | 87780/225000 [162:22:49<195:40:19,  5.13s/it]
 39%|███▉      | 87781/225000 [162:22:53<183:27:13,  4.81s/it]
 39%|███▉      | 87782/225000 [162:22:56<157:51:48,  4.14s/it]
 39%|███▉      | 87783/225000 [162:23:01<173:19:31,  4.55s/it]
 39%|███▉      | 87784/225000 [162:23:05<165:25:34,  4.34s/it]
 39%|███▉      | 87785/225000 [162:23:08<143:04:38,  3.75s/it]
 39%|███▉      | 87786/225000 [162:23:10<132:30:12,  3.48s/it]
 39%|███▉      | 87787/225000 [162:23:17<171:44:20,  4.51s/it]
 39%|███▉      | 87788/225000 [162:23:23<185:47:45,  4.87s/it]
 39%|███▉      | 87789/225000 [162:23:26<161:47:54,  4.25s/it]
 39%|███▉      | 87790/225000 [162:23:28<140:30:30,  3.69s/it]
 39%|███▉      | 87791/225000 [162:23:32<141:19:26,  3.71s/it]
 39%|███▉      | 87792/225000 [162:23:39<180:30:01,  4.74s/it]
 39%|███▉      | 87793/225000 [162:23:42<154:41:21,  4.06s/it]
 39%|███▉      | 87794/225000 [162:23:45<142:56:30,  3.75s/it]
 39%|███▉      | 87795/225000 [162:23:47<128:23:46,  3.37s/it]
 39%|███▉      | 87796/225000 [162:23:50<121:46:05,  3.19s/it]
 39%|███▉      | 87797/225000 [162:23:58<180:34:31,  4.74s/it]
 39%|███▉      | 87798/225000 [162:24:01<155:20:00,  4.08s/it]
 39%|███▉      | 87799/225000 [162:24:03<136:13:23,  3.57s/it]
 39%|███▉      | 87800/225000 [162:24:06<122:34:36,  3.22s/it]
                                                              
{'loss': '0.392', 'grad_norm': '4.629', 'learning_rate': '3.553e-05', 'epoch': '0.3902', 'num_input_tokens_seen': 266340736, 'train_runtime': '5.846e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87800/225000 [162:24:06<122:34:36,  3.22s/it]
 39%|███▉      | 87801/225000 [162:24:08<117:31:00,  3.08s/it]
 39%|███▉      | 87802/225000 [162:24:12<124:10:20,  3.26s/it]
 39%|███▉      | 87803/225000 [162:24:17<140:08:11,  3.68s/it]
 39%|███▉      | 87804/225000 [162:24:26<201:01:24,  5.27s/it]
 39%|███▉      | 87805/225000 [162:24:30<189:19:07,  4.97s/it]
 39%|███▉      | 87806/225000 [162:24:32<161:47:52,  4.25s/it]
 39%|███▉      | 87807/225000 [162:24:38<172:53:02,  4.54s/it]
 39%|███▉      | 87808/225000 [162:24:42<165:41:07,  4.35s/it]
 39%|███▉      | 87809/225000 [162:24:45<155:13:49,  4.07s/it]
 39%|███▉      | 87810/225000 [162:24:50<168:14:50,  4.41s/it]
 39%|███▉      | 87811/225000 [162:24:54<165:34:01,  4.34s/it]
 39%|███▉      | 87812/225000 [162:24:57<145:49:07,  3.83s/it]
 39%|███▉      | 87813/225000 [162:25:04<178:38:57,  4.69s/it]
 39%|███▉      | 87814/225000 [162:25:06<153:39:24,  4.03s/it]
 39%|███▉      | 87815/225000 [162:25:09<142:42:36,  3.74s/it]
 39%|███▉      | 87816/225000 [162:25:12<136:42:31,  3.59s/it]
 39%|███▉      | 87817/225000 [162:25:15<127:35:42,  3.35s/it]
 39%|███▉      | 87818/225000 [162:25:23<182:16:09,  4.78s/it]
 39%|███▉      | 87819/225000 [162:25:28<175:39:30,  4.61s/it]
 39%|███▉      | 87820/225000 [162:25:30<154:19:31,  4.05s/it]
 39%|███▉      | 87821/225000 [162:25:33<138:05:19,  3.62s/it]
 39%|███▉      | 87822/225000 [162:25:37<136:53:50,  3.59s/it]
 39%|███▉      | 87823/225000 [162:25:40<139:52:29,  3.67s/it]
 39%|███▉      | 87824/225000 [162:25:45<153:40:13,  4.03s/it]
 39%|███▉      | 87825/225000 [162:25:53<194:23:49,  5.10s/it]
 39%|███▉      | 87826/225000 [162:25:55<164:23:05,  4.31s/it]
 39%|███▉      | 87827/225000 [162:25:58<149:18:44,  3.92s/it]
 39%|███▉      | 87828/225000 [162:26:01<136:16:40,  3.58s/it]
 39%|███▉      | 87829/225000 [162:26:10<191:53:33,  5.04s/it]
 39%|███▉      | 87830/225000 [162:26:13<170:52:53,  4.48s/it]
 39%|███▉      | 87831/225000 [162:26:15<149:16:23,  3.92s/it]
 39%|███▉      | 87832/225000 [162:26:18<137:52:19,  3.62s/it]
 39%|███▉      | 87833/225000 [162:26:21<128:51:07,  3.38s/it]
 39%|███▉      | 87834/225000 [162:26:24<120:18:24,  3.16s/it]
 39%|███▉      | 87835/225000 [162:26:27<117:33:45,  3.09s/it]
 39%|███▉      | 87836/225000 [162:26:34<168:10:54,  4.41s/it]
 39%|███▉      | 87837/225000 [162:26:40<184:28:44,  4.84s/it]
 39%|███▉      | 87838/225000 [162:26:43<164:17:14,  4.31s/it]
 39%|███▉      | 87839/225000 [162:26:49<182:07:52,  4.78s/it]
 39%|███▉      | 87840/225000 [162:26:53<177:41:20,  4.66s/it]
 39%|███▉      | 87841/225000 [162:26:56<153:36:25,  4.03s/it]
 39%|███▉      | 87842/225000 [162:27:04<197:04:23,  5.17s/it]
 39%|███▉      | 87843/225000 [162:27:06<163:34:36,  4.29s/it]
 39%|███▉      | 87844/225000 [162:27:09<149:29:18,  3.92s/it]
 39%|███▉      | 87845/225000 [162:27:15<168:25:28,  4.42s/it]
 39%|███▉      | 87846/225000 [162:27:17<150:12:12,  3.94s/it]
 39%|███▉      | 87847/225000 [162:27:20<135:36:44,  3.56s/it]
 39%|███▉      | 87848/225000 [162:27:23<132:30:18,  3.48s/it]
 39%|███▉      | 87849/225000 [162:27:26<126:09:18,  3.31s/it]
 39%|███▉      | 87850/225000 [162:27:29<124:55:33,  3.28s/it]
                                                              
{'loss': '0.4002', 'grad_norm': '5.931', 'learning_rate': '3.551e-05', 'epoch': '0.3904', 'num_input_tokens_seen': 266428704, 'train_runtime': '5.849e+05', 'train_tokens_per_second': '455.6'}

 39%|███▉      | 87850/225000 [162:27:29<124:55:33,  3.28s/it]
 39%|███▉      | 87851/225000 [162:27:32<120:34:20,  3.16s/it]
 39%|███▉      | 87852/225000 [162:27:38<149:23:07,  3.92s/it]
 39%|███▉      | 87853/225000 [162:27:46<199:52:51,  5.25s/it]
 39%|███▉      | 87854/225000 [162:27:49<168:10:01,  4.41s/it]
 39%|███▉      | 87855/225000 [162:27:51<147:14:24,  3.86s/it]
 39%|███▉      | 87856/225000 [162:27:54<130:43:29,  3.43s/it]
 39%|███▉      | 87857/225000 [162:27:57<121:54:01,  3.20s/it]
 39%|███▉      | 87858/225000 [162:28:00<123:12:32,  3.23s/it]
 39%|███▉      | 87859/225000 [162:28:05<141:14:07,  3.71s/it]
 39%|███▉      | 87860/225000 [162:28:14<201:57:58,  5.30s/it]
 39%|███▉      | 87861/225000 [162:28:17<173:47:31,  4.56s/it]
 39%|███▉      | 87862/225000 [162:28:19<153:08:16,  4.02s/it]
 39%|███▉      | 87863/225000 [162:28:22<139:11:40,  3.65s/it]
 39%|███▉      | 87864/225000 [162:28:25<129:43:38,  3.41s/it]
 39%|███▉      | 87865/225000 [162:28:28<123:18:05,  3.24s/it]
 39%|███▉      | 87866/225000 [162:28:35<171:39:29,  4.51s/it]
 39%|███▉      | 87867/225000 [162:28:39<165:06:55,  4.33s/it]
 39%|███▉      | 87868/225000 [162:28:42<147:11:48,  3.86s/it]
 39%|███▉      | 87869/225000 [162:28:45<143:11:11,  3.76s/it]
 39%|███▉      | 87870/225000 [162:28:49<140:12:05,  3.68s/it]
 39%|███▉      | 87871/225000 [162:28:53<141:26:33,  3.71s/it]
 39%|███▉      | 87872/225000 [162:28:56<134:18:21,  3.53s/it]
 39%|███▉      | 87873/225000 [162:29:00<138:41:20,  3.64s/it]
 39%|███▉      | 87874/225000 [162:29:03<136:35:52,  3.59s/it]
 39%|███▉      | 87875/225000 [162:29:07<135:46:09,  3.56s/it]
 39%|███▉      | 87876/225000 [162:29:11<140:01:16,  3.68s/it]
 39%|███▉      | 87877/225000 [162:29:14<140:11:50,  3.68s/it]
 39%|███▉      | 87878/225000 [162:29:18<142:38:05,  3.74s/it]
 39%|███▉      | 87879/225000 [162:29:21<136:48:32,  3.59s/it]
 39%|███▉      | 87880/225000 [162:29:27<162:26:00,  4.26s/it]
 39%|███▉      | 87881/225000 [162:29:36<210:12:12,  5.52s/it]
 39%|███▉      | 87882/225000 [162:29:41<201:40:06,  5.29s/it]
 39%|███▉      | 87883/225000 [162:29:43<167:44:16,  4.40s/it]
 39%|███▉      | 87884/225000 [162:29:48<175:55:58,  4.62s/it]
 39%|███▉      | 87885/225000 [162:29:53<175:33:42,  4.61s/it]
 39%|███▉      | 87886/225000 [162:29:55<155:07:31,  4.07s/it]
 39%|███▉      | 87887/225000 [162:29:58<139:42:52,  3.67s/it]
 39%|███▉      | 87888/225000 [162:30:01<129:27:24,  3.40s/it]
 39%|███▉      | 87889/225000 [162:30:04<121:46:07,  3.20s/it]
 39%|███▉      | 87890/225000 [162:30:06<116:40:57,  3.06s/it]
 39%|███▉      | 87891/225000 [162:30:15<176:09:52,  4.63s/it]
 39%|███▉      | 87892/225000 [162:30:20<187:22:18,  4.92s/it]
 39%|███▉      | 87893/225000 [162:30:26<192:18:24,  5.05s/it]
 39%|███▉      | 87894/225000 [162:30:31<195:11:32,  5.13s/it]
 39%|███▉      | 87895/225000 [162:30:34<174:13:18,  4.57s/it]
 39%|███▉      | 87896/225000 [162:30:37<158:41:39,  4.17s/it]
 39%|███▉      | 87897/225000 [162:30:41<151:35:05,  3.98s/it]
 39%|███▉      | 87898/225000 [162:30:44<142:49:45,  3.75s/it]
 39%|███▉      | 87899/225000 [162:30:47<134:19:35,  3.53s/it]
 39%|███▉      | 87900/225000 [162:30:50<129:58:08,  3.41s/it]
                                                              
{'loss': '0.4151', 'grad_norm': '4.282', 'learning_rate': '3.549e-05', 'epoch': '0.3907', 'num_input_tokens_seen': 266514720, 'train_runtime': '5.851e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 87900/225000 [162:30:50<129:58:08,  3.41s/it]
 39%|███▉      | 87901/225000 [162:30:54<132:44:47,  3.49s/it]
 39%|███▉      | 87902/225000 [162:30:57<126:17:44,  3.32s/it]
 39%|███▉      | 87903/225000 [162:31:00<123:30:06,  3.24s/it]
 39%|███▉      | 87904/225000 [162:31:03<122:14:20,  3.21s/it]
 39%|███▉      | 87905/225000 [162:31:06<119:46:26,  3.15s/it]
 39%|███▉      | 87906/225000 [162:31:14<172:56:29,  4.54s/it]
 39%|███▉      | 87907/225000 [162:31:23<221:52:30,  5.83s/it]
 39%|███▉      | 87908/225000 [162:31:25<183:50:32,  4.83s/it]
 39%|███▉      | 87909/225000 [162:31:28<157:27:58,  4.14s/it]
 39%|███▉      | 87910/225000 [162:31:31<143:41:40,  3.77s/it]
 39%|███▉      | 87911/225000 [162:31:36<167:01:42,  4.39s/it]
 39%|███▉      | 87912/225000 [162:31:44<207:41:25,  5.45s/it]
 39%|███▉      | 87913/225000 [162:31:47<171:38:36,  4.51s/it]
 39%|███▉      | 87914/225000 [162:31:49<148:37:55,  3.90s/it]
 39%|███▉      | 87915/225000 [162:31:52<134:54:43,  3.54s/it]
 39%|███▉      | 87916/225000 [162:31:55<132:01:38,  3.47s/it]
 39%|███▉      | 87917/225000 [162:31:58<124:00:45,  3.26s/it]
 39%|███▉      | 87918/225000 [162:32:01<124:38:46,  3.27s/it]
 39%|███▉      | 87919/225000 [162:32:04<122:39:11,  3.22s/it]
 39%|███▉      | 87920/225000 [162:32:10<149:10:49,  3.92s/it]
 39%|███▉      | 87921/225000 [162:32:18<194:30:55,  5.11s/it]
 39%|███▉      | 87922/225000 [162:32:21<170:36:35,  4.48s/it]
 39%|███▉      | 87923/225000 [162:32:23<149:52:05,  3.94s/it]
 39%|███▉      | 87924/225000 [162:32:26<138:33:10,  3.64s/it]
 39%|███▉      | 87925/225000 [162:32:30<135:36:31,  3.56s/it]
 39%|███▉      | 87926/225000 [162:32:33<130:51:59,  3.44s/it]
 39%|███▉      | 87927/225000 [162:32:36<128:03:30,  3.36s/it]
 39%|███▉      | 87928/225000 [162:32:44<178:47:59,  4.70s/it]
 39%|███▉      | 87929/225000 [162:32:51<202:38:16,  5.32s/it]
 39%|███▉      | 87930/225000 [162:32:53<172:01:37,  4.52s/it]
 39%|███▉      | 87931/225000 [162:32:56<149:09:46,  3.92s/it]
 39%|███▉      | 87932/225000 [162:32:58<134:13:59,  3.53s/it]
 39%|███▉      | 87933/225000 [162:33:02<128:30:14,  3.38s/it]
 39%|███▉      | 87934/225000 [162:33:05<127:18:09,  3.34s/it]
 39%|███▉      | 87935/225000 [162:33:09<133:57:04,  3.52s/it]
 39%|███▉      | 87936/225000 [162:33:13<138:20:59,  3.63s/it]
 39%|███▉      | 87937/225000 [162:33:16<138:47:02,  3.65s/it]
 39%|███▉      | 87938/225000 [162:33:20<137:16:11,  3.61s/it]
 39%|███▉      | 87939/225000 [162:33:23<138:06:32,  3.63s/it]
 39%|███▉      | 87940/225000 [162:33:27<137:28:17,  3.61s/it]
 39%|███▉      | 87941/225000 [162:33:30<133:57:33,  3.52s/it]
 39%|███▉      | 87942/225000 [162:33:33<129:34:27,  3.40s/it]
 39%|███▉      | 87943/225000 [162:33:39<153:49:01,  4.04s/it]
 39%|███▉      | 87944/225000 [162:33:45<174:00:31,  4.57s/it]
 39%|███▉      | 87945/225000 [162:33:48<159:47:58,  4.20s/it]
 39%|███▉      | 87946/225000 [162:33:52<150:32:07,  3.95s/it]
 39%|███▉      | 87947/225000 [162:33:55<146:34:55,  3.85s/it]
 39%|███▉      | 87948/225000 [162:33:59<150:38:42,  3.96s/it]
 39%|███▉      | 87949/225000 [162:34:03<145:04:47,  3.81s/it]
 39%|███▉      | 87950/225000 [162:34:06<139:46:47,  3.67s/it]
                                                              
{'loss': '0.3731', 'grad_norm': '4.433', 'learning_rate': '3.548e-05', 'epoch': '0.3909', 'num_input_tokens_seen': 266601272, 'train_runtime': '5.852e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 87950/225000 [162:34:06<139:46:47,  3.67s/it]
 39%|███▉      | 87951/225000 [162:34:10<136:11:32,  3.58s/it]
 39%|███▉      | 87952/225000 [162:34:17<183:03:01,  4.81s/it]
 39%|███▉      | 87953/225000 [162:34:25<211:52:10,  5.57s/it]
 39%|███▉      | 87954/225000 [162:34:27<176:39:40,  4.64s/it]
 39%|███▉      | 87955/225000 [162:34:30<152:14:30,  4.00s/it]
 39%|███▉      | 87956/225000 [162:34:32<138:00:17,  3.63s/it]
 39%|███▉      | 87957/225000 [162:34:35<133:19:34,  3.50s/it]
 39%|███▉      | 87958/225000 [162:34:39<128:02:11,  3.36s/it]
 39%|███▉      | 87959/225000 [162:34:45<167:16:59,  4.39s/it]
 39%|███▉      | 87960/225000 [162:34:54<218:29:15,  5.74s/it]
 39%|███▉      | 87961/225000 [162:34:57<179:50:39,  4.72s/it]
 39%|███▉      | 87962/225000 [162:34:59<155:04:59,  4.07s/it]
 39%|███▉      | 87963/225000 [162:35:02<142:11:25,  3.74s/it]
 39%|███▉      | 87964/225000 [162:35:08<162:48:08,  4.28s/it]
 39%|███▉      | 87965/225000 [162:35:13<178:11:58,  4.68s/it]
 39%|███▉      | 87966/225000 [162:35:16<161:30:24,  4.24s/it]
 39%|███▉      | 87967/225000 [162:35:19<145:02:08,  3.81s/it]
 39%|███▉      | 87968/225000 [162:35:22<132:56:58,  3.49s/it]
 39%|███▉      | 87969/225000 [162:35:25<124:34:30,  3.27s/it]
 39%|███▉      | 87970/225000 [162:35:32<165:45:12,  4.35s/it]
 39%|███▉      | 87971/225000 [162:35:38<188:09:44,  4.94s/it]
 39%|███▉      | 87972/225000 [162:35:41<161:05:38,  4.23s/it]
 39%|███▉      | 87973/225000 [162:35:45<167:46:44,  4.41s/it]
 39%|███▉      | 87974/225000 [162:35:50<173:07:20,  4.55s/it]
 39%|███▉      | 87975/225000 [162:35:53<153:19:55,  4.03s/it]
 39%|███▉      | 87976/225000 [162:35:56<141:07:30,  3.71s/it]
 39%|███▉      | 87977/225000 [162:35:59<130:55:03,  3.44s/it]
 39%|███▉      | 87978/225000 [162:36:02<125:58:04,  3.31s/it]
 39%|███▉      | 87979/225000 [162:36:09<166:40:06,  4.38s/it]
 39%|███▉      | 87980/225000 [162:36:16<203:57:19,  5.36s/it]
 39%|███▉      | 87981/225000 [162:36:19<174:40:05,  4.59s/it]
 39%|███▉      | 87982/225000 [162:36:22<157:29:21,  4.14s/it]
 39%|███▉      | 87983/225000 [162:36:25<146:49:13,  3.86s/it]
 39%|███▉      | 87984/225000 [162:36:28<137:05:25,  3.60s/it]
 39%|███▉      | 87985/225000 [162:36:32<132:53:05,  3.49s/it]
 39%|███▉      | 87986/225000 [162:36:35<129:12:50,  3.40s/it]
 39%|███▉      | 87987/225000 [162:36:38<126:19:46,  3.32s/it]
 39%|███▉      | 87988/225000 [162:36:41<122:39:28,  3.22s/it]
 39%|███▉      | 87989/225000 [162:36:44<120:13:17,  3.16s/it]
 39%|███▉      | 87990/225000 [162:36:47<119:04:52,  3.13s/it]
 39%|███▉      | 87991/225000 [162:36:51<123:12:10,  3.24s/it]
 39%|███▉      | 87992/225000 [162:36:58<172:10:02,  4.52s/it]
 39%|███▉      | 87993/225000 [162:37:06<216:45:44,  5.70s/it]
 39%|███▉      | 87994/225000 [162:37:10<187:26:46,  4.93s/it]
 39%|███▉      | 87995/225000 [162:37:13<165:10:26,  4.34s/it]
 39%|███▉      | 87996/225000 [162:37:16<151:47:13,  3.99s/it]
 39%|███▉      | 87997/225000 [162:37:19<140:05:26,  3.68s/it]
 39%|███▉      | 87998/225000 [162:37:28<202:08:28,  5.31s/it]
 39%|███▉      | 87999/225000 [162:37:32<184:13:29,  4.84s/it]
 39%|███▉      | 88000/225000 [162:37:35<162:27:47,  4.27s/it]
                                                              
{'loss': '0.3888', 'grad_norm': '4.147', 'learning_rate': '3.546e-05', 'epoch': '0.3911', 'num_input_tokens_seen': 266688832, 'train_runtime': '5.855e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88000/225000 [162:37:35<162:27:47,  4.27s/it]
 39%|███▉      | 88001/225000 [162:37:38<149:49:40,  3.94s/it]
 39%|███▉      | 88002/225000 [162:37:41<146:17:47,  3.84s/it]
 39%|███▉      | 88003/225000 [162:37:44<135:59:18,  3.57s/it]
 39%|███▉      | 88004/225000 [162:37:49<147:30:31,  3.88s/it]
 39%|███▉      | 88005/225000 [162:37:56<189:45:34,  4.99s/it]
 39%|███▉      | 88006/225000 [162:37:59<165:38:14,  4.35s/it]
 39%|███▉      | 88007/225000 [162:38:02<146:37:44,  3.85s/it]
 39%|███▉      | 88008/225000 [162:38:05<132:32:45,  3.48s/it]
 39%|███▉      | 88009/225000 [162:38:09<147:14:01,  3.87s/it]
 39%|███▉      | 88010/225000 [162:38:16<182:08:16,  4.79s/it]
 39%|███▉      | 88011/225000 [162:38:19<161:51:22,  4.25s/it]
 39%|███▉      | 88012/225000 [162:38:22<142:26:56,  3.74s/it]
 39%|███▉      | 88013/225000 [162:38:30<197:59:56,  5.20s/it]
 39%|███▉      | 88014/225000 [162:38:34<175:23:41,  4.61s/it]
 39%|███▉      | 88015/225000 [162:38:37<159:15:11,  4.19s/it]
 39%|███▉      | 88016/225000 [162:38:40<143:26:02,  3.77s/it]
 39%|███▉      | 88017/225000 [162:38:42<131:31:41,  3.46s/it]
 39%|███▉      | 88018/225000 [162:38:45<126:10:32,  3.32s/it]
 39%|███▉      | 88019/225000 [162:38:51<149:33:29,  3.93s/it]
 39%|███▉      | 88020/225000 [162:38:59<200:09:28,  5.26s/it]
 39%|███▉      | 88021/225000 [162:39:02<171:13:26,  4.50s/it]
 39%|███▉      | 88022/225000 [162:39:05<153:39:12,  4.04s/it]
 39%|███▉      | 88023/225000 [162:39:08<149:37:38,  3.93s/it]
 39%|███▉      | 88024/225000 [162:39:12<143:59:31,  3.78s/it]
 39%|███▉      | 88025/225000 [162:39:15<136:01:02,  3.57s/it]
 39%|███▉      | 88026/225000 [162:39:19<138:02:55,  3.63s/it]
 39%|███▉      | 88027/225000 [162:39:23<139:50:45,  3.68s/it]
 39%|███▉      | 88028/225000 [162:39:26<137:07:26,  3.60s/it]
 39%|███▉      | 88029/225000 [162:39:29<135:10:11,  3.55s/it]
 39%|███▉      | 88030/225000 [162:39:33<132:29:03,  3.48s/it]
 39%|███▉      | 88031/225000 [162:39:36<133:00:23,  3.50s/it]
 39%|███▉      | 88032/225000 [162:39:43<174:42:36,  4.59s/it]
 39%|███▉      | 88033/225000 [162:39:52<220:07:23,  5.79s/it]
 39%|███▉      | 88034/225000 [162:39:56<203:23:55,  5.35s/it]
 39%|███▉      | 88035/225000 [162:39:59<174:23:45,  4.58s/it]
 39%|███▉      | 88036/225000 [162:40:02<151:13:56,  3.98s/it]
 39%|███▉      | 88037/225000 [162:40:04<135:40:10,  3.57s/it]
 39%|███▉      | 88038/225000 [162:40:09<144:38:28,  3.80s/it]
 39%|███▉      | 88039/225000 [162:40:17<197:33:36,  5.19s/it]
 39%|███▉      | 88040/225000 [162:40:20<169:42:21,  4.46s/it]
 39%|███▉      | 88041/225000 [162:40:22<146:20:17,  3.85s/it]
 39%|███▉      | 88042/225000 [162:40:25<136:02:07,  3.58s/it]
 39%|███▉      | 88043/225000 [162:40:28<123:50:10,  3.26s/it]
 39%|███▉      | 88044/225000 [162:40:30<116:01:45,  3.05s/it]
 39%|███▉      | 88045/225000 [162:40:38<170:40:31,  4.49s/it]
 39%|███▉      | 88046/225000 [162:40:43<175:58:04,  4.63s/it]
 39%|███▉      | 88047/225000 [162:40:46<154:29:10,  4.06s/it]
 39%|███▉      | 88048/225000 [162:40:49<140:07:01,  3.68s/it]
 39%|███▉      | 88049/225000 [162:40:52<133:07:19,  3.50s/it]
 39%|███▉      | 88050/225000 [162:40:55<126:45:50,  3.33s/it]
                                                              
{'loss': '0.3693', 'grad_norm': '6.425', 'learning_rate': '3.544e-05', 'epoch': '0.3913', 'num_input_tokens_seen': 266777496, 'train_runtime': '5.857e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88050/225000 [162:40:55<126:45:50,  3.33s/it]
 39%|███▉      | 88051/225000 [162:40:58<123:05:42,  3.24s/it]
 39%|███▉      | 88052/225000 [162:41:01<125:05:02,  3.29s/it]
 39%|███▉      | 88053/225000 [162:41:04<124:28:21,  3.27s/it]
 39%|███▉      | 88054/225000 [162:41:11<165:38:52,  4.35s/it]
 39%|███▉      | 88055/225000 [162:41:19<208:55:25,  5.49s/it]
 39%|███▉      | 88056/225000 [162:41:22<177:10:54,  4.66s/it]
 39%|███▉      | 88057/225000 [162:41:29<201:37:05,  5.30s/it]
 39%|███▉      | 88058/225000 [162:41:35<206:18:47,  5.42s/it]
 39%|███▉      | 88059/225000 [162:41:37<172:00:21,  4.52s/it]
 39%|███▉      | 88060/225000 [162:41:39<149:38:32,  3.93s/it]
 39%|███▉      | 88061/225000 [162:41:42<133:24:00,  3.51s/it]
 39%|███▉      | 88062/225000 [162:41:50<184:13:42,  4.84s/it]
 39%|███▉      | 88063/225000 [162:41:53<162:59:32,  4.28s/it]
 39%|███▉      | 88064/225000 [162:41:55<142:37:45,  3.75s/it]
 39%|███▉      | 88065/225000 [162:41:59<141:49:55,  3.73s/it]
 39%|███▉      | 88066/225000 [162:42:03<140:27:27,  3.69s/it]
 39%|███▉      | 88067/225000 [162:42:06<134:56:56,  3.55s/it]
 39%|███▉      | 88068/225000 [162:42:09<131:05:04,  3.45s/it]
 39%|███▉      | 88069/225000 [162:42:14<144:32:08,  3.80s/it]
 39%|███▉      | 88070/225000 [162:42:22<197:39:02,  5.20s/it]
 39%|███▉      | 88071/225000 [162:42:26<185:46:04,  4.88s/it]
 39%|███▉      | 88072/225000 [162:42:32<192:48:16,  5.07s/it]
 39%|███▉      | 88073/225000 [162:42:35<170:46:30,  4.49s/it]
 39%|███▉      | 88074/225000 [162:42:38<153:02:54,  4.02s/it]
 39%|███▉      | 88075/225000 [162:42:41<140:48:42,  3.70s/it]
 39%|███▉      | 88076/225000 [162:42:44<138:05:40,  3.63s/it]
 39%|███▉      | 88077/225000 [162:42:47<131:28:01,  3.46s/it]
 39%|███▉      | 88078/225000 [162:42:52<145:01:47,  3.81s/it]
 39%|███▉      | 88079/225000 [162:43:00<189:52:30,  4.99s/it]
 39%|███▉      | 88080/225000 [162:43:02<160:51:21,  4.23s/it]
 39%|███▉      | 88081/225000 [162:43:05<140:46:42,  3.70s/it]
 39%|███▉      | 88082/225000 [162:43:09<147:56:52,  3.89s/it]
 39%|███▉      | 88083/225000 [162:43:15<167:19:30,  4.40s/it]
 39%|███▉      | 88084/225000 [162:43:17<148:37:42,  3.91s/it]
 39%|███▉      | 88085/225000 [162:43:20<134:15:41,  3.53s/it]
 39%|███▉      | 88086/225000 [162:43:24<134:56:44,  3.55s/it]
 39%|███▉      | 88087/225000 [162:43:32<193:11:38,  5.08s/it]
 39%|███▉      | 88088/225000 [162:43:35<167:08:26,  4.39s/it]
 39%|███▉      | 88089/225000 [162:43:38<154:49:05,  4.07s/it]
 39%|███▉      | 88090/225000 [162:43:41<141:30:05,  3.72s/it]
 39%|███▉      | 88091/225000 [162:43:44<133:41:55,  3.52s/it]
 39%|███▉      | 88092/225000 [162:43:47<125:54:11,  3.31s/it]
 39%|███▉      | 88093/225000 [162:43:51<127:23:21,  3.35s/it]
 39%|███▉      | 88094/225000 [162:43:59<182:16:00,  4.79s/it]
 39%|███▉      | 88095/225000 [162:44:02<167:36:56,  4.41s/it]
 39%|███▉      | 88096/225000 [162:44:05<146:10:44,  3.84s/it]
 39%|███▉      | 88097/225000 [162:44:07<130:16:22,  3.43s/it]
 39%|███▉      | 88098/225000 [162:44:10<118:56:36,  3.13s/it]
 39%|███▉      | 88099/225000 [162:44:13<120:30:57,  3.17s/it]
 39%|███▉      | 88100/225000 [162:44:16<118:51:22,  3.13s/it]
                                                              
{'loss': '0.3895', 'grad_norm': '4.652', 'learning_rate': '3.543e-05', 'epoch': '0.3916', 'num_input_tokens_seen': 266865776, 'train_runtime': '5.859e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88100/225000 [162:44:16<118:51:22,  3.13s/it]
 39%|███▉      | 88101/225000 [162:44:25<181:45:38,  4.78s/it]
 39%|███▉      | 88102/225000 [162:44:30<191:04:45,  5.02s/it]
 39%|███▉      | 88103/225000 [162:44:33<160:31:07,  4.22s/it]
 39%|███▉      | 88104/225000 [162:44:35<140:17:04,  3.69s/it]
 39%|███▉      | 88105/225000 [162:44:43<193:05:08,  5.08s/it]
 39%|███▉      | 88106/225000 [162:44:47<182:23:32,  4.80s/it]
 39%|███▉      | 88107/225000 [162:44:50<157:31:38,  4.14s/it]
 39%|███▉      | 88108/225000 [162:44:54<157:29:04,  4.14s/it]
 39%|███▉      | 88109/225000 [162:45:00<170:16:32,  4.48s/it]
 39%|███▉      | 88110/225000 [162:45:03<162:54:29,  4.28s/it]
 39%|███▉      | 88111/225000 [162:45:06<146:35:10,  3.86s/it]
 39%|███▉      | 88112/225000 [162:45:10<140:40:56,  3.70s/it]
 39%|███▉      | 88113/225000 [162:45:12<131:58:38,  3.47s/it]
 39%|███▉      | 88114/225000 [162:45:15<127:00:01,  3.34s/it]
 39%|███▉      | 88115/225000 [162:45:23<177:12:20,  4.66s/it]
 39%|███▉      | 88116/225000 [162:45:27<164:59:41,  4.34s/it]
 39%|███▉      | 88117/225000 [162:45:30<145:57:42,  3.84s/it]
 39%|███▉      | 88118/225000 [162:45:36<180:27:09,  4.75s/it]
 39%|███▉      | 88119/225000 [162:45:39<158:15:51,  4.16s/it]
 39%|███▉      | 88120/225000 [162:45:42<141:13:38,  3.71s/it]
 39%|███▉      | 88121/225000 [162:45:44<128:04:52,  3.37s/it]
 39%|███▉      | 88122/225000 [162:45:51<166:27:18,  4.38s/it]
 39%|███▉      | 88123/225000 [162:45:53<143:14:25,  3.77s/it]
 39%|███▉      | 88124/225000 [162:45:57<137:12:04,  3.61s/it]
 39%|███▉      | 88125/225000 [162:46:03<171:59:11,  4.52s/it]
 39%|███▉      | 88126/225000 [162:46:06<152:51:54,  4.02s/it]
 39%|███▉      | 88127/225000 [162:46:10<144:42:17,  3.81s/it]
 39%|███▉      | 88128/225000 [162:46:13<136:55:52,  3.60s/it]
 39%|███▉      | 88129/225000 [162:46:16<137:11:38,  3.61s/it]
 39%|███▉      | 88130/225000 [162:46:19<131:49:52,  3.47s/it]
 39%|███▉      | 88131/225000 [162:46:22<125:19:34,  3.30s/it]
 39%|███▉      | 88132/225000 [162:46:26<128:24:35,  3.38s/it]
 39%|███▉      | 88133/225000 [162:46:29<125:01:04,  3.29s/it]
 39%|███▉      | 88134/225000 [162:46:36<165:47:39,  4.36s/it]
 39%|███▉      | 88135/225000 [162:46:44<212:16:33,  5.58s/it]
 39%|███▉      | 88136/225000 [162:46:48<188:18:51,  4.95s/it]
 39%|███▉      | 88137/225000 [162:46:50<158:14:43,  4.16s/it]
 39%|███▉      | 88138/225000 [162:46:54<156:40:09,  4.12s/it]
 39%|███▉      | 88139/225000 [162:47:01<194:07:03,  5.11s/it]
 39%|███▉      | 88140/225000 [162:47:05<174:29:36,  4.59s/it]
 39%|███▉      | 88141/225000 [162:47:08<154:58:13,  4.08s/it]
 39%|███▉      | 88142/225000 [162:47:11<143:21:24,  3.77s/it]
 39%|███▉      | 88143/225000 [162:47:14<140:23:36,  3.69s/it]
 39%|███▉      | 88144/225000 [162:47:18<136:48:35,  3.60s/it]
 39%|███▉      | 88145/225000 [162:47:21<136:18:27,  3.59s/it]
 39%|███▉      | 88146/225000 [162:47:24<131:27:15,  3.46s/it]
 39%|███▉      | 88147/225000 [162:47:27<126:48:57,  3.34s/it]
 39%|███▉      | 88148/225000 [162:47:31<125:36:30,  3.30s/it]
 39%|███▉      | 88149/225000 [162:47:39<183:28:00,  4.83s/it]
 39%|███▉      | 88150/225000 [162:47:43<173:55:06,  4.58s/it]
                                                              
{'loss': '0.3854', 'grad_norm': '5.851', 'learning_rate': '3.541e-05', 'epoch': '0.3918', 'num_input_tokens_seen': 266954048, 'train_runtime': '5.861e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88150/225000 [162:47:43<173:55:06,  4.58s/it]
 39%|███▉      | 88151/225000 [162:47:47<161:52:30,  4.26s/it]
 39%|███▉      | 88152/225000 [162:47:50<153:13:31,  4.03s/it]
 39%|███▉      | 88153/225000 [162:47:53<142:25:44,  3.75s/it]
 39%|███▉      | 88154/225000 [162:47:56<137:38:23,  3.62s/it]
 39%|███▉      | 88155/225000 [162:48:02<160:30:37,  4.22s/it]
 39%|███▉      | 88156/225000 [162:48:05<147:31:41,  3.88s/it]
 39%|███▉      | 88157/225000 [162:48:08<139:20:01,  3.67s/it]
 39%|███▉      | 88158/225000 [162:48:12<135:56:22,  3.58s/it]
 39%|███▉      | 88159/225000 [162:48:18<172:21:00,  4.53s/it]
 39%|███▉      | 88160/225000 [162:48:26<204:13:59,  5.37s/it]
 39%|███▉      | 88161/225000 [162:48:29<175:49:20,  4.63s/it]
 39%|███▉      | 88162/225000 [162:48:31<152:39:09,  4.02s/it]
 39%|███▉      | 88163/225000 [162:48:34<140:25:24,  3.69s/it]
 39%|███▉      | 88164/225000 [162:48:37<132:27:05,  3.48s/it]
 39%|███▉      | 88165/225000 [162:48:41<130:55:59,  3.44s/it]
 39%|███▉      | 88166/225000 [162:48:43<124:08:44,  3.27s/it]
 39%|███▉      | 88167/225000 [162:48:51<176:51:20,  4.65s/it]
 39%|███▉      | 88168/225000 [162:48:56<180:33:46,  4.75s/it]
 39%|███▉      | 88169/225000 [162:48:59<157:58:53,  4.16s/it]
 39%|███▉      | 88170/225000 [162:49:06<189:00:36,  4.97s/it]
 39%|███▉      | 88171/225000 [162:49:09<169:39:18,  4.46s/it]
 39%|███▉      | 88172/225000 [162:49:12<152:28:22,  4.01s/it]
 39%|███▉      | 88173/225000 [162:49:16<145:05:38,  3.82s/it]
 39%|███▉      | 88174/225000 [162:49:19<135:14:41,  3.56s/it]
 39%|███▉      | 88175/225000 [162:49:22<129:57:50,  3.42s/it]
 39%|███▉      | 88176/225000 [162:49:30<187:09:14,  4.92s/it]
 39%|███▉      | 88177/225000 [162:49:34<177:17:23,  4.66s/it]
 39%|███▉      | 88178/225000 [162:49:37<153:09:08,  4.03s/it]
 39%|███▉      | 88179/225000 [162:49:40<141:27:27,  3.72s/it]
 39%|███▉      | 88180/225000 [162:49:43<135:03:05,  3.55s/it]
 39%|███▉      | 88181/225000 [162:49:46<133:41:43,  3.52s/it]
 39%|███▉      | 88182/225000 [162:49:50<134:32:46,  3.54s/it]
 39%|███▉      | 88183/225000 [162:49:53<132:24:07,  3.48s/it]
 39%|███▉      | 88184/225000 [162:50:00<174:51:45,  4.60s/it]
 39%|███▉      | 88185/225000 [162:50:09<225:36:40,  5.94s/it]
 39%|███▉      | 88186/225000 [162:50:12<191:29:55,  5.04s/it]
 39%|███▉      | 88187/225000 [162:50:15<164:16:15,  4.32s/it]
 39%|███▉      | 88188/225000 [162:50:18<148:17:31,  3.90s/it]
 39%|███▉      | 88189/225000 [162:50:21<138:09:41,  3.64s/it]
 39%|███▉      | 88190/225000 [162:50:24<130:06:44,  3.42s/it]
 39%|███▉      | 88191/225000 [162:50:28<138:24:48,  3.64s/it]
 39%|███▉      | 88192/225000 [162:50:36<189:55:23,  5.00s/it]
 39%|███▉      | 88193/225000 [162:50:39<162:07:05,  4.27s/it]
 39%|███▉      | 88194/225000 [162:50:42<145:33:49,  3.83s/it]
 39%|███▉      | 88195/225000 [162:50:44<131:35:22,  3.46s/it]
 39%|███▉      | 88196/225000 [162:50:47<125:50:57,  3.31s/it]
 39%|███▉      | 88197/225000 [162:50:53<158:12:08,  4.16s/it]
 39%|███▉      | 88198/225000 [162:51:01<201:49:44,  5.31s/it]
 39%|███▉      | 88199/225000 [162:51:04<169:23:32,  4.46s/it]
 39%|███▉      | 88200/225000 [162:51:06<149:00:14,  3.92s/it]
                                                              
{'loss': '0.3817', 'grad_norm': '3.641', 'learning_rate': '3.539e-05', 'epoch': '0.392', 'num_input_tokens_seen': 267041512, 'train_runtime': '5.863e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88200/225000 [162:51:06<149:00:14,  3.92s/it]
 39%|███▉      | 88201/225000 [162:51:09<137:25:10,  3.62s/it]
 39%|███▉      | 88202/225000 [162:51:12<129:19:28,  3.40s/it]
 39%|███▉      | 88203/225000 [162:51:22<197:47:09,  5.21s/it]
 39%|███▉      | 88204/225000 [162:51:24<168:26:40,  4.43s/it]
 39%|███▉      | 88205/225000 [162:51:30<180:07:58,  4.74s/it]
 39%|███▉      | 88206/225000 [162:51:33<161:59:54,  4.26s/it]
 39%|███▉      | 88207/225000 [162:51:36<144:20:07,  3.80s/it]
 39%|███▉      | 88208/225000 [162:51:38<130:41:23,  3.44s/it]
 39%|███▉      | 88209/225000 [162:51:41<123:11:13,  3.24s/it]
 39%|███▉      | 88210/225000 [162:51:47<155:48:37,  4.10s/it]
 39%|███▉      | 88211/225000 [162:51:55<194:47:52,  5.13s/it]
 39%|███▉      | 88212/225000 [162:51:57<163:13:41,  4.30s/it]
 39%|███▉      | 88213/225000 [162:52:00<145:59:40,  3.84s/it]
 39%|███▉      | 88214/225000 [162:52:03<134:20:11,  3.54s/it]
 39%|███▉      | 88215/225000 [162:52:05<124:12:34,  3.27s/it]
 39%|███▉      | 88216/225000 [162:52:08<120:30:59,  3.17s/it]
 39%|███▉      | 88217/225000 [162:52:12<122:26:10,  3.22s/it]
 39%|███▉      | 88218/225000 [162:52:15<127:05:44,  3.35s/it]
 39%|███▉      | 88219/225000 [162:52:19<130:52:47,  3.44s/it]
 39%|███▉      | 88220/225000 [162:52:22<132:07:53,  3.48s/it]
 39%|███▉      | 88221/225000 [162:52:26<134:09:40,  3.53s/it]
 39%|███▉      | 88222/225000 [162:52:29<133:22:54,  3.51s/it]
 39%|███▉      | 88223/225000 [162:52:33<136:06:16,  3.58s/it]
 39%|███▉      | 88224/225000 [162:52:37<133:21:13,  3.51s/it]
 39%|███▉      | 88225/225000 [162:52:40<134:25:10,  3.54s/it]
 39%|███▉      | 88226/225000 [162:52:44<132:19:35,  3.48s/it]
 39%|███▉      | 88227/225000 [162:52:47<135:35:45,  3.57s/it]
 39%|███▉      | 88228/225000 [162:52:51<135:25:14,  3.56s/it]
 39%|███▉      | 88229/225000 [162:52:57<160:14:24,  4.22s/it]
 39%|███▉      | 88230/225000 [162:53:06<214:46:32,  5.65s/it]
 39%|███▉      | 88231/225000 [162:53:11<208:27:25,  5.49s/it]
 39%|███▉      | 88232/225000 [162:53:14<184:33:23,  4.86s/it]
 39%|███▉      | 88233/225000 [162:53:18<169:23:28,  4.46s/it]
 39%|███▉      | 88234/225000 [162:53:21<157:37:18,  4.15s/it]
 39%|███▉      | 88235/225000 [162:53:25<153:39:02,  4.04s/it]
 39%|███▉      | 88236/225000 [162:53:32<185:42:08,  4.89s/it]
 39%|███▉      | 88237/225000 [162:53:34<159:13:55,  4.19s/it]
 39%|███▉      | 88238/225000 [162:53:37<143:32:54,  3.78s/it]
 39%|███▉      | 88239/225000 [162:53:40<137:20:48,  3.62s/it]
 39%|███▉      | 88240/225000 [162:53:46<160:08:15,  4.22s/it]
 39%|███▉      | 88241/225000 [162:53:50<155:47:19,  4.10s/it]
 39%|███▉      | 88242/225000 [162:53:53<146:37:37,  3.86s/it]
 39%|███▉      | 88243/225000 [162:53:57<143:17:03,  3.77s/it]
 39%|███▉      | 88244/225000 [162:54:00<138:56:39,  3.66s/it]
 39%|███▉      | 88245/225000 [162:54:07<181:25:59,  4.78s/it]
 39%|███▉      | 88246/225000 [162:54:11<167:17:32,  4.40s/it]
 39%|███▉      | 88247/225000 [162:54:14<147:48:49,  3.89s/it]
 39%|███▉      | 88248/225000 [162:54:16<135:31:19,  3.57s/it]
 39%|███▉      | 88249/225000 [162:54:19<125:29:17,  3.30s/it]
 39%|███▉      | 88250/225000 [162:54:22<123:02:50,  3.24s/it]
                                                              
{'loss': '0.3619', 'grad_norm': '8.886', 'learning_rate': '3.538e-05', 'epoch': '0.3922', 'num_input_tokens_seen': 267127992, 'train_runtime': '5.865e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88250/225000 [162:54:22<123:02:50,  3.24s/it]
 39%|███▉      | 88251/225000 [162:54:31<184:22:44,  4.85s/it]
 39%|███▉      | 88252/225000 [162:54:35<176:47:26,  4.65s/it]
 39%|███▉      | 88253/225000 [162:54:41<190:09:12,  5.01s/it]
 39%|███▉      | 88254/225000 [162:54:44<164:35:04,  4.33s/it]
 39%|███▉      | 88255/225000 [162:54:49<171:43:34,  4.52s/it]
 39%|███▉      | 88256/225000 [162:54:52<161:32:57,  4.25s/it]
 39%|███▉      | 88257/225000 [162:54:55<149:44:04,  3.94s/it]
 39%|███▉      | 88258/225000 [162:54:59<141:15:57,  3.72s/it]
 39%|███▉      | 88259/225000 [162:55:02<134:56:47,  3.55s/it]
 39%|███▉      | 88260/225000 [162:55:05<128:48:01,  3.39s/it]
 39%|███▉      | 88261/225000 [162:55:08<124:17:10,  3.27s/it]
 39%|███▉      | 88262/225000 [162:55:11<124:00:33,  3.26s/it]
 39%|███▉      | 88263/225000 [162:55:14<124:55:07,  3.29s/it]
 39%|███▉      | 88264/225000 [162:55:17<122:12:41,  3.22s/it]
 39%|███▉      | 88265/225000 [162:55:21<122:59:04,  3.24s/it]
 39%|███▉      | 88266/225000 [162:55:25<130:49:17,  3.44s/it]
 39%|███▉      | 88267/225000 [162:55:33<187:23:51,  4.93s/it]
 39%|███▉      | 88268/225000 [162:55:40<207:15:09,  5.46s/it]
 39%|███▉      | 88269/225000 [162:55:42<174:47:56,  4.60s/it]
 39%|███▉      | 88270/225000 [162:55:45<151:01:48,  3.98s/it]
 39%|███▉      | 88271/225000 [162:55:49<153:34:24,  4.04s/it]
 39%|███▉      | 88272/225000 [162:55:57<200:05:32,  5.27s/it]
 39%|███▉      | 88273/225000 [162:56:00<166:54:08,  4.39s/it]
 39%|███▉      | 88274/225000 [162:56:02<144:50:47,  3.81s/it]
 39%|███▉      | 88275/225000 [162:56:10<192:29:56,  5.07s/it]
 39%|███▉      | 88276/225000 [162:56:13<167:35:30,  4.41s/it]
 39%|███▉      | 88277/225000 [162:56:15<145:00:35,  3.82s/it]
 39%|███▉      | 88278/225000 [162:56:20<150:38:25,  3.97s/it]
 39%|███▉      | 88279/225000 [162:56:27<192:54:03,  5.08s/it]
 39%|███▉      | 88280/225000 [162:56:30<165:06:12,  4.35s/it]
 39%|███▉      | 88281/225000 [162:56:36<180:17:47,  4.75s/it]
 39%|███▉      | 88282/225000 [162:56:38<154:27:36,  4.07s/it]
 39%|███▉      | 88283/225000 [162:56:41<138:06:38,  3.64s/it]
 39%|███▉      | 88284/225000 [162:56:43<127:48:13,  3.37s/it]
 39%|███▉      | 88285/225000 [162:56:47<129:17:27,  3.40s/it]
 39%|███▉      | 88286/225000 [162:56:56<194:28:51,  5.12s/it]
 39%|███▉      | 88287/225000 [162:56:59<166:36:43,  4.39s/it]
 39%|███▉      | 88288/225000 [162:57:02<148:45:19,  3.92s/it]
 39%|███▉      | 88289/225000 [162:57:04<132:52:14,  3.50s/it]
 39%|███▉      | 88290/225000 [162:57:08<139:59:07,  3.69s/it]
 39%|███▉      | 88291/225000 [162:57:15<177:24:14,  4.67s/it]
 39%|███▉      | 88292/225000 [162:57:19<163:11:14,  4.30s/it]
 39%|███▉      | 88293/225000 [162:57:22<151:29:16,  3.99s/it]
 39%|███▉      | 88294/225000 [162:57:25<142:07:31,  3.74s/it]
 39%|███▉      | 88295/225000 [162:57:28<134:57:54,  3.55s/it]
 39%|███▉      | 88296/225000 [162:57:31<128:55:53,  3.40s/it]
 39%|███▉      | 88297/225000 [162:57:34<124:22:12,  3.28s/it]
 39%|███▉      | 88298/225000 [162:57:37<121:39:42,  3.20s/it]
 39%|███▉      | 88299/225000 [162:57:41<122:41:33,  3.23s/it]
 39%|███▉      | 88300/225000 [162:57:44<126:28:53,  3.33s/it]
                                                              
{'loss': '0.3579', 'grad_norm': '4.98', 'learning_rate': '3.536e-05', 'epoch': '0.3924', 'num_input_tokens_seen': 267216304, 'train_runtime': '5.867e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88300/225000 [162:57:44<126:28:53,  3.33s/it]
 39%|███▉      | 88301/225000 [162:57:52<173:48:22,  4.58s/it]
 39%|███▉      | 88302/225000 [162:57:59<200:34:33,  5.28s/it]
 39%|███▉      | 88303/225000 [162:58:01<172:27:14,  4.54s/it]
 39%|███▉      | 88304/225000 [162:58:04<155:58:47,  4.11s/it]
 39%|███▉      | 88305/225000 [162:58:07<141:54:17,  3.74s/it]
 39%|███▉      | 88306/225000 [162:58:12<147:15:48,  3.88s/it]
 39%|███▉      | 88307/225000 [162:58:20<199:15:51,  5.25s/it]
 39%|███▉      | 88308/225000 [162:58:23<170:36:19,  4.49s/it]
 39%|███▉      | 88309/225000 [162:58:25<147:19:22,  3.88s/it]
 39%|███▉      | 88310/225000 [162:58:28<139:34:29,  3.68s/it]
 39%|███▉      | 88311/225000 [162:58:36<184:30:32,  4.86s/it]
 39%|███▉      | 88312/225000 [162:58:40<180:19:15,  4.75s/it]
 39%|███▉      | 88313/225000 [162:58:43<159:29:55,  4.20s/it]
 39%|███▉      | 88314/225000 [162:58:46<144:57:37,  3.82s/it]
 39%|███▉      | 88315/225000 [162:58:49<132:55:38,  3.50s/it]
 39%|███▉      | 88316/225000 [162:58:52<124:25:45,  3.28s/it]
 39%|███▉      | 88317/225000 [162:58:55<124:29:07,  3.28s/it]
 39%|███▉      | 88318/225000 [162:58:59<126:53:08,  3.34s/it]
 39%|███▉      | 88319/225000 [162:59:02<127:13:44,  3.35s/it]
 39%|███▉      | 88320/225000 [162:59:05<129:04:08,  3.40s/it]
 39%|███▉      | 88321/225000 [162:59:09<129:21:41,  3.41s/it]
 39%|███▉      | 88322/225000 [162:59:16<169:44:11,  4.47s/it]
 39%|███▉      | 88323/225000 [162:59:21<182:28:43,  4.81s/it]
 39%|███▉      | 88324/225000 [162:59:25<168:37:14,  4.44s/it]
 39%|███▉      | 88325/225000 [162:59:29<158:53:14,  4.19s/it]
 39%|███▉      | 88326/225000 [162:59:32<144:50:25,  3.82s/it]
 39%|███▉      | 88327/225000 [162:59:34<132:03:13,  3.48s/it]
 39%|███▉      | 88328/225000 [162:59:37<125:58:13,  3.32s/it]
 39%|███▉      | 88329/225000 [162:59:45<177:04:04,  4.66s/it]
 39%|███▉      | 88330/225000 [162:59:51<193:12:31,  5.09s/it]
 39%|███▉      | 88331/225000 [162:59:54<165:00:21,  4.35s/it]
 39%|███▉      | 88332/225000 [162:59:56<144:47:40,  3.81s/it]
 39%|███▉      | 88333/225000 [163:00:04<184:44:57,  4.87s/it]
 39%|███▉      | 88334/225000 [163:00:08<175:45:08,  4.63s/it]
 39%|███▉      | 88335/225000 [163:00:10<152:33:41,  4.02s/it]
 39%|███▉      | 88336/225000 [163:00:15<161:50:49,  4.26s/it]
 39%|███▉      | 88337/225000 [163:00:20<172:35:27,  4.55s/it]
 39%|███▉      | 88338/225000 [163:00:23<154:36:21,  4.07s/it]
 39%|███▉      | 88339/225000 [163:00:26<144:30:30,  3.81s/it]
 39%|███▉      | 88340/225000 [163:00:30<140:14:09,  3.69s/it]
 39%|███▉      | 88341/225000 [163:00:37<176:11:40,  4.64s/it]
 39%|███▉      | 88342/225000 [163:00:44<207:41:15,  5.47s/it]
 39%|███▉      | 88343/225000 [163:00:47<173:42:49,  4.58s/it]
 39%|███▉      | 88344/225000 [163:00:50<162:30:36,  4.28s/it]
 39%|███▉      | 88345/225000 [163:00:55<170:15:09,  4.49s/it]
 39%|███▉      | 88346/225000 [163:00:58<149:58:21,  3.95s/it]
 39%|███▉      | 88347/225000 [163:01:01<135:05:27,  3.56s/it]
 39%|███▉      | 88348/225000 [163:01:04<129:28:08,  3.41s/it]
 39%|███▉      | 88349/225000 [163:01:07<125:16:29,  3.30s/it]
 39%|███▉      | 88350/225000 [163:01:10<126:58:57,  3.35s/it]
                                                              
{'loss': '0.3342', 'grad_norm': '3.857', 'learning_rate': '3.534e-05', 'epoch': '0.3927', 'num_input_tokens_seen': 267304304, 'train_runtime': '5.869e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88350/225000 [163:01:10<126:58:57,  3.35s/it]
 39%|███▉      | 88351/225000 [163:01:14<138:10:15,  3.64s/it]
 39%|███▉      | 88352/225000 [163:01:24<200:13:22,  5.27s/it]
 39%|███▉      | 88353/225000 [163:01:28<191:22:27,  5.04s/it]
 39%|███▉      | 88354/225000 [163:01:31<168:27:00,  4.44s/it]
 39%|███▉      | 88355/225000 [163:01:34<151:31:56,  3.99s/it]
 39%|███▉      | 88356/225000 [163:01:37<138:13:00,  3.64s/it]
 39%|███▉      | 88357/225000 [163:01:40<128:05:28,  3.37s/it]
 39%|███▉      | 88358/225000 [163:01:43<124:05:32,  3.27s/it]
 39%|███▉      | 88359/225000 [163:01:45<118:49:30,  3.13s/it]
 39%|███▉      | 88360/225000 [163:01:48<117:45:26,  3.10s/it]
 39%|███▉      | 88361/225000 [163:01:52<120:04:36,  3.16s/it]
 39%|███▉      | 88362/225000 [163:02:01<184:23:30,  4.86s/it]
 39%|███▉      | 88363/225000 [163:02:06<193:53:21,  5.11s/it]
 39%|███▉      | 88364/225000 [163:02:09<165:20:23,  4.36s/it]
 39%|███▉      | 88365/225000 [163:02:11<144:45:25,  3.81s/it]
 39%|███▉      | 88366/225000 [163:02:14<134:14:47,  3.54s/it]
 39%|███▉      | 88367/225000 [163:02:22<185:14:06,  4.88s/it]
 39%|███▉      | 88368/225000 [163:02:27<181:03:05,  4.77s/it]
 39%|███▉      | 88369/225000 [163:02:30<158:31:55,  4.18s/it]
 39%|███▉      | 88370/225000 [163:02:33<150:03:44,  3.95s/it]
 39%|███▉      | 88371/225000 [163:02:36<137:12:14,  3.62s/it]
 39%|███▉      | 88372/225000 [163:02:39<134:31:25,  3.54s/it]
 39%|███▉      | 88373/225000 [163:02:42<126:16:21,  3.33s/it]
 39%|███▉      | 88374/225000 [163:02:45<126:10:01,  3.32s/it]
 39%|███▉      | 88375/225000 [163:02:52<168:47:49,  4.45s/it]
 39%|███▉      | 88376/225000 [163:02:59<197:59:54,  5.22s/it]
 39%|███▉      | 88377/225000 [163:03:02<169:08:05,  4.46s/it]
 39%|███▉      | 88378/225000 [163:03:05<147:14:47,  3.88s/it]
 39%|███▉      | 88379/225000 [163:03:07<134:14:45,  3.54s/it]
 39%|███▉      | 88380/225000 [163:03:11<134:32:26,  3.55s/it]
 39%|███▉      | 88381/225000 [163:03:19<189:49:13,  5.00s/it]
 39%|███▉      | 88382/225000 [163:03:24<187:17:46,  4.94s/it]
 39%|███▉      | 88383/225000 [163:03:27<160:34:45,  4.23s/it]
 39%|███▉      | 88384/225000 [163:03:29<143:32:13,  3.78s/it]
 39%|███▉      | 88385/225000 [163:03:33<137:16:34,  3.62s/it]
 39%|███▉      | 88386/225000 [163:03:36<131:33:17,  3.47s/it]
 39%|███▉      | 88387/225000 [163:03:39<126:40:38,  3.34s/it]
 39%|███▉      | 88388/225000 [163:03:42<124:27:55,  3.28s/it]
 39%|███▉      | 88389/225000 [163:03:51<186:27:49,  4.91s/it]
 39%|███▉      | 88390/225000 [163:03:56<187:36:06,  4.94s/it]
 39%|███▉      | 88391/225000 [163:03:59<167:28:08,  4.41s/it]
 39%|███▉      | 88392/225000 [163:04:02<149:50:51,  3.95s/it]
 39%|███▉      | 88393/225000 [163:04:05<140:14:45,  3.70s/it]
 39%|███▉      | 88394/225000 [163:04:08<134:34:59,  3.55s/it]
 39%|███▉      | 88395/225000 [163:04:11<127:36:17,  3.36s/it]
 39%|███▉      | 88396/225000 [163:04:14<124:39:03,  3.28s/it]
 39%|███▉      | 88397/225000 [163:04:17<122:42:06,  3.23s/it]
 39%|███▉      | 88398/225000 [163:04:21<126:10:58,  3.33s/it]
 39%|███▉      | 88399/225000 [163:04:24<124:15:54,  3.27s/it]
 39%|███▉      | 88400/225000 [163:04:29<143:08:02,  3.77s/it]
                                                              
{'loss': '0.3777', 'grad_norm': '4.634', 'learning_rate': '3.533e-05', 'epoch': '0.3929', 'num_input_tokens_seen': 267391256, 'train_runtime': '5.871e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88400/225000 [163:04:29<143:08:02,  3.77s/it]
 39%|███▉      | 88401/225000 [163:04:37<195:49:23,  5.16s/it]
 39%|███▉      | 88402/225000 [163:04:43<203:12:41,  5.36s/it]
 39%|███▉      | 88403/225000 [163:04:46<180:08:14,  4.75s/it]
 39%|███▉      | 88404/225000 [163:04:50<161:56:20,  4.27s/it]
 39%|███▉      | 88405/225000 [163:04:53<155:52:29,  4.11s/it]
 39%|███▉      | 88406/225000 [163:04:57<150:33:19,  3.97s/it]
 39%|███▉      | 88407/225000 [163:05:01<148:13:27,  3.91s/it]
 39%|███▉      | 88408/225000 [163:05:04<137:09:21,  3.61s/it]
 39%|███▉      | 88409/225000 [163:05:08<143:31:47,  3.78s/it]
 39%|███▉      | 88410/225000 [163:05:17<201:53:51,  5.32s/it]
 39%|███▉      | 88411/225000 [163:05:23<214:02:19,  5.64s/it]
 39%|███▉      | 88412/225000 [163:05:26<180:17:00,  4.75s/it]
 39%|███▉      | 88413/225000 [163:05:29<158:02:35,  4.17s/it]
 39%|███▉      | 88414/225000 [163:05:31<142:17:45,  3.75s/it]
 39%|███▉      | 88415/225000 [163:05:40<201:47:45,  5.32s/it]
 39%|███▉      | 88416/225000 [163:05:44<177:35:47,  4.68s/it]
 39%|███▉      | 88417/225000 [163:05:47<158:35:18,  4.18s/it]
 39%|███▉      | 88418/225000 [163:05:50<147:09:03,  3.88s/it]
 39%|███▉      | 88419/225000 [163:05:53<135:29:56,  3.57s/it]
 39%|███▉      | 88420/225000 [163:05:55<124:44:38,  3.29s/it]
 39%|███▉      | 88421/225000 [163:06:01<154:11:12,  4.06s/it]
 39%|███▉      | 88422/225000 [163:06:08<183:57:24,  4.85s/it]
 39%|███▉      | 88423/225000 [163:06:10<157:43:04,  4.16s/it]
 39%|███▉      | 88424/225000 [163:06:13<138:28:50,  3.65s/it]
 39%|███▉      | 88425/225000 [163:06:18<162:00:56,  4.27s/it]
 39%|███▉      | 88426/225000 [163:06:22<158:50:56,  4.19s/it]
 39%|███▉      | 88427/225000 [163:06:25<144:11:22,  3.80s/it]
 39%|███▉      | 88428/225000 [163:06:28<133:10:35,  3.51s/it]
 39%|███▉      | 88429/225000 [163:06:31<121:01:00,  3.19s/it]
 39%|███▉      | 88430/225000 [163:06:39<181:13:14,  4.78s/it]
 39%|███▉      | 88431/225000 [163:06:43<175:49:50,  4.63s/it]
 39%|███▉      | 88432/225000 [163:06:46<155:21:25,  4.10s/it]
 39%|███▉      | 88433/225000 [163:06:50<147:58:01,  3.90s/it]
 39%|███▉      | 88434/225000 [163:06:53<136:55:07,  3.61s/it]
 39%|███▉      | 88435/225000 [163:06:56<128:39:42,  3.39s/it]
 39%|███▉      | 88436/225000 [163:06:59<128:54:02,  3.40s/it]
 39%|███▉      | 88437/225000 [163:07:02<130:29:13,  3.44s/it]
 39%|███▉      | 88438/225000 [163:07:10<180:16:08,  4.75s/it]
 39%|███▉      | 88439/225000 [163:07:16<185:55:40,  4.90s/it]
 39%|███▉      | 88440/225000 [163:07:18<159:45:37,  4.21s/it]
 39%|███▉      | 88441/225000 [163:07:23<162:15:22,  4.28s/it]
 39%|███▉      | 88442/225000 [163:07:28<174:40:39,  4.60s/it]
 39%|███▉      | 88443/225000 [163:07:32<167:33:03,  4.42s/it]
 39%|███▉      | 88444/225000 [163:07:35<155:19:43,  4.09s/it]
 39%|███▉      | 88445/225000 [163:07:39<146:18:33,  3.86s/it]
 39%|███▉      | 88446/225000 [163:07:42<141:16:46,  3.72s/it]
 39%|███▉      | 88447/225000 [163:07:45<138:39:24,  3.66s/it]
 39%|███▉      | 88448/225000 [163:07:52<173:39:13,  4.58s/it]
 39%|███▉      | 88449/225000 [163:07:56<164:42:07,  4.34s/it]
 39%|███▉      | 88450/225000 [163:07:59<152:40:47,  4.03s/it]
                                                              
{'loss': '0.3652', 'grad_norm': '4.655', 'learning_rate': '3.531e-05', 'epoch': '0.3931', 'num_input_tokens_seen': 267480488, 'train_runtime': '5.873e+05', 'train_tokens_per_second': '455.5'}

 39%|███▉      | 88450/225000 [163:07:59<152:40:47,  4.03s/it]
 39%|███▉      | 88451/225000 [163:08:03<147:46:46,  3.90s/it]
 39%|███▉      | 88452/225000 [163:08:06<137:45:16,  3.63s/it]
 39%|███▉      | 88453/225000 [163:08:09<132:12:47,  3.49s/it]
 39%|███▉      | 88454/225000 [163:08:18<192:08:47,  5.07s/it]
 39%|███▉      | 88455/225000 [163:08:21<171:37:34,  4.52s/it]
 39%|███▉      | 88456/225000 [163:08:24<151:54:28,  4.01s/it]
 39%|███▉      | 88457/225000 [163:08:26<135:47:44,  3.58s/it]
 39%|███▉      | 88458/225000 [163:08:29<126:50:56,  3.34s/it]
 39%|███▉      | 88459/225000 [163:08:32<123:09:37,  3.25s/it]
 39%|███▉      | 88460/225000 [163:08:36<124:37:20,  3.29s/it]
 39%|███▉      | 88461/225000 [163:08:39<125:08:07,  3.30s/it]
 39%|███▉      | 88462/225000 [163:08:48<185:27:53,  4.89s/it]
 39%|███▉      | 88463/225000 [163:08:51<173:01:59,  4.56s/it]
 39%|███▉      | 88464/225000 [163:08:55<157:55:05,  4.16s/it]
 39%|███▉      | 88465/225000 [163:09:00<168:16:02,  4.44s/it]
 39%|███▉      | 88466/225000 [163:09:03<155:55:58,  4.11s/it]
 39%|███▉      | 88467/225000 [163:09:06<145:51:47,  3.85s/it]
 39%|███▉      | 88468/225000 [163:09:09<135:01:18,  3.56s/it]
 39%|███▉      | 88469/225000 [163:09:12<130:46:20,  3.45s/it]
 39%|███▉      | 88470/225000 [163:09:16<128:13:22,  3.38s/it]
 39%|███▉      | 88471/225000 [163:09:19<124:29:08,  3.28s/it]
 39%|███▉      | 88472/225000 [163:09:22<125:41:48,  3.31s/it]
 39%|███▉      | 88473/225000 [163:09:26<136:15:16,  3.59s/it]
 39%|███▉      | 88474/225000 [163:09:35<198:05:03,  5.22s/it]
 39%|███▉      | 88475/225000 [163:09:41<199:43:21,  5.27s/it]
 39%|███▉      | 88476/225000 [163:09:44<181:31:46,  4.79s/it]
 39%|███▉      | 88477/225000 [163:09:47<162:23:52,  4.28s/it]
 39%|███▉      | 88478/225000 [163:09:51<151:25:26,  3.99s/it]
 39%|███▉      | 88479/225000 [163:09:54<141:02:47,  3.72s/it]
 39%|███▉      | 88480/225000 [163:09:57<139:35:52,  3.68s/it]
 39%|███▉      | 88481/225000 [163:10:00<132:17:10,  3.49s/it]
 39%|███▉      | 88482/225000 [163:10:04<128:44:45,  3.40s/it]
 39%|███▉      | 88483/225000 [163:10:07<132:37:31,  3.50s/it]
 39%|███▉      | 88484/225000 [163:10:11<137:32:26,  3.63s/it]
 39%|███▉      | 88485/225000 [163:10:21<202:42:39,  5.35s/it]
 39%|███▉      | 88486/225000 [163:10:27<215:06:52,  5.67s/it]
 39%|███▉      | 88487/225000 [163:10:30<185:02:52,  4.88s/it]
 39%|███▉      | 88488/225000 [163:10:33<161:33:26,  4.26s/it]
 39%|███▉      | 88489/225000 [163:10:36<148:10:54,  3.91s/it]
 39%|███▉      | 88490/225000 [163:10:39<138:13:35,  3.65s/it]
 39%|███▉      | 88491/225000 [163:10:45<159:22:11,  4.20s/it]
 39%|███▉      | 88492/225000 [163:10:52<195:52:48,  5.17s/it]
 39%|███▉      | 88493/225000 [163:10:54<164:51:52,  4.35s/it]
 39%|███▉      | 88494/225000 [163:10:57<143:45:21,  3.79s/it]
 39%|███▉      | 88495/225000 [163:10:59<129:17:07,  3.41s/it]
 39%|███▉      | 88496/225000 [163:11:05<151:24:50,  3.99s/it]
 39%|███▉      | 88497/225000 [163:11:10<170:30:41,  4.50s/it]
 39%|███▉      | 88498/225000 [163:11:14<162:47:08,  4.29s/it]
 39%|███▉      | 88499/225000 [163:11:21<189:29:28,  5.00s/it]
 39%|███▉      | 88500/225000 [163:11:24<166:44:41,  4.40s/it]
                                                              
{'loss': '0.3787', 'grad_norm': '5.575', 'learning_rate': '3.529e-05', 'epoch': '0.3933', 'num_input_tokens_seen': 267568016, 'train_runtime': '5.875e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88500/225000 [163:11:24<166:44:41,  4.40s/it]
 39%|███▉      | 88501/225000 [163:11:27<155:47:37,  4.11s/it]
 39%|███▉      | 88502/225000 [163:11:31<151:08:47,  3.99s/it]
 39%|███▉      | 88503/225000 [163:11:36<161:09:54,  4.25s/it]
 39%|███▉      | 88504/225000 [163:11:38<142:16:01,  3.75s/it]
 39%|███▉      | 88505/225000 [163:11:41<129:42:12,  3.42s/it]
 39%|███▉      | 88506/225000 [163:11:45<140:10:12,  3.70s/it]
 39%|███▉      | 88507/225000 [163:11:53<180:07:58,  4.75s/it]
 39%|███▉      | 88508/225000 [163:11:55<152:53:40,  4.03s/it]
 39%|███▉      | 88509/225000 [163:11:58<139:37:51,  3.68s/it]
 39%|███▉      | 88510/225000 [163:12:01<135:05:48,  3.56s/it]
 39%|███▉      | 88511/225000 [163:12:05<136:36:40,  3.60s/it]
 39%|███▉      | 88512/225000 [163:12:08<135:06:28,  3.56s/it]
 39%|███▉      | 88513/225000 [163:12:12<141:32:05,  3.73s/it]
 39%|███▉      | 88514/225000 [163:12:16<139:16:27,  3.67s/it]
 39%|███▉      | 88515/225000 [163:12:21<158:14:41,  4.17s/it]
 39%|███▉      | 88516/225000 [163:12:25<155:36:56,  4.10s/it]
 39%|███▉      | 88517/225000 [163:12:29<148:33:38,  3.92s/it]
 39%|███▉      | 88518/225000 [163:12:34<161:47:05,  4.27s/it]
 39%|███▉      | 88519/225000 [163:12:38<160:05:55,  4.22s/it]
 39%|███▉      | 88520/225000 [163:12:41<149:30:34,  3.94s/it]
 39%|███▉      | 88521/225000 [163:12:45<145:51:24,  3.85s/it]
 39%|███▉      | 88522/225000 [163:12:48<141:42:52,  3.74s/it]
 39%|███▉      | 88523/225000 [163:12:52<139:30:07,  3.68s/it]
 39%|███▉      | 88524/225000 [163:12:56<139:16:58,  3.67s/it]
 39%|███▉      | 88525/225000 [163:12:59<134:32:13,  3.55s/it]
 39%|███▉      | 88526/225000 [163:13:03<137:43:29,  3.63s/it]
 39%|███▉      | 88527/225000 [163:13:11<185:23:32,  4.89s/it]
 39%|███▉      | 88528/225000 [163:13:18<209:27:22,  5.53s/it]
 39%|███▉      | 88529/225000 [163:13:20<176:50:13,  4.66s/it]
 39%|███▉      | 88530/225000 [163:13:23<158:33:17,  4.18s/it]
 39%|███▉      | 88531/225000 [163:13:26<143:32:40,  3.79s/it]
 39%|███▉      | 88532/225000 [163:13:29<131:41:25,  3.47s/it]
 39%|███▉      | 88533/225000 [163:13:36<171:16:05,  4.52s/it]
 39%|███▉      | 88534/225000 [163:13:41<183:15:16,  4.83s/it]
 39%|███▉      | 88535/225000 [163:13:44<156:01:50,  4.12s/it]
 39%|███▉      | 88536/225000 [163:13:47<141:25:07,  3.73s/it]
 39%|███▉      | 88537/225000 [163:13:50<136:56:15,  3.61s/it]
 39%|███▉      | 88538/225000 [163:13:58<186:59:22,  4.93s/it]
 39%|███▉      | 88539/225000 [163:14:00<159:12:52,  4.20s/it]
 39%|███▉      | 88540/225000 [163:14:04<147:51:22,  3.90s/it]
 39%|███▉      | 88541/225000 [163:14:07<142:07:57,  3.75s/it]
 39%|███▉      | 88542/225000 [163:14:10<136:59:31,  3.61s/it]
 39%|███▉      | 88543/225000 [163:14:14<132:03:31,  3.48s/it]
 39%|███▉      | 88544/225000 [163:14:18<147:08:38,  3.88s/it]
 39%|███▉      | 88545/225000 [163:14:27<199:01:26,  5.25s/it]
 39%|███▉      | 88546/225000 [163:14:30<181:00:04,  4.78s/it]
 39%|███▉      | 88547/225000 [163:14:33<155:56:54,  4.11s/it]
 39%|███▉      | 88548/225000 [163:14:36<143:55:00,  3.80s/it]
 39%|███▉      | 88549/225000 [163:14:39<132:14:03,  3.49s/it]
 39%|███▉      | 88550/225000 [163:14:42<124:06:25,  3.27s/it]
                                                              
{'loss': '0.3558', 'grad_norm': '3.792', 'learning_rate': '3.528e-05', 'epoch': '0.3936', 'num_input_tokens_seen': 267655480, 'train_runtime': '5.877e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88550/225000 [163:14:42<124:06:25,  3.27s/it]
 39%|███▉      | 88551/225000 [163:14:50<184:06:38,  4.86s/it]
 39%|███▉      | 88552/225000 [163:14:53<156:35:21,  4.13s/it]
 39%|███▉      | 88553/225000 [163:14:55<140:58:53,  3.72s/it]
 39%|███▉      | 88554/225000 [163:15:03<182:13:21,  4.81s/it]
 39%|███▉      | 88555/225000 [163:15:07<171:16:02,  4.52s/it]
 39%|███▉      | 88556/225000 [163:15:10<163:06:07,  4.30s/it]
 39%|███▉      | 88557/225000 [163:15:13<148:49:55,  3.93s/it]
 39%|███▉      | 88558/225000 [163:15:17<139:28:18,  3.68s/it]
 39%|███▉      | 88559/225000 [163:15:20<136:49:41,  3.61s/it]
 39%|███▉      | 88560/225000 [163:15:23<132:05:07,  3.49s/it]
 39%|███▉      | 88561/225000 [163:15:27<130:36:35,  3.45s/it]
 39%|███▉      | 88562/225000 [163:15:30<127:24:51,  3.36s/it]
 39%|███▉      | 88563/225000 [163:15:33<128:54:19,  3.40s/it]
 39%|███▉      | 88564/225000 [163:15:37<128:30:34,  3.39s/it]
 39%|███▉      | 88565/225000 [163:15:40<131:59:24,  3.48s/it]
 39%|███▉      | 88566/225000 [163:15:44<130:41:13,  3.45s/it]
 39%|███▉      | 88567/225000 [163:15:47<132:34:13,  3.50s/it]
 39%|███▉      | 88568/225000 [163:15:51<133:57:25,  3.53s/it]
 39%|███▉      | 88569/225000 [163:15:54<134:52:10,  3.56s/it]
 39%|███▉      | 88570/225000 [163:16:00<153:29:00,  4.05s/it]
 39%|███▉      | 88571/225000 [163:16:09<210:32:49,  5.56s/it]
 39%|███▉      | 88572/225000 [163:16:14<203:18:17,  5.36s/it]
 39%|███▉      | 88573/225000 [163:16:20<219:15:28,  5.79s/it]
 39%|███▉      | 88574/225000 [163:16:23<177:47:44,  4.69s/it]
 39%|███▉      | 88575/225000 [163:16:29<193:54:37,  5.12s/it]
 39%|███▉      | 88576/225000 [163:16:32<168:53:58,  4.46s/it]
 39%|███▉      | 88577/225000 [163:16:35<152:36:38,  4.03s/it]
 39%|███▉      | 88578/225000 [163:16:38<139:42:49,  3.69s/it]
 39%|███▉      | 88579/225000 [163:16:41<135:23:28,  3.57s/it]
 39%|███▉      | 88580/225000 [163:16:44<127:43:18,  3.37s/it]
 39%|███▉      | 88581/225000 [163:16:47<122:56:06,  3.24s/it]
 39%|███▉      | 88582/225000 [163:16:50<125:23:55,  3.31s/it]
 39%|███▉      | 88583/225000 [163:16:57<166:12:24,  4.39s/it]
 39%|███▉      | 88584/225000 [163:17:05<201:47:37,  5.33s/it]
 39%|███▉      | 88585/225000 [163:17:07<169:33:10,  4.47s/it]
 39%|███▉      | 88586/225000 [163:17:10<147:25:42,  3.89s/it]
 39%|███▉      | 88587/225000 [163:17:13<140:25:35,  3.71s/it]
 39%|███▉      | 88588/225000 [163:17:17<141:36:45,  3.74s/it]
 39%|███▉      | 88589/225000 [163:17:21<149:21:29,  3.94s/it]
 39%|███▉      | 88590/225000 [163:17:25<145:36:32,  3.84s/it]
 39%|███▉      | 88591/225000 [163:17:33<199:10:28,  5.26s/it]
 39%|███▉      | 88592/225000 [163:17:36<172:32:48,  4.55s/it]
 39%|███▉      | 88593/225000 [163:17:39<157:47:06,  4.16s/it]
 39%|███▉      | 88594/225000 [163:17:46<181:37:01,  4.79s/it]
 39%|███▉      | 88595/225000 [163:17:49<160:34:38,  4.24s/it]
 39%|███▉      | 88596/225000 [163:17:51<144:33:42,  3.82s/it]
 39%|███▉      | 88597/225000 [163:17:54<135:12:24,  3.57s/it]
 39%|███▉      | 88598/225000 [163:17:57<127:45:20,  3.37s/it]
 39%|███▉      | 88599/225000 [163:18:00<120:32:34,  3.18s/it]
 39%|███▉      | 88600/225000 [163:18:06<152:52:10,  4.03s/it]
                                                              
{'loss': '0.3914', 'grad_norm': '6.391', 'learning_rate': '3.526e-05', 'epoch': '0.3938', 'num_input_tokens_seen': 267744264, 'train_runtime': '5.879e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88600/225000 [163:18:06<152:52:10,  4.03s/it]
 39%|███▉      | 88601/225000 [163:18:14<197:28:16,  5.21s/it]
 39%|███▉      | 88602/225000 [163:18:17<171:30:55,  4.53s/it]
 39%|███▉      | 88603/225000 [163:18:20<155:35:56,  4.11s/it]
 39%|███▉      | 88604/225000 [163:18:23<141:45:37,  3.74s/it]
 39%|███▉      | 88605/225000 [163:18:26<133:32:48,  3.52s/it]
 39%|███▉      | 88606/225000 [163:18:29<129:22:33,  3.41s/it]
 39%|███▉      | 88607/225000 [163:18:36<166:42:19,  4.40s/it]
 39%|███▉      | 88608/225000 [163:18:44<212:32:45,  5.61s/it]
 39%|███▉      | 88609/225000 [163:18:47<178:09:50,  4.70s/it]
 39%|███▉      | 88610/225000 [163:18:53<195:27:54,  5.16s/it]
 39%|███▉      | 88611/225000 [163:18:56<172:34:32,  4.56s/it]
 39%|███▉      | 88612/225000 [163:18:59<153:33:29,  4.05s/it]
 39%|███▉      | 88613/225000 [163:19:02<143:17:20,  3.78s/it]
 39%|███▉      | 88614/225000 [163:19:05<135:54:07,  3.59s/it]
 39%|███▉      | 88615/225000 [163:19:13<179:16:08,  4.73s/it]
 39%|███▉      | 88616/225000 [163:19:18<187:52:45,  4.96s/it]
 39%|███▉      | 88617/225000 [163:19:21<160:27:54,  4.24s/it]
 39%|███▉      | 88618/225000 [163:19:24<145:03:32,  3.83s/it]
 39%|███▉      | 88619/225000 [163:19:27<133:10:44,  3.52s/it]
 39%|███▉      | 88620/225000 [163:19:35<192:22:59,  5.08s/it]
 39%|███▉      | 88621/225000 [163:19:39<182:30:06,  4.82s/it]
 39%|███▉      | 88622/225000 [163:19:42<155:47:29,  4.11s/it]
 39%|███▉      | 88623/225000 [163:19:45<146:15:58,  3.86s/it]
 39%|███▉      | 88624/225000 [163:19:49<148:55:43,  3.93s/it]
 39%|███▉      | 88625/225000 [163:19:53<142:06:59,  3.75s/it]
 39%|███▉      | 88626/225000 [163:19:56<142:36:26,  3.76s/it]
 39%|███▉      | 88627/225000 [163:20:04<180:51:43,  4.77s/it]
 39%|███▉      | 88628/225000 [163:20:06<156:50:51,  4.14s/it]
 39%|███▉      | 88629/225000 [163:20:09<140:11:13,  3.70s/it]
 39%|███▉      | 88630/225000 [163:20:12<130:45:46,  3.45s/it]
 39%|███▉      | 88631/225000 [163:20:15<125:32:56,  3.31s/it]
 39%|███▉      | 88632/225000 [163:20:22<168:46:34,  4.46s/it]
 39%|███▉      | 88633/225000 [163:20:27<177:34:19,  4.69s/it]
 39%|███▉      | 88634/225000 [163:20:30<160:24:54,  4.23s/it]
 39%|███▉      | 88635/225000 [163:20:33<146:18:12,  3.86s/it]
 39%|███▉      | 88636/225000 [163:20:37<143:29:40,  3.79s/it]
 39%|███▉      | 88637/225000 [163:20:40<135:53:48,  3.59s/it]
 39%|███▉      | 88638/225000 [163:20:44<136:23:54,  3.60s/it]
 39%|███▉      | 88639/225000 [163:20:47<135:56:08,  3.59s/it]
 39%|███▉      | 88640/225000 [163:20:51<137:49:59,  3.64s/it]
 39%|███▉      | 88641/225000 [163:20:54<135:13:47,  3.57s/it]
 39%|███▉      | 88642/225000 [163:20:58<130:38:53,  3.45s/it]
 39%|███▉      | 88643/225000 [163:21:01<128:12:05,  3.38s/it]
 39%|███▉      | 88644/225000 [163:21:04<125:57:07,  3.33s/it]
 39%|███▉      | 88645/225000 [163:21:09<149:31:08,  3.95s/it]
 39%|███▉      | 88646/225000 [163:21:13<140:58:33,  3.72s/it]
 39%|███▉      | 88647/225000 [163:21:16<138:00:36,  3.64s/it]
 39%|███▉      | 88648/225000 [163:21:19<135:46:32,  3.58s/it]
 39%|███▉      | 88649/225000 [163:21:26<166:30:02,  4.40s/it]
 39%|███▉      | 88650/225000 [163:21:34<205:46:59,  5.43s/it]
                                                              
{'loss': '0.351', 'grad_norm': '3.033', 'learning_rate': '3.524e-05', 'epoch': '0.394', 'num_input_tokens_seen': 267832480, 'train_runtime': '5.881e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88650/225000 [163:21:34<205:46:59,  5.43s/it]
 39%|███▉      | 88651/225000 [163:21:37<180:53:06,  4.78s/it]
 39%|███▉      | 88652/225000 [163:21:40<159:21:28,  4.21s/it]
 39%|███▉      | 88653/225000 [163:21:42<141:30:57,  3.74s/it]
 39%|███▉      | 88654/225000 [163:21:48<165:58:11,  4.38s/it]
 39%|███▉      | 88655/225000 [163:21:54<185:39:16,  4.90s/it]
 39%|███▉      | 88656/225000 [163:21:57<156:55:12,  4.14s/it]
 39%|███▉      | 88657/225000 [163:21:59<140:24:29,  3.71s/it]
 39%|███▉      | 88658/225000 [163:22:04<151:09:43,  3.99s/it]
 39%|███▉      | 88659/225000 [163:22:11<182:12:15,  4.81s/it]
 39%|███▉      | 88660/225000 [163:22:14<158:46:49,  4.19s/it]
 39%|███▉      | 88661/225000 [163:22:16<140:00:23,  3.70s/it]
 39%|███▉      | 88662/225000 [163:22:19<135:42:28,  3.58s/it]
 39%|███▉      | 88663/225000 [163:22:22<125:01:41,  3.30s/it]
 39%|███▉      | 88664/225000 [163:22:31<184:09:40,  4.86s/it]
 39%|███▉      | 88665/225000 [163:22:33<157:34:30,  4.16s/it]
 39%|███▉      | 88666/225000 [163:22:36<144:01:54,  3.80s/it]
 39%|███▉      | 88667/225000 [163:22:39<132:30:00,  3.50s/it]
 39%|███▉      | 88668/225000 [163:22:43<142:10:17,  3.75s/it]
 39%|███▉      | 88669/225000 [163:22:51<193:44:41,  5.12s/it]
 39%|███▉      | 88670/225000 [163:22:54<164:35:59,  4.35s/it]
 39%|███▉      | 88671/225000 [163:22:56<142:25:26,  3.76s/it]
 39%|███▉      | 88672/225000 [163:22:59<131:35:34,  3.47s/it]
 39%|███▉      | 88673/225000 [163:23:02<124:56:42,  3.30s/it]
 39%|███▉      | 88674/225000 [163:23:11<188:15:02,  4.97s/it]
 39%|███▉      | 88675/225000 [163:23:16<191:28:38,  5.06s/it]
 39%|███▉      | 88676/225000 [163:23:19<162:21:51,  4.29s/it]
 39%|███▉      | 88677/225000 [163:23:21<140:45:39,  3.72s/it]
 39%|███▉      | 88678/225000 [163:23:24<127:34:06,  3.37s/it]
 39%|███▉      | 88679/225000 [163:23:27<125:33:44,  3.32s/it]
 39%|███▉      | 88680/225000 [163:23:31<131:05:38,  3.46s/it]
 39%|███▉      | 88681/225000 [163:23:34<130:35:57,  3.45s/it]
 39%|███▉      | 88682/225000 [163:23:38<130:26:07,  3.44s/it]
 39%|███▉      | 88683/225000 [163:23:41<125:13:38,  3.31s/it]
 39%|███▉      | 88684/225000 [163:23:44<123:12:16,  3.25s/it]
 39%|███▉      | 88685/225000 [163:23:47<124:53:43,  3.30s/it]
 39%|███▉      | 88686/225000 [163:23:50<125:58:36,  3.33s/it]
 39%|███▉      | 88687/225000 [163:24:00<195:03:47,  5.15s/it]
 39%|███▉      | 88688/225000 [163:24:08<229:24:04,  6.06s/it]
 39%|███▉      | 88689/225000 [163:24:11<191:43:25,  5.06s/it]
 39%|███▉      | 88690/225000 [163:24:14<168:34:16,  4.45s/it]
 39%|███▉      | 88691/225000 [163:24:17<155:04:15,  4.10s/it]
 39%|███▉      | 88692/225000 [163:24:20<146:21:33,  3.87s/it]
 39%|███▉      | 88693/225000 [163:24:23<134:09:45,  3.54s/it]
 39%|███▉      | 88694/225000 [163:24:26<128:58:57,  3.41s/it]
 39%|███▉      | 88695/225000 [163:24:29<122:16:00,  3.23s/it]
 39%|███▉      | 88696/225000 [163:24:32<118:25:09,  3.13s/it]
 39%|███▉      | 88697/225000 [163:24:35<117:29:49,  3.10s/it]
 39%|███▉      | 88698/225000 [163:24:39<121:42:04,  3.21s/it]
 39%|███▉      | 88699/225000 [163:24:42<125:44:52,  3.32s/it]
 39%|███▉      | 88700/225000 [163:24:49<170:13:58,  4.50s/it]
                                                              
{'loss': '0.3522', 'grad_norm': '2.944', 'learning_rate': '3.523e-05', 'epoch': '0.3942', 'num_input_tokens_seen': 267920120, 'train_runtime': '5.883e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88700/225000 [163:24:49<170:13:58,  4.50s/it]
 39%|███▉      | 88701/225000 [163:24:59<228:37:33,  6.04s/it]
 39%|███▉      | 88702/225000 [163:25:03<202:04:09,  5.34s/it]
 39%|███▉      | 88703/225000 [163:25:05<169:27:53,  4.48s/it]
 39%|███▉      | 88704/225000 [163:25:08<146:18:46,  3.86s/it]
 39%|███▉      | 88705/225000 [163:25:15<188:15:19,  4.97s/it]
 39%|███▉      | 88706/225000 [163:25:18<164:11:53,  4.34s/it]
 39%|███▉      | 88707/225000 [163:25:20<143:14:10,  3.78s/it]
 39%|███▉      | 88708/225000 [163:25:24<137:33:47,  3.63s/it]
 39%|███▉      | 88709/225000 [163:25:27<130:34:16,  3.45s/it]
 39%|███▉      | 88710/225000 [163:25:30<129:42:20,  3.43s/it]
 39%|███▉      | 88711/225000 [163:25:34<129:49:27,  3.43s/it]
 39%|███▉      | 88712/225000 [163:25:37<127:09:36,  3.36s/it]
 39%|███▉      | 88713/225000 [163:25:40<126:13:36,  3.33s/it]
 39%|███▉      | 88714/225000 [163:25:44<133:25:09,  3.52s/it]
 39%|███▉      | 88715/225000 [163:25:48<138:46:17,  3.67s/it]
 39%|███▉      | 88716/225000 [163:25:52<138:30:10,  3.66s/it]
 39%|███▉      | 88717/225000 [163:25:56<140:41:58,  3.72s/it]
 39%|███▉      | 88718/225000 [163:26:03<184:43:13,  4.88s/it]
 39%|███▉      | 88719/225000 [163:26:13<245:24:22,  6.48s/it]
 39%|███▉      | 88720/225000 [163:26:16<199:47:32,  5.28s/it]
 39%|███▉      | 88721/225000 [163:26:19<170:48:28,  4.51s/it]
 39%|███▉      | 88722/225000 [163:26:22<155:05:40,  4.10s/it]
 39%|███▉      | 88723/225000 [163:26:25<142:25:21,  3.76s/it]
 39%|███▉      | 88724/225000 [163:26:33<196:16:59,  5.19s/it]
 39%|███▉      | 88725/225000 [163:26:37<181:14:23,  4.79s/it]
 39%|███▉      | 88726/225000 [163:26:40<159:51:20,  4.22s/it]
 39%|███▉      | 88727/225000 [163:26:45<165:19:05,  4.37s/it]
 39%|███▉      | 88728/225000 [163:26:49<170:01:37,  4.49s/it]
 39%|███▉      | 88729/225000 [163:26:53<158:55:51,  4.20s/it]
 39%|███▉      | 88730/225000 [163:26:58<172:12:04,  4.55s/it]
 39%|███▉      | 88731/225000 [163:27:01<151:51:28,  4.01s/it]
 39%|███▉      | 88732/225000 [163:27:04<136:28:49,  3.61s/it]
 39%|███▉      | 88733/225000 [163:27:06<126:45:28,  3.35s/it]
 39%|███▉      | 88734/225000 [163:27:09<123:14:06,  3.26s/it]
 39%|███▉      | 88735/225000 [163:27:12<119:16:05,  3.15s/it]
 39%|███▉      | 88736/225000 [163:27:21<181:11:04,  4.79s/it]
 39%|███▉      | 88737/225000 [163:27:26<185:40:25,  4.91s/it]
 39%|███▉      | 88738/225000 [163:27:29<158:13:57,  4.18s/it]
 39%|███▉      | 88739/225000 [163:27:32<148:01:29,  3.91s/it]
 39%|███▉      | 88740/225000 [163:27:35<137:02:03,  3.62s/it]
 39%|███▉      | 88741/225000 [163:27:38<134:14:20,  3.55s/it]
 39%|███▉      | 88742/225000 [163:27:41<125:04:37,  3.30s/it]
 39%|███▉      | 88743/225000 [163:27:44<122:33:15,  3.24s/it]
 39%|███▉      | 88744/225000 [163:27:51<164:18:36,  4.34s/it]
 39%|███▉      | 88745/225000 [163:28:00<213:09:03,  5.63s/it]
 39%|███▉      | 88746/225000 [163:28:02<180:46:40,  4.78s/it]
 39%|███▉      | 88747/225000 [163:28:05<153:27:49,  4.05s/it]
 39%|███▉      | 88748/225000 [163:28:08<147:23:54,  3.89s/it]
 39%|███▉      | 88749/225000 [163:28:16<191:13:30,  5.05s/it]
 39%|███▉      | 88750/225000 [163:28:19<167:38:37,  4.43s/it]
                                                              
{'loss': '0.348', 'grad_norm': '3.821', 'learning_rate': '3.521e-05', 'epoch': '0.3944', 'num_input_tokens_seen': 268008088, 'train_runtime': '5.885e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88750/225000 [163:28:19<167:38:37,  4.43s/it]
 39%|███▉      | 88751/225000 [163:28:21<145:05:25,  3.83s/it]
 39%|███▉      | 88752/225000 [163:28:24<129:53:35,  3.43s/it]
 39%|███▉      | 88753/225000 [163:28:32<181:54:21,  4.81s/it]
 39%|███▉      | 88754/225000 [163:28:35<161:43:48,  4.27s/it]
 39%|███▉      | 88755/225000 [163:28:39<152:59:59,  4.04s/it]
 39%|███▉      | 88756/225000 [163:28:43<159:40:10,  4.22s/it]
 39%|███▉      | 88757/225000 [163:28:49<181:36:06,  4.80s/it]
 39%|███▉      | 88758/225000 [163:28:52<155:44:44,  4.12s/it]
 39%|███▉      | 88759/225000 [163:28:56<157:14:13,  4.15s/it]
 39%|███▉      | 88760/225000 [163:29:01<162:40:52,  4.30s/it]
 39%|███▉      | 88761/225000 [163:29:03<142:48:47,  3.77s/it]
 39%|███▉      | 88762/225000 [163:29:07<143:54:24,  3.80s/it]
 39%|███▉      | 88763/225000 [163:29:14<179:14:21,  4.74s/it]
 39%|███▉      | 88764/225000 [163:29:17<155:56:55,  4.12s/it]
 39%|███▉      | 88765/225000 [163:29:20<143:47:55,  3.80s/it]
 39%|███▉      | 88766/225000 [163:29:23<134:59:18,  3.57s/it]
 39%|███▉      | 88767/225000 [163:29:26<131:04:16,  3.46s/it]
 39%|███▉      | 88768/225000 [163:29:29<125:08:20,  3.31s/it]
 39%|███▉      | 88769/225000 [163:29:35<155:45:20,  4.12s/it]
 39%|███▉      | 88770/225000 [163:29:43<194:59:32,  5.15s/it]
 39%|███▉      | 88771/225000 [163:29:45<165:32:05,  4.37s/it]
 39%|███▉      | 88772/225000 [163:29:50<167:01:27,  4.41s/it]
 39%|███▉      | 88773/225000 [163:29:54<165:27:47,  4.37s/it]
 39%|███▉      | 88774/225000 [163:29:57<149:37:13,  3.95s/it]
 39%|███▉      | 88775/225000 [163:30:00<144:47:05,  3.83s/it]
 39%|███▉      | 88776/225000 [163:30:04<140:16:38,  3.71s/it]
 39%|███▉      | 88777/225000 [163:30:07<137:11:41,  3.63s/it]
 39%|███▉      | 88778/225000 [163:30:11<135:39:48,  3.59s/it]
 39%|███▉      | 88779/225000 [163:30:14<133:54:11,  3.54s/it]
 39%|███▉      | 88780/225000 [163:30:18<132:35:03,  3.50s/it]
 39%|███▉      | 88781/225000 [163:30:21<136:02:04,  3.60s/it]
 39%|███▉      | 88782/225000 [163:30:25<139:52:56,  3.70s/it]
 39%|███▉      | 88783/225000 [163:30:31<160:32:48,  4.24s/it]
 39%|███▉      | 88784/225000 [163:30:39<208:33:30,  5.51s/it]
 39%|███▉      | 88785/225000 [163:30:44<198:40:35,  5.25s/it]
 39%|███▉      | 88786/225000 [163:30:46<164:44:11,  4.35s/it]
 39%|███▉      | 88787/225000 [163:30:49<145:56:55,  3.86s/it]
 39%|███▉      | 88788/225000 [163:30:56<183:41:27,  4.85s/it]
 39%|███▉      | 88789/225000 [163:30:59<156:44:20,  4.14s/it]
 39%|███▉      | 88790/225000 [163:31:02<144:41:26,  3.82s/it]
 39%|███▉      | 88791/225000 [163:31:08<173:59:50,  4.60s/it]
 39%|███▉      | 88792/225000 [163:31:13<183:14:15,  4.84s/it]
 39%|███▉      | 88793/225000 [163:31:16<156:16:04,  4.13s/it]
 39%|███▉      | 88794/225000 [163:31:22<173:53:29,  4.60s/it]
 39%|███▉      | 88795/225000 [163:31:24<151:38:57,  4.01s/it]
 39%|███▉      | 88796/225000 [163:31:27<140:55:45,  3.72s/it]
 39%|███▉      | 88797/225000 [163:31:30<125:13:46,  3.31s/it]
 39%|███▉      | 88798/225000 [163:31:34<132:24:36,  3.50s/it]
 39%|███▉      | 88799/225000 [163:31:40<170:09:43,  4.50s/it]
 39%|███▉      | 88800/225000 [163:31:44<153:59:20,  4.07s/it]
                                                              
{'loss': '0.3214', 'grad_norm': '4.007', 'learning_rate': '3.519e-05', 'epoch': '0.3947', 'num_input_tokens_seen': 268096200, 'train_runtime': '5.887e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88800/225000 [163:31:44<153:59:20,  4.07s/it]
 39%|███▉      | 88801/225000 [163:31:46<141:01:42,  3.73s/it]
 39%|███▉      | 88802/225000 [163:31:53<174:12:51,  4.60s/it]
 39%|███▉      | 88803/225000 [163:31:57<166:19:43,  4.40s/it]
 39%|███▉      | 88804/225000 [163:32:00<153:22:32,  4.05s/it]
 39%|███▉      | 88805/225000 [163:32:03<141:15:52,  3.73s/it]
 39%|███▉      | 88806/225000 [163:32:07<139:33:16,  3.69s/it]
 39%|███▉      | 88807/225000 [163:32:10<130:49:46,  3.46s/it]
 39%|███▉      | 88808/225000 [163:32:15<151:23:29,  4.00s/it]
 39%|███▉      | 88809/225000 [163:32:24<207:44:05,  5.49s/it]
 39%|███▉      | 88810/225000 [163:32:27<181:57:13,  4.81s/it]
 39%|███▉      | 88811/225000 [163:32:30<161:47:08,  4.28s/it]
 39%|███▉      | 88812/225000 [163:32:33<145:15:25,  3.84s/it]
 39%|███▉      | 88813/225000 [163:32:36<136:22:52,  3.61s/it]
 39%|███▉      | 88814/225000 [163:32:39<128:17:51,  3.39s/it]
 39%|███▉      | 88815/225000 [163:32:42<123:43:25,  3.27s/it]
 39%|███▉      | 88816/225000 [163:32:46<129:13:26,  3.42s/it]
 39%|███▉      | 88817/225000 [163:32:49<122:40:28,  3.24s/it]
 39%|███▉      | 88818/225000 [163:32:57<176:43:44,  4.67s/it]
 39%|███▉      | 88819/225000 [163:33:03<191:06:56,  5.05s/it]
 39%|███▉      | 88820/225000 [163:33:05<163:06:24,  4.31s/it]
 39%|███▉      | 88821/225000 [163:33:09<156:29:53,  4.14s/it]
 39%|███▉      | 88822/225000 [163:33:13<151:18:24,  4.00s/it]
 39%|███▉      | 88823/225000 [163:33:20<189:09:47,  5.00s/it]
 39%|███▉      | 88824/225000 [163:33:23<170:29:32,  4.51s/it]
 39%|███▉      | 88825/225000 [163:33:26<153:48:26,  4.07s/it]
 39%|███▉      | 88826/225000 [163:33:30<147:30:26,  3.90s/it]
 39%|███▉      | 88827/225000 [163:33:33<134:53:13,  3.57s/it]
 39%|███▉      | 88828/225000 [163:33:36<133:37:28,  3.53s/it]
 39%|███▉      | 88829/225000 [163:33:39<125:13:24,  3.31s/it]
 39%|███▉      | 88830/225000 [163:33:46<172:45:16,  4.57s/it]
 39%|███▉      | 88831/225000 [163:33:53<193:04:17,  5.10s/it]
 39%|███▉      | 88832/225000 [163:33:56<169:54:54,  4.49s/it]
 39%|███▉      | 88833/225000 [163:33:58<150:04:24,  3.97s/it]
 39%|███▉      | 88834/225000 [163:34:01<138:16:22,  3.66s/it]
 39%|███▉      | 88835/225000 [163:34:04<128:30:23,  3.40s/it]
 39%|███▉      | 88836/225000 [163:34:07<124:47:10,  3.30s/it]
 39%|███▉      | 88837/225000 [163:34:15<173:20:57,  4.58s/it]
 39%|███▉      | 88838/225000 [163:34:19<166:17:35,  4.40s/it]
 39%|███▉      | 88839/225000 [163:34:22<152:32:17,  4.03s/it]
 39%|███▉      | 88840/225000 [163:34:25<142:11:23,  3.76s/it]
 39%|███▉      | 88841/225000 [163:34:28<135:29:41,  3.58s/it]
 39%|███▉      | 88842/225000 [163:34:32<137:36:58,  3.64s/it]
 39%|███▉      | 88843/225000 [163:34:35<135:23:45,  3.58s/it]
 39%|███▉      | 88844/225000 [163:34:38<128:12:40,  3.39s/it]
 39%|███▉      | 88845/225000 [163:34:41<124:21:31,  3.29s/it]
 39%|███▉      | 88846/225000 [163:34:45<123:18:09,  3.26s/it]
 39%|███▉      | 88847/225000 [163:34:48<125:26:53,  3.32s/it]
 39%|███▉      | 88848/225000 [163:34:55<163:43:54,  4.33s/it]
 39%|███▉      | 88849/225000 [163:35:03<207:16:36,  5.48s/it]
 39%|███▉      | 88850/225000 [163:35:06<176:21:47,  4.66s/it]
                                                              
{'loss': '0.3585', 'grad_norm': '6.135', 'learning_rate': '3.518e-05', 'epoch': '0.3949', 'num_input_tokens_seen': 268182736, 'train_runtime': '5.889e+05', 'train_tokens_per_second': '455.4'}

 39%|███▉      | 88850/225000 [163:35:06<176:21:47,  4.66s/it]
 39%|███▉      | 88851/225000 [163:35:09<160:23:22,  4.24s/it]
 39%|███▉      | 88852/225000 [163:35:13<155:00:07,  4.10s/it]
 39%|███▉      | 88853/225000 [163:35:17<153:14:16,  4.05s/it]
 39%|███▉      | 88854/225000 [163:35:24<185:31:08,  4.91s/it]
 39%|███▉      | 88855/225000 [163:35:27<165:14:44,  4.37s/it]
 39%|███▉      | 88856/225000 [163:35:30<147:34:53,  3.90s/it]
 39%|███▉      | 88857/225000 [163:35:33<137:31:38,  3.64s/it]
 39%|███▉      | 88858/225000 [163:35:35<128:26:29,  3.40s/it]
 39%|███▉      | 88859/225000 [163:35:41<150:55:30,  3.99s/it]
 39%|███▉      | 88860/225000 [163:35:50<208:04:13,  5.50s/it]
 39%|███▉      | 88861/225000 [163:35:53<182:36:44,  4.83s/it]
 39%|███▉      | 88862/225000 [163:35:56<160:20:38,  4.24s/it]
 39%|███▉      | 88863/225000 [163:35:59<142:50:08,  3.78s/it]
 39%|███▉      | 88864/225000 [163:36:01<126:57:45,  3.36s/it]
 39%|███▉      | 88865/225000 [163:36:04<120:35:20,  3.19s/it]
 39%|███▉      | 88866/225000 [163:36:12<180:40:54,  4.78s/it]
 39%|███▉      | 88867/225000 [163:36:17<184:11:43,  4.87s/it]
 39%|███▉      | 88868/225000 [163:36:21<170:53:45,  4.52s/it]
 39%|███▉      | 88869/225000 [163:36:24<157:01:52,  4.15s/it]
 39%|███▉      | 88870/225000 [163:36:27<145:07:10,  3.84s/it]
 39%|███▉      | 88871/225000 [163:36:31<141:36:09,  3.74s/it]
 39%|███▉      | 88872/225000 [163:36:35<144:31:45,  3.82s/it]
 39%|███▉      | 88873/225000 [163:36:39<141:24:59,  3.74s/it]
 39%|███▉      | 88874/225000 [163:36:42<137:36:23,  3.64s/it]
 40%|███▉      | 88875/225000 [163:36:46<137:31:27,  3.64s/it]
 40%|███▉      | 88876/225000 [163:36:52<165:04:21,  4.37s/it]
 40%|███▉      | 88877/225000 [163:37:01<217:21:59,  5.75s/it]
 40%|███▉      | 88878/225000 [163:37:04<193:50:47,  5.13s/it]
 40%|███▉      | 88879/225000 [163:37:08<180:28:29,  4.77s/it]
 40%|███▉      | 88880/225000 [163:37:11<155:35:59,  4.12s/it]
 40%|███▉      | 88881/225000 [163:37:13<138:55:02,  3.67s/it]
 40%|███▉      | 88882/225000 [163:37:17<131:57:09,  3.49s/it]
 40%|███▉      | 88883/225000 [163:37:26<196:24:55,  5.19s/it]
 40%|███▉      | 88884/225000 [163:37:29<169:58:01,  4.50s/it]
 40%|███▉      | 88885/225000 [163:37:31<149:48:14,  3.96s/it]
 40%|███▉      | 88886/225000 [163:37:35<147:52:15,  3.91s/it]
 40%|███▉      | 88887/225000 [163:37:39<143:55:49,  3.81s/it]
 40%|███▉      | 88888/225000 [163:37:42<137:20:27,  3.63s/it]
 40%|███▉      | 88889/225000 [163:37:45<137:19:41,  3.63s/it]
 40%|███▉      | 88890/225000 [163:37:49<135:51:06,  3.59s/it]
 40%|███▉      | 88891/225000 [163:37:53<141:52:23,  3.75s/it]
 40%|███▉      | 88892/225000 [163:38:00<172:59:23,  4.58s/it]
 40%|███▉      | 88893/225000 [163:38:07<204:26:24,  5.41s/it]
 40%|███▉      | 88894/225000 [163:38:10<176:59:12,  4.68s/it]
 40%|███▉      | 88895/225000 [163:38:14<165:27:24,  4.38s/it]
 40%|███▉      | 88896/225000 [163:38:17<156:19:24,  4.13s/it]
 40%|███▉      | 88897/225000 [163:38:20<146:56:16,  3.89s/it]
 40%|███▉      | 88898/225000 [163:38:24<142:12:06,  3.76s/it]
 40%|███▉      | 88899/225000 [163:38:28<141:11:29,  3.73s/it]
 40%|███▉      | 88900/225000 [163:38:31<138:09:25,  3.65s/it]
                                                              
{'loss': '0.3496', 'grad_norm': '6.2', 'learning_rate': '3.516e-05', 'epoch': '0.3951', 'num_input_tokens_seen': 268272256, 'train_runtime': '5.891e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 88900/225000 [163:38:31<138:09:25,  3.65s/it]
 40%|███▉      | 88901/225000 [163:38:34<134:52:24,  3.57s/it]
 40%|███▉      | 88902/225000 [163:38:42<179:57:14,  4.76s/it]
 40%|███▉      | 88903/225000 [163:38:49<208:10:05,  5.51s/it]
 40%|███▉      | 88904/225000 [163:38:52<177:14:12,  4.69s/it]
 40%|███▉      | 88905/225000 [163:38:55<153:56:38,  4.07s/it]
 40%|███▉      | 88906/225000 [163:38:57<138:59:19,  3.68s/it]
 40%|███▉      | 88907/225000 [163:39:01<134:15:06,  3.55s/it]
 40%|███▉      | 88908/225000 [163:39:03<122:24:18,  3.24s/it]
 40%|███▉      | 88909/225000 [163:39:08<143:52:20,  3.81s/it]
 40%|███▉      | 88910/225000 [163:39:17<203:06:55,  5.37s/it]
 40%|███▉      | 88911/225000 [163:39:20<173:40:37,  4.59s/it]
 40%|███▉      | 88912/225000 [163:39:23<156:02:26,  4.13s/it]
 40%|███▉      | 88913/225000 [163:39:26<146:42:20,  3.88s/it]
 40%|███▉      | 88914/225000 [163:39:29<133:28:10,  3.53s/it]
 40%|███▉      | 88915/225000 [163:39:32<125:17:43,  3.31s/it]
 40%|███▉      | 88916/225000 [163:39:35<121:28:23,  3.21s/it]
 40%|███▉      | 88917/225000 [163:39:40<145:22:03,  3.85s/it]
 40%|███▉      | 88918/225000 [163:39:50<210:09:37,  5.56s/it]
 40%|███▉      | 88919/225000 [163:39:53<187:45:48,  4.97s/it]
 40%|███▉      | 88920/225000 [163:39:56<164:56:00,  4.36s/it]
 40%|███▉      | 88921/225000 [163:39:59<148:45:43,  3.94s/it]
 40%|███▉      | 88922/225000 [163:40:02<135:54:02,  3.60s/it]
 40%|███▉      | 88923/225000 [163:40:06<136:43:18,  3.62s/it]
 40%|███▉      | 88924/225000 [163:40:09<131:13:29,  3.47s/it]
 40%|███▉      | 88925/225000 [163:40:14<148:23:07,  3.93s/it]
 40%|███▉      | 88926/225000 [163:40:22<190:55:19,  5.05s/it]
 40%|███▉      | 88927/225000 [163:40:25<175:18:59,  4.64s/it]
 40%|███▉      | 88928/225000 [163:40:28<158:02:02,  4.18s/it]
 40%|███▉      | 88929/225000 [163:40:34<171:43:13,  4.54s/it]
 40%|███▉      | 88930/225000 [163:40:37<157:19:53,  4.16s/it]
 40%|███▉      | 88931/225000 [163:40:40<145:58:15,  3.86s/it]
 40%|███▉      | 88932/225000 [163:40:43<136:38:31,  3.62s/it]
 40%|███▉      | 88933/225000 [163:40:47<133:50:26,  3.54s/it]
 40%|███▉      | 88934/225000 [163:40:50<129:40:12,  3.43s/it]
 40%|███▉      | 88935/225000 [163:40:53<127:34:35,  3.38s/it]
 40%|███▉      | 88936/225000 [163:40:56<124:22:05,  3.29s/it]
 40%|███▉      | 88937/225000 [163:41:00<130:36:15,  3.46s/it]
 40%|███▉      | 88938/225000 [163:41:04<131:45:53,  3.49s/it]
 40%|███▉      | 88939/225000 [163:41:08<141:36:54,  3.75s/it]
 40%|███▉      | 88940/225000 [163:41:11<137:08:06,  3.63s/it]
 40%|███▉      | 88941/225000 [163:41:19<179:52:31,  4.76s/it]
 40%|███▉      | 88942/225000 [163:41:27<221:48:30,  5.87s/it]
 40%|███▉      | 88943/225000 [163:41:31<193:56:50,  5.13s/it]
 40%|███▉      | 88944/225000 [163:41:33<167:17:56,  4.43s/it]
 40%|███▉      | 88945/225000 [163:41:37<155:26:00,  4.11s/it]
 40%|███▉      | 88946/225000 [163:41:40<141:11:12,  3.74s/it]
 40%|███▉      | 88947/225000 [163:41:45<165:21:46,  4.38s/it]
 40%|███▉      | 88948/225000 [163:41:52<190:03:49,  5.03s/it]
 40%|███▉      | 88949/225000 [163:41:54<161:20:58,  4.27s/it]
 40%|███▉      | 88950/225000 [163:41:57<142:47:45,  3.78s/it]
                                                              
{'loss': '0.347', 'grad_norm': '5.721', 'learning_rate': '3.514e-05', 'epoch': '0.3953', 'num_input_tokens_seen': 268361360, 'train_runtime': '5.893e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 88950/225000 [163:41:57<142:47:45,  3.78s/it]
 40%|███▉      | 88951/225000 [163:42:00<131:20:59,  3.48s/it]
 40%|███▉      | 88952/225000 [163:42:06<160:20:03,  4.24s/it]
 40%|███▉      | 88953/225000 [163:42:13<192:56:46,  5.11s/it]
 40%|███▉      | 88954/225000 [163:42:16<174:38:28,  4.62s/it]
 40%|███▉      | 88955/225000 [163:42:20<156:24:08,  4.14s/it]
 40%|███▉      | 88956/225000 [163:42:23<145:34:39,  3.85s/it]
 40%|███▉      | 88957/225000 [163:42:26<142:25:07,  3.77s/it]
 40%|███▉      | 88958/225000 [163:42:30<138:14:41,  3.66s/it]
 40%|███▉      | 88959/225000 [163:42:33<138:22:56,  3.66s/it]
 40%|███▉      | 88960/225000 [163:42:36<130:16:39,  3.45s/it]
 40%|███▉      | 88961/225000 [163:42:39<127:41:19,  3.38s/it]
 40%|███▉      | 88962/225000 [163:42:43<123:26:29,  3.27s/it]
 40%|███▉      | 88963/225000 [163:42:46<124:17:00,  3.29s/it]
 40%|███▉      | 88964/225000 [163:42:49<120:46:24,  3.20s/it]
 40%|███▉      | 88965/225000 [163:42:57<178:02:45,  4.71s/it]
 40%|███▉      | 88966/225000 [163:43:05<209:08:39,  5.53s/it]
 40%|███▉      | 88967/225000 [163:43:07<179:07:54,  4.74s/it]
 40%|███▉      | 88968/225000 [163:43:10<157:43:04,  4.17s/it]
 40%|███▉      | 88969/225000 [163:43:13<140:19:06,  3.71s/it]
 40%|███▉      | 88970/225000 [163:43:15<127:33:10,  3.38s/it]
 40%|███▉      | 88971/225000 [163:43:18<121:16:51,  3.21s/it]
 40%|███▉      | 88972/225000 [163:43:27<178:01:09,  4.71s/it]
 40%|███▉      | 88973/225000 [163:43:32<187:49:52,  4.97s/it]
 40%|███▉      | 88974/225000 [163:43:35<164:35:31,  4.36s/it]
 40%|███▉      | 88975/225000 [163:43:38<143:42:13,  3.80s/it]
 40%|███▉      | 88976/225000 [163:43:43<164:26:56,  4.35s/it]
 40%|███▉      | 88977/225000 [163:43:49<185:21:19,  4.91s/it]
 40%|███▉      | 88978/225000 [163:43:52<157:47:56,  4.18s/it]
 40%|███▉      | 88979/225000 [163:43:54<140:14:12,  3.71s/it]
 40%|███▉      | 88980/225000 [163:43:57<129:37:59,  3.43s/it]
 40%|███▉      | 88981/225000 [163:44:00<126:18:01,  3.34s/it]
 40%|███▉      | 88982/225000 [163:44:04<128:09:45,  3.39s/it]
 40%|███▉      | 88983/225000 [163:44:07<129:00:38,  3.41s/it]
 40%|███▉      | 88984/225000 [163:44:11<129:10:56,  3.42s/it]
 40%|███▉      | 88985/225000 [163:44:15<136:42:22,  3.62s/it]
 40%|███▉      | 88986/225000 [163:44:22<177:27:26,  4.70s/it]
 40%|███▉      | 88987/225000 [163:44:31<224:12:05,  5.93s/it]
 40%|███▉      | 88988/225000 [163:44:33<186:09:01,  4.93s/it]
 40%|███▉      | 88989/225000 [163:44:38<178:57:09,  4.74s/it]
 40%|███▉      | 88990/225000 [163:44:43<179:03:45,  4.74s/it]
 40%|███▉      | 88991/225000 [163:44:45<156:58:40,  4.16s/it]
 40%|███▉      | 88992/225000 [163:44:48<140:51:10,  3.73s/it]
 40%|███▉      | 88993/225000 [163:44:51<129:26:15,  3.43s/it]
 40%|███▉      | 88994/225000 [163:44:54<125:07:01,  3.31s/it]
 40%|███▉      | 88995/225000 [163:45:00<155:40:41,  4.12s/it]
 40%|███▉      | 88996/225000 [163:45:09<209:22:55,  5.54s/it]
 40%|███▉      | 88997/225000 [163:45:12<183:02:32,  4.85s/it]
 40%|███▉      | 88998/225000 [163:45:14<157:27:37,  4.17s/it]
 40%|███▉      | 88999/225000 [163:45:18<149:36:16,  3.96s/it]
 40%|███▉      | 89000/225000 [163:45:21<141:39:15,  3.75s/it]
                                                              
{'loss': '0.3417', 'grad_norm': '4.752', 'learning_rate': '3.513e-05', 'epoch': '0.3956', 'num_input_tokens_seen': 268451400, 'train_runtime': '5.895e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 89000/225000 [163:45:21<141:39:15,  3.75s/it]
 40%|███▉      | 89001/225000 [163:45:24<134:47:11,  3.57s/it]
 40%|███▉      | 89002/225000 [163:45:33<193:51:22,  5.13s/it]
 40%|███▉      | 89003/225000 [163:45:36<170:11:11,  4.51s/it]
 40%|███▉      | 89004/225000 [163:45:39<146:11:23,  3.87s/it]
 40%|███▉      | 89005/225000 [163:45:42<146:11:00,  3.87s/it]
 40%|███▉      | 89006/225000 [163:45:46<137:38:37,  3.64s/it]
 40%|███▉      | 89007/225000 [163:45:49<131:53:36,  3.49s/it]
 40%|███▉      | 89008/225000 [163:45:52<126:04:11,  3.34s/it]
 40%|███▉      | 89009/225000 [163:45:55<122:35:16,  3.25s/it]
 40%|███▉      | 89010/225000 [163:45:58<123:12:39,  3.26s/it]
 40%|███▉      | 89011/225000 [163:46:02<127:16:43,  3.37s/it]
 40%|███▉      | 89012/225000 [163:46:05<127:29:49,  3.38s/it]
 40%|███▉      | 89013/225000 [163:46:08<124:42:43,  3.30s/it]
 40%|███▉      | 89014/225000 [163:46:13<142:10:12,  3.76s/it]
 40%|███▉      | 89015/225000 [163:46:23<217:00:11,  5.74s/it]
 40%|███▉      | 89016/225000 [163:46:28<210:11:25,  5.56s/it]
 40%|███▉      | 89017/225000 [163:46:31<173:38:17,  4.60s/it]
 40%|███▉      | 89018/225000 [163:46:33<148:59:00,  3.94s/it]
 40%|███▉      | 89019/225000 [163:46:36<139:13:47,  3.69s/it]
 40%|███▉      | 89020/225000 [163:46:41<144:53:22,  3.84s/it]
 40%|███▉      | 89021/225000 [163:46:44<140:32:05,  3.72s/it]
 40%|███▉      | 89022/225000 [163:46:47<132:17:30,  3.50s/it]
 40%|███▉      | 89023/225000 [163:46:51<134:23:49,  3.56s/it]
 40%|███▉      | 89024/225000 [163:46:55<139:28:20,  3.69s/it]
 40%|███▉      | 89025/225000 [163:46:59<146:42:31,  3.88s/it]
 40%|███▉      | 89026/225000 [163:47:09<213:04:55,  5.64s/it]
 40%|███▉      | 89027/225000 [163:47:16<226:39:40,  6.00s/it]
 40%|███▉      | 89028/225000 [163:47:18<187:17:44,  4.96s/it]
 40%|███▉      | 89029/225000 [163:47:21<158:38:42,  4.20s/it]
 40%|███▉      | 89030/225000 [163:47:23<136:36:24,  3.62s/it]
 40%|███▉      | 89031/225000 [163:47:26<127:52:27,  3.39s/it]
 40%|███▉      | 89032/225000 [163:47:33<174:39:10,  4.62s/it]
 40%|███▉      | 89033/225000 [163:47:38<177:52:40,  4.71s/it]
 40%|███▉      | 89034/225000 [163:47:41<155:24:31,  4.11s/it]
 40%|███▉      | 89035/225000 [163:47:43<139:19:47,  3.69s/it]
 40%|███▉      | 89036/225000 [163:47:46<127:27:19,  3.37s/it]
 40%|███▉      | 89037/225000 [163:47:49<124:26:27,  3.29s/it]
 40%|███▉      | 89038/225000 [163:47:53<129:39:04,  3.43s/it]
 40%|███▉      | 89039/225000 [163:48:03<199:28:02,  5.28s/it]
 40%|███▉      | 89040/225000 [163:48:06<174:39:41,  4.62s/it]
 40%|███▉      | 89041/225000 [163:48:09<163:00:25,  4.32s/it]
 40%|███▉      | 89042/225000 [163:48:13<154:36:46,  4.09s/it]
 40%|███▉      | 89043/225000 [163:48:17<152:02:36,  4.03s/it]
 40%|███▉      | 89044/225000 [163:48:24<185:00:23,  4.90s/it]
 40%|███▉      | 89045/225000 [163:48:27<167:57:50,  4.45s/it]
 40%|███▉      | 89046/225000 [163:48:30<147:54:46,  3.92s/it]
 40%|███▉      | 89047/225000 [163:48:33<137:16:21,  3.63s/it]
 40%|███▉      | 89048/225000 [163:48:36<134:29:56,  3.56s/it]
 40%|███▉      | 89049/225000 [163:48:39<127:34:10,  3.38s/it]
 40%|███▉      | 89050/225000 [163:48:43<132:54:45,  3.52s/it]
                                                              
{'loss': '0.3584', 'grad_norm': '4.234', 'learning_rate': '3.511e-05', 'epoch': '0.3958', 'num_input_tokens_seen': 268539304, 'train_runtime': '5.897e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 89050/225000 [163:48:43<132:54:45,  3.52s/it]
 40%|███▉      | 89051/225000 [163:48:51<188:51:03,  5.00s/it]
 40%|███▉      | 89052/225000 [163:48:54<164:16:44,  4.35s/it]
 40%|███▉      | 89053/225000 [163:48:57<143:31:30,  3.80s/it]
 40%|███▉      | 89054/225000 [163:49:00<140:34:07,  3.72s/it]
 40%|███▉      | 89055/225000 [163:49:07<180:19:07,  4.78s/it]
 40%|███▉      | 89056/225000 [163:49:11<165:16:42,  4.38s/it]
 40%|███▉      | 89057/225000 [163:49:16<177:20:43,  4.70s/it]
 40%|███▉      | 89058/225000 [163:49:20<162:12:18,  4.30s/it]
 40%|███▉      | 89059/225000 [163:49:22<143:05:52,  3.79s/it]
 40%|███▉      | 89060/225000 [163:49:25<131:55:23,  3.49s/it]
 40%|███▉      | 89061/225000 [163:49:28<126:40:58,  3.35s/it]
 40%|███▉      | 89062/225000 [163:49:32<127:05:21,  3.37s/it]
 40%|███▉      | 89063/225000 [163:49:35<131:27:25,  3.48s/it]
 40%|███▉      | 89064/225000 [163:49:44<193:15:06,  5.12s/it]
 40%|███▉      | 89065/225000 [163:49:47<163:32:48,  4.33s/it]
 40%|███▉      | 89066/225000 [163:49:49<142:15:01,  3.77s/it]
 40%|███▉      | 89067/225000 [163:49:52<132:12:14,  3.50s/it]
 40%|███▉      | 89068/225000 [163:49:55<126:22:45,  3.35s/it]
 40%|███▉      | 89069/225000 [163:49:58<122:12:53,  3.24s/it]
 40%|███▉      | 89070/225000 [163:50:01<122:25:45,  3.24s/it]
 40%|███▉      | 89071/225000 [163:50:05<128:05:18,  3.39s/it]
 40%|███▉      | 89072/225000 [163:50:08<123:17:34,  3.27s/it]
 40%|███▉      | 89073/225000 [163:50:11<123:58:55,  3.28s/it]
 40%|███▉      | 89074/225000 [163:50:15<123:36:13,  3.27s/it]
 40%|███▉      | 89075/225000 [163:50:18<126:52:16,  3.36s/it]
 40%|███▉      | 89076/225000 [163:50:23<139:56:49,  3.71s/it]
 40%|███▉      | 89077/225000 [163:50:32<200:30:47,  5.31s/it]
 40%|███▉      | 89078/225000 [163:50:36<183:52:49,  4.87s/it]
 40%|███▉      | 89079/225000 [163:50:39<170:50:44,  4.53s/it]
 40%|███▉      | 89080/225000 [163:50:45<189:32:42,  5.02s/it]
 40%|███▉      | 89081/225000 [163:50:48<164:46:31,  4.36s/it]
 40%|███▉      | 89082/225000 [163:50:51<147:42:25,  3.91s/it]
 40%|███▉      | 89083/225000 [163:50:55<143:58:39,  3.81s/it]
 40%|███▉      | 89084/225000 [163:50:58<137:27:08,  3.64s/it]
 40%|███▉      | 89085/225000 [163:51:03<158:40:13,  4.20s/it]
 40%|███▉      | 89086/225000 [163:51:11<193:34:14,  5.13s/it]
 40%|███▉      | 89087/225000 [163:51:13<164:25:45,  4.36s/it]
 40%|███▉      | 89088/225000 [163:51:16<144:36:40,  3.83s/it]
 40%|███▉      | 89089/225000 [163:51:22<173:06:35,  4.59s/it]
 40%|███▉      | 89090/225000 [163:51:25<153:53:47,  4.08s/it]
 40%|███▉      | 89091/225000 [163:51:28<143:16:35,  3.80s/it]
 40%|███▉      | 89092/225000 [163:51:32<137:36:22,  3.64s/it]
 40%|███▉      | 89093/225000 [163:51:35<129:40:58,  3.44s/it]
 40%|███▉      | 89094/225000 [163:51:38<125:09:46,  3.32s/it]
 40%|███▉      | 89095/225000 [163:51:41<123:52:23,  3.28s/it]
 40%|███▉      | 89096/225000 [163:51:44<122:43:40,  3.25s/it]
 40%|███▉      | 89097/225000 [163:51:47<119:51:43,  3.18s/it]
 40%|███▉      | 89098/225000 [163:51:50<117:39:26,  3.12s/it]
 40%|███▉      | 89099/225000 [163:51:55<140:08:13,  3.71s/it]
 40%|███▉      | 89100/225000 [163:52:04<199:55:27,  5.30s/it]
                                                              
{'loss': '0.3141', 'grad_norm': '6.557', 'learning_rate': '3.509e-05', 'epoch': '0.396', 'num_input_tokens_seen': 268626240, 'train_runtime': '5.899e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 89100/225000 [163:52:04<199:55:27,  5.30s/it]
 40%|███▉      | 89101/225000 [163:52:08<184:53:44,  4.90s/it]
 40%|███▉      | 89102/225000 [163:52:10<157:27:48,  4.17s/it]
 40%|███▉      | 89103/225000 [163:52:13<140:24:10,  3.72s/it]
 40%|███▉      | 89104/225000 [163:52:16<126:22:37,  3.35s/it]
 40%|███▉      | 89105/225000 [163:52:22<166:14:57,  4.40s/it]
 40%|███▉      | 89106/225000 [163:52:28<173:40:41,  4.60s/it]
 40%|███▉      | 89107/225000 [163:52:31<155:26:26,  4.12s/it]
 40%|███▉      | 89108/225000 [163:52:33<138:19:37,  3.66s/it]
 40%|███▉      | 89109/225000 [163:52:36<133:13:23,  3.53s/it]
 40%|███▉      | 89110/225000 [163:52:39<125:07:11,  3.31s/it]
 40%|███▉      | 89111/225000 [163:52:42<117:19:41,  3.11s/it]
 40%|███▉      | 89112/225000 [163:52:50<174:48:14,  4.63s/it]
 40%|███▉      | 89113/225000 [163:52:56<188:47:24,  5.00s/it]
 40%|███▉      | 89114/225000 [163:52:59<162:47:32,  4.31s/it]
 40%|███▉      | 89115/225000 [163:53:01<145:02:21,  3.84s/it]
 40%|███▉      | 89116/225000 [163:53:04<135:02:29,  3.58s/it]
 40%|███▉      | 89117/225000 [163:53:08<135:22:10,  3.59s/it]
 40%|███▉      | 89118/225000 [163:53:11<130:01:44,  3.44s/it]
 40%|███▉      | 89119/225000 [163:53:14<126:25:27,  3.35s/it]
 40%|███▉      | 89120/225000 [163:53:17<123:39:59,  3.28s/it]
 40%|███▉      | 89121/225000 [163:53:21<130:18:53,  3.45s/it]
 40%|███▉      | 89122/225000 [163:53:26<151:10:59,  4.01s/it]
 40%|███▉      | 89123/225000 [163:53:35<202:26:29,  5.36s/it]
 40%|███▉      | 89124/225000 [163:53:40<195:59:36,  5.19s/it]
 40%|███▉      | 89125/225000 [163:53:44<185:29:47,  4.91s/it]
 40%|███▉      | 89126/225000 [163:53:47<164:44:40,  4.36s/it]
 40%|███▉      | 89127/225000 [163:53:51<158:59:10,  4.21s/it]
 40%|███▉      | 89128/225000 [163:53:54<149:01:41,  3.95s/it]
 40%|███▉      | 89129/225000 [163:53:58<150:35:18,  3.99s/it]
 40%|███▉      | 89130/225000 [163:54:02<146:55:28,  3.89s/it]
 40%|███▉      | 89131/225000 [163:54:05<139:15:08,  3.69s/it]
 40%|███▉      | 89132/225000 [163:54:09<140:35:15,  3.73s/it]
 40%|███▉      | 89133/225000 [163:54:12<134:41:48,  3.57s/it]
 40%|███▉      | 89134/225000 [163:54:16<135:23:22,  3.59s/it]
 40%|███▉      | 89135/225000 [163:54:19<134:45:10,  3.57s/it]
 40%|███▉      | 89136/225000 [163:54:26<167:09:54,  4.43s/it]
 40%|███▉      | 89137/225000 [163:54:35<219:27:30,  5.82s/it]
 40%|███▉      | 89138/225000 [163:54:39<202:55:59,  5.38s/it]
 40%|███▉      | 89139/225000 [163:54:42<173:23:08,  4.59s/it]
 40%|███▉      | 89140/225000 [163:54:45<150:47:52,  4.00s/it]
 40%|███▉      | 89141/225000 [163:54:47<134:27:45,  3.56s/it]
 40%|███▉      | 89142/225000 [163:54:50<123:48:11,  3.28s/it]
 40%|███▉      | 89143/225000 [163:54:53<124:14:35,  3.29s/it]
 40%|███▉      | 89144/225000 [163:55:03<197:04:02,  5.22s/it]
 40%|███▉      | 89145/225000 [163:55:06<171:13:25,  4.54s/it]
 40%|███▉      | 89146/225000 [163:55:08<149:24:59,  3.96s/it]
 40%|███▉      | 89147/225000 [163:55:11<132:47:12,  3.52s/it]
 40%|███▉      | 89148/225000 [163:55:14<124:10:03,  3.29s/it]
 40%|███▉      | 89149/225000 [163:55:22<184:51:04,  4.90s/it]
 40%|███▉      | 89150/225000 [163:55:26<175:53:50,  4.66s/it]
                                                              
{'loss': '0.3239', 'grad_norm': '7.395', 'learning_rate': '3.508e-05', 'epoch': '0.3962', 'num_input_tokens_seen': 268714392, 'train_runtime': '5.901e+05', 'train_tokens_per_second': '455.4'}

 40%|███▉      | 89150/225000 [163:55:26<175:53:50,  4.66s/it]
 40%|███▉      | 89151/225000 [163:55:29<158:09:14,  4.19s/it]
 40%|███▉      | 89152/225000 [163:55:33<149:25:04,  3.96s/it]
 40%|███▉      | 89153/225000 [163:55:36<144:06:07,  3.82s/it]
 40%|███▉      | 89154/225000 [163:55:40<136:54:36,  3.63s/it]
 40%|███▉      | 89155/225000 [163:55:43<138:43:31,  3.68s/it]
 40%|███▉      | 89156/225000 [163:55:47<137:43:14,  3.65s/it]
 40%|███▉      | 89157/225000 [163:55:50<132:17:47,  3.51s/it]
 40%|███▉      | 89158/225000 [163:55:54<131:54:56,  3.50s/it]
 40%|███▉      | 89159/225000 [163:55:57<133:21:07,  3.53s/it]
 40%|███▉      | 89160/225000 [163:56:00<129:44:01,  3.44s/it]
 40%|███▉      | 89161/225000 [163:56:04<126:37:47,  3.36s/it]
 40%|███▉      | 89162/225000 [163:56:07<128:36:50,  3.41s/it]
 40%|███▉      | 89163/225000 [163:56:10<126:07:07,  3.34s/it]
 40%|███▉      | 89164/225000 [163:56:14<131:11:54,  3.48s/it]
 40%|███▉      | 89165/225000 [163:56:22<185:24:59,  4.91s/it]
 40%|███▉      | 89166/225000 [163:56:30<217:11:54,  5.76s/it]
 40%|███▉      | 89167/225000 [163:56:33<188:58:05,  5.01s/it]
 40%|███▉      | 89168/225000 [163:56:36<167:27:20,  4.44s/it]
 40%|███▉      | 89169/225000 [163:56:40<156:36:52,  4.15s/it]
 40%|███▉      | 89170/225000 [163:56:43<147:21:46,  3.91s/it]
 40%|███▉      | 89171/225000 [163:56:47<141:03:19,  3.74s/it]
 40%|███▉      | 89172/225000 [163:56:50<133:07:02,  3.53s/it]
 40%|███▉      | 89173/225000 [163:56:53<126:38:40,  3.36s/it]
 40%|███▉      | 89174/225000 [163:56:56<128:57:41,  3.42s/it]
 40%|███▉      | 89175/225000 [163:57:00<130:25:27,  3.46s/it]
 40%|███▉      | 89176/225000 [163:57:06<158:06:33,  4.19s/it]
 40%|███▉      | 89177/225000 [163:57:16<228:43:49,  6.06s/it]
 40%|███▉      | 89178/225000 [163:57:19<189:46:21,  5.03s/it]
 40%|███▉      | 89179/225000 [163:57:21<160:25:24,  4.25s/it]
 40%|███▉      | 89180/225000 [163:57:27<184:06:14,  4.88s/it]
 40%|███▉      | 89181/225000 [163:57:30<159:32:34,  4.23s/it]
 40%|███▉      | 89182/225000 [163:57:33<140:53:15,  3.73s/it]
 40%|███▉      | 89183/225000 [163:57:41<192:12:49,  5.09s/it]
 40%|███▉      | 89184/225000 [163:57:43<162:06:48,  4.30s/it]
 40%|███▉      | 89185/225000 [163:57:46<143:16:01,  3.80s/it]
 40%|███▉      | 89186/225000 [163:57:49<129:15:48,  3.43s/it]
 40%|███▉      | 89187/225000 [163:57:53<141:34:31,  3.75s/it]
 40%|███▉      | 89188/225000 [163:58:01<187:32:04,  4.97s/it]
 40%|███▉      | 89189/225000 [163:58:04<165:42:40,  4.39s/it]
 40%|███▉      | 89190/225000 [163:58:07<146:11:39,  3.88s/it]
 40%|███▉      | 89191/225000 [163:58:10<135:22:12,  3.59s/it]
 40%|███▉      | 89192/225000 [163:58:12<126:59:18,  3.37s/it]
 40%|███▉      | 89193/225000 [163:58:20<179:20:00,  4.75s/it]
 40%|███▉      | 89194/225000 [163:58:24<162:09:18,  4.30s/it]
 40%|███▉      | 89195/225000 [163:58:27<147:04:32,  3.90s/it]
 40%|███▉      | 89196/225000 [163:58:30<136:41:52,  3.62s/it]
 40%|███▉      | 89197/225000 [163:58:32<127:30:15,  3.38s/it]
 40%|███▉      | 89198/225000 [163:58:36<127:01:00,  3.37s/it]
 40%|███▉      | 89199/225000 [163:58:39<122:06:37,  3.24s/it]
 40%|███▉      | 89200/225000 [163:58:42<124:08:24,  3.29s/it]
                                                              
{'loss': '0.3447', 'grad_norm': '5.24', 'learning_rate': '3.506e-05', 'epoch': '0.3964', 'num_input_tokens_seen': 268801984, 'train_runtime': '5.903e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89200/225000 [163:58:42<124:08:24,  3.29s/it]
 40%|███▉      | 89201/225000 [163:58:48<155:21:40,  4.12s/it]
 40%|███▉      | 89202/225000 [163:58:56<199:29:54,  5.29s/it]
 40%|███▉      | 89203/225000 [163:58:59<168:26:45,  4.47s/it]
 40%|███▉      | 89204/225000 [163:59:02<153:45:57,  4.08s/it]
 40%|███▉      | 89205/225000 [163:59:05<141:25:30,  3.75s/it]
 40%|███▉      | 89206/225000 [163:59:08<132:01:04,  3.50s/it]
 40%|███▉      | 89207/225000 [163:59:11<125:30:33,  3.33s/it]
 40%|███▉      | 89208/225000 [163:59:15<134:46:28,  3.57s/it]
 40%|███▉      | 89209/225000 [163:59:18<131:53:22,  3.50s/it]
 40%|███▉      | 89210/225000 [163:59:27<191:38:19,  5.08s/it]
 40%|███▉      | 89211/225000 [163:59:33<197:38:53,  5.24s/it]
 40%|███▉      | 89212/225000 [163:59:36<173:37:36,  4.60s/it]
 40%|███▉      | 89213/225000 [163:59:39<155:27:34,  4.12s/it]
 40%|███▉      | 89214/225000 [163:59:42<146:20:54,  3.88s/it]
 40%|███▉      | 89215/225000 [163:59:45<137:31:55,  3.65s/it]
 40%|███▉      | 89216/225000 [163:59:51<158:36:55,  4.21s/it]
 40%|███▉      | 89217/225000 [163:59:58<189:14:32,  5.02s/it]
 40%|███▉      | 89218/225000 [164:00:01<168:37:51,  4.47s/it]
 40%|███▉      | 89219/225000 [164:00:04<154:50:01,  4.11s/it]
 40%|███▉      | 89220/225000 [164:00:07<141:53:50,  3.76s/it]
 40%|███▉      | 89221/225000 [164:00:10<133:07:40,  3.53s/it]
 40%|███▉      | 89222/225000 [164:00:13<132:42:52,  3.52s/it]
 40%|███▉      | 89223/225000 [164:00:16<127:07:13,  3.37s/it]
 40%|███▉      | 89224/225000 [164:00:20<125:13:06,  3.32s/it]
 40%|███▉      | 89225/225000 [164:00:23<126:20:38,  3.35s/it]
 40%|███▉      | 89226/225000 [164:00:26<124:59:21,  3.31s/it]
 40%|███▉      | 89227/225000 [164:00:30<128:03:51,  3.40s/it]
 40%|███▉      | 89228/225000 [164:00:37<175:33:02,  4.65s/it]
 40%|███▉      | 89229/225000 [164:00:45<209:22:46,  5.55s/it]
 40%|███▉      | 89230/225000 [164:00:48<174:50:18,  4.64s/it]
 40%|███▉      | 89231/225000 [164:00:50<150:15:06,  3.98s/it]
 40%|███▉      | 89232/225000 [164:00:53<134:14:23,  3.56s/it]
 40%|███▉      | 89233/225000 [164:00:56<127:42:37,  3.39s/it]
 40%|███▉      | 89234/225000 [164:00:59<127:37:35,  3.38s/it]
 40%|███▉      | 89235/225000 [164:01:02<123:16:10,  3.27s/it]
 40%|███▉      | 89236/225000 [164:01:10<180:17:01,  4.78s/it]
 40%|███▉      | 89237/225000 [164:01:17<203:05:15,  5.39s/it]
 40%|███▉      | 89238/225000 [164:01:20<171:12:06,  4.54s/it]
 40%|███▉      | 89239/225000 [164:01:22<147:43:52,  3.92s/it]
 40%|███▉      | 89240/225000 [164:01:25<137:01:09,  3.63s/it]
 40%|███▉      | 89241/225000 [164:01:28<127:00:37,  3.37s/it]
 40%|███▉      | 89242/225000 [164:01:33<152:19:37,  4.04s/it]
 40%|███▉      | 89243/225000 [164:01:40<180:07:42,  4.78s/it]
 40%|███▉      | 89244/225000 [164:01:43<155:49:25,  4.13s/it]
 40%|███▉      | 89245/225000 [164:01:49<184:02:16,  4.88s/it]
 40%|███▉      | 89246/225000 [164:01:53<166:11:20,  4.41s/it]
 40%|███▉      | 89247/225000 [164:01:55<142:37:20,  3.78s/it]
 40%|███▉      | 89248/225000 [164:02:00<153:31:26,  4.07s/it]
 40%|███▉      | 89249/225000 [164:02:05<171:51:22,  4.56s/it]
 40%|███▉      | 89250/225000 [164:02:08<149:44:08,  3.97s/it]
                                                              
{'loss': '0.3372', 'grad_norm': '3.744', 'learning_rate': '3.504e-05', 'epoch': '0.3967', 'num_input_tokens_seen': 268890248, 'train_runtime': '5.905e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89250/225000 [164:02:08<149:44:08,  3.97s/it]
 40%|███▉      | 89251/225000 [164:02:11<137:42:03,  3.65s/it]
 40%|███▉      | 89252/225000 [164:02:18<183:08:35,  4.86s/it]
 40%|███▉      | 89253/225000 [164:02:22<168:23:46,  4.47s/it]
 40%|███▉      | 89254/225000 [164:02:25<147:48:51,  3.92s/it]
 40%|███▉      | 89255/225000 [164:02:28<137:11:56,  3.64s/it]
 40%|███▉      | 89256/225000 [164:02:31<129:45:12,  3.44s/it]
 40%|███▉      | 89257/225000 [164:02:33<122:34:58,  3.25s/it]
 40%|███▉      | 89258/225000 [164:02:39<152:53:04,  4.05s/it]
 40%|███▉      | 89259/225000 [164:02:46<184:22:46,  4.89s/it]
 40%|███▉      | 89260/225000 [164:02:49<156:30:33,  4.15s/it]
 40%|███▉      | 89261/225000 [164:02:51<140:46:01,  3.73s/it]
 40%|███▉      | 89262/225000 [164:02:58<174:26:20,  4.63s/it]
 40%|███▉      | 89263/225000 [164:03:02<169:47:03,  4.50s/it]
 40%|███▉      | 89264/225000 [164:03:05<148:26:45,  3.94s/it]
 40%|███▉      | 89265/225000 [164:03:07<132:08:30,  3.50s/it]
 40%|███▉      | 89266/225000 [164:03:10<126:31:43,  3.36s/it]
 40%|███▉      | 89267/225000 [164:03:17<162:02:44,  4.30s/it]
 40%|███▉      | 89268/225000 [164:03:24<189:49:38,  5.03s/it]
 40%|███▉      | 89269/225000 [164:03:28<178:44:17,  4.74s/it]
 40%|███▉      | 89270/225000 [164:03:33<179:15:09,  4.75s/it]
 40%|███▉      | 89271/225000 [164:03:35<154:10:10,  4.09s/it]
 40%|███▉      | 89272/225000 [164:03:38<136:27:54,  3.62s/it]
 40%|███▉      | 89273/225000 [164:03:40<123:42:50,  3.28s/it]
 40%|███▉      | 89274/225000 [164:03:43<116:42:38,  3.10s/it]
 40%|███▉      | 89275/225000 [164:03:46<114:52:36,  3.05s/it]
 40%|███▉      | 89276/225000 [164:03:53<166:29:13,  4.42s/it]
 40%|███▉      | 89277/225000 [164:03:57<161:01:46,  4.27s/it]
 40%|███▉      | 89278/225000 [164:04:00<142:00:54,  3.77s/it]
 40%|███▉      | 89279/225000 [164:04:02<128:24:29,  3.41s/it]
 40%|███▉      | 89280/225000 [164:04:09<162:13:11,  4.30s/it]
 40%|███▉      | 89281/225000 [164:04:13<166:49:24,  4.43s/it]
 40%|███▉      | 89282/225000 [164:04:16<146:17:00,  3.88s/it]
 40%|███▉      | 89283/225000 [164:04:19<136:55:04,  3.63s/it]
 40%|███▉      | 89284/225000 [164:04:27<184:26:21,  4.89s/it]
 40%|███▉      | 89285/225000 [164:04:30<158:03:11,  4.19s/it]
 40%|███▉      | 89286/225000 [164:04:35<173:27:46,  4.60s/it]
 40%|███▉      | 89287/225000 [164:04:38<157:37:41,  4.18s/it]
 40%|███▉      | 89288/225000 [164:04:41<145:02:05,  3.85s/it]
 40%|███▉      | 89289/225000 [164:04:44<135:02:04,  3.58s/it]
 40%|███▉      | 89290/225000 [164:04:48<130:38:26,  3.47s/it]
 40%|███▉      | 89291/225000 [164:04:52<140:36:11,  3.73s/it]
 40%|███▉      | 89292/225000 [164:04:59<179:50:24,  4.77s/it]
 40%|███▉      | 89293/225000 [164:05:02<164:04:17,  4.35s/it]
 40%|███▉      | 89294/225000 [164:05:06<150:37:50,  4.00s/it]
 40%|███▉      | 89295/225000 [164:05:09<147:06:29,  3.90s/it]
 40%|███▉      | 89296/225000 [164:05:12<135:53:46,  3.61s/it]
 40%|███▉      | 89297/225000 [164:05:15<131:03:03,  3.48s/it]
 40%|███▉      | 89298/225000 [164:05:19<127:14:27,  3.38s/it]
 40%|███▉      | 89299/225000 [164:05:26<173:31:36,  4.60s/it]
 40%|███▉      | 89300/225000 [164:05:32<194:02:59,  5.15s/it]
                                                              
{'loss': '0.3199', 'grad_norm': '4.099', 'learning_rate': '3.503e-05', 'epoch': '0.3969', 'num_input_tokens_seen': 268978336, 'train_runtime': '5.907e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89300/225000 [164:05:32<194:02:59,  5.15s/it]
 40%|███▉      | 89301/225000 [164:05:35<165:30:05,  4.39s/it]
 40%|███▉      | 89302/225000 [164:05:38<148:09:56,  3.93s/it]
 40%|███▉      | 89303/225000 [164:05:41<136:14:17,  3.61s/it]
 40%|███▉      | 89304/225000 [164:05:47<165:20:09,  4.39s/it]
 40%|███▉      | 89305/225000 [164:05:52<175:32:09,  4.66s/it]
 40%|███▉      | 89306/225000 [164:05:55<152:49:29,  4.05s/it]
 40%|███▉      | 89307/225000 [164:05:58<137:45:04,  3.65s/it]
 40%|███▉      | 89308/225000 [164:06:01<129:47:45,  3.44s/it]
 40%|███▉      | 89309/225000 [164:06:04<124:16:51,  3.30s/it]
 40%|███▉      | 89310/225000 [164:06:12<178:54:37,  4.75s/it]
 40%|███▉      | 89311/225000 [164:06:17<184:15:09,  4.89s/it]
 40%|███▉      | 89312/225000 [164:06:19<156:43:15,  4.16s/it]
 40%|███▉      | 89313/225000 [164:06:22<140:54:02,  3.74s/it]
 40%|███▉      | 89314/225000 [164:06:25<133:28:47,  3.54s/it]
 40%|███▉      | 89315/225000 [164:06:32<173:51:27,  4.61s/it]
 40%|███▉      | 89316/225000 [164:06:35<149:40:51,  3.97s/it]
 40%|███▉      | 89317/225000 [164:06:39<147:32:55,  3.91s/it]
 40%|███▉      | 89318/225000 [164:06:46<183:38:19,  4.87s/it]
 40%|███▉      | 89319/225000 [164:06:48<155:15:09,  4.12s/it]
 40%|███▉      | 89320/225000 [164:06:51<139:02:19,  3.69s/it]
 40%|███▉      | 89321/225000 [164:06:56<152:32:00,  4.05s/it]
 40%|███▉      | 89322/225000 [164:07:01<167:41:45,  4.45s/it]
 40%|███▉      | 89323/225000 [164:07:03<143:34:27,  3.81s/it]
 40%|███▉      | 89324/225000 [164:07:06<130:40:47,  3.47s/it]
 40%|███▉      | 89325/225000 [164:07:09<127:59:59,  3.40s/it]
 40%|███▉      | 89326/225000 [164:07:12<126:40:03,  3.36s/it]
 40%|███▉      | 89327/225000 [164:07:16<125:23:28,  3.33s/it]
 40%|███▉      | 89328/225000 [164:07:19<124:35:25,  3.31s/it]
 40%|███▉      | 89329/225000 [164:07:23<127:42:52,  3.39s/it]
 40%|███▉      | 89330/225000 [164:07:32<191:38:42,  5.09s/it]
 40%|███▉      | 89331/225000 [164:07:39<215:29:51,  5.72s/it]
 40%|███▉      | 89332/225000 [164:07:42<190:00:30,  5.04s/it]
 40%|███▉      | 89333/225000 [164:07:45<166:20:04,  4.41s/it]
 40%|███▉      | 89334/225000 [164:07:48<153:13:24,  4.07s/it]
 40%|███▉      | 89335/225000 [164:07:51<141:18:42,  3.75s/it]
 40%|███▉      | 89336/225000 [164:07:55<134:08:50,  3.56s/it]
 40%|███▉      | 89337/225000 [164:07:58<127:16:34,  3.38s/it]
 40%|███▉      | 89338/225000 [164:08:01<124:37:17,  3.31s/it]
 40%|███▉      | 89339/225000 [164:08:04<119:55:55,  3.18s/it]
 40%|███▉      | 89340/225000 [164:08:07<126:08:44,  3.35s/it]
 40%|███▉      | 89341/225000 [164:08:10<121:01:59,  3.21s/it]
 40%|███▉      | 89342/225000 [164:08:17<163:33:17,  4.34s/it]
 40%|███▉      | 89343/225000 [164:08:24<193:36:05,  5.14s/it]
 40%|███▉      | 89344/225000 [164:08:27<163:11:20,  4.33s/it]
 40%|███▉      | 89345/225000 [164:08:33<183:20:18,  4.87s/it]
 40%|███▉      | 89346/225000 [164:08:35<157:37:59,  4.18s/it]
 40%|███▉      | 89347/225000 [164:08:40<167:19:59,  4.44s/it]
 40%|███▉      | 89348/225000 [164:08:43<152:31:41,  4.05s/it]
 40%|███▉      | 89349/225000 [164:08:47<145:47:44,  3.87s/it]
 40%|███▉      | 89350/225000 [164:08:50<141:10:17,  3.75s/it]
                                                              
{'loss': '0.322', 'grad_norm': '5.967', 'learning_rate': '3.501e-05', 'epoch': '0.3971', 'num_input_tokens_seen': 269065496, 'train_runtime': '5.909e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89350/225000 [164:08:50<141:10:17,  3.75s/it]
 40%|███▉      | 89351/225000 [164:08:54<141:07:57,  3.75s/it]
 40%|███▉      | 89352/225000 [164:09:00<162:18:42,  4.31s/it]
 40%|███▉      | 89353/225000 [164:09:05<172:53:24,  4.59s/it]
 40%|███▉      | 89354/225000 [164:09:08<149:43:52,  3.97s/it]
 40%|███▉      | 89355/225000 [164:09:14<173:04:11,  4.59s/it]
 40%|███▉      | 89356/225000 [164:09:19<177:23:18,  4.71s/it]
 40%|███▉      | 89357/225000 [164:09:21<156:10:12,  4.14s/it]
 40%|███▉      | 89358/225000 [164:09:24<140:57:54,  3.74s/it]
 40%|███▉      | 89359/225000 [164:09:27<127:45:19,  3.39s/it]
 40%|███▉      | 89360/225000 [164:09:33<161:49:48,  4.30s/it]
 40%|███▉      | 89361/225000 [164:09:37<162:19:42,  4.31s/it]
 40%|███▉      | 89362/225000 [164:09:40<140:56:19,  3.74s/it]
 40%|███▉      | 89363/225000 [164:09:48<187:19:47,  4.97s/it]
 40%|███▉      | 89364/225000 [164:09:51<162:28:06,  4.31s/it]
 40%|███▉      | 89365/225000 [164:09:53<147:17:37,  3.91s/it]
 40%|███▉      | 89366/225000 [164:09:56<136:41:45,  3.63s/it]
 40%|███▉      | 89367/225000 [164:09:59<127:48:32,  3.39s/it]
 40%|███▉      | 89368/225000 [164:10:03<134:38:54,  3.57s/it]
 40%|███▉      | 89369/225000 [164:10:09<161:21:00,  4.28s/it]
 40%|███▉      | 89370/225000 [164:10:12<147:48:27,  3.92s/it]
 40%|███▉      | 89371/225000 [164:10:15<139:03:06,  3.69s/it]
 40%|███▉      | 89372/225000 [164:10:19<135:53:04,  3.61s/it]
 40%|███▉      | 89373/225000 [164:10:22<133:37:54,  3.55s/it]
 40%|███▉      | 89374/225000 [164:10:25<128:20:11,  3.41s/it]
 40%|███▉      | 89375/225000 [164:10:30<147:40:22,  3.92s/it]
 40%|███▉      | 89376/225000 [164:10:34<144:47:24,  3.84s/it]
 40%|███▉      | 89377/225000 [164:10:38<140:10:13,  3.72s/it]
 40%|███▉      | 89378/225000 [164:10:41<135:46:46,  3.60s/it]
 40%|███▉      | 89379/225000 [164:10:44<133:33:02,  3.55s/it]
 40%|███▉      | 89380/225000 [164:10:48<138:44:14,  3.68s/it]
 40%|███▉      | 89381/225000 [164:10:52<138:06:44,  3.67s/it]
 40%|███▉      | 89382/225000 [164:10:55<133:39:26,  3.55s/it]
 40%|███▉      | 89383/225000 [164:11:04<188:46:59,  5.01s/it]
 40%|███▉      | 89384/225000 [164:11:11<213:21:14,  5.66s/it]
 40%|███▉      | 89385/225000 [164:11:13<176:10:34,  4.68s/it]
 40%|███▉      | 89386/225000 [164:11:16<152:59:57,  4.06s/it]
 40%|███▉      | 89387/225000 [164:11:23<189:01:29,  5.02s/it]
 40%|███▉      | 89388/225000 [164:11:26<161:23:25,  4.28s/it]
 40%|███▉      | 89389/225000 [164:11:28<142:55:50,  3.79s/it]
 40%|███▉      | 89390/225000 [164:11:31<132:42:01,  3.52s/it]
 40%|███▉      | 89391/225000 [164:11:34<124:10:21,  3.30s/it]
 40%|███▉      | 89392/225000 [164:11:40<152:19:40,  4.04s/it]
 40%|███▉      | 89393/225000 [164:11:48<203:33:18,  5.40s/it]
 40%|███▉      | 89394/225000 [164:11:51<175:34:31,  4.66s/it]
 40%|███▉      | 89395/225000 [164:11:54<153:58:09,  4.09s/it]
 40%|███▉      | 89396/225000 [164:11:57<145:47:31,  3.87s/it]
 40%|███▉      | 89397/225000 [164:12:01<138:10:46,  3.67s/it]
 40%|███▉      | 89398/225000 [164:12:04<135:30:40,  3.60s/it]
 40%|███▉      | 89399/225000 [164:12:07<125:43:48,  3.34s/it]
 40%|███▉      | 89400/225000 [164:12:10<127:54:03,  3.40s/it]
                                                              
{'loss': '0.3393', 'grad_norm': '5.293', 'learning_rate': '3.499e-05', 'epoch': '0.3973', 'num_input_tokens_seen': 269152736, 'train_runtime': '5.911e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89400/225000 [164:12:10<127:54:03,  3.40s/it]
 40%|███▉      | 89401/225000 [164:12:20<195:00:10,  5.18s/it]
 40%|███▉      | 89402/225000 [164:12:24<190:35:14,  5.06s/it]
 40%|███▉      | 89403/225000 [164:12:28<171:43:27,  4.56s/it]
 40%|███▉      | 89404/225000 [164:12:31<156:17:11,  4.15s/it]
 40%|███▉      | 89405/225000 [164:12:34<145:49:03,  3.87s/it]
 40%|███▉      | 89406/225000 [164:12:37<137:54:56,  3.66s/it]
 40%|███▉      | 89407/225000 [164:12:41<135:27:39,  3.60s/it]
 40%|███▉      | 89408/225000 [164:12:44<134:29:31,  3.57s/it]
 40%|███▉      | 89409/225000 [164:12:48<134:19:23,  3.57s/it]
 40%|███▉      | 89410/225000 [164:12:51<134:48:29,  3.58s/it]
 40%|███▉      | 89411/225000 [164:12:55<132:06:17,  3.51s/it]
 40%|███▉      | 89412/225000 [164:12:58<128:52:04,  3.42s/it]
 40%|███▉      | 89413/225000 [164:13:06<176:41:56,  4.69s/it]
 40%|███▉      | 89414/225000 [164:13:13<202:24:36,  5.37s/it]
 40%|███▉      | 89415/225000 [164:13:16<173:40:41,  4.61s/it]
 40%|███▉      | 89416/225000 [164:13:19<156:55:44,  4.17s/it]
 40%|███▉      | 89417/225000 [164:13:22<145:35:01,  3.87s/it]
 40%|███▉      | 89418/225000 [164:13:25<140:55:44,  3.74s/it]
 40%|███▉      | 89419/225000 [164:13:28<131:41:36,  3.50s/it]
 40%|███▉      | 89420/225000 [164:13:32<136:36:39,  3.63s/it]
 40%|███▉      | 89421/225000 [164:13:35<128:44:12,  3.42s/it]
 40%|███▉      | 89422/225000 [164:13:42<165:37:05,  4.40s/it]
 40%|███▉      | 89423/225000 [164:13:50<210:14:32,  5.58s/it]
 40%|███▉      | 89424/225000 [164:13:52<173:00:06,  4.59s/it]
 40%|███▉      | 89425/225000 [164:13:55<146:52:44,  3.90s/it]
 40%|███▉      | 89426/225000 [164:13:59<154:45:34,  4.11s/it]
 40%|███▉      | 89427/225000 [164:14:03<149:39:28,  3.97s/it]
 40%|███▉      | 89428/225000 [164:14:06<135:08:59,  3.59s/it]
 40%|███▉      | 89429/225000 [164:14:08<126:26:05,  3.36s/it]
 40%|███▉      | 89430/225000 [164:14:12<123:56:00,  3.29s/it]
 40%|███▉      | 89431/225000 [164:14:15<122:38:33,  3.26s/it]
 40%|███▉      | 89432/225000 [164:14:18<124:41:38,  3.31s/it]
 40%|███▉      | 89433/225000 [164:14:22<127:19:22,  3.38s/it]
 40%|███▉      | 89434/225000 [164:14:28<161:13:56,  4.28s/it]
 40%|███▉      | 89435/225000 [164:14:33<163:46:45,  4.35s/it]
 40%|███▉      | 89436/225000 [164:14:36<148:10:29,  3.93s/it]
 40%|███▉      | 89437/225000 [164:14:39<146:48:54,  3.90s/it]
 40%|███▉      | 89438/225000 [164:14:42<137:50:13,  3.66s/it]
 40%|███▉      | 89439/225000 [164:14:46<133:03:38,  3.53s/it]
 40%|███▉      | 89440/225000 [164:14:49<127:11:26,  3.38s/it]
 40%|███▉      | 89441/225000 [164:14:52<126:46:43,  3.37s/it]
 40%|███▉      | 89442/225000 [164:14:56<128:57:27,  3.42s/it]
 40%|███▉      | 89443/225000 [164:15:00<134:42:56,  3.58s/it]
 40%|███▉      | 89444/225000 [164:15:04<139:16:31,  3.70s/it]
 40%|███▉      | 89445/225000 [164:15:08<152:30:03,  4.05s/it]
 40%|███▉      | 89446/225000 [164:15:18<215:08:34,  5.71s/it]
 40%|███▉      | 89447/225000 [164:15:24<218:06:03,  5.79s/it]
 40%|███▉      | 89448/225000 [164:15:27<184:02:11,  4.89s/it]
 40%|███▉      | 89449/225000 [164:15:30<166:26:11,  4.42s/it]
 40%|███▉      | 89450/225000 [164:15:33<148:12:48,  3.94s/it]
                                                              
{'loss': '0.3335', 'grad_norm': '7.641', 'learning_rate': '3.498e-05', 'epoch': '0.3976', 'num_input_tokens_seen': 269240896, 'train_runtime': '5.913e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89450/225000 [164:15:33<148:12:48,  3.94s/it]
 40%|███▉      | 89451/225000 [164:15:42<201:07:24,  5.34s/it]
 40%|███▉      | 89452/225000 [164:15:44<167:45:18,  4.46s/it]
 40%|███▉      | 89453/225000 [164:15:50<183:35:44,  4.88s/it]
 40%|███▉      | 89454/225000 [164:15:52<155:31:43,  4.13s/it]
 40%|███▉      | 89455/225000 [164:15:55<136:18:27,  3.62s/it]
 40%|███▉      | 89456/225000 [164:16:00<152:52:20,  4.06s/it]
 40%|███▉      | 89457/225000 [164:16:05<170:46:05,  4.54s/it]
 40%|███▉      | 89458/225000 [164:16:09<158:36:19,  4.21s/it]
 40%|███▉      | 89459/225000 [164:16:13<158:50:36,  4.22s/it]
 40%|███▉      | 89460/225000 [164:16:18<171:47:57,  4.56s/it]
 40%|███▉      | 89461/225000 [164:16:23<170:19:14,  4.52s/it]
 40%|███▉      | 89462/225000 [164:16:25<146:17:43,  3.89s/it]
 40%|███▉      | 89463/225000 [164:16:28<130:38:58,  3.47s/it]
 40%|███▉      | 89464/225000 [164:16:32<142:38:13,  3.79s/it]
 40%|███▉      | 89465/225000 [164:16:36<138:10:00,  3.67s/it]
 40%|███▉      | 89466/225000 [164:16:40<148:35:24,  3.95s/it]
 40%|███▉      | 89467/225000 [164:16:44<144:15:04,  3.83s/it]
 40%|███▉      | 89468/225000 [164:16:47<137:55:19,  3.66s/it]
 40%|███▉      | 89469/225000 [164:16:50<128:37:50,  3.42s/it]
 40%|███▉      | 89470/225000 [164:16:53<125:11:16,  3.33s/it]
 40%|███▉      | 89471/225000 [164:16:56<122:00:34,  3.24s/it]
 40%|███▉      | 89472/225000 [164:17:01<144:46:11,  3.85s/it]
 40%|███▉      | 89473/225000 [164:17:11<205:25:58,  5.46s/it]
 40%|███▉      | 89474/225000 [164:17:13<171:16:18,  4.55s/it]
 40%|███▉      | 89475/225000 [164:17:16<151:38:11,  4.03s/it]
 40%|███▉      | 89476/225000 [164:17:19<141:51:27,  3.77s/it]
 40%|███▉      | 89477/225000 [164:17:22<136:12:06,  3.62s/it]
 40%|███▉      | 89478/225000 [164:17:26<134:44:54,  3.58s/it]
 40%|███▉      | 89479/225000 [164:17:29<129:27:55,  3.44s/it]
 40%|███▉      | 89480/225000 [164:17:32<121:49:27,  3.24s/it]
 40%|███▉      | 89481/225000 [164:17:38<156:44:31,  4.16s/it]
 40%|███▉      | 89482/225000 [164:17:46<206:34:12,  5.49s/it]
 40%|███▉      | 89483/225000 [164:17:51<190:15:40,  5.05s/it]
 40%|███▉      | 89484/225000 [164:17:53<160:14:09,  4.26s/it]
 40%|███▉      | 89485/225000 [164:17:56<144:10:22,  3.83s/it]
 40%|███▉      | 89486/225000 [164:18:01<158:13:12,  4.20s/it]
 40%|███▉      | 89487/225000 [164:18:08<191:25:05,  5.09s/it]
 40%|███▉      | 89488/225000 [164:18:10<161:02:10,  4.28s/it]
 40%|███▉      | 89489/225000 [164:18:14<150:49:32,  4.01s/it]
 40%|███▉      | 89490/225000 [164:18:18<148:45:29,  3.95s/it]
 40%|███▉      | 89491/225000 [164:18:21<140:06:29,  3.72s/it]
 40%|███▉      | 89492/225000 [164:18:24<138:29:21,  3.68s/it]
 40%|███▉      | 89493/225000 [164:18:27<132:55:49,  3.53s/it]
 40%|███▉      | 89494/225000 [164:18:31<132:08:04,  3.51s/it]
 40%|███▉      | 89495/225000 [164:18:37<156:49:57,  4.17s/it]
 40%|███▉      | 89496/225000 [164:18:40<147:04:45,  3.91s/it]
 40%|███▉      | 89497/225000 [164:18:43<138:35:50,  3.68s/it]
 40%|███▉      | 89498/225000 [164:18:46<133:25:04,  3.54s/it]
 40%|███▉      | 89499/225000 [164:18:50<136:32:05,  3.63s/it]
 40%|███▉      | 89500/225000 [164:18:54<137:31:25,  3.65s/it]
                                                              
{'loss': '0.3411', 'grad_norm': '6.81', 'learning_rate': '3.496e-05', 'epoch': '0.3978', 'num_input_tokens_seen': 269327880, 'train_runtime': '5.915e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89500/225000 [164:18:54<137:31:25,  3.65s/it]
 40%|███▉      | 89501/225000 [164:18:57<133:37:58,  3.55s/it]
 40%|███▉      | 89502/225000 [164:19:01<133:28:31,  3.55s/it]
 40%|███▉      | 89503/225000 [164:19:04<134:24:17,  3.57s/it]
 40%|███▉      | 89504/225000 [164:19:09<146:49:36,  3.90s/it]
 40%|███▉      | 89505/225000 [164:19:19<211:42:37,  5.62s/it]
 40%|███▉      | 89506/225000 [164:19:23<194:51:37,  5.18s/it]
 40%|███▉      | 89507/225000 [164:19:26<170:28:39,  4.53s/it]
 40%|███▉      | 89508/225000 [164:19:28<148:02:23,  3.93s/it]
 40%|███▉      | 89509/225000 [164:19:35<182:00:13,  4.84s/it]
 40%|███▉      | 89510/225000 [164:19:39<163:44:44,  4.35s/it]
 40%|███▉      | 89511/225000 [164:19:42<154:01:26,  4.09s/it]
 40%|███▉      | 89512/225000 [164:19:45<147:03:10,  3.91s/it]
 40%|███▉      | 89513/225000 [164:19:49<138:18:06,  3.67s/it]
 40%|███▉      | 89514/225000 [164:19:52<132:18:54,  3.52s/it]
 40%|███▉      | 89515/225000 [164:19:55<128:21:58,  3.41s/it]
 40%|███▉      | 89516/225000 [164:19:58<128:51:11,  3.42s/it]
 40%|███▉      | 89517/225000 [164:20:02<131:51:24,  3.50s/it]
 40%|███▉      | 89518/225000 [164:20:06<133:06:25,  3.54s/it]
 40%|███▉      | 89519/225000 [164:20:10<140:18:43,  3.73s/it]
 40%|███▉      | 89520/225000 [164:20:13<134:15:09,  3.57s/it]
 40%|███▉      | 89521/225000 [164:20:17<138:04:27,  3.67s/it]
 40%|███▉      | 89522/225000 [164:20:21<139:28:00,  3.71s/it]
 40%|███▉      | 89523/225000 [164:20:24<136:49:26,  3.64s/it]
 40%|███▉      | 89524/225000 [164:20:28<136:25:30,  3.63s/it]
 40%|███▉      | 89525/225000 [164:20:32<141:43:19,  3.77s/it]
 40%|███▉      | 89526/225000 [164:20:40<195:17:19,  5.19s/it]
 40%|███▉      | 89527/225000 [164:20:47<216:04:05,  5.74s/it]
 40%|███▉      | 89528/225000 [164:20:50<182:00:03,  4.84s/it]
 40%|███▉      | 89529/225000 [164:20:53<156:36:30,  4.16s/it]
 40%|███▉      | 89530/225000 [164:20:55<138:39:21,  3.68s/it]
 40%|███▉      | 89531/225000 [164:21:03<185:24:09,  4.93s/it]
 40%|███▉      | 89532/225000 [164:21:08<183:15:08,  4.87s/it]
 40%|███▉      | 89533/225000 [164:21:11<159:02:39,  4.23s/it]
 40%|███▉      | 89534/225000 [164:21:14<151:50:47,  4.04s/it]
 40%|███▉      | 89535/225000 [164:21:20<173:25:25,  4.61s/it]
 40%|███▉      | 89536/225000 [164:21:23<152:32:01,  4.05s/it]
 40%|███▉      | 89537/225000 [164:21:26<137:33:22,  3.66s/it]
 40%|███▉      | 89538/225000 [164:21:28<126:43:46,  3.37s/it]
 40%|███▉      | 89539/225000 [164:21:31<121:10:49,  3.22s/it]
 40%|███▉      | 89540/225000 [164:21:34<118:27:23,  3.15s/it]
 40%|███▉      | 89541/225000 [164:21:37<115:19:57,  3.07s/it]
 40%|███▉      | 89542/225000 [164:21:43<144:51:56,  3.85s/it]
 40%|███▉      | 89543/225000 [164:21:52<203:15:01,  5.40s/it]
 40%|███▉      | 89544/225000 [164:21:55<178:59:05,  4.76s/it]
 40%|███▉      | 89545/225000 [164:21:58<155:38:16,  4.14s/it]
 40%|███▉      | 89546/225000 [164:22:05<189:11:13,  5.03s/it]
 40%|███▉      | 89547/225000 [164:22:07<162:27:24,  4.32s/it]
 40%|███▉      | 89548/225000 [164:22:10<145:38:56,  3.87s/it]
 40%|███▉      | 89549/225000 [164:22:14<142:07:00,  3.78s/it]
 40%|███▉      | 89550/225000 [164:22:21<185:09:32,  4.92s/it]
                                                              
{'loss': '0.3461', 'grad_norm': '3.693', 'learning_rate': '3.494e-05', 'epoch': '0.398', 'num_input_tokens_seen': 269415136, 'train_runtime': '5.917e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89550/225000 [164:22:21<185:09:32,  4.92s/it]
 40%|███▉      | 89551/225000 [164:22:24<161:57:14,  4.30s/it]
 40%|███▉      | 89552/225000 [164:22:28<149:43:11,  3.98s/it]
 40%|███▉      | 89553/225000 [164:22:30<134:06:35,  3.56s/it]
 40%|███▉      | 89554/225000 [164:22:33<125:44:35,  3.34s/it]
 40%|███▉      | 89555/225000 [164:22:36<125:24:07,  3.33s/it]
 40%|███▉      | 89556/225000 [164:22:45<183:13:45,  4.87s/it]
 40%|███▉      | 89557/225000 [164:22:49<176:44:58,  4.70s/it]
 40%|███▉      | 89558/225000 [164:22:52<153:20:39,  4.08s/it]
 40%|███▉      | 89559/225000 [164:22:54<138:47:12,  3.69s/it]
 40%|███▉      | 89560/225000 [164:23:02<184:26:33,  4.90s/it]
 40%|███▉      | 89561/225000 [164:23:05<158:10:56,  4.20s/it]
 40%|███▉      | 89562/225000 [164:23:07<141:12:23,  3.75s/it]
 40%|███▉      | 89563/225000 [164:23:11<135:10:21,  3.59s/it]
 40%|███▉      | 89564/225000 [164:23:15<138:51:06,  3.69s/it]
 40%|███▉      | 89565/225000 [164:23:18<135:24:35,  3.60s/it]
 40%|███▉      | 89566/225000 [164:23:21<129:12:35,  3.43s/it]
 40%|███▉      | 89567/225000 [164:23:24<123:27:50,  3.28s/it]
 40%|███▉      | 89568/225000 [164:23:29<138:16:34,  3.68s/it]
 40%|███▉      | 89569/225000 [164:23:38<198:49:04,  5.28s/it]
 40%|███▉      | 89570/225000 [164:23:43<204:19:35,  5.43s/it]
 40%|███▉      | 89571/225000 [164:23:46<174:34:53,  4.64s/it]
 40%|███▉      | 89572/225000 [164:23:49<152:52:51,  4.06s/it]
 40%|███▉      | 89573/225000 [164:23:52<142:39:33,  3.79s/it]
 40%|███▉      | 89574/225000 [164:23:55<132:28:34,  3.52s/it]
 40%|███▉      | 89575/225000 [164:23:58<125:59:26,  3.35s/it]
 40%|███▉      | 89576/225000 [164:24:02<131:59:47,  3.51s/it]
 40%|███▉      | 89577/225000 [164:24:11<198:36:53,  5.28s/it]
 40%|███▉      | 89578/225000 [164:24:14<166:49:54,  4.43s/it]
 40%|███▉      | 89579/225000 [164:24:16<148:57:06,  3.96s/it]
 40%|███▉      | 89580/225000 [164:24:19<134:32:20,  3.58s/it]
 40%|███▉      | 89581/225000 [164:24:24<154:12:38,  4.10s/it]
 40%|███▉      | 89582/225000 [164:24:31<185:31:58,  4.93s/it]
 40%|███▉      | 89583/225000 [164:24:34<157:24:54,  4.18s/it]
 40%|███▉      | 89584/225000 [164:24:37<151:09:29,  4.02s/it]
 40%|███▉      | 89585/225000 [164:24:42<156:59:54,  4.17s/it]
 40%|███▉      | 89586/225000 [164:24:46<151:42:11,  4.03s/it]
 40%|███▉      | 89587/225000 [164:24:49<148:31:20,  3.95s/it]
 40%|███▉      | 89588/225000 [164:24:53<144:34:33,  3.84s/it]
 40%|███▉      | 89589/225000 [164:24:56<136:47:30,  3.64s/it]
 40%|███▉      | 89590/225000 [164:24:59<130:53:12,  3.48s/it]
 40%|███▉      | 89591/225000 [164:25:03<131:28:16,  3.50s/it]
 40%|███▉      | 89592/225000 [164:25:06<133:09:14,  3.54s/it]
 40%|███▉      | 89593/225000 [164:25:10<129:25:39,  3.44s/it]
 40%|███▉      | 89594/225000 [164:25:13<129:46:04,  3.45s/it]
 40%|███▉      | 89595/225000 [164:25:17<131:55:16,  3.51s/it]
 40%|███▉      | 89596/225000 [164:25:20<131:25:31,  3.49s/it]
 40%|███▉      | 89597/225000 [164:25:24<129:10:14,  3.43s/it]
 40%|███▉      | 89598/225000 [164:25:27<132:01:25,  3.51s/it]
 40%|███▉      | 89599/225000 [164:25:32<141:31:50,  3.76s/it]
 40%|███▉      | 89600/225000 [164:25:41<201:13:31,  5.35s/it]
                                                              
{'loss': '0.3364', 'grad_norm': '5.018', 'learning_rate': '3.493e-05', 'epoch': '0.3982', 'num_input_tokens_seen': 269501296, 'train_runtime': '5.919e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89600/225000 [164:25:41<201:13:31,  5.35s/it]
 40%|███▉      | 89601/225000 [164:25:44<175:23:16,  4.66s/it]
 40%|███▉      | 89602/225000 [164:25:47<158:44:32,  4.22s/it]
 40%|███▉      | 89603/225000 [164:25:50<146:04:43,  3.88s/it]
 40%|███▉      | 89604/225000 [164:25:53<134:21:42,  3.57s/it]
 40%|███▉      | 89605/225000 [164:25:59<163:10:48,  4.34s/it]
 40%|███▉      | 89606/225000 [164:26:07<205:08:12,  5.45s/it]
 40%|███▉      | 89607/225000 [164:26:10<173:03:41,  4.60s/it]
 40%|███▉      | 89608/225000 [164:26:12<150:46:04,  4.01s/it]
 40%|███▉      | 89609/225000 [164:26:15<135:07:05,  3.59s/it]
 40%|███▉      | 89610/225000 [164:26:20<154:11:42,  4.10s/it]
 40%|███▉      | 89611/225000 [164:26:26<171:28:26,  4.56s/it]
 40%|███▉      | 89612/225000 [164:26:29<151:22:35,  4.03s/it]
 40%|███▉      | 89613/225000 [164:26:32<140:33:55,  3.74s/it]
 40%|███▉      | 89614/225000 [164:26:34<129:17:44,  3.44s/it]
 40%|███▉      | 89615/225000 [164:26:38<129:55:19,  3.45s/it]
 40%|███▉      | 89616/225000 [164:26:41<127:25:59,  3.39s/it]
 40%|███▉      | 89617/225000 [164:26:46<148:28:28,  3.95s/it]
 40%|███▉      | 89618/225000 [164:26:53<182:51:39,  4.86s/it]
 40%|███▉      | 89619/225000 [164:26:56<156:47:57,  4.17s/it]
 40%|███▉      | 89620/225000 [164:26:59<144:25:26,  3.84s/it]
 40%|███▉      | 89621/225000 [164:27:02<131:02:29,  3.48s/it]
 40%|███▉      | 89622/225000 [164:27:09<178:11:57,  4.74s/it]
 40%|███▉      | 89623/225000 [164:27:14<181:47:04,  4.83s/it]
 40%|███▉      | 89624/225000 [164:27:18<170:47:13,  4.54s/it]
 40%|███▉      | 89625/225000 [164:27:22<160:21:24,  4.26s/it]
 40%|███▉      | 89626/225000 [164:27:25<150:07:43,  3.99s/it]
 40%|███▉      | 89627/225000 [164:27:28<139:52:58,  3.72s/it]
 40%|███▉      | 89628/225000 [164:27:32<137:45:31,  3.66s/it]
 40%|███▉      | 89629/225000 [164:27:35<135:00:22,  3.59s/it]
 40%|███▉      | 89630/225000 [164:27:38<129:40:38,  3.45s/it]
 40%|███▉      | 89631/225000 [164:27:42<128:49:17,  3.43s/it]
 40%|███▉      | 89632/225000 [164:27:45<125:34:39,  3.34s/it]
 40%|███▉      | 89633/225000 [164:27:48<124:57:19,  3.32s/it]
 40%|███▉      | 89634/225000 [164:27:51<124:54:03,  3.32s/it]
 40%|███▉      | 89635/225000 [164:27:55<124:45:39,  3.32s/it]
 40%|███▉      | 89636/225000 [164:27:58<128:40:44,  3.42s/it]
 40%|███▉      | 89637/225000 [164:28:04<151:07:44,  4.02s/it]
 40%|███▉      | 89638/225000 [164:28:13<208:35:55,  5.55s/it]
 40%|███▉      | 89639/225000 [164:28:18<197:54:25,  5.26s/it]
 40%|███▉      | 89640/225000 [164:28:20<171:32:17,  4.56s/it]
 40%|███▉      | 89641/225000 [164:28:24<156:03:52,  4.15s/it]
 40%|███▉      | 89642/225000 [164:28:32<206:19:26,  5.49s/it]
 40%|███▉      | 89643/225000 [164:28:35<180:00:43,  4.79s/it]
 40%|███▉      | 89644/225000 [164:28:38<154:12:56,  4.10s/it]
 40%|███▉      | 89645/225000 [164:28:41<141:20:58,  3.76s/it]
 40%|███▉      | 89646/225000 [164:28:45<141:46:09,  3.77s/it]
 40%|███▉      | 89647/225000 [164:28:49<143:12:16,  3.81s/it]
 40%|███▉      | 89648/225000 [164:28:52<138:29:47,  3.68s/it]
 40%|███▉      | 89649/225000 [164:28:56<143:49:51,  3.83s/it]
 40%|███▉      | 89650/225000 [164:29:00<142:36:15,  3.79s/it]
                                                              
{'loss': '0.3291', 'grad_norm': '6.202', 'learning_rate': '3.491e-05', 'epoch': '0.3984', 'num_input_tokens_seen': 269588888, 'train_runtime': '5.921e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89650/225000 [164:29:00<142:36:15,  3.79s/it]
 40%|███▉      | 89651/225000 [164:29:03<140:22:08,  3.73s/it]
 40%|███▉      | 89652/225000 [164:29:08<155:00:59,  4.12s/it]
 40%|███▉      | 89653/225000 [164:29:18<218:07:40,  5.80s/it]
 40%|███▉      | 89654/225000 [164:29:21<184:58:01,  4.92s/it]
 40%|███▉      | 89655/225000 [164:29:24<160:20:50,  4.27s/it]
 40%|███▉      | 89656/225000 [164:29:27<145:17:33,  3.86s/it]
 40%|███▉      | 89657/225000 [164:29:30<137:49:36,  3.67s/it]
 40%|███▉      | 89658/225000 [164:29:33<127:23:55,  3.39s/it]
 40%|███▉      | 89659/225000 [164:29:42<193:00:07,  5.13s/it]
 40%|███▉      | 89660/225000 [164:29:46<184:01:11,  4.89s/it]
 40%|███▉      | 89661/225000 [164:29:49<155:54:56,  4.15s/it]
 40%|███▉      | 89662/225000 [164:29:52<148:38:33,  3.95s/it]
 40%|███▉      | 89663/225000 [164:29:55<141:01:42,  3.75s/it]
 40%|███▉      | 89664/225000 [164:29:59<134:33:02,  3.58s/it]
 40%|███▉      | 89665/225000 [164:30:02<128:05:23,  3.41s/it]
 40%|███▉      | 89666/225000 [164:30:05<126:33:35,  3.37s/it]
 40%|███▉      | 89667/225000 [164:30:08<128:32:00,  3.42s/it]
 40%|███▉      | 89668/225000 [164:30:12<133:00:03,  3.54s/it]
 40%|███▉      | 89669/225000 [164:30:16<132:33:55,  3.53s/it]
 40%|███▉      | 89670/225000 [164:30:23<172:52:11,  4.60s/it]
 40%|███▉      | 89671/225000 [164:30:32<229:35:13,  6.11s/it]
 40%|███▉      | 89672/225000 [164:30:37<207:57:52,  5.53s/it]
 40%|███▉      | 89673/225000 [164:30:40<178:21:58,  4.74s/it]
 40%|███▉      | 89674/225000 [164:30:42<155:14:35,  4.13s/it]
 40%|███▉      | 89675/225000 [164:30:45<143:50:39,  3.83s/it]
 40%|███▉      | 89676/225000 [164:30:48<133:03:27,  3.54s/it]
 40%|███▉      | 89677/225000 [164:30:51<126:25:35,  3.36s/it]
 40%|███▉      | 89678/225000 [164:30:54<119:31:43,  3.18s/it]
 40%|███▉      | 89679/225000 [164:31:01<160:52:26,  4.28s/it]
 40%|███▉      | 89680/225000 [164:31:09<200:29:50,  5.33s/it]
 40%|███▉      | 89681/225000 [164:31:11<169:27:51,  4.51s/it]
 40%|███▉      | 89682/225000 [164:31:14<146:30:12,  3.90s/it]
 40%|███▉      | 89683/225000 [164:31:17<136:33:32,  3.63s/it]
 40%|███▉      | 89684/225000 [164:31:25<192:57:22,  5.13s/it]
 40%|███▉      | 89685/225000 [164:31:29<175:13:07,  4.66s/it]
 40%|███▉      | 89686/225000 [164:31:32<162:41:33,  4.33s/it]
 40%|███▉      | 89687/225000 [164:31:35<145:00:28,  3.86s/it]
 40%|███▉      | 89688/225000 [164:31:38<135:34:19,  3.61s/it]
 40%|███▉      | 89689/225000 [164:31:41<127:38:35,  3.40s/it]
 40%|███▉      | 89690/225000 [164:31:44<125:17:35,  3.33s/it]
 40%|███▉      | 89691/225000 [164:31:47<119:06:51,  3.17s/it]
 40%|███▉      | 89692/225000 [164:31:56<181:46:18,  4.84s/it]
 40%|███▉      | 89693/225000 [164:32:00<179:04:49,  4.76s/it]
 40%|███▉      | 89694/225000 [164:32:03<153:22:47,  4.08s/it]
 40%|███▉      | 89695/225000 [164:32:06<139:36:36,  3.71s/it]
 40%|███▉      | 89696/225000 [164:32:11<156:57:06,  4.18s/it]
 40%|███▉      | 89697/225000 [164:32:15<150:46:44,  4.01s/it]
 40%|███▉      | 89698/225000 [164:32:18<142:21:03,  3.79s/it]
 40%|███▉      | 89699/225000 [164:32:21<139:05:25,  3.70s/it]
 40%|███▉      | 89700/225000 [164:32:25<141:12:09,  3.76s/it]
                                                              
{'loss': '0.3222', 'grad_norm': '7.29', 'learning_rate': '3.489e-05', 'epoch': '0.3987', 'num_input_tokens_seen': 269677088, 'train_runtime': '5.923e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89700/225000 [164:32:25<141:12:09,  3.76s/it]
 40%|███▉      | 89701/225000 [164:32:29<143:01:24,  3.81s/it]
 40%|███▉      | 89702/225000 [164:32:36<178:53:25,  4.76s/it]
 40%|███▉      | 89703/225000 [164:32:40<163:13:31,  4.34s/it]
 40%|███▉      | 89704/225000 [164:32:45<181:32:28,  4.83s/it]
 40%|███▉      | 89705/225000 [164:32:48<158:10:02,  4.21s/it]
 40%|███▉      | 89706/225000 [164:32:51<144:36:52,  3.85s/it]
 40%|███▉      | 89707/225000 [164:32:54<134:33:00,  3.58s/it]
 40%|███▉      | 89708/225000 [164:32:57<128:18:19,  3.41s/it]
 40%|███▉      | 89709/225000 [164:33:00<120:58:01,  3.22s/it]
 40%|███▉      | 89710/225000 [164:33:05<136:14:47,  3.63s/it]
 40%|███▉      | 89711/225000 [164:33:13<191:26:53,  5.09s/it]
 40%|███▉      | 89712/225000 [164:33:17<174:57:27,  4.66s/it]
 40%|███▉      | 89713/225000 [164:33:20<159:21:30,  4.24s/it]
 40%|███▉      | 89714/225000 [164:33:23<144:13:33,  3.84s/it]
 40%|███▉      | 89715/225000 [164:33:26<132:13:44,  3.52s/it]
 40%|███▉      | 89716/225000 [164:33:32<168:25:41,  4.48s/it]
 40%|███▉      | 89717/225000 [164:33:36<162:40:59,  4.33s/it]
 40%|███▉      | 89718/225000 [164:33:40<150:23:08,  4.00s/it]
 40%|███▉      | 89719/225000 [164:33:43<142:51:34,  3.80s/it]
 40%|███▉      | 89720/225000 [164:33:47<142:48:14,  3.80s/it]
 40%|███▉      | 89721/225000 [164:33:50<137:02:19,  3.65s/it]
 40%|███▉      | 89722/225000 [164:33:53<132:50:54,  3.54s/it]
 40%|███▉      | 89723/225000 [164:33:57<131:35:00,  3.50s/it]
 40%|███▉      | 89724/225000 [164:34:00<126:54:10,  3.38s/it]
 40%|███▉      | 89725/225000 [164:34:05<142:16:22,  3.79s/it]
 40%|███▉      | 89726/225000 [164:34:14<201:37:36,  5.37s/it]
 40%|███▉      | 89727/225000 [164:34:19<197:00:44,  5.24s/it]
 40%|███▉      | 89728/225000 [164:34:22<175:36:02,  4.67s/it]
 40%|███▉      | 89729/225000 [164:34:25<152:10:04,  4.05s/it]
 40%|███▉      | 89730/225000 [164:34:27<139:22:57,  3.71s/it]
 40%|███▉      | 89731/225000 [164:34:32<154:10:27,  4.10s/it]
 40%|███▉      | 89732/225000 [164:34:39<179:39:53,  4.78s/it]
 40%|███▉      | 89733/225000 [164:34:43<168:04:42,  4.47s/it]
 40%|███▉      | 89734/225000 [164:34:45<148:56:59,  3.96s/it]
 40%|███▉      | 89735/225000 [164:34:49<141:33:17,  3.77s/it]
 40%|███▉      | 89736/225000 [164:34:52<134:59:48,  3.59s/it]
 40%|███▉      | 89737/225000 [164:34:56<142:56:54,  3.80s/it]
 40%|███▉      | 89738/225000 [164:35:05<194:27:23,  5.18s/it]
 40%|███▉      | 89739/225000 [164:35:07<165:12:22,  4.40s/it]
 40%|███▉      | 89740/225000 [164:35:10<147:34:44,  3.93s/it]
 40%|███▉      | 89741/225000 [164:35:18<196:16:18,  5.22s/it]
 40%|███▉      | 89742/225000 [164:35:21<166:49:37,  4.44s/it]
 40%|███▉      | 89743/225000 [164:35:23<146:23:25,  3.90s/it]
 40%|███▉      | 89744/225000 [164:35:27<137:57:53,  3.67s/it]
 40%|███▉      | 89745/225000 [164:35:30<129:54:04,  3.46s/it]
 40%|███▉      | 89746/225000 [164:35:32<121:27:40,  3.23s/it]
 40%|███▉      | 89747/225000 [164:35:35<120:38:44,  3.21s/it]
 40%|███▉      | 89748/225000 [164:35:44<176:40:02,  4.70s/it]
 40%|███▉      | 89749/225000 [164:35:48<174:52:04,  4.65s/it]
 40%|███▉      | 89750/225000 [164:35:51<153:00:19,  4.07s/it]
                                                              
{'loss': '0.3393', 'grad_norm': '5.837', 'learning_rate': '3.488e-05', 'epoch': '0.3989', 'num_input_tokens_seen': 269764976, 'train_runtime': '5.926e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89750/225000 [164:35:51<153:00:19,  4.07s/it]
 40%|███▉      | 89751/225000 [164:35:56<160:52:58,  4.28s/it]
 40%|███▉      | 89752/225000 [164:36:00<165:24:16,  4.40s/it]
 40%|███▉      | 89753/225000 [164:36:04<153:52:42,  4.10s/it]
 40%|███▉      | 89754/225000 [164:36:08<153:18:45,  4.08s/it]
 40%|███▉      | 89755/225000 [164:36:11<146:06:16,  3.89s/it]
 40%|███▉      | 89756/225000 [164:36:17<165:59:16,  4.42s/it]
 40%|███▉      | 89757/225000 [164:36:21<158:38:17,  4.22s/it]
 40%|███▉      | 89758/225000 [164:36:25<158:02:42,  4.21s/it]
 40%|███▉      | 89759/225000 [164:36:29<159:49:04,  4.25s/it]
 40%|███▉      | 89760/225000 [164:36:32<142:06:34,  3.78s/it]
 40%|███▉      | 89761/225000 [164:36:35<133:04:27,  3.54s/it]
 40%|███▉      | 89762/225000 [164:36:38<129:28:14,  3.45s/it]
 40%|███▉      | 89763/225000 [164:36:41<122:32:35,  3.26s/it]
 40%|███▉      | 89764/225000 [164:36:47<150:24:57,  4.00s/it]
 40%|███▉      | 89765/225000 [164:36:55<200:07:45,  5.33s/it]
 40%|███▉      | 89766/225000 [164:36:57<167:03:13,  4.45s/it]
 40%|███▉      | 89767/225000 [164:37:00<143:11:53,  3.81s/it]
 40%|███▉      | 89768/225000 [164:37:08<191:47:12,  5.11s/it]
 40%|███▉      | 89769/225000 [164:37:10<162:28:08,  4.33s/it]
 40%|███▉      | 89770/225000 [164:37:13<144:47:54,  3.85s/it]
 40%|███▉      | 89771/225000 [164:37:16<132:52:46,  3.54s/it]
 40%|███▉      | 89772/225000 [164:37:18<121:56:17,  3.25s/it]
 40%|███▉      | 89773/225000 [164:37:26<170:29:46,  4.54s/it]
 40%|███▉      | 89774/225000 [164:37:30<168:38:50,  4.49s/it]
 40%|███▉      | 89775/225000 [164:37:33<145:09:11,  3.86s/it]
 40%|███▉      | 89776/225000 [164:37:36<133:48:43,  3.56s/it]
 40%|███▉      | 89777/225000 [164:37:39<130:37:53,  3.48s/it]
 40%|███▉      | 89778/225000 [164:37:42<125:36:08,  3.34s/it]
 40%|███▉      | 89779/225000 [164:37:50<179:44:53,  4.79s/it]
 40%|███▉      | 89780/225000 [164:37:53<153:40:15,  4.09s/it]
 40%|███▉      | 89781/225000 [164:37:55<136:27:38,  3.63s/it]
 40%|███▉      | 89782/225000 [164:37:58<125:54:26,  3.35s/it]
 40%|███▉      | 89783/225000 [164:38:01<119:39:36,  3.19s/it]
 40%|███▉      | 89784/225000 [164:38:06<149:12:24,  3.97s/it]
 40%|███▉      | 89785/225000 [164:38:14<184:32:26,  4.91s/it]
 40%|███▉      | 89786/225000 [164:38:16<159:17:57,  4.24s/it]
 40%|███▉      | 89787/225000 [164:38:19<139:19:35,  3.71s/it]
 40%|███▉      | 89788/225000 [164:38:22<134:50:43,  3.59s/it]
 40%|███▉      | 89789/225000 [164:38:25<133:27:37,  3.55s/it]
 40%|███▉      | 89790/225000 [164:38:28<125:01:30,  3.33s/it]
 40%|███▉      | 89791/225000 [164:38:34<153:15:50,  4.08s/it]
 40%|███▉      | 89792/225000 [164:38:43<208:43:27,  5.56s/it]
 40%|███▉      | 89793/225000 [164:38:46<175:31:45,  4.67s/it]
 40%|███▉      | 89794/225000 [164:38:49<154:22:38,  4.11s/it]
 40%|███▉      | 89795/225000 [164:38:51<137:16:36,  3.66s/it]
 40%|███▉      | 89796/225000 [164:38:54<132:56:20,  3.54s/it]
 40%|███▉      | 89797/225000 [164:39:02<173:53:11,  4.63s/it]
 40%|███▉      | 89798/225000 [164:39:06<172:11:38,  4.58s/it]
 40%|███▉      | 89799/225000 [164:39:09<153:25:44,  4.09s/it]
 40%|███▉      | 89800/225000 [164:39:12<143:03:40,  3.81s/it]
                                                              
{'loss': '0.3324', 'grad_norm': '6.048', 'learning_rate': '3.486e-05', 'epoch': '0.3991', 'num_input_tokens_seen': 269852712, 'train_runtime': '5.928e+05', 'train_tokens_per_second': '455.3'}

 40%|███▉      | 89800/225000 [164:39:12<143:03:40,  3.81s/it]
 40%|███▉      | 89801/225000 [164:39:19<182:14:18,  4.85s/it]
 40%|███▉      | 89802/225000 [164:39:22<155:51:59,  4.15s/it]
 40%|███▉      | 89803/225000 [164:39:27<169:17:22,  4.51s/it]
 40%|███▉      | 89804/225000 [164:39:30<150:23:53,  4.00s/it]
 40%|███▉      | 89805/225000 [164:39:33<133:25:04,  3.55s/it]
 40%|███▉      | 89806/225000 [164:39:36<131:54:28,  3.51s/it]
 40%|███▉      | 89807/225000 [164:39:39<130:16:11,  3.47s/it]
 40%|███▉      | 89808/225000 [164:39:46<162:10:30,  4.32s/it]
 40%|███▉      | 89809/225000 [164:39:49<147:27:08,  3.93s/it]
 40%|███▉      | 89810/225000 [164:39:52<142:39:16,  3.80s/it]
 40%|███▉      | 89811/225000 [164:39:55<134:12:07,  3.57s/it]
 40%|███▉      | 89812/225000 [164:39:59<135:18:06,  3.60s/it]
 40%|███▉      | 89813/225000 [164:40:07<190:32:38,  5.07s/it]
 40%|███▉      | 89814/225000 [164:40:11<173:23:49,  4.62s/it]
 40%|███▉      | 89815/225000 [164:40:13<149:19:54,  3.98s/it]
 40%|███▉      | 89816/225000 [164:40:16<135:20:52,  3.60s/it]
 40%|███▉      | 89817/225000 [164:40:24<179:57:00,  4.79s/it]
 40%|███▉      | 89818/225000 [164:40:29<180:42:24,  4.81s/it]
 40%|███▉      | 89819/225000 [164:40:31<158:24:21,  4.22s/it]
 40%|███▉      | 89820/225000 [164:40:34<142:52:10,  3.80s/it]
 40%|███▉      | 89821/225000 [164:40:38<136:35:35,  3.64s/it]
 40%|███▉      | 89822/225000 [164:40:40<127:37:38,  3.40s/it]
 40%|███▉      | 89823/225000 [164:40:43<122:57:09,  3.27s/it]
 40%|███▉      | 89824/225000 [164:40:46<120:12:56,  3.20s/it]
 40%|███▉      | 89825/225000 [164:40:50<121:34:42,  3.24s/it]
 40%|███▉      | 89826/225000 [164:40:56<154:15:42,  4.11s/it]
 40%|███▉      | 89827/225000 [164:41:04<203:03:05,  5.41s/it]
 40%|███▉      | 89828/225000 [164:41:07<173:08:51,  4.61s/it]
 40%|███▉      | 89829/225000 [164:41:09<148:16:02,  3.95s/it]
 40%|███▉      | 89830/225000 [164:41:12<137:32:20,  3.66s/it]
 40%|███▉      | 89831/225000 [164:41:16<136:30:38,  3.64s/it]
 40%|███▉      | 89832/225000 [164:41:19<134:36:32,  3.59s/it]
 40%|███▉      | 89833/225000 [164:41:23<135:10:52,  3.60s/it]
 40%|███▉      | 89834/225000 [164:41:26<131:55:39,  3.51s/it]
 40%|███▉      | 89835/225000 [164:41:30<132:09:22,  3.52s/it]
 40%|███▉      | 89836/225000 [164:41:34<137:17:49,  3.66s/it]
 40%|███▉      | 89837/225000 [164:41:38<136:25:27,  3.63s/it]
 40%|███▉      | 89838/225000 [164:41:41<133:53:33,  3.57s/it]
 40%|███▉      | 89839/225000 [164:41:45<137:53:38,  3.67s/it]
 40%|███▉      | 89840/225000 [164:41:48<135:02:39,  3.60s/it]
 40%|███▉      | 89841/225000 [164:41:52<134:50:59,  3.59s/it]
 40%|███▉      | 89842/225000 [164:41:56<137:50:13,  3.67s/it]
 40%|███▉      | 89843/225000 [164:42:00<148:22:03,  3.95s/it]
 40%|███▉      | 89844/225000 [164:42:06<171:51:45,  4.58s/it]
 40%|███▉      | 89845/225000 [164:42:10<164:21:51,  4.38s/it]
 40%|███▉      | 89846/225000 [164:42:14<156:51:17,  4.18s/it]
 40%|███▉      | 89847/225000 [164:42:23<216:14:25,  5.76s/it]
 40%|███▉      | 89848/225000 [164:42:28<202:15:24,  5.39s/it]
 40%|███▉      | 89849/225000 [164:42:30<170:03:34,  4.53s/it]
 40%|███▉      | 89850/225000 [164:42:33<146:06:09,  3.89s/it]
                                                              
{'loss': '0.3125', 'grad_norm': '5.813', 'learning_rate': '3.484e-05', 'epoch': '0.3993', 'num_input_tokens_seen': 269939848, 'train_runtime': '5.93e+05', 'train_tokens_per_second': '455.2'}

 40%|███▉      | 89850/225000 [164:42:33<146:06:09,  3.89s/it]
 40%|███▉      | 89851/225000 [164:42:35<130:53:44,  3.49s/it]
 40%|███▉      | 89852/225000 [164:42:39<131:49:28,  3.51s/it]
 40%|███▉      | 89853/225000 [164:42:42<131:33:17,  3.50s/it]
 40%|███▉      | 89854/225000 [164:42:49<167:01:49,  4.45s/it]
 40%|███▉      | 89855/225000 [164:42:53<161:57:32,  4.31s/it]
 40%|███▉      | 89856/225000 [164:42:57<153:04:50,  4.08s/it]
 40%|███▉      | 89857/225000 [164:43:00<144:17:45,  3.84s/it]
 40%|███▉      | 89858/225000 [164:43:04<145:24:04,  3.87s/it]
 40%|███▉      | 89859/225000 [164:43:07<138:09:04,  3.68s/it]
 40%|███▉      | 89860/225000 [164:43:11<135:45:10,  3.62s/it]
 40%|███▉      | 89861/225000 [164:43:16<150:45:29,  4.02s/it]
 40%|███▉      | 89862/225000 [164:43:24<201:16:08,  5.36s/it]
 40%|███▉      | 89863/225000 [164:43:28<180:08:13,  4.80s/it]
 40%|███▉      | 89864/225000 [164:43:30<152:55:49,  4.07s/it]
 40%|███▉      | 89865/225000 [164:43:33<137:01:57,  3.65s/it]
 40%|███▉      | 89866/225000 [164:43:38<152:46:07,  4.07s/it]
 40%|███▉      | 89867/225000 [164:43:46<202:30:33,  5.39s/it]
 40%|███▉      | 89868/225000 [164:43:49<169:17:36,  4.51s/it]
 40%|███▉      | 89869/225000 [164:43:51<147:12:00,  3.92s/it]
 40%|███▉      | 89870/225000 [164:43:56<158:30:04,  4.22s/it]
 40%|███▉      | 89871/225000 [164:44:01<167:48:33,  4.47s/it]
 40%|███▉      | 89872/225000 [164:44:04<155:42:52,  4.15s/it]
 40%|███▉      | 89873/225000 [164:44:07<142:39:31,  3.80s/it]
 40%|███▉      | 89874/225000 [164:44:10<134:09:27,  3.57s/it]
 40%|███▉      | 89875/225000 [164:44:14<129:54:56,  3.46s/it]
 40%|███▉      | 89876/225000 [164:44:17<132:24:42,  3.53s/it]
 40%|███▉      | 89877/225000 [164:44:21<136:55:12,  3.65s/it]
 40%|███▉      | 89878/225000 [164:44:24<129:11:52,  3.44s/it]
 40%|███▉      | 89879/225000 [164:44:31<164:03:31,  4.37s/it]
 40%|███▉      | 89880/225000 [164:44:38<194:56:05,  5.19s/it]
 40%|███▉      | 89881/225000 [164:44:40<163:53:43,  4.37s/it]
 40%|███▉      | 89882/225000 [164:44:44<151:33:43,  4.04s/it]
 40%|███▉      | 89883/225000 [164:44:47<144:16:09,  3.84s/it]
 40%|███▉      | 89884/225000 [164:44:51<141:18:05,  3.76s/it]
 40%|███▉      | 89885/225000 [164:44:54<134:39:20,  3.59s/it]
 40%|███▉      | 89886/225000 [164:44:57<129:57:42,  3.46s/it]
 40%|███▉      | 89887/225000 [164:45:02<148:10:12,  3.95s/it]
 40%|███▉      | 89888/225000 [164:45:05<139:58:49,  3.73s/it]
 40%|███▉      | 89889/225000 [164:45:09<136:56:25,  3.65s/it]
 40%|███▉      | 89890/225000 [164:45:12<130:20:40,  3.47s/it]
 40%|███▉      | 89891/225000 [164:45:15<129:34:02,  3.45s/it]
 40%|███▉      | 89892/225000 [164:45:19<129:37:18,  3.45s/it]
 40%|███▉      | 89893/225000 [164:45:22<133:03:56,  3.55s/it]
 40%|███▉      | 89894/225000 [164:45:29<163:59:23,  4.37s/it]
 40%|███▉      | 89895/225000 [164:45:38<214:44:20,  5.72s/it]
 40%|███▉      | 89896/225000 [164:45:40<178:44:32,  4.76s/it]
 40%|███▉      | 89897/225000 [164:45:42<151:38:57,  4.04s/it]
 40%|███▉      | 89898/225000 [164:45:48<167:19:36,  4.46s/it]
 40%|███▉      | 89899/225000 [164:45:51<155:01:34,  4.13s/it]
 40%|███▉      | 89900/225000 [164:45:54<144:32:13,  3.85s/it]
                                                              
{'loss': '0.3475', 'grad_norm': '4.667', 'learning_rate': '3.483e-05', 'epoch': '0.3996', 'num_input_tokens_seen': 270026720, 'train_runtime': '5.932e+05', 'train_tokens_per_second': '455.2'}

 40%|███▉      | 89900/225000 [164:45:54<144:32:13,  3.85s/it]
 40%|███▉      | 89901/225000 [164:45:58<143:39:19,  3.83s/it]
 40%|███▉      | 89902/225000 [164:46:01<134:47:46,  3.59s/it]
 40%|███▉      | 89903/225000 [164:46:05<132:59:05,  3.54s/it]
 40%|███▉      | 89904/225000 [164:46:08<131:25:42,  3.50s/it]
 40%|███▉      | 89905/225000 [164:46:11<129:55:48,  3.46s/it]
 40%|███▉      | 89906/225000 [164:46:15<134:59:13,  3.60s/it]
 40%|███▉      | 89907/225000 [164:46:19<130:41:08,  3.48s/it]
 40%|███▉      | 89908/225000 [164:46:22<127:14:44,  3.39s/it]
 40%|███▉      | 89909/225000 [164:46:25<128:03:46,  3.41s/it]
 40%|███▉      | 89910/225000 [164:46:29<129:58:27,  3.46s/it]
 40%|███▉      | 89911/225000 [164:46:33<136:27:27,  3.64s/it]
 40%|███▉      | 89912/225000 [164:46:36<134:41:15,  3.59s/it]
 40%|███▉      | 89913/225000 [164:46:42<161:21:08,  4.30s/it]
 40%|███▉      | 89914/225000 [164:46:51<208:01:45,  5.54s/it]
 40%|███▉      | 89915/225000 [164:46:53<174:58:14,  4.66s/it]
 40%|███▉      | 89916/225000 [164:46:56<149:41:10,  3.99s/it]
 40%|███▉      | 89917/225000 [164:46:58<132:27:22,  3.53s/it]
 40%|███▉      | 89918/225000 [164:47:01<125:02:17,  3.33s/it]
 40%|███▉      | 89919/225000 [164:47:05<127:09:38,  3.39s/it]
 40%|███▉      | 89920/225000 [164:47:08<124:33:50,  3.32s/it]
 40%|███▉      | 89921/225000 [164:47:12<136:43:33,  3.64s/it]
 40%|███▉      | 89922/225000 [164:47:22<204:13:15,  5.44s/it]
 40%|███▉      | 89923/225000 [164:47:27<197:17:03,  5.26s/it]
 40%|███▉      | 89924/225000 [164:47:29<163:51:33,  4.37s/it]
 40%|███▉      | 89925/225000 [164:47:33<156:46:33,  4.18s/it]
 40%|███▉      | 89926/225000 [164:47:38<174:29:02,  4.65s/it]
 40%|███▉      | 89927/225000 [164:47:42<160:25:19,  4.28s/it]
 40%|███▉      | 89928/225000 [164:47:45<151:24:56,  4.04s/it]
 40%|███▉      | 89929/225000 [164:47:48<139:34:51,  3.72s/it]
 40%|███▉      | 89930/225000 [164:47:51<133:51:47,  3.57s/it]
 40%|███▉      | 89931/225000 [164:47:55<130:24:58,  3.48s/it]
 40%|███▉      | 89932/225000 [164:47:58<127:35:51,  3.40s/it]
 40%|███▉      | 89933/225000 [164:48:01<121:21:55,  3.23s/it]
 40%|███▉      | 89934/225000 [164:48:04<125:36:23,  3.35s/it]
 40%|███▉      | 89935/225000 [164:48:07<121:04:55,  3.23s/it]
 40%|███▉      | 89936/225000 [164:48:10<118:33:30,  3.16s/it]
 40%|███▉      | 89937/225000 [164:48:14<124:50:28,  3.33s/it]
 40%|███▉      | 89938/225000 [164:48:23<192:23:39,  5.13s/it]
 40%|███▉      | 89939/225000 [164:48:30<207:19:43,  5.53s/it]
 40%|███▉      | 89940/225000 [164:48:33<179:11:43,  4.78s/it]
 40%|███▉      | 89941/225000 [164:48:36<158:25:19,  4.22s/it]
 40%|███▉      | 89942/225000 [164:48:39<144:38:30,  3.86s/it]
 40%|███▉      | 89943/225000 [164:48:42<134:23:20,  3.58s/it]
 40%|███▉      | 89944/225000 [164:48:45<134:08:29,  3.58s/it]
 40%|███▉      | 89945/225000 [164:48:48<125:28:29,  3.34s/it]
 40%|███▉      | 89946/225000 [164:48:57<191:57:36,  5.12s/it]
 40%|███▉      | 89947/225000 [164:49:02<189:16:10,  5.05s/it]
 40%|███▉      | 89948/225000 [164:49:05<159:23:30,  4.25s/it]
 40%|███▉      | 89949/225000 [164:49:08<152:52:30,  4.08s/it]
 40%|███▉      | 89950/225000 [164:49:16<195:39:41,  5.22s/it]
                                                              
{'loss': '0.3436', 'grad_norm': '6.029', 'learning_rate': '3.481e-05', 'epoch': '0.3998', 'num_input_tokens_seen': 270113240, 'train_runtime': '5.934e+05', 'train_tokens_per_second': '455.2'}

 40%|███▉      | 89950/225000 [164:49:16<195:39:41,  5.22s/it]
 40%|███▉      | 89951/225000 [164:49:19<162:44:58,  4.34s/it]
 40%|███▉      | 89952/225000 [164:49:21<146:21:43,  3.90s/it]
 40%|███▉      | 89953/225000 [164:49:29<182:54:32,  4.88s/it]
 40%|███▉      | 89954/225000 [164:49:32<161:48:51,  4.31s/it]
 40%|███▉      | 89955/225000 [164:49:34<143:55:56,  3.84s/it]
 40%|███▉      | 89956/225000 [164:49:37<133:54:54,  3.57s/it]
 40%|███▉      | 89957/225000 [164:49:40<125:16:29,  3.34s/it]
 40%|███▉      | 89958/225000 [164:49:43<123:47:40,  3.30s/it]
 40%|███▉      | 89959/225000 [164:49:50<166:00:55,  4.43s/it]
 40%|███▉      | 89960/225000 [164:49:55<171:58:51,  4.58s/it]
 40%|███▉      | 89961/225000 [164:49:58<151:49:11,  4.05s/it]
 40%|███▉      | 89962/225000 [164:50:01<137:42:37,  3.67s/it]
 40%|███▉      | 89963/225000 [164:50:04<128:12:08,  3.42s/it]
 40%|███▉      | 89964/225000 [164:50:07<131:01:47,  3.49s/it]
 40%|███▉      | 89965/225000 [164:50:10<126:58:43,  3.39s/it]
 40%|███▉      | 89966/225000 [164:50:14<126:25:56,  3.37s/it]
 40%|███▉      | 89967/225000 [164:50:17<123:08:58,  3.28s/it]
 40%|███▉      | 89968/225000 [164:50:22<146:50:40,  3.91s/it]
 40%|███▉      | 89969/225000 [164:50:32<209:56:31,  5.60s/it]
 40%|███▉      | 89970/225000 [164:50:37<201:51:39,  5.38s/it]
 40%|███▉      | 89971/225000 [164:50:39<169:45:55,  4.53s/it]
 40%|███▉      | 89972/225000 [164:50:42<145:37:45,  3.88s/it]
 40%|███▉      | 89973/225000 [164:50:45<135:42:13,  3.62s/it]
 40%|███▉      | 89974/225000 [164:50:48<129:56:35,  3.46s/it]
 40%|███▉      | 89975/225000 [164:50:52<140:33:04,  3.75s/it]
 40%|███▉      | 89976/225000 [164:51:00<192:30:43,  5.13s/it]
 40%|███▉      | 89977/225000 [164:51:03<165:26:25,  4.41s/it]
 40%|███▉      | 89978/225000 [164:51:07<154:22:03,  4.12s/it]
 40%|███▉      | 89979/225000 [164:51:10<143:51:02,  3.84s/it]
 40%|███▉      | 89980/225000 [164:51:13<134:36:06,  3.59s/it]
 40%|███▉      | 89981/225000 [164:51:16<125:33:33,  3.35s/it]
 40%|███▉      | 89982/225000 [164:51:18<119:18:04,  3.18s/it]
 40%|███▉      | 89983/225000 [164:51:21<118:17:05,  3.15s/it]
 40%|███▉      | 89984/225000 [164:51:30<178:41:27,  4.76s/it]
 40%|███▉      | 89985/225000 [164:51:34<172:02:07,  4.59s/it]
 40%|███▉      | 89986/225000 [164:51:37<149:33:21,  3.99s/it]
 40%|███▉      | 89987/225000 [164:51:39<135:32:27,  3.61s/it]
 40%|███▉      | 89988/225000 [164:51:47<176:46:07,  4.71s/it]
 40%|███▉      | 89989/225000 [164:51:51<168:51:37,  4.50s/it]
 40%|███▉      | 89990/225000 [164:51:54<156:01:11,  4.16s/it]
 40%|███▉      | 89991/225000 [164:51:58<148:54:15,  3.97s/it]
 40%|███▉      | 89992/225000 [164:52:01<142:40:48,  3.80s/it]
 40%|███▉      | 89993/225000 [164:52:06<152:03:12,  4.05s/it]
 40%|███▉      | 89994/225000 [164:52:09<146:11:45,  3.90s/it]
 40%|███▉      | 89995/225000 [164:52:13<139:17:19,  3.71s/it]
 40%|███▉      | 89996/225000 [164:52:16<134:58:40,  3.60s/it]
 40%|███▉      | 89997/225000 [164:52:19<132:25:10,  3.53s/it]
 40%|███▉      | 89998/225000 [164:52:23<133:24:18,  3.56s/it]
 40%|███▉      | 89999/225000 [164:52:33<212:26:20,  5.67s/it]
 40%|████      | 90000/225000 [164:52:36<181:48:25,  4.85s/it]
                                                              
{'loss': '0.3271', 'grad_norm': '4.6', 'learning_rate': '3.479e-05', 'epoch': '0.4', 'num_input_tokens_seen': 270201160, 'train_runtime': '5.936e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90000/225000 [164:52:36<181:48:25,  4.85s/it]
 40%|████      | 90001/225000 [164:52:39<157:37:56,  4.20s/it]
 40%|████      | 90002/225000 [164:52:42<146:04:46,  3.90s/it]
 40%|████      | 90003/225000 [164:52:45<136:25:21,  3.64s/it]
 40%|████      | 90004/225000 [164:52:48<129:04:58,  3.44s/it]
 40%|████      | 90005/225000 [164:52:54<149:56:09,  4.00s/it]
 40%|████      | 90006/225000 [164:53:02<203:59:07,  5.44s/it]
 40%|████      | 90007/225000 [164:53:05<168:48:41,  4.50s/it]
 40%|████      | 90008/225000 [164:53:07<144:09:49,  3.84s/it]
 40%|████      | 90009/225000 [164:53:13<170:04:40,  4.54s/it]
 40%|████      | 90010/225000 [164:53:17<167:40:47,  4.47s/it]
 40%|████      | 90011/225000 [164:53:20<150:23:05,  4.01s/it]
 40%|████      | 90012/225000 [164:53:23<134:57:48,  3.60s/it]
 40%|████      | 90013/225000 [164:53:26<129:28:34,  3.45s/it]
 40%|████      | 90014/225000 [164:53:34<176:04:27,  4.70s/it]
 40%|████      | 90015/225000 [164:53:39<186:41:55,  4.98s/it]
 40%|████      | 90016/225000 [164:53:43<165:42:19,  4.42s/it]
 40%|████      | 90017/225000 [164:53:46<153:13:07,  4.09s/it]
 40%|████      | 90018/225000 [164:53:49<139:01:59,  3.71s/it]
 40%|████      | 90019/225000 [164:53:52<139:58:13,  3.73s/it]
 40%|████      | 90020/225000 [164:53:55<130:43:43,  3.49s/it]
 40%|████      | 90021/225000 [164:54:04<189:02:46,  5.04s/it]
 40%|████      | 90022/225000 [164:54:07<161:33:14,  4.31s/it]
 40%|████      | 90023/225000 [164:54:09<144:03:55,  3.84s/it]
 40%|████      | 90024/225000 [164:54:13<136:24:32,  3.64s/it]
 40%|████      | 90025/225000 [164:54:16<130:24:24,  3.48s/it]
 40%|████      | 90026/225000 [164:54:24<184:30:09,  4.92s/it]
 40%|████      | 90027/225000 [164:54:26<157:10:18,  4.19s/it]
 40%|████      | 90028/225000 [164:54:29<136:39:42,  3.65s/it]
 40%|████      | 90029/225000 [164:54:31<123:09:18,  3.28s/it]
 40%|████      | 90030/225000 [164:54:34<119:47:28,  3.20s/it]
 40%|████      | 90031/225000 [164:54:38<126:43:00,  3.38s/it]
 40%|████      | 90032/225000 [164:54:42<132:12:33,  3.53s/it]
 40%|████      | 90033/225000 [164:54:45<128:15:32,  3.42s/it]
 40%|████      | 90034/225000 [164:54:48<128:11:47,  3.42s/it]
 40%|████      | 90035/225000 [164:54:53<135:59:25,  3.63s/it]
 40%|████      | 90036/225000 [164:54:58<155:35:44,  4.15s/it]
 40%|████      | 90037/225000 [164:55:07<210:02:34,  5.60s/it]
 40%|████      | 90038/225000 [164:55:12<200:01:29,  5.34s/it]
 40%|████      | 90039/225000 [164:55:15<171:55:11,  4.59s/it]
 40%|████      | 90040/225000 [164:55:18<157:15:19,  4.19s/it]
 40%|████      | 90041/225000 [164:55:21<147:43:31,  3.94s/it]
 40%|████      | 90042/225000 [164:55:24<134:34:44,  3.59s/it]
 40%|████      | 90043/225000 [164:55:27<126:39:55,  3.38s/it]
 40%|████      | 90044/225000 [164:55:31<136:47:00,  3.65s/it]
 40%|████      | 90045/225000 [164:55:40<191:43:32,  5.11s/it]
 40%|████      | 90046/225000 [164:55:45<192:07:27,  5.13s/it]
 40%|████      | 90047/225000 [164:55:48<166:16:27,  4.44s/it]
 40%|████      | 90048/225000 [164:55:50<147:48:13,  3.94s/it]
 40%|████      | 90049/225000 [164:55:54<139:54:19,  3.73s/it]
 40%|████      | 90050/225000 [164:55:57<138:11:41,  3.69s/it]
                                                              
{'loss': '0.3358', 'grad_norm': '6.267', 'learning_rate': '3.478e-05', 'epoch': '0.4002', 'num_input_tokens_seen': 270288416, 'train_runtime': '5.938e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90050/225000 [164:55:57<138:11:41,  3.69s/it]
 40%|████      | 90051/225000 [164:56:00<133:31:45,  3.56s/it]
 40%|████      | 90052/225000 [164:56:03<123:42:51,  3.30s/it]
 40%|████      | 90053/225000 [164:56:07<124:07:44,  3.31s/it]
 40%|████      | 90054/225000 [164:56:14<170:57:46,  4.56s/it]
 40%|████      | 90055/225000 [164:56:21<196:25:10,  5.24s/it]
 40%|████      | 90056/225000 [164:56:23<164:06:12,  4.38s/it]
 40%|████      | 90057/225000 [164:56:26<143:45:07,  3.84s/it]
 40%|████      | 90058/225000 [164:56:28<127:32:43,  3.40s/it]
 40%|████      | 90059/225000 [164:56:35<167:35:06,  4.47s/it]
 40%|████      | 90060/225000 [164:56:37<141:51:48,  3.78s/it]
 40%|████      | 90061/225000 [164:56:45<180:47:12,  4.82s/it]
 40%|████      | 90062/225000 [164:56:47<155:17:05,  4.14s/it]
 40%|████      | 90063/225000 [164:56:50<144:51:58,  3.86s/it]
 40%|████      | 90064/225000 [164:56:53<131:21:05,  3.50s/it]
 40%|████      | 90065/225000 [164:56:57<135:26:06,  3.61s/it]
 40%|████      | 90066/225000 [164:57:00<131:29:14,  3.51s/it]
 40%|████      | 90067/225000 [164:57:03<129:03:56,  3.44s/it]
 40%|████      | 90068/225000 [164:57:07<131:36:32,  3.51s/it]
 40%|████      | 90069/225000 [164:57:11<132:37:56,  3.54s/it]
 40%|████      | 90070/225000 [164:57:15<139:01:54,  3.71s/it]
 40%|████      | 90071/225000 [164:57:19<142:10:43,  3.79s/it]
 40%|████      | 90072/225000 [164:57:22<136:51:41,  3.65s/it]
 40%|████      | 90073/225000 [164:57:25<132:24:38,  3.53s/it]
 40%|████      | 90074/225000 [164:57:29<134:40:21,  3.59s/it]
 40%|████      | 90075/225000 [164:57:33<135:09:59,  3.61s/it]
 40%|████      | 90076/225000 [164:57:40<181:44:58,  4.85s/it]
 40%|████      | 90077/225000 [164:57:46<194:12:11,  5.18s/it]
 40%|████      | 90078/225000 [164:57:50<174:13:04,  4.65s/it]
 40%|████      | 90079/225000 [164:57:54<170:17:52,  4.54s/it]
 40%|████      | 90080/225000 [164:57:58<164:03:06,  4.38s/it]
 40%|████      | 90081/225000 [164:58:03<173:37:41,  4.63s/it]
 40%|████      | 90082/225000 [164:58:06<150:29:43,  4.02s/it]
 40%|████      | 90083/225000 [164:58:08<132:15:23,  3.53s/it]
 40%|████      | 90084/225000 [164:58:16<174:28:44,  4.66s/it]
 40%|████      | 90085/225000 [164:58:20<170:23:04,  4.55s/it]
 40%|████      | 90086/225000 [164:58:22<145:28:22,  3.88s/it]
 40%|████      | 90087/225000 [164:58:25<131:27:33,  3.51s/it]
 40%|████      | 90088/225000 [164:58:28<128:50:43,  3.44s/it]
 40%|████      | 90089/225000 [164:58:31<121:04:30,  3.23s/it]
 40%|████      | 90090/225000 [164:58:34<118:22:37,  3.16s/it]
 40%|████      | 90091/225000 [164:58:43<181:24:49,  4.84s/it]
 40%|████      | 90092/225000 [164:58:47<177:02:23,  4.72s/it]
 40%|████      | 90093/225000 [164:58:50<151:43:30,  4.05s/it]
 40%|████      | 90094/225000 [164:58:52<137:25:13,  3.67s/it]
 40%|████      | 90095/225000 [164:59:00<186:59:17,  4.99s/it]
 40%|████      | 90096/225000 [164:59:05<181:58:29,  4.86s/it]
 40%|████      | 90097/225000 [164:59:07<155:16:03,  4.14s/it]
 40%|████      | 90098/225000 [164:59:10<137:27:13,  3.67s/it]
 40%|████      | 90099/225000 [164:59:13<125:43:18,  3.36s/it]
 40%|████      | 90100/225000 [164:59:16<123:32:37,  3.30s/it]
                                                              
{'loss': '0.3544', 'grad_norm': '3.209', 'learning_rate': '3.476e-05', 'epoch': '0.4004', 'num_input_tokens_seen': 270375728, 'train_runtime': '5.94e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90100/225000 [164:59:16<123:32:37,  3.30s/it]
 40%|████      | 90101/225000 [164:59:19<120:45:24,  3.22s/it]
 40%|████      | 90102/225000 [164:59:22<120:02:50,  3.20s/it]
 40%|████      | 90103/225000 [164:59:31<181:16:13,  4.84s/it]
 40%|████      | 90104/225000 [164:59:36<184:33:17,  4.93s/it]
 40%|████      | 90105/225000 [164:59:38<156:11:24,  4.17s/it]
 40%|████      | 90106/225000 [164:59:41<141:37:04,  3.78s/it]
 40%|████      | 90107/225000 [164:59:45<138:24:35,  3.69s/it]
 40%|████      | 90108/225000 [164:59:47<128:23:36,  3.43s/it]
 40%|████      | 90109/225000 [164:59:52<145:43:10,  3.89s/it]
 40%|████      | 90110/225000 [165:00:01<197:40:03,  5.28s/it]
 40%|████      | 90111/225000 [165:00:04<170:06:49,  4.54s/it]
 40%|████      | 90112/225000 [165:00:06<150:22:38,  4.01s/it]
 40%|████      | 90113/225000 [165:00:09<134:24:53,  3.59s/it]
 40%|████      | 90114/225000 [165:00:12<127:04:12,  3.39s/it]
 40%|████      | 90115/225000 [165:00:15<118:37:02,  3.17s/it]
 40%|████      | 90116/225000 [165:00:18<120:02:13,  3.20s/it]
 40%|████      | 90117/225000 [165:00:25<162:38:45,  4.34s/it]
 40%|████      | 90118/225000 [165:00:31<184:58:20,  4.94s/it]
 40%|████      | 90119/225000 [165:00:34<160:23:21,  4.28s/it]
 40%|████      | 90120/225000 [165:00:37<144:02:04,  3.84s/it]
 40%|████      | 90121/225000 [165:00:42<158:20:37,  4.23s/it]
 40%|████      | 90122/225000 [165:00:48<174:36:53,  4.66s/it]
 40%|████      | 90123/225000 [165:00:50<154:06:15,  4.11s/it]
 40%|████      | 90124/225000 [165:00:56<167:51:00,  4.48s/it]
 40%|████      | 90125/225000 [165:00:59<157:40:06,  4.21s/it]
 40%|████      | 90126/225000 [165:01:02<142:40:17,  3.81s/it]
 40%|████      | 90127/225000 [165:01:05<135:32:35,  3.62s/it]
 40%|████      | 90128/225000 [165:01:08<128:31:15,  3.43s/it]
 40%|████      | 90129/225000 [165:01:12<126:10:17,  3.37s/it]
 40%|████      | 90130/225000 [165:01:15<121:45:16,  3.25s/it]
 40%|████      | 90131/225000 [165:01:19<134:14:25,  3.58s/it]
 40%|████      | 90132/225000 [165:01:28<191:38:28,  5.12s/it]
 40%|████      | 90133/225000 [165:01:31<167:23:40,  4.47s/it]
 40%|████      | 90134/225000 [165:01:33<146:35:33,  3.91s/it]
 40%|████      | 90135/225000 [165:01:38<154:44:18,  4.13s/it]
 40%|████      | 90136/225000 [165:01:44<182:02:29,  4.86s/it]
 40%|████      | 90137/225000 [165:01:48<166:29:10,  4.44s/it]
 40%|████      | 90138/225000 [165:01:51<150:53:01,  4.03s/it]
 40%|████      | 90139/225000 [165:01:54<146:00:01,  3.90s/it]
 40%|████      | 90140/225000 [165:01:58<137:45:27,  3.68s/it]
 40%|████      | 90141/225000 [165:02:01<132:56:09,  3.55s/it]
 40%|████      | 90142/225000 [165:02:04<132:55:03,  3.55s/it]
 40%|████      | 90143/225000 [165:02:08<131:29:44,  3.51s/it]
 40%|████      | 90144/225000 [165:02:11<124:37:33,  3.33s/it]
 40%|████      | 90145/225000 [165:02:14<122:23:59,  3.27s/it]
 40%|████      | 90146/225000 [165:02:17<125:47:19,  3.36s/it]
 40%|████      | 90147/225000 [165:02:21<122:37:10,  3.27s/it]
 40%|████      | 90148/225000 [165:02:24<124:00:13,  3.31s/it]
 40%|████      | 90149/225000 [165:02:29<137:59:09,  3.68s/it]
 40%|████      | 90150/225000 [165:02:38<198:42:23,  5.30s/it]
                                                              
{'loss': '0.3667', 'grad_norm': '6.568', 'learning_rate': '3.474e-05', 'epoch': '0.4007', 'num_input_tokens_seen': 270462232, 'train_runtime': '5.942e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90150/225000 [165:02:38<198:42:23,  5.30s/it]
 40%|████      | 90151/225000 [165:02:45<226:05:15,  6.04s/it]
 40%|████      | 90152/225000 [165:02:48<193:27:24,  5.16s/it]
 40%|████      | 90153/225000 [165:02:52<170:42:31,  4.56s/it]
 40%|████      | 90154/225000 [165:02:55<163:12:20,  4.36s/it]
 40%|████      | 90155/225000 [165:02:59<151:45:56,  4.05s/it]
 40%|████      | 90156/225000 [165:03:02<146:50:29,  3.92s/it]
 40%|████      | 90157/225000 [165:03:08<165:23:16,  4.42s/it]
 40%|████      | 90158/225000 [165:03:12<156:51:05,  4.19s/it]
 40%|████      | 90159/225000 [165:03:14<136:57:49,  3.66s/it]
 40%|████      | 90160/225000 [165:03:22<179:54:29,  4.80s/it]
 40%|████      | 90161/225000 [165:03:24<153:31:06,  4.10s/it]
 40%|████      | 90162/225000 [165:03:26<134:47:52,  3.60s/it]
 40%|████      | 90163/225000 [165:03:29<122:13:23,  3.26s/it]
 40%|████      | 90164/225000 [165:03:34<143:10:25,  3.82s/it]
 40%|████      | 90165/225000 [165:03:42<193:52:54,  5.18s/it]
 40%|████      | 90166/225000 [165:03:45<165:25:41,  4.42s/it]
 40%|████      | 90167/225000 [165:03:48<146:01:51,  3.90s/it]
 40%|████      | 90168/225000 [165:03:51<135:47:21,  3.63s/it]
 40%|████      | 90169/225000 [165:03:54<126:58:27,  3.39s/it]
 40%|████      | 90170/225000 [165:03:57<122:40:30,  3.28s/it]
 40%|████      | 90171/225000 [165:04:00<124:51:36,  3.33s/it]
 40%|████      | 90172/225000 [165:04:08<181:39:07,  4.85s/it]
 40%|████      | 90173/225000 [165:04:12<161:43:22,  4.32s/it]
 40%|████      | 90174/225000 [165:04:15<148:05:37,  3.95s/it]
 40%|████      | 90175/225000 [165:04:22<181:08:46,  4.84s/it]
 40%|████      | 90176/225000 [165:04:24<152:05:23,  4.06s/it]
 40%|████      | 90177/225000 [165:04:26<136:30:53,  3.65s/it]
 40%|████      | 90178/225000 [165:04:33<172:44:44,  4.61s/it]
 40%|████      | 90179/225000 [165:04:36<151:52:51,  4.06s/it]
 40%|████      | 90180/225000 [165:04:39<142:18:15,  3.80s/it]
 40%|████      | 90181/225000 [165:04:42<132:25:33,  3.54s/it]
 40%|████      | 90182/225000 [165:04:45<122:41:17,  3.28s/it]
 40%|████      | 90183/225000 [165:04:49<128:50:18,  3.44s/it]
 40%|████      | 90184/225000 [165:04:57<184:35:35,  4.93s/it]
 40%|████      | 90185/225000 [165:05:00<165:10:11,  4.41s/it]
 40%|████      | 90186/225000 [165:05:04<153:17:17,  4.09s/it]
 40%|████      | 90187/225000 [165:05:07<142:29:32,  3.81s/it]
 40%|████      | 90188/225000 [165:05:10<137:08:08,  3.66s/it]
 40%|████      | 90189/225000 [165:05:14<135:53:23,  3.63s/it]
 40%|████      | 90190/225000 [165:05:16<127:08:03,  3.40s/it]
 40%|████      | 90191/225000 [165:05:20<129:45:21,  3.47s/it]
 40%|████      | 90192/225000 [165:05:23<124:19:54,  3.32s/it]
 40%|████      | 90193/225000 [165:05:26<121:51:56,  3.25s/it]
 40%|████      | 90194/225000 [165:05:34<170:20:26,  4.55s/it]
 40%|████      | 90195/225000 [165:05:40<187:27:47,  5.01s/it]
 40%|████      | 90196/225000 [165:05:43<161:15:59,  4.31s/it]
 40%|████      | 90197/225000 [165:05:49<181:20:30,  4.84s/it]
 40%|████      | 90198/225000 [165:05:51<155:35:18,  4.16s/it]
 40%|████      | 90199/225000 [165:05:54<139:07:06,  3.72s/it]
 40%|████      | 90200/225000 [165:05:57<127:42:13,  3.41s/it]
                                                              
{'loss': '0.3487', 'grad_norm': '3.708', 'learning_rate': '3.473e-05', 'epoch': '0.4009', 'num_input_tokens_seen': 270548384, 'train_runtime': '5.944e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90200/225000 [165:05:57<127:42:13,  3.41s/it]
 40%|████      | 90201/225000 [165:06:02<154:13:26,  4.12s/it]
 40%|████      | 90202/225000 [165:06:08<173:48:54,  4.64s/it]
 40%|████      | 90203/225000 [165:06:11<157:24:36,  4.20s/it]
 40%|████      | 90204/225000 [165:06:15<145:53:38,  3.90s/it]
 40%|████      | 90205/225000 [165:06:18<139:40:10,  3.73s/it]
 40%|████      | 90206/225000 [165:06:21<134:08:02,  3.58s/it]
 40%|████      | 90207/225000 [165:06:24<131:15:32,  3.51s/it]
 40%|████      | 90208/225000 [165:06:28<130:27:19,  3.48s/it]
 40%|████      | 90209/225000 [165:06:35<166:40:17,  4.45s/it]
 40%|████      | 90210/225000 [165:06:43<211:38:43,  5.65s/it]
 40%|████      | 90211/225000 [165:06:45<173:58:19,  4.65s/it]
 40%|████      | 90212/225000 [165:06:48<150:16:01,  4.01s/it]
 40%|████      | 90213/225000 [165:06:54<170:21:08,  4.55s/it]
 40%|████      | 90214/225000 [165:06:57<158:55:04,  4.24s/it]
 40%|████      | 90215/225000 [165:07:00<142:01:14,  3.79s/it]
 40%|████      | 90216/225000 [165:07:03<132:47:30,  3.55s/it]
 40%|████      | 90217/225000 [165:07:06<130:32:01,  3.49s/it]
 40%|████      | 90218/225000 [165:07:09<125:41:16,  3.36s/it]
 40%|████      | 90219/225000 [165:07:14<139:46:42,  3.73s/it]
 40%|████      | 90220/225000 [165:07:17<134:17:16,  3.59s/it]
 40%|████      | 90221/225000 [165:07:21<132:27:34,  3.54s/it]
 40%|████      | 90222/225000 [165:07:24<131:27:28,  3.51s/it]
 40%|████      | 90223/225000 [165:07:27<127:38:29,  3.41s/it]
 40%|████      | 90224/225000 [165:07:30<125:12:30,  3.34s/it]
 40%|████      | 90225/225000 [165:07:36<149:44:51,  4.00s/it]
 40%|████      | 90226/225000 [165:07:44<197:58:02,  5.29s/it]
 40%|████      | 90227/225000 [165:07:48<176:04:48,  4.70s/it]
 40%|████      | 90228/225000 [165:07:51<160:05:33,  4.28s/it]
 40%|████      | 90229/225000 [165:07:55<153:31:01,  4.10s/it]
 40%|████      | 90230/225000 [165:08:03<200:14:51,  5.35s/it]
 40%|████      | 90231/225000 [165:08:05<168:32:02,  4.50s/it]
 40%|████      | 90232/225000 [165:08:08<148:00:42,  3.95s/it]
 40%|████      | 90233/225000 [165:08:11<134:57:16,  3.61s/it]
 40%|████      | 90234/225000 [165:08:19<185:58:49,  4.97s/it]
 40%|████      | 90235/225000 [165:08:22<162:44:26,  4.35s/it]
 40%|████      | 90236/225000 [165:08:25<149:14:43,  3.99s/it]
 40%|████      | 90237/225000 [165:08:28<132:31:56,  3.54s/it]
 40%|████      | 90238/225000 [165:08:31<126:33:51,  3.38s/it]
 40%|████      | 90239/225000 [165:08:36<153:19:18,  4.10s/it]
 40%|████      | 90240/225000 [165:08:42<173:18:39,  4.63s/it]
 40%|████      | 90241/225000 [165:08:45<153:08:43,  4.09s/it]
 40%|████      | 90242/225000 [165:08:48<138:13:52,  3.69s/it]
 40%|████      | 90243/225000 [165:08:51<131:47:25,  3.52s/it]
 40%|████      | 90244/225000 [165:08:54<132:15:02,  3.53s/it]
 40%|████      | 90245/225000 [165:08:58<136:04:19,  3.64s/it]
 40%|████      | 90246/225000 [165:09:02<133:03:09,  3.55s/it]
 40%|████      | 90247/225000 [165:09:05<132:46:48,  3.55s/it]
 40%|████      | 90248/225000 [165:09:09<132:06:47,  3.53s/it]
 40%|████      | 90249/225000 [165:09:15<167:18:33,  4.47s/it]
 40%|████      | 90250/225000 [165:09:19<157:59:33,  4.22s/it]
                                                              
{'loss': '0.3338', 'grad_norm': '5.393', 'learning_rate': '3.471e-05', 'epoch': '0.4011', 'num_input_tokens_seen': 270636368, 'train_runtime': '5.946e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90250/225000 [165:09:19<157:59:33,  4.22s/it]
 40%|████      | 90251/225000 [165:09:22<144:50:11,  3.87s/it]
 40%|████      | 90252/225000 [165:09:25<136:33:32,  3.65s/it]
 40%|████      | 90253/225000 [165:09:31<156:29:09,  4.18s/it]
 40%|████      | 90254/225000 [165:09:40<213:04:37,  5.69s/it]
 40%|████      | 90255/225000 [165:09:44<194:41:37,  5.20s/it]
 40%|████      | 90256/225000 [165:09:47<172:43:18,  4.61s/it]
 40%|████      | 90257/225000 [165:09:52<178:26:29,  4.77s/it]
 40%|████      | 90258/225000 [165:09:55<155:51:05,  4.16s/it]
 40%|████      | 90259/225000 [165:09:58<142:19:39,  3.80s/it]
 40%|████      | 90260/225000 [165:10:01<132:57:07,  3.55s/it]
 40%|████      | 90261/225000 [165:10:04<128:53:43,  3.44s/it]
 40%|████      | 90262/225000 [165:10:07<128:07:29,  3.42s/it]
 40%|████      | 90263/225000 [165:10:17<193:18:29,  5.16s/it]
 40%|████      | 90264/225000 [165:10:22<200:04:50,  5.35s/it]
 40%|████      | 90265/225000 [165:10:25<168:44:24,  4.51s/it]
 40%|████      | 90266/225000 [165:10:27<144:48:34,  3.87s/it]
 40%|████      | 90267/225000 [165:10:31<136:43:03,  3.65s/it]
 40%|████      | 90268/225000 [165:10:34<131:55:59,  3.53s/it]
 40%|████      | 90269/225000 [165:10:37<128:17:05,  3.43s/it]
 40%|████      | 90270/225000 [165:10:40<126:59:00,  3.39s/it]
 40%|████      | 90271/225000 [165:10:44<125:18:11,  3.35s/it]
 40%|████      | 90272/225000 [165:10:47<131:01:27,  3.50s/it]
 40%|████      | 90273/225000 [165:10:51<131:18:46,  3.51s/it]
 40%|████      | 90274/225000 [165:10:54<128:57:48,  3.45s/it]
 40%|████      | 90275/225000 [165:11:00<156:46:51,  4.19s/it]
 40%|████      | 90276/225000 [165:11:07<189:20:19,  5.06s/it]
 40%|████      | 90277/225000 [165:11:11<171:40:44,  4.59s/it]
 40%|████      | 90278/225000 [165:11:18<197:22:34,  5.27s/it]
 40%|████      | 90279/225000 [165:11:21<170:56:03,  4.57s/it]
 40%|████      | 90280/225000 [165:11:23<151:56:21,  4.06s/it]
 40%|████      | 90281/225000 [165:11:26<139:21:52,  3.72s/it]
 40%|████      | 90282/225000 [165:11:29<132:51:14,  3.55s/it]
 40%|████      | 90283/225000 [165:11:33<132:14:44,  3.53s/it]
 40%|████      | 90284/225000 [165:11:37<138:06:22,  3.69s/it]
 40%|████      | 90285/225000 [165:11:46<191:39:43,  5.12s/it]
 40%|████      | 90286/225000 [165:11:50<179:47:29,  4.80s/it]
 40%|████      | 90287/225000 [165:11:52<157:30:54,  4.21s/it]
 40%|████      | 90288/225000 [165:11:56<151:38:48,  4.05s/it]
 40%|████      | 90289/225000 [165:11:59<142:22:58,  3.81s/it]
 40%|████      | 90290/225000 [165:12:03<136:05:12,  3.64s/it]
 40%|████      | 90291/225000 [165:12:12<205:12:19,  5.48s/it]
 40%|████      | 90292/225000 [165:12:17<191:56:21,  5.13s/it]
 40%|████      | 90293/225000 [165:12:19<163:17:32,  4.36s/it]
 40%|████      | 90294/225000 [165:12:22<141:02:29,  3.77s/it]
 40%|████      | 90295/225000 [165:12:26<148:56:04,  3.98s/it]
 40%|████      | 90296/225000 [165:12:29<140:56:00,  3.77s/it]
 40%|████      | 90297/225000 [165:12:33<140:04:04,  3.74s/it]
 40%|████      | 90298/225000 [165:12:37<139:05:19,  3.72s/it]
 40%|████      | 90299/225000 [165:12:40<133:02:07,  3.56s/it]
 40%|████      | 90300/225000 [165:12:43<130:06:34,  3.48s/it]
                                                              
{'loss': '0.3247', 'grad_norm': '4.555', 'learning_rate': '3.469e-05', 'epoch': '0.4013', 'num_input_tokens_seen': 270723800, 'train_runtime': '5.948e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90300/225000 [165:12:43<130:06:34,  3.48s/it]
 40%|████      | 90301/225000 [165:12:46<128:02:52,  3.42s/it]
 40%|████      | 90302/225000 [165:12:50<130:18:45,  3.48s/it]
 40%|████      | 90303/225000 [165:12:54<132:12:32,  3.53s/it]
 40%|████      | 90304/225000 [165:12:57<133:18:00,  3.56s/it]
 40%|████      | 90305/225000 [165:13:01<134:06:19,  3.58s/it]
 40%|████      | 90306/225000 [165:13:10<199:27:12,  5.33s/it]
 40%|████      | 90307/225000 [165:13:14<185:25:49,  4.96s/it]
 40%|████      | 90308/225000 [165:13:18<168:18:08,  4.50s/it]
 40%|████      | 90309/225000 [165:13:21<154:11:34,  4.12s/it]
 40%|████      | 90310/225000 [165:13:24<142:54:49,  3.82s/it]
 40%|████      | 90311/225000 [165:13:28<141:51:57,  3.79s/it]
 40%|████      | 90312/225000 [165:13:31<138:05:01,  3.69s/it]
 40%|████      | 90313/225000 [165:13:35<137:01:19,  3.66s/it]
 40%|████      | 90314/225000 [165:13:39<134:45:16,  3.60s/it]
 40%|████      | 90315/225000 [165:13:42<138:04:59,  3.69s/it]
 40%|████      | 90316/225000 [165:13:46<133:34:03,  3.57s/it]
 40%|████      | 90317/225000 [165:13:49<131:02:48,  3.50s/it]
 40%|████      | 90318/225000 [165:13:52<125:40:48,  3.36s/it]
 40%|████      | 90319/225000 [165:13:55<121:53:32,  3.26s/it]
 40%|████      | 90320/225000 [165:14:03<170:49:50,  4.57s/it]
 40%|████      | 90321/225000 [165:14:10<202:25:00,  5.41s/it]
 40%|████      | 90322/225000 [165:14:12<167:20:37,  4.47s/it]
 40%|████      | 90323/225000 [165:14:15<146:35:09,  3.92s/it]
 40%|████      | 90324/225000 [165:14:23<189:13:43,  5.06s/it]
 40%|████      | 90325/225000 [165:14:25<161:48:42,  4.33s/it]
 40%|████      | 90326/225000 [165:14:28<142:28:17,  3.81s/it]
 40%|████      | 90327/225000 [165:14:31<138:07:37,  3.69s/it]
 40%|████      | 90328/225000 [165:14:34<126:00:35,  3.37s/it]
 40%|████      | 90329/225000 [165:14:41<167:55:53,  4.49s/it]
 40%|████      | 90330/225000 [165:14:47<188:45:51,  5.05s/it]
 40%|████      | 90331/225000 [165:14:50<164:15:23,  4.39s/it]
 40%|████      | 90332/225000 [165:14:53<149:21:40,  3.99s/it]
 40%|████      | 90333/225000 [165:14:56<139:46:12,  3.74s/it]
 40%|████      | 90334/225000 [165:15:00<131:55:08,  3.53s/it]
 40%|████      | 90335/225000 [165:15:03<126:52:19,  3.39s/it]
 40%|████      | 90336/225000 [165:15:08<146:43:22,  3.92s/it]
 40%|████      | 90337/225000 [165:15:16<196:59:19,  5.27s/it]
 40%|████      | 90338/225000 [165:15:19<170:16:19,  4.55s/it]
 40%|████      | 90339/225000 [165:15:23<159:02:32,  4.25s/it]
 40%|████      | 90340/225000 [165:15:26<145:27:38,  3.89s/it]
 40%|████      | 90341/225000 [165:15:29<139:27:48,  3.73s/it]
 40%|████      | 90342/225000 [165:15:32<133:18:38,  3.56s/it]
 40%|████      | 90343/225000 [165:15:35<126:57:08,  3.39s/it]
 40%|████      | 90344/225000 [165:15:44<188:21:47,  5.04s/it]
 40%|████      | 90345/225000 [165:15:49<183:42:44,  4.91s/it]
 40%|████      | 90346/225000 [165:15:51<157:01:14,  4.20s/it]
 40%|████      | 90347/225000 [165:15:54<137:53:16,  3.69s/it]
 40%|████      | 90348/225000 [165:16:02<186:30:28,  4.99s/it]
 40%|████      | 90349/225000 [165:16:04<156:13:34,  4.18s/it]
 40%|████      | 90350/225000 [165:16:06<136:15:18,  3.64s/it]
                                                              
{'loss': '0.3405', 'grad_norm': '6.663', 'learning_rate': '3.467e-05', 'epoch': '0.4016', 'num_input_tokens_seen': 270810264, 'train_runtime': '5.95e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90350/225000 [165:16:06<136:15:18,  3.64s/it]
 40%|████      | 90351/225000 [165:16:11<147:22:44,  3.94s/it]
 40%|████      | 90352/225000 [165:16:15<153:15:45,  4.10s/it]
 40%|████      | 90353/225000 [165:16:18<135:51:12,  3.63s/it]
 40%|████      | 90354/225000 [165:16:21<127:47:53,  3.42s/it]
 40%|████      | 90355/225000 [165:16:24<125:43:26,  3.36s/it]
 40%|████      | 90356/225000 [165:16:27<125:00:41,  3.34s/it]
 40%|████      | 90357/225000 [165:16:30<120:15:03,  3.22s/it]
 40%|████      | 90358/225000 [165:16:36<147:48:15,  3.95s/it]
 40%|████      | 90359/225000 [165:16:39<140:25:15,  3.75s/it]
 40%|████      | 90360/225000 [165:16:43<137:59:29,  3.69s/it]
 40%|████      | 90361/225000 [165:16:46<136:51:47,  3.66s/it]
 40%|████      | 90362/225000 [165:16:50<132:01:39,  3.53s/it]
 40%|████      | 90363/225000 [165:16:53<131:40:48,  3.52s/it]
 40%|████      | 90364/225000 [165:16:56<128:20:29,  3.43s/it]
 40%|████      | 90365/225000 [165:17:00<125:23:58,  3.35s/it]
 40%|████      | 90366/225000 [165:17:03<126:49:01,  3.39s/it]
 40%|████      | 90367/225000 [165:17:06<124:35:35,  3.33s/it]
 40%|████      | 90368/225000 [165:17:10<130:22:10,  3.49s/it]
 40%|████      | 90369/225000 [165:17:14<134:29:12,  3.60s/it]
 40%|████      | 90370/225000 [165:17:18<134:38:50,  3.60s/it]
 40%|████      | 90371/225000 [165:17:26<185:40:59,  4.97s/it]
 40%|████      | 90372/225000 [165:17:35<231:48:06,  6.20s/it]
 40%|████      | 90373/225000 [165:17:38<198:07:06,  5.30s/it]
 40%|████      | 90374/225000 [165:17:41<175:08:06,  4.68s/it]
 40%|████      | 90375/225000 [165:17:44<158:08:35,  4.23s/it]
 40%|████      | 90376/225000 [165:17:48<149:36:34,  4.00s/it]
 40%|████      | 90377/225000 [165:17:52<149:13:28,  3.99s/it]
 40%|████      | 90378/225000 [165:17:55<140:39:02,  3.76s/it]
 40%|████      | 90379/225000 [165:17:58<131:32:07,  3.52s/it]
 40%|████      | 90380/225000 [165:18:01<127:48:33,  3.42s/it]
 40%|████      | 90381/225000 [165:18:04<123:56:39,  3.31s/it]
 40%|████      | 90382/225000 [165:18:07<121:54:55,  3.26s/it]
 40%|████      | 90383/225000 [165:18:10<118:41:50,  3.17s/it]
 40%|████      | 90384/225000 [165:18:20<189:03:14,  5.06s/it]
 40%|████      | 90385/225000 [165:18:27<215:06:39,  5.75s/it]
 40%|████      | 90386/225000 [165:18:30<180:36:51,  4.83s/it]
 40%|████      | 90387/225000 [165:18:34<175:13:49,  4.69s/it]
 40%|████      | 90388/225000 [165:18:39<175:54:20,  4.70s/it]
 40%|████      | 90389/225000 [165:18:43<167:29:56,  4.48s/it]
 40%|████      | 90390/225000 [165:18:50<195:55:49,  5.24s/it]
 40%|████      | 90391/225000 [165:18:53<171:40:48,  4.59s/it]
 40%|████      | 90392/225000 [165:18:56<153:28:01,  4.10s/it]
 40%|████      | 90393/225000 [165:18:59<141:07:49,  3.77s/it]
 40%|████      | 90394/225000 [165:19:02<131:38:32,  3.52s/it]
 40%|████      | 90395/225000 [165:19:05<128:12:09,  3.43s/it]
 40%|████      | 90396/225000 [165:19:11<157:37:32,  4.22s/it]
 40%|████      | 90397/225000 [165:19:17<175:45:37,  4.70s/it]
 40%|████      | 90398/225000 [165:19:20<154:18:25,  4.13s/it]
 40%|████      | 90399/225000 [165:19:23<140:10:50,  3.75s/it]
 40%|████      | 90400/225000 [165:19:26<133:08:24,  3.56s/it]
                                                              
{'loss': '0.3009', 'grad_norm': '4.369', 'learning_rate': '3.466e-05', 'epoch': '0.4018', 'num_input_tokens_seen': 270897408, 'train_runtime': '5.952e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90400/225000 [165:19:26<133:08:24,  3.56s/it]
 40%|████      | 90401/225000 [165:19:29<123:36:29,  3.31s/it]
 40%|████      | 90402/225000 [165:19:37<181:56:03,  4.87s/it]
 40%|████      | 90403/225000 [165:19:40<158:42:29,  4.24s/it]
 40%|████      | 90404/225000 [165:19:43<143:09:47,  3.83s/it]
 40%|████      | 90405/225000 [165:19:46<141:44:52,  3.79s/it]
 40%|████      | 90406/225000 [165:19:50<135:04:05,  3.61s/it]
 40%|████      | 90407/225000 [165:19:53<128:43:00,  3.44s/it]
 40%|████      | 90408/225000 [165:20:00<172:12:50,  4.61s/it]
 40%|████      | 90409/225000 [165:20:04<170:00:40,  4.55s/it]
 40%|████      | 90410/225000 [165:20:07<146:37:59,  3.92s/it]
 40%|████      | 90411/225000 [165:20:10<133:07:41,  3.56s/it]
 40%|████      | 90412/225000 [165:20:13<135:12:57,  3.62s/it]
 40%|████      | 90413/225000 [165:20:17<132:45:42,  3.55s/it]
 40%|████      | 90414/225000 [165:20:20<129:54:10,  3.47s/it]
 40%|████      | 90415/225000 [165:20:24<137:43:23,  3.68s/it]
 40%|████      | 90416/225000 [165:20:28<134:02:46,  3.59s/it]
 40%|████      | 90417/225000 [165:20:31<131:30:23,  3.52s/it]
 40%|████      | 90418/225000 [165:20:35<139:37:04,  3.73s/it]
 40%|████      | 90419/225000 [165:20:39<138:29:54,  3.70s/it]
 40%|████      | 90420/225000 [165:20:42<134:25:41,  3.60s/it]
 40%|████      | 90421/225000 [165:20:50<179:42:36,  4.81s/it]
 40%|████      | 90422/225000 [165:20:56<193:57:36,  5.19s/it]
 40%|████      | 90423/225000 [165:20:58<165:27:42,  4.43s/it]
 40%|████      | 90424/225000 [165:21:01<143:05:19,  3.83s/it]
 40%|████      | 90425/225000 [165:21:04<137:53:09,  3.69s/it]
 40%|████      | 90426/225000 [165:21:08<134:47:04,  3.61s/it]
 40%|████      | 90427/225000 [165:21:11<133:05:27,  3.56s/it]
 40%|████      | 90428/225000 [165:21:15<135:07:36,  3.61s/it]
 40%|████      | 90429/225000 [165:21:18<129:45:28,  3.47s/it]
 40%|████      | 90430/225000 [165:21:21<129:21:52,  3.46s/it]
 40%|████      | 90431/225000 [165:21:25<132:19:21,  3.54s/it]
 40%|████      | 90432/225000 [165:21:28<128:08:03,  3.43s/it]
 40%|████      | 90433/225000 [165:21:33<138:30:28,  3.71s/it]
 40%|████      | 90434/225000 [165:21:42<197:33:53,  5.29s/it]
 40%|████      | 90435/225000 [165:21:47<195:29:54,  5.23s/it]
 40%|████      | 90436/225000 [165:21:49<163:47:08,  4.38s/it]
 40%|████      | 90437/225000 [165:21:56<189:21:19,  5.07s/it]
 40%|████      | 90438/225000 [165:21:59<167:52:25,  4.49s/it]
 40%|████      | 90439/225000 [165:22:02<150:43:30,  4.03s/it]
 40%|████      | 90440/225000 [165:22:05<139:24:42,  3.73s/it]
 40%|████      | 90441/225000 [165:22:08<130:36:05,  3.49s/it]
 40%|████      | 90442/225000 [165:22:11<123:34:39,  3.31s/it]
 40%|████      | 90443/225000 [165:22:16<146:28:57,  3.92s/it]
 40%|████      | 90444/225000 [165:22:23<174:12:34,  4.66s/it]
 40%|████      | 90445/225000 [165:22:25<148:38:17,  3.98s/it]
 40%|████      | 90446/225000 [165:22:32<187:22:42,  5.01s/it]
 40%|████      | 90447/225000 [165:22:35<166:23:01,  4.45s/it]
 40%|████      | 90448/225000 [165:22:39<156:46:50,  4.19s/it]
 40%|████      | 90449/225000 [165:22:42<144:26:03,  3.86s/it]
 40%|████      | 90450/225000 [165:22:46<143:50:36,  3.85s/it]
                                                              
{'loss': '0.3118', 'grad_norm': '2.596', 'learning_rate': '3.464e-05', 'epoch': '0.402', 'num_input_tokens_seen': 270983152, 'train_runtime': '5.954e+05', 'train_tokens_per_second': '455.2'}

 40%|████      | 90450/225000 [165:22:46<143:50:36,  3.85s/it]
 40%|████      | 90451/225000 [165:22:49<138:05:19,  3.69s/it]
 40%|████      | 90452/225000 [165:22:52<132:00:23,  3.53s/it]
 40%|████      | 90453/225000 [165:22:56<134:15:31,  3.59s/it]
 40%|████      | 90454/225000 [165:23:00<134:13:22,  3.59s/it]
 40%|████      | 90455/225000 [165:23:03<128:59:44,  3.45s/it]
 40%|████      | 90456/225000 [165:23:07<138:33:54,  3.71s/it]
 40%|████      | 90457/225000 [165:23:11<134:41:56,  3.60s/it]
 40%|████      | 90458/225000 [165:23:14<130:18:35,  3.49s/it]
 40%|████      | 90459/225000 [165:23:21<167:43:57,  4.49s/it]
 40%|████      | 90460/225000 [165:23:27<192:43:42,  5.16s/it]
 40%|████      | 90461/225000 [165:23:30<163:38:28,  4.38s/it]
 40%|████      | 90462/225000 [165:23:32<142:33:27,  3.81s/it]
 40%|████      | 90463/225000 [165:23:40<182:29:07,  4.88s/it]
 40%|████      | 90464/225000 [165:23:42<155:26:43,  4.16s/it]
 40%|████      | 90465/225000 [165:23:45<141:45:28,  3.79s/it]
 40%|████      | 90466/225000 [165:23:49<138:10:46,  3.70s/it]
 40%|████      | 90467/225000 [165:23:52<134:19:04,  3.59s/it]
 40%|████      | 90468/225000 [165:23:55<131:24:49,  3.52s/it]
 40%|████      | 90469/225000 [165:23:59<128:04:22,  3.43s/it]
 40%|████      | 90470/225000 [165:24:02<128:12:28,  3.43s/it]
 40%|████      | 90471/225000 [165:24:05<125:23:34,  3.36s/it]
 40%|████      | 90472/225000 [165:24:12<160:15:11,  4.29s/it]
 40%|████      | 90473/225000 [165:24:15<150:57:41,  4.04s/it]
 40%|████      | 90474/225000 [165:24:18<141:18:54,  3.78s/it]
 40%|████      | 90475/225000 [165:24:22<136:41:50,  3.66s/it]
 40%|████      | 90476/225000 [165:24:25<134:21:59,  3.60s/it]
 40%|████      | 90477/225000 [165:24:29<132:39:14,  3.55s/it]
 40%|████      | 90478/225000 [165:24:36<170:53:58,  4.57s/it]
 40%|████      | 90479/225000 [165:24:43<208:19:35,  5.58s/it]
 40%|████      | 90480/225000 [165:24:46<172:52:29,  4.63s/it]
 40%|████      | 90481/225000 [165:24:52<186:28:11,  4.99s/it]
 40%|████      | 90482/225000 [165:24:55<168:47:56,  4.52s/it]
 40%|████      | 90483/225000 [165:24:57<143:31:45,  3.84s/it]
 40%|████      | 90484/225000 [165:25:03<161:21:22,  4.32s/it]
 40%|████      | 90485/225000 [165:25:08<165:49:51,  4.44s/it]
 40%|████      | 90486/225000 [165:25:11<155:37:25,  4.16s/it]
 40%|████      | 90487/225000 [165:25:17<172:05:12,  4.61s/it]
 40%|████      | 90488/225000 [165:25:22<179:47:28,  4.81s/it]
 40%|████      | 90489/225000 [165:25:24<152:01:57,  4.07s/it]
 40%|████      | 90490/225000 [165:25:27<132:46:18,  3.55s/it]
 40%|████      | 90491/225000 [165:25:30<134:54:31,  3.61s/it]
 40%|████      | 90492/225000 [165:25:38<174:46:24,  4.68s/it]
 40%|████      | 90493/225000 [165:25:41<155:50:30,  4.17s/it]
 40%|████      | 90494/225000 [165:25:43<139:59:03,  3.75s/it]
 40%|████      | 90495/225000 [165:25:46<127:51:31,  3.42s/it]
 40%|████      | 90496/225000 [165:25:49<122:17:31,  3.27s/it]
 40%|████      | 90497/225000 [165:25:58<186:32:52,  4.99s/it]
 40%|████      | 90498/225000 [165:26:01<168:57:43,  4.52s/it]
 40%|████      | 90499/225000 [165:26:04<145:51:08,  3.90s/it]
 40%|████      | 90500/225000 [165:26:07<135:43:57,  3.63s/it]
                                                              
{'loss': '0.3421', 'grad_norm': '4.391', 'learning_rate': '3.462e-05', 'epoch': '0.4022', 'num_input_tokens_seen': 271069472, 'train_runtime': '5.956e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90500/225000 [165:26:07<135:43:57,  3.63s/it]
 40%|████      | 90501/225000 [165:26:09<123:35:52,  3.31s/it]
 40%|████      | 90502/225000 [165:26:13<125:43:44,  3.37s/it]
 40%|████      | 90503/225000 [165:26:16<126:18:57,  3.38s/it]
 40%|████      | 90504/225000 [165:26:21<146:23:30,  3.92s/it]
 40%|████      | 90505/225000 [165:26:29<188:47:45,  5.05s/it]
 40%|████      | 90506/225000 [165:26:32<160:49:55,  4.30s/it]
 40%|████      | 90507/225000 [165:26:34<140:18:42,  3.76s/it]
 40%|████      | 90508/225000 [165:26:37<127:12:12,  3.40s/it]
 40%|████      | 90509/225000 [165:26:46<196:59:55,  5.27s/it]
 40%|████      | 90510/225000 [165:26:49<168:11:56,  4.50s/it]
 40%|████      | 90511/225000 [165:26:52<148:53:46,  3.99s/it]
 40%|████      | 90512/225000 [165:26:55<135:59:04,  3.64s/it]
 40%|████      | 90513/225000 [165:26:57<126:34:43,  3.39s/it]
 40%|████      | 90514/225000 [165:27:04<167:03:49,  4.47s/it]
 40%|████      | 90515/225000 [165:27:09<171:13:24,  4.58s/it]
 40%|████      | 90516/225000 [165:27:12<153:48:17,  4.12s/it]
 40%|████      | 90517/225000 [165:27:15<142:39:14,  3.82s/it]
 40%|████      | 90518/225000 [165:27:19<133:47:08,  3.58s/it]
 40%|████      | 90519/225000 [165:27:22<131:38:30,  3.52s/it]
 40%|████      | 90520/225000 [165:27:26<135:51:58,  3.64s/it]
 40%|████      | 90521/225000 [165:27:29<129:35:34,  3.47s/it]
 40%|████      | 90522/225000 [165:27:33<132:18:07,  3.54s/it]
 40%|████      | 90523/225000 [165:27:36<132:59:16,  3.56s/it]
 40%|████      | 90524/225000 [165:27:39<128:29:50,  3.44s/it]
 40%|████      | 90525/225000 [165:27:46<164:14:28,  4.40s/it]
 40%|████      | 90526/225000 [165:27:54<199:30:44,  5.34s/it]
 40%|████      | 90527/225000 [165:27:56<170:02:14,  4.55s/it]
 40%|████      | 90528/225000 [165:27:59<150:19:58,  4.02s/it]
 40%|████      | 90529/225000 [165:28:02<135:31:57,  3.63s/it]
 40%|████      | 90530/225000 [165:28:05<128:37:24,  3.44s/it]
 40%|████      | 90531/225000 [165:28:08<129:53:57,  3.48s/it]
 40%|████      | 90532/225000 [165:28:12<131:51:55,  3.53s/it]
 40%|████      | 90533/225000 [165:28:21<192:51:16,  5.16s/it]
 40%|████      | 90534/225000 [165:28:26<186:54:10,  5.00s/it]
 40%|████      | 90535/225000 [165:28:28<160:13:49,  4.29s/it]
 40%|████      | 90536/225000 [165:28:31<139:52:07,  3.74s/it]
 40%|████      | 90537/225000 [165:28:35<147:21:06,  3.95s/it]
 40%|████      | 90538/225000 [165:28:43<187:09:02,  5.01s/it]
 40%|████      | 90539/225000 [165:28:45<162:29:08,  4.35s/it]
 40%|████      | 90540/225000 [165:28:48<146:39:13,  3.93s/it]
 40%|████      | 90541/225000 [165:28:51<130:15:58,  3.49s/it]
 40%|████      | 90542/225000 [165:28:54<125:37:00,  3.36s/it]
 40%|████      | 90543/225000 [165:28:57<123:04:28,  3.30s/it]
 40%|████      | 90544/225000 [165:29:00<123:01:08,  3.29s/it]
 40%|████      | 90545/225000 [165:29:04<129:56:07,  3.48s/it]
 40%|████      | 90546/225000 [165:29:12<175:42:10,  4.70s/it]
 40%|████      | 90547/225000 [165:29:19<203:39:29,  5.45s/it]
 40%|████      | 90548/225000 [165:29:22<175:51:31,  4.71s/it]
 40%|████      | 90549/225000 [165:29:25<158:49:03,  4.25s/it]
 40%|████      | 90550/225000 [165:29:28<142:59:19,  3.83s/it]
                                                              
{'loss': '0.312', 'grad_norm': '5.388', 'learning_rate': '3.461e-05', 'epoch': '0.4024', 'num_input_tokens_seen': 271156760, 'train_runtime': '5.958e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90550/225000 [165:29:28<142:59:19,  3.83s/it]
 40%|████      | 90551/225000 [165:29:31<134:31:42,  3.60s/it]
 40%|████      | 90552/225000 [165:29:34<129:16:38,  3.46s/it]
 40%|████      | 90553/225000 [165:29:40<153:52:59,  4.12s/it]
 40%|████      | 90554/225000 [165:29:48<200:31:00,  5.37s/it]
 40%|████      | 90555/225000 [165:29:50<166:32:57,  4.46s/it]
 40%|████      | 90556/225000 [165:29:53<144:43:42,  3.88s/it]
 40%|████      | 90557/225000 [165:29:58<153:25:33,  4.11s/it]
 40%|████      | 90558/225000 [165:30:02<159:33:18,  4.27s/it]
 40%|████      | 90559/225000 [165:30:05<137:43:52,  3.69s/it]
 40%|████      | 90560/225000 [165:30:08<131:02:47,  3.51s/it]
 40%|████      | 90561/225000 [165:30:11<127:25:44,  3.41s/it]
 40%|████      | 90562/225000 [165:30:14<123:32:07,  3.31s/it]
 40%|████      | 90563/225000 [165:30:17<121:15:36,  3.25s/it]
 40%|████      | 90564/225000 [165:30:20<118:21:10,  3.17s/it]
 40%|████      | 90565/225000 [165:30:25<139:54:02,  3.75s/it]
 40%|████      | 90566/225000 [165:30:33<189:21:36,  5.07s/it]
 40%|████      | 90567/225000 [165:30:37<170:04:20,  4.55s/it]
 40%|████      | 90568/225000 [165:30:39<145:06:12,  3.89s/it]
 40%|████      | 90569/225000 [165:30:42<132:08:22,  3.54s/it]
 40%|████      | 90570/225000 [165:30:45<131:10:14,  3.51s/it]
 40%|████      | 90571/225000 [165:30:48<127:35:20,  3.42s/it]
 40%|████      | 90572/225000 [165:30:52<125:27:52,  3.36s/it]
 40%|████      | 90573/225000 [165:30:59<165:44:53,  4.44s/it]
 40%|████      | 90574/225000 [165:31:02<154:11:48,  4.13s/it]
 40%|████      | 90575/225000 [165:31:05<138:24:18,  3.71s/it]
 40%|████      | 90576/225000 [165:31:08<131:47:18,  3.53s/it]
 40%|████      | 90577/225000 [165:31:11<125:12:29,  3.35s/it]
 40%|████      | 90578/225000 [165:31:18<173:52:41,  4.66s/it]
 40%|████      | 90579/225000 [165:31:25<198:02:08,  5.30s/it]
 40%|████      | 90580/225000 [165:31:28<170:26:31,  4.56s/it]
 40%|████      | 90581/225000 [165:31:32<161:11:34,  4.32s/it]
 40%|████      | 90582/225000 [165:31:35<147:55:30,  3.96s/it]
 40%|████      | 90583/225000 [165:31:38<138:58:43,  3.72s/it]
 40%|████      | 90584/225000 [165:31:41<134:26:23,  3.60s/it]
 40%|████      | 90585/225000 [165:31:45<130:49:34,  3.50s/it]
 40%|████      | 90586/225000 [165:31:48<126:37:54,  3.39s/it]
 40%|████      | 90587/225000 [165:31:51<129:05:43,  3.46s/it]
 40%|████      | 90588/225000 [165:31:55<128:07:40,  3.43s/it]
 40%|████      | 90589/225000 [165:31:58<129:58:12,  3.48s/it]
 40%|████      | 90590/225000 [165:32:07<192:39:26,  5.16s/it]
 40%|████      | 90591/225000 [165:32:12<184:08:16,  4.93s/it]
 40%|████      | 90592/225000 [165:32:16<175:35:41,  4.70s/it]
 40%|████      | 90593/225000 [165:32:21<176:46:27,  4.73s/it]
 40%|████      | 90594/225000 [165:32:24<161:17:20,  4.32s/it]
 40%|████      | 90595/225000 [165:32:27<147:51:44,  3.96s/it]
 40%|████      | 90596/225000 [165:32:30<134:47:15,  3.61s/it]
 40%|████      | 90597/225000 [165:32:33<125:29:35,  3.36s/it]
 40%|████      | 90598/225000 [165:32:36<122:48:11,  3.29s/it]
 40%|████      | 90599/225000 [165:32:41<138:18:09,  3.70s/it]
 40%|████      | 90600/225000 [165:32:49<190:25:28,  5.10s/it]
                                                              
{'loss': '0.3071', 'grad_norm': '5.49', 'learning_rate': '3.459e-05', 'epoch': '0.4027', 'num_input_tokens_seen': 271243056, 'train_runtime': '5.96e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90600/225000 [165:32:49<190:25:28,  5.10s/it]
 40%|████      | 90601/225000 [165:32:52<161:31:13,  4.33s/it]
 40%|████      | 90602/225000 [165:32:54<139:36:48,  3.74s/it]
 40%|████      | 90603/225000 [165:33:00<168:32:16,  4.51s/it]
 40%|████      | 90604/225000 [165:33:03<145:33:19,  3.90s/it]
 40%|████      | 90605/225000 [165:33:06<139:04:16,  3.73s/it]
 40%|████      | 90606/225000 [165:33:14<187:07:40,  5.01s/it]
 40%|████      | 90607/225000 [165:33:17<163:17:13,  4.37s/it]
 40%|████      | 90608/225000 [165:33:21<156:20:50,  4.19s/it]
 40%|████      | 90609/225000 [165:33:24<147:35:14,  3.95s/it]
 40%|████      | 90610/225000 [165:33:28<146:16:27,  3.92s/it]
 40%|████      | 90611/225000 [165:33:31<140:43:41,  3.77s/it]
 40%|████      | 90612/225000 [165:33:35<137:58:29,  3.70s/it]
 40%|████      | 90613/225000 [165:33:38<131:54:09,  3.53s/it]
 40%|████      | 90614/225000 [165:33:41<130:02:36,  3.48s/it]
 40%|████      | 90615/225000 [165:33:45<129:31:34,  3.47s/it]
 40%|████      | 90616/225000 [165:33:48<129:38:22,  3.47s/it]
 40%|████      | 90617/225000 [165:33:52<134:44:24,  3.61s/it]
 40%|████      | 90618/225000 [165:33:56<135:55:50,  3.64s/it]
 40%|████      | 90619/225000 [165:34:03<171:00:22,  4.58s/it]
 40%|████      | 90620/225000 [165:34:10<197:22:38,  5.29s/it]
 40%|████      | 90621/225000 [165:34:12<165:10:44,  4.43s/it]
 40%|████      | 90622/225000 [165:34:14<142:00:32,  3.80s/it]
 40%|████      | 90623/225000 [165:34:18<140:45:46,  3.77s/it]
 40%|████      | 90624/225000 [165:34:21<134:06:06,  3.59s/it]
 40%|████      | 90625/225000 [165:34:25<133:28:32,  3.58s/it]
 40%|████      | 90626/225000 [165:34:28<133:05:29,  3.57s/it]
 40%|████      | 90627/225000 [165:34:32<132:55:12,  3.56s/it]
 40%|████      | 90628/225000 [165:34:36<134:03:06,  3.59s/it]
 40%|████      | 90629/225000 [165:34:39<129:59:32,  3.48s/it]
 40%|████      | 90630/225000 [165:34:42<128:15:31,  3.44s/it]
 40%|████      | 90631/225000 [165:34:45<126:33:36,  3.39s/it]
 40%|████      | 90632/225000 [165:34:49<127:43:17,  3.42s/it]
 40%|████      | 90633/225000 [165:34:53<133:42:44,  3.58s/it]
 40%|████      | 90634/225000 [165:35:01<185:37:58,  4.97s/it]
 40%|████      | 90635/225000 [165:35:09<222:57:49,  5.97s/it]
 40%|████      | 90636/225000 [165:35:12<189:19:17,  5.07s/it]
 40%|████      | 90637/225000 [165:35:15<163:46:51,  4.39s/it]
 40%|████      | 90638/225000 [165:35:18<149:00:23,  3.99s/it]
 40%|████      | 90639/225000 [165:35:21<135:54:01,  3.64s/it]
 40%|████      | 90640/225000 [165:35:24<126:57:37,  3.40s/it]
 40%|████      | 90641/225000 [165:35:27<127:37:37,  3.42s/it]
 40%|████      | 90642/225000 [165:35:31<127:00:38,  3.40s/it]
 40%|████      | 90643/225000 [165:35:34<126:30:57,  3.39s/it]
 40%|████      | 90644/225000 [165:35:38<130:15:28,  3.49s/it]
 40%|████      | 90645/225000 [165:35:45<168:21:25,  4.51s/it]
 40%|████      | 90646/225000 [165:35:51<192:32:54,  5.16s/it]
 40%|████      | 90647/225000 [165:35:54<168:01:41,  4.50s/it]
 40%|████      | 90648/225000 [165:35:57<151:08:08,  4.05s/it]
 40%|████      | 90649/225000 [165:36:01<145:59:47,  3.91s/it]
 40%|████      | 90650/225000 [165:36:07<166:27:50,  4.46s/it]
                                                              
{'loss': '0.3171', 'grad_norm': '5.943', 'learning_rate': '3.457e-05', 'epoch': '0.4029', 'num_input_tokens_seen': 271329320, 'train_runtime': '5.962e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90650/225000 [165:36:07<166:27:50,  4.46s/it]
 40%|████      | 90651/225000 [165:36:13<191:27:44,  5.13s/it]
 40%|████      | 90652/225000 [165:36:16<162:06:14,  4.34s/it]
 40%|████      | 90653/225000 [165:36:19<142:54:49,  3.83s/it]
 40%|████      | 90654/225000 [165:36:21<132:10:22,  3.54s/it]
 40%|████      | 90655/225000 [165:36:29<173:10:55,  4.64s/it]
 40%|████      | 90656/225000 [165:36:32<161:37:03,  4.33s/it]
 40%|████      | 90657/225000 [165:36:35<142:46:23,  3.83s/it]
 40%|████      | 90658/225000 [165:36:38<136:23:18,  3.65s/it]
 40%|████      | 90659/225000 [165:36:41<129:23:15,  3.47s/it]
 40%|████      | 90660/225000 [165:36:44<124:23:59,  3.33s/it]
 40%|████      | 90661/225000 [165:36:48<125:22:28,  3.36s/it]
 40%|████      | 90662/225000 [165:36:51<127:41:07,  3.42s/it]
 40%|████      | 90663/225000 [165:36:54<126:41:09,  3.39s/it]
 40%|████      | 90664/225000 [165:36:58<125:27:31,  3.36s/it]
 40%|████      | 90665/225000 [165:37:01<120:20:05,  3.22s/it]
 40%|████      | 90666/225000 [165:37:08<166:10:28,  4.45s/it]
 40%|████      | 90667/225000 [165:37:15<194:33:05,  5.21s/it]
 40%|████      | 90668/225000 [165:37:20<186:51:15,  5.01s/it]
 40%|████      | 90669/225000 [165:37:23<168:13:19,  4.51s/it]
 40%|████      | 90670/225000 [165:37:27<158:39:13,  4.25s/it]
 40%|████      | 90671/225000 [165:37:30<144:59:13,  3.89s/it]
 40%|████      | 90672/225000 [165:37:33<135:30:33,  3.63s/it]
 40%|████      | 90673/225000 [165:37:36<129:18:09,  3.47s/it]
 40%|████      | 90674/225000 [165:37:39<127:08:05,  3.41s/it]
 40%|████      | 90675/225000 [165:37:42<126:06:07,  3.38s/it]
 40%|████      | 90676/225000 [165:37:46<126:28:00,  3.39s/it]
 40%|████      | 90677/225000 [165:37:49<126:43:15,  3.40s/it]
 40%|████      | 90678/225000 [165:37:52<123:05:19,  3.30s/it]
 40%|████      | 90679/225000 [165:38:02<200:11:56,  5.37s/it]
 40%|████      | 90680/225000 [165:38:07<194:57:13,  5.23s/it]
 40%|████      | 90681/225000 [165:38:10<172:55:58,  4.63s/it]
 40%|████      | 90682/225000 [165:38:14<160:50:23,  4.31s/it]
 40%|████      | 90683/225000 [165:38:17<146:07:38,  3.92s/it]
 40%|████      | 90684/225000 [165:38:21<145:07:18,  3.89s/it]
 40%|████      | 90685/225000 [165:38:24<138:44:05,  3.72s/it]
 40%|████      | 90686/225000 [165:38:27<129:32:20,  3.47s/it]
 40%|████      | 90687/225000 [165:38:30<126:47:41,  3.40s/it]
 40%|████      | 90688/225000 [165:38:34<125:58:15,  3.38s/it]
 40%|████      | 90689/225000 [165:38:39<144:48:13,  3.88s/it]
 40%|████      | 90690/225000 [165:38:48<203:02:54,  5.44s/it]
 40%|████      | 90691/225000 [165:38:53<196:12:02,  5.26s/it]
 40%|████      | 90692/225000 [165:38:55<168:21:35,  4.51s/it]
 40%|████      | 90693/225000 [165:38:58<149:45:12,  4.01s/it]
 40%|████      | 90694/225000 [165:39:01<134:15:44,  3.60s/it]
 40%|████      | 90695/225000 [165:39:07<161:56:48,  4.34s/it]
 40%|████      | 90696/225000 [165:39:13<180:45:49,  4.85s/it]
 40%|████      | 90697/225000 [165:39:15<154:23:11,  4.14s/it]
 40%|████      | 90698/225000 [165:39:18<135:08:29,  3.62s/it]
 40%|████      | 90699/225000 [165:39:21<126:26:32,  3.39s/it]
 40%|████      | 90700/225000 [165:39:24<126:29:34,  3.39s/it]
                                                              
{'loss': '0.2917', 'grad_norm': '4.171', 'learning_rate': '3.456e-05', 'epoch': '0.4031', 'num_input_tokens_seen': 271415296, 'train_runtime': '5.964e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90700/225000 [165:39:24<126:29:34,  3.39s/it]
 40%|████      | 90701/225000 [165:39:33<186:41:25,  5.00s/it]
 40%|████      | 90702/225000 [165:39:36<166:15:16,  4.46s/it]
 40%|████      | 90703/225000 [165:39:39<154:48:43,  4.15s/it]
 40%|████      | 90704/225000 [165:39:42<139:01:20,  3.73s/it]
 40%|████      | 90705/225000 [165:39:45<128:03:23,  3.43s/it]
 40%|████      | 90706/225000 [165:39:48<123:52:19,  3.32s/it]
 40%|████      | 90707/225000 [165:39:54<155:35:53,  4.17s/it]
 40%|████      | 90708/225000 [165:39:59<164:58:05,  4.42s/it]
 40%|████      | 90709/225000 [165:40:02<150:44:45,  4.04s/it]
 40%|████      | 90710/225000 [165:40:05<137:22:18,  3.68s/it]
 40%|████      | 90711/225000 [165:40:08<127:50:50,  3.43s/it]
 40%|████      | 90712/225000 [165:40:12<133:12:45,  3.57s/it]
 40%|████      | 90713/225000 [165:40:15<126:12:00,  3.38s/it]
 40%|████      | 90714/225000 [165:40:24<186:53:21,  5.01s/it]
 40%|████      | 90715/225000 [165:40:28<183:22:36,  4.92s/it]
 40%|████      | 90716/225000 [165:40:31<157:50:16,  4.23s/it]
 40%|████      | 90717/225000 [165:40:34<141:30:15,  3.79s/it]
 40%|████      | 90718/225000 [165:40:39<153:47:06,  4.12s/it]
 40%|████      | 90719/225000 [165:40:44<172:24:19,  4.62s/it]
 40%|████      | 90720/225000 [165:40:47<148:25:58,  3.98s/it]
 40%|████      | 90721/225000 [165:40:50<140:38:52,  3.77s/it]
 40%|████      | 90722/225000 [165:40:54<136:34:40,  3.66s/it]
 40%|████      | 90723/225000 [165:40:57<130:59:52,  3.51s/it]
 40%|████      | 90724/225000 [165:41:00<126:21:16,  3.39s/it]
 40%|████      | 90725/225000 [165:41:03<125:57:11,  3.38s/it]
 40%|████      | 90726/225000 [165:41:07<127:50:00,  3.43s/it]
 40%|████      | 90727/225000 [165:41:11<131:58:24,  3.54s/it]
 40%|████      | 90728/225000 [165:41:14<128:26:03,  3.44s/it]
 40%|████      | 90729/225000 [165:41:17<129:22:02,  3.47s/it]
 40%|████      | 90730/225000 [165:41:21<126:07:35,  3.38s/it]
 40%|████      | 90731/225000 [165:41:27<166:31:41,  4.46s/it]
 40%|████      | 90732/225000 [165:41:37<220:02:08,  5.90s/it]
 40%|████      | 90733/225000 [165:41:39<182:32:33,  4.89s/it]
 40%|████      | 90734/225000 [165:41:42<156:08:04,  4.19s/it]
 40%|████      | 90735/225000 [165:41:44<138:43:04,  3.72s/it]
 40%|████      | 90736/225000 [165:41:47<128:41:20,  3.45s/it]
 40%|████      | 90737/225000 [165:41:51<126:21:21,  3.39s/it]
 40%|████      | 90738/225000 [165:41:54<122:25:46,  3.28s/it]
 40%|████      | 90739/225000 [165:41:57<123:57:49,  3.32s/it]
 40%|████      | 90740/225000 [165:42:00<121:26:31,  3.26s/it]
 40%|████      | 90741/225000 [165:42:03<118:32:47,  3.18s/it]
 40%|████      | 90742/225000 [165:42:11<167:50:25,  4.50s/it]
 40%|████      | 90743/225000 [165:42:19<212:40:24,  5.70s/it]
 40%|████      | 90744/225000 [165:42:21<174:57:42,  4.69s/it]
 40%|████      | 90745/225000 [165:42:24<150:09:28,  4.03s/it]
 40%|████      | 90746/225000 [165:42:27<135:50:24,  3.64s/it]
 40%|████      | 90747/225000 [165:42:29<123:19:33,  3.31s/it]
 40%|████      | 90748/225000 [165:42:32<120:38:44,  3.24s/it]
 40%|████      | 90749/225000 [165:42:37<138:11:00,  3.71s/it]
 40%|████      | 90750/225000 [165:42:47<207:33:46,  5.57s/it]
                                                              
{'loss': '0.2968', 'grad_norm': '3.698', 'learning_rate': '3.454e-05', 'epoch': '0.4033', 'num_input_tokens_seen': 271501856, 'train_runtime': '5.966e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90750/225000 [165:42:47<207:33:46,  5.57s/it]
 40%|████      | 90751/225000 [165:42:50<174:47:17,  4.69s/it]
 40%|████      | 90752/225000 [165:42:52<152:51:33,  4.10s/it]
 40%|████      | 90753/225000 [165:42:55<138:39:26,  3.72s/it]
 40%|████      | 90754/225000 [165:42:58<129:55:05,  3.48s/it]
 40%|████      | 90755/225000 [165:43:01<124:11:16,  3.33s/it]
 40%|████      | 90756/225000 [165:43:10<184:31:36,  4.95s/it]
 40%|████      | 90757/225000 [165:43:14<174:33:47,  4.68s/it]
 40%|████      | 90758/225000 [165:43:17<157:36:40,  4.23s/it]
 40%|████      | 90759/225000 [165:43:21<149:41:21,  4.01s/it]
 40%|████      | 90760/225000 [165:43:23<137:06:37,  3.68s/it]
 40%|████      | 90761/225000 [165:43:26<127:59:53,  3.43s/it]
 40%|████      | 90762/225000 [165:43:29<124:08:27,  3.33s/it]
 40%|████      | 90763/225000 [165:43:38<177:10:12,  4.75s/it]
 40%|████      | 90764/225000 [165:43:40<152:00:24,  4.08s/it]
 40%|████      | 90765/225000 [165:43:43<134:55:55,  3.62s/it]
 40%|████      | 90766/225000 [165:43:45<124:56:56,  3.35s/it]
 40%|████      | 90767/225000 [165:43:54<180:10:06,  4.83s/it]
 40%|████      | 90768/225000 [165:43:56<157:15:03,  4.22s/it]
 40%|████      | 90769/225000 [165:43:59<141:58:00,  3.81s/it]
 40%|████      | 90770/225000 [165:44:02<128:29:35,  3.45s/it]
 40%|████      | 90771/225000 [165:44:10<180:01:33,  4.83s/it]
 40%|████      | 90772/225000 [165:44:12<154:30:17,  4.14s/it]
 40%|████      | 90773/225000 [165:44:15<134:21:23,  3.60s/it]
 40%|████      | 90774/225000 [165:44:21<167:48:01,  4.50s/it]
 40%|████      | 90775/225000 [165:44:25<161:13:32,  4.32s/it]
 40%|████      | 90776/225000 [165:44:28<144:31:20,  3.88s/it]
 40%|████      | 90777/225000 [165:44:34<170:29:53,  4.57s/it]
 40%|████      | 90778/225000 [165:44:39<171:16:46,  4.59s/it]
 40%|████      | 90779/225000 [165:44:42<152:11:58,  4.08s/it]
 40%|████      | 90780/225000 [165:44:45<139:10:48,  3.73s/it]
 40%|████      | 90781/225000 [165:44:48<133:12:55,  3.57s/it]
 40%|████      | 90782/225000 [165:44:51<125:13:27,  3.36s/it]
 40%|████      | 90783/225000 [165:44:54<125:06:05,  3.36s/it]
 40%|████      | 90784/225000 [165:44:58<126:40:07,  3.40s/it]
 40%|████      | 90785/225000 [165:45:01<125:03:13,  3.35s/it]
 40%|████      | 90786/225000 [165:45:04<125:21:22,  3.36s/it]
 40%|████      | 90787/225000 [165:45:14<197:29:22,  5.30s/it]
 40%|████      | 90788/225000 [165:45:18<186:05:38,  4.99s/it]
 40%|████      | 90789/225000 [165:45:24<190:21:40,  5.11s/it]
 40%|████      | 90790/225000 [165:45:28<177:27:03,  4.76s/it]
 40%|████      | 90791/225000 [165:45:31<156:06:22,  4.19s/it]
 40%|████      | 90792/225000 [165:45:33<140:58:27,  3.78s/it]
 40%|████      | 90793/225000 [165:45:37<138:00:38,  3.70s/it]
 40%|████      | 90794/225000 [165:45:41<138:20:45,  3.71s/it]
 40%|████      | 90795/225000 [165:45:44<136:58:52,  3.67s/it]
 40%|████      | 90796/225000 [165:45:48<136:33:17,  3.66s/it]
 40%|████      | 90797/225000 [165:45:51<133:43:22,  3.59s/it]
 40%|████      | 90798/225000 [165:45:55<135:41:40,  3.64s/it]
 40%|████      | 90799/225000 [165:45:59<134:34:19,  3.61s/it]
 40%|████      | 90800/225000 [165:46:02<131:14:45,  3.52s/it]
                                                              
{'loss': '0.299', 'grad_norm': '4.671', 'learning_rate': '3.452e-05', 'epoch': '0.4036', 'num_input_tokens_seen': 271589344, 'train_runtime': '5.968e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90800/225000 [165:46:02<131:14:45,  3.52s/it]
 40%|████      | 90801/225000 [165:46:05<129:12:56,  3.47s/it]
 40%|████      | 90802/225000 [165:46:09<131:02:28,  3.52s/it]
 40%|████      | 90803/225000 [165:46:12<131:40:41,  3.53s/it]
 40%|████      | 90804/225000 [165:46:16<132:22:04,  3.55s/it]
 40%|████      | 90805/225000 [165:46:19<130:46:27,  3.51s/it]
 40%|████      | 90806/225000 [165:46:23<133:13:59,  3.57s/it]
 40%|████      | 90807/225000 [165:46:26<130:23:07,  3.50s/it]
 40%|████      | 90808/225000 [165:46:33<161:56:36,  4.34s/it]
 40%|████      | 90809/225000 [165:46:42<213:03:54,  5.72s/it]
 40%|████      | 90810/225000 [165:46:45<188:55:44,  5.07s/it]
 40%|████      | 90811/225000 [165:46:48<163:00:56,  4.37s/it]
 40%|████      | 90812/225000 [165:46:51<148:12:15,  3.98s/it]
 40%|████      | 90813/225000 [165:46:54<138:39:14,  3.72s/it]
 40%|████      | 90814/225000 [165:46:57<130:21:57,  3.50s/it]
 40%|████      | 90815/225000 [165:47:00<126:25:30,  3.39s/it]
 40%|████      | 90816/225000 [165:47:03<123:00:15,  3.30s/it]
 40%|████      | 90817/225000 [165:47:06<120:32:25,  3.23s/it]
 40%|████      | 90818/225000 [165:47:14<167:53:37,  4.50s/it]
 40%|████      | 90819/225000 [165:47:22<206:37:35,  5.54s/it]
 40%|████      | 90820/225000 [165:47:24<172:41:07,  4.63s/it]
 40%|████      | 90821/225000 [165:47:28<165:13:10,  4.43s/it]
 40%|████      | 90822/225000 [165:47:32<152:17:02,  4.09s/it]
 40%|████      | 90823/225000 [165:47:35<143:52:20,  3.86s/it]
 40%|████      | 90824/225000 [165:47:39<142:17:28,  3.82s/it]
 40%|████      | 90825/225000 [165:47:42<136:48:30,  3.67s/it]
 40%|████      | 90826/225000 [165:47:46<138:58:38,  3.73s/it]
 40%|████      | 90827/225000 [165:47:49<136:53:42,  3.67s/it]
 40%|████      | 90828/225000 [165:47:53<135:46:22,  3.64s/it]
 40%|████      | 90829/225000 [165:47:57<136:26:38,  3.66s/it]
 40%|████      | 90830/225000 [165:48:02<152:59:58,  4.11s/it]
 40%|████      | 90831/225000 [165:48:07<162:15:28,  4.35s/it]
 40%|████      | 90832/225000 [165:48:10<146:27:11,  3.93s/it]
 40%|████      | 90833/225000 [165:48:13<134:51:59,  3.62s/it]
 40%|████      | 90834/225000 [165:48:15<125:34:38,  3.37s/it]
 40%|████      | 90835/225000 [165:48:19<123:34:05,  3.32s/it]
 40%|████      | 90836/225000 [165:48:27<181:37:56,  4.87s/it]
 40%|████      | 90837/225000 [165:48:33<195:31:10,  5.25s/it]
 40%|████      | 90838/225000 [165:48:36<164:58:06,  4.43s/it]
 40%|████      | 90839/225000 [165:48:38<144:42:12,  3.88s/it]
 40%|████      | 90840/225000 [165:48:41<135:33:50,  3.64s/it]
 40%|████      | 90841/225000 [165:48:48<163:01:23,  4.37s/it]
 40%|████      | 90842/225000 [165:48:53<170:50:54,  4.58s/it]
 40%|████      | 90843/225000 [165:48:55<146:53:26,  3.94s/it]
 40%|████      | 90844/225000 [165:48:58<136:34:19,  3.66s/it]
 40%|████      | 90845/225000 [165:49:02<136:33:07,  3.66s/it]
 40%|████      | 90846/225000 [165:49:05<135:18:24,  3.63s/it]
 40%|████      | 90847/225000 [165:49:09<139:06:27,  3.73s/it]
 40%|████      | 90848/225000 [165:49:13<141:00:08,  3.78s/it]
 40%|████      | 90849/225000 [165:49:17<137:25:51,  3.69s/it]
 40%|████      | 90850/225000 [165:49:20<135:11:22,  3.63s/it]
                                                              
{'loss': '0.2886', 'grad_norm': '4.108', 'learning_rate': '3.451e-05', 'epoch': '0.4038', 'num_input_tokens_seen': 271675168, 'train_runtime': '5.97e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90850/225000 [165:49:20<135:11:22,  3.63s/it]
 40%|████      | 90851/225000 [165:49:24<134:59:23,  3.62s/it]
 40%|████      | 90852/225000 [165:49:27<131:20:51,  3.52s/it]
 40%|████      | 90853/225000 [165:49:36<197:34:20,  5.30s/it]
 40%|████      | 90854/225000 [165:49:42<196:56:31,  5.29s/it]
 40%|████      | 90855/225000 [165:49:44<165:26:24,  4.44s/it]
 40%|████      | 90856/225000 [165:49:47<145:30:59,  3.91s/it]
 40%|████      | 90857/225000 [165:49:50<141:31:49,  3.80s/it]
 40%|████      | 90858/225000 [165:49:58<184:09:45,  4.94s/it]
 40%|████      | 90859/225000 [165:50:01<166:25:06,  4.47s/it]
 40%|████      | 90860/225000 [165:50:04<149:51:04,  4.02s/it]
 40%|████      | 90861/225000 [165:50:07<136:13:28,  3.66s/it]
 40%|████      | 90862/225000 [165:50:10<131:02:26,  3.52s/it]
 40%|████      | 90863/225000 [165:50:13<124:57:31,  3.35s/it]
 40%|████      | 90864/225000 [165:50:16<122:13:17,  3.28s/it]
 40%|████      | 90865/225000 [165:50:23<164:08:06,  4.41s/it]
 40%|████      | 90866/225000 [165:50:30<188:01:23,  5.05s/it]
 40%|████      | 90867/225000 [165:50:32<156:53:51,  4.21s/it]
 40%|████      | 90868/225000 [165:50:36<148:04:58,  3.97s/it]
 40%|████      | 90869/225000 [165:50:39<138:44:44,  3.72s/it]
 40%|████      | 90870/225000 [165:50:42<137:58:55,  3.70s/it]
 40%|████      | 90871/225000 [165:50:46<136:08:02,  3.65s/it]
 40%|████      | 90872/225000 [165:50:49<134:12:49,  3.60s/it]
 40%|████      | 90873/225000 [165:50:53<130:18:14,  3.50s/it]
 40%|████      | 90874/225000 [165:50:56<128:19:23,  3.44s/it]
 40%|████      | 90875/225000 [165:51:00<131:17:20,  3.52s/it]
 40%|████      | 90876/225000 [165:51:03<125:39:37,  3.37s/it]
 40%|████      | 90877/225000 [165:51:09<157:05:48,  4.22s/it]
 40%|████      | 90878/225000 [165:51:17<204:50:09,  5.50s/it]
 40%|████      | 90879/225000 [165:51:21<178:48:01,  4.80s/it]
 40%|████      | 90880/225000 [165:51:23<156:11:05,  4.19s/it]
 40%|████      | 90881/225000 [165:51:26<140:40:13,  3.78s/it]
 40%|████      | 90882/225000 [165:51:29<128:20:27,  3.44s/it]
 40%|████      | 90883/225000 [165:51:37<175:09:53,  4.70s/it]
 40%|████      | 90884/225000 [165:51:41<172:47:23,  4.64s/it]
 40%|████      | 90885/225000 [165:51:44<155:52:04,  4.18s/it]
 40%|████      | 90886/225000 [165:51:47<143:26:03,  3.85s/it]
 40%|████      | 90887/225000 [165:51:51<139:14:06,  3.74s/it]
 40%|████      | 90888/225000 [165:51:53<128:15:33,  3.44s/it]
 40%|████      | 90889/225000 [165:51:56<123:59:23,  3.33s/it]
 40%|████      | 90890/225000 [165:51:59<116:55:11,  3.14s/it]
 40%|████      | 90891/225000 [165:52:08<179:31:49,  4.82s/it]
 40%|████      | 90892/225000 [165:52:13<181:40:32,  4.88s/it]
 40%|████      | 90893/225000 [165:52:16<159:35:09,  4.28s/it]
 40%|████      | 90894/225000 [165:52:18<139:53:21,  3.76s/it]
 40%|████      | 90895/225000 [165:52:22<133:26:37,  3.58s/it]
 40%|████      | 90896/225000 [165:52:25<128:43:30,  3.46s/it]
 40%|████      | 90897/225000 [165:52:30<152:21:28,  4.09s/it]
 40%|████      | 90898/225000 [165:52:40<217:57:14,  5.85s/it]
 40%|████      | 90899/225000 [165:52:43<185:29:29,  4.98s/it]
 40%|████      | 90900/225000 [165:52:46<162:22:26,  4.36s/it]
                                                              
{'loss': '0.3053', 'grad_norm': '5.886', 'learning_rate': '3.449e-05', 'epoch': '0.404', 'num_input_tokens_seen': 271762160, 'train_runtime': '5.972e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90900/225000 [165:52:46<162:22:26,  4.36s/it]
 40%|████      | 90901/225000 [165:52:49<143:55:14,  3.86s/it]
 40%|████      | 90902/225000 [165:52:51<128:38:49,  3.45s/it]
 40%|████      | 90903/225000 [165:52:54<123:31:38,  3.32s/it]
 40%|████      | 90904/225000 [165:53:02<177:45:07,  4.77s/it]
 40%|████      | 90905/225000 [165:53:08<181:54:41,  4.88s/it]
 40%|████      | 90906/225000 [165:53:11<163:33:47,  4.39s/it]
 40%|████      | 90907/225000 [165:53:14<148:42:46,  3.99s/it]
 40%|████      | 90908/225000 [165:53:17<138:34:29,  3.72s/it]
 40%|████      | 90909/225000 [165:53:20<128:23:28,  3.45s/it]
 40%|████      | 90910/225000 [165:53:23<125:15:35,  3.36s/it]
 40%|████      | 90911/225000 [165:53:26<119:12:09,  3.20s/it]
 40%|████      | 90912/225000 [165:53:29<118:51:04,  3.19s/it]
 40%|████      | 90913/225000 [165:53:38<178:46:45,  4.80s/it]
 40%|████      | 90914/225000 [165:53:43<183:03:47,  4.91s/it]
 40%|████      | 90915/225000 [165:53:45<155:38:40,  4.18s/it]
 40%|████      | 90916/225000 [165:53:48<142:18:49,  3.82s/it]
 40%|████      | 90917/225000 [165:53:51<130:26:46,  3.50s/it]
 40%|████      | 90918/225000 [165:53:57<163:39:12,  4.39s/it]
 40%|████      | 90919/225000 [165:54:02<165:25:22,  4.44s/it]
 40%|████      | 90920/225000 [165:54:05<151:19:28,  4.06s/it]
 40%|████      | 90921/225000 [165:54:11<177:19:03,  4.76s/it]
 40%|████      | 90922/225000 [165:54:15<162:17:35,  4.36s/it]
 40%|████      | 90923/225000 [165:54:18<147:48:14,  3.97s/it]
 40%|████      | 90924/225000 [165:54:21<142:27:06,  3.82s/it]
 40%|████      | 90925/225000 [165:54:24<132:36:52,  3.56s/it]
 40%|████      | 90926/225000 [165:54:28<128:51:11,  3.46s/it]
 40%|████      | 90927/225000 [165:54:30<122:04:46,  3.28s/it]
 40%|████      | 90928/225000 [165:54:34<123:22:58,  3.31s/it]
 40%|████      | 90929/225000 [165:54:37<123:55:14,  3.33s/it]
 40%|████      | 90930/225000 [165:54:42<135:59:38,  3.65s/it]
 40%|████      | 90931/225000 [165:54:51<196:08:52,  5.27s/it]
 40%|████      | 90932/225000 [165:54:54<177:17:52,  4.76s/it]
 40%|████      | 90933/225000 [165:54:57<155:07:07,  4.17s/it]
 40%|████      | 90934/225000 [165:55:01<148:25:06,  3.99s/it]
 40%|████      | 90935/225000 [165:55:04<138:56:48,  3.73s/it]
 40%|████      | 90936/225000 [165:55:07<128:28:50,  3.45s/it]
 40%|████      | 90937/225000 [165:55:10<133:27:37,  3.58s/it]
 40%|████      | 90938/225000 [165:55:19<187:42:02,  5.04s/it]
 40%|████      | 90939/225000 [165:55:23<172:08:32,  4.62s/it]
 40%|████      | 90940/225000 [165:55:25<149:28:16,  4.01s/it]
 40%|████      | 90941/225000 [165:55:28<136:09:10,  3.66s/it]
 40%|████      | 90942/225000 [165:55:31<125:09:21,  3.36s/it]
 40%|████      | 90943/225000 [165:55:34<121:48:22,  3.27s/it]
 40%|████      | 90944/225000 [165:55:37<117:29:21,  3.16s/it]
 40%|████      | 90945/225000 [165:55:44<165:55:12,  4.46s/it]
 40%|████      | 90946/225000 [165:55:50<177:57:59,  4.78s/it]
 40%|████      | 90947/225000 [165:55:52<156:51:51,  4.21s/it]
 40%|████      | 90948/225000 [165:55:55<139:40:22,  3.75s/it]
 40%|████      | 90949/225000 [165:56:01<163:56:15,  4.40s/it]
 40%|████      | 90950/225000 [165:56:04<148:32:53,  3.99s/it]
                                                              
{'loss': '0.3148', 'grad_norm': '6.617', 'learning_rate': '3.447e-05', 'epoch': '0.4042', 'num_input_tokens_seen': 271848424, 'train_runtime': '5.974e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 90950/225000 [165:56:04<148:32:53,  3.99s/it]
 40%|████      | 90951/225000 [165:56:06<130:16:34,  3.50s/it]
 40%|████      | 90952/225000 [165:56:09<120:39:13,  3.24s/it]
 40%|████      | 90953/225000 [165:56:17<174:02:33,  4.67s/it]
 40%|████      | 90954/225000 [165:56:20<149:05:09,  4.00s/it]
 40%|████      | 90955/225000 [165:56:22<131:44:59,  3.54s/it]
 40%|████      | 90956/225000 [165:56:25<122:16:26,  3.28s/it]
 40%|████      | 90957/225000 [165:56:27<116:20:21,  3.12s/it]
 40%|████      | 90958/225000 [165:56:31<116:21:33,  3.13s/it]
 40%|████      | 90959/225000 [165:56:34<117:53:32,  3.17s/it]
 40%|████      | 90960/225000 [165:56:40<156:46:53,  4.21s/it]
 40%|████      | 90961/225000 [165:56:48<189:06:57,  5.08s/it]
 40%|████      | 90962/225000 [165:56:51<171:00:21,  4.59s/it]
 40%|████      | 90963/225000 [165:56:55<164:16:48,  4.41s/it]
 40%|████      | 90964/225000 [165:56:58<151:28:35,  4.07s/it]
 40%|████      | 90965/225000 [165:57:02<145:05:09,  3.90s/it]
 40%|████      | 90966/225000 [165:57:05<140:50:44,  3.78s/it]
 40%|████      | 90967/225000 [165:57:09<137:40:38,  3.70s/it]
 40%|████      | 90968/225000 [165:57:13<139:20:36,  3.74s/it]
 40%|████      | 90969/225000 [165:57:16<136:22:31,  3.66s/it]
 40%|████      | 90970/225000 [165:57:24<178:42:02,  4.80s/it]
 40%|████      | 90971/225000 [165:57:26<157:45:25,  4.24s/it]
 40%|████      | 90972/225000 [165:57:30<147:54:27,  3.97s/it]
 40%|████      | 90973/225000 [165:57:33<136:36:30,  3.67s/it]
 40%|████      | 90974/225000 [165:57:39<163:24:57,  4.39s/it]
 40%|████      | 90975/225000 [165:57:47<203:29:49,  5.47s/it]
 40%|████      | 90976/225000 [165:57:49<169:28:30,  4.55s/it]
 40%|████      | 90977/225000 [165:57:52<146:32:29,  3.94s/it]
 40%|████      | 90978/225000 [165:57:54<131:15:12,  3.53s/it]
 40%|████      | 90979/225000 [165:57:57<126:59:59,  3.41s/it]
 40%|████      | 90980/225000 [165:58:03<151:28:03,  4.07s/it]
 40%|████      | 90981/225000 [165:58:09<170:24:26,  4.58s/it]
 40%|████      | 90982/225000 [165:58:11<146:40:16,  3.94s/it]
 40%|████      | 90983/225000 [165:58:14<130:55:58,  3.52s/it]
 40%|████      | 90984/225000 [165:58:17<125:28:26,  3.37s/it]
 40%|████      | 90985/225000 [165:58:20<122:13:14,  3.28s/it]
 40%|████      | 90986/225000 [165:58:24<127:49:36,  3.43s/it]
 40%|████      | 90987/225000 [165:58:32<184:18:47,  4.95s/it]
 40%|████      | 90988/225000 [165:58:35<164:41:10,  4.42s/it]
 40%|████      | 90989/225000 [165:58:39<149:57:32,  4.03s/it]
 40%|████      | 90990/225000 [165:58:41<135:44:56,  3.65s/it]
 40%|████      | 90991/225000 [165:58:44<126:33:09,  3.40s/it]
 40%|████      | 90992/225000 [165:58:47<121:08:37,  3.25s/it]
 40%|████      | 90993/225000 [165:58:50<120:32:02,  3.24s/it]
 40%|████      | 90994/225000 [165:58:59<178:33:41,  4.80s/it]
 40%|████      | 90995/225000 [165:59:03<169:44:12,  4.56s/it]
 40%|████      | 90996/225000 [165:59:05<149:58:57,  4.03s/it]
 40%|████      | 90997/225000 [165:59:08<133:18:37,  3.58s/it]
 40%|████      | 90998/225000 [165:59:11<124:02:03,  3.33s/it]
 40%|████      | 90999/225000 [165:59:17<153:06:22,  4.11s/it]
 40%|████      | 91000/225000 [165:59:23<173:37:38,  4.66s/it]
                                                              
{'loss': '0.2791', 'grad_norm': '5.156', 'learning_rate': '3.446e-05', 'epoch': '0.4044', 'num_input_tokens_seen': 271934504, 'train_runtime': '5.976e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 91000/225000 [165:59:23<173:37:38,  4.66s/it]
 40%|████      | 91001/225000 [165:59:25<149:12:21,  4.01s/it]
 40%|████      | 91002/225000 [165:59:28<133:18:43,  3.58s/it]
 40%|████      | 91003/225000 [165:59:30<124:02:18,  3.33s/it]
 40%|████      | 91004/225000 [165:59:36<144:38:40,  3.89s/it]
 40%|████      | 91005/225000 [165:59:43<186:03:43,  5.00s/it]
 40%|████      | 91006/225000 [165:59:46<159:39:05,  4.29s/it]
 40%|████      | 91007/225000 [165:59:48<140:48:29,  3.78s/it]
 40%|████      | 91008/225000 [165:59:51<130:49:22,  3.51s/it]
 40%|████      | 91009/225000 [165:59:54<120:34:38,  3.24s/it]
 40%|████      | 91010/225000 [165:59:57<116:39:54,  3.13s/it]
 40%|████      | 91011/225000 [166:00:00<115:17:11,  3.10s/it]
 40%|████      | 91012/225000 [166:00:03<111:08:29,  2.99s/it]
 40%|████      | 91013/225000 [166:00:06<118:04:46,  3.17s/it]
 40%|████      | 91014/225000 [166:00:10<122:03:52,  3.28s/it]
 40%|████      | 91015/225000 [166:00:13<122:07:54,  3.28s/it]
 40%|████      | 91016/225000 [166:00:16<123:59:05,  3.33s/it]
 40%|████      | 91017/225000 [166:00:20<125:10:20,  3.36s/it]
 40%|████      | 91018/225000 [166:00:24<128:15:41,  3.45s/it]
 40%|████      | 91019/225000 [166:00:27<126:49:30,  3.41s/it]
 40%|████      | 91020/225000 [166:00:30<125:49:32,  3.38s/it]
 40%|████      | 91021/225000 [166:00:36<149:59:03,  4.03s/it]
 40%|████      | 91022/225000 [166:00:45<206:21:46,  5.54s/it]
 40%|████      | 91023/225000 [166:00:49<196:23:48,  5.28s/it]
 40%|████      | 91024/225000 [166:00:52<163:44:27,  4.40s/it]
 40%|████      | 91025/225000 [166:00:54<143:06:40,  3.85s/it]
 40%|████      | 91026/225000 [166:00:57<133:39:52,  3.59s/it]
 40%|████      | 91027/225000 [166:01:06<194:29:34,  5.23s/it]
 40%|████      | 91028/225000 [166:01:09<164:45:51,  4.43s/it]
 40%|████      | 91029/225000 [166:01:15<180:21:31,  4.85s/it]
 40%|████      | 91030/225000 [166:01:17<154:49:26,  4.16s/it]
 40%|████      | 91031/225000 [166:01:20<137:11:22,  3.69s/it]
 40%|████      | 91032/225000 [166:01:23<131:48:33,  3.54s/it]
 40%|████      | 91033/225000 [166:01:26<120:31:57,  3.24s/it]
 40%|████      | 91034/225000 [166:01:32<154:03:13,  4.14s/it]
 40%|████      | 91035/225000 [166:01:38<174:17:27,  4.68s/it]
 40%|████      | 91036/225000 [166:01:42<165:12:24,  4.44s/it]
 40%|████      | 91037/225000 [166:01:46<165:33:39,  4.45s/it]
 40%|████      | 91038/225000 [166:01:49<150:36:44,  4.05s/it]
 40%|████      | 91039/225000 [166:01:52<140:59:13,  3.79s/it]
 40%|████      | 91040/225000 [166:01:56<138:28:55,  3.72s/it]
 40%|████      | 91041/225000 [166:02:00<135:55:37,  3.65s/it]
 40%|████      | 91042/225000 [166:02:03<138:19:48,  3.72s/it]
 40%|████      | 91043/225000 [166:02:12<191:28:34,  5.15s/it]
 40%|████      | 91044/225000 [166:02:17<189:10:19,  5.08s/it]
 40%|████      | 91045/225000 [166:02:20<168:07:09,  4.52s/it]
 40%|████      | 91046/225000 [166:02:23<150:43:02,  4.05s/it]
 40%|████      | 91047/225000 [166:02:27<145:38:31,  3.91s/it]
 40%|████      | 91048/225000 [166:02:30<135:43:52,  3.65s/it]
 40%|████      | 91049/225000 [166:02:33<131:10:05,  3.53s/it]
 40%|████      | 91050/225000 [166:02:36<124:32:20,  3.35s/it]
                                                              
{'loss': '0.3169', 'grad_norm': '5.001', 'learning_rate': '3.444e-05', 'epoch': '0.4047', 'num_input_tokens_seen': 272020232, 'train_runtime': '5.978e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 91050/225000 [166:02:36<124:32:20,  3.35s/it]
 40%|████      | 91051/225000 [166:02:39<127:49:18,  3.44s/it]
 40%|████      | 91052/225000 [166:02:42<123:31:44,  3.32s/it]
 40%|████      | 91053/225000 [166:02:45<120:22:35,  3.24s/it]
 40%|████      | 91054/225000 [166:02:49<125:21:35,  3.37s/it]
 40%|████      | 91055/225000 [166:02:53<124:54:12,  3.36s/it]
 40%|████      | 91056/225000 [166:03:01<182:03:23,  4.89s/it]
 40%|████      | 91057/225000 [166:03:07<197:53:48,  5.32s/it]
 40%|████      | 91058/225000 [166:03:10<166:37:13,  4.48s/it]
 40%|████      | 91059/225000 [166:03:17<199:32:20,  5.36s/it]
 40%|████      | 91060/225000 [166:03:20<166:29:59,  4.48s/it]
 40%|████      | 91061/225000 [166:03:25<177:03:12,  4.76s/it]
 40%|████      | 91062/225000 [166:03:29<164:56:10,  4.43s/it]
 40%|████      | 91063/225000 [166:03:32<149:09:00,  4.01s/it]
 40%|████      | 91064/225000 [166:03:34<131:13:37,  3.53s/it]
 40%|████      | 91065/225000 [166:03:37<121:02:25,  3.25s/it]
 40%|████      | 91066/225000 [166:03:40<124:12:40,  3.34s/it]
 40%|████      | 91067/225000 [166:03:49<181:10:45,  4.87s/it]
 40%|████      | 91068/225000 [166:03:52<167:35:31,  4.50s/it]
 40%|████      | 91069/225000 [166:03:55<145:47:10,  3.92s/it]
 40%|████      | 91070/225000 [166:03:58<131:04:34,  3.52s/it]
 40%|████      | 91071/225000 [166:04:00<119:44:01,  3.22s/it]
 40%|████      | 91072/225000 [166:04:03<112:00:14,  3.01s/it]
 40%|████      | 91073/225000 [166:04:05<110:23:13,  2.97s/it]
 40%|████      | 91074/225000 [166:04:14<172:14:15,  4.63s/it]
 40%|████      | 91075/225000 [166:04:18<160:17:13,  4.31s/it]
 40%|████      | 91076/225000 [166:04:21<153:53:02,  4.14s/it]
 40%|████      | 91077/225000 [166:04:24<141:38:59,  3.81s/it]
 40%|████      | 91078/225000 [166:04:27<131:08:49,  3.53s/it]
 40%|████      | 91079/225000 [166:04:30<123:02:15,  3.31s/it]
 40%|████      | 91080/225000 [166:04:33<120:24:56,  3.24s/it]
 40%|████      | 91081/225000 [166:04:41<173:36:29,  4.67s/it]
 40%|████      | 91082/225000 [166:04:44<159:51:23,  4.30s/it]
 40%|████      | 91083/225000 [166:04:47<143:47:10,  3.87s/it]
 40%|████      | 91084/225000 [166:04:50<129:50:45,  3.49s/it]
 40%|████      | 91085/225000 [166:04:53<122:07:09,  3.28s/it]
 40%|████      | 91086/225000 [166:04:56<120:19:52,  3.23s/it]
 40%|████      | 91087/225000 [166:05:02<153:34:35,  4.13s/it]
 40%|████      | 91088/225000 [166:05:11<204:32:29,  5.50s/it]
 40%|████      | 91089/225000 [166:05:15<185:15:59,  4.98s/it]
 40%|████      | 91090/225000 [166:05:18<164:09:21,  4.41s/it]
 40%|████      | 91091/225000 [166:05:21<154:19:41,  4.15s/it]
 40%|████      | 91092/225000 [166:05:25<146:01:55,  3.93s/it]
 40%|████      | 91093/225000 [166:05:27<133:30:22,  3.59s/it]
 40%|████      | 91094/225000 [166:05:32<140:45:14,  3.78s/it]
 40%|████      | 91095/225000 [166:05:35<133:12:00,  3.58s/it]
 40%|████      | 91096/225000 [166:05:38<129:47:06,  3.49s/it]
 40%|████      | 91097/225000 [166:05:41<124:17:27,  3.34s/it]
 40%|████      | 91098/225000 [166:05:44<121:49:44,  3.28s/it]
 40%|████      | 91099/225000 [166:05:51<157:08:06,  4.22s/it]
 40%|████      | 91100/225000 [166:05:58<194:22:50,  5.23s/it]
                                                              
{'loss': '0.2899', 'grad_norm': '6.601', 'learning_rate': '3.442e-05', 'epoch': '0.4049', 'num_input_tokens_seen': 272106448, 'train_runtime': '5.98e+05', 'train_tokens_per_second': '455.1'}

 40%|████      | 91100/225000 [166:05:58<194:22:50,  5.23s/it]
 40%|████      | 91101/225000 [166:06:01<164:10:17,  4.41s/it]
 40%|████      | 91102/225000 [166:06:03<143:52:17,  3.87s/it]
 40%|████      | 91103/225000 [166:06:06<134:42:13,  3.62s/it]
 40%|████      | 91104/225000 [166:06:09<127:08:53,  3.42s/it]
 40%|████      | 91105/225000 [166:06:12<123:30:54,  3.32s/it]
 40%|████      | 91106/225000 [166:06:16<123:37:55,  3.32s/it]
 40%|████      | 91107/225000 [166:06:24<178:25:31,  4.80s/it]
 40%|████      | 91108/225000 [166:06:29<183:31:56,  4.93s/it]
 40%|████      | 91109/225000 [166:06:32<155:15:36,  4.17s/it]
 40%|████      | 91110/225000 [166:06:35<142:26:09,  3.83s/it]
 40%|████      | 91111/225000 [166:06:37<131:37:21,  3.54s/it]
 40%|████      | 91112/225000 [166:06:40<124:24:36,  3.35s/it]
 40%|████      | 91113/225000 [166:06:44<125:16:56,  3.37s/it]
 40%|████      | 91114/225000 [166:06:53<195:00:49,  5.24s/it]
 40%|████      | 91115/225000 [166:06:58<182:42:14,  4.91s/it]
 40%|████      | 91116/225000 [166:07:00<153:45:39,  4.13s/it]
 40%|████      | 91117/225000 [166:07:06<181:03:56,  4.87s/it]
 40%|████      | 91118/225000 [166:07:10<161:41:56,  4.35s/it]
 40%|████      | 91119/225000 [166:07:13<151:40:48,  4.08s/it]
 40%|████      | 91120/225000 [166:07:16<144:25:03,  3.88s/it]
 40%|████      | 91121/225000 [166:07:20<140:41:14,  3.78s/it]
 40%|████      | 91122/225000 [166:07:24<138:41:57,  3.73s/it]
 40%|████      | 91123/225000 [166:07:27<135:31:45,  3.64s/it]
 40%|████      | 91124/225000 [166:07:30<133:33:44,  3.59s/it]
 40%|████      | 91125/225000 [166:07:34<132:49:46,  3.57s/it]
 41%|████      | 91126/225000 [166:07:37<130:29:06,  3.51s/it]
 41%|████      | 91127/225000 [166:07:41<132:36:48,  3.57s/it]
 41%|████      | 91128/225000 [166:07:45<131:48:25,  3.54s/it]
 41%|████      | 91129/225000 [166:07:48<129:13:29,  3.48s/it]
 41%|████      | 91130/225000 [166:07:51<126:18:03,  3.40s/it]
 41%|████      | 91131/225000 [166:07:55<127:28:58,  3.43s/it]
 41%|████      | 91132/225000 [166:07:58<125:12:41,  3.37s/it]
 41%|████      | 91133/225000 [166:08:01<123:59:07,  3.33s/it]
 41%|████      | 91134/225000 [166:08:08<169:01:19,  4.55s/it]
 41%|████      | 91135/225000 [166:08:13<169:47:24,  4.57s/it]
 41%|████      | 91136/225000 [166:08:17<157:26:19,  4.23s/it]
 41%|████      | 91137/225000 [166:08:20<149:16:27,  4.01s/it]
 41%|████      | 91138/225000 [166:08:24<144:07:31,  3.88s/it]
 41%|████      | 91139/225000 [166:08:27<139:57:26,  3.76s/it]
 41%|████      | 91140/225000 [166:08:30<135:13:15,  3.64s/it]
 41%|████      | 91141/225000 [166:08:34<131:23:19,  3.53s/it]
 41%|████      | 91142/225000 [166:08:37<127:52:52,  3.44s/it]
 41%|████      | 91143/225000 [166:08:40<127:19:29,  3.42s/it]
 41%|████      | 91144/225000 [166:08:46<155:26:45,  4.18s/it]
 41%|████      | 91145/225000 [166:08:50<149:56:33,  4.03s/it]
 41%|████      | 91146/225000 [166:08:54<145:27:57,  3.91s/it]
 41%|████      | 91147/225000 [166:08:57<144:46:57,  3.89s/it]
 41%|████      | 91148/225000 [166:09:06<192:56:29,  5.19s/it]
 41%|████      | 91149/225000 [166:09:12<208:47:02,  5.62s/it]
 41%|████      | 91150/225000 [166:09:15<176:45:37,  4.75s/it]
                                                              
{'loss': '0.2934', 'grad_norm': '3.805', 'learning_rate': '3.44e-05', 'epoch': '0.4051', 'num_input_tokens_seen': 272192832, 'train_runtime': '5.982e+05', 'train_tokens_per_second': '455.1'}

 41%|████      | 91150/225000 [166:09:15<176:45:37,  4.75s/it]
 41%|████      | 91151/225000 [166:09:18<157:31:50,  4.24s/it]
 41%|████      | 91152/225000 [166:09:24<174:05:48,  4.68s/it]
 41%|████      | 91153/225000 [166:09:29<180:55:23,  4.87s/it]
 41%|████      | 91154/225000 [166:09:32<163:19:45,  4.39s/it]
 41%|████      | 91155/225000 [166:09:36<153:39:02,  4.13s/it]
 41%|████      | 91156/225000 [166:09:39<144:13:58,  3.88s/it]
 41%|████      | 91157/225000 [166:09:42<135:30:36,  3.64s/it]
 41%|████      | 91158/225000 [166:09:46<132:30:04,  3.56s/it]
 41%|████      | 91159/225000 [166:09:49<126:32:37,  3.40s/it]
 41%|████      | 91160/225000 [166:09:52<129:27:52,  3.48s/it]
 41%|████      | 91161/225000 [166:09:55<124:22:32,  3.35s/it]
 41%|████      | 91162/225000 [166:09:59<127:30:13,  3.43s/it]
 41%|████      | 91163/225000 [166:10:04<147:36:14,  3.97s/it]
 41%|████      | 91164/225000 [166:10:13<203:59:18,  5.49s/it]
 41%|████      | 91165/225000 [166:10:18<190:54:18,  5.14s/it]
 41%|████      | 91166/225000 [166:10:20<161:56:26,  4.36s/it]
 41%|████      | 91167/225000 [166:10:23<140:31:45,  3.78s/it]
 41%|████      | 91168/225000 [166:10:27<149:37:08,  4.02s/it]
 41%|████      | 91169/225000 [166:10:32<161:23:57,  4.34s/it]
 41%|████      | 91170/225000 [166:10:35<145:19:46,  3.91s/it]
 41%|████      | 91171/225000 [166:10:38<134:38:02,  3.62s/it]
 41%|████      | 91172/225000 [166:10:41<129:08:09,  3.47s/it]
 41%|████      | 91173/225000 [166:10:44<125:29:42,  3.38s/it]
 41%|████      | 91174/225000 [166:10:47<121:22:46,  3.27s/it]
 41%|████      | 91175/225000 [166:10:51<122:27:55,  3.29s/it]
 41%|████      | 91176/225000 [166:10:54<121:21:35,  3.26s/it]
 41%|████      | 91177/225000 [166:11:01<163:06:10,  4.39s/it]
 41%|████      | 91178/225000 [166:11:08<191:20:02,  5.15s/it]
 41%|████      | 91179/225000 [166:11:11<163:56:15,  4.41s/it]
 41%|████      | 91180/225000 [166:11:13<143:59:51,  3.87s/it]
 41%|████      | 91181/225000 [166:11:16<131:23:13,  3.53s/it]
 41%|████      | 91182/225000 [166:11:19<122:36:52,  3.30s/it]
 41%|████      | 91183/225000 [166:11:22<120:10:17,  3.23s/it]
 41%|████      | 91184/225000 [166:11:24<113:37:48,  3.06s/it]
 41%|████      | 91185/225000 [166:11:32<169:24:24,  4.56s/it]
 41%|████      | 91186/225000 [166:11:38<184:23:33,  4.96s/it]
 41%|████      | 91187/225000 [166:11:41<159:57:33,  4.30s/it]
 41%|████      | 91188/225000 [166:11:47<183:18:14,  4.93s/it]
 41%|████      | 91189/225000 [166:11:50<157:50:51,  4.25s/it]
 41%|████      | 91190/225000 [166:11:53<139:50:32,  3.76s/it]
 41%|████      | 91191/225000 [166:11:56<132:12:16,  3.56s/it]
 41%|████      | 91192/225000 [166:11:59<123:59:54,  3.34s/it]
 41%|████      | 91193/225000 [166:12:02<119:08:02,  3.21s/it]
 41%|████      | 91194/225000 [166:12:04<113:31:27,  3.05s/it]
 41%|████      | 91195/225000 [166:12:12<170:39:55,  4.59s/it]
 41%|████      | 91196/225000 [166:12:18<179:31:13,  4.83s/it]
 41%|████      | 91197/225000 [166:12:20<151:25:55,  4.07s/it]
 41%|████      | 91198/225000 [166:12:22<132:20:20,  3.56s/it]
 41%|████      | 91199/225000 [166:12:25<118:56:48,  3.20s/it]
 41%|████      | 91200/225000 [166:12:28<113:23:18,  3.05s/it]
                                                              
{'loss': '0.3', 'grad_norm': '6.007', 'learning_rate': '3.439e-05', 'epoch': '0.4053', 'num_input_tokens_seen': 272277936, 'train_runtime': '5.983e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91200/225000 [166:12:28<113:23:18,  3.05s/it]
 41%|████      | 91201/225000 [166:12:31<118:25:56,  3.19s/it]
 41%|████      | 91202/225000 [166:12:37<145:54:19,  3.93s/it]
 41%|████      | 91203/225000 [166:12:46<200:27:48,  5.39s/it]
 41%|████      | 91204/225000 [166:12:48<170:36:48,  4.59s/it]
 41%|████      | 91205/225000 [166:12:51<147:02:21,  3.96s/it]
 41%|████      | 91206/225000 [166:12:53<129:55:31,  3.50s/it]
 41%|████      | 91207/225000 [166:12:56<120:24:31,  3.24s/it]
 41%|████      | 91208/225000 [166:13:01<140:24:03,  3.78s/it]
 41%|████      | 91209/225000 [166:13:07<169:14:02,  4.55s/it]
 41%|████      | 91210/225000 [166:13:10<149:00:43,  4.01s/it]
 41%|████      | 91211/225000 [166:13:13<133:59:47,  3.61s/it]
 41%|████      | 91212/225000 [166:13:15<123:28:08,  3.32s/it]
 41%|████      | 91213/225000 [166:13:18<115:02:05,  3.10s/it]
 41%|████      | 91214/225000 [166:13:24<144:52:18,  3.90s/it]
 41%|████      | 91215/225000 [166:13:30<176:33:22,  4.75s/it]
 41%|████      | 91216/225000 [166:13:33<149:36:54,  4.03s/it]
 41%|████      | 91217/225000 [166:13:35<131:19:50,  3.53s/it]
 41%|████      | 91218/225000 [166:13:38<126:19:10,  3.40s/it]
 41%|████      | 91219/225000 [166:13:45<166:34:05,  4.48s/it]
 41%|████      | 91220/225000 [166:13:48<145:55:19,  3.93s/it]
 41%|████      | 91221/225000 [166:13:50<131:33:04,  3.54s/it]
 41%|████      | 91222/225000 [166:13:53<119:31:08,  3.22s/it]
 41%|████      | 91223/225000 [166:14:02<185:39:46,  5.00s/it]
 41%|████      | 91224/225000 [166:14:05<165:06:06,  4.44s/it]
 41%|████      | 91225/225000 [166:14:07<141:11:58,  3.80s/it]
 41%|████      | 91226/225000 [166:14:13<158:07:35,  4.26s/it]
 41%|████      | 91227/225000 [166:14:17<156:07:56,  4.20s/it]
 41%|████      | 91228/225000 [166:14:19<138:11:28,  3.72s/it]
 41%|████      | 91229/225000 [166:14:26<175:11:39,  4.71s/it]
 41%|████      | 91230/225000 [166:14:31<167:45:09,  4.51s/it]
 41%|████      | 91231/225000 [166:14:33<142:29:27,  3.83s/it]
 41%|████      | 91232/225000 [166:14:38<160:25:20,  4.32s/it]
 41%|████      | 91233/225000 [166:14:43<162:09:31,  4.36s/it]
 41%|████      | 91234/225000 [166:14:45<139:46:16,  3.76s/it]
 41%|████      | 91235/225000 [166:14:48<126:53:51,  3.42s/it]
 41%|████      | 91236/225000 [166:14:55<174:50:35,  4.71s/it]
 41%|████      | 91237/225000 [166:14:58<155:23:59,  4.18s/it]
 41%|████      | 91238/225000 [166:15:02<144:31:59,  3.89s/it]
 41%|████      | 91239/225000 [166:15:05<136:54:46,  3.68s/it]
 41%|████      | 91240/225000 [166:15:08<129:51:38,  3.50s/it]
 41%|████      | 91241/225000 [166:15:11<127:34:07,  3.43s/it]
 41%|████      | 91242/225000 [166:15:14<121:33:27,  3.27s/it]
 41%|████      | 91243/225000 [166:15:17<120:56:26,  3.26s/it]
 41%|████      | 91244/225000 [166:15:20<118:07:07,  3.18s/it]
 41%|████      | 91245/225000 [166:15:23<115:21:47,  3.10s/it]
 41%|████      | 91246/225000 [166:15:26<118:02:54,  3.18s/it]
 41%|████      | 91247/225000 [166:15:30<116:36:36,  3.14s/it]
 41%|████      | 91248/225000 [166:15:34<132:56:52,  3.58s/it]
 41%|████      | 91249/225000 [166:15:41<166:06:11,  4.47s/it]
 41%|████      | 91250/225000 [166:15:46<170:20:13,  4.58s/it]
                                                              
{'loss': '0.2977', 'grad_norm': '4.666', 'learning_rate': '3.437e-05', 'epoch': '0.4056', 'num_input_tokens_seen': 272363640, 'train_runtime': '5.985e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91250/225000 [166:15:46<170:20:13,  4.58s/it]
 41%|████      | 91251/225000 [166:15:51<181:05:50,  4.87s/it]
 41%|████      | 91252/225000 [166:15:56<184:42:52,  4.97s/it]
 41%|████      | 91253/225000 [166:15:59<160:00:22,  4.31s/it]
 41%|████      | 91254/225000 [166:16:02<148:10:36,  3.99s/it]
 41%|████      | 91255/225000 [166:16:05<135:31:11,  3.65s/it]
 41%|████      | 91256/225000 [166:16:08<126:06:39,  3.39s/it]
 41%|████      | 91257/225000 [166:16:11<121:07:24,  3.26s/it]
 41%|████      | 91258/225000 [166:16:15<125:41:45,  3.38s/it]
 41%|████      | 91259/225000 [166:16:22<170:03:34,  4.58s/it]
 41%|████      | 91260/225000 [166:16:26<168:57:37,  4.55s/it]
 41%|████      | 91261/225000 [166:16:29<148:28:27,  4.00s/it]
 41%|████      | 91262/225000 [166:16:32<132:53:41,  3.58s/it]
 41%|████      | 91263/225000 [166:16:40<189:32:26,  5.10s/it]
 41%|████      | 91264/225000 [166:16:43<165:50:22,  4.46s/it]
 41%|████      | 91265/225000 [166:16:46<143:22:48,  3.86s/it]
 41%|████      | 91266/225000 [166:16:49<133:02:47,  3.58s/it]
 41%|████      | 91267/225000 [166:16:52<125:59:38,  3.39s/it]
 41%|████      | 91268/225000 [166:16:55<126:17:30,  3.40s/it]
 41%|████      | 91269/225000 [166:16:58<122:00:55,  3.28s/it]
 41%|████      | 91270/225000 [166:17:01<120:30:45,  3.24s/it]
 41%|████      | 91271/225000 [166:17:04<119:37:37,  3.22s/it]
 41%|████      | 91272/225000 [166:17:13<174:33:54,  4.70s/it]
 41%|████      | 91273/225000 [166:17:20<206:10:20,  5.55s/it]
 41%|████      | 91274/225000 [166:17:22<169:43:38,  4.57s/it]
 41%|████      | 91275/225000 [166:17:26<160:49:16,  4.33s/it]
 41%|████      | 91276/225000 [166:17:31<163:36:24,  4.40s/it]
 41%|████      | 91277/225000 [166:17:33<140:33:10,  3.78s/it]
 41%|████      | 91278/225000 [166:17:36<134:09:59,  3.61s/it]
 41%|████      | 91279/225000 [166:17:44<176:10:53,  4.74s/it]
 41%|████      | 91280/225000 [166:17:46<148:42:28,  4.00s/it]
 41%|████      | 91281/225000 [166:17:48<131:49:23,  3.55s/it]
 41%|████      | 91282/225000 [166:17:52<127:35:37,  3.44s/it]
 41%|████      | 91283/225000 [166:17:55<125:07:45,  3.37s/it]
 41%|████      | 91284/225000 [166:17:58<124:23:51,  3.35s/it]
 41%|████      | 91285/225000 [166:18:01<120:04:02,  3.23s/it]
 41%|████      | 91286/225000 [166:18:05<124:19:11,  3.35s/it]
 41%|████      | 91287/225000 [166:18:08<124:44:52,  3.36s/it]
 41%|████      | 91288/225000 [166:18:11<124:23:50,  3.35s/it]
 41%|████      | 91289/225000 [166:18:14<120:22:22,  3.24s/it]
 41%|████      | 91290/225000 [166:18:21<157:31:11,  4.24s/it]
 41%|████      | 91291/225000 [166:18:24<143:59:15,  3.88s/it]
 41%|████      | 91292/225000 [166:18:28<141:23:08,  3.81s/it]
 41%|████      | 91293/225000 [166:18:31<135:53:43,  3.66s/it]
 41%|████      | 91294/225000 [166:18:34<131:52:53,  3.55s/it]
 41%|████      | 91295/225000 [166:18:37<126:44:01,  3.41s/it]
 41%|████      | 91296/225000 [166:18:40<122:48:39,  3.31s/it]
 41%|████      | 91297/225000 [166:18:45<140:31:19,  3.78s/it]
 41%|████      | 91298/225000 [166:18:54<192:17:39,  5.18s/it]
 41%|████      | 91299/225000 [166:19:01<209:55:31,  5.65s/it]
 41%|████      | 91300/225000 [166:19:03<178:15:40,  4.80s/it]
                                                              
{'loss': '0.312', 'grad_norm': '5.189', 'learning_rate': '3.435e-05', 'epoch': '0.4058', 'num_input_tokens_seen': 272448928, 'train_runtime': '5.987e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91300/225000 [166:19:03<178:15:40,  4.80s/it]
 41%|████      | 91301/225000 [166:19:06<155:45:54,  4.19s/it]
 41%|████      | 91302/225000 [166:19:09<138:26:46,  3.73s/it]
 41%|████      | 91303/225000 [166:19:11<126:04:06,  3.39s/it]
 41%|████      | 91304/225000 [166:19:15<125:00:41,  3.37s/it]
 41%|████      | 91305/225000 [166:19:24<194:06:52,  5.23s/it]
 41%|████      | 91306/225000 [166:19:28<180:23:37,  4.86s/it]
 41%|████      | 91307/225000 [166:19:31<153:52:28,  4.14s/it]
 41%|████      | 91308/225000 [166:19:33<135:13:59,  3.64s/it]
 41%|████      | 91309/225000 [166:19:36<128:29:00,  3.46s/it]
 41%|████      | 91310/225000 [166:19:39<123:20:12,  3.32s/it]
 41%|████      | 91311/225000 [166:19:43<127:12:38,  3.43s/it]
 41%|████      | 91312/225000 [166:19:46<128:02:53,  3.45s/it]
 41%|████      | 91313/225000 [166:19:50<125:31:13,  3.38s/it]
 41%|████      | 91314/225000 [166:19:56<164:23:11,  4.43s/it]
 41%|████      | 91315/225000 [166:20:05<207:56:14,  5.60s/it]
 41%|████      | 91316/225000 [166:20:08<177:35:30,  4.78s/it]
 41%|████      | 91317/225000 [166:20:10<153:32:25,  4.13s/it]
 41%|████      | 91318/225000 [166:20:13<138:59:40,  3.74s/it]
 41%|████      | 91319/225000 [166:20:16<126:19:37,  3.40s/it]
 41%|████      | 91320/225000 [166:20:21<148:05:00,  3.99s/it]
 41%|████      | 91321/225000 [166:20:28<183:59:04,  4.95s/it]
 41%|████      | 91322/225000 [166:20:31<158:12:57,  4.26s/it]
 41%|████      | 91323/225000 [166:20:34<143:01:06,  3.85s/it]
 41%|████      | 91324/225000 [166:20:40<167:46:15,  4.52s/it]
 41%|████      | 91325/225000 [166:20:43<157:25:21,  4.24s/it]
 41%|████      | 91326/225000 [166:20:47<144:27:33,  3.89s/it]
 41%|████      | 91327/225000 [166:20:50<134:47:49,  3.63s/it]
 41%|████      | 91328/225000 [166:20:53<130:03:07,  3.50s/it]
 41%|████      | 91329/225000 [166:20:56<124:24:19,  3.35s/it]
 41%|████      | 91330/225000 [166:20:59<121:05:35,  3.26s/it]
 41%|████      | 91331/225000 [166:21:02<125:15:24,  3.37s/it]
 41%|████      | 91332/225000 [166:21:06<122:11:36,  3.29s/it]
 41%|████      | 91333/225000 [166:21:13<163:33:37,  4.41s/it]
 41%|████      | 91334/225000 [166:21:20<201:33:34,  5.43s/it]
 41%|████      | 91335/225000 [166:21:23<171:51:59,  4.63s/it]
 41%|████      | 91336/225000 [166:21:26<150:43:32,  4.06s/it]
 41%|████      | 91337/225000 [166:21:29<135:26:17,  3.65s/it]
 41%|████      | 91338/225000 [166:21:32<128:24:16,  3.46s/it]
 41%|████      | 91339/225000 [166:21:35<125:46:01,  3.39s/it]
 41%|████      | 91340/225000 [166:21:38<120:28:43,  3.24s/it]
 41%|████      | 91341/225000 [166:21:41<115:36:53,  3.11s/it]
 41%|████      | 91342/225000 [166:21:44<120:48:20,  3.25s/it]
 41%|████      | 91343/225000 [166:21:53<185:22:05,  4.99s/it]
 41%|████      | 91344/225000 [166:21:58<185:22:54,  4.99s/it]
 41%|████      | 91345/225000 [166:22:01<157:16:53,  4.24s/it]
 41%|████      | 91346/225000 [166:22:03<137:51:32,  3.71s/it]
 41%|████      | 91347/225000 [166:22:06<127:37:23,  3.44s/it]
 41%|████      | 91348/225000 [166:22:13<167:34:01,  4.51s/it]
 41%|████      | 91349/225000 [166:22:19<182:15:08,  4.91s/it]
 41%|████      | 91350/225000 [166:22:21<154:47:04,  4.17s/it]
                                                              
{'loss': '0.2858', 'grad_norm': '4.63', 'learning_rate': '3.434e-05', 'epoch': '0.406', 'num_input_tokens_seen': 272534848, 'train_runtime': '5.989e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91350/225000 [166:22:21<154:47:04,  4.17s/it]
 41%|████      | 91351/225000 [166:22:24<135:51:01,  3.66s/it]
 41%|████      | 91352/225000 [166:22:31<177:33:04,  4.78s/it]
 41%|████      | 91353/225000 [166:22:34<156:23:35,  4.21s/it]
 41%|████      | 91354/225000 [166:22:37<142:54:09,  3.85s/it]
 41%|████      | 91355/225000 [166:22:40<137:01:02,  3.69s/it]
 41%|████      | 91356/225000 [166:22:43<126:20:17,  3.40s/it]
 41%|████      | 91357/225000 [166:22:46<120:21:58,  3.24s/it]
 41%|████      | 91358/225000 [166:22:49<116:24:37,  3.14s/it]
 41%|████      | 91359/225000 [166:22:53<132:33:04,  3.57s/it]
 41%|████      | 91360/225000 [166:23:00<163:39:14,  4.41s/it]
 41%|████      | 91361/225000 [166:23:02<144:33:27,  3.89s/it]
 41%|████      | 91362/225000 [166:23:05<133:32:49,  3.60s/it]
 41%|████      | 91363/225000 [166:23:08<124:15:49,  3.35s/it]
 41%|████      | 91364/225000 [166:23:12<125:26:16,  3.38s/it]
 41%|████      | 91365/225000 [166:23:21<188:39:46,  5.08s/it]
 41%|████      | 91366/225000 [166:23:24<174:00:28,  4.69s/it]
 41%|████      | 91367/225000 [166:23:27<147:00:03,  3.96s/it]
 41%|████      | 91368/225000 [166:23:30<135:13:47,  3.64s/it]
 41%|████      | 91369/225000 [166:23:32<127:27:44,  3.43s/it]
 41%|████      | 91370/225000 [166:23:35<122:06:26,  3.29s/it]
 41%|████      | 91371/225000 [166:23:39<123:44:29,  3.33s/it]
 41%|████      | 91372/225000 [166:23:42<118:55:51,  3.20s/it]
 41%|████      | 91373/225000 [166:23:46<134:07:06,  3.61s/it]
 41%|████      | 91374/225000 [166:23:56<206:44:39,  5.57s/it]
 41%|████      | 91375/225000 [166:23:59<174:51:51,  4.71s/it]
 41%|████      | 91376/225000 [166:24:02<149:01:45,  4.02s/it]
 41%|████      | 91377/225000 [166:24:05<140:32:13,  3.79s/it]
 41%|████      | 91378/225000 [166:24:07<125:25:38,  3.38s/it]
 41%|████      | 91379/225000 [166:24:14<166:33:09,  4.49s/it]
 41%|████      | 91380/225000 [166:24:19<165:02:55,  4.45s/it]
 41%|████      | 91381/225000 [166:24:22<147:24:03,  3.97s/it]
 41%|████      | 91382/225000 [166:24:25<137:23:34,  3.70s/it]
 41%|████      | 91383/225000 [166:24:27<125:44:34,  3.39s/it]
 41%|████      | 91384/225000 [166:24:30<119:13:07,  3.21s/it]
 41%|████      | 91385/225000 [166:24:39<181:06:41,  4.88s/it]
 41%|████      | 91386/225000 [166:24:42<160:54:39,  4.34s/it]
 41%|████      | 91387/225000 [166:24:45<142:18:11,  3.83s/it]
 41%|████      | 91388/225000 [166:24:47<130:57:40,  3.53s/it]
 41%|████      | 91389/225000 [166:24:51<127:35:30,  3.44s/it]
 41%|████      | 91390/225000 [166:24:55<134:31:49,  3.62s/it]
 41%|████      | 91391/225000 [166:25:06<216:57:22,  5.85s/it]
 41%|████      | 91392/225000 [166:25:08<180:52:12,  4.87s/it]
 41%|████      | 91393/225000 [166:25:11<157:19:25,  4.24s/it]
 41%|████      | 91394/225000 [166:25:14<138:53:18,  3.74s/it]
 41%|████      | 91395/225000 [166:25:17<133:19:59,  3.59s/it]
 41%|████      | 91396/225000 [166:25:20<126:50:40,  3.42s/it]
 41%|████      | 91397/225000 [166:25:23<124:49:39,  3.36s/it]
 41%|████      | 91398/225000 [166:25:27<125:01:38,  3.37s/it]
 41%|████      | 91399/225000 [166:25:36<190:41:28,  5.14s/it]
 41%|████      | 91400/225000 [166:25:41<194:26:30,  5.24s/it]
                                                              
{'loss': '0.3169', 'grad_norm': '5.831', 'learning_rate': '3.432e-05', 'epoch': '0.4062', 'num_input_tokens_seen': 272620208, 'train_runtime': '5.991e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91400/225000 [166:25:41<194:26:30,  5.24s/it]
 41%|████      | 91401/225000 [166:25:44<165:46:35,  4.47s/it]
 41%|████      | 91402/225000 [166:25:47<148:09:42,  3.99s/it]
 41%|████      | 91403/225000 [166:25:51<154:18:14,  4.16s/it]
 41%|████      | 91404/225000 [166:25:58<181:49:55,  4.90s/it]
 41%|████      | 91405/225000 [166:26:00<152:18:58,  4.10s/it]
 41%|████      | 91406/225000 [166:26:03<137:19:54,  3.70s/it]
 41%|████      | 91407/225000 [166:26:06<126:14:42,  3.40s/it]
 41%|████      | 91408/225000 [166:26:09<123:29:18,  3.33s/it]
 41%|████      | 91409/225000 [166:26:13<129:33:25,  3.49s/it]
 41%|████      | 91410/225000 [166:26:21<184:43:26,  4.98s/it]
 41%|████      | 91411/225000 [166:26:24<155:17:44,  4.18s/it]
 41%|████      | 91412/225000 [166:26:27<142:29:11,  3.84s/it]
 41%|████      | 91413/225000 [166:26:30<138:26:56,  3.73s/it]
 41%|████      | 91414/225000 [166:26:34<140:19:25,  3.78s/it]
 41%|████      | 91415/225000 [166:26:38<140:36:34,  3.79s/it]
 41%|████      | 91416/225000 [166:26:41<139:46:11,  3.77s/it]
 41%|████      | 91417/225000 [166:26:45<135:04:32,  3.64s/it]
 41%|████      | 91418/225000 [166:26:49<138:02:33,  3.72s/it]
 41%|████      | 91419/225000 [166:26:53<140:00:01,  3.77s/it]
 41%|████      | 91420/225000 [166:26:57<142:00:55,  3.83s/it]
 41%|████      | 91421/225000 [166:27:00<138:16:42,  3.73s/it]
 41%|████      | 91422/225000 [166:27:03<134:25:56,  3.62s/it]
 41%|████      | 91423/225000 [166:27:07<131:42:53,  3.55s/it]
 41%|████      | 91424/225000 [166:27:16<194:25:03,  5.24s/it]
 41%|████      | 91425/225000 [166:27:20<180:17:30,  4.86s/it]
 41%|████      | 91426/225000 [166:27:23<156:49:25,  4.23s/it]
 41%|████      | 91427/225000 [166:27:26<143:22:08,  3.86s/it]
 41%|████      | 91428/225000 [166:27:34<188:52:31,  5.09s/it]
 41%|████      | 91429/225000 [166:27:37<167:58:36,  4.53s/it]
 41%|████      | 91430/225000 [166:27:40<147:35:55,  3.98s/it]
 41%|████      | 91431/225000 [166:27:42<129:30:09,  3.49s/it]
 41%|████      | 91432/225000 [166:27:46<134:50:49,  3.63s/it]
 41%|████      | 91433/225000 [166:27:49<129:32:10,  3.49s/it]
 41%|████      | 91434/225000 [166:27:53<132:56:29,  3.58s/it]
 41%|████      | 91435/225000 [166:27:56<128:39:02,  3.47s/it]
 41%|████      | 91436/225000 [166:27:59<126:48:21,  3.42s/it]
 41%|████      | 91437/225000 [166:28:03<125:21:55,  3.38s/it]
 41%|████      | 91438/225000 [166:28:10<167:11:42,  4.51s/it]
 41%|████      | 91439/225000 [166:28:17<194:04:38,  5.23s/it]
 41%|████      | 91440/225000 [166:28:20<167:05:39,  4.50s/it]
 41%|████      | 91441/225000 [166:28:22<148:17:14,  4.00s/it]
 41%|████      | 91442/225000 [166:28:25<138:49:34,  3.74s/it]
 41%|████      | 91443/225000 [166:28:29<141:23:58,  3.81s/it]
 41%|████      | 91444/225000 [166:28:33<132:54:34,  3.58s/it]
 41%|████      | 91445/225000 [166:28:36<130:54:39,  3.53s/it]
 41%|████      | 91446/225000 [166:28:39<125:46:47,  3.39s/it]
 41%|████      | 91447/225000 [166:28:43<128:00:55,  3.45s/it]
 41%|████      | 91448/225000 [166:28:48<151:33:41,  4.09s/it]
 41%|████      | 91449/225000 [166:28:58<212:59:22,  5.74s/it]
 41%|████      | 91450/225000 [166:29:00<176:57:50,  4.77s/it]
                                                              
{'loss': '0.2945', 'grad_norm': '4.414', 'learning_rate': '3.43e-05', 'epoch': '0.4064', 'num_input_tokens_seen': 272707112, 'train_runtime': '5.993e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91450/225000 [166:29:00<176:57:50,  4.77s/it]
 41%|████      | 91451/225000 [166:29:03<151:48:01,  4.09s/it]
 41%|████      | 91452/225000 [166:29:06<137:45:33,  3.71s/it]
 41%|████      | 91453/225000 [166:29:08<125:26:15,  3.38s/it]
 41%|████      | 91454/225000 [166:29:11<121:23:45,  3.27s/it]
 41%|████      | 91455/225000 [166:29:15<124:04:58,  3.34s/it]
 41%|████      | 91456/225000 [166:29:18<125:46:25,  3.39s/it]
 41%|████      | 91457/225000 [166:29:23<137:48:22,  3.71s/it]
 41%|████      | 91458/225000 [166:29:31<190:39:58,  5.14s/it]
 41%|████      | 91459/225000 [166:29:36<190:13:52,  5.13s/it]
 41%|████      | 91460/225000 [166:29:39<159:24:24,  4.30s/it]
 41%|████      | 91461/225000 [166:29:46<196:38:31,  5.30s/it]
 41%|████      | 91462/225000 [166:29:49<163:44:29,  4.41s/it]
 41%|████      | 91463/225000 [166:29:51<143:26:48,  3.87s/it]
 41%|████      | 91464/225000 [166:29:55<139:15:05,  3.75s/it]
 41%|████      | 91465/225000 [166:29:59<144:51:14,  3.91s/it]
 41%|████      | 91466/225000 [166:30:03<149:35:16,  4.03s/it]
 41%|████      | 91467/225000 [166:30:06<136:04:36,  3.67s/it]
 41%|████      | 91468/225000 [166:30:09<127:02:47,  3.43s/it]
 41%|████      | 91469/225000 [166:30:12<123:30:02,  3.33s/it]
 41%|████      | 91470/225000 [166:30:15<119:24:31,  3.22s/it]
 41%|████      | 91471/225000 [166:30:18<120:25:50,  3.25s/it]
 41%|████      | 91472/225000 [166:30:27<181:57:33,  4.91s/it]
 41%|████      | 91473/225000 [166:30:30<164:52:07,  4.44s/it]
 41%|████      | 91474/225000 [166:30:33<146:17:53,  3.94s/it]
 41%|████      | 91475/225000 [166:30:36<129:18:45,  3.49s/it]
 41%|████      | 91476/225000 [166:30:38<121:24:51,  3.27s/it]
 41%|████      | 91477/225000 [166:30:47<183:52:57,  4.96s/it]
 41%|████      | 91478/225000 [166:30:52<176:53:43,  4.77s/it]
 41%|████      | 91479/225000 [166:30:54<154:28:59,  4.17s/it]
 41%|████      | 91480/225000 [166:31:00<169:08:30,  4.56s/it]
 41%|████      | 91481/225000 [166:31:03<149:35:18,  4.03s/it]
 41%|████      | 91482/225000 [166:31:05<133:22:24,  3.60s/it]
 41%|████      | 91483/225000 [166:31:09<132:01:40,  3.56s/it]
 41%|████      | 91484/225000 [166:31:17<182:25:06,  4.92s/it]
 41%|████      | 91485/225000 [166:31:19<155:27:13,  4.19s/it]
 41%|████      | 91486/225000 [166:31:22<137:10:53,  3.70s/it]
 41%|████      | 91487/225000 [166:31:25<128:40:19,  3.47s/it]
 41%|████      | 91488/225000 [166:31:28<125:31:36,  3.38s/it]
 41%|████      | 91489/225000 [166:31:36<175:47:02,  4.74s/it]
 41%|████      | 91490/225000 [166:31:42<194:55:55,  5.26s/it]
 41%|████      | 91491/225000 [166:31:45<161:32:02,  4.36s/it]
 41%|████      | 91492/225000 [166:31:49<165:30:31,  4.46s/it]
 41%|████      | 91493/225000 [166:31:53<154:31:01,  4.17s/it]
 41%|████      | 91494/225000 [166:31:56<140:40:03,  3.79s/it]
 41%|████      | 91495/225000 [166:32:03<174:25:28,  4.70s/it]
 41%|████      | 91496/225000 [166:32:06<155:38:46,  4.20s/it]
 41%|████      | 91497/225000 [166:32:09<146:38:44,  3.95s/it]
 41%|████      | 91498/225000 [166:32:12<141:37:23,  3.82s/it]
 41%|████      | 91499/225000 [166:32:15<130:57:21,  3.53s/it]
 41%|████      | 91500/225000 [166:32:19<128:45:41,  3.47s/it]
                                                              
{'loss': '0.3041', 'grad_norm': '5.143', 'learning_rate': '3.429e-05', 'epoch': '0.4067', 'num_input_tokens_seen': 272794592, 'train_runtime': '5.995e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91500/225000 [166:32:19<128:45:41,  3.47s/it]
 41%|████      | 91501/225000 [166:32:22<125:09:28,  3.38s/it]
 41%|████      | 91502/225000 [166:32:25<123:03:34,  3.32s/it]
 41%|████      | 91503/225000 [166:32:28<117:34:08,  3.17s/it]
 41%|████      | 91504/225000 [166:32:31<119:45:57,  3.23s/it]
 41%|████      | 91505/225000 [166:32:40<180:27:22,  4.87s/it]
 41%|████      | 91506/225000 [166:32:47<202:29:20,  5.46s/it]
 41%|████      | 91507/225000 [166:32:50<175:24:49,  4.73s/it]
 41%|████      | 91508/225000 [166:32:53<155:03:25,  4.18s/it]
 41%|████      | 91509/225000 [166:32:55<138:03:17,  3.72s/it]
 41%|████      | 91510/225000 [166:32:58<130:00:57,  3.51s/it]
 41%|████      | 91511/225000 [166:33:01<122:11:26,  3.30s/it]
 41%|████      | 91512/225000 [166:33:07<153:36:39,  4.14s/it]
 41%|████      | 91513/225000 [166:33:15<188:23:01,  5.08s/it]
 41%|████      | 91514/225000 [166:33:18<169:37:27,  4.57s/it]
 41%|████      | 91515/225000 [166:33:24<186:27:15,  5.03s/it]
 41%|████      | 91516/225000 [166:33:27<165:51:43,  4.47s/it]
 41%|████      | 91517/225000 [166:33:30<146:46:54,  3.96s/it]
 41%|████      | 91518/225000 [166:33:37<178:42:51,  4.82s/it]
 41%|████      | 91519/225000 [166:33:39<152:57:45,  4.13s/it]
 41%|████      | 91520/225000 [166:33:42<137:16:58,  3.70s/it]
 41%|████      | 91521/225000 [166:33:45<126:35:45,  3.41s/it]
 41%|████      | 91522/225000 [166:33:48<129:45:07,  3.50s/it]
 41%|████      | 91523/225000 [166:33:52<127:38:19,  3.44s/it]
 41%|████      | 91524/225000 [166:33:55<127:15:19,  3.43s/it]
 41%|████      | 91525/225000 [166:33:59<126:39:28,  3.42s/it]
 41%|████      | 91526/225000 [166:34:02<126:27:50,  3.41s/it]
 41%|████      | 91527/225000 [166:34:09<170:26:18,  4.60s/it]
 41%|████      | 91528/225000 [166:34:12<152:58:17,  4.13s/it]
 41%|████      | 91529/225000 [166:34:15<139:19:54,  3.76s/it]
 41%|████      | 91530/225000 [166:34:18<130:00:59,  3.51s/it]
 41%|████      | 91531/225000 [166:34:21<121:18:37,  3.27s/it]
 41%|████      | 91532/225000 [166:34:27<156:03:48,  4.21s/it]
 41%|████      | 91533/225000 [166:34:37<215:35:14,  5.82s/it]
 41%|████      | 91534/225000 [166:34:39<180:47:48,  4.88s/it]
 41%|████      | 91535/225000 [166:34:42<156:50:49,  4.23s/it]
 41%|████      | 91536/225000 [166:34:46<146:57:31,  3.96s/it]
 41%|████      | 91537/225000 [166:34:49<141:19:47,  3.81s/it]
 41%|████      | 91538/225000 [166:34:53<143:10:48,  3.86s/it]
 41%|████      | 91539/225000 [166:34:57<142:06:51,  3.83s/it]
 41%|████      | 91540/225000 [166:35:00<139:45:07,  3.77s/it]
 41%|████      | 91541/225000 [166:35:04<140:25:47,  3.79s/it]
 41%|████      | 91542/225000 [166:35:08<136:03:44,  3.67s/it]
 41%|████      | 91543/225000 [166:35:11<138:26:15,  3.73s/it]
 41%|████      | 91544/225000 [166:35:19<184:14:29,  4.97s/it]
 41%|████      | 91545/225000 [166:35:24<175:59:31,  4.75s/it]
 41%|████      | 91546/225000 [166:35:27<164:37:20,  4.44s/it]
 41%|████      | 91547/225000 [166:35:31<152:03:16,  4.10s/it]
 41%|████      | 91548/225000 [166:35:34<143:11:40,  3.86s/it]
 41%|████      | 91549/225000 [166:35:38<151:07:23,  4.08s/it]
 41%|████      | 91550/225000 [166:35:46<184:53:24,  4.99s/it]
                                                              
{'loss': '0.2517', 'grad_norm': '4.899', 'learning_rate': '3.427e-05', 'epoch': '0.4069', 'num_input_tokens_seen': 272882768, 'train_runtime': '5.997e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91550/225000 [166:35:46<184:53:24,  4.99s/it]
 41%|████      | 91551/225000 [166:35:49<168:34:15,  4.55s/it]
 41%|████      | 91552/225000 [166:35:52<146:13:49,  3.94s/it]
 41%|████      | 91553/225000 [166:35:55<135:38:47,  3.66s/it]
 41%|████      | 91554/225000 [166:35:58<135:25:38,  3.65s/it]
 41%|████      | 91555/225000 [166:36:02<139:54:33,  3.77s/it]
 41%|████      | 91556/225000 [166:36:06<143:05:10,  3.86s/it]
 41%|████      | 91557/225000 [166:36:10<142:49:36,  3.85s/it]
 41%|████      | 91558/225000 [166:36:14<137:31:10,  3.71s/it]
 41%|████      | 91559/225000 [166:36:17<139:18:49,  3.76s/it]
 41%|████      | 91560/225000 [166:36:21<134:41:41,  3.63s/it]
 41%|████      | 91561/225000 [166:36:27<163:59:03,  4.42s/it]
 41%|████      | 91562/225000 [166:36:36<214:42:57,  5.79s/it]
 41%|████      | 91563/225000 [166:36:39<181:53:09,  4.91s/it]
 41%|████      | 91564/225000 [166:36:41<154:35:11,  4.17s/it]
 41%|████      | 91565/225000 [166:36:44<141:19:48,  3.81s/it]
 41%|████      | 91566/225000 [166:36:48<135:40:12,  3.66s/it]
 41%|████      | 91567/225000 [166:36:51<132:39:40,  3.58s/it]
 41%|████      | 91568/225000 [166:36:59<186:16:52,  5.03s/it]
 41%|████      | 91569/225000 [166:37:04<177:22:29,  4.79s/it]
 41%|████      | 91570/225000 [166:37:07<161:17:09,  4.35s/it]
 41%|████      | 91571/225000 [166:37:10<148:06:32,  4.00s/it]
 41%|████      | 91572/225000 [166:37:14<141:38:30,  3.82s/it]
 41%|████      | 91573/225000 [166:37:17<134:20:25,  3.62s/it]
 41%|████      | 91574/225000 [166:37:20<126:08:32,  3.40s/it]
 41%|████      | 91575/225000 [166:37:23<124:17:46,  3.35s/it]
 41%|████      | 91576/225000 [166:37:32<185:32:35,  5.01s/it]
 41%|████      | 91577/225000 [166:37:37<189:49:55,  5.12s/it]
 41%|████      | 91578/225000 [166:37:40<163:38:06,  4.42s/it]
 41%|████      | 91579/225000 [166:37:43<146:35:27,  3.96s/it]
 41%|████      | 91580/225000 [166:37:46<142:11:57,  3.84s/it]
 41%|████      | 91581/225000 [166:37:53<171:38:11,  4.63s/it]
 41%|████      | 91582/225000 [166:37:56<159:56:20,  4.32s/it]
 41%|████      | 91583/225000 [166:38:00<152:25:10,  4.11s/it]
 41%|████      | 91584/225000 [166:38:03<140:27:34,  3.79s/it]
 41%|████      | 91585/225000 [166:38:06<133:46:42,  3.61s/it]
 41%|████      | 91586/225000 [166:38:10<134:48:30,  3.64s/it]
 41%|████      | 91587/225000 [166:38:13<128:33:21,  3.47s/it]
 41%|████      | 91588/225000 [166:38:16<121:26:52,  3.28s/it]
 41%|████      | 91589/225000 [166:38:24<175:10:45,  4.73s/it]
 41%|████      | 91590/225000 [166:38:31<196:15:41,  5.30s/it]
 41%|████      | 91591/225000 [166:38:33<169:20:09,  4.57s/it]
 41%|████      | 91592/225000 [166:38:37<153:20:42,  4.14s/it]
 41%|████      | 91593/225000 [166:38:40<143:59:09,  3.89s/it]
 41%|████      | 91594/225000 [166:38:43<138:57:59,  3.75s/it]
 41%|████      | 91595/225000 [166:38:46<130:40:33,  3.53s/it]
 41%|████      | 91596/225000 [166:38:49<122:57:21,  3.32s/it]
 41%|████      | 91597/225000 [166:38:56<161:05:08,  4.35s/it]
 41%|████      | 91598/225000 [166:39:03<187:19:13,  5.06s/it]
 41%|████      | 91599/225000 [166:39:06<163:36:19,  4.42s/it]
 41%|████      | 91600/225000 [166:39:09<148:55:41,  4.02s/it]
                                                              
{'loss': '0.3042', 'grad_norm': '6.937', 'learning_rate': '3.425e-05', 'epoch': '0.4071', 'num_input_tokens_seen': 272972104, 'train_runtime': '5.999e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91600/225000 [166:39:09<148:55:41,  4.02s/it]
 41%|████      | 91601/225000 [166:39:14<167:17:51,  4.51s/it]
 41%|████      | 91602/225000 [166:39:18<156:09:36,  4.21s/it]
 41%|████      | 91603/225000 [166:39:22<149:55:57,  4.05s/it]
 41%|████      | 91604/225000 [166:39:25<144:59:47,  3.91s/it]
 41%|████      | 91605/225000 [166:39:29<140:01:13,  3.78s/it]
 41%|████      | 91606/225000 [166:39:32<131:16:31,  3.54s/it]
 41%|████      | 91607/225000 [166:39:35<130:37:50,  3.53s/it]
 41%|████      | 91608/225000 [166:39:41<161:11:27,  4.35s/it]
 41%|████      | 91609/225000 [166:39:50<207:40:11,  5.60s/it]
 41%|████      | 91610/225000 [166:39:53<175:16:07,  4.73s/it]
 41%|████      | 91611/225000 [166:39:56<156:55:13,  4.24s/it]
 41%|████      | 91612/225000 [166:39:59<147:56:53,  3.99s/it]
 41%|████      | 91613/225000 [166:40:02<133:46:11,  3.61s/it]
 41%|████      | 91614/225000 [166:40:05<129:29:13,  3.49s/it]
 41%|████      | 91615/225000 [166:40:08<128:28:54,  3.47s/it]
 41%|████      | 91616/225000 [166:40:16<173:14:37,  4.68s/it]
 41%|████      | 91617/225000 [166:40:23<203:05:12,  5.48s/it]
 41%|████      | 91618/225000 [166:40:26<170:15:08,  4.60s/it]
 41%|████      | 91619/225000 [166:40:28<145:18:28,  3.92s/it]
 41%|████      | 91620/225000 [166:40:35<179:31:48,  4.85s/it]
 41%|████      | 91621/225000 [166:40:38<158:06:07,  4.27s/it]
 41%|████      | 91622/225000 [166:40:42<157:22:45,  4.25s/it]
 41%|████      | 91623/225000 [166:40:49<181:18:15,  4.89s/it]
 41%|████      | 91624/225000 [166:40:51<156:27:22,  4.22s/it]
 41%|████      | 91625/225000 [166:40:54<141:13:35,  3.81s/it]
 41%|████      | 91626/225000 [166:40:57<126:40:05,  3.42s/it]
 41%|████      | 91627/225000 [166:41:00<122:58:11,  3.32s/it]
 41%|████      | 91628/225000 [166:41:06<159:03:10,  4.29s/it]
 41%|████      | 91629/225000 [166:41:11<163:52:50,  4.42s/it]
 41%|████      | 91630/225000 [166:41:14<150:20:55,  4.06s/it]
 41%|████      | 91631/225000 [166:41:17<134:44:46,  3.64s/it]
 41%|████      | 91632/225000 [166:41:19<121:57:52,  3.29s/it]
 41%|████      | 91633/225000 [166:41:22<116:31:47,  3.15s/it]
 41%|████      | 91634/225000 [166:41:30<170:44:46,  4.61s/it]
 41%|████      | 91635/225000 [166:41:34<166:23:46,  4.49s/it]
 41%|████      | 91636/225000 [166:41:37<143:18:51,  3.87s/it]
 41%|████      | 91637/225000 [166:41:44<174:58:43,  4.72s/it]
 41%|████      | 91638/225000 [166:41:49<178:59:44,  4.83s/it]
 41%|████      | 91639/225000 [166:41:51<154:00:06,  4.16s/it]
 41%|████      | 91640/225000 [166:41:54<135:35:20,  3.66s/it]
 41%|████      | 91641/225000 [166:41:56<124:07:40,  3.35s/it]
 41%|████      | 91642/225000 [166:41:59<117:58:13,  3.18s/it]
 41%|████      | 91643/225000 [166:42:08<176:46:29,  4.77s/it]
 41%|████      | 91644/225000 [166:42:13<186:08:38,  5.03s/it]
 41%|████      | 91645/225000 [166:42:15<154:58:59,  4.18s/it]
 41%|████      | 91646/225000 [166:42:18<140:37:14,  3.80s/it]
 41%|████      | 91647/225000 [166:42:24<155:42:51,  4.20s/it]
 41%|████      | 91648/225000 [166:42:26<140:26:17,  3.79s/it]
 41%|████      | 91649/225000 [166:42:30<135:40:04,  3.66s/it]
 41%|████      | 91650/225000 [166:42:33<129:26:24,  3.49s/it]
                                                              
{'loss': '0.2734', 'grad_norm': '3.517', 'learning_rate': '3.424e-05', 'epoch': '0.4073', 'num_input_tokens_seen': 273062144, 'train_runtime': '6.002e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91650/225000 [166:42:33<129:26:24,  3.49s/it]
 41%|████      | 91651/225000 [166:42:35<120:12:55,  3.25s/it]
 41%|████      | 91652/225000 [166:42:39<117:39:23,  3.18s/it]
 41%|████      | 91653/225000 [166:42:47<171:35:11,  4.63s/it]
 41%|████      | 91654/225000 [166:42:54<199:46:55,  5.39s/it]
 41%|████      | 91655/225000 [166:42:56<164:56:35,  4.45s/it]
 41%|████      | 91656/225000 [166:42:59<145:13:37,  3.92s/it]
 41%|████      | 91657/225000 [166:43:06<188:49:42,  5.10s/it]
 41%|████      | 91658/225000 [166:43:09<158:45:13,  4.29s/it]
 41%|████      | 91659/225000 [166:43:12<141:05:37,  3.81s/it]
 41%|████      | 91660/225000 [166:43:15<136:02:05,  3.67s/it]
 41%|████      | 91661/225000 [166:43:18<128:06:08,  3.46s/it]
 41%|████      | 91662/225000 [166:43:21<122:36:57,  3.31s/it]
 41%|████      | 91663/225000 [166:43:25<126:43:51,  3.42s/it]
 41%|████      | 91664/225000 [166:43:33<178:07:20,  4.81s/it]
 41%|████      | 91665/225000 [166:43:39<190:44:09,  5.15s/it]
 41%|████      | 91666/225000 [166:43:41<161:45:43,  4.37s/it]
 41%|████      | 91667/225000 [166:43:44<141:55:50,  3.83s/it]
 41%|████      | 91668/225000 [166:43:46<130:52:17,  3.53s/it]
 41%|████      | 91669/225000 [166:43:50<135:40:57,  3.66s/it]
 41%|████      | 91670/225000 [166:43:59<195:12:58,  5.27s/it]
 41%|████      | 91671/225000 [166:44:04<189:17:51,  5.11s/it]
 41%|████      | 91672/225000 [166:44:07<166:58:39,  4.51s/it]
 41%|████      | 91673/225000 [166:44:10<150:54:59,  4.07s/it]
 41%|████      | 91674/225000 [166:44:13<136:42:06,  3.69s/it]
 41%|████      | 91675/225000 [166:44:16<128:21:08,  3.47s/it]
 41%|████      | 91676/225000 [166:44:19<120:41:02,  3.26s/it]
 41%|████      | 91677/225000 [166:44:22<119:04:44,  3.22s/it]
 41%|████      | 91678/225000 [166:44:28<152:59:49,  4.13s/it]
 41%|████      | 91679/225000 [166:44:38<214:05:19,  5.78s/it]
 41%|████      | 91680/225000 [166:44:41<180:42:14,  4.88s/it]
 41%|████      | 91681/225000 [166:44:43<156:37:06,  4.23s/it]
 41%|████      | 91682/225000 [166:44:51<198:39:14,  5.36s/it]
 41%|████      | 91683/225000 [166:44:55<180:50:19,  4.88s/it]
 41%|████      | 91684/225000 [166:44:58<155:37:28,  4.20s/it]
 41%|████      | 91685/225000 [166:45:00<136:41:32,  3.69s/it]
 41%|████      | 91686/225000 [166:45:04<131:39:57,  3.56s/it]
 41%|████      | 91687/225000 [166:45:07<133:14:04,  3.60s/it]
 41%|████      | 91688/225000 [166:45:11<131:26:42,  3.55s/it]
 41%|████      | 91689/225000 [166:45:19<181:02:00,  4.89s/it]
 41%|████      | 91690/225000 [166:45:25<192:36:40,  5.20s/it]
 41%|████      | 91691/225000 [166:45:28<169:15:52,  4.57s/it]
 41%|████      | 91692/225000 [166:45:31<156:35:16,  4.23s/it]
 41%|████      | 91693/225000 [166:45:34<139:58:13,  3.78s/it]
 41%|████      | 91694/225000 [166:45:37<134:57:37,  3.64s/it]
 41%|████      | 91695/225000 [166:45:45<186:48:46,  5.05s/it]
 41%|████      | 91696/225000 [166:45:49<173:22:34,  4.68s/it]
 41%|████      | 91697/225000 [166:45:52<152:05:34,  4.11s/it]
 41%|████      | 91698/225000 [166:45:55<140:12:10,  3.79s/it]
 41%|████      | 91699/225000 [166:45:59<138:35:09,  3.74s/it]
 41%|████      | 91700/225000 [166:46:02<137:08:06,  3.70s/it]
                                                              
{'loss': '0.2791', 'grad_norm': '3.554', 'learning_rate': '3.422e-05', 'epoch': '0.4076', 'num_input_tokens_seen': 273152728, 'train_runtime': '6.004e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91700/225000 [166:46:02<137:08:06,  3.70s/it]
 41%|████      | 91701/225000 [166:46:06<133:32:31,  3.61s/it]
 41%|████      | 91702/225000 [166:46:09<131:14:10,  3.54s/it]
 41%|████      | 91703/225000 [166:46:19<199:45:58,  5.40s/it]
 41%|████      | 91704/225000 [166:46:23<189:53:16,  5.13s/it]
 41%|████      | 91705/225000 [166:46:27<177:05:46,  4.78s/it]
 41%|████      | 91706/225000 [166:46:31<162:17:32,  4.38s/it]
 41%|████      | 91707/225000 [166:46:35<156:58:55,  4.24s/it]
 41%|████      | 91708/225000 [166:46:38<144:20:05,  3.90s/it]
 41%|████      | 91709/225000 [166:46:41<138:45:59,  3.75s/it]
 41%|████      | 91710/225000 [166:46:45<137:35:55,  3.72s/it]
 41%|████      | 91711/225000 [166:46:48<133:00:11,  3.59s/it]
 41%|████      | 91712/225000 [166:46:52<133:38:07,  3.61s/it]
 41%|████      | 91713/225000 [166:46:56<137:00:10,  3.70s/it]
 41%|████      | 91714/225000 [166:46:59<134:50:28,  3.64s/it]
 41%|████      | 91715/225000 [166:47:03<137:24:54,  3.71s/it]
 41%|████      | 91716/225000 [166:47:07<144:54:40,  3.91s/it]
 41%|████      | 91717/225000 [166:47:11<136:28:53,  3.69s/it]
 41%|████      | 91718/225000 [166:47:19<183:39:41,  4.96s/it]
 41%|████      | 91719/225000 [166:47:28<230:13:34,  6.22s/it]
 41%|████      | 91720/225000 [166:47:31<202:50:17,  5.48s/it]
 41%|████      | 91721/225000 [166:47:35<183:04:24,  4.94s/it]
 41%|████      | 91722/225000 [166:47:39<167:47:10,  4.53s/it]
 41%|████      | 91723/225000 [166:47:42<151:17:18,  4.09s/it]
 41%|████      | 91724/225000 [166:47:46<147:26:46,  3.98s/it]
 41%|████      | 91725/225000 [166:47:54<194:41:19,  5.26s/it]
 41%|████      | 91726/225000 [166:47:59<193:30:10,  5.23s/it]
 41%|████      | 91727/225000 [166:48:01<163:26:00,  4.41s/it]
 41%|████      | 91728/225000 [166:48:05<151:52:24,  4.10s/it]
 41%|████      | 91729/225000 [166:48:08<143:58:21,  3.89s/it]
 41%|████      | 91730/225000 [166:48:11<135:56:28,  3.67s/it]
 41%|████      | 91731/225000 [166:48:14<127:28:45,  3.44s/it]
 41%|████      | 91732/225000 [166:48:22<169:22:57,  4.58s/it]
 41%|████      | 91733/225000 [166:48:27<177:47:24,  4.80s/it]
 41%|████      | 91734/225000 [166:48:30<156:08:14,  4.22s/it]
 41%|████      | 91735/225000 [166:48:33<149:44:58,  4.05s/it]
 41%|████      | 91736/225000 [166:48:36<137:37:37,  3.72s/it]
 41%|████      | 91737/225000 [166:48:39<125:19:17,  3.39s/it]
 41%|████      | 91738/225000 [166:48:46<169:03:43,  4.57s/it]
 41%|████      | 91739/225000 [166:48:53<189:40:51,  5.12s/it]
 41%|████      | 91740/225000 [166:48:55<162:47:55,  4.40s/it]
 41%|████      | 91741/225000 [166:49:01<176:05:41,  4.76s/it]
 41%|████      | 91742/225000 [166:49:05<172:08:35,  4.65s/it]
 41%|████      | 91743/225000 [166:49:08<152:57:34,  4.13s/it]
 41%|████      | 91744/225000 [166:49:11<137:10:00,  3.71s/it]
 41%|████      | 91745/225000 [166:49:14<127:25:45,  3.44s/it]
 41%|████      | 91746/225000 [166:49:17<126:38:44,  3.42s/it]
 41%|████      | 91747/225000 [166:49:26<189:30:45,  5.12s/it]
 41%|████      | 91748/225000 [166:49:29<167:07:28,  4.52s/it]
 41%|████      | 91749/225000 [166:49:32<145:34:54,  3.93s/it]
 41%|████      | 91750/225000 [166:49:38<171:44:54,  4.64s/it]
                                                              
{'loss': '0.2753', 'grad_norm': '5.93', 'learning_rate': '3.42e-05', 'epoch': '0.4078', 'num_input_tokens_seen': 273244328, 'train_runtime': '6.006e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91750/225000 [166:49:38<171:44:54,  4.64s/it]
 41%|████      | 91751/225000 [166:49:41<155:21:45,  4.20s/it]
 41%|████      | 91752/225000 [166:49:44<142:01:48,  3.84s/it]
 41%|████      | 91753/225000 [166:49:47<132:10:57,  3.57s/it]
 41%|████      | 91754/225000 [166:49:50<127:26:17,  3.44s/it]
 41%|████      | 91755/225000 [166:49:53<120:12:50,  3.25s/it]
 41%|████      | 91756/225000 [166:49:56<114:25:34,  3.09s/it]
 41%|████      | 91757/225000 [166:50:03<155:10:19,  4.19s/it]
 41%|████      | 91758/225000 [166:50:10<192:37:21,  5.20s/it]
 41%|████      | 91759/225000 [166:50:13<168:40:40,  4.56s/it]
 41%|████      | 91760/225000 [166:50:16<145:29:19,  3.93s/it]
 41%|████      | 91761/225000 [166:50:19<139:10:56,  3.76s/it]
 41%|████      | 91762/225000 [166:50:27<182:28:21,  4.93s/it]
 41%|████      | 91763/225000 [166:50:33<197:50:41,  5.35s/it]
 41%|████      | 91764/225000 [166:50:36<168:23:14,  4.55s/it]
 41%|████      | 91765/225000 [166:50:43<195:59:28,  5.30s/it]
 41%|████      | 91766/225000 [166:50:46<176:33:15,  4.77s/it]
 41%|████      | 91767/225000 [166:50:50<159:24:38,  4.31s/it]
 41%|████      | 91768/225000 [166:50:53<152:01:20,  4.11s/it]
 41%|████      | 91769/225000 [166:50:56<137:55:13,  3.73s/it]
 41%|████      | 91770/225000 [166:51:00<138:56:52,  3.75s/it]
 41%|████      | 91771/225000 [166:51:03<134:11:29,  3.63s/it]
 41%|████      | 91772/225000 [166:51:07<130:34:53,  3.53s/it]
 41%|████      | 91773/225000 [166:51:10<125:19:07,  3.39s/it]
 41%|████      | 91774/225000 [166:51:13<126:06:04,  3.41s/it]
 41%|████      | 91775/225000 [166:51:17<127:00:04,  3.43s/it]
 41%|████      | 91776/225000 [166:51:26<190:47:53,  5.16s/it]
 41%|████      | 91777/225000 [166:51:32<200:50:36,  5.43s/it]
 41%|████      | 91778/225000 [166:51:35<174:57:58,  4.73s/it]
 41%|████      | 91779/225000 [166:51:38<159:07:04,  4.30s/it]
 41%|████      | 91780/225000 [166:51:42<155:00:16,  4.19s/it]
 41%|████      | 91781/225000 [166:51:46<145:36:20,  3.93s/it]
 41%|████      | 91782/225000 [166:51:52<172:10:17,  4.65s/it]
 41%|████      | 91783/225000 [166:51:59<201:18:23,  5.44s/it]
 41%|████      | 91784/225000 [166:52:02<171:25:35,  4.63s/it]
 41%|████      | 91785/225000 [166:52:05<150:19:35,  4.06s/it]
 41%|████      | 91786/225000 [166:52:08<144:31:12,  3.91s/it]
 41%|████      | 91787/225000 [166:52:16<191:20:18,  5.17s/it]
 41%|████      | 91788/225000 [166:52:19<164:49:00,  4.45s/it]
 41%|████      | 91789/225000 [166:52:22<147:37:22,  3.99s/it]
 41%|████      | 91790/225000 [166:52:26<147:31:13,  3.99s/it]
 41%|████      | 91791/225000 [166:52:32<174:35:39,  4.72s/it]
 41%|████      | 91792/225000 [166:52:35<156:52:07,  4.24s/it]
 41%|████      | 91793/225000 [166:52:38<143:06:07,  3.87s/it]
 41%|████      | 91794/225000 [166:52:42<138:49:39,  3.75s/it]
 41%|████      | 91795/225000 [166:52:45<134:04:47,  3.62s/it]
 41%|████      | 91796/225000 [166:52:49<130:23:37,  3.52s/it]
 41%|████      | 91797/225000 [166:52:52<133:10:09,  3.60s/it]
 41%|████      | 91798/225000 [166:53:02<197:04:48,  5.33s/it]
 41%|████      | 91799/225000 [166:53:06<190:30:58,  5.15s/it]
 41%|████      | 91800/225000 [166:53:09<164:33:03,  4.45s/it]
                                                              
{'loss': '0.2467', 'grad_norm': '5.864', 'learning_rate': '3.418e-05', 'epoch': '0.408', 'num_input_tokens_seen': 273338304, 'train_runtime': '6.008e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91800/225000 [166:53:09<164:33:03,  4.45s/it]
 41%|████      | 91801/225000 [166:53:12<144:18:08,  3.90s/it]
 41%|████      | 91802/225000 [166:53:18<167:21:27,  4.52s/it]
 41%|████      | 91803/225000 [166:53:24<188:11:27,  5.09s/it]
 41%|████      | 91804/225000 [166:53:27<166:56:19,  4.51s/it]
 41%|████      | 91805/225000 [166:53:31<155:11:16,  4.19s/it]
 41%|████      | 91806/225000 [166:53:39<195:44:40,  5.29s/it]
 41%|████      | 91807/225000 [166:53:41<164:41:55,  4.45s/it]
 41%|████      | 91808/225000 [166:53:47<174:26:11,  4.71s/it]
 41%|████      | 91809/225000 [166:53:52<182:48:11,  4.94s/it]
 41%|████      | 91810/225000 [166:53:55<160:37:05,  4.34s/it]
 41%|████      | 91811/225000 [166:54:01<178:01:41,  4.81s/it]
 41%|████      | 91812/225000 [166:54:04<163:37:37,  4.42s/it]
 41%|████      | 91813/225000 [166:54:08<150:00:23,  4.05s/it]
 41%|████      | 91814/225000 [166:54:11<137:44:42,  3.72s/it]
 41%|████      | 91815/225000 [166:54:14<137:19:21,  3.71s/it]
 41%|████      | 91816/225000 [166:54:18<138:59:18,  3.76s/it]
 41%|████      | 91817/225000 [166:54:22<137:05:26,  3.71s/it]
 41%|████      | 91818/225000 [166:54:25<129:03:46,  3.49s/it]
 41%|████      | 91819/225000 [166:54:28<126:22:11,  3.42s/it]
 41%|████      | 91820/225000 [166:54:31<126:40:17,  3.42s/it]
 41%|████      | 91821/225000 [166:54:35<134:27:18,  3.63s/it]
 41%|████      | 91822/225000 [166:54:43<172:36:49,  4.67s/it]
 41%|████      | 91823/225000 [166:54:51<209:23:54,  5.66s/it]
 41%|████      | 91824/225000 [166:54:53<172:51:11,  4.67s/it]
 41%|████      | 91825/225000 [166:54:56<154:45:02,  4.18s/it]
 41%|████      | 91826/225000 [166:55:00<151:32:28,  4.10s/it]
 41%|████      | 91827/225000 [166:55:03<143:26:42,  3.88s/it]
 41%|████      | 91828/225000 [166:55:06<136:15:54,  3.68s/it]
 41%|████      | 91829/225000 [166:55:10<131:59:47,  3.57s/it]
 41%|████      | 91830/225000 [166:55:13<129:34:10,  3.50s/it]
 41%|████      | 91831/225000 [166:55:16<128:36:32,  3.48s/it]
 41%|████      | 91832/225000 [166:55:20<130:11:43,  3.52s/it]
 41%|████      | 91833/225000 [166:55:24<128:59:01,  3.49s/it]
 41%|████      | 91834/225000 [166:55:31<174:44:52,  4.72s/it]
 41%|████      | 91835/225000 [166:55:39<207:04:56,  5.60s/it]
 41%|████      | 91836/225000 [166:55:41<172:26:07,  4.66s/it]
 41%|████      | 91837/225000 [166:55:44<153:19:13,  4.14s/it]
 41%|████      | 91838/225000 [166:55:49<156:51:11,  4.24s/it]
 41%|████      | 91839/225000 [166:55:55<177:04:29,  4.79s/it]
 41%|████      | 91840/225000 [166:55:59<167:27:20,  4.53s/it]
 41%|████      | 91841/225000 [166:56:03<161:09:52,  4.36s/it]
 41%|████      | 91842/225000 [166:56:06<151:46:21,  4.10s/it]
 41%|████      | 91843/225000 [166:56:11<158:40:44,  4.29s/it]
 41%|████      | 91844/225000 [166:56:16<162:55:58,  4.41s/it]
 41%|████      | 91845/225000 [166:56:19<153:10:00,  4.14s/it]
 41%|████      | 91846/225000 [166:56:23<149:13:41,  4.03s/it]
 41%|████      | 91847/225000 [166:56:27<145:52:38,  3.94s/it]
 41%|████      | 91848/225000 [166:56:30<143:41:00,  3.88s/it]
 41%|████      | 91849/225000 [166:56:34<146:40:30,  3.97s/it]
 41%|████      | 91850/225000 [166:56:38<141:30:54,  3.83s/it]
                                                              
{'loss': '0.2423', 'grad_norm': '5.412', 'learning_rate': '3.417e-05', 'epoch': '0.4082', 'num_input_tokens_seen': 273431128, 'train_runtime': '6.01e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91850/225000 [166:56:38<141:30:54,  3.83s/it]
 41%|████      | 91851/225000 [166:56:42<144:08:56,  3.90s/it]
 41%|████      | 91852/225000 [166:56:50<192:03:05,  5.19s/it]
 41%|████      | 91853/225000 [166:56:55<183:47:17,  4.97s/it]
 41%|████      | 91854/225000 [166:56:58<169:41:13,  4.59s/it]
 41%|████      | 91855/225000 [166:57:02<159:47:00,  4.32s/it]
 41%|████      | 91856/225000 [166:57:06<155:54:48,  4.22s/it]
 41%|████      | 91857/225000 [166:57:16<218:23:50,  5.91s/it]
 41%|████      | 91858/225000 [166:57:19<187:32:56,  5.07s/it]
 41%|████      | 91859/225000 [166:57:22<166:35:23,  4.50s/it]
 41%|████      | 91860/225000 [166:57:26<156:22:34,  4.23s/it]
 41%|████      | 91861/225000 [166:57:34<203:03:02,  5.49s/it]
 41%|████      | 91862/225000 [166:57:38<184:36:18,  4.99s/it]
 41%|████      | 91863/225000 [166:57:41<165:22:36,  4.47s/it]
 41%|████      | 91864/225000 [166:57:47<178:28:46,  4.83s/it]
 41%|████      | 91865/225000 [166:57:50<155:37:24,  4.21s/it]
 41%|████      | 91866/225000 [166:57:55<168:54:40,  4.57s/it]
 41%|████      | 91867/225000 [166:58:00<168:45:50,  4.56s/it]
 41%|████      | 91868/225000 [166:58:03<155:34:54,  4.21s/it]
 41%|████      | 91869/225000 [166:58:07<153:15:12,  4.14s/it]
 41%|████      | 91870/225000 [166:58:10<140:34:22,  3.80s/it]
 41%|████      | 91871/225000 [166:58:13<133:32:24,  3.61s/it]
 41%|████      | 91872/225000 [166:58:19<152:32:43,  4.13s/it]
 41%|████      | 91873/225000 [166:58:28<207:16:37,  5.61s/it]
 41%|████      | 91874/225000 [166:58:31<179:44:37,  4.86s/it]
 41%|████      | 91875/225000 [166:58:34<160:07:51,  4.33s/it]
 41%|████      | 91876/225000 [166:58:37<141:59:39,  3.84s/it]
 41%|████      | 91877/225000 [166:58:39<132:21:01,  3.58s/it]
 41%|████      | 91878/225000 [166:58:42<121:56:02,  3.30s/it]
 41%|████      | 91879/225000 [166:58:46<127:19:53,  3.44s/it]
 41%|████      | 91880/225000 [166:58:55<189:50:28,  5.13s/it]
 41%|████      | 91881/225000 [166:58:58<165:39:35,  4.48s/it]
 41%|████      | 91882/225000 [166:59:01<151:53:04,  4.11s/it]
 41%|████      | 91883/225000 [166:59:04<139:43:37,  3.78s/it]
 41%|████      | 91884/225000 [166:59:07<126:52:43,  3.43s/it]
 41%|████      | 91885/225000 [166:59:10<123:15:23,  3.33s/it]
 41%|████      | 91886/225000 [166:59:15<144:46:17,  3.92s/it]
 41%|████      | 91887/225000 [166:59:23<183:25:29,  4.96s/it]
 41%|████      | 91888/225000 [166:59:27<171:45:08,  4.65s/it]
 41%|████      | 91889/225000 [166:59:29<152:42:49,  4.13s/it]
 41%|████      | 91890/225000 [166:59:33<146:15:47,  3.96s/it]
 41%|████      | 91891/225000 [166:59:36<135:43:57,  3.67s/it]
 41%|████      | 91892/225000 [166:59:40<140:05:40,  3.79s/it]
 41%|████      | 91893/225000 [166:59:43<133:03:28,  3.60s/it]
 41%|████      | 91894/225000 [166:59:52<185:17:54,  5.01s/it]
 41%|████      | 91895/225000 [166:59:54<161:28:05,  4.37s/it]
 41%|████      | 91896/225000 [166:59:57<143:46:45,  3.89s/it]
 41%|████      | 91897/225000 [167:00:07<208:07:20,  5.63s/it]
 41%|████      | 91898/225000 [167:00:10<175:23:21,  4.74s/it]
 41%|████      | 91899/225000 [167:00:12<147:31:55,  3.99s/it]
 41%|████      | 91900/225000 [167:00:15<136:09:46,  3.68s/it]
                                                              
{'loss': '0.2463', 'grad_norm': '5.118', 'learning_rate': '3.415e-05', 'epoch': '0.4084', 'num_input_tokens_seen': 273524504, 'train_runtime': '6.012e+05', 'train_tokens_per_second': '455'}

 41%|████      | 91900/225000 [167:00:15<136:09:46,  3.68s/it]
 41%|████      | 91901/225000 [167:00:18<130:28:26,  3.53s/it]
 41%|████      | 91902/225000 [167:00:21<130:05:04,  3.52s/it]
 41%|████      | 91903/225000 [167:00:26<141:26:29,  3.83s/it]
 41%|████      | 91904/225000 [167:00:29<137:24:28,  3.72s/it]
 41%|████      | 91905/225000 [167:00:33<136:37:36,  3.70s/it]
 41%|████      | 91906/225000 [167:00:36<132:41:14,  3.59s/it]
 41%|████      | 91907/225000 [167:00:41<148:10:09,  4.01s/it]
 41%|████      | 91908/225000 [167:00:51<210:06:51,  5.68s/it]
 41%|████      | 91909/225000 [167:00:56<200:01:53,  5.41s/it]
 41%|████      | 91910/225000 [167:00:59<178:39:03,  4.83s/it]
 41%|████      | 91911/225000 [167:01:02<156:29:36,  4.23s/it]
 41%|████      | 91912/225000 [167:01:06<154:58:28,  4.19s/it]
 41%|████      | 91913/225000 [167:01:09<145:24:16,  3.93s/it]
 41%|████      | 91914/225000 [167:01:13<139:25:56,  3.77s/it]
 41%|████      | 91915/225000 [167:01:16<136:44:45,  3.70s/it]
 41%|████      | 91916/225000 [167:01:22<160:39:19,  4.35s/it]
 41%|████      | 91917/225000 [167:01:31<205:45:08,  5.57s/it]
 41%|████      | 91918/225000 [167:01:34<180:34:44,  4.88s/it]
 41%|████      | 91919/225000 [167:01:37<157:27:55,  4.26s/it]
 41%|████      | 91920/225000 [167:01:40<147:58:08,  4.00s/it]
 41%|████      | 91921/225000 [167:01:45<160:40:36,  4.35s/it]
 41%|████      | 91922/225000 [167:01:52<187:59:42,  5.09s/it]
 41%|████      | 91923/225000 [167:01:55<166:41:39,  4.51s/it]
 41%|████      | 91924/225000 [167:01:58<151:18:20,  4.09s/it]
 41%|████      | 91925/225000 [167:02:02<146:46:23,  3.97s/it]
 41%|████      | 91926/225000 [167:02:05<135:37:02,  3.67s/it]
 41%|████      | 91927/225000 [167:02:09<141:26:14,  3.83s/it]
 41%|████      | 91928/225000 [167:02:19<211:26:56,  5.72s/it]
 41%|████      | 91929/225000 [167:02:24<201:52:45,  5.46s/it]
 41%|████      | 91930/225000 [167:02:27<176:36:33,  4.78s/it]
 41%|████      | 91931/225000 [167:02:30<156:37:57,  4.24s/it]
 41%|████      | 91932/225000 [167:02:33<137:08:15,  3.71s/it]
 41%|████      | 91933/225000 [167:02:41<185:07:35,  5.01s/it]
 41%|████      | 91934/225000 [167:02:43<157:01:23,  4.25s/it]
 41%|████      | 91935/225000 [167:02:48<159:37:06,  4.32s/it]
 41%|████      | 91936/225000 [167:02:55<188:23:55,  5.10s/it]
 41%|████      | 91937/225000 [167:02:57<159:20:09,  4.31s/it]
 41%|████      | 91938/225000 [167:03:00<141:45:23,  3.84s/it]
 41%|████      | 91939/225000 [167:03:03<131:00:37,  3.54s/it]
 41%|████      | 91940/225000 [167:03:09<163:24:58,  4.42s/it]
 41%|████      | 91941/225000 [167:03:14<168:09:53,  4.55s/it]
 41%|████      | 91942/225000 [167:03:18<157:25:37,  4.26s/it]
 41%|████      | 91943/225000 [167:03:21<146:46:52,  3.97s/it]
 41%|████      | 91944/225000 [167:03:25<143:33:52,  3.88s/it]
 41%|████      | 91945/225000 [167:03:28<135:46:42,  3.67s/it]
 41%|████      | 91946/225000 [167:03:35<177:35:48,  4.81s/it]
 41%|████      | 91947/225000 [167:03:41<187:12:42,  5.07s/it]
 41%|████      | 91948/225000 [167:03:44<165:33:42,  4.48s/it]
 41%|████      | 91949/225000 [167:03:47<143:38:17,  3.89s/it]
 41%|████      | 91950/225000 [167:03:49<130:08:42,  3.52s/it]
                                                              
{'loss': '0.2648', 'grad_norm': '6.102', 'learning_rate': '3.413e-05', 'epoch': '0.4087', 'num_input_tokens_seen': 273618000, 'train_runtime': '6.014e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 91950/225000 [167:03:49<130:08:42,  3.52s/it]
 41%|████      | 91951/225000 [167:03:53<131:58:30,  3.57s/it]
 41%|████      | 91952/225000 [167:04:04<209:47:55,  5.68s/it]
 41%|████      | 91953/225000 [167:04:07<179:18:35,  4.85s/it]
 41%|████      | 91954/225000 [167:04:10<158:31:17,  4.29s/it]
 41%|████      | 91955/225000 [167:04:17<195:17:59,  5.28s/it]
 41%|████      | 91956/225000 [167:04:20<170:50:51,  4.62s/it]
 41%|████      | 91957/225000 [167:04:23<148:37:58,  4.02s/it]
 41%|████      | 91958/225000 [167:04:26<143:07:23,  3.87s/it]
 41%|████      | 91959/225000 [167:04:29<134:45:17,  3.65s/it]
 41%|████      | 91960/225000 [167:04:35<160:24:25,  4.34s/it]
 41%|████      | 91961/225000 [167:04:39<151:50:08,  4.11s/it]
 41%|████      | 91962/225000 [167:04:43<147:47:45,  4.00s/it]
 41%|████      | 91963/225000 [167:04:46<144:04:18,  3.90s/it]
 41%|████      | 91964/225000 [167:04:51<150:24:34,  4.07s/it]
 41%|████      | 91965/225000 [167:04:55<149:50:03,  4.05s/it]
 41%|████      | 91966/225000 [167:04:58<143:32:17,  3.88s/it]
 41%|████      | 91967/225000 [167:05:02<142:18:28,  3.85s/it]
 41%|████      | 91968/225000 [167:05:08<161:03:11,  4.36s/it]
 41%|████      | 91969/225000 [167:05:18<224:36:09,  6.08s/it]
 41%|████      | 91970/225000 [167:05:22<205:15:57,  5.55s/it]
 41%|████      | 91971/225000 [167:05:25<170:54:51,  4.63s/it]
 41%|████      | 91972/225000 [167:05:28<157:04:21,  4.25s/it]
 41%|████      | 91973/225000 [167:05:31<148:38:57,  4.02s/it]
 41%|████      | 91974/225000 [167:05:35<146:03:28,  3.95s/it]
 41%|████      | 91975/225000 [167:05:39<142:25:34,  3.85s/it]
 41%|████      | 91976/225000 [167:05:43<148:06:50,  4.01s/it]
 41%|████      | 91977/225000 [167:05:47<144:13:56,  3.90s/it]
 41%|████      | 91978/225000 [167:05:52<156:38:39,  4.24s/it]
 41%|████      | 91979/225000 [167:05:55<145:45:07,  3.94s/it]
 41%|████      | 91980/225000 [167:05:59<145:37:21,  3.94s/it]
 41%|████      | 91981/225000 [167:06:03<149:20:29,  4.04s/it]
 41%|████      | 91982/225000 [167:06:08<150:26:56,  4.07s/it]
 41%|████      | 91983/225000 [167:06:12<155:09:48,  4.20s/it]
 41%|████      | 91984/225000 [167:06:20<202:10:08,  5.47s/it]
 41%|████      | 91985/225000 [167:06:28<221:09:33,  5.99s/it]
 41%|████      | 91986/225000 [167:06:32<203:53:50,  5.52s/it]
 41%|████      | 91987/225000 [167:06:35<179:40:58,  4.86s/it]
 41%|████      | 91988/225000 [167:06:40<173:57:24,  4.71s/it]
 41%|████      | 91989/225000 [167:06:43<159:15:28,  4.31s/it]
 41%|████      | 91990/225000 [167:06:47<152:23:02,  4.12s/it]
 41%|████      | 91991/225000 [167:06:51<150:46:57,  4.08s/it]
 41%|████      | 91992/225000 [167:06:54<142:51:41,  3.87s/it]
 41%|████      | 91993/225000 [167:06:58<145:32:32,  3.94s/it]
 41%|████      | 91994/225000 [167:07:05<172:16:51,  4.66s/it]
 41%|████      | 91995/225000 [167:07:13<218:44:25,  5.92s/it]
 41%|████      | 91996/225000 [167:07:16<186:15:52,  5.04s/it]
 41%|████      | 91997/225000 [167:07:19<160:02:24,  4.33s/it]
 41%|████      | 91998/225000 [167:07:23<156:59:57,  4.25s/it]
 41%|████      | 91999/225000 [167:07:29<170:29:43,  4.61s/it]
 41%|████      | 92000/225000 [167:07:33<165:17:51,  4.47s/it]
                                                              
{'loss': '0.2658', 'grad_norm': '5.334', 'learning_rate': '3.412e-05', 'epoch': '0.4089', 'num_input_tokens_seen': 273715992, 'train_runtime': '6.017e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92000/225000 [167:07:33<165:17:51,  4.47s/it]
 41%|████      | 92001/225000 [167:07:36<147:36:15,  4.00s/it]
 41%|████      | 92002/225000 [167:07:39<139:14:24,  3.77s/it]
 41%|████      | 92003/225000 [167:07:48<200:59:07,  5.44s/it]
 41%|████      | 92004/225000 [167:07:54<208:19:06,  5.64s/it]
 41%|████      | 92005/225000 [167:07:57<172:29:35,  4.67s/it]
 41%|████      | 92006/225000 [167:08:00<155:32:40,  4.21s/it]
 41%|████      | 92007/225000 [167:08:03<145:05:55,  3.93s/it]
 41%|████      | 92008/225000 [167:08:07<138:43:15,  3.76s/it]
 41%|████      | 92009/225000 [167:08:10<139:46:45,  3.78s/it]
 41%|████      | 92010/225000 [167:08:14<139:07:51,  3.77s/it]
 41%|████      | 92011/225000 [167:08:18<136:45:36,  3.70s/it]
 41%|████      | 92012/225000 [167:08:21<130:58:40,  3.55s/it]
 41%|████      | 92013/225000 [167:08:24<130:46:12,  3.54s/it]
 41%|████      | 92014/225000 [167:08:31<163:36:49,  4.43s/it]
 41%|████      | 92015/225000 [167:08:42<232:23:41,  6.29s/it]
 41%|████      | 92016/225000 [167:08:44<194:45:57,  5.27s/it]
 41%|████      | 92017/225000 [167:08:49<183:45:54,  4.97s/it]
 41%|████      | 92018/225000 [167:08:52<167:40:55,  4.54s/it]
 41%|████      | 92019/225000 [167:08:56<154:50:59,  4.19s/it]
 41%|████      | 92020/225000 [167:08:59<149:47:39,  4.06s/it]
 41%|████      | 92021/225000 [167:09:06<177:24:33,  4.80s/it]
 41%|████      | 92022/225000 [167:09:09<161:39:27,  4.38s/it]
 41%|████      | 92023/225000 [167:09:13<149:51:30,  4.06s/it]
 41%|████      | 92024/225000 [167:09:15<133:21:35,  3.61s/it]
 41%|████      | 92025/225000 [167:09:18<130:04:16,  3.52s/it]
 41%|████      | 92026/225000 [167:09:24<150:57:25,  4.09s/it]
 41%|████      | 92027/225000 [167:09:33<209:27:11,  5.67s/it]
 41%|████      | 92028/225000 [167:09:36<178:07:44,  4.82s/it]
 41%|████      | 92029/225000 [167:09:41<178:28:27,  4.83s/it]
 41%|████      | 92030/225000 [167:09:47<189:32:31,  5.13s/it]
 41%|████      | 92031/225000 [167:09:50<165:55:11,  4.49s/it]
 41%|████      | 92032/225000 [167:09:53<150:12:17,  4.07s/it]
 41%|████      | 92033/225000 [167:09:55<131:35:00,  3.56s/it]
 41%|████      | 92034/225000 [167:10:04<184:46:03,  5.00s/it]
 41%|████      | 92035/225000 [167:10:07<169:34:51,  4.59s/it]
 41%|████      | 92036/225000 [167:10:10<154:06:19,  4.17s/it]
 41%|████      | 92037/225000 [167:10:14<143:00:48,  3.87s/it]
 41%|████      | 92038/225000 [167:10:19<163:58:14,  4.44s/it]
 41%|████      | 92039/225000 [167:10:25<179:09:09,  4.85s/it]
 41%|████      | 92040/225000 [167:10:28<159:41:08,  4.32s/it]
 41%|████      | 92041/225000 [167:10:31<143:19:57,  3.88s/it]
 41%|████      | 92042/225000 [167:10:35<148:35:40,  4.02s/it]
 41%|████      | 92043/225000 [167:10:44<202:20:31,  5.48s/it]
 41%|████      | 92044/225000 [167:10:48<177:46:37,  4.81s/it]
 41%|████      | 92045/225000 [167:10:50<153:56:37,  4.17s/it]
 41%|████      | 92046/225000 [167:10:54<145:44:21,  3.95s/it]
 41%|████      | 92047/225000 [167:11:00<176:06:51,  4.77s/it]
 41%|████      | 92048/225000 [167:11:04<168:44:10,  4.57s/it]
 41%|████      | 92049/225000 [167:11:08<153:22:53,  4.15s/it]
 41%|████      | 92050/225000 [167:11:11<146:33:51,  3.97s/it]
                                                              
{'loss': '0.2349', 'grad_norm': '4.915', 'learning_rate': '3.41e-05', 'epoch': '0.4091', 'num_input_tokens_seen': 273812016, 'train_runtime': '6.019e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92050/225000 [167:11:11<146:33:51,  3.97s/it]
 41%|████      | 92051/225000 [167:11:19<187:16:53,  5.07s/it]
 41%|████      | 92052/225000 [167:11:24<187:22:18,  5.07s/it]
 41%|████      | 92053/225000 [167:11:28<173:48:54,  4.71s/it]
 41%|████      | 92054/225000 [167:11:32<163:46:49,  4.43s/it]
 41%|████      | 92055/225000 [167:11:34<146:56:30,  3.98s/it]
 41%|████      | 92056/225000 [167:11:38<141:42:35,  3.84s/it]
 41%|████      | 92057/225000 [167:11:42<139:24:50,  3.78s/it]
 41%|████      | 92058/225000 [167:11:46<142:23:06,  3.86s/it]
 41%|████      | 92059/225000 [167:11:49<133:31:57,  3.62s/it]
 41%|████      | 92060/225000 [167:11:52<133:08:31,  3.61s/it]
 41%|████      | 92061/225000 [167:11:56<131:53:54,  3.57s/it]
 41%|████      | 92062/225000 [167:12:06<209:32:06,  5.67s/it]
 41%|████      | 92063/225000 [167:12:12<207:38:49,  5.62s/it]
 41%|████      | 92064/225000 [167:12:15<175:07:08,  4.74s/it]
 41%|████      | 92065/225000 [167:12:17<151:58:19,  4.12s/it]
 41%|████      | 92066/225000 [167:12:20<142:22:44,  3.86s/it]
 41%|████      | 92067/225000 [167:12:24<133:58:55,  3.63s/it]
 41%|████      | 92068/225000 [167:12:31<171:35:48,  4.65s/it]
 41%|████      | 92069/225000 [167:12:37<188:10:56,  5.10s/it]
 41%|████      | 92070/225000 [167:12:40<164:51:50,  4.46s/it]
 41%|████      | 92071/225000 [167:12:43<151:50:32,  4.11s/it]
 41%|████      | 92072/225000 [167:12:50<180:33:57,  4.89s/it]
 41%|████      | 92073/225000 [167:12:56<199:44:15,  5.41s/it]
 41%|████      | 92074/225000 [167:12:59<166:47:45,  4.52s/it]
 41%|████      | 92075/225000 [167:13:03<167:05:10,  4.53s/it]
 41%|████      | 92076/225000 [167:13:07<162:13:30,  4.39s/it]
 41%|████      | 92077/225000 [167:13:15<192:49:08,  5.22s/it]
 41%|████      | 92078/225000 [167:13:17<163:13:57,  4.42s/it]
 41%|████      | 92079/225000 [167:13:24<191:57:57,  5.20s/it]
 41%|████      | 92080/225000 [167:13:27<169:23:29,  4.59s/it]
 41%|████      | 92081/225000 [167:13:30<151:26:51,  4.10s/it]
 41%|████      | 92082/225000 [167:13:33<140:14:40,  3.80s/it]
 41%|████      | 92083/225000 [167:13:37<136:49:56,  3.71s/it]
 41%|████      | 92084/225000 [167:13:43<161:06:29,  4.36s/it]
 41%|████      | 92085/225000 [167:13:51<199:06:30,  5.39s/it]
 41%|████      | 92086/225000 [167:13:54<178:32:45,  4.84s/it]
 41%|████      | 92087/225000 [167:13:58<168:44:09,  4.57s/it]
 41%|████      | 92088/225000 [167:14:01<149:00:52,  4.04s/it]
 41%|████      | 92089/225000 [167:14:04<135:42:09,  3.68s/it]
 41%|████      | 92090/225000 [167:14:07<131:21:45,  3.56s/it]
 41%|████      | 92091/225000 [167:14:12<150:09:38,  4.07s/it]
 41%|████      | 92092/225000 [167:14:21<199:10:04,  5.39s/it]
 41%|████      | 92093/225000 [167:14:25<182:49:28,  4.95s/it]
 41%|████      | 92094/225000 [167:14:28<164:29:04,  4.46s/it]
 41%|████      | 92095/225000 [167:14:31<148:40:15,  4.03s/it]
 41%|████      | 92096/225000 [167:14:34<136:27:25,  3.70s/it]
 41%|████      | 92097/225000 [167:14:41<179:34:09,  4.86s/it]
 41%|████      | 92098/225000 [167:14:47<189:38:16,  5.14s/it]
 41%|████      | 92099/225000 [167:14:50<162:22:41,  4.40s/it]
 41%|████      | 92100/225000 [167:14:54<154:49:01,  4.19s/it]
                                                              
{'loss': '0.2415', 'grad_norm': '4.47', 'learning_rate': '3.408e-05', 'epoch': '0.4093', 'num_input_tokens_seen': 273908856, 'train_runtime': '6.021e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92100/225000 [167:14:54<154:49:01,  4.19s/it]
 41%|████      | 92101/225000 [167:14:57<145:05:10,  3.93s/it]
 41%|████      | 92102/225000 [167:15:04<184:18:55,  4.99s/it]
 41%|████      | 92103/225000 [167:15:07<162:12:59,  4.39s/it]
 41%|████      | 92104/225000 [167:15:11<151:54:07,  4.11s/it]
 41%|████      | 92105/225000 [167:15:14<141:04:33,  3.82s/it]
 41%|████      | 92106/225000 [167:15:17<135:45:19,  3.68s/it]
 41%|████      | 92107/225000 [167:15:26<192:20:32,  5.21s/it]
 41%|████      | 92108/225000 [167:15:30<178:51:09,  4.85s/it]
 41%|████      | 92109/225000 [167:15:35<177:34:35,  4.81s/it]
 41%|████      | 92110/225000 [167:15:37<153:24:37,  4.16s/it]
 41%|████      | 92111/225000 [167:15:40<136:24:08,  3.70s/it]
 41%|████      | 92112/225000 [167:15:43<129:05:41,  3.50s/it]
 41%|████      | 92113/225000 [167:15:46<121:51:40,  3.30s/it]
 41%|████      | 92114/225000 [167:15:53<159:38:30,  4.32s/it]
 41%|████      | 92115/225000 [167:16:00<195:58:09,  5.31s/it]
 41%|████      | 92116/225000 [167:16:04<177:02:27,  4.80s/it]
 41%|████      | 92117/225000 [167:16:07<163:12:05,  4.42s/it]
 41%|████      | 92118/225000 [167:16:11<159:05:48,  4.31s/it]
 41%|████      | 92119/225000 [167:16:15<152:16:29,  4.13s/it]
 41%|████      | 92120/225000 [167:16:19<146:26:08,  3.97s/it]
 41%|████      | 92121/225000 [167:16:22<143:51:00,  3.90s/it]
 41%|████      | 92122/225000 [167:16:26<136:40:10,  3.70s/it]
 41%|████      | 92123/225000 [167:16:30<140:22:12,  3.80s/it]
 41%|████      | 92124/225000 [167:16:33<139:05:35,  3.77s/it]
 41%|████      | 92125/225000 [167:16:37<135:20:31,  3.67s/it]
 41%|████      | 92126/225000 [167:16:40<131:32:54,  3.56s/it]
 41%|████      | 92127/225000 [167:16:49<188:59:55,  5.12s/it]
 41%|████      | 92128/225000 [167:16:55<196:23:25,  5.32s/it]
 41%|████      | 92129/225000 [167:16:58<172:22:27,  4.67s/it]
 41%|████      | 92130/225000 [167:17:01<153:29:47,  4.16s/it]
 41%|████      | 92131/225000 [167:17:09<198:16:06,  5.37s/it]
 41%|████      | 92132/225000 [167:17:14<197:05:06,  5.34s/it]
 41%|████      | 92133/225000 [167:17:17<164:07:31,  4.45s/it]
 41%|████      | 92134/225000 [167:17:20<156:43:47,  4.25s/it]
 41%|████      | 92135/225000 [167:17:24<144:02:45,  3.90s/it]
 41%|████      | 92136/225000 [167:17:27<137:38:14,  3.73s/it]
 41%|████      | 92137/225000 [167:17:31<136:51:29,  3.71s/it]
 41%|████      | 92138/225000 [167:17:34<138:08:48,  3.74s/it]
 41%|████      | 92139/225000 [167:17:38<133:21:45,  3.61s/it]
 41%|████      | 92140/225000 [167:17:42<139:06:33,  3.77s/it]
 41%|████      | 92141/225000 [167:17:47<158:47:21,  4.30s/it]
 41%|████      | 92142/225000 [167:17:56<210:21:17,  5.70s/it]
 41%|████      | 92143/225000 [167:18:01<202:35:15,  5.49s/it]
 41%|████      | 92144/225000 [167:18:04<171:29:48,  4.65s/it]
 41%|████      | 92145/225000 [167:18:12<207:49:49,  5.63s/it]
 41%|████      | 92146/225000 [167:18:15<174:58:57,  4.74s/it]
 41%|████      | 92147/225000 [167:18:21<192:07:30,  5.21s/it]
 41%|████      | 92148/225000 [167:18:24<168:27:05,  4.56s/it]
 41%|████      | 92149/225000 [167:18:26<145:45:29,  3.95s/it]
 41%|████      | 92150/225000 [167:18:31<147:56:25,  4.01s/it]
                                                              
{'loss': '0.2604', 'grad_norm': '4.534', 'learning_rate': '3.407e-05', 'epoch': '0.4096', 'num_input_tokens_seen': 274005384, 'train_runtime': '6.023e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92150/225000 [167:18:31<147:56:25,  4.01s/it]
 41%|████      | 92151/225000 [167:18:39<194:28:06,  5.27s/it]
 41%|████      | 92152/225000 [167:18:43<179:55:29,  4.88s/it]
 41%|████      | 92153/225000 [167:18:47<168:27:53,  4.57s/it]
 41%|████      | 92154/225000 [167:18:50<157:20:01,  4.26s/it]
 41%|████      | 92155/225000 [167:18:55<160:21:44,  4.35s/it]
 41%|████      | 92156/225000 [167:18:58<150:36:35,  4.08s/it]
 41%|████      | 92157/225000 [167:19:04<170:55:16,  4.63s/it]
 41%|████      | 92158/225000 [167:19:10<183:28:01,  4.97s/it]
 41%|████      | 92159/225000 [167:19:14<177:17:51,  4.80s/it]
 41%|████      | 92160/225000 [167:19:18<167:18:56,  4.53s/it]
 41%|████      | 92161/225000 [167:19:22<155:13:01,  4.21s/it]
 41%|████      | 92162/225000 [167:19:25<148:43:20,  4.03s/it]
 41%|████      | 92163/225000 [167:19:28<139:16:14,  3.77s/it]
 41%|████      | 92164/225000 [167:19:32<138:45:05,  3.76s/it]
 41%|████      | 92165/225000 [167:19:37<145:44:24,  3.95s/it]
 41%|████      | 92166/225000 [167:19:40<138:00:29,  3.74s/it]
 41%|████      | 92167/225000 [167:19:43<133:25:12,  3.62s/it]
 41%|████      | 92168/225000 [167:19:50<171:27:08,  4.65s/it]
 41%|████      | 92169/225000 [167:19:55<175:02:46,  4.74s/it]
 41%|████      | 92170/225000 [167:19:59<164:05:56,  4.45s/it]
 41%|████      | 92171/225000 [167:20:03<157:14:26,  4.26s/it]
 41%|████      | 92172/225000 [167:20:09<181:12:24,  4.91s/it]
 41%|████      | 92173/225000 [167:20:13<165:18:49,  4.48s/it]
 41%|████      | 92174/225000 [167:20:17<161:25:42,  4.38s/it]
 41%|████      | 92175/225000 [167:20:21<158:23:59,  4.29s/it]
 41%|████      | 92176/225000 [167:20:24<150:43:40,  4.09s/it]
 41%|████      | 92177/225000 [167:20:29<152:02:31,  4.12s/it]
 41%|████      | 92178/225000 [167:20:32<142:47:51,  3.87s/it]
 41%|████      | 92179/225000 [167:20:36<140:44:44,  3.81s/it]
 41%|████      | 92180/225000 [167:20:41<155:05:59,  4.20s/it]
 41%|████      | 92181/225000 [167:20:50<214:07:17,  5.80s/it]
 41%|████      | 92182/225000 [167:20:56<211:29:03,  5.73s/it]
 41%|████      | 92183/225000 [167:20:59<177:12:19,  4.80s/it]
 41%|████      | 92184/225000 [167:21:02<158:44:56,  4.30s/it]
 41%|████      | 92185/225000 [167:21:05<146:27:08,  3.97s/it]
 41%|████      | 92186/225000 [167:21:08<138:56:56,  3.77s/it]
 41%|████      | 92187/225000 [167:21:12<140:04:55,  3.80s/it]
 41%|████      | 92188/225000 [167:21:16<146:10:26,  3.96s/it]
 41%|████      | 92189/225000 [167:21:23<181:12:06,  4.91s/it]
 41%|████      | 92190/225000 [167:21:31<205:20:25,  5.57s/it]
 41%|████      | 92191/225000 [167:21:34<176:57:19,  4.80s/it]
 41%|████      | 92192/225000 [167:21:37<167:11:11,  4.53s/it]
 41%|████      | 92193/225000 [167:21:44<193:48:09,  5.25s/it]
 41%|████      | 92194/225000 [167:21:50<202:27:25,  5.49s/it]
 41%|████      | 92195/225000 [167:21:53<172:42:56,  4.68s/it]
 41%|████      | 92196/225000 [167:22:01<207:45:43,  5.63s/it]
 41%|████      | 92197/225000 [167:22:05<192:55:21,  5.23s/it]
 41%|████      | 92198/225000 [167:22:09<171:28:51,  4.65s/it]
 41%|████      | 92199/225000 [167:22:12<158:04:14,  4.29s/it]
 41%|████      | 92200/225000 [167:22:15<144:50:02,  3.93s/it]
                                                              
{'loss': '0.2289', 'grad_norm': '7.947', 'learning_rate': '3.405e-05', 'epoch': '0.4098', 'num_input_tokens_seen': 274103544, 'train_runtime': '6.025e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92200/225000 [167:22:15<144:50:02,  3.93s/it]
 41%|████      | 92201/225000 [167:22:18<131:37:26,  3.57s/it]
 41%|████      | 92202/225000 [167:22:21<126:37:29,  3.43s/it]
 41%|████      | 92203/225000 [167:22:29<179:44:34,  4.87s/it]
 41%|████      | 92204/225000 [167:22:35<191:10:03,  5.18s/it]
 41%|████      | 92205/225000 [167:22:39<175:37:54,  4.76s/it]
 41%|████      | 92206/225000 [167:22:43<165:49:51,  4.50s/it]
 41%|████      | 92207/225000 [167:22:46<152:18:17,  4.13s/it]
 41%|████      | 92208/225000 [167:22:49<143:37:43,  3.89s/it]
 41%|████      | 92209/225000 [167:22:53<144:38:05,  3.92s/it]
 41%|████      | 92210/225000 [167:22:57<139:08:08,  3.77s/it]
 41%|████      | 92211/225000 [167:23:00<130:22:52,  3.53s/it]
 41%|████      | 92212/225000 [167:23:05<142:49:32,  3.87s/it]
 41%|████      | 92213/225000 [167:23:14<206:00:31,  5.59s/it]
 41%|████      | 92214/225000 [167:23:19<194:58:22,  5.29s/it]
 41%|████      | 92215/225000 [167:23:23<189:20:24,  5.13s/it]
 41%|████      | 92216/225000 [167:23:29<189:13:26,  5.13s/it]
 41%|████      | 92217/225000 [167:23:34<187:04:58,  5.07s/it]
 41%|████      | 92218/225000 [167:23:36<162:20:29,  4.40s/it]
 41%|████      | 92219/225000 [167:23:40<150:45:36,  4.09s/it]
 41%|████      | 92220/225000 [167:23:46<175:11:07,  4.75s/it]
 41%|████      | 92221/225000 [167:23:49<154:25:10,  4.19s/it]
 41%|████      | 92222/225000 [167:23:52<142:19:05,  3.86s/it]
 41%|████      | 92223/225000 [167:23:59<180:24:23,  4.89s/it]
 41%|████      | 92224/225000 [167:24:05<193:45:41,  5.25s/it]
 41%|████      | 92225/225000 [167:24:09<174:53:22,  4.74s/it]
 41%|████      | 92226/225000 [167:24:13<167:55:32,  4.55s/it]
 41%|████      | 92227/225000 [167:24:17<158:46:41,  4.31s/it]
 41%|████      | 92228/225000 [167:24:20<152:00:55,  4.12s/it]
 41%|████      | 92229/225000 [167:24:24<143:23:18,  3.89s/it]
 41%|████      | 92230/225000 [167:24:27<138:29:46,  3.76s/it]
 41%|████      | 92231/225000 [167:24:31<138:04:14,  3.74s/it]
 41%|████      | 92232/225000 [167:24:38<173:05:11,  4.69s/it]
 41%|████      | 92233/225000 [167:24:43<175:55:12,  4.77s/it]
 41%|████      | 92234/225000 [167:24:47<171:12:40,  4.64s/it]
 41%|████      | 92235/225000 [167:24:50<155:45:36,  4.22s/it]
 41%|████      | 92236/225000 [167:24:54<152:19:07,  4.13s/it]
 41%|████      | 92237/225000 [167:24:58<148:23:28,  4.02s/it]
 41%|████      | 92238/225000 [167:25:02<149:24:03,  4.05s/it]
 41%|████      | 92239/225000 [167:25:06<148:46:07,  4.03s/it]
 41%|████      | 92240/225000 [167:25:11<155:15:49,  4.21s/it]
 41%|████      | 92241/225000 [167:25:15<154:35:03,  4.19s/it]
 41%|████      | 92242/225000 [167:25:19<155:23:36,  4.21s/it]
 41%|████      | 92243/225000 [167:25:23<149:15:20,  4.05s/it]
 41%|████      | 92244/225000 [167:25:27<150:53:22,  4.09s/it]
 41%|████      | 92245/225000 [167:25:32<163:38:31,  4.44s/it]
 41%|████      | 92246/225000 [167:25:41<214:37:12,  5.82s/it]
 41%|████      | 92247/225000 [167:25:49<229:24:17,  6.22s/it]
 41%|████      | 92248/225000 [167:25:51<192:30:12,  5.22s/it]
 41%|████      | 92249/225000 [167:25:54<162:41:45,  4.41s/it]
 41%|████      | 92250/225000 [167:26:02<203:13:16,  5.51s/it]
                                                              
{'loss': '0.2143', 'grad_norm': '3.47', 'learning_rate': '3.403e-05', 'epoch': '0.41', 'num_input_tokens_seen': 274203176, 'train_runtime': '6.028e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92250/225000 [167:26:02<203:13:16,  5.51s/it]
 41%|████      | 92251/225000 [167:26:05<170:33:36,  4.63s/it]
 41%|████      | 92252/225000 [167:26:07<147:33:40,  4.00s/it]
 41%|████      | 92253/225000 [167:26:13<165:09:42,  4.48s/it]
 41%|████      | 92254/225000 [167:26:19<186:55:21,  5.07s/it]
 41%|████      | 92255/225000 [167:26:23<170:31:27,  4.62s/it]
 41%|████      | 92256/225000 [167:26:26<157:43:42,  4.28s/it]
 41%|████      | 92257/225000 [167:26:30<151:08:38,  4.10s/it]
 41%|████      | 92258/225000 [167:26:34<148:21:41,  4.02s/it]
 41%|████      | 92259/225000 [167:26:38<149:07:04,  4.04s/it]
 41%|████      | 92260/225000 [167:26:41<143:28:43,  3.89s/it]
 41%|████      | 92261/225000 [167:26:50<200:00:36,  5.42s/it]
 41%|████      | 92262/225000 [167:26:55<186:21:40,  5.05s/it]
 41%|████      | 92263/225000 [167:26:57<162:33:54,  4.41s/it]
 41%|████      | 92264/225000 [167:27:05<198:04:45,  5.37s/it]
 41%|████      | 92265/225000 [167:27:11<200:39:37,  5.44s/it]
 41%|████      | 92266/225000 [167:27:14<176:56:34,  4.80s/it]
 41%|████      | 92267/225000 [167:27:17<153:21:36,  4.16s/it]
 41%|████      | 92268/225000 [167:27:26<207:25:16,  5.63s/it]
 41%|████      | 92269/225000 [167:27:29<180:39:54,  4.90s/it]
 41%|████      | 92270/225000 [167:27:32<161:55:30,  4.39s/it]
 41%|████      | 92271/225000 [167:27:35<149:21:43,  4.05s/it]
 41%|████      | 92272/225000 [167:27:38<137:22:18,  3.73s/it]
 41%|████      | 92273/225000 [167:27:42<136:48:09,  3.71s/it]
 41%|████      | 92274/225000 [167:27:47<150:19:42,  4.08s/it]
 41%|████      | 92275/225000 [167:27:55<193:16:33,  5.24s/it]
 41%|████      | 92276/225000 [167:27:58<164:00:29,  4.45s/it]
 41%|████      | 92277/225000 [167:28:01<154:40:40,  4.20s/it]
 41%|████      | 92278/225000 [167:28:08<183:56:45,  4.99s/it]
 41%|████      | 92279/225000 [167:28:12<172:38:42,  4.68s/it]
 41%|████      | 92280/225000 [167:28:15<157:59:18,  4.29s/it]
 41%|████      | 92281/225000 [167:28:19<150:19:30,  4.08s/it]
 41%|████      | 92282/225000 [167:28:29<214:23:52,  5.82s/it]
 41%|████      | 92283/225000 [167:28:32<181:53:01,  4.93s/it]
 41%|████      | 92284/225000 [167:28:34<157:46:54,  4.28s/it]
 41%|████      | 92285/225000 [167:28:43<202:30:16,  5.49s/it]
 41%|████      | 92286/225000 [167:28:46<174:22:05,  4.73s/it]
 41%|████      | 92287/225000 [167:28:53<205:13:49,  5.57s/it]
 41%|████      | 92288/225000 [167:28:57<181:41:39,  4.93s/it]
 41%|████      | 92289/225000 [167:29:01<173:01:50,  4.69s/it]
 41%|████      | 92290/225000 [167:29:04<161:48:01,  4.39s/it]
 41%|████      | 92291/225000 [167:29:07<144:29:26,  3.92s/it]
 41%|████      | 92292/225000 [167:29:11<139:12:07,  3.78s/it]
 41%|████      | 92293/225000 [167:29:14<135:34:41,  3.68s/it]
 41%|████      | 92294/225000 [167:29:17<131:43:15,  3.57s/it]
 41%|████      | 92295/225000 [167:29:21<127:07:07,  3.45s/it]
 41%|████      | 92296/225000 [167:29:27<159:06:55,  4.32s/it]
 41%|████      | 92297/225000 [167:29:35<195:16:51,  5.30s/it]
 41%|████      | 92298/225000 [167:29:37<168:45:49,  4.58s/it]
 41%|████      | 92299/225000 [167:29:40<150:42:54,  4.09s/it]
 41%|████      | 92300/225000 [167:29:44<141:36:53,  3.84s/it]
                                                              
{'loss': '0.2389', 'grad_norm': '4.77', 'learning_rate': '3.401e-05', 'epoch': '0.4102', 'num_input_tokens_seen': 274302888, 'train_runtime': '6.03e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92300/225000 [167:29:44<141:36:53,  3.84s/it]
 41%|████      | 92301/225000 [167:29:47<139:28:38,  3.78s/it]
 41%|████      | 92302/225000 [167:29:55<182:21:24,  4.95s/it]
 41%|████      | 92303/225000 [167:30:01<194:22:44,  5.27s/it]
 41%|████      | 92304/225000 [167:30:04<169:18:51,  4.59s/it]
 41%|████      | 92305/225000 [167:30:11<199:46:36,  5.42s/it]
 41%|████      | 92306/225000 [167:30:15<185:02:36,  5.02s/it]
 41%|████      | 92307/225000 [167:30:22<196:38:03,  5.33s/it]
 41%|████      | 92308/225000 [167:30:25<173:31:10,  4.71s/it]
 41%|████      | 92309/225000 [167:30:28<153:38:18,  4.17s/it]
 41%|████      | 92310/225000 [167:30:31<144:21:32,  3.92s/it]
 41%|████      | 92311/225000 [167:30:39<185:57:43,  5.05s/it]
 41%|████      | 92312/225000 [167:30:44<192:11:19,  5.21s/it]
 41%|████      | 92313/225000 [167:30:48<171:25:15,  4.65s/it]
 41%|████      | 92314/225000 [167:30:50<149:41:17,  4.06s/it]
 41%|████      | 92315/225000 [167:30:53<138:09:00,  3.75s/it]
 41%|████      | 92316/225000 [167:30:57<136:38:08,  3.71s/it]
 41%|████      | 92317/225000 [167:31:04<173:23:39,  4.70s/it]
 41%|████      | 92318/225000 [167:31:12<204:28:17,  5.55s/it]
 41%|████      | 92319/225000 [167:31:14<173:07:44,  4.70s/it]
 41%|████      | 92320/225000 [167:31:20<183:42:03,  4.98s/it]
 41%|████      | 92321/225000 [167:31:23<164:19:36,  4.46s/it]
 41%|████      | 92322/225000 [167:31:28<173:38:33,  4.71s/it]
 41%|████      | 92323/225000 [167:31:32<163:47:59,  4.44s/it]
 41%|████      | 92324/225000 [167:31:36<159:36:07,  4.33s/it]
 41%|████      | 92325/225000 [167:31:40<156:32:34,  4.25s/it]
 41%|████      | 92326/225000 [167:31:44<151:12:24,  4.10s/it]
 41%|████      | 92327/225000 [167:31:48<154:06:41,  4.18s/it]
 41%|████      | 92328/225000 [167:31:52<143:38:46,  3.90s/it]
 41%|████      | 92329/225000 [167:31:55<141:48:55,  3.85s/it]
 41%|████      | 92330/225000 [167:32:05<204:14:28,  5.54s/it]
 41%|████      | 92331/225000 [167:32:12<219:25:01,  5.95s/it]
 41%|████      | 92332/225000 [167:32:14<182:10:27,  4.94s/it]
 41%|████      | 92333/225000 [167:32:17<156:54:12,  4.26s/it]
 41%|████      | 92334/225000 [167:32:21<154:57:50,  4.21s/it]
 41%|████      | 92335/225000 [167:32:30<202:44:16,  5.50s/it]
 41%|████      | 92336/225000 [167:32:33<175:23:48,  4.76s/it]
 41%|████      | 92337/225000 [167:32:37<164:40:27,  4.47s/it]
 41%|████      | 92338/225000 [167:32:43<191:45:20,  5.20s/it]
 41%|████      | 92339/225000 [167:32:47<172:03:40,  4.67s/it]
 41%|████      | 92340/225000 [167:32:51<161:51:52,  4.39s/it]
 41%|████      | 92341/225000 [167:32:54<155:16:03,  4.21s/it]
 41%|████      | 92342/225000 [167:32:58<146:45:34,  3.98s/it]
 41%|████      | 92343/225000 [167:33:04<176:04:22,  4.78s/it]
 41%|████      | 92344/225000 [167:33:11<192:15:10,  5.22s/it]
 41%|████      | 92345/225000 [167:33:14<167:55:46,  4.56s/it]
 41%|████      | 92346/225000 [167:33:17<148:58:10,  4.04s/it]
 41%|████      | 92347/225000 [167:33:20<144:00:47,  3.91s/it]
 41%|████      | 92348/225000 [167:33:25<151:01:32,  4.10s/it]
 41%|████      | 92349/225000 [167:33:29<154:54:01,  4.20s/it]
 41%|████      | 92350/225000 [167:33:33<145:29:16,  3.95s/it]
                                                              
{'loss': '0.2393', 'grad_norm': '15.23', 'learning_rate': '3.4e-05', 'epoch': '0.4104', 'num_input_tokens_seen': 274403032, 'train_runtime': '6.032e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92350/225000 [167:33:33<145:29:16,  3.95s/it]
 41%|████      | 92351/225000 [167:33:42<201:05:06,  5.46s/it]
 41%|████      | 92352/225000 [167:33:48<207:43:46,  5.64s/it]
 41%|████      | 92353/225000 [167:33:51<179:21:51,  4.87s/it]
 41%|████      | 92354/225000 [167:33:54<161:03:49,  4.37s/it]
 41%|████      | 92355/225000 [167:33:57<149:45:51,  4.06s/it]
 41%|████      | 92356/225000 [167:34:02<159:20:01,  4.32s/it]
 41%|████      | 92357/225000 [167:34:07<168:29:53,  4.57s/it]
 41%|████      | 92358/225000 [167:34:12<165:25:57,  4.49s/it]
 41%|████      | 92359/225000 [167:34:17<179:54:56,  4.88s/it]
 41%|████      | 92360/225000 [167:34:23<182:35:54,  4.96s/it]
 41%|████      | 92361/225000 [167:34:26<169:45:56,  4.61s/it]
 41%|████      | 92362/225000 [167:34:30<161:01:58,  4.37s/it]
 41%|████      | 92363/225000 [167:34:34<153:48:57,  4.17s/it]
 41%|████      | 92364/225000 [167:34:38<158:10:45,  4.29s/it]
 41%|████      | 92365/225000 [167:34:42<154:31:02,  4.19s/it]
 41%|████      | 92366/225000 [167:34:47<156:49:44,  4.26s/it]
 41%|████      | 92367/225000 [167:34:51<153:36:31,  4.17s/it]
 41%|████      | 92368/225000 [167:34:54<148:39:25,  4.03s/it]
 41%|████      | 92369/225000 [167:34:58<147:44:24,  4.01s/it]
 41%|████      | 92370/225000 [167:35:02<139:55:24,  3.80s/it]
 41%|████      | 92371/225000 [167:35:05<137:39:28,  3.74s/it]
 41%|████      | 92372/225000 [167:35:12<166:33:51,  4.52s/it]
 41%|████      | 92373/225000 [167:35:20<209:24:02,  5.68s/it]
 41%|████      | 92374/225000 [167:35:24<186:40:08,  5.07s/it]
 41%|████      | 92375/225000 [167:35:28<179:29:18,  4.87s/it]
 41%|████      | 92376/225000 [167:35:33<175:44:12,  4.77s/it]
 41%|████      | 92377/225000 [167:35:43<233:37:22,  6.34s/it]
 41%|████      | 92378/225000 [167:35:46<195:05:56,  5.30s/it]
 41%|████      | 92379/225000 [167:35:49<179:04:38,  4.86s/it]
 41%|████      | 92380/225000 [167:35:56<196:28:09,  5.33s/it]
 41%|████      | 92381/225000 [167:35:59<172:12:57,  4.67s/it]
 41%|████      | 92382/225000 [167:36:03<162:44:37,  4.42s/it]
 41%|████      | 92383/225000 [167:36:06<152:19:45,  4.14s/it]
 41%|████      | 92384/225000 [167:36:10<148:47:10,  4.04s/it]
 41%|████      | 92385/225000 [167:36:14<149:26:28,  4.06s/it]
 41%|████      | 92386/225000 [167:36:18<146:39:28,  3.98s/it]
 41%|████      | 92387/225000 [167:36:27<198:04:09,  5.38s/it]
 41%|████      | 92388/225000 [167:36:32<204:12:00,  5.54s/it]
 41%|████      | 92389/225000 [167:36:36<181:26:37,  4.93s/it]
 41%|████      | 92390/225000 [167:36:39<157:44:07,  4.28s/it]
 41%|████      | 92391/225000 [167:36:45<177:27:04,  4.82s/it]
 41%|████      | 92392/225000 [167:36:56<243:27:03,  6.61s/it]
 41%|████      | 92393/225000 [167:36:58<201:33:54,  5.47s/it]
 41%|████      | 92394/225000 [167:37:01<173:11:55,  4.70s/it]
 41%|████      | 92395/225000 [167:37:04<154:59:18,  4.21s/it]
 41%|████      | 92396/225000 [167:37:13<203:35:25,  5.53s/it]
 41%|████      | 92397/225000 [167:37:16<173:43:05,  4.72s/it]
 41%|████      | 92398/225000 [167:37:19<153:39:18,  4.17s/it]
 41%|████      | 92399/225000 [167:37:26<183:25:22,  4.98s/it]
 41%|████      | 92400/225000 [167:37:31<189:25:19,  5.14s/it]
                                                              
{'loss': '0.239', 'grad_norm': '5.262', 'learning_rate': '3.398e-05', 'epoch': '0.4107', 'num_input_tokens_seen': 274504632, 'train_runtime': '6.035e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92400/225000 [167:37:31<189:25:19,  5.14s/it]
 41%|████      | 92401/225000 [167:37:34<168:42:45,  4.58s/it]
 41%|████      | 92402/225000 [167:37:40<179:18:26,  4.87s/it]
 41%|████      | 92403/225000 [167:37:43<160:50:07,  4.37s/it]
 41%|████      | 92404/225000 [167:37:46<145:29:37,  3.95s/it]
 41%|████      | 92405/225000 [167:37:49<137:54:54,  3.74s/it]
 41%|████      | 92406/225000 [167:37:56<169:32:30,  4.60s/it]
 41%|████      | 92407/225000 [167:38:03<195:21:14,  5.30s/it]
 41%|████      | 92408/225000 [167:38:06<175:00:31,  4.75s/it]
 41%|████      | 92409/225000 [167:38:09<155:01:31,  4.21s/it]
 41%|████      | 92410/225000 [167:38:12<143:22:29,  3.89s/it]
 41%|████      | 92411/225000 [167:38:16<140:40:32,  3.82s/it]
 41%|████      | 92412/225000 [167:38:19<134:11:56,  3.64s/it]
 41%|████      | 92413/225000 [167:38:23<129:31:35,  3.52s/it]
 41%|████      | 92414/225000 [167:38:31<181:23:04,  4.92s/it]
 41%|████      | 92415/225000 [167:38:34<167:18:56,  4.54s/it]
 41%|████      | 92416/225000 [167:38:37<149:46:55,  4.07s/it]
 41%|████      | 92417/225000 [167:38:40<138:56:55,  3.77s/it]
 41%|████      | 92418/225000 [167:38:48<185:03:46,  5.03s/it]
 41%|████      | 92419/225000 [167:38:55<202:29:25,  5.50s/it]
 41%|████      | 92420/225000 [167:38:58<179:17:00,  4.87s/it]
 41%|████      | 92421/225000 [167:39:02<164:22:26,  4.46s/it]
 41%|████      | 92422/225000 [167:39:05<147:47:10,  4.01s/it]
 41%|████      | 92423/225000 [167:39:08<136:50:25,  3.72s/it]
 41%|████      | 92424/225000 [167:39:11<130:25:27,  3.54s/it]
 41%|████      | 92425/225000 [167:39:15<132:13:27,  3.59s/it]
 41%|████      | 92426/225000 [167:39:22<167:29:23,  4.55s/it]
 41%|████      | 92427/225000 [167:39:29<194:02:56,  5.27s/it]
 41%|████      | 92428/225000 [167:39:32<177:28:50,  4.82s/it]
 41%|████      | 92429/225000 [167:39:36<166:47:34,  4.53s/it]
 41%|████      | 92430/225000 [167:39:40<157:06:07,  4.27s/it]
 41%|████      | 92431/225000 [167:39:43<149:58:43,  4.07s/it]
 41%|████      | 92432/225000 [167:39:48<157:24:25,  4.27s/it]
 41%|████      | 92433/225000 [167:40:00<236:29:22,  6.42s/it]
 41%|████      | 92434/225000 [167:40:02<195:43:44,  5.32s/it]
 41%|████      | 92435/225000 [167:40:05<171:08:24,  4.65s/it]
 41%|████      | 92436/225000 [167:40:10<165:58:37,  4.51s/it]
 41%|████      | 92437/225000 [167:40:14<160:55:11,  4.37s/it]
 41%|████      | 92438/225000 [167:40:18<164:14:51,  4.46s/it]
 41%|████      | 92439/225000 [167:40:22<160:24:51,  4.36s/it]
 41%|████      | 92440/225000 [167:40:26<152:46:08,  4.15s/it]
 41%|████      | 92441/225000 [167:40:29<143:54:35,  3.91s/it]
 41%|████      | 92442/225000 [167:40:33<142:41:29,  3.88s/it]
 41%|████      | 92443/225000 [167:40:39<161:41:49,  4.39s/it]
 41%|████      | 92444/225000 [167:40:43<155:41:42,  4.23s/it]
 41%|████      | 92445/225000 [167:40:46<148:32:01,  4.03s/it]
 41%|████      | 92446/225000 [167:40:50<143:46:32,  3.90s/it]
 41%|████      | 92447/225000 [167:40:54<143:00:50,  3.88s/it]
 41%|████      | 92448/225000 [167:40:58<145:38:03,  3.96s/it]
 41%|████      | 92449/225000 [167:41:07<204:27:38,  5.55s/it]
 41%|████      | 92450/225000 [167:41:12<196:31:48,  5.34s/it]
                                                              
{'loss': '0.2248', 'grad_norm': '4.28', 'learning_rate': '3.396e-05', 'epoch': '0.4109', 'num_input_tokens_seen': 274604976, 'train_runtime': '6.037e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92450/225000 [167:41:12<196:31:48,  5.34s/it]
 41%|████      | 92451/225000 [167:41:16<182:26:41,  4.96s/it]
 41%|████      | 92452/225000 [167:41:20<172:09:58,  4.68s/it]
 41%|████      | 92453/225000 [167:41:24<164:34:47,  4.47s/it]
 41%|████      | 92454/225000 [167:41:33<215:37:45,  5.86s/it]
 41%|████      | 92455/225000 [167:41:37<192:07:11,  5.22s/it]
 41%|████      | 92456/225000 [167:41:41<175:54:59,  4.78s/it]
 41%|████      | 92457/225000 [167:41:44<164:31:18,  4.47s/it]
 41%|████      | 92458/225000 [167:41:48<154:33:39,  4.20s/it]
 41%|████      | 92459/225000 [167:41:55<186:38:48,  5.07s/it]
 41%|████      | 92460/225000 [167:42:02<206:35:13,  5.61s/it]
 41%|████      | 92461/225000 [167:42:05<179:13:04,  4.87s/it]
 41%|████      | 92462/225000 [167:42:08<160:59:21,  4.37s/it]
 41%|████      | 92463/225000 [167:42:11<141:28:02,  3.84s/it]
 41%|████      | 92464/225000 [167:42:14<135:46:58,  3.69s/it]
 41%|████      | 92465/225000 [167:42:24<200:33:01,  5.45s/it]
 41%|████      | 92466/225000 [167:42:27<173:48:28,  4.72s/it]
 41%|████      | 92467/225000 [167:42:30<162:27:01,  4.41s/it]
 41%|████      | 92468/225000 [167:42:34<148:53:21,  4.04s/it]
 41%|████      | 92469/225000 [167:42:37<142:35:44,  3.87s/it]
 41%|████      | 92470/225000 [167:42:41<144:33:31,  3.93s/it]
 41%|████      | 92471/225000 [167:42:49<190:30:59,  5.18s/it]
 41%|████      | 92472/225000 [167:42:57<223:04:56,  6.06s/it]
 41%|████      | 92473/225000 [167:43:00<190:14:12,  5.17s/it]
 41%|████      | 92474/225000 [167:43:04<173:07:38,  4.70s/it]
 41%|████      | 92475/225000 [167:43:07<159:14:40,  4.33s/it]
 41%|████      | 92476/225000 [167:43:11<155:23:36,  4.22s/it]
 41%|████      | 92477/225000 [167:43:15<149:24:59,  4.06s/it]
 41%|████      | 92478/225000 [167:43:19<142:43:47,  3.88s/it]
 41%|████      | 92479/225000 [167:43:22<140:54:42,  3.83s/it]
 41%|████      | 92480/225000 [167:43:27<155:12:51,  4.22s/it]
 41%|████      | 92481/225000 [167:43:38<227:26:01,  6.18s/it]
 41%|████      | 92482/225000 [167:43:42<203:57:38,  5.54s/it]
 41%|████      | 92483/225000 [167:43:45<173:38:06,  4.72s/it]
 41%|████      | 92484/225000 [167:43:48<150:50:24,  4.10s/it]
 41%|████      | 92485/225000 [167:43:51<143:32:08,  3.90s/it]
 41%|████      | 92486/225000 [167:43:55<141:34:19,  3.85s/it]
 41%|████      | 92487/225000 [167:44:01<172:00:41,  4.67s/it]
 41%|████      | 92488/225000 [167:44:10<212:05:51,  5.76s/it]
 41%|████      | 92489/225000 [167:44:13<184:10:51,  5.00s/it]
 41%|████      | 92490/225000 [167:44:16<164:16:05,  4.46s/it]
 41%|████      | 92491/225000 [167:44:19<150:59:28,  4.10s/it]
 41%|████      | 92492/225000 [167:44:24<160:25:23,  4.36s/it]
 41%|████      | 92493/225000 [167:44:33<206:34:02,  5.61s/it]
 41%|████      | 92494/225000 [167:44:37<185:02:11,  5.03s/it]
 41%|████      | 92495/225000 [167:44:40<163:46:29,  4.45s/it]
 41%|████      | 92496/225000 [167:44:43<150:37:02,  4.09s/it]
 41%|████      | 92497/225000 [167:44:47<146:06:21,  3.97s/it]
 41%|████      | 92498/225000 [167:44:51<148:37:17,  4.04s/it]
 41%|████      | 92499/225000 [167:44:58<185:57:51,  5.05s/it]
 41%|████      | 92500/225000 [167:45:05<207:42:43,  5.64s/it]
                                                              
{'loss': '0.2276', 'grad_norm': '6.315', 'learning_rate': '3.395e-05', 'epoch': '0.4111', 'num_input_tokens_seen': 274706208, 'train_runtime': '6.039e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92500/225000 [167:45:05<207:42:43,  5.64s/it]
 41%|████      | 92501/225000 [167:45:08<179:01:49,  4.86s/it]
 41%|████      | 92502/225000 [167:45:11<158:03:50,  4.29s/it]
 41%|████      | 92503/225000 [167:45:17<177:11:44,  4.81s/it]
 41%|████      | 92504/225000 [167:45:24<196:54:57,  5.35s/it]
 41%|████      | 92505/225000 [167:45:27<170:13:10,  4.63s/it]
 41%|████      | 92506/225000 [167:45:30<155:51:54,  4.24s/it]
 41%|████      | 92507/225000 [167:45:38<193:20:17,  5.25s/it]
 41%|████      | 92508/225000 [167:45:41<170:49:25,  4.64s/it]
 41%|████      | 92509/225000 [167:45:48<191:44:05,  5.21s/it]
 41%|████      | 92510/225000 [167:45:50<165:10:46,  4.49s/it]
 41%|████      | 92511/225000 [167:45:53<142:53:06,  3.88s/it]
 41%|████      | 92512/225000 [167:46:01<187:30:10,  5.09s/it]
 41%|████      | 92513/225000 [167:46:04<171:33:51,  4.66s/it]
 41%|████      | 92514/225000 [167:46:07<150:50:38,  4.10s/it]
 41%|████      | 92515/225000 [167:46:10<134:00:59,  3.64s/it]
 41%|████      | 92516/225000 [167:46:12<122:30:37,  3.33s/it]
 41%|████      | 92517/225000 [167:46:18<147:50:20,  4.02s/it]
 41%|████      | 92518/225000 [167:46:26<195:15:24,  5.31s/it]
 41%|████      | 92519/225000 [167:46:29<165:25:12,  4.50s/it]
 41%|████      | 92520/225000 [167:46:32<149:40:57,  4.07s/it]
 41%|████      | 92521/225000 [167:46:36<144:24:35,  3.92s/it]
 41%|████      | 92522/225000 [167:46:40<145:07:22,  3.94s/it]
 41%|████      | 92523/225000 [167:46:49<207:17:48,  5.63s/it]
 41%|████      | 92524/225000 [167:46:53<182:30:50,  4.96s/it]
 41%|████      | 92525/225000 [167:46:56<170:57:11,  4.65s/it]
 41%|████      | 92526/225000 [167:47:01<170:27:39,  4.63s/it]
 41%|████      | 92527/225000 [167:47:06<169:53:39,  4.62s/it]
 41%|████      | 92528/225000 [167:47:09<160:55:39,  4.37s/it]
 41%|████      | 92529/225000 [167:47:13<155:13:39,  4.22s/it]
 41%|████      | 92530/225000 [167:47:17<148:13:06,  4.03s/it]
 41%|████      | 92531/225000 [167:47:21<148:53:27,  4.05s/it]
 41%|████      | 92532/225000 [167:47:25<147:28:01,  4.01s/it]
 41%|████      | 92533/225000 [167:47:29<150:27:12,  4.09s/it]
 41%|████      | 92534/225000 [167:47:39<214:48:11,  5.84s/it]
 41%|████      | 92535/225000 [167:47:43<191:34:29,  5.21s/it]
 41%|████      | 92536/225000 [167:47:46<164:38:19,  4.47s/it]
 41%|████      | 92537/225000 [167:47:49<152:40:15,  4.15s/it]
 41%|████      | 92538/225000 [167:47:53<147:39:18,  4.01s/it]
 41%|████      | 92539/225000 [167:47:58<167:26:56,  4.55s/it]
 41%|████      | 92540/225000 [167:48:03<170:13:51,  4.63s/it]
 41%|████      | 92541/225000 [167:48:07<162:36:57,  4.42s/it]
 41%|████      | 92542/225000 [167:48:11<157:03:20,  4.27s/it]
 41%|████      | 92543/225000 [167:48:16<160:07:30,  4.35s/it]
 41%|████      | 92544/225000 [167:48:19<152:37:50,  4.15s/it]
 41%|████      | 92545/225000 [167:48:24<154:25:10,  4.20s/it]
 41%|████      | 92546/225000 [167:48:29<167:48:21,  4.56s/it]
 41%|████      | 92547/225000 [167:48:36<199:11:58,  5.41s/it]
 41%|████      | 92548/225000 [167:48:43<210:48:09,  5.73s/it]
 41%|████      | 92549/225000 [167:48:47<188:56:25,  5.14s/it]
 41%|████      | 92550/225000 [167:48:51<176:31:42,  4.80s/it]
                                                              
{'loss': '0.2389', 'grad_norm': '5.699', 'learning_rate': '3.393e-05', 'epoch': '0.4113', 'num_input_tokens_seen': 274807520, 'train_runtime': '6.041e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92550/225000 [167:48:51<176:31:42,  4.80s/it]
 41%|████      | 92551/225000 [167:48:55<174:00:01,  4.73s/it]
 41%|████      | 92552/225000 [167:49:00<177:03:28,  4.81s/it]
 41%|████      | 92553/225000 [167:49:08<203:42:54,  5.54s/it]
 41%|████      | 92554/225000 [167:49:10<173:05:13,  4.70s/it]
 41%|████      | 92555/225000 [167:49:18<201:46:43,  5.48s/it]
 41%|████      | 92556/225000 [167:49:22<188:15:58,  5.12s/it]
 41%|████      | 92557/225000 [167:49:27<189:06:19,  5.14s/it]
 41%|████      | 92558/225000 [167:49:30<165:57:52,  4.51s/it]
 41%|████      | 92559/225000 [167:49:33<147:31:33,  4.01s/it]
 41%|████      | 92560/225000 [167:49:36<137:41:39,  3.74s/it]
 41%|████      | 92561/225000 [167:49:40<137:36:38,  3.74s/it]
 41%|████      | 92562/225000 [167:49:49<203:28:03,  5.53s/it]
 41%|████      | 92563/225000 [167:49:53<181:20:20,  4.93s/it]
 41%|████      | 92564/225000 [167:49:57<168:00:37,  4.57s/it]
 41%|████      | 92565/225000 [167:50:00<153:50:45,  4.18s/it]
 41%|████      | 92566/225000 [167:50:04<148:36:58,  4.04s/it]
 41%|████      | 92567/225000 [167:50:07<141:38:51,  3.85s/it]
 41%|████      | 92568/225000 [167:50:11<141:34:14,  3.85s/it]
 41%|████      | 92569/225000 [167:50:15<140:59:55,  3.83s/it]
 41%|████      | 92570/225000 [167:50:19<144:53:18,  3.94s/it]
 41%|████      | 92571/225000 [167:50:23<145:13:00,  3.95s/it]
 41%|████      | 92572/225000 [167:50:32<200:11:26,  5.44s/it]
 41%|████      | 92573/225000 [167:50:38<208:24:07,  5.67s/it]
 41%|████      | 92574/225000 [167:50:41<175:54:30,  4.78s/it]
 41%|████      | 92575/225000 [167:50:49<209:56:02,  5.71s/it]
 41%|████      | 92576/225000 [167:50:52<182:15:16,  4.95s/it]
 41%|████      | 92577/225000 [167:50:55<164:41:19,  4.48s/it]
 41%|████      | 92578/225000 [167:50:58<147:05:20,  4.00s/it]
 41%|████      | 92579/225000 [167:51:02<142:07:39,  3.86s/it]
 41%|████      | 92580/225000 [167:51:05<140:19:45,  3.82s/it]
 41%|████      | 92581/225000 [167:51:10<148:13:41,  4.03s/it]
 41%|████      | 92582/225000 [167:51:19<210:18:51,  5.72s/it]
 41%|████      | 92583/225000 [167:51:23<186:23:43,  5.07s/it]
 41%|████      | 92584/225000 [167:51:26<161:13:16,  4.38s/it]
 41%|████      | 92585/225000 [167:51:29<143:38:52,  3.91s/it]
 41%|████      | 92586/225000 [167:51:36<185:21:58,  5.04s/it]
 41%|████      | 92587/225000 [167:51:43<204:43:00,  5.57s/it]
 41%|████      | 92588/225000 [167:51:47<181:55:47,  4.95s/it]
 41%|████      | 92589/225000 [167:51:50<161:18:36,  4.39s/it]
 41%|████      | 92590/225000 [167:51:53<154:36:39,  4.20s/it]
 41%|████      | 92591/225000 [167:51:57<142:57:12,  3.89s/it]
 41%|████      | 92592/225000 [167:52:00<136:15:47,  3.70s/it]
 41%|████      | 92593/225000 [167:52:04<141:14:03,  3.84s/it]
 41%|████      | 92594/225000 [167:52:08<139:50:15,  3.80s/it]
 41%|████      | 92595/225000 [167:52:12<139:19:04,  3.79s/it]
 41%|████      | 92596/225000 [167:52:16<147:58:09,  4.02s/it]
 41%|████      | 92597/225000 [167:52:26<208:57:53,  5.68s/it]
 41%|████      | 92598/225000 [167:52:31<210:05:00,  5.71s/it]
 41%|████      | 92599/225000 [167:52:36<193:22:06,  5.26s/it]
 41%|████      | 92600/225000 [167:52:41<195:37:26,  5.32s/it]
                                                              
{'loss': '0.2043', 'grad_norm': '6.253', 'learning_rate': '3.391e-05', 'epoch': '0.4116', 'num_input_tokens_seen': 274910280, 'train_runtime': '6.044e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92600/225000 [167:52:41<195:37:26,  5.32s/it]
 41%|████      | 92601/225000 [167:52:44<169:58:57,  4.62s/it]
 41%|████      | 92602/225000 [167:52:48<157:48:43,  4.29s/it]
 41%|████      | 92603/225000 [167:52:56<206:21:07,  5.61s/it]
 41%|████      | 92604/225000 [167:52:59<175:29:43,  4.77s/it]
 41%|████      | 92605/225000 [167:53:05<184:13:32,  5.01s/it]
 41%|████      | 92606/225000 [167:53:08<169:24:51,  4.61s/it]
 41%|████      | 92607/225000 [167:53:12<164:03:23,  4.46s/it]
 41%|████      | 92608/225000 [167:53:16<156:31:06,  4.26s/it]
 41%|████      | 92609/225000 [167:53:20<152:30:45,  4.15s/it]
 41%|████      | 92610/225000 [167:53:24<146:42:57,  3.99s/it]
 41%|████      | 92611/225000 [167:53:31<186:12:26,  5.06s/it]
 41%|████      | 92612/225000 [167:53:39<216:28:48,  5.89s/it]
 41%|████      | 92613/225000 [167:53:43<190:40:39,  5.19s/it]
 41%|████      | 92614/225000 [167:53:46<167:55:06,  4.57s/it]
 41%|████      | 92615/225000 [167:53:49<152:12:21,  4.14s/it]
 41%|████      | 92616/225000 [167:53:52<142:44:50,  3.88s/it]
 41%|████      | 92617/225000 [167:54:01<197:09:17,  5.36s/it]
 41%|████      | 92618/225000 [167:54:06<187:48:36,  5.11s/it]
 41%|████      | 92619/225000 [167:54:09<166:18:25,  4.52s/it]
 41%|████      | 92620/225000 [167:54:12<149:32:43,  4.07s/it]
 41%|████      | 92621/225000 [167:54:15<136:18:36,  3.71s/it]
 41%|████      | 92622/225000 [167:54:24<201:34:07,  5.48s/it]
 41%|████      | 92623/225000 [167:54:27<175:13:56,  4.77s/it]
 41%|████      | 92624/225000 [167:54:30<154:03:10,  4.19s/it]
 41%|████      | 92625/225000 [167:54:34<153:56:00,  4.19s/it]
 41%|████      | 92626/225000 [167:54:41<185:58:41,  5.06s/it]
 41%|████      | 92627/225000 [167:54:45<168:49:49,  4.59s/it]
 41%|████      | 92628/225000 [167:54:48<149:48:04,  4.07s/it]
 41%|████      | 92629/225000 [167:54:57<202:25:12,  5.51s/it]
 41%|████      | 92630/225000 [167:55:01<188:28:10,  5.13s/it]
 41%|████      | 92631/225000 [167:55:07<205:12:01,  5.58s/it]
 41%|████      | 92632/225000 [167:55:11<183:55:34,  5.00s/it]
 41%|████      | 92633/225000 [167:55:14<158:41:37,  4.32s/it]
 41%|████      | 92634/225000 [167:55:17<148:59:44,  4.05s/it]
 41%|████      | 92635/225000 [167:55:24<174:57:45,  4.76s/it]
 41%|████      | 92636/225000 [167:55:30<189:14:44,  5.15s/it]
 41%|████      | 92637/225000 [167:55:35<190:31:56,  5.18s/it]
 41%|████      | 92638/225000 [167:55:39<177:53:07,  4.84s/it]
 41%|████      | 92639/225000 [167:55:43<169:36:08,  4.61s/it]
 41%|████      | 92640/225000 [167:55:47<163:04:54,  4.44s/it]
 41%|████      | 92641/225000 [167:55:51<157:27:29,  4.28s/it]
 41%|████      | 92642/225000 [167:55:54<147:26:57,  4.01s/it]
 41%|████      | 92643/225000 [167:55:58<143:30:00,  3.90s/it]
 41%|████      | 92644/225000 [167:56:02<142:31:03,  3.88s/it]
 41%|████      | 92645/225000 [167:56:06<145:28:41,  3.96s/it]
 41%|████      | 92646/225000 [167:56:10<143:12:36,  3.90s/it]
 41%|████      | 92647/225000 [167:56:13<140:03:26,  3.81s/it]
 41%|████      | 92648/225000 [167:56:22<189:11:50,  5.15s/it]
 41%|████      | 92649/225000 [167:56:28<201:12:12,  5.47s/it]
 41%|████      | 92650/225000 [167:56:32<181:16:38,  4.93s/it]
                                                              
{'loss': '0.1821', 'grad_norm': '3.339', 'learning_rate': '3.389e-05', 'epoch': '0.4118', 'num_input_tokens_seen': 275013520, 'train_runtime': '6.046e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92650/225000 [167:56:32<181:16:38,  4.93s/it]
 41%|████      | 92651/225000 [167:56:36<170:10:57,  4.63s/it]
 41%|████      | 92652/225000 [167:56:40<168:58:25,  4.60s/it]
 41%|████      | 92653/225000 [167:56:44<162:58:01,  4.43s/it]
 41%|████      | 92654/225000 [167:56:48<153:18:48,  4.17s/it]
 41%|████      | 92655/225000 [167:56:51<143:28:54,  3.90s/it]
 41%|████      | 92656/225000 [167:56:55<142:58:49,  3.89s/it]
 41%|████      | 92657/225000 [167:56:59<145:58:18,  3.97s/it]
 41%|████      | 92658/225000 [167:57:07<196:09:20,  5.34s/it]
 41%|████      | 92659/225000 [167:57:17<247:41:10,  6.74s/it]
 41%|████      | 92660/225000 [167:57:21<207:37:05,  5.65s/it]
 41%|████      | 92661/225000 [167:57:23<172:47:46,  4.70s/it]
 41%|████      | 92662/225000 [167:57:31<204:02:37,  5.55s/it]
 41%|████      | 92663/225000 [167:57:33<172:41:57,  4.70s/it]
 41%|████      | 92664/225000 [167:57:36<149:37:45,  4.07s/it]
 41%|████      | 92665/225000 [167:57:40<150:05:18,  4.08s/it]
 41%|████      | 92666/225000 [167:57:44<152:20:34,  4.14s/it]
 41%|████      | 92667/225000 [167:57:54<216:30:19,  5.89s/it]
 41%|████      | 92668/225000 [167:57:57<181:42:43,  4.94s/it]
 41%|████      | 92669/225000 [167:58:00<157:35:51,  4.29s/it]
 41%|████      | 92670/225000 [167:58:03<143:12:11,  3.90s/it]
 41%|████      | 92671/225000 [167:58:12<197:41:09,  5.38s/it]
 41%|████      | 92672/225000 [167:58:15<173:02:19,  4.71s/it]
 41%|████      | 92673/225000 [167:58:21<194:44:56,  5.30s/it]
 41%|████      | 92674/225000 [167:58:25<180:00:50,  4.90s/it]
 41%|████      | 92675/225000 [167:58:28<156:17:38,  4.25s/it]
 41%|████      | 92676/225000 [167:58:31<142:09:06,  3.87s/it]
 41%|████      | 92677/225000 [167:58:41<204:12:09,  5.56s/it]
 41%|████      | 92678/225000 [167:58:43<172:56:00,  4.70s/it]
 41%|████      | 92679/225000 [167:58:47<156:35:10,  4.26s/it]
 41%|████      | 92680/225000 [167:58:50<145:44:12,  3.97s/it]
 41%|████      | 92681/225000 [167:58:53<142:31:03,  3.88s/it]
 41%|████      | 92682/225000 [167:59:01<187:14:06,  5.09s/it]
 41%|████      | 92683/225000 [167:59:08<202:01:51,  5.50s/it]
 41%|████      | 92684/225000 [167:59:12<185:30:16,  5.05s/it]
 41%|████      | 92685/225000 [167:59:15<164:38:18,  4.48s/it]
 41%|████      | 92686/225000 [167:59:19<153:44:23,  4.18s/it]
 41%|████      | 92687/225000 [167:59:23<152:02:19,  4.14s/it]
 41%|████      | 92688/225000 [167:59:26<147:54:35,  4.02s/it]
 41%|████      | 92689/225000 [167:59:30<140:49:07,  3.83s/it]
 41%|████      | 92690/225000 [167:59:34<142:22:42,  3.87s/it]
 41%|████      | 92691/225000 [167:59:39<161:22:04,  4.39s/it]
 41%|████      | 92692/225000 [167:59:51<240:42:14,  6.55s/it]
 41%|████      | 92693/225000 [167:59:54<202:15:58,  5.50s/it]
 41%|████      | 92694/225000 [167:59:57<178:54:42,  4.87s/it]
 41%|████      | 92695/225000 [168:00:01<163:37:08,  4.45s/it]
 41%|████      | 92696/225000 [168:00:04<151:01:02,  4.11s/it]
 41%|████      | 92697/225000 [168:00:08<149:16:17,  4.06s/it]
 41%|████      | 92698/225000 [168:00:16<186:56:16,  5.09s/it]
 41%|████      | 92699/225000 [168:00:22<205:35:32,  5.59s/it]
 41%|████      | 92700/225000 [168:00:25<178:54:46,  4.87s/it]
                                                              
{'loss': '0.2041', 'grad_norm': '7.451', 'learning_rate': '3.388e-05', 'epoch': '0.412', 'num_input_tokens_seen': 275117544, 'train_runtime': '6.048e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92700/225000 [168:00:25<178:54:46,  4.87s/it]
 41%|████      | 92701/225000 [168:00:34<218:10:24,  5.94s/it]
 41%|████      | 92702/225000 [168:00:36<178:28:38,  4.86s/it]
 41%|████      | 92703/225000 [168:00:40<162:10:00,  4.41s/it]
 41%|████      | 92704/225000 [168:00:43<153:21:15,  4.17s/it]
 41%|████      | 92705/225000 [168:00:47<148:02:54,  4.03s/it]
 41%|████      | 92706/225000 [168:00:53<166:25:45,  4.53s/it]
 41%|████      | 92707/225000 [168:01:01<204:15:35,  5.56s/it]
 41%|████      | 92708/225000 [168:01:03<170:55:39,  4.65s/it]
 41%|████      | 92709/225000 [168:01:06<151:48:42,  4.13s/it]
 41%|████      | 92710/225000 [168:01:09<139:50:35,  3.81s/it]
 41%|████      | 92711/225000 [168:01:19<208:20:58,  5.67s/it]
 41%|████      | 92712/225000 [168:01:24<199:18:33,  5.42s/it]
 41%|████      | 92713/225000 [168:01:27<171:35:21,  4.67s/it]
 41%|████      | 92714/225000 [168:01:35<209:51:14,  5.71s/it]
 41%|████      | 92715/225000 [168:01:38<183:26:08,  4.99s/it]
 41%|████      | 92716/225000 [168:01:45<203:21:05,  5.53s/it]
 41%|████      | 92717/225000 [168:01:48<177:29:28,  4.83s/it]
 41%|████      | 92718/225000 [168:01:52<159:52:32,  4.35s/it]
 41%|████      | 92719/225000 [168:02:00<205:11:33,  5.58s/it]
 41%|████      | 92720/225000 [168:02:03<181:35:41,  4.94s/it]
 41%|████      | 92721/225000 [168:02:07<166:21:29,  4.53s/it]
 41%|████      | 92722/225000 [168:02:11<161:46:00,  4.40s/it]
 41%|████      | 92723/225000 [168:02:15<153:52:21,  4.19s/it]
 41%|████      | 92724/225000 [168:02:20<170:00:10,  4.63s/it]
 41%|████      | 92725/225000 [168:02:28<206:43:22,  5.63s/it]
 41%|████      | 92726/225000 [168:02:31<176:13:53,  4.80s/it]
 41%|████      | 92727/225000 [168:02:34<153:26:33,  4.18s/it]
 41%|████      | 92728/225000 [168:02:38<147:01:37,  4.00s/it]
 41%|████      | 92729/225000 [168:02:41<142:27:11,  3.88s/it]
 41%|████      | 92730/225000 [168:02:45<142:15:56,  3.87s/it]
 41%|████      | 92731/225000 [168:02:48<136:09:06,  3.71s/it]
 41%|████      | 92732/225000 [168:02:52<136:45:39,  3.72s/it]
 41%|████      | 92733/225000 [168:02:56<133:48:05,  3.64s/it]
 41%|████      | 92734/225000 [168:03:03<178:25:29,  4.86s/it]
 41%|████      | 92735/225000 [168:03:13<230:43:33,  6.28s/it]
 41%|████      | 92736/225000 [168:03:17<202:04:57,  5.50s/it]
 41%|████      | 92737/225000 [168:03:20<180:38:17,  4.92s/it]
 41%|████      | 92738/225000 [168:03:23<162:12:55,  4.42s/it]
 41%|████      | 92739/225000 [168:03:26<147:21:43,  4.01s/it]
 41%|████      | 92740/225000 [168:03:30<147:55:24,  4.03s/it]
 41%|████      | 92741/225000 [168:03:35<151:59:20,  4.14s/it]
 41%|████      | 92742/225000 [168:03:46<224:36:54,  6.11s/it]
 41%|████      | 92743/225000 [168:03:51<213:24:55,  5.81s/it]
 41%|████      | 92744/225000 [168:03:54<190:54:54,  5.20s/it]
 41%|████      | 92745/225000 [168:04:03<231:46:31,  6.31s/it]
 41%|████      | 92746/225000 [168:04:07<201:01:31,  5.47s/it]
 41%|████      | 92747/225000 [168:04:09<169:46:31,  4.62s/it]
 41%|████      | 92748/225000 [168:04:17<200:37:20,  5.46s/it]
 41%|████      | 92749/225000 [168:04:22<196:48:48,  5.36s/it]
 41%|████      | 92750/225000 [168:04:25<171:42:42,  4.67s/it]
                                                              
{'loss': '0.1729', 'grad_norm': '7.215', 'learning_rate': '3.386e-05', 'epoch': '0.4122', 'num_input_tokens_seen': 275223752, 'train_runtime': '6.051e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92750/225000 [168:04:25<171:42:42,  4.67s/it]
 41%|████      | 92751/225000 [168:04:31<186:10:48,  5.07s/it]
 41%|████      | 92752/225000 [168:04:34<164:27:23,  4.48s/it]
 41%|████      | 92753/225000 [168:04:38<152:12:26,  4.14s/it]
 41%|████      | 92754/225000 [168:04:41<148:17:04,  4.04s/it]
 41%|████      | 92755/225000 [168:04:45<146:03:07,  3.98s/it]
 41%|████      | 92756/225000 [168:04:49<143:40:02,  3.91s/it]
 41%|████      | 92757/225000 [168:04:53<148:18:46,  4.04s/it]
 41%|████      | 92758/225000 [168:05:03<210:45:40,  5.74s/it]
 41%|████      | 92759/225000 [168:05:09<216:48:40,  5.90s/it]
 41%|████      | 92760/225000 [168:05:13<191:01:07,  5.20s/it]
 41%|████      | 92761/225000 [168:05:17<180:14:10,  4.91s/it]
 41%|████      | 92762/225000 [168:05:22<178:52:06,  4.87s/it]
 41%|████      | 92763/225000 [168:05:25<164:11:31,  4.47s/it]
 41%|████      | 92764/225000 [168:05:29<159:13:57,  4.33s/it]
 41%|████      | 92765/225000 [168:05:34<157:18:27,  4.28s/it]
 41%|████      | 92766/225000 [168:05:37<153:24:56,  4.18s/it]
 41%|████      | 92767/225000 [168:05:41<146:18:08,  3.98s/it]
 41%|████      | 92768/225000 [168:05:46<159:37:23,  4.35s/it]
 41%|████      | 92769/225000 [168:05:58<242:28:00,  6.60s/it]
 41%|████      | 92770/225000 [168:06:02<209:01:02,  5.69s/it]
 41%|████      | 92771/225000 [168:06:08<213:18:30,  5.81s/it]
 41%|████      | 92772/225000 [168:06:11<190:06:47,  5.18s/it]
 41%|████      | 92773/225000 [168:06:14<160:05:17,  4.36s/it]
 41%|████      | 92774/225000 [168:06:22<198:15:35,  5.40s/it]
 41%|████      | 92775/225000 [168:06:25<178:13:15,  4.85s/it]
 41%|████      | 92776/225000 [168:06:30<172:47:20,  4.70s/it]
 41%|████      | 92777/225000 [168:06:33<162:39:07,  4.43s/it]
 41%|████      | 92778/225000 [168:06:38<160:27:33,  4.37s/it]
 41%|████      | 92779/225000 [168:06:41<150:43:39,  4.10s/it]
 41%|████      | 92780/225000 [168:06:45<149:05:10,  4.06s/it]
 41%|████      | 92781/225000 [168:06:49<152:49:22,  4.16s/it]
 41%|████      | 92782/225000 [168:06:53<149:13:48,  4.06s/it]
 41%|████      | 92783/225000 [168:06:58<151:18:48,  4.12s/it]
 41%|████      | 92784/225000 [168:07:04<179:15:46,  4.88s/it]
 41%|████      | 92785/225000 [168:07:14<238:24:56,  6.49s/it]
 41%|████      | 92786/225000 [168:07:18<201:50:28,  5.50s/it]
 41%|████      | 92787/225000 [168:07:21<174:19:56,  4.75s/it]
 41%|████      | 92788/225000 [168:07:24<156:18:03,  4.26s/it]
 41%|████      | 92789/225000 [168:07:32<199:52:14,  5.44s/it]
 41%|████      | 92790/225000 [168:07:38<201:15:27,  5.48s/it]
 41%|████      | 92791/225000 [168:07:42<187:55:42,  5.12s/it]
 41%|████      | 92792/225000 [168:07:46<177:35:53,  4.84s/it]
 41%|████      | 92793/225000 [168:07:50<168:47:24,  4.60s/it]
 41%|████      | 92794/225000 [168:07:54<156:48:17,  4.27s/it]
 41%|████      | 92795/225000 [168:07:58<154:25:17,  4.20s/it]
 41%|████      | 92796/225000 [168:08:01<151:03:47,  4.11s/it]
 41%|████      | 92797/225000 [168:08:05<142:05:17,  3.87s/it]
 41%|████      | 92798/225000 [168:08:13<193:14:45,  5.26s/it]
 41%|████      | 92799/225000 [168:08:19<202:49:50,  5.52s/it]
 41%|████      | 92800/225000 [168:08:23<176:00:40,  4.79s/it]
                                                              
{'loss': '0.1967', 'grad_norm': '5.562', 'learning_rate': '3.384e-05', 'epoch': '0.4124', 'num_input_tokens_seen': 275329488, 'train_runtime': '6.053e+05', 'train_tokens_per_second': '454.9'}

 41%|████      | 92800/225000 [168:08:23<176:00:40,  4.79s/it]
 41%|████      | 92801/225000 [168:08:26<159:31:48,  4.34s/it]
 41%|████      | 92802/225000 [168:08:30<153:03:31,  4.17s/it]
 41%|████      | 92803/225000 [168:08:33<146:10:04,  3.98s/it]
 41%|████      | 92804/225000 [168:08:37<146:19:46,  3.98s/it]
 41%|████      | 92805/225000 [168:08:45<184:20:54,  5.02s/it]
 41%|████      | 92806/225000 [168:08:51<199:31:34,  5.43s/it]
 41%|████      | 92807/225000 [168:08:54<174:09:20,  4.74s/it]
 41%|████      | 92808/225000 [168:08:57<155:40:22,  4.24s/it]
 41%|████      | 92809/225000 [168:09:02<161:22:03,  4.39s/it]
 41%|████      | 92810/225000 [168:09:10<204:12:51,  5.56s/it]
 41%|████      | 92811/225000 [168:09:13<168:50:22,  4.60s/it]
 41%|████      | 92812/225000 [168:09:15<145:11:44,  3.95s/it]
 41%|████▏     | 92813/225000 [168:09:18<138:23:10,  3.77s/it]
 41%|████▏     | 92814/225000 [168:09:23<151:06:49,  4.12s/it]
 41%|████▏     | 92815/225000 [168:09:32<204:33:42,  5.57s/it]
 41%|████▏     | 92816/225000 [168:09:36<188:07:06,  5.12s/it]
 41%|████▏     | 92817/225000 [168:09:40<169:14:57,  4.61s/it]
 41%|████▏     | 92818/225000 [168:09:43<155:19:29,  4.23s/it]
 41%|████▏     | 92819/225000 [168:09:47<151:47:16,  4.13s/it]
 41%|████▏     | 92820/225000 [168:09:56<202:24:17,  5.51s/it]
 41%|████▏     | 92821/225000 [168:09:58<171:42:32,  4.68s/it]
 41%|████▏     | 92822/225000 [168:10:01<153:45:30,  4.19s/it]
 41%|████▏     | 92823/225000 [168:10:08<184:30:42,  5.03s/it]
 41%|████▏     | 92824/225000 [168:10:14<191:37:14,  5.22s/it]
 41%|████▏     | 92825/225000 [168:10:19<190:02:54,  5.18s/it]
 41%|████▏     | 92826/225000 [168:10:22<164:28:13,  4.48s/it]
 41%|████▏     | 92827/225000 [168:10:29<194:32:51,  5.30s/it]
 41%|████▏     | 92828/225000 [168:10:32<163:17:04,  4.45s/it]
 41%|████▏     | 92829/225000 [168:10:34<142:46:19,  3.89s/it]
 41%|████▏     | 92830/225000 [168:10:38<142:10:57,  3.87s/it]
 41%|████▏     | 92831/225000 [168:10:47<195:31:49,  5.33s/it]
 41%|████▏     | 92832/225000 [168:10:50<166:34:09,  4.54s/it]
 41%|████▏     | 92833/225000 [168:10:57<194:39:32,  5.30s/it]
 41%|████▏     | 92834/225000 [168:11:00<170:20:50,  4.64s/it]
 41%|████▏     | 92835/225000 [168:11:02<149:42:26,  4.08s/it]
 41%|████▏     | 92836/225000 [168:11:06<147:00:24,  4.00s/it]
 41%|████▏     | 92837/225000 [168:11:16<211:47:49,  5.77s/it]
 41%|████▏     | 92838/225000 [168:11:19<181:26:14,  4.94s/it]
 41%|████▏     | 92839/225000 [168:11:23<165:51:26,  4.52s/it]
 41%|████▏     | 92840/225000 [168:11:26<146:35:34,  3.99s/it]
 41%|████▏     | 92841/225000 [168:11:33<179:35:00,  4.89s/it]
 41%|████▏     | 92842/225000 [168:11:39<198:44:37,  5.41s/it]
 41%|████▏     | 92843/225000 [168:11:43<176:24:37,  4.81s/it]
 41%|████▏     | 92844/225000 [168:11:46<165:25:25,  4.51s/it]
 41%|████▏     | 92845/225000 [168:11:50<154:07:38,  4.20s/it]
 41%|████▏     | 92846/225000 [168:11:54<152:04:15,  4.14s/it]
 41%|████▏     | 92847/225000 [168:11:57<141:31:40,  3.86s/it]
 41%|████▏     | 92848/225000 [168:12:01<141:19:35,  3.85s/it]
 41%|████▏     | 92849/225000 [168:12:10<196:42:32,  5.36s/it]
 41%|████▏     | 92850/225000 [168:12:15<198:37:26,  5.41s/it]
                                                              
{'loss': '0.2067', 'grad_norm': '5.539', 'learning_rate': '3.383e-05', 'epoch': '0.4127', 'num_input_tokens_seen': 275432928, 'train_runtime': '6.055e+05', 'train_tokens_per_second': '454.9'}

 41%|████▏     | 92850/225000 [168:12:15<198:37:26,  5.41s/it]
 41%|████▏     | 92851/225000 [168:12:19<178:58:56,  4.88s/it]
 41%|████▏     | 92852/225000 [168:12:22<160:13:28,  4.36s/it]
 41%|████▏     | 92853/225000 [168:12:25<149:29:08,  4.07s/it]
 41%|████▏     | 92854/225000 [168:12:30<151:50:51,  4.14s/it]
 41%|████▏     | 92855/225000 [168:12:33<146:29:03,  3.99s/it]
 41%|████▏     | 92856/225000 [168:12:37<146:29:45,  3.99s/it]
 41%|████▏     | 92857/225000 [168:12:41<144:22:51,  3.93s/it]
 41%|████▏     | 92858/225000 [168:12:46<153:42:50,  4.19s/it]
 41%|████▏     | 92859/225000 [168:12:57<234:27:44,  6.39s/it]
 41%|████▏     | 92860/225000 [168:13:03<222:55:13,  6.07s/it]
 41%|████▏     | 92861/225000 [168:13:07<196:51:51,  5.36s/it]
 41%|████▏     | 92862/225000 [168:13:09<166:33:41,  4.54s/it]
 41%|████▏     | 92863/225000 [168:13:12<150:10:24,  4.09s/it]
 41%|████▏     | 92864/225000 [168:13:16<141:41:05,  3.86s/it]
 41%|████▏     | 92865/225000 [168:13:20<153:20:16,  4.18s/it]
 41%|████▏     | 92866/225000 [168:13:31<219:22:46,  5.98s/it]
 41%|████▏     | 92867/225000 [168:13:34<190:58:34,  5.20s/it]
 41%|████▏     | 92868/225000 [168:13:37<163:01:55,  4.44s/it]
 41%|████▏     | 92869/225000 [168:13:44<190:38:39,  5.19s/it]
 41%|████▏     | 92870/225000 [168:13:47<171:15:08,  4.67s/it]
 41%|████▏     | 92871/225000 [168:13:50<153:37:54,  4.19s/it]
 41%|████▏     | 92872/225000 [168:13:58<192:39:38,  5.25s/it]
 41%|████▏     | 92873/225000 [168:14:04<199:33:48,  5.44s/it]
 41%|████▏     | 92874/225000 [168:14:07<179:41:25,  4.90s/it]
 41%|████▏     | 92875/225000 [168:14:12<172:12:46,  4.69s/it]
 41%|████▏     | 92876/225000 [168:14:16<165:17:07,  4.50s/it]
 41%|████▏     | 92877/225000 [168:14:20<161:52:54,  4.41s/it]
 41%|████▏     | 92878/225000 [168:14:25<175:14:41,  4.77s/it]
 41%|████▏     | 92879/225000 [168:14:32<194:24:19,  5.30s/it]
 41%|████▏     | 92880/225000 [168:14:37<194:05:38,  5.29s/it]
 41%|████▏     | 92881/225000 [168:14:42<187:31:05,  5.11s/it]
 41%|████▏     | 92882/225000 [168:14:46<177:33:13,  4.84s/it]
 41%|████▏     | 92883/225000 [168:14:50<162:14:35,  4.42s/it]
 41%|████▏     | 92884/225000 [168:14:54<164:18:22,  4.48s/it]
 41%|████▏     | 92885/225000 [168:14:58<159:01:20,  4.33s/it]
 41%|████▏     | 92886/225000 [168:15:02<156:05:05,  4.25s/it]
 41%|████▏     | 92887/225000 [168:15:07<163:03:14,  4.44s/it]
 41%|████▏     | 92888/225000 [168:15:11<156:36:30,  4.27s/it]
 41%|████▏     | 92889/225000 [168:15:14<144:26:40,  3.94s/it]
 41%|████▏     | 92890/225000 [168:15:19<155:17:14,  4.23s/it]
 41%|████▏     | 92891/225000 [168:15:24<166:37:51,  4.54s/it]
 41%|████▏     | 92892/225000 [168:15:29<165:10:27,  4.50s/it]
 41%|████▏     | 92893/225000 [168:15:34<174:33:17,  4.76s/it]
 41%|████▏     | 92894/225000 [168:15:39<174:49:04,  4.76s/it]
 41%|████▏     | 92895/225000 [168:15:43<163:56:47,  4.47s/it]
 41%|████▏     | 92896/225000 [168:15:50<199:47:35,  5.44s/it]
 41%|████▏     | 92897/225000 [168:15:59<230:24:11,  6.28s/it]
 41%|████▏     | 92898/225000 [168:16:04<224:32:29,  6.12s/it]
 41%|████▏     | 92899/225000 [168:16:08<198:17:40,  5.40s/it]
 41%|████▏     | 92900/225000 [168:16:13<191:03:11,  5.21s/it]
                                                              
{'loss': '0.1883', 'grad_norm': '2.927', 'learning_rate': '3.381e-05', 'epoch': '0.4129', 'num_input_tokens_seen': 275538832, 'train_runtime': '6.058e+05', 'train_tokens_per_second': '454.9'}

 41%|████▏     | 92900/225000 [168:16:13<191:03:11,  5.21s/it]
 41%|████▏     | 92901/225000 [168:16:18<188:34:44,  5.14s/it]
 41%|████▏     | 92902/225000 [168:16:22<180:04:14,  4.91s/it]
 41%|████▏     | 92903/225000 [168:16:26<165:31:56,  4.51s/it]
 41%|████▏     | 92904/225000 [168:16:30<159:45:55,  4.35s/it]
 41%|████▏     | 92905/225000 [168:16:34<157:19:21,  4.29s/it]
 41%|████▏     | 92906/225000 [168:16:38<151:41:22,  4.13s/it]
 41%|████▏     | 92907/225000 [168:16:43<160:29:55,  4.37s/it]
 41%|████▏     | 92908/225000 [168:16:47<162:56:21,  4.44s/it]
 41%|████▏     | 92909/225000 [168:16:51<154:09:14,  4.20s/it]
 41%|████▏     | 92910/225000 [168:17:00<205:55:41,  5.61s/it]
 41%|████▏     | 92911/225000 [168:17:07<225:12:09,  6.14s/it]
 41%|████▏     | 92912/225000 [168:17:10<191:08:07,  5.21s/it]
 41%|████▏     | 92913/225000 [168:17:13<169:20:44,  4.62s/it]
 41%|████▏     | 92914/225000 [168:17:17<157:16:27,  4.29s/it]
 41%|████▏     | 92915/225000 [168:17:20<143:07:25,  3.90s/it]
 41%|████▏     | 92916/225000 [168:17:26<164:08:17,  4.47s/it]
 41%|████▏     | 92917/225000 [168:17:34<207:27:55,  5.65s/it]
 41%|████▏     | 92918/225000 [168:17:37<178:05:14,  4.85s/it]
 41%|████▏     | 92919/225000 [168:17:40<159:36:31,  4.35s/it]
 41%|████▏     | 92920/225000 [168:17:46<170:42:00,  4.65s/it]
 41%|████▏     | 92921/225000 [168:17:52<194:32:35,  5.30s/it]
 41%|████▏     | 92922/225000 [168:17:56<169:45:47,  4.63s/it]
 41%|████▏     | 92923/225000 [168:18:04<210:35:13,  5.74s/it]
 41%|████▏     | 92924/225000 [168:18:06<173:25:41,  4.73s/it]
 41%|████▏     | 92925/225000 [168:18:09<156:00:19,  4.25s/it]
 41%|████▏     | 92926/225000 [168:18:17<189:06:21,  5.15s/it]
 41%|████▏     | 92927/225000 [168:18:21<177:44:56,  4.85s/it]
 41%|████▏     | 92928/225000 [168:18:26<180:14:34,  4.91s/it]
 41%|████▏     | 92929/225000 [168:18:29<160:07:02,  4.36s/it]
 41%|████▏     | 92930/225000 [168:18:33<154:24:28,  4.21s/it]
 41%|████▏     | 92931/225000 [168:18:37<150:07:52,  4.09s/it]
 41%|████▏     | 92932/225000 [168:18:42<164:20:45,  4.48s/it]
 41%|████▏     | 92933/225000 [168:18:52<227:51:20,  6.21s/it]
 41%|████▏     | 92934/225000 [168:18:56<200:19:07,  5.46s/it]
 41%|████▏     | 92935/225000 [168:19:00<180:10:47,  4.91s/it]
 41%|████▏     | 92936/225000 [168:19:04<176:37:49,  4.81s/it]
 41%|████▏     | 92937/225000 [168:19:08<166:26:47,  4.54s/it]
 41%|████▏     | 92938/225000 [168:19:12<163:39:29,  4.46s/it]
 41%|████▏     | 92939/225000 [168:19:16<160:26:52,  4.37s/it]
 41%|████▏     | 92940/225000 [168:19:20<152:50:46,  4.17s/it]
 41%|████▏     | 92941/225000 [168:19:24<152:55:20,  4.17s/it]
 41%|████▏     | 92942/225000 [168:19:28<147:32:31,  4.02s/it]
 41%|████▏     | 92943/225000 [168:19:38<213:56:35,  5.83s/it]
 41%|████▏     | 92944/225000 [168:19:46<234:02:23,  6.38s/it]
 41%|████▏     | 92945/225000 [168:19:50<208:55:58,  5.70s/it]
 41%|████▏     | 92946/225000 [168:19:54<195:06:44,  5.32s/it]
 41%|████▏     | 92947/225000 [168:19:58<179:22:31,  4.89s/it]
 41%|████▏     | 92948/225000 [168:20:03<174:58:22,  4.77s/it]
 41%|████▏     | 92949/225000 [168:20:07<166:53:23,  4.55s/it]
 41%|████▏     | 92950/225000 [168:20:11<163:15:04,  4.45s/it]
                                                              
{'loss': '0.1933', 'grad_norm': '4.483', 'learning_rate': '3.379e-05', 'epoch': '0.4131', 'num_input_tokens_seen': 275646328, 'train_runtime': '6.06e+05', 'train_tokens_per_second': '454.9'}

 41%|████▏     | 92950/225000 [168:20:11<163:15:04,  4.45s/it]
 41%|████▏     | 92951/225000 [168:20:14<152:28:05,  4.16s/it]
 41%|████▏     | 92952/225000 [168:20:18<147:49:16,  4.03s/it]
 41%|████▏     | 92953/225000 [168:20:28<208:50:02,  5.69s/it]
 41%|████▏     | 92954/225000 [168:20:34<214:33:42,  5.85s/it]
 41%|████▏     | 92955/225000 [168:20:37<184:47:25,  5.04s/it]
 41%|████▏     | 92956/225000 [168:20:40<165:15:28,  4.51s/it]
 41%|████▏     | 92957/225000 [168:20:51<230:51:19,  6.29s/it]
 41%|████▏     | 92958/225000 [168:20:54<194:55:00,  5.31s/it]
 41%|████▏     | 92959/225000 [168:20:58<182:39:00,  4.98s/it]
 41%|████▏     | 92960/225000 [168:21:03<186:01:01,  5.07s/it]
 41%|████▏     | 92961/225000 [168:21:06<164:10:34,  4.48s/it]
 41%|████▏     | 92962/225000 [168:21:10<158:00:58,  4.31s/it]
 41%|████▏     | 92963/225000 [168:21:17<187:11:28,  5.10s/it]
 41%|████▏     | 92964/225000 [168:21:25<211:41:22,  5.77s/it]
 41%|████▏     | 92965/225000 [168:21:28<186:14:13,  5.08s/it]
 41%|████▏     | 92966/225000 [168:21:32<173:31:20,  4.73s/it]
 41%|████▏     | 92967/225000 [168:21:35<159:44:37,  4.36s/it]
 41%|████▏     | 92968/225000 [168:21:39<148:09:41,  4.04s/it]
 41%|████▏     | 92969/225000 [168:21:43<146:51:09,  4.00s/it]
 41%|████▏     | 92970/225000 [168:21:47<150:12:15,  4.10s/it]
 41%|████▏     | 92971/225000 [168:21:55<197:53:13,  5.40s/it]
 41%|████▏     | 92972/225000 [168:21:59<179:31:32,  4.90s/it]
 41%|████▏     | 92973/225000 [168:22:02<153:25:23,  4.18s/it]
 41%|████▏     | 92974/225000 [168:22:07<164:03:49,  4.47s/it]
 41%|████▏     | 92975/225000 [168:22:13<180:21:30,  4.92s/it]
 41%|████▏     | 92976/225000 [168:22:16<160:51:24,  4.39s/it]
 41%|████▏     | 92977/225000 [168:22:19<148:08:21,  4.04s/it]
 41%|████▏     | 92978/225000 [168:22:23<141:05:26,  3.85s/it]
 41%|████▏     | 92979/225000 [168:22:33<209:31:23,  5.71s/it]
 41%|████▏     | 92980/225000 [168:22:37<190:44:28,  5.20s/it]
 41%|████▏     | 92981/225000 [168:22:40<173:48:08,  4.74s/it]
 41%|████▏     | 92982/225000 [168:22:48<207:27:07,  5.66s/it]
 41%|████▏     | 92983/225000 [168:22:52<189:45:38,  5.17s/it]
 41%|████▏     | 92984/225000 [168:22:56<173:34:22,  4.73s/it]
 41%|████▏     | 92985/225000 [168:22:59<157:46:23,  4.30s/it]
 41%|████▏     | 92986/225000 [168:23:03<150:58:04,  4.12s/it]
 41%|████▏     | 92987/225000 [168:23:12<209:54:21,  5.72s/it]
 41%|████▏     | 92988/225000 [168:23:16<182:59:20,  4.99s/it]
 41%|████▏     | 92989/225000 [168:23:18<160:18:02,  4.37s/it]
 41%|████▏     | 92990/225000 [168:23:22<146:27:11,  3.99s/it]
 41%|████▏     | 92991/225000 [168:23:29<184:31:50,  5.03s/it]
 41%|████▏     | 92992/225000 [168:23:33<171:27:25,  4.68s/it]
 41%|████▏     | 92993/225000 [168:23:36<155:31:21,  4.24s/it]
 41%|████▏     | 92994/225000 [168:23:40<147:42:42,  4.03s/it]
 41%|████▏     | 92995/225000 [168:23:44<154:47:38,  4.22s/it]
 41%|████▏     | 92996/225000 [168:23:54<220:10:08,  6.00s/it]
 41%|████▏     | 92997/225000 [168:23:57<185:59:41,  5.07s/it]
 41%|████▏     | 92998/225000 [168:24:01<165:50:01,  4.52s/it]
 41%|████▏     | 92999/225000 [168:24:04<149:35:47,  4.08s/it]
 41%|████▏     | 93000/225000 [168:24:11<180:30:01,  4.92s/it]
                                                              
{'loss': '0.1951', 'grad_norm': '3.71', 'learning_rate': '3.377e-05', 'epoch': '0.4133', 'num_input_tokens_seen': 275753232, 'train_runtime': '6.063e+05', 'train_tokens_per_second': '454.9'}

 41%|████▏     | 93000/225000 [168:24:11<180:30:01,  4.92s/it]
 41%|████▏     | 93001/225000 [168:24:17<192:59:39,  5.26s/it]
 41%|████▏     | 93002/225000 [168:24:20<174:33:02,  4.76s/it]
 41%|████▏     | 93003/225000 [168:24:24<160:53:55,  4.39s/it]
 41%|████▏     | 93004/225000 [168:24:27<145:15:38,  3.96s/it]
 41%|████▏     | 93005/225000 [168:24:38<224:17:51,  6.12s/it]
 41%|████▏     | 93006/225000 [168:24:40<183:05:29,  4.99s/it]
 41%|████▏     | 93007/225000 [168:24:43<162:49:56,  4.44s/it]
 41%|████▏     | 93008/225000 [168:24:51<200:47:30,  5.48s/it]
 41%|████▏     | 93009/225000 [168:24:53<165:08:30,  4.50s/it]
 41%|████▏     | 93010/225000 [168:24:57<152:11:24,  4.15s/it]
 41%|████▏     | 93011/225000 [168:25:00<140:39:08,  3.84s/it]
 41%|████▏     | 93012/225000 [168:25:03<132:29:02,  3.61s/it]
 41%|████▏     | 93013/225000 [168:25:07<137:10:43,  3.74s/it]
 41%|████▏     | 93014/225000 [168:25:10<131:11:06,  3.58s/it]
 41%|████▏     | 93015/225000 [168:25:14<128:04:29,  3.49s/it]
 41%|████▏     | 93016/225000 [168:25:17<127:05:59,  3.47s/it]
 41%|████▏     | 93017/225000 [168:25:28<208:06:21,  5.68s/it]
 41%|████▏     | 93018/225000 [168:25:31<185:48:24,  5.07s/it]
 41%|████▏     | 93019/225000 [168:25:35<166:41:31,  4.55s/it]
 41%|████▏     | 93020/225000 [168:25:39<166:57:33,  4.55s/it]
 41%|████▏     | 93021/225000 [168:25:44<166:05:51,  4.53s/it]
 41%|████▏     | 93022/225000 [168:25:49<170:36:05,  4.65s/it]
 41%|████▏     | 93023/225000 [168:25:54<176:16:43,  4.81s/it]
 41%|████▏     | 93024/225000 [168:26:05<248:45:37,  6.79s/it]
 41%|████▏     | 93025/225000 [168:26:09<214:50:43,  5.86s/it]
 41%|████▏     | 93026/225000 [168:26:15<215:25:34,  5.88s/it]
 41%|████▏     | 93027/225000 [168:26:20<206:18:53,  5.63s/it]
 41%|████▏     | 93028/225000 [168:26:24<192:09:12,  5.24s/it]
 41%|████▏     | 93029/225000 [168:26:28<178:44:42,  4.88s/it]
 41%|████▏     | 93030/225000 [168:26:32<167:59:48,  4.58s/it]
 41%|████▏     | 93031/225000 [168:26:36<163:46:48,  4.47s/it]
 41%|████▏     | 93032/225000 [168:26:41<163:30:45,  4.46s/it]
 41%|████▏     | 93033/225000 [168:26:45<154:19:17,  4.21s/it]
 41%|████▏     | 93034/225000 [168:26:49<161:18:33,  4.40s/it]
 41%|████▏     | 93035/225000 [168:26:54<160:50:00,  4.39s/it]
 41%|████▏     | 93036/225000 [168:26:58<163:41:53,  4.47s/it]
 41%|████▏     | 93037/225000 [168:27:02<154:35:47,  4.22s/it]
 41%|████▏     | 93038/225000 [168:27:08<173:22:50,  4.73s/it]
 41%|████▏     | 93039/225000 [168:27:17<224:17:17,  6.12s/it]
 41%|████▏     | 93040/225000 [168:27:20<191:23:41,  5.22s/it]
 41%|████▏     | 93041/225000 [168:27:24<169:11:40,  4.62s/it]
 41%|████▏     | 93042/225000 [168:27:28<163:17:23,  4.45s/it]
 41%|████▏     | 93043/225000 [168:27:34<184:00:02,  5.02s/it]
 41%|████▏     | 93044/225000 [168:27:41<203:23:02,  5.55s/it]
 41%|████▏     | 93045/225000 [168:27:45<186:41:02,  5.09s/it]
 41%|████▏     | 93046/225000 [168:27:51<197:11:22,  5.38s/it]
 41%|████▏     | 93047/225000 [168:27:54<171:30:34,  4.68s/it]
 41%|████▏     | 93048/225000 [168:28:02<210:21:15,  5.74s/it]
 41%|████▏     | 93049/225000 [168:28:06<188:14:10,  5.14s/it]
 41%|████▏     | 93050/225000 [168:28:09<170:24:47,  4.65s/it]
                                                              
{'loss': '0.1869', 'grad_norm': '2.739', 'learning_rate': '3.376e-05', 'epoch': '0.4136', 'num_input_tokens_seen': 275860440, 'train_runtime': '6.065e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93050/225000 [168:28:09<170:24:47,  4.65s/it]
 41%|████▏     | 93051/225000 [168:28:13<158:08:49,  4.31s/it]
 41%|████▏     | 93052/225000 [168:28:17<155:17:56,  4.24s/it]
 41%|████▏     | 93053/225000 [168:28:21<152:57:36,  4.17s/it]
 41%|████▏     | 93054/225000 [168:28:24<142:37:55,  3.89s/it]
 41%|████▏     | 93055/225000 [168:28:35<213:20:36,  5.82s/it]
 41%|████▏     | 93056/225000 [168:28:38<191:24:44,  5.22s/it]
 41%|████▏     | 93057/225000 [168:28:42<178:51:40,  4.88s/it]
 41%|████▏     | 93058/225000 [168:28:47<171:05:30,  4.67s/it]
 41%|████▏     | 93059/225000 [168:28:51<164:34:42,  4.49s/it]
 41%|████▏     | 93060/225000 [168:28:54<154:16:17,  4.21s/it]
 41%|████▏     | 93061/225000 [168:29:03<199:32:46,  5.44s/it]
 41%|████▏     | 93062/225000 [168:29:11<234:00:13,  6.38s/it]
 41%|████▏     | 93063/225000 [168:29:16<213:42:29,  5.83s/it]
 41%|████▏     | 93064/225000 [168:29:21<212:30:07,  5.80s/it]
 41%|████▏     | 93065/225000 [168:29:26<200:18:55,  5.47s/it]
 41%|████▏     | 93066/225000 [168:29:30<182:48:29,  4.99s/it]
 41%|████▏     | 93067/225000 [168:29:34<172:27:33,  4.71s/it]
 41%|████▏     | 93068/225000 [168:29:38<165:50:24,  4.53s/it]
 41%|████▏     | 93069/225000 [168:29:42<154:02:18,  4.20s/it]
 41%|████▏     | 93070/225000 [168:29:48<182:10:19,  4.97s/it]
 41%|████▏     | 93071/225000 [168:29:57<224:40:55,  6.13s/it]
 41%|████▏     | 93072/225000 [168:30:00<190:46:34,  5.21s/it]
 41%|████▏     | 93073/225000 [168:30:03<167:18:54,  4.57s/it]
 41%|████▏     | 93074/225000 [168:30:10<190:56:36,  5.21s/it]
 41%|████▏     | 93075/225000 [168:30:15<185:27:18,  5.06s/it]
 41%|████▏     | 93076/225000 [168:30:18<161:09:52,  4.40s/it]
 41%|████▏     | 93077/225000 [168:30:21<151:15:19,  4.13s/it]
 41%|████▏     | 93078/225000 [168:30:25<146:55:00,  4.01s/it]
 41%|████▏     | 93079/225000 [168:30:33<195:14:56,  5.33s/it]
 41%|████▏     | 93080/225000 [168:30:37<179:02:30,  4.89s/it]
 41%|████▏     | 93081/225000 [168:30:40<160:45:47,  4.39s/it]
 41%|████▏     | 93082/225000 [168:30:44<148:36:22,  4.06s/it]
 41%|████▏     | 93083/225000 [168:30:48<151:55:40,  4.15s/it]
 41%|████▏     | 93084/225000 [168:30:52<152:30:13,  4.16s/it]
 41%|████▏     | 93085/225000 [168:30:56<150:14:21,  4.10s/it]
 41%|████▏     | 93086/225000 [168:31:00<148:04:57,  4.04s/it]
 41%|████▏     | 93087/225000 [168:31:04<148:50:08,  4.06s/it]
 41%|████▏     | 93088/225000 [168:31:08<149:18:34,  4.07s/it]
 41%|████▏     | 93089/225000 [168:31:17<203:10:53,  5.55s/it]
 41%|████▏     | 93090/225000 [168:31:27<248:24:15,  6.78s/it]
 41%|████▏     | 93091/225000 [168:31:30<208:17:59,  5.68s/it]
 41%|████▏     | 93092/225000 [168:31:37<220:51:31,  6.03s/it]
 41%|████▏     | 93093/225000 [168:31:40<185:46:37,  5.07s/it]
 41%|████▏     | 93094/225000 [168:31:48<220:06:44,  6.01s/it]
 41%|████▏     | 93095/225000 [168:31:51<186:55:23,  5.10s/it]
 41%|████▏     | 93096/225000 [168:31:55<172:31:30,  4.71s/it]
 41%|████▏     | 93097/225000 [168:31:58<161:10:55,  4.40s/it]
 41%|████▏     | 93098/225000 [168:32:02<152:28:34,  4.16s/it]
 41%|████▏     | 93099/225000 [168:32:06<148:47:35,  4.06s/it]
 41%|████▏     | 93100/225000 [168:32:10<148:52:38,  4.06s/it]
                                                              
{'loss': '0.1815', 'grad_norm': '4.226', 'learning_rate': '3.374e-05', 'epoch': '0.4138', 'num_input_tokens_seen': 275968408, 'train_runtime': '6.067e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93100/225000 [168:32:10<148:52:38,  4.06s/it]
 41%|████▏     | 93101/225000 [168:32:14<146:59:17,  4.01s/it]
 41%|████▏     | 93102/225000 [168:32:20<167:36:20,  4.57s/it]
 41%|████▏     | 93103/225000 [168:32:30<233:35:37,  6.38s/it]
 41%|████▏     | 93104/225000 [168:32:33<195:59:14,  5.35s/it]
 41%|████▏     | 93105/225000 [168:32:37<177:34:29,  4.85s/it]
 41%|████▏     | 93106/225000 [168:32:40<163:30:54,  4.46s/it]
 41%|████▏     | 93107/225000 [168:32:44<155:46:59,  4.25s/it]
 41%|████▏     | 93108/225000 [168:32:49<158:43:11,  4.33s/it]
 41%|████▏     | 93109/225000 [168:32:52<148:43:26,  4.06s/it]
 41%|████▏     | 93110/225000 [168:32:55<140:47:50,  3.84s/it]
 41%|████▏     | 93111/225000 [168:33:06<218:53:04,  5.97s/it]
 41%|████▏     | 93112/225000 [168:33:10<191:36:00,  5.23s/it]
 41%|████▏     | 93113/225000 [168:33:13<166:21:08,  4.54s/it]
 41%|████▏     | 93114/225000 [168:33:16<149:44:49,  4.09s/it]
 41%|████▏     | 93115/225000 [168:33:21<157:37:50,  4.30s/it]
 41%|████▏     | 93116/225000 [168:33:31<220:39:20,  6.02s/it]
 41%|████▏     | 93117/225000 [168:33:35<205:03:59,  5.60s/it]
 41%|████▏     | 93118/225000 [168:33:38<173:36:28,  4.74s/it]
 41%|████▏     | 93119/225000 [168:33:44<191:45:06,  5.23s/it]
 41%|████▏     | 93120/225000 [168:33:48<177:44:25,  4.85s/it]
 41%|████▏     | 93121/225000 [168:33:51<157:08:25,  4.29s/it]
 41%|████▏     | 93122/225000 [168:33:55<147:32:36,  4.03s/it]
 41%|████▏     | 93123/225000 [168:33:59<154:35:29,  4.22s/it]
 41%|████▏     | 93124/225000 [168:34:08<204:59:48,  5.60s/it]
 41%|████▏     | 93125/225000 [168:34:12<183:07:14,  5.00s/it]
 41%|████▏     | 93126/225000 [168:34:19<207:42:13,  5.67s/it]
 41%|████▏     | 93127/225000 [168:34:23<185:27:28,  5.06s/it]
 41%|████▏     | 93128/225000 [168:34:26<166:37:14,  4.55s/it]
 41%|████▏     | 93129/225000 [168:34:29<150:21:16,  4.10s/it]
 41%|████▏     | 93130/225000 [168:34:33<145:34:36,  3.97s/it]
 41%|████▏     | 93131/225000 [168:34:37<143:44:45,  3.92s/it]
 41%|████▏     | 93132/225000 [168:34:41<148:22:48,  4.05s/it]
 41%|████▏     | 93133/225000 [168:34:47<174:23:58,  4.76s/it]
 41%|████▏     | 93134/225000 [168:34:54<189:34:47,  5.18s/it]
 41%|████▏     | 93135/225000 [168:34:58<181:11:40,  4.95s/it]
 41%|████▏     | 93136/225000 [168:35:01<162:12:45,  4.43s/it]
 41%|████▏     | 93137/225000 [168:35:04<148:40:17,  4.06s/it]
 41%|████▏     | 93138/225000 [168:35:08<146:38:39,  4.00s/it]
 41%|████▏     | 93139/225000 [168:35:15<178:19:26,  4.87s/it]
 41%|████▏     | 93140/225000 [168:35:25<229:20:42,  6.26s/it]
 41%|████▏     | 93141/225000 [168:35:28<201:46:08,  5.51s/it]
 41%|████▏     | 93142/225000 [168:35:32<182:26:02,  4.98s/it]
 41%|████▏     | 93143/225000 [168:35:36<174:44:38,  4.77s/it]
 41%|████▏     | 93144/225000 [168:35:46<230:31:26,  6.29s/it]
 41%|████▏     | 93145/225000 [168:35:52<227:20:03,  6.21s/it]
 41%|████▏     | 93146/225000 [168:35:56<197:33:26,  5.39s/it]
 41%|████▏     | 93147/225000 [168:35:59<176:58:55,  4.83s/it]
 41%|████▏     | 93148/225000 [168:36:03<160:25:29,  4.38s/it]
 41%|████▏     | 93149/225000 [168:36:06<150:36:57,  4.11s/it]
 41%|████▏     | 93150/225000 [168:36:13<180:26:29,  4.93s/it]
                                                              
{'loss': '0.1933', 'grad_norm': '4.711', 'learning_rate': '3.372e-05', 'epoch': '0.414', 'num_input_tokens_seen': 276075680, 'train_runtime': '6.07e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93150/225000 [168:36:13<180:26:29,  4.93s/it]
 41%|████▏     | 93151/225000 [168:36:21<217:52:00,  5.95s/it]
 41%|████▏     | 93152/225000 [168:36:24<187:27:21,  5.12s/it]
 41%|████▏     | 93153/225000 [168:36:29<176:33:27,  4.82s/it]
 41%|████▏     | 93154/225000 [168:36:36<208:52:17,  5.70s/it]
 41%|████▏     | 93155/225000 [168:36:40<184:25:46,  5.04s/it]
 41%|████▏     | 93156/225000 [168:36:43<166:44:46,  4.55s/it]
 41%|████▏     | 93157/225000 [168:36:47<161:38:29,  4.41s/it]
 41%|████▏     | 93158/225000 [168:36:52<159:54:44,  4.37s/it]
 41%|████▏     | 93159/225000 [168:36:56<165:08:16,  4.51s/it]
 41%|████▏     | 93160/225000 [168:37:01<161:59:56,  4.42s/it]
 41%|████▏     | 93161/225000 [168:37:05<157:05:38,  4.29s/it]
 41%|████▏     | 93162/225000 [168:37:15<226:33:15,  6.19s/it]
 41%|████▏     | 93163/225000 [168:37:20<216:34:28,  5.91s/it]
 41%|████▏     | 93164/225000 [168:37:24<194:17:07,  5.31s/it]
 41%|████▏     | 93165/225000 [168:37:28<174:43:41,  4.77s/it]
 41%|████▏     | 93166/225000 [168:37:32<162:38:44,  4.44s/it]
 41%|████▏     | 93167/225000 [168:37:35<154:13:09,  4.21s/it]
 41%|████▏     | 93168/225000 [168:37:38<140:14:26,  3.83s/it]
 41%|████▏     | 93169/225000 [168:37:44<158:29:48,  4.33s/it]
 41%|████▏     | 93170/225000 [168:37:54<225:48:22,  6.17s/it]
 41%|████▏     | 93171/225000 [168:37:57<186:49:22,  5.10s/it]
 41%|████▏     | 93172/225000 [168:38:00<164:53:35,  4.50s/it]
 41%|████▏     | 93173/225000 [168:38:08<206:24:08,  5.64s/it]
 41%|████▏     | 93174/225000 [168:38:12<192:12:07,  5.25s/it]
 41%|████▏     | 93175/225000 [168:38:16<177:35:26,  4.85s/it]
 41%|████▏     | 93176/225000 [168:38:20<167:18:23,  4.57s/it]
 41%|████▏     | 93177/225000 [168:38:31<233:11:43,  6.37s/it]
 41%|████▏     | 93178/225000 [168:38:34<195:21:56,  5.34s/it]
 41%|████▏     | 93179/225000 [168:38:37<173:13:27,  4.73s/it]
 41%|████▏     | 93180/225000 [168:38:40<158:03:08,  4.32s/it]
 41%|████▏     | 93181/225000 [168:38:44<150:01:44,  4.10s/it]
 41%|████▏     | 93182/225000 [168:38:51<185:06:58,  5.06s/it]
 41%|████▏     | 93183/225000 [168:38:57<196:59:58,  5.38s/it]
 41%|████▏     | 93184/225000 [168:39:03<202:54:13,  5.54s/it]
 41%|████▏     | 93185/225000 [168:39:08<194:16:36,  5.31s/it]
 41%|████▏     | 93186/225000 [168:39:11<172:06:26,  4.70s/it]
 41%|████▏     | 93187/225000 [168:39:17<185:41:57,  5.07s/it]
 41%|████▏     | 93188/225000 [168:39:25<217:54:52,  5.95s/it]
 41%|████▏     | 93189/225000 [168:39:29<188:18:26,  5.14s/it]
 41%|████▏     | 93190/225000 [168:39:32<170:17:30,  4.65s/it]
 41%|████▏     | 93191/225000 [168:39:40<210:00:53,  5.74s/it]
 41%|████▏     | 93192/225000 [168:39:44<180:56:22,  4.94s/it]
 41%|████▏     | 93193/225000 [168:39:47<161:44:41,  4.42s/it]
 41%|████▏     | 93194/225000 [168:39:56<210:07:59,  5.74s/it]
 41%|████▏     | 93195/225000 [168:39:59<181:51:38,  4.97s/it]
 41%|████▏     | 93196/225000 [168:40:02<162:33:18,  4.44s/it]
 41%|████▏     | 93197/225000 [168:40:06<159:43:36,  4.36s/it]
 41%|████▏     | 93198/225000 [168:40:10<150:11:29,  4.10s/it]
 41%|████▏     | 93199/225000 [168:40:14<151:47:56,  4.15s/it]
 41%|████▏     | 93200/225000 [168:40:22<199:48:40,  5.46s/it]
                                                              
{'loss': '0.1998', 'grad_norm': '4.135', 'learning_rate': '3.371e-05', 'epoch': '0.4142', 'num_input_tokens_seen': 276185600, 'train_runtime': '6.072e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93200/225000 [168:40:22<199:48:40,  5.46s/it]
 41%|████▏     | 93201/225000 [168:40:29<213:54:30,  5.84s/it]
 41%|████▏     | 93202/225000 [168:40:33<187:49:57,  5.13s/it]
 41%|████▏     | 93203/225000 [168:40:36<173:50:05,  4.75s/it]
 41%|████▏     | 93204/225000 [168:40:40<159:45:56,  4.36s/it]
 41%|████▏     | 93205/225000 [168:40:45<166:19:23,  4.54s/it]
 41%|████▏     | 93206/225000 [168:40:55<223:18:56,  6.10s/it]
 41%|████▏     | 93207/225000 [168:40:58<191:25:06,  5.23s/it]
 41%|████▏     | 93208/225000 [168:41:01<170:13:51,  4.65s/it]
 41%|████▏     | 93209/225000 [168:41:04<151:16:46,  4.13s/it]
 41%|████▏     | 93210/225000 [168:41:13<208:14:11,  5.69s/it]
 41%|████▏     | 93211/225000 [168:41:19<211:36:20,  5.78s/it]
 41%|████▏     | 93212/225000 [168:41:22<177:14:41,  4.84s/it]
 41%|████▏     | 93213/225000 [168:41:25<154:20:02,  4.22s/it]
 41%|████▏     | 93214/225000 [168:41:31<181:54:47,  4.97s/it]
 41%|████▏     | 93215/225000 [168:41:37<183:58:23,  5.03s/it]
 41%|████▏     | 93216/225000 [168:41:40<163:57:11,  4.48s/it]
 41%|████▏     | 93217/225000 [168:41:44<166:03:45,  4.54s/it]
 41%|████▏     | 93218/225000 [168:41:53<207:53:22,  5.68s/it]
 41%|████▏     | 93219/225000 [168:41:58<200:45:48,  5.48s/it]
 41%|████▏     | 93220/225000 [168:42:03<192:59:07,  5.27s/it]
 41%|████▏     | 93221/225000 [168:42:07<185:06:52,  5.06s/it]
 41%|████▏     | 93222/225000 [168:42:11<169:43:17,  4.64s/it]
 41%|████▏     | 93223/225000 [168:42:16<172:48:54,  4.72s/it]
 41%|████▏     | 93224/225000 [168:42:20<163:12:58,  4.46s/it]
 41%|████▏     | 93225/225000 [168:42:24<159:34:01,  4.36s/it]
 41%|████▏     | 93226/225000 [168:42:28<154:05:57,  4.21s/it]
 41%|████▏     | 93227/225000 [168:42:32<155:46:14,  4.26s/it]
 41%|████▏     | 93228/225000 [168:42:36<153:05:28,  4.18s/it]
 41%|████▏     | 93229/225000 [168:42:42<173:15:00,  4.73s/it]
 41%|████▏     | 93230/225000 [168:42:46<166:21:46,  4.55s/it]
 41%|████▏     | 93231/225000 [168:42:55<209:03:46,  5.71s/it]
 41%|████▏     | 93232/225000 [168:42:59<191:39:58,  5.24s/it]
 41%|████▏     | 93233/225000 [168:43:03<181:14:40,  4.95s/it]
 41%|████▏     | 93234/225000 [168:43:07<172:55:53,  4.72s/it]
 41%|████▏     | 93235/225000 [168:43:16<220:33:06,  6.03s/it]
 41%|████▏     | 93236/225000 [168:43:20<192:28:17,  5.26s/it]
 41%|████▏     | 93237/225000 [168:43:23<168:24:31,  4.60s/it]
 41%|████▏     | 93238/225000 [168:43:32<216:46:14,  5.92s/it]
 41%|████▏     | 93239/225000 [168:43:36<201:36:22,  5.51s/it]
 41%|████▏     | 93240/225000 [168:43:40<182:40:09,  4.99s/it]
 41%|████▏     | 93241/225000 [168:43:44<171:07:47,  4.68s/it]
 41%|████▏     | 93242/225000 [168:43:48<158:45:04,  4.34s/it]
 41%|████▏     | 93243/225000 [168:43:51<152:26:07,  4.16s/it]
 41%|████▏     | 93244/225000 [168:43:56<153:27:48,  4.19s/it]
 41%|████▏     | 93245/225000 [168:43:59<148:56:18,  4.07s/it]
 41%|████▏     | 93246/225000 [168:44:03<147:10:48,  4.02s/it]
 41%|████▏     | 93247/225000 [168:44:14<223:22:04,  6.10s/it]
 41%|████▏     | 93248/225000 [168:44:19<204:54:07,  5.60s/it]
 41%|████▏     | 93249/225000 [168:44:22<175:45:05,  4.80s/it]
 41%|████▏     | 93250/225000 [168:44:29<204:39:48,  5.59s/it]
                                                              
{'loss': '0.1788', 'grad_norm': '3.02', 'learning_rate': '3.369e-05', 'epoch': '0.4144', 'num_input_tokens_seen': 276294688, 'train_runtime': '6.075e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93250/225000 [168:44:29<204:39:48,  5.59s/it]
 41%|████▏     | 93251/225000 [168:44:32<179:40:05,  4.91s/it]
 41%|████▏     | 93252/225000 [168:44:35<160:26:34,  4.38s/it]
 41%|████▏     | 93253/225000 [168:44:40<158:40:40,  4.34s/it]
 41%|████▏     | 93254/225000 [168:44:47<190:48:31,  5.21s/it]
 41%|████▏     | 93255/225000 [168:44:51<173:22:35,  4.74s/it]
 41%|████▏     | 93256/225000 [168:44:54<161:00:20,  4.40s/it]
 41%|████▏     | 93257/225000 [168:44:58<153:42:26,  4.20s/it]
 41%|████▏     | 93258/225000 [168:45:06<192:13:14,  5.25s/it]
 41%|████▏     | 93259/225000 [168:45:11<195:02:58,  5.33s/it]
 41%|████▏     | 93260/225000 [168:45:14<172:32:28,  4.71s/it]
 41%|████▏     | 93261/225000 [168:45:23<212:37:31,  5.81s/it]
 41%|████▏     | 93262/225000 [168:45:26<180:15:26,  4.93s/it]
 41%|████▏     | 93263/225000 [168:45:29<166:24:21,  4.55s/it]
 41%|████▏     | 93264/225000 [168:45:33<160:43:46,  4.39s/it]
 41%|████▏     | 93265/225000 [168:45:37<154:34:40,  4.22s/it]
 41%|████▏     | 93266/225000 [168:45:41<154:30:01,  4.22s/it]
 41%|████▏     | 93267/225000 [168:45:46<160:22:48,  4.38s/it]
 41%|████▏     | 93268/225000 [168:45:51<164:49:06,  4.50s/it]
 41%|████▏     | 93269/225000 [168:46:01<227:25:51,  6.22s/it]
 41%|████▏     | 93270/225000 [168:46:06<213:52:39,  5.84s/it]
 41%|████▏     | 93271/225000 [168:46:09<182:24:16,  4.98s/it]
 41%|████▏     | 93272/225000 [168:46:12<163:20:23,  4.46s/it]
 41%|████▏     | 93273/225000 [168:46:22<220:01:55,  6.01s/it]
 41%|████▏     | 93274/225000 [168:46:25<185:19:33,  5.06s/it]
 41%|████▏     | 93275/225000 [168:46:28<167:06:32,  4.57s/it]
 41%|████▏     | 93276/225000 [168:46:36<197:53:30,  5.41s/it]
 41%|████▏     | 93277/225000 [168:46:38<165:56:24,  4.54s/it]
 41%|████▏     | 93278/225000 [168:46:42<156:00:41,  4.26s/it]
 41%|████▏     | 93279/225000 [168:46:46<151:05:50,  4.13s/it]
 41%|████▏     | 93280/225000 [168:46:49<145:32:19,  3.98s/it]
 41%|████▏     | 93281/225000 [168:46:58<200:34:23,  5.48s/it]
 41%|████▏     | 93282/225000 [168:47:07<239:05:20,  6.53s/it]
 41%|████▏     | 93283/225000 [168:47:10<203:05:53,  5.55s/it]
 41%|████▏     | 93284/225000 [168:47:14<179:02:24,  4.89s/it]
 41%|████▏     | 93285/225000 [168:47:17<156:46:51,  4.29s/it]
 41%|████▏     | 93286/225000 [168:47:24<188:53:07,  5.16s/it]
 41%|████▏     | 93287/225000 [168:47:31<206:31:21,  5.64s/it]
 41%|████▏     | 93288/225000 [168:47:34<176:18:58,  4.82s/it]
 41%|████▏     | 93289/225000 [168:47:36<155:24:46,  4.25s/it]
 41%|████▏     | 93290/225000 [168:47:43<186:00:56,  5.08s/it]
 41%|████▏     | 93291/225000 [168:47:46<161:58:17,  4.43s/it]
 41%|████▏     | 93292/225000 [168:47:56<213:32:05,  5.84s/it]
 41%|████▏     | 93293/225000 [168:47:59<192:50:17,  5.27s/it]
 41%|████▏     | 93294/225000 [168:48:03<176:55:00,  4.84s/it]
 41%|████▏     | 93295/225000 [168:48:07<161:14:15,  4.41s/it]
 41%|████▏     | 93296/225000 [168:48:11<154:52:31,  4.23s/it]
 41%|████▏     | 93297/225000 [168:48:15<156:31:45,  4.28s/it]
 41%|████▏     | 93298/225000 [168:48:18<148:58:55,  4.07s/it]
 41%|████▏     | 93299/225000 [168:48:22<138:47:12,  3.79s/it]
 41%|████▏     | 93300/225000 [168:48:32<214:17:25,  5.86s/it]
                                                              
{'loss': '0.1627', 'grad_norm': '2.014', 'learning_rate': '3.367e-05', 'epoch': '0.4147', 'num_input_tokens_seen': 276404832, 'train_runtime': '6.077e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93300/225000 [168:48:32<214:17:25,  5.86s/it]
 41%|████▏     | 93301/225000 [168:48:37<204:07:40,  5.58s/it]
 41%|████▏     | 93302/225000 [168:48:40<170:29:56,  4.66s/it]
 41%|████▏     | 93303/225000 [168:48:43<151:54:33,  4.15s/it]
 41%|████▏     | 93304/225000 [168:48:52<204:58:57,  5.60s/it]
 41%|████▏     | 93305/225000 [168:48:56<189:49:57,  5.19s/it]
 41%|████▏     | 93306/225000 [168:49:00<178:14:23,  4.87s/it]
 41%|████▏     | 93307/225000 [168:49:05<181:16:26,  4.96s/it]
 41%|████▏     | 93308/225000 [168:49:09<172:46:20,  4.72s/it]
 41%|████▏     | 93309/225000 [168:49:13<165:56:48,  4.54s/it]
 41%|████▏     | 93310/225000 [168:49:18<160:10:20,  4.38s/it]
 41%|████▏     | 93311/225000 [168:49:22<161:33:17,  4.42s/it]
 41%|████▏     | 93312/225000 [168:49:26<162:07:50,  4.43s/it]
 41%|████▏     | 93313/225000 [168:49:31<159:21:41,  4.36s/it]
 41%|████▏     | 93314/225000 [168:49:35<153:47:19,  4.20s/it]
 41%|████▏     | 93315/225000 [168:49:39<160:46:02,  4.40s/it]
 41%|████▏     | 93316/225000 [168:49:50<227:45:38,  6.23s/it]
 41%|████▏     | 93317/225000 [168:49:56<231:50:51,  6.34s/it]
 41%|████▏     | 93318/225000 [168:50:00<199:12:49,  5.45s/it]
 41%|████▏     | 93319/225000 [168:50:03<175:22:10,  4.79s/it]
 41%|████▏     | 93320/225000 [168:50:06<155:32:17,  4.25s/it]
 41%|████▏     | 93321/225000 [168:50:14<199:36:37,  5.46s/it]
 41%|████▏     | 93322/225000 [168:50:21<208:55:16,  5.71s/it]
 41%|████▏     | 93323/225000 [168:50:24<181:55:43,  4.97s/it]
 41%|████▏     | 93324/225000 [168:50:27<158:36:16,  4.34s/it]
 41%|████▏     | 93325/225000 [168:50:31<152:49:30,  4.18s/it]
 41%|████▏     | 93326/225000 [168:50:40<215:35:26,  5.89s/it]
 41%|████▏     | 93327/225000 [168:50:46<213:04:30,  5.83s/it]
 41%|████▏     | 93328/225000 [168:50:50<189:05:39,  5.17s/it]
 41%|████▏     | 93329/225000 [168:50:53<169:16:52,  4.63s/it]
 41%|████▏     | 93330/225000 [168:50:56<148:56:49,  4.07s/it]
 41%|████▏     | 93331/225000 [168:50:59<135:50:46,  3.71s/it]
 41%|████▏     | 93332/225000 [168:51:02<135:16:28,  3.70s/it]
 41%|████▏     | 93333/225000 [168:51:11<191:27:57,  5.24s/it]
 41%|████▏     | 93334/225000 [168:51:17<201:12:55,  5.50s/it]
 41%|████▏     | 93335/225000 [168:51:21<177:06:58,  4.84s/it]
 41%|████▏     | 93336/225000 [168:51:28<207:35:17,  5.68s/it]
 41%|████▏     | 93337/225000 [168:51:32<187:56:48,  5.14s/it]
 41%|████▏     | 93338/225000 [168:51:35<160:23:32,  4.39s/it]
 41%|████▏     | 93339/225000 [168:51:41<185:13:51,  5.06s/it]
 41%|████▏     | 93340/225000 [168:51:46<178:25:55,  4.88s/it]
 41%|████▏     | 93341/225000 [168:51:50<164:36:30,  4.50s/it]
 41%|████▏     | 93342/225000 [168:51:53<154:38:09,  4.23s/it]
 41%|████▏     | 93343/225000 [168:51:56<142:23:40,  3.89s/it]
 41%|████▏     | 93344/225000 [168:52:00<137:48:01,  3.77s/it]
 41%|████▏     | 93345/225000 [168:52:05<152:07:43,  4.16s/it]
 41%|████▏     | 93346/225000 [168:52:13<196:39:45,  5.38s/it]
 41%|████▏     | 93347/225000 [168:52:16<175:50:23,  4.81s/it]
 41%|████▏     | 93348/225000 [168:52:20<164:15:11,  4.49s/it]
 41%|████▏     | 93349/225000 [168:52:28<197:10:48,  5.39s/it]
 41%|████▏     | 93350/225000 [168:52:31<169:06:50,  4.62s/it]
                                                              
{'loss': '0.1896', 'grad_norm': '3.065', 'learning_rate': '3.366e-05', 'epoch': '0.4149', 'num_input_tokens_seen': 276513384, 'train_runtime': '6.08e+05', 'train_tokens_per_second': '454.8'}

 41%|████▏     | 93350/225000 [168:52:31<169:06:50,  4.62s/it]
 41%|████▏     | 93351/225000 [168:52:35<162:46:37,  4.45s/it]
 41%|████▏     | 93352/225000 [168:52:38<155:29:25,  4.25s/it]
 41%|████▏     | 93353/225000 [168:52:42<150:06:16,  4.10s/it]
 41%|████▏     | 93354/225000 [168:52:47<153:53:17,  4.21s/it]
 41%|████▏     | 93355/225000 [168:52:50<149:17:52,  4.08s/it]
 41%|████▏     | 93356/225000 [168:52:54<148:13:26,  4.05s/it]
 41%|████▏     | 93357/225000 [168:52:59<150:27:14,  4.11s/it]
 41%|████▏     | 93358/225000 [168:53:04<166:44:22,  4.56s/it]
 41%|████▏     | 93359/225000 [168:53:15<237:21:08,  6.49s/it]
 41%|████▏     | 93360/225000 [168:53:21<229:25:31,  6.27s/it]
 41%|████▏     | 93361/225000 [168:53:24<198:05:41,  5.42s/it]
 41%|████▏     | 93362/225000 [168:53:31<209:01:13,  5.72s/it]
 41%|████▏     | 93363/225000 [168:53:34<181:32:24,  4.96s/it]
 41%|████▏     | 93364/225000 [168:53:37<158:56:27,  4.35s/it]
 41%|████▏     | 93365/225000 [168:53:43<175:16:04,  4.79s/it]
 41%|████▏     | 93366/225000 [168:53:52<222:19:00,  6.08s/it]
 41%|████▏     | 93367/225000 [168:53:55<191:01:53,  5.22s/it]
 41%|████▏     | 93368/225000 [168:53:58<169:21:07,  4.63s/it]
 41%|████▏     | 93369/225000 [168:54:01<152:40:26,  4.18s/it]
 41%|████▏     | 93370/225000 [168:54:05<149:32:36,  4.09s/it]
 41%|████▏     | 93371/225000 [168:54:09<146:52:21,  4.02s/it]
 41%|████▏     | 93372/225000 [168:54:16<176:31:42,  4.83s/it]
 41%|████▏     | 93373/225000 [168:54:26<239:33:05,  6.55s/it]
 41%|████▏     | 93374/225000 [168:54:29<198:42:01,  5.43s/it]
 42%|████▏     | 93375/225000 [168:54:33<174:26:31,  4.77s/it]
 42%|████▏     | 93376/225000 [168:54:37<169:26:40,  4.63s/it]
 42%|████▏     | 93377/225000 [168:54:42<175:02:29,  4.79s/it]
 42%|████▏     | 93378/225000 [168:54:46<168:15:12,  4.60s/it]
 42%|████▏     | 93379/225000 [168:54:50<163:49:02,  4.48s/it]
 42%|████▏     | 93380/225000 [168:54:54<159:19:16,  4.36s/it]
 42%|████▏     | 93381/225000 [168:54:59<159:57:14,  4.38s/it]
 42%|████▏     | 93382/225000 [168:55:03<162:29:37,  4.44s/it]
 42%|████▏     | 93383/225000 [168:55:08<160:59:30,  4.40s/it]
 42%|████▏     | 93384/225000 [168:55:13<167:13:51,  4.57s/it]
 42%|████▏     | 93385/225000 [168:55:18<175:00:29,  4.79s/it]
 42%|████▏     | 93386/225000 [168:55:22<165:10:38,  4.52s/it]
 42%|████▏     | 93387/225000 [168:55:34<247:02:14,  6.76s/it]
 42%|████▏     | 93388/225000 [168:55:39<229:30:26,  6.28s/it]
 42%|████▏     | 93389/225000 [168:55:43<205:05:58,  5.61s/it]
 42%|████▏     | 93390/225000 [168:55:48<197:11:09,  5.39s/it]
 42%|████▏     | 93391/225000 [168:55:53<196:34:54,  5.38s/it]
 42%|████▏     | 93392/225000 [168:55:58<186:29:46,  5.10s/it]
 42%|████▏     | 93393/225000 [168:56:02<174:06:08,  4.76s/it]
 42%|████▏     | 93394/225000 [168:56:06<170:05:13,  4.65s/it]
 42%|████▏     | 93395/225000 [168:56:11<169:26:13,  4.63s/it]
 42%|████▏     | 93396/225000 [168:56:15<163:04:04,  4.46s/it]
 42%|████▏     | 93397/225000 [168:56:19<158:00:28,  4.32s/it]
 42%|████▏     | 93398/225000 [168:56:23<155:43:56,  4.26s/it]
 42%|████▏     | 93399/225000 [168:56:27<154:50:50,  4.24s/it]
 42%|████▏     | 93400/225000 [168:56:32<158:45:33,  4.34s/it]
                                                              
{'loss': '0.1739', 'grad_norm': '4.587', 'learning_rate': '3.364e-05', 'epoch': '0.4151', 'num_input_tokens_seen': 276624016, 'train_runtime': '6.082e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93400/225000 [168:56:32<158:45:33,  4.34s/it]
 42%|████▏     | 93401/225000 [168:56:36<162:56:37,  4.46s/it]
 42%|████▏     | 93402/225000 [168:56:43<186:00:43,  5.09s/it]
 42%|████▏     | 93403/225000 [168:56:53<235:18:13,  6.44s/it]
 42%|████▏     | 93404/225000 [168:56:57<218:24:33,  5.97s/it]
 42%|████▏     | 93405/225000 [168:57:04<227:12:47,  6.22s/it]
 42%|████▏     | 93406/225000 [168:57:08<198:50:26,  5.44s/it]
 42%|████▏     | 93407/225000 [168:57:12<182:49:23,  5.00s/it]
 42%|████▏     | 93408/225000 [168:57:16<175:04:04,  4.79s/it]
 42%|████▏     | 93409/225000 [168:57:20<164:51:56,  4.51s/it]
 42%|████▏     | 93410/225000 [168:57:24<159:09:39,  4.35s/it]
 42%|████▏     | 93411/225000 [168:57:28<155:46:23,  4.26s/it]
 42%|████▏     | 93412/225000 [168:57:32<157:36:53,  4.31s/it]
 42%|████▏     | 93413/225000 [168:57:36<151:18:50,  4.14s/it]
 42%|████▏     | 93414/225000 [168:57:40<146:21:54,  4.00s/it]
 42%|████▏     | 93415/225000 [168:57:49<204:01:15,  5.58s/it]
 42%|████▏     | 93416/225000 [168:57:57<231:46:19,  6.34s/it]
 42%|████▏     | 93417/225000 [168:58:00<191:38:39,  5.24s/it]
 42%|████▏     | 93418/225000 [168:58:03<170:34:18,  4.67s/it]
 42%|████▏     | 93419/225000 [168:58:06<152:55:51,  4.18s/it]
 42%|████▏     | 93420/225000 [168:58:14<195:44:39,  5.36s/it]
 42%|████▏     | 93421/225000 [168:58:20<199:25:54,  5.46s/it]
 42%|████▏     | 93422/225000 [168:58:26<202:34:54,  5.54s/it]
 42%|████▏     | 93423/225000 [168:58:30<189:22:08,  5.18s/it]
 42%|████▏     | 93424/225000 [168:58:34<177:18:25,  4.85s/it]
 42%|████▏     | 93425/225000 [168:58:38<166:58:48,  4.57s/it]
 42%|████▏     | 93426/225000 [168:58:43<164:09:59,  4.49s/it]
 42%|████▏     | 93427/225000 [168:58:47<162:28:28,  4.45s/it]
 42%|████▏     | 93428/225000 [168:58:57<222:39:29,  6.09s/it]
 42%|████▏     | 93429/225000 [168:59:03<219:51:27,  6.02s/it]
 42%|████▏     | 93430/225000 [168:59:05<183:53:14,  5.03s/it]
 42%|████▏     | 93431/225000 [168:59:09<163:32:32,  4.47s/it]
 42%|████▏     | 93432/225000 [168:59:16<198:11:01,  5.42s/it]
 42%|████▏     | 93433/225000 [168:59:21<187:40:17,  5.14s/it]
 42%|████▏     | 93434/225000 [168:59:27<198:43:36,  5.44s/it]
 42%|████▏     | 93435/225000 [168:59:30<169:46:13,  4.65s/it]
 42%|████▏     | 93436/225000 [168:59:32<150:32:44,  4.12s/it]
 42%|████▏     | 93437/225000 [168:59:39<175:52:58,  4.81s/it]
 42%|████▏     | 93438/225000 [168:59:45<188:32:41,  5.16s/it]
 42%|████▏     | 93439/225000 [168:59:48<162:08:42,  4.44s/it]
 42%|████▏     | 93440/225000 [168:59:51<151:45:30,  4.15s/it]
 42%|████▏     | 93441/225000 [168:59:58<177:34:07,  4.86s/it]
 42%|████▏     | 93442/225000 [169:00:05<202:13:15,  5.53s/it]
 42%|████▏     | 93443/225000 [169:00:08<180:03:14,  4.93s/it]
 42%|████▏     | 93444/225000 [169:00:16<207:41:04,  5.68s/it]
 42%|████▏     | 93445/225000 [169:00:20<188:56:24,  5.17s/it]
 42%|████▏     | 93446/225000 [169:00:27<208:28:20,  5.70s/it]
 42%|████▏     | 93447/225000 [169:00:29<176:38:18,  4.83s/it]
 42%|████▏     | 93448/225000 [169:00:36<201:22:12,  5.51s/it]
 42%|████▏     | 93449/225000 [169:00:39<173:52:55,  4.76s/it]
 42%|████▏     | 93450/225000 [169:00:42<152:13:01,  4.17s/it]
                                                              
{'loss': '0.1778', 'grad_norm': '3.569', 'learning_rate': '3.362e-05', 'epoch': '0.4153', 'num_input_tokens_seen': 276734176, 'train_runtime': '6.084e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93450/225000 [169:00:42<152:13:01,  4.17s/it]
 42%|████▏     | 93451/225000 [169:00:46<147:57:36,  4.05s/it]
 42%|████▏     | 93452/225000 [169:00:50<145:56:20,  3.99s/it]
 42%|████▏     | 93453/225000 [169:00:53<139:14:41,  3.81s/it]
 42%|████▏     | 93454/225000 [169:00:58<145:38:53,  3.99s/it]
 42%|████▏     | 93455/225000 [169:01:01<139:46:06,  3.83s/it]
 42%|████▏     | 93456/225000 [169:01:10<191:07:30,  5.23s/it]
 42%|████▏     | 93457/225000 [169:01:18<220:19:38,  6.03s/it]
 42%|████▏     | 93458/225000 [169:01:21<193:39:47,  5.30s/it]
 42%|████▏     | 93459/225000 [169:01:24<171:21:04,  4.69s/it]
 42%|████▏     | 93460/225000 [169:01:29<173:43:21,  4.75s/it]
 42%|████▏     | 93461/225000 [169:01:40<233:51:15,  6.40s/it]
 42%|████▏     | 93462/225000 [169:01:43<204:53:07,  5.61s/it]
 42%|████▏     | 93463/225000 [169:01:46<175:33:39,  4.80s/it]
 42%|████▏     | 93464/225000 [169:01:53<196:05:34,  5.37s/it]
 42%|████▏     | 93465/225000 [169:01:57<180:27:30,  4.94s/it]
 42%|████▏     | 93466/225000 [169:02:01<167:18:57,  4.58s/it]
 42%|████▏     | 93467/225000 [169:02:05<170:14:41,  4.66s/it]
 42%|████▏     | 93468/225000 [169:02:09<159:13:43,  4.36s/it]
 42%|████▏     | 93469/225000 [169:02:13<157:54:39,  4.32s/it]
 42%|████▏     | 93470/225000 [169:02:18<159:23:06,  4.36s/it]
 42%|████▏     | 93471/225000 [169:02:25<193:19:35,  5.29s/it]
 42%|████▏     | 93472/225000 [169:02:30<190:17:44,  5.21s/it]
 42%|████▏     | 93473/225000 [169:02:35<187:29:09,  5.13s/it]
 42%|████▏     | 93474/225000 [169:02:40<186:41:51,  5.11s/it]
 42%|████▏     | 93475/225000 [169:02:45<183:32:17,  5.02s/it]
 42%|████▏     | 93476/225000 [169:02:49<174:32:54,  4.78s/it]
 42%|████▏     | 93477/225000 [169:02:54<168:15:38,  4.61s/it]
 42%|████▏     | 93478/225000 [169:02:57<155:42:03,  4.26s/it]
 42%|████▏     | 93479/225000 [169:03:02<160:06:56,  4.38s/it]
 42%|████▏     | 93480/225000 [169:03:08<176:30:08,  4.83s/it]
 42%|████▏     | 93481/225000 [169:03:18<236:25:23,  6.47s/it]
 42%|████▏     | 93482/225000 [169:03:22<205:58:17,  5.64s/it]
 42%|████▏     | 93483/225000 [169:03:24<175:17:46,  4.80s/it]
 42%|████▏     | 93484/225000 [169:03:28<161:33:25,  4.42s/it]
 42%|████▏     | 93485/225000 [169:03:32<154:47:01,  4.24s/it]
 42%|████▏     | 93486/225000 [169:03:41<210:31:35,  5.76s/it]
 42%|████▏     | 93487/225000 [169:03:45<187:32:07,  5.13s/it]
 42%|████▏     | 93488/225000 [169:03:48<165:36:07,  4.53s/it]
 42%|████▏     | 93489/225000 [169:04:00<244:10:25,  6.68s/it]
 42%|████▏     | 93490/225000 [169:04:03<210:05:30,  5.75s/it]
 42%|████▏     | 93491/225000 [169:04:06<183:02:23,  5.01s/it]
 42%|████▏     | 93492/225000 [169:04:10<165:25:38,  4.53s/it]
 42%|████▏     | 93493/225000 [169:04:14<157:48:31,  4.32s/it]
 42%|████▏     | 93494/225000 [169:04:26<242:18:03,  6.63s/it]
 42%|████▏     | 93495/225000 [169:04:29<211:49:24,  5.80s/it]
 42%|████▏     | 93496/225000 [169:04:33<184:30:05,  5.05s/it]
 42%|████▏     | 93497/225000 [169:04:37<171:08:35,  4.69s/it]
 42%|████▏     | 93498/225000 [169:04:40<155:51:55,  4.27s/it]
 42%|████▏     | 93499/225000 [169:04:44<157:44:02,  4.32s/it]
 42%|████▏     | 93500/225000 [169:04:50<176:36:58,  4.84s/it]
                                                              
{'loss': '0.1453', 'grad_norm': '2.359', 'learning_rate': '3.36e-05', 'epoch': '0.4156', 'num_input_tokens_seen': 276846688, 'train_runtime': '6.087e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93500/225000 [169:04:50<176:36:58,  4.84s/it]
 42%|████▏     | 93501/225000 [169:04:55<170:02:21,  4.66s/it]
 42%|████▏     | 93502/225000 [169:04:59<169:58:46,  4.65s/it]
 42%|████▏     | 93503/225000 [169:05:04<170:24:32,  4.67s/it]
 42%|████▏     | 93504/225000 [169:05:16<250:36:10,  6.86s/it]
 42%|████▏     | 93505/225000 [169:05:19<211:15:36,  5.78s/it]
 42%|████▏     | 93506/225000 [169:05:24<198:08:11,  5.42s/it]
 42%|████▏     | 93507/225000 [169:05:30<211:39:29,  5.79s/it]
 42%|████▏     | 93508/225000 [169:05:34<184:07:27,  5.04s/it]
 42%|████▏     | 93509/225000 [169:05:42<221:13:03,  6.06s/it]
 42%|████▏     | 93510/225000 [169:05:45<183:53:05,  5.03s/it]
 42%|████▏     | 93511/225000 [169:05:48<164:17:23,  4.50s/it]
 42%|████▏     | 93512/225000 [169:05:52<155:48:09,  4.27s/it]
 42%|████▏     | 93513/225000 [169:05:57<160:42:37,  4.40s/it]
 42%|████▏     | 93514/225000 [169:06:00<155:31:12,  4.26s/it]
 42%|████▏     | 93515/225000 [169:06:05<154:29:51,  4.23s/it]
 42%|████▏     | 93516/225000 [169:06:08<150:18:01,  4.12s/it]
 42%|████▏     | 93517/225000 [169:06:13<156:42:47,  4.29s/it]
 42%|████▏     | 93518/225000 [169:06:22<202:27:34,  5.54s/it]
 42%|████▏     | 93519/225000 [169:06:26<185:10:52,  5.07s/it]
 42%|████▏     | 93520/225000 [169:06:30<174:58:12,  4.79s/it]
 42%|████▏     | 93521/225000 [169:06:34<164:01:24,  4.49s/it]
 42%|████▏     | 93522/225000 [169:06:37<156:23:17,  4.28s/it]
 42%|████▏     | 93523/225000 [169:06:42<158:55:04,  4.35s/it]
 42%|████▏     | 93524/225000 [169:06:46<155:42:01,  4.26s/it]
 42%|████▏     | 93525/225000 [169:06:51<161:15:32,  4.42s/it]
 42%|████▏     | 93526/225000 [169:06:59<206:47:27,  5.66s/it]
 42%|████▏     | 93527/225000 [169:07:08<239:06:11,  6.55s/it]
 42%|████▏     | 93528/225000 [169:07:13<220:18:01,  6.03s/it]
 42%|████▏     | 93529/225000 [169:07:19<218:53:02,  5.99s/it]
 42%|████▏     | 93530/225000 [169:07:22<190:11:48,  5.21s/it]
 42%|████▏     | 93531/225000 [169:07:26<177:30:51,  4.86s/it]
 42%|████▏     | 93532/225000 [169:07:30<163:39:50,  4.48s/it]
 42%|████▏     | 93533/225000 [169:07:33<156:53:54,  4.30s/it]
 42%|████▏     | 93534/225000 [169:07:43<218:43:14,  5.99s/it]
 42%|████▏     | 93535/225000 [169:07:46<183:21:24,  5.02s/it]
 42%|████▏     | 93536/225000 [169:07:49<162:49:21,  4.46s/it]
 42%|████▏     | 93537/225000 [169:07:58<204:20:06,  5.60s/it]
 42%|████▏     | 93538/225000 [169:08:01<181:08:34,  4.96s/it]
 42%|████▏     | 93539/225000 [169:08:04<161:46:08,  4.43s/it]
 42%|████▏     | 93540/225000 [169:08:08<152:02:03,  4.16s/it]
 42%|████▏     | 93541/225000 [169:08:12<148:43:04,  4.07s/it]
 42%|████▏     | 93542/225000 [169:08:16<155:10:49,  4.25s/it]
 42%|████▏     | 93543/225000 [169:08:26<214:58:04,  5.89s/it]
 42%|████▏     | 93544/225000 [169:08:33<223:09:27,  6.11s/it]
 42%|████▏     | 93545/225000 [169:08:36<193:15:12,  5.29s/it]
 42%|████▏     | 93546/225000 [169:08:43<206:49:01,  5.66s/it]
 42%|████▏     | 93547/225000 [169:08:47<197:20:41,  5.40s/it]
 42%|████▏     | 93548/225000 [169:08:50<172:17:06,  4.72s/it]
 42%|████▏     | 93549/225000 [169:08:54<157:47:30,  4.32s/it]
 42%|████▏     | 93550/225000 [169:08:58<156:19:32,  4.28s/it]
                                                              
{'loss': '0.1631', 'grad_norm': '4.625', 'learning_rate': '3.359e-05', 'epoch': '0.4158', 'num_input_tokens_seen': 276959648, 'train_runtime': '6.089e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93550/225000 [169:08:58<156:19:32,  4.28s/it]
 42%|████▏     | 93551/225000 [169:09:04<174:15:24,  4.77s/it]
 42%|████▏     | 93552/225000 [169:09:13<219:54:24,  6.02s/it]
 42%|████▏     | 93553/225000 [169:09:17<194:03:35,  5.31s/it]
 42%|████▏     | 93554/225000 [169:09:24<216:30:09,  5.93s/it]
 42%|████▏     | 93555/225000 [169:09:28<198:59:29,  5.45s/it]
 42%|████▏     | 93556/225000 [169:09:32<175:32:10,  4.81s/it]
 42%|████▏     | 93557/225000 [169:09:35<156:50:09,  4.30s/it]
 42%|████▏     | 93558/225000 [169:09:39<151:54:53,  4.16s/it]
 42%|████▏     | 93559/225000 [169:09:43<151:10:13,  4.14s/it]
 42%|████▏     | 93560/225000 [169:09:47<149:26:26,  4.09s/it]
 42%|████▏     | 93561/225000 [169:09:50<146:13:23,  4.00s/it]
 42%|████▏     | 93562/225000 [169:09:55<154:59:04,  4.24s/it]
 42%|████▏     | 93563/225000 [169:10:06<222:22:43,  6.09s/it]
 42%|████▏     | 93564/225000 [169:10:12<226:47:30,  6.21s/it]
 42%|████▏     | 93565/225000 [169:10:16<199:29:21,  5.46s/it]
 42%|████▏     | 93566/225000 [169:10:20<184:36:49,  5.06s/it]
 42%|████▏     | 93567/225000 [169:10:24<175:47:23,  4.81s/it]
 42%|████▏     | 93568/225000 [169:10:28<169:39:21,  4.65s/it]
 42%|████▏     | 93569/225000 [169:10:33<171:10:49,  4.69s/it]
 42%|████▏     | 93570/225000 [169:10:43<229:07:03,  6.28s/it]
 42%|████▏     | 93571/225000 [169:10:49<223:13:49,  6.11s/it]
 42%|████▏     | 93572/225000 [169:10:52<190:47:20,  5.23s/it]
 42%|████▏     | 93573/225000 [169:11:00<224:50:04,  6.16s/it]
 42%|████▏     | 93574/225000 [169:11:04<194:58:32,  5.34s/it]
 42%|████▏     | 93575/225000 [169:11:08<184:00:16,  5.04s/it]
 42%|████▏     | 93576/225000 [169:11:12<171:14:36,  4.69s/it]
 42%|████▏     | 93577/225000 [169:11:17<168:28:39,  4.62s/it]
 42%|████▏     | 93578/225000 [169:11:21<171:30:40,  4.70s/it]
 42%|████▏     | 93579/225000 [169:11:27<185:39:57,  5.09s/it]
 42%|████▏     | 93580/225000 [169:11:32<175:53:10,  4.82s/it]
 42%|████▏     | 93581/225000 [169:11:36<166:57:56,  4.57s/it]
 42%|████▏     | 93582/225000 [169:11:41<172:23:40,  4.72s/it]
 42%|████▏     | 93583/225000 [169:11:44<161:36:31,  4.43s/it]
 42%|████▏     | 93584/225000 [169:11:49<163:11:36,  4.47s/it]
 42%|████▏     | 93585/225000 [169:11:53<154:34:00,  4.23s/it]
 42%|████▏     | 93586/225000 [169:11:58<170:42:42,  4.68s/it]
 42%|████▏     | 93587/225000 [169:12:10<242:55:52,  6.65s/it]
 42%|████▏     | 93588/225000 [169:12:14<217:00:42,  5.94s/it]
 42%|████▏     | 93589/225000 [169:12:17<183:59:47,  5.04s/it]
 42%|████▏     | 93590/225000 [169:12:21<174:10:31,  4.77s/it]
 42%|████▏     | 93591/225000 [169:12:31<236:23:11,  6.48s/it]
 42%|████▏     | 93592/225000 [169:12:35<200:22:32,  5.49s/it]
 42%|████▏     | 93593/225000 [169:12:38<175:56:36,  4.82s/it]
 42%|████▏     | 93594/225000 [169:12:41<155:24:07,  4.26s/it]
 42%|████▏     | 93595/225000 [169:12:48<182:34:06,  5.00s/it]
 42%|████▏     | 93596/225000 [169:12:51<169:48:34,  4.65s/it]
 42%|████▏     | 93597/225000 [169:12:55<162:38:05,  4.46s/it]
 42%|████▏     | 93598/225000 [169:12:59<154:39:13,  4.24s/it]
 42%|████▏     | 93599/225000 [169:13:04<156:11:30,  4.28s/it]
 42%|████▏     | 93600/225000 [169:13:08<158:29:48,  4.34s/it]
                                                              
{'loss': '0.1547', 'grad_norm': '5.524', 'learning_rate': '3.357e-05', 'epoch': '0.416', 'num_input_tokens_seen': 277071152, 'train_runtime': '6.092e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93600/225000 [169:13:08<158:29:48,  4.34s/it]
 42%|████▏     | 93601/225000 [169:13:18<224:07:21,  6.14s/it]
 42%|████▏     | 93602/225000 [169:13:23<203:27:36,  5.57s/it]
 42%|████▏     | 93603/225000 [169:13:27<186:57:53,  5.12s/it]
 42%|████▏     | 93604/225000 [169:13:30<165:31:12,  4.53s/it]
 42%|████▏     | 93605/225000 [169:13:37<198:28:57,  5.44s/it]
 42%|████▏     | 93606/225000 [169:13:43<202:42:28,  5.55s/it]
 42%|████▏     | 93607/225000 [169:13:48<191:08:02,  5.24s/it]
 42%|████▏     | 93608/225000 [169:13:51<172:05:58,  4.72s/it]
 42%|████▏     | 93609/225000 [169:13:56<168:06:01,  4.61s/it]
 42%|████▏     | 93610/225000 [169:13:59<159:16:26,  4.36s/it]
 42%|████▏     | 93611/225000 [169:14:04<159:55:30,  4.38s/it]
 42%|████▏     | 93612/225000 [169:14:13<215:28:21,  5.90s/it]
 42%|████▏     | 93613/225000 [169:14:21<235:27:07,  6.45s/it]
 42%|████▏     | 93614/225000 [169:14:27<234:35:22,  6.43s/it]
 42%|████▏     | 93615/225000 [169:14:30<196:19:01,  5.38s/it]
 42%|████▏     | 93616/225000 [169:14:34<176:24:12,  4.83s/it]
 42%|████▏     | 93617/225000 [169:14:42<213:54:59,  5.86s/it]
 42%|████▏     | 93618/225000 [169:14:47<201:22:49,  5.52s/it]
 42%|████▏     | 93619/225000 [169:14:50<179:50:08,  4.93s/it]
 42%|████▏     | 93620/225000 [169:14:54<170:02:18,  4.66s/it]
 42%|████▏     | 93621/225000 [169:14:58<161:22:17,  4.42s/it]
 42%|████▏     | 93622/225000 [169:15:03<160:48:37,  4.41s/it]
 42%|████▏     | 93623/225000 [169:15:06<153:40:15,  4.21s/it]
 42%|████▏     | 93624/225000 [169:15:10<152:17:43,  4.17s/it]
 42%|████▏     | 93625/225000 [169:15:15<151:11:11,  4.14s/it]
 42%|████▏     | 93626/225000 [169:15:19<156:31:31,  4.29s/it]
 42%|████▏     | 93627/225000 [169:15:23<150:35:10,  4.13s/it]
 42%|████▏     | 93628/225000 [169:15:31<199:12:49,  5.46s/it]
 42%|████▏     | 93629/225000 [169:15:38<214:53:59,  5.89s/it]
 42%|████▏     | 93630/225000 [169:15:41<184:31:00,  5.06s/it]
 42%|████▏     | 93631/225000 [169:15:50<220:11:50,  6.03s/it]
 42%|████▏     | 93632/225000 [169:15:54<200:04:43,  5.48s/it]
 42%|████▏     | 93633/225000 [169:15:59<197:46:54,  5.42s/it]
 42%|████▏     | 93634/225000 [169:16:04<193:55:24,  5.31s/it]
 42%|████▏     | 93635/225000 [169:16:08<173:54:24,  4.77s/it]
 42%|████▏     | 93636/225000 [169:16:16<209:25:29,  5.74s/it]
 42%|████▏     | 93637/225000 [169:16:19<182:30:27,  5.00s/it]
 42%|████▏     | 93638/225000 [169:16:23<169:22:54,  4.64s/it]
 42%|████▏     | 93639/225000 [169:16:26<152:38:54,  4.18s/it]
 42%|████▏     | 93640/225000 [169:16:30<146:04:09,  4.00s/it]
 42%|████▏     | 93641/225000 [169:16:35<161:30:53,  4.43s/it]
 42%|████▏     | 93642/225000 [169:16:46<229:57:59,  6.30s/it]
 42%|████▏     | 93643/225000 [169:16:49<193:26:02,  5.30s/it]
 42%|████▏     | 93644/225000 [169:16:54<194:06:27,  5.32s/it]
 42%|████▏     | 93645/225000 [169:16:58<184:40:07,  5.06s/it]
 42%|████▏     | 93646/225000 [169:17:03<179:14:12,  4.91s/it]
 42%|████▏     | 93647/225000 [169:17:09<193:41:21,  5.31s/it]
 42%|████▏     | 93648/225000 [169:17:13<180:01:35,  4.93s/it]
 42%|████▏     | 93649/225000 [169:17:17<164:31:19,  4.51s/it]
 42%|████▏     | 93650/225000 [169:17:20<153:54:42,  4.22s/it]
                                                              
{'loss': '0.1741', 'grad_norm': '4.785', 'learning_rate': '3.355e-05', 'epoch': '0.4162', 'num_input_tokens_seen': 277183800, 'train_runtime': '6.094e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93650/225000 [169:17:20<153:54:42,  4.22s/it]
 42%|████▏     | 93651/225000 [169:17:24<146:07:27,  4.00s/it]
 42%|████▏     | 93652/225000 [169:17:32<192:35:32,  5.28s/it]
 42%|████▏     | 93653/225000 [169:17:39<204:39:07,  5.61s/it]
 42%|████▏     | 93654/225000 [169:17:43<193:03:44,  5.29s/it]
 42%|████▏     | 93655/225000 [169:17:48<192:27:11,  5.27s/it]
 42%|████▏     | 93656/225000 [169:17:56<218:06:38,  5.98s/it]
 42%|████▏     | 93657/225000 [169:17:59<190:31:53,  5.22s/it]
 42%|████▏     | 93658/225000 [169:18:03<171:25:08,  4.70s/it]
 42%|████▏     | 93659/225000 [169:18:07<164:09:59,  4.50s/it]
 42%|████▏     | 93660/225000 [169:18:12<171:11:05,  4.69s/it]
 42%|████▏     | 93661/225000 [169:18:17<169:18:50,  4.64s/it]
 42%|████▏     | 93662/225000 [169:18:21<171:32:09,  4.70s/it]
 42%|████▏     | 93663/225000 [169:18:26<165:56:45,  4.55s/it]
 42%|████▏     | 93664/225000 [169:18:31<171:26:32,  4.70s/it]
 42%|████▏     | 93665/225000 [169:18:34<157:43:59,  4.32s/it]
 42%|████▏     | 93666/225000 [169:18:40<171:35:38,  4.70s/it]
 42%|████▏     | 93667/225000 [169:18:46<184:34:20,  5.06s/it]
 42%|████▏     | 93668/225000 [169:18:51<183:54:53,  5.04s/it]
 42%|████▏     | 93669/225000 [169:19:00<236:04:37,  6.47s/it]
 42%|████▏     | 93670/225000 [169:19:09<254:24:49,  6.97s/it]
 42%|████▏     | 93671/225000 [169:19:12<214:43:32,  5.89s/it]
 42%|████▏     | 93672/225000 [169:19:17<204:28:33,  5.61s/it]
 42%|████▏     | 93673/225000 [169:19:25<236:23:04,  6.48s/it]
 42%|████▏     | 93674/225000 [169:19:29<201:59:41,  5.54s/it]
 42%|████▏     | 93675/225000 [169:19:32<179:33:40,  4.92s/it]
 42%|████▏     | 93676/225000 [169:19:35<158:42:31,  4.35s/it]
 42%|████▏     | 93677/225000 [169:19:46<224:38:12,  6.16s/it]
 42%|████▏     | 93678/225000 [169:19:49<196:33:54,  5.39s/it]
 42%|████▏     | 93679/225000 [169:19:57<218:13:06,  5.98s/it]
 42%|████▏     | 93680/225000 [169:20:01<199:47:07,  5.48s/it]
 42%|████▏     | 93681/225000 [169:20:05<189:10:33,  5.19s/it]
 42%|████▏     | 93682/225000 [169:20:10<178:46:30,  4.90s/it]
 42%|████▏     | 93683/225000 [169:20:14<172:36:05,  4.73s/it]
 42%|████▏     | 93684/225000 [169:20:18<167:31:12,  4.59s/it]
 42%|████▏     | 93685/225000 [169:20:22<155:23:58,  4.26s/it]
 42%|████▏     | 93686/225000 [169:20:26<151:04:09,  4.14s/it]
 42%|████▏     | 93687/225000 [169:20:31<164:02:12,  4.50s/it]
 42%|████▏     | 93688/225000 [169:20:42<232:24:34,  6.37s/it]
 42%|████▏     | 93689/225000 [169:20:47<217:25:27,  5.96s/it]
 42%|████▏     | 93690/225000 [169:20:51<196:40:06,  5.39s/it]
 42%|████▏     | 93691/225000 [169:20:55<180:30:35,  4.95s/it]
 42%|████▏     | 93692/225000 [169:20:58<168:38:51,  4.62s/it]
 42%|████▏     | 93693/225000 [169:21:02<153:35:44,  4.21s/it]
 42%|████▏     | 93694/225000 [169:21:11<209:27:29,  5.74s/it]
 42%|████▏     | 93695/225000 [169:21:16<196:04:16,  5.38s/it]
 42%|████▏     | 93696/225000 [169:21:19<178:19:59,  4.89s/it]
 42%|████▏     | 93697/225000 [169:21:22<158:45:00,  4.35s/it]
 42%|████▏     | 93698/225000 [169:21:26<148:19:11,  4.07s/it]
 42%|████▏     | 93699/225000 [169:21:29<143:23:48,  3.93s/it]
 42%|████▏     | 93700/225000 [169:21:35<163:20:11,  4.48s/it]
                                                              
{'loss': '0.1627', 'grad_norm': '2.867', 'learning_rate': '3.354e-05', 'epoch': '0.4164', 'num_input_tokens_seen': 277298568, 'train_runtime': '6.097e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93700/225000 [169:21:35<163:20:11,  4.48s/it]
 42%|████▏     | 93701/225000 [169:21:46<237:52:48,  6.52s/it]
 42%|████▏     | 93702/225000 [169:21:49<194:26:41,  5.33s/it]
 42%|████▏     | 93703/225000 [169:21:52<171:46:36,  4.71s/it]
 42%|████▏     | 93704/225000 [169:21:56<161:12:18,  4.42s/it]
 42%|████▏     | 93705/225000 [169:22:00<160:09:36,  4.39s/it]
 42%|████▏     | 93706/225000 [169:22:11<225:05:52,  6.17s/it]
 42%|████▏     | 93707/225000 [169:22:15<207:25:58,  5.69s/it]
 42%|████▏     | 93708/225000 [169:22:23<227:46:46,  6.25s/it]
 42%|████▏     | 93709/225000 [169:22:26<198:26:37,  5.44s/it]
 42%|████▏     | 93710/225000 [169:22:30<182:09:23,  4.99s/it]
 42%|████▏     | 93711/225000 [169:22:34<168:02:51,  4.61s/it]
 42%|████▏     | 93712/225000 [169:22:38<158:03:59,  4.33s/it]
 42%|████▏     | 93713/225000 [169:22:46<204:45:49,  5.61s/it]
 42%|████▏     | 93714/225000 [169:22:52<208:52:46,  5.73s/it]
 42%|████▏     | 93715/225000 [169:22:59<216:57:35,  5.95s/it]
 42%|████▏     | 93716/225000 [169:23:02<184:45:08,  5.07s/it]
 42%|████▏     | 93717/225000 [169:23:09<212:10:51,  5.82s/it]
 42%|████▏     | 93718/225000 [169:23:13<186:42:43,  5.12s/it]
 42%|████▏     | 93719/225000 [169:23:16<162:09:30,  4.45s/it]
 42%|████▏     | 93720/225000 [169:23:26<227:46:30,  6.25s/it]
 42%|████▏     | 93721/225000 [169:23:30<197:28:21,  5.42s/it]
 42%|████▏     | 93722/225000 [169:23:37<213:47:07,  5.86s/it]
 42%|████▏     | 93723/225000 [169:23:40<186:55:55,  5.13s/it]
 42%|████▏     | 93724/225000 [169:23:47<205:17:15,  5.63s/it]
 42%|████▏     | 93725/225000 [169:23:50<183:29:28,  5.03s/it]
 42%|████▏     | 93726/225000 [169:23:54<165:38:46,  4.54s/it]
 42%|████▏     | 93727/225000 [169:24:03<212:57:58,  5.84s/it]
 42%|████▏     | 93728/225000 [169:24:06<188:07:56,  5.16s/it]
 42%|████▏     | 93729/225000 [169:24:10<168:32:31,  4.62s/it]
 42%|████▏     | 93730/225000 [169:24:13<153:08:38,  4.20s/it]
 42%|████▏     | 93731/225000 [169:24:22<203:20:32,  5.58s/it]
 42%|████▏     | 93732/225000 [169:24:25<178:49:01,  4.90s/it]
 42%|████▏     | 93733/225000 [169:24:29<171:17:18,  4.70s/it]
 42%|████▏     | 93734/225000 [169:24:36<200:35:41,  5.50s/it]
 42%|████▏     | 93735/225000 [169:24:40<180:06:16,  4.94s/it]
 42%|████▏     | 93736/225000 [169:24:44<167:29:42,  4.59s/it]
 42%|████▏     | 93737/225000 [169:24:48<161:16:16,  4.42s/it]
 42%|████▏     | 93738/225000 [169:24:56<197:06:20,  5.41s/it]
 42%|████▏     | 93739/225000 [169:25:02<209:43:16,  5.75s/it]
 42%|████▏     | 93740/225000 [169:25:06<183:53:38,  5.04s/it]
 42%|████▏     | 93741/225000 [169:25:08<160:13:49,  4.39s/it]
 42%|████▏     | 93742/225000 [169:25:18<212:02:58,  5.82s/it]
 42%|████▏     | 93743/225000 [169:25:21<188:18:40,  5.16s/it]
 42%|████▏     | 93744/225000 [169:25:24<163:51:03,  4.49s/it]
 42%|████▏     | 93745/225000 [169:25:28<152:32:43,  4.18s/it]
 42%|████▏     | 93746/225000 [169:25:31<145:54:24,  4.00s/it]
 42%|████▏     | 93747/225000 [169:25:41<214:28:06,  5.88s/it]
 42%|████▏     | 93748/225000 [169:25:44<179:19:11,  4.92s/it]
 42%|████▏     | 93749/225000 [169:25:52<207:01:06,  5.68s/it]
 42%|████▏     | 93750/225000 [169:25:55<184:07:33,  5.05s/it]
                                                              
{'loss': '0.1686', 'grad_norm': '3.586', 'learning_rate': '3.352e-05', 'epoch': '0.4167', 'num_input_tokens_seen': 277413360, 'train_runtime': '6.1e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93750/225000 [169:25:55<184:07:33,  5.05s/it]
 42%|████▏     | 93751/225000 [169:25:58<164:43:30,  4.52s/it]
 42%|████▏     | 93752/225000 [169:26:02<150:21:58,  4.12s/it]
 42%|████▏     | 93753/225000 [169:26:10<201:30:24,  5.53s/it]
 42%|████▏     | 93754/225000 [169:26:16<202:34:48,  5.56s/it]
 42%|████▏     | 93755/225000 [169:26:24<233:16:09,  6.40s/it]
 42%|████▏     | 93756/225000 [169:26:28<200:08:43,  5.49s/it]
 42%|████▏     | 93757/225000 [169:26:31<173:22:54,  4.76s/it]
 42%|████▏     | 93758/225000 [169:26:35<161:42:09,  4.44s/it]
 42%|████▏     | 93759/225000 [169:26:43<208:43:42,  5.73s/it]
 42%|████▏     | 93760/225000 [169:26:48<198:47:21,  5.45s/it]
 42%|████▏     | 93761/225000 [169:26:51<169:50:36,  4.66s/it]
 42%|████▏     | 93762/225000 [169:26:59<203:11:57,  5.57s/it]
 42%|████▏     | 93763/225000 [169:27:02<183:52:54,  5.04s/it]
 42%|████▏     | 93764/225000 [169:27:07<180:47:48,  4.96s/it]
 42%|████▏     | 93765/225000 [169:27:17<229:24:31,  6.29s/it]
 42%|████▏     | 93766/225000 [169:27:20<198:04:09,  5.43s/it]
 42%|████▏     | 93767/225000 [169:27:24<182:06:39,  5.00s/it]
 42%|████▏     | 93768/225000 [169:27:33<221:16:07,  6.07s/it]
 42%|████▏     | 93769/225000 [169:27:36<188:44:28,  5.18s/it]
 42%|████▏     | 93770/225000 [169:27:40<178:57:41,  4.91s/it]
 42%|████▏     | 93771/225000 [169:27:44<169:21:53,  4.65s/it]
 42%|████▏     | 93772/225000 [169:27:48<163:29:58,  4.49s/it]
 42%|████▏     | 93773/225000 [169:27:52<156:54:22,  4.30s/it]
 42%|████▏     | 93774/225000 [169:27:56<153:01:18,  4.20s/it]
 42%|████▏     | 93775/225000 [169:28:00<154:10:56,  4.23s/it]
 42%|████▏     | 93776/225000 [169:28:04<149:28:44,  4.10s/it]
 42%|████▏     | 93777/225000 [169:28:09<154:27:37,  4.24s/it]
 42%|████▏     | 93778/225000 [169:28:20<229:07:28,  6.29s/it]
 42%|████▏     | 93779/225000 [169:28:26<234:17:20,  6.43s/it]
 42%|████▏     | 93780/225000 [169:28:29<195:37:56,  5.37s/it]
 42%|████▏     | 93781/225000 [169:28:32<169:55:31,  4.66s/it]
 42%|████▏     | 93782/225000 [169:28:36<158:06:43,  4.34s/it]
 42%|████▏     | 93783/225000 [169:28:44<202:48:51,  5.56s/it]
 42%|████▏     | 93784/225000 [169:28:51<216:06:49,  5.93s/it]
 42%|████▏     | 93785/225000 [169:28:54<186:44:41,  5.12s/it]
 42%|████▏     | 93786/225000 [169:28:58<173:37:56,  4.76s/it]
 42%|████▏     | 93787/225000 [169:29:03<172:15:53,  4.73s/it]
 42%|████▏     | 93788/225000 [169:29:07<169:24:05,  4.65s/it]
 42%|████▏     | 93789/225000 [169:29:11<161:46:42,  4.44s/it]
 42%|████▏     | 93790/225000 [169:29:21<221:58:52,  6.09s/it]
 42%|████▏     | 93791/225000 [169:29:25<194:13:13,  5.33s/it]
 42%|████▏     | 93792/225000 [169:29:29<177:34:57,  4.87s/it]
 42%|████▏     | 93793/225000 [169:29:32<162:23:43,  4.46s/it]
 42%|████▏     | 93794/225000 [169:29:36<161:11:10,  4.42s/it]
 42%|████▏     | 93795/225000 [169:29:44<192:53:00,  5.29s/it]
 42%|████▏     | 93796/225000 [169:29:53<233:09:46,  6.40s/it]
 42%|████▏     | 93797/225000 [169:29:56<194:34:09,  5.34s/it]
 42%|████▏     | 93798/225000 [169:30:02<201:27:24,  5.53s/it]
 42%|████▏     | 93799/225000 [169:30:09<225:09:15,  6.18s/it]
 42%|████▏     | 93800/225000 [169:30:13<196:16:27,  5.39s/it]
                                                              
{'loss': '0.1534', 'grad_norm': '2.327', 'learning_rate': '3.35e-05', 'epoch': '0.4169', 'num_input_tokens_seen': 277527608, 'train_runtime': '6.102e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93800/225000 [169:30:13<196:16:27,  5.39s/it]
 42%|████▏     | 93801/225000 [169:30:16<169:19:57,  4.65s/it]
 42%|████▏     | 93802/225000 [169:30:20<166:35:02,  4.57s/it]
 42%|████▏     | 93803/225000 [169:30:24<162:06:52,  4.45s/it]
 42%|████▏     | 93804/225000 [169:30:29<161:17:57,  4.43s/it]
 42%|████▏     | 93805/225000 [169:30:36<192:56:48,  5.29s/it]
 42%|████▏     | 93806/225000 [169:30:46<239:45:54,  6.58s/it]
 42%|████▏     | 93807/225000 [169:30:49<202:52:24,  5.57s/it]
 42%|████▏     | 93808/225000 [169:30:53<192:06:38,  5.27s/it]
 42%|████▏     | 93809/225000 [169:31:02<231:42:24,  6.36s/it]
 42%|████▏     | 93810/225000 [169:31:06<200:27:41,  5.50s/it]
 42%|████▏     | 93811/225000 [169:31:09<174:36:48,  4.79s/it]
 42%|████▏     | 93812/225000 [169:31:17<207:55:22,  5.71s/it]
 42%|████▏     | 93813/225000 [169:31:21<189:35:29,  5.20s/it]
 42%|████▏     | 93814/225000 [169:31:25<179:15:06,  4.92s/it]
 42%|████▏     | 93815/225000 [169:31:29<170:54:23,  4.69s/it]
 42%|████▏     | 93816/225000 [169:31:33<166:35:19,  4.57s/it]
 42%|████▏     | 93817/225000 [169:31:37<158:19:01,  4.34s/it]
 42%|████▏     | 93818/225000 [169:31:42<157:02:43,  4.31s/it]
 42%|████▏     | 93819/225000 [169:31:45<150:28:29,  4.13s/it]
 42%|████▏     | 93820/225000 [169:31:49<148:51:56,  4.09s/it]
 42%|████▏     | 93821/225000 [169:32:00<221:34:27,  6.08s/it]
 42%|████▏     | 93822/225000 [169:32:06<217:25:18,  5.97s/it]
 42%|████▏     | 93823/225000 [169:32:09<194:11:18,  5.33s/it]
 42%|████▏     | 93824/225000 [169:32:13<178:59:57,  4.91s/it]
 42%|████▏     | 93825/225000 [169:32:17<164:48:54,  4.52s/it]
 42%|████▏     | 93826/225000 [169:32:21<154:03:20,  4.23s/it]
 42%|████▏     | 93827/225000 [169:32:25<156:55:09,  4.31s/it]
 42%|████▏     | 93828/225000 [169:32:29<150:14:52,  4.12s/it]
 42%|████▏     | 93829/225000 [169:32:33<154:27:27,  4.24s/it]
 42%|████▏     | 93830/225000 [169:32:41<195:15:44,  5.36s/it]
 42%|████▏     | 93831/225000 [169:32:46<184:17:15,  5.06s/it]
 42%|████▏     | 93832/225000 [169:32:49<171:05:24,  4.70s/it]
 42%|████▏     | 93833/225000 [169:32:53<156:42:51,  4.30s/it]
 42%|████▏     | 93834/225000 [169:32:57<152:07:11,  4.18s/it]
 42%|████▏     | 93835/225000 [169:33:01<154:54:08,  4.25s/it]
 42%|████▏     | 93836/225000 [169:33:05<155:16:45,  4.26s/it]
 42%|████▏     | 93837/225000 [169:33:11<167:41:38,  4.60s/it]
 42%|████▏     | 93838/225000 [169:33:15<166:49:31,  4.58s/it]
 42%|████▏     | 93839/225000 [169:33:26<234:09:15,  6.43s/it]
 42%|████▏     | 93840/225000 [169:33:36<267:46:07,  7.35s/it]
 42%|████▏     | 93841/225000 [169:33:40<230:22:24,  6.32s/it]
 42%|████▏     | 93842/225000 [169:33:43<198:35:19,  5.45s/it]
 42%|████▏     | 93843/225000 [169:33:47<183:11:04,  5.03s/it]
 42%|████▏     | 93844/225000 [169:33:51<176:35:21,  4.85s/it]
 42%|████▏     | 93845/225000 [169:34:00<221:09:40,  6.07s/it]
 42%|████▏     | 93846/225000 [169:34:04<192:38:23,  5.29s/it]
 42%|████▏     | 93847/225000 [169:34:07<165:46:46,  4.55s/it]
 42%|████▏     | 93848/225000 [169:34:15<211:05:49,  5.79s/it]
 42%|████▏     | 93849/225000 [169:34:19<183:39:44,  5.04s/it]
 42%|████▏     | 93850/225000 [169:34:28<230:09:40,  6.32s/it]
                                                              
{'loss': '0.1705', 'grad_norm': '2.741', 'learning_rate': '3.348e-05', 'epoch': '0.4171', 'num_input_tokens_seen': 277641768, 'train_runtime': '6.105e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93850/225000 [169:34:28<230:09:40,  6.32s/it]
 42%|████▏     | 93851/225000 [169:34:32<203:15:25,  5.58s/it]
 42%|████▏     | 93852/225000 [169:34:36<186:43:38,  5.13s/it]
 42%|████▏     | 93853/225000 [169:34:40<173:56:29,  4.77s/it]
 42%|████▏     | 93854/225000 [169:34:46<192:30:08,  5.28s/it]
 42%|████▏     | 93855/225000 [169:34:50<177:26:51,  4.87s/it]
 42%|████▏     | 93856/225000 [169:34:54<164:47:15,  4.52s/it]
 42%|████▏     | 93857/225000 [169:34:58<159:28:59,  4.38s/it]
 42%|████▏     | 93858/225000 [169:35:01<150:05:32,  4.12s/it]
 42%|████▏     | 93859/225000 [169:35:06<150:28:49,  4.13s/it]
 42%|████▏     | 93860/225000 [169:35:14<193:41:49,  5.32s/it]
 42%|████▏     | 93861/225000 [169:35:20<207:23:07,  5.69s/it]
 42%|████▏     | 93862/225000 [169:35:23<179:35:57,  4.93s/it]
 42%|████▏     | 93863/225000 [169:35:32<215:54:14,  5.93s/it]
 42%|████▏     | 93864/225000 [169:35:35<191:09:12,  5.25s/it]
 42%|████▏     | 93865/225000 [169:35:39<175:09:01,  4.81s/it]
 42%|████▏     | 93866/225000 [169:35:43<167:57:03,  4.61s/it]
 42%|████▏     | 93867/225000 [169:35:49<181:33:05,  4.98s/it]
 42%|████▏     | 93868/225000 [169:35:59<232:30:49,  6.38s/it]
 42%|████▏     | 93869/225000 [169:36:05<228:33:20,  6.27s/it]
 42%|████▏     | 93870/225000 [169:36:10<221:34:08,  6.08s/it]
 42%|████▏     | 93871/225000 [169:36:13<187:38:03,  5.15s/it]
 42%|████▏     | 93872/225000 [169:36:18<179:29:04,  4.93s/it]
 42%|████▏     | 93873/225000 [169:36:22<166:52:06,  4.58s/it]
 42%|████▏     | 93874/225000 [169:36:25<157:09:21,  4.31s/it]
 42%|████▏     | 93875/225000 [169:36:30<157:57:58,  4.34s/it]
 42%|████▏     | 93876/225000 [169:36:36<181:25:52,  4.98s/it]
 42%|████▏     | 93877/225000 [169:36:48<251:26:47,  6.90s/it]
 42%|████▏     | 93878/225000 [169:36:52<223:14:34,  6.13s/it]
 42%|████▏     | 93879/225000 [169:36:56<197:32:18,  5.42s/it]
 42%|████▏     | 93880/225000 [169:36:59<180:35:01,  4.96s/it]
 42%|████▏     | 93881/225000 [169:37:03<168:57:55,  4.64s/it]
 42%|████▏     | 93882/225000 [169:37:07<157:02:00,  4.31s/it]
 42%|████▏     | 93883/225000 [169:37:15<197:01:46,  5.41s/it]
 42%|████▏     | 93884/225000 [169:37:21<204:46:50,  5.62s/it]
 42%|████▏     | 93885/225000 [169:37:24<173:36:06,  4.77s/it]
 42%|████▏     | 93886/225000 [169:37:31<202:23:08,  5.56s/it]
 42%|████▏     | 93887/225000 [169:37:39<228:10:00,  6.26s/it]
 42%|████▏     | 93888/225000 [169:37:43<200:23:20,  5.50s/it]
 42%|████▏     | 93889/225000 [169:37:46<174:59:33,  4.80s/it]
 42%|████▏     | 93890/225000 [169:37:54<206:25:11,  5.67s/it]
 42%|████▏     | 93891/225000 [169:37:58<190:17:19,  5.22s/it]
 42%|████▏     | 93892/225000 [169:38:02<179:16:16,  4.92s/it]
 42%|████▏     | 93893/225000 [169:38:06<163:20:11,  4.48s/it]
 42%|████▏     | 93894/225000 [169:38:10<160:54:36,  4.42s/it]
 42%|████▏     | 93895/225000 [169:38:14<156:20:50,  4.29s/it]
 42%|████▏     | 93896/225000 [169:38:22<197:06:26,  5.41s/it]
 42%|████▏     | 93897/225000 [169:38:30<223:33:22,  6.14s/it]
 42%|████▏     | 93898/225000 [169:38:34<198:59:20,  5.46s/it]
 42%|████▏     | 93899/225000 [169:38:37<179:48:51,  4.94s/it]
 42%|████▏     | 93900/225000 [169:38:41<170:05:02,  4.67s/it]
                                                              
{'loss': '0.1317', 'grad_norm': '1.744', 'learning_rate': '3.347e-05', 'epoch': '0.4173', 'num_input_tokens_seen': 277758112, 'train_runtime': '6.107e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93900/225000 [169:38:41<170:05:02,  4.67s/it]
 42%|████▏     | 93901/225000 [169:38:45<154:05:41,  4.23s/it]
 42%|████▏     | 93902/225000 [169:38:55<220:53:10,  6.07s/it]
 42%|████▏     | 93903/225000 [169:39:00<205:25:11,  5.64s/it]
 42%|████▏     | 93904/225000 [169:39:03<181:56:26,  5.00s/it]
 42%|████▏     | 93905/225000 [169:39:07<167:49:55,  4.61s/it]
 42%|████▏     | 93906/225000 [169:39:16<220:42:39,  6.06s/it]
 42%|████▏     | 93907/225000 [169:39:21<201:54:13,  5.54s/it]
 42%|████▏     | 93908/225000 [169:39:24<174:04:00,  4.78s/it]
 42%|████▏     | 93909/225000 [169:39:28<172:44:08,  4.74s/it]
 42%|████▏     | 93910/225000 [169:39:33<172:59:41,  4.75s/it]
 42%|████▏     | 93911/225000 [169:39:37<162:39:11,  4.47s/it]
 42%|████▏     | 93912/225000 [169:39:42<168:01:59,  4.61s/it]
 42%|████▏     | 93913/225000 [169:39:46<161:23:47,  4.43s/it]
 42%|████▏     | 93914/225000 [169:39:50<162:19:51,  4.46s/it]
 42%|████▏     | 93915/225000 [169:39:55<168:10:34,  4.62s/it]
 42%|████▏     | 93916/225000 [169:40:04<217:41:17,  5.98s/it]
 42%|████▏     | 93917/225000 [169:40:11<226:59:09,  6.23s/it]
 42%|████▏     | 93918/225000 [169:40:14<192:11:55,  5.28s/it]
 42%|████▏     | 93919/225000 [169:40:18<175:07:26,  4.81s/it]
 42%|████▏     | 93920/225000 [169:40:22<168:08:41,  4.62s/it]
 42%|████▏     | 93921/225000 [169:40:27<166:14:39,  4.57s/it]
 42%|████▏     | 93922/225000 [169:40:36<221:26:21,  6.08s/it]
 42%|████▏     | 93923/225000 [169:40:41<212:18:53,  5.83s/it]
 42%|████▏     | 93924/225000 [169:40:45<186:45:40,  5.13s/it]
 42%|████▏     | 93925/225000 [169:40:49<171:55:26,  4.72s/it]
 42%|████▏     | 93926/225000 [169:40:53<164:04:00,  4.51s/it]
 42%|████▏     | 93927/225000 [169:40:58<167:15:54,  4.59s/it]
 42%|████▏     | 93928/225000 [169:41:04<186:02:39,  5.11s/it]
 42%|████▏     | 93929/225000 [169:41:14<245:26:39,  6.74s/it]
 42%|████▏     | 93930/225000 [169:41:17<201:19:35,  5.53s/it]
 42%|████▏     | 93931/225000 [169:41:25<224:48:03,  6.17s/it]
 42%|████▏     | 93932/225000 [169:41:28<189:04:29,  5.19s/it]
 42%|████▏     | 93933/225000 [169:41:35<211:13:52,  5.80s/it]
 42%|████▏     | 93934/225000 [169:41:38<181:28:40,  4.98s/it]
 42%|████▏     | 93935/225000 [169:41:45<202:16:13,  5.56s/it]
 42%|████▏     | 93936/225000 [169:41:50<196:34:53,  5.40s/it]
 42%|████▏     | 93937/225000 [169:41:54<183:00:06,  5.03s/it]
 42%|████▏     | 93938/225000 [169:41:59<177:15:28,  4.87s/it]
 42%|████▏     | 93939/225000 [169:42:03<170:15:48,  4.68s/it]
 42%|████▏     | 93940/225000 [169:42:07<164:59:38,  4.53s/it]
 42%|████▏     | 93941/225000 [169:42:11<160:25:06,  4.41s/it]
 42%|████▏     | 93942/225000 [169:42:16<163:08:06,  4.48s/it]
 42%|████▏     | 93943/225000 [169:42:20<163:25:15,  4.49s/it]
 42%|████▏     | 93944/225000 [169:42:29<204:52:04,  5.63s/it]
 42%|████▏     | 93945/225000 [169:42:32<181:17:19,  4.98s/it]
 42%|████▏     | 93946/225000 [169:42:37<185:44:11,  5.10s/it]
 42%|████▏     | 93947/225000 [169:42:42<180:13:57,  4.95s/it]
 42%|████▏     | 93948/225000 [169:42:46<169:40:45,  4.66s/it]
 42%|████▏     | 93949/225000 [169:42:51<172:27:53,  4.74s/it]
 42%|████▏     | 93950/225000 [169:42:56<172:07:20,  4.73s/it]
                                                              
{'loss': '0.1502', 'grad_norm': '3.142', 'learning_rate': '3.345e-05', 'epoch': '0.4176', 'num_input_tokens_seen': 277873736, 'train_runtime': '6.11e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 93950/225000 [169:42:56<172:07:20,  4.73s/it]
 42%|████▏     | 93951/225000 [169:43:01<177:00:38,  4.86s/it]
 42%|████▏     | 93952/225000 [169:43:06<178:22:31,  4.90s/it]
 42%|████▏     | 93953/225000 [169:43:15<222:59:52,  6.13s/it]
 42%|████▏     | 93954/225000 [169:43:22<236:32:07,  6.50s/it]
 42%|████▏     | 93955/225000 [169:43:29<239:58:09,  6.59s/it]
 42%|████▏     | 93956/225000 [169:43:33<210:53:34,  5.79s/it]
 42%|████▏     | 93957/225000 [169:43:37<191:12:18,  5.25s/it]
 42%|████▏     | 93958/225000 [169:43:40<173:02:41,  4.75s/it]
 42%|████▏     | 93959/225000 [169:43:44<159:46:10,  4.39s/it]
 42%|████▏     | 93960/225000 [169:43:48<155:22:36,  4.27s/it]
 42%|████▏     | 93961/225000 [169:43:52<157:08:46,  4.32s/it]
 42%|████▏     | 93962/225000 [169:44:03<221:44:22,  6.09s/it]
 42%|████▏     | 93963/225000 [169:44:07<200:49:37,  5.52s/it]
 42%|████▏     | 93964/225000 [169:44:14<213:56:35,  5.88s/it]
 42%|████▏     | 93965/225000 [169:44:17<191:08:41,  5.25s/it]
 42%|████▏     | 93966/225000 [169:44:21<175:33:59,  4.82s/it]
 42%|████▏     | 93967/225000 [169:44:29<210:04:31,  5.77s/it]
 42%|████▏     | 93968/225000 [169:44:36<221:45:57,  6.09s/it]
 42%|████▏     | 93969/225000 [169:44:39<192:13:32,  5.28s/it]
 42%|████▏     | 93970/225000 [169:44:44<186:12:33,  5.12s/it]
 42%|████▏     | 93971/225000 [169:44:48<172:10:22,  4.73s/it]
 42%|████▏     | 93972/225000 [169:44:52<165:27:25,  4.55s/it]
 42%|████▏     | 93973/225000 [169:44:57<167:55:40,  4.61s/it]
 42%|████▏     | 93974/225000 [169:45:01<161:07:52,  4.43s/it]
 42%|████▏     | 93975/225000 [169:45:06<164:51:46,  4.53s/it]
 42%|████▏     | 93976/225000 [169:45:13<201:58:58,  5.55s/it]
 42%|████▏     | 93977/225000 [169:45:22<231:47:44,  6.37s/it]
 42%|████▏     | 93978/225000 [169:45:25<201:54:32,  5.55s/it]
 42%|████▏     | 93979/225000 [169:45:34<231:26:56,  6.36s/it]
 42%|████▏     | 93980/225000 [169:45:39<216:25:36,  5.95s/it]
 42%|████▏     | 93981/225000 [169:45:42<191:00:53,  5.25s/it]
 42%|████▏     | 93982/225000 [169:45:46<170:20:46,  4.68s/it]
 42%|████▏     | 93983/225000 [169:45:49<160:18:05,  4.40s/it]
 42%|████▏     | 93984/225000 [169:45:53<154:50:27,  4.25s/it]
 42%|████▏     | 93985/225000 [169:45:58<156:52:22,  4.31s/it]
 42%|████▏     | 93986/225000 [169:46:08<219:37:51,  6.04s/it]
 42%|████▏     | 93987/225000 [169:46:12<198:15:52,  5.45s/it]
 42%|████▏     | 93988/225000 [169:46:15<174:43:17,  4.80s/it]
 42%|████▏     | 93989/225000 [169:46:20<175:07:39,  4.81s/it]
 42%|████▏     | 93990/225000 [169:46:24<168:00:30,  4.62s/it]
 42%|████▏     | 93991/225000 [169:46:28<164:44:30,  4.53s/it]
 42%|████▏     | 93992/225000 [169:46:33<169:23:53,  4.65s/it]
 42%|████▏     | 93993/225000 [169:46:43<223:21:20,  6.14s/it]
 42%|████▏     | 93994/225000 [169:46:50<234:40:36,  6.45s/it]
 42%|████▏     | 93995/225000 [169:46:54<201:15:38,  5.53s/it]
 42%|████▏     | 93996/225000 [169:46:57<175:19:07,  4.82s/it]
 42%|████▏     | 93997/225000 [169:47:03<192:07:44,  5.28s/it]
 42%|████▏     | 93998/225000 [169:47:09<198:53:23,  5.47s/it]
 42%|████▏     | 93999/225000 [169:47:14<189:00:12,  5.19s/it]
 42%|████▏     | 94000/225000 [169:47:18<184:08:28,  5.06s/it]
                                                              
{'loss': '0.1356', 'grad_norm': '3.553', 'learning_rate': '3.343e-05', 'epoch': '0.4178', 'num_input_tokens_seen': 277990328, 'train_runtime': '6.112e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94000/225000 [169:47:18<184:08:28,  5.06s/it]
 42%|████▏     | 94001/225000 [169:47:23<181:28:43,  4.99s/it]
 42%|████▏     | 94002/225000 [169:47:27<170:23:11,  4.68s/it]
 42%|████▏     | 94003/225000 [169:47:37<226:32:33,  6.23s/it]
 42%|████▏     | 94004/225000 [169:47:44<238:10:46,  6.55s/it]
 42%|████▏     | 94005/225000 [169:47:48<213:21:57,  5.86s/it]
 42%|████▏     | 94006/225000 [169:47:52<192:26:42,  5.29s/it]
 42%|████▏     | 94007/225000 [169:47:56<176:44:02,  4.86s/it]
 42%|████▏     | 94008/225000 [169:48:01<170:28:33,  4.69s/it]
 42%|████▏     | 94009/225000 [169:48:05<171:45:52,  4.72s/it]
 42%|████▏     | 94010/225000 [169:48:15<225:07:27,  6.19s/it]
 42%|████▏     | 94011/225000 [169:48:20<208:37:37,  5.73s/it]
 42%|████▏     | 94012/225000 [169:48:23<184:34:46,  5.07s/it]
 42%|████▏     | 94013/225000 [169:48:29<188:37:33,  5.18s/it]
 42%|████▏     | 94014/225000 [169:48:37<223:56:04,  6.15s/it]
 42%|████▏     | 94015/225000 [169:48:41<197:19:14,  5.42s/it]
 42%|████▏     | 94016/225000 [169:48:44<175:11:37,  4.82s/it]
 42%|████▏     | 94017/225000 [169:48:48<166:07:50,  4.57s/it]
 42%|████▏     | 94018/225000 [169:48:55<193:56:58,  5.33s/it]
 42%|████▏     | 94019/225000 [169:49:03<217:54:23,  5.99s/it]
 42%|████▏     | 94020/225000 [169:49:09<223:48:56,  6.15s/it]
 42%|████▏     | 94021/225000 [169:49:12<188:34:24,  5.18s/it]
 42%|████▏     | 94022/225000 [169:49:20<212:53:36,  5.85s/it]
 42%|████▏     | 94023/225000 [169:49:23<190:19:58,  5.23s/it]
 42%|████▏     | 94024/225000 [169:49:28<188:00:40,  5.17s/it]
 42%|████▏     | 94025/225000 [169:49:36<212:02:20,  5.83s/it]
 42%|████▏     | 94026/225000 [169:49:39<187:00:23,  5.14s/it]
 42%|████▏     | 94027/225000 [169:49:43<171:08:25,  4.70s/it]
 42%|████▏     | 94028/225000 [169:49:51<204:40:52,  5.63s/it]
 42%|████▏     | 94029/225000 [169:49:58<221:30:54,  6.09s/it]
 42%|████▏     | 94030/225000 [169:50:02<201:03:55,  5.53s/it]
 42%|████▏     | 94031/225000 [169:50:06<183:34:04,  5.05s/it]
 42%|████▏     | 94032/225000 [169:50:10<175:44:34,  4.83s/it]
 42%|████▏     | 94033/225000 [169:50:15<171:20:31,  4.71s/it]
 42%|████▏     | 94034/225000 [169:50:20<173:40:40,  4.77s/it]
 42%|████▏     | 94035/225000 [169:50:24<162:58:35,  4.48s/it]
 42%|████▏     | 94036/225000 [169:50:30<179:55:33,  4.95s/it]
 42%|████▏     | 94037/225000 [169:50:40<237:27:23,  6.53s/it]
 42%|████▏     | 94038/225000 [169:50:43<204:47:45,  5.63s/it]
 42%|████▏     | 94039/225000 [169:50:47<178:06:40,  4.90s/it]
 42%|████▏     | 94040/225000 [169:50:58<250:28:22,  6.89s/it]
 42%|████▏     | 94041/225000 [169:51:02<214:06:26,  5.89s/it]
 42%|████▏     | 94042/225000 [169:51:05<191:07:06,  5.25s/it]
 42%|████▏     | 94043/225000 [169:51:14<232:54:33,  6.40s/it]
 42%|████▏     | 94044/225000 [169:51:18<199:56:05,  5.50s/it]
 42%|████▏     | 94045/225000 [169:51:22<184:06:52,  5.06s/it]
 42%|████▏     | 94046/225000 [169:51:26<177:44:07,  4.89s/it]
 42%|████▏     | 94047/225000 [169:51:32<183:05:57,  5.03s/it]
 42%|████▏     | 94048/225000 [169:51:41<230:56:00,  6.35s/it]
 42%|████▏     | 94049/225000 [169:51:45<205:48:24,  5.66s/it]
 42%|████▏     | 94050/225000 [169:51:49<187:41:57,  5.16s/it]
                                                              
{'loss': '0.1365', 'grad_norm': '6.921', 'learning_rate': '3.341e-05', 'epoch': '0.418', 'num_input_tokens_seen': 278111328, 'train_runtime': '6.115e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94050/225000 [169:51:49<187:41:57,  5.16s/it]
 42%|████▏     | 94051/225000 [169:51:53<173:03:00,  4.76s/it]
 42%|████▏     | 94052/225000 [169:51:57<168:09:40,  4.62s/it]
 42%|████▏     | 94053/225000 [169:52:02<165:41:14,  4.56s/it]
 42%|████▏     | 94054/225000 [169:52:09<196:45:01,  5.41s/it]
 42%|████▏     | 94055/225000 [169:52:18<236:03:39,  6.49s/it]
 42%|████▏     | 94056/225000 [169:52:21<199:42:56,  5.49s/it]
 42%|████▏     | 94057/225000 [169:52:25<182:29:13,  5.02s/it]
 42%|████▏     | 94058/225000 [169:52:29<165:10:49,  4.54s/it]
 42%|████▏     | 94059/225000 [169:52:38<218:26:28,  6.01s/it]
 42%|████▏     | 94060/225000 [169:52:41<181:24:45,  4.99s/it]
 42%|████▏     | 94061/225000 [169:52:47<199:13:25,  5.48s/it]
 42%|████▏     | 94062/225000 [169:52:53<196:51:41,  5.41s/it]
 42%|████▏     | 94063/225000 [169:52:57<184:25:43,  5.07s/it]
 42%|████▏     | 94064/225000 [169:53:02<181:57:11,  5.00s/it]
 42%|████▏     | 94065/225000 [169:53:06<179:01:47,  4.92s/it]
 42%|████▏     | 94066/225000 [169:53:11<178:49:34,  4.92s/it]
 42%|████▏     | 94067/225000 [169:53:15<168:54:06,  4.64s/it]
 42%|████▏     | 94068/225000 [169:53:20<164:52:22,  4.53s/it]
 42%|████▏     | 94069/225000 [169:53:24<163:04:53,  4.48s/it]
 42%|████▏     | 94070/225000 [169:53:29<173:58:49,  4.78s/it]
 42%|████▏     | 94071/225000 [169:53:34<170:41:20,  4.69s/it]
 42%|████▏     | 94072/225000 [169:53:38<159:17:59,  4.38s/it]
 42%|████▏     | 94073/225000 [169:53:43<164:59:46,  4.54s/it]
 42%|████▏     | 94074/225000 [169:53:51<204:16:30,  5.62s/it]
 42%|████▏     | 94075/225000 [169:53:56<201:03:44,  5.53s/it]
 42%|████▏     | 94076/225000 [169:54:00<184:20:00,  5.07s/it]
 42%|████▏     | 94077/225000 [169:54:04<175:34:48,  4.83s/it]
 42%|████▏     | 94078/225000 [169:54:09<172:06:50,  4.73s/it]
 42%|████▏     | 94079/225000 [169:54:13<171:24:51,  4.71s/it]
 42%|████▏     | 94080/225000 [169:54:21<198:31:41,  5.46s/it]
 42%|████▏     | 94081/225000 [169:54:28<220:56:24,  6.08s/it]
 42%|████▏     | 94082/225000 [169:54:33<212:40:03,  5.85s/it]
 42%|████▏     | 94083/225000 [169:54:40<216:05:47,  5.94s/it]
 42%|████▏     | 94084/225000 [169:54:44<201:12:50,  5.53s/it]
 42%|████▏     | 94085/225000 [169:54:50<202:55:08,  5.58s/it]
 42%|████▏     | 94086/225000 [169:54:55<199:35:01,  5.49s/it]
 42%|████▏     | 94087/225000 [169:54:59<184:27:12,  5.07s/it]
 42%|████▏     | 94088/225000 [169:55:03<169:01:42,  4.65s/it]
 42%|████▏     | 94089/225000 [169:55:07<160:00:03,  4.40s/it]
 42%|████▏     | 94090/225000 [169:55:11<163:41:07,  4.50s/it]
 42%|████▏     | 94091/225000 [169:55:22<231:05:48,  6.36s/it]
 42%|████▏     | 94092/225000 [169:55:26<203:09:45,  5.59s/it]
 42%|████▏     | 94093/225000 [169:55:30<186:19:03,  5.12s/it]
 42%|████▏     | 94094/225000 [169:55:34<169:23:05,  4.66s/it]
 42%|████▏     | 94095/225000 [169:55:43<224:23:14,  6.17s/it]
 42%|████▏     | 94096/225000 [169:55:47<192:53:26,  5.30s/it]
 42%|████▏     | 94097/225000 [169:55:50<173:01:39,  4.76s/it]
 42%|████▏     | 94098/225000 [169:55:54<161:09:36,  4.43s/it]
 42%|████▏     | 94099/225000 [169:55:57<150:51:41,  4.15s/it]
 42%|████▏     | 94100/225000 [169:56:02<153:30:42,  4.22s/it]
                                                              
{'loss': '0.145', 'grad_norm': '1.291', 'learning_rate': '3.34e-05', 'epoch': '0.4182', 'num_input_tokens_seen': 278228168, 'train_runtime': '6.118e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94100/225000 [169:56:02<153:30:42,  4.22s/it]
 42%|████▏     | 94101/225000 [169:56:09<187:09:42,  5.15s/it]
 42%|████▏     | 94102/225000 [169:56:19<240:08:43,  6.60s/it]
 42%|████▏     | 94103/225000 [169:56:26<240:37:55,  6.62s/it]
 42%|████▏     | 94104/225000 [169:56:31<224:58:39,  6.19s/it]
 42%|████▏     | 94105/225000 [169:56:34<196:33:32,  5.41s/it]
 42%|████▏     | 94106/225000 [169:56:37<171:52:41,  4.73s/it]
 42%|████▏     | 94107/225000 [169:56:45<205:08:51,  5.64s/it]
 42%|████▏     | 94108/225000 [169:56:49<184:23:09,  5.07s/it]
 42%|████▏     | 94109/225000 [169:56:54<181:42:50,  5.00s/it]
 42%|████▏     | 94110/225000 [169:56:59<180:48:28,  4.97s/it]
 42%|████▏     | 94111/225000 [169:57:08<223:11:00,  6.14s/it]
 42%|████▏     | 94112/225000 [169:57:13<214:20:14,  5.90s/it]
 42%|████▏     | 94113/225000 [169:57:16<184:59:50,  5.09s/it]
 42%|████▏     | 94114/225000 [169:57:20<177:14:25,  4.87s/it]
 42%|████▏     | 94115/225000 [169:57:29<213:26:34,  5.87s/it]
 42%|████▏     | 94116/225000 [169:57:33<192:18:54,  5.29s/it]
 42%|████▏     | 94117/225000 [169:57:39<204:46:18,  5.63s/it]
 42%|████▏     | 94118/225000 [169:57:42<177:06:33,  4.87s/it]
 42%|████▏     | 94119/225000 [169:57:46<161:14:50,  4.44s/it]
 42%|████▏     | 94120/225000 [169:57:55<210:27:41,  5.79s/it]
 42%|████▏     | 94121/225000 [169:57:58<183:02:56,  5.04s/it]
 42%|████▏     | 94122/225000 [169:58:04<191:42:25,  5.27s/it]
 42%|████▏     | 94123/225000 [169:58:07<168:12:49,  4.63s/it]
 42%|████▏     | 94124/225000 [169:58:10<148:02:23,  4.07s/it]
 42%|████▏     | 94125/225000 [169:58:13<141:01:45,  3.88s/it]
 42%|████▏     | 94126/225000 [169:58:24<219:06:33,  6.03s/it]
 42%|████▏     | 94127/225000 [169:58:27<185:48:50,  5.11s/it]
 42%|████▏     | 94128/225000 [169:58:31<171:14:59,  4.71s/it]
 42%|████▏     | 94129/225000 [169:58:40<217:13:21,  5.98s/it]
 42%|████▏     | 94130/225000 [169:58:43<188:28:29,  5.18s/it]
 42%|████▏     | 94131/225000 [169:58:46<168:17:45,  4.63s/it]
 42%|████▏     | 94132/225000 [169:58:50<157:52:12,  4.34s/it]
 42%|████▏     | 94133/225000 [169:58:54<159:29:22,  4.39s/it]
 42%|████▏     | 94134/225000 [169:59:00<171:32:01,  4.72s/it]
 42%|████▏     | 94135/225000 [169:59:11<243:42:00,  6.70s/it]
 42%|████▏     | 94136/225000 [169:59:15<212:03:53,  5.83s/it]
 42%|████▏     | 94137/225000 [169:59:22<223:35:01,  6.15s/it]
 42%|████▏     | 94138/225000 [169:59:25<190:34:54,  5.24s/it]
 42%|████▏     | 94139/225000 [169:59:33<221:21:44,  6.09s/it]
 42%|████▏     | 94140/225000 [169:59:36<184:05:48,  5.06s/it]
 42%|████▏     | 94141/225000 [169:59:40<169:06:12,  4.65s/it]
 42%|████▏     | 94142/225000 [169:59:47<201:41:23,  5.55s/it]
 42%|████▏     | 94143/225000 [169:59:50<174:02:02,  4.79s/it]
 42%|████▏     | 94144/225000 [169:59:54<160:35:58,  4.42s/it]
 42%|████▏     | 94145/225000 [170:00:02<199:02:25,  5.48s/it]
 42%|████▏     | 94146/225000 [170:00:06<186:01:29,  5.12s/it]
 42%|████▏     | 94147/225000 [170:00:13<203:34:43,  5.60s/it]
 42%|████▏     | 94148/225000 [170:00:15<170:28:00,  4.69s/it]
 42%|████▏     | 94149/225000 [170:00:21<178:28:25,  4.91s/it]
 42%|████▏     | 94150/225000 [170:00:26<184:33:56,  5.08s/it]
                                                              
{'loss': '0.1572', 'grad_norm': '1.279', 'learning_rate': '3.338e-05', 'epoch': '0.4184', 'num_input_tokens_seen': 278343872, 'train_runtime': '6.12e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94150/225000 [170:00:26<184:33:56,  5.08s/it]
 42%|████▏     | 94151/225000 [170:00:30<166:04:15,  4.57s/it]
 42%|████▏     | 94152/225000 [170:00:34<160:36:06,  4.42s/it]
 42%|████▏     | 94153/225000 [170:00:45<235:28:17,  6.48s/it]
 42%|████▏     | 94154/225000 [170:00:48<193:54:09,  5.33s/it]
 42%|████▏     | 94155/225000 [170:00:53<199:33:08,  5.49s/it]
 42%|████▏     | 94156/225000 [170:00:58<189:08:49,  5.20s/it]
 42%|████▏     | 94157/225000 [170:01:02<177:56:19,  4.90s/it]
 42%|████▏     | 94158/225000 [170:01:06<169:31:48,  4.66s/it]
 42%|████▏     | 94159/225000 [170:01:11<165:33:42,  4.56s/it]
 42%|████▏     | 94160/225000 [170:01:14<157:48:30,  4.34s/it]
 42%|████▏     | 94161/225000 [170:01:18<151:49:03,  4.18s/it]
 42%|████▏     | 94162/225000 [170:01:23<163:44:35,  4.51s/it]
 42%|████▏     | 94163/225000 [170:01:37<258:25:02,  7.11s/it]
 42%|████▏     | 94164/225000 [170:01:40<219:30:37,  6.04s/it]
 42%|████▏     | 94165/225000 [170:01:43<187:45:08,  5.17s/it]
 42%|████▏     | 94166/225000 [170:01:46<164:53:15,  4.54s/it]
 42%|████▏     | 94167/225000 [170:01:57<225:37:25,  6.21s/it]
 42%|████▏     | 94168/225000 [170:02:00<196:15:49,  5.40s/it]
 42%|████▏     | 94169/225000 [170:02:04<184:01:04,  5.06s/it]
 42%|████▏     | 94170/225000 [170:02:08<163:49:57,  4.51s/it]
 42%|████▏     | 94171/225000 [170:02:11<156:14:39,  4.30s/it]
 42%|████▏     | 94172/225000 [170:02:15<154:10:32,  4.24s/it]
 42%|████▏     | 94173/225000 [170:02:24<198:26:15,  5.46s/it]
 42%|████▏     | 94174/225000 [170:02:33<241:25:36,  6.64s/it]
 42%|████▏     | 94175/225000 [170:02:38<218:22:24,  6.01s/it]
 42%|████▏     | 94176/225000 [170:02:41<188:23:07,  5.18s/it]
 42%|████▏     | 94177/225000 [170:02:45<175:48:07,  4.84s/it]
 42%|████▏     | 94178/225000 [170:02:49<171:45:31,  4.73s/it]
 42%|████▏     | 94179/225000 [170:02:53<159:34:50,  4.39s/it]
 42%|████▏     | 94180/225000 [170:03:02<205:49:42,  5.66s/it]
 42%|████▏     | 94181/225000 [170:03:09<218:48:06,  6.02s/it]
 42%|████▏     | 94182/225000 [170:03:12<187:47:17,  5.17s/it]
 42%|████▏     | 94183/225000 [170:03:19<210:50:36,  5.80s/it]
 42%|████▏     | 94184/225000 [170:03:25<209:30:25,  5.77s/it]
 42%|████▏     | 94185/225000 [170:03:28<184:04:49,  5.07s/it]
 42%|████▏     | 94186/225000 [170:03:32<169:08:04,  4.65s/it]
 42%|████▏     | 94187/225000 [170:03:36<160:03:15,  4.40s/it]
 42%|████▏     | 94188/225000 [170:03:40<163:24:11,  4.50s/it]
 42%|████▏     | 94189/225000 [170:03:44<159:00:49,  4.38s/it]
 42%|████▏     | 94190/225000 [170:03:49<160:21:50,  4.41s/it]
 42%|████▏     | 94191/225000 [170:03:59<217:28:52,  5.99s/it]
 42%|████▏     | 94192/225000 [170:04:05<225:24:11,  6.20s/it]
 42%|████▏     | 94193/225000 [170:04:10<203:45:52,  5.61s/it]
 42%|████▏     | 94194/225000 [170:04:14<196:00:40,  5.39s/it]
 42%|████▏     | 94195/225000 [170:04:21<208:56:09,  5.75s/it]
 42%|████▏     | 94196/225000 [170:04:25<185:18:20,  5.10s/it]
 42%|████▏     | 94197/225000 [170:04:28<163:29:12,  4.50s/it]
 42%|████▏     | 94198/225000 [170:04:32<158:05:29,  4.35s/it]
 42%|████▏     | 94199/225000 [170:04:39<189:44:25,  5.22s/it]
 42%|████▏     | 94200/225000 [170:04:43<176:58:02,  4.87s/it]
                                                              
{'loss': '0.1341', 'grad_norm': '1.529', 'learning_rate': '3.336e-05', 'epoch': '0.4187', 'num_input_tokens_seen': 278460416, 'train_runtime': '6.123e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94200/225000 [170:04:43<176:58:02,  4.87s/it]
 42%|████▏     | 94201/225000 [170:04:47<172:19:37,  4.74s/it]
 42%|████▏     | 94202/225000 [170:04:52<168:40:48,  4.64s/it]
 42%|████▏     | 94203/225000 [170:04:56<160:58:46,  4.43s/it]
 42%|████▏     | 94204/225000 [170:05:00<157:28:20,  4.33s/it]
 42%|████▏     | 94205/225000 [170:05:11<232:22:52,  6.40s/it]
 42%|████▏     | 94206/225000 [170:05:15<210:19:39,  5.79s/it]
 42%|████▏     | 94207/225000 [170:05:20<193:00:56,  5.31s/it]
 42%|████▏     | 94208/225000 [170:05:25<190:26:25,  5.24s/it]
 42%|████▏     | 94209/225000 [170:05:30<192:32:49,  5.30s/it]
 42%|████▏     | 94210/225000 [170:05:34<180:06:05,  4.96s/it]
 42%|████▏     | 94211/225000 [170:05:39<174:00:26,  4.79s/it]
 42%|████▏     | 94212/225000 [170:05:43<166:05:46,  4.57s/it]
 42%|████▏     | 94213/225000 [170:05:47<162:41:54,  4.48s/it]
 42%|████▏     | 94214/225000 [170:05:52<162:26:37,  4.47s/it]
 42%|████▏     | 94215/225000 [170:05:56<163:22:49,  4.50s/it]
 42%|████▏     | 94216/225000 [170:06:00<159:05:28,  4.38s/it]
 42%|████▏     | 94217/225000 [170:06:12<236:22:34,  6.51s/it]
 42%|████▏     | 94218/225000 [170:06:20<257:31:52,  7.09s/it]
 42%|████▏     | 94219/225000 [170:06:24<224:10:28,  6.17s/it]
 42%|████▏     | 94220/225000 [170:06:32<237:45:54,  6.54s/it]
 42%|████▏     | 94221/225000 [170:06:35<207:56:53,  5.72s/it]
 42%|████▏     | 94222/225000 [170:06:39<183:08:22,  5.04s/it]
 42%|████▏     | 94223/225000 [170:06:43<169:13:04,  4.66s/it]
 42%|████▏     | 94224/225000 [170:06:47<164:04:30,  4.52s/it]
 42%|████▏     | 94225/225000 [170:06:58<235:21:26,  6.48s/it]
 42%|████▏     | 94226/225000 [170:07:01<198:48:05,  5.47s/it]
 42%|████▏     | 94227/225000 [170:07:07<206:35:31,  5.69s/it]
 42%|████▏     | 94228/225000 [170:07:12<195:50:24,  5.39s/it]
 42%|████▏     | 94229/225000 [170:07:16<181:30:00,  5.00s/it]
 42%|████▏     | 94230/225000 [170:07:20<173:02:33,  4.76s/it]
 42%|████▏     | 94231/225000 [170:07:24<166:59:41,  4.60s/it]
 42%|████▏     | 94232/225000 [170:07:29<166:15:08,  4.58s/it]
 42%|████▏     | 94233/225000 [170:07:35<178:19:46,  4.91s/it]
 42%|████▏     | 94234/225000 [170:07:44<229:37:48,  6.32s/it]
 42%|████▏     | 94235/225000 [170:07:49<212:55:50,  5.86s/it]
 42%|████▏     | 94236/225000 [170:07:53<195:03:02,  5.37s/it]
 42%|████▏     | 94237/225000 [170:07:57<177:10:46,  4.88s/it]
 42%|████▏     | 94238/225000 [170:08:06<219:59:18,  6.06s/it]
 42%|████▏     | 94239/225000 [170:08:12<219:41:36,  6.05s/it]
 42%|████▏     | 94240/225000 [170:08:18<220:34:52,  6.07s/it]
 42%|████▏     | 94241/225000 [170:08:24<218:43:14,  6.02s/it]
 42%|████▏     | 94242/225000 [170:08:28<197:53:18,  5.45s/it]
 42%|████▏     | 94243/225000 [170:08:31<177:21:52,  4.88s/it]
 42%|████▏     | 94244/225000 [170:08:35<161:29:23,  4.45s/it]
 42%|████▏     | 94245/225000 [170:08:40<172:11:14,  4.74s/it]
 42%|████▏     | 94246/225000 [170:08:50<223:36:44,  6.16s/it]
 42%|████▏     | 94247/225000 [170:08:53<194:44:34,  5.36s/it]
 42%|████▏     | 94248/225000 [170:08:57<178:37:55,  4.92s/it]
 42%|████▏     | 94249/225000 [170:09:06<223:03:14,  6.14s/it]
 42%|████▏     | 94250/225000 [170:09:10<194:42:16,  5.36s/it]
                                                              
{'loss': '0.1459', 'grad_norm': '5.609', 'learning_rate': '3.335e-05', 'epoch': '0.4189', 'num_input_tokens_seen': 278579864, 'train_runtime': '6.126e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94250/225000 [170:09:10<194:42:16,  5.36s/it]
 42%|████▏     | 94251/225000 [170:09:17<215:08:10,  5.92s/it]
 42%|████▏     | 94252/225000 [170:09:22<204:46:00,  5.64s/it]
 42%|████▏     | 94253/225000 [170:09:26<182:52:00,  5.04s/it]
 42%|████▏     | 94254/225000 [170:09:33<209:50:23,  5.78s/it]
 42%|████▏     | 94255/225000 [170:09:38<196:02:45,  5.40s/it]
 42%|████▏     | 94256/225000 [170:09:41<177:45:48,  4.89s/it]
 42%|████▏     | 94257/225000 [170:09:49<206:51:59,  5.70s/it]
 42%|████▏     | 94258/225000 [170:09:52<180:10:24,  4.96s/it]
 42%|████▏     | 94259/225000 [170:09:55<162:38:18,  4.48s/it]
 42%|████▏     | 94260/225000 [170:10:04<207:15:16,  5.71s/it]
 42%|████▏     | 94261/225000 [170:10:08<188:24:54,  5.19s/it]
 42%|████▏     | 94262/225000 [170:10:14<192:56:53,  5.31s/it]
 42%|████▏     | 94263/225000 [170:10:19<196:54:35,  5.42s/it]
 42%|████▏     | 94264/225000 [170:10:22<172:54:19,  4.76s/it]
 42%|████▏     | 94265/225000 [170:10:26<157:31:27,  4.34s/it]
 42%|████▏     | 94266/225000 [170:10:35<207:43:41,  5.72s/it]
 42%|████▏     | 94267/225000 [170:10:41<209:29:12,  5.77s/it]
 42%|████▏     | 94268/225000 [170:10:44<183:48:40,  5.06s/it]
 42%|████▏     | 94269/225000 [170:10:49<181:11:40,  4.99s/it]
 42%|████▏     | 94270/225000 [170:10:56<206:48:00,  5.69s/it]
 42%|████▏     | 94271/225000 [170:11:02<203:10:31,  5.60s/it]
 42%|████▏     | 94272/225000 [170:11:05<177:38:27,  4.89s/it]
 42%|████▏     | 94273/225000 [170:11:08<159:35:44,  4.39s/it]
 42%|████▏     | 94274/225000 [170:11:12<155:20:08,  4.28s/it]
 42%|████▏     | 94275/225000 [170:11:20<197:44:39,  5.45s/it]
 42%|████▏     | 94276/225000 [170:11:28<217:30:25,  5.99s/it]
 42%|████▏     | 94277/225000 [170:11:31<187:30:44,  5.16s/it]
 42%|████▏     | 94278/225000 [170:11:37<204:28:42,  5.63s/it]
 42%|████▏     | 94279/225000 [170:11:40<171:47:34,  4.73s/it]
 42%|████▏     | 94280/225000 [170:11:48<203:15:21,  5.60s/it]
 42%|████▏     | 94281/225000 [170:11:53<196:43:02,  5.42s/it]
 42%|████▏     | 94282/225000 [170:11:59<203:05:15,  5.59s/it]
 42%|████▏     | 94283/225000 [170:12:02<177:37:45,  4.89s/it]
 42%|████▏     | 94284/225000 [170:12:05<160:41:57,  4.43s/it]
 42%|████▏     | 94285/225000 [170:12:08<144:46:52,  3.99s/it]
 42%|████▏     | 94286/225000 [170:12:16<181:07:40,  4.99s/it]
 42%|████▏     | 94287/225000 [170:12:23<209:55:30,  5.78s/it]
 42%|████▏     | 94288/225000 [170:12:28<197:43:34,  5.45s/it]
 42%|████▏     | 94289/225000 [170:12:32<184:49:48,  5.09s/it]
 42%|████▏     | 94290/225000 [170:12:36<171:38:10,  4.73s/it]
 42%|████▏     | 94291/225000 [170:12:40<168:21:16,  4.64s/it]
 42%|████▏     | 94292/225000 [170:12:45<168:57:51,  4.65s/it]
 42%|████▏     | 94293/225000 [170:12:54<219:54:32,  6.06s/it]
 42%|████▏     | 94294/225000 [170:12:59<201:12:58,  5.54s/it]
 42%|████▏     | 94295/225000 [170:13:04<201:41:17,  5.56s/it]
 42%|████▏     | 94296/225000 [170:13:11<208:04:33,  5.73s/it]
 42%|████▏     | 94297/225000 [170:13:15<190:38:21,  5.25s/it]
 42%|████▏     | 94298/225000 [170:13:19<177:35:35,  4.89s/it]
 42%|████▏     | 94299/225000 [170:13:23<168:19:08,  4.64s/it]
 42%|████▏     | 94300/225000 [170:13:27<159:02:49,  4.38s/it]
                                                              
{'loss': '0.141', 'grad_norm': '4.302', 'learning_rate': '3.333e-05', 'epoch': '0.4191', 'num_input_tokens_seen': 278697224, 'train_runtime': '6.128e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94300/225000 [170:13:27<159:02:49,  4.38s/it]
 42%|████▏     | 94301/225000 [170:13:30<152:37:07,  4.20s/it]
 42%|████▏     | 94302/225000 [170:13:36<163:09:17,  4.49s/it]
 42%|████▏     | 94303/225000 [170:13:46<230:10:32,  6.34s/it]
 42%|████▏     | 94304/225000 [170:13:50<201:09:36,  5.54s/it]
 42%|████▏     | 94305/225000 [170:13:54<184:03:10,  5.07s/it]
 42%|████▏     | 94306/225000 [170:13:58<169:07:17,  4.66s/it]
 42%|████▏     | 94307/225000 [170:14:03<176:06:00,  4.85s/it]
 42%|████▏     | 94308/225000 [170:14:13<234:10:55,  6.45s/it]
 42%|████▏     | 94309/225000 [170:14:16<197:33:37,  5.44s/it]
 42%|████▏     | 94310/225000 [170:14:21<193:34:11,  5.33s/it]
 42%|████▏     | 94311/225000 [170:14:29<217:47:06,  6.00s/it]
 42%|████▏     | 94312/225000 [170:14:33<196:12:24,  5.40s/it]
 42%|████▏     | 94313/225000 [170:14:37<182:28:15,  5.03s/it]
 42%|████▏     | 94314/225000 [170:14:41<174:05:54,  4.80s/it]
 42%|████▏     | 94315/225000 [170:14:45<158:34:25,  4.37s/it]
 42%|████▏     | 94316/225000 [170:14:49<163:26:54,  4.50s/it]
 42%|████▏     | 94317/225000 [170:15:00<227:53:33,  6.28s/it]
 42%|████▏     | 94318/225000 [170:15:03<199:12:34,  5.49s/it]
 42%|████▏     | 94319/225000 [170:15:07<183:31:46,  5.06s/it]
 42%|████▏     | 94320/225000 [170:15:11<169:16:21,  4.66s/it]
 42%|████▏     | 94321/225000 [170:15:15<162:38:29,  4.48s/it]
 42%|████▏     | 94322/225000 [170:15:26<232:30:02,  6.41s/it]
 42%|████▏     | 94323/225000 [170:15:31<216:45:09,  5.97s/it]
 42%|████▏     | 94324/225000 [170:15:35<193:22:43,  5.33s/it]
 42%|████▏     | 94325/225000 [170:15:39<179:35:21,  4.95s/it]
 42%|████▏     | 94326/225000 [170:15:42<163:51:14,  4.51s/it]
 42%|████▏     | 94327/225000 [170:15:46<153:34:13,  4.23s/it]
 42%|████▏     | 94328/225000 [170:15:50<151:34:43,  4.18s/it]
 42%|████▏     | 94329/225000 [170:16:01<228:13:55,  6.29s/it]
 42%|████▏     | 94330/225000 [170:16:05<204:35:52,  5.64s/it]
 42%|████▏     | 94331/225000 [170:16:10<189:14:10,  5.21s/it]
 42%|████▏     | 94332/225000 [170:16:14<180:24:56,  4.97s/it]
 42%|████▏     | 94333/225000 [170:16:19<180:10:39,  4.96s/it]
 42%|████▏     | 94334/225000 [170:16:24<177:47:42,  4.90s/it]
 42%|████▏     | 94335/225000 [170:16:28<169:17:35,  4.66s/it]
 42%|████▏     | 94336/225000 [170:16:37<219:41:34,  6.05s/it]
 42%|████▏     | 94337/225000 [170:16:45<238:18:57,  6.57s/it]
 42%|████▏     | 94338/225000 [170:16:48<204:05:12,  5.62s/it]
 42%|████▏     | 94339/225000 [170:16:52<183:28:34,  5.06s/it]
 42%|████▏     | 94340/225000 [170:16:59<208:15:45,  5.74s/it]
 42%|████▏     | 94341/225000 [170:17:07<225:47:39,  6.22s/it]
 42%|████▏     | 94342/225000 [170:17:10<192:01:13,  5.29s/it]
 42%|████▏     | 94343/225000 [170:17:19<236:36:15,  6.52s/it]
 42%|████▏     | 94344/225000 [170:17:22<196:50:54,  5.42s/it]
 42%|████▏     | 94345/225000 [170:17:27<191:27:09,  5.28s/it]
 42%|████▏     | 94346/225000 [170:17:34<209:55:44,  5.78s/it]
 42%|████▏     | 94347/225000 [170:17:38<194:26:35,  5.36s/it]
 42%|████▏     | 94348/225000 [170:17:43<186:23:40,  5.14s/it]
 42%|████▏     | 94349/225000 [170:17:48<183:57:17,  5.07s/it]
 42%|████▏     | 94350/225000 [170:17:52<174:30:52,  4.81s/it]
                                                              
{'loss': '0.1497', 'grad_norm': '4.529', 'learning_rate': '3.331e-05', 'epoch': '0.4193', 'num_input_tokens_seen': 278816112, 'train_runtime': '6.131e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94350/225000 [170:17:52<174:30:52,  4.81s/it]
 42%|████▏     | 94351/225000 [170:17:59<202:00:00,  5.57s/it]
 42%|████▏     | 94352/225000 [170:18:06<212:33:21,  5.86s/it]
 42%|████▏     | 94353/225000 [170:18:09<180:43:40,  4.98s/it]
 42%|████▏     | 94354/225000 [170:18:13<167:52:17,  4.63s/it]
 42%|████▏     | 94355/225000 [170:18:17<161:49:02,  4.46s/it]
 42%|████▏     | 94356/225000 [170:18:29<242:55:24,  6.69s/it]
 42%|████▏     | 94357/225000 [170:18:32<210:40:35,  5.81s/it]
 42%|████▏     | 94358/225000 [170:18:35<178:52:45,  4.93s/it]
 42%|████▏     | 94359/225000 [170:18:44<222:04:38,  6.12s/it]
 42%|████▏     | 94360/225000 [170:18:48<195:29:45,  5.39s/it]
 42%|████▏     | 94361/225000 [170:18:54<204:35:26,  5.64s/it]
 42%|████▏     | 94362/225000 [170:18:58<183:42:49,  5.06s/it]
 42%|████▏     | 94363/225000 [170:19:02<171:07:51,  4.72s/it]
 42%|████▏     | 94364/225000 [170:19:06<160:46:32,  4.43s/it]
 42%|████▏     | 94365/225000 [170:19:09<148:00:38,  4.08s/it]
 42%|████▏     | 94366/225000 [170:19:13<150:41:27,  4.15s/it]
 42%|████▏     | 94367/225000 [170:19:17<147:15:56,  4.06s/it]
 42%|████▏     | 94368/225000 [170:19:26<197:24:13,  5.44s/it]
 42%|████▏     | 94369/225000 [170:19:30<188:34:34,  5.20s/it]
 42%|████▏     | 94370/225000 [170:19:35<179:34:20,  4.95s/it]
 42%|████▏     | 94371/225000 [170:19:41<198:22:33,  5.47s/it]
 42%|████▏     | 94372/225000 [170:19:45<175:26:49,  4.84s/it]
 42%|████▏     | 94373/225000 [170:19:54<219:55:42,  6.06s/it]
 42%|████▏     | 94374/225000 [170:19:57<191:09:03,  5.27s/it]
 42%|████▏     | 94375/225000 [170:20:01<178:10:52,  4.91s/it]
 42%|████▏     | 94376/225000 [170:20:08<199:20:35,  5.49s/it]
 42%|████▏     | 94377/225000 [170:20:16<222:22:37,  6.13s/it]
 42%|████▏     | 94378/225000 [170:20:18<187:55:19,  5.18s/it]
 42%|████▏     | 94379/225000 [170:20:22<167:56:27,  4.63s/it]
 42%|████▏     | 94380/225000 [170:20:31<220:48:00,  6.09s/it]
 42%|████▏     | 94381/225000 [170:20:35<198:02:09,  5.46s/it]
 42%|████▏     | 94382/225000 [170:20:39<180:13:11,  4.97s/it]
 42%|████▏     | 94383/225000 [170:20:42<154:57:33,  4.27s/it]
 42%|████▏     | 94384/225000 [170:20:52<219:24:44,  6.05s/it]
 42%|████▏     | 94385/225000 [170:20:55<187:04:26,  5.16s/it]
 42%|████▏     | 94386/225000 [170:20:59<174:26:46,  4.81s/it]
 42%|████▏     | 94387/225000 [170:21:04<170:51:21,  4.71s/it]
 42%|████▏     | 94388/225000 [170:21:07<160:27:40,  4.42s/it]
 42%|████▏     | 94389/225000 [170:21:12<160:31:40,  4.42s/it]
 42%|████▏     | 94390/225000 [170:21:16<163:19:46,  4.50s/it]
 42%|████▏     | 94391/225000 [170:21:21<162:57:17,  4.49s/it]
 42%|████▏     | 94392/225000 [170:21:28<194:59:22,  5.37s/it]
 42%|████▏     | 94393/225000 [170:21:38<240:36:16,  6.63s/it]
 42%|████▏     | 94394/225000 [170:21:42<212:33:47,  5.86s/it]
 42%|████▏     | 94395/225000 [170:21:46<191:38:31,  5.28s/it]
 42%|████▏     | 94396/225000 [170:21:50<181:02:28,  4.99s/it]
 42%|████▏     | 94397/225000 [170:21:54<168:48:32,  4.65s/it]
 42%|████▏     | 94398/225000 [170:22:04<226:41:33,  6.25s/it]
 42%|████▏     | 94399/225000 [170:22:11<231:08:47,  6.37s/it]
 42%|████▏     | 94400/225000 [170:22:15<204:42:38,  5.64s/it]
                                                              
{'loss': '0.1476', 'grad_norm': '8.247', 'learning_rate': '3.329e-05', 'epoch': '0.4196', 'num_input_tokens_seen': 278933576, 'train_runtime': '6.133e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94400/225000 [170:22:15<204:42:38,  5.64s/it]
 42%|████▏     | 94401/225000 [170:22:19<188:01:01,  5.18s/it]
 42%|████▏     | 94402/225000 [170:22:26<211:15:52,  5.82s/it]
 42%|████▏     | 94403/225000 [170:22:29<177:56:12,  4.90s/it]
 42%|████▏     | 94404/225000 [170:22:37<210:39:45,  5.81s/it]
 42%|████▏     | 94405/225000 [170:22:40<187:36:21,  5.17s/it]
 42%|████▏     | 94406/225000 [170:22:45<177:55:09,  4.90s/it]
 42%|████▏     | 94407/225000 [170:22:48<162:02:58,  4.47s/it]
 42%|████▏     | 94408/225000 [170:22:57<214:59:30,  5.93s/it]
 42%|████▏     | 94409/225000 [170:23:03<205:59:48,  5.68s/it]
 42%|████▏     | 94410/225000 [170:23:06<177:38:41,  4.90s/it]
 42%|████▏     | 94411/225000 [170:23:13<204:33:19,  5.64s/it]
 42%|████▏     | 94412/225000 [170:23:16<180:46:47,  4.98s/it]
 42%|████▏     | 94413/225000 [170:23:24<205:40:20,  5.67s/it]
 42%|████▏     | 94414/225000 [170:23:28<186:52:44,  5.15s/it]
 42%|████▏     | 94415/225000 [170:23:32<183:27:46,  5.06s/it]
 42%|████▏     | 94416/225000 [170:23:37<173:51:08,  4.79s/it]
 42%|████▏     | 94417/225000 [170:23:41<167:40:05,  4.62s/it]
 42%|████▏     | 94418/225000 [170:23:48<189:17:26,  5.22s/it]
 42%|████▏     | 94419/225000 [170:23:56<228:39:04,  6.30s/it]
 42%|████▏     | 94420/225000 [170:24:00<201:51:17,  5.57s/it]
 42%|████▏     | 94421/225000 [170:24:04<178:19:41,  4.92s/it]
 42%|████▏     | 94422/225000 [170:24:07<163:31:41,  4.51s/it]
 42%|████▏     | 94423/225000 [170:24:18<232:53:27,  6.42s/it]
 42%|████▏     | 94424/225000 [170:24:21<199:26:03,  5.50s/it]
 42%|████▏     | 94425/225000 [170:24:25<179:13:00,  4.94s/it]
 42%|████▏     | 94426/225000 [170:24:29<169:07:54,  4.66s/it]
 42%|████▏     | 94427/225000 [170:24:33<161:10:02,  4.44s/it]
 42%|████▏     | 94428/225000 [170:24:44<234:55:38,  6.48s/it]
 42%|████▏     | 94429/225000 [170:24:49<212:47:33,  5.87s/it]
 42%|████▏     | 94430/225000 [170:24:52<185:48:05,  5.12s/it]
 42%|████▏     | 94431/225000 [170:24:56<173:51:18,  4.79s/it]
 42%|████▏     | 94432/225000 [170:25:00<169:50:50,  4.68s/it]
 42%|████▏     | 94433/225000 [170:25:11<238:54:51,  6.59s/it]
 42%|████▏     | 94434/225000 [170:25:18<233:55:40,  6.45s/it]
 42%|████▏     | 94435/225000 [170:25:21<198:24:55,  5.47s/it]
 42%|████▏     | 94436/225000 [170:25:26<193:21:15,  5.33s/it]
 42%|████▏     | 94437/225000 [170:25:34<220:58:37,  6.09s/it]
 42%|████▏     | 94438/225000 [170:25:37<187:23:31,  5.17s/it]
 42%|████▏     | 94439/225000 [170:25:41<176:45:54,  4.87s/it]
 42%|████▏     | 94440/225000 [170:25:45<166:09:07,  4.58s/it]
 42%|████▏     | 94441/225000 [170:25:50<169:05:10,  4.66s/it]
 42%|████▏     | 94442/225000 [170:25:54<167:47:59,  4.63s/it]
 42%|████▏     | 94443/225000 [170:26:01<191:45:59,  5.29s/it]
 42%|████▏     | 94444/225000 [170:26:09<220:52:21,  6.09s/it]
 42%|████▏     | 94445/225000 [170:26:13<194:24:32,  5.36s/it]
 42%|████▏     | 94446/225000 [170:26:17<181:43:21,  5.01s/it]
 42%|████▏     | 94447/225000 [170:26:22<179:20:16,  4.95s/it]
 42%|████▏     | 94448/225000 [170:26:31<224:19:39,  6.19s/it]
 42%|████▏     | 94449/225000 [170:26:34<187:49:37,  5.18s/it]
 42%|████▏     | 94450/225000 [170:26:41<207:56:24,  5.73s/it]
                                                              
{'loss': '0.1276', 'grad_norm': '4.815', 'learning_rate': '3.328e-05', 'epoch': '0.4198', 'num_input_tokens_seen': 279053824, 'train_runtime': '6.136e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94450/225000 [170:26:41<207:56:24,  5.73s/it]
 42%|████▏     | 94451/225000 [170:26:45<188:49:36,  5.21s/it]
 42%|████▏     | 94452/225000 [170:26:48<170:21:55,  4.70s/it]
 42%|████▏     | 94453/225000 [170:26:52<165:53:55,  4.57s/it]
 42%|████▏     | 94454/225000 [170:26:56<154:16:01,  4.25s/it]
 42%|████▏     | 94455/225000 [170:27:00<152:53:24,  4.22s/it]
 42%|████▏     | 94456/225000 [170:27:04<153:08:07,  4.22s/it]
 42%|████▏     | 94457/225000 [170:27:10<167:10:59,  4.61s/it]
 42%|████▏     | 94458/225000 [170:27:21<240:35:40,  6.63s/it]
 42%|████▏     | 94459/225000 [170:27:26<227:06:21,  6.26s/it]
 42%|████▏     | 94460/225000 [170:27:30<199:58:25,  5.51s/it]
 42%|████▏     | 94461/225000 [170:27:36<198:52:03,  5.48s/it]
 42%|████▏     | 94462/225000 [170:27:45<245:53:51,  6.78s/it]
 42%|████▏     | 94463/225000 [170:27:49<206:53:09,  5.71s/it]
 42%|████▏     | 94464/225000 [170:27:53<187:19:26,  5.17s/it]
 42%|████▏     | 94465/225000 [170:27:57<176:00:15,  4.85s/it]
 42%|████▏     | 94466/225000 [170:28:01<165:52:33,  4.57s/it]
 42%|████▏     | 94467/225000 [170:28:12<242:10:47,  6.68s/it]
 42%|████▏     | 94468/225000 [170:28:16<208:11:48,  5.74s/it]
 42%|████▏     | 94469/225000 [170:28:19<185:43:49,  5.12s/it]
 42%|████▏     | 94470/225000 [170:28:25<195:33:41,  5.39s/it]
 42%|████▏     | 94471/225000 [170:28:32<203:56:39,  5.62s/it]
 42%|████▏     | 94472/225000 [170:28:35<174:59:40,  4.83s/it]
 42%|████▏     | 94473/225000 [170:28:38<159:40:18,  4.40s/it]
 42%|████▏     | 94474/225000 [170:28:48<218:35:58,  6.03s/it]
 42%|████▏     | 94475/225000 [170:28:51<190:46:35,  5.26s/it]
 42%|████▏     | 94476/225000 [170:28:55<175:48:45,  4.85s/it]
 42%|████▏     | 94477/225000 [170:28:59<165:05:12,  4.55s/it]
 42%|████▏     | 94478/225000 [170:29:09<223:09:51,  6.16s/it]
 42%|████▏     | 94479/225000 [170:29:12<189:48:31,  5.24s/it]
 42%|████▏     | 94480/225000 [170:29:15<170:03:26,  4.69s/it]
 42%|████▏     | 94481/225000 [170:29:19<156:28:21,  4.32s/it]
 42%|████▏     | 94482/225000 [170:29:24<163:07:11,  4.50s/it]
 42%|████▏     | 94483/225000 [170:29:28<163:07:36,  4.50s/it]
 42%|████▏     | 94484/225000 [170:29:32<157:56:06,  4.36s/it]
 42%|████▏     | 94485/225000 [170:29:37<162:20:48,  4.48s/it]
 42%|████▏     | 94486/225000 [170:29:46<209:08:22,  5.77s/it]
 42%|████▏     | 94487/225000 [170:29:55<248:43:50,  6.86s/it]
 42%|████▏     | 94488/225000 [170:29:58<208:13:54,  5.74s/it]
 42%|████▏     | 94489/225000 [170:30:02<180:51:56,  4.99s/it]
 42%|████▏     | 94490/225000 [170:30:11<230:16:32,  6.35s/it]
 42%|████▏     | 94491/225000 [170:30:16<215:49:20,  5.95s/it]
 42%|████▏     | 94492/225000 [170:30:20<189:45:17,  5.23s/it]
 42%|████▏     | 94493/225000 [170:30:24<173:39:53,  4.79s/it]
 42%|████▏     | 94494/225000 [170:30:28<165:59:40,  4.58s/it]
 42%|████▏     | 94495/225000 [170:30:38<228:16:07,  6.30s/it]
 42%|████▏     | 94496/225000 [170:30:41<198:12:48,  5.47s/it]
 42%|████▏     | 94497/225000 [170:30:45<181:17:01,  5.00s/it]
 42%|████▏     | 94498/225000 [170:30:55<236:49:47,  6.53s/it]
 42%|████▏     | 94499/225000 [170:30:59<206:51:06,  5.71s/it]
 42%|████▏     | 94500/225000 [170:31:04<194:16:47,  5.36s/it]
                                                              
{'loss': '0.1372', 'grad_norm': '2.845', 'learning_rate': '3.326e-05', 'epoch': '0.42', 'num_input_tokens_seen': 279174768, 'train_runtime': '6.139e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94500/225000 [170:31:04<194:16:47,  5.36s/it]
 42%|████▏     | 94501/225000 [170:31:16<266:13:56,  7.34s/it]
 42%|████▏     | 94502/225000 [170:31:19<217:00:47,  5.99s/it]
 42%|████▏     | 94503/225000 [170:31:24<206:42:15,  5.70s/it]
 42%|████▏     | 94504/225000 [170:31:29<200:46:09,  5.54s/it]
 42%|████▏     | 94505/225000 [170:31:34<194:13:24,  5.36s/it]
 42%|████▏     | 94506/225000 [170:31:38<186:35:02,  5.15s/it]
 42%|████▏     | 94507/225000 [170:31:43<183:14:43,  5.06s/it]
 42%|████▏     | 94508/225000 [170:31:47<171:34:10,  4.73s/it]
 42%|████▏     | 94509/225000 [170:31:52<170:44:33,  4.71s/it]
 42%|████▏     | 94510/225000 [170:31:56<166:36:14,  4.60s/it]
 42%|████▏     | 94511/225000 [170:32:01<166:21:16,  4.59s/it]
 42%|████▏     | 94512/225000 [170:32:06<170:59:54,  4.72s/it]
 42%|████▏     | 94513/225000 [170:32:10<168:50:26,  4.66s/it]
 42%|████▏     | 94514/225000 [170:32:14<161:49:34,  4.46s/it]
 42%|████▏     | 94515/225000 [170:32:27<248:01:58,  6.84s/it]
 42%|████▏     | 94516/225000 [170:32:31<223:16:17,  6.16s/it]
 42%|████▏     | 94517/225000 [170:32:36<202:45:55,  5.59s/it]
 42%|████▏     | 94518/225000 [170:32:46<253:32:23,  7.00s/it]
 42%|████▏     | 94519/225000 [170:32:50<226:20:28,  6.24s/it]
 42%|████▏     | 94520/225000 [170:32:54<199:44:01,  5.51s/it]
 42%|████▏     | 94521/225000 [170:32:59<194:54:39,  5.38s/it]
 42%|████▏     | 94522/225000 [170:33:04<189:05:47,  5.22s/it]
 42%|████▏     | 94523/225000 [170:33:09<186:56:39,  5.16s/it]
 42%|████▏     | 94524/225000 [170:33:13<174:18:47,  4.81s/it]
 42%|████▏     | 94525/225000 [170:33:17<167:59:50,  4.64s/it]
 42%|████▏     | 94526/225000 [170:33:22<171:45:30,  4.74s/it]
 42%|████▏     | 94527/225000 [170:33:37<276:03:38,  7.62s/it]
 42%|████▏     | 94528/225000 [170:33:41<236:56:16,  6.54s/it]
 42%|████▏     | 94529/225000 [170:33:44<203:35:45,  5.62s/it]
 42%|████▏     | 94530/225000 [170:33:48<186:57:57,  5.16s/it]
 42%|████▏     | 94531/225000 [170:33:59<251:12:45,  6.93s/it]
 42%|████▏     | 94532/225000 [170:34:04<228:06:33,  6.29s/it]
 42%|████▏     | 94533/225000 [170:34:06<185:46:04,  5.13s/it]
 42%|████▏     | 94534/225000 [170:34:10<174:04:22,  4.80s/it]
 42%|████▏     | 94535/225000 [170:34:15<167:02:50,  4.61s/it]
 42%|████▏     | 94536/225000 [170:34:23<206:57:50,  5.71s/it]
 42%|████▏     | 94537/225000 [170:34:31<235:54:31,  6.51s/it]
 42%|████▏     | 94538/225000 [170:34:36<218:23:10,  6.03s/it]
 42%|████▏     | 94539/225000 [170:34:45<247:31:41,  6.83s/it]
 42%|████▏     | 94540/225000 [170:34:48<209:38:01,  5.78s/it]
 42%|████▏     | 94541/225000 [170:34:53<193:27:13,  5.34s/it]
 42%|████▏     | 94542/225000 [170:34:56<176:07:42,  4.86s/it]
 42%|████▏     | 94543/225000 [170:35:03<195:05:41,  5.38s/it]
 42%|████▏     | 94544/225000 [170:35:14<261:19:02,  7.21s/it]
 42%|████▏     | 94545/225000 [170:35:19<234:00:45,  6.46s/it]
 42%|████▏     | 94546/225000 [170:35:24<216:17:22,  5.97s/it]
 42%|████▏     | 94547/225000 [170:35:28<195:10:45,  5.39s/it]
 42%|████▏     | 94548/225000 [170:35:32<176:18:41,  4.87s/it]
 42%|████▏     | 94549/225000 [170:35:35<165:36:39,  4.57s/it]
 42%|████▏     | 94550/225000 [170:35:46<235:03:21,  6.49s/it]
                                                              
{'loss': '0.2522', 'grad_norm': '2.24', 'learning_rate': '3.324e-05', 'epoch': '0.4202', 'num_input_tokens_seen': 279303072, 'train_runtime': '6.141e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94550/225000 [170:35:46<235:03:21,  6.49s/it]
 42%|████▏     | 94551/225000 [170:35:54<250:49:21,  6.92s/it]
 42%|████▏     | 94552/225000 [170:35:59<222:34:38,  6.14s/it]
 42%|████▏     | 94553/225000 [170:36:04<211:03:40,  5.82s/it]
 42%|████▏     | 94554/225000 [170:36:09<206:22:36,  5.70s/it]
 42%|████▏     | 94555/225000 [170:36:17<230:20:13,  6.36s/it]
 42%|████▏     | 94556/225000 [170:36:21<199:38:48,  5.51s/it]
 42%|████▏     | 94557/225000 [170:36:29<230:19:35,  6.36s/it]
 42%|████▏     | 94558/225000 [170:36:33<202:02:48,  5.58s/it]
 42%|████▏     | 94559/225000 [170:36:42<248:24:19,  6.86s/it]
 42%|████▏     | 94560/225000 [170:36:47<220:27:02,  6.08s/it]
 42%|████▏     | 94561/225000 [170:36:51<195:15:32,  5.39s/it]
 42%|████▏     | 94562/225000 [170:36:55<184:12:05,  5.08s/it]
 42%|████▏     | 94563/225000 [170:37:01<200:17:11,  5.53s/it]
 42%|████▏     | 94564/225000 [170:37:11<246:18:25,  6.80s/it]
 42%|████▏     | 94565/225000 [170:37:16<228:09:03,  6.30s/it]
 42%|████▏     | 94566/225000 [170:37:23<237:02:06,  6.54s/it]
 42%|████▏     | 94567/225000 [170:37:28<218:45:58,  6.04s/it]
 42%|████▏     | 94568/225000 [170:37:31<183:46:16,  5.07s/it]
 42%|████▏     | 94569/225000 [170:37:36<181:48:10,  5.02s/it]
 42%|████▏     | 94570/225000 [170:37:47<249:32:34,  6.89s/it]
 42%|████▏     | 94571/225000 [170:37:51<214:01:57,  5.91s/it]
 42%|████▏     | 94572/225000 [170:37:54<185:16:03,  5.11s/it]
 42%|████▏     | 94573/225000 [170:38:10<297:17:36,  8.21s/it]
 42%|████▏     | 94574/225000 [170:38:13<248:55:09,  6.87s/it]
 42%|████▏     | 94575/225000 [170:38:17<218:12:13,  6.02s/it]
 42%|████▏     | 94576/225000 [170:38:21<190:55:59,  5.27s/it]
 42%|████▏     | 94577/225000 [170:38:25<177:17:23,  4.89s/it]
 42%|████▏     | 94578/225000 [170:38:29<163:18:53,  4.51s/it]
 42%|████▏     | 94579/225000 [170:38:42<255:13:56,  7.05s/it]
 42%|████▏     | 94580/225000 [170:38:46<227:49:39,  6.29s/it]
 42%|████▏     | 94581/225000 [170:38:50<207:15:40,  5.72s/it]
 42%|████▏     | 94582/225000 [170:39:02<266:01:01,  7.34s/it]
 42%|████▏     | 94583/225000 [170:39:05<224:33:55,  6.20s/it]
 42%|████▏     | 94584/225000 [170:39:11<225:48:48,  6.23s/it]
 42%|████▏     | 94585/225000 [170:39:20<254:21:41,  7.02s/it]
 42%|████▏     | 94586/225000 [170:39:25<230:56:37,  6.38s/it]
 42%|████▏     | 94587/225000 [170:39:30<219:37:34,  6.06s/it]
 42%|████▏     | 94588/225000 [170:39:38<231:20:29,  6.39s/it]
 42%|████▏     | 94589/225000 [170:39:41<199:21:40,  5.50s/it]
 42%|████▏     | 94590/225000 [170:39:45<182:21:41,  5.03s/it]
 42%|████▏     | 94591/225000 [170:39:48<163:13:20,  4.51s/it]
 42%|████▏     | 94592/225000 [170:39:54<174:58:42,  4.83s/it]
 42%|████▏     | 94593/225000 [170:40:03<224:17:48,  6.19s/it]
 42%|████▏     | 94594/225000 [170:40:08<205:14:47,  5.67s/it]
 42%|████▏     | 94595/225000 [170:40:11<181:01:07,  5.00s/it]
 42%|████▏     | 94596/225000 [170:40:16<179:09:07,  4.95s/it]
 42%|████▏     | 94597/225000 [170:40:23<203:21:44,  5.61s/it]
 42%|████▏     | 94598/225000 [170:40:28<198:37:21,  5.48s/it]
 42%|████▏     | 94599/225000 [170:40:33<185:15:34,  5.11s/it]
 42%|████▏     | 94600/225000 [170:40:38<187:52:56,  5.19s/it]
                                                              
{'loss': '0.2893', 'grad_norm': '5.458', 'learning_rate': '3.323e-05', 'epoch': '0.4204', 'num_input_tokens_seen': 279437440, 'train_runtime': '6.144e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94600/225000 [170:40:38<187:52:56,  5.19s/it]
 42%|████▏     | 94601/225000 [170:40:44<201:31:52,  5.56s/it]
 42%|████▏     | 94602/225000 [170:40:47<175:13:49,  4.84s/it]
 42%|████▏     | 94603/225000 [170:40:57<221:01:17,  6.10s/it]
 42%|████▏     | 94604/225000 [170:41:00<192:44:34,  5.32s/it]
 42%|████▏     | 94605/225000 [170:41:04<176:02:01,  4.86s/it]
 42%|████▏     | 94606/225000 [170:41:09<176:01:23,  4.86s/it]
 42%|████▏     | 94607/225000 [170:41:22<271:56:53,  7.51s/it]
 42%|████▏     | 94608/225000 [170:41:29<259:04:36,  7.15s/it]
 42%|████▏     | 94609/225000 [170:41:36<257:52:42,  7.12s/it]
 42%|████▏     | 94610/225000 [170:41:43<256:20:22,  7.08s/it]
 42%|████▏     | 94611/225000 [170:41:47<225:05:49,  6.21s/it]
 42%|████▏     | 94612/225000 [170:41:51<206:01:18,  5.69s/it]
 42%|████▏     | 94613/225000 [170:42:03<265:59:28,  7.34s/it]
 42%|████▏     | 94614/225000 [170:42:06<223:25:11,  6.17s/it]
 42%|████▏     | 94615/225000 [170:42:10<202:39:08,  5.60s/it]
 42%|████▏     | 94616/225000 [170:42:17<211:54:01,  5.85s/it]
 42%|████▏     | 94617/225000 [170:42:26<244:15:51,  6.74s/it]
 42%|████▏     | 94618/225000 [170:42:36<289:57:03,  8.01s/it]
 42%|████▏     | 94619/225000 [170:42:43<272:00:01,  7.51s/it]
 42%|████▏     | 94620/225000 [170:42:47<231:17:11,  6.39s/it]
 42%|████▏     | 94621/225000 [170:42:56<262:14:16,  7.24s/it]
 42%|████▏     | 94622/225000 [170:42:59<221:30:19,  6.12s/it]
 42%|████▏     | 94623/225000 [170:43:02<187:44:15,  5.18s/it]
 42%|████▏     | 94624/225000 [170:43:07<179:45:41,  4.96s/it]
 42%|████▏     | 94625/225000 [170:43:21<280:07:46,  7.74s/it]
 42%|████▏     | 94626/225000 [170:43:31<303:46:29,  8.39s/it]
 42%|████▏     | 94627/225000 [170:43:34<245:29:49,  6.78s/it]
 42%|████▏     | 94628/225000 [170:43:43<275:09:45,  7.60s/it]
 42%|████▏     | 94629/225000 [170:43:46<223:48:42,  6.18s/it]
 42%|████▏     | 94630/225000 [170:43:54<236:24:18,  6.53s/it]
 42%|████▏     | 94631/225000 [170:43:58<215:32:15,  5.95s/it]
 42%|████▏     | 94632/225000 [170:44:07<240:42:00,  6.65s/it]
 42%|████▏     | 94633/225000 [170:44:10<208:16:56,  5.75s/it]
 42%|████▏     | 94634/225000 [170:44:14<183:27:16,  5.07s/it]
 42%|████▏     | 94635/225000 [170:44:18<170:25:45,  4.71s/it]
 42%|████▏     | 94636/225000 [170:44:22<163:06:49,  4.50s/it]
 42%|████▏     | 94637/225000 [170:44:26<161:54:12,  4.47s/it]
 42%|████▏     | 94638/225000 [170:44:31<163:57:18,  4.53s/it]
 42%|████▏     | 94639/225000 [170:44:45<274:49:25,  7.59s/it]
 42%|████▏     | 94640/225000 [170:44:49<234:34:43,  6.48s/it]
 42%|████▏     | 94641/225000 [170:44:53<203:03:19,  5.61s/it]
 42%|████▏     | 94642/225000 [170:45:00<225:18:13,  6.22s/it]
 42%|████▏     | 94643/225000 [170:45:11<272:50:00,  7.53s/it]
 42%|████▏     | 94644/225000 [170:45:20<289:25:26,  7.99s/it]
 42%|████▏     | 94645/225000 [170:45:31<318:49:12,  8.80s/it]
 42%|████▏     | 94646/225000 [170:45:34<261:57:06,  7.23s/it]
 42%|████▏     | 94647/225000 [170:45:41<258:18:31,  7.13s/it]
 42%|████▏     | 94648/225000 [170:45:45<218:31:37,  6.04s/it]
 42%|████▏     | 94649/225000 [170:45:49<197:25:14,  5.45s/it]
 42%|████▏     | 94650/225000 [170:45:54<194:10:48,  5.36s/it]
                                                              
{'loss': '0.3594', 'grad_norm': '4.293', 'learning_rate': '3.321e-05', 'epoch': '0.4207', 'num_input_tokens_seen': 279582840, 'train_runtime': '6.148e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94650/225000 [170:45:54<194:10:48,  5.36s/it]
 42%|████▏     | 94651/225000 [170:46:06<267:11:28,  7.38s/it]
 42%|████▏     | 94652/225000 [170:46:11<237:22:40,  6.56s/it]
 42%|████▏     | 94653/225000 [170:46:21<273:07:26,  7.54s/it]
 42%|████▏     | 94654/225000 [170:46:25<243:03:13,  6.71s/it]
 42%|████▏     | 94655/225000 [170:46:29<212:40:54,  5.87s/it]
 42%|████▏     | 94656/225000 [170:46:42<288:45:21,  7.98s/it]
 42%|████▏     | 94657/225000 [170:46:52<313:54:38,  8.67s/it]
 42%|████▏     | 94658/225000 [170:46:56<262:51:23,  7.26s/it]
 42%|████▏     | 94659/225000 [170:47:03<261:17:16,  7.22s/it]
 42%|████▏     | 94660/225000 [170:47:07<225:13:29,  6.22s/it]
 42%|████▏     | 94661/225000 [170:47:17<258:32:55,  7.14s/it]
 42%|████▏     | 94662/225000 [170:47:21<228:01:54,  6.30s/it]
 42%|████▏     | 94663/225000 [170:47:32<279:47:30,  7.73s/it]
 42%|████▏     | 94664/225000 [170:47:36<235:31:15,  6.51s/it]
 42%|████▏     | 94665/225000 [170:47:41<225:13:49,  6.22s/it]
 42%|████▏     | 94666/225000 [170:47:45<201:42:19,  5.57s/it]
 42%|████▏     | 94667/225000 [170:47:49<182:41:54,  5.05s/it]
 42%|████▏     | 94668/225000 [170:47:54<178:02:30,  4.92s/it]
 42%|████▏     | 94669/225000 [170:47:59<182:42:47,  5.05s/it]
 42%|████▏     | 94670/225000 [170:48:04<178:01:02,  4.92s/it]
 42%|████▏     | 94671/225000 [170:48:19<291:44:50,  8.06s/it]
 42%|████▏     | 94672/225000 [170:48:22<233:17:34,  6.44s/it]
 42%|████▏     | 94673/225000 [170:48:36<318:14:15,  8.79s/it]
 42%|████▏     | 94674/225000 [170:48:40<261:47:31,  7.23s/it]
 42%|████▏     | 94675/225000 [170:48:44<227:54:04,  6.30s/it]
 42%|████▏     | 94676/225000 [170:48:47<197:45:18,  5.46s/it]
 42%|████▏     | 94677/225000 [170:48:58<251:06:27,  6.94s/it]
 42%|████▏     | 94678/225000 [170:49:02<227:54:48,  6.30s/it]
 42%|████▏     | 94679/225000 [170:49:06<202:59:59,  5.61s/it]
 42%|████▏     | 94680/225000 [170:49:18<265:30:07,  7.33s/it]
 42%|████▏     | 94681/225000 [170:49:25<268:34:31,  7.42s/it]
 42%|████▏     | 94682/225000 [170:49:29<225:41:26,  6.23s/it]
 42%|████▏     | 94683/225000 [170:49:33<200:43:11,  5.54s/it]
 42%|████▏     | 94684/225000 [170:49:43<254:12:27,  7.02s/it]
 42%|████▏     | 94685/225000 [170:49:52<267:40:17,  7.39s/it]
 42%|████▏     | 94686/225000 [170:49:59<270:08:30,  7.46s/it]
 42%|████▏     | 94687/225000 [170:50:03<228:01:50,  6.30s/it]
 42%|████▏     | 94688/225000 [170:50:10<236:36:02,  6.54s/it]
 42%|████▏     | 94689/225000 [170:50:17<243:40:46,  6.73s/it]
 42%|████▏     | 94690/225000 [170:50:24<250:07:21,  6.91s/it]
 42%|████▏     | 94691/225000 [170:50:29<225:17:22,  6.22s/it]
 42%|████▏     | 94692/225000 [170:50:41<286:31:32,  7.92s/it]
 42%|████▏     | 94693/225000 [170:50:45<248:51:59,  6.88s/it]
 42%|████▏     | 94694/225000 [170:50:49<217:28:06,  6.01s/it]
 42%|████▏     | 94695/225000 [170:50:54<207:23:19,  5.73s/it]
 42%|████▏     | 94696/225000 [170:51:05<255:09:15,  7.05s/it]
 42%|████▏     | 94697/225000 [170:51:10<237:58:57,  6.57s/it]
 42%|████▏     | 94698/225000 [170:51:20<276:59:36,  7.65s/it]
 42%|████▏     | 94699/225000 [170:51:28<283:55:23,  7.84s/it]
 42%|████▏     | 94700/225000 [170:51:35<266:13:24,  7.36s/it]
                                                              
{'loss': '0.4321', 'grad_norm': '1.996', 'learning_rate': '3.319e-05', 'epoch': '0.4209', 'num_input_tokens_seen': 279740112, 'train_runtime': '6.151e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94700/225000 [170:51:35<266:13:24,  7.36s/it]
 42%|████▏     | 94701/225000 [170:51:46<310:54:17,  8.59s/it]
 42%|████▏     | 94702/225000 [170:51:52<279:56:27,  7.73s/it]
 42%|████▏     | 94703/225000 [170:51:56<243:33:05,  6.73s/it]
 42%|████▏     | 94704/225000 [170:52:00<214:31:40,  5.93s/it]
 42%|████▏     | 94705/225000 [170:52:10<250:44:49,  6.93s/it]
 42%|████▏     | 94706/225000 [170:52:14<223:06:45,  6.16s/it]
 42%|████▏     | 94707/225000 [170:52:25<274:15:21,  7.58s/it]
 42%|████▏     | 94708/225000 [170:52:32<274:21:30,  7.58s/it]
 42%|████▏     | 94709/225000 [170:52:40<278:26:53,  7.69s/it]
 42%|████▏     | 94710/225000 [170:52:49<293:39:39,  8.11s/it]
 42%|████▏     | 94711/225000 [170:52:53<243:51:39,  6.74s/it]
 42%|████▏     | 94712/225000 [170:53:03<276:00:38,  7.63s/it]
 42%|████▏     | 94713/225000 [170:53:08<246:29:55,  6.81s/it]
 42%|████▏     | 94714/225000 [170:53:15<257:56:03,  7.13s/it]
 42%|████▏     | 94715/225000 [170:53:19<213:26:22,  5.90s/it]
 42%|████▏     | 94716/225000 [170:53:32<294:46:40,  8.15s/it]
 42%|████▏     | 94717/225000 [170:53:42<313:26:48,  8.66s/it]
 42%|████▏     | 94718/225000 [170:53:46<265:39:02,  7.34s/it]
 42%|████▏     | 94719/225000 [170:53:56<296:42:30,  8.20s/it]
 42%|████▏     | 94720/225000 [170:54:06<314:11:03,  8.68s/it]
 42%|████▏     | 94721/225000 [170:54:10<267:20:15,  7.39s/it]
 42%|████▏     | 94722/225000 [170:54:15<232:22:43,  6.42s/it]
 42%|████▏     | 94723/225000 [170:54:19<215:53:55,  5.97s/it]
 42%|████▏     | 94724/225000 [170:54:31<276:23:35,  7.64s/it]
 42%|████▏     | 94725/225000 [170:54:35<233:14:04,  6.45s/it]
 42%|████▏     | 94726/225000 [170:54:38<204:00:52,  5.64s/it]
 42%|████▏     | 94727/225000 [170:54:51<282:40:27,  7.81s/it]
 42%|████▏     | 94728/225000 [170:54:56<247:49:51,  6.85s/it]
 42%|████▏     | 94729/225000 [170:54:59<211:30:37,  5.85s/it]
 42%|████▏     | 94730/225000 [170:55:09<255:14:04,  7.05s/it]
 42%|████▏     | 94731/225000 [170:55:17<261:26:55,  7.23s/it]
 42%|████▏     | 94732/225000 [170:55:20<221:41:42,  6.13s/it]
 42%|████▏     | 94733/225000 [170:55:28<236:46:24,  6.54s/it]
 42%|████▏     | 94734/225000 [170:55:36<250:22:00,  6.92s/it]
 42%|████▏     | 94735/225000 [170:55:45<278:40:45,  7.70s/it]
 42%|████▏     | 94736/225000 [170:55:56<308:02:57,  8.51s/it]
 42%|████▏     | 94737/225000 [170:56:02<287:56:08,  7.96s/it]
 42%|████▏     | 94738/225000 [170:56:08<261:16:42,  7.22s/it]
 42%|████▏     | 94739/225000 [170:56:11<220:55:44,  6.11s/it]
 42%|████▏     | 94740/225000 [170:56:24<288:53:12,  7.98s/it]
 42%|████▏     | 94741/225000 [170:56:29<261:03:25,  7.21s/it]
 42%|████▏     | 94742/225000 [170:56:38<279:52:30,  7.74s/it]
 42%|████▏     | 94743/225000 [170:56:47<294:51:38,  8.15s/it]
 42%|████▏     | 94744/225000 [170:56:58<325:15:51,  8.99s/it]
 42%|████▏     | 94745/225000 [170:57:02<269:33:43,  7.45s/it]
 42%|████▏     | 94746/225000 [170:57:08<251:53:59,  6.96s/it]
 42%|████▏     | 94747/225000 [170:57:22<324:34:00,  8.97s/it]
 42%|████▏     | 94748/225000 [170:57:26<279:33:59,  7.73s/it]
 42%|████▏     | 94749/225000 [170:57:33<271:32:10,  7.50s/it]
 42%|████▏     | 94750/225000 [170:57:37<231:12:30,  6.39s/it]
                                                              
{'loss': '0.5482', 'grad_norm': '3.263', 'learning_rate': '3.317e-05', 'epoch': '0.4211', 'num_input_tokens_seen': 279907504, 'train_runtime': '6.155e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94750/225000 [170:57:37<231:12:30,  6.39s/it]
 42%|████▏     | 94751/225000 [170:57:41<204:21:31,  5.65s/it]
 42%|████▏     | 94752/225000 [170:57:48<216:18:21,  5.98s/it]
 42%|████▏     | 94753/225000 [170:58:00<279:54:00,  7.74s/it]
 42%|████▏     | 94754/225000 [170:58:06<260:01:40,  7.19s/it]
 42%|████▏     | 94755/225000 [170:58:10<228:15:51,  6.31s/it]
 42%|████▏     | 94756/225000 [170:58:22<289:07:04,  7.99s/it]
 42%|████▏     | 94757/225000 [170:58:31<298:32:06,  8.25s/it]
 42%|████▏     | 94758/225000 [170:58:35<255:05:05,  7.05s/it]
 42%|████▏     | 94759/225000 [170:58:48<324:04:03,  8.96s/it]
 42%|████▏     | 94760/225000 [170:58:56<314:26:06,  8.69s/it]
 42%|████▏     | 94761/225000 [170:59:04<306:05:39,  8.46s/it]
 42%|████▏     | 94762/225000 [170:59:12<294:01:05,  8.13s/it]
 42%|████▏     | 94763/225000 [170:59:16<251:09:42,  6.94s/it]
 42%|████▏     | 94764/225000 [170:59:23<258:32:44,  7.15s/it]
 42%|████▏     | 94765/225000 [170:59:28<231:34:52,  6.40s/it]
 42%|████▏     | 94766/225000 [170:59:32<207:29:56,  5.74s/it]
 42%|████▏     | 94767/225000 [170:59:47<308:08:44,  8.52s/it]
 42%|████▏     | 94768/225000 [170:59:50<246:42:08,  6.82s/it]
 42%|████▏     | 94769/225000 [171:00:01<291:37:22,  8.06s/it]
 42%|████▏     | 94770/225000 [171:00:10<304:59:12,  8.43s/it]
 42%|████▏     | 94771/225000 [171:00:15<269:08:55,  7.44s/it]
 42%|████▏     | 94772/225000 [171:00:20<234:22:39,  6.48s/it]
 42%|████▏     | 94773/225000 [171:00:34<323:02:15,  8.93s/it]
 42%|████▏     | 94774/225000 [171:00:39<277:58:09,  7.68s/it]
 42%|████▏     | 94775/225000 [171:00:45<253:24:00,  7.01s/it]
 42%|████▏     | 94776/225000 [171:00:53<268:47:06,  7.43s/it]
 42%|████▏     | 94777/225000 [171:00:58<245:41:42,  6.79s/it]
 42%|████▏     | 94778/225000 [171:01:02<217:27:28,  6.01s/it]
 42%|████▏     | 94779/225000 [171:01:14<272:51:30,  7.54s/it]
 42%|████▏     | 94780/225000 [171:01:20<261:48:48,  7.24s/it]
 42%|████▏     | 94781/225000 [171:01:24<225:11:16,  6.23s/it]
 42%|████▏     | 94782/225000 [171:01:34<262:40:48,  7.26s/it]
 42%|████▏     | 94783/225000 [171:01:41<266:19:54,  7.36s/it]
 42%|████▏     | 94784/225000 [171:01:56<346:07:29,  9.57s/it]
 42%|████▏     | 94785/225000 [171:02:05<335:05:00,  9.26s/it]
 42%|████▏     | 94786/225000 [171:02:16<360:23:02,  9.96s/it]
 42%|████▏     | 94787/225000 [171:02:22<311:26:54,  8.61s/it]
 42%|████▏     | 94788/225000 [171:02:33<340:08:07,  9.40s/it]
 42%|████▏     | 94789/225000 [171:02:40<312:24:31,  8.64s/it]
 42%|████▏     | 94790/225000 [171:02:46<290:06:23,  8.02s/it]
 42%|████▏     | 94791/225000 [171:02:57<318:13:26,  8.80s/it]
 42%|████▏     | 94792/225000 [171:03:03<290:18:35,  8.03s/it]
 42%|████▏     | 94793/225000 [171:03:10<275:00:36,  7.60s/it]
 42%|████▏     | 94794/225000 [171:03:25<357:56:21,  9.90s/it]
 42%|████▏     | 94795/225000 [171:03:31<312:24:58,  8.64s/it]
 42%|████▏     | 94796/225000 [171:03:36<272:00:46,  7.52s/it]
 42%|████▏     | 94797/225000 [171:03:46<306:22:24,  8.47s/it]
 42%|████▏     | 94798/225000 [171:03:58<341:03:44,  9.43s/it]
 42%|████▏     | 94799/225000 [171:04:14<412:38:02, 11.41s/it]
 42%|████▏     | 94800/225000 [171:04:18<329:36:40,  9.11s/it]
                                                              
{'loss': '0.5771', 'grad_norm': '2.202', 'learning_rate': '3.316e-05', 'epoch': '0.4213', 'num_input_tokens_seen': 280093200, 'train_runtime': '6.159e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94800/225000 [171:04:18<329:36:40,  9.11s/it]
 42%|████▏     | 94801/225000 [171:04:29<351:34:30,  9.72s/it]
 42%|████▏     | 94802/225000 [171:04:37<338:42:58,  9.37s/it]
 42%|████▏     | 94803/225000 [171:04:46<334:42:19,  9.25s/it]
 42%|████▏     | 94804/225000 [171:04:54<316:37:30,  8.75s/it]
 42%|████▏     | 94805/225000 [171:04:59<281:10:51,  7.77s/it]
 42%|████▏     | 94806/225000 [171:05:09<295:47:39,  8.18s/it]
 42%|████▏     | 94807/225000 [171:05:16<283:56:03,  7.85s/it]
 42%|████▏     | 94808/225000 [171:05:21<256:34:09,  7.09s/it]
 42%|████▏     | 94809/225000 [171:05:27<249:08:58,  6.89s/it]
 42%|████▏     | 94810/225000 [171:05:37<279:39:37,  7.73s/it]
 42%|████▏     | 94811/225000 [171:05:41<241:08:24,  6.67s/it]
 42%|████▏     | 94812/225000 [171:05:47<229:59:52,  6.36s/it]
 42%|████▏     | 94813/225000 [171:05:59<296:51:46,  8.21s/it]
 42%|████▏     | 94814/225000 [171:06:04<262:12:13,  7.25s/it]
 42%|████▏     | 94815/225000 [171:06:17<322:29:06,  8.92s/it]
 42%|████▏     | 94816/225000 [171:06:25<306:46:21,  8.48s/it]
 42%|████▏     | 94817/225000 [171:06:31<284:54:56,  7.88s/it]
 42%|████▏     | 94818/225000 [171:06:36<247:10:17,  6.84s/it]
 42%|████▏     | 94819/225000 [171:06:46<289:49:33,  8.01s/it]
 42%|████▏     | 94820/225000 [171:06:52<266:35:57,  7.37s/it]
 42%|████▏     | 94821/225000 [171:06:56<224:56:44,  6.22s/it]
 42%|████▏     | 94822/225000 [171:07:06<271:20:20,  7.50s/it]
 42%|████▏     | 94823/225000 [171:07:10<232:51:51,  6.44s/it]
 42%|████▏     | 94824/225000 [171:07:16<230:26:08,  6.37s/it]
 42%|████▏     | 94825/225000 [171:07:24<238:45:44,  6.60s/it]
 42%|████▏     | 94826/225000 [171:07:28<216:46:37,  6.00s/it]
 42%|████▏     | 94827/225000 [171:07:41<286:02:58,  7.91s/it]
 42%|████▏     | 94828/225000 [171:07:49<291:04:33,  8.05s/it]
 42%|████▏     | 94829/225000 [171:07:54<262:00:38,  7.25s/it]
 42%|████▏     | 94830/225000 [171:08:05<295:05:50,  8.16s/it]
 42%|████▏     | 94831/225000 [171:08:14<303:46:48,  8.40s/it]
 42%|████▏     | 94832/225000 [171:08:17<251:35:38,  6.96s/it]
 42%|████▏     | 94833/225000 [171:08:27<278:55:54,  7.71s/it]
 42%|████▏     | 94834/225000 [171:08:39<331:12:52,  9.16s/it]
 42%|████▏     | 94835/225000 [171:08:51<358:08:21,  9.91s/it]
 42%|████▏     | 94836/225000 [171:09:01<363:39:40, 10.06s/it]
 42%|████▏     | 94837/225000 [171:09:10<345:38:38,  9.56s/it]
 42%|████▏     | 94838/225000 [171:09:17<321:21:47,  8.89s/it]
 42%|████▏     | 94839/225000 [171:09:24<299:08:11,  8.27s/it]
 42%|████▏     | 94840/225000 [171:09:36<340:53:50,  9.43s/it]
 42%|████▏     | 94841/225000 [171:09:46<347:43:40,  9.62s/it]
 42%|████▏     | 94842/225000 [171:09:50<287:32:35,  7.95s/it]
 42%|████▏     | 94843/225000 [171:09:57<280:47:18,  7.77s/it]
 42%|████▏     | 94844/225000 [171:10:05<280:51:56,  7.77s/it]
 42%|████▏     | 94845/225000 [171:10:18<340:57:24,  9.43s/it]
 42%|████▏     | 94846/225000 [171:10:26<324:36:10,  8.98s/it]
 42%|████▏     | 94847/225000 [171:10:32<291:11:02,  8.05s/it]
 42%|████▏     | 94848/225000 [171:10:37<250:36:36,  6.93s/it]
 42%|████▏     | 94849/225000 [171:10:41<227:18:07,  6.29s/it]
 42%|████▏     | 94850/225000 [171:10:46<204:34:31,  5.66s/it]
                                                              
{'loss': '0.5614', 'grad_norm': '3.25', 'learning_rate': '3.314e-05', 'epoch': '0.4216', 'num_input_tokens_seen': 280275752, 'train_runtime': '6.162e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94850/225000 [171:10:46<204:34:31,  5.66s/it]
 42%|████▏     | 94851/225000 [171:10:54<232:33:23,  6.43s/it]
 42%|████▏     | 94852/225000 [171:11:06<290:20:06,  8.03s/it]
 42%|████▏     | 94853/225000 [171:11:18<335:35:36,  9.28s/it]
 42%|████▏     | 94854/225000 [171:11:32<394:36:26, 10.92s/it]
 42%|████▏     | 94855/225000 [171:11:36<309:46:04,  8.57s/it]
 42%|████▏     | 94856/225000 [171:11:42<289:04:08,  8.00s/it]
 42%|████▏     | 94857/225000 [171:11:51<295:24:38,  8.17s/it]
 42%|████▏     | 94858/225000 [171:11:56<265:31:09,  7.34s/it]
 42%|████▏     | 94859/225000 [171:12:04<273:29:45,  7.57s/it]
 42%|████▏     | 94860/225000 [171:12:08<236:07:32,  6.53s/it]
 42%|████▏     | 94861/225000 [171:12:24<338:57:28,  9.38s/it]
 42%|████▏     | 94862/225000 [171:12:29<287:51:59,  7.96s/it]
 42%|████▏     | 94863/225000 [171:12:42<342:46:12,  9.48s/it]
 42%|████▏     | 94864/225000 [171:12:50<330:14:34,  9.14s/it]
 42%|████▏     | 94865/225000 [171:12:58<311:43:37,  8.62s/it]
 42%|████▏     | 94866/225000 [171:13:07<318:15:16,  8.80s/it]
 42%|████▏     | 94867/225000 [171:13:13<281:33:00,  7.79s/it]
 42%|████▏     | 94868/225000 [171:13:25<328:36:54,  9.09s/it]
 42%|████▏     | 94869/225000 [171:13:30<284:35:19,  7.87s/it]
 42%|████▏     | 94870/225000 [171:13:44<351:58:03,  9.74s/it]
 42%|████▏     | 94871/225000 [171:13:47<282:20:43,  7.81s/it]
 42%|████▏     | 94872/225000 [171:13:56<294:09:25,  8.14s/it]
 42%|████▏     | 94873/225000 [171:14:06<313:16:52,  8.67s/it]
 42%|████▏     | 94874/225000 [171:14:13<295:15:56,  8.17s/it]
 42%|████▏     | 94875/225000 [171:14:25<337:48:15,  9.35s/it]
 42%|████▏     | 94876/225000 [171:14:38<372:37:26, 10.31s/it]
 42%|████▏     | 94877/225000 [171:14:44<331:44:54,  9.18s/it]
 42%|████▏     | 94878/225000 [171:14:48<270:37:36,  7.49s/it]
 42%|████▏     | 94879/225000 [171:15:05<377:15:57, 10.44s/it]
 42%|████▏     | 94880/225000 [171:15:10<320:12:37,  8.86s/it]
 42%|████▏     | 94881/225000 [171:15:20<328:50:13,  9.10s/it]
 42%|████▏     | 94882/225000 [171:15:36<410:30:43, 11.36s/it]
 42%|████▏     | 94883/225000 [171:15:41<334:52:50,  9.27s/it]
 42%|████▏     | 94884/225000 [171:15:47<305:50:55,  8.46s/it]
 42%|████▏     | 94885/225000 [171:15:56<304:24:03,  8.42s/it]
 42%|████▏     | 94886/225000 [171:16:00<257:56:16,  7.14s/it]
 42%|████▏     | 94887/225000 [171:16:13<328:06:09,  9.08s/it]
 42%|████▏     | 94888/225000 [171:16:24<348:45:08,  9.65s/it]
 42%|████▏     | 94889/225000 [171:16:27<276:34:12,  7.65s/it]
 42%|████▏     | 94890/225000 [171:16:37<297:42:40,  8.24s/it]
 42%|████▏     | 94891/225000 [171:16:49<338:50:04,  9.38s/it]
 42%|████▏     | 94892/225000 [171:16:54<290:18:14,  8.03s/it]
 42%|████▏     | 94893/225000 [171:17:03<300:14:18,  8.31s/it]
 42%|████▏     | 94894/225000 [171:17:09<280:42:01,  7.77s/it]
 42%|████▏     | 94895/225000 [171:17:17<276:00:02,  7.64s/it]
 42%|████▏     | 94896/225000 [171:17:26<290:02:10,  8.03s/it]
 42%|████▏     | 94897/225000 [171:17:29<243:58:08,  6.75s/it]
 42%|████▏     | 94898/225000 [171:17:39<275:51:28,  7.63s/it]
 42%|████▏     | 94899/225000 [171:17:45<260:24:00,  7.21s/it]
 42%|████▏     | 94900/225000 [171:17:54<276:18:34,  7.65s/it]
                                                              
{'loss': '0.6272', 'grad_norm': '2.445', 'learning_rate': '3.312e-05', 'epoch': '0.4218', 'num_input_tokens_seen': 280483824, 'train_runtime': '6.167e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94900/225000 [171:17:54<276:18:34,  7.65s/it]
 42%|████▏     | 94901/225000 [171:17:58<241:26:20,  6.68s/it]
 42%|████▏     | 94902/225000 [171:18:11<305:38:52,  8.46s/it]
 42%|████▏     | 94903/225000 [171:18:14<247:47:12,  6.86s/it]
 42%|████▏     | 94904/225000 [171:18:25<287:15:49,  7.95s/it]
 42%|████▏     | 94905/225000 [171:18:33<293:29:13,  8.12s/it]
 42%|████▏     | 94906/225000 [171:18:38<252:53:47,  7.00s/it]
 42%|████▏     | 94907/225000 [171:18:43<235:51:22,  6.53s/it]
 42%|████▏     | 94908/225000 [171:18:58<330:21:42,  9.14s/it]
 42%|████▏     | 94909/225000 [171:19:02<271:40:38,  7.52s/it]
 42%|████▏     | 94910/225000 [171:19:18<360:10:47,  9.97s/it]
 42%|████▏     | 94911/225000 [171:19:21<290:35:57,  8.04s/it]
 42%|████▏     | 94912/225000 [171:19:36<364:02:03, 10.07s/it]
 42%|████▏     | 94913/225000 [171:19:40<295:32:29,  8.18s/it]
 42%|████▏     | 94914/225000 [171:19:52<338:46:21,  9.38s/it]
 42%|████▏     | 94915/225000 [171:19:57<286:28:39,  7.93s/it]
 42%|████▏     | 94916/225000 [171:20:02<259:04:46,  7.17s/it]
 42%|████▏     | 94917/225000 [171:20:15<326:39:59,  9.04s/it]
 42%|████▏     | 94918/225000 [171:20:24<326:49:18,  9.04s/it]
 42%|████▏     | 94919/225000 [171:20:35<341:53:31,  9.46s/it]
 42%|████▏     | 94920/225000 [171:20:48<377:46:30, 10.46s/it]
 42%|████▏     | 94921/225000 [171:20:55<346:01:42,  9.58s/it]
 42%|████▏     | 94922/225000 [171:20:59<282:31:10,  7.82s/it]
 42%|████▏     | 94923/225000 [171:21:13<354:53:42,  9.82s/it]
 42%|████▏     | 94924/225000 [171:21:25<376:19:15, 10.42s/it]
 42%|████▏     | 94925/225000 [171:21:30<313:13:30,  8.67s/it]
 42%|████▏     | 94926/225000 [171:21:39<316:20:45,  8.76s/it]
 42%|████▏     | 94927/225000 [171:21:51<353:22:38,  9.78s/it]
 42%|████▏     | 94928/225000 [171:21:57<315:02:32,  8.72s/it]
 42%|████▏     | 94929/225000 [171:22:08<342:26:02,  9.48s/it]
 42%|████▏     | 94930/225000 [171:22:14<295:51:03,  8.19s/it]
 42%|████▏     | 94931/225000 [171:22:22<295:05:02,  8.17s/it]
 42%|████▏     | 94932/225000 [171:22:30<298:05:37,  8.25s/it]
 42%|████▏     | 94933/225000 [171:22:34<251:41:50,  6.97s/it]
 42%|████▏     | 94934/225000 [171:22:44<282:54:37,  7.83s/it]
 42%|████▏     | 94935/225000 [171:22:50<268:22:27,  7.43s/it]
 42%|████▏     | 94936/225000 [171:23:01<298:11:18,  8.25s/it]
 42%|████▏     | 94937/225000 [171:23:12<330:32:03,  9.15s/it]
 42%|████▏     | 94938/225000 [171:23:21<335:23:10,  9.28s/it]
 42%|████▏     | 94939/225000 [171:23:32<345:46:17,  9.57s/it]
 42%|████▏     | 94940/225000 [171:23:44<377:23:18, 10.45s/it]
 42%|████▏     | 94941/225000 [171:23:49<315:14:44,  8.73s/it]
 42%|████▏     | 94942/225000 [171:23:56<300:02:18,  8.31s/it]
 42%|████▏     | 94943/225000 [171:24:02<276:28:40,  7.65s/it]
 42%|████▏     | 94944/225000 [171:24:08<256:38:09,  7.10s/it]
 42%|████▏     | 94945/225000 [171:24:21<320:47:59,  8.88s/it]
 42%|████▏     | 94946/225000 [171:24:35<375:37:22, 10.40s/it]
 42%|████▏     | 94947/225000 [171:24:42<332:19:59,  9.20s/it]
 42%|████▏     | 94948/225000 [171:24:54<368:16:02, 10.19s/it]
 42%|████▏     | 94949/225000 [171:24:58<303:40:43,  8.41s/it]
 42%|████▏     | 94950/225000 [171:25:09<331:04:15,  9.16s/it]
                                                              
{'loss': '0.6433', 'grad_norm': '1.885', 'learning_rate': '3.311e-05', 'epoch': '0.422', 'num_input_tokens_seen': 280690288, 'train_runtime': '6.171e+05', 'train_tokens_per_second': '454.8'}

 42%|████▏     | 94950/225000 [171:25:09<331:04:15,  9.16s/it]
 42%|████▏     | 94951/225000 [171:25:19<336:50:03,  9.32s/it]
 42%|████▏     | 94952/225000 [171:25:28<336:30:29,  9.32s/it]
 42%|████▏     | 94953/225000 [171:25:40<365:33:42, 10.12s/it]
 42%|████▏     | 94954/225000 [171:25:46<321:35:37,  8.90s/it]
 42%|████▏     | 94955/225000 [171:25:57<345:08:27,  9.55s/it]
 42%|████▏     | 94956/225000 [171:26:07<349:48:56,  9.68s/it]
 42%|████▏     | 94957/225000 [171:26:16<338:38:46,  9.37s/it]
 42%|████▏     | 94958/225000 [171:26:26<346:11:09,  9.58s/it]
 42%|████▏     | 94959/225000 [171:26:35<335:17:35,  9.28s/it]
 42%|████▏     | 94960/225000 [171:26:39<277:29:46,  7.68s/it]
 42%|████▏     | 94961/225000 [171:26:55<367:57:18, 10.19s/it]
 42%|████▏     | 94962/225000 [171:26:59<300:17:24,  8.31s/it]
 42%|████▏     | 94963/225000 [171:27:06<286:41:33,  7.94s/it]
 42%|████▏     | 94964/225000 [171:27:10<246:58:59,  6.84s/it]
 42%|████▏     | 94965/225000 [171:27:18<262:57:09,  7.28s/it]
 42%|████▏     | 94966/225000 [171:27:23<231:13:55,  6.40s/it]
 42%|████▏     | 94967/225000 [171:27:30<242:06:51,  6.70s/it]
 42%|████▏     | 94968/225000 [171:27:38<255:20:50,  7.07s/it]
 42%|████▏     | 94969/225000 [171:27:49<294:47:02,  8.16s/it]
 42%|████▏     | 94970/225000 [171:27:59<323:55:12,  8.97s/it]
 42%|████▏     | 94971/225000 [171:28:06<297:07:35,  8.23s/it]
 42%|████▏     | 94972/225000 [171:28:14<299:18:22,  8.29s/it]
 42%|████▏     | 94973/225000 [171:28:31<394:52:58, 10.93s/it]
 42%|████▏     | 94974/225000 [171:28:40<368:39:41, 10.21s/it]
 42%|████▏     | 94975/225000 [171:28:57<438:11:47, 12.13s/it]
 42%|████▏     | 94976/225000 [171:29:00<348:26:12,  9.65s/it]
 42%|████▏     | 94977/225000 [171:29:13<381:43:07, 10.57s/it]
 42%|████▏     | 94978/225000 [171:29:19<328:02:31,  9.08s/it]
 42%|████▏     | 94979/225000 [171:29:28<329:37:25,  9.13s/it]
 42%|████▏     | 94980/225000 [171:29:33<287:03:34,  7.95s/it]
 42%|████▏     | 94981/225000 [171:29:44<312:58:56,  8.67s/it]
 42%|████▏     | 94982/225000 [171:29:50<287:58:05,  7.97s/it]
 42%|████▏     | 94983/225000 [171:29:53<238:58:53,  6.62s/it]
 42%|████▏     | 94984/225000 [171:30:04<285:18:17,  7.90s/it]
 42%|████▏     | 94985/225000 [171:30:13<289:59:01,  8.03s/it]
 42%|████▏     | 94986/225000 [171:30:24<325:50:13,  9.02s/it]
 42%|████▏     | 94987/225000 [171:30:28<269:05:36,  7.45s/it]
 42%|████▏     | 94988/225000 [171:30:39<314:35:47,  8.71s/it]
 42%|████▏     | 94989/225000 [171:30:43<261:19:10,  7.24s/it]
 42%|████▏     | 94990/225000 [171:30:49<251:45:49,  6.97s/it]
 42%|████▏     | 94991/225000 [171:30:56<243:15:24,  6.74s/it]
 42%|████▏     | 94992/225000 [171:31:10<325:45:10,  9.02s/it]
 42%|████▏     | 94993/225000 [171:31:13<260:24:01,  7.21s/it]
 42%|████▏     | 94994/225000 [171:31:24<298:58:19,  8.28s/it]
 42%|████▏     | 94995/225000 [171:31:30<275:51:52,  7.64s/it]
 42%|████▏     | 94996/225000 [171:31:36<253:44:51,  7.03s/it]
 42%|████▏     | 94997/225000 [171:31:45<277:34:52,  7.69s/it]
 42%|████▏     | 94998/225000 [171:31:57<326:52:52,  9.05s/it]
 42%|████▏     | 94999/225000 [171:32:13<397:33:18, 11.01s/it]
 42%|████▏     | 95000/225000 [171:32:18<334:48:46,  9.27s/it]
                                                              
{'loss': '0.5894', 'grad_norm': '1.488', 'learning_rate': '3.309e-05', 'epoch': '0.4222', 'num_input_tokens_seen': 280891856, 'train_runtime': '6.175e+05', 'train_tokens_per_second': '454.9'}

 42%|████▏     | 95000/225000 [171:32:18<334:48:46,  9.27s/it]
 42%|████▏     | 95001/225000 [171:32:23<288:12:12,  7.98s/it]
 42%|████▏     | 95002/225000 [171:32:30<284:00:55,  7.87s/it]
 42%|████▏     | 95003/225000 [171:32:35<249:27:27,  6.91s/it]
 42%|████▏     | 95004/225000 [171:32:40<232:54:40,  6.45s/it]
 42%|████▏     | 95005/225000 [171:32:53<303:58:21,  8.42s/it]
 42%|████▏     | 95006/225000 [171:33:05<339:34:03,  9.40s/it]
 42%|████▏     | 95007/225000 [171:33:13<322:53:48,  8.94s/it]
 42%|████▏     | 95008/225000 [171:33:17<268:01:51,  7.42s/it]
 42%|████▏     | 95009/225000 [171:33:25<273:13:11,  7.57s/it]
 42%|████▏     | 95010/225000 [171:33:30<247:50:03,  6.86s/it]
 42%|████▏     | 95011/225000 [171:33:36<240:48:13,  6.67s/it]
 42%|████▏     | 95012/225000 [171:33:46<271:38:33,  7.52s/it]
 42%|████▏     | 95013/225000 [171:33:58<319:01:56,  8.84s/it]
 42%|████▏     | 95014/225000 [171:34:03<284:41:32,  7.88s/it]
 42%|████▏     | 95015/225000 [171:34:15<326:34:32,  9.04s/it]
 42%|████▏     | 95016/225000 [171:34:28<370:39:19, 10.27s/it]
 42%|████▏     | 95017/225000 [171:34:32<299:20:49,  8.29s/it]
 42%|████▏     | 95018/225000 [171:34:43<333:31:35,  9.24s/it]
 42%|████▏     | 95019/225000 [171:34:54<347:00:44,  9.61s/it]
 42%|████▏     | 95020/225000 [171:35:06<377:17:25, 10.45s/it]
 42%|████▏     | 95021/225000 [171:35:10<302:46:37,  8.39s/it]
 42%|████▏     | 95022/225000 [171:35:24<361:02:05, 10.00s/it]
 42%|████▏     | 95023/225000 [171:35:31<338:03:15,  9.36s/it]
 42%|████▏     | 95024/225000 [171:35:41<343:14:54,  9.51s/it]
 42%|████▏     | 95025/225000 [171:35:52<361:49:40, 10.02s/it]
 42%|████▏     | 95026/225000 [171:36:08<418:33:39, 11.59s/it]
 42%|████▏     | 95027/225000 [171:36:15<367:11:39, 10.17s/it]
 42%|████▏     | 95028/225000 [171:36:27<390:07:04, 10.81s/it]
 42%|████▏     | 95029/225000 [171:36:39<407:28:52, 11.29s/it]
 42%|████▏     | 95030/225000 [171:36:50<405:13:10, 11.22s/it]
 42%|████▏     | 95031/225000 [171:37:02<414:31:51, 11.48s/it]
 42%|████▏     | 95032/225000 [171:37:12<388:56:32, 10.77s/it]
 42%|████▏     | 95033/225000 [171:37:17<332:53:50,  9.22s/it]
 42%|████▏     | 95034/225000 [171:37:20<265:47:00,  7.36s/it]
 42%|████▏     | 95035/225000 [171:37:34<340:11:14,  9.42s/it]
 42%|████▏     | 95036/225000 [171:37:49<394:32:35, 10.93s/it]
 42%|████▏     | 95037/225000 [171:37:59<391:17:36, 10.84s/it]
 42%|████▏     | 95038/225000 [171:38:14<436:29:38, 12.09s/it]
 42%|████▏     | 95039/225000 [171:38:33<510:24:01, 14.14s/it]
 42%|████▏     | 95040/225000 [171:38:38<411:16:40, 11.39s/it]
 42%|████▏     | 95041/225000 [171:38:48<393:44:25, 10.91s/it]
 42%|████▏     | 95042/225000 [171:38:53<326:14:49,  9.04s/it]
 42%|████▏     | 95043/225000 [171:39:06<368:13:24, 10.20s/it]
 42%|████▏     | 95044/225000 [171:39:11<314:14:48,  8.71s/it]
 42%|████▏     | 95045/225000 [171:39:15<267:42:09,  7.42s/it]
 42%|████▏     | 95046/225000 [171:39:21<250:22:30,  6.94s/it]
 42%|████▏     | 95047/225000 [171:39:28<246:48:14,  6.84s/it]
 42%|████▏     | 95048/225000 [171:39:35<250:17:19,  6.93s/it]
 42%|████▏     | 95049/225000 [171:39:42<248:24:14,  6.88s/it]
 42%|████▏     | 95050/225000 [171:39:49<253:41:11,  7.03s/it]
                                                              
{'loss': '0.6192', 'grad_norm': '2.083', 'learning_rate': '3.307e-05', 'epoch': '0.4224', 'num_input_tokens_seen': 281109624, 'train_runtime': '6.18e+05', 'train_tokens_per_second': '454.9'}

 42%|████▏     | 95050/225000 [171:39:49<253:41:11,  7.03s/it]
 42%|████▏     | 95051/225000 [171:39:57<258:16:28,  7.16s/it]
 42%|████▏     | 95052/225000 [171:40:06<283:48:23,  7.86s/it]
 42%|████▏     | 95053/225000 [171:40:23<383:34:39, 10.63s/it]
 42%|████▏     | 95054/225000 [171:40:26<304:46:56,  8.44s/it]
 42%|████▏     | 95055/225000 [171:40:31<265:32:10,  7.36s/it]
 42%|████▏     | 95056/225000 [171:40:43<309:22:52,  8.57s/it]
 42%|████▏     | 95057/225000 [171:40:49<280:35:35,  7.77s/it]
 42%|████▏     | 95058/225000 [171:40:59<305:10:35,  8.45s/it]
 42%|████▏     | 95059/225000 [171:41:04<271:03:45,  7.51s/it]
 42%|████▏     | 95060/225000 [171:41:14<299:42:14,  8.30s/it]
 42%|████▏     | 95061/225000 [171:41:18<255:31:07,  7.08s/it]
 42%|████▏     | 95062/225000 [171:41:24<242:54:26,  6.73s/it]
 42%|████▏     | 95063/225000 [171:41:38<319:12:33,  8.84s/it]
 42%|████▏     | 95064/225000 [171:41:41<259:38:14,  7.19s/it]
 42%|████▏     | 95065/225000 [171:41:51<289:05:31,  8.01s/it]
 42%|████▏     | 95066/225000 [171:42:04<336:54:42,  9.33s/it]
 42%|████▏     | 95067/225000 [171:42:12<325:14:49,  9.01s/it]
 42%|████▏     | 95068/225000 [171:42:23<341:52:05,  9.47s/it]
 42%|████▏     | 95069/225000 [171:42:32<340:15:13,  9.43s/it]
 42%|████▏     | 95070/225000 [171:42:36<281:43:50,  7.81s/it]
 42%|████▏     | 95071/225000 [171:42:44<282:06:24,  7.82s/it]
 42%|████▏     | 95072/225000 [171:42:56<334:43:03,  9.27s/it]
 42%|████▏     | 95073/225000 [171:43:07<351:45:53,  9.75s/it]
 42%|████▏     | 95074/225000 [171:43:24<425:09:37, 11.78s/it]
 42%|████▏     | 95075/225000 [171:43:26<326:41:56,  9.05s/it]
 42%|████▏     | 95076/225000 [171:43:39<366:16:33, 10.15s/it]
 42%|████▏     | 95077/225000 [171:43:50<372:04:56, 10.31s/it]
 42%|████▏     | 95078/225000 [171:43:55<311:16:42,  8.63s/it]
 42%|████▏     | 95079/225000 [171:44:05<335:39:34,  9.30s/it]
 42%|████▏     | 95080/225000 [171:44:18<366:06:25, 10.14s/it]
 42%|████▏     | 95081/225000 [171:44:27<361:55:30, 10.03s/it]
 42%|████▏     | 95082/225000 [171:44:32<303:27:48,  8.41s/it]
 42%|████▏     | 95083/225000 [171:44:36<252:04:14,  6.98s/it]
 42%|████▏     | 95084/225000 [171:44:39<213:01:47,  5.90s/it]
 42%|████▏     | 95085/225000 [171:45:00<379:13:13, 10.51s/it]
 42%|████▏     | 95086/225000 [171:45:06<325:23:38,  9.02s/it]
 42%|████▏     | 95087/225000 [171:45:22<406:45:14, 11.27s/it]
 42%|████▏     | 95088/225000 [171:45:31<381:00:22, 10.56s/it]
 42%|████▏     | 95089/225000 [171:45:37<334:21:55,  9.27s/it]
 42%|████▏     | 95090/225000 [171:45:41<276:52:09,  7.67s/it]
 42%|████▏     | 95091/225000 [171:45:48<266:32:11,  7.39s/it]
 42%|████▏     | 95092/225000 [171:45:52<231:28:41,  6.41s/it]
 42%|████▏     | 95093/225000 [171:46:06<315:35:42,  8.75s/it]
 42%|████▏     | 95094/225000 [171:46:13<289:03:10,  8.01s/it]
 42%|████▏     | 95095/225000 [171:46:23<315:58:22,  8.76s/it]
 42%|████▏     | 95096/225000 [171:46:36<355:31:40,  9.85s/it]
 42%|████▏     | 95097/225000 [171:46:43<329:20:33,  9.13s/it]
 42%|████▏     | 95098/225000 [171:47:06<481:45:00, 13.35s/it]
 42%|████▏     | 95099/225000 [171:47:15<431:54:23, 11.97s/it]
 42%|████▏     | 95100/225000 [171:47:22<377:18:04, 10.46s/it]
                                                              
{'loss': '0.644', 'grad_norm': '1.706', 'learning_rate': '3.305e-05', 'epoch': '0.4227', 'num_input_tokens_seen': 281324264, 'train_runtime': '6.184e+05', 'train_tokens_per_second': '454.9'}

 42%|████▏     | 95100/225000 [171:47:22<377:18:04, 10.46s/it]
 42%|████▏     | 95101/225000 [171:47:34<397:33:19, 11.02s/it]
 42%|████▏     | 95102/225000 [171:47:43<371:11:45, 10.29s/it]
 42%|████▏     | 95103/225000 [171:47:50<334:53:56,  9.28s/it]
 42%|████▏     | 95104/225000 [171:47:57<313:23:29,  8.69s/it]
 42%|████▏     | 95105/225000 [171:48:10<355:15:16,  9.85s/it]
 42%|████▏     | 95106/225000 [171:48:25<418:34:35, 11.60s/it]
 42%|████▏     | 95107/225000 [171:48:35<397:40:23, 11.02s/it]
 42%|████▏     | 95108/225000 [171:48:47<405:35:11, 11.24s/it]
 42%|████▏     | 95109/225000 [171:48:58<403:48:50, 11.19s/it]
 42%|████▏     | 95110/225000 [171:49:04<345:29:23,  9.58s/it]
 42%|████▏     | 95111/225000 [171:49:17<386:57:40, 10.73s/it]
 42%|████▏     | 95112/225000 [171:49:22<322:10:56,  8.93s/it]
 42%|████▏     | 95113/225000 [171:49:26<269:33:58,  7.47s/it]
 42%|████▏     | 95114/225000 [171:49:44<381:52:32, 10.58s/it]
 42%|████▏     | 95115/225000 [171:49:52<359:18:06,  9.96s/it]
 42%|████▏     | 95116/225000 [171:50:00<333:01:13,  9.23s/it]
 42%|████▏     | 95117/225000 [171:50:05<293:20:27,  8.13s/it]
 42%|████▏     | 95118/225000 [171:50:10<254:55:11,  7.07s/it]
 42%|████▏     | 95119/225000 [171:50:20<289:41:08,  8.03s/it]
 42%|████▏     | 95120/225000 [171:50:33<338:14:44,  9.38s/it]
 42%|████▏     | 95121/225000 [171:50:43<343:49:47,  9.53s/it]
 42%|████▏     | 95122/225000 [171:50:53<351:47:32,  9.75s/it]
 42%|████▏     | 95123/225000 [171:50:59<315:49:24,  8.75s/it]
 42%|████▏     | 95124/225000 [171:51:04<271:16:56,  7.52s/it]
 42%|████▏     | 95125/225000 [171:51:12<283:01:21,  7.85s/it]
 42%|████▏     | 95126/225000 [171:51:25<334:32:48,  9.27s/it]
 42%|████▏     | 95127/225000 [171:51:29<274:36:37,  7.61s/it]
 42%|████▏     | 95128/225000 [171:51:40<317:06:05,  8.79s/it]
 42%|████▏     | 95129/225000 [171:51:54<371:59:28, 10.31s/it]
 42%|████▏     | 95130/225000 [171:51:57<294:40:10,  8.17s/it]
 42%|████▏     | 95131/225000 [171:52:03<270:26:45,  7.50s/it]
 42%|████▏     | 95132/225000 [171:52:15<315:19:30,  8.74s/it]
 42%|████▏     | 95133/225000 [171:52:23<311:56:13,  8.65s/it]
 42%|████▏     | 95134/225000 [171:52:35<343:59:00,  9.54s/it]
 42%|████▏     | 95135/225000 [171:52:38<275:15:01,  7.63s/it]
 42%|████▏     | 95136/225000 [171:52:48<299:20:26,  8.30s/it]
 42%|████▏     | 95137/225000 [171:52:52<253:53:21,  7.04s/it]
 42%|████▏     | 95138/225000 [171:53:00<260:09:49,  7.21s/it]
 42%|████▏     | 95139/225000 [171:53:14<332:20:20,  9.21s/it]
 42%|████▏     | 95140/225000 [171:53:23<336:20:20,  9.32s/it]
 42%|████▏     | 95141/225000 [171:53:34<353:48:29,  9.81s/it]
 42%|████▏     | 95142/225000 [171:53:41<326:52:35,  9.06s/it]
 42%|████▏     | 95143/225000 [171:53:48<303:06:09,  8.40s/it]
 42%|████▏     | 95144/225000 [171:53:57<310:36:12,  8.61s/it]
 42%|████▏     | 95145/225000 [171:54:04<286:45:46,  7.95s/it]
 42%|████▏     | 95146/225000 [171:54:08<248:35:33,  6.89s/it]
 42%|████▏     | 95147/225000 [171:54:18<278:44:29,  7.73s/it]
 42%|████▏     | 95148/225000 [171:54:31<339:58:08,  9.43s/it]
 42%|████▏     | 95149/225000 [171:54:37<302:55:04,  8.40s/it]
 42%|████▏     | 95150/225000 [171:54:46<305:55:49,  8.48s/it]
                                                              
{'loss': '0.6677', 'grad_norm': '2.094', 'learning_rate': '3.304e-05', 'epoch': '0.4229', 'num_input_tokens_seen': 281551552, 'train_runtime': '6.189e+05', 'train_tokens_per_second': '454.9'}

 42%|████▏     | 95150/225000 [171:54:46<305:55:49,  8.48s/it]
 42%|████▏     | 95151/225000 [171:54:57<330:24:30,  9.16s/it]
 42%|████▏     | 95152/225000 [171:55:02<289:15:56,  8.02s/it]
 42%|████▏     | 95153/225000 [171:55:15<339:40:02,  9.42s/it]
 42%|████▏     | 95154/225000 [171:55:22<320:37:16,  8.89s/it]
 42%|████▏     | 95155/225000 [171:55:32<323:58:15,  8.98s/it]
 42%|████▏     | 95156/225000 [171:55:46<384:55:41, 10.67s/it]
 42%|████▏     | 95157/225000 [171:55:55<360:11:12,  9.99s/it]
 42%|████▏     | 95158/225000 [171:56:01<316:22:10,  8.77s/it]
 42%|████▏     | 95159/225000 [171:56:04<255:13:12,  7.08s/it]
 42%|████▏     | 95160/225000 [171:56:16<307:49:52,  8.54s/it]
 42%|████▏     | 95161/225000 [171:56:27<332:59:31,  9.23s/it]
 42%|████▏     | 95162/225000 [171:56:37<351:50:11,  9.76s/it]
 42%|████▏     | 95163/225000 [171:56:46<333:44:20,  9.25s/it]
 42%|████▏     | 95164/225000 [171:56:54<328:40:22,  9.11s/it]
 42%|████▏     | 95165/225000 [171:57:06<358:16:54,  9.93s/it]
 42%|████▏     | 95166/225000 [171:57:16<357:51:52,  9.92s/it]
 42%|████▏     | 95167/225000 [171:57:29<394:44:25, 10.95s/it]
 42%|████▏     | 95168/225000 [171:57:37<361:06:19, 10.01s/it]
 42%|████▏     | 95169/225000 [171:57:47<354:42:56,  9.84s/it]
 42%|████▏     | 95170/225000 [171:57:57<355:50:37,  9.87s/it]
 42%|████▏     | 95171/225000 [171:58:10<393:50:43, 10.92s/it]
 42%|████▏     | 95172/225000 [171:58:15<334:16:30,  9.27s/it]
 42%|████▏     | 95173/225000 [171:58:21<296:30:34,  8.22s/it]
 42%|████▏     | 95174/225000 [171:58:28<279:42:38,  7.76s/it]
 42%|████▏     | 95175/225000 [171:58:41<342:57:28,  9.51s/it]
 42%|████▏     | 95176/225000 [171:58:53<362:12:44, 10.04s/it]
 42%|████▏     | 95177/225000 [171:59:17<519:47:07, 14.41s/it]
 42%|████▏     | 95178/225000 [171:59:26<453:37:36, 12.58s/it]
 42%|████▏     | 95179/225000 [171:59:34<404:45:59, 11.22s/it]
 42%|████▏     | 95180/225000 [171:59:41<363:36:45, 10.08s/it]
 42%|████▏     | 95181/225000 [171:59:50<348:45:48,  9.67s/it]
 42%|████▏     | 95182/225000 [172:00:04<393:45:17, 10.92s/it]
 42%|████▏     | 95183/225000 [172:00:08<326:58:39,  9.07s/it]
 42%|████▏     | 95184/225000 [172:00:21<369:02:34, 10.23s/it]
 42%|████▏     | 95185/225000 [172:00:30<349:36:38,  9.70s/it]
 42%|████▏     | 95186/225000 [172:00:36<315:13:47,  8.74s/it]
 42%|████▏     | 95187/225000 [172:00:44<299:52:33,  8.32s/it]
 42%|████▏     | 95188/225000 [172:00:57<350:44:23,  9.73s/it]
 42%|████▏     | 95189/225000 [172:01:05<333:52:26,  9.26s/it]
 42%|████▏     | 95190/225000 [172:01:16<350:13:41,  9.71s/it]
 42%|████▏     | 95191/225000 [172:01:22<312:07:53,  8.66s/it]
 42%|████▏     | 95192/225000 [172:01:37<383:16:56, 10.63s/it]
 42%|████▏     | 95193/225000 [172:01:42<324:24:45,  9.00s/it]
 42%|████▏     | 95194/225000 [172:01:47<280:26:58,  7.78s/it]
 42%|████▏     | 95195/225000 [172:02:05<386:02:12, 10.71s/it]
 42%|████▏     | 95196/225000 [172:02:12<344:47:03,  9.56s/it]
 42%|████▏     | 95197/225000 [172:02:22<353:40:19,  9.81s/it]
 42%|████▏     | 95198/225000 [172:02:31<341:20:58,  9.47s/it]
 42%|████▏     | 95199/225000 [172:02:39<331:43:47,  9.20s/it]
 42%|████▏     | 95200/225000 [172:02:50<347:40:28,  9.64s/it]
                                                              
{'loss': '0.7288', 'grad_norm': '1.965', 'learning_rate': '3.302e-05', 'epoch': '0.4231', 'num_input_tokens_seen': 281789944, 'train_runtime': '6.194e+05', 'train_tokens_per_second': '455'}

 42%|████▏     | 95200/225000 [172:02:50<347:40:28,  9.64s/it]
 42%|████▏     | 95201/225000 [172:03:00<354:35:06,  9.83s/it]
 42%|████▏     | 95202/225000 [172:03:05<302:46:45,  8.40s/it]
 42%|████▏     | 95203/225000 [172:03:15<314:43:16,  8.73s/it]
 42%|████▏     | 95204/225000 [172:03:27<356:29:29,  9.89s/it]
 42%|████▏     | 95205/225000 [172:03:34<320:50:54,  8.90s/it]
 42%|████▏     | 95206/225000 [172:03:39<282:57:53,  7.85s/it]
 42%|████▏     | 95207/225000 [172:03:52<337:39:27,  9.37s/it]
 42%|████▏     | 95208/225000 [172:03:59<307:53:55,  8.54s/it]
 42%|████▏     | 95209/225000 [172:04:05<284:29:24,  7.89s/it]
 42%|████▏     | 95210/225000 [172:04:14<292:37:16,  8.12s/it]
 42%|████▏     | 95211/225000 [172:04:28<355:16:55,  9.85s/it]
 42%|████▏     | 95212/225000 [172:04:39<370:36:28, 10.28s/it]
 42%|████▏     | 95213/225000 [172:04:49<367:27:48, 10.19s/it]
 42%|████▏     | 95214/225000 [172:04:56<327:39:00,  9.09s/it]
 42%|████▏     | 95215/225000 [172:05:07<347:46:09,  9.65s/it]
 42%|████▏     | 95216/225000 [172:05:20<385:19:53, 10.69s/it]
 42%|████▏     | 95217/225000 [172:05:30<378:35:06, 10.50s/it]
 42%|████▏     | 95218/225000 [172:05:47<450:04:25, 12.48s/it]
 42%|████▏     | 95219/225000 [172:06:00<458:09:37, 12.71s/it]
 42%|████▏     | 95220/225000 [172:06:10<431:01:20, 11.96s/it]
 42%|████▏     | 95221/225000 [172:06:20<403:48:12, 11.20s/it]
 42%|████▏     | 95222/225000 [172:06:30<394:07:36, 10.93s/it]
 42%|████▏     | 95223/225000 [172:06:43<419:05:01, 11.63s/it]
 42%|████▏     | 95224/225000 [172:06:48<347:33:15,  9.64s/it]
 42%|████▏     | 95225/225000 [172:06:57<336:40:55,  9.34s/it]
 42%|████▏     | 95226/225000 [172:07:09<365:43:55, 10.15s/it]
 42%|████▏     | 95227/225000 [172:07:26<438:44:12, 12.17s/it]
 42%|████▏     | 95228/225000 [172:07:44<503:54:48, 13.98s/it]
 42%|████▏     | 95229/225000 [172:07:52<444:00:18, 12.32s/it]
 42%|████▏     | 95230/225000 [172:08:05<451:25:20, 12.52s/it]
 42%|████▏     | 95231/225000 [172:08:19<461:54:43, 12.81s/it]
 42%|████▏     | 95232/225000 [172:08:30<438:06:15, 12.15s/it]
 42%|████▏     | 95233/225000 [172:08:38<400:12:25, 11.10s/it]
 42%|████▏     | 95234/225000 [172:08:49<401:23:56, 11.14s/it]
 42%|████▏     | 95235/225000 [172:09:06<461:37:11, 12.81s/it]
 42%|████▏     | 95236/225000 [172:09:17<438:45:40, 12.17s/it]
 42%|████▏     | 95237/225000 [172:09:20<342:47:39,  9.51s/it]
 42%|████▏     | 95238/225000 [172:09:27<319:29:12,  8.86s/it]
 42%|████▏     | 95239/225000 [172:09:38<336:32:18,  9.34s/it]
 42%|████▏     | 95240/225000 [172:09:52<392:09:37, 10.88s/it]
 42%|████▏     | 95241/225000 [172:09:59<348:49:42,  9.68s/it]
 42%|████▏     | 95242/225000 [172:10:11<371:53:27, 10.32s/it]
 42%|████▏     | 95243/225000 [172:10:30<463:41:18, 12.86s/it]
 42%|████▏     | 95244/225000 [172:10:44<481:10:18, 13.35s/it]
 42%|████▏     | 95245/225000 [172:10:54<436:12:49, 12.10s/it]
 42%|████▏     | 95246/225000 [172:10:57<347:16:26,  9.64s/it]
 42%|████▏     | 95247/225000 [172:11:14<422:14:36, 11.72s/it]
 42%|████▏     | 95248/225000 [172:11:21<372:31:15, 10.34s/it]
 42%|████▏     | 95249/225000 [172:11:30<361:36:04, 10.03s/it]
 42%|████▏     | 95250/225000 [172:11:41<367:50:49, 10.21s/it]
                                                              
{'loss': '0.7361', 'grad_norm': '1.869', 'learning_rate': '3.3e-05', 'epoch': '0.4233', 'num_input_tokens_seen': 282051728, 'train_runtime': '6.199e+05', 'train_tokens_per_second': '455'}

 42%|████▏     | 95250/225000 [172:11:41<367:50:49, 10.21s/it]
 42%|████▏     | 95251/225000 [172:11:49<340:00:28,  9.43s/it]
 42%|████▏     | 95252/225000 [172:12:05<409:56:06, 11.37s/it]
 42%|████▏     | 95253/225000 [172:12:09<337:41:38,  9.37s/it]
 42%|████▏     | 95254/225000 [172:12:17<315:54:41,  8.77s/it]
 42%|████▏     | 95255/225000 [172:12:22<274:42:53,  7.62s/it]
 42%|████▏     | 95256/225000 [172:12:33<315:36:34,  8.76s/it]
 42%|████▏     | 95257/225000 [172:12:37<261:20:20,  7.25s/it]
 42%|████▏     | 95258/225000 [172:12:55<378:24:32, 10.50s/it]
 42%|████▏     | 95259/225000 [172:13:03<349:10:45,  9.69s/it]
 42%|████▏     | 95260/225000 [172:13:15<373:11:12, 10.36s/it]
 42%|████▏     | 95261/225000 [172:13:22<346:32:19,  9.62s/it]
 42%|████▏     | 95262/225000 [172:13:31<330:17:51,  9.17s/it]
 42%|████▏     | 95263/225000 [172:13:41<346:23:21,  9.61s/it]
 42%|████▏     | 95264/225000 [172:13:45<283:00:48,  7.85s/it]
 42%|████▏     | 95265/225000 [172:13:59<346:52:33,  9.63s/it]
 42%|████▏     | 95266/225000 [172:14:13<395:58:38, 10.99s/it]
 42%|████▏     | 95267/225000 [172:14:24<401:34:43, 11.14s/it]
 42%|████▏     | 95268/225000 [172:14:31<352:40:42,  9.79s/it]
 42%|████▏     | 95269/225000 [172:14:38<327:37:18,  9.09s/it]
 42%|████▏     | 95270/225000 [172:14:55<405:05:33, 11.24s/it]
 42%|████▏     | 95271/225000 [172:15:07<421:04:34, 11.68s/it]
 42%|████▏     | 95272/225000 [172:15:18<407:08:00, 11.30s/it]
 42%|████▏     | 95273/225000 [172:15:29<407:13:20, 11.30s/it]
 42%|████▏     | 95274/225000 [172:15:40<400:09:19, 11.10s/it]
 42%|████▏     | 95275/225000 [172:15:45<339:26:25,  9.42s/it]
 42%|████▏     | 95276/225000 [172:15:55<340:56:43,  9.46s/it]
 42%|████▏     | 95277/225000 [172:16:10<403:42:09, 11.20s/it]
 42%|████▏     | 95278/225000 [172:16:19<376:02:36, 10.44s/it]
 42%|████▏     | 95279/225000 [172:16:31<393:59:58, 10.93s/it]
 42%|████▏     | 95280/225000 [172:16:42<395:25:29, 10.97s/it]
 42%|████▏     | 95281/225000 [172:16:46<319:41:30,  8.87s/it]
 42%|████▏     | 95282/225000 [172:16:52<286:06:03,  7.94s/it]
 42%|████▏     | 95283/225000 [172:17:09<383:54:58, 10.65s/it]
 42%|████▏     | 95284/225000 [172:17:19<386:09:46, 10.72s/it]
 42%|████▏     | 95285/225000 [172:17:35<437:28:35, 12.14s/it]
 42%|████▏     | 95286/225000 [172:17:47<434:08:43, 12.05s/it]
 42%|████▏     | 95287/225000 [172:18:02<471:16:52, 13.08s/it]
 42%|████▏     | 95288/225000 [172:18:13<445:59:40, 12.38s/it]
 42%|████▏     | 95289/225000 [172:18:20<392:22:32, 10.89s/it]
 42%|████▏     | 95290/225000 [172:18:39<480:41:41, 13.34s/it]
 42%|████▏     | 95291/225000 [172:18:49<437:04:55, 12.13s/it]
 42%|████▏     | 95292/225000 [172:18:59<417:34:39, 11.59s/it]
 42%|████▏     | 95293/225000 [172:19:20<516:29:42, 14.34s/it]
 42%|████▏     | 95294/225000 [172:19:27<440:19:54, 12.22s/it]
 42%|████▏     | 95295/225000 [172:19:44<493:29:45, 13.70s/it]
 42%|████▏     | 95296/225000 [172:19:52<433:56:11, 12.04s/it]
 42%|████▏     | 95297/225000 [172:20:01<400:45:11, 11.12s/it]
 42%|████▏     | 95298/225000 [172:20:19<466:32:38, 12.95s/it]
 42%|████▏     | 95299/225000 [172:20:23<371:23:15, 10.31s/it]
 42%|████▏     | 95300/225000 [172:20:37<413:58:04, 11.49s/it]
                                                              
{'loss': '0.7376', 'grad_norm': '2.67', 'learning_rate': '3.298e-05', 'epoch': '0.4236', 'num_input_tokens_seen': 282314992, 'train_runtime': '6.204e+05', 'train_tokens_per_second': '455'}

 42%|████▏     | 95300/225000 [172:20:37<413:58:04, 11.49s/it]
 42%|████▏     | 95301/225000 [172:20:44<365:28:56, 10.14s/it]
 42%|████▏     | 95302/225000 [172:20:53<354:06:24,  9.83s/it]
 42%|████▏     | 95303/225000 [172:20:58<299:57:40,  8.33s/it]
 42%|████▏     | 95304/225000 [172:21:23<483:43:34, 13.43s/it]
 42%|████▏     | 95305/225000 [172:21:33<448:33:33, 12.45s/it]
 42%|████▏     | 95306/225000 [172:21:41<396:10:47, 11.00s/it]
 42%|████▏     | 95307/225000 [172:21:50<378:14:08, 10.50s/it]
 42%|████▏     | 95308/225000 [172:22:05<422:51:50, 11.74s/it]
 42%|████▏     | 95309/225000 [172:22:15<405:03:33, 11.24s/it]
 42%|████▏     | 95310/225000 [172:22:23<369:51:10, 10.27s/it]
 42%|████▏     | 95311/225000 [172:22:42<462:44:42, 12.85s/it]
 42%|████▏     | 95312/225000 [172:22:52<435:48:14, 12.10s/it]
 42%|████▏     | 95313/225000 [172:22:59<374:37:57, 10.40s/it]
 42%|████▏     | 95314/225000 [172:23:17<455:36:42, 12.65s/it]
 42%|████▏     | 95315/225000 [172:23:29<456:08:19, 12.66s/it]
 42%|████▏     | 95316/225000 [172:23:35<379:45:02, 10.54s/it]
 42%|████▏     | 95317/225000 [172:23:54<476:20:30, 13.22s/it]
 42%|████▏     | 95318/225000 [172:24:04<441:52:32, 12.27s/it]
 42%|████▏     | 95319/225000 [172:24:12<386:52:06, 10.74s/it]
 42%|████▏     | 95320/225000 [172:24:19<347:48:55,  9.66s/it]
 42%|████▏     | 95321/225000 [172:24:25<315:03:49,  8.75s/it]
 42%|████▏     | 95322/225000 [172:24:33<305:06:54,  8.47s/it]
 42%|████▏     | 95323/225000 [172:24:43<321:35:24,  8.93s/it]
 42%|████▏     | 95324/225000 [172:24:55<349:50:12,  9.71s/it]
 42%|████▏     | 95325/225000 [172:25:06<365:27:09, 10.15s/it]
 42%|████▏     | 95326/225000 [172:25:15<350:16:58,  9.72s/it]
 42%|████▏     | 95327/225000 [172:25:25<359:38:42,  9.98s/it]
 42%|████▏     | 95328/225000 [172:25:32<323:45:57,  8.99s/it]
 42%|████▏     | 95329/225000 [172:25:39<299:24:38,  8.31s/it]
 42%|████▏     | 95330/225000 [172:25:56<398:51:41, 11.07s/it]
 42%|████▏     | 95331/225000 [172:26:10<430:01:33, 11.94s/it]
 42%|████▏     | 95332/225000 [172:26:20<405:53:51, 11.27s/it]
 42%|████▏     | 95333/225000 [172:26:31<404:39:42, 11.23s/it]
 42%|████▏     | 95334/225000 [172:26:44<421:09:49, 11.69s/it]
 42%|████▏     | 95335/225000 [172:26:50<365:00:59, 10.13s/it]
 42%|████▏     | 95336/225000 [172:27:03<388:14:13, 10.78s/it]
 42%|████▏     | 95337/225000 [172:27:10<351:01:42,  9.75s/it]
 42%|████▏     | 95338/225000 [172:27:22<372:59:04, 10.36s/it]
 42%|████▏     | 95339/225000 [172:27:36<419:25:58, 11.65s/it]
 42%|████▏     | 95340/225000 [172:27:43<364:09:48, 10.11s/it]
 42%|████▏     | 95341/225000 [172:27:53<364:17:27, 10.11s/it]
 42%|████▏     | 95342/225000 [172:28:08<416:54:20, 11.58s/it]
 42%|████▏     | 95343/225000 [172:28:14<359:10:07,  9.97s/it]
 42%|████▏     | 95344/225000 [172:28:31<434:31:49, 12.07s/it]
 42%|████▏     | 95345/225000 [172:28:51<515:34:38, 14.32s/it]
 42%|████▏     | 95346/225000 [172:28:59<447:46:13, 12.43s/it]
 42%|████▏     | 95347/225000 [172:29:20<538:40:56, 14.96s/it]
 42%|████▏     | 95348/225000 [172:29:34<533:41:03, 14.82s/it]
 42%|████▏     | 95349/225000 [172:29:44<484:00:17, 13.44s/it]
 42%|████▏     | 95350/225000 [172:29:53<431:39:01, 11.99s/it]
                                                              
{'loss': '0.7415', 'grad_norm': '4.061', 'learning_rate': '3.297e-05', 'epoch': '0.4238', 'num_input_tokens_seen': 282581648, 'train_runtime': '6.21e+05', 'train_tokens_per_second': '455'}

 42%|████▏     | 95350/225000 [172:29:53<431:39:01, 11.99s/it]
 42%|████▏     | 95351/225000 [172:30:02<396:17:39, 11.00s/it]
 42%|████▏     | 95352/225000 [172:30:12<394:54:23, 10.97s/it]
 42%|████▏     | 95353/225000 [172:30:24<405:29:10, 11.26s/it]
 42%|████▏     | 95354/225000 [172:30:37<417:36:14, 11.60s/it]
 42%|████▏     | 95355/225000 [172:31:00<543:03:34, 15.08s/it]
 42%|████▏     | 95356/225000 [172:31:07<451:51:38, 12.55s/it]
 42%|████▏     | 95357/225000 [172:31:20<462:43:58, 12.85s/it]
 42%|████▏     | 95358/225000 [172:31:34<472:51:06, 13.13s/it]
 42%|████▏     | 95359/225000 [172:31:43<431:49:48, 11.99s/it]
 42%|████▏     | 95360/225000 [172:31:53<407:32:14, 11.32s/it]
 42%|████▏     | 95361/225000 [172:32:04<401:27:11, 11.15s/it]
 42%|████▏     | 95362/225000 [172:32:11<359:53:07,  9.99s/it]
 42%|████▏     | 95363/225000 [172:32:27<422:35:30, 11.74s/it]
 42%|████▏     | 95364/225000 [172:32:43<468:44:52, 13.02s/it]
 42%|████▏     | 95365/225000 [172:32:49<393:53:24, 10.94s/it]
 42%|████▏     | 95366/225000 [172:33:01<401:44:24, 11.16s/it]
 42%|████▏     | 95367/225000 [172:33:05<324:27:18,  9.01s/it]
 42%|████▏     | 95368/225000 [172:33:19<379:24:22, 10.54s/it]
 42%|████▏     | 95369/225000 [172:33:30<388:07:25, 10.78s/it]
 42%|████▏     | 95370/225000 [172:33:44<418:58:54, 11.64s/it]
 42%|████▏     | 95371/225000 [172:33:54<399:57:16, 11.11s/it]
 42%|████▏     | 95372/225000 [172:34:01<356:39:31,  9.91s/it]
 42%|████▏     | 95373/225000 [172:34:17<428:28:05, 11.90s/it]
 42%|████▏     | 95374/225000 [172:34:34<477:41:00, 13.27s/it]
 42%|████▏     | 95375/225000 [172:34:44<444:49:06, 12.35s/it]
 42%|████▏     | 95376/225000 [172:34:53<411:52:18, 11.44s/it]
 42%|████▏     | 95377/225000 [172:34:58<340:51:10,  9.47s/it]
 42%|████▏     | 95378/225000 [172:35:05<309:01:25,  8.58s/it]
 42%|████▏     | 95379/225000 [172:35:28<469:40:26, 13.04s/it]
 42%|████▏     | 95380/225000 [172:35:32<374:18:34, 10.40s/it]
 42%|████▏     | 95381/225000 [172:35:39<339:03:56,  9.42s/it]
 42%|████▏     | 95382/225000 [172:35:56<418:31:05, 11.62s/it]
 42%|████▏     | 95383/225000 [172:36:03<364:25:57, 10.12s/it]
 42%|████▏     | 95384/225000 [172:36:16<394:18:28, 10.95s/it]
 42%|████▏     | 95385/225000 [172:36:27<402:37:22, 11.18s/it]
 42%|████▏     | 95386/225000 [172:36:34<351:45:41,  9.77s/it]
 42%|████▏     | 95387/225000 [172:36:50<418:18:54, 11.62s/it]
 42%|████▏     | 95388/225000 [172:37:11<524:46:14, 14.58s/it]
 42%|████▏     | 95389/225000 [172:37:24<504:17:30, 14.01s/it]
 42%|████▏     | 95390/225000 [172:37:34<462:12:23, 12.84s/it]
 42%|████▏     | 95391/225000 [172:37:43<420:01:08, 11.67s/it]
 42%|████▏     | 95392/225000 [172:37:55<421:39:34, 11.71s/it]
 42%|████▏     | 95393/225000 [172:38:04<389:41:48, 10.82s/it]
 42%|████▏     | 95394/225000 [172:38:22<470:38:11, 13.07s/it]
 42%|████▏     | 95395/225000 [172:38:27<381:21:16, 10.59s/it]
 42%|████▏     | 95396/225000 [172:38:41<422:41:46, 11.74s/it]
 42%|████▏     | 95397/225000 [172:38:53<427:20:58, 11.87s/it]
 42%|████▏     | 95398/225000 [172:39:11<493:11:53, 13.70s/it]
 42%|████▏     | 95399/225000 [172:39:21<444:35:50, 12.35s/it]
 42%|████▏     | 95400/225000 [172:39:27<383:52:37, 10.66s/it]
                                                              
{'loss': '0.7516', 'grad_norm': '2.461', 'learning_rate': '3.295e-05', 'epoch': '0.424', 'num_input_tokens_seen': 282859704, 'train_runtime': '6.216e+05', 'train_tokens_per_second': '455.1'}

 42%|████▏     | 95400/225000 [172:39:27<383:52:37, 10.66s/it]
 42%|████▏     | 95401/225000 [172:39:45<457:59:09, 12.72s/it]
 42%|████▏     | 95402/225000 [172:39:51<387:20:32, 10.76s/it]
 42%|████▏     | 95403/225000 [172:40:05<426:42:27, 11.85s/it]
 42%|████▏     | 95404/225000 [172:40:13<381:06:11, 10.59s/it]
 42%|████▏     | 95405/225000 [172:40:17<311:00:58,  8.64s/it]
 42%|████▏     | 95406/225000 [172:40:25<300:25:32,  8.35s/it]
 42%|████▏     | 95407/225000 [172:40:41<386:44:39, 10.74s/it]
 42%|████▏     | 95408/225000 [172:40:49<351:27:21,  9.76s/it]
 42%|████▏     | 95409/225000 [172:41:02<390:40:08, 10.85s/it]
 42%|████▏     | 95410/225000 [172:41:07<322:37:48,  8.96s/it]
 42%|████▏     | 95411/225000 [172:41:22<391:01:02, 10.86s/it]
 42%|████▏     | 95412/225000 [172:41:38<452:08:00, 12.56s/it]
 42%|████▏     | 95413/225000 [172:41:48<418:54:23, 11.64s/it]
 42%|████▏     | 95414/225000 [172:41:54<365:13:21, 10.15s/it]
 42%|████▏     | 95415/225000 [172:42:03<343:43:52,  9.55s/it]
 42%|████▏     | 95416/225000 [172:42:15<379:02:36, 10.53s/it]
 42%|████▏     | 95417/225000 [172:42:31<432:57:02, 12.03s/it]
 42%|████▏     | 95418/225000 [172:42:36<362:26:51, 10.07s/it]
 42%|████▏     | 95419/225000 [172:42:56<466:37:52, 12.96s/it]
 42%|████▏     | 95420/225000 [172:43:04<413:29:28, 11.49s/it]
 42%|████▏     | 95421/225000 [172:43:14<395:41:14, 10.99s/it]
 42%|████▏     | 95422/225000 [172:43:32<472:11:44, 13.12s/it]
 42%|████▏     | 95423/225000 [172:43:41<421:43:32, 11.72s/it]
 42%|████▏     | 95424/225000 [172:43:50<401:33:03, 11.16s/it]
 42%|████▏     | 95425/225000 [172:44:08<474:15:29, 13.18s/it]
 42%|████▏     | 95426/225000 [172:44:15<398:54:22, 11.08s/it]
 42%|████▏     | 95427/225000 [172:44:35<500:37:33, 13.91s/it]
 42%|████▏     | 95428/225000 [172:44:46<465:13:58, 12.93s/it]
 42%|████▏     | 95429/225000 [172:44:51<383:24:58, 10.65s/it]
 42%|████▏     | 95430/225000 [172:45:12<492:52:03, 13.69s/it]
 42%|████▏     | 95431/225000 [172:45:24<471:41:02, 13.11s/it]
 42%|████▏     | 95432/225000 [172:45:38<489:11:15, 13.59s/it]
 42%|████▏     | 95433/225000 [172:45:54<509:13:37, 14.15s/it]
 42%|████▏     | 95434/225000 [172:46:04<470:27:31, 13.07s/it]
 42%|████▏     | 95435/225000 [172:46:15<441:26:46, 12.27s/it]
 42%|████▏     | 95436/225000 [172:46:22<390:05:17, 10.84s/it]
 42%|████▏     | 95437/225000 [172:46:29<344:11:19,  9.56s/it]
 42%|████▏     | 95438/225000 [172:46:44<403:39:09, 11.22s/it]
 42%|████▏     | 95439/225000 [172:46:56<409:56:03, 11.39s/it]
 42%|████▏     | 95440/225000 [172:47:07<411:23:18, 11.43s/it]
 42%|████▏     | 95441/225000 [172:47:16<380:44:01, 10.58s/it]
 42%|████▏     | 95442/225000 [172:47:30<425:50:21, 11.83s/it]
 42%|████▏     | 95443/225000 [172:47:40<403:39:19, 11.22s/it]
 42%|████▏     | 95444/225000 [172:47:52<414:02:12, 11.50s/it]
 42%|████▏     | 95445/225000 [172:48:03<399:04:51, 11.09s/it]
 42%|████▏     | 95446/225000 [172:48:16<423:29:42, 11.77s/it]
 42%|████▏     | 95447/225000 [172:48:36<512:30:58, 14.24s/it]
 42%|████▏     | 95448/225000 [172:48:46<468:35:33, 13.02s/it]
 42%|████▏     | 95449/225000 [172:48:56<432:37:02, 12.02s/it]
 42%|████▏     | 95450/225000 [172:49:07<427:38:27, 11.88s/it]
                                                              
{'loss': '0.8029', 'grad_norm': '1.82', 'learning_rate': '3.293e-05', 'epoch': '0.4242', 'num_input_tokens_seen': 283149152, 'train_runtime': '6.221e+05', 'train_tokens_per_second': '455.1'}

 42%|████▏     | 95450/225000 [172:49:07<427:38:27, 11.88s/it]
 42%|████▏     | 95451/225000 [172:49:26<500:45:58, 13.92s/it]
 42%|████▏     | 95452/225000 [172:49:39<493:52:41, 13.72s/it]
 42%|████▏     | 95453/225000 [172:49:50<462:26:37, 12.85s/it]
 42%|████▏     | 95454/225000 [172:49:57<402:53:31, 11.20s/it]
 42%|████▏     | 95455/225000 [172:50:11<429:31:17, 11.94s/it]
 42%|████▏     | 95456/225000 [172:50:16<356:13:28,  9.90s/it]
 42%|████▏     | 95457/225000 [172:50:39<490:02:26, 13.62s/it]
 42%|████▏     | 95458/225000 [172:50:51<481:07:13, 13.37s/it]
 42%|████▏     | 95459/225000 [172:51:00<427:37:40, 11.88s/it]
 42%|████▏     | 95460/225000 [172:51:14<448:30:15, 12.46s/it]
 42%|████▏     | 95461/225000 [172:51:39<588:04:17, 16.34s/it]
 42%|████▏     | 95462/225000 [172:51:49<521:08:35, 14.48s/it]
 42%|████▏     | 95463/225000 [172:51:58<463:41:04, 12.89s/it]
 42%|████▏     | 95464/225000 [172:52:05<398:47:57, 11.08s/it]
 42%|████▏     | 95465/225000 [172:52:23<476:04:19, 13.23s/it]
 42%|████▏     | 95466/225000 [172:52:30<402:25:00, 11.18s/it]
 42%|████▏     | 95467/225000 [172:52:34<329:40:40,  9.16s/it]
 42%|████▏     | 95468/225000 [172:52:50<398:51:28, 11.09s/it]
 42%|████▏     | 95469/225000 [172:53:09<485:05:27, 13.48s/it]
 42%|████▏     | 95470/225000 [172:53:23<493:16:41, 13.71s/it]
 42%|████▏     | 95471/225000 [172:53:38<501:59:05, 13.95s/it]
 42%|████▏     | 95472/225000 [172:53:44<415:57:32, 11.56s/it]
 42%|████▏     | 95473/225000 [172:53:48<337:38:57,  9.38s/it]
 42%|████▏     | 95474/225000 [172:53:55<308:57:15,  8.59s/it]
 42%|████▏     | 95475/225000 [172:54:13<415:13:32, 11.54s/it]
 42%|████▏     | 95476/225000 [172:54:24<404:58:14, 11.26s/it]
 42%|████▏     | 95477/225000 [172:54:39<453:04:14, 12.59s/it]
 42%|████▏     | 95478/225000 [172:55:02<565:37:20, 15.72s/it]
 42%|████▏     | 95479/225000 [172:55:08<457:38:00, 12.72s/it]
 42%|████▏     | 95480/225000 [172:55:29<547:31:16, 15.22s/it]
 42%|████▏     | 95481/225000 [172:55:38<480:31:52, 13.36s/it]
 42%|████▏     | 95482/225000 [172:55:52<480:34:09, 13.36s/it]
 42%|████▏     | 95483/225000 [172:56:02<454:12:38, 12.63s/it]
 42%|████▏     | 95484/225000 [172:56:13<434:59:40, 12.09s/it]
 42%|████▏     | 95485/225000 [172:56:24<423:06:17, 11.76s/it]
 42%|████▏     | 95486/225000 [172:56:40<468:28:47, 13.02s/it]
 42%|████▏     | 95487/225000 [172:56:58<518:40:50, 14.42s/it]
 42%|████▏     | 95488/225000 [172:57:07<465:51:01, 12.95s/it]
 42%|████▏     | 95489/225000 [172:57:25<513:52:08, 14.28s/it]
 42%|████▏     | 95490/225000 [172:57:49<615:33:33, 17.11s/it]
 42%|████▏     | 95491/225000 [172:58:01<562:26:50, 15.63s/it]
 42%|████▏     | 95492/225000 [172:58:08<475:34:39, 13.22s/it]
 42%|████▏     | 95493/225000 [172:58:17<429:12:44, 11.93s/it]
 42%|████▏     | 95494/225000 [172:58:38<524:24:07, 14.58s/it]
 42%|████▏     | 95495/225000 [172:58:49<483:14:25, 13.43s/it]
 42%|████▏     | 95496/225000 [172:59:02<483:20:18, 13.44s/it]
 42%|████▏     | 95497/225000 [172:59:18<503:32:36, 14.00s/it]
 42%|████▏     | 95498/225000 [172:59:29<477:56:37, 13.29s/it]
 42%|████▏     | 95499/225000 [172:59:32<366:30:21, 10.19s/it]
 42%|████▏     | 95500/225000 [172:59:40<340:17:15,  9.46s/it]
                                                              
{'loss': '0.7923', 'grad_norm': '1.776', 'learning_rate': '3.292e-05', 'epoch': '0.4244', 'num_input_tokens_seen': 283459008, 'train_runtime': '6.228e+05', 'train_tokens_per_second': '455.2'}

 42%|████▏     | 95500/225000 [172:59:40<340:17:15,  9.46s/it]
 42%|████▏     | 95501/225000 [172:59:51<360:51:07, 10.03s/it]
 42%|████▏     | 95502/225000 [173:00:10<455:57:52, 12.68s/it]
 42%|████▏     | 95503/225000 [173:00:20<425:12:02, 11.82s/it]
 42%|████▏     | 95504/225000 [173:00:33<436:38:44, 12.14s/it]
 42%|████▏     | 95505/225000 [173:00:37<345:33:31,  9.61s/it]
 42%|████▏     | 95506/225000 [173:00:52<408:22:23, 11.35s/it]
 42%|████▏     | 95507/225000 [173:01:11<494:39:30, 13.75s/it]
 42%|████▏     | 95508/225000 [173:01:33<577:30:11, 16.06s/it]
 42%|████▏     | 95509/225000 [173:01:39<475:00:32, 13.21s/it]
 42%|████▏     | 95510/225000 [173:01:43<373:50:38, 10.39s/it]
 42%|████▏     | 95511/225000 [173:01:53<369:04:54, 10.26s/it]
 42%|████▏     | 95512/225000 [173:02:06<396:56:43, 11.04s/it]
 42%|████▏     | 95513/225000 [173:02:12<349:09:11,  9.71s/it]
 42%|████▏     | 95514/225000 [173:02:23<358:56:35,  9.98s/it]
 42%|████▏     | 95515/225000 [173:02:40<429:09:45, 11.93s/it]
 42%|████▏     | 95516/225000 [173:02:46<369:19:53, 10.27s/it]
 42%|████▏     | 95517/225000 [173:03:04<454:35:23, 12.64s/it]
 42%|████▏     | 95518/225000 [173:03:16<444:02:38, 12.35s/it]
 42%|████▏     | 95519/225000 [173:03:27<436:09:07, 12.13s/it]
 42%|████▏     | 95520/225000 [173:03:36<398:38:50, 11.08s/it]
 42%|████▏     | 95521/225000 [173:03:49<415:53:20, 11.56s/it]
 42%|████▏     | 95522/225000 [173:04:02<431:00:10, 11.98s/it]
 42%|████▏     | 95523/225000 [173:04:19<493:02:57, 13.71s/it]
 42%|████▏     | 95524/225000 [173:04:24<389:21:41, 10.83s/it]
 42%|████▏     | 95525/225000 [173:04:30<346:01:33,  9.62s/it]
 42%|████▏     | 95526/225000 [173:04:47<418:51:48, 11.65s/it]
 42%|████▏     | 95527/225000 [173:05:07<511:28:17, 14.22s/it]
 42%|████▏     | 95528/225000 [173:05:28<589:35:34, 16.39s/it]
 42%|████▏     | 95529/225000 [173:05:43<573:42:32, 15.95s/it]
 42%|████▏     | 95530/225000 [173:05:52<495:49:45, 13.79s/it]
 42%|████▏     | 95531/225000 [173:06:01<443:33:54, 12.33s/it]
 42%|████▏     | 95532/225000 [173:06:05<351:24:05,  9.77s/it]
 42%|████▏     | 95533/225000 [173:06:12<318:10:05,  8.85s/it]
 42%|████▏     | 95534/225000 [173:06:24<352:39:53,  9.81s/it]
 42%|████▏     | 95535/225000 [173:06:30<315:46:57,  8.78s/it]
 42%|████▏     | 95536/225000 [173:06:41<345:26:04,  9.61s/it]
 42%|████▏     | 95537/225000 [173:06:54<375:00:46, 10.43s/it]
 42%|████▏     | 95538/225000 [173:07:18<521:13:38, 14.49s/it]
 42%|████▏     | 95539/225000 [173:07:30<491:20:27, 13.66s/it]
 42%|████▏     | 95540/225000 [173:07:43<492:50:43, 13.70s/it]
 42%|████▏     | 95541/225000 [173:07:55<473:30:38, 13.17s/it]
 42%|████▏     | 95542/225000 [173:08:19<588:13:28, 16.36s/it]
 42%|████▏     | 95543/225000 [173:08:27<501:40:24, 13.95s/it]
 42%|████▏     | 95544/225000 [173:08:43<520:35:41, 14.48s/it]
 42%|████▏     | 95545/225000 [173:08:55<491:18:59, 13.66s/it]
 42%|████▏     | 95546/225000 [173:08:58<375:43:43, 10.45s/it]
 42%|████▏     | 95547/225000 [173:09:07<365:02:23, 10.15s/it]
 42%|████▏     | 95548/225000 [173:09:22<412:07:06, 11.46s/it]
 42%|████▏     | 95549/225000 [173:09:27<342:20:31,  9.52s/it]
 42%|████▏     | 95550/225000 [173:09:40<387:17:04, 10.77s/it]
                                                              
{'loss': '0.756', 'grad_norm': '1.15', 'learning_rate': '3.29e-05', 'epoch': '0.4247', 'num_input_tokens_seen': 283749968, 'train_runtime': '6.234e+05', 'train_tokens_per_second': '455.2'}

 42%|████▏     | 95550/225000 [173:09:40<387:17:04, 10.77s/it]
 42%|████▏     | 95551/225000 [173:09:52<391:29:56, 10.89s/it]
 42%|████▏     | 95552/225000 [173:10:08<451:19:28, 12.55s/it]
 42%|████▏     | 95553/225000 [173:10:23<473:28:29, 13.17s/it]
 42%|████▏     | 95554/225000 [173:10:31<426:03:24, 11.85s/it]
 42%|████▏     | 95555/225000 [173:10:48<477:16:29, 13.27s/it]
 42%|████▏     | 95556/225000 [173:11:00<462:31:07, 12.86s/it]
 42%|████▏     | 95557/225000 [173:11:23<569:10:04, 15.83s/it]
 42%|████▏     | 95558/225000 [173:11:43<618:45:58, 17.21s/it]
 42%|████▏     | 95559/225000 [173:11:59<599:32:40, 16.67s/it]
 42%|████▏     | 95560/225000 [173:12:14<588:31:40, 16.37s/it]
 42%|████▏     | 95561/225000 [173:12:27<546:38:53, 15.20s/it]
 42%|████▏     | 95562/225000 [173:12:40<523:24:32, 14.56s/it]
 42%|████▏     | 95563/225000 [173:12:47<447:58:00, 12.46s/it]
 42%|████▏     | 95564/225000 [173:12:58<425:33:09, 11.84s/it]
 42%|████▏     | 95565/225000 [173:13:13<463:00:46, 12.88s/it]
 42%|████▏     | 95566/225000 [173:13:21<409:19:36, 11.38s/it]
 42%|████▏     | 95567/225000 [173:13:34<422:43:24, 11.76s/it]
 42%|████▏     | 95568/225000 [173:13:47<440:20:56, 12.25s/it]
 42%|████▏     | 95569/225000 [173:14:18<642:03:45, 17.86s/it]
 42%|████▏     | 95570/225000 [173:14:23<506:32:58, 14.09s/it]
 42%|████▏     | 95571/225000 [173:14:28<408:26:33, 11.36s/it]
 42%|████▏     | 95572/225000 [173:14:53<550:38:14, 15.32s/it]
 42%|████▏     | 95573/225000 [173:15:04<505:38:15, 14.06s/it]
 42%|████▏     | 95574/225000 [173:15:18<502:01:21, 13.96s/it]
 42%|████▏     | 95575/225000 [173:15:33<514:48:34, 14.32s/it]
 42%|████▏     | 95576/225000 [173:15:42<457:56:13, 12.74s/it]
 42%|████▏     | 95577/225000 [173:16:01<527:51:02, 14.68s/it]
 42%|████▏     | 95578/225000 [173:16:08<442:12:04, 12.30s/it]
 42%|████▏     | 95579/225000 [173:16:18<422:30:41, 11.75s/it]
 42%|████▏     | 95580/225000 [173:16:29<415:57:04, 11.57s/it]
 42%|████▏     | 95581/225000 [173:16:44<450:18:43, 12.53s/it]
 42%|████▏     | 95582/225000 [173:17:04<530:48:15, 14.77s/it]
 42%|████▏     | 95583/225000 [173:17:19<528:16:29, 14.70s/it]
 42%|████▏     | 95584/225000 [173:17:25<441:48:55, 12.29s/it]
 42%|████▏     | 95585/225000 [173:17:38<451:22:26, 12.56s/it]
 42%|████▏     | 95586/225000 [173:17:54<488:05:51, 13.58s/it]
 42%|████▏     | 95587/225000 [173:18:14<554:40:11, 15.43s/it]
 42%|████▏     | 95588/225000 [173:18:18<434:21:18, 12.08s/it]
 42%|████▏     | 95589/225000 [173:18:34<474:04:30, 13.19s/it]
 42%|████▏     | 95590/225000 [173:18:53<530:58:28, 14.77s/it]
 42%|████▏     | 95591/225000 [173:19:00<447:41:49, 12.45s/it]
 42%|████▏     | 95592/225000 [173:19:10<419:27:20, 11.67s/it]
 42%|████▏     | 95593/225000 [173:19:19<391:26:27, 10.89s/it]
 42%|████▏     | 95594/225000 [173:19:36<465:47:00, 12.96s/it]
 42%|████▏     | 95595/225000 [173:19:50<471:06:37, 13.11s/it]
 42%|████▏     | 95596/225000 [173:19:58<417:34:27, 11.62s/it]
 42%|████▏     | 95597/225000 [173:20:13<452:52:00, 12.60s/it]
 42%|████▏     | 95598/225000 [173:20:30<499:14:48, 13.89s/it]
 42%|████▏     | 95599/225000 [173:20:39<450:37:07, 12.54s/it]
 42%|████▏     | 95600/225000 [173:20:46<383:53:18, 10.68s/it]
                                                              
{'loss': '0.8401', 'grad_norm': '1.885', 'learning_rate': '3.288e-05', 'epoch': '0.4249', 'num_input_tokens_seen': 284071976, 'train_runtime': '6.24e+05', 'train_tokens_per_second': '455.2'}

 42%|████▏     | 95600/225000 [173:20:46<383:53:18, 10.68s/it]
 42%|████▏     | 95601/225000 [173:20:54<364:42:15, 10.15s/it]
 42%|████▏     | 95602/225000 [173:21:08<399:16:56, 11.11s/it]
 42%|████▏     | 95603/225000 [173:21:29<503:17:48, 14.00s/it]
 42%|████▏     | 95604/225000 [173:21:44<523:45:19, 14.57s/it]
 42%|████▏     | 95605/225000 [173:21:55<481:49:31, 13.41s/it]
 42%|████▏     | 95606/225000 [173:22:15<547:10:41, 15.22s/it]
 42%|████▏     | 95607/225000 [173:22:28<525:36:13, 14.62s/it]
 42%|████▏     | 95608/225000 [173:22:44<537:29:06, 14.95s/it]
 42%|████▏     | 95609/225000 [173:22:50<442:16:46, 12.31s/it]
 42%|████▏     | 95610/225000 [173:22:58<399:13:46, 11.11s/it]
 42%|████▏     | 95611/225000 [173:23:18<492:48:59, 13.71s/it]
 42%|████▏     | 95612/225000 [173:23:28<450:48:07, 12.54s/it]
 42%|████▏     | 95613/225000 [173:23:48<533:09:44, 14.83s/it]
 42%|████▏     | 95614/225000 [173:24:01<510:36:23, 14.21s/it]
 42%|████▏     | 95615/225000 [173:24:11<465:21:50, 12.95s/it]
 42%|████▏     | 95616/225000 [173:24:28<510:18:12, 14.20s/it]
 42%|████▏     | 95617/225000 [173:24:45<543:37:18, 15.13s/it]
 42%|████▏     | 95618/225000 [173:25:02<559:14:49, 15.56s/it]
 42%|████▏     | 95619/225000 [173:25:13<518:49:06, 14.44s/it]
 42%|████▏     | 95620/225000 [173:25:24<481:29:24, 13.40s/it]
 42%|████▏     | 95621/225000 [173:25:38<484:00:55, 13.47s/it]
 42%|████▏     | 95622/225000 [173:25:47<436:46:39, 12.15s/it]
 42%|████▏     | 95623/225000 [173:26:03<481:42:52, 13.40s/it]
 42%|████▏     | 95624/225000 [173:26:12<430:54:13, 11.99s/it]
 42%|████▎     | 95625/225000 [173:26:31<505:54:00, 14.08s/it]
 43%|████▎     | 95626/225000 [173:26:46<517:58:09, 14.41s/it]
 43%|████▎     | 95627/225000 [173:27:02<527:54:24, 14.69s/it]
 43%|████▎     | 95628/225000 [173:27:14<503:59:37, 14.02s/it]
 43%|████▎     | 95629/225000 [173:27:34<571:39:54, 15.91s/it]
 43%|████▎     | 95630/225000 [173:27:47<533:56:46, 14.86s/it]
 43%|████▎     | 95631/225000 [173:27:54<447:51:32, 12.46s/it]
 43%|████▎     | 95632/225000 [173:28:00<383:26:23, 10.67s/it]
 43%|████▎     | 95633/225000 [173:28:13<413:15:47, 11.50s/it]
 43%|████▎     | 95634/225000 [173:28:23<394:44:55, 10.99s/it]
 43%|████▎     | 95635/225000 [173:28:34<390:33:49, 10.87s/it]
 43%|████▎     | 95636/225000 [173:28:43<375:42:20, 10.46s/it]
 43%|████▎     | 95637/225000 [173:29:02<462:11:28, 12.86s/it]
 43%|████▎     | 95638/225000 [173:29:12<429:04:04, 11.94s/it]
 43%|████▎     | 95639/225000 [173:29:25<440:31:26, 12.26s/it]
 43%|████▎     | 95640/225000 [173:29:41<486:44:42, 13.55s/it]
 43%|████▎     | 95641/225000 [173:29:50<439:20:32, 12.23s/it]
 43%|████▎     | 95642/225000 [173:30:05<470:42:07, 13.10s/it]
 43%|████▎     | 95643/225000 [173:30:24<532:39:48, 14.82s/it]
 43%|████▎     | 95644/225000 [173:30:34<478:59:52, 13.33s/it]
 43%|████▎     | 95645/225000 [173:30:52<526:13:51, 14.65s/it]
 43%|████▎     | 95646/225000 [173:31:07<532:07:40, 14.81s/it]
 43%|████▎     | 95647/225000 [173:31:26<573:59:15, 15.97s/it]
 43%|████▎     | 95648/225000 [173:31:36<513:17:32, 14.29s/it]
 43%|████▎     | 95649/225000 [173:31:47<481:51:14, 13.41s/it]
 43%|████▎     | 95650/225000 [173:31:59<457:01:53, 12.72s/it]
                                                              
{'loss': '0.8005', 'grad_norm': '1.48', 'learning_rate': '3.286e-05', 'epoch': '0.4251', 'num_input_tokens_seen': 284394016, 'train_runtime': '6.247e+05', 'train_tokens_per_second': '455.2'}

 43%|████▎     | 95650/225000 [173:31:59<457:01:53, 12.72s/it]
 43%|████▎     | 95651/225000 [173:32:16<507:05:07, 14.11s/it]
 43%|████▎     | 95652/225000 [173:32:23<432:57:54, 12.05s/it]
 43%|████▎     | 95653/225000 [173:32:33<404:51:01, 11.27s/it]
 43%|████▎     | 95654/225000 [173:32:54<509:07:44, 14.17s/it]
 43%|████▎     | 95655/225000 [173:33:11<544:28:13, 15.15s/it]
 43%|████▎     | 95656/225000 [173:33:26<541:17:19, 15.07s/it]
 43%|████▎     | 95657/225000 [173:33:34<470:56:16, 13.11s/it]
 43%|████▎     | 95658/225000 [173:33:44<430:17:52, 11.98s/it]
 43%|████▎     | 95659/225000 [173:34:01<486:55:56, 13.55s/it]
 43%|████▎     | 95660/225000 [173:34:15<491:55:07, 13.69s/it]
 43%|████▎     | 95661/225000 [173:34:31<515:50:16, 14.36s/it]
 43%|████▎     | 95662/225000 [173:34:46<520:07:47, 14.48s/it]
 43%|████▎     | 95663/225000 [173:34:56<470:49:21, 13.10s/it]
 43%|████▎     | 95664/225000 [173:35:12<512:01:32, 14.25s/it]
 43%|████▎     | 95665/225000 [173:35:17<410:12:25, 11.42s/it]
 43%|████▎     | 95666/225000 [173:35:45<580:26:58, 16.16s/it]
 43%|████▎     | 95667/225000 [173:35:49<450:43:08, 12.55s/it]
 43%|████▎     | 95668/225000 [173:36:03<471:29:57, 13.12s/it]
 43%|████▎     | 95669/225000 [173:36:21<521:47:02, 14.52s/it]
 43%|████▎     | 95670/225000 [173:36:37<536:13:44, 14.93s/it]
 43%|████▎     | 95671/225000 [173:36:50<521:36:03, 14.52s/it]
 43%|████▎     | 95672/225000 [173:36:57<434:18:41, 12.09s/it]
 43%|████▎     | 95673/225000 [173:37:07<417:39:37, 11.63s/it]
 43%|████▎     | 95674/225000 [173:37:29<528:18:41, 14.71s/it]
 43%|████▎     | 95675/225000 [173:37:35<433:10:06, 12.06s/it]
 43%|████▎     | 95676/225000 [173:37:52<482:21:53, 13.43s/it]
 43%|████▎     | 95677/225000 [173:38:05<480:42:03, 13.38s/it]
 43%|████▎     | 95678/225000 [173:38:20<497:09:09, 13.84s/it]
 43%|████▎     | 95679/225000 [173:38:32<482:18:26, 13.43s/it]
 43%|████▎     | 95680/225000 [173:38:38<400:06:23, 11.14s/it]
 43%|████▎     | 95681/225000 [173:38:49<400:03:46, 11.14s/it]
 43%|████▎     | 95682/225000 [173:39:01<410:31:47, 11.43s/it]
 43%|████▎     | 95683/225000 [173:39:12<397:35:40, 11.07s/it]
 43%|████▎     | 95684/225000 [173:39:34<524:05:39, 14.59s/it]
 43%|████▎     | 95685/225000 [173:39:44<468:41:03, 13.05s/it]
 43%|████▎     | 95686/225000 [173:39:58<476:47:57, 13.27s/it]
 43%|████▎     | 95687/225000 [173:40:17<540:39:08, 15.05s/it]
 43%|████▎     | 95688/225000 [173:40:25<471:30:52, 13.13s/it]
 43%|████▎     | 95689/225000 [173:40:49<585:25:52, 16.30s/it]
 43%|████▎     | 95690/225000 [173:41:01<532:03:29, 14.81s/it]
 43%|████▎     | 95691/225000 [173:41:14<516:06:24, 14.37s/it]
 43%|████▎     | 95692/225000 [173:41:18<409:18:12, 11.40s/it]
 43%|████▎     | 95693/225000 [173:41:35<466:24:25, 12.99s/it]
 43%|████▎     | 95694/225000 [173:41:41<386:46:34, 10.77s/it]
 43%|████▎     | 95695/225000 [173:41:55<428:01:26, 11.92s/it]
 43%|████▎     | 95696/225000 [173:42:08<438:20:59, 12.20s/it]
 43%|████▎     | 95697/225000 [173:42:22<454:46:26, 12.66s/it]
 43%|████▎     | 95698/225000 [173:42:34<450:11:48, 12.53s/it]
 43%|████▎     | 95699/225000 [173:42:48<467:46:29, 13.02s/it]
 43%|████▎     | 95700/225000 [173:43:03<488:20:01, 13.60s/it]
                                                              
{'loss': '0.82', 'grad_norm': '1.771', 'learning_rate': '3.285e-05', 'epoch': '0.4253', 'num_input_tokens_seen': 284715904, 'train_runtime': '6.254e+05', 'train_tokens_per_second': '455.3'}

 43%|████▎     | 95700/225000 [173:43:03<488:20:01, 13.60s/it]
 43%|████▎     | 95701/225000 [173:43:12<439:18:48, 12.23s/it]
 43%|████▎     | 95702/225000 [173:43:29<489:48:40, 13.64s/it]
 43%|████▎     | 95703/225000 [173:43:40<461:12:59, 12.84s/it]
 43%|████▎     | 95704/225000 [173:43:58<518:04:57, 14.43s/it]
 43%|████▎     | 95705/225000 [173:44:07<459:23:07, 12.79s/it]
 43%|████▎     | 95706/225000 [173:44:32<592:26:22, 16.50s/it]
 43%|████▎     | 95707/225000 [173:44:44<542:36:28, 15.11s/it]
 43%|████▎     | 95708/225000 [173:44:54<488:53:05, 13.61s/it]
 43%|████▎     | 95709/225000 [173:45:01<415:45:45, 11.58s/it]
 43%|████▎     | 95710/225000 [173:45:15<441:31:17, 12.29s/it]
 43%|████▎     | 95711/225000 [173:45:29<459:36:58, 12.80s/it]
 43%|████▎     | 95712/225000 [173:45:42<459:03:06, 12.78s/it]
 43%|████▎     | 95713/225000 [173:46:01<529:28:20, 14.74s/it]
 43%|████▎     | 95714/225000 [173:46:14<506:11:20, 14.09s/it]
 43%|████▎     | 95715/225000 [173:46:34<572:58:24, 15.95s/it]
 43%|████▎     | 95716/225000 [173:46:41<477:11:46, 13.29s/it]
 43%|████▎     | 95717/225000 [173:46:59<530:35:29, 14.77s/it]
 43%|████▎     | 95718/225000 [173:47:14<529:06:15, 14.73s/it]
 43%|████▎     | 95719/225000 [173:47:26<502:28:36, 13.99s/it]
 43%|████▎     | 95720/225000 [173:47:50<604:04:03, 16.82s/it]
 43%|████▎     | 95721/225000 [173:48:06<593:36:47, 16.53s/it]
 43%|████▎     | 95722/225000 [173:48:18<549:44:10, 15.31s/it]
 43%|████▎     | 95723/225000 [173:48:38<597:48:42, 16.65s/it]
 43%|████▎     | 95724/225000 [173:48:49<534:31:27, 14.89s/it]
 43%|████▎     | 95725/225000 [173:48:53<426:08:19, 11.87s/it]
 43%|████▎     | 95726/225000 [173:49:06<429:21:51, 11.96s/it]
 43%|████▎     | 95727/225000 [173:49:15<400:56:06, 11.17s/it]
 43%|████▎     | 95728/225000 [173:49:29<434:48:24, 12.11s/it]
 43%|████▎     | 95729/225000 [173:49:36<373:54:40, 10.41s/it]
 43%|████▎     | 95730/225000 [173:49:52<437:29:52, 12.18s/it]
 43%|████▎     | 95731/225000 [173:50:02<417:57:07, 11.64s/it]
 43%|████▎     | 95732/225000 [173:50:19<477:05:01, 13.29s/it]
 43%|████▎     | 95733/225000 [173:50:28<424:06:05, 11.81s/it]
 43%|████▎     | 95734/225000 [173:50:43<463:24:06, 12.91s/it]
 43%|████▎     | 95735/225000 [173:51:03<539:59:16, 15.04s/it]
 43%|████▎     | 95736/225000 [173:51:13<483:39:53, 13.47s/it]
 43%|████▎     | 95737/225000 [173:51:21<427:56:13, 11.92s/it]
 43%|████▎     | 95738/225000 [173:51:26<349:15:53,  9.73s/it]
 43%|████▎     | 95739/225000 [173:51:47<473:42:15, 13.19s/it]
 43%|████▎     | 95740/225000 [173:51:51<370:16:35, 10.31s/it]
 43%|████▎     | 95741/225000 [173:52:06<424:53:34, 11.83s/it]
 43%|████▎     | 95742/225000 [173:52:18<428:27:52, 11.93s/it]
 43%|████▎     | 95743/225000 [173:52:31<437:42:54, 12.19s/it]
 43%|████▎     | 95744/225000 [173:52:45<458:54:06, 12.78s/it]
 43%|████▎     | 95745/225000 [173:52:56<435:06:54, 12.12s/it]
 43%|████▎     | 95746/225000 [173:53:10<459:30:53, 12.80s/it]
 43%|████▎     | 95747/225000 [173:53:20<421:48:29, 11.75s/it]
 43%|████▎     | 95748/225000 [173:53:31<415:10:44, 11.56s/it]
 43%|████▎     | 95749/225000 [173:53:53<528:15:10, 14.71s/it]
 43%|████▎     | 95750/225000 [173:54:12<571:54:34, 15.93s/it]
                                                              
{'loss': '0.8226', 'grad_norm': '1.599', 'learning_rate': '3.283e-05', 'epoch': '0.4256', 'num_input_tokens_seen': 285048944, 'train_runtime': '6.261e+05', 'train_tokens_per_second': '455.3'}

 43%|████▎     | 95750/225000 [173:54:12<571:54:34, 15.93s/it]
 43%|████▎     | 95751/225000 [173:54:18<466:01:31, 12.98s/it]
 43%|████▎     | 95752/225000 [173:54:28<442:16:12, 12.32s/it]
 43%|████▎     | 95753/225000 [173:54:43<469:05:02, 13.07s/it]
 43%|████▎     | 95754/225000 [173:54:50<396:58:50, 11.06s/it]
 43%|████▎     | 95755/225000 [173:54:55<338:56:10,  9.44s/it]
 43%|████▎     | 95756/225000 [173:55:17<474:28:34, 13.22s/it]
 43%|████▎     | 95757/225000 [173:55:30<470:05:13, 13.09s/it]
 43%|████▎     | 95758/225000 [173:55:41<447:16:42, 12.46s/it]
 43%|████▎     | 95759/225000 [173:55:50<408:18:00, 11.37s/it]
 43%|████▎     | 95760/225000 [173:55:58<377:33:29, 10.52s/it]
 43%|████▎     | 95761/225000 [173:56:14<431:48:34, 12.03s/it]
 43%|████▎     | 95762/225000 [173:56:22<393:09:08, 10.95s/it]
 43%|████▎     | 95763/225000 [173:56:40<465:33:45, 12.97s/it]
 43%|████▎     | 95764/225000 [173:56:48<410:30:35, 11.44s/it]
 43%|████▎     | 95765/225000 [173:57:04<456:53:32, 12.73s/it]
 43%|████▎     | 95766/225000 [173:57:16<447:34:30, 12.47s/it]
 43%|████▎     | 95767/225000 [173:57:26<425:16:36, 11.85s/it]
 43%|████▎     | 95768/225000 [173:57:41<459:51:57, 12.81s/it]
 43%|████▎     | 95769/225000 [173:57:48<394:50:35, 11.00s/it]
 43%|████▎     | 95770/225000 [173:58:01<420:32:11, 11.72s/it]
 43%|████▎     | 95771/225000 [173:58:15<444:56:02, 12.39s/it]
 43%|████▎     | 95772/225000 [173:58:37<547:59:47, 15.27s/it]
 43%|████▎     | 95773/225000 [173:58:53<558:16:23, 15.55s/it]
 43%|████▎     | 95774/225000 [173:59:14<611:48:02, 17.04s/it]
 43%|████▎     | 95775/225000 [173:59:33<632:49:31, 17.63s/it]
 43%|████▎     | 95776/225000 [173:59:49<620:40:29, 17.29s/it]
 43%|████▎     | 95777/225000 [174:00:06<617:49:13, 17.21s/it]
 43%|████▎     | 95778/225000 [174:00:10<475:08:20, 13.24s/it]
 43%|████▎     | 95779/225000 [174:00:25<492:38:11, 13.72s/it]
 43%|████▎     | 95780/225000 [174:00:30<395:37:17, 11.02s/it]
 43%|████▎     | 95781/225000 [174:00:44<431:11:48, 12.01s/it]
 43%|████▎     | 95782/225000 [174:01:02<490:01:40, 13.65s/it]
 43%|████▎     | 95783/225000 [174:01:22<562:27:03, 15.67s/it]
 43%|████▎     | 95784/225000 [174:01:28<453:54:22, 12.65s/it]
 43%|████▎     | 95785/225000 [174:01:48<536:34:25, 14.95s/it]
 43%|████▎     | 95786/225000 [174:01:57<468:21:59, 13.05s/it]
 43%|████▎     | 95787/225000 [174:02:04<406:24:57, 11.32s/it]
 43%|████▎     | 95788/225000 [174:02:25<506:21:22, 14.11s/it]
 43%|████▎     | 95789/225000 [174:02:39<508:28:57, 14.17s/it]
 43%|████▎     | 95790/225000 [174:02:57<546:07:24, 15.22s/it]
 43%|████▎     | 95791/225000 [174:03:05<478:13:59, 13.32s/it]
 43%|████▎     | 95792/225000 [174:03:17<456:39:29, 12.72s/it]
 43%|████▎     | 95793/225000 [174:03:34<503:02:13, 14.02s/it]
 43%|████▎     | 95794/225000 [174:03:40<415:22:05, 11.57s/it]
 43%|████▎     | 95795/225000 [174:03:56<463:21:48, 12.91s/it]
 43%|████▎     | 95796/225000 [174:04:02<389:03:56, 10.84s/it]
 43%|████▎     | 95797/225000 [174:04:20<471:53:10, 13.15s/it]
 43%|████▎     | 95798/225000 [174:04:31<444:52:14, 12.40s/it]
 43%|████▎     | 95799/225000 [174:04:35<350:09:37,  9.76s/it]
 43%|████▎     | 95800/225000 [174:04:45<360:11:07, 10.04s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '2', 'learning_rate': '3.281e-05', 'epoch': '0.4258', 'num_input_tokens_seen': 285361344, 'train_runtime': '6.267e+05', 'train_tokens_per_second': '455.4'}

 43%|████▎     | 95800/225000 [174:04:45<360:11:07, 10.04s/it]
 43%|████▎     | 95801/225000 [174:04:53<332:27:51,  9.26s/it]
 43%|████▎     | 95802/225000 [174:05:13<452:22:51, 12.61s/it]
 43%|████▎     | 95803/225000 [174:05:31<505:19:10, 14.08s/it]
 43%|████▎     | 95804/225000 [174:05:49<555:00:56, 15.47s/it]
 43%|████▎     | 95805/225000 [174:06:07<579:35:39, 16.15s/it]
 43%|████▎     | 95806/225000 [174:06:21<556:25:25, 15.50s/it]
 43%|████▎     | 95807/225000 [174:06:35<535:51:43, 14.93s/it]
 43%|████▎     | 95808/225000 [174:07:02<674:37:15, 18.80s/it]
 43%|████▎     | 95809/225000 [174:07:16<620:06:12, 17.28s/it]
 43%|████▎     | 95810/225000 [174:07:30<583:19:04, 16.25s/it]
 43%|████▎     | 95811/225000 [174:07:47<593:33:07, 16.54s/it]
 43%|████▎     | 95812/225000 [174:08:00<553:52:21, 15.43s/it]
 43%|████▎     | 95813/225000 [174:08:14<538:23:58, 15.00s/it]
 43%|████▎     | 95814/225000 [174:08:34<595:00:51, 16.58s/it]
 43%|████▎     | 95815/225000 [174:08:47<547:21:22, 15.25s/it]
 43%|████▎     | 95816/225000 [174:08:52<439:32:54, 12.25s/it]
 43%|████▎     | 95817/225000 [174:09:01<407:43:11, 11.36s/it]
 43%|████▎     | 95818/225000 [174:09:19<483:37:09, 13.48s/it]
 43%|████▎     | 95819/225000 [174:09:31<460:58:54, 12.85s/it]
 43%|████▎     | 95820/225000 [174:09:55<578:03:15, 16.11s/it]
 43%|████▎     | 95821/225000 [174:10:08<546:41:00, 15.24s/it]
 43%|████▎     | 95822/225000 [174:10:21<521:00:45, 14.52s/it]
 43%|████▎     | 95823/225000 [174:10:36<529:12:33, 14.75s/it]
 43%|████▎     | 95824/225000 [174:10:53<555:46:25, 15.49s/it]
 43%|████▎     | 95825/225000 [174:11:04<507:35:25, 14.15s/it]
 43%|████▎     | 95826/225000 [174:11:21<542:11:17, 15.11s/it]
 43%|████▎     | 95827/225000 [174:11:34<516:06:07, 14.38s/it]
 43%|████▎     | 95828/225000 [174:11:47<496:35:41, 13.84s/it]
 43%|████▎     | 95829/225000 [174:12:03<518:24:14, 14.45s/it]
 43%|████▎     | 95830/225000 [174:12:16<504:22:53, 14.06s/it]
 43%|████▎     | 95831/225000 [174:12:24<447:11:45, 12.46s/it]
 43%|████▎     | 95832/225000 [174:12:43<511:20:14, 14.25s/it]
 43%|████▎     | 95833/225000 [174:13:01<547:34:57, 15.26s/it]
 43%|████▎     | 95834/225000 [174:13:10<489:33:45, 13.64s/it]
 43%|████▎     | 95835/225000 [174:13:26<510:42:47, 14.23s/it]
 43%|████▎     | 95836/225000 [174:13:31<415:43:23, 11.59s/it]
 43%|████▎     | 95837/225000 [174:13:44<430:04:41, 11.99s/it]
 43%|████▎     | 95838/225000 [174:13:54<409:44:40, 11.42s/it]
 43%|████▎     | 95839/225000 [174:14:11<469:52:23, 13.10s/it]
 43%|████▎     | 95840/225000 [174:14:25<479:00:58, 13.35s/it]
 43%|████▎     | 95841/225000 [174:14:41<506:32:07, 14.12s/it]
 43%|████▎     | 95842/225000 [174:14:58<539:42:10, 15.04s/it]
 43%|████▎     | 95843/225000 [174:15:06<461:02:21, 12.85s/it]
 43%|████▎     | 95844/225000 [174:15:20<476:14:33, 13.27s/it]
 43%|████▎     | 95845/225000 [174:15:34<474:00:54, 13.21s/it]
 43%|████▎     | 95846/225000 [174:15:51<518:10:40, 14.44s/it]
 43%|████▎     | 95847/225000 [174:16:14<616:19:47, 17.18s/it]
 43%|████▎     | 95848/225000 [174:16:30<595:44:54, 16.61s/it]
 43%|████▎     | 95849/225000 [174:16:42<552:35:35, 15.40s/it]
 43%|████▎     | 95850/225000 [174:17:03<611:19:05, 17.04s/it]
                                                              
{'loss': '0.8333', 'grad_norm': '1.168', 'learning_rate': '3.279e-05', 'epoch': '0.426', 'num_input_tokens_seen': 285716296, 'train_runtime': '6.274e+05', 'train_tokens_per_second': '455.4'}

 43%|████▎     | 95850/225000 [174:17:03<611:19:05, 17.04s/it]
 43%|████▎     | 95851/225000 [174:17:19<600:18:49, 16.73s/it]
 43%|████▎     | 95852/225000 [174:17:37<615:24:31, 17.15s/it]
 43%|████▎     | 95853/225000 [174:17:55<625:33:37, 17.44s/it]
 43%|████▎     | 95854/225000 [174:18:04<527:10:06, 14.70s/it]
 43%|████▎     | 95855/225000 [174:18:23<578:02:33, 16.11s/it]
 43%|████▎     | 95856/225000 [174:18:32<504:52:27, 14.07s/it]
 43%|████▎     | 95857/225000 [174:18:47<513:26:52, 14.31s/it]
 43%|████▎     | 95858/225000 [174:18:54<429:16:32, 11.97s/it]
 43%|████▎     | 95859/225000 [174:19:02<388:42:51, 10.84s/it]
 43%|████▎     | 95860/225000 [174:19:23<501:51:05, 13.99s/it]
 43%|████▎     | 95861/225000 [174:19:36<484:52:58, 13.52s/it]
 43%|████▎     | 95862/225000 [174:19:49<477:32:35, 13.31s/it]
 43%|████▎     | 95863/225000 [174:20:12<589:34:05, 16.44s/it]
 43%|████▎     | 95864/225000 [174:20:21<504:50:26, 14.07s/it]
 43%|████▎     | 95865/225000 [174:20:35<500:19:13, 13.95s/it]
 43%|████▎     | 95866/225000 [174:20:55<574:49:28, 16.02s/it]
 43%|████▎     | 95867/225000 [174:21:03<486:37:00, 13.57s/it]
 43%|████▎     | 95868/225000 [174:21:19<515:04:18, 14.36s/it]
 43%|████▎     | 95869/225000 [174:21:28<455:17:51, 12.69s/it]
 43%|████▎     | 95870/225000 [174:21:42<469:04:16, 13.08s/it]
 43%|████▎     | 95871/225000 [174:21:52<436:57:00, 12.18s/it]
 43%|████▎     | 95872/225000 [174:22:09<486:17:42, 13.56s/it]
 43%|████▎     | 95873/225000 [174:22:17<427:00:39, 11.90s/it]
 43%|████▎     | 95874/225000 [174:22:36<501:33:02, 13.98s/it]
 43%|████▎     | 95875/225000 [174:22:41<405:39:43, 11.31s/it]
 43%|████▎     | 95876/225000 [174:22:52<397:15:51, 11.08s/it]
 43%|████▎     | 95877/225000 [174:22:58<348:32:55,  9.72s/it]
 43%|████▎     | 95878/225000 [174:23:22<499:26:01, 13.92s/it]
 43%|████▎     | 95879/225000 [174:23:39<538:19:28, 15.01s/it]
 43%|████▎     | 95880/225000 [174:23:52<512:49:12, 14.30s/it]
 43%|████▎     | 95881/225000 [174:24:04<486:34:53, 13.57s/it]
 43%|████▎     | 95882/225000 [174:24:28<605:05:26, 16.87s/it]
 43%|████▎     | 95883/225000 [174:24:35<498:26:31, 13.90s/it]
 43%|████▎     | 95884/225000 [174:24:57<580:57:26, 16.20s/it]
 43%|████▎     | 95885/225000 [174:25:14<585:28:24, 16.32s/it]
 43%|████▎     | 95886/225000 [174:25:29<570:52:48, 15.92s/it]
 43%|████▎     | 95887/225000 [174:25:40<523:58:00, 14.61s/it]
 43%|████▎     | 95888/225000 [174:25:46<432:14:12, 12.05s/it]
 43%|████▎     | 95889/225000 [174:26:04<492:47:59, 13.74s/it]
 43%|████▎     | 95890/225000 [174:26:22<535:43:16, 14.94s/it]
 43%|████▎     | 95891/225000 [174:26:41<583:41:48, 16.28s/it]
 43%|████▎     | 95892/225000 [174:26:53<536:13:32, 14.95s/it]
 43%|████▎     | 95893/225000 [174:26:58<428:32:58, 11.95s/it]
 43%|████▎     | 95894/225000 [174:27:14<473:16:51, 13.20s/it]
 43%|████▎     | 95895/225000 [174:27:28<479:03:21, 13.36s/it]
 43%|████▎     | 95896/225000 [174:27:41<473:34:20, 13.21s/it]
 43%|████▎     | 95897/225000 [174:27:49<422:46:44, 11.79s/it]
 43%|████▎     | 95898/225000 [174:28:06<480:42:44, 13.40s/it]
 43%|████▎     | 95899/225000 [174:28:17<450:03:49, 12.55s/it]
 43%|████▎     | 95900/225000 [174:28:24<389:10:13, 10.85s/it]
                                                              
{'loss': '0.8295', 'grad_norm': '1.814', 'learning_rate': '3.278e-05', 'epoch': '0.4262', 'num_input_tokens_seen': 286062800, 'train_runtime': '6.281e+05', 'train_tokens_per_second': '455.4'}

 43%|████▎     | 95900/225000 [174:28:24<389:10:13, 10.85s/it]
 43%|████▎     | 95901/225000 [174:28:38<430:49:06, 12.01s/it]
 43%|████▎     | 95902/225000 [174:28:51<442:13:22, 12.33s/it]
 43%|████▎     | 95903/225000 [174:29:13<537:39:47, 14.99s/it]
 43%|████▎     | 95904/225000 [174:29:21<469:56:52, 13.11s/it]
 43%|████▎     | 95905/225000 [174:29:37<494:19:35, 13.79s/it]
 43%|████▎     | 95906/225000 [174:29:55<542:53:54, 15.14s/it]
 43%|████▎     | 95907/225000 [174:30:13<577:07:19, 16.09s/it]
 43%|████▎     | 95908/225000 [174:30:27<553:36:30, 15.44s/it]
 43%|████▎     | 95909/225000 [174:30:47<602:45:55, 16.81s/it]
 43%|████▎     | 95910/225000 [174:31:05<615:04:15, 17.15s/it]
 43%|████▎     | 95911/225000 [174:31:24<629:26:04, 17.55s/it]
 43%|████▎     | 95912/225000 [174:31:32<524:31:27, 14.63s/it]
 43%|████▎     | 95913/225000 [174:31:36<419:10:37, 11.69s/it]
 43%|████▎     | 95914/225000 [174:31:52<466:20:43, 13.01s/it]
 43%|████▎     | 95915/225000 [174:32:07<482:58:21, 13.47s/it]
 43%|████▎     | 95916/225000 [174:32:23<508:02:20, 14.17s/it]
 43%|████▎     | 95917/225000 [174:32:44<585:48:33, 16.34s/it]
 43%|████▎     | 95918/225000 [174:32:53<509:29:25, 14.21s/it]
 43%|████▎     | 95919/225000 [174:33:00<431:24:11, 12.03s/it]
 43%|████▎     | 95920/225000 [174:33:07<367:56:44, 10.26s/it]
 43%|████▎     | 95921/225000 [174:33:23<435:43:51, 12.15s/it]
 43%|████▎     | 95922/225000 [174:33:33<411:52:47, 11.49s/it]
 43%|████▎     | 95923/225000 [174:33:49<457:32:40, 12.76s/it]
 43%|████▎     | 95924/225000 [174:34:02<466:53:46, 13.02s/it]
 43%|████▎     | 95925/225000 [174:34:18<495:48:46, 13.83s/it]
 43%|████▎     | 95926/225000 [174:34:33<509:46:28, 14.22s/it]
 43%|████▎     | 95927/225000 [174:34:50<538:13:25, 15.01s/it]
 43%|████▎     | 95928/225000 [174:35:11<604:17:36, 16.85s/it]
 43%|████▎     | 95929/225000 [174:35:30<622:29:34, 17.36s/it]
 43%|████▎     | 95930/225000 [174:35:42<562:40:35, 15.69s/it]
 43%|████▎     | 95931/225000 [174:36:03<621:42:50, 17.34s/it]
 43%|████▎     | 95932/225000 [174:36:20<616:27:04, 17.19s/it]
 43%|████▎     | 95933/225000 [174:36:38<630:03:30, 17.57s/it]
 43%|████▎     | 95934/225000 [174:36:50<564:34:01, 15.75s/it]
 43%|████▎     | 95935/225000 [174:37:13<643:18:56, 17.94s/it]
 43%|████▎     | 95936/225000 [174:37:29<623:00:39, 17.38s/it]
 43%|████▎     | 95937/225000 [174:37:38<539:00:59, 15.03s/it]
 43%|████▎     | 95938/225000 [174:37:49<492:12:39, 13.73s/it]
 43%|████▎     | 95939/225000 [174:38:04<508:49:51, 14.19s/it]
 43%|████▎     | 95940/225000 [174:38:27<596:31:19, 16.64s/it]
 43%|████▎     | 95941/225000 [174:38:39<547:29:06, 15.27s/it]
 43%|████▎     | 95942/225000 [174:38:59<604:36:36, 16.87s/it]
 43%|████▎     | 95943/225000 [174:39:14<580:00:38, 16.18s/it]
 43%|████▎     | 95944/225000 [174:39:23<506:30:28, 14.13s/it]
 43%|████▎     | 95945/225000 [174:39:34<469:37:30, 13.10s/it]
 43%|████▎     | 95946/225000 [174:39:44<436:47:50, 12.18s/it]
 43%|████▎     | 95947/225000 [174:39:57<450:04:59, 12.56s/it]
 43%|████▎     | 95948/225000 [174:40:02<360:30:35, 10.06s/it]
 43%|████▎     | 95949/225000 [174:40:18<430:23:23, 12.01s/it]
 43%|████▎     | 95950/225000 [174:40:30<427:34:54, 11.93s/it]
                                                              
{'loss': '0.8262', 'grad_norm': '1.717', 'learning_rate': '3.276e-05', 'epoch': '0.4264', 'num_input_tokens_seen': 286428640, 'train_runtime': '6.288e+05', 'train_tokens_per_second': '455.5'}

 43%|████▎     | 95950/225000 [174:40:30<427:34:54, 11.93s/it]
 43%|████▎     | 95951/225000 [174:40:38<385:20:43, 10.75s/it]
 43%|████▎     | 95952/225000 [174:40:44<337:54:43,  9.43s/it]
 43%|████▎     | 95953/225000 [174:41:00<404:10:00, 11.27s/it]
 43%|████▎     | 95954/225000 [174:41:19<490:50:16, 13.69s/it]
 43%|████▎     | 95955/225000 [174:41:39<562:22:34, 15.69s/it]
 43%|████▎     | 95956/225000 [174:41:52<534:01:49, 14.90s/it]
 43%|████▎     | 95957/225000 [174:42:13<592:31:11, 16.53s/it]
 43%|████▎     | 95958/225000 [174:42:32<626:08:33, 17.47s/it]
 43%|████▎     | 95959/225000 [174:42:41<533:27:04, 14.88s/it]
 43%|████▎     | 95960/225000 [174:43:03<607:03:41, 16.94s/it]
 43%|████▎     | 95961/225000 [174:43:16<564:40:42, 15.75s/it]
 43%|████▎     | 95962/225000 [174:43:32<569:16:54, 15.88s/it]
 43%|████▎     | 95963/225000 [174:44:01<711:14:08, 19.84s/it]
 43%|████▎     | 95964/225000 [174:44:13<623:48:05, 17.40s/it]
 43%|████▎     | 95965/225000 [174:44:28<592:28:30, 16.53s/it]
 43%|████▎     | 95966/225000 [174:44:38<525:36:46, 14.66s/it]
 43%|████▎     | 95967/225000 [174:44:50<500:12:36, 13.96s/it]
 43%|████▎     | 95968/225000 [174:45:12<585:36:59, 16.34s/it]
 43%|████▎     | 95969/225000 [174:45:31<617:02:43, 17.22s/it]
 43%|████▎     | 95970/225000 [174:45:45<581:26:51, 16.22s/it]
 43%|████▎     | 95971/225000 [174:46:03<603:20:13, 16.83s/it]
 43%|████▎     | 95972/225000 [174:46:16<561:37:28, 15.67s/it]
 43%|████▎     | 95973/225000 [174:46:33<572:58:32, 15.99s/it]
 43%|████▎     | 95974/225000 [174:46:50<578:46:49, 16.15s/it]
 43%|████▎     | 95975/225000 [174:47:06<576:42:51, 16.09s/it]
 43%|████▎     | 95976/225000 [174:47:25<616:02:44, 17.19s/it]
 43%|████▎     | 95977/225000 [174:47:37<551:30:00, 15.39s/it]
 43%|████▎     | 95978/225000 [174:47:50<527:10:43, 14.71s/it]
 43%|████▎     | 95979/225000 [174:48:02<499:40:58, 13.94s/it]
 43%|████▎     | 95980/225000 [174:48:19<536:19:14, 14.96s/it]
 43%|████▎     | 95981/225000 [174:48:29<476:57:36, 13.31s/it]
 43%|████▎     | 95982/225000 [174:48:44<501:42:49, 14.00s/it]
 43%|████▎     | 95983/225000 [174:48:57<486:54:25, 13.59s/it]
 43%|████▎     | 95984/225000 [174:49:09<475:06:34, 13.26s/it]
 43%|████▎     | 95985/225000 [174:49:22<471:41:56, 13.16s/it]
 43%|████▎     | 95986/225000 [174:49:32<429:32:55, 11.99s/it]
 43%|████▎     | 95987/225000 [174:50:02<633:17:50, 17.67s/it]
 43%|████▎     | 95988/225000 [174:50:15<581:37:49, 16.23s/it]
 43%|████▎     | 95989/225000 [174:50:28<542:52:29, 15.15s/it]
 43%|████▎     | 95990/225000 [174:50:39<495:32:45, 13.83s/it]
 43%|████▎     | 95991/225000 [174:50:55<521:23:01, 14.55s/it]
 43%|████▎     | 95992/225000 [174:51:07<499:26:16, 13.94s/it]
 43%|████▎     | 95993/225000 [174:51:15<433:39:23, 12.10s/it]
 43%|████▎     | 95994/225000 [174:51:24<399:41:58, 11.15s/it]
 43%|████▎     | 95995/225000 [174:51:40<446:23:07, 12.46s/it]
 43%|████▎     | 95996/225000 [174:51:50<423:54:40, 11.83s/it]
 43%|████▎     | 95997/225000 [174:52:05<461:49:53, 12.89s/it]
 43%|████▎     | 95998/225000 [174:52:13<399:32:04, 11.15s/it]
 43%|████▎     | 95999/225000 [174:52:33<503:27:12, 14.05s/it]
 43%|████▎     | 96000/225000 [174:52:38<404:37:24, 11.29s/it]
                                                              
{'loss': '0.8659', 'grad_norm': '2.234', 'learning_rate': '3.274e-05', 'epoch': '0.4267', 'num_input_tokens_seen': 286803432, 'train_runtime': '6.296e+05', 'train_tokens_per_second': '455.6'}

 43%|████▎     | 96000/225000 [174:52:38<404:37:24, 11.29s/it]
 43%|████▎     | 96001/225000 [174:52:50<409:35:05, 11.43s/it]
 43%|████▎     | 96002/225000 [174:52:58<373:19:41, 10.42s/it]
 43%|████▎     | 96003/225000 [174:53:08<370:01:06, 10.33s/it]
 43%|████▎     | 96004/225000 [174:53:29<487:33:13, 13.61s/it]
 43%|████▎     | 96005/225000 [174:53:38<429:14:23, 11.98s/it]
 43%|████▎     | 96006/225000 [174:53:55<487:18:00, 13.60s/it]
 43%|████▎     | 96007/225000 [174:54:09<486:51:49, 13.59s/it]
 43%|████▎     | 96008/225000 [174:54:28<553:10:06, 15.44s/it]
 43%|████▎     | 96009/225000 [174:54:34<445:49:32, 12.44s/it]
 43%|████▎     | 96010/225000 [174:54:42<405:40:32, 11.32s/it]
 43%|████▎     | 96011/225000 [174:54:51<380:18:32, 10.61s/it]
 43%|████▎     | 96012/225000 [174:55:08<442:01:37, 12.34s/it]
 43%|████▎     | 96013/225000 [174:55:20<444:44:49, 12.41s/it]
 43%|████▎     | 96014/225000 [174:55:39<511:45:30, 14.28s/it]
 43%|████▎     | 96015/225000 [174:55:55<528:20:27, 14.75s/it]
 43%|████▎     | 96016/225000 [174:56:01<438:31:41, 12.24s/it]
 43%|████▎     | 96017/225000 [174:56:20<509:14:54, 14.21s/it]
 43%|████▎     | 96018/225000 [174:56:40<574:18:45, 16.03s/it]
 43%|████▎     | 96019/225000 [174:56:50<501:39:38, 14.00s/it]
 43%|████▎     | 96020/225000 [174:57:08<544:10:14, 15.19s/it]
 43%|████▎     | 96021/225000 [174:57:22<532:28:29, 14.86s/it]
 43%|████▎     | 96022/225000 [174:57:39<555:37:31, 15.51s/it]
 43%|████▎     | 96023/225000 [174:57:50<510:53:00, 14.26s/it]
 43%|████▎     | 96024/225000 [174:58:07<545:22:38, 15.22s/it]
 43%|████▎     | 96025/225000 [174:58:37<697:21:48, 19.47s/it]
 43%|████▎     | 96026/225000 [174:58:43<553:37:55, 15.45s/it]
 43%|████▎     | 96027/225000 [174:58:56<527:32:26, 14.73s/it]
 43%|████▎     | 96028/225000 [174:59:06<481:59:15, 13.45s/it]
 43%|████▎     | 96029/225000 [174:59:16<441:17:22, 12.32s/it]
 43%|████▎     | 96030/225000 [174:59:24<389:11:51, 10.86s/it]
 43%|████▎     | 96031/225000 [174:59:40<453:12:07, 12.65s/it]
 43%|████▎     | 96032/225000 [174:59:52<436:52:01, 12.19s/it]
 43%|████▎     | 96033/225000 [175:00:10<499:22:56, 13.94s/it]
 43%|████▎     | 96034/225000 [175:00:23<491:45:57, 13.73s/it]
 43%|████▎     | 96035/225000 [175:00:43<564:23:53, 15.75s/it]
 43%|████▎     | 96036/225000 [175:00:53<500:27:08, 13.97s/it]
 43%|████▎     | 96037/225000 [175:01:17<605:03:25, 16.89s/it]
 43%|████▎     | 96038/225000 [175:01:32<589:00:27, 16.44s/it]
 43%|████▎     | 96039/225000 [175:01:57<679:35:51, 18.97s/it]
 43%|████▎     | 96040/225000 [175:02:12<636:34:36, 17.77s/it]
 43%|████▎     | 96041/225000 [175:02:30<640:07:37, 17.87s/it]
 43%|████▎     | 96042/225000 [175:02:44<597:59:02, 16.69s/it]
 43%|████▎     | 96043/225000 [175:03:07<661:48:51, 18.48s/it]
 43%|████▎     | 96044/225000 [175:03:23<636:39:54, 17.77s/it]
 43%|████▎     | 96045/225000 [175:03:42<653:03:40, 18.23s/it]
 43%|████▎     | 96046/225000 [175:04:01<657:00:59, 18.34s/it]
 43%|████▎     | 96047/225000 [175:04:13<592:40:34, 16.55s/it]
 43%|████▎     | 96048/225000 [175:04:28<575:48:27, 16.08s/it]
 43%|████▎     | 96049/225000 [175:04:52<657:54:39, 18.37s/it]
 43%|████▎     | 96050/225000 [175:05:10<661:05:13, 18.46s/it]
                                                              
{'loss': '0.8902', 'grad_norm': '1.472', 'learning_rate': '3.272e-05', 'epoch': '0.4269', 'num_input_tokens_seen': 287175520, 'train_runtime': '6.303e+05', 'train_tokens_per_second': '455.6'}

 43%|████▎     | 96050/225000 [175:05:10<661:05:13, 18.46s/it]
 43%|████▎     | 96051/225000 [175:05:23<594:07:35, 16.59s/it]
 43%|████▎     | 96052/225000 [175:05:28<471:53:37, 13.17s/it]
 43%|████▎     | 96053/225000 [175:05:36<415:58:47, 11.61s/it]
 43%|████▎     | 96054/225000 [175:05:55<500:59:25, 13.99s/it]
 43%|████▎     | 96055/225000 [175:06:00<404:16:53, 11.29s/it]
 43%|████▎     | 96056/225000 [175:06:14<428:44:11, 11.97s/it]
 43%|████▎     | 96057/225000 [175:06:25<416:34:21, 11.63s/it]
 43%|████▎     | 96058/225000 [175:06:51<574:00:37, 16.03s/it]
 43%|████▎     | 96059/225000 [175:07:12<623:42:25, 17.41s/it]
 43%|████▎     | 96060/225000 [175:07:30<631:18:23, 17.63s/it]
 43%|████▎     | 96061/225000 [175:07:48<632:41:01, 17.66s/it]
 43%|████▎     | 96062/225000 [175:08:05<630:32:00, 17.60s/it]
 43%|████▎     | 96063/225000 [175:08:13<526:35:19, 14.70s/it]
 43%|████▎     | 96064/225000 [175:08:34<593:21:29, 16.57s/it]
 43%|████▎     | 96065/225000 [175:08:48<563:12:32, 15.73s/it]
 43%|████▎     | 96066/225000 [175:09:11<649:58:46, 18.15s/it]
 43%|████▎     | 96067/225000 [175:09:24<590:57:51, 16.50s/it]
 43%|████▎     | 96068/225000 [175:09:41<600:21:51, 16.76s/it]
 43%|████▎     | 96069/225000 [175:09:49<505:09:56, 14.11s/it]
 43%|████▎     | 96070/225000 [175:10:03<496:19:28, 13.86s/it]
 43%|████▎     | 96071/225000 [175:10:14<467:21:00, 13.05s/it]
 43%|████▎     | 96072/225000 [175:10:32<518:47:54, 14.49s/it]
 43%|████▎     | 96073/225000 [175:10:46<512:25:46, 14.31s/it]
 43%|████▎     | 96074/225000 [175:11:07<591:27:16, 16.52s/it]
 43%|████▎     | 96075/225000 [175:11:22<577:32:39, 16.13s/it]
 43%|████▎     | 96076/225000 [175:11:36<549:16:12, 15.34s/it]
 43%|████▎     | 96077/225000 [175:11:48<518:08:53, 14.47s/it]
 43%|████▎     | 96078/225000 [175:11:58<466:23:56, 13.02s/it]
 43%|████▎     | 96079/225000 [175:12:18<542:35:19, 15.15s/it]
 43%|████▎     | 96080/225000 [175:12:31<513:32:39, 14.34s/it]
 43%|████▎     | 96081/225000 [175:12:50<573:03:02, 16.00s/it]
 43%|████▎     | 96082/225000 [175:13:05<557:26:50, 15.57s/it]
 43%|████▎     | 96083/225000 [175:13:18<526:19:43, 14.70s/it]
 43%|████▎     | 96084/225000 [175:13:43<637:24:44, 17.80s/it]
 43%|████▎     | 96085/225000 [175:13:58<609:06:17, 17.01s/it]
 43%|████▎     | 96086/225000 [175:14:07<521:04:31, 14.55s/it]
 43%|████▎     | 96087/225000 [175:14:24<554:49:44, 15.49s/it]
 43%|████▎     | 96088/225000 [175:14:37<521:20:34, 14.56s/it]
 43%|████▎     | 96089/225000 [175:14:54<550:58:43, 15.39s/it]
 43%|████▎     | 96090/225000 [175:15:07<522:42:59, 14.60s/it]
 43%|████▎     | 96091/225000 [175:15:21<521:36:40, 14.57s/it]
 43%|████▎     | 96092/225000 [175:15:36<521:01:13, 14.55s/it]
 43%|████▎     | 96093/225000 [175:15:45<463:04:38, 12.93s/it]
 43%|████▎     | 96094/225000 [175:15:58<461:59:20, 12.90s/it]
 43%|████▎     | 96095/225000 [175:16:11<468:31:02, 13.08s/it]
 43%|████▎     | 96096/225000 [175:16:28<509:54:28, 14.24s/it]
 43%|████▎     | 96097/225000 [175:16:43<519:51:57, 14.52s/it]
 43%|████▎     | 96098/225000 [175:16:59<532:08:48, 14.86s/it]
 43%|████▎     | 96099/225000 [175:17:20<595:00:15, 16.62s/it]
 43%|████▎     | 96100/225000 [175:17:31<540:25:25, 15.09s/it]
                                                              
{'loss': '0.893', 'grad_norm': '1.276', 'learning_rate': '3.271e-05', 'epoch': '0.4271', 'num_input_tokens_seen': 287550712, 'train_runtime': '6.311e+05', 'train_tokens_per_second': '455.7'}

 43%|████▎     | 96100/225000 [175:17:31<540:25:25, 15.09s/it]
 43%|████▎     | 96101/225000 [175:17:53<605:30:17, 16.91s/it]
 43%|████▎     | 96102/225000 [175:18:09<598:39:23, 16.72s/it]
 43%|████▎     | 96103/225000 [175:18:30<644:31:26, 18.00s/it]
 43%|████▎     | 96104/225000 [175:18:44<604:53:47, 16.89s/it]
 43%|████▎     | 96105/225000 [175:19:02<611:32:29, 17.08s/it]
 43%|████▎     | 96106/225000 [175:19:14<565:48:45, 15.80s/it]
 43%|████▎     | 96107/225000 [175:19:25<505:04:28, 14.11s/it]
 43%|████▎     | 96108/225000 [175:19:44<567:09:50, 15.84s/it]
 43%|████▎     | 96109/225000 [175:19:57<528:35:40, 14.76s/it]
 43%|████▎     | 96110/225000 [175:20:09<499:01:49, 13.94s/it]
 43%|████▎     | 96111/225000 [175:20:34<616:29:52, 17.22s/it]
 43%|████▎     | 96112/225000 [175:20:42<526:44:26, 14.71s/it]
 43%|████▎     | 96113/225000 [175:20:59<547:40:54, 15.30s/it]
 43%|████▎     | 96114/225000 [175:21:13<527:29:30, 14.73s/it]
 43%|████▎     | 96115/225000 [175:21:23<476:13:24, 13.30s/it]
 43%|████▎     | 96116/225000 [175:21:36<474:22:42, 13.25s/it]
 43%|████▎     | 96117/225000 [175:21:44<422:12:07, 11.79s/it]
 43%|████▎     | 96118/225000 [175:22:07<544:51:57, 15.22s/it]
 43%|████▎     | 96119/225000 [175:22:23<545:40:59, 15.24s/it]
 43%|████▎     | 96120/225000 [175:22:38<548:49:12, 15.33s/it]
 43%|████▎     | 96121/225000 [175:22:51<524:42:43, 14.66s/it]
 43%|████▎     | 96122/225000 [175:23:04<509:42:40, 14.24s/it]
 43%|████▎     | 96123/225000 [175:23:16<485:29:17, 13.56s/it]
 43%|████▎     | 96124/225000 [175:23:29<471:09:57, 13.16s/it]
 43%|████▎     | 96125/225000 [175:23:41<458:34:29, 12.81s/it]
 43%|████▎     | 96126/225000 [175:23:58<508:09:10, 14.19s/it]
 43%|████▎     | 96127/225000 [175:24:15<537:11:49, 15.01s/it]
 43%|████▎     | 96128/225000 [175:24:27<510:18:48, 14.26s/it]
 43%|████▎     | 96129/225000 [175:24:47<570:39:03, 15.94s/it]
 43%|████▎     | 96130/225000 [175:25:11<648:23:39, 18.11s/it]
 43%|████▎     | 96131/225000 [175:25:23<586:58:13, 16.40s/it]
 43%|████▎     | 96132/225000 [175:25:46<659:55:03, 18.44s/it]
 43%|████▎     | 96133/225000 [175:25:57<579:41:29, 16.19s/it]
 43%|████▎     | 96134/225000 [175:26:07<517:31:04, 14.46s/it]
 43%|████▎     | 96135/225000 [175:26:30<609:03:20, 17.01s/it]
 43%|████▎     | 96136/225000 [175:26:46<593:53:10, 16.59s/it]
 43%|████▎     | 96137/225000 [175:27:00<565:56:11, 15.81s/it]
 43%|████▎     | 96138/225000 [175:27:16<566:05:02, 15.81s/it]
 43%|████▎     | 96139/225000 [175:27:35<602:17:10, 16.83s/it]
 43%|████▎     | 96140/225000 [175:27:49<569:27:35, 15.91s/it]
 43%|████▎     | 96141/225000 [175:28:03<547:33:41, 15.30s/it]
 43%|████▎     | 96142/225000 [175:28:22<586:11:00, 16.38s/it]
 43%|████▎     | 96143/225000 [175:28:38<588:28:29, 16.44s/it]
 43%|████▎     | 96144/225000 [175:28:54<584:05:00, 16.32s/it]
 43%|████▎     | 96145/225000 [175:29:11<588:01:39, 16.43s/it]
 43%|████▎     | 96146/225000 [175:29:23<545:44:28, 15.25s/it]
 43%|████▎     | 96147/225000 [175:29:41<569:05:14, 15.90s/it]
 43%|████▎     | 96148/225000 [175:29:48<475:13:38, 13.28s/it]
 43%|████▎     | 96149/225000 [175:30:02<480:55:24, 13.44s/it]
 43%|████▎     | 96150/225000 [175:30:11<439:30:59, 12.28s/it]
                                                              
{'loss': '0.8545', 'grad_norm': '1.763', 'learning_rate': '3.269e-05', 'epoch': '0.4273', 'num_input_tokens_seen': 287923392, 'train_runtime': '6.318e+05', 'train_tokens_per_second': '455.7'}

 43%|████▎     | 96150/225000 [175:30:11<439:30:59, 12.28s/it]
 43%|████▎     | 96151/225000 [175:30:26<461:56:23, 12.91s/it]
 43%|████▎     | 96152/225000 [175:30:44<522:01:11, 14.59s/it]
 43%|████▎     | 96153/225000 [175:31:09<630:59:55, 17.63s/it]
 43%|████▎     | 96154/225000 [175:31:25<616:07:24, 17.21s/it]
 43%|████▎     | 96155/225000 [175:31:50<701:22:45, 19.60s/it]
 43%|████▎     | 96156/225000 [175:31:59<582:18:57, 16.27s/it]
 43%|████▎     | 96157/225000 [175:32:21<642:04:43, 17.94s/it]
 43%|████▎     | 96158/225000 [175:32:31<561:59:14, 15.70s/it]
 43%|████▎     | 96159/225000 [175:32:50<591:56:17, 16.54s/it]
 43%|████▎     | 96160/225000 [175:33:02<544:20:42, 15.21s/it]
 43%|████▎     | 96161/225000 [175:33:12<495:42:35, 13.85s/it]
 43%|████▎     | 96162/225000 [175:33:34<573:16:30, 16.02s/it]
 43%|████▎     | 96163/225000 [175:33:46<538:20:30, 15.04s/it]
 43%|████▎     | 96164/225000 [175:34:03<560:28:13, 15.66s/it]
 43%|████▎     | 96165/225000 [175:34:22<591:25:21, 16.53s/it]
 43%|████▎     | 96166/225000 [175:34:46<668:27:26, 18.68s/it]
 43%|████▎     | 96167/225000 [175:35:01<634:23:06, 17.73s/it]
 43%|████▎     | 96168/225000 [175:35:19<632:08:35, 17.66s/it]
 43%|████▎     | 96169/225000 [175:35:28<537:23:56, 15.02s/it]
 43%|████▎     | 96170/225000 [175:35:44<556:09:16, 15.54s/it]
 43%|████▎     | 96171/225000 [175:35:55<506:14:36, 14.15s/it]
 43%|████▎     | 96172/225000 [175:36:06<467:33:09, 13.07s/it]
 43%|████▎     | 96173/225000 [175:36:24<523:11:36, 14.62s/it]
 43%|████▎     | 96174/225000 [175:36:45<590:02:59, 16.49s/it]
 43%|████▎     | 96175/225000 [175:37:05<628:20:55, 17.56s/it]
 43%|████▎     | 96176/225000 [175:37:14<542:50:26, 15.17s/it]
 43%|████▎     | 96177/225000 [175:37:41<661:24:27, 18.48s/it]
 43%|████▎     | 96178/225000 [175:37:48<537:52:38, 15.03s/it]
 43%|████▎     | 96179/225000 [175:37:58<492:14:54, 13.76s/it]
 43%|████▎     | 96180/225000 [175:38:20<571:22:51, 15.97s/it]
 43%|████▎     | 96181/225000 [175:38:32<530:56:05, 14.84s/it]
 43%|████▎     | 96182/225000 [175:38:53<597:04:43, 16.69s/it]
 43%|████▎     | 96183/225000 [175:39:05<550:02:22, 15.37s/it]
 43%|████▎     | 96184/225000 [175:39:27<621:22:09, 17.37s/it]
 43%|████▎     | 96185/225000 [175:39:34<505:53:46, 14.14s/it]
 43%|████▎     | 96186/225000 [175:39:40<422:15:44, 11.80s/it]
 43%|████▎     | 96187/225000 [175:40:07<590:11:37, 16.49s/it]
 43%|████▎     | 96188/225000 [175:40:24<594:26:39, 16.61s/it]
 43%|████▎     | 96189/225000 [175:40:37<554:41:31, 15.50s/it]
 43%|████▎     | 96190/225000 [175:40:57<596:39:34, 16.68s/it]
 43%|████▎     | 96191/225000 [175:41:14<602:09:16, 16.83s/it]
 43%|████▎     | 96192/225000 [175:41:30<597:16:50, 16.69s/it]
 43%|████▎     | 96193/225000 [175:41:36<484:47:33, 13.55s/it]
 43%|████▎     | 96194/225000 [175:41:59<581:47:39, 16.26s/it]
 43%|████▎     | 96195/225000 [175:42:16<591:47:42, 16.54s/it]
 43%|████▎     | 96196/225000 [175:42:35<617:21:16, 17.25s/it]
 43%|████▎     | 96197/225000 [175:42:56<659:33:48, 18.43s/it]
 43%|████▎     | 96198/225000 [175:43:08<588:55:08, 16.46s/it]
 43%|████▎     | 96199/225000 [175:43:20<535:21:07, 14.96s/it]
 43%|████▎     | 96200/225000 [175:43:29<478:15:37, 13.37s/it]
                                                              
{'loss': '0.8846', 'grad_norm': '1.775', 'learning_rate': '3.267e-05', 'epoch': '0.4276', 'num_input_tokens_seen': 288325392, 'train_runtime': '6.326e+05', 'train_tokens_per_second': '455.8'}

 43%|████▎     | 96200/225000 [175:43:29<478:15:37, 13.37s/it]
 43%|████▎     | 96201/225000 [175:43:48<534:00:55, 14.93s/it]
 43%|████▎     | 96202/225000 [175:44:02<525:46:13, 14.70s/it]
 43%|████▎     | 96203/225000 [175:44:17<526:01:37, 14.70s/it]
 43%|████▎     | 96204/225000 [175:44:35<558:46:22, 15.62s/it]
 43%|████▎     | 96205/225000 [175:44:55<606:13:44, 16.94s/it]
 43%|████▎     | 96206/225000 [175:45:14<635:20:33, 17.76s/it]
 43%|████▎     | 96207/225000 [175:45:31<624:26:35, 17.45s/it]
 43%|████▎     | 96208/225000 [175:45:47<608:15:50, 17.00s/it]
 43%|████▎     | 96209/225000 [175:45:54<500:49:56, 14.00s/it]
 43%|████▎     | 96210/225000 [175:46:11<534:37:52, 14.94s/it]
 43%|████▎     | 96211/225000 [175:46:26<536:37:21, 15.00s/it]
 43%|████▎     | 96212/225000 [175:46:46<585:38:05, 16.37s/it]
 43%|████▎     | 96213/225000 [175:47:08<648:01:43, 18.11s/it]
 43%|████▎     | 96214/225000 [175:47:15<527:10:34, 14.74s/it]
 43%|████▎     | 96215/225000 [175:47:34<571:50:56, 15.99s/it]
 43%|████▎     | 96216/225000 [175:47:47<543:03:50, 15.18s/it]
 43%|████▎     | 96217/225000 [175:48:08<600:54:26, 16.80s/it]
 43%|████▎     | 96218/225000 [175:48:26<614:52:48, 17.19s/it]
 43%|████▎     | 96219/225000 [175:48:38<561:13:03, 15.69s/it]
 43%|████▎     | 96220/225000 [175:48:52<549:33:59, 15.36s/it]
 43%|████▎     | 96221/225000 [175:49:14<617:16:40, 17.26s/it]
 43%|████▎     | 96222/225000 [175:49:34<647:47:59, 18.11s/it]
 43%|████▎     | 96223/225000 [175:49:50<619:41:01, 17.32s/it]
 43%|████▎     | 96224/225000 [175:50:05<601:44:50, 16.82s/it]
 43%|████▎     | 96225/225000 [175:50:22<595:58:54, 16.66s/it]
 43%|████▎     | 96226/225000 [175:50:29<494:50:38, 13.83s/it]
 43%|████▎     | 96227/225000 [175:50:42<483:31:40, 13.52s/it]
 43%|████▎     | 96228/225000 [175:50:57<499:04:06, 13.95s/it]
 43%|████▎     | 96229/225000 [175:51:11<503:07:58, 14.07s/it]
 43%|████▎     | 96230/225000 [175:51:30<552:15:05, 15.44s/it]
 43%|████▎     | 96231/225000 [175:51:42<517:30:36, 14.47s/it]
 43%|████▎     | 96232/225000 [175:51:55<505:07:35, 14.12s/it]
 43%|████▎     | 96233/225000 [175:52:08<488:07:42, 13.65s/it]
 43%|████▎     | 96234/225000 [175:52:30<583:49:53, 16.32s/it]
 43%|████▎     | 96235/225000 [175:52:39<506:08:44, 14.15s/it]
 43%|████▎     | 96236/225000 [175:52:52<487:39:35, 13.63s/it]
 43%|████▎     | 96237/225000 [175:53:00<426:17:42, 11.92s/it]
 43%|████▎     | 96238/225000 [175:53:24<554:42:32, 15.51s/it]
 43%|████▎     | 96239/225000 [175:53:42<587:04:20, 16.41s/it]
 43%|████▎     | 96240/225000 [175:54:02<621:48:53, 17.39s/it]
 43%|████▎     | 96241/225000 [175:54:14<564:35:15, 15.79s/it]
 43%|████▎     | 96242/225000 [175:54:31<580:49:10, 16.24s/it]
 43%|████▎     | 96243/225000 [175:54:52<634:06:15, 17.73s/it]
 43%|████▎     | 96244/225000 [175:55:04<566:30:10, 15.84s/it]
 43%|████▎     | 96245/225000 [175:55:23<599:20:14, 16.76s/it]
 43%|████▎     | 96246/225000 [175:55:34<537:03:33, 15.02s/it]
 43%|████▎     | 96247/225000 [175:56:01<666:52:36, 18.65s/it]
 43%|████▎     | 96248/225000 [175:56:19<663:09:38, 18.54s/it]
 43%|████▎     | 96249/225000 [175:56:32<607:03:51, 16.97s/it]
 43%|████▎     | 96250/225000 [175:56:49<602:51:03, 16.86s/it]
                                                              
{'loss': '0.8711', 'grad_norm': '1.51', 'learning_rate': '3.266e-05', 'epoch': '0.4278', 'num_input_tokens_seen': 288716072, 'train_runtime': '6.334e+05', 'train_tokens_per_second': '455.8'}

 43%|████▎     | 96250/225000 [175:56:49<602:51:03, 16.86s/it]
 43%|████▎     | 96251/225000 [175:57:04<578:43:37, 16.18s/it]
 43%|████▎     | 96252/225000 [175:57:25<632:08:42, 17.68s/it]
 43%|████▎     | 96253/225000 [175:57:44<651:57:50, 18.23s/it]
 43%|████▎     | 96254/225000 [175:58:01<632:35:24, 17.69s/it]
 43%|████▎     | 96255/225000 [175:58:16<604:18:03, 16.90s/it]
 43%|████▎     | 96256/225000 [175:58:30<577:21:47, 16.14s/it]
 43%|████▎     | 96257/225000 [175:58:42<528:01:09, 14.76s/it]
 43%|████▎     | 96258/225000 [175:58:57<531:07:08, 14.85s/it]
 43%|████▎     | 96259/225000 [175:59:12<535:03:41, 14.96s/it]
 43%|████▎     | 96260/225000 [175:59:27<534:49:52, 14.96s/it]
 43%|████▎     | 96261/225000 [175:59:41<522:34:11, 14.61s/it]
 43%|████▎     | 96262/225000 [175:59:49<457:46:24, 12.80s/it]
 43%|████▎     | 96263/225000 [176:00:08<521:27:40, 14.58s/it]
 43%|████▎     | 96264/225000 [176:00:24<532:17:22, 14.89s/it]
 43%|████▎     | 96265/225000 [176:00:35<496:11:25, 13.88s/it]
 43%|████▎     | 96266/225000 [176:00:40<404:29:44, 11.31s/it]
 43%|████▎     | 96267/225000 [176:01:06<558:39:49, 15.62s/it]
 43%|████▎     | 96268/225000 [176:01:27<611:19:13, 17.10s/it]
 43%|████▎     | 96269/225000 [176:01:42<588:41:21, 16.46s/it]
 43%|████▎     | 96270/225000 [176:02:06<673:57:41, 18.85s/it]
 43%|████▎     | 96271/225000 [176:02:20<626:11:59, 17.51s/it]
 43%|████▎     | 96272/225000 [176:02:33<576:46:16, 16.13s/it]
 43%|████▎     | 96273/225000 [176:02:40<478:03:06, 13.37s/it]
 43%|████▎     | 96274/225000 [176:03:00<549:45:47, 15.37s/it]
 43%|████▎     | 96275/225000 [176:03:26<658:09:27, 18.41s/it]
 43%|████▎     | 96276/225000 [176:03:41<621:54:28, 17.39s/it]
 43%|████▎     | 96277/225000 [176:03:58<619:44:08, 17.33s/it]
 43%|████▎     | 96278/225000 [176:04:11<578:51:33, 16.19s/it]
 43%|████▎     | 96279/225000 [176:04:24<540:57:55, 15.13s/it]
 43%|████▎     | 96280/225000 [176:04:33<476:11:16, 13.32s/it]
 43%|████▎     | 96281/225000 [176:04:46<475:18:39, 13.29s/it]
 43%|████▎     | 96282/225000 [176:05:12<606:37:02, 16.97s/it]
 43%|████▎     | 96283/225000 [176:05:31<624:22:19, 17.46s/it]
 43%|████▎     | 96284/225000 [176:05:49<630:02:32, 17.62s/it]
 43%|████▎     | 96285/225000 [176:06:03<597:42:28, 16.72s/it]
 43%|████▎     | 96286/225000 [176:06:10<496:09:16, 13.88s/it]
 43%|████▎     | 96287/225000 [176:06:25<500:39:05, 14.00s/it]
 43%|████▎     | 96288/225000 [176:06:37<485:53:06, 13.59s/it]
 43%|████▎     | 96289/225000 [176:06:52<492:54:58, 13.79s/it]
 43%|████▎     | 96290/225000 [176:07:08<516:06:26, 14.44s/it]
 43%|████▎     | 96291/225000 [176:07:27<573:01:10, 16.03s/it]
 43%|████▎     | 96292/225000 [176:07:39<528:59:09, 14.80s/it]
 43%|████▎     | 96293/225000 [176:07:55<534:59:47, 14.96s/it]
 43%|████▎     | 96294/225000 [176:08:07<501:58:35, 14.04s/it]
 43%|████▎     | 96295/225000 [176:08:27<574:10:38, 16.06s/it]
 43%|████▎     | 96296/225000 [176:08:43<575:41:25, 16.10s/it]
 43%|████▎     | 96297/225000 [176:08:53<502:29:44, 14.06s/it]
 43%|████▎     | 96298/225000 [176:09:13<565:57:58, 15.83s/it]
 43%|████▎     | 96299/225000 [176:09:31<595:12:08, 16.65s/it]
 43%|████▎     | 96300/225000 [176:09:42<529:40:41, 14.82s/it]
                                                              
{'loss': '0.8708', 'grad_norm': '1.257', 'learning_rate': '3.264e-05', 'epoch': '0.428', 'num_input_tokens_seen': 289110232, 'train_runtime': '6.342e+05', 'train_tokens_per_second': '455.9'}

 43%|████▎     | 96300/225000 [176:09:42<529:40:41, 14.82s/it]
 43%|████▎     | 96301/225000 [176:09:55<509:55:13, 14.26s/it]
 43%|████▎     | 96302/225000 [176:10:13<554:07:44, 15.50s/it]
 43%|████▎     | 96303/225000 [176:10:30<572:26:29, 16.01s/it]
 43%|████▎     | 96304/225000 [176:10:42<523:18:41, 14.64s/it]
 43%|████▎     | 96305/225000 [176:10:57<528:26:47, 14.78s/it]
 43%|████▎     | 96306/225000 [176:11:28<698:11:52, 19.53s/it]
 43%|████▎     | 96307/225000 [176:11:37<591:00:52, 16.53s/it]
 43%|████▎     | 96308/225000 [176:11:58<638:27:36, 17.86s/it]
 43%|████▎     | 96309/225000 [176:12:04<513:35:19, 14.37s/it]
 43%|████▎     | 96310/225000 [176:12:21<537:23:00, 15.03s/it]
 43%|████▎     | 96311/225000 [176:12:38<555:31:15, 15.54s/it]
 43%|████▎     | 96312/225000 [176:12:41<427:57:45, 11.97s/it]
 43%|████▎     | 96313/225000 [176:13:04<543:51:30, 15.21s/it]
 43%|████▎     | 96314/225000 [176:13:22<578:37:23, 16.19s/it]
 43%|████▎     | 96315/225000 [176:13:38<574:25:16, 16.07s/it]
 43%|████▎     | 96316/225000 [176:13:55<583:32:11, 16.32s/it]
 43%|████▎     | 96317/225000 [176:14:13<600:03:35, 16.79s/it]
 43%|████▎     | 96318/225000 [176:14:22<517:17:43, 14.47s/it]
 43%|████▎     | 96319/225000 [176:14:36<508:18:04, 14.22s/it]
 43%|████▎     | 96320/225000 [176:14:57<587:48:05, 16.44s/it]
 43%|████▎     | 96321/225000 [176:15:13<581:59:25, 16.28s/it]
 43%|████▎     | 96322/225000 [176:15:35<639:43:01, 17.90s/it]
 43%|████▎     | 96323/225000 [176:15:47<576:22:52, 16.13s/it]
 43%|████▎     | 96324/225000 [176:15:55<484:56:36, 13.57s/it]
 43%|████▎     | 96325/225000 [176:16:14<546:08:09, 15.28s/it]
 43%|████▎     | 96326/225000 [176:16:36<622:06:50, 17.41s/it]
 43%|████▎     | 96327/225000 [176:17:01<696:22:21, 19.48s/it]
 43%|████▎     | 96328/225000 [176:17:19<689:47:43, 19.30s/it]
 43%|████▎     | 96329/225000 [176:17:31<611:48:04, 17.12s/it]
 43%|████▎     | 96330/225000 [176:17:48<606:29:50, 16.97s/it]
 43%|████▎     | 96331/225000 [176:18:00<548:37:25, 15.35s/it]
 43%|████▎     | 96332/225000 [176:18:23<631:32:19, 17.67s/it]
 43%|████▎     | 96333/225000 [176:18:43<664:57:37, 18.61s/it]
 43%|████▎     | 96334/225000 [176:18:58<616:05:15, 17.24s/it]
 43%|████▎     | 96335/225000 [176:19:17<644:26:46, 18.03s/it]
 43%|████▎     | 96336/225000 [176:19:32<602:59:34, 16.87s/it]
 43%|████▎     | 96337/225000 [176:19:40<512:19:47, 14.34s/it]
 43%|████▎     | 96338/225000 [176:19:51<474:25:21, 13.27s/it]
 43%|████▎     | 96339/225000 [176:20:10<533:38:40, 14.93s/it]
 43%|████▎     | 96340/225000 [176:20:22<508:32:52, 14.23s/it]
 43%|████▎     | 96341/225000 [176:20:48<632:02:47, 17.69s/it]
 43%|████▎     | 96342/225000 [176:20:59<564:34:44, 15.80s/it]
 43%|████▎     | 96343/225000 [176:21:15<567:42:16, 15.89s/it]
 43%|████▎     | 96344/225000 [176:21:31<566:01:27, 15.84s/it]
 43%|████▎     | 96345/225000 [176:21:52<615:59:35, 17.24s/it]
 43%|████▎     | 96346/225000 [176:22:05<573:07:19, 16.04s/it]
 43%|████▎     | 96347/225000 [176:22:17<529:21:16, 14.81s/it]
 43%|████▎     | 96348/225000 [176:22:30<509:01:46, 14.24s/it]
 43%|████▎     | 96349/225000 [176:22:38<445:52:20, 12.48s/it]
 43%|████▎     | 96350/225000 [176:22:53<469:25:11, 13.14s/it]
                                                              
{'loss': '0.8747', 'grad_norm': '1.934', 'learning_rate': '3.262e-05', 'epoch': '0.4282', 'num_input_tokens_seen': 289517480, 'train_runtime': '6.35e+05', 'train_tokens_per_second': '456'}

 43%|████▎     | 96350/225000 [176:22:53<469:25:11, 13.14s/it]
 43%|████▎     | 96351/225000 [176:23:08<494:29:44, 13.84s/it]
 43%|████▎     | 96352/225000 [176:23:24<515:58:36, 14.44s/it]
 43%|████▎     | 96353/225000 [176:23:44<578:08:58, 16.18s/it]
 43%|████▎     | 96354/225000 [176:23:57<536:43:28, 15.02s/it]
 43%|████▎     | 96355/225000 [176:24:11<524:36:54, 14.68s/it]
 43%|████▎     | 96356/225000 [176:24:30<572:26:11, 16.02s/it]
 43%|████▎     | 96357/225000 [176:24:46<570:32:08, 15.97s/it]
 43%|████▎     | 96358/225000 [176:25:07<624:51:08, 17.49s/it]
 43%|████▎     | 96359/225000 [176:25:16<539:16:51, 15.09s/it]
 43%|████▎     | 96360/225000 [176:25:29<519:18:31, 14.53s/it]
 43%|████▎     | 96361/225000 [176:25:49<578:15:39, 16.18s/it]
 43%|████▎     | 96362/225000 [176:25:59<511:59:06, 14.33s/it]
 43%|████▎     | 96363/225000 [176:26:13<500:37:55, 14.01s/it]
 43%|████▎     | 96364/225000 [176:26:30<541:13:24, 15.15s/it]
 43%|████▎     | 96365/225000 [176:26:45<536:39:20, 15.02s/it]
 43%|████▎     | 96366/225000 [176:26:55<486:15:12, 13.61s/it]
 43%|████▎     | 96367/225000 [176:27:23<639:52:06, 17.91s/it]
 43%|████▎     | 96368/225000 [176:27:41<635:37:28, 17.79s/it]
 43%|████▎     | 96369/225000 [176:28:01<663:13:33, 18.56s/it]
 43%|████▎     | 96370/225000 [176:28:19<655:09:01, 18.34s/it]
 43%|████▎     | 96371/225000 [176:28:32<600:54:31, 16.82s/it]
 43%|████▎     | 96372/225000 [176:28:46<571:49:04, 16.00s/it]
 43%|████▎     | 96373/225000 [176:29:01<553:31:29, 15.49s/it]
 43%|████▎     | 96374/225000 [176:29:22<611:04:21, 17.10s/it]
 43%|████▎     | 96375/225000 [176:29:34<563:58:26, 15.78s/it]
 43%|████▎     | 96376/225000 [176:29:42<476:30:12, 13.34s/it]
 43%|████▎     | 96377/225000 [176:30:06<586:59:19, 16.43s/it]
 43%|████▎     | 96378/225000 [176:30:25<617:11:58, 17.27s/it]
 43%|████▎     | 96379/225000 [176:30:51<716:18:52, 20.05s/it]
 43%|████▎     | 96380/225000 [176:31:01<605:40:39, 16.95s/it]
 43%|████▎     | 96381/225000 [176:31:18<601:26:57, 16.83s/it]
 43%|████▎     | 96382/225000 [176:31:34<599:49:42, 16.79s/it]
 43%|████▎     | 96383/225000 [176:31:49<582:10:01, 16.29s/it]
 43%|████▎     | 96384/225000 [176:32:05<576:51:53, 16.15s/it]
 43%|████▎     | 96385/225000 [176:32:23<598:54:20, 16.76s/it]
 43%|████▎     | 96386/225000 [176:32:34<527:12:05, 14.76s/it]
 43%|████▎     | 96387/225000 [176:32:50<547:55:03, 15.34s/it]
 43%|████▎     | 96388/225000 [176:33:14<637:09:28, 17.83s/it]
 43%|████▎     | 96389/225000 [176:33:34<661:35:14, 18.52s/it]
 43%|████▎     | 96390/225000 [176:33:45<576:31:52, 16.14s/it]
 43%|████▎     | 96391/225000 [176:33:55<518:56:50, 14.53s/it]
 43%|████▎     | 96392/225000 [176:34:17<592:37:30, 16.59s/it]
 43%|████▎     | 96393/225000 [176:34:24<488:55:09, 13.69s/it]
 43%|████▎     | 96394/225000 [176:34:45<568:20:18, 15.91s/it]
 43%|████▎     | 96395/225000 [176:34:59<552:22:18, 15.46s/it]
 43%|████▎     | 96396/225000 [176:35:19<600:24:23, 16.81s/it]
 43%|████▎     | 96397/225000 [176:35:34<581:04:28, 16.27s/it]
 43%|████▎     | 96398/225000 [176:35:45<525:52:33, 14.72s/it]
 43%|████▎     | 96399/225000 [176:36:04<566:09:59, 15.85s/it]
 43%|████▎     | 96400/225000 [176:36:18<544:05:47, 15.23s/it]
                                                              
{'loss': '0.8655', 'grad_norm': '1.389', 'learning_rate': '3.26e-05', 'epoch': '0.4284', 'num_input_tokens_seen': 289927072, 'train_runtime': '6.358e+05', 'train_tokens_per_second': '456'}

 43%|████▎     | 96400/225000 [176:36:18<544:05:47, 15.23s/it]
 43%|████▎     | 96401/225000 [176:36:37<591:26:53, 16.56s/it]
 43%|████▎     | 96402/225000 [176:36:50<551:42:03, 15.44s/it]
 43%|████▎     | 96403/225000 [176:37:07<573:04:16, 16.04s/it]
 43%|████▎     | 96404/225000 [176:37:30<647:41:35, 18.13s/it]
 43%|████▎     | 96405/225000 [176:37:44<597:55:12, 16.74s/it]
 43%|████▎     | 96406/225000 [176:38:05<643:29:17, 18.01s/it]
 43%|████▎     | 96407/225000 [176:38:24<655:07:22, 18.34s/it]
 43%|████▎     | 96408/225000 [176:38:39<614:44:08, 17.21s/it]
 43%|████▎     | 96409/225000 [176:39:00<656:55:31, 18.39s/it]
 43%|████▎     | 96410/225000 [176:39:17<648:28:59, 18.15s/it]
 43%|████▎     | 96411/225000 [176:39:27<559:54:09, 15.68s/it]
 43%|████▎     | 96412/225000 [176:39:42<552:47:36, 15.48s/it]
 43%|████▎     | 96413/225000 [176:40:03<609:54:48, 17.08s/it]
 43%|████▎     | 96414/225000 [176:40:18<590:41:23, 16.54s/it]
 43%|████▎     | 96415/225000 [176:40:25<487:03:02, 13.64s/it]
 43%|████▎     | 96416/225000 [176:40:41<506:38:17, 14.18s/it]
 43%|████▎     | 96417/225000 [176:41:05<615:36:41, 17.24s/it]
 43%|████▎     | 96418/225000 [176:41:11<495:55:04, 13.88s/it]
 43%|████▎     | 96419/225000 [176:41:31<557:56:28, 15.62s/it]
 43%|████▎     | 96420/225000 [176:41:46<557:55:58, 15.62s/it]
 43%|████▎     | 96421/225000 [176:41:57<508:34:38, 14.24s/it]
 43%|████▎     | 96422/225000 [176:42:13<520:40:52, 14.58s/it]
 43%|████▎     | 96423/225000 [176:42:33<583:12:48, 16.33s/it]
 43%|████▎     | 96424/225000 [176:42:56<650:25:03, 18.21s/it]
 43%|████▎     | 96425/225000 [176:43:08<585:56:58, 16.41s/it]
 43%|████▎     | 96426/225000 [176:43:26<598:48:19, 16.77s/it]
 43%|████▎     | 96427/225000 [176:43:43<607:25:07, 17.01s/it]
 43%|████▎     | 96428/225000 [176:43:57<575:33:02, 16.12s/it]
 43%|████▎     | 96429/225000 [176:44:17<609:55:57, 17.08s/it]
 43%|████▎     | 96430/225000 [176:44:25<518:03:17, 14.51s/it]
 43%|████▎     | 96431/225000 [176:44:47<598:20:05, 16.75s/it]
 43%|████▎     | 96432/225000 [176:45:07<632:13:41, 17.70s/it]
 43%|████▎     | 96433/225000 [176:45:23<617:01:40, 17.28s/it]
 43%|████▎     | 96434/225000 [176:45:31<513:55:10, 14.39s/it]
 43%|████▎     | 96435/225000 [176:45:45<514:29:11, 14.41s/it]
 43%|████▎     | 96436/225000 [176:45:55<460:36:33, 12.90s/it]
 43%|████▎     | 96437/225000 [176:46:12<512:23:50, 14.35s/it]
 43%|████▎     | 96438/225000 [176:46:29<531:15:01, 14.88s/it]
 43%|████▎     | 96439/225000 [176:46:43<525:27:07, 14.71s/it]
 43%|████▎     | 96440/225000 [176:46:53<471:14:22, 13.20s/it]
 43%|████▎     | 96441/225000 [176:47:15<567:13:06, 15.88s/it]
 43%|████▎     | 96442/225000 [176:47:26<515:58:26, 14.45s/it]
 43%|████▎     | 96443/225000 [176:47:42<535:14:28, 14.99s/it]
 43%|████▎     | 96444/225000 [176:47:52<481:07:11, 13.47s/it]
 43%|████▎     | 96445/225000 [176:48:13<561:43:58, 15.73s/it]
 43%|████▎     | 96446/225000 [176:48:29<561:22:50, 15.72s/it]
 43%|████▎     | 96447/225000 [176:48:35<465:10:27, 13.03s/it]
 43%|████▎     | 96448/225000 [176:48:49<471:35:46, 13.21s/it]
 43%|████▎     | 96449/225000 [176:49:03<481:56:48, 13.50s/it]
 43%|████▎     | 96450/225000 [176:49:18<497:53:07, 13.94s/it]
                                                              
{'loss': '0.8265', 'grad_norm': '1.148', 'learning_rate': '3.259e-05', 'epoch': '0.4287', 'num_input_tokens_seen': 290319184, 'train_runtime': '6.366e+05', 'train_tokens_per_second': '456.1'}

 43%|████▎     | 96450/225000 [176:49:18<497:53:07, 13.94s/it]
 43%|████▎     | 96451/225000 [176:49:35<533:26:42, 14.94s/it]
 43%|████▎     | 96452/225000 [176:49:56<595:30:36, 16.68s/it]
 43%|████▎     | 96453/225000 [176:50:11<580:07:58, 16.25s/it]
 43%|████▎     | 96454/225000 [176:50:22<518:40:45, 14.53s/it]
 43%|████▎     | 96455/225000 [176:50:37<522:18:02, 14.63s/it]
 43%|████▎     | 96456/225000 [176:50:59<601:35:03, 16.85s/it]
 43%|████▎     | 96457/225000 [176:51:06<494:26:41, 13.85s/it]
 43%|████▎     | 96458/225000 [176:51:34<648:55:51, 18.17s/it]
 43%|████▎     | 96459/225000 [176:51:51<639:15:44, 17.90s/it]
 43%|████▎     | 96460/225000 [176:52:18<737:29:41, 20.65s/it]
 43%|████▎     | 96461/225000 [176:52:35<697:26:17, 19.53s/it]
 43%|████▎     | 96462/225000 [176:52:48<630:05:47, 17.65s/it]
 43%|████▎     | 96463/225000 [176:53:01<574:48:14, 16.10s/it]
 43%|████▎     | 96464/225000 [176:53:15<556:28:19, 15.59s/it]
 43%|████▎     | 96465/225000 [176:53:26<503:40:17, 14.11s/it]
 43%|████▎     | 96466/225000 [176:53:39<496:20:35, 13.90s/it]
 43%|████▎     | 96467/225000 [176:53:59<561:14:13, 15.72s/it]
 43%|████▎     | 96468/225000 [176:54:17<584:12:55, 16.36s/it]
 43%|████▎     | 96469/225000 [176:54:35<598:58:31, 16.78s/it]
 43%|████▎     | 96470/225000 [176:54:54<621:36:01, 17.41s/it]
 43%|████▎     | 96471/225000 [176:55:07<574:30:37, 16.09s/it]
 43%|████▎     | 96472/225000 [176:55:23<576:47:28, 16.16s/it]
 43%|████▎     | 96473/225000 [176:55:40<586:00:45, 16.41s/it]
 43%|████▎     | 96474/225000 [176:55:53<550:00:10, 15.41s/it]
 43%|████▎     | 96475/225000 [176:56:07<530:16:36, 14.85s/it]
 43%|████▎     | 96476/225000 [176:56:25<563:28:07, 15.78s/it]
 43%|████▎     | 96477/225000 [176:56:46<622:45:51, 17.44s/it]
 43%|████▎     | 96478/225000 [176:57:01<598:24:45, 16.76s/it]
 43%|████▎     | 96479/225000 [176:57:20<620:27:11, 17.38s/it]
 43%|████▎     | 96480/225000 [176:57:30<535:30:40, 15.00s/it]
 43%|████▎     | 96481/225000 [176:57:48<577:43:28, 16.18s/it]
 43%|████▎     | 96482/225000 [176:58:08<618:14:46, 17.32s/it]
 43%|████▎     | 96483/225000 [176:58:29<657:53:23, 18.43s/it]
 43%|████▎     | 96484/225000 [176:58:42<597:58:07, 16.75s/it]
 43%|████▎     | 96485/225000 [176:59:02<625:52:44, 17.53s/it]
 43%|████▎     | 96486/225000 [176:59:23<662:34:50, 18.56s/it]
 43%|████▎     | 96487/225000 [176:59:40<645:39:10, 18.09s/it]
 43%|████▎     | 96488/225000 [176:59:52<583:47:38, 16.35s/it]
 43%|████▎     | 96489/225000 [177:00:11<617:54:11, 17.31s/it]
 43%|████▎     | 96490/225000 [177:00:35<683:18:21, 19.14s/it]
 43%|████▎     | 96491/225000 [177:00:53<673:42:49, 18.87s/it]
 43%|████▎     | 96492/225000 [177:01:07<618:47:16, 17.33s/it]
 43%|████▎     | 96493/225000 [177:01:33<717:41:57, 20.11s/it]
 43%|████▎     | 96494/225000 [177:01:48<655:17:29, 18.36s/it]
 43%|████▎     | 96495/225000 [177:02:01<601:23:32, 16.85s/it]
 43%|████▎     | 96496/225000 [177:02:20<625:15:34, 17.52s/it]
 43%|████▎     | 96497/225000 [177:02:34<585:48:00, 16.41s/it]
 43%|████▎     | 96498/225000 [177:02:49<572:17:27, 16.03s/it]
 43%|████▎     | 96499/225000 [177:03:04<559:30:10, 15.67s/it]
 43%|████▎     | 96500/225000 [177:03:18<546:24:09, 15.31s/it]
                                                              
{'loss': '0.8997', 'grad_norm': '1.36', 'learning_rate': '3.257e-05', 'epoch': '0.4289', 'num_input_tokens_seen': 290740944, 'train_runtime': '6.374e+05', 'train_tokens_per_second': '456.1'}

 43%|████▎     | 96500/225000 [177:03:18<546:24:09, 15.31s/it]
 43%|████▎     | 96501/225000 [177:03:32<525:40:58, 14.73s/it]
 43%|████▎     | 96502/225000 [177:03:50<564:56:50, 15.83s/it]
 43%|████▎     | 96503/225000 [177:04:15<662:35:32, 18.56s/it]
 43%|████▎     | 96504/225000 [177:04:43<764:46:08, 21.43s/it]
 43%|████▎     | 96505/225000 [177:05:00<711:17:36, 19.93s/it]
 43%|████▎     | 96506/225000 [177:05:15<666:19:43, 18.67s/it]
 43%|████▎     | 96507/225000 [177:05:30<627:22:53, 17.58s/it]
 43%|████▎     | 96508/225000 [177:05:49<635:48:51, 17.81s/it]
 43%|████▎     | 96509/225000 [177:06:02<589:29:48, 16.52s/it]
 43%|████▎     | 96510/225000 [177:06:15<549:00:21, 15.38s/it]
 43%|████▎     | 96511/225000 [177:06:35<596:27:29, 16.71s/it]
 43%|████▎     | 96512/225000 [177:06:47<547:45:56, 15.35s/it]
 43%|████▎     | 96513/225000 [177:06:56<484:49:24, 13.58s/it]
 43%|████▎     | 96514/225000 [177:07:15<543:19:19, 15.22s/it]
 43%|████▎     | 96515/225000 [177:07:46<708:08:03, 19.84s/it]
 43%|████▎     | 96516/225000 [177:08:00<645:27:41, 18.09s/it]
 43%|████▎     | 96517/225000 [177:08:12<580:38:24, 16.27s/it]
 43%|████▎     | 96518/225000 [177:08:30<601:44:25, 16.86s/it]
 43%|████▎     | 96519/225000 [177:08:41<537:52:23, 15.07s/it]
 43%|████▎     | 96520/225000 [177:08:55<528:20:44, 14.80s/it]
 43%|████▎     | 96521/225000 [177:09:04<457:22:11, 12.82s/it]
 43%|████▎     | 96522/225000 [177:09:20<500:10:18, 14.01s/it]
 43%|████▎     | 96523/225000 [177:09:35<510:37:54, 14.31s/it]
 43%|████▎     | 96524/225000 [177:09:57<584:16:35, 16.37s/it]
 43%|████▎     | 96525/225000 [177:10:17<626:47:58, 17.56s/it]
 43%|████▎     | 96526/225000 [177:10:36<642:29:47, 18.00s/it]
 43%|████▎     | 96527/225000 [177:10:57<670:31:02, 18.79s/it]
 43%|████▎     | 96528/225000 [177:11:07<584:09:16, 16.37s/it]
 43%|████▎     | 96529/225000 [177:11:27<614:55:22, 17.23s/it]
 43%|████▎     | 96530/225000 [177:11:44<612:32:38, 17.16s/it]
 43%|████▎     | 96531/225000 [177:11:52<522:40:25, 14.65s/it]
 43%|████▎     | 96532/225000 [177:12:12<574:20:38, 16.09s/it]
 43%|████▎     | 96533/225000 [177:12:25<542:31:23, 15.20s/it]
 43%|████▎     | 96534/225000 [177:12:47<616:42:13, 17.28s/it]
 43%|████▎     | 96535/225000 [177:13:06<636:50:12, 17.85s/it]
 43%|████▎     | 96536/225000 [177:13:22<615:17:39, 17.24s/it]
 43%|████▎     | 96537/225000 [177:13:40<623:33:34, 17.47s/it]
 43%|████▎     | 96538/225000 [177:13:57<615:30:24, 17.25s/it]
 43%|████▎     | 96539/225000 [177:14:09<561:30:14, 15.74s/it]
 43%|████▎     | 96540/225000 [177:14:28<594:34:16, 16.66s/it]
 43%|████▎     | 96541/225000 [177:14:50<654:05:23, 18.33s/it]
 43%|████▎     | 96542/225000 [177:15:05<617:45:10, 17.31s/it]
 43%|████▎     | 96543/225000 [177:15:29<688:10:34, 19.29s/it]
 43%|████▎     | 96544/225000 [177:15:40<601:14:18, 16.85s/it]
 43%|████▎     | 96545/225000 [177:15:57<601:31:57, 16.86s/it]
 43%|████▎     | 96546/225000 [177:16:17<638:59:03, 17.91s/it]
 43%|████▎     | 96547/225000 [177:16:39<679:30:41, 19.04s/it]
 43%|████▎     | 96548/225000 [177:16:55<647:10:38, 18.14s/it]
 43%|████▎     | 96549/225000 [177:17:15<667:33:47, 18.71s/it]
 43%|████▎     | 96550/225000 [177:17:33<664:35:53, 18.63s/it]
                                                              
{'loss': '0.8728', 'grad_norm': '1.127', 'learning_rate': '3.255e-05', 'epoch': '0.4291', 'num_input_tokens_seen': 291168376, 'train_runtime': '6.383e+05', 'train_tokens_per_second': '456.2'}

 43%|████▎     | 96550/225000 [177:17:33<664:35:53, 18.63s/it]
 43%|████▎     | 96551/225000 [177:17:54<680:54:36, 19.08s/it]
 43%|████▎     | 96552/225000 [177:18:07<624:36:56, 17.51s/it]
 43%|████▎     | 96553/225000 [177:18:16<529:17:25, 14.83s/it]
 43%|████▎     | 96554/225000 [177:18:38<606:55:57, 17.01s/it]
 43%|████▎     | 96555/225000 [177:19:05<711:14:26, 19.93s/it]
 43%|████▎     | 96556/225000 [177:19:23<694:00:25, 19.45s/it]
 43%|████▎     | 96557/225000 [177:19:41<671:12:19, 18.81s/it]
 43%|████▎     | 96558/225000 [177:19:59<666:34:42, 18.68s/it]
 43%|████▎     | 96559/225000 [177:20:09<569:54:23, 15.97s/it]
 43%|████▎     | 96560/225000 [177:20:23<548:06:16, 15.36s/it]
 43%|████▎     | 96561/225000 [177:20:43<607:35:31, 17.03s/it]
 43%|████▎     | 96562/225000 [177:21:05<660:08:54, 18.50s/it]
 43%|████▎     | 96563/225000 [177:21:20<623:37:54, 17.48s/it]
 43%|████▎     | 96564/225000 [177:21:41<657:06:37, 18.42s/it]
 43%|████▎     | 96565/225000 [177:22:06<728:12:15, 20.41s/it]
 43%|████▎     | 96566/225000 [177:22:28<738:54:27, 20.71s/it]
 43%|████▎     | 96567/225000 [177:22:43<686:24:10, 19.24s/it]
 43%|████▎     | 96568/225000 [177:23:04<700:21:37, 19.63s/it]
 43%|████▎     | 96569/225000 [177:23:23<693:28:56, 19.44s/it]
 43%|████▎     | 96570/225000 [177:23:43<704:35:51, 19.75s/it]
 43%|████▎     | 96571/225000 [177:23:53<593:14:05, 16.63s/it]
 43%|████▎     | 96572/225000 [177:24:03<522:54:01, 14.66s/it]
 43%|████▎     | 96573/225000 [177:24:12<468:46:44, 13.14s/it]
 43%|████▎     | 96574/225000 [177:24:38<598:52:01, 16.79s/it]
 43%|████▎     | 96575/225000 [177:24:59<642:19:30, 18.01s/it]
 43%|████▎     | 96576/225000 [177:25:20<674:29:20, 18.91s/it]
 43%|████▎     | 96577/225000 [177:25:29<576:09:30, 16.15s/it]
 43%|████▎     | 96578/225000 [177:25:50<623:18:31, 17.47s/it]
 43%|████▎     | 96579/225000 [177:26:03<582:01:06, 16.32s/it]
 43%|████▎     | 96580/225000 [177:26:18<560:15:06, 15.71s/it]
 43%|████▎     | 96581/225000 [177:26:35<582:02:02, 16.32s/it]
 43%|████▎     | 96582/225000 [177:26:53<599:57:25, 16.82s/it]
 43%|████▎     | 96583/225000 [177:27:06<556:54:40, 15.61s/it]
 43%|████▎     | 96584/225000 [177:27:23<568:59:58, 15.95s/it]
 43%|████▎     | 96585/225000 [177:27:42<598:46:42, 16.79s/it]
 43%|████▎     | 96586/225000 [177:28:02<640:29:36, 17.96s/it]
 43%|████▎     | 96587/225000 [177:28:18<616:08:47, 17.27s/it]
 43%|████▎     | 96588/225000 [177:28:48<755:58:26, 21.19s/it]
 43%|████▎     | 96589/225000 [177:28:57<625:19:29, 17.53s/it]
 43%|████▎     | 96590/225000 [177:29:12<592:10:55, 16.60s/it]
 43%|████▎     | 96591/225000 [177:29:32<627:02:50, 17.58s/it]
 43%|████▎     | 96592/225000 [177:29:51<641:09:29, 17.98s/it]
 43%|████▎     | 96593/225000 [177:30:04<590:43:41, 16.56s/it]
 43%|████▎     | 96594/225000 [177:30:24<625:34:27, 17.54s/it]
 43%|████▎     | 96595/225000 [177:30:37<575:11:46, 16.13s/it]
 43%|████▎     | 96596/225000 [177:31:00<654:00:35, 18.34s/it]
 43%|████▎     | 96597/225000 [177:31:17<644:36:38, 18.07s/it]
 43%|████▎     | 96598/225000 [177:31:36<648:09:08, 18.17s/it]
 43%|████▎     | 96599/225000 [177:31:53<641:15:57, 17.98s/it]
 43%|████▎     | 96600/225000 [177:32:10<622:07:49, 17.44s/it]
                                                              
{'loss': '0.8732', 'grad_norm': '1.41', 'learning_rate': '3.253e-05', 'epoch': '0.4293', 'num_input_tokens_seen': 291609624, 'train_runtime': '6.391e+05', 'train_tokens_per_second': '456.3'}

 43%|████▎     | 96600/225000 [177:32:10<622:07:49, 17.44s/it]
 43%|████▎     | 96601/225000 [177:32:23<579:58:21, 16.26s/it]
 43%|████▎     | 96602/225000 [177:32:43<617:24:08, 17.31s/it]
 43%|████▎     | 96603/225000 [177:33:01<628:46:23, 17.63s/it]
 43%|████▎     | 96604/225000 [177:33:18<623:50:52, 17.49s/it]
 43%|████▎     | 96605/225000 [177:33:31<567:06:29, 15.90s/it]
 43%|████▎     | 96606/225000 [177:33:47<576:52:27, 16.17s/it]
 43%|████▎     | 96607/225000 [177:34:04<584:13:44, 16.38s/it]
 43%|████▎     | 96608/225000 [177:34:17<547:13:37, 15.34s/it]
 43%|████▎     | 96609/225000 [177:34:30<522:58:43, 14.66s/it]
 43%|████▎     | 96610/225000 [177:34:59<669:03:02, 18.76s/it]
 43%|████▎     | 96611/225000 [177:35:17<670:42:11, 18.81s/it]
 43%|████▎     | 96612/225000 [177:35:30<600:24:56, 16.84s/it]
 43%|████▎     | 96613/225000 [177:35:50<642:01:44, 18.00s/it]
 43%|████▎     | 96614/225000 [177:36:08<632:07:14, 17.72s/it]
 43%|████▎     | 96615/225000 [177:36:25<627:32:41, 17.60s/it]
 43%|████▎     | 96616/225000 [177:36:39<594:48:30, 16.68s/it]
 43%|████▎     | 96617/225000 [177:36:51<536:40:01, 15.05s/it]
 43%|████▎     | 96618/225000 [177:37:12<600:23:02, 16.84s/it]
 43%|████▎     | 96619/225000 [177:37:36<685:14:03, 19.22s/it]
 43%|████▎     | 96620/225000 [177:38:03<762:36:34, 21.38s/it]
 43%|████▎     | 96621/225000 [177:38:15<663:40:59, 18.61s/it]
 43%|████▎     | 96622/225000 [177:38:26<578:10:12, 16.21s/it]
 43%|████▎     | 96623/225000 [177:38:45<609:50:24, 17.10s/it]
 43%|████▎     | 96624/225000 [177:39:00<592:29:38, 16.62s/it]
 43%|████▎     | 96625/225000 [177:39:13<556:05:50, 15.59s/it]
 43%|████▎     | 96626/225000 [177:39:42<690:28:40, 19.36s/it]
 43%|████▎     | 96627/225000 [177:39:50<575:14:10, 16.13s/it]
 43%|████▎     | 96628/225000 [177:39:58<485:52:43, 13.63s/it]
 43%|████▎     | 96629/225000 [177:40:17<548:24:08, 15.38s/it]
 43%|████▎     | 96630/225000 [177:40:39<611:22:09, 17.15s/it]
 43%|████▎     | 96631/225000 [177:40:54<595:37:55, 16.70s/it]
 43%|████▎     | 96632/225000 [177:41:00<480:00:17, 13.46s/it]
 43%|████▎     | 96633/225000 [177:41:17<513:45:21, 14.41s/it]
 43%|████▎     | 96634/225000 [177:41:27<466:37:32, 13.09s/it]
 43%|████▎     | 96635/225000 [177:41:37<430:39:18, 12.08s/it]
 43%|████▎     | 96636/225000 [177:41:53<477:56:13, 13.40s/it]
 43%|████▎     | 96637/225000 [177:42:08<490:37:20, 13.76s/it]
 43%|████▎     | 96638/225000 [177:42:23<502:17:01, 14.09s/it]
 43%|████▎     | 96639/225000 [177:42:45<592:59:11, 16.63s/it]
 43%|████▎     | 96640/225000 [177:43:05<627:56:25, 17.61s/it]
 43%|████▎     | 96641/225000 [177:43:15<541:31:07, 15.19s/it]
 43%|████▎     | 96642/225000 [177:43:27<511:17:03, 14.34s/it]
 43%|████▎     | 96643/225000 [177:43:52<623:54:13, 17.50s/it]
 43%|████▎     | 96644/225000 [177:44:03<560:23:22, 15.72s/it]
 43%|████▎     | 96645/225000 [177:44:23<603:18:26, 16.92s/it]
 43%|████▎     | 96646/225000 [177:44:54<752:04:53, 21.09s/it]
 43%|████▎     | 96647/225000 [177:45:10<699:57:59, 19.63s/it]
 43%|████▎     | 96648/225000 [177:45:28<681:46:58, 19.12s/it]
 43%|████▎     | 96649/225000 [177:45:47<683:27:48, 19.17s/it]
 43%|████▎     | 96650/225000 [177:46:01<624:50:51, 17.53s/it]
                                                              
{'loss': '0.8582', 'grad_norm': '1.385', 'learning_rate': '3.252e-05', 'epoch': '0.4296', 'num_input_tokens_seen': 292036728, 'train_runtime': '6.4e+05', 'train_tokens_per_second': '456.3'}

 43%|████▎     | 96650/225000 [177:46:01<624:50:51, 17.53s/it]
 43%|████▎     | 96651/225000 [177:46:23<672:00:34, 18.85s/it]
 43%|████▎     | 96652/225000 [177:46:37<616:50:57, 17.30s/it]
 43%|████▎     | 96653/225000 [177:47:00<681:44:20, 19.12s/it]
 43%|████▎     | 96654/225000 [177:47:21<697:12:03, 19.56s/it]
 43%|████▎     | 96655/225000 [177:47:34<634:35:45, 17.80s/it]
 43%|████▎     | 96656/225000 [177:47:50<607:05:00, 17.03s/it]
 43%|████▎     | 96657/225000 [177:48:12<662:38:00, 18.59s/it]
 43%|████▎     | 96658/225000 [177:48:33<687:43:49, 19.29s/it]
 43%|████▎     | 96659/225000 [177:48:48<648:25:47, 18.19s/it]
 43%|████▎     | 96660/225000 [177:49:01<588:51:23, 16.52s/it]
 43%|████▎     | 96661/225000 [177:49:24<655:39:17, 18.39s/it]
 43%|████▎     | 96662/225000 [177:49:35<582:36:14, 16.34s/it]
 43%|████▎     | 96663/225000 [177:49:59<664:25:59, 18.64s/it]
 43%|████▎     | 96664/225000 [177:50:10<577:31:07, 16.20s/it]
 43%|████▎     | 96665/225000 [177:50:25<567:52:57, 15.93s/it]
 43%|████▎     | 96666/225000 [177:50:46<617:42:51, 17.33s/it]
 43%|████▎     | 96667/225000 [177:50:58<559:40:01, 15.70s/it]
 43%|████▎     | 96668/225000 [177:51:12<549:07:34, 15.40s/it]
 43%|████▎     | 96669/225000 [177:51:39<673:00:28, 18.88s/it]
 43%|████▎     | 96670/225000 [177:51:58<670:14:30, 18.80s/it]
 43%|████▎     | 96671/225000 [177:52:21<711:56:49, 19.97s/it]
 43%|████▎     | 96672/225000 [177:52:40<703:41:14, 19.74s/it]
 43%|████▎     | 96673/225000 [177:52:55<653:27:11, 18.33s/it]
 43%|████▎     | 96674/225000 [177:53:12<638:55:41, 17.92s/it]
 43%|████▎     | 96675/225000 [177:53:36<709:22:06, 19.90s/it]
 43%|████▎     | 96676/225000 [177:53:56<703:20:53, 19.73s/it]
 43%|████▎     | 96677/225000 [177:54:14<688:08:24, 19.31s/it]
 43%|████▎     | 96678/225000 [177:54:45<816:06:16, 22.90s/it]
 43%|████▎     | 96679/225000 [177:55:01<742:47:34, 20.84s/it]
 43%|████▎     | 96680/225000 [177:55:17<682:59:43, 19.16s/it]
 43%|████▎     | 96681/225000 [177:55:33<655:08:46, 18.38s/it]
 43%|████▎     | 96682/225000 [177:55:47<609:32:41, 17.10s/it]
 43%|████▎     | 96683/225000 [177:56:06<624:58:43, 17.53s/it]
 43%|████▎     | 96684/225000 [177:56:16<544:59:13, 15.29s/it]
 43%|████▎     | 96685/225000 [177:56:35<582:10:01, 16.33s/it]
 43%|████▎     | 96686/225000 [177:56:56<641:37:04, 18.00s/it]
 43%|████▎     | 96687/225000 [177:57:16<654:46:45, 18.37s/it]
 43%|████▎     | 96688/225000 [177:57:43<752:51:33, 21.12s/it]
 43%|████▎     | 96689/225000 [177:57:52<615:26:32, 17.27s/it]
 43%|████▎     | 96690/225000 [177:58:17<703:22:56, 19.73s/it]
 43%|████▎     | 96691/225000 [177:58:38<713:32:21, 20.02s/it]
 43%|████▎     | 96692/225000 [177:58:55<679:21:50, 19.06s/it]
 43%|████▎     | 96693/225000 [177:59:14<685:27:17, 19.23s/it]
 43%|████▎     | 96694/225000 [177:59:29<644:01:48, 18.07s/it]
 43%|████▎     | 96695/225000 [177:59:52<696:26:02, 19.54s/it]
 43%|████▎     | 96696/225000 [178:00:12<698:06:07, 19.59s/it]
 43%|████▎     | 96697/225000 [178:00:30<674:07:57, 18.92s/it]
 43%|████▎     | 96698/225000 [178:00:52<712:30:24, 19.99s/it]
 43%|████▎     | 96699/225000 [178:01:07<653:47:23, 18.34s/it]
 43%|████▎     | 96700/225000 [178:01:17<570:43:02, 16.01s/it]
                                                              
{'loss': '0.8919', 'grad_norm': '1.463', 'learning_rate': '3.25e-05', 'epoch': '0.4298', 'num_input_tokens_seen': 292498208, 'train_runtime': '6.409e+05', 'train_tokens_per_second': '456.4'}

 43%|████▎     | 96700/225000 [178:01:17<570:43:02, 16.01s/it]
 43%|████▎     | 96701/225000 [178:01:33<573:13:48, 16.08s/it]
 43%|████▎     | 96702/225000 [178:01:55<634:19:49, 17.80s/it]
 43%|████▎     | 96703/225000 [178:02:10<600:21:16, 16.85s/it]
 43%|████▎     | 96704/225000 [178:02:25<578:42:41, 16.24s/it]
 43%|████▎     | 96705/225000 [178:02:46<638:40:55, 17.92s/it]
 43%|████▎     | 96706/225000 [178:03:11<708:10:42, 19.87s/it]
 43%|████▎     | 96707/225000 [178:03:20<597:21:02, 16.76s/it]
 43%|████▎     | 96708/225000 [178:03:37<601:13:02, 16.87s/it]
 43%|████▎     | 96709/225000 [178:04:03<696:09:46, 19.54s/it]
 43%|████▎     | 96710/225000 [178:04:27<741:53:20, 20.82s/it]
 43%|████▎     | 96711/225000 [178:04:34<589:35:04, 16.54s/it]
 43%|████▎     | 96712/225000 [178:04:48<562:10:22, 15.78s/it]
 43%|████▎     | 96713/225000 [178:04:58<503:17:26, 14.12s/it]
 43%|████▎     | 96714/225000 [178:05:10<478:50:29, 13.44s/it]
 43%|████▎     | 96715/225000 [178:05:26<514:19:56, 14.43s/it]
 43%|████▎     | 96716/225000 [178:05:51<619:34:35, 17.39s/it]
 43%|████▎     | 96717/225000 [178:06:02<558:10:35, 15.66s/it]
 43%|████▎     | 96718/225000 [178:06:28<660:22:46, 18.53s/it]
 43%|████▎     | 96719/225000 [178:06:46<657:18:07, 18.45s/it]
 43%|████▎     | 96720/225000 [178:07:10<716:47:31, 20.12s/it]
 43%|████▎     | 96721/225000 [178:07:28<695:47:56, 19.53s/it]
 43%|████▎     | 96722/225000 [178:07:38<593:29:24, 16.66s/it]
 43%|████▎     | 96723/225000 [178:08:09<745:38:06, 20.93s/it]
 43%|████▎     | 96724/225000 [178:08:22<660:46:03, 18.54s/it]
 43%|████▎     | 96725/225000 [178:08:31<564:34:42, 15.84s/it]
 43%|████▎     | 96726/225000 [178:08:51<603:26:26, 16.94s/it]
 43%|████▎     | 96727/225000 [178:09:09<620:22:51, 17.41s/it]
 43%|████▎     | 96728/225000 [178:09:25<602:25:07, 16.91s/it]
 43%|████▎     | 96729/225000 [178:09:46<646:07:43, 18.13s/it]
 43%|████▎     | 96730/225000 [178:10:04<641:42:06, 18.01s/it]
 43%|████▎     | 96731/225000 [178:10:14<555:46:07, 15.60s/it]
 43%|████▎     | 96732/225000 [178:10:37<637:28:04, 17.89s/it]
 43%|████▎     | 96733/225000 [178:11:00<694:53:31, 19.50s/it]
 43%|████▎     | 96734/225000 [178:11:15<645:04:08, 18.10s/it]
 43%|████▎     | 96735/225000 [178:11:36<669:23:14, 18.79s/it]
 43%|████▎     | 96736/225000 [178:11:54<668:25:54, 18.76s/it]
 43%|████▎     | 96737/225000 [178:12:13<671:38:09, 18.85s/it]
 43%|████▎     | 96738/225000 [178:12:35<704:36:00, 19.78s/it]
 43%|████▎     | 96739/225000 [178:12:50<653:36:31, 18.35s/it]
 43%|████▎     | 96740/225000 [178:13:07<632:32:40, 17.75s/it]
 43%|████▎     | 96741/225000 [178:13:29<677:35:25, 19.02s/it]
 43%|████▎     | 96742/225000 [178:13:44<642:46:53, 18.04s/it]
 43%|████▎     | 96743/225000 [178:14:12<745:24:33, 20.92s/it]
 43%|████▎     | 96744/225000 [178:14:29<704:33:50, 19.78s/it]
 43%|████▎     | 96745/225000 [178:14:42<630:52:44, 17.71s/it]
 43%|████▎     | 96746/225000 [178:15:07<706:04:05, 19.82s/it]
 43%|████▎     | 96747/225000 [178:15:29<729:33:59, 20.48s/it]
 43%|████▎     | 96748/225000 [178:15:42<649:15:10, 18.22s/it]
 43%|████▎     | 96749/225000 [178:16:06<710:23:00, 19.94s/it]
 43%|████▎     | 96750/225000 [178:16:27<721:24:52, 20.25s/it]
                                                              
{'loss': '0.8691', 'grad_norm': '1.29', 'learning_rate': '3.248e-05', 'epoch': '0.43', 'num_input_tokens_seen': 292957152, 'train_runtime': '6.418e+05', 'train_tokens_per_second': '456.5'}

 43%|████▎     | 96750/225000 [178:16:27<721:24:52, 20.25s/it]
 43%|████▎     | 96751/225000 [178:16:46<708:13:47, 19.88s/it]
 43%|████▎     | 96752/225000 [178:17:12<773:11:10, 21.70s/it]
 43%|████▎     | 96753/225000 [178:17:27<703:24:44, 19.75s/it]
 43%|████▎     | 96754/225000 [178:17:43<668:16:57, 18.76s/it]
 43%|████▎     | 96755/225000 [178:18:03<683:33:14, 19.19s/it]
 43%|████▎     | 96756/225000 [178:18:25<705:10:02, 19.80s/it]
 43%|████▎     | 96757/225000 [178:18:42<681:17:25, 19.12s/it]
 43%|████▎     | 96758/225000 [178:19:03<697:49:08, 19.59s/it]
 43%|████▎     | 96759/225000 [178:19:27<742:10:15, 20.83s/it]
 43%|████▎     | 96760/225000 [178:19:48<748:51:44, 21.02s/it]
 43%|████▎     | 96761/225000 [178:20:05<699:50:32, 19.65s/it]
 43%|████▎     | 96762/225000 [178:20:18<628:41:35, 17.65s/it]
 43%|████▎     | 96763/225000 [178:20:38<657:42:58, 18.46s/it]
 43%|████▎     | 96764/225000 [178:20:56<649:46:25, 18.24s/it]
 43%|████▎     | 96765/225000 [178:21:12<633:13:54, 17.78s/it]
 43%|████▎     | 96766/225000 [178:21:30<628:20:13, 17.64s/it]
 43%|████▎     | 96767/225000 [178:21:54<698:28:44, 19.61s/it]
 43%|████▎     | 96768/225000 [178:22:21<779:38:29, 21.89s/it]
 43%|████▎     | 96769/225000 [178:22:46<807:53:29, 22.68s/it]
 43%|████▎     | 96770/225000 [178:22:57<685:30:41, 19.25s/it]
 43%|████▎     | 96771/225000 [178:23:12<643:12:15, 18.06s/it]
 43%|████▎     | 96772/225000 [178:23:32<661:20:03, 18.57s/it]
 43%|████▎     | 96773/225000 [178:23:52<673:54:12, 18.92s/it]
 43%|████▎     | 96774/225000 [178:24:09<662:43:35, 18.61s/it]
 43%|████▎     | 96775/225000 [178:24:25<625:16:42, 17.56s/it]
 43%|████▎     | 96776/225000 [178:24:36<560:11:46, 15.73s/it]
 43%|████▎     | 96777/225000 [178:24:55<593:50:51, 16.67s/it]
 43%|████▎     | 96778/225000 [178:25:22<708:25:14, 19.89s/it]
 43%|████▎     | 96779/225000 [178:25:43<713:08:01, 20.02s/it]
 43%|████▎     | 96780/225000 [178:26:02<701:59:24, 19.71s/it]
 43%|████▎     | 96781/225000 [178:26:14<627:02:29, 17.61s/it]
 43%|████▎     | 96782/225000 [178:26:36<668:51:38, 18.78s/it]
 43%|████▎     | 96783/225000 [178:26:50<616:43:18, 17.32s/it]
 43%|████▎     | 96784/225000 [178:27:08<622:50:10, 17.49s/it]
 43%|████▎     | 96785/225000 [178:27:28<654:41:18, 18.38s/it]
 43%|████▎     | 96786/225000 [178:27:48<675:25:25, 18.96s/it]
 43%|████▎     | 96787/225000 [178:28:09<694:54:59, 19.51s/it]
 43%|████▎     | 96788/225000 [178:28:34<748:11:53, 21.01s/it]
 43%|████▎     | 96789/225000 [178:28:46<658:37:22, 18.49s/it]
 43%|████▎     | 96790/225000 [178:28:59<598:57:49, 16.82s/it]
 43%|████▎     | 96791/225000 [178:29:09<529:01:57, 14.85s/it]
 43%|████▎     | 96792/225000 [178:29:35<647:37:30, 18.18s/it]
 43%|████▎     | 96793/225000 [178:29:55<657:50:48, 18.47s/it]
 43%|████▎     | 96794/225000 [178:30:15<679:09:49, 19.07s/it]
 43%|████▎     | 96795/225000 [178:30:37<709:34:18, 19.92s/it]
 43%|████▎     | 96796/225000 [178:30:59<732:40:00, 20.57s/it]
 43%|████▎     | 96797/225000 [178:31:15<685:57:58, 19.26s/it]
 43%|████▎     | 96798/225000 [178:31:44<789:08:20, 22.16s/it]
 43%|████▎     | 96799/225000 [178:32:02<741:47:23, 20.83s/it]
 43%|████▎     | 96800/225000 [178:32:11<616:38:57, 17.32s/it]
                                                              
{'loss': '0.9045', 'grad_norm': '1.381', 'learning_rate': '3.246e-05', 'epoch': '0.4302', 'num_input_tokens_seen': 293428984, 'train_runtime': '6.427e+05', 'train_tokens_per_second': '456.5'}

 43%|████▎     | 96800/225000 [178:32:11<616:38:57, 17.32s/it]
 43%|████▎     | 96801/225000 [178:32:22<546:23:43, 15.34s/it]
 43%|████▎     | 96802/225000 [178:32:43<612:10:05, 17.19s/it]
 43%|████▎     | 96803/225000 [178:33:01<619:33:25, 17.40s/it]
 43%|████▎     | 96804/225000 [178:33:23<664:10:20, 18.65s/it]
 43%|████▎     | 96805/225000 [178:33:33<573:15:07, 16.10s/it]
 43%|████▎     | 96806/225000 [178:33:45<530:50:47, 14.91s/it]
 43%|████▎     | 96807/225000 [178:33:59<520:14:28, 14.61s/it]
 43%|████▎     | 96808/225000 [178:34:13<510:40:52, 14.34s/it]
 43%|████▎     | 96809/225000 [178:34:29<528:40:16, 14.85s/it]
 43%|████▎     | 96810/225000 [178:34:50<596:06:22, 16.74s/it]
 43%|████▎     | 96811/225000 [178:35:07<602:13:20, 16.91s/it]
 43%|████▎     | 96812/225000 [178:35:21<569:42:57, 16.00s/it]
 43%|████▎     | 96813/225000 [178:35:36<555:20:10, 15.60s/it]
 43%|████▎     | 96814/225000 [178:35:49<527:20:07, 14.81s/it]
 43%|████▎     | 96815/225000 [178:35:59<485:10:50, 13.63s/it]
 43%|████▎     | 96816/225000 [178:36:24<602:40:57, 16.93s/it]
 43%|████▎     | 96817/225000 [178:36:44<630:59:15, 17.72s/it]
 43%|████▎     | 96818/225000 [178:36:59<602:19:09, 16.92s/it]
 43%|████▎     | 96819/225000 [178:37:13<576:57:11, 16.20s/it]
 43%|████▎     | 96820/225000 [178:37:41<701:19:27, 19.70s/it]
 43%|████▎     | 96821/225000 [178:37:51<591:18:07, 16.61s/it]
 43%|████▎     | 96822/225000 [178:38:08<604:37:03, 16.98s/it]
 43%|████▎     | 96823/225000 [178:38:26<613:44:47, 17.24s/it]
 43%|████▎     | 96824/225000 [178:38:45<630:07:32, 17.70s/it]
 43%|████▎     | 96825/225000 [178:39:13<736:47:54, 20.69s/it]
 43%|████▎     | 96826/225000 [178:39:31<714:36:02, 20.07s/it]
 43%|████▎     | 96827/225000 [178:39:48<678:43:40, 19.06s/it]
 43%|████▎     | 96828/225000 [178:40:06<668:47:52, 18.78s/it]
 43%|████▎     | 96829/225000 [178:40:24<656:56:09, 18.45s/it]
 43%|████▎     | 96830/225000 [178:40:41<647:58:36, 18.20s/it]
 43%|████▎     | 96831/225000 [178:40:56<612:15:50, 17.20s/it]
 43%|████▎     | 96832/225000 [178:41:14<616:31:05, 17.32s/it]
 43%|████▎     | 96833/225000 [178:41:34<649:16:28, 18.24s/it]
 43%|████▎     | 96834/225000 [178:41:49<613:07:22, 17.22s/it]
 43%|████▎     | 96835/225000 [178:42:02<568:31:16, 15.97s/it]
 43%|████▎     | 96836/225000 [178:42:25<647:15:33, 18.18s/it]
 43%|████▎     | 96837/225000 [178:42:40<608:50:23, 17.10s/it]
 43%|████▎     | 96838/225000 [178:43:06<698:39:33, 19.62s/it]
 43%|████▎     | 96839/225000 [178:43:32<769:10:03, 21.61s/it]
 43%|████▎     | 96840/225000 [178:43:45<676:39:52, 19.01s/it]
 43%|████▎     | 96841/225000 [178:44:07<708:59:35, 19.92s/it]
 43%|████▎     | 96842/225000 [178:44:32<767:22:53, 21.56s/it]
 43%|████▎     | 96843/225000 [178:44:51<740:49:38, 20.81s/it]
 43%|████▎     | 96844/225000 [178:45:12<735:52:39, 20.67s/it]
 43%|████▎     | 96845/225000 [178:45:24<645:57:38, 18.15s/it]
 43%|████▎     | 96846/225000 [178:45:45<679:05:07, 19.08s/it]
 43%|████▎     | 96847/225000 [178:46:02<661:16:47, 18.58s/it]
 43%|████▎     | 96848/225000 [178:46:23<686:09:01, 19.28s/it]
 43%|████▎     | 96849/225000 [178:46:52<786:20:44, 22.09s/it]
 43%|████▎     | 96850/225000 [178:47:05<691:31:56, 19.43s/it]
                                                              
{'loss': '0.8926', 'grad_norm': '1.238', 'learning_rate': '3.245e-05', 'epoch': '0.4304', 'num_input_tokens_seen': 293880744, 'train_runtime': '6.436e+05', 'train_tokens_per_second': '456.6'}

 43%|████▎     | 96850/225000 [178:47:05<691:31:56, 19.43s/it]
 43%|████▎     | 96851/225000 [178:47:15<591:41:28, 16.62s/it]
 43%|████▎     | 96852/225000 [178:47:34<609:41:17, 17.13s/it]
 43%|████▎     | 96853/225000 [178:48:01<714:57:19, 20.09s/it]
 43%|████▎     | 96854/225000 [178:48:23<738:10:00, 20.74s/it]
 43%|████▎     | 96855/225000 [178:48:42<721:59:12, 20.28s/it]
 43%|████▎     | 96856/225000 [178:49:03<733:11:22, 20.60s/it]
 43%|████▎     | 96857/225000 [178:49:31<802:29:58, 22.55s/it]
 43%|████▎     | 96858/225000 [178:49:45<717:01:06, 20.14s/it]
 43%|████▎     | 96859/225000 [178:49:59<648:22:05, 18.22s/it]
 43%|████▎     | 96860/225000 [178:50:21<687:49:23, 19.32s/it]
 43%|████▎     | 96861/225000 [178:50:37<659:46:28, 18.54s/it]
 43%|████▎     | 96862/225000 [178:50:54<639:13:34, 17.96s/it]
 43%|████▎     | 96863/225000 [178:51:08<598:59:10, 16.83s/it]
 43%|████▎     | 96864/225000 [178:51:22<570:10:29, 16.02s/it]
 43%|████▎     | 96865/225000 [178:51:49<679:35:15, 19.09s/it]
 43%|████▎     | 96866/225000 [178:52:04<638:15:57, 17.93s/it]
 43%|████▎     | 96867/225000 [178:52:26<680:34:44, 19.12s/it]
 43%|████▎     | 96868/225000 [178:52:46<696:05:14, 19.56s/it]
 43%|████▎     | 96869/225000 [178:53:05<689:16:35, 19.37s/it]
 43%|████▎     | 96870/225000 [178:53:27<720:01:23, 20.23s/it]
 43%|████▎     | 96871/225000 [178:53:41<652:20:34, 18.33s/it]
 43%|████▎     | 96872/225000 [178:53:55<602:28:29, 16.93s/it]
 43%|████▎     | 96873/225000 [178:54:19<678:48:18, 19.07s/it]
 43%|████▎     | 96874/225000 [178:54:37<661:10:38, 18.58s/it]
 43%|████▎     | 96875/225000 [178:54:59<699:37:55, 19.66s/it]
 43%|████▎     | 96876/225000 [178:55:12<630:14:54, 17.71s/it]
 43%|████▎     | 96877/225000 [178:55:35<691:47:16, 19.44s/it]
 43%|████▎     | 96878/225000 [178:55:55<692:58:00, 19.47s/it]
 43%|████▎     | 96879/225000 [178:56:18<726:44:05, 20.42s/it]
 43%|████▎     | 96880/225000 [178:56:29<626:31:15, 17.60s/it]
 43%|████▎     | 96881/225000 [178:56:52<689:58:30, 19.39s/it]
 43%|████▎     | 96882/225000 [178:57:04<611:09:29, 17.17s/it]
 43%|████▎     | 96883/225000 [178:57:28<680:42:00, 19.13s/it]
 43%|████▎     | 96884/225000 [178:57:38<588:37:28, 16.54s/it]
 43%|████▎     | 96885/225000 [178:57:54<577:36:48, 16.23s/it]
 43%|████▎     | 96886/225000 [178:58:15<635:50:40, 17.87s/it]
 43%|████▎     | 96887/225000 [178:58:31<615:37:33, 17.30s/it]
 43%|████▎     | 96888/225000 [178:58:49<614:16:53, 17.26s/it]
 43%|████▎     | 96889/225000 [178:59:06<616:54:16, 17.34s/it]
 43%|████▎     | 96890/225000 [178:59:21<590:39:31, 16.60s/it]
 43%|████▎     | 96891/225000 [178:59:37<583:02:40, 16.38s/it]
 43%|████▎     | 96892/225000 [178:59:59<646:52:02, 18.18s/it]
 43%|████▎     | 96893/225000 [179:00:13<597:15:18, 16.78s/it]
 43%|████▎     | 96894/225000 [179:00:39<695:48:24, 19.55s/it]
 43%|████▎     | 96895/225000 [179:01:02<737:28:05, 20.72s/it]
 43%|████▎     | 96896/225000 [179:01:22<729:41:31, 20.51s/it]
 43%|████▎     | 96897/225000 [179:01:34<640:04:19, 17.99s/it]
 43%|████▎     | 96898/225000 [179:01:51<623:40:12, 17.53s/it]
 43%|████▎     | 96899/225000 [179:02:01<545:12:24, 15.32s/it]
 43%|████▎     | 96900/225000 [179:02:21<599:04:07, 16.84s/it]
                                                              
{'loss': '0.9075', 'grad_norm': '1.518', 'learning_rate': '3.243e-05', 'epoch': '0.4307', 'num_input_tokens_seen': 294346592, 'train_runtime': '6.445e+05', 'train_tokens_per_second': '456.7'}

 43%|████▎     | 96900/225000 [179:02:21<599:04:07, 16.84s/it]
 43%|████▎     | 96901/225000 [179:02:46<685:17:47, 19.26s/it]
 43%|████▎     | 96902/225000 [179:03:07<701:17:27, 19.71s/it]
 43%|████▎     | 96903/225000 [179:03:30<737:55:52, 20.74s/it]
 43%|████▎     | 96904/225000 [179:03:44<664:23:05, 18.67s/it]
 43%|████▎     | 96905/225000 [179:04:00<633:30:12, 17.80s/it]
 43%|████▎     | 96906/225000 [179:04:19<652:40:13, 18.34s/it]
 43%|████▎     | 96907/225000 [179:04:44<724:24:32, 20.36s/it]
 43%|████▎     | 96908/225000 [179:05:02<690:09:46, 19.40s/it]
 43%|████▎     | 96909/225000 [179:05:13<603:38:40, 16.97s/it]
 43%|████▎     | 96910/225000 [179:05:25<547:37:32, 15.39s/it]
 43%|████▎     | 96911/225000 [179:05:45<602:40:28, 16.94s/it]
 43%|████▎     | 96912/225000 [179:06:08<661:18:45, 18.59s/it]
 43%|████▎     | 96913/225000 [179:06:34<748:32:50, 21.04s/it]
 43%|████▎     | 96914/225000 [179:07:00<801:07:32, 22.52s/it]
 43%|████▎     | 96915/225000 [179:07:25<822:35:54, 23.12s/it]
 43%|████▎     | 96916/225000 [179:07:38<711:44:56, 20.00s/it]
 43%|████▎     | 96917/225000 [179:07:56<695:32:34, 19.55s/it]
 43%|████▎     | 96918/225000 [179:08:15<684:07:38, 19.23s/it]
 43%|████▎     | 96919/225000 [179:08:42<770:48:19, 21.67s/it]
 43%|████▎     | 96920/225000 [179:08:55<674:43:19, 18.96s/it]
 43%|████▎     | 96921/225000 [179:09:11<648:30:04, 18.23s/it]
 43%|████▎     | 96922/225000 [179:09:33<687:22:40, 19.32s/it]
 43%|████▎     | 96923/225000 [179:09:55<716:21:24, 20.14s/it]
 43%|████▎     | 96924/225000 [179:10:11<677:13:20, 19.04s/it]
 43%|████▎     | 96925/225000 [179:10:28<650:33:22, 18.29s/it]
 43%|████▎     | 96926/225000 [179:10:44<629:44:10, 17.70s/it]
 43%|████▎     | 96927/225000 [179:11:05<657:17:36, 18.48s/it]
 43%|████▎     | 96928/225000 [179:11:18<603:04:07, 16.95s/it]
 43%|████▎     | 96929/225000 [179:11:33<581:39:23, 16.35s/it]
 43%|████▎     | 96930/225000 [179:11:56<656:01:47, 18.44s/it]
 43%|████▎     | 96931/225000 [179:12:23<745:40:39, 20.96s/it]
 43%|████▎     | 96932/225000 [179:12:44<747:00:54, 21.00s/it]
 43%|████▎     | 96933/225000 [179:13:04<729:37:09, 20.51s/it]
 43%|████▎     | 96934/225000 [179:13:17<656:27:10, 18.45s/it]
 43%|████▎     | 96935/225000 [179:13:36<660:55:45, 18.58s/it]
 43%|████▎     | 96936/225000 [179:13:52<634:14:28, 17.83s/it]
 43%|████▎     | 96937/225000 [179:14:15<686:11:43, 19.29s/it]
 43%|████▎     | 96938/225000 [179:14:34<682:15:12, 19.18s/it]
 43%|████▎     | 96939/225000 [179:14:53<679:52:33, 19.11s/it]
 43%|████▎     | 96940/225000 [179:15:05<604:07:03, 16.98s/it]
 43%|████▎     | 96941/225000 [179:15:32<716:20:18, 20.14s/it]
 43%|████▎     | 96942/225000 [179:15:49<677:29:17, 19.05s/it]
 43%|████▎     | 96943/225000 [179:16:01<608:22:52, 17.10s/it]
 43%|████▎     | 96944/225000 [179:16:30<728:44:35, 20.49s/it]
 43%|████▎     | 96945/225000 [179:16:50<730:02:57, 20.52s/it]
 43%|████▎     | 96946/225000 [179:17:09<706:42:58, 19.87s/it]
 43%|████▎     | 96947/225000 [179:17:21<629:03:34, 17.68s/it]
 43%|████▎     | 96948/225000 [179:17:49<740:49:56, 20.83s/it]
 43%|████▎     | 96949/225000 [179:18:06<697:47:12, 19.62s/it]
 43%|████▎     | 96950/225000 [179:18:23<672:05:08, 18.90s/it]
                                                              
{'loss': '0.8747', 'grad_norm': '2.126', 'learning_rate': '3.241e-05', 'epoch': '0.4309', 'num_input_tokens_seen': 294819560, 'train_runtime': '6.455e+05', 'train_tokens_per_second': '456.7'}

 43%|████▎     | 96950/225000 [179:18:23<672:05:08, 18.90s/it]
 43%|████▎     | 96951/225000 [179:18:41<653:48:12, 18.38s/it]
 43%|████▎     | 96952/225000 [179:18:59<654:28:29, 18.40s/it]
 43%|████▎     | 96953/225000 [179:19:12<599:09:18, 16.85s/it]
 43%|████▎     | 96954/225000 [179:19:19<494:21:49, 13.90s/it]
 43%|████▎     | 96955/225000 [179:19:28<435:11:54, 12.24s/it]
 43%|████▎     | 96956/225000 [179:19:41<443:07:04, 12.46s/it]
 43%|████▎     | 96957/225000 [179:19:56<477:23:53, 13.42s/it]
 43%|████▎     | 96958/225000 [179:20:19<571:29:31, 16.07s/it]
 43%|████▎     | 96959/225000 [179:20:37<600:12:43, 16.88s/it]
 43%|████▎     | 96960/225000 [179:20:55<608:48:38, 17.12s/it]
 43%|████▎     | 96961/225000 [179:21:11<601:33:25, 16.91s/it]
 43%|████▎     | 96962/225000 [179:21:34<664:14:23, 18.68s/it]
 43%|████▎     | 96963/225000 [179:22:01<747:29:15, 21.02s/it]
 43%|████▎     | 96964/225000 [179:22:17<697:11:05, 19.60s/it]
 43%|████▎     | 96965/225000 [179:22:32<642:39:24, 18.07s/it]
 43%|████▎     | 96966/225000 [179:22:56<708:48:31, 19.93s/it]
 43%|████▎     | 96967/225000 [179:23:09<635:20:56, 17.86s/it]
 43%|████▎     | 96968/225000 [179:23:33<701:15:07, 19.72s/it]
 43%|████▎     | 96969/225000 [179:23:42<583:29:14, 16.41s/it]
 43%|████▎     | 96970/225000 [179:23:55<555:44:00, 15.63s/it]
 43%|████▎     | 96971/225000 [179:24:17<622:04:41, 17.49s/it]
 43%|████▎     | 96972/225000 [179:24:37<649:03:13, 18.25s/it]
 43%|████▎     | 96973/225000 [179:24:59<682:07:57, 19.18s/it]
 43%|████▎     | 96974/225000 [179:25:13<632:50:23, 17.80s/it]
 43%|████▎     | 96975/225000 [179:25:31<638:55:57, 17.97s/it]
 43%|████▎     | 96976/225000 [179:25:56<706:59:33, 19.88s/it]
 43%|████▎     | 96977/225000 [179:26:13<677:47:34, 19.06s/it]
 43%|████▎     | 96978/225000 [179:26:35<704:34:44, 19.81s/it]
 43%|████▎     | 96979/225000 [179:26:59<752:53:25, 21.17s/it]
 43%|████▎     | 96980/225000 [179:27:15<698:59:00, 19.66s/it]
 43%|████▎     | 96981/225000 [179:27:37<724:30:30, 20.37s/it]
 43%|████▎     | 96982/225000 [179:27:54<684:54:38, 19.26s/it]
 43%|████▎     | 96983/225000 [179:28:03<582:20:08, 16.38s/it]
 43%|████▎     | 96984/225000 [179:28:31<701:27:44, 19.73s/it]
 43%|████▎     | 96985/225000 [179:28:43<623:47:29, 17.54s/it]
 43%|████▎     | 96986/225000 [179:28:57<580:04:14, 16.31s/it]
 43%|████▎     | 96987/225000 [179:29:26<713:43:17, 20.07s/it]
 43%|████▎     | 96988/225000 [179:29:36<606:04:40, 17.04s/it]
 43%|████▎     | 96989/225000 [179:29:54<616:13:54, 17.33s/it]
 43%|████▎     | 96990/225000 [179:30:11<611:33:21, 17.20s/it]
 43%|████▎     | 96991/225000 [179:30:29<629:52:04, 17.71s/it]
 43%|████▎     | 96992/225000 [179:30:54<698:54:34, 19.66s/it]
 43%|████▎     | 96993/225000 [179:31:17<738:38:09, 20.77s/it]
 43%|████▎     | 96994/225000 [179:31:29<645:57:13, 18.17s/it]
 43%|████▎     | 96995/225000 [179:31:47<643:42:57, 18.10s/it]
 43%|████▎     | 96996/225000 [179:32:18<785:07:57, 22.08s/it]
 43%|████▎     | 96997/225000 [179:32:43<812:15:04, 22.84s/it]
 43%|████▎     | 96998/225000 [179:33:04<788:49:40, 22.19s/it]
 43%|████▎     | 96999/225000 [179:33:23<757:07:36, 21.29s/it]
 43%|████▎     | 97000/225000 [179:33:48<798:22:47, 22.45s/it]
                                                              
{'loss': '0.9009', 'grad_norm': '1.13', 'learning_rate': '3.239e-05', 'epoch': '0.4311', 'num_input_tokens_seen': 295301624, 'train_runtime': '6.464e+05', 'train_tokens_per_second': '456.8'}

 43%|████▎     | 97000/225000 [179:33:48<798:22:47, 22.45s/it]
 43%|████▎     | 97001/225000 [179:34:06<752:44:45, 21.17s/it]
 43%|████▎     | 97002/225000 [179:34:33<808:19:11, 22.73s/it]
 43%|████▎     | 97003/225000 [179:34:56<818:08:42, 23.01s/it]
 43%|████▎     | 97004/225000 [179:35:18<802:27:16, 22.57s/it]
 43%|████▎     | 97005/225000 [179:35:39<791:32:20, 22.26s/it]
 43%|████▎     | 97006/225000 [179:35:55<718:27:03, 20.21s/it]
 43%|████▎     | 97007/225000 [179:36:13<701:26:54, 19.73s/it]
 43%|████▎     | 97008/225000 [179:36:31<681:10:47, 19.16s/it]
 43%|████▎     | 97009/225000 [179:36:45<623:31:18, 17.54s/it]
 43%|████▎     | 97010/225000 [179:37:15<757:59:00, 21.32s/it]
 43%|████▎     | 97011/225000 [179:37:32<710:46:22, 19.99s/it]
 43%|████▎     | 97012/225000 [179:37:51<698:53:29, 19.66s/it]
 43%|████▎     | 97013/225000 [179:38:11<699:46:24, 19.68s/it]
 43%|████▎     | 97014/225000 [179:38:22<608:36:25, 17.12s/it]
 43%|████▎     | 97015/225000 [179:38:44<666:42:45, 18.75s/it]
 43%|████▎     | 97016/225000 [179:38:59<624:01:42, 17.55s/it]
 43%|████▎     | 97017/225000 [179:39:23<688:20:49, 19.36s/it]
 43%|████▎     | 97018/225000 [179:39:41<681:28:23, 19.17s/it]
 43%|████▎     | 97019/225000 [179:40:06<743:04:04, 20.90s/it]
 43%|████▎     | 97020/225000 [179:40:37<842:07:28, 23.69s/it]
 43%|████▎     | 97021/225000 [179:40:59<831:35:42, 23.39s/it]
 43%|████▎     | 97022/225000 [179:41:15<749:44:52, 21.09s/it]
 43%|████▎     | 97023/225000 [179:41:37<759:30:17, 21.36s/it]
 43%|████▎     | 97024/225000 [179:41:49<657:18:05, 18.49s/it]
 43%|████▎     | 97025/225000 [179:42:11<695:45:51, 19.57s/it]
 43%|████▎     | 97026/225000 [179:42:28<666:51:01, 18.76s/it]
 43%|████▎     | 97027/225000 [179:42:44<640:03:23, 18.01s/it]
 43%|████▎     | 97028/225000 [179:43:14<764:05:43, 21.49s/it]
 43%|████▎     | 97029/225000 [179:43:34<753:25:02, 21.19s/it]
 43%|████▎     | 97030/225000 [179:43:46<651:04:11, 18.32s/it]
 43%|████▎     | 97031/225000 [179:44:07<678:20:36, 19.08s/it]
 43%|████▎     | 97032/225000 [179:44:19<605:10:04, 17.02s/it]
 43%|████▎     | 97033/225000 [179:44:34<588:50:38, 16.57s/it]
 43%|████▎     | 97034/225000 [179:45:00<689:15:30, 19.39s/it]
 43%|████▎     | 97035/225000 [179:45:22<716:40:58, 20.16s/it]
 43%|████▎     | 97036/225000 [179:45:40<692:50:19, 19.49s/it]
 43%|████▎     | 97037/225000 [179:45:54<635:34:59, 17.88s/it]
 43%|████▎     | 97038/225000 [179:46:16<681:37:50, 19.18s/it]
 43%|████▎     | 97039/225000 [179:46:40<726:00:50, 20.43s/it]
 43%|████▎     | 97040/225000 [179:47:03<751:46:43, 21.15s/it]
 43%|████▎     | 97041/225000 [179:47:14<642:41:10, 18.08s/it]
 43%|████▎     | 97042/225000 [179:47:45<783:55:08, 22.05s/it]
 43%|████▎     | 97043/225000 [179:48:03<741:31:33, 20.86s/it]
 43%|████▎     | 97044/225000 [179:48:16<654:50:34, 18.42s/it]
 43%|████▎     | 97045/225000 [179:48:35<669:09:26, 18.83s/it]
 43%|████▎     | 97046/225000 [179:48:54<668:50:31, 18.82s/it]
 43%|████▎     | 97047/225000 [179:49:12<662:30:25, 18.64s/it]
 43%|████▎     | 97048/225000 [179:49:24<590:27:08, 16.61s/it]
 43%|████▎     | 97049/225000 [179:49:44<624:19:14, 17.57s/it]
 43%|████▎     | 97050/225000 [179:50:11<723:13:54, 20.35s/it]
                                                              
{'loss': '0.9163', 'grad_norm': '1.186', 'learning_rate': '3.238e-05', 'epoch': '0.4313', 'num_input_tokens_seen': 295792392, 'train_runtime': '6.474e+05', 'train_tokens_per_second': '456.9'}

 43%|████▎     | 97050/225000 [179:50:11<723:13:54, 20.35s/it]
 43%|████▎     | 97051/225000 [179:50:27<677:00:10, 19.05s/it]
 43%|████▎     | 97052/225000 [179:50:45<669:32:47, 18.84s/it]
 43%|████▎     | 97053/225000 [179:50:58<600:43:04, 16.90s/it]
 43%|████▎     | 97054/225000 [179:51:11<562:31:47, 15.83s/it]
 43%|████▎     | 97055/225000 [179:51:32<613:56:57, 17.27s/it]
 43%|████▎     | 97056/225000 [179:51:55<680:29:02, 19.15s/it]
 43%|████▎     | 97057/225000 [179:52:15<691:09:06, 19.45s/it]
 43%|████▎     | 97058/225000 [179:52:32<657:24:38, 18.50s/it]
 43%|████▎     | 97059/225000 [179:52:49<642:04:10, 18.07s/it]
 43%|████▎     | 97060/225000 [179:53:01<584:38:49, 16.45s/it]
 43%|████▎     | 97061/225000 [179:53:20<612:33:33, 17.24s/it]
 43%|████▎     | 97062/225000 [179:53:45<691:51:35, 19.47s/it]
 43%|████▎     | 97063/225000 [179:53:59<630:19:02, 17.74s/it]
 43%|████▎     | 97064/225000 [179:54:18<647:57:15, 18.23s/it]
 43%|████▎     | 97065/225000 [179:54:31<594:33:20, 16.73s/it]
 43%|████▎     | 97066/225000 [179:54:54<654:47:57, 18.43s/it]
 43%|████▎     | 97067/225000 [179:55:16<695:22:53, 19.57s/it]
 43%|████▎     | 97068/225000 [179:55:37<707:04:11, 19.90s/it]
 43%|████▎     | 97069/225000 [179:55:54<681:20:35, 19.17s/it]
 43%|████▎     | 97070/225000 [179:56:14<691:04:31, 19.45s/it]
 43%|████▎     | 97071/225000 [179:56:24<586:15:06, 16.50s/it]
 43%|████▎     | 97072/225000 [179:56:46<645:45:21, 18.17s/it]
 43%|████▎     | 97073/225000 [179:57:07<673:33:42, 18.95s/it]
 43%|████▎     | 97074/225000 [179:57:21<625:42:34, 17.61s/it]
 43%|████▎     | 97075/225000 [179:57:37<603:15:16, 16.98s/it]
 43%|████▎     | 97076/225000 [179:57:54<606:16:55, 17.06s/it]
 43%|████▎     | 97077/225000 [179:58:18<682:41:30, 19.21s/it]
 43%|████▎     | 97078/225000 [179:58:37<675:30:39, 19.01s/it]
 43%|████▎     | 97079/225000 [179:59:00<722:50:42, 20.34s/it]
 43%|████▎     | 97080/225000 [179:59:23<750:10:58, 21.11s/it]
 43%|████▎     | 97081/225000 [179:59:38<683:24:44, 19.23s/it]
 43%|████▎     | 97082/225000 [180:00:07<786:09:54, 22.13s/it]
 43%|████▎     | 97083/225000 [180:00:28<772:57:14, 21.75s/it]
 43%|████▎     | 97084/225000 [180:00:47<741:32:07, 20.87s/it]
 43%|████▎     | 97085/225000 [180:01:05<720:23:28, 20.27s/it]
 43%|████▎     | 97086/225000 [180:01:22<679:37:20, 19.13s/it]
 43%|████▎     | 97087/225000 [180:01:50<774:05:31, 21.79s/it]
 43%|████▎     | 97088/225000 [180:02:08<739:43:15, 20.82s/it]
 43%|████▎     | 97089/225000 [180:02:31<757:42:47, 21.33s/it]
 43%|████▎     | 97090/225000 [180:02:49<720:30:14, 20.28s/it]
 43%|████▎     | 97091/225000 [180:03:07<694:08:35, 19.54s/it]
 43%|████▎     | 97092/225000 [180:03:15<576:47:38, 16.23s/it]
 43%|████▎     | 97093/225000 [180:03:45<726:08:27, 20.44s/it]
 43%|████▎     | 97094/225000 [180:04:02<687:37:37, 19.35s/it]
 43%|████▎     | 97095/225000 [180:04:17<644:05:18, 18.13s/it]
 43%|████▎     | 97096/225000 [180:04:39<679:26:03, 19.12s/it]
 43%|████▎     | 97097/225000 [180:04:56<659:54:38, 18.57s/it]
 43%|████▎     | 97098/225000 [180:05:08<583:08:26, 16.41s/it]
 43%|████▎     | 97099/225000 [180:05:30<652:28:42, 18.37s/it]
 43%|████▎     | 97100/225000 [180:05:49<650:34:47, 18.31s/it]
                                                              
{'loss': '0.8816', 'grad_norm': '1.462', 'learning_rate': '3.236e-05', 'epoch': '0.4316', 'num_input_tokens_seen': 296264824, 'train_runtime': '6.483e+05', 'train_tokens_per_second': '457'}

 43%|████▎     | 97100/225000 [180:05:49<650:34:47, 18.31s/it]
 43%|████▎     | 97101/225000 [180:06:13<710:14:36, 19.99s/it]
 43%|████▎     | 97102/225000 [180:06:25<630:31:19, 17.75s/it]
 43%|████▎     | 97103/225000 [180:06:45<653:06:43, 18.38s/it]
 43%|████▎     | 97104/225000 [180:07:12<740:24:47, 20.84s/it]
 43%|████▎     | 97105/225000 [180:07:23<641:41:39, 18.06s/it]
 43%|████▎     | 97106/225000 [180:07:46<695:44:40, 19.58s/it]
 43%|████▎     | 97107/225000 [180:07:57<600:05:12, 16.89s/it]
 43%|████▎     | 97108/225000 [180:08:26<728:41:04, 20.51s/it]
 43%|████▎     | 97109/225000 [180:08:46<722:00:28, 20.32s/it]
 43%|████▎     | 97110/225000 [180:09:02<676:04:03, 19.03s/it]
 43%|████▎     | 97111/225000 [180:09:15<612:36:30, 17.24s/it]
 43%|████▎     | 97112/225000 [180:09:38<679:08:32, 19.12s/it]
 43%|████▎     | 97113/225000 [180:09:54<642:56:45, 18.10s/it]
 43%|████▎     | 97114/225000 [180:10:17<695:38:31, 19.58s/it]
 43%|████▎     | 97115/225000 [180:10:44<770:55:47, 21.70s/it]
 43%|████▎     | 97116/225000 [180:11:03<744:56:32, 20.97s/it]
 43%|████▎     | 97117/225000 [180:11:22<719:10:44, 20.25s/it]
 43%|████▎     | 97118/225000 [180:11:33<630:10:43, 17.74s/it]
 43%|████▎     | 97119/225000 [180:12:02<749:25:12, 21.10s/it]
 43%|████▎     | 97120/225000 [180:12:15<662:14:13, 18.64s/it]
 43%|████▎     | 97121/225000 [180:12:39<711:53:32, 20.04s/it]
 43%|████▎     | 97122/225000 [180:12:52<638:45:17, 17.98s/it]
 43%|████▎     | 97123/225000 [180:13:15<698:29:35, 19.66s/it]
 43%|████▎     | 97124/225000 [180:13:30<649:32:51, 18.29s/it]
 43%|████▎     | 97125/225000 [180:13:45<608:15:59, 17.12s/it]
 43%|████▎     | 97126/225000 [180:14:01<603:10:23, 16.98s/it]
 43%|████▎     | 97127/225000 [180:14:25<676:43:24, 19.05s/it]
 43%|████▎     | 97128/225000 [180:14:52<759:06:34, 21.37s/it]
 43%|████▎     | 97129/225000 [180:15:22<851:26:05, 23.97s/it]
 43%|████▎     | 97130/225000 [180:15:43<819:25:33, 23.07s/it]
 43%|████▎     | 97131/225000 [180:16:06<819:50:50, 23.08s/it]
 43%|████▎     | 97132/225000 [180:16:23<754:08:00, 21.23s/it]
 43%|████▎     | 97133/225000 [180:16:34<640:31:32, 18.03s/it]
 43%|████▎     | 97134/225000 [180:16:51<635:44:31, 17.90s/it]
 43%|████▎     | 97135/225000 [180:17:15<701:53:07, 19.76s/it]
 43%|████▎     | 97136/225000 [180:17:27<610:57:52, 17.20s/it]
 43%|████▎     | 97137/225000 [180:17:50<672:48:54, 18.94s/it]
 43%|████▎     | 97138/225000 [180:18:05<633:57:52, 17.85s/it]
 43%|████▎     | 97139/225000 [180:18:21<619:49:26, 17.45s/it]
 43%|████▎     | 97140/225000 [180:18:41<645:56:02, 18.19s/it]
 43%|████▎     | 97141/225000 [180:19:00<650:45:16, 18.32s/it]
 43%|████▎     | 97142/225000 [180:19:25<719:09:17, 20.25s/it]
 43%|████▎     | 97143/225000 [180:19:45<724:20:29, 20.39s/it]
 43%|████▎     | 97144/225000 [180:19:54<599:59:18, 16.89s/it]
 43%|████▎     | 97145/225000 [180:20:15<646:14:11, 18.20s/it]
 43%|████▎     | 97146/225000 [180:20:26<569:02:14, 16.02s/it]
 43%|████▎     | 97147/225000 [180:20:45<594:50:11, 16.75s/it]
 43%|████▎     | 97148/225000 [180:21:09<669:58:50, 18.87s/it]
 43%|████▎     | 97149/225000 [180:21:30<695:56:07, 19.60s/it]
 43%|████▎     | 97150/225000 [180:21:54<746:11:17, 21.01s/it]
                                                              
{'loss': '0.8902', 'grad_norm': '1.352', 'learning_rate': '3.234e-05', 'epoch': '0.4318', 'num_input_tokens_seen': 296749896, 'train_runtime': '6.493e+05', 'train_tokens_per_second': '457'}

 43%|████▎     | 97150/225000 [180:21:54<746:11:17, 21.01s/it]
 43%|████▎     | 97151/225000 [180:22:14<733:16:46, 20.65s/it]
 43%|████▎     | 97152/225000 [180:22:34<730:55:36, 20.58s/it]
 43%|████▎     | 97153/225000 [180:22:56<740:17:08, 20.85s/it]
 43%|████▎     | 97154/225000 [180:23:14<713:33:09, 20.09s/it]
 43%|████▎     | 97155/225000 [180:23:31<682:25:12, 19.22s/it]
 43%|████▎     | 97156/225000 [180:23:48<651:59:43, 18.36s/it]
 43%|████▎     | 97157/225000 [180:24:02<612:16:21, 17.24s/it]
 43%|████▎     | 97158/225000 [180:24:18<590:53:24, 16.64s/it]
 43%|████▎     | 97159/225000 [180:24:40<653:53:45, 18.41s/it]
 43%|████▎     | 97160/225000 [180:25:09<761:40:56, 21.45s/it]
 43%|████▎     | 97161/225000 [180:25:21<660:27:27, 18.60s/it]
 43%|████▎     | 97162/225000 [180:25:38<643:39:20, 18.13s/it]
 43%|████▎     | 97163/225000 [180:25:55<631:33:47, 17.79s/it]
 43%|████▎     | 97164/225000 [180:26:23<746:03:07, 21.01s/it]
 43%|████▎     | 97165/225000 [180:26:39<686:32:49, 19.33s/it]
 43%|████▎     | 97166/225000 [180:26:55<657:13:02, 18.51s/it]
 43%|████▎     | 97167/225000 [180:27:11<630:46:17, 17.76s/it]
 43%|████▎     | 97168/225000 [180:27:32<666:36:37, 18.77s/it]
 43%|████▎     | 97169/225000 [180:27:44<594:48:15, 16.75s/it]
 43%|████▎     | 97170/225000 [180:28:04<622:26:25, 17.53s/it]
 43%|████▎     | 97171/225000 [180:28:26<671:21:19, 18.91s/it]
 43%|████▎     | 97172/225000 [180:28:46<688:25:50, 19.39s/it]
 43%|████▎     | 97173/225000 [180:29:04<671:37:40, 18.92s/it]
 43%|████▎     | 97174/225000 [180:29:30<739:25:01, 20.82s/it]
 43%|████▎     | 97175/225000 [180:29:54<783:28:43, 22.07s/it]
 43%|████▎     | 97176/225000 [180:30:15<765:35:43, 21.56s/it]
 43%|████▎     | 97177/225000 [180:30:38<785:34:11, 22.12s/it]
 43%|████▎     | 97178/225000 [180:30:57<750:44:56, 21.14s/it]
 43%|████▎     | 97179/225000 [180:31:14<707:53:48, 19.94s/it]
 43%|████▎     | 97180/225000 [180:31:30<660:15:25, 18.60s/it]
 43%|████▎     | 97181/225000 [180:31:44<610:29:18, 17.19s/it]
 43%|████▎     | 97182/225000 [180:31:59<590:28:04, 16.63s/it]
 43%|████▎     | 97183/225000 [180:32:27<712:13:41, 20.06s/it]
 43%|████▎     | 97184/225000 [180:32:55<799:56:15, 22.53s/it]
 43%|████▎     | 97185/225000 [180:33:15<764:35:44, 21.54s/it]
 43%|████▎     | 97186/225000 [180:33:42<823:06:22, 23.18s/it]
 43%|████▎     | 97187/225000 [180:33:59<756:43:17, 21.31s/it]
 43%|████▎     | 97188/225000 [180:34:22<776:50:24, 21.88s/it]
 43%|████▎     | 97189/225000 [180:34:34<677:30:37, 19.08s/it]
 43%|████▎     | 97190/225000 [180:34:48<616:54:02, 17.38s/it]
 43%|████▎     | 97191/225000 [180:35:07<638:13:35, 17.98s/it]
 43%|████▎     | 97192/225000 [180:35:27<658:08:54, 18.54s/it]
 43%|████▎     | 97193/225000 [180:35:43<632:41:05, 17.82s/it]
 43%|████▎     | 97194/225000 [180:36:03<652:27:51, 18.38s/it]
 43%|████▎     | 97195/225000 [180:36:28<729:15:33, 20.54s/it]
 43%|████▎     | 97196/225000 [180:36:41<648:28:46, 18.27s/it]
 43%|████▎     | 97197/225000 [180:37:03<685:01:09, 19.30s/it]
 43%|████▎     | 97198/225000 [180:37:32<789:24:33, 22.24s/it]
 43%|████▎     | 97199/225000 [180:37:50<745:08:50, 20.99s/it]
 43%|████▎     | 97200/225000 [180:38:14<779:50:52, 21.97s/it]
                                                              
{'loss': '0.8932', 'grad_norm': '1.473', 'learning_rate': '3.232e-05', 'epoch': '0.432', 'num_input_tokens_seen': 297238528, 'train_runtime': '6.503e+05', 'train_tokens_per_second': '457.1'}

 43%|████▎     | 97200/225000 [180:38:14<779:50:52, 21.97s/it]
 43%|████▎     | 97201/225000 [180:38:31<720:23:25, 20.29s/it]
 43%|████▎     | 97202/225000 [180:39:00<817:17:13, 23.02s/it]
 43%|████▎     | 97203/225000 [180:39:21<791:34:48, 22.30s/it]
 43%|████▎     | 97204/225000 [180:39:41<764:31:45, 21.54s/it]
 43%|████▎     | 97205/225000 [180:39:55<690:39:44, 19.46s/it]
 43%|████▎     | 97206/225000 [180:40:14<678:42:21, 19.12s/it]
 43%|████▎     | 97207/225000 [180:40:34<697:51:33, 19.66s/it]
 43%|████▎     | 97208/225000 [180:40:53<688:50:54, 19.41s/it]
 43%|████▎     | 97209/225000 [180:41:14<708:32:00, 19.96s/it]
 43%|████▎     | 97210/225000 [180:41:37<737:37:46, 20.78s/it]
 43%|████▎     | 97211/225000 [180:41:57<730:43:23, 20.59s/it]
 43%|████▎     | 97212/225000 [180:42:14<685:11:12, 19.30s/it]
 43%|████▎     | 97213/225000 [180:42:40<764:56:59, 21.55s/it]
 43%|████▎     | 97214/225000 [180:42:58<726:26:11, 20.47s/it]
 43%|████▎     | 97215/225000 [180:43:12<651:02:18, 18.34s/it]
 43%|████▎     | 97216/225000 [180:43:38<734:31:38, 20.69s/it]
 43%|████▎     | 97217/225000 [180:43:50<642:03:50, 18.09s/it]
 43%|████▎     | 97218/225000 [180:44:06<616:27:33, 17.37s/it]
 43%|████▎     | 97219/225000 [180:44:24<627:26:59, 17.68s/it]
 43%|████▎     | 97220/225000 [180:44:41<624:14:48, 17.59s/it]
 43%|████▎     | 97221/225000 [180:44:57<608:18:09, 17.14s/it]
 43%|████▎     | 97222/225000 [180:45:15<614:58:00, 17.33s/it]
 43%|████▎     | 97223/225000 [180:45:38<668:54:37, 18.85s/it]
 43%|████▎     | 97224/225000 [180:45:52<621:25:03, 17.51s/it]
 43%|████▎     | 97225/225000 [180:46:16<691:59:44, 19.50s/it]
 43%|████▎     | 97226/225000 [180:46:35<690:08:57, 19.44s/it]
 43%|████▎     | 97227/225000 [180:46:52<662:44:54, 18.67s/it]
 43%|████▎     | 97228/225000 [180:47:17<721:26:39, 20.33s/it]
 43%|████▎     | 97229/225000 [180:47:31<659:52:35, 18.59s/it]
 43%|████▎     | 97230/225000 [180:47:55<719:19:03, 20.27s/it]
 43%|████▎     | 97231/225000 [180:48:24<811:51:35, 22.87s/it]
 43%|████▎     | 97232/225000 [180:48:46<804:27:16, 22.67s/it]
 43%|████▎     | 97233/225000 [180:49:01<720:11:54, 20.29s/it]
 43%|████▎     | 97234/225000 [180:49:19<693:28:43, 19.54s/it]
 43%|████▎     | 97235/225000 [180:49:38<686:59:24, 19.36s/it]
 43%|████▎     | 97236/225000 [180:49:57<684:03:44, 19.27s/it]
 43%|████▎     | 97237/225000 [180:50:13<652:00:23, 18.37s/it]
 43%|████▎     | 97238/225000 [180:50:22<550:38:58, 15.52s/it]
 43%|████▎     | 97239/225000 [180:50:40<577:22:37, 16.27s/it]
 43%|████▎     | 97240/225000 [180:51:07<691:30:36, 19.49s/it]
 43%|████▎     | 97241/225000 [180:51:34<767:34:58, 21.63s/it]
 43%|████▎     | 97242/225000 [180:51:52<729:14:25, 20.55s/it]
 43%|████▎     | 97243/225000 [180:52:18<785:49:15, 22.14s/it]
 43%|████▎     | 97244/225000 [180:52:38<766:26:21, 21.60s/it]
 43%|████▎     | 97245/225000 [180:52:55<719:02:40, 20.26s/it]
 43%|████▎     | 97246/225000 [180:53:12<683:44:43, 19.27s/it]
 43%|████▎     | 97247/225000 [180:53:35<722:15:26, 20.35s/it]
 43%|████▎     | 97248/225000 [180:53:53<700:20:00, 19.74s/it]
 43%|████▎     | 97249/225000 [180:54:14<706:57:51, 19.92s/it]
 43%|████▎     | 97250/225000 [180:54:29<660:20:07, 18.61s/it]
                                                              
{'loss': '0.8858', 'grad_norm': '1.619', 'learning_rate': '3.231e-05', 'epoch': '0.4322', 'num_input_tokens_seen': 297735744, 'train_runtime': '6.513e+05', 'train_tokens_per_second': '457.2'}

 43%|████▎     | 97250/225000 [180:54:29<660:20:07, 18.61s/it]
 43%|████▎     | 97251/225000 [180:54:53<721:54:24, 20.34s/it]
 43%|████▎     | 97252/225000 [180:55:21<803:23:45, 22.64s/it]
 43%|████▎     | 97253/225000 [180:55:40<763:51:39, 21.53s/it]
 43%|████▎     | 97254/225000 [180:56:00<741:15:06, 20.89s/it]
 43%|████▎     | 97255/225000 [180:56:12<643:58:10, 18.15s/it]
 43%|████▎     | 97256/225000 [180:56:26<602:41:48, 16.98s/it]
 43%|████▎     | 97257/225000 [180:56:47<643:34:01, 18.14s/it]
 43%|████▎     | 97258/225000 [180:57:01<597:55:40, 16.85s/it]
 43%|████▎     | 97259/225000 [180:57:21<637:31:47, 17.97s/it]
 43%|████▎     | 97260/225000 [180:57:39<636:25:14, 17.94s/it]
 43%|████▎     | 97261/225000 [180:58:05<717:55:34, 20.23s/it]
 43%|████▎     | 97262/225000 [180:58:18<649:15:33, 18.30s/it]
 43%|████▎     | 97263/225000 [180:58:34<623:00:04, 17.56s/it]
 43%|████▎     | 97264/225000 [180:58:53<634:17:34, 17.88s/it]
 43%|████▎     | 97265/225000 [180:59:09<615:15:55, 17.34s/it]
 43%|████▎     | 97266/225000 [180:59:23<581:00:14, 16.37s/it]
 43%|████▎     | 97267/225000 [180:59:45<636:30:02, 17.94s/it]
 43%|████▎     | 97268/225000 [181:00:05<658:13:29, 18.55s/it]
 43%|████▎     | 97269/225000 [181:00:22<642:00:21, 18.09s/it]
 43%|████▎     | 97270/225000 [181:00:49<742:34:55, 20.93s/it]
 43%|████▎     | 97271/225000 [181:01:11<747:26:24, 21.07s/it]
 43%|████▎     | 97272/225000 [181:01:29<722:37:54, 20.37s/it]
 43%|████▎     | 97273/225000 [181:01:54<773:30:06, 21.80s/it]
 43%|████▎     | 97274/225000 [181:02:10<705:47:50, 19.89s/it]
 43%|████▎     | 97275/225000 [181:02:22<620:50:30, 17.50s/it]
 43%|████▎     | 97276/225000 [181:02:35<570:56:45, 16.09s/it]
 43%|████▎     | 97277/225000 [181:02:55<618:10:23, 17.42s/it]
 43%|████▎     | 97278/225000 [181:03:19<681:52:26, 19.22s/it]
 43%|████▎     | 97279/225000 [181:03:38<686:22:31, 19.35s/it]
 43%|████▎     | 97280/225000 [181:03:59<697:05:18, 19.65s/it]
 43%|████▎     | 97281/225000 [181:04:12<629:31:22, 17.74s/it]
 43%|████▎     | 97282/225000 [181:04:37<712:06:02, 20.07s/it]
 43%|████▎     | 97283/225000 [181:04:55<682:20:43, 19.23s/it]
 43%|████▎     | 97284/225000 [181:05:19<735:45:00, 20.74s/it]
 43%|████▎     | 97285/225000 [181:05:34<680:25:42, 19.18s/it]
 43%|████▎     | 97286/225000 [181:06:00<746:33:31, 21.04s/it]
 43%|████▎     | 97287/225000 [181:06:23<770:58:15, 21.73s/it]
 43%|████▎     | 97288/225000 [181:06:36<673:27:09, 18.98s/it]
 43%|████▎     | 97289/225000 [181:06:56<684:27:50, 19.29s/it]
 43%|████▎     | 97290/225000 [181:07:15<681:59:04, 19.22s/it]
 43%|████▎     | 97291/225000 [181:07:29<623:41:47, 17.58s/it]
 43%|████▎     | 97292/225000 [181:07:52<690:26:03, 19.46s/it]
 43%|████▎     | 97293/225000 [181:08:06<624:44:45, 17.61s/it]
 43%|████▎     | 97294/225000 [181:08:27<665:12:12, 18.75s/it]
 43%|████▎     | 97295/225000 [181:08:46<670:54:05, 18.91s/it]
 43%|████▎     | 97296/225000 [181:09:01<622:17:28, 17.54s/it]
 43%|████▎     | 97297/225000 [181:09:28<728:53:00, 20.55s/it]
 43%|████▎     | 97298/225000 [181:09:54<783:15:43, 22.08s/it]
 43%|████▎     | 97299/225000 [181:10:15<775:44:29, 21.87s/it]
 43%|████▎     | 97300/225000 [181:10:31<714:51:53, 20.15s/it]
                                                              
{'loss': '0.9003', 'grad_norm': '1.241', 'learning_rate': '3.229e-05', 'epoch': '0.4324', 'num_input_tokens_seen': 298216672, 'train_runtime': '6.522e+05', 'train_tokens_per_second': '457.2'}

 43%|████▎     | 97300/225000 [181:10:31<714:51:53, 20.15s/it]
 43%|████▎     | 97301/225000 [181:10:50<699:32:30, 19.72s/it]
 43%|████▎     | 97302/225000 [181:11:14<742:03:41, 20.92s/it]
 43%|████▎     | 97303/225000 [181:11:25<642:07:19, 18.10s/it]
 43%|████▎     | 97304/225000 [181:11:43<636:47:40, 17.95s/it]
 43%|████▎     | 97305/225000 [181:12:01<637:09:55, 17.96s/it]
 43%|████▎     | 97306/225000 [181:12:21<657:23:34, 18.53s/it]
 43%|████▎     | 97307/225000 [181:12:42<685:50:51, 19.34s/it]
 43%|████▎     | 97308/225000 [181:12:52<590:01:44, 16.63s/it]
 43%|████▎     | 97309/225000 [181:13:08<580:09:56, 16.36s/it]
 43%|████▎     | 97310/225000 [181:13:28<614:37:05, 17.33s/it]
 43%|████▎     | 97311/225000 [181:13:57<737:44:29, 20.80s/it]
 43%|████▎     | 97312/225000 [181:14:16<726:45:01, 20.49s/it]
 43%|████▎     | 97313/225000 [181:14:39<746:44:07, 21.05s/it]
 43%|████▎     | 97314/225000 [181:15:07<824:00:27, 23.23s/it]
 43%|████▎     | 97315/225000 [181:15:19<703:22:11, 19.83s/it]
 43%|████▎     | 97316/225000 [181:15:36<677:05:52, 19.09s/it]
 43%|████▎     | 97317/225000 [181:15:56<679:48:01, 19.17s/it]
 43%|████▎     | 97318/225000 [181:16:21<746:59:22, 21.06s/it]
 43%|████▎     | 97319/225000 [181:16:45<781:33:47, 22.04s/it]
 43%|████▎     | 97320/225000 [181:17:08<788:18:03, 22.23s/it]
 43%|████▎     | 97321/225000 [181:17:36<844:33:07, 23.81s/it]
 43%|████▎     | 97322/225000 [181:17:53<776:24:37, 21.89s/it]
 43%|████▎     | 97323/225000 [181:18:04<663:32:24, 18.71s/it]
 43%|████▎     | 97324/225000 [181:18:28<712:25:34, 20.09s/it]
 43%|████▎     | 97325/225000 [181:18:36<592:42:44, 16.71s/it]
 43%|████▎     | 97326/225000 [181:19:03<702:12:34, 19.80s/it]
 43%|████▎     | 97327/225000 [181:19:23<696:54:39, 19.65s/it]
 43%|████▎     | 97328/225000 [181:19:35<616:49:31, 17.39s/it]
 43%|████▎     | 97329/225000 [181:19:58<675:48:09, 19.06s/it]
 43%|████▎     | 97330/225000 [181:20:27<779:01:54, 21.97s/it]
 43%|████▎     | 97331/225000 [181:20:35<639:04:31, 18.02s/it]
 43%|████▎     | 97332/225000 [181:20:58<684:43:35, 19.31s/it]
 43%|████▎     | 97333/225000 [181:21:19<704:40:52, 19.87s/it]
 43%|████▎     | 97334/225000 [181:21:31<621:43:12, 17.53s/it]
 43%|████▎     | 97335/225000 [181:21:56<698:04:49, 19.69s/it]
 43%|████▎     | 97336/225000 [181:22:11<650:53:14, 18.35s/it]
 43%|████▎     | 97337/225000 [181:22:33<693:40:33, 19.56s/it]
 43%|████▎     | 97338/225000 [181:22:55<720:28:59, 20.32s/it]
 43%|████▎     | 97339/225000 [181:23:19<760:05:39, 21.43s/it]
 43%|████▎     | 97340/225000 [181:23:44<788:26:48, 22.23s/it]
 43%|████▎     | 97341/225000 [181:24:08<810:08:25, 22.85s/it]
 43%|████▎     | 97342/225000 [181:24:23<730:02:30, 20.59s/it]
 43%|████▎     | 97343/225000 [181:24:47<765:09:44, 21.58s/it]
 43%|████▎     | 97344/225000 [181:25:05<730:25:08, 20.60s/it]
 43%|████▎     | 97345/225000 [181:25:26<727:13:22, 20.51s/it]
 43%|████▎     | 97346/225000 [181:25:42<688:23:42, 19.41s/it]
 43%|████▎     | 97347/225000 [181:26:01<679:42:17, 19.17s/it]
 43%|████▎     | 97348/225000 [181:26:16<631:10:44, 17.80s/it]
 43%|████▎     | 97349/225000 [181:26:37<666:17:23, 18.79s/it]
 43%|████▎     | 97350/225000 [181:27:02<731:51:23, 20.64s/it]
                                                              
{'loss': '0.8721', 'grad_norm': '1.012', 'learning_rate': '3.227e-05', 'epoch': '0.4327', 'num_input_tokens_seen': 298726392, 'train_runtime': '6.532e+05', 'train_tokens_per_second': '457.3'}

 43%|████▎     | 97350/225000 [181:27:02<731:51:23, 20.64s/it]
 43%|████▎     | 97351/225000 [181:27:25<762:01:09, 21.49s/it]
 43%|████▎     | 97352/225000 [181:27:46<749:14:18, 21.13s/it]
 43%|████▎     | 97353/225000 [181:28:09<769:29:23, 21.70s/it]
 43%|████▎     | 97354/225000 [181:28:23<691:26:41, 19.50s/it]
 43%|████▎     | 97355/225000 [181:28:42<691:22:14, 19.50s/it]
 43%|████▎     | 97356/225000 [181:29:12<793:31:31, 22.38s/it]
 43%|████▎     | 97357/225000 [181:29:32<769:42:20, 21.71s/it]
 43%|████▎     | 97358/225000 [181:29:46<696:28:24, 19.64s/it]
 43%|████▎     | 97359/225000 [181:30:06<697:34:08, 19.67s/it]
 43%|████▎     | 97360/225000 [181:30:25<686:40:36, 19.37s/it]
 43%|████▎     | 97361/225000 [181:30:43<668:16:42, 18.85s/it]
 43%|████▎     | 97362/225000 [181:30:55<601:25:46, 16.96s/it]
 43%|████▎     | 97363/225000 [181:31:12<606:03:25, 17.09s/it]
 43%|████▎     | 97364/225000 [181:31:43<754:16:23, 21.27s/it]
 43%|████▎     | 97365/225000 [181:32:04<746:32:53, 21.06s/it]
 43%|████▎     | 97366/225000 [181:32:24<734:51:32, 20.73s/it]
 43%|████▎     | 97367/225000 [181:32:49<775:58:48, 21.89s/it]
 43%|████▎     | 97368/225000 [181:33:08<748:56:22, 21.12s/it]
 43%|████▎     | 97369/225000 [181:33:32<784:04:31, 22.12s/it]
 43%|████▎     | 97370/225000 [181:33:49<724:25:44, 20.43s/it]
 43%|████▎     | 97371/225000 [181:34:09<717:15:33, 20.23s/it]
 43%|████▎     | 97372/225000 [181:34:23<659:02:24, 18.59s/it]
 43%|████▎     | 97373/225000 [181:34:36<596:54:57, 16.84s/it]
 43%|████▎     | 97374/225000 [181:35:02<689:23:54, 19.45s/it]
 43%|████▎     | 97375/225000 [181:35:23<707:25:01, 19.95s/it]
 43%|████▎     | 97376/225000 [181:35:42<696:40:45, 19.65s/it]
 43%|████▎     | 97377/225000 [181:36:06<747:56:54, 21.10s/it]
 43%|████▎     | 97378/225000 [181:36:18<650:26:19, 18.35s/it]
 43%|████▎     | 97379/225000 [181:36:37<657:02:35, 18.53s/it]
 43%|████▎     | 97380/225000 [181:36:50<601:30:50, 16.97s/it]
 43%|████▎     | 97381/225000 [181:37:17<704:37:28, 19.88s/it]
 43%|████▎     | 97382/225000 [181:37:43<763:45:22, 21.54s/it]
 43%|████▎     | 97383/225000 [181:37:58<700:42:10, 19.77s/it]
 43%|████▎     | 97384/225000 [181:38:25<774:35:58, 21.85s/it]
 43%|████▎     | 97385/225000 [181:38:45<751:04:17, 21.19s/it]
 43%|████▎     | 97386/225000 [181:39:02<713:26:12, 20.13s/it]
 43%|████▎     | 97387/225000 [181:39:14<625:19:24, 17.64s/it]
 43%|████▎     | 97388/225000 [181:39:36<667:56:15, 18.84s/it]
 43%|████▎     | 97389/225000 [181:39:56<681:38:44, 19.23s/it]
 43%|████▎     | 97390/225000 [181:40:14<671:01:34, 18.93s/it]
 43%|████▎     | 97391/225000 [181:40:39<736:03:42, 20.77s/it]
 43%|████▎     | 97392/225000 [181:41:02<753:58:06, 21.27s/it]
 43%|████▎     | 97393/225000 [181:41:14<657:04:39, 18.54s/it]
 43%|████▎     | 97394/225000 [181:41:30<637:20:45, 17.98s/it]
 43%|████▎     | 97395/225000 [181:41:57<725:41:10, 20.47s/it]
 43%|████▎     | 97396/225000 [181:42:11<655:26:09, 18.49s/it]
 43%|████▎     | 97397/225000 [181:42:36<727:22:27, 20.52s/it]
 43%|████▎     | 97398/225000 [181:42:51<669:18:08, 18.88s/it]
 43%|████▎     | 97399/225000 [181:43:07<636:02:08, 17.94s/it]
 43%|████▎     | 97400/225000 [181:43:31<699:47:10, 19.74s/it]
                                                              
{'loss': '0.891', 'grad_norm': '1.091', 'learning_rate': '3.225e-05', 'epoch': '0.4329', 'num_input_tokens_seen': 299223128, 'train_runtime': '6.542e+05', 'train_tokens_per_second': '457.4'}

 43%|████▎     | 97400/225000 [181:43:31<699:47:10, 19.74s/it]
 43%|████▎     | 97401/225000 [181:43:53<723:44:17, 20.42s/it]
 43%|████▎     | 97402/225000 [181:44:20<800:40:38, 22.59s/it]
 43%|████▎     | 97403/225000 [181:44:48<856:02:17, 24.15s/it]
 43%|████▎     | 97404/225000 [181:45:02<747:53:21, 21.10s/it]
 43%|████▎     | 97405/225000 [181:45:18<691:42:16, 19.52s/it]
 43%|████▎     | 97406/225000 [181:45:40<715:48:10, 20.20s/it]
 43%|████▎     | 97407/225000 [181:45:59<706:51:12, 19.94s/it]
 43%|████▎     | 97408/225000 [181:46:11<626:01:39, 17.66s/it]
 43%|████▎     | 97409/225000 [181:46:34<677:19:43, 19.11s/it]
 43%|████▎     | 97410/225000 [181:47:02<770:02:31, 21.73s/it]
 43%|████▎     | 97411/225000 [181:47:16<691:48:10, 19.52s/it]
 43%|████▎     | 97412/225000 [181:47:35<690:36:50, 19.49s/it]
 43%|████▎     | 97413/225000 [181:47:54<686:44:22, 19.38s/it]
 43%|████▎     | 97414/225000 [181:48:11<659:39:03, 18.61s/it]
 43%|████▎     | 97415/225000 [181:48:35<713:32:33, 20.13s/it]
 43%|████▎     | 97416/225000 [181:49:04<808:18:01, 22.81s/it]
 43%|████▎     | 97417/225000 [181:49:25<785:23:21, 22.16s/it]
 43%|████▎     | 97418/225000 [181:49:51<830:02:57, 23.42s/it]
 43%|████▎     | 97419/225000 [181:50:16<846:55:28, 23.90s/it]
 43%|████▎     | 97420/225000 [181:50:35<794:02:22, 22.41s/it]
 43%|████▎     | 97421/225000 [181:50:53<750:00:51, 21.16s/it]
 43%|████▎     | 97422/225000 [181:51:11<717:32:02, 20.25s/it]
 43%|████▎     | 97423/225000 [181:51:33<729:59:56, 20.60s/it]
 43%|████▎     | 97424/225000 [181:51:51<703:25:59, 19.85s/it]
 43%|████▎     | 97425/225000 [181:52:05<637:58:38, 18.00s/it]
 43%|████▎     | 97426/225000 [181:52:30<715:56:51, 20.20s/it]
 43%|████▎     | 97427/225000 [181:52:47<687:36:13, 19.40s/it]
 43%|████▎     | 97428/225000 [181:53:10<723:23:48, 20.41s/it]
 43%|████▎     | 97429/225000 [181:53:26<678:23:03, 19.14s/it]
 43%|████▎     | 97430/225000 [181:53:44<662:26:49, 18.69s/it]
 43%|████▎     | 97431/225000 [181:54:07<711:09:07, 20.07s/it]
 43%|████▎     | 97432/225000 [181:54:29<725:16:04, 20.47s/it]
 43%|████▎     | 97433/225000 [181:54:50<734:44:30, 20.73s/it]
 43%|████▎     | 97434/225000 [181:55:04<660:13:24, 18.63s/it]
 43%|████▎     | 97435/225000 [181:55:27<706:09:19, 19.93s/it]
 43%|████▎     | 97436/225000 [181:55:48<718:52:45, 20.29s/it]
 43%|████▎     | 97437/225000 [181:56:12<764:47:57, 21.58s/it]
 43%|████▎     | 97438/225000 [181:56:27<694:28:21, 19.60s/it]
 43%|████▎     | 97439/225000 [181:56:49<716:42:02, 20.23s/it]
 43%|████▎     | 97440/225000 [181:57:18<804:43:17, 22.71s/it]
 43%|████▎     | 97441/225000 [181:57:37<765:02:05, 21.59s/it]
 43%|████▎     | 97442/225000 [181:57:59<772:07:06, 21.79s/it]
 43%|████▎     | 97443/225000 [181:58:21<776:41:55, 21.92s/it]
 43%|████▎     | 97444/225000 [181:58:45<793:48:52, 22.40s/it]
 43%|████▎     | 97445/225000 [181:59:10<828:27:32, 23.38s/it]
 43%|████▎     | 97446/225000 [181:59:39<886:50:23, 25.03s/it]
 43%|████▎     | 97447/225000 [182:00:00<846:57:19, 23.90s/it]
 43%|████▎     | 97448/225000 [182:00:22<825:08:52, 23.29s/it]
 43%|████▎     | 97449/225000 [182:00:38<747:48:32, 21.11s/it]
 43%|████▎     | 97450/225000 [182:00:55<696:55:37, 19.67s/it]
                                                              
{'loss': '0.9008', 'grad_norm': '1.264', 'learning_rate': '3.224e-05', 'epoch': '0.4331', 'num_input_tokens_seen': 299751040, 'train_runtime': '6.553e+05', 'train_tokens_per_second': '457.5'}

 43%|████▎     | 97450/225000 [182:00:55<696:55:37, 19.67s/it]
 43%|████▎     | 97451/225000 [182:01:21<763:25:57, 21.55s/it]
 43%|████▎     | 97452/225000 [182:01:35<692:18:57, 19.54s/it]
 43%|████▎     | 97453/225000 [182:01:54<682:18:15, 19.26s/it]
 43%|████▎     | 97454/225000 [182:02:13<675:19:10, 19.06s/it]
 43%|████▎     | 97455/225000 [182:02:34<697:59:05, 19.70s/it]
 43%|████▎     | 97456/225000 [182:02:58<744:18:20, 21.01s/it]
 43%|████▎     | 97457/225000 [182:03:23<792:26:48, 22.37s/it]
 43%|████▎     | 97458/225000 [182:03:44<778:18:44, 21.97s/it]
 43%|████▎     | 97459/225000 [182:04:02<729:49:01, 20.60s/it]
 43%|████▎     | 97460/225000 [182:04:25<754:53:13, 21.31s/it]
 43%|████▎     | 97461/225000 [182:04:40<691:53:44, 19.53s/it]
 43%|████▎     | 97462/225000 [182:05:00<693:00:06, 19.56s/it]
 43%|████▎     | 97463/225000 [182:05:14<632:30:57, 17.85s/it]
 43%|████▎     | 97464/225000 [182:05:37<694:38:16, 19.61s/it]
 43%|████▎     | 97465/225000 [182:05:53<647:22:22, 18.27s/it]
 43%|████▎     | 97466/225000 [182:06:08<612:21:20, 17.29s/it]
 43%|████▎     | 97467/225000 [182:06:26<622:22:36, 17.57s/it]
 43%|████▎     | 97468/225000 [182:06:46<653:45:19, 18.45s/it]
 43%|████▎     | 97469/225000 [182:07:12<725:21:02, 20.48s/it]
 43%|████▎     | 97470/225000 [182:07:30<706:00:26, 19.93s/it]
 43%|████▎     | 97471/225000 [182:07:44<639:51:26, 18.06s/it]
 43%|████▎     | 97472/225000 [182:08:11<735:29:48, 20.76s/it]
 43%|████▎     | 97473/225000 [182:08:34<756:47:17, 21.36s/it]
 43%|████▎     | 97474/225000 [182:08:55<752:37:26, 21.25s/it]
 43%|████▎     | 97475/225000 [182:09:18<776:51:41, 21.93s/it]
 43%|████▎     | 97476/225000 [182:09:31<679:44:34, 19.19s/it]
 43%|████▎     | 97477/225000 [182:09:53<707:02:49, 19.96s/it]
 43%|████▎     | 97478/225000 [182:10:18<758:05:48, 21.40s/it]
 43%|████▎     | 97479/225000 [182:10:43<798:07:35, 22.53s/it]
 43%|████▎     | 97480/225000 [182:11:05<798:08:14, 22.53s/it]
 43%|████▎     | 97481/225000 [182:11:25<767:05:23, 21.66s/it]
 43%|████▎     | 97482/225000 [182:11:42<715:13:15, 20.19s/it]
 43%|████▎     | 97483/225000 [182:11:57<661:23:03, 18.67s/it]
 43%|████▎     | 97484/225000 [182:12:11<611:07:47, 17.25s/it]
 43%|████▎     | 97485/225000 [182:12:29<622:14:04, 17.57s/it]
 43%|████▎     | 97486/225000 [182:12:56<723:59:05, 20.44s/it]
 43%|████▎     | 97487/225000 [182:13:23<788:38:08, 22.27s/it]
 43%|████▎     | 97488/225000 [182:13:42<759:37:56, 21.45s/it]
 43%|████▎     | 97489/225000 [182:14:01<736:54:04, 20.80s/it]
 43%|████▎     | 97490/225000 [182:14:25<770:59:30, 21.77s/it]
 43%|████▎     | 97491/225000 [182:14:50<799:29:55, 22.57s/it]
 43%|████▎     | 97492/225000 [182:15:19<866:48:17, 24.47s/it]
 43%|████▎     | 97493/225000 [182:15:37<804:12:27, 22.71s/it]
 43%|████▎     | 97494/225000 [182:15:59<790:27:46, 22.32s/it]
 43%|████▎     | 97495/225000 [182:16:15<725:03:46, 20.47s/it]
 43%|████▎     | 97496/225000 [182:16:39<757:47:04, 21.40s/it]
 43%|████▎     | 97497/225000 [182:16:56<713:48:31, 20.15s/it]
 43%|████▎     | 97498/225000 [182:17:26<817:50:22, 23.09s/it]
 43%|████▎     | 97499/225000 [182:17:44<769:02:39, 21.71s/it]
 43%|████▎     | 97500/225000 [182:18:03<739:06:17, 20.87s/it]
                                                              
{'loss': '0.8997', 'grad_norm': '1.185', 'learning_rate': '3.222e-05', 'epoch': '0.4333', 'num_input_tokens_seen': 300268136, 'train_runtime': '6.563e+05', 'train_tokens_per_second': '457.5'}

 43%|████▎     | 97500/225000 [182:18:03<739:06:17, 20.87s/it]
 43%|████▎     | 97501/225000 [182:18:19<689:44:04, 19.48s/it]
 43%|████▎     | 97502/225000 [182:18:46<770:37:25, 21.76s/it]
 43%|████▎     | 97503/225000 [182:19:09<781:18:07, 22.06s/it]
 43%|████▎     | 97504/225000 [182:19:21<668:33:04, 18.88s/it]
 43%|████▎     | 97505/225000 [182:19:35<622:15:11, 17.57s/it]
 43%|████▎     | 97506/225000 [182:19:54<633:46:28, 17.90s/it]
 43%|████▎     | 97507/225000 [182:20:13<647:45:02, 18.29s/it]
 43%|████▎     | 97508/225000 [182:20:27<599:21:37, 16.92s/it]
 43%|████▎     | 97509/225000 [182:20:42<579:26:31, 16.36s/it]
 43%|████▎     | 97510/225000 [182:21:12<729:38:59, 20.60s/it]
 43%|████▎     | 97511/225000 [182:21:35<753:42:36, 21.28s/it]
 43%|████▎     | 97512/225000 [182:21:50<686:25:45, 19.38s/it]
 43%|████▎     | 97513/225000 [182:22:11<698:43:19, 19.73s/it]
 43%|████▎     | 97514/225000 [182:22:33<727:30:26, 20.54s/it]
 43%|████▎     | 97515/225000 [182:22:55<745:08:00, 21.04s/it]
 43%|████▎     | 97516/225000 [182:23:14<714:45:58, 20.18s/it]
 43%|████▎     | 97517/225000 [182:23:27<639:46:48, 18.07s/it]
 43%|████▎     | 97518/225000 [182:23:44<627:56:14, 17.73s/it]
 43%|████▎     | 97519/225000 [182:24:01<623:06:25, 17.60s/it]
 43%|████▎     | 97520/225000 [182:24:22<662:22:34, 18.71s/it]
 43%|████▎     | 97521/225000 [182:24:37<616:14:43, 17.40s/it]
 43%|████▎     | 97522/225000 [182:25:03<710:12:40, 20.06s/it]
 43%|████▎     | 97523/225000 [182:25:21<695:37:16, 19.64s/it]
 43%|████▎     | 97524/225000 [182:25:55<838:21:14, 23.68s/it]
 43%|████▎     | 97525/225000 [182:26:13<778:54:53, 22.00s/it]
 43%|████▎     | 97526/225000 [182:26:28<710:47:51, 20.07s/it]
 43%|████▎     | 97527/225000 [182:26:54<770:31:02, 21.76s/it]
 43%|████▎     | 97528/225000 [182:27:05<659:13:13, 18.62s/it]
 43%|████▎     | 97529/225000 [182:27:36<787:02:39, 22.23s/it]
 43%|████▎     | 97530/225000 [182:27:50<699:25:11, 19.75s/it]
 43%|████▎     | 97531/225000 [182:28:10<700:24:17, 19.78s/it]
 43%|████▎     | 97532/225000 [182:28:29<696:10:17, 19.66s/it]
 43%|████▎     | 97533/225000 [182:28:50<712:31:01, 20.12s/it]
 43%|████▎     | 97534/225000 [182:29:13<740:41:12, 20.92s/it]
 43%|████▎     | 97535/225000 [182:29:29<692:32:11, 19.56s/it]
 43%|████▎     | 97536/225000 [182:29:48<684:17:01, 19.33s/it]
 43%|████▎     | 97537/225000 [182:30:08<685:31:06, 19.36s/it]
 43%|████▎     | 97538/225000 [182:30:26<675:53:31, 19.09s/it]
 43%|████▎     | 97539/225000 [182:30:43<649:33:05, 18.35s/it]
 43%|████▎     | 97540/225000 [182:31:01<644:01:13, 18.19s/it]
 43%|████▎     | 97541/225000 [182:31:21<671:22:22, 18.96s/it]
 43%|████▎     | 97542/225000 [182:31:34<604:55:28, 17.09s/it]
 43%|████▎     | 97543/225000 [182:31:59<693:16:14, 19.58s/it]
 43%|████▎     | 97544/225000 [182:32:20<707:42:51, 19.99s/it]
 43%|████▎     | 97545/225000 [182:32:41<717:50:29, 20.28s/it]
 43%|████▎     | 97546/225000 [182:33:09<792:53:06, 22.40s/it]
 43%|████▎     | 97547/225000 [182:33:31<787:33:12, 22.25s/it]
 43%|████▎     | 97548/225000 [182:33:48<735:42:09, 20.78s/it]
 43%|████▎     | 97549/225000 [182:34:02<669:24:03, 18.91s/it]
 43%|████▎     | 97550/225000 [182:34:18<637:45:26, 18.01s/it]
                                                              
{'loss': '0.9036', 'grad_norm': '1.481', 'learning_rate': '3.22e-05', 'epoch': '0.4336', 'num_input_tokens_seen': 300779712, 'train_runtime': '6.573e+05', 'train_tokens_per_second': '457.6'}

 43%|████▎     | 97550/225000 [182:34:18<637:45:26, 18.01s/it]
 43%|████▎     | 97551/225000 [182:34:37<642:10:28, 18.14s/it]
 43%|████▎     | 97552/225000 [182:35:07<772:09:15, 21.81s/it]
 43%|████▎     | 97553/225000 [182:35:23<712:38:06, 20.13s/it]
 43%|████▎     | 97554/225000 [182:35:42<695:28:39, 19.65s/it]
 43%|████▎     | 97555/225000 [182:36:07<757:07:49, 21.39s/it]
 43%|████▎     | 97556/225000 [182:36:28<746:54:31, 21.10s/it]
 43%|████▎     | 97557/225000 [182:36:46<719:10:12, 20.32s/it]
 43%|████▎     | 97558/225000 [182:37:03<676:28:17, 19.11s/it]
 43%|████▎     | 97559/225000 [182:37:26<718:11:10, 20.29s/it]
 43%|████▎     | 97560/225000 [182:37:39<643:33:48, 18.18s/it]
 43%|████▎     | 97561/225000 [182:38:00<679:57:47, 19.21s/it]
 43%|████▎     | 97562/225000 [182:38:16<638:11:50, 18.03s/it]
 43%|████▎     | 97563/225000 [182:38:33<632:07:12, 17.86s/it]
 43%|████▎     | 97564/225000 [182:38:57<698:43:52, 19.74s/it]
 43%|████▎     | 97565/225000 [182:39:15<678:54:57, 19.18s/it]
 43%|████▎     | 97566/225000 [182:39:37<701:13:15, 19.81s/it]
 43%|████▎     | 97567/225000 [182:39:58<719:02:37, 20.31s/it]
 43%|████▎     | 97568/225000 [182:40:17<703:45:10, 19.88s/it]
 43%|████▎     | 97569/225000 [182:40:34<672:42:02, 19.00s/it]
 43%|████▎     | 97570/225000 [182:40:57<712:56:30, 20.14s/it]
 43%|████▎     | 97571/225000 [182:41:10<642:16:41, 18.15s/it]
 43%|████▎     | 97572/225000 [182:41:30<664:45:53, 18.78s/it]
 43%|████▎     | 97573/225000 [182:41:44<608:58:27, 17.20s/it]
 43%|████▎     | 97574/225000 [182:42:11<715:31:45, 20.21s/it]
 43%|████▎     | 97575/225000 [182:42:32<719:54:03, 20.34s/it]
 43%|████▎     | 97576/225000 [182:42:54<740:12:00, 20.91s/it]
 43%|████▎     | 97577/225000 [182:43:13<724:56:21, 20.48s/it]
 43%|████▎     | 97578/225000 [182:43:39<782:01:55, 22.09s/it]
 43%|████▎     | 97579/225000 [182:43:58<750:24:43, 21.20s/it]
 43%|████▎     | 97580/225000 [182:44:28<842:10:40, 23.79s/it]
 43%|████▎     | 97581/225000 [182:44:46<775:34:01, 21.91s/it]
 43%|████▎     | 97582/225000 [182:45:12<823:51:36, 23.28s/it]
 43%|████▎     | 97583/225000 [182:45:25<715:50:04, 20.22s/it]
 43%|████▎     | 97584/225000 [182:45:44<701:12:30, 19.81s/it]
 43%|████▎     | 97585/225000 [182:46:10<760:41:56, 21.49s/it]
 43%|████▎     | 97586/225000 [182:46:28<730:39:40, 20.64s/it]
 43%|████▎     | 97587/225000 [182:46:48<722:27:54, 20.41s/it]
 43%|████▎     | 97588/225000 [182:47:06<691:33:38, 19.54s/it]
 43%|████▎     | 97589/225000 [182:47:33<772:30:28, 21.83s/it]
 43%|████▎     | 97590/225000 [182:47:54<764:07:14, 21.59s/it]
 43%|████▎     | 97591/225000 [182:48:18<787:40:52, 22.26s/it]
 43%|████▎     | 97592/225000 [182:48:30<685:31:07, 19.37s/it]
 43%|████▎     | 97593/225000 [182:48:45<639:54:06, 18.08s/it]
 43%|████▎     | 97594/225000 [182:49:08<685:31:58, 19.37s/it]
 43%|████▎     | 97595/225000 [182:49:31<729:03:30, 20.60s/it]
 43%|████▎     | 97596/225000 [182:49:53<737:37:24, 20.84s/it]
 43%|████▎     | 97597/225000 [182:50:01<605:00:29, 17.10s/it]
 43%|████▎     | 97598/225000 [182:50:26<691:17:08, 19.53s/it]
 43%|████▎     | 97599/225000 [182:50:43<656:29:02, 18.55s/it]
 43%|████▎     | 97600/225000 [182:51:03<672:26:17, 19.00s/it]
                                                              
{'loss': '0.8968', 'grad_norm': '1.324', 'learning_rate': '3.218e-05', 'epoch': '0.4338', 'num_input_tokens_seen': 301289192, 'train_runtime': '6.583e+05', 'train_tokens_per_second': '457.7'}

 43%|████▎     | 97600/225000 [182:51:03<672:26:17, 19.00s/it]
 43%|████▎     | 97601/225000 [182:51:29<746:54:10, 21.11s/it]
 43%|████▎     | 97602/225000 [182:51:53<780:41:49, 22.06s/it]
 43%|████▎     | 97603/225000 [182:52:19<822:37:24, 23.25s/it]
 43%|████▎     | 97604/225000 [182:52:36<757:31:14, 21.41s/it]
 43%|████▎     | 97605/225000 [182:53:01<797:55:19, 22.55s/it]
 43%|████▎     | 97606/225000 [182:53:14<695:32:14, 19.66s/it]
 43%|████▎     | 97607/225000 [182:53:39<752:16:40, 21.26s/it]
 43%|████▎     | 97608/225000 [182:54:06<811:58:11, 22.95s/it]
 43%|████▎     | 97609/225000 [182:54:24<761:16:55, 21.51s/it]
 43%|████▎     | 97610/225000 [182:54:41<708:20:28, 20.02s/it]
 43%|████▎     | 97611/225000 [182:55:02<720:46:40, 20.37s/it]
 43%|████▎     | 97612/225000 [182:55:31<812:29:15, 22.96s/it]
 43%|████▎     | 97613/225000 [182:55:50<776:38:31, 21.95s/it]
 43%|████▎     | 97614/225000 [182:56:15<801:08:28, 22.64s/it]
 43%|████▎     | 97615/225000 [182:56:35<773:47:46, 21.87s/it]
 43%|████▎     | 97616/225000 [182:56:54<749:44:19, 21.19s/it]
 43%|████▎     | 97617/225000 [182:57:15<747:05:56, 21.11s/it]
 43%|████▎     | 97618/225000 [182:57:35<734:47:33, 20.77s/it]
 43%|████▎     | 97619/225000 [182:57:46<632:06:06, 17.86s/it]
 43%|████▎     | 97620/225000 [182:58:09<679:10:50, 19.19s/it]
 43%|████▎     | 97621/225000 [182:58:28<678:47:47, 19.18s/it]
 43%|████▎     | 97622/225000 [182:58:56<776:27:10, 21.94s/it]
 43%|████▎     | 97623/225000 [182:59:15<738:30:15, 20.87s/it]
 43%|████▎     | 97624/225000 [182:59:36<744:24:12, 21.04s/it]
 43%|████▎     | 97625/225000 [182:59:58<755:17:06, 21.35s/it]
 43%|████▎     | 97626/225000 [183:00:17<732:27:15, 20.70s/it]
 43%|████▎     | 97627/225000 [183:00:39<738:27:21, 20.87s/it]
 43%|████▎     | 97628/225000 [183:00:51<653:40:24, 18.48s/it]
 43%|████▎     | 97629/225000 [183:01:09<642:47:51, 18.17s/it]
 43%|████▎     | 97630/225000 [183:01:46<839:09:33, 23.72s/it]
 43%|████▎     | 97631/225000 [183:02:11<862:44:58, 24.39s/it]
 43%|████▎     | 97632/225000 [183:02:30<805:12:26, 22.76s/it]
 43%|████▎     | 97633/225000 [183:02:54<818:39:10, 23.14s/it]
 43%|████▎     | 97634/225000 [183:03:13<766:40:01, 21.67s/it]
 43%|████▎     | 97635/225000 [183:03:33<747:26:58, 21.13s/it]
 43%|████▎     | 97636/225000 [183:03:54<752:28:11, 21.27s/it]
 43%|████▎     | 97637/225000 [183:04:13<727:42:22, 20.57s/it]
 43%|████▎     | 97638/225000 [183:04:37<763:23:35, 21.58s/it]
 43%|████▎     | 97639/225000 [183:04:59<762:53:14, 21.56s/it]
 43%|████▎     | 97640/225000 [183:05:15<712:20:48, 20.14s/it]
 43%|████▎     | 97641/225000 [183:05:42<777:53:28, 21.99s/it]
 43%|████▎     | 97642/225000 [183:06:02<762:50:43, 21.56s/it]
 43%|████▎     | 97643/225000 [183:06:20<718:16:11, 20.30s/it]
 43%|████▎     | 97644/225000 [183:06:49<811:38:03, 22.94s/it]
 43%|████▎     | 97645/225000 [183:07:14<833:12:58, 23.55s/it]
 43%|████▎     | 97646/225000 [183:07:37<831:04:52, 23.49s/it]
 43%|████▎     | 97647/225000 [183:07:49<703:59:22, 19.90s/it]
 43%|████▎     | 97648/225000 [183:08:13<757:02:40, 21.40s/it]
 43%|████▎     | 97649/225000 [183:08:32<731:06:05, 20.67s/it]
 43%|████▎     | 97650/225000 [183:08:56<761:25:08, 21.52s/it]
                                                              
{'loss': '0.9168', 'grad_norm': '1.179', 'learning_rate': '3.217e-05', 'epoch': '0.434', 'num_input_tokens_seen': 301835016, 'train_runtime': '6.593e+05', 'train_tokens_per_second': '457.8'}

 43%|████▎     | 97650/225000 [183:08:56<761:25:08, 21.52s/it]
 43%|████▎     | 97651/225000 [183:09:19<773:12:55, 21.86s/it]
 43%|████▎     | 97652/225000 [183:09:39<761:01:19, 21.51s/it]
 43%|████▎     | 97653/225000 [183:10:02<775:50:59, 21.93s/it]
 43%|████▎     | 97654/225000 [183:10:28<817:51:28, 23.12s/it]
 43%|████▎     | 97655/225000 [183:10:56<872:01:52, 24.65s/it]
 43%|████▎     | 97656/225000 [183:11:14<795:34:01, 22.49s/it]
 43%|████▎     | 97657/225000 [183:11:32<747:10:29, 21.12s/it]
 43%|████▎     | 97658/225000 [183:11:53<753:29:43, 21.30s/it]
 43%|████▎     | 97659/225000 [183:12:17<778:26:13, 22.01s/it]
 43%|████▎     | 97660/225000 [183:12:40<783:16:29, 22.14s/it]
 43%|████▎     | 97661/225000 [183:12:56<718:20:21, 20.31s/it]
 43%|████▎     | 97662/225000 [183:13:16<716:26:29, 20.25s/it]
 43%|████▎     | 97663/225000 [183:13:32<679:16:24, 19.20s/it]
 43%|████▎     | 97664/225000 [183:14:00<762:42:33, 21.56s/it]
 43%|████▎     | 97665/225000 [183:14:14<683:38:25, 19.33s/it]
 43%|████▎     | 97666/225000 [183:14:36<720:15:16, 20.36s/it]
 43%|████▎     | 97667/225000 [183:14:57<719:39:42, 20.35s/it]
 43%|████▎     | 97668/225000 [183:15:23<778:28:17, 22.01s/it]
 43%|████▎     | 97669/225000 [183:15:47<804:04:43, 22.73s/it]
 43%|████▎     | 97670/225000 [183:16:05<758:24:53, 21.44s/it]
 43%|████▎     | 97671/225000 [183:16:20<689:40:01, 19.50s/it]
 43%|████▎     | 97672/225000 [183:16:41<700:44:58, 19.81s/it]
 43%|████▎     | 97673/225000 [183:16:58<666:24:43, 18.84s/it]
 43%|████▎     | 97674/225000 [183:17:20<701:47:01, 19.84s/it]
 43%|████▎     | 97675/225000 [183:17:42<729:20:45, 20.62s/it]
 43%|████▎     | 97676/225000 [183:18:04<744:25:20, 21.05s/it]
 43%|████▎     | 97677/225000 [183:18:17<658:56:24, 18.63s/it]
 43%|████▎     | 97678/225000 [183:18:41<709:23:18, 20.06s/it]
 43%|████▎     | 97679/225000 [183:18:58<684:07:19, 19.34s/it]
 43%|████▎     | 97680/225000 [183:19:17<673:24:27, 19.04s/it]
 43%|████▎     | 97681/225000 [183:19:31<625:46:21, 17.69s/it]
 43%|████▎     | 97682/225000 [183:19:46<598:14:38, 16.92s/it]
 43%|████▎     | 97683/225000 [183:20:00<567:17:44, 16.04s/it]
 43%|████▎     | 97684/225000 [183:20:30<707:39:48, 20.01s/it]
 43%|████▎     | 97685/225000 [183:20:43<637:59:37, 18.04s/it]
 43%|████▎     | 97686/225000 [183:21:00<624:23:48, 17.66s/it]
 43%|████▎     | 97687/225000 [183:21:25<702:30:04, 19.86s/it]
 43%|████▎     | 97688/225000 [183:21:50<755:27:38, 21.36s/it]
 43%|████▎     | 97689/225000 [183:22:03<667:45:44, 18.88s/it]
 43%|████▎     | 97690/225000 [183:22:25<705:07:26, 19.94s/it]
 43%|████▎     | 97691/225000 [183:22:44<694:21:28, 19.63s/it]
 43%|████▎     | 97692/225000 [183:23:09<753:34:50, 21.31s/it]
 43%|████▎     | 97693/225000 [183:23:35<797:54:24, 22.56s/it]
 43%|████▎     | 97694/225000 [183:23:57<794:00:01, 22.45s/it]
 43%|████▎     | 97695/225000 [183:24:10<691:02:09, 19.54s/it]
 43%|████▎     | 97696/225000 [183:24:24<636:07:09, 17.99s/it]
 43%|████▎     | 97697/225000 [183:24:48<698:53:47, 19.76s/it]
 43%|████▎     | 97698/225000 [183:25:13<756:35:21, 21.40s/it]
 43%|████▎     | 97699/225000 [183:25:34<745:35:59, 21.09s/it]
 43%|████▎     | 97700/225000 [183:25:53<723:40:19, 20.47s/it]
                                                              
{'loss': '0.8828', 'grad_norm': '1.144', 'learning_rate': '3.215e-05', 'epoch': '0.4342', 'num_input_tokens_seen': 302365936, 'train_runtime': '6.604e+05', 'train_tokens_per_second': '457.9'}

 43%|████▎     | 97700/225000 [183:25:53<723:40:19, 20.47s/it]
 43%|████▎     | 97701/225000 [183:26:10<690:05:55, 19.52s/it]
 43%|████▎     | 97702/225000 [183:26:31<710:19:51, 20.09s/it]
 43%|████▎     | 97703/225000 [183:26:55<750:31:25, 21.23s/it]
 43%|████▎     | 97704/225000 [183:27:18<768:10:02, 21.72s/it]
 43%|████▎     | 97705/225000 [183:27:36<733:24:55, 20.74s/it]
 43%|████▎     | 97706/225000 [183:28:00<763:17:20, 21.59s/it]
 43%|████▎     | 97707/225000 [183:28:19<739:51:10, 20.92s/it]
 43%|████▎     | 97708/225000 [183:28:42<753:35:48, 21.31s/it]
 43%|████▎     | 97709/225000 [183:29:02<739:33:32, 20.92s/it]
 43%|████▎     | 97710/225000 [183:29:14<643:42:02, 18.21s/it]
 43%|████▎     | 97711/225000 [183:29:30<626:09:18, 17.71s/it]
 43%|████▎     | 97712/225000 [183:29:46<610:45:16, 17.27s/it]
 43%|████▎     | 97713/225000 [183:30:02<598:33:06, 16.93s/it]
 43%|████▎     | 97714/225000 [183:30:16<560:47:05, 15.86s/it]
 43%|████▎     | 97715/225000 [183:30:36<607:02:16, 17.17s/it]
 43%|████▎     | 97716/225000 [183:30:56<640:50:52, 18.13s/it]
 43%|████▎     | 97717/225000 [183:31:17<663:25:36, 18.76s/it]
 43%|████▎     | 97718/225000 [183:31:35<656:21:36, 18.56s/it]
 43%|████▎     | 97719/225000 [183:32:02<748:56:35, 21.18s/it]
 43%|████▎     | 97720/225000 [183:32:26<779:07:44, 22.04s/it]
 43%|████▎     | 97721/225000 [183:32:43<722:50:53, 20.45s/it]
 43%|████▎     | 97722/225000 [183:33:02<707:41:38, 20.02s/it]
 43%|████▎     | 97723/225000 [183:33:25<737:06:10, 20.85s/it]
 43%|████▎     | 97724/225000 [183:33:46<737:58:57, 20.87s/it]
 43%|████▎     | 97725/225000 [183:34:04<712:35:21, 20.16s/it]
 43%|████▎     | 97726/225000 [183:34:29<760:52:11, 21.52s/it]
 43%|████▎     | 97727/225000 [183:34:46<713:46:33, 20.19s/it]
 43%|████▎     | 97728/225000 [183:35:13<783:21:03, 22.16s/it]
 43%|████▎     | 97729/225000 [183:35:28<715:18:55, 20.23s/it]
 43%|████▎     | 97730/225000 [183:35:48<705:51:04, 19.97s/it]
 43%|████▎     | 97731/225000 [183:36:04<667:24:04, 18.88s/it]
 43%|████▎     | 97732/225000 [183:36:30<746:06:08, 21.10s/it]
 43%|████▎     | 97733/225000 [183:36:52<756:12:08, 21.39s/it]
 43%|████▎     | 97734/225000 [183:37:15<771:59:01, 21.84s/it]
 43%|████▎     | 97735/225000 [183:37:27<663:02:19, 18.76s/it]
 43%|████▎     | 97736/225000 [183:37:46<668:11:09, 18.90s/it]
 43%|████▎     | 97737/225000 [183:38:10<719:50:01, 20.36s/it]
 43%|████▎     | 97738/225000 [183:38:26<679:07:36, 19.21s/it]
 43%|████▎     | 97739/225000 [183:38:57<800:03:06, 22.63s/it]
 43%|████▎     | 97740/225000 [183:39:22<824:01:08, 23.31s/it]
 43%|████▎     | 97741/225000 [183:39:52<891:42:28, 25.23s/it]
 43%|████▎     | 97742/225000 [183:40:14<862:53:01, 24.41s/it]
 43%|████▎     | 97743/225000 [183:40:37<848:30:06, 24.00s/it]
 43%|████▎     | 97744/225000 [183:41:00<839:53:21, 23.76s/it]
 43%|████▎     | 97745/225000 [183:41:20<796:02:16, 22.52s/it]
 43%|████▎     | 97746/225000 [183:41:43<802:46:38, 22.71s/it]
 43%|████▎     | 97747/225000 [183:42:08<831:18:55, 23.52s/it]
 43%|████▎     | 97748/225000 [183:42:24<747:00:19, 21.13s/it]
 43%|████▎     | 97749/225000 [183:42:40<697:17:13, 19.73s/it]
 43%|████▎     | 97750/225000 [183:43:08<778:18:06, 22.02s/it]
                                                              
{'loss': '0.8957', 'grad_norm': '1.351', 'learning_rate': '3.213e-05', 'epoch': '0.4344', 'num_input_tokens_seen': 302902776, 'train_runtime': '6.614e+05', 'train_tokens_per_second': '458'}

 43%|████▎     | 97750/225000 [183:43:08<778:18:06, 22.02s/it]
 43%|████▎     | 97751/225000 [183:43:24<718:27:13, 20.33s/it]
 43%|████▎     | 97752/225000 [183:43:40<670:23:25, 18.97s/it]
 43%|████▎     | 97753/225000 [183:44:01<691:04:46, 19.55s/it]
 43%|████▎     | 97754/225000 [183:44:32<809:05:52, 22.89s/it]
 43%|████▎     | 97755/225000 [183:44:42<678:53:43, 19.21s/it]
 43%|████▎     | 97756/225000 [183:45:03<700:47:26, 19.83s/it]
 43%|████▎     | 97757/225000 [183:45:26<725:54:43, 20.54s/it]
 43%|████▎     | 97758/225000 [183:45:41<670:38:08, 18.97s/it]
 43%|████▎     | 97759/225000 [183:46:07<746:19:46, 21.12s/it]
 43%|████▎     | 97760/225000 [183:46:25<712:34:43, 20.16s/it]
 43%|████▎     | 97761/225000 [183:46:48<737:53:31, 20.88s/it]
 43%|████▎     | 97762/225000 [183:47:12<778:20:18, 22.02s/it]
 43%|████▎     | 97763/225000 [183:47:35<786:35:32, 22.26s/it]
 43%|████▎     | 97764/225000 [183:47:53<743:32:30, 21.04s/it]
 43%|████▎     | 97765/225000 [183:48:12<720:32:03, 20.39s/it]
 43%|████▎     | 97766/225000 [183:48:32<715:20:51, 20.24s/it]
 43%|████▎     | 97767/225000 [183:48:52<715:33:21, 20.25s/it]
 43%|████▎     | 97768/225000 [183:49:17<760:01:56, 21.50s/it]
 43%|████▎     | 97769/225000 [183:49:46<838:47:57, 23.73s/it]
 43%|████▎     | 97770/225000 [183:50:03<770:28:30, 21.80s/it]
 43%|████▎     | 97771/225000 [183:50:23<755:26:03, 21.38s/it]
 43%|████▎     | 97772/225000 [183:50:43<737:59:09, 20.88s/it]
 43%|████▎     | 97773/225000 [183:51:10<804:57:05, 22.78s/it]
 43%|████▎     | 97774/225000 [183:51:27<741:11:52, 20.97s/it]
 43%|████▎     | 97775/225000 [183:51:49<749:52:44, 21.22s/it]
 43%|████▎     | 97776/225000 [183:52:12<772:50:12, 21.87s/it]
 43%|████▎     | 97777/225000 [183:52:32<748:29:09, 21.18s/it]
 43%|████▎     | 97778/225000 [183:52:54<755:37:50, 21.38s/it]
 43%|████▎     | 97779/225000 [183:53:14<745:12:59, 21.09s/it]
 43%|████▎     | 97780/225000 [183:53:39<787:35:15, 22.29s/it]
 43%|████▎     | 97781/225000 [183:53:57<742:43:18, 21.02s/it]
 43%|████▎     | 97782/225000 [183:54:18<743:34:08, 21.04s/it]
 43%|████▎     | 97783/225000 [183:54:36<711:40:31, 20.14s/it]
 43%|████▎     | 97784/225000 [183:54:54<688:57:35, 19.50s/it]
 43%|████▎     | 97785/225000 [183:55:26<814:10:48, 23.04s/it]
 43%|████▎     | 97786/225000 [183:55:48<805:37:47, 22.80s/it]
 43%|████▎     | 97787/225000 [183:56:12<820:14:52, 23.21s/it]
 43%|████▎     | 97788/225000 [183:56:30<760:06:05, 21.51s/it]
 43%|████▎     | 97789/225000 [183:56:52<767:54:54, 21.73s/it]
 43%|████▎     | 97790/225000 [183:57:19<822:48:12, 23.29s/it]
 43%|████▎     | 97791/225000 [183:57:35<745:57:51, 21.11s/it]
 43%|████▎     | 97792/225000 [183:58:08<876:07:42, 24.79s/it]
 43%|████▎     | 97793/225000 [183:58:24<776:00:49, 21.96s/it]
 43%|████▎     | 97794/225000 [183:58:39<702:41:50, 19.89s/it]
 43%|████▎     | 97795/225000 [183:58:58<694:36:55, 19.66s/it]
 43%|████▎     | 97796/225000 [183:59:24<766:55:46, 21.70s/it]
 43%|████▎     | 97797/225000 [183:59:52<828:24:37, 23.45s/it]
 43%|████▎     | 97798/225000 [184:00:14<813:27:58, 23.02s/it]
 43%|████▎     | 97799/225000 [184:00:39<840:44:58, 23.79s/it]
 43%|████▎     | 97800/225000 [184:00:59<799:20:02, 22.62s/it]
                                                              
{'loss': '0.8876', 'grad_norm': '1.66', 'learning_rate': '3.211e-05', 'epoch': '0.4347', 'num_input_tokens_seen': 303449856, 'train_runtime': '6.625e+05', 'train_tokens_per_second': '458.1'}

 43%|████▎     | 97800/225000 [184:00:59<799:20:02, 22.62s/it]
 43%|████▎     | 97801/225000 [184:01:22<796:01:35, 22.53s/it]
 43%|████▎     | 97802/225000 [184:01:46<815:26:18, 23.08s/it]
 43%|████▎     | 97803/225000 [184:02:00<720:36:59, 20.40s/it]
 43%|████▎     | 97804/225000 [184:02:27<785:44:50, 22.24s/it]
 43%|████▎     | 97805/225000 [184:02:44<739:20:28, 20.93s/it]
 43%|████▎     | 97806/225000 [184:03:06<749:44:51, 21.22s/it]
 43%|████▎     | 97807/225000 [184:03:24<710:24:30, 20.11s/it]
 43%|████▎     | 97808/225000 [184:03:52<794:53:07, 22.50s/it]
 43%|████▎     | 97809/225000 [184:04:16<816:47:37, 23.12s/it]
 43%|████▎     | 97810/225000 [184:04:39<806:31:02, 22.83s/it]
 43%|████▎     | 97811/225000 [184:05:05<841:26:08, 23.82s/it]
 43%|████▎     | 97812/225000 [184:05:30<854:52:31, 24.20s/it]
 43%|████▎     | 97813/225000 [184:05:49<802:43:53, 22.72s/it]
 43%|████▎     | 97814/225000 [184:06:15<832:22:15, 23.56s/it]
 43%|████▎     | 97815/225000 [184:06:34<783:23:08, 22.17s/it]
 43%|████▎     | 97816/225000 [184:06:58<802:58:06, 22.73s/it]
 43%|████▎     | 97817/225000 [184:07:16<761:21:14, 21.55s/it]
 43%|████▎     | 97818/225000 [184:07:36<736:14:04, 20.84s/it]
 43%|████▎     | 97819/225000 [184:08:02<796:36:00, 22.55s/it]
 43%|████▎     | 97820/225000 [184:08:28<836:43:39, 23.68s/it]
 43%|████▎     | 97821/225000 [184:08:44<750:17:38, 21.24s/it]
 43%|████▎     | 97822/225000 [184:09:10<799:10:38, 22.62s/it]
 43%|████▎     | 97823/225000 [184:09:26<729:56:42, 20.66s/it]
 43%|████▎     | 97824/225000 [184:09:46<724:41:09, 20.51s/it]
 43%|████▎     | 97825/225000 [184:10:07<732:00:42, 20.72s/it]
 43%|████▎     | 97826/225000 [184:10:26<710:52:16, 20.12s/it]
 43%|████▎     | 97827/225000 [184:10:50<755:03:49, 21.37s/it]
 43%|████▎     | 97828/225000 [184:11:08<720:29:39, 20.40s/it]
 43%|████▎     | 97829/225000 [184:11:33<770:00:53, 21.80s/it]
 43%|████▎     | 97830/225000 [184:11:53<740:31:52, 20.96s/it]
 43%|████▎     | 97831/225000 [184:12:17<774:59:38, 21.94s/it]
 43%|████▎     | 97832/225000 [184:12:44<828:01:17, 23.44s/it]
 43%|████▎     | 97833/225000 [184:13:08<836:40:39, 23.69s/it]
 43%|████▎     | 97834/225000 [184:13:19<699:44:45, 19.81s/it]
 43%|████▎     | 97835/225000 [184:13:34<653:00:02, 18.49s/it]
 43%|████▎     | 97836/225000 [184:13:57<702:06:16, 19.88s/it]
 43%|████▎     | 97837/225000 [184:14:20<731:16:28, 20.70s/it]
 43%|████▎     | 97838/225000 [184:14:37<697:46:57, 19.75s/it]
 43%|████▎     | 97839/225000 [184:15:00<724:25:17, 20.51s/it]
 43%|████▎     | 97840/225000 [184:15:19<710:08:09, 20.10s/it]
 43%|████▎     | 97841/225000 [184:15:32<641:07:47, 18.15s/it]
 43%|████▎     | 97842/225000 [184:15:52<654:00:46, 18.52s/it]
 43%|████▎     | 97843/225000 [184:16:11<661:03:55, 18.72s/it]
 43%|████▎     | 97844/225000 [184:16:41<780:34:45, 22.10s/it]
 43%|████▎     | 97845/225000 [184:17:02<770:36:10, 21.82s/it]
 43%|████▎     | 97846/225000 [184:17:26<793:22:47, 22.46s/it]
 43%|████▎     | 97847/225000 [184:17:55<864:30:11, 24.48s/it]
 43%|████▎     | 97848/225000 [184:18:10<761:34:20, 21.56s/it]
 43%|████▎     | 97849/225000 [184:18:31<750:48:38, 21.26s/it]
 43%|████▎     | 97850/225000 [184:18:48<711:31:55, 20.15s/it]
                                                              
{'loss': '0.8809', 'grad_norm': '1.4', 'learning_rate': '3.21e-05', 'epoch': '0.4349', 'num_input_tokens_seen': 303988432, 'train_runtime': '6.635e+05', 'train_tokens_per_second': '458.1'}

 43%|████▎     | 97850/225000 [184:18:48<711:31:55, 20.15s/it]
 43%|████▎     | 97851/225000 [184:19:11<739:37:41, 20.94s/it]
 43%|████▎     | 97852/225000 [184:19:34<763:42:10, 21.62s/it]
 43%|████▎     | 97853/225000 [184:19:48<683:49:05, 19.36s/it]
 43%|████▎     | 97854/225000 [184:20:06<664:52:18, 18.83s/it]
 43%|████▎     | 97855/225000 [184:20:31<733:19:22, 20.76s/it]
 43%|████▎     | 97856/225000 [184:20:50<713:18:11, 20.20s/it]
 43%|████▎     | 97857/225000 [184:21:07<684:34:14, 19.38s/it]
 43%|████▎     | 97858/225000 [184:21:27<682:19:28, 19.32s/it]
 43%|████▎     | 97859/225000 [184:21:58<808:04:10, 22.88s/it]
 43%|████▎     | 97860/225000 [184:22:18<778:15:59, 22.04s/it]
 43%|████▎     | 97861/225000 [184:22:35<726:19:43, 20.57s/it]
 43%|████▎     | 97862/225000 [184:22:56<725:49:52, 20.55s/it]
 43%|████▎     | 97863/225000 [184:23:18<742:55:35, 21.04s/it]
 43%|████▎     | 97864/225000 [184:23:38<730:14:52, 20.68s/it]
 43%|████▎     | 97865/225000 [184:23:58<725:51:32, 20.55s/it]
 43%|████▎     | 97866/225000 [184:24:23<771:18:18, 21.84s/it]
 43%|████▎     | 97867/225000 [184:24:42<745:10:34, 21.10s/it]
 43%|████▎     | 97868/225000 [184:25:03<746:51:52, 21.15s/it]
 43%|████▎     | 97869/225000 [184:25:27<776:43:03, 21.99s/it]
 43%|████▎     | 97870/225000 [184:25:41<685:58:24, 19.43s/it]
 43%|████▎     | 97871/225000 [184:26:07<764:23:16, 21.65s/it]
 43%|████▎     | 97872/225000 [184:26:26<735:12:09, 20.82s/it]
 43%|████▎     | 97873/225000 [184:26:39<650:05:55, 18.41s/it]
 43%|████▎     | 97874/225000 [184:27:01<681:39:32, 19.30s/it]
 44%|████▎     | 97875/225000 [184:27:22<701:22:57, 19.86s/it]
 44%|████▎     | 97876/225000 [184:27:46<750:28:23, 21.25s/it]
 44%|████▎     | 97877/225000 [184:28:08<757:20:16, 21.45s/it]
 44%|████▎     | 97878/225000 [184:28:33<791:13:50, 22.41s/it]
 44%|████▎     | 97879/225000 [184:28:58<818:11:25, 23.17s/it]
 44%|████▎     | 97880/225000 [184:29:10<706:14:29, 20.00s/it]
 44%|████▎     | 97881/225000 [184:29:36<762:55:03, 21.61s/it]
 44%|████▎     | 97882/225000 [184:29:51<698:40:06, 19.79s/it]
 44%|████▎     | 97883/225000 [184:30:13<722:17:13, 20.46s/it]
 44%|████▎     | 97884/225000 [184:30:35<734:08:55, 20.79s/it]
 44%|████▎     | 97885/225000 [184:30:56<736:27:04, 20.86s/it]
 44%|████▎     | 97886/225000 [184:31:22<791:18:59, 22.41s/it]
 44%|████▎     | 97887/225000 [184:31:42<767:00:03, 21.72s/it]
 44%|████▎     | 97888/225000 [184:31:59<721:41:27, 20.44s/it]
 44%|████▎     | 97889/225000 [184:32:26<783:55:36, 22.20s/it]
 44%|████▎     | 97890/225000 [184:32:50<808:02:22, 22.89s/it]
 44%|████▎     | 97891/225000 [184:33:15<827:45:51, 23.44s/it]
 44%|████▎     | 97892/225000 [184:33:35<789:34:33, 22.36s/it]
 44%|████▎     | 97893/225000 [184:33:56<776:58:14, 22.01s/it]
 44%|████▎     | 97894/225000 [184:34:18<774:54:03, 21.95s/it]
 44%|████▎     | 97895/225000 [184:34:40<779:45:37, 22.09s/it]
 44%|████▎     | 97896/225000 [184:35:06<815:29:38, 23.10s/it]
 44%|████▎     | 97897/225000 [184:35:16<678:35:29, 19.22s/it]
 44%|████▎     | 97898/225000 [184:35:31<638:16:22, 18.08s/it]
 44%|████▎     | 97899/225000 [184:36:01<763:32:13, 21.63s/it]
 44%|████▎     | 97900/225000 [184:36:19<728:07:24, 20.62s/it]
                                                              
{'loss': '0.898', 'grad_norm': '1.153', 'learning_rate': '3.208e-05', 'epoch': '0.4351', 'num_input_tokens_seen': 304520944, 'train_runtime': '6.646e+05', 'train_tokens_per_second': '458.2'}

 44%|████▎     | 97900/225000 [184:36:19<728:07:24, 20.62s/it]
 44%|████▎     | 97901/225000 [184:36:41<740:17:15, 20.97s/it]
 44%|████▎     | 97902/225000 [184:36:57<690:46:05, 19.57s/it]
 44%|████▎     | 97903/225000 [184:37:32<852:29:35, 24.15s/it]
 44%|████▎     | 97904/225000 [184:37:54<824:19:22, 23.35s/it]
 44%|████▎     | 97905/225000 [184:38:23<884:07:57, 25.04s/it]
 44%|████▎     | 97906/225000 [184:38:42<822:42:23, 23.30s/it]
 44%|████▎     | 97907/225000 [184:39:10<871:43:47, 24.69s/it]
 44%|████▎     | 97908/225000 [184:39:33<851:36:14, 24.12s/it]
 44%|████▎     | 97909/225000 [184:39:53<807:10:11, 22.86s/it]
 44%|████▎     | 97910/225000 [184:40:14<795:33:40, 22.54s/it]
 44%|████▎     | 97911/225000 [184:40:28<700:45:19, 19.85s/it]
 44%|████▎     | 97912/225000 [184:40:45<670:51:39, 19.00s/it]
 44%|████▎     | 97913/225000 [184:41:07<698:34:52, 19.79s/it]
 44%|████▎     | 97914/225000 [184:41:24<671:22:59, 19.02s/it]
 44%|████▎     | 97915/225000 [184:41:43<667:22:04, 18.90s/it]
 44%|████▎     | 97916/225000 [184:42:11<764:45:58, 21.66s/it]
 44%|████▎     | 97917/225000 [184:42:37<816:59:23, 23.14s/it]
 44%|████▎     | 97918/225000 [184:42:56<767:37:37, 21.75s/it]
 44%|████▎     | 97919/225000 [184:43:18<777:47:06, 22.03s/it]
 44%|████▎     | 97920/225000 [184:43:40<773:07:45, 21.90s/it]
 44%|████▎     | 97921/225000 [184:44:06<817:32:52, 23.16s/it]
 44%|████▎     | 97922/225000 [184:44:24<761:01:32, 21.56s/it]
 44%|████▎     | 97923/225000 [184:44:44<746:18:03, 21.14s/it]
 44%|████▎     | 97924/225000 [184:44:59<682:52:58, 19.35s/it]
 44%|████▎     | 97925/225000 [184:45:31<815:14:51, 23.10s/it]
 44%|████▎     | 97926/225000 [184:45:52<794:26:09, 22.51s/it]
 44%|████▎     | 97927/225000 [184:46:07<707:35:51, 20.05s/it]
 44%|████▎     | 97928/225000 [184:46:33<769:59:45, 21.81s/it]
 44%|████▎     | 97929/225000 [184:46:47<687:48:03, 19.49s/it]
 44%|████▎     | 97930/225000 [184:47:13<756:57:37, 21.45s/it]
 44%|████▎     | 97931/225000 [184:47:31<726:18:45, 20.58s/it]
 44%|████▎     | 97932/225000 [184:47:56<767:39:16, 21.75s/it]
 44%|████▎     | 97933/225000 [184:48:15<741:30:47, 21.01s/it]
 44%|████▎     | 97934/225000 [184:48:35<731:33:29, 20.73s/it]
 44%|████▎     | 97935/225000 [184:48:57<743:31:52, 21.07s/it]
 44%|████▎     | 97936/225000 [184:49:19<753:56:33, 21.36s/it]
 44%|████▎     | 97937/225000 [184:49:42<773:57:35, 21.93s/it]
 44%|████▎     | 97938/225000 [184:50:02<756:03:02, 21.42s/it]
 44%|████▎     | 97939/225000 [184:50:22<734:19:24, 20.81s/it]
 44%|████▎     | 97940/225000 [184:50:45<758:52:01, 21.50s/it]
 44%|████▎     | 97941/225000 [184:51:08<776:04:42, 21.99s/it]
 44%|████▎     | 97942/225000 [184:51:26<738:30:04, 20.92s/it]
 44%|████▎     | 97943/225000 [184:51:46<722:41:25, 20.48s/it]
 44%|████▎     | 97944/225000 [184:52:09<753:14:01, 21.34s/it]
 44%|████▎     | 97945/225000 [184:52:36<805:46:22, 22.83s/it]
 44%|████▎     | 97946/225000 [184:52:54<756:08:39, 21.42s/it]
 44%|████▎     | 97947/225000 [184:53:13<738:48:58, 20.93s/it]
 44%|████▎     | 97948/225000 [184:53:33<721:26:50, 20.44s/it]
 44%|████▎     | 97949/225000 [184:53:50<686:07:29, 19.44s/it]
 44%|████▎     | 97950/225000 [184:54:18<778:15:44, 22.05s/it]
                                                              
{'loss': '0.9083', 'grad_norm': '0.9744', 'learning_rate': '3.206e-05', 'epoch': '0.4353', 'num_input_tokens_seen': 305062072, 'train_runtime': '6.657e+05', 'train_tokens_per_second': '458.3'}

 44%|████▎     | 97950/225000 [184:54:18<778:15:44, 22.05s/it]
 44%|████▎     | 97951/225000 [184:54:40<775:37:19, 21.98s/it]
 44%|████▎     | 97952/225000 [184:55:04<796:34:20, 22.57s/it]
 44%|████▎     | 97953/225000 [184:55:18<706:03:56, 20.01s/it]
 44%|████▎     | 97954/225000 [184:55:34<670:50:15, 19.01s/it]
 44%|████▎     | 97955/225000 [184:55:58<714:36:11, 20.25s/it]
 44%|████▎     | 97956/225000 [184:56:21<749:58:02, 21.25s/it]
 44%|████▎     | 97957/225000 [184:56:42<743:46:43, 21.08s/it]
 44%|████▎     | 97958/225000 [184:56:58<694:38:49, 19.68s/it]
 44%|████▎     | 97959/225000 [184:57:17<680:54:39, 19.30s/it]
 44%|████▎     | 97960/225000 [184:57:25<567:18:58, 16.08s/it]
 44%|████▎     | 97961/225000 [184:57:50<662:00:54, 18.76s/it]
 44%|████▎     | 97962/225000 [184:58:11<680:54:20, 19.30s/it]
 44%|████▎     | 97963/225000 [184:58:31<693:57:59, 19.67s/it]
 44%|████▎     | 97964/225000 [184:58:57<753:19:44, 21.35s/it]
 44%|████▎     | 97965/225000 [184:59:19<765:32:27, 21.69s/it]
 44%|████▎     | 97966/225000 [184:59:41<768:09:24, 21.77s/it]
 44%|████▎     | 97967/225000 [185:00:01<749:23:55, 21.24s/it]
 44%|████▎     | 97968/225000 [185:00:32<850:23:22, 24.10s/it]
 44%|████▎     | 97969/225000 [185:00:49<780:04:56, 22.11s/it]
 44%|████▎     | 97970/225000 [185:01:11<778:07:04, 22.05s/it]
 44%|████▎     | 97971/225000 [185:01:39<838:19:58, 23.76s/it]
 44%|████▎     | 97972/225000 [185:01:58<788:10:01, 22.34s/it]
 44%|████▎     | 97973/225000 [185:02:20<781:32:46, 22.15s/it]
 44%|████▎     | 97974/225000 [185:02:37<726:14:55, 20.58s/it]
 44%|████▎     | 97975/225000 [185:02:57<719:56:52, 20.40s/it]
 44%|████▎     | 97976/225000 [185:03:22<775:44:47, 21.99s/it]
 44%|████▎     | 97977/225000 [185:03:37<703:55:55, 19.95s/it]
 44%|████▎     | 97978/225000 [185:03:53<657:29:08, 18.63s/it]
 44%|████▎     | 97979/225000 [185:04:11<645:47:33, 18.30s/it]
 44%|████▎     | 97980/225000 [185:04:36<719:28:36, 20.39s/it]
 44%|████▎     | 97981/225000 [185:04:48<628:23:45, 17.81s/it]
 44%|████▎     | 97982/225000 [185:05:17<748:54:37, 21.23s/it]
 44%|████▎     | 97983/225000 [185:05:31<670:52:24, 19.01s/it]
 44%|████▎     | 97984/225000 [185:05:56<737:47:01, 20.91s/it]
 44%|████▎     | 97985/225000 [185:06:22<792:35:58, 22.46s/it]
 44%|████▎     | 97986/225000 [185:06:46<806:14:33, 22.85s/it]
 44%|████▎     | 97987/225000 [185:06:58<693:18:07, 19.65s/it]
 44%|████▎     | 97988/225000 [185:07:21<727:05:23, 20.61s/it]
 44%|████▎     | 97989/225000 [185:07:44<756:25:24, 21.44s/it]
 44%|████▎     | 97990/225000 [185:08:11<808:23:36, 22.91s/it]
 44%|████▎     | 97991/225000 [185:08:29<761:47:41, 21.59s/it]
 44%|████▎     | 97992/225000 [185:08:52<778:04:12, 22.05s/it]
 44%|████▎     | 97993/225000 [185:09:11<747:20:17, 21.18s/it]
 44%|████▎     | 97994/225000 [185:09:31<733:58:32, 20.80s/it]
 44%|████▎     | 97995/225000 [185:09:58<790:43:08, 22.41s/it]
 44%|████▎     | 97996/225000 [185:10:15<743:34:06, 21.08s/it]
 44%|████▎     | 97997/225000 [185:10:33<711:00:38, 20.15s/it]
 44%|████▎     | 97998/225000 [185:11:06<845:36:50, 23.97s/it]
 44%|████▎     | 97999/225000 [185:11:35<893:17:24, 25.32s/it]
 44%|████▎     | 98000/225000 [185:12:00<887:07:15, 25.15s/it]
                                                              
{'loss': '0.9195', 'grad_norm': '1.012', 'learning_rate': '3.204e-05', 'epoch': '0.4356', 'num_input_tokens_seen': 305606696, 'train_runtime': '6.667e+05', 'train_tokens_per_second': '458.4'}

 44%|████▎     | 98000/225000 [185:12:00<887:07:15, 25.15s/it]
 44%|████▎     | 98001/225000 [185:12:20<834:34:53, 23.66s/it]
 44%|████▎     | 98002/225000 [185:12:36<752:33:16, 21.33s/it]
 44%|████▎     | 98003/225000 [185:12:57<749:20:30, 21.24s/it]
 44%|████▎     | 98004/225000 [185:13:13<701:52:07, 19.90s/it]
 44%|████▎     | 98005/225000 [185:13:28<643:58:31, 18.26s/it]
 44%|████▎     | 98006/225000 [185:14:00<792:41:45, 22.47s/it]
 44%|████▎     | 98007/225000 [185:14:26<833:06:15, 23.62s/it]
 44%|████▎     | 98008/225000 [185:14:48<810:40:52, 22.98s/it]
 44%|████▎     | 98009/225000 [185:15:08<778:14:59, 22.06s/it]
 44%|████▎     | 98010/225000 [185:15:26<739:39:49, 20.97s/it]
 44%|████▎     | 98011/225000 [185:15:41<677:01:42, 19.19s/it]
 44%|████▎     | 98012/225000 [185:16:02<695:09:29, 19.71s/it]
 44%|████▎     | 98013/225000 [185:16:26<739:32:10, 20.97s/it]
 44%|████▎     | 98014/225000 [185:16:51<782:43:47, 22.19s/it]
 44%|████▎     | 98015/225000 [185:17:05<692:23:21, 19.63s/it]
 44%|████▎     | 98016/225000 [185:17:29<740:03:57, 20.98s/it]
 44%|████▎     | 98017/225000 [185:17:47<708:07:46, 20.08s/it]
 44%|████▎     | 98018/225000 [185:18:08<716:49:27, 20.32s/it]
 44%|████▎     | 98019/225000 [185:18:28<713:12:58, 20.22s/it]
 44%|████▎     | 98020/225000 [185:18:50<729:19:33, 20.68s/it]
 44%|████▎     | 98021/225000 [185:19:02<643:30:29, 18.24s/it]
 44%|████▎     | 98022/225000 [185:19:21<653:29:17, 18.53s/it]
 44%|████▎     | 98023/225000 [185:19:46<720:45:14, 20.43s/it]
 44%|████▎     | 98024/225000 [185:20:08<732:55:46, 20.78s/it]
 44%|████▎     | 98025/225000 [185:20:23<668:57:39, 18.97s/it]
 44%|████▎     | 98026/225000 [185:20:53<791:25:03, 22.44s/it]
 44%|████▎     | 98027/225000 [185:21:10<738:27:18, 20.94s/it]
 44%|████▎     | 98028/225000 [185:21:29<714:43:43, 20.26s/it]
 44%|████▎     | 98029/225000 [185:21:55<774:29:41, 21.96s/it]
 44%|████▎     | 98030/225000 [185:22:24<843:31:48, 23.92s/it]
 44%|████▎     | 98031/225000 [185:22:52<889:24:20, 25.22s/it]
 44%|████▎     | 98032/225000 [185:23:16<880:47:23, 24.97s/it]
 44%|████▎     | 98033/225000 [185:23:37<838:15:14, 23.77s/it]
 44%|████▎     | 98034/225000 [185:23:54<763:54:40, 21.66s/it]
 44%|████▎     | 98035/225000 [185:24:05<651:58:25, 18.49s/it]
 44%|████▎     | 98036/225000 [185:24:34<765:37:43, 21.71s/it]
 44%|████▎     | 98037/225000 [185:24:58<791:36:46, 22.45s/it]
 44%|████▎     | 98038/225000 [185:25:21<794:06:21, 22.52s/it]
 44%|████▎     | 98039/225000 [185:25:38<739:00:21, 20.95s/it]
 44%|████▎     | 98040/225000 [185:26:04<786:14:28, 22.29s/it]
 44%|████▎     | 98041/225000 [185:26:29<813:41:20, 23.07s/it]
 44%|████▎     | 98042/225000 [185:26:54<840:11:30, 23.82s/it]
 44%|████▎     | 98043/225000 [185:27:13<791:09:10, 22.43s/it]
 44%|████▎     | 98044/225000 [185:27:41<845:19:04, 23.97s/it]
 44%|████▎     | 98045/225000 [185:28:07<868:40:43, 24.63s/it]
 44%|████▎     | 98046/225000 [185:28:31<862:55:12, 24.47s/it]
 44%|████▎     | 98047/225000 [185:28:53<832:29:56, 23.61s/it]
 44%|████▎     | 98048/225000 [185:29:18<851:33:27, 24.15s/it]
 44%|████▎     | 98049/225000 [185:29:33<747:11:36, 21.19s/it]
 44%|████▎     | 98050/225000 [185:29:50<707:18:14, 20.06s/it]
                                                              
{'loss': '0.909', 'grad_norm': '1.169', 'learning_rate': '3.203e-05', 'epoch': '0.4358', 'num_input_tokens_seen': 306156320, 'train_runtime': '6.678e+05', 'train_tokens_per_second': '458.5'}

 44%|████▎     | 98050/225000 [185:29:50<707:18:14, 20.06s/it]
 44%|████▎     | 98051/225000 [185:30:16<771:58:21, 21.89s/it]
 44%|████▎     | 98052/225000 [185:30:35<742:20:46, 21.05s/it]
 44%|████▎     | 98053/225000 [185:30:58<760:28:35, 21.57s/it]
 44%|████▎     | 98054/225000 [185:31:12<683:32:11, 19.38s/it]
 44%|████▎     | 98055/225000 [185:31:39<765:33:01, 21.71s/it]
 44%|████▎     | 98056/225000 [185:31:56<706:36:13, 20.04s/it]
 44%|████▎     | 98057/225000 [185:32:14<688:41:35, 19.53s/it]
 44%|████▎     | 98058/225000 [185:32:44<795:06:31, 22.55s/it]
 44%|████▎     | 98059/225000 [185:33:00<730:56:16, 20.73s/it]
 44%|████▎     | 98060/225000 [185:33:18<702:00:46, 19.91s/it]
 44%|████▎     | 98061/225000 [185:33:45<772:26:12, 21.91s/it]
 44%|████▎     | 98062/225000 [185:34:05<760:07:12, 21.56s/it]
 44%|████▎     | 98063/225000 [185:34:22<709:06:00, 20.11s/it]
 44%|████▎     | 98064/225000 [185:34:43<716:08:29, 20.31s/it]
 44%|████▎     | 98065/225000 [185:35:03<713:02:26, 20.22s/it]
 44%|████▎     | 98066/225000 [185:35:34<828:28:09, 23.50s/it]
 44%|████▎     | 98067/225000 [185:35:54<795:18:57, 22.56s/it]
 44%|████▎     | 98068/225000 [185:36:10<719:20:37, 20.40s/it]
 44%|████▎     | 98069/225000 [185:36:31<730:38:31, 20.72s/it]
 44%|████▎     | 98070/225000 [185:37:00<812:02:51, 23.03s/it]
 44%|████▎     | 98071/225000 [185:37:21<797:39:48, 22.62s/it]
 44%|████▎     | 98072/225000 [185:37:38<737:31:33, 20.92s/it]
 44%|████▎     | 98073/225000 [185:38:02<771:16:03, 21.88s/it]
 44%|████▎     | 98074/225000 [185:38:25<780:11:35, 22.13s/it]
 44%|████▎     | 98075/225000 [185:38:48<784:00:02, 22.24s/it]
 44%|████▎     | 98076/225000 [185:39:13<813:56:38, 23.09s/it]
 44%|████▎     | 98077/225000 [185:39:41<873:25:52, 24.77s/it]
 44%|████▎     | 98078/225000 [185:40:04<853:26:42, 24.21s/it]
 44%|████▎     | 98079/225000 [185:40:29<862:55:40, 24.48s/it]
 44%|████▎     | 98080/225000 [185:40:48<803:18:54, 22.79s/it]
 44%|████▎     | 98081/225000 [185:41:08<774:24:56, 21.97s/it]
 44%|████▎     | 98082/225000 [185:41:29<761:29:20, 21.60s/it]
 44%|████▎     | 98083/225000 [185:41:55<806:18:30, 22.87s/it]
 44%|████▎     | 98084/225000 [185:42:13<756:00:10, 21.44s/it]
 44%|████▎     | 98085/225000 [185:42:32<728:23:17, 20.66s/it]
 44%|████▎     | 98086/225000 [185:42:57<774:36:00, 21.97s/it]
 44%|████▎     | 98087/225000 [185:43:19<781:41:26, 22.17s/it]
 44%|████▎     | 98088/225000 [185:43:45<815:34:57, 23.13s/it]
 44%|████▎     | 98089/225000 [185:44:08<813:47:22, 23.08s/it]
 44%|████▎     | 98090/225000 [185:44:25<756:57:30, 21.47s/it]
 44%|████▎     | 98091/225000 [185:44:41<693:01:58, 19.66s/it]
 44%|████▎     | 98092/225000 [185:44:55<629:45:25, 17.86s/it]
 44%|████▎     | 98093/225000 [185:45:17<678:53:01, 19.26s/it]
 44%|████▎     | 98094/225000 [185:45:49<813:21:26, 23.07s/it]
 44%|████▎     | 98095/225000 [185:46:20<892:01:54, 25.30s/it]
 44%|████▎     | 98096/225000 [185:46:39<831:54:23, 23.60s/it]
 44%|████▎     | 98097/225000 [185:47:09<893:13:52, 25.34s/it]
 44%|████▎     | 98098/225000 [185:47:26<810:56:10, 23.00s/it]
 44%|████▎     | 98099/225000 [185:47:51<833:43:36, 23.65s/it]
 44%|████▎     | 98100/225000 [185:48:19<874:27:24, 24.81s/it]
                                                              
{'loss': '0.913', 'grad_norm': '1.231', 'learning_rate': '3.201e-05', 'epoch': '0.436', 'num_input_tokens_seen': 306710832, 'train_runtime': '6.689e+05', 'train_tokens_per_second': '458.5'}

 44%|████▎     | 98100/225000 [185:48:19<874:27:24, 24.81s/it]
 44%|████▎     | 98101/225000 [185:48:38<817:11:13, 23.18s/it]
 44%|████▎     | 98102/225000 [185:49:01<813:23:59, 23.08s/it]
 44%|████▎     | 98103/225000 [185:49:25<827:30:53, 23.48s/it]
 44%|████▎     | 98104/225000 [185:49:57<909:01:44, 25.79s/it]
 44%|████▎     | 98105/225000 [185:50:16<845:24:52, 23.98s/it]
 44%|████▎     | 98106/225000 [185:50:42<860:53:27, 24.42s/it]
 44%|████▎     | 98107/225000 [185:51:03<830:31:40, 23.56s/it]
 44%|████▎     | 98108/225000 [185:51:29<850:29:24, 24.13s/it]
 44%|████▎     | 98109/225000 [185:51:52<839:04:39, 23.81s/it]
 44%|████▎     | 98110/225000 [185:52:06<739:55:44, 20.99s/it]
 44%|████▎     | 98111/225000 [185:52:22<686:22:31, 19.47s/it]
 44%|████▎     | 98112/225000 [185:52:45<721:40:53, 20.48s/it]
 44%|████▎     | 98113/225000 [185:53:01<676:20:03, 19.19s/it]
 44%|████▎     | 98114/225000 [185:53:34<824:00:49, 23.38s/it]
 44%|████▎     | 98115/225000 [185:53:53<769:19:30, 21.83s/it]
 44%|████▎     | 98116/225000 [185:54:11<736:48:54, 20.91s/it]
 44%|████▎     | 98117/225000 [185:54:30<714:26:37, 20.27s/it]
 44%|████▎     | 98118/225000 [185:55:01<826:10:47, 23.44s/it]
 44%|████▎     | 98119/225000 [185:55:23<805:49:12, 22.86s/it]
 44%|████▎     | 98120/225000 [185:55:49<842:51:47, 23.91s/it]
 44%|████▎     | 98121/225000 [185:56:08<793:14:06, 22.51s/it]
 44%|████▎     | 98122/225000 [185:56:29<780:01:55, 22.13s/it]
 44%|████▎     | 98123/225000 [185:56:56<823:47:18, 23.37s/it]
 44%|████▎     | 98124/225000 [185:57:20<836:17:31, 23.73s/it]
 44%|████▎     | 98125/225000 [185:57:40<796:28:10, 22.60s/it]
 44%|████▎     | 98126/225000 [185:58:03<796:22:54, 22.60s/it]
 44%|████▎     | 98127/225000 [185:58:24<781:54:59, 22.19s/it]
 44%|████▎     | 98128/225000 [185:58:53<852:30:10, 24.19s/it]
 44%|████▎     | 98129/225000 [185:59:18<861:01:22, 24.43s/it]
 44%|████▎     | 98130/225000 [185:59:34<775:10:56, 22.00s/it]
 44%|████▎     | 98131/225000 [185:59:57<783:59:18, 22.25s/it]
 44%|████▎     | 98132/225000 [186:00:25<847:25:16, 24.05s/it]
 44%|████▎     | 98133/225000 [186:00:39<738:16:04, 20.95s/it]
 44%|████▎     | 98134/225000 [186:00:54<677:27:27, 19.22s/it]
 44%|████▎     | 98135/225000 [186:01:19<738:55:27, 20.97s/it]
 44%|████▎     | 98136/225000 [186:01:40<734:22:05, 20.84s/it]
 44%|████▎     | 98137/225000 [186:01:58<704:55:53, 20.00s/it]
 44%|████▎     | 98138/225000 [186:02:21<737:06:15, 20.92s/it]
 44%|████▎     | 98139/225000 [186:02:44<764:47:22, 21.70s/it]
 44%|████▎     | 98140/225000 [186:03:01<711:44:28, 20.20s/it]
 44%|████▎     | 98141/225000 [186:03:32<824:54:56, 23.41s/it]
 44%|████▎     | 98142/225000 [186:03:47<739:32:39, 20.99s/it]
 44%|████▎     | 98143/225000 [186:04:12<781:00:04, 22.16s/it]
 44%|████▎     | 98144/225000 [186:04:40<838:23:22, 23.79s/it]
 44%|████▎     | 98145/225000 [186:05:09<898:46:58, 25.51s/it]
 44%|████▎     | 98146/225000 [186:05:32<871:37:44, 24.74s/it]
 44%|████▎     | 98147/225000 [186:05:50<798:44:30, 22.67s/it]
 44%|████▎     | 98148/225000 [186:06:16<837:02:23, 23.75s/it]
 44%|████▎     | 98149/225000 [186:06:37<799:37:34, 22.69s/it]
 44%|████▎     | 98150/225000 [186:07:00<810:48:46, 23.01s/it]
                                                              
{'loss': '0.8846', 'grad_norm': '1.106', 'learning_rate': '3.199e-05', 'epoch': '0.4362', 'num_input_tokens_seen': 307283120, 'train_runtime': '6.7e+05', 'train_tokens_per_second': '458.6'}

 44%|████▎     | 98150/225000 [186:07:00<810:48:46, 23.01s/it]
 44%|████▎     | 98151/225000 [186:07:23<805:29:47, 22.86s/it]
 44%|████▎     | 98152/225000 [186:07:31<652:07:56, 18.51s/it]
 44%|████▎     | 98153/225000 [186:07:51<666:15:47, 18.91s/it]
 44%|████▎     | 98154/225000 [186:08:17<743:06:31, 21.09s/it]
 44%|████▎     | 98155/225000 [186:08:43<794:37:11, 22.55s/it]
 44%|████▎     | 98156/225000 [186:09:05<787:58:17, 22.36s/it]
 44%|████▎     | 98157/225000 [186:09:31<828:46:53, 23.52s/it]
 44%|████▎     | 98158/225000 [186:09:51<790:08:12, 22.43s/it]
 44%|████▎     | 98159/225000 [186:10:10<752:27:32, 21.36s/it]
 44%|████▎     | 98160/225000 [186:10:39<827:55:02, 23.50s/it]
 44%|████▎     | 98161/225000 [186:10:55<751:39:31, 21.33s/it]
 44%|████▎     | 98162/225000 [186:11:14<728:59:12, 20.69s/it]
 44%|████▎     | 98163/225000 [186:11:36<740:05:45, 21.01s/it]
 44%|████▎     | 98164/225000 [186:12:03<807:54:46, 22.93s/it]
 44%|████▎     | 98165/225000 [186:12:31<858:23:51, 24.36s/it]
 44%|████▎     | 98166/225000 [186:12:51<816:55:17, 23.19s/it]
 44%|████▎     | 98167/225000 [186:13:12<788:41:38, 22.39s/it]
 44%|████▎     | 98168/225000 [186:13:42<875:04:55, 24.84s/it]
 44%|████▎     | 98169/225000 [186:14:06<863:56:55, 24.52s/it]
 44%|████▎     | 98170/225000 [186:14:28<833:42:55, 23.66s/it]
 44%|████▎     | 98171/225000 [186:14:42<735:12:44, 20.87s/it]
 44%|████▎     | 98172/225000 [186:15:07<772:38:47, 21.93s/it]
 44%|████▎     | 98173/225000 [186:15:33<818:58:46, 23.25s/it]
 44%|████▎     | 98174/225000 [186:15:55<806:03:06, 22.88s/it]
 44%|████▎     | 98175/225000 [186:16:22<849:02:12, 24.10s/it]
 44%|████▎     | 98176/225000 [186:16:46<848:14:40, 24.08s/it]
 44%|████▎     | 98177/225000 [186:17:05<795:17:14, 22.58s/it]
 44%|████▎     | 98178/225000 [186:17:28<802:07:37, 22.77s/it]
 44%|████▎     | 98179/225000 [186:17:49<782:09:20, 22.20s/it]
 44%|████▎     | 98180/225000 [186:18:09<760:58:20, 21.60s/it]
 44%|████▎     | 98181/225000 [186:18:30<749:00:45, 21.26s/it]
 44%|████▎     | 98182/225000 [186:18:54<776:06:58, 22.03s/it]
 44%|████▎     | 98183/225000 [186:19:15<773:22:35, 21.95s/it]
 44%|████▎     | 98184/225000 [186:19:40<804:46:38, 22.85s/it]
 44%|████▎     | 98185/225000 [186:20:03<801:16:30, 22.75s/it]
 44%|████▎     | 98186/225000 [186:20:27<811:33:38, 23.04s/it]
 44%|████▎     | 98187/225000 [186:20:38<694:13:22, 19.71s/it]
 44%|████▎     | 98188/225000 [186:21:06<775:14:18, 22.01s/it]
 44%|████▎     | 98189/225000 [186:21:28<777:35:19, 22.07s/it]
 44%|████▎     | 98190/225000 [186:21:53<803:05:17, 22.80s/it]
 44%|████▎     | 98191/225000 [186:22:15<797:50:38, 22.65s/it]
 44%|████▎     | 98192/225000 [186:22:41<832:18:01, 23.63s/it]
 44%|████▎     | 98193/225000 [186:23:03<816:42:25, 23.19s/it]
 44%|████▎     | 98194/225000 [186:23:22<770:14:38, 21.87s/it]
 44%|████▎     | 98195/225000 [186:23:39<723:40:37, 20.55s/it]
 44%|████▎     | 98196/225000 [186:23:58<702:31:39, 19.94s/it]
 44%|████▎     | 98197/225000 [186:24:27<795:48:50, 22.59s/it]
 44%|████▎     | 98198/225000 [186:24:49<794:09:39, 22.55s/it]
 44%|████▎     | 98199/225000 [186:25:17<853:42:15, 24.24s/it]
 44%|████▎     | 98200/225000 [186:25:36<794:08:40, 22.55s/it]
                                                              
{'loss': '0.903', 'grad_norm': '1.366', 'learning_rate': '3.197e-05', 'epoch': '0.4364', 'num_input_tokens_seen': 307851264, 'train_runtime': '6.711e+05', 'train_tokens_per_second': '458.7'}

 44%|████▎     | 98200/225000 [186:25:36<794:08:40, 22.55s/it]
 44%|████▎     | 98201/225000 [186:26:01<822:43:50, 23.36s/it]
 44%|████▎     | 98202/225000 [186:26:13<703:42:26, 19.98s/it]
 44%|████▎     | 98203/225000 [186:26:37<746:12:52, 21.19s/it]
 44%|████▎     | 98204/225000 [186:26:54<698:40:26, 19.84s/it]
 44%|████▎     | 98205/225000 [186:27:08<636:52:42, 18.08s/it]
 44%|████▎     | 98206/225000 [186:27:29<668:42:29, 18.99s/it]
 44%|████▎     | 98207/225000 [186:27:52<710:04:41, 20.16s/it]
 44%|████▎     | 98208/225000 [186:28:18<772:31:48, 21.93s/it]
 44%|████▎     | 98209/225000 [186:28:45<830:03:33, 23.57s/it]
 44%|████▎     | 98210/225000 [186:29:15<891:53:13, 25.32s/it]
 44%|████▎     | 98211/225000 [186:29:32<804:39:48, 22.85s/it]
 44%|████▎     | 98212/225000 [186:30:02<879:50:12, 24.98s/it]
 44%|████▎     | 98213/225000 [186:30:23<842:14:09, 23.91s/it]
 44%|████▎     | 98214/225000 [186:30:49<858:03:45, 24.36s/it]
 44%|████▎     | 98215/225000 [186:31:17<899:17:52, 25.54s/it]
 44%|████▎     | 98216/225000 [186:31:35<819:08:31, 23.26s/it]
 44%|████▎     | 98217/225000 [186:32:00<840:01:23, 23.85s/it]
 44%|████▎     | 98218/225000 [186:32:22<819:01:33, 23.26s/it]
 44%|████▎     | 98219/225000 [186:32:48<852:51:22, 24.22s/it]
 44%|████▎     | 98220/225000 [186:33:13<857:37:57, 24.35s/it]
 44%|████▎     | 98221/225000 [186:33:35<834:09:57, 23.69s/it]
 44%|████▎     | 98222/225000 [186:34:02<866:59:26, 24.62s/it]
 44%|████▎     | 98223/225000 [186:34:26<858:55:11, 24.39s/it]
 44%|████▎     | 98224/225000 [186:34:46<819:12:26, 23.26s/it]
 44%|████▎     | 98225/225000 [186:35:12<841:38:59, 23.90s/it]
 44%|████▎     | 98226/225000 [186:35:33<813:41:35, 23.11s/it]
 44%|████▎     | 98227/225000 [186:35:50<744:00:29, 21.13s/it]
 44%|████▎     | 98228/225000 [186:36:09<728:54:41, 20.70s/it]
 44%|████▎     | 98229/225000 [186:36:38<813:18:59, 23.10s/it]
 44%|████▎     | 98230/225000 [186:37:03<835:14:58, 23.72s/it]
 44%|████▎     | 98231/225000 [186:37:28<843:27:37, 23.95s/it]
 44%|████▎     | 98232/225000 [186:37:47<797:42:06, 22.65s/it]
 44%|████▎     | 98233/225000 [186:38:09<784:09:18, 22.27s/it]
 44%|████▎     | 98234/225000 [186:38:29<762:51:51, 21.66s/it]
 44%|████▎     | 98235/225000 [186:38:42<674:26:49, 19.15s/it]
 44%|████▎     | 98236/225000 [186:38:57<634:07:31, 18.01s/it]
 44%|████▎     | 98237/225000 [186:39:26<745:06:59, 21.16s/it]
 44%|████▎     | 98238/225000 [186:39:50<779:22:37, 22.13s/it]
 44%|████▎     | 98239/225000 [186:40:09<739:16:51, 21.00s/it]
 44%|████▎     | 98240/225000 [186:40:32<764:13:05, 21.70s/it]
 44%|████▎     | 98241/225000 [186:41:05<884:30:43, 25.12s/it]
 44%|████▎     | 98242/225000 [186:41:27<847:53:34, 24.08s/it]
 44%|████▎     | 98243/225000 [186:41:57<913:05:13, 25.93s/it]
 44%|████▎     | 98244/225000 [186:42:16<835:58:38, 23.74s/it]
 44%|████▎     | 98245/225000 [186:42:33<768:16:36, 21.82s/it]
 44%|████▎     | 98246/225000 [186:43:06<883:58:01, 25.11s/it]
 44%|████▎     | 98247/225000 [186:43:29<865:34:18, 24.58s/it]
 44%|████▎     | 98248/225000 [186:43:43<751:04:18, 21.33s/it]
 44%|████▎     | 98249/225000 [186:43:58<686:14:46, 19.49s/it]
 44%|████▎     | 98250/225000 [186:44:26<770:26:33, 21.88s/it]
                                                              
{'loss': '0.8844', 'grad_norm': '1.104', 'learning_rate': '3.196e-05', 'epoch': '0.4367', 'num_input_tokens_seen': 308441904, 'train_runtime': '6.723e+05', 'train_tokens_per_second': '458.8'}

 44%|████▎     | 98250/225000 [186:44:26<770:26:33, 21.88s/it]
 44%|████▎     | 98251/225000 [186:44:47<761:15:30, 21.62s/it]
 44%|████▎     | 98252/225000 [186:45:05<731:43:27, 20.78s/it]
 44%|████▎     | 98253/225000 [186:45:32<790:03:31, 22.44s/it]
 44%|████▎     | 98254/225000 [186:45:52<769:35:29, 21.86s/it]
 44%|████▎     | 98255/225000 [186:46:20<827:34:52, 23.51s/it]
 44%|████▎     | 98256/225000 [186:46:44<838:57:15, 23.83s/it]
 44%|████▎     | 98257/225000 [186:47:10<863:53:22, 24.54s/it]
 44%|████▎     | 98258/225000 [186:47:31<824:17:38, 23.41s/it]
 44%|████▎     | 98259/225000 [186:47:56<840:28:09, 23.87s/it]
 44%|████▎     | 98260/225000 [186:48:23<869:08:25, 24.69s/it]
 44%|████▎     | 98261/225000 [186:48:47<863:47:51, 24.54s/it]
 44%|████▎     | 98262/225000 [186:49:00<745:29:36, 21.18s/it]
 44%|████▎     | 98263/225000 [186:49:14<664:03:36, 18.86s/it]
 44%|████▎     | 98264/225000 [186:49:38<719:40:31, 20.44s/it]
 44%|████▎     | 98265/225000 [186:49:51<644:24:58, 18.31s/it]
 44%|████▎     | 98266/225000 [186:50:21<764:30:04, 21.72s/it]
 44%|████▎     | 98267/225000 [186:50:40<741:28:25, 21.06s/it]
 44%|████▎     | 98268/225000 [186:50:58<703:15:32, 19.98s/it]
 44%|████▎     | 98269/225000 [186:51:15<676:19:27, 19.21s/it]
 44%|████▎     | 98270/225000 [186:51:36<690:15:39, 19.61s/it]
 44%|████▎     | 98271/225000 [186:52:02<759:39:29, 21.58s/it]
 44%|████▎     | 98272/225000 [186:52:22<745:46:19, 21.19s/it]
 44%|████▎     | 98273/225000 [186:52:54<858:34:56, 24.39s/it]
 44%|████▎     | 98274/225000 [186:53:15<824:19:18, 23.42s/it]
 44%|████▎     | 98275/225000 [186:53:31<747:31:55, 21.24s/it]
 44%|████▎     | 98276/225000 [186:53:51<733:16:40, 20.83s/it]
 44%|████▎     | 98277/225000 [186:54:20<816:16:25, 23.19s/it]
 44%|████▎     | 98278/225000 [186:54:42<806:46:37, 22.92s/it]
 44%|████▎     | 98279/225000 [186:55:15<915:49:25, 26.02s/it]
 44%|████▎     | 98280/225000 [186:55:34<836:10:09, 23.75s/it]
 44%|████▎     | 98281/225000 [186:55:54<799:56:48, 22.73s/it]
 44%|████▎     | 98282/225000 [186:56:14<770:35:02, 21.89s/it]
 44%|████▎     | 98283/225000 [186:56:30<709:19:44, 20.15s/it]
 44%|████▎     | 98284/225000 [186:56:59<800:00:02, 22.73s/it]
 44%|████▎     | 98285/225000 [186:57:20<780:42:11, 22.18s/it]
 44%|████▎     | 98286/225000 [186:57:39<746:41:59, 21.21s/it]
 44%|████▎     | 98287/225000 [186:57:59<737:18:59, 20.95s/it]
 44%|████▎     | 98288/225000 [186:58:24<776:44:45, 22.07s/it]
 44%|████▎     | 98289/225000 [186:58:40<717:39:43, 20.39s/it]
 44%|████▎     | 98290/225000 [186:58:58<688:05:13, 19.55s/it]
 44%|████▎     | 98291/225000 [186:59:23<745:20:38, 21.18s/it]
 44%|████▎     | 98292/225000 [186:59:47<776:22:35, 22.06s/it]
 44%|████▎     | 98293/225000 [187:00:13<816:01:38, 23.18s/it]
 44%|████▎     | 98294/225000 [187:00:41<873:28:16, 24.82s/it]
 44%|████▎     | 98295/225000 [187:01:08<889:13:50, 25.27s/it]
 44%|████▎     | 98296/225000 [187:01:22<773:29:05, 21.98s/it]
 44%|████▎     | 98297/225000 [187:01:54<876:51:42, 24.91s/it]
 44%|████▎     | 98298/225000 [187:02:19<877:10:41, 24.92s/it]
 44%|████▎     | 98299/225000 [187:02:37<807:24:10, 22.94s/it]
 44%|████▎     | 98300/225000 [187:03:01<815:24:35, 23.17s/it]
                                                              
{'loss': '0.9052', 'grad_norm': '1.206', 'learning_rate': '3.194e-05', 'epoch': '0.4369', 'num_input_tokens_seen': 309015640, 'train_runtime': '6.734e+05', 'train_tokens_per_second': '458.9'}

 44%|████▎     | 98300/225000 [187:03:01<815:24:35, 23.17s/it]
 44%|████▎     | 98301/225000 [187:03:23<809:28:10, 23.00s/it]
 44%|████▎     | 98302/225000 [187:03:42<757:44:50, 21.53s/it]
 44%|████▎     | 98303/225000 [187:04:03<752:43:44, 21.39s/it]
 44%|████▎     | 98304/225000 [187:04:23<740:13:31, 21.03s/it]
 44%|████▎     | 98305/225000 [187:04:39<692:08:08, 19.67s/it]
 44%|████▎     | 98306/225000 [187:05:04<748:46:53, 21.28s/it]
 44%|████▎     | 98307/225000 [187:05:25<743:52:13, 21.14s/it]
 44%|████▎     | 98308/225000 [187:05:47<756:37:26, 21.50s/it]
 44%|████▎     | 98309/225000 [187:06:13<795:14:59, 22.60s/it]
 44%|████▎     | 98310/225000 [187:06:41<853:37:30, 24.26s/it]
 44%|████▎     | 98311/225000 [187:06:57<771:58:55, 21.94s/it]
 44%|████▎     | 98312/225000 [187:07:18<760:16:04, 21.60s/it]
 44%|████▎     | 98313/225000 [187:07:31<671:00:24, 19.07s/it]
 44%|████▎     | 98314/225000 [187:07:53<694:19:32, 19.73s/it]
 44%|████▎     | 98315/225000 [187:08:16<735:26:44, 20.90s/it]
 44%|████▎     | 98316/225000 [187:08:34<706:18:16, 20.07s/it]
 44%|████▎     | 98317/225000 [187:08:56<720:08:54, 20.46s/it]
 44%|████▎     | 98318/225000 [187:09:17<725:27:20, 20.62s/it]
 44%|████▎     | 98319/225000 [187:09:38<729:47:48, 20.74s/it]
 44%|████▎     | 98320/225000 [187:09:57<718:21:35, 20.41s/it]
 44%|████▎     | 98321/225000 [187:10:22<763:14:15, 21.69s/it]
 44%|████▎     | 98322/225000 [187:10:46<787:00:34, 22.37s/it]
 44%|████▎     | 98323/225000 [187:11:09<795:45:34, 22.61s/it]
 44%|████▎     | 98324/225000 [187:11:23<707:29:51, 20.11s/it]
 44%|████▎     | 98325/225000 [187:11:49<760:47:13, 21.62s/it]
 44%|████▎     | 98326/225000 [187:12:14<798:37:16, 22.70s/it]
 44%|████▎     | 98327/225000 [187:12:35<784:02:37, 22.28s/it]
 44%|████▎     | 98328/225000 [187:12:55<756:17:50, 21.49s/it]
 44%|████▎     | 98329/225000 [187:13:18<774:46:50, 22.02s/it]
 44%|████▎     | 98330/225000 [187:13:41<783:12:45, 22.26s/it]
 44%|████▎     | 98331/225000 [187:13:55<697:43:08, 19.83s/it]
 44%|████▎     | 98332/225000 [187:14:09<640:57:16, 18.22s/it]
 44%|████▎     | 98333/225000 [187:14:28<639:43:35, 18.18s/it]
 44%|████▎     | 98334/225000 [187:14:58<768:35:49, 21.84s/it]
 44%|████▎     | 98335/225000 [187:15:17<741:43:40, 21.08s/it]
 44%|████▎     | 98336/225000 [187:15:44<803:16:49, 22.83s/it]
 44%|████▎     | 98337/225000 [187:16:08<818:53:07, 23.27s/it]
 44%|████▎     | 98338/225000 [187:16:32<822:31:32, 23.38s/it]
 44%|████▎     | 98339/225000 [187:16:55<819:44:40, 23.30s/it]
 44%|████▎     | 98340/225000 [187:17:19<823:39:17, 23.41s/it]
 44%|████▎     | 98341/225000 [187:17:43<834:50:55, 23.73s/it]
 44%|████▎     | 98342/225000 [187:18:03<797:31:32, 22.67s/it]
 44%|████▎     | 98343/225000 [187:18:32<855:04:37, 24.30s/it]
 44%|████▎     | 98344/225000 [187:19:02<924:35:28, 26.28s/it]
 44%|████▎     | 98345/225000 [187:19:27<901:26:00, 25.62s/it]
 44%|████▎     | 98346/225000 [187:19:54<920:21:33, 26.16s/it]
 44%|████▎     | 98347/225000 [187:20:16<880:53:20, 25.04s/it]
 44%|████▎     | 98348/225000 [187:20:39<860:02:18, 24.45s/it]
 44%|████▎     | 98349/225000 [187:20:54<757:08:40, 21.52s/it]
 44%|████▎     | 98350/225000 [187:21:17<776:05:30, 22.06s/it]
                                                              
{'loss': '0.9014', 'grad_norm': '1.214', 'learning_rate': '3.192e-05', 'epoch': '0.4371', 'num_input_tokens_seen': 309604120, 'train_runtime': '6.745e+05', 'train_tokens_per_second': '459'}

 44%|████▎     | 98350/225000 [187:21:18<776:05:30, 22.06s/it]
 44%|████▎     | 98351/225000 [187:21:39<767:22:04, 21.81s/it]
 44%|████▎     | 98352/225000 [187:22:01<769:23:57, 21.87s/it]
 44%|████▎     | 98353/225000 [187:22:25<794:47:28, 22.59s/it]
 44%|████▎     | 98354/225000 [187:22:47<784:57:26, 22.31s/it]
 44%|████▎     | 98355/225000 [187:23:12<814:22:31, 23.15s/it]
 44%|████▎     | 98356/225000 [187:23:42<887:23:18, 25.23s/it]
 44%|████▎     | 98357/225000 [187:24:06<875:07:11, 24.88s/it]
 44%|████▎     | 98358/225000 [187:24:29<854:23:03, 24.29s/it]
 44%|████▎     | 98359/225000 [187:24:42<740:12:53, 21.04s/it]
 44%|████▎     | 98360/225000 [187:25:06<764:24:40, 21.73s/it]
 44%|████▎     | 98361/225000 [187:25:29<782:36:22, 22.25s/it]
 44%|████▎     | 98362/225000 [187:25:40<659:49:29, 18.76s/it]
 44%|████▎     | 98363/225000 [187:26:00<676:48:48, 19.24s/it]
 44%|████▎     | 98364/225000 [187:26:16<640:54:26, 18.22s/it]
 44%|████▎     | 98365/225000 [187:26:47<780:57:04, 22.20s/it]
 44%|████▎     | 98366/225000 [187:27:13<815:42:29, 23.19s/it]
 44%|████▎     | 98367/225000 [187:27:35<804:28:24, 22.87s/it]
 44%|████▎     | 98368/225000 [187:27:52<742:57:32, 21.12s/it]
 44%|████▎     | 98369/225000 [187:28:22<835:33:27, 23.75s/it]
 44%|████▎     | 98370/225000 [187:28:48<862:04:55, 24.51s/it]
 44%|████▎     | 98371/225000 [187:29:08<817:06:42, 23.23s/it]
 44%|████▎     | 98372/225000 [187:29:23<720:34:26, 20.49s/it]
 44%|████▎     | 98373/225000 [187:29:54<835:45:52, 23.76s/it]
 44%|████▎     | 98374/225000 [187:30:16<817:49:08, 23.25s/it]
 44%|████▎     | 98375/225000 [187:30:43<859:58:41, 24.45s/it]
 44%|████▎     | 98376/225000 [187:31:06<847:03:25, 24.08s/it]
 44%|████▎     | 98377/225000 [187:31:30<840:20:39, 23.89s/it]
 44%|████▎     | 98378/225000 [187:31:47<769:21:10, 21.87s/it]
 44%|████▎     | 98379/225000 [187:32:07<752:10:01, 21.39s/it]
 44%|████▎     | 98380/225000 [187:32:32<783:47:57, 22.28s/it]
 44%|████▎     | 98381/225000 [187:32:51<747:32:05, 21.25s/it]
 44%|████▎     | 98382/225000 [187:33:15<779:50:02, 22.17s/it]
 44%|████▎     | 98383/225000 [187:33:37<774:28:02, 22.02s/it]
 44%|████▎     | 98384/225000 [187:33:56<751:23:10, 21.36s/it]
 44%|████▎     | 98385/225000 [187:34:26<841:09:52, 23.92s/it]
 44%|████▎     | 98386/225000 [187:34:44<772:36:22, 21.97s/it]
 44%|████▎     | 98387/225000 [187:35:04<751:20:34, 21.36s/it]
 44%|████▎     | 98388/225000 [187:35:34<849:21:42, 24.15s/it]
 44%|████▎     | 98389/225000 [187:35:57<832:49:52, 23.68s/it]
 44%|████▎     | 98390/225000 [187:36:17<793:25:30, 22.56s/it]
 44%|████▎     | 98391/225000 [187:36:48<880:15:00, 25.03s/it]
 44%|████▎     | 98392/225000 [187:37:12<869:53:00, 24.73s/it]
 44%|████▎     | 98393/225000 [187:37:36<868:52:35, 24.71s/it]
 44%|████▎     | 98394/225000 [187:37:59<850:19:33, 24.18s/it]
 44%|████▎     | 98395/225000 [187:38:24<855:41:04, 24.33s/it]
 44%|████▎     | 98396/225000 [187:38:42<791:39:36, 22.51s/it]
 44%|████▎     | 98397/225000 [187:39:02<759:42:45, 21.60s/it]
 44%|████▎     | 98398/225000 [187:39:23<758:27:15, 21.57s/it]
 44%|████▎     | 98399/225000 [187:39:40<708:19:46, 20.14s/it]
 44%|████▎     | 98400/225000 [187:40:00<706:34:58, 20.09s/it]
                                                              
{'loss': '0.8771', 'grad_norm': '1.224', 'learning_rate': '3.19e-05', 'epoch': '0.4373', 'num_input_tokens_seen': 310178520, 'train_runtime': '6.756e+05', 'train_tokens_per_second': '459.1'}

 44%|████▎     | 98400/225000 [187:40:00<706:34:58, 20.09s/it]
 44%|████▎     | 98401/225000 [187:40:20<704:43:51, 20.04s/it]
 44%|████▎     | 98402/225000 [187:40:38<679:42:36, 19.33s/it]
 44%|████▎     | 98403/225000 [187:41:01<720:50:04, 20.50s/it]
 44%|████▎     | 98404/225000 [187:41:20<711:45:34, 20.24s/it]
 44%|████▎     | 98405/225000 [187:41:48<786:25:29, 22.36s/it]
 44%|████▎     | 98406/225000 [187:42:13<818:32:34, 23.28s/it]
 44%|████▎     | 98407/225000 [187:42:35<799:13:51, 22.73s/it]
 44%|████▎     | 98408/225000 [187:43:01<833:15:01, 23.70s/it]
 44%|████▎     | 98409/225000 [187:43:18<764:04:22, 21.73s/it]
 44%|████▎     | 98410/225000 [187:43:42<793:00:40, 22.55s/it]
 44%|████▎     | 98411/225000 [187:44:06<810:04:03, 23.04s/it]
 44%|████▎     | 98412/225000 [187:44:37<889:13:07, 25.29s/it]
 44%|████▎     | 98413/225000 [187:45:00<867:03:31, 24.66s/it]
 44%|████▎     | 98414/225000 [187:45:21<822:58:34, 23.40s/it]
 44%|████▎     | 98415/225000 [187:45:44<824:43:40, 23.45s/it]
 44%|████▎     | 98416/225000 [187:46:11<862:12:48, 24.52s/it]
 44%|████▎     | 98417/225000 [187:46:33<831:32:47, 23.65s/it]
 44%|████▎     | 98418/225000 [187:46:49<758:21:13, 21.57s/it]
 44%|████▎     | 98419/225000 [187:47:12<771:03:58, 21.93s/it]
 44%|████▎     | 98420/225000 [187:47:31<740:56:12, 21.07s/it]
 44%|████▎     | 98421/225000 [187:47:50<721:24:12, 20.52s/it]
 44%|████▎     | 98422/225000 [187:48:18<793:00:46, 22.55s/it]
 44%|████▎     | 98423/225000 [187:48:43<823:03:06, 23.41s/it]
 44%|████▎     | 98424/225000 [187:49:09<845:00:00, 24.03s/it]
 44%|████▎     | 98425/225000 [187:49:29<809:59:10, 23.04s/it]
 44%|████▎     | 98426/225000 [187:49:55<834:14:25, 23.73s/it]
 44%|████▎     | 98427/225000 [187:50:18<834:04:35, 23.72s/it]
 44%|████▎     | 98428/225000 [187:50:41<824:04:43, 23.44s/it]
 44%|████▎     | 98429/225000 [187:51:01<787:37:09, 22.40s/it]
 44%|████▎     | 98430/225000 [187:51:21<762:21:05, 21.68s/it]
 44%|████▎     | 98431/225000 [187:51:51<847:17:45, 24.10s/it]
 44%|████▎     | 98432/225000 [187:52:11<801:29:17, 22.80s/it]
 44%|████▎     | 98433/225000 [187:52:30<769:08:14, 21.88s/it]
 44%|████▎     | 98434/225000 [187:52:58<825:03:50, 23.47s/it]
 44%|████▎     | 98435/225000 [187:53:10<704:54:09, 20.05s/it]
 44%|████▎     | 98436/225000 [187:53:33<734:10:44, 20.88s/it]
 44%|████▎     | 98437/225000 [187:53:50<697:13:20, 19.83s/it]
 44%|████▍     | 98438/225000 [187:54:10<699:36:11, 19.90s/it]
 44%|████▍     | 98439/225000 [187:54:27<673:55:28, 19.17s/it]
 44%|████▍     | 98440/225000 [187:54:46<667:45:24, 18.99s/it]
 44%|████▍     | 98441/225000 [187:55:03<645:34:14, 18.36s/it]
 44%|████▍     | 98442/225000 [187:55:40<842:49:42, 23.97s/it]
 44%|████▍     | 98443/225000 [187:56:01<816:01:02, 23.21s/it]
 44%|████▍     | 98444/225000 [187:56:27<845:38:35, 24.06s/it]
 44%|████▍     | 98445/225000 [187:56:43<760:41:47, 21.64s/it]
 44%|████▍     | 98446/225000 [187:56:57<678:10:01, 19.29s/it]
 44%|████▍     | 98447/225000 [187:57:12<634:15:10, 18.04s/it]
 44%|████▍     | 98448/225000 [187:57:41<744:43:13, 21.18s/it]
 44%|████▍     | 98449/225000 [187:58:13<862:16:51, 24.53s/it]
 44%|████▍     | 98450/225000 [187:58:33<811:09:44, 23.08s/it]
                                                              
{'loss': '0.9004', 'grad_norm': '0.9122', 'learning_rate': '3.189e-05', 'epoch': '0.4376', 'num_input_tokens_seen': 310751448, 'train_runtime': '6.767e+05', 'train_tokens_per_second': '459.2'}

 44%|████▍     | 98450/225000 [187:58:33<811:09:44, 23.08s/it]
 44%|████▍     | 98451/225000 [187:58:51<760:17:37, 21.63s/it]
 44%|████▍     | 98452/225000 [187:59:10<732:12:16, 20.83s/it]
 44%|████▍     | 98453/225000 [187:59:37<799:43:28, 22.75s/it]
 44%|████▍     | 98454/225000 [188:00:01<808:08:13, 22.99s/it]
 44%|████▍     | 98455/225000 [188:00:28<851:58:09, 24.24s/it]
 44%|████▍     | 98456/225000 [188:00:52<849:34:35, 24.17s/it]
 44%|████▍     | 98457/225000 [188:01:16<851:46:03, 24.23s/it]
 44%|████▍     | 98458/225000 [188:01:45<895:59:53, 25.49s/it]
 44%|████▍     | 98459/225000 [188:02:11<904:34:25, 25.73s/it]
 44%|████▍     | 98460/225000 [188:02:31<845:13:01, 24.05s/it]
 44%|████▍     | 98461/225000 [188:02:52<811:32:15, 23.09s/it]
 44%|████▍     | 98462/225000 [188:03:20<861:35:32, 24.51s/it]
 44%|████▍     | 98463/225000 [188:03:35<765:20:37, 21.77s/it]
 44%|████▍     | 98464/225000 [188:03:55<742:54:07, 21.14s/it]
 44%|████▍     | 98465/225000 [188:04:27<856:15:32, 24.36s/it]
 44%|████▍     | 98466/225000 [188:04:50<839:57:02, 23.90s/it]
 44%|████▍     | 98467/225000 [188:05:11<816:32:40, 23.23s/it]
 44%|████▍     | 98468/225000 [188:05:42<898:21:58, 25.56s/it]
 44%|████▍     | 98469/225000 [188:06:05<871:11:05, 24.79s/it]
 44%|████▍     | 98470/225000 [188:06:29<854:56:22, 24.32s/it]
 44%|████▍     | 98471/225000 [188:06:52<843:10:38, 23.99s/it]
 44%|████▍     | 98472/225000 [188:07:14<822:31:53, 23.40s/it]
 44%|████▍     | 98473/225000 [188:07:38<831:31:02, 23.66s/it]
 44%|████▍     | 98474/225000 [188:07:53<741:16:12, 21.09s/it]
 44%|████▍     | 98475/225000 [188:08:21<812:55:49, 23.13s/it]
 44%|████▍     | 98476/225000 [188:08:51<887:38:46, 25.26s/it]
 44%|████▍     | 98477/225000 [188:09:15<870:00:28, 24.75s/it]
 44%|████▍     | 98478/225000 [188:09:31<779:57:48, 22.19s/it]
 44%|████▍     | 98479/225000 [188:09:52<765:18:04, 21.78s/it]
 44%|████▍     | 98480/225000 [188:10:14<769:40:57, 21.90s/it]
 44%|████▍     | 98481/225000 [188:10:45<868:00:48, 24.70s/it]
 44%|████▍     | 98482/225000 [188:11:01<772:02:34, 21.97s/it]
 44%|████▍     | 98483/225000 [188:11:20<743:44:16, 21.16s/it]
 44%|████▍     | 98484/225000 [188:11:35<680:46:46, 19.37s/it]
 44%|████▍     | 98485/225000 [188:11:56<691:30:45, 19.68s/it]
 44%|████▍     | 98486/225000 [188:12:22<762:03:47, 21.68s/it]
 44%|████▍     | 98487/225000 [188:12:41<731:17:17, 20.81s/it]
 44%|████▍     | 98488/225000 [188:13:03<746:42:27, 21.25s/it]
 44%|████▍     | 98489/225000 [188:13:33<836:00:21, 23.79s/it]
 44%|████▍     | 98490/225000 [188:13:50<768:42:06, 21.87s/it]
 44%|████▍     | 98491/225000 [188:14:15<801:52:05, 22.82s/it]
 44%|████▍     | 98492/225000 [188:14:40<822:02:36, 23.39s/it]
 44%|████▍     | 98493/225000 [188:14:57<758:34:33, 21.59s/it]
 44%|████▍     | 98494/225000 [188:15:16<729:38:57, 20.76s/it]
 44%|████▍     | 98495/225000 [188:15:38<735:47:12, 20.94s/it]
 44%|████▍     | 98496/225000 [188:16:01<762:16:44, 21.69s/it]
 44%|████▍     | 98497/225000 [188:16:30<839:47:57, 23.90s/it]
 44%|████▍     | 98498/225000 [188:16:45<745:26:20, 21.21s/it]
 44%|████▍     | 98499/225000 [188:17:07<754:10:20, 21.46s/it]
 44%|████▍     | 98500/225000 [188:17:31<781:54:47, 22.25s/it]
                                                              
{'loss': '0.8804', 'grad_norm': '1.022', 'learning_rate': '3.187e-05', 'epoch': '0.4378', 'num_input_tokens_seen': 311339600, 'train_runtime': '6.779e+05', 'train_tokens_per_second': '459.3'}

 44%|████▍     | 98500/225000 [188:17:31<781:54:47, 22.25s/it]
 44%|████▍     | 98501/225000 [188:17:52<768:23:22, 21.87s/it]
 44%|████▍     | 98502/225000 [188:18:17<799:08:38, 22.74s/it]
 44%|████▍     | 98503/225000 [188:18:38<777:03:50, 22.11s/it]
 44%|████▍     | 98504/225000 [188:19:01<793:50:15, 22.59s/it]
 44%|████▍     | 98505/225000 [188:19:17<721:29:55, 20.53s/it]
 44%|████▍     | 98506/225000 [188:19:44<788:26:19, 22.44s/it]
 44%|████▍     | 98507/225000 [188:20:05<775:14:01, 22.06s/it]
 44%|████▍     | 98508/225000 [188:20:35<861:41:29, 24.52s/it]
 44%|████▍     | 98509/225000 [188:20:50<761:49:10, 21.68s/it]
 44%|████▍     | 98510/225000 [188:21:09<727:34:00, 20.71s/it]
 44%|████▍     | 98511/225000 [188:21:31<745:54:51, 21.23s/it]
 44%|████▍     | 98512/225000 [188:21:49<707:58:04, 20.15s/it]
 44%|████▍     | 98513/225000 [188:22:04<652:51:55, 18.58s/it]
 44%|████▍     | 98514/225000 [188:22:22<653:08:07, 18.59s/it]
 44%|████▍     | 98515/225000 [188:22:48<727:17:22, 20.70s/it]
 44%|████▍     | 98516/225000 [188:23:10<737:42:32, 21.00s/it]
 44%|████▍     | 98517/225000 [188:23:30<729:40:42, 20.77s/it]
 44%|████▍     | 98518/225000 [188:23:49<707:36:33, 20.14s/it]
 44%|████▍     | 98519/225000 [188:24:16<786:04:51, 22.37s/it]
 44%|████▍     | 98520/225000 [188:24:45<853:07:56, 24.28s/it]
 44%|████▍     | 98521/225000 [188:25:01<768:36:08, 21.88s/it]
 44%|████▍     | 98522/225000 [188:25:21<740:55:08, 21.09s/it]
 44%|████▍     | 98523/225000 [188:25:37<692:38:15, 19.72s/it]
 44%|████▍     | 98524/225000 [188:26:02<752:41:00, 21.42s/it]
 44%|████▍     | 98525/225000 [188:26:20<713:51:08, 20.32s/it]
 44%|████▍     | 98526/225000 [188:26:38<688:15:45, 19.59s/it]
 44%|████▍     | 98527/225000 [188:27:06<774:08:10, 22.04s/it]
 44%|████▍     | 98528/225000 [188:27:32<814:23:27, 23.18s/it]
 44%|████▍     | 98529/225000 [188:28:00<870:38:38, 24.78s/it]
 44%|████▍     | 98530/225000 [188:28:28<897:27:39, 25.55s/it]
 44%|████▍     | 98531/225000 [188:28:44<802:27:22, 22.84s/it]
 44%|████▍     | 98532/225000 [188:29:09<822:11:34, 23.40s/it]
 44%|████▍     | 98533/225000 [188:29:32<821:40:43, 23.39s/it]
 44%|████▍     | 98534/225000 [188:29:48<739:23:41, 21.05s/it]
 44%|████▍     | 98535/225000 [188:30:07<716:02:49, 20.38s/it]
 44%|████▍     | 98536/225000 [188:30:37<820:38:02, 23.36s/it]
 44%|████▍     | 98537/225000 [188:31:00<817:04:32, 23.26s/it]
 44%|████▍     | 98538/225000 [188:31:25<840:45:38, 23.93s/it]
 44%|████▍     | 98539/225000 [188:31:42<758:24:41, 21.59s/it]
 44%|████▍     | 98540/225000 [188:32:05<773:55:06, 22.03s/it]
 44%|████▍     | 98541/225000 [188:32:24<751:26:03, 21.39s/it]
 44%|████▍     | 98542/225000 [188:32:51<803:59:17, 22.89s/it]
 44%|████▍     | 98543/225000 [188:33:04<705:16:40, 20.08s/it]
 44%|████▍     | 98544/225000 [188:33:29<748:56:26, 21.32s/it]
 44%|████▍     | 98545/225000 [188:33:56<812:15:08, 23.12s/it]
 44%|████▍     | 98546/225000 [188:34:14<754:52:27, 21.49s/it]
 44%|████▍     | 98547/225000 [188:34:31<707:52:00, 20.15s/it]
 44%|████▍     | 98548/225000 [188:35:02<830:33:04, 23.65s/it]
 44%|████▍     | 98549/225000 [188:35:21<777:17:05, 22.13s/it]
 44%|████▍     | 98550/225000 [188:35:50<850:42:01, 24.22s/it]
                                                              
{'loss': '0.886', 'grad_norm': '0.8896', 'learning_rate': '3.185e-05', 'epoch': '0.438', 'num_input_tokens_seen': 311919584, 'train_runtime': '6.79e+05', 'train_tokens_per_second': '459.4'}

 44%|████▍     | 98550/225000 [188:35:50<850:42:01, 24.22s/it]
 44%|████▍     | 98551/225000 [188:36:10<806:22:22, 22.96s/it]
 44%|████▍     | 98552/225000 [188:36:29<759:44:29, 21.63s/it]
 44%|████▍     | 98553/225000 [188:36:49<750:39:48, 21.37s/it]
 44%|████▍     | 98554/225000 [188:37:12<765:58:39, 21.81s/it]
 44%|████▍     | 98555/225000 [188:37:41<838:57:19, 23.89s/it]
 44%|████▍     | 98556/225000 [188:38:08<876:04:58, 24.94s/it]
 44%|████▍     | 98557/225000 [188:38:30<836:44:06, 23.82s/it]
 44%|████▍     | 98558/225000 [188:38:52<818:19:42, 23.30s/it]
 44%|████▍     | 98559/225000 [188:39:24<918:19:54, 26.15s/it]
 44%|████▍     | 98560/225000 [188:39:41<815:38:41, 23.22s/it]
 44%|████▍     | 98561/225000 [188:40:09<864:05:12, 24.60s/it]
 44%|████▍     | 98562/225000 [188:40:31<836:32:34, 23.82s/it]
 44%|████▍     | 98563/225000 [188:40:57<866:58:59, 24.69s/it]
 44%|████▍     | 98564/225000 [188:41:24<884:16:49, 25.18s/it]
 44%|████▍     | 98565/225000 [188:41:40<788:33:24, 22.45s/it]
 44%|████▍     | 98566/225000 [188:42:01<776:08:10, 22.10s/it]
 44%|████▍     | 98567/225000 [188:42:26<803:42:13, 22.88s/it]
 44%|████▍     | 98568/225000 [188:42:44<754:36:59, 21.49s/it]
 44%|████▍     | 98569/225000 [188:43:02<720:26:12, 20.51s/it]
 44%|████▍     | 98570/225000 [188:43:34<840:15:57, 23.93s/it]
 44%|████▍     | 98571/225000 [188:44:01<875:42:04, 24.94s/it]
 44%|████▍     | 98572/225000 [188:44:24<845:39:04, 24.08s/it]
 44%|████▍     | 98573/225000 [188:44:50<868:40:30, 24.74s/it]
 44%|████▍     | 98574/225000 [188:45:15<875:15:33, 24.92s/it]
 44%|████▍     | 98575/225000 [188:45:41<883:00:29, 25.14s/it]
 44%|████▍     | 98576/225000 [188:45:53<750:59:59, 21.39s/it]
 44%|████▍     | 98577/225000 [188:46:15<751:22:37, 21.40s/it]
 44%|████▍     | 98578/225000 [188:46:37<762:26:10, 21.71s/it]
 44%|████▍     | 98579/225000 [188:47:02<798:27:42, 22.74s/it]
 44%|████▍     | 98580/225000 [188:47:28<830:52:43, 23.66s/it]
 44%|████▍     | 98581/225000 [188:47:56<871:27:17, 24.82s/it]
 44%|████▍     | 98582/225000 [188:48:17<828:54:19, 23.60s/it]
 44%|████▍     | 98583/225000 [188:48:43<863:32:24, 24.59s/it]
 44%|████▍     | 98584/225000 [188:48:58<753:19:57, 21.45s/it]
 44%|████▍     | 98585/225000 [188:49:25<816:37:45, 23.26s/it]
 44%|████▍     | 98586/225000 [188:49:45<779:57:15, 22.21s/it]
 44%|████▍     | 98587/225000 [188:49:54<642:08:42, 18.29s/it]
 44%|████▍     | 98588/225000 [188:50:14<657:40:22, 18.73s/it]
 44%|████▍     | 98589/225000 [188:50:33<661:05:17, 18.83s/it]
 44%|████▍     | 98590/225000 [188:51:02<769:41:21, 21.92s/it]
 44%|████▍     | 98591/225000 [188:51:32<856:48:17, 24.40s/it]
 44%|████▍     | 98592/225000 [188:51:54<830:29:28, 23.65s/it]
 44%|████▍     | 98593/225000 [188:52:16<815:18:28, 23.22s/it]
 44%|████▍     | 98594/225000 [188:52:38<799:32:06, 22.77s/it]
 44%|████▍     | 98595/225000 [188:53:03<824:54:16, 23.49s/it]
 44%|████▍     | 98596/225000 [188:53:24<802:03:31, 22.84s/it]
 44%|████▍     | 98597/225000 [188:53:43<760:58:32, 21.67s/it]
 44%|████▍     | 98598/225000 [188:54:00<712:50:55, 20.30s/it]
 44%|████▍     | 98599/225000 [188:54:31<825:13:08, 23.50s/it]
 44%|████▍     | 98600/225000 [188:54:56<836:46:25, 23.83s/it]
                                                              
{'loss': '0.8894', 'grad_norm': '1.014', 'learning_rate': '3.183e-05', 'epoch': '0.4382', 'num_input_tokens_seen': 312510248, 'train_runtime': '6.801e+05', 'train_tokens_per_second': '459.5'}

 44%|████▍     | 98600/225000 [188:54:56<836:46:25, 23.83s/it]
 44%|████▍     | 98601/225000 [188:55:19<825:29:59, 23.51s/it]
 44%|████▍     | 98602/225000 [188:55:30<698:12:01, 19.89s/it]
 44%|████▍     | 98603/225000 [188:56:01<810:27:00, 23.08s/it]
 44%|████▍     | 98604/225000 [188:56:28<851:45:25, 24.26s/it]
 44%|████▍     | 98605/225000 [188:56:55<878:25:29, 25.02s/it]
 44%|████▍     | 98606/225000 [188:57:15<825:43:37, 23.52s/it]
 44%|████▍     | 98607/225000 [188:57:34<783:52:19, 22.33s/it]
 44%|████▍     | 98608/225000 [188:57:50<712:20:41, 20.29s/it]
 44%|████▍     | 98609/225000 [188:58:21<833:40:30, 23.75s/it]
 44%|████▍     | 98610/225000 [188:58:42<798:17:52, 22.74s/it]
 44%|████▍     | 98611/225000 [188:59:07<828:33:52, 23.60s/it]
 44%|████▍     | 98612/225000 [188:59:19<704:58:11, 20.08s/it]
 44%|████▍     | 98613/225000 [188:59:46<774:34:40, 22.06s/it]
 44%|████▍     | 98614/225000 [189:00:14<838:24:34, 23.88s/it]
 44%|████▍     | 98615/225000 [189:00:41<874:07:55, 24.90s/it]
 44%|████▍     | 98616/225000 [189:01:06<867:59:05, 24.72s/it]
 44%|████▍     | 98617/225000 [189:01:25<807:12:06, 22.99s/it]
 44%|████▍     | 98618/225000 [189:01:47<804:50:51, 22.93s/it]
 44%|████▍     | 98619/225000 [189:02:18<882:21:59, 25.13s/it]
 44%|████▍     | 98620/225000 [189:02:35<803:10:51, 22.88s/it]
 44%|████▍     | 98621/225000 [189:03:03<857:14:17, 24.42s/it]
 44%|████▍     | 98622/225000 [189:03:25<831:00:48, 23.67s/it]
 44%|████▍     | 98623/225000 [189:03:47<810:48:16, 23.10s/it]
 44%|████▍     | 98624/225000 [189:04:11<819:55:04, 23.36s/it]
 44%|████▍     | 98625/225000 [189:04:28<756:35:25, 21.55s/it]
 44%|████▍     | 98626/225000 [189:04:56<818:41:48, 23.32s/it]
 44%|████▍     | 98627/225000 [189:05:16<789:45:32, 22.50s/it]
 44%|████▍     | 98628/225000 [189:05:38<778:26:29, 22.18s/it]
 44%|████▍     | 98629/225000 [189:06:09<874:07:14, 24.90s/it]
 44%|████▍     | 98630/225000 [189:06:34<879:16:38, 25.05s/it]
 44%|████▍     | 98631/225000 [189:07:03<919:21:07, 26.19s/it]
 44%|████▍     | 98632/225000 [189:07:31<936:34:11, 26.68s/it]
 44%|████▍     | 98633/225000 [189:07:45<803:38:03, 22.89s/it]
 44%|████▍     | 98634/225000 [189:08:10<825:41:20, 23.52s/it]
 44%|████▍     | 98635/225000 [189:08:34<827:25:23, 23.57s/it]
 44%|████▍     | 98636/225000 [189:08:50<748:39:13, 21.33s/it]
 44%|████▍     | 98637/225000 [189:09:14<778:40:30, 22.18s/it]
 44%|████▍     | 98638/225000 [189:09:40<822:02:03, 23.42s/it]
 44%|████▍     | 98639/225000 [189:10:10<882:43:56, 25.15s/it]
 44%|████▍     | 98640/225000 [189:10:34<878:39:27, 25.03s/it]
 44%|████▍     | 98641/225000 [189:11:03<911:49:16, 25.98s/it]
 44%|████▍     | 98642/225000 [189:11:23<854:18:20, 24.34s/it]
 44%|████▍     | 98643/225000 [189:11:43<804:28:59, 22.92s/it]
 44%|████▍     | 98644/225000 [189:12:10<854:45:25, 24.35s/it]
 44%|████▍     | 98645/225000 [189:12:37<881:42:52, 25.12s/it]
 44%|████▍     | 98646/225000 [189:13:03<889:53:51, 25.35s/it]
 44%|████▍     | 98647/225000 [189:13:24<840:01:57, 23.93s/it]
 44%|████▍     | 98648/225000 [189:13:45<806:24:43, 22.98s/it]
 44%|████▍     | 98649/225000 [189:14:05<781:02:48, 22.25s/it]
 44%|████▍     | 98650/225000 [189:14:26<766:45:01, 21.85s/it]
                                                              
{'loss': '0.8818', 'grad_norm': '1.132', 'learning_rate': '3.182e-05', 'epoch': '0.4384', 'num_input_tokens_seen': 313109056, 'train_runtime': '6.813e+05', 'train_tokens_per_second': '459.6'}

 44%|████▍     | 98650/225000 [189:14:26<766:45:01, 21.85s/it]
 44%|████▍     | 98651/225000 [189:14:46<751:45:07, 21.42s/it]
 44%|████▍     | 98652/225000 [189:15:02<691:16:08, 19.70s/it]
 44%|████▍     | 98653/225000 [189:15:25<727:55:59, 20.74s/it]
 44%|████▍     | 98654/225000 [189:15:46<728:55:00, 20.77s/it]
 44%|████▍     | 98655/225000 [189:16:09<750:27:00, 21.38s/it]
 44%|████▍     | 98656/225000 [189:16:31<759:59:01, 21.65s/it]
 44%|████▍     | 98657/225000 [189:16:58<816:50:29, 23.27s/it]
 44%|████▍     | 98658/225000 [189:17:26<865:47:27, 24.67s/it]
 44%|████▍     | 98659/225000 [189:17:50<852:52:26, 24.30s/it]
 44%|████▍     | 98660/225000 [189:18:08<792:09:26, 22.57s/it]
 44%|████▍     | 98661/225000 [189:18:37<853:53:49, 24.33s/it]
 44%|████▍     | 98662/225000 [189:19:04<883:46:43, 25.18s/it]
 44%|████▍     | 98663/225000 [189:19:26<850:09:39, 24.23s/it]
 44%|████▍     | 98664/225000 [189:19:55<897:39:43, 25.58s/it]
 44%|████▍     | 98665/225000 [189:20:19<886:43:47, 25.27s/it]
 44%|████▍     | 98666/225000 [189:20:39<828:57:15, 23.62s/it]
 44%|████▍     | 98667/225000 [189:20:57<773:30:44, 22.04s/it]
 44%|████▍     | 98668/225000 [189:21:26<844:41:48, 24.07s/it]
 44%|████▍     | 98669/225000 [189:21:55<891:52:49, 25.42s/it]
 44%|████▍     | 98670/225000 [189:22:17<864:18:17, 24.63s/it]
 44%|████▍     | 98671/225000 [189:22:32<761:36:11, 21.70s/it]
 44%|████▍     | 98672/225000 [189:22:56<780:05:30, 22.23s/it]
 44%|████▍     | 98673/225000 [189:23:22<819:48:56, 23.36s/it]
 44%|████▍     | 98674/225000 [189:23:43<799:29:16, 22.78s/it]
 44%|████▍     | 98675/225000 [189:24:04<783:10:36, 22.32s/it]
 44%|████▍     | 98676/225000 [189:24:31<830:26:57, 23.67s/it]
 44%|████▍     | 98677/225000 [189:24:50<778:13:53, 22.18s/it]
 44%|████▍     | 98678/225000 [189:25:20<864:04:47, 24.63s/it]
 44%|████▍     | 98679/225000 [189:25:51<933:15:33, 26.60s/it]
 44%|████▍     | 98680/225000 [189:26:15<904:46:17, 25.79s/it]
 44%|████▍     | 98681/225000 [189:26:38<867:35:47, 24.73s/it]
 44%|████▍     | 98682/225000 [189:26:58<819:12:24, 23.35s/it]
 44%|████▍     | 98683/225000 [189:27:24<850:23:48, 24.24s/it]
 44%|████▍     | 98684/225000 [189:27:43<792:04:57, 22.57s/it]
 44%|████▍     | 98685/225000 [189:28:05<789:07:55, 22.49s/it]
 44%|████▍     | 98686/225000 [189:28:27<779:58:52, 22.23s/it]
 44%|████▍     | 98687/225000 [189:28:56<851:06:10, 24.26s/it]
 44%|████▍     | 98688/225000 [189:29:18<828:21:39, 23.61s/it]
 44%|████▍     | 98689/225000 [189:29:34<752:49:37, 21.46s/it]
 44%|████▍     | 98690/225000 [189:29:55<746:45:00, 21.28s/it]
 44%|████▍     | 98691/225000 [189:30:17<750:50:43, 21.40s/it]
 44%|████▍     | 98692/225000 [189:30:35<718:14:57, 20.47s/it]
 44%|████▍     | 98693/225000 [189:31:00<764:13:55, 21.78s/it]
 44%|████▍     | 98694/225000 [189:31:20<742:36:33, 21.17s/it]
 44%|████▍     | 98695/225000 [189:31:54<884:07:30, 25.20s/it]
 44%|████▍     | 98696/225000 [189:32:16<848:48:20, 24.19s/it]
 44%|████▍     | 98697/225000 [189:32:41<861:35:46, 24.56s/it]
 44%|████▍     | 98698/225000 [189:33:08<877:30:28, 25.01s/it]
 44%|████▍     | 98699/225000 [189:33:34<891:20:53, 25.41s/it]
 44%|████▍     | 98700/225000 [189:33:54<839:40:44, 23.93s/it]
                                                              
{'loss': '0.9067', 'grad_norm': '0.905', 'learning_rate': '3.18e-05', 'epoch': '0.4387', 'num_input_tokens_seen': 313723488, 'train_runtime': '6.824e+05', 'train_tokens_per_second': '459.7'}

 44%|████▍     | 98700/225000 [189:33:54<839:40:44, 23.93s/it]
 44%|████▍     | 98701/225000 [189:34:12<768:36:29, 21.91s/it]
 44%|████▍     | 98702/225000 [189:34:32<755:36:02, 21.54s/it]
 44%|████▍     | 98703/225000 [189:34:52<732:43:47, 20.89s/it]
 44%|████▍     | 98704/225000 [189:35:19<800:50:40, 22.83s/it]
 44%|████▍     | 98705/225000 [189:35:41<794:37:00, 22.65s/it]
 44%|████▍     | 98706/225000 [189:36:08<835:00:22, 23.80s/it]
 44%|████▍     | 98707/225000 [189:36:32<835:40:17, 23.82s/it]
 44%|████▍     | 98708/225000 [189:36:56<839:43:25, 23.94s/it]
 44%|████▍     | 98709/225000 [189:37:17<815:56:52, 23.26s/it]
 44%|████▍     | 98710/225000 [189:37:38<784:24:20, 22.36s/it]
 44%|████▍     | 98711/225000 [189:37:53<711:09:34, 20.27s/it]
 44%|████▍     | 98712/225000 [189:38:22<807:12:26, 23.01s/it]
 44%|████▍     | 98713/225000 [189:38:39<739:35:36, 21.08s/it]
 44%|████▍     | 98714/225000 [189:39:09<828:24:54, 23.62s/it]
 44%|████▍     | 98715/225000 [189:39:36<870:23:01, 24.81s/it]
 44%|████▍     | 98716/225000 [189:39:56<817:59:30, 23.32s/it]
 44%|████▍     | 98717/225000 [189:40:25<872:45:45, 24.88s/it]
 44%|████▍     | 98718/225000 [189:40:45<828:00:34, 23.60s/it]
 44%|████▍     | 98719/225000 [189:41:07<804:04:52, 22.92s/it]
 44%|████▍     | 98720/225000 [189:41:33<845:52:35, 24.11s/it]
 44%|████▍     | 98721/225000 [189:41:56<828:29:10, 23.62s/it]
 44%|████▍     | 98722/225000 [189:42:22<851:26:12, 24.27s/it]
 44%|████▍     | 98723/225000 [189:42:42<807:36:50, 23.02s/it]
 44%|████▍     | 98724/225000 [189:43:07<830:53:18, 23.69s/it]
 44%|████▍     | 98725/225000 [189:43:33<856:27:15, 24.42s/it]
 44%|████▍     | 98726/225000 [189:43:59<875:23:46, 24.96s/it]
 44%|████▍     | 98727/225000 [189:44:25<877:22:10, 25.01s/it]
 44%|████▍     | 98728/225000 [189:44:48<861:29:22, 24.56s/it]
 44%|████▍     | 98729/225000 [189:45:10<833:00:36, 23.75s/it]
 44%|████▍     | 98730/225000 [189:45:36<858:16:20, 24.47s/it]
 44%|████▍     | 98731/225000 [189:45:52<766:19:22, 21.85s/it]
 44%|████▍     | 98732/225000 [189:46:23<861:15:46, 24.56s/it]
 44%|████▍     | 98733/225000 [189:46:36<740:36:47, 21.12s/it]
 44%|████▍     | 98734/225000 [189:46:55<720:08:27, 20.53s/it]
 44%|████▍     | 98735/225000 [189:47:23<797:44:59, 22.75s/it]
 44%|████▍     | 98736/225000 [189:47:44<785:27:29, 22.39s/it]
 44%|████▍     | 98737/225000 [189:48:06<775:22:49, 22.11s/it]
 44%|████▍     | 98738/225000 [189:48:33<829:19:33, 23.65s/it]
 44%|████▍     | 98739/225000 [189:48:54<800:07:44, 22.81s/it]
 44%|████▍     | 98740/225000 [189:49:12<748:26:39, 21.34s/it]
 44%|████▍     | 98741/225000 [189:49:39<807:54:25, 23.04s/it]
 44%|████▍     | 98742/225000 [189:50:06<848:45:25, 24.20s/it]
 44%|████▍     | 98743/225000 [189:50:26<806:26:34, 22.99s/it]
 44%|████▍     | 98744/225000 [189:50:47<790:32:16, 22.54s/it]
 44%|████▍     | 98745/225000 [189:51:08<772:36:57, 22.03s/it]
 44%|████▍     | 98746/225000 [189:51:33<804:24:07, 22.94s/it]
 44%|████▍     | 98747/225000 [189:51:53<766:59:51, 21.87s/it]
 44%|████▍     | 98748/225000 [189:52:16<779:37:16, 22.23s/it]
 44%|████▍     | 98749/225000 [189:52:41<807:51:40, 23.04s/it]
 44%|████▍     | 98750/225000 [189:52:59<761:46:54, 21.72s/it]
                                                              
{'loss': '0.8875', 'grad_norm': '1.285', 'learning_rate': '3.178e-05', 'epoch': '0.4389', 'num_input_tokens_seen': 314314408, 'train_runtime': '6.836e+05', 'train_tokens_per_second': '459.8'}

 44%|████▍     | 98750/225000 [189:52:59<761:46:54, 21.72s/it]
 44%|████▍     | 98751/225000 [189:53:22<768:41:05, 21.92s/it]
 44%|████▍     | 98752/225000 [189:53:41<744:24:24, 21.23s/it]
 44%|████▍     | 98753/225000 [189:54:14<859:57:52, 24.52s/it]
 44%|████▍     | 98754/225000 [189:54:41<890:16:27, 25.39s/it]
 44%|████▍     | 98755/225000 [189:55:04<864:47:39, 24.66s/it]
 44%|████▍     | 98756/225000 [189:55:31<890:52:49, 25.40s/it]
 44%|████▍     | 98757/225000 [189:55:47<791:09:26, 22.56s/it]
 44%|████▍     | 98758/225000 [189:56:09<785:40:16, 22.40s/it]
 44%|████▍     | 98759/225000 [189:56:33<807:23:17, 23.02s/it]
 44%|████▍     | 98760/225000 [189:56:59<829:07:05, 23.64s/it]
 44%|████▍     | 98761/225000 [189:57:20<806:43:19, 23.01s/it]
 44%|████▍     | 98762/225000 [189:57:47<842:55:10, 24.04s/it]
 44%|████▍     | 98763/225000 [189:58:08<812:49:00, 23.18s/it]
 44%|████▍     | 98764/225000 [189:58:32<824:20:59, 23.51s/it]
 44%|████▍     | 98765/225000 [189:58:54<804:41:57, 22.95s/it]
 44%|████▍     | 98766/225000 [189:59:19<832:25:24, 23.74s/it]
 44%|████▍     | 98767/225000 [189:59:40<802:57:49, 22.90s/it]
 44%|████▍     | 98768/225000 [190:00:02<795:45:38, 22.69s/it]
 44%|████▍     | 98769/225000 [190:00:28<824:27:03, 23.51s/it]
 44%|████▍     | 98770/225000 [190:00:48<792:55:38, 22.61s/it]
 44%|████▍     | 98771/225000 [190:01:09<776:13:42, 22.14s/it]
 44%|████▍     | 98772/225000 [190:01:39<857:19:26, 24.45s/it]
 44%|████▍     | 98773/225000 [190:02:06<884:38:33, 25.23s/it]
 44%|████▍     | 98774/225000 [190:02:37<938:12:22, 26.76s/it]
 44%|████▍     | 98775/225000 [190:02:57<876:10:07, 24.99s/it]
 44%|████▍     | 98776/225000 [190:03:26<912:40:30, 26.03s/it]
 44%|████▍     | 98777/225000 [190:03:39<780:02:48, 22.25s/it]
 44%|████▍     | 98778/225000 [190:04:04<806:39:07, 23.01s/it]
 44%|████▍     | 98779/225000 [190:04:21<747:22:43, 21.32s/it]
 44%|████▍     | 98780/225000 [190:04:54<867:46:02, 24.75s/it]
 44%|████▍     | 98781/225000 [190:05:17<848:32:25, 24.20s/it]
 44%|████▍     | 98782/225000 [190:05:46<901:55:24, 25.72s/it]
 44%|████▍     | 98783/225000 [190:06:11<890:40:48, 25.40s/it]
 44%|████▍     | 98784/225000 [190:06:41<937:05:15, 26.73s/it]
 44%|████▍     | 98785/225000 [190:07:05<910:04:44, 25.96s/it]
 44%|████▍     | 98786/225000 [190:07:25<847:34:01, 24.18s/it]
 44%|████▍     | 98787/225000 [190:07:40<754:23:38, 21.52s/it]
 44%|████▍     | 98788/225000 [190:08:04<773:37:04, 22.07s/it]
 44%|████▍     | 98789/225000 [190:08:25<769:29:14, 21.95s/it]
 44%|████▍     | 98790/225000 [190:08:47<761:17:57, 21.72s/it]
 44%|████▍     | 98791/225000 [190:09:06<733:19:38, 20.92s/it]
 44%|████▍     | 98792/225000 [190:09:33<803:57:13, 22.93s/it]
 44%|████▍     | 98793/225000 [190:09:54<776:26:26, 22.15s/it]
 44%|████▍     | 98794/225000 [190:10:20<821:30:56, 23.43s/it]
 44%|████▍     | 98795/225000 [190:10:39<773:01:22, 22.05s/it]
 44%|████▍     | 98796/225000 [190:11:08<848:46:51, 24.21s/it]
 44%|████▍     | 98797/225000 [190:11:27<788:56:09, 22.50s/it]
 44%|████▍     | 98798/225000 [190:11:52<817:04:33, 23.31s/it]
 44%|████▍     | 98799/225000 [190:12:17<836:58:26, 23.88s/it]
 44%|████▍     | 98800/225000 [190:12:43<860:34:00, 24.55s/it]
                                                              
{'loss': '0.8959', 'grad_norm': '1.048', 'learning_rate': '3.176e-05', 'epoch': '0.4391', 'num_input_tokens_seen': 314926344, 'train_runtime': '6.848e+05', 'train_tokens_per_second': '459.9'}

 44%|████▍     | 98800/225000 [190:12:43<860:34:00, 24.55s/it]
 44%|████▍     | 98801/225000 [190:13:03<814:09:49, 23.23s/it]
 44%|████▍     | 98802/225000 [190:13:34<889:04:32, 25.36s/it]
 44%|████▍     | 98803/225000 [190:14:02<922:48:26, 26.32s/it]
 44%|████▍     | 98804/225000 [190:14:22<857:45:07, 24.47s/it]
 44%|████▍     | 98805/225000 [190:14:41<799:04:16, 22.80s/it]
 44%|████▍     | 98806/225000 [190:15:10<861:25:56, 24.57s/it]
 44%|████▍     | 98807/225000 [190:15:34<858:31:00, 24.49s/it]
 44%|████▍     | 98808/225000 [190:15:59<862:48:33, 24.61s/it]
 44%|████▍     | 98809/225000 [190:16:22<844:50:24, 24.10s/it]
 44%|████▍     | 98810/225000 [190:16:43<809:37:59, 23.10s/it]
 44%|████▍     | 98811/225000 [190:17:02<766:25:50, 21.87s/it]
 44%|████▍     | 98812/225000 [190:17:24<768:51:37, 21.93s/it]
 44%|████▍     | 98813/225000 [190:17:57<883:35:39, 25.21s/it]
 44%|████▍     | 98814/225000 [190:18:20<865:38:10, 24.70s/it]
 44%|████▍     | 98815/225000 [190:18:40<816:09:40, 23.28s/it]
 44%|████▍     | 98816/225000 [190:19:05<827:49:03, 23.62s/it]
 44%|████▍     | 98817/225000 [190:19:31<859:23:33, 24.52s/it]
 44%|████▍     | 98818/225000 [190:19:52<824:10:32, 23.51s/it]
 44%|████▍     | 98819/225000 [190:20:21<879:07:44, 25.08s/it]
 44%|████▍     | 98820/225000 [190:20:50<919:58:31, 26.25s/it]
 44%|████▍     | 98821/225000 [190:21:08<828:45:02, 23.65s/it]
 44%|████▍     | 98822/225000 [190:21:37<888:21:49, 25.35s/it]
 44%|████▍     | 98823/225000 [190:22:03<900:41:23, 25.70s/it]
 44%|████▍     | 98824/225000 [190:22:21<809:55:44, 23.11s/it]
 44%|████▍     | 98825/225000 [190:22:51<887:55:42, 25.33s/it]
 44%|████▍     | 98826/225000 [190:23:16<879:03:41, 25.08s/it]
 44%|████▍     | 98827/225000 [190:23:43<906:33:24, 25.87s/it]
 44%|████▍     | 98828/225000 [190:24:04<848:03:24, 24.20s/it]
 44%|████▍     | 98829/225000 [190:24:20<763:37:52, 21.79s/it]
 44%|████▍     | 98830/225000 [190:24:36<707:41:07, 20.19s/it]
 44%|████▍     | 98831/225000 [190:25:09<837:00:40, 23.88s/it]
 44%|████▍     | 98832/225000 [190:25:33<844:23:00, 24.09s/it]
 44%|████▍     | 98833/225000 [190:25:54<805:42:20, 22.99s/it]
 44%|████▍     | 98834/225000 [190:26:14<773:48:23, 22.08s/it]
 44%|████▍     | 98835/225000 [190:26:43<854:52:01, 24.39s/it]
 44%|████▍     | 98836/225000 [190:27:02<796:31:37, 22.73s/it]
 44%|████▍     | 98837/225000 [190:27:24<784:30:46, 22.39s/it]
 44%|████▍     | 98838/225000 [190:27:50<824:02:45, 23.51s/it]
 44%|████▍     | 98839/225000 [190:28:11<801:58:24, 22.88s/it]
 44%|████▍     | 98840/225000 [190:28:35<810:40:47, 23.13s/it]
 44%|████▍     | 98841/225000 [190:28:58<803:58:24, 22.94s/it]
 44%|████▍     | 98842/225000 [190:29:25<853:49:44, 24.36s/it]
 44%|████▍     | 98843/225000 [190:29:47<822:03:50, 23.46s/it]
 44%|████▍     | 98844/225000 [190:30:10<823:03:24, 23.49s/it]
 44%|████▍     | 98845/225000 [190:30:33<811:47:51, 23.17s/it]
 44%|████▍     | 98846/225000 [190:30:53<782:57:55, 22.34s/it]
 44%|████▍     | 98847/225000 [190:31:14<767:13:58, 21.89s/it]
 44%|████▍     | 98848/225000 [190:31:35<758:25:16, 21.64s/it]
 44%|████▍     | 98849/225000 [190:32:02<817:41:34, 23.33s/it]
 44%|████▍     | 98850/225000 [190:32:29<848:27:11, 24.21s/it]
                                                              
{'loss': '0.8758', 'grad_norm': '1.031', 'learning_rate': '3.175e-05', 'epoch': '0.4393', 'num_input_tokens_seen': 315536232, 'train_runtime': '6.859e+05', 'train_tokens_per_second': '460'}

 44%|████▍     | 98850/225000 [190:32:29<848:27:11, 24.21s/it]
 44%|████▍     | 98851/225000 [190:32:54<865:16:16, 24.69s/it]
 44%|████▍     | 98852/225000 [190:33:15<822:32:31, 23.47s/it]
 44%|████▍     | 98853/225000 [190:33:32<758:59:50, 21.66s/it]
 44%|████▍     | 98854/225000 [190:33:52<741:23:23, 21.16s/it]
 44%|████▍     | 98855/225000 [190:34:21<816:16:00, 23.30s/it]
 44%|████▍     | 98856/225000 [190:34:34<713:53:22, 20.37s/it]
 44%|████▍     | 98857/225000 [190:35:00<767:08:45, 21.89s/it]
 44%|████▍     | 98858/225000 [190:35:22<769:01:10, 21.95s/it]
 44%|████▍     | 98859/225000 [190:35:54<879:48:47, 25.11s/it]
 44%|████▍     | 98860/225000 [190:36:15<829:56:12, 23.69s/it]
 44%|████▍     | 98861/225000 [190:36:38<824:56:38, 23.54s/it]
 44%|████▍     | 98862/225000 [190:37:08<890:41:56, 25.42s/it]
 44%|████▍     | 98863/225000 [190:37:29<845:01:45, 24.12s/it]
 44%|████▍     | 98864/225000 [190:37:50<812:39:57, 23.19s/it]
 44%|████▍     | 98865/225000 [190:38:08<765:26:21, 21.85s/it]
 44%|████▍     | 98866/225000 [190:38:29<757:27:00, 21.62s/it]
 44%|████▍     | 98867/225000 [190:38:52<767:38:36, 21.91s/it]
 44%|████▍     | 98868/225000 [190:39:17<804:01:21, 22.95s/it]
 44%|████▍     | 98869/225000 [190:39:43<826:50:18, 23.60s/it]
 44%|████▍     | 98870/225000 [190:40:10<863:18:26, 24.64s/it]
 44%|████▍     | 98871/225000 [190:40:29<803:20:18, 22.93s/it]
 44%|████▍     | 98872/225000 [190:40:55<840:00:09, 23.98s/it]
 44%|████▍     | 98873/225000 [190:41:17<823:40:16, 23.51s/it]
 44%|████▍     | 98874/225000 [190:41:35<759:30:48, 21.68s/it]
 44%|████▍     | 98875/225000 [190:42:04<839:03:35, 23.95s/it]
 44%|████▍     | 98876/225000 [190:42:27<832:16:29, 23.76s/it]
 44%|████▍     | 98877/225000 [190:42:50<816:22:24, 23.30s/it]
 44%|████▍     | 98878/225000 [190:43:19<875:58:19, 25.00s/it]
 44%|████▍     | 98879/225000 [190:43:50<945:34:03, 26.99s/it]
 44%|████▍     | 98880/225000 [190:44:16<930:49:09, 26.57s/it]
 44%|████▍     | 98881/225000 [190:44:41<921:01:26, 26.29s/it]
 44%|████▍     | 98882/225000 [190:44:57<805:01:46, 22.98s/it]
 44%|████▍     | 98883/225000 [190:45:22<833:57:15, 23.81s/it]
 44%|████▍     | 98884/225000 [190:45:51<879:42:14, 25.11s/it]
 44%|████▍     | 98885/225000 [190:46:15<875:39:44, 25.00s/it]
 44%|████▍     | 98886/225000 [190:46:34<808:51:18, 23.09s/it]
 44%|████▍     | 98887/225000 [190:46:59<829:21:55, 23.67s/it]
 44%|████▍     | 98888/225000 [190:47:22<826:06:21, 23.58s/it]
 44%|████▍     | 98889/225000 [190:47:46<822:55:18, 23.49s/it]
 44%|████▍     | 98890/225000 [190:48:10<835:54:21, 23.86s/it]
 44%|████▍     | 98891/225000 [190:48:31<801:58:06, 22.89s/it]
 44%|████▍     | 98892/225000 [190:49:00<861:12:15, 24.58s/it]
 44%|████▍     | 98893/225000 [190:49:20<813:20:18, 23.22s/it]
 44%|████▍     | 98894/225000 [190:49:38<757:52:39, 21.64s/it]
 44%|████▍     | 98895/225000 [190:50:02<783:32:47, 22.37s/it]
 44%|████▍     | 98896/225000 [190:50:22<761:18:44, 21.73s/it]
 44%|████▍     | 98897/225000 [190:50:48<810:33:08, 23.14s/it]
 44%|████▍     | 98898/225000 [190:51:08<774:04:15, 22.10s/it]
 44%|████▍     | 98899/225000 [190:51:32<793:33:59, 22.66s/it]
 44%|████▍     | 98900/225000 [190:51:57<815:35:45, 23.28s/it]
                                                              
{'loss': '0.8786', 'grad_norm': '1.016', 'learning_rate': '3.173e-05', 'epoch': '0.4396', 'num_input_tokens_seen': 316147440, 'train_runtime': '6.871e+05', 'train_tokens_per_second': '460.1'}

 44%|████▍     | 98900/225000 [190:51:57<815:35:45, 23.28s/it]
 44%|████▍     | 98901/225000 [190:52:08<689:18:57, 19.68s/it]
 44%|████▍     | 98902/225000 [190:52:29<707:30:44, 20.20s/it]
 44%|████▍     | 98903/225000 [190:52:56<779:00:39, 22.24s/it]
 44%|████▍     | 98904/225000 [190:53:24<834:42:28, 23.83s/it]
 44%|████▍     | 98905/225000 [190:53:42<774:42:29, 22.12s/it]
 44%|████▍     | 98906/225000 [190:54:05<785:54:45, 22.44s/it]
 44%|████▍     | 98907/225000 [190:54:30<810:16:45, 23.13s/it]
 44%|████▍     | 98908/225000 [190:54:57<850:50:03, 24.29s/it]
 44%|████▍     | 98909/225000 [190:55:24<882:48:45, 25.21s/it]
 44%|████▍     | 98910/225000 [190:55:50<889:46:16, 25.40s/it]
 44%|████▍     | 98911/225000 [190:56:18<910:42:05, 26.00s/it]
 44%|████▍     | 98912/225000 [190:56:43<907:21:06, 25.91s/it]
 44%|████▍     | 98913/225000 [190:57:07<883:00:06, 25.21s/it]
 44%|████▍     | 98914/225000 [190:57:29<855:02:15, 24.41s/it]
 44%|████▍     | 98915/225000 [190:57:57<893:23:49, 25.51s/it]
 44%|████▍     | 98916/225000 [190:58:22<884:35:19, 25.26s/it]
 44%|████▍     | 98917/225000 [190:58:38<784:40:49, 22.40s/it]
 44%|████▍     | 98918/225000 [190:59:02<801:13:07, 22.88s/it]
 44%|████▍     | 98919/225000 [190:59:24<793:08:27, 22.65s/it]
 44%|████▍     | 98920/225000 [190:59:46<788:02:52, 22.50s/it]
 44%|████▍     | 98921/225000 [191:00:24<947:03:35, 27.04s/it]
 44%|████▍     | 98922/225000 [191:00:45<886:29:49, 25.31s/it]
 44%|████▍     | 98923/225000 [191:01:15<930:55:16, 26.58s/it]
 44%|████▍     | 98924/225000 [191:01:39<911:16:17, 26.02s/it]
 44%|████▍     | 98925/225000 [191:02:02<874:56:53, 24.98s/it]
 44%|████▍     | 98926/225000 [191:02:21<810:39:44, 23.15s/it]
 44%|████▍     | 98927/225000 [191:02:49<866:32:49, 24.74s/it]
 44%|████▍     | 98928/225000 [191:03:12<849:40:50, 24.26s/it]
 44%|████▍     | 98929/225000 [191:03:42<911:21:22, 26.02s/it]
 44%|████▍     | 98930/225000 [191:04:03<853:30:35, 24.37s/it]
 44%|████▍     | 98931/225000 [191:04:26<835:10:49, 23.85s/it]
 44%|████▍     | 98932/225000 [191:04:56<901:35:59, 25.75s/it]
 44%|████▍     | 98933/225000 [191:05:24<923:55:38, 26.38s/it]
 44%|████▍     | 98934/225000 [191:05:46<883:50:22, 25.24s/it]
 44%|████▍     | 98935/225000 [191:06:12<891:32:53, 25.46s/it]
 44%|████▍     | 98936/225000 [191:06:27<785:17:21, 22.43s/it]
 44%|████▍     | 98937/225000 [191:06:50<782:28:29, 22.35s/it]
 44%|████▍     | 98938/225000 [191:07:16<828:02:30, 23.65s/it]
 44%|████▍     | 98939/225000 [191:07:42<850:34:24, 24.29s/it]
 44%|████▍     | 98940/225000 [191:08:07<855:26:02, 24.43s/it]
 44%|████▍     | 98941/225000 [191:08:30<840:14:08, 24.00s/it]
 44%|████▍     | 98942/225000 [191:08:52<824:26:20, 23.54s/it]
 44%|████▍     | 98943/225000 [191:09:12<780:58:53, 22.30s/it]
 44%|████▍     | 98944/225000 [191:09:44<888:42:46, 25.38s/it]
 44%|████▍     | 98945/225000 [191:10:12<917:14:24, 26.20s/it]
 44%|████▍     | 98946/225000 [191:10:39<917:09:55, 26.19s/it]
 44%|████▍     | 98947/225000 [191:10:52<778:34:11, 22.24s/it]
 44%|████▍     | 98948/225000 [191:11:08<712:15:41, 20.34s/it]
 44%|████▍     | 98949/225000 [191:11:36<794:04:24, 22.68s/it]
 44%|████▍     | 98950/225000 [191:12:03<847:12:45, 24.20s/it]
                                                              
{'loss': '0.9008', 'grad_norm': '0.9591', 'learning_rate': '3.171e-05', 'epoch': '0.4398', 'num_input_tokens_seen': 316761728, 'train_runtime': '6.883e+05', 'train_tokens_per_second': '460.2'}

 44%|████▍     | 98950/225000 [191:12:03<847:12:45, 24.20s/it]
 44%|████▍     | 98951/225000 [191:12:27<838:40:41, 23.95s/it]
 44%|████▍     | 98952/225000 [191:12:52<851:17:12, 24.31s/it]
 44%|████▍     | 98953/225000 [191:13:10<786:46:49, 22.47s/it]
 44%|████▍     | 98954/225000 [191:13:37<828:10:07, 23.65s/it]
 44%|████▍     | 98955/225000 [191:13:53<756:01:54, 21.59s/it]
 44%|████▍     | 98956/225000 [191:14:13<735:09:58, 21.00s/it]
 44%|████▍     | 98957/225000 [191:14:39<789:22:47, 22.55s/it]
 44%|████▍     | 98958/225000 [191:15:08<859:13:53, 24.54s/it]
 44%|████▍     | 98959/225000 [191:15:37<900:49:36, 25.73s/it]
 44%|████▍     | 98960/225000 [191:16:04<921:01:31, 26.31s/it]
 44%|████▍     | 98961/225000 [191:16:28<896:11:07, 25.60s/it]
 44%|████▍     | 98962/225000 [191:16:50<858:23:08, 24.52s/it]
 44%|████▍     | 98963/225000 [191:17:08<785:55:44, 22.45s/it]
 44%|████▍     | 98964/225000 [191:17:36<842:04:43, 24.05s/it]
 44%|████▍     | 98965/225000 [191:18:05<900:56:16, 25.73s/it]
 44%|████▍     | 98966/225000 [191:18:31<894:58:16, 25.56s/it]
 44%|████▍     | 98967/225000 [191:18:52<848:38:04, 24.24s/it]
 44%|████▍     | 98968/225000 [191:19:23<924:56:43, 26.42s/it]
 44%|████▍     | 98969/225000 [191:19:49<913:52:40, 26.10s/it]
 44%|████▍     | 98970/225000 [191:20:00<761:01:42, 21.74s/it]
 44%|████▍     | 98971/225000 [191:20:21<755:22:55, 21.58s/it]
 44%|████▍     | 98972/225000 [191:20:46<783:38:52, 22.38s/it]
 44%|████▍     | 98973/225000 [191:21:09<792:23:01, 22.63s/it]
 44%|████▍     | 98974/225000 [191:21:42<898:42:29, 25.67s/it]
 44%|████▍     | 98975/225000 [191:22:09<918:12:55, 26.23s/it]
 44%|████▍     | 98976/225000 [191:22:33<893:54:59, 25.54s/it]
 44%|████▍     | 98977/225000 [191:22:52<820:51:03, 23.45s/it]
 44%|████▍     | 98978/225000 [191:23:19<865:43:47, 24.73s/it]
 44%|████▍     | 98979/225000 [191:23:41<830:49:02, 23.73s/it]
 44%|████▍     | 98980/225000 [191:24:03<817:40:11, 23.36s/it]
 44%|████▍     | 98981/225000 [191:24:29<842:59:28, 24.08s/it]
 44%|████▍     | 98982/225000 [191:24:50<815:21:30, 23.29s/it]
 44%|████▍     | 98983/225000 [191:25:13<803:55:25, 22.97s/it]
 44%|████▍     | 98984/225000 [191:25:37<821:09:29, 23.46s/it]
 44%|████▍     | 98985/225000 [191:26:00<809:52:47, 23.14s/it]
 44%|████▍     | 98986/225000 [191:26:29<876:00:50, 25.03s/it]
 44%|████▍     | 98987/225000 [191:26:48<808:48:39, 23.11s/it]
 44%|████▍     | 98988/225000 [191:27:09<784:31:55, 22.41s/it]
 44%|████▍     | 98989/225000 [191:27:42<898:14:02, 25.66s/it]
 44%|████▍     | 98990/225000 [191:28:07<889:22:41, 25.41s/it]
 44%|████▍     | 98991/225000 [191:28:29<856:57:07, 24.48s/it]
 44%|████▍     | 98992/225000 [191:28:55<876:32:50, 25.04s/it]
 44%|████▍     | 98993/225000 [191:29:24<912:39:25, 26.07s/it]
 44%|████▍     | 98994/225000 [191:29:50<912:29:36, 26.07s/it]
 44%|████▍     | 98995/225000 [191:30:12<871:12:20, 24.89s/it]
 44%|████▍     | 98996/225000 [191:30:38<885:37:57, 25.30s/it]
 44%|████▍     | 98997/225000 [191:30:58<824:36:31, 23.56s/it]
 44%|████▍     | 98998/225000 [191:31:23<843:18:28, 24.09s/it]
 44%|████▍     | 98999/225000 [191:31:43<794:35:45, 22.70s/it]
 44%|████▍     | 99000/225000 [191:32:06<800:50:40, 22.88s/it]
                                                              
{'loss': '0.8972', 'grad_norm': '1.139', 'learning_rate': '3.169e-05', 'epoch': '0.44', 'num_input_tokens_seen': 317376000, 'train_runtime': '6.895e+05', 'train_tokens_per_second': '460.3'}

 44%|████▍     | 99000/225000 [191:32:06<800:50:40, 22.88s/it]
 44%|████▍     | 99001/225000 [191:32:27<785:09:23, 22.43s/it]
 44%|████▍     | 99002/225000 [191:32:49<783:47:05, 22.39s/it]
 44%|████▍     | 99003/225000 [191:33:11<775:55:58, 22.17s/it]
 44%|████▍     | 99004/225000 [191:33:40<847:54:16, 24.23s/it]
 44%|████▍     | 99005/225000 [191:34:03<835:31:56, 23.87s/it]
 44%|████▍     | 99006/225000 [191:34:22<778:26:16, 22.24s/it]
 44%|████▍     | 99007/225000 [191:34:51<853:45:00, 24.39s/it]
 44%|████▍     | 99008/225000 [191:35:07<763:05:28, 21.80s/it]
 44%|████▍     | 99009/225000 [191:35:33<805:16:48, 23.01s/it]
 44%|████▍     | 99010/225000 [191:36:02<876:24:03, 25.04s/it]
 44%|████▍     | 99011/225000 [191:36:23<829:59:47, 23.72s/it]
 44%|████▍     | 99012/225000 [191:36:47<835:01:57, 23.86s/it]
 44%|████▍     | 99013/225000 [191:37:12<843:31:47, 24.10s/it]
 44%|████▍     | 99014/225000 [191:37:29<773:53:38, 22.11s/it]
 44%|████▍     | 99015/225000 [191:37:54<798:37:27, 22.82s/it]
 44%|████▍     | 99016/225000 [191:38:14<773:08:15, 22.09s/it]
 44%|████▍     | 99017/225000 [191:38:39<799:28:23, 22.85s/it]
 44%|████▍     | 99018/225000 [191:39:10<888:56:25, 25.40s/it]
 44%|████▍     | 99019/225000 [191:39:38<918:28:09, 26.25s/it]
 44%|████▍     | 99020/225000 [191:40:06<930:11:22, 26.58s/it]
 44%|████▍     | 99021/225000 [191:40:32<929:36:41, 26.56s/it]
 44%|████▍     | 99022/225000 [191:40:51<843:34:51, 24.11s/it]
 44%|████▍     | 99023/225000 [191:41:13<824:30:58, 23.56s/it]
 44%|████▍     | 99024/225000 [191:41:38<840:05:51, 24.01s/it]
 44%|████▍     | 99025/225000 [191:42:02<836:17:42, 23.90s/it]
 44%|████▍     | 99026/225000 [191:42:35<933:48:26, 26.69s/it]
 44%|████▍     | 99027/225000 [191:42:58<891:57:08, 25.49s/it]
 44%|████▍     | 99028/225000 [191:43:21<874:36:25, 24.99s/it]
 44%|████▍     | 99029/225000 [191:43:42<832:14:57, 23.78s/it]
 44%|████▍     | 99030/225000 [191:44:08<850:32:20, 24.31s/it]
 44%|████▍     | 99031/225000 [191:44:32<851:21:36, 24.33s/it]
 44%|████▍     | 99032/225000 [191:44:54<826:24:29, 23.62s/it]
 44%|████▍     | 99033/225000 [191:45:25<897:09:12, 25.64s/it]
 44%|████▍     | 99034/225000 [191:45:45<844:52:55, 24.15s/it]
 44%|████▍     | 99035/225000 [191:46:08<834:54:56, 23.86s/it]
 44%|████▍     | 99036/225000 [191:46:38<891:35:49, 25.48s/it]
 44%|████▍     | 99037/225000 [191:47:04<903:16:45, 25.82s/it]
 44%|████▍     | 99038/225000 [191:47:24<842:43:38, 24.09s/it]
 44%|████▍     | 99039/225000 [191:47:45<807:30:54, 23.08s/it]
 44%|████▍     | 99040/225000 [191:48:13<856:56:59, 24.49s/it]
 44%|████▍     | 99041/225000 [191:48:41<899:18:01, 25.70s/it]
 44%|████▍     | 99042/225000 [191:49:08<904:06:52, 25.84s/it]
 44%|████▍     | 99043/225000 [191:49:27<840:40:29, 24.03s/it]
 44%|████▍     | 99044/225000 [191:49:53<862:05:47, 24.64s/it]
 44%|████▍     | 99045/225000 [191:50:17<855:37:09, 24.45s/it]
 44%|████▍     | 99046/225000 [191:50:36<789:45:05, 22.57s/it]
 44%|████▍     | 99047/225000 [191:51:05<860:17:15, 24.59s/it]
 44%|████▍     | 99048/225000 [191:51:33<892:36:44, 25.51s/it]
 44%|████▍     | 99049/225000 [191:51:54<848:21:21, 24.25s/it]
 44%|████▍     | 99050/225000 [191:52:08<745:39:08, 21.31s/it]
                                                              
{'loss': '0.8987', 'grad_norm': '1.195', 'learning_rate': '3.168e-05', 'epoch': '0.4402', 'num_input_tokens_seen': 318001096, 'train_runtime': '6.907e+05', 'train_tokens_per_second': '460.4'}

 44%|████▍     | 99050/225000 [191:52:08<745:39:08, 21.31s/it]
 44%|████▍     | 99051/225000 [191:52:33<780:52:48, 22.32s/it]
 44%|████▍     | 99052/225000 [191:52:55<778:50:39, 22.26s/it]
 44%|████▍     | 99053/225000 [191:53:19<798:44:01, 22.83s/it]
 44%|████▍     | 99054/225000 [191:53:47<848:37:49, 24.26s/it]
 44%|████▍     | 99055/225000 [191:54:09<826:24:17, 23.62s/it]
 44%|████▍     | 99056/225000 [191:54:39<897:25:55, 25.65s/it]
 44%|████▍     | 99057/225000 [191:54:57<808:10:17, 23.10s/it]
 44%|████▍     | 99058/225000 [191:55:25<864:15:48, 24.70s/it]
 44%|████▍     | 99059/225000 [191:55:41<777:29:19, 22.22s/it]
 44%|████▍     | 99060/225000 [191:56:12<860:27:26, 24.60s/it]
 44%|████▍     | 99061/225000 [191:56:39<889:45:17, 25.43s/it]
 44%|████▍     | 99062/225000 [191:57:01<854:46:42, 24.43s/it]
 44%|████▍     | 99063/225000 [191:57:23<829:37:43, 23.72s/it]
 44%|████▍     | 99064/225000 [191:57:49<856:57:24, 24.50s/it]
 44%|████▍     | 99065/225000 [191:58:08<790:55:33, 22.61s/it]
 44%|████▍     | 99066/225000 [191:58:39<885:59:53, 25.33s/it]
 44%|████▍     | 99067/225000 [191:59:11<952:05:55, 27.22s/it]
 44%|████▍     | 99068/225000 [191:59:31<882:00:09, 25.21s/it]
 44%|████▍     | 99069/225000 [191:59:52<828:11:30, 23.68s/it]
 44%|████▍     | 99070/225000 [192:00:17<842:53:17, 24.10s/it]
 44%|████▍     | 99071/225000 [192:00:44<877:24:41, 25.08s/it]
 44%|████▍     | 99072/225000 [192:01:08<863:29:59, 24.69s/it]
 44%|████▍     | 99073/225000 [192:01:22<758:21:40, 21.68s/it]
 44%|████▍     | 99074/225000 [192:01:51<832:06:52, 23.79s/it]
 44%|████▍     | 99075/225000 [192:02:07<748:55:35, 21.41s/it]
 44%|████▍     | 99076/225000 [192:02:28<746:31:23, 21.34s/it]
 44%|████▍     | 99077/225000 [192:02:56<818:54:20, 23.41s/it]
 44%|████▍     | 99078/225000 [192:03:09<702:14:09, 20.08s/it]
 44%|████▍     | 99079/225000 [192:03:35<764:10:47, 21.85s/it]
 44%|████▍     | 99080/225000 [192:04:12<921:13:09, 26.34s/it]
 44%|████▍     | 99081/225000 [192:04:35<892:22:30, 25.51s/it]
 44%|████▍     | 99082/225000 [192:04:59<871:46:23, 24.92s/it]
 44%|████▍     | 99083/225000 [192:05:33<965:12:45, 27.60s/it]
 44%|████▍     | 99084/225000 [192:05:54<902:27:36, 25.80s/it]
 44%|████▍     | 99085/225000 [192:06:22<921:19:42, 26.34s/it]
 44%|████▍     | 99086/225000 [192:06:43<872:39:16, 24.95s/it]
 44%|████▍     | 99087/225000 [192:07:12<907:12:23, 25.94s/it]
 44%|████▍     | 99088/225000 [192:07:35<883:29:37, 25.26s/it]
 44%|████▍     | 99089/225000 [192:08:04<922:00:51, 26.36s/it]
 44%|████▍     | 99090/225000 [192:08:25<859:29:34, 24.57s/it]
 44%|████▍     | 99091/225000 [192:08:46<828:34:18, 23.69s/it]
 44%|████▍     | 99092/225000 [192:09:09<822:20:26, 23.51s/it]
 44%|████▍     | 99093/225000 [192:09:38<874:41:52, 25.01s/it]
 44%|████▍     | 99094/225000 [192:09:55<793:10:36, 22.68s/it]
 44%|████▍     | 99095/225000 [192:10:23<851:57:16, 24.36s/it]
 44%|████▍     | 99096/225000 [192:10:47<847:43:19, 24.24s/it]
 44%|████▍     | 99097/225000 [192:11:10<828:32:58, 23.69s/it]
 44%|████▍     | 99098/225000 [192:11:33<827:40:09, 23.67s/it]
 44%|████▍     | 99099/225000 [192:11:55<804:44:22, 23.01s/it]
 44%|████▍     | 99100/225000 [192:12:19<820:36:17, 23.46s/it]
                                                              
{'loss': '0.8799', 'grad_norm': '0.9295', 'learning_rate': '3.166e-05', 'epoch': '0.4404', 'num_input_tokens_seen': 318623592, 'train_runtime': '6.919e+05', 'train_tokens_per_second': '460.5'}

 44%|████▍     | 99100/225000 [192:12:19<820:36:17, 23.46s/it]
 44%|████▍     | 99101/225000 [192:12:46<853:22:08, 24.40s/it]
 44%|████▍     | 99102/225000 [192:13:15<899:19:08, 25.72s/it]
 44%|████▍     | 99103/225000 [192:13:39<881:26:37, 25.20s/it]
 44%|████▍     | 99104/225000 [192:14:01<848:40:03, 24.27s/it]
 44%|████▍     | 99105/225000 [192:14:26<860:54:52, 24.62s/it]
 44%|████▍     | 99106/225000 [192:14:58<929:58:58, 26.59s/it]
 44%|████▍     | 99107/225000 [192:15:17<852:32:33, 24.38s/it]
 44%|████▍     | 99108/225000 [192:15:47<910:32:18, 26.04s/it]
 44%|████▍     | 99109/225000 [192:16:12<908:08:59, 25.97s/it]
 44%|████▍     | 99110/225000 [192:16:39<918:58:57, 26.28s/it]
 44%|████▍     | 99111/225000 [192:17:09<956:10:01, 27.34s/it]
 44%|████▍     | 99112/225000 [192:17:33<922:08:22, 26.37s/it]
 44%|████▍     | 99113/225000 [192:18:03<953:02:26, 27.25s/it]
 44%|████▍     | 99114/225000 [192:18:29<948:14:23, 27.12s/it]
 44%|████▍     | 99115/225000 [192:18:52<899:34:36, 25.73s/it]
 44%|████▍     | 99116/225000 [192:19:15<876:15:30, 25.06s/it]
 44%|████▍     | 99117/225000 [192:19:43<904:48:18, 25.88s/it]
 44%|████▍     | 99118/225000 [192:20:05<857:53:31, 24.53s/it]
 44%|████▍     | 99119/225000 [192:20:34<904:32:20, 25.87s/it]
 44%|████▍     | 99120/225000 [192:20:49<797:32:19, 22.81s/it]
 44%|████▍     | 99121/225000 [192:21:12<796:36:50, 22.78s/it]
 44%|████▍     | 99122/225000 [192:21:37<819:23:30, 23.43s/it]
 44%|████▍     | 99123/225000 [192:21:59<807:16:46, 23.09s/it]
 44%|████▍     | 99124/225000 [192:22:22<805:35:51, 23.04s/it]
 44%|████▍     | 99125/225000 [192:22:53<883:49:05, 25.28s/it]
 44%|████▍     | 99126/225000 [192:23:09<784:56:01, 22.45s/it]
 44%|████▍     | 99127/225000 [192:23:40<880:16:03, 25.18s/it]
 44%|████▍     | 99128/225000 [192:24:02<843:19:04, 24.12s/it]
 44%|████▍     | 99129/225000 [192:24:27<857:23:07, 24.52s/it]
 44%|████▍     | 99130/225000 [192:24:44<777:54:08, 22.25s/it]
 44%|████▍     | 99131/225000 [192:25:11<825:19:56, 23.61s/it]
 44%|████▍     | 99132/225000 [192:25:41<895:37:19, 25.62s/it]
 44%|████▍     | 99133/225000 [192:25:57<787:59:17, 22.54s/it]
 44%|████▍     | 99134/225000 [192:26:23<829:15:24, 23.72s/it]
 44%|████▍     | 99135/225000 [192:26:45<814:30:37, 23.30s/it]
 44%|████▍     | 99136/225000 [192:27:10<825:14:29, 23.60s/it]
 44%|████▍     | 99137/225000 [192:27:33<821:52:27, 23.51s/it]
 44%|████▍     | 99138/225000 [192:28:00<856:33:19, 24.50s/it]
 44%|████▍     | 99139/225000 [192:28:27<881:16:07, 25.21s/it]
 44%|████▍     | 99140/225000 [192:28:47<828:24:06, 23.69s/it]
 44%|████▍     | 99141/225000 [192:29:04<764:42:31, 21.87s/it]
 44%|████▍     | 99142/225000 [192:29:21<707:00:22, 20.22s/it]
 44%|████▍     | 99143/225000 [192:29:45<747:46:27, 21.39s/it]
 44%|████▍     | 99144/225000 [192:30:13<813:23:42, 23.27s/it]
 44%|████▍     | 99145/225000 [192:30:34<798:23:03, 22.84s/it]
 44%|████▍     | 99146/225000 [192:30:59<817:07:35, 23.37s/it]
 44%|████▍     | 99147/225000 [192:31:25<848:04:28, 24.26s/it]
 44%|████▍     | 99148/225000 [192:31:56<918:50:19, 26.28s/it]
 44%|████▍     | 99149/225000 [192:32:17<860:23:16, 24.61s/it]
 44%|████▍     | 99150/225000 [192:32:33<766:27:56, 21.93s/it]
                                                              
{'loss': '0.9137', 'grad_norm': '0.9726', 'learning_rate': '3.164e-05', 'epoch': '0.4407', 'num_input_tokens_seen': 319256104, 'train_runtime': '6.932e+05', 'train_tokens_per_second': '460.6'}

 44%|████▍     | 99150/225000 [192:32:33<766:27:56, 21.93s/it]
 44%|████▍     | 99151/225000 [192:32:52<741:55:30, 21.22s/it]
 44%|████▍     | 99152/225000 [192:33:17<781:17:10, 22.35s/it]
 44%|████▍     | 99153/225000 [192:33:52<910:21:00, 26.04s/it]
 44%|████▍     | 99154/225000 [192:34:18<915:32:42, 26.19s/it]
 44%|████▍     | 99155/225000 [192:34:44<903:21:20, 25.84s/it]
 44%|████▍     | 99156/225000 [192:34:59<791:48:32, 22.65s/it]
 44%|████▍     | 99157/225000 [192:35:24<819:02:46, 23.43s/it]
 44%|████▍     | 99158/225000 [192:35:48<829:31:03, 23.73s/it]
 44%|████▍     | 99159/225000 [192:36:13<837:38:40, 23.96s/it]
 44%|████▍     | 99160/225000 [192:36:43<900:43:29, 25.77s/it]
 44%|████▍     | 99161/225000 [192:37:12<931:09:02, 26.64s/it]
 44%|████▍     | 99162/225000 [192:37:36<911:46:32, 26.08s/it]
 44%|████▍     | 99163/225000 [192:37:54<820:02:36, 23.46s/it]
 44%|████▍     | 99164/225000 [192:38:22<873:19:10, 24.98s/it]
 44%|████▍     | 99165/225000 [192:38:46<864:36:35, 24.74s/it]
 44%|████▍     | 99166/225000 [192:39:14<890:47:14, 25.48s/it]
 44%|████▍     | 99167/225000 [192:39:32<817:51:38, 23.40s/it]
 44%|████▍     | 99168/225000 [192:40:01<872:22:50, 24.96s/it]
 44%|████▍     | 99169/225000 [192:40:27<889:59:46, 25.46s/it]
 44%|████▍     | 99170/225000 [192:40:51<870:58:33, 24.92s/it]
 44%|████▍     | 99171/225000 [192:41:12<826:39:58, 23.65s/it]
 44%|████▍     | 99172/225000 [192:41:34<814:04:48, 23.29s/it]
 44%|████▍     | 99173/225000 [192:41:50<736:52:06, 21.08s/it]
 44%|████▍     | 99174/225000 [192:42:11<731:11:15, 20.92s/it]
 44%|████▍     | 99175/225000 [192:42:30<717:19:20, 20.52s/it]
 44%|████▍     | 99176/225000 [192:43:02<839:38:55, 24.02s/it]
 44%|████▍     | 99177/225000 [192:43:19<761:47:01, 21.80s/it]
 44%|████▍     | 99178/225000 [192:43:41<768:46:24, 22.00s/it]
 44%|████▍     | 99179/225000 [192:44:06<792:58:47, 22.69s/it]
 44%|████▍     | 99180/225000 [192:44:25<758:26:10, 21.70s/it]
 44%|████▍     | 99181/225000 [192:44:48<774:37:43, 22.16s/it]
 44%|████▍     | 99182/225000 [192:45:15<823:01:32, 23.55s/it]
 44%|████▍     | 99183/225000 [192:45:45<893:01:44, 25.55s/it]
 44%|████▍     | 99184/225000 [192:46:13<910:24:12, 26.05s/it]
 44%|████▍     | 99185/225000 [192:46:36<884:43:39, 25.32s/it]
 44%|████▍     | 99186/225000 [192:47:01<873:45:14, 25.00s/it]
 44%|████▍     | 99187/225000 [192:47:20<819:31:20, 23.45s/it]
 44%|████▍     | 99188/225000 [192:47:41<787:03:28, 22.52s/it]
 44%|████▍     | 99189/225000 [192:48:10<863:09:09, 24.70s/it]
 44%|████▍     | 99190/225000 [192:48:33<835:20:18, 23.90s/it]
 44%|████▍     | 99191/225000 [192:49:01<886:59:42, 25.38s/it]
 44%|████▍     | 99192/225000 [192:49:20<813:55:09, 23.29s/it]
 44%|████▍     | 99193/225000 [192:49:51<901:46:05, 25.80s/it]
 44%|████▍     | 99194/225000 [192:50:17<896:40:04, 25.66s/it]
 44%|████▍     | 99195/225000 [192:50:37<838:06:41, 23.98s/it]
 44%|████▍     | 99196/225000 [192:51:03<859:22:36, 24.59s/it]
 44%|████▍     | 99197/225000 [192:51:21<790:55:01, 22.63s/it]
 44%|████▍     | 99198/225000 [192:51:46<812:32:21, 23.25s/it]
 44%|████▍     | 99199/225000 [192:52:06<781:39:39, 22.37s/it]
 44%|████▍     | 99200/225000 [192:52:36<861:07:49, 24.64s/it]
                                                              
{'loss': '0.906', 'grad_norm': '0.9757', 'learning_rate': '3.162e-05', 'epoch': '0.4409', 'num_input_tokens_seen': 319892944, 'train_runtime': '6.944e+05', 'train_tokens_per_second': '460.7'}

 44%|████▍     | 99200/225000 [192:52:36<861:07:49, 24.64s/it]
 44%|████▍     | 99201/225000 [192:52:53<787:14:53, 22.53s/it]
 44%|████▍     | 99202/225000 [192:53:21<836:59:08, 23.95s/it]
 44%|████▍     | 99203/225000 [192:53:45<837:06:09, 23.96s/it]
 44%|████▍     | 99204/225000 [192:54:15<903:56:03, 25.87s/it]
 44%|████▍     | 99205/225000 [192:54:34<833:47:10, 23.86s/it]
 44%|████▍     | 99206/225000 [192:54:54<792:29:09, 22.68s/it]
 44%|████▍     | 99207/225000 [192:55:14<765:34:03, 21.91s/it]
 44%|████▍     | 99208/225000 [192:55:32<727:10:56, 20.81s/it]
 44%|████▍     | 99209/225000 [192:56:04<837:40:54, 23.97s/it]
 44%|████▍     | 99210/225000 [192:56:24<801:17:05, 22.93s/it]
 44%|████▍     | 99211/225000 [192:56:43<754:58:01, 21.61s/it]
 44%|████▍     | 99212/225000 [192:57:03<738:18:01, 21.13s/it]
 44%|████▍     | 99213/225000 [192:57:24<740:56:46, 21.21s/it]
 44%|████▍     | 99214/225000 [192:57:45<733:25:34, 20.99s/it]
 44%|████▍     | 99215/225000 [192:58:12<794:52:29, 22.75s/it]
 44%|████▍     | 99216/225000 [192:58:41<860:28:10, 24.63s/it]
 44%|████▍     | 99217/225000 [192:59:03<835:47:23, 23.92s/it]
 44%|████▍     | 99218/225000 [192:59:29<853:55:43, 24.44s/it]
 44%|████▍     | 99219/225000 [192:59:54<863:37:13, 24.72s/it]
 44%|████▍     | 99220/225000 [193:00:15<827:25:42, 23.68s/it]
 44%|████▍     | 99221/225000 [193:00:38<821:00:37, 23.50s/it]
 44%|████▍     | 99222/225000 [193:01:05<851:07:18, 24.36s/it]
 44%|████▍     | 99223/225000 [193:01:30<862:12:59, 24.68s/it]
 44%|████▍     | 99224/225000 [193:01:43<737:23:34, 21.11s/it]
 44%|████▍     | 99225/225000 [193:02:07<773:39:00, 22.14s/it]
 44%|████▍     | 99226/225000 [193:02:24<719:31:46, 20.59s/it]
 44%|████▍     | 99227/225000 [193:02:48<750:23:17, 21.48s/it]
 44%|████▍     | 99228/225000 [193:03:04<696:57:41, 19.95s/it]
 44%|████▍     | 99229/225000 [193:03:30<757:36:37, 21.69s/it]
 44%|████▍     | 99230/225000 [193:03:48<717:20:04, 20.53s/it]
 44%|████▍     | 99231/225000 [193:04:13<764:59:17, 21.90s/it]
 44%|████▍     | 99232/225000 [193:04:41<832:41:16, 23.83s/it]
 44%|████▍     | 99233/225000 [193:05:05<829:04:23, 23.73s/it]
 44%|████▍     | 99234/225000 [193:05:28<826:56:52, 23.67s/it]
 44%|████▍     | 99235/225000 [193:05:57<879:19:46, 25.17s/it]
 44%|████▍     | 99236/225000 [193:06:19<848:31:07, 24.29s/it]
 44%|████▍     | 99237/225000 [193:06:44<856:11:55, 24.51s/it]
 44%|████▍     | 99238/225000 [193:07:10<871:38:52, 24.95s/it]
 44%|████▍     | 99239/225000 [193:07:39<907:09:12, 25.97s/it]
 44%|████▍     | 99240/225000 [193:08:03<896:16:16, 25.66s/it]
 44%|████▍     | 99241/225000 [193:08:31<917:21:14, 26.26s/it]
 44%|████▍     | 99242/225000 [193:08:57<917:57:44, 26.28s/it]
 44%|████▍     | 99243/225000 [193:09:18<857:35:58, 24.55s/it]
 44%|████▍     | 99244/225000 [193:09:39<816:14:29, 23.37s/it]
 44%|████▍     | 99245/225000 [193:09:59<789:22:28, 22.60s/it]
 44%|████▍     | 99246/225000 [193:10:19<756:59:09, 21.67s/it]
 44%|████▍     | 99247/225000 [193:10:47<829:16:16, 23.74s/it]
 44%|████▍     | 99248/225000 [193:11:11<830:41:58, 23.78s/it]
 44%|████▍     | 99249/225000 [193:11:42<907:13:06, 25.97s/it]
 44%|████▍     | 99250/225000 [193:12:07<887:25:54, 25.41s/it]
                                                              
{'loss': '0.8739', 'grad_norm': '1.1', 'learning_rate': '3.16e-05', 'epoch': '0.4411', 'num_input_tokens_seen': 320509224, 'train_runtime': '6.955e+05', 'train_tokens_per_second': '460.8'}

 44%|████▍     | 99250/225000 [193:12:07<887:25:54, 25.41s/it]
 44%|████▍     | 99251/225000 [193:12:34<914:16:25, 26.17s/it]
 44%|████▍     | 99252/225000 [193:12:56<866:48:33, 24.82s/it]
 44%|████▍     | 99253/225000 [193:13:29<949:37:32, 27.19s/it]
 44%|████▍     | 99254/225000 [193:13:52<907:17:18, 25.97s/it]
 44%|████▍     | 99255/225000 [193:14:13<859:18:47, 24.60s/it]
 44%|████▍     | 99256/225000 [193:14:32<800:55:32, 22.93s/it]
 44%|████▍     | 99257/225000 [193:15:05<902:55:05, 25.85s/it]
 44%|████▍     | 99258/225000 [193:15:23<818:33:14, 23.44s/it]
 44%|████▍     | 99259/225000 [193:15:45<803:52:27, 23.02s/it]
 44%|████▍     | 99260/225000 [193:16:13<859:50:17, 24.62s/it]
 44%|████▍     | 99261/225000 [193:16:33<811:32:41, 23.24s/it]
 44%|████▍     | 99262/225000 [193:16:59<836:48:05, 23.96s/it]
 44%|████▍     | 99263/225000 [193:17:15<759:15:11, 21.74s/it]
 44%|████▍     | 99264/225000 [193:17:42<811:49:25, 23.24s/it]
 44%|████▍     | 99265/225000 [193:18:09<843:30:34, 24.15s/it]
 44%|████▍     | 99266/225000 [193:18:30<813:22:19, 23.29s/it]
 44%|████▍     | 99267/225000 [193:18:56<849:08:31, 24.31s/it]
 44%|████▍     | 99268/225000 [193:19:18<818:43:40, 23.44s/it]
 44%|████▍     | 99269/225000 [193:19:44<843:34:13, 24.15s/it]
 44%|████▍     | 99270/225000 [193:20:04<805:39:32, 23.07s/it]
 44%|████▍     | 99271/225000 [193:20:32<852:11:56, 24.40s/it]
 44%|████▍     | 99272/225000 [193:21:03<928:31:21, 26.59s/it]
 44%|████▍     | 99273/225000 [193:21:29<914:53:29, 26.20s/it]
 44%|████▍     | 99274/225000 [193:21:54<908:12:01, 26.01s/it]
 44%|████▍     | 99275/225000 [193:22:13<827:52:17, 23.71s/it]
 44%|████▍     | 99276/225000 [193:22:37<829:41:57, 23.76s/it]
 44%|████▍     | 99277/225000 [193:22:57<795:57:57, 22.79s/it]
 44%|████▍     | 99278/225000 [193:23:21<810:06:46, 23.20s/it]
 44%|████▍     | 99279/225000 [193:23:49<853:52:24, 24.45s/it]
 44%|████▍     | 99280/225000 [193:24:13<855:17:58, 24.49s/it]
 44%|████▍     | 99281/225000 [193:24:36<833:54:19, 23.88s/it]
 44%|████▍     | 99282/225000 [193:25:04<884:55:51, 25.34s/it]
 44%|████▍     | 99283/225000 [193:25:27<855:00:01, 24.48s/it]
 44%|████▍     | 99284/225000 [193:25:53<871:07:13, 24.95s/it]
 44%|████▍     | 99285/225000 [193:26:14<828:36:45, 23.73s/it]
 44%|████▍     | 99286/225000 [193:26:29<739:23:26, 21.17s/it]
 44%|████▍     | 99287/225000 [193:26:56<797:29:51, 22.84s/it]
 44%|████▍     | 99288/225000 [193:27:20<814:35:10, 23.33s/it]
 44%|████▍     | 99289/225000 [193:27:38<753:25:12, 21.58s/it]
 44%|████▍     | 99290/225000 [193:28:06<826:03:56, 23.66s/it]
 44%|████▍     | 99291/225000 [193:28:31<838:19:16, 24.01s/it]
 44%|████▍     | 99292/225000 [193:28:54<831:46:06, 23.82s/it]
 44%|████▍     | 99293/225000 [193:29:19<835:24:27, 23.92s/it]
 44%|████▍     | 99294/225000 [193:29:45<862:12:52, 24.69s/it]
 44%|████▍     | 99295/225000 [193:30:06<824:38:47, 23.62s/it]
 44%|████▍     | 99296/225000 [193:30:34<870:51:03, 24.94s/it]
 44%|████▍     | 99297/225000 [193:30:58<863:31:33, 24.73s/it]
 44%|████▍     | 99298/225000 [193:31:24<877:40:30, 25.14s/it]
 44%|████▍     | 99299/225000 [193:31:55<936:46:38, 26.83s/it]
 44%|████▍     | 99300/225000 [193:32:15<858:37:55, 24.59s/it]
                                                              
{'loss': '0.8997', 'grad_norm': '1.117', 'learning_rate': '3.159e-05', 'epoch': '0.4413', 'num_input_tokens_seen': 321141048, 'train_runtime': '6.967e+05', 'train_tokens_per_second': '460.9'}

 44%|████▍     | 99300/225000 [193:32:15<858:37:55, 24.59s/it]
 44%|████▍     | 99301/225000 [193:32:32<787:37:05, 22.56s/it]
 44%|████▍     | 99302/225000 [193:32:56<801:13:58, 22.95s/it]
 44%|████▍     | 99303/225000 [193:33:18<791:11:23, 22.66s/it]
 44%|████▍     | 99304/225000 [193:33:45<831:23:33, 23.81s/it]
 44%|████▍     | 99305/225000 [193:34:04<785:09:36, 22.49s/it]
 44%|████▍     | 99306/225000 [193:34:31<833:33:14, 23.87s/it]
 44%|████▍     | 99307/225000 [193:35:00<887:58:00, 25.43s/it]
 44%|████▍     | 99308/225000 [193:35:30<929:22:27, 26.62s/it]
 44%|████▍     | 99309/225000 [193:35:59<957:35:48, 27.43s/it]
 44%|████▍     | 99310/225000 [193:36:23<919:21:40, 26.33s/it]
 44%|████▍     | 99311/225000 [193:36:54<975:09:12, 27.93s/it]
 44%|████▍     | 99312/225000 [193:37:14<882:46:50, 25.28s/it]
 44%|████▍     | 99313/225000 [193:37:43<926:16:10, 26.53s/it]
 44%|████▍     | 99314/225000 [193:38:07<903:38:00, 25.88s/it]
 44%|████▍     | 99315/225000 [193:38:27<832:31:26, 23.85s/it]
 44%|████▍     | 99316/225000 [193:38:55<881:25:25, 25.25s/it]
 44%|████▍     | 99317/225000 [193:39:21<883:53:10, 25.32s/it]
 44%|████▍     | 99318/225000 [193:39:42<844:05:46, 24.18s/it]
 44%|████▍     | 99319/225000 [193:40:07<852:41:44, 24.42s/it]
 44%|████▍     | 99320/225000 [193:40:36<895:08:04, 25.64s/it]
 44%|████▍     | 99321/225000 [193:40:57<848:01:53, 24.29s/it]
 44%|████▍     | 99322/225000 [193:41:24<877:30:18, 25.14s/it]
 44%|████▍     | 99323/225000 [193:41:45<833:32:29, 23.88s/it]
 44%|████▍     | 99324/225000 [193:42:17<920:35:27, 26.37s/it]
 44%|████▍     | 99325/225000 [193:42:39<872:50:57, 25.00s/it]
 44%|████▍     | 99326/225000 [193:43:07<910:47:18, 26.09s/it]
 44%|████▍     | 99327/225000 [193:43:28<850:22:03, 24.36s/it]
 44%|████▍     | 99328/225000 [193:43:49<820:17:49, 23.50s/it]
 44%|████▍     | 99329/225000 [193:44:17<865:35:50, 24.80s/it]
 44%|████▍     | 99330/225000 [193:44:45<896:54:48, 25.69s/it]
 44%|████▍     | 99331/225000 [193:45:13<923:32:55, 26.46s/it]
 44%|████▍     | 99332/225000 [193:45:31<835:30:24, 23.93s/it]
 44%|████▍     | 99333/225000 [193:46:00<886:50:38, 25.41s/it]
 44%|████▍     | 99334/225000 [193:46:21<846:45:46, 24.26s/it]
 44%|████▍     | 99335/225000 [193:46:44<828:15:52, 23.73s/it]
 44%|████▍     | 99336/225000 [193:47:07<821:28:59, 23.53s/it]
 44%|████▍     | 99337/225000 [193:47:31<830:31:03, 23.79s/it]
 44%|████▍     | 99338/225000 [193:47:57<845:26:34, 24.22s/it]
 44%|████▍     | 99339/225000 [193:48:22<860:48:53, 24.66s/it]
 44%|████▍     | 99340/225000 [193:48:46<850:28:09, 24.36s/it]
 44%|████▍     | 99341/225000 [193:49:15<903:33:19, 25.89s/it]
 44%|████▍     | 99342/225000 [193:49:46<957:02:42, 27.42s/it]
 44%|████▍     | 99343/225000 [193:50:06<872:24:10, 24.99s/it]
 44%|████▍     | 99344/225000 [193:50:30<868:44:04, 24.89s/it]
 44%|████▍     | 99345/225000 [193:50:54<858:35:23, 24.60s/it]
 44%|████▍     | 99346/225000 [193:51:19<860:40:02, 24.66s/it]
 44%|████▍     | 99347/225000 [193:51:41<835:51:12, 23.95s/it]
 44%|████▍     | 99348/225000 [193:52:10<882:41:45, 25.29s/it]
 44%|████▍     | 99349/225000 [193:52:35<879:37:11, 25.20s/it]
 44%|████▍     | 99350/225000 [193:53:01<891:28:47, 25.54s/it]
                                                              
{'loss': '0.8896', 'grad_norm': '1.023', 'learning_rate': '3.157e-05', 'epoch': '0.4416', 'num_input_tokens_seen': 321798008, 'train_runtime': '6.98e+05', 'train_tokens_per_second': '461'}

 44%|████▍     | 99350/225000 [193:53:01<891:28:47, 25.54s/it]
 44%|████▍     | 99351/225000 [193:53:30<921:29:45, 26.40s/it]
 44%|████▍     | 99352/225000 [193:53:57<934:39:41, 26.78s/it]
 44%|████▍     | 99353/225000 [193:54:23<926:09:55, 26.54s/it]
 44%|████▍     | 99354/225000 [193:54:47<901:50:24, 25.84s/it]
 44%|████▍     | 99355/225000 [193:55:02<787:16:42, 22.56s/it]
 44%|████▍     | 99356/225000 [193:55:28<817:58:40, 23.44s/it]
 44%|████▍     | 99357/225000 [193:55:59<899:14:34, 25.77s/it]
 44%|████▍     | 99358/225000 [193:56:18<830:35:30, 23.80s/it]
 44%|████▍     | 99359/225000 [193:56:44<849:59:25, 24.35s/it]
 44%|████▍     | 99360/225000 [193:57:07<840:41:21, 24.09s/it]
 44%|████▍     | 99361/225000 [193:57:36<885:17:35, 25.37s/it]
 44%|████▍     | 99362/225000 [193:58:08<953:14:44, 27.31s/it]
 44%|████▍     | 99363/225000 [193:58:32<918:36:14, 26.32s/it]
 44%|████▍     | 99364/225000 [193:58:51<850:18:26, 24.36s/it]
 44%|████▍     | 99365/225000 [193:59:11<804:13:21, 23.04s/it]
 44%|████▍     | 99366/225000 [193:59:42<881:53:28, 25.27s/it]
 44%|████▍     | 99367/225000 [194:00:05<857:14:15, 24.56s/it]
 44%|████▍     | 99368/225000 [194:00:31<878:45:24, 25.18s/it]
 44%|████▍     | 99369/225000 [194:00:51<824:26:36, 23.62s/it]
 44%|████▍     | 99370/225000 [194:01:12<795:18:16, 22.79s/it]
 44%|████▍     | 99371/225000 [194:01:42<874:23:45, 25.06s/it]
 44%|████▍     | 99372/225000 [194:02:12<918:10:42, 26.31s/it]
 44%|████▍     | 99373/225000 [194:02:35<889:02:06, 25.48s/it]
 44%|████▍     | 99374/225000 [194:03:04<926:42:19, 26.56s/it]
 44%|████▍     | 99375/225000 [194:03:26<875:50:20, 25.10s/it]
 44%|████▍     | 99376/225000 [194:03:50<869:03:37, 24.90s/it]
 44%|████▍     | 99377/225000 [194:04:20<915:02:04, 26.22s/it]
 44%|████▍     | 99378/225000 [194:04:48<937:40:14, 26.87s/it]
 44%|████▍     | 99379/225000 [194:05:12<907:28:10, 26.01s/it]
 44%|████▍     | 99380/225000 [194:05:36<882:43:01, 25.30s/it]
 44%|████▍     | 99381/225000 [194:06:01<882:28:57, 25.29s/it]
 44%|████▍     | 99382/225000 [194:06:26<876:38:26, 25.12s/it]
 44%|████▍     | 99383/225000 [194:06:42<788:21:46, 22.59s/it]
 44%|████▍     | 99384/225000 [194:07:12<859:31:42, 24.63s/it]
 44%|████▍     | 99385/225000 [194:07:38<871:46:06, 24.98s/it]
 44%|████▍     | 99386/225000 [194:08:01<853:51:40, 24.47s/it]
 44%|████▍     | 99387/225000 [194:08:24<833:56:35, 23.90s/it]
 44%|████▍     | 99388/225000 [194:08:46<818:01:50, 23.44s/it]
 44%|████▍     | 99389/225000 [194:09:08<799:54:32, 22.93s/it]
 44%|████▍     | 99390/225000 [194:09:26<755:18:15, 21.65s/it]
 44%|████▍     | 99391/225000 [194:09:54<822:15:36, 23.57s/it]
 44%|████▍     | 99392/225000 [194:10:21<851:29:05, 24.40s/it]
 44%|████▍     | 99393/225000 [194:10:47<869:13:01, 24.91s/it]
 44%|████▍     | 99394/225000 [194:11:13<882:30:43, 25.29s/it]
 44%|████▍     | 99395/225000 [194:11:33<824:58:15, 23.64s/it]
 44%|████▍     | 99396/225000 [194:11:54<800:54:00, 22.96s/it]
 44%|████▍     | 99397/225000 [194:12:17<796:00:02, 22.81s/it]
 44%|████▍     | 99398/225000 [194:12:36<755:31:08, 21.65s/it]
 44%|████▍     | 99399/225000 [194:12:51<685:53:03, 19.66s/it]
 44%|████▍     | 99400/225000 [194:13:16<748:09:37, 21.44s/it]
                                                              
{'loss': '0.9435', 'grad_norm': '0.8443', 'learning_rate': '3.155e-05', 'epoch': '0.4418', 'num_input_tokens_seen': 322431632, 'train_runtime': '6.992e+05', 'train_tokens_per_second': '461.1'}

 44%|████▍     | 99400/225000 [194:13:16<748:09:37, 21.44s/it]
 44%|████▍     | 99401/225000 [194:13:42<795:37:28, 22.80s/it]
 44%|████▍     | 99402/225000 [194:14:00<746:19:13, 21.39s/it]
 44%|████▍     | 99403/225000 [194:14:22<748:45:33, 21.46s/it]
 44%|████▍     | 99404/225000 [194:14:48<799:01:19, 22.90s/it]
 44%|████▍     | 99405/225000 [194:15:07<758:58:42, 21.76s/it]
 44%|████▍     | 99406/225000 [194:15:42<896:11:24, 25.69s/it]
 44%|████▍     | 99407/225000 [194:15:59<806:48:23, 23.13s/it]
 44%|████▍     | 99408/225000 [194:16:20<780:03:04, 22.36s/it]
 44%|████▍     | 99409/225000 [194:16:47<827:05:28, 23.71s/it]
 44%|████▍     | 99410/225000 [194:17:09<808:21:57, 23.17s/it]
 44%|████▍     | 99411/225000 [194:17:38<876:54:26, 25.14s/it]
 44%|████▍     | 99412/225000 [194:18:02<865:42:59, 24.82s/it]
 44%|████▍     | 99413/225000 [194:18:22<812:52:53, 23.30s/it]
 44%|████▍     | 99414/225000 [194:18:52<876:47:54, 25.13s/it]
 44%|████▍     | 99415/225000 [194:19:16<873:20:20, 25.03s/it]
 44%|████▍     | 99416/225000 [194:19:40<859:58:15, 24.65s/it]
 44%|████▍     | 99417/225000 [194:20:04<847:28:02, 24.29s/it]
 44%|████▍     | 99418/225000 [194:20:25<816:05:23, 23.39s/it]
 44%|████▍     | 99419/225000 [194:20:45<783:17:15, 22.45s/it]
 44%|████▍     | 99420/225000 [194:21:01<709:44:45, 20.35s/it]
 44%|████▍     | 99421/225000 [194:21:24<740:24:54, 21.23s/it]
 44%|████▍     | 99422/225000 [194:21:48<767:54:09, 22.01s/it]
 44%|████▍     | 99423/225000 [194:22:07<736:58:38, 21.13s/it]
 44%|████▍     | 99424/225000 [194:22:34<803:10:34, 23.03s/it]
 44%|████▍     | 99425/225000 [194:23:00<829:22:31, 23.78s/it]
 44%|████▍     | 99426/225000 [194:23:26<853:19:57, 24.46s/it]
 44%|████▍     | 99427/225000 [194:23:57<920:08:26, 26.38s/it]
 44%|████▍     | 99428/225000 [194:24:21<901:39:52, 25.85s/it]
 44%|████▍     | 99429/225000 [194:24:44<870:58:03, 24.97s/it]
 44%|████▍     | 99430/225000 [194:25:15<929:40:03, 26.65s/it]
 44%|████▍     | 99431/225000 [194:25:37<880:29:11, 25.24s/it]
 44%|████▍     | 99432/225000 [194:25:58<834:47:55, 23.93s/it]
 44%|████▍     | 99433/225000 [194:26:14<761:30:58, 21.83s/it]
 44%|████▍     | 99434/225000 [194:26:40<801:14:34, 22.97s/it]
 44%|████▍     | 99435/225000 [194:27:07<843:25:11, 24.18s/it]
 44%|████▍     | 99436/225000 [194:27:36<890:02:45, 25.52s/it]
 44%|████▍     | 99437/225000 [194:28:01<891:02:06, 25.55s/it]
 44%|████▍     | 99438/225000 [194:28:23<853:31:10, 24.47s/it]
 44%|████▍     | 99439/225000 [194:28:44<817:11:52, 23.43s/it]
 44%|████▍     | 99440/225000 [194:29:09<833:40:17, 23.90s/it]
 44%|████▍     | 99441/225000 [194:29:34<836:25:13, 23.98s/it]
 44%|████▍     | 99442/225000 [194:29:55<810:56:55, 23.25s/it]
 44%|████▍     | 99443/225000 [194:30:23<858:27:39, 24.61s/it]
 44%|████▍     | 99444/225000 [194:30:44<822:14:26, 23.58s/it]
 44%|████▍     | 99445/225000 [194:31:16<906:24:23, 25.99s/it]
 44%|████▍     | 99446/225000 [194:31:42<910:17:49, 26.10s/it]
 44%|████▍     | 99447/225000 [194:32:06<885:04:35, 25.38s/it]
 44%|████▍     | 99448/225000 [194:32:29<867:07:03, 24.86s/it]
 44%|████▍     | 99449/225000 [194:32:48<802:20:12, 23.01s/it]
 44%|████▍     | 99450/225000 [194:33:19<881:52:06, 25.29s/it]
                                                              
{'loss': '0.864', 'grad_norm': '0.924', 'learning_rate': '3.153e-05', 'epoch': '0.442', 'num_input_tokens_seen': 323066488, 'train_runtime': '7.004e+05', 'train_tokens_per_second': '461.3'}

 44%|████▍     | 99450/225000 [194:33:19<881:52:06, 25.29s/it]
 44%|████▍     | 99451/225000 [194:33:45<889:49:46, 25.52s/it]
 44%|████▍     | 99452/225000 [194:34:08<863:34:46, 24.76s/it]
 44%|████▍     | 99453/225000 [194:34:41<950:39:16, 27.26s/it]
 44%|████▍     | 99454/225000 [194:35:01<881:44:06, 25.28s/it]
 44%|████▍     | 99455/225000 [194:35:29<901:43:20, 25.86s/it]
 44%|████▍     | 99456/225000 [194:35:53<881:58:54, 25.29s/it]
 44%|████▍     | 99457/225000 [194:36:17<873:39:41, 25.05s/it]
 44%|████▍     | 99458/225000 [194:36:46<910:14:31, 26.10s/it]
 44%|████▍     | 99459/225000 [194:37:10<892:48:37, 25.60s/it]
 44%|████▍     | 99460/225000 [194:37:37<905:57:56, 25.98s/it]
 44%|████▍     | 99461/225000 [194:37:56<836:12:09, 23.98s/it]
 44%|████▍     | 99462/225000 [194:38:22<859:24:28, 24.64s/it]
 44%|████▍     | 99463/225000 [194:38:47<855:29:03, 24.53s/it]
 44%|████▍     | 99464/225000 [194:39:10<843:34:37, 24.19s/it]
 44%|████▍     | 99465/225000 [194:39:34<835:40:25, 23.96s/it]
 44%|████▍     | 99466/225000 [194:39:59<849:58:28, 24.38s/it]
 44%|████▍     | 99467/225000 [194:40:29<906:57:35, 26.01s/it]
 44%|████▍     | 99468/225000 [194:40:55<912:46:02, 26.18s/it]
 44%|████▍     | 99469/225000 [194:41:13<820:48:49, 23.54s/it]
 44%|████▍     | 99470/225000 [194:41:44<897:15:25, 25.73s/it]
 44%|████▍     | 99471/225000 [194:42:13<931:36:49, 26.72s/it]
 44%|████▍     | 99472/225000 [194:42:34<874:31:40, 25.08s/it]
 44%|████▍     | 99473/225000 [194:42:53<814:17:54, 23.35s/it]
 44%|████▍     | 99474/225000 [194:43:22<872:33:25, 25.02s/it]
 44%|████▍     | 99475/225000 [194:43:39<783:16:58, 22.46s/it]
 44%|████▍     | 99476/225000 [194:44:08<857:20:11, 24.59s/it]
 44%|████▍     | 99477/225000 [194:44:34<876:31:23, 25.14s/it]
 44%|████▍     | 99478/225000 [194:45:01<888:37:00, 25.49s/it]
 44%|████▍     | 99479/225000 [194:45:18<800:59:50, 22.97s/it]
 44%|████▍     | 99480/225000 [194:45:47<863:05:29, 24.75s/it]
 44%|████▍     | 99481/225000 [194:46:10<846:01:35, 24.26s/it]
 44%|████▍     | 99482/225000 [194:46:31<816:05:40, 23.41s/it]
 44%|████▍     | 99483/225000 [194:46:50<769:36:13, 22.07s/it]
 44%|████▍     | 99484/225000 [194:47:14<788:00:25, 22.60s/it]
 44%|████▍     | 99485/225000 [194:47:38<802:51:03, 23.03s/it]
 44%|████▍     | 99486/225000 [194:48:06<849:08:41, 24.36s/it]
 44%|████▍     | 99487/225000 [194:48:33<882:33:28, 25.31s/it]
 44%|████▍     | 99488/225000 [194:48:57<862:45:36, 24.75s/it]
 44%|████▍     | 99489/225000 [194:49:25<896:18:26, 25.71s/it]
 44%|████▍     | 99490/225000 [194:49:49<882:08:03, 25.30s/it]
 44%|████▍     | 99491/225000 [194:50:17<910:48:11, 26.12s/it]
 44%|████▍     | 99492/225000 [194:50:43<911:49:39, 26.15s/it]
 44%|████▍     | 99493/225000 [194:51:02<833:37:25, 23.91s/it]
 44%|████▍     | 99494/225000 [194:51:17<744:58:25, 21.37s/it]
 44%|████▍     | 99495/225000 [194:51:41<774:20:06, 22.21s/it]
 44%|████▍     | 99496/225000 [194:52:07<808:24:10, 23.19s/it]
 44%|████▍     | 99497/225000 [194:52:28<782:49:27, 22.45s/it]
 44%|████▍     | 99498/225000 [194:52:51<795:38:45, 22.82s/it]
 44%|████▍     | 99499/225000 [194:53:14<792:21:47, 22.73s/it]
 44%|████▍     | 99500/225000 [194:53:39<817:20:35, 23.45s/it]
                                                              
{'loss': '0.8856', 'grad_norm': '1.456', 'learning_rate': '3.152e-05', 'epoch': '0.4422', 'num_input_tokens_seen': 323710416, 'train_runtime': '7.016e+05', 'train_tokens_per_second': '461.4'}

 44%|████▍     | 99500/225000 [194:53:39<817:20:35, 23.45s/it]
 44%|████▍     | 99501/225000 [194:54:03<823:04:54, 23.61s/it]
 44%|████▍     | 99502/225000 [194:54:30<854:08:20, 24.50s/it]
 44%|████▍     | 99503/225000 [194:54:56<879:30:32, 25.23s/it]
 44%|████▍     | 99504/225000 [194:55:22<881:20:49, 25.28s/it]
 44%|████▍     | 99505/225000 [194:55:38<783:21:11, 22.47s/it]
 44%|████▍     | 99506/225000 [194:56:03<816:09:14, 23.41s/it]
 44%|████▍     | 99507/225000 [194:56:29<843:35:18, 24.20s/it]
 44%|████▍     | 99508/225000 [194:56:46<761:58:34, 21.86s/it]
 44%|████▍     | 99509/225000 [194:57:15<842:00:55, 24.16s/it]
 44%|████▍     | 99510/225000 [194:57:42<873:00:52, 25.04s/it]
 44%|████▍     | 99511/225000 [194:58:08<879:12:32, 25.22s/it]
 44%|████▍     | 99512/225000 [194:58:27<816:04:36, 23.41s/it]
 44%|████▍     | 99513/225000 [194:58:59<905:50:43, 25.99s/it]
 44%|████▍     | 99514/225000 [194:59:23<882:48:26, 25.33s/it]
 44%|████▍     | 99515/225000 [194:59:49<887:26:39, 25.46s/it]
 44%|████▍     | 99516/225000 [195:00:08<818:03:55, 23.47s/it]
 44%|████▍     | 99517/225000 [195:00:40<907:38:29, 26.04s/it]
 44%|████▍     | 99518/225000 [195:01:02<864:46:27, 24.81s/it]
 44%|████▍     | 99519/225000 [195:01:26<858:31:02, 24.63s/it]
 44%|████▍     | 99520/225000 [195:01:45<802:09:34, 23.01s/it]
 44%|████▍     | 99521/225000 [195:02:19<912:07:30, 26.17s/it]
 44%|████▍     | 99522/225000 [195:02:45<911:08:15, 26.14s/it]
 44%|████▍     | 99523/225000 [195:03:13<938:25:56, 26.92s/it]
 44%|████▍     | 99524/225000 [195:03:37<899:21:13, 25.80s/it]
 44%|████▍     | 99525/225000 [195:04:00<868:47:07, 24.93s/it]
 44%|████▍     | 99526/225000 [195:04:22<839:20:15, 24.08s/it]
 44%|████▍     | 99527/225000 [195:04:48<863:56:28, 24.79s/it]
 44%|████▍     | 99528/225000 [195:05:11<848:52:22, 24.36s/it]
 44%|████▍     | 99529/225000 [195:05:36<854:13:59, 24.51s/it]
 44%|████▍     | 99530/225000 [195:06:05<902:37:48, 25.90s/it]
 44%|████▍     | 99531/225000 [195:06:26<847:32:36, 24.32s/it]
 44%|████▍     | 99532/225000 [195:06:56<902:55:31, 25.91s/it]
 44%|████▍     | 99533/225000 [195:07:16<843:50:53, 24.21s/it]
 44%|████▍     | 99534/225000 [195:07:45<891:59:59, 25.59s/it]
 44%|████▍     | 99535/225000 [195:08:06<842:36:20, 24.18s/it]
 44%|████▍     | 99536/225000 [195:08:31<855:47:39, 24.56s/it]
 44%|████▍     | 99537/225000 [195:08:55<848:05:06, 24.33s/it]
 44%|████▍     | 99538/225000 [195:09:18<836:26:58, 24.00s/it]
 44%|████▍     | 99539/225000 [195:09:46<880:37:14, 25.27s/it]
 44%|████▍     | 99540/225000 [195:10:12<880:48:22, 25.27s/it]
 44%|████▍     | 99541/225000 [195:10:38<896:57:12, 25.74s/it]
 44%|████▍     | 99542/225000 [195:10:58<828:59:05, 23.79s/it]
 44%|████▍     | 99543/225000 [195:11:27<886:29:20, 25.44s/it]
 44%|████▍     | 99544/225000 [195:11:55<917:43:31, 26.33s/it]
 44%|████▍     | 99545/225000 [195:12:19<888:56:30, 25.51s/it]
 44%|████▍     | 99546/225000 [195:12:50<943:34:44, 27.08s/it]
 44%|████▍     | 99547/225000 [195:13:16<935:30:53, 26.85s/it]
 44%|████▍     | 99548/225000 [195:13:37<873:32:09, 25.07s/it]
 44%|████▍     | 99549/225000 [195:14:06<912:00:12, 26.17s/it]
 44%|████▍     | 99550/225000 [195:14:26<849:24:50, 24.38s/it]
                                                              
{'loss': '0.8859', 'grad_norm': '0.9822', 'learning_rate': '3.15e-05', 'epoch': '0.4424', 'num_input_tokens_seen': 324360352, 'train_runtime': '7.029e+05', 'train_tokens_per_second': '461.5'}

 44%|████▍     | 99550/225000 [195:14:26<849:24:50, 24.38s/it]
 44%|████▍     | 99551/225000 [195:14:46<809:58:57, 23.24s/it]
 44%|████▍     | 99552/225000 [195:15:14<857:11:26, 24.60s/it]
 44%|████▍     | 99553/225000 [195:15:40<864:28:00, 24.81s/it]
 44%|████▍     | 99554/225000 [195:15:59<811:41:42, 23.29s/it]
 44%|████▍     | 99555/225000 [195:16:24<823:05:39, 23.62s/it]
 44%|████▍     | 99556/225000 [195:16:54<895:27:27, 25.70s/it]
 44%|████▍     | 99557/225000 [195:17:19<885:25:37, 25.41s/it]
 44%|████▍     | 99558/225000 [195:17:38<823:14:05, 23.63s/it]
 44%|████▍     | 99559/225000 [195:17:59<790:04:36, 22.67s/it]
 44%|████▍     | 99560/225000 [195:18:29<868:39:36, 24.93s/it]
 44%|████▍     | 99561/225000 [195:18:49<814:02:11, 23.36s/it]
 44%|████▍     | 99562/225000 [195:19:13<820:25:39, 23.55s/it]
 44%|████▍     | 99563/225000 [195:19:37<828:53:37, 23.79s/it]
 44%|████▍     | 99564/225000 [195:19:54<754:05:49, 21.64s/it]
 44%|████▍     | 99565/225000 [195:20:16<763:36:25, 21.92s/it]
 44%|████▍     | 99566/225000 [195:20:39<777:02:26, 22.30s/it]
 44%|████▍     | 99567/225000 [195:21:06<817:36:51, 23.47s/it]
 44%|████▍     | 99568/225000 [195:21:32<846:33:18, 24.30s/it]
 44%|████▍     | 99569/225000 [195:21:59<875:31:31, 25.13s/it]
 44%|████▍     | 99570/225000 [195:22:22<853:36:02, 24.50s/it]
 44%|████▍     | 99571/225000 [195:22:45<841:50:12, 24.16s/it]
 44%|████▍     | 99572/225000 [195:23:07<815:16:07, 23.40s/it]
 44%|████▍     | 99573/225000 [195:23:35<864:04:12, 24.80s/it]
 44%|████▍     | 99574/225000 [195:23:57<838:06:53, 24.06s/it]
 44%|████▍     | 99575/225000 [195:24:21<830:49:17, 23.85s/it]
 44%|████▍     | 99576/225000 [195:24:45<837:29:39, 24.04s/it]
 44%|████▍     | 99577/225000 [195:25:14<885:10:57, 25.41s/it]
 44%|████▍     | 99578/225000 [195:25:36<854:37:35, 24.53s/it]
 44%|████▍     | 99579/225000 [195:26:00<844:20:37, 24.24s/it]
 44%|████▍     | 99580/225000 [195:26:27<876:15:34, 25.15s/it]
 44%|████▍     | 99581/225000 [195:26:46<812:33:08, 23.32s/it]
 44%|████▍     | 99582/225000 [195:27:09<809:43:54, 23.24s/it]
 44%|████▍     | 99583/225000 [195:27:42<913:27:26, 26.22s/it]
 44%|████▍     | 99584/225000 [195:28:06<889:25:20, 25.53s/it]
 44%|████▍     | 99585/225000 [195:28:29<858:18:52, 24.64s/it]
 44%|████▍     | 99586/225000 [195:28:50<823:22:03, 23.63s/it]
 44%|████▍     | 99587/225000 [195:29:16<843:31:28, 24.21s/it]
 44%|████▍     | 99588/225000 [195:29:39<834:52:41, 23.97s/it]
 44%|████▍     | 99589/225000 [195:30:04<847:12:28, 24.32s/it]
 44%|████▍     | 99590/225000 [195:30:28<841:45:55, 24.16s/it]
 44%|████▍     | 99591/225000 [195:30:55<869:07:07, 24.95s/it]
 44%|████▍     | 99592/225000 [195:31:26<935:43:31, 26.86s/it]
 44%|████▍     | 99593/225000 [195:31:49<897:43:25, 25.77s/it]
 44%|████▍     | 99594/225000 [195:32:13<870:16:26, 24.98s/it]
 44%|████▍     | 99595/225000 [195:32:46<958:28:05, 27.51s/it]
 44%|████▍     | 99596/225000 [195:33:05<870:14:05, 24.98s/it]
 44%|████▍     | 99597/225000 [195:33:29<863:34:53, 24.79s/it]
 44%|████▍     | 99598/225000 [195:33:50<824:16:55, 23.66s/it]
 44%|████▍     | 99599/225000 [195:34:16<848:25:53, 24.36s/it]
 44%|████▍     | 99600/225000 [195:34:46<905:44:33, 26.00s/it]
                                                              
{'loss': '0.902', 'grad_norm': '0.9215', 'learning_rate': '3.148e-05', 'epoch': '0.4427', 'num_input_tokens_seen': 324990944, 'train_runtime': '7.041e+05', 'train_tokens_per_second': '461.6'}

 44%|████▍     | 99600/225000 [195:34:46<905:44:33, 26.00s/it]
 44%|████▍     | 99601/225000 [195:35:06<845:03:11, 24.26s/it]
 44%|████▍     | 99602/225000 [195:35:35<885:21:32, 25.42s/it]
 44%|████▍     | 99603/225000 [195:35:58<867:45:36, 24.91s/it]
 44%|████▍     | 99604/225000 [195:36:28<918:47:59, 26.38s/it]
 44%|████▍     | 99605/225000 [195:36:58<953:29:14, 27.37s/it]
 44%|████▍     | 99606/225000 [195:37:18<878:04:57, 25.21s/it]
 44%|████▍     | 99607/225000 [195:37:45<898:14:10, 25.79s/it]
 44%|████▍     | 99608/225000 [195:38:09<881:29:00, 25.31s/it]
 44%|████▍     | 99609/225000 [195:38:37<901:29:08, 25.88s/it]
 44%|████▍     | 99610/225000 [195:39:08<955:49:33, 27.44s/it]
 44%|████▍     | 99611/225000 [195:39:33<933:58:26, 26.82s/it]
 44%|████▍     | 99612/225000 [195:39:59<930:59:41, 26.73s/it]
 44%|████▍     | 99613/225000 [195:40:23<899:34:49, 25.83s/it]
 44%|████▍     | 99614/225000 [195:40:54<956:20:08, 27.46s/it]
 44%|████▍     | 99615/225000 [195:41:17<900:51:19, 25.86s/it]
 44%|████▍     | 99616/225000 [195:41:41<883:13:39, 25.36s/it]
 44%|████▍     | 99617/225000 [195:41:54<754:15:24, 21.66s/it]
 44%|████▍     | 99618/225000 [195:42:28<880:41:40, 25.29s/it]
 44%|████▍     | 99619/225000 [195:42:52<875:26:50, 25.14s/it]
 44%|████▍     | 99620/225000 [195:43:13<830:04:04, 23.83s/it]
 44%|████▍     | 99621/225000 [195:43:36<822:22:02, 23.61s/it]
 44%|████▍     | 99622/225000 [195:44:10<924:27:04, 26.54s/it]
 44%|████▍     | 99623/225000 [195:44:35<913:02:30, 26.22s/it]
 44%|████▍     | 99624/225000 [195:44:57<868:26:46, 24.94s/it]
 44%|████▍     | 99625/225000 [195:45:21<855:31:19, 24.57s/it]
 44%|████▍     | 99626/225000 [195:45:37<771:36:29, 22.16s/it]
 44%|████▍     | 99627/225000 [195:46:02<801:45:47, 23.02s/it]
 44%|████▍     | 99628/225000 [195:46:34<890:51:10, 25.58s/it]
 44%|████▍     | 99629/225000 [195:47:01<904:29:20, 25.97s/it]
 44%|████▍     | 99630/225000 [195:47:32<955:32:58, 27.44s/it]
 44%|████▍     | 99631/225000 [195:47:55<913:48:28, 26.24s/it]
 44%|████▍     | 99632/225000 [195:48:22<923:18:42, 26.51s/it]
 44%|████▍     | 99633/225000 [195:48:49<928:49:59, 26.67s/it]
 44%|████▍     | 99634/225000 [195:49:16<925:09:44, 26.57s/it]
 44%|████▍     | 99635/225000 [195:49:43<937:18:48, 26.92s/it]
 44%|████▍     | 99636/225000 [195:50:04<876:10:38, 25.16s/it]
 44%|████▍     | 99637/225000 [195:50:19<768:18:50, 22.06s/it]
 44%|████▍     | 99638/225000 [195:50:51<871:26:32, 25.03s/it]
 44%|████▍     | 99639/225000 [195:51:14<849:04:06, 24.38s/it]
 44%|████▍     | 99640/225000 [195:51:39<858:32:30, 24.65s/it]
 44%|████▍     | 99641/225000 [195:52:04<862:11:47, 24.76s/it]
 44%|████▍     | 99642/225000 [195:52:27<842:17:04, 24.19s/it]
 44%|████▍     | 99643/225000 [195:52:52<854:14:07, 24.53s/it]
 44%|████▍     | 99644/225000 [195:53:22<905:41:24, 26.01s/it]
 44%|████▍     | 99645/225000 [195:53:44<864:41:38, 24.83s/it]
 44%|████▍     | 99646/225000 [195:54:15<924:31:13, 26.55s/it]
 44%|████▍     | 99647/225000 [195:54:37<882:05:22, 25.33s/it]
 44%|████▍     | 99648/225000 [195:55:01<862:33:37, 24.77s/it]
 44%|████▍     | 99649/225000 [195:55:24<847:19:49, 24.33s/it]
 44%|████▍     | 99650/225000 [195:55:51<875:45:29, 25.15s/it]
                                                              
{'loss': '0.8751', 'grad_norm': '0.9828', 'learning_rate': '3.146e-05', 'epoch': '0.4429', 'num_input_tokens_seen': 325650040, 'train_runtime': '7.054e+05', 'train_tokens_per_second': '461.7'}

 44%|████▍     | 99650/225000 [195:55:51<875:45:29, 25.15s/it]
 44%|████▍     | 99651/225000 [195:56:10<812:22:04, 23.33s/it]
 44%|████▍     | 99652/225000 [195:56:31<789:55:54, 22.69s/it]
 44%|████▍     | 99653/225000 [195:56:58<828:59:36, 23.81s/it]
 44%|████▍     | 99654/225000 [195:57:26<872:07:57, 25.05s/it]
 44%|████▍     | 99655/225000 [195:57:48<843:38:13, 24.23s/it]
 44%|████▍     | 99656/225000 [195:58:17<897:49:01, 25.79s/it]
 44%|████▍     | 99657/225000 [195:58:42<883:41:04, 25.38s/it]
 44%|████▍     | 99658/225000 [195:59:07<877:25:35, 25.20s/it]
 44%|████▍     | 99659/225000 [195:59:27<824:57:14, 23.69s/it]
 44%|████▍     | 99660/225000 [195:59:53<847:18:37, 24.34s/it]
 44%|████▍     | 99661/225000 [196:00:18<859:23:25, 24.68s/it]
 44%|████▍     | 99662/225000 [196:00:37<800:20:48, 22.99s/it]
 44%|████▍     | 99663/225000 [196:01:10<905:03:02, 26.00s/it]
 44%|████▍     | 99664/225000 [196:01:27<814:48:11, 23.40s/it]
 44%|████▍     | 99665/225000 [196:01:52<823:50:41, 23.66s/it]
 44%|████▍     | 99666/225000 [196:02:12<785:11:14, 22.55s/it]
 44%|████▍     | 99667/225000 [196:02:41<860:34:22, 24.72s/it]
 44%|████▍     | 99668/225000 [196:03:00<791:21:26, 22.73s/it]
 44%|████▍     | 99669/225000 [196:03:32<893:15:48, 25.66s/it]
 44%|████▍     | 99670/225000 [196:03:53<843:17:37, 24.22s/it]
 44%|████▍     | 99671/225000 [196:04:16<829:00:09, 23.81s/it]
 44%|████▍     | 99672/225000 [196:04:42<851:42:16, 24.46s/it]
 44%|████▍     | 99673/225000 [196:04:59<775:56:31, 22.29s/it]
 44%|████▍     | 99674/225000 [196:05:31<876:37:10, 25.18s/it]
 44%|████▍     | 99675/225000 [196:06:00<914:54:01, 26.28s/it]
 44%|████▍     | 99676/225000 [196:06:19<845:54:26, 24.30s/it]
 44%|████▍     | 99677/225000 [196:06:44<852:20:50, 24.48s/it]
 44%|████▍     | 99678/225000 [196:07:15<913:42:39, 26.25s/it]
 44%|████▍     | 99679/225000 [196:07:42<929:40:29, 26.71s/it]
 44%|████▍     | 99680/225000 [196:08:02<854:14:41, 24.54s/it]
 44%|████▍     | 99681/225000 [196:08:30<896:12:30, 25.75s/it]
 44%|████▍     | 99682/225000 [196:08:59<920:27:58, 26.44s/it]
 44%|████▍     | 99683/225000 [196:09:25<921:14:11, 26.46s/it]
 44%|████▍     | 99684/225000 [196:09:50<901:53:46, 25.91s/it]
 44%|████▍     | 99685/225000 [196:10:16<901:42:51, 25.90s/it]
 44%|████▍     | 99686/225000 [196:10:40<882:48:47, 25.36s/it]
 44%|████▍     | 99687/225000 [196:11:03<862:10:03, 24.77s/it]
 44%|████▍     | 99688/225000 [196:11:25<831:03:18, 23.87s/it]
 44%|████▍     | 99689/225000 [196:11:50<840:16:05, 24.14s/it]
 44%|████▍     | 99690/225000 [196:12:16<865:56:44, 24.88s/it]
 44%|████▍     | 99691/225000 [196:12:45<906:27:57, 26.04s/it]
 44%|████▍     | 99692/225000 [196:13:10<898:10:44, 25.80s/it]
 44%|████▍     | 99693/225000 [196:13:39<928:13:24, 26.67s/it]
 44%|████▍     | 99694/225000 [196:14:10<970:17:36, 27.88s/it]
 44%|████▍     | 99695/225000 [196:14:25<841:58:55, 24.19s/it]
 44%|████▍     | 99696/225000 [196:14:50<847:09:17, 24.34s/it]
 44%|████▍     | 99697/225000 [196:15:14<846:49:11, 24.33s/it]
 44%|████▍     | 99698/225000 [196:15:34<804:44:20, 23.12s/it]
 44%|████▍     | 99699/225000 [196:16:01<837:38:27, 24.07s/it]
 44%|████▍     | 99700/225000 [196:16:26<847:03:47, 24.34s/it]
                                                              
{'loss': '0.8921', 'grad_norm': '1.104', 'learning_rate': '3.145e-05', 'epoch': '0.4431', 'num_input_tokens_seen': 326288360, 'train_runtime': '7.066e+05', 'train_tokens_per_second': '461.8'}

 44%|████▍     | 99700/225000 [196:16:26<847:03:47, 24.34s/it]
 44%|████▍     | 99701/225000 [196:16:56<905:51:22, 26.03s/it]
 44%|████▍     | 99702/225000 [196:17:19<881:35:39, 25.33s/it]
 44%|████▍     | 99703/225000 [196:17:49<926:18:48, 26.61s/it]
 44%|████▍     | 99704/225000 [196:18:12<888:56:41, 25.54s/it]
 44%|████▍     | 99705/225000 [196:18:44<959:10:24, 27.56s/it]
 44%|████▍     | 99706/225000 [196:19:09<924:55:45, 26.58s/it]
 44%|████▍     | 99707/225000 [196:19:28<852:57:53, 24.51s/it]
 44%|████▍     | 99708/225000 [196:19:52<849:30:21, 24.41s/it]
 44%|████▍     | 99709/225000 [196:20:21<895:27:30, 25.73s/it]
 44%|████▍     | 99710/225000 [196:20:49<914:42:59, 26.28s/it]
 44%|████▍     | 99711/225000 [196:21:19<955:49:54, 27.46s/it]
 44%|████▍     | 99712/225000 [196:21:38<868:55:38, 24.97s/it]
 44%|████▍     | 99713/225000 [196:22:11<951:06:44, 27.33s/it]
 44%|████▍     | 99714/225000 [196:22:37<941:17:23, 27.05s/it]
 44%|████▍     | 99715/225000 [196:22:59<886:24:12, 25.47s/it]
 44%|████▍     | 99716/225000 [196:23:33<969:52:44, 27.87s/it]
 44%|████▍     | 99717/225000 [196:23:54<904:37:13, 25.99s/it]
 44%|████▍     | 99718/225000 [196:24:21<910:56:07, 26.18s/it]
 44%|████▍     | 99719/225000 [196:24:40<838:56:10, 24.11s/it]
 44%|████▍     | 99720/225000 [196:25:11<905:58:01, 26.03s/it]
 44%|████▍     | 99721/225000 [196:25:41<951:46:23, 27.35s/it]
 44%|████▍     | 99722/225000 [196:26:03<895:57:00, 25.75s/it]
 44%|████▍     | 99723/225000 [196:26:23<831:13:56, 23.89s/it]
 44%|████▍     | 99724/225000 [196:26:50<864:33:15, 24.84s/it]
 44%|████▍     | 99725/225000 [196:27:17<887:03:13, 25.49s/it]
 44%|████▍     | 99726/225000 [196:27:36<823:20:01, 23.66s/it]
 44%|████▍     | 99727/225000 [196:27:58<807:55:16, 23.22s/it]
 44%|████▍     | 99728/225000 [196:28:28<874:10:38, 25.12s/it]
 44%|████▍     | 99729/225000 [196:28:55<891:32:08, 25.62s/it]
 44%|████▍     | 99730/225000 [196:29:16<844:53:11, 24.28s/it]
 44%|████▍     | 99731/225000 [196:29:39<828:16:04, 23.80s/it]
 44%|████▍     | 99732/225000 [196:30:02<827:54:41, 23.79s/it]
 44%|████▍     | 99733/225000 [196:30:28<843:53:01, 24.25s/it]
 44%|████▍     | 99734/225000 [196:30:53<858:13:11, 24.66s/it]
 44%|████▍     | 99735/225000 [196:31:19<873:13:59, 25.10s/it]
 44%|████▍     | 99736/225000 [196:31:43<862:51:33, 24.80s/it]
 44%|████▍     | 99737/225000 [196:32:05<826:21:36, 23.75s/it]
 44%|████▍     | 99738/225000 [196:32:35<891:50:18, 25.63s/it]
 44%|████▍     | 99739/225000 [196:33:04<927:43:22, 26.66s/it]
 44%|████▍     | 99740/225000 [196:33:26<884:20:06, 25.42s/it]
 44%|████▍     | 99741/225000 [196:33:54<905:41:39, 26.03s/it]
 44%|████▍     | 99742/225000 [196:34:15<852:44:08, 24.51s/it]
 44%|████▍     | 99743/225000 [196:34:41<865:53:44, 24.89s/it]
 44%|████▍     | 99744/225000 [196:35:04<851:20:39, 24.47s/it]
 44%|████▍     | 99745/225000 [196:35:27<839:33:10, 24.13s/it]
 44%|████▍     | 99746/225000 [196:35:49<817:17:17, 23.49s/it]
 44%|████▍     | 99747/225000 [196:36:14<825:53:51, 23.74s/it]
 44%|████▍     | 99748/225000 [196:36:43<884:27:54, 25.42s/it]
 44%|████▍     | 99749/225000 [196:37:11<907:54:17, 26.10s/it]
 44%|████▍     | 99750/225000 [196:37:36<904:07:09, 25.99s/it]
                                                              
{'loss': '0.8904', 'grad_norm': '1.023', 'learning_rate': '3.143e-05', 'epoch': '0.4433', 'num_input_tokens_seen': 326940760, 'train_runtime': '7.079e+05', 'train_tokens_per_second': '461.9'}

 44%|████▍     | 99750/225000 [196:37:36<904:07:09, 25.99s/it]
 44%|████▍     | 99751/225000 [196:37:58<854:36:08, 24.56s/it]
 44%|████▍     | 99752/225000 [196:38:23<864:43:22, 24.85s/it]
 44%|████▍     | 99753/225000 [196:38:42<804:30:42, 23.12s/it]
 44%|████▍     | 99754/225000 [196:39:08<832:09:21, 23.92s/it]
 44%|████▍     | 99755/225000 [196:39:31<818:57:55, 23.54s/it]
 44%|████▍     | 99756/225000 [196:40:03<910:02:00, 26.16s/it]
 44%|████▍     | 99757/225000 [196:40:30<920:58:26, 26.47s/it]
 44%|████▍     | 99758/225000 [196:40:51<864:50:33, 24.86s/it]
 44%|████▍     | 99759/225000 [196:41:20<907:28:55, 26.09s/it]
 44%|████▍     | 99760/225000 [196:41:48<923:45:28, 26.55s/it]
 44%|████▍     | 99761/225000 [196:42:12<902:09:43, 25.93s/it]
 44%|████▍     | 99762/225000 [196:42:37<890:11:08, 25.59s/it]
 44%|████▍     | 99763/225000 [196:43:02<882:15:35, 25.36s/it]
 44%|████▍     | 99764/225000 [196:43:23<839:44:50, 24.14s/it]
 44%|████▍     | 99765/225000 [196:43:51<879:40:30, 25.29s/it]
 44%|████▍     | 99766/225000 [196:44:11<822:37:57, 23.65s/it]
 44%|████▍     | 99767/225000 [196:44:43<905:37:26, 26.03s/it]
 44%|████▍     | 99768/225000 [196:45:11<933:49:14, 26.84s/it]
 44%|████▍     | 99769/225000 [196:45:41<958:57:08, 27.57s/it]
 44%|████▍     | 99770/225000 [196:46:07<946:55:40, 27.22s/it]
 44%|████▍     | 99771/225000 [196:46:29<891:19:53, 25.62s/it]
 44%|████▍     | 99772/225000 [196:46:56<909:52:49, 26.16s/it]
 44%|████▍     | 99773/225000 [196:47:18<866:25:05, 24.91s/it]
 44%|████▍     | 99774/225000 [196:47:45<883:37:17, 25.40s/it]
 44%|████▍     | 99775/225000 [196:48:08<855:35:05, 24.60s/it]
 44%|████▍     | 99776/225000 [196:48:38<915:17:52, 26.31s/it]
 44%|████▍     | 99777/225000 [196:49:00<870:16:27, 25.02s/it]
 44%|████▍     | 99778/225000 [196:49:21<825:08:19, 23.72s/it]
 44%|████▍     | 99779/225000 [196:49:54<924:29:35, 26.58s/it]
 44%|████▍     | 99780/225000 [196:50:18<897:14:00, 25.79s/it]
 44%|████▍     | 99781/225000 [196:50:42<884:45:58, 25.44s/it]
 44%|████▍     | 99782/225000 [196:51:11<915:11:17, 26.31s/it]
 44%|████▍     | 99783/225000 [196:51:37<908:55:35, 26.13s/it]
 44%|████▍     | 99784/225000 [196:51:53<805:08:59, 23.15s/it]
 44%|████▍     | 99785/225000 [196:52:15<793:56:56, 22.83s/it]
 44%|████▍     | 99786/225000 [196:52:48<901:39:01, 25.92s/it]
 44%|████▍     | 99787/225000 [196:53:11<867:35:36, 24.94s/it]
 44%|████▍     | 99788/225000 [196:53:42<935:10:14, 26.89s/it]
 44%|████▍     | 99789/225000 [196:54:10<942:52:23, 27.11s/it]
 44%|████▍     | 99790/225000 [196:54:36<934:31:06, 26.87s/it]
 44%|████▍     | 99791/225000 [196:54:57<869:13:06, 24.99s/it]
 44%|████▍     | 99792/225000 [196:55:28<935:35:54, 26.90s/it]
 44%|████▍     | 99793/225000 [196:55:56<947:03:31, 27.23s/it]
 44%|████▍     | 99794/225000 [196:56:22<932:51:27, 26.82s/it]
 44%|████▍     | 99795/225000 [196:56:50<946:19:09, 27.21s/it]
 44%|████▍     | 99796/225000 [196:57:15<926:08:50, 26.63s/it]
 44%|████▍     | 99797/225000 [196:57:34<847:39:08, 24.37s/it]
 44%|████▍     | 99798/225000 [196:57:58<840:05:09, 24.16s/it]
 44%|████▍     | 99799/225000 [196:58:20<820:38:01, 23.60s/it]
 44%|████▍     | 99800/225000 [196:58:51<890:58:20, 25.62s/it]
                                                              
{'loss': '0.9029', 'grad_norm': '0.8324', 'learning_rate': '3.141e-05', 'epoch': '0.4436', 'num_input_tokens_seen': 327603992, 'train_runtime': '7.091e+05', 'train_tokens_per_second': '462'}

 44%|████▍     | 99800/225000 [196:58:51<890:58:20, 25.62s/it]
 44%|████▍     | 99801/225000 [196:59:24<974:58:05, 28.03s/it]
 44%|████▍     | 99802/225000 [196:59:49<945:24:11, 27.18s/it]
 44%|████▍     | 99803/225000 [197:00:13<906:39:19, 26.07s/it]
 44%|████▍     | 99804/225000 [197:00:40<915:22:58, 26.32s/it]
 44%|████▍     | 99805/225000 [197:00:54<794:02:06, 22.83s/it]
 44%|████▍     | 99806/225000 [197:01:26<888:32:24, 25.55s/it]
 44%|████▍     | 99807/225000 [197:01:52<892:09:42, 25.65s/it]
 44%|████▍     | 99808/225000 [197:02:21<928:56:47, 26.71s/it]
 44%|████▍     | 99809/225000 [197:02:42<864:25:09, 24.86s/it]
 44%|████▍     | 99810/225000 [197:03:07<865:27:29, 24.89s/it]
 44%|████▍     | 99811/225000 [197:03:42<967:30:31, 27.82s/it]
 44%|████▍     | 99812/225000 [197:04:07<939:54:31, 27.03s/it]
 44%|████▍     | 99813/225000 [197:04:34<946:30:30, 27.22s/it]
 44%|████▍     | 99814/225000 [197:04:56<884:51:18, 25.45s/it]
 44%|████▍     | 99815/225000 [197:05:22<897:57:17, 25.82s/it]
 44%|████▍     | 99816/225000 [197:05:46<871:35:19, 25.06s/it]
 44%|████▍     | 99817/225000 [197:06:06<823:19:55, 23.68s/it]
 44%|████▍     | 99818/225000 [197:06:23<750:59:52, 21.60s/it]
 44%|████▍     | 99819/225000 [197:06:55<859:56:56, 24.73s/it]
 44%|████▍     | 99820/225000 [197:07:16<824:41:17, 23.72s/it]
 44%|████▍     | 99821/225000 [197:07:45<874:43:12, 25.16s/it]
 44%|████▍     | 99822/225000 [197:08:16<937:17:54, 26.96s/it]
 44%|████▍     | 99823/225000 [197:08:37<876:31:14, 25.21s/it]
 44%|████▍     | 99824/225000 [197:09:00<851:09:56, 24.48s/it]
 44%|████▍     | 99825/225000 [197:09:23<831:30:04, 23.91s/it]
 44%|████▍     | 99826/225000 [197:09:52<886:39:41, 25.50s/it]
 44%|████▍     | 99827/225000 [197:10:18<890:00:58, 25.60s/it]
 44%|████▍     | 99828/225000 [197:10:42<881:23:21, 25.35s/it]
 44%|████▍     | 99829/225000 [197:11:05<856:51:38, 24.64s/it]
 44%|████▍     | 99830/225000 [197:11:24<799:47:12, 23.00s/it]
 44%|████▍     | 99831/225000 [197:11:54<868:12:20, 24.97s/it]
 44%|████▍     | 99832/225000 [197:12:26<939:24:22, 27.02s/it]
 44%|████▍     | 99833/225000 [197:12:52<927:01:35, 26.66s/it]
 44%|████▍     | 99834/225000 [197:13:17<908:08:17, 26.12s/it]
 44%|████▍     | 99835/225000 [197:13:36<836:46:34, 24.07s/it]
 44%|████▍     | 99836/225000 [197:14:01<844:56:55, 24.30s/it]
 44%|████▍     | 99837/225000 [197:14:30<892:43:11, 25.68s/it]
 44%|████▍     | 99838/225000 [197:15:00<941:11:07, 27.07s/it]
 44%|████▍     | 99839/225000 [197:15:26<927:48:47, 26.69s/it]
 44%|████▍     | 99840/225000 [197:15:47<868:50:30, 24.99s/it]
 44%|████▍     | 99841/225000 [197:16:16<916:44:18, 26.37s/it]
 44%|████▍     | 99842/225000 [197:16:43<917:15:52, 26.38s/it]
 44%|████▍     | 99843/225000 [197:17:11<933:17:06, 26.84s/it]
 44%|████▍     | 99844/225000 [197:17:29<842:11:55, 24.23s/it]
 44%|████▍     | 99845/225000 [197:17:59<902:27:26, 25.96s/it]
 44%|████▍     | 99846/225000 [197:18:27<927:43:40, 26.69s/it]
 44%|████▍     | 99847/225000 [197:18:46<849:30:29, 24.44s/it]
 44%|████▍     | 99848/225000 [197:19:09<829:36:29, 23.86s/it]
 44%|████▍     | 99849/225000 [197:19:30<802:57:49, 23.10s/it]
 44%|████▍     | 99850/225000 [197:20:00<878:17:19, 25.26s/it]
                                                              
{'loss': '0.8878', 'grad_norm': '0.8334', 'learning_rate': '3.139e-05', 'epoch': '0.4438', 'num_input_tokens_seen': 328262456, 'train_runtime': '7.104e+05', 'train_tokens_per_second': '462.1'}

 44%|████▍     | 99850/225000 [197:20:00<878:17:19, 25.26s/it]
 44%|████▍     | 99851/225000 [197:20:20<821:31:42, 23.63s/it]
 44%|████▍     | 99852/225000 [197:20:48<859:47:42, 24.73s/it]
 44%|████▍     | 99853/225000 [197:21:04<777:29:35, 22.37s/it]
 44%|████▍     | 99854/225000 [197:21:33<838:43:08, 24.13s/it]
 44%|████▍     | 99855/225000 [197:22:03<903:03:56, 25.98s/it]
 44%|████▍     | 99856/225000 [197:22:24<853:49:55, 24.56s/it]
 44%|████▍     | 99857/225000 [197:22:49<852:11:11, 24.51s/it]
 44%|████▍     | 99858/225000 [197:23:18<898:02:05, 25.83s/it]
 44%|████▍     | 99859/225000 [197:23:43<896:31:21, 25.79s/it]
 44%|████▍     | 99860/225000 [197:24:07<871:49:43, 25.08s/it]
 44%|████▍     | 99861/225000 [197:24:33<887:06:56, 25.52s/it]
 44%|████▍     | 99862/225000 [197:24:55<847:04:00, 24.37s/it]
 44%|████▍     | 99863/225000 [197:25:21<865:05:14, 24.89s/it]
 44%|████▍     | 99864/225000 [197:25:54<946:21:47, 27.23s/it]
 44%|████▍     | 99865/225000 [197:26:17<901:00:52, 25.92s/it]
 44%|████▍     | 99866/225000 [197:26:45<928:59:42, 26.73s/it]
 44%|████▍     | 99867/225000 [197:27:08<885:42:47, 25.48s/it]
 44%|████▍     | 99868/225000 [197:27:33<885:17:58, 25.47s/it]
 44%|████▍     | 99869/225000 [197:27:59<890:26:52, 25.62s/it]
 44%|████▍     | 99870/225000 [197:28:22<857:05:25, 24.66s/it]
 44%|████▍     | 99871/225000 [197:28:57<972:15:09, 27.97s/it]
 44%|████▍     | 99872/225000 [197:29:18<898:25:06, 25.85s/it]
 44%|████▍     | 99873/225000 [197:29:46<916:44:10, 26.38s/it]
 44%|████▍     | 99874/225000 [197:30:12<911:31:36, 26.23s/it]
 44%|████▍     | 99875/225000 [197:30:34<874:35:35, 25.16s/it]
 44%|████▍     | 99876/225000 [197:30:54<813:20:10, 23.40s/it]
 44%|████▍     | 99877/225000 [197:31:24<888:30:11, 25.56s/it]
 44%|████▍     | 99878/225000 [197:31:51<897:24:51, 25.82s/it]
 44%|████▍     | 99879/225000 [197:32:16<887:59:44, 25.55s/it]
 44%|████▍     | 99880/225000 [197:32:46<935:02:21, 26.90s/it]
 44%|████▍     | 99881/225000 [197:33:09<899:27:09, 25.88s/it]
 44%|████▍     | 99882/225000 [197:33:32<869:37:47, 25.02s/it]
 44%|████▍     | 99883/225000 [197:34:00<902:37:40, 25.97s/it]
 44%|████▍     | 99884/225000 [197:34:26<898:28:41, 25.85s/it]
 44%|████▍     | 99885/225000 [197:34:51<886:17:08, 25.50s/it]
 44%|████▍     | 99886/225000 [197:35:18<902:49:26, 25.98s/it]
 44%|████▍     | 99887/225000 [197:35:43<901:17:29, 25.93s/it]
 44%|████▍     | 99888/225000 [197:36:05<855:54:56, 24.63s/it]
 44%|████▍     | 99889/225000 [197:36:30<858:55:14, 24.71s/it]
 44%|████▍     | 99890/225000 [197:36:55<858:16:58, 24.70s/it]
 44%|████▍     | 99891/225000 [197:37:19<857:53:49, 24.69s/it]
 44%|████▍     | 99892/225000 [197:37:35<764:06:02, 21.99s/it]
 44%|████▍     | 99893/225000 [197:38:04<833:14:29, 23.98s/it]
 44%|████▍     | 99894/225000 [197:38:23<789:34:28, 22.72s/it]
 44%|████▍     | 99895/225000 [197:38:54<873:19:27, 25.13s/it]
 44%|████▍     | 99896/225000 [197:39:17<849:53:50, 24.46s/it]
 44%|████▍     | 99897/225000 [197:39:32<746:11:52, 21.47s/it]
 44%|████▍     | 99898/225000 [197:39:59<810:41:47, 23.33s/it]
 44%|████▍     | 99899/225000 [197:40:25<839:39:24, 24.16s/it]
 44%|████▍     | 99900/225000 [197:40:47<814:58:43, 23.45s/it]
                                                              
{'loss': '0.8879', 'grad_norm': '0.8395', 'learning_rate': '3.138e-05', 'epoch': '0.444', 'num_input_tokens_seen': 328904856, 'train_runtime': '7.116e+05', 'train_tokens_per_second': '462.2'}

 44%|████▍     | 99900/225000 [197:40:47<814:58:43, 23.45s/it]
 44%|████▍     | 99901/225000 [197:41:16<873:56:04, 25.15s/it]
 44%|████▍     | 99902/225000 [197:41:36<816:12:49, 23.49s/it]
 44%|████▍     | 99903/225000 [197:41:59<817:46:00, 23.53s/it]
 44%|████▍     | 99904/225000 [197:42:28<867:04:09, 24.95s/it]
 44%|████▍     | 99905/225000 [197:42:54<881:52:32, 25.38s/it]
 44%|████▍     | 99906/225000 [197:43:21<901:07:22, 25.93s/it]
 44%|████▍     | 99907/225000 [197:43:44<871:03:40, 25.07s/it]
 44%|████▍     | 99908/225000 [197:44:15<933:55:11, 26.88s/it]
 44%|████▍     | 99909/225000 [197:44:44<954:37:40, 27.47s/it]
 44%|████▍     | 99910/225000 [197:45:06<896:33:04, 25.80s/it]
 44%|████▍     | 99911/225000 [197:45:35<927:15:23, 26.69s/it]
 44%|████▍     | 99912/225000 [197:45:57<877:00:02, 25.24s/it]
 44%|████▍     | 99913/225000 [197:46:27<933:12:45, 26.86s/it]
 44%|████▍     | 99914/225000 [197:46:56<954:31:45, 27.47s/it]
 44%|████▍     | 99915/225000 [197:47:16<872:48:00, 25.12s/it]
 44%|████▍     | 99916/225000 [197:47:44<898:14:38, 25.85s/it]
 44%|████▍     | 99917/225000 [197:48:10<903:16:07, 26.00s/it]
 44%|████▍     | 99918/225000 [197:48:36<908:49:09, 26.16s/it]
 44%|████▍     | 99919/225000 [197:48:51<790:18:33, 22.75s/it]
 44%|████▍     | 99920/225000 [197:49:22<873:55:04, 25.15s/it]
 44%|████▍     | 99921/225000 [197:49:45<847:25:44, 24.39s/it]
 44%|████▍     | 99922/225000 [197:50:13<885:47:36, 25.49s/it]
 44%|████▍     | 99923/225000 [197:50:40<908:33:25, 26.15s/it]
 44%|████▍     | 99924/225000 [197:51:05<896:14:55, 25.80s/it]
 44%|████▍     | 99925/225000 [197:51:29<873:35:15, 25.14s/it]
 44%|████▍     | 99926/225000 [197:51:58<917:22:08, 26.40s/it]
 44%|████▍     | 99927/225000 [197:52:22<884:29:29, 25.46s/it]
 44%|████▍     | 99928/225000 [197:52:46<872:05:00, 25.10s/it]
 44%|████▍     | 99929/225000 [197:53:12<883:49:19, 25.44s/it]
 44%|████▍     | 99930/225000 [197:53:37<879:39:01, 25.32s/it]
 44%|████▍     | 99931/225000 [197:53:55<802:26:22, 23.10s/it]
 44%|████▍     | 99932/225000 [197:54:15<775:22:34, 22.32s/it]
 44%|████▍     | 99933/225000 [197:54:41<806:14:31, 23.21s/it]
 44%|████▍     | 99934/225000 [197:55:15<925:17:17, 26.63s/it]
 44%|████▍     | 99935/225000 [197:55:45<953:31:32, 27.45s/it]
 44%|████▍     | 99936/225000 [197:56:15<979:08:43, 28.18s/it]
 44%|████▍     | 99937/225000 [197:56:37<917:44:17, 26.42s/it]
 44%|████▍     | 99938/225000 [197:57:03<915:51:02, 26.36s/it]
 44%|████▍     | 99939/225000 [197:57:33<951:36:45, 27.39s/it]
 44%|████▍     | 99940/225000 [197:57:55<892:04:48, 25.68s/it]
 44%|████▍     | 99941/225000 [197:58:15<837:09:37, 24.10s/it]
 44%|████▍     | 99942/225000 [197:58:38<823:10:53, 23.70s/it]
 44%|████▍     | 99943/225000 [197:59:06<866:54:03, 24.96s/it]
 44%|████▍     | 99944/225000 [197:59:30<856:49:49, 24.67s/it]
 44%|████▍     | 99945/225000 [197:59:52<837:09:41, 24.10s/it]
 44%|████▍     | 99946/225000 [198:00:08<744:33:59, 21.43s/it]
 44%|████▍     | 99947/225000 [198:00:38<841:23:10, 24.22s/it]
 44%|████▍     | 99948/225000 [198:01:06<876:12:40, 25.22s/it]
 44%|████▍     | 99949/225000 [198:01:25<810:42:07, 23.34s/it]
 44%|████▍     | 99950/225000 [198:01:55<877:56:21, 25.27s/it]
                                                              
{'loss': '0.8689', 'grad_norm': '0.8678', 'learning_rate': '3.136e-05', 'epoch': '0.4442', 'num_input_tokens_seen': 329569936, 'train_runtime': '7.129e+05', 'train_tokens_per_second': '462.3'}

 44%|████▍     | 99950/225000 [198:01:55<877:56:21, 25.27s/it]
 44%|████▍     | 99951/225000 [198:02:23<909:40:44, 26.19s/it]
 44%|████▍     | 99952/225000 [198:02:43<847:49:55, 24.41s/it]
 44%|████▍     | 99953/225000 [198:03:10<872:23:07, 25.12s/it]
 44%|████▍     | 99954/225000 [198:03:41<933:28:19, 26.87s/it]
 44%|████▍     | 99955/225000 [198:04:00<847:35:53, 24.40s/it]
 44%|████▍     | 99956/225000 [198:04:21<820:27:24, 23.62s/it]
 44%|████▍     | 99957/225000 [198:04:54<917:56:29, 26.43s/it]
 44%|████▍     | 99958/225000 [198:05:19<900:34:23, 25.93s/it]
 44%|████▍     | 99959/225000 [198:05:48<925:38:55, 26.65s/it]
 44%|████▍     | 99960/225000 [198:06:15<937:30:53, 26.99s/it]
 44%|████▍     | 99961/225000 [198:06:43<942:34:37, 27.14s/it]
 44%|████▍     | 99962/225000 [198:07:04<883:40:19, 25.44s/it]
 44%|████▍     | 99963/225000 [198:07:36<950:04:00, 27.35s/it]
 44%|████▍     | 99964/225000 [198:07:59<904:15:34, 26.04s/it]
 44%|████▍     | 99965/225000 [198:08:19<843:03:20, 24.27s/it]
 44%|████▍     | 99966/225000 [198:08:41<814:07:02, 23.44s/it]
 44%|████▍     | 99967/225000 [198:09:10<871:08:03, 25.08s/it]
 44%|████▍     | 99968/225000 [198:09:41<940:03:34, 27.07s/it]
 44%|████▍     | 99969/225000 [198:10:06<913:24:43, 26.30s/it]
 44%|████▍     | 99970/225000 [198:10:35<940:58:48, 27.09s/it]
 44%|████▍     | 99971/225000 [198:10:58<900:46:55, 25.94s/it]
 44%|████▍     | 99972/225000 [198:11:19<849:57:05, 24.47s/it]
 44%|████▍     | 99973/225000 [198:11:42<836:24:23, 24.08s/it]
 44%|████▍     | 99974/225000 [198:12:05<817:31:45, 23.54s/it]
 44%|████▍     | 99975/225000 [198:12:32<859:15:30, 24.74s/it]
 44%|████▍     | 99976/225000 [198:12:59<879:42:13, 25.33s/it]
 44%|████▍     | 99977/225000 [198:13:24<881:20:41, 25.38s/it]
 44%|████▍     | 99978/225000 [198:13:46<846:11:42, 24.37s/it]
 44%|████▍     | 99979/225000 [198:14:19<936:40:46, 26.97s/it]
 44%|████▍     | 99980/225000 [198:14:44<911:14:25, 26.24s/it]
 44%|████▍     | 99981/225000 [198:15:08<891:10:09, 25.66s/it]
 44%|████▍     | 99982/225000 [198:15:31<863:46:26, 24.87s/it]
 44%|████▍     | 99983/225000 [198:16:00<902:03:27, 25.98s/it]
 44%|████▍     | 99984/225000 [198:16:30<948:35:12, 27.32s/it]
 44%|████▍     | 99985/225000 [198:16:49<863:19:52, 24.86s/it]
 44%|████▍     | 99986/225000 [198:17:16<878:58:25, 25.31s/it]
 44%|████▍     | 99987/225000 [198:17:44<905:31:49, 26.08s/it]
 44%|████▍     | 99988/225000 [198:18:04<849:59:26, 24.48s/it]
 44%|████▍     | 99989/225000 [198:18:36<923:17:03, 26.59s/it]
 44%|████▍     | 99990/225000 [198:18:55<848:53:59, 24.45s/it]
 44%|████▍     | 99991/225000 [198:19:24<898:08:10, 25.86s/it]
 44%|████▍     | 99992/225000 [198:19:45<844:41:22, 24.33s/it]
 44%|████▍     | 99993/225000 [198:20:15<904:09:34, 26.04s/it]
 44%|████▍     | 99994/225000 [198:20:42<911:45:37, 26.26s/it]
 44%|████▍     | 99995/225000 [198:21:04<865:23:28, 24.92s/it]
 44%|████▍     | 99996/225000 [198:21:33<905:07:48, 26.07s/it]
 44%|████▍     | 99997/225000 [198:21:58<898:31:28, 25.88s/it]
 44%|████▍     | 99998/225000 [198:22:26<921:26:29, 26.54s/it]
 44%|████▍     | 99999/225000 [198:22:51<907:12:56, 26.13s/it]
 44%|████▍     | 100000/225000 [198:23:19<927:31:19, 26.71s/it]
                                                               
{'loss': '0.8885', 'grad_norm': '0.8625', 'learning_rate': '3.134e-05', 'epoch': '0.4444', 'num_input_tokens_seen': 330235104, 'train_runtime': '7.142e+05', 'train_tokens_per_second': '462.4'}

 44%|████▍     | 100000/225000 [198:23:19<927:31:19, 26.71s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                               
{'eval_loss': '1.429', 'eval_runtime': '432.8', 'eval_samples_per_second': '1.155', 'eval_steps_per_second': '0.578', 'epoch': '0.4444', 'num_input_tokens_seen': 330235104}

 44%|████▍     | 100000/225000 [198:30:32<927:31:19, 26.71s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-100000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 44%|████▍     | 100001/225000 [198:30:52<5367:12:18, 154.58s/it]
 44%|████▍     | 100002/225000 [198:31:15<3999:32:04, 115.19s/it]
 44%|████▍     | 100003/225000 [198:31:40<3050:26:43, 87.85s/it] 
 44%|████▍     | 100004/225000 [198:32:13<2479:03:14, 71.40s/it]
 44%|████▍     | 100005/225000 [198:32:33<1948:12:54, 56.11s/it]
 44%|████▍     | 100006/225000 [198:33:02<1666:08:46, 47.99s/it]
 44%|████▍     | 100007/225000 [198:33:29<1442:34:10, 41.55s/it]
 44%|████▍     | 100008/225000 [198:33:53<1264:52:01, 36.43s/it]
 44%|████▍     | 100009/225000 [198:34:20<1165:21:53, 33.56s/it]
 44%|████▍     | 100010/225000 [198:34:38<1003:30:19, 28.90s/it]
 44%|████▍     | 100011/225000 [198:35:11<1045:21:17, 30.11s/it]
 44%|████▍     | 100012/225000 [198:35:30<933:45:08, 26.89s/it] 
 44%|████▍     | 100013/225000 [198:35:58<945:09:29, 27.22s/it]
 44%|████▍     | 100014/225000 [198:36:33<1019:47:30, 29.37s/it]
 44%|████▍     | 100015/225000 [198:37:01<1006:34:27, 28.99s/it]
 44%|████▍     | 100016/225000 [198:37:26<970:00:46, 27.94s/it] 
 44%|████▍     | 100017/225000 [198:37:52<946:52:07, 27.27s/it]
 44%|████▍     | 100018/225000 [198:38:19<944:48:23, 27.21s/it]
 44%|████▍     | 100019/225000 [198:38:46<941:02:04, 27.11s/it]
 44%|████▍     | 100020/225000 [198:39:18<991:00:38, 28.55s/it]
 44%|████▍     | 100021/225000 [198:39:39<914:17:13, 26.34s/it]
 44%|████▍     | 100022/225000 [198:39:59<850:45:12, 24.51s/it]
 44%|████▍     | 100023/225000 [198:40:29<909:05:19, 26.19s/it]
 44%|████▍     | 100024/225000 [198:40:59<948:22:08, 27.32s/it]
 44%|████▍     | 100025/225000 [198:41:25<929:19:37, 26.77s/it]
 44%|████▍     | 100026/225000 [198:41:53<947:27:52, 27.29s/it]
 44%|████▍     | 100027/225000 [198:42:20<941:06:04, 27.11s/it]
 44%|████▍     | 100028/225000 [198:42:41<872:58:15, 25.15s/it]
 44%|████▍     | 100029/225000 [198:43:10<919:26:42, 26.49s/it]
 44%|████▍     | 100030/225000 [198:43:31<858:22:43, 24.73s/it]
 44%|████▍     | 100031/225000 [198:43:59<893:25:13, 25.74s/it]
 44%|████▍     | 100032/225000 [198:44:28<930:52:59, 26.82s/it]
 44%|████▍     | 100033/225000 [198:44:50<879:39:41, 25.34s/it]
 44%|████▍     | 100034/225000 [198:45:13<853:00:28, 24.57s/it]
 44%|████▍     | 100035/225000 [198:45:33<805:28:22, 23.20s/it]
 44%|████▍     | 100036/225000 [198:45:57<818:03:53, 23.57s/it]
 44%|████▍     | 100037/225000 [198:46:22<829:39:46, 23.90s/it]
 44%|████▍     | 100038/225000 [198:46:53<904:03:22, 26.04s/it]
 44%|████▍     | 100039/225000 [198:47:16<873:53:11, 25.18s/it]
 44%|████▍     | 100040/225000 [198:47:38<840:34:19, 24.22s/it]
 44%|████▍     | 100041/225000 [198:48:05<863:06:39, 24.87s/it]
 44%|████▍     | 100042/225000 [198:48:39<961:18:17, 27.69s/it]
 44%|████▍     | 100043/225000 [198:49:01<903:30:04, 26.03s/it]
 44%|████▍     | 100044/225000 [198:49:21<841:10:14, 24.23s/it]
 44%|████▍     | 100045/225000 [198:49:48<874:19:12, 25.19s/it]
 44%|████▍     | 100046/225000 [198:50:15<892:07:02, 25.70s/it]
 44%|████▍     | 100047/225000 [198:50:40<880:56:32, 25.38s/it]
 44%|████▍     | 100048/225000 [198:51:11<939:02:40, 27.05s/it]
 44%|████▍     | 100049/225000 [198:51:40<964:44:58, 27.80s/it]
 44%|████▍     | 100050/225000 [198:52:06<946:01:42, 27.26s/it]
                                                               
{'loss': '0.9244', 'grad_norm': '0.9264', 'learning_rate': '3.132e-05', 'epoch': '0.4447', 'num_input_tokens_seen': 330910952, 'train_runtime': '7.159e+05', 'train_tokens_per_second': '462.2'}

 44%|████▍     | 100050/225000 [198:52:06<946:01:42, 27.26s/it]
 44%|████▍     | 100051/225000 [198:52:28<881:47:18, 25.41s/it]
 44%|████▍     | 100052/225000 [198:52:53<879:43:45, 25.35s/it]
 44%|████▍     | 100053/225000 [198:53:22<924:33:33, 26.64s/it]
 44%|████▍     | 100054/225000 [198:53:46<893:14:31, 25.74s/it]
 44%|████▍     | 100055/225000 [198:54:13<908:14:35, 26.17s/it]
 44%|████▍     | 100056/225000 [198:54:37<882:27:01, 25.43s/it]
 44%|████▍     | 100057/225000 [198:55:08<938:37:48, 27.04s/it]
 44%|████▍     | 100058/225000 [198:55:32<914:12:49, 26.34s/it]
 44%|████▍     | 100059/225000 [198:55:54<865:21:45, 24.93s/it]
 44%|████▍     | 100060/225000 [198:56:19<864:06:05, 24.90s/it]
 44%|████▍     | 100061/225000 [198:56:35<768:17:51, 22.14s/it]
 44%|████▍     | 100062/225000 [198:57:07<878:27:38, 25.31s/it]
 44%|████▍     | 100063/225000 [198:57:28<833:52:46, 24.03s/it]
 44%|████▍     | 100064/225000 [198:58:00<912:52:41, 26.30s/it]
 44%|████▍     | 100065/225000 [198:58:27<918:47:14, 26.47s/it]
 44%|████▍     | 100066/225000 [198:58:48<866:48:19, 24.98s/it]
 44%|████▍     | 100067/225000 [198:59:14<879:18:13, 25.34s/it]
 44%|████▍     | 100068/225000 [198:59:41<890:26:15, 25.66s/it]
 44%|████▍     | 100069/225000 [199:00:13<953:32:29, 27.48s/it]
 44%|████▍     | 100070/225000 [199:00:36<915:48:04, 26.39s/it]
 44%|████▍     | 100071/225000 [199:01:02<911:02:19, 26.25s/it]
 44%|████▍     | 100072/225000 [199:01:24<857:39:00, 24.71s/it]
 44%|████▍     | 100073/225000 [199:01:47<842:24:26, 24.28s/it]
 44%|████▍     | 100074/225000 [199:02:05<774:54:54, 22.33s/it]
 44%|████▍     | 100075/225000 [199:02:31<820:49:25, 23.65s/it]
 44%|████▍     | 100076/225000 [199:03:01<887:51:01, 25.59s/it]
 44%|████▍     | 100077/225000 [199:03:24<857:59:54, 24.73s/it]
 44%|████▍     | 100078/225000 [199:03:52<890:18:12, 25.66s/it]
 44%|████▍     | 100079/225000 [199:04:19<899:41:36, 25.93s/it]
 44%|████▍     | 100080/225000 [199:04:47<926:06:57, 26.69s/it]
 44%|████▍     | 100081/225000 [199:05:12<905:12:02, 26.09s/it]
 44%|████▍     | 100082/225000 [199:05:44<974:35:22, 28.09s/it]
 44%|████▍     | 100083/225000 [199:06:06<906:09:41, 26.11s/it]
 44%|████▍     | 100084/225000 [199:06:29<873:03:13, 25.16s/it]
 44%|████▍     | 100085/225000 [199:06:50<833:39:24, 24.03s/it]
 44%|████▍     | 100086/225000 [199:07:23<923:34:25, 26.62s/it]
 44%|████▍     | 100087/225000 [199:07:47<894:24:55, 25.78s/it]
 44%|████▍     | 100088/225000 [199:08:11<878:33:25, 25.32s/it]
 44%|████▍     | 100089/225000 [199:08:35<866:59:31, 24.99s/it]
 44%|████▍     | 100090/225000 [199:09:02<889:58:26, 25.65s/it]
 44%|████▍     | 100091/225000 [199:09:26<872:44:24, 25.15s/it]
 44%|████▍     | 100092/225000 [199:09:50<860:08:05, 24.79s/it]
 44%|████▍     | 100093/225000 [199:10:08<789:41:40, 22.76s/it]
 44%|████▍     | 100094/225000 [199:10:38<857:18:54, 24.71s/it]
 44%|████▍     | 100095/225000 [199:11:11<950:41:06, 27.40s/it]
 44%|████▍     | 100096/225000 [199:11:39<949:58:25, 27.38s/it]
 44%|████▍     | 100097/225000 [199:11:59<876:17:24, 25.26s/it]
 44%|████▍     | 100098/225000 [199:12:27<904:00:50, 26.06s/it]
 44%|████▍     | 100099/225000 [199:12:51<880:41:52, 25.38s/it]
 44%|████▍     | 100100/225000 [199:13:12<841:13:52, 24.25s/it]
                                                               
{'loss': '0.8894', 'grad_norm': '0.8314', 'learning_rate': '3.131e-05', 'epoch': '0.4449', 'num_input_tokens_seen': 331579600, 'train_runtime': '7.172e+05', 'train_tokens_per_second': '462.3'}

 44%|████▍     | 100100/225000 [199:13:12<841:13:52, 24.25s/it]
 44%|████▍     | 100101/225000 [199:13:39<866:30:08, 24.98s/it]
 44%|████▍     | 100102/225000 [199:14:04<865:20:49, 24.94s/it]
 44%|████▍     | 100103/225000 [199:14:31<893:15:08, 25.75s/it]
 44%|████▍     | 100104/225000 [199:14:59<916:13:24, 26.41s/it]
 44%|████▍     | 100105/225000 [199:15:27<929:35:11, 26.79s/it]
 44%|████▍     | 100106/225000 [199:15:58<973:05:46, 28.05s/it]
 44%|████▍     | 100107/225000 [199:16:15<861:28:30, 24.83s/it]
 44%|████▍     | 100108/225000 [199:16:42<881:55:32, 25.42s/it]
 44%|████▍     | 100109/225000 [199:17:10<902:25:56, 26.01s/it]
 44%|████▍     | 100110/225000 [199:17:29<837:09:48, 24.13s/it]
 44%|████▍     | 100111/225000 [199:17:51<811:16:56, 23.39s/it]
 44%|████▍     | 100112/225000 [199:18:17<840:52:05, 24.24s/it]
 44%|████▍     | 100113/225000 [199:18:45<874:26:00, 25.21s/it]
 44%|████▍     | 100114/225000 [199:19:03<806:40:12, 23.25s/it]
 44%|████▍     | 100115/225000 [199:19:37<910:05:37, 26.23s/it]
 44%|████▍     | 100116/225000 [199:20:03<917:12:56, 26.44s/it]
 44%|████▍     | 100117/225000 [199:20:25<866:58:32, 24.99s/it]
 44%|████▍     | 100118/225000 [199:21:00<967:16:34, 27.88s/it]
 44%|████▍     | 100119/225000 [199:21:27<956:54:35, 27.59s/it]
 44%|████▍     | 100120/225000 [199:21:50<915:31:52, 26.39s/it]
 44%|████▍     | 100121/225000 [199:22:16<907:10:01, 26.15s/it]
 44%|████▍     | 100122/225000 [199:22:44<928:03:26, 26.75s/it]
 44%|████▍     | 100123/225000 [199:23:10<920:32:06, 26.54s/it]
 44%|████▍     | 100124/225000 [199:23:36<916:17:29, 26.42s/it]
 44%|████▍     | 100125/225000 [199:24:05<939:50:46, 27.09s/it]
 45%|████▍     | 100126/225000 [199:24:28<896:56:26, 25.86s/it]
 45%|████▍     | 100127/225000 [199:24:56<921:39:12, 26.57s/it]
 45%|████▍     | 100128/225000 [199:25:18<875:08:10, 25.23s/it]
 45%|████▍     | 100129/225000 [199:25:46<899:51:35, 25.94s/it]
 45%|████▍     | 100130/225000 [199:26:11<894:53:37, 25.80s/it]
 45%|████▍     | 100131/225000 [199:26:33<858:08:37, 24.74s/it]
 45%|████▍     | 100132/225000 [199:26:59<861:53:18, 24.85s/it]
 45%|████▍     | 100133/225000 [199:27:31<946:06:29, 27.28s/it]
 45%|████▍     | 100134/225000 [199:27:57<926:12:08, 26.70s/it]
 45%|████▍     | 100135/225000 [199:28:19<880:42:31, 25.39s/it]
 45%|████▍     | 100136/225000 [199:28:46<891:30:42, 25.70s/it]
 45%|████▍     | 100137/225000 [199:29:12<901:51:00, 26.00s/it]
 45%|████▍     | 100138/225000 [199:29:39<906:38:16, 26.14s/it]
 45%|████▍     | 100139/225000 [199:30:04<895:57:48, 25.83s/it]
 45%|████▍     | 100140/225000 [199:30:24<836:18:54, 24.11s/it]
 45%|████▍     | 100141/225000 [199:30:45<802:33:00, 23.14s/it]
 45%|████▍     | 100142/225000 [199:31:11<834:35:03, 24.06s/it]
 45%|████▍     | 100143/225000 [199:31:30<779:30:58, 22.48s/it]
 45%|████▍     | 100144/225000 [199:31:54<800:48:16, 23.09s/it]
 45%|████▍     | 100145/225000 [199:32:15<775:21:41, 22.36s/it]
 45%|████▍     | 100146/225000 [199:32:49<900:09:05, 25.95s/it]
 45%|████▍     | 100147/225000 [199:33:16<906:41:58, 26.14s/it]
 45%|████▍     | 100148/225000 [199:33:45<940:22:42, 27.12s/it]
 45%|████▍     | 100149/225000 [199:34:04<855:44:04, 24.67s/it]
 45%|████▍     | 100150/225000 [199:34:29<859:05:31, 24.77s/it]
                                                               
{'loss': '0.8695', 'grad_norm': '1.032', 'learning_rate': '3.129e-05', 'epoch': '0.4451', 'num_input_tokens_seen': 332251880, 'train_runtime': '7.185e+05', 'train_tokens_per_second': '462.4'}

 45%|████▍     | 100150/225000 [199:34:29<859:05:31, 24.77s/it]
 45%|████▍     | 100151/225000 [199:34:59<905:37:45, 26.11s/it]
 45%|████▍     | 100152/225000 [199:35:23<893:28:38, 25.76s/it]
 45%|████▍     | 100153/225000 [199:35:42<819:47:48, 23.64s/it]
 45%|████▍     | 100154/225000 [199:36:11<874:16:33, 25.21s/it]
 45%|████▍     | 100155/225000 [199:36:39<902:30:17, 26.02s/it]
 45%|████▍     | 100156/225000 [199:37:01<864:22:21, 24.93s/it]
 45%|████▍     | 100157/225000 [199:37:28<879:54:07, 25.37s/it]
 45%|████▍     | 100158/225000 [199:37:52<865:22:25, 24.95s/it]
 45%|████▍     | 100159/225000 [199:38:23<926:22:53, 26.71s/it]
 45%|████▍     | 100160/225000 [199:38:51<943:24:51, 27.21s/it]
 45%|████▍     | 100161/225000 [199:39:15<908:11:11, 26.19s/it]
 45%|████▍     | 100162/225000 [199:39:47<973:37:20, 28.08s/it]
 45%|████▍     | 100163/225000 [199:40:10<923:32:28, 26.63s/it]
 45%|████▍     | 100164/225000 [199:40:36<909:21:24, 26.22s/it]
 45%|████▍     | 100165/225000 [199:41:07<960:06:34, 27.69s/it]
 45%|████▍     | 100166/225000 [199:41:33<942:20:27, 27.18s/it]
 45%|████▍     | 100167/225000 [199:41:56<895:46:07, 25.83s/it]
 45%|████▍     | 100168/225000 [199:42:31<991:52:37, 28.60s/it]
 45%|████▍     | 100169/225000 [199:42:56<955:11:12, 27.55s/it]
 45%|████▍     | 100170/225000 [199:43:24<959:21:26, 27.67s/it]
 45%|████▍     | 100171/225000 [199:43:47<910:01:08, 26.24s/it]
 45%|████▍     | 100172/225000 [199:44:18<961:48:32, 27.74s/it]
 45%|████▍     | 100173/225000 [199:44:43<933:02:46, 26.91s/it]
 45%|████▍     | 100174/225000 [199:45:05<883:10:07, 25.47s/it]
 45%|████▍     | 100175/225000 [199:45:29<872:39:55, 25.17s/it]
 45%|████▍     | 100176/225000 [199:46:01<940:44:40, 27.13s/it]
 45%|████▍     | 100177/225000 [199:46:33<986:00:21, 28.44s/it]
 45%|████▍     | 100178/225000 [199:47:04<1017:56:32, 29.36s/it]
 45%|████▍     | 100179/225000 [199:47:30<984:01:43, 28.38s/it] 
 45%|████▍     | 100180/225000 [199:47:54<937:36:09, 27.04s/it]
 45%|████▍     | 100181/225000 [199:48:24<963:16:24, 27.78s/it]
 45%|████▍     | 100182/225000 [199:48:50<945:56:48, 27.28s/it]
 45%|████▍     | 100183/225000 [199:49:17<943:57:41, 27.23s/it]
 45%|████▍     | 100184/225000 [199:49:39<890:11:04, 25.68s/it]
 45%|████▍     | 100185/225000 [199:50:00<842:00:18, 24.29s/it]
 45%|████▍     | 100186/225000 [199:50:24<842:01:37, 24.29s/it]
 45%|████▍     | 100187/225000 [199:50:49<851:29:10, 24.56s/it]
 45%|████▍     | 100188/225000 [199:51:11<817:28:35, 23.58s/it]
 45%|████▍     | 100189/225000 [199:51:39<872:01:29, 25.15s/it]
 45%|████▍     | 100190/225000 [199:51:57<796:52:34, 22.98s/it]
 45%|████▍     | 100191/225000 [199:52:26<855:03:45, 24.66s/it]
 45%|████▍     | 100192/225000 [199:52:51<860:16:25, 24.81s/it]
 45%|████▍     | 100193/225000 [199:53:22<920:22:50, 26.55s/it]
 45%|████▍     | 100194/225000 [199:53:51<949:02:28, 27.37s/it]
 45%|████▍     | 100195/225000 [199:54:08<840:28:06, 24.24s/it]
 45%|████▍     | 100196/225000 [199:54:33<844:01:46, 24.35s/it]
 45%|████▍     | 100197/225000 [199:55:06<939:27:11, 27.10s/it]
 45%|████▍     | 100198/225000 [199:55:38<987:29:54, 28.49s/it]
 45%|████▍     | 100199/225000 [199:56:03<958:23:16, 27.65s/it]
 45%|████▍     | 100200/225000 [199:56:27<918:59:54, 26.51s/it]
                                                               
{'loss': '0.8788', 'grad_norm': '1.161', 'learning_rate': '3.127e-05', 'epoch': '0.4453', 'num_input_tokens_seen': 332926336, 'train_runtime': '7.198e+05', 'train_tokens_per_second': '462.5'}

 45%|████▍     | 100200/225000 [199:56:27<918:59:54, 26.51s/it]
 45%|████▍     | 100201/225000 [199:56:59<973:54:34, 28.09s/it]
 45%|████▍     | 100202/225000 [199:57:27<976:41:15, 28.17s/it]
 45%|████▍     | 100203/225000 [199:57:48<895:52:30, 25.84s/it]
 45%|████▍     | 100204/225000 [199:58:15<911:45:23, 26.30s/it]
 45%|████▍     | 100205/225000 [199:58:42<920:35:14, 26.56s/it]
 45%|████▍     | 100206/225000 [199:59:10<927:00:39, 26.74s/it]
 45%|████▍     | 100207/225000 [199:59:39<958:41:58, 27.66s/it]
 45%|████▍     | 100208/225000 [200:00:07<957:50:29, 27.63s/it]
 45%|████▍     | 100209/225000 [200:00:39<1000:27:33, 28.86s/it]
 45%|████▍     | 100210/225000 [200:01:04<966:30:44, 27.88s/it] 
 45%|████▍     | 100211/225000 [200:01:32<962:00:30, 27.75s/it]
 45%|████▍     | 100212/225000 [200:02:04<1006:18:34, 29.03s/it]
 45%|████▍     | 100213/225000 [200:02:34<1019:01:32, 29.40s/it]
 45%|████▍     | 100214/225000 [200:02:54<920:52:45, 26.57s/it] 
 45%|████▍     | 100215/225000 [200:03:20<920:32:44, 26.56s/it]
 45%|████▍     | 100216/225000 [200:03:45<896:14:02, 25.86s/it]
 45%|████▍     | 100217/225000 [200:04:03<821:06:03, 23.69s/it]
 45%|████▍     | 100218/225000 [200:04:30<846:46:41, 24.43s/it]
 45%|████▍     | 100219/225000 [200:04:59<899:02:39, 25.94s/it]
 45%|████▍     | 100220/225000 [200:05:26<908:49:49, 26.22s/it]
 45%|████▍     | 100221/225000 [200:05:49<881:42:47, 25.44s/it]
 45%|████▍     | 100222/225000 [200:06:12<848:37:59, 24.48s/it]
 45%|████▍     | 100223/225000 [200:06:33<811:17:13, 23.41s/it]
 45%|████▍     | 100224/225000 [200:06:59<844:11:20, 24.36s/it]
 45%|████▍     | 100225/225000 [200:07:24<846:35:25, 24.43s/it]
 45%|████▍     | 100226/225000 [200:07:48<842:22:34, 24.30s/it]
 45%|████▍     | 100227/225000 [200:08:22<944:38:16, 27.26s/it]
 45%|████▍     | 100228/225000 [200:08:53<984:46:13, 28.41s/it]
 45%|████▍     | 100229/225000 [200:09:22<995:04:15, 28.71s/it]
 45%|████▍     | 100230/225000 [200:09:45<931:32:31, 26.88s/it]
 45%|████▍     | 100231/225000 [200:10:10<908:52:20, 26.22s/it]
 45%|████▍     | 100232/225000 [200:10:33<872:50:19, 25.18s/it]
 45%|████▍     | 100233/225000 [200:10:55<839:49:35, 24.23s/it]
 45%|████▍     | 100234/225000 [200:11:29<950:08:37, 27.42s/it]
 45%|████▍     | 100235/225000 [200:11:53<908:34:51, 26.22s/it]
 45%|████▍     | 100236/225000 [200:12:18<898:05:55, 25.91s/it]
 45%|████▍     | 100237/225000 [200:12:46<922:11:38, 26.61s/it]
 45%|████▍     | 100238/225000 [200:13:13<928:39:54, 26.80s/it]
 45%|████▍     | 100239/225000 [200:13:40<923:13:14, 26.64s/it]
 45%|████▍     | 100240/225000 [200:14:07<925:42:20, 26.71s/it]
 45%|████▍     | 100241/225000 [200:14:33<926:01:12, 26.72s/it]
 45%|████▍     | 100242/225000 [200:14:51<833:34:08, 24.05s/it]
 45%|████▍     | 100243/225000 [200:15:11<788:13:22, 22.75s/it]
 45%|████▍     | 100244/225000 [200:15:36<814:32:30, 23.50s/it]
 45%|████▍     | 100245/225000 [200:16:00<816:02:47, 23.55s/it]
 45%|████▍     | 100246/225000 [200:16:32<909:19:47, 26.24s/it]
 45%|████▍     | 100247/225000 [200:17:02<945:40:13, 27.29s/it]
 45%|████▍     | 100248/225000 [200:17:28<926:43:00, 26.74s/it]
 45%|████▍     | 100249/225000 [200:17:59<973:24:01, 28.09s/it]
 45%|████▍     | 100250/225000 [200:18:20<898:52:15, 25.94s/it]
                                                               
{'loss': '0.8896', 'grad_norm': '0.9573', 'learning_rate': '3.125e-05', 'epoch': '0.4456', 'num_input_tokens_seen': 333607120, 'train_runtime': '7.211e+05', 'train_tokens_per_second': '462.6'}

 45%|████▍     | 100250/225000 [200:18:20<898:52:15, 25.94s/it]
 45%|████▍     | 100251/225000 [200:18:53<975:58:03, 28.16s/it]
 45%|████▍     | 100252/225000 [200:19:14<903:03:22, 26.06s/it]
 45%|████▍     | 100253/225000 [200:19:43<934:12:57, 26.96s/it]
 45%|████▍     | 100254/225000 [200:20:07<895:53:15, 25.85s/it]
 45%|████▍     | 100255/225000 [200:20:33<906:58:08, 26.17s/it]
 45%|████▍     | 100256/225000 [200:21:02<932:02:38, 26.90s/it]
 45%|████▍     | 100257/225000 [200:21:21<852:32:09, 24.60s/it]
 45%|████▍     | 100258/225000 [200:21:39<785:26:27, 22.67s/it]
 45%|████▍     | 100259/225000 [200:22:12<884:28:27, 25.53s/it]
 45%|████▍     | 100260/225000 [200:22:36<876:04:50, 25.28s/it]
 45%|████▍     | 100261/225000 [200:23:04<903:15:47, 26.07s/it]
 45%|████▍     | 100262/225000 [200:23:34<946:11:15, 27.31s/it]
 45%|████▍     | 100263/225000 [200:23:57<894:19:57, 25.81s/it]
 45%|████▍     | 100264/225000 [200:24:26<928:37:56, 26.80s/it]
 45%|████▍     | 100265/225000 [200:24:55<948:21:03, 27.37s/it]
 45%|████▍     | 100266/225000 [200:25:19<920:07:09, 26.56s/it]
 45%|████▍     | 100267/225000 [200:25:45<913:49:41, 26.37s/it]
 45%|████▍     | 100268/225000 [200:26:05<849:13:42, 24.51s/it]
 45%|████▍     | 100269/225000 [200:26:35<899:37:12, 25.96s/it]
 45%|████▍     | 100270/225000 [200:26:55<841:39:23, 24.29s/it]
 45%|████▍     | 100271/225000 [200:27:22<864:41:59, 24.96s/it]
 45%|████▍     | 100272/225000 [200:27:44<840:06:33, 24.25s/it]
 45%|████▍     | 100273/225000 [200:28:09<850:34:02, 24.55s/it]
 45%|████▍     | 100274/225000 [200:28:30<813:05:12, 23.47s/it]
 45%|████▍     | 100275/225000 [200:29:04<918:27:44, 26.51s/it]
 45%|████▍     | 100276/225000 [200:29:32<937:31:13, 27.06s/it]
 45%|████▍     | 100277/225000 [200:30:02<965:11:52, 27.86s/it]
 45%|████▍     | 100278/225000 [200:30:28<941:17:28, 27.17s/it]
 45%|████▍     | 100279/225000 [200:30:52<909:40:35, 26.26s/it]
 45%|████▍     | 100280/225000 [200:31:23<956:59:53, 27.62s/it]
 45%|████▍     | 100281/225000 [200:31:46<918:38:52, 26.52s/it]
 45%|████▍     | 100282/225000 [200:32:13<923:18:45, 26.65s/it]
 45%|████▍     | 100283/225000 [200:32:43<952:06:57, 27.48s/it]
 45%|████▍     | 100284/225000 [200:33:04<887:52:20, 25.63s/it]
 45%|████▍     | 100285/225000 [200:33:27<861:52:26, 24.88s/it]
 45%|████▍     | 100286/225000 [200:33:58<924:17:27, 26.68s/it]
 45%|████▍     | 100287/225000 [200:34:21<879:22:53, 25.38s/it]
 45%|████▍     | 100288/225000 [200:34:39<805:35:33, 23.25s/it]
 45%|████▍     | 100289/225000 [200:35:12<911:53:22, 26.32s/it]
 45%|████▍     | 100290/225000 [200:35:36<880:38:25, 25.42s/it]
 45%|████▍     | 100291/225000 [200:36:03<900:53:19, 26.01s/it]
 45%|████▍     | 100292/225000 [200:36:33<939:59:28, 27.14s/it]
 45%|████▍     | 100293/225000 [200:36:56<899:24:36, 25.96s/it]
 45%|████▍     | 100294/225000 [200:37:24<921:38:33, 26.61s/it]
 45%|████▍     | 100295/225000 [200:37:50<910:52:23, 26.30s/it]
 45%|████▍     | 100296/225000 [200:38:18<927:27:06, 26.77s/it]
 45%|████▍     | 100297/225000 [200:38:48<967:49:13, 27.94s/it]
 45%|████▍     | 100298/225000 [200:39:04<836:40:44, 24.15s/it]
 45%|████▍     | 100299/225000 [200:39:31<868:46:05, 25.08s/it]
 45%|████▍     | 100300/225000 [200:40:03<944:41:59, 27.27s/it]
                                                               
{'loss': '0.8929', 'grad_norm': '0.8221', 'learning_rate': '3.123e-05', 'epoch': '0.4458', 'num_input_tokens_seen': 334285432, 'train_runtime': '7.224e+05', 'train_tokens_per_second': '462.7'}

 45%|████▍     | 100300/225000 [200:40:03<944:41:59, 27.27s/it]
 45%|████▍     | 100301/225000 [200:40:34<977:47:36, 28.23s/it]
 45%|████▍     | 100302/225000 [200:40:56<915:39:29, 26.43s/it]
 45%|████▍     | 100303/225000 [200:41:18<873:24:17, 25.22s/it]
 45%|████▍     | 100304/225000 [200:41:50<943:05:26, 27.23s/it]
 45%|████▍     | 100305/225000 [200:42:16<929:29:24, 26.83s/it]
 45%|████▍     | 100306/225000 [200:42:34<840:50:42, 24.28s/it]
 45%|████▍     | 100307/225000 [200:43:03<888:14:32, 25.64s/it]
 45%|████▍     | 100308/225000 [200:43:32<916:04:45, 26.45s/it]
 45%|████▍     | 100309/225000 [200:43:59<925:12:41, 26.71s/it]
 45%|████▍     | 100310/225000 [200:44:27<938:05:00, 27.08s/it]
 45%|████▍     | 100311/225000 [200:44:57<972:54:01, 28.09s/it]
 45%|████▍     | 100312/225000 [200:45:22<936:48:29, 27.05s/it]
 45%|████▍     | 100313/225000 [200:45:49<935:35:57, 27.01s/it]
 45%|████▍     | 100314/225000 [200:46:15<926:18:40, 26.74s/it]
 45%|████▍     | 100315/225000 [200:46:43<941:29:24, 27.18s/it]
 45%|████▍     | 100316/225000 [200:47:04<880:33:18, 25.42s/it]
 45%|████▍     | 100317/225000 [200:47:30<886:21:10, 25.59s/it]
 45%|████▍     | 100318/225000 [200:47:54<863:46:24, 24.94s/it]
 45%|████▍     | 100319/225000 [200:48:24<913:17:25, 26.37s/it]
 45%|████▍     | 100320/225000 [200:48:52<936:12:38, 27.03s/it]
 45%|████▍     | 100321/225000 [200:49:18<925:56:20, 26.74s/it]
 45%|████▍     | 100322/225000 [200:49:47<942:21:11, 27.21s/it]
 45%|████▍     | 100323/225000 [200:50:11<911:24:57, 26.32s/it]
 45%|████▍     | 100324/225000 [200:50:38<918:03:12, 26.51s/it]
 45%|████▍     | 100325/225000 [200:51:04<917:51:30, 26.50s/it]
 45%|████▍     | 100326/225000 [200:51:29<895:27:35, 25.86s/it]
 45%|████▍     | 100327/225000 [200:52:00<951:55:31, 27.49s/it]
 45%|████▍     | 100328/225000 [200:52:21<887:08:43, 25.62s/it]
 45%|████▍     | 100329/225000 [200:52:57<990:13:52, 28.59s/it]
 45%|████▍     | 100330/225000 [200:53:22<955:08:55, 27.58s/it]
 45%|████▍     | 100331/225000 [200:53:54<1002:43:11, 28.96s/it]
 45%|████▍     | 100332/225000 [200:54:19<959:13:23, 27.70s/it] 
 45%|████▍     | 100333/225000 [200:54:46<958:32:25, 27.68s/it]
 45%|████▍     | 100334/225000 [200:55:15<963:59:57, 27.84s/it]
 45%|████▍     | 100335/225000 [200:55:37<910:22:09, 26.29s/it]
 45%|████▍     | 100336/225000 [200:56:07<945:13:19, 27.30s/it]
 45%|████▍     | 100337/225000 [200:56:39<993:33:51, 28.69s/it]
 45%|████▍     | 100338/225000 [200:56:58<894:27:25, 25.83s/it]
 45%|████▍     | 100339/225000 [200:57:27<930:28:16, 26.87s/it]
 45%|████▍     | 100340/225000 [200:57:55<933:34:38, 26.96s/it]
 45%|████▍     | 100341/225000 [200:58:23<945:08:19, 27.29s/it]
 45%|████▍     | 100342/225000 [200:58:50<948:36:39, 27.39s/it]
 45%|████▍     | 100343/225000 [200:59:16<929:35:12, 26.85s/it]
 45%|████▍     | 100344/225000 [200:59:41<914:01:03, 26.40s/it]
 45%|████▍     | 100345/225000 [201:00:03<869:56:55, 25.12s/it]
 45%|████▍     | 100346/225000 [201:00:36<945:45:45, 27.31s/it]
 45%|████▍     | 100347/225000 [201:00:57<885:21:09, 25.57s/it]
 45%|████▍     | 100348/225000 [201:01:23<890:42:17, 25.72s/it]
 45%|████▍     | 100349/225000 [201:01:49<892:32:44, 25.78s/it]
 45%|████▍     | 100350/225000 [201:02:16<907:41:34, 26.21s/it]
                                                               
{'loss': '0.887', 'grad_norm': '1.025', 'learning_rate': '3.122e-05', 'epoch': '0.446', 'num_input_tokens_seen': 334964336, 'train_runtime': '7.237e+05', 'train_tokens_per_second': '462.8'}

 45%|████▍     | 100350/225000 [201:02:16<907:41:34, 26.21s/it]
 45%|████▍     | 100351/225000 [201:02:43<909:03:26, 26.25s/it]
 45%|████▍     | 100352/225000 [201:03:10<923:41:53, 26.68s/it]
 45%|████▍     | 100353/225000 [201:03:40<953:14:27, 27.53s/it]
 45%|████▍     | 100354/225000 [201:04:02<895:31:13, 25.86s/it]
 45%|████▍     | 100355/225000 [201:04:29<905:30:41, 26.15s/it]
 45%|████▍     | 100356/225000 [201:04:54<890:46:48, 25.73s/it]
 45%|████▍     | 100357/225000 [201:05:18<878:09:35, 25.36s/it]
 45%|████▍     | 100358/225000 [201:05:55<1002:03:04, 28.94s/it]
 45%|████▍     | 100359/225000 [201:06:25<1008:52:58, 29.14s/it]
 45%|████▍     | 100360/225000 [201:06:50<964:41:26, 27.86s/it] 
 45%|████▍     | 100361/225000 [201:07:19<973:55:21, 28.13s/it]
 45%|████▍     | 100362/225000 [201:07:41<910:37:08, 26.30s/it]
 45%|████▍     | 100363/225000 [201:08:08<921:45:59, 26.62s/it]
 45%|████▍     | 100364/225000 [201:08:34<920:45:46, 26.60s/it]
 45%|████▍     | 100365/225000 [201:09:01<923:59:41, 26.69s/it]
 45%|████▍     | 100366/225000 [201:09:28<928:01:39, 26.81s/it]
 45%|████▍     | 100367/225000 [201:09:57<943:13:57, 27.25s/it]
 45%|████▍     | 100368/225000 [201:10:23<936:07:21, 27.04s/it]
 45%|████▍     | 100369/225000 [201:10:46<890:17:51, 25.72s/it]
 45%|████▍     | 100370/225000 [201:11:18<954:41:11, 27.58s/it]
 45%|████▍     | 100371/225000 [201:11:48<982:05:14, 28.37s/it]
 45%|████▍     | 100372/225000 [201:12:15<968:05:10, 27.96s/it]
 45%|████▍     | 100373/225000 [201:12:45<984:28:09, 28.44s/it]
 45%|████▍     | 100374/225000 [201:13:17<1021:08:04, 29.50s/it]
 45%|████▍     | 100375/225000 [201:13:45<1006:49:26, 29.08s/it]
 45%|████▍     | 100376/225000 [201:14:09<957:24:31, 27.66s/it] 
 45%|████▍     | 100377/225000 [201:14:33<914:22:46, 26.41s/it]
 45%|████▍     | 100378/225000 [201:15:02<945:04:47, 27.30s/it]
 45%|████▍     | 100379/225000 [201:15:23<875:58:25, 25.30s/it]
 45%|████▍     | 100380/225000 [201:15:57<970:16:38, 28.03s/it]
 45%|████▍     | 100381/225000 [201:16:25<969:34:18, 28.01s/it]
 45%|████▍     | 100382/225000 [201:16:46<901:12:01, 26.03s/it]
 45%|████▍     | 100383/225000 [201:17:20<979:11:02, 28.29s/it]
 45%|████▍     | 100384/225000 [201:17:43<923:38:58, 26.68s/it]
 45%|████▍     | 100385/225000 [201:18:12<946:15:30, 27.34s/it]
 45%|████▍     | 100386/225000 [201:18:37<925:44:56, 26.74s/it]
 45%|████▍     | 100387/225000 [201:19:02<908:46:35, 26.25s/it]
 45%|████▍     | 100388/225000 [201:19:29<914:30:59, 26.42s/it]
 45%|████▍     | 100389/225000 [201:19:55<911:58:35, 26.35s/it]
 45%|████▍     | 100390/225000 [201:20:23<931:32:29, 26.91s/it]
 45%|████▍     | 100391/225000 [201:20:50<930:13:29, 26.87s/it]
 45%|████▍     | 100392/225000 [201:21:16<919:35:01, 26.57s/it]
 45%|████▍     | 100393/225000 [201:21:37<861:41:35, 24.90s/it]
 45%|████▍     | 100394/225000 [201:22:09<940:01:52, 27.16s/it]
 45%|████▍     | 100395/225000 [201:22:34<914:20:15, 26.42s/it]
 45%|████▍     | 100396/225000 [201:23:08<995:50:47, 28.77s/it]
 45%|████▍     | 100397/225000 [201:23:35<975:28:05, 28.18s/it]
 45%|████▍     | 100398/225000 [201:24:01<954:50:00, 27.59s/it]
 45%|████▍     | 100399/225000 [201:24:23<890:50:32, 25.74s/it]
 45%|████▍     | 100400/225000 [201:24:53<937:52:02, 27.10s/it]
                                                               
{'loss': '0.8965', 'grad_norm': '0.8128', 'learning_rate': '3.12e-05', 'epoch': '0.4462', 'num_input_tokens_seen': 335666480, 'train_runtime': '7.251e+05', 'train_tokens_per_second': '462.9'}

 45%|████▍     | 100400/225000 [201:24:53<937:52:02, 27.10s/it]
 45%|████▍     | 100401/225000 [201:25:21<949:58:53, 27.45s/it]
 45%|████▍     | 100402/225000 [201:25:43<894:21:17, 25.84s/it]
 45%|████▍     | 100403/225000 [201:26:17<975:54:43, 28.20s/it]
 45%|████▍     | 100404/225000 [201:26:46<982:32:38, 28.39s/it]
 45%|████▍     | 100405/225000 [201:27:13<964:57:43, 27.88s/it]
 45%|████▍     | 100406/225000 [201:27:38<934:46:28, 27.01s/it]
 45%|████▍     | 100407/225000 [201:28:06<946:54:17, 27.36s/it]
 45%|████▍     | 100408/225000 [201:28:24<848:26:59, 24.52s/it]
 45%|████▍     | 100409/225000 [201:28:43<790:00:22, 22.83s/it]
 45%|████▍     | 100410/225000 [201:29:14<873:58:58, 25.25s/it]
 45%|████▍     | 100411/225000 [201:29:43<914:20:35, 26.42s/it]
 45%|████▍     | 100412/225000 [201:30:15<975:05:15, 28.18s/it]
 45%|████▍     | 100413/225000 [201:30:42<966:58:28, 27.94s/it]
 45%|████▍     | 100414/225000 [201:31:06<919:58:00, 26.58s/it]
 45%|████▍     | 100415/225000 [201:31:35<951:13:59, 27.49s/it]
 45%|████▍     | 100416/225000 [201:32:00<923:02:21, 26.67s/it]
 45%|████▍     | 100417/225000 [201:32:23<884:30:54, 25.56s/it]
 45%|████▍     | 100418/225000 [201:32:56<959:22:12, 27.72s/it]
 45%|████▍     | 100419/225000 [201:33:23<951:51:40, 27.51s/it]
 45%|████▍     | 100420/225000 [201:33:46<906:05:27, 26.18s/it]
 45%|████▍     | 100421/225000 [201:34:12<908:19:16, 26.25s/it]
 45%|████▍     | 100422/225000 [201:34:38<902:49:47, 26.09s/it]
 45%|████▍     | 100423/225000 [201:35:07<936:36:39, 27.07s/it]
 45%|████▍     | 100424/225000 [201:35:38<976:33:30, 28.22s/it]
 45%|████▍     | 100425/225000 [201:36:09<999:28:43, 28.88s/it]
 45%|████▍     | 100426/225000 [201:36:31<929:09:47, 26.85s/it]
 45%|████▍     | 100427/225000 [201:36:49<841:15:38, 24.31s/it]
 45%|████▍     | 100428/225000 [201:37:15<852:23:11, 24.63s/it]
 45%|████▍     | 100429/225000 [201:37:41<871:15:54, 25.18s/it]
 45%|████▍     | 100430/225000 [201:38:13<938:52:01, 27.13s/it]
 45%|████▍     | 100431/225000 [201:38:42<957:05:42, 27.66s/it]
 45%|████▍     | 100432/225000 [201:39:11<972:51:13, 28.12s/it]
 45%|████▍     | 100433/225000 [201:39:36<941:34:03, 27.21s/it]
 45%|████▍     | 100434/225000 [201:40:03<941:47:12, 27.22s/it]
 45%|████▍     | 100435/225000 [201:40:24<877:25:45, 25.36s/it]
 45%|████▍     | 100436/225000 [201:40:49<875:27:29, 25.30s/it]
 45%|████▍     | 100437/225000 [201:41:21<942:36:53, 27.24s/it]
 45%|████▍     | 100438/225000 [201:41:44<899:40:07, 26.00s/it]
 45%|████▍     | 100439/225000 [201:42:11<906:40:00, 26.20s/it]
 45%|████▍     | 100440/225000 [201:42:35<882:14:49, 25.50s/it]
 45%|████▍     | 100441/225000 [201:43:09<974:26:56, 28.16s/it]
 45%|████▍     | 100442/225000 [201:43:33<934:47:30, 27.02s/it]
 45%|████▍     | 100443/225000 [201:44:06<994:06:48, 28.73s/it]
 45%|████▍     | 100444/225000 [201:44:31<950:26:51, 27.47s/it]
 45%|████▍     | 100445/225000 [201:45:01<979:22:00, 28.31s/it]
 45%|████▍     | 100446/225000 [201:45:26<945:16:47, 27.32s/it]
 45%|████▍     | 100447/225000 [201:45:49<897:27:54, 25.94s/it]
 45%|████▍     | 100448/225000 [201:46:16<914:13:17, 26.42s/it]
 45%|████▍     | 100449/225000 [201:46:40<887:12:32, 25.64s/it]
 45%|████▍     | 100450/225000 [201:47:00<827:56:03, 23.93s/it]
                                                               
{'loss': '0.9032', 'grad_norm': '0.9219', 'learning_rate': '3.118e-05', 'epoch': '0.4464', 'num_input_tokens_seen': 336342816, 'train_runtime': '7.264e+05', 'train_tokens_per_second': '463'}

 45%|████▍     | 100450/225000 [201:47:00<827:56:03, 23.93s/it]
 45%|████▍     | 100451/225000 [201:47:34<928:16:32, 26.83s/it]
 45%|████▍     | 100452/225000 [201:48:07<999:21:50, 28.89s/it]
 45%|████▍     | 100453/225000 [201:48:33<965:20:12, 27.90s/it]
 45%|████▍     | 100454/225000 [201:49:00<954:42:15, 27.60s/it]
 45%|████▍     | 100455/225000 [201:49:28<963:18:37, 27.84s/it]
 45%|████▍     | 100456/225000 [201:49:54<944:02:14, 27.29s/it]
 45%|████▍     | 100457/225000 [201:50:22<945:30:02, 27.33s/it]
 45%|████▍     | 100458/225000 [201:50:47<922:36:20, 26.67s/it]
 45%|████▍     | 100459/225000 [201:51:13<912:43:04, 26.38s/it]
 45%|████▍     | 100460/225000 [201:51:42<947:43:07, 27.40s/it]
 45%|████▍     | 100461/225000 [201:52:09<938:11:32, 27.12s/it]
 45%|████▍     | 100462/225000 [201:52:34<921:31:44, 26.64s/it]
 45%|████▍     | 100463/225000 [201:53:04<949:50:40, 27.46s/it]
 45%|████▍     | 100464/225000 [201:53:35<992:58:01, 28.70s/it]
 45%|████▍     | 100465/225000 [201:54:03<985:18:45, 28.48s/it]
 45%|████▍     | 100466/225000 [201:54:26<927:12:05, 26.80s/it]
 45%|████▍     | 100467/225000 [201:54:54<936:44:47, 27.08s/it]
 45%|████▍     | 100468/225000 [201:55:21<941:34:26, 27.22s/it]
 45%|████▍     | 100469/225000 [201:55:54<1000:50:04, 28.93s/it]
 45%|████▍     | 100470/225000 [201:56:25<1015:16:18, 29.35s/it]
 45%|████▍     | 100471/225000 [201:56:54<1018:57:16, 29.46s/it]
 45%|████▍     | 100472/225000 [201:57:24<1024:02:22, 29.60s/it]
 45%|████▍     | 100473/225000 [201:57:48<963:56:06, 27.87s/it] 
 45%|████▍     | 100474/225000 [201:58:17<975:28:25, 28.20s/it]
 45%|████▍     | 100475/225000 [201:58:38<903:53:16, 26.13s/it]
 45%|████▍     | 100476/225000 [201:59:09<952:34:10, 27.54s/it]
 45%|████▍     | 100477/225000 [201:59:42<1006:11:37, 29.09s/it]
 45%|████▍     | 100478/225000 [202:00:03<927:17:12, 26.81s/it] 
 45%|████▍     | 100479/225000 [202:00:36<986:55:17, 28.53s/it]
 45%|████▍     | 100480/225000 [202:00:54<878:40:57, 25.40s/it]
 45%|████▍     | 100481/225000 [202:01:22<904:30:15, 26.15s/it]
 45%|████▍     | 100482/225000 [202:01:52<949:46:23, 27.46s/it]
 45%|████▍     | 100483/225000 [202:02:16<908:42:10, 26.27s/it]
 45%|████▍     | 100484/225000 [202:02:46<945:57:27, 27.35s/it]
 45%|████▍     | 100485/225000 [202:03:08<895:38:40, 25.90s/it]
 45%|████▍     | 100486/225000 [202:03:36<916:08:08, 26.49s/it]
 45%|████▍     | 100487/225000 [202:04:01<901:32:43, 26.07s/it]
 45%|████▍     | 100488/225000 [202:04:32<946:04:06, 27.35s/it]
 45%|████▍     | 100489/225000 [202:04:57<924:48:10, 26.74s/it]
 45%|████▍     | 100490/225000 [202:05:29<978:13:02, 28.28s/it]
 45%|████▍     | 100491/225000 [202:05:55<959:11:01, 27.73s/it]
 45%|████▍     | 100492/225000 [202:06:17<895:38:21, 25.90s/it]
 45%|████▍     | 100493/225000 [202:06:50<965:54:50, 27.93s/it]
 45%|████▍     | 100494/225000 [202:07:20<994:57:13, 28.77s/it]
 45%|████▍     | 100495/225000 [202:07:40<905:29:16, 26.18s/it]
 45%|████▍     | 100496/225000 [202:08:00<838:53:01, 24.26s/it]
 45%|████▍     | 100497/225000 [202:08:32<920:54:40, 26.63s/it]
 45%|████▍     | 100498/225000 [202:09:02<957:01:13, 27.67s/it]
 45%|████▍     | 100499/225000 [202:09:29<945:42:47, 27.35s/it]
 45%|████▍     | 100500/225000 [202:09:53<910:14:10, 26.32s/it]
                                                               
{'loss': '0.8901', 'grad_norm': '1.05', 'learning_rate': '3.116e-05', 'epoch': '0.4467', 'num_input_tokens_seen': 337041280, 'train_runtime': '7.278e+05', 'train_tokens_per_second': '463.1'}

 45%|████▍     | 100500/225000 [202:09:53<910:14:10, 26.32s/it]
 45%|████▍     | 100501/225000 [202:10:20<918:59:24, 26.57s/it]
 45%|████▍     | 100502/225000 [202:10:45<902:57:53, 26.11s/it]
 45%|████▍     | 100503/225000 [202:11:17<958:15:07, 27.71s/it]
 45%|████▍     | 100504/225000 [202:11:44<956:20:46, 27.65s/it]
 45%|████▍     | 100505/225000 [202:12:11<951:33:25, 27.52s/it]
 45%|████▍     | 100506/225000 [202:12:42<985:14:20, 28.49s/it]
 45%|████▍     | 100507/225000 [202:13:03<910:14:56, 26.32s/it]
 45%|████▍     | 100508/225000 [202:13:32<939:01:48, 27.15s/it]
 45%|████▍     | 100509/225000 [202:13:59<936:19:22, 27.08s/it]
 45%|████▍     | 100510/225000 [202:14:24<914:30:05, 26.45s/it]
 45%|████▍     | 100511/225000 [202:14:53<942:23:34, 27.25s/it]
 45%|████▍     | 100512/225000 [202:15:14<873:20:26, 25.26s/it]
 45%|████▍     | 100513/225000 [202:15:44<918:00:52, 26.55s/it]
 45%|████▍     | 100514/225000 [202:16:11<926:50:35, 26.80s/it]
 45%|████▍     | 100515/225000 [202:16:36<909:52:03, 26.31s/it]
 45%|████▍     | 100516/225000 [202:17:05<934:47:22, 27.03s/it]
 45%|████▍     | 100517/225000 [202:17:36<977:27:42, 28.27s/it]
 45%|████▍     | 100518/225000 [202:18:03<960:06:35, 27.77s/it]
 45%|████▍     | 100519/225000 [202:18:31<964:08:41, 27.88s/it]
 45%|████▍     | 100520/225000 [202:18:54<917:12:46, 26.53s/it]
 45%|████▍     | 100521/225000 [202:19:23<938:15:51, 27.14s/it]
 45%|████▍     | 100522/225000 [202:19:48<922:46:50, 26.69s/it]
 45%|████▍     | 100523/225000 [202:20:13<905:07:58, 26.18s/it]
 45%|████▍     | 100524/225000 [202:20:41<920:47:25, 26.63s/it]
 45%|████▍     | 100525/225000 [202:21:10<947:14:57, 27.40s/it]
 45%|████▍     | 100526/225000 [202:21:33<898:14:50, 25.98s/it]
 45%|████▍     | 100527/225000 [202:21:56<873:36:24, 25.27s/it]
 45%|████▍     | 100528/225000 [202:22:27<931:07:57, 26.93s/it]
 45%|████▍     | 100529/225000 [202:22:52<905:45:48, 26.20s/it]
 45%|████▍     | 100530/225000 [202:23:21<938:57:42, 27.16s/it]
 45%|████▍     | 100531/225000 [202:23:47<929:46:30, 26.89s/it]
 45%|████▍     | 100532/225000 [202:24:16<949:33:03, 27.46s/it]
 45%|████▍     | 100533/225000 [202:24:50<1015:26:30, 29.37s/it]
 45%|████▍     | 100534/225000 [202:25:13<945:04:07, 27.33s/it] 
 45%|████▍     | 100535/225000 [202:25:46<1007:41:58, 29.15s/it]
 45%|████▍     | 100536/225000 [202:26:14<994:18:31, 28.76s/it] 
 45%|████▍     | 100537/225000 [202:26:43<996:21:40, 28.82s/it]
 45%|████▍     | 100538/225000 [202:27:03<911:33:28, 26.37s/it]
 45%|████▍     | 100539/225000 [202:27:33<948:26:07, 27.43s/it]
 45%|████▍     | 100540/225000 [202:28:04<986:01:10, 28.52s/it]
 45%|████▍     | 100541/225000 [202:28:34<992:49:21, 28.72s/it]
 45%|████▍     | 100542/225000 [202:29:06<1025:46:57, 29.67s/it]
 45%|████▍     | 100543/225000 [202:29:26<934:05:41, 27.02s/it] 
 45%|████▍     | 100544/225000 [202:29:50<900:48:47, 26.06s/it]
 45%|████▍     | 100545/225000 [202:30:15<884:57:27, 25.60s/it]
 45%|████▍     | 100546/225000 [202:30:42<899:19:15, 26.01s/it]
 45%|████▍     | 100547/225000 [202:30:59<807:51:59, 23.37s/it]
 45%|████▍     | 100548/225000 [202:31:32<906:42:42, 26.23s/it]
 45%|████▍     | 100549/225000 [202:31:55<873:01:52, 25.25s/it]
 45%|████▍     | 100550/225000 [202:32:27<945:33:10, 27.35s/it]
                                                               
{'loss': '0.887', 'grad_norm': '0.8374', 'learning_rate': '3.115e-05', 'epoch': '0.4469', 'num_input_tokens_seen': 337734112, 'train_runtime': '7.291e+05', 'train_tokens_per_second': '463.2'}

 45%|████▍     | 100550/225000 [202:32:27<945:33:10, 27.35s/it]
 45%|████▍     | 100551/225000 [202:32:58<986:56:01, 28.55s/it]
 45%|████▍     | 100552/225000 [202:33:20<914:10:58, 26.45s/it]
 45%|████▍     | 100553/225000 [202:33:51<958:59:15, 27.74s/it]
 45%|████▍     | 100554/225000 [202:34:17<947:00:24, 27.40s/it]
 45%|████▍     | 100555/225000 [202:34:42<920:02:59, 26.62s/it]
 45%|████▍     | 100556/225000 [202:35:12<956:01:21, 27.66s/it]
 45%|████▍     | 100557/225000 [202:35:35<905:26:56, 26.19s/it]
 45%|████▍     | 100558/225000 [202:36:02<914:08:47, 26.45s/it]
 45%|████▍     | 100559/225000 [202:36:29<920:21:16, 26.63s/it]
 45%|████▍     | 100560/225000 [202:36:59<956:00:32, 27.66s/it]
 45%|████▍     | 100561/225000 [202:37:16<844:19:33, 24.43s/it]
 45%|████▍     | 100562/225000 [202:37:49<936:25:12, 27.09s/it]
 45%|████▍     | 100563/225000 [202:38:15<922:38:35, 26.69s/it]
 45%|████▍     | 100564/225000 [202:38:41<910:26:04, 26.34s/it]
 45%|████▍     | 100565/225000 [202:39:06<899:16:47, 26.02s/it]
 45%|████▍     | 100566/225000 [202:39:26<837:33:23, 24.23s/it]
 45%|████▍     | 100567/225000 [202:39:49<822:18:36, 23.79s/it]
 45%|████▍     | 100568/225000 [202:40:13<830:35:28, 24.03s/it]
 45%|████▍     | 100569/225000 [202:40:43<887:39:46, 25.68s/it]
 45%|████▍     | 100570/225000 [202:41:08<879:00:01, 25.43s/it]
 45%|████▍     | 100571/225000 [202:41:31<861:27:47, 24.92s/it]
 45%|████▍     | 100572/225000 [202:42:02<924:43:17, 26.75s/it]
 45%|████▍     | 100573/225000 [202:42:26<895:12:37, 25.90s/it]
 45%|████▍     | 100574/225000 [202:43:00<980:36:05, 28.37s/it]
 45%|████▍     | 100575/225000 [202:43:33<1027:35:25, 29.73s/it]
 45%|████▍     | 100576/225000 [202:43:56<952:33:45, 27.56s/it] 
 45%|████▍     | 100577/225000 [202:44:26<984:16:33, 28.48s/it]
 45%|████▍     | 100578/225000 [202:44:56<999:24:09, 28.92s/it]
 45%|████▍     | 100579/225000 [202:45:20<940:20:56, 27.21s/it]
 45%|████▍     | 100580/225000 [202:45:48<956:24:58, 27.67s/it]
 45%|████▍     | 100581/225000 [202:46:14<937:12:16, 27.12s/it]
 45%|████▍     | 100582/225000 [202:46:41<934:44:48, 27.05s/it]
 45%|████▍     | 100583/225000 [202:47:02<868:50:33, 25.14s/it]
 45%|████▍     | 100584/225000 [202:47:31<907:15:50, 26.25s/it]
 45%|████▍     | 100585/225000 [202:47:56<903:42:34, 26.15s/it]
 45%|████▍     | 100586/225000 [202:48:26<936:23:01, 27.09s/it]
 45%|████▍     | 100587/225000 [202:48:57<983:08:36, 28.45s/it]
 45%|████▍     | 100588/225000 [202:49:24<969:04:20, 28.04s/it]
 45%|████▍     | 100589/225000 [202:49:51<952:42:00, 27.57s/it]
 45%|████▍     | 100590/225000 [202:50:22<984:09:43, 28.48s/it]
 45%|████▍     | 100591/225000 [202:50:48<960:18:29, 27.79s/it]
 45%|████▍     | 100592/225000 [202:51:15<949:56:50, 27.49s/it]
 45%|████▍     | 100593/225000 [202:51:41<940:25:49, 27.21s/it]
 45%|████▍     | 100594/225000 [202:52:13<991:00:19, 28.68s/it]
 45%|████▍     | 100595/225000 [202:52:36<932:37:55, 26.99s/it]
 45%|████▍     | 100596/225000 [202:53:09<994:37:56, 28.78s/it]
 45%|████▍     | 100597/225000 [202:53:35<966:21:06, 27.96s/it]
 45%|████▍     | 100598/225000 [202:54:06<993:01:19, 28.74s/it]
 45%|████▍     | 100599/225000 [202:54:34<985:02:26, 28.51s/it]
 45%|████▍     | 100600/225000 [202:54:57<929:41:59, 26.90s/it]
                                                               
{'loss': '0.88', 'grad_norm': '0.9148', 'learning_rate': '3.113e-05', 'epoch': '0.4471', 'num_input_tokens_seen': 338421976, 'train_runtime': '7.305e+05', 'train_tokens_per_second': '463.3'}

 45%|████▍     | 100600/225000 [202:54:57<929:41:59, 26.90s/it]
 45%|████▍     | 100601/225000 [202:55:24<931:07:12, 26.95s/it]
 45%|████▍     | 100602/225000 [202:55:57<989:23:33, 28.63s/it]
 45%|████▍     | 100603/225000 [202:56:23<963:57:10, 27.90s/it]
 45%|████▍     | 100604/225000 [202:56:52<976:43:49, 28.27s/it]
 45%|████▍     | 100605/225000 [202:57:18<950:31:16, 27.51s/it]
 45%|████▍     | 100606/225000 [202:57:52<1019:26:27, 29.50s/it]
 45%|████▍     | 100607/225000 [202:58:17<979:48:29, 28.36s/it] 
 45%|████▍     | 100608/225000 [202:58:39<911:54:42, 26.39s/it]
 45%|████▍     | 100609/225000 [202:59:03<882:39:57, 25.55s/it]
 45%|████▍     | 100610/225000 [202:59:26<856:55:11, 24.80s/it]
 45%|████▍     | 100611/225000 [202:59:59<938:37:24, 27.17s/it]
 45%|████▍     | 100612/225000 [203:00:15<828:13:28, 23.97s/it]
 45%|████▍     | 100613/225000 [203:00:50<945:47:44, 27.37s/it]
 45%|████▍     | 100614/225000 [203:01:14<907:15:02, 26.26s/it]
 45%|████▍     | 100615/225000 [203:01:43<938:50:53, 27.17s/it]
 45%|████▍     | 100616/225000 [203:02:08<911:45:31, 26.39s/it]
 45%|████▍     | 100617/225000 [203:02:39<963:29:37, 27.89s/it]
 45%|████▍     | 100618/225000 [203:03:08<974:15:48, 28.20s/it]
 45%|████▍     | 100619/225000 [203:03:34<946:02:13, 27.38s/it]
 45%|████▍     | 100620/225000 [203:04:03<964:32:17, 27.92s/it]
 45%|████▍     | 100621/225000 [203:04:25<909:18:44, 26.32s/it]
 45%|████▍     | 100622/225000 [203:04:53<917:12:39, 26.55s/it]
 45%|████▍     | 100623/225000 [203:05:20<929:06:03, 26.89s/it]
 45%|████▍     | 100624/225000 [203:05:45<910:51:22, 26.36s/it]
 45%|████▍     | 100625/225000 [203:06:12<914:03:08, 26.46s/it]
 45%|████▍     | 100626/225000 [203:06:37<898:42:22, 26.01s/it]
 45%|████▍     | 100627/225000 [203:07:07<942:35:12, 27.28s/it]
 45%|████▍     | 100628/225000 [203:07:36<957:32:13, 27.72s/it]
 45%|████▍     | 100629/225000 [203:08:05<968:13:53, 28.03s/it]
 45%|████▍     | 100630/225000 [203:08:31<951:45:36, 27.55s/it]
 45%|████▍     | 100631/225000 [203:08:57<931:53:29, 26.97s/it]
 45%|████▍     | 100632/225000 [203:09:27<970:20:08, 28.09s/it]
 45%|████▍     | 100633/225000 [203:09:50<914:05:35, 26.46s/it]
 45%|████▍     | 100634/225000 [203:10:21<958:31:27, 27.75s/it]
 45%|████▍     | 100635/225000 [203:10:51<986:52:06, 28.57s/it]
 45%|████▍     | 100636/225000 [203:11:14<929:03:20, 26.89s/it]
 45%|████▍     | 100637/225000 [203:11:38<893:59:29, 25.88s/it]
 45%|████▍     | 100638/225000 [203:12:04<891:58:14, 25.82s/it]
 45%|████▍     | 100639/225000 [203:12:33<933:50:26, 27.03s/it]
 45%|████▍     | 100640/225000 [203:13:02<946:58:23, 27.41s/it]
 45%|████▍     | 100641/225000 [203:13:25<899:06:03, 26.03s/it]
 45%|████▍     | 100642/225000 [203:13:49<885:53:45, 25.65s/it]
 45%|████▍     | 100643/225000 [203:14:16<899:37:49, 26.04s/it]
 45%|████▍     | 100644/225000 [203:14:44<920:48:37, 26.66s/it]
 45%|████▍     | 100645/225000 [203:15:12<933:19:44, 27.02s/it]
 45%|████▍     | 100646/225000 [203:15:38<923:20:29, 26.73s/it]
 45%|████▍     | 100647/225000 [203:16:04<911:23:05, 26.38s/it]
 45%|████▍     | 100648/225000 [203:16:32<930:53:17, 26.95s/it]
 45%|████▍     | 100649/225000 [203:16:55<892:10:39, 25.83s/it]
 45%|████▍     | 100650/225000 [203:17:24<925:21:44, 26.79s/it]
                                                               
{'loss': '0.8664', 'grad_norm': '0.8916', 'learning_rate': '3.111e-05', 'epoch': '0.4473', 'num_input_tokens_seen': 339109336, 'train_runtime': '7.318e+05', 'train_tokens_per_second': '463.4'}

 45%|████▍     | 100650/225000 [203:17:24<925:21:44, 26.79s/it]
 45%|████▍     | 100651/225000 [203:17:47<886:54:50, 25.68s/it]
 45%|████▍     | 100652/225000 [203:18:14<899:27:53, 26.04s/it]
 45%|████▍     | 100653/225000 [203:18:41<909:14:27, 26.32s/it]
 45%|████▍     | 100654/225000 [203:19:11<939:26:46, 27.20s/it]
 45%|████▍     | 100655/225000 [203:19:40<963:00:45, 27.88s/it]
 45%|████▍     | 100656/225000 [203:20:10<986:04:46, 28.55s/it]
 45%|████▍     | 100657/225000 [203:20:37<965:23:22, 27.95s/it]
 45%|████▍     | 100658/225000 [203:20:55<869:37:17, 25.18s/it]
 45%|████▍     | 100659/225000 [203:21:21<876:42:34, 25.38s/it]
 45%|████▍     | 100660/225000 [203:21:49<899:08:29, 26.03s/it]
 45%|████▍     | 100661/225000 [203:22:15<896:41:24, 25.96s/it]
 45%|████▍     | 100662/225000 [203:22:44<930:11:03, 26.93s/it]
 45%|████▍     | 100663/225000 [203:23:10<919:29:22, 26.62s/it]
 45%|████▍     | 100664/225000 [203:23:38<940:26:25, 27.23s/it]
 45%|████▍     | 100665/225000 [203:24:02<908:17:46, 26.30s/it]
 45%|████▍     | 100666/225000 [203:24:30<916:35:39, 26.54s/it]
 45%|████▍     | 100667/225000 [203:25:00<954:16:55, 27.63s/it]
 45%|████▍     | 100668/225000 [203:25:20<882:22:28, 25.55s/it]
 45%|████▍     | 100669/225000 [203:25:54<969:08:13, 28.06s/it]
 45%|████▍     | 100670/225000 [203:26:19<936:53:01, 27.13s/it]
 45%|████▍     | 100671/225000 [203:26:48<948:27:10, 27.46s/it]
 45%|████▍     | 100672/225000 [203:27:14<936:43:53, 27.12s/it]
 45%|████▍     | 100673/225000 [203:27:39<916:12:39, 26.53s/it]
 45%|████▍     | 100674/225000 [203:28:08<945:59:51, 27.39s/it]
 45%|████▍     | 100675/225000 [203:28:36<945:42:46, 27.38s/it]
 45%|████▍     | 100676/225000 [203:29:03<940:08:28, 27.22s/it]
 45%|████▍     | 100677/225000 [203:29:28<924:22:09, 26.77s/it]
 45%|████▍     | 100678/225000 [203:29:56<937:27:28, 27.15s/it]
 45%|████▍     | 100679/225000 [203:30:22<918:00:04, 26.58s/it]
 45%|████▍     | 100680/225000 [203:30:52<960:54:24, 27.83s/it]
 45%|████▍     | 100681/225000 [203:31:17<923:30:48, 26.74s/it]
 45%|████▍     | 100682/225000 [203:31:42<907:13:38, 26.27s/it]
 45%|████▍     | 100683/225000 [203:32:08<906:28:23, 26.25s/it]
 45%|████▍     | 100684/225000 [203:32:37<934:12:58, 27.05s/it]
 45%|████▍     | 100685/225000 [203:32:59<884:16:25, 25.61s/it]
 45%|████▍     | 100686/225000 [203:33:30<943:53:55, 27.33s/it]
 45%|████▍     | 100687/225000 [203:33:55<919:00:42, 26.61s/it]
 45%|████▍     | 100688/225000 [203:34:23<928:45:02, 26.90s/it]
 45%|████▍     | 100689/225000 [203:34:49<915:04:03, 26.50s/it]
 45%|████▍     | 100690/225000 [203:35:16<929:39:18, 26.92s/it]
 45%|████▍     | 100691/225000 [203:35:42<913:40:47, 26.46s/it]
 45%|████▍     | 100692/225000 [203:36:08<911:25:40, 26.40s/it]
 45%|████▍     | 100693/225000 [203:36:36<924:26:43, 26.77s/it]
 45%|████▍     | 100694/225000 [203:37:04<937:51:33, 27.16s/it]
 45%|████▍     | 100695/225000 [203:37:31<938:22:12, 27.18s/it]
 45%|████▍     | 100696/225000 [203:37:59<950:10:14, 27.52s/it]
 45%|████▍     | 100697/225000 [203:38:25<926:32:28, 26.83s/it]
 45%|████▍     | 100698/225000 [203:38:50<911:50:14, 26.41s/it]
 45%|████▍     | 100699/225000 [203:39:17<916:53:46, 26.56s/it]
 45%|████▍     | 100700/225000 [203:39:49<974:10:39, 28.21s/it]
                                                               
{'loss': '0.8712', 'grad_norm': '0.995', 'learning_rate': '3.109e-05', 'epoch': '0.4476', 'num_input_tokens_seen': 339792712, 'train_runtime': '7.332e+05', 'train_tokens_per_second': '463.4'}

 45%|████▍     | 100700/225000 [203:39:49<974:10:39, 28.21s/it]
 45%|████▍     | 100701/225000 [203:40:13<935:48:43, 27.10s/it]
 45%|████▍     | 100702/225000 [203:40:39<916:40:47, 26.55s/it]
 45%|████▍     | 100703/225000 [203:41:00<865:23:22, 25.06s/it]
 45%|████▍     | 100704/225000 [203:41:23<844:31:50, 24.46s/it]
 45%|████▍     | 100705/225000 [203:41:55<920:41:29, 26.67s/it]
 45%|████▍     | 100706/225000 [203:42:24<940:58:30, 27.25s/it]
 45%|████▍     | 100707/225000 [203:42:50<933:17:09, 27.03s/it]
 45%|████▍     | 100708/225000 [203:43:18<942:11:42, 27.29s/it]
 45%|████▍     | 100709/225000 [203:43:45<934:45:18, 27.07s/it]
 45%|████▍     | 100710/225000 [203:44:11<928:09:42, 26.88s/it]
 45%|████▍     | 100711/225000 [203:44:31<859:07:49, 24.88s/it]
 45%|████▍     | 100712/225000 [203:45:02<915:34:18, 26.52s/it]
 45%|████▍     | 100713/225000 [203:45:34<976:43:46, 28.29s/it]
 45%|████▍     | 100714/225000 [203:46:05<1006:40:31, 29.16s/it]
 45%|████▍     | 100715/225000 [203:46:26<916:46:27, 26.55s/it] 
 45%|████▍     | 100716/225000 [203:46:59<982:37:26, 28.46s/it]
 45%|████▍     | 100717/225000 [203:47:23<938:14:32, 27.18s/it]
 45%|████▍     | 100718/225000 [203:47:49<925:58:58, 26.82s/it]
 45%|████▍     | 100719/225000 [203:48:19<956:22:02, 27.70s/it]
 45%|████▍     | 100720/225000 [203:48:50<991:20:25, 28.72s/it]
 45%|████▍     | 100721/225000 [203:49:20<1011:39:19, 29.30s/it]
 45%|████▍     | 100722/225000 [203:49:42<935:00:23, 27.08s/it] 
 45%|████▍     | 100723/225000 [203:50:12<956:08:47, 27.70s/it]
 45%|████▍     | 100724/225000 [203:50:42<983:30:48, 28.49s/it]
 45%|████▍     | 100725/225000 [203:50:58<853:33:01, 24.73s/it]
 45%|████▍     | 100726/225000 [203:51:29<919:21:09, 26.63s/it]
 45%|████▍     | 100727/225000 [203:51:52<879:22:09, 25.47s/it]
 45%|████▍     | 100728/225000 [203:52:22<925:04:15, 26.80s/it]
 45%|████▍     | 100729/225000 [203:52:48<920:33:04, 26.67s/it]
 45%|████▍     | 100730/225000 [203:53:12<890:48:28, 25.81s/it]
 45%|████▍     | 100731/225000 [203:53:40<918:27:58, 26.61s/it]
 45%|████▍     | 100732/225000 [203:54:11<961:55:08, 27.87s/it]
 45%|████▍     | 100733/225000 [203:54:40<973:52:11, 28.21s/it]
 45%|████▍     | 100734/225000 [203:55:10<988:56:25, 28.65s/it]
 45%|████▍     | 100735/225000 [203:55:34<945:08:33, 27.38s/it]
 45%|████▍     | 100736/225000 [203:56:05<981:53:48, 28.45s/it]
 45%|████▍     | 100737/225000 [203:56:34<984:41:46, 28.53s/it]
 45%|████▍     | 100738/225000 [203:57:03<997:09:49, 28.89s/it]
 45%|████▍     | 100739/225000 [203:57:29<966:16:47, 27.99s/it]
 45%|████▍     | 100740/225000 [203:58:00<988:37:55, 28.64s/it]
 45%|████▍     | 100741/225000 [203:58:24<945:46:45, 27.40s/it]
 45%|████▍     | 100742/225000 [203:58:44<871:46:53, 25.26s/it]
 45%|████▍     | 100743/225000 [203:59:09<868:53:24, 25.17s/it]
 45%|████▍     | 100744/225000 [203:59:34<868:58:35, 25.18s/it]
 45%|████▍     | 100745/225000 [204:00:03<901:11:16, 26.11s/it]
 45%|████▍     | 100746/225000 [204:00:26<866:37:02, 25.11s/it]
 45%|████▍     | 100747/225000 [204:00:52<882:01:07, 25.55s/it]
 45%|████▍     | 100748/225000 [204:01:19<896:19:17, 25.97s/it]
 45%|████▍     | 100749/225000 [204:01:44<885:53:33, 25.67s/it]
 45%|████▍     | 100750/225000 [204:02:08<865:56:24, 25.09s/it]
                                                               
{'loss': '0.8993', 'grad_norm': '0.9807', 'learning_rate': '3.108e-05', 'epoch': '0.4478', 'num_input_tokens_seen': 340488400, 'train_runtime': '7.345e+05', 'train_tokens_per_second': '463.5'}

 45%|████▍     | 100750/225000 [204:02:08<865:56:24, 25.09s/it]
 45%|████▍     | 100751/225000 [204:02:37<905:40:03, 26.24s/it]
 45%|████▍     | 100752/225000 [204:02:59<866:11:12, 25.10s/it]
 45%|████▍     | 100753/225000 [204:03:28<909:53:03, 26.36s/it]
 45%|████▍     | 100754/225000 [204:03:51<867:21:53, 25.13s/it]
 45%|████▍     | 100755/225000 [204:04:08<789:12:27, 22.87s/it]
 45%|████▍     | 100756/225000 [204:04:36<842:08:52, 24.40s/it]
 45%|████▍     | 100757/225000 [204:05:05<883:54:10, 25.61s/it]
 45%|████▍     | 100758/225000 [204:05:28<858:47:29, 24.88s/it]
 45%|████▍     | 100759/225000 [204:05:53<865:45:19, 25.09s/it]
 45%|████▍     | 100760/225000 [204:06:19<872:01:54, 25.27s/it]
 45%|████▍     | 100761/225000 [204:06:42<852:16:48, 24.70s/it]
 45%|████▍     | 100762/225000 [204:07:10<884:58:08, 25.64s/it]
 45%|████▍     | 100763/225000 [204:07:41<938:31:35, 27.20s/it]
 45%|████▍     | 100764/225000 [204:08:08<934:09:21, 27.07s/it]
 45%|████▍     | 100765/225000 [204:08:30<882:58:59, 25.59s/it]
 45%|████▍     | 100766/225000 [204:08:52<846:49:58, 24.54s/it]
 45%|████▍     | 100767/225000 [204:09:23<913:04:32, 26.46s/it]
 45%|████▍     | 100768/225000 [204:09:51<926:17:36, 26.84s/it]
 45%|████▍     | 100769/225000 [204:10:17<924:24:51, 26.79s/it]
 45%|████▍     | 100770/225000 [204:10:41<892:23:57, 25.86s/it]
 45%|████▍     | 100771/225000 [204:11:08<900:50:15, 26.11s/it]
 45%|████▍     | 100772/225000 [204:11:37<931:31:20, 26.99s/it]
 45%|████▍     | 100773/225000 [204:11:56<846:56:04, 24.54s/it]
 45%|████▍     | 100774/225000 [204:12:18<818:07:18, 23.71s/it]
 45%|████▍     | 100775/225000 [204:12:47<879:47:09, 25.50s/it]
 45%|████▍     | 100776/225000 [204:13:05<797:43:47, 23.12s/it]
 45%|████▍     | 100777/225000 [204:13:32<844:36:37, 24.48s/it]
 45%|████▍     | 100778/225000 [204:14:00<876:18:09, 25.40s/it]
 45%|████▍     | 100779/225000 [204:14:27<893:14:54, 25.89s/it]
 45%|████▍     | 100780/225000 [204:14:54<909:54:33, 26.37s/it]
 45%|████▍     | 100781/225000 [204:15:26<964:09:34, 27.94s/it]
 45%|████▍     | 100782/225000 [204:15:55<972:57:16, 28.20s/it]
 45%|████▍     | 100783/225000 [204:16:20<942:58:04, 27.33s/it]
 45%|████▍     | 100784/225000 [204:16:49<955:25:35, 27.69s/it]
 45%|████▍     | 100785/225000 [204:17:09<874:17:35, 25.34s/it]
 45%|████▍     | 100786/225000 [204:17:40<939:35:00, 27.23s/it]
 45%|████▍     | 100787/225000 [204:18:08<947:07:56, 27.45s/it]
 45%|████▍     | 100788/225000 [204:18:34<932:41:46, 27.03s/it]
 45%|████▍     | 100789/225000 [204:19:04<956:36:24, 27.73s/it]
 45%|████▍     | 100790/225000 [204:19:33<974:03:11, 28.23s/it]
 45%|████▍     | 100791/225000 [204:19:58<945:11:18, 27.39s/it]
 45%|████▍     | 100792/225000 [204:20:29<973:37:01, 28.22s/it]
 45%|████▍     | 100793/225000 [204:20:53<939:03:39, 27.22s/it]
 45%|████▍     | 100794/225000 [204:21:19<922:30:31, 26.74s/it]
 45%|████▍     | 100795/225000 [204:21:36<817:22:50, 23.69s/it]
 45%|████▍     | 100796/225000 [204:22:09<914:15:25, 26.50s/it]
 45%|████▍     | 100797/225000 [204:22:32<885:49:42, 25.68s/it]
 45%|████▍     | 100798/225000 [204:22:54<839:44:51, 24.34s/it]
 45%|████▍     | 100799/225000 [204:23:24<903:40:22, 26.19s/it]
 45%|████▍     | 100800/225000 [204:23:52<917:41:38, 26.60s/it]
                                                               
{'loss': '0.8956', 'grad_norm': '1.076', 'learning_rate': '3.106e-05', 'epoch': '0.448', 'num_input_tokens_seen': 341159336, 'train_runtime': '7.358e+05', 'train_tokens_per_second': '463.6'}

 45%|████▍     | 100800/225000 [204:23:52<917:41:38, 26.60s/it]
 45%|████▍     | 100801/225000 [204:24:23<969:35:02, 28.10s/it]
 45%|████▍     | 100802/225000 [204:24:53<990:03:47, 28.70s/it]
 45%|████▍     | 100803/225000 [204:25:11<874:25:21, 25.35s/it]
 45%|████▍     | 100804/225000 [204:25:38<889:53:23, 25.79s/it]
 45%|████▍     | 100805/225000 [204:25:58<832:36:05, 24.13s/it]
 45%|████▍     | 100806/225000 [204:26:25<865:09:47, 25.08s/it]
 45%|████▍     | 100807/225000 [204:26:59<951:44:23, 27.59s/it]
 45%|████▍     | 100808/225000 [204:27:26<947:05:23, 27.45s/it]
 45%|████▍     | 100809/225000 [204:27:47<879:49:47, 25.50s/it]
 45%|████▍     | 100810/225000 [204:28:19<950:10:18, 27.54s/it]
 45%|████▍     | 100811/225000 [204:28:46<940:21:25, 27.26s/it]
 45%|████▍     | 100812/225000 [204:29:14<951:04:05, 27.57s/it]
 45%|████▍     | 100813/225000 [204:29:42<954:06:01, 27.66s/it]
 45%|████▍     | 100814/225000 [204:30:13<987:15:05, 28.62s/it]
 45%|████▍     | 100815/225000 [204:30:32<890:25:29, 25.81s/it]
 45%|████▍     | 100816/225000 [204:30:57<877:13:01, 25.43s/it]
 45%|████▍     | 100817/225000 [204:31:29<953:03:32, 27.63s/it]
 45%|████▍     | 100818/225000 [204:31:55<933:30:43, 27.06s/it]
 45%|████▍     | 100819/225000 [204:32:23<937:22:59, 27.17s/it]
 45%|████▍     | 100820/225000 [204:32:41<850:03:47, 24.64s/it]
 45%|████▍     | 100821/225000 [204:33:10<892:25:47, 25.87s/it]
 45%|████▍     | 100822/225000 [204:33:34<877:59:49, 25.45s/it]
 45%|████▍     | 100823/225000 [204:33:55<822:33:19, 23.85s/it]
 45%|████▍     | 100824/225000 [204:34:27<907:07:09, 26.30s/it]
 45%|████▍     | 100825/225000 [204:34:58<962:57:22, 27.92s/it]
 45%|████▍     | 100826/225000 [204:35:28<981:13:31, 28.45s/it]
 45%|████▍     | 100827/225000 [204:35:57<990:55:22, 28.73s/it]
 45%|████▍     | 100828/225000 [204:36:23<954:49:57, 27.68s/it]
 45%|████▍     | 100829/225000 [204:36:56<1013:05:24, 29.37s/it]
 45%|████▍     | 100830/225000 [204:37:19<949:03:42, 27.52s/it] 
 45%|████▍     | 100831/225000 [204:37:46<941:01:48, 27.28s/it]
 45%|████▍     | 100832/225000 [204:38:16<972:11:04, 28.19s/it]
 45%|████▍     | 100833/225000 [204:38:41<942:40:39, 27.33s/it]
 45%|████▍     | 100834/225000 [204:39:09<946:58:44, 27.46s/it]
 45%|████▍     | 100835/225000 [204:39:32<895:19:34, 25.96s/it]
 45%|████▍     | 100836/225000 [204:40:04<960:34:22, 27.85s/it]
 45%|████▍     | 100837/225000 [204:40:33<969:58:26, 28.12s/it]
 45%|████▍     | 100838/225000 [204:41:00<961:31:37, 27.88s/it]
 45%|████▍     | 100839/225000 [204:41:27<952:12:30, 27.61s/it]
 45%|████▍     | 100840/225000 [204:41:59<996:39:26, 28.90s/it]
 45%|████▍     | 100841/225000 [204:42:29<1005:33:46, 29.16s/it]
 45%|████▍     | 100842/225000 [204:42:45<875:45:16, 25.39s/it] 
 45%|████▍     | 100843/225000 [204:43:17<936:31:12, 27.15s/it]
 45%|████▍     | 100844/225000 [204:43:41<908:57:57, 26.36s/it]
 45%|████▍     | 100845/225000 [204:44:12<956:26:21, 27.73s/it]
 45%|████▍     | 100846/225000 [204:44:41<973:52:45, 28.24s/it]
 45%|████▍     | 100847/225000 [204:45:05<927:34:20, 26.90s/it]
 45%|████▍     | 100848/225000 [204:45:31<921:05:18, 26.71s/it]
 45%|████▍     | 100849/225000 [204:45:55<892:51:26, 25.89s/it]
 45%|████▍     | 100850/225000 [204:46:16<839:04:38, 24.33s/it]
                                                               
{'loss': '0.8996', 'grad_norm': '0.968', 'learning_rate': '3.104e-05', 'epoch': '0.4482', 'num_input_tokens_seen': 341844560, 'train_runtime': '7.372e+05', 'train_tokens_per_second': '463.7'}

 45%|████▍     | 100850/225000 [204:46:16<839:04:38, 24.33s/it]
 45%|████▍     | 100851/225000 [204:46:45<881:19:10, 25.56s/it]
 45%|████▍     | 100852/225000 [204:47:12<897:23:13, 26.02s/it]
 45%|████▍     | 100853/225000 [204:47:45<971:02:23, 28.16s/it]
 45%|████▍     | 100854/225000 [204:48:16<1001:21:47, 29.04s/it]
 45%|████▍     | 100855/225000 [204:48:46<1016:28:40, 29.48s/it]
 45%|████▍     | 100856/225000 [204:49:09<942:36:00, 27.33s/it] 
 45%|████▍     | 100857/225000 [204:49:29<871:49:11, 25.28s/it]
 45%|████▍     | 100858/225000 [204:50:02<951:12:21, 27.58s/it]
 45%|████▍     | 100859/225000 [204:50:30<953:38:17, 27.65s/it]
 45%|████▍     | 100860/225000 [204:50:57<950:08:17, 27.55s/it]
 45%|████▍     | 100861/225000 [204:51:30<1001:16:31, 29.04s/it]
 45%|████▍     | 100862/225000 [204:51:58<992:47:20, 28.79s/it] 
 45%|████▍     | 100863/225000 [204:52:23<953:37:07, 27.66s/it]
 45%|████▍     | 100864/225000 [204:52:41<852:12:01, 24.71s/it]
 45%|████▍     | 100865/225000 [204:53:16<958:56:19, 27.81s/it]
 45%|████▍     | 100866/225000 [204:53:37<889:29:40, 25.80s/it]
 45%|████▍     | 100867/225000 [204:54:04<899:06:36, 26.08s/it]
 45%|████▍     | 100868/225000 [204:54:29<892:05:01, 25.87s/it]
 45%|████▍     | 100869/225000 [204:54:56<904:11:21, 26.22s/it]
 45%|████▍     | 100870/225000 [204:55:18<855:03:07, 24.80s/it]
 45%|████▍     | 100871/225000 [204:55:50<935:45:35, 27.14s/it]
 45%|████▍     | 100872/225000 [204:56:18<944:39:11, 27.40s/it]
 45%|████▍     | 100873/225000 [204:56:47<961:15:45, 27.88s/it]
 45%|████▍     | 100874/225000 [204:57:19<1000:44:10, 29.02s/it]
 45%|████▍     | 100875/225000 [204:57:48<1001:35:48, 29.05s/it]
 45%|████▍     | 100876/225000 [204:58:07<902:12:02, 26.17s/it] 
 45%|████▍     | 100877/225000 [204:58:39<957:21:49, 27.77s/it]
 45%|████▍     | 100878/225000 [204:59:03<922:48:20, 26.76s/it]
 45%|████▍     | 100879/225000 [204:59:22<838:19:18, 24.31s/it]
 45%|████▍     | 100880/225000 [204:59:42<790:50:02, 22.94s/it]
 45%|████▍     | 100881/225000 [205:00:09<833:29:27, 24.17s/it]
 45%|████▍     | 100882/225000 [205:00:34<845:33:06, 24.52s/it]
 45%|████▍     | 100883/225000 [205:01:01<870:46:56, 25.26s/it]
 45%|████▍     | 100884/225000 [205:01:28<883:06:12, 25.61s/it]
 45%|████▍     | 100885/225000 [205:01:50<852:06:38, 24.72s/it]
 45%|████▍     | 100886/225000 [205:02:16<867:58:37, 25.18s/it]
 45%|████▍     | 100887/225000 [205:02:50<953:00:57, 27.64s/it]
 45%|████▍     | 100888/225000 [205:03:13<906:27:56, 26.29s/it]
 45%|████▍     | 100889/225000 [205:03:43<945:50:25, 27.44s/it]
 45%|████▍     | 100890/225000 [205:04:10<939:30:12, 27.25s/it]
 45%|████▍     | 100891/225000 [205:04:36<925:09:20, 26.84s/it]
 45%|████▍     | 100892/225000 [205:04:57<869:42:39, 25.23s/it]
 45%|████▍     | 100893/225000 [205:05:25<894:34:18, 25.95s/it]
 45%|████▍     | 100894/225000 [205:05:50<887:36:26, 25.75s/it]
 45%|████▍     | 100895/225000 [205:06:14<865:53:37, 25.12s/it]
 45%|████▍     | 100896/225000 [205:06:39<863:37:50, 25.05s/it]
 45%|████▍     | 100897/225000 [205:07:04<871:09:49, 25.27s/it]
 45%|████▍     | 100898/225000 [205:07:28<851:26:20, 24.70s/it]
 45%|████▍     | 100899/225000 [205:07:58<906:24:57, 26.29s/it]
 45%|████▍     | 100900/225000 [205:08:22<886:24:43, 25.71s/it]
                                                               
{'loss': '0.8722', 'grad_norm': '0.9993', 'learning_rate': '3.102e-05', 'epoch': '0.4484', 'num_input_tokens_seen': 342524424, 'train_runtime': '7.385e+05', 'train_tokens_per_second': '463.8'}

 45%|████▍     | 100900/225000 [205:08:22<886:24:43, 25.71s/it]
 45%|████▍     | 100901/225000 [205:08:50<903:47:44, 26.22s/it]
 45%|████▍     | 100902/225000 [205:09:22<962:48:30, 27.93s/it]
 45%|████▍     | 100903/225000 [205:09:44<904:51:15, 26.25s/it]
 45%|████▍     | 100904/225000 [205:10:14<941:50:16, 27.32s/it]
 45%|████▍     | 100905/225000 [205:10:33<858:20:08, 24.90s/it]
 45%|████▍     | 100906/225000 [205:10:59<867:20:09, 25.16s/it]
 45%|████▍     | 100907/225000 [205:11:33<958:29:12, 27.81s/it]
 45%|████▍     | 100908/225000 [205:11:55<905:05:27, 26.26s/it]
 45%|████▍     | 100909/225000 [205:12:19<879:55:33, 25.53s/it]
 45%|████▍     | 100910/225000 [205:12:49<920:18:50, 26.70s/it]
 45%|████▍     | 100911/225000 [205:13:16<927:04:36, 26.90s/it]
 45%|████▍     | 100912/225000 [205:13:44<934:12:40, 27.10s/it]
 45%|████▍     | 100913/225000 [205:14:17<995:38:36, 28.89s/it]
 45%|████▍     | 100914/225000 [205:14:39<925:11:27, 26.84s/it]
 45%|████▍     | 100915/225000 [205:15:03<902:23:15, 26.18s/it]
 45%|████▍     | 100916/225000 [205:15:32<931:54:58, 27.04s/it]
 45%|████▍     | 100917/225000 [205:15:55<886:52:57, 25.73s/it]
 45%|████▍     | 100918/225000 [205:16:24<925:24:43, 26.85s/it]
 45%|████▍     | 100919/225000 [205:16:46<870:57:49, 25.27s/it]
 45%|████▍     | 100920/225000 [205:17:07<829:17:34, 24.06s/it]
 45%|████▍     | 100921/225000 [205:17:31<827:41:42, 24.01s/it]
 45%|████▍     | 100922/225000 [205:18:02<894:13:02, 25.94s/it]
 45%|████▍     | 100923/225000 [205:18:26<874:32:27, 25.37s/it]
 45%|████▍     | 100924/225000 [205:18:54<905:00:45, 26.26s/it]
 45%|████▍     | 100925/225000 [205:19:11<809:39:35, 23.49s/it]
 45%|████▍     | 100926/225000 [205:19:41<877:26:01, 25.46s/it]
 45%|████▍     | 100927/225000 [205:20:14<959:11:50, 27.83s/it]
 45%|████▍     | 100928/225000 [205:20:38<917:07:25, 26.61s/it]
 45%|████▍     | 100929/225000 [205:21:04<903:46:48, 26.22s/it]
 45%|████▍     | 100930/225000 [205:21:34<946:01:13, 27.45s/it]
 45%|████▍     | 100931/225000 [205:21:58<908:35:59, 26.36s/it]
 45%|████▍     | 100932/225000 [205:22:11<776:31:48, 22.53s/it]
 45%|████▍     | 100933/225000 [205:22:35<794:07:27, 23.04s/it]
 45%|████▍     | 100934/225000 [205:23:05<863:01:14, 25.04s/it]
 45%|████▍     | 100935/225000 [205:23:38<939:35:42, 27.26s/it]
 45%|████▍     | 100936/225000 [205:24:03<922:15:57, 26.76s/it]
 45%|████▍     | 100937/225000 [205:24:25<872:46:14, 25.33s/it]
 45%|████▍     | 100938/225000 [205:24:46<826:20:18, 23.98s/it]
 45%|████▍     | 100939/225000 [205:25:12<845:54:24, 24.55s/it]
 45%|████▍     | 100940/225000 [205:25:39<867:33:07, 25.17s/it]
 45%|████▍     | 100941/225000 [205:26:02<853:08:42, 24.76s/it]
 45%|████▍     | 100942/225000 [205:26:14<719:31:39, 20.88s/it]
 45%|████▍     | 100943/225000 [205:26:43<801:46:25, 23.27s/it]
 45%|████▍     | 100944/225000 [205:27:09<825:50:46, 23.97s/it]
 45%|████▍     | 100945/225000 [205:27:22<718:38:45, 20.85s/it]
 45%|████▍     | 100946/225000 [205:27:57<867:50:37, 25.18s/it]
 45%|████▍     | 100947/225000 [205:28:21<849:12:34, 24.64s/it]
 45%|████▍     | 100948/225000 [205:28:54<939:41:56, 27.27s/it]
 45%|████▍     | 100949/225000 [205:29:18<901:13:14, 26.15s/it]
 45%|████▍     | 100950/225000 [205:29:39<851:04:05, 24.70s/it]
                                                               
{'loss': '0.8871', 'grad_norm': '0.9996', 'learning_rate': '3.1e-05', 'epoch': '0.4487', 'num_input_tokens_seen': 343173656, 'train_runtime': '7.398e+05', 'train_tokens_per_second': '463.9'}

 45%|████▍     | 100950/225000 [205:29:39<851:04:05, 24.70s/it]
 45%|████▍     | 100951/225000 [205:30:03<844:38:33, 24.51s/it]
 45%|████▍     | 100952/225000 [205:30:31<879:54:48, 25.54s/it]
 45%|████▍     | 100953/225000 [205:30:56<876:35:40, 25.44s/it]
 45%|████▍     | 100954/225000 [205:31:24<903:19:08, 26.22s/it]
 45%|████▍     | 100955/225000 [205:31:52<917:25:54, 26.63s/it]
 45%|████▍     | 100956/225000 [205:32:13<856:56:42, 24.87s/it]
 45%|████▍     | 100957/225000 [205:32:33<811:35:57, 23.55s/it]
 45%|████▍     | 100958/225000 [205:33:00<843:12:34, 24.47s/it]
 45%|████▍     | 100959/225000 [205:33:25<855:41:02, 24.83s/it]
 45%|████▍     | 100960/225000 [205:33:51<863:14:55, 25.05s/it]
 45%|████▍     | 100961/225000 [205:34:15<851:49:52, 24.72s/it]
 45%|████▍     | 100962/225000 [205:34:49<948:48:38, 27.54s/it]
 45%|████▍     | 100963/225000 [205:35:14<920:30:12, 26.72s/it]
 45%|████▍     | 100964/225000 [205:35:38<891:04:38, 25.86s/it]
 45%|████▍     | 100965/225000 [205:36:08<936:36:46, 27.18s/it]
 45%|████▍     | 100966/225000 [205:36:27<853:32:41, 24.77s/it]
 45%|████▍     | 100967/225000 [205:36:54<875:26:12, 25.41s/it]
 45%|████▍     | 100968/225000 [205:37:16<840:00:19, 24.38s/it]
 45%|████▍     | 100969/225000 [205:37:44<879:37:32, 25.53s/it]
 45%|████▍     | 100970/225000 [205:38:10<881:09:22, 25.58s/it]
 45%|████▍     | 100971/225000 [205:38:40<931:25:14, 27.03s/it]
 45%|████▍     | 100972/225000 [205:39:03<887:15:22, 25.75s/it]
 45%|████▍     | 100973/225000 [205:39:37<968:56:52, 28.12s/it]
 45%|████▍     | 100974/225000 [205:39:56<875:16:51, 25.41s/it]
 45%|████▍     | 100975/225000 [205:40:19<853:32:33, 24.78s/it]
 45%|████▍     | 100976/225000 [205:40:38<795:36:17, 23.09s/it]
 45%|████▍     | 100977/225000 [205:40:58<762:35:52, 22.14s/it]
 45%|████▍     | 100978/225000 [205:41:30<862:19:33, 25.03s/it]
 45%|████▍     | 100979/225000 [205:41:54<853:05:26, 24.76s/it]
 45%|████▍     | 100980/225000 [205:42:15<816:20:27, 23.70s/it]
 45%|████▍     | 100981/225000 [205:42:44<865:24:43, 25.12s/it]
 45%|████▍     | 100982/225000 [205:43:13<911:16:49, 26.45s/it]
 45%|████▍     | 100983/225000 [205:43:32<833:17:29, 24.19s/it]
 45%|████▍     | 100984/225000 [205:43:53<799:29:54, 23.21s/it]
 45%|████▍     | 100985/225000 [205:44:30<944:16:30, 27.41s/it]
 45%|████▍     | 100986/225000 [205:44:55<909:47:14, 26.41s/it]
 45%|████▍     | 100987/225000 [205:45:11<803:38:48, 23.33s/it]
 45%|████▍     | 100988/225000 [205:45:38<843:50:03, 24.50s/it]
 45%|████▍     | 100989/225000 [205:46:00<822:22:25, 23.87s/it]
 45%|████▍     | 100990/225000 [205:46:25<830:23:48, 24.11s/it]
 45%|████▍     | 100991/225000 [205:46:54<877:39:04, 25.48s/it]
 45%|████▍     | 100992/225000 [205:47:26<950:55:18, 27.61s/it]
 45%|████▍     | 100993/225000 [205:47:47<883:57:23, 25.66s/it]
 45%|████▍     | 100994/225000 [205:48:16<919:16:18, 26.69s/it]
 45%|████▍     | 100995/225000 [205:48:33<817:09:01, 23.72s/it]
 45%|████▍     | 100996/225000 [205:49:05<903:48:17, 26.24s/it]
 45%|████▍     | 100997/225000 [205:49:27<856:09:29, 24.86s/it]
 45%|████▍     | 100998/225000 [205:49:53<863:50:30, 25.08s/it]
 45%|████▍     | 100999/225000 [205:50:16<844:14:38, 24.51s/it]
 45%|████▍     | 101000/225000 [205:50:40<844:52:58, 24.53s/it]
                                                               
{'loss': '0.8715', 'grad_norm': '0.8749', 'learning_rate': '3.099e-05', 'epoch': '0.4489', 'num_input_tokens_seen': 343829088, 'train_runtime': '7.41e+05', 'train_tokens_per_second': '464'}

 45%|████▍     | 101000/225000 [205:50:40<844:52:58, 24.53s/it]
 45%|████▍     | 101001/225000 [205:51:12<920:20:57, 26.72s/it]
 45%|████▍     | 101002/225000 [205:51:32<851:35:23, 24.72s/it]
 45%|████▍     | 101003/225000 [205:51:57<855:37:13, 24.84s/it]
 45%|████▍     | 101004/225000 [205:52:16<792:25:07, 23.01s/it]
 45%|████▍     | 101005/225000 [205:52:47<872:27:59, 25.33s/it]
 45%|████▍     | 101006/225000 [205:53:07<815:53:09, 23.69s/it]
 45%|████▍     | 101007/225000 [205:53:32<829:59:41, 24.10s/it]
 45%|████▍     | 101008/225000 [205:53:55<825:20:48, 23.96s/it]
 45%|████▍     | 101009/225000 [205:54:23<865:12:22, 25.12s/it]
 45%|████▍     | 101010/225000 [205:54:48<866:26:03, 25.16s/it]
 45%|████▍     | 101011/225000 [205:55:13<857:01:42, 24.88s/it]
 45%|████▍     | 101012/225000 [205:55:38<859:57:08, 24.97s/it]
 45%|████▍     | 101013/225000 [205:55:59<817:35:48, 23.74s/it]
 45%|████▍     | 101014/225000 [205:56:31<906:36:53, 26.32s/it]
 45%|████▍     | 101015/225000 [205:56:51<838:19:29, 24.34s/it]
 45%|████▍     | 101016/225000 [205:57:23<916:23:41, 26.61s/it]
 45%|████▍     | 101017/225000 [205:57:45<876:38:14, 25.45s/it]
 45%|████▍     | 101018/225000 [205:58:17<938:13:49, 27.24s/it]
 45%|████▍     | 101019/225000 [205:58:44<932:10:44, 27.07s/it]
 45%|████▍     | 101020/225000 [205:59:00<818:42:43, 23.77s/it]
 45%|████▍     | 101021/225000 [205:59:30<888:00:29, 25.79s/it]
 45%|████▍     | 101022/225000 [205:59:51<839:12:31, 24.37s/it]
 45%|████▍     | 101023/225000 [206:00:13<810:04:23, 23.52s/it]
 45%|████▍     | 101024/225000 [206:00:34<785:15:42, 22.80s/it]
 45%|████▍     | 101025/225000 [206:01:03<846:34:16, 24.58s/it]
 45%|████▍     | 101026/225000 [206:01:32<898:23:17, 26.09s/it]
 45%|████▍     | 101027/225000 [206:01:53<840:42:20, 24.41s/it]
 45%|████▍     | 101028/225000 [206:02:19<859:05:38, 24.95s/it]
 45%|████▍     | 101029/225000 [206:02:45<874:36:26, 25.40s/it]
 45%|████▍     | 101030/225000 [206:03:10<870:49:12, 25.29s/it]
 45%|████▍     | 101031/225000 [206:03:24<753:09:34, 21.87s/it]
 45%|████▍     | 101032/225000 [206:03:55<849:11:58, 24.66s/it]
 45%|████▍     | 101033/225000 [206:04:12<770:43:24, 22.38s/it]
 45%|████▍     | 101034/225000 [206:04:38<808:29:13, 23.48s/it]
 45%|████▍     | 101035/225000 [206:05:05<836:18:18, 24.29s/it]
 45%|████▍     | 101036/225000 [206:05:27<812:24:49, 23.59s/it]
 45%|████▍     | 101037/225000 [206:05:48<784:52:26, 22.79s/it]
 45%|████▍     | 101038/225000 [206:06:17<855:45:23, 24.85s/it]
 45%|████▍     | 101039/225000 [206:06:39<828:57:52, 24.07s/it]
 45%|████▍     | 101040/225000 [206:07:04<832:08:30, 24.17s/it]
 45%|████▍     | 101041/225000 [206:07:22<773:18:00, 22.46s/it]
 45%|████▍     | 101042/225000 [206:07:41<729:10:02, 21.18s/it]
 45%|████▍     | 101043/225000 [206:08:03<742:38:35, 21.57s/it]
 45%|████▍     | 101044/225000 [206:08:28<774:14:08, 22.49s/it]
 45%|████▍     | 101045/225000 [206:08:54<815:50:48, 23.69s/it]
 45%|████▍     | 101046/225000 [206:09:29<926:29:11, 26.91s/it]
 45%|████▍     | 101047/225000 [206:09:51<879:06:37, 25.53s/it]
 45%|████▍     | 101048/225000 [206:10:17<883:39:48, 25.66s/it]
 45%|████▍     | 101049/225000 [206:10:38<841:56:16, 24.45s/it]
 45%|████▍     | 101050/225000 [206:11:08<893:38:37, 25.95s/it]
                                                               
{'loss': '0.8939', 'grad_norm': '1.018', 'learning_rate': '3.097e-05', 'epoch': '0.4491', 'num_input_tokens_seen': 344465368, 'train_runtime': '7.423e+05', 'train_tokens_per_second': '464.1'}

 45%|████▍     | 101050/225000 [206:11:08<893:38:37, 25.95s/it]
 45%|████▍     | 101051/225000 [206:11:23<781:09:03, 22.69s/it]
 45%|████▍     | 101052/225000 [206:11:53<852:55:07, 24.77s/it]
 45%|████▍     | 101053/225000 [206:12:18<855:00:03, 24.83s/it]
 45%|████▍     | 101054/225000 [206:12:44<869:27:20, 25.25s/it]
 45%|████▍     | 101055/225000 [206:13:02<800:26:57, 23.25s/it]
 45%|████▍     | 101056/225000 [206:13:17<713:33:17, 20.73s/it]
 45%|████▍     | 101057/225000 [206:13:47<801:35:23, 23.28s/it]
 45%|████▍     | 101058/225000 [206:14:15<852:20:15, 24.76s/it]
 45%|████▍     | 101059/225000 [206:14:36<814:25:57, 23.66s/it]
 45%|████▍     | 101060/225000 [206:14:58<802:31:42, 23.31s/it]
 45%|████▍     | 101061/225000 [206:15:28<868:31:32, 25.23s/it]
 45%|████▍     | 101062/225000 [206:15:52<858:16:55, 24.93s/it]
 45%|████▍     | 101063/225000 [206:16:14<825:06:13, 23.97s/it]
 45%|████▍     | 101064/225000 [206:16:39<841:26:31, 24.44s/it]
 45%|████▍     | 101065/225000 [206:17:08<887:12:34, 25.77s/it]
 45%|████▍     | 101066/225000 [206:17:31<856:54:43, 24.89s/it]
 45%|████▍     | 101067/225000 [206:17:59<889:56:12, 25.85s/it]
 45%|████▍     | 101068/225000 [206:18:33<972:16:11, 28.24s/it]
 45%|████▍     | 101069/225000 [206:19:03<993:37:46, 28.86s/it]
 45%|████▍     | 101070/225000 [206:19:20<866:07:09, 25.16s/it]
 45%|████▍     | 101071/225000 [206:19:48<896:59:34, 26.06s/it]
 45%|████▍     | 101072/225000 [206:20:10<855:08:47, 24.84s/it]
 45%|████▍     | 101073/225000 [206:20:43<935:32:31, 27.18s/it]
 45%|████▍     | 101074/225000 [206:21:13<965:02:16, 28.03s/it]
 45%|████▍     | 101075/225000 [206:21:38<931:50:10, 27.07s/it]
 45%|████▍     | 101076/225000 [206:22:08<964:22:02, 28.01s/it]
 45%|████▍     | 101077/225000 [206:22:26<860:19:51, 24.99s/it]
 45%|████▍     | 101078/225000 [206:22:41<755:24:26, 21.94s/it]
 45%|████▍     | 101079/225000 [206:23:03<760:29:50, 22.09s/it]
 45%|████▍     | 101080/225000 [206:23:32<832:17:10, 24.18s/it]
 45%|████▍     | 101081/225000 [206:23:58<848:15:35, 24.64s/it]
 45%|████▍     | 101082/225000 [206:24:18<804:17:00, 23.37s/it]
 45%|████▍     | 101083/225000 [206:24:35<733:27:31, 21.31s/it]
 45%|████▍     | 101084/225000 [206:25:02<794:31:40, 23.08s/it]
 45%|████▍     | 101085/225000 [206:25:23<773:12:41, 22.46s/it]
 45%|████▍     | 101086/225000 [206:25:46<780:01:54, 22.66s/it]
 45%|████▍     | 101087/225000 [206:26:03<716:08:01, 20.81s/it]
 45%|████▍     | 101088/225000 [206:26:27<758:27:50, 22.04s/it]
 45%|████▍     | 101089/225000 [206:26:54<803:34:47, 23.35s/it]
 45%|████▍     | 101090/225000 [206:27:17<800:35:39, 23.26s/it]
 45%|████▍     | 101091/225000 [206:27:45<847:30:39, 24.62s/it]
 45%|████▍     | 101092/225000 [206:28:14<896:39:46, 26.05s/it]
 45%|████▍     | 101093/225000 [206:28:38<878:31:31, 25.52s/it]
 45%|████▍     | 101094/225000 [206:29:05<885:16:10, 25.72s/it]
 45%|████▍     | 101095/225000 [206:29:27<851:59:11, 24.75s/it]
 45%|████▍     | 101096/225000 [206:29:49<824:41:31, 23.96s/it]
 45%|████▍     | 101097/225000 [206:30:18<870:28:28, 25.29s/it]
 45%|████▍     | 101098/225000 [206:30:43<870:16:58, 25.29s/it]
 45%|████▍     | 101099/225000 [206:31:02<808:24:45, 23.49s/it]
 45%|████▍     | 101100/225000 [206:31:33<880:49:30, 25.59s/it]
                                                               
{'loss': '0.8864', 'grad_norm': '1.011', 'learning_rate': '3.095e-05', 'epoch': '0.4493', 'num_input_tokens_seen': 345096032, 'train_runtime': '7.435e+05', 'train_tokens_per_second': '464.2'}

 45%|████▍     | 101100/225000 [206:31:33<880:49:30, 25.59s/it]
 45%|████▍     | 101101/225000 [206:32:00<900:25:07, 26.16s/it]
 45%|████▍     | 101102/225000 [206:32:19<821:49:11, 23.88s/it]
 45%|████▍     | 101103/225000 [206:32:38<777:29:06, 22.59s/it]
 45%|████▍     | 101104/225000 [206:32:58<745:07:12, 21.65s/it]
 45%|████▍     | 101105/225000 [206:33:23<779:42:29, 22.66s/it]
 45%|████▍     | 101106/225000 [206:33:46<789:06:58, 22.93s/it]
 45%|████▍     | 101107/225000 [206:34:06<759:50:52, 22.08s/it]
 45%|████▍     | 101108/225000 [206:34:30<777:58:20, 22.61s/it]
 45%|████▍     | 101109/225000 [206:35:01<861:06:40, 25.02s/it]
 45%|████▍     | 101110/225000 [206:35:28<880:16:30, 25.58s/it]
 45%|████▍     | 101111/225000 [206:35:45<791:34:02, 23.00s/it]
 45%|████▍     | 101112/225000 [206:36:14<858:12:00, 24.94s/it]
 45%|████▍     | 101113/225000 [206:36:30<767:28:01, 22.30s/it]
 45%|████▍     | 101114/225000 [206:36:50<736:19:23, 21.40s/it]
 45%|████▍     | 101115/225000 [206:37:19<819:24:39, 23.81s/it]
 45%|████▍     | 101116/225000 [206:37:43<824:09:19, 23.95s/it]
 45%|████▍     | 101117/225000 [206:38:05<804:55:39, 23.39s/it]
 45%|████▍     | 101118/225000 [206:38:35<864:27:39, 25.12s/it]
 45%|████▍     | 101119/225000 [206:39:00<862:39:39, 25.07s/it]
 45%|████▍     | 101120/225000 [206:39:24<853:45:37, 24.81s/it]
 45%|████▍     | 101121/225000 [206:39:42<788:01:47, 22.90s/it]
 45%|████▍     | 101122/225000 [206:40:11<845:35:37, 24.57s/it]
 45%|████▍     | 101123/225000 [206:40:37<862:54:30, 25.08s/it]
 45%|████▍     | 101124/225000 [206:40:52<755:56:57, 21.97s/it]
 45%|████▍     | 101125/225000 [206:41:16<783:09:19, 22.76s/it]
 45%|████▍     | 101126/225000 [206:41:47<865:44:28, 25.16s/it]
 45%|████▍     | 101127/225000 [206:42:04<783:57:34, 22.78s/it]
 45%|████▍     | 101128/225000 [206:42:31<819:48:22, 23.83s/it]
 45%|████▍     | 101129/225000 [206:42:59<868:50:30, 25.25s/it]
 45%|████▍     | 101130/225000 [206:43:21<831:06:04, 24.15s/it]
 45%|████▍     | 101131/225000 [206:43:44<822:32:02, 23.91s/it]
 45%|████▍     | 101132/225000 [206:44:15<896:16:49, 26.05s/it]
 45%|████▍     | 101133/225000 [206:44:36<843:27:07, 24.51s/it]
 45%|████▍     | 101134/225000 [206:45:02<860:52:10, 25.02s/it]
 45%|████▍     | 101135/225000 [206:45:28<873:34:51, 25.39s/it]
 45%|████▍     | 101136/225000 [206:45:48<809:23:00, 23.52s/it]
 45%|████▍     | 101137/225000 [206:46:11<812:23:25, 23.61s/it]
 45%|████▍     | 101138/225000 [206:46:40<862:41:16, 25.07s/it]
 45%|████▍     | 101139/225000 [206:47:13<944:04:49, 27.44s/it]
 45%|████▍     | 101140/225000 [206:47:30<837:04:24, 24.33s/it]
 45%|████▍     | 101141/225000 [206:47:54<832:03:49, 24.18s/it]
 45%|████▍     | 101142/225000 [206:48:18<829:33:33, 24.11s/it]
 45%|████▍     | 101143/225000 [206:48:46<869:23:22, 25.27s/it]
 45%|████▍     | 101144/225000 [206:49:02<777:03:13, 22.59s/it]
 45%|████▍     | 101145/225000 [206:49:23<761:08:53, 22.12s/it]
 45%|████▍     | 101146/225000 [206:49:54<850:17:21, 24.71s/it]
 45%|████▍     | 101147/225000 [206:50:25<911:50:24, 26.50s/it]
 45%|████▍     | 101148/225000 [206:50:50<898:16:43, 26.11s/it]
 45%|████▍     | 101149/225000 [206:51:13<873:33:49, 25.39s/it]
 45%|████▍     | 101150/225000 [206:51:39<878:42:21, 25.54s/it]
                                                               
{'loss': '0.9053', 'grad_norm': '0.9851', 'learning_rate': '3.093e-05', 'epoch': '0.4496', 'num_input_tokens_seen': 345719040, 'train_runtime': '7.447e+05', 'train_tokens_per_second': '464.2'}

 45%|████▍     | 101150/225000 [206:51:39<878:42:21, 25.54s/it]
 45%|████▍     | 101151/225000 [206:52:02<848:20:40, 24.66s/it]
 45%|████▍     | 101152/225000 [206:52:20<783:05:30, 22.76s/it]
 45%|████▍     | 101153/225000 [206:52:48<836:09:56, 24.31s/it]
 45%|████▍     | 101154/225000 [206:53:07<775:02:00, 22.53s/it]
 45%|████▍     | 101155/225000 [206:53:32<804:55:10, 23.40s/it]
 45%|████▍     | 101156/225000 [206:53:53<782:54:36, 22.76s/it]
 45%|████▍     | 101157/225000 [206:54:23<856:02:09, 24.88s/it]
 45%|████▍     | 101158/225000 [206:54:45<821:14:51, 23.87s/it]
 45%|████▍     | 101159/225000 [206:55:10<834:28:51, 24.26s/it]
 45%|████▍     | 101160/225000 [206:55:32<812:00:35, 23.60s/it]
 45%|████▍     | 101161/225000 [206:55:53<785:14:36, 22.83s/it]
 45%|████▍     | 101162/225000 [206:56:14<763:42:05, 22.20s/it]
 45%|████▍     | 101163/225000 [206:56:36<761:26:35, 22.14s/it]
 45%|████▍     | 101164/225000 [206:57:03<816:18:07, 23.73s/it]
 45%|████▍     | 101165/225000 [206:57:30<847:54:48, 24.65s/it]
 45%|████▍     | 101166/225000 [206:57:52<825:10:20, 23.99s/it]
 45%|████▍     | 101167/225000 [206:58:18<843:17:39, 24.52s/it]
 45%|████▍     | 101168/225000 [206:58:43<852:07:34, 24.77s/it]
 45%|████▍     | 101169/225000 [206:59:15<925:20:18, 26.90s/it]
 45%|████▍     | 101170/225000 [206:59:43<938:47:36, 27.29s/it]
 45%|████▍     | 101171/225000 [207:00:04<868:10:48, 25.24s/it]
 45%|████▍     | 101172/225000 [207:00:32<893:47:53, 25.99s/it]
 45%|████▍     | 101173/225000 [207:00:57<885:02:32, 25.73s/it]
 45%|████▍     | 101174/225000 [207:01:19<845:12:47, 24.57s/it]
 45%|████▍     | 101175/225000 [207:01:39<804:33:05, 23.39s/it]
 45%|████▍     | 101176/225000 [207:02:03<806:31:43, 23.45s/it]
 45%|████▍     | 101177/225000 [207:02:32<868:49:57, 25.26s/it]
 45%|████▍     | 101178/225000 [207:03:01<904:14:13, 26.29s/it]
 45%|████▍     | 101179/225000 [207:03:22<850:43:05, 24.73s/it]
 45%|████▍     | 101180/225000 [207:03:39<770:50:04, 22.41s/it]
 45%|████▍     | 101181/225000 [207:04:03<782:34:31, 22.75s/it]
 45%|████▍     | 101182/225000 [207:04:25<776:53:28, 22.59s/it]
 45%|████▍     | 101183/225000 [207:04:56<868:53:34, 25.26s/it]
 45%|████▍     | 101184/225000 [207:05:24<893:37:08, 25.98s/it]
 45%|████▍     | 101185/225000 [207:05:43<825:23:19, 24.00s/it]
 45%|████▍     | 101186/225000 [207:06:02<773:05:17, 22.48s/it]
 45%|████▍     | 101187/225000 [207:06:21<738:32:48, 21.47s/it]
 45%|████▍     | 101188/225000 [207:06:41<722:42:35, 21.01s/it]
 45%|████▍     | 101189/225000 [207:07:04<734:49:14, 21.37s/it]
 45%|████▍     | 101190/225000 [207:07:33<819:50:26, 23.84s/it]
 45%|████▍     | 101191/225000 [207:08:04<889:48:03, 25.87s/it]
 45%|████▍     | 101192/225000 [207:08:19<780:11:30, 22.69s/it]
 45%|████▍     | 101193/225000 [207:08:38<743:16:39, 21.61s/it]
 45%|████▍     | 101194/225000 [207:08:59<736:03:15, 21.40s/it]
 45%|████▍     | 101195/225000 [207:09:14<667:03:49, 19.40s/it]
 45%|████▍     | 101196/225000 [207:09:36<691:16:27, 20.10s/it]
 45%|████▍     | 101197/225000 [207:10:04<782:22:40, 22.75s/it]
 45%|████▍     | 101198/225000 [207:10:30<806:12:50, 23.44s/it]
 45%|████▍     | 101199/225000 [207:10:57<844:25:14, 24.55s/it]
 45%|████▍     | 101200/225000 [207:11:14<769:30:00, 22.38s/it]
                                                               
{'loss': '0.8953', 'grad_norm': '0.9871', 'learning_rate': '3.092e-05', 'epoch': '0.4498', 'num_input_tokens_seen': 346318448, 'train_runtime': '7.459e+05', 'train_tokens_per_second': '464.3'}

 45%|████▍     | 101200/225000 [207:11:14<769:30:00, 22.38s/it]
 45%|████▍     | 101201/225000 [207:11:46<869:08:22, 25.27s/it]
 45%|████▍     | 101202/225000 [207:12:08<838:38:04, 24.39s/it]
 45%|████▍     | 101203/225000 [207:12:34<847:58:33, 24.66s/it]
 45%|████▍     | 101204/225000 [207:13:05<916:19:37, 26.65s/it]
 45%|████▍     | 101205/225000 [207:13:31<913:14:06, 26.56s/it]
 45%|████▍     | 101206/225000 [207:13:57<902:04:26, 26.23s/it]
 45%|████▍     | 101207/225000 [207:14:14<806:54:09, 23.47s/it]
 45%|████▍     | 101208/225000 [207:14:30<733:38:27, 21.34s/it]
 45%|████▍     | 101209/225000 [207:14:50<722:16:41, 21.00s/it]
 45%|████▍     | 101210/225000 [207:15:25<858:36:26, 24.97s/it]
 45%|████▍     | 101211/225000 [207:15:53<899:21:16, 26.15s/it]
 45%|████▍     | 101212/225000 [207:16:18<879:15:23, 25.57s/it]
 45%|████▍     | 101213/225000 [207:16:46<904:44:57, 26.31s/it]
 45%|████▍     | 101214/225000 [207:17:05<835:36:06, 24.30s/it]
 45%|████▍     | 101215/225000 [207:17:32<859:41:49, 25.00s/it]
 45%|████▍     | 101216/225000 [207:17:57<857:17:58, 24.93s/it]
 45%|████▍     | 101217/225000 [207:18:21<847:11:27, 24.64s/it]
 45%|████▍     | 101218/225000 [207:18:47<864:53:33, 25.15s/it]
 45%|████▍     | 101219/225000 [207:19:11<855:34:41, 24.88s/it]
 45%|████▍     | 101220/225000 [207:19:36<856:49:38, 24.92s/it]
 45%|████▍     | 101221/225000 [207:20:05<891:14:19, 25.92s/it]
 45%|████▍     | 101222/225000 [207:20:34<922:45:45, 26.84s/it]
 45%|████▍     | 101223/225000 [207:20:50<815:29:35, 23.72s/it]
 45%|████▍     | 101224/225000 [207:21:11<784:41:47, 22.82s/it]
 45%|████▍     | 101225/225000 [207:21:28<729:52:35, 21.23s/it]
 45%|████▍     | 101226/225000 [207:21:58<817:57:08, 23.79s/it]
 45%|████▍     | 101227/225000 [207:22:23<834:14:02, 24.26s/it]
 45%|████▍     | 101228/225000 [207:22:51<868:28:43, 25.26s/it]
 45%|████▍     | 101229/225000 [207:23:10<801:03:26, 23.30s/it]
 45%|████▍     | 101230/225000 [207:23:33<804:47:21, 23.41s/it]
 45%|████▍     | 101231/225000 [207:24:05<887:15:28, 25.81s/it]
 45%|████▍     | 101232/225000 [207:24:24<824:35:55, 23.98s/it]
 45%|████▍     | 101233/225000 [207:24:47<810:17:58, 23.57s/it]
 45%|████▍     | 101234/225000 [207:25:08<787:56:04, 22.92s/it]
 45%|████▍     | 101235/225000 [207:25:37<841:27:15, 24.48s/it]
 45%|████▍     | 101236/225000 [207:25:55<779:12:42, 22.67s/it]
 45%|████▍     | 101237/225000 [207:26:24<846:58:31, 24.64s/it]
 45%|████▍     | 101238/225000 [207:26:54<903:35:13, 26.28s/it]
 45%|████▍     | 101239/225000 [207:27:13<827:20:47, 24.07s/it]
 45%|████▍     | 101240/225000 [207:27:32<776:49:50, 22.60s/it]
 45%|████▍     | 101241/225000 [207:27:59<820:07:38, 23.86s/it]
 45%|████▍     | 101242/225000 [207:28:25<834:38:28, 24.28s/it]
 45%|████▍     | 101243/225000 [207:28:50<852:02:15, 24.79s/it]
 45%|████▍     | 101244/225000 [207:29:11<807:01:58, 23.48s/it]
 45%|████▍     | 101245/225000 [207:29:33<789:22:37, 22.96s/it]
 45%|████▍     | 101246/225000 [207:29:52<753:26:36, 21.92s/it]
 45%|████▍     | 101247/225000 [207:30:15<758:13:31, 22.06s/it]
 45%|████▍     | 101248/225000 [207:30:40<791:44:04, 23.03s/it]
 45%|████▍     | 101249/225000 [207:30:56<725:00:06, 21.09s/it]
 45%|████▌     | 101250/225000 [207:31:16<710:56:06, 20.68s/it]
                                                               
{'loss': '0.89', 'grad_norm': '1.163', 'learning_rate': '3.09e-05', 'epoch': '0.45', 'num_input_tokens_seen': 346932680, 'train_runtime': '7.471e+05', 'train_tokens_per_second': '464.4'}

 45%|████▌     | 101250/225000 [207:31:16<710:56:06, 20.68s/it]
 45%|████▌     | 101251/225000 [207:31:35<694:33:33, 20.21s/it]
 45%|████▌     | 101252/225000 [207:32:00<738:56:41, 21.50s/it]
 45%|████▌     | 101253/225000 [207:32:31<836:18:39, 24.33s/it]
 45%|████▌     | 101254/225000 [207:32:51<790:19:00, 22.99s/it]
 45%|████▌     | 101255/225000 [207:33:18<836:44:41, 24.34s/it]
 45%|████▌     | 101256/225000 [207:33:48<891:22:28, 25.93s/it]
 45%|████▌     | 101257/225000 [207:34:04<793:06:36, 23.07s/it]
 45%|████▌     | 101258/225000 [207:34:28<802:53:10, 23.36s/it]
 45%|████▌     | 101259/225000 [207:34:52<805:59:09, 23.45s/it]
 45%|████▌     | 101260/225000 [207:35:20<855:58:54, 24.90s/it]
 45%|████▌     | 101261/225000 [207:35:40<809:22:20, 23.55s/it]
 45%|████▌     | 101262/225000 [207:36:05<824:19:11, 23.98s/it]
 45%|████▌     | 101263/225000 [207:36:29<818:55:35, 23.83s/it]
 45%|████▌     | 101264/225000 [207:37:00<898:56:22, 26.15s/it]
 45%|████▌     | 101265/225000 [207:37:15<778:06:50, 22.64s/it]
 45%|████▌     | 101266/225000 [207:37:42<823:26:26, 23.96s/it]
 45%|████▌     | 101267/225000 [207:38:02<780:26:40, 22.71s/it]
 45%|████▌     | 101268/225000 [207:38:29<827:37:47, 24.08s/it]
 45%|████▌     | 101269/225000 [207:38:45<748:09:09, 21.77s/it]
 45%|████▌     | 101270/225000 [207:39:13<813:02:00, 23.66s/it]
 45%|████▌     | 101271/225000 [207:39:38<817:24:01, 23.78s/it]
 45%|████▌     | 101272/225000 [207:40:03<837:15:27, 24.36s/it]
 45%|████▌     | 101273/225000 [207:40:32<886:01:43, 25.78s/it]
 45%|████▌     | 101274/225000 [207:40:50<804:55:08, 23.42s/it]
 45%|████▌     | 101275/225000 [207:41:14<810:11:29, 23.57s/it]
 45%|████▌     | 101276/225000 [207:41:36<791:10:42, 23.02s/it]
 45%|████▌     | 101277/225000 [207:41:55<747:39:38, 21.75s/it]
 45%|████▌     | 101278/225000 [207:42:25<832:20:24, 24.22s/it]
 45%|████▌     | 101279/225000 [207:42:45<797:05:28, 23.19s/it]
 45%|████▌     | 101280/225000 [207:43:06<774:08:49, 22.53s/it]
 45%|████▌     | 101281/225000 [207:43:29<775:31:37, 22.57s/it]
 45%|████▌     | 101282/225000 [207:43:49<742:49:28, 21.62s/it]
 45%|████▌     | 101283/225000 [207:44:13<777:05:27, 22.61s/it]
 45%|████▌     | 101284/225000 [207:44:35<764:34:55, 22.25s/it]
 45%|████▌     | 101285/225000 [207:44:53<727:10:43, 21.16s/it]
 45%|████▌     | 101286/225000 [207:45:17<748:48:55, 21.79s/it]
 45%|████▌     | 101287/225000 [207:45:49<856:02:42, 24.91s/it]
 45%|████▌     | 101288/225000 [207:46:05<760:00:01, 22.12s/it]
 45%|████▌     | 101289/225000 [207:46:25<744:55:17, 21.68s/it]
 45%|████▌     | 101290/225000 [207:46:51<788:56:07, 22.96s/it]
 45%|████▌     | 101291/225000 [207:47:12<769:37:03, 22.40s/it]
 45%|████▌     | 101292/225000 [207:47:30<717:48:48, 20.89s/it]
 45%|████▌     | 101293/225000 [207:47:54<755:35:01, 21.99s/it]
 45%|████▌     | 101294/225000 [207:48:26<860:09:26, 25.03s/it]
 45%|████▌     | 101295/225000 [207:48:49<835:46:33, 24.32s/it]
 45%|████▌     | 101296/225000 [207:49:14<839:20:47, 24.43s/it]
 45%|████▌     | 101297/225000 [207:49:26<716:38:56, 20.86s/it]
 45%|████▌     | 101298/225000 [207:49:45<698:40:55, 20.33s/it]
 45%|████▌     | 101299/225000 [207:50:09<734:51:46, 21.39s/it]
 45%|████▌     | 101300/225000 [207:50:38<808:31:12, 23.53s/it]
                                                               
{'loss': '0.9113', 'grad_norm': '1.071', 'learning_rate': '3.088e-05', 'epoch': '0.4502', 'num_input_tokens_seen': 347528216, 'train_runtime': '7.482e+05', 'train_tokens_per_second': '464.5'}

 45%|████▌     | 101300/225000 [207:50:38<808:31:12, 23.53s/it]
 45%|████▌     | 101301/225000 [207:51:04<834:49:43, 24.30s/it]
 45%|████▌     | 101302/225000 [207:51:27<828:10:13, 24.10s/it]
 45%|████▌     | 101303/225000 [207:51:56<872:42:38, 25.40s/it]
 45%|████▌     | 101304/225000 [207:52:18<845:03:20, 24.59s/it]
 45%|████▌     | 101305/225000 [207:52:41<826:47:42, 24.06s/it]
 45%|████▌     | 101306/225000 [207:53:03<797:26:51, 23.21s/it]
 45%|████▌     | 101307/225000 [207:53:24<783:07:08, 22.79s/it]
 45%|████▌     | 101308/225000 [207:53:50<810:47:24, 23.60s/it]
 45%|████▌     | 101309/225000 [207:54:17<849:59:44, 24.74s/it]
 45%|████▌     | 101310/225000 [207:54:43<859:13:01, 25.01s/it]
 45%|████▌     | 101311/225000 [207:55:08<863:20:51, 25.13s/it]
 45%|████▌     | 101312/225000 [207:55:30<832:08:30, 24.22s/it]
 45%|████▌     | 101313/225000 [207:55:55<833:45:52, 24.27s/it]
 45%|████▌     | 101314/225000 [207:56:18<819:48:22, 23.86s/it]
 45%|████▌     | 101315/225000 [207:56:40<804:12:32, 23.41s/it]
 45%|████▌     | 101316/225000 [207:57:01<779:14:16, 22.68s/it]
 45%|████▌     | 101317/225000 [207:57:30<840:58:08, 24.48s/it]
 45%|████▌     | 101318/225000 [207:57:54<842:43:32, 24.53s/it]
 45%|████▌     | 101319/225000 [207:58:13<786:10:05, 22.88s/it]
 45%|████▌     | 101320/225000 [207:58:37<794:35:44, 23.13s/it]
 45%|████▌     | 101321/225000 [207:59:06<852:00:40, 24.80s/it]
 45%|████▌     | 101322/225000 [207:59:28<821:56:39, 23.93s/it]
 45%|████▌     | 101323/225000 [207:59:52<828:14:05, 24.11s/it]
 45%|████▌     | 101324/225000 [208:00:15<813:35:28, 23.68s/it]
 45%|████▌     | 101325/225000 [208:00:43<856:28:36, 24.93s/it]
 45%|████▌     | 101326/225000 [208:00:56<734:17:18, 21.37s/it]
 45%|████▌     | 101327/225000 [208:01:17<730:11:10, 21.26s/it]
 45%|████▌     | 101328/225000 [208:01:39<742:46:31, 21.62s/it]
 45%|████▌     | 101329/225000 [208:01:57<701:37:23, 20.42s/it]
 45%|████▌     | 101330/225000 [208:02:25<780:13:26, 22.71s/it]
 45%|████▌     | 101331/225000 [208:02:55<851:31:46, 24.79s/it]
 45%|████▌     | 101332/225000 [208:03:12<775:43:28, 22.58s/it]
 45%|████▌     | 101333/225000 [208:03:26<684:03:31, 19.91s/it]
 45%|████▌     | 101334/225000 [208:03:41<641:41:12, 18.68s/it]
 45%|████▌     | 101335/225000 [208:04:04<676:12:12, 19.68s/it]
 45%|████▌     | 101336/225000 [208:04:29<740:51:14, 21.57s/it]
 45%|████▌     | 101337/225000 [208:04:50<732:34:12, 21.33s/it]
 45%|████▌     | 101338/225000 [208:05:23<854:02:50, 24.86s/it]
 45%|████▌     | 101339/225000 [208:05:53<901:45:11, 26.25s/it]
 45%|████▌     | 101340/225000 [208:06:17<882:09:57, 25.68s/it]
 45%|████▌     | 101341/225000 [208:06:40<847:45:19, 24.68s/it]
 45%|████▌     | 101342/225000 [208:06:57<773:53:29, 22.53s/it]
 45%|████▌     | 101343/225000 [208:07:20<775:40:34, 22.58s/it]
 45%|████▌     | 101344/225000 [208:07:39<744:59:33, 21.69s/it]
 45%|████▌     | 101345/225000 [208:08:01<746:54:50, 21.75s/it]
 45%|████▌     | 101346/225000 [208:08:19<705:04:12, 20.53s/it]
 45%|████▌     | 101347/225000 [208:08:45<762:25:15, 22.20s/it]
 45%|████▌     | 101348/225000 [208:08:59<681:09:24, 19.83s/it]
 45%|████▌     | 101349/225000 [208:09:25<746:31:48, 21.73s/it]
 45%|████▌     | 101350/225000 [208:09:46<737:18:00, 21.47s/it]
                                                               
{'loss': '0.9014', 'grad_norm': '1.38', 'learning_rate': '3.086e-05', 'epoch': '0.4504', 'num_input_tokens_seen': 348114024, 'train_runtime': '7.494e+05', 'train_tokens_per_second': '464.5'}

 45%|████▌     | 101350/225000 [208:09:46<737:18:00, 21.47s/it]
 45%|████▌     | 101351/225000 [208:10:15<809:09:32, 23.56s/it]
 45%|████▌     | 101352/225000 [208:10:35<777:29:29, 22.64s/it]
 45%|████▌     | 101353/225000 [208:10:57<771:28:09, 22.46s/it]
 45%|████▌     | 101354/225000 [208:11:19<760:59:27, 22.16s/it]
 45%|████▌     | 101355/225000 [208:11:44<791:34:23, 23.05s/it]
 45%|████▌     | 101356/225000 [208:12:06<783:04:32, 22.80s/it]
 45%|████▌     | 101357/225000 [208:12:22<712:55:57, 20.76s/it]
 45%|████▌     | 101358/225000 [208:12:39<668:43:43, 19.47s/it]
 45%|████▌     | 101359/225000 [208:13:03<717:59:58, 20.91s/it]
 45%|████▌     | 101360/225000 [208:13:23<709:23:30, 20.66s/it]
 45%|████▌     | 101361/225000 [208:13:49<760:58:47, 22.16s/it]
 45%|████▌     | 101362/225000 [208:14:10<757:42:22, 22.06s/it]
 45%|████▌     | 101363/225000 [208:14:34<772:30:56, 22.49s/it]
 45%|████▌     | 101364/225000 [208:14:54<742:56:39, 21.63s/it]
 45%|████▌     | 101365/225000 [208:15:05<633:43:29, 18.45s/it]
 45%|████▌     | 101366/225000 [208:15:23<633:36:57, 18.45s/it]
 45%|████▌     | 101367/225000 [208:15:45<665:15:07, 19.37s/it]
 45%|████▌     | 101368/225000 [208:16:10<729:24:50, 21.24s/it]
 45%|████▌     | 101369/225000 [208:16:35<764:37:13, 22.26s/it]
 45%|████▌     | 101370/225000 [208:16:53<727:39:48, 21.19s/it]
 45%|████▌     | 101371/225000 [208:17:22<798:24:52, 23.25s/it]
 45%|████▌     | 101372/225000 [208:17:38<724:47:18, 21.11s/it]
 45%|████▌     | 101373/225000 [208:17:59<730:06:39, 21.26s/it]
 45%|████▌     | 101374/225000 [208:18:24<761:37:09, 22.18s/it]
 45%|████▌     | 101375/225000 [208:18:43<735:25:34, 21.42s/it]
 45%|████▌     | 101376/225000 [208:19:13<824:10:10, 24.00s/it]
 45%|████▌     | 101377/225000 [208:19:38<831:52:04, 24.22s/it]
 45%|████▌     | 101378/225000 [208:20:01<822:06:08, 23.94s/it]
 45%|████▌     | 101379/225000 [208:20:26<830:37:01, 24.19s/it]
 45%|████▌     | 101380/225000 [208:20:46<789:29:24, 22.99s/it]
 45%|████▌     | 101381/225000 [208:21:09<782:19:20, 22.78s/it]
 45%|████▌     | 101382/225000 [208:21:38<850:55:45, 24.78s/it]
 45%|████▌     | 101383/225000 [208:22:04<858:50:35, 25.01s/it]
 45%|████▌     | 101384/225000 [208:22:24<810:47:14, 23.61s/it]
 45%|████▌     | 101385/225000 [208:22:40<737:03:54, 21.47s/it]
 45%|████▌     | 101386/225000 [208:23:07<787:01:01, 22.92s/it]
 45%|████▌     | 101387/225000 [208:23:20<684:08:19, 19.92s/it]
 45%|████▌     | 101388/225000 [208:23:49<786:03:27, 22.89s/it]
 45%|████▌     | 101389/225000 [208:24:10<760:26:23, 22.15s/it]
 45%|████▌     | 101390/225000 [208:24:33<771:17:49, 22.46s/it]
 45%|████▌     | 101391/225000 [208:24:56<772:31:33, 22.50s/it]
 45%|████▌     | 101392/225000 [208:25:18<771:48:03, 22.48s/it]
 45%|████▌     | 101393/225000 [208:25:42<787:12:23, 22.93s/it]
 45%|████▌     | 101394/225000 [208:26:04<781:19:21, 22.76s/it]
 45%|████▌     | 101395/225000 [208:26:37<878:20:31, 25.58s/it]
 45%|████▌     | 101396/225000 [208:27:00<856:40:01, 24.95s/it]
 45%|████▌     | 101397/225000 [208:27:16<768:12:39, 22.37s/it]
 45%|████▌     | 101398/225000 [208:27:44<818:17:04, 23.83s/it]
 45%|████▌     | 101399/225000 [208:28:06<801:31:23, 23.35s/it]
 45%|████▌     | 101400/225000 [208:28:29<802:16:32, 23.37s/it]
                                                               
{'loss': '0.8578', 'grad_norm': '1.015', 'learning_rate': '3.085e-05', 'epoch': '0.4507', 'num_input_tokens_seen': 348693192, 'train_runtime': '7.505e+05', 'train_tokens_per_second': '464.6'}

 45%|████▌     | 101400/225000 [208:28:29<802:16:32, 23.37s/it]
 45%|████▌     | 101401/225000 [208:28:44<717:15:36, 20.89s/it]
 45%|████▌     | 101402/225000 [208:29:14<807:09:26, 23.51s/it]
 45%|████▌     | 101403/225000 [208:29:42<851:56:18, 24.81s/it]
 45%|████▌     | 101404/225000 [208:30:02<805:48:32, 23.47s/it]
 45%|████▌     | 101405/225000 [208:30:27<819:40:54, 23.88s/it]
 45%|████▌     | 101406/225000 [208:30:51<820:13:51, 23.89s/it]
 45%|████▌     | 101407/225000 [208:31:11<778:29:17, 22.68s/it]
 45%|████▌     | 101408/225000 [208:31:30<748:35:44, 21.81s/it]
 45%|████▌     | 101409/225000 [208:31:54<766:54:23, 22.34s/it]
 45%|████▌     | 101410/225000 [208:32:19<798:10:21, 23.25s/it]
 45%|████▌     | 101411/225000 [208:32:41<778:08:33, 22.67s/it]
 45%|████▌     | 101412/225000 [208:33:00<743:39:50, 21.66s/it]
 45%|████▌     | 101413/225000 [208:33:20<727:14:41, 21.18s/it]
 45%|████▌     | 101414/225000 [208:33:47<788:50:49, 22.98s/it]
 45%|████▌     | 101415/225000 [208:33:59<668:10:52, 19.46s/it]
 45%|████▌     | 101416/225000 [208:34:28<766:15:01, 22.32s/it]
 45%|████▌     | 101417/225000 [208:34:55<819:32:44, 23.87s/it]
 45%|████▌     | 101418/225000 [208:35:18<808:33:29, 23.55s/it]
 45%|████▌     | 101419/225000 [208:35:45<849:36:08, 24.75s/it]
 45%|████▌     | 101420/225000 [208:36:02<768:07:01, 22.38s/it]
 45%|████▌     | 101421/225000 [208:36:26<778:56:17, 22.69s/it]
 45%|████▌     | 101422/225000 [208:36:45<741:42:57, 21.61s/it]
 45%|████▌     | 101423/225000 [208:37:03<707:28:38, 20.61s/it]
 45%|████▌     | 101424/225000 [208:37:22<687:21:21, 20.02s/it]
 45%|████▌     | 101425/225000 [208:37:46<735:31:51, 21.43s/it]
 45%|████▌     | 101426/225000 [208:38:06<717:54:23, 20.91s/it]
 45%|████▌     | 101427/225000 [208:38:28<723:21:11, 21.07s/it]
 45%|████▌     | 101428/225000 [208:38:51<747:10:51, 21.77s/it]
 45%|████▌     | 101429/225000 [208:39:18<799:47:11, 23.30s/it]
 45%|████▌     | 101430/225000 [208:39:43<815:48:19, 23.77s/it]
 45%|████▌     | 101431/225000 [208:40:02<767:46:21, 22.37s/it]
 45%|████▌     | 101432/225000 [208:40:16<683:35:52, 19.92s/it]
 45%|████▌     | 101433/225000 [208:40:41<736:05:14, 21.45s/it]
 45%|████▌     | 101434/225000 [208:41:02<729:20:11, 21.25s/it]
 45%|████▌     | 101435/225000 [208:41:30<805:08:05, 23.46s/it]
 45%|████▌     | 101436/225000 [208:41:53<796:17:43, 23.20s/it]
 45%|████▌     | 101437/225000 [208:42:24<874:36:45, 25.48s/it]
 45%|████▌     | 101438/225000 [208:42:42<802:46:12, 23.39s/it]
 45%|████▌     | 101439/225000 [208:42:58<722:35:35, 21.05s/it]
 45%|████▌     | 101440/225000 [208:43:23<764:16:22, 22.27s/it]
 45%|████▌     | 101441/225000 [208:43:45<760:51:28, 22.17s/it]
 45%|████▌     | 101442/225000 [208:44:01<695:01:14, 20.25s/it]
 45%|████▌     | 101443/225000 [208:44:29<778:00:31, 22.67s/it]
 45%|████▌     | 101444/225000 [208:44:47<726:58:37, 21.18s/it]
 45%|████▌     | 101445/225000 [208:45:16<805:53:09, 23.48s/it]
 45%|████▌     | 101446/225000 [208:45:43<843:03:30, 24.56s/it]
 45%|████▌     | 101447/225000 [208:46:05<816:29:42, 23.79s/it]
 45%|████▌     | 101448/225000 [208:46:29<824:08:58, 24.01s/it]
 45%|████▌     | 101449/225000 [208:46:55<838:31:51, 24.43s/it]
 45%|████▌     | 101450/225000 [208:47:13<776:29:11, 22.63s/it]
                                                               
{'loss': '0.9104', 'grad_norm': '1.152', 'learning_rate': '3.083e-05', 'epoch': '0.4509', 'num_input_tokens_seen': 349258888, 'train_runtime': '7.516e+05', 'train_tokens_per_second': '464.7'}

 45%|████▌     | 101450/225000 [208:47:13<776:29:11, 22.63s/it]
 45%|████▌     | 101451/225000 [208:47:42<840:08:12, 24.48s/it]
 45%|████▌     | 101452/225000 [208:48:02<796:33:30, 23.21s/it]
 45%|████▌     | 101453/225000 [208:48:24<783:27:10, 22.83s/it]
 45%|████▌     | 101454/225000 [208:48:49<809:50:31, 23.60s/it]
 45%|████▌     | 101455/225000 [208:49:10<783:31:07, 22.83s/it]
 45%|████▌     | 101456/225000 [208:49:38<832:13:24, 24.25s/it]
 45%|████▌     | 101457/225000 [208:49:54<743:37:16, 21.67s/it]
 45%|████▌     | 101458/225000 [208:50:25<843:14:27, 24.57s/it]
 45%|████▌     | 101459/225000 [208:50:41<754:32:08, 21.99s/it]
 45%|████▌     | 101460/225000 [208:51:03<758:00:44, 22.09s/it]
 45%|████▌     | 101461/225000 [208:51:23<732:40:21, 21.35s/it]
 45%|████▌     | 101462/225000 [208:51:40<693:46:07, 20.22s/it]
 45%|████▌     | 101463/225000 [208:52:01<694:03:55, 20.23s/it]
 45%|████▌     | 101464/225000 [208:52:26<749:40:08, 21.85s/it]
 45%|████▌     | 101465/225000 [208:52:46<731:19:12, 21.31s/it]
 45%|████▌     | 101466/225000 [208:53:04<692:12:47, 20.17s/it]
 45%|████▌     | 101467/225000 [208:53:20<653:39:57, 19.05s/it]
 45%|████▌     | 101468/225000 [208:53:40<656:24:47, 19.13s/it]
 45%|████▌     | 101469/225000 [208:53:56<632:29:47, 18.43s/it]
 45%|████▌     | 101470/225000 [208:54:23<711:39:13, 20.74s/it]
 45%|████▌     | 101471/225000 [208:54:42<699:22:15, 20.38s/it]
 45%|████▌     | 101472/225000 [208:55:05<726:33:41, 21.17s/it]
 45%|████▌     | 101473/225000 [208:55:36<821:14:01, 23.93s/it]
 45%|████▌     | 101474/225000 [208:55:59<814:00:04, 23.72s/it]
 45%|████▌     | 101475/225000 [208:56:23<822:35:47, 23.97s/it]
 45%|████▌     | 101476/225000 [208:56:47<818:22:29, 23.85s/it]
 45%|████▌     | 101477/225000 [208:57:08<791:09:59, 23.06s/it]
 45%|████▌     | 101478/225000 [208:57:36<844:52:41, 24.62s/it]
 45%|████▌     | 101479/225000 [208:57:56<788:14:10, 22.97s/it]
 45%|████▌     | 101480/225000 [208:58:15<747:42:00, 21.79s/it]
 45%|████▌     | 101481/225000 [208:58:34<728:05:59, 21.22s/it]
 45%|████▌     | 101482/225000 [208:58:55<718:33:20, 20.94s/it]
 45%|████▌     | 101483/225000 [208:59:16<726:30:21, 21.17s/it]
 45%|████▌     | 101484/225000 [208:59:32<673:48:31, 19.64s/it]
 45%|████▌     | 101485/225000 [208:59:55<704:12:52, 20.53s/it]
 45%|████▌     | 101486/225000 [209:00:22<772:54:55, 22.53s/it]
 45%|████▌     | 101487/225000 [209:00:42<747:10:37, 21.78s/it]
 45%|████▌     | 101488/225000 [209:01:01<710:16:14, 20.70s/it]
 45%|████▌     | 101489/225000 [209:01:19<691:44:03, 20.16s/it]
 45%|████▌     | 101490/225000 [209:01:36<656:15:21, 19.13s/it]
 45%|████▌     | 101491/225000 [209:02:02<724:14:17, 21.11s/it]
 45%|████▌     | 101492/225000 [209:02:21<706:01:12, 20.58s/it]
 45%|████▌     | 101493/225000 [209:02:52<808:06:15, 23.55s/it]
 45%|████▌     | 101494/225000 [209:03:13<782:10:30, 22.80s/it]
 45%|████▌     | 101495/225000 [209:03:28<709:26:41, 20.68s/it]
 45%|████▌     | 101496/225000 [209:03:46<679:04:01, 19.79s/it]
 45%|████▌     | 101497/225000 [209:04:11<729:52:07, 21.28s/it]
 45%|████▌     | 101498/225000 [209:04:30<704:52:32, 20.55s/it]
 45%|████▌     | 101499/225000 [209:04:55<755:30:38, 22.02s/it]
 45%|████▌     | 101500/225000 [209:05:20<784:44:20, 22.87s/it]
                                                               
{'loss': '0.8836', 'grad_norm': '1.025', 'learning_rate': '3.081e-05', 'epoch': '0.4511', 'num_input_tokens_seen': 349803712, 'train_runtime': '7.527e+05', 'train_tokens_per_second': '464.7'}

 45%|████▌     | 101500/225000 [209:05:20<784:44:20, 22.87s/it]
 45%|████▌     | 101501/225000 [209:05:48<833:42:02, 24.30s/it]
 45%|████▌     | 101502/225000 [209:06:11<823:06:05, 23.99s/it]
 45%|████▌     | 101503/225000 [209:06:33<801:12:42, 23.36s/it]
 45%|████▌     | 101504/225000 [209:06:58<819:29:22, 23.89s/it]
 45%|████▌     | 101505/225000 [209:07:17<770:40:18, 22.47s/it]
 45%|████▌     | 101506/225000 [209:07:45<829:48:15, 24.19s/it]
 45%|████▌     | 101507/225000 [209:08:07<804:10:28, 23.44s/it]
 45%|████▌     | 101508/225000 [209:08:33<829:40:05, 24.19s/it]
 45%|████▌     | 101509/225000 [209:08:50<760:00:58, 22.16s/it]
 45%|████▌     | 101510/225000 [209:09:03<656:49:24, 19.15s/it]
 45%|████▌     | 101511/225000 [209:09:22<664:38:15, 19.38s/it]
 45%|████▌     | 101512/225000 [209:09:47<716:18:18, 20.88s/it]
 45%|████▌     | 101513/225000 [209:10:10<742:22:24, 21.64s/it]
 45%|████▌     | 101514/225000 [209:10:36<783:47:07, 22.85s/it]
 45%|████▌     | 101515/225000 [209:10:56<753:57:28, 21.98s/it]
 45%|████▌     | 101516/225000 [209:11:29<864:37:35, 25.21s/it]
 45%|████▌     | 101517/225000 [209:11:49<813:42:36, 23.72s/it]
 45%|████▌     | 101518/225000 [209:12:08<766:51:58, 22.36s/it]
 45%|████▌     | 101519/225000 [209:12:39<853:26:55, 24.88s/it]
 45%|████▌     | 101520/225000 [209:13:07<888:37:50, 25.91s/it]
 45%|████▌     | 101521/225000 [209:13:26<814:12:15, 23.74s/it]
 45%|████▌     | 101522/225000 [209:13:54<862:00:13, 25.13s/it]
 45%|████▌     | 101523/225000 [209:14:20<869:38:02, 25.35s/it]
 45%|████▌     | 101524/225000 [209:14:48<891:45:40, 26.00s/it]
 45%|████▌     | 101525/225000 [209:15:06<816:36:50, 23.81s/it]
 45%|████▌     | 101526/225000 [209:15:28<797:45:27, 23.26s/it]
 45%|████▌     | 101527/225000 [209:15:49<769:20:30, 22.43s/it]
 45%|████▌     | 101528/225000 [209:16:06<716:00:29, 20.88s/it]
 45%|████▌     | 101529/225000 [209:16:23<679:54:48, 19.82s/it]
 45%|████▌     | 101530/225000 [209:16:57<826:22:58, 24.09s/it]
 45%|████▌     | 101531/225000 [209:17:12<727:44:56, 21.22s/it]
 45%|████▌     | 101532/225000 [209:17:31<703:22:22, 20.51s/it]
 45%|████▌     | 101533/225000 [209:17:46<647:16:05, 18.87s/it]
 45%|████▌     | 101534/225000 [209:18:12<722:45:58, 21.07s/it]
 45%|████▌     | 101535/225000 [209:18:39<784:28:54, 22.87s/it]
 45%|████▌     | 101536/225000 [209:18:53<688:02:17, 20.06s/it]
 45%|████▌     | 101537/225000 [209:19:06<623:46:23, 18.19s/it]
 45%|████▌     | 101538/225000 [209:19:36<737:13:15, 21.50s/it]
 45%|████▌     | 101539/225000 [209:19:53<694:03:31, 20.24s/it]
 45%|████▌     | 101540/225000 [209:20:15<711:33:46, 20.75s/it]
 45%|████▌     | 101541/225000 [209:20:36<718:53:30, 20.96s/it]
 45%|████▌     | 101542/225000 [209:20:56<704:06:03, 20.53s/it]
 45%|████▌     | 101543/225000 [209:21:25<795:01:02, 23.18s/it]
 45%|████▌     | 101544/225000 [209:21:38<690:10:13, 20.13s/it]
 45%|████▌     | 101545/225000 [209:21:51<616:13:54, 17.97s/it]
 45%|████▌     | 101546/225000 [209:22:10<625:58:03, 18.25s/it]
 45%|████▌     | 101547/225000 [209:22:40<750:26:29, 21.88s/it]
 45%|████▌     | 101548/225000 [209:23:05<780:29:14, 22.76s/it]
 45%|████▌     | 101549/225000 [209:23:28<778:16:20, 22.70s/it]
 45%|████▌     | 101550/225000 [209:23:51<784:36:51, 22.88s/it]
                                                               
{'loss': '0.908', 'grad_norm': '1.496', 'learning_rate': '3.079e-05', 'epoch': '0.4513', 'num_input_tokens_seen': 350359640, 'train_runtime': '7.538e+05', 'train_tokens_per_second': '464.8'}

 45%|████▌     | 101550/225000 [209:23:51<784:36:51, 22.88s/it]
 45%|████▌     | 101551/225000 [209:24:15<790:08:20, 23.04s/it]
 45%|████▌     | 101552/225000 [209:24:33<746:03:55, 21.76s/it]
 45%|████▌     | 101553/225000 [209:24:56<757:49:52, 22.10s/it]
 45%|████▌     | 101554/225000 [209:25:24<818:35:50, 23.87s/it]
 45%|████▌     | 101555/225000 [209:25:38<717:53:30, 20.94s/it]
 45%|████▌     | 101556/225000 [209:25:56<685:48:45, 20.00s/it]
 45%|████▌     | 101557/225000 [209:26:18<709:54:13, 20.70s/it]
 45%|████▌     | 101558/225000 [209:26:44<764:14:53, 22.29s/it]
 45%|████▌     | 101559/225000 [209:27:04<737:41:04, 21.51s/it]
 45%|████▌     | 101560/225000 [209:27:18<654:07:22, 19.08s/it]
 45%|████▌     | 101561/225000 [209:27:44<733:51:47, 21.40s/it]
 45%|████▌     | 101562/225000 [209:28:02<690:22:33, 20.13s/it]
 45%|████▌     | 101563/225000 [209:28:27<742:04:28, 21.64s/it]
 45%|████▌     | 101564/225000 [209:28:41<661:51:45, 19.30s/it]
 45%|████▌     | 101565/225000 [209:29:06<724:19:43, 21.13s/it]
 45%|████▌     | 101566/225000 [209:29:21<661:48:09, 19.30s/it]
 45%|████▌     | 101567/225000 [209:29:47<732:49:34, 21.37s/it]
 45%|████▌     | 101568/225000 [209:30:08<726:58:25, 21.20s/it]
 45%|████▌     | 101569/225000 [209:30:22<657:59:03, 19.19s/it]
 45%|████▌     | 101570/225000 [209:30:37<614:09:36, 17.91s/it]
 45%|████▌     | 101571/225000 [209:31:03<691:28:20, 20.17s/it]
 45%|████▌     | 101572/225000 [209:31:13<586:56:19, 17.12s/it]
 45%|████▌     | 101573/225000 [209:31:37<660:28:32, 19.26s/it]
 45%|████▌     | 101574/225000 [209:32:07<773:12:52, 22.55s/it]
 45%|████▌     | 101575/225000 [209:32:25<723:35:59, 21.11s/it]
 45%|████▌     | 101576/225000 [209:32:48<740:32:21, 21.60s/it]
 45%|████▌     | 101577/225000 [209:33:06<708:55:38, 20.68s/it]
 45%|████▌     | 101578/225000 [209:33:31<745:15:14, 21.74s/it]
 45%|████▌     | 101579/225000 [209:33:48<698:21:49, 20.37s/it]
 45%|████▌     | 101580/225000 [209:34:06<677:17:03, 19.76s/it]
 45%|████▌     | 101581/225000 [209:34:32<744:19:48, 21.71s/it]
 45%|████▌     | 101582/225000 [209:34:55<754:10:09, 22.00s/it]
 45%|████▌     | 101583/225000 [209:35:14<724:47:39, 21.14s/it]
 45%|████▌     | 101584/225000 [209:35:34<715:52:34, 20.88s/it]
 45%|████▌     | 101585/225000 [209:35:59<751:39:16, 21.93s/it]
 45%|████▌     | 101586/225000 [209:36:26<805:51:38, 23.51s/it]
 45%|████▌     | 101587/225000 [209:36:48<789:57:43, 23.04s/it]
 45%|████▌     | 101588/225000 [209:37:13<806:12:03, 23.52s/it]
 45%|████▌     | 101589/225000 [209:37:38<821:19:11, 23.96s/it]
 45%|████▌     | 101590/225000 [209:37:59<792:31:23, 23.12s/it]
 45%|████▌     | 101591/225000 [209:38:11<676:28:10, 19.73s/it]
 45%|████▌     | 101592/225000 [209:38:28<657:27:25, 19.18s/it]
 45%|████▌     | 101593/225000 [209:38:52<703:23:17, 20.52s/it]
 45%|████▌     | 101594/225000 [209:39:12<693:16:42, 20.22s/it]
 45%|████▌     | 101595/225000 [209:39:25<620:42:38, 18.11s/it]
 45%|████▌     | 101596/225000 [209:39:53<723:58:51, 21.12s/it]
 45%|████▌     | 101597/225000 [209:40:22<801:12:30, 23.37s/it]
 45%|████▌     | 101598/225000 [209:40:31<659:37:52, 19.24s/it]
 45%|████▌     | 101599/225000 [209:40:58<733:53:30, 21.41s/it]
 45%|████▌     | 101600/225000 [209:41:13<668:47:12, 19.51s/it]
                                                               
{'loss': '0.8863', 'grad_norm': '1.259', 'learning_rate': '3.077e-05', 'epoch': '0.4516', 'num_input_tokens_seen': 350891784, 'train_runtime': '7.549e+05', 'train_tokens_per_second': '464.8'}

 45%|████▌     | 101600/225000 [209:41:13<668:47:12, 19.51s/it]
 45%|████▌     | 101601/225000 [209:41:29<630:53:22, 18.41s/it]
 45%|████▌     | 101602/225000 [209:41:55<716:56:44, 20.92s/it]
 45%|████▌     | 101603/225000 [209:42:14<689:23:41, 20.11s/it]
 45%|████▌     | 101604/225000 [209:42:43<781:04:14, 22.79s/it]
 45%|████▌     | 101605/225000 [209:42:51<637:01:29, 18.58s/it]
 45%|████▌     | 101606/225000 [209:43:10<634:09:40, 18.50s/it]
 45%|████▌     | 101607/225000 [209:43:32<678:06:11, 19.78s/it]
 45%|████▌     | 101608/225000 [209:43:56<712:19:57, 20.78s/it]
 45%|████▌     | 101609/225000 [209:44:21<763:22:23, 22.27s/it]
 45%|████▌     | 101610/225000 [209:44:45<776:04:33, 22.64s/it]
 45%|████▌     | 101611/225000 [209:45:10<802:58:59, 23.43s/it]
 45%|████▌     | 101612/225000 [209:45:30<770:14:52, 22.47s/it]
 45%|████▌     | 101613/225000 [209:45:54<783:58:39, 22.87s/it]
 45%|████▌     | 101614/225000 [209:46:09<701:04:38, 20.46s/it]
 45%|████▌     | 101615/225000 [209:46:36<767:59:24, 22.41s/it]
 45%|████▌     | 101616/225000 [209:47:03<816:40:19, 23.83s/it]
 45%|████▌     | 101617/225000 [209:47:28<825:00:31, 24.07s/it]
 45%|████▌     | 101618/225000 [209:47:42<726:57:13, 21.21s/it]
 45%|████▌     | 101619/225000 [209:48:01<702:23:50, 20.49s/it]
 45%|████▌     | 101620/225000 [209:48:24<723:22:29, 21.11s/it]
 45%|████▌     | 101621/225000 [209:48:45<731:57:04, 21.36s/it]
 45%|████▌     | 101622/225000 [209:49:04<698:48:24, 20.39s/it]
 45%|████▌     | 101623/225000 [209:49:18<632:17:37, 18.45s/it]
 45%|████▌     | 101624/225000 [209:49:38<648:57:48, 18.94s/it]
 45%|████▌     | 101625/225000 [209:50:01<690:54:30, 20.16s/it]
 45%|████▌     | 101626/225000 [209:50:21<695:59:42, 20.31s/it]
 45%|████▌     | 101627/225000 [209:50:48<763:08:46, 22.27s/it]
 45%|████▌     | 101628/225000 [209:51:12<781:37:03, 22.81s/it]
 45%|████▌     | 101629/225000 [209:51:40<830:44:48, 24.24s/it]
 45%|████▌     | 101630/225000 [209:52:03<822:41:56, 24.01s/it]
 45%|████▌     | 101631/225000 [209:52:26<814:46:39, 23.78s/it]
 45%|████▌     | 101632/225000 [209:52:49<798:04:50, 23.29s/it]
 45%|████▌     | 101633/225000 [209:53:11<792:40:08, 23.13s/it]
 45%|████▌     | 101634/225000 [209:53:29<738:22:07, 21.55s/it]
 45%|████▌     | 101635/225000 [209:53:44<668:27:45, 19.51s/it]
 45%|████▌     | 101636/225000 [209:54:03<664:39:36, 19.40s/it]
 45%|████▌     | 101637/225000 [209:54:21<653:10:51, 19.06s/it]
 45%|████▌     | 101638/225000 [209:54:49<743:44:19, 21.70s/it]
 45%|████▌     | 101639/225000 [209:55:11<739:19:50, 21.58s/it]
 45%|████▌     | 101640/225000 [209:55:22<634:35:28, 18.52s/it]
 45%|████▌     | 101641/225000 [209:55:39<624:04:52, 18.21s/it]
 45%|████▌     | 101642/225000 [209:55:53<573:17:46, 16.73s/it]
 45%|████▌     | 101643/225000 [209:56:19<668:49:09, 19.52s/it]
 45%|████▌     | 101644/225000 [209:56:40<683:41:44, 19.95s/it]
 45%|████▌     | 101645/225000 [209:57:04<728:28:48, 21.26s/it]
 45%|████▌     | 101646/225000 [209:57:26<739:05:41, 21.57s/it]
 45%|████▌     | 101647/225000 [209:57:48<741:04:39, 21.63s/it]
 45%|████▌     | 101648/225000 [209:58:11<750:28:06, 21.90s/it]
 45%|████▌     | 101649/225000 [209:58:33<757:17:06, 22.10s/it]
 45%|████▌     | 101650/225000 [209:58:58<781:46:18, 22.82s/it]
                                                               
{'loss': '0.8915', 'grad_norm': '0.9695', 'learning_rate': '3.076e-05', 'epoch': '0.4518', 'num_input_tokens_seen': 351435552, 'train_runtime': '7.559e+05', 'train_tokens_per_second': '464.9'}

 45%|████▌     | 101650/225000 [209:58:58<781:46:18, 22.82s/it]
 45%|████▌     | 101651/225000 [209:59:12<698:16:04, 20.38s/it]
 45%|████▌     | 101652/225000 [209:59:35<720:31:57, 21.03s/it]
 45%|████▌     | 101653/225000 [209:59:51<666:34:58, 19.45s/it]
 45%|████▌     | 101654/225000 [210:00:21<781:58:39, 22.82s/it]
 45%|████▌     | 101655/225000 [210:00:42<764:17:25, 22.31s/it]
 45%|████▌     | 101656/225000 [210:01:04<753:49:13, 22.00s/it]
 45%|████▌     | 101657/225000 [210:01:21<707:47:48, 20.66s/it]
 45%|████▌     | 101658/225000 [210:01:41<702:25:19, 20.50s/it]
 45%|████▌     | 101659/225000 [210:02:09<777:36:14, 22.70s/it]
 45%|████▌     | 101660/225000 [210:02:30<755:37:22, 22.05s/it]
 45%|████▌     | 101661/225000 [210:02:53<772:03:00, 22.53s/it]
 45%|████▌     | 101662/225000 [210:03:13<738:22:34, 21.55s/it]
 45%|████▌     | 101663/225000 [210:03:29<682:31:17, 19.92s/it]
 45%|████▌     | 101664/225000 [210:03:49<690:14:53, 20.15s/it]
 45%|████▌     | 101665/225000 [210:04:09<687:47:48, 20.08s/it]
 45%|████▌     | 101666/225000 [210:04:37<761:12:13, 22.22s/it]
 45%|████▌     | 101667/225000 [210:05:00<771:32:48, 22.52s/it]
 45%|████▌     | 101668/225000 [210:05:25<798:15:44, 23.30s/it]
 45%|████▌     | 101669/225000 [210:05:43<744:41:37, 21.74s/it]
 45%|████▌     | 101670/225000 [210:06:08<777:30:18, 22.70s/it]
 45%|████▌     | 101671/225000 [210:06:25<722:33:21, 21.09s/it]
 45%|████▌     | 101672/225000 [210:06:51<767:11:53, 22.39s/it]
 45%|████▌     | 101673/225000 [210:07:17<803:23:21, 23.45s/it]
 45%|████▌     | 101674/225000 [210:07:36<757:44:25, 22.12s/it]
 45%|████▌     | 101675/225000 [210:08:05<826:44:07, 24.13s/it]
 45%|████▌     | 101676/225000 [210:08:24<777:57:25, 22.71s/it]
 45%|████▌     | 101677/225000 [210:08:51<827:50:57, 24.17s/it]
 45%|████▌     | 101678/225000 [210:09:05<720:16:45, 21.03s/it]
 45%|████▌     | 101679/225000 [210:09:30<763:10:27, 22.28s/it]
 45%|████▌     | 101680/225000 [210:09:52<752:29:26, 21.97s/it]
 45%|████▌     | 101681/225000 [210:10:18<794:50:58, 23.20s/it]
 45%|████▌     | 101682/225000 [210:10:44<826:28:52, 24.13s/it]
 45%|████▌     | 101683/225000 [210:11:10<846:44:53, 24.72s/it]
 45%|████▌     | 101684/225000 [210:11:28<781:26:05, 22.81s/it]
 45%|████▌     | 101685/225000 [210:11:50<773:21:36, 22.58s/it]
 45%|████▌     | 101686/225000 [210:12:11<753:52:45, 22.01s/it]
 45%|████▌     | 101687/225000 [210:12:24<661:24:46, 19.31s/it]
 45%|████▌     | 101688/225000 [210:12:48<704:56:23, 20.58s/it]
 45%|████▌     | 101689/225000 [210:13:03<646:30:03, 18.87s/it]
 45%|████▌     | 101690/225000 [210:13:18<613:23:03, 17.91s/it]
 45%|████▌     | 101691/225000 [210:13:38<626:55:06, 18.30s/it]
 45%|████▌     | 101692/225000 [210:14:05<720:56:22, 21.05s/it]
 45%|████▌     | 101693/225000 [210:14:16<613:56:58, 17.92s/it]
 45%|████▌     | 101694/225000 [210:14:44<718:33:00, 20.98s/it]
 45%|████▌     | 101695/225000 [210:15:08<751:02:51, 21.93s/it]
 45%|████▌     | 101696/225000 [210:15:24<691:15:10, 20.18s/it]
 45%|████▌     | 101697/225000 [210:15:38<630:44:28, 18.42s/it]
 45%|████▌     | 101698/225000 [210:16:04<710:18:56, 20.74s/it]
 45%|████▌     | 101699/225000 [210:16:27<725:50:15, 21.19s/it]
 45%|████▌     | 101700/225000 [210:16:44<690:56:55, 20.17s/it]
                                                               
{'loss': '0.924', 'grad_norm': '1.014', 'learning_rate': '3.074e-05', 'epoch': '0.452', 'num_input_tokens_seen': 351977720, 'train_runtime': '7.57e+05', 'train_tokens_per_second': '465'}

 45%|████▌     | 101700/225000 [210:16:44<690:56:55, 20.17s/it]
 45%|████▌     | 101701/225000 [210:17:06<700:30:00, 20.45s/it]
 45%|████▌     | 101702/225000 [210:17:32<762:44:34, 22.27s/it]
 45%|████▌     | 101703/225000 [210:17:51<729:57:54, 21.31s/it]
 45%|████▌     | 101704/225000 [210:18:07<676:49:32, 19.76s/it]
 45%|████▌     | 101705/225000 [210:18:37<777:11:48, 22.69s/it]
 45%|████▌     | 101706/225000 [210:18:53<713:22:29, 20.83s/it]
 45%|████▌     | 101707/225000 [210:19:11<681:34:33, 19.90s/it]
 45%|████▌     | 101708/225000 [210:19:30<667:12:38, 19.48s/it]
 45%|████▌     | 101709/225000 [210:19:44<616:25:47, 18.00s/it]
 45%|████▌     | 101710/225000 [210:20:09<682:39:18, 19.93s/it]
 45%|████▌     | 101711/225000 [210:20:32<721:28:29, 21.07s/it]
 45%|████▌     | 101712/225000 [210:20:44<624:03:50, 18.22s/it]
 45%|████▌     | 101713/225000 [210:21:04<641:49:47, 18.74s/it]
 45%|████▌     | 101714/225000 [210:21:27<689:41:14, 20.14s/it]
 45%|████▌     | 101715/225000 [210:21:54<761:33:18, 22.24s/it]
 45%|████▌     | 101716/225000 [210:22:16<758:07:24, 22.14s/it]
 45%|████▌     | 101717/225000 [210:22:34<715:48:05, 20.90s/it]
 45%|████▌     | 101718/225000 [210:23:02<790:57:57, 23.10s/it]
 45%|████▌     | 101719/225000 [210:23:25<785:10:09, 22.93s/it]
 45%|████▌     | 101720/225000 [210:23:52<822:03:59, 24.01s/it]
 45%|████▌     | 101721/225000 [210:24:13<798:01:43, 23.30s/it]
 45%|████▌     | 101722/225000 [210:24:31<739:13:55, 21.59s/it]
 45%|████▌     | 101723/225000 [210:24:40<616:14:06, 18.00s/it]
 45%|████▌     | 101724/225000 [210:25:01<638:56:06, 18.66s/it]
 45%|████▌     | 101725/225000 [210:25:13<575:51:56, 16.82s/it]
 45%|████▌     | 101726/225000 [210:25:34<614:43:23, 17.95s/it]
 45%|████▌     | 101727/225000 [210:25:53<629:32:12, 18.38s/it]
 45%|████▌     | 101728/225000 [210:26:08<591:00:27, 17.26s/it]
 45%|████▌     | 101729/225000 [210:26:34<680:44:09, 19.88s/it]
 45%|████▌     | 101730/225000 [210:26:53<679:05:53, 19.83s/it]
 45%|████▌     | 101731/225000 [210:27:24<793:53:14, 23.19s/it]
 45%|████▌     | 101732/225000 [210:27:42<736:52:58, 21.52s/it]
 45%|████▌     | 101733/225000 [210:28:09<788:37:49, 23.03s/it]
 45%|████▌     | 101734/225000 [210:28:23<697:09:24, 20.36s/it]
 45%|████▌     | 101735/225000 [210:28:36<625:42:23, 18.27s/it]
 45%|████▌     | 101736/225000 [210:29:02<706:02:14, 20.62s/it]
 45%|████▌     | 101737/225000 [210:29:20<671:27:31, 19.61s/it]
 45%|████▌     | 101738/225000 [210:29:40<677:26:52, 19.79s/it]
 45%|████▌     | 101739/225000 [210:30:01<691:41:14, 20.20s/it]
 45%|████▌     | 101740/225000 [210:30:25<733:10:26, 21.41s/it]
 45%|████▌     | 101741/225000 [210:30:42<683:27:53, 19.96s/it]
 45%|████▌     | 101742/225000 [210:30:57<635:01:18, 18.55s/it]
 45%|████▌     | 101743/225000 [210:31:25<732:19:35, 21.39s/it]
 45%|████▌     | 101744/225000 [210:31:47<735:18:15, 21.48s/it]
 45%|████▌     | 101745/225000 [210:32:17<825:26:02, 24.11s/it]
 45%|████▌     | 101746/225000 [210:32:36<770:46:39, 22.51s/it]
 45%|████▌     | 101747/225000 [210:33:01<794:22:02, 23.20s/it]
 45%|████▌     | 101748/225000 [210:33:17<722:06:45, 21.09s/it]
 45%|████▌     | 101749/225000 [210:33:37<711:52:12, 20.79s/it]
 45%|████▌     | 101750/225000 [210:33:55<682:18:07, 19.93s/it]
                                                               
{'loss': '0.8942', 'grad_norm': '1.543', 'learning_rate': '3.072e-05', 'epoch': '0.4522', 'num_input_tokens_seen': 352491264, 'train_runtime': '7.58e+05', 'train_tokens_per_second': '465'}

 45%|████▌     | 101750/225000 [210:33:55<682:18:07, 19.93s/it]
 45%|████▌     | 101751/225000 [210:34:14<672:15:22, 19.64s/it]
 45%|████▌     | 101752/225000 [210:34:39<728:06:22, 21.27s/it]
 45%|████▌     | 101753/225000 [210:34:56<686:31:57, 20.05s/it]
 45%|████▌     | 101754/225000 [210:35:19<721:17:30, 21.07s/it]
 45%|████▌     | 101755/225000 [210:35:45<764:37:41, 22.33s/it]
 45%|████▌     | 101756/225000 [210:36:05<745:15:47, 21.77s/it]
 45%|████▌     | 101757/225000 [210:36:33<804:15:44, 23.49s/it]
 45%|████▌     | 101758/225000 [210:36:47<711:19:23, 20.78s/it]
 45%|████▌     | 101759/225000 [210:37:12<756:33:58, 22.10s/it]
 45%|████▌     | 101760/225000 [210:37:30<708:57:16, 20.71s/it]
 45%|████▌     | 101761/225000 [210:38:00<810:34:47, 23.68s/it]
 45%|████▌     | 101762/225000 [210:38:18<753:31:05, 22.01s/it]
 45%|████▌     | 101763/225000 [210:38:35<695:43:36, 20.32s/it]
 45%|████▌     | 101764/225000 [210:38:53<671:26:57, 19.61s/it]
 45%|████▌     | 101765/225000 [210:39:11<660:50:47, 19.30s/it]
 45%|████▌     | 101766/225000 [210:39:33<688:52:12, 20.12s/it]
 45%|████▌     | 101767/225000 [210:40:01<763:31:55, 22.31s/it]
 45%|████▌     | 101768/225000 [210:40:18<709:44:48, 20.73s/it]
 45%|████▌     | 101769/225000 [210:40:37<691:27:35, 20.20s/it]
 45%|████▌     | 101770/225000 [210:41:00<720:41:23, 21.05s/it]
 45%|████▌     | 101771/225000 [210:41:21<721:53:19, 21.09s/it]
 45%|████▌     | 101772/225000 [210:41:48<782:09:09, 22.85s/it]
 45%|████▌     | 101773/225000 [210:42:12<790:17:17, 23.09s/it]
 45%|████▌     | 101774/225000 [210:42:30<744:17:12, 21.74s/it]
 45%|████▌     | 101775/225000 [210:42:44<663:10:55, 19.37s/it]
 45%|████▌     | 101776/225000 [210:43:05<674:36:29, 19.71s/it]
 45%|████▌     | 101777/225000 [210:43:24<670:05:16, 19.58s/it]
 45%|████▌     | 101778/225000 [210:43:43<661:51:00, 19.34s/it]
 45%|████▌     | 101779/225000 [210:44:08<720:02:35, 21.04s/it]
 45%|████▌     | 101780/225000 [210:44:19<619:52:33, 18.11s/it]
 45%|████▌     | 101781/225000 [210:44:42<674:01:41, 19.69s/it]
 45%|████▌     | 101782/225000 [210:44:58<636:08:21, 18.59s/it]
 45%|████▌     | 101783/225000 [210:45:29<758:27:41, 22.16s/it]
 45%|████▌     | 101784/225000 [210:45:52<768:50:29, 22.46s/it]
 45%|████▌     | 101785/225000 [210:46:15<778:10:37, 22.74s/it]
 45%|████▌     | 101786/225000 [210:46:37<762:37:52, 22.28s/it]
 45%|████▌     | 101787/225000 [210:46:45<623:21:52, 18.21s/it]
 45%|████▌     | 101788/225000 [210:47:06<649:56:07, 18.99s/it]
 45%|████▌     | 101789/225000 [210:47:24<635:15:53, 18.56s/it]
 45%|████▌     | 101790/225000 [210:47:48<697:36:18, 20.38s/it]
 45%|████▌     | 101791/225000 [210:48:18<795:13:44, 23.24s/it]
 45%|████▌     | 101792/225000 [210:48:38<762:52:52, 22.29s/it]
 45%|████▌     | 101793/225000 [210:48:59<743:20:20, 21.72s/it]
 45%|████▌     | 101794/225000 [210:49:21<752:02:56, 21.97s/it]
 45%|████▌     | 101795/225000 [210:49:41<728:29:42, 21.29s/it]
 45%|████▌     | 101796/225000 [210:50:01<718:53:30, 21.01s/it]
 45%|████▌     | 101797/225000 [210:50:20<694:29:41, 20.29s/it]
 45%|████▌     | 101798/225000 [210:50:46<758:02:18, 22.15s/it]
 45%|████▌     | 101799/225000 [210:51:10<773:17:56, 22.60s/it]
 45%|████▌     | 101800/225000 [210:51:26<704:27:24, 20.58s/it]
                                                               
{'loss': '0.8853', 'grad_norm': '1.238', 'learning_rate': '3.07e-05', 'epoch': '0.4524', 'num_input_tokens_seen': 353016328, 'train_runtime': '7.591e+05', 'train_tokens_per_second': '465.1'}

 45%|████▌     | 101800/225000 [210:51:26<704:27:24, 20.58s/it]
 45%|████▌     | 101801/225000 [210:51:45<691:10:08, 20.20s/it]
 45%|████▌     | 101802/225000 [210:51:54<576:52:49, 16.86s/it]
 45%|████▌     | 101803/225000 [210:52:08<544:48:27, 15.92s/it]
 45%|████▌     | 101804/225000 [210:52:34<647:11:07, 18.91s/it]
 45%|████▌     | 101805/225000 [210:53:00<719:39:01, 21.03s/it]
 45%|████▌     | 101806/225000 [210:53:19<695:20:29, 20.32s/it]
 45%|████▌     | 101807/225000 [210:53:34<645:04:40, 18.85s/it]
 45%|████▌     | 101808/225000 [210:54:00<716:30:15, 20.94s/it]
 45%|████▌     | 101809/225000 [210:54:26<772:45:39, 22.58s/it]
 45%|████▌     | 101810/225000 [210:54:45<732:38:23, 21.41s/it]
 45%|████▌     | 101811/225000 [210:55:05<721:14:13, 21.08s/it]
 45%|████▌     | 101812/225000 [210:55:32<779:40:39, 22.79s/it]
 45%|████▌     | 101813/225000 [210:55:50<735:02:27, 21.48s/it]
 45%|████▌     | 101814/225000 [210:56:14<761:49:30, 22.26s/it]
 45%|████▌     | 101815/225000 [210:56:37<768:42:35, 22.47s/it]
 45%|████▌     | 101816/225000 [210:56:56<733:09:01, 21.43s/it]
 45%|████▌     | 101817/225000 [210:57:16<712:54:20, 20.83s/it]
 45%|████▌     | 101818/225000 [210:57:33<679:21:47, 19.85s/it]
 45%|████▌     | 101819/225000 [210:57:57<715:51:05, 20.92s/it]
 45%|████▌     | 101820/225000 [210:58:15<690:11:48, 20.17s/it]
 45%|████▌     | 101821/225000 [210:58:38<717:24:00, 20.97s/it]
 45%|████▌     | 101822/225000 [210:58:56<686:33:33, 20.07s/it]
 45%|████▌     | 101823/225000 [210:59:20<725:47:31, 21.21s/it]
 45%|████▌     | 101824/225000 [210:59:38<695:51:47, 20.34s/it]
 45%|████▌     | 101825/225000 [210:59:57<675:15:24, 19.74s/it]
 45%|████▌     | 101826/225000 [211:00:17<684:32:31, 20.01s/it]
 45%|████▌     | 101827/225000 [211:00:37<686:20:42, 20.06s/it]
 45%|████▌     | 101828/225000 [211:00:55<659:28:37, 19.27s/it]
 45%|████▌     | 101829/225000 [211:01:22<743:00:02, 21.72s/it]
 45%|████▌     | 101830/225000 [211:01:43<729:30:56, 21.32s/it]
 45%|████▌     | 101831/225000 [211:01:58<673:23:00, 19.68s/it]
 45%|████▌     | 101832/225000 [211:02:17<657:32:24, 19.22s/it]
 45%|████▌     | 101833/225000 [211:02:39<688:14:47, 20.12s/it]
 45%|████▌     | 101834/225000 [211:03:05<746:18:47, 21.81s/it]
 45%|████▌     | 101835/225000 [211:03:22<704:46:00, 20.60s/it]
 45%|████▌     | 101836/225000 [211:03:47<744:34:57, 21.76s/it]
 45%|████▌     | 101837/225000 [211:04:09<748:55:25, 21.89s/it]
 45%|████▌     | 101838/225000 [211:04:25<689:38:18, 20.16s/it]
 45%|████▌     | 101839/225000 [211:04:51<748:44:31, 21.89s/it]
 45%|████▌     | 101840/225000 [211:05:14<754:49:37, 22.06s/it]
 45%|████▌     | 101841/225000 [211:05:33<730:45:53, 21.36s/it]
 45%|████▌     | 101842/225000 [211:05:58<761:35:27, 22.26s/it]
 45%|████▌     | 101843/225000 [211:06:16<723:37:45, 21.15s/it]
 45%|████▌     | 101844/225000 [211:06:33<683:20:14, 19.97s/it]
 45%|████▌     | 101845/225000 [211:06:47<613:20:34, 17.93s/it]
 45%|████▌     | 101846/225000 [211:07:05<616:56:02, 18.03s/it]
 45%|████▌     | 101847/225000 [211:07:38<767:32:23, 22.44s/it]
 45%|████▌     | 101848/225000 [211:07:58<748:21:07, 21.88s/it]
 45%|████▌     | 101849/225000 [211:08:14<690:52:39, 20.20s/it]
 45%|████▌     | 101850/225000 [211:08:29<637:39:58, 18.64s/it]
                                                               
{'loss': '0.879', 'grad_norm': '0.9034', 'learning_rate': '3.069e-05', 'epoch': '0.4527', 'num_input_tokens_seen': 353534176, 'train_runtime': '7.601e+05', 'train_tokens_per_second': '465.1'}

 45%|████▌     | 101850/225000 [211:08:29<637:39:58, 18.64s/it]
 45%|████▌     | 101851/225000 [211:08:59<747:00:35, 21.84s/it]
 45%|████▌     | 101852/225000 [211:09:18<723:16:31, 21.14s/it]
 45%|████▌     | 101853/225000 [211:09:34<663:07:25, 19.39s/it]
 45%|████▌     | 101854/225000 [211:10:01<742:35:20, 21.71s/it]
 45%|████▌     | 101855/225000 [211:10:26<776:33:07, 22.70s/it]
 45%|████▌     | 101856/225000 [211:10:48<773:58:31, 22.63s/it]
 45%|████▌     | 101857/225000 [211:11:12<788:42:19, 23.06s/it]
 45%|████▌     | 101858/225000 [211:11:27<701:57:32, 20.52s/it]
 45%|████▌     | 101859/225000 [211:11:50<726:56:15, 21.25s/it]
 45%|████▌     | 101860/225000 [211:12:06<674:21:47, 19.72s/it]
 45%|████▌     | 101861/225000 [211:12:18<594:20:49, 17.38s/it]
 45%|████▌     | 101862/225000 [211:12:38<618:41:54, 18.09s/it]
 45%|████▌     | 101863/225000 [211:12:59<653:54:33, 19.12s/it]
 45%|████▌     | 101864/225000 [211:13:14<606:38:17, 17.74s/it]
 45%|████▌     | 101865/225000 [211:13:25<544:44:25, 15.93s/it]
 45%|████▌     | 101866/225000 [211:13:53<665:49:21, 19.47s/it]
 45%|████▌     | 101867/225000 [211:14:11<648:11:22, 18.95s/it]
 45%|████▌     | 101868/225000 [211:14:29<637:06:08, 18.63s/it]
 45%|████▌     | 101869/225000 [211:14:49<654:14:53, 19.13s/it]
 45%|████▌     | 101870/225000 [211:15:19<763:14:13, 22.32s/it]
 45%|████▌     | 101871/225000 [211:15:36<709:34:07, 20.75s/it]
 45%|████▌     | 101872/225000 [211:15:43<574:49:52, 16.81s/it]
 45%|████▌     | 101873/225000 [211:16:04<612:02:19, 17.89s/it]
 45%|████▌     | 101874/225000 [211:16:22<619:56:13, 18.13s/it]
 45%|████▌     | 101875/225000 [211:16:41<626:36:33, 18.32s/it]
 45%|████▌     | 101876/225000 [211:17:05<680:45:12, 19.90s/it]
 45%|████▌     | 101877/225000 [211:17:23<666:01:15, 19.47s/it]
 45%|████▌     | 101878/225000 [211:17:45<685:31:04, 20.04s/it]
 45%|████▌     | 101879/225000 [211:18:08<715:08:10, 20.91s/it]
 45%|████▌     | 101880/225000 [211:18:33<766:01:36, 22.40s/it]
 45%|████▌     | 101881/225000 [211:18:50<702:41:56, 20.55s/it]
 45%|████▌     | 101882/225000 [211:19:11<711:45:21, 20.81s/it]
 45%|████▌     | 101883/225000 [211:19:31<705:09:23, 20.62s/it]
 45%|████▌     | 101884/225000 [211:20:01<796:28:05, 23.29s/it]
 45%|████▌     | 101885/225000 [211:20:22<779:37:09, 22.80s/it]
 45%|████▌     | 101886/225000 [211:20:43<755:27:00, 22.09s/it]
 45%|████▌     | 101887/225000 [211:20:53<635:58:39, 18.60s/it]
 45%|████▌     | 101888/225000 [211:21:10<613:17:06, 17.93s/it]
 45%|████▌     | 101889/225000 [211:21:33<668:48:45, 19.56s/it]
 45%|████▌     | 101890/225000 [211:21:58<720:42:12, 21.07s/it]
 45%|████▌     | 101891/225000 [211:22:20<736:52:14, 21.55s/it]
 45%|████▌     | 101892/225000 [211:22:39<711:41:25, 20.81s/it]
 45%|████▌     | 101893/225000 [211:22:53<632:57:17, 18.51s/it]
 45%|████▌     | 101894/225000 [211:23:12<638:28:31, 18.67s/it]
 45%|████▌     | 101895/225000 [211:23:35<691:41:45, 20.23s/it]
 45%|████▌     | 101896/225000 [211:23:56<696:48:43, 20.38s/it]
 45%|████▌     | 101897/225000 [211:24:12<652:35:25, 19.08s/it]
 45%|████▌     | 101898/225000 [211:24:35<692:21:13, 20.25s/it]
 45%|████▌     | 101899/225000 [211:24:50<636:32:12, 18.62s/it]
 45%|████▌     | 101900/225000 [211:25:09<638:16:14, 18.67s/it]
                                                               
{'loss': '0.8834', 'grad_norm': '1.431', 'learning_rate': '3.067e-05', 'epoch': '0.4529', 'num_input_tokens_seen': 354029944, 'train_runtime': '7.611e+05', 'train_tokens_per_second': '465.1'}

 45%|████▌     | 101900/225000 [211:25:09<638:16:14, 18.67s/it]
 45%|████▌     | 101901/225000 [211:25:30<664:21:30, 19.43s/it]
 45%|████▌     | 101902/225000 [211:25:47<634:16:55, 18.55s/it]
 45%|████▌     | 101903/225000 [211:26:07<653:45:59, 19.12s/it]
 45%|████▌     | 101904/225000 [211:26:29<687:35:05, 20.11s/it]
 45%|████▌     | 101905/225000 [211:26:42<606:00:33, 17.72s/it]
 45%|████▌     | 101906/225000 [211:27:07<683:14:00, 19.98s/it]
 45%|████▌     | 101907/225000 [211:27:19<600:19:19, 17.56s/it]
 45%|████▌     | 101908/225000 [211:27:43<670:49:21, 19.62s/it]
 45%|████▌     | 101909/225000 [211:28:09<736:53:08, 21.55s/it]
 45%|████▌     | 101910/225000 [211:28:24<666:41:44, 19.50s/it]
 45%|████▌     | 101911/225000 [211:28:49<728:21:51, 21.30s/it]
 45%|████▌     | 101912/225000 [211:28:59<609:55:38, 17.84s/it]
 45%|████▌     | 101913/225000 [211:29:16<603:57:07, 17.66s/it]
 45%|████▌     | 101914/225000 [211:29:40<663:18:54, 19.40s/it]
 45%|████▌     | 101915/225000 [211:29:52<593:08:54, 17.35s/it]
 45%|████▌     | 101916/225000 [211:30:17<666:56:15, 19.51s/it]
 45%|████▌     | 101917/225000 [211:30:37<673:06:43, 19.69s/it]
 45%|████▌     | 101918/225000 [211:30:58<689:13:23, 20.16s/it]
 45%|████▌     | 101919/225000 [211:31:12<620:01:10, 18.13s/it]
 45%|████▌     | 101920/225000 [211:31:39<708:19:38, 20.72s/it]
 45%|████▌     | 101921/225000 [211:31:57<690:07:29, 20.19s/it]
 45%|████▌     | 101922/225000 [211:32:26<780:40:12, 22.83s/it]
 45%|████▌     | 101923/225000 [211:32:48<767:23:56, 22.45s/it]
 45%|████▌     | 101924/225000 [211:33:02<678:55:06, 19.86s/it]
 45%|████▌     | 101925/225000 [211:33:23<688:43:14, 20.15s/it]
 45%|████▌     | 101926/225000 [211:33:47<728:22:04, 21.31s/it]
 45%|████▌     | 101927/225000 [211:34:00<641:46:50, 18.77s/it]
 45%|████▌     | 101928/225000 [211:34:18<638:25:53, 18.67s/it]
 45%|████▌     | 101929/225000 [211:34:42<690:56:22, 20.21s/it]
 45%|████▌     | 101930/225000 [211:34:58<652:25:52, 19.08s/it]
 45%|████▌     | 101931/225000 [211:35:23<706:24:14, 20.66s/it]
 45%|████▌     | 101932/225000 [211:35:37<641:24:17, 18.76s/it]
 45%|████▌     | 101933/225000 [211:36:10<791:13:47, 23.15s/it]
 45%|████▌     | 101934/225000 [211:36:28<736:59:38, 21.56s/it]
 45%|████▌     | 101935/225000 [211:36:44<680:21:55, 19.90s/it]
 45%|████▌     | 101936/225000 [211:37:04<683:27:27, 19.99s/it]
 45%|████▌     | 101937/225000 [211:37:24<676:11:40, 19.78s/it]
 45%|████▌     | 101938/225000 [211:37:43<670:52:54, 19.63s/it]
 45%|████▌     | 101939/225000 [211:38:03<671:18:04, 19.64s/it]
 45%|████▌     | 101940/225000 [211:38:22<671:52:17, 19.65s/it]
 45%|████▌     | 101941/225000 [211:38:37<618:48:48, 18.10s/it]
 45%|████▌     | 101942/225000 [211:38:53<603:31:03, 17.66s/it]
 45%|████▌     | 101943/225000 [211:39:19<682:47:17, 19.97s/it]
 45%|████▌     | 101944/225000 [211:39:35<643:53:13, 18.84s/it]
 45%|████▌     | 101945/225000 [211:39:57<671:52:09, 19.66s/it]
 45%|████▌     | 101946/225000 [211:40:16<666:35:24, 19.50s/it]
 45%|████▌     | 101947/225000 [211:40:33<641:22:56, 18.76s/it]
 45%|████▌     | 101948/225000 [211:40:50<631:11:42, 18.47s/it]
 45%|████▌     | 101949/225000 [211:41:05<586:18:29, 17.15s/it]
 45%|████▌     | 101950/225000 [211:41:19<556:44:01, 16.29s/it]
                                                               
{'loss': '0.891', 'grad_norm': '4.321', 'learning_rate': '3.065e-05', 'epoch': '0.4531', 'num_input_tokens_seen': 354530680, 'train_runtime': '7.621e+05', 'train_tokens_per_second': '465.2'}

 45%|████▌     | 101950/225000 [211:41:19<556:44:01, 16.29s/it]
 45%|████▌     | 101951/225000 [211:41:36<569:21:39, 16.66s/it]
 45%|████▌     | 101952/225000 [211:41:57<613:00:59, 17.93s/it]
 45%|████▌     | 101953/225000 [211:42:20<658:35:23, 19.27s/it]
 45%|████▌     | 101954/225000 [211:42:41<680:46:51, 19.92s/it]
 45%|████▌     | 101955/225000 [211:43:06<736:47:03, 21.56s/it]
 45%|████▌     | 101956/225000 [211:43:25<709:12:36, 20.75s/it]
 45%|████▌     | 101957/225000 [211:43:42<664:46:10, 19.45s/it]
 45%|████▌     | 101958/225000 [211:43:59<637:19:07, 18.65s/it]
 45%|████▌     | 101959/225000 [211:44:14<604:48:07, 17.70s/it]
 45%|████▌     | 101960/225000 [211:44:34<624:23:38, 18.27s/it]
 45%|████▌     | 101961/225000 [211:45:01<721:19:09, 21.11s/it]
 45%|████▌     | 101962/225000 [211:45:14<638:18:26, 18.68s/it]
 45%|████▌     | 101963/225000 [211:45:37<684:10:43, 20.02s/it]
 45%|████▌     | 101964/225000 [211:46:03<744:13:28, 21.78s/it]
 45%|████▌     | 101965/225000 [211:46:30<790:54:19, 23.14s/it]
 45%|████▌     | 101966/225000 [211:46:51<776:12:05, 22.71s/it]
 45%|████▌     | 101967/225000 [211:47:10<738:38:25, 21.61s/it]
 45%|████▌     | 101968/225000 [211:47:27<691:53:28, 20.25s/it]
 45%|████▌     | 101969/225000 [211:47:50<719:30:59, 21.05s/it]
 45%|████▌     | 101970/225000 [211:48:13<739:04:37, 21.63s/it]
 45%|████▌     | 101971/225000 [211:48:32<708:42:31, 20.74s/it]
 45%|████▌     | 101972/225000 [211:48:59<767:39:22, 22.46s/it]
 45%|████▌     | 101973/225000 [211:49:16<716:33:27, 20.97s/it]
 45%|████▌     | 101974/225000 [211:49:34<686:19:08, 20.08s/it]
 45%|████▌     | 101975/225000 [211:49:47<612:21:37, 17.92s/it]
 45%|████▌     | 101976/225000 [211:50:15<719:17:33, 21.05s/it]
 45%|████▌     | 101977/225000 [211:50:35<701:06:57, 20.52s/it]
 45%|████▌     | 101978/225000 [211:50:52<666:38:35, 19.51s/it]
 45%|████▌     | 101979/225000 [211:51:18<732:00:36, 21.42s/it]
 45%|████▌     | 101980/225000 [211:51:42<762:53:47, 22.33s/it]
 45%|████▌     | 101981/225000 [211:51:57<686:02:28, 20.08s/it]
 45%|████▌     | 101982/225000 [211:52:24<760:13:00, 22.25s/it]
 45%|████▌     | 101983/225000 [211:52:38<677:35:29, 19.83s/it]
 45%|████▌     | 101984/225000 [211:52:47<563:40:15, 16.50s/it]
 45%|████▌     | 101985/225000 [211:52:59<520:37:43, 15.24s/it]
 45%|████▌     | 101986/225000 [211:53:27<650:32:24, 19.04s/it]
 45%|████▌     | 101987/225000 [211:53:47<652:52:58, 19.11s/it]
 45%|████▌     | 101988/225000 [211:54:08<679:52:59, 19.90s/it]
 45%|████▌     | 101989/225000 [211:54:33<727:40:06, 21.30s/it]
 45%|████▌     | 101990/225000 [211:54:48<665:26:11, 19.47s/it]
 45%|████▌     | 101991/225000 [211:55:17<759:09:28, 22.22s/it]
 45%|████▌     | 101992/225000 [211:55:33<703:09:47, 20.58s/it]
 45%|████▌     | 101993/225000 [211:56:02<780:30:58, 22.84s/it]
 45%|████▌     | 101994/225000 [211:56:11<646:24:39, 18.92s/it]
 45%|████▌     | 101995/225000 [211:56:23<572:44:27, 16.76s/it]
 45%|████▌     | 101996/225000 [211:56:36<530:05:34, 15.51s/it]
 45%|████▌     | 101997/225000 [211:56:45<466:55:52, 13.67s/it]
 45%|████▌     | 101998/225000 [211:57:11<597:47:00, 17.50s/it]
 45%|████▌     | 101999/225000 [211:57:32<624:31:34, 18.28s/it]
 45%|████▌     | 102000/225000 [211:57:46<588:26:20, 17.22s/it]
                                                               
{'loss': '0.8818', 'grad_norm': '1.448', 'learning_rate': '3.063e-05', 'epoch': '0.4533', 'num_input_tokens_seen': 355016600, 'train_runtime': '7.631e+05', 'train_tokens_per_second': '465.2'}

 45%|████▌     | 102000/225000 [211:57:46<588:26:20, 17.22s/it]
 45%|████▌     | 102001/225000 [211:58:03<584:51:48, 17.12s/it]
 45%|████▌     | 102002/225000 [211:58:21<591:53:48, 17.32s/it]
 45%|████▌     | 102003/225000 [211:58:41<619:45:39, 18.14s/it]
 45%|████▌     | 102004/225000 [211:59:04<667:01:36, 19.52s/it]
 45%|████▌     | 102005/225000 [211:59:27<701:23:22, 20.53s/it]
 45%|████▌     | 102006/225000 [211:59:54<767:56:34, 22.48s/it]
 45%|████▌     | 102007/225000 [212:00:10<703:28:12, 20.59s/it]
 45%|████▌     | 102008/225000 [212:00:27<665:49:47, 19.49s/it]
 45%|████▌     | 102009/225000 [212:00:43<634:11:59, 18.56s/it]
 45%|████▌     | 102010/225000 [212:00:56<574:14:16, 16.81s/it]
 45%|████▌     | 102011/225000 [212:01:15<596:17:39, 17.45s/it]
 45%|████▌     | 102012/225000 [212:01:40<678:32:20, 19.86s/it]
 45%|████▌     | 102013/225000 [212:02:08<755:06:29, 22.10s/it]
 45%|████▌     | 102014/225000 [212:02:35<806:15:28, 23.60s/it]
 45%|████▌     | 102015/225000 [212:02:53<746:38:37, 21.86s/it]
 45%|████▌     | 102016/225000 [212:03:07<674:25:00, 19.74s/it]
 45%|████▌     | 102017/225000 [212:03:27<670:36:30, 19.63s/it]
 45%|████▌     | 102018/225000 [212:03:41<610:53:42, 17.88s/it]
 45%|████▌     | 102019/225000 [212:03:57<593:11:40, 17.36s/it]
 45%|████▌     | 102020/225000 [212:04:22<669:44:14, 19.61s/it]
 45%|████▌     | 102021/225000 [212:04:38<632:59:25, 18.53s/it]
 45%|████▌     | 102022/225000 [212:04:52<591:07:34, 17.30s/it]
 45%|████▌     | 102023/225000 [212:05:05<546:17:44, 15.99s/it]
 45%|████▌     | 102024/225000 [212:05:28<617:00:09, 18.06s/it]
 45%|████▌     | 102025/225000 [212:05:48<635:38:58, 18.61s/it]
 45%|████▌     | 102026/225000 [212:06:06<628:20:57, 18.39s/it]
 45%|████▌     | 102027/225000 [212:06:35<740:20:00, 21.67s/it]
 45%|████▌     | 102028/225000 [212:07:00<773:06:49, 22.63s/it]
 45%|████▌     | 102029/225000 [212:07:23<780:42:45, 22.86s/it]
 45%|████▌     | 102030/225000 [212:07:50<822:58:24, 24.09s/it]
 45%|████▌     | 102031/225000 [212:08:10<777:15:36, 22.75s/it]
 45%|████▌     | 102032/225000 [212:08:22<674:11:02, 19.74s/it]
 45%|████▌     | 102033/225000 [212:08:50<756:51:02, 22.16s/it]
 45%|████▌     | 102034/225000 [212:09:14<771:40:38, 22.59s/it]
 45%|████▌     | 102035/225000 [212:09:35<751:46:55, 22.01s/it]
 45%|████▌     | 102036/225000 [212:09:50<686:48:26, 20.11s/it]
 45%|████▌     | 102037/225000 [212:10:10<680:22:23, 19.92s/it]
 45%|████▌     | 102038/225000 [212:10:27<652:18:30, 19.10s/it]
 45%|████▌     | 102039/225000 [212:10:41<605:00:33, 17.71s/it]
 45%|████▌     | 102040/225000 [212:11:05<670:21:12, 19.63s/it]
 45%|████▌     | 102041/225000 [212:11:23<648:36:05, 18.99s/it]
 45%|████▌     | 102042/225000 [212:11:42<644:34:47, 18.87s/it]
 45%|████▌     | 102043/225000 [212:12:02<658:23:40, 19.28s/it]
 45%|████▌     | 102044/225000 [212:12:20<643:41:23, 18.85s/it]
 45%|████▌     | 102045/225000 [212:12:47<730:54:23, 21.40s/it]
 45%|████▌     | 102046/225000 [212:13:07<716:58:21, 20.99s/it]
 45%|████▌     | 102047/225000 [212:13:20<637:49:29, 18.68s/it]
 45%|████▌     | 102048/225000 [212:13:40<646:45:23, 18.94s/it]
 45%|████▌     | 102049/225000 [212:14:04<703:18:27, 20.59s/it]
 45%|████▌     | 102050/225000 [212:14:24<690:56:29, 20.23s/it]
                                                               
{'loss': '0.9138', 'grad_norm': '1.266', 'learning_rate': '3.061e-05', 'epoch': '0.4536', 'num_input_tokens_seen': 355504280, 'train_runtime': '7.641e+05', 'train_tokens_per_second': '465.3'}

 45%|████▌     | 102050/225000 [212:14:24<690:56:29, 20.23s/it]
 45%|████▌     | 102051/225000 [212:14:47<723:06:06, 21.17s/it]
 45%|████▌     | 102052/225000 [212:15:17<813:05:18, 23.81s/it]
 45%|████▌     | 102053/225000 [212:15:37<777:59:04, 22.78s/it]
 45%|████▌     | 102054/225000 [212:15:58<756:10:27, 22.14s/it]
 45%|████▌     | 102055/225000 [212:16:17<726:52:15, 21.28s/it]
 45%|████▌     | 102056/225000 [212:16:43<776:27:13, 22.74s/it]
 45%|████▌     | 102057/225000 [212:17:05<768:01:03, 22.49s/it]
 45%|████▌     | 102058/225000 [212:17:25<735:16:19, 21.53s/it]
 45%|████▌     | 102059/225000 [212:17:37<645:32:57, 18.90s/it]
 45%|████▌     | 102060/225000 [212:17:58<658:56:38, 19.30s/it]
 45%|████▌     | 102061/225000 [212:18:13<622:49:59, 18.24s/it]
 45%|████▌     | 102062/225000 [212:18:33<635:07:52, 18.60s/it]
 45%|████▌     | 102063/225000 [212:18:52<644:40:53, 18.88s/it]
 45%|████▌     | 102064/225000 [212:19:10<627:42:23, 18.38s/it]
 45%|████▌     | 102065/225000 [212:19:29<640:29:01, 18.76s/it]
 45%|████▌     | 102066/225000 [212:19:49<654:43:41, 19.17s/it]
 45%|████▌     | 102067/225000 [212:20:14<711:10:28, 20.83s/it]
 45%|████▌     | 102068/225000 [212:20:29<646:24:57, 18.93s/it]
 45%|████▌     | 102069/225000 [212:20:38<547:57:40, 16.05s/it]
 45%|████▌     | 102070/225000 [212:21:03<638:46:55, 18.71s/it]
 45%|████▌     | 102071/225000 [212:21:23<652:14:42, 19.10s/it]
 45%|████▌     | 102072/225000 [212:21:46<692:32:55, 20.28s/it]
 45%|████▌     | 102073/225000 [212:22:07<697:46:27, 20.43s/it]
 45%|████▌     | 102074/225000 [212:22:22<648:29:15, 18.99s/it]
 45%|████▌     | 102075/225000 [212:22:38<612:17:29, 17.93s/it]
 45%|████▌     | 102076/225000 [212:23:07<732:31:06, 21.45s/it]
 45%|████▌     | 102077/225000 [212:23:19<631:57:51, 18.51s/it]
 45%|████▌     | 102078/225000 [212:23:42<675:43:41, 19.79s/it]
 45%|████▌     | 102079/225000 [212:24:05<715:32:25, 20.96s/it]
 45%|████▌     | 102080/225000 [212:24:30<749:30:45, 21.95s/it]
 45%|████▌     | 102081/225000 [212:24:57<803:09:41, 23.52s/it]
 45%|████▌     | 102082/225000 [212:25:06<659:46:47, 19.32s/it]
 45%|████▌     | 102083/225000 [212:25:18<580:34:07, 17.00s/it]
 45%|████▌     | 102084/225000 [212:25:37<601:05:10, 17.60s/it]
 45%|████▌     | 102085/225000 [212:25:52<578:13:50, 16.94s/it]
 45%|████▌     | 102086/225000 [212:26:06<545:34:18, 15.98s/it]
 45%|████▌     | 102087/225000 [212:26:23<558:30:08, 16.36s/it]
 45%|████▌     | 102088/225000 [212:26:40<559:04:39, 16.37s/it]
 45%|████▌     | 102089/225000 [212:27:02<613:18:36, 17.96s/it]
 45%|████▌     | 102090/225000 [212:27:24<655:35:24, 19.20s/it]
 45%|████▌     | 102091/225000 [212:27:40<631:30:44, 18.50s/it]
 45%|████▌     | 102092/225000 [212:28:03<674:29:28, 19.76s/it]
 45%|████▌     | 102093/225000 [212:28:13<576:16:16, 16.88s/it]
 45%|████▌     | 102094/225000 [212:28:35<629:45:25, 18.45s/it]
 45%|████▌     | 102095/225000 [212:28:52<613:14:38, 17.96s/it]
 45%|████▌     | 102096/225000 [212:29:19<703:06:53, 20.60s/it]
 45%|████▌     | 102097/225000 [212:29:46<765:23:23, 22.42s/it]
 45%|████▌     | 102098/225000 [212:30:01<687:56:19, 20.15s/it]
 45%|████▌     | 102099/225000 [212:30:20<684:35:14, 20.05s/it]
 45%|████▌     | 102100/225000 [212:30:43<713:25:19, 20.90s/it]
                                                               
{'loss': '0.895', 'grad_norm': '1.038', 'learning_rate': '3.06e-05', 'epoch': '0.4538', 'num_input_tokens_seen': 355984216, 'train_runtime': '7.65e+05', 'train_tokens_per_second': '465.3'}

 45%|████▌     | 102100/225000 [212:30:43<713:25:19, 20.90s/it]
 45%|████▌     | 102101/225000 [212:30:59<658:21:50, 19.29s/it]
 45%|████▌     | 102102/225000 [212:31:17<648:10:49, 18.99s/it]
 45%|████▌     | 102103/225000 [212:31:24<520:24:01, 15.24s/it]
 45%|████▌     | 102104/225000 [212:31:48<619:34:26, 18.15s/it]
 45%|████▌     | 102105/225000 [212:32:11<669:03:29, 19.60s/it]
 45%|████▌     | 102106/225000 [212:32:33<685:37:48, 20.08s/it]
 45%|████▌     | 102107/225000 [212:32:49<645:09:23, 18.90s/it]
 45%|████▌     | 102108/225000 [212:33:06<632:20:13, 18.52s/it]
 45%|████▌     | 102109/225000 [212:33:21<595:04:06, 17.43s/it]
 45%|████▌     | 102110/225000 [212:33:40<604:15:51, 17.70s/it]
 45%|████▌     | 102111/225000 [212:34:05<680:50:02, 19.94s/it]
 45%|████▌     | 102112/225000 [212:34:24<676:46:04, 19.83s/it]
 45%|████▌     | 102113/225000 [212:34:45<685:26:23, 20.08s/it]
 45%|████▌     | 102114/225000 [212:34:58<610:34:41, 17.89s/it]
 45%|████▌     | 102115/225000 [212:35:16<613:51:48, 17.98s/it]
 45%|████▌     | 102116/225000 [212:35:32<594:55:56, 17.43s/it]
 45%|████▌     | 102117/225000 [212:35:50<596:02:13, 17.46s/it]
 45%|████▌     | 102118/225000 [212:36:09<619:19:04, 18.14s/it]
 45%|████▌     | 102119/225000 [212:36:28<622:54:55, 18.25s/it]
 45%|████▌     | 102120/225000 [212:36:46<618:36:53, 18.12s/it]
 45%|████▌     | 102121/225000 [212:37:08<656:01:16, 19.22s/it]
 45%|████▌     | 102122/225000 [212:37:27<662:45:46, 19.42s/it]
 45%|████▌     | 102123/225000 [212:37:48<670:00:27, 19.63s/it]
 45%|████▌     | 102124/225000 [212:38:04<637:37:42, 18.68s/it]
 45%|████▌     | 102125/225000 [212:38:27<681:19:03, 19.96s/it]
 45%|████▌     | 102126/225000 [212:38:51<727:41:52, 21.32s/it]
 45%|████▌     | 102127/225000 [212:39:14<738:45:43, 21.64s/it]
 45%|████▌     | 102128/225000 [212:39:29<671:58:16, 19.69s/it]
 45%|████▌     | 102129/225000 [212:39:46<647:17:06, 18.96s/it]
 45%|████▌     | 102130/225000 [212:40:12<721:09:44, 21.13s/it]
 45%|████▌     | 102131/225000 [212:40:41<795:50:16, 23.32s/it]
 45%|████▌     | 102132/225000 [212:41:02<777:47:03, 22.79s/it]
 45%|████▌     | 102133/225000 [212:41:26<787:36:38, 23.08s/it]
 45%|████▌     | 102134/225000 [212:41:42<712:12:30, 20.87s/it]
 45%|████▌     | 102135/225000 [212:42:00<686:08:27, 20.10s/it]
 45%|████▌     | 102136/225000 [212:42:16<637:04:20, 18.67s/it]
 45%|████▌     | 102137/225000 [212:42:36<652:57:07, 19.13s/it]
 45%|████▌     | 102138/225000 [212:42:48<579:00:50, 16.97s/it]
 45%|████▌     | 102139/225000 [212:43:04<571:22:08, 16.74s/it]
 45%|████▌     | 102140/225000 [212:43:21<571:24:13, 16.74s/it]
 45%|████▌     | 102141/225000 [212:43:35<546:12:40, 16.01s/it]
 45%|████▌     | 102142/225000 [212:43:54<573:20:06, 16.80s/it]
 45%|████▌     | 102143/225000 [212:44:06<523:48:09, 15.35s/it]
 45%|████▌     | 102144/225000 [212:44:19<505:31:43, 14.81s/it]
 45%|████▌     | 102145/225000 [212:44:29<455:27:36, 13.35s/it]
 45%|████▌     | 102146/225000 [212:44:55<587:42:42, 17.22s/it]
 45%|████▌     | 102147/225000 [212:45:17<631:57:50, 18.52s/it]
 45%|████▌     | 102148/225000 [212:45:29<569:31:41, 16.69s/it]
 45%|████▌     | 102149/225000 [212:45:56<673:07:21, 19.73s/it]
 45%|████▌     | 102150/225000 [212:46:19<709:09:55, 20.78s/it]
                                                               
{'loss': '0.9008', 'grad_norm': '1.162', 'learning_rate': '3.058e-05', 'epoch': '0.454', 'num_input_tokens_seen': 356446784, 'train_runtime': '7.66e+05', 'train_tokens_per_second': '465.3'}

 45%|████▌     | 102150/225000 [212:46:19<709:09:55, 20.78s/it]
 45%|████▌     | 102151/225000 [212:46:41<719:27:40, 21.08s/it]
 45%|████▌     | 102152/225000 [212:47:04<735:38:32, 21.56s/it]
 45%|████▌     | 102153/225000 [212:47:20<682:51:53, 20.01s/it]
 45%|████▌     | 102154/225000 [212:47:46<746:35:13, 21.88s/it]
 45%|████▌     | 102155/225000 [212:47:59<656:30:54, 19.24s/it]
 45%|████▌     | 102156/225000 [212:48:17<633:56:53, 18.58s/it]
 45%|████▌     | 102157/225000 [212:48:38<661:36:05, 19.39s/it]
 45%|████▌     | 102158/225000 [212:48:59<677:26:11, 19.85s/it]
 45%|████▌     | 102159/225000 [212:49:24<728:23:19, 21.35s/it]
 45%|████▌     | 102160/225000 [212:49:46<739:14:11, 21.66s/it]
 45%|████▌     | 102161/225000 [212:50:05<712:01:45, 20.87s/it]
 45%|████▌     | 102162/225000 [212:50:27<722:17:11, 21.17s/it]
 45%|████▌     | 102163/225000 [212:50:51<757:24:28, 22.20s/it]
 45%|████▌     | 102164/225000 [212:51:24<863:10:16, 25.30s/it]
 45%|████▌     | 102165/225000 [212:51:41<781:38:47, 22.91s/it]
 45%|████▌     | 102166/225000 [212:51:59<731:09:12, 21.43s/it]
 45%|████▌     | 102167/225000 [212:52:27<794:56:19, 23.30s/it]
 45%|████▌     | 102168/225000 [212:52:37<662:45:09, 19.42s/it]
 45%|████▌     | 102169/225000 [212:52:57<666:07:28, 19.52s/it]
 45%|████▌     | 102170/225000 [212:53:12<616:15:40, 18.06s/it]
 45%|████▌     | 102171/225000 [212:53:42<736:15:00, 21.58s/it]
 45%|████▌     | 102172/225000 [212:54:02<726:06:58, 21.28s/it]
 45%|████▌     | 102173/225000 [212:54:22<714:48:29, 20.95s/it]
 45%|████▌     | 102174/225000 [212:54:45<736:12:32, 21.58s/it]
 45%|████▌     | 102175/225000 [212:55:02<686:58:14, 20.14s/it]
 45%|████▌     | 102176/225000 [212:55:15<610:23:09, 17.89s/it]
 45%|████▌     | 102177/225000 [212:55:35<636:34:47, 18.66s/it]
 45%|████▌     | 102178/225000 [212:56:03<732:57:01, 21.48s/it]
 45%|████▌     | 102179/225000 [212:56:23<713:35:21, 20.92s/it]
 45%|████▌     | 102180/225000 [212:56:42<698:51:53, 20.48s/it]
 45%|████▌     | 102181/225000 [212:56:59<660:02:59, 19.35s/it]
 45%|████▌     | 102182/225000 [212:57:12<590:37:42, 17.31s/it]
 45%|████▌     | 102183/225000 [212:57:32<619:16:18, 18.15s/it]
 45%|████▌     | 102184/225000 [212:57:39<510:58:46, 14.98s/it]
 45%|████▌     | 102185/225000 [212:57:59<556:44:12, 16.32s/it]
 45%|████▌     | 102186/225000 [212:58:09<499:26:16, 14.64s/it]
 45%|████▌     | 102187/225000 [212:58:30<556:05:18, 16.30s/it]
 45%|████▌     | 102188/225000 [212:58:47<571:27:47, 16.75s/it]
 45%|████▌     | 102189/225000 [212:59:05<575:45:30, 16.88s/it]
 45%|████▌     | 102190/225000 [212:59:23<593:04:13, 17.39s/it]
 45%|████▌     | 102191/225000 [212:59:33<516:46:10, 15.15s/it]
 45%|████▌     | 102192/225000 [212:59:57<606:35:41, 17.78s/it]
 45%|████▌     | 102193/225000 [213:00:16<623:54:25, 18.29s/it]
 45%|████▌     | 102194/225000 [213:00:34<615:00:30, 18.03s/it]
 45%|████▌     | 102195/225000 [213:00:58<674:47:48, 19.78s/it]
 45%|████▌     | 102196/225000 [213:01:13<629:22:37, 18.45s/it]
 45%|████▌     | 102197/225000 [213:01:27<579:57:32, 17.00s/it]
 45%|████▌     | 102198/225000 [213:01:34<482:12:34, 14.14s/it]
 45%|████▌     | 102199/225000 [213:01:58<584:16:39, 17.13s/it]
 45%|████▌     | 102200/225000 [213:02:17<601:38:49, 17.64s/it]
                                                               
{'loss': '0.9032', 'grad_norm': '1.051', 'learning_rate': '3.056e-05', 'epoch': '0.4542', 'num_input_tokens_seen': 356920976, 'train_runtime': '7.669e+05', 'train_tokens_per_second': '465.4'}

 45%|████▌     | 102200/225000 [213:02:17<601:38:49, 17.64s/it]
 45%|████▌     | 102201/225000 [213:02:38<631:41:30, 18.52s/it]
 45%|████▌     | 102202/225000 [213:02:56<631:45:12, 18.52s/it]
 45%|████▌     | 102203/225000 [213:03:11<593:02:42, 17.39s/it]
 45%|████▌     | 102204/225000 [213:03:37<681:57:26, 19.99s/it]
 45%|████▌     | 102205/225000 [213:03:56<669:33:16, 19.63s/it]
 45%|████▌     | 102206/225000 [213:04:17<688:19:39, 20.18s/it]
 45%|████▌     | 102207/225000 [213:04:36<677:35:17, 19.87s/it]
 45%|████▌     | 102208/225000 [213:04:59<707:35:29, 20.75s/it]
 45%|████▌     | 102209/225000 [213:05:17<678:14:07, 19.88s/it]
 45%|████▌     | 102210/225000 [213:05:33<642:17:19, 18.83s/it]
 45%|████▌     | 102211/225000 [213:05:53<649:16:41, 19.04s/it]
 45%|████▌     | 102212/225000 [213:06:06<592:16:33, 17.36s/it]
 45%|████▌     | 102213/225000 [213:06:27<620:35:59, 18.20s/it]
 45%|████▌     | 102214/225000 [213:06:42<591:08:50, 17.33s/it]
 45%|████▌     | 102215/225000 [213:07:09<693:04:06, 20.32s/it]
 45%|████▌     | 102216/225000 [213:07:22<619:29:34, 18.16s/it]
 45%|████▌     | 102217/225000 [213:07:40<609:58:57, 17.88s/it]
 45%|████▌     | 102218/225000 [213:07:54<571:44:08, 16.76s/it]
 45%|████▌     | 102219/225000 [213:08:11<578:10:23, 16.95s/it]
 45%|████▌     | 102220/225000 [213:08:29<584:59:36, 17.15s/it]
 45%|████▌     | 102221/225000 [213:08:50<623:52:01, 18.29s/it]
 45%|████▌     | 102222/225000 [213:09:01<552:33:06, 16.20s/it]
 45%|████▌     | 102223/225000 [213:09:21<590:12:22, 17.31s/it]
 45%|████▌     | 102224/225000 [213:09:44<648:11:23, 19.01s/it]
 45%|████▌     | 102225/225000 [213:10:07<688:16:34, 20.18s/it]
 45%|████▌     | 102226/225000 [213:10:15<564:19:35, 16.55s/it]
 45%|████▌     | 102227/225000 [213:10:36<609:25:50, 17.87s/it]
 45%|████▌     | 102228/225000 [213:10:49<560:50:38, 16.45s/it]
 45%|████▌     | 102229/225000 [213:11:17<684:24:06, 20.07s/it]
 45%|████▌     | 102230/225000 [213:11:40<704:57:11, 20.67s/it]
 45%|████▌     | 102231/225000 [213:11:54<643:22:22, 18.87s/it]
 45%|████▌     | 102232/225000 [213:12:19<699:23:49, 20.51s/it]
 45%|████▌     | 102233/225000 [213:12:34<646:59:26, 18.97s/it]
 45%|████▌     | 102234/225000 [213:12:57<685:35:08, 20.10s/it]
 45%|████▌     | 102235/225000 [213:13:12<642:01:07, 18.83s/it]
 45%|████▌     | 102236/225000 [213:13:28<604:50:22, 17.74s/it]
 45%|████▌     | 102237/225000 [213:13:38<529:50:47, 15.54s/it]
 45%|████▌     | 102238/225000 [213:13:52<515:34:01, 15.12s/it]
 45%|████▌     | 102239/225000 [213:14:09<530:31:51, 15.56s/it]
 45%|████▌     | 102240/225000 [213:14:24<524:52:32, 15.39s/it]
 45%|████▌     | 102241/225000 [213:14:48<613:10:39, 17.98s/it]
 45%|████▌     | 102242/225000 [213:15:03<581:08:41, 17.04s/it]
 45%|████▌     | 102243/225000 [213:15:21<596:12:59, 17.48s/it]
 45%|████▌     | 102244/225000 [213:15:36<566:56:51, 16.63s/it]
 45%|████▌     | 102245/225000 [213:16:02<668:46:39, 19.61s/it]
 45%|████▌     | 102246/225000 [213:16:20<650:40:57, 19.08s/it]
 45%|████▌     | 102247/225000 [213:16:38<641:36:37, 18.82s/it]
 45%|████▌     | 102248/225000 [213:16:57<642:05:16, 18.83s/it]
 45%|████▌     | 102249/225000 [213:17:15<628:59:57, 18.45s/it]
 45%|████▌     | 102250/225000 [213:17:38<676:21:30, 19.84s/it]
                                                               
{'loss': '0.8676', 'grad_norm': '0.9512', 'learning_rate': '3.054e-05', 'epoch': '0.4544', 'num_input_tokens_seen': 357363848, 'train_runtime': '7.679e+05', 'train_tokens_per_second': '465.4'}

 45%|████▌     | 102250/225000 [213:17:38<676:21:30, 19.84s/it]
 45%|████▌     | 102251/225000 [213:18:01<707:00:25, 20.74s/it]
 45%|████▌     | 102252/225000 [213:18:18<670:15:44, 19.66s/it]
 45%|████▌     | 102253/225000 [213:18:34<630:49:38, 18.50s/it]
 45%|████▌     | 102254/225000 [213:18:54<646:41:19, 18.97s/it]
 45%|████▌     | 102255/225000 [213:19:15<665:04:53, 19.51s/it]
 45%|████▌     | 102256/225000 [213:19:33<649:36:18, 19.05s/it]
 45%|████▌     | 102257/225000 [213:19:53<663:01:24, 19.45s/it]
 45%|████▌     | 102258/225000 [213:20:27<815:03:36, 23.91s/it]
 45%|████▌     | 102259/225000 [213:20:39<693:48:31, 20.35s/it]
 45%|████▌     | 102260/225000 [213:21:02<722:44:46, 21.20s/it]
 45%|████▌     | 102261/225000 [213:21:23<712:34:52, 20.90s/it]
 45%|████▌     | 102262/225000 [213:21:43<706:05:43, 20.71s/it]
 45%|████▌     | 102263/225000 [213:22:05<724:49:30, 21.26s/it]
 45%|████▌     | 102264/225000 [213:22:21<666:56:17, 19.56s/it]
 45%|████▌     | 102265/225000 [213:22:43<691:10:49, 20.27s/it]
 45%|████▌     | 102266/225000 [213:23:08<735:30:00, 21.57s/it]
 45%|████▌     | 102267/225000 [213:23:23<670:04:06, 19.65s/it]
 45%|████▌     | 102268/225000 [213:23:45<698:12:04, 20.48s/it]
 45%|████▌     | 102269/225000 [213:24:00<644:13:17, 18.90s/it]
 45%|████▌     | 102270/225000 [213:24:09<538:28:07, 15.79s/it]
 45%|████▌     | 102271/225000 [213:24:37<660:38:16, 19.38s/it]
 45%|████▌     | 102272/225000 [213:24:54<643:44:25, 18.88s/it]
 45%|████▌     | 102273/225000 [213:25:09<596:05:19, 17.49s/it]
 45%|████▌     | 102274/225000 [213:25:32<657:05:54, 19.28s/it]
 45%|████▌     | 102275/225000 [213:25:47<607:46:30, 17.83s/it]
 45%|████▌     | 102276/225000 [213:26:05<612:22:16, 17.96s/it]
 45%|████▌     | 102277/225000 [213:26:17<555:52:26, 16.31s/it]
 45%|████▌     | 102278/225000 [213:26:36<578:38:21, 16.97s/it]
 45%|████▌     | 102279/225000 [213:26:52<568:11:45, 16.67s/it]
 45%|████▌     | 102280/225000 [213:27:04<526:29:59, 15.44s/it]
 45%|████▌     | 102281/225000 [213:27:26<594:29:33, 17.44s/it]
 45%|████▌     | 102282/225000 [213:27:44<591:57:07, 17.37s/it]
 45%|████▌     | 102283/225000 [213:27:56<541:30:27, 15.89s/it]
 45%|████▌     | 102284/225000 [213:28:09<515:58:53, 15.14s/it]
 45%|████▌     | 102285/225000 [213:28:31<586:36:24, 17.21s/it]
 45%|████▌     | 102286/225000 [213:28:52<619:17:30, 18.17s/it]
 45%|████▌     | 102287/225000 [213:29:09<610:12:49, 17.90s/it]
 45%|████▌     | 102288/225000 [213:29:32<660:11:28, 19.37s/it]
 45%|████▌     | 102289/225000 [213:29:50<648:53:20, 19.04s/it]
 45%|████▌     | 102290/225000 [213:30:07<625:21:21, 18.35s/it]
 45%|████▌     | 102291/225000 [213:30:28<650:08:35, 19.07s/it]
 45%|████▌     | 102292/225000 [213:30:51<696:51:15, 20.44s/it]
 45%|████▌     | 102293/225000 [213:31:07<642:36:05, 18.85s/it]
 45%|████▌     | 102294/225000 [213:31:25<636:45:22, 18.68s/it]
 45%|████▌     | 102295/225000 [213:31:39<587:56:03, 17.25s/it]
 45%|████▌     | 102296/225000 [213:32:03<656:21:16, 19.26s/it]
 45%|████▌     | 102297/225000 [213:32:22<656:21:18, 19.26s/it]
 45%|████▌     | 102298/225000 [213:32:46<705:02:22, 20.69s/it]
 45%|████▌     | 102299/225000 [213:32:59<632:11:39, 18.55s/it]
 45%|████▌     | 102300/225000 [213:33:07<522:42:52, 15.34s/it]
                                                               
{'loss': '0.8957', 'grad_norm': '1.464', 'learning_rate': '3.053e-05', 'epoch': '0.4547', 'num_input_tokens_seen': 357836280, 'train_runtime': '7.688e+05', 'train_tokens_per_second': '465.5'}

 45%|████▌     | 102300/225000 [213:33:07<522:42:52, 15.34s/it]
 45%|████▌     | 102301/225000 [213:33:22<517:48:48, 15.19s/it]
 45%|████▌     | 102302/225000 [213:33:36<500:22:33, 14.68s/it]
 45%|████▌     | 102303/225000 [213:33:54<533:19:58, 15.65s/it]
 45%|████▌     | 102304/225000 [213:34:13<570:47:13, 16.75s/it]
 45%|████▌     | 102305/225000 [213:34:33<606:56:22, 17.81s/it]
 45%|████▌     | 102306/225000 [213:34:50<592:58:50, 17.40s/it]
 45%|████▌     | 102307/225000 [213:35:14<667:47:10, 19.59s/it]
 45%|████▌     | 102308/225000 [213:35:33<653:10:49, 19.17s/it]
 45%|████▌     | 102309/225000 [213:35:56<698:21:28, 20.49s/it]
 45%|████▌     | 102310/225000 [213:36:23<768:40:44, 22.55s/it]
 45%|████▌     | 102311/225000 [213:36:40<705:49:45, 20.71s/it]
 45%|████▌     | 102312/225000 [213:37:00<704:48:37, 20.68s/it]
 45%|████▌     | 102313/225000 [213:37:20<692:46:23, 20.33s/it]
 45%|████▌     | 102314/225000 [213:37:37<661:28:18, 19.41s/it]
 45%|████▌     | 102315/225000 [213:38:01<709:30:21, 20.82s/it]
 45%|████▌     | 102316/225000 [213:38:07<558:53:04, 16.40s/it]
 45%|████▌     | 102317/225000 [213:38:23<553:09:41, 16.23s/it]
 45%|████▌     | 102318/225000 [213:38:46<617:54:13, 18.13s/it]
 45%|████▌     | 102319/225000 [213:38:55<530:36:06, 15.57s/it]
 45%|████▌     | 102320/225000 [213:39:18<601:12:02, 17.64s/it]
 45%|████▌     | 102321/225000 [213:39:32<566:38:31, 16.63s/it]
 45%|████▌     | 102322/225000 [213:39:51<590:19:18, 17.32s/it]
 45%|████▌     | 102323/225000 [213:40:04<549:18:24, 16.12s/it]
 45%|████▌     | 102324/225000 [213:40:25<596:37:50, 17.51s/it]
 45%|████▌     | 102325/225000 [213:40:35<517:38:47, 15.19s/it]
 45%|████▌     | 102326/225000 [213:40:55<570:59:45, 16.76s/it]
 45%|████▌     | 102327/225000 [213:41:17<624:08:46, 18.32s/it]
 45%|████▌     | 102328/225000 [213:41:39<661:33:41, 19.41s/it]
 45%|████▌     | 102329/225000 [213:41:56<637:00:34, 18.69s/it]
 45%|████▌     | 102330/225000 [213:42:19<681:43:01, 20.01s/it]
 45%|████▌     | 102331/225000 [213:42:32<604:42:57, 17.75s/it]
 45%|████▌     | 102332/225000 [213:42:49<597:25:22, 17.53s/it]
 45%|████▌     | 102333/225000 [213:43:06<590:59:00, 17.34s/it]
 45%|████▌     | 102334/225000 [213:43:27<626:57:38, 18.40s/it]
 45%|████▌     | 102335/225000 [213:43:40<572:44:42, 16.81s/it]
 45%|████▌     | 102336/225000 [213:43:49<496:39:44, 14.58s/it]
 45%|████▌     | 102337/225000 [213:44:13<587:57:55, 17.26s/it]
 45%|████▌     | 102338/225000 [213:44:29<576:50:47, 16.93s/it]
 45%|████▌     | 102339/225000 [213:44:40<513:51:46, 15.08s/it]
 45%|████▌     | 102340/225000 [213:44:49<458:03:59, 13.44s/it]
 45%|████▌     | 102341/225000 [213:45:10<537:10:02, 15.77s/it]
 45%|████▌     | 102342/225000 [213:45:32<596:40:21, 17.51s/it]
 45%|████▌     | 102343/225000 [213:45:50<600:25:45, 17.62s/it]
 45%|████▌     | 102344/225000 [213:46:05<573:31:45, 16.83s/it]
 45%|████▌     | 102345/225000 [213:46:24<597:34:55, 17.54s/it]
 45%|████▌     | 102346/225000 [213:46:47<648:23:32, 19.03s/it]
 45%|████▌     | 102347/225000 [213:47:04<635:25:40, 18.65s/it]
 45%|████▌     | 102348/225000 [213:47:21<611:18:03, 17.94s/it]
 45%|████▌     | 102349/225000 [213:47:39<614:05:56, 18.02s/it]
 45%|████▌     | 102350/225000 [213:47:50<547:17:22, 16.06s/it]
                                                               
{'loss': '0.9017', 'grad_norm': '1.321', 'learning_rate': '3.051e-05', 'epoch': '0.4549', 'num_input_tokens_seen': 358272112, 'train_runtime': '7.697e+05', 'train_tokens_per_second': '465.5'}

 45%|████▌     | 102350/225000 [213:47:50<547:17:22, 16.06s/it]
 45%|████▌     | 102351/225000 [213:47:59<473:37:12, 13.90s/it]
 45%|████▌     | 102352/225000 [213:48:20<546:06:33, 16.03s/it]
 45%|████▌     | 102353/225000 [213:48:34<522:10:27, 15.33s/it]
 45%|████▌     | 102354/225000 [213:48:52<555:50:19, 16.32s/it]
 45%|████▌     | 102355/225000 [213:49:05<517:27:41, 15.19s/it]
 45%|████▌     | 102356/225000 [213:49:20<512:59:14, 15.06s/it]
 45%|████▌     | 102357/225000 [213:49:40<564:49:07, 16.58s/it]
 45%|████▌     | 102358/225000 [213:49:46<462:43:59, 13.58s/it]
 45%|████▌     | 102359/225000 [213:50:00<464:59:48, 13.65s/it]
 45%|████▌     | 102360/225000 [213:50:20<528:40:26, 15.52s/it]
 45%|████▌     | 102361/225000 [213:50:40<574:00:46, 16.85s/it]
 45%|████▌     | 102362/225000 [213:50:49<492:34:19, 14.46s/it]
 45%|████▌     | 102363/225000 [213:51:17<626:25:26, 18.39s/it]
 45%|████▌     | 102364/225000 [213:51:34<618:19:34, 18.15s/it]
 45%|████▌     | 102365/225000 [213:51:48<578:34:15, 16.98s/it]
 45%|████▌     | 102366/225000 [213:52:09<610:13:07, 17.91s/it]
 45%|████▌     | 102367/225000 [213:52:19<532:59:23, 15.65s/it]
 45%|████▌     | 102368/225000 [213:52:35<537:48:27, 15.79s/it]
 45%|████▌     | 102369/225000 [213:52:51<540:28:41, 15.87s/it]
 45%|████▌     | 102370/225000 [213:53:03<504:19:43, 14.81s/it]
 45%|████▌     | 102371/225000 [213:53:28<604:41:22, 17.75s/it]
 45%|████▌     | 102372/225000 [213:53:41<559:34:47, 16.43s/it]
 45%|████▌     | 102373/225000 [213:54:04<622:05:02, 18.26s/it]
 45%|████▌     | 102374/225000 [213:54:17<565:30:06, 16.60s/it]
 46%|████▌     | 102375/225000 [213:54:38<615:16:29, 18.06s/it]
 46%|████▌     | 102376/225000 [213:54:59<645:45:39, 18.96s/it]
 46%|████▌     | 102377/225000 [213:55:10<566:50:27, 16.64s/it]
 46%|████▌     | 102378/225000 [213:55:33<626:43:34, 18.40s/it]
 46%|████▌     | 102379/225000 [213:55:53<645:43:23, 18.96s/it]
 46%|████▌     | 102380/225000 [213:56:06<587:30:47, 17.25s/it]
 46%|████▌     | 102381/225000 [213:56:21<564:55:52, 16.59s/it]
 46%|████▌     | 102382/225000 [213:56:28<458:25:34, 13.46s/it]
 46%|████▌     | 102383/225000 [213:56:45<495:16:28, 14.54s/it]
 46%|████▌     | 102384/225000 [213:57:05<550:11:54, 16.15s/it]
 46%|████▌     | 102385/225000 [213:57:21<556:47:37, 16.35s/it]
 46%|████▌     | 102386/225000 [213:57:38<563:03:16, 16.53s/it]
 46%|████▌     | 102387/225000 [213:57:55<565:49:28, 16.61s/it]
 46%|████▌     | 102388/225000 [213:58:08<532:08:38, 15.62s/it]
 46%|████▌     | 102389/225000 [213:58:35<639:45:42, 18.78s/it]
 46%|████▌     | 102390/225000 [213:58:51<618:24:00, 18.16s/it]
 46%|████▌     | 102391/225000 [213:59:04<561:10:42, 16.48s/it]
 46%|████▌     | 102392/225000 [213:59:17<530:00:49, 15.56s/it]
 46%|████▌     | 102393/225000 [213:59:33<532:46:29, 15.64s/it]
 46%|████▌     | 102394/225000 [213:59:57<618:35:12, 18.16s/it]
 46%|████▌     | 102395/225000 [214:00:17<634:25:04, 18.63s/it]
 46%|████▌     | 102396/225000 [214:00:40<675:45:59, 19.84s/it]
 46%|████▌     | 102397/225000 [214:00:55<634:16:21, 18.62s/it]
 46%|████▌     | 102398/225000 [214:01:11<601:15:25, 17.65s/it]
 46%|████▌     | 102399/225000 [214:01:40<720:08:17, 21.15s/it]
 46%|████▌     | 102400/225000 [214:01:58<686:28:23, 20.16s/it]
                                                               
{'loss': '0.9045', 'grad_norm': '1.038', 'learning_rate': '3.049e-05', 'epoch': '0.4551', 'num_input_tokens_seen': 358700656, 'train_runtime': '7.705e+05', 'train_tokens_per_second': '465.5'}

 46%|████▌     | 102400/225000 [214:01:58<686:28:23, 20.16s/it]
 46%|████▌     | 102401/225000 [214:02:21<714:24:01, 20.98s/it]
 46%|████▌     | 102402/225000 [214:02:35<646:03:21, 18.97s/it]
 46%|████▌     | 102403/225000 [214:02:46<567:43:23, 16.67s/it]
 46%|████▌     | 102404/225000 [214:02:58<520:11:10, 15.28s/it]
 46%|████▌     | 102405/225000 [214:03:17<555:32:30, 16.31s/it]
 46%|████▌     | 102406/225000 [214:03:42<645:54:28, 18.97s/it]
 46%|████▌     | 102407/225000 [214:03:58<613:10:11, 18.01s/it]
 46%|████▌     | 102408/225000 [214:04:17<628:06:08, 18.44s/it]
 46%|████▌     | 102409/225000 [214:04:30<563:40:52, 16.55s/it]
 46%|████▌     | 102410/225000 [214:04:43<531:17:11, 15.60s/it]
 46%|████▌     | 102411/225000 [214:05:06<609:58:57, 17.91s/it]
 46%|████▌     | 102412/225000 [214:05:30<673:09:56, 19.77s/it]
 46%|████▌     | 102413/225000 [214:05:49<662:58:07, 19.47s/it]
 46%|████▌     | 102414/225000 [214:06:07<641:13:05, 18.83s/it]
 46%|████▌     | 102415/225000 [214:06:19<577:14:21, 16.95s/it]
 46%|████▌     | 102416/225000 [214:06:32<538:53:22, 15.83s/it]
 46%|████▌     | 102417/225000 [214:06:50<560:42:59, 16.47s/it]
 46%|████▌     | 102418/225000 [214:07:09<585:45:02, 17.20s/it]
 46%|████▌     | 102419/225000 [214:07:28<598:02:43, 17.56s/it]
 46%|████▌     | 102420/225000 [214:07:51<652:42:29, 19.17s/it]
 46%|████▌     | 102421/225000 [214:08:08<635:56:21, 18.68s/it]
 46%|████▌     | 102422/225000 [214:08:16<527:54:14, 15.50s/it]
 46%|████▌     | 102423/225000 [214:08:47<684:58:45, 20.12s/it]
 46%|████▌     | 102424/225000 [214:09:04<657:30:06, 19.31s/it]
 46%|████▌     | 102425/225000 [214:09:28<700:55:12, 20.59s/it]
 46%|████▌     | 102426/225000 [214:09:43<647:12:45, 19.01s/it]
 46%|████▌     | 102427/225000 [214:10:04<659:38:37, 19.37s/it]
 46%|████▌     | 102428/225000 [214:10:24<667:41:48, 19.61s/it]
 46%|████▌     | 102429/225000 [214:10:36<588:24:26, 17.28s/it]
 46%|████▌     | 102430/225000 [214:10:47<525:13:54, 15.43s/it]
 46%|████▌     | 102431/225000 [214:10:59<495:42:56, 14.56s/it]
 46%|████▌     | 102432/225000 [214:11:23<594:50:52, 17.47s/it]
 46%|████▌     | 102433/225000 [214:11:44<624:35:56, 18.35s/it]
 46%|████▌     | 102434/225000 [214:11:59<591:33:54, 17.38s/it]
 46%|████▌     | 102435/225000 [214:12:10<522:11:32, 15.34s/it]
 46%|████▌     | 102436/225000 [214:12:35<622:12:29, 18.28s/it]
 46%|████▌     | 102437/225000 [214:12:46<554:36:34, 16.29s/it]
 46%|████▌     | 102438/225000 [214:13:07<599:06:45, 17.60s/it]
 46%|████▌     | 102439/225000 [214:13:28<638:27:37, 18.75s/it]
 46%|████▌     | 102440/225000 [214:13:51<675:57:13, 19.86s/it]
 46%|████▌     | 102441/225000 [214:14:00<563:19:53, 16.55s/it]
 46%|████▌     | 102442/225000 [214:14:27<668:18:41, 19.63s/it]
 46%|████▌     | 102443/225000 [214:14:43<640:10:49, 18.80s/it]
 46%|████▌     | 102444/225000 [214:14:56<577:24:38, 16.96s/it]
 46%|████▌     | 102445/225000 [214:15:13<582:12:33, 17.10s/it]
 46%|████▌     | 102446/225000 [214:15:37<646:56:27, 19.00s/it]
 46%|████▌     | 102447/225000 [214:15:52<606:40:59, 17.82s/it]
 46%|████▌     | 102448/225000 [214:16:06<563:29:08, 16.55s/it]
 46%|████▌     | 102449/225000 [214:16:29<635:23:54, 18.67s/it]
 46%|████▌     | 102450/225000 [214:16:41<562:02:46, 16.51s/it]
                                                               
{'loss': '0.8955', 'grad_norm': '1.721', 'learning_rate': '3.047e-05', 'epoch': '0.4553', 'num_input_tokens_seen': 359129200, 'train_runtime': '7.714e+05', 'train_tokens_per_second': '465.6'}

 46%|████▌     | 102450/225000 [214:16:41<562:02:46, 16.51s/it]
 46%|████▌     | 102451/225000 [214:16:57<564:12:26, 16.57s/it]
 46%|████▌     | 102452/225000 [214:17:07<491:57:14, 14.45s/it]
 46%|████▌     | 102453/225000 [214:17:18<459:10:01, 13.49s/it]
 46%|████▌     | 102454/225000 [214:17:43<572:54:02, 16.83s/it]
 46%|████▌     | 102455/225000 [214:17:56<532:52:32, 15.65s/it]
 46%|████▌     | 102456/225000 [214:18:16<578:30:04, 16.99s/it]
 46%|████▌     | 102457/225000 [214:18:27<520:13:35, 15.28s/it]
 46%|████▌     | 102458/225000 [214:18:38<473:39:10, 13.91s/it]
 46%|████▌     | 102459/225000 [214:18:59<549:02:31, 16.13s/it]
 46%|████▌     | 102460/225000 [214:19:20<593:20:39, 17.43s/it]
 46%|████▌     | 102461/225000 [214:19:39<617:50:21, 18.15s/it]
 46%|████▌     | 102462/225000 [214:20:00<642:33:53, 18.88s/it]
 46%|████▌     | 102463/225000 [214:20:07<520:25:52, 15.29s/it]
 46%|████▌     | 102464/225000 [214:20:33<631:39:40, 18.56s/it]
 46%|████▌     | 102465/225000 [214:20:40<508:50:45, 14.95s/it]
 46%|████▌     | 102466/225000 [214:20:48<440:58:01, 12.96s/it]
 46%|████▌     | 102467/225000 [214:21:03<463:42:28, 13.62s/it]
 46%|████▌     | 102468/225000 [214:21:28<578:55:14, 17.01s/it]
 46%|████▌     | 102469/225000 [214:21:47<602:06:05, 17.69s/it]
 46%|████▌     | 102470/225000 [214:21:58<532:23:01, 15.64s/it]
 46%|████▌     | 102471/225000 [214:22:14<532:47:38, 15.65s/it]
 46%|████▌     | 102472/225000 [214:22:28<514:07:52, 15.11s/it]
 46%|████▌     | 102473/225000 [214:22:51<598:47:31, 17.59s/it]
 46%|████▌     | 102474/225000 [214:23:04<546:39:30, 16.06s/it]
 46%|████▌     | 102475/225000 [214:23:24<595:03:13, 17.48s/it]
 46%|████▌     | 102476/225000 [214:23:39<564:00:57, 16.57s/it]
 46%|████▌     | 102477/225000 [214:23:50<506:46:26, 14.89s/it]
 46%|████▌     | 102478/225000 [214:24:02<481:07:59, 14.14s/it]
 46%|████▌     | 102479/225000 [214:24:17<493:31:51, 14.50s/it]
 46%|████▌     | 102480/225000 [214:24:41<586:10:53, 17.22s/it]
 46%|████▌     | 102481/225000 [214:24:53<533:45:19, 15.68s/it]
 46%|████▌     | 102482/225000 [214:25:04<479:49:22, 14.10s/it]
 46%|████▌     | 102483/225000 [214:25:23<536:26:48, 15.76s/it]
 46%|████▌     | 102484/225000 [214:25:42<565:49:09, 16.63s/it]
 46%|████▌     | 102485/225000 [214:26:09<669:53:15, 19.68s/it]
 46%|████▌     | 102486/225000 [214:26:17<551:32:10, 16.21s/it]
 46%|████▌     | 102487/225000 [214:26:32<538:32:13, 15.82s/it]
 46%|████▌     | 102488/225000 [214:26:45<517:36:26, 15.21s/it]
 46%|████▌     | 102489/225000 [214:27:00<510:00:50, 14.99s/it]
 46%|████▌     | 102490/225000 [214:27:14<501:19:27, 14.73s/it]
 46%|████▌     | 102491/225000 [214:27:31<522:29:58, 15.35s/it]
 46%|████▌     | 102492/225000 [214:27:42<482:54:21, 14.19s/it]
 46%|████▌     | 102493/225000 [214:28:15<669:51:20, 19.68s/it]
 46%|████▌     | 102494/225000 [214:28:26<584:20:17, 17.17s/it]
 46%|████▌     | 102495/225000 [214:28:36<505:27:21, 14.85s/it]
 46%|████▌     | 102496/225000 [214:29:05<649:10:12, 19.08s/it]
 46%|████▌     | 102497/225000 [214:29:24<653:38:22, 19.21s/it]
 46%|████▌     | 102498/225000 [214:29:39<613:16:48, 18.02s/it]
 46%|████▌     | 102499/225000 [214:29:55<590:23:38, 17.35s/it]
 46%|████▌     | 102500/225000 [214:30:18<644:14:46, 18.93s/it]
                                                               
{'loss': '0.9', 'grad_norm': '1.46', 'learning_rate': '3.045e-05', 'epoch': '0.4556', 'num_input_tokens_seen': 359525280, 'train_runtime': '7.722e+05', 'train_tokens_per_second': '465.6'}

 46%|████▌     | 102500/225000 [214:30:18<644:14:46, 18.93s/it]
 46%|████▌     | 102501/225000 [214:30:36<637:32:20, 18.74s/it]
 46%|████▌     | 102502/225000 [214:30:45<543:24:03, 15.97s/it]
 46%|████▌     | 102503/225000 [214:31:01<541:53:20, 15.93s/it]
 46%|████▌     | 102504/225000 [214:31:28<648:38:13, 19.06s/it]
 46%|████▌     | 102505/225000 [214:31:40<582:30:24, 17.12s/it]
 46%|████▌     | 102506/225000 [214:31:59<601:53:02, 17.69s/it]
 46%|████▌     | 102507/225000 [214:32:15<579:56:39, 17.04s/it]
 46%|████▌     | 102508/225000 [214:32:39<656:21:56, 19.29s/it]
 46%|████▌     | 102509/225000 [214:32:57<635:07:34, 18.67s/it]
 46%|████▌     | 102510/225000 [214:33:15<634:11:59, 18.64s/it]
 46%|████▌     | 102511/225000 [214:33:36<653:47:51, 19.22s/it]
 46%|████▌     | 102512/225000 [214:33:54<642:22:44, 18.88s/it]
 46%|████▌     | 102513/225000 [214:34:09<608:54:54, 17.90s/it]
 46%|████▌     | 102514/225000 [214:34:25<583:31:09, 17.15s/it]
 46%|████▌     | 102515/225000 [214:34:36<522:14:08, 15.35s/it]
 46%|████▌     | 102516/225000 [214:34:52<528:23:20, 15.53s/it]
 46%|████▌     | 102517/225000 [214:35:10<559:13:37, 16.44s/it]
 46%|████▌     | 102518/225000 [214:35:27<561:16:35, 16.50s/it]
 46%|████▌     | 102519/225000 [214:35:40<524:49:25, 15.43s/it]
 46%|████▌     | 102520/225000 [214:36:05<626:09:17, 18.40s/it]
 46%|████▌     | 102521/225000 [214:36:17<557:05:39, 16.37s/it]
 46%|████▌     | 102522/225000 [214:36:33<547:59:14, 16.11s/it]
 46%|████▌     | 102523/225000 [214:36:49<548:24:20, 16.12s/it]
 46%|████▌     | 102524/225000 [214:37:00<503:16:10, 14.79s/it]
 46%|████▌     | 102525/225000 [214:37:16<508:13:57, 14.94s/it]
 46%|████▌     | 102526/225000 [214:37:40<601:41:31, 17.69s/it]
 46%|████▌     | 102527/225000 [214:37:52<548:47:03, 16.13s/it]
 46%|████▌     | 102528/225000 [214:38:07<532:02:05, 15.64s/it]
 46%|████▌     | 102529/225000 [214:38:23<542:09:31, 15.94s/it]
 46%|████▌     | 102530/225000 [214:38:43<576:27:07, 16.94s/it]
 46%|████▌     | 102531/225000 [214:39:00<582:51:45, 17.13s/it]
 46%|████▌     | 102532/225000 [214:39:19<600:25:29, 17.65s/it]
 46%|████▌     | 102533/225000 [214:39:28<508:02:03, 14.93s/it]
 46%|████▌     | 102534/225000 [214:39:45<534:52:14, 15.72s/it]
 46%|████▌     | 102535/225000 [214:39:59<518:20:03, 15.24s/it]
 46%|████▌     | 102536/225000 [214:40:19<561:17:26, 16.50s/it]
 46%|████▌     | 102537/225000 [214:40:31<516:29:51, 15.18s/it]
 46%|████▌     | 102538/225000 [214:40:46<515:38:29, 15.16s/it]
 46%|████▌     | 102539/225000 [214:41:06<563:54:18, 16.58s/it]
 46%|████▌     | 102540/225000 [214:41:26<595:54:49, 17.52s/it]
 46%|████▌     | 102541/225000 [214:41:47<632:55:58, 18.61s/it]
 46%|████▌     | 102542/225000 [214:41:59<571:25:27, 16.80s/it]
 46%|████▌     | 102543/225000 [214:42:16<567:22:54, 16.68s/it]
 46%|████▌     | 102544/225000 [214:42:34<585:42:55, 17.22s/it]
 46%|████▌     | 102545/225000 [214:42:44<513:04:08, 15.08s/it]
 46%|████▌     | 102546/225000 [214:42:56<483:29:23, 14.21s/it]
 46%|████▌     | 102547/225000 [214:43:10<479:43:35, 14.10s/it]
 46%|████▌     | 102548/225000 [214:43:29<522:29:46, 15.36s/it]
 46%|████▌     | 102549/225000 [214:43:45<531:13:57, 15.62s/it]
 46%|████▌     | 102550/225000 [214:44:05<578:51:24, 17.02s/it]
                                                               
{'loss': '0.9033', 'grad_norm': '1.041', 'learning_rate': '3.044e-05', 'epoch': '0.4558', 'num_input_tokens_seen': 359940848, 'train_runtime': '7.73e+05', 'train_tokens_per_second': '465.6'}

 46%|████▌     | 102550/225000 [214:44:05<578:51:24, 17.02s/it]
 46%|████▌     | 102551/225000 [214:44:22<575:35:36, 16.92s/it]
 46%|████▌     | 102552/225000 [214:44:45<635:39:39, 18.69s/it]
 46%|████▌     | 102553/225000 [214:45:03<630:47:06, 18.55s/it]
 46%|████▌     | 102554/225000 [214:45:11<527:42:58, 15.52s/it]
 46%|████▌     | 102555/225000 [214:45:27<528:45:45, 15.55s/it]
 46%|████▌     | 102556/225000 [214:45:43<535:28:52, 15.74s/it]
 46%|████▌     | 102557/225000 [214:45:57<520:16:55, 15.30s/it]
 46%|████▌     | 102558/225000 [214:46:16<549:22:02, 16.15s/it]
 46%|████▌     | 102559/225000 [214:46:33<561:49:52, 16.52s/it]
 46%|████▌     | 102560/225000 [214:46:49<554:43:53, 16.31s/it]
 46%|████▌     | 102561/225000 [214:47:10<605:27:02, 17.80s/it]
 46%|████▌     | 102562/225000 [214:47:33<654:24:14, 19.24s/it]
 46%|████▌     | 102563/225000 [214:47:51<649:28:19, 19.10s/it]
 46%|████▌     | 102564/225000 [214:48:03<578:00:29, 17.00s/it]
 46%|████▌     | 102565/225000 [214:48:22<597:28:37, 17.57s/it]
 46%|████▌     | 102566/225000 [214:48:41<611:14:06, 17.97s/it]
 46%|████▌     | 102567/225000 [214:48:57<590:52:50, 17.37s/it]
 46%|████▌     | 102568/225000 [214:49:10<544:01:48, 16.00s/it]
 46%|████▌     | 102569/225000 [214:49:32<603:16:06, 17.74s/it]
 46%|████▌     | 102570/225000 [214:49:58<693:05:18, 20.38s/it]
 46%|████▌     | 102571/225000 [214:50:05<553:54:08, 16.29s/it]
 46%|████▌     | 102572/225000 [214:50:24<580:17:29, 17.06s/it]
 46%|████▌     | 102573/225000 [214:50:44<613:54:44, 18.05s/it]
 46%|████▌     | 102574/225000 [214:50:53<522:56:49, 15.38s/it]
 46%|████▌     | 102575/225000 [214:51:06<491:32:33, 14.45s/it]
 46%|████▌     | 102576/225000 [214:51:28<575:23:21, 16.92s/it]
 46%|████▌     | 102577/225000 [214:51:45<569:25:26, 16.74s/it]
 46%|████▌     | 102578/225000 [214:51:59<547:43:13, 16.11s/it]
 46%|████▌     | 102579/225000 [214:52:13<525:58:14, 15.47s/it]
 46%|████▌     | 102580/225000 [214:52:36<603:39:04, 17.75s/it]
 46%|████▌     | 102581/225000 [214:52:53<593:50:57, 17.46s/it]
 46%|████▌     | 102582/225000 [214:53:13<613:24:24, 18.04s/it]
 46%|████▌     | 102583/225000 [214:53:34<650:57:11, 19.14s/it]
 46%|████▌     | 102584/225000 [214:53:49<605:00:08, 17.79s/it]
 46%|████▌     | 102585/225000 [214:54:02<557:41:50, 16.40s/it]
 46%|████▌     | 102586/225000 [214:54:15<522:31:41, 15.37s/it]
 46%|████▌     | 102587/225000 [214:54:29<506:10:42, 14.89s/it]
 46%|████▌     | 102588/225000 [214:54:46<526:46:49, 15.49s/it]
 46%|████▌     | 102589/225000 [214:54:54<453:13:17, 13.33s/it]
 46%|████▌     | 102590/225000 [214:55:17<553:33:24, 16.28s/it]
 46%|████▌     | 102591/225000 [214:55:29<504:23:26, 14.83s/it]
 46%|████▌     | 102592/225000 [214:55:42<492:29:17, 14.48s/it]
 46%|████▌     | 102593/225000 [214:56:11<640:54:13, 18.85s/it]
 46%|████▌     | 102594/225000 [214:56:31<649:12:22, 19.09s/it]
 46%|████▌     | 102595/225000 [214:56:48<631:36:50, 18.58s/it]
 46%|████▌     | 102596/225000 [214:57:07<632:56:20, 18.62s/it]
 46%|████▌     | 102597/225000 [214:57:18<556:59:56, 16.38s/it]
 46%|████▌     | 102598/225000 [214:57:35<562:47:48, 16.55s/it]
 46%|████▌     | 102599/225000 [214:57:47<518:27:52, 15.25s/it]
 46%|████▌     | 102600/225000 [214:58:01<503:55:13, 14.82s/it]
                                                               
{'loss': '0.932', 'grad_norm': '1.782', 'learning_rate': '3.042e-05', 'epoch': '0.456', 'num_input_tokens_seen': 360342064, 'train_runtime': '7.739e+05', 'train_tokens_per_second': '465.6'}

 46%|████▌     | 102600/225000 [214:58:01<503:55:13, 14.82s/it]
 46%|████▌     | 102601/225000 [214:58:11<451:56:46, 13.29s/it]
 46%|████▌     | 102602/225000 [214:58:29<503:02:13, 14.80s/it]
 46%|████▌     | 102603/225000 [214:58:44<506:33:06, 14.90s/it]
 46%|████▌     | 102604/225000 [214:58:58<492:21:04, 14.48s/it]
 46%|████▌     | 102605/225000 [214:59:16<524:09:26, 15.42s/it]
 46%|████▌     | 102606/225000 [214:59:33<546:11:39, 16.07s/it]
 46%|████▌     | 102607/225000 [214:59:49<548:54:18, 16.15s/it]
 46%|████▌     | 102608/225000 [215:00:11<602:03:58, 17.71s/it]
 46%|████▌     | 102609/225000 [215:00:23<547:37:07, 16.11s/it]
 46%|████▌     | 102610/225000 [215:00:41<567:58:55, 16.71s/it]
 46%|████▌     | 102611/225000 [215:01:02<606:58:06, 17.85s/it]
 46%|████▌     | 102612/225000 [215:01:24<648:54:02, 19.09s/it]
 46%|████▌     | 102613/225000 [215:01:47<693:18:20, 20.39s/it]
 46%|████▌     | 102614/225000 [215:02:01<621:24:11, 18.28s/it]
 46%|████▌     | 102615/225000 [215:02:18<613:58:41, 18.06s/it]
 46%|████▌     | 102616/225000 [215:02:36<616:20:03, 18.13s/it]
 46%|████▌     | 102617/225000 [215:02:51<579:58:58, 17.06s/it]
 46%|████▌     | 102618/225000 [215:03:10<598:26:26, 17.60s/it]
 46%|████▌     | 102619/225000 [215:03:27<596:46:07, 17.55s/it]
 46%|████▌     | 102620/225000 [215:03:43<574:51:14, 16.91s/it]
 46%|████▌     | 102621/225000 [215:03:52<493:15:09, 14.51s/it]
 46%|████▌     | 102622/225000 [215:04:01<443:35:11, 13.05s/it]
 46%|████▌     | 102623/225000 [215:04:22<525:36:55, 15.46s/it]
 46%|████▌     | 102624/225000 [215:04:41<554:43:49, 16.32s/it]
 46%|████▌     | 102625/225000 [215:05:02<610:14:32, 17.95s/it]
 46%|████▌     | 102626/225000 [215:05:23<632:02:23, 18.59s/it]
 46%|████▌     | 102627/225000 [215:05:40<619:03:23, 18.21s/it]
 46%|████▌     | 102628/225000 [215:05:55<584:25:07, 17.19s/it]
 46%|████▌     | 102629/225000 [215:06:11<571:15:51, 16.81s/it]
 46%|████▌     | 102630/225000 [215:06:20<496:00:17, 14.59s/it]
 46%|████▌     | 102631/225000 [215:06:35<498:14:38, 14.66s/it]
 46%|████▌     | 102632/225000 [215:06:48<482:28:13, 14.19s/it]
 46%|████▌     | 102633/225000 [215:07:07<531:39:39, 15.64s/it]
 46%|████▌     | 102634/225000 [215:07:26<567:49:16, 16.71s/it]
 46%|████▌     | 102635/225000 [215:07:41<550:29:24, 16.20s/it]
 46%|████▌     | 102636/225000 [215:08:00<577:43:27, 17.00s/it]
 46%|████▌     | 102637/225000 [215:08:08<490:50:57, 14.44s/it]
 46%|████▌     | 102638/225000 [215:08:27<530:02:56, 15.59s/it]
 46%|████▌     | 102639/225000 [215:08:41<520:39:40, 15.32s/it]
 46%|████▌     | 102640/225000 [215:08:57<526:26:39, 15.49s/it]
 46%|████▌     | 102641/225000 [215:09:21<610:48:35, 17.97s/it]
 46%|████▌     | 102642/225000 [215:09:38<595:59:22, 17.54s/it]
 46%|████▌     | 102643/225000 [215:09:51<552:41:00, 16.26s/it]
 46%|████▌     | 102644/225000 [215:10:04<523:23:35, 15.40s/it]
 46%|████▌     | 102645/225000 [215:10:20<524:50:37, 15.44s/it]
 46%|████▌     | 102646/225000 [215:10:30<475:17:40, 13.98s/it]
 46%|████▌     | 102647/225000 [215:10:57<602:22:01, 17.72s/it]
 46%|████▌     | 102648/225000 [215:11:19<649:06:18, 19.10s/it]
 46%|████▌     | 102649/225000 [215:11:32<583:31:55, 17.17s/it]
 46%|████▌     | 102650/225000 [215:11:47<563:55:23, 16.59s/it]
                                                               
{'loss': '0.8873', 'grad_norm': '1.312', 'learning_rate': '3.04e-05', 'epoch': '0.4562', 'num_input_tokens_seen': 360758104, 'train_runtime': '7.747e+05', 'train_tokens_per_second': '465.7'}

 46%|████▌     | 102650/225000 [215:11:47<563:55:23, 16.59s/it]
 46%|████▌     | 102651/225000 [215:12:10<631:21:25, 18.58s/it]
 46%|████▌     | 102652/225000 [215:12:30<646:17:09, 19.02s/it]
 46%|████▌     | 102653/225000 [215:12:48<636:03:43, 18.72s/it]
 46%|████▌     | 102654/225000 [215:13:09<660:59:32, 19.45s/it]
 46%|████▌     | 102655/225000 [215:13:24<613:44:18, 18.06s/it]
 46%|████▌     | 102656/225000 [215:13:41<595:26:26, 17.52s/it]
 46%|████▌     | 102657/225000 [215:13:57<588:55:09, 17.33s/it]
 46%|████▌     | 102658/225000 [215:14:12<559:21:23, 16.46s/it]
 46%|████▌     | 102659/225000 [215:14:21<484:10:17, 14.25s/it]
 46%|████▌     | 102660/225000 [215:14:46<591:30:41, 17.41s/it]
 46%|████▌     | 102661/225000 [215:14:58<538:24:01, 15.84s/it]
 46%|████▌     | 102662/225000 [215:15:23<632:03:24, 18.60s/it]
 46%|████▌     | 102663/225000 [215:15:27<488:30:24, 14.38s/it]
 46%|████▌     | 102664/225000 [215:15:44<514:16:04, 15.13s/it]
 46%|████▌     | 102665/225000 [215:15:53<451:58:13, 13.30s/it]
 46%|████▌     | 102666/225000 [215:16:09<471:34:10, 13.88s/it]
 46%|████▌     | 102667/225000 [215:16:29<533:45:49, 15.71s/it]
 46%|████▌     | 102668/225000 [215:16:51<600:37:56, 17.68s/it]
 46%|████▌     | 102669/225000 [215:17:07<584:25:31, 17.20s/it]
 46%|████▌     | 102670/225000 [215:17:22<566:47:29, 16.68s/it]
 46%|████▌     | 102671/225000 [215:17:43<608:20:11, 17.90s/it]
 46%|████▌     | 102672/225000 [215:18:04<640:18:11, 18.84s/it]
 46%|████▌     | 102673/225000 [215:18:17<576:32:52, 16.97s/it]
 46%|████▌     | 102674/225000 [215:18:33<569:04:46, 16.75s/it]
 46%|████▌     | 102675/225000 [215:18:57<642:51:50, 18.92s/it]
 46%|████▌     | 102676/225000 [215:19:14<621:15:37, 18.28s/it]
 46%|████▌     | 102677/225000 [215:19:30<603:36:32, 17.76s/it]
 46%|████▌     | 102678/225000 [215:19:51<628:34:49, 18.50s/it]
 46%|████▌     | 102679/225000 [215:20:11<650:37:13, 19.15s/it]
 46%|████▌     | 102680/225000 [215:20:30<641:09:15, 18.87s/it]
 46%|████▌     | 102681/225000 [215:20:45<602:20:03, 17.73s/it]
 46%|████▌     | 102682/225000 [215:20:57<547:03:42, 16.10s/it]
 46%|████▌     | 102683/225000 [215:21:14<558:53:43, 16.45s/it]
 46%|████▌     | 102684/225000 [215:21:34<592:09:16, 17.43s/it]
 46%|████▌     | 102685/225000 [215:21:49<564:36:39, 16.62s/it]
 46%|████▌     | 102686/225000 [215:22:02<527:04:36, 15.51s/it]
 46%|████▌     | 102687/225000 [215:22:16<516:23:04, 15.20s/it]
 46%|████▌     | 102688/225000 [215:22:31<510:36:22, 15.03s/it]
 46%|████▌     | 102689/225000 [215:22:41<464:53:57, 13.68s/it]
 46%|████▌     | 102690/225000 [215:22:53<441:52:10, 13.01s/it]
 46%|████▌     | 102691/225000 [215:23:10<486:11:52, 14.31s/it]
 46%|████▌     | 102692/225000 [215:23:35<592:42:34, 17.45s/it]
 46%|████▌     | 102693/225000 [215:23:57<640:01:28, 18.84s/it]
 46%|████▌     | 102694/225000 [215:24:10<584:41:30, 17.21s/it]
 46%|████▌     | 102695/225000 [215:24:27<583:23:57, 17.17s/it]
 46%|████▌     | 102696/225000 [215:24:46<602:58:54, 17.75s/it]
 46%|████▌     | 102697/225000 [215:25:01<571:28:06, 16.82s/it]
 46%|████▌     | 102698/225000 [215:25:16<549:54:18, 16.19s/it]
 46%|████▌     | 102699/225000 [215:25:28<514:54:08, 15.16s/it]
 46%|████▌     | 102700/225000 [215:25:44<516:02:26, 15.19s/it]
                                                               
{'loss': '0.8876', 'grad_norm': '1.283', 'learning_rate': '3.038e-05', 'epoch': '0.4564', 'num_input_tokens_seen': 361164312, 'train_runtime': '7.755e+05', 'train_tokens_per_second': '465.7'}

 46%|████▌     | 102700/225000 [215:25:44<516:02:26, 15.19s/it]
 46%|████▌     | 102701/225000 [215:25:58<510:05:59, 15.02s/it]
 46%|████▌     | 102702/225000 [215:26:16<539:08:37, 15.87s/it]
 46%|████▌     | 102703/225000 [215:26:29<509:41:45, 15.00s/it]
 46%|████▌     | 102704/225000 [215:26:39<460:15:09, 13.55s/it]
 46%|████▌     | 102705/225000 [215:27:01<546:03:56, 16.07s/it]
 46%|████▌     | 102706/225000 [215:27:18<547:28:18, 16.12s/it]
 46%|████▌     | 102707/225000 [215:27:39<602:46:50, 17.74s/it]
 46%|████▌     | 102708/225000 [215:28:02<652:27:25, 19.21s/it]
 46%|████▌     | 102709/225000 [215:28:24<683:57:02, 20.13s/it]
 46%|████▌     | 102710/225000 [215:28:34<577:05:23, 16.99s/it]
 46%|████▌     | 102711/225000 [215:28:50<566:05:13, 16.66s/it]
 46%|████▌     | 102712/225000 [215:29:10<607:23:33, 17.88s/it]
 46%|████▌     | 102713/225000 [215:29:30<621:30:35, 18.30s/it]
 46%|████▌     | 102714/225000 [215:29:41<550:27:07, 16.20s/it]
 46%|████▌     | 102715/225000 [215:29:53<513:08:55, 15.11s/it]
 46%|████▌     | 102716/225000 [215:30:17<596:37:39, 17.56s/it]
 46%|████▌     | 102717/225000 [215:30:34<588:57:20, 17.34s/it]
 46%|████▌     | 102718/225000 [215:30:52<603:13:00, 17.76s/it]
 46%|████▌     | 102719/225000 [215:31:00<498:14:53, 14.67s/it]
 46%|████▌     | 102720/225000 [215:31:22<571:19:07, 16.82s/it]
 46%|████▌     | 102721/225000 [215:31:41<599:19:33, 17.64s/it]
 46%|████▌     | 102722/225000 [215:31:58<587:32:23, 17.30s/it]
 46%|████▌     | 102723/225000 [215:32:12<552:58:11, 16.28s/it]
 46%|████▌     | 102724/225000 [215:32:28<557:01:16, 16.40s/it]
 46%|████▌     | 102725/225000 [215:32:49<597:44:50, 17.60s/it]
 46%|████▌     | 102726/225000 [215:33:02<550:59:10, 16.22s/it]
 46%|████▌     | 102727/225000 [215:33:19<560:08:37, 16.49s/it]
 46%|████▌     | 102728/225000 [215:33:32<527:00:16, 15.52s/it]
 46%|████▌     | 102729/225000 [215:33:47<520:09:00, 15.31s/it]
 46%|████▌     | 102730/225000 [215:34:01<511:21:22, 15.06s/it]
 46%|████▌     | 102731/225000 [215:34:17<522:44:39, 15.39s/it]
 46%|████▌     | 102732/225000 [215:34:31<500:08:28, 14.73s/it]
 46%|████▌     | 102733/225000 [215:34:42<463:15:32, 13.64s/it]
 46%|████▌     | 102734/225000 [215:35:04<548:12:49, 16.14s/it]
 46%|████▌     | 102735/225000 [215:35:17<520:30:38, 15.33s/it]
 46%|████▌     | 102736/225000 [215:35:28<473:22:57, 13.94s/it]
 46%|████▌     | 102737/225000 [215:35:43<484:38:09, 14.27s/it]
 46%|████▌     | 102738/225000 [215:35:57<483:12:37, 14.23s/it]
 46%|████▌     | 102739/225000 [215:36:07<436:40:21, 12.86s/it]
 46%|████▌     | 102740/225000 [215:36:18<417:59:46, 12.31s/it]
 46%|████▌     | 102741/225000 [215:36:40<517:33:55, 15.24s/it]
 46%|████▌     | 102742/225000 [215:36:57<541:57:01, 15.96s/it]
 46%|████▌     | 102743/225000 [215:37:08<492:07:47, 14.49s/it]
 46%|████▌     | 102744/225000 [215:37:38<647:35:36, 19.07s/it]
 46%|████▌     | 102745/225000 [215:37:54<616:32:01, 18.15s/it]
 46%|████▌     | 102746/225000 [215:38:13<627:30:14, 18.48s/it]
 46%|████▌     | 102747/225000 [215:38:25<552:55:48, 16.28s/it]
 46%|████▌     | 102748/225000 [215:38:47<614:57:55, 18.11s/it]
 46%|████▌     | 102749/225000 [215:39:09<651:21:29, 19.18s/it]
 46%|████▌     | 102750/225000 [215:39:23<604:30:06, 17.80s/it]
                                                               
{'loss': '0.9016', 'grad_norm': '1.208', 'learning_rate': '3.036e-05', 'epoch': '0.4567', 'num_input_tokens_seen': 361567400, 'train_runtime': '7.764e+05', 'train_tokens_per_second': '465.7'}

 46%|████▌     | 102750/225000 [215:39:23<604:30:06, 17.80s/it]
 46%|████▌     | 102751/225000 [215:39:40<591:03:41, 17.41s/it]
 46%|████▌     | 102752/225000 [215:40:05<668:31:05, 19.69s/it]
 46%|████▌     | 102753/225000 [215:40:20<622:57:31, 18.35s/it]
 46%|████▌     | 102754/225000 [215:40:30<540:19:00, 15.91s/it]
 46%|████▌     | 102755/225000 [215:40:36<440:19:51, 12.97s/it]
 46%|████▌     | 102756/225000 [215:40:47<420:53:43, 12.40s/it]
 46%|████▌     | 102757/225000 [215:40:58<402:47:36, 11.86s/it]
 46%|████▌     | 102758/225000 [215:41:12<429:06:44, 12.64s/it]
 46%|████▌     | 102759/225000 [215:41:40<576:20:59, 16.97s/it]
 46%|████▌     | 102760/225000 [215:42:03<645:31:13, 19.01s/it]
 46%|████▌     | 102761/225000 [215:42:12<540:21:40, 15.91s/it]
 46%|████▌     | 102762/225000 [215:42:30<563:31:57, 16.60s/it]
 46%|████▌     | 102763/225000 [215:42:46<551:06:51, 16.23s/it]
 46%|████▌     | 102764/225000 [215:42:51<442:12:00, 13.02s/it]
 46%|████▌     | 102765/225000 [215:43:05<453:14:14, 13.35s/it]
 46%|████▌     | 102766/225000 [215:43:31<576:19:45, 16.97s/it]
 46%|████▌     | 102767/225000 [215:43:34<441:42:04, 13.01s/it]
 46%|████▌     | 102768/225000 [215:43:41<377:54:27, 11.13s/it]
 46%|████▌     | 102769/225000 [215:43:58<432:49:01, 12.75s/it]
 46%|████▌     | 102770/225000 [215:44:12<449:10:42, 13.23s/it]
 46%|████▌     | 102771/225000 [215:44:27<470:44:18, 13.86s/it]
 46%|████▌     | 102772/225000 [215:44:41<472:18:16, 13.91s/it]
 46%|████▌     | 102773/225000 [215:44:56<477:55:29, 14.08s/it]
 46%|████▌     | 102774/225000 [215:45:17<546:01:01, 16.08s/it]
 46%|████▌     | 102775/225000 [215:45:37<587:05:45, 17.29s/it]
 46%|████▌     | 102776/225000 [215:45:51<553:50:37, 16.31s/it]
 46%|████▌     | 102777/225000 [215:46:08<568:03:34, 16.73s/it]
 46%|████▌     | 102778/225000 [215:46:24<558:15:09, 16.44s/it]
 46%|████▌     | 102779/225000 [215:46:37<516:38:34, 15.22s/it]
 46%|████▌     | 102780/225000 [215:46:58<579:16:39, 17.06s/it]
 46%|████▌     | 102781/225000 [215:47:13<557:43:11, 16.43s/it]
 46%|████▌     | 102782/225000 [215:47:23<497:08:13, 14.64s/it]
 46%|████▌     | 102783/225000 [215:47:41<523:54:26, 15.43s/it]
 46%|████▌     | 102784/225000 [215:47:53<489:41:53, 14.42s/it]
 46%|████▌     | 102785/225000 [215:48:03<451:57:14, 13.31s/it]
 46%|████▌     | 102786/225000 [215:48:19<472:22:21, 13.91s/it]
 46%|████▌     | 102787/225000 [215:48:42<571:39:46, 16.84s/it]
 46%|████▌     | 102788/225000 [215:49:07<647:04:42, 19.06s/it]
 46%|████▌     | 102789/225000 [215:49:17<555:51:05, 16.37s/it]
 46%|████▌     | 102790/225000 [215:49:31<531:55:51, 15.67s/it]
 46%|████▌     | 102791/225000 [215:49:47<540:47:52, 15.93s/it]
 46%|████▌     | 102792/225000 [215:50:04<553:06:01, 16.29s/it]
 46%|████▌     | 102793/225000 [215:50:16<502:30:40, 14.80s/it]
 46%|████▌     | 102794/225000 [215:50:41<613:13:51, 18.06s/it]
 46%|████▌     | 102795/225000 [215:50:58<594:34:35, 17.52s/it]
 46%|████▌     | 102796/225000 [215:51:14<581:38:21, 17.13s/it]
 46%|████▌     | 102797/225000 [215:51:37<643:54:14, 18.97s/it]
 46%|████▌     | 102798/225000 [215:51:53<616:29:09, 18.16s/it]
 46%|████▌     | 102799/225000 [215:52:12<623:38:21, 18.37s/it]
 46%|████▌     | 102800/225000 [215:52:29<609:10:21, 17.95s/it]
                                                               
{'loss': '0.9143', 'grad_norm': '1.057', 'learning_rate': '3.035e-05', 'epoch': '0.4569', 'num_input_tokens_seen': 361951976, 'train_runtime': '7.771e+05', 'train_tokens_per_second': '465.7'}

 46%|████▌     | 102800/225000 [215:52:29<609:10:21, 17.95s/it]
 46%|████▌     | 102801/225000 [215:52:51<642:34:28, 18.93s/it]
 46%|████▌     | 102802/225000 [215:53:06<606:35:02, 17.87s/it]
 46%|████▌     | 102803/225000 [215:53:26<630:14:01, 18.57s/it]
 46%|████▌     | 102804/225000 [215:53:43<610:17:33, 17.98s/it]
 46%|████▌     | 102805/225000 [215:53:55<551:46:22, 16.26s/it]
 46%|████▌     | 102806/225000 [215:54:02<461:13:21, 13.59s/it]
 46%|████▌     | 102807/225000 [215:54:13<430:12:14, 12.67s/it]
 46%|████▌     | 102808/225000 [215:54:27<448:49:11, 13.22s/it]
 46%|████▌     | 102809/225000 [215:54:42<468:23:23, 13.80s/it]
 46%|████▌     | 102810/225000 [215:54:57<477:53:54, 14.08s/it]
 46%|████▌     | 102811/225000 [215:55:21<580:56:30, 17.12s/it]
 46%|████▌     | 102812/225000 [215:55:45<641:51:18, 18.91s/it]
 46%|████▌     | 102813/225000 [215:56:05<662:35:17, 19.52s/it]
 46%|████▌     | 102814/225000 [215:56:16<575:55:02, 16.97s/it]
 46%|████▌     | 102815/225000 [215:56:33<572:56:33, 16.88s/it]
 46%|████▌     | 102816/225000 [215:56:52<594:16:19, 17.51s/it]
 46%|████▌     | 102817/225000 [215:57:05<547:24:11, 16.13s/it]
 46%|████▌     | 102818/225000 [215:57:22<556:25:36, 16.39s/it]
 46%|████▌     | 102819/225000 [215:57:44<612:04:51, 18.03s/it]
 46%|████▌     | 102820/225000 [215:58:03<620:04:13, 18.27s/it]
 46%|████▌     | 102821/225000 [215:58:14<548:35:54, 16.16s/it]
 46%|████▌     | 102822/225000 [215:58:26<509:18:37, 15.01s/it]
 46%|████▌     | 102823/225000 [215:58:32<419:02:19, 12.35s/it]
 46%|████▌     | 102824/225000 [215:58:58<556:52:51, 16.41s/it]
 46%|████▌     | 102825/225000 [215:59:18<593:29:43, 17.49s/it]
 46%|████▌     | 102826/225000 [215:59:24<475:30:26, 14.01s/it]
 46%|████▌     | 102827/225000 [215:59:37<458:53:00, 13.52s/it]
 46%|████▌     | 102828/225000 [215:59:50<460:11:31, 13.56s/it]
 46%|████▌     | 102829/225000 [216:00:04<466:45:17, 13.75s/it]
 46%|████▌     | 102830/225000 [216:00:22<509:37:00, 15.02s/it]
 46%|████▌     | 102831/225000 [216:00:39<521:55:04, 15.38s/it]
 46%|████▌     | 102832/225000 [216:00:48<462:30:04, 13.63s/it]
 46%|████▌     | 102833/225000 [216:01:07<517:07:05, 15.24s/it]
 46%|████▌     | 102834/225000 [216:01:20<491:43:58, 14.49s/it]
 46%|████▌     | 102835/225000 [216:01:43<575:38:07, 16.96s/it]
 46%|████▌     | 102836/225000 [216:01:52<502:34:46, 14.81s/it]
 46%|████▌     | 102837/225000 [216:01:59<423:21:21, 12.48s/it]
 46%|████▌     | 102838/225000 [216:02:21<513:05:26, 15.12s/it]
 46%|████▌     | 102839/225000 [216:02:34<489:38:45, 14.43s/it]
 46%|████▌     | 102840/225000 [216:02:44<451:19:41, 13.30s/it]
 46%|████▌     | 102841/225000 [216:02:57<449:13:47, 13.24s/it]
 46%|████▌     | 102842/225000 [216:03:17<511:46:10, 15.08s/it]
 46%|████▌     | 102843/225000 [216:03:28<475:04:51, 14.00s/it]
 46%|████▌     | 102844/225000 [216:03:48<537:27:34, 15.84s/it]
 46%|████▌     | 102845/225000 [216:04:05<548:21:12, 16.16s/it]
 46%|████▌     | 102846/225000 [216:04:26<593:00:35, 17.48s/it]
 46%|████▌     | 102847/225000 [216:04:47<627:43:43, 18.50s/it]
 46%|████▌     | 102848/225000 [216:05:04<618:00:42, 18.21s/it]
 46%|████▌     | 102849/225000 [216:05:24<636:21:30, 18.75s/it]
 46%|████▌     | 102850/225000 [216:05:48<682:22:53, 20.11s/it]
                                                               
{'loss': '0.923', 'grad_norm': '0.9006', 'learning_rate': '3.033e-05', 'epoch': '0.4571', 'num_input_tokens_seen': 362334872, 'train_runtime': '7.779e+05', 'train_tokens_per_second': '465.8'}

 46%|████▌     | 102850/225000 [216:05:48<682:22:53, 20.11s/it]
 46%|████▌     | 102851/225000 [216:05:57<570:38:08, 16.82s/it]
 46%|████▌     | 102852/225000 [216:06:15<583:25:06, 17.19s/it]
 46%|████▌     | 102853/225000 [216:06:26<526:07:01, 15.51s/it]
 46%|████▌     | 102854/225000 [216:06:40<503:13:45, 14.83s/it]
 46%|████▌     | 102855/225000 [216:06:53<487:13:22, 14.36s/it]
 46%|████▌     | 102856/225000 [216:07:16<575:54:51, 16.97s/it]
 46%|████▌     | 102857/225000 [216:07:36<603:26:09, 17.79s/it]
 46%|████▌     | 102858/225000 [216:07:52<593:31:40, 17.49s/it]
 46%|████▌     | 102859/225000 [216:08:13<627:25:09, 18.49s/it]
 46%|████▌     | 102860/225000 [216:08:24<548:17:07, 16.16s/it]
 46%|████▌     | 102861/225000 [216:08:37<513:08:24, 15.12s/it]
 46%|████▌     | 102862/225000 [216:09:02<619:06:26, 18.25s/it]
 46%|████▌     | 102863/225000 [216:09:20<618:39:33, 18.24s/it]
 46%|████▌     | 102864/225000 [216:09:29<523:54:09, 15.44s/it]
 46%|████▌     | 102865/225000 [216:09:41<489:06:31, 14.42s/it]
 46%|████▌     | 102866/225000 [216:09:54<468:15:19, 13.80s/it]
 46%|████▌     | 102867/225000 [216:10:16<553:23:34, 16.31s/it]
 46%|████▌     | 102868/225000 [216:10:30<527:38:52, 15.55s/it]
 46%|████▌     | 102869/225000 [216:10:48<558:30:05, 16.46s/it]
 46%|████▌     | 102870/225000 [216:11:09<598:51:59, 17.65s/it]
 46%|████▌     | 102871/225000 [216:11:34<680:41:36, 20.06s/it]
 46%|████▌     | 102872/225000 [216:11:52<653:30:08, 19.26s/it]
 46%|████▌     | 102873/225000 [216:12:11<657:09:44, 19.37s/it]
 46%|████▌     | 102874/225000 [216:12:25<597:15:46, 17.61s/it]
 46%|████▌     | 102875/225000 [216:12:40<576:50:31, 17.00s/it]
 46%|████▌     | 102876/225000 [216:12:58<584:58:57, 17.24s/it]
 46%|████▌     | 102877/225000 [216:13:12<545:43:24, 16.09s/it]
 46%|████▌     | 102878/225000 [216:13:35<618:25:19, 18.23s/it]
 46%|████▌     | 102879/225000 [216:13:46<546:05:27, 16.10s/it]
 46%|████▌     | 102880/225000 [216:13:57<497:15:05, 14.66s/it]
 46%|████▌     | 102881/225000 [216:14:26<643:17:20, 18.96s/it]
 46%|████▌     | 102882/225000 [216:14:36<550:01:40, 16.21s/it]
 46%|████▌     | 102883/225000 [216:14:56<585:13:08, 17.25s/it]
 46%|████▌     | 102884/225000 [216:15:13<588:35:26, 17.35s/it]
 46%|████▌     | 102885/225000 [216:15:33<606:49:26, 17.89s/it]
 46%|████▌     | 102886/225000 [216:15:39<489:31:24, 14.43s/it]
 46%|████▌     | 102887/225000 [216:15:47<427:57:31, 12.62s/it]
 46%|████▌     | 102888/225000 [216:16:05<477:05:31, 14.07s/it]
 46%|████▌     | 102889/225000 [216:16:15<435:13:32, 12.83s/it]
 46%|████▌     | 102890/225000 [216:16:28<437:43:39, 12.90s/it]
 46%|████▌     | 102891/225000 [216:16:36<391:11:12, 11.53s/it]
 46%|████▌     | 102892/225000 [216:17:02<536:35:39, 15.82s/it]
 46%|████▌     | 102893/225000 [216:17:17<527:51:51, 15.56s/it]
 46%|████▌     | 102894/225000 [216:17:31<517:28:24, 15.26s/it]
 46%|████▌     | 102895/225000 [216:17:47<519:23:50, 15.31s/it]
 46%|████▌     | 102896/225000 [216:17:54<439:53:10, 12.97s/it]
 46%|████▌     | 102897/225000 [216:18:13<499:28:35, 14.73s/it]
 46%|████▌     | 102898/225000 [216:18:34<565:31:02, 16.67s/it]
 46%|████▌     | 102899/225000 [216:18:46<516:32:19, 15.23s/it]
 46%|████▌     | 102900/225000 [216:19:08<583:14:31, 17.20s/it]
                                                               
{'loss': '0.875', 'grad_norm': '0.9266', 'learning_rate': '3.031e-05', 'epoch': '0.4573', 'num_input_tokens_seen': 362726528, 'train_runtime': '7.787e+05', 'train_tokens_per_second': '465.8'}

 46%|████▌     | 102900/225000 [216:19:08<583:14:31, 17.20s/it]
 46%|████▌     | 102901/225000 [216:19:30<633:31:37, 18.68s/it]
 46%|████▌     | 102902/225000 [216:19:48<627:49:12, 18.51s/it]
 46%|████▌     | 102903/225000 [216:20:01<571:44:21, 16.86s/it]
 46%|████▌     | 102904/225000 [216:20:14<534:39:32, 15.76s/it]
 46%|████▌     | 102905/225000 [216:20:30<529:01:13, 15.60s/it]
 46%|████▌     | 102906/225000 [216:20:36<432:25:18, 12.75s/it]
 46%|████▌     | 102907/225000 [216:20:58<532:34:08, 15.70s/it]
 46%|████▌     | 102908/225000 [216:21:11<498:10:03, 14.69s/it]
 46%|████▌     | 102909/225000 [216:21:37<614:14:31, 18.11s/it]
 46%|████▌     | 102910/225000 [216:21:52<582:41:14, 17.18s/it]
 46%|████▌     | 102911/225000 [216:22:05<541:37:29, 15.97s/it]
 46%|████▌     | 102912/225000 [216:22:22<551:26:19, 16.26s/it]
 46%|████▌     | 102913/225000 [216:22:34<513:18:50, 15.14s/it]
 46%|████▌     | 102914/225000 [216:22:50<521:34:48, 15.38s/it]
 46%|████▌     | 102915/225000 [216:23:08<544:22:09, 16.05s/it]
 46%|████▌     | 102916/225000 [216:23:26<561:14:22, 16.55s/it]
 46%|████▌     | 102917/225000 [216:23:35<491:49:43, 14.50s/it]
 46%|████▌     | 102918/225000 [216:23:42<407:15:17, 12.01s/it]
 46%|████▌     | 102919/225000 [216:23:50<373:21:00, 11.01s/it]
 46%|████▌     | 102920/225000 [216:24:09<448:09:20, 13.22s/it]
 46%|████▌     | 102921/225000 [216:24:32<552:23:22, 16.29s/it]
 46%|████▌     | 102922/225000 [216:24:41<474:21:18, 13.99s/it]
 46%|████▌     | 102923/225000 [216:24:57<496:40:07, 14.65s/it]
 46%|████▌     | 102924/225000 [216:25:12<504:26:30, 14.88s/it]
 46%|████▌     | 102925/225000 [216:25:29<526:42:24, 15.53s/it]
 46%|████▌     | 102926/225000 [216:25:46<533:46:20, 15.74s/it]
 46%|████▌     | 102927/225000 [216:26:03<547:41:19, 16.15s/it]
 46%|████▌     | 102928/225000 [216:26:20<563:21:26, 16.61s/it]
 46%|████▌     | 102929/225000 [216:26:38<570:09:45, 16.81s/it]
 46%|████▌     | 102930/225000 [216:26:56<582:10:49, 17.17s/it]
 46%|████▌     | 102931/225000 [216:27:01<458:43:52, 13.53s/it]
 46%|████▌     | 102932/225000 [216:27:14<456:42:02, 13.47s/it]
 46%|████▌     | 102933/225000 [216:27:29<466:45:01, 13.77s/it]
 46%|████▌     | 102934/225000 [216:27:46<508:23:44, 14.99s/it]
 46%|████▌     | 102935/225000 [216:27:58<469:36:26, 13.85s/it]
 46%|████▌     | 102936/225000 [216:28:18<534:46:48, 15.77s/it]
 46%|████▌     | 102937/225000 [216:28:32<520:30:46, 15.35s/it]
 46%|████▌     | 102938/225000 [216:28:48<528:08:46, 15.58s/it]
 46%|████▌     | 102939/225000 [216:29:01<496:46:05, 14.65s/it]
 46%|████▌     | 102940/225000 [216:29:18<522:24:32, 15.41s/it]
 46%|████▌     | 102941/225000 [216:29:30<488:22:17, 14.40s/it]
 46%|████▌     | 102942/225000 [216:29:48<519:47:59, 15.33s/it]
 46%|████▌     | 102943/225000 [216:30:01<501:48:06, 14.80s/it]
 46%|████▌     | 102944/225000 [216:30:19<530:55:53, 15.66s/it]
 46%|████▌     | 102945/225000 [216:30:34<521:58:02, 15.40s/it]
 46%|████▌     | 102946/225000 [216:30:51<539:13:08, 15.90s/it]
 46%|████▌     | 102947/225000 [216:31:04<516:35:03, 15.24s/it]
 46%|████▌     | 102948/225000 [216:31:15<467:26:16, 13.79s/it]
 46%|████▌     | 102949/225000 [216:31:32<504:46:29, 14.89s/it]
 46%|████▌     | 102950/225000 [216:31:43<467:48:32, 13.80s/it]
                                                               
{'loss': '0.9204', 'grad_norm': '1.205', 'learning_rate': '3.029e-05', 'epoch': '0.4576', 'num_input_tokens_seen': 363088088, 'train_runtime': '7.795e+05', 'train_tokens_per_second': '465.8'}

 46%|████▌     | 102950/225000 [216:31:43<467:48:32, 13.80s/it]
 46%|████▌     | 102951/225000 [216:32:01<509:47:43, 15.04s/it]
 46%|████▌     | 102952/225000 [216:32:18<524:54:44, 15.48s/it]
 46%|████▌     | 102953/225000 [216:32:33<520:04:22, 15.34s/it]
 46%|████▌     | 102954/225000 [216:32:46<502:09:20, 14.81s/it]
 46%|████▌     | 102955/225000 [216:33:06<553:46:14, 16.33s/it]
 46%|████▌     | 102956/225000 [216:33:22<550:46:33, 16.25s/it]
 46%|████▌     | 102957/225000 [216:33:38<542:19:11, 16.00s/it]
 46%|████▌     | 102958/225000 [216:33:53<538:56:15, 15.90s/it]
 46%|████▌     | 102959/225000 [216:34:06<500:46:34, 14.77s/it]
 46%|████▌     | 102960/225000 [216:34:18<480:09:01, 14.16s/it]
 46%|████▌     | 102961/225000 [216:34:31<466:55:00, 13.77s/it]
 46%|████▌     | 102962/225000 [216:34:44<461:27:33, 13.61s/it]
 46%|████▌     | 102963/225000 [216:35:05<527:48:50, 15.57s/it]
 46%|████▌     | 102964/225000 [216:35:21<539:11:50, 15.91s/it]
 46%|████▌     | 102965/225000 [216:35:29<458:02:10, 13.51s/it]
 46%|████▌     | 102966/225000 [216:35:49<518:14:23, 15.29s/it]
 46%|████▌     | 102967/225000 [216:35:55<425:08:04, 12.54s/it]
 46%|████▌     | 102968/225000 [216:36:14<492:57:30, 14.54s/it]
 46%|████▌     | 102969/225000 [216:36:24<446:47:40, 13.18s/it]
 46%|████▌     | 102970/225000 [216:36:35<424:34:09, 12.53s/it]
 46%|████▌     | 102971/225000 [216:36:50<448:38:58, 13.24s/it]
 46%|████▌     | 102972/225000 [216:37:05<466:55:32, 13.77s/it]
 46%|████▌     | 102973/225000 [216:37:18<463:03:36, 13.66s/it]
 46%|████▌     | 102974/225000 [216:37:40<544:51:04, 16.07s/it]
 46%|████▌     | 102975/225000 [216:37:53<511:15:01, 15.08s/it]
 46%|████▌     | 102976/225000 [216:38:07<504:41:16, 14.89s/it]
 46%|████▌     | 102977/225000 [216:38:25<530:35:13, 15.65s/it]
 46%|████▌     | 102978/225000 [216:38:44<568:00:35, 16.76s/it]
 46%|████▌     | 102979/225000 [216:39:01<566:57:27, 16.73s/it]
 46%|████▌     | 102980/225000 [216:39:17<565:12:35, 16.68s/it]
 46%|████▌     | 102981/225000 [216:39:38<605:59:13, 17.88s/it]
 46%|████▌     | 102982/225000 [216:39:57<618:27:51, 18.25s/it]
 46%|████▌     | 102983/225000 [216:40:13<595:01:35, 17.56s/it]
 46%|████▌     | 102984/225000 [216:40:27<560:48:46, 16.55s/it]
 46%|████▌     | 102985/225000 [216:40:41<532:33:21, 15.71s/it]
 46%|████▌     | 102986/225000 [216:40:57<534:42:00, 15.78s/it]
 46%|████▌     | 102987/225000 [216:41:15<563:22:40, 16.62s/it]
 46%|████▌     | 102988/225000 [216:41:30<544:06:09, 16.05s/it]
 46%|████▌     | 102989/225000 [216:41:48<563:49:37, 16.64s/it]
 46%|████▌     | 102990/225000 [216:41:55<460:29:40, 13.59s/it]
 46%|████▌     | 102991/225000 [216:42:08<456:09:59, 13.46s/it]
 46%|████▌     | 102992/225000 [216:42:23<477:27:00, 14.09s/it]
 46%|████▌     | 102993/225000 [216:42:36<462:42:34, 13.65s/it]
 46%|████▌     | 102994/225000 [216:42:54<505:50:38, 14.93s/it]
 46%|████▌     | 102995/225000 [216:43:11<526:38:26, 15.54s/it]
 46%|████▌     | 102996/225000 [216:43:22<478:39:16, 14.12s/it]
 46%|████▌     | 102997/225000 [216:43:32<437:42:13, 12.92s/it]
 46%|████▌     | 102998/225000 [216:43:47<459:48:34, 13.57s/it]
 46%|████▌     | 102999/225000 [216:44:04<493:51:40, 14.57s/it]
 46%|████▌     | 103000/225000 [216:44:14<450:27:50, 13.29s/it]
                                                               
{'loss': '0.8876', 'grad_norm': '1.555', 'learning_rate': '3.028e-05', 'epoch': '0.4578', 'num_input_tokens_seen': 363454096, 'train_runtime': '7.803e+05', 'train_tokens_per_second': '465.8'}

 46%|████▌     | 103000/225000 [216:44:14<450:27:50, 13.29s/it]
 46%|████▌     | 103001/225000 [216:44:31<489:47:37, 14.45s/it]
 46%|████▌     | 103002/225000 [216:44:48<512:32:30, 15.12s/it]
 46%|████▌     | 103003/225000 [216:44:59<475:02:24, 14.02s/it]
 46%|████▌     | 103004/225000 [216:45:16<505:58:38, 14.93s/it]
 46%|████▌     | 103005/225000 [216:45:39<587:14:22, 17.33s/it]
 46%|████▌     | 103006/225000 [216:45:58<601:50:05, 17.76s/it]
 46%|████▌     | 103007/225000 [216:46:17<613:33:49, 18.11s/it]
 46%|████▌     | 103008/225000 [216:46:27<535:38:59, 15.81s/it]
 46%|████▌     | 103009/225000 [216:46:43<536:25:38, 15.83s/it]
 46%|████▌     | 103010/225000 [216:46:48<423:46:28, 12.51s/it]
 46%|████▌     | 103011/225000 [216:47:04<456:55:13, 13.48s/it]
 46%|████▌     | 103012/225000 [216:47:17<450:15:53, 13.29s/it]
 46%|████▌     | 103013/225000 [216:47:38<535:02:24, 15.79s/it]
 46%|████▌     | 103014/225000 [216:47:49<479:27:10, 14.15s/it]
 46%|████▌     | 103015/225000 [216:48:00<453:25:48, 13.38s/it]
 46%|████▌     | 103016/225000 [216:48:11<422:34:30, 12.47s/it]
 46%|████▌     | 103017/225000 [216:48:28<477:08:44, 14.08s/it]
 46%|████▌     | 103018/225000 [216:48:46<510:14:19, 15.06s/it]
 46%|████▌     | 103019/225000 [216:48:53<431:49:15, 12.74s/it]
 46%|████▌     | 103020/225000 [216:49:09<466:01:54, 13.75s/it]
 46%|████▌     | 103021/225000 [216:49:25<488:41:02, 14.42s/it]
 46%|████▌     | 103022/225000 [216:49:46<549:53:36, 16.23s/it]
 46%|████▌     | 103023/225000 [216:49:55<479:51:28, 14.16s/it]
 46%|████▌     | 103024/225000 [216:50:20<588:53:20, 17.38s/it]
 46%|████▌     | 103025/225000 [216:50:36<577:32:44, 17.05s/it]
 46%|████▌     | 103026/225000 [216:50:44<481:20:09, 14.21s/it]
 46%|████▌     | 103027/225000 [216:50:53<428:47:40, 12.66s/it]
 46%|████▌     | 103028/225000 [216:51:15<530:23:38, 15.65s/it]
 46%|████▌     | 103029/225000 [216:51:45<676:26:10, 19.97s/it]
 46%|████▌     | 103030/225000 [216:51:56<580:20:07, 17.13s/it]
 46%|████▌     | 103031/225000 [216:52:08<527:52:34, 15.58s/it]
 46%|████▌     | 103032/225000 [216:52:21<499:23:24, 14.74s/it]
 46%|████▌     | 103033/225000 [216:52:35<499:40:51, 14.75s/it]
 46%|████▌     | 103034/225000 [216:52:44<440:09:05, 12.99s/it]
 46%|████▌     | 103035/225000 [216:53:11<574:50:45, 16.97s/it]
 46%|████▌     | 103036/225000 [216:53:26<554:30:23, 16.37s/it]
 46%|████▌     | 103037/225000 [216:53:35<484:31:33, 14.30s/it]
 46%|████▌     | 103038/225000 [216:54:01<598:18:23, 17.66s/it]
 46%|████▌     | 103039/225000 [216:54:13<542:55:05, 16.03s/it]
 46%|████▌     | 103040/225000 [216:54:17<420:24:45, 12.41s/it]
 46%|████▌     | 103041/225000 [216:54:23<361:30:12, 10.67s/it]
 46%|████▌     | 103042/225000 [216:54:39<411:50:49, 12.16s/it]
 46%|████▌     | 103043/225000 [216:54:52<422:09:40, 12.46s/it]
 46%|████▌     | 103044/225000 [216:55:12<498:25:50, 14.71s/it]
 46%|████▌     | 103045/225000 [216:55:25<479:34:51, 14.16s/it]
 46%|████▌     | 103046/225000 [216:55:45<542:38:00, 16.02s/it]
 46%|████▌     | 103047/225000 [216:56:01<539:39:40, 15.93s/it]
 46%|████▌     | 103048/225000 [216:56:11<475:01:31, 14.02s/it]
 46%|████▌     | 103049/225000 [216:56:21<438:40:20, 12.95s/it]
 46%|████▌     | 103050/225000 [216:56:31<410:48:34, 12.13s/it]
                                                               
{'loss': '0.8752', 'grad_norm': '1.898', 'learning_rate': '3.026e-05', 'epoch': '0.458', 'num_input_tokens_seen': 363807728, 'train_runtime': '7.81e+05', 'train_tokens_per_second': '465.8'}

 46%|████▌     | 103050/225000 [216:56:31<410:48:34, 12.13s/it]
 46%|████▌     | 103051/225000 [216:56:36<336:06:49,  9.92s/it]
 46%|████▌     | 103052/225000 [216:56:59<470:26:06, 13.89s/it]
 46%|████▌     | 103053/225000 [216:57:11<450:54:17, 13.31s/it]
 46%|████▌     | 103054/225000 [216:57:28<488:23:18, 14.42s/it]
 46%|████▌     | 103055/225000 [216:57:43<496:12:37, 14.65s/it]
 46%|████▌     | 103056/225000 [216:57:58<494:22:46, 14.59s/it]
 46%|████▌     | 103057/225000 [216:58:17<538:02:40, 15.88s/it]
 46%|████▌     | 103058/225000 [216:58:27<477:34:55, 14.10s/it]
 46%|████▌     | 103059/225000 [216:58:40<468:01:17, 13.82s/it]
 46%|████▌     | 103060/225000 [216:58:56<487:42:40, 14.40s/it]
 46%|████▌     | 103061/225000 [216:59:07<459:06:56, 13.55s/it]
 46%|████▌     | 103062/225000 [216:59:14<389:54:19, 11.51s/it]
 46%|████▌     | 103063/225000 [216:59:23<362:16:03, 10.70s/it]
 46%|████▌     | 103064/225000 [216:59:37<402:28:37, 11.88s/it]
 46%|████▌     | 103065/225000 [216:59:52<427:04:37, 12.61s/it]
 46%|████▌     | 103066/225000 [217:00:02<400:32:20, 11.83s/it]
 46%|████▌     | 103067/225000 [217:00:22<490:47:33, 14.49s/it]
 46%|████▌     | 103068/225000 [217:00:29<412:43:15, 12.19s/it]
 46%|████▌     | 103069/225000 [217:00:49<487:50:25, 14.40s/it]
 46%|████▌     | 103070/225000 [217:01:00<459:06:35, 13.56s/it]
 46%|████▌     | 103071/225000 [217:01:22<537:22:11, 15.87s/it]
 46%|████▌     | 103072/225000 [217:01:34<506:14:34, 14.95s/it]
 46%|████▌     | 103073/225000 [217:01:50<511:49:15, 15.11s/it]
 46%|████▌     | 103074/225000 [217:02:03<496:11:40, 14.65s/it]
 46%|████▌     | 103075/225000 [217:02:22<536:11:46, 15.83s/it]
 46%|████▌     | 103076/225000 [217:02:32<472:35:12, 13.95s/it]
 46%|████▌     | 103077/225000 [217:02:38<399:40:26, 11.80s/it]
 46%|████▌     | 103078/225000 [217:02:56<457:50:15, 13.52s/it]
 46%|████▌     | 103079/225000 [217:03:11<471:11:09, 13.91s/it]
 46%|████▌     | 103080/225000 [217:03:22<440:52:52, 13.02s/it]
 46%|████▌     | 103081/225000 [217:03:31<404:40:52, 11.95s/it]
 46%|████▌     | 103082/225000 [217:03:53<506:03:51, 14.94s/it]
 46%|████▌     | 103083/225000 [217:04:13<557:23:23, 16.46s/it]
 46%|████▌     | 103084/225000 [217:04:29<554:17:42, 16.37s/it]
 46%|████▌     | 103085/225000 [217:04:39<483:55:40, 14.29s/it]
 46%|████▌     | 103086/225000 [217:04:52<472:15:44, 13.95s/it]
 46%|████▌     | 103087/225000 [217:05:09<505:52:01, 14.94s/it]
 46%|████▌     | 103088/225000 [217:05:23<492:12:02, 14.53s/it]
 46%|████▌     | 103089/225000 [217:05:33<454:50:44, 13.43s/it]
 46%|████▌     | 103090/225000 [217:05:44<427:17:31, 12.62s/it]
 46%|████▌     | 103091/225000 [217:05:49<345:09:44, 10.19s/it]
 46%|████▌     | 103092/225000 [217:05:59<341:31:34, 10.09s/it]
 46%|████▌     | 103093/225000 [217:06:10<357:49:23, 10.57s/it]
 46%|████▌     | 103094/225000 [217:06:29<444:37:20, 13.13s/it]
 46%|████▌     | 103095/225000 [217:06:47<486:55:38, 14.38s/it]
 46%|████▌     | 103096/225000 [217:07:06<540:43:19, 15.97s/it]
 46%|████▌     | 103097/225000 [217:07:24<559:25:00, 16.52s/it]
 46%|████▌     | 103098/225000 [217:07:39<546:30:56, 16.14s/it]
 46%|████▌     | 103099/225000 [217:07:59<577:39:31, 17.06s/it]
 46%|████▌     | 103100/225000 [217:08:03<444:58:22, 13.14s/it]
                                                               
{'loss': '0.9115', 'grad_norm': '2.464', 'learning_rate': '3.024e-05', 'epoch': '0.4582', 'num_input_tokens_seen': 364156160, 'train_runtime': '7.817e+05', 'train_tokens_per_second': '465.9'}

 46%|████▌     | 103100/225000 [217:08:03<444:58:22, 13.14s/it]
 46%|████▌     | 103101/225000 [217:08:06<350:18:54, 10.35s/it]
 46%|████▌     | 103102/225000 [217:08:20<383:30:13, 11.33s/it]
 46%|████▌     | 103103/225000 [217:08:35<418:08:59, 12.35s/it]
 46%|████▌     | 103104/225000 [217:08:45<393:03:36, 11.61s/it]
 46%|████▌     | 103105/225000 [217:08:59<416:35:59, 12.30s/it]
 46%|████▌     | 103106/225000 [217:09:04<347:07:51, 10.25s/it]
 46%|████▌     | 103107/225000 [217:09:28<487:03:55, 14.39s/it]
 46%|████▌     | 103108/225000 [217:09:46<525:13:21, 15.51s/it]
 46%|████▌     | 103109/225000 [217:10:01<515:16:03, 15.22s/it]
 46%|████▌     | 103110/225000 [217:10:22<571:27:25, 16.88s/it]
 46%|████▌     | 103111/225000 [217:10:31<501:09:56, 14.80s/it]
 46%|████▌     | 103112/225000 [217:10:40<441:06:46, 13.03s/it]
 46%|████▌     | 103113/225000 [217:10:48<383:55:08, 11.34s/it]
 46%|████▌     | 103114/225000 [217:11:04<430:16:51, 12.71s/it]
 46%|████▌     | 103115/225000 [217:11:26<527:38:03, 15.58s/it]
 46%|████▌     | 103116/225000 [217:11:38<491:26:31, 14.52s/it]
 46%|████▌     | 103117/225000 [217:12:01<580:09:17, 17.14s/it]
 46%|████▌     | 103118/225000 [217:12:16<554:51:27, 16.39s/it]
 46%|████▌     | 103119/225000 [217:12:30<530:27:11, 15.67s/it]
 46%|████▌     | 103120/225000 [217:12:39<459:13:46, 13.56s/it]
 46%|████▌     | 103121/225000 [217:12:47<409:42:46, 12.10s/it]
 46%|████▌     | 103122/225000 [217:12:59<402:54:37, 11.90s/it]
 46%|████▌     | 103123/225000 [217:13:17<467:21:03, 13.80s/it]
 46%|████▌     | 103124/225000 [217:13:30<456:39:54, 13.49s/it]
 46%|████▌     | 103125/225000 [217:13:45<476:44:15, 14.08s/it]
 46%|████▌     | 103126/225000 [217:13:57<457:27:51, 13.51s/it]
 46%|████▌     | 103127/225000 [217:14:13<479:30:50, 14.16s/it]
 46%|████▌     | 103128/225000 [217:14:25<456:37:26, 13.49s/it]
 46%|████▌     | 103129/225000 [217:14:37<446:09:22, 13.18s/it]
 46%|████▌     | 103130/225000 [217:14:51<446:35:03, 13.19s/it]
 46%|████▌     | 103131/225000 [217:15:06<469:58:44, 13.88s/it]
 46%|████▌     | 103132/225000 [217:15:25<518:20:03, 15.31s/it]
 46%|████▌     | 103133/225000 [217:15:40<519:30:41, 15.35s/it]
 46%|████▌     | 103134/225000 [217:15:49<450:27:05, 13.31s/it]
 46%|████▌     | 103135/225000 [217:15:59<416:18:11, 12.30s/it]
 46%|████▌     | 103136/225000 [217:16:12<429:51:13, 12.70s/it]
 46%|████▌     | 103137/225000 [217:16:20<384:21:22, 11.35s/it]
 46%|████▌     | 103138/225000 [217:16:41<476:46:42, 14.08s/it]
 46%|████▌     | 103139/225000 [217:17:02<550:45:48, 16.27s/it]
 46%|████▌     | 103140/225000 [217:17:10<462:24:54, 13.66s/it]
 46%|████▌     | 103141/225000 [217:17:24<462:14:19, 13.66s/it]
 46%|████▌     | 103142/225000 [217:17:42<507:41:38, 15.00s/it]
 46%|████▌     | 103143/225000 [217:17:58<520:24:50, 15.37s/it]
 46%|████▌     | 103144/225000 [217:18:09<476:15:24, 14.07s/it]
 46%|████▌     | 103145/225000 [217:18:21<459:35:37, 13.58s/it]
 46%|████▌     | 103146/225000 [217:18:36<474:23:32, 14.02s/it]
 46%|████▌     | 103147/225000 [217:18:47<441:34:19, 13.05s/it]
 46%|████▌     | 103148/225000 [217:19:15<591:27:43, 17.47s/it]
 46%|████▌     | 103149/225000 [217:19:25<516:49:05, 15.27s/it]
 46%|████▌     | 103150/225000 [217:19:37<481:05:19, 14.21s/it]
                                                               
{'loss': '0.9024', 'grad_norm': '1.675', 'learning_rate': '3.022e-05', 'epoch': '0.4584', 'num_input_tokens_seen': 364501368, 'train_runtime': '7.824e+05', 'train_tokens_per_second': '465.9'}

 46%|████▌     | 103150/225000 [217:19:37<481:05:19, 14.21s/it]
 46%|████▌     | 103151/225000 [217:19:49<458:50:11, 13.56s/it]
 46%|████▌     | 103152/225000 [217:20:10<533:08:55, 15.75s/it]
 46%|████▌     | 103153/225000 [217:20:24<522:53:14, 15.45s/it]
 46%|████▌     | 103154/225000 [217:20:28<396:42:41, 11.72s/it]
 46%|████▌     | 103155/225000 [217:20:42<421:28:20, 12.45s/it]
 46%|████▌     | 103156/225000 [217:21:00<476:38:14, 14.08s/it]
 46%|████▌     | 103157/225000 [217:21:14<484:48:54, 14.32s/it]
 46%|████▌     | 103158/225000 [217:21:24<440:50:47, 13.03s/it]
 46%|████▌     | 103159/225000 [217:21:34<407:24:07, 12.04s/it]
 46%|████▌     | 103160/225000 [217:21:48<429:22:13, 12.69s/it]
 46%|████▌     | 103161/225000 [217:21:51<330:11:04,  9.76s/it]
 46%|████▌     | 103162/225000 [217:22:03<353:28:40, 10.44s/it]
 46%|████▌     | 103163/225000 [217:22:24<460:00:56, 13.59s/it]
 46%|████▌     | 103164/225000 [217:22:36<442:24:49, 13.07s/it]
 46%|████▌     | 103165/225000 [217:22:55<499:17:30, 14.75s/it]
 46%|████▌     | 103166/225000 [217:23:05<449:26:57, 13.28s/it]
 46%|████▌     | 103167/225000 [217:23:24<514:23:41, 15.20s/it]
 46%|████▌     | 103168/225000 [217:23:32<440:39:47, 13.02s/it]
 46%|████▌     | 103169/225000 [217:23:46<445:24:37, 13.16s/it]
 46%|████▌     | 103170/225000 [217:24:05<502:31:54, 14.85s/it]
 46%|████▌     | 103171/225000 [217:24:22<525:21:41, 15.52s/it]
 46%|████▌     | 103172/225000 [217:24:34<493:41:40, 14.59s/it]
 46%|████▌     | 103173/225000 [217:24:50<510:26:39, 15.08s/it]
 46%|████▌     | 103174/225000 [217:25:02<479:47:07, 14.18s/it]
 46%|████▌     | 103175/225000 [217:25:12<429:46:36, 12.70s/it]
 46%|████▌     | 103176/225000 [217:25:26<451:55:31, 13.35s/it]
 46%|████▌     | 103177/225000 [217:25:45<505:18:21, 14.93s/it]
 46%|████▌     | 103178/225000 [217:25:57<471:02:18, 13.92s/it]
 46%|████▌     | 103179/225000 [217:26:17<537:53:59, 15.90s/it]
 46%|████▌     | 103180/225000 [217:26:27<479:55:17, 14.18s/it]
 46%|████▌     | 103181/225000 [217:26:36<419:21:56, 12.39s/it]
 46%|████▌     | 103182/225000 [217:26:42<363:12:56, 10.73s/it]
 46%|████▌     | 103183/225000 [217:26:57<404:00:17, 11.94s/it]
 46%|████▌     | 103184/225000 [217:27:09<400:53:53, 11.85s/it]
 46%|████▌     | 103185/225000 [217:27:26<456:57:07, 13.50s/it]
 46%|████▌     | 103186/225000 [217:27:44<496:07:40, 14.66s/it]
 46%|████▌     | 103187/225000 [217:27:59<499:45:14, 14.77s/it]
 46%|████▌     | 103188/225000 [217:28:18<544:53:27, 16.10s/it]
 46%|████▌     | 103189/225000 [217:28:33<538:10:06, 15.91s/it]
 46%|████▌     | 103190/225000 [217:28:52<563:51:06, 16.66s/it]
 46%|████▌     | 103191/225000 [217:29:08<556:00:32, 16.43s/it]
 46%|████▌     | 103192/225000 [217:29:24<560:40:14, 16.57s/it]
 46%|████▌     | 103193/225000 [217:29:31<460:13:08, 13.60s/it]
 46%|████▌     | 103194/225000 [217:29:37<380:34:19, 11.25s/it]
 46%|████▌     | 103195/225000 [217:29:48<382:07:55, 11.29s/it]
 46%|████▌     | 103196/225000 [217:30:06<445:15:55, 13.16s/it]
 46%|████▌     | 103197/225000 [217:30:14<398:38:24, 11.78s/it]
 46%|████▌     | 103198/225000 [217:30:30<442:34:59, 13.08s/it]
 46%|████▌     | 103199/225000 [217:30:50<504:31:21, 14.91s/it]
 46%|████▌     | 103200/225000 [217:31:08<538:35:36, 15.92s/it]
                                                               
{'loss': '0.94', 'grad_norm': '1.298', 'learning_rate': '3.02e-05', 'epoch': '0.4587', 'num_input_tokens_seen': 364837928, 'train_runtime': '7.831e+05', 'train_tokens_per_second': '465.9'}

 46%|████▌     | 103200/225000 [217:31:08<538:35:36, 15.92s/it]
 46%|████▌     | 103201/225000 [217:31:21<506:14:47, 14.96s/it]
 46%|████▌     | 103202/225000 [217:31:35<499:20:28, 14.76s/it]
 46%|████▌     | 103203/225000 [217:31:49<492:23:02, 14.55s/it]
 46%|████▌     | 103204/225000 [217:32:10<554:09:46, 16.38s/it]
 46%|████▌     | 103205/225000 [217:32:28<569:32:54, 16.83s/it]
 46%|████▌     | 103206/225000 [217:32:45<570:47:38, 16.87s/it]
 46%|████▌     | 103207/225000 [217:32:55<506:48:24, 14.98s/it]
 46%|████▌     | 103208/225000 [217:33:06<465:48:39, 13.77s/it]
 46%|████▌     | 103209/225000 [217:33:24<510:26:56, 15.09s/it]
 46%|████▌     | 103210/225000 [217:33:38<499:09:52, 14.75s/it]
 46%|████▌     | 103211/225000 [217:33:54<508:41:05, 15.04s/it]
 46%|████▌     | 103212/225000 [217:34:11<531:37:38, 15.71s/it]
 46%|████▌     | 103213/225000 [217:34:18<442:40:24, 13.09s/it]
 46%|████▌     | 103214/225000 [217:34:39<519:02:26, 15.34s/it]
 46%|████▌     | 103215/225000 [217:34:59<572:33:26, 16.92s/it]
 46%|████▌     | 103216/225000 [217:35:09<496:14:11, 14.67s/it]
 46%|████▌     | 103217/225000 [217:35:24<501:07:27, 14.81s/it]
 46%|████▌     | 103218/225000 [217:35:42<530:55:05, 15.69s/it]
 46%|████▌     | 103219/225000 [217:36:01<564:28:16, 16.69s/it]
 46%|████▌     | 103220/225000 [217:36:18<572:51:18, 16.93s/it]
 46%|████▌     | 103221/225000 [217:36:26<479:10:07, 14.17s/it]
 46%|████▌     | 103222/225000 [217:36:37<449:20:39, 13.28s/it]
 46%|████▌     | 103223/225000 [217:36:47<412:14:25, 12.19s/it]
 46%|████▌     | 103224/225000 [217:37:06<485:09:48, 14.34s/it]
 46%|████▌     | 103225/225000 [217:37:22<501:42:35, 14.83s/it]
 46%|████▌     | 103226/225000 [217:37:36<489:44:28, 14.48s/it]
 46%|████▌     | 103227/225000 [217:37:55<542:27:56, 16.04s/it]
 46%|████▌     | 103228/225000 [217:38:09<512:53:34, 15.16s/it]
 46%|████▌     | 103229/225000 [217:38:22<500:07:01, 14.79s/it]
 46%|████▌     | 103230/225000 [217:38:40<524:19:31, 15.50s/it]
 46%|████▌     | 103231/225000 [217:38:51<480:20:16, 14.20s/it]
 46%|████▌     | 103232/225000 [217:39:01<436:45:51, 12.91s/it]
 46%|████▌     | 103233/225000 [217:39:25<548:18:29, 16.21s/it]
 46%|████▌     | 103234/225000 [217:39:34<482:43:08, 14.27s/it]
 46%|████▌     | 103235/225000 [217:39:45<442:35:46, 13.09s/it]
 46%|████▌     | 103236/225000 [217:40:02<483:56:30, 14.31s/it]
 46%|████▌     | 103237/225000 [217:40:23<549:16:04, 16.24s/it]
 46%|████▌     | 103238/225000 [217:40:41<573:49:41, 16.97s/it]
 46%|████▌     | 103239/225000 [217:40:54<530:59:00, 15.70s/it]
 46%|████▌     | 103240/225000 [217:41:05<487:44:53, 14.42s/it]
 46%|████▌     | 103241/225000 [217:41:28<566:45:17, 16.76s/it]
 46%|████▌     | 103242/225000 [217:41:48<599:29:00, 17.72s/it]
 46%|████▌     | 103243/225000 [217:42:07<612:57:55, 18.12s/it]
 46%|████▌     | 103244/225000 [217:42:23<592:01:41, 17.50s/it]
 46%|████▌     | 103245/225000 [217:42:36<548:23:25, 16.21s/it]
 46%|████▌     | 103246/225000 [217:42:47<495:59:29, 14.67s/it]
 46%|████▌     | 103247/225000 [217:43:04<519:06:43, 15.35s/it]
 46%|████▌     | 103248/225000 [217:43:16<491:01:58, 14.52s/it]
 46%|████▌     | 103249/225000 [217:43:38<558:13:03, 16.51s/it]
 46%|████▌     | 103250/225000 [217:43:53<547:00:09, 16.17s/it]
                                                               
{'loss': '0.8959', 'grad_norm': '1.788', 'learning_rate': '3.019e-05', 'epoch': '0.4589', 'num_input_tokens_seen': 365205040, 'train_runtime': '7.838e+05', 'train_tokens_per_second': '465.9'}

 46%|████▌     | 103250/225000 [217:43:53<547:00:09, 16.17s/it]
 46%|████▌     | 103251/225000 [217:44:05<503:27:01, 14.89s/it]
 46%|████▌     | 103252/225000 [217:44:21<517:17:02, 15.30s/it]
 46%|████▌     | 103253/225000 [217:44:33<483:29:38, 14.30s/it]
 46%|████▌     | 103254/225000 [217:44:53<541:44:30, 16.02s/it]
 46%|████▌     | 103255/225000 [217:45:11<556:24:26, 16.45s/it]
 46%|████▌     | 103256/225000 [217:45:20<485:36:22, 14.36s/it]
 46%|████▌     | 103257/225000 [217:45:31<455:25:42, 13.47s/it]
 46%|████▌     | 103258/225000 [217:45:47<478:56:52, 14.16s/it]
 46%|████▌     | 103259/225000 [217:46:02<487:43:53, 14.42s/it]
 46%|████▌     | 103260/225000 [217:46:12<440:27:01, 13.02s/it]
 46%|████▌     | 103261/225000 [217:46:33<521:22:06, 15.42s/it]
 46%|████▌     | 103262/225000 [217:46:41<450:04:44, 13.31s/it]
 46%|████▌     | 103263/225000 [217:46:57<476:19:39, 14.09s/it]
 46%|████▌     | 103264/225000 [217:47:10<457:41:50, 13.54s/it]
 46%|████▌     | 103265/225000 [217:47:27<496:56:24, 14.70s/it]
 46%|████▌     | 103266/225000 [217:47:37<450:19:38, 13.32s/it]
 46%|████▌     | 103267/225000 [217:47:57<512:39:17, 15.16s/it]
 46%|████▌     | 103268/225000 [217:48:03<428:55:43, 12.68s/it]
 46%|████▌     | 103269/225000 [217:48:10<370:24:23, 10.95s/it]
 46%|████▌     | 103270/225000 [217:48:24<397:53:28, 11.77s/it]
 46%|████▌     | 103271/225000 [217:48:44<476:39:00, 14.10s/it]
 46%|████▌     | 103272/225000 [217:48:50<402:52:08, 11.91s/it]
 46%|████▌     | 103273/225000 [217:49:02<399:17:59, 11.81s/it]
 46%|████▌     | 103274/225000 [217:49:15<407:29:39, 12.05s/it]
 46%|████▌     | 103275/225000 [217:49:29<427:47:35, 12.65s/it]
 46%|████▌     | 103276/225000 [217:49:45<463:49:57, 13.72s/it]
 46%|████▌     | 103277/225000 [217:49:55<431:04:28, 12.75s/it]
 46%|████▌     | 103278/225000 [217:50:16<506:56:07, 14.99s/it]
 46%|████▌     | 103279/225000 [217:50:31<516:31:29, 15.28s/it]
 46%|████▌     | 103280/225000 [217:50:43<475:18:35, 14.06s/it]
 46%|████▌     | 103281/225000 [217:50:51<415:35:18, 12.29s/it]
 46%|████▌     | 103282/225000 [217:51:10<483:34:22, 14.30s/it]
 46%|████▌     | 103283/225000 [217:51:22<463:05:52, 13.70s/it]
 46%|████▌     | 103284/225000 [217:51:36<460:53:06, 13.63s/it]
 46%|████▌     | 103285/225000 [217:51:48<450:17:35, 13.32s/it]
 46%|████▌     | 103286/225000 [217:51:59<429:39:04, 12.71s/it]
 46%|████▌     | 103287/225000 [217:52:25<564:19:52, 16.69s/it]
 46%|████▌     | 103288/225000 [217:52:34<482:42:31, 14.28s/it]
 46%|████▌     | 103289/225000 [217:52:56<563:58:05, 16.68s/it]
 46%|████▌     | 103290/225000 [217:53:08<509:46:48, 15.08s/it]
 46%|████▌     | 103291/225000 [217:53:25<528:42:20, 15.64s/it]
 46%|████▌     | 103292/225000 [217:53:38<509:57:54, 15.08s/it]
 46%|████▌     | 103293/225000 [217:53:51<480:55:22, 14.23s/it]
 46%|████▌     | 103294/225000 [217:54:03<463:00:48, 13.70s/it]
 46%|████▌     | 103295/225000 [217:54:15<448:30:21, 13.27s/it]
 46%|████▌     | 103296/225000 [217:54:34<501:12:27, 14.83s/it]
 46%|████▌     | 103297/225000 [217:54:50<512:09:00, 15.15s/it]
 46%|████▌     | 103298/225000 [217:54:57<431:19:12, 12.76s/it]
 46%|████▌     | 103299/225000 [217:55:09<420:24:39, 12.44s/it]
 46%|████▌     | 103300/225000 [217:55:23<436:49:12, 12.92s/it]
                                                               
{'loss': '0.9049', 'grad_norm': '1.729', 'learning_rate': '3.017e-05', 'epoch': '0.4591', 'num_input_tokens_seen': 365537456, 'train_runtime': '7.845e+05', 'train_tokens_per_second': '465.9'}

 46%|████▌     | 103300/225000 [217:55:23<436:49:12, 12.92s/it]
 46%|████▌     | 103301/225000 [217:55:39<470:26:33, 13.92s/it]
 46%|████▌     | 103302/225000 [217:55:59<532:17:10, 15.75s/it]
 46%|████▌     | 103303/225000 [217:56:21<591:49:30, 17.51s/it]
 46%|████▌     | 103304/225000 [217:56:36<574:48:47, 17.00s/it]
 46%|████▌     | 103305/225000 [217:56:49<532:34:24, 15.75s/it]
 46%|████▌     | 103306/225000 [217:57:08<565:27:37, 16.73s/it]
 46%|████▌     | 103307/225000 [217:57:28<599:39:22, 17.74s/it]
 46%|████▌     | 103308/225000 [217:57:43<563:58:57, 16.68s/it]
 46%|████▌     | 103309/225000 [217:57:55<520:21:17, 15.39s/it]
 46%|████▌     | 103310/225000 [217:58:08<498:03:00, 14.73s/it]
 46%|████▌     | 103311/225000 [217:58:21<483:33:41, 14.31s/it]
 46%|████▌     | 103312/225000 [217:58:36<485:26:45, 14.36s/it]
 46%|████▌     | 103313/225000 [217:58:46<439:15:32, 13.00s/it]
 46%|████▌     | 103314/225000 [217:59:01<463:22:10, 13.71s/it]
 46%|████▌     | 103315/225000 [217:59:19<501:17:57, 14.83s/it]
 46%|████▌     | 103316/225000 [217:59:33<497:36:02, 14.72s/it]
 46%|████▌     | 103317/225000 [217:59:53<547:41:39, 16.20s/it]
 46%|████▌     | 103318/225000 [218:00:06<520:40:25, 15.40s/it]
 46%|████▌     | 103319/225000 [218:00:23<533:40:01, 15.79s/it]
 46%|████▌     | 103320/225000 [218:00:28<429:22:07, 12.70s/it]
 46%|████▌     | 103321/225000 [218:00:39<407:51:28, 12.07s/it]
 46%|████▌     | 103322/225000 [218:00:48<380:48:05, 11.27s/it]
 46%|████▌     | 103323/225000 [218:01:05<434:19:52, 12.85s/it]
 46%|████▌     | 103324/225000 [218:01:24<494:08:42, 14.62s/it]
 46%|████▌     | 103325/225000 [218:01:46<568:29:21, 16.82s/it]
 46%|████▌     | 103326/225000 [218:01:58<524:48:27, 15.53s/it]
 46%|████▌     | 103327/225000 [218:02:07<459:33:10, 13.60s/it]
 46%|████▌     | 103328/225000 [218:02:20<450:34:12, 13.33s/it]
 46%|████▌     | 103329/225000 [218:02:35<463:24:15, 13.71s/it]
 46%|████▌     | 103330/225000 [218:02:56<541:28:56, 16.02s/it]
 46%|████▌     | 103331/225000 [218:03:07<486:42:23, 14.40s/it]
 46%|████▌     | 103332/225000 [218:03:24<516:21:06, 15.28s/it]
 46%|████▌     | 103333/225000 [218:03:32<444:16:18, 13.15s/it]
 46%|████▌     | 103334/225000 [218:03:37<364:16:47, 10.78s/it]
 46%|████▌     | 103335/225000 [218:03:50<386:27:17, 11.43s/it]
 46%|████▌     | 103336/225000 [218:03:57<339:33:29, 10.05s/it]
 46%|████▌     | 103337/225000 [218:04:09<362:19:47, 10.72s/it]
 46%|████▌     | 103338/225000 [218:04:27<435:09:07, 12.88s/it]
 46%|████▌     | 103339/225000 [218:04:43<459:27:42, 13.60s/it]
 46%|████▌     | 103340/225000 [218:04:58<479:43:11, 14.20s/it]
 46%|████▌     | 103341/225000 [218:05:23<586:56:41, 17.37s/it]
 46%|████▌     | 103342/225000 [218:05:33<511:49:15, 15.15s/it]
 46%|████▌     | 103343/225000 [218:05:37<397:28:19, 11.76s/it]
 46%|████▌     | 103344/225000 [218:05:50<416:14:35, 12.32s/it]
 46%|████▌     | 103345/225000 [218:06:12<508:30:38, 15.05s/it]
 46%|████▌     | 103346/225000 [218:06:22<454:21:05, 13.45s/it]
 46%|████▌     | 103347/225000 [218:06:35<450:57:35, 13.34s/it]
 46%|████▌     | 103348/225000 [218:06:44<408:18:41, 12.08s/it]
 46%|████▌     | 103349/225000 [218:06:52<369:52:24, 10.95s/it]
 46%|████▌     | 103350/225000 [218:07:01<345:13:16, 10.22s/it]
                                                               
{'loss': '0.8945', 'grad_norm': '1.498', 'learning_rate': '3.015e-05', 'epoch': '0.4593', 'num_input_tokens_seen': 365877976, 'train_runtime': '7.852e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103350/225000 [218:07:01<345:13:16, 10.22s/it]
 46%|████▌     | 103351/225000 [218:07:18<418:25:10, 12.38s/it]
 46%|████▌     | 103352/225000 [218:07:33<445:13:12, 13.18s/it]
 46%|████▌     | 103353/225000 [218:07:43<414:52:59, 12.28s/it]
 46%|████▌     | 103354/225000 [218:07:53<392:26:31, 11.61s/it]
 46%|████▌     | 103355/225000 [218:08:06<402:36:10, 11.91s/it]
 46%|████▌     | 103356/225000 [218:08:20<423:02:10, 12.52s/it]
 46%|████▌     | 103357/225000 [218:08:30<402:42:37, 11.92s/it]
 46%|████▌     | 103358/225000 [218:08:46<439:54:58, 13.02s/it]
 46%|████▌     | 103359/225000 [218:08:58<428:49:06, 12.69s/it]
 46%|████▌     | 103360/225000 [218:09:11<432:54:22, 12.81s/it]
 46%|████▌     | 103361/225000 [218:09:25<445:25:15, 13.18s/it]
 46%|████▌     | 103362/225000 [218:09:39<448:33:01, 13.28s/it]
 46%|████▌     | 103363/225000 [218:09:49<422:02:21, 12.49s/it]
 46%|████▌     | 103364/225000 [218:09:58<385:26:20, 11.41s/it]
 46%|████▌     | 103365/225000 [218:10:23<522:29:11, 15.46s/it]
 46%|████▌     | 103366/225000 [218:10:39<524:51:01, 15.53s/it]
 46%|████▌     | 103367/225000 [218:10:52<497:45:47, 14.73s/it]
 46%|████▌     | 103368/225000 [218:11:04<474:14:22, 14.04s/it]
 46%|████▌     | 103369/225000 [218:11:17<459:53:31, 13.61s/it]
 46%|████▌     | 103370/225000 [218:11:34<502:25:33, 14.87s/it]
 46%|████▌     | 103371/225000 [218:11:51<520:17:13, 15.40s/it]
 46%|████▌     | 103372/225000 [218:12:17<632:22:28, 18.72s/it]
 46%|████▌     | 103373/225000 [218:12:29<560:56:18, 16.60s/it]
 46%|████▌     | 103374/225000 [218:12:38<482:09:13, 14.27s/it]
 46%|████▌     | 103375/225000 [218:12:50<463:28:09, 13.72s/it]
 46%|████▌     | 103376/225000 [218:13:12<541:13:00, 16.02s/it]
 46%|████▌     | 103377/225000 [218:13:28<544:33:07, 16.12s/it]
 46%|████▌     | 103378/225000 [218:13:43<530:34:00, 15.70s/it]
 46%|████▌     | 103379/225000 [218:13:53<469:29:47, 13.90s/it]
 46%|████▌     | 103380/225000 [218:14:04<441:06:11, 13.06s/it]
 46%|████▌     | 103381/225000 [218:14:18<455:46:20, 13.49s/it]
 46%|████▌     | 103382/225000 [218:14:22<362:12:03, 10.72s/it]
 46%|████▌     | 103383/225000 [218:14:35<384:15:37, 11.37s/it]
 46%|████▌     | 103384/225000 [218:14:46<375:01:28, 11.10s/it]
 46%|████▌     | 103385/225000 [218:15:00<406:36:31, 12.04s/it]
 46%|████▌     | 103386/225000 [218:15:14<427:17:29, 12.65s/it]
 46%|████▌     | 103387/225000 [218:15:24<398:34:42, 11.80s/it]
 46%|████▌     | 103388/225000 [218:15:40<437:48:42, 12.96s/it]
 46%|████▌     | 103389/225000 [218:15:48<394:55:39, 11.69s/it]
 46%|████▌     | 103390/225000 [218:15:58<371:43:28, 11.00s/it]
 46%|████▌     | 103391/225000 [218:16:09<377:10:45, 11.17s/it]
 46%|████▌     | 103392/225000 [218:16:21<386:49:23, 11.45s/it]
 46%|████▌     | 103393/225000 [218:16:30<358:54:32, 10.62s/it]
 46%|████▌     | 103394/225000 [218:16:49<439:30:38, 13.01s/it]
 46%|████▌     | 103395/225000 [218:17:07<491:40:20, 14.56s/it]
 46%|████▌     | 103396/225000 [218:17:25<529:19:58, 15.67s/it]
 46%|████▌     | 103397/225000 [218:17:39<512:29:16, 15.17s/it]
 46%|████▌     | 103398/225000 [218:17:59<555:36:48, 16.45s/it]
 46%|████▌     | 103399/225000 [218:18:12<522:24:35, 15.47s/it]
 46%|████▌     | 103400/225000 [218:18:22<468:40:28, 13.88s/it]
                                                               
{'loss': '0.9196', 'grad_norm': '1.514', 'learning_rate': '3.013e-05', 'epoch': '0.4596', 'num_input_tokens_seen': 366206040, 'train_runtime': '7.859e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103400/225000 [218:18:22<468:40:28, 13.88s/it]
 46%|████▌     | 103401/225000 [218:18:31<424:53:21, 12.58s/it]
 46%|████▌     | 103402/225000 [218:18:42<407:15:31, 12.06s/it]
 46%|████▌     | 103403/225000 [218:18:59<456:40:34, 13.52s/it]
 46%|████▌     | 103404/225000 [218:19:10<431:36:48, 12.78s/it]
 46%|████▌     | 103405/225000 [218:19:21<413:51:52, 12.25s/it]
 46%|████▌     | 103406/225000 [218:19:36<435:34:19, 12.90s/it]
 46%|████▌     | 103407/225000 [218:19:52<475:17:41, 14.07s/it]
 46%|████▌     | 103408/225000 [218:19:58<388:38:40, 11.51s/it]
 46%|████▌     | 103409/225000 [218:20:13<426:38:17, 12.63s/it]
 46%|████▌     | 103410/225000 [218:20:19<355:59:15, 10.54s/it]
 46%|████▌     | 103411/225000 [218:20:41<468:30:00, 13.87s/it]
 46%|████▌     | 103412/225000 [218:20:53<453:25:40, 13.43s/it]
 46%|████▌     | 103413/225000 [218:21:11<501:57:09, 14.86s/it]
 46%|████▌     | 103414/225000 [218:21:19<431:06:22, 12.76s/it]
 46%|████▌     | 103415/225000 [218:21:32<436:04:57, 12.91s/it]
 46%|████▌     | 103416/225000 [218:21:46<447:28:12, 13.25s/it]
 46%|████▌     | 103417/225000 [218:22:02<469:13:24, 13.89s/it]
 46%|████▌     | 103418/225000 [218:22:21<526:01:55, 15.58s/it]
 46%|████▌     | 103419/225000 [218:22:30<452:31:20, 13.40s/it]
 46%|████▌     | 103420/225000 [218:22:52<546:28:51, 16.18s/it]
 46%|████▌     | 103421/225000 [218:23:05<510:54:00, 15.13s/it]
 46%|████▌     | 103422/225000 [218:23:24<551:10:57, 16.32s/it]
 46%|████▌     | 103423/225000 [218:23:43<582:18:52, 17.24s/it]
 46%|████▌     | 103424/225000 [218:23:53<502:37:15, 14.88s/it]
 46%|████▌     | 103425/225000 [218:24:06<485:26:09, 14.37s/it]
 46%|████▌     | 103426/225000 [218:24:22<501:26:40, 14.85s/it]
 46%|████▌     | 103427/225000 [218:24:38<518:01:51, 15.34s/it]
 46%|████▌     | 103428/225000 [218:24:48<458:49:03, 13.59s/it]
 46%|████▌     | 103429/225000 [218:24:58<418:47:17, 12.40s/it]
 46%|████▌     | 103430/225000 [218:25:10<416:50:30, 12.34s/it]
 46%|████▌     | 103431/225000 [218:25:21<405:56:38, 12.02s/it]
 46%|████▌     | 103432/225000 [218:25:35<422:44:27, 12.52s/it]
 46%|████▌     | 103433/225000 [218:25:48<428:02:12, 12.68s/it]
 46%|████▌     | 103434/225000 [218:26:10<530:28:22, 15.71s/it]
 46%|████▌     | 103435/225000 [218:26:21<473:21:22, 14.02s/it]
 46%|████▌     | 103436/225000 [218:26:35<475:58:34, 14.10s/it]
 46%|████▌     | 103437/225000 [218:26:41<391:58:47, 11.61s/it]
 46%|████▌     | 103438/225000 [218:26:57<439:46:30, 13.02s/it]
 46%|████▌     | 103439/225000 [218:27:08<417:54:23, 12.38s/it]
 46%|████▌     | 103440/225000 [218:27:11<320:55:48,  9.50s/it]
 46%|████▌     | 103441/225000 [218:27:22<342:08:15, 10.13s/it]
 46%|████▌     | 103442/225000 [218:27:39<410:00:00, 12.14s/it]
 46%|████▌     | 103443/225000 [218:27:55<450:17:18, 13.34s/it]
 46%|████▌     | 103444/225000 [218:28:15<518:46:01, 15.36s/it]
 46%|████▌     | 103445/225000 [218:28:42<636:47:30, 18.86s/it]
 46%|████▌     | 103446/225000 [218:28:54<563:39:15, 16.69s/it]
 46%|████▌     | 103447/225000 [218:29:12<575:25:13, 17.04s/it]
 46%|████▌     | 103448/225000 [218:29:26<544:36:24, 16.13s/it]
 46%|████▌     | 103449/225000 [218:29:42<546:59:24, 16.20s/it]
 46%|████▌     | 103450/225000 [218:29:54<501:55:38, 14.87s/it]
                                                               
{'loss': '0.9241', 'grad_norm': '1.642', 'learning_rate': '3.012e-05', 'epoch': '0.4598', 'num_input_tokens_seen': 366546688, 'train_runtime': '7.866e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103450/225000 [218:29:54<501:55:38, 14.87s/it]
 46%|████▌     | 103451/225000 [218:30:06<475:45:57, 14.09s/it]
 46%|████▌     | 103452/225000 [218:30:20<470:11:52, 13.93s/it]
 46%|████▌     | 103453/225000 [218:30:32<454:42:42, 13.47s/it]
 46%|████▌     | 103454/225000 [218:30:44<442:26:51, 13.10s/it]
 46%|████▌     | 103455/225000 [218:30:56<424:16:13, 12.57s/it]
 46%|████▌     | 103456/225000 [218:31:08<423:16:08, 12.54s/it]
 46%|████▌     | 103457/225000 [218:31:21<425:07:15, 12.59s/it]
 46%|████▌     | 103458/225000 [218:31:36<446:21:10, 13.22s/it]
 46%|████▌     | 103459/225000 [218:31:50<457:12:56, 13.54s/it]
 46%|████▌     | 103460/225000 [218:32:11<533:36:55, 15.81s/it]
 46%|████▌     | 103461/225000 [218:32:21<479:00:27, 14.19s/it]
 46%|████▌     | 103462/225000 [218:32:33<451:51:47, 13.38s/it]
 46%|████▌     | 103463/225000 [218:32:46<446:26:57, 13.22s/it]
 46%|████▌     | 103464/225000 [218:32:56<414:17:38, 12.27s/it]
 46%|████▌     | 103465/225000 [218:33:02<350:20:21, 10.38s/it]
 46%|████▌     | 103466/225000 [218:33:20<430:03:10, 12.74s/it]
 46%|████▌     | 103467/225000 [218:33:34<442:30:01, 13.11s/it]
 46%|████▌     | 103468/225000 [218:33:45<425:54:46, 12.62s/it]
 46%|████▌     | 103469/225000 [218:33:56<405:41:53, 12.02s/it]
 46%|████▌     | 103470/225000 [218:34:10<427:36:14, 12.67s/it]
 46%|████▌     | 103471/225000 [218:34:20<402:02:41, 11.91s/it]
 46%|████▌     | 103472/225000 [218:34:38<456:03:48, 13.51s/it]
 46%|████▌     | 103473/225000 [218:34:47<413:27:35, 12.25s/it]
 46%|████▌     | 103474/225000 [218:34:57<394:16:21, 11.68s/it]
 46%|████▌     | 103475/225000 [218:35:06<360:50:29, 10.69s/it]
 46%|████▌     | 103476/225000 [218:35:22<413:40:15, 12.25s/it]
 46%|████▌     | 103477/225000 [218:35:37<450:27:54, 13.34s/it]
 46%|████▌     | 103478/225000 [218:35:52<458:41:30, 13.59s/it]
 46%|████▌     | 103479/225000 [218:36:08<486:48:21, 14.42s/it]
 46%|████▌     | 103480/225000 [218:36:20<467:28:07, 13.85s/it]
 46%|████▌     | 103481/225000 [218:36:37<499:13:22, 14.79s/it]
 46%|████▌     | 103482/225000 [218:36:54<517:02:01, 15.32s/it]
 46%|████▌     | 103483/225000 [218:37:08<501:55:32, 14.87s/it]
 46%|████▌     | 103484/225000 [218:37:16<434:43:36, 12.88s/it]
 46%|████▌     | 103485/225000 [218:37:36<509:56:01, 15.11s/it]
 46%|████▌     | 103486/225000 [218:37:41<402:03:46, 11.91s/it]
 46%|████▌     | 103487/225000 [218:37:55<428:23:31, 12.69s/it]
 46%|████▌     | 103488/225000 [218:38:08<427:38:47, 12.67s/it]
 46%|████▌     | 103489/225000 [218:38:20<424:14:52, 12.57s/it]
 46%|████▌     | 103490/225000 [218:38:29<383:16:27, 11.36s/it]
 46%|████▌     | 103491/225000 [218:38:45<427:08:16, 12.65s/it]
 46%|████▌     | 103492/225000 [218:38:53<389:31:10, 11.54s/it]
 46%|████▌     | 103493/225000 [218:39:06<398:03:36, 11.79s/it]
 46%|████▌     | 103494/225000 [218:39:19<409:51:22, 12.14s/it]
 46%|████▌     | 103495/225000 [218:39:39<491:07:17, 14.55s/it]
 46%|████▌     | 103496/225000 [218:39:47<426:01:56, 12.62s/it]
 46%|████▌     | 103497/225000 [218:39:59<423:08:25, 12.54s/it]
 46%|████▌     | 103498/225000 [218:40:17<478:37:03, 14.18s/it]
 46%|████▌     | 103499/225000 [218:40:36<527:32:04, 15.63s/it]
 46%|████▌     | 103500/225000 [218:40:46<464:05:10, 13.75s/it]
                                                               
{'loss': '0.8958', 'grad_norm': '1.647', 'learning_rate': '3.01e-05', 'epoch': '0.46', 'num_input_tokens_seen': 366846896, 'train_runtime': '7.872e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103500/225000 [218:40:46<464:05:10, 13.75s/it]
 46%|████▌     | 103501/225000 [218:41:06<528:58:32, 15.67s/it]
 46%|████▌     | 103502/225000 [218:41:16<467:59:45, 13.87s/it]
 46%|████▌     | 103503/225000 [218:41:25<423:55:42, 12.56s/it]
 46%|████▌     | 103504/225000 [218:41:39<434:00:11, 12.86s/it]
 46%|████▌     | 103505/225000 [218:41:47<387:49:19, 11.49s/it]
 46%|████▌     | 103506/225000 [218:41:59<390:24:11, 11.57s/it]
 46%|████▌     | 103507/225000 [218:42:07<355:01:54, 10.52s/it]
 46%|████▌     | 103508/225000 [218:42:24<419:55:54, 12.44s/it]
 46%|████▌     | 103509/225000 [218:42:32<376:35:32, 11.16s/it]
 46%|████▌     | 103510/225000 [218:42:53<474:40:00, 14.07s/it]
 46%|████▌     | 103511/225000 [218:43:01<415:52:10, 12.32s/it]
 46%|████▌     | 103512/225000 [218:43:07<356:31:39, 10.56s/it]
 46%|████▌     | 103513/225000 [218:43:24<421:20:28, 12.49s/it]
 46%|████▌     | 103514/225000 [218:43:41<461:37:29, 13.68s/it]
 46%|████▌     | 103515/225000 [218:43:48<398:55:50, 11.82s/it]
 46%|████▌     | 103516/225000 [218:43:58<371:25:54, 11.01s/it]
 46%|████▌     | 103517/225000 [218:44:08<362:00:50, 10.73s/it]
 46%|████▌     | 103518/225000 [218:44:21<384:25:03, 11.39s/it]
 46%|████▌     | 103519/225000 [218:44:27<334:38:26,  9.92s/it]
 46%|████▌     | 103520/225000 [218:44:37<333:04:42,  9.87s/it]
 46%|████▌     | 103521/225000 [218:44:47<331:46:23,  9.83s/it]
 46%|████▌     | 103522/225000 [218:44:59<354:07:49, 10.49s/it]
 46%|████▌     | 103523/225000 [218:45:07<329:13:24,  9.76s/it]
 46%|████▌     | 103524/225000 [218:45:27<437:18:42, 12.96s/it]
 46%|████▌     | 103525/225000 [218:45:38<415:46:24, 12.32s/it]
 46%|████▌     | 103526/225000 [218:45:48<391:23:34, 11.60s/it]
 46%|████▌     | 103527/225000 [218:45:51<310:54:16,  9.21s/it]
 46%|████▌     | 103528/225000 [218:45:59<293:59:43,  8.71s/it]
 46%|████▌     | 103529/225000 [218:46:12<333:09:20,  9.87s/it]
 46%|████▌     | 103530/225000 [218:46:34<458:28:21, 13.59s/it]
 46%|████▌     | 103531/225000 [218:46:44<429:07:24, 12.72s/it]
 46%|████▌     | 103532/225000 [218:47:03<488:27:10, 14.48s/it]
 46%|████▌     | 103533/225000 [218:47:22<529:02:32, 15.68s/it]
 46%|████▌     | 103534/225000 [218:47:36<515:51:38, 15.29s/it]
 46%|████▌     | 103535/225000 [218:47:44<442:05:58, 13.10s/it]
 46%|████▌     | 103536/225000 [218:47:59<464:21:56, 13.76s/it]
 46%|████▌     | 103537/225000 [218:48:18<511:25:45, 15.16s/it]
 46%|████▌     | 103538/225000 [218:48:27<452:49:42, 13.42s/it]
 46%|████▌     | 103539/225000 [218:48:39<443:10:31, 13.14s/it]
 46%|████▌     | 103540/225000 [218:48:49<406:38:33, 12.05s/it]
 46%|████▌     | 103541/225000 [218:49:04<439:21:22, 13.02s/it]
 46%|████▌     | 103542/225000 [218:49:12<384:30:24, 11.40s/it]
 46%|████▌     | 103543/225000 [218:49:29<446:25:16, 13.23s/it]
 46%|████▌     | 103544/225000 [218:49:44<459:43:40, 13.63s/it]
 46%|████▌     | 103545/225000 [218:50:01<494:32:19, 14.66s/it]
 46%|████▌     | 103546/225000 [218:50:23<563:46:03, 16.71s/it]
 46%|████▌     | 103547/225000 [218:50:39<560:05:26, 16.60s/it]
 46%|████▌     | 103548/225000 [218:50:58<584:20:44, 17.32s/it]
 46%|████▌     | 103549/225000 [218:51:14<570:23:13, 16.91s/it]
 46%|████▌     | 103550/225000 [218:51:25<513:35:47, 15.22s/it]
                                                               
{'loss': '0.875', 'grad_norm': '1.282', 'learning_rate': '3.008e-05', 'epoch': '0.4602', 'num_input_tokens_seen': 367150000, 'train_runtime': '7.879e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103550/225000 [218:51:25<513:35:47, 15.22s/it]
 46%|████▌     | 103551/225000 [218:51:36<464:47:40, 13.78s/it]
 46%|████▌     | 103552/225000 [218:51:41<378:04:54, 11.21s/it]
 46%|████▌     | 103553/225000 [218:52:00<456:35:31, 13.53s/it]
 46%|████▌     | 103554/225000 [218:52:13<455:58:12, 13.52s/it]
 46%|████▌     | 103555/225000 [218:52:20<384:27:42, 11.40s/it]
 46%|████▌     | 103556/225000 [218:52:30<374:08:56, 11.09s/it]
 46%|████▌     | 103557/225000 [218:52:53<494:54:46, 14.67s/it]
 46%|████▌     | 103558/225000 [218:53:16<584:04:42, 17.31s/it]
 46%|████▌     | 103559/225000 [218:53:26<504:23:46, 14.95s/it]
 46%|████▌     | 103560/225000 [218:53:38<471:05:29, 13.97s/it]
 46%|████▌     | 103561/225000 [218:53:51<463:04:07, 13.73s/it]
 46%|████▌     | 103562/225000 [218:54:07<489:01:25, 14.50s/it]
 46%|████▌     | 103563/225000 [218:54:26<531:16:01, 15.75s/it]
 46%|████▌     | 103564/225000 [218:54:34<453:57:41, 13.46s/it]
 46%|████▌     | 103565/225000 [218:54:50<486:15:13, 14.42s/it]
 46%|████▌     | 103566/225000 [218:54:56<398:27:05, 11.81s/it]
 46%|████▌     | 103567/225000 [218:55:15<463:52:25, 13.75s/it]
 46%|████▌     | 103568/225000 [218:55:27<448:26:18, 13.29s/it]
 46%|████▌     | 103569/225000 [218:55:44<488:54:14, 14.49s/it]
 46%|████▌     | 103570/225000 [218:55:59<489:14:41, 14.50s/it]
 46%|████▌     | 103571/225000 [218:56:13<486:56:06, 14.44s/it]
 46%|████▌     | 103572/225000 [218:56:24<449:11:26, 13.32s/it]
 46%|████▌     | 103573/225000 [218:56:35<427:45:01, 12.68s/it]
 46%|████▌     | 103574/225000 [218:56:42<374:20:38, 11.10s/it]
 46%|████▌     | 103575/225000 [218:56:55<394:12:48, 11.69s/it]
 46%|████▌     | 103576/225000 [218:57:02<341:12:16, 10.12s/it]
 46%|████▌     | 103577/225000 [218:57:20<419:47:01, 12.45s/it]
 46%|████▌     | 103578/225000 [218:57:27<368:26:32, 10.92s/it]
 46%|████▌     | 103579/225000 [218:57:48<468:54:42, 13.90s/it]
 46%|████▌     | 103580/225000 [218:58:02<468:23:45, 13.89s/it]
 46%|████▌     | 103581/225000 [218:58:10<411:46:39, 12.21s/it]
 46%|████▌     | 103582/225000 [218:58:25<436:34:46, 12.94s/it]
 46%|████▌     | 103583/225000 [218:58:40<458:16:43, 13.59s/it]
 46%|████▌     | 103584/225000 [218:58:49<415:08:26, 12.31s/it]
 46%|████▌     | 103585/225000 [218:58:54<337:16:43, 10.00s/it]
 46%|████▌     | 103586/225000 [218:59:03<335:40:04,  9.95s/it]
 46%|████▌     | 103587/225000 [218:59:22<418:01:32, 12.39s/it]
 46%|████▌     | 103588/225000 [218:59:41<488:14:53, 14.48s/it]
 46%|████▌     | 103589/225000 [218:59:59<521:07:50, 15.45s/it]
 46%|████▌     | 103590/225000 [219:00:04<421:49:32, 12.51s/it]
 46%|████▌     | 103591/225000 [219:00:18<438:01:28, 12.99s/it]
 46%|████▌     | 103592/225000 [219:00:35<480:04:27, 14.24s/it]
 46%|████▌     | 103593/225000 [219:00:56<541:00:08, 16.04s/it]
 46%|████▌     | 103594/225000 [219:01:05<473:31:54, 14.04s/it]
 46%|████▌     | 103595/225000 [219:01:13<415:52:03, 12.33s/it]
 46%|████▌     | 103596/225000 [219:01:25<407:08:48, 12.07s/it]
 46%|████▌     | 103597/225000 [219:01:37<409:46:31, 12.15s/it]
 46%|████▌     | 103598/225000 [219:01:47<388:28:36, 11.52s/it]
 46%|████▌     | 103599/225000 [219:02:04<444:08:49, 13.17s/it]
 46%|████▌     | 103600/225000 [219:02:16<425:27:33, 12.62s/it]
                                                               
{'loss': '0.938', 'grad_norm': '1.908', 'learning_rate': '3.006e-05', 'epoch': '0.4604', 'num_input_tokens_seen': 367448312, 'train_runtime': '7.885e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103600/225000 [219:02:16<425:27:33, 12.62s/it]
 46%|████▌     | 103601/225000 [219:02:28<427:33:08, 12.68s/it]
 46%|████▌     | 103602/225000 [219:02:44<457:05:21, 13.55s/it]
 46%|████▌     | 103603/225000 [219:03:04<519:45:36, 15.41s/it]
 46%|████▌     | 103604/225000 [219:03:16<485:52:10, 14.41s/it]
 46%|████▌     | 103605/225000 [219:03:24<418:20:51, 12.41s/it]
 46%|████▌     | 103606/225000 [219:03:35<409:59:13, 12.16s/it]
 46%|████▌     | 103607/225000 [219:03:52<455:58:20, 13.52s/it]
 46%|████▌     | 103608/225000 [219:04:01<415:59:25, 12.34s/it]
 46%|████▌     | 103609/225000 [219:04:05<323:32:39,  9.60s/it]
 46%|████▌     | 103610/225000 [219:04:13<315:03:07,  9.34s/it]
 46%|████▌     | 103611/225000 [219:04:27<356:46:27, 10.58s/it]
 46%|████▌     | 103612/225000 [219:04:41<394:22:09, 11.70s/it]
 46%|████▌     | 103613/225000 [219:04:55<412:21:22, 12.23s/it]
 46%|████▌     | 103614/225000 [219:05:11<451:41:41, 13.40s/it]
 46%|████▌     | 103615/225000 [219:05:37<578:00:32, 17.14s/it]
 46%|████▌     | 103616/225000 [219:05:44<474:05:51, 14.06s/it]
 46%|████▌     | 103617/225000 [219:06:01<503:48:58, 14.94s/it]
 46%|████▌     | 103618/225000 [219:06:07<418:08:43, 12.40s/it]
 46%|████▌     | 103619/225000 [219:06:22<447:27:10, 13.27s/it]
 46%|████▌     | 103620/225000 [219:06:40<491:45:15, 14.58s/it]
 46%|████▌     | 103621/225000 [219:06:46<409:02:45, 12.13s/it]
 46%|████▌     | 103622/225000 [219:06:55<377:40:19, 11.20s/it]
 46%|████▌     | 103623/225000 [219:07:06<374:44:39, 11.11s/it]
 46%|████▌     | 103624/225000 [219:07:15<350:46:29, 10.40s/it]
 46%|████▌     | 103625/225000 [219:07:28<372:49:36, 11.06s/it]
 46%|████▌     | 103626/225000 [219:07:40<389:46:28, 11.56s/it]
 46%|████▌     | 103627/225000 [219:07:52<388:30:18, 11.52s/it]
 46%|████▌     | 103628/225000 [219:08:05<405:54:40, 12.04s/it]
 46%|████▌     | 103629/225000 [219:08:17<400:08:48, 11.87s/it]
 46%|████▌     | 103630/225000 [219:08:27<383:56:47, 11.39s/it]
 46%|████▌     | 103631/225000 [219:08:39<392:53:44, 11.65s/it]
 46%|████▌     | 103632/225000 [219:08:48<369:56:28, 10.97s/it]
 46%|████▌     | 103633/225000 [219:09:03<401:27:19, 11.91s/it]
 46%|████▌     | 103634/225000 [219:09:20<453:53:10, 13.46s/it]
 46%|████▌     | 103635/225000 [219:09:39<518:02:48, 15.37s/it]
 46%|████▌     | 103636/225000 [219:09:49<460:26:52, 13.66s/it]
 46%|████▌     | 103637/225000 [219:10:00<428:48:01, 12.72s/it]
 46%|████▌     | 103638/225000 [219:10:04<343:07:14, 10.18s/it]
 46%|████▌     | 103639/225000 [219:10:13<330:50:13,  9.81s/it]
 46%|████▌     | 103640/225000 [219:10:26<360:12:47, 10.69s/it]
 46%|████▌     | 103641/225000 [219:10:38<381:50:54, 11.33s/it]
 46%|████▌     | 103642/225000 [219:11:00<489:10:07, 14.51s/it]
 46%|████▌     | 103643/225000 [219:11:18<522:01:09, 15.49s/it]
 46%|████▌     | 103644/225000 [219:11:27<451:02:50, 13.38s/it]
 46%|████▌     | 103645/225000 [219:11:41<459:39:57, 13.64s/it]
 46%|████▌     | 103646/225000 [219:11:51<419:59:51, 12.46s/it]
 46%|████▌     | 103647/225000 [219:12:08<468:53:51, 13.91s/it]
 46%|████▌     | 103648/225000 [219:12:26<511:23:27, 15.17s/it]
 46%|████▌     | 103649/225000 [219:12:36<456:18:16, 13.54s/it]
 46%|████▌     | 103650/225000 [219:12:52<489:15:05, 14.51s/it]
                                                               
{'loss': '0.9021', 'grad_norm': '1.652', 'learning_rate': '3.004e-05', 'epoch': '0.4607', 'num_input_tokens_seen': 367761168, 'train_runtime': '7.892e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103650/225000 [219:12:52<489:15:05, 14.51s/it]
 46%|████▌     | 103651/225000 [219:13:04<459:11:38, 13.62s/it]
 46%|████▌     | 103652/225000 [219:13:14<421:18:35, 12.50s/it]
 46%|████▌     | 103653/225000 [219:13:19<343:35:34, 10.19s/it]
 46%|████▌     | 103654/225000 [219:13:35<406:55:11, 12.07s/it]
 46%|████▌     | 103655/225000 [219:13:52<454:38:22, 13.49s/it]
 46%|████▌     | 103656/225000 [219:14:06<457:48:10, 13.58s/it]
 46%|████▌     | 103657/225000 [219:14:12<388:17:57, 11.52s/it]
 46%|████▌     | 103658/225000 [219:14:22<364:27:42, 10.81s/it]
 46%|████▌     | 103659/225000 [219:14:33<366:06:31, 10.86s/it]
 46%|████▌     | 103660/225000 [219:14:55<485:34:38, 14.41s/it]
 46%|████▌     | 103661/225000 [219:15:08<469:23:00, 13.93s/it]
 46%|████▌     | 103662/225000 [219:15:22<474:29:28, 14.08s/it]
 46%|████▌     | 103663/225000 [219:15:39<494:28:38, 14.67s/it]
 46%|████▌     | 103664/225000 [219:15:52<478:29:18, 14.20s/it]
 46%|████▌     | 103665/225000 [219:16:00<415:07:42, 12.32s/it]
 46%|████▌     | 103666/225000 [219:16:10<391:39:20, 11.62s/it]
 46%|████▌     | 103667/225000 [219:16:19<373:54:27, 11.09s/it]
 46%|████▌     | 103668/225000 [219:16:29<355:49:47, 10.56s/it]
 46%|████▌     | 103669/225000 [219:16:49<455:36:50, 13.52s/it]
 46%|████▌     | 103670/225000 [219:17:02<453:42:08, 13.46s/it]
 46%|████▌     | 103671/225000 [219:17:12<410:27:22, 12.18s/it]
 46%|████▌     | 103672/225000 [219:17:32<496:46:49, 14.74s/it]
 46%|████▌     | 103673/225000 [219:17:41<434:24:05, 12.89s/it]
 46%|████▌     | 103674/225000 [219:17:55<442:06:11, 13.12s/it]
 46%|████▌     | 103675/225000 [219:18:00<363:43:07, 10.79s/it]
 46%|████▌     | 103676/225000 [219:18:15<403:23:58, 11.97s/it]
 46%|████▌     | 103677/225000 [219:18:25<384:32:16, 11.41s/it]
 46%|████▌     | 103678/225000 [219:18:34<360:49:12, 10.71s/it]
 46%|████▌     | 103679/225000 [219:18:46<378:31:02, 11.23s/it]
 46%|████▌     | 103680/225000 [219:19:17<574:38:00, 17.05s/it]
 46%|████▌     | 103681/225000 [219:19:32<555:15:46, 16.48s/it]
 46%|████▌     | 103682/225000 [219:19:55<621:53:32, 18.45s/it]
 46%|████▌     | 103683/225000 [219:20:13<614:41:25, 18.24s/it]
 46%|████▌     | 103684/225000 [219:20:19<487:20:04, 14.46s/it]
 46%|████▌     | 103685/225000 [219:20:24<395:47:42, 11.75s/it]
 46%|████▌     | 103686/225000 [219:20:31<351:33:28, 10.43s/it]
 46%|████▌     | 103687/225000 [219:20:47<409:00:01, 12.14s/it]
 46%|████▌     | 103688/225000 [219:20:55<364:20:48, 10.81s/it]
 46%|████▌     | 103689/225000 [219:21:09<394:11:39, 11.70s/it]
 46%|████▌     | 103690/225000 [219:21:18<369:25:42, 10.96s/it]
 46%|████▌     | 103691/225000 [219:21:34<416:55:29, 12.37s/it]
 46%|████▌     | 103692/225000 [219:21:48<437:08:42, 12.97s/it]
 46%|████▌     | 103693/225000 [219:21:59<413:35:14, 12.27s/it]
 46%|████▌     | 103694/225000 [219:22:10<402:54:58, 11.96s/it]
 46%|████▌     | 103695/225000 [219:22:19<372:28:58, 11.05s/it]
 46%|████▌     | 103696/225000 [219:22:28<353:46:08, 10.50s/it]
 46%|████▌     | 103697/225000 [219:22:33<293:46:21,  8.72s/it]
 46%|████▌     | 103698/225000 [219:22:40<281:44:32,  8.36s/it]
 46%|████▌     | 103699/225000 [219:22:49<286:06:32,  8.49s/it]
 46%|████▌     | 103700/225000 [219:22:55<262:31:52,  7.79s/it]
                                                               
{'loss': '0.9154', 'grad_norm': '1.509', 'learning_rate': '3.003e-05', 'epoch': '0.4609', 'num_input_tokens_seen': 368051728, 'train_runtime': '7.898e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103700/225000 [219:22:55<262:31:52,  7.79s/it]
 46%|████▌     | 103701/225000 [219:22:59<225:44:57,  6.70s/it]
 46%|████▌     | 103702/225000 [219:23:11<274:51:39,  8.16s/it]
 46%|████▌     | 103703/225000 [219:23:20<278:55:38,  8.28s/it]
 46%|████▌     | 103704/225000 [219:23:45<455:10:50, 13.51s/it]
 46%|████▌     | 103705/225000 [219:23:58<449:27:14, 13.34s/it]
 46%|████▌     | 103706/225000 [219:24:06<398:18:21, 11.82s/it]
 46%|████▌     | 103707/225000 [219:24:13<341:35:15, 10.14s/it]
 46%|████▌     | 103708/225000 [219:24:31<424:13:21, 12.59s/it]
 46%|████▌     | 103709/225000 [219:24:45<439:11:30, 13.04s/it]
 46%|████▌     | 103710/225000 [219:24:59<449:26:00, 13.34s/it]
 46%|████▌     | 103711/225000 [219:25:14<464:53:34, 13.80s/it]
 46%|████▌     | 103712/225000 [219:25:26<447:02:52, 13.27s/it]
 46%|████▌     | 103713/225000 [219:25:34<394:18:18, 11.70s/it]
 46%|████▌     | 103714/225000 [219:25:56<499:02:47, 14.81s/it]
 46%|████▌     | 103715/225000 [219:26:12<510:22:14, 15.15s/it]
 46%|████▌     | 103716/225000 [219:26:24<481:17:33, 14.29s/it]
 46%|████▌     | 103717/225000 [219:26:40<492:24:33, 14.62s/it]
 46%|████▌     | 103718/225000 [219:26:47<413:25:29, 12.27s/it]
 46%|████▌     | 103719/225000 [219:26:59<414:05:44, 12.29s/it]
 46%|████▌     | 103720/225000 [219:27:08<379:39:23, 11.27s/it]
 46%|████▌     | 103721/225000 [219:27:22<405:26:34, 12.04s/it]
 46%|████▌     | 103722/225000 [219:27:27<342:47:37, 10.18s/it]
 46%|████▌     | 103723/225000 [219:27:36<329:03:09,  9.77s/it]
 46%|████▌     | 103724/225000 [219:27:49<355:18:04, 10.55s/it]
 46%|████▌     | 103725/225000 [219:28:07<434:29:56, 12.90s/it]
 46%|████▌     | 103726/225000 [219:28:19<430:33:58, 12.78s/it]
 46%|████▌     | 103727/225000 [219:28:26<368:42:19, 10.95s/it]
 46%|████▌     | 103728/225000 [219:28:40<396:41:22, 11.78s/it]
 46%|████▌     | 103729/225000 [219:28:47<354:18:29, 10.52s/it]
 46%|████▌     | 103730/225000 [219:28:58<354:50:27, 10.53s/it]
 46%|████▌     | 103731/225000 [219:29:14<411:25:35, 12.21s/it]
 46%|████▌     | 103732/225000 [219:29:23<379:12:31, 11.26s/it]
 46%|████▌     | 103733/225000 [219:29:43<463:33:01, 13.76s/it]
 46%|████▌     | 103734/225000 [219:30:05<551:13:10, 16.36s/it]
 46%|████▌     | 103735/225000 [219:30:17<505:18:02, 15.00s/it]
 46%|████▌     | 103736/225000 [219:30:36<545:51:30, 16.21s/it]
 46%|████▌     | 103737/225000 [219:30:56<580:22:37, 17.23s/it]
 46%|████▌     | 103738/225000 [219:31:03<483:31:20, 14.35s/it]
 46%|████▌     | 103739/225000 [219:31:13<433:52:57, 12.88s/it]
 46%|████▌     | 103740/225000 [219:31:29<471:09:30, 13.99s/it]
 46%|████▌     | 103741/225000 [219:31:49<531:21:23, 15.78s/it]
 46%|████▌     | 103742/225000 [219:31:56<436:36:08, 12.96s/it]
 46%|████▌     | 103743/225000 [219:32:00<350:24:28, 10.40s/it]
 46%|████▌     | 103744/225000 [219:32:07<312:32:59,  9.28s/it]
 46%|████▌     | 103745/225000 [219:32:14<292:54:26,  8.70s/it]
 46%|████▌     | 103746/225000 [219:32:26<324:48:49,  9.64s/it]
 46%|████▌     | 103747/225000 [219:32:36<328:14:33,  9.75s/it]
 46%|████▌     | 103748/225000 [219:32:53<397:17:20, 11.80s/it]
 46%|████▌     | 103749/225000 [219:33:05<403:22:22, 11.98s/it]
 46%|████▌     | 103750/225000 [219:33:16<394:19:35, 11.71s/it]
                                                               
{'loss': '0.9031', 'grad_norm': '1.186', 'learning_rate': '3.001e-05', 'epoch': '0.4611', 'num_input_tokens_seen': 368361776, 'train_runtime': '7.904e+05', 'train_tokens_per_second': '466'}

 46%|████▌     | 103750/225000 [219:33:16<394:19:35, 11.71s/it]
 46%|████▌     | 103751/225000 [219:33:26<374:33:34, 11.12s/it]
 46%|████▌     | 103752/225000 [219:33:41<412:16:59, 12.24s/it]
 46%|████▌     | 103753/225000 [219:33:57<453:38:52, 13.47s/it]
 46%|████▌     | 103754/225000 [219:34:11<463:46:42, 13.77s/it]
 46%|████▌     | 103755/225000 [219:34:22<431:25:43, 12.81s/it]
 46%|████▌     | 103756/225000 [219:34:32<398:23:47, 11.83s/it]
 46%|████▌     | 103757/225000 [219:34:46<423:00:57, 12.56s/it]
 46%|████▌     | 103758/225000 [219:34:58<418:17:40, 12.42s/it]
 46%|████▌     | 103759/225000 [219:35:10<414:45:13, 12.32s/it]
 46%|████▌     | 103760/225000 [219:35:32<510:49:05, 15.17s/it]
 46%|████▌     | 103761/225000 [219:35:38<419:59:23, 12.47s/it]
 46%|████▌     | 103762/225000 [219:35:52<436:55:05, 12.97s/it]
 46%|████▌     | 103763/225000 [219:36:01<396:48:54, 11.78s/it]
 46%|████▌     | 103764/225000 [219:36:08<344:00:27, 10.22s/it]
 46%|████▌     | 103765/225000 [219:36:22<380:49:50, 11.31s/it]
 46%|████▌     | 103766/225000 [219:36:30<350:29:03, 10.41s/it]
 46%|████▌     | 103767/225000 [219:36:38<328:45:59,  9.76s/it]
 46%|████▌     | 103768/225000 [219:36:54<392:00:52, 11.64s/it]
 46%|████▌     | 103769/225000 [219:37:06<395:35:23, 11.75s/it]
 46%|████▌     | 103770/225000 [219:37:15<364:46:22, 10.83s/it]
 46%|████▌     | 103771/225000 [219:37:27<380:26:29, 11.30s/it]
 46%|████▌     | 103772/225000 [219:37:39<390:01:43, 11.58s/it]
 46%|████▌     | 103773/225000 [219:37:46<335:09:55,  9.95s/it]
 46%|████▌     | 103774/225000 [219:37:53<312:07:27,  9.27s/it]
 46%|████▌     | 103775/225000 [219:38:10<388:26:55, 11.54s/it]
 46%|████▌     | 103776/225000 [219:38:21<383:27:53, 11.39s/it]
 46%|████▌     | 103777/225000 [219:38:29<346:05:50, 10.28s/it]
 46%|████▌     | 103778/225000 [219:38:50<459:17:28, 13.64s/it]
 46%|████▌     | 103779/225000 [219:38:57<392:31:36, 11.66s/it]
 46%|████▌     | 103780/225000 [219:39:20<505:08:32, 15.00s/it]
 46%|████▌     | 103781/225000 [219:39:40<556:20:58, 16.52s/it]
 46%|████▌     | 103782/225000 [219:39:55<539:38:43, 16.03s/it]
 46%|████▌     | 103783/225000 [219:40:12<552:26:36, 16.41s/it]
 46%|████▌     | 103784/225000 [219:40:20<462:50:17, 13.75s/it]
 46%|████▌     | 103785/225000 [219:40:44<569:28:46, 16.91s/it]
 46%|████▌     | 103786/225000 [219:41:03<591:16:25, 17.56s/it]
 46%|████▌     | 103787/225000 [219:41:14<527:05:31, 15.65s/it]
 46%|████▌     | 103788/225000 [219:41:29<518:20:56, 15.39s/it]
 46%|████▌     | 103789/225000 [219:41:40<467:17:26, 13.88s/it]
 46%|████▌     | 103790/225000 [219:41:44<375:56:18, 11.17s/it]
 46%|████▌     | 103791/225000 [219:41:57<389:52:55, 11.58s/it]
 46%|████▌     | 103792/225000 [219:42:09<394:15:22, 11.71s/it]
 46%|████▌     | 103793/225000 [219:42:21<393:56:27, 11.70s/it]
 46%|████▌     | 103794/225000 [219:42:28<348:21:44, 10.35s/it]
 46%|████▌     | 103795/225000 [219:42:53<499:01:32, 14.82s/it]
 46%|████▌     | 103796/225000 [219:43:10<516:27:35, 15.34s/it]
 46%|████▌     | 103797/225000 [219:43:22<490:38:03, 14.57s/it]
 46%|████▌     | 103798/225000 [219:43:36<484:20:36, 14.39s/it]
 46%|████▌     | 103799/225000 [219:43:50<480:40:42, 14.28s/it]
 46%|████▌     | 103800/225000 [219:43:55<382:16:50, 11.35s/it]
                                                               
{'loss': '0.9157', 'grad_norm': '1.931', 'learning_rate': '2.999e-05', 'epoch': '0.4613', 'num_input_tokens_seen': 368670832, 'train_runtime': '7.91e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 103800/225000 [219:43:55<382:16:50, 11.35s/it]
 46%|████▌     | 103801/225000 [219:44:17<490:23:52, 14.57s/it]
 46%|████▌     | 103802/225000 [219:44:31<486:50:55, 14.46s/it]
 46%|████▌     | 103803/225000 [219:44:52<546:41:46, 16.24s/it]
 46%|████▌     | 103804/225000 [219:45:01<478:48:55, 14.22s/it]
 46%|████▌     | 103805/225000 [219:45:18<507:45:07, 15.08s/it]
 46%|████▌     | 103806/225000 [219:45:41<581:27:29, 17.27s/it]
 46%|████▌     | 103807/225000 [219:45:48<483:24:31, 14.36s/it]
 46%|████▌     | 103808/225000 [219:46:02<474:10:35, 14.09s/it]
 46%|████▌     | 103809/225000 [219:46:07<387:51:33, 11.52s/it]
 46%|████▌     | 103810/225000 [219:46:18<377:00:08, 11.20s/it]
 46%|████▌     | 103811/225000 [219:46:32<408:26:18, 12.13s/it]
 46%|████▌     | 103812/225000 [219:46:45<421:09:31, 12.51s/it]
 46%|████▌     | 103813/225000 [219:47:02<458:39:07, 13.62s/it]
 46%|████▌     | 103814/225000 [219:47:06<361:50:28, 10.75s/it]
 46%|████▌     | 103815/225000 [219:47:25<448:08:19, 13.31s/it]
 46%|████▌     | 103816/225000 [219:47:42<490:54:46, 14.58s/it]
 46%|████▌     | 103817/225000 [219:47:52<436:01:53, 12.95s/it]
 46%|████▌     | 103818/225000 [219:48:09<478:31:57, 14.22s/it]
 46%|████▌     | 103819/225000 [219:48:26<507:49:17, 15.09s/it]
 46%|████▌     | 103820/225000 [219:48:37<471:18:35, 14.00s/it]
 46%|████▌     | 103821/225000 [219:48:53<489:27:46, 14.54s/it]
 46%|████▌     | 103822/225000 [219:49:04<456:37:34, 13.57s/it]
 46%|████▌     | 103823/225000 [219:49:23<510:18:00, 15.16s/it]
 46%|████▌     | 103824/225000 [219:49:33<456:32:15, 13.56s/it]
 46%|████▌     | 103825/225000 [219:49:47<460:40:29, 13.69s/it]
 46%|████▌     | 103826/225000 [219:49:58<428:49:02, 12.74s/it]
 46%|████▌     | 103827/225000 [219:50:10<427:59:56, 12.72s/it]
 46%|████▌     | 103828/225000 [219:50:32<516:32:22, 15.35s/it]
 46%|████▌     | 103829/225000 [219:50:38<420:48:42, 12.50s/it]
 46%|████▌     | 103830/225000 [219:51:00<519:31:29, 15.44s/it]
 46%|████▌     | 103831/225000 [219:51:11<470:42:15, 13.98s/it]
 46%|████▌     | 103832/225000 [219:51:14<367:58:36, 10.93s/it]
 46%|████▌     | 103833/225000 [219:51:26<379:05:28, 11.26s/it]
 46%|████▌     | 103834/225000 [219:51:48<482:24:18, 14.33s/it]
 46%|████▌     | 103835/225000 [219:52:01<469:22:43, 13.95s/it]
 46%|████▌     | 103836/225000 [219:52:09<414:16:09, 12.31s/it]
 46%|████▌     | 103837/225000 [219:52:13<323:07:32,  9.60s/it]
 46%|████▌     | 103838/225000 [219:52:23<327:19:59,  9.73s/it]
 46%|████▌     | 103839/225000 [219:52:38<386:52:32, 11.50s/it]
 46%|████▌     | 103840/225000 [219:52:46<344:10:24, 10.23s/it]
 46%|████▌     | 103841/225000 [219:52:55<336:01:03,  9.98s/it]
 46%|████▌     | 103842/225000 [219:53:13<413:08:07, 12.28s/it]
 46%|████▌     | 103843/225000 [219:53:29<453:18:39, 13.47s/it]
 46%|████▌     | 103844/225000 [219:53:35<383:41:20, 11.40s/it]
 46%|████▌     | 103845/225000 [219:53:49<406:39:24, 12.08s/it]
 46%|████▌     | 103846/225000 [219:54:00<389:47:35, 11.58s/it]
 46%|████▌     | 103847/225000 [219:54:14<419:25:34, 12.46s/it]
 46%|████▌     | 103848/225000 [219:54:24<397:29:07, 11.81s/it]
 46%|████▌     | 103849/225000 [219:54:41<448:41:45, 13.33s/it]
 46%|████▌     | 103850/225000 [219:54:59<497:42:39, 14.79s/it]
                                                               
{'loss': '0.9524', 'grad_norm': '1.824', 'learning_rate': '2.997e-05', 'epoch': '0.4616', 'num_input_tokens_seen': 368977272, 'train_runtime': '7.917e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 103850/225000 [219:54:59<497:42:39, 14.79s/it]
 46%|████▌     | 103851/225000 [219:55:07<422:15:45, 12.55s/it]
 46%|████▌     | 103852/225000 [219:55:17<398:36:14, 11.84s/it]
 46%|████▌     | 103853/225000 [219:55:26<368:04:56, 10.94s/it]
 46%|████▌     | 103854/225000 [219:55:32<315:56:15,  9.39s/it]
 46%|████▌     | 103855/225000 [219:55:48<391:19:50, 11.63s/it]
 46%|████▌     | 103856/225000 [219:55:56<355:32:54, 10.57s/it]
 46%|████▌     | 103857/225000 [219:56:13<411:32:18, 12.23s/it]
 46%|████▌     | 103858/225000 [219:56:33<497:34:35, 14.79s/it]
 46%|████▌     | 103859/225000 [219:56:47<485:51:18, 14.44s/it]
 46%|████▌     | 103860/225000 [219:56:55<422:21:31, 12.55s/it]
 46%|████▌     | 103861/225000 [219:57:13<476:52:00, 14.17s/it]
 46%|████▌     | 103862/225000 [219:57:26<461:03:19, 13.70s/it]
 46%|████▌     | 103863/225000 [219:57:35<420:03:39, 12.48s/it]
 46%|████▌     | 103864/225000 [219:58:00<541:44:30, 16.10s/it]
 46%|████▌     | 103865/225000 [219:58:22<597:43:15, 17.76s/it]
 46%|████▌     | 103866/225000 [219:58:32<524:27:24, 15.59s/it]
 46%|████▌     | 103867/225000 [219:58:45<497:26:53, 14.78s/it]
 46%|████▌     | 103868/225000 [219:58:57<472:06:47, 14.03s/it]
 46%|████▌     | 103869/225000 [219:59:05<409:27:49, 12.17s/it]
 46%|████▌     | 103870/225000 [219:59:26<493:31:26, 14.67s/it]
 46%|████▌     | 103871/225000 [219:59:34<428:50:43, 12.75s/it]
 46%|████▌     | 103872/225000 [219:59:39<351:24:09, 10.44s/it]
 46%|████▌     | 103873/225000 [219:59:47<331:22:49,  9.85s/it]
 46%|████▌     | 103874/225000 [220:00:06<417:39:24, 12.41s/it]
 46%|████▌     | 103875/225000 [220:00:15<387:55:14, 11.53s/it]
 46%|████▌     | 103876/225000 [220:00:28<400:52:30, 11.91s/it]
 46%|████▌     | 103877/225000 [220:00:48<486:54:52, 14.47s/it]
 46%|████▌     | 103878/225000 [220:01:03<490:54:30, 14.59s/it]
 46%|████▌     | 103879/225000 [220:01:12<430:07:16, 12.78s/it]
 46%|████▌     | 103880/225000 [220:01:33<512:26:38, 15.23s/it]
 46%|████▌     | 103881/225000 [220:01:39<416:10:46, 12.37s/it]
 46%|████▌     | 103882/225000 [220:01:44<350:08:16, 10.41s/it]
 46%|████▌     | 103883/225000 [220:02:02<426:25:51, 12.67s/it]
 46%|████▌     | 103884/225000 [220:02:13<404:55:07, 12.04s/it]
 46%|████▌     | 103885/225000 [220:02:20<358:49:03, 10.67s/it]
 46%|████▌     | 103886/225000 [220:02:41<457:23:38, 13.60s/it]
 46%|████▌     | 103887/225000 [220:02:55<466:26:06, 13.86s/it]
 46%|████▌     | 103888/225000 [220:03:15<521:28:46, 15.50s/it]
 46%|████▌     | 103889/225000 [220:03:27<490:41:08, 14.59s/it]
 46%|████▌     | 103890/225000 [220:03:44<515:04:58, 15.31s/it]
 46%|████▌     | 103891/225000 [220:03:55<468:17:29, 13.92s/it]
 46%|████▌     | 103892/225000 [220:04:05<428:07:17, 12.73s/it]
 46%|████▌     | 103893/225000 [220:04:12<375:15:33, 11.15s/it]
 46%|████▌     | 103894/225000 [220:04:25<389:13:16, 11.57s/it]
 46%|████▌     | 103895/225000 [220:04:45<476:23:52, 14.16s/it]
 46%|████▌     | 103896/225000 [220:04:58<470:38:55, 13.99s/it]
 46%|████▌     | 103897/225000 [220:05:05<396:18:36, 11.78s/it]
 46%|████▌     | 103898/225000 [220:05:18<405:59:16, 12.07s/it]
 46%|████▌     | 103899/225000 [220:05:30<403:55:08, 12.01s/it]
 46%|████▌     | 103900/225000 [220:05:48<465:55:43, 13.85s/it]
                                                               
{'loss': '0.9168', 'grad_norm': '1.223', 'learning_rate': '2.995e-05', 'epoch': '0.4618', 'num_input_tokens_seen': 369288640, 'train_runtime': '7.923e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 103900/225000 [220:05:48<465:55:43, 13.85s/it]
 46%|████▌     | 103901/225000 [220:05:55<398:51:14, 11.86s/it]
 46%|████▌     | 103902/225000 [220:06:02<346:48:02, 10.31s/it]
 46%|████▌     | 103903/225000 [220:06:13<360:24:44, 10.71s/it]
 46%|████▌     | 103904/225000 [220:06:27<391:40:24, 11.64s/it]
 46%|████▌     | 103905/225000 [220:06:50<501:59:35, 14.92s/it]
 46%|████▌     | 103906/225000 [220:07:03<480:06:45, 14.27s/it]
 46%|████▌     | 103907/225000 [220:07:17<482:03:11, 14.33s/it]
 46%|████▌     | 103908/225000 [220:07:37<544:04:36, 16.18s/it]
 46%|████▌     | 103909/225000 [220:07:42<426:28:35, 12.68s/it]
 46%|████▌     | 103910/225000 [220:07:57<445:41:58, 13.25s/it]
 46%|████▌     | 103911/225000 [220:08:07<416:33:41, 12.38s/it]
 46%|████▌     | 103912/225000 [220:08:20<420:30:12, 12.50s/it]
 46%|████▌     | 103913/225000 [220:08:35<444:07:01, 13.20s/it]
 46%|████▌     | 103914/225000 [220:08:55<514:22:31, 15.29s/it]
 46%|████▌     | 103915/225000 [220:09:04<455:54:01, 13.55s/it]
 46%|████▌     | 103916/225000 [220:09:21<488:57:55, 14.54s/it]
 46%|████▌     | 103917/225000 [220:09:30<435:27:30, 12.95s/it]
 46%|████▌     | 103918/225000 [220:09:37<375:27:06, 11.16s/it]
 46%|████▌     | 103919/225000 [220:09:46<346:02:09, 10.29s/it]
 46%|████▌     | 103920/225000 [220:09:49<276:58:49,  8.24s/it]
 46%|████▌     | 103921/225000 [220:10:07<378:53:12, 11.27s/it]
 46%|████▌     | 103922/225000 [220:10:21<399:44:03, 11.89s/it]
 46%|████▌     | 103923/225000 [220:10:29<366:02:05, 10.88s/it]
 46%|████▌     | 103924/225000 [220:10:42<381:36:55, 11.35s/it]
 46%|████▌     | 103925/225000 [220:10:52<373:59:14, 11.12s/it]
 46%|████▌     | 103926/225000 [220:11:09<429:47:16, 12.78s/it]
 46%|████▌     | 103927/225000 [220:11:15<363:43:14, 10.81s/it]
 46%|████▌     | 103928/225000 [220:11:27<371:54:25, 11.06s/it]
 46%|████▌     | 103929/225000 [220:11:43<422:51:33, 12.57s/it]
 46%|████▌     | 103930/225000 [220:11:50<372:45:32, 11.08s/it]
 46%|████▌     | 103931/225000 [220:12:05<411:51:54, 12.25s/it]
 46%|████▌     | 103932/225000 [220:12:11<342:23:54, 10.18s/it]
 46%|████▌     | 103933/225000 [220:12:26<390:07:27, 11.60s/it]
 46%|████▌     | 103934/225000 [220:12:37<386:30:39, 11.49s/it]
 46%|████▌     | 103935/225000 [220:12:50<401:25:31, 11.94s/it]
 46%|████▌     | 103936/225000 [220:13:07<455:51:15, 13.56s/it]
 46%|████▌     | 103937/225000 [220:13:16<404:24:48, 12.03s/it]
 46%|████▌     | 103938/225000 [220:13:35<475:41:27, 14.15s/it]
 46%|████▌     | 103939/225000 [220:13:46<444:11:48, 13.21s/it]
 46%|████▌     | 103940/225000 [220:13:59<445:44:27, 13.26s/it]
 46%|████▌     | 103941/225000 [220:14:11<430:03:27, 12.79s/it]
 46%|████▌     | 103942/225000 [220:14:26<453:45:12, 13.49s/it]
 46%|████▌     | 103943/225000 [220:14:40<459:00:31, 13.65s/it]
 46%|████▌     | 103944/225000 [220:14:47<389:22:16, 11.58s/it]
 46%|████▌     | 103945/225000 [220:15:04<449:41:12, 13.37s/it]
 46%|████▌     | 103946/225000 [220:15:23<499:42:26, 14.86s/it]
 46%|████▌     | 103947/225000 [220:15:28<405:44:13, 12.07s/it]
 46%|████▌     | 103948/225000 [220:15:42<420:52:54, 12.52s/it]
 46%|████▌     | 103949/225000 [220:15:49<369:10:24, 10.98s/it]
 46%|████▌     | 103950/225000 [220:15:57<340:35:00, 10.13s/it]
                                                               
{'loss': '0.8944', 'grad_norm': '1.642', 'learning_rate': '2.994e-05', 'epoch': '0.462', 'num_input_tokens_seen': 369583936, 'train_runtime': '7.93e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 103950/225000 [220:15:57<340:35:00, 10.13s/it]
 46%|████▌     | 103951/225000 [220:16:05<316:53:22,  9.42s/it]
 46%|████▌     | 103952/225000 [220:16:13<303:07:28,  9.02s/it]
 46%|████▌     | 103953/225000 [220:16:33<413:26:53, 12.30s/it]
 46%|████▌     | 103954/225000 [220:16:46<424:07:32, 12.61s/it]
 46%|████▌     | 103955/225000 [220:16:56<396:22:49, 11.79s/it]
 46%|████▌     | 103956/225000 [220:17:17<481:06:14, 14.31s/it]
 46%|████▌     | 103957/225000 [220:17:32<488:32:31, 14.53s/it]
 46%|████▌     | 103958/225000 [220:17:40<425:16:24, 12.65s/it]
 46%|████▌     | 103959/225000 [220:17:53<432:17:42, 12.86s/it]
 46%|████▌     | 103960/225000 [220:18:08<455:55:41, 13.56s/it]
 46%|████▌     | 103961/225000 [220:18:18<411:16:41, 12.23s/it]
 46%|████▌     | 103962/225000 [220:18:30<415:37:48, 12.36s/it]
 46%|████▌     | 103963/225000 [220:18:37<362:28:14, 10.78s/it]
 46%|████▌     | 103964/225000 [220:18:40<276:32:28,  8.23s/it]
 46%|████▌     | 103965/225000 [220:18:50<298:54:50,  8.89s/it]
 46%|████▌     | 103966/225000 [220:19:14<447:20:41, 13.31s/it]
 46%|████▌     | 103967/225000 [220:19:23<408:57:46, 12.16s/it]
 46%|████▌     | 103968/225000 [220:19:39<442:27:37, 13.16s/it]
 46%|████▌     | 103969/225000 [220:19:56<489:32:01, 14.56s/it]
 46%|████▌     | 103970/225000 [220:20:11<489:54:09, 14.57s/it]
 46%|████▌     | 103971/225000 [220:20:29<521:40:48, 15.52s/it]
 46%|████▌     | 103972/225000 [220:20:38<458:28:02, 13.64s/it]
 46%|████▌     | 103973/225000 [220:20:46<401:52:49, 11.95s/it]
 46%|████▌     | 103974/225000 [220:20:59<417:35:04, 12.42s/it]
 46%|████▌     | 103975/225000 [220:21:07<369:18:11, 10.99s/it]
 46%|████▌     | 103976/225000 [220:21:21<399:38:03, 11.89s/it]
 46%|████▌     | 103977/225000 [220:21:32<388:01:24, 11.54s/it]
 46%|████▌     | 103978/225000 [220:21:55<501:56:34, 14.93s/it]
 46%|████▌     | 103979/225000 [220:21:59<398:42:36, 11.86s/it]
 46%|████▌     | 103980/225000 [220:22:20<484:00:46, 14.40s/it]
 46%|████▌     | 103981/225000 [220:22:28<421:21:36, 12.53s/it]
 46%|████▌     | 103982/225000 [220:22:50<515:48:25, 15.34s/it]
 46%|████▌     | 103983/225000 [220:23:06<528:18:21, 15.72s/it]
 46%|████▌     | 103984/225000 [220:23:22<529:11:51, 15.74s/it]
 46%|████▌     | 103985/225000 [220:23:35<497:05:10, 14.79s/it]
 46%|████▌     | 103986/225000 [220:23:52<524:59:48, 15.62s/it]
 46%|████▌     | 103987/225000 [220:24:01<452:48:54, 13.47s/it]
 46%|████▌     | 103988/225000 [220:24:17<484:06:07, 14.40s/it]
 46%|████▌     | 103989/225000 [220:24:30<464:40:09, 13.82s/it]
 46%|████▌     | 103990/225000 [220:24:41<440:50:01, 13.11s/it]
 46%|████▌     | 103991/225000 [220:24:47<369:42:03, 11.00s/it]
 46%|████▌     | 103992/225000 [220:24:55<335:52:15,  9.99s/it]
 46%|████▌     | 103993/225000 [220:25:01<296:32:50,  8.82s/it]
 46%|████▌     | 103994/225000 [220:25:13<328:24:25,  9.77s/it]
 46%|████▌     | 103995/225000 [220:25:30<401:26:41, 11.94s/it]
 46%|████▌     | 103996/225000 [220:25:38<365:50:25, 10.88s/it]
 46%|████▌     | 103997/225000 [220:25:51<384:31:22, 11.44s/it]
 46%|████▌     | 103998/225000 [220:26:06<414:10:13, 12.32s/it]
 46%|████▌     | 103999/225000 [220:26:16<391:38:36, 11.65s/it]
 46%|████▌     | 104000/225000 [220:26:25<368:36:49, 10.97s/it]
                                                               
{'loss': '0.9167', 'grad_norm': '1.612', 'learning_rate': '2.992e-05', 'epoch': '0.4622', 'num_input_tokens_seen': 369880000, 'train_runtime': '7.936e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 104000/225000 [220:26:25<368:36:49, 10.97s/it]
 46%|████▌     | 104001/225000 [220:26:32<327:10:31,  9.73s/it]
 46%|████▌     | 104002/225000 [220:26:51<423:48:35, 12.61s/it]
 46%|████▌     | 104003/225000 [220:27:06<448:15:04, 13.34s/it]
 46%|████▌     | 104004/225000 [220:27:27<518:09:35, 15.42s/it]
 46%|████▌     | 104005/225000 [220:27:36<453:57:01, 13.51s/it]
 46%|████▌     | 104006/225000 [220:27:50<463:15:36, 13.78s/it]
 46%|████▌     | 104007/225000 [220:28:04<468:03:15, 13.93s/it]
 46%|████▌     | 104008/225000 [220:28:15<437:49:27, 13.03s/it]
 46%|████▌     | 104009/225000 [220:28:21<367:11:48, 10.93s/it]
 46%|████▌     | 104010/225000 [220:28:40<448:15:28, 13.34s/it]
 46%|████▌     | 104011/225000 [220:28:45<366:32:44, 10.91s/it]
 46%|████▌     | 104012/225000 [220:28:50<304:13:21,  9.05s/it]
 46%|████▌     | 104013/225000 [220:28:55<262:26:30,  7.81s/it]
 46%|████▌     | 104014/225000 [220:29:11<341:06:34, 10.15s/it]
 46%|████▌     | 104015/225000 [220:29:31<439:32:44, 13.08s/it]
 46%|████▌     | 104016/225000 [220:29:37<369:07:06, 10.98s/it]
 46%|████▌     | 104017/225000 [220:29:51<398:20:27, 11.85s/it]
 46%|████▌     | 104018/225000 [220:30:02<396:50:26, 11.81s/it]
 46%|████▌     | 104019/225000 [220:30:09<344:52:13, 10.26s/it]
 46%|████▌     | 104020/225000 [220:30:25<401:05:48, 11.94s/it]
 46%|████▌     | 104021/225000 [220:30:42<458:09:58, 13.63s/it]
 46%|████▌     | 104022/225000 [220:30:53<426:35:28, 12.69s/it]
 46%|████▌     | 104023/225000 [220:30:59<355:51:06, 10.59s/it]
 46%|████▌     | 104024/225000 [220:31:12<385:41:35, 11.48s/it]
 46%|████▌     | 104025/225000 [220:31:20<345:32:49, 10.28s/it]
 46%|████▌     | 104026/225000 [220:31:41<459:46:20, 13.68s/it]
 46%|████▌     | 104027/225000 [220:31:51<417:25:41, 12.42s/it]
 46%|████▌     | 104028/225000 [220:32:04<425:09:20, 12.65s/it]
 46%|████▌     | 104029/225000 [220:32:14<398:30:42, 11.86s/it]
 46%|████▌     | 104030/225000 [220:32:22<356:33:06, 10.61s/it]
 46%|████▌     | 104031/225000 [220:32:27<308:49:43,  9.19s/it]
 46%|████▌     | 104032/225000 [220:32:38<326:48:20,  9.73s/it]
 46%|████▌     | 104033/225000 [220:33:00<445:37:48, 13.26s/it]
 46%|████▌     | 104034/225000 [220:33:13<448:23:12, 13.34s/it]
 46%|████▌     | 104035/225000 [220:33:23<409:27:16, 12.19s/it]
 46%|████▌     | 104036/225000 [220:33:35<412:52:57, 12.29s/it]
 46%|████▌     | 104037/225000 [220:33:46<396:29:52, 11.80s/it]
 46%|████▌     | 104038/225000 [220:33:54<358:57:53, 10.68s/it]
 46%|████▌     | 104039/225000 [220:34:01<315:25:15,  9.39s/it]
 46%|████▌     | 104040/225000 [220:34:16<371:35:52, 11.06s/it]
 46%|████▌     | 104041/225000 [220:34:22<330:02:19,  9.82s/it]
 46%|████▌     | 104042/225000 [220:34:31<319:36:32,  9.51s/it]
 46%|████▌     | 104043/225000 [220:34:40<308:18:25,  9.18s/it]
 46%|████▌     | 104044/225000 [220:34:52<341:57:18, 10.18s/it]
 46%|████▌     | 104045/225000 [220:35:05<371:43:29, 11.06s/it]
 46%|████▌     | 104046/225000 [220:35:15<354:01:49, 10.54s/it]
 46%|████▌     | 104047/225000 [220:35:33<433:13:25, 12.89s/it]
 46%|████▌     | 104048/225000 [220:35:45<420:20:55, 12.51s/it]
 46%|████▌     | 104049/225000 [220:35:59<434:44:46, 12.94s/it]
 46%|████▌     | 104050/225000 [220:36:03<346:04:03, 10.30s/it]
                                                               
{'loss': '0.9017', 'grad_norm': '2.417', 'learning_rate': '2.99e-05', 'epoch': '0.4624', 'num_input_tokens_seen': 370164968, 'train_runtime': '7.942e+05', 'train_tokens_per_second': '466.1'}

 46%|████▌     | 104050/225000 [220:36:03<346:04:03, 10.30s/it]
 46%|████▌     | 104051/225000 [220:36:23<444:46:14, 13.24s/it]
 46%|████▌     | 104052/225000 [220:36:34<423:07:31, 12.59s/it]
 46%|████▌     | 104053/225000 [220:36:44<396:51:28, 11.81s/it]
 46%|████▌     | 104054/225000 [220:36:52<363:21:18, 10.82s/it]
 46%|████▌     | 104055/225000 [220:36:59<317:28:31,  9.45s/it]
 46%|████▌     | 104056/225000 [220:37:10<337:54:40, 10.06s/it]
 46%|████▌     | 104057/225000 [220:37:21<350:58:44, 10.45s/it]
 46%|████▌     | 104058/225000 [220:37:33<357:30:37, 10.64s/it]
 46%|████▌     | 104059/225000 [220:37:37<296:09:19,  8.82s/it]
 46%|████▌     | 104060/225000 [220:37:55<388:25:06, 11.56s/it]
 46%|████▌     | 104061/225000 [220:38:10<423:40:08, 12.61s/it]
 46%|████▌     | 104062/225000 [220:38:19<387:50:37, 11.55s/it]
 46%|████▋     | 104063/225000 [220:38:34<418:24:14, 12.45s/it]
 46%|████▋     | 104064/225000 [220:38:39<348:43:18, 10.38s/it]
 46%|████▋     | 104065/225000 [220:38:49<342:56:13, 10.21s/it]
 46%|████▋     | 104066/225000 [220:38:53<274:29:01,  8.17s/it]
 46%|████▋     | 104067/225000 [220:39:18<448:51:30, 13.36s/it]
 46%|████▋     | 104068/225000 [220:39:24<376:38:19, 11.21s/it]
 46%|████▋     | 104069/225000 [220:39:38<405:25:25, 12.07s/it]
 46%|████▋     | 104070/225000 [220:39:52<418:57:56, 12.47s/it]
 46%|████▋     | 104071/225000 [220:40:09<463:56:49, 13.81s/it]
 46%|████▋     | 104072/225000 [220:40:17<410:13:59, 12.21s/it]
 46%|████▋     | 104073/225000 [220:40:24<355:23:27, 10.58s/it]
 46%|████▋     | 104074/225000 [220:40:33<338:35:35, 10.08s/it]
 46%|████▋     | 104075/225000 [220:40:51<423:03:16, 12.59s/it]
 46%|████▋     | 104076/225000 [220:41:03<410:34:05, 12.22s/it]
 46%|████▋     | 104077/225000 [220:41:09<355:12:53, 10.58s/it]
 46%|████▋     | 104078/225000 [220:41:18<332:37:16,  9.90s/it]
 46%|████▋     | 104079/225000 [220:41:27<324:11:36,  9.65s/it]
 46%|████▋     | 104080/225000 [220:41:40<358:14:11, 10.67s/it]
 46%|████▋     | 104081/225000 [220:41:53<383:56:34, 11.43s/it]
 46%|████▋     | 104082/225000 [220:42:00<339:09:30, 10.10s/it]
 46%|████▋     | 104083/225000 [220:42:09<331:07:10,  9.86s/it]
 46%|████▋     | 104084/225000 [220:42:23<374:52:46, 11.16s/it]
 46%|████▋     | 104085/225000 [220:42:31<339:18:48, 10.10s/it]
 46%|████▋     | 104086/225000 [220:42:44<365:45:41, 10.89s/it]
 46%|████▋     | 104087/225000 [220:42:57<393:30:00, 11.72s/it]
 46%|████▋     | 104088/225000 [220:43:14<437:05:18, 13.01s/it]
 46%|████▋     | 104089/225000 [220:43:20<369:17:10, 11.00s/it]
 46%|████▋     | 104090/225000 [220:43:37<432:04:56, 12.86s/it]
 46%|████▋     | 104091/225000 [220:43:48<409:22:57, 12.19s/it]
 46%|████▋     | 104092/225000 [220:43:56<371:04:13, 11.05s/it]
 46%|████▋     | 104093/225000 [220:44:10<401:29:17, 11.95s/it]
 46%|████▋     | 104094/225000 [220:44:26<439:57:33, 13.10s/it]
 46%|████▋     | 104095/225000 [220:44:39<439:48:58, 13.10s/it]
 46%|████▋     | 104096/225000 [220:44:52<441:49:23, 13.16s/it]
 46%|████▋     | 104097/225000 [220:45:03<418:06:36, 12.45s/it]
 46%|████▋     | 104098/225000 [220:45:09<354:03:02, 10.54s/it]
 46%|████▋     | 104099/225000 [220:45:21<368:05:30, 10.96s/it]
 46%|████▋     | 104100/225000 [220:45:30<348:11:05, 10.37s/it]
                                                               
{'loss': '0.9378', 'grad_norm': '1.727', 'learning_rate': '2.988e-05', 'epoch': '0.4627', 'num_input_tokens_seen': 370426600, 'train_runtime': '7.947e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104100/225000 [220:45:30<348:11:05, 10.37s/it]
 46%|████▋     | 104101/225000 [220:45:38<327:40:52,  9.76s/it]
 46%|████▋     | 104102/225000 [220:45:54<386:37:44, 11.51s/it]
 46%|████▋     | 104103/225000 [220:46:04<370:55:11, 11.05s/it]
 46%|████▋     | 104104/225000 [220:46:14<359:04:30, 10.69s/it]
 46%|████▋     | 104105/225000 [220:46:27<387:44:36, 11.55s/it]
 46%|████▋     | 104106/225000 [220:46:46<454:15:33, 13.53s/it]
 46%|████▋     | 104107/225000 [220:46:55<417:09:37, 12.42s/it]
 46%|████▋     | 104108/225000 [220:47:01<349:36:47, 10.41s/it]
 46%|████▋     | 104109/225000 [220:47:19<428:20:32, 12.76s/it]
 46%|████▋     | 104110/225000 [220:47:26<364:51:00, 10.86s/it]
 46%|████▋     | 104111/225000 [220:47:36<362:02:35, 10.78s/it]
 46%|████▋     | 104112/225000 [220:47:50<389:35:05, 11.60s/it]
 46%|████▋     | 104113/225000 [220:48:11<489:43:58, 14.58s/it]
 46%|████▋     | 104114/225000 [220:48:28<505:31:15, 15.05s/it]
 46%|████▋     | 104115/225000 [220:48:41<484:56:52, 14.44s/it]
 46%|████▋     | 104116/225000 [220:48:53<460:10:04, 13.70s/it]
 46%|████▋     | 104117/225000 [220:49:02<413:25:38, 12.31s/it]
 46%|████▋     | 104118/225000 [220:49:08<354:35:36, 10.56s/it]
 46%|████▋     | 104119/225000 [220:49:11<277:57:31,  8.28s/it]
 46%|████▋     | 104120/225000 [220:49:20<283:23:45,  8.44s/it]
 46%|████▋     | 104121/225000 [220:49:33<327:03:11,  9.74s/it]
 46%|████▋     | 104122/225000 [220:49:36<261:40:46,  7.79s/it]
 46%|████▋     | 104123/225000 [220:49:48<302:24:24,  9.01s/it]
 46%|████▋     | 104124/225000 [220:50:04<371:14:48, 11.06s/it]
 46%|████▋     | 104125/225000 [220:50:23<456:39:49, 13.60s/it]
 46%|████▋     | 104126/225000 [220:50:39<481:03:17, 14.33s/it]
 46%|████▋     | 104127/225000 [220:50:46<408:55:10, 12.18s/it]
 46%|████▋     | 104128/225000 [220:51:01<435:42:39, 12.98s/it]
 46%|████▋     | 104129/225000 [220:51:08<377:15:35, 11.24s/it]
 46%|████▋     | 104130/225000 [220:51:21<390:15:51, 11.62s/it]
 46%|████▋     | 104131/225000 [220:51:34<405:44:14, 12.08s/it]
 46%|████▋     | 104132/225000 [220:51:45<393:28:08, 11.72s/it]
 46%|████▋     | 104133/225000 [220:51:58<407:49:26, 12.15s/it]
 46%|████▋     | 104134/225000 [220:52:08<384:18:58, 11.45s/it]
 46%|████▋     | 104135/225000 [220:52:13<325:22:24,  9.69s/it]
 46%|████▋     | 104136/225000 [220:52:21<304:21:46,  9.07s/it]
 46%|████▋     | 104137/225000 [220:52:29<295:44:50,  8.81s/it]
 46%|████▋     | 104138/225000 [220:52:40<315:26:36,  9.40s/it]
 46%|████▋     | 104139/225000 [220:52:53<351:05:41, 10.46s/it]
 46%|████▋     | 104140/225000 [220:53:11<425:03:16, 12.66s/it]
 46%|████▋     | 104141/225000 [220:53:24<435:48:47, 12.98s/it]
 46%|████▋     | 104142/225000 [220:53:34<404:05:10, 12.04s/it]
 46%|████▋     | 104143/225000 [220:53:44<380:30:00, 11.33s/it]
 46%|████▋     | 104144/225000 [220:53:52<344:09:38, 10.25s/it]
 46%|████▋     | 104145/225000 [220:54:01<335:16:04,  9.99s/it]
 46%|████▋     | 104146/225000 [220:54:25<478:25:14, 14.25s/it]
 46%|████▋     | 104147/225000 [220:54:35<431:51:46, 12.86s/it]
 46%|████▋     | 104148/225000 [220:54:47<424:25:58, 12.64s/it]
 46%|████▋     | 104149/225000 [220:54:58<404:28:37, 12.05s/it]
 46%|████▋     | 104150/225000 [220:55:14<445:36:52, 13.27s/it]
                                                               
{'loss': '0.9149', 'grad_norm': '1.426', 'learning_rate': '2.986e-05', 'epoch': '0.4629', 'num_input_tokens_seen': 370700328, 'train_runtime': '7.953e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104150/225000 [220:55:14<445:36:52, 13.27s/it]
 46%|████▋     | 104151/225000 [220:55:26<436:35:05, 13.01s/it]
 46%|████▋     | 104152/225000 [220:55:38<425:35:34, 12.68s/it]
 46%|████▋     | 104153/225000 [220:55:49<407:38:02, 12.14s/it]
 46%|████▋     | 104154/225000 [220:55:56<353:56:14, 10.54s/it]
 46%|████▋     | 104155/225000 [220:55:58<270:20:08,  8.05s/it]
 46%|████▋     | 104156/225000 [220:56:16<373:53:11, 11.14s/it]
 46%|████▋     | 104157/225000 [220:56:31<409:46:38, 12.21s/it]
 46%|████▋     | 104158/225000 [220:56:40<375:04:08, 11.17s/it]
 46%|████▋     | 104159/225000 [220:56:47<333:17:09,  9.93s/it]
 46%|████▋     | 104160/225000 [220:56:56<324:05:01,  9.65s/it]
 46%|████▋     | 104161/225000 [220:57:16<427:32:18, 12.74s/it]
 46%|████▋     | 104162/225000 [220:57:25<387:46:47, 11.55s/it]
 46%|████▋     | 104163/225000 [220:57:31<331:16:01,  9.87s/it]
 46%|████▋     | 104164/225000 [220:57:42<348:39:10, 10.39s/it]
 46%|████▋     | 104165/225000 [220:57:57<397:22:08, 11.84s/it]
 46%|████▋     | 104166/225000 [220:58:09<398:46:20, 11.88s/it]
 46%|████▋     | 104167/225000 [220:58:23<413:50:52, 12.33s/it]
 46%|████▋     | 104168/225000 [220:58:33<395:34:39, 11.79s/it]
 46%|████▋     | 104169/225000 [220:58:37<317:28:17,  9.46s/it]
 46%|████▋     | 104170/225000 [220:58:49<335:21:08,  9.99s/it]
 46%|████▋     | 104171/225000 [220:59:00<355:01:07, 10.58s/it]
 46%|████▋     | 104172/225000 [220:59:13<373:21:08, 11.12s/it]
 46%|████▋     | 104173/225000 [220:59:30<428:49:35, 12.78s/it]
 46%|████▋     | 104174/225000 [220:59:36<367:21:48, 10.95s/it]
 46%|████▋     | 104175/225000 [220:59:42<317:21:24,  9.46s/it]
 46%|████▋     | 104176/225000 [220:59:51<311:23:53,  9.28s/it]
 46%|████▋     | 104177/225000 [221:00:11<417:45:26, 12.45s/it]
 46%|████▋     | 104178/225000 [221:00:27<458:49:50, 13.67s/it]
 46%|████▋     | 104179/225000 [221:00:33<375:45:51, 11.20s/it]
 46%|████▋     | 104180/225000 [221:00:46<398:44:45, 11.88s/it]
 46%|████▋     | 104181/225000 [221:00:57<387:45:42, 11.55s/it]
 46%|████▋     | 104182/225000 [221:01:15<454:49:00, 13.55s/it]
 46%|████▋     | 104183/225000 [221:01:26<424:18:00, 12.64s/it]
 46%|████▋     | 104184/225000 [221:01:42<459:12:24, 13.68s/it]
 46%|████▋     | 104185/225000 [221:01:48<379:48:42, 11.32s/it]
 46%|████▋     | 104186/225000 [221:02:02<411:30:34, 12.26s/it]
 46%|████▋     | 104187/225000 [221:02:12<384:07:03, 11.45s/it]
 46%|████▋     | 104188/225000 [221:02:24<393:53:37, 11.74s/it]
 46%|████▋     | 104189/225000 [221:02:30<336:36:42, 10.03s/it]
 46%|████▋     | 104190/225000 [221:02:46<389:31:45, 11.61s/it]
 46%|████▋     | 104191/225000 [221:02:58<398:13:07, 11.87s/it]
 46%|████▋     | 104192/225000 [221:03:13<433:17:21, 12.91s/it]
 46%|████▋     | 104193/225000 [221:03:30<466:16:48, 13.89s/it]
 46%|████▋     | 104194/225000 [221:03:35<381:19:27, 11.36s/it]
 46%|████▋     | 104195/225000 [221:03:47<387:43:17, 11.55s/it]
 46%|████▋     | 104196/225000 [221:04:03<437:08:33, 13.03s/it]
 46%|████▋     | 104197/225000 [221:04:11<384:13:55, 11.45s/it]
 46%|████▋     | 104198/225000 [221:04:21<366:05:32, 10.91s/it]
 46%|████▋     | 104199/225000 [221:04:36<413:24:02, 12.32s/it]
 46%|████▋     | 104200/225000 [221:04:50<423:33:44, 12.62s/it]
                                                               
{'loss': '0.9002', 'grad_norm': '1.53', 'learning_rate': '2.985e-05', 'epoch': '0.4631', 'num_input_tokens_seen': 370965552, 'train_runtime': '7.959e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104200/225000 [221:04:50<423:33:44, 12.62s/it]
 46%|████▋     | 104201/225000 [221:04:59<392:22:26, 11.69s/it]
 46%|████▋     | 104202/225000 [221:05:16<438:31:36, 13.07s/it]
 46%|████▋     | 104203/225000 [221:05:29<440:15:52, 13.12s/it]
 46%|████▋     | 104204/225000 [221:05:40<419:55:27, 12.51s/it]
 46%|████▋     | 104205/225000 [221:05:52<410:41:55, 12.24s/it]
 46%|████▋     | 104206/225000 [221:06:03<403:32:20, 12.03s/it]
 46%|████▋     | 104207/225000 [221:06:16<412:24:47, 12.29s/it]
 46%|████▋     | 104208/225000 [221:06:26<389:04:34, 11.60s/it]
 46%|████▋     | 104209/225000 [221:06:40<415:44:12, 12.39s/it]
 46%|████▋     | 104210/225000 [221:06:46<351:17:33, 10.47s/it]
 46%|████▋     | 104211/225000 [221:06:56<342:49:29, 10.22s/it]
 46%|████▋     | 104212/225000 [221:07:12<397:59:45, 11.86s/it]
 46%|████▋     | 104213/225000 [221:07:31<478:46:01, 14.27s/it]
 46%|████▋     | 104214/225000 [221:07:46<484:24:20, 14.44s/it]
 46%|████▋     | 104215/225000 [221:08:03<506:38:26, 15.10s/it]
 46%|████▋     | 104216/225000 [221:08:24<563:41:51, 16.80s/it]
 46%|████▋     | 104217/225000 [221:08:31<473:35:07, 14.12s/it]
 46%|████▋     | 104218/225000 [221:08:40<418:41:28, 12.48s/it]
 46%|████▋     | 104219/225000 [221:08:46<354:01:13, 10.55s/it]
 46%|████▋     | 104220/225000 [221:09:01<394:45:36, 11.77s/it]
 46%|████▋     | 104221/225000 [221:09:07<343:11:25, 10.23s/it]
 46%|████▋     | 104222/225000 [221:09:26<428:34:56, 12.77s/it]
 46%|████▋     | 104223/225000 [221:09:45<487:00:53, 14.52s/it]
 46%|████▋     | 104224/225000 [221:09:52<418:11:43, 12.47s/it]
 46%|████▋     | 104225/225000 [221:10:13<495:33:37, 14.77s/it]
 46%|████▋     | 104226/225000 [221:10:29<514:57:51, 15.35s/it]
 46%|████▋     | 104227/225000 [221:10:43<501:55:40, 14.96s/it]
 46%|████▋     | 104228/225000 [221:10:54<456:09:36, 13.60s/it]
 46%|████▋     | 104229/225000 [221:10:59<374:14:49, 11.16s/it]
 46%|████▋     | 104230/225000 [221:11:09<359:49:23, 10.73s/it]
 46%|████▋     | 104231/225000 [221:11:20<362:38:15, 10.81s/it]
 46%|████▋     | 104232/225000 [221:11:24<293:02:05,  8.74s/it]
 46%|████▋     | 104233/225000 [221:11:32<292:16:25,  8.71s/it]
 46%|████▋     | 104234/225000 [221:11:42<296:20:19,  8.83s/it]
 46%|████▋     | 104235/225000 [221:11:55<342:22:13, 10.21s/it]
 46%|████▋     | 104236/225000 [221:12:04<334:10:59,  9.96s/it]
 46%|████▋     | 104237/225000 [221:12:18<374:58:46, 11.18s/it]
 46%|████▋     | 104238/225000 [221:12:27<346:09:55, 10.32s/it]
 46%|████▋     | 104239/225000 [221:12:38<353:11:08, 10.53s/it]
 46%|████▋     | 104240/225000 [221:12:48<345:27:23, 10.30s/it]
 46%|████▋     | 104241/225000 [221:12:57<340:46:35, 10.16s/it]
 46%|████▋     | 104242/225000 [221:13:16<421:38:11, 12.57s/it]
 46%|████▋     | 104243/225000 [221:13:26<400:48:24, 11.95s/it]
 46%|████▋     | 104244/225000 [221:13:42<439:22:54, 13.10s/it]
 46%|████▋     | 104245/225000 [221:13:49<376:36:42, 11.23s/it]
 46%|████▋     | 104246/225000 [221:14:03<409:40:33, 12.21s/it]
 46%|████▋     | 104247/225000 [221:14:09<348:58:15, 10.40s/it]
 46%|████▋     | 104248/225000 [221:14:26<412:27:09, 12.30s/it]
 46%|████▋     | 104249/225000 [221:14:34<373:00:38, 11.12s/it]
 46%|████▋     | 104250/225000 [221:14:50<414:39:55, 12.36s/it]
                                                               
{'loss': '0.9232', 'grad_norm': '1.435', 'learning_rate': '2.983e-05', 'epoch': '0.4633', 'num_input_tokens_seen': 371240528, 'train_runtime': '7.965e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104250/225000 [221:14:50<414:39:55, 12.36s/it]
 46%|████▋     | 104251/225000 [221:15:04<429:44:46, 12.81s/it]
 46%|████▋     | 104252/225000 [221:15:09<350:41:43, 10.46s/it]
 46%|████▋     | 104253/225000 [221:15:24<399:15:33, 11.90s/it]
 46%|████▋     | 104254/225000 [221:15:30<340:22:34, 10.15s/it]
 46%|████▋     | 104255/225000 [221:15:43<370:28:49, 11.05s/it]
 46%|████▋     | 104256/225000 [221:15:58<410:10:20, 12.23s/it]
 46%|████▋     | 104257/225000 [221:16:08<388:54:52, 11.60s/it]
 46%|████▋     | 104258/225000 [221:16:23<419:11:24, 12.50s/it]
 46%|████▋     | 104259/225000 [221:16:36<424:13:10, 12.65s/it]
 46%|████▋     | 104260/225000 [221:16:43<365:37:10, 10.90s/it]
 46%|████▋     | 104261/225000 [221:16:48<309:00:06,  9.21s/it]
 46%|████▋     | 104262/225000 [221:17:03<363:58:08, 10.85s/it]
 46%|████▋     | 104263/225000 [221:17:15<385:00:04, 11.48s/it]
 46%|████▋     | 104264/225000 [221:17:23<349:45:10, 10.43s/it]
 46%|████▋     | 104265/225000 [221:17:39<406:13:57, 12.11s/it]
 46%|████▋     | 104266/225000 [221:17:52<412:27:35, 12.30s/it]
 46%|████▋     | 104267/225000 [221:17:59<356:25:51, 10.63s/it]
 46%|████▋     | 104268/225000 [221:18:14<401:07:53, 11.96s/it]
 46%|████▋     | 104269/225000 [221:18:28<418:16:34, 12.47s/it]
 46%|████▋     | 104270/225000 [221:18:43<447:53:20, 13.36s/it]
 46%|████▋     | 104271/225000 [221:18:50<382:50:46, 11.42s/it]
 46%|████▋     | 104272/225000 [221:19:00<368:45:12, 11.00s/it]
 46%|████▋     | 104273/225000 [221:19:06<313:42:05,  9.35s/it]
 46%|████▋     | 104274/225000 [221:19:12<289:37:20,  8.64s/it]
 46%|████▋     | 104275/225000 [221:19:28<355:19:05, 10.60s/it]
 46%|████▋     | 104276/225000 [221:19:46<428:27:56, 12.78s/it]
 46%|████▋     | 104277/225000 [221:19:56<401:18:47, 11.97s/it]
 46%|████▋     | 104278/225000 [221:20:07<399:44:28, 11.92s/it]
 46%|████▋     | 104279/225000 [221:20:20<403:28:24, 12.03s/it]
 46%|████▋     | 104280/225000 [221:20:26<344:56:17, 10.29s/it]
 46%|████▋     | 104281/225000 [221:20:37<355:04:50, 10.59s/it]
 46%|████▋     | 104282/225000 [221:20:46<339:22:37, 10.12s/it]
 46%|████▋     | 104283/225000 [221:20:58<358:59:52, 10.71s/it]
 46%|████▋     | 104284/225000 [221:21:08<352:16:16, 10.51s/it]
 46%|████▋     | 104285/225000 [221:21:20<359:48:19, 10.73s/it]
 46%|████▋     | 104286/225000 [221:21:31<364:03:25, 10.86s/it]
 46%|████▋     | 104287/225000 [221:21:43<377:01:11, 11.24s/it]
 46%|████▋     | 104288/225000 [221:21:51<349:01:58, 10.41s/it]
 46%|████▋     | 104289/225000 [221:22:01<337:51:40, 10.08s/it]
 46%|████▋     | 104290/225000 [221:22:24<473:07:48, 14.11s/it]
 46%|████▋     | 104291/225000 [221:22:39<484:47:50, 14.46s/it]
 46%|████▋     | 104292/225000 [221:22:52<461:44:41, 13.77s/it]
 46%|████▋     | 104293/225000 [221:23:05<455:28:49, 13.58s/it]
 46%|████▋     | 104294/225000 [221:23:18<449:58:37, 13.42s/it]
 46%|████▋     | 104295/225000 [221:23:25<388:44:03, 11.59s/it]
 46%|████▋     | 104296/225000 [221:23:39<412:39:23, 12.31s/it]
 46%|████▋     | 104297/225000 [221:23:46<355:37:14, 10.61s/it]
 46%|████▋     | 104298/225000 [221:23:54<331:50:26,  9.90s/it]
 46%|████▋     | 104299/225000 [221:24:01<301:59:42,  9.01s/it]
 46%|████▋     | 104300/225000 [221:24:16<360:39:40, 10.76s/it]
                                                               
{'loss': '0.9151', 'grad_norm': '1.414', 'learning_rate': '2.981e-05', 'epoch': '0.4636', 'num_input_tokens_seen': 371498288, 'train_runtime': '7.971e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104300/225000 [221:24:16<360:39:40, 10.76s/it]
 46%|████▋     | 104301/225000 [221:24:27<363:44:43, 10.85s/it]
 46%|████▋     | 104302/225000 [221:24:35<332:45:56,  9.93s/it]
 46%|████▋     | 104303/225000 [221:24:44<325:50:21,  9.72s/it]
 46%|████▋     | 104304/225000 [221:24:56<351:30:50, 10.48s/it]
 46%|████▋     | 104305/225000 [221:25:07<351:28:17, 10.48s/it]
 46%|████▋     | 104306/225000 [221:25:23<407:31:31, 12.16s/it]
 46%|████▋     | 104307/225000 [221:25:30<362:36:02, 10.82s/it]
 46%|████▋     | 104308/225000 [221:25:39<343:45:05, 10.25s/it]
 46%|████▋     | 104309/225000 [221:25:45<300:37:50,  8.97s/it]
 46%|████▋     | 104310/225000 [221:25:52<278:17:13,  8.30s/it]
 46%|████▋     | 104311/225000 [221:26:05<324:45:12,  9.69s/it]
 46%|████▋     | 104312/225000 [221:26:19<370:58:55, 11.07s/it]
 46%|████▋     | 104313/225000 [221:26:27<335:10:27, 10.00s/it]
 46%|████▋     | 104314/225000 [221:26:45<422:20:59, 12.60s/it]
 46%|████▋     | 104315/225000 [221:26:59<428:01:22, 12.77s/it]
 46%|████▋     | 104316/225000 [221:27:11<425:50:05, 12.70s/it]
 46%|████▋     | 104317/225000 [221:27:18<366:36:46, 10.94s/it]
 46%|████▋     | 104318/225000 [221:27:25<328:02:43,  9.79s/it]
 46%|████▋     | 104319/225000 [221:27:36<342:22:20, 10.21s/it]
 46%|████▋     | 104320/225000 [221:27:49<368:03:42, 10.98s/it]
 46%|████▋     | 104321/225000 [221:27:57<342:26:27, 10.22s/it]
 46%|████▋     | 104322/225000 [221:28:06<326:59:05,  9.75s/it]
 46%|████▋     | 104323/225000 [221:28:16<333:06:00,  9.94s/it]
 46%|████▋     | 104324/225000 [221:28:27<336:50:31, 10.05s/it]
 46%|████▋     | 104325/225000 [221:28:38<349:52:56, 10.44s/it]
 46%|████▋     | 104326/225000 [221:28:57<434:37:12, 12.97s/it]
 46%|████▋     | 104327/225000 [221:29:07<409:54:32, 12.23s/it]
 46%|████▋     | 104328/225000 [221:29:20<415:39:35, 12.40s/it]
 46%|████▋     | 104329/225000 [221:29:29<379:06:24, 11.31s/it]
 46%|████▋     | 104330/225000 [221:29:35<325:57:04,  9.72s/it]
 46%|████▋     | 104331/225000 [221:29:49<366:25:53, 10.93s/it]
 46%|████▋     | 104332/225000 [221:30:00<365:29:09, 10.90s/it]
 46%|████▋     | 104333/225000 [221:30:14<404:38:39, 12.07s/it]
 46%|████▋     | 104334/225000 [221:30:28<422:56:05, 12.62s/it]
 46%|████▋     | 104335/225000 [221:30:42<429:41:04, 12.82s/it]
 46%|████▋     | 104336/225000 [221:30:48<363:25:06, 10.84s/it]
 46%|████▋     | 104337/225000 [221:31:02<392:23:01, 11.71s/it]
 46%|████▋     | 104338/225000 [221:31:20<463:41:13, 13.83s/it]
 46%|████▋     | 104339/225000 [221:31:33<451:51:00, 13.48s/it]
 46%|████▋     | 104340/225000 [221:31:51<495:11:43, 14.77s/it]
 46%|████▋     | 104341/225000 [221:32:02<462:30:03, 13.80s/it]
 46%|████▋     | 104342/225000 [221:32:10<405:05:14, 12.09s/it]
 46%|████▋     | 104343/225000 [221:32:16<342:14:42, 10.21s/it]
 46%|████▋     | 104344/225000 [221:32:28<358:17:53, 10.69s/it]
 46%|████▋     | 104345/225000 [221:32:44<408:34:47, 12.19s/it]
 46%|████▋     | 104346/225000 [221:32:56<408:27:15, 12.19s/it]
 46%|████▋     | 104347/225000 [221:33:04<369:34:59, 11.03s/it]
 46%|████▋     | 104348/225000 [221:33:21<422:51:18, 12.62s/it]
 46%|████▋     | 104349/225000 [221:33:27<356:12:14, 10.63s/it]
 46%|████▋     | 104350/225000 [221:33:35<334:29:17,  9.98s/it]
                                                               
{'loss': '0.9139', 'grad_norm': '2.255', 'learning_rate': '2.979e-05', 'epoch': '0.4638', 'num_input_tokens_seen': 371760040, 'train_runtime': '7.976e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104350/225000 [221:33:35<334:29:17,  9.98s/it]
 46%|████▋     | 104351/225000 [221:33:48<362:23:09, 10.81s/it]
 46%|████▋     | 104352/225000 [221:34:05<431:06:44, 12.86s/it]
 46%|████▋     | 104353/225000 [221:34:22<472:57:37, 14.11s/it]
 46%|████▋     | 104354/225000 [221:34:33<435:43:01, 13.00s/it]
 46%|████▋     | 104355/225000 [221:34:41<382:06:52, 11.40s/it]
 46%|████▋     | 104356/225000 [221:34:44<298:56:33,  8.92s/it]
 46%|████▋     | 104357/225000 [221:34:51<286:47:31,  8.56s/it]
 46%|████▋     | 104358/225000 [221:35:01<297:53:45,  8.89s/it]
 46%|████▋     | 104359/225000 [221:35:18<377:13:26, 11.26s/it]
 46%|████▋     | 104360/225000 [221:35:40<482:15:54, 14.39s/it]
 46%|████▋     | 104361/225000 [221:35:56<497:56:49, 14.86s/it]
 46%|████▋     | 104362/225000 [221:36:01<408:31:40, 12.19s/it]
 46%|████▋     | 104363/225000 [221:36:15<416:55:49, 12.44s/it]
 46%|████▋     | 104364/225000 [221:36:33<475:52:18, 14.20s/it]
 46%|████▋     | 104365/225000 [221:36:47<470:44:48, 14.05s/it]
 46%|████▋     | 104366/225000 [221:37:06<524:08:32, 15.64s/it]
 46%|████▋     | 104367/225000 [221:37:28<587:58:22, 17.55s/it]
 46%|████▋     | 104368/225000 [221:37:40<532:41:58, 15.90s/it]
 46%|████▋     | 104369/225000 [221:37:47<445:24:17, 13.29s/it]
 46%|████▋     | 104370/225000 [221:38:04<486:16:15, 14.51s/it]
 46%|████▋     | 104371/225000 [221:38:15<444:23:32, 13.26s/it]
 46%|████▋     | 104372/225000 [221:38:20<366:47:14, 10.95s/it]
 46%|████▋     | 104373/225000 [221:38:26<315:09:29,  9.41s/it]
 46%|████▋     | 104374/225000 [221:38:32<277:05:43,  8.27s/it]
 46%|████▋     | 104375/225000 [221:38:39<265:59:25,  7.94s/it]
 46%|████▋     | 104376/225000 [221:38:52<318:19:23,  9.50s/it]
 46%|████▋     | 104377/225000 [221:39:06<367:04:14, 10.96s/it]
 46%|████▋     | 104378/225000 [221:39:13<327:45:14,  9.78s/it]
 46%|████▋     | 104379/225000 [221:39:32<411:12:44, 12.27s/it]
 46%|████▋     | 104380/225000 [221:39:41<379:42:47, 11.33s/it]
 46%|████▋     | 104381/225000 [221:40:01<469:42:00, 14.02s/it]
 46%|████▋     | 104382/225000 [221:40:16<478:49:23, 14.29s/it]
 46%|████▋     | 104383/225000 [221:40:21<391:05:32, 11.67s/it]
 46%|████▋     | 104384/225000 [221:40:32<376:31:45, 11.24s/it]
 46%|████▋     | 104385/225000 [221:40:45<394:12:22, 11.77s/it]
 46%|████▋     | 104386/225000 [221:40:59<416:32:32, 12.43s/it]
 46%|████▋     | 104387/225000 [221:41:17<479:55:25, 14.32s/it]
 46%|████▋     | 104388/225000 [221:41:24<397:27:07, 11.86s/it]
 46%|████▋     | 104389/225000 [221:41:37<411:13:06, 12.27s/it]
 46%|████▋     | 104390/225000 [221:41:50<416:23:30, 12.43s/it]
 46%|████▋     | 104391/225000 [221:41:55<344:58:16, 10.30s/it]
 46%|████▋     | 104392/225000 [221:42:10<395:53:24, 11.82s/it]
 46%|████▋     | 104393/225000 [221:42:20<370:12:04, 11.05s/it]
 46%|████▋     | 104394/225000 [221:42:29<352:32:33, 10.52s/it]
 46%|████▋     | 104395/225000 [221:42:46<423:08:01, 12.63s/it]
 46%|████▋     | 104396/225000 [221:42:55<385:58:12, 11.52s/it]
 46%|████▋     | 104397/225000 [221:43:02<339:11:59, 10.13s/it]
 46%|████▋     | 104398/225000 [221:43:20<415:22:47, 12.40s/it]
 46%|████▋     | 104399/225000 [221:43:25<346:40:32, 10.35s/it]
 46%|████▋     | 104400/225000 [221:43:39<374:36:33, 11.18s/it]
                                                               
{'loss': '0.9203', 'grad_norm': '11.37', 'learning_rate': '2.977e-05', 'epoch': '0.464', 'num_input_tokens_seen': 372026216, 'train_runtime': '7.982e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104400/225000 [221:43:39<374:36:33, 11.18s/it]
 46%|████▋     | 104401/225000 [221:43:47<345:12:16, 10.30s/it]
 46%|████▋     | 104402/225000 [221:43:58<356:16:31, 10.64s/it]
 46%|████▋     | 104403/225000 [221:44:05<321:59:35,  9.61s/it]
 46%|████▋     | 104404/225000 [221:44:19<361:06:02, 10.78s/it]
 46%|████▋     | 104405/225000 [221:44:28<338:41:00, 10.11s/it]
 46%|████▋     | 104406/225000 [221:44:34<303:28:34,  9.06s/it]
 46%|████▋     | 104407/225000 [221:44:48<347:12:40, 10.37s/it]
 46%|████▋     | 104408/225000 [221:45:00<367:34:12, 10.97s/it]
 46%|████▋     | 104409/225000 [221:45:09<346:57:13, 10.36s/it]
 46%|████▋     | 104410/225000 [221:45:35<501:22:52, 14.97s/it]
 46%|████▋     | 104411/225000 [221:45:43<434:05:39, 12.96s/it]
 46%|████▋     | 104412/225000 [221:45:56<431:46:51, 12.89s/it]
 46%|████▋     | 104413/225000 [221:46:11<460:10:17, 13.74s/it]
 46%|████▋     | 104414/225000 [221:46:23<440:33:29, 13.15s/it]
 46%|████▋     | 104415/225000 [221:46:36<439:54:54, 13.13s/it]
 46%|████▋     | 104416/225000 [221:46:44<388:16:16, 11.59s/it]
 46%|████▋     | 104417/225000 [221:47:00<434:12:32, 12.96s/it]
 46%|████▋     | 104418/225000 [221:47:14<436:58:20, 13.05s/it]
 46%|████▋     | 104419/225000 [221:47:26<429:39:48, 12.83s/it]
 46%|████▋     | 104420/225000 [221:47:37<414:25:05, 12.37s/it]
 46%|████▋     | 104421/225000 [221:47:43<346:10:32, 10.34s/it]
 46%|████▋     | 104422/225000 [221:47:54<358:48:56, 10.71s/it]
 46%|████▋     | 104423/225000 [221:48:17<474:09:05, 14.16s/it]
 46%|████▋     | 104424/225000 [221:48:22<382:24:18, 11.42s/it]
 46%|████▋     | 104425/225000 [221:48:33<386:58:44, 11.55s/it]
 46%|████▋     | 104426/225000 [221:48:46<393:18:48, 11.74s/it]
 46%|████▋     | 104427/225000 [221:48:57<388:25:43, 11.60s/it]
 46%|████▋     | 104428/225000 [221:49:04<339:42:04, 10.14s/it]
 46%|████▋     | 104429/225000 [221:49:11<307:31:53,  9.18s/it]
 46%|████▋     | 104430/225000 [221:49:21<318:59:40,  9.52s/it]
 46%|████▋     | 104431/225000 [221:49:26<273:02:22,  8.15s/it]
 46%|████▋     | 104432/225000 [221:49:34<268:02:17,  8.00s/it]
 46%|████▋     | 104433/225000 [221:49:40<256:37:46,  7.66s/it]
 46%|████▋     | 104434/225000 [221:49:54<320:16:59,  9.56s/it]
 46%|████▋     | 104435/225000 [221:50:03<312:03:45,  9.32s/it]
 46%|████▋     | 104436/225000 [221:50:12<306:15:03,  9.14s/it]
 46%|████▋     | 104437/225000 [221:50:21<309:17:58,  9.24s/it]
 46%|████▋     | 104438/225000 [221:50:40<399:37:05, 11.93s/it]
 46%|████▋     | 104439/225000 [221:50:49<371:50:06, 11.10s/it]
 46%|████▋     | 104440/225000 [221:51:00<369:39:00, 11.04s/it]
 46%|████▋     | 104441/225000 [221:51:09<355:34:45, 10.62s/it]
 46%|████▋     | 104442/225000 [221:51:21<366:24:45, 10.94s/it]
 46%|████▋     | 104443/225000 [221:51:29<340:04:48, 10.16s/it]
 46%|████▋     | 104444/225000 [221:51:37<316:48:01,  9.46s/it]
 46%|████▋     | 104445/225000 [221:51:47<316:27:28,  9.45s/it]
 46%|████▋     | 104446/225000 [221:51:55<305:40:15,  9.13s/it]
 46%|████▋     | 104447/225000 [221:52:10<364:20:22, 10.88s/it]
 46%|████▋     | 104448/225000 [221:52:28<438:57:45, 13.11s/it]
 46%|████▋     | 104449/225000 [221:52:45<477:51:20, 14.27s/it]
 46%|████▋     | 104450/225000 [221:52:53<413:00:13, 12.33s/it]
                                                               
{'loss': '0.9227', 'grad_norm': '1.6', 'learning_rate': '2.976e-05', 'epoch': '0.4642', 'num_input_tokens_seen': 372286208, 'train_runtime': '7.988e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104450/225000 [221:52:53<413:00:13, 12.33s/it]
 46%|████▋     | 104451/225000 [221:53:08<438:50:26, 13.11s/it]
 46%|████▋     | 104452/225000 [221:53:18<407:55:57, 12.18s/it]
 46%|████▋     | 104453/225000 [221:53:24<344:18:22, 10.28s/it]
 46%|████▋     | 104454/225000 [221:53:34<339:42:56, 10.15s/it]
 46%|████▋     | 104455/225000 [221:53:42<319:46:43,  9.55s/it]
 46%|████▋     | 104456/225000 [221:53:46<267:31:15,  7.99s/it]
 46%|████▋     | 104457/225000 [221:53:57<295:23:48,  8.82s/it]
 46%|████▋     | 104458/225000 [221:54:07<311:27:57,  9.30s/it]
 46%|████▋     | 104459/225000 [221:54:21<351:18:15, 10.49s/it]
 46%|████▋     | 104460/225000 [221:54:34<385:43:49, 11.52s/it]
 46%|████▋     | 104461/225000 [221:54:40<324:25:20,  9.69s/it]
 46%|████▋     | 104462/225000 [221:54:47<296:25:18,  8.85s/it]
 46%|████▋     | 104463/225000 [221:54:53<266:24:16,  7.96s/it]
 46%|████▋     | 104464/225000 [221:55:10<357:35:59, 10.68s/it]
 46%|████▋     | 104465/225000 [221:55:27<420:10:30, 12.55s/it]
 46%|████▋     | 104466/225000 [221:55:36<388:15:26, 11.60s/it]
 46%|████▋     | 104467/225000 [221:55:46<373:08:19, 11.14s/it]
 46%|████▋     | 104468/225000 [221:55:51<308:59:16,  9.23s/it]
 46%|████▋     | 104469/225000 [221:56:02<329:10:15,  9.83s/it]
 46%|████▋     | 104470/225000 [221:56:15<357:01:57, 10.66s/it]
 46%|████▋     | 104471/225000 [221:56:22<320:54:10,  9.58s/it]
 46%|████▋     | 104472/225000 [221:56:28<288:29:02,  8.62s/it]
 46%|████▋     | 104473/225000 [221:56:40<324:56:19,  9.71s/it]
 46%|████▋     | 104474/225000 [221:56:50<326:49:34,  9.76s/it]
 46%|████▋     | 104475/225000 [221:56:59<317:31:55,  9.48s/it]
 46%|████▋     | 104476/225000 [221:57:10<334:23:49,  9.99s/it]
 46%|████▋     | 104477/225000 [221:57:16<287:56:32,  8.60s/it]
 46%|████▋     | 104478/225000 [221:57:27<319:38:28,  9.55s/it]
 46%|████▋     | 104479/225000 [221:57:38<333:20:19,  9.96s/it]
 46%|████▋     | 104480/225000 [221:57:49<344:16:42, 10.28s/it]
 46%|████▋     | 104481/225000 [221:57:56<305:11:12,  9.12s/it]
 46%|████▋     | 104482/225000 [221:58:00<260:13:44,  7.77s/it]
 46%|████▋     | 104483/225000 [221:58:11<287:18:26,  8.58s/it]
 46%|████▋     | 104484/225000 [221:58:25<342:55:25, 10.24s/it]
 46%|████▋     | 104485/225000 [221:58:42<408:41:35, 12.21s/it]
 46%|████▋     | 104486/225000 [221:58:54<412:00:14, 12.31s/it]
 46%|████▋     | 104487/225000 [221:59:05<398:18:10, 11.90s/it]
 46%|████▋     | 104488/225000 [221:59:21<442:33:58, 13.22s/it]
 46%|████▋     | 104489/225000 [221:59:37<466:32:54, 13.94s/it]
 46%|████▋     | 104490/225000 [221:59:51<461:05:31, 13.77s/it]
 46%|████▋     | 104491/225000 [222:00:02<438:25:17, 13.10s/it]
 46%|████▋     | 104492/225000 [222:00:14<425:53:50, 12.72s/it]
 46%|████▋     | 104493/225000 [222:00:19<353:22:21, 10.56s/it]
 46%|████▋     | 104494/225000 [222:00:22<276:47:21,  8.27s/it]
 46%|████▋     | 104495/225000 [222:00:27<245:02:37,  7.32s/it]
 46%|████▋     | 104496/225000 [222:00:40<301:52:20,  9.02s/it]
 46%|████▋     | 104497/225000 [222:00:52<329:28:25,  9.84s/it]
 46%|████▋     | 104498/225000 [222:01:02<328:53:52,  9.83s/it]
 46%|████▋     | 104499/225000 [222:01:15<360:49:43, 10.78s/it]
 46%|████▋     | 104500/225000 [222:01:25<352:16:19, 10.52s/it]
                                                               
{'loss': '0.9263', 'grad_norm': '1.579', 'learning_rate': '2.974e-05', 'epoch': '0.4644', 'num_input_tokens_seen': 372524936, 'train_runtime': '7.993e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104500/225000 [222:01:25<352:16:19, 10.52s/it]
 46%|████▋     | 104501/225000 [222:01:33<324:42:24,  9.70s/it]
 46%|████▋     | 104502/225000 [222:01:43<330:57:10,  9.89s/it]
 46%|████▋     | 104503/225000 [222:01:50<301:21:18,  9.00s/it]
 46%|████▋     | 104504/225000 [222:02:02<334:56:14, 10.01s/it]
 46%|████▋     | 104505/225000 [222:02:13<341:48:33, 10.21s/it]
 46%|████▋     | 104506/225000 [222:02:33<438:35:11, 13.10s/it]
 46%|████▋     | 104507/225000 [222:02:45<427:24:54, 12.77s/it]
 46%|████▋     | 104508/225000 [222:02:54<388:27:23, 11.61s/it]
 46%|████▋     | 104509/225000 [222:03:12<457:33:19, 13.67s/it]
 46%|████▋     | 104510/225000 [222:03:18<377:32:52, 11.28s/it]
 46%|████▋     | 104511/225000 [222:03:21<294:08:11,  8.79s/it]
 46%|████▋     | 104512/225000 [222:03:31<305:18:50,  9.12s/it]
 46%|████▋     | 104513/225000 [222:03:41<319:46:59,  9.55s/it]
 46%|████▋     | 104514/225000 [222:03:53<344:17:33, 10.29s/it]
 46%|████▋     | 104515/225000 [222:04:03<342:03:06, 10.22s/it]
 46%|████▋     | 104516/225000 [222:04:10<302:00:12,  9.02s/it]
 46%|████▋     | 104517/225000 [222:04:19<301:16:00,  9.00s/it]
 46%|████▋     | 104518/225000 [222:04:33<357:58:06, 10.70s/it]
 46%|████▋     | 104519/225000 [222:04:38<303:09:54,  9.06s/it]
 46%|████▋     | 104520/225000 [222:04:45<281:30:05,  8.41s/it]
 46%|████▋     | 104521/225000 [222:04:55<290:47:36,  8.69s/it]
 46%|████▋     | 104522/225000 [222:05:08<332:19:03,  9.93s/it]
 46%|████▋     | 104523/225000 [222:05:17<332:01:14,  9.92s/it]
 46%|████▋     | 104524/225000 [222:05:26<322:52:57,  9.65s/it]
 46%|████▋     | 104525/225000 [222:05:31<270:59:55,  8.10s/it]
 46%|████▋     | 104526/225000 [222:05:45<327:51:12,  9.80s/it]
 46%|████▋     | 104527/225000 [222:05:56<342:55:56, 10.25s/it]
 46%|████▋     | 104528/225000 [222:06:03<314:04:41,  9.39s/it]
 46%|████▋     | 104529/225000 [222:06:13<315:39:26,  9.43s/it]
 46%|████▋     | 104530/225000 [222:06:27<362:10:59, 10.82s/it]
 46%|████▋     | 104531/225000 [222:06:35<331:18:30,  9.90s/it]
 46%|████▋     | 104532/225000 [222:06:51<394:44:01, 11.80s/it]
 46%|████▋     | 104533/225000 [222:07:07<440:24:44, 13.16s/it]
 46%|████▋     | 104534/225000 [222:07:12<360:00:40, 10.76s/it]
 46%|████▋     | 104535/225000 [222:07:21<341:50:42, 10.22s/it]
 46%|████▋     | 104536/225000 [222:07:27<300:22:23,  8.98s/it]
 46%|████▋     | 104537/225000 [222:07:40<337:41:04, 10.09s/it]
 46%|████▋     | 104538/225000 [222:07:48<313:06:53,  9.36s/it]
 46%|████▋     | 104539/225000 [222:07:59<334:26:36,  9.99s/it]
 46%|████▋     | 104540/225000 [222:08:10<338:58:17, 10.13s/it]
 46%|████▋     | 104541/225000 [222:08:19<328:59:06,  9.83s/it]
 46%|████▋     | 104542/225000 [222:08:33<372:47:42, 11.14s/it]
 46%|████▋     | 104543/225000 [222:08:38<307:12:35,  9.18s/it]
 46%|████▋     | 104544/225000 [222:08:54<381:11:36, 11.39s/it]
 46%|████▋     | 104545/225000 [222:09:02<343:39:57, 10.27s/it]
 46%|████▋     | 104546/225000 [222:09:07<292:50:54,  8.75s/it]
 46%|████▋     | 104547/225000 [222:09:14<274:34:07,  8.21s/it]
 46%|████▋     | 104548/225000 [222:09:24<287:39:56,  8.60s/it]
 46%|████▋     | 104549/225000 [222:09:37<335:40:16, 10.03s/it]
 46%|████▋     | 104550/225000 [222:09:49<354:29:46, 10.60s/it]
                                                               
{'loss': '0.9282', 'grad_norm': '1.383', 'learning_rate': '2.972e-05', 'epoch': '0.4647', 'num_input_tokens_seen': 372747736, 'train_runtime': '7.998e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104550/225000 [222:09:49<354:29:46, 10.60s/it]
 46%|████▋     | 104551/225000 [222:09:51<270:41:49,  8.09s/it]
 46%|████▋     | 104552/225000 [222:10:00<282:55:17,  8.46s/it]
 46%|████▋     | 104553/225000 [222:10:13<326:37:28,  9.76s/it]
 46%|████▋     | 104554/225000 [222:10:28<372:33:42, 11.14s/it]
 46%|████▋     | 104555/225000 [222:10:43<420:53:35, 12.58s/it]
 46%|████▋     | 104556/225000 [222:10:58<438:40:19, 13.11s/it]
 46%|████▋     | 104557/225000 [222:11:15<477:49:48, 14.28s/it]
 46%|████▋     | 104558/225000 [222:11:21<398:15:39, 11.90s/it]
 46%|████▋     | 104559/225000 [222:11:30<370:07:42, 11.06s/it]
 46%|████▋     | 104560/225000 [222:11:46<414:15:25, 12.38s/it]
 46%|████▋     | 104561/225000 [222:11:55<384:43:15, 11.50s/it]
 46%|████▋     | 104562/225000 [222:12:06<381:25:30, 11.40s/it]
 46%|████▋     | 104563/225000 [222:12:21<412:02:18, 12.32s/it]
 46%|████▋     | 104564/225000 [222:12:30<385:37:20, 11.53s/it]
 46%|████▋     | 104565/225000 [222:12:40<360:55:38, 10.79s/it]
 46%|████▋     | 104566/225000 [222:12:56<416:33:10, 12.45s/it]
 46%|████▋     | 104567/225000 [222:13:02<353:03:25, 10.55s/it]
 46%|████▋     | 104568/225000 [222:13:18<410:07:39, 12.26s/it]
 46%|████▋     | 104569/225000 [222:13:30<409:38:34, 12.25s/it]
 46%|████▋     | 104570/225000 [222:13:39<373:27:05, 11.16s/it]
 46%|████▋     | 104571/225000 [222:13:50<367:49:23, 11.00s/it]
 46%|████▋     | 104572/225000 [222:14:01<371:31:58, 11.11s/it]
 46%|████▋     | 104573/225000 [222:14:10<347:25:30, 10.39s/it]
 46%|████▋     | 104574/225000 [222:14:22<361:49:40, 10.82s/it]
 46%|████▋     | 104575/225000 [222:14:25<288:03:07,  8.61s/it]
 46%|████▋     | 104576/225000 [222:14:32<273:35:28,  8.18s/it]
 46%|████▋     | 104577/225000 [222:14:40<274:12:57,  8.20s/it]
 46%|████▋     | 104578/225000 [222:14:47<257:42:14,  7.70s/it]
 46%|████▋     | 104579/225000 [222:14:52<229:53:09,  6.87s/it]
 46%|████▋     | 104580/225000 [222:14:55<195:28:02,  5.84s/it]
 46%|████▋     | 104581/225000 [222:15:09<271:36:32,  8.12s/it]
 46%|████▋     | 104582/225000 [222:15:21<307:51:25,  9.20s/it]
 46%|████▋     | 104583/225000 [222:15:26<266:04:40,  7.95s/it]
 46%|████▋     | 104584/225000 [222:15:34<274:22:03,  8.20s/it]
 46%|████▋     | 104585/225000 [222:15:42<267:00:13,  7.98s/it]
 46%|████▋     | 104586/225000 [222:16:00<366:33:13, 10.96s/it]
 46%|████▋     | 104587/225000 [222:16:16<420:15:39, 12.56s/it]
 46%|████▋     | 104588/225000 [222:16:26<389:46:14, 11.65s/it]
 46%|████▋     | 104589/225000 [222:16:30<319:37:42,  9.56s/it]
 46%|████▋     | 104590/225000 [222:16:37<292:58:30,  8.76s/it]
 46%|████▋     | 104591/225000 [222:16:44<276:01:24,  8.25s/it]
 46%|████▋     | 104592/225000 [222:17:08<430:28:23, 12.87s/it]
 46%|████▋     | 104593/225000 [222:17:25<475:28:49, 14.22s/it]
 46%|████▋     | 104594/225000 [222:17:33<414:30:06, 12.39s/it]
 46%|████▋     | 104595/225000 [222:17:37<322:20:50,  9.64s/it]
 46%|████▋     | 104596/225000 [222:17:50<364:17:03, 10.89s/it]
 46%|████▋     | 104597/225000 [222:17:57<317:47:00,  9.50s/it]
 46%|████▋     | 104598/225000 [222:18:08<331:22:50,  9.91s/it]
 46%|████▋     | 104599/225000 [222:18:24<402:03:13, 12.02s/it]
 46%|████▋     | 104600/225000 [222:18:39<429:46:04, 12.85s/it]
                                                               
{'loss': '0.9226', 'grad_norm': '1.658', 'learning_rate': '2.97e-05', 'epoch': '0.4649', 'num_input_tokens_seen': 372989640, 'train_runtime': '8.003e+05', 'train_tokens_per_second': '466.1'}

 46%|████▋     | 104600/225000 [222:18:39<429:46:04, 12.85s/it]
 46%|████▋     | 104601/225000 [222:18:53<437:22:23, 13.08s/it]
 46%|████▋     | 104602/225000 [222:19:09<469:48:20, 14.05s/it]
 46%|████▋     | 104603/225000 [222:19:28<513:35:43, 15.36s/it]
 46%|████▋     | 104604/225000 [222:19:41<497:38:23, 14.88s/it]
 46%|████▋     | 104605/225000 [222:19:59<527:38:11, 15.78s/it]
 46%|████▋     | 104606/225000 [222:20:12<493:17:45, 14.75s/it]
 46%|████▋     | 104607/225000 [222:20:17<404:28:49, 12.09s/it]
 46%|████▋     | 104608/225000 [222:20:26<370:46:39, 11.09s/it]
 46%|████▋     | 104609/225000 [222:20:38<375:01:27, 11.21s/it]
 46%|████▋     | 104610/225000 [222:20:43<312:31:23,  9.35s/it]
 46%|████▋     | 104611/225000 [222:20:49<279:59:29,  8.37s/it]
 46%|████▋     | 104612/225000 [222:21:06<369:44:36, 11.06s/it]
 46%|████▋     | 104613/225000 [222:21:16<357:44:46, 10.70s/it]
 46%|████▋     | 104614/225000 [222:21:31<398:30:28, 11.92s/it]
 46%|████▋     | 104615/225000 [222:21:42<388:59:53, 11.63s/it]
 46%|████▋     | 104616/225000 [222:21:56<417:30:17, 12.49s/it]
 46%|████▋     | 104617/225000 [222:22:11<439:30:43, 13.14s/it]
 46%|████▋     | 104618/225000 [222:22:22<422:01:11, 12.62s/it]
 46%|████▋     | 104619/225000 [222:22:37<444:21:47, 13.29s/it]
 46%|████▋     | 104620/225000 [222:22:44<382:08:17, 11.43s/it]
 46%|████▋     | 104621/225000 [222:22:52<347:23:21, 10.39s/it]
 46%|████▋     | 104622/225000 [222:23:02<346:13:50, 10.35s/it]
 46%|████▋     | 104623/225000 [222:23:05<272:26:22,  8.15s/it]
 46%|████▋     | 104624/225000 [222:23:18<318:18:25,  9.52s/it]
 46%|████▋     | 104625/225000 [222:23:27<315:31:03,  9.44s/it]
 47%|████▋     | 104626/225000 [222:23:37<318:02:40,  9.51s/it]
 47%|████▋     | 104627/225000 [222:23:46<316:42:54,  9.47s/it]
 47%|████▋     | 104628/225000 [222:23:57<323:27:09,  9.67s/it]
 47%|████▋     | 104629/225000 [222:24:12<380:36:21, 11.38s/it]
 47%|████▋     | 104630/225000 [222:24:21<353:03:00, 10.56s/it]
 47%|████▋     | 104631/225000 [222:24:33<372:07:44, 11.13s/it]
 47%|████▋     | 104632/225000 [222:24:41<344:31:17, 10.30s/it]
 47%|████▋     | 104633/225000 [222:24:49<317:19:25,  9.49s/it]
 47%|████▋     | 104634/225000 [222:24:59<319:02:43,  9.54s/it]
 47%|████▋     | 104635/225000 [222:25:11<348:31:32, 10.42s/it]
 47%|████▋     | 104636/225000 [222:25:20<335:10:19, 10.02s/it]
 47%|████▋     | 104637/225000 [222:25:24<275:18:55,  8.23s/it]
 47%|████▋     | 104638/225000 [222:25:34<290:09:57,  8.68s/it]
 47%|████▋     | 104639/225000 [222:25:36<226:22:39,  6.77s/it]
 47%|████▋     | 104640/225000 [222:25:46<259:29:25,  7.76s/it]
 47%|████▋     | 104641/225000 [222:25:52<237:25:31,  7.10s/it]
 47%|████▋     | 104642/225000 [222:26:03<281:04:05,  8.41s/it]
 47%|████▋     | 104643/225000 [222:26:14<304:34:06,  9.11s/it]
 47%|████▋     | 104644/225000 [222:26:27<345:36:20, 10.34s/it]
 47%|████▋     | 104645/225000 [222:26:38<352:51:13, 10.55s/it]
 47%|████▋     | 104646/225000 [222:26:50<362:48:30, 10.85s/it]
 47%|████▋     | 104647/225000 [222:27:01<359:59:32, 10.77s/it]
 47%|████▋     | 104648/225000 [222:27:14<387:35:55, 11.59s/it]
 47%|████▋     | 104649/225000 [222:27:27<403:39:07, 12.07s/it]
 47%|████▋     | 104650/225000 [222:27:34<345:06:26, 10.32s/it]
                                                               
{'loss': '0.9399', 'grad_norm': '1.554', 'learning_rate': '2.969e-05', 'epoch': '0.4651', 'num_input_tokens_seen': 373244024, 'train_runtime': '8.009e+05', 'train_tokens_per_second': '466.1'}

 47%|████▋     | 104650/225000 [222:27:34<345:06:26, 10.32s/it]
 47%|████▋     | 104651/225000 [222:27:54<451:35:10, 13.51s/it]
 47%|████▋     | 104652/225000 [222:28:02<396:13:49, 11.85s/it]
 47%|████▋     | 104653/225000 [222:28:11<357:46:00, 10.70s/it]
 47%|████▋     | 104654/225000 [222:28:27<410:52:10, 12.29s/it]
 47%|████▋     | 104655/225000 [222:28:42<439:26:54, 13.15s/it]
 47%|████▋     | 104656/225000 [222:28:47<357:47:34, 10.70s/it]
 47%|████▋     | 104657/225000 [222:29:07<457:23:11, 13.68s/it]
 47%|████▋     | 104658/225000 [222:29:21<462:15:44, 13.83s/it]
 47%|████▋     | 104659/225000 [222:29:26<369:22:49, 11.05s/it]
 47%|████▋     | 104660/225000 [222:29:38<380:55:33, 11.40s/it]
 47%|████▋     | 104661/225000 [222:29:49<370:53:49, 11.10s/it]
 47%|████▋     | 104662/225000 [222:29:53<302:09:54,  9.04s/it]
 47%|████▋     | 104663/225000 [222:30:06<339:57:26, 10.17s/it]
 47%|████▋     | 104664/225000 [222:30:13<311:28:41,  9.32s/it]
 47%|████▋     | 104665/225000 [222:30:28<366:36:43, 10.97s/it]
 47%|████▋     | 104666/225000 [222:30:50<478:51:30, 14.33s/it]
 47%|████▋     | 104667/225000 [222:30:57<407:00:45, 12.18s/it]
 47%|████▋     | 104668/225000 [222:31:05<361:58:54, 10.83s/it]
 47%|████▋     | 104669/225000 [222:31:08<284:02:03,  8.50s/it]
 47%|████▋     | 104670/225000 [222:31:12<240:30:39,  7.20s/it]
 47%|████▋     | 104671/225000 [222:31:26<311:10:49,  9.31s/it]
 47%|████▋     | 104672/225000 [222:31:36<313:09:55,  9.37s/it]
 47%|████▋     | 104673/225000 [222:31:43<293:11:13,  8.77s/it]
 47%|████▋     | 104674/225000 [222:31:51<287:18:02,  8.60s/it]
 47%|████▋     | 104675/225000 [222:31:57<259:45:38,  7.77s/it]
 47%|████▋     | 104676/225000 [222:32:07<280:37:53,  8.40s/it]
 47%|████▋     | 104677/225000 [222:32:11<233:59:31,  7.00s/it]
 47%|████▋     | 104678/225000 [222:32:21<268:58:59,  8.05s/it]
 47%|████▋     | 104679/225000 [222:32:33<307:38:49,  9.20s/it]
 47%|████▋     | 104680/225000 [222:32:40<284:14:21,  8.50s/it]
 47%|████▋     | 104681/225000 [222:32:49<283:49:03,  8.49s/it]
 47%|████▋     | 104682/225000 [222:33:02<337:24:53, 10.10s/it]
 47%|████▋     | 104683/225000 [222:33:12<335:27:42, 10.04s/it]
 47%|████▋     | 104684/225000 [222:33:22<335:03:24, 10.03s/it]
 47%|████▋     | 104685/225000 [222:33:29<305:55:26,  9.15s/it]
 47%|████▋     | 104686/225000 [222:33:38<302:08:28,  9.04s/it]
 47%|████▋     | 104687/225000 [222:33:50<325:51:43,  9.75s/it]
 47%|████▋     | 104688/225000 [222:33:54<268:15:21,  8.03s/it]
 47%|████▋     | 104689/225000 [222:34:04<289:01:46,  8.65s/it]
 47%|████▋     | 104690/225000 [222:34:23<397:56:38, 11.91s/it]
 47%|████▋     | 104691/225000 [222:34:32<369:21:32, 11.05s/it]
 47%|████▋     | 104692/225000 [222:34:39<327:15:05,  9.79s/it]
 47%|████▋     | 104693/225000 [222:34:53<368:44:51, 11.03s/it]
 47%|████▋     | 104694/225000 [222:35:04<373:05:33, 11.16s/it]
 47%|████▋     | 104695/225000 [222:35:17<388:56:00, 11.64s/it]
 47%|████▋     | 104696/225000 [222:35:21<308:31:00,  9.23s/it]
 47%|████▋     | 104697/225000 [222:35:36<368:07:36, 11.02s/it]
 47%|████▋     | 104698/225000 [222:35:52<417:11:03, 12.48s/it]
 47%|████▋     | 104699/225000 [222:36:01<383:47:24, 11.48s/it]
 47%|████▋     | 104700/225000 [222:36:13<387:09:30, 11.59s/it]
                                                               
{'loss': '0.9316', 'grad_norm': '1.794', 'learning_rate': '2.967e-05', 'epoch': '0.4653', 'num_input_tokens_seen': 373472168, 'train_runtime': '8.014e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104700/225000 [222:36:13<387:09:30, 11.59s/it]
 47%|████▋     | 104701/225000 [222:36:29<427:56:20, 12.81s/it]
 47%|████▋     | 104702/225000 [222:36:36<375:50:46, 11.25s/it]
 47%|████▋     | 104703/225000 [222:36:48<381:12:02, 11.41s/it]
 47%|████▋     | 104704/225000 [222:36:55<341:28:05, 10.22s/it]
 47%|████▋     | 104705/225000 [222:37:11<395:05:41, 11.82s/it]
 47%|████▋     | 104706/225000 [222:37:26<431:54:07, 12.93s/it]
 47%|████▋     | 104707/225000 [222:37:30<340:44:51, 10.20s/it]
 47%|████▋     | 104708/225000 [222:37:37<304:28:16,  9.11s/it]
 47%|████▋     | 104709/225000 [222:37:39<235:37:10,  7.05s/it]
 47%|████▋     | 104710/225000 [222:37:50<277:12:58,  8.30s/it]
 47%|████▋     | 104711/225000 [222:38:03<320:29:10,  9.59s/it]
 47%|████▋     | 104712/225000 [222:38:15<345:52:06, 10.35s/it]
 47%|████▋     | 104713/225000 [222:38:27<362:30:39, 10.85s/it]
 47%|████▋     | 104714/225000 [222:38:41<393:28:28, 11.78s/it]
 47%|████▋     | 104715/225000 [222:38:55<412:47:58, 12.35s/it]
 47%|████▋     | 104716/225000 [222:39:09<435:18:51, 13.03s/it]
 47%|████▋     | 104717/225000 [222:39:20<408:58:19, 12.24s/it]
 47%|████▋     | 104718/225000 [222:39:30<392:58:18, 11.76s/it]
 47%|████▋     | 104719/225000 [222:39:39<361:34:11, 10.82s/it]
 47%|████▋     | 104720/225000 [222:39:50<359:35:27, 10.76s/it]
 47%|████▋     | 104721/225000 [222:39:55<310:38:37,  9.30s/it]
 47%|████▋     | 104722/225000 [222:40:00<264:16:53,  7.91s/it]
 47%|████▋     | 104723/225000 [222:40:03<215:27:55,  6.45s/it]
 47%|████▋     | 104724/225000 [222:40:06<182:05:57,  5.45s/it]
 47%|████▋     | 104725/225000 [222:40:18<243:53:52,  7.30s/it]
 47%|████▋     | 104726/225000 [222:40:36<353:54:59, 10.59s/it]
 47%|████▋     | 104727/225000 [222:40:45<333:55:14,  9.99s/it]
 47%|████▋     | 104728/225000 [222:40:57<355:09:22, 10.63s/it]
 47%|████▋     | 104729/225000 [222:41:12<395:19:38, 11.83s/it]
 47%|████▋     | 104730/225000 [222:41:16<322:21:44,  9.65s/it]
 47%|████▋     | 104731/225000 [222:41:20<264:17:50,  7.91s/it]
 47%|████▋     | 104732/225000 [222:41:24<224:19:14,  6.71s/it]
 47%|████▋     | 104733/225000 [222:41:33<251:24:11,  7.53s/it]
 47%|████▋     | 104734/225000 [222:41:45<296:11:14,  8.87s/it]
 47%|████▋     | 104735/225000 [222:42:00<354:24:33, 10.61s/it]
 47%|████▋     | 104736/225000 [222:42:11<362:36:45, 10.85s/it]
 47%|████▋     | 104737/225000 [222:42:15<291:47:55,  8.73s/it]
 47%|████▋     | 104738/225000 [222:42:24<295:03:20,  8.83s/it]
 47%|████▋     | 104739/225000 [222:42:32<283:06:06,  8.47s/it]
 47%|████▋     | 104740/225000 [222:42:41<291:42:45,  8.73s/it]
 47%|████▋     | 104741/225000 [222:42:49<283:09:58,  8.48s/it]
 47%|████▋     | 104742/225000 [222:42:53<236:21:08,  7.08s/it]
 47%|████▋     | 104743/225000 [222:43:11<346:12:21, 10.36s/it]
 47%|████▋     | 104744/225000 [222:43:14<274:12:16,  8.21s/it]
 47%|████▋     | 104745/225000 [222:43:17<219:21:44,  6.57s/it]
 47%|████▋     | 104746/225000 [222:43:30<282:51:29,  8.47s/it]
 47%|████▋     | 104747/225000 [222:43:42<315:38:52,  9.45s/it]
 47%|████▋     | 104748/225000 [222:43:59<393:47:14, 11.79s/it]
 47%|████▋     | 104749/225000 [222:44:08<367:08:46, 10.99s/it]
 47%|████▋     | 104750/225000 [222:44:13<311:31:39,  9.33s/it]
                                                               
{'loss': '0.9326', 'grad_norm': '2.034', 'learning_rate': '2.965e-05', 'epoch': '0.4656', 'num_input_tokens_seen': 373692848, 'train_runtime': '8.019e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104750/225000 [222:44:13<311:31:39,  9.33s/it]
 47%|████▋     | 104751/225000 [222:44:31<396:11:08, 11.86s/it]
 47%|████▋     | 104752/225000 [222:44:48<441:46:16, 13.23s/it]
 47%|████▋     | 104753/225000 [222:44:57<399:15:33, 11.95s/it]
 47%|████▋     | 104754/225000 [222:45:04<358:37:40, 10.74s/it]
 47%|████▋     | 104755/225000 [222:45:07<281:27:53,  8.43s/it]
 47%|████▋     | 104756/225000 [222:45:22<346:29:17, 10.37s/it]
 47%|████▋     | 104757/225000 [222:45:29<308:25:22,  9.23s/it]
 47%|████▋     | 104758/225000 [222:45:37<299:06:52,  8.96s/it]
 47%|████▋     | 104759/225000 [222:45:51<348:22:48, 10.43s/it]
 47%|████▋     | 104760/225000 [222:46:03<367:04:48, 10.99s/it]
 47%|████▋     | 104761/225000 [222:46:06<284:58:43,  8.53s/it]
 47%|████▋     | 104762/225000 [222:46:21<351:20:53, 10.52s/it]
 47%|████▋     | 104763/225000 [222:46:32<354:38:59, 10.62s/it]
 47%|████▋     | 104764/225000 [222:46:45<380:03:06, 11.38s/it]
 47%|████▋     | 104765/225000 [222:46:49<304:10:11,  9.11s/it]
 47%|████▋     | 104766/225000 [222:46:55<270:34:15,  8.10s/it]
 47%|████▋     | 104767/225000 [222:47:00<243:32:31,  7.29s/it]
 47%|████▋     | 104768/225000 [222:47:07<234:43:09,  7.03s/it]
 47%|████▋     | 104769/225000 [222:47:25<349:43:45, 10.47s/it]
 47%|████▋     | 104770/225000 [222:47:33<322:40:01,  9.66s/it]
 47%|████▋     | 104771/225000 [222:47:46<358:34:53, 10.74s/it]
 47%|████▋     | 104772/225000 [222:47:54<332:59:20,  9.97s/it]
 47%|████▋     | 104773/225000 [222:48:06<351:50:55, 10.54s/it]
 47%|████▋     | 104774/225000 [222:48:18<360:34:30, 10.80s/it]
 47%|████▋     | 104775/225000 [222:48:35<426:38:34, 12.78s/it]
 47%|████▋     | 104776/225000 [222:48:39<337:16:16, 10.10s/it]
 47%|████▋     | 104777/225000 [222:48:45<293:17:27,  8.78s/it]
 47%|████▋     | 104778/225000 [222:49:02<374:53:36, 11.23s/it]
 47%|████▋     | 104779/225000 [222:49:23<472:14:02, 14.14s/it]
 47%|████▋     | 104780/225000 [222:49:26<369:10:14, 11.05s/it]
 47%|████▋     | 104781/225000 [222:49:36<351:14:21, 10.52s/it]
 47%|████▋     | 104782/225000 [222:49:52<409:37:10, 12.27s/it]
 47%|████▋     | 104783/225000 [222:50:02<384:44:52, 11.52s/it]
 47%|████▋     | 104784/225000 [222:50:14<394:56:14, 11.83s/it]
 47%|████▋     | 104785/225000 [222:50:24<373:59:04, 11.20s/it]
 47%|████▋     | 104786/225000 [222:50:40<417:57:47, 12.52s/it]
 47%|████▋     | 104787/225000 [222:50:47<364:06:06, 10.90s/it]
 47%|████▋     | 104788/225000 [222:51:02<410:46:00, 12.30s/it]
 47%|████▋     | 104789/225000 [222:51:12<382:10:53, 11.45s/it]
 47%|████▋     | 104790/225000 [222:51:22<368:27:39, 11.03s/it]
 47%|████▋     | 104791/225000 [222:51:38<418:08:53, 12.52s/it]
 47%|████▋     | 104792/225000 [222:52:00<512:09:19, 15.34s/it]
 47%|████▋     | 104793/225000 [222:52:05<414:19:33, 12.41s/it]
 47%|████▋     | 104794/225000 [222:52:24<473:16:42, 14.17s/it]
 47%|████▋     | 104795/225000 [222:52:34<435:06:11, 13.03s/it]
 47%|████▋     | 104796/225000 [222:52:53<496:58:41, 14.88s/it]
 47%|████▋     | 104797/225000 [222:52:59<410:25:31, 12.29s/it]
 47%|████▋     | 104798/225000 [222:53:14<432:56:39, 12.97s/it]
 47%|████▋     | 104799/225000 [222:53:18<341:46:07, 10.24s/it]
 47%|████▋     | 104800/225000 [222:53:24<298:13:51,  8.93s/it]
                                                               
{'loss': '0.9672', 'grad_norm': '1.958', 'learning_rate': '2.963e-05', 'epoch': '0.4658', 'num_input_tokens_seen': 373936232, 'train_runtime': '8.024e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104800/225000 [222:53:24<298:13:51,  8.93s/it]
 47%|████▋     | 104801/225000 [222:53:37<338:28:30, 10.14s/it]
 47%|████▋     | 104802/225000 [222:53:53<399:02:09, 11.95s/it]
 47%|████▋     | 104803/225000 [222:53:56<311:52:16,  9.34s/it]
 47%|████▋     | 104804/225000 [222:53:59<249:22:30,  7.47s/it]
 47%|████▋     | 104805/225000 [222:54:14<320:24:51,  9.60s/it]
 47%|████▋     | 104806/225000 [222:54:32<408:47:05, 12.24s/it]
 47%|████▋     | 104807/225000 [222:54:45<417:02:16, 12.49s/it]
 47%|████▋     | 104808/225000 [222:54:54<374:56:46, 11.23s/it]
 47%|████▋     | 104809/225000 [222:55:05<372:23:17, 11.15s/it]
 47%|████▋     | 104810/225000 [222:55:08<298:04:47,  8.93s/it]
 47%|████▋     | 104811/225000 [222:55:13<254:43:10,  7.63s/it]
 47%|████▋     | 104812/225000 [222:55:18<226:26:24,  6.78s/it]
 47%|████▋     | 104813/225000 [222:55:24<219:35:43,  6.58s/it]
 47%|████▋     | 104814/225000 [222:55:35<265:37:13,  7.96s/it]
 47%|████▋     | 104815/225000 [222:55:54<377:12:44, 11.30s/it]
 47%|████▋     | 104816/225000 [222:56:11<432:23:47, 12.95s/it]
 47%|████▋     | 104817/225000 [222:56:24<432:16:01, 12.95s/it]
 47%|████▋     | 104818/225000 [222:56:32<383:13:41, 11.48s/it]
 47%|████▋     | 104819/225000 [222:56:43<379:43:07, 11.37s/it]
 47%|████▋     | 104820/225000 [222:56:49<329:35:43,  9.87s/it]
 47%|████▋     | 104821/225000 [222:56:53<268:28:56,  8.04s/it]
 47%|████▋     | 104822/225000 [222:57:11<368:49:23, 11.05s/it]
 47%|████▋     | 104823/225000 [222:57:23<371:22:04, 11.12s/it]
 47%|████▋     | 104824/225000 [222:57:26<291:10:09,  8.72s/it]
 47%|████▋     | 104825/225000 [222:57:39<333:37:08,  9.99s/it]
 47%|████▋     | 104826/225000 [222:57:49<336:23:37, 10.08s/it]
 47%|████▋     | 104827/225000 [222:58:05<393:59:41, 11.80s/it]
 47%|████▋     | 104828/225000 [222:58:16<388:52:45, 11.65s/it]
 47%|████▋     | 104829/225000 [222:58:25<367:09:10, 11.00s/it]
 47%|████▋     | 104830/225000 [222:58:36<366:17:58, 10.97s/it]
 47%|████▋     | 104831/225000 [222:58:49<378:49:20, 11.35s/it]
 47%|████▋     | 104832/225000 [222:59:09<470:14:59, 14.09s/it]
 47%|████▋     | 104833/225000 [222:59:14<383:17:22, 11.48s/it]
 47%|████▋     | 104834/225000 [222:59:26<386:52:47, 11.59s/it]
 47%|████▋     | 104835/225000 [222:59:41<415:51:12, 12.46s/it]
 47%|████▋     | 104836/225000 [222:59:45<329:10:16,  9.86s/it]
 47%|████▋     | 104837/225000 [222:59:51<296:10:29,  8.87s/it]
 47%|████▋     | 104838/225000 [222:59:56<255:38:39,  7.66s/it]
 47%|████▋     | 104839/225000 [223:00:07<286:37:13,  8.59s/it]
 47%|████▋     | 104840/225000 [223:00:16<289:36:37,  8.68s/it]
 47%|████▋     | 104841/225000 [223:00:30<347:05:57, 10.40s/it]
 47%|████▋     | 104842/225000 [223:00:45<391:23:47, 11.73s/it]
 47%|████▋     | 104843/225000 [223:00:57<392:35:56, 11.76s/it]
 47%|████▋     | 104844/225000 [223:01:09<394:49:44, 11.83s/it]
 47%|████▋     | 104845/225000 [223:01:26<449:44:28, 13.47s/it]
 47%|████▋     | 104846/225000 [223:01:31<364:57:41, 10.93s/it]
 47%|████▋     | 104847/225000 [223:01:46<400:19:38, 11.99s/it]
 47%|████▋     | 104848/225000 [223:01:52<340:34:27, 10.20s/it]
 47%|████▋     | 104849/225000 [223:02:02<348:13:42, 10.43s/it]
 47%|████▋     | 104850/225000 [223:02:17<389:32:28, 11.67s/it]
                                                               
{'loss': '0.9157', 'grad_norm': '2.393', 'learning_rate': '2.961e-05', 'epoch': '0.466', 'num_input_tokens_seen': 374174320, 'train_runtime': '8.029e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104850/225000 [223:02:17<389:32:28, 11.67s/it]
 47%|████▋     | 104851/225000 [223:02:26<357:27:16, 10.71s/it]
 47%|████▋     | 104852/225000 [223:02:38<375:46:43, 11.26s/it]
 47%|████▋     | 104853/225000 [223:02:43<316:25:05,  9.48s/it]
 47%|████▋     | 104854/225000 [223:03:03<414:10:02, 12.41s/it]
 47%|████▋     | 104855/225000 [223:03:05<312:09:16,  9.35s/it]
 47%|████▋     | 104856/225000 [223:03:12<291:17:23,  8.73s/it]
 47%|████▋     | 104857/225000 [223:03:17<252:00:13,  7.55s/it]
 47%|████▋     | 104858/225000 [223:03:26<272:07:03,  8.15s/it]
 47%|████▋     | 104859/225000 [223:03:44<364:10:58, 10.91s/it]
 47%|████▋     | 104860/225000 [223:03:59<406:52:06, 12.19s/it]
 47%|████▋     | 104861/225000 [223:04:09<385:24:57, 11.55s/it]
 47%|████▋     | 104862/225000 [223:04:19<372:43:58, 11.17s/it]
 47%|████▋     | 104863/225000 [223:04:29<361:09:06, 10.82s/it]
 47%|████▋     | 104864/225000 [223:04:42<381:46:47, 11.44s/it]
 47%|████▋     | 104865/225000 [223:04:50<348:31:15, 10.44s/it]
 47%|████▋     | 104866/225000 [223:05:04<375:32:01, 11.25s/it]
 47%|████▋     | 104867/225000 [223:05:12<350:42:51, 10.51s/it]
 47%|████▋     | 104868/225000 [223:05:17<289:47:26,  8.68s/it]
 47%|████▋     | 104869/225000 [223:05:28<316:00:36,  9.47s/it]
 47%|████▋     | 104870/225000 [223:05:35<287:20:33,  8.61s/it]
 47%|████▋     | 104871/225000 [223:05:42<272:10:01,  8.16s/it]
 47%|████▋     | 104872/225000 [223:06:00<369:29:21, 11.07s/it]
 47%|████▋     | 104873/225000 [223:06:04<305:51:20,  9.17s/it]
 47%|████▋     | 104874/225000 [223:06:20<370:41:57, 11.11s/it]
 47%|████▋     | 104875/225000 [223:06:28<338:59:28, 10.16s/it]
 47%|████▋     | 104876/225000 [223:06:38<335:47:05, 10.06s/it]
 47%|████▋     | 104877/225000 [223:06:44<301:00:20,  9.02s/it]
 47%|████▋     | 104878/225000 [223:06:50<263:15:05,  7.89s/it]
 47%|████▋     | 104879/225000 [223:07:11<399:26:58, 11.97s/it]
 47%|████▋     | 104880/225000 [223:07:16<324:31:47,  9.73s/it]
 47%|████▋     | 104881/225000 [223:07:29<361:35:44, 10.84s/it]
 47%|████▋     | 104882/225000 [223:07:44<406:24:35, 12.18s/it]
 47%|████▋     | 104883/225000 [223:07:54<378:15:07, 11.34s/it]
 47%|████▋     | 104884/225000 [223:08:05<377:44:02, 11.32s/it]
 47%|████▋     | 104885/225000 [223:08:18<396:29:10, 11.88s/it]
 47%|████▋     | 104886/225000 [223:08:29<383:09:39, 11.48s/it]
 47%|████▋     | 104887/225000 [223:08:40<381:35:49, 11.44s/it]
 47%|████▋     | 104888/225000 [223:08:56<428:50:17, 12.85s/it]
 47%|████▋     | 104889/225000 [223:09:01<346:58:47, 10.40s/it]
 47%|████▋     | 104890/225000 [223:09:09<328:59:32,  9.86s/it]
 47%|████▋     | 104891/225000 [223:09:28<412:35:04, 12.37s/it]
 47%|████▋     | 104892/225000 [223:09:33<337:50:51, 10.13s/it]
 47%|████▋     | 104893/225000 [223:09:46<374:04:58, 11.21s/it]
 47%|████▋     | 104894/225000 [223:09:55<348:08:42, 10.44s/it]
 47%|████▋     | 104895/225000 [223:09:59<280:07:15,  8.40s/it]
 47%|████▋     | 104896/225000 [223:10:17<375:29:02, 11.25s/it]
 47%|████▋     | 104897/225000 [223:10:21<310:07:14,  9.30s/it]
 47%|████▋     | 104898/225000 [223:10:30<305:21:42,  9.15s/it]
 47%|████▋     | 104899/225000 [223:10:52<428:46:50, 12.85s/it]
 47%|████▋     | 104900/225000 [223:11:02<408:20:40, 12.24s/it]
                                                               
{'loss': '0.9514', 'grad_norm': '1.58', 'learning_rate': '2.96e-05', 'epoch': '0.4662', 'num_input_tokens_seen': 374405944, 'train_runtime': '8.035e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104900/225000 [223:11:02<408:20:40, 12.24s/it]
 47%|████▋     | 104901/225000 [223:11:09<354:18:01, 10.62s/it]
 47%|████▋     | 104902/225000 [223:11:19<349:39:15, 10.48s/it]
 47%|████▋     | 104903/225000 [223:11:38<429:13:31, 12.87s/it]
 47%|████▋     | 104904/225000 [223:11:53<450:08:35, 13.49s/it]
 47%|████▋     | 104905/225000 [223:12:01<393:17:59, 11.79s/it]
 47%|████▋     | 104906/225000 [223:12:04<313:22:52,  9.39s/it]
 47%|████▋     | 104907/225000 [223:12:17<343:58:50, 10.31s/it]
 47%|████▋     | 104908/225000 [223:12:24<311:41:51,  9.34s/it]
 47%|████▋     | 104909/225000 [223:12:37<349:30:51, 10.48s/it]
 47%|████▋     | 104910/225000 [223:12:45<325:44:29,  9.76s/it]
 47%|████▋     | 104911/225000 [223:13:02<401:56:29, 12.05s/it]
 47%|████▋     | 104912/225000 [223:13:15<405:19:07, 12.15s/it]
 47%|████▋     | 104913/225000 [223:13:22<356:14:51, 10.68s/it]
 47%|████▋     | 104914/225000 [223:13:30<327:09:06,  9.81s/it]
 47%|████▋     | 104915/225000 [223:13:41<343:05:19, 10.29s/it]
 47%|████▋     | 104916/225000 [223:13:54<365:37:15, 10.96s/it]
 47%|████▋     | 104917/225000 [223:14:06<374:20:36, 11.22s/it]
 47%|████▋     | 104918/225000 [223:14:11<317:00:09,  9.50s/it]
 47%|████▋     | 104919/225000 [223:14:18<292:31:51,  8.77s/it]
 47%|████▋     | 104920/225000 [223:14:26<283:51:00,  8.51s/it]
 47%|████▋     | 104921/225000 [223:14:35<285:20:23,  8.55s/it]
 47%|████▋     | 104922/225000 [223:14:42<268:30:50,  8.05s/it]
 47%|████▋     | 104923/225000 [223:14:56<331:28:51,  9.94s/it]
 47%|████▋     | 104924/225000 [223:14:59<261:27:24,  7.84s/it]
 47%|████▋     | 104925/225000 [223:15:06<254:24:54,  7.63s/it]
 47%|████▋     | 104926/225000 [223:15:20<319:33:51,  9.58s/it]
 47%|████▋     | 104927/225000 [223:15:24<263:43:30,  7.91s/it]
 47%|████▋     | 104928/225000 [223:15:37<315:13:35,  9.45s/it]
 47%|████▋     | 104929/225000 [223:15:48<330:10:53,  9.90s/it]
 47%|████▋     | 104930/225000 [223:15:57<321:32:40,  9.64s/it]
 47%|████▋     | 104931/225000 [223:16:12<368:21:21, 11.04s/it]
 47%|████▋     | 104932/225000 [223:16:23<370:05:34, 11.10s/it]
 47%|████▋     | 104933/225000 [223:16:44<473:04:16, 14.18s/it]
 47%|████▋     | 104934/225000 [223:16:51<402:53:03, 12.08s/it]
 47%|████▋     | 104935/225000 [223:16:57<337:54:21, 10.13s/it]
 47%|████▋     | 104936/225000 [223:17:00<266:25:06,  7.99s/it]
 47%|████▋     | 104937/225000 [223:17:09<273:35:14,  8.20s/it]
 47%|████▋     | 104938/225000 [223:17:16<260:45:06,  7.82s/it]
 47%|████▋     | 104939/225000 [223:17:22<245:37:50,  7.37s/it]
 47%|████▋     | 104940/225000 [223:17:26<217:52:36,  6.53s/it]
 47%|████▋     | 104941/225000 [223:17:44<323:07:47,  9.69s/it]
 47%|████▋     | 104942/225000 [223:18:02<408:53:53, 12.26s/it]
 47%|████▋     | 104943/225000 [223:18:07<336:40:24, 10.10s/it]
 47%|████▋     | 104944/225000 [223:18:12<290:03:47,  8.70s/it]
 47%|████▋     | 104945/225000 [223:18:22<303:07:35,  9.09s/it]
 47%|████▋     | 104946/225000 [223:18:27<256:20:25,  7.69s/it]
 47%|████▋     | 104947/225000 [223:18:42<336:02:38, 10.08s/it]
 47%|████▋     | 104948/225000 [223:18:57<378:00:25, 11.34s/it]
 47%|████▋     | 104949/225000 [223:19:06<362:44:13, 10.88s/it]
 47%|████▋     | 104950/225000 [223:19:19<378:16:46, 11.34s/it]
                                                               
{'loss': '0.9443', 'grad_norm': '2.284', 'learning_rate': '2.958e-05', 'epoch': '0.4664', 'num_input_tokens_seen': 374626776, 'train_runtime': '8.04e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 104950/225000 [223:19:19<378:16:46, 11.34s/it]
 47%|████▋     | 104951/225000 [223:19:29<363:22:14, 10.90s/it]
 47%|████▋     | 104952/225000 [223:19:32<283:48:13,  8.51s/it]
 47%|████▋     | 104953/225000 [223:19:40<286:47:08,  8.60s/it]
 47%|████▋     | 104954/225000 [223:19:54<332:12:56,  9.96s/it]
 47%|████▋     | 104955/225000 [223:20:10<399:56:13, 11.99s/it]
 47%|████▋     | 104956/225000 [223:20:23<411:16:54, 12.33s/it]
 47%|████▋     | 104957/225000 [223:20:29<342:45:15, 10.28s/it]
 47%|████▋     | 104958/225000 [223:20:40<352:45:34, 10.58s/it]
 47%|████▋     | 104959/225000 [223:20:46<302:45:28,  9.08s/it]
 47%|████▋     | 104960/225000 [223:21:01<366:39:55, 11.00s/it]
 47%|████▋     | 104961/225000 [223:21:08<328:27:12,  9.85s/it]
 47%|████▋     | 104962/225000 [223:21:18<328:15:02,  9.84s/it]
 47%|████▋     | 104963/225000 [223:21:35<393:10:08, 11.79s/it]
 47%|████▋     | 104964/225000 [223:21:40<326:09:01,  9.78s/it]
 47%|████▋     | 104965/225000 [223:21:43<262:22:28,  7.87s/it]
 47%|████▋     | 104966/225000 [223:21:59<340:10:46, 10.20s/it]
 47%|████▋     | 104967/225000 [223:22:10<347:35:30, 10.42s/it]
 47%|████▋     | 104968/225000 [223:22:18<328:17:13,  9.85s/it]
 47%|████▋     | 104969/225000 [223:22:33<379:54:41, 11.39s/it]
 47%|████▋     | 104970/225000 [223:22:40<333:50:01, 10.01s/it]
 47%|████▋     | 104971/225000 [223:22:47<299:19:08,  8.98s/it]
 47%|████▋     | 104972/225000 [223:22:54<285:33:05,  8.56s/it]
 47%|████▋     | 104973/225000 [223:23:11<372:13:09, 11.16s/it]
 47%|████▋     | 104974/225000 [223:23:33<472:25:10, 14.17s/it]
 47%|████▋     | 104975/225000 [223:23:36<360:35:58, 10.82s/it]
 47%|████▋     | 104976/225000 [223:23:44<334:04:43, 10.02s/it]
 47%|████▋     | 104977/225000 [223:23:53<325:20:56,  9.76s/it]
 47%|████▋     | 104978/225000 [223:23:56<262:39:05,  7.88s/it]
 47%|████▋     | 104979/225000 [223:23:59<208:46:28,  6.26s/it]
 47%|████▋     | 104980/225000 [223:24:14<302:17:53,  9.07s/it]
 47%|████▋     | 104981/225000 [223:24:21<278:02:49,  8.34s/it]
 47%|████▋     | 104982/225000 [223:24:39<374:55:09, 11.25s/it]
 47%|████▋     | 104983/225000 [223:24:46<330:50:32,  9.92s/it]
 47%|████▋     | 104984/225000 [223:24:48<255:09:59,  7.65s/it]
 47%|████▋     | 104985/225000 [223:24:58<272:54:29,  8.19s/it]
 47%|████▋     | 104986/225000 [223:25:10<315:44:23,  9.47s/it]
 47%|████▋     | 104987/225000 [223:25:29<404:58:13, 12.15s/it]
 47%|████▋     | 104988/225000 [223:25:41<408:46:10, 12.26s/it]
 47%|████▋     | 104989/225000 [223:25:53<406:39:50, 12.20s/it]
 47%|████▋     | 104990/225000 [223:26:00<352:01:04, 10.56s/it]
 47%|████▋     | 104991/225000 [223:26:09<340:43:40, 10.22s/it]
 47%|████▋     | 104992/225000 [223:26:22<365:02:24, 10.95s/it]
 47%|████▋     | 104993/225000 [223:26:30<334:39:10, 10.04s/it]
 47%|████▋     | 104994/225000 [223:26:36<290:13:20,  8.71s/it]
 47%|████▋     | 104995/225000 [223:26:43<277:11:28,  8.32s/it]
 47%|████▋     | 104996/225000 [223:26:45<217:56:11,  6.54s/it]
 47%|████▋     | 104997/225000 [223:27:01<310:55:15,  9.33s/it]
 47%|████▋     | 104998/225000 [223:27:09<294:31:40,  8.84s/it]
 47%|████▋     | 104999/225000 [223:27:13<244:34:51,  7.34s/it]
 47%|████▋     | 105000/225000 [223:27:35<398:53:45, 11.97s/it]
                                                               
{'loss': '0.9463', 'grad_norm': '1.31', 'learning_rate': '2.956e-05', 'epoch': '0.4667', 'num_input_tokens_seen': 374845504, 'train_runtime': '8.045e+05', 'train_tokens_per_second': '466'}

 47%|████▋     | 105000/225000 [223:27:35<398:53:45, 11.97s/it]
 47%|████▋     | 105001/225000 [223:27:40<320:37:17,  9.62s/it]
 47%|████▋     | 105002/225000 [223:27:45<281:54:54,  8.46s/it]
 47%|████▋     | 105003/225000 [223:28:04<384:30:59, 11.54s/it]
 47%|████▋     | 105004/225000 [223:28:16<392:36:19, 11.78s/it]
 47%|████▋     | 105005/225000 [223:28:29<400:23:54, 12.01s/it]
 47%|████▋     | 105006/225000 [223:28:39<385:02:48, 11.55s/it]
 47%|████▋     | 105007/225000 [223:28:46<335:37:28, 10.07s/it]
 47%|████▋     | 105008/225000 [223:28:49<265:08:50,  7.95s/it]
 47%|████▋     | 105009/225000 [223:28:59<283:25:01,  8.50s/it]
 47%|████▋     | 105010/225000 [223:29:07<282:43:16,  8.48s/it]
 47%|████▋     | 105011/225000 [223:29:30<426:33:03, 12.80s/it]
 47%|████▋     | 105012/225000 [223:29:36<360:06:50, 10.80s/it]
 47%|████▋     | 105013/225000 [223:29:39<284:06:17,  8.52s/it]
 47%|████▋     | 105014/225000 [223:29:48<288:40:59,  8.66s/it]
 47%|████▋     | 105015/225000 [223:29:56<276:00:00,  8.28s/it]
 47%|████▋     | 105016/225000 [223:30:03<264:29:55,  7.94s/it]
 47%|████▋     | 105017/225000 [223:30:09<240:42:07,  7.22s/it]
 47%|████▋     | 105018/225000 [223:30:24<319:10:24,  9.58s/it]
 47%|████▋     | 105019/225000 [223:30:32<305:01:03,  9.15s/it]
 47%|████▋     | 105020/225000 [223:30:40<298:27:58,  8.96s/it]
 47%|████▋     | 105021/225000 [223:30:47<273:08:21,  8.20s/it]
 47%|████▋     | 105022/225000 [223:30:52<241:04:18,  7.23s/it]
 47%|████▋     | 105023/225000 [223:31:04<290:43:55,  8.72s/it]
 47%|████▋     | 105024/225000 [223:31:17<337:57:43, 10.14s/it]
 47%|████▋     | 105025/225000 [223:31:32<384:15:26, 11.53s/it]
 47%|████▋     | 105026/225000 [223:31:36<308:38:09,  9.26s/it]
 47%|████▋     | 105027/225000 [223:31:42<271:04:22,  8.13s/it]
 47%|████▋     | 105028/225000 [223:31:59<363:30:34, 10.91s/it]
 47%|████▋     | 105029/225000 [223:32:05<317:48:07,  9.54s/it]
 47%|████▋     | 105030/225000 [223:32:14<308:52:37,  9.27s/it]
 47%|████▋     | 105031/225000 [223:32:24<320:57:02,  9.63s/it]
 47%|████▋     | 105032/225000 [223:32:30<281:15:32,  8.44s/it]
 47%|████▋     | 105033/225000 [223:32:36<258:48:45,  7.77s/it]
 47%|████▋     | 105034/225000 [223:32:50<314:55:36,  9.45s/it]
 47%|████▋     | 105035/225000 [223:33:00<321:59:00,  9.66s/it]
 47%|████▋     | 105036/225000 [223:33:05<281:01:16,  8.43s/it]
 47%|████▋     | 105037/225000 [223:33:12<258:05:23,  7.75s/it]
 47%|████▋     | 105038/225000 [223:33:29<356:50:29, 10.71s/it]
 47%|████▋     | 105039/225000 [223:33:38<336:23:44, 10.10s/it]
 47%|████▋     | 105040/225000 [223:33:51<369:44:57, 11.10s/it]
 47%|████▋     | 105041/225000 [223:34:00<343:26:19, 10.31s/it]
 47%|████▋     | 105042/225000 [223:34:08<324:39:44,  9.74s/it]
 47%|████▋     | 105043/225000 [223:34:18<325:17:24,  9.76s/it]
 47%|████▋     | 105044/225000 [223:34:27<318:19:12,  9.55s/it]
 47%|████▋     | 105045/225000 [223:34:37<318:11:10,  9.55s/it]
 47%|████▋     | 105046/225000 [223:34:39<252:11:19,  7.57s/it]
 47%|████▋     | 105047/225000 [223:34:45<227:46:01,  6.84s/it]
 47%|████▋     | 105048/225000 [223:34:56<277:40:51,  8.33s/it]
 47%|████▋     | 105049/225000 [223:35:09<318:14:28,  9.55s/it]
 47%|████▋     | 105050/225000 [223:35:17<303:39:27,  9.11s/it]
                                                               
{'loss': '0.9088', 'grad_norm': '1.91', 'learning_rate': '2.954e-05', 'epoch': '0.4669', 'num_input_tokens_seen': 375045624, 'train_runtime': '8.049e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105050/225000 [223:35:17<303:39:27,  9.11s/it]
 47%|████▋     | 105051/225000 [223:35:25<290:22:30,  8.71s/it]
 47%|████▋     | 105052/225000 [223:35:32<276:12:06,  8.29s/it]
 47%|████▋     | 105053/225000 [223:35:40<276:47:59,  8.31s/it]
 47%|████▋     | 105054/225000 [223:35:52<314:02:35,  9.43s/it]
 47%|████▋     | 105055/225000 [223:36:01<308:45:00,  9.27s/it]
 47%|████▋     | 105056/225000 [223:36:08<280:18:10,  8.41s/it]
 47%|████▋     | 105057/225000 [223:36:14<257:03:54,  7.72s/it]
 47%|████▋     | 105058/225000 [223:36:24<285:14:23,  8.56s/it]
 47%|████▋     | 105059/225000 [223:36:37<324:49:51,  9.75s/it]
 47%|████▋     | 105060/225000 [223:36:45<312:35:45,  9.38s/it]
 47%|████▋     | 105061/225000 [223:36:54<304:44:07,  9.15s/it]
 47%|████▋     | 105062/225000 [223:36:59<261:22:58,  7.85s/it]
 47%|████▋     | 105063/225000 [223:37:07<264:53:21,  7.95s/it]
 47%|████▋     | 105064/225000 [223:37:27<383:00:34, 11.50s/it]
 47%|████▋     | 105065/225000 [223:37:31<307:32:20,  9.23s/it]
 47%|████▋     | 105066/225000 [223:37:39<299:04:23,  8.98s/it]
 47%|████▋     | 105067/225000 [223:37:43<247:12:45,  7.42s/it]
 47%|████▋     | 105068/225000 [223:38:05<395:19:59, 11.87s/it]
 47%|████▋     | 105069/225000 [223:38:10<330:02:53,  9.91s/it]
 47%|████▋     | 105070/225000 [223:38:23<354:59:24, 10.66s/it]
 47%|████▋     | 105071/225000 [223:38:33<351:27:00, 10.55s/it]
 47%|████▋     | 105072/225000 [223:38:47<384:06:07, 11.53s/it]
 47%|████▋     | 105073/225000 [223:38:57<373:44:15, 11.22s/it]
 47%|████▋     | 105074/225000 [223:39:12<403:28:29, 12.11s/it]
 47%|████▋     | 105075/225000 [223:39:26<424:22:32, 12.74s/it]
 47%|████▋     | 105076/225000 [223:39:35<387:17:49, 11.63s/it]
 47%|████▋     | 105077/225000 [223:39:43<354:00:23, 10.63s/it]
 47%|████▋     | 105078/225000 [223:39:45<270:35:13,  8.12s/it]
 47%|████▋     | 105079/225000 [223:39:52<253:15:54,  7.60s/it]
 47%|████▋     | 105080/225000 [223:39:58<241:43:45,  7.26s/it]
 47%|████▋     | 105081/225000 [223:40:07<254:28:45,  7.64s/it]
 47%|████▋     | 105082/225000 [223:40:20<306:39:44,  9.21s/it]
 47%|████▋     | 105083/225000 [223:40:29<302:44:27,  9.09s/it]
 47%|████▋     | 105084/225000 [223:40:40<322:07:10,  9.67s/it]
 47%|████▋     | 105085/225000 [223:40:51<339:38:34, 10.20s/it]
 47%|████▋     | 105086/225000 [223:41:05<375:55:01, 11.29s/it]
 47%|████▋     | 105087/225000 [223:41:11<328:34:43,  9.86s/it]
 47%|████▋     | 105088/225000 [223:41:18<300:49:24,  9.03s/it]
 47%|████▋     | 105089/225000 [223:41:24<266:26:08,  8.00s/it]
 47%|████▋     | 105090/225000 [223:41:35<292:14:04,  8.77s/it]
 47%|████▋     | 105091/225000 [223:41:52<382:07:08, 11.47s/it]
 47%|████▋     | 105092/225000 [223:41:57<314:42:38,  9.45s/it]
 47%|████▋     | 105093/225000 [223:42:07<322:05:42,  9.67s/it]
 47%|████▋     | 105094/225000 [223:42:22<377:02:29, 11.32s/it]
 47%|████▋     | 105095/225000 [223:42:35<386:40:33, 11.61s/it]
 47%|████▋     | 105096/225000 [223:42:51<429:59:16, 12.91s/it]
 47%|████▋     | 105097/225000 [223:42:54<331:01:59,  9.94s/it]
 47%|████▋     | 105098/225000 [223:43:05<343:50:15, 10.32s/it]
 47%|████▋     | 105099/225000 [223:43:17<357:29:49, 10.73s/it]
 47%|████▋     | 105100/225000 [223:43:28<367:46:47, 11.04s/it]
                                                               
{'loss': '0.9312', 'grad_norm': '1.726', 'learning_rate': '2.952e-05', 'epoch': '0.4671', 'num_input_tokens_seen': 375267720, 'train_runtime': '8.054e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105100/225000 [223:43:28<367:46:47, 11.04s/it]
 47%|████▋     | 105101/225000 [223:43:41<379:05:35, 11.38s/it]
 47%|████▋     | 105102/225000 [223:43:51<366:14:29, 11.00s/it]
 47%|████▋     | 105103/225000 [223:44:01<356:57:07, 10.72s/it]
 47%|████▋     | 105104/225000 [223:44:13<374:42:44, 11.25s/it]
 47%|████▋     | 105105/225000 [223:44:21<337:19:35, 10.13s/it]
 47%|████▋     | 105106/225000 [223:44:31<339:01:40, 10.18s/it]
 47%|████▋     | 105107/225000 [223:44:38<303:47:19,  9.12s/it]
 47%|████▋     | 105108/225000 [223:44:46<292:09:48,  8.77s/it]
 47%|████▋     | 105109/225000 [223:44:54<289:44:09,  8.70s/it]
 47%|████▋     | 105110/225000 [223:45:14<401:20:27, 12.05s/it]
 47%|████▋     | 105111/225000 [223:45:27<413:31:07, 12.42s/it]
 47%|████▋     | 105112/225000 [223:45:43<443:09:18, 13.31s/it]
 47%|████▋     | 105113/225000 [223:45:50<387:37:51, 11.64s/it]
 47%|████▋     | 105114/225000 [223:46:00<364:28:41, 10.94s/it]
 47%|████▋     | 105115/225000 [223:46:04<301:15:37,  9.05s/it]
 47%|████▋     | 105116/225000 [223:46:08<243:12:53,  7.30s/it]
 47%|████▋     | 105117/225000 [223:46:21<303:56:50,  9.13s/it]
 47%|████▋     | 105118/225000 [223:46:29<288:42:29,  8.67s/it]
 47%|████▋     | 105119/225000 [223:46:38<297:41:57,  8.94s/it]
 47%|████▋     | 105120/225000 [223:46:42<242:19:58,  7.28s/it]
 47%|████▋     | 105121/225000 [223:46:58<333:06:39, 10.00s/it]
 47%|████▋     | 105122/225000 [223:47:06<316:46:04,  9.51s/it]
 47%|████▋     | 105123/225000 [223:47:12<280:26:19,  8.42s/it]
 47%|████▋     | 105124/225000 [223:47:15<227:04:38,  6.82s/it]
 47%|████▋     | 105125/225000 [223:47:25<260:50:17,  7.83s/it]
 47%|████▋     | 105126/225000 [223:47:39<313:58:17,  9.43s/it]
 47%|████▋     | 105127/225000 [223:47:55<381:49:15, 11.47s/it]
 47%|████▋     | 105128/225000 [223:48:00<323:10:45,  9.71s/it]
 47%|████▋     | 105129/225000 [223:48:18<399:44:38, 12.01s/it]
 47%|████▋     | 105130/225000 [223:48:21<311:29:06,  9.35s/it]
 47%|████▋     | 105131/225000 [223:48:27<279:55:15,  8.41s/it]
 47%|████▋     | 105132/225000 [223:48:37<295:00:34,  8.86s/it]
 47%|████▋     | 105133/225000 [223:48:42<252:48:50,  7.59s/it]
 47%|████▋     | 105134/225000 [223:48:50<264:01:32,  7.93s/it]
 47%|████▋     | 105135/225000 [223:49:09<373:52:04, 11.23s/it]
 47%|████▋     | 105136/225000 [223:49:26<430:46:43, 12.94s/it]
 47%|████▋     | 105137/225000 [223:49:34<382:25:16, 11.49s/it]
 47%|████▋     | 105138/225000 [223:49:42<343:29:20, 10.32s/it]
 47%|████▋     | 105139/225000 [223:49:54<358:50:43, 10.78s/it]
 47%|████▋     | 105140/225000 [223:50:04<351:33:49, 10.56s/it]
 47%|████▋     | 105141/225000 [223:50:21<413:36:38, 12.42s/it]
 47%|████▋     | 105142/225000 [223:50:28<365:50:49, 10.99s/it]
 47%|████▋     | 105143/225000 [223:50:47<443:21:51, 13.32s/it]
 47%|████▋     | 105144/225000 [223:50:52<360:22:44, 10.82s/it]
 47%|████▋     | 105145/225000 [223:51:04<371:57:29, 11.17s/it]
 47%|████▋     | 105146/225000 [223:51:15<374:08:14, 11.24s/it]
 47%|████▋     | 105147/225000 [223:51:31<421:51:36, 12.67s/it]
 47%|████▋     | 105148/225000 [223:51:42<398:18:20, 11.96s/it]
 47%|████▋     | 105149/225000 [223:51:54<404:39:02, 12.15s/it]
 47%|████▋     | 105150/225000 [223:51:58<324:24:08,  9.74s/it]
                                                               
{'loss': '0.9438', 'grad_norm': '2.266', 'learning_rate': '2.951e-05', 'epoch': '0.4673', 'num_input_tokens_seen': 375507584, 'train_runtime': '8.059e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105150/225000 [223:51:58<324:24:08,  9.74s/it]
 47%|████▋     | 105151/225000 [223:52:02<259:35:32,  7.80s/it]
 47%|████▋     | 105152/225000 [223:52:11<274:07:20,  8.23s/it]
 47%|████▋     | 105153/225000 [223:52:19<272:06:58,  8.17s/it]
 47%|████▋     | 105154/225000 [223:52:28<276:27:56,  8.30s/it]
 47%|████▋     | 105155/225000 [223:52:41<325:38:40,  9.78s/it]
 47%|████▋     | 105156/225000 [223:52:50<323:08:48,  9.71s/it]
 47%|████▋     | 105157/225000 [223:52:56<282:42:34,  8.49s/it]
 47%|████▋     | 105158/225000 [223:53:09<330:07:08,  9.92s/it]
 47%|████▋     | 105159/225000 [223:53:16<301:02:20,  9.04s/it]
 47%|████▋     | 105160/225000 [223:53:28<330:12:39,  9.92s/it]
 47%|████▋     | 105161/225000 [223:53:37<320:03:38,  9.61s/it]
 47%|████▋     | 105162/225000 [223:53:41<259:50:24,  7.81s/it]
 47%|████▋     | 105163/225000 [223:53:47<248:48:22,  7.47s/it]
 47%|████▋     | 105164/225000 [223:53:58<275:38:13,  8.28s/it]
 47%|████▋     | 105165/225000 [223:54:07<286:55:02,  8.62s/it]
 47%|████▋     | 105166/225000 [223:54:22<346:18:57, 10.40s/it]
 47%|████▋     | 105167/225000 [223:54:32<341:46:48, 10.27s/it]
 47%|████▋     | 105168/225000 [223:54:41<336:17:34, 10.10s/it]
 47%|████▋     | 105169/225000 [223:54:47<296:34:42,  8.91s/it]
 47%|████▋     | 105170/225000 [223:54:57<304:59:24,  9.16s/it]
 47%|████▋     | 105171/225000 [223:55:05<288:58:36,  8.68s/it]
 47%|████▋     | 105172/225000 [223:55:15<309:16:55,  9.29s/it]
 47%|████▋     | 105173/225000 [223:55:19<248:15:10,  7.46s/it]
 47%|████▋     | 105174/225000 [223:55:23<220:12:12,  6.62s/it]
 47%|████▋     | 105175/225000 [223:55:34<264:20:13,  7.94s/it]
 47%|████▋     | 105176/225000 [223:55:53<376:19:06, 11.31s/it]
 47%|████▋     | 105177/225000 [223:56:01<335:04:00, 10.07s/it]
 47%|████▋     | 105178/225000 [223:56:15<383:08:00, 11.51s/it]
 47%|████▋     | 105179/225000 [223:56:20<314:22:19,  9.45s/it]
 47%|████▋     | 105180/225000 [223:56:28<298:21:28,  8.96s/it]
 47%|████▋     | 105181/225000 [223:56:36<285:55:22,  8.59s/it]
 47%|████▋     | 105182/225000 [223:56:43<276:36:26,  8.31s/it]
 47%|████▋     | 105183/225000 [223:56:53<288:52:04,  8.68s/it]
 47%|████▋     | 105184/225000 [223:56:57<241:22:24,  7.25s/it]
 47%|████▋     | 105185/225000 [223:57:16<363:30:43, 10.92s/it]
 47%|████▋     | 105186/225000 [223:57:25<336:59:33, 10.13s/it]
 47%|████▋     | 105187/225000 [223:57:31<296:11:42,  8.90s/it]
 47%|████▋     | 105188/225000 [223:57:41<314:52:24,  9.46s/it]
 47%|████▋     | 105189/225000 [223:57:47<279:52:54,  8.41s/it]
 47%|████▋     | 105190/225000 [223:57:52<241:20:22,  7.25s/it]
 47%|████▋     | 105191/225000 [223:57:58<231:20:46,  6.95s/it]
 47%|████▋     | 105192/225000 [223:58:15<331:26:46,  9.96s/it]
 47%|████▋     | 105193/225000 [223:58:34<418:35:26, 12.58s/it]
 47%|████▋     | 105194/225000 [223:58:41<368:31:02, 11.07s/it]
 47%|████▋     | 105195/225000 [223:58:45<291:57:36,  8.77s/it]
 47%|████▋     | 105196/225000 [223:58:58<336:04:01, 10.10s/it]
 47%|████▋     | 105197/225000 [223:59:06<320:35:21,  9.63s/it]
 47%|████▋     | 105198/225000 [223:59:20<354:54:42, 10.66s/it]
 47%|████▋     | 105199/225000 [223:59:31<361:58:39, 10.88s/it]
 47%|████▋     | 105200/225000 [223:59:49<432:38:47, 13.00s/it]
                                                               
{'loss': '0.9252', 'grad_norm': '1.819', 'learning_rate': '2.949e-05', 'epoch': '0.4676', 'num_input_tokens_seen': 375718552, 'train_runtime': '8.064e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105200/225000 [223:59:49<432:38:47, 13.00s/it]
 47%|████▋     | 105201/225000 [223:59:57<388:31:51, 11.68s/it]
 47%|████▋     | 105202/225000 [224:00:07<371:17:21, 11.16s/it]
 47%|████▋     | 105203/225000 [224:00:13<318:55:02,  9.58s/it]
 47%|████▋     | 105204/225000 [224:00:26<351:35:18, 10.57s/it]
 47%|████▋     | 105205/225000 [224:00:30<287:22:24,  8.64s/it]
 47%|████▋     | 105206/225000 [224:00:45<351:32:18, 10.56s/it]
 47%|████▋     | 105207/225000 [224:00:55<345:47:02, 10.39s/it]
 47%|████▋     | 105208/225000 [224:01:06<351:48:11, 10.57s/it]
 47%|████▋     | 105209/225000 [224:01:12<306:19:10,  9.21s/it]
 47%|████▋     | 105210/225000 [224:01:22<307:08:06,  9.23s/it]
 47%|████▋     | 105211/225000 [224:01:31<311:42:22,  9.37s/it]
 47%|████▋     | 105212/225000 [224:01:40<304:11:27,  9.14s/it]
 47%|████▋     | 105213/225000 [224:01:57<383:35:13, 11.53s/it]
 47%|████▋     | 105214/225000 [224:02:14<437:11:08, 13.14s/it]
 47%|████▋     | 105215/225000 [224:02:18<349:55:54, 10.52s/it]
 47%|████▋     | 105216/225000 [224:02:22<285:01:22,  8.57s/it]
 47%|████▋     | 105217/225000 [224:02:28<258:33:15,  7.77s/it]
 47%|████▋     | 105218/225000 [224:02:43<331:19:25,  9.96s/it]
 47%|████▋     | 105219/225000 [224:02:46<261:06:51,  7.85s/it]
 47%|████▋     | 105220/225000 [224:02:53<252:43:45,  7.60s/it]
 47%|████▋     | 105221/225000 [224:03:08<319:03:42,  9.59s/it]
 47%|████▋     | 105222/225000 [224:03:21<359:51:41, 10.82s/it]
 47%|████▋     | 105223/225000 [224:03:26<303:53:27,  9.13s/it]
 47%|████▋     | 105224/225000 [224:03:42<365:56:16, 11.00s/it]
 47%|████▋     | 105225/225000 [224:03:47<309:10:24,  9.29s/it]
 47%|████▋     | 105226/225000 [224:03:55<297:33:34,  8.94s/it]
 47%|████▋     | 105227/225000 [224:04:01<262:10:16,  7.88s/it]
 47%|████▋     | 105228/225000 [224:04:18<355:58:17, 10.70s/it]
 47%|████▋     | 105229/225000 [224:04:28<348:09:16, 10.46s/it]
 47%|████▋     | 105230/225000 [224:04:41<374:38:15, 11.26s/it]
 47%|████▋     | 105231/225000 [224:04:51<365:09:42, 10.98s/it]
 47%|████▋     | 105232/225000 [224:05:01<349:00:44, 10.49s/it]
 47%|████▋     | 105233/225000 [224:05:16<400:12:42, 12.03s/it]
 47%|████▋     | 105234/225000 [224:05:27<390:47:14, 11.75s/it]
 47%|████▋     | 105235/225000 [224:05:38<376:22:51, 11.31s/it]
 47%|████▋     | 105236/225000 [224:05:41<301:35:43,  9.07s/it]
 47%|████▋     | 105237/225000 [224:05:56<361:02:57, 10.85s/it]
 47%|████▋     | 105238/225000 [224:06:01<297:50:15,  8.95s/it]
 47%|████▋     | 105239/225000 [224:06:08<280:36:56,  8.44s/it]
 47%|████▋     | 105240/225000 [224:06:19<305:47:04,  9.19s/it]
 47%|████▋     | 105241/225000 [224:06:27<288:08:30,  8.66s/it]
 47%|████▋     | 105242/225000 [224:06:34<277:36:16,  8.34s/it]
 47%|████▋     | 105243/225000 [224:06:52<368:41:11, 11.08s/it]
 47%|████▋     | 105244/225000 [224:07:09<435:41:00, 13.10s/it]
 47%|████▋     | 105245/225000 [224:07:15<358:58:15, 10.79s/it]
 47%|████▋     | 105246/225000 [224:07:29<392:29:11, 11.80s/it]
 47%|████▋     | 105247/225000 [224:07:39<371:08:46, 11.16s/it]
 47%|████▋     | 105248/225000 [224:07:44<313:51:39,  9.44s/it]
 47%|████▋     | 105249/225000 [224:07:59<365:53:27, 11.00s/it]
 47%|████▋     | 105250/225000 [224:08:15<423:27:56, 12.73s/it]
                                                               
{'loss': '0.9393', 'grad_norm': '1.636', 'learning_rate': '2.947e-05', 'epoch': '0.4678', 'num_input_tokens_seen': 375936576, 'train_runtime': '8.069e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105250/225000 [224:08:15<423:27:56, 12.73s/it]
 47%|████▋     | 105251/225000 [224:08:21<352:05:19, 10.58s/it]
 47%|████▋     | 105252/225000 [224:08:27<302:10:14,  9.08s/it]
 47%|████▋     | 105253/225000 [224:08:39<335:02:52, 10.07s/it]
 47%|████▋     | 105254/225000 [224:08:44<289:02:43,  8.69s/it]
 47%|████▋     | 105255/225000 [224:09:06<417:21:44, 12.55s/it]
 47%|████▋     | 105256/225000 [224:09:18<416:22:06, 12.52s/it]
 47%|████▋     | 105257/225000 [224:09:27<372:27:10, 11.20s/it]
 47%|████▋     | 105258/225000 [224:09:32<312:53:20,  9.41s/it]
 47%|████▋     | 105259/225000 [224:09:36<258:32:28,  7.77s/it]
 47%|████▋     | 105260/225000 [224:09:43<253:33:25,  7.62s/it]
 47%|████▋     | 105261/225000 [224:09:57<317:41:31,  9.55s/it]
 47%|████▋     | 105262/225000 [224:10:00<247:22:54,  7.44s/it]
 47%|████▋     | 105263/225000 [224:10:03<210:33:51,  6.33s/it]
 47%|████▋     | 105264/225000 [224:10:21<323:14:15,  9.72s/it]
 47%|████▋     | 105265/225000 [224:10:27<290:32:54,  8.74s/it]
 47%|████▋     | 105266/225000 [224:10:42<353:40:00, 10.63s/it]
 47%|████▋     | 105267/225000 [224:11:04<464:39:10, 13.97s/it]
 47%|████▋     | 105268/225000 [224:11:07<356:43:37, 10.73s/it]
 47%|████▋     | 105269/225000 [224:11:11<288:44:51,  8.68s/it]
 47%|████▋     | 105270/225000 [224:11:18<269:18:08,  8.10s/it]
 47%|████▋     | 105271/225000 [224:11:28<285:48:33,  8.59s/it]
 47%|████▋     | 105272/225000 [224:11:38<300:18:44,  9.03s/it]
 47%|████▋     | 105273/225000 [224:11:46<292:55:47,  8.81s/it]
 47%|████▋     | 105274/225000 [224:11:57<311:41:44,  9.37s/it]
 47%|████▋     | 105275/225000 [224:12:00<244:52:36,  7.36s/it]
 47%|████▋     | 105276/225000 [224:12:08<251:11:31,  7.55s/it]
 47%|████▋     | 105277/225000 [224:12:13<234:51:59,  7.06s/it]
 47%|████▋     | 105278/225000 [224:12:27<296:11:05,  8.91s/it]
 47%|████▋     | 105279/225000 [224:12:45<387:26:32, 11.65s/it]
 47%|████▋     | 105280/225000 [224:12:53<352:25:03, 10.60s/it]
 47%|████▋     | 105281/225000 [224:13:04<358:57:08, 10.79s/it]
 47%|████▋     | 105282/225000 [224:13:08<288:56:28,  8.69s/it]
 47%|████▋     | 105283/225000 [224:13:16<280:03:45,  8.42s/it]
 47%|████▋     | 105284/225000 [224:13:29<331:38:57,  9.97s/it]
 47%|████▋     | 105285/225000 [224:13:37<310:42:46,  9.34s/it]
 47%|████▋     | 105286/225000 [224:13:49<338:26:05, 10.18s/it]
 47%|████▋     | 105287/225000 [224:13:58<327:07:42,  9.84s/it]
 47%|████▋     | 105288/225000 [224:14:05<295:52:20,  8.90s/it]
 47%|████▋     | 105289/225000 [224:14:24<395:48:58, 11.90s/it]
 47%|████▋     | 105290/225000 [224:14:33<367:05:32, 11.04s/it]
 47%|████▋     | 105291/225000 [224:14:45<375:14:09, 11.28s/it]
 47%|████▋     | 105292/225000 [224:14:51<323:43:31,  9.74s/it]
 47%|████▋     | 105293/225000 [224:15:04<358:34:57, 10.78s/it]
 47%|████▋     | 105294/225000 [224:15:15<360:58:56, 10.86s/it]
 47%|████▋     | 105295/225000 [224:15:22<317:36:57,  9.55s/it]
 47%|████▋     | 105296/225000 [224:15:38<389:20:01, 11.71s/it]
 47%|████▋     | 105297/225000 [224:15:56<447:55:53, 13.47s/it]
 47%|████▋     | 105298/225000 [224:16:01<363:51:10, 10.94s/it]
 47%|████▋     | 105299/225000 [224:16:08<328:59:52,  9.89s/it]
 47%|████▋     | 105300/225000 [224:16:20<342:22:37, 10.30s/it]
                                                               
{'loss': '0.9411', 'grad_norm': '1.518', 'learning_rate': '2.945e-05', 'epoch': '0.468', 'num_input_tokens_seen': 376148624, 'train_runtime': '8.074e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105300/225000 [224:16:20<342:22:37, 10.30s/it]
 47%|████▋     | 105301/225000 [224:16:27<309:19:13,  9.30s/it]
 47%|████▋     | 105302/225000 [224:16:37<315:40:12,  9.49s/it]
 47%|████▋     | 105303/225000 [224:16:49<340:31:07, 10.24s/it]
 47%|████▋     | 105304/225000 [224:16:59<338:54:33, 10.19s/it]
 47%|████▋     | 105305/225000 [224:17:05<304:52:54,  9.17s/it]
 47%|████▋     | 105306/225000 [224:17:18<334:45:00, 10.07s/it]
 47%|████▋     | 105307/225000 [224:17:21<263:48:51,  7.93s/it]
 47%|████▋     | 105308/225000 [224:17:29<264:53:10,  7.97s/it]
 47%|████▋     | 105309/225000 [224:17:39<289:59:36,  8.72s/it]
 47%|████▋     | 105310/225000 [224:17:47<276:43:18,  8.32s/it]
 47%|████▋     | 105311/225000 [224:18:02<349:03:33, 10.50s/it]
 47%|████▋     | 105312/225000 [224:18:11<337:33:01, 10.15s/it]
 47%|████▋     | 105313/225000 [224:18:24<365:41:38, 11.00s/it]
 47%|████▋     | 105314/225000 [224:18:30<315:14:20,  9.48s/it]
 47%|████▋     | 105315/225000 [224:18:36<276:00:57,  8.30s/it]
 47%|████▋     | 105316/225000 [224:18:45<279:54:37,  8.42s/it]
 47%|████▋     | 105317/225000 [224:18:54<292:18:39,  8.79s/it]
 47%|████▋     | 105318/225000 [224:19:05<307:54:32,  9.26s/it]
 47%|████▋     | 105319/225000 [224:19:14<307:52:46,  9.26s/it]
 47%|████▋     | 105320/225000 [224:19:23<307:22:10,  9.25s/it]
 47%|████▋     | 105321/225000 [224:19:28<262:25:27,  7.89s/it]
 47%|████▋     | 105322/225000 [224:19:38<284:58:27,  8.57s/it]
 47%|████▋     | 105323/225000 [224:19:43<247:32:40,  7.45s/it]
 47%|████▋     | 105324/225000 [224:19:48<229:42:04,  6.91s/it]
 47%|████▋     | 105325/225000 [224:20:01<285:52:27,  8.60s/it]
 47%|████▋     | 105326/225000 [224:20:10<286:27:40,  8.62s/it]
 47%|████▋     | 105327/225000 [224:20:17<271:47:13,  8.18s/it]
 47%|████▋     | 105328/225000 [224:20:23<248:36:01,  7.48s/it]
 47%|████▋     | 105329/225000 [224:20:34<282:49:34,  8.51s/it]
 47%|████▋     | 105330/225000 [224:20:38<240:54:09,  7.25s/it]
 47%|████▋     | 105331/225000 [224:20:45<234:46:07,  7.06s/it]
 47%|████▋     | 105332/225000 [224:20:56<282:39:15,  8.50s/it]
 47%|████▋     | 105333/225000 [224:21:04<275:09:52,  8.28s/it]
 47%|████▋     | 105334/225000 [224:21:10<254:04:37,  7.64s/it]
 47%|████▋     | 105335/225000 [224:21:16<231:37:22,  6.97s/it]
 47%|████▋     | 105336/225000 [224:21:27<272:33:40,  8.20s/it]
 47%|████▋     | 105337/225000 [224:21:43<356:05:25, 10.71s/it]
 47%|████▋     | 105338/225000 [224:21:49<305:04:43,  9.18s/it]
 47%|████▋     | 105339/225000 [224:22:04<366:20:09, 11.02s/it]
 47%|████▋     | 105340/225000 [224:22:13<339:34:10, 10.22s/it]
 47%|████▋     | 105341/225000 [224:22:20<309:56:39,  9.32s/it]
 47%|████▋     | 105342/225000 [224:22:26<281:44:52,  8.48s/it]
 47%|████▋     | 105343/225000 [224:22:37<307:24:44,  9.25s/it]
 47%|████▋     | 105344/225000 [224:22:46<304:40:18,  9.17s/it]
 47%|████▋     | 105345/225000 [224:22:52<267:15:56,  8.04s/it]
 47%|████▋     | 105346/225000 [224:23:00<269:12:40,  8.10s/it]
 47%|████▋     | 105347/225000 [224:23:10<287:14:21,  8.64s/it]
 47%|████▋     | 105348/225000 [224:23:25<356:09:42, 10.72s/it]
 47%|████▋     | 105349/225000 [224:23:31<307:56:14,  9.27s/it]
 47%|████▋     | 105350/225000 [224:23:37<268:28:18,  8.08s/it]
                                                               
{'loss': '0.8976', 'grad_norm': '1.855', 'learning_rate': '2.943e-05', 'epoch': '0.4682', 'num_input_tokens_seen': 376355824, 'train_runtime': '8.078e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105350/225000 [224:23:37<268:28:18,  8.08s/it]
 47%|████▋     | 105351/225000 [224:23:51<329:22:22,  9.91s/it]
 47%|████▋     | 105352/225000 [224:23:56<284:51:01,  8.57s/it]
 47%|████▋     | 105353/225000 [224:24:03<262:55:37,  7.91s/it]
 47%|████▋     | 105354/225000 [224:24:14<294:24:18,  8.86s/it]
 47%|████▋     | 105355/225000 [224:24:17<237:05:01,  7.13s/it]
 47%|████▋     | 105356/225000 [224:24:35<350:50:46, 10.56s/it]
 47%|████▋     | 105357/225000 [224:24:44<333:16:11, 10.03s/it]
 47%|████▋     | 105358/225000 [224:24:47<265:39:08,  7.99s/it]
 47%|████▋     | 105359/225000 [224:24:50<211:43:22,  6.37s/it]
 47%|████▋     | 105360/225000 [224:25:13<378:07:06, 11.38s/it]
 47%|████▋     | 105361/225000 [224:25:27<405:02:16, 12.19s/it]
 47%|████▋     | 105362/225000 [224:25:40<416:14:01, 12.52s/it]
 47%|████▋     | 105363/225000 [224:25:54<427:38:08, 12.87s/it]
 47%|████▋     | 105364/225000 [224:25:58<338:17:20, 10.18s/it]
 47%|████▋     | 105365/225000 [224:26:02<272:57:31,  8.21s/it]
 47%|████▋     | 105366/225000 [224:26:08<250:57:25,  7.55s/it]
 47%|████▋     | 105367/225000 [224:26:20<295:02:12,  8.88s/it]
 47%|████▋     | 105368/225000 [224:26:25<260:23:05,  7.84s/it]
 47%|████▋     | 105369/225000 [224:26:30<236:21:16,  7.11s/it]
 47%|████▋     | 105370/225000 [224:26:35<206:29:26,  6.21s/it]
 47%|████▋     | 105371/225000 [224:26:46<254:13:04,  7.65s/it]
 47%|████▋     | 105372/225000 [224:26:52<238:33:01,  7.18s/it]
 47%|████▋     | 105373/225000 [224:27:03<278:33:02,  8.38s/it]
 47%|████▋     | 105374/225000 [224:27:07<232:47:31,  7.01s/it]
 47%|████▋     | 105375/225000 [224:27:11<205:21:21,  6.18s/it]
 47%|████▋     | 105376/225000 [224:27:20<235:25:00,  7.08s/it]
 47%|████▋     | 105377/225000 [224:27:28<243:22:52,  7.32s/it]
 47%|████▋     | 105378/225000 [224:27:41<299:08:46,  9.00s/it]
 47%|████▋     | 105379/225000 [224:27:46<261:04:38,  7.86s/it]
 47%|████▋     | 105380/225000 [224:27:56<280:15:20,  8.43s/it]
 47%|████▋     | 105381/225000 [224:28:13<366:47:34, 11.04s/it]
 47%|████▋     | 105382/225000 [224:28:16<288:35:51,  8.69s/it]
 47%|████▋     | 105383/225000 [224:28:28<322:40:01,  9.71s/it]
 47%|████▋     | 105384/225000 [224:28:35<290:49:44,  8.75s/it]
 47%|████▋     | 105385/225000 [224:28:49<347:59:10, 10.47s/it]
 47%|████▋     | 105386/225000 [224:28:54<287:04:36,  8.64s/it]
 47%|████▋     | 105387/225000 [224:28:58<247:57:26,  7.46s/it]
 47%|████▋     | 105388/225000 [224:29:06<250:15:47,  7.53s/it]
 47%|████▋     | 105389/225000 [224:29:15<263:09:46,  7.92s/it]
 47%|████▋     | 105390/225000 [224:29:22<257:30:17,  7.75s/it]
 47%|████▋     | 105391/225000 [224:29:29<247:46:44,  7.46s/it]
 47%|████▋     | 105392/225000 [224:29:35<236:13:05,  7.11s/it]
 47%|████▋     | 105393/225000 [224:29:39<197:24:06,  5.94s/it]
 47%|████▋     | 105394/225000 [224:29:51<267:15:23,  8.04s/it]
 47%|████▋     | 105395/225000 [224:30:01<282:13:10,  8.49s/it]
 47%|████▋     | 105396/225000 [224:30:11<296:51:42,  8.94s/it]
 47%|████▋     | 105397/225000 [224:30:20<297:33:58,  8.96s/it]
 47%|████▋     | 105398/225000 [224:30:26<265:27:18,  7.99s/it]
 47%|████▋     | 105399/225000 [224:30:33<255:17:58,  7.68s/it]
 47%|████▋     | 105400/225000 [224:30:48<331:31:06,  9.98s/it]
                                                               
{'loss': '0.9389', 'grad_norm': '1.692', 'learning_rate': '2.942e-05', 'epoch': '0.4684', 'num_input_tokens_seen': 376549928, 'train_runtime': '8.082e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105400/225000 [224:30:48<331:31:06,  9.98s/it]
 47%|████▋     | 105401/225000 [224:30:58<335:22:24, 10.09s/it]
 47%|████▋     | 105402/225000 [224:31:09<345:00:37, 10.39s/it]
 47%|████▋     | 105403/225000 [224:31:14<283:53:16,  8.55s/it]
 47%|████▋     | 105404/225000 [224:31:18<237:33:34,  7.15s/it]
 47%|████▋     | 105405/225000 [224:31:33<321:10:39,  9.67s/it]
 47%|████▋     | 105406/225000 [224:31:38<276:24:51,  8.32s/it]
 47%|████▋     | 105407/225000 [224:31:47<282:15:21,  8.50s/it]
 47%|████▋     | 105408/225000 [224:32:03<350:01:24, 10.54s/it]
 47%|████▋     | 105409/225000 [224:32:16<375:04:20, 11.29s/it]
 47%|████▋     | 105410/225000 [224:32:33<434:34:26, 13.08s/it]
 47%|████▋     | 105411/225000 [224:32:38<355:11:49, 10.69s/it]
 47%|████▋     | 105412/225000 [224:32:41<278:38:50,  8.39s/it]
 47%|████▋     | 105413/225000 [224:32:43<217:58:24,  6.56s/it]
 47%|████▋     | 105414/225000 [224:32:50<218:38:40,  6.58s/it]
 47%|████▋     | 105415/225000 [224:32:56<217:42:28,  6.55s/it]
 47%|████▋     | 105416/225000 [224:32:59<175:30:51,  5.28s/it]
 47%|████▋     | 105417/225000 [224:33:05<181:20:44,  5.46s/it]
 47%|████▋     | 105418/225000 [224:33:18<258:59:40,  7.80s/it]
 47%|████▋     | 105419/225000 [224:33:33<332:08:09, 10.00s/it]
 47%|████▋     | 105420/225000 [224:33:42<323:22:39,  9.74s/it]
 47%|████▋     | 105421/225000 [224:33:52<320:51:58,  9.66s/it]
 47%|████▋     | 105422/225000 [224:34:02<329:15:19,  9.91s/it]
 47%|████▋     | 105423/225000 [224:34:12<333:02:21, 10.03s/it]
 47%|████▋     | 105424/225000 [224:34:30<412:15:17, 12.41s/it]
 47%|████▋     | 105425/225000 [224:34:45<438:43:54, 13.21s/it]
 47%|████▋     | 105426/225000 [224:34:49<343:59:41, 10.36s/it]
 47%|████▋     | 105427/225000 [224:34:57<320:12:24,  9.64s/it]
 47%|████▋     | 105428/225000 [224:35:12<375:56:36, 11.32s/it]
 47%|████▋     | 105429/225000 [224:35:25<385:04:33, 11.59s/it]
 47%|████▋     | 105430/225000 [224:35:36<383:35:53, 11.55s/it]
 47%|████▋     | 105431/225000 [224:35:47<373:58:39, 11.26s/it]
 47%|████▋     | 105432/225000 [224:35:55<347:46:18, 10.47s/it]
 47%|████▋     | 105433/225000 [224:36:04<331:46:09,  9.99s/it]
 47%|████▋     | 105434/225000 [224:36:06<255:45:04,  7.70s/it]
 47%|████▋     | 105435/225000 [224:36:14<259:06:02,  7.80s/it]
 47%|████▋     | 105436/225000 [224:36:22<253:44:16,  7.64s/it]
 47%|████▋     | 105437/225000 [224:36:26<215:59:59,  6.50s/it]
 47%|████▋     | 105438/225000 [224:36:32<213:13:06,  6.42s/it]
 47%|████▋     | 105439/225000 [224:36:40<227:19:49,  6.84s/it]
 47%|████▋     | 105440/225000 [224:36:51<276:19:12,  8.32s/it]
 47%|████▋     | 105441/225000 [224:37:01<288:09:29,  8.68s/it]
 47%|████▋     | 105442/225000 [224:37:11<301:01:48,  9.06s/it]
 47%|████▋     | 105443/225000 [224:37:16<257:17:45,  7.75s/it]
 47%|████▋     | 105444/225000 [224:37:24<259:44:29,  7.82s/it]
 47%|████▋     | 105445/225000 [224:37:36<310:16:47,  9.34s/it]
 47%|████▋     | 105446/225000 [224:37:41<263:34:19,  7.94s/it]
 47%|████▋     | 105447/225000 [224:37:45<221:02:53,  6.66s/it]
 47%|████▋     | 105448/225000 [224:37:57<276:50:21,  8.34s/it]
 47%|████▋     | 105449/225000 [224:38:02<239:37:34,  7.22s/it]
 47%|████▋     | 105450/225000 [224:38:12<271:27:06,  8.17s/it]
                                                               
{'loss': '0.917', 'grad_norm': '2.28', 'learning_rate': '2.94e-05', 'epoch': '0.4687', 'num_input_tokens_seen': 376764464, 'train_runtime': '8.087e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105450/225000 [224:38:12<271:27:06,  8.17s/it]
 47%|████▋     | 105451/225000 [224:38:15<220:48:59,  6.65s/it]
 47%|████▋     | 105452/225000 [224:38:24<238:33:13,  7.18s/it]
 47%|████▋     | 105453/225000 [224:38:40<333:30:16, 10.04s/it]
 47%|████▋     | 105454/225000 [224:38:51<339:32:43, 10.23s/it]
 47%|████▋     | 105455/225000 [224:38:56<287:31:58,  8.66s/it]
 47%|████▋     | 105456/225000 [224:39:09<329:07:35,  9.91s/it]
 47%|████▋     | 105457/225000 [224:39:15<290:32:01,  8.75s/it]
 47%|████▋     | 105458/225000 [224:39:21<263:10:21,  7.93s/it]
 47%|████▋     | 105459/225000 [224:39:26<231:27:19,  6.97s/it]
 47%|████▋     | 105460/225000 [224:39:40<306:37:56,  9.23s/it]
 47%|████▋     | 105461/225000 [224:39:49<302:45:01,  9.12s/it]
 47%|████▋     | 105462/225000 [224:39:54<260:44:52,  7.85s/it]
 47%|████▋     | 105463/225000 [224:39:56<207:08:47,  6.24s/it]
 47%|████▋     | 105464/225000 [224:40:11<295:24:02,  8.90s/it]
 47%|████▋     | 105465/225000 [224:40:26<355:01:27, 10.69s/it]
 47%|████▋     | 105466/225000 [224:40:42<408:23:03, 12.30s/it]
 47%|████▋     | 105467/225000 [224:40:56<426:28:20, 12.84s/it]
 47%|████▋     | 105468/225000 [224:41:00<331:42:51,  9.99s/it]
 47%|████▋     | 105469/225000 [224:41:14<375:37:01, 11.31s/it]
 47%|████▋     | 105470/225000 [224:41:20<321:43:30,  9.69s/it]
 47%|████▋     | 105471/225000 [224:41:27<298:10:32,  8.98s/it]
 47%|████▋     | 105472/225000 [224:41:36<293:12:12,  8.83s/it]
 47%|████▋     | 105473/225000 [224:41:46<308:36:43,  9.30s/it]
 47%|████▋     | 105474/225000 [224:41:52<277:27:44,  8.36s/it]
 47%|████▋     | 105475/225000 [224:42:01<278:36:36,  8.39s/it]
 47%|████▋     | 105476/225000 [224:42:08<270:18:52,  8.14s/it]
 47%|████▋     | 105477/225000 [224:42:16<259:41:24,  7.82s/it]
 47%|████▋     | 105478/225000 [224:42:35<373:02:19, 11.24s/it]
 47%|████▋     | 105479/225000 [224:42:43<339:29:33, 10.23s/it]
 47%|████▋     | 105480/225000 [224:42:46<273:56:17,  8.25s/it]
 47%|████▋     | 105481/225000 [224:42:55<283:00:48,  8.52s/it]
 47%|████▋     | 105482/225000 [224:43:10<342:08:04, 10.31s/it]
 47%|████▋     | 105483/225000 [224:43:29<425:24:12, 12.81s/it]
 47%|████▋     | 105484/225000 [224:43:36<374:44:23, 11.29s/it]
 47%|████▋     | 105485/225000 [224:43:39<292:25:37,  8.81s/it]
 47%|████▋     | 105486/225000 [224:43:42<233:57:16,  7.05s/it]
 47%|████▋     | 105487/225000 [224:43:49<226:35:16,  6.83s/it]
 47%|████▋     | 105488/225000 [224:43:53<202:02:11,  6.09s/it]
 47%|████▋     | 105489/225000 [224:43:58<195:52:14,  5.90s/it]
 47%|████▋     | 105490/225000 [224:44:04<192:14:10,  5.79s/it]
 47%|████▋     | 105491/225000 [224:44:19<286:31:58,  8.63s/it]
 47%|████▋     | 105492/225000 [224:44:31<320:58:27,  9.67s/it]
 47%|████▋     | 105493/225000 [224:44:37<282:50:11,  8.52s/it]
 47%|████▋     | 105494/225000 [224:44:49<315:54:38,  9.52s/it]
 47%|████▋     | 105495/225000 [224:45:03<363:59:40, 10.97s/it]
 47%|████▋     | 105496/225000 [224:45:08<299:22:34,  9.02s/it]
 47%|████▋     | 105497/225000 [224:45:13<263:23:30,  7.93s/it]
 47%|████▋     | 105498/225000 [224:45:26<308:16:13,  9.29s/it]
 47%|████▋     | 105499/225000 [224:45:34<295:52:19,  8.91s/it]
 47%|████▋     | 105500/225000 [224:45:50<365:54:19, 11.02s/it]
                                                               
{'loss': '0.9197', 'grad_norm': '1.271', 'learning_rate': '2.938e-05', 'epoch': '0.4689', 'num_input_tokens_seen': 376972416, 'train_runtime': '8.092e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105500/225000 [224:45:50<365:54:19, 11.02s/it]
 47%|████▋     | 105501/225000 [224:45:58<337:16:37, 10.16s/it]
 47%|████▋     | 105502/225000 [224:46:18<440:16:33, 13.26s/it]
 47%|████▋     | 105503/225000 [224:46:27<400:22:46, 12.06s/it]
 47%|████▋     | 105504/225000 [224:46:51<514:15:07, 15.49s/it]
 47%|████▋     | 105505/225000 [224:46:58<425:20:03, 12.81s/it]
 47%|████▋     | 105506/225000 [224:47:03<354:50:57, 10.69s/it]
 47%|████▋     | 105507/225000 [224:47:15<362:15:28, 10.91s/it]
 47%|████▋     | 105508/225000 [224:47:27<377:30:53, 11.37s/it]
 47%|████▋     | 105509/225000 [224:47:39<379:59:21, 11.45s/it]
 47%|████▋     | 105510/225000 [224:47:50<378:46:19, 11.41s/it]
 47%|████▋     | 105511/225000 [224:47:56<328:01:45,  9.88s/it]
 47%|████▋     | 105512/225000 [224:48:02<284:45:40,  8.58s/it]
 47%|████▋     | 105513/225000 [224:48:13<308:37:39,  9.30s/it]
 47%|████▋     | 105514/225000 [224:48:26<344:02:42, 10.37s/it]
 47%|████▋     | 105515/225000 [224:48:33<312:22:18,  9.41s/it]
 47%|████▋     | 105516/225000 [224:48:36<248:54:43,  7.50s/it]
 47%|████▋     | 105517/225000 [224:48:42<238:33:08,  7.19s/it]
 47%|████▋     | 105518/225000 [224:48:53<267:39:48,  8.06s/it]
 47%|████▋     | 105519/225000 [224:49:04<297:11:57,  8.95s/it]
 47%|████▋     | 105520/225000 [224:49:17<342:09:28, 10.31s/it]
 47%|████▋     | 105521/225000 [224:49:28<350:07:15, 10.55s/it]
 47%|████▋     | 105522/225000 [224:49:36<322:04:08,  9.70s/it]
 47%|████▋     | 105523/225000 [224:49:47<335:58:21, 10.12s/it]
 47%|████▋     | 105524/225000 [224:49:59<356:39:01, 10.75s/it]
 47%|████▋     | 105525/225000 [224:50:05<308:21:19,  9.29s/it]
 47%|████▋     | 105526/225000 [224:50:08<241:44:43,  7.28s/it]
 47%|████▋     | 105527/225000 [224:50:14<228:54:21,  6.90s/it]
 47%|████▋     | 105528/225000 [224:50:29<309:11:04,  9.32s/it]
 47%|████▋     | 105529/225000 [224:50:34<266:12:31,  8.02s/it]
 47%|████▋     | 105530/225000 [224:50:40<252:43:30,  7.62s/it]
 47%|████▋     | 105531/225000 [224:50:43<200:56:05,  6.05s/it]
 47%|████▋     | 105532/225000 [224:50:54<247:45:20,  7.47s/it]
 47%|████▋     | 105533/225000 [224:50:57<204:57:29,  6.18s/it]
 47%|████▋     | 105534/225000 [224:50:59<168:03:27,  5.06s/it]
 47%|████▋     | 105535/225000 [224:51:05<178:33:18,  5.38s/it]
 47%|████▋     | 105536/225000 [224:51:14<209:10:21,  6.30s/it]
 47%|████▋     | 105537/225000 [224:51:28<292:52:21,  8.83s/it]
 47%|████▋     | 105538/225000 [224:51:40<316:31:28,  9.54s/it]
 47%|████▋     | 105539/225000 [224:51:43<258:09:38,  7.78s/it]
 47%|████▋     | 105540/225000 [224:51:53<272:33:48,  8.21s/it]
 47%|████▋     | 105541/225000 [224:51:58<245:55:11,  7.41s/it]
 47%|████▋     | 105542/225000 [224:52:09<282:16:42,  8.51s/it]
 47%|████▋     | 105543/225000 [224:52:20<304:18:49,  9.17s/it]
 47%|████▋     | 105544/225000 [224:52:31<326:24:41,  9.84s/it]
 47%|████▋     | 105545/225000 [224:52:40<318:10:55,  9.59s/it]
 47%|████▋     | 105546/225000 [224:52:50<317:07:44,  9.56s/it]
 47%|████▋     | 105547/225000 [224:52:56<282:11:04,  8.50s/it]
 47%|████▋     | 105548/225000 [224:53:04<283:06:01,  8.53s/it]
 47%|████▋     | 105549/225000 [224:53:15<302:28:00,  9.12s/it]
 47%|████▋     | 105550/225000 [224:53:20<260:12:00,  7.84s/it]
                                                               
{'loss': '0.9567', 'grad_norm': '2.077', 'learning_rate': '2.936e-05', 'epoch': '0.4691', 'num_input_tokens_seen': 377179808, 'train_runtime': '8.096e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105550/225000 [224:53:20<260:12:00,  7.84s/it]
 47%|████▋     | 105551/225000 [224:53:30<281:58:39,  8.50s/it]
 47%|████▋     | 105552/225000 [224:53:34<235:29:40,  7.10s/it]
 47%|████▋     | 105553/225000 [224:53:37<202:28:12,  6.10s/it]
 47%|████▋     | 105554/225000 [224:53:46<223:18:34,  6.73s/it]
 47%|████▋     | 105555/225000 [224:53:49<185:55:32,  5.60s/it]
 47%|████▋     | 105556/225000 [224:53:52<159:29:09,  4.81s/it]
 47%|████▋     | 105557/225000 [224:53:55<144:02:42,  4.34s/it]
 47%|████▋     | 105558/225000 [224:54:01<164:15:45,  4.95s/it]
 47%|████▋     | 105559/225000 [224:54:07<175:17:55,  5.28s/it]
 47%|████▋     | 105560/225000 [224:54:16<212:19:27,  6.40s/it]
 47%|████▋     | 105561/225000 [224:54:24<227:39:46,  6.86s/it]
 47%|████▋     | 105562/225000 [224:54:38<299:21:34,  9.02s/it]
 47%|████▋     | 105563/225000 [224:54:46<283:25:11,  8.54s/it]
 47%|████▋     | 105564/225000 [224:54:55<292:20:53,  8.81s/it]
 47%|████▋     | 105565/225000 [224:54:58<235:21:45,  7.09s/it]
 47%|████▋     | 105566/225000 [224:55:03<213:06:28,  6.42s/it]
 47%|████▋     | 105567/225000 [224:55:12<235:45:57,  7.11s/it]
 47%|████▋     | 105568/225000 [224:55:22<265:01:59,  7.99s/it]
 47%|████▋     | 105569/225000 [224:55:27<234:20:15,  7.06s/it]
 47%|████▋     | 105570/225000 [224:55:30<198:35:10,  5.99s/it]
 47%|████▋     | 105571/225000 [224:55:34<180:14:45,  5.43s/it]
 47%|████▋     | 105572/225000 [224:55:38<160:30:28,  4.84s/it]
 47%|████▋     | 105573/225000 [224:55:43<166:58:18,  5.03s/it]
 47%|████▋     | 105574/225000 [224:55:47<153:10:25,  4.62s/it]
 47%|████▋     | 105575/225000 [224:55:52<155:56:20,  4.70s/it]
 47%|████▋     | 105576/225000 [224:56:05<241:49:28,  7.29s/it]
 47%|████▋     | 105577/225000 [224:56:14<256:02:13,  7.72s/it]
 47%|████▋     | 105578/225000 [224:56:27<309:23:40,  9.33s/it]
 47%|████▋     | 105579/225000 [224:56:34<287:48:19,  8.68s/it]
 47%|████▋     | 105580/225000 [224:56:37<232:06:49,  7.00s/it]
 47%|████▋     | 105581/225000 [224:56:47<264:59:17,  7.99s/it]
 47%|████▋     | 105582/225000 [224:57:06<374:19:31, 11.28s/it]
 47%|████▋     | 105583/225000 [224:57:13<323:33:29,  9.75s/it]
 47%|████▋     | 105584/225000 [224:57:25<352:44:35, 10.63s/it]
 47%|████▋     | 105585/225000 [224:57:28<276:20:59,  8.33s/it]
 47%|████▋     | 105586/225000 [224:57:33<245:11:22,  7.39s/it]
 47%|████▋     | 105587/225000 [224:57:40<241:42:24,  7.29s/it]
 47%|████▋     | 105588/225000 [224:57:54<299:54:57,  9.04s/it]
 47%|████▋     | 105589/225000 [224:58:04<311:43:30,  9.40s/it]
 47%|████▋     | 105590/225000 [224:58:08<257:43:53,  7.77s/it]
 47%|████▋     | 105591/225000 [224:58:18<281:49:12,  8.50s/it]
 47%|████▋     | 105592/225000 [224:58:20<219:52:37,  6.63s/it]
 47%|████▋     | 105593/225000 [224:58:26<212:18:21,  6.40s/it]
 47%|████▋     | 105594/225000 [224:58:30<186:25:34,  5.62s/it]
 47%|████▋     | 105595/225000 [224:58:48<310:37:04,  9.36s/it]
 47%|████▋     | 105596/225000 [224:58:50<240:09:51,  7.24s/it]
 47%|████▋     | 105597/225000 [224:59:03<296:18:03,  8.93s/it]
 47%|████▋     | 105598/225000 [224:59:06<237:53:42,  7.17s/it]
 47%|████▋     | 105599/225000 [224:59:20<299:42:16,  9.04s/it]
 47%|████▋     | 105600/225000 [224:59:33<345:30:14, 10.42s/it]
                                                               
{'loss': '0.9189', 'grad_norm': '1.652', 'learning_rate': '2.934e-05', 'epoch': '0.4693', 'num_input_tokens_seen': 377346280, 'train_runtime': '8.1e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105600/225000 [224:59:33<345:30:14, 10.42s/it]
 47%|████▋     | 105601/225000 [224:59:37<283:00:25,  8.53s/it]
 47%|████▋     | 105602/225000 [224:59:41<232:06:28,  7.00s/it]
 47%|████▋     | 105603/225000 [224:59:50<252:55:59,  7.63s/it]
 47%|████▋     | 105604/225000 [225:00:05<328:10:59,  9.90s/it]
 47%|████▋     | 105605/225000 [225:00:10<280:01:15,  8.44s/it]
 47%|████▋     | 105606/225000 [225:00:20<289:52:34,  8.74s/it]
 47%|████▋     | 105607/225000 [225:00:25<260:00:01,  7.84s/it]
 47%|████▋     | 105608/225000 [225:00:32<244:05:48,  7.36s/it]
 47%|████▋     | 105609/225000 [225:00:36<216:00:34,  6.51s/it]
 47%|████▋     | 105610/225000 [225:00:44<234:03:20,  7.06s/it]
 47%|████▋     | 105611/225000 [225:01:00<320:35:13,  9.67s/it]
 47%|████▋     | 105612/225000 [225:01:10<326:08:52,  9.83s/it]
 47%|████▋     | 105613/225000 [225:01:26<381:43:04, 11.51s/it]
 47%|████▋     | 105614/225000 [225:01:31<314:03:12,  9.47s/it]
 47%|████▋     | 105615/225000 [225:01:40<316:46:03,  9.55s/it]
 47%|████▋     | 105616/225000 [225:01:47<292:14:18,  8.81s/it]
 47%|████▋     | 105617/225000 [225:01:53<262:01:40,  7.90s/it]
 47%|████▋     | 105618/225000 [225:02:04<287:57:18,  8.68s/it]
 47%|████▋     | 105619/225000 [225:02:10<265:57:07,  8.02s/it]
 47%|████▋     | 105620/225000 [225:02:16<242:15:16,  7.31s/it]
 47%|████▋     | 105621/225000 [225:02:25<260:11:47,  7.85s/it]
 47%|████▋     | 105622/225000 [225:02:40<329:26:09,  9.93s/it]
 47%|████▋     | 105623/225000 [225:02:50<328:33:17,  9.91s/it]
 47%|████▋     | 105624/225000 [225:02:58<311:10:09,  9.38s/it]
 47%|████▋     | 105625/225000 [225:03:11<345:04:35, 10.41s/it]
 47%|████▋     | 105626/225000 [225:03:18<315:36:38,  9.52s/it]
 47%|████▋     | 105627/225000 [225:03:30<340:09:19, 10.26s/it]
 47%|████▋     | 105628/225000 [225:03:40<340:33:48, 10.27s/it]
 47%|████▋     | 105629/225000 [225:03:54<379:24:59, 11.44s/it]
 47%|████▋     | 105630/225000 [225:04:03<352:27:12, 10.63s/it]
 47%|████▋     | 105631/225000 [225:04:10<315:09:04,  9.50s/it]
 47%|████▋     | 105632/225000 [225:04:19<312:18:52,  9.42s/it]
 47%|████▋     | 105633/225000 [225:04:31<339:18:26, 10.23s/it]
 47%|████▋     | 105634/225000 [225:04:37<288:26:10,  8.70s/it]
 47%|████▋     | 105635/225000 [225:04:40<240:47:22,  7.26s/it]
 47%|████▋     | 105636/225000 [225:04:53<290:01:49,  8.75s/it]
 47%|████▋     | 105637/225000 [225:05:01<288:12:59,  8.69s/it]
 47%|████▋     | 105638/225000 [225:05:08<273:02:59,  8.24s/it]
 47%|████▋     | 105639/225000 [225:05:14<243:34:24,  7.35s/it]
 47%|████▋     | 105640/225000 [225:05:22<251:24:49,  7.58s/it]
 47%|████▋     | 105641/225000 [225:05:25<206:04:19,  6.22s/it]
 47%|████▋     | 105642/225000 [225:05:32<212:06:35,  6.40s/it]
 47%|████▋     | 105643/225000 [225:05:40<233:58:00,  7.06s/it]
 47%|████▋     | 105644/225000 [225:05:52<277:26:23,  8.37s/it]
 47%|████▋     | 105645/225000 [225:06:08<361:19:16, 10.90s/it]
 47%|████▋     | 105646/225000 [225:06:21<381:54:53, 11.52s/it]
 47%|████▋     | 105647/225000 [225:06:26<314:20:16,  9.48s/it]
 47%|████▋     | 105648/225000 [225:06:31<265:26:20,  8.01s/it]
 47%|████▋     | 105649/225000 [225:06:39<268:38:17,  8.10s/it]
 47%|████▋     | 105650/225000 [225:06:42<218:11:27,  6.58s/it]
                                                               
{'loss': '0.9415', 'grad_norm': '2.685', 'learning_rate': '2.933e-05', 'epoch': '0.4696', 'num_input_tokens_seen': 377544312, 'train_runtime': '8.104e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105650/225000 [225:06:42<218:11:27,  6.58s/it]
 47%|████▋     | 105651/225000 [225:06:51<243:53:53,  7.36s/it]
 47%|████▋     | 105652/225000 [225:07:02<277:48:39,  8.38s/it]
 47%|████▋     | 105653/225000 [225:07:07<247:49:17,  7.48s/it]
 47%|████▋     | 105654/225000 [225:07:12<217:57:25,  6.57s/it]
 47%|████▋     | 105655/225000 [225:07:26<295:44:29,  8.92s/it]
 47%|████▋     | 105656/225000 [225:07:40<340:44:07, 10.28s/it]
 47%|████▋     | 105657/225000 [225:07:49<327:29:45,  9.88s/it]
 47%|████▋     | 105658/225000 [225:07:52<259:05:50,  7.82s/it]
 47%|████▋     | 105659/225000 [225:08:00<267:03:25,  8.06s/it]
 47%|████▋     | 105660/225000 [225:08:12<304:37:12,  9.19s/it]
 47%|████▋     | 105661/225000 [225:08:23<318:21:06,  9.60s/it]
 47%|████▋     | 105662/225000 [225:08:26<257:39:41,  7.77s/it]
 47%|████▋     | 105663/225000 [225:08:40<322:21:15,  9.72s/it]
 47%|████▋     | 105664/225000 [225:08:48<302:53:23,  9.14s/it]
 47%|████▋     | 105665/225000 [225:08:54<267:20:22,  8.06s/it]
 47%|████▋     | 105666/225000 [225:09:10<347:01:37, 10.47s/it]
 47%|████▋     | 105667/225000 [225:09:16<300:38:56,  9.07s/it]
 47%|████▋     | 105668/225000 [225:09:21<266:36:48,  8.04s/it]
 47%|████▋     | 105669/225000 [225:09:30<275:28:12,  8.31s/it]
 47%|████▋     | 105670/225000 [225:09:47<363:01:58, 10.95s/it]
 47%|████▋     | 105671/225000 [225:09:53<313:18:35,  9.45s/it]
 47%|████▋     | 105672/225000 [225:09:59<271:39:38,  8.20s/it]
 47%|████▋     | 105673/225000 [225:10:02<220:31:32,  6.65s/it]
 47%|████▋     | 105674/225000 [225:10:17<310:12:13,  9.36s/it]
 47%|████▋     | 105675/225000 [225:10:30<338:36:19, 10.22s/it]
 47%|████▋     | 105676/225000 [225:10:36<299:31:37,  9.04s/it]
 47%|████▋     | 105677/225000 [225:10:54<390:56:05, 11.79s/it]
 47%|████▋     | 105678/225000 [225:10:59<322:46:30,  9.74s/it]
 47%|████▋     | 105679/225000 [225:11:14<376:08:43, 11.35s/it]
 47%|████▋     | 105680/225000 [225:11:16<287:00:04,  8.66s/it]
 47%|████▋     | 105681/225000 [225:11:23<264:39:24,  7.99s/it]
 47%|████▋     | 105682/225000 [225:11:30<256:10:52,  7.73s/it]
 47%|████▋     | 105683/225000 [225:11:39<271:00:37,  8.18s/it]
 47%|████▋     | 105684/225000 [225:11:46<257:15:44,  7.76s/it]
 47%|████▋     | 105685/225000 [225:11:49<211:28:15,  6.38s/it]
 47%|████▋     | 105686/225000 [225:12:01<261:53:56,  7.90s/it]
 47%|████▋     | 105687/225000 [225:12:08<253:20:17,  7.64s/it]
 47%|████▋     | 105688/225000 [225:12:19<291:44:27,  8.80s/it]
 47%|████▋     | 105689/225000 [225:12:36<374:44:44, 11.31s/it]
 47%|████▋     | 105690/225000 [225:12:45<345:17:10, 10.42s/it]
 47%|████▋     | 105691/225000 [225:12:55<344:37:02, 10.40s/it]
 47%|████▋     | 105692/225000 [225:13:02<308:48:20,  9.32s/it]
 47%|████▋     | 105693/225000 [225:13:17<362:43:20, 10.94s/it]
 47%|████▋     | 105694/225000 [225:13:24<326:14:24,  9.84s/it]
 47%|████▋     | 105695/225000 [225:13:37<355:25:52, 10.73s/it]
 47%|████▋     | 105696/225000 [225:13:39<272:59:42,  8.24s/it]
 47%|████▋     | 105697/225000 [225:13:45<255:14:48,  7.70s/it]
 47%|████▋     | 105698/225000 [225:13:55<269:24:50,  8.13s/it]
 47%|████▋     | 105699/225000 [225:13:58<225:41:53,  6.81s/it]
 47%|████▋     | 105700/225000 [225:14:01<187:31:17,  5.66s/it]
                                                               
{'loss': '0.9654', 'grad_norm': '3.085', 'learning_rate': '2.931e-05', 'epoch': '0.4698', 'num_input_tokens_seen': 377739200, 'train_runtime': '8.108e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105700/225000 [225:14:01<187:31:17,  5.66s/it]
 47%|████▋     | 105701/225000 [225:14:04<162:11:48,  4.89s/it]
 47%|████▋     | 105702/225000 [225:14:09<157:53:37,  4.76s/it]
 47%|████▋     | 105703/225000 [225:14:15<173:48:55,  5.25s/it]
 47%|████▋     | 105704/225000 [225:14:25<217:31:13,  6.56s/it]
 47%|████▋     | 105705/225000 [225:14:28<181:36:02,  5.48s/it]
 47%|████▋     | 105706/225000 [225:14:31<156:15:01,  4.72s/it]
 47%|████▋     | 105707/225000 [225:14:37<166:49:58,  5.03s/it]
 47%|████▋     | 105708/225000 [225:14:50<249:47:15,  7.54s/it]
 47%|████▋     | 105709/225000 [225:15:03<304:23:53,  9.19s/it]
 47%|████▋     | 105710/225000 [225:15:06<245:07:58,  7.40s/it]
 47%|████▋     | 105711/225000 [225:15:17<277:42:31,  8.38s/it]
 47%|████▋     | 105712/225000 [225:15:28<306:34:36,  9.25s/it]
 47%|████▋     | 105713/225000 [225:15:38<312:03:59,  9.42s/it]
 47%|████▋     | 105714/225000 [225:15:46<296:32:01,  8.95s/it]
 47%|████▋     | 105715/225000 [225:15:53<280:57:22,  8.48s/it]
 47%|████▋     | 105716/225000 [225:16:03<298:05:05,  9.00s/it]
 47%|████▋     | 105717/225000 [225:16:09<268:18:42,  8.10s/it]
 47%|████▋     | 105718/225000 [225:16:23<324:44:47,  9.80s/it]
 47%|████▋     | 105719/225000 [225:16:28<280:00:57,  8.45s/it]
 47%|████▋     | 105720/225000 [225:16:33<237:15:45,  7.16s/it]
 47%|████▋     | 105721/225000 [225:16:44<279:20:46,  8.43s/it]
 47%|████▋     | 105722/225000 [225:16:58<330:05:43,  9.96s/it]
 47%|████▋     | 105723/225000 [225:17:04<290:28:41,  8.77s/it]
 47%|████▋     | 105724/225000 [225:17:09<257:02:14,  7.76s/it]
 47%|████▋     | 105725/225000 [225:17:15<244:17:51,  7.37s/it]
 47%|████▋     | 105726/225000 [225:17:20<216:51:42,  6.55s/it]
 47%|████▋     | 105727/225000 [225:17:23<183:14:44,  5.53s/it]
 47%|████▋     | 105728/225000 [225:17:35<242:42:07,  7.33s/it]
 47%|████▋     | 105729/225000 [225:17:37<193:05:45,  5.83s/it]
 47%|████▋     | 105730/225000 [225:17:51<269:41:09,  8.14s/it]
 47%|████▋     | 105731/225000 [225:18:03<311:03:39,  9.39s/it]
 47%|████▋     | 105732/225000 [225:18:09<277:14:10,  8.37s/it]
 47%|████▋     | 105733/225000 [225:18:12<221:07:17,  6.67s/it]
 47%|████▋     | 105734/225000 [225:18:14<176:52:39,  5.34s/it]
 47%|████▋     | 105735/225000 [225:18:17<153:44:09,  4.64s/it]
 47%|████▋     | 105736/225000 [225:18:20<138:00:41,  4.17s/it]
 47%|████▋     | 105737/225000 [225:18:30<201:55:56,  6.10s/it]
 47%|████▋     | 105738/225000 [225:18:34<175:50:32,  5.31s/it]
 47%|████▋     | 105739/225000 [225:18:49<272:28:50,  8.23s/it]
 47%|████▋     | 105740/225000 [225:18:54<244:02:36,  7.37s/it]
 47%|████▋     | 105741/225000 [225:18:57<200:55:28,  6.07s/it]
 47%|████▋     | 105742/225000 [225:19:11<274:39:58,  8.29s/it]
 47%|████▋     | 105743/225000 [225:19:16<247:04:43,  7.46s/it]
 47%|████▋     | 105744/225000 [225:19:29<301:43:56,  9.11s/it]
 47%|████▋     | 105745/225000 [225:19:36<275:17:19,  8.31s/it]
 47%|████▋     | 105746/225000 [225:19:46<297:13:29,  8.97s/it]
 47%|████▋     | 105747/225000 [225:19:57<318:00:09,  9.60s/it]
 47%|████▋     | 105748/225000 [225:20:09<343:05:18, 10.36s/it]
 47%|████▋     | 105749/225000 [225:20:15<294:23:32,  8.89s/it]
 47%|████▋     | 105750/225000 [225:20:25<308:36:15,  9.32s/it]
                                                               
{'loss': '0.9504', 'grad_norm': '2.146', 'learning_rate': '2.929e-05', 'epoch': '0.47', 'num_input_tokens_seen': 377913768, 'train_runtime': '8.112e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105750/225000 [225:20:25<308:36:15,  9.32s/it]
 47%|████▋     | 105751/225000 [225:20:39<353:58:35, 10.69s/it]
 47%|████▋     | 105752/225000 [225:20:45<305:31:27,  9.22s/it]
 47%|████▋     | 105753/225000 [225:20:52<279:15:58,  8.43s/it]
 47%|████▋     | 105754/225000 [225:20:57<245:46:33,  7.42s/it]
 47%|████▋     | 105755/225000 [225:21:03<237:46:02,  7.18s/it]
 47%|████▋     | 105756/225000 [225:21:14<271:11:53,  8.19s/it]
 47%|████▋     | 105757/225000 [225:21:26<307:31:41,  9.28s/it]
 47%|████▋     | 105758/225000 [225:21:30<259:30:36,  7.83s/it]
 47%|████▋     | 105759/225000 [225:21:45<326:05:14,  9.84s/it]
 47%|████▋     | 105760/225000 [225:21:49<274:41:42,  8.29s/it]
 47%|████▋     | 105761/225000 [225:21:55<246:12:15,  7.43s/it]
 47%|████▋     | 105762/225000 [225:21:59<217:43:15,  6.57s/it]
 47%|████▋     | 105763/225000 [225:22:16<323:08:59,  9.76s/it]
 47%|████▋     | 105764/225000 [225:22:27<328:38:39,  9.92s/it]
 47%|████▋     | 105765/225000 [225:22:35<316:16:04,  9.55s/it]
 47%|████▋     | 105766/225000 [225:22:41<272:06:51,  8.22s/it]
 47%|████▋     | 105767/225000 [225:22:49<278:05:45,  8.40s/it]
 47%|████▋     | 105768/225000 [225:22:55<253:46:16,  7.66s/it]
 47%|████▋     | 105769/225000 [225:23:05<269:41:13,  8.14s/it]
 47%|████▋     | 105770/225000 [225:23:14<287:08:26,  8.67s/it]
 47%|████▋     | 105771/225000 [225:23:20<254:09:35,  7.67s/it]
 47%|████▋     | 105772/225000 [225:23:26<234:36:01,  7.08s/it]
 47%|████▋     | 105773/225000 [225:23:35<258:46:21,  7.81s/it]
 47%|████▋     | 105774/225000 [225:23:41<243:20:35,  7.35s/it]
 47%|████▋     | 105775/225000 [225:23:48<233:31:33,  7.05s/it]
 47%|████▋     | 105776/225000 [225:23:53<212:38:58,  6.42s/it]
 47%|████▋     | 105777/225000 [225:23:56<180:10:50,  5.44s/it]
 47%|████▋     | 105778/225000 [225:24:06<228:12:10,  6.89s/it]
 47%|████▋     | 105779/225000 [225:24:08<184:11:33,  5.56s/it]
 47%|████▋     | 105780/225000 [225:24:16<201:11:21,  6.08s/it]
 47%|████▋     | 105781/225000 [225:24:21<195:26:05,  5.90s/it]
 47%|████▋     | 105782/225000 [225:24:28<199:48:00,  6.03s/it]
 47%|████▋     | 105783/225000 [225:24:39<256:18:00,  7.74s/it]
 47%|████▋     | 105784/225000 [225:24:49<271:26:23,  8.20s/it]
 47%|████▋     | 105785/225000 [225:24:58<286:58:58,  8.67s/it]
 47%|████▋     | 105786/225000 [225:25:13<347:39:59, 10.50s/it]
 47%|████▋     | 105787/225000 [225:25:17<285:11:14,  8.61s/it]
 47%|████▋     | 105788/225000 [225:25:20<223:09:13,  6.74s/it]
 47%|████▋     | 105789/225000 [225:25:22<180:42:10,  5.46s/it]
 47%|████▋     | 105790/225000 [225:25:27<179:24:46,  5.42s/it]
 47%|████▋     | 105791/225000 [225:25:38<226:17:48,  6.83s/it]
 47%|████▋     | 105792/225000 [225:25:42<203:44:43,  6.15s/it]
 47%|████▋     | 105793/225000 [225:25:49<207:48:38,  6.28s/it]
 47%|████▋     | 105794/225000 [225:26:06<316:07:50,  9.55s/it]
 47%|████▋     | 105795/225000 [225:26:14<304:57:06,  9.21s/it]
 47%|████▋     | 105796/225000 [225:26:19<259:52:28,  7.85s/it]
 47%|████▋     | 105797/225000 [225:26:21<204:12:18,  6.17s/it]
 47%|████▋     | 105798/225000 [225:26:26<192:00:59,  5.80s/it]
 47%|████▋     | 105799/225000 [225:26:29<163:05:10,  4.93s/it]
 47%|████▋     | 105800/225000 [225:26:31<137:10:57,  4.14s/it]
                                                               
{'loss': '0.9996', 'grad_norm': '5.07', 'learning_rate': '2.927e-05', 'epoch': '0.4702', 'num_input_tokens_seen': 378091680, 'train_runtime': '8.116e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105800/225000 [225:26:31<137:10:57,  4.14s/it]
 47%|████▋     | 105801/225000 [225:26:43<209:59:54,  6.34s/it]
 47%|████▋     | 105802/225000 [225:26:47<186:41:15,  5.64s/it]
 47%|████▋     | 105803/225000 [225:26:57<234:06:58,  7.07s/it]
 47%|████▋     | 105804/225000 [225:27:09<277:12:43,  8.37s/it]
 47%|████▋     | 105805/225000 [225:27:14<249:26:13,  7.53s/it]
 47%|████▋     | 105806/225000 [225:27:17<204:34:15,  6.18s/it]
 47%|████▋     | 105807/225000 [225:27:21<176:31:36,  5.33s/it]
 47%|████▋     | 105808/225000 [225:27:41<323:37:50,  9.77s/it]
 47%|████▋     | 105809/225000 [225:27:44<258:54:17,  7.82s/it]
 47%|████▋     | 105810/225000 [225:27:54<282:12:52,  8.52s/it]
 47%|████▋     | 105811/225000 [225:27:59<241:16:02,  7.29s/it]
 47%|████▋     | 105812/225000 [225:28:10<280:29:08,  8.47s/it]
 47%|████▋     | 105813/225000 [225:28:17<268:19:03,  8.10s/it]
 47%|████▋     | 105814/225000 [225:28:28<299:55:29,  9.06s/it]
 47%|████▋     | 105815/225000 [225:28:34<262:43:38,  7.94s/it]
 47%|████▋     | 105816/225000 [225:28:41<260:50:35,  7.88s/it]
 47%|████▋     | 105817/225000 [225:28:47<235:34:23,  7.12s/it]
 47%|████▋     | 105818/225000 [225:28:53<225:23:36,  6.81s/it]
 47%|████▋     | 105819/225000 [225:28:57<199:44:26,  6.03s/it]
 47%|████▋     | 105820/225000 [225:29:00<166:38:30,  5.03s/it]
 47%|████▋     | 105821/225000 [225:29:06<178:28:37,  5.39s/it]
 47%|████▋     | 105822/225000 [225:29:13<190:38:20,  5.76s/it]
 47%|████▋     | 105823/225000 [225:29:18<188:26:36,  5.69s/it]
 47%|████▋     | 105824/225000 [225:29:26<210:05:41,  6.35s/it]
 47%|████▋     | 105825/225000 [225:29:29<178:58:31,  5.41s/it]
 47%|████▋     | 105826/225000 [225:29:33<162:41:29,  4.91s/it]
 47%|████▋     | 105827/225000 [225:29:46<247:26:08,  7.47s/it]
 47%|████▋     | 105828/225000 [225:29:49<201:35:40,  6.09s/it]
 47%|████▋     | 105829/225000 [225:30:00<249:22:47,  7.53s/it]
 47%|████▋     | 105830/225000 [225:30:08<251:13:25,  7.59s/it]
 47%|████▋     | 105831/225000 [225:30:19<281:46:47,  8.51s/it]
 47%|████▋     | 105832/225000 [225:30:38<386:40:07, 11.68s/it]
 47%|████▋     | 105833/225000 [225:30:46<357:32:00, 10.80s/it]
 47%|████▋     | 105834/225000 [225:30:54<323:39:05,  9.78s/it]
 47%|████▋     | 105835/225000 [225:31:00<288:10:16,  8.71s/it]
 47%|████▋     | 105836/225000 [225:31:09<293:25:27,  8.86s/it]
 47%|████▋     | 105837/225000 [225:31:22<328:00:27,  9.91s/it]
 47%|████▋     | 105838/225000 [225:31:27<287:42:48,  8.69s/it]
 47%|████▋     | 105839/225000 [225:31:40<324:55:22,  9.82s/it]
 47%|████▋     | 105840/225000 [225:31:47<298:32:05,  9.02s/it]
 47%|████▋     | 105841/225000 [225:31:50<238:48:45,  7.21s/it]
 47%|████▋     | 105842/225000 [225:31:52<189:31:01,  5.73s/it]
 47%|████▋     | 105843/225000 [225:31:59<194:27:49,  5.88s/it]
 47%|████▋     | 105844/225000 [225:32:11<258:54:57,  7.82s/it]
 47%|████▋     | 105845/225000 [225:32:16<234:24:22,  7.08s/it]
 47%|████▋     | 105846/225000 [225:32:21<214:44:48,  6.49s/it]
 47%|████▋     | 105847/225000 [225:32:33<261:42:38,  7.91s/it]
 47%|████▋     | 105848/225000 [225:32:36<220:23:18,  6.66s/it]
 47%|████▋     | 105849/225000 [225:32:42<207:47:54,  6.28s/it]
 47%|████▋     | 105850/225000 [225:32:57<294:08:55,  8.89s/it]
                                                               
{'loss': '0.9534', 'grad_norm': '1.637', 'learning_rate': '2.925e-05', 'epoch': '0.4704', 'num_input_tokens_seen': 378266088, 'train_runtime': '8.12e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105850/225000 [225:32:57<294:08:55,  8.89s/it]
 47%|████▋     | 105851/225000 [225:33:02<263:04:36,  7.95s/it]
 47%|████▋     | 105852/225000 [225:33:10<260:39:17,  7.88s/it]
 47%|████▋     | 105853/225000 [225:33:17<249:55:30,  7.55s/it]
 47%|████▋     | 105854/225000 [225:33:24<249:09:17,  7.53s/it]
 47%|████▋     | 105855/225000 [225:33:30<231:38:49,  7.00s/it]
 47%|████▋     | 105856/225000 [225:33:44<299:37:06,  9.05s/it]
 47%|████▋     | 105857/225000 [225:33:53<297:15:00,  8.98s/it]
 47%|████▋     | 105858/225000 [225:33:59<268:46:43,  8.12s/it]
 47%|████▋     | 105859/225000 [225:34:07<270:32:42,  8.17s/it]
 47%|████▋     | 105860/225000 [225:34:17<289:04:45,  8.73s/it]
 47%|████▋     | 105861/225000 [225:34:25<275:49:36,  8.33s/it]
 47%|████▋     | 105862/225000 [225:34:29<238:09:58,  7.20s/it]
 47%|████▋     | 105863/225000 [225:34:45<327:15:00,  9.89s/it]
 47%|████▋     | 105864/225000 [225:34:53<300:58:02,  9.09s/it]
 47%|████▋     | 105865/225000 [225:34:57<248:38:07,  7.51s/it]
 47%|████▋     | 105866/225000 [225:35:00<205:42:02,  6.22s/it]
 47%|████▋     | 105867/225000 [225:35:06<203:48:57,  6.16s/it]
 47%|████▋     | 105868/225000 [225:35:11<196:20:41,  5.93s/it]
 47%|████▋     | 105869/225000 [225:35:26<288:50:37,  8.73s/it]
 47%|████▋     | 105870/225000 [225:35:31<249:53:18,  7.55s/it]
 47%|████▋     | 105871/225000 [225:35:36<226:11:49,  6.84s/it]
 47%|████▋     | 105872/225000 [225:35:40<195:37:53,  5.91s/it]
 47%|████▋     | 105873/225000 [225:35:46<192:43:42,  5.82s/it]
 47%|████▋     | 105874/225000 [225:35:53<205:46:09,  6.22s/it]
 47%|████▋     | 105875/225000 [225:36:08<289:31:17,  8.75s/it]
 47%|████▋     | 105876/225000 [225:36:18<305:36:41,  9.24s/it]
 47%|████▋     | 105877/225000 [225:36:26<290:25:53,  8.78s/it]
 47%|████▋     | 105878/225000 [225:36:30<251:01:08,  7.59s/it]
 47%|████▋     | 105879/225000 [225:36:45<317:11:36,  9.59s/it]
 47%|████▋     | 105880/225000 [225:36:48<252:53:55,  7.64s/it]
 47%|████▋     | 105881/225000 [225:36:59<292:18:13,  8.83s/it]
 47%|████▋     | 105882/225000 [225:37:06<266:35:05,  8.06s/it]
 47%|████▋     | 105883/225000 [225:37:12<254:27:11,  7.69s/it]
 47%|████▋     | 105884/225000 [225:37:25<299:12:01,  9.04s/it]
 47%|████▋     | 105885/225000 [225:37:32<285:35:25,  8.63s/it]
 47%|████▋     | 105886/225000 [225:37:38<252:08:27,  7.62s/it]
 47%|████▋     | 105887/225000 [225:37:50<301:02:52,  9.10s/it]
 47%|████▋     | 105888/225000 [225:37:59<297:49:01,  9.00s/it]
 47%|████▋     | 105889/225000 [225:38:07<292:54:59,  8.85s/it]
 47%|████▋     | 105890/225000 [225:38:20<331:25:55, 10.02s/it]
 47%|████▋     | 105891/225000 [225:38:31<336:07:25, 10.16s/it]
 47%|████▋     | 105892/225000 [225:38:41<341:10:45, 10.31s/it]
 47%|████▋     | 105893/225000 [225:38:44<269:02:13,  8.13s/it]
 47%|████▋     | 105894/225000 [225:38:50<245:24:28,  7.42s/it]
 47%|████▋     | 105895/225000 [225:38:53<202:19:23,  6.12s/it]
 47%|████▋     | 105896/225000 [225:39:00<207:39:51,  6.28s/it]
 47%|████▋     | 105897/225000 [225:39:08<225:03:07,  6.80s/it]
 47%|████▋     | 105898/225000 [225:39:14<220:25:42,  6.66s/it]
 47%|████▋     | 105899/225000 [225:39:28<289:09:54,  8.74s/it]
 47%|████▋     | 105900/225000 [225:39:36<284:10:12,  8.59s/it]
                                                               
{'loss': '0.9782', 'grad_norm': '2.495', 'learning_rate': '2.924e-05', 'epoch': '0.4707', 'num_input_tokens_seen': 378446376, 'train_runtime': '8.124e+05', 'train_tokens_per_second': '465.9'}

 47%|████▋     | 105900/225000 [225:39:36<284:10:12,  8.59s/it]
 47%|████▋     | 105901/225000 [225:39:42<260:59:39,  7.89s/it]
 47%|████▋     | 105902/225000 [225:39:45<207:57:14,  6.29s/it]
 47%|████▋     | 105903/225000 [225:39:50<201:23:33,  6.09s/it]
 47%|████▋     | 105904/225000 [225:39:54<172:15:39,  5.21s/it]
 47%|████▋     | 105905/225000 [225:40:10<282:26:33,  8.54s/it]
 47%|████▋     | 105906/225000 [225:40:15<252:20:47,  7.63s/it]
 47%|████▋     | 105907/225000 [225:40:31<330:52:16, 10.00s/it]
 47%|████▋     | 105908/225000 [225:40:40<318:19:53,  9.62s/it]
 47%|████▋     | 105909/225000 [225:40:48<303:55:51,  9.19s/it]
 47%|████▋     | 105910/225000 [225:40:58<309:40:13,  9.36s/it]
 47%|████▋     | 105911/225000 [225:41:02<259:59:49,  7.86s/it]
 47%|████▋     | 105912/225000 [225:41:11<266:45:45,  8.06s/it]
 47%|████▋     | 105913/225000 [225:41:27<345:41:49, 10.45s/it]
 47%|████▋     | 105914/225000 [225:41:30<273:00:15,  8.25s/it]
 47%|████▋     | 105915/225000 [225:41:35<243:21:48,  7.36s/it]
 47%|████▋     | 105916/225000 [225:41:43<251:23:30,  7.60s/it]
 47%|████▋     | 105917/225000 [225:41:52<261:43:46,  7.91s/it]
 47%|████▋     | 105918/225000 [225:42:07<335:57:30, 10.16s/it]
 47%|████▋     | 105919/225000 [225:42:16<322:35:42,  9.75s/it]
 47%|████▋     | 105920/225000 [225:42:18<249:37:31,  7.55s/it]
 47%|████▋     | 105921/225000 [225:42:26<253:11:09,  7.65s/it]
 47%|████▋     | 105922/225000 [225:42:34<258:10:31,  7.81s/it]
 47%|████▋     | 105923/225000 [225:42:45<286:48:37,  8.67s/it]
 47%|████▋     | 105924/225000 [225:42:55<301:50:58,  9.13s/it]
 47%|████▋     | 105925/225000 [225:42:58<236:49:48,  7.16s/it]
 47%|████▋     | 105926/225000 [225:43:13<312:02:09,  9.43s/it]
 47%|████▋     | 105927/225000 [225:43:17<266:15:40,  8.05s/it]
 47%|████▋     | 105928/225000 [225:43:30<315:55:14,  9.55s/it]
 47%|████▋     | 105929/225000 [225:43:41<320:52:41,  9.70s/it]
 47%|████▋     | 105930/225000 [225:43:49<313:21:59,  9.47s/it]
 47%|████▋     | 105931/225000 [225:44:00<328:18:43,  9.93s/it]
 47%|████▋     | 105932/225000 [225:44:08<308:55:47,  9.34s/it]
 47%|████▋     | 105933/225000 [225:44:16<288:43:54,  8.73s/it]
 47%|████▋     | 105934/225000 [225:44:26<304:48:33,  9.22s/it]
 47%|████▋     | 105935/225000 [225:44:33<281:30:39,  8.51s/it]
 47%|████▋     | 105936/225000 [225:44:43<297:34:08,  9.00s/it]
 47%|████▋     | 105937/225000 [225:44:51<286:03:57,  8.65s/it]
 47%|████▋     | 105938/225000 [225:44:57<264:45:08,  8.01s/it]
 47%|████▋     | 105939/225000 [225:45:03<238:56:55,  7.22s/it]
 47%|████▋     | 105940/225000 [225:45:12<254:40:15,  7.70s/it]
 47%|████▋     | 105941/225000 [225:45:29<353:23:05, 10.69s/it]
 47%|████▋     | 105942/225000 [225:45:36<314:55:23,  9.52s/it]
 47%|████▋     | 105943/225000 [225:45:45<307:17:27,  9.29s/it]
 47%|████▋     | 105944/225000 [225:45:54<307:11:14,  9.29s/it]
 47%|████▋     | 105945/225000 [225:46:02<288:39:38,  8.73s/it]
 47%|████▋     | 105946/225000 [225:46:11<296:46:26,  8.97s/it]
 47%|████▋     | 105947/225000 [225:46:17<261:19:02,  7.90s/it]
 47%|████▋     | 105948/225000 [225:46:22<239:54:19,  7.25s/it]
 47%|████▋     | 105949/225000 [225:46:25<197:56:06,  5.99s/it]
 47%|████▋     | 105950/225000 [225:46:42<305:45:49,  9.25s/it]
                                                               
{'loss': '0.9273', 'grad_norm': '1.665', 'learning_rate': '2.922e-05', 'epoch': '0.4709', 'num_input_tokens_seen': 378635904, 'train_runtime': '8.128e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 105950/225000 [225:46:42<305:45:49,  9.25s/it]
 47%|████▋     | 105951/225000 [225:46:50<287:19:30,  8.69s/it]
 47%|████▋     | 105952/225000 [225:46:56<260:51:33,  7.89s/it]
 47%|████▋     | 105953/225000 [225:46:59<214:10:33,  6.48s/it]
 47%|████▋     | 105954/225000 [225:47:11<271:08:20,  8.20s/it]
 47%|████▋     | 105955/225000 [225:47:23<308:29:37,  9.33s/it]
 47%|████▋     | 105956/225000 [225:47:26<247:16:38,  7.48s/it]
 47%|████▋     | 105957/225000 [225:47:34<255:02:15,  7.71s/it]
 47%|████▋     | 105958/225000 [225:47:55<382:20:16, 11.56s/it]
 47%|████▋     | 105959/225000 [225:47:58<302:56:24,  9.16s/it]
 47%|████▋     | 105960/225000 [225:48:11<340:27:35, 10.30s/it]
 47%|████▋     | 105961/225000 [225:48:20<319:46:07,  9.67s/it]
 47%|████▋     | 105962/225000 [225:48:27<294:45:22,  8.91s/it]
 47%|████▋     | 105963/225000 [225:48:30<238:56:31,  7.23s/it]
 47%|████▋     | 105964/225000 [225:48:32<189:31:21,  5.73s/it]
 47%|████▋     | 105965/225000 [225:48:39<197:50:45,  5.98s/it]
 47%|████▋     | 105966/225000 [225:48:42<168:44:23,  5.10s/it]
 47%|████▋     | 105967/225000 [225:48:51<208:02:45,  6.29s/it]
 47%|████▋     | 105968/225000 [225:49:00<234:01:51,  7.08s/it]
 47%|████▋     | 105969/225000 [225:49:08<249:03:23,  7.53s/it]
 47%|████▋     | 105970/225000 [225:49:23<318:23:33,  9.63s/it]
 47%|████▋     | 105971/225000 [225:49:35<338:46:12, 10.25s/it]
 47%|████▋     | 105972/225000 [225:49:46<347:16:12, 10.50s/it]
 47%|████▋     | 105973/225000 [225:49:51<296:21:10,  8.96s/it]
 47%|████▋     | 105974/225000 [225:50:05<346:09:06, 10.47s/it]
 47%|████▋     | 105975/225000 [225:50:14<325:38:05,  9.85s/it]
 47%|████▋     | 105976/225000 [225:50:21<305:38:21,  9.24s/it]
 47%|████▋     | 105977/225000 [225:50:25<245:28:49,  7.42s/it]
 47%|████▋     | 105978/225000 [225:50:27<195:56:12,  5.93s/it]
 47%|████▋     | 105979/225000 [225:50:33<201:16:42,  6.09s/it]
 47%|████▋     | 105980/225000 [225:50:38<185:51:37,  5.62s/it]
 47%|████▋     | 105981/225000 [225:50:53<275:07:40,  8.32s/it]
 47%|████▋     | 105982/225000 [225:50:56<224:27:34,  6.79s/it]
 47%|████▋     | 105983/225000 [225:50:59<189:01:23,  5.72s/it]
 47%|████▋     | 105984/225000 [225:51:03<171:38:21,  5.19s/it]
 47%|████▋     | 105985/225000 [225:51:07<162:13:06,  4.91s/it]
 47%|████▋     | 105986/225000 [225:51:18<222:49:57,  6.74s/it]
 47%|████▋     | 105987/225000 [225:51:25<222:52:38,  6.74s/it]
 47%|████▋     | 105988/225000 [225:51:29<195:05:12,  5.90s/it]
 47%|████▋     | 105989/225000 [225:51:38<227:34:09,  6.88s/it]
 47%|████▋     | 105990/225000 [225:51:44<220:42:39,  6.68s/it]
 47%|████▋     | 105991/225000 [225:51:58<285:28:59,  8.64s/it]
 47%|████▋     | 105992/225000 [225:52:02<247:33:32,  7.49s/it]
 47%|████▋     | 105993/225000 [225:52:15<300:51:31,  9.10s/it]
 47%|████▋     | 105994/225000 [225:52:20<255:52:44,  7.74s/it]
 47%|████▋     | 105995/225000 [225:52:26<241:20:03,  7.30s/it]
 47%|████▋     | 105996/225000 [225:52:30<205:49:54,  6.23s/it]
 47%|████▋     | 105997/225000 [225:52:34<188:36:05,  5.71s/it]
 47%|████▋     | 105998/225000 [225:52:37<161:37:37,  4.89s/it]
 47%|████▋     | 105999/225000 [225:52:48<217:00:12,  6.56s/it]
 47%|████▋     | 106000/225000 [225:52:55<222:59:46,  6.75s/it]
                                                               
{'loss': '0.9422', 'grad_norm': '4.309', 'learning_rate': '2.92e-05', 'epoch': '0.4711', 'num_input_tokens_seen': 378804808, 'train_runtime': '8.132e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106000/225000 [225:52:55<222:59:46,  6.75s/it]
 47%|████▋     | 106001/225000 [225:53:03<241:30:32,  7.31s/it]
 47%|████▋     | 106002/225000 [225:53:12<251:14:58,  7.60s/it]
 47%|████▋     | 106003/225000 [225:53:20<256:58:38,  7.77s/it]
 47%|████▋     | 106004/225000 [225:53:23<208:59:43,  6.32s/it]
 47%|████▋     | 106005/225000 [225:53:31<230:10:35,  6.96s/it]
 47%|████▋     | 106006/225000 [225:53:38<227:05:02,  6.87s/it]
 47%|████▋     | 106007/225000 [225:53:41<188:29:03,  5.70s/it]
 47%|████▋     | 106008/225000 [225:53:51<229:11:05,  6.93s/it]
 47%|████▋     | 106009/225000 [225:54:01<263:41:02,  7.98s/it]
 47%|████▋     | 106010/225000 [225:54:08<256:53:00,  7.77s/it]
 47%|████▋     | 106011/225000 [225:54:15<248:34:48,  7.52s/it]
 47%|████▋     | 106012/225000 [225:54:23<249:27:33,  7.55s/it]
 47%|████▋     | 106013/225000 [225:54:27<212:45:12,  6.44s/it]
 47%|████▋     | 106014/225000 [225:54:30<179:20:03,  5.43s/it]
 47%|████▋     | 106015/225000 [225:54:34<161:11:25,  4.88s/it]
 47%|████▋     | 106016/225000 [225:54:37<143:11:36,  4.33s/it]
 47%|████▋     | 106017/225000 [225:54:40<134:10:18,  4.06s/it]
 47%|████▋     | 106018/225000 [225:54:58<271:04:35,  8.20s/it]
 47%|████▋     | 106019/225000 [225:55:02<233:03:58,  7.05s/it]
 47%|████▋     | 106020/225000 [225:55:12<264:32:10,  8.00s/it]
 47%|████▋     | 106021/225000 [225:55:19<252:29:27,  7.64s/it]
 47%|████▋     | 106022/225000 [225:55:22<207:55:17,  6.29s/it]
 47%|████▋     | 106023/225000 [225:55:35<269:49:16,  8.16s/it]
 47%|████▋     | 106024/225000 [225:55:42<256:17:07,  7.75s/it]
 47%|████▋     | 106025/225000 [225:55:55<313:24:16,  9.48s/it]
 47%|████▋     | 106026/225000 [225:55:57<241:19:02,  7.30s/it]
 47%|████▋     | 106027/225000 [225:56:02<215:06:33,  6.51s/it]
 47%|████▋     | 106028/225000 [225:56:11<236:07:38,  7.15s/it]
 47%|████▋     | 106029/225000 [225:56:17<226:45:25,  6.86s/it]
 47%|████▋     | 106030/225000 [225:56:22<211:39:49,  6.40s/it]
 47%|████▋     | 106031/225000 [225:56:25<170:29:31,  5.16s/it]
 47%|████▋     | 106032/225000 [225:56:30<175:14:51,  5.30s/it]
 47%|████▋     | 106033/225000 [225:56:37<189:05:49,  5.72s/it]
 47%|████▋     | 106034/225000 [225:56:48<245:02:56,  7.42s/it]
 47%|████▋     | 106035/225000 [225:57:10<383:11:34, 11.60s/it]
 47%|████▋     | 106036/225000 [225:57:14<313:15:59,  9.48s/it]
 47%|████▋     | 106037/225000 [225:57:23<307:08:45,  9.29s/it]
 47%|████▋     | 106038/225000 [225:57:29<273:19:28,  8.27s/it]
 47%|████▋     | 106039/225000 [225:57:32<221:12:13,  6.69s/it]
 47%|████▋     | 106040/225000 [225:57:47<305:25:08,  9.24s/it]
 47%|████▋     | 106041/225000 [225:57:57<307:05:39,  9.29s/it]
 47%|████▋     | 106042/225000 [225:58:05<294:23:03,  8.91s/it]
 47%|████▋     | 106043/225000 [225:58:19<351:06:46, 10.63s/it]
 47%|████▋     | 106044/225000 [225:58:22<275:11:51,  8.33s/it]
 47%|████▋     | 106045/225000 [225:58:29<263:36:17,  7.98s/it]
 47%|████▋     | 106046/225000 [225:58:38<267:51:46,  8.11s/it]
 47%|████▋     | 106047/225000 [225:58:43<243:11:24,  7.36s/it]
 47%|████▋     | 106048/225000 [225:58:46<200:07:08,  6.06s/it]
 47%|████▋     | 106049/225000 [225:58:52<199:14:18,  6.03s/it]
 47%|████▋     | 106050/225000 [225:59:01<227:25:01,  6.88s/it]
                                                               
{'loss': '0.941', 'grad_norm': '1.848', 'learning_rate': '2.918e-05', 'epoch': '0.4713', 'num_input_tokens_seen': 378985504, 'train_runtime': '8.135e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106050/225000 [225:59:01<227:25:01,  6.88s/it]
 47%|████▋     | 106051/225000 [225:59:04<188:52:37,  5.72s/it]
 47%|████▋     | 106052/225000 [225:59:11<196:06:21,  5.94s/it]
 47%|████▋     | 106053/225000 [225:59:14<167:42:06,  5.08s/it]
 47%|████▋     | 106054/225000 [225:59:22<204:34:03,  6.19s/it]
 47%|████▋     | 106055/225000 [225:59:28<193:30:00,  5.86s/it]
 47%|████▋     | 106056/225000 [225:59:32<181:18:55,  5.49s/it]
 47%|████▋     | 106057/225000 [225:59:38<182:16:19,  5.52s/it]
 47%|████▋     | 106058/225000 [225:59:42<165:57:40,  5.02s/it]
 47%|████▋     | 106059/225000 [225:59:46<156:12:34,  4.73s/it]
 47%|████▋     | 106060/225000 [225:59:59<245:32:30,  7.43s/it]
 47%|████▋     | 106061/225000 [226:00:02<202:08:01,  6.12s/it]
 47%|████▋     | 106062/225000 [226:00:13<247:31:25,  7.49s/it]
 47%|████▋     | 106063/225000 [226:00:33<367:44:35, 11.13s/it]
 47%|████▋     | 106064/225000 [226:00:44<371:48:13, 11.25s/it]
 47%|████▋     | 106065/225000 [226:00:53<349:09:00, 10.57s/it]
 47%|████▋     | 106066/225000 [226:01:00<312:31:29,  9.46s/it]
 47%|████▋     | 106067/225000 [226:01:02<241:38:40,  7.31s/it]
 47%|████▋     | 106068/225000 [226:01:09<229:24:55,  6.94s/it]
 47%|████▋     | 106069/225000 [226:01:20<273:47:13,  8.29s/it]
 47%|████▋     | 106070/225000 [226:01:30<290:09:59,  8.78s/it]
 47%|████▋     | 106071/225000 [226:01:33<234:09:29,  7.09s/it]
 47%|████▋     | 106072/225000 [226:01:38<211:03:19,  6.39s/it]
 47%|████▋     | 106073/225000 [226:01:42<190:28:14,  5.77s/it]
 47%|████▋     | 106074/225000 [226:01:46<169:27:00,  5.13s/it]
 47%|████▋     | 106075/225000 [226:02:02<278:21:23,  8.43s/it]
 47%|████▋     | 106076/225000 [226:02:05<223:23:01,  6.76s/it]
 47%|████▋     | 106077/225000 [226:02:10<205:29:38,  6.22s/it]
 47%|████▋     | 106078/225000 [226:02:22<265:38:21,  8.04s/it]
 47%|████▋     | 106079/225000 [226:02:28<249:48:05,  7.56s/it]
 47%|████▋     | 106080/225000 [226:02:37<260:32:48,  7.89s/it]
 47%|████▋     | 106081/225000 [226:02:41<220:04:23,  6.66s/it]
 47%|████▋     | 106082/225000 [226:02:48<223:34:59,  6.77s/it]
 47%|████▋     | 106083/225000 [226:02:52<194:24:28,  5.89s/it]
 47%|████▋     | 106084/225000 [226:02:58<201:44:41,  6.11s/it]
 47%|████▋     | 106085/225000 [226:03:05<203:56:57,  6.17s/it]
 47%|████▋     | 106086/225000 [226:03:09<182:04:42,  5.51s/it]
 47%|████▋     | 106087/225000 [226:03:24<281:47:25,  8.53s/it]
 47%|████▋     | 106088/225000 [226:03:26<219:27:42,  6.64s/it]
 47%|████▋     | 106089/225000 [226:03:37<259:02:08,  7.84s/it]
 47%|████▋     | 106090/225000 [226:03:46<266:20:28,  8.06s/it]
 47%|████▋     | 106091/225000 [226:03:55<276:59:57,  8.39s/it]
 47%|████▋     | 106092/225000 [226:04:11<356:11:18, 10.78s/it]
 47%|████▋     | 106093/225000 [226:04:20<334:12:59, 10.12s/it]
 47%|████▋     | 106094/225000 [226:04:27<310:18:19,  9.39s/it]
 47%|████▋     | 106095/225000 [226:04:33<270:04:43,  8.18s/it]
 47%|████▋     | 106096/225000 [226:04:36<220:43:28,  6.68s/it]
 47%|████▋     | 106097/225000 [226:04:42<214:46:13,  6.50s/it]
 47%|████▋     | 106098/225000 [226:05:00<332:00:04, 10.05s/it]
 47%|████▋     | 106099/225000 [226:05:11<340:00:28, 10.29s/it]
 47%|████▋     | 106100/225000 [226:05:13<259:39:34,  7.86s/it]
                                                               
{'loss': '0.9462', 'grad_norm': '5.045', 'learning_rate': '2.916e-05', 'epoch': '0.4716', 'num_input_tokens_seen': 379157256, 'train_runtime': '8.139e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106100/225000 [226:05:13<259:39:34,  7.86s/it]
 47%|████▋     | 106101/225000 [226:05:23<272:02:00,  8.24s/it]
 47%|████▋     | 106102/225000 [226:05:26<220:17:49,  6.67s/it]
 47%|████▋     | 106103/225000 [226:05:28<176:36:03,  5.35s/it]
 47%|████▋     | 106104/225000 [226:05:36<208:48:33,  6.32s/it]
 47%|████▋     | 106105/225000 [226:05:53<313:03:47,  9.48s/it]
 47%|████▋     | 106106/225000 [226:05:57<256:42:12,  7.77s/it]
 47%|████▋     | 106107/225000 [226:06:04<250:14:22,  7.58s/it]
 47%|████▋     | 106108/225000 [226:06:13<259:55:21,  7.87s/it]
 47%|████▋     | 106109/225000 [226:06:19<245:11:16,  7.42s/it]
 47%|████▋     | 106110/225000 [226:06:25<231:26:41,  7.01s/it]
 47%|████▋     | 106111/225000 [226:06:28<193:05:34,  5.85s/it]
 47%|████▋     | 106112/225000 [226:06:37<217:39:59,  6.59s/it]
 47%|████▋     | 106113/225000 [226:06:45<235:55:23,  7.14s/it]
 47%|████▋     | 106114/225000 [226:06:51<223:07:14,  6.76s/it]
 47%|████▋     | 106115/225000 [226:06:53<180:21:58,  5.46s/it]
 47%|████▋     | 106116/225000 [226:07:03<221:34:30,  6.71s/it]
 47%|████▋     | 106117/225000 [226:07:06<188:59:27,  5.72s/it]
 47%|████▋     | 106118/225000 [226:07:10<166:07:32,  5.03s/it]
 47%|████▋     | 106119/225000 [226:07:20<218:21:15,  6.61s/it]
 47%|████▋     | 106120/225000 [226:07:30<248:08:17,  7.51s/it]
 47%|████▋     | 106121/225000 [226:07:34<219:05:32,  6.63s/it]
 47%|████▋     | 106122/225000 [226:07:41<216:51:57,  6.57s/it]
 47%|████▋     | 106123/225000 [226:07:48<223:47:05,  6.78s/it]
 47%|████▋     | 106124/225000 [226:07:51<185:57:30,  5.63s/it]
 47%|████▋     | 106125/225000 [226:07:56<181:24:42,  5.49s/it]
 47%|████▋     | 106126/225000 [226:08:02<189:38:44,  5.74s/it]
 47%|████▋     | 106127/225000 [226:08:07<175:19:11,  5.31s/it]
 47%|████▋     | 106128/225000 [226:08:15<204:38:09,  6.20s/it]
 47%|████▋     | 106129/225000 [226:08:19<182:20:30,  5.52s/it]
 47%|████▋     | 106130/225000 [226:08:22<157:08:59,  4.76s/it]
 47%|████▋     | 106131/225000 [226:08:33<222:49:43,  6.75s/it]
 47%|████▋     | 106132/225000 [226:08:40<224:17:05,  6.79s/it]
 47%|████▋     | 106133/225000 [226:08:46<215:06:39,  6.51s/it]
 47%|████▋     | 106134/225000 [226:08:51<199:11:08,  6.03s/it]
 47%|████▋     | 106135/225000 [226:09:05<274:43:53,  8.32s/it]
 47%|████▋     | 106136/225000 [226:09:14<284:49:49,  8.63s/it]
 47%|████▋     | 106137/225000 [226:09:23<284:58:57,  8.63s/it]
 47%|████▋     | 106138/225000 [226:09:28<247:16:42,  7.49s/it]
 47%|████▋     | 106139/225000 [226:09:36<254:27:57,  7.71s/it]
 47%|████▋     | 106140/225000 [226:09:42<235:39:25,  7.14s/it]
 47%|████▋     | 106141/225000 [226:09:50<247:39:03,  7.50s/it]
 47%|████▋     | 106142/225000 [226:09:55<227:56:49,  6.90s/it]
 47%|████▋     | 106143/225000 [226:09:59<190:30:35,  5.77s/it]
 47%|████▋     | 106144/225000 [226:10:02<172:41:13,  5.23s/it]
 47%|████▋     | 106145/225000 [226:10:17<260:02:55,  7.88s/it]
 47%|████▋     | 106146/225000 [226:10:24<252:44:44,  7.66s/it]
 47%|████▋     | 106147/225000 [226:10:30<244:16:29,  7.40s/it]
 47%|████▋     | 106148/225000 [226:10:33<199:14:41,  6.04s/it]
 47%|████▋     | 106149/225000 [226:10:43<235:07:59,  7.12s/it]
 47%|████▋     | 106150/225000 [226:10:47<208:57:38,  6.33s/it]
                                                               
{'loss': '0.9824', 'grad_norm': '3.132', 'learning_rate': '2.915e-05', 'epoch': '0.4718', 'num_input_tokens_seen': 379308072, 'train_runtime': '8.142e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106150/225000 [226:10:47<208:57:38,  6.33s/it]
 47%|████▋     | 106151/225000 [226:10:56<232:21:50,  7.04s/it]
 47%|████▋     | 106152/225000 [226:11:02<215:36:02,  6.53s/it]
 47%|████▋     | 106153/225000 [226:11:07<208:33:07,  6.32s/it]
 47%|████▋     | 106154/225000 [226:11:10<175:11:59,  5.31s/it]
 47%|████▋     | 106155/225000 [226:11:23<250:22:36,  7.58s/it]
 47%|████▋     | 106156/225000 [226:11:32<259:06:33,  7.85s/it]
 47%|████▋     | 106157/225000 [226:11:41<271:17:58,  8.22s/it]
 47%|████▋     | 106158/225000 [226:11:54<326:15:47,  9.88s/it]
 47%|████▋     | 106159/225000 [226:12:07<347:56:35, 10.54s/it]
 47%|████▋     | 106160/225000 [226:12:13<310:16:39,  9.40s/it]
 47%|████▋     | 106161/225000 [226:12:17<257:57:53,  7.81s/it]
 47%|████▋     | 106162/225000 [226:12:22<230:12:53,  6.97s/it]
 47%|████▋     | 106163/225000 [226:12:30<240:08:29,  7.27s/it]
 47%|████▋     | 106164/225000 [226:12:35<217:30:23,  6.59s/it]
 47%|████▋     | 106165/225000 [226:12:51<307:30:28,  9.32s/it]
 47%|████▋     | 106166/225000 [226:13:01<312:15:43,  9.46s/it]
 47%|████▋     | 106167/225000 [226:13:12<327:20:50,  9.92s/it]
 47%|████▋     | 106168/225000 [226:13:16<275:07:33,  8.33s/it]
 47%|████▋     | 106169/225000 [226:13:24<264:06:07,  8.00s/it]
 47%|████▋     | 106170/225000 [226:13:28<226:34:50,  6.86s/it]
 47%|████▋     | 106171/225000 [226:13:33<206:05:47,  6.24s/it]
 47%|████▋     | 106172/225000 [226:13:38<195:38:05,  5.93s/it]
 47%|████▋     | 106173/225000 [226:13:54<298:14:46,  9.04s/it]
 47%|████▋     | 106174/225000 [226:13:57<239:43:06,  7.26s/it]
 47%|████▋     | 106175/225000 [226:14:10<290:04:00,  8.79s/it]
 47%|████▋     | 106176/225000 [226:14:16<264:59:14,  8.03s/it]
 47%|████▋     | 106177/225000 [226:14:19<219:25:28,  6.65s/it]
 47%|████▋     | 106178/225000 [226:14:22<175:27:07,  5.32s/it]
 47%|████▋     | 106179/225000 [226:14:38<284:15:31,  8.61s/it]
 47%|████▋     | 106180/225000 [226:14:44<260:02:23,  7.88s/it]
 47%|████▋     | 106181/225000 [226:14:47<214:26:37,  6.50s/it]
 47%|████▋     | 106182/225000 [226:14:56<231:22:26,  7.01s/it]
 47%|████▋     | 106183/225000 [226:15:01<214:47:45,  6.51s/it]
 47%|████▋     | 106184/225000 [226:15:05<187:31:15,  5.68s/it]
 47%|████▋     | 106185/225000 [226:15:15<234:56:51,  7.12s/it]
 47%|████▋     | 106186/225000 [226:15:21<221:17:17,  6.70s/it]
 47%|████▋     | 106187/225000 [226:15:26<204:54:09,  6.21s/it]
 47%|████▋     | 106188/225000 [226:15:29<177:19:24,  5.37s/it]
 47%|████▋     | 106189/225000 [226:15:39<217:54:32,  6.60s/it]
 47%|████▋     | 106190/225000 [226:16:02<383:46:07, 11.63s/it]
 47%|████▋     | 106191/225000 [226:16:08<328:03:41,  9.94s/it]
 47%|████▋     | 106192/225000 [226:16:12<267:28:41,  8.10s/it]
 47%|████▋     | 106193/225000 [226:16:17<238:00:36,  7.21s/it]
 47%|████▋     | 106194/225000 [226:16:23<226:30:07,  6.86s/it]
 47%|████▋     | 106195/225000 [226:16:28<209:44:26,  6.36s/it]
 47%|████▋     | 106196/225000 [226:16:48<346:34:16, 10.50s/it]
 47%|████▋     | 106197/225000 [226:16:55<304:47:22,  9.24s/it]
 47%|████▋     | 106198/225000 [226:17:00<267:01:20,  8.09s/it]
 47%|████▋     | 106199/225000 [226:17:06<247:35:57,  7.50s/it]
 47%|████▋     | 106200/225000 [226:17:17<278:39:16,  8.44s/it]
                                                               
{'loss': '0.9483', 'grad_norm': '2.191', 'learning_rate': '2.913e-05', 'epoch': '0.472', 'num_input_tokens_seen': 379483504, 'train_runtime': '8.146e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106200/225000 [226:17:17<278:39:16,  8.44s/it]
 47%|████▋     | 106201/225000 [226:17:24<268:47:17,  8.15s/it]
 47%|████▋     | 106202/225000 [226:17:35<291:19:43,  8.83s/it]
 47%|████▋     | 106203/225000 [226:17:37<225:24:38,  6.83s/it]
 47%|████▋     | 106204/225000 [226:17:43<221:18:29,  6.71s/it]
 47%|████▋     | 106205/225000 [226:17:49<213:01:42,  6.46s/it]
 47%|████▋     | 106206/225000 [226:17:58<240:13:17,  7.28s/it]
 47%|████▋     | 106207/225000 [226:18:03<212:36:01,  6.44s/it]
 47%|████▋     | 106208/225000 [226:18:12<237:13:36,  7.19s/it]
 47%|████▋     | 106209/225000 [226:18:18<222:35:28,  6.75s/it]
 47%|████▋     | 106210/225000 [226:18:33<307:58:10,  9.33s/it]
 47%|████▋     | 106211/225000 [226:18:45<338:17:36, 10.25s/it]
 47%|████▋     | 106212/225000 [226:18:57<352:58:48, 10.70s/it]
 47%|████▋     | 106213/225000 [226:19:03<300:57:23,  9.12s/it]
 47%|████▋     | 106214/225000 [226:19:10<284:59:05,  8.64s/it]
 47%|████▋     | 106215/225000 [226:19:17<268:10:48,  8.13s/it]
 47%|████▋     | 106216/225000 [226:19:28<295:51:34,  8.97s/it]
 47%|████▋     | 106217/225000 [226:19:44<362:20:27, 10.98s/it]
 47%|████▋     | 106218/225000 [226:19:54<358:33:14, 10.87s/it]
 47%|████▋     | 106219/225000 [226:20:02<330:18:54, 10.01s/it]
 47%|████▋     | 106220/225000 [226:20:13<339:29:08, 10.29s/it]
 47%|████▋     | 106221/225000 [226:20:18<285:26:21,  8.65s/it]
 47%|████▋     | 106222/225000 [226:20:31<330:53:21, 10.03s/it]
 47%|████▋     | 106223/225000 [226:20:38<294:59:21,  8.94s/it]
 47%|████▋     | 106224/225000 [226:20:41<238:03:43,  7.22s/it]
 47%|████▋     | 106225/225000 [226:20:50<262:11:14,  7.95s/it]
 47%|████▋     | 106226/225000 [226:20:53<209:40:46,  6.36s/it]
 47%|████▋     | 106227/225000 [226:21:02<238:16:22,  7.22s/it]
 47%|████▋     | 106228/225000 [226:21:07<213:50:35,  6.48s/it]
 47%|████▋     | 106229/225000 [226:21:12<201:01:37,  6.09s/it]
 47%|████▋     | 106230/225000 [226:21:20<216:54:44,  6.57s/it]
 47%|████▋     | 106231/225000 [226:21:25<201:53:13,  6.12s/it]
 47%|████▋     | 106232/225000 [226:21:34<228:57:52,  6.94s/it]
 47%|████▋     | 106233/225000 [226:21:41<226:40:13,  6.87s/it]
 47%|████▋     | 106234/225000 [226:21:49<240:22:39,  7.29s/it]
 47%|████▋     | 106235/225000 [226:21:52<199:13:52,  6.04s/it]
 47%|████▋     | 106236/225000 [226:22:04<260:36:39,  7.90s/it]
 47%|████▋     | 106237/225000 [226:22:06<204:31:26,  6.20s/it]
 47%|████▋     | 106238/225000 [226:22:12<193:47:02,  5.87s/it]
 47%|████▋     | 106239/225000 [226:22:21<230:05:01,  6.97s/it]
 47%|████▋     | 106240/225000 [226:22:30<247:17:43,  7.50s/it]
 47%|████▋     | 106241/225000 [226:22:38<253:00:43,  7.67s/it]
 47%|████▋     | 106242/225000 [226:22:53<326:56:29,  9.91s/it]
 47%|████▋     | 106243/225000 [226:22:56<255:35:23,  7.75s/it]
 47%|████▋     | 106244/225000 [226:22:59<213:05:37,  6.46s/it]
 47%|████▋     | 106245/225000 [226:23:06<218:26:03,  6.62s/it]
 47%|████▋     | 106246/225000 [226:23:13<219:04:47,  6.64s/it]
 47%|████▋     | 106247/225000 [226:23:25<276:23:40,  8.38s/it]
 47%|████▋     | 106248/225000 [226:23:39<331:48:10, 10.06s/it]
 47%|████▋     | 106249/225000 [226:23:51<350:45:39, 10.63s/it]
 47%|████▋     | 106250/225000 [226:23:55<286:15:52,  8.68s/it]
                                                               
{'loss': '0.9508', 'grad_norm': '3.073', 'learning_rate': '2.911e-05', 'epoch': '0.4722', 'num_input_tokens_seen': 379652600, 'train_runtime': '8.15e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106250/225000 [226:23:55<286:15:52,  8.68s/it]
 47%|████▋     | 106251/225000 [226:23:58<221:55:21,  6.73s/it]
 47%|████▋     | 106252/225000 [226:24:08<253:47:57,  7.69s/it]
 47%|████▋     | 106253/225000 [226:24:11<210:33:11,  6.38s/it]
 47%|████▋     | 106254/225000 [226:24:20<235:35:17,  7.14s/it]
 47%|████▋     | 106255/225000 [226:24:28<249:41:53,  7.57s/it]
 47%|████▋     | 106256/225000 [226:24:36<251:25:30,  7.62s/it]
 47%|████▋     | 106257/225000 [226:24:46<270:58:43,  8.22s/it]
 47%|████▋     | 106258/225000 [226:24:53<258:43:17,  7.84s/it]
 47%|████▋     | 106259/225000 [226:24:56<212:03:43,  6.43s/it]
 47%|████▋     | 106260/225000 [226:25:02<214:49:01,  6.51s/it]
 47%|████▋     | 106261/225000 [226:25:10<226:21:22,  6.86s/it]
 47%|████▋     | 106262/225000 [226:25:18<239:39:49,  7.27s/it]
 47%|████▋     | 106263/225000 [226:25:35<330:35:15, 10.02s/it]
 47%|████▋     | 106264/225000 [226:25:40<282:00:46,  8.55s/it]
 47%|████▋     | 106265/225000 [226:25:51<305:43:55,  9.27s/it]
 47%|████▋     | 106266/225000 [226:26:00<308:15:44,  9.35s/it]
 47%|████▋     | 106267/225000 [226:26:04<253:53:10,  7.70s/it]
 47%|████▋     | 106268/225000 [226:26:09<226:38:29,  6.87s/it]
 47%|████▋     | 106269/225000 [226:26:12<189:09:11,  5.74s/it]
 47%|████▋     | 106270/225000 [226:26:21<214:28:15,  6.50s/it]
 47%|████▋     | 106271/225000 [226:26:26<203:21:16,  6.17s/it]
 47%|████▋     | 106272/225000 [226:26:34<223:30:15,  6.78s/it]
 47%|████▋     | 106273/225000 [226:26:44<249:32:31,  7.57s/it]
 47%|████▋     | 106274/225000 [226:26:57<310:00:25,  9.40s/it]
 47%|████▋     | 106275/225000 [226:27:03<275:02:18,  8.34s/it]
 47%|████▋     | 106276/225000 [226:27:14<299:17:12,  9.08s/it]
 47%|████▋     | 106277/225000 [226:27:20<271:24:31,  8.23s/it]
 47%|████▋     | 106278/225000 [226:27:23<222:08:52,  6.74s/it]
 47%|████▋     | 106279/225000 [226:27:30<221:59:27,  6.73s/it]
 47%|████▋     | 106280/225000 [226:27:34<197:41:34,  5.99s/it]
 47%|████▋     | 106281/225000 [226:27:37<167:27:34,  5.08s/it]
 47%|████▋     | 106282/225000 [226:27:48<220:41:57,  6.69s/it]
 47%|████▋     | 106283/225000 [226:27:55<225:22:05,  6.83s/it]
 47%|████▋     | 106284/225000 [226:28:07<278:49:42,  8.46s/it]
 47%|████▋     | 106285/225000 [226:28:19<311:48:12,  9.46s/it]
 47%|████▋     | 106286/225000 [226:28:30<325:01:40,  9.86s/it]
 47%|████▋     | 106287/225000 [226:28:43<359:08:26, 10.89s/it]
 47%|████▋     | 106288/225000 [226:28:57<392:09:33, 11.89s/it]
 47%|████▋     | 106289/225000 [226:29:11<406:28:57, 12.33s/it]
 47%|████▋     | 106290/225000 [226:29:21<388:34:04, 11.78s/it]
 47%|████▋     | 106291/225000 [226:29:29<350:22:54, 10.63s/it]
 47%|████▋     | 106292/225000 [226:29:43<382:01:10, 11.59s/it]
 47%|████▋     | 106293/225000 [226:29:53<369:46:11, 11.21s/it]
 47%|████▋     | 106294/225000 [226:30:01<337:42:04, 10.24s/it]
 47%|████▋     | 106295/225000 [226:30:14<362:28:24, 10.99s/it]
 47%|████▋     | 106296/225000 [226:30:19<302:58:25,  9.19s/it]
 47%|████▋     | 106297/225000 [226:30:22<240:52:59,  7.31s/it]
 47%|████▋     | 106298/225000 [226:30:30<246:00:23,  7.46s/it]
 47%|████▋     | 106299/225000 [226:30:38<256:09:43,  7.77s/it]
 47%|████▋     | 106300/225000 [226:30:47<270:27:03,  8.20s/it]
                                                               
{'loss': '0.9072', 'grad_norm': '1.806', 'learning_rate': '2.909e-05', 'epoch': '0.4724', 'num_input_tokens_seen': 379839360, 'train_runtime': '8.154e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106300/225000 [226:30:47<270:27:03,  8.20s/it]
 47%|████▋     | 106301/225000 [226:30:53<239:30:56,  7.26s/it]
 47%|████▋     | 106302/225000 [226:30:57<212:40:12,  6.45s/it]
 47%|████▋     | 106303/225000 [226:31:10<280:14:24,  8.50s/it]
 47%|████▋     | 106304/225000 [226:31:13<226:42:30,  6.88s/it]
 47%|████▋     | 106305/225000 [226:31:23<256:08:34,  7.77s/it]
 47%|████▋     | 106306/225000 [226:31:29<231:05:44,  7.01s/it]
 47%|████▋     | 106307/225000 [226:31:31<185:23:40,  5.62s/it]
 47%|████▋     | 106308/225000 [226:31:40<217:35:30,  6.60s/it]
 47%|████▋     | 106309/225000 [226:31:54<289:49:54,  8.79s/it]
 47%|████▋     | 106310/225000 [226:32:00<263:52:03,  8.00s/it]
 47%|████▋     | 106311/225000 [226:32:03<220:29:01,  6.69s/it]
 47%|████▋     | 106312/225000 [226:32:06<175:29:13,  5.32s/it]
 47%|████▋     | 106313/225000 [226:32:12<184:22:10,  5.59s/it]
 47%|████▋     | 106314/225000 [226:32:17<179:16:33,  5.44s/it]
 47%|████▋     | 106315/225000 [226:32:25<206:29:13,  6.26s/it]
 47%|████▋     | 106316/225000 [226:32:30<190:24:54,  5.78s/it]
 47%|████▋     | 106317/225000 [226:32:44<274:34:57,  8.33s/it]
 47%|████▋     | 106318/225000 [226:32:48<236:08:21,  7.16s/it]
 47%|████▋     | 106319/225000 [226:32:59<272:16:28,  8.26s/it]
 47%|████▋     | 106320/225000 [226:33:03<224:50:12,  6.82s/it]
 47%|████▋     | 106321/225000 [226:33:06<189:49:53,  5.76s/it]
 47%|████▋     | 106322/225000 [226:33:17<245:46:04,  7.46s/it]
 47%|████▋     | 106323/225000 [226:33:32<314:16:20,  9.53s/it]
 47%|████▋     | 106324/225000 [226:33:43<333:18:19, 10.11s/it]
 47%|████▋     | 106325/225000 [226:33:52<316:25:16,  9.60s/it]
 47%|████▋     | 106326/225000 [226:33:59<294:14:32,  8.93s/it]
 47%|████▋     | 106327/225000 [226:34:09<302:42:55,  9.18s/it]
 47%|████▋     | 106328/225000 [226:34:14<262:12:03,  7.95s/it]
 47%|████▋     | 106329/225000 [226:34:20<248:38:18,  7.54s/it]
 47%|████▋     | 106330/225000 [226:34:24<204:20:14,  6.20s/it]
 47%|████▋     | 106331/225000 [226:34:26<165:57:22,  5.03s/it]
 47%|████▋     | 106332/225000 [226:34:30<153:32:50,  4.66s/it]
 47%|████▋     | 106333/225000 [226:34:37<182:30:02,  5.54s/it]
 47%|████▋     | 106334/225000 [226:34:50<257:31:46,  7.81s/it]
 47%|████▋     | 106335/225000 [226:34:58<252:05:41,  7.65s/it]
 47%|████▋     | 106336/225000 [226:35:01<213:55:30,  6.49s/it]
 47%|████▋     | 106337/225000 [226:35:05<189:27:25,  5.75s/it]
 47%|████▋     | 106338/225000 [226:35:16<239:02:03,  7.25s/it]
 47%|████▋     | 106339/225000 [226:35:22<226:08:21,  6.86s/it]
 47%|████▋     | 106340/225000 [226:35:25<188:00:35,  5.70s/it]
 47%|████▋     | 106341/225000 [226:35:37<248:23:29,  7.54s/it]
 47%|████▋     | 106342/225000 [226:35:43<232:57:19,  7.07s/it]
 47%|████▋     | 106343/225000 [226:35:53<258:32:42,  7.84s/it]
 47%|████▋     | 106344/225000 [226:36:01<264:21:43,  8.02s/it]
 47%|████▋     | 106345/225000 [226:36:08<249:25:22,  7.57s/it]
 47%|████▋     | 106346/225000 [226:36:19<291:20:40,  8.84s/it]
 47%|████▋     | 106347/225000 [226:36:26<273:15:40,  8.29s/it]
 47%|████▋     | 106348/225000 [226:36:30<231:44:09,  7.03s/it]
 47%|████▋     | 106349/225000 [226:36:43<288:01:43,  8.74s/it]
 47%|████▋     | 106350/225000 [226:36:52<286:00:13,  8.68s/it]
                                                               
{'loss': '0.9384', 'grad_norm': '1.133', 'learning_rate': '2.907e-05', 'epoch': '0.4727', 'num_input_tokens_seen': 380008496, 'train_runtime': '8.158e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106350/225000 [226:36:52<286:00:13,  8.68s/it]
 47%|████▋     | 106351/225000 [226:36:59<268:05:25,  8.13s/it]
 47%|████▋     | 106352/225000 [226:37:05<248:58:57,  7.55s/it]
 47%|████▋     | 106353/225000 [226:37:13<251:44:18,  7.64s/it]
 47%|████▋     | 106354/225000 [226:37:15<197:33:31,  5.99s/it]
 47%|████▋     | 106355/225000 [226:37:21<203:06:40,  6.16s/it]
 47%|████▋     | 106356/225000 [226:37:30<224:17:23,  6.81s/it]
 47%|████▋     | 106357/225000 [226:37:40<262:52:29,  7.98s/it]
 47%|████▋     | 106358/225000 [226:37:46<238:31:37,  7.24s/it]
 47%|████▋     | 106359/225000 [226:38:00<308:54:23,  9.37s/it]
 47%|████▋     | 106360/225000 [226:38:08<297:22:02,  9.02s/it]
 47%|████▋     | 106361/225000 [226:38:21<333:53:25, 10.13s/it]
 47%|████▋     | 106362/225000 [226:38:28<298:48:25,  9.07s/it]
 47%|████▋     | 106363/225000 [226:38:32<256:28:39,  7.78s/it]
 47%|████▋     | 106364/225000 [226:38:37<226:55:43,  6.89s/it]
 47%|████▋     | 106365/225000 [226:38:41<196:38:24,  5.97s/it]
 47%|████▋     | 106366/225000 [226:38:47<194:49:54,  5.91s/it]
 47%|████▋     | 106367/225000 [226:38:50<172:00:11,  5.22s/it]
 47%|████▋     | 106368/225000 [226:38:55<163:56:29,  4.97s/it]
 47%|████▋     | 106369/225000 [226:39:02<186:32:00,  5.66s/it]
 47%|████▋     | 106370/225000 [226:39:11<222:25:53,  6.75s/it]
 47%|████▋     | 106371/225000 [226:39:20<242:32:54,  7.36s/it]
 47%|████▋     | 106372/225000 [226:39:26<222:26:13,  6.75s/it]
 47%|████▋     | 106373/225000 [226:39:29<187:33:08,  5.69s/it]
 47%|████▋     | 106374/225000 [226:39:33<170:09:35,  5.16s/it]
 47%|████▋     | 106375/225000 [226:39:37<161:46:18,  4.91s/it]
 47%|████▋     | 106376/225000 [226:39:40<145:05:27,  4.40s/it]
 47%|████▋     | 106377/225000 [226:39:46<160:37:38,  4.87s/it]
 47%|████▋     | 106378/225000 [226:39:49<143:38:28,  4.36s/it]
 47%|████▋     | 106379/225000 [226:39:56<167:26:50,  5.08s/it]
 47%|████▋     | 106380/225000 [226:40:14<294:22:03,  8.93s/it]
 47%|████▋     | 106381/225000 [226:40:18<239:54:44,  7.28s/it]
 47%|████▋     | 106382/225000 [226:40:20<190:52:15,  5.79s/it]
 47%|████▋     | 106383/225000 [226:40:33<266:06:01,  8.08s/it]
 47%|████▋     | 106384/225000 [226:40:37<219:38:29,  6.67s/it]
 47%|████▋     | 106385/225000 [226:40:46<250:28:17,  7.60s/it]
 47%|████▋     | 106386/225000 [226:40:57<281:55:00,  8.56s/it]
 47%|████▋     | 106387/225000 [226:41:01<231:55:41,  7.04s/it]
 47%|████▋     | 106388/225000 [226:41:05<209:02:15,  6.34s/it]
 47%|████▋     | 106389/225000 [226:41:13<218:24:54,  6.63s/it]
 47%|████▋     | 106390/225000 [226:41:27<290:53:37,  8.83s/it]
 47%|████▋     | 106391/225000 [226:41:30<238:03:50,  7.23s/it]
 47%|████▋     | 106392/225000 [226:41:39<250:05:51,  7.59s/it]
 47%|████▋     | 106393/225000 [226:41:47<258:49:56,  7.86s/it]
 47%|████▋     | 106394/225000 [226:41:51<220:47:32,  6.70s/it]
 47%|████▋     | 106395/225000 [226:42:01<251:09:01,  7.62s/it]
 47%|████▋     | 106396/225000 [226:42:04<207:55:28,  6.31s/it]
 47%|████▋     | 106397/225000 [226:42:07<170:49:28,  5.19s/it]
 47%|████▋     | 106398/225000 [226:42:20<254:13:16,  7.72s/it]
 47%|████▋     | 106399/225000 [226:42:26<236:31:54,  7.18s/it]
 47%|████▋     | 106400/225000 [226:42:36<258:45:50,  7.85s/it]
                                                               
{'loss': '0.98', 'grad_norm': '1.85', 'learning_rate': '2.906e-05', 'epoch': '0.4729', 'num_input_tokens_seen': 380165312, 'train_runtime': '8.162e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106400/225000 [226:42:36<258:45:50,  7.85s/it]
 47%|████▋     | 106401/225000 [226:42:39<212:31:48,  6.45s/it]
 47%|████▋     | 106402/225000 [226:42:56<322:29:05,  9.79s/it]
 47%|████▋     | 106403/225000 [226:43:00<260:00:28,  7.89s/it]
 47%|████▋     | 106404/225000 [226:43:09<276:55:41,  8.41s/it]
 47%|████▋     | 106405/225000 [226:43:14<243:48:25,  7.40s/it]
 47%|████▋     | 106406/225000 [226:43:18<207:26:25,  6.30s/it]
 47%|████▋     | 106407/225000 [226:43:25<214:56:36,  6.52s/it]
 47%|████▋     | 106408/225000 [226:43:31<210:00:03,  6.37s/it]
 47%|████▋     | 106409/225000 [226:43:34<178:02:27,  5.40s/it]
 47%|████▋     | 106410/225000 [226:43:43<204:42:03,  6.21s/it]
 47%|████▋     | 106411/225000 [226:43:55<269:46:02,  8.19s/it]
 47%|████▋     | 106412/225000 [226:43:58<218:33:51,  6.63s/it]
 47%|████▋     | 106413/225000 [226:44:05<215:00:26,  6.53s/it]
 47%|████▋     | 106414/225000 [226:44:10<203:28:38,  6.18s/it]
 47%|████▋     | 106415/225000 [226:44:26<300:45:30,  9.13s/it]
 47%|████▋     | 106416/225000 [226:44:32<273:28:11,  8.30s/it]
 47%|████▋     | 106417/225000 [226:44:45<317:12:53,  9.63s/it]
 47%|████▋     | 106418/225000 [226:44:53<302:11:10,  9.17s/it]
 47%|████▋     | 106419/225000 [226:44:59<269:10:09,  8.17s/it]
 47%|████▋     | 106420/225000 [226:45:05<245:36:31,  7.46s/it]
 47%|████▋     | 106421/225000 [226:45:11<236:08:25,  7.17s/it]
 47%|████▋     | 106422/225000 [226:45:15<196:35:05,  5.97s/it]
 47%|████▋     | 106423/225000 [226:45:18<169:10:02,  5.14s/it]
 47%|████▋     | 106424/225000 [226:45:25<186:50:52,  5.67s/it]
 47%|████▋     | 106425/225000 [226:45:29<176:46:16,  5.37s/it]
 47%|████▋     | 106426/225000 [226:45:36<185:18:22,  5.63s/it]
 47%|████▋     | 106427/225000 [226:45:42<191:28:35,  5.81s/it]
 47%|████▋     | 106428/225000 [226:45:47<181:51:42,  5.52s/it]
 47%|████▋     | 106429/225000 [226:46:00<257:34:20,  7.82s/it]
 47%|████▋     | 106430/225000 [226:46:10<281:13:24,  8.54s/it]
 47%|████▋     | 106431/225000 [226:46:13<228:07:06,  6.93s/it]
 47%|████▋     | 106432/225000 [226:46:22<245:10:37,  7.44s/it]
 47%|████▋     | 106433/225000 [226:46:30<252:44:23,  7.67s/it]
 47%|████▋     | 106434/225000 [226:46:33<207:40:33,  6.31s/it]
 47%|████▋     | 106435/225000 [226:46:49<304:39:52,  9.25s/it]
 47%|████▋     | 106436/225000 [226:46:54<257:32:53,  7.82s/it]
 47%|████▋     | 106437/225000 [226:46:56<205:39:47,  6.24s/it]
 47%|████▋     | 106438/225000 [226:46:59<170:48:15,  5.19s/it]
 47%|████▋     | 106439/225000 [226:47:07<203:04:24,  6.17s/it]
 47%|████▋     | 106440/225000 [226:47:16<229:53:21,  6.98s/it]
 47%|████▋     | 106441/225000 [226:47:19<190:17:21,  5.78s/it]
 47%|████▋     | 106442/225000 [226:47:25<184:51:02,  5.61s/it]
 47%|████▋     | 106443/225000 [226:47:32<204:38:38,  6.21s/it]
 47%|████▋     | 106444/225000 [226:47:36<181:40:04,  5.52s/it]
 47%|████▋     | 106445/225000 [226:47:38<150:45:45,  4.58s/it]
 47%|████▋     | 106446/225000 [226:47:55<264:19:53,  8.03s/it]
 47%|████▋     | 106447/225000 [226:48:01<246:30:42,  7.49s/it]
 47%|████▋     | 106448/225000 [226:48:16<322:36:38,  9.80s/it]
 47%|████▋     | 106449/225000 [226:48:18<250:01:43,  7.59s/it]
 47%|████▋     | 106450/225000 [226:48:30<289:14:33,  8.78s/it]
                                                               
{'loss': '0.9593', 'grad_norm': '1.628', 'learning_rate': '2.904e-05', 'epoch': '0.4731', 'num_input_tokens_seen': 380321192, 'train_runtime': '8.165e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106450/225000 [226:48:30<289:14:33,  8.78s/it]
 47%|████▋     | 106451/225000 [226:48:44<336:30:19, 10.22s/it]
 47%|████▋     | 106452/225000 [226:48:51<311:00:43,  9.44s/it]
 47%|████▋     | 106453/225000 [226:48:53<239:05:58,  7.26s/it]
 47%|████▋     | 106454/225000 [226:48:58<218:12:21,  6.63s/it]
 47%|████▋     | 106455/225000 [226:49:04<211:25:35,  6.42s/it]
 47%|████▋     | 106456/225000 [226:49:09<194:20:04,  5.90s/it]
 47%|████▋     | 106457/225000 [226:49:11<159:43:35,  4.85s/it]
 47%|████▋     | 106458/225000 [226:49:22<215:20:20,  6.54s/it]
 47%|████▋     | 106459/225000 [226:49:30<231:46:12,  7.04s/it]
 47%|████▋     | 106460/225000 [226:49:40<257:08:58,  7.81s/it]
 47%|████▋     | 106461/225000 [226:49:50<282:01:14,  8.56s/it]
 47%|████▋     | 106462/225000 [226:50:04<333:04:50, 10.12s/it]
 47%|████▋     | 106463/225000 [226:50:08<277:07:21,  8.42s/it]
 47%|████▋     | 106464/225000 [226:50:19<299:23:28,  9.09s/it]
 47%|████▋     | 106465/225000 [226:50:21<232:11:54,  7.05s/it]
 47%|████▋     | 106466/225000 [226:50:31<256:29:21,  7.79s/it]
 47%|████▋     | 106467/225000 [226:50:41<276:43:02,  8.40s/it]
 47%|████▋     | 106468/225000 [226:50:53<314:23:38,  9.55s/it]
 47%|████▋     | 106469/225000 [226:50:57<263:05:36,  7.99s/it]
 47%|████▋     | 106470/225000 [226:51:03<239:42:57,  7.28s/it]
 47%|████▋     | 106471/225000 [226:51:07<210:52:28,  6.40s/it]
 47%|████▋     | 106472/225000 [226:51:10<171:21:11,  5.20s/it]
 47%|████▋     | 106473/225000 [226:51:20<220:15:15,  6.69s/it]
 47%|████▋     | 106474/225000 [226:51:24<200:26:55,  6.09s/it]
 47%|████▋     | 106475/225000 [226:51:27<164:41:15,  5.00s/it]
 47%|████▋     | 106476/225000 [226:51:35<200:09:49,  6.08s/it]
 47%|████▋     | 106477/225000 [226:51:42<200:18:33,  6.08s/it]
 47%|████▋     | 106478/225000 [226:51:47<198:13:53,  6.02s/it]
 47%|████▋     | 106479/225000 [226:51:57<234:57:40,  7.14s/it]
 47%|████▋     | 106480/225000 [226:52:03<224:37:20,  6.82s/it]
 47%|████▋     | 106481/225000 [226:52:09<211:19:41,  6.42s/it]
 47%|████▋     | 106482/225000 [226:52:15<208:08:25,  6.32s/it]
 47%|████▋     | 106483/225000 [226:52:21<207:04:45,  6.29s/it]
 47%|████▋     | 106484/225000 [226:52:23<168:21:44,  5.11s/it]
 47%|████▋     | 106485/225000 [226:52:27<149:30:12,  4.54s/it]
 47%|████▋     | 106486/225000 [226:52:29<128:06:25,  3.89s/it]
 47%|████▋     | 106487/225000 [226:52:40<199:58:42,  6.07s/it]
 47%|████▋     | 106488/225000 [226:52:43<171:32:51,  5.21s/it]
 47%|████▋     | 106489/225000 [226:52:52<208:31:44,  6.33s/it]
 47%|████▋     | 106490/225000 [226:52:55<169:22:41,  5.15s/it]
 47%|████▋     | 106491/225000 [226:53:01<181:54:58,  5.53s/it]
 47%|████▋     | 106492/225000 [226:53:10<216:41:54,  6.58s/it]
 47%|████▋     | 106493/225000 [226:53:17<215:35:27,  6.55s/it]
 47%|████▋     | 106494/225000 [226:53:21<193:11:42,  5.87s/it]
 47%|████▋     | 106495/225000 [226:53:29<213:07:34,  6.47s/it]
 47%|████▋     | 106496/225000 [226:53:36<218:16:52,  6.63s/it]
 47%|████▋     | 106497/225000 [226:53:46<257:28:13,  7.82s/it]
 47%|████▋     | 106498/225000 [226:53:49<202:12:34,  6.14s/it]
 47%|████▋     | 106499/225000 [226:53:54<196:59:48,  5.98s/it]
 47%|████▋     | 106500/225000 [226:54:04<233:14:03,  7.09s/it]
                                                               
{'loss': '0.9778', 'grad_norm': '2.778', 'learning_rate': '2.902e-05', 'epoch': '0.4733', 'num_input_tokens_seen': 380476272, 'train_runtime': '8.168e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106500/225000 [226:54:04<233:14:03,  7.09s/it]
 47%|████▋     | 106501/225000 [226:54:10<222:32:36,  6.76s/it]
 47%|████▋     | 106502/225000 [226:54:13<186:21:06,  5.66s/it]
 47%|████▋     | 106503/225000 [226:54:17<174:18:34,  5.30s/it]
 47%|████▋     | 106504/225000 [226:54:34<286:55:59,  8.72s/it]
 47%|████▋     | 106505/225000 [226:54:41<265:50:19,  8.08s/it]
 47%|████▋     | 106506/225000 [226:54:49<267:05:40,  8.11s/it]
 47%|████▋     | 106507/225000 [226:54:54<236:52:46,  7.20s/it]
 47%|████▋     | 106508/225000 [226:54:57<196:42:04,  5.98s/it]
 47%|████▋     | 106509/225000 [226:55:05<215:57:15,  6.56s/it]
 47%|████▋     | 106510/225000 [226:55:09<188:15:38,  5.72s/it]
 47%|████▋     | 106511/225000 [226:55:11<153:37:34,  4.67s/it]
 47%|████▋     | 106512/225000 [226:55:16<153:16:23,  4.66s/it]
 47%|████▋     | 106513/225000 [226:55:26<211:58:46,  6.44s/it]
 47%|████▋     | 106514/225000 [226:55:37<258:12:06,  7.85s/it]
 47%|████▋     | 106515/225000 [226:55:47<274:48:25,  8.35s/it]
 47%|████▋     | 106516/225000 [226:55:54<258:04:05,  7.84s/it]
 47%|████▋     | 106517/225000 [226:55:58<221:08:13,  6.72s/it]
 47%|████▋     | 106518/225000 [226:56:15<325:50:04,  9.90s/it]
 47%|████▋     | 106519/225000 [226:56:22<301:35:43,  9.16s/it]
 47%|████▋     | 106520/225000 [226:56:25<232:29:37,  7.06s/it]
 47%|████▋     | 106521/225000 [226:56:37<290:10:54,  8.82s/it]
 47%|████▋     | 106522/225000 [226:56:43<258:35:19,  7.86s/it]
 47%|████▋     | 106523/225000 [226:56:48<229:26:35,  6.97s/it]
 47%|████▋     | 106524/225000 [226:57:03<310:32:53,  9.44s/it]
 47%|████▋     | 106525/225000 [226:57:10<280:13:28,  8.51s/it]
 47%|████▋     | 106526/225000 [226:57:13<225:17:37,  6.85s/it]
 47%|████▋     | 106527/225000 [226:57:19<221:45:31,  6.74s/it]
 47%|████▋     | 106528/225000 [226:57:21<178:04:50,  5.41s/it]
 47%|████▋     | 106529/225000 [226:57:27<185:38:38,  5.64s/it]
 47%|████▋     | 106530/225000 [226:57:30<154:05:02,  4.68s/it]
 47%|████▋     | 106531/225000 [226:57:32<132:08:45,  4.02s/it]
 47%|████▋     | 106532/225000 [226:57:35<116:08:52,  3.53s/it]
 47%|████▋     | 106533/225000 [226:57:37<107:08:13,  3.26s/it]
 47%|████▋     | 106534/225000 [226:57:51<204:14:59,  6.21s/it]
 47%|████▋     | 106535/225000 [226:57:56<199:45:17,  6.07s/it]
 47%|████▋     | 106536/225000 [226:57:59<162:25:11,  4.94s/it]
 47%|████▋     | 106537/225000 [226:58:04<164:36:10,  5.00s/it]
 47%|████▋     | 106538/225000 [226:58:14<221:13:40,  6.72s/it]
 47%|████▋     | 106539/225000 [226:58:25<259:16:58,  7.88s/it]
 47%|████▋     | 106540/225000 [226:58:33<261:26:38,  7.95s/it]
 47%|████▋     | 106541/225000 [226:58:36<215:56:11,  6.56s/it]
 47%|████▋     | 106542/225000 [226:58:46<245:25:39,  7.46s/it]
 47%|████▋     | 106543/225000 [226:58:52<232:20:50,  7.06s/it]
 47%|████▋     | 106544/225000 [226:59:02<261:09:14,  7.94s/it]
 47%|████▋     | 106545/225000 [226:59:08<239:26:35,  7.28s/it]
 47%|████▋     | 106546/225000 [226:59:12<204:39:05,  6.22s/it]
 47%|████▋     | 106547/225000 [226:59:17<194:36:15,  5.91s/it]
 47%|████▋     | 106548/225000 [226:59:22<189:45:18,  5.77s/it]
 47%|████▋     | 106549/225000 [226:59:29<197:22:20,  6.00s/it]
 47%|████▋     | 106550/225000 [226:59:36<210:00:26,  6.38s/it]
                                                               
{'loss': '0.9956', 'grad_norm': '1.705', 'learning_rate': '2.9e-05', 'epoch': '0.4736', 'num_input_tokens_seen': 380628680, 'train_runtime': '8.172e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106550/225000 [226:59:36<210:00:26,  6.38s/it]
 47%|████▋     | 106551/225000 [226:59:54<327:55:07,  9.97s/it]
 47%|████▋     | 106552/225000 [227:00:04<321:20:41,  9.77s/it]
 47%|████▋     | 106553/225000 [227:00:13<314:01:18,  9.54s/it]
 47%|████▋     | 106554/225000 [227:00:16<251:32:44,  7.65s/it]
 47%|████▋     | 106555/225000 [227:00:25<261:45:25,  7.96s/it]
 47%|████▋     | 106556/225000 [227:00:33<264:06:56,  8.03s/it]
 47%|████▋     | 106557/225000 [227:00:37<221:40:40,  6.74s/it]
 47%|████▋     | 106558/225000 [227:00:42<208:08:47,  6.33s/it]
 47%|████▋     | 106559/225000 [227:00:52<244:50:32,  7.44s/it]
 47%|████▋     | 106560/225000 [227:00:59<237:37:46,  7.22s/it]
 47%|████▋     | 106561/225000 [227:01:07<246:27:43,  7.49s/it]
 47%|████▋     | 106562/225000 [227:01:09<196:20:37,  5.97s/it]
 47%|████▋     | 106563/225000 [227:01:14<180:10:27,  5.48s/it]
 47%|████▋     | 106564/225000 [227:01:19<180:50:43,  5.50s/it]
 47%|████▋     | 106565/225000 [227:01:23<164:34:45,  5.00s/it]
 47%|████▋     | 106566/225000 [227:01:28<161:36:38,  4.91s/it]
 47%|████▋     | 106567/225000 [227:01:40<239:54:21,  7.29s/it]
 47%|████▋     | 106568/225000 [227:01:46<226:27:40,  6.88s/it]
 47%|████▋     | 106569/225000 [227:01:56<249:41:58,  7.59s/it]
 47%|████▋     | 106570/225000 [227:01:59<205:22:45,  6.24s/it]
 47%|████▋     | 106571/225000 [227:02:03<181:26:05,  5.52s/it]
 47%|████▋     | 106572/225000 [227:02:11<209:28:36,  6.37s/it]
 47%|████▋     | 106573/225000 [227:02:23<264:07:59,  8.03s/it]
 47%|████▋     | 106574/225000 [227:02:33<289:53:15,  8.81s/it]
 47%|████▋     | 106575/225000 [227:02:37<233:28:19,  7.10s/it]
 47%|████▋     | 106576/225000 [227:02:45<248:43:35,  7.56s/it]
 47%|████▋     | 106577/225000 [227:02:54<264:48:52,  8.05s/it]
 47%|████▋     | 106578/225000 [227:03:08<315:03:58,  9.58s/it]
 47%|████▋     | 106579/225000 [227:03:19<332:41:59, 10.11s/it]
 47%|████▋     | 106580/225000 [227:03:25<290:46:35,  8.84s/it]
 47%|████▋     | 106581/225000 [227:03:27<228:35:01,  6.95s/it]
 47%|████▋     | 106582/225000 [227:03:37<255:02:04,  7.75s/it]
 47%|████▋     | 106583/225000 [227:03:40<207:35:36,  6.31s/it]
 47%|████▋     | 106584/225000 [227:03:53<271:03:57,  8.24s/it]
 47%|████▋     | 106585/225000 [227:03:59<251:06:59,  7.63s/it]
 47%|████▋     | 106586/225000 [227:04:02<206:59:52,  6.29s/it]
 47%|████▋     | 106587/225000 [227:04:12<244:18:39,  7.43s/it]
 47%|████▋     | 106588/225000 [227:04:15<195:56:03,  5.96s/it]
 47%|████▋     | 106589/225000 [227:04:26<253:04:57,  7.69s/it]
 47%|████▋     | 106590/225000 [227:04:32<228:23:51,  6.94s/it]
 47%|████▋     | 106591/225000 [227:04:40<247:08:13,  7.51s/it]
 47%|████▋     | 106592/225000 [227:04:50<267:03:17,  8.12s/it]
 47%|████▋     | 106593/225000 [227:04:54<224:22:03,  6.82s/it]
 47%|████▋     | 106594/225000 [227:04:56<178:43:54,  5.43s/it]
 47%|████▋     | 106595/225000 [227:05:01<172:42:39,  5.25s/it]
 47%|████▋     | 106596/225000 [227:05:11<226:07:47,  6.88s/it]
 47%|████▋     | 106597/225000 [227:05:20<243:52:47,  7.42s/it]
 47%|████▋     | 106598/225000 [227:05:33<295:09:50,  8.97s/it]
 47%|████▋     | 106599/225000 [227:05:40<281:38:38,  8.56s/it]
 47%|████▋     | 106600/225000 [227:05:45<240:26:17,  7.31s/it]
                                                               
{'loss': '0.982', 'grad_norm': '2.436', 'learning_rate': '2.898e-05', 'epoch': '0.4738', 'num_input_tokens_seen': 380785960, 'train_runtime': '8.175e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106600/225000 [227:05:45<240:26:17,  7.31s/it]
 47%|████▋     | 106601/225000 [227:05:50<217:19:08,  6.61s/it]
 47%|████▋     | 106602/225000 [227:05:53<185:59:43,  5.66s/it]
 47%|████▋     | 106603/225000 [227:05:56<162:01:53,  4.93s/it]
 47%|████▋     | 106604/225000 [227:06:05<204:09:18,  6.21s/it]
 47%|████▋     | 106605/225000 [227:06:13<220:59:31,  6.72s/it]
 47%|████▋     | 106606/225000 [227:06:17<192:23:08,  5.85s/it]
 47%|████▋     | 106607/225000 [227:06:20<164:36:14,  5.01s/it]
 47%|████▋     | 106608/225000 [227:06:27<186:32:53,  5.67s/it]
 47%|████▋     | 106609/225000 [227:06:33<183:48:15,  5.59s/it]
 47%|████▋     | 106610/225000 [227:06:40<202:06:20,  6.15s/it]
 47%|████▋     | 106611/225000 [227:06:47<207:33:20,  6.31s/it]
 47%|████▋     | 106612/225000 [227:06:50<176:07:41,  5.36s/it]
 47%|████▋     | 106613/225000 [227:06:54<162:05:52,  4.93s/it]
 47%|████▋     | 106614/225000 [227:06:58<147:23:19,  4.48s/it]
 47%|████▋     | 106615/225000 [227:07:03<161:46:43,  4.92s/it]
 47%|████▋     | 106616/225000 [227:07:09<164:14:19,  4.99s/it]
 47%|████▋     | 106617/225000 [227:07:27<295:27:15,  8.98s/it]
 47%|████▋     | 106618/225000 [227:07:30<240:37:46,  7.32s/it]
 47%|████▋     | 106619/225000 [227:07:39<253:47:29,  7.72s/it]
 47%|████▋     | 106620/225000 [227:07:47<260:16:45,  7.92s/it]
 47%|████▋     | 106621/225000 [227:07:59<295:44:56,  8.99s/it]
 47%|████▋     | 106622/225000 [227:08:11<329:44:47, 10.03s/it]
 47%|████▋     | 106623/225000 [227:08:22<333:42:50, 10.15s/it]
 47%|████▋     | 106624/225000 [227:08:25<269:48:24,  8.21s/it]
 47%|████▋     | 106625/225000 [227:08:32<257:06:10,  7.82s/it]
 47%|████▋     | 106626/225000 [227:08:37<228:04:47,  6.94s/it]
 47%|████▋     | 106627/225000 [227:08:48<270:38:08,  8.23s/it]
 47%|████▋     | 106628/225000 [227:08:56<262:23:54,  7.98s/it]
 47%|████▋     | 106629/225000 [227:09:00<220:45:02,  6.71s/it]
 47%|████▋     | 106630/225000 [227:09:06<214:14:22,  6.52s/it]
 47%|████▋     | 106631/225000 [227:09:10<189:36:12,  5.77s/it]
 47%|████▋     | 106632/225000 [227:09:13<164:10:59,  4.99s/it]
 47%|████▋     | 106633/225000 [227:09:20<182:38:22,  5.55s/it]
 47%|████▋     | 106634/225000 [227:09:24<171:48:30,  5.23s/it]
 47%|████▋     | 106635/225000 [227:09:31<184:00:00,  5.60s/it]
 47%|████▋     | 106636/225000 [227:09:37<195:09:49,  5.94s/it]
 47%|████▋     | 106637/225000 [227:09:50<260:31:20,  7.92s/it]
 47%|████▋     | 106638/225000 [227:09:53<216:12:53,  6.58s/it]
 47%|████▋     | 106639/225000 [227:10:06<272:55:50,  8.30s/it]
 47%|████▋     | 106640/225000 [227:10:10<238:05:24,  7.24s/it]
 47%|████▋     | 106641/225000 [227:10:16<222:06:12,  6.76s/it]
 47%|████▋     | 106642/225000 [227:10:27<265:29:13,  8.08s/it]
 47%|████▋     | 106643/225000 [227:10:38<287:35:59,  8.75s/it]
 47%|████▋     | 106644/225000 [227:10:48<300:14:20,  9.13s/it]
 47%|████▋     | 106645/225000 [227:10:59<321:07:28,  9.77s/it]
 47%|████▋     | 106646/225000 [227:11:16<393:06:07, 11.96s/it]
 47%|████▋     | 106647/225000 [227:11:29<406:44:40, 12.37s/it]
 47%|████▋     | 106648/225000 [227:11:32<314:35:22,  9.57s/it]
 47%|████▋     | 106649/225000 [227:11:40<295:00:24,  8.97s/it]
 47%|████▋     | 106650/225000 [227:11:45<256:34:53,  7.80s/it]
                                                               
{'loss': '0.9165', 'grad_norm': '3.386', 'learning_rate': '2.897e-05', 'epoch': '0.474', 'num_input_tokens_seen': 380953528, 'train_runtime': '8.179e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106650/225000 [227:11:45<256:34:53,  7.80s/it]
 47%|████▋     | 106651/225000 [227:11:49<217:13:56,  6.61s/it]
 47%|████▋     | 106652/225000 [227:11:53<192:52:55,  5.87s/it]
 47%|████▋     | 106653/225000 [227:11:57<172:51:35,  5.26s/it]
 47%|████▋     | 106654/225000 [227:12:04<195:56:46,  5.96s/it]
 47%|████▋     | 106655/225000 [227:12:18<275:06:19,  8.37s/it]
 47%|████▋     | 106656/225000 [227:12:21<221:17:43,  6.73s/it]
 47%|████▋     | 106657/225000 [227:12:30<238:28:01,  7.25s/it]
 47%|████▋     | 106658/225000 [227:12:32<188:02:23,  5.72s/it]
 47%|████▋     | 106659/225000 [227:12:34<155:46:00,  4.74s/it]
 47%|████▋     | 106660/225000 [227:12:48<241:31:12,  7.35s/it]
 47%|████▋     | 106661/225000 [227:12:53<219:13:11,  6.67s/it]
 47%|████▋     | 106662/225000 [227:12:57<191:40:52,  5.83s/it]
 47%|████▋     | 106663/225000 [227:13:06<221:27:46,  6.74s/it]
 47%|████▋     | 106664/225000 [227:13:11<204:06:25,  6.21s/it]
 47%|████▋     | 106665/225000 [227:13:14<180:28:08,  5.49s/it]
 47%|████▋     | 106666/225000 [227:13:17<156:33:08,  4.76s/it]
 47%|████▋     | 106667/225000 [227:13:26<195:55:02,  5.96s/it]
 47%|████▋     | 106668/225000 [227:13:30<175:00:46,  5.32s/it]
 47%|████▋     | 106669/225000 [227:13:41<230:44:31,  7.02s/it]
 47%|████▋     | 106670/225000 [227:13:50<249:26:05,  7.59s/it]
 47%|████▋     | 106671/225000 [227:13:52<195:58:52,  5.96s/it]
 47%|████▋     | 106672/225000 [227:13:56<178:57:13,  5.44s/it]
 47%|████▋     | 106673/225000 [227:14:01<171:52:29,  5.23s/it]
 47%|████▋     | 106674/225000 [227:14:07<175:11:10,  5.33s/it]
 47%|████▋     | 106675/225000 [227:14:16<214:30:05,  6.53s/it]
 47%|████▋     | 106676/225000 [227:14:24<229:50:11,  6.99s/it]
 47%|████▋     | 106677/225000 [227:14:33<251:23:19,  7.65s/it]
 47%|████▋     | 106678/225000 [227:14:41<251:10:32,  7.64s/it]
 47%|████▋     | 106679/225000 [227:14:48<246:27:11,  7.50s/it]
 47%|████▋     | 106680/225000 [227:14:53<218:36:04,  6.65s/it]
 47%|████▋     | 106681/225000 [227:14:56<183:17:42,  5.58s/it]
 47%|████▋     | 106682/225000 [227:15:04<210:05:23,  6.39s/it]
 47%|████▋     | 106683/225000 [227:15:15<254:18:36,  7.74s/it]
 47%|████▋     | 106684/225000 [227:15:24<269:29:20,  8.20s/it]
 47%|████▋     | 106685/225000 [227:15:31<258:30:29,  7.87s/it]
 47%|████▋     | 106686/225000 [227:15:49<353:38:11, 10.76s/it]
 47%|████▋     | 106687/225000 [227:15:56<323:50:30,  9.85s/it]
 47%|████▋     | 106688/225000 [227:15:59<253:07:00,  7.70s/it]
 47%|████▋     | 106689/225000 [227:16:03<214:24:24,  6.52s/it]
 47%|████▋     | 106690/225000 [227:16:08<197:38:06,  6.01s/it]
 47%|████▋     | 106691/225000 [227:16:22<278:21:02,  8.47s/it]
 47%|████▋     | 106692/225000 [227:16:27<247:21:58,  7.53s/it]
 47%|████▋     | 106693/225000 [227:16:40<295:32:59,  8.99s/it]
 47%|████▋     | 106694/225000 [227:16:45<259:14:38,  7.89s/it]
 47%|████▋     | 106695/225000 [227:16:51<236:12:53,  7.19s/it]
 47%|████▋     | 106696/225000 [227:16:53<187:03:31,  5.69s/it]
 47%|████▋     | 106697/225000 [227:17:00<197:38:16,  6.01s/it]
 47%|████▋     | 106698/225000 [227:17:06<199:42:23,  6.08s/it]
 47%|████▋     | 106699/225000 [227:17:09<169:16:16,  5.15s/it]
 47%|████▋     | 106700/225000 [227:17:12<147:38:17,  4.49s/it]
                                                               
{'loss': '0.9605', 'grad_norm': '3.368', 'learning_rate': '2.895e-05', 'epoch': '0.4742', 'num_input_tokens_seen': 381109176, 'train_runtime': '8.182e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106700/225000 [227:17:12<147:38:17,  4.49s/it]
 47%|████▋     | 106701/225000 [227:17:14<125:15:59,  3.81s/it]
 47%|████▋     | 106702/225000 [227:17:30<244:05:16,  7.43s/it]
 47%|████▋     | 106703/225000 [227:17:35<221:10:20,  6.73s/it]
 47%|████▋     | 106704/225000 [227:17:40<205:10:35,  6.24s/it]
 47%|████▋     | 106705/225000 [227:17:49<233:47:32,  7.11s/it]
 47%|████▋     | 106706/225000 [227:17:58<246:42:14,  7.51s/it]
 47%|████▋     | 106707/225000 [227:18:08<270:55:25,  8.24s/it]
 47%|████▋     | 106708/225000 [227:18:18<292:33:46,  8.90s/it]
 47%|████▋     | 106709/225000 [227:18:27<289:18:55,  8.80s/it]
 47%|████▋     | 106710/225000 [227:18:30<231:43:12,  7.05s/it]
 47%|████▋     | 106711/225000 [227:18:38<241:48:32,  7.36s/it]
 47%|████▋     | 106712/225000 [227:18:42<208:21:52,  6.34s/it]
 47%|████▋     | 106713/225000 [227:18:45<175:58:48,  5.36s/it]
 47%|████▋     | 106714/225000 [227:18:47<145:52:43,  4.44s/it]
 47%|████▋     | 106715/225000 [227:18:49<125:30:06,  3.82s/it]
 47%|████▋     | 106716/225000 [227:19:04<237:19:17,  7.22s/it]
 47%|████▋     | 106717/225000 [227:19:10<215:47:05,  6.57s/it]
 47%|████▋     | 106718/225000 [227:19:13<182:24:21,  5.55s/it]
 47%|████▋     | 106719/225000 [227:19:21<211:49:42,  6.45s/it]
 47%|████▋     | 106720/225000 [227:19:29<225:52:52,  6.87s/it]
 47%|████▋     | 106721/225000 [227:19:31<180:23:08,  5.49s/it]
 47%|████▋     | 106722/225000 [227:19:38<194:15:40,  5.91s/it]
 47%|████▋     | 106723/225000 [227:19:45<201:58:57,  6.15s/it]
 47%|████▋     | 106724/225000 [227:19:55<237:16:03,  7.22s/it]
 47%|████▋     | 106725/225000 [227:20:10<317:53:09,  9.68s/it]
 47%|████▋     | 106726/225000 [227:20:18<296:44:13,  9.03s/it]
 47%|████▋     | 106727/225000 [227:20:23<259:07:50,  7.89s/it]
 47%|████▋     | 106728/225000 [227:20:27<224:19:47,  6.83s/it]
 47%|████▋     | 106729/225000 [227:20:38<260:47:22,  7.94s/it]
 47%|████▋     | 106730/225000 [227:20:47<275:23:44,  8.38s/it]
 47%|████▋     | 106731/225000 [227:20:50<221:07:24,  6.73s/it]
 47%|████▋     | 106732/225000 [227:20:59<240:53:35,  7.33s/it]
 47%|████▋     | 106733/225000 [227:21:02<197:45:39,  6.02s/it]
 47%|████▋     | 106734/225000 [227:21:10<218:02:30,  6.64s/it]
 47%|████▋     | 106735/225000 [227:21:19<243:08:51,  7.40s/it]
 47%|████▋     | 106736/225000 [227:21:25<233:00:37,  7.09s/it]
 47%|████▋     | 106737/225000 [227:21:35<256:55:29,  7.82s/it]
 47%|████▋     | 106738/225000 [227:21:42<247:30:30,  7.53s/it]
 47%|████▋     | 106739/225000 [227:21:47<228:11:16,  6.95s/it]
 47%|████▋     | 106740/225000 [227:21:51<195:42:49,  5.96s/it]
 47%|████▋     | 106741/225000 [227:22:01<233:53:23,  7.12s/it]
 47%|████▋     | 106742/225000 [227:22:12<273:38:14,  8.33s/it]
 47%|████▋     | 106743/225000 [227:22:21<281:02:07,  8.56s/it]
 47%|████▋     | 106744/225000 [227:22:27<257:22:30,  7.84s/it]
 47%|████▋     | 106745/225000 [227:22:33<240:22:05,  7.32s/it]
 47%|████▋     | 106746/225000 [227:22:46<289:01:13,  8.80s/it]
 47%|████▋     | 106747/225000 [227:22:53<280:04:47,  8.53s/it]
 47%|████▋     | 106748/225000 [227:22:56<219:18:28,  6.68s/it]
 47%|████▋     | 106749/225000 [227:23:08<269:41:46,  8.21s/it]
 47%|████▋     | 106750/225000 [227:23:15<257:00:05,  7.82s/it]
                                                               
{'loss': '0.9559', 'grad_norm': '1.826', 'learning_rate': '2.893e-05', 'epoch': '0.4744', 'num_input_tokens_seen': 381272736, 'train_runtime': '8.186e+05', 'train_tokens_per_second': '465.8'}

 47%|████▋     | 106750/225000 [227:23:15<257:00:05,  7.82s/it]
 47%|████▋     | 106751/225000 [227:23:25<281:18:31,  8.56s/it]
 47%|████▋     | 106752/225000 [227:23:34<286:32:17,  8.72s/it]
 47%|████▋     | 106753/225000 [227:23:42<281:29:39,  8.57s/it]
 47%|████▋     | 106754/225000 [227:23:45<228:03:25,  6.94s/it]
 47%|████▋     | 106755/225000 [227:23:57<270:40:06,  8.24s/it]
 47%|████▋     | 106756/225000 [227:23:59<211:29:59,  6.44s/it]
 47%|████▋     | 106757/225000 [227:24:09<251:00:59,  7.64s/it]
 47%|████▋     | 106758/225000 [227:24:13<215:23:22,  6.56s/it]
 47%|████▋     | 106759/225000 [227:24:21<232:06:52,  7.07s/it]
 47%|████▋     | 106760/225000 [227:24:31<258:07:10,  7.86s/it]
 47%|████▋     | 106761/225000 [227:24:47<334:13:53, 10.18s/it]
 47%|████▋     | 106762/225000 [227:24:58<341:40:56, 10.40s/it]
 47%|████▋     | 106763/225000 [227:25:05<315:24:24,  9.60s/it]
 47%|████▋     | 106764/225000 [227:25:11<271:09:35,  8.26s/it]
 47%|████▋     | 106765/225000 [227:25:14<219:52:41,  6.69s/it]
 47%|████▋     | 106766/225000 [227:25:25<265:20:04,  8.08s/it]
 47%|████▋     | 106767/225000 [227:25:28<215:15:35,  6.55s/it]
 47%|████▋     | 106768/225000 [227:25:35<218:21:35,  6.65s/it]
 47%|████▋     | 106769/225000 [227:25:37<175:17:29,  5.34s/it]
 47%|████▋     | 106770/225000 [227:25:40<152:35:31,  4.65s/it]
 47%|████▋     | 106771/225000 [227:25:53<230:15:33,  7.01s/it]
 47%|████▋     | 106772/225000 [227:25:59<220:48:16,  6.72s/it]
 47%|████▋     | 106773/225000 [227:26:09<257:07:56,  7.83s/it]
 47%|████▋     | 106774/225000 [227:26:16<251:00:35,  7.64s/it]
 47%|████▋     | 106775/225000 [227:26:30<305:58:17,  9.32s/it]
 47%|████▋     | 106776/225000 [227:26:33<243:39:07,  7.42s/it]
 47%|████▋     | 106777/225000 [227:26:43<273:23:34,  8.33s/it]
 47%|████▋     | 106778/225000 [227:26:48<236:22:48,  7.20s/it]
 47%|████▋     | 106779/225000 [227:26:51<195:32:51,  5.95s/it]
 47%|████▋     | 106780/225000 [227:26:58<211:41:33,  6.45s/it]
 47%|████▋     | 106781/225000 [227:27:03<195:41:54,  5.96s/it]
 47%|████▋     | 106782/225000 [227:27:13<232:53:50,  7.09s/it]
 47%|████▋     | 106783/225000 [227:27:15<187:33:52,  5.71s/it]
 47%|████▋     | 106784/225000 [227:27:21<185:46:32,  5.66s/it]
 47%|████▋     | 106785/225000 [227:27:33<249:24:08,  7.60s/it]
 47%|████▋     | 106786/225000 [227:27:41<259:33:12,  7.90s/it]
 47%|████▋     | 106787/225000 [227:27:50<263:10:45,  8.01s/it]
 47%|████▋     | 106788/225000 [227:27:53<220:34:41,  6.72s/it]
 47%|████▋     | 106789/225000 [227:28:01<226:58:14,  6.91s/it]
 47%|████▋     | 106790/225000 [227:28:09<237:32:18,  7.23s/it]
 47%|████▋     | 106791/225000 [227:28:11<190:29:31,  5.80s/it]
 47%|████▋     | 106792/225000 [227:28:21<229:39:39,  6.99s/it]
 47%|████▋     | 106793/225000 [227:28:26<207:46:34,  6.33s/it]
 47%|████▋     | 106794/225000 [227:28:36<244:01:45,  7.43s/it]
 47%|████▋     | 106795/225000 [227:28:41<225:06:06,  6.86s/it]
 47%|████▋     | 106796/225000 [227:28:47<208:37:27,  6.35s/it]
 47%|████▋     | 106797/225000 [227:28:50<176:38:52,  5.38s/it]
 47%|████▋     | 106798/225000 [227:28:56<185:02:06,  5.64s/it]
 47%|████▋     | 106799/225000 [227:28:59<157:06:24,  4.78s/it]
 47%|████▋     | 106800/225000 [227:29:04<160:09:41,  4.88s/it]
                                                               
{'loss': '0.9491', 'grad_norm': '2.577', 'learning_rate': '2.891e-05', 'epoch': '0.4747', 'num_input_tokens_seen': 381422528, 'train_runtime': '8.189e+05', 'train_tokens_per_second': '465.7'}

 47%|████▋     | 106800/225000 [227:29:04<160:09:41,  4.88s/it]
 47%|████▋     | 106801/225000 [227:29:08<151:14:52,  4.61s/it]
 47%|████▋     | 106802/225000 [227:29:16<190:39:21,  5.81s/it]
 47%|████▋     | 106803/225000 [227:29:20<167:25:17,  5.10s/it]
 47%|████▋     | 106804/225000 [227:29:22<143:51:36,  4.38s/it]
 47%|████▋     | 106805/225000 [227:29:33<205:15:36,  6.25s/it]
 47%|████▋     | 106806/225000 [227:29:39<202:48:12,  6.18s/it]
 47%|████▋     | 106807/225000 [227:29:44<193:30:35,  5.89s/it]
 47%|████▋     | 106808/225000 [227:29:51<201:56:18,  6.15s/it]
 47%|████▋     | 106809/225000 [227:30:00<229:06:21,  6.98s/it]
 47%|████▋     | 106810/225000 [227:30:03<191:05:08,  5.82s/it]
 47%|████▋     | 106811/225000 [227:30:13<231:53:50,  7.06s/it]
 47%|████▋     | 106812/225000 [227:30:24<265:45:54,  8.10s/it]
 47%|████▋     | 106813/225000 [227:30:37<317:55:15,  9.68s/it]
 47%|████▋     | 106814/225000 [227:30:40<253:12:48,  7.71s/it]
 47%|████▋     | 106815/225000 [227:30:46<236:09:49,  7.19s/it]
 47%|████▋     | 106816/225000 [227:30:50<203:31:07,  6.20s/it]
 47%|████▋     | 106817/225000 [227:30:53<172:22:52,  5.25s/it]
 47%|████▋     | 106818/225000 [227:30:55<143:50:54,  4.38s/it]
 47%|████▋     | 106819/225000 [227:31:00<151:25:17,  4.61s/it]
 47%|████▋     | 106820/225000 [227:31:06<164:29:17,  5.01s/it]
 47%|████▋     | 106821/225000 [227:31:16<212:39:06,  6.48s/it]
 47%|████▋     | 106822/225000 [227:31:24<220:26:08,  6.72s/it]
 47%|████▋     | 106823/225000 [227:31:29<203:10:59,  6.19s/it]
 47%|████▋     | 106824/225000 [227:31:32<179:45:35,  5.48s/it]
 47%|████▋     | 106825/225000 [227:31:35<156:05:35,  4.76s/it]
 47%|████▋     | 106826/225000 [227:31:39<147:54:00,  4.51s/it]
 47%|████▋     | 106827/225000 [227:31:50<204:50:34,  6.24s/it]
 47%|████▋     | 106828/225000 [227:31:59<230:55:33,  7.03s/it]
 47%|████▋     | 106829/225000 [227:32:06<232:39:49,  7.09s/it]
 47%|████▋     | 106830/225000 [227:32:11<210:00:35,  6.40s/it]
 47%|████▋     | 106831/225000 [227:32:15<188:08:13,  5.73s/it]
 47%|████▋     | 106832/225000 [227:32:22<204:25:53,  6.23s/it]
 47%|████▋     | 106833/225000 [227:32:31<232:21:45,  7.08s/it]
 47%|████▋     | 106834/225000 [227:32:44<289:41:28,  8.83s/it]
 47%|████▋     | 106835/225000 [227:32:47<234:26:19,  7.14s/it]
 47%|████▋     | 106836/225000 [227:32:53<225:17:24,  6.86s/it]
 47%|████▋     | 106837/225000 [227:33:00<217:21:58,  6.62s/it]
 47%|████▋     | 106838/225000 [227:33:05<202:47:28,  6.18s/it]
 47%|████▋     | 106839/225000 [227:33:08<174:02:41,  5.30s/it]
 47%|████▋     | 106840/225000 [227:33:10<145:38:50,  4.44s/it]
 47%|████▋     | 106841/225000 [227:33:19<185:01:29,  5.64s/it]
 47%|████▋     | 106842/225000 [227:33:27<208:00:11,  6.34s/it]
 47%|████▋     | 106843/225000 [227:33:34<212:55:26,  6.49s/it]
 47%|████▋     | 106844/225000 [227:33:42<227:50:35,  6.94s/it]
 47%|████▋     | 106845/225000 [227:33:47<217:13:58,  6.62s/it]
 47%|████▋     | 106846/225000 [227:33:53<210:50:21,  6.42s/it]
 47%|████▋     | 106847/225000 [227:33:58<190:14:39,  5.80s/it]
 47%|████▋     | 106848/225000 [227:34:05<200:43:21,  6.12s/it]
 47%|████▋     | 106849/225000 [227:34:12<214:13:28,  6.53s/it]
 47%|████▋     | 106850/225000 [227:34:18<205:59:39,  6.28s/it]
                                                               
{'loss': '0.9102', 'grad_norm': '2.367', 'learning_rate': '2.889e-05', 'epoch': '0.4749', 'num_input_tokens_seen': 381568208, 'train_runtime': '8.193e+05', 'train_tokens_per_second': '465.7'}

 47%|████▋     | 106850/225000 [227:34:18<205:59:39,  6.28s/it]
 47%|████▋     | 106851/225000 [227:34:35<316:35:59,  9.65s/it]
 47%|████▋     | 106852/225000 [227:34:53<394:25:14, 12.02s/it]
 47%|████▋     | 106853/225000 [227:35:03<375:49:34, 11.45s/it]
 47%|████▋     | 106854/225000 [227:35:07<307:04:32,  9.36s/it]
 47%|████▋     | 106855/225000 [227:35:13<266:11:29,  8.11s/it]
 47%|████▋     | 106856/225000 [227:35:18<233:59:15,  7.13s/it]
 47%|████▋     | 106857/225000 [227:35:23<218:43:09,  6.66s/it]
 47%|████▋     | 106858/225000 [227:35:25<175:07:23,  5.34s/it]
 47%|████▋     | 106859/225000 [227:35:39<256:24:30,  7.81s/it]
 47%|████▋     | 106860/225000 [227:35:51<293:20:17,  8.94s/it]
 47%|████▋     | 106861/225000 [227:35:55<250:18:52,  7.63s/it]
 47%|████▋     | 106862/225000 [227:35:58<204:12:14,  6.22s/it]
 47%|████▋     | 106863/225000 [227:36:08<243:32:27,  7.42s/it]
 47%|████▋     | 106864/225000 [227:36:12<206:59:56,  6.31s/it]
 47%|████▋     | 106865/225000 [227:36:17<190:51:12,  5.82s/it]
 47%|████▋     | 106866/225000 [227:36:30<261:48:17,  7.98s/it]
 47%|████▋     | 106867/225000 [227:36:33<221:07:34,  6.74s/it]
 47%|████▋     | 106868/225000 [227:36:40<215:09:17,  6.56s/it]
 47%|████▋     | 106869/225000 [227:36:48<233:37:25,  7.12s/it]
 47%|████▋     | 106870/225000 [227:36:55<228:50:27,  6.97s/it]
 47%|████▋     | 106871/225000 [227:36:58<192:29:47,  5.87s/it]
 47%|████▋     | 106872/225000 [227:37:10<254:51:56,  7.77s/it]
 47%|████▋     | 106873/225000 [227:37:20<273:43:23,  8.34s/it]
 47%|████▋     | 106874/225000 [227:37:29<278:21:21,  8.48s/it]
 48%|████▊     | 106875/225000 [227:37:31<219:56:09,  6.70s/it]
 48%|████▊     | 106876/225000 [227:37:34<183:13:02,  5.58s/it]
 48%|████▊     | 106877/225000 [227:37:45<236:49:03,  7.22s/it]
 48%|████▊     | 106878/225000 [227:37:47<187:52:53,  5.73s/it]
 48%|████▊     | 106879/225000 [227:37:51<161:30:48,  4.92s/it]
 48%|████▊     | 106880/225000 [227:37:54<144:25:31,  4.40s/it]
 48%|████▊     | 106881/225000 [227:37:59<150:41:25,  4.59s/it]
 48%|████▊     | 106882/225000 [227:38:10<212:18:41,  6.47s/it]
 48%|████▊     | 106883/225000 [227:38:14<195:16:28,  5.95s/it]
 48%|████▊     | 106884/225000 [227:38:17<166:56:46,  5.09s/it]
 48%|████▊     | 106885/225000 [227:38:28<223:17:46,  6.81s/it]
 48%|████▊     | 106886/225000 [227:38:34<211:05:36,  6.43s/it]
 48%|████▊     | 106887/225000 [227:38:36<169:04:13,  5.15s/it]
 48%|████▊     | 106888/225000 [227:38:45<204:34:14,  6.24s/it]
 48%|████▊     | 106889/225000 [227:38:53<225:40:25,  6.88s/it]
 48%|████▊     | 106890/225000 [227:39:02<242:19:28,  7.39s/it]
 48%|████▊     | 106891/225000 [227:39:15<302:49:25,  9.23s/it]
 48%|████▊     | 106892/225000 [227:39:29<349:39:30, 10.66s/it]
 48%|████▊     | 106893/225000 [227:39:32<274:37:46,  8.37s/it]
 48%|████▊     | 106894/225000 [227:39:34<214:19:59,  6.53s/it]
 48%|████▊     | 106895/225000 [227:39:42<227:33:14,  6.94s/it]
 48%|████▊     | 106896/225000 [227:39:47<207:48:49,  6.33s/it]
 48%|████▊     | 106897/225000 [227:39:53<201:44:52,  6.15s/it]
 48%|████▊     | 106898/225000 [227:40:04<245:13:44,  7.48s/it]
 48%|████▊     | 106899/225000 [227:40:12<253:53:43,  7.74s/it]
 48%|████▊     | 106900/225000 [227:40:15<207:42:08,  6.33s/it]
                                                               
{'loss': '0.9218', 'grad_norm': '2.256', 'learning_rate': '2.888e-05', 'epoch': '0.4751', 'num_input_tokens_seen': 381729960, 'train_runtime': '8.196e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 106900/225000 [227:40:15<207:42:08,  6.33s/it]
 48%|████▊     | 106901/225000 [227:40:18<175:31:13,  5.35s/it]
 48%|████▊     | 106902/225000 [227:40:23<169:08:32,  5.16s/it]
 48%|████▊     | 106903/225000 [227:40:33<217:53:15,  6.64s/it]
 48%|████▊     | 106904/225000 [227:40:38<198:33:16,  6.05s/it]
 48%|████▊     | 106905/225000 [227:40:55<310:19:07,  9.46s/it]
 48%|████▊     | 106906/225000 [227:41:04<307:17:53,  9.37s/it]
 48%|████▊     | 106907/225000 [227:41:09<267:01:22,  8.14s/it]
 48%|████▊     | 106908/225000 [227:41:21<305:30:48,  9.31s/it]
 48%|████▊     | 106909/225000 [227:41:33<326:45:05,  9.96s/it]
 48%|████▊     | 106910/225000 [227:41:41<308:10:59,  9.40s/it]
 48%|████▊     | 106911/225000 [227:41:48<281:25:33,  8.58s/it]
 48%|████▊     | 106912/225000 [227:41:54<262:00:12,  7.99s/it]
 48%|████▊     | 106913/225000 [227:42:03<266:59:42,  8.14s/it]
 48%|████▊     | 106914/225000 [227:42:06<216:42:42,  6.61s/it]
 48%|████▊     | 106915/225000 [227:42:11<200:57:50,  6.13s/it]
 48%|████▊     | 106916/225000 [227:42:15<186:29:32,  5.69s/it]
 48%|████▊     | 106917/225000 [227:42:19<163:45:47,  4.99s/it]
 48%|████▊     | 106918/225000 [227:42:28<207:03:40,  6.31s/it]
 48%|████▊     | 106919/225000 [227:42:33<194:12:04,  5.92s/it]
 48%|████▊     | 106920/225000 [227:42:47<270:09:00,  8.24s/it]
 48%|████▊     | 106921/225000 [227:42:56<277:53:04,  8.47s/it]
 48%|████▊     | 106922/225000 [227:42:59<223:39:10,  6.82s/it]
 48%|████▊     | 106923/225000 [227:43:03<193:53:13,  5.91s/it]
 48%|████▊     | 106924/225000 [227:43:07<180:02:43,  5.49s/it]
 48%|████▊     | 106925/225000 [227:43:21<259:09:53,  7.90s/it]
 48%|████▊     | 106926/225000 [227:43:23<202:45:16,  6.18s/it]
 48%|████▊     | 106927/225000 [227:43:30<217:33:51,  6.63s/it]
 48%|████▊     | 106928/225000 [227:43:33<174:42:35,  5.33s/it]
 48%|████▊     | 106929/225000 [227:43:43<218:55:24,  6.68s/it]
 48%|████▊     | 106930/225000 [227:43:48<205:24:21,  6.26s/it]
 48%|████▊     | 106931/225000 [227:43:57<235:07:37,  7.17s/it]
 48%|████▊     | 106932/225000 [227:44:11<295:57:07,  9.02s/it]
 48%|████▊     | 106933/225000 [227:44:16<260:13:52,  7.93s/it]
 48%|████▊     | 106934/225000 [227:44:22<243:03:41,  7.41s/it]
 48%|████▊     | 106935/225000 [227:44:28<227:23:20,  6.93s/it]
 48%|████▊     | 106936/225000 [227:44:31<192:27:35,  5.87s/it]
 48%|████▊     | 106937/225000 [227:44:38<201:51:23,  6.16s/it]
 48%|████▊     | 106938/225000 [227:44:46<220:14:08,  6.72s/it]
 48%|████▊     | 106939/225000 [227:44:54<234:44:39,  7.16s/it]
 48%|████▊     | 106940/225000 [227:45:06<277:43:25,  8.47s/it]
 48%|████▊     | 106941/225000 [227:45:17<300:51:00,  9.17s/it]
 48%|████▊     | 106942/225000 [227:45:23<276:28:25,  8.43s/it]
 48%|████▊     | 106943/225000 [227:45:26<222:53:29,  6.80s/it]
 48%|████▊     | 106944/225000 [227:45:41<297:00:59,  9.06s/it]
 48%|████▊     | 106945/225000 [227:45:46<259:30:47,  7.91s/it]
 48%|████▊     | 106946/225000 [227:45:51<226:24:39,  6.90s/it]
 48%|████▊     | 106947/225000 [227:45:55<202:37:12,  6.18s/it]
 48%|████▊     | 106948/225000 [227:46:04<231:12:13,  7.05s/it]
 48%|████▊     | 106949/225000 [227:46:06<183:21:35,  5.59s/it]
 48%|████▊     | 106950/225000 [227:46:14<208:02:03,  6.34s/it]
                                                               
{'loss': '0.9', 'grad_norm': '1.756', 'learning_rate': '2.886e-05', 'epoch': '0.4753', 'num_input_tokens_seen': 381890688, 'train_runtime': '8.2e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 106950/225000 [227:46:14<208:02:03,  6.34s/it]
 48%|████▊     | 106951/225000 [227:46:27<266:16:26,  8.12s/it]
 48%|████▊     | 106952/225000 [227:46:30<214:45:48,  6.55s/it]
 48%|████▊     | 106953/225000 [227:46:32<178:48:50,  5.45s/it]
 48%|████▊     | 106954/225000 [227:46:41<205:54:45,  6.28s/it]
 48%|████▊     | 106955/225000 [227:46:48<216:38:59,  6.61s/it]
 48%|████▊     | 106956/225000 [227:46:51<182:19:54,  5.56s/it]
 48%|████▊     | 106957/225000 [227:46:55<162:08:59,  4.95s/it]
 48%|████▊     | 106958/225000 [227:46:57<135:33:41,  4.13s/it]
 48%|████▊     | 106959/225000 [227:47:16<282:35:18,  8.62s/it]
 48%|████▊     | 106960/225000 [227:47:22<253:43:04,  7.74s/it]
 48%|████▊     | 106961/225000 [227:47:29<253:24:51,  7.73s/it]
 48%|████▊     | 106962/225000 [227:47:35<231:34:46,  7.06s/it]
 48%|████▊     | 106963/225000 [227:47:37<184:56:48,  5.64s/it]
 48%|████▊     | 106964/225000 [227:47:43<188:06:47,  5.74s/it]
 48%|████▊     | 106965/225000 [227:47:45<155:00:20,  4.73s/it]
 48%|████▊     | 106966/225000 [227:47:48<131:33:04,  4.01s/it]
 48%|████▊     | 106967/225000 [227:47:51<121:14:58,  3.70s/it]
 48%|████▊     | 106968/225000 [227:48:02<197:47:17,  6.03s/it]
 48%|████▊     | 106969/225000 [227:48:04<159:24:02,  4.86s/it]
 48%|████▊     | 106970/225000 [227:48:08<147:44:23,  4.51s/it]
 48%|████▊     | 106971/225000 [227:48:11<132:42:07,  4.05s/it]
 48%|████▊     | 106972/225000 [227:48:16<142:38:56,  4.35s/it]
 48%|████▊     | 106973/225000 [227:48:30<232:18:24,  7.09s/it]
 48%|████▊     | 106974/225000 [227:48:36<223:34:05,  6.82s/it]
 48%|████▊     | 106975/225000 [227:48:41<210:00:29,  6.41s/it]
 48%|████▊     | 106976/225000 [227:48:47<200:09:09,  6.11s/it]
 48%|████▊     | 106977/225000 [227:48:50<169:58:00,  5.18s/it]
 48%|████▊     | 106978/225000 [227:49:00<223:59:33,  6.83s/it]
 48%|████▊     | 106979/225000 [227:49:03<185:51:44,  5.67s/it]
 48%|████▊     | 106980/225000 [227:49:06<159:51:51,  4.88s/it]
 48%|████▊     | 106981/225000 [227:49:12<166:40:23,  5.08s/it]
 48%|████▊     | 106982/225000 [227:49:16<155:38:41,  4.75s/it]
 48%|████▊     | 106983/225000 [227:49:24<190:50:30,  5.82s/it]
 48%|████▊     | 106984/225000 [227:49:27<161:58:46,  4.94s/it]
 48%|████▊     | 106985/225000 [227:49:36<202:20:11,  6.17s/it]
 48%|████▊     | 106986/225000 [227:49:46<240:12:18,  7.33s/it]
 48%|████▊     | 106987/225000 [227:49:51<217:34:11,  6.64s/it]
 48%|████▊     | 106988/225000 [227:49:55<189:19:26,  5.78s/it]
 48%|████▊     | 106989/225000 [227:50:01<188:32:24,  5.75s/it]
 48%|████▊     | 106990/225000 [227:50:06<180:58:42,  5.52s/it]
 48%|████▊     | 106991/225000 [227:50:16<226:15:52,  6.90s/it]
 48%|████▊     | 106992/225000 [227:50:19<187:39:44,  5.72s/it]
 48%|████▊     | 106993/225000 [227:50:21<153:13:10,  4.67s/it]
 48%|████▊     | 106994/225000 [227:50:23<129:45:23,  3.96s/it]
 48%|████▊     | 106995/225000 [227:50:26<113:31:48,  3.46s/it]
 48%|████▊     | 106996/225000 [227:50:28<107:54:32,  3.29s/it]
 48%|████▊     | 106997/225000 [227:50:42<205:44:44,  6.28s/it]
 48%|████▊     | 106998/225000 [227:50:53<255:33:01,  7.80s/it]
 48%|████▊     | 106999/225000 [227:50:59<241:39:05,  7.37s/it]
 48%|████▊     | 107000/225000 [227:51:08<249:07:53,  7.60s/it]
                                                               
{'loss': '0.9531', 'grad_norm': '2.808', 'learning_rate': '2.884e-05', 'epoch': '0.4756', 'num_input_tokens_seen': 382016176, 'train_runtime': '8.203e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107000/225000 [227:51:08<249:07:53,  7.60s/it]
 48%|████▊     | 107001/225000 [227:51:12<213:28:21,  6.51s/it]
 48%|████▊     | 107002/225000 [227:51:14<173:14:50,  5.29s/it]
 48%|████▊     | 107003/225000 [227:51:19<174:48:59,  5.33s/it]
 48%|████▊     | 107004/225000 [227:51:26<187:29:49,  5.72s/it]
 48%|████▊     | 107005/225000 [227:51:34<211:27:00,  6.45s/it]
 48%|████▊     | 107006/225000 [227:51:44<241:10:35,  7.36s/it]
 48%|████▊     | 107007/225000 [227:51:52<251:41:19,  7.68s/it]
 48%|████▊     | 107008/225000 [227:52:05<303:24:35,  9.26s/it]
 48%|████▊     | 107009/225000 [227:52:10<260:29:13,  7.95s/it]
 48%|████▊     | 107010/225000 [227:52:13<212:22:15,  6.48s/it]
 48%|████▊     | 107011/225000 [227:52:18<195:43:21,  5.97s/it]
 48%|████▊     | 107012/225000 [227:52:20<159:14:44,  4.86s/it]
 48%|████▊     | 107013/225000 [227:52:23<138:24:12,  4.22s/it]
 48%|████▊     | 107014/225000 [227:52:26<131:48:38,  4.02s/it]
 48%|████▊     | 107015/225000 [227:52:35<182:38:51,  5.57s/it]
 48%|████▊     | 107016/225000 [227:52:49<262:27:25,  8.01s/it]
 48%|████▊     | 107017/225000 [227:52:51<206:23:18,  6.30s/it]
 48%|████▊     | 107018/225000 [227:52:55<184:06:04,  5.62s/it]
 48%|████▊     | 107019/225000 [227:53:01<186:00:06,  5.68s/it]
 48%|████▊     | 107020/225000 [227:53:09<201:12:49,  6.14s/it]
 48%|████▊     | 107021/225000 [227:53:14<199:27:23,  6.09s/it]
 48%|████▊     | 107022/225000 [227:53:19<184:52:41,  5.64s/it]
 48%|████▊     | 107023/225000 [227:53:21<151:40:20,  4.63s/it]
 48%|████▊     | 107024/225000 [227:53:25<137:21:35,  4.19s/it]
 48%|████▊     | 107025/225000 [227:53:35<199:55:16,  6.10s/it]
 48%|████▊     | 107026/225000 [227:53:37<162:33:44,  4.96s/it]
 48%|████▊     | 107027/225000 [227:53:40<135:41:09,  4.14s/it]
 48%|████▊     | 107028/225000 [227:53:44<136:21:29,  4.16s/it]
 48%|████▊     | 107029/225000 [227:53:49<149:11:45,  4.55s/it]
 48%|████▊     | 107030/225000 [227:53:59<198:53:56,  6.07s/it]
 48%|████▊     | 107031/225000 [227:54:02<170:47:35,  5.21s/it]
 48%|████▊     | 107032/225000 [227:54:05<148:11:20,  4.52s/it]
 48%|████▊     | 107033/225000 [227:54:09<142:24:59,  4.35s/it]
 48%|████▊     | 107034/225000 [227:54:14<152:09:52,  4.64s/it]
 48%|████▊     | 107035/225000 [227:54:19<155:01:31,  4.73s/it]
 48%|████▊     | 107036/225000 [227:54:30<213:42:01,  6.52s/it]
 48%|████▊     | 107037/225000 [227:54:33<178:38:32,  5.45s/it]
 48%|████▊     | 107038/225000 [227:54:36<156:20:01,  4.77s/it]
 48%|████▊     | 107039/225000 [227:54:41<155:37:34,  4.75s/it]
 48%|████▊     | 107040/225000 [227:54:44<144:20:03,  4.40s/it]
 48%|████▊     | 107041/225000 [227:55:01<265:59:00,  8.12s/it]
 48%|████▊     | 107042/225000 [227:55:04<217:50:24,  6.65s/it]
 48%|████▊     | 107043/225000 [227:55:15<254:09:20,  7.76s/it]
 48%|████▊     | 107044/225000 [227:55:17<199:04:54,  6.08s/it]
 48%|████▊     | 107045/225000 [227:55:26<227:36:53,  6.95s/it]
 48%|████▊     | 107046/225000 [227:55:30<202:48:38,  6.19s/it]
 48%|████▊     | 107047/225000 [227:55:43<269:16:51,  8.22s/it]
 48%|████▊     | 107048/225000 [227:55:54<294:10:56,  8.98s/it]
 48%|████▊     | 107049/225000 [227:55:56<227:00:08,  6.93s/it]
 48%|████▊     | 107050/225000 [227:56:01<211:12:08,  6.45s/it]
                                                               
{'loss': '0.9217', 'grad_norm': '2.041', 'learning_rate': '2.882e-05', 'epoch': '0.4758', 'num_input_tokens_seen': 382144384, 'train_runtime': '8.206e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107050/225000 [227:56:01<211:12:08,  6.45s/it]
 48%|████▊     | 107051/225000 [227:56:10<233:58:36,  7.14s/it]
 48%|████▊     | 107052/225000 [227:56:16<221:37:59,  6.76s/it]
 48%|████▊     | 107053/225000 [227:56:28<268:57:13,  8.21s/it]
 48%|████▊     | 107054/225000 [227:56:30<209:50:15,  6.40s/it]
 48%|████▊     | 107055/225000 [227:56:40<245:36:09,  7.50s/it]
 48%|████▊     | 107056/225000 [227:56:42<193:34:06,  5.91s/it]
 48%|████▊     | 107057/225000 [227:56:46<174:12:07,  5.32s/it]
 48%|████▊     | 107058/225000 [227:56:52<176:53:44,  5.40s/it]
 48%|████▊     | 107059/225000 [227:56:55<153:47:29,  4.69s/it]
 48%|████▊     | 107060/225000 [227:57:03<189:44:31,  5.79s/it]
 48%|████▊     | 107061/225000 [227:57:19<294:23:25,  8.99s/it]
 48%|████▊     | 107062/225000 [227:57:30<305:36:46,  9.33s/it]
 48%|████▊     | 107063/225000 [227:57:43<344:30:33, 10.52s/it]
 48%|████▊     | 107064/225000 [227:57:45<262:24:32,  8.01s/it]
 48%|████▊     | 107065/225000 [227:57:51<238:45:17,  7.29s/it]
 48%|████▊     | 107066/225000 [227:57:59<251:40:31,  7.68s/it]
 48%|████▊     | 107067/225000 [227:58:07<254:57:47,  7.78s/it]
 48%|████▊     | 107068/225000 [227:58:10<202:45:01,  6.19s/it]
 48%|████▊     | 107069/225000 [227:58:12<166:05:28,  5.07s/it]
 48%|████▊     | 107070/225000 [227:58:16<155:14:11,  4.74s/it]
 48%|████▊     | 107071/225000 [227:58:19<139:11:37,  4.25s/it]
 48%|████▊     | 107072/225000 [227:58:26<165:15:10,  5.04s/it]
 48%|████▊     | 107073/225000 [227:58:29<143:06:18,  4.37s/it]
 48%|████▊     | 107074/225000 [227:58:37<181:52:52,  5.55s/it]
 48%|████▊     | 107075/225000 [227:58:43<187:55:53,  5.74s/it]
 48%|████▊     | 107076/225000 [227:58:49<184:32:41,  5.63s/it]
 48%|████▊     | 107077/225000 [227:58:51<152:47:13,  4.66s/it]
 48%|████▊     | 107078/225000 [227:59:01<199:51:03,  6.10s/it]
 48%|████▊     | 107079/225000 [227:59:05<178:17:21,  5.44s/it]
 48%|████▊     | 107080/225000 [227:59:09<163:14:04,  4.98s/it]
 48%|████▊     | 107081/225000 [227:59:11<137:10:09,  4.19s/it]
 48%|████▊     | 107082/225000 [227:59:16<145:43:11,  4.45s/it]
 48%|████▊     | 107083/225000 [227:59:18<127:21:44,  3.89s/it]
 48%|████▊     | 107084/225000 [227:59:23<136:52:56,  4.18s/it]
 48%|████▊     | 107085/225000 [227:59:27<134:06:08,  4.09s/it]
 48%|████▊     | 107086/225000 [227:59:30<124:21:40,  3.80s/it]
 48%|████▊     | 107087/225000 [227:59:34<125:27:52,  3.83s/it]
 48%|████▊     | 107088/225000 [227:59:41<154:23:09,  4.71s/it]
 48%|████▊     | 107089/225000 [227:59:53<229:21:52,  7.00s/it]
 48%|████▊     | 107090/225000 [227:59:56<182:50:06,  5.58s/it]
 48%|████▊     | 107091/225000 [228:00:00<167:02:33,  5.10s/it]
 48%|████▊     | 107092/225000 [228:00:03<151:08:52,  4.61s/it]
 48%|████▊     | 107093/225000 [228:00:05<127:43:02,  3.90s/it]
 48%|████▊     | 107094/225000 [228:00:08<118:48:03,  3.63s/it]
 48%|████▊     | 107095/225000 [228:00:12<115:06:42,  3.51s/it]
 48%|████▊     | 107096/225000 [228:00:26<226:24:08,  6.91s/it]
 48%|████▊     | 107097/225000 [228:00:29<187:29:28,  5.72s/it]
 48%|████▊     | 107098/225000 [228:00:39<226:03:38,  6.90s/it]
 48%|████▊     | 107099/225000 [228:00:43<197:13:06,  6.02s/it]
 48%|████▊     | 107100/225000 [228:00:49<192:36:17,  5.88s/it]
                                                               
{'loss': '0.9566', 'grad_norm': '1.546', 'learning_rate': '2.88e-05', 'epoch': '0.476', 'num_input_tokens_seen': 382275648, 'train_runtime': '8.208e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107100/225000 [228:00:49<192:36:17,  5.88s/it]
 48%|████▊     | 107101/225000 [228:00:55<202:45:59,  6.19s/it]
 48%|████▊     | 107102/225000 [228:01:04<228:29:51,  6.98s/it]
 48%|████▊     | 107103/225000 [228:01:07<190:35:42,  5.82s/it]
 48%|████▊     | 107104/225000 [228:01:10<155:07:56,  4.74s/it]
 48%|████▊     | 107105/225000 [228:01:15<157:20:53,  4.80s/it]
 48%|████▊     | 107106/225000 [228:01:18<146:52:33,  4.48s/it]
 48%|████▊     | 107107/225000 [228:01:33<250:39:28,  7.65s/it]
 48%|████▊     | 107108/225000 [228:01:40<243:07:06,  7.42s/it]
 48%|████▊     | 107109/225000 [228:01:50<269:27:37,  8.23s/it]
 48%|████▊     | 107110/225000 [228:01:53<218:50:37,  6.68s/it]
 48%|████▊     | 107111/225000 [228:01:58<195:18:25,  5.96s/it]
 48%|████▊     | 107112/225000 [228:02:05<207:53:42,  6.35s/it]
 48%|████▊     | 107113/225000 [228:02:08<173:46:34,  5.31s/it]
 48%|████▊     | 107114/225000 [228:02:12<163:01:55,  4.98s/it]
 48%|████▊     | 107115/225000 [228:02:19<184:39:54,  5.64s/it]
 48%|████▊     | 107116/225000 [228:02:27<200:59:18,  6.14s/it]
 48%|████▊     | 107117/225000 [228:02:30<170:26:04,  5.20s/it]
 48%|████▊     | 107118/225000 [228:02:33<148:44:17,  4.54s/it]
 48%|████▊     | 107119/225000 [228:02:42<194:45:54,  5.95s/it]
 48%|████▊     | 107120/225000 [228:02:45<170:27:20,  5.21s/it]
 48%|████▊     | 107121/225000 [228:02:56<223:12:40,  6.82s/it]
 48%|████▊     | 107122/225000 [228:03:03<230:31:02,  7.04s/it]
 48%|████▊     | 107123/225000 [228:03:10<223:24:09,  6.82s/it]
 48%|████▊     | 107124/225000 [228:03:19<250:04:13,  7.64s/it]
 48%|████▊     | 107125/225000 [228:03:24<226:03:31,  6.90s/it]
 48%|████▊     | 107126/225000 [228:03:35<266:49:53,  8.15s/it]
 48%|████▊     | 107127/225000 [228:03:42<251:01:26,  7.67s/it]
 48%|████▊     | 107128/225000 [228:03:44<198:25:58,  6.06s/it]
 48%|████▊     | 107129/225000 [228:03:49<188:42:20,  5.76s/it]
 48%|████▊     | 107130/225000 [228:03:58<211:53:05,  6.47s/it]
 48%|████▊     | 107131/225000 [228:04:02<192:04:40,  5.87s/it]
 48%|████▊     | 107132/225000 [228:04:10<209:27:18,  6.40s/it]
 48%|████▊     | 107133/225000 [228:04:13<178:06:52,  5.44s/it]
 48%|████▊     | 107134/225000 [228:04:21<203:54:24,  6.23s/it]
 48%|████▊     | 107135/225000 [228:04:26<189:24:57,  5.79s/it]
 48%|████▊     | 107136/225000 [228:04:29<161:36:14,  4.94s/it]
 48%|████▊     | 107137/225000 [228:04:42<241:50:23,  7.39s/it]
 48%|████▊     | 107138/225000 [228:04:44<191:25:42,  5.85s/it]
 48%|████▊     | 107139/225000 [228:04:49<186:23:16,  5.69s/it]
 48%|████▊     | 107140/225000 [228:04:55<182:59:38,  5.59s/it]
 48%|████▊     | 107141/225000 [228:05:05<231:11:31,  7.06s/it]
 48%|████▊     | 107142/225000 [228:05:11<217:41:45,  6.65s/it]
 48%|████▊     | 107143/225000 [228:05:13<177:10:14,  5.41s/it]
 48%|████▊     | 107144/225000 [228:05:20<191:38:38,  5.85s/it]
 48%|████▊     | 107145/225000 [228:05:22<155:21:21,  4.75s/it]
 48%|████▊     | 107146/225000 [228:05:31<194:57:02,  5.96s/it]
 48%|████▊     | 107147/225000 [228:05:38<202:46:48,  6.19s/it]
 48%|████▊     | 107148/225000 [228:05:45<211:22:19,  6.46s/it]
 48%|████▊     | 107149/225000 [228:05:51<203:02:53,  6.20s/it]
 48%|████▊     | 107150/225000 [228:05:58<219:14:06,  6.70s/it]
                                                               
{'loss': '0.9674', 'grad_norm': '1.533', 'learning_rate': '2.879e-05', 'epoch': '0.4762', 'num_input_tokens_seen': 382406104, 'train_runtime': '8.212e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107150/225000 [228:05:58<219:14:06,  6.70s/it]
 48%|████▊     | 107151/225000 [228:06:09<261:11:55,  7.98s/it]
 48%|████▊     | 107152/225000 [228:06:15<233:14:29,  7.13s/it]
 48%|████▊     | 107153/225000 [228:06:18<194:05:49,  5.93s/it]
 48%|████▊     | 107154/225000 [228:06:23<192:39:00,  5.89s/it]
 48%|████▊     | 107155/225000 [228:06:30<195:21:03,  5.97s/it]
 48%|████▊     | 107156/225000 [228:06:34<182:45:08,  5.58s/it]
 48%|████▊     | 107157/225000 [228:06:43<209:40:36,  6.41s/it]
 48%|████▊     | 107158/225000 [228:06:48<204:00:26,  6.23s/it]
 48%|████▊     | 107159/225000 [228:06:52<177:39:55,  5.43s/it]
 48%|████▊     | 107160/225000 [228:06:56<159:16:11,  4.87s/it]
 48%|████▊     | 107161/225000 [228:06:59<145:54:58,  4.46s/it]
 48%|████▊     | 107162/225000 [228:07:08<190:59:47,  5.84s/it]
 48%|████▊     | 107163/225000 [228:07:10<155:57:42,  4.76s/it]
 48%|████▊     | 107164/225000 [228:07:20<200:36:45,  6.13s/it]
 48%|████▊     | 107165/225000 [228:07:22<161:39:08,  4.94s/it]
 48%|████▊     | 107166/225000 [228:07:24<135:53:30,  4.15s/it]
 48%|████▊     | 107167/225000 [228:07:29<142:47:06,  4.36s/it]
 48%|████▊     | 107168/225000 [228:07:32<129:16:25,  3.95s/it]
 48%|████▊     | 107169/225000 [228:07:37<142:39:31,  4.36s/it]
 48%|████▊     | 107170/225000 [228:07:48<206:38:53,  6.31s/it]
 48%|████▊     | 107171/225000 [228:07:55<209:50:57,  6.41s/it]
 48%|████▊     | 107172/225000 [228:08:02<214:06:00,  6.54s/it]
 48%|████▊     | 107173/225000 [228:08:05<180:34:33,  5.52s/it]
 48%|████▊     | 107174/225000 [228:08:08<157:26:43,  4.81s/it]
 48%|████▊     | 107175/225000 [228:08:17<195:39:32,  5.98s/it]
 48%|████▊     | 107176/225000 [228:08:22<188:41:52,  5.77s/it]
 48%|████▊     | 107177/225000 [228:08:43<340:59:25, 10.42s/it]
 48%|████▊     | 107178/225000 [228:08:51<318:46:37,  9.74s/it]
 48%|████▊     | 107179/225000 [228:08:58<283:47:29,  8.67s/it]
 48%|████▊     | 107180/225000 [228:09:07<293:48:51,  8.98s/it]
 48%|████▊     | 107181/225000 [228:09:20<326:46:22,  9.98s/it]
 48%|████▊     | 107182/225000 [228:09:25<281:32:27,  8.60s/it]
 48%|████▊     | 107183/225000 [228:09:27<220:47:26,  6.75s/it]
 48%|████▊     | 107184/225000 [228:09:34<214:50:39,  6.56s/it]
 48%|████▊     | 107185/225000 [228:09:38<195:00:19,  5.96s/it]
 48%|████▊     | 107186/225000 [228:09:41<168:44:59,  5.16s/it]
 48%|████▊     | 107187/225000 [228:09:59<289:49:43,  8.86s/it]
 48%|████▊     | 107188/225000 [228:10:04<253:04:28,  7.73s/it]
 48%|████▊     | 107189/225000 [228:10:06<198:52:53,  6.08s/it]
 48%|████▊     | 107190/225000 [228:10:11<185:02:45,  5.65s/it]
 48%|████▊     | 107191/225000 [228:10:13<153:04:10,  4.68s/it]
 48%|████▊     | 107192/225000 [228:10:29<259:13:33,  7.92s/it]
 48%|████▊     | 107193/225000 [228:10:32<217:09:58,  6.64s/it]
 48%|████▊     | 107194/225000 [228:10:37<195:08:38,  5.96s/it]
 48%|████▊     | 107195/225000 [228:10:40<170:55:32,  5.22s/it]
 48%|████▊     | 107196/225000 [228:10:54<250:19:55,  7.65s/it]
 48%|████▊     | 107197/225000 [228:10:59<227:42:09,  6.96s/it]
 48%|████▊     | 107198/225000 [228:11:04<208:32:37,  6.37s/it]
 48%|████▊     | 107199/225000 [228:11:19<290:39:05,  8.88s/it]
 48%|████▊     | 107200/225000 [228:11:28<291:39:21,  8.91s/it]
                                                               
{'loss': '0.9251', 'grad_norm': '1.91', 'learning_rate': '2.877e-05', 'epoch': '0.4764', 'num_input_tokens_seen': 382557592, 'train_runtime': '8.215e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107200/225000 [228:11:28<291:39:21,  8.91s/it]
 48%|████▊     | 107201/225000 [228:11:30<226:18:38,  6.92s/it]
 48%|████▊     | 107202/225000 [228:11:36<216:30:50,  6.62s/it]
 48%|████▊     | 107203/225000 [228:11:44<233:27:02,  7.13s/it]
 48%|████▊     | 107204/225000 [228:11:46<186:12:38,  5.69s/it]
 48%|████▊     | 107205/225000 [228:11:50<160:31:25,  4.91s/it]
 48%|████▊     | 107206/225000 [228:12:01<222:09:05,  6.79s/it]
 48%|████▊     | 107207/225000 [228:12:04<184:49:22,  5.65s/it]
 48%|████▊     | 107208/225000 [228:12:09<181:02:33,  5.53s/it]
 48%|████▊     | 107209/225000 [228:12:15<187:18:09,  5.72s/it]
 48%|████▊     | 107210/225000 [228:12:26<232:56:33,  7.12s/it]
 48%|████▊     | 107211/225000 [228:12:34<243:56:10,  7.46s/it]
 48%|████▊     | 107212/225000 [228:12:40<230:42:04,  7.05s/it]
 48%|████▊     | 107213/225000 [228:12:43<192:24:30,  5.88s/it]
 48%|████▊     | 107214/225000 [228:12:48<186:38:27,  5.70s/it]
 48%|████▊     | 107215/225000 [228:12:54<188:39:07,  5.77s/it]
 48%|████▊     | 107216/225000 [228:13:03<214:26:09,  6.55s/it]
 48%|████▊     | 107217/225000 [228:13:07<189:13:40,  5.78s/it]
 48%|████▊     | 107218/225000 [228:13:26<323:36:41,  9.89s/it]
 48%|████▊     | 107219/225000 [228:13:32<283:48:50,  8.67s/it]
 48%|████▊     | 107220/225000 [228:13:34<221:07:28,  6.76s/it]
 48%|████▊     | 107221/225000 [228:13:37<177:27:26,  5.42s/it]
 48%|████▊     | 107222/225000 [228:13:47<228:30:14,  6.98s/it]
 48%|████▊     | 107223/225000 [228:13:53<221:05:47,  6.76s/it]
 48%|████▊     | 107224/225000 [228:14:01<231:39:09,  7.08s/it]
 48%|████▊     | 107225/225000 [228:14:07<221:43:35,  6.78s/it]
 48%|████▊     | 107226/225000 [228:14:16<237:07:33,  7.25s/it]
 48%|████▊     | 107227/225000 [228:14:19<197:46:33,  6.05s/it]
 48%|████▊     | 107228/225000 [228:14:25<197:21:00,  6.03s/it]
 48%|████▊     | 107229/225000 [228:14:27<161:18:35,  4.93s/it]
 48%|████▊     | 107230/225000 [228:14:38<214:10:08,  6.55s/it]
 48%|████▊     | 107231/225000 [228:14:47<241:22:31,  7.38s/it]
 48%|████▊     | 107232/225000 [228:14:55<248:30:44,  7.60s/it]
 48%|████▊     | 107233/225000 [228:14:58<203:29:33,  6.22s/it]
 48%|████▊     | 107234/225000 [228:15:07<229:39:09,  7.02s/it]
 48%|████▊     | 107235/225000 [228:15:10<190:24:31,  5.82s/it]
 48%|████▊     | 107236/225000 [228:15:15<182:43:23,  5.59s/it]
 48%|████▊     | 107237/225000 [228:15:30<271:18:06,  8.29s/it]
 48%|████▊     | 107238/225000 [228:15:41<301:15:53,  9.21s/it]
 48%|████▊     | 107239/225000 [228:15:47<269:05:01,  8.23s/it]
 48%|████▊     | 107240/225000 [228:15:50<218:46:57,  6.69s/it]
 48%|████▊     | 107241/225000 [228:15:53<181:23:04,  5.55s/it]
 48%|████▊     | 107242/225000 [228:15:57<172:49:37,  5.28s/it]
 48%|████▊     | 107243/225000 [228:16:03<172:06:41,  5.26s/it]
 48%|████▊     | 107244/225000 [228:16:11<200:31:54,  6.13s/it]
 48%|████▊     | 107245/225000 [228:16:23<262:40:54,  8.03s/it]
 48%|████▊     | 107246/225000 [228:16:26<214:26:33,  6.56s/it]
 48%|████▊     | 107247/225000 [228:16:37<251:45:55,  7.70s/it]
 48%|████▊     | 107248/225000 [228:16:39<197:23:28,  6.03s/it]
 48%|████▊     | 107249/225000 [228:16:45<198:53:59,  6.08s/it]
 48%|████▊     | 107250/225000 [228:16:51<199:19:28,  6.09s/it]
                                                               
{'loss': '0.9274', 'grad_norm': '1.709', 'learning_rate': '2.875e-05', 'epoch': '0.4767', 'num_input_tokens_seen': 382708240, 'train_runtime': '8.218e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107250/225000 [228:16:51<199:19:28,  6.09s/it]
 48%|████▊     | 107251/225000 [228:16:55<176:05:31,  5.38s/it]
 48%|████▊     | 107252/225000 [228:17:07<238:20:28,  7.29s/it]
 48%|████▊     | 107253/225000 [228:17:09<188:12:16,  5.75s/it]
 48%|████▊     | 107254/225000 [228:17:11<153:23:03,  4.69s/it]
 48%|████▊     | 107255/225000 [228:17:21<203:26:29,  6.22s/it]
 48%|████▊     | 107256/225000 [228:17:26<193:08:48,  5.91s/it]
 48%|████▊     | 107257/225000 [228:17:37<242:54:02,  7.43s/it]
 48%|████▊     | 107258/225000 [228:17:40<199:28:16,  6.10s/it]
 48%|████▊     | 107259/225000 [228:17:47<211:39:41,  6.47s/it]
 48%|████▊     | 107260/225000 [228:17:53<199:19:24,  6.09s/it]
 48%|████▊     | 107261/225000 [228:18:07<284:17:22,  8.69s/it]
 48%|████▊     | 107262/225000 [228:18:11<232:07:03,  7.10s/it]
 48%|████▊     | 107263/225000 [228:18:22<269:08:47,  8.23s/it]
 48%|████▊     | 107264/225000 [228:18:26<235:05:21,  7.19s/it]
 48%|████▊     | 107265/225000 [228:18:29<186:47:37,  5.71s/it]
 48%|████▊     | 107266/225000 [228:18:32<159:35:07,  4.88s/it]
 48%|████▊     | 107267/225000 [228:18:35<141:16:44,  4.32s/it]
 48%|████▊     | 107268/225000 [228:18:51<255:31:42,  7.81s/it]
 48%|████▊     | 107269/225000 [228:19:10<365:44:57, 11.18s/it]
 48%|████▊     | 107270/225000 [228:19:16<314:05:10,  9.60s/it]
 48%|████▊     | 107271/225000 [228:19:19<249:19:02,  7.62s/it]
 48%|████▊     | 107272/225000 [228:19:29<278:25:12,  8.51s/it]
 48%|████▊     | 107273/225000 [228:19:34<247:08:40,  7.56s/it]
 48%|████▊     | 107274/225000 [228:19:37<203:04:28,  6.21s/it]
 48%|████▊     | 107275/225000 [228:19:41<175:40:06,  5.37s/it]
 48%|████▊     | 107276/225000 [228:19:44<151:29:00,  4.63s/it]
 48%|████▊     | 107277/225000 [228:19:54<206:04:20,  6.30s/it]
 48%|████▊     | 107278/225000 [228:19:58<185:19:53,  5.67s/it]
 48%|████▊     | 107279/225000 [228:20:09<233:21:46,  7.14s/it]
 48%|████▊     | 107280/225000 [228:20:20<270:18:32,  8.27s/it]
 48%|████▊     | 107281/225000 [228:20:23<225:41:29,  6.90s/it]
 48%|████▊     | 107282/225000 [228:20:29<210:37:45,  6.44s/it]
 48%|████▊     | 107283/225000 [228:20:38<238:33:01,  7.30s/it]
 48%|████▊     | 107284/225000 [228:20:45<231:35:19,  7.08s/it]
 48%|████▊     | 107285/225000 [228:20:49<207:39:02,  6.35s/it]
 48%|████▊     | 107286/225000 [228:20:53<179:57:32,  5.50s/it]
 48%|████▊     | 107287/225000 [228:20:56<156:25:59,  4.78s/it]
 48%|████▊     | 107288/225000 [228:21:03<179:20:51,  5.49s/it]
 48%|████▊     | 107289/225000 [228:21:05<147:53:28,  4.52s/it]
 48%|████▊     | 107290/225000 [228:21:17<220:57:06,  6.76s/it]
 48%|████▊     | 107291/225000 [228:21:20<183:51:35,  5.62s/it]
 48%|████▊     | 107292/225000 [228:21:32<243:46:20,  7.46s/it]
 48%|████▊     | 107293/225000 [228:21:34<192:25:35,  5.89s/it]
 48%|████▊     | 107294/225000 [228:21:39<179:24:01,  5.49s/it]
 48%|████▊     | 107295/225000 [228:21:48<213:33:41,  6.53s/it]
 48%|████▊     | 107296/225000 [228:21:56<229:13:08,  7.01s/it]
 48%|████▊     | 107297/225000 [228:21:58<186:23:08,  5.70s/it]
 48%|████▊     | 107298/225000 [228:22:10<247:21:14,  7.57s/it]
 48%|████▊     | 107299/225000 [228:22:16<223:27:43,  6.83s/it]
 48%|████▊     | 107300/225000 [228:22:19<186:34:30,  5.71s/it]
                                                               
{'loss': '0.9546', 'grad_norm': '2.967', 'learning_rate': '2.873e-05', 'epoch': '0.4769', 'num_input_tokens_seen': 382855336, 'train_runtime': '8.221e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107300/225000 [228:22:19<186:34:30,  5.71s/it]
 48%|████▊     | 107301/225000 [228:22:32<262:42:38,  8.04s/it]
 48%|████▊     | 107302/225000 [228:22:35<210:31:15,  6.44s/it]
 48%|████▊     | 107303/225000 [228:22:43<224:34:06,  6.87s/it]
 48%|████▊     | 107304/225000 [228:22:45<179:21:50,  5.49s/it]
 48%|████▊     | 107305/225000 [228:22:47<148:12:20,  4.53s/it]
 48%|████▊     | 107306/225000 [228:22:51<141:03:46,  4.31s/it]
 48%|████▊     | 107307/225000 [228:22:56<149:30:51,  4.57s/it]
 48%|████▊     | 107308/225000 [228:23:00<142:11:51,  4.35s/it]
 48%|████▊     | 107309/225000 [228:23:05<149:01:56,  4.56s/it]
 48%|████▊     | 107310/225000 [228:23:16<209:51:01,  6.42s/it]
 48%|████▊     | 107311/225000 [228:23:21<194:07:00,  5.94s/it]
 48%|████▊     | 107312/225000 [228:23:29<222:24:20,  6.80s/it]
 48%|████▊     | 107313/225000 [228:23:39<251:17:59,  7.69s/it]
 48%|████▊     | 107314/225000 [228:23:41<197:02:30,  6.03s/it]
 48%|████▊     | 107315/225000 [228:23:50<221:30:54,  6.78s/it]
 48%|████▊     | 107316/225000 [228:23:55<206:17:26,  6.31s/it]
 48%|████▊     | 107317/225000 [228:24:03<224:44:28,  6.87s/it]
 48%|████▊     | 107318/225000 [228:24:21<327:54:32, 10.03s/it]
 48%|████▊     | 107319/225000 [228:24:26<277:24:49,  8.49s/it]
 48%|████▊     | 107320/225000 [228:24:29<224:42:12,  6.87s/it]
 48%|████▊     | 107321/225000 [228:24:32<187:49:11,  5.75s/it]
 48%|████▊     | 107322/225000 [228:24:40<210:21:44,  6.44s/it]
 48%|████▊     | 107323/225000 [228:24:46<206:02:09,  6.30s/it]
 48%|████▊     | 107324/225000 [228:24:59<267:58:45,  8.20s/it]
 48%|████▊     | 107325/225000 [228:25:06<264:45:53,  8.10s/it]
 48%|████▊     | 107326/225000 [228:25:11<234:12:58,  7.17s/it]
 48%|████▊     | 107327/225000 [228:25:15<195:23:56,  5.98s/it]
 48%|████▊     | 107328/225000 [228:25:21<197:35:44,  6.05s/it]
 48%|████▊     | 107329/225000 [228:25:23<161:46:18,  4.95s/it]
 48%|████▊     | 107330/225000 [228:25:31<193:42:01,  5.93s/it]
 48%|████▊     | 107331/225000 [228:25:37<194:57:59,  5.96s/it]
 48%|████▊     | 107332/225000 [228:25:48<238:30:35,  7.30s/it]
 48%|████▊     | 107333/225000 [228:25:54<226:06:31,  6.92s/it]
 48%|████▊     | 107334/225000 [228:25:57<191:36:15,  5.86s/it]
 48%|████▊     | 107335/225000 [228:26:07<226:19:45,  6.92s/it]
 48%|████▊     | 107336/225000 [228:26:15<242:14:05,  7.41s/it]
 48%|████▊     | 107337/225000 [228:26:18<193:59:12,  5.94s/it]
 48%|████▊     | 107338/225000 [228:26:28<238:22:24,  7.29s/it]
 48%|████▊     | 107339/225000 [228:26:38<265:03:41,  8.11s/it]
 48%|████▊     | 107340/225000 [228:26:44<244:39:43,  7.49s/it]
 48%|████▊     | 107341/225000 [228:26:50<224:50:12,  6.88s/it]
 48%|████▊     | 107342/225000 [228:26:54<204:00:41,  6.24s/it]
 48%|████▊     | 107343/225000 [228:27:00<200:37:04,  6.14s/it]
 48%|████▊     | 107344/225000 [228:27:10<236:54:17,  7.25s/it]
 48%|████▊     | 107345/225000 [228:27:19<250:07:29,  7.65s/it]
 48%|████▊     | 107346/225000 [228:27:21<198:56:53,  6.09s/it]
 48%|████▊     | 107347/225000 [228:27:32<244:42:34,  7.49s/it]
 48%|████▊     | 107348/225000 [228:27:36<212:35:22,  6.50s/it]
 48%|████▊     | 107349/225000 [228:27:38<170:12:03,  5.21s/it]
 48%|████▊     | 107350/225000 [228:27:50<231:16:32,  7.08s/it]
                                                               
{'loss': '0.9269', 'grad_norm': '1.051', 'learning_rate': '2.871e-05', 'epoch': '0.4771', 'num_input_tokens_seen': 383009408, 'train_runtime': '8.225e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107350/225000 [228:27:50<231:16:32,  7.08s/it]
 48%|████▊     | 107351/225000 [228:27:55<213:45:01,  6.54s/it]
 48%|████▊     | 107352/225000 [228:28:12<319:15:12,  9.77s/it]
 48%|████▊     | 107353/225000 [228:28:16<256:38:19,  7.85s/it]
 48%|████▊     | 107354/225000 [228:28:21<229:23:39,  7.02s/it]
 48%|████▊     | 107355/225000 [228:28:25<201:46:49,  6.17s/it]
 48%|████▊     | 107356/225000 [228:28:35<239:51:37,  7.34s/it]
 48%|████▊     | 107357/225000 [228:28:39<206:43:38,  6.33s/it]
 48%|████▊     | 107358/225000 [228:28:47<219:24:38,  6.71s/it]
 48%|████▊     | 107359/225000 [228:28:50<181:24:17,  5.55s/it]
 48%|████▊     | 107360/225000 [228:29:05<278:03:44,  8.51s/it]
 48%|████▊     | 107361/225000 [228:29:07<215:47:26,  6.60s/it]
 48%|████▊     | 107362/225000 [228:29:13<205:08:55,  6.28s/it]
 48%|████▊     | 107363/225000 [228:29:16<173:15:43,  5.30s/it]
 48%|████▊     | 107364/225000 [228:29:25<212:54:08,  6.52s/it]
 48%|████▊     | 107365/225000 [228:29:30<195:18:09,  5.98s/it]
 48%|████▊     | 107366/225000 [228:29:33<166:27:18,  5.09s/it]
 48%|████▊     | 107367/225000 [228:29:36<145:51:36,  4.46s/it]
 48%|████▊     | 107368/225000 [228:29:38<126:43:11,  3.88s/it]
 48%|████▊     | 107369/225000 [228:29:42<127:43:03,  3.91s/it]
 48%|████▊     | 107370/225000 [228:29:51<179:04:28,  5.48s/it]
 48%|████▊     | 107371/225000 [228:30:02<230:01:10,  7.04s/it]
 48%|████▊     | 107372/225000 [228:30:04<184:39:41,  5.65s/it]
 48%|████▊     | 107373/225000 [228:30:07<153:03:40,  4.68s/it]
 48%|████▊     | 107374/225000 [228:30:19<223:45:06,  6.85s/it]
 48%|████▊     | 107375/225000 [228:30:22<189:24:44,  5.80s/it]
 48%|████▊     | 107376/225000 [228:30:27<183:30:20,  5.62s/it]
 48%|████▊     | 107377/225000 [228:30:35<199:10:44,  6.10s/it]
 48%|████▊     | 107378/225000 [228:30:46<251:39:42,  7.70s/it]
 48%|████▊     | 107379/225000 [228:30:50<210:50:34,  6.45s/it]
 48%|████▊     | 107380/225000 [228:30:55<202:23:12,  6.19s/it]
 48%|████▊     | 107381/225000 [228:31:01<196:25:14,  6.01s/it]
 48%|████▊     | 107382/225000 [228:31:06<190:03:43,  5.82s/it]
 48%|████▊     | 107383/225000 [228:31:18<253:26:15,  7.76s/it]
 48%|████▊     | 107384/225000 [228:31:25<241:53:55,  7.40s/it]
 48%|████▊     | 107385/225000 [228:31:33<251:49:42,  7.71s/it]
 48%|████▊     | 107386/225000 [228:31:38<226:17:11,  6.93s/it]
 48%|████▊     | 107387/225000 [228:31:46<229:33:36,  7.03s/it]
 48%|████▊     | 107388/225000 [228:31:49<190:20:13,  5.83s/it]
 48%|████▊     | 107389/225000 [228:31:57<211:24:51,  6.47s/it]
 48%|████▊     | 107390/225000 [228:32:04<217:59:29,  6.67s/it]
 48%|████▊     | 107391/225000 [228:32:15<263:57:06,  8.08s/it]
 48%|████▊     | 107392/225000 [228:32:17<207:04:08,  6.34s/it]
 48%|████▊     | 107393/225000 [228:32:24<211:17:21,  6.47s/it]
 48%|████▊     | 107394/225000 [228:32:36<265:35:42,  8.13s/it]
 48%|████▊     | 107395/225000 [228:32:41<234:12:29,  7.17s/it]
 48%|████▊     | 107396/225000 [228:32:52<269:56:08,  8.26s/it]
 48%|████▊     | 107397/225000 [228:33:00<271:35:11,  8.31s/it]
 48%|████▊     | 107398/225000 [228:33:03<219:04:30,  6.71s/it]
 48%|████▊     | 107399/225000 [228:33:08<197:05:05,  6.03s/it]
 48%|████▊     | 107400/225000 [228:33:14<195:34:29,  5.99s/it]
                                                               
{'loss': '0.8932', 'grad_norm': '2.025', 'learning_rate': '2.869e-05', 'epoch': '0.4773', 'num_input_tokens_seen': 383152088, 'train_runtime': '8.228e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107400/225000 [228:33:14<195:34:29,  5.99s/it]
 48%|████▊     | 107401/225000 [228:33:22<217:57:29,  6.67s/it]
 48%|████▊     | 107402/225000 [228:33:28<211:03:03,  6.46s/it]
 48%|████▊     | 107403/225000 [228:33:35<219:59:33,  6.73s/it]
 48%|████▊     | 107404/225000 [228:33:38<177:20:48,  5.43s/it]
 48%|████▊     | 107405/225000 [228:33:40<150:49:01,  4.62s/it]
 48%|████▊     | 107406/225000 [228:33:43<128:26:55,  3.93s/it]
 48%|████▊     | 107407/225000 [228:33:48<143:30:25,  4.39s/it]
 48%|████▊     | 107408/225000 [228:33:54<156:30:05,  4.79s/it]
 48%|████▊     | 107409/225000 [228:34:06<228:15:51,  6.99s/it]
 48%|████▊     | 107410/225000 [228:34:12<216:26:06,  6.63s/it]
 48%|████▊     | 107411/225000 [228:34:17<200:15:48,  6.13s/it]
 48%|████▊     | 107412/225000 [228:34:19<162:01:20,  4.96s/it]
 48%|████▊     | 107413/225000 [228:34:23<150:55:50,  4.62s/it]
 48%|████▊     | 107414/225000 [228:34:37<242:55:49,  7.44s/it]
 48%|████▊     | 107415/225000 [228:34:39<191:54:26,  5.88s/it]
 48%|████▊     | 107416/225000 [228:34:49<230:16:12,  7.05s/it]
 48%|████▊     | 107417/225000 [228:34:55<218:27:25,  6.69s/it]
 48%|████▊     | 107418/225000 [228:35:01<211:28:41,  6.47s/it]
 48%|████▊     | 107419/225000 [228:35:04<184:22:07,  5.64s/it]
 48%|████▊     | 107420/225000 [228:35:08<166:00:58,  5.08s/it]
 48%|████▊     | 107421/225000 [228:35:11<146:03:53,  4.47s/it]
 48%|████▊     | 107422/225000 [228:35:17<153:50:53,  4.71s/it]
 48%|████▊     | 107423/225000 [228:35:21<150:44:50,  4.62s/it]
 48%|████▊     | 107424/225000 [228:35:32<217:26:13,  6.66s/it]
 48%|████▊     | 107425/225000 [228:35:41<235:10:11,  7.20s/it]
 48%|████▊     | 107426/225000 [228:35:44<196:54:19,  6.03s/it]
 48%|████▊     | 107427/225000 [228:35:50<197:56:34,  6.06s/it]
 48%|████▊     | 107428/225000 [228:35:53<169:50:00,  5.20s/it]
 48%|████▊     | 107429/225000 [228:35:57<157:03:31,  4.81s/it]
 48%|████▊     | 107430/225000 [228:36:00<133:15:38,  4.08s/it]
 48%|████▊     | 107431/225000 [228:36:08<177:20:10,  5.43s/it]
 48%|████▊     | 107432/225000 [228:36:14<177:47:30,  5.44s/it]
 48%|████▊     | 107433/225000 [228:36:17<156:48:33,  4.80s/it]
 48%|████▊     | 107434/225000 [228:36:20<134:15:19,  4.11s/it]
 48%|████▊     | 107435/225000 [228:36:27<171:20:31,  5.25s/it]
 48%|████▊     | 107436/225000 [228:36:38<227:45:15,  6.97s/it]
 48%|████▊     | 107437/225000 [228:36:41<180:42:23,  5.53s/it]
 48%|████▊     | 107438/225000 [228:36:46<176:47:42,  5.41s/it]
 48%|████▊     | 107439/225000 [228:36:58<247:23:43,  7.58s/it]
 48%|████▊     | 107440/225000 [228:37:04<223:41:46,  6.85s/it]
 48%|████▊     | 107441/225000 [228:37:10<222:33:28,  6.82s/it]
 48%|████▊     | 107442/225000 [228:37:13<178:40:26,  5.47s/it]
 48%|████▊     | 107443/225000 [228:37:18<175:26:26,  5.37s/it]
 48%|████▊     | 107444/225000 [228:37:28<218:55:55,  6.70s/it]
 48%|████▊     | 107445/225000 [228:37:35<227:59:22,  6.98s/it]
 48%|████▊     | 107446/225000 [228:37:42<226:40:53,  6.94s/it]
 48%|████▊     | 107447/225000 [228:37:48<215:23:46,  6.60s/it]
 48%|████▊     | 107448/225000 [228:37:50<174:11:17,  5.33s/it]
 48%|████▊     | 107449/225000 [228:37:57<189:09:04,  5.79s/it]
 48%|████▊     | 107450/225000 [228:38:02<184:40:36,  5.66s/it]
                                                               
{'loss': '0.9293', 'grad_norm': '3.315', 'learning_rate': '2.868e-05', 'epoch': '0.4776', 'num_input_tokens_seen': 383285864, 'train_runtime': '8.231e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107450/225000 [228:38:02<184:40:36,  5.66s/it]
 48%|████▊     | 107451/225000 [228:38:08<184:22:44,  5.65s/it]
 48%|████▊     | 107452/225000 [228:38:24<286:07:47,  8.76s/it]
 48%|████▊     | 107453/225000 [228:38:26<221:53:40,  6.80s/it]
 48%|████▊     | 107454/225000 [228:38:30<191:50:13,  5.88s/it]
 48%|████▊     | 107455/225000 [228:38:35<184:50:16,  5.66s/it]
 48%|████▊     | 107456/225000 [228:38:46<235:49:22,  7.22s/it]
 48%|████▊     | 107457/225000 [228:38:52<224:49:09,  6.89s/it]
 48%|████▊     | 107458/225000 [228:38:57<209:04:09,  6.40s/it]
 48%|████▊     | 107459/225000 [228:39:01<178:02:48,  5.45s/it]
 48%|████▊     | 107460/225000 [228:39:10<218:34:15,  6.69s/it]
 48%|████▊     | 107461/225000 [228:39:16<205:38:13,  6.30s/it]
 48%|████▊     | 107462/225000 [228:39:24<223:37:17,  6.85s/it]
 48%|████▊     | 107463/225000 [228:39:29<211:50:10,  6.49s/it]
 48%|████▊     | 107464/225000 [228:39:32<177:28:36,  5.44s/it]
 48%|████▊     | 107465/225000 [228:39:36<161:22:13,  4.94s/it]
 48%|████▊     | 107466/225000 [228:39:38<135:06:17,  4.14s/it]
 48%|████▊     | 107467/225000 [228:39:48<190:53:52,  5.85s/it]
 48%|████▊     | 107468/225000 [228:39:51<163:27:45,  5.01s/it]
 48%|████▊     | 107469/225000 [228:39:54<136:18:23,  4.18s/it]
 48%|████▊     | 107470/225000 [228:39:56<117:26:48,  3.60s/it]
 48%|████▊     | 107471/225000 [228:39:59<111:35:55,  3.42s/it]
 48%|████▊     | 107472/225000 [228:40:05<141:36:34,  4.34s/it]
 48%|████▊     | 107473/225000 [228:40:16<204:58:02,  6.28s/it]
 48%|████▊     | 107474/225000 [228:40:24<221:25:01,  6.78s/it]
 48%|████▊     | 107475/225000 [228:40:29<205:37:00,  6.30s/it]
 48%|████▊     | 107476/225000 [228:40:35<204:03:09,  6.25s/it]
 48%|████▊     | 107477/225000 [228:40:39<179:04:44,  5.49s/it]
 48%|████▊     | 107478/225000 [228:40:44<170:07:01,  5.21s/it]
 48%|████▊     | 107479/225000 [228:40:57<251:31:22,  7.70s/it]
 48%|████▊     | 107480/225000 [228:41:02<223:32:26,  6.85s/it]
 48%|████▊     | 107481/225000 [228:41:05<187:54:38,  5.76s/it]
 48%|████▊     | 107482/225000 [228:41:14<214:12:13,  6.56s/it]
 48%|████▊     | 107483/225000 [228:41:17<187:09:05,  5.73s/it]
 48%|████▊     | 107484/225000 [228:41:21<160:44:23,  4.92s/it]
 48%|████▊     | 107485/225000 [228:41:29<193:19:26,  5.92s/it]
 48%|████▊     | 107486/225000 [228:41:31<157:45:01,  4.83s/it]
 48%|████▊     | 107487/225000 [228:41:44<236:30:30,  7.25s/it]
 48%|████▊     | 107488/225000 [228:41:54<262:42:13,  8.05s/it]
 48%|████▊     | 107489/225000 [228:42:05<289:41:44,  8.87s/it]
 48%|████▊     | 107490/225000 [228:42:10<257:12:37,  7.88s/it]
 48%|████▊     | 107491/225000 [228:42:17<247:28:40,  7.58s/it]
 48%|████▊     | 107492/225000 [228:42:23<230:46:19,  7.07s/it]
 48%|████▊     | 107493/225000 [228:42:25<184:26:14,  5.65s/it]
 48%|████▊     | 107494/225000 [228:42:29<162:06:57,  4.97s/it]
 48%|████▊     | 107495/225000 [228:42:31<136:31:48,  4.18s/it]
 48%|████▊     | 107496/225000 [228:42:33<117:27:59,  3.60s/it]
 48%|████▊     | 107497/225000 [228:42:36<113:00:29,  3.46s/it]
 48%|████▊     | 107498/225000 [228:42:45<158:45:53,  4.86s/it]
 48%|████▊     | 107499/225000 [228:42:56<218:50:22,  6.70s/it]
 48%|████▊     | 107500/225000 [228:43:03<222:34:21,  6.82s/it]
                                                               
{'loss': '0.9714', 'grad_norm': '3.288', 'learning_rate': '2.866e-05', 'epoch': '0.4778', 'num_input_tokens_seen': 383417848, 'train_runtime': '8.234e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107500/225000 [228:43:03<222:34:21,  6.82s/it]
 48%|████▊     | 107501/225000 [228:43:22<342:17:49, 10.49s/it]
 48%|████▊     | 107502/225000 [228:43:26<277:01:02,  8.49s/it]
 48%|████▊     | 107503/225000 [228:43:31<251:21:40,  7.70s/it]
 48%|████▊     | 107504/225000 [228:43:41<267:12:23,  8.19s/it]
 48%|████▊     | 107505/225000 [228:43:52<294:41:47,  9.03s/it]
 48%|████▊     | 107506/225000 [228:44:00<288:55:34,  8.85s/it]
 48%|████▊     | 107507/225000 [228:44:05<249:04:36,  7.63s/it]
 48%|████▊     | 107508/225000 [228:44:14<260:13:48,  7.97s/it]
 48%|████▊     | 107509/225000 [228:44:19<232:32:20,  7.13s/it]
 48%|████▊     | 107510/225000 [228:44:22<189:59:39,  5.82s/it]
 48%|████▊     | 107511/225000 [228:44:30<211:27:23,  6.48s/it]
 48%|████▊     | 107512/225000 [228:44:32<168:32:31,  5.16s/it]
 48%|████▊     | 107513/225000 [228:44:40<197:30:41,  6.05s/it]
 48%|████▊     | 107514/225000 [228:44:43<169:26:34,  5.19s/it]
 48%|████▊     | 107515/225000 [228:44:51<200:43:16,  6.15s/it]
 48%|████▊     | 107516/225000 [228:44:55<171:27:03,  5.25s/it]
 48%|████▊     | 107517/225000 [228:44:57<143:58:42,  4.41s/it]
 48%|████▊     | 107518/225000 [228:44:59<124:31:10,  3.82s/it]
 48%|████▊     | 107519/225000 [228:45:02<111:06:25,  3.40s/it]
 48%|████▊     | 107520/225000 [228:45:12<173:22:20,  5.31s/it]
 48%|████▊     | 107521/225000 [228:45:14<146:42:56,  4.50s/it]
 48%|████▊     | 107522/225000 [228:45:21<164:21:18,  5.04s/it]
 48%|████▊     | 107523/225000 [228:45:23<139:23:53,  4.27s/it]
 48%|████▊     | 107524/225000 [228:45:26<125:28:51,  3.85s/it]
 48%|████▊     | 107525/225000 [228:45:35<178:43:44,  5.48s/it]
 48%|████▊     | 107526/225000 [228:45:41<179:59:46,  5.52s/it]
 48%|████▊     | 107527/225000 [228:45:44<157:59:28,  4.84s/it]
 48%|████▊     | 107528/225000 [228:45:52<192:10:49,  5.89s/it]
 48%|████▊     | 107529/225000 [228:45:58<187:47:30,  5.76s/it]
 48%|████▊     | 107530/225000 [228:46:06<213:55:57,  6.56s/it]
 48%|████▊     | 107531/225000 [228:46:10<189:40:57,  5.81s/it]
 48%|████▊     | 107532/225000 [228:46:17<201:15:53,  6.17s/it]
 48%|████▊     | 107533/225000 [228:46:24<205:00:37,  6.28s/it]
 48%|████▊     | 107534/225000 [228:46:37<276:39:05,  8.48s/it]
 48%|████▊     | 107535/225000 [228:46:40<215:41:18,  6.61s/it]
 48%|████▊     | 107536/225000 [228:46:42<172:17:24,  5.28s/it]
 48%|████▊     | 107537/225000 [228:46:46<157:39:55,  4.83s/it]
 48%|████▊     | 107538/225000 [228:46:51<160:57:21,  4.93s/it]
 48%|████▊     | 107539/225000 [228:47:03<232:15:10,  7.12s/it]
 48%|████▊     | 107540/225000 [228:47:13<257:25:48,  7.89s/it]
 48%|████▊     | 107541/225000 [228:47:15<203:16:48,  6.23s/it]
 48%|████▊     | 107542/225000 [228:47:21<197:32:22,  6.05s/it]
 48%|████▊     | 107543/225000 [228:47:33<259:59:58,  7.97s/it]
 48%|████▊     | 107544/225000 [228:47:39<235:03:09,  7.20s/it]
 48%|████▊     | 107545/225000 [228:47:42<193:59:12,  5.95s/it]
 48%|████▊     | 107546/225000 [228:47:45<165:18:27,  5.07s/it]
 48%|████▊     | 107547/225000 [228:47:50<166:19:27,  5.10s/it]
 48%|████▊     | 107548/225000 [228:48:02<238:24:16,  7.31s/it]
 48%|████▊     | 107549/225000 [228:48:04<187:42:29,  5.75s/it]
 48%|████▊     | 107550/225000 [228:48:07<160:42:28,  4.93s/it]
                                                               
{'loss': '0.9254', 'grad_norm': '4.175', 'learning_rate': '2.864e-05', 'epoch': '0.478', 'num_input_tokens_seen': 383552336, 'train_runtime': '8.237e+05', 'train_tokens_per_second': '465.7'}

 48%|████▊     | 107550/225000 [228:48:07<160:42:28,  4.93s/it]
 48%|████▊     | 107551/225000 [228:48:21<241:08:42,  7.39s/it]
 48%|████▊     | 107552/225000 [228:48:24<198:45:11,  6.09s/it]
 48%|████▊     | 107553/225000 [228:48:26<161:29:07,  4.95s/it]
 48%|████▊     | 107554/225000 [228:48:30<155:04:27,  4.75s/it]
 48%|████▊     | 107555/225000 [228:48:37<172:27:01,  5.29s/it]
 48%|████▊     | 107556/225000 [228:48:39<141:45:47,  4.35s/it]
 48%|████▊     | 107557/225000 [228:48:49<201:21:40,  6.17s/it]
 48%|████▊     | 107558/225000 [228:48:54<188:09:13,  5.77s/it]
 48%|████▊     | 107559/225000 [228:48:56<153:54:30,  4.72s/it]
 48%|████▊     | 107560/225000 [228:49:03<172:01:01,  5.27s/it]
 48%|████▊     | 107561/225000 [228:49:12<212:16:15,  6.51s/it]
 48%|████▊     | 107562/225000 [228:49:15<172:22:47,  5.28s/it]
 48%|████▊     | 107563/225000 [228:49:18<147:35:22,  4.52s/it]
 48%|████▊     | 107564/225000 [228:49:21<137:29:14,  4.21s/it]
 48%|████▊     | 107565/225000 [228:49:32<203:14:52,  6.23s/it]
 48%|████▊     | 107566/225000 [228:49:42<239:42:03,  7.35s/it]
 48%|████▊     | 107567/225000 [228:49:53<271:50:48,  8.33s/it]
 48%|████▊     | 107568/225000 [228:50:00<262:16:38,  8.04s/it]
 48%|████▊     | 107569/225000 [228:50:05<233:02:08,  7.14s/it]
 48%|████▊     | 107570/225000 [228:50:17<281:45:24,  8.64s/it]
 48%|████▊     | 107571/225000 [228:50:24<261:04:47,  8.00s/it]
 48%|████▊     | 107572/225000 [228:50:26<205:23:38,  6.30s/it]
 48%|████▊     | 107573/225000 [228:50:29<173:58:01,  5.33s/it]
 48%|████▊     | 107574/225000 [228:50:32<152:58:14,  4.69s/it]
 48%|████▊     | 107575/225000 [228:50:35<137:16:12,  4.21s/it]
 48%|████▊     | 107576/225000 [228:50:45<191:03:57,  5.86s/it]
 48%|████▊     | 107577/225000 [228:50:49<171:11:14,  5.25s/it]
 48%|████▊     | 107578/225000 [228:50:54<168:33:14,  5.17s/it]
 48%|████▊     | 107579/225000 [228:51:00<174:11:06,  5.34s/it]
 48%|████▊     | 107580/225000 [228:51:06<188:33:16,  5.78s/it]
 48%|████▊     | 107581/225000 [228:51:18<246:17:24,  7.55s/it]
 48%|████▊     | 107582/225000 [228:51:26<248:06:45,  7.61s/it]
 48%|████▊     | 107583/225000 [228:51:30<216:03:51,  6.62s/it]
 48%|████▊     | 107584/225000 [228:51:32<174:42:54,  5.36s/it]
 48%|████▊     | 107585/225000 [228:51:41<202:32:49,  6.21s/it]
 48%|████▊     | 107586/225000 [228:51:44<178:10:56,  5.46s/it]
 48%|████▊     | 107587/225000 [228:51:48<162:15:29,  4.97s/it]
 48%|████▊     | 107588/225000 [228:51:56<193:45:08,  5.94s/it]
 48%|████▊     | 107589/225000 [228:52:04<209:34:31,  6.43s/it]
 48%|████▊     | 107590/225000 [228:52:06<168:32:38,  5.17s/it]
 48%|████▊     | 107591/225000 [228:52:09<142:30:06,  4.37s/it]
 48%|████▊     | 107592/225000 [228:52:12<130:39:07,  4.01s/it]
 48%|████▊     | 107593/225000 [228:52:14<115:49:33,  3.55s/it]
 48%|████▊     | 107594/225000 [228:52:18<119:32:04,  3.67s/it]
 48%|████▊     | 107595/225000 [228:52:27<172:41:21,  5.30s/it]
 48%|████▊     | 107596/225000 [228:52:30<145:37:00,  4.47s/it]
 48%|████▊     | 107597/225000 [228:52:32<126:27:07,  3.88s/it]
 48%|████▊     | 107598/225000 [228:52:35<113:10:39,  3.47s/it]
 48%|████▊     | 107599/225000 [228:52:38<109:50:04,  3.37s/it]
 48%|████▊     | 107600/225000 [228:52:41<107:30:37,  3.30s/it]
                                                               
{'loss': '0.9424', 'grad_norm': '3.4', 'learning_rate': '2.862e-05', 'epoch': '0.4782', 'num_input_tokens_seen': 383675840, 'train_runtime': '8.24e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107600/225000 [228:52:41<107:30:37,  3.30s/it]
 48%|████▊     | 107601/225000 [228:52:54<203:47:16,  6.25s/it]
 48%|████▊     | 107602/225000 [228:53:02<218:08:07,  6.69s/it]
 48%|████▊     | 107603/225000 [228:53:14<265:17:25,  8.14s/it]
 48%|████▊     | 107604/225000 [228:53:17<214:35:33,  6.58s/it]
 48%|████▊     | 107605/225000 [228:53:23<211:16:30,  6.48s/it]
 48%|████▊     | 107606/225000 [228:53:27<188:13:23,  5.77s/it]
 48%|████▊     | 107607/225000 [228:53:35<211:06:06,  6.47s/it]
 48%|████▊     | 107608/225000 [228:53:42<215:19:33,  6.60s/it]
 48%|████▊     | 107609/225000 [228:53:48<214:33:28,  6.58s/it]
 48%|████▊     | 107610/225000 [228:53:51<173:39:06,  5.33s/it]
 48%|████▊     | 107611/225000 [228:53:53<145:23:45,  4.46s/it]
 48%|████▊     | 107612/225000 [228:54:12<281:28:46,  8.63s/it]
 48%|████▊     | 107613/225000 [228:54:18<258:07:31,  7.92s/it]
 48%|████▊     | 107614/225000 [228:54:27<271:03:14,  8.31s/it]
 48%|████▊     | 107615/225000 [228:54:31<231:14:03,  7.09s/it]
 48%|████▊     | 107616/225000 [228:54:35<193:05:45,  5.92s/it]
 48%|████▊     | 107617/225000 [228:54:44<226:49:05,  6.96s/it]
 48%|████▊     | 107618/225000 [228:54:46<180:06:19,  5.52s/it]
 48%|████▊     | 107619/225000 [228:54:54<200:26:54,  6.15s/it]
 48%|████▊     | 107620/225000 [228:54:57<170:19:05,  5.22s/it]
 48%|████▊     | 107621/225000 [228:55:07<220:31:56,  6.76s/it]
 48%|████▊     | 107622/225000 [228:55:09<176:09:16,  5.40s/it]
 48%|████▊     | 107623/225000 [228:55:20<231:25:25,  7.10s/it]
 48%|████▊     | 107624/225000 [228:55:23<184:56:29,  5.67s/it]
 48%|████▊     | 107625/225000 [228:55:26<164:54:36,  5.06s/it]
 48%|████▊     | 107626/225000 [228:55:36<214:09:32,  6.57s/it]
 48%|████▊     | 107627/225000 [228:55:40<179:29:37,  5.51s/it]
 48%|████▊     | 107628/225000 [228:55:44<173:12:13,  5.31s/it]
 48%|████▊     | 107629/225000 [228:55:49<168:59:42,  5.18s/it]
 48%|████▊     | 107630/225000 [228:56:00<223:16:53,  6.85s/it]
 48%|████▊     | 107631/225000 [228:56:09<245:42:36,  7.54s/it]
 48%|████▊     | 107632/225000 [228:56:14<220:09:46,  6.75s/it]
 48%|████▊     | 107633/225000 [228:56:23<240:11:45,  7.37s/it]
 48%|████▊     | 107634/225000 [228:56:26<195:15:03,  5.99s/it]
 48%|████▊     | 107635/225000 [228:56:30<183:52:46,  5.64s/it]
 48%|████▊     | 107636/225000 [228:56:36<185:40:54,  5.70s/it]
 48%|████▊     | 107637/225000 [228:56:45<211:38:05,  6.49s/it]
 48%|████▊     | 107638/225000 [228:56:50<202:49:47,  6.22s/it]
 48%|████▊     | 107639/225000 [228:56:59<224:37:26,  6.89s/it]
 48%|████▊     | 107640/225000 [228:57:01<179:08:12,  5.49s/it]
 48%|████▊     | 107641/225000 [228:57:06<171:28:33,  5.26s/it]
 48%|████▊     | 107642/225000 [228:57:14<199:01:23,  6.11s/it]
 48%|████▊     | 107643/225000 [228:57:16<161:14:13,  4.95s/it]
 48%|████▊     | 107644/225000 [228:57:19<142:32:48,  4.37s/it]
 48%|████▊     | 107645/225000 [228:57:21<121:52:02,  3.74s/it]
 48%|████▊     | 107646/225000 [228:57:24<115:07:52,  3.53s/it]
 48%|████▊     | 107647/225000 [228:57:29<124:53:19,  3.83s/it]
 48%|████▊     | 107648/225000 [228:57:31<110:23:28,  3.39s/it]
 48%|████▊     | 107649/225000 [228:57:42<183:33:53,  5.63s/it]
 48%|████▊     | 107650/225000 [228:57:54<249:52:25,  7.67s/it]
                                                               
{'loss': '0.9397', 'grad_norm': '3.369', 'learning_rate': '2.86e-05', 'epoch': '0.4784', 'num_input_tokens_seen': 383801112, 'train_runtime': '8.243e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107650/225000 [228:57:54<249:52:25,  7.67s/it]
 48%|████▊     | 107651/225000 [228:58:00<230:00:09,  7.06s/it]
 48%|████▊     | 107652/225000 [228:58:03<191:44:40,  5.88s/it]
 48%|████▊     | 107653/225000 [228:58:06<157:41:30,  4.84s/it]
 48%|████▊     | 107654/225000 [228:58:12<177:14:19,  5.44s/it]
 48%|████▊     | 107655/225000 [228:58:18<177:08:29,  5.43s/it]
 48%|████▊     | 107656/225000 [228:58:26<206:56:13,  6.35s/it]
 48%|████▊     | 107657/225000 [228:58:29<169:03:20,  5.19s/it]
 48%|████▊     | 107658/225000 [228:58:34<169:52:56,  5.21s/it]
 48%|████▊     | 107659/225000 [228:58:43<207:39:40,  6.37s/it]
 48%|████▊     | 107660/225000 [228:58:55<261:37:11,  8.03s/it]
 48%|████▊     | 107661/225000 [228:59:04<267:30:35,  8.21s/it]
 48%|████▊     | 107662/225000 [228:59:14<290:31:50,  8.91s/it]
 48%|████▊     | 107663/225000 [228:59:21<265:25:07,  8.14s/it]
 48%|████▊     | 107664/225000 [228:59:23<210:01:33,  6.44s/it]
 48%|████▊     | 107665/225000 [228:59:32<237:24:26,  7.28s/it]
 48%|████▊     | 107666/225000 [228:59:37<208:34:52,  6.40s/it]
 48%|████▊     | 107667/225000 [228:59:43<203:02:04,  6.23s/it]
 48%|████▊     | 107668/225000 [228:59:48<191:42:21,  5.88s/it]
 48%|████▊     | 107669/225000 [228:59:50<156:56:47,  4.82s/it]
 48%|████▊     | 107670/225000 [228:59:52<134:05:05,  4.11s/it]
 48%|████▊     | 107671/225000 [228:59:58<150:10:46,  4.61s/it]
 48%|████▊     | 107672/225000 [229:00:01<135:33:28,  4.16s/it]
 48%|████▊     | 107673/225000 [229:00:13<205:38:46,  6.31s/it]
 48%|████▊     | 107674/225000 [229:00:21<222:57:38,  6.84s/it]
 48%|████▊     | 107675/225000 [229:00:23<178:41:06,  5.48s/it]
 48%|████▊     | 107676/225000 [229:00:32<215:08:05,  6.60s/it]
 48%|████▊     | 107677/225000 [229:00:44<265:49:59,  8.16s/it]
 48%|████▊     | 107678/225000 [229:00:46<207:52:17,  6.38s/it]
 48%|████▊     | 107679/225000 [229:00:51<196:04:13,  6.02s/it]
 48%|████▊     | 107680/225000 [229:00:55<172:22:04,  5.29s/it]
 48%|████▊     | 107681/225000 [229:00:59<156:36:12,  4.81s/it]
 48%|████▊     | 107682/225000 [229:01:09<206:25:16,  6.33s/it]
 48%|████▊     | 107683/225000 [229:01:17<230:48:19,  7.08s/it]
 48%|████▊     | 107684/225000 [229:01:21<199:09:01,  6.11s/it]
 48%|████▊     | 107685/225000 [229:01:25<172:18:39,  5.29s/it]
 48%|████▊     | 107686/225000 [229:01:31<185:35:12,  5.70s/it]
 48%|████▊     | 107687/225000 [229:01:35<169:46:16,  5.21s/it]
 48%|████▊     | 107688/225000 [229:01:42<184:32:11,  5.66s/it]
 48%|████▊     | 107689/225000 [229:01:50<206:27:16,  6.34s/it]
 48%|████▊     | 107690/225000 [229:02:04<277:14:45,  8.51s/it]
 48%|████▊     | 107691/225000 [229:02:12<278:56:09,  8.56s/it]
 48%|████▊     | 107692/225000 [229:02:21<278:01:17,  8.53s/it]
 48%|████▊     | 107693/225000 [229:02:23<216:01:34,  6.63s/it]
 48%|████▊     | 107694/225000 [229:02:26<180:32:27,  5.54s/it]
 48%|████▊     | 107695/225000 [229:02:28<147:50:02,  4.54s/it]
 48%|████▊     | 107696/225000 [229:02:38<196:46:16,  6.04s/it]
 48%|████▊     | 107697/225000 [229:02:41<166:33:37,  5.11s/it]
 48%|████▊     | 107698/225000 [229:02:43<145:17:21,  4.46s/it]
 48%|████▊     | 107699/225000 [229:02:53<198:17:44,  6.09s/it]
 48%|████▊     | 107700/225000 [229:02:56<167:21:21,  5.14s/it]
                                                               
{'loss': '0.9454', 'grad_norm': '3.246', 'learning_rate': '2.859e-05', 'epoch': '0.4787', 'num_input_tokens_seen': 383933984, 'train_runtime': '8.246e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107700/225000 [229:02:56<167:21:21,  5.14s/it]
 48%|████▊     | 107701/225000 [229:03:05<202:34:44,  6.22s/it]
 48%|████▊     | 107702/225000 [229:03:08<173:12:59,  5.32s/it]
 48%|████▊     | 107703/225000 [229:03:24<274:28:52,  8.42s/it]
 48%|████▊     | 107704/225000 [229:03:29<239:42:24,  7.36s/it]
 48%|████▊     | 107705/225000 [229:03:32<197:31:24,  6.06s/it]
 48%|████▊     | 107706/225000 [229:03:35<171:20:57,  5.26s/it]
 48%|████▊     | 107707/225000 [229:03:45<213:39:29,  6.56s/it]
 48%|████▊     | 107708/225000 [229:03:55<245:36:49,  7.54s/it]
 48%|████▊     | 107709/225000 [229:03:59<209:50:52,  6.44s/it]
 48%|████▊     | 107710/225000 [229:04:04<204:20:12,  6.27s/it]
 48%|████▊     | 107711/225000 [229:04:10<200:02:36,  6.14s/it]
 48%|████▊     | 107712/225000 [229:04:13<162:27:26,  4.99s/it]
 48%|████▊     | 107713/225000 [229:04:16<150:45:55,  4.63s/it]
 48%|████▊     | 107714/225000 [229:04:21<155:37:36,  4.78s/it]
 48%|████▊     | 107715/225000 [229:04:33<218:41:08,  6.71s/it]
 48%|████▊     | 107716/225000 [229:04:38<203:07:15,  6.23s/it]
 48%|████▊     | 107717/225000 [229:04:48<237:49:46,  7.30s/it]
 48%|████▊     | 107718/225000 [229:04:51<196:23:54,  6.03s/it]
 48%|████▊     | 107719/225000 [229:04:53<165:35:39,  5.08s/it]
 48%|████▊     | 107720/225000 [229:04:57<145:37:10,  4.47s/it]
 48%|████▊     | 107721/225000 [229:05:05<186:03:04,  5.71s/it]
 48%|████▊     | 107722/225000 [229:05:07<152:10:32,  4.67s/it]
 48%|████▊     | 107723/225000 [229:05:12<153:54:52,  4.72s/it]
 48%|████▊     | 107724/225000 [229:05:17<153:41:21,  4.72s/it]
 48%|████▊     | 107725/225000 [229:05:26<200:45:32,  6.16s/it]
 48%|████▊     | 107726/225000 [229:05:31<189:22:02,  5.81s/it]
 48%|████▊     | 107727/225000 [229:05:36<181:20:58,  5.57s/it]
 48%|████▊     | 107728/225000 [229:05:41<170:52:08,  5.25s/it]
 48%|████▊     | 107729/225000 [229:05:48<185:05:24,  5.68s/it]
 48%|████▊     | 107730/225000 [229:05:54<192:08:01,  5.90s/it]
 48%|████▊     | 107731/225000 [229:05:57<161:44:21,  4.97s/it]
 48%|████▊     | 107732/225000 [229:06:00<147:12:10,  4.52s/it]
 48%|████▊     | 107733/225000 [229:06:04<139:16:46,  4.28s/it]
 48%|████▊     | 107734/225000 [229:06:07<125:07:34,  3.84s/it]
 48%|████▊     | 107735/225000 [229:06:14<156:12:36,  4.80s/it]
 48%|████▊     | 107736/225000 [229:06:16<130:39:49,  4.01s/it]
 48%|████▊     | 107737/225000 [229:06:20<127:00:27,  3.90s/it]
 48%|████▊     | 107738/225000 [229:06:23<121:27:58,  3.73s/it]
 48%|████▊     | 107739/225000 [229:06:27<124:14:23,  3.81s/it]
 48%|████▊     | 107740/225000 [229:06:36<175:17:17,  5.38s/it]
 48%|████▊     | 107741/225000 [229:06:41<174:58:41,  5.37s/it]
 48%|████▊     | 107742/225000 [229:06:44<151:31:55,  4.65s/it]
 48%|████▊     | 107743/225000 [229:06:47<132:04:58,  4.06s/it]
 48%|████▊     | 107744/225000 [229:06:56<181:40:04,  5.58s/it]
 48%|████▊     | 107745/225000 [229:06:58<148:25:41,  4.56s/it]
 48%|████▊     | 107746/225000 [229:07:07<189:30:03,  5.82s/it]
 48%|████▊     | 107747/225000 [229:07:13<186:11:39,  5.72s/it]
 48%|████▊     | 107748/225000 [229:07:15<156:47:21,  4.81s/it]
 48%|████▊     | 107749/225000 [229:07:28<231:48:07,  7.12s/it]
 48%|████▊     | 107750/225000 [229:07:38<261:31:33,  8.03s/it]
                                                               
{'loss': '0.9048', 'grad_norm': '2.047', 'learning_rate': '2.857e-05', 'epoch': '0.4789', 'num_input_tokens_seen': 384051216, 'train_runtime': '8.249e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107750/225000 [229:07:38<261:31:33,  8.03s/it]
 48%|████▊     | 107751/225000 [229:07:48<276:19:27,  8.48s/it]
 48%|████▊     | 107752/225000 [229:07:53<248:51:00,  7.64s/it]
 48%|████▊     | 107753/225000 [229:07:57<212:40:42,  6.53s/it]
 48%|████▊     | 107754/225000 [229:07:59<170:10:55,  5.23s/it]
 48%|████▊     | 107755/225000 [229:08:02<141:08:44,  4.33s/it]
 48%|████▊     | 107756/225000 [229:08:10<183:07:22,  5.62s/it]
 48%|████▊     | 107757/225000 [229:08:16<182:38:00,  5.61s/it]
 48%|████▊     | 107758/225000 [229:08:21<174:03:58,  5.34s/it]
 48%|████▊     | 107759/225000 [229:08:23<146:14:49,  4.49s/it]
 48%|████▊     | 107760/225000 [229:08:26<133:57:22,  4.11s/it]
 48%|████▊     | 107761/225000 [229:08:29<119:39:05,  3.67s/it]
 48%|████▊     | 107762/225000 [229:08:38<175:14:26,  5.38s/it]
 48%|████▊     | 107763/225000 [229:08:45<187:12:28,  5.75s/it]
 48%|████▊     | 107764/225000 [229:08:54<217:22:04,  6.67s/it]
 48%|████▊     | 107765/225000 [229:08:57<187:10:46,  5.75s/it]
 48%|████▊     | 107766/225000 [229:09:03<185:44:36,  5.70s/it]
 48%|████▊     | 107767/225000 [229:09:06<163:36:31,  5.02s/it]
 48%|████▊     | 107768/225000 [229:09:09<143:36:16,  4.41s/it]
 48%|████▊     | 107769/225000 [229:09:13<137:31:25,  4.22s/it]
 48%|████▊     | 107770/225000 [229:09:28<242:42:10,  7.45s/it]
 48%|████▊     | 107771/225000 [229:09:36<244:18:22,  7.50s/it]
 48%|████▊     | 107772/225000 [229:09:45<260:02:10,  7.99s/it]
 48%|████▊     | 107773/225000 [229:09:50<230:50:37,  7.09s/it]
 48%|████▊     | 107774/225000 [229:09:57<233:35:46,  7.17s/it]
 48%|████▊     | 107775/225000 [229:10:02<209:04:31,  6.42s/it]
 48%|████▊     | 107776/225000 [229:10:04<168:05:28,  5.16s/it]
 48%|████▊     | 107777/225000 [229:10:07<149:56:25,  4.60s/it]
 48%|████▊     | 107778/225000 [229:10:18<205:04:54,  6.30s/it]
 48%|████▊     | 107779/225000 [229:10:21<172:59:40,  5.31s/it]
 48%|████▊     | 107780/225000 [229:10:23<142:34:22,  4.38s/it]
 48%|████▊     | 107781/225000 [229:10:25<121:19:31,  3.73s/it]
 48%|████▊     | 107782/225000 [229:10:35<179:34:22,  5.52s/it]
 48%|████▊     | 107783/225000 [229:10:39<172:09:24,  5.29s/it]
 48%|████▊     | 107784/225000 [229:10:42<145:57:37,  4.48s/it]
 48%|████▊     | 107785/225000 [229:10:46<137:40:30,  4.23s/it]
 48%|████▊     | 107786/225000 [229:10:55<184:24:41,  5.66s/it]
 48%|████▊     | 107787/225000 [229:10:57<153:30:11,  4.71s/it]
 48%|████▊     | 107788/225000 [229:11:01<140:28:34,  4.31s/it]
 48%|████▊     | 107789/225000 [229:11:10<190:19:58,  5.85s/it]
 48%|████▊     | 107790/225000 [229:11:12<154:51:07,  4.76s/it]
 48%|████▊     | 107791/225000 [229:11:22<208:04:06,  6.39s/it]
 48%|████▊     | 107792/225000 [229:11:30<222:31:06,  6.83s/it]
 48%|████▊     | 107793/225000 [229:11:39<245:20:16,  7.54s/it]
 48%|████▊     | 107794/225000 [229:11:42<197:06:51,  6.05s/it]
 48%|████▊     | 107795/225000 [229:11:44<159:40:38,  4.90s/it]
 48%|████▊     | 107796/225000 [229:11:49<159:37:00,  4.90s/it]
 48%|████▊     | 107797/225000 [229:11:52<143:56:22,  4.42s/it]
 48%|████▊     | 107798/225000 [229:11:55<124:02:13,  3.81s/it]
 48%|████▊     | 107799/225000 [229:12:05<189:36:43,  5.82s/it]
 48%|████▊     | 107800/225000 [229:12:10<179:22:08,  5.51s/it]
                                                               
{'loss': '0.9448', 'grad_norm': '2.343', 'learning_rate': '2.855e-05', 'epoch': '0.4791', 'num_input_tokens_seen': 384170192, 'train_runtime': '8.251e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107800/225000 [229:12:10<179:22:08,  5.51s/it]
 48%|████▊     | 107801/225000 [229:12:14<160:55:08,  4.94s/it]
 48%|████▊     | 107802/225000 [229:12:24<216:12:27,  6.64s/it]
 48%|████▊     | 107803/225000 [229:12:27<171:57:06,  5.28s/it]
 48%|████▊     | 107804/225000 [229:12:30<157:08:52,  4.83s/it]
 48%|████▊     | 107805/225000 [229:12:33<138:54:16,  4.27s/it]
 48%|████▊     | 107806/225000 [229:12:42<178:06:40,  5.47s/it]
 48%|████▊     | 107807/225000 [229:12:49<198:41:52,  6.10s/it]
 48%|████▊     | 107808/225000 [229:12:52<168:06:49,  5.16s/it]
 48%|████▊     | 107809/225000 [229:13:00<195:51:42,  6.02s/it]
 48%|████▊     | 107810/225000 [229:13:08<210:18:56,  6.46s/it]
 48%|████▊     | 107811/225000 [229:13:14<213:01:43,  6.54s/it]
 48%|████▊     | 107812/225000 [229:13:18<183:17:08,  5.63s/it]
 48%|████▊     | 107813/225000 [229:13:21<161:48:30,  4.97s/it]
 48%|████▊     | 107814/225000 [229:13:26<159:18:33,  4.89s/it]
 48%|████▊     | 107815/225000 [229:13:29<140:21:00,  4.31s/it]
 48%|████▊     | 107816/225000 [229:13:31<119:26:46,  3.67s/it]
 48%|████▊     | 107817/225000 [229:13:36<130:41:44,  4.02s/it]
 48%|████▊     | 107818/225000 [229:13:39<124:38:35,  3.83s/it]
 48%|████▊     | 107819/225000 [229:13:49<181:37:13,  5.58s/it]
 48%|████▊     | 107820/225000 [229:13:51<149:56:41,  4.61s/it]
 48%|████▊     | 107821/225000 [229:13:54<134:37:39,  4.14s/it]
 48%|████▊     | 107822/225000 [229:13:59<144:21:22,  4.43s/it]
 48%|████▊     | 107823/225000 [229:14:09<197:47:31,  6.08s/it]
 48%|████▊     | 107824/225000 [229:14:12<161:11:23,  4.95s/it]
 48%|████▊     | 107825/225000 [229:14:18<170:18:14,  5.23s/it]
 48%|████▊     | 107826/225000 [229:14:20<141:35:10,  4.35s/it]
 48%|████▊     | 107827/225000 [229:14:24<140:45:12,  4.32s/it]
 48%|████▊     | 107828/225000 [229:14:32<175:02:34,  5.38s/it]
 48%|████▊     | 107829/225000 [229:14:37<175:55:50,  5.41s/it]
 48%|████▊     | 107830/225000 [229:14:42<171:51:11,  5.28s/it]
 48%|████▊     | 107831/225000 [229:14:53<225:08:29,  6.92s/it]
 48%|████▊     | 107832/225000 [229:15:04<259:33:57,  7.98s/it]
 48%|████▊     | 107833/225000 [229:15:08<228:25:43,  7.02s/it]
 48%|████▊     | 107834/225000 [229:15:21<281:16:48,  8.64s/it]
 48%|████▊     | 107835/225000 [229:15:27<254:11:06,  7.81s/it]
 48%|████▊     | 107836/225000 [229:15:36<271:51:48,  8.35s/it]
 48%|████▊     | 107837/225000 [229:15:39<219:01:11,  6.73s/it]
 48%|████▊     | 107838/225000 [229:15:46<215:44:54,  6.63s/it]
 48%|████▊     | 107839/225000 [229:15:48<172:57:06,  5.31s/it]
 48%|████▊     | 107840/225000 [229:15:53<175:09:56,  5.38s/it]
 48%|████▊     | 107841/225000 [229:16:06<241:09:02,  7.41s/it]
 48%|████▊     | 107842/225000 [229:16:09<206:46:04,  6.35s/it]
 48%|████▊     | 107843/225000 [229:16:12<169:59:11,  5.22s/it]
 48%|████▊     | 107844/225000 [229:16:27<264:04:54,  8.11s/it]
 48%|████▊     | 107845/225000 [229:16:31<221:12:31,  6.80s/it]
 48%|████▊     | 107846/225000 [229:16:39<236:43:04,  7.27s/it]
 48%|████▊     | 107847/225000 [229:16:41<186:52:25,  5.74s/it]
 48%|████▊     | 107848/225000 [229:16:46<177:22:23,  5.45s/it]
 48%|████▊     | 107849/225000 [229:16:49<150:38:03,  4.63s/it]
 48%|████▊     | 107850/225000 [229:16:53<149:19:15,  4.59s/it]
                                                               
{'loss': '0.9325', 'grad_norm': '2.49', 'learning_rate': '2.853e-05', 'epoch': '0.4793', 'num_input_tokens_seen': 384301368, 'train_runtime': '8.254e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107850/225000 [229:16:53<149:19:15,  4.59s/it]
 48%|████▊     | 107851/225000 [229:17:04<213:20:57,  6.56s/it]
 48%|████▊     | 107852/225000 [229:17:08<186:20:19,  5.73s/it]
 48%|████▊     | 107853/225000 [229:17:13<177:31:02,  5.46s/it]
 48%|████▊     | 107854/225000 [229:17:15<146:19:47,  4.50s/it]
 48%|████▊     | 107855/225000 [229:17:22<173:14:50,  5.32s/it]
 48%|████▊     | 107856/225000 [229:17:31<207:34:28,  6.38s/it]
 48%|████▊     | 107857/225000 [229:17:41<240:59:16,  7.41s/it]
 48%|████▊     | 107858/225000 [229:17:46<216:23:00,  6.65s/it]
 48%|████▊     | 107859/225000 [229:17:58<265:38:49,  8.16s/it]
 48%|████▊     | 107860/225000 [229:18:01<222:27:49,  6.84s/it]
 48%|████▊     | 107861/225000 [229:18:05<185:59:11,  5.72s/it]
 48%|████▊     | 107862/225000 [229:18:07<154:05:13,  4.74s/it]
 48%|████▊     | 107863/225000 [229:18:15<187:06:27,  5.75s/it]
 48%|████▊     | 107864/225000 [229:18:21<185:37:25,  5.70s/it]
 48%|████▊     | 107865/225000 [229:18:28<205:51:17,  6.33s/it]
 48%|████▊     | 107866/225000 [229:18:37<226:03:58,  6.95s/it]
 48%|████▊     | 107867/225000 [229:18:43<213:21:45,  6.56s/it]
 48%|████▊     | 107868/225000 [229:18:45<171:07:47,  5.26s/it]
 48%|████▊     | 107869/225000 [229:18:47<142:12:09,  4.37s/it]
 48%|████▊     | 107870/225000 [229:18:50<132:01:44,  4.06s/it]
 48%|████▊     | 107871/225000 [229:18:54<127:03:15,  3.91s/it]
 48%|████▊     | 107872/225000 [229:19:04<189:36:34,  5.83s/it]
 48%|████▊     | 107873/225000 [229:19:07<161:59:11,  4.98s/it]
 48%|████▊     | 107874/225000 [229:19:10<142:15:45,  4.37s/it]
 48%|████▊     | 107875/225000 [229:19:14<136:22:10,  4.19s/it]
 48%|████▊     | 107876/225000 [229:19:22<172:40:15,  5.31s/it]
 48%|████▊     | 107877/225000 [229:19:27<172:05:43,  5.29s/it]
 48%|████▊     | 107878/225000 [229:19:30<150:26:38,  4.62s/it]
 48%|████▊     | 107879/225000 [229:19:44<238:12:46,  7.32s/it]
 48%|████▊     | 107880/225000 [229:19:47<195:08:51,  6.00s/it]
 48%|████▊     | 107881/225000 [229:19:50<166:41:06,  5.12s/it]
 48%|████▊     | 107882/225000 [229:20:03<242:17:49,  7.45s/it]
 48%|████▊     | 107883/225000 [229:20:06<206:45:01,  6.36s/it]
 48%|████▊     | 107884/225000 [229:20:09<166:57:31,  5.13s/it]
 48%|████▊     | 107885/225000 [229:20:11<138:58:26,  4.27s/it]
 48%|████▊     | 107886/225000 [229:20:14<126:32:20,  3.89s/it]
 48%|████▊     | 107887/225000 [229:20:16<110:31:07,  3.40s/it]
 48%|████▊     | 107888/225000 [229:20:19<106:43:52,  3.28s/it]
 48%|████▊     | 107889/225000 [229:20:29<174:09:32,  5.35s/it]
 48%|████▊     | 107890/225000 [229:20:35<173:39:00,  5.34s/it]
 48%|████▊     | 107891/225000 [229:20:46<228:56:54,  7.04s/it]
 48%|████▊     | 107892/225000 [229:20:52<223:14:02,  6.86s/it]
 48%|████▊     | 107893/225000 [229:20:58<211:13:33,  6.49s/it]
 48%|████▊     | 107894/225000 [229:21:01<175:38:06,  5.40s/it]
 48%|████▊     | 107895/225000 [229:21:03<146:33:43,  4.51s/it]
 48%|████▊     | 107896/225000 [229:21:05<124:45:32,  3.84s/it]
 48%|████▊     | 107897/225000 [229:21:09<118:21:16,  3.64s/it]
 48%|████▊     | 107898/225000 [229:21:16<156:59:11,  4.83s/it]
 48%|████▊     | 107899/225000 [229:21:19<133:48:57,  4.11s/it]
 48%|████▊     | 107900/225000 [229:21:29<193:57:12,  5.96s/it]
                                                               
{'loss': '0.9643', 'grad_norm': '2.035', 'learning_rate': '2.851e-05', 'epoch': '0.4796', 'num_input_tokens_seen': 384425288, 'train_runtime': '8.257e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107900/225000 [229:21:29<193:57:12,  5.96s/it]
 48%|████▊     | 107901/225000 [229:21:32<166:00:47,  5.10s/it]
 48%|████▊     | 107902/225000 [229:21:34<138:43:52,  4.27s/it]
 48%|████▊     | 107903/225000 [229:21:40<151:20:55,  4.65s/it]
 48%|████▊     | 107904/225000 [229:21:52<220:16:39,  6.77s/it]
 48%|████▊     | 107905/225000 [229:21:59<227:14:30,  6.99s/it]
 48%|████▊     | 107906/225000 [229:22:06<224:14:49,  6.89s/it]
 48%|████▊     | 107907/225000 [229:22:11<211:26:42,  6.50s/it]
 48%|████▊     | 107908/225000 [229:22:20<235:18:59,  7.23s/it]
 48%|████▊     | 107909/225000 [229:22:24<201:44:40,  6.20s/it]
 48%|████▊     | 107910/225000 [229:22:27<169:26:48,  5.21s/it]
 48%|████▊     | 107911/225000 [229:22:35<196:23:47,  6.04s/it]
 48%|████▊     | 107912/225000 [229:22:38<166:05:14,  5.11s/it]
 48%|████▊     | 107913/225000 [229:22:40<137:59:19,  4.24s/it]
 48%|████▊     | 107914/225000 [229:22:50<195:47:16,  6.02s/it]
 48%|████▊     | 107915/225000 [229:22:57<203:47:06,  6.27s/it]
 48%|████▊     | 107916/225000 [229:22:59<164:42:18,  5.06s/it]
 48%|████▊     | 107917/225000 [229:23:04<163:37:19,  5.03s/it]
 48%|████▊     | 107918/225000 [229:23:08<152:24:38,  4.69s/it]
 48%|████▊     | 107919/225000 [229:23:11<136:39:36,  4.20s/it]
 48%|████▊     | 107920/225000 [229:23:14<117:45:13,  3.62s/it]
 48%|████▊     | 107921/225000 [229:23:16<105:36:28,  3.25s/it]
 48%|████▊     | 107922/225000 [229:23:30<207:01:13,  6.37s/it]
 48%|████▊     | 107923/225000 [229:23:33<174:11:57,  5.36s/it]
 48%|████▊     | 107924/225000 [229:23:35<144:31:30,  4.44s/it]
 48%|████▊     | 107925/225000 [229:23:38<134:12:16,  4.13s/it]
 48%|████▊     | 107926/225000 [229:23:43<142:01:38,  4.37s/it]
 48%|████▊     | 107927/225000 [229:23:54<202:03:15,  6.21s/it]
 48%|████▊     | 107928/225000 [229:23:59<196:46:30,  6.05s/it]
 48%|████▊     | 107929/225000 [229:24:06<201:05:22,  6.18s/it]
 48%|████▊     | 107930/225000 [229:24:09<172:13:58,  5.30s/it]
 48%|████▊     | 107931/225000 [229:24:20<229:20:18,  7.05s/it]
 48%|████▊     | 107932/225000 [229:24:24<197:22:21,  6.07s/it]
 48%|████▊     | 107933/225000 [229:24:30<197:20:20,  6.07s/it]
 48%|████▊     | 107934/225000 [229:24:33<163:59:06,  5.04s/it]
 48%|████▊     | 107935/225000 [229:24:35<137:51:56,  4.24s/it]
 48%|████▊     | 107936/225000 [229:24:38<127:36:51,  3.92s/it]
 48%|████▊     | 107937/225000 [229:24:41<112:15:24,  3.45s/it]
 48%|████▊     | 107938/225000 [229:24:44<112:46:59,  3.47s/it]
 48%|████▊     | 107939/225000 [229:24:47<109:39:17,  3.37s/it]
 48%|████▊     | 107940/225000 [229:24:53<127:53:41,  3.93s/it]
 48%|████▊     | 107941/225000 [229:25:05<208:44:13,  6.42s/it]
 48%|████▊     | 107942/225000 [229:25:15<246:32:48,  7.58s/it]
 48%|████▊     | 107943/225000 [229:25:23<254:29:04,  7.83s/it]
 48%|████▊     | 107944/225000 [229:25:28<222:23:27,  6.84s/it]
 48%|████▊     | 107945/225000 [229:25:31<185:53:49,  5.72s/it]
 48%|████▊     | 107946/225000 [229:25:35<164:10:30,  5.05s/it]
 48%|████▊     | 107947/225000 [229:25:44<202:07:15,  6.22s/it]
 48%|████▊     | 107948/225000 [229:25:48<186:59:22,  5.75s/it]
 48%|████▊     | 107949/225000 [229:25:53<181:10:03,  5.57s/it]
 48%|████▊     | 107950/225000 [229:26:04<227:47:52,  7.01s/it]
                                                               
{'loss': '0.9506', 'grad_norm': '2.098', 'learning_rate': '2.85e-05', 'epoch': '0.4798', 'num_input_tokens_seen': 384546712, 'train_runtime': '8.26e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 107950/225000 [229:26:04<227:47:52,  7.01s/it]
 48%|████▊     | 107951/225000 [229:26:07<188:57:19,  5.81s/it]
 48%|████▊     | 107952/225000 [229:26:09<154:36:28,  4.76s/it]
 48%|████▊     | 107953/225000 [229:26:15<164:16:58,  5.05s/it]
 48%|████▊     | 107954/225000 [229:26:28<245:50:10,  7.56s/it]
 48%|████▊     | 107955/225000 [229:26:31<201:44:06,  6.20s/it]
 48%|████▊     | 107956/225000 [229:26:34<164:31:00,  5.06s/it]
 48%|████▊     | 107957/225000 [229:26:36<137:50:21,  4.24s/it]
 48%|████▊     | 107958/225000 [229:26:43<161:56:28,  4.98s/it]
 48%|████▊     | 107959/225000 [229:26:49<171:55:47,  5.29s/it]
 48%|████▊     | 107960/225000 [229:26:51<143:38:04,  4.42s/it]
 48%|████▊     | 107961/225000 [229:26:54<130:38:40,  4.02s/it]
 48%|████▊     | 107962/225000 [229:27:03<181:14:26,  5.57s/it]
 48%|████▊     | 107963/225000 [229:27:06<156:24:28,  4.81s/it]
 48%|████▊     | 107964/225000 [229:27:15<191:59:05,  5.91s/it]
 48%|████▊     | 107965/225000 [229:27:18<163:59:00,  5.04s/it]
 48%|████▊     | 107966/225000 [229:27:24<171:53:56,  5.29s/it]
 48%|████▊     | 107967/225000 [229:27:34<220:27:03,  6.78s/it]
 48%|████▊     | 107968/225000 [229:27:39<207:38:44,  6.39s/it]
 48%|████▊     | 107969/225000 [229:27:44<189:05:54,  5.82s/it]
 48%|████▊     | 107970/225000 [229:27:47<161:58:47,  4.98s/it]
 48%|████▊     | 107971/225000 [229:27:56<201:41:07,  6.20s/it]
 48%|████▊     | 107972/225000 [229:28:04<214:58:32,  6.61s/it]
 48%|████▊     | 107973/225000 [229:28:07<186:47:24,  5.75s/it]
 48%|████▊     | 107974/225000 [229:28:12<180:15:59,  5.55s/it]
 48%|████▊     | 107975/225000 [229:28:15<155:47:56,  4.79s/it]
 48%|████▊     | 107976/225000 [229:28:20<158:34:34,  4.88s/it]
 48%|████▊     | 107977/225000 [229:28:23<132:53:09,  4.09s/it]
 48%|████▊     | 107978/225000 [229:28:28<144:02:14,  4.43s/it]
 48%|████▊     | 107979/225000 [229:28:40<216:10:24,  6.65s/it]
 48%|████▊     | 107980/225000 [229:28:43<181:02:56,  5.57s/it]
 48%|████▊     | 107981/225000 [229:28:46<155:45:27,  4.79s/it]
 48%|████▊     | 107982/225000 [229:28:51<154:58:53,  4.77s/it]
 48%|████▊     | 107983/225000 [229:28:55<153:27:04,  4.72s/it]
 48%|████▊     | 107984/225000 [229:29:02<177:55:43,  5.47s/it]
 48%|████▊     | 107985/225000 [229:29:05<154:21:05,  4.75s/it]
 48%|████▊     | 107986/225000 [229:29:10<155:30:09,  4.78s/it]
 48%|████▊     | 107987/225000 [229:29:16<162:28:06,  5.00s/it]
 48%|████▊     | 107988/225000 [229:29:22<169:44:53,  5.22s/it]
 48%|████▊     | 107989/225000 [229:29:28<179:04:46,  5.51s/it]
 48%|████▊     | 107990/225000 [229:29:37<214:35:13,  6.60s/it]
 48%|████▊     | 107991/225000 [229:29:40<179:22:06,  5.52s/it]
 48%|████▊     | 107992/225000 [229:29:55<271:32:53,  8.35s/it]
 48%|████▊     | 107993/225000 [229:30:03<265:34:25,  8.17s/it]
 48%|████▊     | 107994/225000 [229:30:14<299:30:16,  9.22s/it]
 48%|████▊     | 107995/225000 [229:30:25<312:30:19,  9.62s/it]
 48%|████▊     | 107996/225000 [229:30:29<257:46:51,  7.93s/it]
 48%|████▊     | 107997/225000 [229:30:39<278:16:08,  8.56s/it]
 48%|████▊     | 107998/225000 [229:30:43<233:41:59,  7.19s/it]
 48%|████▊     | 107999/225000 [229:30:46<190:43:23,  5.87s/it]
 48%|████▊     | 108000/225000 [229:30:55<226:16:35,  6.96s/it]
                                                               
{'loss': '0.9238', 'grad_norm': '2.012', 'learning_rate': '2.848e-05', 'epoch': '0.48', 'num_input_tokens_seen': 384681336, 'train_runtime': '8.263e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108000/225000 [229:30:55<226:16:35,  6.96s/it]
 48%|████▊     | 108001/225000 [229:31:03<237:27:16,  7.31s/it]
 48%|████▊     | 108002/225000 [229:31:09<219:51:17,  6.76s/it]
 48%|████▊     | 108003/225000 [229:31:19<251:54:56,  7.75s/it]
 48%|████▊     | 108004/225000 [229:31:21<196:57:38,  6.06s/it]
 48%|████▊     | 108005/225000 [229:31:28<210:27:48,  6.48s/it]
 48%|████▊     | 108006/225000 [229:31:31<169:25:21,  5.21s/it]
 48%|████▊     | 108007/225000 [229:31:33<141:01:47,  4.34s/it]
 48%|████▊     | 108008/225000 [229:31:35<120:45:47,  3.72s/it]
 48%|████▊     | 108009/225000 [229:31:41<141:52:36,  4.37s/it]
 48%|████▊     | 108010/225000 [229:31:51<194:12:00,  5.98s/it]
 48%|████▊     | 108011/225000 [229:32:00<227:02:32,  6.99s/it]
 48%|████▊     | 108012/225000 [229:32:02<180:47:31,  5.56s/it]
 48%|████▊     | 108013/225000 [229:32:08<181:38:25,  5.59s/it]
 48%|████▊     | 108014/225000 [229:32:12<163:52:47,  5.04s/it]
 48%|████▊     | 108015/225000 [229:32:16<152:28:11,  4.69s/it]
 48%|████▊     | 108016/225000 [229:32:19<139:20:02,  4.29s/it]
 48%|████▊     | 108017/225000 [229:32:23<136:16:27,  4.19s/it]
 48%|████▊     | 108018/225000 [229:32:32<188:03:25,  5.79s/it]
 48%|████▊     | 108019/225000 [229:32:35<160:04:26,  4.93s/it]
 48%|████▊     | 108020/225000 [229:32:38<140:48:49,  4.33s/it]
 48%|████▊     | 108021/225000 [229:32:43<147:33:24,  4.54s/it]
 48%|████▊     | 108022/225000 [229:32:46<126:02:31,  3.88s/it]
 48%|████▊     | 108023/225000 [229:33:00<223:57:55,  6.89s/it]
 48%|████▊     | 108024/225000 [229:33:07<225:30:23,  6.94s/it]
 48%|████▊     | 108025/225000 [229:33:09<179:29:19,  5.52s/it]
 48%|████▊     | 108026/225000 [229:33:11<147:27:36,  4.54s/it]
 48%|████▊     | 108027/225000 [229:33:13<126:06:48,  3.88s/it]
 48%|████▊     | 108028/225000 [229:33:28<225:46:07,  6.95s/it]
 48%|████▊     | 108029/225000 [229:33:39<272:02:03,  8.37s/it]
 48%|████▊     | 108030/225000 [229:33:44<237:12:34,  7.30s/it]
 48%|████▊     | 108031/225000 [229:33:50<223:25:40,  6.88s/it]
 48%|████▊     | 108032/225000 [229:33:53<185:43:34,  5.72s/it]
 48%|████▊     | 108033/225000 [229:34:00<199:14:07,  6.13s/it]
 48%|████▊     | 108034/225000 [229:34:04<175:28:57,  5.40s/it]
 48%|████▊     | 108035/225000 [229:34:10<181:13:07,  5.58s/it]
 48%|████▊     | 108036/225000 [229:34:19<219:23:39,  6.75s/it]
 48%|████▊     | 108037/225000 [229:34:25<205:50:05,  6.34s/it]
 48%|████▊     | 108038/225000 [229:34:29<183:42:02,  5.65s/it]
 48%|████▊     | 108039/225000 [229:34:38<216:00:47,  6.65s/it]
 48%|████▊     | 108040/225000 [229:34:40<171:38:17,  5.28s/it]
 48%|████▊     | 108041/225000 [229:34:45<170:08:36,  5.24s/it]
 48%|████▊     | 108042/225000 [229:34:55<222:29:23,  6.85s/it]
 48%|████▊     | 108043/225000 [229:34:58<184:19:15,  5.67s/it]
 48%|████▊     | 108044/225000 [229:35:01<158:34:36,  4.88s/it]
 48%|████▊     | 108045/225000 [229:35:05<147:33:03,  4.54s/it]
 48%|████▊     | 108046/225000 [229:35:07<124:42:58,  3.84s/it]
 48%|████▊     | 108047/225000 [229:35:13<137:46:36,  4.24s/it]
 48%|████▊     | 108048/225000 [229:35:23<195:07:08,  6.01s/it]
 48%|████▊     | 108049/225000 [229:35:26<171:27:11,  5.28s/it]
 48%|████▊     | 108050/225000 [229:35:29<150:34:13,  4.63s/it]
                                                               
{'loss': '0.9261', 'grad_norm': '3.329', 'learning_rate': '2.846e-05', 'epoch': '0.4802', 'num_input_tokens_seen': 384810560, 'train_runtime': '8.265e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108050/225000 [229:35:29<150:34:13,  4.63s/it]
 48%|████▊     | 108051/225000 [229:35:32<130:04:02,  4.00s/it]
 48%|████▊     | 108052/225000 [229:35:37<136:24:13,  4.20s/it]
 48%|████▊     | 108053/225000 [229:35:42<146:46:21,  4.52s/it]
 48%|████▊     | 108054/225000 [229:35:47<149:33:40,  4.60s/it]
 48%|████▊     | 108055/225000 [229:35:57<206:47:15,  6.37s/it]
 48%|████▊     | 108056/225000 [229:36:12<291:31:51,  8.97s/it]
 48%|████▊     | 108057/225000 [229:36:20<281:47:47,  8.67s/it]
 48%|████▊     | 108058/225000 [229:36:24<234:23:02,  7.22s/it]
 48%|████▊     | 108059/225000 [229:36:26<187:00:43,  5.76s/it]
 48%|████▊     | 108060/225000 [229:36:29<153:24:49,  4.72s/it]
 48%|████▊     | 108061/225000 [229:36:32<144:33:29,  4.45s/it]
 48%|████▊     | 108062/225000 [229:36:36<131:20:00,  4.04s/it]
 48%|████▊     | 108063/225000 [229:36:38<118:49:30,  3.66s/it]
 48%|████▊     | 108064/225000 [229:36:42<121:48:56,  3.75s/it]
 48%|████▊     | 108065/225000 [229:36:46<123:46:05,  3.81s/it]
 48%|████▊     | 108066/225000 [229:36:49<110:54:18,  3.41s/it]
 48%|████▊     | 108067/225000 [229:36:51<101:54:22,  3.14s/it]
 48%|████▊     | 108068/225000 [229:37:07<227:06:45,  6.99s/it]
 48%|████▊     | 108069/225000 [229:37:14<229:16:25,  7.06s/it]
 48%|████▊     | 108070/225000 [229:37:23<241:48:13,  7.44s/it]
 48%|████▊     | 108071/225000 [229:37:28<217:04:11,  6.68s/it]
 48%|████▊     | 108072/225000 [229:37:31<181:24:38,  5.59s/it]
 48%|████▊     | 108073/225000 [229:37:39<207:09:29,  6.38s/it]
 48%|████▊     | 108074/225000 [229:37:51<259:29:16,  7.99s/it]
 48%|████▊     | 108075/225000 [229:37:53<207:06:06,  6.38s/it]
 48%|████▊     | 108076/225000 [229:37:56<171:32:45,  5.28s/it]
 48%|████▊     | 108077/225000 [229:37:59<147:09:15,  4.53s/it]
 48%|████▊     | 108078/225000 [229:38:02<129:58:05,  4.00s/it]
 48%|████▊     | 108079/225000 [229:38:12<195:42:54,  6.03s/it]
 48%|████▊     | 108080/225000 [229:38:22<229:56:11,  7.08s/it]
 48%|████▊     | 108081/225000 [229:38:34<276:45:12,  8.52s/it]
 48%|████▊     | 108082/225000 [229:38:37<220:44:06,  6.80s/it]
 48%|████▊     | 108083/225000 [229:38:39<176:40:46,  5.44s/it]
 48%|████▊     | 108084/225000 [229:38:44<171:46:29,  5.29s/it]
 48%|████▊     | 108085/225000 [229:38:47<149:28:48,  4.60s/it]
 48%|████▊     | 108086/225000 [229:38:55<189:03:26,  5.82s/it]
 48%|████▊     | 108087/225000 [229:38:58<159:15:17,  4.90s/it]
 48%|████▊     | 108088/225000 [229:39:00<133:41:20,  4.12s/it]
 48%|████▊     | 108089/225000 [229:39:03<114:51:19,  3.54s/it]
 48%|████▊     | 108090/225000 [229:39:05<101:48:34,  3.14s/it]
 48%|████▊     | 108091/225000 [229:39:10<118:28:13,  3.65s/it]
 48%|████▊     | 108092/225000 [229:39:13<115:20:47,  3.55s/it]
 48%|████▊     | 108093/225000 [229:39:22<172:41:48,  5.32s/it]
 48%|████▊     | 108094/225000 [229:39:31<201:31:28,  6.21s/it]
 48%|████▊     | 108095/225000 [229:39:35<179:16:54,  5.52s/it]
 48%|████▊     | 108096/225000 [229:39:37<151:37:02,  4.67s/it]
 48%|████▊     | 108097/225000 [229:39:46<187:43:16,  5.78s/it]
 48%|████▊     | 108098/225000 [229:39:49<162:21:42,  5.00s/it]
 48%|████▊     | 108099/225000 [229:39:55<172:35:32,  5.32s/it]
 48%|████▊     | 108100/225000 [229:39:57<144:52:25,  4.46s/it]
                                                               
{'loss': '0.9098', 'grad_norm': '5.351', 'learning_rate': '2.844e-05', 'epoch': '0.4804', 'num_input_tokens_seen': 384925072, 'train_runtime': '8.268e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108100/225000 [229:39:57<144:52:25,  4.46s/it]
 48%|████▊     | 108101/225000 [229:40:01<132:29:20,  4.08s/it]
 48%|████▊     | 108102/225000 [229:40:03<121:20:15,  3.74s/it]
 48%|████▊     | 108103/225000 [229:40:13<179:24:44,  5.53s/it]
 48%|████▊     | 108104/225000 [229:40:22<206:53:49,  6.37s/it]
 48%|████▊     | 108105/225000 [229:40:25<176:03:33,  5.42s/it]
 48%|████▊     | 108106/225000 [229:40:33<199:27:43,  6.14s/it]
 48%|████▊     | 108107/225000 [229:40:37<182:26:59,  5.62s/it]
 48%|████▊     | 108108/225000 [229:40:46<218:11:48,  6.72s/it]
 48%|████▊     | 108109/225000 [229:40:51<198:30:30,  6.11s/it]
 48%|████▊     | 108110/225000 [229:40:54<171:10:55,  5.27s/it]
 48%|████▊     | 108111/225000 [229:40:58<157:43:39,  4.86s/it]
 48%|████▊     | 108112/225000 [229:41:07<193:29:13,  5.96s/it]
 48%|████▊     | 108113/225000 [229:41:16<222:05:01,  6.84s/it]
 48%|████▊     | 108114/225000 [229:41:19<184:05:59,  5.67s/it]
 48%|████▊     | 108115/225000 [229:41:21<157:34:17,  4.85s/it]
 48%|████▊     | 108116/225000 [229:41:31<205:21:49,  6.33s/it]
 48%|████▊     | 108117/225000 [229:41:36<190:56:19,  5.88s/it]
 48%|████▊     | 108118/225000 [229:41:42<189:47:12,  5.85s/it]
 48%|████▊     | 108119/225000 [229:41:44<154:41:49,  4.76s/it]
 48%|████▊     | 108120/225000 [229:41:47<137:33:37,  4.24s/it]
 48%|████▊     | 108121/225000 [229:41:52<148:51:34,  4.59s/it]
 48%|████▊     | 108122/225000 [229:41:55<133:00:50,  4.10s/it]
 48%|████▊     | 108123/225000 [229:42:04<180:06:12,  5.55s/it]
 48%|████▊     | 108124/225000 [229:42:07<150:26:41,  4.63s/it]
 48%|████▊     | 108125/225000 [229:42:10<139:35:34,  4.30s/it]
 48%|████▊     | 108126/225000 [229:42:13<121:33:35,  3.74s/it]
 48%|████▊     | 108127/225000 [229:42:17<123:35:24,  3.81s/it]
 48%|████▊     | 108128/225000 [229:42:25<165:14:57,  5.09s/it]
 48%|████▊     | 108129/225000 [229:42:27<140:00:55,  4.31s/it]
 48%|████▊     | 108130/225000 [229:42:31<135:39:38,  4.18s/it]
 48%|████▊     | 108131/225000 [229:42:42<196:59:50,  6.07s/it]
 48%|████▊     | 108132/225000 [229:42:47<187:31:42,  5.78s/it]
 48%|████▊     | 108133/225000 [229:42:55<207:00:08,  6.38s/it]
 48%|████▊     | 108134/225000 [229:42:59<183:06:43,  5.64s/it]
 48%|████▊     | 108135/225000 [229:43:02<162:57:49,  5.02s/it]
 48%|████▊     | 108136/225000 [229:43:05<141:46:19,  4.37s/it]
 48%|████▊     | 108137/225000 [229:43:08<127:28:16,  3.93s/it]
 48%|████▊     | 108138/225000 [229:43:19<194:43:03,  6.00s/it]
 48%|████▊     | 108139/225000 [229:43:21<157:20:08,  4.85s/it]
 48%|████▊     | 108140/225000 [229:43:23<132:09:40,  4.07s/it]
 48%|████▊     | 108141/225000 [229:43:28<143:16:25,  4.41s/it]
 48%|████▊     | 108142/225000 [229:43:38<197:45:56,  6.09s/it]
 48%|████▊     | 108143/225000 [229:43:42<179:02:54,  5.52s/it]
 48%|████▊     | 108144/225000 [229:43:52<213:56:51,  6.59s/it]
 48%|████▊     | 108145/225000 [229:43:56<194:18:02,  5.99s/it]
 48%|████▊     | 108146/225000 [229:44:03<200:31:11,  6.18s/it]
 48%|████▊     | 108147/225000 [229:44:06<168:21:00,  5.19s/it]
 48%|████▊     | 108148/225000 [229:44:10<164:34:07,  5.07s/it]
 48%|████▊     | 108149/225000 [229:44:14<152:06:44,  4.69s/it]
 48%|████▊     | 108150/225000 [229:44:24<197:56:12,  6.10s/it]
                                                               
{'loss': '0.9113', 'grad_norm': '1.915', 'learning_rate': '2.842e-05', 'epoch': '0.4807', 'num_input_tokens_seen': 385043224, 'train_runtime': '8.271e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108150/225000 [229:44:24<197:56:12,  6.10s/it]
 48%|████▊     | 108151/225000 [229:44:32<216:10:30,  6.66s/it]
 48%|████▊     | 108152/225000 [229:44:41<242:36:57,  7.47s/it]
 48%|████▊     | 108153/225000 [229:44:44<202:10:19,  6.23s/it]
 48%|████▊     | 108154/225000 [229:44:47<163:17:13,  5.03s/it]
 48%|████▊     | 108155/225000 [229:44:49<142:37:23,  4.39s/it]
 48%|████▊     | 108156/225000 [229:44:52<128:22:57,  3.96s/it]
 48%|████▊     | 108157/225000 [229:45:00<166:58:46,  5.14s/it]
 48%|████▊     | 108158/225000 [229:45:07<180:17:29,  5.55s/it]
 48%|████▊     | 108159/225000 [229:45:10<158:09:11,  4.87s/it]
 48%|████▊     | 108160/225000 [229:45:14<153:00:10,  4.71s/it]
 48%|████▊     | 108161/225000 [229:45:22<179:20:59,  5.53s/it]
 48%|████▊     | 108162/225000 [229:45:25<155:03:43,  4.78s/it]
 48%|████▊     | 108163/225000 [229:45:29<146:54:32,  4.53s/it]
 48%|████▊     | 108164/225000 [229:45:38<189:51:25,  5.85s/it]
 48%|████▊     | 108165/225000 [229:45:43<184:39:37,  5.69s/it]
 48%|████▊     | 108166/225000 [229:45:47<170:37:33,  5.26s/it]
 48%|████▊     | 108167/225000 [229:45:59<234:43:34,  7.23s/it]
 48%|████▊     | 108168/225000 [229:46:01<185:39:10,  5.72s/it]
 48%|████▊     | 108169/225000 [229:46:06<176:19:11,  5.43s/it]
 48%|████▊     | 108170/225000 [229:46:09<153:07:08,  4.72s/it]
 48%|████▊     | 108171/225000 [229:46:12<138:18:31,  4.26s/it]
 48%|████▊     | 108172/225000 [229:46:24<211:00:25,  6.50s/it]
 48%|████▊     | 108173/225000 [229:46:27<176:31:15,  5.44s/it]
 48%|████▊     | 108174/225000 [229:46:33<176:41:35,  5.44s/it]
 48%|████▊     | 108175/225000 [229:46:35<145:55:19,  4.50s/it]
 48%|████▊     | 108176/225000 [229:46:40<151:08:58,  4.66s/it]
 48%|████▊     | 108177/225000 [229:46:43<134:54:31,  4.16s/it]
 48%|████▊     | 108178/225000 [229:46:53<189:37:38,  5.84s/it]
 48%|████▊     | 108179/225000 [229:46:56<161:27:08,  4.98s/it]
 48%|████▊     | 108180/225000 [229:46:58<133:48:10,  4.12s/it]
 48%|████▊     | 108181/225000 [229:47:09<202:30:12,  6.24s/it]
 48%|████▊     | 108182/225000 [229:47:14<191:45:42,  5.91s/it]
 48%|████▊     | 108183/225000 [229:47:18<171:40:21,  5.29s/it]
 48%|████▊     | 108184/225000 [229:47:34<274:55:18,  8.47s/it]
 48%|████▊     | 108185/225000 [229:47:38<232:18:54,  7.16s/it]
 48%|████▊     | 108186/225000 [229:47:46<239:06:19,  7.37s/it]
 48%|████▊     | 108187/225000 [229:47:48<192:46:24,  5.94s/it]
 48%|████▊     | 108188/225000 [229:47:57<217:23:08,  6.70s/it]
 48%|████▊     | 108189/225000 [229:48:00<181:04:07,  5.58s/it]
 48%|████▊     | 108190/225000 [229:48:03<155:38:14,  4.80s/it]
 48%|████▊     | 108191/225000 [229:48:16<239:37:35,  7.39s/it]
 48%|████▊     | 108192/225000 [229:48:18<189:15:18,  5.83s/it]
 48%|████▊     | 108193/225000 [229:48:21<153:50:29,  4.74s/it]
 48%|████▊     | 108194/225000 [229:48:24<136:42:42,  4.21s/it]
 48%|████▊     | 108195/225000 [229:48:28<138:04:11,  4.26s/it]
 48%|████▊     | 108196/225000 [229:48:35<167:22:43,  5.16s/it]
 48%|████▊     | 108197/225000 [229:48:39<154:29:28,  4.76s/it]
 48%|████▊     | 108198/225000 [229:48:42<137:33:40,  4.24s/it]
 48%|████▊     | 108199/225000 [229:48:46<133:30:39,  4.12s/it]
 48%|████▊     | 108200/225000 [229:48:48<115:17:51,  3.55s/it]
                                                               
{'loss': '0.9592', 'grad_norm': '6.538', 'learning_rate': '2.841e-05', 'epoch': '0.4809', 'num_input_tokens_seen': 385166800, 'train_runtime': '8.273e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108200/225000 [229:48:48<115:17:51,  3.55s/it]
 48%|████▊     | 108201/225000 [229:48:57<162:55:07,  5.02s/it]
 48%|████▊     | 108202/225000 [229:49:05<194:43:14,  6.00s/it]
 48%|████▊     | 108203/225000 [229:49:08<170:26:12,  5.25s/it]
 48%|████▊     | 108204/225000 [229:49:13<165:34:29,  5.10s/it]
 48%|████▊     | 108205/225000 [229:49:17<149:56:19,  4.62s/it]
 48%|████▊     | 108206/225000 [229:49:20<134:15:15,  4.14s/it]
 48%|████▊     | 108207/225000 [229:49:23<129:39:39,  4.00s/it]
 48%|████▊     | 108208/225000 [229:49:36<217:28:03,  6.70s/it]
 48%|████▊     | 108209/225000 [229:49:39<174:10:44,  5.37s/it]
 48%|████▊     | 108210/225000 [229:49:42<155:45:43,  4.80s/it]
 48%|████▊     | 108211/225000 [229:49:52<209:54:50,  6.47s/it]
 48%|████▊     | 108212/225000 [229:50:00<222:35:54,  6.86s/it]
 48%|████▊     | 108213/225000 [229:50:03<185:23:21,  5.71s/it]
 48%|████▊     | 108214/225000 [229:50:08<178:29:10,  5.50s/it]
 48%|████▊     | 108215/225000 [229:50:12<162:32:12,  5.01s/it]
 48%|████▊     | 108216/225000 [229:50:28<271:14:27,  8.36s/it]
 48%|████▊     | 108217/225000 [229:50:34<248:40:19,  7.67s/it]
 48%|████▊     | 108218/225000 [229:50:37<203:27:01,  6.27s/it]
 48%|████▊     | 108219/225000 [229:50:42<189:25:53,  5.84s/it]
 48%|████▊     | 108220/225000 [229:50:47<179:07:04,  5.52s/it]
 48%|████▊     | 108221/225000 [229:50:50<152:23:25,  4.70s/it]
 48%|████▊     | 108222/225000 [229:51:00<204:04:56,  6.29s/it]
 48%|████▊     | 108223/225000 [229:51:07<217:09:02,  6.69s/it]
 48%|████▊     | 108224/225000 [229:51:14<217:08:12,  6.69s/it]
 48%|████▊     | 108225/225000 [229:51:16<172:58:53,  5.33s/it]
 48%|████▊     | 108226/225000 [229:51:20<157:30:46,  4.86s/it]
 48%|████▊     | 108227/225000 [229:51:25<155:38:58,  4.80s/it]
 48%|████▊     | 108228/225000 [229:51:29<152:41:58,  4.71s/it]
 48%|████▊     | 108229/225000 [229:51:33<149:57:39,  4.62s/it]
 48%|████▊     | 108230/225000 [229:51:44<205:02:24,  6.32s/it]
 48%|████▊     | 108231/225000 [229:51:49<191:02:38,  5.89s/it]
 48%|████▊     | 108232/225000 [229:51:58<227:17:20,  7.01s/it]
 48%|████▊     | 108233/225000 [229:52:02<191:02:31,  5.89s/it]
 48%|████▊     | 108234/225000 [229:52:09<206:29:28,  6.37s/it]
 48%|████▊     | 108235/225000 [229:52:13<178:20:25,  5.50s/it]
 48%|████▊     | 108236/225000 [229:52:16<158:14:53,  4.88s/it]
 48%|████▊     | 108237/225000 [229:52:24<184:46:11,  5.70s/it]
 48%|████▊     | 108238/225000 [229:52:30<192:35:06,  5.94s/it]
 48%|████▊     | 108239/225000 [229:52:39<223:01:40,  6.88s/it]
 48%|████▊     | 108240/225000 [229:52:48<245:36:50,  7.57s/it]
 48%|████▊     | 108241/225000 [229:52:53<220:57:35,  6.81s/it]
 48%|████▊     | 108242/225000 [229:52:56<179:05:46,  5.52s/it]
 48%|████▊     | 108243/225000 [229:53:01<178:46:31,  5.51s/it]
 48%|████▊     | 108244/225000 [229:53:05<163:21:37,  5.04s/it]
 48%|████▊     | 108245/225000 [229:53:09<147:45:57,  4.56s/it]
 48%|████▊     | 108246/225000 [229:53:13<141:33:03,  4.36s/it]
 48%|████▊     | 108247/225000 [229:53:19<159:12:48,  4.91s/it]
 48%|████▊     | 108248/225000 [229:53:29<211:43:58,  6.53s/it]
 48%|████▊     | 108249/225000 [229:53:31<170:06:35,  5.25s/it]
 48%|████▊     | 108250/225000 [229:53:35<149:28:54,  4.61s/it]
                                                               
{'loss': '0.9722', 'grad_norm': '2.952', 'learning_rate': '2.839e-05', 'epoch': '0.4811', 'num_input_tokens_seen': 385300224, 'train_runtime': '8.276e+05', 'train_tokens_per_second': '465.6'}

 48%|████▊     | 108250/225000 [229:53:35<149:28:54,  4.61s/it]
 48%|████▊     | 108251/225000 [229:53:40<160:07:43,  4.94s/it]
 48%|████▊     | 108252/225000 [229:53:55<251:48:45,  7.76s/it]
 48%|████▊     | 108253/225000 [229:54:00<225:59:40,  6.97s/it]
 48%|████▊     | 108254/225000 [229:54:03<190:33:11,  5.88s/it]
 48%|████▊     | 108255/225000 [229:54:14<242:06:33,  7.47s/it]
 48%|████▊     | 108256/225000 [229:54:18<206:18:39,  6.36s/it]
 48%|████▊     | 108257/225000 [229:54:23<191:56:08,  5.92s/it]
 48%|████▊     | 108258/225000 [229:54:25<155:40:53,  4.80s/it]
 48%|████▊     | 108259/225000 [229:54:35<201:46:56,  6.22s/it]
 48%|████▊     | 108260/225000 [229:54:37<163:07:16,  5.03s/it]
 48%|████▊     | 108261/225000 [229:54:43<176:52:14,  5.45s/it]
 48%|████▊     | 108262/225000 [229:54:52<209:47:27,  6.47s/it]
 48%|████▊     | 108263/225000 [229:55:02<245:29:22,  7.57s/it]
 48%|████▊     | 108264/225000 [229:55:08<224:35:45,  6.93s/it]
 48%|████▊     | 108265/225000 [229:55:14<222:05:36,  6.85s/it]
 48%|████▊     | 108266/225000 [229:55:19<204:06:13,  6.29s/it]
 48%|████▊     | 108267/225000 [229:55:22<172:55:14,  5.33s/it]
 48%|████▊     | 108268/225000 [229:55:25<143:38:18,  4.43s/it]
 48%|████▊     | 108269/225000 [229:55:33<179:32:04,  5.54s/it]
 48%|████▊     | 108270/225000 [229:55:44<237:21:38,  7.32s/it]
 48%|████▊     | 108271/225000 [229:55:47<195:34:55,  6.03s/it]
 48%|████▊     | 108272/225000 [229:55:55<209:37:53,  6.47s/it]
 48%|████▊     | 108273/225000 [229:55:57<169:17:00,  5.22s/it]
 48%|████▊     | 108274/225000 [229:56:00<147:31:53,  4.55s/it]
 48%|████▊     | 108275/225000 [229:56:05<146:07:23,  4.51s/it]
 48%|████▊     | 108276/225000 [229:56:07<124:33:01,  3.84s/it]
 48%|████▊     | 108277/225000 [229:56:19<207:11:28,  6.39s/it]
 48%|████▊     | 108278/225000 [229:56:24<196:29:07,  6.06s/it]
 48%|████▊     | 108279/225000 [229:56:27<166:11:02,  5.13s/it]
 48%|████▊     | 108280/225000 [229:56:37<209:32:54,  6.46s/it]
 48%|████▊     | 108281/225000 [229:56:42<190:26:55,  5.87s/it]
 48%|████▊     | 108282/225000 [229:56:44<155:37:42,  4.80s/it]
 48%|████▊     | 108283/225000 [229:56:46<131:09:53,  4.05s/it]
 48%|████▊     | 108284/225000 [229:56:53<160:43:47,  4.96s/it]
 48%|████▊     | 108285/225000 [229:56:59<171:16:05,  5.28s/it]
 48%|████▊     | 108286/225000 [229:57:07<196:00:34,  6.05s/it]
 48%|████▊     | 108287/225000 [229:57:10<166:48:25,  5.15s/it]
 48%|████▊     | 108288/225000 [229:57:13<142:08:38,  4.38s/it]
 48%|████▊     | 108289/225000 [229:57:16<133:20:08,  4.11s/it]
 48%|████▊     | 108290/225000 [229:57:20<126:17:53,  3.90s/it]
 48%|████▊     | 108291/225000 [229:57:27<164:17:01,  5.07s/it]
 48%|████▊     | 108292/225000 [229:57:36<194:46:07,  6.01s/it]
 48%|████▊     | 108293/225000 [229:57:38<156:49:26,  4.84s/it]
 48%|████▊     | 108294/225000 [229:57:44<174:49:06,  5.39s/it]
 48%|████▊     | 108295/225000 [229:57:50<174:12:25,  5.37s/it]
 48%|████▊     | 108296/225000 [229:57:53<152:22:54,  4.70s/it]
 48%|████▊     | 108297/225000 [229:58:01<188:22:28,  5.81s/it]
 48%|████▊     | 108298/225000 [229:58:09<210:37:40,  6.50s/it]
 48%|████▊     | 108299/225000 [229:58:19<240:15:26,  7.41s/it]
 48%|████▊     | 108300/225000 [229:58:23<204:22:56,  6.30s/it]
                                                               
{'loss': '0.903', 'grad_norm': '3.283', 'learning_rate': '2.837e-05', 'epoch': '0.4813', 'num_input_tokens_seen': 385419784, 'train_runtime': '8.279e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108300/225000 [229:58:23<204:22:56,  6.30s/it]
 48%|████▊     | 108301/225000 [229:58:26<179:46:39,  5.55s/it]
 48%|████▊     | 108302/225000 [229:58:34<204:14:26,  6.30s/it]
 48%|████▊     | 108303/225000 [229:58:38<173:46:08,  5.36s/it]
 48%|████▊     | 108304/225000 [229:58:42<167:23:33,  5.16s/it]
 48%|████▊     | 108305/225000 [229:58:53<221:21:44,  6.83s/it]
 48%|████▊     | 108306/225000 [229:59:01<232:18:47,  7.17s/it]
 48%|████▊     | 108307/225000 [229:59:09<244:30:15,  7.54s/it]
 48%|████▊     | 108308/225000 [229:59:15<228:40:48,  7.05s/it]
 48%|████▊     | 108309/225000 [229:59:21<211:38:30,  6.53s/it]
 48%|████▊     | 108310/225000 [229:59:26<197:32:43,  6.09s/it]
 48%|████▊     | 108311/225000 [229:59:31<191:12:41,  5.90s/it]
 48%|████▊     | 108312/225000 [229:59:37<189:48:16,  5.86s/it]
 48%|████▊     | 108313/225000 [229:59:48<241:18:16,  7.44s/it]
 48%|████▊     | 108314/225000 [229:59:51<193:29:29,  5.97s/it]
 48%|████▊     | 108315/225000 [230:00:07<290:25:49,  8.96s/it]
 48%|████▊     | 108316/225000 [230:00:12<253:10:21,  7.81s/it]
 48%|████▊     | 108317/225000 [230:00:14<198:34:51,  6.13s/it]
 48%|████▊     | 108318/225000 [230:00:19<192:07:01,  5.93s/it]
 48%|████▊     | 108319/225000 [230:00:35<286:09:34,  8.83s/it]
 48%|████▊     | 108320/225000 [230:00:44<286:59:56,  8.85s/it]
 48%|████▊     | 108321/225000 [230:00:46<226:22:33,  6.98s/it]
 48%|████▊     | 108322/225000 [230:00:54<236:07:08,  7.29s/it]
 48%|████▊     | 108323/225000 [230:01:02<237:41:07,  7.33s/it]
 48%|████▊     | 108324/225000 [230:01:05<200:46:26,  6.19s/it]
 48%|████▊     | 108325/225000 [230:01:11<199:12:22,  6.15s/it]
 48%|████▊     | 108326/225000 [230:01:18<206:02:09,  6.36s/it]
 48%|████▊     | 108327/225000 [230:01:21<174:48:32,  5.39s/it]
 48%|████▊     | 108328/225000 [230:01:24<146:00:26,  4.51s/it]
 48%|████▊     | 108329/225000 [230:01:27<133:35:35,  4.12s/it]
 48%|████▊     | 108330/225000 [230:01:30<123:45:18,  3.82s/it]
 48%|████▊     | 108331/225000 [230:01:36<144:43:59,  4.47s/it]
 48%|████▊     | 108332/225000 [230:01:42<154:35:43,  4.77s/it]
 48%|████▊     | 108333/225000 [230:01:46<146:34:17,  4.52s/it]
 48%|████▊     | 108334/225000 [230:01:49<138:29:31,  4.27s/it]
 48%|████▊     | 108335/225000 [230:01:55<149:36:51,  4.62s/it]
 48%|████▊     | 108336/225000 [230:01:57<128:26:32,  3.96s/it]
 48%|████▊     | 108337/225000 [230:02:00<113:28:56,  3.50s/it]
 48%|████▊     | 108338/225000 [230:02:08<164:13:45,  5.07s/it]
 48%|████▊     | 108339/225000 [230:02:11<138:22:55,  4.27s/it]
 48%|████▊     | 108340/225000 [230:02:23<217:27:32,  6.71s/it]
 48%|████▊     | 108341/225000 [230:02:29<212:11:48,  6.55s/it]
 48%|████▊     | 108342/225000 [230:02:32<174:24:51,  5.38s/it]
 48%|████▊     | 108343/225000 [230:02:38<177:47:29,  5.49s/it]
 48%|████▊     | 108344/225000 [230:02:44<188:49:44,  5.83s/it]
 48%|████▊     | 108345/225000 [230:02:47<155:44:59,  4.81s/it]
 48%|████▊     | 108346/225000 [230:02:50<140:18:32,  4.33s/it]
 48%|████▊     | 108347/225000 [230:02:52<121:46:35,  3.76s/it]
 48%|████▊     | 108348/225000 [230:03:03<187:05:48,  5.77s/it]
 48%|████▊     | 108349/225000 [230:03:06<157:42:58,  4.87s/it]
 48%|████▊     | 108350/225000 [230:03:08<136:50:57,  4.22s/it]
                                                               
{'loss': '0.9157', 'grad_norm': '3.727', 'learning_rate': '2.835e-05', 'epoch': '0.4816', 'num_input_tokens_seen': 385548824, 'train_runtime': '8.282e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108350/225000 [230:03:08<136:50:57,  4.22s/it]
 48%|████▊     | 108351/225000 [230:03:21<219:58:58,  6.79s/it]
 48%|████▊     | 108352/225000 [230:03:23<177:08:02,  5.47s/it]
 48%|████▊     | 108353/225000 [230:03:29<177:38:37,  5.48s/it]
 48%|████▊     | 108354/225000 [230:03:32<153:54:48,  4.75s/it]
 48%|████▊     | 108355/225000 [230:03:35<137:59:20,  4.26s/it]
 48%|████▊     | 108356/225000 [230:03:38<126:56:39,  3.92s/it]
 48%|████▊     | 108357/225000 [230:03:41<119:07:26,  3.68s/it]
 48%|████▊     | 108358/225000 [230:03:44<105:50:38,  3.27s/it]
 48%|████▊     | 108359/225000 [230:03:49<124:49:05,  3.85s/it]
 48%|████▊     | 108360/225000 [230:04:01<203:45:27,  6.29s/it]
 48%|████▊     | 108361/225000 [230:04:03<163:34:04,  5.05s/it]
 48%|████▊     | 108362/225000 [230:04:09<170:59:22,  5.28s/it]
 48%|████▊     | 108363/225000 [230:04:12<145:29:05,  4.49s/it]
 48%|████▊     | 108364/225000 [230:04:15<133:48:53,  4.13s/it]
 48%|████▊     | 108365/225000 [230:04:19<132:00:26,  4.07s/it]
 48%|████▊     | 108366/225000 [230:04:28<184:59:49,  5.71s/it]
 48%|████▊     | 108367/225000 [230:04:30<149:55:41,  4.63s/it]
 48%|████▊     | 108368/225000 [230:04:33<126:41:34,  3.91s/it]
 48%|████▊     | 108369/225000 [230:04:41<167:53:50,  5.18s/it]
 48%|████▊     | 108370/225000 [230:04:46<168:29:11,  5.20s/it]
 48%|████▊     | 108371/225000 [230:04:56<214:56:06,  6.63s/it]
 48%|████▊     | 108372/225000 [230:05:03<214:23:03,  6.62s/it]
 48%|████▊     | 108373/225000 [230:05:06<178:46:23,  5.52s/it]
 48%|████▊     | 108374/225000 [230:05:10<169:46:47,  5.24s/it]
 48%|████▊     | 108375/225000 [230:05:16<171:51:37,  5.31s/it]
 48%|████▊     | 108376/225000 [230:05:18<144:30:50,  4.46s/it]
 48%|████▊     | 108377/225000 [230:05:24<161:25:35,  4.98s/it]
 48%|████▊     | 108378/225000 [230:05:27<144:07:13,  4.45s/it]
 48%|████▊     | 108379/225000 [230:05:30<124:55:37,  3.86s/it]
 48%|████▊     | 108380/225000 [230:05:36<148:01:03,  4.57s/it]
 48%|████▊     | 108381/225000 [230:05:39<134:34:57,  4.15s/it]
 48%|████▊     | 108382/225000 [230:05:44<134:55:39,  4.17s/it]
 48%|████▊     | 108383/225000 [230:05:46<118:24:24,  3.66s/it]
 48%|████▊     | 108384/225000 [230:05:51<135:35:27,  4.19s/it]
 48%|████▊     | 108385/225000 [230:05:55<126:00:23,  3.89s/it]
 48%|████▊     | 108386/225000 [230:05:58<118:30:41,  3.66s/it]
 48%|████▊     | 108387/225000 [230:06:05<149:51:07,  4.63s/it]
 48%|████▊     | 108388/225000 [230:06:08<135:25:24,  4.18s/it]
 48%|████▊     | 108389/225000 [230:06:12<132:52:01,  4.10s/it]
 48%|████▊     | 108390/225000 [230:06:14<116:37:02,  3.60s/it]
 48%|████▊     | 108391/225000 [230:06:20<135:32:04,  4.18s/it]
 48%|████▊     | 108392/225000 [230:06:31<204:54:10,  6.33s/it]
 48%|████▊     | 108393/225000 [230:06:33<167:26:50,  5.17s/it]
 48%|████▊     | 108394/225000 [230:06:44<221:57:15,  6.85s/it]
 48%|████▊     | 108395/225000 [230:06:49<198:29:35,  6.13s/it]
 48%|████▊     | 108396/225000 [230:06:51<162:28:48,  5.02s/it]
 48%|████▊     | 108397/225000 [230:06:53<135:50:37,  4.19s/it]
 48%|████▊     | 108398/225000 [230:07:04<194:09:59,  5.99s/it]
 48%|████▊     | 108399/225000 [230:07:06<156:53:01,  4.84s/it]
 48%|████▊     | 108400/225000 [230:07:09<138:52:54,  4.29s/it]
                                                               
{'loss': '0.9317', 'grad_norm': '3.039', 'learning_rate': '2.833e-05', 'epoch': '0.4818', 'num_input_tokens_seen': 385657760, 'train_runtime': '8.284e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108400/225000 [230:07:09<138:52:54,  4.29s/it]
 48%|████▊     | 108401/225000 [230:07:14<144:31:40,  4.46s/it]
 48%|████▊     | 108402/225000 [230:07:24<201:24:56,  6.22s/it]
 48%|████▊     | 108403/225000 [230:07:27<170:26:16,  5.26s/it]
 48%|████▊     | 108404/225000 [230:07:43<273:38:30,  8.45s/it]
 48%|████▊     | 108405/225000 [230:07:48<239:52:43,  7.41s/it]
 48%|████▊     | 108406/225000 [230:07:50<192:20:28,  5.94s/it]
 48%|████▊     | 108407/225000 [230:07:54<167:19:07,  5.17s/it]
 48%|████▊     | 108408/225000 [230:08:03<211:36:22,  6.53s/it]
 48%|████▊     | 108409/225000 [230:08:06<170:11:10,  5.25s/it]
 48%|████▊     | 108410/225000 [230:08:10<164:37:16,  5.08s/it]
 48%|████▊     | 108411/225000 [230:08:20<209:57:11,  6.48s/it]
 48%|████▊     | 108412/225000 [230:08:27<214:31:46,  6.62s/it]
 48%|████▊     | 108413/225000 [230:08:33<207:34:31,  6.41s/it]
 48%|████▊     | 108414/225000 [230:08:41<222:05:24,  6.86s/it]
 48%|████▊     | 108415/225000 [230:08:43<179:14:59,  5.54s/it]
 48%|████▊     | 108416/225000 [230:08:46<148:13:34,  4.58s/it]
 48%|████▊     | 108417/225000 [230:08:54<181:40:05,  5.61s/it]
 48%|████▊     | 108418/225000 [230:08:57<163:12:06,  5.04s/it]
 48%|████▊     | 108419/225000 [230:09:07<205:52:15,  6.36s/it]
 48%|████▊     | 108420/225000 [230:09:10<171:19:58,  5.29s/it]
 48%|████▊     | 108421/225000 [230:09:12<141:05:30,  4.36s/it]
 48%|████▊     | 108422/225000 [230:09:14<120:33:05,  3.72s/it]
 48%|████▊     | 108423/225000 [230:09:22<157:32:03,  4.86s/it]
 48%|████▊     | 108424/225000 [230:09:24<132:04:42,  4.08s/it]
 48%|████▊     | 108425/225000 [230:09:30<155:40:13,  4.81s/it]
 48%|████▊     | 108426/225000 [230:09:40<204:17:00,  6.31s/it]
 48%|████▊     | 108427/225000 [230:09:42<164:44:05,  5.09s/it]
 48%|████▊     | 108428/225000 [230:09:45<136:37:15,  4.22s/it]
 48%|████▊     | 108429/225000 [230:09:50<144:00:40,  4.45s/it]
 48%|████▊     | 108430/225000 [230:09:53<130:24:33,  4.03s/it]
 48%|████▊     | 108431/225000 [230:09:56<125:26:51,  3.87s/it]
 48%|████▊     | 108432/225000 [230:10:00<122:50:22,  3.79s/it]
 48%|████▊     | 108433/225000 [230:10:03<114:03:06,  3.52s/it]
 48%|████▊     | 108434/225000 [230:10:07<120:42:18,  3.73s/it]
 48%|████▊     | 108435/225000 [230:10:17<179:29:06,  5.54s/it]
 48%|████▊     | 108436/225000 [230:10:22<177:40:16,  5.49s/it]
 48%|████▊     | 108437/225000 [230:10:26<165:54:58,  5.12s/it]
 48%|████▊     | 108438/225000 [230:10:30<150:25:15,  4.65s/it]
 48%|████▊     | 108439/225000 [230:10:38<181:15:10,  5.60s/it]
 48%|████▊     | 108440/225000 [230:10:44<192:47:26,  5.95s/it]
 48%|████▊     | 108441/225000 [230:10:50<187:27:26,  5.79s/it]
 48%|████▊     | 108442/225000 [230:10:58<212:19:35,  6.56s/it]
 48%|████▊     | 108443/225000 [230:11:01<172:25:24,  5.33s/it]
 48%|████▊     | 108444/225000 [230:11:04<151:22:33,  4.68s/it]
 48%|████▊     | 108445/225000 [230:11:09<158:54:13,  4.91s/it]
 48%|████▊     | 108446/225000 [230:11:12<142:27:46,  4.40s/it]
 48%|████▊     | 108447/225000 [230:11:21<183:49:33,  5.68s/it]
 48%|████▊     | 108448/225000 [230:11:28<193:18:24,  5.97s/it]
 48%|████▊     | 108449/225000 [230:11:31<165:07:37,  5.10s/it]
 48%|████▊     | 108450/225000 [230:11:33<137:15:30,  4.24s/it]
                                                               
{'loss': '0.9061', 'grad_norm': '3.995', 'learning_rate': '2.832e-05', 'epoch': '0.482', 'num_input_tokens_seen': 385771528, 'train_runtime': '8.287e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108450/225000 [230:11:33<137:15:30,  4.24s/it]
 48%|████▊     | 108451/225000 [230:11:35<118:13:05,  3.65s/it]
 48%|████▊     | 108452/225000 [230:11:42<145:13:18,  4.49s/it]
 48%|████▊     | 108453/225000 [230:11:45<131:21:12,  4.06s/it]
 48%|████▊     | 108454/225000 [230:11:50<140:13:42,  4.33s/it]
 48%|████▊     | 108455/225000 [230:11:55<151:12:02,  4.67s/it]
 48%|████▊     | 108456/225000 [230:12:01<166:41:05,  5.15s/it]
 48%|████▊     | 108457/225000 [230:12:05<147:37:51,  4.56s/it]
 48%|████▊     | 108458/225000 [230:12:07<126:51:30,  3.92s/it]
 48%|████▊     | 108459/225000 [230:12:10<112:42:59,  3.48s/it]
 48%|████▊     | 108460/225000 [230:12:27<248:30:56,  7.68s/it]
 48%|████▊     | 108461/225000 [230:12:29<195:25:22,  6.04s/it]
 48%|████▊     | 108462/225000 [230:12:31<157:54:33,  4.88s/it]
 48%|████▊     | 108463/225000 [230:12:34<132:28:01,  4.09s/it]
 48%|████▊     | 108464/225000 [230:12:39<147:22:46,  4.55s/it]
 48%|████▊     | 108465/225000 [230:12:44<148:16:38,  4.58s/it]
 48%|████▊     | 108466/225000 [230:12:47<134:00:38,  4.14s/it]
 48%|████▊     | 108467/225000 [230:12:57<186:54:45,  5.77s/it]
 48%|████▊     | 108468/225000 [230:13:04<202:26:49,  6.25s/it]
 48%|████▊     | 108469/225000 [230:13:08<177:58:18,  5.50s/it]
 48%|████▊     | 108470/225000 [230:13:15<196:44:46,  6.08s/it]
 48%|████▊     | 108471/225000 [230:13:18<166:13:00,  5.14s/it]
 48%|████▊     | 108472/225000 [230:13:21<147:37:34,  4.56s/it]
 48%|████▊     | 108473/225000 [230:13:32<205:48:11,  6.36s/it]
 48%|████▊     | 108474/225000 [230:13:41<236:42:49,  7.31s/it]
 48%|████▊     | 108475/225000 [230:13:47<221:21:19,  6.84s/it]
 48%|████▊     | 108476/225000 [230:13:54<217:40:45,  6.73s/it]
 48%|████▊     | 108477/225000 [230:14:07<282:56:22,  8.74s/it]
 48%|████▊     | 108478/225000 [230:14:11<232:09:07,  7.17s/it]
 48%|████▊     | 108479/225000 [230:14:14<195:21:57,  6.04s/it]
 48%|████▊     | 108480/225000 [230:14:18<177:37:36,  5.49s/it]
 48%|████▊     | 108481/225000 [230:14:21<151:48:09,  4.69s/it]
 48%|████▊     | 108482/225000 [230:14:24<140:12:11,  4.33s/it]
 48%|████▊     | 108483/225000 [230:14:28<130:33:53,  4.03s/it]
 48%|████▊     | 108484/225000 [230:14:31<118:29:28,  3.66s/it]
 48%|████▊     | 108485/225000 [230:14:40<174:08:55,  5.38s/it]
 48%|████▊     | 108486/225000 [230:14:46<182:24:25,  5.64s/it]
 48%|████▊     | 108487/225000 [230:14:49<157:30:33,  4.87s/it]
 48%|████▊     | 108488/225000 [230:14:54<157:50:17,  4.88s/it]
 48%|████▊     | 108489/225000 [230:14:57<140:14:07,  4.33s/it]
 48%|████▊     | 108490/225000 [230:14:59<119:39:29,  3.70s/it]
 48%|████▊     | 108491/225000 [230:15:03<113:23:00,  3.50s/it]
 48%|████▊     | 108492/225000 [230:15:07<123:40:49,  3.82s/it]
 48%|████▊     | 108493/225000 [230:15:19<200:28:59,  6.19s/it]
 48%|████▊     | 108494/225000 [230:15:24<189:26:45,  5.85s/it]
 48%|████▊     | 108495/225000 [230:15:32<212:27:08,  6.56s/it]
 48%|████▊     | 108496/225000 [230:15:37<199:17:31,  6.16s/it]
 48%|████▊     | 108497/225000 [230:15:41<177:58:26,  5.50s/it]
 48%|████▊     | 108498/225000 [230:15:54<244:45:49,  7.56s/it]
 48%|████▊     | 108499/225000 [230:15:57<208:20:05,  6.44s/it]
 48%|████▊     | 108500/225000 [230:16:03<194:43:44,  6.02s/it]
                                                               
{'loss': '0.8835', 'grad_norm': '2.98', 'learning_rate': '2.83e-05', 'epoch': '0.4822', 'num_input_tokens_seen': 385894704, 'train_runtime': '8.29e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108500/225000 [230:16:03<194:43:44,  6.02s/it]
 48%|████▊     | 108501/225000 [230:16:05<162:53:18,  5.03s/it]
 48%|████▊     | 108502/225000 [230:16:11<170:16:04,  5.26s/it]
 48%|████▊     | 108503/225000 [230:16:21<213:53:08,  6.61s/it]
 48%|████▊     | 108504/225000 [230:16:24<179:06:41,  5.53s/it]
 48%|████▊     | 108505/225000 [230:16:27<154:30:53,  4.77s/it]
 48%|████▊     | 108506/225000 [230:16:29<130:08:08,  4.02s/it]
 48%|████▊     | 108507/225000 [230:16:32<120:08:20,  3.71s/it]
 48%|████▊     | 108508/225000 [230:16:37<131:56:55,  4.08s/it]
 48%|████▊     | 108509/225000 [230:16:44<160:19:08,  4.95s/it]
 48%|████▊     | 108510/225000 [230:16:47<141:11:45,  4.36s/it]
 48%|████▊     | 108511/225000 [230:16:52<151:46:20,  4.69s/it]
 48%|████▊     | 108512/225000 [230:16:58<156:06:20,  4.82s/it]
 48%|████▊     | 108513/225000 [230:17:02<155:56:06,  4.82s/it]
 48%|████▊     | 108514/225000 [230:17:09<176:45:03,  5.46s/it]
 48%|████▊     | 108515/225000 [230:17:13<156:12:21,  4.83s/it]
 48%|████▊     | 108516/225000 [230:17:16<141:45:37,  4.38s/it]
 48%|████▊     | 108517/225000 [230:17:20<133:49:22,  4.14s/it]
 48%|████▊     | 108518/225000 [230:17:23<127:46:09,  3.95s/it]
 48%|████▊     | 108519/225000 [230:17:26<114:47:29,  3.55s/it]
 48%|████▊     | 108520/225000 [230:17:35<171:46:38,  5.31s/it]
 48%|████▊     | 108521/225000 [230:17:38<145:20:21,  4.49s/it]
 48%|████▊     | 108522/225000 [230:17:43<150:07:49,  4.64s/it]
 48%|████▊     | 108523/225000 [230:17:56<230:19:30,  7.12s/it]
 48%|████▊     | 108524/225000 [230:17:59<189:43:26,  5.86s/it]
 48%|████▊     | 108525/225000 [230:18:07<215:02:18,  6.65s/it]
 48%|████▊     | 108526/225000 [230:18:10<179:17:47,  5.54s/it]
 48%|████▊     | 108527/225000 [230:18:19<209:35:21,  6.48s/it]
 48%|████▊     | 108528/225000 [230:18:22<176:17:46,  5.45s/it]
 48%|████▊     | 108529/225000 [230:18:25<152:55:44,  4.73s/it]
 48%|████▊     | 108530/225000 [230:18:27<129:11:24,  3.99s/it]
 48%|████▊     | 108531/225000 [230:18:32<137:07:22,  4.24s/it]
 48%|████▊     | 108532/225000 [230:18:34<119:49:26,  3.70s/it]
 48%|████▊     | 108533/225000 [230:18:37<107:18:54,  3.32s/it]
 48%|████▊     | 108534/225000 [230:18:44<145:07:35,  4.49s/it]
 48%|████▊     | 108535/225000 [230:18:50<162:05:39,  5.01s/it]
 48%|████▊     | 108536/225000 [230:18:59<203:20:55,  6.29s/it]
 48%|████▊     | 108537/225000 [230:19:06<203:12:51,  6.28s/it]
 48%|████▊     | 108538/225000 [230:19:12<200:36:18,  6.20s/it]
 48%|████▊     | 108539/225000 [230:19:14<164:45:04,  5.09s/it]
 48%|████▊     | 108540/225000 [230:19:23<199:43:45,  6.17s/it]
 48%|████▊     | 108541/225000 [230:19:26<169:25:13,  5.24s/it]
 48%|████▊     | 108542/225000 [230:19:28<141:19:29,  4.37s/it]
 48%|████▊     | 108543/225000 [230:19:36<178:12:09,  5.51s/it]
 48%|████▊     | 108544/225000 [230:19:51<262:11:06,  8.10s/it]
 48%|████▊     | 108545/225000 [230:19:53<205:03:21,  6.34s/it]
 48%|████▊     | 108546/225000 [230:20:00<217:37:34,  6.73s/it]
 48%|████▊     | 108547/225000 [230:20:05<192:15:16,  5.94s/it]
 48%|████▊     | 108548/225000 [230:20:13<217:54:12,  6.74s/it]
 48%|████▊     | 108549/225000 [230:20:15<173:29:41,  5.36s/it]
 48%|████▊     | 108550/225000 [230:20:19<159:33:07,  4.93s/it]
                                                               
{'loss': '0.8781', 'grad_norm': '2.501', 'learning_rate': '2.828e-05', 'epoch': '0.4824', 'num_input_tokens_seen': 386003328, 'train_runtime': '8.292e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108550/225000 [230:20:19<159:33:07,  4.93s/it]
 48%|████▊     | 108551/225000 [230:20:25<167:32:16,  5.18s/it]
 48%|████▊     | 108552/225000 [230:20:28<148:43:03,  4.60s/it]
 48%|████▊     | 108553/225000 [230:20:31<127:16:00,  3.93s/it]
 48%|████▊     | 108554/225000 [230:20:34<117:07:18,  3.62s/it]
 48%|████▊     | 108555/225000 [230:20:43<169:29:23,  5.24s/it]
 48%|████▊     | 108556/225000 [230:20:49<182:27:14,  5.64s/it]
 48%|████▊     | 108557/225000 [230:20:54<175:34:56,  5.43s/it]
 48%|████▊     | 108558/225000 [230:21:00<179:56:46,  5.56s/it]
 48%|████▊     | 108559/225000 [230:21:03<155:34:49,  4.81s/it]
 48%|████▊     | 108560/225000 [230:21:05<131:40:29,  4.07s/it]
 48%|████▊     | 108561/225000 [230:21:09<129:37:53,  4.01s/it]
 48%|████▊     | 108562/225000 [230:21:17<162:04:03,  5.01s/it]
 48%|████▊     | 108563/225000 [230:21:19<137:37:08,  4.25s/it]
 48%|████▊     | 108564/225000 [230:21:28<183:48:44,  5.68s/it]
 48%|████▊     | 108565/225000 [230:21:36<209:11:27,  6.47s/it]
 48%|████▊     | 108566/225000 [230:21:39<169:02:57,  5.23s/it]
 48%|████▊     | 108567/225000 [230:21:41<141:02:00,  4.36s/it]
 48%|████▊     | 108568/225000 [230:21:46<147:39:17,  4.57s/it]
 48%|████▊     | 108569/225000 [230:21:53<167:26:17,  5.18s/it]
 48%|████▊     | 108570/225000 [230:21:58<164:31:10,  5.09s/it]
 48%|████▊     | 108571/225000 [230:22:00<136:45:27,  4.23s/it]
 48%|████▊     | 108572/225000 [230:22:03<123:33:47,  3.82s/it]
 48%|████▊     | 108573/225000 [230:22:06<123:24:36,  3.82s/it]
 48%|████▊     | 108574/225000 [230:22:16<183:46:34,  5.68s/it]
 48%|████▊     | 108575/225000 [230:22:19<153:33:58,  4.75s/it]
 48%|████▊     | 108576/225000 [230:22:22<132:30:08,  4.10s/it]
 48%|████▊     | 108577/225000 [230:22:32<194:15:20,  6.01s/it]
 48%|████▊     | 108578/225000 [230:22:38<192:31:18,  5.95s/it]
 48%|████▊     | 108579/225000 [230:22:40<156:36:00,  4.84s/it]
 48%|████▊     | 108580/225000 [230:22:45<158:40:48,  4.91s/it]
 48%|████▊     | 108581/225000 [230:22:51<166:38:08,  5.15s/it]
 48%|████▊     | 108582/225000 [230:22:53<138:15:42,  4.28s/it]
 48%|████▊     | 108583/225000 [230:22:55<118:53:42,  3.68s/it]
 48%|████▊     | 108584/225000 [230:23:02<144:11:26,  4.46s/it]
 48%|████▊     | 108585/225000 [230:23:10<180:09:52,  5.57s/it]
 48%|████▊     | 108586/225000 [230:23:14<162:01:39,  5.01s/it]
 48%|████▊     | 108587/225000 [230:23:16<136:03:18,  4.21s/it]
 48%|████▊     | 108588/225000 [230:23:21<146:16:06,  4.52s/it]
 48%|████▊     | 108589/225000 [230:23:25<140:53:02,  4.36s/it]
 48%|████▊     | 108590/225000 [230:23:28<121:48:24,  3.77s/it]
 48%|████▊     | 108591/225000 [230:23:31<115:45:01,  3.58s/it]
 48%|████▊     | 108592/225000 [230:23:36<133:19:39,  4.12s/it]
 48%|████▊     | 108593/225000 [230:23:43<163:23:47,  5.05s/it]
 48%|████▊     | 108594/225000 [230:23:48<159:47:39,  4.94s/it]
 48%|████▊     | 108595/225000 [230:23:51<141:40:47,  4.38s/it]
 48%|████▊     | 108596/225000 [230:23:53<121:13:16,  3.75s/it]
 48%|████▊     | 108597/225000 [230:23:56<107:32:24,  3.33s/it]
 48%|████▊     | 108598/225000 [230:23:59<106:30:22,  3.29s/it]
 48%|████▊     | 108599/225000 [230:24:01<97:01:42,  3.00s/it] 
 48%|████▊     | 108600/225000 [230:24:08<132:10:58,  4.09s/it]
                                                               
{'loss': '0.9187', 'grad_norm': '3.465', 'learning_rate': '2.826e-05', 'epoch': '0.4827', 'num_input_tokens_seen': 386107488, 'train_runtime': '8.294e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108600/225000 [230:24:08<132:10:58,  4.09s/it]
 48%|████▊     | 108601/225000 [230:24:22<227:08:44,  7.03s/it]
 48%|████▊     | 108602/225000 [230:24:26<203:13:09,  6.29s/it]
 48%|████▊     | 108603/225000 [230:24:28<163:25:44,  5.05s/it]
 48%|████▊     | 108604/225000 [230:24:32<144:13:11,  4.46s/it]
 48%|████▊     | 108605/225000 [230:24:37<150:04:25,  4.64s/it]
 48%|████▊     | 108606/225000 [230:24:45<181:46:01,  5.62s/it]
 48%|████▊     | 108607/225000 [230:24:53<207:47:21,  6.43s/it]
 48%|████▊     | 108608/225000 [230:24:55<165:36:23,  5.12s/it]
 48%|████▊     | 108609/225000 [230:24:57<140:05:38,  4.33s/it]
 48%|████▊     | 108610/225000 [230:25:11<229:38:33,  7.10s/it]
 48%|████▊     | 108611/225000 [230:25:18<230:32:53,  7.13s/it]
 48%|████▊     | 108612/225000 [230:25:20<182:46:38,  5.65s/it]
 48%|████▊     | 108613/225000 [230:25:23<150:57:24,  4.67s/it]
 48%|████▊     | 108614/225000 [230:25:26<135:57:51,  4.21s/it]
 48%|████▊     | 108615/225000 [230:25:29<124:23:42,  3.85s/it]
 48%|████▊     | 108616/225000 [230:25:31<110:18:32,  3.41s/it]
 48%|████▊     | 108617/225000 [230:25:34<102:12:25,  3.16s/it]
 48%|████▊     | 108618/225000 [230:25:38<108:01:55,  3.34s/it]
 48%|████▊     | 108619/225000 [230:25:40<98:26:40,  3.05s/it] 
 48%|████▊     | 108620/225000 [230:25:47<134:20:21,  4.16s/it]
 48%|████▊     | 108621/225000 [230:25:49<116:17:49,  3.60s/it]
 48%|████▊     | 108622/225000 [230:25:51<102:50:36,  3.18s/it]
 48%|████▊     | 108623/225000 [230:25:55<108:54:39,  3.37s/it]
 48%|████▊     | 108624/225000 [230:26:01<132:20:31,  4.09s/it]
 48%|████▊     | 108625/225000 [230:26:06<143:34:47,  4.44s/it]
 48%|████▊     | 108626/225000 [230:26:16<199:30:52,  6.17s/it]
 48%|████▊     | 108627/225000 [230:26:20<171:50:48,  5.32s/it]
 48%|████▊     | 108628/225000 [230:26:26<184:27:00,  5.71s/it]
 48%|████▊     | 108629/225000 [230:26:35<218:57:21,  6.77s/it]
 48%|████▊     | 108630/225000 [230:26:38<174:10:28,  5.39s/it]
 48%|████▊     | 108631/225000 [230:26:52<256:23:29,  7.93s/it]
 48%|████▊     | 108632/225000 [230:26:54<201:25:48,  6.23s/it]
 48%|████▊     | 108633/225000 [230:26:56<163:07:16,  5.05s/it]
 48%|████▊     | 108634/225000 [230:26:58<136:22:59,  4.22s/it]
 48%|████▊     | 108635/225000 [230:27:16<269:54:13,  8.35s/it]
 48%|████▊     | 108636/225000 [230:27:19<213:47:29,  6.61s/it]
 48%|████▊     | 108637/225000 [230:27:26<215:27:40,  6.67s/it]
 48%|████▊     | 108638/225000 [230:27:28<171:39:28,  5.31s/it]
 48%|████▊     | 108639/225000 [230:27:30<141:55:54,  4.39s/it]
 48%|████▊     | 108640/225000 [230:27:33<130:22:42,  4.03s/it]
 48%|████▊     | 108641/225000 [230:27:38<134:45:35,  4.17s/it]
 48%|████▊     | 108642/225000 [230:27:43<146:36:05,  4.54s/it]
 48%|████▊     | 108643/225000 [230:27:46<133:55:48,  4.14s/it]
 48%|████▊     | 108644/225000 [230:27:49<117:56:30,  3.65s/it]
 48%|████▊     | 108645/225000 [230:27:58<169:27:58,  5.24s/it]
 48%|████▊     | 108646/225000 [230:28:00<141:44:43,  4.39s/it]
 48%|████▊     | 108647/225000 [230:28:03<123:18:37,  3.82s/it]
 48%|████▊     | 108648/225000 [230:28:08<137:30:51,  4.25s/it]
 48%|████▊     | 108649/225000 [230:28:10<119:14:45,  3.69s/it]
 48%|████▊     | 108650/225000 [230:28:17<150:54:39,  4.67s/it]
                                                               
{'loss': '0.9024', 'grad_norm': '1.779', 'learning_rate': '2.824e-05', 'epoch': '0.4829', 'num_input_tokens_seen': 386214040, 'train_runtime': '8.297e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108650/225000 [230:28:17<150:54:39,  4.67s/it]
 48%|████▊     | 108651/225000 [230:28:21<143:33:56,  4.44s/it]
 48%|████▊     | 108652/225000 [230:28:24<131:27:20,  4.07s/it]
 48%|████▊     | 108653/225000 [230:28:27<115:54:17,  3.59s/it]
 48%|████▊     | 108654/225000 [230:28:29<105:01:07,  3.25s/it]
 48%|████▊     | 108655/225000 [230:28:34<114:38:26,  3.55s/it]
 48%|████▊     | 108656/225000 [230:28:36<103:53:38,  3.21s/it]
 48%|████▊     | 108657/225000 [230:28:39<97:03:58,  3.00s/it] 
 48%|████▊     | 108658/225000 [230:28:51<185:03:18,  5.73s/it]
 48%|████▊     | 108659/225000 [230:28:54<161:38:27,  5.00s/it]
 48%|████▊     | 108660/225000 [230:28:57<145:38:43,  4.51s/it]
 48%|████▊     | 108661/225000 [230:29:00<127:31:04,  3.95s/it]
 48%|████▊     | 108662/225000 [230:29:09<177:17:29,  5.49s/it]
 48%|████▊     | 108663/225000 [230:29:11<146:26:17,  4.53s/it]
 48%|████▊     | 108664/225000 [230:29:24<228:10:56,  7.06s/it]
 48%|████▊     | 108665/225000 [230:29:26<180:29:28,  5.59s/it]
 48%|████▊     | 108666/225000 [230:29:36<221:28:01,  6.85s/it]
 48%|████▊     | 108667/225000 [230:29:44<227:50:14,  7.05s/it]
 48%|████▊     | 108668/225000 [230:29:46<183:10:43,  5.67s/it]
 48%|████▊     | 108669/225000 [230:29:49<151:40:44,  4.69s/it]
 48%|████▊     | 108670/225000 [230:29:56<181:56:24,  5.63s/it]
 48%|████▊     | 108671/225000 [230:30:02<180:22:51,  5.58s/it]
 48%|████▊     | 108672/225000 [230:30:04<149:09:06,  4.62s/it]
 48%|████▊     | 108673/225000 [230:30:07<134:00:58,  4.15s/it]
 48%|████▊     | 108674/225000 [230:30:10<117:07:51,  3.62s/it]
 48%|████▊     | 108675/225000 [230:30:12<104:45:29,  3.24s/it]
 48%|████▊     | 108676/225000 [230:30:22<165:51:30,  5.13s/it]
 48%|████▊     | 108677/225000 [230:30:25<144:44:08,  4.48s/it]
 48%|████▊     | 108678/225000 [230:30:33<187:06:31,  5.79s/it]
 48%|████▊     | 108679/225000 [230:30:37<169:50:27,  5.26s/it]
 48%|████▊     | 108680/225000 [230:30:41<150:46:54,  4.67s/it]
 48%|████▊     | 108681/225000 [230:30:51<205:01:40,  6.35s/it]
 48%|████▊     | 108682/225000 [230:30:59<219:03:55,  6.78s/it]
 48%|████▊     | 108683/225000 [230:31:01<176:10:42,  5.45s/it]
 48%|████▊     | 108684/225000 [230:31:04<151:30:16,  4.69s/it]
 48%|████▊     | 108685/225000 [230:31:07<131:36:24,  4.07s/it]
 48%|████▊     | 108686/225000 [230:31:09<117:09:51,  3.63s/it]
 48%|████▊     | 108687/225000 [230:31:12<107:19:58,  3.32s/it]
 48%|████▊     | 108688/225000 [230:31:16<111:34:56,  3.45s/it]
 48%|████▊     | 108689/225000 [230:31:18<103:39:43,  3.21s/it]
 48%|████▊     | 108690/225000 [230:31:28<170:07:29,  5.27s/it]
 48%|████▊     | 108691/225000 [230:31:35<187:05:08,  5.79s/it]
 48%|████▊     | 108692/225000 [230:31:38<160:21:42,  4.96s/it]
 48%|████▊     | 108693/225000 [230:31:49<211:57:11,  6.56s/it]
 48%|████▊     | 108694/225000 [230:31:54<202:43:50,  6.28s/it]
 48%|████▊     | 108695/225000 [230:31:57<171:42:05,  5.31s/it]
 48%|████▊     | 108696/225000 [230:32:01<154:14:08,  4.77s/it]
 48%|████▊     | 108697/225000 [230:32:05<146:47:53,  4.54s/it]
 48%|████▊     | 108698/225000 [230:32:09<140:19:49,  4.34s/it]
 48%|████▊     | 108699/225000 [230:32:12<125:10:47,  3.87s/it]
 48%|████▊     | 108700/225000 [230:32:15<120:15:58,  3.72s/it]
                                                               
{'loss': '0.8992', 'grad_norm': '3.206', 'learning_rate': '2.822e-05', 'epoch': '0.4831', 'num_input_tokens_seen': 386314616, 'train_runtime': '8.299e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108700/225000 [230:32:15<120:15:58,  3.72s/it]
 48%|████▊     | 108701/225000 [230:32:24<170:11:25,  5.27s/it]
 48%|████▊     | 108702/225000 [230:32:31<189:04:44,  5.85s/it]
 48%|████▊     | 108703/225000 [230:32:38<203:00:35,  6.28s/it]
 48%|████▊     | 108704/225000 [230:32:43<188:51:35,  5.85s/it]
 48%|████▊     | 108705/225000 [230:32:52<220:31:27,  6.83s/it]
 48%|████▊     | 108706/225000 [230:32:59<219:03:39,  6.78s/it]
 48%|████▊     | 108707/225000 [230:33:02<182:23:49,  5.65s/it]
 48%|████▊     | 108708/225000 [230:33:05<161:16:50,  4.99s/it]
 48%|████▊     | 108709/225000 [230:33:15<202:48:36,  6.28s/it]
 48%|████▊     | 108710/225000 [230:33:17<164:38:06,  5.10s/it]
 48%|████▊     | 108711/225000 [230:33:22<165:45:08,  5.13s/it]
 48%|████▊     | 108712/225000 [230:33:25<139:08:26,  4.31s/it]
 48%|████▊     | 108713/225000 [230:33:27<125:24:13,  3.88s/it]
 48%|████▊     | 108714/225000 [230:33:31<118:03:06,  3.65s/it]
 48%|████▊     | 108715/225000 [230:33:33<105:26:52,  3.26s/it]
 48%|████▊     | 108716/225000 [230:33:35<96:24:46,  2.98s/it] 
 48%|████▊     | 108717/225000 [230:33:38<97:44:24,  3.03s/it]
 48%|████▊     | 108718/225000 [230:33:50<176:36:13,  5.47s/it]
 48%|████▊     | 108719/225000 [230:33:55<172:06:50,  5.33s/it]
 48%|████▊     | 108720/225000 [230:33:57<149:07:17,  4.62s/it]
 48%|████▊     | 108721/225000 [230:34:00<126:26:46,  3.91s/it]
 48%|████▊     | 108722/225000 [230:34:06<144:59:41,  4.49s/it]
 48%|████▊     | 108723/225000 [230:34:19<226:52:12,  7.02s/it]
 48%|████▊     | 108724/225000 [230:34:21<180:32:30,  5.59s/it]
 48%|████▊     | 108725/225000 [230:34:27<188:34:26,  5.84s/it]
 48%|████▊     | 108726/225000 [230:34:31<168:41:36,  5.22s/it]
 48%|████▊     | 108727/225000 [230:34:34<149:04:52,  4.62s/it]
 48%|████▊     | 108728/225000 [230:34:37<135:29:12,  4.19s/it]
 48%|████▊     | 108729/225000 [230:34:42<140:30:10,  4.35s/it]
 48%|████▊     | 108730/225000 [230:34:47<150:13:33,  4.65s/it]
 48%|████▊     | 108731/225000 [230:34:53<158:58:26,  4.92s/it]
 48%|████▊     | 108732/225000 [230:34:55<134:44:45,  4.17s/it]
 48%|████▊     | 108733/225000 [230:35:08<218:07:13,  6.75s/it]
 48%|████▊     | 108734/225000 [230:35:11<176:26:01,  5.46s/it]
 48%|████▊     | 108735/225000 [230:35:15<161:49:31,  5.01s/it]
 48%|████▊     | 108736/225000 [230:35:21<177:56:12,  5.51s/it]
 48%|████▊     | 108737/225000 [230:35:27<184:02:14,  5.70s/it]
 48%|████▊     | 108738/225000 [230:35:30<156:20:37,  4.84s/it]
 48%|████▊     | 108739/225000 [230:35:42<224:03:16,  6.94s/it]
 48%|████▊     | 108740/225000 [230:35:45<185:04:25,  5.73s/it]
 48%|████▊     | 108741/225000 [230:35:47<150:43:15,  4.67s/it]
 48%|████▊     | 108742/225000 [230:35:50<135:08:47,  4.18s/it]
 48%|████▊     | 108743/225000 [230:35:53<123:23:11,  3.82s/it]
 48%|████▊     | 108744/225000 [230:36:06<208:33:10,  6.46s/it]
 48%|████▊     | 108745/225000 [230:36:09<175:12:42,  5.43s/it]
 48%|████▊     | 108746/225000 [230:36:21<243:01:25,  7.53s/it]
 48%|████▊     | 108747/225000 [230:36:26<217:56:47,  6.75s/it]
 48%|████▊     | 108748/225000 [230:36:29<182:02:35,  5.64s/it]
 48%|████▊     | 108749/225000 [230:36:35<179:48:11,  5.57s/it]
 48%|████▊     | 108750/225000 [230:36:48<253:18:48,  7.84s/it]
                                                               
{'loss': '0.9188', 'grad_norm': '2.391', 'learning_rate': '2.821e-05', 'epoch': '0.4833', 'num_input_tokens_seen': 386435808, 'train_runtime': '8.302e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108750/225000 [230:36:48<253:18:48,  7.84s/it]
 48%|████▊     | 108751/225000 [230:36:56<254:51:25,  7.89s/it]
 48%|████▊     | 108752/225000 [230:36:58<201:18:21,  6.23s/it]
 48%|████▊     | 108753/225000 [230:37:01<170:39:09,  5.28s/it]
 48%|████▊     | 108754/225000 [230:37:09<197:32:48,  6.12s/it]
 48%|████▊     | 108755/225000 [230:37:16<199:37:53,  6.18s/it]
 48%|████▊     | 108756/225000 [230:37:27<251:05:46,  7.78s/it]
 48%|████▊     | 108757/225000 [230:37:30<203:59:12,  6.32s/it]
 48%|████▊     | 108758/225000 [230:37:34<178:58:31,  5.54s/it]
 48%|████▊     | 108759/225000 [230:37:38<162:06:00,  5.02s/it]
 48%|████▊     | 108760/225000 [230:37:41<149:57:50,  4.64s/it]
 48%|████▊     | 108761/225000 [230:37:45<140:56:17,  4.36s/it]
 48%|████▊     | 108762/225000 [230:37:49<134:41:02,  4.17s/it]
 48%|████▊     | 108763/225000 [230:37:53<130:51:34,  4.05s/it]
 48%|████▊     | 108764/225000 [230:37:58<146:29:05,  4.54s/it]
 48%|████▊     | 108765/225000 [230:38:01<128:55:39,  3.99s/it]
 48%|████▊     | 108766/225000 [230:38:05<129:11:20,  4.00s/it]
 48%|████▊     | 108767/225000 [230:38:10<137:54:48,  4.27s/it]
 48%|████▊     | 108768/225000 [230:38:17<168:30:45,  5.22s/it]
 48%|████▊     | 108769/225000 [230:38:28<224:52:17,  6.96s/it]
 48%|████▊     | 108770/225000 [230:38:31<182:02:50,  5.64s/it]
 48%|████▊     | 108771/225000 [230:38:34<152:58:14,  4.74s/it]
 48%|████▊     | 108772/225000 [230:38:36<132:44:17,  4.11s/it]
 48%|████▊     | 108773/225000 [230:38:46<188:36:32,  5.84s/it]
 48%|████▊     | 108774/225000 [230:38:48<153:07:08,  4.74s/it]
 48%|████▊     | 108775/225000 [230:38:50<128:18:16,  3.97s/it]
 48%|████▊     | 108776/225000 [230:38:56<143:06:50,  4.43s/it]
 48%|████▊     | 108777/225000 [230:39:02<156:37:30,  4.85s/it]
 48%|████▊     | 108778/225000 [230:39:04<132:48:57,  4.11s/it]
 48%|████▊     | 108779/225000 [230:39:07<117:07:20,  3.63s/it]
 48%|████▊     | 108780/225000 [230:39:12<136:14:13,  4.22s/it]
 48%|████▊     | 108781/225000 [230:39:25<217:19:07,  6.73s/it]
 48%|████▊     | 108782/225000 [230:39:31<212:52:52,  6.59s/it]
 48%|████▊     | 108783/225000 [230:39:34<172:02:19,  5.33s/it]
 48%|████▊     | 108784/225000 [230:39:46<238:21:01,  7.38s/it]
 48%|████▊     | 108785/225000 [230:39:48<187:29:22,  5.81s/it]
 48%|████▊     | 108786/225000 [230:39:57<218:56:12,  6.78s/it]
 48%|████▊     | 108787/225000 [230:39:59<173:48:10,  5.38s/it]
 48%|████▊     | 108788/225000 [230:40:01<143:07:20,  4.43s/it]
 48%|████▊     | 108789/225000 [230:40:04<122:58:44,  3.81s/it]
 48%|████▊     | 108790/225000 [230:40:12<169:33:54,  5.25s/it]
 48%|████▊     | 108791/225000 [230:40:14<140:09:22,  4.34s/it]
 48%|████▊     | 108792/225000 [230:40:17<119:29:01,  3.70s/it]
 48%|████▊     | 108793/225000 [230:40:20<112:35:14,  3.49s/it]
 48%|████▊     | 108794/225000 [230:40:31<186:41:15,  5.78s/it]
 48%|████▊     | 108795/225000 [230:40:34<159:58:42,  4.96s/it]
 48%|████▊     | 108796/225000 [230:40:37<141:59:10,  4.40s/it]
 48%|████▊     | 108797/225000 [230:40:40<126:04:18,  3.91s/it]
 48%|████▊     | 108798/225000 [230:40:42<114:53:08,  3.56s/it]
 48%|████▊     | 108799/225000 [230:40:45<108:06:27,  3.35s/it]
 48%|████▊     | 108800/225000 [230:40:48<102:37:16,  3.18s/it]
                                                               
{'loss': '0.8947', 'grad_norm': '5.045', 'learning_rate': '2.819e-05', 'epoch': '0.4836', 'num_input_tokens_seen': 386538496, 'train_runtime': '8.304e+05', 'train_tokens_per_second': '465.5'}

 48%|████▊     | 108800/225000 [230:40:48<102:37:16,  3.18s/it]
 48%|████▊     | 108801/225000 [230:40:51<102:17:02,  3.17s/it]
 48%|████▊     | 108802/225000 [230:40:55<105:33:37,  3.27s/it]
 48%|████▊     | 108803/225000 [230:40:57<100:48:13,  3.12s/it]
 48%|████▊     | 108804/225000 [230:41:09<184:56:56,  5.73s/it]
 48%|████▊     | 108805/225000 [230:41:11<150:44:02,  4.67s/it]
 48%|████▊     | 108806/225000 [230:41:17<156:04:59,  4.84s/it]
 48%|████▊     | 108807/225000 [230:41:26<204:24:22,  6.33s/it]
 48%|████▊     | 108808/225000 [230:41:34<213:20:52,  6.61s/it]
 48%|████▊     | 108809/225000 [230:41:44<249:31:51,  7.73s/it]
 48%|████▊     | 108810/225000 [230:41:54<272:37:27,  8.45s/it]
 48%|████▊     | 108811/225000 [230:41:57<220:11:31,  6.82s/it]
 48%|████▊     | 108812/225000 [230:42:00<182:37:01,  5.66s/it]
 48%|████▊     | 108813/225000 [230:42:03<159:30:24,  4.94s/it]
 48%|████▊     | 108814/225000 [230:42:06<132:51:31,  4.12s/it]
 48%|████▊     | 108815/225000 [230:42:08<114:19:16,  3.54s/it]
 48%|████▊     | 108816/225000 [230:42:10<101:24:29,  3.14s/it]
 48%|████▊     | 108817/225000 [230:42:22<188:04:24,  5.83s/it]
 48%|████▊     | 108818/225000 [230:42:28<189:42:31,  5.88s/it]
 48%|████▊     | 108819/225000 [230:42:31<163:20:51,  5.06s/it]
 48%|████▊     | 108820/225000 [230:42:34<137:59:21,  4.28s/it]
 48%|████▊     | 108821/225000 [230:42:36<119:46:57,  3.71s/it]
 48%|████▊     | 108822/225000 [230:42:41<134:29:49,  4.17s/it]
 48%|████▊     | 108823/225000 [230:42:44<117:20:27,  3.64s/it]
 48%|████▊     | 108824/225000 [230:42:50<144:48:59,  4.49s/it]
 48%|████▊     | 108825/225000 [230:43:00<196:43:02,  6.10s/it]
 48%|████▊     | 108826/225000 [230:43:02<158:37:28,  4.92s/it]
 48%|████▊     | 108827/225000 [230:43:08<166:56:05,  5.17s/it]
 48%|████▊     | 108828/225000 [230:43:18<211:43:57,  6.56s/it]
 48%|████▊     | 108829/225000 [230:43:20<169:35:15,  5.26s/it]
 48%|████▊     | 108830/225000 [230:43:22<140:22:18,  4.35s/it]
 48%|████▊     | 108831/225000 [230:43:25<119:48:19,  3.71s/it]
 48%|████▊     | 108832/225000 [230:43:27<106:40:33,  3.31s/it]
 48%|████▊     | 108833/225000 [230:43:31<115:54:55,  3.59s/it]
 48%|████▊     | 108834/225000 [230:43:34<109:00:35,  3.38s/it]
 48%|████▊     | 108835/225000 [230:43:51<243:42:59,  7.55s/it]
 48%|████▊     | 108836/225000 [230:43:57<226:48:26,  7.03s/it]
 48%|████▊     | 108837/225000 [230:43:59<179:25:11,  5.56s/it]
 48%|████▊     | 108838/225000 [230:44:04<167:31:47,  5.19s/it]
 48%|████▊     | 108839/225000 [230:44:07<147:04:24,  4.56s/it]
 48%|████▊     | 108840/225000 [230:44:16<194:36:29,  6.03s/it]
 48%|████▊     | 108841/225000 [230:44:23<199:17:19,  6.18s/it]
 48%|████▊     | 108842/225000 [230:44:31<220:56:55,  6.85s/it]
 48%|████▊     | 108843/225000 [230:44:36<202:27:58,  6.27s/it]
 48%|████▊     | 108844/225000 [230:44:45<231:38:06,  7.18s/it]
 48%|████▊     | 108845/225000 [230:44:49<197:52:39,  6.13s/it]
 48%|████▊     | 108846/225000 [230:44:52<164:55:55,  5.11s/it]
 48%|████▊     | 108847/225000 [230:44:55<143:08:27,  4.44s/it]
 48%|████▊     | 108848/225000 [230:45:02<177:02:39,  5.49s/it]
 48%|████▊     | 108849/225000 [230:45:13<228:40:25,  7.09s/it]
 48%|████▊     | 108850/225000 [230:45:16<189:19:04,  5.87s/it]
                                                               
{'loss': '0.9474', 'grad_norm': '3.182', 'learning_rate': '2.817e-05', 'epoch': '0.4838', 'num_input_tokens_seen': 386651200, 'train_runtime': '8.307e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 108850/225000 [230:45:16<189:19:04,  5.87s/it]
 48%|████▊     | 108851/225000 [230:45:23<197:46:56,  6.13s/it]
 48%|████▊     | 108852/225000 [230:45:26<169:33:37,  5.26s/it]
 48%|████▊     | 108853/225000 [230:45:29<143:34:29,  4.45s/it]
 48%|████▊     | 108854/225000 [230:45:31<125:19:59,  3.88s/it]
 48%|████▊     | 108855/225000 [230:45:40<174:17:18,  5.40s/it]
 48%|████▊     | 108856/225000 [230:45:43<143:18:07,  4.44s/it]
 48%|████▊     | 108857/225000 [230:45:49<165:24:47,  5.13s/it]
 48%|████▊     | 108858/225000 [230:45:54<162:04:03,  5.02s/it]
 48%|████▊     | 108859/225000 [230:45:58<149:39:08,  4.64s/it]
 48%|████▊     | 108860/225000 [230:46:00<126:43:31,  3.93s/it]
 48%|████▊     | 108861/225000 [230:46:03<114:06:09,  3.54s/it]
 48%|████▊     | 108862/225000 [230:46:08<130:39:09,  4.05s/it]
 48%|████▊     | 108863/225000 [230:46:12<128:20:12,  3.98s/it]
 48%|████▊     | 108864/225000 [230:46:21<178:06:44,  5.52s/it]
 48%|████▊     | 108865/225000 [230:46:25<163:07:03,  5.06s/it]
 48%|████▊     | 108866/225000 [230:46:32<181:55:27,  5.64s/it]
 48%|████▊     | 108867/225000 [230:46:35<158:31:17,  4.91s/it]
 48%|████▊     | 108868/225000 [230:46:43<187:44:30,  5.82s/it]
 48%|████▊     | 108869/225000 [230:46:48<183:43:52,  5.70s/it]
 48%|████▊     | 108870/225000 [230:46:54<182:01:46,  5.64s/it]
 48%|████▊     | 108871/225000 [230:46:58<165:50:50,  5.14s/it]
 48%|████▊     | 108872/225000 [230:47:03<169:13:29,  5.25s/it]
 48%|████▊     | 108873/225000 [230:47:10<186:40:29,  5.79s/it]
 48%|████▊     | 108874/225000 [230:47:16<185:50:36,  5.76s/it]
 48%|████▊     | 108875/225000 [230:47:20<171:59:56,  5.33s/it]
 48%|████▊     | 108876/225000 [230:47:23<142:22:49,  4.41s/it]
 48%|████▊     | 108877/225000 [230:47:25<122:31:29,  3.80s/it]
 48%|████▊     | 108878/225000 [230:47:37<201:23:38,  6.24s/it]
 48%|████▊     | 108879/225000 [230:47:42<186:45:49,  5.79s/it]
 48%|████▊     | 108880/225000 [230:47:48<192:11:24,  5.96s/it]
 48%|████▊     | 108881/225000 [230:47:53<178:33:01,  5.54s/it]
 48%|████▊     | 108882/225000 [230:48:02<215:14:43,  6.67s/it]
 48%|████▊     | 108883/225000 [230:48:07<196:09:52,  6.08s/it]
 48%|████▊     | 108884/225000 [230:48:09<159:15:34,  4.94s/it]
 48%|████▊     | 108885/225000 [230:48:21<231:13:13,  7.17s/it]
 48%|████▊     | 108886/225000 [230:48:24<183:53:57,  5.70s/it]
 48%|████▊     | 108887/225000 [230:48:26<150:39:30,  4.67s/it]
 48%|████▊     | 108888/225000 [230:48:31<154:07:37,  4.78s/it]
 48%|████▊     | 108889/225000 [230:48:34<133:01:38,  4.12s/it]
 48%|████▊     | 108890/225000 [230:48:43<188:57:26,  5.86s/it]
 48%|████▊     | 108891/225000 [230:48:48<180:07:09,  5.58s/it]
 48%|████▊     | 108892/225000 [230:48:51<149:50:18,  4.65s/it]
 48%|████▊     | 108893/225000 [230:48:57<160:07:25,  4.96s/it]
 48%|████▊     | 108894/225000 [230:49:00<141:01:40,  4.37s/it]
 48%|████▊     | 108895/225000 [230:49:06<162:47:19,  5.05s/it]
 48%|████▊     | 108896/225000 [230:49:09<137:56:39,  4.28s/it]
 48%|████▊     | 108897/225000 [230:49:14<148:48:27,  4.61s/it]
 48%|████▊     | 108898/225000 [230:49:23<191:29:14,  5.94s/it]
 48%|████▊     | 108899/225000 [230:49:28<186:10:57,  5.77s/it]
 48%|████▊     | 108900/225000 [230:49:31<151:08:29,  4.69s/it]
                                                               
{'loss': '0.8604', 'grad_norm': '4.16', 'learning_rate': '2.815e-05', 'epoch': '0.484', 'num_input_tokens_seen': 386762008, 'train_runtime': '8.31e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 108900/225000 [230:49:31<151:08:29,  4.69s/it]
 48%|████▊     | 108901/225000 [230:49:33<126:34:42,  3.92s/it]
 48%|████▊     | 108902/225000 [230:49:36<117:24:48,  3.64s/it]
 48%|████▊     | 108903/225000 [230:49:39<111:27:01,  3.46s/it]
 48%|████▊     | 108904/225000 [230:49:48<166:22:41,  5.16s/it]
 48%|████▊     | 108905/225000 [230:49:50<138:43:44,  4.30s/it]
 48%|████▊     | 108906/225000 [230:49:55<144:10:29,  4.47s/it]
 48%|████▊     | 108907/225000 [230:49:59<140:24:44,  4.35s/it]
 48%|████▊     | 108908/225000 [230:50:02<126:42:59,  3.93s/it]
 48%|████▊     | 108909/225000 [230:50:12<183:07:04,  5.68s/it]
 48%|████▊     | 108910/225000 [230:50:21<217:15:28,  6.74s/it]
 48%|████▊     | 108911/225000 [230:50:35<285:50:30,  8.86s/it]
 48%|████▊     | 108912/225000 [230:50:37<221:31:55,  6.87s/it]
 48%|████▊     | 108913/225000 [230:50:39<176:58:23,  5.49s/it]
 48%|████▊     | 108914/225000 [230:50:45<178:48:41,  5.55s/it]
 48%|████▊     | 108915/225000 [230:50:48<154:02:04,  4.78s/it]
 48%|████▊     | 108916/225000 [230:50:50<129:30:34,  4.02s/it]
 48%|████▊     | 108917/225000 [230:50:56<147:54:54,  4.59s/it]
 48%|████▊     | 108918/225000 [230:51:00<145:00:28,  4.50s/it]
 48%|████▊     | 108919/225000 [230:51:03<123:27:32,  3.83s/it]
 48%|████▊     | 108920/225000 [230:51:06<116:03:28,  3.60s/it]
 48%|████▊     | 108921/225000 [230:51:09<111:07:59,  3.45s/it]
 48%|████▊     | 108922/225000 [230:51:11<100:05:44,  3.10s/it]
 48%|████▊     | 108923/225000 [230:51:14<99:29:33,  3.09s/it] 
 48%|████▊     | 108924/225000 [230:51:29<216:34:17,  6.72s/it]
 48%|████▊     | 108925/225000 [230:51:34<200:15:08,  6.21s/it]
 48%|████▊     | 108926/225000 [230:51:38<169:42:01,  5.26s/it]
 48%|████▊     | 108927/225000 [230:51:40<140:29:11,  4.36s/it]
 48%|████▊     | 108928/225000 [230:51:43<126:17:14,  3.92s/it]
 48%|████▊     | 108929/225000 [230:51:49<150:41:47,  4.67s/it]
 48%|████▊     | 108930/225000 [230:51:59<202:28:19,  6.28s/it]
 48%|████▊     | 108931/225000 [230:52:01<162:59:30,  5.06s/it]
 48%|████▊     | 108932/225000 [230:52:08<181:08:06,  5.62s/it]
 48%|████▊     | 108933/225000 [230:52:17<212:00:20,  6.58s/it]
 48%|████▊     | 108934/225000 [230:52:19<170:11:03,  5.28s/it]
 48%|████▊     | 108935/225000 [230:52:23<154:28:49,  4.79s/it]
 48%|████▊     | 108936/225000 [230:52:25<129:36:30,  4.02s/it]
 48%|████▊     | 108937/225000 [230:52:35<186:19:07,  5.78s/it]
 48%|████▊     | 108938/225000 [230:52:41<185:04:55,  5.74s/it]
 48%|████▊     | 108939/225000 [230:52:44<158:34:06,  4.92s/it]
 48%|████▊     | 108940/225000 [230:52:47<147:21:06,  4.57s/it]
 48%|████▊     | 108941/225000 [230:52:51<133:40:17,  4.15s/it]
 48%|████▊     | 108942/225000 [230:53:04<219:16:36,  6.80s/it]
 48%|████▊     | 108943/225000 [230:53:06<178:10:32,  5.53s/it]
 48%|████▊     | 108944/225000 [230:53:09<152:51:27,  4.74s/it]
 48%|████▊     | 108945/225000 [230:53:26<271:29:23,  8.42s/it]
 48%|████▊     | 108946/225000 [230:53:35<278:15:14,  8.63s/it]
 48%|████▊     | 108947/225000 [230:53:42<261:12:38,  8.10s/it]
 48%|████▊     | 108948/225000 [230:53:44<203:48:04,  6.32s/it]
 48%|████▊     | 108949/225000 [230:53:46<163:50:07,  5.08s/it]
 48%|████▊     | 108950/225000 [230:53:55<193:33:32,  6.00s/it]
                                                               
{'loss': '0.8907', 'grad_norm': '1.71', 'learning_rate': '2.813e-05', 'epoch': '0.4842', 'num_input_tokens_seen': 386881344, 'train_runtime': '8.312e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 108950/225000 [230:53:55<193:33:32,  6.00s/it]
 48%|████▊     | 108951/225000 [230:53:59<175:48:47,  5.45s/it]
 48%|████▊     | 108952/225000 [230:54:09<221:37:35,  6.88s/it]
 48%|████▊     | 108953/225000 [230:54:17<234:53:34,  7.29s/it]
 48%|████▊     | 108954/225000 [230:54:19<186:21:43,  5.78s/it]
 48%|████▊     | 108955/225000 [230:54:22<151:44:43,  4.71s/it]
 48%|████▊     | 108956/225000 [230:54:34<229:15:29,  7.11s/it]
 48%|████▊     | 108957/225000 [230:54:37<188:51:50,  5.86s/it]
 48%|████▊     | 108958/225000 [230:54:48<239:24:53,  7.43s/it]
 48%|████▊     | 108959/225000 [230:54:53<209:27:49,  6.50s/it]
 48%|████▊     | 108960/225000 [230:54:57<187:00:33,  5.80s/it]
 48%|████▊     | 108961/225000 [230:55:01<171:03:49,  5.31s/it]
 48%|████▊     | 108962/225000 [230:55:05<153:36:16,  4.77s/it]
 48%|████▊     | 108963/225000 [230:55:07<133:43:50,  4.15s/it]
 48%|████▊     | 108964/225000 [230:55:15<165:09:54,  5.12s/it]
 48%|████▊     | 108965/225000 [230:55:18<144:33:03,  4.48s/it]
 48%|████▊     | 108966/225000 [230:55:20<123:27:37,  3.83s/it]
 48%|████▊     | 108967/225000 [230:55:31<195:48:41,  6.08s/it]
 48%|████▊     | 108968/225000 [230:55:37<189:56:17,  5.89s/it]
 48%|████▊     | 108969/225000 [230:55:39<157:06:25,  4.87s/it]
 48%|████▊     | 108970/225000 [230:55:42<134:03:43,  4.16s/it]
 48%|████▊     | 108971/225000 [230:55:47<147:09:06,  4.57s/it]
 48%|████▊     | 108972/225000 [230:55:57<195:34:36,  6.07s/it]
 48%|████▊     | 108973/225000 [230:56:03<192:26:59,  5.97s/it]
 48%|████▊     | 108974/225000 [230:56:05<156:20:35,  4.85s/it]
 48%|████▊     | 108975/225000 [230:56:10<157:01:08,  4.87s/it]
 48%|████▊     | 108976/225000 [230:56:12<134:34:16,  4.18s/it]
 48%|████▊     | 108977/225000 [230:56:22<183:54:31,  5.71s/it]
 48%|████▊     | 108978/225000 [230:56:30<209:14:45,  6.49s/it]
 48%|████▊     | 108979/225000 [230:56:36<201:28:34,  6.25s/it]
 48%|████▊     | 108980/225000 [230:56:46<245:40:45,  7.62s/it]
 48%|████▊     | 108981/225000 [230:56:50<201:43:29,  6.26s/it]
 48%|████▊     | 108982/225000 [230:56:52<163:15:54,  5.07s/it]
 48%|████▊     | 108983/225000 [230:56:54<136:37:48,  4.24s/it]
 48%|████▊     | 108984/225000 [230:56:59<145:10:27,  4.50s/it]
 48%|████▊     | 108985/225000 [230:57:09<194:27:02,  6.03s/it]
 48%|████▊     | 108986/225000 [230:57:11<157:37:21,  4.89s/it]
 48%|████▊     | 108987/225000 [230:57:13<131:42:20,  4.09s/it]
 48%|████▊     | 108988/225000 [230:57:16<120:29:33,  3.74s/it]
 48%|████▊     | 108989/225000 [230:57:21<127:44:23,  3.96s/it]
 48%|████▊     | 108990/225000 [230:57:24<117:52:34,  3.66s/it]
 48%|████▊     | 108991/225000 [230:57:26<110:21:24,  3.42s/it]
 48%|████▊     | 108992/225000 [230:57:36<171:08:36,  5.31s/it]
 48%|████▊     | 108993/225000 [230:57:39<142:08:47,  4.41s/it]
 48%|████▊     | 108994/225000 [230:57:41<127:14:42,  3.95s/it]
 48%|████▊     | 108995/225000 [230:57:53<203:22:24,  6.31s/it]
 48%|████▊     | 108996/225000 [230:57:57<174:17:29,  5.41s/it]
 48%|████▊     | 108997/225000 [230:58:03<187:34:58,  5.82s/it]
 48%|████▊     | 108998/225000 [230:58:08<181:13:24,  5.62s/it]
 48%|████▊     | 108999/225000 [230:58:16<204:05:52,  6.33s/it]
 48%|████▊     | 109000/225000 [230:58:19<167:50:30,  5.21s/it]
                                                               
{'loss': '0.8793', 'grad_norm': '5.132', 'learning_rate': '2.812e-05', 'epoch': '0.4844', 'num_input_tokens_seen': 386988960, 'train_runtime': '8.315e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 109000/225000 [230:58:19<167:50:30,  5.21s/it]
 48%|████▊     | 109001/225000 [230:58:28<200:15:53,  6.22s/it]
 48%|████▊     | 109002/225000 [230:58:30<164:44:24,  5.11s/it]
 48%|████▊     | 109003/225000 [230:58:41<219:21:42,  6.81s/it]
 48%|████▊     | 109004/225000 [230:58:46<206:39:08,  6.41s/it]
 48%|████▊     | 109005/225000 [230:58:49<166:02:42,  5.15s/it]
 48%|████▊     | 109006/225000 [230:58:51<137:22:01,  4.26s/it]
 48%|████▊     | 109007/225000 [230:58:54<127:01:32,  3.94s/it]
 48%|████▊     | 109008/225000 [230:58:58<125:47:43,  3.90s/it]
 48%|████▊     | 109009/225000 [230:59:10<207:12:44,  6.43s/it]
 48%|████▊     | 109010/225000 [230:59:16<203:45:34,  6.32s/it]
 48%|████▊     | 109011/225000 [230:59:19<165:45:42,  5.14s/it]
 48%|████▊     | 109012/225000 [230:59:21<139:19:47,  4.32s/it]
 48%|████▊     | 109013/225000 [230:59:23<120:38:02,  3.74s/it]
 48%|████▊     | 109014/225000 [230:59:29<137:42:30,  4.27s/it]
 48%|████▊     | 109015/225000 [230:59:36<168:44:22,  5.24s/it]
 48%|████▊     | 109016/225000 [230:59:39<142:47:02,  4.43s/it]
 48%|████▊     | 109017/225000 [230:59:42<131:44:59,  4.09s/it]
 48%|████▊     | 109018/225000 [230:59:51<174:50:22,  5.43s/it]
 48%|████▊     | 109019/225000 [230:59:56<172:26:28,  5.35s/it]
 48%|████▊     | 109020/225000 [230:59:58<144:01:54,  4.47s/it]
 48%|████▊     | 109021/225000 [231:00:01<124:21:26,  3.86s/it]
 48%|████▊     | 109022/225000 [231:00:03<109:58:43,  3.41s/it]
 48%|████▊     | 109023/225000 [231:00:12<163:16:48,  5.07s/it]
 48%|████▊     | 109024/225000 [231:00:19<183:10:52,  5.69s/it]
 48%|████▊     | 109025/225000 [231:00:22<158:44:47,  4.93s/it]
 48%|████▊     | 109026/225000 [231:00:28<162:02:45,  5.03s/it]
 48%|████▊     | 109027/225000 [231:00:30<137:06:44,  4.26s/it]
 48%|████▊     | 109028/225000 [231:00:35<146:26:02,  4.55s/it]
 48%|████▊     | 109029/225000 [231:00:40<149:42:46,  4.65s/it]
 48%|████▊     | 109030/225000 [231:00:43<126:53:55,  3.94s/it]
 48%|████▊     | 109031/225000 [231:00:47<136:58:27,  4.25s/it]
 48%|████▊     | 109032/225000 [231:00:50<122:39:23,  3.81s/it]
 48%|████▊     | 109033/225000 [231:01:00<176:39:03,  5.48s/it]
 48%|████▊     | 109034/225000 [231:01:03<151:37:19,  4.71s/it]
 48%|████▊     | 109035/225000 [231:01:12<192:45:32,  5.98s/it]
 48%|████▊     | 109036/225000 [231:01:15<171:00:03,  5.31s/it]
 48%|████▊     | 109037/225000 [231:01:17<140:36:23,  4.37s/it]
 48%|████▊     | 109038/225000 [231:01:26<180:21:33,  5.60s/it]
 48%|████▊     | 109039/225000 [231:01:28<148:46:50,  4.62s/it]
 48%|████▊     | 109040/225000 [231:01:32<138:44:20,  4.31s/it]
 48%|████▊     | 109041/225000 [231:01:34<122:20:54,  3.80s/it]
 48%|████▊     | 109042/225000 [231:01:41<151:51:51,  4.71s/it]
 48%|████▊     | 109043/225000 [231:01:45<142:42:28,  4.43s/it]
 48%|████▊     | 109044/225000 [231:01:48<130:54:55,  4.06s/it]
 48%|████▊     | 109045/225000 [231:01:51<114:40:36,  3.56s/it]
 48%|████▊     | 109046/225000 [231:01:53<103:05:20,  3.20s/it]
 48%|████▊     | 109047/225000 [231:01:58<123:29:38,  3.83s/it]
 48%|████▊     | 109048/225000 [231:02:02<124:37:57,  3.87s/it]
 48%|████▊     | 109049/225000 [231:02:08<146:28:28,  4.55s/it]
 48%|████▊     | 109050/225000 [231:02:14<154:54:00,  4.81s/it]
                                                               
{'loss': '0.894', 'grad_norm': '3.064', 'learning_rate': '2.81e-05', 'epoch': '0.4847', 'num_input_tokens_seen': 387090112, 'train_runtime': '8.317e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 109050/225000 [231:02:14<154:54:00,  4.81s/it]
 48%|████▊     | 109051/225000 [231:02:20<164:04:16,  5.09s/it]
 48%|████▊     | 109052/225000 [231:02:26<173:14:03,  5.38s/it]
 48%|████▊     | 109053/225000 [231:02:28<142:37:46,  4.43s/it]
 48%|████▊     | 109054/225000 [231:02:32<144:14:47,  4.48s/it]
 48%|████▊     | 109055/225000 [231:02:35<122:39:59,  3.81s/it]
 48%|████▊     | 109056/225000 [231:02:40<133:51:45,  4.16s/it]
 48%|████▊     | 109057/225000 [231:02:46<155:10:45,  4.82s/it]
 48%|████▊     | 109058/225000 [231:02:52<169:41:24,  5.27s/it]
 48%|████▊     | 109059/225000 [231:02:55<148:08:35,  4.60s/it]
 48%|████▊     | 109060/225000 [231:03:06<210:07:49,  6.52s/it]
 48%|████▊     | 109061/225000 [231:03:09<170:16:39,  5.29s/it]
 48%|████▊     | 109062/225000 [231:03:16<185:59:20,  5.78s/it]
 48%|████▊     | 109063/225000 [231:03:19<163:08:25,  5.07s/it]
 48%|████▊     | 109064/225000 [231:03:33<249:47:26,  7.76s/it]
 48%|████▊     | 109065/225000 [231:03:35<196:35:31,  6.10s/it]
 48%|████▊     | 109066/225000 [231:03:43<212:30:17,  6.60s/it]
 48%|████▊     | 109067/225000 [231:03:45<170:12:33,  5.29s/it]
 48%|████▊     | 109068/225000 [231:03:48<143:41:35,  4.46s/it]
 48%|████▊     | 109069/225000 [231:03:51<129:47:36,  4.03s/it]
 48%|████▊     | 109070/225000 [231:03:54<120:34:20,  3.74s/it]
 48%|████▊     | 109071/225000 [231:03:58<120:33:37,  3.74s/it]
 48%|████▊     | 109072/225000 [231:04:05<158:54:41,  4.93s/it]
 48%|████▊     | 109073/225000 [231:04:08<140:01:47,  4.35s/it]
 48%|████▊     | 109074/225000 [231:04:11<126:52:40,  3.94s/it]
 48%|████▊     | 109075/225000 [231:04:16<132:17:21,  4.11s/it]
 48%|████▊     | 109076/225000 [231:04:18<113:27:39,  3.52s/it]
 48%|████▊     | 109077/225000 [231:04:24<141:23:22,  4.39s/it]
 48%|████▊     | 109078/225000 [231:04:28<135:43:24,  4.21s/it]
 48%|████▊     | 109079/225000 [231:04:30<116:03:32,  3.60s/it]
 48%|████▊     | 109080/225000 [231:04:34<112:04:26,  3.48s/it]
 48%|████▊     | 109081/225000 [231:04:43<166:19:17,  5.17s/it]
 48%|████▊     | 109082/225000 [231:04:49<178:10:48,  5.53s/it]
 48%|████▊     | 109083/225000 [231:04:58<209:39:45,  6.51s/it]
 48%|████▊     | 109084/225000 [231:05:05<218:14:23,  6.78s/it]
 48%|████▊     | 109085/225000 [231:05:12<221:17:37,  6.87s/it]
 48%|████▊     | 109086/225000 [231:05:23<255:21:01,  7.93s/it]
 48%|████▊     | 109087/225000 [231:05:26<212:30:20,  6.60s/it]
 48%|████▊     | 109088/225000 [231:05:29<175:56:30,  5.46s/it]
 48%|████▊     | 109089/225000 [231:05:32<150:07:21,  4.66s/it]
 48%|████▊     | 109090/225000 [231:05:35<138:46:42,  4.31s/it]
 48%|████▊     | 109091/225000 [231:05:39<130:15:38,  4.05s/it]
 48%|████▊     | 109092/225000 [231:05:42<117:58:08,  3.66s/it]
 48%|████▊     | 109093/225000 [231:05:44<108:43:01,  3.38s/it]
 48%|████▊     | 109094/225000 [231:05:52<154:06:15,  4.79s/it]
 48%|████▊     | 109095/225000 [231:05:55<135:37:07,  4.21s/it]
 48%|████▊     | 109096/225000 [231:05:58<123:07:13,  3.82s/it]
 48%|████▊     | 109097/225000 [231:06:07<167:29:39,  5.20s/it]
 48%|████▊     | 109098/225000 [231:06:10<153:22:11,  4.76s/it]
 48%|████▊     | 109099/225000 [231:06:13<128:28:24,  3.99s/it]
 48%|████▊     | 109100/225000 [231:06:15<116:41:43,  3.62s/it]
                                                               
{'loss': '0.911', 'grad_norm': '3.077', 'learning_rate': '2.808e-05', 'epoch': '0.4849', 'num_input_tokens_seen': 387191208, 'train_runtime': '8.32e+05', 'train_tokens_per_second': '465.4'}

 48%|████▊     | 109100/225000 [231:06:15<116:41:43,  3.62s/it]
 48%|████▊     | 109101/225000 [231:06:18<111:48:41,  3.47s/it]
 48%|████▊     | 109102/225000 [231:06:31<202:28:59,  6.29s/it]
 48%|████▊     | 109103/225000 [231:06:37<196:35:10,  6.11s/it]
 48%|████▊     | 109104/225000 [231:06:42<189:19:22,  5.88s/it]
 48%|████▊     | 109105/225000 [231:06:48<189:34:01,  5.89s/it]
 48%|████▊     | 109106/225000 [231:06:51<161:23:51,  5.01s/it]
 48%|████▊     | 109107/225000 [231:06:53<133:54:18,  4.16s/it]
 48%|████▊     | 109108/225000 [231:06:56<115:57:25,  3.60s/it]
 48%|████▊     | 109109/225000 [231:07:01<130:23:25,  4.05s/it]
 48%|████▊     | 109110/225000 [231:07:04<123:28:21,  3.84s/it]
 48%|████▊     | 109111/225000 [231:07:15<193:00:38,  6.00s/it]
 48%|████▊     | 109112/225000 [231:07:17<156:02:21,  4.85s/it]
 48%|████▊     | 109113/225000 [231:07:24<173:03:19,  5.38s/it]
 48%|████▊     | 109114/225000 [231:07:35<225:03:31,  6.99s/it]
 48%|████▊     | 109115/225000 [231:07:38<186:31:30,  5.79s/it]
 48%|████▊     | 109116/225000 [231:07:40<152:35:06,  4.74s/it]
 48%|████▊     | 109117/225000 [231:07:45<153:07:06,  4.76s/it]
 48%|████▊     | 109118/225000 [231:07:48<139:06:05,  4.32s/it]
 48%|████▊     | 109119/225000 [231:07:52<134:21:59,  4.17s/it]
 48%|████▊     | 109120/225000 [231:07:55<126:59:21,  3.95s/it]
 48%|████▊     | 109121/225000 [231:07:58<116:01:52,  3.60s/it]
 48%|████▊     | 109122/225000 [231:08:02<118:33:49,  3.68s/it]
 48%|████▊     | 109123/225000 [231:08:05<113:21:15,  3.52s/it]
 48%|████▊     | 109124/225000 [231:08:09<115:54:16,  3.60s/it]
 48%|████▊     | 109125/225000 [231:08:12<112:07:07,  3.48s/it]
 49%|████▊     | 109126/225000 [231:08:22<174:35:23,  5.42s/it]
 49%|████▊     | 109127/225000 [231:08:24<144:56:14,  4.50s/it]
 49%|████▊     | 109128/225000 [231:08:31<161:00:12,  5.00s/it]
 49%|████▊     | 109129/225000 [231:08:35<150:29:19,  4.68s/it]
 49%|████▊     | 109130/225000 [231:08:37<128:44:54,  4.00s/it]
 49%|████▊     | 109131/225000 [231:08:54<255:26:43,  7.94s/it]
 49%|████▊     | 109132/225000 [231:09:02<257:10:00,  7.99s/it]
 49%|████▊     | 109133/225000 [231:09:14<296:38:42,  9.22s/it]
 49%|████▊     | 109134/225000 [231:09:19<250:02:31,  7.77s/it]
 49%|████▊     | 109135/225000 [231:09:21<197:50:06,  6.15s/it]
 49%|████▊     | 109136/225000 [231:09:24<168:20:51,  5.23s/it]
 49%|████▊     | 109137/225000 [231:09:27<148:57:58,  4.63s/it]
 49%|████▊     | 109138/225000 [231:09:37<196:04:09,  6.09s/it]
 49%|████▊     | 109139/225000 [231:09:39<161:58:20,  5.03s/it]
 49%|████▊     | 109140/225000 [231:09:48<198:01:49,  6.15s/it]
 49%|████▊     | 109141/225000 [231:09:51<166:41:45,  5.18s/it]
 49%|████▊     | 109142/225000 [231:10:02<224:43:11,  6.98s/it]
 49%|████▊     | 109143/225000 [231:10:05<185:25:48,  5.76s/it]
 49%|████▊     | 109144/225000 [231:10:07<150:26:16,  4.67s/it]
 49%|████▊     | 109145/225000 [231:10:18<210:43:41,  6.55s/it]
 49%|████▊     | 109146/225000 [231:10:26<218:41:10,  6.80s/it]
 49%|████▊     | 109147/225000 [231:10:28<174:50:38,  5.43s/it]
 49%|████▊     | 109148/225000 [231:10:37<215:27:07,  6.69s/it]
 49%|████▊     | 109149/225000 [231:10:40<172:02:01,  5.35s/it]
 49%|████▊     | 109150/225000 [231:10:44<164:18:14,  5.11s/it]
                                                               
{'loss': '0.8729', 'grad_norm': '2.669', 'learning_rate': '2.806e-05', 'epoch': '0.4851', 'num_input_tokens_seen': 387301464, 'train_runtime': '8.322e+05', 'train_tokens_per_second': '465.4'}

 49%|████▊     | 109150/225000 [231:10:44<164:18:14,  5.11s/it]
 49%|████▊     | 109151/225000 [231:10:56<226:52:59,  7.05s/it]
 49%|████▊     | 109152/225000 [231:11:01<210:51:57,  6.55s/it]
 49%|████▊     | 109153/225000 [231:11:09<220:25:55,  6.85s/it]
 49%|████▊     | 109154/225000 [231:11:11<175:31:53,  5.45s/it]
 49%|████▊     | 109155/225000 [231:11:14<151:53:37,  4.72s/it]
 49%|████▊     | 109156/225000 [231:11:19<153:30:18,  4.77s/it]
 49%|████▊     | 109157/225000 [231:11:22<137:31:37,  4.27s/it]
 49%|████▊     | 109158/225000 [231:11:28<149:50:59,  4.66s/it]
 49%|████▊     | 109159/225000 [231:11:31<134:45:39,  4.19s/it]
 49%|████▊     | 109160/225000 [231:11:36<150:59:43,  4.69s/it]
 49%|████▊     | 109161/225000 [231:11:41<149:48:44,  4.66s/it]
 49%|████▊     | 109162/225000 [231:11:46<154:47:37,  4.81s/it]
 49%|████▊     | 109163/225000 [231:11:55<192:47:15,  5.99s/it]
 49%|████▊     | 109164/225000 [231:12:01<196:40:31,  6.11s/it]
 49%|████▊     | 109165/225000 [231:12:08<200:19:42,  6.23s/it]
 49%|████▊     | 109166/225000 [231:12:10<161:15:26,  5.01s/it]
 49%|████▊     | 109167/225000 [231:12:13<141:27:36,  4.40s/it]
 49%|████▊     | 109168/225000 [231:12:19<155:47:52,  4.84s/it]
 49%|████▊     | 109169/225000 [231:12:23<150:17:06,  4.67s/it]
 49%|████▊     | 109170/225000 [231:12:26<133:31:20,  4.15s/it]
 49%|████▊     | 109171/225000 [231:12:35<178:41:41,  5.55s/it]
 49%|████▊     | 109172/225000 [231:12:37<147:26:22,  4.58s/it]
 49%|████▊     | 109173/225000 [231:12:40<126:15:08,  3.92s/it]
 49%|████▊     | 109174/225000 [231:12:42<111:56:01,  3.48s/it]
 49%|████▊     | 109175/225000 [231:12:45<107:57:20,  3.36s/it]
 49%|████▊     | 109176/225000 [231:12:51<128:36:02,  4.00s/it]
 49%|████▊     | 109177/225000 [231:12:53<113:36:39,  3.53s/it]
 49%|████▊     | 109178/225000 [231:12:56<103:39:17,  3.22s/it]
 49%|████▊     | 109179/225000 [231:13:07<180:09:45,  5.60s/it]
 49%|████▊     | 109180/225000 [231:13:09<151:15:28,  4.70s/it]
 49%|████▊     | 109181/225000 [231:13:12<136:13:14,  4.23s/it]
 49%|████▊     | 109182/225000 [231:13:18<148:06:34,  4.60s/it]
 49%|████▊     | 109183/225000 [231:13:21<133:15:43,  4.14s/it]
 49%|████▊     | 109184/225000 [231:13:23<117:09:44,  3.64s/it]
 49%|████▊     | 109185/225000 [231:13:36<200:24:00,  6.23s/it]
 49%|████▊     | 109186/225000 [231:13:38<162:51:10,  5.06s/it]
 49%|████▊     | 109187/225000 [231:13:43<164:22:52,  5.11s/it]
 49%|████▊     | 109188/225000 [231:13:46<140:14:11,  4.36s/it]
 49%|████▊     | 109189/225000 [231:13:54<177:39:47,  5.52s/it]
 49%|████▊     | 109190/225000 [231:14:03<205:51:43,  6.40s/it]
 49%|████▊     | 109191/225000 [231:14:05<168:03:33,  5.22s/it]
 49%|████▊     | 109192/225000 [231:14:08<142:00:50,  4.41s/it]
 49%|████▊     | 109193/225000 [231:14:11<131:10:30,  4.08s/it]
 49%|████▊     | 109194/225000 [231:14:13<116:36:59,  3.63s/it]
 49%|████▊     | 109195/225000 [231:14:24<184:56:14,  5.75s/it]
 49%|████▊     | 109196/225000 [231:14:29<179:36:13,  5.58s/it]
 49%|████▊     | 109197/225000 [231:14:33<156:27:58,  4.86s/it]
 49%|████▊     | 109198/225000 [231:14:35<133:03:59,  4.14s/it]
 49%|████▊     | 109199/225000 [231:14:38<123:58:11,  3.85s/it]
 49%|████▊     | 109200/225000 [231:14:45<150:37:00,  4.68s/it]
                                                               
{'loss': '0.9143', 'grad_norm': '4.614', 'learning_rate': '2.804e-05', 'epoch': '0.4853', 'num_input_tokens_seen': 387403192, 'train_runtime': '8.325e+05', 'train_tokens_per_second': '465.4'}

 49%|████▊     | 109200/225000 [231:14:45<150:37:00,  4.68s/it]
 49%|████▊     | 109201/225000 [231:14:48<135:51:40,  4.22s/it]
 49%|████▊     | 109202/225000 [231:14:51<125:13:08,  3.89s/it]
 49%|████▊     | 109203/225000 [231:15:01<185:58:17,  5.78s/it]
 49%|████▊     | 109204/225000 [231:15:04<159:39:32,  4.96s/it]
 49%|████▊     | 109205/225000 [231:15:08<145:00:08,  4.51s/it]
 49%|████▊     | 109206/225000 [231:15:14<164:48:43,  5.12s/it]
 49%|████▊     | 109207/225000 [231:15:19<164:03:32,  5.10s/it]
 49%|████▊     | 109208/225000 [231:15:24<162:59:32,  5.07s/it]
 49%|████▊     | 109209/225000 [231:15:34<205:09:42,  6.38s/it]
 49%|████▊     | 109210/225000 [231:15:39<194:50:47,  6.06s/it]
 49%|████▊     | 109211/225000 [231:15:42<166:24:27,  5.17s/it]
 49%|████▊     | 109212/225000 [231:15:45<140:32:25,  4.37s/it]
 49%|████▊     | 109213/225000 [231:15:47<121:56:53,  3.79s/it]
 49%|████▊     | 109214/225000 [231:15:50<116:08:39,  3.61s/it]
 49%|████▊     | 109215/225000 [231:15:56<140:07:44,  4.36s/it]
 49%|████▊     | 109216/225000 [231:16:02<150:09:07,  4.67s/it]
 49%|████▊     | 109217/225000 [231:16:07<156:08:19,  4.85s/it]
 49%|████▊     | 109218/225000 [231:16:20<236:11:03,  7.34s/it]
 49%|████▊     | 109219/225000 [231:16:22<185:54:41,  5.78s/it]
 49%|████▊     | 109220/225000 [231:16:25<155:27:39,  4.83s/it]
 49%|████▊     | 109221/225000 [231:16:37<227:39:53,  7.08s/it]
 49%|████▊     | 109222/225000 [231:16:40<180:12:59,  5.60s/it]
 49%|████▊     | 109223/225000 [231:16:46<184:46:34,  5.75s/it]
 49%|████▊     | 109224/225000 [231:16:51<177:21:39,  5.51s/it]
 49%|████▊     | 109225/225000 [231:16:54<153:44:03,  4.78s/it]
 49%|████▊     | 109226/225000 [231:16:56<129:46:53,  4.04s/it]
 49%|████▊     | 109227/225000 [231:16:59<120:01:41,  3.73s/it]
 49%|████▊     | 109228/225000 [231:17:08<170:50:11,  5.31s/it]
 49%|████▊     | 109229/225000 [231:17:16<193:21:16,  6.01s/it]
 49%|████▊     | 109230/225000 [231:17:18<156:35:25,  4.87s/it]
 49%|████▊     | 109231/225000 [231:17:28<203:30:06,  6.33s/it]
 49%|████▊     | 109232/225000 [231:17:32<187:13:17,  5.82s/it]
 49%|████▊     | 109233/225000 [231:17:35<159:47:05,  4.97s/it]
 49%|████▊     | 109234/225000 [231:17:38<134:34:55,  4.19s/it]
 49%|████▊     | 109235/225000 [231:17:40<117:11:14,  3.64s/it]
 49%|████▊     | 109236/225000 [231:17:49<172:26:01,  5.36s/it]
 49%|████▊     | 109237/225000 [231:17:52<149:26:42,  4.65s/it]
 49%|████▊     | 109238/225000 [231:17:55<133:43:13,  4.16s/it]
 49%|████▊     | 109239/225000 [231:17:58<115:32:34,  3.59s/it]
 49%|████▊     | 109240/225000 [231:18:00<101:47:08,  3.17s/it]
 49%|████▊     | 109241/225000 [231:18:02<94:35:31,  2.94s/it] 
 49%|████▊     | 109242/225000 [231:18:05<91:44:01,  2.85s/it]
 49%|████▊     | 109243/225000 [231:18:15<163:47:54,  5.09s/it]
 49%|████▊     | 109244/225000 [231:18:17<135:12:56,  4.21s/it]
 49%|████▊     | 109245/225000 [231:18:22<137:33:23,  4.28s/it]
 49%|████▊     | 109246/225000 [231:18:32<193:06:46,  6.01s/it]
 49%|████▊     | 109247/225000 [231:18:34<157:26:02,  4.90s/it]
 49%|████▊     | 109248/225000 [231:18:36<132:54:10,  4.13s/it]
 49%|████▊     | 109249/225000 [231:18:44<170:05:03,  5.29s/it]
 49%|████▊     | 109250/225000 [231:18:47<144:36:35,  4.50s/it]
                                                               
{'loss': '0.8835', 'grad_norm': '5.699', 'learning_rate': '2.803e-05', 'epoch': '0.4856', 'num_input_tokens_seen': 387503392, 'train_runtime': '8.327e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109250/225000 [231:18:47<144:36:35,  4.50s/it]
 49%|████▊     | 109251/225000 [231:19:01<236:05:13,  7.34s/it]
 49%|████▊     | 109252/225000 [231:19:08<233:56:18,  7.28s/it]
 49%|████▊     | 109253/225000 [231:19:11<187:12:31,  5.82s/it]
 49%|████▊     | 109254/225000 [231:19:16<181:36:50,  5.65s/it]
 49%|████▊     | 109255/225000 [231:19:18<150:28:58,  4.68s/it]
 49%|████▊     | 109256/225000 [231:19:24<157:09:21,  4.89s/it]
 49%|████▊     | 109257/225000 [231:19:30<169:18:13,  5.27s/it]
 49%|████▊     | 109258/225000 [231:19:32<142:26:44,  4.43s/it]
 49%|████▊     | 109259/225000 [231:19:35<125:52:09,  3.92s/it]
 49%|████▊     | 109260/225000 [231:19:37<111:23:43,  3.46s/it]
 49%|████▊     | 109261/225000 [231:19:43<129:53:58,  4.04s/it]
 49%|████▊     | 109262/225000 [231:19:50<157:49:20,  4.91s/it]
 49%|████▊     | 109263/225000 [231:19:52<134:49:13,  4.19s/it]
 49%|████▊     | 109264/225000 [231:19:55<120:18:18,  3.74s/it]
 49%|████▊     | 109265/225000 [231:19:57<107:40:41,  3.35s/it]
 49%|████▊     | 109266/225000 [231:20:01<107:01:02,  3.33s/it]
 49%|████▊     | 109267/225000 [231:20:03<98:02:09,  3.05s/it] 
 49%|████▊     | 109268/225000 [231:20:14<177:37:22,  5.53s/it]
 49%|████▊     | 109269/225000 [231:20:17<153:49:17,  4.78s/it]
 49%|████▊     | 109270/225000 [231:20:21<144:11:24,  4.49s/it]
 49%|████▊     | 109271/225000 [231:20:24<129:54:08,  4.04s/it]
 49%|████▊     | 109272/225000 [231:20:27<115:04:13,  3.58s/it]
 49%|████▊     | 109273/225000 [231:20:30<115:52:15,  3.60s/it]
 49%|████▊     | 109274/225000 [231:20:33<105:49:33,  3.29s/it]
 49%|████▊     | 109275/225000 [231:20:40<147:28:25,  4.59s/it]
 49%|████▊     | 109276/225000 [231:20:45<149:41:18,  4.66s/it]
 49%|████▊     | 109277/225000 [231:20:51<156:48:36,  4.88s/it]
 49%|████▊     | 109278/225000 [231:20:53<134:33:18,  4.19s/it]
 49%|████▊     | 109279/225000 [231:21:02<182:39:23,  5.68s/it]
 49%|████▊     | 109280/225000 [231:21:14<239:50:33,  7.46s/it]
 49%|████▊     | 109281/225000 [231:21:17<192:02:58,  5.97s/it]
 49%|████▊     | 109282/225000 [231:21:26<221:18:50,  6.89s/it]
 49%|████▊     | 109283/225000 [231:21:28<176:23:22,  5.49s/it]
 49%|████▊     | 109284/225000 [231:21:32<166:43:14,  5.19s/it]
 49%|████▊     | 109285/225000 [231:21:35<142:08:11,  4.42s/it]
 49%|████▊     | 109286/225000 [231:21:38<126:13:08,  3.93s/it]
 49%|████▊     | 109287/225000 [231:21:41<121:40:09,  3.79s/it]
 49%|████▊     | 109288/225000 [231:21:44<111:16:53,  3.46s/it]
 49%|████▊     | 109289/225000 [231:21:47<105:10:00,  3.27s/it]
 49%|████▊     | 109290/225000 [231:21:56<168:26:50,  5.24s/it]
 49%|████▊     | 109291/225000 [231:22:00<147:44:01,  4.60s/it]
 49%|████▊     | 109292/225000 [231:22:03<136:05:54,  4.23s/it]
 49%|████▊     | 109293/225000 [231:22:11<175:58:19,  5.48s/it]
 49%|████▊     | 109294/225000 [231:22:14<152:58:42,  4.76s/it]
 49%|████▊     | 109295/225000 [231:22:20<160:22:46,  4.99s/it]
 49%|████▊     | 109296/225000 [231:22:27<184:33:56,  5.74s/it]
 49%|████▊     | 109297/225000 [231:22:31<161:01:35,  5.01s/it]
 49%|████▊     | 109298/225000 [231:22:34<144:32:01,  4.50s/it]
 49%|████▊     | 109299/225000 [231:22:37<133:00:55,  4.14s/it]
 49%|████▊     | 109300/225000 [231:22:41<132:20:43,  4.12s/it]
                                                               
{'loss': '0.902', 'grad_norm': '2.811', 'learning_rate': '2.801e-05', 'epoch': '0.4858', 'num_input_tokens_seen': 387601896, 'train_runtime': '8.33e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109300/225000 [231:22:41<132:20:43,  4.12s/it]
 49%|████▊     | 109301/225000 [231:22:51<188:59:53,  5.88s/it]
 49%|████▊     | 109302/225000 [231:22:56<172:31:51,  5.37s/it]
 49%|████▊     | 109303/225000 [231:23:00<165:35:02,  5.15s/it]
 49%|████▊     | 109304/225000 [231:23:03<137:40:49,  4.28s/it]
 49%|████▊     | 109305/225000 [231:23:08<144:40:22,  4.50s/it]
 49%|████▊     | 109306/225000 [231:23:11<130:38:12,  4.06s/it]
 49%|████▊     | 109307/225000 [231:23:14<125:15:15,  3.90s/it]
 49%|████▊     | 109308/225000 [231:23:27<210:10:43,  6.54s/it]
 49%|████▊     | 109309/225000 [231:23:29<169:12:18,  5.27s/it]
 49%|████▊     | 109310/225000 [231:23:31<139:35:02,  4.34s/it]
 49%|████▊     | 109311/225000 [231:23:39<171:12:48,  5.33s/it]
 49%|████▊     | 109312/225000 [231:23:42<154:01:20,  4.79s/it]
 49%|████▊     | 109313/225000 [231:23:47<156:37:39,  4.87s/it]
 49%|████▊     | 109314/225000 [231:23:53<165:57:08,  5.16s/it]
 49%|████▊     | 109315/225000 [231:24:01<192:32:16,  5.99s/it]
 49%|████▊     | 109316/225000 [231:24:03<156:21:20,  4.87s/it]
 49%|████▊     | 109317/225000 [231:24:06<137:50:52,  4.29s/it]
 49%|████▊     | 109318/225000 [231:24:10<132:30:43,  4.12s/it]
 49%|████▊     | 109319/225000 [231:24:14<128:47:38,  4.01s/it]
 49%|████▊     | 109320/225000 [231:24:19<137:07:50,  4.27s/it]
 49%|████▊     | 109321/225000 [231:24:26<169:55:07,  5.29s/it]
 49%|████▊     | 109322/225000 [231:24:31<161:39:12,  5.03s/it]
 49%|████▊     | 109323/225000 [231:24:34<145:41:19,  4.53s/it]
 49%|████▊     | 109324/225000 [231:24:38<134:52:45,  4.20s/it]
 49%|████▊     | 109325/225000 [231:24:41<127:04:51,  3.95s/it]
 49%|████▊     | 109326/225000 [231:24:44<121:56:29,  3.80s/it]
 49%|████▊     | 109327/225000 [231:24:52<158:29:21,  4.93s/it]
 49%|████▊     | 109328/225000 [231:24:54<131:41:44,  4.10s/it]
 49%|████▊     | 109329/225000 [231:24:56<113:41:31,  3.54s/it]
 49%|████▊     | 109330/225000 [231:25:04<153:46:47,  4.79s/it]
 49%|████▊     | 109331/225000 [231:25:10<160:58:21,  5.01s/it]
 49%|████▊     | 109332/225000 [231:25:17<178:44:58,  5.56s/it]
 49%|████▊     | 109333/225000 [231:25:24<195:16:39,  6.08s/it]
 49%|████▊     | 109334/225000 [231:25:29<187:08:44,  5.82s/it]
 49%|████▊     | 109335/225000 [231:25:31<153:32:35,  4.78s/it]
 49%|████▊     | 109336/225000 [231:25:40<190:26:52,  5.93s/it]
 49%|████▊     | 109337/225000 [231:25:45<179:25:49,  5.58s/it]
 49%|████▊     | 109338/225000 [231:25:51<186:28:56,  5.80s/it]
 49%|████▊     | 109339/225000 [231:25:54<155:55:32,  4.85s/it]
 49%|████▊     | 109340/225000 [231:26:07<234:02:42,  7.28s/it]
 49%|████▊     | 109341/225000 [231:26:13<228:56:49,  7.13s/it]
 49%|████▊     | 109342/225000 [231:26:16<183:40:30,  5.72s/it]
 49%|████▊     | 109343/225000 [231:26:18<151:11:36,  4.71s/it]
 49%|████▊     | 109344/225000 [231:26:24<165:16:07,  5.14s/it]
 49%|████▊     | 109345/225000 [231:26:36<223:48:20,  6.97s/it]
 49%|████▊     | 109346/225000 [231:26:39<190:30:54,  5.93s/it]
 49%|████▊     | 109347/225000 [231:26:42<160:40:05,  5.00s/it]
 49%|████▊     | 109348/225000 [231:26:44<135:19:33,  4.21s/it]
 49%|████▊     | 109349/225000 [231:26:49<142:58:19,  4.45s/it]
 49%|████▊     | 109350/225000 [231:26:54<149:21:05,  4.65s/it]
                                                               
{'loss': '0.919', 'grad_norm': '2.732', 'learning_rate': '2.799e-05', 'epoch': '0.486', 'num_input_tokens_seen': 387705536, 'train_runtime': '8.332e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109350/225000 [231:26:54<149:21:05,  4.65s/it]
 49%|████▊     | 109351/225000 [231:26:58<134:29:38,  4.19s/it]
 49%|████▊     | 109352/225000 [231:27:00<121:18:25,  3.78s/it]
 49%|████▊     | 109353/225000 [231:27:11<190:46:34,  5.94s/it]
 49%|████▊     | 109354/225000 [231:27:17<185:20:54,  5.77s/it]
 49%|████▊     | 109355/225000 [231:27:20<159:41:09,  4.97s/it]
 49%|████▊     | 109356/225000 [231:27:25<163:31:42,  5.09s/it]
 49%|████▊     | 109357/225000 [231:27:28<137:18:09,  4.27s/it]
 49%|████▊     | 109358/225000 [231:27:30<119:09:47,  3.71s/it]
 49%|████▊     | 109359/225000 [231:27:35<135:58:15,  4.23s/it]
 49%|████▊     | 109360/225000 [231:27:42<156:06:26,  4.86s/it]
 49%|████▊     | 109361/225000 [231:27:46<149:43:39,  4.66s/it]
 49%|████▊     | 109362/225000 [231:27:51<149:37:11,  4.66s/it]
 49%|████▊     | 109363/225000 [231:27:59<183:39:34,  5.72s/it]
 49%|████▊     | 109364/225000 [231:28:04<181:17:05,  5.64s/it]
 49%|████▊     | 109365/225000 [231:28:07<155:42:56,  4.85s/it]
 49%|████▊     | 109366/225000 [231:28:09<130:30:24,  4.06s/it]
 49%|████▊     | 109367/225000 [231:28:12<113:32:45,  3.54s/it]
 49%|████▊     | 109368/225000 [231:28:14<105:35:22,  3.29s/it]
 49%|████▊     | 109369/225000 [231:28:18<106:48:25,  3.33s/it]
 49%|████▊     | 109370/225000 [231:28:21<106:53:19,  3.33s/it]
 49%|████▊     | 109371/225000 [231:28:31<164:34:36,  5.12s/it]
 49%|████▊     | 109372/225000 [231:28:41<217:34:25,  6.77s/it]
 49%|████▊     | 109373/225000 [231:28:44<181:26:18,  5.65s/it]
 49%|████▊     | 109374/225000 [231:28:53<212:12:42,  6.61s/it]
 49%|████▊     | 109375/225000 [231:28:55<171:32:09,  5.34s/it]
 49%|████▊     | 109376/225000 [231:28:58<140:19:05,  4.37s/it]
 49%|████▊     | 109377/225000 [231:29:01<127:17:46,  3.96s/it]
 49%|████▊     | 109378/225000 [231:29:04<117:44:03,  3.67s/it]
 49%|████▊     | 109379/225000 [231:29:10<144:46:09,  4.51s/it]
 49%|████▊     | 109380/225000 [231:29:13<133:50:25,  4.17s/it]
 49%|████▊     | 109381/225000 [231:29:17<133:26:29,  4.15s/it]
 49%|████▊     | 109382/225000 [231:29:27<186:24:00,  5.80s/it]
 49%|████▊     | 109383/225000 [231:29:30<159:20:36,  4.96s/it]
 49%|████▊     | 109384/225000 [231:29:33<135:10:35,  4.21s/it]
 49%|████▊     | 109385/225000 [231:29:36<123:23:58,  3.84s/it]
 49%|████▊     | 109386/225000 [231:29:49<220:20:34,  6.86s/it]
 49%|████▊     | 109387/225000 [231:29:52<178:33:01,  5.56s/it]
 49%|████▊     | 109388/225000 [231:29:57<169:18:30,  5.27s/it]
 49%|████▊     | 109389/225000 [231:29:59<139:55:06,  4.36s/it]
 49%|████▊     | 109390/225000 [231:30:02<126:57:49,  3.95s/it]
 49%|████▊     | 109391/225000 [231:30:12<190:19:40,  5.93s/it]
 49%|████▊     | 109392/225000 [231:30:15<154:45:13,  4.82s/it]
 49%|████▊     | 109393/225000 [231:30:18<137:47:32,  4.29s/it]
 49%|████▊     | 109394/225000 [231:30:20<118:01:13,  3.68s/it]
 49%|████▊     | 109395/225000 [231:30:29<169:47:26,  5.29s/it]
 49%|████▊     | 109396/225000 [231:30:32<148:59:55,  4.64s/it]
 49%|████▊     | 109397/225000 [231:30:34<127:20:06,  3.97s/it]
 49%|████▊     | 109398/225000 [231:30:38<119:16:25,  3.71s/it]
 49%|████▊     | 109399/225000 [231:30:40<106:00:11,  3.30s/it]
 49%|████▊     | 109400/225000 [231:30:43<100:17:20,  3.12s/it]
                                                               
{'loss': '0.9109', 'grad_norm': '3.94', 'learning_rate': '2.797e-05', 'epoch': '0.4862', 'num_input_tokens_seen': 387803048, 'train_runtime': '8.334e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109400/225000 [231:30:43<100:17:20,  3.12s/it]
 49%|████▊     | 109401/225000 [231:30:53<165:32:52,  5.16s/it]
 49%|████▊     | 109402/225000 [231:30:56<151:09:10,  4.71s/it]
 49%|████▊     | 109403/225000 [231:31:01<153:41:39,  4.79s/it]
 49%|████▊     | 109404/225000 [231:31:04<136:50:37,  4.26s/it]
 49%|████▊     | 109405/225000 [231:31:14<191:28:09,  5.96s/it]
 49%|████▊     | 109406/225000 [231:31:18<173:35:14,  5.41s/it]
 49%|████▊     | 109407/225000 [231:31:20<142:25:52,  4.44s/it]
 49%|████▊     | 109408/225000 [231:31:26<152:49:32,  4.76s/it]
 49%|████▊     | 109409/225000 [231:31:28<128:57:16,  4.02s/it]
 49%|████▊     | 109410/225000 [231:31:31<115:03:16,  3.58s/it]
 49%|████▊     | 109411/225000 [231:31:37<136:55:45,  4.26s/it]
 49%|████▊     | 109412/225000 [231:31:41<136:37:44,  4.26s/it]
 49%|████▊     | 109413/225000 [231:31:49<174:53:13,  5.45s/it]
 49%|████▊     | 109414/225000 [231:31:51<144:12:27,  4.49s/it]
 49%|████▊     | 109415/225000 [231:31:55<137:43:31,  4.29s/it]
 49%|████▊     | 109416/225000 [231:32:07<208:49:12,  6.50s/it]
 49%|████▊     | 109417/225000 [231:32:09<167:00:24,  5.20s/it]
 49%|████▊     | 109418/225000 [231:32:11<139:18:48,  4.34s/it]
 49%|████▊     | 109419/225000 [231:32:14<121:49:23,  3.79s/it]
 49%|████▊     | 109420/225000 [231:32:16<109:23:15,  3.41s/it]
 49%|████▊     | 109421/225000 [231:32:19<100:56:51,  3.14s/it]
 49%|████▊     | 109422/225000 [231:32:22<102:07:02,  3.18s/it]
 49%|████▊     | 109423/225000 [231:32:30<146:52:43,  4.57s/it]
 49%|████▊     | 109424/225000 [231:32:33<127:56:17,  3.99s/it]
 49%|████▊     | 109425/225000 [231:32:42<183:12:52,  5.71s/it]
 49%|████▊     | 109426/225000 [231:32:51<213:53:35,  6.66s/it]
 49%|████▊     | 109427/225000 [231:32:53<170:08:53,  5.30s/it]
 49%|████▊     | 109428/225000 [231:32:56<147:40:01,  4.60s/it]
 49%|████▊     | 109429/225000 [231:33:07<202:17:20,  6.30s/it]
 49%|████▊     | 109430/225000 [231:33:11<187:14:13,  5.83s/it]
 49%|████▊     | 109431/225000 [231:33:14<161:10:58,  5.02s/it]
 49%|████▊     | 109432/225000 [231:33:17<134:34:31,  4.19s/it]
 49%|████▊     | 109433/225000 [231:33:26<186:19:39,  5.80s/it]
 49%|████▊     | 109434/225000 [231:33:29<151:59:04,  4.73s/it]
 49%|████▊     | 109435/225000 [231:33:36<182:56:40,  5.70s/it]
 49%|████▊     | 109436/225000 [231:33:47<232:07:23,  7.23s/it]
 49%|████▊     | 109437/225000 [231:33:51<198:51:31,  6.19s/it]
 49%|████▊     | 109438/225000 [231:33:57<193:36:23,  6.03s/it]
 49%|████▊     | 109439/225000 [231:34:05<212:17:33,  6.61s/it]
 49%|████▊     | 109440/225000 [231:34:10<197:31:06,  6.15s/it]
 49%|████▊     | 109441/225000 [231:34:12<160:25:56,  5.00s/it]
 49%|████▊     | 109442/225000 [231:34:15<142:32:04,  4.44s/it]
 49%|████▊     | 109443/225000 [231:34:18<122:23:46,  3.81s/it]
 49%|████▊     | 109444/225000 [231:34:28<184:23:39,  5.74s/it]
 49%|████▊     | 109445/225000 [231:34:33<177:40:01,  5.54s/it]
 49%|████▊     | 109446/225000 [231:34:38<178:47:38,  5.57s/it]
 49%|████▊     | 109447/225000 [231:34:41<147:13:23,  4.59s/it]
 49%|████▊     | 109448/225000 [231:34:44<132:19:50,  4.12s/it]
 49%|████▊     | 109449/225000 [231:34:58<232:05:51,  7.23s/it]
 49%|████▊     | 109450/225000 [231:35:01<185:46:03,  5.79s/it]
                                                               
{'loss': '0.9653', 'grad_norm': '4.367', 'learning_rate': '2.795e-05', 'epoch': '0.4864', 'num_input_tokens_seen': 387908080, 'train_runtime': '8.337e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109450/225000 [231:35:01<185:46:03,  5.79s/it]
 49%|████▊     | 109451/225000 [231:35:03<154:24:07,  4.81s/it]
 49%|████▊     | 109452/225000 [231:35:15<220:16:00,  6.86s/it]
 49%|████▊     | 109453/225000 [231:35:18<185:23:38,  5.78s/it]
 49%|████▊     | 109454/225000 [231:35:21<152:56:25,  4.77s/it]
 49%|████▊     | 109455/225000 [231:35:24<138:06:57,  4.30s/it]
 49%|████▊     | 109456/225000 [231:35:26<121:14:00,  3.78s/it]
 49%|████▊     | 109457/225000 [231:35:35<169:58:55,  5.30s/it]
 49%|████▊     | 109458/225000 [231:35:38<142:23:03,  4.44s/it]
 49%|████▊     | 109459/225000 [231:35:40<120:56:00,  3.77s/it]
 49%|████▊     | 109460/225000 [231:35:43<118:09:13,  3.68s/it]
 49%|████▊     | 109461/225000 [231:35:53<173:48:34,  5.42s/it]
 49%|████▊     | 109462/225000 [231:35:55<144:00:05,  4.49s/it]
 49%|████▊     | 109463/225000 [231:36:00<148:47:53,  4.64s/it]
 49%|████▊     | 109464/225000 [231:36:09<192:01:24,  5.98s/it]
 49%|████▊     | 109465/225000 [231:36:11<155:46:27,  4.85s/it]
 49%|████▊     | 109466/225000 [231:36:14<130:03:17,  4.05s/it]
 49%|████▊     | 109467/225000 [231:36:23<180:11:09,  5.61s/it]
 49%|████▊     | 109468/225000 [231:36:25<150:19:39,  4.68s/it]
 49%|████▊     | 109469/225000 [231:36:32<170:03:10,  5.30s/it]
 49%|████▊     | 109470/225000 [231:36:35<146:33:04,  4.57s/it]
 49%|████▊     | 109471/225000 [231:36:43<180:44:10,  5.63s/it]
 49%|████▊     | 109472/225000 [231:36:46<150:16:27,  4.68s/it]
 49%|████▊     | 109473/225000 [231:36:48<129:53:02,  4.05s/it]
 49%|████▊     | 109474/225000 [231:36:51<116:00:28,  3.62s/it]
 49%|████▊     | 109475/225000 [231:36:54<113:09:56,  3.53s/it]
 49%|████▊     | 109476/225000 [231:36:57<103:45:35,  3.23s/it]
 49%|████▊     | 109477/225000 [231:36:59<97:45:52,  3.05s/it] 
 49%|████▊     | 109478/225000 [231:37:08<154:58:48,  4.83s/it]
 49%|████▊     | 109479/225000 [231:37:14<160:18:06,  5.00s/it]
 49%|████▊     | 109480/225000 [231:37:16<134:25:57,  4.19s/it]
 49%|████▊     | 109481/225000 [231:37:18<116:12:27,  3.62s/it]
 49%|████▊     | 109482/225000 [231:37:29<184:14:14,  5.74s/it]
 49%|████▊     | 109483/225000 [231:37:32<158:24:31,  4.94s/it]
 49%|████▊     | 109484/225000 [231:37:38<166:42:35,  5.20s/it]
 49%|████▊     | 109485/225000 [231:37:40<138:57:43,  4.33s/it]
 49%|████▊     | 109486/225000 [231:37:47<161:47:47,  5.04s/it]
 49%|████▊     | 109487/225000 [231:37:50<147:50:17,  4.61s/it]
 49%|████▊     | 109488/225000 [231:37:53<126:11:03,  3.93s/it]
 49%|████▊     | 109489/225000 [231:37:55<114:18:42,  3.56s/it]
 49%|████▊     | 109490/225000 [231:37:58<105:51:15,  3.30s/it]
 49%|████▊     | 109491/225000 [231:38:08<173:03:21,  5.39s/it]
 49%|████▊     | 109492/225000 [231:38:11<147:03:10,  4.58s/it]
 49%|████▊     | 109493/225000 [231:38:14<128:52:31,  4.02s/it]
 49%|████▊     | 109494/225000 [231:38:16<115:47:41,  3.61s/it]
 49%|████▊     | 109495/225000 [231:38:19<106:54:35,  3.33s/it]
 49%|████▊     | 109496/225000 [231:38:22<99:08:06,  3.09s/it] 
 49%|████▊     | 109497/225000 [231:38:24<95:55:51,  2.99s/it]
 49%|████▊     | 109498/225000 [231:38:27<92:56:14,  2.90s/it]
 49%|████▊     | 109499/225000 [231:38:37<161:50:00,  5.04s/it]
 49%|████▊     | 109500/225000 [231:38:40<138:09:25,  4.31s/it]
                                                               
{'loss': '0.907', 'grad_norm': '5.064', 'learning_rate': '2.793e-05', 'epoch': '0.4867', 'num_input_tokens_seen': 387998576, 'train_runtime': '8.339e+05', 'train_tokens_per_second': '465.3'}

 49%|████▊     | 109500/225000 [231:38:40<138:09:25,  4.31s/it]
 49%|████▊     | 109501/225000 [231:38:47<168:39:11,  5.26s/it]
 49%|████▊     | 109502/225000 [231:38:52<169:03:00,  5.27s/it]
 49%|████▊     | 109503/225000 [231:38:59<177:31:52,  5.53s/it]
 49%|████▊     | 109504/225000 [231:39:03<170:11:05,  5.30s/it]
 49%|████▊     | 109505/225000 [231:39:08<163:29:48,  5.10s/it]
 49%|████▊     | 109506/225000 [231:39:17<204:12:54,  6.37s/it]
 49%|████▊     | 109507/225000 [231:39:21<178:30:18,  5.56s/it]
 49%|████▊     | 109508/225000 [231:39:24<157:07:51,  4.90s/it]
 49%|████▊     | 109509/225000 [231:39:28<147:59:14,  4.61s/it]
 49%|████▊     | 109510/225000 [231:39:32<135:10:23,  4.21s/it]
 49%|████▊     | 109511/225000 [231:39:41<181:30:48,  5.66s/it]
 49%|████▊     | 109512/225000 [231:39:44<156:01:51,  4.86s/it]
 49%|████▊     | 109513/225000 [231:39:47<140:35:13,  4.38s/it]
 49%|████▊     | 109514/225000 [231:39:50<129:27:06,  4.04s/it]
 49%|████▊     | 109515/225000 [231:39:57<156:29:54,  4.88s/it]
 49%|████▊     | 109516/225000 [231:40:00<139:38:53,  4.35s/it]
 49%|████▊     | 109517/225000 [231:40:09<187:59:57,  5.86s/it]
 49%|████▊     | 109518/225000 [231:40:12<153:47:24,  4.79s/it]
 49%|████▊     | 109519/225000 [231:40:14<129:22:32,  4.03s/it]
 49%|████▊     | 109520/225000 [231:40:16<112:00:26,  3.49s/it]
 49%|████▊     | 109521/225000 [231:40:18<99:23:55,  3.10s/it] 
 49%|████▊     | 109522/225000 [231:40:24<123:38:06,  3.85s/it]
 49%|████▊     | 109523/225000 [231:40:28<123:23:19,  3.85s/it]
 49%|████▊     | 109524/225000 [231:40:37<174:31:18,  5.44s/it]
 49%|████▊     | 109525/225000 [231:40:39<144:32:58,  4.51s/it]
 49%|████▊     | 109526/225000 [231:40:42<122:59:40,  3.83s/it]
 49%|████▊     | 109527/225000 [231:40:44<107:59:07,  3.37s/it]
 49%|████▊     | 109528/225000 [231:40:46<97:12:20,  3.03s/it] 
 49%|████▊     | 109529/225000 [231:40:56<166:31:07,  5.19s/it]
 49%|████▊     | 109530/225000 [231:41:06<207:02:23,  6.45s/it]
 49%|████▊     | 109531/225000 [231:41:08<167:35:38,  5.23s/it]
 49%|████▊     | 109532/225000 [231:41:11<146:21:44,  4.56s/it]
 49%|████▊     | 109533/225000 [231:41:15<137:10:18,  4.28s/it]
 49%|████▊     | 109534/225000 [231:41:18<131:55:10,  4.11s/it]
 49%|████▊     | 109535/225000 [231:41:22<121:35:36,  3.79s/it]
 49%|████▊     | 109536/225000 [231:41:25<114:28:09,  3.57s/it]
 49%|████▊     | 109537/225000 [231:41:31<145:50:22,  4.55s/it]
 49%|████▊     | 109538/225000 [231:41:39<172:17:44,  5.37s/it]
 49%|████▊     | 109539/225000 [231:41:41<144:13:34,  4.50s/it]
 49%|████▊     | 109540/225000 [231:41:43<122:12:52,  3.81s/it]
 49%|████▊     | 109541/225000 [231:41:52<167:46:23,  5.23s/it]
 49%|████▊     | 109542/225000 [231:42:03<227:28:08,  7.09s/it]
 49%|████▊     | 109543/225000 [231:42:05<179:57:20,  5.61s/it]
 49%|████▊     | 109544/225000 [231:42:17<234:30:48,  7.31s/it]
 49%|████▊     | 109545/225000 [231:42:19<185:07:02,  5.77s/it]
 49%|████▊     | 109546/225000 [231:42:25<183:33:19,  5.72s/it]
 49%|████▊     | 109547/225000 [231:42:29<174:42:45,  5.45s/it]
 49%|████▊     | 109548/225000 [231:42:32<151:25:00,  4.72s/it]
 49%|████▊     | 109549/225000 [231:42:40<175:35:49,  5.48s/it]
 49%|████▊     | 109550/225000 [231:42:46<188:42:03,  5.88s/it]
                                                               
{'loss': '0.8868', 'grad_norm': '4.62', 'learning_rate': '2.792e-05', 'epoch': '0.4869', 'num_input_tokens_seen': 388092392, 'train_runtime': '8.342e+05', 'train_tokens_per_second': '465.2'}

 49%|████▊     | 109550/225000 [231:42:46<188:42:03,  5.88s/it]
 49%|████▊     | 109551/225000 [231:42:52<181:04:02,  5.65s/it]
 49%|████▊     | 109552/225000 [231:42:55<157:31:32,  4.91s/it]
 49%|████▊     | 109553/225000 [231:42:58<143:02:10,  4.46s/it]
 49%|████▊     | 109554/225000 [231:43:02<132:39:00,  4.14s/it]
 49%|████▊     | 109555/225000 [231:43:11<186:09:57,  5.81s/it]
 49%|████▊     | 109556/225000 [231:43:17<185:58:54,  5.80s/it]
 49%|████▊     | 109557/225000 [231:43:19<152:18:04,  4.75s/it]
 49%|████▊     | 109558/225000 [231:43:27<181:39:17,  5.66s/it]
 49%|████▊     | 109559/225000 [231:43:32<177:37:09,  5.54s/it]
 49%|████▊     | 109560/225000 [231:43:38<182:59:49,  5.71s/it]
 49%|████▊     | 109561/225000 [231:43:43<175:35:54,  5.48s/it]
 49%|████▊     | 109562/225000 [231:43:57<251:58:37,  7.86s/it]
 49%|████▊     | 109563/225000 [231:44:03<237:34:50,  7.41s/it]
 49%|████▊     | 109564/225000 [231:44:06<194:34:09,  6.07s/it]
 49%|████▊     | 109565/225000 [231:44:12<194:14:31,  6.06s/it]
 49%|████▊     | 109566/225000 [231:44:20<215:00:20,  6.71s/it]
 49%|████▊     | 109567/225000 [231:44:26<203:55:28,  6.36s/it]
 49%|████▊     | 109568/225000 [231:44:28<163:42:17,  5.11s/it]
 49%|████▊     | 109569/225000 [231:44:31<137:57:11,  4.30s/it]
 49%|████▊     | 109570/225000 [231:44:33<121:11:23,  3.78s/it]
 49%|████▊     | 109571/225000 [231:44:36<109:29:47,  3.41s/it]
 49%|████▊     | 109572/225000 [231:44:38<100:34:57,  3.14s/it]
 49%|████▊     | 109573/225000 [231:44:48<161:57:46,  5.05s/it]
 49%|████▊     | 109574/225000 [231:44:53<167:42:31,  5.23s/it]
 49%|████▊     | 109575/225000 [231:44:56<139:35:52,  4.35s/it]
 49%|████▊     | 109576/225000 [231:44:58<119:53:29,  3.74s/it]
 49%|████▊     | 109577/225000 [231:45:00<106:30:36,  3.32s/it]
 49%|████▊     | 109578/225000 [231:45:11<181:54:49,  5.67s/it]
 49%|████▊     | 109579/225000 [231:45:15<162:13:59,  5.06s/it]
 49%|████▊     | 109580/225000 [231:45:18<141:07:02,  4.40s/it]
 49%|████▊     | 109581/225000 [231:45:20<120:13:24,  3.75s/it]
 49%|████▊     | 109582/225000 [231:45:22<105:40:45,  3.30s/it]
 49%|████▊     | 109583/225000 [231:45:25<95:54:15,  2.99s/it] 
 49%|████▊     | 109584/225000 [231:45:30<116:00:34,  3.62s/it]
 49%|████▊     | 109585/225000 [231:45:34<117:23:54,  3.66s/it]
 49%|████▊     | 109586/225000 [231:45:47<213:35:53,  6.66s/it]
 49%|████▊     | 109587/225000 [231:45:49<171:35:25,  5.35s/it]
 49%|████▊     | 109588/225000 [231:45:53<149:43:06,  4.67s/it]
 49%|████▊     | 109589/225000 [231:45:55<127:00:48,  3.96s/it]
 49%|████▊     | 109590/225000 [231:46:04<172:19:36,  5.38s/it]
 49%|████▊     | 109591/225000 [231:46:14<222:46:00,  6.95s/it]
 49%|████▊     | 109592/225000 [231:46:17<184:27:37,  5.75s/it]
 49%|████▊     | 109593/225000 [231:46:19<150:22:18,  4.69s/it]
 49%|████▊     | 109594/225000 [231:46:22<128:32:25,  4.01s/it]
 49%|████▊     | 109595/225000 [231:46:24<115:27:43,  3.60s/it]
 49%|████▊     | 109596/225000 [231:46:35<184:36:47,  5.76s/it]
 49%|████▊     | 109597/225000 [231:46:41<181:11:41,  5.65s/it]
 49%|████▊     | 109598/225000 [231:46:43<148:59:02,  4.65s/it]
 49%|████▊     | 109599/225000 [231:46:45<128:22:25,  4.00s/it]
 49%|████▊     | 109600/225000 [231:46:48<114:30:05,  3.57s/it]
                                                               
{'loss': '0.8112', 'grad_norm': '6.595', 'learning_rate': '2.79e-05', 'epoch': '0.4871', 'num_input_tokens_seen': 388195896, 'train_runtime': '8.344e+05', 'train_tokens_per_second': '465.2'}

 49%|████▊     | 109600/225000 [231:46:48<114:30:05,  3.57s/it]
 49%|████▊     | 109601/225000 [231:46:51<109:56:52,  3.43s/it]
 49%|████▊     | 109602/225000 [231:46:55<119:02:40,  3.71s/it]
 49%|████▊     | 109603/225000 [231:47:02<143:16:48,  4.47s/it]
 49%|████▊     | 109604/225000 [231:47:05<130:50:40,  4.08s/it]
 49%|████▊     | 109605/225000 [231:47:09<130:20:36,  4.07s/it]
 49%|████▊     | 109606/225000 [231:47:11<115:12:08,  3.59s/it]
 49%|████▊     | 109607/225000 [231:47:19<152:31:41,  4.76s/it]
 49%|████▊     | 109608/225000 [231:47:27<186:26:34,  5.82s/it]
 49%|████▊     | 109609/225000 [231:47:31<166:56:55,  5.21s/it]
 49%|████▊     | 109610/225000 [231:47:34<145:28:22,  4.54s/it]
 49%|████▊     | 109611/225000 [231:47:41<167:51:00,  5.24s/it]
 49%|████▊     | 109612/225000 [231:47:43<143:00:25,  4.46s/it]
 49%|████▊     | 109613/225000 [231:47:46<123:28:19,  3.85s/it]
 49%|████▊     | 109614/225000 [231:47:49<116:54:17,  3.65s/it]
 49%|████▊     | 109615/225000 [231:47:51<105:30:42,  3.29s/it]
 49%|████▊     | 109616/225000 [231:47:54<97:50:44,  3.05s/it] 
 49%|████▊     | 109617/225000 [231:47:57<98:55:00,  3.09s/it]
 49%|████▊     | 109618/225000 [231:48:00<99:58:05,  3.12s/it]
 49%|████▊     | 109619/225000 [231:48:09<149:19:21,  4.66s/it]
 49%|████▊     | 109620/225000 [231:48:11<128:26:25,  4.01s/it]
 49%|████▊     | 109621/225000 [231:48:14<120:05:33,  3.75s/it]
 49%|████▊     | 109622/225000 [231:48:20<142:03:38,  4.43s/it]
 49%|████▊     | 109623/225000 [231:48:31<199:42:57,  6.23s/it]
 49%|████▊     | 109624/225000 [231:48:36<192:20:59,  6.00s/it]
 49%|████▊     | 109625/225000 [231:48:40<169:01:02,  5.27s/it]
 49%|████▊     | 109626/225000 [231:48:46<178:04:41,  5.56s/it]
 49%|████▊     | 109627/225000 [231:48:52<184:22:38,  5.75s/it]
 49%|████▊     | 109628/225000 [231:49:02<225:59:07,  7.05s/it]
 49%|████▊     | 109629/225000 [231:49:09<221:23:53,  6.91s/it]
 49%|████▊     | 109630/225000 [231:49:20<259:15:55,  8.09s/it]
 49%|████▊     | 109631/225000 [231:49:31<288:54:28,  9.02s/it]
 49%|████▊     | 109632/225000 [231:49:33<225:09:56,  7.03s/it]
 49%|████▊     | 109633/225000 [231:49:37<193:10:08,  6.03s/it]
 49%|████▊     | 109634/225000 [231:49:45<209:54:12,  6.55s/it]
 49%|████▊     | 109635/225000 [231:49:47<167:47:22,  5.24s/it]
 49%|████▊     | 109636/225000 [231:49:50<146:38:03,  4.58s/it]
 49%|████▊     | 109637/225000 [231:49:52<125:13:22,  3.91s/it]
 49%|████▊     | 109638/225000 [231:49:55<118:38:39,  3.70s/it]
 49%|████▊     | 109639/225000 [231:49:59<118:46:16,  3.71s/it]
 49%|████▊     | 109640/225000 [231:50:03<119:23:02,  3.73s/it]
 49%|████▊     | 109641/225000 [231:50:07<119:11:12,  3.72s/it]
 49%|████▊     | 109642/225000 [231:50:10<114:10:51,  3.56s/it]
 49%|████▊     | 109643/225000 [231:50:20<172:58:26,  5.40s/it]
 49%|████▊     | 109644/225000 [231:50:22<141:58:12,  4.43s/it]
 49%|████▊     | 109645/225000 [231:50:24<119:57:28,  3.74s/it]
 49%|████▊     | 109646/225000 [231:50:26<105:13:55,  3.28s/it]
 49%|████▊     | 109647/225000 [231:50:28<95:01:22,  2.97s/it] 
 49%|████▊     | 109648/225000 [231:50:36<139:10:52,  4.34s/it]
 49%|████▊     | 109649/225000 [231:50:44<172:16:57,  5.38s/it]
 49%|████▊     | 109650/225000 [231:50:52<198:38:34,  6.20s/it]
                                                               
{'loss': '0.9178', 'grad_norm': '3.445', 'learning_rate': '2.788e-05', 'epoch': '0.4873', 'num_input_tokens_seen': 388299912, 'train_runtime': '8.347e+05', 'train_tokens_per_second': '465.2'}

 49%|████▊     | 109650/225000 [231:50:52<198:38:34,  6.20s/it]
 49%|████▊     | 109651/225000 [231:50:58<195:13:40,  6.09s/it]
 49%|████▊     | 109652/225000 [231:51:00<161:07:19,  5.03s/it]
 49%|████▊     | 109653/225000 [231:51:02<134:48:42,  4.21s/it]
 49%|████▊     | 109654/225000 [231:51:05<120:02:24,  3.75s/it]
 49%|████▊     | 109655/225000 [231:51:15<175:36:58,  5.48s/it]
 49%|████▊     | 109656/225000 [231:51:17<144:19:01,  4.50s/it]
 49%|████▊     | 109657/225000 [231:51:20<129:42:16,  4.05s/it]
 49%|████▊     | 109658/225000 [231:51:22<111:59:01,  3.50s/it]
 49%|████▊     | 109659/225000 [231:51:25<107:52:58,  3.37s/it]
 49%|████▊     | 109660/225000 [231:51:30<118:01:34,  3.68s/it]
 49%|████▊     | 109661/225000 [231:51:33<113:21:55,  3.54s/it]
 49%|████▊     | 109662/225000 [231:51:43<177:37:09,  5.54s/it]
 49%|████▊     | 109663/225000 [231:51:48<175:11:15,  5.47s/it]
 49%|████▊     | 109664/225000 [231:51:51<148:27:51,  4.63s/it]
 49%|████▊     | 109665/225000 [231:51:54<137:55:27,  4.31s/it]
 49%|████▊     | 109666/225000 [231:51:58<126:10:42,  3.94s/it]
 49%|████▊     | 109667/225000 [231:52:07<182:06:09,  5.68s/it]
 49%|████▊     | 109668/225000 [231:52:10<154:03:31,  4.81s/it]
 49%|████▊     | 109669/225000 [231:52:12<129:21:31,  4.04s/it]
 49%|████▊     | 109670/225000 [231:52:19<150:40:16,  4.70s/it]
 49%|████▊     | 109671/225000 [231:52:21<127:05:46,  3.97s/it]
 49%|████▊     | 109672/225000 [231:52:23<110:04:45,  3.44s/it]
 49%|████▊     | 109673/225000 [231:52:33<177:31:58,  5.54s/it]
 49%|████▊     | 109674/225000 [231:52:41<192:57:54,  6.02s/it]
 49%|████▊     | 109675/225000 [231:52:57<295:16:20,  9.22s/it]
 49%|████▊     | 109676/225000 [231:53:03<263:54:10,  8.24s/it]
 49%|████▊     | 109677/225000 [231:53:07<216:24:27,  6.76s/it]
 49%|████▊     | 109678/225000 [231:53:10<183:22:54,  5.72s/it]
 49%|████▊     | 109679/225000 [231:53:12<153:28:41,  4.79s/it]
 49%|████▊     | 109680/225000 [231:53:16<140:07:33,  4.37s/it]
 49%|████▊     | 109681/225000 [231:53:19<123:25:46,  3.85s/it]
 49%|████▊     | 109682/225000 [231:53:21<112:48:08,  3.52s/it]
 49%|████▊     | 109683/225000 [231:53:29<154:35:39,  4.83s/it]
 49%|████▊     | 109684/225000 [231:53:31<129:16:27,  4.04s/it]
 49%|████▊     | 109685/225000 [231:53:33<111:10:08,  3.47s/it]
 49%|████▊     | 109686/225000 [231:53:39<127:01:13,  3.97s/it]
 49%|████▊     | 109687/225000 [231:53:45<154:37:32,  4.83s/it]
 49%|████▉     | 109688/225000 [231:53:51<160:35:31,  5.01s/it]
 49%|████▉     | 109689/225000 [231:53:53<135:26:58,  4.23s/it]
 49%|████▉     | 109690/225000 [231:53:56<117:46:49,  3.68s/it]
 49%|████▉     | 109691/225000 [231:53:58<104:50:19,  3.27s/it]
 49%|████▉     | 109692/225000 [231:54:05<138:36:30,  4.33s/it]
 49%|████▉     | 109693/225000 [231:54:07<120:14:51,  3.75s/it]
 49%|████▉     | 109694/225000 [231:54:14<146:48:41,  4.58s/it]
 49%|████▉     | 109695/225000 [231:54:21<170:20:30,  5.32s/it]
 49%|████▉     | 109696/225000 [231:54:23<139:28:27,  4.35s/it]
 49%|████▉     | 109697/225000 [231:54:26<123:51:39,  3.87s/it]
 49%|████▉     | 109698/225000 [231:54:36<183:35:43,  5.73s/it]
 49%|████▉     | 109699/225000 [231:54:39<156:37:52,  4.89s/it]
 49%|████▉     | 109700/225000 [231:54:41<134:35:44,  4.20s/it]
                                                               
{'loss': '0.9055', 'grad_norm': '3.745', 'learning_rate': '2.786e-05', 'epoch': '0.4876', 'num_input_tokens_seen': 388398904, 'train_runtime': '8.349e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109700/225000 [231:54:41<134:35:44,  4.20s/it]
 49%|████▉     | 109701/225000 [231:54:51<192:26:45,  6.01s/it]
 49%|████▉     | 109702/225000 [231:54:54<155:16:49,  4.85s/it]
 49%|████▉     | 109703/225000 [231:54:57<137:04:27,  4.28s/it]
 49%|████▉     | 109704/225000 [231:55:00<125:03:16,  3.90s/it]
 49%|████▉     | 109705/225000 [231:55:08<171:51:45,  5.37s/it]
 49%|████▉     | 109706/225000 [231:55:11<146:23:18,  4.57s/it]
 49%|████▉     | 109707/225000 [231:55:21<195:42:20,  6.11s/it]
 49%|████▉     | 109708/225000 [231:55:24<172:53:23,  5.40s/it]
 49%|████▉     | 109709/225000 [231:55:27<149:28:36,  4.67s/it]
 49%|████▉     | 109710/225000 [231:55:30<128:21:12,  4.01s/it]
 49%|████▉     | 109711/225000 [231:55:33<121:49:16,  3.80s/it]
 49%|████▉     | 109712/225000 [231:55:45<199:02:57,  6.22s/it]
 49%|████▉     | 109713/225000 [231:55:50<182:12:02,  5.69s/it]
 49%|████▉     | 109714/225000 [231:55:53<163:14:33,  5.10s/it]
 49%|████▉     | 109715/225000 [231:55:57<151:11:05,  4.72s/it]
 49%|████▉     | 109716/225000 [231:56:06<194:12:17,  6.06s/it]
 49%|████▉     | 109717/225000 [231:56:11<179:27:40,  5.60s/it]
 49%|████▉     | 109718/225000 [231:56:16<176:15:40,  5.50s/it]
 49%|████▉     | 109719/225000 [231:56:19<146:50:50,  4.59s/it]
 49%|████▉     | 109720/225000 [231:56:21<125:50:49,  3.93s/it]
 49%|████▉     | 109721/225000 [231:56:26<139:46:14,  4.36s/it]
 49%|████▉     | 109722/225000 [231:56:29<121:11:38,  3.78s/it]
 49%|████▉     | 109723/225000 [231:56:31<110:24:19,  3.45s/it]
 49%|████▉     | 109724/225000 [231:56:35<115:00:36,  3.59s/it]
 49%|████▉     | 109725/225000 [231:56:46<178:20:23,  5.57s/it]
 49%|████▉     | 109726/225000 [231:56:48<149:52:10,  4.68s/it]
 49%|████▉     | 109727/225000 [231:56:51<129:37:46,  4.05s/it]
 49%|████▉     | 109728/225000 [231:56:53<116:16:51,  3.63s/it]
 49%|████▉     | 109729/225000 [231:56:56<106:51:38,  3.34s/it]
 49%|████▉     | 109730/225000 [231:56:59<100:44:30,  3.15s/it]
 49%|████▉     | 109731/225000 [231:57:01<95:58:27,  3.00s/it] 
 49%|████▉     | 109732/225000 [231:57:11<164:12:56,  5.13s/it]
 49%|████▉     | 109733/225000 [231:57:15<152:50:41,  4.77s/it]
 49%|████▉     | 109734/225000 [231:57:18<136:28:22,  4.26s/it]
 49%|████▉     | 109735/225000 [231:57:21<121:00:36,  3.78s/it]
 49%|████▉     | 109736/225000 [231:57:30<174:05:08,  5.44s/it]
 49%|████▉     | 109737/225000 [231:57:33<144:17:28,  4.51s/it]
 49%|████▉     | 109738/225000 [231:57:36<130:09:11,  4.07s/it]
 49%|████▉     | 109739/225000 [231:57:38<113:40:20,  3.55s/it]
 49%|████▉     | 109740/225000 [231:57:41<102:30:14,  3.20s/it]
 49%|████▉     | 109741/225000 [231:57:44<103:32:43,  3.23s/it]
 49%|████▉     | 109742/225000 [231:57:51<139:22:04,  4.35s/it]
 49%|████▉     | 109743/225000 [231:57:53<120:01:48,  3.75s/it]
 49%|████▉     | 109744/225000 [231:57:58<133:09:23,  4.16s/it]
 49%|████▉     | 109745/225000 [231:58:02<126:51:24,  3.96s/it]
 49%|████▉     | 109746/225000 [231:58:05<115:50:23,  3.62s/it]
 49%|████▉     | 109747/225000 [231:58:07<106:40:36,  3.33s/it]
 49%|████▉     | 109748/225000 [231:58:10<102:04:21,  3.19s/it]
 49%|████▉     | 109749/225000 [231:58:13<98:18:21,  3.07s/it] 
 49%|████▉     | 109750/225000 [231:58:16<98:40:13,  3.08s/it]
                                                              
{'loss': '0.8491', 'grad_norm': '5.019', 'learning_rate': '2.784e-05', 'epoch': '0.4878', 'num_input_tokens_seen': 388493800, 'train_runtime': '8.351e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109750/225000 [231:58:16<98:40:13,  3.08s/it]
 49%|████▉     | 109751/225000 [231:58:19<96:03:34,  3.00s/it]
 49%|████▉     | 109752/225000 [231:58:22<94:17:04,  2.95s/it]
 49%|████▉     | 109753/225000 [231:58:31<156:35:40,  4.89s/it]
 49%|████▉     | 109754/225000 [231:58:36<156:26:23,  4.89s/it]
 49%|████▉     | 109755/225000 [231:58:39<136:34:37,  4.27s/it]
 49%|████▉     | 109756/225000 [231:58:42<122:14:51,  3.82s/it]
 49%|████▉     | 109757/225000 [231:58:52<189:56:45,  5.93s/it]
 49%|████▉     | 109758/225000 [231:58:55<154:55:15,  4.84s/it]
 49%|████▉     | 109759/225000 [231:58:59<152:12:13,  4.75s/it]
 49%|████▉     | 109760/225000 [231:59:01<127:19:42,  3.98s/it]
 49%|████▉     | 109761/225000 [231:59:08<149:57:20,  4.68s/it]
 49%|████▉     | 109762/225000 [231:59:12<148:10:41,  4.63s/it]
 49%|████▉     | 109763/225000 [231:59:17<152:12:23,  4.75s/it]
 49%|████▉     | 109764/225000 [231:59:21<143:57:42,  4.50s/it]
 49%|████▉     | 109765/225000 [231:59:24<130:33:49,  4.08s/it]
 49%|████▉     | 109766/225000 [231:59:28<128:30:24,  4.01s/it]
 49%|████▉     | 109767/225000 [231:59:30<111:50:49,  3.49s/it]
 49%|████▉     | 109768/225000 [231:59:33<100:34:31,  3.14s/it]
 49%|████▉     | 109769/225000 [231:59:35<92:24:02,  2.89s/it] 
 49%|████▉     | 109770/225000 [231:59:50<204:44:36,  6.40s/it]
 49%|████▉     | 109771/225000 [231:59:56<209:03:44,  6.53s/it]
 49%|████▉     | 109772/225000 [232:00:02<198:47:06,  6.21s/it]
 49%|████▉     | 109773/225000 [232:00:04<162:46:41,  5.09s/it]
 49%|████▉     | 109774/225000 [232:00:10<167:32:48,  5.23s/it]
 49%|████▉     | 109775/225000 [232:00:12<141:01:19,  4.41s/it]
 49%|████▉     | 109776/225000 [232:00:16<129:34:37,  4.05s/it]
 49%|████▉     | 109777/225000 [232:00:19<121:26:31,  3.79s/it]
 49%|████▉     | 109778/225000 [232:00:30<188:29:20,  5.89s/it]
 49%|████▉     | 109779/225000 [232:00:33<166:40:57,  5.21s/it]
 49%|████▉     | 109780/225000 [232:00:36<145:20:15,  4.54s/it]
 49%|████▉     | 109781/225000 [232:00:40<138:05:51,  4.31s/it]
 49%|████▉     | 109782/225000 [232:00:45<144:15:47,  4.51s/it]
 49%|████▉     | 109783/225000 [232:00:48<131:42:10,  4.12s/it]
 49%|████▉     | 109784/225000 [232:00:51<119:42:55,  3.74s/it]
 49%|████▉     | 109785/225000 [232:00:55<119:13:45,  3.73s/it]
 49%|████▉     | 109786/225000 [232:00:58<118:29:52,  3.70s/it]
 49%|████▉     | 109787/225000 [232:01:02<117:41:43,  3.68s/it]
 49%|████▉     | 109788/225000 [232:01:13<183:46:12,  5.74s/it]
 49%|████▉     | 109789/225000 [232:01:15<149:59:10,  4.69s/it]
 49%|████▉     | 109790/225000 [232:01:21<160:01:53,  5.00s/it]
 49%|████▉     | 109791/225000 [232:01:23<134:53:45,  4.22s/it]
 49%|████▉     | 109792/225000 [232:01:33<187:22:24,  5.86s/it]
 49%|████▉     | 109793/225000 [232:01:36<164:39:03,  5.15s/it]
 49%|████▉     | 109794/225000 [232:01:39<145:09:47,  4.54s/it]
 49%|████▉     | 109795/225000 [232:01:52<221:48:52,  6.93s/it]
 49%|████▉     | 109796/225000 [232:01:54<178:50:44,  5.59s/it]
 49%|████▉     | 109797/225000 [232:01:57<148:42:27,  4.65s/it]
 49%|████▉     | 109798/225000 [232:01:59<127:49:34,  3.99s/it]
 49%|████▉     | 109799/225000 [232:02:03<123:35:31,  3.86s/it]
 49%|████▉     | 109800/225000 [232:02:05<109:56:55,  3.44s/it]
                                                               
{'loss': '0.9158', 'grad_norm': '5.752', 'learning_rate': '2.783e-05', 'epoch': '0.488', 'num_input_tokens_seen': 388595816, 'train_runtime': '8.353e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109800/225000 [232:02:05<109:56:55,  3.44s/it]
 49%|████▉     | 109801/225000 [232:02:11<130:18:11,  4.07s/it]
 49%|████▉     | 109802/225000 [232:02:18<160:06:05,  5.00s/it]
 49%|████▉     | 109803/225000 [232:02:29<216:37:29,  6.77s/it]
 49%|████▉     | 109804/225000 [232:02:34<201:21:54,  6.29s/it]
 49%|████▉     | 109805/225000 [232:02:43<230:06:56,  7.19s/it]
 49%|████▉     | 109806/225000 [232:02:45<182:01:21,  5.69s/it]
 49%|████▉     | 109807/225000 [232:02:48<148:14:16,  4.63s/it]
 49%|████▉     | 109808/225000 [232:02:57<197:41:28,  6.18s/it]
 49%|████▉     | 109809/225000 [232:03:00<167:45:25,  5.24s/it]
 49%|████▉     | 109810/225000 [232:03:03<139:30:59,  4.36s/it]
 49%|████▉     | 109811/225000 [232:03:06<127:22:17,  3.98s/it]
 49%|████▉     | 109812/225000 [232:03:09<118:38:19,  3.71s/it]
 49%|████▉     | 109813/225000 [232:03:14<133:07:41,  4.16s/it]
 49%|████▉     | 109814/225000 [232:03:16<115:07:18,  3.60s/it]
 49%|████▉     | 109815/225000 [232:03:24<155:41:39,  4.87s/it]
 49%|████▉     | 109816/225000 [232:03:29<157:52:14,  4.93s/it]
 49%|████▉     | 109817/225000 [232:03:36<171:37:58,  5.36s/it]
 49%|████▉     | 109818/225000 [232:03:40<165:45:52,  5.18s/it]
 49%|████▉     | 109819/225000 [232:03:43<136:51:40,  4.28s/it]
 49%|████▉     | 109820/225000 [232:03:47<140:39:52,  4.40s/it]
 49%|████▉     | 109821/225000 [232:03:54<162:08:03,  5.07s/it]
 49%|████▉     | 109822/225000 [232:04:02<186:34:55,  5.83s/it]
 49%|████▉     | 109823/225000 [232:04:08<190:12:28,  5.95s/it]
 49%|████▉     | 109824/225000 [232:04:10<156:19:30,  4.89s/it]
 49%|████▉     | 109825/225000 [232:04:13<140:21:29,  4.39s/it]
 49%|████▉     | 109826/225000 [232:04:29<244:20:48,  7.64s/it]
 49%|████▉     | 109827/225000 [232:04:31<192:35:43,  6.02s/it]
 49%|████▉     | 109828/225000 [232:04:37<195:23:24,  6.11s/it]
 49%|████▉     | 109829/225000 [232:04:49<251:11:48,  7.85s/it]
 49%|████▉     | 109830/225000 [232:04:51<195:52:53,  6.12s/it]
 49%|████▉     | 109831/225000 [232:04:53<157:26:40,  4.92s/it]
 49%|████▉     | 109832/225000 [232:04:56<132:28:40,  4.14s/it]
 49%|████▉     | 109833/225000 [232:05:00<139:42:14,  4.37s/it]
 49%|████▉     | 109834/225000 [232:05:14<226:29:42,  7.08s/it]
 49%|████▉     | 109835/225000 [232:05:17<186:41:06,  5.84s/it]
 49%|████▉     | 109836/225000 [232:05:19<151:38:38,  4.74s/it]
 49%|████▉     | 109837/225000 [232:05:27<178:46:22,  5.59s/it]
 49%|████▉     | 109838/225000 [232:05:33<184:54:20,  5.78s/it]
 49%|████▉     | 109839/225000 [232:05:41<210:21:13,  6.58s/it]
 49%|████▉     | 109840/225000 [232:05:44<171:13:28,  5.35s/it]
 49%|████▉     | 109841/225000 [232:05:50<179:24:00,  5.61s/it]
 49%|████▉     | 109842/225000 [232:05:55<176:40:43,  5.52s/it]
 49%|████▉     | 109843/225000 [232:05:58<147:28:50,  4.61s/it]
 49%|████▉     | 109844/225000 [232:06:05<172:13:50,  5.38s/it]
 49%|████▉     | 109845/225000 [232:06:08<148:38:38,  4.65s/it]
 49%|████▉     | 109846/225000 [232:06:11<132:55:57,  4.16s/it]
 49%|████▉     | 109847/225000 [232:06:19<171:36:23,  5.36s/it]
 49%|████▉     | 109848/225000 [232:06:21<141:39:24,  4.43s/it]
 49%|████▉     | 109849/225000 [232:06:25<136:57:09,  4.28s/it]
 49%|████▉     | 109850/225000 [232:06:31<147:01:29,  4.60s/it]
                                                               
{'loss': '0.8871', 'grad_norm': '4.082', 'learning_rate': '2.781e-05', 'epoch': '0.4882', 'num_input_tokens_seen': 388702624, 'train_runtime': '8.356e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109850/225000 [232:06:31<147:01:29,  4.60s/it]
 49%|████▉     | 109851/225000 [232:06:40<190:34:10,  5.96s/it]
 49%|████▉     | 109852/225000 [232:06:48<212:51:41,  6.65s/it]
 49%|████▉     | 109853/225000 [232:06:51<174:40:07,  5.46s/it]
 49%|████▉     | 109854/225000 [232:06:55<161:21:52,  5.05s/it]
 49%|████▉     | 109855/225000 [232:07:10<257:05:46,  8.04s/it]
 49%|████▉     | 109856/225000 [232:07:20<277:03:22,  8.66s/it]
 49%|████▉     | 109857/225000 [232:07:28<275:32:09,  8.61s/it]
 49%|████▉     | 109858/225000 [232:07:31<214:31:41,  6.71s/it]
 49%|████▉     | 109859/225000 [232:07:33<171:09:37,  5.35s/it]
 49%|████▉     | 109860/225000 [232:07:35<143:27:25,  4.49s/it]
 49%|████▉     | 109861/225000 [232:07:38<122:38:47,  3.83s/it]
 49%|████▉     | 109862/225000 [232:07:46<163:24:30,  5.11s/it]
 49%|████▉     | 109863/225000 [232:07:48<136:50:04,  4.28s/it]
 49%|████▉     | 109864/225000 [232:07:58<190:41:00,  5.96s/it]
 49%|████▉     | 109865/225000 [232:08:05<200:07:18,  6.26s/it]
 49%|████▉     | 109866/225000 [232:08:08<167:27:44,  5.24s/it]
 49%|████▉     | 109867/225000 [232:08:14<182:12:53,  5.70s/it]
 49%|████▉     | 109868/225000 [232:08:22<198:15:59,  6.20s/it]
 49%|████▉     | 109869/225000 [232:08:24<160:19:00,  5.01s/it]
 49%|████▉     | 109870/225000 [232:08:27<142:22:46,  4.45s/it]
 49%|████▉     | 109871/225000 [232:08:30<130:29:11,  4.08s/it]
 49%|████▉     | 109872/225000 [232:08:34<120:51:05,  3.78s/it]
 49%|████▉     | 109873/225000 [232:08:42<167:17:39,  5.23s/it]
 49%|████▉     | 109874/225000 [232:08:44<138:10:03,  4.32s/it]
 49%|████▉     | 109875/225000 [232:08:47<123:19:33,  3.86s/it]
 49%|████▉     | 109876/225000 [232:08:55<162:44:10,  5.09s/it]
 49%|████▉     | 109877/225000 [232:09:06<219:58:27,  6.88s/it]
 49%|████▉     | 109878/225000 [232:09:13<220:43:07,  6.90s/it]
 49%|████▉     | 109879/225000 [232:09:16<177:46:41,  5.56s/it]
 49%|████▉     | 109880/225000 [232:09:24<203:15:01,  6.36s/it]
 49%|████▉     | 109881/225000 [232:09:26<163:29:13,  5.11s/it]
 49%|████▉     | 109882/225000 [232:09:30<156:32:16,  4.90s/it]
 49%|████▉     | 109883/225000 [232:09:33<132:40:06,  4.15s/it]
 49%|████▉     | 109884/225000 [232:09:38<142:32:11,  4.46s/it]
 49%|████▉     | 109885/225000 [232:09:41<129:23:59,  4.05s/it]
 49%|████▉     | 109886/225000 [232:09:43<113:02:40,  3.54s/it]
 49%|████▉     | 109887/225000 [232:09:46<101:47:44,  3.18s/it]
 49%|████▉     | 109888/225000 [232:09:49<101:11:50,  3.16s/it]
 49%|████▉     | 109889/225000 [232:09:53<106:36:20,  3.33s/it]
 49%|████▉     | 109890/225000 [232:09:55<99:44:00,  3.12s/it] 
 49%|████▉     | 109891/225000 [232:09:58<95:14:57,  2.98s/it]
 49%|████▉     | 109892/225000 [232:10:01<92:44:03,  2.90s/it]
 49%|████▉     | 109893/225000 [232:10:04<97:24:35,  3.05s/it]
 49%|████▉     | 109894/225000 [232:10:07<99:30:31,  3.11s/it]
 49%|████▉     | 109895/225000 [232:10:10<95:44:53,  2.99s/it]
 49%|████▉     | 109896/225000 [232:10:13<93:01:04,  2.91s/it]
 49%|████▉     | 109897/225000 [232:10:15<91:27:25,  2.86s/it]
 49%|████▉     | 109898/225000 [232:10:23<136:37:20,  4.27s/it]
 49%|████▉     | 109899/225000 [232:10:35<208:27:06,  6.52s/it]
 49%|████▉     | 109900/225000 [232:10:37<169:57:33,  5.32s/it]
                                                               
{'loss': '0.9134', 'grad_norm': '4.349', 'learning_rate': '2.779e-05', 'epoch': '0.4884', 'num_input_tokens_seen': 388801232, 'train_runtime': '8.358e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109900/225000 [232:10:37<169:57:33,  5.32s/it]
 49%|████▉     | 109901/225000 [232:10:40<142:39:49,  4.46s/it]
 49%|████▉     | 109902/225000 [232:10:43<130:48:41,  4.09s/it]
 49%|████▉     | 109903/225000 [232:10:47<129:49:03,  4.06s/it]
 49%|████▉     | 109904/225000 [232:10:50<121:32:17,  3.80s/it]
 49%|████▉     | 109905/225000 [232:10:52<107:48:37,  3.37s/it]
 49%|████▉     | 109906/225000 [232:10:56<105:11:11,  3.29s/it]
 49%|████▉     | 109907/225000 [232:11:01<129:09:11,  4.04s/it]
 49%|████▉     | 109908/225000 [232:11:05<124:10:41,  3.88s/it]
 49%|████▉     | 109909/225000 [232:11:07<111:50:47,  3.50s/it]
 49%|████▉     | 109910/225000 [232:11:10<104:04:24,  3.26s/it]
 49%|████▉     | 109911/225000 [232:11:21<172:47:26,  5.40s/it]
 49%|████▉     | 109912/225000 [232:11:26<175:27:52,  5.49s/it]
 49%|████▉     | 109913/225000 [232:11:29<144:42:47,  4.53s/it]
 49%|████▉     | 109914/225000 [232:11:31<123:46:26,  3.87s/it]
 49%|████▉     | 109915/225000 [232:11:42<189:10:27,  5.92s/it]
 49%|████▉     | 109916/225000 [232:11:45<164:39:03,  5.15s/it]
 49%|████▉     | 109917/225000 [232:11:48<145:14:35,  4.54s/it]
 49%|████▉     | 109918/225000 [232:11:50<123:59:22,  3.88s/it]
 49%|████▉     | 109919/225000 [232:11:56<136:23:56,  4.27s/it]
 49%|████▉     | 109920/225000 [232:12:00<139:36:45,  4.37s/it]
 49%|████▉     | 109921/225000 [232:12:07<164:03:52,  5.13s/it]
 49%|████▉     | 109922/225000 [232:12:15<193:42:01,  6.06s/it]
 49%|████▉     | 109923/225000 [232:12:18<159:17:19,  4.98s/it]
 49%|████▉     | 109924/225000 [232:12:21<138:58:19,  4.35s/it]
 49%|████▉     | 109925/225000 [232:12:30<185:10:50,  5.79s/it]
 49%|████▉     | 109926/225000 [232:12:32<154:16:22,  4.83s/it]
 49%|████▉     | 109927/225000 [232:12:35<131:53:25,  4.13s/it]
 49%|████▉     | 109928/225000 [232:12:43<170:43:47,  5.34s/it]
 49%|████▉     | 109929/225000 [232:12:45<142:11:02,  4.45s/it]
 49%|████▉     | 109930/225000 [232:12:49<129:43:01,  4.06s/it]
 49%|████▉     | 109931/225000 [232:12:55<149:01:27,  4.66s/it]
 49%|████▉     | 109932/225000 [232:12:58<139:09:44,  4.35s/it]
 49%|████▉     | 109933/225000 [232:13:03<142:43:20,  4.47s/it]
 49%|████▉     | 109934/225000 [232:13:09<153:27:52,  4.80s/it]
 49%|████▉     | 109935/225000 [232:13:15<167:51:56,  5.25s/it]
 49%|████▉     | 109936/225000 [232:13:17<140:42:17,  4.40s/it]
 49%|████▉     | 109937/225000 [232:13:21<130:36:02,  4.09s/it]
 49%|████▉     | 109938/225000 [232:13:24<124:22:38,  3.89s/it]
 49%|████▉     | 109939/225000 [232:13:34<185:45:09,  5.81s/it]
 49%|████▉     | 109940/225000 [232:13:37<154:44:14,  4.84s/it]
 49%|████▉     | 109941/225000 [232:13:40<132:41:11,  4.15s/it]
 49%|████▉     | 109942/225000 [232:13:49<185:18:03,  5.80s/it]
 49%|████▉     | 109943/225000 [232:13:51<151:01:00,  4.73s/it]
 49%|████▉     | 109944/225000 [232:13:54<127:59:15,  4.00s/it]
 49%|████▉     | 109945/225000 [232:13:57<119:10:21,  3.73s/it]
 49%|████▉     | 109946/225000 [232:14:00<113:40:20,  3.56s/it]
 49%|████▉     | 109947/225000 [232:14:04<117:38:38,  3.68s/it]
 49%|████▉     | 109948/225000 [232:14:06<105:52:25,  3.31s/it]
 49%|████▉     | 109949/225000 [232:14:10<105:08:28,  3.29s/it]
 49%|████▉     | 109950/225000 [232:14:12<99:04:07,  3.10s/it] 
                                                              
{'loss': '0.9558', 'grad_norm': '4.112', 'learning_rate': '2.777e-05', 'epoch': '0.4887', 'num_input_tokens_seen': 388898736, 'train_runtime': '8.361e+05', 'train_tokens_per_second': '465.2'}

 49%|████▉     | 109950/225000 [232:14:12<99:04:07,  3.10s/it]
 49%|████▉     | 109951/225000 [232:14:15<95:04:39,  2.98s/it]
 49%|████▉     | 109952/225000 [232:14:18<92:00:43,  2.88s/it]
 49%|████▉     | 109953/225000 [232:14:20<90:23:41,  2.83s/it]
 49%|████▉     | 109954/225000 [232:14:28<136:12:48,  4.26s/it]
 49%|████▉     | 109955/225000 [232:14:35<159:14:36,  4.98s/it]
 49%|████▉     | 109956/225000 [232:14:39<149:06:25,  4.67s/it]
 49%|████▉     | 109957/225000 [232:14:50<216:07:24,  6.76s/it]
 49%|████▉     | 109958/225000 [232:14:52<172:04:08,  5.38s/it]
 49%|████▉     | 109959/225000 [232:14:55<141:50:31,  4.44s/it]
 49%|████▉     | 109960/225000 [232:15:04<193:04:48,  6.04s/it]
 49%|████▉     | 109961/225000 [232:15:08<167:10:37,  5.23s/it]
 49%|████▉     | 109962/225000 [232:15:10<141:57:27,  4.44s/it]
 49%|████▉     | 109963/225000 [232:15:19<179:09:12,  5.61s/it]
 49%|████▉     | 109964/225000 [232:15:26<196:22:57,  6.15s/it]
 49%|████▉     | 109965/225000 [232:15:34<214:16:55,  6.71s/it]
 49%|████▉     | 109966/225000 [232:15:39<200:24:14,  6.27s/it]
 49%|████▉     | 109967/225000 [232:15:46<208:34:52,  6.53s/it]
 49%|████▉     | 109968/225000 [232:15:53<208:10:09,  6.51s/it]
 49%|████▉     | 109969/225000 [232:15:56<171:32:43,  5.37s/it]
 49%|████▉     | 109970/225000 [232:15:58<146:13:57,  4.58s/it]
 49%|████▉     | 109971/225000 [232:16:02<134:58:08,  4.22s/it]
 49%|████▉     | 109972/225000 [232:16:10<169:16:51,  5.30s/it]
 49%|████▉     | 109973/225000 [232:16:12<139:30:03,  4.37s/it]
 49%|████▉     | 109974/225000 [232:16:18<154:06:16,  4.82s/it]
 49%|████▉     | 109975/225000 [232:16:26<185:29:22,  5.81s/it]
 49%|████▉     | 109976/225000 [232:16:28<152:08:57,  4.76s/it]
 49%|████▉     | 109977/225000 [232:16:33<155:21:15,  4.86s/it]
 49%|████▉     | 109978/225000 [232:16:45<223:34:34,  7.00s/it]
 49%|████▉     | 109979/225000 [232:16:49<191:09:04,  5.98s/it]
 49%|████▉     | 109980/225000 [232:16:52<166:04:11,  5.20s/it]
 49%|████▉     | 109981/225000 [232:16:55<141:01:19,  4.41s/it]
 49%|████▉     | 109982/225000 [232:16:57<123:56:32,  3.88s/it]
 49%|████▉     | 109983/225000 [232:17:05<160:26:09,  5.02s/it]
 49%|████▉     | 109984/225000 [232:17:07<136:23:36,  4.27s/it]
 49%|████▉     | 109985/225000 [232:17:12<135:13:40,  4.23s/it]
 49%|████▉     | 109986/225000 [232:17:14<121:40:52,  3.81s/it]
 49%|████▉     | 109987/225000 [232:17:23<166:08:40,  5.20s/it]
 49%|████▉     | 109988/225000 [232:17:25<140:32:31,  4.40s/it]
 49%|████▉     | 109989/225000 [232:17:28<123:41:30,  3.87s/it]
 49%|████▉     | 109990/225000 [232:17:31<118:58:06,  3.72s/it]
 49%|████▉     | 109991/225000 [232:17:35<115:15:56,  3.61s/it]
 49%|████▉     | 109992/225000 [232:17:37<106:02:18,  3.32s/it]
 49%|████▉     | 109993/225000 [232:17:41<106:29:58,  3.33s/it]
 49%|████▉     | 109994/225000 [232:17:51<175:45:53,  5.50s/it]
 49%|████▉     | 109995/225000 [232:17:54<143:53:51,  4.50s/it]
 49%|████▉     | 109996/225000 [232:18:04<196:26:56,  6.15s/it]
 49%|████▉     | 109997/225000 [232:18:06<159:13:43,  4.98s/it]
 49%|████▉     | 109998/225000 [232:18:09<140:32:12,  4.40s/it]
 49%|████▉     | 109999/225000 [232:18:15<154:52:50,  4.85s/it]
 49%|████▉     | 110000/225000 [232:18:22<173:50:00,  5.44s/it]
                                                               
{'loss': '0.8941', 'grad_norm': '4.52', 'learning_rate': '2.775e-05', 'epoch': '0.4889', 'num_input_tokens_seen': 388998968, 'train_runtime': '8.363e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110000/225000 [232:18:22<173:50:00,  5.44s/it]
 49%|████▉     | 110001/225000 [232:18:25<149:50:43,  4.69s/it]
 49%|████▉     | 110002/225000 [232:18:27<125:37:27,  3.93s/it]
 49%|████▉     | 110003/225000 [232:18:30<115:18:38,  3.61s/it]
 49%|████▉     | 110004/225000 [232:18:33<115:13:16,  3.61s/it]
 49%|████▉     | 110005/225000 [232:18:36<107:39:12,  3.37s/it]
 49%|████▉     | 110006/225000 [232:18:40<115:43:29,  3.62s/it]
 49%|████▉     | 110007/225000 [232:18:43<108:21:58,  3.39s/it]
 49%|████▉     | 110008/225000 [232:18:46<109:15:22,  3.42s/it]
 49%|████▉     | 110009/225000 [232:18:55<155:24:27,  4.87s/it]
 49%|████▉     | 110010/225000 [232:18:58<135:55:08,  4.26s/it]
 49%|████▉     | 110011/225000 [232:19:00<122:30:51,  3.84s/it]
 49%|████▉     | 110012/225000 [232:19:04<115:44:30,  3.62s/it]
 49%|████▉     | 110013/225000 [232:19:12<166:44:02,  5.22s/it]
 49%|████▉     | 110014/225000 [232:19:21<197:20:23,  6.18s/it]
 49%|████▉     | 110015/225000 [232:19:23<161:30:10,  5.06s/it]
 49%|████▉     | 110016/225000 [232:19:30<175:26:04,  5.49s/it]
 49%|████▉     | 110017/225000 [232:19:38<202:35:21,  6.34s/it]
 49%|████▉     | 110018/225000 [232:19:42<177:39:06,  5.56s/it]
 49%|████▉     | 110019/225000 [232:19:44<145:24:53,  4.55s/it]
 49%|████▉     | 110020/225000 [232:19:47<129:23:42,  4.05s/it]
 49%|████▉     | 110021/225000 [232:19:57<188:11:17,  5.89s/it]
 49%|████▉     | 110022/225000 [232:20:00<155:35:35,  4.87s/it]
 49%|████▉     | 110023/225000 [232:20:05<163:18:58,  5.11s/it]
 49%|████▉     | 110024/225000 [232:20:08<135:34:29,  4.24s/it]
 49%|████▉     | 110025/225000 [232:20:13<142:20:59,  4.46s/it]
 49%|████▉     | 110026/225000 [232:20:23<202:34:37,  6.34s/it]
 49%|████▉     | 110027/225000 [232:20:26<172:09:09,  5.39s/it]
 49%|████▉     | 110028/225000 [232:20:29<147:07:31,  4.61s/it]
 49%|████▉     | 110029/225000 [232:20:37<177:56:08,  5.57s/it]
 49%|████▉     | 110030/225000 [232:20:40<155:06:06,  4.86s/it]
 49%|████▉     | 110031/225000 [232:20:51<212:17:18,  6.65s/it]
 49%|████▉     | 110032/225000 [232:20:53<170:24:08,  5.34s/it]
 49%|████▉     | 110033/225000 [232:20:56<149:04:56,  4.67s/it]
 49%|████▉     | 110034/225000 [232:21:02<158:00:29,  4.95s/it]
 49%|████▉     | 110035/225000 [232:21:05<140:01:53,  4.38s/it]
 49%|████▉     | 110036/225000 [232:21:10<148:03:28,  4.64s/it]
 49%|████▉     | 110037/225000 [232:21:13<133:22:13,  4.18s/it]
 49%|████▉     | 110038/225000 [232:21:23<182:34:25,  5.72s/it]
 49%|████▉     | 110039/225000 [232:21:26<158:55:25,  4.98s/it]
 49%|████▉     | 110040/225000 [232:21:29<135:24:12,  4.24s/it]
 49%|████▉     | 110041/225000 [232:21:31<119:41:30,  3.75s/it]
 49%|████▉     | 110042/225000 [232:21:34<109:44:53,  3.44s/it]
 49%|████▉     | 110043/225000 [232:21:43<168:20:05,  5.27s/it]
 49%|████▉     | 110044/225000 [232:21:46<139:07:35,  4.36s/it]
 49%|████▉     | 110045/225000 [232:21:49<125:10:18,  3.92s/it]
 49%|████▉     | 110046/225000 [232:21:52<116:17:34,  3.64s/it]
 49%|████▉     | 110047/225000 [232:21:55<115:11:40,  3.61s/it]
 49%|████▉     | 110048/225000 [232:21:59<122:39:10,  3.84s/it]
 49%|████▉     | 110049/225000 [232:22:02<106:57:50,  3.35s/it]
 49%|████▉     | 110050/225000 [232:22:05<103:27:44,  3.24s/it]
                                                               
{'loss': '0.8289', 'grad_norm': '4.228', 'learning_rate': '2.774e-05', 'epoch': '0.4891', 'num_input_tokens_seen': 389093552, 'train_runtime': '8.365e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110050/225000 [232:22:05<103:27:44,  3.24s/it]
 49%|████▉     | 110051/225000 [232:22:13<154:05:35,  4.83s/it]
 49%|████▉     | 110052/225000 [232:22:22<196:46:29,  6.16s/it]
 49%|████▉     | 110053/225000 [232:22:25<159:14:27,  4.99s/it]
 49%|████▉     | 110054/225000 [232:22:28<139:56:31,  4.38s/it]
 49%|████▉     | 110055/225000 [232:22:31<126:15:05,  3.95s/it]
 49%|████▉     | 110056/225000 [232:22:33<109:24:44,  3.43s/it]
 49%|████▉     | 110057/225000 [232:22:42<161:33:15,  5.06s/it]
 49%|████▉     | 110058/225000 [232:22:45<141:38:04,  4.44s/it]
 49%|████▉     | 110059/225000 [232:22:47<120:45:09,  3.78s/it]
 49%|████▉     | 110060/225000 [232:22:50<113:48:19,  3.56s/it]
 49%|████▉     | 110061/225000 [232:22:56<135:32:12,  4.25s/it]
 49%|████▉     | 110062/225000 [232:22:58<116:53:58,  3.66s/it]
 49%|████▉     | 110063/225000 [232:23:01<114:13:47,  3.58s/it]
 49%|████▉     | 110064/225000 [232:23:07<136:25:44,  4.27s/it]
 49%|████▉     | 110065/225000 [232:23:13<151:18:06,  4.74s/it]
 49%|████▉     | 110066/225000 [232:23:16<134:46:18,  4.22s/it]
 49%|████▉     | 110067/225000 [232:23:22<149:24:05,  4.68s/it]
 49%|████▉     | 110068/225000 [232:23:37<244:16:15,  7.65s/it]
 49%|████▉     | 110069/225000 [232:23:39<199:09:00,  6.24s/it]
 49%|████▉     | 110070/225000 [232:23:42<160:17:34,  5.02s/it]
 49%|████▉     | 110071/225000 [232:23:50<194:40:47,  6.10s/it]
 49%|████▉     | 110072/225000 [232:23:56<196:02:26,  6.14s/it]
 49%|████▉     | 110073/225000 [232:24:02<190:49:55,  5.98s/it]
 49%|████▉     | 110074/225000 [232:24:07<183:49:10,  5.76s/it]
 49%|████▉     | 110075/225000 [232:24:11<162:48:08,  5.10s/it]
 49%|████▉     | 110076/225000 [232:24:17<173:14:39,  5.43s/it]
 49%|████▉     | 110077/225000 [232:24:24<187:58:07,  5.89s/it]
 49%|████▉     | 110078/225000 [232:24:32<204:16:47,  6.40s/it]
 49%|████▉     | 110079/225000 [232:24:34<170:12:45,  5.33s/it]
 49%|████▉     | 110080/225000 [232:24:37<147:10:58,  4.61s/it]
 49%|████▉     | 110081/225000 [232:24:41<132:35:31,  4.15s/it]
 49%|████▉     | 110082/225000 [232:24:43<114:27:18,  3.59s/it]
 49%|████▉     | 110083/225000 [232:24:47<119:40:05,  3.75s/it]
 49%|████▉     | 110084/225000 [232:24:49<105:44:05,  3.31s/it]
 49%|████▉     | 110085/225000 [232:24:51<95:38:53,  3.00s/it] 
 49%|████▉     | 110086/225000 [232:24:54<89:46:17,  2.81s/it]
 49%|████▉     | 110087/225000 [232:24:56<86:04:50,  2.70s/it]
 49%|████▉     | 110088/225000 [232:25:04<132:30:58,  4.15s/it]
 49%|████▉     | 110089/225000 [232:25:07<120:28:37,  3.77s/it]
 49%|████▉     | 110090/225000 [232:25:09<105:13:30,  3.30s/it]
 49%|████▉     | 110091/225000 [232:25:17<148:09:21,  4.64s/it]
 49%|████▉     | 110092/225000 [232:25:26<191:32:26,  6.00s/it]
 49%|████▉     | 110093/225000 [232:25:29<162:28:24,  5.09s/it]
 49%|████▉     | 110094/225000 [232:25:32<141:13:13,  4.42s/it]
 49%|████▉     | 110095/225000 [232:25:35<126:11:43,  3.95s/it]
 49%|████▉     | 110096/225000 [232:25:43<171:00:27,  5.36s/it]
 49%|████▉     | 110097/225000 [232:25:52<208:47:45,  6.54s/it]
 49%|████▉     | 110098/225000 [232:25:55<168:02:03,  5.26s/it]
 49%|████▉     | 110099/225000 [232:25:58<150:11:01,  4.71s/it]
 49%|████▉     | 110100/225000 [232:26:02<141:54:56,  4.45s/it]
                                                               
{'loss': '0.8985', 'grad_norm': '4.705', 'learning_rate': '2.772e-05', 'epoch': '0.4893', 'num_input_tokens_seen': 389189824, 'train_runtime': '8.368e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110100/225000 [232:26:02<141:54:56,  4.45s/it]
 49%|████▉     | 110101/225000 [232:26:07<150:14:43,  4.71s/it]
 49%|████▉     | 110102/225000 [232:26:15<180:56:14,  5.67s/it]
 49%|████▉     | 110103/225000 [232:26:19<163:31:06,  5.12s/it]
 49%|████▉     | 110104/225000 [232:26:22<145:45:52,  4.57s/it]
 49%|████▉     | 110105/225000 [232:26:29<168:40:14,  5.28s/it]
 49%|████▉     | 110106/225000 [232:26:32<140:21:05,  4.40s/it]
 49%|████▉     | 110107/225000 [232:26:35<127:21:20,  3.99s/it]
 49%|████▉     | 110108/225000 [232:26:37<111:35:25,  3.50s/it]
 49%|████▉     | 110109/225000 [232:26:39<99:40:41,  3.12s/it] 
 49%|████▉     | 110110/225000 [232:26:44<118:52:26,  3.72s/it]
 49%|████▉     | 110111/225000 [232:26:50<141:10:02,  4.42s/it]
 49%|████▉     | 110112/225000 [232:27:00<193:42:04,  6.07s/it]
 49%|████▉     | 110113/225000 [232:27:03<165:03:16,  5.17s/it]
 49%|████▉     | 110114/225000 [232:27:07<146:39:14,  4.60s/it]
 49%|████▉     | 110115/225000 [232:27:09<126:54:54,  3.98s/it]
 49%|████▉     | 110116/225000 [232:27:18<168:21:16,  5.28s/it]
 49%|████▉     | 110117/225000 [232:27:20<141:29:14,  4.43s/it]
 49%|████▉     | 110118/225000 [232:27:22<121:04:15,  3.79s/it]
 49%|████▉     | 110119/225000 [232:27:25<113:39:28,  3.56s/it]
 49%|████▉     | 110120/225000 [232:27:29<117:06:39,  3.67s/it]
 49%|████▉     | 110121/225000 [232:27:34<130:53:18,  4.10s/it]
 49%|████▉     | 110122/225000 [232:27:47<214:35:19,  6.72s/it]
 49%|████▉     | 110123/225000 [232:27:55<222:29:10,  6.97s/it]
 49%|████▉     | 110124/225000 [232:28:02<223:08:19,  6.99s/it]
 49%|████▉     | 110125/225000 [232:28:08<215:00:43,  6.74s/it]
 49%|████▉     | 110126/225000 [232:28:13<202:08:16,  6.33s/it]
 49%|████▉     | 110127/225000 [232:28:18<190:43:29,  5.98s/it]
 49%|████▉     | 110128/225000 [232:28:21<155:27:25,  4.87s/it]
 49%|████▉     | 110129/225000 [232:28:25<148:47:25,  4.66s/it]
 49%|████▉     | 110130/225000 [232:28:32<168:53:09,  5.29s/it]
 49%|████▉     | 110131/225000 [232:28:34<139:18:50,  4.37s/it]
 49%|████▉     | 110132/225000 [232:28:36<118:42:08,  3.72s/it]
 49%|████▉     | 110133/225000 [232:28:38<104:27:45,  3.27s/it]
 49%|████▉     | 110134/225000 [232:28:41<94:18:07,  2.96s/it] 
 49%|████▉     | 110135/225000 [232:28:52<172:17:53,  5.40s/it]
 49%|████▉     | 110136/225000 [232:28:55<149:57:03,  4.70s/it]
 49%|████▉     | 110137/225000 [232:29:04<198:15:16,  6.21s/it]
 49%|████▉     | 110138/225000 [232:29:15<240:49:43,  7.55s/it]
 49%|████▉     | 110139/225000 [232:29:20<215:19:26,  6.75s/it]
 49%|████▉     | 110140/225000 [232:29:25<197:19:57,  6.18s/it]
 49%|████▉     | 110141/225000 [232:29:28<163:19:46,  5.12s/it]
 49%|████▉     | 110142/225000 [232:29:30<140:07:52,  4.39s/it]
 49%|████▉     | 110143/225000 [232:29:33<123:48:01,  3.88s/it]
 49%|████▉     | 110144/225000 [232:29:36<112:05:55,  3.51s/it]
 49%|████▉     | 110145/225000 [232:29:45<167:57:21,  5.26s/it]
 49%|████▉     | 110146/225000 [232:29:49<152:31:02,  4.78s/it]
 49%|████▉     | 110147/225000 [232:29:51<127:16:55,  3.99s/it]
 49%|████▉     | 110148/225000 [232:30:01<184:46:36,  5.79s/it]
 49%|████▉     | 110149/225000 [232:30:03<153:54:51,  4.82s/it]
 49%|████▉     | 110150/225000 [232:30:07<139:20:41,  4.37s/it]
                                                               
{'loss': '0.8941', 'grad_norm': '2.399', 'learning_rate': '2.77e-05', 'epoch': '0.4896', 'num_input_tokens_seen': 389287400, 'train_runtime': '8.37e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110150/225000 [232:30:07<139:20:41,  4.37s/it]
 49%|████▉     | 110151/225000 [232:30:09<122:31:05,  3.84s/it]
 49%|████▉     | 110152/225000 [232:30:12<110:41:15,  3.47s/it]
 49%|████▉     | 110153/225000 [232:30:14<102:33:15,  3.21s/it]
 49%|████▉     | 110154/225000 [232:30:18<103:39:06,  3.25s/it]
 49%|████▉     | 110155/225000 [232:30:25<145:40:54,  4.57s/it]
 49%|████▉     | 110156/225000 [232:30:28<122:32:35,  3.84s/it]
 49%|████▉     | 110157/225000 [232:30:32<128:15:26,  4.02s/it]
 49%|████▉     | 110158/225000 [232:30:34<111:03:30,  3.48s/it]
 49%|████▉     | 110159/225000 [232:30:44<174:27:00,  5.47s/it]
 49%|████▉     | 110160/225000 [232:30:47<152:28:14,  4.78s/it]
 49%|████▉     | 110161/225000 [232:30:51<137:00:33,  4.29s/it]
 49%|████▉     | 110162/225000 [232:30:54<126:02:13,  3.95s/it]
 49%|████▉     | 110163/225000 [232:30:57<117:43:02,  3.69s/it]
 49%|████▉     | 110164/225000 [232:31:04<147:57:34,  4.64s/it]
 49%|████▉     | 110165/225000 [232:31:14<199:13:21,  6.25s/it]
 49%|████▉     | 110166/225000 [232:31:17<167:56:26,  5.26s/it]
 49%|████▉     | 110167/225000 [232:31:19<137:52:17,  4.32s/it]
 49%|████▉     | 110168/225000 [232:31:21<120:23:36,  3.77s/it]
 49%|████▉     | 110169/225000 [232:31:25<123:33:04,  3.87s/it]
 49%|████▉     | 110170/225000 [232:31:35<173:38:01,  5.44s/it]
 49%|████▉     | 110171/225000 [232:31:38<157:53:45,  4.95s/it]
 49%|████▉     | 110172/225000 [232:31:41<139:38:12,  4.38s/it]
 49%|████▉     | 110173/225000 [232:31:44<126:01:52,  3.95s/it]
 49%|████▉     | 110174/225000 [232:31:53<173:53:10,  5.45s/it]
 49%|████▉     | 110175/225000 [232:32:00<182:55:31,  5.74s/it]
 49%|████▉     | 110176/225000 [232:32:05<179:39:53,  5.63s/it]
 49%|████▉     | 110177/225000 [232:32:08<149:31:43,  4.69s/it]
 49%|████▉     | 110178/225000 [232:32:14<164:21:51,  5.15s/it]
 49%|████▉     | 110179/225000 [232:32:16<138:13:57,  4.33s/it]
 49%|████▉     | 110180/225000 [232:32:25<182:44:50,  5.73s/it]
 49%|████▉     | 110181/225000 [232:32:28<158:57:31,  4.98s/it]
 49%|████▉     | 110182/225000 [232:32:31<135:06:28,  4.24s/it]
 49%|████▉     | 110183/225000 [232:32:33<118:31:19,  3.72s/it]
 49%|████▉     | 110184/225000 [232:32:36<106:33:45,  3.34s/it]
 49%|████▉     | 110185/225000 [232:32:40<109:21:07,  3.43s/it]
 49%|████▉     | 110186/225000 [232:32:43<106:59:37,  3.35s/it]
 49%|████▉     | 110187/225000 [232:32:45<98:47:46,  3.10s/it] 
 49%|████▉     | 110188/225000 [232:32:48<95:09:36,  2.98s/it]
 49%|████▉     | 110189/225000 [232:32:50<90:40:45,  2.84s/it]
 49%|████▉     | 110190/225000 [232:32:54<101:40:41,  3.19s/it]
 49%|████▉     | 110191/225000 [232:33:02<144:26:44,  4.53s/it]
 49%|████▉     | 110192/225000 [232:33:12<198:12:50,  6.22s/it]
 49%|████▉     | 110193/225000 [232:33:18<192:41:50,  6.04s/it]
 49%|████▉     | 110194/225000 [232:33:21<163:29:48,  5.13s/it]
 49%|████▉     | 110195/225000 [232:33:24<141:46:40,  4.45s/it]
 49%|████▉     | 110196/225000 [232:33:27<127:10:27,  3.99s/it]
 49%|████▉     | 110197/225000 [232:33:30<116:49:52,  3.66s/it]
 49%|████▉     | 110198/225000 [232:33:33<116:19:18,  3.65s/it]
 49%|████▉     | 110199/225000 [232:33:36<108:47:30,  3.41s/it]
 49%|████▉     | 110200/225000 [232:33:39<107:36:23,  3.37s/it]
                                                               
{'loss': '0.9232', 'grad_norm': '6.125', 'learning_rate': '2.768e-05', 'epoch': '0.4898', 'num_input_tokens_seen': 389384400, 'train_runtime': '8.372e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110200/225000 [232:33:39<107:36:23,  3.37s/it]
 49%|████▉     | 110201/225000 [232:33:42<103:26:36,  3.24s/it]
 49%|████▉     | 110202/225000 [232:33:45<100:26:11,  3.15s/it]
 49%|████▉     | 110203/225000 [232:33:48<98:21:11,  3.08s/it] 
 49%|████▉     | 110204/225000 [232:33:51<96:42:25,  3.03s/it]
 49%|████▉     | 110205/225000 [232:33:54<96:13:34,  3.02s/it]
 49%|████▉     | 110206/225000 [232:33:57<95:36:49,  3.00s/it]
 49%|████▉     | 110207/225000 [232:34:13<217:57:49,  6.84s/it]
 49%|████▉     | 110208/225000 [232:34:18<205:00:21,  6.43s/it]
 49%|████▉     | 110209/225000 [232:34:21<166:47:07,  5.23s/it]
 49%|████▉     | 110210/225000 [232:34:26<171:16:45,  5.37s/it]
 49%|████▉     | 110211/225000 [232:34:30<150:35:02,  4.72s/it]
 49%|████▉     | 110212/225000 [232:34:32<128:49:34,  4.04s/it]
 49%|████▉     | 110213/225000 [232:34:38<145:22:10,  4.56s/it]
 49%|████▉     | 110214/225000 [232:34:49<212:49:14,  6.67s/it]
 49%|████▉     | 110215/225000 [232:34:53<184:58:28,  5.80s/it]
 49%|████▉     | 110216/225000 [232:34:56<158:02:47,  4.96s/it]
 49%|████▉     | 110217/225000 [232:35:01<154:56:10,  4.86s/it]
 49%|████▉     | 110218/225000 [232:35:03<129:36:29,  4.07s/it]
 49%|████▉     | 110219/225000 [232:35:08<141:15:13,  4.43s/it]
 49%|████▉     | 110220/225000 [232:35:12<138:54:57,  4.36s/it]
 49%|████▉     | 110221/225000 [232:35:16<131:13:09,  4.12s/it]
 49%|████▉     | 110222/225000 [232:35:24<168:50:35,  5.30s/it]
 49%|████▉     | 110223/225000 [232:35:26<138:52:44,  4.36s/it]
 49%|████▉     | 110224/225000 [232:35:36<187:26:58,  5.88s/it]
 49%|████▉     | 110225/225000 [232:35:38<151:53:21,  4.76s/it]
 49%|████▉     | 110226/225000 [232:35:40<127:42:37,  4.01s/it]
 49%|████▉     | 110227/225000 [232:35:43<118:11:34,  3.71s/it]
 49%|████▉     | 110228/225000 [232:35:48<127:55:35,  4.01s/it]
 49%|████▉     | 110229/225000 [232:35:50<111:40:07,  3.50s/it]
 49%|████▉     | 110230/225000 [232:35:54<113:43:33,  3.57s/it]
 49%|████▉     | 110231/225000 [232:36:01<149:28:51,  4.69s/it]
 49%|████▉     | 110232/225000 [232:36:03<126:59:22,  3.98s/it]
 49%|████▉     | 110233/225000 [232:36:06<112:04:21,  3.52s/it]
 49%|████▉     | 110234/225000 [232:36:10<120:52:08,  3.79s/it]
 49%|████▉     | 110235/225000 [232:36:14<120:26:56,  3.78s/it]
 49%|████▉     | 110236/225000 [232:36:17<112:15:36,  3.52s/it]
 49%|████▉     | 110237/225000 [232:36:21<120:46:23,  3.79s/it]
 49%|████▉     | 110238/225000 [232:36:24<113:16:15,  3.55s/it]
 49%|████▉     | 110239/225000 [232:36:27<108:49:10,  3.41s/it]
 49%|████▉     | 110240/225000 [232:36:31<104:56:48,  3.29s/it]
 49%|████▉     | 110241/225000 [232:36:34<108:54:13,  3.42s/it]
 49%|████▉     | 110242/225000 [232:36:38<110:46:43,  3.48s/it]
 49%|████▉     | 110243/225000 [232:36:41<106:55:56,  3.35s/it]
 49%|████▉     | 110244/225000 [232:36:51<174:20:26,  5.47s/it]
 49%|████▉     | 110245/225000 [232:36:54<146:54:32,  4.61s/it]
 49%|████▉     | 110246/225000 [232:36:57<134:22:14,  4.22s/it]
 49%|████▉     | 110247/225000 [232:37:00<118:03:18,  3.70s/it]
 49%|████▉     | 110248/225000 [232:37:10<176:15:25,  5.53s/it]
 49%|████▉     | 110249/225000 [232:37:12<145:15:05,  4.56s/it]
 49%|████▉     | 110250/225000 [232:37:15<133:23:58,  4.19s/it]
                                                               
{'loss': '0.8726', 'grad_norm': '2.56', 'learning_rate': '2.766e-05', 'epoch': '0.49', 'num_input_tokens_seen': 389475928, 'train_runtime': '8.374e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110250/225000 [232:37:15<133:23:58,  4.19s/it]
 49%|████▉     | 110251/225000 [232:37:17<114:32:59,  3.59s/it]
 49%|████▉     | 110252/225000 [232:37:22<128:01:26,  4.02s/it]
 49%|████▉     | 110253/225000 [232:37:33<193:09:33,  6.06s/it]
 49%|████▉     | 110254/225000 [232:37:39<191:33:51,  6.01s/it]
 49%|████▉     | 110255/225000 [232:37:41<157:23:12,  4.94s/it]
 49%|████▉     | 110256/225000 [232:37:47<161:51:20,  5.08s/it]
 49%|████▉     | 110257/225000 [232:37:51<149:52:54,  4.70s/it]
 49%|████▉     | 110258/225000 [232:37:53<127:06:14,  3.99s/it]
 49%|████▉     | 110259/225000 [232:37:56<115:19:41,  3.62s/it]
 49%|████▉     | 110260/225000 [232:38:06<180:41:06,  5.67s/it]
 49%|████▉     | 110261/225000 [232:38:09<157:03:46,  4.93s/it]
 49%|████▉     | 110262/225000 [232:38:19<202:36:02,  6.36s/it]
 49%|████▉     | 110263/225000 [232:38:22<169:05:17,  5.31s/it]
 49%|████▉     | 110264/225000 [232:38:25<146:49:25,  4.61s/it]
 49%|████▉     | 110265/225000 [232:38:33<175:55:16,  5.52s/it]
 49%|████▉     | 110266/225000 [232:38:36<159:06:29,  4.99s/it]
 49%|████▉     | 110267/225000 [232:38:41<159:36:05,  5.01s/it]
 49%|████▉     | 110268/225000 [232:38:56<251:18:21,  7.89s/it]
 49%|████▉     | 110269/225000 [232:39:01<226:13:59,  7.10s/it]
 49%|████▉     | 110270/225000 [232:39:04<181:15:21,  5.69s/it]
 49%|████▉     | 110271/225000 [232:39:06<149:16:33,  4.68s/it]
 49%|████▉     | 110272/225000 [232:39:08<127:02:46,  3.99s/it]
 49%|████▉     | 110273/225000 [232:39:13<130:55:04,  4.11s/it]
 49%|████▉     | 110274/225000 [232:39:15<114:39:43,  3.60s/it]
 49%|████▉     | 110275/225000 [232:39:18<110:11:26,  3.46s/it]
 49%|████▉     | 110276/225000 [232:39:21<99:47:42,  3.13s/it] 
 49%|████▉     | 110277/225000 [232:39:28<136:49:45,  4.29s/it]
 49%|████▉     | 110278/225000 [232:39:37<188:46:16,  5.92s/it]
 49%|████▉     | 110279/225000 [232:39:41<169:30:24,  5.32s/it]
 49%|████▉     | 110280/225000 [232:39:44<142:00:32,  4.46s/it]
 49%|████▉     | 110281/225000 [232:39:47<129:50:17,  4.07s/it]
 49%|████▉     | 110282/225000 [232:39:50<121:00:42,  3.80s/it]
 49%|████▉     | 110283/225000 [232:39:53<108:12:48,  3.40s/it]
 49%|████▉     | 110284/225000 [232:39:55<99:46:18,  3.13s/it] 
 49%|████▉     | 110285/225000 [232:39:58<93:32:21,  2.94s/it]
 49%|████▉     | 110286/225000 [232:40:00<91:58:28,  2.89s/it]
 49%|████▉     | 110287/225000 [232:40:04<98:26:04,  3.09s/it]
 49%|████▉     | 110288/225000 [232:40:07<95:52:20,  3.01s/it]
 49%|████▉     | 110289/225000 [232:40:11<105:26:03,  3.31s/it]
 49%|████▉     | 110290/225000 [232:40:14<103:12:44,  3.24s/it]
 49%|████▉     | 110291/225000 [232:40:17<103:13:04,  3.24s/it]
 49%|████▉     | 110292/225000 [232:40:25<147:16:16,  4.62s/it]
 49%|████▉     | 110293/225000 [232:40:33<185:30:39,  5.82s/it]
 49%|████▉     | 110294/225000 [232:40:36<151:37:55,  4.76s/it]
 49%|████▉     | 110295/225000 [232:40:38<127:59:25,  4.02s/it]
 49%|████▉     | 110296/225000 [232:40:40<111:52:18,  3.51s/it]
 49%|████▉     | 110297/225000 [232:40:43<100:18:50,  3.15s/it]
 49%|████▉     | 110298/225000 [232:40:54<177:03:01,  5.56s/it]
 49%|████▉     | 110299/225000 [232:40:57<152:19:15,  4.78s/it]
 49%|████▉     | 110300/225000 [232:40:59<127:51:43,  4.01s/it]
                                                               
{'loss': '0.8346', 'grad_norm': '5.959', 'learning_rate': '2.764e-05', 'epoch': '0.4902', 'num_input_tokens_seen': 389569744, 'train_runtime': '8.377e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110300/225000 [232:40:59<127:51:43,  4.01s/it]
 49%|████▉     | 110301/225000 [232:41:01<110:47:28,  3.48s/it]
 49%|████▉     | 110302/225000 [232:41:04<106:23:31,  3.34s/it]
 49%|████▉     | 110303/225000 [232:41:07<100:16:33,  3.15s/it]
 49%|████▉     | 110304/225000 [232:41:10<96:36:45,  3.03s/it] 
 49%|████▉     | 110305/225000 [232:41:13<94:13:22,  2.96s/it]
 49%|████▉     | 110306/225000 [232:41:16<98:41:58,  3.10s/it]
 49%|████▉     | 110307/225000 [232:41:25<159:57:24,  5.02s/it]
 49%|████▉     | 110308/225000 [232:41:28<136:28:38,  4.28s/it]
 49%|████▉     | 110309/225000 [232:41:31<120:05:28,  3.77s/it]
 49%|████▉     | 110310/225000 [232:41:33<108:07:16,  3.39s/it]
 49%|████▉     | 110311/225000 [232:41:36<107:02:19,  3.36s/it]
 49%|████▉     | 110312/225000 [232:41:47<177:35:42,  5.57s/it]
 49%|████▉     | 110313/225000 [232:41:56<205:43:41,  6.46s/it]
 49%|████▉     | 110314/225000 [232:41:58<168:10:39,  5.28s/it]
 49%|████▉     | 110315/225000 [232:42:01<142:27:38,  4.47s/it]
 49%|████▉     | 110316/225000 [232:42:03<124:47:01,  3.92s/it]
 49%|████▉     | 110317/225000 [232:42:06<111:22:38,  3.50s/it]
 49%|████▉     | 110318/225000 [232:42:09<110:08:17,  3.46s/it]
 49%|████▉     | 110319/225000 [232:42:12<105:16:26,  3.30s/it]
 49%|████▉     | 110320/225000 [232:42:15<97:14:00,  3.05s/it] 
 49%|████▉     | 110321/225000 [232:42:20<122:02:02,  3.83s/it]
 49%|████▉     | 110322/225000 [232:42:28<154:40:49,  4.86s/it]
 49%|████▉     | 110323/225000 [232:42:30<129:04:07,  4.05s/it]
 49%|████▉     | 110324/225000 [232:42:32<111:49:05,  3.51s/it]
 49%|████▉     | 110325/225000 [232:42:35<107:12:51,  3.37s/it]
 49%|████▉     | 110326/225000 [232:42:37<97:09:15,  3.05s/it] 
 49%|████▉     | 110327/225000 [232:42:51<198:48:51,  6.24s/it]
 49%|████▉     | 110328/225000 [232:42:56<188:38:55,  5.92s/it]
 49%|████▉     | 110329/225000 [232:42:58<152:59:55,  4.80s/it]
 49%|████▉     | 110330/225000 [232:43:03<147:29:10,  4.63s/it]
 49%|████▉     | 110331/225000 [232:43:06<136:28:45,  4.28s/it]
 49%|████▉     | 110332/225000 [232:43:09<127:06:49,  3.99s/it]
 49%|████▉     | 110333/225000 [232:43:16<153:10:25,  4.81s/it]
 49%|████▉     | 110334/225000 [232:43:22<163:54:24,  5.15s/it]
 49%|████▉     | 110335/225000 [232:43:27<163:11:33,  5.12s/it]
 49%|████▉     | 110336/225000 [232:43:43<268:34:50,  8.43s/it]
 49%|████▉     | 110337/225000 [232:43:47<223:54:31,  7.03s/it]
 49%|████▉     | 110338/225000 [232:43:49<178:12:44,  5.60s/it]
 49%|████▉     | 110339/225000 [232:43:52<146:11:43,  4.59s/it]
 49%|████▉     | 110340/225000 [232:43:57<157:14:33,  4.94s/it]
 49%|████▉     | 110341/225000 [232:44:09<225:02:17,  7.07s/it]
 49%|████▉     | 110342/225000 [232:44:12<178:14:02,  5.60s/it]
 49%|████▉     | 110343/225000 [232:44:17<176:33:56,  5.54s/it]
 49%|████▉     | 110344/225000 [232:44:25<204:28:19,  6.42s/it]
 49%|████▉     | 110345/225000 [232:44:32<203:33:10,  6.39s/it]
 49%|████▉     | 110346/225000 [232:44:34<165:21:53,  5.19s/it]
 49%|████▉     | 110347/225000 [232:44:37<139:38:57,  4.38s/it]
 49%|████▉     | 110348/225000 [232:44:43<157:33:39,  4.95s/it]
 49%|████▉     | 110349/225000 [232:44:45<134:23:57,  4.22s/it]
 49%|████▉     | 110350/225000 [232:44:48<117:47:02,  3.70s/it]
                                                               
{'loss': '0.9254', 'grad_norm': '5.906', 'learning_rate': '2.763e-05', 'epoch': '0.4904', 'num_input_tokens_seen': 389664680, 'train_runtime': '8.379e+05', 'train_tokens_per_second': '465.1'}

 49%|████▉     | 110350/225000 [232:44:48<117:47:02,  3.70s/it]
 49%|████▉     | 110351/225000 [232:44:51<113:04:52,  3.55s/it]
 49%|████▉     | 110352/225000 [232:44:57<131:57:38,  4.14s/it]
 49%|████▉     | 110353/225000 [232:45:06<178:35:27,  5.61s/it]
 49%|████▉     | 110354/225000 [232:45:09<153:36:53,  4.82s/it]
 49%|████▉     | 110355/225000 [232:45:11<130:28:29,  4.10s/it]
 49%|████▉     | 110356/225000 [232:45:14<116:23:41,  3.65s/it]
 49%|████▉     | 110357/225000 [232:45:23<174:12:32,  5.47s/it]
 49%|████▉     | 110358/225000 [232:45:26<143:43:33,  4.51s/it]
 49%|████▉     | 110359/225000 [232:45:28<123:01:23,  3.86s/it]
 49%|████▉     | 110360/225000 [232:45:30<107:53:00,  3.39s/it]
 49%|████▉     | 110361/225000 [232:45:33<97:23:28,  3.06s/it] 
 49%|████▉     | 110362/225000 [232:45:43<163:40:00,  5.14s/it]
 49%|████▉     | 110363/225000 [232:45:45<139:45:31,  4.39s/it]
 49%|████▉     | 110364/225000 [232:45:49<129:53:15,  4.08s/it]
 49%|████▉     | 110365/225000 [232:45:51<116:11:06,  3.65s/it]
 49%|████▉     | 110366/225000 [232:45:54<106:28:07,  3.34s/it]
 49%|████▉     | 110367/225000 [232:45:56<99:39:59,  3.13s/it] 
 49%|████▉     | 110368/225000 [232:45:59<94:44:50,  2.98s/it]
 49%|████▉     | 110369/225000 [232:46:09<160:07:59,  5.03s/it]
 49%|████▉     | 110370/225000 [232:46:11<133:47:37,  4.20s/it]
 49%|████▉     | 110371/225000 [232:46:13<115:44:09,  3.63s/it]
 49%|████▉     | 110372/225000 [232:46:16<102:49:52,  3.23s/it]
 49%|████▉     | 110373/225000 [232:46:18<94:30:36,  2.97s/it] 
 49%|████▉     | 110374/225000 [232:46:24<126:07:11,  3.96s/it]
 49%|████▉     | 110375/225000 [232:46:31<153:56:08,  4.83s/it]
 49%|████▉     | 110376/225000 [232:46:39<177:51:41,  5.59s/it]
 49%|████▉     | 110377/225000 [232:46:41<146:59:05,  4.62s/it]
 49%|████▉     | 110378/225000 [232:46:43<125:10:26,  3.93s/it]
 49%|████▉     | 110379/225000 [232:46:46<110:15:56,  3.46s/it]
 49%|████▉     | 110380/225000 [232:46:48<99:52:13,  3.14s/it] 
 49%|████▉     | 110381/225000 [232:46:50<92:29:14,  2.90s/it]
 49%|████▉     | 110382/225000 [232:47:01<168:39:07,  5.30s/it]
 49%|████▉     | 110383/225000 [232:47:04<139:28:01,  4.38s/it]
 49%|████▉     | 110384/225000 [232:47:06<119:09:19,  3.74s/it]
 49%|████▉     | 110385/225000 [232:47:15<176:00:11,  5.53s/it]
 49%|████▉     | 110386/225000 [232:47:18<148:16:04,  4.66s/it]
 49%|████▉     | 110387/225000 [232:47:20<125:32:25,  3.94s/it]
 49%|████▉     | 110388/225000 [232:47:23<110:50:58,  3.48s/it]
 49%|████▉     | 110389/225000 [232:47:26<108:23:41,  3.40s/it]
 49%|████▉     | 110390/225000 [232:47:29<106:17:37,  3.34s/it]
 49%|████▉     | 110391/225000 [232:47:32<105:03:20,  3.30s/it]
 49%|████▉     | 110392/225000 [232:47:42<169:02:19,  5.31s/it]
 49%|████▉     | 110393/225000 [232:47:45<140:06:04,  4.40s/it]
 49%|████▉     | 110394/225000 [232:48:00<244:39:24,  7.69s/it]
 49%|████▉     | 110395/225000 [232:48:03<196:12:05,  6.16s/it]
 49%|████▉     | 110396/225000 [232:48:05<163:08:06,  5.12s/it]
 49%|████▉     | 110397/225000 [232:48:08<139:53:33,  4.39s/it]
 49%|████▉     | 110398/225000 [232:48:11<123:30:16,  3.88s/it]
 49%|████▉     | 110399/225000 [232:48:18<153:15:28,  4.81s/it]
 49%|████▉     | 110400/225000 [232:48:22<150:07:56,  4.72s/it]
                                                               
{'loss': '0.8683', 'grad_norm': '2.819', 'learning_rate': '2.761e-05', 'epoch': '0.4907', 'num_input_tokens_seen': 389749200, 'train_runtime': '8.381e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110400/225000 [232:48:22<150:07:56,  4.72s/it]
 49%|████▉     | 110401/225000 [232:48:25<135:25:43,  4.25s/it]
 49%|████▉     | 110402/225000 [232:48:29<133:24:29,  4.19s/it]
 49%|████▉     | 110403/225000 [232:48:39<187:09:30,  5.88s/it]
 49%|████▉     | 110404/225000 [232:48:43<171:07:14,  5.38s/it]
 49%|████▉     | 110405/225000 [232:48:49<175:06:08,  5.50s/it]
 49%|████▉     | 110406/225000 [232:48:51<143:27:56,  4.51s/it]
 49%|████▉     | 110407/225000 [232:48:54<121:39:44,  3.82s/it]
 49%|████▉     | 110408/225000 [232:48:56<109:30:31,  3.44s/it]
 49%|████▉     | 110409/225000 [232:48:59<100:44:40,  3.17s/it]
 49%|████▉     | 110410/225000 [232:49:08<161:09:32,  5.06s/it]
 49%|████▉     | 110411/225000 [232:49:10<133:56:56,  4.21s/it]
 49%|████▉     | 110412/225000 [232:49:13<115:16:39,  3.62s/it]
 49%|████▉     | 110413/225000 [232:49:15<102:20:49,  3.22s/it]
 49%|████▉     | 110414/225000 [232:49:24<159:35:15,  5.01s/it]
 49%|████▉     | 110415/225000 [232:49:26<131:50:24,  4.14s/it]
 49%|████▉     | 110416/225000 [232:49:28<113:02:45,  3.55s/it]
 49%|████▉     | 110417/225000 [232:49:31<100:17:08,  3.15s/it]
 49%|████▉     | 110418/225000 [232:49:33<93:10:41,  2.93s/it] 
 49%|████▉     | 110419/225000 [232:49:36<92:08:55,  2.90s/it]
 49%|████▉     | 110420/225000 [232:49:39<91:10:37,  2.86s/it]
 49%|████▉     | 110421/225000 [232:49:41<90:02:10,  2.83s/it]
 49%|████▉     | 110422/225000 [232:49:44<89:16:18,  2.80s/it]
 49%|████▉     | 110423/225000 [232:49:47<89:01:32,  2.80s/it]
 49%|████▉     | 110424/225000 [232:49:57<159:42:51,  5.02s/it]
 49%|████▉     | 110425/225000 [232:50:05<186:12:51,  5.85s/it]
 49%|████▉     | 110426/225000 [232:50:07<153:30:38,  4.82s/it]
 49%|████▉     | 110427/225000 [232:50:10<137:29:34,  4.32s/it]
 49%|████▉     | 110428/225000 [232:50:13<121:44:20,  3.83s/it]
 49%|████▉     | 110429/225000 [232:50:19<137:28:59,  4.32s/it]
 49%|████▉     | 110430/225000 [232:50:21<120:02:04,  3.77s/it]
 49%|████▉     | 110431/225000 [232:50:24<107:22:48,  3.37s/it]
 49%|████▉     | 110432/225000 [232:50:26<98:51:31,  3.11s/it] 
 49%|████▉     | 110433/225000 [232:50:37<171:40:08,  5.39s/it]
 49%|████▉     | 110434/225000 [232:50:39<141:06:31,  4.43s/it]
 49%|████▉     | 110435/225000 [232:50:41<121:47:38,  3.83s/it]
 49%|████▉     | 110436/225000 [232:50:44<109:51:16,  3.45s/it]
 49%|████▉     | 110437/225000 [232:50:47<102:23:37,  3.22s/it]
 49%|████▉     | 110438/225000 [232:50:50<99:31:51,  3.13s/it] 
 49%|████▉     | 110439/225000 [232:50:52<94:52:30,  2.98s/it]
 49%|████▉     | 110440/225000 [232:50:55<94:30:40,  2.97s/it]
 49%|████▉     | 110441/225000 [232:50:58<90:57:45,  2.86s/it]
 49%|████▉     | 110442/225000 [232:51:06<147:07:59,  4.62s/it]
 49%|████▉     | 110443/225000 [232:51:09<125:04:18,  3.93s/it]
 49%|████▉     | 110444/225000 [232:51:11<111:47:42,  3.51s/it]
 49%|████▉     | 110445/225000 [232:51:14<102:41:23,  3.23s/it]
 49%|████▉     | 110446/225000 [232:51:16<96:30:57,  3.03s/it] 
 49%|████▉     | 110447/225000 [232:51:20<98:19:31,  3.09s/it]
 49%|████▉     | 110448/225000 [232:51:22<93:29:34,  2.94s/it]
 49%|████▉     | 110449/225000 [232:51:25<89:51:53,  2.82s/it]
 49%|████▉     | 110450/225000 [232:51:32<128:02:58,  4.02s/it]
                                                               
{'loss': '0.8954', 'grad_norm': '4.176', 'learning_rate': '2.759e-05', 'epoch': '0.4909', 'num_input_tokens_seen': 389823424, 'train_runtime': '8.383e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110450/225000 [232:51:32<128:02:58,  4.02s/it]
 49%|████▉     | 110451/225000 [232:51:41<175:53:43,  5.53s/it]
 49%|████▉     | 110452/225000 [232:51:49<200:53:45,  6.31s/it]
 49%|████▉     | 110453/225000 [232:51:51<161:12:10,  5.07s/it]
 49%|████▉     | 110454/225000 [232:51:54<141:41:50,  4.45s/it]
 49%|████▉     | 110455/225000 [232:51:56<120:12:16,  3.78s/it]
 49%|████▉     | 110456/225000 [232:51:58<105:20:23,  3.31s/it]
 49%|████▉     | 110457/225000 [232:52:05<137:26:15,  4.32s/it]
 49%|████▉     | 110458/225000 [232:52:07<117:06:20,  3.68s/it]
 49%|████▉     | 110459/225000 [232:52:12<129:34:14,  4.07s/it]
 49%|████▉     | 110460/225000 [232:52:15<112:48:46,  3.55s/it]
 49%|████▉     | 110461/225000 [232:52:17<102:46:41,  3.23s/it]
 49%|████▉     | 110462/225000 [232:52:26<155:34:07,  4.89s/it]
 49%|████▉     | 110463/225000 [232:52:29<134:18:47,  4.22s/it]
 49%|████▉     | 110464/225000 [232:52:31<119:02:28,  3.74s/it]
 49%|████▉     | 110465/225000 [232:52:34<108:24:17,  3.41s/it]
 49%|████▉     | 110466/225000 [232:52:36<100:57:31,  3.17s/it]
 49%|████▉     | 110467/225000 [232:52:40<101:44:28,  3.20s/it]
 49%|████▉     | 110468/225000 [232:52:42<96:46:33,  3.04s/it] 
 49%|████▉     | 110469/225000 [232:52:46<99:49:07,  3.14s/it]
 49%|████▉     | 110470/225000 [232:52:55<163:11:29,  5.13s/it]
 49%|████▉     | 110471/225000 [232:52:59<143:38:29,  4.52s/it]
 49%|████▉     | 110472/225000 [232:53:01<122:46:20,  3.86s/it]
 49%|████▉     | 110473/225000 [232:53:03<107:28:02,  3.38s/it]
 49%|████▉     | 110474/225000 [232:53:07<112:08:42,  3.53s/it]
 49%|████▉     | 110475/225000 [232:53:09<101:36:35,  3.19s/it]
 49%|████▉     | 110476/225000 [232:53:19<163:01:10,  5.12s/it]
 49%|████▉     | 110477/225000 [232:53:27<191:06:45,  6.01s/it]
 49%|████▉     | 110478/225000 [232:53:30<159:08:32,  5.00s/it]
 49%|████▉     | 110479/225000 [232:53:33<137:11:42,  4.31s/it]
 49%|████▉     | 110480/225000 [232:53:35<121:23:29,  3.82s/it]
 49%|████▉     | 110481/225000 [232:53:44<168:53:52,  5.31s/it]
 49%|████▉     | 110482/225000 [232:53:47<145:34:23,  4.58s/it]
 49%|████▉     | 110483/225000 [232:53:50<131:49:09,  4.14s/it]
 49%|████▉     | 110484/225000 [232:53:53<123:40:07,  3.89s/it]
 49%|████▉     | 110485/225000 [232:53:56<117:09:41,  3.68s/it]
 49%|████▉     | 110486/225000 [232:54:00<114:13:22,  3.59s/it]
 49%|████▉     | 110487/225000 [232:54:04<117:07:41,  3.68s/it]
 49%|████▉     | 110488/225000 [232:54:07<112:51:18,  3.55s/it]
 49%|████▉     | 110489/225000 [232:54:10<110:24:04,  3.47s/it]
 49%|████▉     | 110490/225000 [232:54:15<121:01:05,  3.80s/it]
 49%|████▉     | 110491/225000 [232:54:18<116:19:36,  3.66s/it]
 49%|████▉     | 110492/225000 [232:54:21<112:49:16,  3.55s/it]
 49%|████▉     | 110493/225000 [232:54:25<110:36:55,  3.48s/it]
 49%|████▉     | 110494/225000 [232:54:28<108:37:16,  3.41s/it]
 49%|████▉     | 110495/225000 [232:54:31<105:52:06,  3.33s/it]
 49%|████▉     | 110496/225000 [232:54:42<174:18:03,  5.48s/it]
 49%|████▉     | 110497/225000 [232:54:46<166:55:32,  5.25s/it]
 49%|████▉     | 110498/225000 [232:54:59<235:19:13,  7.40s/it]
 49%|████▉     | 110499/225000 [232:55:01<188:30:02,  5.93s/it]
 49%|████▉     | 110500/225000 [232:55:05<171:03:36,  5.38s/it]
                                                               
{'loss': '0.844', 'grad_norm': '2.119', 'learning_rate': '2.757e-05', 'epoch': '0.4911', 'num_input_tokens_seen': 389910416, 'train_runtime': '8.385e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110500/225000 [232:55:05<171:03:36,  5.38s/it]
 49%|████▉     | 110501/225000 [232:55:22<276:14:25,  8.69s/it]
 49%|████▉     | 110502/225000 [232:55:24<214:40:31,  6.75s/it]
 49%|████▉     | 110503/225000 [232:55:29<195:54:31,  6.16s/it]
 49%|████▉     | 110504/225000 [232:55:31<158:36:32,  4.99s/it]
 49%|████▉     | 110505/225000 [232:55:34<136:30:15,  4.29s/it]
 49%|████▉     | 110506/225000 [232:55:38<135:36:23,  4.26s/it]
 49%|████▉     | 110507/225000 [232:55:48<187:04:44,  5.88s/it]
 49%|████▉     | 110508/225000 [232:55:52<171:10:41,  5.38s/it]
 49%|████▉     | 110509/225000 [232:55:55<146:11:29,  4.60s/it]
 49%|████▉     | 110510/225000 [232:55:58<131:36:52,  4.14s/it]
 49%|████▉     | 110511/225000 [232:56:00<118:33:38,  3.73s/it]
 49%|████▉     | 110512/225000 [232:56:10<175:16:44,  5.51s/it]
 49%|████▉     | 110513/225000 [232:56:18<193:57:09,  6.10s/it]
 49%|████▉     | 110514/225000 [232:56:24<198:26:25,  6.24s/it]
 49%|████▉     | 110515/225000 [232:56:27<166:11:10,  5.23s/it]
 49%|████▉     | 110516/225000 [232:56:30<142:47:59,  4.49s/it]
 49%|████▉     | 110517/225000 [232:56:33<133:21:55,  4.19s/it]
 49%|████▉     | 110518/225000 [232:56:37<127:03:07,  4.00s/it]
 49%|████▉     | 110519/225000 [232:56:40<115:31:52,  3.63s/it]
 49%|████▉     | 110520/225000 [232:56:42<107:04:12,  3.37s/it]
 49%|████▉     | 110521/225000 [232:56:46<108:15:31,  3.40s/it]
 49%|████▉     | 110522/225000 [232:56:49<109:13:57,  3.44s/it]
 49%|████▉     | 110523/225000 [232:56:52<103:05:44,  3.24s/it]
 49%|████▉     | 110524/225000 [232:57:02<165:21:24,  5.20s/it]
 49%|████▉     | 110525/225000 [232:57:05<141:21:42,  4.45s/it]
 49%|████▉     | 110526/225000 [232:57:15<194:45:51,  6.12s/it]
 49%|████▉     | 110527/225000 [232:57:21<195:23:59,  6.15s/it]
 49%|████▉     | 110528/225000 [232:57:28<206:12:11,  6.48s/it]
 49%|████▉     | 110529/225000 [232:57:30<165:34:05,  5.21s/it]
 49%|████▉     | 110530/225000 [232:57:36<170:38:22,  5.37s/it]
 49%|████▉     | 110531/225000 [232:57:39<148:22:21,  4.67s/it]
 49%|████▉     | 110532/225000 [232:57:41<124:35:41,  3.92s/it]
 49%|████▉     | 110533/225000 [232:57:44<112:57:54,  3.55s/it]
 49%|████▉     | 110534/225000 [232:57:47<104:54:38,  3.30s/it]
 49%|████▉     | 110535/225000 [232:57:49<99:27:28,  3.13s/it] 
 49%|████▉     | 110536/225000 [232:58:00<170:44:12,  5.37s/it]
 49%|████▉     | 110537/225000 [232:58:05<163:12:47,  5.13s/it]
 49%|████▉     | 110538/225000 [232:58:10<170:14:25,  5.35s/it]
 49%|████▉     | 110539/225000 [232:58:13<141:18:33,  4.44s/it]
 49%|████▉     | 110540/225000 [232:58:18<146:44:11,  4.62s/it]
 49%|████▉     | 110541/225000 [232:58:29<209:21:14,  6.58s/it]
 49%|████▉     | 110542/225000 [232:58:31<167:53:45,  5.28s/it]
 49%|████▉     | 110543/225000 [232:58:33<139:16:11,  4.38s/it]
 49%|████▉     | 110544/225000 [232:58:36<119:14:39,  3.75s/it]
 49%|████▉     | 110545/225000 [232:58:38<105:28:56,  3.32s/it]
 49%|████▉     | 110546/225000 [232:58:53<213:04:18,  6.70s/it]
 49%|████▉     | 110547/225000 [232:58:55<173:55:05,  5.47s/it]
 49%|████▉     | 110548/225000 [232:58:58<151:07:19,  4.75s/it]
 49%|████▉     | 110549/225000 [232:59:05<172:44:14,  5.43s/it]
 49%|████▉     | 110550/225000 [232:59:08<145:27:19,  4.58s/it]
                                                               
{'loss': '0.8459', 'grad_norm': '5.761', 'learning_rate': '2.755e-05', 'epoch': '0.4913', 'num_input_tokens_seen': 390008440, 'train_runtime': '8.387e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110550/225000 [232:59:08<145:27:19,  4.58s/it]
 49%|████▉     | 110551/225000 [232:59:10<126:26:18,  3.98s/it]
 49%|████▉     | 110552/225000 [232:59:13<113:35:30,  3.57s/it]
 49%|████▉     | 110553/225000 [232:59:16<104:05:05,  3.27s/it]
 49%|████▉     | 110554/225000 [232:59:19<104:24:47,  3.28s/it]
 49%|████▉     | 110555/225000 [232:59:22<97:57:12,  3.08s/it] 
 49%|████▉     | 110556/225000 [232:59:31<157:43:55,  4.96s/it]
 49%|████▉     | 110557/225000 [232:59:39<183:48:49,  5.78s/it]
 49%|████▉     | 110558/225000 [232:59:41<150:30:52,  4.73s/it]
 49%|████▉     | 110559/225000 [232:59:44<134:16:28,  4.22s/it]
 49%|████▉     | 110560/225000 [232:59:47<123:29:06,  3.88s/it]
 49%|████▉     | 110561/225000 [232:59:49<108:27:46,  3.41s/it]
 49%|████▉     | 110562/225000 [232:59:52<97:39:36,  3.07s/it] 
 49%|████▉     | 110563/225000 [232:59:56<107:31:26,  3.38s/it]
 49%|████▉     | 110564/225000 [233:00:02<135:15:30,  4.26s/it]
 49%|████▉     | 110565/225000 [233:00:05<119:05:41,  3.75s/it]
 49%|████▉     | 110566/225000 [233:00:08<114:46:54,  3.61s/it]
 49%|████▉     | 110567/225000 [233:00:17<168:32:54,  5.30s/it]
 49%|████▉     | 110568/225000 [233:00:19<139:05:14,  4.38s/it]
 49%|████▉     | 110569/225000 [233:00:22<118:53:01,  3.74s/it]
 49%|████▉     | 110570/225000 [233:00:24<104:46:00,  3.30s/it]
 49%|████▉     | 110571/225000 [233:00:27<101:43:25,  3.20s/it]
 49%|████▉     | 110572/225000 [233:00:29<92:41:56,  2.92s/it] 
 49%|████▉     | 110573/225000 [233:00:37<143:12:23,  4.51s/it]
 49%|████▉     | 110574/225000 [233:00:46<183:11:42,  5.76s/it]
 49%|████▉     | 110575/225000 [233:00:53<196:11:18,  6.17s/it]
 49%|████▉     | 110576/225000 [233:00:59<189:58:06,  5.98s/it]
 49%|████▉     | 110577/225000 [233:01:01<154:30:26,  4.86s/it]
 49%|████▉     | 110578/225000 [233:01:03<129:25:47,  4.07s/it]
 49%|████▉     | 110579/225000 [233:01:08<135:39:20,  4.27s/it]
 49%|████▉     | 110580/225000 [233:01:11<120:28:36,  3.79s/it]
 49%|████▉     | 110581/225000 [233:01:14<112:59:26,  3.56s/it]
 49%|████▉     | 110582/225000 [233:01:30<236:53:18,  7.45s/it]
 49%|████▉     | 110583/225000 [233:01:35<209:55:14,  6.60s/it]
 49%|████▉     | 110584/225000 [233:01:37<169:41:29,  5.34s/it]
 49%|████▉     | 110585/225000 [233:01:40<142:29:03,  4.48s/it]
 49%|████▉     | 110586/225000 [233:01:49<191:45:13,  6.03s/it]
 49%|████▉     | 110587/225000 [233:01:55<187:56:55,  5.91s/it]
 49%|████▉     | 110588/225000 [233:01:57<155:36:11,  4.90s/it]
 49%|████▉     | 110589/225000 [233:02:00<130:49:18,  4.12s/it]
 49%|████▉     | 110590/225000 [233:02:02<113:15:39,  3.56s/it]
 49%|████▉     | 110591/225000 [233:02:04<100:46:54,  3.17s/it]
 49%|████▉     | 110592/225000 [233:02:08<109:35:00,  3.45s/it]
 49%|████▉     | 110593/225000 [233:02:13<119:54:50,  3.77s/it]
 49%|████▉     | 110594/225000 [233:02:15<104:23:01,  3.28s/it]
 49%|████▉     | 110595/225000 [233:02:18<99:10:49,  3.12s/it] 
 49%|████▉     | 110596/225000 [233:02:28<168:13:14,  5.29s/it]
 49%|████▉     | 110597/225000 [233:02:30<138:47:59,  4.37s/it]
 49%|████▉     | 110598/225000 [233:02:33<118:40:28,  3.73s/it]
 49%|████▉     | 110599/225000 [233:02:35<104:11:16,  3.28s/it]
 49%|████▉     | 110600/225000 [233:02:49<203:52:11,  6.42s/it]
                                                               
{'loss': '0.8216', 'grad_norm': '1.582', 'learning_rate': '2.754e-05', 'epoch': '0.4916', 'num_input_tokens_seen': 390097264, 'train_runtime': '8.39e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110600/225000 [233:02:49<203:52:11,  6.42s/it]
 49%|████▉     | 110601/225000 [233:02:51<163:16:31,  5.14s/it]
 49%|████▉     | 110602/225000 [233:02:53<135:19:10,  4.26s/it]
 49%|████▉     | 110603/225000 [233:02:55<116:12:43,  3.66s/it]
 49%|████▉     | 110604/225000 [233:02:57<102:25:00,  3.22s/it]
 49%|████▉     | 110605/225000 [233:03:00<100:17:58,  3.16s/it]
 49%|████▉     | 110606/225000 [233:03:05<117:03:32,  3.68s/it]
 49%|████▉     | 110607/225000 [233:03:11<141:11:43,  4.44s/it]
 49%|████▉     | 110608/225000 [233:03:14<122:47:26,  3.86s/it]
 49%|████▉     | 110609/225000 [233:03:17<117:12:16,  3.69s/it]
 49%|████▉     | 110610/225000 [233:03:20<106:05:20,  3.34s/it]
 49%|████▉     | 110611/225000 [233:03:22<98:45:38,  3.11s/it] 
 49%|████▉     | 110612/225000 [233:03:25<93:18:21,  2.94s/it]
 49%|████▉     | 110613/225000 [233:03:35<163:48:36,  5.16s/it]
 49%|████▉     | 110614/225000 [233:03:39<150:16:19,  4.73s/it]
 49%|████▉     | 110615/225000 [233:03:44<155:33:58,  4.90s/it]
 49%|████▉     | 110616/225000 [233:03:47<130:21:58,  4.10s/it]
 49%|████▉     | 110617/225000 [233:03:54<164:07:47,  5.17s/it]
 49%|████▉     | 110618/225000 [233:04:00<167:18:35,  5.27s/it]
 49%|████▉     | 110619/225000 [233:04:05<171:47:53,  5.41s/it]
 49%|████▉     | 110620/225000 [233:04:11<172:29:59,  5.43s/it]
 49%|████▉     | 110621/225000 [233:04:14<148:49:29,  4.68s/it]
 49%|████▉     | 110622/225000 [233:04:17<133:25:40,  4.20s/it]
 49%|████▉     | 110623/225000 [233:04:20<121:59:47,  3.84s/it]
 49%|████▉     | 110624/225000 [233:04:23<114:31:42,  3.60s/it]
 49%|████▉     | 110625/225000 [233:04:29<140:18:47,  4.42s/it]
 49%|████▉     | 110626/225000 [233:04:33<134:13:49,  4.22s/it]
 49%|████▉     | 110627/225000 [233:04:36<122:11:51,  3.85s/it]
 49%|████▉     | 110628/225000 [233:04:39<114:33:46,  3.61s/it]
 49%|████▉     | 110629/225000 [233:04:43<115:46:47,  3.64s/it]
 49%|████▉     | 110630/225000 [233:04:53<176:13:05,  5.55s/it]
 49%|████▉     | 110631/225000 [233:04:58<169:53:11,  5.35s/it]
 49%|████▉     | 110632/225000 [233:05:01<148:10:48,  4.66s/it]
 49%|████▉     | 110633/225000 [233:05:03<125:33:59,  3.95s/it]
 49%|████▉     | 110634/225000 [233:05:05<108:59:25,  3.43s/it]
 49%|████▉     | 110635/225000 [233:05:11<129:10:43,  4.07s/it]
 49%|████▉     | 110636/225000 [233:05:15<127:32:25,  4.01s/it]
 49%|████▉     | 110637/225000 [233:05:17<111:53:05,  3.52s/it]
 49%|████▉     | 110638/225000 [233:05:22<128:19:10,  4.04s/it]
 49%|████▉     | 110639/225000 [233:05:25<112:28:41,  3.54s/it]
 49%|████▉     | 110640/225000 [233:05:27<101:47:20,  3.20s/it]
 49%|████▉     | 110641/225000 [233:05:30<101:26:55,  3.19s/it]
 49%|████▉     | 110642/225000 [233:05:38<141:29:06,  4.45s/it]
 49%|████▉     | 110643/225000 [233:05:43<146:13:40,  4.60s/it]
 49%|████▉     | 110644/225000 [233:05:46<139:12:23,  4.38s/it]
 49%|████▉     | 110645/225000 [233:05:49<119:26:56,  3.76s/it]
 49%|████▉     | 110646/225000 [233:06:07<256:40:25,  8.08s/it]
 49%|████▉     | 110647/225000 [233:06:10<206:42:46,  6.51s/it]
 49%|████▉     | 110648/225000 [233:06:12<167:24:13,  5.27s/it]
 49%|████▉     | 110649/225000 [233:06:15<145:28:32,  4.58s/it]
 49%|████▉     | 110650/225000 [233:06:17<122:14:44,  3.85s/it]
                                                               
{'loss': '0.8646', 'grad_norm': '4.118', 'learning_rate': '2.752e-05', 'epoch': '0.4918', 'num_input_tokens_seen': 390184776, 'train_runtime': '8.392e+05', 'train_tokens_per_second': '465'}

 49%|████▉     | 110650/225000 [233:06:17<122:14:44,  3.85s/it]
 49%|████▉     | 110651/225000 [233:06:22<135:03:28,  4.25s/it]
 49%|████▉     | 110652/225000 [233:06:27<140:52:23,  4.44s/it]
 49%|████▉     | 110653/225000 [233:06:30<119:36:02,  3.77s/it]
 49%|████▉     | 110654/225000 [233:06:34<128:58:03,  4.06s/it]
 49%|████▉     | 110655/225000 [233:06:39<137:14:59,  4.32s/it]
 49%|████▉     | 110656/225000 [233:06:42<123:25:46,  3.89s/it]
 49%|████▉     | 110657/225000 [233:06:45<113:39:40,  3.58s/it]
 49%|████▉     | 110658/225000 [233:06:48<105:50:07,  3.33s/it]
 49%|████▉     | 110659/225000 [233:06:51<101:19:48,  3.19s/it]
 49%|████▉     | 110660/225000 [233:06:53<98:20:21,  3.10s/it] 
 49%|████▉     | 110661/225000 [233:06:56<95:51:42,  3.02s/it]
 49%|████▉     | 110662/225000 [233:06:59<93:57:05,  2.96s/it]
 49%|████▉     | 110663/225000 [233:07:02<93:01:08,  2.93s/it]
 49%|████▉     | 110664/225000 [233:07:06<105:28:49,  3.32s/it]
 49%|████▉     | 110665/225000 [233:07:16<171:01:12,  5.38s/it]
 49%|████▉     | 110666/225000 [233:07:19<145:22:50,  4.58s/it]
 49%|████▉     | 110667/225000 [233:07:22<132:00:28,  4.16s/it]
 49%|████▉     | 110668/225000 [233:07:25<122:20:15,  3.85s/it]
 49%|████▉     | 110669/225000 [233:07:29<115:51:39,  3.65s/it]
 49%|████▉     | 110670/225000 [233:07:31<104:16:49,  3.28s/it]
 49%|████▉     | 110671/225000 [233:07:39<153:13:38,  4.82s/it]
 49%|████▉     | 110672/225000 [233:07:42<131:06:11,  4.13s/it]
 49%|████▉     | 110673/225000 [233:07:52<184:52:37,  5.82s/it]
 49%|████▉     | 110674/225000 [233:07:54<150:42:59,  4.75s/it]
 49%|████▉     | 110675/225000 [233:08:06<217:57:44,  6.86s/it]
 49%|████▉     | 110676/225000 [233:08:10<191:44:53,  6.04s/it]
 49%|████▉     | 110677/225000 [233:08:12<159:27:47,  5.02s/it]
 49%|████▉     | 110678/225000 [233:08:21<189:23:08,  5.96s/it]
 49%|████▉     | 110679/225000 [233:08:23<153:17:10,  4.83s/it]
 49%|████▉     | 110680/225000 [233:08:25<128:22:30,  4.04s/it]
 49%|████▉     | 110681/225000 [233:08:30<136:27:28,  4.30s/it]
 49%|████▉     | 110682/225000 [233:08:37<160:01:47,  5.04s/it]
 49%|████▉     | 110683/225000 [233:08:39<135:30:31,  4.27s/it]
 49%|████▉     | 110684/225000 [233:08:42<125:53:06,  3.96s/it]
 49%|████▉     | 110685/225000 [233:08:45<111:40:11,  3.52s/it]
 49%|████▉     | 110686/225000 [233:08:56<180:37:19,  5.69s/it]
 49%|████▉     | 110687/225000 [233:09:01<175:20:19,  5.52s/it]
 49%|████▉     | 110688/225000 [233:09:04<154:04:00,  4.85s/it]
 49%|████▉     | 110689/225000 [233:09:06<128:43:32,  4.05s/it]
 49%|████▉     | 110690/225000 [233:09:09<117:55:01,  3.71s/it]
 49%|████▉     | 110691/225000 [233:09:17<155:33:22,  4.90s/it]
 49%|████▉     | 110692/225000 [233:09:25<186:19:34,  5.87s/it]
 49%|████▉     | 110693/225000 [233:09:27<150:37:33,  4.74s/it]
 49%|████▉     | 110694/225000 [233:09:33<163:53:17,  5.16s/it]
 49%|████▉     | 110695/225000 [233:09:36<138:33:58,  4.36s/it]
 49%|████▉     | 110696/225000 [233:09:44<173:33:46,  5.47s/it]
 49%|████▉     | 110697/225000 [233:09:48<159:28:42,  5.02s/it]
 49%|████▉     | 110698/225000 [233:09:50<135:37:24,  4.27s/it]
 49%|████▉     | 110699/225000 [233:09:53<125:27:23,  3.95s/it]
 49%|████▉     | 110700/225000 [233:09:56<112:06:15,  3.53s/it]
                                                               
{'loss': '0.8937', 'grad_norm': '5.559', 'learning_rate': '2.75e-05', 'epoch': '0.492', 'num_input_tokens_seen': 390273192, 'train_runtime': '8.394e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110700/225000 [233:09:56<112:06:15,  3.53s/it]
 49%|████▉     | 110701/225000 [233:09:59<102:44:17,  3.24s/it]
 49%|████▉     | 110702/225000 [233:10:01<95:32:06,  3.01s/it] 
 49%|████▉     | 110703/225000 [233:10:04<91:00:54,  2.87s/it]
 49%|████▉     | 110704/225000 [233:10:06<88:10:22,  2.78s/it]
 49%|████▉     | 110705/225000 [233:10:12<117:26:05,  3.70s/it]
 49%|████▉     | 110706/225000 [233:10:19<144:44:20,  4.56s/it]
 49%|████▉     | 110707/225000 [233:10:21<122:08:46,  3.85s/it]
 49%|████▉     | 110708/225000 [233:10:24<113:47:27,  3.58s/it]
 49%|████▉     | 110709/225000 [233:10:27<108:19:00,  3.41s/it]
 49%|████▉     | 110710/225000 [233:10:30<106:46:47,  3.36s/it]
 49%|████▉     | 110711/225000 [233:10:33<104:50:06,  3.30s/it]
 49%|████▉     | 110712/225000 [233:10:36<104:13:30,  3.28s/it]
 49%|████▉     | 110713/225000 [233:10:40<110:32:13,  3.48s/it]
 49%|████▉     | 110714/225000 [233:10:51<181:07:51,  5.71s/it]
 49%|████▉     | 110715/225000 [233:10:54<151:45:14,  4.78s/it]
 49%|████▉     | 110716/225000 [233:11:11<272:43:48,  8.59s/it]
 49%|████▉     | 110717/225000 [233:11:14<217:10:59,  6.84s/it]
 49%|████▉     | 110718/225000 [233:11:17<180:16:04,  5.68s/it]
 49%|████▉     | 110719/225000 [233:11:19<147:29:45,  4.65s/it]
 49%|████▉     | 110720/225000 [233:11:22<124:31:46,  3.92s/it]
 49%|████▉     | 110721/225000 [233:11:26<133:51:05,  4.22s/it]
 49%|████▉     | 110722/225000 [233:11:29<122:24:26,  3.86s/it]
 49%|████▉     | 110723/225000 [233:11:33<118:38:28,  3.74s/it]
 49%|████▉     | 110724/225000 [233:11:43<179:53:48,  5.67s/it]
 49%|████▉     | 110725/225000 [233:11:47<167:07:11,  5.26s/it]
 49%|████▉     | 110726/225000 [233:11:50<142:31:13,  4.49s/it]
 49%|████▉     | 110727/225000 [233:11:53<128:10:38,  4.04s/it]
 49%|████▉     | 110728/225000 [233:11:59<147:26:29,  4.64s/it]
 49%|████▉     | 110729/225000 [233:12:03<135:14:12,  4.26s/it]
 49%|████▉     | 110730/225000 [233:12:12<182:40:29,  5.76s/it]
 49%|████▉     | 110731/225000 [233:12:15<160:39:32,  5.06s/it]
 49%|████▉     | 110732/225000 [233:12:18<141:49:52,  4.47s/it]
 49%|████▉     | 110733/225000 [233:12:21<122:51:27,  3.87s/it]
 49%|████▉     | 110734/225000 [233:12:25<126:35:57,  3.99s/it]
 49%|████▉     | 110735/225000 [233:12:28<119:38:35,  3.77s/it]
 49%|████▉     | 110736/225000 [233:12:31<107:20:05,  3.38s/it]
 49%|████▉     | 110737/225000 [233:12:34<105:50:41,  3.33s/it]
 49%|████▉     | 110738/225000 [233:12:38<112:29:16,  3.54s/it]
 49%|████▉     | 110739/225000 [233:12:40<102:17:06,  3.22s/it]
 49%|████▉     | 110740/225000 [233:12:45<116:02:04,  3.66s/it]
 49%|████▉     | 110741/225000 [233:12:57<196:36:28,  6.19s/it]
 49%|████▉     | 110742/225000 [233:13:02<185:38:49,  5.85s/it]
 49%|████▉     | 110743/225000 [233:13:05<152:17:07,  4.80s/it]
 49%|████▉     | 110744/225000 [233:13:07<128:33:16,  4.05s/it]
 49%|████▉     | 110745/225000 [233:13:10<121:11:00,  3.82s/it]
 49%|████▉     | 110746/225000 [233:13:14<123:20:16,  3.89s/it]
 49%|████▉     | 110747/225000 [233:13:20<145:06:56,  4.57s/it]
 49%|████▉     | 110748/225000 [233:13:23<122:29:04,  3.86s/it]
 49%|████▉     | 110749/225000 [233:13:26<114:37:57,  3.61s/it]
 49%|████▉     | 110750/225000 [233:13:30<121:00:40,  3.81s/it]
                                                               
{'loss': '0.8788', 'grad_norm': '2.273', 'learning_rate': '2.748e-05', 'epoch': '0.4922', 'num_input_tokens_seen': 390365136, 'train_runtime': '8.396e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110750/225000 [233:13:30<121:00:40,  3.81s/it]
 49%|████▉     | 110751/225000 [233:13:33<113:01:46,  3.56s/it]
 49%|████▉     | 110752/225000 [233:13:36<107:11:11,  3.38s/it]
 49%|████▉     | 110753/225000 [233:13:46<174:58:35,  5.51s/it]
 49%|████▉     | 110754/225000 [233:13:50<157:38:12,  4.97s/it]
 49%|████▉     | 110755/225000 [233:13:54<144:24:36,  4.55s/it]
 49%|████▉     | 110756/225000 [233:13:57<129:21:30,  4.08s/it]
 49%|████▉     | 110757/225000 [233:14:02<141:20:00,  4.45s/it]
 49%|████▉     | 110758/225000 [233:14:05<131:48:24,  4.15s/it]
 49%|████▉     | 110759/225000 [233:14:10<138:01:26,  4.35s/it]
 49%|████▉     | 110760/225000 [233:14:13<119:22:33,  3.76s/it]
 49%|████▉     | 110761/225000 [233:14:15<109:45:01,  3.46s/it]
 49%|████▉     | 110762/225000 [233:14:18<102:55:57,  3.24s/it]
 49%|████▉     | 110763/225000 [233:14:33<212:20:57,  6.69s/it]
 49%|████▉     | 110764/225000 [233:14:41<228:53:11,  7.21s/it]
 49%|████▉     | 110765/225000 [233:14:47<215:07:17,  6.78s/it]
 49%|████▉     | 110766/225000 [233:14:55<227:21:23,  7.16s/it]
 49%|████▉     | 110767/225000 [233:14:57<180:20:08,  5.68s/it]
 49%|████▉     | 110768/225000 [233:15:01<164:06:51,  5.17s/it]
 49%|████▉     | 110769/225000 [233:15:04<143:04:29,  4.51s/it]
 49%|████▉     | 110770/225000 [233:15:07<128:14:03,  4.04s/it]
 49%|████▉     | 110771/225000 [233:15:16<176:27:31,  5.56s/it]
 49%|████▉     | 110772/225000 [233:15:19<146:55:25,  4.63s/it]
 49%|████▉     | 110773/225000 [233:15:21<127:51:46,  4.03s/it]
 49%|████▉     | 110774/225000 [233:15:24<114:11:16,  3.60s/it]
 49%|████▉     | 110775/225000 [233:15:27<111:47:27,  3.52s/it]
 49%|████▉     | 110776/225000 [233:15:30<103:19:11,  3.26s/it]
 49%|████▉     | 110777/225000 [233:15:34<106:18:28,  3.35s/it]
 49%|████▉     | 110778/225000 [233:15:36<99:46:28,  3.14s/it] 
 49%|████▉     | 110779/225000 [233:15:39<95:21:45,  3.01s/it]
 49%|████▉     | 110780/225000 [233:15:42<91:46:14,  2.89s/it]
 49%|████▉     | 110781/225000 [233:15:44<88:45:30,  2.80s/it]
 49%|████▉     | 110782/225000 [233:15:53<149:04:03,  4.70s/it]
 49%|████▉     | 110783/225000 [233:15:56<132:58:14,  4.19s/it]
 49%|████▉     | 110784/225000 [233:15:59<117:59:07,  3.72s/it]
 49%|████▉     | 110785/225000 [233:16:02<107:58:53,  3.40s/it]
 49%|████▉     | 110786/225000 [233:16:04<100:11:09,  3.16s/it]
 49%|████▉     | 110787/225000 [233:16:07<101:40:44,  3.20s/it]
 49%|████▉     | 110788/225000 [233:16:20<189:39:34,  5.98s/it]
 49%|████▉     | 110789/225000 [233:16:22<155:13:00,  4.89s/it]
 49%|████▉     | 110790/225000 [233:16:24<129:57:40,  4.10s/it]
 49%|████▉     | 110791/225000 [233:16:27<117:18:19,  3.70s/it]
 49%|████▉     | 110792/225000 [233:16:39<193:33:26,  6.10s/it]
 49%|████▉     | 110793/225000 [233:16:41<156:14:28,  4.92s/it]
 49%|████▉     | 110794/225000 [233:16:48<177:31:16,  5.60s/it]
 49%|████▉     | 110795/225000 [233:16:53<164:58:21,  5.20s/it]
 49%|████▉     | 110796/225000 [233:16:55<137:06:17,  4.32s/it]
 49%|████▉     | 110797/225000 [233:16:57<118:34:51,  3.74s/it]
 49%|████▉     | 110798/225000 [233:17:06<168:07:52,  5.30s/it]
 49%|████▉     | 110799/225000 [233:17:08<138:51:35,  4.38s/it]
 49%|████▉     | 110800/225000 [233:17:11<125:37:49,  3.96s/it]
                                                               
{'loss': '0.8186', 'grad_norm': '2.967', 'learning_rate': '2.746e-05', 'epoch': '0.4924', 'num_input_tokens_seen': 390453496, 'train_runtime': '8.398e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110800/225000 [233:17:11<125:37:49,  3.96s/it]
 49%|████▉     | 110801/225000 [233:17:14<116:10:02,  3.66s/it]
 49%|████▉     | 110802/225000 [233:17:23<163:30:57,  5.15s/it]
 49%|████▉     | 110803/225000 [233:17:25<136:13:15,  4.29s/it]
 49%|████▉     | 110804/225000 [233:17:28<117:33:38,  3.71s/it]
 49%|████▉     | 110805/225000 [233:17:30<104:55:18,  3.31s/it]
 49%|████▉     | 110806/225000 [233:17:33<102:41:41,  3.24s/it]
 49%|████▉     | 110807/225000 [233:17:35<94:29:51,  2.98s/it] 
 49%|████▉     | 110808/225000 [233:17:38<88:22:25,  2.79s/it]
 49%|████▉     | 110809/225000 [233:17:41<91:10:33,  2.87s/it]
 49%|████▉     | 110810/225000 [233:17:43<85:54:30,  2.71s/it]
 49%|████▉     | 110811/225000 [233:17:51<135:28:18,  4.27s/it]
 49%|████▉     | 110812/225000 [233:17:54<118:18:56,  3.73s/it]
 49%|████▉     | 110813/225000 [233:17:57<114:25:27,  3.61s/it]
 49%|████▉     | 110814/225000 [233:17:59<101:13:49,  3.19s/it]
 49%|████▉     | 110815/225000 [233:18:10<171:42:14,  5.41s/it]
 49%|████▉     | 110816/225000 [233:18:13<150:20:43,  4.74s/it]
 49%|████▉     | 110817/225000 [233:18:16<135:07:16,  4.26s/it]
 49%|████▉     | 110818/225000 [233:18:18<117:11:12,  3.69s/it]
 49%|████▉     | 110819/225000 [233:18:21<104:27:17,  3.29s/it]
 49%|████▉     | 110820/225000 [233:18:27<131:41:47,  4.15s/it]
 49%|████▉     | 110821/225000 [233:18:30<122:27:20,  3.86s/it]
 49%|████▉     | 110822/225000 [233:18:33<116:17:39,  3.67s/it]
 49%|████▉     | 110823/225000 [233:18:40<140:44:32,  4.44s/it]
 49%|████▉     | 110824/225000 [233:18:48<178:03:10,  5.61s/it]
 49%|████▉     | 110825/225000 [233:18:57<211:20:17,  6.66s/it]
 49%|████▉     | 110826/225000 [233:18:59<168:50:40,  5.32s/it]
 49%|████▉     | 110827/225000 [233:19:01<139:48:07,  4.41s/it]
 49%|████▉     | 110828/225000 [233:19:11<185:44:14,  5.86s/it]
 49%|████▉     | 110829/225000 [233:19:16<177:09:53,  5.59s/it]
 49%|████▉     | 110830/225000 [233:19:21<174:03:32,  5.49s/it]
 49%|████▉     | 110831/225000 [233:19:24<153:38:40,  4.84s/it]
 49%|████▉     | 110832/225000 [233:19:35<213:12:13,  6.72s/it]
 49%|████▉     | 110833/225000 [233:19:38<171:30:14,  5.41s/it]
 49%|████▉     | 110834/225000 [233:19:40<141:55:50,  4.48s/it]
 49%|████▉     | 110835/225000 [233:19:42<121:28:08,  3.83s/it]
 49%|████▉     | 110836/225000 [233:19:45<107:18:10,  3.38s/it]
 49%|████▉     | 110837/225000 [233:19:51<139:01:02,  4.38s/it]
 49%|████▉     | 110838/225000 [233:19:55<133:09:11,  4.20s/it]
 49%|████▉     | 110839/225000 [233:19:57<115:17:04,  3.64s/it]
 49%|████▉     | 110840/225000 [233:20:01<110:11:13,  3.47s/it]
 49%|████▉     | 110841/225000 [233:20:03<99:29:33,  3.14s/it] 
 49%|████▉     | 110842/225000 [233:20:06<99:13:51,  3.13s/it]
 49%|████▉     | 110843/225000 [233:20:08<92:15:45,  2.91s/it]
 49%|████▉     | 110844/225000 [233:20:11<91:03:31,  2.87s/it]
 49%|████▉     | 110845/225000 [233:20:21<152:40:20,  4.81s/it]
 49%|████▉     | 110846/225000 [233:20:29<183:19:58,  5.78s/it]
 49%|████▉     | 110847/225000 [233:20:31<150:41:33,  4.75s/it]
 49%|████▉     | 110848/225000 [233:20:35<144:25:59,  4.55s/it]
 49%|████▉     | 110849/225000 [233:20:38<125:31:04,  3.96s/it]
 49%|████▉     | 110850/225000 [233:20:40<112:22:12,  3.54s/it]
                                                               
{'loss': '0.8344', 'grad_norm': '5.526', 'learning_rate': '2.744e-05', 'epoch': '0.4927', 'num_input_tokens_seen': 390544008, 'train_runtime': '8.4e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110850/225000 [233:20:40<112:22:12,  3.54s/it]
 49%|████▉     | 110851/225000 [233:20:43<103:17:05,  3.26s/it]
 49%|████▉     | 110852/225000 [233:20:47<109:55:39,  3.47s/it]
 49%|████▉     | 110853/225000 [233:20:50<107:40:51,  3.40s/it]
 49%|████▉     | 110854/225000 [233:20:53<106:26:03,  3.36s/it]
 49%|████▉     | 110855/225000 [233:20:56<105:13:18,  3.32s/it]
 49%|████▉     | 110856/225000 [233:21:00<104:59:01,  3.31s/it]
 49%|████▉     | 110857/225000 [233:21:03<104:28:22,  3.30s/it]
 49%|████▉     | 110858/225000 [233:21:07<109:10:07,  3.44s/it]
 49%|████▉     | 110859/225000 [233:21:10<106:41:25,  3.37s/it]
 49%|████▉     | 110860/225000 [233:21:13<105:15:35,  3.32s/it]
 49%|████▉     | 110861/225000 [233:21:19<128:50:41,  4.06s/it]
 49%|████▉     | 110862/225000 [233:21:23<124:20:37,  3.92s/it]
 49%|████▉     | 110863/225000 [233:21:28<135:03:57,  4.26s/it]
 49%|████▉     | 110864/225000 [233:21:31<123:39:05,  3.90s/it]
 49%|████▉     | 110865/225000 [233:21:41<181:27:22,  5.72s/it]
 49%|████▉     | 110866/225000 [233:21:44<159:08:57,  5.02s/it]
 49%|████▉     | 110867/225000 [233:21:47<135:24:06,  4.27s/it]
 49%|████▉     | 110868/225000 [233:21:49<119:18:38,  3.76s/it]
 49%|████▉     | 110869/225000 [233:21:52<108:21:35,  3.42s/it]
 49%|████▉     | 110870/225000 [233:21:59<146:06:26,  4.61s/it]
 49%|████▉     | 110871/225000 [233:22:02<127:03:02,  4.01s/it]
 49%|████▉     | 110872/225000 [233:22:04<113:30:16,  3.58s/it]
 49%|████▉     | 110873/225000 [233:22:08<109:26:16,  3.45s/it]
 49%|████▉     | 110874/225000 [233:22:10<101:55:07,  3.21s/it]
 49%|████▉     | 110875/225000 [233:22:15<113:04:31,  3.57s/it]
 49%|████▉     | 110876/225000 [233:22:17<104:15:24,  3.29s/it]
 49%|████▉     | 110877/225000 [233:22:20<98:05:56,  3.09s/it] 
 49%|████▉     | 110878/225000 [233:22:23<96:40:48,  3.05s/it]
 49%|████▉     | 110879/225000 [233:22:26<96:26:02,  3.04s/it]
 49%|████▉     | 110880/225000 [233:22:34<146:15:23,  4.61s/it]
 49%|████▉     | 110881/225000 [233:22:38<143:35:56,  4.53s/it]
 49%|████▉     | 110882/225000 [233:22:44<151:47:21,  4.79s/it]
 49%|████▉     | 110883/225000 [233:22:49<155:21:55,  4.90s/it]
 49%|████▉     | 110884/225000 [233:22:58<199:12:51,  6.28s/it]
 49%|████▉     | 110885/225000 [233:23:03<182:18:02,  5.75s/it]
 49%|████▉     | 110886/225000 [233:23:09<185:10:52,  5.84s/it]
 49%|████▉     | 110887/225000 [233:23:12<160:40:01,  5.07s/it]
 49%|████▉     | 110888/225000 [233:23:15<136:26:06,  4.30s/it]
 49%|████▉     | 110889/225000 [233:23:18<127:14:08,  4.01s/it]
 49%|████▉     | 110890/225000 [233:23:32<222:46:44,  7.03s/it]
 49%|████▉     | 110891/225000 [233:23:36<187:28:14,  5.91s/it]
 49%|████▉     | 110892/225000 [233:23:38<153:38:53,  4.85s/it]
 49%|████▉     | 110893/225000 [233:23:40<129:47:25,  4.09s/it]
 49%|████▉     | 110894/225000 [233:23:43<113:23:45,  3.58s/it]
 49%|████▉     | 110895/225000 [233:23:45<101:31:41,  3.20s/it]
 49%|████▉     | 110896/225000 [233:23:54<155:59:12,  4.92s/it]
 49%|████▉     | 110897/225000 [233:23:56<132:12:42,  4.17s/it]
 49%|████▉     | 110898/225000 [233:23:59<119:13:25,  3.76s/it]
 49%|████▉     | 110899/225000 [233:24:11<196:09:43,  6.19s/it]
 49%|████▉     | 110900/225000 [233:24:13<157:57:34,  4.98s/it]
                                                               
{'loss': '0.819', 'grad_norm': '4.776', 'learning_rate': '2.743e-05', 'epoch': '0.4929', 'num_input_tokens_seen': 390629968, 'train_runtime': '8.403e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110900/225000 [233:24:13<157:57:34,  4.98s/it]
 49%|████▉     | 110901/225000 [233:24:15<131:40:54,  4.15s/it]
 49%|████▉     | 110902/225000 [233:24:26<190:37:03,  6.01s/it]
 49%|████▉     | 110903/225000 [233:24:28<154:38:48,  4.88s/it]
 49%|████▉     | 110904/225000 [233:24:30<129:01:15,  4.07s/it]
 49%|████▉     | 110905/225000 [233:24:35<134:05:35,  4.23s/it]
 49%|████▉     | 110906/225000 [233:24:37<118:34:01,  3.74s/it]
 49%|████▉     | 110907/225000 [233:24:40<107:04:30,  3.38s/it]
 49%|████▉     | 110908/225000 [233:24:42<98:14:45,  3.10s/it] 
 49%|████▉     | 110909/225000 [233:24:45<93:10:52,  2.94s/it]
 49%|████▉     | 110910/225000 [233:24:47<89:15:14,  2.82s/it]
 49%|████▉     | 110911/225000 [233:24:50<86:52:17,  2.74s/it]
 49%|████▉     | 110912/225000 [233:24:53<85:34:21,  2.70s/it]
 49%|████▉     | 110913/225000 [233:25:00<128:00:10,  4.04s/it]
 49%|████▉     | 110914/225000 [233:25:05<141:43:27,  4.47s/it]
 49%|████▉     | 110915/225000 [233:25:08<123:43:31,  3.90s/it]
 49%|████▉     | 110916/225000 [233:25:12<125:30:35,  3.96s/it]
 49%|████▉     | 110917/225000 [233:25:14<112:24:11,  3.55s/it]
 49%|████▉     | 110918/225000 [233:25:17<105:59:16,  3.34s/it]
 49%|████▉     | 110919/225000 [233:25:20<99:11:42,  3.13s/it] 
 49%|████▉     | 110920/225000 [233:25:23<93:48:28,  2.96s/it]
 49%|████▉     | 110921/225000 [233:25:32<158:48:36,  5.01s/it]
 49%|████▉     | 110922/225000 [233:25:35<132:34:46,  4.18s/it]
 49%|████▉     | 110923/225000 [233:25:43<175:12:20,  5.53s/it]
 49%|████▉     | 110924/225000 [233:25:46<150:24:48,  4.75s/it]
 49%|████▉     | 110925/225000 [233:25:54<184:21:36,  5.82s/it]
 49%|████▉     | 110926/225000 [233:25:57<149:26:05,  4.72s/it]
 49%|████▉     | 110927/225000 [233:26:01<148:14:46,  4.68s/it]
 49%|████▉     | 110928/225000 [233:26:03<124:08:07,  3.92s/it]
 49%|████▉     | 110929/225000 [233:26:10<150:17:47,  4.74s/it]
 49%|████▉     | 110930/225000 [233:26:12<127:47:15,  4.03s/it]
 49%|████▉     | 110931/225000 [233:26:15<111:56:51,  3.53s/it]
 49%|████▉     | 110932/225000 [233:26:17<101:18:23,  3.20s/it]
 49%|████▉     | 110933/225000 [233:26:20<93:34:41,  2.95s/it] 
 49%|████▉     | 110934/225000 [233:26:22<87:55:22,  2.77s/it]
 49%|████▉     | 110935/225000 [233:26:24<84:19:10,  2.66s/it]
 49%|████▉     | 110936/225000 [233:26:29<108:09:10,  3.41s/it]
 49%|████▉     | 110937/225000 [233:26:42<192:46:37,  6.08s/it]
 49%|████▉     | 110938/225000 [233:26:45<165:05:30,  5.21s/it]
 49%|████▉     | 110939/225000 [233:26:53<191:21:23,  6.04s/it]
 49%|████▉     | 110940/225000 [233:26:55<155:08:45,  4.90s/it]
 49%|████▉     | 110941/225000 [233:26:57<129:23:42,  4.08s/it]
 49%|████▉     | 110942/225000 [233:27:09<199:45:16,  6.30s/it]
 49%|████▉     | 110943/225000 [233:27:16<204:43:18,  6.46s/it]
 49%|████▉     | 110944/225000 [233:27:18<165:44:30,  5.23s/it]
 49%|████▉     | 110945/225000 [233:27:21<145:52:19,  4.60s/it]
 49%|████▉     | 110946/225000 [233:27:23<123:59:31,  3.91s/it]
 49%|████▉     | 110947/225000 [233:27:26<109:13:06,  3.45s/it]
 49%|████▉     | 110948/225000 [233:27:29<105:57:45,  3.34s/it]
 49%|████▉     | 110949/225000 [233:27:31<96:22:07,  3.04s/it] 
 49%|████▉     | 110950/225000 [233:27:34<97:12:56,  3.07s/it]
                                                              
{'loss': '0.8325', 'grad_norm': '7.189', 'learning_rate': '2.741e-05', 'epoch': '0.4931', 'num_input_tokens_seen': 390711480, 'train_runtime': '8.405e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 110950/225000 [233:27:34<97:12:56,  3.07s/it]
 49%|████▉     | 110951/225000 [233:27:46<174:01:17,  5.49s/it]
 49%|████▉     | 110952/225000 [233:27:49<157:04:34,  4.96s/it]
 49%|████▉     | 110953/225000 [233:27:52<138:18:13,  4.37s/it]
 49%|████▉     | 110954/225000 [233:27:55<118:36:18,  3.74s/it]
 49%|████▉     | 110955/225000 [233:27:57<105:09:06,  3.32s/it]
 49%|████▉     | 110956/225000 [233:27:59<94:57:18,  3.00s/it] 
 49%|████▉     | 110957/225000 [233:28:01<87:38:49,  2.77s/it]
 49%|████▉     | 110958/225000 [233:28:04<90:10:05,  2.85s/it]
 49%|████▉     | 110959/225000 [233:28:07<87:46:44,  2.77s/it]
 49%|████▉     | 110960/225000 [233:28:17<159:28:52,  5.03s/it]
 49%|████▉     | 110961/225000 [233:28:21<144:15:19,  4.55s/it]
 49%|████▉     | 110962/225000 [233:28:29<179:43:49,  5.67s/it]
 49%|████▉     | 110963/225000 [233:28:33<161:36:29,  5.10s/it]
 49%|████▉     | 110964/225000 [233:28:35<134:01:08,  4.23s/it]
 49%|████▉     | 110965/225000 [233:28:38<123:47:49,  3.91s/it]
 49%|████▉     | 110966/225000 [233:28:42<119:25:39,  3.77s/it]
 49%|████▉     | 110967/225000 [233:28:44<108:49:42,  3.44s/it]
 49%|████▉     | 110968/225000 [233:28:47<101:10:08,  3.19s/it]
 49%|████▉     | 110969/225000 [233:28:50<104:10:47,  3.29s/it]
 49%|████▉     | 110970/225000 [233:28:53<98:56:12,  3.12s/it] 
 49%|████▉     | 110971/225000 [233:28:56<95:32:09,  3.02s/it]
 49%|████▉     | 110972/225000 [233:28:59<93:00:18,  2.94s/it]
 49%|████▉     | 110973/225000 [233:29:01<91:32:48,  2.89s/it]
 49%|████▉     | 110974/225000 [233:29:04<90:16:33,  2.85s/it]
 49%|████▉     | 110975/225000 [233:29:07<89:05:51,  2.81s/it]
 49%|████▉     | 110976/225000 [233:29:17<160:19:23,  5.06s/it]
 49%|████▉     | 110977/225000 [233:29:27<203:19:30,  6.42s/it]
 49%|████▉     | 110978/225000 [233:29:31<185:15:40,  5.85s/it]
 49%|████▉     | 110979/225000 [233:29:35<159:54:58,  5.05s/it]
 49%|████▉     | 110980/225000 [233:29:37<134:39:46,  4.25s/it]
 49%|████▉     | 110981/225000 [233:29:40<124:14:20,  3.92s/it]
 49%|████▉     | 110982/225000 [233:29:43<113:26:50,  3.58s/it]
 49%|████▉     | 110983/225000 [233:29:45<102:47:49,  3.25s/it]
 49%|████▉     | 110984/225000 [233:29:49<104:18:13,  3.29s/it]
 49%|████▉     | 110985/225000 [233:30:07<244:30:49,  7.72s/it]
 49%|████▉     | 110986/225000 [233:30:09<194:09:18,  6.13s/it]
 49%|████▉     | 110987/225000 [233:30:12<161:37:27,  5.10s/it]
 49%|████▉     | 110988/225000 [233:30:15<139:20:59,  4.40s/it]
 49%|████▉     | 110989/225000 [233:30:24<188:14:05,  5.94s/it]
 49%|████▉     | 110990/225000 [233:30:27<158:02:36,  4.99s/it]
 49%|████▉     | 110991/225000 [233:30:30<136:42:30,  4.32s/it]
 49%|████▉     | 110992/225000 [233:30:32<122:11:12,  3.86s/it]
 49%|████▉     | 110993/225000 [233:30:35<111:57:03,  3.54s/it]
 49%|████▉     | 110994/225000 [233:30:39<116:16:21,  3.67s/it]
 49%|████▉     | 110995/225000 [233:30:42<107:54:35,  3.41s/it]
 49%|████▉     | 110996/225000 [233:30:46<112:23:42,  3.55s/it]
 49%|████▉     | 110997/225000 [233:30:49<105:06:39,  3.32s/it]
 49%|████▉     | 110998/225000 [233:30:52<100:27:39,  3.17s/it]
 49%|████▉     | 110999/225000 [233:30:54<96:42:05,  3.05s/it] 
 49%|████▉     | 111000/225000 [233:31:04<162:03:22,  5.12s/it]
                                                               
{'loss': '0.8711', 'grad_norm': '3.932', 'learning_rate': '2.739e-05', 'epoch': '0.4933', 'num_input_tokens_seen': 390797856, 'train_runtime': '8.407e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 111000/225000 [233:31:04<162:03:22,  5.12s/it]
 49%|████▉     | 111001/225000 [233:31:06<133:56:48,  4.23s/it]
 49%|████▉     | 111002/225000 [233:31:09<114:39:45,  3.62s/it]
 49%|████▉     | 111003/225000 [233:31:18<169:12:18,  5.34s/it]
 49%|████▉     | 111004/225000 [233:31:21<145:57:04,  4.61s/it]
 49%|████▉     | 111005/225000 [233:31:23<124:27:14,  3.93s/it]
 49%|████▉     | 111006/225000 [233:31:26<109:03:59,  3.44s/it]
 49%|████▉     | 111007/225000 [233:31:28<98:36:12,  3.11s/it] 
 49%|████▉     | 111008/225000 [233:31:33<118:47:11,  3.75s/it]
 49%|████▉     | 111009/225000 [233:31:44<190:13:02,  6.01s/it]
 49%|████▉     | 111010/225000 [233:31:47<159:09:42,  5.03s/it]
 49%|████▉     | 111011/225000 [233:31:50<140:23:08,  4.43s/it]
 49%|████▉     | 111012/225000 [233:31:52<119:51:32,  3.79s/it]
 49%|████▉     | 111013/225000 [233:31:55<105:45:29,  3.34s/it]
 49%|████▉     | 111014/225000 [233:31:57<95:24:39,  3.01s/it] 
 49%|████▉     | 111015/225000 [233:31:59<88:49:37,  2.81s/it]
 49%|████▉     | 111016/225000 [233:32:02<86:15:24,  2.72s/it]
 49%|████▉     | 111017/225000 [233:32:05<85:41:52,  2.71s/it]
 49%|████▉     | 111018/225000 [233:32:07<85:01:25,  2.69s/it]
 49%|████▉     | 111019/225000 [233:32:11<95:04:36,  3.00s/it]
 49%|████▉     | 111020/225000 [233:32:19<145:12:24,  4.59s/it]
 49%|████▉     | 111021/225000 [233:32:23<134:07:25,  4.24s/it]
 49%|████▉     | 111022/225000 [233:32:26<122:13:21,  3.86s/it]
 49%|████▉     | 111023/225000 [233:32:28<106:24:37,  3.36s/it]
 49%|████▉     | 111024/225000 [233:32:32<117:30:13,  3.71s/it]
 49%|████▉     | 111025/225000 [233:32:35<106:30:20,  3.36s/it]
 49%|████▉     | 111026/225000 [233:32:43<151:34:52,  4.79s/it]
 49%|████▉     | 111027/225000 [233:32:46<131:55:01,  4.17s/it]
 49%|████▉     | 111028/225000 [233:32:52<150:49:10,  4.76s/it]
 49%|████▉     | 111029/225000 [233:32:54<129:10:22,  4.08s/it]
 49%|████▉     | 111030/225000 [233:32:57<113:26:13,  3.58s/it]
 49%|████▉     | 111031/225000 [233:32:59<103:02:38,  3.25s/it]
 49%|████▉     | 111032/225000 [233:33:10<174:15:10,  5.50s/it]
 49%|████▉     | 111033/225000 [233:33:13<149:30:18,  4.72s/it]
 49%|████▉     | 111034/225000 [233:33:21<185:52:32,  5.87s/it]
 49%|████▉     | 111035/225000 [233:33:24<150:50:25,  4.76s/it]
 49%|████▉     | 111036/225000 [233:33:28<148:21:02,  4.69s/it]
 49%|████▉     | 111037/225000 [233:33:31<132:42:57,  4.19s/it]
 49%|████▉     | 111038/225000 [233:33:34<121:59:00,  3.85s/it]
 49%|████▉     | 111039/225000 [233:33:42<160:42:08,  5.08s/it]
 49%|████▉     | 111040/225000 [233:33:44<133:19:17,  4.21s/it]
 49%|████▉     | 111041/225000 [233:33:47<113:43:49,  3.59s/it]
 49%|████▉     | 111042/225000 [233:33:49<101:04:27,  3.19s/it]
 49%|████▉     | 111043/225000 [233:33:57<153:31:59,  4.85s/it]
 49%|████▉     | 111044/225000 [233:34:01<141:39:57,  4.48s/it]
 49%|████▉     | 111045/225000 [233:34:03<120:48:35,  3.82s/it]
 49%|████▉     | 111046/225000 [233:34:06<106:29:30,  3.36s/it]
 49%|████▉     | 111047/225000 [233:34:08<97:04:15,  3.07s/it] 
 49%|████▉     | 111048/225000 [233:34:15<129:21:32,  4.09s/it]
 49%|████▉     | 111049/225000 [233:34:19<129:39:46,  4.10s/it]
 49%|████▉     | 111050/225000 [233:34:22<122:29:44,  3.87s/it]
                                                               
{'loss': '0.8685', 'grad_norm': '3.41', 'learning_rate': '2.737e-05', 'epoch': '0.4936', 'num_input_tokens_seen': 390879920, 'train_runtime': '8.409e+05', 'train_tokens_per_second': '464.9'}

 49%|████▉     | 111050/225000 [233:34:22<122:29:44,  3.87s/it]
 49%|████▉     | 111051/225000 [233:34:31<172:55:35,  5.46s/it]
 49%|████▉     | 111052/225000 [233:34:33<141:56:21,  4.48s/it]
 49%|████▉     | 111053/225000 [233:34:36<127:22:23,  4.02s/it]
 49%|████▉     | 111054/225000 [233:34:45<169:56:23,  5.37s/it]
 49%|████▉     | 111055/225000 [233:34:48<148:25:34,  4.69s/it]
 49%|████▉     | 111056/225000 [233:34:52<141:30:38,  4.47s/it]
 49%|████▉     | 111057/225000 [233:34:54<121:07:20,  3.83s/it]
 49%|████▉     | 111058/225000 [233:34:57<106:44:41,  3.37s/it]
 49%|████▉     | 111059/225000 [233:35:02<123:03:18,  3.89s/it]
 49%|████▉     | 111060/225000 [233:35:04<112:12:22,  3.55s/it]
 49%|████▉     | 111061/225000 [233:35:07<103:58:56,  3.29s/it]
 49%|████▉     | 111062/225000 [233:35:10<97:36:55,  3.08s/it] 
 49%|████▉     | 111063/225000 [233:35:13<97:13:56,  3.07s/it]
 49%|████▉     | 111064/225000 [233:35:15<93:58:53,  2.97s/it]
 49%|████▉     | 111065/225000 [233:35:18<91:31:42,  2.89s/it]
 49%|████▉     | 111066/225000 [233:35:21<89:42:48,  2.83s/it]
 49%|████▉     | 111067/225000 [233:35:24<95:32:46,  3.02s/it]
 49%|████▉     | 111068/225000 [233:35:28<102:30:21,  3.24s/it]
 49%|████▉     | 111069/225000 [233:35:31<97:12:38,  3.07s/it] 
 49%|████▉     | 111070/225000 [233:35:38<138:31:15,  4.38s/it]
 49%|████▉     | 111071/225000 [233:35:45<161:40:13,  5.11s/it]
 49%|████▉     | 111072/225000 [233:35:47<134:28:50,  4.25s/it]
 49%|████▉     | 111073/225000 [233:35:49<115:09:52,  3.64s/it]
 49%|████▉     | 111074/225000 [233:35:52<102:09:53,  3.23s/it]
 49%|████▉     | 111075/225000 [233:35:54<93:12:09,  2.95s/it] 
 49%|████▉     | 111076/225000 [233:35:57<89:30:38,  2.83s/it]
 49%|████▉     | 111077/225000 [233:35:59<88:23:50,  2.79s/it]
 49%|████▉     | 111078/225000 [233:36:03<93:31:16,  2.96s/it]
 49%|████▉     | 111079/225000 [233:36:08<115:34:57,  3.65s/it]
 49%|████▉     | 111080/225000 [233:36:11<113:08:36,  3.58s/it]
 49%|████▉     | 111081/225000 [233:36:15<111:12:49,  3.51s/it]
 49%|████▉     | 111082/225000 [233:36:18<110:17:52,  3.49s/it]
 49%|████▉     | 111083/225000 [233:36:29<185:35:05,  5.86s/it]
 49%|████▉     | 111084/225000 [233:36:33<166:33:01,  5.26s/it]
 49%|████▉     | 111085/225000 [233:36:36<145:54:33,  4.61s/it]
 49%|████▉     | 111086/225000 [233:36:41<147:03:09,  4.65s/it]
 49%|████▉     | 111087/225000 [233:36:49<181:54:36,  5.75s/it]
 49%|████▉     | 111088/225000 [233:37:00<226:08:41,  7.15s/it]
 49%|████▉     | 111089/225000 [233:37:02<179:50:05,  5.68s/it]
 49%|████▉     | 111090/225000 [233:37:07<171:47:03,  5.43s/it]
 49%|████▉     | 111091/225000 [233:37:09<141:40:01,  4.48s/it]
 49%|████▉     | 111092/225000 [233:37:17<170:00:32,  5.37s/it]
 49%|████▉     | 111093/225000 [233:37:22<167:45:33,  5.30s/it]
 49%|████▉     | 111094/225000 [233:37:32<210:48:22,  6.66s/it]
 49%|████▉     | 111095/225000 [233:37:34<172:39:57,  5.46s/it]
 49%|████▉     | 111096/225000 [233:37:38<156:04:57,  4.93s/it]
 49%|████▉     | 111097/225000 [233:37:47<198:06:16,  6.26s/it]
 49%|████▉     | 111098/225000 [233:37:50<166:09:28,  5.25s/it]
 49%|████▉     | 111099/225000 [233:37:59<202:58:18,  6.42s/it]
 49%|████▉     | 111100/225000 [233:38:06<201:34:24,  6.37s/it]
                                                               
{'loss': '0.8441', 'grad_norm': '2.845', 'learning_rate': '2.735e-05', 'epoch': '0.4938', 'num_input_tokens_seen': 390971104, 'train_runtime': '8.411e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111100/225000 [233:38:06<201:34:24,  6.37s/it]
 49%|████▉     | 111101/225000 [233:38:08<162:06:15,  5.12s/it]
 49%|████▉     | 111102/225000 [233:38:10<137:04:16,  4.33s/it]
 49%|████▉     | 111103/225000 [233:38:13<123:00:25,  3.89s/it]
 49%|████▉     | 111104/225000 [233:38:16<112:06:34,  3.54s/it]
 49%|████▉     | 111105/225000 [233:38:19<111:35:12,  3.53s/it]
 49%|████▉     | 111106/225000 [233:38:22<104:34:39,  3.31s/it]
 49%|████▉     | 111107/225000 [233:38:26<105:35:22,  3.34s/it]
 49%|████▉     | 111108/225000 [233:38:29<103:40:24,  3.28s/it]
 49%|████▉     | 111109/225000 [233:38:32<102:16:44,  3.23s/it]
 49%|████▉     | 111110/225000 [233:38:35<97:36:42,  3.09s/it] 
 49%|████▉     | 111111/225000 [233:38:37<94:56:28,  3.00s/it]
 49%|████▉     | 111112/225000 [233:38:41<98:52:55,  3.13s/it]
 49%|████▉     | 111113/225000 [233:38:44<95:30:52,  3.02s/it]
 49%|████▉     | 111114/225000 [233:38:47<99:43:48,  3.15s/it]
 49%|████▉     | 111115/225000 [233:38:51<103:06:44,  3.26s/it]
 49%|████▉     | 111116/225000 [233:39:09<245:35:29,  7.76s/it]
 49%|████▉     | 111117/225000 [233:39:18<259:54:22,  8.22s/it]
 49%|████▉     | 111118/225000 [233:39:20<202:56:48,  6.42s/it]
 49%|████▉     | 111119/225000 [233:39:23<163:16:07,  5.16s/it]
 49%|████▉     | 111120/225000 [233:39:30<184:45:10,  5.84s/it]
 49%|████▉     | 111121/225000 [233:39:34<168:46:15,  5.34s/it]
 49%|████▉     | 111122/225000 [233:39:40<169:36:51,  5.36s/it]
 49%|████▉     | 111123/225000 [233:39:42<141:53:35,  4.49s/it]
 49%|████▉     | 111124/225000 [233:39:44<122:29:00,  3.87s/it]
 49%|████▉     | 111125/225000 [233:39:47<111:38:41,  3.53s/it]
 49%|████▉     | 111126/225000 [233:39:50<101:24:50,  3.21s/it]
 49%|████▉     | 111127/225000 [233:39:52<94:25:02,  2.98s/it] 
 49%|████▉     | 111128/225000 [233:39:58<125:15:10,  3.96s/it]
 49%|████▉     | 111129/225000 [233:40:01<111:26:23,  3.52s/it]
 49%|████▉     | 111130/225000 [233:40:03<100:44:50,  3.19s/it]
 49%|████▉     | 111131/225000 [233:40:07<107:58:25,  3.41s/it]
 49%|████▉     | 111132/225000 [233:40:10<105:49:47,  3.35s/it]
 49%|████▉     | 111133/225000 [233:40:26<220:11:46,  6.96s/it]
 49%|████▉     | 111134/225000 [233:40:29<183:11:52,  5.79s/it]
 49%|████▉     | 111135/225000 [233:40:36<196:41:46,  6.22s/it]
 49%|████▉     | 111136/225000 [233:40:38<158:02:46,  5.00s/it]
 49%|████▉     | 111137/225000 [233:40:40<131:08:41,  4.15s/it]
 49%|████▉     | 111138/225000 [233:40:43<112:05:39,  3.54s/it]
 49%|████▉     | 111139/225000 [233:40:50<152:49:10,  4.83s/it]
 49%|████▉     | 111140/225000 [233:40:58<182:55:16,  5.78s/it]
 49%|████▉     | 111141/225000 [233:41:06<205:06:54,  6.49s/it]
 49%|████▉     | 111142/225000 [233:41:09<164:25:23,  5.20s/it]
 49%|████▉     | 111143/225000 [233:41:11<135:19:53,  4.28s/it]
 49%|████▉     | 111144/225000 [233:41:14<123:17:08,  3.90s/it]
 49%|████▉     | 111145/225000 [233:41:16<106:46:57,  3.38s/it]
 49%|████▉     | 111146/225000 [233:41:19<102:41:00,  3.25s/it]
 49%|████▉     | 111147/225000 [233:41:21<93:51:55,  2.97s/it] 
 49%|████▉     | 111148/225000 [233:41:25<102:04:16,  3.23s/it]
 49%|████▉     | 111149/225000 [233:41:28<95:35:36,  3.02s/it] 
 49%|████▉     | 111150/225000 [233:41:30<90:37:37,  2.87s/it]
                                                              
{'loss': '0.8538', 'grad_norm': '4.717', 'learning_rate': '2.734e-05', 'epoch': '0.494', 'num_input_tokens_seen': 391060656, 'train_runtime': '8.413e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111150/225000 [233:41:30<90:37:37,  2.87s/it]
 49%|████▉     | 111151/225000 [233:41:33<87:44:09,  2.77s/it]
 49%|████▉     | 111152/225000 [233:41:35<85:08:02,  2.69s/it]
 49%|████▉     | 111153/225000 [233:41:38<83:14:33,  2.63s/it]
 49%|████▉     | 111154/225000 [233:41:43<113:00:11,  3.57s/it]
 49%|████▉     | 111155/225000 [233:41:54<177:58:08,  5.63s/it]
 49%|████▉     | 111156/225000 [233:41:56<148:43:30,  4.70s/it]
 49%|████▉     | 111157/225000 [233:41:59<129:11:28,  4.09s/it]
 49%|████▉     | 111158/225000 [233:42:02<115:26:48,  3.65s/it]
 49%|████▉     | 111159/225000 [233:42:05<112:22:34,  3.55s/it]
 49%|████▉     | 111160/225000 [233:42:15<176:41:52,  5.59s/it]
 49%|████▉     | 111161/225000 [233:42:18<145:21:43,  4.60s/it]
 49%|████▉     | 111162/225000 [233:42:20<123:18:43,  3.90s/it]
 49%|████▉     | 111163/225000 [233:42:22<107:57:41,  3.41s/it]
 49%|████▉     | 111164/225000 [233:42:27<123:47:58,  3.92s/it]
 49%|████▉     | 111165/225000 [233:42:30<108:36:12,  3.43s/it]
 49%|████▉     | 111166/225000 [233:42:33<105:45:08,  3.34s/it]
 49%|████▉     | 111167/225000 [233:42:40<145:48:41,  4.61s/it]
 49%|████▉     | 111168/225000 [233:42:43<123:48:44,  3.92s/it]
 49%|████▉     | 111169/225000 [233:42:49<151:24:38,  4.79s/it]
 49%|████▉     | 111170/225000 [233:42:54<148:40:57,  4.70s/it]
 49%|████▉     | 111171/225000 [233:42:57<135:58:39,  4.30s/it]
 49%|████▉     | 111172/225000 [233:43:09<205:42:56,  6.51s/it]
 49%|████▉     | 111173/225000 [233:43:12<168:48:24,  5.34s/it]
 49%|████▉     | 111174/225000 [233:43:17<165:44:07,  5.24s/it]
 49%|████▉     | 111175/225000 [233:43:19<143:18:14,  4.53s/it]
 49%|████▉     | 111176/225000 [233:43:30<197:07:59,  6.23s/it]
 49%|████▉     | 111177/225000 [233:43:33<165:57:50,  5.25s/it]
 49%|████▉     | 111178/225000 [233:43:37<162:01:10,  5.12s/it]
 49%|████▉     | 111179/225000 [233:43:46<198:56:34,  6.29s/it]
 49%|████▉     | 111180/225000 [233:43:49<167:59:37,  5.31s/it]
 49%|████▉     | 111181/225000 [233:43:57<192:03:08,  6.07s/it]
 49%|████▉     | 111182/225000 [233:44:00<163:52:05,  5.18s/it]
 49%|████▉     | 111183/225000 [233:44:04<146:54:05,  4.65s/it]
 49%|████▉     | 111184/225000 [233:44:09<155:56:03,  4.93s/it]
 49%|████▉     | 111185/225000 [233:44:12<134:09:50,  4.24s/it]
 49%|████▉     | 111186/225000 [233:44:15<119:16:04,  3.77s/it]
 49%|████▉     | 111187/225000 [233:44:26<192:51:54,  6.10s/it]
 49%|████▉     | 111188/225000 [233:44:29<162:51:50,  5.15s/it]
 49%|████▉     | 111189/225000 [233:44:36<180:30:24,  5.71s/it]
 49%|████▉     | 111190/225000 [233:44:38<146:57:23,  4.65s/it]
 49%|████▉     | 111191/225000 [233:44:41<124:12:08,  3.93s/it]
 49%|████▉     | 111192/225000 [233:44:43<108:19:34,  3.43s/it]
 49%|████▉     | 111193/225000 [233:44:46<106:30:38,  3.37s/it]
 49%|████▉     | 111194/225000 [233:44:48<96:16:32,  3.05s/it] 
 49%|████▉     | 111195/225000 [233:44:51<89:06:49,  2.82s/it]
 49%|████▉     | 111196/225000 [233:44:55<104:19:44,  3.30s/it]
 49%|████▉     | 111197/225000 [233:45:06<173:39:18,  5.49s/it]
 49%|████▉     | 111198/225000 [233:45:08<142:57:01,  4.52s/it]
 49%|████▉     | 111199/225000 [233:45:13<143:07:31,  4.53s/it]
 49%|████▉     | 111200/225000 [233:45:15<126:24:56,  4.00s/it]
                                                               
{'loss': '0.8524', 'grad_norm': '5.604', 'learning_rate': '2.732e-05', 'epoch': '0.4942', 'num_input_tokens_seen': 391145264, 'train_runtime': '8.415e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111200/225000 [233:45:15<126:24:56,  4.00s/it]
 49%|████▉     | 111201/225000 [233:45:18<111:31:32,  3.53s/it]
 49%|████▉     | 111202/225000 [233:45:20<100:53:54,  3.19s/it]
 49%|████▉     | 111203/225000 [233:45:23<93:39:46,  2.96s/it] 
 49%|████▉     | 111204/225000 [233:45:28<115:30:08,  3.65s/it]
 49%|████▉     | 111205/225000 [233:45:33<133:43:05,  4.23s/it]
 49%|████▉     | 111206/225000 [233:45:41<162:29:35,  5.14s/it]
 49%|████▉     | 111207/225000 [233:45:45<155:34:37,  4.92s/it]
 49%|████▉     | 111208/225000 [233:45:47<129:34:41,  4.10s/it]
 49%|████▉     | 111209/225000 [233:45:50<113:33:33,  3.59s/it]
 49%|████▉     | 111210/225000 [233:45:54<121:38:24,  3.85s/it]
 49%|████▉     | 111211/225000 [233:45:59<135:45:01,  4.29s/it]
 49%|████▉     | 111212/225000 [233:46:02<116:40:48,  3.69s/it]
 49%|████▉     | 111213/225000 [233:46:04<103:22:42,  3.27s/it]
 49%|████▉     | 111214/225000 [233:46:06<94:17:42,  2.98s/it] 
 49%|████▉     | 111215/225000 [233:46:09<87:54:39,  2.78s/it]
 49%|████▉     | 111216/225000 [233:46:11<83:40:46,  2.65s/it]
 49%|████▉     | 111217/225000 [233:46:17<119:08:33,  3.77s/it]
 49%|████▉     | 111218/225000 [233:46:28<185:42:06,  5.88s/it]
 49%|████▉     | 111219/225000 [233:46:38<221:02:38,  6.99s/it]
 49%|████▉     | 111220/225000 [233:46:47<243:34:25,  7.71s/it]
 49%|████▉     | 111221/225000 [233:46:49<190:59:49,  6.04s/it]
 49%|████▉     | 111222/225000 [233:46:54<177:13:57,  5.61s/it]
 49%|████▉     | 111223/225000 [233:47:01<187:29:19,  5.93s/it]
 49%|████▉     | 111224/225000 [233:47:05<175:30:11,  5.55s/it]
 49%|████▉     | 111225/225000 [233:47:07<144:02:32,  4.56s/it]
 49%|████▉     | 111226/225000 [233:47:10<121:34:21,  3.85s/it]
 49%|████▉     | 111227/225000 [233:47:12<106:17:13,  3.36s/it]
 49%|████▉     | 111228/225000 [233:47:14<96:17:49,  3.05s/it] 
 49%|████▉     | 111229/225000 [233:47:17<93:45:02,  2.97s/it]
 49%|████▉     | 111230/225000 [233:47:20<91:41:32,  2.90s/it]
 49%|████▉     | 111231/225000 [233:47:27<130:57:29,  4.14s/it]
 49%|████▉     | 111232/225000 [233:47:31<129:58:34,  4.11s/it]
 49%|████▉     | 111233/225000 [233:47:41<191:14:48,  6.05s/it]
 49%|████▉     | 111234/225000 [233:47:46<180:58:12,  5.73s/it]
 49%|████▉     | 111235/225000 [233:47:49<147:36:59,  4.67s/it]
 49%|████▉     | 111236/225000 [233:47:51<124:32:44,  3.94s/it]
 49%|████▉     | 111237/225000 [233:47:53<109:56:16,  3.48s/it]
 49%|████▉     | 111238/225000 [233:48:03<170:35:07,  5.40s/it]
 49%|████▉     | 111239/225000 [233:48:09<171:21:23,  5.42s/it]
 49%|████▉     | 111240/225000 [233:48:11<142:11:48,  4.50s/it]
 49%|████▉     | 111241/225000 [233:48:13<121:47:30,  3.85s/it]
 49%|████▉     | 111242/225000 [233:48:20<148:29:45,  4.70s/it]
 49%|████▉     | 111243/225000 [233:48:22<124:32:15,  3.94s/it]
 49%|████▉     | 111244/225000 [233:48:34<199:43:03,  6.32s/it]
 49%|████▉     | 111245/225000 [233:48:36<160:25:34,  5.08s/it]
 49%|████▉     | 111246/225000 [233:48:41<161:32:10,  5.11s/it]
 49%|████▉     | 111247/225000 [233:48:45<143:33:19,  4.54s/it]
 49%|████▉     | 111248/225000 [233:48:48<130:54:38,  4.14s/it]
 49%|████▉     | 111249/225000 [233:48:51<122:22:03,  3.87s/it]
 49%|████▉     | 111250/225000 [233:48:59<163:28:54,  5.17s/it]
                                                               
{'loss': '0.8249', 'grad_norm': '2.934', 'learning_rate': '2.73e-05', 'epoch': '0.4944', 'num_input_tokens_seen': 391232080, 'train_runtime': '8.417e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111250/225000 [233:48:59<163:28:54,  5.17s/it]
 49%|████▉     | 111251/225000 [233:49:06<183:10:23,  5.80s/it]
 49%|████▉     | 111252/225000 [233:49:09<149:05:39,  4.72s/it]
 49%|████▉     | 111253/225000 [233:49:11<124:50:45,  3.95s/it]
 49%|████▉     | 111254/225000 [233:49:17<148:05:10,  4.69s/it]
 49%|████▉     | 111255/225000 [233:49:20<127:14:10,  4.03s/it]
 49%|████▉     | 111256/225000 [233:49:22<112:23:00,  3.56s/it]
 49%|████▉     | 111257/225000 [233:49:25<101:07:46,  3.20s/it]
 49%|████▉     | 111258/225000 [233:49:28<101:09:31,  3.20s/it]
 49%|████▉     | 111259/225000 [233:49:30<94:17:20,  2.98s/it] 
 49%|████▉     | 111260/225000 [233:49:33<89:14:17,  2.82s/it]
 49%|████▉     | 111261/225000 [233:49:35<86:14:12,  2.73s/it]
 49%|████▉     | 111262/225000 [233:49:38<83:47:29,  2.65s/it]
 49%|████▉     | 111263/225000 [233:49:40<82:24:15,  2.61s/it]
 49%|████▉     | 111264/225000 [233:49:43<83:28:01,  2.64s/it]
 49%|████▉     | 111265/225000 [233:49:53<155:39:06,  4.93s/it]
 49%|████▉     | 111266/225000 [233:49:56<135:06:15,  4.28s/it]
 49%|████▉     | 111267/225000 [233:49:59<123:25:04,  3.91s/it]
 49%|████▉     | 111268/225000 [233:50:07<165:21:39,  5.23s/it]
 49%|████▉     | 111269/225000 [233:50:10<137:57:10,  4.37s/it]
 49%|████▉     | 111270/225000 [233:50:15<143:00:10,  4.53s/it]
 49%|████▉     | 111271/225000 [233:50:17<122:27:13,  3.88s/it]
 49%|████▉     | 111272/225000 [233:50:19<107:56:31,  3.42s/it]
 49%|████▉     | 111273/225000 [233:50:26<136:10:56,  4.31s/it]
 49%|████▉     | 111274/225000 [233:50:32<159:37:16,  5.05s/it]
 49%|████▉     | 111275/225000 [233:50:35<131:49:02,  4.17s/it]
 49%|████▉     | 111276/225000 [233:50:37<112:23:25,  3.56s/it]
 49%|████▉     | 111277/225000 [233:50:39<99:59:45,  3.17s/it] 
 49%|████▉     | 111278/225000 [233:50:41<91:47:35,  2.91s/it]
 49%|████▉     | 111279/225000 [233:50:50<147:00:00,  4.65s/it]
 49%|████▉     | 111280/225000 [233:50:58<181:56:17,  5.76s/it]
 49%|████▉     | 111281/225000 [233:51:03<173:39:13,  5.50s/it]
 49%|████▉     | 111282/225000 [233:51:05<142:49:36,  4.52s/it]
 49%|████▉     | 111283/225000 [233:51:08<120:54:33,  3.83s/it]
 49%|████▉     | 111284/225000 [233:51:10<108:36:45,  3.44s/it]
 49%|████▉     | 111285/225000 [233:51:14<113:26:08,  3.59s/it]
 49%|████▉     | 111286/225000 [233:51:24<175:15:46,  5.55s/it]
 49%|████▉     | 111287/225000 [233:51:27<152:01:27,  4.81s/it]
 49%|████▉     | 111288/225000 [233:51:30<127:36:02,  4.04s/it]
 49%|████▉     | 111289/225000 [233:51:33<118:07:06,  3.74s/it]
 49%|████▉     | 111290/225000 [233:51:35<104:25:00,  3.31s/it]
 49%|████▉     | 111291/225000 [233:51:38<106:28:53,  3.37s/it]
 49%|████▉     | 111292/225000 [233:51:41<96:28:05,  3.05s/it] 
 49%|████▉     | 111293/225000 [233:51:45<105:59:58,  3.36s/it]
 49%|████▉     | 111294/225000 [233:51:48<106:25:56,  3.37s/it]
 49%|████▉     | 111295/225000 [233:51:52<106:53:24,  3.38s/it]
 49%|████▉     | 111296/225000 [233:51:55<107:31:55,  3.40s/it]
 49%|████▉     | 111297/225000 [233:51:59<108:22:34,  3.43s/it]
 49%|████▉     | 111298/225000 [233:52:09<170:14:46,  5.39s/it]
 49%|████▉     | 111299/225000 [233:52:11<139:36:46,  4.42s/it]
 49%|████▉     | 111300/225000 [233:52:15<139:49:57,  4.43s/it]
                                                               
{'loss': '0.8789', 'grad_norm': '3.147', 'learning_rate': '2.728e-05', 'epoch': '0.4947', 'num_input_tokens_seen': 391314312, 'train_runtime': '8.419e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111300/225000 [233:52:15<139:49:57,  4.43s/it]
 49%|████▉     | 111301/225000 [233:52:18<128:50:45,  4.08s/it]
 49%|████▉     | 111302/225000 [233:52:21<114:32:09,  3.63s/it]
 49%|████▉     | 111303/225000 [233:52:24<105:08:59,  3.33s/it]
 49%|████▉     | 111304/225000 [233:52:26<98:23:19,  3.12s/it] 
 49%|████▉     | 111305/225000 [233:52:29<93:32:49,  2.96s/it]
 49%|████▉     | 111306/225000 [233:52:31<90:33:09,  2.87s/it]
 49%|████▉     | 111307/225000 [233:52:35<95:03:28,  3.01s/it]
 49%|████▉     | 111308/225000 [233:52:39<108:54:58,  3.45s/it]
 49%|████▉     | 111309/225000 [233:52:48<154:32:55,  4.89s/it]
 49%|████▉     | 111310/225000 [233:52:51<138:12:48,  4.38s/it]
 49%|████▉     | 111311/225000 [233:52:53<119:23:15,  3.78s/it]
 49%|████▉     | 111312/225000 [233:52:56<106:26:25,  3.37s/it]
 49%|████▉     | 111313/225000 [233:53:01<127:45:55,  4.05s/it]
 49%|████▉     | 111314/225000 [233:53:04<112:09:43,  3.55s/it]
 49%|████▉     | 111315/225000 [233:53:06<101:21:32,  3.21s/it]
 49%|████▉     | 111316/225000 [233:53:09<100:53:16,  3.19s/it]
 49%|████▉     | 111317/225000 [233:53:17<142:18:35,  4.51s/it]
 49%|████▉     | 111318/225000 [233:53:20<135:12:06,  4.28s/it]
 49%|████▉     | 111319/225000 [233:53:23<123:20:50,  3.91s/it]
 49%|████▉     | 111320/225000 [233:53:27<120:30:27,  3.82s/it]
 49%|████▉     | 111321/225000 [233:53:30<112:56:05,  3.58s/it]
 49%|████▉     | 111322/225000 [233:53:33<107:42:54,  3.41s/it]
 49%|████▉     | 111323/225000 [233:53:36<104:37:32,  3.31s/it]
 49%|████▉     | 111324/225000 [233:53:40<108:05:58,  3.42s/it]
 49%|████▉     | 111325/225000 [233:53:43<104:36:45,  3.31s/it]
 49%|████▉     | 111326/225000 [233:53:46<101:41:14,  3.22s/it]
 49%|████▉     | 111327/225000 [233:53:52<125:24:09,  3.97s/it]
 49%|████▉     | 111328/225000 [233:53:55<121:24:05,  3.84s/it]
 49%|████▉     | 111329/225000 [233:53:58<114:15:21,  3.62s/it]
 49%|████▉     | 111330/225000 [233:54:07<164:44:27,  5.22s/it]
 49%|████▉     | 111331/225000 [233:54:12<164:08:17,  5.20s/it]
 49%|████▉     | 111332/225000 [233:54:17<162:30:56,  5.15s/it]
 49%|████▉     | 111333/225000 [233:54:27<206:14:46,  6.53s/it]
 49%|████▉     | 111334/225000 [233:54:30<166:34:49,  5.28s/it]
 49%|████▉     | 111335/225000 [233:54:32<140:44:43,  4.46s/it]
 49%|████▉     | 111336/225000 [233:54:34<120:23:44,  3.81s/it]
 49%|████▉     | 111337/225000 [233:54:40<134:12:45,  4.25s/it]
 49%|████▉     | 111338/225000 [233:54:44<135:15:46,  4.28s/it]
 49%|████▉     | 111339/225000 [233:54:56<206:29:50,  6.54s/it]
 49%|████▉     | 111340/225000 [233:54:58<164:54:05,  5.22s/it]
 49%|████▉     | 111341/225000 [233:55:00<137:04:56,  4.34s/it]
 49%|████▉     | 111342/225000 [233:55:05<142:11:47,  4.50s/it]
 49%|████▉     | 111343/225000 [233:55:08<128:19:38,  4.06s/it]
 49%|████▉     | 111344/225000 [233:55:10<110:51:53,  3.51s/it]
 49%|████▉     | 111345/225000 [233:55:20<169:20:44,  5.36s/it]
 49%|████▉     | 111346/225000 [233:55:23<149:22:14,  4.73s/it]
 49%|████▉     | 111347/225000 [233:55:31<176:35:52,  5.59s/it]
 49%|████▉     | 111348/225000 [233:55:39<203:20:25,  6.44s/it]
 49%|████▉     | 111349/225000 [233:55:42<162:55:05,  5.16s/it]
 49%|████▉     | 111350/225000 [233:55:44<134:29:54,  4.26s/it]
                                                               
{'loss': '0.8381', 'grad_norm': '4.996', 'learning_rate': '2.726e-05', 'epoch': '0.4949', 'num_input_tokens_seen': 391396432, 'train_runtime': '8.421e+05', 'train_tokens_per_second': '464.8'}

 49%|████▉     | 111350/225000 [233:55:44<134:29:54,  4.26s/it]
 49%|████▉     | 111351/225000 [233:55:47<122:27:03,  3.88s/it]
 49%|████▉     | 111352/225000 [233:55:49<106:16:40,  3.37s/it]
 49%|████▉     | 111353/225000 [233:55:51<95:20:17,  3.02s/it] 
 49%|████▉     | 111354/225000 [233:55:54<90:18:14,  2.86s/it]
 49%|████▉     | 111355/225000 [233:55:57<91:42:05,  2.90s/it]
 49%|████▉     | 111356/225000 [233:56:05<145:42:00,  4.62s/it]
 49%|████▉     | 111357/225000 [233:56:07<122:38:46,  3.89s/it]
 49%|████▉     | 111358/225000 [233:56:16<165:20:05,  5.24s/it]
 49%|████▉     | 111359/225000 [233:56:18<137:36:13,  4.36s/it]
 49%|████▉     | 111360/225000 [233:56:20<118:52:12,  3.77s/it]
 49%|████▉     | 111361/225000 [233:56:23<105:21:33,  3.34s/it]
 49%|████▉     | 111362/225000 [233:56:25<96:00:14,  3.04s/it] 
 49%|████▉     | 111363/225000 [233:56:33<137:54:06,  4.37s/it]
 49%|████▉     | 111364/225000 [233:56:37<138:27:48,  4.39s/it]
 49%|████▉     | 111365/225000 [233:56:40<120:34:31,  3.82s/it]
 49%|████▉     | 111366/225000 [233:56:50<186:39:45,  5.91s/it]
 49%|████▉     | 111367/225000 [233:56:53<151:31:42,  4.80s/it]
 49%|████▉     | 111368/225000 [233:56:55<127:18:40,  4.03s/it]
 49%|████▉     | 111369/225000 [233:56:57<113:07:14,  3.58s/it]
 49%|████▉     | 111370/225000 [233:57:01<109:24:18,  3.47s/it]
 49%|████▉     | 111371/225000 [233:57:04<107:23:40,  3.40s/it]
 49%|████▉     | 111372/225000 [233:57:07<105:31:20,  3.34s/it]
 49%|████▉     | 111373/225000 [233:57:16<161:46:48,  5.13s/it]
 49%|████▉     | 111374/225000 [233:57:18<133:27:44,  4.23s/it]
 50%|████▉     | 111375/225000 [233:57:21<116:30:49,  3.69s/it]
 50%|████▉     | 111376/225000 [233:57:26<127:11:05,  4.03s/it]
 50%|████▉     | 111377/225000 [233:57:29<121:45:05,  3.86s/it]
 50%|████▉     | 111378/225000 [233:57:33<117:42:58,  3.73s/it]
 50%|████▉     | 111379/225000 [233:57:35<108:49:50,  3.45s/it]
 50%|████▉     | 111380/225000 [233:57:46<178:09:31,  5.64s/it]
 50%|████▉     | 111381/225000 [233:57:48<146:57:48,  4.66s/it]
 50%|████▉     | 111382/225000 [233:57:54<152:39:10,  4.84s/it]
 50%|████▉     | 111383/225000 [233:57:56<129:29:39,  4.10s/it]
 50%|████▉     | 111384/225000 [233:57:58<113:01:44,  3.58s/it]
 50%|████▉     | 111385/225000 [233:58:08<170:27:25,  5.40s/it]
 50%|████▉     | 111386/225000 [233:58:14<179:18:34,  5.68s/it]
 50%|████▉     | 111387/225000 [233:58:18<161:19:04,  5.11s/it]
 50%|████▉     | 111388/225000 [233:58:26<186:47:06,  5.92s/it]
 50%|████▉     | 111389/225000 [233:58:29<158:52:11,  5.03s/it]
 50%|████▉     | 111390/225000 [233:58:33<150:53:55,  4.78s/it]
 50%|████▉     | 111391/225000 [233:58:36<131:58:09,  4.18s/it]
 50%|████▉     | 111392/225000 [233:58:40<126:11:03,  4.00s/it]
 50%|████▉     | 111393/225000 [233:58:44<127:25:24,  4.04s/it]
 50%|████▉     | 111394/225000 [233:58:49<138:10:03,  4.38s/it]
 50%|████▉     | 111395/225000 [233:58:52<123:21:52,  3.91s/it]
 50%|████▉     | 111396/225000 [233:58:55<119:21:27,  3.78s/it]
 50%|████▉     | 111397/225000 [233:59:02<143:58:49,  4.56s/it]
 50%|████▉     | 111398/225000 [233:59:05<130:14:37,  4.13s/it]
 50%|████▉     | 111399/225000 [233:59:07<114:09:12,  3.62s/it]
 50%|████▉     | 111400/225000 [233:59:10<109:41:33,  3.48s/it]
                                                               
{'loss': '0.8538', 'grad_norm': '3.367', 'learning_rate': '2.724e-05', 'epoch': '0.4951', 'num_input_tokens_seen': 391485632, 'train_runtime': '8.424e+05', 'train_tokens_per_second': '464.8'}

 50%|████▉     | 111400/225000 [233:59:10<109:41:33,  3.48s/it]
 50%|████▉     | 111401/225000 [233:59:13<99:08:53,  3.14s/it] 
 50%|████▉     | 111402/225000 [233:59:15<91:59:05,  2.92s/it]
 50%|████▉     | 111403/225000 [233:59:17<87:09:59,  2.76s/it]
 50%|████▉     | 111404/225000 [233:59:20<83:56:04,  2.66s/it]
 50%|████▉     | 111405/225000 [233:59:22<81:21:08,  2.58s/it]
 50%|████▉     | 111406/225000 [233:59:32<151:17:49,  4.79s/it]
 50%|████▉     | 111407/225000 [233:59:36<139:13:19,  4.41s/it]
 50%|████▉     | 111408/225000 [233:59:39<124:48:01,  3.96s/it]
 50%|████▉     | 111409/225000 [233:59:41<113:20:12,  3.59s/it]
 50%|████▉     | 111410/225000 [233:59:51<175:44:38,  5.57s/it]
 50%|████▉     | 111411/225000 [233:59:56<167:32:29,  5.31s/it]
 50%|████▉     | 111412/225000 [233:59:59<146:28:33,  4.64s/it]
 50%|████▉     | 111413/225000 [234:00:03<141:02:56,  4.47s/it]
 50%|████▉     | 111414/225000 [234:00:07<136:12:03,  4.32s/it]
 50%|████▉     | 111415/225000 [234:00:13<153:48:02,  4.87s/it]
 50%|████▉     | 111416/225000 [234:00:16<130:23:44,  4.13s/it]
 50%|████▉     | 111417/225000 [234:00:18<113:58:52,  3.61s/it]
 50%|████▉     | 111418/225000 [234:00:21<102:26:44,  3.25s/it]
 50%|████▉     | 111419/225000 [234:00:30<162:10:33,  5.14s/it]
 50%|████▉     | 111420/225000 [234:00:33<136:36:03,  4.33s/it]
 50%|████▉     | 111421/225000 [234:00:37<132:33:31,  4.20s/it]
 50%|████▉     | 111422/225000 [234:00:39<115:23:34,  3.66s/it]
 50%|████▉     | 111423/225000 [234:00:41<103:08:43,  3.27s/it]
 50%|████▉     | 111424/225000 [234:00:44<94:37:42,  3.00s/it] 
 50%|████▉     | 111425/225000 [234:00:54<160:32:09,  5.09s/it]
 50%|████▉     | 111426/225000 [234:00:57<139:44:15,  4.43s/it]
 50%|████▉     | 111427/225000 [234:00:59<119:36:40,  3.79s/it]
 50%|████▉     | 111428/225000 [234:01:10<188:26:49,  5.97s/it]
 50%|████▉     | 111429/225000 [234:01:13<160:39:11,  5.09s/it]
 50%|████▉     | 111430/225000 [234:01:15<133:31:40,  4.23s/it]
 50%|████▉     | 111431/225000 [234:01:20<138:36:28,  4.39s/it]
 50%|████▉     | 111432/225000 [234:01:24<134:02:50,  4.25s/it]
 50%|████▉     | 111433/225000 [234:01:26<118:42:49,  3.76s/it]
 50%|████▉     | 111434/225000 [234:01:29<105:50:48,  3.36s/it]
 50%|████▉     | 111435/225000 [234:01:31<96:58:52,  3.07s/it] 
 50%|████▉     | 111436/225000 [234:01:34<91:02:19,  2.89s/it]
 50%|████▉     | 111437/225000 [234:01:36<86:29:15,  2.74s/it]
 50%|████▉     | 111438/225000 [234:01:39<85:32:05,  2.71s/it]
 50%|████▉     | 111439/225000 [234:01:44<110:52:27,  3.51s/it]
 50%|████▉     | 111440/225000 [234:01:52<150:45:51,  4.78s/it]
 50%|████▉     | 111441/225000 [234:01:58<165:32:16,  5.25s/it]
 50%|████▉     | 111442/225000 [234:02:01<137:46:02,  4.37s/it]
 50%|████▉     | 111443/225000 [234:02:05<142:33:49,  4.52s/it]
 50%|████▉     | 111444/225000 [234:02:08<121:37:26,  3.86s/it]
 50%|████▉     | 111445/225000 [234:02:15<155:15:35,  4.92s/it]
 50%|████▉     | 111446/225000 [234:02:18<139:56:04,  4.44s/it]
 50%|████▉     | 111447/225000 [234:02:21<121:58:00,  3.87s/it]
 50%|████▉     | 111448/225000 [234:02:25<119:39:22,  3.79s/it]
 50%|████▉     | 111449/225000 [234:02:27<107:20:32,  3.40s/it]
 50%|████▉     | 111450/225000 [234:02:36<161:20:25,  5.12s/it]
                                                               
{'loss': '0.8411', 'grad_norm': '2.982', 'learning_rate': '2.723e-05', 'epoch': '0.4953', 'num_input_tokens_seen': 391566448, 'train_runtime': '8.426e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111450/225000 [234:02:36<161:20:25,  5.12s/it]
 50%|████▉     | 111451/225000 [234:02:39<138:25:45,  4.39s/it]
 50%|████▉     | 111452/225000 [234:02:41<117:27:13,  3.72s/it]
 50%|████▉     | 111453/225000 [234:02:43<103:18:06,  3.28s/it]
 50%|████▉     | 111454/225000 [234:02:46<93:04:26,  2.95s/it] 
 50%|████▉     | 111455/225000 [234:02:48<86:44:26,  2.75s/it]
 50%|████▉     | 111456/225000 [234:02:50<82:23:47,  2.61s/it]
 50%|████▉     | 111457/225000 [234:02:53<83:05:02,  2.63s/it]
 50%|████▉     | 111458/225000 [234:02:56<84:08:57,  2.67s/it]
 50%|████▉     | 111459/225000 [234:02:58<85:11:47,  2.70s/it]
 50%|████▉     | 111460/225000 [234:03:01<86:32:27,  2.74s/it]
 50%|████▉     | 111461/225000 [234:03:04<90:23:48,  2.87s/it]
 50%|████▉     | 111462/225000 [234:03:07<90:08:43,  2.86s/it]
 50%|████▉     | 111463/225000 [234:03:10<89:45:47,  2.85s/it]
 50%|████▉     | 111464/225000 [234:03:20<160:17:04,  5.08s/it]
 50%|████▉     | 111465/225000 [234:03:23<139:33:44,  4.43s/it]
 50%|████▉     | 111466/225000 [234:03:33<195:03:09,  6.18s/it]
 50%|████▉     | 111467/225000 [234:03:39<187:01:56,  5.93s/it]
 50%|████▉     | 111468/225000 [234:03:44<180:41:57,  5.73s/it]
 50%|████▉     | 111469/225000 [234:03:46<149:17:32,  4.73s/it]
 50%|████▉     | 111470/225000 [234:03:49<127:25:58,  4.04s/it]
 50%|████▉     | 111471/225000 [234:03:51<111:55:36,  3.55s/it]
 50%|████▉     | 111472/225000 [234:03:54<105:33:20,  3.35s/it]
 50%|████▉     | 111473/225000 [234:04:04<169:18:44,  5.37s/it]
 50%|████▉     | 111474/225000 [234:04:10<170:52:58,  5.42s/it]
 50%|████▉     | 111475/225000 [234:04:12<140:12:29,  4.45s/it]
 50%|████▉     | 111476/225000 [234:04:15<128:07:44,  4.06s/it]
 50%|████▉     | 111477/225000 [234:04:18<119:19:23,  3.78s/it]
 50%|████▉     | 111478/225000 [234:04:22<118:04:34,  3.74s/it]
 50%|████▉     | 111479/225000 [234:04:25<113:19:55,  3.59s/it]
 50%|████▉     | 111480/225000 [234:04:28<109:11:58,  3.46s/it]
 50%|████▉     | 111481/225000 [234:04:32<106:53:19,  3.39s/it]
 50%|████▉     | 111482/225000 [234:04:35<104:48:14,  3.32s/it]
 50%|████▉     | 111483/225000 [234:04:45<166:56:12,  5.29s/it]
 50%|████▉     | 111484/225000 [234:04:47<136:56:58,  4.34s/it]
 50%|████▉     | 111485/225000 [234:04:49<116:56:09,  3.71s/it]
 50%|████▉     | 111486/225000 [234:04:52<110:13:44,  3.50s/it]
 50%|████▉     | 111487/225000 [234:04:54<98:33:25,  3.13s/it] 
 50%|████▉     | 111488/225000 [234:04:56<90:46:00,  2.88s/it]
 50%|████▉     | 111489/225000 [234:05:07<165:32:54,  5.25s/it]
 50%|████▉     | 111490/225000 [234:05:10<138:52:02,  4.40s/it]
 50%|████▉     | 111491/225000 [234:05:12<117:41:22,  3.73s/it]
 50%|████▉     | 111492/225000 [234:05:15<112:19:13,  3.56s/it]
 50%|████▉     | 111493/225000 [234:05:18<104:24:44,  3.31s/it]
 50%|████▉     | 111494/225000 [234:05:21<99:16:12,  3.15s/it] 
 50%|████▉     | 111495/225000 [234:05:23<96:16:40,  3.05s/it]
 50%|████▉     | 111496/225000 [234:05:26<90:33:24,  2.87s/it]
 50%|████▉     | 111497/225000 [234:05:28<87:11:29,  2.77s/it]
 50%|████▉     | 111498/225000 [234:05:42<193:30:28,  6.14s/it]
 50%|████▉     | 111499/225000 [234:05:48<184:27:16,  5.85s/it]
 50%|████▉     | 111500/225000 [234:05:50<151:52:24,  4.82s/it]
                                                               
{'loss': '0.8577', 'grad_norm': '6.906', 'learning_rate': '2.721e-05', 'epoch': '0.4956', 'num_input_tokens_seen': 391646584, 'train_runtime': '8.428e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111500/225000 [234:05:50<151:52:24,  4.82s/it]
 50%|████▉     | 111501/225000 [234:05:52<126:36:09,  4.02s/it]
 50%|████▉     | 111502/225000 [234:05:58<146:27:13,  4.65s/it]
 50%|████▉     | 111503/225000 [234:06:02<141:35:55,  4.49s/it]
 50%|████▉     | 111504/225000 [234:06:08<151:05:48,  4.79s/it]
 50%|████▉     | 111505/225000 [234:06:10<129:33:11,  4.11s/it]
 50%|████▉     | 111506/225000 [234:06:16<143:34:11,  4.55s/it]
 50%|████▉     | 111507/225000 [234:06:26<193:52:18,  6.15s/it]
 50%|████▉     | 111508/225000 [234:06:28<155:59:39,  4.95s/it]
 50%|████▉     | 111509/225000 [234:06:30<132:02:56,  4.19s/it]
 50%|████▉     | 111510/225000 [234:06:36<142:25:20,  4.52s/it]
 50%|████▉     | 111511/225000 [234:06:39<129:19:11,  4.10s/it]
 50%|████▉     | 111512/225000 [234:06:44<138:48:31,  4.40s/it]
 50%|████▉     | 111513/225000 [234:06:54<188:39:53,  5.98s/it]
 50%|████▉     | 111514/225000 [234:07:02<208:23:05,  6.61s/it]
 50%|████▉     | 111515/225000 [234:07:04<165:34:22,  5.25s/it]
 50%|████▉     | 111516/225000 [234:07:06<136:29:31,  4.33s/it]
 50%|████▉     | 111517/225000 [234:07:15<184:16:13,  5.85s/it]
 50%|████▉     | 111518/225000 [234:07:18<156:57:09,  4.98s/it]
 50%|████▉     | 111519/225000 [234:07:20<130:55:35,  4.15s/it]
 50%|████▉     | 111520/225000 [234:07:23<119:54:47,  3.80s/it]
 50%|████▉     | 111521/225000 [234:07:28<126:32:38,  4.01s/it]
 50%|████▉     | 111522/225000 [234:07:31<112:56:27,  3.58s/it]
 50%|████▉     | 111523/225000 [234:07:33<100:50:02,  3.20s/it]
 50%|████▉     | 111524/225000 [234:07:43<168:13:09,  5.34s/it]
 50%|████▉     | 111525/225000 [234:07:46<147:41:37,  4.69s/it]
 50%|████▉     | 111526/225000 [234:07:48<123:45:13,  3.93s/it]
 50%|████▉     | 111527/225000 [234:07:59<182:32:41,  5.79s/it]
 50%|████▉     | 111528/225000 [234:08:01<148:43:04,  4.72s/it]
 50%|████▉     | 111529/225000 [234:08:03<124:32:16,  3.95s/it]
 50%|████▉     | 111530/225000 [234:08:05<107:38:31,  3.42s/it]
 50%|████▉     | 111531/225000 [234:08:10<119:59:28,  3.81s/it]
 50%|████▉     | 111532/225000 [234:08:13<109:14:32,  3.47s/it]
 50%|████▉     | 111533/225000 [234:08:16<107:16:45,  3.40s/it]
 50%|████▉     | 111534/225000 [234:08:19<107:16:30,  3.40s/it]
 50%|████▉     | 111535/225000 [234:08:29<165:17:52,  5.24s/it]
 50%|████▉     | 111536/225000 [234:08:31<137:04:12,  4.35s/it]
 50%|████▉     | 111537/225000 [234:08:33<117:48:47,  3.74s/it]
 50%|████▉     | 111538/225000 [234:08:36<104:21:01,  3.31s/it]
 50%|████▉     | 111539/225000 [234:08:38<95:08:33,  3.02s/it] 
 50%|████▉     | 111540/225000 [234:08:40<88:36:51,  2.81s/it]
 50%|████▉     | 111541/225000 [234:08:44<94:09:06,  2.99s/it]
 50%|████▉     | 111542/225000 [234:08:47<95:34:14,  3.03s/it]
 50%|████▉     | 111543/225000 [234:08:50<96:45:36,  3.07s/it]
 50%|████▉     | 111544/225000 [234:09:01<173:23:13,  5.50s/it]
 50%|████▉     | 111545/225000 [234:09:06<167:57:54,  5.33s/it]
 50%|████▉     | 111546/225000 [234:09:11<163:42:36,  5.19s/it]
 50%|████▉     | 111547/225000 [234:09:17<168:48:38,  5.36s/it]
 50%|████▉     | 111548/225000 [234:09:22<164:20:20,  5.21s/it]
 50%|████▉     | 111549/225000 [234:09:25<146:20:35,  4.64s/it]
 50%|████▉     | 111550/225000 [234:09:28<127:40:05,  4.05s/it]
                                                               
{'loss': '0.8429', 'grad_norm': '5.001', 'learning_rate': '2.719e-05', 'epoch': '0.4958', 'num_input_tokens_seen': 391734056, 'train_runtime': '8.43e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111550/225000 [234:09:28<127:40:05,  4.05s/it]
 50%|████▉     | 111551/225000 [234:09:30<114:40:19,  3.64s/it]
 50%|████▉     | 111552/225000 [234:09:33<105:28:38,  3.35s/it]
 50%|████▉     | 111553/225000 [234:09:36<98:22:48,  3.12s/it] 
 50%|████▉     | 111554/225000 [234:09:38<94:09:52,  2.99s/it]
 50%|████▉     | 111555/225000 [234:09:48<160:49:51,  5.10s/it]
 50%|████▉     | 111556/225000 [234:09:51<134:12:17,  4.26s/it]
 50%|████▉     | 111557/225000 [234:09:54<123:04:49,  3.91s/it]
 50%|████▉     | 111558/225000 [234:09:56<108:09:47,  3.43s/it]
 50%|████▉     | 111559/225000 [234:10:01<122:38:56,  3.89s/it]
 50%|████▉     | 111560/225000 [234:10:03<107:30:54,  3.41s/it]
 50%|████▉     | 111561/225000 [234:10:16<197:24:14,  6.26s/it]
 50%|████▉     | 111562/225000 [234:10:22<196:37:35,  6.24s/it]
 50%|████▉     | 111563/225000 [234:10:25<162:51:12,  5.17s/it]
 50%|████▉     | 111564/225000 [234:10:27<136:20:56,  4.33s/it]
 50%|████▉     | 111565/225000 [234:10:30<118:19:01,  3.75s/it]
 50%|████▉     | 111566/225000 [234:10:36<140:22:12,  4.45s/it]
 50%|████▉     | 111567/225000 [234:10:38<120:25:46,  3.82s/it]
 50%|████▉     | 111568/225000 [234:10:41<107:06:36,  3.40s/it]
 50%|████▉     | 111569/225000 [234:10:43<101:22:17,  3.22s/it]
 50%|████▉     | 111570/225000 [234:10:53<165:44:46,  5.26s/it]
 50%|████▉     | 111571/225000 [234:10:56<142:05:30,  4.51s/it]
 50%|████▉     | 111572/225000 [234:11:05<178:32:09,  5.67s/it]
 50%|████▉     | 111573/225000 [234:11:08<156:09:11,  4.96s/it]
 50%|████▉     | 111574/225000 [234:11:10<134:12:53,  4.26s/it]
 50%|████▉     | 111575/225000 [234:11:18<169:59:09,  5.40s/it]
 50%|████▉     | 111576/225000 [234:11:23<163:26:49,  5.19s/it]
 50%|████▉     | 111577/225000 [234:11:26<144:45:24,  4.59s/it]
 50%|████▉     | 111578/225000 [234:11:32<151:48:11,  4.82s/it]
 50%|████▉     | 111579/225000 [234:11:35<136:39:02,  4.34s/it]
 50%|████▉     | 111580/225000 [234:11:40<141:33:14,  4.49s/it]
 50%|████▉     | 111581/225000 [234:11:45<148:45:03,  4.72s/it]
 50%|████▉     | 111582/225000 [234:11:50<148:06:33,  4.70s/it]
 50%|████▉     | 111583/225000 [234:11:52<125:01:51,  3.97s/it]
 50%|████▉     | 111584/225000 [234:11:56<123:15:58,  3.91s/it]
 50%|████▉     | 111585/225000 [234:12:05<177:28:44,  5.63s/it]
 50%|████▉     | 111586/225000 [234:12:08<144:45:41,  4.60s/it]
 50%|████▉     | 111587/225000 [234:12:10<122:28:55,  3.89s/it]
 50%|████▉     | 111588/225000 [234:12:17<155:10:03,  4.93s/it]
 50%|████▉     | 111589/225000 [234:12:20<136:47:22,  4.34s/it]
 50%|████▉     | 111590/225000 [234:12:22<116:38:27,  3.70s/it]
 50%|████▉     | 111591/225000 [234:12:25<102:15:00,  3.25s/it]
 50%|████▉     | 111592/225000 [234:12:36<182:14:05,  5.78s/it]
 50%|████▉     | 111593/225000 [234:12:41<176:39:20,  5.61s/it]
 50%|████▉     | 111594/225000 [234:12:47<173:04:23,  5.49s/it]
 50%|████▉     | 111595/225000 [234:12:49<145:04:02,  4.61s/it]
 50%|████▉     | 111596/225000 [234:13:02<222:06:46,  7.05s/it]
 50%|████▉     | 111597/225000 [234:13:08<211:59:51,  6.73s/it]
 50%|████▉     | 111598/225000 [234:13:11<175:25:33,  5.57s/it]
 50%|████▉     | 111599/225000 [234:13:14<154:28:28,  4.90s/it]
 50%|████▉     | 111600/225000 [234:13:17<135:57:42,  4.32s/it]
                                                               
{'loss': '0.8185', 'grad_norm': '3.584', 'learning_rate': '2.717e-05', 'epoch': '0.496', 'num_input_tokens_seen': 391826328, 'train_runtime': '8.432e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111600/225000 [234:13:17<135:57:42,  4.32s/it]
 50%|████▉     | 111601/225000 [234:13:22<142:36:46,  4.53s/it]
 50%|████▉     | 111602/225000 [234:13:24<121:25:50,  3.86s/it]
 50%|████▉     | 111603/225000 [234:13:29<132:39:16,  4.21s/it]
 50%|████▉     | 111604/225000 [234:13:32<118:12:51,  3.75s/it]
 50%|████▉     | 111605/225000 [234:13:37<131:39:48,  4.18s/it]
 50%|████▉     | 111606/225000 [234:13:41<125:36:31,  3.99s/it]
 50%|████▉     | 111607/225000 [234:13:45<130:46:38,  4.15s/it]
 50%|████▉     | 111608/225000 [234:13:48<113:49:39,  3.61s/it]
 50%|████▉     | 111609/225000 [234:13:50<102:04:08,  3.24s/it]
 50%|████▉     | 111610/225000 [234:13:52<93:46:37,  2.98s/it] 
 50%|████▉     | 111611/225000 [234:13:55<88:01:07,  2.79s/it]
 50%|████▉     | 111612/225000 [234:14:00<109:18:28,  3.47s/it]
 50%|████▉     | 111613/225000 [234:14:12<192:31:46,  6.11s/it]
 50%|████▉     | 111614/225000 [234:14:14<155:37:53,  4.94s/it]
 50%|████▉     | 111615/225000 [234:14:17<130:07:08,  4.13s/it]
 50%|████▉     | 111616/225000 [234:14:19<111:49:06,  3.55s/it]
 50%|████▉     | 111617/225000 [234:14:22<110:47:18,  3.52s/it]
 50%|████▉     | 111618/225000 [234:14:25<103:28:14,  3.29s/it]
 50%|████▉     | 111619/225000 [234:14:28<99:01:13,  3.14s/it] 
 50%|████▉     | 111620/225000 [234:14:31<102:38:56,  3.26s/it]
 50%|████▉     | 111621/225000 [234:14:35<104:54:38,  3.33s/it]
 50%|████▉     | 111622/225000 [234:14:38<100:11:49,  3.18s/it]
 50%|████▉     | 111623/225000 [234:14:46<152:48:01,  4.85s/it]
 50%|████▉     | 111624/225000 [234:14:49<135:13:21,  4.29s/it]
 50%|████▉     | 111625/225000 [234:14:52<117:26:18,  3.73s/it]
 50%|████▉     | 111626/225000 [234:14:54<102:46:26,  3.26s/it]
 50%|████▉     | 111627/225000 [234:14:57<100:08:56,  3.18s/it]
 50%|████▉     | 111628/225000 [234:15:04<140:17:10,  4.45s/it]
 50%|████▉     | 111629/225000 [234:15:07<122:32:04,  3.89s/it]
 50%|████▉     | 111630/225000 [234:15:10<110:27:35,  3.51s/it]
 50%|████▉     | 111631/225000 [234:15:13<108:59:32,  3.46s/it]
 50%|████▉     | 111632/225000 [234:15:16<101:12:42,  3.21s/it]
 50%|████▉     | 111633/225000 [234:15:18<96:15:50,  3.06s/it] 
 50%|████▉     | 111634/225000 [234:15:29<167:13:43,  5.31s/it]
 50%|████▉     | 111635/225000 [234:15:31<138:45:40,  4.41s/it]
 50%|████▉     | 111636/225000 [234:15:33<118:42:08,  3.77s/it]
 50%|████▉     | 111637/225000 [234:15:39<136:07:58,  4.32s/it]
 50%|████▉     | 111638/225000 [234:15:41<117:06:42,  3.72s/it]
 50%|████▉     | 111639/225000 [234:15:49<153:30:40,  4.88s/it]
 50%|████▉     | 111640/225000 [234:15:52<133:53:32,  4.25s/it]
 50%|████▉     | 111641/225000 [234:15:54<114:02:50,  3.62s/it]
 50%|████▉     | 111642/225000 [234:15:56<100:32:37,  3.19s/it]
 50%|████▉     | 111643/225000 [234:15:59<95:18:32,  3.03s/it] 
 50%|████▉     | 111644/225000 [234:16:01<92:45:25,  2.95s/it]
 50%|████▉     | 111645/225000 [234:16:04<90:34:15,  2.88s/it]
 50%|████▉     | 111646/225000 [234:16:07<88:54:18,  2.82s/it]
 50%|████▉     | 111647/225000 [234:16:10<88:10:49,  2.80s/it]
 50%|████▉     | 111648/225000 [234:16:13<94:11:07,  2.99s/it]
 50%|████▉     | 111649/225000 [234:16:17<102:25:49,  3.25s/it]
 50%|████▉     | 111650/225000 [234:16:25<144:02:16,  4.57s/it]
                                                               
{'loss': '0.8318', 'grad_norm': '5.827', 'learning_rate': '2.715e-05', 'epoch': '0.4962', 'num_input_tokens_seen': 391903952, 'train_runtime': '8.434e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111650/225000 [234:16:25<144:02:16,  4.57s/it]
 50%|████▉     | 111651/225000 [234:16:29<141:28:38,  4.49s/it]
 50%|████▉     | 111652/225000 [234:16:32<127:42:06,  4.06s/it]
 50%|████▉     | 111653/225000 [234:16:41<174:46:40,  5.55s/it]
 50%|████▉     | 111654/225000 [234:16:43<143:41:52,  4.56s/it]
 50%|████▉     | 111655/225000 [234:16:46<122:51:34,  3.90s/it]
 50%|████▉     | 111656/225000 [234:16:51<136:05:51,  4.32s/it]
 50%|████▉     | 111657/225000 [234:16:56<145:47:10,  4.63s/it]
 50%|████▉     | 111658/225000 [234:17:01<151:27:24,  4.81s/it]
 50%|████▉     | 111659/225000 [234:17:04<127:46:44,  4.06s/it]
 50%|████▉     | 111660/225000 [234:17:06<110:29:10,  3.51s/it]
 50%|████▉     | 111661/225000 [234:17:11<123:24:27,  3.92s/it]
 50%|████▉     | 111662/225000 [234:17:16<132:11:31,  4.20s/it]
 50%|████▉     | 111663/225000 [234:17:22<155:10:26,  4.93s/it]
 50%|████▉     | 111664/225000 [234:17:25<130:03:00,  4.13s/it]
 50%|████▉     | 111665/225000 [234:17:27<112:41:48,  3.58s/it]
 50%|████▉     | 111666/225000 [234:17:29<99:57:33,  3.18s/it] 
 50%|████▉     | 111667/225000 [234:17:33<106:02:51,  3.37s/it]
 50%|████▉     | 111668/225000 [234:17:37<114:59:12,  3.65s/it]
 50%|████▉     | 111669/225000 [234:17:40<102:41:32,  3.26s/it]
 50%|████▉     | 111670/225000 [234:17:42<93:44:40,  2.98s/it] 
 50%|████▉     | 111671/225000 [234:17:50<137:32:38,  4.37s/it]
 50%|████▉     | 111672/225000 [234:17:53<131:18:44,  4.17s/it]
 50%|████▉     | 111673/225000 [234:17:56<113:19:01,  3.60s/it]
 50%|████▉     | 111674/225000 [234:17:58<101:51:25,  3.24s/it]
 50%|████▉     | 111675/225000 [234:18:07<158:28:35,  5.03s/it]
 50%|████▉     | 111676/225000 [234:18:09<132:44:35,  4.22s/it]
 50%|████▉     | 111677/225000 [234:18:15<140:22:08,  4.46s/it]
 50%|████▉     | 111678/225000 [234:18:18<128:08:00,  4.07s/it]
 50%|████▉     | 111679/225000 [234:18:20<112:13:11,  3.57s/it]
 50%|████▉     | 111680/225000 [234:18:29<161:44:53,  5.14s/it]
 50%|████▉     | 111681/225000 [234:18:33<156:57:41,  4.99s/it]
 50%|████▉     | 111682/225000 [234:18:36<131:33:22,  4.18s/it]
 50%|████▉     | 111683/225000 [234:18:38<114:06:09,  3.62s/it]
 50%|████▉     | 111684/225000 [234:18:40<101:48:25,  3.23s/it]
 50%|████▉     | 111685/225000 [234:18:43<93:30:11,  2.97s/it] 
 50%|████▉     | 111686/225000 [234:18:45<87:54:25,  2.79s/it]
 50%|████▉     | 111687/225000 [234:18:49<101:42:36,  3.23s/it]
 50%|████▉     | 111688/225000 [234:18:56<131:59:35,  4.19s/it]
 50%|████▉     | 111689/225000 [234:19:01<145:18:59,  4.62s/it]
 50%|████▉     | 111690/225000 [234:19:05<130:22:02,  4.14s/it]
 50%|████▉     | 111691/225000 [234:19:08<119:56:51,  3.81s/it]
 50%|████▉     | 111692/225000 [234:19:13<133:08:59,  4.23s/it]
 50%|████▉     | 111693/225000 [234:19:16<123:44:25,  3.93s/it]
 50%|████▉     | 111694/225000 [234:19:24<161:18:51,  5.13s/it]
 50%|████▉     | 111695/225000 [234:19:31<176:49:20,  5.62s/it]
 50%|████▉     | 111696/225000 [234:19:33<144:45:38,  4.60s/it]
 50%|████▉     | 111697/225000 [234:19:35<121:47:46,  3.87s/it]
 50%|████▉     | 111698/225000 [234:19:38<114:03:14,  3.62s/it]
 50%|████▉     | 111699/225000 [234:19:41<108:32:47,  3.45s/it]
 50%|████▉     | 111700/225000 [234:19:45<110:39:16,  3.52s/it]
                                                               
{'loss': '0.8529', 'grad_norm': '4.927', 'learning_rate': '2.714e-05', 'epoch': '0.4964', 'num_input_tokens_seen': 391987032, 'train_runtime': '8.436e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111700/225000 [234:19:45<110:39:16,  3.52s/it]
 50%|████▉     | 111701/225000 [234:19:51<131:09:43,  4.17s/it]
 50%|████▉     | 111702/225000 [234:19:54<120:43:57,  3.84s/it]
 50%|████▉     | 111703/225000 [234:20:03<173:47:53,  5.52s/it]
 50%|████▉     | 111704/225000 [234:20:06<149:19:32,  4.74s/it]
 50%|████▉     | 111705/225000 [234:20:08<125:16:02,  3.98s/it]
 50%|████▉     | 111706/225000 [234:20:18<177:04:48,  5.63s/it]
 50%|████▉     | 111707/225000 [234:20:23<173:16:18,  5.51s/it]
 50%|████▉     | 111708/225000 [234:20:25<143:18:26,  4.55s/it]
 50%|████▉     | 111709/225000 [234:20:27<121:51:33,  3.87s/it]
 50%|████▉     | 111710/225000 [234:20:30<106:40:21,  3.39s/it]
 50%|████▉     | 111711/225000 [234:20:32<96:16:32,  3.06s/it] 
 50%|████▉     | 111712/225000 [234:20:34<89:21:16,  2.84s/it]
 50%|████▉     | 111713/225000 [234:20:46<168:11:09,  5.34s/it]
 50%|████▉     | 111714/225000 [234:20:50<161:48:22,  5.14s/it]
 50%|████▉     | 111715/225000 [234:20:52<134:37:37,  4.28s/it]
 50%|████▉     | 111716/225000 [234:20:55<115:26:41,  3.67s/it]
 50%|████▉     | 111717/225000 [234:20:57<101:40:56,  3.23s/it]
 50%|████▉     | 111718/225000 [234:20:59<92:43:03,  2.95s/it] 
 50%|████▉     | 111719/225000 [234:21:02<93:50:04,  2.98s/it]
 50%|████▉     | 111720/225000 [234:21:08<120:16:30,  3.82s/it]
 50%|████▉     | 111721/225000 [234:21:10<106:57:20,  3.40s/it]
 50%|████▉     | 111722/225000 [234:21:13<97:27:30,  3.10s/it] 
 50%|████▉     | 111723/225000 [234:21:15<91:29:14,  2.91s/it]
 50%|████▉     | 111724/225000 [234:21:26<165:36:02,  5.26s/it]
 50%|████▉     | 111725/225000 [234:21:28<135:59:58,  4.32s/it]
 50%|████▉     | 111726/225000 [234:21:30<116:19:46,  3.70s/it]
 50%|████▉     | 111727/225000 [234:21:35<123:07:39,  3.91s/it]
 50%|████▉     | 111728/225000 [234:21:45<177:34:16,  5.64s/it]
 50%|████▉     | 111729/225000 [234:21:47<148:45:03,  4.73s/it]
 50%|████▉     | 111730/225000 [234:21:51<138:51:49,  4.41s/it]
 50%|████▉     | 111731/225000 [234:21:53<118:25:13,  3.76s/it]
 50%|████▉     | 111732/225000 [234:21:59<134:31:32,  4.28s/it]
 50%|████▉     | 111733/225000 [234:22:01<117:05:24,  3.72s/it]
 50%|████▉     | 111734/225000 [234:22:03<104:53:37,  3.33s/it]
 50%|████▉     | 111735/225000 [234:22:06<95:57:28,  3.05s/it] 
 50%|████▉     | 111736/225000 [234:22:08<89:54:00,  2.86s/it]
 50%|████▉     | 111737/225000 [234:22:16<132:54:44,  4.22s/it]
 50%|████▉     | 111738/225000 [234:22:18<120:18:35,  3.82s/it]
 50%|████▉     | 111739/225000 [234:22:21<104:56:20,  3.34s/it]
 50%|████▉     | 111740/225000 [234:22:23<98:59:46,  3.15s/it] 
 50%|████▉     | 111741/225000 [234:22:27<99:32:11,  3.16s/it]
 50%|████▉     | 111742/225000 [234:22:30<98:57:44,  3.15s/it]
 50%|████▉     | 111743/225000 [234:22:34<105:14:24,  3.35s/it]
 50%|████▉     | 111744/225000 [234:22:37<103:44:08,  3.30s/it]
 50%|████▉     | 111745/225000 [234:22:40<102:22:53,  3.25s/it]
 50%|████▉     | 111746/225000 [234:22:43<101:54:29,  3.24s/it]
 50%|████▉     | 111747/225000 [234:22:46<101:33:13,  3.23s/it]
 50%|████▉     | 111748/225000 [234:22:49<101:36:52,  3.23s/it]
 50%|████▉     | 111749/225000 [234:22:53<101:27:43,  3.23s/it]
 50%|████▉     | 111750/225000 [234:22:57<109:42:37,  3.49s/it]
                                                               
{'loss': '0.8092', 'grad_norm': '4.506', 'learning_rate': '2.712e-05', 'epoch': '0.4967', 'num_input_tokens_seen': 392061600, 'train_runtime': '8.438e+05', 'train_tokens_per_second': '464.7'}

 50%|████▉     | 111750/225000 [234:22:57<109:42:37,  3.49s/it]
 50%|████▉     | 111751/225000 [234:23:00<106:48:35,  3.40s/it]
 50%|████▉     | 111752/225000 [234:23:03<105:28:46,  3.35s/it]
 50%|████▉     | 111753/225000 [234:23:06<102:42:30,  3.26s/it]
 50%|████▉     | 111754/225000 [234:23:10<102:40:30,  3.26s/it]
 50%|████▉     | 111755/225000 [234:23:22<193:56:19,  6.17s/it]
 50%|████▉     | 111756/225000 [234:23:25<161:29:43,  5.13s/it]
 50%|████▉     | 111757/225000 [234:23:28<142:01:40,  4.52s/it]
 50%|████▉     | 111758/225000 [234:23:31<128:07:09,  4.07s/it]
 50%|████▉     | 111759/225000 [234:23:40<175:04:41,  5.57s/it]
 50%|████▉     | 111760/225000 [234:23:43<143:46:12,  4.57s/it]
 50%|████▉     | 111761/225000 [234:23:45<121:40:56,  3.87s/it]
 50%|████▉     | 111762/225000 [234:23:47<107:52:31,  3.43s/it]
 50%|████▉     | 111763/225000 [234:23:49<96:15:14,  3.06s/it] 
 50%|████▉     | 111764/225000 [234:23:52<91:20:11,  2.90s/it]
 50%|████▉     | 111765/225000 [234:23:58<124:02:42,  3.94s/it]
 50%|████▉     | 111766/225000 [234:24:01<115:05:43,  3.66s/it]
 50%|████▉     | 111767/225000 [234:24:06<128:46:44,  4.09s/it]
 50%|████▉     | 111768/225000 [234:24:13<148:35:15,  4.72s/it]
 50%|████▉     | 111769/225000 [234:24:15<127:17:49,  4.05s/it]
 50%|████▉     | 111770/225000 [234:24:18<115:03:36,  3.66s/it]
 50%|████▉     | 111771/225000 [234:24:20<105:05:40,  3.34s/it]
 50%|████▉     | 111772/225000 [234:24:34<201:45:56,  6.41s/it]
 50%|████▉     | 111773/225000 [234:24:36<161:14:15,  5.13s/it]
 50%|████▉     | 111774/225000 [234:24:41<155:15:20,  4.94s/it]
 50%|████▉     | 111775/225000 [234:24:50<199:00:05,  6.33s/it]
 50%|████▉     | 111776/225000 [234:24:53<160:44:08,  5.11s/it]
 50%|████▉     | 111777/225000 [234:24:56<141:03:22,  4.48s/it]
 50%|████▉     | 111778/225000 [234:24:58<119:49:25,  3.81s/it]
 50%|████▉     | 111779/225000 [234:25:00<104:56:10,  3.34s/it]
 50%|████▉     | 111780/225000 [234:25:04<111:07:59,  3.53s/it]
 50%|████▉     | 111781/225000 [234:25:09<127:44:38,  4.06s/it]
 50%|████▉     | 111782/225000 [234:25:12<110:01:42,  3.50s/it]
 50%|████▉     | 111783/225000 [234:25:16<123:44:53,  3.93s/it]
 50%|████▉     | 111784/225000 [234:25:19<108:34:36,  3.45s/it]
 50%|████▉     | 111785/225000 [234:25:21<98:38:57,  3.14s/it] 
 50%|████▉     | 111786/225000 [234:25:24<92:36:37,  2.94s/it]
 50%|████▉     | 111787/225000 [234:25:28<101:59:14,  3.24s/it]
 50%|████▉     | 111788/225000 [234:25:31<99:07:57,  3.15s/it] 
 50%|████▉     | 111789/225000 [234:25:34<102:51:41,  3.27s/it]
 50%|████▉     | 111790/225000 [234:25:41<139:26:36,  4.43s/it]
 50%|████▉     | 111791/225000 [234:25:44<123:24:22,  3.92s/it]
 50%|████▉     | 111792/225000 [234:25:47<111:31:16,  3.55s/it]
 50%|████▉     | 111793/225000 [234:25:49<103:25:47,  3.29s/it]
 50%|████▉     | 111794/225000 [234:25:52<97:38:50,  3.11s/it] 
 50%|████▉     | 111795/225000 [234:25:55<93:54:29,  2.99s/it]
 50%|████▉     | 111796/225000 [234:25:57<90:22:44,  2.87s/it]
 50%|████▉     | 111797/225000 [234:26:01<95:12:25,  3.03s/it]
 50%|████▉     | 111798/225000 [234:26:12<175:10:18,  5.57s/it]
 50%|████▉     | 111799/225000 [234:26:16<157:30:03,  5.01s/it]
 50%|████▉     | 111800/225000 [234:26:18<132:42:33,  4.22s/it]
                                                               
{'loss': '0.8938', 'grad_norm': '3.837', 'learning_rate': '2.71e-05', 'epoch': '0.4969', 'num_input_tokens_seen': 392142720, 'train_runtime': '8.44e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 111800/225000 [234:26:18<132:42:33,  4.22s/it]
 50%|████▉     | 111801/225000 [234:26:20<113:22:07,  3.61s/it]
 50%|████▉     | 111802/225000 [234:26:23<100:02:14,  3.18s/it]
 50%|████▉     | 111803/225000 [234:26:30<136:42:05,  4.35s/it]
 50%|████▉     | 111804/225000 [234:26:34<132:22:28,  4.21s/it]
 50%|████▉     | 111805/225000 [234:26:36<117:10:41,  3.73s/it]
 50%|████▉     | 111806/225000 [234:26:41<127:36:53,  4.06s/it]
 50%|████▉     | 111807/225000 [234:26:43<111:54:13,  3.56s/it]
 50%|████▉     | 111808/225000 [234:26:46<101:54:01,  3.24s/it]
 50%|████▉     | 111809/225000 [234:26:49<101:32:15,  3.23s/it]
 50%|████▉     | 111810/225000 [234:26:52<96:51:23,  3.08s/it] 
 50%|████▉     | 111811/225000 [234:26:55<93:51:22,  2.99s/it]
 50%|████▉     | 111812/225000 [234:26:57<91:39:42,  2.92s/it]
 50%|████▉     | 111813/225000 [234:27:01<95:33:28,  3.04s/it]
 50%|████▉     | 111814/225000 [234:27:06<111:52:56,  3.56s/it]
 50%|████▉     | 111815/225000 [234:27:14<155:55:28,  4.96s/it]
 50%|████▉     | 111816/225000 [234:27:16<129:41:41,  4.13s/it]
 50%|████▉     | 111817/225000 [234:27:18<111:44:21,  3.55s/it]
 50%|████▉     | 111818/225000 [234:27:20<99:03:14,  3.15s/it] 
 50%|████▉     | 111819/225000 [234:27:28<139:25:19,  4.43s/it]
 50%|████▉     | 111820/225000 [234:27:30<121:58:48,  3.88s/it]
 50%|████▉     | 111821/225000 [234:27:33<109:59:02,  3.50s/it]
 50%|████▉     | 111822/225000 [234:27:36<101:22:02,  3.22s/it]
 50%|████▉     | 111823/225000 [234:27:38<95:24:57,  3.04s/it] 
 50%|████▉     | 111824/225000 [234:27:41<91:29:21,  2.91s/it]
 50%|████▉     | 111825/225000 [234:27:46<112:52:30,  3.59s/it]
 50%|████▉     | 111826/225000 [234:27:49<110:00:14,  3.50s/it]
 50%|████▉     | 111827/225000 [234:27:52<107:23:46,  3.42s/it]
 50%|████▉     | 111828/225000 [234:28:03<176:50:42,  5.63s/it]
 50%|████▉     | 111829/225000 [234:28:09<178:06:05,  5.67s/it]
 50%|████▉     | 111830/225000 [234:28:12<149:37:17,  4.76s/it]
 50%|████▉     | 111831/225000 [234:28:20<184:53:32,  5.88s/it]
 50%|████▉     | 111832/225000 [234:28:22<149:22:20,  4.75s/it]
 50%|████▉     | 111833/225000 [234:28:24<125:16:22,  3.99s/it]
 50%|████▉     | 111834/225000 [234:28:31<150:35:01,  4.79s/it]
 50%|████▉     | 111835/225000 [234:28:33<126:26:50,  4.02s/it]
 50%|████▉     | 111836/225000 [234:28:36<109:39:12,  3.49s/it]
 50%|████▉     | 111837/225000 [234:28:38<98:05:31,  3.12s/it] 
 50%|████▉     | 111838/225000 [234:28:47<157:20:07,  5.01s/it]
 50%|████▉     | 111839/225000 [234:28:51<143:41:13,  4.57s/it]
 50%|████▉     | 111840/225000 [234:28:54<128:13:03,  4.08s/it]
 50%|████▉     | 111841/225000 [234:28:56<110:12:01,  3.51s/it]
 50%|████▉     | 111842/225000 [234:28:58<98:57:58,  3.15s/it] 
 50%|████▉     | 111843/225000 [234:29:01<94:12:35,  3.00s/it]
 50%|████▉     | 111844/225000 [234:29:11<160:36:52,  5.11s/it]
 50%|████▉     | 111845/225000 [234:29:19<192:49:37,  6.13s/it]
 50%|████▉     | 111846/225000 [234:29:22<156:21:22,  4.97s/it]
 50%|████▉     | 111847/225000 [234:29:24<130:46:34,  4.16s/it]
 50%|████▉     | 111848/225000 [234:29:26<114:17:53,  3.64s/it]
 50%|████▉     | 111849/225000 [234:29:29<108:42:26,  3.46s/it]
 50%|████▉     | 111850/225000 [234:29:32<100:36:48,  3.20s/it]
                                                               
{'loss': '0.8914', 'grad_norm': '3.393', 'learning_rate': '2.708e-05', 'epoch': '0.4971', 'num_input_tokens_seen': 392221544, 'train_runtime': '8.442e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 111850/225000 [234:29:32<100:36:48,  3.20s/it]
 50%|████▉     | 111851/225000 [234:29:35<101:57:49,  3.24s/it]
 50%|████▉     | 111852/225000 [234:29:48<191:58:51,  6.11s/it]
 50%|████▉     | 111853/225000 [234:29:51<161:47:04,  5.15s/it]
 50%|████▉     | 111854/225000 [234:29:53<132:53:25,  4.23s/it]
 50%|████▉     | 111855/225000 [234:29:56<117:41:29,  3.74s/it]
 50%|████▉     | 111856/225000 [234:30:06<179:59:33,  5.73s/it]
 50%|████▉     | 111857/225000 [234:30:09<155:55:10,  4.96s/it]
 50%|████▉     | 111858/225000 [234:30:13<139:40:52,  4.44s/it]
 50%|████▉     | 111859/225000 [234:30:15<122:31:32,  3.90s/it]
 50%|████▉     | 111860/225000 [234:30:18<110:42:22,  3.52s/it]
 50%|████▉     | 111861/225000 [234:30:20<100:07:42,  3.19s/it]
 50%|████▉     | 111862/225000 [234:30:23<92:44:50,  2.95s/it] 
 50%|████▉     | 111863/225000 [234:30:25<87:28:34,  2.78s/it]
 50%|████▉     | 111864/225000 [234:30:28<84:57:10,  2.70s/it]
 50%|████▉     | 111865/225000 [234:30:31<89:18:22,  2.84s/it]
 50%|████▉     | 111866/225000 [234:30:35<105:06:23,  3.34s/it]
 50%|████▉     | 111867/225000 [234:30:38<104:14:28,  3.32s/it]
 50%|████▉     | 111868/225000 [234:30:45<135:41:00,  4.32s/it]
 50%|████▉     | 111869/225000 [234:30:52<160:56:39,  5.12s/it]
 50%|████▉     | 111870/225000 [234:30:55<137:25:33,  4.37s/it]
 50%|████▉     | 111871/225000 [234:30:58<130:26:27,  4.15s/it]
 50%|████▉     | 111872/225000 [234:31:01<118:33:02,  3.77s/it]
 50%|████▉     | 111873/225000 [234:31:04<112:26:10,  3.58s/it]
 50%|████▉     | 111874/225000 [234:31:08<114:50:19,  3.65s/it]
 50%|████▉     | 111875/225000 [234:31:20<191:50:33,  6.11s/it]
 50%|████▉     | 111876/225000 [234:31:23<160:07:30,  5.10s/it]
 50%|████▉     | 111877/225000 [234:31:25<133:51:09,  4.26s/it]
 50%|████▉     | 111878/225000 [234:31:27<115:03:06,  3.66s/it]
 50%|████▉     | 111879/225000 [234:31:30<102:08:36,  3.25s/it]
 50%|████▉     | 111880/225000 [234:31:34<109:53:02,  3.50s/it]
 50%|████▉     | 111881/225000 [234:31:38<113:42:32,  3.62s/it]
 50%|████▉     | 111882/225000 [234:31:41<110:58:13,  3.53s/it]
 50%|████▉     | 111883/225000 [234:31:44<108:13:27,  3.44s/it]
 50%|████▉     | 111884/225000 [234:31:48<107:09:57,  3.41s/it]
 50%|████▉     | 111885/225000 [234:31:53<129:41:11,  4.13s/it]
 50%|████▉     | 111886/225000 [234:31:58<131:03:43,  4.17s/it]
 50%|████▉     | 111887/225000 [234:32:00<116:14:26,  3.70s/it]
 50%|████▉     | 111888/225000 [234:32:04<114:04:24,  3.63s/it]
 50%|████▉     | 111889/225000 [234:32:11<151:34:34,  4.82s/it]
 50%|████▉     | 111890/225000 [234:32:14<128:36:39,  4.09s/it]
 50%|████▉     | 111891/225000 [234:32:16<112:53:47,  3.59s/it]
 50%|████▉     | 111892/225000 [234:32:18<101:42:25,  3.24s/it]
 50%|████▉     | 111893/225000 [234:32:23<118:10:37,  3.76s/it]
 50%|████▉     | 111894/225000 [234:32:26<107:45:24,  3.43s/it]
 50%|████▉     | 111895/225000 [234:32:38<187:31:15,  5.97s/it]
 50%|████▉     | 111896/225000 [234:32:41<155:36:15,  4.95s/it]
 50%|████▉     | 111897/225000 [234:32:44<136:20:34,  4.34s/it]
 50%|████▉     | 111898/225000 [234:32:46<119:58:43,  3.82s/it]
 50%|████▉     | 111899/225000 [234:32:49<108:52:28,  3.47s/it]
 50%|████▉     | 111900/225000 [234:32:52<103:47:11,  3.30s/it]
                                                               
{'loss': '0.9055', 'grad_norm': '4.256', 'learning_rate': '2.706e-05', 'epoch': '0.4973', 'num_input_tokens_seen': 392305976, 'train_runtime': '8.444e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 111900/225000 [234:32:52<103:47:11,  3.30s/it]
 50%|████▉     | 111901/225000 [234:33:04<187:42:21,  5.97s/it]
 50%|████▉     | 111902/225000 [234:33:07<156:19:39,  4.98s/it]
 50%|████▉     | 111903/225000 [234:33:09<132:03:25,  4.20s/it]
 50%|████▉     | 111904/225000 [234:33:14<142:44:56,  4.54s/it]
 50%|████▉     | 111905/225000 [234:33:17<122:28:06,  3.90s/it]
 50%|████▉     | 111906/225000 [234:33:19<110:20:15,  3.51s/it]
 50%|████▉     | 111907/225000 [234:33:34<220:09:19,  7.01s/it]
 50%|████▉     | 111908/225000 [234:33:38<184:15:10,  5.87s/it]
 50%|████▉     | 111909/225000 [234:33:40<152:04:23,  4.84s/it]
 50%|████▉     | 111910/225000 [234:33:43<136:29:37,  4.35s/it]
 50%|████▉     | 111911/225000 [234:33:46<118:51:49,  3.78s/it]
 50%|████▉     | 111912/225000 [234:33:52<141:19:50,  4.50s/it]
 50%|████▉     | 111913/225000 [234:33:54<121:50:08,  3.88s/it]
 50%|████▉     | 111914/225000 [234:33:57<111:39:01,  3.55s/it]
 50%|████▉     | 111915/225000 [234:34:00<106:31:50,  3.39s/it]
 50%|████▉     | 111916/225000 [234:34:04<111:51:18,  3.56s/it]
 50%|████▉     | 111917/225000 [234:34:07<106:01:08,  3.38s/it]
 50%|████▉     | 111918/225000 [234:34:14<135:14:59,  4.31s/it]
 50%|████▉     | 111919/225000 [234:34:19<146:05:00,  4.65s/it]
 50%|████▉     | 111920/225000 [234:34:22<132:52:34,  4.23s/it]
 50%|████▉     | 111921/225000 [234:34:25<116:57:47,  3.72s/it]
 50%|████▉     | 111922/225000 [234:34:27<106:04:56,  3.38s/it]
 50%|████▉     | 111923/225000 [234:34:30<98:45:54,  3.14s/it] 
 50%|████▉     | 111924/225000 [234:34:33<97:17:47,  3.10s/it]
 50%|████▉     | 111925/225000 [234:34:36<99:26:05,  3.17s/it]
 50%|████▉     | 111926/225000 [234:34:39<96:16:25,  3.07s/it]
 50%|████▉     | 111927/225000 [234:34:42<91:48:01,  2.92s/it]
 50%|████▉     | 111928/225000 [234:34:44<88:58:38,  2.83s/it]
 50%|████▉     | 111929/225000 [234:35:01<216:27:45,  6.89s/it]
 50%|████▉     | 111930/225000 [234:35:03<175:50:38,  5.60s/it]
 50%|████▉     | 111931/225000 [234:35:06<147:05:57,  4.68s/it]
 50%|████▉     | 111932/225000 [234:35:15<191:58:27,  6.11s/it]
 50%|████▉     | 111933/225000 [234:35:17<155:00:26,  4.94s/it]
 50%|████▉     | 111934/225000 [234:35:20<133:33:08,  4.25s/it]
 50%|████▉     | 111935/225000 [234:35:23<124:07:08,  3.95s/it]
 50%|████▉     | 111936/225000 [234:35:33<178:50:42,  5.69s/it]
 50%|████▉     | 111937/225000 [234:35:36<153:31:12,  4.89s/it]
 50%|████▉     | 111938/225000 [234:35:39<132:08:16,  4.21s/it]
 50%|████▉     | 111939/225000 [234:35:41<115:54:01,  3.69s/it]
 50%|████▉     | 111940/225000 [234:35:44<105:23:14,  3.36s/it]
 50%|████▉     | 111941/225000 [234:35:46<97:27:47,  3.10s/it] 
 50%|████▉     | 111942/225000 [234:35:49<93:59:26,  2.99s/it]
 50%|████▉     | 111943/225000 [234:36:04<207:46:52,  6.62s/it]
 50%|████▉     | 111944/225000 [234:36:07<174:58:48,  5.57s/it]
 50%|████▉     | 111945/225000 [234:36:10<149:15:59,  4.75s/it]
 50%|████▉     | 111946/225000 [234:36:19<192:01:00,  6.11s/it]
 50%|████▉     | 111947/225000 [234:36:22<161:43:50,  5.15s/it]
 50%|████▉     | 111948/225000 [234:36:24<133:23:46,  4.25s/it]
 50%|████▉     | 111949/225000 [234:36:31<155:22:06,  4.95s/it]
 50%|████▉     | 111950/225000 [234:36:36<156:39:56,  4.99s/it]
                                                               
{'loss': '0.8601', 'grad_norm': '2.623', 'learning_rate': '2.704e-05', 'epoch': '0.4976', 'num_input_tokens_seen': 392397760, 'train_runtime': '8.446e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 111950/225000 [234:36:36<156:39:56,  4.99s/it]
 50%|████▉     | 111951/225000 [234:36:44<186:16:50,  5.93s/it]
 50%|████▉     | 111952/225000 [234:36:47<159:12:21,  5.07s/it]
 50%|████▉     | 111953/225000 [234:36:50<140:56:11,  4.49s/it]
 50%|████▉     | 111954/225000 [234:36:53<123:47:16,  3.94s/it]
 50%|████▉     | 111955/225000 [234:36:56<117:45:07,  3.75s/it]
 50%|████▉     | 111956/225000 [234:36:59<109:21:01,  3.48s/it]
 50%|████▉     | 111957/225000 [234:37:02<105:11:15,  3.35s/it]
 50%|████▉     | 111958/225000 [234:37:05<97:54:32,  3.12s/it] 
 50%|████▉     | 111959/225000 [234:37:07<93:02:10,  2.96s/it]
 50%|████▉     | 111960/225000 [234:37:10<91:42:03,  2.92s/it]
 50%|████▉     | 111961/225000 [234:37:19<145:30:43,  4.63s/it]
 50%|████▉     | 111962/225000 [234:37:22<126:39:19,  4.03s/it]
 50%|████▉     | 111963/225000 [234:37:24<113:51:09,  3.63s/it]
 50%|████▉     | 111964/225000 [234:37:27<104:40:41,  3.33s/it]
 50%|████▉     | 111965/225000 [234:37:30<107:01:01,  3.41s/it]
 50%|████▉     | 111966/225000 [234:37:33<99:32:31,  3.17s/it] 
 50%|████▉     | 111967/225000 [234:37:36<94:17:45,  3.00s/it]
 50%|████▉     | 111968/225000 [234:37:39<93:41:12,  2.98s/it]
 50%|████▉     | 111969/225000 [234:37:43<107:22:09,  3.42s/it]
 50%|████▉     | 111970/225000 [234:37:46<99:00:58,  3.15s/it] 
 50%|████▉     | 111971/225000 [234:37:55<158:32:56,  5.05s/it]
 50%|████▉     | 111972/225000 [234:37:58<137:43:50,  4.39s/it]
 50%|████▉     | 111973/225000 [234:38:00<120:46:23,  3.85s/it]
 50%|████▉     | 111974/225000 [234:38:03<111:24:09,  3.55s/it]
 50%|████▉     | 111975/225000 [234:38:13<173:36:31,  5.53s/it]
 50%|████▉     | 111976/225000 [234:38:16<148:57:21,  4.74s/it]
 50%|████▉     | 111977/225000 [234:38:20<133:37:39,  4.26s/it]
 50%|████▉     | 111978/225000 [234:38:22<118:07:24,  3.76s/it]
 50%|████▉     | 111979/225000 [234:38:25<110:13:55,  3.51s/it]
 50%|████▉     | 111980/225000 [234:38:28<101:40:45,  3.24s/it]
 50%|████▉     | 111981/225000 [234:38:30<95:53:37,  3.05s/it] 
 50%|████▉     | 111982/225000 [234:38:41<165:45:02,  5.28s/it]
 50%|████▉     | 111983/225000 [234:38:44<142:52:47,  4.55s/it]
 50%|████▉     | 111984/225000 [234:38:46<127:00:30,  4.05s/it]
 50%|████▉     | 111985/225000 [234:38:50<122:17:24,  3.90s/it]
 50%|████▉     | 111986/225000 [234:38:53<116:18:32,  3.70s/it]
 50%|████▉     | 111987/225000 [234:39:03<174:50:53,  5.57s/it]
 50%|████▉     | 111988/225000 [234:39:06<145:20:38,  4.63s/it]
 50%|████▉     | 111989/225000 [234:39:11<153:42:07,  4.90s/it]
 50%|████▉     | 111990/225000 [234:39:20<186:25:56,  5.94s/it]
 50%|████▉     | 111991/225000 [234:39:23<159:15:46,  5.07s/it]
 50%|████▉     | 111992/225000 [234:39:26<147:02:35,  4.68s/it]
 50%|████▉     | 111993/225000 [234:39:29<129:02:05,  4.11s/it]
 50%|████▉     | 111994/225000 [234:39:32<119:41:48,  3.81s/it]
 50%|████▉     | 111995/225000 [234:39:35<112:35:47,  3.59s/it]
 50%|████▉     | 111996/225000 [234:39:45<169:06:16,  5.39s/it]
 50%|████▉     | 111997/225000 [234:39:47<140:41:59,  4.48s/it]
 50%|████▉     | 111998/225000 [234:39:50<120:51:38,  3.85s/it]
 50%|████▉     | 111999/225000 [234:39:52<107:18:08,  3.42s/it]
 50%|████▉     | 112000/225000 [234:39:55<100:11:18,  3.19s/it]
                                                               
{'loss': '0.8959', 'grad_norm': '3.669', 'learning_rate': '2.703e-05', 'epoch': '0.4978', 'num_input_tokens_seen': 392481600, 'train_runtime': '8.448e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 112000/225000 [234:39:55<100:11:18,  3.19s/it]
 50%|████▉     | 112001/225000 [234:39:58<99:34:38,  3.17s/it] 
 50%|████▉     | 112002/225000 [234:40:08<167:43:26,  5.34s/it]
 50%|████▉     | 112003/225000 [234:40:12<149:08:45,  4.75s/it]
 50%|████▉     | 112004/225000 [234:40:14<124:07:45,  3.95s/it]
 50%|████▉     | 112005/225000 [234:40:23<178:01:35,  5.67s/it]
 50%|████▉     | 112006/225000 [234:40:26<144:55:17,  4.62s/it]
 50%|████▉     | 112007/225000 [234:40:28<121:49:36,  3.88s/it]
 50%|████▉     | 112008/225000 [234:40:30<108:43:20,  3.46s/it]
 50%|████▉     | 112009/225000 [234:40:33<100:55:06,  3.22s/it]
 50%|████▉     | 112010/225000 [234:40:36<101:50:31,  3.24s/it]
 50%|████▉     | 112011/225000 [234:40:48<181:31:26,  5.78s/it]
 50%|████▉     | 112012/225000 [234:40:50<151:44:39,  4.83s/it]
 50%|████▉     | 112013/225000 [234:40:53<127:25:47,  4.06s/it]
 50%|████▉     | 112014/225000 [234:40:55<113:44:53,  3.62s/it]
 50%|████▉     | 112015/225000 [234:41:04<159:53:41,  5.09s/it]
 50%|████▉     | 112016/225000 [234:41:12<191:18:49,  6.10s/it]
 50%|████▉     | 112017/225000 [234:41:14<153:50:57,  4.90s/it]
 50%|████▉     | 112018/225000 [234:41:17<130:24:45,  4.16s/it]
 50%|████▉     | 112019/225000 [234:41:20<123:54:58,  3.95s/it]
 50%|████▉     | 112020/225000 [234:41:24<119:04:38,  3.79s/it]
 50%|████▉     | 112021/225000 [234:41:27<115:54:25,  3.69s/it]
 50%|████▉     | 112022/225000 [234:41:31<116:02:40,  3.70s/it]
 50%|████▉     | 112023/225000 [234:41:33<105:08:54,  3.35s/it]
 50%|████▉     | 112024/225000 [234:41:39<126:14:22,  4.02s/it]
 50%|████▉     | 112025/225000 [234:41:41<110:44:17,  3.53s/it]
 50%|████▉     | 112026/225000 [234:41:45<109:36:09,  3.49s/it]
 50%|████▉     | 112027/225000 [234:41:47<98:59:35,  3.15s/it] 
 50%|████▉     | 112028/225000 [234:41:50<93:26:33,  2.98s/it]
 50%|████▉     | 112029/225000 [234:42:03<190:24:28,  6.07s/it]
 50%|████▉     | 112030/225000 [234:42:05<153:05:22,  4.88s/it]
 50%|████▉     | 112031/225000 [234:42:07<127:38:21,  4.07s/it]
 50%|████▉     | 112032/225000 [234:42:10<118:24:12,  3.77s/it]
 50%|████▉     | 112033/225000 [234:42:13<106:09:45,  3.38s/it]
 50%|████▉     | 112034/225000 [234:42:15<97:41:00,  3.11s/it] 
 50%|████▉     | 112035/225000 [234:42:18<94:13:46,  3.00s/it]
 50%|████▉     | 112036/225000 [234:42:21<91:16:13,  2.91s/it]
 50%|████▉     | 112037/225000 [234:42:28<135:21:18,  4.31s/it]
 50%|████▉     | 112038/225000 [234:42:32<128:02:05,  4.08s/it]
 50%|████▉     | 112039/225000 [234:42:40<163:20:58,  5.21s/it]
 50%|████▉     | 112040/225000 [234:42:43<142:21:42,  4.54s/it]
 50%|████▉     | 112041/225000 [234:42:46<127:22:40,  4.06s/it]
 50%|████▉     | 112042/225000 [234:42:49<116:58:20,  3.73s/it]
 50%|████▉     | 112043/225000 [234:42:52<113:31:25,  3.62s/it]
 50%|████▉     | 112044/225000 [234:42:56<116:23:26,  3.71s/it]
 50%|████▉     | 112045/225000 [234:43:06<174:50:58,  5.57s/it]
 50%|████▉     | 112046/225000 [234:43:09<149:31:01,  4.77s/it]
 50%|████▉     | 112047/225000 [234:43:11<125:22:06,  4.00s/it]
 50%|████▉     | 112048/225000 [234:43:20<171:20:16,  5.46s/it]
 50%|████▉     | 112049/225000 [234:43:23<153:44:48,  4.90s/it]
 50%|████▉     | 112050/225000 [234:43:26<128:23:57,  4.09s/it]
                                                               
{'loss': '0.8929', 'grad_norm': '2.304', 'learning_rate': '2.701e-05', 'epoch': '0.498', 'num_input_tokens_seen': 392567032, 'train_runtime': '8.45e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 112050/225000 [234:43:26<128:23:57,  4.09s/it]
 50%|████▉     | 112051/225000 [234:43:35<175:19:58,  5.59s/it]
 50%|████▉     | 112052/225000 [234:43:37<143:36:30,  4.58s/it]
 50%|████▉     | 112053/225000 [234:43:39<121:57:08,  3.89s/it]
 50%|████▉     | 112054/225000 [234:43:41<106:09:14,  3.38s/it]
 50%|████▉     | 112055/225000 [234:43:47<128:45:42,  4.10s/it]
 50%|████▉     | 112056/225000 [234:43:55<161:20:41,  5.14s/it]
 50%|████▉     | 112057/225000 [234:43:58<139:39:24,  4.45s/it]
 50%|████▉     | 112058/225000 [234:44:01<132:14:04,  4.21s/it]
 50%|████▉     | 112059/225000 [234:44:09<170:22:29,  5.43s/it]
 50%|████▉     | 112060/225000 [234:44:13<147:57:36,  4.72s/it]
 50%|████▉     | 112061/225000 [234:44:21<182:31:20,  5.82s/it]
 50%|████▉     | 112062/225000 [234:44:27<188:34:45,  6.01s/it]
 50%|████▉     | 112063/225000 [234:44:30<156:05:15,  4.98s/it]
 50%|████▉     | 112064/225000 [234:44:33<137:59:30,  4.40s/it]
 50%|████▉     | 112065/225000 [234:44:36<120:45:32,  3.85s/it]
 50%|████▉     | 112066/225000 [234:44:39<115:24:37,  3.68s/it]
 50%|████▉     | 112067/225000 [234:44:41<105:39:48,  3.37s/it]
 50%|████▉     | 112068/225000 [234:44:44<97:31:15,  3.11s/it] 
 50%|████▉     | 112069/225000 [234:44:50<126:06:02,  4.02s/it]
 50%|████▉     | 112070/225000 [234:44:57<149:18:06,  4.76s/it]
 50%|████▉     | 112071/225000 [234:44:59<128:09:42,  4.09s/it]
 50%|████▉     | 112072/225000 [234:45:01<111:11:11,  3.54s/it]
 50%|████▉     | 112073/225000 [234:45:07<126:20:21,  4.03s/it]
 50%|████▉     | 112074/225000 [234:45:09<112:33:02,  3.59s/it]
 50%|████▉     | 112075/225000 [234:45:12<110:26:39,  3.52s/it]
 50%|████▉     | 112076/225000 [234:45:16<112:04:43,  3.57s/it]
 50%|████▉     | 112077/225000 [234:45:22<134:48:35,  4.30s/it]
 50%|████▉     | 112078/225000 [234:45:24<115:50:21,  3.69s/it]
 50%|████▉     | 112079/225000 [234:45:27<102:40:04,  3.27s/it]
 50%|████▉     | 112080/225000 [234:45:33<127:08:31,  4.05s/it]
 50%|████▉     | 112081/225000 [234:45:36<118:13:19,  3.77s/it]
 50%|████▉     | 112082/225000 [234:45:39<115:25:05,  3.68s/it]
 50%|████▉     | 112083/225000 [234:45:49<170:44:06,  5.44s/it]
 50%|████▉     | 112084/225000 [234:45:52<151:04:48,  4.82s/it]
 50%|████▉     | 112085/225000 [234:45:55<131:43:07,  4.20s/it]
 50%|████▉     | 112086/225000 [234:45:58<123:13:44,  3.93s/it]
 50%|████▉     | 112087/225000 [234:46:01<112:19:55,  3.58s/it]
 50%|████▉     | 112088/225000 [234:46:04<110:55:26,  3.54s/it]
 50%|████▉     | 112089/225000 [234:46:07<106:41:25,  3.40s/it]
 50%|████▉     | 112090/225000 [234:46:10<100:35:30,  3.21s/it]
 50%|████▉     | 112091/225000 [234:46:13<97:10:51,  3.10s/it] 
 50%|████▉     | 112092/225000 [234:46:16<100:09:21,  3.19s/it]
 50%|████▉     | 112093/225000 [234:46:20<102:11:47,  3.26s/it]
 50%|████▉     | 112094/225000 [234:46:24<109:59:27,  3.51s/it]
 50%|████▉     | 112095/225000 [234:46:27<106:04:54,  3.38s/it]
 50%|████▉     | 112096/225000 [234:46:30<101:09:26,  3.23s/it]
 50%|████▉     | 112097/225000 [234:46:33<100:23:11,  3.20s/it]
 50%|████▉     | 112098/225000 [234:46:37<106:45:29,  3.40s/it]
 50%|████▉     | 112099/225000 [234:46:40<101:54:33,  3.25s/it]
 50%|████▉     | 112100/225000 [234:46:52<181:48:48,  5.80s/it]
                                                               
{'loss': '0.9202', 'grad_norm': '2.384', 'learning_rate': '2.699e-05', 'epoch': '0.4982', 'num_input_tokens_seen': 392652624, 'train_runtime': '8.452e+05', 'train_tokens_per_second': '464.6'}

 50%|████▉     | 112100/225000 [234:46:52<181:48:48,  5.80s/it]
 50%|████▉     | 112101/225000 [234:46:55<157:03:46,  5.01s/it]
 50%|████▉     | 112102/225000 [234:46:58<136:24:51,  4.35s/it]
 50%|████▉     | 112103/225000 [234:47:00<117:59:50,  3.76s/it]
 50%|████▉     | 112104/225000 [234:47:03<106:58:25,  3.41s/it]
 50%|████▉     | 112105/225000 [234:47:09<139:38:04,  4.45s/it]
 50%|████▉     | 112106/225000 [234:47:13<135:49:25,  4.33s/it]
 50%|████▉     | 112107/225000 [234:47:17<124:32:20,  3.97s/it]
 50%|████▉     | 112108/225000 [234:47:20<122:19:00,  3.90s/it]
 50%|████▉     | 112109/225000 [234:47:23<112:22:17,  3.58s/it]
 50%|████▉     | 112110/225000 [234:47:26<102:04:46,  3.26s/it]
 50%|████▉     | 112111/225000 [234:47:29<103:29:01,  3.30s/it]
 50%|████▉     | 112112/225000 [234:47:32<96:30:33,  3.08s/it] 
 50%|████▉     | 112113/225000 [234:47:35<96:58:46,  3.09s/it]
 50%|████▉     | 112114/225000 [234:47:45<161:47:01,  5.16s/it]
 50%|████▉     | 112115/225000 [234:47:48<148:10:53,  4.73s/it]
 50%|████▉     | 112116/225000 [234:47:52<138:03:07,  4.40s/it]
 50%|████▉     | 112117/225000 [234:47:55<123:52:00,  3.95s/it]
 50%|████▉     | 112118/225000 [234:47:58<117:39:43,  3.75s/it]
 50%|████▉     | 112119/225000 [234:48:09<180:35:06,  5.76s/it]
 50%|████▉     | 112120/225000 [234:48:13<166:05:30,  5.30s/it]
 50%|████▉     | 112121/225000 [234:48:16<141:05:28,  4.50s/it]
 50%|████▉     | 112122/225000 [234:48:18<119:45:42,  3.82s/it]
 50%|████▉     | 112123/225000 [234:48:21<113:27:11,  3.62s/it]
 50%|████▉     | 112124/225000 [234:48:23<102:39:43,  3.27s/it]
 50%|████▉     | 112125/225000 [234:48:27<101:36:59,  3.24s/it]
 50%|████▉     | 112126/225000 [234:48:30<100:02:07,  3.19s/it]
 50%|████▉     | 112127/225000 [234:48:33<104:45:58,  3.34s/it]
 50%|████▉     | 112128/225000 [234:48:48<214:57:01,  6.86s/it]
 50%|████▉     | 112129/225000 [234:48:52<180:15:00,  5.75s/it]
 50%|████▉     | 112130/225000 [234:48:54<150:55:12,  4.81s/it]
 50%|████▉     | 112131/225000 [234:49:06<216:45:01,  6.91s/it]
 50%|████▉     | 112132/225000 [234:49:10<193:29:24,  6.17s/it]
 50%|████▉     | 112133/225000 [234:49:13<157:24:34,  5.02s/it]
 50%|████▉     | 112134/225000 [234:49:15<132:32:46,  4.23s/it]
 50%|████▉     | 112135/225000 [234:49:18<117:01:56,  3.73s/it]
 50%|████▉     | 112136/225000 [234:49:21<114:04:09,  3.64s/it]
 50%|████▉     | 112137/225000 [234:49:31<175:32:56,  5.60s/it]
 50%|████▉     | 112138/225000 [234:49:34<143:43:50,  4.58s/it]
 50%|████▉     | 112139/225000 [234:49:37<128:05:54,  4.09s/it]
 50%|████▉     | 112140/225000 [234:49:47<186:23:49,  5.95s/it]
 50%|████▉     | 112141/225000 [234:49:51<165:35:17,  5.28s/it]
 50%|████▉     | 112142/225000 [234:49:53<137:53:51,  4.40s/it]
 50%|████▉     | 112143/225000 [234:49:55<119:13:21,  3.80s/it]
 50%|████▉     | 112144/225000 [234:50:05<173:53:14,  5.55s/it]
 50%|████▉     | 112145/225000 [234:50:07<143:48:57,  4.59s/it]
 50%|████▉     | 112146/225000 [234:50:10<128:26:48,  4.10s/it]
 50%|████▉     | 112147/225000 [234:50:13<117:54:41,  3.76s/it]
 50%|████▉     | 112148/225000 [234:50:16<108:11:32,  3.45s/it]
 50%|████▉     | 112149/225000 [234:50:20<112:41:25,  3.59s/it]
 50%|████▉     | 112150/225000 [234:50:23<111:48:30,  3.57s/it]
                                                               
{'loss': '0.9245', 'grad_norm': '2.672', 'learning_rate': '2.697e-05', 'epoch': '0.4984', 'num_input_tokens_seen': 392739776, 'train_runtime': '8.454e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112150/225000 [234:50:23<111:48:30,  3.57s/it]
 50%|████▉     | 112151/225000 [234:50:27<115:03:52,  3.67s/it]
 50%|████▉     | 112152/225000 [234:50:31<111:36:14,  3.56s/it]
 50%|████▉     | 112153/225000 [234:50:34<110:43:51,  3.53s/it]
 50%|████▉     | 112154/225000 [234:50:45<185:30:58,  5.92s/it]
 50%|████▉     | 112155/225000 [234:50:49<160:14:00,  5.11s/it]
 50%|████▉     | 112156/225000 [234:50:51<137:28:22,  4.39s/it]
 50%|████▉     | 112157/225000 [234:50:54<121:29:39,  3.88s/it]
 50%|████▉     | 112158/225000 [234:51:00<136:23:17,  4.35s/it]
 50%|████▉     | 112159/225000 [234:51:08<171:18:49,  5.47s/it]
 50%|████▉     | 112160/225000 [234:51:11<155:44:51,  4.97s/it]
 50%|████▉     | 112161/225000 [234:51:14<132:22:33,  4.22s/it]
 50%|████▉     | 112162/225000 [234:51:17<123:39:43,  3.95s/it]
 50%|████▉     | 112163/225000 [234:51:20<113:02:19,  3.61s/it]
 50%|████▉     | 112164/225000 [234:51:23<110:13:36,  3.52s/it]
 50%|████▉     | 112165/225000 [234:51:29<131:57:10,  4.21s/it]
 50%|████▉     | 112166/225000 [234:51:32<118:17:43,  3.77s/it]
 50%|████▉     | 112167/225000 [234:51:35<112:58:03,  3.60s/it]
 50%|████▉     | 112168/225000 [234:51:49<210:26:38,  6.71s/it]
 50%|████▉     | 112169/225000 [234:51:52<171:38:16,  5.48s/it]
 50%|████▉     | 112170/225000 [234:51:54<146:08:04,  4.66s/it]
 50%|████▉     | 112171/225000 [234:51:58<133:12:14,  4.25s/it]
 50%|████▉     | 112172/225000 [234:52:00<115:02:08,  3.67s/it]
 50%|████▉     | 112173/225000 [234:52:02<102:10:14,  3.26s/it]
 50%|████▉     | 112174/225000 [234:52:05<95:08:46,  3.04s/it] 
 50%|████▉     | 112175/225000 [234:52:08<92:06:27,  2.94s/it]
 50%|████▉     | 112176/225000 [234:52:11<93:49:34,  2.99s/it]
 50%|████▉     | 112177/225000 [234:52:14<96:46:00,  3.09s/it]
 50%|████▉     | 112178/225000 [234:52:17<96:21:26,  3.07s/it]
 50%|████▉     | 112179/225000 [234:52:27<165:08:31,  5.27s/it]
 50%|████▉     | 112180/225000 [234:52:30<143:05:55,  4.57s/it]
 50%|████▉     | 112181/225000 [234:52:33<126:33:08,  4.04s/it]
 50%|████▉     | 112182/225000 [234:52:36<112:30:14,  3.59s/it]
 50%|████▉     | 112183/225000 [234:52:39<108:35:17,  3.47s/it]
 50%|████▉     | 112184/225000 [234:52:42<108:59:37,  3.48s/it]
 50%|████▉     | 112185/225000 [234:52:46<108:50:35,  3.47s/it]
 50%|████▉     | 112186/225000 [234:52:50<117:41:27,  3.76s/it]
 50%|████▉     | 112187/225000 [234:53:03<206:30:29,  6.59s/it]
 50%|████▉     | 112188/225000 [234:53:06<171:53:03,  5.49s/it]
 50%|████▉     | 112189/225000 [234:53:11<162:45:53,  5.19s/it]
 50%|████▉     | 112190/225000 [234:53:13<137:36:34,  4.39s/it]
 50%|████▉     | 112191/225000 [234:53:22<175:16:58,  5.59s/it]
 50%|████▉     | 112192/225000 [234:53:27<167:29:13,  5.34s/it]
 50%|████▉     | 112193/225000 [234:53:30<146:55:46,  4.69s/it]
 50%|████▉     | 112194/225000 [234:53:32<126:05:57,  4.02s/it]
 50%|████▉     | 112195/225000 [234:53:35<116:39:14,  3.72s/it]
 50%|████▉     | 112196/225000 [234:53:38<109:18:54,  3.49s/it]
 50%|████▉     | 112197/225000 [234:53:41<104:44:31,  3.34s/it]
 50%|████▉     | 112198/225000 [234:53:50<159:59:08,  5.11s/it]
 50%|████▉     | 112199/225000 [234:53:53<140:10:37,  4.47s/it]
 50%|████▉     | 112200/225000 [234:53:56<124:36:56,  3.98s/it]
                                                               
{'loss': '0.8999', 'grad_norm': '2.706', 'learning_rate': '2.695e-05', 'epoch': '0.4987', 'num_input_tokens_seen': 392826536, 'train_runtime': '8.456e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112200/225000 [234:53:56<124:36:56,  3.98s/it]
 50%|████▉     | 112201/225000 [234:53:59<110:41:49,  3.53s/it]
 50%|████▉     | 112202/225000 [234:54:02<107:50:42,  3.44s/it]
 50%|████▉     | 112203/225000 [234:54:11<160:41:39,  5.13s/it]
 50%|████▉     | 112204/225000 [234:54:14<140:30:10,  4.48s/it]
 50%|████▉     | 112205/225000 [234:54:16<121:50:50,  3.89s/it]
 50%|████▉     | 112206/225000 [234:54:20<114:46:27,  3.66s/it]
 50%|████▉     | 112207/225000 [234:54:23<113:01:16,  3.61s/it]
 50%|████▉     | 112208/225000 [234:54:35<187:55:30,  6.00s/it]
 50%|████▉     | 112209/225000 [234:54:41<190:44:20,  6.09s/it]
 50%|████▉     | 112210/225000 [234:54:43<156:16:04,  4.99s/it]
 50%|████▉     | 112211/225000 [234:54:47<147:02:07,  4.69s/it]
 50%|████▉     | 112212/225000 [234:54:50<127:39:20,  4.07s/it]
 50%|████▉     | 112213/225000 [234:54:52<112:02:16,  3.58s/it]
 50%|████▉     | 112214/225000 [234:54:55<104:14:01,  3.33s/it]
 50%|████▉     | 112215/225000 [234:54:58<98:31:32,  3.14s/it] 
 50%|████▉     | 112216/225000 [234:55:00<92:13:34,  2.94s/it]
 50%|████▉     | 112217/225000 [234:55:04<96:18:58,  3.07s/it]
 50%|████▉     | 112218/225000 [234:55:07<93:26:46,  2.98s/it]
 50%|████▉     | 112219/225000 [234:55:10<94:08:34,  3.01s/it]
 50%|████▉     | 112220/225000 [234:55:12<92:07:43,  2.94s/it]
 50%|████▉     | 112221/225000 [234:55:22<158:02:17,  5.04s/it]
 50%|████▉     | 112222/225000 [234:55:25<139:17:44,  4.45s/it]
 50%|████▉     | 112223/225000 [234:55:28<126:16:41,  4.03s/it]
 50%|████▉     | 112224/225000 [234:55:32<117:32:15,  3.75s/it]
 50%|████▉     | 112225/225000 [234:55:34<108:24:01,  3.46s/it]
 50%|████▉     | 112226/225000 [234:55:38<109:50:44,  3.51s/it]
 50%|████▉     | 112227/225000 [234:55:41<108:51:14,  3.47s/it]
 50%|████▉     | 112228/225000 [234:55:48<138:29:43,  4.42s/it]
 50%|████▉     | 112229/225000 [234:55:54<153:11:29,  4.89s/it]
 50%|████▉     | 112230/225000 [234:55:57<133:12:18,  4.25s/it]
 50%|████▉     | 112231/225000 [234:56:03<150:13:43,  4.80s/it]
 50%|████▉     | 112232/225000 [234:56:05<129:41:07,  4.14s/it]
 50%|████▉     | 112233/225000 [234:56:11<142:23:05,  4.55s/it]
 50%|████▉     | 112234/225000 [234:56:16<146:20:22,  4.67s/it]
 50%|████▉     | 112235/225000 [234:56:23<167:41:35,  5.35s/it]
 50%|████▉     | 112236/225000 [234:56:26<143:13:51,  4.57s/it]
 50%|████▉     | 112237/225000 [234:56:28<126:03:10,  4.02s/it]
 50%|████▉     | 112238/225000 [234:56:32<119:02:23,  3.80s/it]
 50%|████▉     | 112239/225000 [234:56:35<113:20:23,  3.62s/it]
 50%|████▉     | 112240/225000 [234:56:47<197:00:44,  6.29s/it]
 50%|████▉     | 112241/225000 [234:56:50<159:02:36,  5.08s/it]
 50%|████▉     | 112242/225000 [234:56:55<158:05:46,  5.05s/it]
 50%|████▉     | 112243/225000 [234:56:57<138:34:22,  4.42s/it]
 50%|████▉     | 112244/225000 [234:57:03<144:29:23,  4.61s/it]
 50%|████▉     | 112245/225000 [234:57:09<164:43:46,  5.26s/it]
 50%|████▉     | 112246/225000 [234:57:14<158:02:41,  5.05s/it]
 50%|████▉     | 112247/225000 [234:57:17<144:07:27,  4.60s/it]
 50%|████▉     | 112248/225000 [234:57:21<134:30:12,  4.29s/it]
 50%|████▉     | 112249/225000 [234:57:24<121:30:15,  3.88s/it]
 50%|████▉     | 112250/225000 [234:57:27<115:46:40,  3.70s/it]
                                                               
{'loss': '0.9312', 'grad_norm': '3.521', 'learning_rate': '2.694e-05', 'epoch': '0.4989', 'num_input_tokens_seen': 392911032, 'train_runtime': '8.458e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112250/225000 [234:57:27<115:46:40,  3.70s/it]
 50%|████▉     | 112251/225000 [234:57:31<118:45:53,  3.79s/it]
 50%|████▉     | 112252/225000 [234:57:35<122:46:51,  3.92s/it]
 50%|████▉     | 112253/225000 [234:57:46<184:39:30,  5.90s/it]
 50%|████▉     | 112254/225000 [234:57:48<152:07:34,  4.86s/it]
 50%|████▉     | 112255/225000 [234:57:51<129:27:43,  4.13s/it]
 50%|████▉     | 112256/225000 [234:57:54<122:57:32,  3.93s/it]
 50%|████▉     | 112257/225000 [234:57:57<112:06:28,  3.58s/it]
 50%|████▉     | 112258/225000 [234:58:00<107:55:14,  3.45s/it]
 50%|████▉     | 112259/225000 [234:58:09<161:30:48,  5.16s/it]
 50%|████▉     | 112260/225000 [234:58:12<140:53:32,  4.50s/it]
 50%|████▉     | 112261/225000 [234:58:15<126:25:13,  4.04s/it]
 50%|████▉     | 112262/225000 [234:58:18<116:16:42,  3.71s/it]
 50%|████▉     | 112263/225000 [234:58:21<104:44:58,  3.34s/it]
 50%|████▉     | 112264/225000 [234:58:24<103:45:38,  3.31s/it]
 50%|████▉     | 112265/225000 [234:58:27<102:12:06,  3.26s/it]
 50%|████▉     | 112266/225000 [234:58:30<99:33:06,  3.18s/it] 
 50%|████▉     | 112267/225000 [234:58:41<170:01:14,  5.43s/it]
 50%|████▉     | 112268/225000 [234:58:44<147:39:38,  4.72s/it]
 50%|████▉     | 112269/225000 [234:58:50<160:03:37,  5.11s/it]
 50%|████▉     | 112270/225000 [234:58:53<146:40:31,  4.68s/it]
 50%|████▉     | 112271/225000 [234:59:02<179:10:40,  5.72s/it]
 50%|████▉     | 112272/225000 [234:59:04<146:32:31,  4.68s/it]
 50%|████▉     | 112273/225000 [234:59:09<152:28:57,  4.87s/it]
 50%|████▉     | 112274/225000 [234:59:17<176:40:43,  5.64s/it]
 50%|████▉     | 112275/225000 [234:59:19<147:51:05,  4.72s/it]
 50%|████▉     | 112276/225000 [234:59:28<185:48:10,  5.93s/it]
 50%|████▉     | 112277/225000 [234:59:30<153:16:46,  4.90s/it]
 50%|████▉     | 112278/225000 [234:59:36<157:53:21,  5.04s/it]
 50%|████▉     | 112279/225000 [234:59:39<139:00:29,  4.44s/it]
 50%|████▉     | 112280/225000 [234:59:41<118:43:19,  3.79s/it]
 50%|████▉     | 112281/225000 [234:59:47<134:14:14,  4.29s/it]
 50%|████▉     | 112282/225000 [234:59:53<156:09:56,  4.99s/it]
 50%|████▉     | 112283/225000 [234:59:56<134:48:38,  4.31s/it]
 50%|████▉     | 112284/225000 [234:59:59<119:52:32,  3.83s/it]
 50%|████▉     | 112285/225000 [235:00:01<107:24:50,  3.43s/it]
 50%|████▉     | 112286/225000 [235:00:14<194:05:36,  6.20s/it]
 50%|████▉     | 112287/225000 [235:00:27<258:31:38,  8.26s/it]
 50%|████▉     | 112288/225000 [235:00:29<203:43:16,  6.51s/it]
 50%|████▉     | 112289/225000 [235:00:35<195:53:21,  6.26s/it]
 50%|████▉     | 112290/225000 [235:00:37<158:50:43,  5.07s/it]
 50%|████▉     | 112291/225000 [235:00:41<143:56:44,  4.60s/it]
 50%|████▉     | 112292/225000 [235:00:52<205:37:56,  6.57s/it]
 50%|████▉     | 112293/225000 [235:00:54<164:36:29,  5.26s/it]
 50%|████▉     | 112294/225000 [235:00:57<140:06:17,  4.48s/it]
 50%|████▉     | 112295/225000 [235:01:03<158:46:44,  5.07s/it]
 50%|████▉     | 112296/225000 [235:01:09<168:58:23,  5.40s/it]
 50%|████▉     | 112297/225000 [235:01:12<142:33:46,  4.55s/it]
 50%|████▉     | 112298/225000 [235:01:17<145:13:52,  4.64s/it]
 50%|████▉     | 112299/225000 [235:01:24<164:43:30,  5.26s/it]
 50%|████▉     | 112300/225000 [235:01:27<144:09:37,  4.60s/it]
                                                               
{'loss': '0.9397', 'grad_norm': '2.693', 'learning_rate': '2.692e-05', 'epoch': '0.4991', 'num_input_tokens_seen': 393006784, 'train_runtime': '8.461e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112300/225000 [235:01:27<144:09:37,  4.60s/it]
 50%|████▉     | 112301/225000 [235:01:30<128:08:18,  4.09s/it]
 50%|████▉     | 112302/225000 [235:01:38<168:08:56,  5.37s/it]
 50%|████▉     | 112303/225000 [235:01:47<200:06:22,  6.39s/it]
 50%|████▉     | 112304/225000 [235:01:49<163:59:40,  5.24s/it]
 50%|████▉     | 112305/225000 [235:01:53<147:13:56,  4.70s/it]
 50%|████▉     | 112306/225000 [235:01:56<130:17:09,  4.16s/it]
 50%|████▉     | 112307/225000 [235:01:59<119:34:39,  3.82s/it]
 50%|████▉     | 112308/225000 [235:02:07<162:08:26,  5.18s/it]
 50%|████▉     | 112309/225000 [235:02:10<137:55:37,  4.41s/it]
 50%|████▉     | 112310/225000 [235:02:12<118:57:45,  3.80s/it]
 50%|████▉     | 112311/225000 [235:02:14<106:24:21,  3.40s/it]
 50%|████▉     | 112312/225000 [235:02:17<97:08:59,  3.10s/it] 
 50%|████▉     | 112313/225000 [235:02:20<93:40:01,  2.99s/it]
 50%|████▉     | 112314/225000 [235:02:27<132:37:41,  4.24s/it]
 50%|████▉     | 112315/225000 [235:02:30<126:22:07,  4.04s/it]
 50%|████▉     | 112316/225000 [235:02:38<164:39:37,  5.26s/it]
 50%|████▉     | 112317/225000 [235:02:46<187:51:25,  6.00s/it]
 50%|████▉     | 112318/225000 [235:02:52<184:49:54,  5.91s/it]
 50%|████▉     | 112319/225000 [235:02:54<151:59:48,  4.86s/it]
 50%|████▉     | 112320/225000 [235:03:05<206:17:24,  6.59s/it]
 50%|████▉     | 112321/225000 [235:03:09<181:39:25,  5.80s/it]
 50%|████▉     | 112322/225000 [235:03:11<148:48:17,  4.75s/it]
 50%|████▉     | 112323/225000 [235:03:17<157:46:52,  5.04s/it]
 50%|████▉     | 112324/225000 [235:03:23<166:06:44,  5.31s/it]
 50%|████▉     | 112325/225000 [235:03:25<141:04:40,  4.51s/it]
 50%|████▉     | 112326/225000 [235:03:28<125:53:30,  4.02s/it]
 50%|████▉     | 112327/225000 [235:03:34<137:40:19,  4.40s/it]
 50%|████▉     | 112328/225000 [235:03:39<147:55:25,  4.73s/it]
 50%|████▉     | 112329/225000 [235:03:42<129:40:48,  4.14s/it]
 50%|████▉     | 112330/225000 [235:03:54<206:27:56,  6.60s/it]
 50%|████▉     | 112331/225000 [235:03:57<167:12:29,  5.34s/it]
 50%|████▉     | 112332/225000 [235:03:59<143:11:12,  4.58s/it]
 50%|████▉     | 112333/225000 [235:04:02<124:50:57,  3.99s/it]
 50%|████▉     | 112334/225000 [235:04:05<118:47:54,  3.80s/it]
 50%|████▉     | 112335/225000 [235:04:08<111:22:00,  3.56s/it]
 50%|████▉     | 112336/225000 [235:04:16<151:39:30,  4.85s/it]
 50%|████▉     | 112337/225000 [235:04:19<129:18:23,  4.13s/it]
 50%|████▉     | 112338/225000 [235:04:21<115:40:33,  3.70s/it]
 50%|████▉     | 112339/225000 [235:04:24<108:52:21,  3.48s/it]
 50%|████▉     | 112340/225000 [235:04:32<144:34:37,  4.62s/it]
 50%|████▉     | 112341/225000 [235:04:35<133:26:39,  4.26s/it]
 50%|████▉     | 112342/225000 [235:04:38<123:54:17,  3.96s/it]
 50%|████▉     | 112343/225000 [235:04:44<136:38:15,  4.37s/it]
 50%|████▉     | 112344/225000 [235:04:47<126:06:43,  4.03s/it]
 50%|████▉     | 112345/225000 [235:04:51<126:21:34,  4.04s/it]
 50%|████▉     | 112346/225000 [235:05:00<177:49:26,  5.68s/it]
 50%|████▉     | 112347/225000 [235:05:04<155:40:02,  4.97s/it]
 50%|████▉     | 112348/225000 [235:05:07<138:59:56,  4.44s/it]
 50%|████▉     | 112349/225000 [235:05:10<126:42:38,  4.05s/it]
 50%|████▉     | 112350/225000 [235:05:13<116:04:11,  3.71s/it]
                                                               
{'loss': '0.9504', 'grad_norm': '4.073', 'learning_rate': '2.69e-05', 'epoch': '0.4993', 'num_input_tokens_seen': 393103872, 'train_runtime': '8.463e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112350/225000 [235:05:13<116:04:11,  3.71s/it]
 50%|████▉     | 112351/225000 [235:05:15<104:01:18,  3.32s/it]
 50%|████▉     | 112352/225000 [235:05:19<102:52:58,  3.29s/it]
 50%|████▉     | 112353/225000 [235:05:23<118:28:45,  3.79s/it]
 50%|████▉     | 112354/225000 [235:05:32<160:13:03,  5.12s/it]
 50%|████▉     | 112355/225000 [235:05:39<184:58:12,  5.91s/it]
 50%|████▉     | 112356/225000 [235:05:43<161:23:50,  5.16s/it]
 50%|████▉     | 112357/225000 [235:05:53<208:09:44,  6.65s/it]
 50%|████▉     | 112358/225000 [235:05:55<166:31:58,  5.32s/it]
 50%|████▉     | 112359/225000 [235:05:58<138:25:33,  4.42s/it]
 50%|████▉     | 112360/225000 [235:06:00<119:13:57,  3.81s/it]
 50%|████▉     | 112361/225000 [235:06:03<113:21:32,  3.62s/it]
 50%|████▉     | 112362/225000 [235:06:06<107:21:58,  3.43s/it]
 50%|████▉     | 112363/225000 [235:06:09<100:15:42,  3.20s/it]
 50%|████▉     | 112364/225000 [235:06:15<129:53:57,  4.15s/it]
 50%|████▉     | 112365/225000 [235:06:24<173:06:33,  5.53s/it]
 50%|████▉     | 112366/225000 [235:06:28<164:01:20,  5.24s/it]
 50%|████▉     | 112367/225000 [235:06:32<145:22:21,  4.65s/it]
 50%|████▉     | 112368/225000 [235:06:34<127:36:57,  4.08s/it]
 50%|████▉     | 112369/225000 [235:06:41<146:34:49,  4.69s/it]
 50%|████▉     | 112370/225000 [235:06:44<134:31:04,  4.30s/it]
 50%|████▉     | 112371/225000 [235:06:46<117:12:41,  3.75s/it]
 50%|████▉     | 112372/225000 [235:06:49<105:15:15,  3.36s/it]
 50%|████▉     | 112373/225000 [235:06:52<98:30:01,  3.15s/it] 
 50%|████▉     | 112374/225000 [235:07:00<146:22:47,  4.68s/it]
 50%|████▉     | 112375/225000 [235:07:05<153:29:18,  4.91s/it]
 50%|████▉     | 112376/225000 [235:07:08<130:53:19,  4.18s/it]
 50%|████▉     | 112377/225000 [235:07:11<117:36:16,  3.76s/it]
 50%|████▉     | 112378/225000 [235:07:13<105:18:47,  3.37s/it]
 50%|████▉     | 112379/225000 [235:07:27<204:00:42,  6.52s/it]
 50%|████▉     | 112380/225000 [235:07:29<163:13:30,  5.22s/it]
 50%|████▉     | 112381/225000 [235:07:34<163:04:07,  5.21s/it]
 50%|████▉     | 112382/225000 [235:07:37<140:24:17,  4.49s/it]
 50%|████▉     | 112383/225000 [235:07:40<125:05:20,  4.00s/it]
 50%|████▉     | 112384/225000 [235:07:43<120:46:31,  3.86s/it]
 50%|████▉     | 112385/225000 [235:07:47<117:13:57,  3.75s/it]
 50%|████▉     | 112386/225000 [235:07:50<114:08:43,  3.65s/it]
 50%|████▉     | 112387/225000 [235:07:55<119:05:38,  3.81s/it]
 50%|████▉     | 112388/225000 [235:08:03<163:03:41,  5.21s/it]
 50%|████▉     | 112389/225000 [235:08:08<162:23:44,  5.19s/it]
 50%|████▉     | 112390/225000 [235:08:12<148:52:01,  4.76s/it]
 50%|████▉     | 112391/225000 [235:08:22<196:36:18,  6.29s/it]
 50%|████▉     | 112392/225000 [235:08:25<166:13:21,  5.31s/it]
 50%|████▉     | 112393/225000 [235:08:28<145:17:45,  4.65s/it]
 50%|████▉     | 112394/225000 [235:08:32<136:44:00,  4.37s/it]
 50%|████▉     | 112395/225000 [235:08:35<131:07:40,  4.19s/it]
 50%|████▉     | 112396/225000 [235:08:39<127:51:51,  4.09s/it]
 50%|████▉     | 112397/225000 [235:08:43<125:08:14,  4.00s/it]
 50%|████▉     | 112398/225000 [235:08:47<121:44:50,  3.89s/it]
 50%|████▉     | 112399/225000 [235:08:58<189:00:12,  6.04s/it]
 50%|████▉     | 112400/225000 [235:09:02<174:47:35,  5.59s/it]
                                                               
{'loss': '0.9389', 'grad_norm': '2.523', 'learning_rate': '2.688e-05', 'epoch': '0.4996', 'num_input_tokens_seen': 393201112, 'train_runtime': '8.465e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112400/225000 [235:09:02<174:47:35,  5.59s/it]
 50%|████▉     | 112401/225000 [235:09:05<146:18:10,  4.68s/it]
 50%|████▉     | 112402/225000 [235:09:08<129:19:06,  4.13s/it]
 50%|████▉     | 112403/225000 [235:09:13<144:24:30,  4.62s/it]
 50%|████▉     | 112404/225000 [235:09:17<133:40:33,  4.27s/it]
 50%|████▉     | 112405/225000 [235:09:27<191:18:33,  6.12s/it]
 50%|████▉     | 112406/225000 [235:09:30<161:54:10,  5.18s/it]
 50%|████▉     | 112407/225000 [235:09:33<142:54:40,  4.57s/it]
 50%|████▉     | 112408/225000 [235:09:36<121:26:25,  3.88s/it]
 50%|████▉     | 112409/225000 [235:09:42<147:20:38,  4.71s/it]
 50%|████▉     | 112410/225000 [235:09:45<131:44:12,  4.21s/it]
 50%|████▉     | 112411/225000 [235:09:49<127:26:24,  4.07s/it]
 50%|████▉     | 112412/225000 [235:09:52<119:40:04,  3.83s/it]
 50%|████▉     | 112413/225000 [235:10:02<178:26:00,  5.71s/it]
 50%|████▉     | 112414/225000 [235:10:05<146:33:08,  4.69s/it]
 50%|████▉     | 112415/225000 [235:10:07<126:57:37,  4.06s/it]
 50%|████▉     | 112416/225000 [235:10:13<137:45:53,  4.41s/it]
 50%|████▉     | 112417/225000 [235:10:18<151:30:07,  4.84s/it]
 50%|████▉     | 112418/225000 [235:10:21<128:07:57,  4.10s/it]
 50%|████▉     | 112419/225000 [235:10:25<125:29:33,  4.01s/it]
 50%|████▉     | 112420/225000 [235:10:33<168:30:38,  5.39s/it]
 50%|████▉     | 112421/225000 [235:10:42<199:13:07,  6.37s/it]
 50%|████▉     | 112422/225000 [235:10:44<161:39:12,  5.17s/it]
 50%|████▉     | 112423/225000 [235:10:47<136:33:18,  4.37s/it]
 50%|████▉     | 112424/225000 [235:10:52<148:56:59,  4.76s/it]
 50%|████▉     | 112425/225000 [235:10:57<146:41:40,  4.69s/it]
 50%|████▉     | 112426/225000 [235:11:03<155:26:39,  4.97s/it]
 50%|████▉     | 112427/225000 [235:11:05<135:24:37,  4.33s/it]
 50%|████▉     | 112428/225000 [235:11:10<134:49:23,  4.31s/it]
 50%|████▉     | 112429/225000 [235:11:13<125:56:58,  4.03s/it]
 50%|████▉     | 112430/225000 [235:11:22<168:59:37,  5.40s/it]
 50%|████▉     | 112431/225000 [235:11:25<151:28:20,  4.84s/it]
 50%|████▉     | 112432/225000 [235:11:30<152:33:07,  4.88s/it]
 50%|████▉     | 112433/225000 [235:11:37<169:18:52,  5.41s/it]
 50%|████▉     | 112434/225000 [235:11:40<145:29:59,  4.65s/it]
 50%|████▉     | 112435/225000 [235:11:43<131:05:25,  4.19s/it]
 50%|████▉     | 112436/225000 [235:11:46<122:26:52,  3.92s/it]
 50%|████▉     | 112437/225000 [235:11:56<182:51:26,  5.85s/it]
 50%|████▉     | 112438/225000 [235:12:00<164:02:13,  5.25s/it]
 50%|████▉     | 112439/225000 [235:12:13<230:35:16,  7.37s/it]
 50%|████▉     | 112440/225000 [235:12:16<193:14:49,  6.18s/it]
 50%|████▉     | 112441/225000 [235:12:24<209:22:29,  6.70s/it]
 50%|████▉     | 112442/225000 [235:12:36<259:17:34,  8.29s/it]
 50%|████▉     | 112443/225000 [235:12:40<220:28:38,  7.05s/it]
 50%|████▉     | 112444/225000 [235:12:45<198:54:14,  6.36s/it]
 50%|████▉     | 112445/225000 [235:12:47<161:24:58,  5.16s/it]
 50%|████▉     | 112446/225000 [235:12:57<207:55:29,  6.65s/it]
 50%|████▉     | 112447/225000 [235:13:00<173:03:17,  5.54s/it]
 50%|████▉     | 112448/225000 [235:13:03<149:11:55,  4.77s/it]
 50%|████▉     | 112449/225000 [235:13:08<146:53:54,  4.70s/it]
 50%|████▉     | 112450/225000 [235:13:11<133:42:18,  4.28s/it]
                                                               
{'loss': '0.9335', 'grad_norm': '1.925', 'learning_rate': '2.686e-05', 'epoch': '0.4998', 'num_input_tokens_seen': 393299384, 'train_runtime': '8.468e+05', 'train_tokens_per_second': '464.5'}

 50%|████▉     | 112450/225000 [235:13:11<133:42:18,  4.28s/it]
 50%|████▉     | 112451/225000 [235:13:15<125:53:57,  4.03s/it]
 50%|████▉     | 112452/225000 [235:13:17<113:36:42,  3.63s/it]
 50%|████▉     | 112453/225000 [235:13:21<111:26:15,  3.56s/it]
 50%|████▉     | 112454/225000 [235:13:24<111:17:01,  3.56s/it]
 50%|████▉     | 112455/225000 [235:13:28<115:30:56,  3.70s/it]
 50%|████▉     | 112456/225000 [235:13:31<110:34:35,  3.54s/it]
 50%|████▉     | 112457/225000 [235:13:35<108:28:00,  3.47s/it]
 50%|████▉     | 112458/225000 [235:13:38<104:26:00,  3.34s/it]
 50%|████▉     | 112459/225000 [235:13:41<101:39:44,  3.25s/it]
 50%|████▉     | 112460/225000 [235:13:44<100:38:13,  3.22s/it]
 50%|████▉     | 112461/225000 [235:13:47<96:37:16,  3.09s/it] 
 50%|████▉     | 112462/225000 [235:13:57<167:57:06,  5.37s/it]
 50%|████▉     | 112463/225000 [235:14:06<199:22:02,  6.38s/it]
 50%|████▉     | 112464/225000 [235:14:11<188:03:09,  6.02s/it]
 50%|████▉     | 112465/225000 [235:14:14<156:11:46,  5.00s/it]
 50%|████▉     | 112466/225000 [235:14:16<133:10:06,  4.26s/it]
 50%|████▉     | 112467/225000 [235:14:20<128:38:43,  4.12s/it]
 50%|████▉     | 112468/225000 [235:14:23<112:18:31,  3.59s/it]
 50%|████▉     | 112469/225000 [235:14:28<131:36:52,  4.21s/it]
 50%|████▉     | 112470/225000 [235:14:32<129:30:46,  4.14s/it]
 50%|████▉     | 112471/225000 [235:14:42<184:36:31,  5.91s/it]
 50%|████▉     | 112472/225000 [235:14:45<155:03:15,  4.96s/it]
 50%|████▉     | 112473/225000 [235:14:48<132:30:23,  4.24s/it]
 50%|████▉     | 112474/225000 [235:14:53<145:45:19,  4.66s/it]
 50%|████▉     | 112475/225000 [235:14:57<138:36:38,  4.43s/it]
 50%|████▉     | 112476/225000 [235:15:03<155:43:14,  4.98s/it]
 50%|████▉     | 112477/225000 [235:15:07<138:29:43,  4.43s/it]
 50%|████▉     | 112478/225000 [235:15:09<122:51:20,  3.93s/it]
 50%|████▉     | 112479/225000 [235:15:13<120:24:29,  3.85s/it]
 50%|████▉     | 112480/225000 [235:15:17<119:07:39,  3.81s/it]
 50%|████▉     | 112481/225000 [235:15:21<121:02:18,  3.87s/it]
 50%|████▉     | 112482/225000 [235:15:24<119:58:32,  3.84s/it]
 50%|████▉     | 112483/225000 [235:15:28<121:00:32,  3.87s/it]
 50%|████▉     | 112484/225000 [235:15:32<119:08:19,  3.81s/it]
 50%|████▉     | 112485/225000 [235:15:36<121:38:26,  3.89s/it]
 50%|████▉     | 112486/225000 [235:15:39<111:29:36,  3.57s/it]
 50%|████▉     | 112487/225000 [235:15:42<110:39:22,  3.54s/it]
 50%|████▉     | 112488/225000 [235:15:46<109:31:18,  3.50s/it]
 50%|████▉     | 112489/225000 [235:15:49<109:01:07,  3.49s/it]
 50%|████▉     | 112490/225000 [235:15:53<114:17:26,  3.66s/it]
 50%|████▉     | 112491/225000 [235:16:01<149:01:19,  4.77s/it]
 50%|████▉     | 112492/225000 [235:16:13<223:15:32,  7.14s/it]
 50%|████▉     | 112493/225000 [235:16:19<204:51:03,  6.55s/it]
 50%|████▉     | 112494/225000 [235:16:21<167:30:39,  5.36s/it]
 50%|████▉     | 112495/225000 [235:16:25<149:08:38,  4.77s/it]
 50%|████▉     | 112496/225000 [235:16:28<134:34:52,  4.31s/it]
 50%|████▉     | 112497/225000 [235:16:39<198:11:31,  6.34s/it]
 50%|████▉     | 112498/225000 [235:16:42<164:59:52,  5.28s/it]
 50%|████▉     | 112499/225000 [235:16:45<142:37:57,  4.56s/it]
 50%|█████     | 112500/225000 [235:16:53<181:01:42,  5.79s/it]
                                                               
{'loss': '0.9366', 'grad_norm': '1.981', 'learning_rate': '2.684e-05', 'epoch': '0.5', 'num_input_tokens_seen': 393393144, 'train_runtime': '8.47e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112500/225000 [235:16:53<181:01:42,  5.79s/it]
 50%|█████     | 112501/225000 [235:16:56<156:31:35,  5.01s/it]
 50%|█████     | 112502/225000 [235:16:59<135:19:49,  4.33s/it]
 50%|█████     | 112503/225000 [235:17:02<117:58:02,  3.78s/it]
 50%|█████     | 112504/225000 [235:17:05<109:39:29,  3.51s/it]
 50%|█████     | 112505/225000 [235:17:08<107:24:07,  3.44s/it]
 50%|█████     | 112506/225000 [235:17:13<125:50:33,  4.03s/it]
 50%|█████     | 112507/225000 [235:17:26<205:26:53,  6.57s/it]
 50%|█████     | 112508/225000 [235:17:28<165:12:47,  5.29s/it]
 50%|█████     | 112509/225000 [235:17:31<147:51:35,  4.73s/it]
 50%|█████     | 112510/225000 [235:17:35<138:36:32,  4.44s/it]
 50%|█████     | 112511/225000 [235:17:39<134:34:56,  4.31s/it]
 50%|█████     | 112512/225000 [235:17:43<131:22:24,  4.20s/it]
 50%|█████     | 112513/225000 [235:17:53<185:38:37,  5.94s/it]
 50%|█████     | 112514/225000 [235:17:56<158:25:26,  5.07s/it]
 50%|█████     | 112515/225000 [235:17:59<139:42:55,  4.47s/it]
 50%|█████     | 112516/225000 [235:18:09<187:44:37,  6.01s/it]
 50%|█████     | 112517/225000 [235:18:18<220:50:35,  7.07s/it]
 50%|█████     | 112518/225000 [235:18:21<178:34:15,  5.72s/it]
 50%|█████     | 112519/225000 [235:18:29<201:08:16,  6.44s/it]
 50%|█████     | 112520/225000 [235:18:38<225:49:54,  7.23s/it]
 50%|█████     | 112521/225000 [235:18:43<199:27:37,  6.38s/it]
 50%|█████     | 112522/225000 [235:18:45<160:05:55,  5.12s/it]
 50%|█████     | 112523/225000 [235:18:50<165:11:07,  5.29s/it]
 50%|█████     | 112524/225000 [235:18:54<152:57:16,  4.90s/it]
 50%|█████     | 112525/225000 [235:19:03<186:08:06,  5.96s/it]
 50%|█████     | 112526/225000 [235:19:06<155:30:18,  4.98s/it]
 50%|█████     | 112527/225000 [235:19:11<158:13:12,  5.06s/it]
 50%|█████     | 112528/225000 [235:19:13<133:03:07,  4.26s/it]
 50%|█████     | 112529/225000 [235:19:16<116:16:03,  3.72s/it]
 50%|█████     | 112530/225000 [235:19:19<108:56:39,  3.49s/it]
 50%|█████     | 112531/225000 [235:19:27<152:23:23,  4.88s/it]
 50%|█████     | 112532/225000 [235:19:30<136:30:49,  4.37s/it]
 50%|█████     | 112533/225000 [235:19:34<133:12:50,  4.26s/it]
 50%|█████     | 112534/225000 [235:19:43<179:29:15,  5.75s/it]
 50%|█████     | 112535/225000 [235:19:46<150:46:02,  4.83s/it]
 50%|█████     | 112536/225000 [235:19:51<157:39:17,  5.05s/it]
 50%|█████     | 112537/225000 [235:19:55<139:40:30,  4.47s/it]
 50%|█████     | 112538/225000 [235:19:57<124:34:52,  3.99s/it]
 50%|█████     | 112539/225000 [235:20:00<111:24:54,  3.57s/it]
 50%|█████     | 112540/225000 [235:20:03<105:51:37,  3.39s/it]
 50%|█████     | 112541/225000 [235:20:06<101:38:49,  3.25s/it]
 50%|█████     | 112542/225000 [235:20:14<146:06:43,  4.68s/it]
 50%|█████     | 112543/225000 [235:20:18<142:43:16,  4.57s/it]
 50%|█████     | 112544/225000 [235:20:22<139:34:38,  4.47s/it]
 50%|█████     | 112545/225000 [235:20:26<128:49:45,  4.12s/it]
 50%|█████     | 112546/225000 [235:20:29<118:31:45,  3.79s/it]
 50%|█████     | 112547/225000 [235:20:32<110:48:56,  3.55s/it]
 50%|█████     | 112548/225000 [235:20:36<112:58:15,  3.62s/it]
 50%|█████     | 112549/225000 [235:20:38<105:21:01,  3.37s/it]
 50%|█████     | 112550/225000 [235:20:48<167:07:21,  5.35s/it]
                                                               
{'loss': '0.9532', 'grad_norm': '2.59', 'learning_rate': '2.683e-05', 'epoch': '0.5002', 'num_input_tokens_seen': 393490344, 'train_runtime': '8.472e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112550/225000 [235:20:48<167:07:21,  5.35s/it]
 50%|█████     | 112551/225000 [235:20:55<179:42:48,  5.75s/it]
 50%|█████     | 112552/225000 [235:20:59<165:20:03,  5.29s/it]
 50%|█████     | 112553/225000 [235:21:02<142:15:22,  4.55s/it]
 50%|█████     | 112554/225000 [235:21:05<132:00:47,  4.23s/it]
 50%|█████     | 112555/225000 [235:21:09<128:04:55,  4.10s/it]
 50%|█████     | 112556/225000 [235:21:12<119:12:03,  3.82s/it]
 50%|█████     | 112557/225000 [235:21:16<116:41:24,  3.74s/it]
 50%|█████     | 112558/225000 [235:21:27<184:23:42,  5.90s/it]
 50%|█████     | 112559/225000 [235:21:29<151:21:12,  4.85s/it]
 50%|█████     | 112560/225000 [235:21:32<135:18:37,  4.33s/it]
 50%|█████     | 112561/225000 [235:21:35<119:04:12,  3.81s/it]
 50%|█████     | 112562/225000 [235:21:38<114:31:33,  3.67s/it]
 50%|█████     | 112563/225000 [235:21:42<111:24:52,  3.57s/it]
 50%|█████     | 112564/225000 [235:21:45<112:13:18,  3.59s/it]
 50%|█████     | 112565/225000 [235:21:49<109:29:01,  3.51s/it]
 50%|█████     | 112566/225000 [235:21:53<115:24:28,  3.70s/it]
 50%|█████     | 112567/225000 [235:22:01<157:56:01,  5.06s/it]
 50%|█████     | 112568/225000 [235:22:07<164:13:31,  5.26s/it]
 50%|█████     | 112569/225000 [235:22:12<164:19:38,  5.26s/it]
 50%|█████     | 112570/225000 [235:22:15<144:06:05,  4.61s/it]
 50%|█████     | 112571/225000 [235:22:20<146:15:15,  4.68s/it]
 50%|█████     | 112572/225000 [235:22:23<125:55:17,  4.03s/it]
 50%|█████     | 112573/225000 [235:22:25<112:10:50,  3.59s/it]
 50%|█████     | 112574/225000 [235:22:30<128:12:23,  4.11s/it]
 50%|█████     | 112575/225000 [235:22:34<127:11:25,  4.07s/it]
 50%|█████     | 112576/225000 [235:22:38<125:15:28,  4.01s/it]
 50%|█████     | 112577/225000 [235:22:43<128:51:52,  4.13s/it]
 50%|█████     | 112578/225000 [235:22:47<130:06:19,  4.17s/it]
 50%|█████     | 112579/225000 [235:22:50<120:49:18,  3.87s/it]
 50%|█████     | 112580/225000 [235:22:53<114:48:52,  3.68s/it]
 50%|█████     | 112581/225000 [235:22:57<117:43:44,  3.77s/it]
 50%|█████     | 112582/225000 [235:23:01<112:53:58,  3.62s/it]
 50%|█████     | 112583/225000 [235:23:04<108:33:06,  3.48s/it]
 50%|█████     | 112584/225000 [235:23:07<109:59:35,  3.52s/it]
 50%|█████     | 112585/225000 [235:23:11<110:34:33,  3.54s/it]
 50%|█████     | 112586/225000 [235:23:23<192:26:14,  6.16s/it]
 50%|█████     | 112587/225000 [235:23:29<186:53:38,  5.99s/it]
 50%|█████     | 112588/225000 [235:23:32<163:45:55,  5.24s/it]
 50%|█████     | 112589/225000 [235:23:35<140:09:46,  4.49s/it]
 50%|█████     | 112590/225000 [235:23:44<180:38:16,  5.79s/it]
 50%|█████     | 112591/225000 [235:23:47<154:33:38,  4.95s/it]
 50%|█████     | 112592/225000 [235:23:50<138:30:20,  4.44s/it]
 50%|█████     | 112593/225000 [235:24:00<193:18:58,  6.19s/it]
 50%|█████     | 112594/225000 [235:24:06<186:38:44,  5.98s/it]
 50%|█████     | 112595/225000 [235:24:09<163:55:50,  5.25s/it]
 50%|█████     | 112596/225000 [235:24:15<170:41:10,  5.47s/it]
 50%|█████     | 112597/225000 [235:24:28<233:46:16,  7.49s/it]
 50%|█████     | 112598/225000 [235:24:30<189:07:27,  6.06s/it]
 50%|█████     | 112599/225000 [235:24:40<221:46:02,  7.10s/it]
 50%|█████     | 112600/225000 [235:24:42<177:17:33,  5.68s/it]
                                                               
{'loss': '0.9281', 'grad_norm': '1.773', 'learning_rate': '2.681e-05', 'epoch': '0.5004', 'num_input_tokens_seen': 393590424, 'train_runtime': '8.475e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112600/225000 [235:24:42<177:17:33,  5.68s/it]
 50%|█████     | 112601/225000 [235:24:45<149:42:40,  4.80s/it]
 50%|█████     | 112602/225000 [235:24:47<128:39:06,  4.12s/it]
 50%|█████     | 112603/225000 [235:24:53<139:32:37,  4.47s/it]
 50%|█████     | 112604/225000 [235:24:55<120:38:00,  3.86s/it]
 50%|█████     | 112605/225000 [235:25:06<186:25:05,  5.97s/it]
 50%|█████     | 112606/225000 [235:25:09<154:30:55,  4.95s/it]
 50%|█████     | 112607/225000 [235:25:12<138:17:46,  4.43s/it]
 50%|█████     | 112608/225000 [235:25:14<120:01:33,  3.84s/it]
 50%|█████     | 112609/225000 [235:25:21<147:30:26,  4.72s/it]
 50%|█████     | 112610/225000 [235:25:25<135:34:39,  4.34s/it]
 50%|█████     | 112611/225000 [235:25:28<126:59:12,  4.07s/it]
 50%|█████     | 112612/225000 [235:25:31<119:09:41,  3.82s/it]
 50%|█████     | 112613/225000 [235:25:34<110:44:51,  3.55s/it]
 50%|█████     | 112614/225000 [235:25:38<111:15:39,  3.56s/it]
 50%|█████     | 112615/225000 [235:25:41<105:04:22,  3.37s/it]
 50%|█████     | 112616/225000 [235:25:44<100:47:29,  3.23s/it]
 50%|█████     | 112617/225000 [235:25:52<152:23:02,  4.88s/it]
 50%|█████     | 112618/225000 [235:25:55<128:04:25,  4.10s/it]
 50%|█████     | 112619/225000 [235:25:58<123:57:45,  3.97s/it]
 50%|█████     | 112620/225000 [235:26:10<194:26:07,  6.23s/it]
 50%|█████     | 112621/225000 [235:26:13<163:31:39,  5.24s/it]
 50%|█████     | 112622/225000 [235:26:15<135:44:58,  4.35s/it]
 50%|█████     | 112623/225000 [235:26:17<118:58:34,  3.81s/it]
 50%|█████     | 112624/225000 [235:26:24<144:35:58,  4.63s/it]
 50%|█████     | 112625/225000 [235:26:28<136:10:31,  4.36s/it]
 50%|█████     | 112626/225000 [235:26:31<129:19:21,  4.14s/it]
 50%|█████     | 112627/225000 [235:26:35<124:29:05,  3.99s/it]
 50%|█████     | 112628/225000 [235:26:39<121:16:36,  3.89s/it]
 50%|█████     | 112629/225000 [235:26:50<193:54:11,  6.21s/it]
 50%|█████     | 112630/225000 [235:26:53<156:48:47,  5.02s/it]
 50%|█████     | 112631/225000 [235:26:56<141:48:03,  4.54s/it]
 50%|█████     | 112632/225000 [235:26:59<125:17:30,  4.01s/it]
 50%|█████     | 112633/225000 [235:27:02<117:28:45,  3.76s/it]
 50%|█████     | 112634/225000 [235:27:10<159:01:51,  5.10s/it]
 50%|█████     | 112635/225000 [235:27:14<145:32:50,  4.66s/it]
 50%|█████     | 112636/225000 [235:27:18<142:17:34,  4.56s/it]
 50%|█████     | 112637/225000 [235:27:22<136:51:24,  4.38s/it]
 50%|█████     | 112638/225000 [235:27:27<140:05:28,  4.49s/it]
 50%|█████     | 112639/225000 [235:27:31<139:34:09,  4.47s/it]
 50%|█████     | 112640/225000 [235:27:35<129:25:49,  4.15s/it]
 50%|█████     | 112641/225000 [235:27:38<124:42:01,  4.00s/it]
 50%|█████     | 112642/225000 [235:27:43<134:28:40,  4.31s/it]
 50%|█████     | 112643/225000 [235:27:48<141:12:30,  4.52s/it]
 50%|█████     | 112644/225000 [235:27:52<133:28:24,  4.28s/it]
 50%|█████     | 112645/225000 [235:27:56<130:14:48,  4.17s/it]
 50%|█████     | 112646/225000 [235:27:59<119:43:03,  3.84s/it]
 50%|█████     | 112647/225000 [235:28:02<115:11:43,  3.69s/it]
 50%|█████     | 112648/225000 [235:28:14<189:30:29,  6.07s/it]
 50%|█████     | 112649/225000 [235:28:16<155:14:55,  4.97s/it]
 50%|█████     | 112650/225000 [235:28:23<172:39:57,  5.53s/it]
                                                               
{'loss': '0.9418', 'grad_norm': '1.615', 'learning_rate': '2.679e-05', 'epoch': '0.5007', 'num_input_tokens_seen': 393681520, 'train_runtime': '8.477e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112650/225000 [235:28:23<172:39:57,  5.53s/it]
 50%|█████     | 112651/225000 [235:28:26<150:51:20,  4.83s/it]
 50%|█████     | 112652/225000 [235:28:29<130:57:57,  4.20s/it]
 50%|█████     | 112653/225000 [235:28:32<121:39:48,  3.90s/it]
 50%|█████     | 112654/225000 [235:28:36<117:37:27,  3.77s/it]
 50%|█████     | 112655/225000 [235:28:45<168:51:15,  5.41s/it]
 50%|█████     | 112656/225000 [235:28:50<168:44:30,  5.41s/it]
 50%|█████     | 112657/225000 [235:28:57<181:26:24,  5.81s/it]
 50%|█████     | 112658/225000 [235:29:00<152:44:24,  4.89s/it]
 50%|█████     | 112659/225000 [235:29:04<141:21:55,  4.53s/it]
 50%|█████     | 112660/225000 [235:29:07<130:30:14,  4.18s/it]
 50%|█████     | 112661/225000 [235:29:10<120:50:21,  3.87s/it]
 50%|█████     | 112662/225000 [235:29:14<116:49:58,  3.74s/it]
 50%|█████     | 112663/225000 [235:29:16<106:27:55,  3.41s/it]
 50%|█████     | 112664/225000 [235:29:27<178:59:15,  5.74s/it]
 50%|█████     | 112665/225000 [235:29:35<192:44:51,  6.18s/it]
 50%|█████     | 112666/225000 [235:29:38<163:41:31,  5.25s/it]
 50%|█████     | 112667/225000 [235:29:41<146:47:18,  4.70s/it]
 50%|█████     | 112668/225000 [235:29:50<190:08:29,  6.09s/it]
 50%|█████     | 112669/225000 [235:29:54<168:29:18,  5.40s/it]
 50%|█████     | 112670/225000 [235:29:57<141:08:56,  4.52s/it]
 50%|█████     | 112671/225000 [235:30:00<125:41:27,  4.03s/it]
 50%|█████     | 112672/225000 [235:30:05<137:13:12,  4.40s/it]
 50%|█████     | 112673/225000 [235:30:08<125:56:10,  4.04s/it]
 50%|█████     | 112674/225000 [235:30:17<174:50:15,  5.60s/it]
 50%|█████     | 112675/225000 [235:30:25<196:43:46,  6.31s/it]
 50%|█████     | 112676/225000 [235:30:28<165:44:27,  5.31s/it]
 50%|█████     | 112677/225000 [235:30:31<142:00:05,  4.55s/it]
 50%|█████     | 112678/225000 [235:30:33<120:59:45,  3.88s/it]
 50%|█████     | 112679/225000 [235:30:36<112:50:53,  3.62s/it]
 50%|█████     | 112680/225000 [235:30:40<112:06:20,  3.59s/it]
 50%|█████     | 112681/225000 [235:30:50<173:52:24,  5.57s/it]
 50%|█████     | 112682/225000 [235:30:53<151:52:09,  4.87s/it]
 50%|█████     | 112683/225000 [235:30:57<136:31:40,  4.38s/it]
 50%|█████     | 112684/225000 [235:31:00<126:18:46,  4.05s/it]
 50%|█████     | 112685/225000 [235:31:03<115:35:39,  3.71s/it]
 50%|█████     | 112686/225000 [235:31:07<118:25:35,  3.80s/it]
 50%|█████     | 112687/225000 [235:31:11<119:43:28,  3.84s/it]
 50%|█████     | 112688/225000 [235:31:15<128:40:06,  4.12s/it]
 50%|█████     | 112689/225000 [235:31:19<125:51:28,  4.03s/it]
 50%|█████     | 112690/225000 [235:31:23<123:35:41,  3.96s/it]
 50%|█████     | 112691/225000 [235:31:27<121:41:39,  3.90s/it]
 50%|█████     | 112692/225000 [235:31:39<199:44:46,  6.40s/it]
 50%|█████     | 112693/225000 [235:31:48<226:55:04,  7.27s/it]
 50%|█████     | 112694/225000 [235:31:52<189:26:21,  6.07s/it]
 50%|█████     | 112695/225000 [235:31:54<154:04:08,  4.94s/it]
 50%|█████     | 112696/225000 [235:31:58<144:17:00,  4.63s/it]
 50%|█████     | 112697/225000 [235:32:01<126:04:19,  4.04s/it]
 50%|█████     | 112698/225000 [235:32:05<128:34:43,  4.12s/it]
 50%|█████     | 112699/225000 [235:32:12<159:04:03,  5.10s/it]
 50%|█████     | 112700/225000 [235:32:16<144:06:48,  4.62s/it]
                                                               
{'loss': '0.926', 'grad_norm': '1.8', 'learning_rate': '2.677e-05', 'epoch': '0.5009', 'num_input_tokens_seen': 393782152, 'train_runtime': '8.479e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112700/225000 [235:32:16<144:06:48,  4.62s/it]
 50%|█████     | 112701/225000 [235:32:18<123:29:33,  3.96s/it]
 50%|█████     | 112702/225000 [235:32:21<115:19:54,  3.70s/it]
 50%|█████     | 112703/225000 [235:32:24<103:51:41,  3.33s/it]
 50%|█████     | 112704/225000 [235:32:27<106:13:26,  3.41s/it]
 50%|█████     | 112705/225000 [235:32:36<155:40:27,  4.99s/it]
 50%|█████     | 112706/225000 [235:32:38<130:22:57,  4.18s/it]
 50%|█████     | 112707/225000 [235:32:41<118:12:21,  3.79s/it]
 50%|█████     | 112708/225000 [235:32:43<104:49:02,  3.36s/it]
 50%|█████     | 112709/225000 [235:32:50<137:16:39,  4.40s/it]
 50%|█████     | 112710/225000 [235:32:53<120:46:35,  3.87s/it]
 50%|█████     | 112711/225000 [235:32:55<106:54:09,  3.43s/it]
 50%|█████     | 112712/225000 [235:32:58<103:25:58,  3.32s/it]
 50%|█████     | 112713/225000 [235:33:02<104:38:32,  3.35s/it]
 50%|█████     | 112714/225000 [235:33:13<179:06:11,  5.74s/it]
 50%|█████     | 112715/225000 [235:33:17<157:51:05,  5.06s/it]
 50%|█████     | 112716/225000 [235:33:20<144:07:27,  4.62s/it]
 50%|█████     | 112717/225000 [235:33:31<203:38:46,  6.53s/it]
 50%|█████     | 112718/225000 [235:33:37<192:59:14,  6.19s/it]
 50%|█████     | 112719/225000 [235:33:40<169:07:21,  5.42s/it]
 50%|█████     | 112720/225000 [235:33:43<146:48:05,  4.71s/it]
 50%|█████     | 112721/225000 [235:33:52<184:37:16,  5.92s/it]
 50%|█████     | 112722/225000 [235:33:58<188:49:49,  6.05s/it]
 50%|█████     | 112723/225000 [235:34:08<219:08:24,  7.03s/it]
 50%|█████     | 112724/225000 [235:34:10<175:52:23,  5.64s/it]
 50%|█████     | 112725/225000 [235:34:14<159:12:33,  5.10s/it]
 50%|█████     | 112726/225000 [235:34:17<135:36:20,  4.35s/it]
 50%|█████     | 112727/225000 [235:34:23<158:09:35,  5.07s/it]
 50%|█████     | 112728/225000 [235:34:26<135:06:15,  4.33s/it]
 50%|█████     | 112729/225000 [235:34:29<127:01:34,  4.07s/it]
 50%|█████     | 112730/225000 [235:34:32<115:49:31,  3.71s/it]
 50%|█████     | 112731/225000 [235:34:35<106:41:12,  3.42s/it]
 50%|█████     | 112732/225000 [235:34:38<106:11:52,  3.41s/it]
 50%|█████     | 112733/225000 [235:34:42<112:40:23,  3.61s/it]
 50%|█████     | 112734/225000 [235:34:46<109:34:44,  3.51s/it]
 50%|█████     | 112735/225000 [235:34:49<111:06:40,  3.56s/it]
 50%|█████     | 112736/225000 [235:34:53<111:13:38,  3.57s/it]
 50%|█████     | 112737/225000 [235:34:59<137:54:43,  4.42s/it]
 50%|█████     | 112738/225000 [235:35:08<177:12:43,  5.68s/it]
 50%|█████     | 112739/225000 [235:35:11<149:57:54,  4.81s/it]
 50%|█████     | 112740/225000 [235:35:13<129:04:37,  4.14s/it]
 50%|█████     | 112741/225000 [235:35:16<117:41:53,  3.77s/it]
 50%|█████     | 112742/225000 [235:35:25<164:06:09,  5.26s/it]
 50%|█████     | 112743/225000 [235:35:28<145:15:11,  4.66s/it]
 50%|█████     | 112744/225000 [235:35:38<188:05:45,  6.03s/it]
 50%|█████     | 112745/225000 [235:35:41<162:57:51,  5.23s/it]
 50%|█████     | 112746/225000 [235:35:44<140:20:44,  4.50s/it]
 50%|█████     | 112747/225000 [235:35:47<132:17:45,  4.24s/it]
 50%|█████     | 112748/225000 [235:35:55<161:36:24,  5.18s/it]
 50%|█████     | 112749/225000 [235:35:57<137:36:03,  4.41s/it]
 50%|█████     | 112750/225000 [235:36:01<129:05:50,  4.14s/it]
                                                               
{'loss': '0.9383', 'grad_norm': '1.763', 'learning_rate': '2.675e-05', 'epoch': '0.5011', 'num_input_tokens_seen': 393876792, 'train_runtime': '8.482e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112750/225000 [235:36:01<129:05:50,  4.14s/it]
 50%|█████     | 112751/225000 [235:36:04<118:34:00,  3.80s/it]
 50%|█████     | 112752/225000 [235:36:10<141:21:38,  4.53s/it]
 50%|█████     | 112753/225000 [235:36:14<134:49:44,  4.32s/it]
 50%|█████     | 112754/225000 [235:36:17<127:06:52,  4.08s/it]
 50%|█████     | 112755/225000 [235:36:23<143:31:24,  4.60s/it]
 50%|█████     | 112756/225000 [235:36:29<158:32:20,  5.08s/it]
 50%|█████     | 112757/225000 [235:36:36<171:24:24,  5.50s/it]
 50%|█████     | 112758/225000 [235:36:38<142:53:23,  4.58s/it]
 50%|█████     | 112759/225000 [235:36:43<139:14:07,  4.47s/it]
 50%|█████     | 112760/225000 [235:36:45<123:29:13,  3.96s/it]
 50%|█████     | 112761/225000 [235:36:52<146:47:58,  4.71s/it]
 50%|█████     | 112762/225000 [235:36:57<155:24:18,  4.98s/it]
 50%|█████     | 112763/225000 [235:37:00<136:33:47,  4.38s/it]
 50%|█████     | 112764/225000 [235:37:03<123:37:28,  3.97s/it]
 50%|█████     | 112765/225000 [235:37:07<121:08:47,  3.89s/it]
 50%|█████     | 112766/225000 [235:37:11<123:39:51,  3.97s/it]
 50%|█████     | 112767/225000 [235:37:15<118:56:29,  3.82s/it]
 50%|█████     | 112768/225000 [235:37:19<123:38:22,  3.97s/it]
 50%|█████     | 112769/225000 [235:37:22<117:19:25,  3.76s/it]
 50%|█████     | 112770/225000 [235:37:27<124:42:25,  4.00s/it]
 50%|█████     | 112771/225000 [235:37:30<118:49:00,  3.81s/it]
 50%|█████     | 112772/225000 [235:37:34<116:57:56,  3.75s/it]
 50%|█████     | 112773/225000 [235:37:37<110:02:53,  3.53s/it]
 50%|█████     | 112774/225000 [235:37:42<127:44:41,  4.10s/it]
 50%|█████     | 112775/225000 [235:37:53<189:06:38,  6.07s/it]
 50%|█████     | 112776/225000 [235:37:57<175:01:41,  5.61s/it]
 50%|█████     | 112777/225000 [235:38:00<150:07:44,  4.82s/it]
 50%|█████     | 112778/225000 [235:38:03<128:47:53,  4.13s/it]
 50%|█████     | 112779/225000 [235:38:12<171:14:27,  5.49s/it]
 50%|█████     | 112780/225000 [235:38:18<175:04:42,  5.62s/it]
 50%|█████     | 112781/225000 [235:38:23<170:26:03,  5.47s/it]
 50%|█████     | 112782/225000 [235:38:27<160:20:12,  5.14s/it]
 50%|█████     | 112783/225000 [235:38:33<170:44:16,  5.48s/it]
 50%|█████     | 112784/225000 [235:38:37<151:54:32,  4.87s/it]
 50%|█████     | 112785/225000 [235:38:40<139:35:09,  4.48s/it]
 50%|█████     | 112786/225000 [235:38:44<131:45:15,  4.23s/it]
 50%|█████     | 112787/225000 [235:38:53<177:55:06,  5.71s/it]
 50%|█████     | 112788/225000 [235:38:56<147:54:05,  4.74s/it]
 50%|█████     | 112789/225000 [235:39:06<201:02:44,  6.45s/it]
 50%|█████     | 112790/225000 [235:39:08<161:26:13,  5.18s/it]
 50%|█████     | 112791/225000 [235:39:11<136:01:34,  4.36s/it]
 50%|█████     | 112792/225000 [235:39:13<119:56:06,  3.85s/it]
 50%|█████     | 112793/225000 [235:39:17<116:19:41,  3.73s/it]
 50%|█████     | 112794/225000 [235:39:25<156:30:52,  5.02s/it]
 50%|█████     | 112795/225000 [235:39:28<142:17:45,  4.57s/it]
 50%|█████     | 112796/225000 [235:39:31<126:53:58,  4.07s/it]
 50%|█████     | 112797/225000 [235:39:35<121:56:33,  3.91s/it]
 50%|█████     | 112798/225000 [235:39:43<164:24:24,  5.27s/it]
 50%|█████     | 112799/225000 [235:39:47<145:37:43,  4.67s/it]
 50%|█████     | 112800/225000 [235:39:50<130:13:40,  4.18s/it]
                                                               
{'loss': '0.9339', 'grad_norm': '2.448', 'learning_rate': '2.673e-05', 'epoch': '0.5013', 'num_input_tokens_seen': 393977392, 'train_runtime': '8.484e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112800/225000 [235:39:50<130:13:40,  4.18s/it]
 50%|█████     | 112801/225000 [235:39:53<119:19:25,  3.83s/it]
 50%|█████     | 112802/225000 [235:39:56<114:21:23,  3.67s/it]
 50%|█████     | 112803/225000 [235:40:02<133:51:47,  4.30s/it]
 50%|█████     | 112804/225000 [235:40:15<215:30:29,  6.91s/it]
 50%|█████     | 112805/225000 [235:40:20<199:03:38,  6.39s/it]
 50%|█████     | 112806/225000 [235:40:29<223:58:50,  7.19s/it]
 50%|█████     | 112807/225000 [235:40:32<183:21:10,  5.88s/it]
 50%|█████     | 112808/225000 [235:40:36<163:51:34,  5.26s/it]
 50%|█████     | 112809/225000 [235:40:39<142:43:46,  4.58s/it]
 50%|█████     | 112810/225000 [235:40:48<190:24:23,  6.11s/it]
 50%|█████     | 112811/225000 [235:40:53<182:21:20,  5.85s/it]
 50%|█████     | 112812/225000 [235:40:57<156:49:34,  5.03s/it]
 50%|█████     | 112813/225000 [235:41:00<139:18:31,  4.47s/it]
 50%|█████     | 112814/225000 [235:41:02<122:20:48,  3.93s/it]
 50%|█████     | 112815/225000 [235:41:06<122:14:43,  3.92s/it]
 50%|█████     | 112816/225000 [235:41:10<118:36:09,  3.81s/it]
 50%|█████     | 112817/225000 [235:41:13<109:09:16,  3.50s/it]
 50%|█████     | 112818/225000 [235:41:16<111:12:33,  3.57s/it]
 50%|█████     | 112819/225000 [235:41:22<134:47:27,  4.33s/it]
 50%|█████     | 112820/225000 [235:41:26<123:56:35,  3.98s/it]
 50%|█████     | 112821/225000 [235:41:29<115:02:22,  3.69s/it]
 50%|█████     | 112822/225000 [235:41:37<161:18:29,  5.18s/it]
 50%|█████     | 112823/225000 [235:41:40<134:54:39,  4.33s/it]
 50%|█████     | 112824/225000 [235:41:42<114:57:19,  3.69s/it]
 50%|█████     | 112825/225000 [235:41:49<151:40:50,  4.87s/it]
 50%|█████     | 112826/225000 [235:41:55<156:47:55,  5.03s/it]
 50%|█████     | 112827/225000 [235:41:58<134:52:44,  4.33s/it]
 50%|█████     | 112828/225000 [235:42:04<157:31:23,  5.06s/it]
 50%|█████     | 112829/225000 [235:42:08<148:47:08,  4.78s/it]
 50%|█████     | 112830/225000 [235:42:12<134:46:33,  4.33s/it]
 50%|█████     | 112831/225000 [235:42:15<124:39:14,  4.00s/it]
 50%|█████     | 112832/225000 [235:42:18<120:23:08,  3.86s/it]
 50%|█████     | 112833/225000 [235:42:26<156:24:03,  5.02s/it]
 50%|█████     | 112834/225000 [235:42:29<133:45:47,  4.29s/it]
 50%|█████     | 112835/225000 [235:42:32<120:05:41,  3.85s/it]
 50%|█████     | 112836/225000 [235:42:35<117:52:04,  3.78s/it]
 50%|█████     | 112837/225000 [235:42:46<180:48:01,  5.80s/it]
 50%|█████     | 112838/225000 [235:42:49<152:56:32,  4.91s/it]
 50%|█████     | 112839/225000 [235:42:51<130:39:03,  4.19s/it]
 50%|█████     | 112840/225000 [235:42:54<114:19:15,  3.67s/it]
 50%|█████     | 112841/225000 [235:42:59<127:33:30,  4.09s/it]
 50%|█████     | 112842/225000 [235:43:04<143:06:51,  4.59s/it]
 50%|█████     | 112843/225000 [235:43:07<126:38:23,  4.06s/it]
 50%|█████     | 112844/225000 [235:43:10<115:47:07,  3.72s/it]
 50%|█████     | 112845/225000 [235:43:14<119:45:52,  3.84s/it]
 50%|█████     | 112846/225000 [235:43:17<113:12:18,  3.63s/it]
 50%|█████     | 112847/225000 [235:43:21<115:14:38,  3.70s/it]
 50%|█████     | 112848/225000 [235:43:24<109:10:28,  3.50s/it]
 50%|█████     | 112849/225000 [235:43:28<108:24:26,  3.48s/it]
 50%|█████     | 112850/225000 [235:43:34<131:01:31,  4.21s/it]
                                                               
{'loss': '0.9597', 'grad_norm': '1.727', 'learning_rate': '2.672e-05', 'epoch': '0.5016', 'num_input_tokens_seen': 394075240, 'train_runtime': '8.486e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112850/225000 [235:43:34<131:01:31,  4.21s/it]
 50%|█████     | 112851/225000 [235:43:40<146:44:30,  4.71s/it]
 50%|█████     | 112852/225000 [235:43:45<156:37:11,  5.03s/it]
 50%|█████     | 112853/225000 [235:43:49<145:30:37,  4.67s/it]
 50%|█████     | 112854/225000 [235:43:59<191:52:45,  6.16s/it]
 50%|█████     | 112855/225000 [235:44:04<182:22:17,  5.85s/it]
 50%|█████     | 112856/225000 [235:44:06<150:47:03,  4.84s/it]
 50%|█████     | 112857/225000 [235:44:12<161:07:18,  5.17s/it]
 50%|█████     | 112858/225000 [235:44:18<168:53:52,  5.42s/it]
 50%|█████     | 112859/225000 [235:44:21<143:27:09,  4.61s/it]
 50%|█████     | 112860/225000 [235:44:23<123:20:39,  3.96s/it]
 50%|█████     | 112861/225000 [235:44:28<124:12:09,  3.99s/it]
 50%|█████     | 112862/225000 [235:44:34<146:23:39,  4.70s/it]
 50%|█████     | 112863/225000 [235:44:37<127:29:50,  4.09s/it]
 50%|█████     | 112864/225000 [235:44:40<125:20:33,  4.02s/it]
 50%|█████     | 112865/225000 [235:44:44<117:58:21,  3.79s/it]
 50%|█████     | 112866/225000 [235:44:47<116:18:28,  3.73s/it]
 50%|█████     | 112867/225000 [235:44:51<119:01:26,  3.82s/it]
 50%|█████     | 112868/225000 [235:44:55<114:29:56,  3.68s/it]
 50%|█████     | 112869/225000 [235:44:58<113:30:39,  3.64s/it]
 50%|█████     | 112870/225000 [235:45:03<126:51:09,  4.07s/it]
 50%|█████     | 112871/225000 [235:45:17<212:57:10,  6.84s/it]
 50%|█████     | 112872/225000 [235:45:20<179:31:17,  5.76s/it]
 50%|█████     | 112873/225000 [235:45:23<154:20:32,  4.96s/it]
 50%|█████     | 112874/225000 [235:45:26<133:59:57,  4.30s/it]
 50%|█████     | 112875/225000 [235:45:29<124:42:01,  4.00s/it]
 50%|█████     | 112876/225000 [235:45:32<111:15:21,  3.57s/it]
 50%|█████     | 112877/225000 [235:45:35<109:13:05,  3.51s/it]
 50%|█████     | 112878/225000 [235:45:48<196:54:30,  6.32s/it]
 50%|█████     | 112879/225000 [235:45:51<167:56:50,  5.39s/it]
 50%|█████     | 112880/225000 [235:45:54<148:14:16,  4.76s/it]
 50%|█████     | 112881/225000 [235:46:08<234:22:10,  7.53s/it]
 50%|█████     | 112882/225000 [235:46:12<196:31:26,  6.31s/it]
 50%|█████     | 112883/225000 [235:46:14<161:19:51,  5.18s/it]
 50%|█████     | 112884/225000 [235:46:17<138:22:49,  4.44s/it]
 50%|█████     | 112885/225000 [235:46:23<152:13:18,  4.89s/it]
 50%|█████     | 112886/225000 [235:46:28<152:00:48,  4.88s/it]
 50%|█████     | 112887/225000 [235:46:32<143:08:36,  4.60s/it]
 50%|█████     | 112888/225000 [235:46:35<127:37:23,  4.10s/it]
 50%|█████     | 112889/225000 [235:46:38<117:46:41,  3.78s/it]
 50%|█████     | 112890/225000 [235:46:41<112:11:36,  3.60s/it]
 50%|█████     | 112891/225000 [235:46:44<109:56:38,  3.53s/it]
 50%|█████     | 112892/225000 [235:46:48<109:01:25,  3.50s/it]
 50%|█████     | 112893/225000 [235:46:51<110:06:52,  3.54s/it]
 50%|█████     | 112894/225000 [235:46:56<122:55:48,  3.95s/it]
 50%|█████     | 112895/225000 [235:47:05<163:38:59,  5.26s/it]
 50%|█████     | 112896/225000 [235:47:09<159:08:19,  5.11s/it]
 50%|█████     | 112897/225000 [235:47:13<142:06:55,  4.56s/it]
 50%|█████     | 112898/225000 [235:47:22<185:14:27,  5.95s/it]
 50%|█████     | 112899/225000 [235:47:24<154:28:16,  4.96s/it]
 50%|█████     | 112900/225000 [235:47:27<134:30:58,  4.32s/it]
                                                               
{'loss': '0.9595', 'grad_norm': '1.5', 'learning_rate': '2.67e-05', 'epoch': '0.5018', 'num_input_tokens_seen': 394174616, 'train_runtime': '8.488e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112900/225000 [235:47:27<134:30:58,  4.32s/it]
 50%|█████     | 112901/225000 [235:47:30<120:37:15,  3.87s/it]
 50%|█████     | 112902/225000 [235:47:35<125:41:45,  4.04s/it]
 50%|█████     | 112903/225000 [235:47:46<190:56:53,  6.13s/it]
 50%|█████     | 112904/225000 [235:47:49<162:20:45,  5.21s/it]
 50%|█████     | 112905/225000 [235:47:53<155:58:35,  5.01s/it]
 50%|█████     | 112906/225000 [235:47:56<135:46:57,  4.36s/it]
 50%|█████     | 112907/225000 [235:47:59<124:12:20,  3.99s/it]
 50%|█████     | 112908/225000 [235:48:02<110:55:26,  3.56s/it]
 50%|█████     | 112909/225000 [235:48:05<107:48:46,  3.46s/it]
 50%|█████     | 112910/225000 [235:48:09<113:01:18,  3.63s/it]
 50%|█████     | 112911/225000 [235:48:12<111:53:46,  3.59s/it]
 50%|█████     | 112912/225000 [235:48:20<152:38:23,  4.90s/it]
 50%|█████     | 112913/225000 [235:48:31<205:55:42,  6.61s/it]
 50%|█████     | 112914/225000 [235:48:34<175:55:25,  5.65s/it]
 50%|█████     | 112915/225000 [235:48:37<149:58:18,  4.82s/it]
 50%|█████     | 112916/225000 [235:48:40<134:24:53,  4.32s/it]
 50%|█████     | 112917/225000 [235:48:44<125:22:18,  4.03s/it]
 50%|█████     | 112918/225000 [235:48:49<133:25:44,  4.29s/it]
 50%|█████     | 112919/225000 [235:48:57<175:48:45,  5.65s/it]
 50%|█████     | 112920/225000 [235:49:00<147:46:40,  4.75s/it]
 50%|█████     | 112921/225000 [235:49:05<145:06:33,  4.66s/it]
 50%|█████     | 112922/225000 [235:49:10<150:28:03,  4.83s/it]
 50%|█████     | 112923/225000 [235:49:19<194:24:19,  6.24s/it]
 50%|█████     | 112924/225000 [235:49:23<166:44:57,  5.36s/it]
 50%|█████     | 112925/225000 [235:49:28<166:23:49,  5.34s/it]
 50%|█████     | 112926/225000 [235:49:34<169:11:26,  5.43s/it]
 50%|█████     | 112927/225000 [235:49:43<206:30:14,  6.63s/it]
 50%|█████     | 112928/225000 [235:49:45<166:44:15,  5.36s/it]
 50%|█████     | 112929/225000 [235:49:52<176:53:40,  5.68s/it]
 50%|█████     | 112930/225000 [235:49:54<148:10:02,  4.76s/it]
 50%|█████     | 112931/225000 [235:49:57<124:46:27,  4.01s/it]
 50%|█████     | 112932/225000 [235:50:00<114:20:56,  3.67s/it]
 50%|█████     | 112933/225000 [235:50:05<133:17:04,  4.28s/it]
 50%|█████     | 112934/225000 [235:50:12<158:45:56,  5.10s/it]
 50%|█████     | 112935/225000 [235:50:16<141:00:12,  4.53s/it]
 50%|█████     | 112936/225000 [235:50:18<122:22:23,  3.93s/it]
 50%|█████     | 112937/225000 [235:50:27<169:52:42,  5.46s/it]
 50%|█████     | 112938/225000 [235:50:30<148:59:51,  4.79s/it]
 50%|█████     | 112939/225000 [235:50:35<151:30:01,  4.87s/it]
 50%|█████     | 112940/225000 [235:50:39<141:35:41,  4.55s/it]
 50%|█████     | 112941/225000 [235:50:42<126:01:25,  4.05s/it]
 50%|█████     | 112942/225000 [235:50:48<145:06:34,  4.66s/it]
 50%|█████     | 112943/225000 [235:50:52<135:27:59,  4.35s/it]
 50%|█████     | 112944/225000 [235:50:56<130:44:57,  4.20s/it]
 50%|█████     | 112945/225000 [235:50:59<121:27:48,  3.90s/it]
 50%|█████     | 112946/225000 [235:51:03<120:39:45,  3.88s/it]
 50%|█████     | 112947/225000 [235:51:06<117:21:51,  3.77s/it]
 50%|█████     | 112948/225000 [235:51:19<201:02:51,  6.46s/it]
 50%|█████     | 112949/225000 [235:51:21<164:16:13,  5.28s/it]
 50%|█████     | 112950/225000 [235:51:24<140:03:44,  4.50s/it]
                                                               
{'loss': '0.9367', 'grad_norm': '1.745', 'learning_rate': '2.668e-05', 'epoch': '0.502', 'num_input_tokens_seen': 394272424, 'train_runtime': '8.491e+05', 'train_tokens_per_second': '464.4'}

 50%|█████     | 112950/225000 [235:51:24<140:03:44,  4.50s/it]
 50%|█████     | 112951/225000 [235:51:28<132:19:50,  4.25s/it]
 50%|█████     | 112952/225000 [235:51:37<175:01:14,  5.62s/it]
 50%|█████     | 112953/225000 [235:51:41<162:23:33,  5.22s/it]
 50%|█████     | 112954/225000 [235:51:45<151:38:46,  4.87s/it]
 50%|█████     | 112955/225000 [235:51:48<139:24:41,  4.48s/it]
 50%|█████     | 112956/225000 [235:51:52<127:49:10,  4.11s/it]
 50%|█████     | 112957/225000 [235:51:55<122:15:12,  3.93s/it]
 50%|█████     | 112958/225000 [235:51:59<121:11:28,  3.89s/it]
 50%|█████     | 112959/225000 [235:52:04<127:09:48,  4.09s/it]
 50%|█████     | 112960/225000 [235:52:08<128:27:27,  4.13s/it]
 50%|█████     | 112961/225000 [235:52:11<121:26:03,  3.90s/it]
 50%|█████     | 112962/225000 [235:52:15<118:04:24,  3.79s/it]
 50%|█████     | 112963/225000 [235:52:26<190:27:28,  6.12s/it]
 50%|█████     | 112964/225000 [235:52:29<159:58:59,  5.14s/it]
 50%|█████     | 112965/225000 [235:52:35<167:47:01,  5.39s/it]
 50%|█████     | 112966/225000 [235:52:42<178:07:45,  5.72s/it]
 50%|█████     | 112967/225000 [235:52:49<193:17:33,  6.21s/it]
 50%|█████     | 112968/225000 [235:52:52<160:26:59,  5.16s/it]
 50%|█████     | 112969/225000 [235:52:55<140:01:57,  4.50s/it]
 50%|█████     | 112970/225000 [235:53:04<186:24:28,  5.99s/it]
 50%|█████     | 112971/225000 [235:53:07<158:46:07,  5.10s/it]
 50%|█████     | 112972/225000 [235:53:10<139:18:34,  4.48s/it]
 50%|█████     | 112973/225000 [235:53:13<126:00:09,  4.05s/it]
 50%|█████     | 112974/225000 [235:53:25<199:11:38,  6.40s/it]
 50%|█████     | 112975/225000 [235:53:29<176:35:42,  5.68s/it]
 50%|█████     | 112976/225000 [235:53:35<182:02:35,  5.85s/it]
 50%|█████     | 112977/225000 [235:53:38<149:34:01,  4.81s/it]
 50%|█████     | 112978/225000 [235:53:41<132:45:35,  4.27s/it]
 50%|█████     | 112979/225000 [235:53:47<150:30:49,  4.84s/it]
 50%|█████     | 112980/225000 [235:53:52<156:02:21,  5.01s/it]
 50%|█████     | 112981/225000 [235:53:55<135:28:09,  4.35s/it]
 50%|█████     | 112982/225000 [235:53:58<125:32:06,  4.03s/it]
 50%|█████     | 112983/225000 [235:54:02<118:23:30,  3.80s/it]
 50%|█████     | 112984/225000 [235:54:14<199:15:26,  6.40s/it]
 50%|█████     | 112985/225000 [235:54:19<180:40:50,  5.81s/it]
 50%|█████     | 112986/225000 [235:54:21<149:31:43,  4.81s/it]
 50%|█████     | 112987/225000 [235:54:25<138:47:57,  4.46s/it]
 50%|█████     | 112988/225000 [235:54:28<126:19:54,  4.06s/it]
 50%|█████     | 112989/225000 [235:54:36<165:50:03,  5.33s/it]
 50%|█████     | 112990/225000 [235:54:39<141:54:46,  4.56s/it]
 50%|█████     | 112991/225000 [235:54:48<186:31:33,  5.99s/it]
 50%|█████     | 112992/225000 [235:54:53<177:06:40,  5.69s/it]
 50%|█████     | 112993/225000 [235:54:56<150:47:30,  4.85s/it]
 50%|█████     | 112994/225000 [235:54:59<132:02:44,  4.24s/it]
 50%|█████     | 112995/225000 [235:55:02<121:14:48,  3.90s/it]
 50%|█████     | 112996/225000 [235:55:05<114:43:43,  3.69s/it]
 50%|█████     | 112997/225000 [235:55:11<136:48:06,  4.40s/it]
 50%|█████     | 112998/225000 [235:55:17<153:03:23,  4.92s/it]
 50%|█████     | 112999/225000 [235:55:21<143:48:55,  4.62s/it]
 50%|█████     | 113000/225000 [235:55:26<146:33:12,  4.71s/it]
                                                               
{'loss': '0.9532', 'grad_norm': '1.633', 'learning_rate': '2.666e-05', 'epoch': '0.5022', 'num_input_tokens_seen': 394376760, 'train_runtime': '8.493e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113000/225000 [235:55:26<146:33:12,  4.71s/it]
 50%|█████     | 113001/225000 [235:55:29<130:08:05,  4.18s/it]
 50%|█████     | 113002/225000 [235:55:33<127:54:32,  4.11s/it]
 50%|█████     | 113003/225000 [235:55:38<134:27:05,  4.32s/it]
 50%|█████     | 113004/225000 [235:55:41<127:26:46,  4.10s/it]
 50%|█████     | 113005/225000 [235:55:44<116:59:18,  3.76s/it]
 50%|█████     | 113006/225000 [235:55:58<204:59:02,  6.59s/it]
 50%|█████     | 113007/225000 [235:56:01<178:06:22,  5.73s/it]
 50%|█████     | 113008/225000 [235:56:04<151:08:57,  4.86s/it]
 50%|█████     | 113009/225000 [235:56:07<131:48:14,  4.24s/it]
 50%|█████     | 113010/225000 [235:56:10<123:04:28,  3.96s/it]
 50%|█████     | 113011/225000 [235:56:19<167:48:33,  5.39s/it]
 50%|█████     | 113012/225000 [235:56:22<143:03:14,  4.60s/it]
 50%|█████     | 113013/225000 [235:56:24<123:30:57,  3.97s/it]
 50%|█████     | 113014/225000 [235:56:28<121:00:45,  3.89s/it]
 50%|█████     | 113015/225000 [235:56:32<123:02:56,  3.96s/it]
 50%|█████     | 113016/225000 [235:56:36<122:19:53,  3.93s/it]
 50%|█████     | 113017/225000 [235:56:40<121:20:21,  3.90s/it]
 50%|█████     | 113018/225000 [235:56:43<110:58:40,  3.57s/it]
 50%|█████     | 113019/225000 [235:56:46<108:06:29,  3.48s/it]
 50%|█████     | 113020/225000 [235:56:49<107:05:47,  3.44s/it]
 50%|█████     | 113021/225000 [235:56:55<133:11:58,  4.28s/it]
 50%|█████     | 113022/225000 [235:57:12<251:53:41,  8.10s/it]
 50%|█████     | 113023/225000 [235:57:16<211:36:40,  6.80s/it]
 50%|█████     | 113024/225000 [235:57:20<180:51:05,  5.81s/it]
 50%|█████     | 113025/225000 [235:57:24<161:48:27,  5.20s/it]
 50%|█████     | 113026/225000 [235:57:34<207:16:21,  6.66s/it]
 50%|█████     | 113027/225000 [235:57:36<167:30:08,  5.39s/it]
 50%|█████     | 113028/225000 [235:57:40<151:07:27,  4.86s/it]
 50%|█████     | 113029/225000 [235:57:43<137:24:45,  4.42s/it]
 50%|█████     | 113030/225000 [235:57:47<128:41:49,  4.14s/it]
 50%|█████     | 113031/225000 [235:57:54<156:22:21,  5.03s/it]
 50%|█████     | 113032/225000 [235:57:58<152:19:06,  4.90s/it]
 50%|█████     | 113033/225000 [235:58:01<131:34:22,  4.23s/it]
 50%|█████     | 113034/225000 [235:58:04<119:38:09,  3.85s/it]
 50%|█████     | 113035/225000 [235:58:13<172:45:04,  5.55s/it]
 50%|█████     | 113036/225000 [235:58:19<172:48:26,  5.56s/it]
 50%|█████     | 113037/225000 [235:58:22<148:33:24,  4.78s/it]
 50%|█████     | 113038/225000 [235:58:25<135:24:40,  4.35s/it]
 50%|█████     | 113039/225000 [235:58:28<123:18:25,  3.96s/it]
 50%|█████     | 113040/225000 [235:58:31<111:46:49,  3.59s/it]
 50%|█████     | 113041/225000 [235:58:34<108:48:57,  3.50s/it]
 50%|█████     | 113042/225000 [235:58:42<150:40:41,  4.85s/it]
 50%|█████     | 113043/225000 [235:58:50<174:45:23,  5.62s/it]
 50%|█████     | 113044/225000 [235:58:52<147:08:16,  4.73s/it]
 50%|█████     | 113045/225000 [235:58:56<141:03:03,  4.54s/it]
 50%|█████     | 113046/225000 [235:59:00<130:41:13,  4.20s/it]
 50%|█████     | 113047/225000 [235:59:08<163:57:20,  5.27s/it]
 50%|█████     | 113048/225000 [235:59:10<137:58:36,  4.44s/it]
 50%|█████     | 113049/225000 [235:59:13<123:22:09,  3.97s/it]
 50%|█████     | 113050/225000 [235:59:16<113:18:33,  3.64s/it]
                                                               
{'loss': '0.9498', 'grad_norm': '1.662', 'learning_rate': '2.664e-05', 'epoch': '0.5024', 'num_input_tokens_seen': 394476064, 'train_runtime': '8.496e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113050/225000 [235:59:16<113:18:33,  3.64s/it]
 50%|█████     | 113051/225000 [235:59:19<104:41:21,  3.37s/it]
 50%|█████     | 113052/225000 [235:59:27<148:41:22,  4.78s/it]
 50%|█████     | 113053/225000 [235:59:33<159:54:02,  5.14s/it]
 50%|█████     | 113054/225000 [235:59:36<146:01:42,  4.70s/it]
 50%|█████     | 113055/225000 [235:59:46<187:58:31,  6.05s/it]
 50%|█████     | 113056/225000 [235:59:49<159:43:33,  5.14s/it]
 50%|█████     | 113057/225000 [235:59:51<135:17:49,  4.35s/it]
 50%|█████     | 113058/225000 [235:59:54<118:05:23,  3.80s/it]
 50%|█████     | 113059/225000 [235:59:59<133:56:45,  4.31s/it]
 50%|█████     | 113060/225000 [236:00:06<157:06:28,  5.05s/it]
 50%|█████     | 113061/225000 [236:00:08<133:58:11,  4.31s/it]
 50%|█████     | 113062/225000 [236:00:13<137:57:56,  4.44s/it]
 50%|█████     | 113063/225000 [236:00:17<129:47:12,  4.17s/it]
 50%|█████     | 113064/225000 [236:00:22<141:39:59,  4.56s/it]
 50%|█████     | 113065/225000 [236:00:28<157:36:08,  5.07s/it]
 50%|█████     | 113066/225000 [236:00:33<154:46:37,  4.98s/it]
 50%|█████     | 113067/225000 [236:00:37<146:18:32,  4.71s/it]
 50%|█████     | 113068/225000 [236:00:41<139:53:11,  4.50s/it]
 50%|█████     | 113069/225000 [236:00:45<134:08:35,  4.31s/it]
 50%|█████     | 113070/225000 [236:00:49<132:48:23,  4.27s/it]
 50%|█████     | 113071/225000 [236:01:00<189:48:01,  6.10s/it]
 50%|█████     | 113072/225000 [236:01:06<190:24:15,  6.12s/it]
 50%|█████     | 113073/225000 [236:01:09<164:19:22,  5.29s/it]
 50%|█████     | 113074/225000 [236:01:22<233:22:05,  7.51s/it]
 50%|█████     | 113075/225000 [236:01:26<198:59:22,  6.40s/it]
 50%|█████     | 113076/225000 [236:01:28<163:31:40,  5.26s/it]
 50%|█████     | 113077/225000 [236:01:31<140:46:14,  4.53s/it]
 50%|█████     | 113078/225000 [236:01:34<121:15:02,  3.90s/it]
 50%|█████     | 113079/225000 [236:01:39<138:25:27,  4.45s/it]
 50%|█████     | 113080/225000 [236:01:43<134:03:52,  4.31s/it]
 50%|█████     | 113081/225000 [236:01:51<168:46:40,  5.43s/it]
 50%|█████     | 113082/225000 [236:01:55<151:25:17,  4.87s/it]
 50%|█████     | 113083/225000 [236:02:01<163:34:49,  5.26s/it]
 50%|█████     | 113084/225000 [236:02:04<138:31:05,  4.46s/it]
 50%|█████     | 113085/225000 [236:02:07<132:07:55,  4.25s/it]
 50%|█████     | 113086/225000 [236:02:16<168:58:23,  5.44s/it]
 50%|█████     | 113087/225000 [236:02:20<162:30:15,  5.23s/it]
 50%|█████     | 113088/225000 [236:02:24<146:54:06,  4.73s/it]
 50%|█████     | 113089/225000 [236:02:28<138:48:19,  4.47s/it]
 50%|█████     | 113090/225000 [236:02:31<131:07:33,  4.22s/it]
 50%|█████     | 113091/225000 [236:02:35<125:26:33,  4.04s/it]
 50%|█████     | 113092/225000 [236:02:39<122:53:08,  3.95s/it]
 50%|█████     | 113093/225000 [236:02:50<193:51:53,  6.24s/it]
 50%|█████     | 113094/225000 [236:02:53<160:08:54,  5.15s/it]
 50%|█████     | 113095/225000 [236:02:56<135:49:46,  4.37s/it]
 50%|█████     | 113096/225000 [236:02:59<127:51:44,  4.11s/it]
 50%|█████     | 113097/225000 [236:03:05<147:44:00,  4.75s/it]
 50%|█████     | 113098/225000 [236:03:08<129:01:38,  4.15s/it]
 50%|█████     | 113099/225000 [236:03:22<218:47:20,  7.04s/it]
 50%|█████     | 113100/225000 [236:03:25<184:08:26,  5.92s/it]
                                                               
{'loss': '0.946', 'grad_norm': '1.365', 'learning_rate': '2.663e-05', 'epoch': '0.5027', 'num_input_tokens_seen': 394577808, 'train_runtime': '8.498e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113100/225000 [236:03:25<184:08:26,  5.92s/it]
 50%|█████     | 113101/225000 [236:03:29<166:53:28,  5.37s/it]
 50%|█████     | 113102/225000 [236:03:32<140:19:59,  4.51s/it]
 50%|█████     | 113103/225000 [236:03:35<128:13:19,  4.13s/it]
 50%|█████     | 113104/225000 [236:03:47<204:06:45,  6.57s/it]
 50%|█████     | 113105/225000 [236:03:51<179:17:55,  5.77s/it]
 50%|█████     | 113106/225000 [236:03:54<150:10:44,  4.83s/it]
 50%|█████     | 113107/225000 [236:04:04<197:02:31,  6.34s/it]
 50%|█████     | 113108/225000 [236:04:08<175:55:59,  5.66s/it]
 50%|█████     | 113109/225000 [236:04:10<148:24:14,  4.77s/it]
 50%|█████     | 113110/225000 [236:04:13<125:26:06,  4.04s/it]
 50%|█████     | 113111/225000 [236:04:16<114:52:45,  3.70s/it]
 50%|█████     | 113112/225000 [236:04:19<114:25:55,  3.68s/it]
 50%|█████     | 113113/225000 [236:04:27<154:47:32,  4.98s/it]
 50%|█████     | 113114/225000 [236:04:30<132:14:53,  4.26s/it]
 50%|█████     | 113115/225000 [236:04:33<122:15:26,  3.93s/it]
 50%|█████     | 113116/225000 [236:04:36<109:54:28,  3.54s/it]
 50%|█████     | 113117/225000 [236:04:39<108:14:27,  3.48s/it]
 50%|█████     | 113118/225000 [236:04:42<100:29:24,  3.23s/it]
 50%|█████     | 113119/225000 [236:04:46<106:33:50,  3.43s/it]
 50%|█████     | 113120/225000 [236:04:49<102:51:54,  3.31s/it]
 50%|█████     | 113121/225000 [236:04:52<107:54:21,  3.47s/it]
 50%|█████     | 113122/225000 [236:04:56<109:27:55,  3.52s/it]
 50%|█████     | 113123/225000 [236:05:08<189:37:48,  6.10s/it]
 50%|█████     | 113124/225000 [236:05:16<203:15:18,  6.54s/it]
 50%|█████     | 113125/225000 [236:05:20<177:52:29,  5.72s/it]
 50%|█████     | 113126/225000 [236:05:23<154:43:30,  4.98s/it]
 50%|█████     | 113127/225000 [236:05:26<137:49:00,  4.43s/it]
 50%|█████     | 113128/225000 [236:05:30<131:30:08,  4.23s/it]
 50%|█████     | 113129/225000 [236:05:33<118:05:55,  3.80s/it]
 50%|█████     | 113130/225000 [236:05:36<112:04:49,  3.61s/it]
 50%|█████     | 113131/225000 [236:05:39<109:30:59,  3.52s/it]
 50%|█████     | 113132/225000 [236:05:42<104:48:10,  3.37s/it]
 50%|█████     | 113133/225000 [236:05:47<120:59:41,  3.89s/it]
 50%|█████     | 113134/225000 [236:05:53<140:37:54,  4.53s/it]
 50%|█████     | 113135/225000 [236:05:58<147:06:48,  4.73s/it]
 50%|█████     | 113136/225000 [236:06:05<160:49:29,  5.18s/it]
 50%|█████     | 113137/225000 [236:06:07<138:52:38,  4.47s/it]
 50%|█████     | 113138/225000 [236:06:11<132:35:52,  4.27s/it]
 50%|█████     | 113139/225000 [236:06:15<124:32:24,  4.01s/it]
 50%|█████     | 113140/225000 [236:06:24<173:48:03,  5.59s/it]
 50%|█████     | 113141/225000 [236:06:27<152:09:03,  4.90s/it]
 50%|█████     | 113142/225000 [236:06:30<136:03:18,  4.38s/it]
 50%|█████     | 113143/225000 [236:06:41<191:21:57,  6.16s/it]
 50%|█████     | 113144/225000 [236:06:44<160:36:40,  5.17s/it]
 50%|█████     | 113145/225000 [236:06:47<146:28:41,  4.71s/it]
 50%|█████     | 113146/225000 [236:06:50<132:35:01,  4.27s/it]
 50%|█████     | 113147/225000 [236:06:58<165:12:28,  5.32s/it]
 50%|█████     | 113148/225000 [236:07:01<141:59:31,  4.57s/it]
 50%|█████     | 113149/225000 [236:07:04<124:08:26,  4.00s/it]
 50%|█████     | 113150/225000 [236:07:07<113:58:22,  3.67s/it]
                                                               
{'loss': '0.912', 'grad_norm': '2.089', 'learning_rate': '2.661e-05', 'epoch': '0.5029', 'num_input_tokens_seen': 394675800, 'train_runtime': '8.5e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113150/225000 [236:07:07<113:58:22,  3.67s/it]
 50%|█████     | 113151/225000 [236:07:14<153:07:38,  4.93s/it]
 50%|█████     | 113152/225000 [236:07:18<138:41:30,  4.46s/it]
 50%|█████     | 113153/225000 [236:07:22<132:00:19,  4.25s/it]
 50%|█████     | 113154/225000 [236:07:24<119:12:24,  3.84s/it]
 50%|█████     | 113155/225000 [236:07:33<160:50:05,  5.18s/it]
 50%|█████     | 113156/225000 [236:07:36<144:14:52,  4.64s/it]
 50%|█████     | 113157/225000 [236:07:40<135:56:41,  4.38s/it]
 50%|█████     | 113158/225000 [236:07:43<121:56:45,  3.93s/it]
 50%|█████     | 113159/225000 [236:07:46<113:29:51,  3.65s/it]
 50%|█████     | 113160/225000 [236:07:50<117:33:25,  3.78s/it]
 50%|█████     | 113161/225000 [236:07:54<118:01:19,  3.80s/it]
 50%|█████     | 113162/225000 [236:08:03<165:58:20,  5.34s/it]
 50%|█████     | 113163/225000 [236:08:10<180:18:11,  5.80s/it]
 50%|█████     | 113164/225000 [236:08:17<197:49:37,  6.37s/it]
 50%|█████     | 113165/225000 [236:08:20<160:38:37,  5.17s/it]
 50%|█████     | 113166/225000 [236:08:22<138:06:48,  4.45s/it]
 50%|█████     | 113167/225000 [236:08:25<122:37:17,  3.95s/it]
 50%|█████     | 113168/225000 [236:08:28<114:19:12,  3.68s/it]
 50%|█████     | 113169/225000 [236:08:32<115:21:52,  3.71s/it]
 50%|█████     | 113170/225000 [236:08:40<153:10:38,  4.93s/it]
 50%|█████     | 113171/225000 [236:08:51<213:57:24,  6.89s/it]
 50%|█████     | 113172/225000 [236:08:59<222:14:08,  7.15s/it]
 50%|█████     | 113173/225000 [236:09:07<227:30:49,  7.32s/it]
 50%|█████     | 113174/225000 [236:09:10<193:33:18,  6.23s/it]
 50%|█████     | 113175/225000 [236:09:16<184:35:30,  5.94s/it]
 50%|█████     | 113176/225000 [236:09:20<167:44:02,  5.40s/it]
 50%|█████     | 113177/225000 [236:09:23<148:18:06,  4.77s/it]
 50%|█████     | 113178/225000 [236:09:31<181:37:15,  5.85s/it]
 50%|█████     | 113179/225000 [236:09:34<148:59:24,  4.80s/it]
 50%|█████     | 113180/225000 [236:09:38<145:38:40,  4.69s/it]
 50%|█████     | 113181/225000 [236:09:42<140:26:11,  4.52s/it]
 50%|█████     | 113182/225000 [236:09:45<121:57:16,  3.93s/it]
 50%|█████     | 113183/225000 [236:09:49<119:47:41,  3.86s/it]
 50%|█████     | 113184/225000 [236:09:52<117:07:51,  3.77s/it]
 50%|█████     | 113185/225000 [236:09:56<116:44:53,  3.76s/it]
 50%|█████     | 113186/225000 [236:10:07<182:45:57,  5.88s/it]
 50%|█████     | 113187/225000 [236:10:09<152:10:57,  4.90s/it]
 50%|█████     | 113188/225000 [236:10:13<143:08:00,  4.61s/it]
 50%|█████     | 113189/225000 [236:10:17<139:28:17,  4.49s/it]
 50%|█████     | 113190/225000 [236:10:21<129:35:48,  4.17s/it]
 50%|█████     | 113191/225000 [236:10:25<126:27:22,  4.07s/it]
 50%|█████     | 113192/225000 [236:10:35<187:31:11,  6.04s/it]
 50%|█████     | 113193/225000 [236:10:38<156:02:49,  5.02s/it]
 50%|█████     | 113194/225000 [236:10:41<133:30:15,  4.30s/it]
 50%|█████     | 113195/225000 [236:10:43<118:28:17,  3.81s/it]
 50%|█████     | 113196/225000 [236:10:47<113:39:08,  3.66s/it]
 50%|█████     | 113197/225000 [236:10:51<117:00:07,  3.77s/it]
 50%|█████     | 113198/225000 [236:10:58<150:35:04,  4.85s/it]
 50%|█████     | 113199/225000 [236:11:04<163:35:08,  5.27s/it]
 50%|█████     | 113200/225000 [236:11:07<143:05:33,  4.61s/it]
                                                               
{'loss': '0.9105', 'grad_norm': '1.347', 'learning_rate': '2.659e-05', 'epoch': '0.5031', 'num_input_tokens_seen': 394772896, 'train_runtime': '8.503e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113200/225000 [236:11:07<143:05:33,  4.61s/it]
 50%|█████     | 113201/225000 [236:11:11<130:00:19,  4.19s/it]
 50%|█████     | 113202/225000 [236:11:14<124:35:44,  4.01s/it]
 50%|█████     | 113203/225000 [236:11:18<124:11:02,  4.00s/it]
 50%|█████     | 113204/225000 [236:11:21<111:45:42,  3.60s/it]
 50%|█████     | 113205/225000 [236:11:25<114:06:34,  3.67s/it]
 50%|█████     | 113206/225000 [236:11:32<148:45:38,  4.79s/it]
 50%|█████     | 113207/225000 [236:11:38<158:54:20,  5.12s/it]
 50%|█████     | 113208/225000 [236:11:40<133:59:26,  4.31s/it]
 50%|█████     | 113209/225000 [236:11:50<186:17:32,  6.00s/it]
 50%|█████     | 113210/225000 [236:11:54<164:05:05,  5.28s/it]
 50%|█████     | 113211/225000 [236:11:57<141:18:17,  4.55s/it]
 50%|█████     | 113212/225000 [236:12:03<160:46:12,  5.18s/it]
 50%|█████     | 113213/225000 [236:12:06<139:03:19,  4.48s/it]
 50%|█████     | 113214/225000 [236:12:09<122:52:17,  3.96s/it]
 50%|█████     | 113215/225000 [236:12:12<112:25:13,  3.62s/it]
 50%|█████     | 113216/225000 [236:12:15<108:12:07,  3.48s/it]
 50%|█████     | 113217/225000 [236:12:18<102:24:47,  3.30s/it]
 50%|█████     | 113218/225000 [236:12:32<208:13:08,  6.71s/it]
 50%|█████     | 113219/225000 [236:12:38<194:40:40,  6.27s/it]
 50%|█████     | 113220/225000 [236:12:41<164:45:48,  5.31s/it]
 50%|█████     | 113221/225000 [236:12:49<189:22:15,  6.10s/it]
 50%|█████     | 113222/225000 [236:12:52<167:19:21,  5.39s/it]
 50%|█████     | 113223/225000 [236:12:56<150:34:38,  4.85s/it]
 50%|█████     | 113224/225000 [236:13:07<208:31:01,  6.72s/it]
 50%|█████     | 113225/225000 [236:13:10<176:11:11,  5.67s/it]
 50%|█████     | 113226/225000 [236:13:14<153:41:09,  4.95s/it]
 50%|█████     | 113227/225000 [236:13:18<145:48:31,  4.70s/it]
 50%|█████     | 113228/225000 [236:13:24<162:53:58,  5.25s/it]
 50%|█████     | 113229/225000 [236:13:27<138:51:23,  4.47s/it]
 50%|█████     | 113230/225000 [236:13:30<128:31:57,  4.14s/it]
 50%|█████     | 113231/225000 [236:13:34<120:28:37,  3.88s/it]
 50%|█████     | 113232/225000 [236:13:39<133:26:31,  4.30s/it]
 50%|█████     | 113233/225000 [236:13:46<164:37:45,  5.30s/it]
 50%|█████     | 113234/225000 [236:13:51<157:22:17,  5.07s/it]
 50%|█████     | 113235/225000 [236:13:54<140:21:03,  4.52s/it]
 50%|█████     | 113236/225000 [236:13:57<122:45:59,  3.95s/it]
 50%|█████     | 113237/225000 [236:14:07<176:03:18,  5.67s/it]
 50%|█████     | 113238/225000 [236:14:11<164:55:56,  5.31s/it]
 50%|█████     | 113239/225000 [236:14:18<175:59:18,  5.67s/it]
 50%|█████     | 113240/225000 [236:14:21<159:36:12,  5.14s/it]
 50%|█████     | 113241/225000 [236:14:25<141:28:24,  4.56s/it]
 50%|█████     | 113242/225000 [236:14:28<131:42:15,  4.24s/it]
 50%|█████     | 113243/225000 [236:14:32<132:32:45,  4.27s/it]
 50%|█████     | 113244/225000 [236:14:36<128:57:43,  4.15s/it]
 50%|█████     | 113245/225000 [236:14:40<122:22:41,  3.94s/it]
 50%|█████     | 113246/225000 [236:14:43<115:53:35,  3.73s/it]
 50%|█████     | 113247/225000 [236:14:46<113:12:18,  3.65s/it]
 50%|█████     | 113248/225000 [236:14:57<180:43:03,  5.82s/it]
 50%|█████     | 113249/225000 [236:15:00<153:52:35,  4.96s/it]
 50%|█████     | 113250/225000 [236:15:06<158:09:15,  5.09s/it]
                                                               
{'loss': '0.9416', 'grad_norm': '1.939', 'learning_rate': '2.657e-05', 'epoch': '0.5033', 'num_input_tokens_seen': 394873456, 'train_runtime': '8.505e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113250/225000 [236:15:06<158:09:15,  5.09s/it]
 50%|█████     | 113251/225000 [236:15:09<141:30:09,  4.56s/it]
 50%|█████     | 113252/225000 [236:15:12<127:49:10,  4.12s/it]
 50%|█████     | 113253/225000 [236:15:15<115:15:13,  3.71s/it]
 50%|█████     | 113254/225000 [236:15:19<116:56:05,  3.77s/it]
 50%|█████     | 113255/225000 [236:15:23<116:26:21,  3.75s/it]
 50%|█████     | 113256/225000 [236:15:26<113:05:28,  3.64s/it]
 50%|█████     | 113257/225000 [236:15:36<174:02:02,  5.61s/it]
 50%|█████     | 113258/225000 [236:15:40<157:30:07,  5.07s/it]
 50%|█████     | 113259/225000 [236:15:44<150:18:45,  4.84s/it]
 50%|█████     | 113260/225000 [236:15:49<146:45:29,  4.73s/it]
 50%|█████     | 113261/225000 [236:15:52<130:28:38,  4.20s/it]
 50%|█████     | 113262/225000 [236:15:55<121:38:24,  3.92s/it]
 50%|█████     | 113263/225000 [236:15:59<123:11:30,  3.97s/it]
 50%|█████     | 113264/225000 [236:16:09<175:51:41,  5.67s/it]
 50%|█████     | 113265/225000 [236:16:13<165:20:37,  5.33s/it]
 50%|█████     | 113266/225000 [236:16:16<146:35:50,  4.72s/it]
 50%|█████     | 113267/225000 [236:16:20<133:08:26,  4.29s/it]
 50%|█████     | 113268/225000 [236:16:23<125:33:02,  4.05s/it]
 50%|█████     | 113269/225000 [236:16:27<123:17:42,  3.97s/it]
 50%|█████     | 113270/225000 [236:16:35<163:03:38,  5.25s/it]
 50%|█████     | 113271/225000 [236:16:44<191:06:08,  6.16s/it]
 50%|█████     | 113272/225000 [236:16:48<172:22:14,  5.55s/it]
 50%|█████     | 113273/225000 [236:16:51<155:52:50,  5.02s/it]
 50%|█████     | 113274/225000 [236:16:55<138:13:27,  4.45s/it]
 50%|█████     | 113275/225000 [236:17:05<189:08:39,  6.09s/it]
 50%|█████     | 113276/225000 [236:17:07<158:47:36,  5.12s/it]
 50%|█████     | 113277/225000 [236:17:11<146:53:18,  4.73s/it]
 50%|█████     | 113278/225000 [236:17:15<133:49:11,  4.31s/it]
 50%|█████     | 113279/225000 [236:17:17<120:52:32,  3.89s/it]
 50%|█████     | 113280/225000 [236:17:23<132:47:00,  4.28s/it]
 50%|█████     | 113281/225000 [236:17:32<183:31:42,  5.91s/it]
 50%|█████     | 113282/225000 [236:17:35<154:22:44,  4.97s/it]
 50%|█████     | 113283/225000 [236:17:38<135:32:32,  4.37s/it]
 50%|█████     | 113284/225000 [236:17:41<123:06:18,  3.97s/it]
 50%|█████     | 113285/225000 [236:17:44<113:50:40,  3.67s/it]
 50%|█████     | 113286/225000 [236:17:48<113:08:49,  3.65s/it]
 50%|█████     | 113287/225000 [236:17:51<108:44:35,  3.50s/it]
 50%|█████     | 113288/225000 [236:18:01<169:16:43,  5.46s/it]
 50%|█████     | 113289/225000 [236:18:09<197:09:12,  6.35s/it]
 50%|█████     | 113290/225000 [236:18:12<161:56:22,  5.22s/it]
 50%|█████     | 113291/225000 [236:18:14<137:19:32,  4.43s/it]
 50%|█████     | 113292/225000 [236:18:18<131:48:55,  4.25s/it]
 50%|█████     | 113293/225000 [236:18:25<152:34:39,  4.92s/it]
 50%|█████     | 113294/225000 [236:18:35<204:12:59,  6.58s/it]
 50%|█████     | 113295/225000 [236:18:41<196:35:00,  6.34s/it]
 50%|█████     | 113296/225000 [236:18:44<162:25:09,  5.23s/it]
 50%|█████     | 113297/225000 [236:18:46<138:28:40,  4.46s/it]
 50%|█████     | 113298/225000 [236:18:49<122:52:16,  3.96s/it]
 50%|█████     | 113299/225000 [236:18:52<116:25:51,  3.75s/it]
 50%|█████     | 113300/225000 [236:18:55<109:23:44,  3.53s/it]
                                                               
{'loss': '0.9408', 'grad_norm': '1.871', 'learning_rate': '2.655e-05', 'epoch': '0.5036', 'num_input_tokens_seen': 394975504, 'train_runtime': '8.507e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113300/225000 [236:18:55<109:23:44,  3.53s/it]
 50%|█████     | 113301/225000 [236:19:03<152:00:41,  4.90s/it]
 50%|█████     | 113302/225000 [236:19:08<149:44:49,  4.83s/it]
 50%|█████     | 113303/225000 [236:19:11<134:11:20,  4.32s/it]
 50%|█████     | 113304/225000 [236:19:15<129:08:37,  4.16s/it]
 50%|█████     | 113305/225000 [236:19:24<173:13:11,  5.58s/it]
 50%|█████     | 113306/225000 [236:19:27<150:56:03,  4.86s/it]
 50%|█████     | 113307/225000 [236:19:30<131:20:24,  4.23s/it]
 50%|█████     | 113308/225000 [236:19:34<126:47:43,  4.09s/it]
 50%|█████     | 113309/225000 [236:19:43<178:50:59,  5.76s/it]
 50%|█████     | 113310/225000 [236:19:46<154:21:27,  4.98s/it]
 50%|█████     | 113311/225000 [236:19:50<141:13:22,  4.55s/it]
 50%|█████     | 113312/225000 [236:19:59<186:04:42,  6.00s/it]
 50%|█████     | 113313/225000 [236:20:03<164:58:07,  5.32s/it]
 50%|█████     | 113314/225000 [236:20:09<172:52:32,  5.57s/it]
 50%|█████     | 113315/225000 [236:20:12<144:22:47,  4.65s/it]
 50%|█████     | 113316/225000 [236:20:19<172:14:45,  5.55s/it]
 50%|█████     | 113317/225000 [236:20:26<181:16:20,  5.84s/it]
 50%|█████     | 113318/225000 [236:20:31<173:53:16,  5.61s/it]
 50%|█████     | 113319/225000 [236:20:35<153:57:41,  4.96s/it]
 50%|█████     | 113320/225000 [236:20:42<173:36:43,  5.60s/it]
 50%|█████     | 113321/225000 [236:20:51<212:15:00,  6.84s/it]
 50%|█████     | 113322/225000 [236:20:54<169:20:13,  5.46s/it]
 50%|█████     | 113323/225000 [236:20:56<145:31:17,  4.69s/it]
 50%|█████     | 113324/225000 [236:21:00<131:42:10,  4.25s/it]
 50%|█████     | 113325/225000 [236:21:04<129:28:47,  4.17s/it]
 50%|█████     | 113326/225000 [236:21:07<121:19:34,  3.91s/it]
 50%|█████     | 113327/225000 [236:21:17<175:26:08,  5.66s/it]
 50%|█████     | 113328/225000 [236:21:21<163:46:25,  5.28s/it]
 50%|█████     | 113329/225000 [236:21:26<164:04:11,  5.29s/it]
 50%|█████     | 113330/225000 [236:21:30<144:04:13,  4.64s/it]
 50%|█████     | 113331/225000 [236:21:33<132:12:22,  4.26s/it]
 50%|█████     | 113332/225000 [236:21:36<123:12:28,  3.97s/it]
 50%|█████     | 113333/225000 [236:21:40<118:44:15,  3.83s/it]
 50%|█████     | 113334/225000 [236:21:43<110:27:24,  3.56s/it]
 50%|█████     | 113335/225000 [236:21:46<107:14:09,  3.46s/it]
 50%|█████     | 113336/225000 [236:21:49<100:44:02,  3.25s/it]
 50%|█████     | 113337/225000 [236:21:52<105:08:16,  3.39s/it]
 50%|█████     | 113338/225000 [236:22:02<161:08:16,  5.20s/it]
 50%|█████     | 113339/225000 [236:22:07<161:06:01,  5.19s/it]
 50%|█████     | 113340/225000 [236:22:10<138:04:07,  4.45s/it]
 50%|█████     | 113341/225000 [236:22:16<152:17:34,  4.91s/it]
 50%|█████     | 113342/225000 [236:22:18<131:58:57,  4.26s/it]
 50%|█████     | 113343/225000 [236:22:23<134:14:26,  4.33s/it]
 50%|█████     | 113344/225000 [236:22:26<124:47:28,  4.02s/it]
 50%|█████     | 113345/225000 [236:22:29<116:48:11,  3.77s/it]
 50%|█████     | 113346/225000 [236:22:33<115:35:03,  3.73s/it]
 50%|█████     | 113347/225000 [236:22:43<178:19:01,  5.75s/it]
 50%|█████     | 113348/225000 [236:22:47<157:04:04,  5.06s/it]
 50%|█████     | 113349/225000 [236:22:50<134:33:20,  4.34s/it]
 50%|█████     | 113350/225000 [236:22:53<127:52:49,  4.12s/it]
                                                               
{'loss': '0.9344', 'grad_norm': '1.053', 'learning_rate': '2.653e-05', 'epoch': '0.5038', 'num_input_tokens_seen': 395075912, 'train_runtime': '8.51e+05', 'train_tokens_per_second': '464.3'}

 50%|█████     | 113350/225000 [236:22:53<127:52:49,  4.12s/it]
 50%|█████     | 113351/225000 [236:23:04<190:06:23,  6.13s/it]
 50%|█████     | 113352/225000 [236:23:07<161:46:58,  5.22s/it]
 50%|█████     | 113353/225000 [236:23:10<139:10:17,  4.49s/it]
 50%|█████     | 113354/225000 [236:23:19<185:59:12,  6.00s/it]
 50%|█████     | 113355/225000 [236:23:24<174:43:20,  5.63s/it]
 50%|█████     | 113356/225000 [236:23:27<150:26:46,  4.85s/it]
 50%|█████     | 113357/225000 [236:23:31<142:49:31,  4.61s/it]
 50%|█████     | 113358/225000 [236:23:34<128:37:04,  4.15s/it]
 50%|█████     | 113359/225000 [236:23:37<118:03:52,  3.81s/it]
 50%|█████     | 113360/225000 [236:23:41<116:40:43,  3.76s/it]
 50%|█████     | 113361/225000 [236:23:44<112:40:05,  3.63s/it]
 50%|█████     | 113362/225000 [236:23:52<150:04:33,  4.84s/it]
 50%|█████     | 113363/225000 [236:23:59<168:00:53,  5.42s/it]
 50%|█████     | 113364/225000 [236:24:02<151:23:24,  4.88s/it]
 50%|█████     | 113365/225000 [236:24:05<131:04:31,  4.23s/it]
 50%|█████     | 113366/225000 [236:24:08<120:37:15,  3.89s/it]
 50%|█████     | 113367/225000 [236:24:15<147:44:17,  4.76s/it]
 50%|█████     | 113368/225000 [236:24:27<216:49:05,  6.99s/it]
 50%|█████     | 113369/225000 [236:24:31<184:38:25,  5.95s/it]
 50%|█████     | 113370/225000 [236:24:33<153:54:29,  4.96s/it]
 50%|█████     | 113371/225000 [236:24:41<177:06:44,  5.71s/it]
 50%|█████     | 113372/225000 [236:24:45<160:54:04,  5.19s/it]
 50%|█████     | 113373/225000 [236:24:48<139:01:33,  4.48s/it]
 50%|█████     | 113374/225000 [236:24:53<149:07:37,  4.81s/it]
 50%|█████     | 113375/225000 [236:25:04<201:36:58,  6.50s/it]
 50%|█████     | 113376/225000 [236:25:06<167:12:59,  5.39s/it]
 50%|█████     | 113377/225000 [236:25:13<173:42:36,  5.60s/it]
 50%|█████     | 113378/225000 [236:25:17<159:53:19,  5.16s/it]
 50%|█████     | 113379/225000 [236:25:20<140:00:48,  4.52s/it]
 50%|█████     | 113380/225000 [236:25:23<128:11:25,  4.13s/it]
 50%|█████     | 113381/225000 [236:25:26<120:35:52,  3.89s/it]
 50%|█████     | 113382/225000 [236:25:36<175:20:36,  5.66s/it]
 50%|█████     | 113383/225000 [236:25:39<150:13:23,  4.85s/it]
 50%|█████     | 113384/225000 [236:25:42<132:09:37,  4.26s/it]
 50%|█████     | 113385/225000 [236:25:47<144:16:28,  4.65s/it]
 50%|█████     | 113386/225000 [236:25:53<154:27:48,  4.98s/it]
 50%|█████     | 113387/225000 [236:25:57<147:08:42,  4.75s/it]
 50%|█████     | 113388/225000 [236:26:08<201:03:58,  6.49s/it]
 50%|█████     | 113389/225000 [236:26:11<169:25:06,  5.46s/it]
 50%|█████     | 113390/225000 [236:26:15<153:06:38,  4.94s/it]
 50%|█████     | 113391/225000 [236:26:19<150:28:02,  4.85s/it]
 50%|█████     | 113392/225000 [236:26:23<137:29:23,  4.43s/it]
 50%|█████     | 113393/225000 [236:26:26<128:48:08,  4.15s/it]
 50%|█████     | 113394/225000 [236:26:30<126:02:37,  4.07s/it]
 50%|█████     | 113395/225000 [236:26:40<181:50:41,  5.87s/it]
 50%|█████     | 113396/225000 [236:26:43<151:54:47,  4.90s/it]
 50%|█████     | 113397/225000 [236:26:45<129:46:32,  4.19s/it]
 50%|█████     | 113398/225000 [236:26:48<113:29:45,  3.66s/it]
 50%|█████     | 113399/225000 [236:26:57<164:30:44,  5.31s/it]
 50%|█████     | 113400/225000 [236:27:03<168:09:11,  5.42s/it]
                                                               
{'loss': '0.9472', 'grad_norm': '1.306', 'learning_rate': '2.652e-05', 'epoch': '0.504', 'num_input_tokens_seen': 395179160, 'train_runtime': '8.512e+05', 'train_tokens_per_second': '464.2'}

 50%|█████     | 113400/225000 [236:27:03<168:09:11,  5.42s/it]
 50%|█████     | 113401/225000 [236:27:07<156:38:49,  5.05s/it]
 50%|█████     | 113402/225000 [236:27:11<144:07:23,  4.65s/it]
 50%|█████     | 113403/225000 [236:27:15<140:29:26,  4.53s/it]
 50%|█████     | 113404/225000 [236:27:18<130:47:11,  4.22s/it]
 50%|█████     | 113405/225000 [236:27:29<190:34:35,  6.15s/it]
 50%|█████     | 113406/225000 [236:27:33<166:51:56,  5.38s/it]
 50%|█████     | 113407/225000 [236:27:35<140:55:07,  4.55s/it]
 50%|█████     | 113408/225000 [236:27:38<127:57:16,  4.13s/it]
 50%|█████     | 113409/225000 [236:27:42<128:12:21,  4.14s/it]
 50%|█████     | 113410/225000 [236:27:46<126:45:09,  4.09s/it]
 50%|█████     | 113411/225000 [236:27:50<118:47:56,  3.83s/it]
 50%|█████     | 113412/225000 [236:27:53<115:56:12,  3.74s/it]
 50%|█████     | 113413/225000 [236:27:56<108:45:40,  3.51s/it]
 50%|█████     | 113414/225000 [236:28:06<167:51:45,  5.42s/it]
 50%|█████     | 113415/225000 [236:28:09<147:31:52,  4.76s/it]
 50%|█████     | 113416/225000 [236:28:12<129:46:46,  4.19s/it]
 50%|█████     | 113417/225000 [236:28:16<125:34:19,  4.05s/it]
 50%|█████     | 113418/225000 [236:28:20<128:25:34,  4.14s/it]
 50%|█████     | 113419/225000 [236:28:23<117:54:51,  3.80s/it]
 50%|█████     | 113420/225000 [236:28:33<174:12:11,  5.62s/it]
 50%|█████     | 113421/225000 [236:28:37<158:02:13,  5.10s/it]
 50%|█████     | 113422/225000 [236:28:42<159:19:23,  5.14s/it]
 50%|█████     | 113423/225000 [236:28:48<161:11:37,  5.20s/it]
 50%|█████     | 113424/225000 [236:28:52<150:39:48,  4.86s/it]
 50%|█████     | 113425/225000 [236:28:55<140:20:51,  4.53s/it]
 50%|█████     | 113426/225000 [236:28:59<131:01:20,  4.23s/it]
 50%|█████     | 113427/225000 [236:29:03<126:56:46,  4.10s/it]
 50%|█████     | 113428/225000 [236:29:06<119:57:15,  3.87s/it]
 50%|█████     | 113429/225000 [236:29:10<120:45:51,  3.90s/it]
 50%|█████     | 113430/225000 [236:29:13<112:53:38,  3.64s/it]
 50%|█████     | 113431/225000 [236:29:17<112:49:02,  3.64s/it]
 50%|█████     | 113432/225000 [236:29:21<120:25:31,  3.89s/it]
 50%|█████     | 113433/225000 [236:29:26<133:00:39,  4.29s/it]
 50%|█████     | 113434/225000 [236:29:30<122:58:22,  3.97s/it]
 50%|█████     | 113435/225000 [236:29:33<119:57:26,  3.87s/it]
 50%|█████     | 113436/225000 [236:29:44<183:24:14,  5.92s/it]
 50%|█████     | 113437/225000 [236:29:51<197:59:42,  6.39s/it]
 50%|█████     | 113438/225000 [236:30:01<227:22:26,  7.34s/it]
 50%|█████     | 113439/225000 [236:30:07<216:17:18,  6.98s/it]
 50%|█████     | 113440/225000 [236:30:11<183:45:27,  5.93s/it]
 50%|█████     | 113441/225000 [236:30:15<171:53:16,  5.55s/it]
 50%|█████     | 113442/225000 [236:30:18<148:16:56,  4.79s/it]
 50%|█████     | 113443/225000 [236:30:30<208:40:10,  6.73s/it]
 50%|█████     | 113444/225000 [236:30:33<176:01:43,  5.68s/it]
 50%|█████     | 113445/225000 [236:30:36<155:30:59,  5.02s/it]
 50%|█████     | 113446/225000 [236:30:40<141:42:43,  4.57s/it]
 50%|█████     | 113447/225000 [236:30:43<130:08:58,  4.20s/it]
 50%|█████     | 113448/225000 [236:30:54<191:05:00,  6.17s/it]
 50%|█████     | 113449/225000 [236:30:58<169:28:01,  5.47s/it]
 50%|█████     | 113450/225000 [236:31:05<182:25:25,  5.89s/it]
                                                               
{'loss': '0.935', 'grad_norm': '1.433', 'learning_rate': '2.65e-05', 'epoch': '0.5042', 'num_input_tokens_seen': 395284992, 'train_runtime': '8.515e+05', 'train_tokens_per_second': '464.2'}

 50%|█████     | 113450/225000 [236:31:05<182:25:25,  5.89s/it]
 50%|█████     | 113451/225000 [236:31:08<160:59:08,  5.20s/it]
 50%|█████     | 113452/225000 [236:31:11<139:13:34,  4.49s/it]
 50%|█████     | 113453/225000 [236:31:14<125:31:19,  4.05s/it]
 50%|█████     | 113454/225000 [236:31:18<125:30:24,  4.05s/it]
 50%|█████     | 113455/225000 [236:31:24<145:25:02,  4.69s/it]
 50%|█████     | 113456/225000 [236:31:38<225:15:00,  7.27s/it]
 50%|█████     | 113457/225000 [236:31:41<187:44:28,  6.06s/it]
 50%|█████     | 113458/225000 [236:31:44<159:05:42,  5.13s/it]
 50%|█████     | 113459/225000 [236:31:47<137:28:54,  4.44s/it]
 50%|█████     | 113460/225000 [236:31:50<125:44:57,  4.06s/it]
 50%|█████     | 113461/225000 [236:31:53<118:47:25,  3.83s/it]
 50%|█████     | 113462/225000 [236:32:01<155:17:24,  5.01s/it]
 50%|█████     | 113463/225000 [236:32:04<142:46:10,  4.61s/it]
 50%|█████     | 113464/225000 [236:32:08<132:25:12,  4.27s/it]
 50%|█████     | 113465/225000 [236:32:11<121:12:26,  3.91s/it]
 50%|█████     | 113466/225000 [236:32:14<115:39:36,  3.73s/it]
 50%|█████     | 113467/225000 [236:32:17<106:01:09,  3.42s/it]
 50%|█████     | 113468/225000 [236:32:21<113:34:45,  3.67s/it]
 50%|█████     | 113469/225000 [236:32:27<130:57:39,  4.23s/it]
 50%|█████     | 113470/225000 [236:32:30<124:50:48,  4.03s/it]
 50%|█████     | 113471/225000 [236:32:34<116:45:37,  3.77s/it]
 50%|█████     | 113472/225000 [236:32:42<157:08:31,  5.07s/it]
 50%|█████     | 113473/225000 [236:32:50<187:22:59,  6.05s/it]
 50%|█████     | 113474/225000 [236:32:53<156:06:55,  5.04s/it]
 50%|█████     | 113475/225000 [236:32:56<139:57:50,  4.52s/it]
 50%|█████     | 113476/225000 [236:33:03<160:05:15,  5.17s/it]
 50%|█████     | 113477/225000 [236:33:06<143:30:05,  4.63s/it]
 50%|█████     | 113478/225000 [236:33:11<142:37:16,  4.60s/it]
 50%|█████     | 113479/225000 [236:33:14<134:57:34,  4.36s/it]
 50%|█████     | 113480/225000 [236:33:18<128:45:17,  4.16s/it]
 50%|█████     | 113481/225000 [236:33:30<199:37:36,  6.44s/it]
 50%|█████     | 113482/225000 [236:33:34<174:03:32,  5.62s/it]
 50%|█████     | 113483/225000 [236:33:37<151:20:28,  4.89s/it]
 50%|█████     | 113484/225000 [236:33:40<136:47:33,  4.42s/it]
 50%|█████     | 113485/225000 [236:33:43<127:07:19,  4.10s/it]
 50%|█████     | 113486/225000 [236:33:47<117:51:51,  3.81s/it]
 50%|█████     | 113487/225000 [236:33:53<146:44:45,  4.74s/it]
 50%|█████     | 113488/225000 [236:34:02<178:46:48,  5.77s/it]
 50%|█████     | 113489/225000 [236:34:05<158:10:53,  5.11s/it]
 50%|█████     | 113490/225000 [236:34:08<136:59:57,  4.42s/it]
 50%|█████     | 113491/225000 [236:34:11<128:10:58,  4.14s/it]
 50%|█████     | 113492/225000 [236:34:15<120:07:22,  3.88s/it]
 50%|█████     | 113493/225000 [236:34:23<163:55:28,  5.29s/it]
 50%|█████     | 113494/225000 [236:34:30<172:39:44,  5.57s/it]
 50%|█████     | 113495/225000 [236:34:32<148:00:35,  4.78s/it]
 50%|█████     | 113496/225000 [236:34:36<139:55:23,  4.52s/it]
 50%|█████     | 113497/225000 [236:34:40<131:43:50,  4.25s/it]
 50%|█████     | 113498/225000 [236:34:43<121:11:05,  3.91s/it]
 50%|█████     | 113499/225000 [236:34:52<164:49:54,  5.32s/it]
 50%|█████     | 113500/225000 [236:34:57<159:52:29,  5.16s/it]
                                                               
{'loss': '0.9384', 'grad_norm': '1.218', 'learning_rate': '2.648e-05', 'epoch': '0.5044', 'num_input_tokens_seen': 395390344, 'train_runtime': '8.517e+05', 'train_tokens_per_second': '464.2'}

 50%|█████     | 113500/225000 [236:34:57<159:52:29,  5.16s/it]
 50%|█████     | 113501/225000 [236:35:01<157:17:39,  5.08s/it]
 50%|█████     | 113502/225000 [236:35:05<140:23:55,  4.53s/it]
 50%|█████     | 113503/225000 [236:35:09<138:13:44,  4.46s/it]
 50%|█████     | 113504/225000 [236:35:13<132:04:50,  4.26s/it]
 50%|█████     | 113505/225000 [236:35:16<126:09:28,  4.07s/it]
 50%|█████     | 113506/225000 [236:35:21<134:37:11,  4.35s/it]
 50%|█████     | 113507/225000 [236:35:31<187:24:39,  6.05s/it]
 50%|█████     | 113508/225000 [236:35:34<155:31:44,  5.02s/it]
 50%|█████     | 113509/225000 [236:35:39<150:42:06,  4.87s/it]
 50%|█████     | 113510/225000 [236:35:42<141:41:37,  4.58s/it]
 50%|█████     | 113511/225000 [236:35:46<131:05:09,  4.23s/it]
 50%|█████     | 113512/225000 [236:35:51<136:23:52,  4.40s/it]
 50%|█████     | 113513/225000 [236:35:55<131:46:39,  4.26s/it]
 50%|█████     | 113514/225000 [236:35:58<126:43:28,  4.09s/it]
 50%|█████     | 113515/225000 [236:36:08<179:12:47,  5.79s/it]
 50%|█████     | 113516/225000 [236:36:13<175:23:54,  5.66s/it]
 50%|█████     | 113517/225000 [236:36:16<150:37:56,  4.86s/it]
 50%|█████     | 113518/225000 [236:36:20<134:29:28,  4.34s/it]
 50%|█████     | 113519/225000 [236:36:24<131:57:16,  4.26s/it]
 50%|█████     | 113520/225000 [236:36:31<159:36:51,  5.15s/it]
 50%|█████     | 113521/225000 [236:36:34<145:21:33,  4.69s/it]
 50%|█████     | 113522/225000 [236:36:37<128:49:54,  4.16s/it]
 50%|█████     | 113523/225000 [236:36:41<124:42:14,  4.03s/it]
 50%|█████     | 113524/225000 [236:36:45<123:11:30,  3.98s/it]
 50%|█████     | 113525/225000 [236:36:49<123:05:54,  3.98s/it]
 50%|█████     | 113526/225000 [236:36:56<147:51:46,  4.78s/it]
 50%|█████     | 113527/225000 [236:37:04<183:08:31,  5.91s/it]
 50%|█████     | 113528/225000 [236:37:07<155:07:34,  5.01s/it]
 50%|█████     | 113529/225000 [236:37:11<140:43:01,  4.54s/it]
 50%|█████     | 113530/225000 [236:37:14<127:36:54,  4.12s/it]
 50%|█████     | 113531/225000 [236:37:18<125:21:37,  4.05s/it]
 50%|█████     | 113532/225000 [236:37:27<175:59:20,  5.68s/it]
 50%|█████     | 113533/225000 [236:37:32<168:19:22,  5.44s/it]
 50%|█████     | 113534/225000 [236:37:34<141:24:00,  4.57s/it]
 50%|█████     | 113535/225000 [236:37:43<175:27:22,  5.67s/it]
 50%|█████     | 113536/225000 [236:37:46<155:40:10,  5.03s/it]
 50%|█████     | 113537/225000 [236:37:49<136:06:34,  4.40s/it]
 50%|█████     | 113538/225000 [236:37:52<120:46:36,  3.90s/it]
 50%|█████     | 113539/225000 [236:37:58<141:05:04,  4.56s/it]
 50%|█████     | 113540/225000 [236:38:04<153:34:05,  4.96s/it]
 50%|█████     | 113541/225000 [236:38:07<139:25:03,  4.50s/it]
 50%|█████     | 113542/225000 [236:38:10<125:21:10,  4.05s/it]
 50%|█████     | 113543/225000 [236:38:16<144:37:28,  4.67s/it]
 50%|█████     | 113544/225000 [236:38:22<152:29:32,  4.93s/it]
 50%|█████     | 113545/225000 [236:38:26<146:20:24,  4.73s/it]
 50%|█████     | 113546/225000 [236:38:30<136:27:21,  4.41s/it]
 50%|█████     | 113547/225000 [236:38:40<188:13:38,  6.08s/it]
 50%|█████     | 113548/225000 [236:38:43<164:20:27,  5.31s/it]
 50%|█████     | 113549/225000 [236:38:46<143:33:02,  4.64s/it]
 50%|█████     | 113550/225000 [236:38:49<127:13:03,  4.11s/it]
                                                               
{'loss': '0.9186', 'grad_norm': '1.316', 'learning_rate': '2.646e-05', 'epoch': '0.5047', 'num_input_tokens_seen': 395493560, 'train_runtime': '8.519e+05', 'train_tokens_per_second': '464.2'}

 50%|█████     | 113550/225000 [236:38:49<127:13:03,  4.11s/it]
 50%|█████     | 113551/225000 [236:38:55<142:10:21,  4.59s/it]
 50%|█████     | 113552/225000 [236:39:07<208:46:58,  6.74s/it]
 50%|█████     | 113553/225000 [236:39:09<170:43:50,  5.52s/it]
 50%|█████     | 113554/225000 [236:39:12<147:51:00,  4.78s/it]
 50%|█████     | 113555/225000 [236:39:16<132:31:16,  4.28s/it]
 50%|█████     | 113556/225000 [236:39:26<186:42:16,  6.03s/it]
 50%|█████     | 113557/225000 [236:39:28<153:34:55,  4.96s/it]
 50%|█████     | 113558/225000 [236:39:31<133:34:04,  4.31s/it]
 50%|█████     | 113559/225000 [236:39:33<114:23:00,  3.70s/it]
 50%|█████     | 113560/225000 [236:39:36<107:33:09,  3.47s/it]
 50%|█████     | 113561/225000 [236:39:40<111:21:02,  3.60s/it]
 50%|█████     | 113562/225000 [236:39:50<174:25:16,  5.63s/it]
 50%|█████     | 113563/225000 [236:39:54<152:24:55,  4.92s/it]
 50%|█████     | 113564/225000 [236:39:57<135:45:45,  4.39s/it]
 50%|█████     | 113565/225000 [236:40:00<121:33:35,  3.93s/it]
 50%|█████     | 113566/225000 [236:40:04<128:05:29,  4.14s/it]
 50%|█████     | 113567/225000 [236:40:12<159:16:17,  5.15s/it]
 50%|█████     | 113568/225000 [236:40:16<153:15:21,  4.95s/it]
 50%|█████     | 113569/225000 [236:40:20<139:52:51,  4.52s/it]
 50%|█████     | 113570/225000 [236:40:24<137:24:39,  4.44s/it]
 50%|█████     | 113571/225000 [236:40:34<189:08:25,  6.11s/it]
 50%|█████     | 113572/225000 [236:40:37<161:07:44,  5.21s/it]
 50%|█████     | 113573/225000 [236:40:42<156:19:17,  5.05s/it]
 50%|█████     | 113574/225000 [236:40:45<134:52:27,  4.36s/it]
 50%|█████     | 113575/225000 [236:40:48<123:07:19,  3.98s/it]
 50%|█████     | 113576/225000 [236:40:52<123:47:29,  4.00s/it]
 50%|█████     | 113577/225000 [236:40:56<123:26:36,  3.99s/it]
 50%|█████     | 113578/225000 [236:40:59<119:45:51,  3.87s/it]
 50%|█████     | 113579/225000 [236:41:08<163:53:01,  5.30s/it]
 50%|█████     | 113580/225000 [236:41:14<173:31:30,  5.61s/it]
 50%|█████     | 113581/225000 [236:41:17<144:36:52,  4.67s/it]
 50%|█████     | 113582/225000 [236:41:27<195:53:11,  6.33s/it]
 50%|█████     | 113583/225000 [236:41:30<166:06:03,  5.37s/it]
 50%|█████     | 113584/225000 [236:41:34<151:33:52,  4.90s/it]
 50%|█████     | 113585/225000 [236:41:37<136:31:08,  4.41s/it]
 50%|█████     | 113586/225000 [236:41:40<125:57:49,  4.07s/it]
 50%|█████     | 113587/225000 [236:41:51<186:26:47,  6.02s/it]
 50%|█████     | 113588/225000 [236:41:55<167:06:57,  5.40s/it]
 50%|█████     | 113589/225000 [236:41:59<151:04:12,  4.88s/it]
 50%|█████     | 113590/225000 [236:42:01<131:56:27,  4.26s/it]
 50%|█████     | 113591/225000 [236:42:05<124:34:49,  4.03s/it]
 50%|█████     | 113592/225000 [236:42:08<118:46:41,  3.84s/it]
 50%|█████     | 113593/225000 [236:42:12<116:01:53,  3.75s/it]
 50%|█████     | 113594/225000 [236:42:15<109:51:26,  3.55s/it]
 50%|█████     | 113595/225000 [236:42:22<142:49:50,  4.62s/it]
 50%|█████     | 113596/225000 [236:42:29<160:55:00,  5.20s/it]
 50%|█████     | 113597/225000 [236:42:32<142:14:11,  4.60s/it]
 50%|█████     | 113598/225000 [236:42:35<131:16:39,  4.24s/it]
 50%|█████     | 113599/225000 [236:42:39<128:09:51,  4.14s/it]
 50%|█████     | 113600/225000 [236:42:43<121:29:00,  3.93s/it]
                                                               
{'loss': '0.9435', 'grad_norm': '1.602', 'learning_rate': '2.644e-05', 'epoch': '0.5049', 'num_input_tokens_seen': 395592176, 'train_runtime': '8.522e+05', 'train_tokens_per_second': '464.2'}

 50%|█████     | 113600/225000 [236:42:43<121:29:00,  3.93s/it]
 50%|█████     | 113601/225000 [236:42:46<117:44:45,  3.81s/it]
 50%|█████     | 113602/225000 [236:42:50<119:40:07,  3.87s/it]
 50%|█████     | 113603/225000 [236:42:54<119:39:14,  3.87s/it]
 50%|█████     | 113604/225000 [236:43:03<169:29:56,  5.48s/it]
 50%|█████     | 113605/225000 [236:43:15<225:44:04,  7.30s/it]
 50%|█████     | 113606/225000 [236:43:17<177:47:08,  5.75s/it]
 50%|█████     | 113607/225000 [236:43:20<152:39:16,  4.93s/it]
 50%|█████     | 113608/225000 [236:43:23<136:16:39,  4.40s/it]
 50%|█████     | 113609/225000 [236:43:33<185:44:13,  6.00s/it]
 50%|█████     | 113610/225000 [236:43:36<160:39:24,  5.19s/it]
 50%|█████     | 113611/225000 [236:43:39<142:37:32,  4.61s/it]
 50%|█████     | 113612/225000 [236:43:43<137:53:51,  4.46s/it]
 50%|█████     | 113613/225000 [236:43:46<124:03:47,  4.01s/it]
 50%|█████     | 113614/225000 [236:43:50<119:07:35,  3.85s/it]
 50%|█████     | 113615/225000 [236:43:54<122:42:54,  3.97s/it]
 50%|█████     | 113616/225000 [236:43:58<123:43:02,  4.00s/it]
 50%|█████     | 113617/225000 [236:44:02<124:58:23,  4.04s/it]
 50%|█████     | 113618/225000 [236:44:06<117:37:54,  3.80s/it]
 50%|█████     | 113619/225000 [236:44:10<125:26:16,  4.05s/it]
 50%|█████     | 113620/225000 [236:44:14<120:41:05,  3.90s/it]
 50%|█████     | 113621/225000 [236:44:18<119:16:35,  3.86s/it]
 50%|█████     | 113622/225000 [236:44:22<124:08:00,  4.01s/it]
 50%|█████     | 113623/225000 [236:44:27<131:58:40,  4.27s/it]
 50%|█████     | 113624/225000 [236:44:35<170:19:37,  5.51s/it]
 50%|█████     | 113625/225000 [236:44:41<171:47:47,  5.55s/it]
 51%|█████     | 113626/225000 [236:44:46<169:11:01,  5.47s/it]
 51%|█████     | 113627/225000 [236:44:54<196:21:13,  6.35s/it]
 51%|█████     | 113628/225000 [236:44:57<164:38:57,  5.32s/it]
 51%|█████     | 113629/225000 [236:45:04<174:52:12,  5.65s/it]
 51%|█████     | 113630/225000 [236:45:10<177:55:06,  5.75s/it]
 51%|█████     | 113631/225000 [236:45:13<155:29:07,  5.03s/it]
 51%|█████     | 113632/225000 [236:45:16<136:47:58,  4.42s/it]
 51%|█████     | 113633/225000 [236:45:19<123:08:51,  3.98s/it]
 51%|█████     | 113634/225000 [236:45:29<178:56:39,  5.78s/it]
 51%|█████     | 113635/225000 [236:45:33<159:49:56,  5.17s/it]
 51%|█████     | 113636/225000 [236:45:36<141:30:11,  4.57s/it]
 51%|█████     | 113637/225000 [236:45:40<136:24:26,  4.41s/it]
 51%|█████     | 113638/225000 [236:45:43<125:51:51,  4.07s/it]
 51%|█████     | 113639/225000 [236:45:47<120:20:58,  3.89s/it]
 51%|█████     | 113640/225000 [236:45:52<130:27:05,  4.22s/it]
 51%|█████     | 113641/225000 [236:46:01<178:54:22,  5.78s/it]
 51%|█████     | 113642/225000 [236:46:04<147:17:54,  4.76s/it]
 51%|█████     | 113643/225000 [236:46:07<138:48:16,  4.49s/it]
 51%|█████     | 113644/225000 [236:46:10<123:23:16,  3.99s/it]
 51%|█████     | 113645/225000 [236:46:14<117:14:58,  3.79s/it]
 51%|█████     | 113646/225000 [236:46:23<164:39:15,  5.32s/it]
 51%|█████     | 113647/225000 [236:46:25<140:20:48,  4.54s/it]
 51%|█████     | 113648/225000 [236:46:28<124:10:03,  4.01s/it]
 51%|█████     | 113649/225000 [236:46:31<113:22:16,  3.67s/it]
 51%|█████     | 113650/225000 [236:46:36<123:46:21,  4.00s/it]
                                                               
{'loss': '0.914', 'grad_norm': '1.486', 'learning_rate': '2.642e-05', 'epoch': '0.5051', 'num_input_tokens_seen': 395693848, 'train_runtime': '8.524e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113650/225000 [236:46:36<123:46:21,  4.00s/it]
 51%|█████     | 113651/225000 [236:46:43<153:57:56,  4.98s/it]
 51%|█████     | 113652/225000 [236:46:52<196:38:53,  6.36s/it]
 51%|█████     | 113653/225000 [236:46:56<172:59:59,  5.59s/it]
 51%|█████     | 113654/225000 [236:47:02<178:03:04,  5.76s/it]
 51%|█████     | 113655/225000 [236:47:07<166:44:06,  5.39s/it]
 51%|█████     | 113656/225000 [236:47:11<156:15:50,  5.05s/it]
 51%|█████     | 113657/225000 [236:47:14<138:27:14,  4.48s/it]
 51%|█████     | 113658/225000 [236:47:18<127:21:33,  4.12s/it]
 51%|█████     | 113659/225000 [236:47:28<186:46:28,  6.04s/it]
 51%|█████     | 113660/225000 [236:47:32<168:04:02,  5.43s/it]
 51%|█████     | 113661/225000 [236:47:36<149:42:59,  4.84s/it]
 51%|█████     | 113662/225000 [236:47:46<203:27:05,  6.58s/it]
 51%|█████     | 113663/225000 [236:47:49<165:44:14,  5.36s/it]
 51%|█████     | 113664/225000 [236:47:55<174:39:14,  5.65s/it]
 51%|█████     | 113665/225000 [236:47:58<152:29:06,  4.93s/it]
 51%|█████     | 113666/225000 [236:48:03<146:12:41,  4.73s/it]
 51%|█████     | 113667/225000 [236:48:12<191:19:27,  6.19s/it]
 51%|█████     | 113668/225000 [236:48:15<160:44:33,  5.20s/it]
 51%|█████     | 113669/225000 [236:48:21<171:52:01,  5.56s/it]
 51%|█████     | 113670/225000 [236:48:28<180:30:28,  5.84s/it]
 51%|█████     | 113671/225000 [236:48:31<150:14:21,  4.86s/it]
 51%|█████     | 113672/225000 [236:48:34<139:26:32,  4.51s/it]
 51%|█████     | 113673/225000 [236:48:39<137:20:28,  4.44s/it]
 51%|█████     | 113674/225000 [236:48:42<126:31:44,  4.09s/it]
 51%|█████     | 113675/225000 [236:48:52<185:42:59,  6.01s/it]
 51%|█████     | 113676/225000 [236:49:00<199:43:40,  6.46s/it]
 51%|█████     | 113677/225000 [236:49:02<164:37:56,  5.32s/it]
 51%|█████     | 113678/225000 [236:49:06<147:44:46,  4.78s/it]
 51%|█████     | 113679/225000 [236:49:10<138:26:17,  4.48s/it]
 51%|█████     | 113680/225000 [236:49:17<162:28:13,  5.25s/it]
 51%|█████     | 113681/225000 [236:49:22<159:50:15,  5.17s/it]
 51%|█████     | 113682/225000 [236:49:24<135:10:39,  4.37s/it]
 51%|█████     | 113683/225000 [236:49:27<121:17:26,  3.92s/it]
 51%|█████     | 113684/225000 [236:49:34<152:28:00,  4.93s/it]
 51%|█████     | 113685/225000 [236:49:37<130:30:56,  4.22s/it]
 51%|█████     | 113686/225000 [236:49:40<122:45:52,  3.97s/it]
 51%|█████     | 113687/225000 [236:49:50<172:07:50,  5.57s/it]
 51%|█████     | 113688/225000 [236:49:55<165:52:15,  5.36s/it]
 51%|█████     | 113689/225000 [236:49:57<137:48:30,  4.46s/it]
 51%|█████     | 113690/225000 [236:50:03<154:05:29,  4.98s/it]
 51%|█████     | 113691/225000 [236:50:07<140:16:02,  4.54s/it]
 51%|█████     | 113692/225000 [236:50:10<130:43:27,  4.23s/it]
 51%|█████     | 113693/225000 [236:50:14<125:04:41,  4.05s/it]
 51%|█████     | 113694/225000 [236:50:17<121:10:53,  3.92s/it]
 51%|█████     | 113695/225000 [236:50:21<120:26:02,  3.90s/it]
 51%|█████     | 113696/225000 [236:50:25<120:45:04,  3.91s/it]
 51%|█████     | 113697/225000 [236:50:29<123:02:48,  3.98s/it]
 51%|█████     | 113698/225000 [236:50:33<116:03:44,  3.75s/it]
 51%|█████     | 113699/225000 [236:50:37<121:56:20,  3.94s/it]
 51%|█████     | 113700/225000 [236:50:43<141:57:14,  4.59s/it]
                                                               
{'loss': '0.9246', 'grad_norm': '1.392', 'learning_rate': '2.641e-05', 'epoch': '0.5053', 'num_input_tokens_seen': 395800488, 'train_runtime': '8.526e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113700/225000 [236:50:43<141:57:14,  4.59s/it]
 51%|█████     | 113701/225000 [236:50:53<188:29:48,  6.10s/it]
 51%|█████     | 113702/225000 [236:50:56<163:50:32,  5.30s/it]
 51%|█████     | 113703/225000 [236:51:02<169:30:18,  5.48s/it]
 51%|█████     | 113704/225000 [236:51:05<145:04:08,  4.69s/it]
 51%|█████     | 113705/225000 [236:51:08<128:52:15,  4.17s/it]
 51%|█████     | 113706/225000 [236:51:11<121:40:30,  3.94s/it]
 51%|█████     | 113707/225000 [236:51:15<124:15:46,  4.02s/it]
 51%|█████     | 113708/225000 [236:51:19<124:46:49,  4.04s/it]
 51%|█████     | 113709/225000 [236:51:23<119:17:17,  3.86s/it]
 51%|█████     | 113710/225000 [236:51:34<185:40:02,  6.01s/it]
 51%|█████     | 113711/225000 [236:51:37<158:21:47,  5.12s/it]
 51%|█████     | 113712/225000 [236:51:40<140:32:27,  4.55s/it]
 51%|█████     | 113713/225000 [236:51:47<163:54:26,  5.30s/it]
 51%|█████     | 113714/225000 [236:51:56<197:58:22,  6.40s/it]
 51%|█████     | 113715/225000 [236:51:59<162:48:44,  5.27s/it]
 51%|█████     | 113716/225000 [236:52:02<142:07:15,  4.60s/it]
 51%|█████     | 113717/225000 [236:52:06<141:45:24,  4.59s/it]
 51%|█████     | 113718/225000 [236:52:11<137:50:09,  4.46s/it]
 51%|█████     | 113719/225000 [236:52:14<131:47:26,  4.26s/it]
 51%|█████     | 113720/225000 [236:52:19<130:50:17,  4.23s/it]
 51%|█████     | 113721/225000 [236:52:22<122:31:42,  3.96s/it]
 51%|█████     | 113722/225000 [236:52:26<124:16:37,  4.02s/it]
 51%|█████     | 113723/225000 [236:52:31<132:37:40,  4.29s/it]
 51%|█████     | 113724/225000 [236:52:35<133:52:54,  4.33s/it]
 51%|█████     | 113725/225000 [236:52:40<134:12:24,  4.34s/it]
 51%|█████     | 113726/225000 [236:52:44<135:08:49,  4.37s/it]
 51%|█████     | 113727/225000 [236:52:51<156:32:41,  5.06s/it]
 51%|█████     | 113728/225000 [236:52:55<150:37:34,  4.87s/it]
 51%|█████     | 113729/225000 [236:52:59<142:21:06,  4.61s/it]
 51%|█████     | 113730/225000 [236:53:03<134:16:47,  4.34s/it]
 51%|█████     | 113731/225000 [236:53:08<140:36:04,  4.55s/it]
 51%|█████     | 113732/225000 [236:53:12<130:51:52,  4.23s/it]
 51%|█████     | 113733/225000 [236:53:16<136:47:04,  4.43s/it]
 51%|█████     | 113734/225000 [236:53:20<132:47:00,  4.30s/it]
 51%|█████     | 113735/225000 [236:53:25<137:19:03,  4.44s/it]
 51%|█████     | 113736/225000 [236:53:30<142:19:46,  4.61s/it]
 51%|█████     | 113737/225000 [236:53:34<132:28:13,  4.29s/it]
 51%|█████     | 113738/225000 [236:53:42<173:30:28,  5.61s/it]
 51%|█████     | 113739/225000 [236:53:46<152:10:04,  4.92s/it]
 51%|█████     | 113740/225000 [236:53:58<216:49:17,  7.02s/it]
 51%|█████     | 113741/225000 [236:54:00<177:40:25,  5.75s/it]
 51%|█████     | 113742/225000 [236:54:03<147:37:32,  4.78s/it]
 51%|█████     | 113743/225000 [236:54:06<132:50:43,  4.30s/it]
 51%|█████     | 113744/225000 [236:54:11<134:04:47,  4.34s/it]
 51%|█████     | 113745/225000 [236:54:15<134:41:31,  4.36s/it]
 51%|█████     | 113746/225000 [236:54:18<126:37:29,  4.10s/it]
 51%|█████     | 113747/225000 [236:54:23<128:34:51,  4.16s/it]
 51%|█████     | 113748/225000 [236:54:26<119:31:00,  3.87s/it]
 51%|█████     | 113749/225000 [236:54:30<119:30:13,  3.87s/it]
 51%|█████     | 113750/225000 [236:54:34<121:57:06,  3.95s/it]
                                                               
{'loss': '0.9275', 'grad_norm': '1.653', 'learning_rate': '2.639e-05', 'epoch': '0.5056', 'num_input_tokens_seen': 395902272, 'train_runtime': '8.529e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113750/225000 [236:54:34<121:57:06,  3.95s/it]
 51%|█████     | 113751/225000 [236:54:39<135:28:25,  4.38s/it]
 51%|█████     | 113752/225000 [236:54:49<182:11:40,  5.90s/it]
 51%|█████     | 113753/225000 [236:54:53<166:48:03,  5.40s/it]
 51%|█████     | 113754/225000 [236:54:57<151:29:11,  4.90s/it]
 51%|█████     | 113755/225000 [236:55:00<139:18:22,  4.51s/it]
 51%|█████     | 113756/225000 [236:55:12<204:34:16,  6.62s/it]
 51%|█████     | 113757/225000 [236:55:14<165:45:32,  5.36s/it]
 51%|█████     | 113758/225000 [236:55:18<147:45:27,  4.78s/it]
 51%|█████     | 113759/225000 [236:55:27<184:25:16,  5.97s/it]
 51%|█████     | 113760/225000 [236:55:32<183:56:38,  5.95s/it]
 51%|█████     | 113761/225000 [236:55:36<161:13:56,  5.22s/it]
 51%|█████     | 113762/225000 [236:55:39<141:01:03,  4.56s/it]
 51%|█████     | 113763/225000 [236:55:46<162:04:05,  5.25s/it]
 51%|█████     | 113764/225000 [236:55:49<145:36:31,  4.71s/it]
 51%|█████     | 113765/225000 [236:55:53<137:38:04,  4.45s/it]
 51%|█████     | 113766/225000 [236:55:57<128:56:56,  4.17s/it]
 51%|█████     | 113767/225000 [236:56:01<130:04:55,  4.21s/it]
 51%|█████     | 113768/225000 [236:56:11<188:10:13,  6.09s/it]
 51%|█████     | 113769/225000 [236:56:15<167:16:59,  5.41s/it]
 51%|█████     | 113770/225000 [236:56:19<147:40:48,  4.78s/it]
 51%|█████     | 113771/225000 [236:56:22<137:18:49,  4.44s/it]
 51%|█████     | 113772/225000 [236:56:27<139:45:14,  4.52s/it]
 51%|█████     | 113773/225000 [236:56:30<127:02:06,  4.11s/it]
 51%|█████     | 113774/225000 [236:56:33<118:33:50,  3.84s/it]
 51%|█████     | 113775/225000 [236:56:43<173:15:18,  5.61s/it]
 51%|█████     | 113776/225000 [236:56:49<174:06:45,  5.64s/it]
 51%|█████     | 113777/225000 [236:56:57<202:19:52,  6.55s/it]
 51%|█████     | 113778/225000 [236:57:01<176:43:40,  5.72s/it]
 51%|█████     | 113779/225000 [236:57:04<152:30:22,  4.94s/it]
 51%|█████     | 113780/225000 [236:57:07<133:47:18,  4.33s/it]
 51%|█████     | 113781/225000 [236:57:12<138:04:40,  4.47s/it]
 51%|█████     | 113782/225000 [236:57:16<131:53:02,  4.27s/it]
 51%|█████     | 113783/225000 [236:57:20<132:39:53,  4.29s/it]
 51%|█████     | 113784/225000 [236:57:25<136:44:56,  4.43s/it]
 51%|█████     | 113785/225000 [236:57:34<175:43:12,  5.69s/it]
 51%|█████     | 113786/225000 [236:57:37<150:52:50,  4.88s/it]
 51%|█████     | 113787/225000 [236:57:39<131:48:38,  4.27s/it]
 51%|█████     | 113788/225000 [236:57:51<196:30:23,  6.36s/it]
 51%|█████     | 113789/225000 [236:57:55<178:49:30,  5.79s/it]
 51%|█████     | 113790/225000 [236:57:59<161:03:53,  5.21s/it]
 51%|█████     | 113791/225000 [236:58:03<150:49:38,  4.88s/it]
 51%|█████     | 113792/225000 [236:58:07<138:38:55,  4.49s/it]
 51%|█████     | 113793/225000 [236:58:09<122:41:44,  3.97s/it]
 51%|█████     | 113794/225000 [236:58:13<122:32:04,  3.97s/it]
 51%|█████     | 113795/225000 [236:58:18<128:39:17,  4.16s/it]
 51%|█████     | 113796/225000 [236:58:24<142:36:36,  4.62s/it]
 51%|█████     | 113797/225000 [236:58:34<192:13:22,  6.22s/it]
 51%|█████     | 113798/225000 [236:58:37<165:49:18,  5.37s/it]
 51%|█████     | 113799/225000 [236:58:41<149:11:05,  4.83s/it]
 51%|█████     | 113800/225000 [236:58:45<146:57:57,  4.76s/it]
                                                               
{'loss': '0.9335', 'grad_norm': '1.142', 'learning_rate': '2.637e-05', 'epoch': '0.5058', 'num_input_tokens_seen': 396011696, 'train_runtime': '8.531e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113800/225000 [236:58:45<146:57:57,  4.76s/it]
 51%|█████     | 113801/225000 [236:58:57<210:41:02,  6.82s/it]
 51%|█████     | 113802/225000 [236:59:01<184:28:24,  5.97s/it]
 51%|█████     | 113803/225000 [236:59:04<160:54:59,  5.21s/it]
 51%|█████     | 113804/225000 [236:59:07<138:59:32,  4.50s/it]
 51%|█████     | 113805/225000 [236:59:10<129:08:22,  4.18s/it]
 51%|█████     | 113806/225000 [236:59:26<230:13:42,  7.45s/it]
 51%|█████     | 113807/225000 [236:59:30<202:17:50,  6.55s/it]
 51%|█████     | 113808/225000 [236:59:34<175:44:20,  5.69s/it]
 51%|█████     | 113809/225000 [236:59:37<155:18:37,  5.03s/it]
 51%|█████     | 113810/225000 [236:59:40<136:36:43,  4.42s/it]
 51%|█████     | 113811/225000 [236:59:43<126:27:53,  4.09s/it]
 51%|█████     | 113812/225000 [236:59:47<118:34:03,  3.84s/it]
 51%|█████     | 113813/225000 [236:59:50<113:57:04,  3.69s/it]
 51%|█████     | 113814/225000 [237:00:04<206:12:26,  6.68s/it]
 51%|█████     | 113815/225000 [237:00:07<171:42:47,  5.56s/it]
 51%|█████     | 113816/225000 [237:00:09<144:43:48,  4.69s/it]
 51%|█████     | 113817/225000 [237:00:17<169:02:50,  5.47s/it]
 51%|█████     | 113818/225000 [237:00:19<140:44:57,  4.56s/it]
 51%|█████     | 113819/225000 [237:00:28<181:22:24,  5.87s/it]
 51%|█████     | 113820/225000 [237:00:34<181:41:41,  5.88s/it]
 51%|█████     | 113821/225000 [237:00:39<176:23:16,  5.71s/it]
 51%|█████     | 113822/225000 [237:00:43<156:49:03,  5.08s/it]
 51%|█████     | 113823/225000 [237:00:46<140:52:43,  4.56s/it]
 51%|█████     | 113824/225000 [237:00:50<133:03:36,  4.31s/it]
 51%|█████     | 113825/225000 [237:00:54<132:46:24,  4.30s/it]
 51%|█████     | 113826/225000 [237:00:58<128:23:35,  4.16s/it]
 51%|█████     | 113827/225000 [237:01:02<123:14:57,  3.99s/it]
 51%|█████     | 113828/225000 [237:01:05<122:23:23,  3.96s/it]
 51%|█████     | 113829/225000 [237:01:10<123:34:36,  4.00s/it]
 51%|█████     | 113830/225000 [237:01:13<122:24:38,  3.96s/it]
 51%|█████     | 113831/225000 [237:01:18<123:49:24,  4.01s/it]
 51%|█████     | 113832/225000 [237:01:28<178:36:18,  5.78s/it]
 51%|█████     | 113833/225000 [237:01:32<164:36:49,  5.33s/it]
 51%|█████     | 113834/225000 [237:01:35<146:05:50,  4.73s/it]
 51%|█████     | 113835/225000 [237:01:39<135:55:28,  4.40s/it]
 51%|█████     | 113836/225000 [237:01:46<160:39:49,  5.20s/it]
 51%|█████     | 113837/225000 [237:01:53<183:26:44,  5.94s/it]
 51%|█████     | 113838/225000 [237:01:56<153:12:06,  4.96s/it]
 51%|█████     | 113839/225000 [237:01:59<133:42:48,  4.33s/it]
 51%|█████     | 113840/225000 [237:02:06<160:22:47,  5.19s/it]
 51%|█████     | 113841/225000 [237:02:10<149:32:06,  4.84s/it]
 51%|█████     | 113842/225000 [237:02:14<136:00:31,  4.40s/it]
 51%|█████     | 113843/225000 [237:02:18<134:17:32,  4.35s/it]
 51%|█████     | 113844/225000 [237:02:22<129:56:58,  4.21s/it]
 51%|█████     | 113845/225000 [237:02:26<126:18:35,  4.09s/it]
 51%|█████     | 113846/225000 [237:02:34<171:00:42,  5.54s/it]
 51%|█████     | 113847/225000 [237:02:40<167:59:36,  5.44s/it]
 51%|█████     | 113848/225000 [237:02:49<205:36:04,  6.66s/it]
 51%|█████     | 113849/225000 [237:02:52<168:50:44,  5.47s/it]
 51%|█████     | 113850/225000 [237:02:55<143:36:27,  4.65s/it]
                                                               
{'loss': '0.9255', 'grad_norm': '1.563', 'learning_rate': '2.635e-05', 'epoch': '0.506', 'num_input_tokens_seen': 396115432, 'train_runtime': '8.534e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113850/225000 [237:02:55<143:36:27,  4.65s/it]
 51%|█████     | 113851/225000 [237:03:03<176:55:22,  5.73s/it]
 51%|█████     | 113852/225000 [237:03:07<158:07:22,  5.12s/it]
 51%|█████     | 113853/225000 [237:03:12<162:13:05,  5.25s/it]
 51%|█████     | 113854/225000 [237:03:19<176:34:23,  5.72s/it]
 51%|█████     | 113855/225000 [237:03:25<183:37:48,  5.95s/it]
 51%|█████     | 113856/225000 [237:03:35<213:33:45,  6.92s/it]
 51%|█████     | 113857/225000 [237:03:38<177:04:08,  5.74s/it]
 51%|█████     | 113858/225000 [237:03:42<162:03:20,  5.25s/it]
 51%|█████     | 113859/225000 [237:03:48<175:38:22,  5.69s/it]
 51%|█████     | 113860/225000 [237:03:52<153:20:01,  4.97s/it]
 51%|█████     | 113861/225000 [237:03:55<141:46:19,  4.59s/it]
 51%|█████     | 113862/225000 [237:03:59<130:22:55,  4.22s/it]
 51%|█████     | 113863/225000 [237:04:02<124:37:46,  4.04s/it]
 51%|█████     | 113864/225000 [237:04:07<127:10:37,  4.12s/it]
 51%|█████     | 113865/225000 [237:04:15<165:10:44,  5.35s/it]
 51%|█████     | 113866/225000 [237:04:19<157:07:09,  5.09s/it]
 51%|█████     | 113867/225000 [237:04:23<143:43:57,  4.66s/it]
 51%|█████     | 113868/225000 [237:04:27<135:36:13,  4.39s/it]
 51%|█████     | 113869/225000 [237:04:32<140:28:10,  4.55s/it]
 51%|█████     | 113870/225000 [237:04:43<198:45:39,  6.44s/it]
 51%|█████     | 113871/225000 [237:04:45<165:35:44,  5.36s/it]
 51%|█████     | 113872/225000 [237:04:49<151:24:16,  4.90s/it]
 51%|█████     | 113873/225000 [237:04:54<146:30:17,  4.75s/it]
 51%|█████     | 113874/225000 [237:04:58<142:06:42,  4.60s/it]
 51%|█████     | 113875/225000 [237:05:02<136:26:56,  4.42s/it]
 51%|█████     | 113876/225000 [237:05:06<135:47:17,  4.40s/it]
 51%|█████     | 113877/225000 [237:05:11<134:59:52,  4.37s/it]
 51%|█████     | 113878/225000 [237:05:14<126:18:38,  4.09s/it]
 51%|█████     | 113879/225000 [237:05:18<121:57:00,  3.95s/it]
 51%|█████     | 113880/225000 [237:05:21<120:35:06,  3.91s/it]
 51%|█████     | 113881/225000 [237:05:26<127:25:31,  4.13s/it]
 51%|█████     | 113882/225000 [237:05:30<125:01:23,  4.05s/it]
 51%|█████     | 113883/225000 [237:05:43<210:57:38,  6.83s/it]
 51%|█████     | 113884/225000 [237:05:46<173:22:11,  5.62s/it]
 51%|█████     | 113885/225000 [237:05:57<225:05:53,  7.29s/it]
 51%|█████     | 113886/225000 [237:06:00<183:12:13,  5.94s/it]
 51%|█████     | 113887/225000 [237:06:06<185:13:57,  6.00s/it]
 51%|█████     | 113888/225000 [237:06:09<152:08:24,  4.93s/it]
 51%|█████     | 113889/225000 [237:06:11<132:20:56,  4.29s/it]
 51%|█████     | 113890/225000 [237:06:19<160:52:54,  5.21s/it]
 51%|█████     | 113891/225000 [237:06:22<146:04:34,  4.73s/it]
 51%|█████     | 113892/225000 [237:06:25<131:03:49,  4.25s/it]
 51%|█████     | 113893/225000 [237:06:29<120:31:35,  3.91s/it]
 51%|█████     | 113894/225000 [237:06:32<120:16:21,  3.90s/it]
 51%|█████     | 113895/225000 [237:06:36<119:49:03,  3.88s/it]
 51%|█████     | 113896/225000 [237:06:40<120:22:50,  3.90s/it]
 51%|█████     | 113897/225000 [237:06:44<122:01:42,  3.95s/it]
 51%|█████     | 113898/225000 [237:06:54<179:05:27,  5.80s/it]
 51%|█████     | 113899/225000 [237:07:00<173:32:09,  5.62s/it]
 51%|█████     | 113900/225000 [237:07:04<158:16:52,  5.13s/it]
                                                               
{'loss': '0.9444', 'grad_norm': '1.256', 'learning_rate': '2.633e-05', 'epoch': '0.5062', 'num_input_tokens_seen': 396220320, 'train_runtime': '8.536e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113900/225000 [237:07:04<158:16:52,  5.13s/it]
 51%|█████     | 113901/225000 [237:07:07<143:38:19,  4.65s/it]
 51%|█████     | 113902/225000 [237:07:19<208:25:40,  6.75s/it]
 51%|█████     | 113903/225000 [237:07:23<182:03:08,  5.90s/it]
 51%|█████     | 113904/225000 [237:07:25<149:34:14,  4.85s/it]
 51%|█████     | 113905/225000 [237:07:34<189:35:34,  6.14s/it]
 51%|█████     | 113906/225000 [237:07:40<181:44:17,  5.89s/it]
 51%|█████     | 113907/225000 [237:07:43<155:32:43,  5.04s/it]
 51%|█████     | 113908/225000 [237:07:48<162:07:47,  5.25s/it]
 51%|█████     | 113909/225000 [237:07:52<143:51:14,  4.66s/it]
 51%|█████     | 113910/225000 [237:08:00<181:13:23,  5.87s/it]
 51%|█████     | 113911/225000 [237:08:04<164:51:42,  5.34s/it]
 51%|█████     | 113912/225000 [237:08:08<152:03:13,  4.93s/it]
 51%|█████     | 113913/225000 [237:08:12<141:09:36,  4.57s/it]
 51%|█████     | 113914/225000 [237:08:16<137:19:04,  4.45s/it]
 51%|█████     | 113915/225000 [237:08:20<130:19:13,  4.22s/it]
 51%|█████     | 113916/225000 [237:08:24<129:31:31,  4.20s/it]
 51%|█████     | 113917/225000 [237:08:28<123:22:13,  4.00s/it]
 51%|█████     | 113918/225000 [237:08:39<189:46:31,  6.15s/it]
 51%|█████     | 113919/225000 [237:08:42<160:54:17,  5.21s/it]
 51%|█████     | 113920/225000 [237:08:45<139:42:16,  4.53s/it]
 51%|█████     | 113921/225000 [237:08:50<141:19:14,  4.58s/it]
 51%|█████     | 113922/225000 [237:08:58<173:35:07,  5.63s/it]
 51%|█████     | 113923/225000 [237:09:01<153:42:23,  4.98s/it]
 51%|█████     | 113924/225000 [237:09:05<143:10:19,  4.64s/it]
 51%|█████     | 113925/225000 [237:09:08<131:08:01,  4.25s/it]
 51%|█████     | 113926/225000 [237:09:12<123:09:01,  3.99s/it]
 51%|█████     | 113927/225000 [237:09:22<180:31:09,  5.85s/it]
 51%|█████     | 113928/225000 [237:09:25<156:48:33,  5.08s/it]
 51%|█████     | 113929/225000 [237:09:28<136:44:57,  4.43s/it]
 51%|█████     | 113930/225000 [237:09:36<165:25:30,  5.36s/it]
 51%|█████     | 113931/225000 [237:09:39<148:10:02,  4.80s/it]
 51%|█████     | 113932/225000 [237:09:43<142:32:31,  4.62s/it]
 51%|█████     | 113933/225000 [237:09:47<136:17:34,  4.42s/it]
 51%|█████     | 113934/225000 [237:09:51<128:33:00,  4.17s/it]
 51%|█████     | 113935/225000 [237:10:02<197:04:36,  6.39s/it]
 51%|█████     | 113936/225000 [237:10:06<167:28:41,  5.43s/it]
 51%|█████     | 113937/225000 [237:10:09<145:01:13,  4.70s/it]
 51%|█████     | 113938/225000 [237:10:16<168:47:57,  5.47s/it]
 51%|█████     | 113939/225000 [237:10:19<150:12:02,  4.87s/it]
 51%|█████     | 113940/225000 [237:10:23<135:41:41,  4.40s/it]
 51%|█████     | 113941/225000 [237:10:26<123:43:54,  4.01s/it]
 51%|█████     | 113942/225000 [237:10:34<164:23:04,  5.33s/it]
 51%|█████     | 113943/225000 [237:10:37<140:49:02,  4.56s/it]
 51%|█████     | 113944/225000 [237:10:41<133:42:18,  4.33s/it]
 51%|█████     | 113945/225000 [237:10:44<123:49:26,  4.01s/it]
 51%|█████     | 113946/225000 [237:10:48<119:43:49,  3.88s/it]
 51%|█████     | 113947/225000 [237:10:51<112:19:34,  3.64s/it]
 51%|█████     | 113948/225000 [237:10:54<109:46:21,  3.56s/it]
 51%|█████     | 113949/225000 [237:10:58<109:53:22,  3.56s/it]
 51%|█████     | 113950/225000 [237:11:02<114:13:59,  3.70s/it]
                                                               
{'loss': '0.9304', 'grad_norm': '1.308', 'learning_rate': '2.632e-05', 'epoch': '0.5064', 'num_input_tokens_seen': 396325848, 'train_runtime': '8.539e+05', 'train_tokens_per_second': '464.2'}

 51%|█████     | 113950/225000 [237:11:02<114:13:59,  3.70s/it]
 51%|█████     | 113951/225000 [237:11:07<132:19:01,  4.29s/it]
 51%|█████     | 113952/225000 [237:11:17<181:34:48,  5.89s/it]
 51%|█████     | 113953/225000 [237:11:20<155:57:54,  5.06s/it]
 51%|█████     | 113954/225000 [237:11:23<141:47:34,  4.60s/it]
 51%|█████     | 113955/225000 [237:11:29<151:36:53,  4.92s/it]
 51%|█████     | 113956/225000 [237:11:38<185:49:29,  6.02s/it]
 51%|█████     | 113957/225000 [237:11:40<154:31:15,  5.01s/it]
 51%|█████     | 113958/225000 [237:11:43<133:42:14,  4.33s/it]
 51%|█████     | 113959/225000 [237:11:49<149:40:17,  4.85s/it]
 51%|█████     | 113960/225000 [237:11:53<139:04:48,  4.51s/it]
 51%|█████     | 113961/225000 [237:11:56<123:51:45,  4.02s/it]
 51%|█████     | 113962/225000 [237:12:04<158:33:16,  5.14s/it]
 51%|█████     | 113963/225000 [237:12:07<140:22:32,  4.55s/it]
 51%|█████     | 113964/225000 [237:12:10<123:53:55,  4.02s/it]
 51%|█████     | 113965/225000 [237:12:13<121:49:58,  3.95s/it]
 51%|█████     | 113966/225000 [237:12:22<165:29:04,  5.37s/it]
 51%|█████     | 113967/225000 [237:12:25<144:12:14,  4.68s/it]
 51%|█████     | 113968/225000 [237:12:29<139:43:59,  4.53s/it]
 51%|█████     | 113969/225000 [237:12:33<129:46:50,  4.21s/it]
 51%|█████     | 113970/225000 [237:12:37<127:23:01,  4.13s/it]
 51%|█████     | 113971/225000 [237:12:47<186:13:05,  6.04s/it]
 51%|█████     | 113972/225000 [237:12:50<158:30:28,  5.14s/it]
 51%|█████     | 113973/225000 [237:12:53<137:59:55,  4.47s/it]
 51%|█████     | 113974/225000 [237:12:57<129:35:23,  4.20s/it]
 51%|█████     | 113975/225000 [237:13:06<180:36:36,  5.86s/it]
 51%|█████     | 113976/225000 [237:13:09<152:35:12,  4.95s/it]
 51%|█████     | 113977/225000 [237:13:12<136:35:34,  4.43s/it]
 51%|█████     | 113978/225000 [237:13:20<163:46:49,  5.31s/it]
 51%|█████     | 113979/225000 [237:13:23<145:33:27,  4.72s/it]
 51%|█████     | 113980/225000 [237:13:28<142:30:11,  4.62s/it]
 51%|█████     | 113981/225000 [237:13:31<131:48:12,  4.27s/it]
 51%|█████     | 113982/225000 [237:13:34<123:36:03,  4.01s/it]
 51%|█████     | 113983/225000 [237:13:38<123:50:44,  4.02s/it]
 51%|█████     | 113984/225000 [237:13:42<122:23:51,  3.97s/it]
 51%|█████     | 113985/225000 [237:13:49<146:55:24,  4.76s/it]
 51%|█████     | 113986/225000 [237:13:59<195:22:13,  6.34s/it]
 51%|█████     | 113987/225000 [237:14:02<166:27:04,  5.40s/it]
 51%|█████     | 113988/225000 [237:14:06<148:38:27,  4.82s/it]
 51%|█████     | 113989/225000 [237:14:09<137:07:12,  4.45s/it]
 51%|█████     | 113990/225000 [237:14:13<129:16:48,  4.19s/it]
 51%|█████     | 113991/225000 [237:14:17<128:58:02,  4.18s/it]
 51%|█████     | 113992/225000 [237:14:28<188:49:59,  6.12s/it]
 51%|█████     | 113993/225000 [237:14:30<158:21:53,  5.14s/it]
 51%|█████     | 113994/225000 [237:14:36<164:47:51,  5.34s/it]
 51%|█████     | 113995/225000 [237:14:41<162:13:03,  5.26s/it]
 51%|█████     | 113996/225000 [237:14:44<137:03:08,  4.44s/it]
 51%|█████     | 113997/225000 [237:14:52<167:55:31,  5.45s/it]
 51%|█████     | 113998/225000 [237:14:57<171:43:21,  5.57s/it]
 51%|█████     | 113999/225000 [237:15:04<179:22:41,  5.82s/it]
 51%|█████     | 114000/225000 [237:15:08<163:08:27,  5.29s/it]
                                                               
{'loss': '0.9387', 'grad_norm': '1.374', 'learning_rate': '2.63e-05', 'epoch': '0.5067', 'num_input_tokens_seen': 396429584, 'train_runtime': '8.541e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114000/225000 [237:15:08<163:08:27,  5.29s/it]
 51%|█████     | 114001/225000 [237:15:11<145:29:03,  4.72s/it]
 51%|█████     | 114002/225000 [237:15:15<135:31:54,  4.40s/it]
 51%|█████     | 114003/225000 [237:15:21<152:53:28,  4.96s/it]
 51%|█████     | 114004/225000 [237:15:30<188:27:59,  6.11s/it]
 51%|█████     | 114005/225000 [237:15:32<153:14:52,  4.97s/it]
 51%|█████     | 114006/225000 [237:15:37<154:38:33,  5.02s/it]
 51%|█████     | 114007/225000 [237:15:45<182:04:08,  5.91s/it]
 51%|█████     | 114008/225000 [237:15:52<186:48:44,  6.06s/it]
 51%|█████     | 114009/225000 [237:15:58<189:52:53,  6.16s/it]
 51%|█████     | 114010/225000 [237:16:01<159:41:29,  5.18s/it]
 51%|█████     | 114011/225000 [237:16:08<176:16:39,  5.72s/it]
 51%|█████     | 114012/225000 [237:16:11<153:36:40,  4.98s/it]
 51%|█████     | 114013/225000 [237:16:15<138:29:54,  4.49s/it]
 51%|█████     | 114014/225000 [237:16:18<127:47:29,  4.15s/it]
 51%|█████     | 114015/225000 [237:16:21<114:31:36,  3.71s/it]
 51%|█████     | 114016/225000 [237:16:25<117:25:48,  3.81s/it]
 51%|█████     | 114017/225000 [237:16:30<132:56:00,  4.31s/it]
 51%|█████     | 114018/225000 [237:16:37<154:08:55,  5.00s/it]
 51%|█████     | 114019/225000 [237:16:40<140:46:22,  4.57s/it]
 51%|█████     | 114020/225000 [237:16:48<170:38:32,  5.54s/it]
 51%|█████     | 114021/225000 [237:16:52<151:20:48,  4.91s/it]
 51%|█████     | 114022/225000 [237:16:56<141:35:53,  4.59s/it]
 51%|█████     | 114023/225000 [237:16:59<135:08:23,  4.38s/it]
 51%|█████     | 114024/225000 [237:17:03<123:27:24,  4.00s/it]
 51%|█████     | 114025/225000 [237:17:10<156:07:42,  5.06s/it]
 51%|█████     | 114026/225000 [237:17:15<156:45:57,  5.09s/it]
 51%|█████     | 114027/225000 [237:17:19<142:01:30,  4.61s/it]
 51%|█████     | 114028/225000 [237:17:24<147:28:01,  4.78s/it]
 51%|█████     | 114029/225000 [237:17:27<135:30:48,  4.40s/it]
 51%|█████     | 114030/225000 [237:17:33<150:58:04,  4.90s/it]
 51%|█████     | 114031/225000 [237:17:40<167:18:09,  5.43s/it]
 51%|█████     | 114032/225000 [237:17:47<177:01:55,  5.74s/it]
 51%|█████     | 114033/225000 [237:17:50<156:39:55,  5.08s/it]
 51%|█████     | 114034/225000 [237:17:53<139:33:44,  4.53s/it]
 51%|█████     | 114035/225000 [237:18:00<159:33:38,  5.18s/it]
 51%|█████     | 114036/225000 [237:18:08<187:04:10,  6.07s/it]
 51%|█████     | 114037/225000 [237:18:12<165:19:28,  5.36s/it]
 51%|█████     | 114038/225000 [237:18:21<200:37:20,  6.51s/it]
 51%|█████     | 114039/225000 [237:18:25<180:16:07,  5.85s/it]
 51%|█████     | 114040/225000 [237:18:29<158:18:35,  5.14s/it]
 51%|█████     | 114041/225000 [237:18:32<139:08:35,  4.51s/it]
 51%|█████     | 114042/225000 [237:18:39<166:16:44,  5.39s/it]
 51%|█████     | 114043/225000 [237:18:43<146:30:56,  4.75s/it]
 51%|█████     | 114044/225000 [237:18:47<145:24:05,  4.72s/it]
 51%|█████     | 114045/225000 [237:18:54<166:31:10,  5.40s/it]
 51%|█████     | 114046/225000 [237:19:01<178:28:50,  5.79s/it]
 51%|█████     | 114047/225000 [237:19:06<169:18:59,  5.49s/it]
 51%|█████     | 114048/225000 [237:19:08<141:32:33,  4.59s/it]
 51%|█████     | 114049/225000 [237:19:13<138:52:21,  4.51s/it]
 51%|█████     | 114050/225000 [237:19:16<130:45:52,  4.24s/it]
                                                               
{'loss': '0.937', 'grad_norm': '1.766', 'learning_rate': '2.628e-05', 'epoch': '0.5069', 'num_input_tokens_seen': 396535536, 'train_runtime': '8.544e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114050/225000 [237:19:16<130:45:52,  4.24s/it]
 51%|█████     | 114051/225000 [237:19:21<137:42:05,  4.47s/it]
 51%|█████     | 114052/225000 [237:19:25<135:16:32,  4.39s/it]
 51%|█████     | 114053/225000 [237:19:30<139:01:22,  4.51s/it]
 51%|█████     | 114054/225000 [237:19:33<125:14:29,  4.06s/it]
 51%|█████     | 114055/225000 [237:19:37<119:55:47,  3.89s/it]
 51%|█████     | 114056/225000 [237:19:44<151:24:07,  4.91s/it]
 51%|█████     | 114057/225000 [237:19:51<168:44:12,  5.48s/it]
 51%|█████     | 114058/225000 [237:19:54<144:49:13,  4.70s/it]
 51%|█████     | 114059/225000 [237:20:00<158:20:00,  5.14s/it]
 51%|█████     | 114060/225000 [237:20:04<148:11:54,  4.81s/it]
 51%|█████     | 114061/225000 [237:20:08<138:15:31,  4.49s/it]
 51%|█████     | 114062/225000 [237:20:12<133:02:04,  4.32s/it]
 51%|█████     | 114063/225000 [237:20:15<127:00:16,  4.12s/it]
 51%|█████     | 114064/225000 [237:20:19<125:43:09,  4.08s/it]
 51%|█████     | 114065/225000 [237:20:23<123:44:12,  4.02s/it]
 51%|█████     | 114066/225000 [237:20:27<121:19:54,  3.94s/it]
 51%|█████     | 114067/225000 [237:20:37<177:52:39,  5.77s/it]
 51%|█████     | 114068/225000 [237:20:41<166:23:34,  5.40s/it]
 51%|█████     | 114069/225000 [237:20:45<145:37:09,  4.73s/it]
 51%|█████     | 114070/225000 [237:20:47<128:43:22,  4.18s/it]
 51%|█████     | 114071/225000 [237:20:57<174:18:47,  5.66s/it]
 51%|█████     | 114072/225000 [237:21:01<164:58:25,  5.35s/it]
 51%|█████     | 114073/225000 [237:21:05<147:59:25,  4.80s/it]
 51%|█████     | 114074/225000 [237:21:11<157:01:51,  5.10s/it]
 51%|█████     | 114075/225000 [237:21:14<141:09:07,  4.58s/it]
 51%|█████     | 114076/225000 [237:21:17<130:24:04,  4.23s/it]
 51%|█████     | 114077/225000 [237:21:21<127:36:49,  4.14s/it]
 51%|█████     | 114078/225000 [237:21:25<124:42:10,  4.05s/it]
 51%|█████     | 114079/225000 [237:21:29<120:29:22,  3.91s/it]
 51%|█████     | 114080/225000 [237:21:32<118:25:45,  3.84s/it]
 51%|█████     | 114081/225000 [237:21:42<169:29:43,  5.50s/it]
 51%|█████     | 114082/225000 [237:21:47<165:29:52,  5.37s/it]
 51%|█████     | 114083/225000 [237:21:51<150:30:10,  4.88s/it]
 51%|█████     | 114084/225000 [237:21:54<139:30:03,  4.53s/it]
 51%|█████     | 114085/225000 [237:21:59<137:46:18,  4.47s/it]
 51%|█████     | 114086/225000 [237:22:02<129:02:22,  4.19s/it]
 51%|█████     | 114087/225000 [237:22:06<121:31:24,  3.94s/it]
 51%|█████     | 114088/225000 [237:22:09<116:17:29,  3.77s/it]
 51%|█████     | 114089/225000 [237:22:13<122:18:40,  3.97s/it]
 51%|█████     | 114090/225000 [237:22:24<184:52:32,  6.00s/it]
 51%|█████     | 114091/225000 [237:22:29<174:11:10,  5.65s/it]
 51%|█████     | 114092/225000 [237:22:33<156:53:55,  5.09s/it]
 51%|█████     | 114093/225000 [237:22:37<154:24:12,  5.01s/it]
 51%|█████     | 114094/225000 [237:22:47<196:12:26,  6.37s/it]
 51%|█████     | 114095/225000 [237:22:50<169:00:07,  5.49s/it]
 51%|█████     | 114096/225000 [237:22:54<149:29:33,  4.85s/it]
 51%|█████     | 114097/225000 [237:22:59<148:45:12,  4.83s/it]
 51%|█████     | 114098/225000 [237:23:02<138:12:27,  4.49s/it]
 51%|█████     | 114099/225000 [237:23:05<124:54:22,  4.05s/it]
 51%|█████     | 114100/225000 [237:23:09<123:47:42,  4.02s/it]
                                                               
{'loss': '0.9209', 'grad_norm': '1.437', 'learning_rate': '2.626e-05', 'epoch': '0.5071', 'num_input_tokens_seen': 396636424, 'train_runtime': '8.546e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114100/225000 [237:23:09<123:47:42,  4.02s/it]
 51%|█████     | 114101/225000 [237:23:13<123:09:39,  4.00s/it]
 51%|█████     | 114102/225000 [237:23:16<114:52:56,  3.73s/it]
 51%|█████     | 114103/225000 [237:23:20<112:24:22,  3.65s/it]
 51%|█████     | 114104/225000 [237:23:24<116:39:17,  3.79s/it]
 51%|█████     | 114105/225000 [237:23:28<121:25:22,  3.94s/it]
 51%|█████     | 114106/225000 [237:23:34<140:30:18,  4.56s/it]
 51%|█████     | 114107/225000 [237:23:46<208:20:29,  6.76s/it]
 51%|█████     | 114108/225000 [237:23:49<176:45:26,  5.74s/it]
 51%|█████     | 114109/225000 [237:23:53<154:28:32,  5.01s/it]
 51%|█████     | 114110/225000 [237:23:57<144:52:00,  4.70s/it]
 51%|█████     | 114111/225000 [237:24:01<140:32:25,  4.56s/it]
 51%|█████     | 114112/225000 [237:24:05<137:36:07,  4.47s/it]
 51%|█████     | 114113/225000 [237:24:17<208:25:06,  6.77s/it]
 51%|█████     | 114114/225000 [237:24:23<193:17:50,  6.28s/it]
 51%|█████     | 114115/225000 [237:24:27<177:55:47,  5.78s/it]
 51%|█████     | 114116/225000 [237:24:31<162:51:33,  5.29s/it]
 51%|█████     | 114117/225000 [237:24:34<143:09:17,  4.65s/it]
 51%|█████     | 114118/225000 [237:24:39<141:44:30,  4.60s/it]
 51%|█████     | 114119/225000 [237:24:42<131:19:30,  4.26s/it]
 51%|█████     | 114120/225000 [237:24:50<159:39:56,  5.18s/it]
 51%|█████     | 114121/225000 [237:24:55<160:36:59,  5.21s/it]
 51%|█████     | 114122/225000 [237:24:59<149:03:54,  4.84s/it]
 51%|█████     | 114123/225000 [237:25:03<139:29:20,  4.53s/it]
 51%|█████     | 114124/225000 [237:25:14<201:17:08,  6.54s/it]
 51%|█████     | 114125/225000 [237:25:17<172:26:46,  5.60s/it]
 51%|█████     | 114126/225000 [237:25:21<152:06:27,  4.94s/it]
 51%|█████     | 114127/225000 [237:25:25<141:46:18,  4.60s/it]
 51%|█████     | 114128/225000 [237:25:28<133:01:54,  4.32s/it]
 51%|█████     | 114129/225000 [237:25:32<131:16:32,  4.26s/it]
 51%|█████     | 114130/225000 [237:25:43<187:16:34,  6.08s/it]
 51%|█████     | 114131/225000 [237:25:47<169:52:30,  5.52s/it]
 51%|█████     | 114132/225000 [237:25:51<153:09:18,  4.97s/it]
 51%|█████     | 114133/225000 [237:25:55<143:43:45,  4.67s/it]
 51%|█████     | 114134/225000 [237:25:59<142:37:15,  4.63s/it]
 51%|█████     | 114135/225000 [237:26:03<133:14:59,  4.33s/it]
 51%|█████     | 114136/225000 [237:26:08<138:12:08,  4.49s/it]
 51%|█████     | 114137/225000 [237:26:17<185:55:45,  6.04s/it]
 51%|█████     | 114138/225000 [237:26:20<159:52:48,  5.19s/it]
 51%|█████     | 114139/225000 [237:26:23<139:43:45,  4.54s/it]
 51%|█████     | 114140/225000 [237:26:33<184:02:59,  5.98s/it]
 51%|█████     | 114141/225000 [237:26:36<155:54:37,  5.06s/it]
 51%|█████     | 114142/225000 [237:26:39<134:35:46,  4.37s/it]
 51%|█████     | 114143/225000 [237:26:42<123:50:56,  4.02s/it]
 51%|█████     | 114144/225000 [237:26:50<164:01:09,  5.33s/it]
 51%|█████     | 114145/225000 [237:26:55<158:25:21,  5.14s/it]
 51%|█████     | 114146/225000 [237:26:58<139:30:58,  4.53s/it]
 51%|█████     | 114147/225000 [237:27:02<138:23:26,  4.49s/it]
 51%|█████     | 114148/225000 [237:27:06<126:52:23,  4.12s/it]
 51%|█████     | 114149/225000 [237:27:15<178:46:32,  5.81s/it]
 51%|█████     | 114150/225000 [237:27:20<165:51:02,  5.39s/it]
                                                               
{'loss': '0.9383', 'grad_norm': '1.273', 'learning_rate': '2.624e-05', 'epoch': '0.5073', 'num_input_tokens_seen': 396745192, 'train_runtime': '8.548e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114150/225000 [237:27:20<165:51:02,  5.39s/it]
 51%|█████     | 114151/225000 [237:27:29<201:20:32,  6.54s/it]
 51%|█████     | 114152/225000 [237:27:32<173:05:10,  5.62s/it]
 51%|█████     | 114153/225000 [237:27:36<152:19:05,  4.95s/it]
 51%|█████     | 114154/225000 [237:27:40<141:23:05,  4.59s/it]
 51%|█████     | 114155/225000 [237:27:48<176:53:10,  5.74s/it]
 51%|█████     | 114156/225000 [237:27:53<166:10:28,  5.40s/it]
 51%|█████     | 114157/225000 [237:27:55<141:11:22,  4.59s/it]
 51%|█████     | 114158/225000 [237:27:59<131:37:41,  4.28s/it]
 51%|█████     | 114159/225000 [237:28:06<156:44:41,  5.09s/it]
 51%|█████     | 114160/225000 [237:28:15<198:11:49,  6.44s/it]
 51%|█████     | 114161/225000 [237:28:20<177:02:51,  5.75s/it]
 51%|█████     | 114162/225000 [237:28:23<157:27:27,  5.11s/it]
 51%|█████     | 114163/225000 [237:28:26<135:30:13,  4.40s/it]
 51%|█████     | 114164/225000 [237:28:29<123:53:33,  4.02s/it]
 51%|█████     | 114165/225000 [237:28:35<143:03:44,  4.65s/it]
 51%|█████     | 114166/225000 [237:28:43<175:25:31,  5.70s/it]
 51%|█████     | 114167/225000 [237:28:46<151:30:34,  4.92s/it]
 51%|█████     | 114168/225000 [237:28:50<135:15:07,  4.39s/it]
 51%|█████     | 114169/225000 [237:28:56<150:28:59,  4.89s/it]
 51%|█████     | 114170/225000 [237:29:03<171:31:56,  5.57s/it]
 51%|█████     | 114171/225000 [237:29:06<148:24:17,  4.82s/it]
 51%|█████     | 114172/225000 [237:29:13<169:20:59,  5.50s/it]
 51%|█████     | 114173/225000 [237:29:17<156:42:32,  5.09s/it]
 51%|█████     | 114174/225000 [237:29:21<149:35:22,  4.86s/it]
 51%|█████     | 114175/225000 [237:29:30<186:49:34,  6.07s/it]
 51%|█████     | 114176/225000 [237:29:35<173:28:42,  5.64s/it]
 51%|█████     | 114177/225000 [237:29:39<163:08:31,  5.30s/it]
 51%|█████     | 114178/225000 [237:29:43<146:15:06,  4.75s/it]
 51%|█████     | 114179/225000 [237:29:47<140:13:58,  4.56s/it]
 51%|█████     | 114180/225000 [237:29:51<136:08:56,  4.42s/it]
 51%|█████     | 114181/225000 [237:29:55<128:30:24,  4.17s/it]
 51%|█████     | 114182/225000 [237:30:03<169:17:20,  5.50s/it]
 51%|█████     | 114183/225000 [237:30:10<180:35:17,  5.87s/it]
 51%|█████     | 114184/225000 [237:30:13<157:53:06,  5.13s/it]
 51%|█████     | 114185/225000 [237:30:17<139:34:59,  4.53s/it]
 51%|█████     | 114186/225000 [237:30:21<135:57:57,  4.42s/it]
 51%|█████     | 114187/225000 [237:30:30<184:34:03,  6.00s/it]
 51%|█████     | 114188/225000 [237:30:34<162:13:20,  5.27s/it]
 51%|█████     | 114189/225000 [237:30:37<145:48:08,  4.74s/it]
 51%|█████     | 114190/225000 [237:30:49<209:41:37,  6.81s/it]
 51%|█████     | 114191/225000 [237:30:52<175:56:43,  5.72s/it]
 51%|█████     | 114192/225000 [237:31:00<196:34:33,  6.39s/it]
 51%|█████     | 114193/225000 [237:31:08<210:37:45,  6.84s/it]
 51%|█████     | 114194/225000 [237:31:12<181:58:04,  5.91s/it]
 51%|█████     | 114195/225000 [237:31:16<164:24:52,  5.34s/it]
 51%|█████     | 114196/225000 [237:31:22<168:13:23,  5.47s/it]
 51%|█████     | 114197/225000 [237:31:27<170:50:17,  5.55s/it]
 51%|█████     | 114198/225000 [237:31:37<205:18:27,  6.67s/it]
 51%|█████     | 114199/225000 [237:31:40<176:37:49,  5.74s/it]
 51%|█████     | 114200/225000 [237:31:44<157:30:18,  5.12s/it]
                                                               
{'loss': '0.9122', 'grad_norm': '1.27', 'learning_rate': '2.622e-05', 'epoch': '0.5076', 'num_input_tokens_seen': 396860912, 'train_runtime': '8.551e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114200/225000 [237:31:44<157:30:18,  5.12s/it]
 51%|█████     | 114201/225000 [237:31:47<141:03:50,  4.58s/it]
 51%|█████     | 114202/225000 [237:31:50<128:21:03,  4.17s/it]
 51%|█████     | 114203/225000 [237:32:00<176:13:42,  5.73s/it]
 51%|█████     | 114204/225000 [237:32:03<156:42:31,  5.09s/it]
 51%|█████     | 114205/225000 [237:32:07<140:05:27,  4.55s/it]
 51%|█████     | 114206/225000 [237:32:10<123:55:44,  4.03s/it]
 51%|█████     | 114207/225000 [237:32:18<166:33:15,  5.41s/it]
 51%|█████     | 114208/225000 [237:32:22<148:37:42,  4.83s/it]
 51%|█████     | 114209/225000 [237:32:24<129:01:59,  4.19s/it]
 51%|█████     | 114210/225000 [237:32:27<116:12:26,  3.78s/it]
 51%|█████     | 114211/225000 [237:32:32<123:17:52,  4.01s/it]
 51%|█████     | 114212/225000 [237:32:41<167:46:49,  5.45s/it]
 51%|█████     | 114213/225000 [237:32:43<139:11:34,  4.52s/it]
 51%|█████     | 114214/225000 [237:32:46<124:51:07,  4.06s/it]
 51%|█████     | 114215/225000 [237:32:53<151:50:52,  4.93s/it]
 51%|█████     | 114216/225000 [237:32:57<143:58:28,  4.68s/it]
 51%|█████     | 114217/225000 [237:33:02<145:07:43,  4.72s/it]
 51%|█████     | 114218/225000 [237:33:06<139:55:14,  4.55s/it]
 51%|█████     | 114219/225000 [237:33:10<139:32:25,  4.53s/it]
 51%|█████     | 114220/225000 [237:33:15<136:02:03,  4.42s/it]
 51%|█████     | 114221/225000 [237:33:19<134:02:28,  4.36s/it]
 51%|█████     | 114222/225000 [237:33:23<133:11:54,  4.33s/it]
 51%|█████     | 114223/225000 [237:33:26<124:34:00,  4.05s/it]
 51%|█████     | 114224/225000 [237:33:34<154:58:41,  5.04s/it]
 51%|█████     | 114225/225000 [237:33:42<187:16:49,  6.09s/it]
 51%|█████     | 114226/225000 [237:33:46<162:19:37,  5.28s/it]
 51%|█████     | 114227/225000 [237:33:49<142:48:09,  4.64s/it]
 51%|█████     | 114228/225000 [237:33:57<174:01:43,  5.66s/it]
 51%|█████     | 114229/225000 [237:34:00<150:21:45,  4.89s/it]
 51%|█████     | 114230/225000 [237:34:03<133:05:42,  4.33s/it]
 51%|█████     | 114231/225000 [237:34:06<125:12:16,  4.07s/it]
 51%|█████     | 114232/225000 [237:34:10<117:36:42,  3.82s/it]
 51%|█████     | 114233/225000 [237:34:13<113:21:13,  3.68s/it]
 51%|█████     | 114234/225000 [237:34:22<166:03:45,  5.40s/it]
 51%|█████     | 114235/225000 [237:34:26<146:09:23,  4.75s/it]
 51%|█████     | 114236/225000 [237:34:29<131:51:54,  4.29s/it]
 51%|█████     | 114237/225000 [237:34:42<213:58:39,  6.95s/it]
 51%|█████     | 114238/225000 [237:34:45<176:42:40,  5.74s/it]
 51%|█████     | 114239/225000 [237:34:48<152:52:18,  4.97s/it]
 51%|█████     | 114240/225000 [237:34:52<142:15:51,  4.62s/it]
 51%|█████     | 114241/225000 [237:35:01<185:29:42,  6.03s/it]
 51%|█████     | 114242/225000 [237:35:06<177:26:53,  5.77s/it]
 51%|█████     | 114243/225000 [237:35:10<160:44:40,  5.22s/it]
 51%|█████     | 114244/225000 [237:35:14<143:48:18,  4.67s/it]
 51%|█████     | 114245/225000 [237:35:17<126:36:48,  4.12s/it]
 51%|█████     | 114246/225000 [237:35:21<125:35:31,  4.08s/it]
 51%|█████     | 114247/225000 [237:35:24<119:25:23,  3.88s/it]
 51%|█████     | 114248/225000 [237:35:28<124:29:02,  4.05s/it]
 51%|█████     | 114249/225000 [237:35:34<137:20:12,  4.46s/it]
 51%|█████     | 114250/225000 [237:35:42<171:12:40,  5.57s/it]
                                                               
{'loss': '0.9256', 'grad_norm': '1.605', 'learning_rate': '2.621e-05', 'epoch': '0.5078', 'num_input_tokens_seen': 396965896, 'train_runtime': '8.553e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114250/225000 [237:35:42<171:12:40,  5.57s/it]
 51%|█████     | 114251/225000 [237:35:45<149:04:18,  4.85s/it]
 51%|█████     | 114252/225000 [237:35:48<133:42:12,  4.35s/it]
 51%|█████     | 114253/225000 [237:35:51<121:05:44,  3.94s/it]
 51%|█████     | 114254/225000 [237:35:58<149:50:01,  4.87s/it]
 51%|█████     | 114255/225000 [237:36:05<162:40:05,  5.29s/it]
 51%|█████     | 114256/225000 [237:36:07<140:02:46,  4.55s/it]
 51%|█████     | 114257/225000 [237:36:13<147:02:27,  4.78s/it]
 51%|█████     | 114258/225000 [237:36:23<195:11:01,  6.35s/it]
 51%|█████     | 114259/225000 [237:36:27<172:13:26,  5.60s/it]
 51%|█████     | 114260/225000 [237:36:31<160:14:07,  5.21s/it]
 51%|█████     | 114261/225000 [237:36:34<142:11:38,  4.62s/it]
 51%|█████     | 114262/225000 [237:36:39<148:19:55,  4.82s/it]
 51%|█████     | 114263/225000 [237:36:48<178:56:05,  5.82s/it]
 51%|█████     | 114264/225000 [237:36:51<153:04:30,  4.98s/it]
 51%|█████     | 114265/225000 [237:36:53<133:01:05,  4.32s/it]
 51%|█████     | 114266/225000 [237:37:00<156:50:29,  5.10s/it]
 51%|█████     | 114267/225000 [237:37:05<149:55:37,  4.87s/it]
 51%|█████     | 114268/225000 [237:37:08<138:11:07,  4.49s/it]
 51%|█████     | 114269/225000 [237:37:11<125:12:56,  4.07s/it]
 51%|█████     | 114270/225000 [237:37:15<116:57:26,  3.80s/it]
 51%|█████     | 114271/225000 [237:37:21<144:22:24,  4.69s/it]
 51%|█████     | 114272/225000 [237:37:29<174:07:43,  5.66s/it]
 51%|█████     | 114273/225000 [237:37:33<158:56:54,  5.17s/it]
 51%|█████     | 114274/225000 [237:37:38<150:55:25,  4.91s/it]
 51%|█████     | 114275/225000 [237:37:50<222:46:00,  7.24s/it]
 51%|█████     | 114276/225000 [237:37:54<187:29:59,  6.10s/it]
 51%|█████     | 114277/225000 [237:37:57<164:29:58,  5.35s/it]
 51%|█████     | 114278/225000 [237:38:00<142:28:45,  4.63s/it]
 51%|█████     | 114279/225000 [237:38:04<131:03:39,  4.26s/it]
 51%|█████     | 114280/225000 [237:38:13<180:30:02,  5.87s/it]
 51%|█████     | 114281/225000 [237:38:17<159:29:15,  5.19s/it]
 51%|█████     | 114282/225000 [237:38:20<140:05:56,  4.56s/it]
 51%|█████     | 114283/225000 [237:38:23<130:44:33,  4.25s/it]
 51%|█████     | 114284/225000 [237:38:32<173:20:29,  5.64s/it]
 51%|█████     | 114285/225000 [237:38:43<215:59:26,  7.02s/it]
 51%|█████     | 114286/225000 [237:38:46<183:56:11,  5.98s/it]
 51%|█████     | 114287/225000 [237:38:49<151:51:40,  4.94s/it]
 51%|█████     | 114288/225000 [237:38:52<136:42:11,  4.45s/it]
 51%|█████     | 114289/225000 [237:39:00<173:32:46,  5.64s/it]
 51%|█████     | 114290/225000 [237:39:03<148:37:17,  4.83s/it]
 51%|█████     | 114291/225000 [237:39:10<163:00:52,  5.30s/it]
 51%|█████     | 114292/225000 [237:39:19<202:39:22,  6.59s/it]
 51%|█████     | 114293/225000 [237:39:23<173:59:28,  5.66s/it]
 51%|█████     | 114294/225000 [237:39:26<153:22:51,  4.99s/it]
 51%|█████     | 114295/225000 [237:39:30<144:59:56,  4.72s/it]
 51%|█████     | 114296/225000 [237:39:35<142:28:58,  4.63s/it]
 51%|█████     | 114297/225000 [237:39:39<141:26:40,  4.60s/it]
 51%|█████     | 114298/225000 [237:39:44<140:47:23,  4.58s/it]
 51%|█████     | 114299/225000 [237:39:47<132:00:53,  4.29s/it]
 51%|█████     | 114300/225000 [237:39:56<173:49:07,  5.65s/it]
                                                               
{'loss': '0.9366', 'grad_norm': '2.152', 'learning_rate': '2.619e-05', 'epoch': '0.508', 'num_input_tokens_seen': 397076224, 'train_runtime': '8.556e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114300/225000 [237:39:56<173:49:07,  5.65s/it]
 51%|█████     | 114301/225000 [237:40:02<172:12:27,  5.60s/it]
 51%|█████     | 114302/225000 [237:40:04<146:00:13,  4.75s/it]
 51%|█████     | 114303/225000 [237:40:10<155:32:54,  5.06s/it]
 51%|█████     | 114304/225000 [237:40:17<169:40:20,  5.52s/it]
 51%|█████     | 114305/225000 [237:40:28<218:02:46,  7.09s/it]
 51%|█████     | 114306/225000 [237:40:32<191:56:07,  6.24s/it]
 51%|█████     | 114307/225000 [237:40:35<163:01:22,  5.30s/it]
 51%|█████     | 114308/225000 [237:40:39<150:36:35,  4.90s/it]
 51%|█████     | 114309/225000 [237:40:48<186:49:01,  6.08s/it]
 51%|█████     | 114310/225000 [237:40:52<167:32:46,  5.45s/it]
 51%|█████     | 114311/225000 [237:40:56<156:26:52,  5.09s/it]
 51%|█████     | 114312/225000 [237:41:00<142:07:46,  4.62s/it]
 51%|█████     | 114313/225000 [237:41:03<130:40:02,  4.25s/it]
 51%|█████     | 114314/225000 [237:41:07<126:06:50,  4.10s/it]
 51%|█████     | 114315/225000 [237:41:17<184:40:52,  6.01s/it]
 51%|█████     | 114316/225000 [237:41:25<201:55:53,  6.57s/it]
 51%|█████     | 114317/225000 [237:41:29<178:14:44,  5.80s/it]
 51%|█████     | 114318/225000 [237:41:32<152:48:24,  4.97s/it]
 51%|█████     | 114319/225000 [237:41:43<204:52:37,  6.66s/it]
 51%|█████     | 114320/225000 [237:41:46<172:38:41,  5.62s/it]
 51%|█████     | 114321/225000 [237:41:51<170:02:39,  5.53s/it]
 51%|█████     | 114322/225000 [237:41:58<181:27:09,  5.90s/it]
 51%|█████     | 114323/225000 [237:42:02<165:40:38,  5.39s/it]
 51%|█████     | 114324/225000 [237:42:06<149:36:55,  4.87s/it]
 51%|█████     | 114325/225000 [237:42:10<140:27:50,  4.57s/it]
 51%|█████     | 114326/225000 [237:42:14<137:20:08,  4.47s/it]
 51%|█████     | 114327/225000 [237:42:18<135:37:49,  4.41s/it]
 51%|█████     | 114328/225000 [237:42:28<184:17:04,  5.99s/it]
 51%|█████     | 114329/225000 [237:42:33<177:39:30,  5.78s/it]
 51%|█████     | 114330/225000 [237:42:36<154:57:58,  5.04s/it]
 51%|█████     | 114331/225000 [237:42:40<140:34:48,  4.57s/it]
 51%|█████     | 114332/225000 [237:42:43<131:01:44,  4.26s/it]
 51%|█████     | 114333/225000 [237:42:53<179:23:52,  5.84s/it]
 51%|█████     | 114334/225000 [237:42:59<184:54:58,  6.02s/it]
 51%|█████     | 114335/225000 [237:43:02<156:47:03,  5.10s/it]
 51%|█████     | 114336/225000 [237:43:06<142:46:48,  4.64s/it]
 51%|█████     | 114337/225000 [237:43:10<136:27:09,  4.44s/it]
 51%|█████     | 114338/225000 [237:43:18<171:43:18,  5.59s/it]
 51%|█████     | 114339/225000 [237:43:22<159:21:39,  5.18s/it]
 51%|█████     | 114340/225000 [237:43:28<165:36:04,  5.39s/it]
 51%|█████     | 114341/225000 [237:43:32<146:24:07,  4.76s/it]
 51%|█████     | 114342/225000 [237:43:35<136:04:13,  4.43s/it]
 51%|█████     | 114343/225000 [237:43:38<122:44:07,  3.99s/it]
 51%|█████     | 114344/225000 [237:43:42<119:28:54,  3.89s/it]
 51%|█████     | 114345/225000 [237:43:46<122:38:39,  3.99s/it]
 51%|█████     | 114346/225000 [237:43:51<127:16:57,  4.14s/it]
 51%|█████     | 114347/225000 [237:44:03<199:42:45,  6.50s/it]
 51%|█████     | 114348/225000 [237:44:07<178:00:59,  5.79s/it]
 51%|█████     | 114349/225000 [237:44:10<151:07:51,  4.92s/it]
 51%|█████     | 114350/225000 [237:44:13<134:43:41,  4.38s/it]
                                                               
{'loss': '0.9236', 'grad_norm': '1.488', 'learning_rate': '2.617e-05', 'epoch': '0.5082', 'num_input_tokens_seen': 397188728, 'train_runtime': '8.559e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114350/225000 [237:44:13<134:43:41,  4.38s/it]
 51%|█████     | 114351/225000 [237:44:17<132:46:19,  4.32s/it]
 51%|█████     | 114352/225000 [237:44:28<193:14:14,  6.29s/it]
 51%|█████     | 114353/225000 [237:44:31<162:45:16,  5.30s/it]
 51%|█████     | 114354/225000 [237:44:34<142:11:22,  4.63s/it]
 51%|█████     | 114355/225000 [237:44:39<144:31:45,  4.70s/it]
 51%|█████     | 114356/225000 [237:44:42<134:46:43,  4.39s/it]
 51%|█████     | 114357/225000 [237:44:53<191:40:21,  6.24s/it]
 51%|█████     | 114358/225000 [237:44:56<161:42:05,  5.26s/it]
 51%|█████     | 114359/225000 [237:44:59<145:31:23,  4.73s/it]
 51%|█████     | 114360/225000 [237:45:02<129:24:11,  4.21s/it]
 51%|█████     | 114361/225000 [237:45:11<174:13:01,  5.67s/it]
 51%|█████     | 114362/225000 [237:45:15<157:05:04,  5.11s/it]
 51%|█████     | 114363/225000 [237:45:20<150:52:11,  4.91s/it]
 51%|█████     | 114364/225000 [237:45:23<140:01:49,  4.56s/it]
 51%|█████     | 114365/225000 [237:45:29<145:49:28,  4.75s/it]
 51%|█████     | 114366/225000 [237:45:41<219:49:28,  7.15s/it]
 51%|█████     | 114367/225000 [237:45:45<183:13:47,  5.96s/it]
 51%|█████     | 114368/225000 [237:45:54<214:13:49,  6.97s/it]
 51%|█████     | 114369/225000 [237:45:57<180:13:29,  5.86s/it]
 51%|█████     | 114370/225000 [237:46:00<149:42:32,  4.87s/it]
 51%|█████     | 114371/225000 [237:46:10<202:33:29,  6.59s/it]
 51%|█████     | 114372/225000 [237:46:15<183:17:41,  5.96s/it]
 51%|█████     | 114373/225000 [237:46:18<160:55:14,  5.24s/it]
 51%|█████     | 114374/225000 [237:46:24<160:35:20,  5.23s/it]
 51%|█████     | 114375/225000 [237:46:28<151:27:58,  4.93s/it]
 51%|█████     | 114376/225000 [237:46:31<133:33:47,  4.35s/it]
 51%|█████     | 114377/225000 [237:46:35<131:33:51,  4.28s/it]
 51%|█████     | 114378/225000 [237:46:38<120:41:47,  3.93s/it]
 51%|█████     | 114379/225000 [237:46:43<126:41:47,  4.12s/it]
 51%|█████     | 114380/225000 [237:46:47<125:59:31,  4.10s/it]
 51%|█████     | 114381/225000 [237:46:58<190:01:38,  6.18s/it]
 51%|█████     | 114382/225000 [237:47:02<168:57:21,  5.50s/it]
 51%|█████     | 114383/225000 [237:47:04<143:17:25,  4.66s/it]
 51%|█████     | 114384/225000 [237:47:07<129:09:15,  4.20s/it]
 51%|█████     | 114385/225000 [237:47:14<151:08:14,  4.92s/it]
 51%|█████     | 114386/225000 [237:47:27<221:11:33,  7.20s/it]
 51%|█████     | 114387/225000 [237:47:30<185:52:09,  6.05s/it]
 51%|█████     | 114388/225000 [237:47:34<163:26:32,  5.32s/it]
 51%|█████     | 114389/225000 [237:47:45<216:19:22,  7.04s/it]
 51%|█████     | 114390/225000 [237:47:47<176:20:40,  5.74s/it]
 51%|█████     | 114391/225000 [237:47:54<183:05:59,  5.96s/it]
 51%|█████     | 114392/225000 [237:47:57<155:25:22,  5.06s/it]
 51%|█████     | 114393/225000 [237:48:01<147:02:08,  4.79s/it]
 51%|█████     | 114394/225000 [237:48:09<179:27:13,  5.84s/it]
 51%|█████     | 114395/225000 [237:48:12<154:48:37,  5.04s/it]
 51%|█████     | 114396/225000 [237:48:15<135:34:43,  4.41s/it]
 51%|█████     | 114397/225000 [237:48:18<123:03:11,  4.01s/it]
 51%|█████     | 114398/225000 [237:48:31<205:00:01,  6.67s/it]
 51%|█████     | 114399/225000 [237:48:37<197:02:14,  6.41s/it]
 51%|█████     | 114400/225000 [237:48:41<172:51:09,  5.63s/it]
                                                               
{'loss': '0.9349', 'grad_norm': '1.251', 'learning_rate': '2.615e-05', 'epoch': '0.5084', 'num_input_tokens_seen': 397303872, 'train_runtime': '8.561e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114400/225000 [237:48:41<172:51:09,  5.63s/it]
 51%|█████     | 114401/225000 [237:48:47<180:15:05,  5.87s/it]
 51%|█████     | 114402/225000 [237:48:52<168:58:10,  5.50s/it]
 51%|█████     | 114403/225000 [237:48:55<146:16:33,  4.76s/it]
 51%|█████     | 114404/225000 [237:48:58<129:55:21,  4.23s/it]
 51%|█████     | 114405/225000 [237:49:01<121:42:51,  3.96s/it]
 51%|█████     | 114406/225000 [237:49:05<115:40:29,  3.77s/it]
 51%|█████     | 114407/225000 [237:49:14<164:04:18,  5.34s/it]
 51%|█████     | 114408/225000 [237:49:18<159:28:32,  5.19s/it]
 51%|█████     | 114409/225000 [237:49:22<145:43:53,  4.74s/it]
 51%|█████     | 114410/225000 [237:49:28<159:27:37,  5.19s/it]
 51%|█████     | 114411/225000 [237:49:31<138:54:07,  4.52s/it]
 51%|█████     | 114412/225000 [237:49:35<134:29:58,  4.38s/it]
 51%|█████     | 114413/225000 [237:49:39<125:40:18,  4.09s/it]
 51%|█████     | 114414/225000 [237:49:42<117:02:01,  3.81s/it]
 51%|█████     | 114415/225000 [237:49:49<143:17:26,  4.66s/it]
 51%|█████     | 114416/225000 [237:49:56<170:02:44,  5.54s/it]
 51%|█████     | 114417/225000 [237:49:59<144:18:59,  4.70s/it]
 51%|█████     | 114418/225000 [237:50:06<164:24:45,  5.35s/it]
 51%|█████     | 114419/225000 [237:50:09<144:44:57,  4.71s/it]
 51%|█████     | 114420/225000 [237:50:12<132:18:55,  4.31s/it]
 51%|█████     | 114421/225000 [237:50:23<188:01:03,  6.12s/it]
 51%|█████     | 114422/225000 [237:50:26<164:13:05,  5.35s/it]
 51%|█████     | 114423/225000 [237:50:30<147:10:18,  4.79s/it]
 51%|█████     | 114424/225000 [237:50:34<144:06:59,  4.69s/it]
 51%|█████     | 114425/225000 [237:50:38<135:58:39,  4.43s/it]
 51%|█████     | 114426/225000 [237:50:48<189:20:21,  6.16s/it]
 51%|█████     | 114427/225000 [237:50:53<173:30:41,  5.65s/it]
 51%|█████     | 114428/225000 [237:50:56<155:36:28,  5.07s/it]
 51%|█████     | 114429/225000 [237:51:00<146:08:34,  4.76s/it]
 51%|█████     | 114430/225000 [237:51:04<131:12:26,  4.27s/it]
 51%|█████     | 114431/225000 [237:51:14<189:28:00,  6.17s/it]
 51%|█████     | 114432/225000 [237:51:19<176:15:55,  5.74s/it]
 51%|█████     | 114433/225000 [237:51:24<165:33:43,  5.39s/it]
 51%|█████     | 114434/225000 [237:51:27<150:56:43,  4.91s/it]
 51%|█████     | 114435/225000 [237:51:31<139:03:16,  4.53s/it]
 51%|█████     | 114436/225000 [237:51:35<134:52:24,  4.39s/it]
 51%|█████     | 114437/225000 [237:51:39<133:14:55,  4.34s/it]
 51%|█████     | 114438/225000 [237:51:43<132:21:20,  4.31s/it]
 51%|█████     | 114439/225000 [237:51:54<186:18:01,  6.07s/it]
 51%|█████     | 114440/225000 [237:52:01<198:32:00,  6.46s/it]
 51%|█████     | 114441/225000 [237:52:04<170:00:33,  5.54s/it]
 51%|█████     | 114442/225000 [237:52:08<148:24:15,  4.83s/it]
 51%|█████     | 114443/225000 [237:52:15<171:58:49,  5.60s/it]
 51%|█████     | 114444/225000 [237:52:21<176:05:12,  5.73s/it]
 51%|█████     | 114445/225000 [237:52:24<152:27:41,  4.96s/it]
 51%|█████     | 114446/225000 [237:52:28<141:15:01,  4.60s/it]
 51%|█████     | 114447/225000 [237:52:32<131:50:11,  4.29s/it]
 51%|█████     | 114448/225000 [237:52:41<177:29:02,  5.78s/it]
 51%|█████     | 114449/225000 [237:52:43<148:21:06,  4.83s/it]
 51%|█████     | 114450/225000 [237:52:47<138:57:18,  4.52s/it]
                                                               
{'loss': '0.9255', 'grad_norm': '1.265', 'learning_rate': '2.613e-05', 'epoch': '0.5087', 'num_input_tokens_seen': 397411584, 'train_runtime': '8.564e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114450/225000 [237:52:47<138:57:18,  4.52s/it]
 51%|█████     | 114451/225000 [237:52:52<145:18:34,  4.73s/it]
 51%|█████     | 114452/225000 [237:52:58<149:56:58,  4.88s/it]
 51%|█████     | 114453/225000 [237:53:04<161:23:48,  5.26s/it]
 51%|█████     | 114454/225000 [237:53:08<147:39:51,  4.81s/it]
 51%|█████     | 114455/225000 [237:53:11<135:44:45,  4.42s/it]
 51%|█████     | 114456/225000 [237:53:20<175:19:52,  5.71s/it]
 51%|█████     | 114457/225000 [237:53:27<185:31:07,  6.04s/it]
 51%|█████     | 114458/225000 [237:53:31<165:57:08,  5.40s/it]
 51%|█████     | 114459/225000 [237:53:34<145:49:26,  4.75s/it]
 51%|█████     | 114460/225000 [237:53:37<134:02:53,  4.37s/it]
 51%|█████     | 114461/225000 [237:53:48<194:43:39,  6.34s/it]
 51%|█████     | 114462/225000 [237:53:51<165:15:10,  5.38s/it]
 51%|█████     | 114463/225000 [237:53:55<147:41:14,  4.81s/it]
 51%|█████     | 114464/225000 [237:53:58<136:52:41,  4.46s/it]
 51%|█████     | 114465/225000 [237:54:02<128:47:41,  4.19s/it]
 51%|█████     | 114466/225000 [237:54:12<186:15:03,  6.07s/it]
 51%|█████     | 114467/225000 [237:54:16<166:16:48,  5.42s/it]
 51%|█████     | 114468/225000 [237:54:20<148:03:35,  4.82s/it]
 51%|█████     | 114469/225000 [237:54:24<138:58:28,  4.53s/it]
 51%|█████     | 114470/225000 [237:54:28<134:53:16,  4.39s/it]
 51%|█████     | 114471/225000 [237:54:31<124:55:57,  4.07s/it]
 51%|█████     | 114472/225000 [237:54:35<125:37:12,  4.09s/it]
 51%|█████     | 114473/225000 [237:54:38<117:11:59,  3.82s/it]
 51%|█████     | 114474/225000 [237:54:42<117:33:35,  3.83s/it]
 51%|█████     | 114475/225000 [237:54:52<176:31:08,  5.75s/it]
 51%|█████     | 114476/225000 [237:54:57<168:27:45,  5.49s/it]
 51%|█████     | 114477/225000 [237:55:01<152:26:14,  4.97s/it]
 51%|█████     | 114478/225000 [237:55:08<167:22:27,  5.45s/it]
 51%|█████     | 114479/225000 [237:55:12<154:48:33,  5.04s/it]
 51%|█████     | 114480/225000 [237:55:17<152:42:03,  4.97s/it]
 51%|█████     | 114481/225000 [237:55:25<182:20:18,  5.94s/it]
 51%|█████     | 114482/225000 [237:55:29<166:48:12,  5.43s/it]
 51%|█████     | 114483/225000 [237:55:33<149:53:43,  4.88s/it]
 51%|█████     | 114484/225000 [237:55:36<140:09:38,  4.57s/it]
 51%|█████     | 114485/225000 [237:55:48<201:25:26,  6.56s/it]
 51%|█████     | 114486/225000 [237:55:51<174:13:45,  5.68s/it]
 51%|█████     | 114487/225000 [237:55:55<155:22:59,  5.06s/it]
 51%|█████     | 114488/225000 [237:55:59<143:21:37,  4.67s/it]
 51%|█████     | 114489/225000 [237:56:09<195:39:29,  6.37s/it]
 51%|█████     | 114490/225000 [237:56:14<181:18:44,  5.91s/it]
 51%|█████     | 114491/225000 [237:56:17<159:15:58,  5.19s/it]
 51%|█████     | 114492/225000 [237:56:21<145:21:51,  4.74s/it]
 51%|█████     | 114493/225000 [237:56:24<134:27:46,  4.38s/it]
 51%|█████     | 114494/225000 [237:56:28<123:15:09,  4.02s/it]
 51%|█████     | 114495/225000 [237:56:32<126:50:51,  4.13s/it]
 51%|█████     | 114496/225000 [237:56:35<120:26:50,  3.92s/it]
 51%|█████     | 114497/225000 [237:56:40<129:17:36,  4.21s/it]
 51%|█████     | 114498/225000 [237:56:51<185:31:47,  6.04s/it]
 51%|█████     | 114499/225000 [237:56:54<159:04:24,  5.18s/it]
 51%|█████     | 114500/225000 [237:56:58<146:02:41,  4.76s/it]
                                                               
{'loss': '0.9176', 'grad_norm': '1.133', 'learning_rate': '2.611e-05', 'epoch': '0.5089', 'num_input_tokens_seen': 397519432, 'train_runtime': '8.566e+05', 'train_tokens_per_second': '464.1'}

 51%|█████     | 114500/225000 [237:56:58<146:02:41,  4.76s/it]
 51%|█████     | 114501/225000 [237:57:04<161:13:20,  5.25s/it]
 51%|█████     | 114502/225000 [237:57:09<160:23:19,  5.23s/it]
 51%|█████     | 114503/225000 [237:57:12<142:19:52,  4.64s/it]
 51%|█████     | 114504/225000 [237:57:24<203:32:13,  6.63s/it]
 51%|█████     | 114505/225000 [237:57:27<172:13:59,  5.61s/it]
 51%|█████     | 114506/225000 [237:57:30<150:55:06,  4.92s/it]
 51%|█████     | 114507/225000 [237:57:35<151:32:32,  4.94s/it]
 51%|█████     | 114508/225000 [237:57:42<167:21:53,  5.45s/it]
 51%|█████     | 114509/225000 [237:57:45<147:22:47,  4.80s/it]
 51%|█████     | 114510/225000 [237:57:49<137:52:44,  4.49s/it]
 51%|█████     | 114511/225000 [237:57:58<177:57:25,  5.80s/it]
 51%|█████     | 114512/225000 [237:58:04<178:59:40,  5.83s/it]
 51%|█████     | 114513/225000 [237:58:07<154:27:42,  5.03s/it]
 51%|█████     | 114514/225000 [237:58:17<200:58:38,  6.55s/it]
 51%|█████     | 114515/225000 [237:58:22<186:30:29,  6.08s/it]
 51%|█████     | 114516/225000 [237:58:29<193:54:28,  6.32s/it]
 51%|█████     | 114517/225000 [237:58:32<162:36:43,  5.30s/it]
 51%|█████     | 114518/225000 [237:58:35<143:41:31,  4.68s/it]
 51%|█████     | 114519/225000 [237:58:43<178:32:07,  5.82s/it]
 51%|█████     | 114520/225000 [237:58:48<167:31:06,  5.46s/it]
 51%|█████     | 114521/225000 [237:58:51<148:10:34,  4.83s/it]
 51%|█████     | 114522/225000 [237:58:56<141:31:52,  4.61s/it]
 51%|█████     | 114523/225000 [237:58:59<128:46:56,  4.20s/it]
 51%|█████     | 114524/225000 [237:59:02<123:01:57,  4.01s/it]
 51%|█████     | 114525/225000 [237:59:10<158:14:15,  5.16s/it]
 51%|█████     | 114526/225000 [237:59:17<169:16:03,  5.52s/it]
 51%|█████     | 114527/225000 [237:59:19<145:19:14,  4.74s/it]
 51%|█████     | 114528/225000 [237:59:23<132:33:14,  4.32s/it]
 51%|█████     | 114529/225000 [237:59:31<166:19:58,  5.42s/it]
 51%|█████     | 114530/225000 [237:59:40<197:22:48,  6.43s/it]
 51%|█████     | 114531/225000 [237:59:42<160:53:00,  5.24s/it]
 51%|█████     | 114532/225000 [237:59:49<179:23:03,  5.85s/it]
 51%|█████     | 114533/225000 [237:59:53<160:41:12,  5.24s/it]
 51%|█████     | 114534/225000 [237:59:56<140:11:00,  4.57s/it]
 51%|█████     | 114535/225000 [238:00:00<130:50:30,  4.26s/it]
 51%|█████     | 114536/225000 [238:00:10<184:18:38,  6.01s/it]
 51%|█████     | 114537/225000 [238:00:13<158:55:19,  5.18s/it]
 51%|█████     | 114538/225000 [238:00:20<173:42:30,  5.66s/it]
 51%|█████     | 114539/225000 [238:00:23<151:42:19,  4.94s/it]
 51%|█████     | 114540/225000 [238:00:26<136:33:20,  4.45s/it]
 51%|█████     | 114541/225000 [238:00:33<159:53:27,  5.21s/it]
 51%|█████     | 114542/225000 [238:00:42<193:13:51,  6.30s/it]
 51%|█████     | 114543/225000 [238:00:46<169:51:45,  5.54s/it]
 51%|█████     | 114544/225000 [238:00:50<159:46:53,  5.21s/it]
 51%|█████     | 114545/225000 [238:00:55<150:25:12,  4.90s/it]
 51%|█████     | 114546/225000 [238:01:00<153:08:14,  4.99s/it]
 51%|█████     | 114547/225000 [238:01:05<156:29:25,  5.10s/it]
 51%|█████     | 114548/225000 [238:01:13<185:37:39,  6.05s/it]
 51%|█████     | 114549/225000 [238:01:17<159:52:27,  5.21s/it]
 51%|█████     | 114550/225000 [238:01:20<138:23:04,  4.51s/it]
                                                               
{'loss': '0.923', 'grad_norm': '1.358', 'learning_rate': '2.61e-05', 'epoch': '0.5091', 'num_input_tokens_seen': 397630352, 'train_runtime': '8.569e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114550/225000 [238:01:20<138:23:04,  4.51s/it]
 51%|█████     | 114551/225000 [238:01:29<182:39:58,  5.95s/it]
 51%|█████     | 114552/225000 [238:01:32<155:46:55,  5.08s/it]
 51%|█████     | 114553/225000 [238:01:35<134:46:32,  4.39s/it]
 51%|█████     | 114554/225000 [238:01:39<134:37:50,  4.39s/it]
 51%|█████     | 114555/225000 [238:01:43<131:12:15,  4.28s/it]
 51%|█████     | 114556/225000 [238:01:49<150:52:46,  4.92s/it]
 51%|█████     | 114557/225000 [238:01:56<164:49:24,  5.37s/it]
 51%|█████     | 114558/225000 [238:01:59<143:55:46,  4.69s/it]
 51%|█████     | 114559/225000 [238:02:03<135:08:49,  4.41s/it]
 51%|█████     | 114560/225000 [238:02:07<130:44:56,  4.26s/it]
 51%|█████     | 114561/225000 [238:02:16<181:40:36,  5.92s/it]
 51%|█████     | 114562/225000 [238:02:20<160:19:14,  5.23s/it]
 51%|█████     | 114563/225000 [238:02:24<148:28:23,  4.84s/it]
 51%|█████     | 114564/225000 [238:02:29<145:40:03,  4.75s/it]
 51%|█████     | 114565/225000 [238:02:38<184:33:11,  6.02s/it]
 51%|█████     | 114566/225000 [238:02:41<159:16:15,  5.19s/it]
 51%|█████     | 114567/225000 [238:02:44<142:13:33,  4.64s/it]
 51%|█████     | 114568/225000 [238:02:51<159:20:23,  5.19s/it]
 51%|█████     | 114569/225000 [238:02:56<165:17:36,  5.39s/it]
 51%|█████     | 114570/225000 [238:03:00<152:05:59,  4.96s/it]
 51%|█████     | 114571/225000 [238:03:04<136:24:15,  4.45s/it]
 51%|█████     | 114572/225000 [238:03:11<164:08:30,  5.35s/it]
 51%|█████     | 114573/225000 [238:03:16<155:09:29,  5.06s/it]
 51%|█████     | 114574/225000 [238:03:22<167:07:09,  5.45s/it]
 51%|█████     | 114575/225000 [238:03:26<151:20:18,  4.93s/it]
 51%|█████     | 114576/225000 [238:03:29<141:31:28,  4.61s/it]
 51%|█████     | 114577/225000 [238:03:33<135:55:05,  4.43s/it]
 51%|█████     | 114578/225000 [238:03:38<139:17:45,  4.54s/it]
 51%|█████     | 114579/225000 [238:03:42<135:04:33,  4.40s/it]
 51%|█████     | 114580/225000 [238:03:46<128:22:13,  4.19s/it]
 51%|█████     | 114581/225000 [238:03:51<134:11:27,  4.38s/it]
 51%|█████     | 114582/225000 [238:03:55<129:12:20,  4.21s/it]
 51%|█████     | 114583/225000 [238:03:59<128:10:45,  4.18s/it]
 51%|█████     | 114584/225000 [238:04:15<238:33:48,  7.78s/it]
 51%|█████     | 114585/225000 [238:04:18<195:54:54,  6.39s/it]
 51%|█████     | 114586/225000 [238:04:29<237:52:35,  7.76s/it]
 51%|█████     | 114587/225000 [238:04:34<213:05:38,  6.95s/it]
 51%|█████     | 114588/225000 [238:04:37<175:38:22,  5.73s/it]
 51%|█████     | 114589/225000 [238:04:40<147:21:41,  4.80s/it]
 51%|█████     | 114590/225000 [238:04:49<184:31:35,  6.02s/it]
 51%|█████     | 114591/225000 [238:04:52<165:02:24,  5.38s/it]
 51%|█████     | 114592/225000 [238:04:56<147:29:55,  4.81s/it]
 51%|█████     | 114593/225000 [238:04:59<132:07:29,  4.31s/it]
 51%|█████     | 114594/225000 [238:05:03<126:48:05,  4.13s/it]
 51%|█████     | 114595/225000 [238:05:15<197:45:44,  6.45s/it]
 51%|█████     | 114596/225000 [238:05:20<184:09:45,  6.01s/it]
 51%|█████     | 114597/225000 [238:05:24<165:44:30,  5.40s/it]
 51%|█████     | 114598/225000 [238:05:27<151:58:28,  4.96s/it]
 51%|█████     | 114599/225000 [238:05:32<145:42:55,  4.75s/it]
 51%|█████     | 114600/225000 [238:05:36<142:00:13,  4.63s/it]
                                                               
{'loss': '0.9224', 'grad_norm': '1.242', 'learning_rate': '2.608e-05', 'epoch': '0.5093', 'num_input_tokens_seen': 397742560, 'train_runtime': '8.571e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114600/225000 [238:05:36<142:00:13,  4.63s/it]
 51%|█████     | 114601/225000 [238:05:40<137:01:48,  4.47s/it]
 51%|█████     | 114602/225000 [238:05:44<132:32:28,  4.32s/it]
 51%|█████     | 114603/225000 [238:05:47<121:58:53,  3.98s/it]
 51%|█████     | 114604/225000 [238:05:54<148:49:29,  4.85s/it]
 51%|█████     | 114605/225000 [238:06:04<191:52:37,  6.26s/it]
 51%|█████     | 114606/225000 [238:06:07<167:57:43,  5.48s/it]
 51%|█████     | 114607/225000 [238:06:10<145:18:38,  4.74s/it]
 51%|█████     | 114608/225000 [238:06:19<176:08:00,  5.74s/it]
 51%|█████     | 114609/225000 [238:06:25<179:23:11,  5.85s/it]
 51%|█████     | 114610/225000 [238:06:28<155:03:05,  5.06s/it]
 51%|█████     | 114611/225000 [238:06:31<134:13:57,  4.38s/it]
 51%|█████     | 114612/225000 [238:06:39<168:00:39,  5.48s/it]
 51%|█████     | 114613/225000 [238:06:48<200:41:29,  6.55s/it]
 51%|█████     | 114614/225000 [238:06:51<168:18:54,  5.49s/it]
 51%|█████     | 114615/225000 [238:06:58<181:10:19,  5.91s/it]
 51%|█████     | 114616/225000 [238:07:00<152:28:55,  4.97s/it]
 51%|█████     | 114617/225000 [238:07:04<136:47:35,  4.46s/it]
 51%|█████     | 114618/225000 [238:07:11<167:14:12,  5.45s/it]
 51%|█████     | 114619/225000 [238:07:14<141:43:32,  4.62s/it]
 51%|█████     | 114620/225000 [238:07:23<178:15:56,  5.81s/it]
 51%|█████     | 114621/225000 [238:07:26<153:34:05,  5.01s/it]
 51%|█████     | 114622/225000 [238:07:30<146:12:05,  4.77s/it]
 51%|█████     | 114623/225000 [238:07:34<142:45:13,  4.66s/it]
 51%|█████     | 114624/225000 [238:07:38<135:43:39,  4.43s/it]
 51%|█████     | 114625/225000 [238:07:49<195:16:12,  6.37s/it]
 51%|█████     | 114626/225000 [238:07:54<177:37:25,  5.79s/it]
 51%|█████     | 114627/225000 [238:07:58<163:17:33,  5.33s/it]
 51%|█████     | 114628/225000 [238:08:02<148:55:53,  4.86s/it]
 51%|█████     | 114629/225000 [238:08:06<140:28:16,  4.58s/it]
 51%|█████     | 114630/225000 [238:08:09<126:49:39,  4.14s/it]
 51%|█████     | 114631/225000 [238:08:13<129:54:34,  4.24s/it]
 51%|█████     | 114632/225000 [238:08:17<127:30:23,  4.16s/it]
 51%|█████     | 114633/225000 [238:08:20<119:15:13,  3.89s/it]
 51%|█████     | 114634/225000 [238:08:25<129:39:11,  4.23s/it]
 51%|█████     | 114635/225000 [238:08:35<181:15:40,  5.91s/it]
 51%|█████     | 114636/225000 [238:08:38<153:28:25,  5.01s/it]
 51%|█████     | 114637/225000 [238:08:47<187:46:33,  6.13s/it]
 51%|█████     | 114638/225000 [238:08:50<163:42:47,  5.34s/it]
 51%|█████     | 114639/225000 [238:08:54<149:07:57,  4.86s/it]
 51%|█████     | 114640/225000 [238:08:58<140:55:42,  4.60s/it]
 51%|█████     | 114641/225000 [238:09:02<133:45:20,  4.36s/it]
 51%|█████     | 114642/225000 [238:09:13<195:17:20,  6.37s/it]
 51%|█████     | 114643/225000 [238:09:16<163:51:20,  5.35s/it]
 51%|█████     | 114644/225000 [238:09:19<143:46:12,  4.69s/it]
 51%|█████     | 114645/225000 [238:09:28<179:26:04,  5.85s/it]
 51%|█████     | 114646/225000 [238:09:33<170:16:54,  5.55s/it]
 51%|█████     | 114647/225000 [238:09:36<151:56:42,  4.96s/it]
 51%|█████     | 114648/225000 [238:09:40<137:54:37,  4.50s/it]
 51%|█████     | 114649/225000 [238:09:46<157:36:36,  5.14s/it]
 51%|█████     | 114650/225000 [238:09:55<190:59:55,  6.23s/it]
                                                               
{'loss': '0.9104', 'grad_norm': '1.386', 'learning_rate': '2.606e-05', 'epoch': '0.5096', 'num_input_tokens_seen': 397854464, 'train_runtime': '8.574e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114650/225000 [238:09:55<190:59:55,  6.23s/it]
 51%|█████     | 114651/225000 [238:09:59<169:25:58,  5.53s/it]
 51%|█████     | 114652/225000 [238:10:03<157:17:53,  5.13s/it]
 51%|█████     | 114653/225000 [238:10:06<135:34:14,  4.42s/it]
 51%|█████     | 114654/225000 [238:10:10<128:26:42,  4.19s/it]
 51%|█████     | 114655/225000 [238:10:14<127:33:40,  4.16s/it]
 51%|█████     | 114656/225000 [238:10:17<120:22:45,  3.93s/it]
 51%|█████     | 114657/225000 [238:10:20<115:51:16,  3.78s/it]
 51%|█████     | 114658/225000 [238:10:28<150:32:10,  4.91s/it]
 51%|█████     | 114659/225000 [238:10:39<207:31:20,  6.77s/it]
 51%|█████     | 114660/225000 [238:10:42<171:16:04,  5.59s/it]
 51%|█████     | 114661/225000 [238:10:52<209:01:58,  6.82s/it]
 51%|█████     | 114662/225000 [238:10:58<208:14:23,  6.79s/it]
 51%|█████     | 114663/225000 [238:11:02<180:41:14,  5.90s/it]
 51%|█████     | 114664/225000 [238:11:08<176:47:08,  5.77s/it]
 51%|█████     | 114665/225000 [238:11:11<155:15:49,  5.07s/it]
 51%|█████     | 114666/225000 [238:11:14<137:47:48,  4.50s/it]
 51%|█████     | 114667/225000 [238:11:23<174:36:29,  5.70s/it]
 51%|█████     | 114668/225000 [238:11:29<182:48:27,  5.96s/it]
 51%|█████     | 114669/225000 [238:11:33<162:43:42,  5.31s/it]
 51%|█████     | 114670/225000 [238:11:41<184:53:47,  6.03s/it]
 51%|█████     | 114671/225000 [238:11:44<158:29:16,  5.17s/it]
 51%|█████     | 114672/225000 [238:11:47<141:05:42,  4.60s/it]
 51%|█████     | 114673/225000 [238:11:55<172:09:18,  5.62s/it]
 51%|█████     | 114674/225000 [238:11:59<153:54:31,  5.02s/it]
 51%|█████     | 114675/225000 [238:12:02<137:18:36,  4.48s/it]
 51%|█████     | 114676/225000 [238:12:06<128:08:34,  4.18s/it]
 51%|█████     | 114677/225000 [238:12:09<118:11:17,  3.86s/it]
 51%|█████     | 114678/225000 [238:12:13<123:00:06,  4.01s/it]
 51%|█████     | 114679/225000 [238:12:18<129:17:44,  4.22s/it]
 51%|█████     | 114680/225000 [238:12:28<184:00:43,  6.00s/it]
 51%|█████     | 114681/225000 [238:12:33<172:24:40,  5.63s/it]
 51%|█████     | 114682/225000 [238:12:35<146:56:15,  4.80s/it]
 51%|█████     | 114683/225000 [238:12:50<238:01:50,  7.77s/it]
 51%|█████     | 114684/225000 [238:12:54<198:21:59,  6.47s/it]
 51%|█████     | 114685/225000 [238:12:57<171:29:17,  5.60s/it]
 51%|█████     | 114686/225000 [238:13:01<156:29:36,  5.11s/it]
 51%|█████     | 114687/225000 [238:13:05<148:44:07,  4.85s/it]
 51%|█████     | 114688/225000 [238:13:10<142:52:07,  4.66s/it]
 51%|█████     | 114689/225000 [238:13:14<137:06:55,  4.47s/it]
 51%|█████     | 114690/225000 [238:13:18<131:27:46,  4.29s/it]
 51%|█████     | 114691/225000 [238:13:21<125:02:34,  4.08s/it]
 51%|█████     | 114692/225000 [238:13:30<169:41:31,  5.54s/it]
 51%|█████     | 114693/225000 [238:13:37<183:43:37,  6.00s/it]
 51%|█████     | 114694/225000 [238:13:40<157:42:40,  5.15s/it]
 51%|█████     | 114695/225000 [238:13:49<193:08:35,  6.30s/it]
 51%|█████     | 114696/225000 [238:13:59<224:24:05,  7.32s/it]
 51%|█████     | 114697/225000 [238:14:02<185:12:00,  6.04s/it]
 51%|█████     | 114698/225000 [238:14:06<163:04:14,  5.32s/it]
 51%|█████     | 114699/225000 [238:14:12<174:34:11,  5.70s/it]
 51%|█████     | 114700/225000 [238:14:15<151:16:49,  4.94s/it]
                                                               
{'loss': '0.9144', 'grad_norm': '1.163', 'learning_rate': '2.604e-05', 'epoch': '0.5098', 'num_input_tokens_seen': 397966760, 'train_runtime': '8.577e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114700/225000 [238:14:15<151:16:49,  4.94s/it]
 51%|█████     | 114701/225000 [238:14:19<137:31:45,  4.49s/it]
 51%|█████     | 114702/225000 [238:14:27<173:13:30,  5.65s/it]
 51%|█████     | 114703/225000 [238:14:31<154:54:30,  5.06s/it]
 51%|█████     | 114704/225000 [238:14:33<131:52:44,  4.30s/it]
 51%|█████     | 114705/225000 [238:14:37<126:35:21,  4.13s/it]
 51%|█████     | 114706/225000 [238:14:40<117:32:05,  3.84s/it]
 51%|█████     | 114707/225000 [238:14:50<169:03:29,  5.52s/it]
 51%|█████     | 114708/225000 [238:14:59<200:42:28,  6.55s/it]
 51%|█████     | 114709/225000 [238:15:02<169:22:01,  5.53s/it]
 51%|█████     | 114710/225000 [238:15:12<215:38:27,  7.04s/it]
 51%|█████     | 114711/225000 [238:15:16<179:02:52,  5.84s/it]
 51%|█████     | 114712/225000 [238:15:19<160:47:08,  5.25s/it]
 51%|█████     | 114713/225000 [238:15:23<145:08:30,  4.74s/it]
 51%|█████     | 114714/225000 [238:15:28<152:16:15,  4.97s/it]
 51%|█████     | 114715/225000 [238:15:40<212:57:59,  6.95s/it]
 51%|█████     | 114716/225000 [238:15:42<171:30:43,  5.60s/it]
 51%|█████     | 114717/225000 [238:15:49<182:47:29,  5.97s/it]
 51%|█████     | 114718/225000 [238:15:53<162:43:02,  5.31s/it]
 51%|█████     | 114719/225000 [238:15:57<146:54:16,  4.80s/it]
 51%|█████     | 114720/225000 [238:15:59<127:29:48,  4.16s/it]
 51%|█████     | 114721/225000 [238:16:04<129:14:54,  4.22s/it]
 51%|█████     | 114722/225000 [238:16:08<126:38:40,  4.13s/it]
 51%|█████     | 114723/225000 [238:16:11<120:52:40,  3.95s/it]
 51%|█████     | 114724/225000 [238:16:21<173:43:45,  5.67s/it]
 51%|█████     | 114725/225000 [238:16:27<177:46:16,  5.80s/it]
 51%|█████     | 114726/225000 [238:16:30<154:36:28,  5.05s/it]
 51%|█████     | 114727/225000 [238:16:34<143:27:09,  4.68s/it]
 51%|█████     | 114728/225000 [238:16:38<139:57:00,  4.57s/it]
 51%|█████     | 114729/225000 [238:16:42<134:52:12,  4.40s/it]
 51%|█████     | 114730/225000 [238:16:47<134:58:36,  4.41s/it]
 51%|█████     | 114731/225000 [238:16:51<130:29:04,  4.26s/it]
 51%|█████     | 114732/225000 [238:16:56<141:06:42,  4.61s/it]
 51%|█████     | 114733/225000 [238:17:06<190:21:55,  6.22s/it]
 51%|█████     | 114734/225000 [238:17:09<161:18:57,  5.27s/it]
 51%|█████     | 114735/225000 [238:17:15<165:37:03,  5.41s/it]
 51%|█████     | 114736/225000 [238:17:26<218:19:20,  7.13s/it]
 51%|█████     | 114737/225000 [238:17:29<181:54:35,  5.94s/it]
 51%|█████     | 114738/225000 [238:17:36<188:46:01,  6.16s/it]
 51%|█████     | 114739/225000 [238:17:39<163:02:16,  5.32s/it]
 51%|█████     | 114740/225000 [238:17:44<160:56:48,  5.25s/it]
 51%|█████     | 114741/225000 [238:17:51<170:17:41,  5.56s/it]
 51%|█████     | 114742/225000 [238:17:54<147:57:53,  4.83s/it]
 51%|█████     | 114743/225000 [238:18:00<157:13:18,  5.13s/it]
 51%|█████     | 114744/225000 [238:18:04<147:17:22,  4.81s/it]
 51%|█████     | 114745/225000 [238:18:07<134:10:07,  4.38s/it]
 51%|█████     | 114746/225000 [238:18:18<193:33:05,  6.32s/it]
 51%|█████     | 114747/225000 [238:18:21<169:02:13,  5.52s/it]
 51%|█████     | 114748/225000 [238:18:27<168:43:12,  5.51s/it]
 51%|█████     | 114749/225000 [238:18:32<167:18:09,  5.46s/it]
 51%|█████     | 114750/225000 [238:18:36<147:08:51,  4.80s/it]
                                                               
{'loss': '0.9032', 'grad_norm': '1.492', 'learning_rate': '2.602e-05', 'epoch': '0.51', 'num_input_tokens_seen': 398077976, 'train_runtime': '8.579e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114750/225000 [238:18:36<147:08:51,  4.80s/it]
 51%|█████     | 114751/225000 [238:18:42<164:27:30,  5.37s/it]
 51%|█████     | 114752/225000 [238:18:49<173:27:17,  5.66s/it]
 51%|█████     | 114753/225000 [238:18:52<148:05:27,  4.84s/it]
 51%|█████     | 114754/225000 [238:18:55<135:51:17,  4.44s/it]
 51%|█████     | 114755/225000 [238:18:59<128:51:57,  4.21s/it]
 51%|█████     | 114756/225000 [238:19:10<193:24:29,  6.32s/it]
 51%|█████     | 114757/225000 [238:19:13<164:00:07,  5.36s/it]
 51%|█████     | 114758/225000 [238:19:17<147:56:31,  4.83s/it]
 51%|█████     | 114759/225000 [238:19:20<131:39:42,  4.30s/it]
 51%|█████     | 114760/225000 [238:19:29<179:38:01,  5.87s/it]
 51%|█████     | 114761/225000 [238:19:32<155:00:49,  5.06s/it]
 51%|█████     | 114762/225000 [238:19:36<136:51:26,  4.47s/it]
 51%|█████     | 114763/225000 [238:19:39<127:53:39,  4.18s/it]
 51%|█████     | 114764/225000 [238:19:50<190:39:29,  6.23s/it]
 51%|█████     | 114765/225000 [238:19:53<159:06:14,  5.20s/it]
 51%|█████     | 114766/225000 [238:19:59<169:25:01,  5.53s/it]
 51%|█████     | 114767/225000 [238:20:05<171:33:57,  5.60s/it]
 51%|█████     | 114768/225000 [238:20:08<150:13:44,  4.91s/it]
 51%|█████     | 114769/225000 [238:20:12<144:26:04,  4.72s/it]
 51%|█████     | 114770/225000 [238:20:21<175:49:41,  5.74s/it]
 51%|█████     | 114771/225000 [238:20:24<156:31:32,  5.11s/it]
 51%|█████     | 114772/225000 [238:20:28<143:01:03,  4.67s/it]
 51%|█████     | 114773/225000 [238:20:31<129:12:11,  4.22s/it]
 51%|█████     | 114774/225000 [238:20:35<123:29:55,  4.03s/it]
 51%|█████     | 114775/225000 [238:20:44<169:28:07,  5.53s/it]
 51%|█████     | 114776/225000 [238:20:49<164:42:08,  5.38s/it]
 51%|█████     | 114777/225000 [238:20:56<183:04:17,  5.98s/it]
 51%|█████     | 114778/225000 [238:20:59<157:10:22,  5.13s/it]
 51%|█████     | 114779/225000 [238:21:03<145:01:15,  4.74s/it]
 51%|█████     | 114780/225000 [238:21:08<147:54:10,  4.83s/it]
 51%|█████     | 114781/225000 [238:21:16<175:35:03,  5.73s/it]
 51%|█████     | 114782/225000 [238:21:20<157:58:14,  5.16s/it]
 51%|█████     | 114783/225000 [238:21:24<146:55:40,  4.80s/it]
 51%|█████     | 114784/225000 [238:21:28<138:51:01,  4.54s/it]
 51%|█████     | 114785/225000 [238:21:30<122:57:44,  4.02s/it]
 51%|█████     | 114786/225000 [238:21:33<113:13:28,  3.70s/it]
 51%|█████     | 114787/225000 [238:21:39<132:10:04,  4.32s/it]
 51%|█████     | 114788/225000 [238:21:48<172:23:26,  5.63s/it]
 51%|█████     | 114789/225000 [238:21:51<145:31:33,  4.75s/it]
 51%|█████     | 114790/225000 [238:21:56<154:44:43,  5.05s/it]
 51%|█████     | 114791/225000 [238:21:59<136:10:02,  4.45s/it]
 51%|█████     | 114792/225000 [238:22:06<158:22:49,  5.17s/it]
 51%|█████     | 114793/225000 [238:22:09<137:19:26,  4.49s/it]
 51%|█████     | 114794/225000 [238:22:13<132:20:52,  4.32s/it]
 51%|█████     | 114795/225000 [238:22:21<168:25:06,  5.50s/it]
 51%|█████     | 114796/225000 [238:22:25<154:01:10,  5.03s/it]
 51%|█████     | 114797/225000 [238:22:35<200:30:11,  6.55s/it]
 51%|█████     | 114798/225000 [238:22:39<177:20:22,  5.79s/it]
 51%|█████     | 114799/225000 [238:22:46<184:53:09,  6.04s/it]
 51%|█████     | 114800/225000 [238:22:49<160:04:16,  5.23s/it]
                                                               
{'loss': '0.9299', 'grad_norm': '1.376', 'learning_rate': '2.601e-05', 'epoch': '0.5102', 'num_input_tokens_seen': 398185320, 'train_runtime': '8.582e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114800/225000 [238:22:49<160:04:16,  5.23s/it]
 51%|█████     | 114801/225000 [238:22:53<144:42:57,  4.73s/it]
 51%|█████     | 114802/225000 [238:22:57<134:37:14,  4.40s/it]
 51%|█████     | 114803/225000 [238:23:01<139:39:48,  4.56s/it]
 51%|█████     | 114804/225000 [238:23:06<135:10:18,  4.42s/it]
 51%|█████     | 114805/225000 [238:23:10<133:45:28,  4.37s/it]
 51%|█████     | 114806/225000 [238:23:14<129:36:28,  4.23s/it]
 51%|█████     | 114807/225000 [238:23:18<127:28:15,  4.16s/it]
 51%|█████     | 114808/225000 [238:23:23<135:21:53,  4.42s/it]
 51%|█████     | 114809/225000 [238:23:38<232:24:48,  7.59s/it]
 51%|█████     | 114810/225000 [238:23:42<203:55:20,  6.66s/it]
 51%|█████     | 114811/225000 [238:23:45<167:43:34,  5.48s/it]
 51%|█████     | 114812/225000 [238:23:54<201:26:12,  6.58s/it]
 51%|█████     | 114813/225000 [238:24:03<221:24:22,  7.23s/it]
 51%|█████     | 114814/225000 [238:24:07<195:22:55,  6.38s/it]
 51%|█████     | 114815/225000 [238:24:11<172:56:37,  5.65s/it]
 51%|█████     | 114816/225000 [238:24:15<155:48:43,  5.09s/it]
 51%|█████     | 114817/225000 [238:24:18<141:15:37,  4.62s/it]
 51%|█████     | 114818/225000 [238:24:22<133:05:55,  4.35s/it]
 51%|█████     | 114819/225000 [238:24:33<191:12:45,  6.25s/it]
 51%|█████     | 114820/225000 [238:24:36<162:00:02,  5.29s/it]
 51%|█████     | 114821/225000 [238:24:40<148:40:35,  4.86s/it]
 51%|█████     | 114822/225000 [238:24:48<178:30:12,  5.83s/it]
 51%|█████     | 114823/225000 [238:24:51<154:45:04,  5.06s/it]
 51%|█████     | 114824/225000 [238:24:54<138:20:12,  4.52s/it]
 51%|█████     | 114825/225000 [238:24:58<127:13:42,  4.16s/it]
 51%|█████     | 114826/225000 [238:25:08<182:55:41,  5.98s/it]
 51%|█████     | 114827/225000 [238:25:11<155:14:11,  5.07s/it]
 51%|█████     | 114828/225000 [238:25:15<145:16:32,  4.75s/it]
 51%|█████     | 114829/225000 [238:25:18<131:05:41,  4.28s/it]
 51%|█████     | 114830/225000 [238:25:22<127:37:03,  4.17s/it]
 51%|█████     | 114831/225000 [238:25:34<201:49:00,  6.59s/it]
 51%|█████     | 114832/225000 [238:25:38<175:13:59,  5.73s/it]
 51%|█████     | 114833/225000 [238:25:42<161:24:58,  5.27s/it]
 51%|█████     | 114834/225000 [238:25:46<145:03:49,  4.74s/it]
 51%|█████     | 114835/225000 [238:25:59<222:44:43,  7.28s/it]
 51%|█████     | 114836/225000 [238:26:03<192:31:48,  6.29s/it]
 51%|█████     | 114837/225000 [238:26:06<166:25:22,  5.44s/it]
 51%|█████     | 114838/225000 [238:26:10<153:55:50,  5.03s/it]
 51%|█████     | 114839/225000 [238:26:14<144:07:57,  4.71s/it]
 51%|█████     | 114840/225000 [238:26:18<135:27:15,  4.43s/it]
 51%|█████     | 114841/225000 [238:26:22<132:40:47,  4.34s/it]
 51%|█████     | 114842/225000 [238:26:26<128:59:07,  4.22s/it]
 51%|█████     | 114843/225000 [238:26:32<143:41:52,  4.70s/it]
 51%|█████     | 114844/225000 [238:26:41<187:19:05,  6.12s/it]
 51%|█████     | 114845/225000 [238:26:46<173:07:09,  5.66s/it]
 51%|█████     | 114846/225000 [238:26:52<174:05:32,  5.69s/it]
 51%|█████     | 114847/225000 [238:26:57<169:31:45,  5.54s/it]
 51%|█████     | 114848/225000 [238:27:01<156:09:32,  5.10s/it]
 51%|█████     | 114849/225000 [238:27:13<223:33:38,  7.31s/it]
 51%|█████     | 114850/225000 [238:27:18<194:54:59,  6.37s/it]
                                                               
{'loss': '0.9173', 'grad_norm': '1.456', 'learning_rate': '2.599e-05', 'epoch': '0.5104', 'num_input_tokens_seen': 398300696, 'train_runtime': '8.584e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114850/225000 [238:27:18<194:54:59,  6.37s/it]
 51%|█████     | 114851/225000 [238:27:22<171:35:17,  5.61s/it]
 51%|█████     | 114852/225000 [238:27:26<158:46:34,  5.19s/it]
 51%|█████     | 114853/225000 [238:27:29<140:32:59,  4.59s/it]
 51%|█████     | 114854/225000 [238:27:40<199:00:36,  6.50s/it]
 51%|█████     | 114855/225000 [238:27:43<166:57:58,  5.46s/it]
 51%|█████     | 114856/225000 [238:27:46<146:46:00,  4.80s/it]
 51%|█████     | 114857/225000 [238:27:49<130:17:44,  4.26s/it]
 51%|█████     | 114858/225000 [238:27:53<124:47:24,  4.08s/it]
 51%|█████     | 114859/225000 [238:28:07<219:45:15,  7.18s/it]
 51%|█████     | 114860/225000 [238:28:10<183:20:18,  5.99s/it]
 51%|█████     | 114861/225000 [238:28:14<159:40:37,  5.22s/it]
 51%|█████     | 114862/225000 [238:28:20<172:07:56,  5.63s/it]
 51%|█████     | 114863/225000 [238:28:25<163:59:00,  5.36s/it]
 51%|█████     | 114864/225000 [238:28:29<149:18:02,  4.88s/it]
 51%|█████     | 114865/225000 [238:28:34<149:38:03,  4.89s/it]
 51%|█████     | 114866/225000 [238:28:41<172:29:34,  5.64s/it]
 51%|█████     | 114867/225000 [238:28:46<163:37:45,  5.35s/it]
 51%|█████     | 114868/225000 [238:28:52<172:10:19,  5.63s/it]
 51%|█████     | 114869/225000 [238:28:56<158:23:36,  5.18s/it]
 51%|█████     | 114870/225000 [238:29:00<141:58:30,  4.64s/it]
 51%|█████     | 114871/225000 [238:29:04<139:34:01,  4.56s/it]
 51%|█████     | 114872/225000 [238:29:08<133:40:22,  4.37s/it]
 51%|█████     | 114873/225000 [238:29:18<186:05:03,  6.08s/it]
 51%|█████     | 114874/225000 [238:29:21<158:44:43,  5.19s/it]
 51%|█████     | 114875/225000 [238:29:25<150:12:50,  4.91s/it]
 51%|█████     | 114876/225000 [238:29:29<140:33:02,  4.59s/it]
 51%|█████     | 114877/225000 [238:29:33<132:24:54,  4.33s/it]
 51%|█████     | 114878/225000 [238:29:37<125:13:34,  4.09s/it]
 51%|█████     | 114879/225000 [238:29:40<122:45:12,  4.01s/it]
 51%|█████     | 114880/225000 [238:29:44<122:34:11,  4.01s/it]
 51%|█████     | 114881/225000 [238:29:49<125:11:53,  4.09s/it]
 51%|█████     | 114882/225000 [238:29:53<130:35:55,  4.27s/it]
 51%|█████     | 114883/225000 [238:30:03<184:05:40,  6.02s/it]
 51%|█████     | 114884/225000 [238:30:07<159:03:34,  5.20s/it]
 51%|█████     | 114885/225000 [238:30:10<142:36:09,  4.66s/it]
 51%|█████     | 114886/225000 [238:30:13<129:25:31,  4.23s/it]
 51%|█████     | 114887/225000 [238:30:23<176:37:20,  5.77s/it]
 51%|█████     | 114888/225000 [238:30:26<153:13:52,  5.01s/it]
 51%|█████     | 114889/225000 [238:30:29<139:06:58,  4.55s/it]
 51%|█████     | 114890/225000 [238:30:33<127:06:42,  4.16s/it]
 51%|█████     | 114891/225000 [238:30:37<124:54:13,  4.08s/it]
 51%|█████     | 114892/225000 [238:30:43<144:51:56,  4.74s/it]
 51%|█████     | 114893/225000 [238:30:51<179:28:33,  5.87s/it]
 51%|█████     | 114894/225000 [238:30:55<154:40:20,  5.06s/it]
 51%|█████     | 114895/225000 [238:30:58<138:10:35,  4.52s/it]
 51%|█████     | 114896/225000 [238:31:03<139:57:40,  4.58s/it]
 51%|█████     | 114897/225000 [238:31:09<157:47:10,  5.16s/it]
 51%|█████     | 114898/225000 [238:31:15<166:43:25,  5.45s/it]
 51%|█████     | 114899/225000 [238:31:22<177:17:23,  5.80s/it]
 51%|█████     | 114900/225000 [238:31:26<159:13:59,  5.21s/it]
                                                               
{'loss': '0.9134', 'grad_norm': '1.815', 'learning_rate': '2.597e-05', 'epoch': '0.5107', 'num_input_tokens_seen': 398410352, 'train_runtime': '8.587e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114900/225000 [238:31:26<159:13:59,  5.21s/it]
 51%|█████     | 114901/225000 [238:31:30<148:36:05,  4.86s/it]
 51%|█████     | 114902/225000 [238:31:37<169:44:47,  5.55s/it]
 51%|█████     | 114903/225000 [238:31:49<228:10:09,  7.46s/it]
 51%|█████     | 114904/225000 [238:31:56<226:50:02,  7.42s/it]
 51%|█████     | 114905/225000 [238:31:59<190:21:07,  6.22s/it]
 51%|█████     | 114906/225000 [238:32:03<165:02:47,  5.40s/it]
 51%|█████     | 114907/225000 [238:32:06<142:54:47,  4.67s/it]
 51%|█████     | 114908/225000 [238:32:14<177:28:30,  5.80s/it]
 51%|█████     | 114909/225000 [238:32:20<176:44:03,  5.78s/it]
 51%|█████     | 114910/225000 [238:32:23<149:58:00,  4.90s/it]
 51%|█████     | 114911/225000 [238:32:27<138:14:11,  4.52s/it]
 51%|█████     | 114912/225000 [238:32:31<136:55:55,  4.48s/it]
 51%|█████     | 114913/225000 [238:32:35<137:00:38,  4.48s/it]
 51%|█████     | 114914/225000 [238:32:40<136:07:23,  4.45s/it]
 51%|█████     | 114915/225000 [238:32:44<132:51:52,  4.34s/it]
 51%|█████     | 114916/225000 [238:32:48<127:15:33,  4.16s/it]
 51%|█████     | 114917/225000 [238:32:56<163:57:53,  5.36s/it]
 51%|█████     | 114918/225000 [238:33:03<179:14:44,  5.86s/it]
 51%|█████     | 114919/225000 [238:33:06<158:30:19,  5.18s/it]
 51%|█████     | 114920/225000 [238:33:10<141:12:58,  4.62s/it]
 51%|█████     | 114921/225000 [238:33:16<156:41:58,  5.12s/it]
 51%|█████     | 114922/225000 [238:33:24<184:13:41,  6.03s/it]
 51%|█████     | 114923/225000 [238:33:31<186:53:54,  6.11s/it]
 51%|█████     | 114924/225000 [238:33:33<156:55:50,  5.13s/it]
 51%|█████     | 114925/225000 [238:33:40<171:54:39,  5.62s/it]
 51%|█████     | 114926/225000 [238:33:43<148:49:28,  4.87s/it]
 51%|█████     | 114927/225000 [238:33:54<203:34:29,  6.66s/it]
 51%|█████     | 114928/225000 [238:33:58<177:30:11,  5.81s/it]
 51%|█████     | 114929/225000 [238:34:05<193:23:10,  6.32s/it]
 51%|█████     | 114930/225000 [238:34:12<194:42:48,  6.37s/it]
 51%|█████     | 114931/225000 [238:34:15<167:42:59,  5.49s/it]
 51%|█████     | 114932/225000 [238:34:19<149:21:13,  4.88s/it]
 51%|█████     | 114933/225000 [238:34:22<137:11:07,  4.49s/it]
 51%|█████     | 114934/225000 [238:34:26<130:00:45,  4.25s/it]
 51%|█████     | 114935/225000 [238:34:31<135:41:47,  4.44s/it]
 51%|█████     | 114936/225000 [238:34:36<143:41:22,  4.70s/it]
 51%|█████     | 114937/225000 [238:34:40<134:27:34,  4.40s/it]
 51%|█████     | 114938/225000 [238:34:44<130:13:44,  4.26s/it]
 51%|█████     | 114939/225000 [238:34:53<177:25:00,  5.80s/it]
 51%|█████     | 114940/225000 [238:34:57<156:48:52,  5.13s/it]
 51%|█████     | 114941/225000 [238:35:01<143:48:10,  4.70s/it]
 51%|█████     | 114942/225000 [238:35:04<133:16:06,  4.36s/it]
 51%|█████     | 114943/225000 [238:35:12<163:01:45,  5.33s/it]
 51%|█████     | 114944/225000 [238:35:18<175:45:13,  5.75s/it]
 51%|█████     | 114945/225000 [238:35:23<160:43:34,  5.26s/it]
 51%|█████     | 114946/225000 [238:35:26<145:31:46,  4.76s/it]
 51%|█████     | 114947/225000 [238:35:31<147:16:35,  4.82s/it]
 51%|█████     | 114948/225000 [238:35:35<138:08:27,  4.52s/it]
 51%|█████     | 114949/225000 [238:35:39<136:01:19,  4.45s/it]
 51%|█████     | 114950/225000 [238:35:44<136:04:47,  4.45s/it]
                                                               
{'loss': '0.9087', 'grad_norm': '1.15', 'learning_rate': '2.595e-05', 'epoch': '0.5109', 'num_input_tokens_seen': 398522848, 'train_runtime': '8.589e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 114950/225000 [238:35:44<136:04:47,  4.45s/it]
 51%|█████     | 114951/225000 [238:35:47<129:56:51,  4.25s/it]
 51%|█████     | 114952/225000 [238:35:51<125:43:15,  4.11s/it]
 51%|█████     | 114953/225000 [238:35:56<129:45:07,  4.24s/it]
 51%|█████     | 114954/225000 [238:36:00<130:50:07,  4.28s/it]
 51%|█████     | 114955/225000 [238:36:05<136:18:36,  4.46s/it]
 51%|█████     | 114956/225000 [238:36:09<129:39:15,  4.24s/it]
 51%|█████     | 114957/225000 [238:36:14<138:00:31,  4.51s/it]
 51%|█████     | 114958/225000 [238:36:25<197:07:11,  6.45s/it]
 51%|█████     | 114959/225000 [238:36:29<179:51:45,  5.88s/it]
 51%|█████     | 114960/225000 [238:36:33<161:54:44,  5.30s/it]
 51%|█████     | 114961/225000 [238:36:38<152:26:39,  4.99s/it]
 51%|█████     | 114962/225000 [238:36:41<137:34:57,  4.50s/it]
 51%|█████     | 114963/225000 [238:36:45<133:44:19,  4.38s/it]
 51%|█████     | 114964/225000 [238:36:49<126:59:07,  4.15s/it]
 51%|█████     | 114965/225000 [238:37:02<210:30:28,  6.89s/it]
 51%|█████     | 114966/225000 [238:37:08<200:20:55,  6.55s/it]
 51%|█████     | 114967/225000 [238:37:11<172:55:32,  5.66s/it]
 51%|█████     | 114968/225000 [238:37:15<156:18:22,  5.11s/it]
 51%|█████     | 114969/225000 [238:37:19<142:02:55,  4.65s/it]
 51%|█████     | 114970/225000 [238:37:23<135:06:54,  4.42s/it]
 51%|█████     | 114971/225000 [238:37:26<124:40:32,  4.08s/it]
 51%|█████     | 114972/225000 [238:37:33<149:00:12,  4.88s/it]
 51%|█████     | 114973/225000 [238:37:41<183:24:46,  6.00s/it]
 51%|█████     | 114974/225000 [238:37:45<158:23:13,  5.18s/it]
 51%|█████     | 114975/225000 [238:37:52<181:54:30,  5.95s/it]
 51%|█████     | 114976/225000 [238:37:56<158:08:20,  5.17s/it]
 51%|█████     | 114977/225000 [238:38:02<172:47:16,  5.65s/it]
 51%|█████     | 114978/225000 [238:38:08<172:31:10,  5.64s/it]
 51%|█████     | 114979/225000 [238:38:11<147:18:14,  4.82s/it]
 51%|█████     | 114980/225000 [238:38:15<142:17:34,  4.66s/it]
 51%|█████     | 114981/225000 [238:38:20<144:33:50,  4.73s/it]
 51%|█████     | 114982/225000 [238:38:24<137:51:51,  4.51s/it]
 51%|█████     | 114983/225000 [238:38:31<156:29:19,  5.12s/it]
 51%|█████     | 114984/225000 [238:38:34<143:55:30,  4.71s/it]
 51%|█████     | 114985/225000 [238:38:38<133:21:01,  4.36s/it]
 51%|█████     | 114986/225000 [238:38:42<126:04:51,  4.13s/it]
 51%|█████     | 114987/225000 [238:38:45<123:44:54,  4.05s/it]
 51%|█████     | 114988/225000 [238:38:50<130:22:02,  4.27s/it]
 51%|█████     | 114989/225000 [238:39:02<197:27:44,  6.46s/it]
 51%|█████     | 114990/225000 [238:39:05<163:39:59,  5.36s/it]
 51%|█████     | 114991/225000 [238:39:08<143:11:58,  4.69s/it]
 51%|█████     | 114992/225000 [238:39:15<164:49:10,  5.39s/it]
 51%|█████     | 114993/225000 [238:39:18<143:27:49,  4.69s/it]
 51%|█████     | 114994/225000 [238:39:22<135:21:14,  4.43s/it]
 51%|█████     | 114995/225000 [238:39:24<121:15:16,  3.97s/it]
 51%|█████     | 114996/225000 [238:39:33<166:55:42,  5.46s/it]
 51%|█████     | 114997/225000 [238:39:38<160:35:04,  5.26s/it]
 51%|█████     | 114998/225000 [238:39:42<144:12:27,  4.72s/it]
 51%|█████     | 114999/225000 [238:39:46<138:08:41,  4.52s/it]
 51%|█████     | 115000/225000 [238:39:50<134:23:50,  4.40s/it]
                                                               
{'loss': '0.9155', 'grad_norm': '1.323', 'learning_rate': '2.593e-05', 'epoch': '0.5111', 'num_input_tokens_seen': 398630216, 'train_runtime': '8.592e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 115000/225000 [238:39:50<134:23:50,  4.40s/it]
 51%|█████     | 115001/225000 [238:39:53<125:56:29,  4.12s/it]
 51%|█████     | 115002/225000 [238:39:57<122:13:35,  4.00s/it]
 51%|█████     | 115003/225000 [238:40:01<125:28:33,  4.11s/it]
 51%|█████     | 115004/225000 [238:40:11<176:08:41,  5.76s/it]
 51%|█████     | 115005/225000 [238:40:17<179:11:58,  5.86s/it]
 51%|█████     | 115006/225000 [238:40:22<166:14:58,  5.44s/it]
 51%|█████     | 115007/225000 [238:40:27<165:34:59,  5.42s/it]
 51%|█████     | 115008/225000 [238:40:30<139:49:41,  4.58s/it]
 51%|█████     | 115009/225000 [238:40:36<156:03:27,  5.11s/it]
 51%|█████     | 115010/225000 [238:40:40<151:04:31,  4.94s/it]
 51%|█████     | 115011/225000 [238:40:45<147:53:19,  4.84s/it]
 51%|█████     | 115012/225000 [238:40:51<155:51:41,  5.10s/it]
 51%|█████     | 115013/225000 [238:40:57<163:40:39,  5.36s/it]
 51%|█████     | 115014/225000 [238:41:01<149:39:25,  4.90s/it]
 51%|█████     | 115015/225000 [238:41:04<137:57:03,  4.52s/it]
 51%|█████     | 115016/225000 [238:41:15<199:41:49,  6.54s/it]
 51%|█████     | 115017/225000 [238:41:19<171:36:10,  5.62s/it]
 51%|█████     | 115018/225000 [238:41:23<156:00:11,  5.11s/it]
 51%|█████     | 115019/225000 [238:41:27<146:04:25,  4.78s/it]
 51%|█████     | 115020/225000 [238:41:31<138:28:48,  4.53s/it]
 51%|█████     | 115021/225000 [238:41:34<130:10:45,  4.26s/it]
 51%|█████     | 115022/225000 [238:41:41<147:40:44,  4.83s/it]
 51%|█████     | 115023/225000 [238:41:50<189:42:55,  6.21s/it]
 51%|█████     | 115024/225000 [238:41:53<162:03:35,  5.30s/it]
 51%|█████     | 115025/225000 [238:41:57<149:59:30,  4.91s/it]
 51%|█████     | 115026/225000 [238:42:05<173:29:51,  5.68s/it]
 51%|█████     | 115027/225000 [238:42:09<157:45:04,  5.16s/it]
 51%|█████     | 115028/225000 [238:42:13<147:01:07,  4.81s/it]
 51%|█████     | 115029/225000 [238:42:16<136:37:34,  4.47s/it]
 51%|█████     | 115030/225000 [238:42:21<140:03:42,  4.59s/it]
 51%|█████     | 115031/225000 [238:42:25<132:22:06,  4.33s/it]
 51%|█████     | 115032/225000 [238:42:28<122:03:06,  4.00s/it]
 51%|█████     | 115033/225000 [238:42:32<122:47:34,  4.02s/it]
 51%|█████     | 115034/225000 [238:42:37<127:24:23,  4.17s/it]
 51%|█████     | 115035/225000 [238:42:47<184:08:29,  6.03s/it]
 51%|█████     | 115036/225000 [238:42:55<198:04:12,  6.48s/it]
 51%|█████     | 115037/225000 [238:42:58<171:22:27,  5.61s/it]
 51%|█████     | 115038/225000 [238:43:03<160:36:51,  5.26s/it]
 51%|█████     | 115039/225000 [238:43:11<190:22:51,  6.23s/it]
 51%|█████     | 115040/225000 [238:43:14<163:57:27,  5.37s/it]
 51%|█████     | 115041/225000 [238:43:19<156:31:04,  5.12s/it]
 51%|█████     | 115042/225000 [238:43:24<158:46:05,  5.20s/it]
 51%|█████     | 115043/225000 [238:43:28<140:52:01,  4.61s/it]
 51%|█████     | 115044/225000 [238:43:31<130:47:46,  4.28s/it]
 51%|█████     | 115045/225000 [238:43:35<126:47:44,  4.15s/it]
 51%|█████     | 115046/225000 [238:43:40<136:06:14,  4.46s/it]
 51%|█████     | 115047/225000 [238:43:51<194:38:41,  6.37s/it]
 51%|█████     | 115048/225000 [238:43:54<166:02:26,  5.44s/it]
 51%|█████     | 115049/225000 [238:43:58<149:10:51,  4.88s/it]
 51%|█████     | 115050/225000 [238:44:02<138:19:09,  4.53s/it]
                                                               
{'loss': '0.9106', 'grad_norm': '1.322', 'learning_rate': '2.591e-05', 'epoch': '0.5113', 'num_input_tokens_seen': 398740328, 'train_runtime': '8.594e+05', 'train_tokens_per_second': '464'}

 51%|█████     | 115050/225000 [238:44:02<138:19:09,  4.53s/it]
 51%|█████     | 115051/225000 [238:44:10<178:08:19,  5.83s/it]
 51%|█████     | 115052/225000 [238:44:13<151:51:49,  4.97s/it]
 51%|█████     | 115053/225000 [238:44:17<142:24:47,  4.66s/it]
 51%|█████     | 115054/225000 [238:44:26<178:12:42,  5.84s/it]
 51%|█████     | 115055/225000 [238:44:29<151:15:40,  4.95s/it]
 51%|█████     | 115056/225000 [238:44:32<138:39:39,  4.54s/it]
 51%|█████     | 115057/225000 [238:44:36<126:21:50,  4.14s/it]
 51%|█████     | 115058/225000 [238:44:44<169:33:36,  5.55s/it]
 51%|█████     | 115059/225000 [238:44:49<156:20:49,  5.12s/it]
 51%|█████     | 115060/225000 [238:44:52<137:24:13,  4.50s/it]
 51%|█████     | 115061/225000 [238:44:55<131:21:47,  4.30s/it]
 51%|█████     | 115062/225000 [238:45:03<156:49:08,  5.14s/it]
 51%|█████     | 115063/225000 [238:45:10<182:23:52,  5.97s/it]
 51%|█████     | 115064/225000 [238:45:13<152:39:05,  5.00s/it]
 51%|█████     | 115065/225000 [238:45:21<180:27:14,  5.91s/it]
 51%|█████     | 115066/225000 [238:45:25<161:41:28,  5.29s/it]
 51%|█████     | 115067/225000 [238:45:28<144:09:54,  4.72s/it]
 51%|█████     | 115068/225000 [238:45:35<157:22:58,  5.15s/it]
 51%|█████     | 115069/225000 [238:45:41<168:45:08,  5.53s/it]
 51%|█████     | 115070/225000 [238:45:51<212:53:08,  6.97s/it]
 51%|█████     | 115071/225000 [238:45:57<197:15:10,  6.46s/it]
 51%|█████     | 115072/225000 [238:45:59<162:46:37,  5.33s/it]
 51%|█████     | 115073/225000 [238:46:09<203:02:44,  6.65s/it]
 51%|█████     | 115074/225000 [238:46:15<195:34:06,  6.40s/it]
 51%|█████     | 115075/225000 [238:46:18<164:55:33,  5.40s/it]
 51%|█████     | 115076/225000 [238:46:22<151:29:34,  4.96s/it]
 51%|█████     | 115077/225000 [238:46:26<140:42:06,  4.61s/it]
 51%|█████     | 115078/225000 [238:46:29<132:45:28,  4.35s/it]
 51%|█████     | 115079/225000 [238:46:33<129:17:06,  4.23s/it]
 51%|█████     | 115080/225000 [238:46:44<190:53:52,  6.25s/it]
 51%|█████     | 115081/225000 [238:46:48<169:50:35,  5.56s/it]
 51%|█████     | 115082/225000 [238:46:53<158:04:13,  5.18s/it]
 51%|█████     | 115083/225000 [238:47:03<202:42:12,  6.64s/it]
 51%|█████     | 115084/225000 [238:47:08<187:57:06,  6.16s/it]
 51%|█████     | 115085/225000 [238:47:13<178:11:58,  5.84s/it]
 51%|█████     | 115086/225000 [238:47:18<176:30:08,  5.78s/it]
 51%|█████     | 115087/225000 [238:47:23<163:47:18,  5.36s/it]
 51%|█████     | 115088/225000 [238:47:26<148:49:03,  4.87s/it]
 51%|█████     | 115089/225000 [238:47:30<139:14:28,  4.56s/it]
 51%|█████     | 115090/225000 [238:47:34<131:19:56,  4.30s/it]
 51%|█████     | 115091/225000 [238:47:38<130:01:11,  4.26s/it]
 51%|█████     | 115092/225000 [238:47:43<132:51:00,  4.35s/it]
 51%|█████     | 115093/225000 [238:47:47<133:22:19,  4.37s/it]
 51%|█████     | 115094/225000 [238:47:59<197:46:10,  6.48s/it]
 51%|█████     | 115095/225000 [238:48:04<188:29:03,  6.17s/it]
 51%|█████     | 115096/225000 [238:48:07<158:23:20,  5.19s/it]
 51%|█████     | 115097/225000 [238:48:13<163:52:02,  5.37s/it]
 51%|█████     | 115098/225000 [238:48:19<175:06:38,  5.74s/it]
 51%|█████     | 115099/225000 [238:48:23<155:56:01,  5.11s/it]
 51%|█████     | 115100/225000 [238:48:27<142:07:49,  4.66s/it]
                                                               
{'loss': '0.9134', 'grad_norm': '1.277', 'learning_rate': '2.59e-05', 'epoch': '0.5116', 'num_input_tokens_seen': 398855024, 'train_runtime': '8.597e+05', 'train_tokens_per_second': '463.9'}

 51%|█████     | 115100/225000 [238:48:27<142:07:49,  4.66s/it]
 51%|█████     | 115101/225000 [238:48:31<138:47:09,  4.55s/it]
 51%|█████     | 115102/225000 [238:48:42<201:42:50,  6.61s/it]
 51%|█████     | 115103/225000 [238:48:46<175:08:14,  5.74s/it]
 51%|█████     | 115104/225000 [238:48:50<155:33:49,  5.10s/it]
 51%|█████     | 115105/225000 [238:48:54<148:26:27,  4.86s/it]
 51%|█████     | 115106/225000 [238:48:58<145:07:57,  4.75s/it]
 51%|█████     | 115107/225000 [238:49:02<133:39:33,  4.38s/it]
 51%|█████     | 115108/225000 [238:49:06<133:21:05,  4.37s/it]
 51%|█████     | 115109/225000 [238:49:11<136:23:28,  4.47s/it]
 51%|█████     | 115110/225000 [238:49:21<188:05:36,  6.16s/it]
 51%|█████     | 115111/225000 [238:49:26<178:43:04,  5.85s/it]
 51%|█████     | 115112/225000 [238:49:30<157:06:22,  5.15s/it]
 51%|█████     | 115113/225000 [238:49:36<165:02:11,  5.41s/it]
 51%|█████     | 115114/225000 [238:49:42<173:09:00,  5.67s/it]
 51%|█████     | 115115/225000 [238:49:49<181:57:16,  5.96s/it]
 51%|█████     | 115116/225000 [238:49:58<211:44:25,  6.94s/it]
 51%|█████     | 115117/225000 [238:50:04<203:28:26,  6.67s/it]
 51%|█████     | 115118/225000 [238:50:12<221:30:40,  7.26s/it]
 51%|█████     | 115119/225000 [238:50:15<181:48:05,  5.96s/it]
 51%|█████     | 115120/225000 [238:50:19<158:01:21,  5.18s/it]
 51%|█████     | 115121/225000 [238:50:27<189:37:58,  6.21s/it]
 51%|█████     | 115122/225000 [238:50:31<167:08:08,  5.48s/it]
 51%|█████     | 115123/225000 [238:50:34<141:54:49,  4.65s/it]
 51%|█████     | 115124/225000 [238:50:38<138:19:41,  4.53s/it]
 51%|█████     | 115125/225000 [238:50:43<140:58:17,  4.62s/it]
 51%|█████     | 115126/225000 [238:50:47<138:34:14,  4.54s/it]
 51%|█████     | 115127/225000 [238:50:51<134:51:56,  4.42s/it]
 51%|█████     | 115128/225000 [238:50:56<136:29:59,  4.47s/it]
 51%|█████     | 115129/225000 [238:51:01<139:06:52,  4.56s/it]
 51%|█████     | 115130/225000 [238:51:06<140:52:05,  4.62s/it]
 51%|█████     | 115131/225000 [238:51:11<144:42:48,  4.74s/it]
 51%|█████     | 115132/225000 [238:51:15<141:57:56,  4.65s/it]
 51%|█████     | 115133/225000 [238:51:26<198:42:48,  6.51s/it]
 51%|█████     | 115134/225000 [238:51:34<217:21:11,  7.12s/it]
 51%|█████     | 115135/225000 [238:51:37<177:08:45,  5.80s/it]
 51%|█████     | 115136/225000 [238:51:41<156:00:41,  5.11s/it]
 51%|█████     | 115137/225000 [238:51:45<145:08:10,  4.76s/it]
 51%|█████     | 115138/225000 [238:51:55<199:43:13,  6.54s/it]
 51%|█████     | 115139/225000 [238:52:01<196:35:37,  6.44s/it]
 51%|█████     | 115140/225000 [238:52:08<196:02:14,  6.42s/it]
 51%|█████     | 115141/225000 [238:52:12<172:36:17,  5.66s/it]
 51%|█████     | 115142/225000 [238:52:16<156:53:17,  5.14s/it]
 51%|█████     | 115143/225000 [238:52:20<148:17:29,  4.86s/it]
 51%|█████     | 115144/225000 [238:52:24<138:16:26,  4.53s/it]
 51%|█████     | 115145/225000 [238:52:28<138:15:18,  4.53s/it]
 51%|█████     | 115146/225000 [238:52:32<132:22:36,  4.34s/it]
 51%|█████     | 115147/225000 [238:52:47<226:00:33,  7.41s/it]
 51%|█████     | 115148/225000 [238:52:50<185:04:22,  6.07s/it]
 51%|█████     | 115149/225000 [238:52:53<158:33:17,  5.20s/it]
 51%|█████     | 115150/225000 [238:53:02<198:25:49,  6.50s/it]
                                                               
{'loss': '0.9084', 'grad_norm': '0.9745', 'learning_rate': '2.588e-05', 'epoch': '0.5118', 'num_input_tokens_seen': 398972016, 'train_runtime': '8.6e+05', 'train_tokens_per_second': '463.9'}

 51%|█████     | 115150/225000 [238:53:02<198:25:49,  6.50s/it]
 51%|█████     | 115151/225000 [238:53:07<177:29:34,  5.82s/it]
 51%|█████     | 115152/225000 [238:53:09<151:31:36,  4.97s/it]
 51%|█████     | 115153/225000 [238:53:13<135:50:42,  4.45s/it]
 51%|█████     | 115154/225000 [238:53:20<165:14:33,  5.42s/it]
 51%|█████     | 115155/225000 [238:53:25<154:38:30,  5.07s/it]
 51%|█████     | 115156/225000 [238:53:28<137:25:08,  4.50s/it]
 51%|█████     | 115157/225000 [238:53:31<121:20:16,  3.98s/it]
 51%|█████     | 115158/225000 [238:53:40<171:56:09,  5.64s/it]
 51%|█████     | 115159/225000 [238:53:46<174:12:37,  5.71s/it]
 51%|█████     | 115160/225000 [238:53:51<165:35:17,  5.43s/it]
 51%|█████     | 115161/225000 [238:53:54<147:25:27,  4.83s/it]
 51%|█████     | 115162/225000 [238:54:02<178:49:19,  5.86s/it]
 51%|█████     | 115163/225000 [238:54:06<157:22:12,  5.16s/it]
 51%|█████     | 115164/225000 [238:54:09<141:21:41,  4.63s/it]
 51%|█████     | 115165/225000 [238:54:13<135:11:23,  4.43s/it]
 51%|█████     | 115166/225000 [238:54:17<128:20:11,  4.21s/it]
 51%|█████     | 115167/225000 [238:54:22<135:20:13,  4.44s/it]
 51%|█████     | 115168/225000 [238:54:26<133:43:22,  4.38s/it]
 51%|█████     | 115169/225000 [238:54:32<148:10:24,  4.86s/it]
 51%|█████     | 115170/225000 [238:54:45<218:36:00,  7.17s/it]
 51%|█████     | 115171/225000 [238:54:48<182:58:18,  6.00s/it]
 51%|█████     | 115172/225000 [238:54:51<154:33:31,  5.07s/it]
 51%|█████     | 115173/225000 [238:54:59<181:24:50,  5.95s/it]
 51%|█████     | 115174/225000 [238:55:04<169:03:56,  5.54s/it]
 51%|█████     | 115175/225000 [238:55:10<179:59:50,  5.90s/it]
 51%|█████     | 115176/225000 [238:55:15<168:13:12,  5.51s/it]
 51%|█████     | 115177/225000 [238:55:20<167:36:24,  5.49s/it]
 51%|█████     | 115178/225000 [238:55:24<153:52:56,  5.04s/it]
 51%|█████     | 115179/225000 [238:55:29<149:25:59,  4.90s/it]
 51%|█████     | 115180/225000 [238:55:34<148:47:43,  4.88s/it]
 51%|█████     | 115181/225000 [238:55:38<141:42:49,  4.65s/it]
 51%|█████     | 115182/225000 [238:55:42<137:18:53,  4.50s/it]
 51%|█████     | 115183/225000 [238:55:46<133:47:39,  4.39s/it]
 51%|█████     | 115184/225000 [238:55:51<139:20:26,  4.57s/it]
 51%|█████     | 115185/225000 [238:56:00<182:04:50,  5.97s/it]
 51%|█████     | 115186/225000 [238:56:12<235:41:02,  7.73s/it]
 51%|█████     | 115187/225000 [238:56:18<222:10:36,  7.28s/it]
 51%|█████     | 115188/225000 [238:56:22<190:03:33,  6.23s/it]
 51%|█████     | 115189/225000 [238:56:26<170:07:39,  5.58s/it]
 51%|█████     | 115190/225000 [238:56:30<150:13:47,  4.93s/it]
 51%|█████     | 115191/225000 [238:56:34<140:50:06,  4.62s/it]
 51%|█████     | 115192/225000 [238:56:37<130:20:27,  4.27s/it]
 51%|█████     | 115193/225000 [238:56:41<130:30:28,  4.28s/it]
 51%|█████     | 115194/225000 [238:56:46<133:56:21,  4.39s/it]
 51%|█████     | 115195/225000 [238:56:50<131:45:20,  4.32s/it]
 51%|█████     | 115196/225000 [238:56:54<132:11:23,  4.33s/it]
 51%|█████     | 115197/225000 [238:57:04<181:18:41,  5.94s/it]
 51%|█████     | 115198/225000 [238:57:12<194:11:17,  6.37s/it]
 51%|█████     | 115199/225000 [238:57:19<206:41:55,  6.78s/it]
 51%|█████     | 115200/225000 [238:57:25<199:31:21,  6.54s/it]
                                                               
{'loss': '0.927', 'grad_norm': '1.445', 'learning_rate': '2.586e-05', 'epoch': '0.512', 'num_input_tokens_seen': 399087288, 'train_runtime': '8.602e+05', 'train_tokens_per_second': '463.9'}

 51%|█████     | 115200/225000 [238:57:25<199:31:21,  6.54s/it]
 51%|█████     | 115201/225000 [238:57:29<176:26:27,  5.78s/it]
 51%|█████     | 115202/225000 [238:57:33<155:23:52,  5.10s/it]
 51%|█████     | 115203/225000 [238:57:36<138:07:53,  4.53s/it]
 51%|█████     | 115204/225000 [238:57:46<190:16:51,  6.24s/it]
 51%|█████     | 115205/225000 [238:57:53<197:22:03,  6.47s/it]
 51%|█████     | 115206/225000 [238:57:58<177:41:13,  5.83s/it]
 51%|█████     | 115207/225000 [238:58:02<162:10:09,  5.32s/it]
 51%|█████     | 115208/225000 [238:58:05<147:42:26,  4.84s/it]
 51%|█████     | 115209/225000 [238:58:10<141:50:06,  4.65s/it]
 51%|█████     | 115210/225000 [238:58:13<132:45:54,  4.35s/it]
 51%|█████     | 115211/225000 [238:58:17<127:31:24,  4.18s/it]
 51%|█████     | 115212/225000 [238:58:28<185:34:08,  6.08s/it]
 51%|█████     | 115213/225000 [238:58:32<170:01:48,  5.58s/it]
 51%|█████     | 115214/225000 [238:58:36<157:42:46,  5.17s/it]
 51%|█████     | 115215/225000 [238:58:48<215:23:22,  7.06s/it]
 51%|█████     | 115216/225000 [238:58:51<182:36:44,  5.99s/it]
 51%|█████     | 115217/225000 [238:58:55<163:46:25,  5.37s/it]
 51%|█████     | 115218/225000 [238:58:59<147:20:06,  4.83s/it]
 51%|█████     | 115219/225000 [238:59:03<146:54:10,  4.82s/it]
 51%|█████     | 115220/225000 [238:59:07<137:33:53,  4.51s/it]
 51%|█████     | 115221/225000 [238:59:10<124:57:47,  4.10s/it]
 51%|█████     | 115222/225000 [238:59:21<184:49:11,  6.06s/it]
 51%|█████     | 115223/225000 [238:59:24<160:34:11,  5.27s/it]
 51%|█████     | 115224/225000 [238:59:35<206:20:50,  6.77s/it]
 51%|█████     | 115225/225000 [238:59:38<175:01:38,  5.74s/it]
 51%|█████     | 115226/225000 [238:59:45<183:58:21,  6.03s/it]
 51%|█████     | 115227/225000 [238:59:51<184:09:44,  6.04s/it]
 51%|█████     | 115228/225000 [238:59:55<168:55:42,  5.54s/it]
 51%|█████     | 115229/225000 [239:00:00<159:28:20,  5.23s/it]
 51%|█████     | 115230/225000 [239:00:04<155:05:37,  5.09s/it]
 51%|█████     | 115231/225000 [239:00:08<144:35:10,  4.74s/it]
 51%|█████     | 115232/225000 [239:00:12<133:35:18,  4.38s/it]
 51%|█████     | 115233/225000 [239:00:16<132:43:53,  4.35s/it]
 51%|█████     | 115234/225000 [239:00:21<134:52:41,  4.42s/it]
 51%|█████     | 115235/225000 [239:00:31<184:35:31,  6.05s/it]
 51%|█████     | 115236/225000 [239:00:35<172:26:30,  5.66s/it]
 51%|█████     | 115237/225000 [239:00:39<149:38:59,  4.91s/it]
 51%|█████     | 115238/225000 [239:00:49<203:38:16,  6.68s/it]
 51%|█████     | 115239/225000 [239:00:53<179:52:38,  5.90s/it]
 51%|█████     | 115240/225000 [239:00:58<165:46:15,  5.44s/it]
 51%|█████     | 115241/225000 [239:01:09<222:03:29,  7.28s/it]
 51%|█████     | 115242/225000 [239:01:12<182:17:55,  5.98s/it]
 51%|█████     | 115243/225000 [239:01:18<179:53:28,  5.90s/it]
 51%|█████     | 115244/225000 [239:01:23<170:53:15,  5.61s/it]
 51%|█████     | 115245/225000 [239:01:26<148:48:11,  4.88s/it]
 51%|█████     | 115246/225000 [239:01:31<144:38:28,  4.74s/it]
 51%|█████     | 115247/225000 [239:01:36<147:47:40,  4.85s/it]
 51%|█████     | 115248/225000 [239:01:40<142:41:23,  4.68s/it]
 51%|█████     | 115249/225000 [239:01:45<146:51:25,  4.82s/it]
 51%|█████     | 115250/225000 [239:01:50<146:59:06,  4.82s/it]
                                                               
{'loss': '0.9193', 'grad_norm': '1.399', 'learning_rate': '2.584e-05', 'epoch': '0.5122', 'num_input_tokens_seen': 399203256, 'train_runtime': '8.605e+05', 'train_tokens_per_second': '463.9'}

 51%|█████     | 115250/225000 [239:01:50<146:59:06,  4.82s/it]
 51%|█████     | 115251/225000 [239:01:54<143:08:40,  4.70s/it]
 51%|█████     | 115252/225000 [239:01:59<139:59:44,  4.59s/it]
 51%|█████     | 115253/225000 [239:02:04<143:47:23,  4.72s/it]
 51%|█████     | 115254/225000 [239:02:15<204:44:52,  6.72s/it]
 51%|█████     | 115255/225000 [239:02:19<178:05:05,  5.84s/it]
 51%|█████     | 115256/225000 [239:02:24<170:56:39,  5.61s/it]
 51%|█████     | 115257/225000 [239:02:29<162:02:23,  5.32s/it]
 51%|█████     | 115258/225000 [239:02:34<160:19:23,  5.26s/it]
 51%|█████     | 115259/225000 [239:02:38<153:07:42,  5.02s/it]
 51%|█████     | 115260/225000 [239:02:42<146:20:01,  4.80s/it]
 51%|█████     | 115261/225000 [239:02:47<145:34:15,  4.78s/it]
 51%|█████     | 115262/225000 [239:02:52<142:27:44,  4.67s/it]
 51%|█████     | 115263/225000 [239:03:03<205:11:22,  6.73s/it]
 51%|█████     | 115264/225000 [239:03:08<185:24:00,  6.08s/it]
 51%|█████     | 115265/225000 [239:03:12<165:24:43,  5.43s/it]
 51%|█████     | 115266/225000 [239:03:16<154:29:43,  5.07s/it]
 51%|█████     | 115267/225000 [239:03:27<211:25:27,  6.94s/it]
 51%|█████     | 115268/225000 [239:03:31<186:47:49,  6.13s/it]
 51%|█████     | 115269/225000 [239:03:39<202:11:25,  6.63s/it]
 51%|█████     | 115270/225000 [239:03:44<182:58:08,  6.00s/it]
 51%|█████     | 115271/225000 [239:03:48<165:05:08,  5.42s/it]
 51%|█████     | 115272/225000 [239:03:52<152:05:36,  4.99s/it]
 51%|█████     | 115273/225000 [239:03:56<143:48:42,  4.72s/it]
 51%|█████     | 115274/225000 [239:04:00<139:24:12,  4.57s/it]
 51%|█████     | 115275/225000 [239:04:04<132:07:48,  4.34s/it]
 51%|█████     | 115276/225000 [239:04:08<127:08:15,  4.17s/it]
 51%|█████     | 115277/225000 [239:04:12<128:55:46,  4.23s/it]
 51%|█████     | 115278/225000 [239:04:17<138:29:47,  4.54s/it]
 51%|█████     | 115279/225000 [239:04:22<138:54:58,  4.56s/it]
 51%|█████     | 115280/225000 [239:04:29<158:20:25,  5.20s/it]
 51%|█████     | 115281/225000 [239:04:39<202:42:01,  6.65s/it]
 51%|█████     | 115282/225000 [239:04:43<180:46:25,  5.93s/it]
 51%|█████     | 115283/225000 [239:04:47<164:40:34,  5.40s/it]
 51%|█████     | 115284/225000 [239:04:50<145:29:05,  4.77s/it]
 51%|█████     | 115285/225000 [239:04:54<136:10:55,  4.47s/it]
 51%|█████     | 115286/225000 [239:04:58<128:50:59,  4.23s/it]
 51%|█████     | 115287/225000 [239:05:02<127:34:22,  4.19s/it]
 51%|█████     | 115288/225000 [239:05:06<127:14:56,  4.18s/it]
 51%|█████     | 115289/225000 [239:05:17<192:31:12,  6.32s/it]
 51%|█████     | 115290/225000 [239:05:27<227:25:02,  7.46s/it]
 51%|█████     | 115291/225000 [239:05:32<201:26:28,  6.61s/it]
 51%|█████     | 115292/225000 [239:05:36<175:01:32,  5.74s/it]
 51%|█████     | 115293/225000 [239:05:42<178:49:38,  5.87s/it]
 51%|█████     | 115294/225000 [239:05:49<189:50:26,  6.23s/it]
 51%|█████     | 115295/225000 [239:05:52<161:14:19,  5.29s/it]
 51%|█████     | 115296/225000 [239:05:56<149:43:44,  4.91s/it]
 51%|█████     | 115297/225000 [239:06:04<172:56:01,  5.67s/it]
 51%|█████     | 115298/225000 [239:06:07<152:36:42,  5.01s/it]
 51%|█████     | 115299/225000 [239:06:15<175:22:02,  5.75s/it]
 51%|█████     | 115300/225000 [239:06:18<150:38:41,  4.94s/it]
                                                               
{'loss': '0.9032', 'grad_norm': '1.392', 'learning_rate': '2.582e-05', 'epoch': '0.5124', 'num_input_tokens_seen': 399317672, 'train_runtime': '8.608e+05', 'train_tokens_per_second': '463.9'}

 51%|█████     | 115300/225000 [239:06:18<150:38:41,  4.94s/it]
 51%|█████     | 115301/225000 [239:06:23<155:11:21,  5.09s/it]
 51%|█████     | 115302/225000 [239:06:31<177:57:11,  5.84s/it]
 51%|█████     | 115303/225000 [239:06:37<185:42:46,  6.09s/it]
 51%|█████     | 115304/225000 [239:06:46<206:52:02,  6.79s/it]
 51%|█████     | 115305/225000 [239:06:49<174:01:59,  5.71s/it]
 51%|█████     | 115306/225000 [239:06:54<166:00:45,  5.45s/it]
 51%|█████     | 115307/225000 [239:06:58<158:35:52,  5.21s/it]
 51%|█████     | 115308/225000 [239:07:10<217:54:55,  7.15s/it]
 51%|█████     | 115309/225000 [239:07:14<185:22:00,  6.08s/it]
 51%|█████     | 115310/225000 [239:07:18<168:36:44,  5.53s/it]
 51%|█████     | 115311/225000 [239:07:22<151:54:52,  4.99s/it]
 51%|█████     | 115312/225000 [239:07:25<138:03:10,  4.53s/it]
 51%|█████▏    | 115313/225000 [239:07:34<176:48:37,  5.80s/it]
 51%|█████▏    | 115314/225000 [239:07:40<182:26:16,  5.99s/it]
 51%|█████▏    | 115315/225000 [239:07:43<157:00:10,  5.15s/it]
 51%|█████▏    | 115316/225000 [239:07:47<144:39:42,  4.75s/it]
 51%|█████▏    | 115317/225000 [239:07:56<182:30:05,  5.99s/it]
 51%|█████▏    | 115318/225000 [239:07:59<155:57:30,  5.12s/it]
 51%|█████▏    | 115319/225000 [239:08:07<175:38:33,  5.77s/it]
 51%|█████▏    | 115320/225000 [239:08:11<167:02:38,  5.48s/it]
 51%|█████▏    | 115321/225000 [239:08:15<152:23:46,  5.00s/it]
 51%|█████▏    | 115322/225000 [239:08:19<144:31:35,  4.74s/it]
 51%|█████▏    | 115323/225000 [239:08:24<142:45:15,  4.69s/it]
 51%|█████▏    | 115324/225000 [239:08:29<141:49:49,  4.66s/it]
 51%|█████▏    | 115325/225000 [239:08:33<137:34:39,  4.52s/it]
 51%|█████▏    | 115326/225000 [239:08:37<139:36:42,  4.58s/it]
 51%|█████▏    | 115327/225000 [239:08:42<142:16:22,  4.67s/it]
 51%|█████▏    | 115328/225000 [239:08:47<142:24:56,  4.67s/it]
 51%|█████▏    | 115329/225000 [239:08:51<140:17:43,  4.61s/it]
 51%|█████▏    | 115330/225000 [239:09:01<190:10:59,  6.24s/it]
 51%|█████▏    | 115331/225000 [239:09:10<212:48:37,  6.99s/it]
 51%|█████▏    | 115332/225000 [239:09:13<175:44:15,  5.77s/it]
 51%|█████▏    | 115333/225000 [239:09:16<150:40:23,  4.95s/it]
 51%|█████▏    | 115334/225000 [239:09:23<168:20:24,  5.53s/it]
 51%|█████▏    | 115335/225000 [239:09:29<174:23:48,  5.72s/it]
 51%|█████▏    | 115336/225000 [239:09:34<166:03:58,  5.45s/it]
 51%|█████▏    | 115337/225000 [239:09:39<159:00:01,  5.22s/it]
 51%|█████▏    | 115338/225000 [239:09:42<139:42:54,  4.59s/it]
 51%|█████▏    | 115339/225000 [239:09:46<136:54:36,  4.49s/it]
 51%|█████▏    | 115340/225000 [239:09:50<132:11:12,  4.34s/it]
 51%|█████▏    | 115341/225000 [239:09:55<133:11:17,  4.37s/it]
 51%|█████▏    | 115342/225000 [239:10:00<138:41:04,  4.55s/it]
 51%|█████▏    | 115343/225000 [239:10:10<190:53:44,  6.27s/it]
 51%|█████▏    | 115344/225000 [239:10:14<176:22:26,  5.79s/it]
 51%|█████▏    | 115345/225000 [239:10:19<161:24:54,  5.30s/it]
 51%|█████▏    | 115346/225000 [239:10:27<185:39:24,  6.10s/it]
 51%|█████▏    | 115347/225000 [239:10:30<161:42:32,  5.31s/it]
 51%|█████▏    | 115348/225000 [239:10:33<141:48:43,  4.66s/it]
 51%|█████▏    | 115349/225000 [239:10:37<135:37:24,  4.45s/it]
 51%|█████▏    | 115350/225000 [239:10:48<193:14:30,  6.34s/it]
                                                               
{'loss': '0.9056', 'grad_norm': '1.113', 'learning_rate': '2.58e-05', 'epoch': '0.5127', 'num_input_tokens_seen': 399432200, 'train_runtime': '8.61e+05', 'train_tokens_per_second': '463.9'}

 51%|█████▏    | 115350/225000 [239:10:48<193:14:30,  6.34s/it]
 51%|█████▏    | 115351/225000 [239:10:51<161:44:18,  5.31s/it]
 51%|█████▏    | 115352/225000 [239:10:58<176:58:36,  5.81s/it]
 51%|█████▏    | 115353/225000 [239:11:02<157:48:34,  5.18s/it]
 51%|█████▏    | 115354/225000 [239:11:04<136:39:31,  4.49s/it]
 51%|█████▏    | 115355/225000 [239:11:08<128:59:00,  4.23s/it]
 51%|█████▏    | 115356/225000 [239:11:17<169:17:53,  5.56s/it]
 51%|█████▏    | 115357/225000 [239:11:24<180:57:46,  5.94s/it]
 51%|█████▏    | 115358/225000 [239:11:28<164:16:11,  5.39s/it]
 51%|█████▏    | 115359/225000 [239:11:31<148:32:35,  4.88s/it]
 51%|█████▏    | 115360/225000 [239:11:36<145:29:33,  4.78s/it]
 51%|█████▏    | 115361/225000 [239:11:42<154:27:55,  5.07s/it]
 51%|█████▏    | 115362/225000 [239:11:53<210:52:05,  6.92s/it]
 51%|█████▏    | 115363/225000 [239:11:58<190:33:21,  6.26s/it]
 51%|█████▏    | 115364/225000 [239:12:01<166:43:17,  5.47s/it]
 51%|█████▏    | 115365/225000 [239:12:07<171:07:03,  5.62s/it]
 51%|█████▏    | 115366/225000 [239:12:14<182:45:08,  6.00s/it]
 51%|█████▏    | 115367/225000 [239:12:18<159:33:27,  5.24s/it]
 51%|█████▏    | 115368/225000 [239:12:23<165:46:56,  5.44s/it]
 51%|█████▏    | 115369/225000 [239:12:30<172:26:46,  5.66s/it]
 51%|█████▏    | 115370/225000 [239:12:33<148:25:22,  4.87s/it]
 51%|█████▏    | 115371/225000 [239:12:36<136:42:59,  4.49s/it]
 51%|█████▏    | 115372/225000 [239:12:45<173:05:44,  5.68s/it]
 51%|█████▏    | 115373/225000 [239:12:51<176:31:05,  5.80s/it]
 51%|█████▏    | 115374/225000 [239:12:55<165:49:02,  5.45s/it]
 51%|█████▏    | 115375/225000 [239:13:01<166:17:01,  5.46s/it]
 51%|█████▏    | 115376/225000 [239:13:06<160:47:00,  5.28s/it]
 51%|█████▏    | 115377/225000 [239:13:11<157:06:19,  5.16s/it]
 51%|█████▏    | 115378/225000 [239:13:16<159:50:52,  5.25s/it]
 51%|█████▏    | 115379/225000 [239:13:20<150:47:57,  4.95s/it]
 51%|█████▏    | 115380/225000 [239:13:25<143:54:07,  4.73s/it]
 51%|█████▏    | 115381/225000 [239:13:29<139:41:43,  4.59s/it]
 51%|█████▏    | 115382/225000 [239:13:33<136:58:36,  4.50s/it]
 51%|█████▏    | 115383/225000 [239:13:46<212:59:47,  7.00s/it]
 51%|█████▏    | 115384/225000 [239:13:49<175:01:11,  5.75s/it]
 51%|█████▏    | 115385/225000 [239:13:58<203:33:57,  6.69s/it]
 51%|█████▏    | 115386/225000 [239:14:01<176:08:34,  5.78s/it]
 51%|█████▏    | 115387/225000 [239:14:08<183:42:00,  6.03s/it]
 51%|█████▏    | 115388/225000 [239:14:14<183:48:42,  6.04s/it]
 51%|█████▏    | 115389/225000 [239:14:18<163:51:03,  5.38s/it]
 51%|█████▏    | 115390/225000 [239:14:22<151:19:49,  4.97s/it]
 51%|█████▏    | 115391/225000 [239:14:25<138:45:38,  4.56s/it]
 51%|█████▏    | 115392/225000 [239:14:34<177:19:35,  5.82s/it]
 51%|█████▏    | 115393/225000 [239:14:38<158:10:22,  5.20s/it]
 51%|█████▏    | 115394/225000 [239:14:46<188:09:42,  6.18s/it]
 51%|█████▏    | 115395/225000 [239:14:49<158:38:42,  5.21s/it]
 51%|█████▏    | 115396/225000 [239:14:53<145:50:49,  4.79s/it]
 51%|█████▏    | 115397/225000 [239:15:00<162:05:41,  5.32s/it]
 51%|█████▏    | 115398/225000 [239:15:03<143:01:30,  4.70s/it]
 51%|█████▏    | 115399/225000 [239:15:08<147:15:45,  4.84s/it]
 51%|█████▏    | 115400/225000 [239:15:20<207:59:24,  6.83s/it]
                                                               
{'loss': '0.9191', 'grad_norm': '1.19', 'learning_rate': '2.579e-05', 'epoch': '0.5129', 'num_input_tokens_seen': 399548752, 'train_runtime': '8.613e+05', 'train_tokens_per_second': '463.9'}

 51%|█████▏    | 115400/225000 [239:15:20<207:59:24,  6.83s/it]
 51%|█████▏    | 115401/225000 [239:15:24<184:13:12,  6.05s/it]
 51%|█████▏    | 115402/225000 [239:15:30<184:08:46,  6.05s/it]
 51%|█████▏    | 115403/225000 [239:15:33<153:22:06,  5.04s/it]
 51%|█████▏    | 115404/225000 [239:15:35<133:43:49,  4.39s/it]
 51%|█████▏    | 115405/225000 [239:15:39<126:22:42,  4.15s/it]
 51%|█████▏    | 115406/225000 [239:15:43<127:09:00,  4.18s/it]
 51%|█████▏    | 115407/225000 [239:15:48<130:29:44,  4.29s/it]
 51%|█████▏    | 115408/225000 [239:15:52<127:32:02,  4.19s/it]
 51%|█████▏    | 115409/225000 [239:15:56<124:05:07,  4.08s/it]
 51%|█████▏    | 115410/225000 [239:16:07<188:48:28,  6.20s/it]
 51%|█████▏    | 115411/225000 [239:16:14<195:10:15,  6.41s/it]
 51%|█████▏    | 115412/225000 [239:16:17<163:49:55,  5.38s/it]
 51%|█████▏    | 115413/225000 [239:16:25<191:09:00,  6.28s/it]
 51%|█████▏    | 115414/225000 [239:16:28<165:35:42,  5.44s/it]
 51%|█████▏    | 115415/225000 [239:16:32<152:26:15,  5.01s/it]
 51%|█████▏    | 115416/225000 [239:16:41<181:25:00,  5.96s/it]
 51%|█████▏    | 115417/225000 [239:16:46<176:45:25,  5.81s/it]
 51%|█████▏    | 115418/225000 [239:16:50<159:36:51,  5.24s/it]
 51%|█████▏    | 115419/225000 [239:16:55<160:29:09,  5.27s/it]
 51%|█████▏    | 115420/225000 [239:16:59<143:21:25,  4.71s/it]
 51%|█████▏    | 115421/225000 [239:17:02<134:15:44,  4.41s/it]
 51%|█████▏    | 115422/225000 [239:17:11<173:35:12,  5.70s/it]
 51%|█████▏    | 115423/225000 [239:17:17<171:39:29,  5.64s/it]
 51%|█████▏    | 115424/225000 [239:17:24<184:48:43,  6.07s/it]
 51%|█████▏    | 115425/225000 [239:17:28<163:50:13,  5.38s/it]
 51%|█████▏    | 115426/225000 [239:17:32<151:20:34,  4.97s/it]
 51%|█████▏    | 115427/225000 [239:17:36<145:32:08,  4.78s/it]
 51%|█████▏    | 115428/225000 [239:17:40<140:57:53,  4.63s/it]
 51%|█████▏    | 115429/225000 [239:17:44<136:14:15,  4.48s/it]
 51%|█████▏    | 115430/225000 [239:17:49<134:51:12,  4.43s/it]
 51%|█████▏    | 115431/225000 [239:17:55<153:22:27,  5.04s/it]
 51%|█████▏    | 115432/225000 [239:18:02<174:39:17,  5.74s/it]
 51%|█████▏    | 115433/225000 [239:18:08<169:46:10,  5.58s/it]
 51%|█████▏    | 115434/225000 [239:18:11<150:01:53,  4.93s/it]
 51%|█████▏    | 115435/225000 [239:18:16<147:37:12,  4.85s/it]
 51%|█████▏    | 115436/225000 [239:18:20<146:29:01,  4.81s/it]
 51%|█████▏    | 115437/225000 [239:18:26<155:13:54,  5.10s/it]
 51%|█████▏    | 115438/225000 [239:18:31<155:32:50,  5.11s/it]
 51%|█████▏    | 115439/225000 [239:18:36<150:12:06,  4.94s/it]
 51%|█████▏    | 115440/225000 [239:18:40<143:44:21,  4.72s/it]
 51%|█████▏    | 115441/225000 [239:18:45<146:10:38,  4.80s/it]
 51%|█████▏    | 115442/225000 [239:18:50<143:23:51,  4.71s/it]
 51%|█████▏    | 115443/225000 [239:18:57<164:33:06,  5.41s/it]
 51%|█████▏    | 115444/225000 [239:19:04<179:57:26,  5.91s/it]
 51%|█████▏    | 115445/225000 [239:19:07<154:29:37,  5.08s/it]
 51%|█████▏    | 115446/225000 [239:19:15<185:06:30,  6.08s/it]
 51%|█████▏    | 115447/225000 [239:19:21<181:39:46,  5.97s/it]
 51%|█████▏    | 115448/225000 [239:19:25<159:52:07,  5.25s/it]
 51%|█████▏    | 115449/225000 [239:19:28<146:34:54,  4.82s/it]
 51%|█████▏    | 115450/225000 [239:19:36<170:58:13,  5.62s/it]
                                                               
{'loss': '0.9088', 'grad_norm': '0.961', 'learning_rate': '2.577e-05', 'epoch': '0.5131', 'num_input_tokens_seen': 399661696, 'train_runtime': '8.616e+05', 'train_tokens_per_second': '463.9'}

 51%|█████▏    | 115450/225000 [239:19:36<170:58:13,  5.62s/it]
 51%|█████▏    | 115451/225000 [239:19:41<166:03:35,  5.46s/it]
 51%|█████▏    | 115452/225000 [239:19:44<143:18:41,  4.71s/it]
 51%|█████▏    | 115453/225000 [239:19:47<129:34:31,  4.26s/it]
 51%|█████▏    | 115454/225000 [239:19:53<145:09:20,  4.77s/it]
 51%|█████▏    | 115455/225000 [239:19:59<156:46:58,  5.15s/it]
 51%|█████▏    | 115456/225000 [239:20:03<140:46:26,  4.63s/it]
 51%|█████▏    | 115457/225000 [239:20:12<186:39:25,  6.13s/it]
 51%|█████▏    | 115458/225000 [239:20:15<159:06:59,  5.23s/it]
 51%|█████▏    | 115459/225000 [239:20:26<207:28:11,  6.82s/it]
 51%|█████▏    | 115460/225000 [239:20:29<170:28:21,  5.60s/it]
 51%|█████▏    | 115461/225000 [239:20:32<154:06:36,  5.06s/it]
 51%|█████▏    | 115462/225000 [239:20:37<150:59:21,  4.96s/it]
 51%|█████▏    | 115463/225000 [239:20:41<144:26:33,  4.75s/it]
 51%|█████▏    | 115464/225000 [239:20:46<144:51:51,  4.76s/it]
 51%|█████▏    | 115465/225000 [239:20:50<136:58:51,  4.50s/it]
 51%|█████▏    | 115466/225000 [239:20:54<132:18:31,  4.35s/it]
 51%|█████▏    | 115467/225000 [239:20:58<131:20:41,  4.32s/it]
 51%|█████▏    | 115468/225000 [239:21:07<169:18:35,  5.56s/it]
 51%|█████▏    | 115469/225000 [239:21:16<202:00:00,  6.64s/it]
 51%|█████▏    | 115470/225000 [239:21:19<170:06:27,  5.59s/it]
 51%|█████▏    | 115471/225000 [239:21:29<207:44:13,  6.83s/it]
 51%|█████▏    | 115472/225000 [239:21:32<173:55:23,  5.72s/it]
 51%|█████▏    | 115473/225000 [239:21:35<146:55:41,  4.83s/it]
 51%|█████▏    | 115474/225000 [239:21:44<187:43:04,  6.17s/it]
 51%|█████▏    | 115475/225000 [239:21:48<166:38:24,  5.48s/it]
 51%|█████▏    | 115476/225000 [239:21:51<146:37:29,  4.82s/it]
 51%|█████▏    | 115477/225000 [239:21:55<135:31:11,  4.45s/it]
 51%|█████▏    | 115478/225000 [239:22:03<173:30:20,  5.70s/it]
 51%|█████▏    | 115479/225000 [239:22:09<176:35:04,  5.80s/it]
 51%|█████▏    | 115480/225000 [239:22:13<156:44:52,  5.15s/it]
 51%|█████▏    | 115481/225000 [239:22:17<141:40:55,  4.66s/it]
 51%|█████▏    | 115482/225000 [239:22:20<129:06:29,  4.24s/it]
 51%|█████▏    | 115483/225000 [239:22:30<184:41:37,  6.07s/it]
 51%|█████▏    | 115484/225000 [239:22:33<159:49:33,  5.25s/it]
 51%|█████▏    | 115485/225000 [239:22:37<148:04:38,  4.87s/it]
 51%|█████▏    | 115486/225000 [239:22:41<136:32:17,  4.49s/it]
 51%|█████▏    | 115487/225000 [239:22:45<129:14:06,  4.25s/it]
 51%|█████▏    | 115488/225000 [239:22:49<127:11:26,  4.18s/it]
 51%|█████▏    | 115489/225000 [239:22:54<135:25:02,  4.45s/it]
 51%|█████▏    | 115490/225000 [239:22:59<139:07:46,  4.57s/it]
 51%|█████▏    | 115491/225000 [239:23:10<198:41:21,  6.53s/it]
 51%|█████▏    | 115492/225000 [239:23:16<197:51:21,  6.50s/it]
 51%|█████▏    | 115493/225000 [239:23:19<167:06:02,  5.49s/it]
 51%|█████▏    | 115494/225000 [239:23:25<169:33:56,  5.57s/it]
 51%|█████▏    | 115495/225000 [239:23:33<187:15:42,  6.16s/it]
 51%|█████▏    | 115496/225000 [239:23:37<170:37:18,  5.61s/it]
 51%|█████▏    | 115497/225000 [239:23:41<152:16:31,  5.01s/it]
 51%|█████▏    | 115498/225000 [239:23:45<143:35:15,  4.72s/it]
 51%|█████▏    | 115499/225000 [239:23:52<169:07:56,  5.56s/it]
 51%|█████▏    | 115500/225000 [239:23:59<184:14:15,  6.06s/it]
                                                               
{'loss': '0.9115', 'grad_norm': '1.327', 'learning_rate': '2.575e-05', 'epoch': '0.5133', 'num_input_tokens_seen': 399776272, 'train_runtime': '8.618e+05', 'train_tokens_per_second': '463.9'}

 51%|█████▏    | 115500/225000 [239:23:59<184:14:15,  6.06s/it]
 51%|█████▏    | 115501/225000 [239:24:03<165:03:14,  5.43s/it]
 51%|█████▏    | 115502/225000 [239:24:12<190:31:46,  6.26s/it]
 51%|█████▏    | 115503/225000 [239:24:15<164:46:13,  5.42s/it]
 51%|█████▏    | 115504/225000 [239:24:22<175:37:34,  5.77s/it]
 51%|█████▏    | 115505/225000 [239:24:26<166:51:13,  5.49s/it]
 51%|█████▏    | 115506/225000 [239:24:30<148:28:49,  4.88s/it]
 51%|█████▏    | 115507/225000 [239:24:33<135:24:31,  4.45s/it]
 51%|█████▏    | 115508/225000 [239:24:38<138:51:54,  4.57s/it]
 51%|█████▏    | 115509/225000 [239:24:49<193:39:06,  6.37s/it]
 51%|█████▏    | 115510/225000 [239:24:52<164:34:49,  5.41s/it]
 51%|█████▏    | 115511/225000 [239:24:55<139:22:03,  4.58s/it]
 51%|█████▏    | 115512/225000 [239:25:02<167:53:20,  5.52s/it]
 51%|█████▏    | 115513/225000 [239:25:06<152:18:49,  5.01s/it]
 51%|█████▏    | 115514/225000 [239:25:09<133:43:23,  4.40s/it]
 51%|█████▏    | 115515/225000 [239:25:18<173:12:33,  5.70s/it]
 51%|█████▏    | 115516/225000 [239:25:21<154:48:37,  5.09s/it]
 51%|█████▏    | 115517/225000 [239:25:25<137:42:47,  4.53s/it]
 51%|█████▏    | 115518/225000 [239:25:28<127:51:02,  4.20s/it]
 51%|█████▏    | 115519/225000 [239:25:38<182:03:20,  5.99s/it]
 51%|█████▏    | 115520/225000 [239:25:42<158:49:53,  5.22s/it]
 51%|█████▏    | 115521/225000 [239:25:45<145:43:17,  4.79s/it]
 51%|█████▏    | 115522/225000 [239:25:49<134:37:39,  4.43s/it]
 51%|█████▏    | 115523/225000 [239:25:53<130:58:42,  4.31s/it]
 51%|█████▏    | 115524/225000 [239:25:58<132:37:25,  4.36s/it]
 51%|█████▏    | 115525/225000 [239:26:04<147:52:32,  4.86s/it]
 51%|█████▏    | 115526/225000 [239:26:14<199:02:22,  6.55s/it]
 51%|█████▏    | 115527/225000 [239:26:18<175:54:01,  5.78s/it]
 51%|█████▏    | 115528/225000 [239:26:22<157:27:33,  5.18s/it]
 51%|█████▏    | 115529/225000 [239:26:25<142:47:05,  4.70s/it]
 51%|█████▏    | 115530/225000 [239:26:30<137:45:47,  4.53s/it]
 51%|█████▏    | 115531/225000 [239:26:40<190:05:29,  6.25s/it]
 51%|█████▏    | 115532/225000 [239:26:43<162:24:12,  5.34s/it]
 51%|█████▏    | 115533/225000 [239:26:47<149:21:53,  4.91s/it]
 51%|█████▏    | 115534/225000 [239:26:52<147:12:12,  4.84s/it]
 51%|█████▏    | 115535/225000 [239:26:56<143:34:33,  4.72s/it]
 51%|█████▏    | 115536/225000 [239:27:00<139:25:28,  4.59s/it]
 51%|█████▏    | 115537/225000 [239:27:08<168:10:59,  5.53s/it]
 51%|█████▏    | 115538/225000 [239:27:13<159:52:18,  5.26s/it]
 51%|█████▏    | 115539/225000 [239:27:18<158:34:14,  5.22s/it]
 51%|█████▏    | 115540/225000 [239:27:24<168:38:51,  5.55s/it]
 51%|█████▏    | 115541/225000 [239:27:28<157:29:35,  5.18s/it]
 51%|█████▏    | 115542/225000 [239:27:32<140:00:25,  4.60s/it]
 51%|█████▏    | 115543/225000 [239:27:35<126:44:00,  4.17s/it]
 51%|█████▏    | 115544/225000 [239:27:39<125:59:08,  4.14s/it]
 51%|█████▏    | 115545/225000 [239:27:43<123:52:34,  4.07s/it]
 51%|█████▏    | 115546/225000 [239:27:53<179:11:49,  5.89s/it]
 51%|█████▏    | 115547/225000 [239:27:57<164:15:20,  5.40s/it]
 51%|█████▏    | 115548/225000 [239:28:04<173:26:37,  5.70s/it]
 51%|█████▏    | 115549/225000 [239:28:09<168:05:47,  5.53s/it]
 51%|█████▏    | 115550/225000 [239:28:16<182:16:50,  6.00s/it]
                                                               
{'loss': '0.9041', 'grad_norm': '1.106', 'learning_rate': '2.573e-05', 'epoch': '0.5136', 'num_input_tokens_seen': 399887960, 'train_runtime': '8.621e+05', 'train_tokens_per_second': '463.9'}

 51%|█████▏    | 115550/225000 [239:28:16<182:16:50,  6.00s/it]
 51%|█████▏    | 115551/225000 [239:28:20<163:35:20,  5.38s/it]
 51%|█████▏    | 115552/225000 [239:28:23<147:06:27,  4.84s/it]
 51%|█████▏    | 115553/225000 [239:28:27<133:41:09,  4.40s/it]
 51%|█████▏    | 115554/225000 [239:28:30<126:56:25,  4.18s/it]
 51%|█████▏    | 115555/225000 [239:28:45<219:53:19,  7.23s/it]
 51%|█████▏    | 115556/225000 [239:28:48<181:19:07,  5.96s/it]
 51%|█████▏    | 115557/225000 [239:28:58<220:06:58,  7.24s/it]
 51%|█████▏    | 115558/225000 [239:29:01<185:19:35,  6.10s/it]
 51%|█████▏    | 115559/225000 [239:29:06<170:32:44,  5.61s/it]
 51%|█████▏    | 115560/225000 [239:29:13<184:50:00,  6.08s/it]
 51%|█████▏    | 115561/225000 [239:29:17<162:01:50,  5.33s/it]
 51%|█████▏    | 115562/225000 [239:29:20<146:00:55,  4.80s/it]
 51%|█████▏    | 115563/225000 [239:29:27<164:47:49,  5.42s/it]
 51%|█████▏    | 115564/225000 [239:29:30<141:39:59,  4.66s/it]
 51%|█████▏    | 115565/225000 [239:29:34<137:11:57,  4.51s/it]
 51%|█████▏    | 115566/225000 [239:29:38<127:56:12,  4.21s/it]
 51%|█████▏    | 115567/225000 [239:29:49<190:45:40,  6.28s/it]
 51%|█████▏    | 115568/225000 [239:29:52<163:30:06,  5.38s/it]
 51%|█████▏    | 115569/225000 [239:29:58<167:41:58,  5.52s/it]
 51%|█████▏    | 115570/225000 [239:30:03<162:33:02,  5.35s/it]
 51%|█████▏    | 115571/225000 [239:30:06<145:48:14,  4.80s/it]
 51%|█████▏    | 115572/225000 [239:30:10<136:01:12,  4.47s/it]
 51%|█████▏    | 115573/225000 [239:30:19<177:20:37,  5.83s/it]
 51%|█████▏    | 115574/225000 [239:30:23<161:55:52,  5.33s/it]
 51%|█████▏    | 115575/225000 [239:30:27<146:33:54,  4.82s/it]
 51%|█████▏    | 115576/225000 [239:30:30<134:18:49,  4.42s/it]
 51%|█████▏    | 115577/225000 [239:30:34<129:20:04,  4.26s/it]
 51%|█████▏    | 115578/225000 [239:30:38<125:52:16,  4.14s/it]
 51%|█████▏    | 115579/225000 [239:30:47<170:00:03,  5.59s/it]
 51%|█████▏    | 115580/225000 [239:30:55<187:43:27,  6.18s/it]
 51%|█████▏    | 115581/225000 [239:30:58<162:00:34,  5.33s/it]
 51%|█████▏    | 115582/225000 [239:31:02<146:19:59,  4.81s/it]
 51%|█████▏    | 115583/225000 [239:31:07<149:10:17,  4.91s/it]
 51%|█████▏    | 115584/225000 [239:31:11<141:36:41,  4.66s/it]
 51%|█████▏    | 115585/225000 [239:31:15<135:43:07,  4.47s/it]
 51%|█████▏    | 115586/225000 [239:31:19<133:32:08,  4.39s/it]
 51%|█████▏    | 115587/225000 [239:31:24<137:36:38,  4.53s/it]
 51%|█████▏    | 115588/225000 [239:31:28<137:46:06,  4.53s/it]
 51%|█████▏    | 115589/225000 [239:31:39<196:56:05,  6.48s/it]
 51%|█████▏    | 115590/225000 [239:31:44<177:31:11,  5.84s/it]
 51%|█████▏    | 115591/225000 [239:31:50<177:27:31,  5.84s/it]
 51%|█████▏    | 115592/225000 [239:31:56<179:35:14,  5.91s/it]
 51%|█████▏    | 115593/225000 [239:31:59<156:24:24,  5.15s/it]
 51%|█████▏    | 115594/225000 [239:32:02<139:49:48,  4.60s/it]
 51%|█████▏    | 115595/225000 [239:32:07<139:53:18,  4.60s/it]
 51%|█████▏    | 115596/225000 [239:32:11<134:28:05,  4.42s/it]
 51%|█████▏    | 115597/225000 [239:32:20<179:03:34,  5.89s/it]
 51%|█████▏    | 115598/225000 [239:32:26<180:35:40,  5.94s/it]
 51%|█████▏    | 115599/225000 [239:32:30<155:07:20,  5.10s/it]
 51%|█████▏    | 115600/225000 [239:32:38<187:13:47,  6.16s/it]
                                                               
{'loss': '0.8986', 'grad_norm': '1.489', 'learning_rate': '2.571e-05', 'epoch': '0.5138', 'num_input_tokens_seen': 400001752, 'train_runtime': '8.624e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115600/225000 [239:32:38<187:13:47,  6.16s/it]
 51%|█████▏    | 115601/225000 [239:32:41<156:52:46,  5.16s/it]
 51%|█████▏    | 115602/225000 [239:32:44<139:46:41,  4.60s/it]
 51%|█████▏    | 115603/225000 [239:32:54<182:05:57,  5.99s/it]
 51%|█████▏    | 115604/225000 [239:32:57<159:11:38,  5.24s/it]
 51%|█████▏    | 115605/225000 [239:33:02<158:42:19,  5.22s/it]
 51%|█████▏    | 115606/225000 [239:33:09<171:32:53,  5.65s/it]
 51%|█████▏    | 115607/225000 [239:33:15<175:57:35,  5.79s/it]
 51%|█████▏    | 115608/225000 [239:33:18<148:04:43,  4.87s/it]
 51%|█████▏    | 115609/225000 [239:33:21<134:37:44,  4.43s/it]
 51%|█████▏    | 115610/225000 [239:33:25<128:13:31,  4.22s/it]
 51%|█████▏    | 115611/225000 [239:33:35<181:44:05,  5.98s/it]
 51%|█████▏    | 115612/225000 [239:33:39<162:57:56,  5.36s/it]
 51%|█████▏    | 115613/225000 [239:33:43<150:58:15,  4.97s/it]
 51%|█████▏    | 115614/225000 [239:33:47<146:00:04,  4.81s/it]
 51%|█████▏    | 115615/225000 [239:33:58<197:13:57,  6.49s/it]
 51%|█████▏    | 115616/225000 [239:34:02<179:24:53,  5.90s/it]
 51%|█████▏    | 115617/225000 [239:34:07<170:57:28,  5.63s/it]
 51%|█████▏    | 115618/225000 [239:34:12<164:25:37,  5.41s/it]
 51%|█████▏    | 115619/225000 [239:34:17<162:02:01,  5.33s/it]
 51%|█████▏    | 115620/225000 [239:34:22<159:42:44,  5.26s/it]
 51%|█████▏    | 115621/225000 [239:34:35<226:22:05,  7.45s/it]
 51%|█████▏    | 115622/225000 [239:34:41<211:01:15,  6.95s/it]
 51%|█████▏    | 115623/225000 [239:34:45<186:25:45,  6.14s/it]
 51%|█████▏    | 115624/225000 [239:34:53<206:06:11,  6.78s/it]
 51%|█████▏    | 115625/225000 [239:34:57<179:55:53,  5.92s/it]
 51%|█████▏    | 115626/225000 [239:35:01<162:55:50,  5.36s/it]
 51%|█████▏    | 115627/225000 [239:35:10<197:06:52,  6.49s/it]
 51%|█████▏    | 115628/225000 [239:35:13<164:55:21,  5.43s/it]
 51%|█████▏    | 115629/225000 [239:35:17<149:06:33,  4.91s/it]
 51%|█████▏    | 115630/225000 [239:35:26<184:56:09,  6.09s/it]
 51%|█████▏    | 115631/225000 [239:35:29<159:57:06,  5.26s/it]
 51%|█████▏    | 115632/225000 [239:35:33<148:35:01,  4.89s/it]
 51%|█████▏    | 115633/225000 [239:35:41<171:31:01,  5.65s/it]
 51%|█████▏    | 115634/225000 [239:35:44<152:08:16,  5.01s/it]
 51%|█████▏    | 115635/225000 [239:35:52<176:45:54,  5.82s/it]
 51%|█████▏    | 115636/225000 [239:35:55<154:06:34,  5.07s/it]
 51%|█████▏    | 115637/225000 [239:36:01<157:15:15,  5.18s/it]
 51%|█████▏    | 115638/225000 [239:36:08<176:22:00,  5.81s/it]
 51%|█████▏    | 115639/225000 [239:36:11<155:45:12,  5.13s/it]
 51%|█████▏    | 115640/225000 [239:36:15<141:43:12,  4.67s/it]
 51%|█████▏    | 115641/225000 [239:36:26<196:19:38,  6.46s/it]
 51%|█████▏    | 115642/225000 [239:36:29<167:02:12,  5.50s/it]
 51%|█████▏    | 115643/225000 [239:36:33<155:20:28,  5.11s/it]
 51%|█████▏    | 115644/225000 [239:36:41<177:43:46,  5.85s/it]
 51%|█████▏    | 115645/225000 [239:36:44<155:14:11,  5.11s/it]
 51%|█████▏    | 115646/225000 [239:36:52<178:41:41,  5.88s/it]
 51%|█████▏    | 115647/225000 [239:36:55<150:59:05,  4.97s/it]
 51%|█████▏    | 115648/225000 [239:37:00<152:13:23,  5.01s/it]
 51%|█████▏    | 115649/225000 [239:37:04<148:18:05,  4.88s/it]
 51%|█████▏    | 115650/225000 [239:37:09<145:08:49,  4.78s/it]
                                                               
{'loss': '0.897', 'grad_norm': '1.093', 'learning_rate': '2.57e-05', 'epoch': '0.514', 'num_input_tokens_seen': 400120384, 'train_runtime': '8.626e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115650/225000 [239:37:09<145:08:49,  4.78s/it]
 51%|█████▏    | 115651/225000 [239:37:13<144:07:33,  4.74s/it]
 51%|█████▏    | 115652/225000 [239:37:18<141:41:20,  4.66s/it]
 51%|█████▏    | 115653/225000 [239:37:24<151:42:32,  4.99s/it]
 51%|█████▏    | 115654/225000 [239:37:28<149:27:14,  4.92s/it]
 51%|█████▏    | 115655/225000 [239:37:33<149:55:34,  4.94s/it]
 51%|█████▏    | 115656/225000 [239:37:38<147:24:21,  4.85s/it]
 51%|█████▏    | 115657/225000 [239:37:43<147:42:19,  4.86s/it]
 51%|█████▏    | 115658/225000 [239:37:54<203:57:10,  6.71s/it]
 51%|█████▏    | 115659/225000 [239:37:58<175:40:57,  5.78s/it]
 51%|█████▏    | 115660/225000 [239:38:00<146:36:38,  4.83s/it]
 51%|█████▏    | 115661/225000 [239:38:06<151:19:05,  4.98s/it]
 51%|█████▏    | 115662/225000 [239:38:14<182:50:29,  6.02s/it]
 51%|█████▏    | 115663/225000 [239:38:18<166:19:04,  5.48s/it]
 51%|█████▏    | 115664/225000 [239:38:24<171:57:46,  5.66s/it]
 51%|█████▏    | 115665/225000 [239:38:32<187:41:05,  6.18s/it]
 51%|█████▏    | 115666/225000 [239:38:35<162:44:10,  5.36s/it]
 51%|█████▏    | 115667/225000 [239:38:42<178:22:59,  5.87s/it]
 51%|█████▏    | 115668/225000 [239:38:50<191:29:39,  6.31s/it]
 51%|█████▏    | 115669/225000 [239:38:59<220:05:07,  7.25s/it]
 51%|█████▏    | 115670/225000 [239:39:02<183:53:59,  6.06s/it]
 51%|█████▏    | 115671/225000 [239:39:09<193:52:35,  6.38s/it]
 51%|█████▏    | 115672/225000 [239:39:13<168:03:36,  5.53s/it]
 51%|█████▏    | 115673/225000 [239:39:17<151:19:42,  4.98s/it]
 51%|█████▏    | 115674/225000 [239:39:25<182:35:49,  6.01s/it]
 51%|█████▏    | 115675/225000 [239:39:29<161:21:56,  5.31s/it]
 51%|█████▏    | 115676/225000 [239:39:32<142:06:35,  4.68s/it]
 51%|█████▏    | 115677/225000 [239:39:37<141:43:50,  4.67s/it]
 51%|█████▏    | 115678/225000 [239:39:47<194:53:02,  6.42s/it]
 51%|█████▏    | 115679/225000 [239:39:50<158:59:26,  5.24s/it]
 51%|█████▏    | 115680/225000 [239:39:57<175:54:57,  5.79s/it]
 51%|█████▏    | 115681/225000 [239:40:00<153:07:01,  5.04s/it]
 51%|█████▏    | 115682/225000 [239:40:04<139:36:57,  4.60s/it]
 51%|█████▏    | 115683/225000 [239:40:08<142:11:13,  4.68s/it]
 51%|█████▏    | 115684/225000 [239:40:13<145:56:15,  4.81s/it]
 51%|█████▏    | 115685/225000 [239:40:24<194:36:57,  6.41s/it]
 51%|█████▏    | 115686/225000 [239:40:28<174:07:51,  5.73s/it]
 51%|█████▏    | 115687/225000 [239:40:33<171:07:57,  5.64s/it]
 51%|█████▏    | 115688/225000 [239:40:38<162:34:14,  5.35s/it]
 51%|█████▏    | 115689/225000 [239:40:43<158:09:02,  5.21s/it]
 51%|█████▏    | 115690/225000 [239:40:47<146:23:24,  4.82s/it]
 51%|█████▏    | 115691/225000 [239:40:51<142:41:01,  4.70s/it]
 51%|█████▏    | 115692/225000 [239:40:56<144:04:28,  4.75s/it]
 51%|█████▏    | 115693/225000 [239:41:01<143:14:11,  4.72s/it]
 51%|█████▏    | 115694/225000 [239:41:05<143:27:16,  4.72s/it]
 51%|█████▏    | 115695/225000 [239:41:16<200:37:24,  6.61s/it]
 51%|█████▏    | 115696/225000 [239:41:20<177:02:44,  5.83s/it]
 51%|█████▏    | 115697/225000 [239:41:28<192:56:51,  6.35s/it]
 51%|█████▏    | 115698/225000 [239:41:32<171:37:55,  5.65s/it]
 51%|█████▏    | 115699/225000 [239:41:36<153:36:00,  5.06s/it]
 51%|█████▏    | 115700/225000 [239:41:43<170:40:03,  5.62s/it]
                                                               
{'loss': '0.8977', 'grad_norm': '1.107', 'learning_rate': '2.568e-05', 'epoch': '0.5142', 'num_input_tokens_seen': 400240800, 'train_runtime': '8.629e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115700/225000 [239:41:43<170:40:03,  5.62s/it]
 51%|█████▏    | 115701/225000 [239:41:52<202:35:31,  6.67s/it]
 51%|█████▏    | 115702/225000 [239:41:57<188:42:05,  6.22s/it]
 51%|█████▏    | 115703/225000 [239:42:00<161:23:18,  5.32s/it]
 51%|█████▏    | 115704/225000 [239:42:03<144:19:40,  4.75s/it]
 51%|█████▏    | 115705/225000 [239:42:09<150:49:17,  4.97s/it]
 51%|█████▏    | 115706/225000 [239:42:18<191:23:01,  6.30s/it]
 51%|█████▏    | 115707/225000 [239:42:21<158:39:56,  5.23s/it]
 51%|█████▏    | 115708/225000 [239:42:29<180:06:11,  5.93s/it]
 51%|█████▏    | 115709/225000 [239:42:33<166:22:42,  5.48s/it]
 51%|█████▏    | 115710/225000 [239:42:38<159:55:19,  5.27s/it]
 51%|█████▏    | 115711/225000 [239:42:42<153:13:01,  5.05s/it]
 51%|█████▏    | 115712/225000 [239:42:46<140:33:01,  4.63s/it]
 51%|█████▏    | 115713/225000 [239:42:50<137:55:17,  4.54s/it]
 51%|█████▏    | 115714/225000 [239:42:55<134:43:35,  4.44s/it]
 51%|█████▏    | 115715/225000 [239:43:00<139:02:50,  4.58s/it]
 51%|█████▏    | 115716/225000 [239:43:07<166:33:54,  5.49s/it]
 51%|█████▏    | 115717/225000 [239:43:15<192:55:43,  6.36s/it]
 51%|█████▏    | 115718/225000 [239:43:18<161:39:17,  5.33s/it]
 51%|█████▏    | 115719/225000 [239:43:21<140:10:25,  4.62s/it]
 51%|█████▏    | 115720/225000 [239:43:30<175:44:56,  5.79s/it]
 51%|█████▏    | 115721/225000 [239:43:35<166:38:48,  5.49s/it]
 51%|█████▏    | 115722/225000 [239:43:39<155:17:20,  5.12s/it]
 51%|█████▏    | 115723/225000 [239:43:44<151:28:12,  4.99s/it]
 51%|█████▏    | 115724/225000 [239:43:48<149:50:09,  4.94s/it]
 51%|█████▏    | 115725/225000 [239:43:53<144:46:35,  4.77s/it]
 51%|█████▏    | 115726/225000 [239:43:57<143:29:39,  4.73s/it]
 51%|█████▏    | 115727/225000 [239:44:02<137:23:42,  4.53s/it]
 51%|█████▏    | 115728/225000 [239:44:06<133:08:13,  4.39s/it]
 51%|█████▏    | 115729/225000 [239:44:10<133:12:45,  4.39s/it]
 51%|█████▏    | 115730/225000 [239:44:14<134:17:18,  4.42s/it]
 51%|█████▏    | 115731/225000 [239:44:19<139:20:48,  4.59s/it]
 51%|█████▏    | 115732/225000 [239:44:32<213:03:58,  7.02s/it]
 51%|█████▏    | 115733/225000 [239:44:38<203:38:38,  6.71s/it]
 51%|█████▏    | 115734/225000 [239:44:42<174:34:59,  5.75s/it]
 51%|█████▏    | 115735/225000 [239:44:46<165:04:05,  5.44s/it]
 51%|█████▏    | 115736/225000 [239:44:56<202:53:16,  6.68s/it]
 51%|█████▏    | 115737/225000 [239:44:59<169:30:31,  5.58s/it]
 51%|█████▏    | 115738/225000 [239:45:03<151:05:52,  4.98s/it]
 51%|█████▏    | 115739/225000 [239:45:12<188:42:22,  6.22s/it]
 51%|█████▏    | 115740/225000 [239:45:15<162:24:56,  5.35s/it]
 51%|█████▏    | 115741/225000 [239:45:18<144:24:16,  4.76s/it]
 51%|█████▏    | 115742/225000 [239:45:22<131:43:34,  4.34s/it]
 51%|█████▏    | 115743/225000 [239:45:26<128:19:07,  4.23s/it]
 51%|█████▏    | 115744/225000 [239:45:30<127:41:26,  4.21s/it]
 51%|█████▏    | 115745/225000 [239:45:42<200:23:53,  6.60s/it]
 51%|█████▏    | 115746/225000 [239:45:46<180:37:57,  5.95s/it]
 51%|█████▏    | 115747/225000 [239:45:49<153:20:59,  5.05s/it]
 51%|█████▏    | 115748/225000 [239:45:54<150:02:44,  4.94s/it]
 51%|█████▏    | 115749/225000 [239:45:59<147:25:14,  4.86s/it]
 51%|█████▏    | 115750/225000 [239:46:04<147:14:22,  4.85s/it]
                                                               
{'loss': '0.9229', 'grad_norm': '1.254', 'learning_rate': '2.566e-05', 'epoch': '0.5144', 'num_input_tokens_seen': 400356216, 'train_runtime': '8.632e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115750/225000 [239:46:04<147:14:22,  4.85s/it]
 51%|█████▏    | 115751/225000 [239:46:08<144:01:50,  4.75s/it]
 51%|█████▏    | 115752/225000 [239:46:13<141:15:49,  4.65s/it]
 51%|█████▏    | 115753/225000 [239:46:24<199:02:59,  6.56s/it]
 51%|█████▏    | 115754/225000 [239:46:30<196:26:54,  6.47s/it]
 51%|█████▏    | 115755/225000 [239:46:33<168:32:11,  5.55s/it]
 51%|█████▏    | 115756/225000 [239:46:40<178:38:18,  5.89s/it]
 51%|█████▏    | 115757/225000 [239:46:44<164:23:00,  5.42s/it]
 51%|█████▏    | 115758/225000 [239:46:49<155:10:50,  5.11s/it]
 51%|█████▏    | 115759/225000 [239:46:53<147:33:03,  4.86s/it]
 51%|█████▏    | 115760/225000 [239:46:57<142:10:22,  4.69s/it]
 51%|█████▏    | 115761/225000 [239:47:06<178:00:56,  5.87s/it]
 51%|█████▏    | 115762/225000 [239:47:18<231:36:36,  7.63s/it]
 51%|█████▏    | 115763/225000 [239:47:23<211:03:59,  6.96s/it]
 51%|█████▏    | 115764/225000 [239:47:30<211:04:33,  6.96s/it]
 51%|█████▏    | 115765/225000 [239:47:36<201:11:49,  6.63s/it]
 51%|█████▏    | 115766/225000 [239:47:42<200:15:06,  6.60s/it]
 51%|█████▏    | 115767/225000 [239:47:47<182:18:57,  6.01s/it]
 51%|█████▏    | 115768/225000 [239:47:53<184:30:59,  6.08s/it]
 51%|█████▏    | 115769/225000 [239:47:56<159:42:46,  5.26s/it]
 51%|█████▏    | 115770/225000 [239:48:00<139:38:03,  4.60s/it]
 51%|█████▏    | 115771/225000 [239:48:04<137:46:07,  4.54s/it]
 51%|█████▏    | 115772/225000 [239:48:13<178:21:57,  5.88s/it]
 51%|█████▏    | 115773/225000 [239:48:17<163:09:20,  5.38s/it]
 51%|█████▏    | 115774/225000 [239:48:21<152:56:43,  5.04s/it]
 51%|█████▏    | 115775/225000 [239:48:25<144:04:29,  4.75s/it]
 51%|█████▏    | 115776/225000 [239:48:29<131:21:55,  4.33s/it]
 51%|█████▏    | 115777/225000 [239:48:35<144:53:01,  4.78s/it]
 51%|█████▏    | 115778/225000 [239:48:47<215:42:45,  7.11s/it]
 51%|█████▏    | 115779/225000 [239:48:52<193:52:41,  6.39s/it]
 51%|█████▏    | 115780/225000 [239:48:57<183:07:54,  6.04s/it]
 51%|█████▏    | 115781/225000 [239:49:02<169:33:48,  5.59s/it]
 51%|█████▏    | 115782/225000 [239:49:08<176:18:02,  5.81s/it]
 51%|█████▏    | 115783/225000 [239:49:14<175:59:19,  5.80s/it]
 51%|█████▏    | 115784/225000 [239:49:20<177:32:46,  5.85s/it]
 51%|█████▏    | 115785/225000 [239:49:26<177:28:27,  5.85s/it]
 51%|█████▏    | 115786/225000 [239:49:30<164:54:38,  5.44s/it]
 51%|█████▏    | 115787/225000 [239:49:36<168:46:58,  5.56s/it]
 51%|█████▏    | 115788/225000 [239:49:40<154:16:00,  5.09s/it]
 51%|█████▏    | 115789/225000 [239:49:44<142:35:29,  4.70s/it]
 51%|█████▏    | 115790/225000 [239:49:48<138:41:42,  4.57s/it]
 51%|█████▏    | 115791/225000 [239:49:53<140:02:27,  4.62s/it]
 51%|█████▏    | 115792/225000 [239:49:57<136:13:40,  4.49s/it]
 51%|█████▏    | 115793/225000 [239:50:01<135:51:00,  4.48s/it]
 51%|█████▏    | 115794/225000 [239:50:06<133:09:19,  4.39s/it]
 51%|█████▏    | 115795/225000 [239:50:12<150:46:49,  4.97s/it]
 51%|█████▏    | 115796/225000 [239:50:24<212:50:24,  7.02s/it]
 51%|█████▏    | 115797/225000 [239:50:29<196:13:20,  6.47s/it]
 51%|█████▏    | 115798/225000 [239:50:33<172:23:30,  5.68s/it]
 51%|█████▏    | 115799/225000 [239:50:36<147:40:55,  4.87s/it]
 51%|█████▏    | 115800/225000 [239:50:39<137:33:47,  4.54s/it]
                                                               
{'loss': '0.8983', 'grad_norm': '1.326', 'learning_rate': '2.564e-05', 'epoch': '0.5147', 'num_input_tokens_seen': 400474192, 'train_runtime': '8.634e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115800/225000 [239:50:39<137:33:47,  4.54s/it]
 51%|█████▏    | 115801/225000 [239:50:51<197:20:34,  6.51s/it]
 51%|█████▏    | 115802/225000 [239:50:54<170:15:17,  5.61s/it]
 51%|█████▏    | 115803/225000 [239:50:58<154:05:10,  5.08s/it]
 51%|█████▏    | 115804/225000 [239:51:03<158:09:04,  5.21s/it]
 51%|█████▏    | 115805/225000 [239:51:13<200:55:35,  6.62s/it]
 51%|█████▏    | 115806/225000 [239:51:17<178:22:08,  5.88s/it]
 51%|█████▏    | 115807/225000 [239:51:22<163:01:31,  5.37s/it]
 51%|█████▏    | 115808/225000 [239:51:26<153:58:54,  5.08s/it]
 51%|█████▏    | 115809/225000 [239:51:36<197:58:25,  6.53s/it]
 51%|█████▏    | 115810/225000 [239:51:39<168:52:26,  5.57s/it]
 51%|█████▏    | 115811/225000 [239:51:46<182:35:22,  6.02s/it]
 51%|█████▏    | 115812/225000 [239:51:50<159:18:11,  5.25s/it]
 51%|█████▏    | 115813/225000 [239:51:54<145:49:09,  4.81s/it]
 51%|█████▏    | 115814/225000 [239:52:01<167:23:32,  5.52s/it]
 51%|█████▏    | 115815/225000 [239:52:08<181:32:16,  5.99s/it]
 51%|█████▏    | 115816/225000 [239:52:12<164:32:39,  5.43s/it]
 51%|█████▏    | 115817/225000 [239:52:16<151:51:37,  5.01s/it]
 51%|█████▏    | 115818/225000 [239:52:20<142:18:51,  4.69s/it]
 51%|█████▏    | 115819/225000 [239:52:24<139:38:22,  4.60s/it]
 51%|█████▏    | 115820/225000 [239:52:29<138:13:17,  4.56s/it]
 51%|█████▏    | 115821/225000 [239:52:33<135:28:41,  4.47s/it]
 51%|█████▏    | 115822/225000 [239:52:40<160:04:16,  5.28s/it]
 51%|█████▏    | 115823/225000 [239:52:49<195:30:12,  6.45s/it]
 51%|█████▏    | 115824/225000 [239:52:52<164:55:34,  5.44s/it]
 51%|█████▏    | 115825/225000 [239:52:55<141:25:59,  4.66s/it]
 51%|█████▏    | 115826/225000 [239:53:02<161:21:17,  5.32s/it]
 51%|█████▏    | 115827/225000 [239:53:06<150:31:32,  4.96s/it]
 51%|█████▏    | 115828/225000 [239:53:11<146:55:17,  4.84s/it]
 51%|█████▏    | 115829/225000 [239:53:16<149:12:45,  4.92s/it]
 51%|█████▏    | 115830/225000 [239:53:21<149:25:51,  4.93s/it]
 51%|█████▏    | 115831/225000 [239:53:26<149:11:36,  4.92s/it]
 51%|█████▏    | 115832/225000 [239:53:34<176:44:38,  5.83s/it]
 51%|█████▏    | 115833/225000 [239:53:38<163:17:18,  5.38s/it]
 51%|█████▏    | 115834/225000 [239:53:42<153:12:18,  5.05s/it]
 51%|█████▏    | 115835/225000 [239:53:55<219:06:45,  7.23s/it]
 51%|█████▏    | 115836/225000 [239:53:59<190:25:36,  6.28s/it]
 51%|█████▏    | 115837/225000 [239:54:03<174:33:56,  5.76s/it]
 51%|█████▏    | 115838/225000 [239:54:08<164:42:18,  5.43s/it]
 51%|█████▏    | 115839/225000 [239:54:13<156:57:02,  5.18s/it]
 51%|█████▏    | 115840/225000 [239:54:17<149:54:03,  4.94s/it]
 51%|█████▏    | 115841/225000 [239:54:24<169:47:30,  5.60s/it]
 51%|█████▏    | 115842/225000 [239:54:29<165:16:29,  5.45s/it]
 51%|█████▏    | 115843/225000 [239:54:34<159:43:50,  5.27s/it]
 51%|█████▏    | 115844/225000 [239:54:39<153:49:37,  5.07s/it]
 51%|█████▏    | 115845/225000 [239:54:43<150:23:53,  4.96s/it]
 51%|█████▏    | 115846/225000 [239:54:56<216:30:09,  7.14s/it]
 51%|█████▏    | 115847/225000 [239:54:59<183:46:07,  6.06s/it]
 51%|█████▏    | 115848/225000 [239:55:04<173:38:35,  5.73s/it]
 51%|█████▏    | 115849/225000 [239:55:11<185:30:29,  6.12s/it]
 51%|█████▏    | 115850/225000 [239:55:16<174:36:48,  5.76s/it]
                                                               
{'loss': '0.8961', 'grad_norm': '1.034', 'learning_rate': '2.562e-05', 'epoch': '0.5149', 'num_input_tokens_seen': 400591336, 'train_runtime': '8.637e+05', 'train_tokens_per_second': '463.8'}

 51%|█████▏    | 115850/225000 [239:55:16<174:36:48,  5.76s/it]
 51%|█████▏    | 115851/225000 [239:55:20<162:49:03,  5.37s/it]
 51%|█████▏    | 115852/225000 [239:55:25<154:10:35,  5.09s/it]
 51%|█████▏    | 115853/225000 [239:55:37<215:53:15,  7.12s/it]
 51%|█████▏    | 115854/225000 [239:55:40<180:39:22,  5.96s/it]
 51%|█████▏    | 115855/225000 [239:55:44<164:14:45,  5.42s/it]
 51%|█████▏    | 115856/225000 [239:55:48<150:14:49,  4.96s/it]
 51%|█████▏    | 115857/225000 [239:55:53<148:21:45,  4.89s/it]
 51%|█████▏    | 115858/225000 [239:55:57<140:34:17,  4.64s/it]
 51%|█████▏    | 115859/225000 [239:56:01<136:43:55,  4.51s/it]
 51%|█████▏    | 115860/225000 [239:56:05<134:20:51,  4.43s/it]
 51%|█████▏    | 115861/225000 [239:56:10<134:13:09,  4.43s/it]
 51%|█████▏    | 115862/225000 [239:56:14<130:26:16,  4.30s/it]
 51%|█████▏    | 115863/225000 [239:56:18<133:31:06,  4.40s/it]
 51%|█████▏    | 115864/225000 [239:56:22<127:44:51,  4.21s/it]
 51%|█████▏    | 115865/225000 [239:56:27<132:20:29,  4.37s/it]
 51%|█████▏    | 115866/225000 [239:56:32<138:49:37,  4.58s/it]
 51%|█████▏    | 115867/225000 [239:56:39<161:46:50,  5.34s/it]
 51%|█████▏    | 115868/225000 [239:56:51<221:26:51,  7.31s/it]
 51%|█████▏    | 115869/225000 [239:56:55<189:49:13,  6.26s/it]
 51%|█████▏    | 115870/225000 [239:56:59<172:27:08,  5.69s/it]
 51%|█████▏    | 115871/225000 [239:57:04<161:33:54,  5.33s/it]
 51%|█████▏    | 115872/225000 [239:57:07<148:07:34,  4.89s/it]
 51%|█████▏    | 115873/225000 [239:57:11<139:04:26,  4.59s/it]
 51%|█████▏    | 115874/225000 [239:57:16<137:31:25,  4.54s/it]
 52%|█████▏    | 115875/225000 [239:57:24<170:46:18,  5.63s/it]
 52%|█████▏    | 115876/225000 [239:57:33<197:40:21,  6.52s/it]
 52%|█████▏    | 115877/225000 [239:57:36<165:20:33,  5.45s/it]
 52%|█████▏    | 115878/225000 [239:57:44<191:43:10,  6.32s/it]
 52%|█████▏    | 115879/225000 [239:57:47<164:35:11,  5.43s/it]
 52%|█████▏    | 115880/225000 [239:57:50<140:57:13,  4.65s/it]
 52%|█████▏    | 115881/225000 [239:57:59<178:02:06,  5.87s/it]
 52%|█████▏    | 115882/225000 [239:58:02<155:31:29,  5.13s/it]
 52%|█████▏    | 115883/225000 [239:58:06<139:12:32,  4.59s/it]
 52%|█████▏    | 115884/225000 [239:58:09<132:23:53,  4.37s/it]
 52%|█████▏    | 115885/225000 [239:58:20<188:37:40,  6.22s/it]
 52%|█████▏    | 115886/225000 [239:58:24<164:49:44,  5.44s/it]
 52%|█████▏    | 115887/225000 [239:58:27<150:22:46,  4.96s/it]
 52%|█████▏    | 115888/225000 [239:58:32<142:51:41,  4.71s/it]
 52%|█████▏    | 115889/225000 [239:58:35<135:48:20,  4.48s/it]
 52%|█████▏    | 115890/225000 [239:58:40<132:35:11,  4.37s/it]
 52%|█████▏    | 115891/225000 [239:58:49<182:16:47,  6.01s/it]
 52%|█████▏    | 115892/225000 [239:58:55<177:56:36,  5.87s/it]
 52%|█████▏    | 115893/225000 [239:58:58<154:00:19,  5.08s/it]
 52%|█████▏    | 115894/225000 [239:59:05<170:12:34,  5.62s/it]
 52%|█████▏    | 115895/225000 [239:59:13<194:12:22,  6.41s/it]
 52%|█████▏    | 115896/225000 [239:59:19<191:05:56,  6.31s/it]
 52%|█████▏    | 115897/225000 [239:59:26<192:40:15,  6.36s/it]
 52%|█████▏    | 115898/225000 [239:59:36<228:11:43,  7.53s/it]
 52%|█████▏    | 115899/225000 [239:59:39<190:23:58,  6.28s/it]
 52%|█████▏    | 115900/225000 [239:59:43<163:51:22,  5.41s/it]
                                                               
{'loss': '0.9095', 'grad_norm': '1.172', 'learning_rate': '2.56e-05', 'epoch': '0.5151', 'num_input_tokens_seen': 400707032, 'train_runtime': '8.64e+05', 'train_tokens_per_second': '463.8'}

 52%|█████▏    | 115900/225000 [239:59:43<163:51:22,  5.41s/it]
 52%|█████▏    | 115901/225000 [239:59:52<196:56:27,  6.50s/it]
 52%|█████▏    | 115902/225000 [239:59:55<165:31:48,  5.46s/it]
 52%|█████▏    | 115903/225000 [239:59:59<154:35:17,  5.10s/it]
 52%|█████▏    | 115904/225000 [240:00:05<162:16:54,  5.36s/it]
 52%|█████▏    | 115905/225000 [240:00:08<142:22:14,  4.70s/it]
 52%|█████▏    | 115906/225000 [240:00:16<165:57:38,  5.48s/it]
 52%|█████▏    | 115907/225000 [240:00:19<146:08:59,  4.82s/it]
 52%|█████▏    | 115908/225000 [240:00:27<174:38:25,  5.76s/it]
 52%|█████▏    | 115909/225000 [240:00:30<153:04:16,  5.05s/it]
 52%|█████▏    | 115910/225000 [240:00:34<143:32:01,  4.74s/it]
 52%|█████▏    | 115911/225000 [240:00:40<156:01:14,  5.15s/it]
 52%|█████▏    | 115912/225000 [240:00:51<205:31:11,  6.78s/it]
 52%|█████▏    | 115913/225000 [240:00:54<168:36:55,  5.56s/it]
 52%|█████▏    | 115914/225000 [240:00:58<156:59:58,  5.18s/it]
 52%|█████▏    | 115915/225000 [240:01:04<161:13:06,  5.32s/it]
 52%|█████▏    | 115916/225000 [240:01:16<222:56:31,  7.36s/it]
 52%|█████▏    | 115917/225000 [240:01:20<199:04:49,  6.57s/it]
 52%|█████▏    | 115918/225000 [240:01:25<183:13:16,  6.05s/it]
 52%|█████▏    | 115919/225000 [240:01:30<169:39:05,  5.60s/it]
 52%|█████▏    | 115920/225000 [240:01:34<161:04:16,  5.32s/it]
 52%|█████▏    | 115921/225000 [240:01:40<159:09:15,  5.25s/it]
 52%|█████▏    | 115922/225000 [240:01:45<156:17:02,  5.16s/it]
 52%|█████▏    | 115923/225000 [240:01:50<160:29:44,  5.30s/it]
 52%|█████▏    | 115924/225000 [240:01:55<153:38:33,  5.07s/it]
 52%|█████▏    | 115925/225000 [240:02:02<176:22:03,  5.82s/it]
 52%|█████▏    | 115926/225000 [240:02:08<179:27:39,  5.92s/it]
 52%|█████▏    | 115927/225000 [240:02:12<161:57:47,  5.35s/it]
 52%|█████▏    | 115928/225000 [240:02:23<210:24:32,  6.94s/it]
 52%|█████▏    | 115929/225000 [240:02:26<175:04:19,  5.78s/it]
 52%|█████▏    | 115930/225000 [240:02:32<178:07:53,  5.88s/it]
 52%|█████▏    | 115931/225000 [240:02:38<174:01:36,  5.74s/it]
 52%|█████▏    | 115932/225000 [240:02:41<155:11:20,  5.12s/it]
 52%|█████▏    | 115933/225000 [240:02:51<195:17:17,  6.45s/it]
 52%|█████▏    | 115934/225000 [240:02:54<163:22:42,  5.39s/it]
 52%|█████▏    | 115935/225000 [240:02:57<144:23:08,  4.77s/it]
 52%|█████▏    | 115936/225000 [240:03:01<140:00:34,  4.62s/it]
 52%|█████▏    | 115937/225000 [240:03:06<136:39:18,  4.51s/it]
 52%|█████▏    | 115938/225000 [240:03:10<137:56:02,  4.55s/it]
 52%|█████▏    | 115939/225000 [240:03:17<153:39:37,  5.07s/it]
 52%|█████▏    | 115940/225000 [240:03:27<199:32:26,  6.59s/it]
 52%|█████▏    | 115941/225000 [240:03:30<167:29:37,  5.53s/it]
 52%|█████▏    | 115942/225000 [240:03:33<141:45:30,  4.68s/it]
 52%|█████▏    | 115943/225000 [240:03:41<173:22:42,  5.72s/it]
 52%|█████▏    | 115944/225000 [240:03:44<154:25:40,  5.10s/it]
 52%|█████▏    | 115945/225000 [240:03:49<153:46:09,  5.08s/it]
 52%|█████▏    | 115946/225000 [240:03:53<144:41:13,  4.78s/it]
 52%|█████▏    | 115947/225000 [240:03:59<153:58:32,  5.08s/it]
 52%|█████▏    | 115948/225000 [240:04:09<200:54:12,  6.63s/it]
 52%|█████▏    | 115949/225000 [240:04:14<185:27:02,  6.12s/it]
 52%|█████▏    | 115950/225000 [240:04:18<160:56:04,  5.31s/it]
                                                               
{'loss': '0.8944', 'grad_norm': '1.206', 'learning_rate': '2.559e-05', 'epoch': '0.5153', 'num_input_tokens_seen': 400827024, 'train_runtime': '8.643e+05', 'train_tokens_per_second': '463.8'}

 52%|█████▏    | 115950/225000 [240:04:18<160:56:04,  5.31s/it]
 52%|█████▏    | 115951/225000 [240:04:21<144:29:21,  4.77s/it]
 52%|█████▏    | 115952/225000 [240:04:34<218:16:33,  7.21s/it]
 52%|█████▏    | 115953/225000 [240:04:40<202:45:50,  6.69s/it]
 52%|█████▏    | 115954/225000 [240:04:44<176:37:57,  5.83s/it]
 52%|█████▏    | 115955/225000 [240:04:48<166:42:44,  5.50s/it]
 52%|█████▏    | 115956/225000 [240:04:53<162:25:03,  5.36s/it]
 52%|█████▏    | 115957/225000 [240:04:58<153:52:37,  5.08s/it]
 52%|█████▏    | 115958/225000 [240:05:10<220:08:32,  7.27s/it]
 52%|█████▏    | 115959/225000 [240:05:13<182:32:28,  6.03s/it]
 52%|█████▏    | 115960/225000 [240:05:16<154:46:12,  5.11s/it]
 52%|█████▏    | 115961/225000 [240:05:23<172:26:10,  5.69s/it]
 52%|█████▏    | 115962/225000 [240:05:28<168:25:49,  5.56s/it]
 52%|█████▏    | 115963/225000 [240:05:32<147:58:28,  4.89s/it]
 52%|█████▏    | 115964/225000 [240:05:44<211:27:46,  6.98s/it]
 52%|█████▏    | 115965/225000 [240:05:47<178:26:58,  5.89s/it]
 52%|█████▏    | 115966/225000 [240:05:54<188:08:33,  6.21s/it]
 52%|█████▏    | 115967/225000 [240:05:57<159:14:27,  5.26s/it]
 52%|█████▏    | 115968/225000 [240:06:00<137:46:58,  4.55s/it]
 52%|█████▏    | 115969/225000 [240:06:10<185:08:41,  6.11s/it]
 52%|█████▏    | 115970/225000 [240:06:13<160:45:49,  5.31s/it]
 52%|█████▏    | 115971/225000 [240:06:16<141:54:09,  4.69s/it]
 52%|█████▏    | 115972/225000 [240:06:20<132:16:44,  4.37s/it]
 52%|█████▏    | 115973/225000 [240:06:26<147:48:39,  4.88s/it]
 52%|█████▏    | 115974/225000 [240:06:33<168:15:27,  5.56s/it]
 52%|█████▏    | 115975/225000 [240:06:39<170:58:20,  5.65s/it]
 52%|█████▏    | 115976/225000 [240:06:45<172:58:47,  5.71s/it]
 52%|█████▏    | 115977/225000 [240:06:48<147:33:27,  4.87s/it]
 52%|█████▏    | 115978/225000 [240:06:52<139:19:12,  4.60s/it]
 52%|█████▏    | 115979/225000 [240:07:00<175:08:54,  5.78s/it]
 52%|█████▏    | 115980/225000 [240:07:04<152:59:10,  5.05s/it]
 52%|█████▏    | 115981/225000 [240:07:10<168:57:44,  5.58s/it]
 52%|█████▏    | 115982/225000 [240:07:17<177:18:52,  5.86s/it]
 52%|█████▏    | 115983/225000 [240:07:21<156:30:45,  5.17s/it]
 52%|█████▏    | 115984/225000 [240:07:25<147:20:22,  4.87s/it]
 52%|█████▏    | 115985/225000 [240:07:34<190:29:01,  6.29s/it]
 52%|█████▏    | 115986/225000 [240:07:37<161:24:25,  5.33s/it]
 52%|█████▏    | 115987/225000 [240:07:44<174:48:21,  5.77s/it]
 52%|█████▏    | 115988/225000 [240:07:50<171:38:05,  5.67s/it]
 52%|█████▏    | 115989/225000 [240:07:54<161:04:31,  5.32s/it]
 52%|█████▏    | 115990/225000 [240:07:59<156:10:32,  5.16s/it]
 52%|█████▏    | 115991/225000 [240:08:03<150:38:06,  4.97s/it]
 52%|█████▏    | 115992/225000 [240:08:07<141:03:34,  4.66s/it]
 52%|█████▏    | 115993/225000 [240:08:12<142:49:35,  4.72s/it]
 52%|█████▏    | 115994/225000 [240:08:17<139:18:56,  4.60s/it]
 52%|█████▏    | 115995/225000 [240:08:20<132:25:39,  4.37s/it]
 52%|█████▏    | 115996/225000 [240:08:31<186:04:26,  6.15s/it]
 52%|█████▏    | 115997/225000 [240:08:38<195:01:46,  6.44s/it]
 52%|█████▏    | 115998/225000 [240:08:44<196:57:16,  6.50s/it]
 52%|█████▏    | 115999/225000 [240:08:48<169:00:43,  5.58s/it]
 52%|█████▏    | 116000/225000 [240:08:52<155:10:35,  5.13s/it]
                                                               
{'loss': '0.8892', 'grad_norm': '1.25', 'learning_rate': '2.557e-05', 'epoch': '0.5156', 'num_input_tokens_seen': 400947080, 'train_runtime': '8.645e+05', 'train_tokens_per_second': '463.8'}

 52%|█████▏    | 116000/225000 [240:08:52<155:10:35,  5.13s/it]
 52%|█████▏    | 116001/225000 [240:08:56<148:20:53,  4.90s/it]
 52%|█████▏    | 116002/225000 [240:08:59<132:34:48,  4.38s/it]
 52%|█████▏    | 116003/225000 [240:09:08<166:09:06,  5.49s/it]
 52%|█████▏    | 116004/225000 [240:09:14<176:52:51,  5.84s/it]
 52%|█████▏    | 116005/225000 [240:09:18<157:28:34,  5.20s/it]
 52%|█████▏    | 116006/225000 [240:09:27<190:03:55,  6.28s/it]
 52%|█████▏    | 116007/225000 [240:09:30<160:03:50,  5.29s/it]
 52%|█████▏    | 116008/225000 [240:09:34<146:31:16,  4.84s/it]
 52%|█████▏    | 116009/225000 [240:09:37<137:39:20,  4.55s/it]
 52%|█████▏    | 116010/225000 [240:09:52<226:40:29,  7.49s/it]
 52%|█████▏    | 116011/225000 [240:09:58<217:24:26,  7.18s/it]
 52%|█████▏    | 116012/225000 [240:10:02<185:51:28,  6.14s/it]
 52%|█████▏    | 116013/225000 [240:10:10<201:36:23,  6.66s/it]
 52%|█████▏    | 116014/225000 [240:10:16<193:14:43,  6.38s/it]
 52%|█████▏    | 116015/225000 [240:10:19<163:37:29,  5.40s/it]
 52%|█████▏    | 116016/225000 [240:10:27<187:41:35,  6.20s/it]
 52%|█████▏    | 116017/225000 [240:10:32<176:00:43,  5.81s/it]
 52%|█████▏    | 116018/225000 [240:10:35<156:36:40,  5.17s/it]
 52%|█████▏    | 116019/225000 [240:10:40<153:23:04,  5.07s/it]
 52%|█████▏    | 116020/225000 [240:10:44<144:46:25,  4.78s/it]
 52%|█████▏    | 116021/225000 [240:10:49<143:44:03,  4.75s/it]
 52%|█████▏    | 116022/225000 [240:10:53<139:10:20,  4.60s/it]
 52%|█████▏    | 116023/225000 [240:11:01<166:22:12,  5.50s/it]
 52%|█████▏    | 116024/225000 [240:11:10<201:58:04,  6.67s/it]
 52%|█████▏    | 116025/225000 [240:11:13<170:37:37,  5.64s/it]
 52%|█████▏    | 116026/225000 [240:11:16<146:47:45,  4.85s/it]
 52%|█████▏    | 116027/225000 [240:11:24<170:05:55,  5.62s/it]
 52%|█████▏    | 116028/225000 [240:11:28<156:36:57,  5.17s/it]
 52%|█████▏    | 116029/225000 [240:11:31<138:59:48,  4.59s/it]
 52%|█████▏    | 116030/225000 [240:11:35<132:19:26,  4.37s/it]
 52%|█████▏    | 116031/225000 [240:11:43<168:04:39,  5.55s/it]
 52%|█████▏    | 116032/225000 [240:11:48<160:14:37,  5.29s/it]
 52%|█████▏    | 116033/225000 [240:11:52<148:47:13,  4.92s/it]
 52%|█████▏    | 116034/225000 [240:11:57<148:46:32,  4.92s/it]
 52%|█████▏    | 116035/225000 [240:12:01<144:43:58,  4.78s/it]
 52%|█████▏    | 116036/225000 [240:12:06<141:44:11,  4.68s/it]
 52%|█████▏    | 116037/225000 [240:12:16<189:18:00,  6.25s/it]
 52%|█████▏    | 116038/225000 [240:12:23<197:55:42,  6.54s/it]
 52%|█████▏    | 116039/225000 [240:12:29<196:49:48,  6.50s/it]
 52%|█████▏    | 116040/225000 [240:12:34<176:49:54,  5.84s/it]
 52%|█████▏    | 116041/225000 [240:12:38<163:58:22,  5.42s/it]
 52%|█████▏    | 116042/225000 [240:12:43<158:11:32,  5.23s/it]
 52%|█████▏    | 116043/225000 [240:12:48<152:54:26,  5.05s/it]
 52%|█████▏    | 116044/225000 [240:12:52<144:44:26,  4.78s/it]
 52%|█████▏    | 116045/225000 [240:13:00<172:48:36,  5.71s/it]
 52%|█████▏    | 116046/225000 [240:13:09<208:01:13,  6.87s/it]
 52%|█████▏    | 116047/225000 [240:13:13<176:37:48,  5.84s/it]
 52%|█████▏    | 116048/225000 [240:13:16<153:54:36,  5.09s/it]
 52%|█████▏    | 116049/225000 [240:13:24<179:37:00,  5.93s/it]
 52%|█████▏    | 116050/225000 [240:13:28<167:25:23,  5.53s/it]
                                                               
{'loss': '0.9001', 'grad_norm': '1.051', 'learning_rate': '2.555e-05', 'epoch': '0.5158', 'num_input_tokens_seen': 401066200, 'train_runtime': '8.648e+05', 'train_tokens_per_second': '463.8'}

 52%|█████▏    | 116050/225000 [240:13:28<167:25:23,  5.53s/it]
 52%|█████▏    | 116051/225000 [240:13:32<153:28:04,  5.07s/it]
 52%|█████▏    | 116052/225000 [240:13:37<146:54:38,  4.85s/it]
 52%|█████▏    | 116053/225000 [240:13:41<142:47:01,  4.72s/it]
 52%|█████▏    | 116054/225000 [240:13:52<196:31:12,  6.49s/it]
 52%|█████▏    | 116055/225000 [240:13:56<179:22:13,  5.93s/it]
 52%|█████▏    | 116056/225000 [240:14:00<157:09:12,  5.19s/it]
 52%|█████▏    | 116057/225000 [240:14:08<184:55:23,  6.11s/it]
 52%|█████▏    | 116058/225000 [240:14:13<174:33:53,  5.77s/it]
 52%|█████▏    | 116059/225000 [240:14:17<153:53:38,  5.09s/it]
 52%|█████▏    | 116060/225000 [240:14:20<138:45:42,  4.59s/it]
 52%|█████▏    | 116061/225000 [240:14:24<132:34:44,  4.38s/it]
 52%|█████▏    | 116062/225000 [240:14:31<156:29:29,  5.17s/it]
 52%|█████▏    | 116063/225000 [240:14:39<180:42:31,  5.97s/it]
 52%|█████▏    | 116064/225000 [240:14:43<164:14:59,  5.43s/it]
 52%|█████▏    | 116065/225000 [240:14:47<154:08:00,  5.09s/it]
 52%|█████▏    | 116066/225000 [240:14:51<141:27:38,  4.67s/it]
 52%|█████▏    | 116067/225000 [240:14:54<130:41:36,  4.32s/it]
 52%|█████▏    | 116068/225000 [240:14:58<122:25:11,  4.05s/it]
 52%|█████▏    | 116069/225000 [240:15:02<120:30:58,  3.98s/it]
 52%|█████▏    | 116070/225000 [240:15:06<123:53:40,  4.09s/it]
 52%|█████▏    | 116071/225000 [240:15:17<183:15:07,  6.06s/it]
 52%|█████▏    | 116072/225000 [240:15:22<175:32:03,  5.80s/it]
 52%|█████▏    | 116073/225000 [240:15:32<215:29:43,  7.12s/it]
 52%|█████▏    | 116074/225000 [240:15:35<180:55:00,  5.98s/it]
 52%|█████▏    | 116075/225000 [240:15:38<151:35:05,  5.01s/it]
 52%|█████▏    | 116076/225000 [240:15:47<182:51:29,  6.04s/it]
 52%|█████▏    | 116077/225000 [240:15:51<169:04:13,  5.59s/it]
 52%|█████▏    | 116078/225000 [240:15:58<182:37:27,  6.04s/it]
 52%|█████▏    | 116079/225000 [240:16:02<164:40:21,  5.44s/it]
 52%|█████▏    | 116080/225000 [240:16:11<197:04:04,  6.51s/it]
 52%|█████▏    | 116081/225000 [240:16:15<175:19:44,  5.79s/it]
 52%|█████▏    | 116082/225000 [240:16:19<158:06:05,  5.23s/it]
 52%|█████▏    | 116083/225000 [240:16:25<165:05:47,  5.46s/it]
 52%|█████▏    | 116084/225000 [240:16:33<186:56:09,  6.18s/it]
 52%|█████▏    | 116085/225000 [240:16:38<172:03:29,  5.69s/it]
 52%|█████▏    | 116086/225000 [240:16:42<161:58:37,  5.35s/it]
 52%|█████▏    | 116087/225000 [240:16:47<156:47:45,  5.18s/it]
 52%|█████▏    | 116088/225000 [240:16:57<199:16:12,  6.59s/it]
 52%|█████▏    | 116089/225000 [240:17:05<209:52:33,  6.94s/it]
 52%|█████▏    | 116090/225000 [240:17:09<184:45:19,  6.11s/it]
 52%|█████▏    | 116091/225000 [240:17:13<169:45:36,  5.61s/it]
 52%|█████▏    | 116092/225000 [240:17:17<152:09:52,  5.03s/it]
 52%|█████▏    | 116093/225000 [240:17:21<143:36:26,  4.75s/it]
 52%|█████▏    | 116094/225000 [240:17:25<139:42:56,  4.62s/it]
 52%|█████▏    | 116095/225000 [240:17:38<210:32:44,  6.96s/it]
 52%|█████▏    | 116096/225000 [240:17:42<184:43:07,  6.11s/it]
 52%|█████▏    | 116097/225000 [240:17:46<167:48:50,  5.55s/it]
 52%|█████▏    | 116098/225000 [240:17:51<164:58:09,  5.45s/it]
 52%|█████▏    | 116099/225000 [240:17:58<175:19:22,  5.80s/it]
 52%|█████▏    | 116100/225000 [240:18:02<155:35:44,  5.14s/it]
                                                               
{'loss': '0.8958', 'grad_norm': '1.057', 'learning_rate': '2.553e-05', 'epoch': '0.516', 'num_input_tokens_seen': 401186016, 'train_runtime': '8.651e+05', 'train_tokens_per_second': '463.8'}

 52%|█████▏    | 116100/225000 [240:18:02<155:35:44,  5.14s/it]
 52%|█████▏    | 116101/225000 [240:18:10<188:32:42,  6.23s/it]
 52%|█████▏    | 116102/225000 [240:18:17<190:39:13,  6.30s/it]
 52%|█████▏    | 116103/225000 [240:18:22<177:38:43,  5.87s/it]
 52%|█████▏    | 116104/225000 [240:18:29<190:20:36,  6.29s/it]
 52%|█████▏    | 116105/225000 [240:18:33<171:26:13,  5.67s/it]
 52%|█████▏    | 116106/225000 [240:18:37<155:23:13,  5.14s/it]
 52%|█████▏    | 116107/225000 [240:18:46<186:51:12,  6.18s/it]
 52%|█████▏    | 116108/225000 [240:18:51<176:48:50,  5.85s/it]
 52%|█████▏    | 116109/225000 [240:18:54<154:02:42,  5.09s/it]
 52%|█████▏    | 116110/225000 [240:18:58<141:09:53,  4.67s/it]
 52%|█████▏    | 116111/225000 [240:19:05<165:42:21,  5.48s/it]
 52%|█████▏    | 116112/225000 [240:19:11<172:56:23,  5.72s/it]
 52%|█████▏    | 116113/225000 [240:19:18<181:27:02,  6.00s/it]
 52%|█████▏    | 116114/225000 [240:19:21<157:42:19,  5.21s/it]
 52%|█████▏    | 116115/225000 [240:19:25<139:09:33,  4.60s/it]
 52%|█████▏    | 116116/225000 [240:19:34<178:52:27,  5.91s/it]
 52%|█████▏    | 116117/225000 [240:19:38<163:07:26,  5.39s/it]
 52%|█████▏    | 116118/225000 [240:19:44<172:46:44,  5.71s/it]
 52%|█████▏    | 116119/225000 [240:19:49<166:35:18,  5.51s/it]
 52%|█████▏    | 116120/225000 [240:19:52<144:28:38,  4.78s/it]
 52%|█████▏    | 116121/225000 [240:19:56<136:50:29,  4.52s/it]
 52%|█████▏    | 116122/225000 [240:20:05<176:36:28,  5.84s/it]
 52%|█████▏    | 116123/225000 [240:20:09<153:23:07,  5.07s/it]
 52%|█████▏    | 116124/225000 [240:20:12<139:32:10,  4.61s/it]
 52%|█████▏    | 116125/225000 [240:20:16<131:55:10,  4.36s/it]
 52%|█████▏    | 116126/225000 [240:20:20<126:38:01,  4.19s/it]
 52%|█████▏    | 116127/225000 [240:20:29<171:25:16,  5.67s/it]
 52%|█████▏    | 116128/225000 [240:20:36<188:57:38,  6.25s/it]
 52%|█████▏    | 116129/225000 [240:20:42<187:34:57,  6.20s/it]
 52%|█████▏    | 116130/225000 [240:20:45<157:17:57,  5.20s/it]
 52%|█████▏    | 116131/225000 [240:20:49<139:40:50,  4.62s/it]
 52%|█████▏    | 116132/225000 [240:20:56<163:49:42,  5.42s/it]
 52%|█████▏    | 116133/225000 [240:21:01<158:50:48,  5.25s/it]
 52%|█████▏    | 116134/225000 [240:21:04<139:51:17,  4.62s/it]
 52%|█████▏    | 116135/225000 [240:21:07<130:23:34,  4.31s/it]
 52%|█████▏    | 116136/225000 [240:21:17<177:17:01,  5.86s/it]
 52%|█████▏    | 116137/225000 [240:21:22<173:34:11,  5.74s/it]
 52%|█████▏    | 116138/225000 [240:21:25<146:51:39,  4.86s/it]
 52%|█████▏    | 116139/225000 [240:21:30<150:26:05,  4.97s/it]
 52%|█████▏    | 116140/225000 [240:21:37<165:40:19,  5.48s/it]
 52%|█████▏    | 116141/225000 [240:21:41<152:46:24,  5.05s/it]
 52%|█████▏    | 116142/225000 [240:21:48<170:17:34,  5.63s/it]
 52%|█████▏    | 116143/225000 [240:21:57<199:22:56,  6.59s/it]
 52%|█████▏    | 116144/225000 [240:22:06<218:50:38,  7.24s/it]
 52%|█████▏    | 116145/225000 [240:22:11<200:57:05,  6.65s/it]
 52%|█████▏    | 116146/225000 [240:22:15<179:26:02,  5.93s/it]
 52%|█████▏    | 116147/225000 [240:22:20<166:18:02,  5.50s/it]
 52%|█████▏    | 116148/225000 [240:22:24<157:03:42,  5.19s/it]
 52%|█████▏    | 116149/225000 [240:22:29<154:24:24,  5.11s/it]
 52%|█████▏    | 116150/225000 [240:22:33<146:49:33,  4.86s/it]
                                                               
{'loss': '0.8935', 'grad_norm': '1.29', 'learning_rate': '2.551e-05', 'epoch': '0.5162', 'num_input_tokens_seen': 401305016, 'train_runtime': '8.654e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116150/225000 [240:22:33<146:49:33,  4.86s/it]
 52%|█████▏    | 116151/225000 [240:22:38<145:39:33,  4.82s/it]
 52%|█████▏    | 116152/225000 [240:22:43<145:07:40,  4.80s/it]
 52%|█████▏    | 116153/225000 [240:22:48<144:47:19,  4.79s/it]
 52%|█████▏    | 116154/225000 [240:22:53<146:35:56,  4.85s/it]
 52%|█████▏    | 116155/225000 [240:22:58<147:43:39,  4.89s/it]
 52%|█████▏    | 116156/225000 [240:23:07<184:44:56,  6.11s/it]
 52%|█████▏    | 116157/225000 [240:23:10<164:48:21,  5.45s/it]
 52%|█████▏    | 116158/225000 [240:23:15<154:27:34,  5.11s/it]
 52%|█████▏    | 116159/225000 [240:23:19<148:26:40,  4.91s/it]
 52%|█████▏    | 116160/225000 [240:23:29<189:02:58,  6.25s/it]
 52%|█████▏    | 116161/225000 [240:23:33<174:58:56,  5.79s/it]
 52%|█████▏    | 116162/225000 [240:23:38<162:26:55,  5.37s/it]
 52%|█████▏    | 116163/225000 [240:23:43<158:27:07,  5.24s/it]
 52%|█████▏    | 116164/225000 [240:23:47<146:56:45,  4.86s/it]
 52%|█████▏    | 116165/225000 [240:23:51<139:47:13,  4.62s/it]
 52%|█████▏    | 116166/225000 [240:23:55<136:39:14,  4.52s/it]
 52%|█████▏    | 116167/225000 [240:24:04<181:58:03,  6.02s/it]
 52%|█████▏    | 116168/225000 [240:24:11<185:34:11,  6.14s/it]
 52%|█████▏    | 116169/225000 [240:24:15<168:09:07,  5.56s/it]
 52%|█████▏    | 116170/225000 [240:24:22<177:00:53,  5.86s/it]
 52%|█████▏    | 116171/225000 [240:24:25<157:44:16,  5.22s/it]
 52%|█████▏    | 116172/225000 [240:24:30<151:12:57,  5.00s/it]
 52%|█████▏    | 116173/225000 [240:24:34<144:39:42,  4.79s/it]
 52%|█████▏    | 116174/225000 [240:24:39<142:42:04,  4.72s/it]
 52%|█████▏    | 116175/225000 [240:24:47<176:04:59,  5.82s/it]
 52%|█████▏    | 116176/225000 [240:24:54<189:13:52,  6.26s/it]
 52%|█████▏    | 116177/225000 [240:24:58<161:17:09,  5.34s/it]
 52%|█████▏    | 116178/225000 [240:25:02<154:57:56,  5.13s/it]
 52%|█████▏    | 116179/225000 [240:25:06<144:27:26,  4.78s/it]
 52%|█████▏    | 116180/225000 [240:25:11<140:35:35,  4.65s/it]
 52%|█████▏    | 116181/225000 [240:25:15<134:41:22,  4.46s/it]
 52%|█████▏    | 116182/225000 [240:25:18<128:36:33,  4.25s/it]
 52%|█████▏    | 116183/225000 [240:25:24<142:38:46,  4.72s/it]
 52%|█████▏    | 116184/225000 [240:25:36<206:14:27,  6.82s/it]
 52%|█████▏    | 116185/225000 [240:25:40<181:37:52,  6.01s/it]
 52%|█████▏    | 116186/225000 [240:25:43<157:21:22,  5.21s/it]
 52%|█████▏    | 116187/225000 [240:25:47<146:10:14,  4.84s/it]
 52%|█████▏    | 116188/225000 [240:25:58<198:08:08,  6.56s/it]
 52%|█████▏    | 116189/225000 [240:26:04<190:15:30,  6.29s/it]
 52%|█████▏    | 116190/225000 [240:26:07<163:08:49,  5.40s/it]
 52%|█████▏    | 116191/225000 [240:26:11<150:50:58,  4.99s/it]
 52%|█████▏    | 116192/225000 [240:26:15<145:21:20,  4.81s/it]
 52%|█████▏    | 116193/225000 [240:26:19<137:54:08,  4.56s/it]
 52%|█████▏    | 116194/225000 [240:26:24<140:58:54,  4.66s/it]
 52%|█████▏    | 116195/225000 [240:26:29<142:03:13,  4.70s/it]
 52%|█████▏    | 116196/225000 [240:26:34<146:00:19,  4.83s/it]
 52%|█████▏    | 116197/225000 [240:26:46<206:39:25,  6.84s/it]
 52%|█████▏    | 116198/225000 [240:26:54<222:32:26,  7.36s/it]
 52%|█████▏    | 116199/225000 [240:27:00<211:45:16,  7.01s/it]
 52%|█████▏    | 116200/225000 [240:27:07<207:11:47,  6.86s/it]
                                                               
{'loss': '0.8988', 'grad_norm': '1.096', 'learning_rate': '2.549e-05', 'epoch': '0.5164', 'num_input_tokens_seen': 401423976, 'train_runtime': '8.656e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116200/225000 [240:27:07<207:11:47,  6.86s/it]
 52%|█████▏    | 116201/225000 [240:27:10<173:10:27,  5.73s/it]
 52%|█████▏    | 116202/225000 [240:27:14<162:15:02,  5.37s/it]
 52%|█████▏    | 116203/225000 [240:27:19<156:53:15,  5.19s/it]
 52%|█████▏    | 116204/225000 [240:27:23<140:38:42,  4.65s/it]
 52%|█████▏    | 116205/225000 [240:27:28<148:37:21,  4.92s/it]
 52%|█████▏    | 116206/225000 [240:27:33<146:48:34,  4.86s/it]
 52%|█████▏    | 116207/225000 [240:27:38<146:17:40,  4.84s/it]
 52%|█████▏    | 116208/225000 [240:27:42<144:46:55,  4.79s/it]
 52%|█████▏    | 116209/225000 [240:27:46<135:43:58,  4.49s/it]
 52%|█████▏    | 116210/225000 [240:27:51<142:56:55,  4.73s/it]
 52%|█████▏    | 116211/225000 [240:28:05<219:08:57,  7.25s/it]
 52%|█████▏    | 116212/225000 [240:28:09<192:49:03,  6.38s/it]
 52%|█████▏    | 116213/225000 [240:28:16<195:53:44,  6.48s/it]
 52%|█████▏    | 116214/225000 [240:28:19<165:42:25,  5.48s/it]
 52%|█████▏    | 116215/225000 [240:28:29<210:05:25,  6.95s/it]
 52%|█████▏    | 116216/225000 [240:28:32<174:16:03,  5.77s/it]
 52%|█████▏    | 116217/225000 [240:28:39<180:09:25,  5.96s/it]
 52%|█████▏    | 116218/225000 [240:28:42<154:32:56,  5.11s/it]
 52%|█████▏    | 116219/225000 [240:28:47<154:40:38,  5.12s/it]
 52%|█████▏    | 116220/225000 [240:28:53<159:18:07,  5.27s/it]
 52%|█████▏    | 116221/225000 [240:28:56<144:45:55,  4.79s/it]
 52%|█████▏    | 116222/225000 [240:28:59<128:49:07,  4.26s/it]
 52%|█████▏    | 116223/225000 [240:29:05<146:32:16,  4.85s/it]
 52%|█████▏    | 116224/225000 [240:29:16<200:14:00,  6.63s/it]
 52%|█████▏    | 116225/225000 [240:29:19<168:02:25,  5.56s/it]
 52%|█████▏    | 116226/225000 [240:29:28<199:43:13,  6.61s/it]
 52%|█████▏    | 116227/225000 [240:29:32<171:24:36,  5.67s/it]
 52%|█████▏    | 116228/225000 [240:29:36<160:20:35,  5.31s/it]
 52%|█████▏    | 116229/225000 [240:29:40<144:58:18,  4.80s/it]
 52%|█████▏    | 116230/225000 [240:29:44<139:40:56,  4.62s/it]
 52%|█████▏    | 116231/225000 [240:29:48<133:44:31,  4.43s/it]
 52%|█████▏    | 116232/225000 [240:29:58<183:04:38,  6.06s/it]
 52%|█████▏    | 116233/225000 [240:30:03<177:52:04,  5.89s/it]
 52%|█████▏    | 116234/225000 [240:30:07<158:05:47,  5.23s/it]
 52%|█████▏    | 116235/225000 [240:30:13<159:51:41,  5.29s/it]
 52%|█████▏    | 116236/225000 [240:30:18<165:27:01,  5.48s/it]
 52%|█████▏    | 116237/225000 [240:30:22<151:06:32,  5.00s/it]
 52%|█████▏    | 116238/225000 [240:30:27<148:39:22,  4.92s/it]
 52%|█████▏    | 116239/225000 [240:30:32<148:13:38,  4.91s/it]
 52%|█████▏    | 116240/225000 [240:30:36<140:16:25,  4.64s/it]
 52%|█████▏    | 116241/225000 [240:30:41<141:02:10,  4.67s/it]
 52%|█████▏    | 116242/225000 [240:30:49<171:51:35,  5.69s/it]
 52%|█████▏    | 116243/225000 [240:30:56<185:33:36,  6.14s/it]
 52%|█████▏    | 116244/225000 [240:31:00<167:51:01,  5.56s/it]
 52%|█████▏    | 116245/225000 [240:31:04<153:06:28,  5.07s/it]
 52%|█████▏    | 116246/225000 [240:31:08<138:17:10,  4.58s/it]
 52%|█████▏    | 116247/225000 [240:31:15<168:41:14,  5.58s/it]
 52%|█████▏    | 116248/225000 [240:31:22<174:58:40,  5.79s/it]
 52%|█████▏    | 116249/225000 [240:31:29<192:26:10,  6.37s/it]
 52%|█████▏    | 116250/225000 [240:31:33<165:07:19,  5.47s/it]
                                                               
{'loss': '0.8944', 'grad_norm': '1.253', 'learning_rate': '2.548e-05', 'epoch': '0.5167', 'num_input_tokens_seen': 401540008, 'train_runtime': '8.659e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116250/225000 [240:31:33<165:07:19,  5.47s/it]
 52%|█████▏    | 116251/225000 [240:31:37<156:14:39,  5.17s/it]
 52%|█████▏    | 116252/225000 [240:31:42<154:58:42,  5.13s/it]
 52%|█████▏    | 116253/225000 [240:31:48<163:51:04,  5.42s/it]
 52%|█████▏    | 116254/225000 [240:31:55<178:17:57,  5.90s/it]
 52%|█████▏    | 116255/225000 [240:31:59<152:27:19,  5.05s/it]
 52%|█████▏    | 116256/225000 [240:32:06<169:47:10,  5.62s/it]
 52%|█████▏    | 116257/225000 [240:32:09<151:47:00,  5.02s/it]
 52%|█████▏    | 116258/225000 [240:32:13<137:52:06,  4.56s/it]
 52%|█████▏    | 116259/225000 [240:32:16<131:18:46,  4.35s/it]
 52%|█████▏    | 116260/225000 [240:32:21<130:27:35,  4.32s/it]
 52%|█████▏    | 116261/225000 [240:32:29<169:01:27,  5.60s/it]
 52%|█████▏    | 116262/225000 [240:32:37<192:00:45,  6.36s/it]
 52%|█████▏    | 116263/225000 [240:32:40<161:35:20,  5.35s/it]
 52%|█████▏    | 116264/225000 [240:32:47<176:30:38,  5.84s/it]
 52%|█████▏    | 116265/225000 [240:32:53<177:18:06,  5.87s/it]
 52%|█████▏    | 116266/225000 [240:32:57<153:55:21,  5.10s/it]
 52%|█████▏    | 116267/225000 [240:33:07<200:58:36,  6.65s/it]
 52%|█████▏    | 116268/225000 [240:33:11<181:21:07,  6.00s/it]
 52%|█████▏    | 116269/225000 [240:33:22<225:43:54,  7.47s/it]
 52%|█████▏    | 116270/225000 [240:33:26<189:01:14,  6.26s/it]
 52%|█████▏    | 116271/225000 [240:33:29<163:58:50,  5.43s/it]
 52%|█████▏    | 116272/225000 [240:33:38<193:27:53,  6.41s/it]
 52%|█████▏    | 116273/225000 [240:33:42<173:33:54,  5.75s/it]
 52%|█████▏    | 116274/225000 [240:33:49<187:00:54,  6.19s/it]
 52%|█████▏    | 116275/225000 [240:33:55<185:59:02,  6.16s/it]
 52%|█████▏    | 116276/225000 [240:34:02<185:58:52,  6.16s/it]
 52%|█████▏    | 116277/225000 [240:34:07<175:57:24,  5.83s/it]
 52%|█████▏    | 116278/225000 [240:34:11<164:46:55,  5.46s/it]
 52%|█████▏    | 116279/225000 [240:34:16<153:49:37,  5.09s/it]
 52%|█████▏    | 116280/225000 [240:34:23<172:36:09,  5.72s/it]
 52%|█████▏    | 116281/225000 [240:34:27<156:32:35,  5.18s/it]
 52%|█████▏    | 116282/225000 [240:34:31<151:37:35,  5.02s/it]
 52%|█████▏    | 116283/225000 [240:34:41<195:42:23,  6.48s/it]
 52%|█████▏    | 116284/225000 [240:34:46<180:14:53,  5.97s/it]
 52%|█████▏    | 116285/225000 [240:34:50<165:46:58,  5.49s/it]
 52%|█████▏    | 116286/225000 [240:34:55<158:26:49,  5.25s/it]
 52%|█████▏    | 116287/225000 [240:35:05<202:19:40,  6.70s/it]
 52%|█████▏    | 116288/225000 [240:35:08<167:11:18,  5.54s/it]
 52%|█████▏    | 116289/225000 [240:35:11<147:41:37,  4.89s/it]
 52%|█████▏    | 116290/225000 [240:35:21<195:15:31,  6.47s/it]
 52%|█████▏    | 116291/225000 [240:35:25<171:19:40,  5.67s/it]
 52%|█████▏    | 116292/225000 [240:35:29<154:58:45,  5.13s/it]
 52%|█████▏    | 116293/225000 [240:35:33<147:34:30,  4.89s/it]
 52%|█████▏    | 116294/225000 [240:35:37<138:18:56,  4.58s/it]
 52%|█████▏    | 116295/225000 [240:35:41<132:39:39,  4.39s/it]
 52%|█████▏    | 116296/225000 [240:35:52<187:52:49,  6.22s/it]
 52%|█████▏    | 116297/225000 [240:35:56<168:38:00,  5.58s/it]
 52%|█████▏    | 116298/225000 [240:36:01<165:16:41,  5.47s/it]
 52%|█████▏    | 116299/225000 [240:36:09<183:59:16,  6.09s/it]
 52%|█████▏    | 116300/225000 [240:36:12<156:59:17,  5.20s/it]
                                                               
{'loss': '0.906', 'grad_norm': '1.124', 'learning_rate': '2.546e-05', 'epoch': '0.5169', 'num_input_tokens_seen': 401664880, 'train_runtime': '8.662e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116300/225000 [240:36:12<156:59:17,  5.20s/it]
 52%|█████▏    | 116301/225000 [240:36:20<189:38:03,  6.28s/it]
 52%|█████▏    | 116302/225000 [240:36:24<161:33:02,  5.35s/it]
 52%|█████▏    | 116303/225000 [240:36:31<180:23:10,  5.97s/it]
 52%|█████▏    | 116304/225000 [240:36:35<158:36:58,  5.25s/it]
 52%|█████▏    | 116305/225000 [240:36:41<167:26:57,  5.55s/it]
 52%|█████▏    | 116306/225000 [240:36:48<183:49:42,  6.09s/it]
 52%|█████▏    | 116307/225000 [240:36:55<185:52:38,  6.16s/it]
 52%|█████▏    | 116308/225000 [240:36:59<166:16:38,  5.51s/it]
 52%|█████▏    | 116309/225000 [240:37:02<145:40:33,  4.82s/it]
 52%|█████▏    | 116310/225000 [240:37:09<167:15:32,  5.54s/it]
 52%|█████▏    | 116311/225000 [240:37:15<173:26:45,  5.74s/it]
 52%|█████▏    | 116312/225000 [240:37:19<157:56:47,  5.23s/it]
 52%|█████▏    | 116313/225000 [240:37:23<148:22:32,  4.91s/it]
 52%|█████▏    | 116314/225000 [240:37:28<145:46:32,  4.83s/it]
 52%|█████▏    | 116315/225000 [240:37:32<136:55:37,  4.54s/it]
 52%|█████▏    | 116316/225000 [240:37:36<130:14:35,  4.31s/it]
 52%|█████▏    | 116317/225000 [240:37:48<205:39:38,  6.81s/it]
 52%|█████▏    | 116318/225000 [240:37:53<189:24:44,  6.27s/it]
 52%|█████▏    | 116319/225000 [240:38:01<201:47:01,  6.68s/it]
 52%|█████▏    | 116320/225000 [240:38:05<177:51:17,  5.89s/it]
 52%|█████▏    | 116321/225000 [240:38:09<156:36:38,  5.19s/it]
 52%|█████▏    | 116322/225000 [240:38:12<141:08:54,  4.68s/it]
 52%|█████▏    | 116323/225000 [240:38:21<181:30:02,  6.01s/it]
 52%|█████▏    | 116324/225000 [240:38:25<162:46:22,  5.39s/it]
 52%|█████▏    | 116325/225000 [240:38:28<140:15:38,  4.65s/it]
 52%|█████▏    | 116326/225000 [240:38:31<128:44:11,  4.26s/it]
 52%|█████▏    | 116327/225000 [240:38:39<155:32:08,  5.15s/it]
 52%|█████▏    | 116328/225000 [240:38:46<173:41:57,  5.75s/it]
 52%|█████▏    | 116329/225000 [240:38:50<161:43:34,  5.36s/it]
 52%|█████▏    | 116330/225000 [240:38:54<148:20:54,  4.91s/it]
 52%|█████▏    | 116331/225000 [240:38:58<142:53:05,  4.73s/it]
 52%|█████▏    | 116332/225000 [240:39:03<138:20:22,  4.58s/it]
 52%|█████▏    | 116333/225000 [240:39:07<136:11:30,  4.51s/it]
 52%|█████▏    | 116334/225000 [240:39:11<133:09:03,  4.41s/it]
 52%|█████▏    | 116335/225000 [240:39:15<126:34:31,  4.19s/it]
 52%|█████▏    | 116336/225000 [240:39:19<129:42:46,  4.30s/it]
 52%|█████▏    | 116337/225000 [240:39:25<137:57:46,  4.57s/it]
 52%|█████▏    | 116338/225000 [240:39:29<138:24:12,  4.59s/it]
 52%|█████▏    | 116339/225000 [240:39:37<168:09:57,  5.57s/it]
 52%|█████▏    | 116340/225000 [240:39:43<170:12:03,  5.64s/it]
 52%|█████▏    | 116341/225000 [240:39:48<166:08:49,  5.50s/it]
 52%|█████▏    | 116342/225000 [240:39:57<200:27:12,  6.64s/it]
 52%|█████▏    | 116343/225000 [240:40:01<169:09:12,  5.60s/it]
 52%|█████▏    | 116344/225000 [240:40:04<151:48:22,  5.03s/it]
 52%|█████▏    | 116345/225000 [240:40:12<178:40:35,  5.92s/it]
 52%|█████▏    | 116346/225000 [240:40:18<177:58:50,  5.90s/it]
 52%|█████▏    | 116347/225000 [240:40:21<149:08:00,  4.94s/it]
 52%|█████▏    | 116348/225000 [240:40:24<135:48:56,  4.50s/it]
 52%|█████▏    | 116349/225000 [240:40:31<157:43:55,  5.23s/it]
 52%|█████▏    | 116350/225000 [240:40:37<166:24:10,  5.51s/it]
                                                               
{'loss': '0.8946', 'grad_norm': '1.149', 'learning_rate': '2.544e-05', 'epoch': '0.5171', 'num_input_tokens_seen': 401778768, 'train_runtime': '8.664e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116350/225000 [240:40:37<166:24:10,  5.51s/it]
 52%|█████▏    | 116351/225000 [240:40:41<149:22:34,  4.95s/it]
 52%|█████▏    | 116352/225000 [240:40:45<136:29:26,  4.52s/it]
 52%|█████▏    | 116353/225000 [240:40:58<215:23:51,  7.14s/it]
 52%|█████▏    | 116354/225000 [240:41:01<177:13:46,  5.87s/it]
 52%|█████▏    | 116355/225000 [240:41:08<187:22:58,  6.21s/it]
 52%|█████▏    | 116356/225000 [240:41:11<156:49:37,  5.20s/it]
 52%|█████▏    | 116357/225000 [240:41:15<148:59:53,  4.94s/it]
 52%|█████▏    | 116358/225000 [240:41:19<141:12:11,  4.68s/it]
 52%|█████▏    | 116359/225000 [240:41:24<141:13:14,  4.68s/it]
 52%|█████▏    | 116360/225000 [240:41:28<140:30:06,  4.66s/it]
 52%|█████▏    | 116361/225000 [240:41:32<132:36:46,  4.39s/it]
 52%|█████▏    | 116362/225000 [240:41:36<129:39:20,  4.30s/it]
 52%|█████▏    | 116363/225000 [240:41:40<125:01:45,  4.14s/it]
 52%|█████▏    | 116364/225000 [240:41:50<176:50:21,  5.86s/it]
 52%|█████▏    | 116365/225000 [240:41:59<205:34:05,  6.81s/it]
 52%|█████▏    | 116366/225000 [240:42:02<174:45:30,  5.79s/it]
 52%|█████▏    | 116367/225000 [240:42:06<153:55:18,  5.10s/it]
 52%|█████▏    | 116368/225000 [240:42:10<146:58:15,  4.87s/it]
 52%|█████▏    | 116369/225000 [240:42:24<228:33:32,  7.57s/it]
 52%|█████▏    | 116370/225000 [240:42:28<194:52:42,  6.46s/it]
 52%|█████▏    | 116371/225000 [240:42:32<173:06:00,  5.74s/it]
 52%|█████▏    | 116372/225000 [240:42:36<162:43:33,  5.39s/it]
 52%|█████▏    | 116373/225000 [240:42:40<146:36:03,  4.86s/it]
 52%|█████▏    | 116374/225000 [240:42:44<140:08:03,  4.64s/it]
 52%|█████▏    | 116375/225000 [240:42:48<136:41:04,  4.53s/it]
 52%|█████▏    | 116376/225000 [240:42:52<129:06:45,  4.28s/it]
 52%|█████▏    | 116377/225000 [240:42:56<126:06:00,  4.18s/it]
 52%|█████▏    | 116378/225000 [240:43:00<125:17:18,  4.15s/it]
 52%|█████▏    | 116379/225000 [240:43:05<131:08:58,  4.35s/it]
 52%|█████▏    | 116380/225000 [240:43:11<145:59:00,  4.84s/it]
 52%|█████▏    | 116381/225000 [240:43:22<203:30:34,  6.74s/it]
 52%|█████▏    | 116382/225000 [240:43:25<169:19:44,  5.61s/it]
 52%|█████▏    | 116383/225000 [240:43:29<150:45:44,  5.00s/it]
 52%|█████▏    | 116384/225000 [240:43:33<141:41:00,  4.70s/it]
 52%|█████▏    | 116385/225000 [240:43:37<139:12:43,  4.61s/it]
 52%|█████▏    | 116386/225000 [240:43:42<139:42:53,  4.63s/it]
 52%|█████▏    | 116387/225000 [240:43:47<144:02:30,  4.77s/it]
 52%|█████▏    | 116388/225000 [240:43:58<200:39:06,  6.65s/it]
 52%|█████▏    | 116389/225000 [240:44:05<201:07:54,  6.67s/it]
 52%|█████▏    | 116390/225000 [240:44:08<168:57:20,  5.60s/it]
 52%|█████▏    | 116391/225000 [240:44:16<189:12:05,  6.27s/it]
 52%|█████▏    | 116392/225000 [240:44:19<166:25:51,  5.52s/it]
 52%|█████▏    | 116393/225000 [240:44:28<195:46:11,  6.49s/it]
 52%|█████▏    | 116394/225000 [240:44:32<173:19:46,  5.75s/it]
 52%|█████▏    | 116395/225000 [240:44:36<152:55:50,  5.07s/it]
 52%|█████▏    | 116396/225000 [240:44:40<149:54:23,  4.97s/it]
 52%|█████▏    | 116397/225000 [240:44:50<190:01:18,  6.30s/it]
 52%|█████▏    | 116398/225000 [240:44:54<172:32:27,  5.72s/it]
 52%|█████▏    | 116399/225000 [240:44:57<151:28:32,  5.02s/it]
 52%|█████▏    | 116400/225000 [240:45:01<140:55:59,  4.67s/it]
                                                               
{'loss': '0.8899', 'grad_norm': '1.123', 'learning_rate': '2.542e-05', 'epoch': '0.5173', 'num_input_tokens_seen': 401894816, 'train_runtime': '8.667e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116400/225000 [240:45:01<140:55:59,  4.67s/it]
 52%|█████▏    | 116401/225000 [240:45:05<130:14:43,  4.32s/it]
 52%|█████▏    | 116402/225000 [240:45:09<129:40:42,  4.30s/it]
 52%|█████▏    | 116403/225000 [240:45:17<162:32:03,  5.39s/it]
 52%|█████▏    | 116404/225000 [240:45:26<199:45:25,  6.62s/it]
 52%|█████▏    | 116405/225000 [240:45:30<175:48:59,  5.83s/it]
 52%|█████▏    | 116406/225000 [240:45:34<153:19:21,  5.08s/it]
 52%|█████▏    | 116407/225000 [240:45:43<189:26:11,  6.28s/it]
 52%|█████▏    | 116408/225000 [240:45:47<169:00:35,  5.60s/it]
 52%|█████▏    | 116409/225000 [240:45:51<151:46:12,  5.03s/it]
 52%|█████▏    | 116410/225000 [240:45:54<139:47:14,  4.63s/it]
 52%|█████▏    | 116411/225000 [240:45:58<134:25:09,  4.46s/it]
 52%|█████▏    | 116412/225000 [240:46:09<191:12:59,  6.34s/it]
 52%|█████▏    | 116413/225000 [240:46:13<168:12:28,  5.58s/it]
 52%|█████▏    | 116414/225000 [240:46:18<163:48:22,  5.43s/it]
 52%|█████▏    | 116415/225000 [240:46:28<203:28:26,  6.75s/it]
 52%|█████▏    | 116416/225000 [240:46:36<213:53:58,  7.09s/it]
 52%|█████▏    | 116417/225000 [240:46:39<179:04:21,  5.94s/it]
 52%|█████▏    | 116418/225000 [240:46:43<161:47:48,  5.36s/it]
 52%|█████▏    | 116419/225000 [240:46:47<145:45:12,  4.83s/it]
 52%|█████▏    | 116420/225000 [240:46:51<146:38:08,  4.86s/it]
 52%|█████▏    | 116421/225000 [240:47:02<200:22:26,  6.64s/it]
 52%|█████▏    | 116422/225000 [240:47:06<173:12:51,  5.74s/it]
 52%|█████▏    | 116423/225000 [240:47:10<160:30:28,  5.32s/it]
 52%|█████▏    | 116424/225000 [240:47:15<156:24:00,  5.19s/it]
 52%|█████▏    | 116425/225000 [240:47:19<142:37:00,  4.73s/it]
 52%|█████▏    | 116426/225000 [240:47:23<135:31:27,  4.49s/it]
 52%|█████▏    | 116427/225000 [240:47:27<137:16:35,  4.55s/it]
 52%|█████▏    | 116428/225000 [240:47:34<154:47:28,  5.13s/it]
 52%|█████▏    | 116429/225000 [240:47:43<194:00:56,  6.43s/it]
 52%|█████▏    | 116430/225000 [240:47:46<162:24:41,  5.39s/it]
 52%|█████▏    | 116431/225000 [240:47:49<141:38:26,  4.70s/it]
 52%|█████▏    | 116432/225000 [240:47:53<134:50:17,  4.47s/it]
 52%|█████▏    | 116433/225000 [240:47:58<134:37:03,  4.46s/it]
 52%|█████▏    | 116434/225000 [240:48:03<138:00:17,  4.58s/it]
 52%|█████▏    | 116435/225000 [240:48:08<145:13:21,  4.82s/it]
 52%|█████▏    | 116436/225000 [240:48:13<147:29:35,  4.89s/it]
 52%|█████▏    | 116437/225000 [240:48:18<145:56:21,  4.84s/it]
 52%|█████▏    | 116438/225000 [240:48:23<149:21:58,  4.95s/it]
 52%|█████▏    | 116439/225000 [240:48:27<141:24:22,  4.69s/it]
 52%|█████▏    | 116440/225000 [240:48:38<200:28:17,  6.65s/it]
 52%|█████▏    | 116441/225000 [240:48:45<197:59:03,  6.57s/it]
 52%|█████▏    | 116442/225000 [240:48:49<177:30:21,  5.89s/it]
 52%|█████▏    | 116443/225000 [240:48:53<158:25:17,  5.25s/it]
 52%|█████▏    | 116444/225000 [240:48:57<147:33:48,  4.89s/it]
 52%|█████▏    | 116445/225000 [240:49:11<228:21:59,  7.57s/it]
 52%|█████▏    | 116446/225000 [240:49:15<203:11:27,  6.74s/it]
 52%|█████▏    | 116447/225000 [240:49:26<240:48:49,  7.99s/it]
 52%|█████▏    | 116448/225000 [240:49:30<198:51:07,  6.59s/it]
 52%|█████▏    | 116449/225000 [240:49:33<170:43:10,  5.66s/it]
 52%|█████▏    | 116450/225000 [240:49:44<215:39:55,  7.15s/it]
                                                               
{'loss': '0.8937', 'grad_norm': '0.9429', 'learning_rate': '2.54e-05', 'epoch': '0.5176', 'num_input_tokens_seen': 402015568, 'train_runtime': '8.67e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116450/225000 [240:49:44<215:39:55,  7.15s/it]
 52%|█████▏    | 116451/225000 [240:49:47<178:41:16,  5.93s/it]
 52%|█████▏    | 116452/225000 [240:49:50<155:22:47,  5.15s/it]
 52%|█████▏    | 116453/225000 [240:49:54<146:43:08,  4.87s/it]
 52%|█████▏    | 116454/225000 [240:49:59<142:05:10,  4.71s/it]
 52%|█████▏    | 116455/225000 [240:50:04<145:41:59,  4.83s/it]
 52%|█████▏    | 116456/225000 [240:50:08<142:25:51,  4.72s/it]
 52%|█████▏    | 116457/225000 [240:50:13<138:32:45,  4.60s/it]
 52%|█████▏    | 116458/225000 [240:50:18<141:35:49,  4.70s/it]
 52%|█████▏    | 116459/225000 [240:50:22<143:06:31,  4.75s/it]
 52%|█████▏    | 116460/225000 [240:50:27<140:15:37,  4.65s/it]
 52%|█████▏    | 116461/225000 [240:50:32<148:39:00,  4.93s/it]
 52%|█████▏    | 116462/225000 [240:50:43<200:18:18,  6.64s/it]
 52%|█████▏    | 116463/225000 [240:50:50<204:12:41,  6.77s/it]
 52%|█████▏    | 116464/225000 [240:50:54<176:01:01,  5.84s/it]
 52%|█████▏    | 116465/225000 [240:50:58<164:14:46,  5.45s/it]
 52%|█████▏    | 116466/225000 [240:51:02<150:51:48,  5.00s/it]
 52%|█████▏    | 116467/225000 [240:51:07<149:23:55,  4.96s/it]
 52%|█████▏    | 116468/225000 [240:51:12<144:12:02,  4.78s/it]
 52%|█████▏    | 116469/225000 [240:51:16<142:49:29,  4.74s/it]
 52%|█████▏    | 116470/225000 [240:51:23<163:39:06,  5.43s/it]
 52%|█████▏    | 116471/225000 [240:51:32<195:28:30,  6.48s/it]
 52%|█████▏    | 116472/225000 [240:51:36<167:38:00,  5.56s/it]
 52%|█████▏    | 116473/225000 [240:51:44<191:04:15,  6.34s/it]
 52%|█████▏    | 116474/225000 [240:51:47<165:20:07,  5.48s/it]
 52%|█████▏    | 116475/225000 [240:51:52<162:37:43,  5.39s/it]
 52%|█████▏    | 116476/225000 [240:51:59<177:00:40,  5.87s/it]
 52%|█████▏    | 116477/225000 [240:52:03<153:55:13,  5.11s/it]
 52%|█████▏    | 116478/225000 [240:52:08<152:31:11,  5.06s/it]
 52%|█████▏    | 116479/225000 [240:52:12<148:33:31,  4.93s/it]
 52%|█████▏    | 116480/225000 [240:52:17<145:00:06,  4.81s/it]
 52%|█████▏    | 116481/225000 [240:52:21<138:42:37,  4.60s/it]
 52%|█████▏    | 116482/225000 [240:52:25<134:09:10,  4.45s/it]
 52%|█████▏    | 116483/225000 [240:52:30<137:53:25,  4.57s/it]
 52%|█████▏    | 116484/225000 [240:52:35<141:31:52,  4.70s/it]
 52%|█████▏    | 116485/225000 [240:52:40<146:38:16,  4.86s/it]
 52%|█████▏    | 116486/225000 [240:52:45<144:36:04,  4.80s/it]
 52%|█████▏    | 116487/225000 [240:52:50<146:26:26,  4.86s/it]
 52%|█████▏    | 116488/225000 [240:52:57<170:09:12,  5.65s/it]
 52%|█████▏    | 116489/225000 [240:53:09<224:49:04,  7.46s/it]
 52%|█████▏    | 116490/225000 [240:53:12<188:55:34,  6.27s/it]
 52%|█████▏    | 116491/225000 [240:53:16<165:28:38,  5.49s/it]
 52%|█████▏    | 116492/225000 [240:53:24<184:27:48,  6.12s/it]
 52%|█████▏    | 116493/225000 [240:53:32<203:25:35,  6.75s/it]
 52%|█████▏    | 116494/225000 [240:53:41<228:38:09,  7.59s/it]
 52%|█████▏    | 116495/225000 [240:53:45<189:25:32,  6.28s/it]
 52%|█████▏    | 116496/225000 [240:53:48<161:56:53,  5.37s/it]
 52%|█████▏    | 116497/225000 [240:53:59<216:26:26,  7.18s/it]
 52%|█████▏    | 116498/225000 [240:54:03<185:25:33,  6.15s/it]
 52%|█████▏    | 116499/225000 [240:54:07<165:20:38,  5.49s/it]
 52%|█████▏    | 116500/225000 [240:54:14<175:38:27,  5.83s/it]
                                                               
{'loss': '0.898', 'grad_norm': '1.259', 'learning_rate': '2.538e-05', 'epoch': '0.5178', 'num_input_tokens_seen': 402133064, 'train_runtime': '8.673e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116500/225000 [240:54:14<175:38:27,  5.83s/it]
 52%|█████▏    | 116501/225000 [240:54:19<167:41:26,  5.56s/it]
 52%|█████▏    | 116502/225000 [240:54:22<146:45:01,  4.87s/it]
 52%|█████▏    | 116503/225000 [240:54:31<183:43:02,  6.10s/it]
 52%|█████▏    | 116504/225000 [240:54:34<155:56:56,  5.17s/it]
 52%|█████▏    | 116505/225000 [240:54:43<192:07:36,  6.38s/it]
 52%|█████▏    | 116506/225000 [240:54:49<184:35:29,  6.13s/it]
 52%|█████▏    | 116507/225000 [240:54:52<164:31:39,  5.46s/it]
 52%|█████▏    | 116508/225000 [240:54:56<150:24:40,  4.99s/it]
 52%|█████▏    | 116509/225000 [240:55:07<205:47:02,  6.83s/it]
 52%|█████▏    | 116510/225000 [240:55:11<177:11:16,  5.88s/it]
 52%|█████▏    | 116511/225000 [240:55:17<173:40:24,  5.76s/it]
 52%|█████▏    | 116512/225000 [240:55:23<177:08:22,  5.88s/it]
 52%|█████▏    | 116513/225000 [240:55:28<171:58:18,  5.71s/it]
 52%|█████▏    | 116514/225000 [240:55:34<172:52:49,  5.74s/it]
 52%|█████▏    | 116515/225000 [240:55:39<163:04:10,  5.41s/it]
 52%|█████▏    | 116516/225000 [240:55:44<164:23:54,  5.46s/it]
 52%|█████▏    | 116517/225000 [240:55:49<157:35:37,  5.23s/it]
 52%|█████▏    | 116518/225000 [240:55:54<158:36:12,  5.26s/it]
 52%|█████▏    | 116519/225000 [240:55:59<156:53:11,  5.21s/it]
 52%|█████▏    | 116520/225000 [240:56:05<164:52:43,  5.47s/it]
 52%|█████▏    | 116521/225000 [240:56:11<170:14:21,  5.65s/it]
 52%|█████▏    | 116522/225000 [240:56:16<160:00:50,  5.31s/it]
 52%|█████▏    | 116523/225000 [240:56:20<145:07:13,  4.82s/it]
 52%|█████▏    | 116524/225000 [240:56:24<142:43:36,  4.74s/it]
 52%|█████▏    | 116525/225000 [240:56:34<187:02:30,  6.21s/it]
 52%|█████▏    | 116526/225000 [240:56:39<183:04:40,  6.08s/it]
 52%|█████▏    | 116527/225000 [240:56:43<156:19:39,  5.19s/it]
 52%|█████▏    | 116528/225000 [240:56:47<151:20:21,  5.02s/it]
 52%|█████▏    | 116529/225000 [240:56:54<168:46:14,  5.60s/it]
 52%|█████▏    | 116530/225000 [240:56:59<157:40:56,  5.23s/it]
 52%|█████▏    | 116531/225000 [240:57:03<152:12:03,  5.05s/it]
 52%|█████▏    | 116532/225000 [240:57:08<146:28:19,  4.86s/it]
 52%|█████▏    | 116533/225000 [240:57:13<150:47:07,  5.00s/it]
 52%|█████▏    | 116534/225000 [240:57:18<148:02:53,  4.91s/it]
 52%|█████▏    | 116535/225000 [240:57:22<142:22:42,  4.73s/it]
 52%|█████▏    | 116536/225000 [240:57:29<165:21:33,  5.49s/it]
 52%|█████▏    | 116537/225000 [240:57:41<226:34:09,  7.52s/it]
 52%|█████▏    | 116538/225000 [240:57:45<188:22:47,  6.25s/it]
 52%|█████▏    | 116539/225000 [240:57:51<189:16:30,  6.28s/it]
 52%|█████▏    | 116540/225000 [240:57:56<173:02:38,  5.74s/it]
 52%|█████▏    | 116541/225000 [240:57:59<153:28:06,  5.09s/it]
 52%|█████▏    | 116542/225000 [240:58:05<155:54:36,  5.18s/it]
 52%|█████▏    | 116543/225000 [240:58:09<145:33:44,  4.83s/it]
 52%|█████▏    | 116544/225000 [240:58:14<152:06:09,  5.05s/it]
 52%|█████▏    | 116545/225000 [240:58:22<180:20:44,  5.99s/it]
 52%|█████▏    | 116546/225000 [240:58:26<161:44:11,  5.37s/it]
 52%|█████▏    | 116547/225000 [240:58:34<179:25:45,  5.96s/it]
 52%|█████▏    | 116548/225000 [240:58:40<183:03:36,  6.08s/it]
 52%|█████▏    | 116549/225000 [240:58:44<163:16:19,  5.42s/it]
 52%|█████▏    | 116550/225000 [240:58:47<144:42:14,  4.80s/it]
                                                               
{'loss': '0.8872', 'grad_norm': '1.381', 'learning_rate': '2.537e-05', 'epoch': '0.518', 'num_input_tokens_seen': 402254496, 'train_runtime': '8.675e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116550/225000 [240:58:47<144:42:14,  4.80s/it]
 52%|█████▏    | 116551/225000 [240:58:57<185:38:01,  6.16s/it]
 52%|█████▏    | 116552/225000 [240:59:01<171:34:24,  5.70s/it]
 52%|█████▏    | 116553/225000 [240:59:11<211:40:44,  7.03s/it]
 52%|█████▏    | 116554/225000 [240:59:15<177:41:11,  5.90s/it]
 52%|█████▏    | 116555/225000 [240:59:18<156:18:52,  5.19s/it]
 52%|█████▏    | 116556/225000 [240:59:22<144:49:33,  4.81s/it]
 52%|█████▏    | 116557/225000 [240:59:27<149:35:42,  4.97s/it]
 52%|█████▏    | 116558/225000 [240:59:37<196:45:09,  6.53s/it]
 52%|█████▏    | 116559/225000 [240:59:40<164:29:25,  5.46s/it]
 52%|█████▏    | 116560/225000 [240:59:48<184:18:26,  6.12s/it]
 52%|█████▏    | 116561/225000 [240:59:51<159:08:37,  5.28s/it]
 52%|█████▏    | 116562/225000 [240:59:57<158:01:41,  5.25s/it]
 52%|█████▏    | 116563/225000 [241:00:01<153:36:41,  5.10s/it]
 52%|█████▏    | 116564/225000 [241:00:07<156:13:48,  5.19s/it]
 52%|█████▏    | 116565/225000 [241:00:12<153:25:05,  5.09s/it]
 52%|█████▏    | 116566/225000 [241:00:16<149:16:09,  4.96s/it]
 52%|█████▏    | 116567/225000 [241:00:21<148:40:08,  4.94s/it]
 52%|█████▏    | 116568/225000 [241:00:27<157:45:50,  5.24s/it]
 52%|█████▏    | 116569/225000 [241:00:35<178:47:58,  5.94s/it]
 52%|█████▏    | 116570/225000 [241:00:39<161:44:23,  5.37s/it]
 52%|█████▏    | 116571/225000 [241:00:43<152:55:59,  5.08s/it]
 52%|█████▏    | 116572/225000 [241:00:48<149:51:56,  4.98s/it]
 52%|█████▏    | 116573/225000 [241:00:58<199:51:12,  6.64s/it]
 52%|█████▏    | 116574/225000 [241:01:02<173:20:24,  5.76s/it]
 52%|█████▏    | 116575/225000 [241:01:06<154:32:25,  5.13s/it]
 52%|█████▏    | 116576/225000 [241:01:19<227:56:27,  7.57s/it]
 52%|█████▏    | 116577/225000 [241:01:23<193:05:33,  6.41s/it]
 52%|█████▏    | 116578/225000 [241:01:26<163:42:44,  5.44s/it]
 52%|█████▏    | 116579/225000 [241:01:34<188:41:55,  6.27s/it]
 52%|█████▏    | 116580/225000 [241:01:37<161:46:25,  5.37s/it]
 52%|█████▏    | 116581/225000 [241:01:41<144:17:22,  4.79s/it]
 52%|█████▏    | 116582/225000 [241:01:44<133:26:00,  4.43s/it]
 52%|█████▏    | 116583/225000 [241:01:48<129:59:18,  4.32s/it]
 52%|█████▏    | 116584/225000 [241:02:00<196:40:05,  6.53s/it]
 52%|█████▏    | 116585/225000 [241:02:05<184:41:27,  6.13s/it]
 52%|█████▏    | 116586/225000 [241:02:08<156:14:15,  5.19s/it]
 52%|█████▏    | 116587/225000 [241:02:12<139:12:48,  4.62s/it]
 52%|█████▏    | 116588/225000 [241:02:15<128:55:48,  4.28s/it]
 52%|█████▏    | 116589/225000 [241:02:18<119:38:07,  3.97s/it]
 52%|█████▏    | 116590/225000 [241:02:29<179:43:28,  5.97s/it]
 52%|█████▏    | 116591/225000 [241:02:32<153:41:42,  5.10s/it]
 52%|█████▏    | 116592/225000 [241:02:41<190:21:37,  6.32s/it]
 52%|█████▏    | 116593/225000 [241:02:45<168:47:40,  5.61s/it]
 52%|█████▏    | 116594/225000 [241:02:49<149:05:13,  4.95s/it]
 52%|█████▏    | 116595/225000 [241:02:52<135:38:16,  4.50s/it]
 52%|█████▏    | 116596/225000 [241:02:57<135:33:34,  4.50s/it]
 52%|█████▏    | 116597/225000 [241:03:01<139:14:18,  4.62s/it]
 52%|█████▏    | 116598/225000 [241:03:06<137:08:09,  4.55s/it]
 52%|█████▏    | 116599/225000 [241:03:11<141:08:08,  4.69s/it]
 52%|█████▏    | 116600/225000 [241:03:16<143:15:18,  4.76s/it]
                                                               
{'loss': '0.8937', 'grad_norm': '1.024', 'learning_rate': '2.535e-05', 'epoch': '0.5182', 'num_input_tokens_seen': 402373136, 'train_runtime': '8.678e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116600/225000 [241:03:16<143:15:18,  4.76s/it]
 52%|█████▏    | 116601/225000 [241:03:20<141:52:11,  4.71s/it]
 52%|█████▏    | 116602/225000 [241:03:26<148:52:53,  4.94s/it]
 52%|█████▏    | 116603/225000 [241:03:32<161:10:20,  5.35s/it]
 52%|█████▏    | 116604/225000 [241:03:44<219:55:27,  7.30s/it]
 52%|█████▏    | 116605/225000 [241:03:50<208:15:07,  6.92s/it]
 52%|█████▏    | 116606/225000 [241:03:54<178:07:00,  5.92s/it]
 52%|█████▏    | 116607/225000 [241:03:57<155:14:30,  5.16s/it]
 52%|█████▏    | 116608/225000 [241:04:00<139:03:11,  4.62s/it]
 52%|█████▏    | 116609/225000 [241:04:06<147:07:10,  4.89s/it]
 52%|█████▏    | 116610/225000 [241:04:11<147:14:55,  4.89s/it]
 52%|█████▏    | 116611/225000 [241:04:24<221:49:26,  7.37s/it]
 52%|█████▏    | 116612/225000 [241:04:28<194:23:40,  6.46s/it]
 52%|█████▏    | 116613/225000 [241:04:32<172:25:42,  5.73s/it]
 52%|█████▏    | 116614/225000 [241:04:36<156:59:18,  5.21s/it]
 52%|█████▏    | 116615/225000 [241:04:40<144:01:13,  4.78s/it]
 52%|█████▏    | 116616/225000 [241:04:45<147:56:53,  4.91s/it]
 52%|█████▏    | 116617/225000 [241:04:50<147:45:42,  4.91s/it]
 52%|█████▏    | 116618/225000 [241:04:55<149:37:20,  4.97s/it]
 52%|█████▏    | 116619/225000 [241:04:59<141:02:34,  4.68s/it]
 52%|█████▏    | 116620/225000 [241:05:04<137:07:47,  4.55s/it]
 52%|█████▏    | 116621/225000 [241:05:11<167:24:51,  5.56s/it]
 52%|█████▏    | 116622/225000 [241:05:23<219:08:51,  7.28s/it]
 52%|█████▏    | 116623/225000 [241:05:27<190:46:22,  6.34s/it]
 52%|█████▏    | 116624/225000 [241:05:30<165:20:08,  5.49s/it]
 52%|█████▏    | 116625/225000 [241:05:34<152:38:50,  5.07s/it]
 52%|█████▏    | 116626/225000 [241:05:41<165:28:48,  5.50s/it]
 52%|█████▏    | 116627/225000 [241:05:48<178:18:55,  5.92s/it]
 52%|█████▏    | 116628/225000 [241:05:52<161:20:22,  5.36s/it]
 52%|█████▏    | 116629/225000 [241:05:56<147:27:41,  4.90s/it]
 52%|█████▏    | 116630/225000 [241:06:01<146:42:08,  4.87s/it]
 52%|█████▏    | 116631/225000 [241:06:05<145:14:33,  4.82s/it]
 52%|█████▏    | 116632/225000 [241:06:10<141:59:48,  4.72s/it]
 52%|█████▏    | 116633/225000 [241:06:14<138:26:10,  4.60s/it]
 52%|█████▏    | 116634/225000 [241:06:19<143:05:57,  4.75s/it]
 52%|█████▏    | 116635/225000 [241:06:31<208:00:41,  6.91s/it]
 52%|█████▏    | 116636/225000 [241:06:36<192:56:34,  6.41s/it]
 52%|█████▏    | 116637/225000 [241:06:41<173:31:32,  5.76s/it]
 52%|█████▏    | 116638/225000 [241:06:52<222:53:57,  7.41s/it]
 52%|█████▏    | 116639/225000 [241:06:56<197:36:23,  6.56s/it]
 52%|█████▏    | 116640/225000 [241:07:02<190:43:11,  6.34s/it]
 52%|█████▏    | 116641/225000 [241:07:05<158:39:00,  5.27s/it]
 52%|█████▏    | 116642/225000 [241:07:08<138:21:42,  4.60s/it]
 52%|█████▏    | 116643/225000 [241:07:16<167:29:41,  5.56s/it]
 52%|█████▏    | 116644/225000 [241:07:20<151:15:08,  5.03s/it]
 52%|█████▏    | 116645/225000 [241:07:23<135:58:21,  4.52s/it]
 52%|█████▏    | 116646/225000 [241:07:34<194:44:45,  6.47s/it]
 52%|█████▏    | 116647/225000 [241:07:40<188:06:21,  6.25s/it]
 52%|█████▏    | 116648/225000 [241:07:43<160:42:07,  5.34s/it]
 52%|█████▏    | 116649/225000 [241:07:50<180:01:36,  5.98s/it]
 52%|█████▏    | 116650/225000 [241:07:54<157:13:22,  5.22s/it]
                                                               
{'loss': '0.8959', 'grad_norm': '1.018', 'learning_rate': '2.533e-05', 'epoch': '0.5184', 'num_input_tokens_seen': 402493472, 'train_runtime': '8.681e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116650/225000 [241:07:54<157:13:22,  5.22s/it]
 52%|█████▏    | 116651/225000 [241:07:57<141:47:22,  4.71s/it]
 52%|█████▏    | 116652/225000 [241:08:01<132:10:10,  4.39s/it]
 52%|█████▏    | 116653/225000 [241:08:09<159:32:57,  5.30s/it]
 52%|█████▏    | 116654/225000 [241:08:16<179:52:14,  5.98s/it]
 52%|█████▏    | 116655/225000 [241:08:21<165:59:56,  5.52s/it]
 52%|█████▏    | 116656/225000 [241:08:27<171:57:36,  5.71s/it]
 52%|█████▏    | 116657/225000 [241:08:36<203:03:50,  6.75s/it]
 52%|█████▏    | 116658/225000 [241:08:39<171:03:00,  5.68s/it]
 52%|█████▏    | 116659/225000 [241:08:44<168:28:13,  5.60s/it]
 52%|█████▏    | 116660/225000 [241:08:49<161:54:35,  5.38s/it]
 52%|█████▏    | 116661/225000 [241:08:53<149:27:53,  4.97s/it]
 52%|█████▏    | 116662/225000 [241:08:58<143:40:02,  4.77s/it]
 52%|█████▏    | 116663/225000 [241:09:02<143:40:41,  4.77s/it]
 52%|█████▏    | 116664/225000 [241:09:07<138:17:59,  4.60s/it]
 52%|█████▏    | 116665/225000 [241:09:11<137:07:57,  4.56s/it]
 52%|█████▏    | 116666/225000 [241:09:16<138:54:43,  4.62s/it]
 52%|█████▏    | 116667/225000 [241:09:20<136:00:25,  4.52s/it]
 52%|█████▏    | 116668/225000 [241:09:32<202:07:30,  6.72s/it]
 52%|█████▏    | 116669/225000 [241:09:37<184:10:19,  6.12s/it]
 52%|█████▏    | 116670/225000 [241:09:43<183:59:29,  6.11s/it]
 52%|█████▏    | 116671/225000 [241:09:47<168:42:44,  5.61s/it]
 52%|█████▏    | 116672/225000 [241:09:52<158:13:26,  5.26s/it]
 52%|█████▏    | 116673/225000 [241:10:02<203:43:51,  6.77s/it]
 52%|█████▏    | 116674/225000 [241:10:07<185:39:45,  6.17s/it]
 52%|█████▏    | 116675/225000 [241:10:11<169:27:25,  5.63s/it]
 52%|█████▏    | 116676/225000 [241:10:16<161:11:11,  5.36s/it]
 52%|█████▏    | 116677/225000 [241:10:20<149:13:13,  4.96s/it]
 52%|█████▏    | 116678/225000 [241:10:25<149:53:26,  4.98s/it]
 52%|█████▏    | 116679/225000 [241:10:29<139:40:17,  4.64s/it]
 52%|█████▏    | 116680/225000 [241:10:42<217:23:46,  7.23s/it]
 52%|█████▏    | 116681/225000 [241:10:46<192:10:30,  6.39s/it]
 52%|█████▏    | 116682/225000 [241:10:51<173:53:31,  5.78s/it]
 52%|█████▏    | 116683/225000 [241:10:55<155:49:16,  5.18s/it]
 52%|█████▏    | 116684/225000 [241:10:59<149:27:47,  4.97s/it]
 52%|█████▏    | 116685/225000 [241:11:04<148:34:08,  4.94s/it]
 52%|█████▏    | 116686/225000 [241:11:09<149:42:34,  4.98s/it]
 52%|█████▏    | 116687/225000 [241:11:13<144:30:57,  4.80s/it]
 52%|█████▏    | 116688/225000 [241:11:25<202:32:34,  6.73s/it]
 52%|█████▏    | 116689/225000 [241:11:30<190:09:10,  6.32s/it]
 52%|█████▏    | 116690/225000 [241:11:35<175:27:04,  5.83s/it]
 52%|█████▏    | 116691/225000 [241:11:44<208:03:12,  6.92s/it]
 52%|█████▏    | 116692/225000 [241:11:48<181:09:37,  6.02s/it]
 52%|█████▏    | 116693/225000 [241:11:53<171:06:46,  5.69s/it]
 52%|█████▏    | 116694/225000 [241:11:58<163:40:42,  5.44s/it]
 52%|█████▏    | 116695/225000 [241:12:03<159:32:11,  5.30s/it]
 52%|█████▏    | 116696/225000 [241:12:07<150:36:54,  5.01s/it]
 52%|█████▏    | 116697/225000 [241:12:12<145:19:56,  4.83s/it]
 52%|█████▏    | 116698/225000 [241:12:16<144:17:03,  4.80s/it]
 52%|█████▏    | 116699/225000 [241:12:21<143:07:18,  4.76s/it]
 52%|█████▏    | 116700/225000 [241:12:26<145:10:29,  4.83s/it]
                                                               
{'loss': '0.8954', 'grad_norm': '1.096', 'learning_rate': '2.531e-05', 'epoch': '0.5187', 'num_input_tokens_seen': 402614992, 'train_runtime': '8.683e+05', 'train_tokens_per_second': '463.7'}

 52%|█████▏    | 116700/225000 [241:12:26<145:10:29,  4.83s/it]
 52%|█████▏    | 116701/225000 [241:12:35<182:40:52,  6.07s/it]
 52%|█████▏    | 116702/225000 [241:12:45<217:09:41,  7.22s/it]
 52%|█████▏    | 116703/225000 [241:12:48<183:01:34,  6.08s/it]
 52%|█████▏    | 116704/225000 [241:12:52<162:10:12,  5.39s/it]
 52%|█████▏    | 116705/225000 [241:12:56<148:37:35,  4.94s/it]
 52%|█████▏    | 116706/225000 [241:13:05<185:50:10,  6.18s/it]
 52%|█████▏    | 116707/225000 [241:13:15<218:36:31,  7.27s/it]
 52%|█████▏    | 116708/225000 [241:13:18<185:56:25,  6.18s/it]
 52%|█████▏    | 116709/225000 [241:13:25<187:35:11,  6.24s/it]
 52%|█████▏    | 116710/225000 [241:13:29<167:32:18,  5.57s/it]
 52%|█████▏    | 116711/225000 [241:13:32<148:55:41,  4.95s/it]
 52%|█████▏    | 116712/225000 [241:13:41<186:57:27,  6.22s/it]
 52%|█████▏    | 116713/225000 [241:13:46<173:27:30,  5.77s/it]
 52%|█████▏    | 116714/225000 [241:13:50<152:31:47,  5.07s/it]
 52%|█████▏    | 116715/225000 [241:13:52<132:57:35,  4.42s/it]
 52%|█████▏    | 116716/225000 [241:14:00<163:00:28,  5.42s/it]
 52%|█████▏    | 116717/225000 [241:14:06<168:19:47,  5.60s/it]
 52%|█████▏    | 116718/225000 [241:14:13<176:59:06,  5.88s/it]
 52%|█████▏    | 116719/225000 [241:14:17<164:09:43,  5.46s/it]
 52%|█████▏    | 116720/225000 [241:14:22<157:51:04,  5.25s/it]
 52%|█████▏    | 116721/225000 [241:14:26<142:42:37,  4.74s/it]
 52%|█████▏    | 116722/225000 [241:14:31<147:15:40,  4.90s/it]
 52%|█████▏    | 116723/225000 [241:14:40<183:47:04,  6.11s/it]
 52%|█████▏    | 116724/225000 [241:14:43<157:46:33,  5.25s/it]
 52%|█████▏    | 116725/225000 [241:14:46<137:19:22,  4.57s/it]
 52%|█████▏    | 116726/225000 [241:14:55<180:25:25,  6.00s/it]
 52%|█████▏    | 116727/225000 [241:14:59<162:54:35,  5.42s/it]
 52%|█████▏    | 116728/225000 [241:15:03<147:34:15,  4.91s/it]
 52%|█████▏    | 116729/225000 [241:15:11<174:17:45,  5.80s/it]
 52%|█████▏    | 116730/225000 [241:15:16<166:40:02,  5.54s/it]
 52%|█████▏    | 116731/225000 [241:15:20<151:26:28,  5.04s/it]
 52%|█████▏    | 116732/225000 [241:15:25<155:03:32,  5.16s/it]
 52%|█████▏    | 116733/225000 [241:15:30<149:33:49,  4.97s/it]
 52%|█████▏    | 116734/225000 [241:15:34<140:13:19,  4.66s/it]
 52%|█████▏    | 116735/225000 [241:15:41<161:50:36,  5.38s/it]
 52%|█████▏    | 116736/225000 [241:15:49<189:11:51,  6.29s/it]
 52%|█████▏    | 116737/225000 [241:15:52<160:05:42,  5.32s/it]
 52%|█████▏    | 116738/225000 [241:15:58<160:46:15,  5.35s/it]
 52%|█████▏    | 116739/225000 [241:16:03<161:34:21,  5.37s/it]
 52%|█████▏    | 116740/225000 [241:16:08<155:04:39,  5.16s/it]
 52%|█████▏    | 116741/225000 [241:16:13<153:16:47,  5.10s/it]
 52%|█████▏    | 116742/225000 [241:16:17<146:25:13,  4.87s/it]
 52%|█████▏    | 116743/225000 [241:16:21<140:17:55,  4.67s/it]
 52%|█████▏    | 116744/225000 [241:16:26<142:40:00,  4.74s/it]
 52%|█████▏    | 116745/225000 [241:16:31<142:45:37,  4.75s/it]
 52%|█████▏    | 116746/225000 [241:16:36<146:59:29,  4.89s/it]
 52%|█████▏    | 116747/225000 [241:16:43<167:52:10,  5.58s/it]
 52%|█████▏    | 116748/225000 [241:16:55<226:25:27,  7.53s/it]
 52%|█████▏    | 116749/225000 [241:16:58<186:20:20,  6.20s/it]
 52%|█████▏    | 116750/225000 [241:17:02<162:28:21,  5.40s/it]
                                                               
{'loss': '0.8936', 'grad_norm': '1.188', 'learning_rate': '2.529e-05', 'epoch': '0.5189', 'num_input_tokens_seen': 402733584, 'train_runtime': '8.686e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 116750/225000 [241:17:02<162:28:21,  5.40s/it]
 52%|█████▏    | 116751/225000 [241:17:06<147:42:37,  4.91s/it]
 52%|█████▏    | 116752/225000 [241:17:10<144:16:49,  4.80s/it]
 52%|█████▏    | 116753/225000 [241:17:18<169:27:57,  5.64s/it]
 52%|█████▏    | 116754/225000 [241:17:25<186:47:11,  6.21s/it]
 52%|█████▏    | 116755/225000 [241:17:31<176:56:39,  5.88s/it]
 52%|█████▏    | 116756/225000 [241:17:39<196:42:55,  6.54s/it]
 52%|█████▏    | 116757/225000 [241:17:43<179:13:31,  5.96s/it]
 52%|█████▏    | 116758/225000 [241:17:48<169:08:48,  5.63s/it]
 52%|█████▏    | 116759/225000 [241:17:52<151:01:46,  5.02s/it]
 52%|█████▏    | 116760/225000 [241:17:56<144:59:57,  4.82s/it]
 52%|█████▏    | 116761/225000 [241:18:00<138:05:27,  4.59s/it]
 52%|█████▏    | 116762/225000 [241:18:06<153:24:10,  5.10s/it]
 52%|█████▏    | 116763/225000 [241:18:18<207:08:35,  6.89s/it]
 52%|█████▏    | 116764/225000 [241:18:21<173:08:21,  5.76s/it]
 52%|█████▏    | 116765/225000 [241:18:25<156:49:45,  5.22s/it]
 52%|█████▏    | 116766/225000 [241:18:32<178:02:56,  5.92s/it]
 52%|█████▏    | 116767/225000 [241:18:35<150:22:08,  5.00s/it]
 52%|█████▏    | 116768/225000 [241:18:42<167:12:24,  5.56s/it]
 52%|█████▏    | 116769/225000 [241:18:49<184:20:02,  6.13s/it]
 52%|█████▏    | 116770/225000 [241:18:53<158:17:49,  5.27s/it]
 52%|█████▏    | 116771/225000 [241:18:57<148:09:23,  4.93s/it]
 52%|█████▏    | 116772/225000 [241:19:02<149:18:08,  4.97s/it]
 52%|█████▏    | 116773/225000 [241:19:06<144:06:53,  4.79s/it]
 52%|█████▏    | 116774/225000 [241:19:11<142:30:16,  4.74s/it]
 52%|█████▏    | 116775/225000 [241:19:15<141:38:15,  4.71s/it]
 52%|█████▏    | 116776/225000 [241:19:20<139:32:38,  4.64s/it]
 52%|█████▏    | 116777/225000 [241:19:24<136:32:23,  4.54s/it]
 52%|█████▏    | 116778/225000 [241:19:29<141:26:46,  4.71s/it]
 52%|█████▏    | 116779/225000 [241:19:34<142:18:52,  4.73s/it]
 52%|█████▏    | 116780/225000 [241:19:44<188:57:55,  6.29s/it]
 52%|█████▏    | 116781/225000 [241:19:53<210:27:48,  7.00s/it]
 52%|█████▏    | 116782/225000 [241:19:56<173:07:21,  5.76s/it]
 52%|█████▏    | 116783/225000 [241:20:03<189:34:41,  6.31s/it]
 52%|█████▏    | 116784/225000 [241:20:07<164:10:22,  5.46s/it]
 52%|█████▏    | 116785/225000 [241:20:18<214:04:43,  7.12s/it]
 52%|█████▏    | 116786/225000 [241:20:22<188:14:12,  6.26s/it]
 52%|█████▏    | 116787/225000 [241:20:29<197:33:00,  6.57s/it]
 52%|█████▏    | 116788/225000 [241:20:33<176:20:49,  5.87s/it]
 52%|█████▏    | 116789/225000 [241:20:37<158:49:18,  5.28s/it]
 52%|█████▏    | 116790/225000 [241:20:44<174:49:48,  5.82s/it]
 52%|█████▏    | 116791/225000 [241:20:50<172:34:37,  5.74s/it]
 52%|█████▏    | 116792/225000 [241:20:54<154:04:16,  5.13s/it]
 52%|█████▏    | 116793/225000 [241:20:57<141:05:47,  4.69s/it]
 52%|█████▏    | 116794/225000 [241:21:02<140:58:34,  4.69s/it]
 52%|█████▏    | 116795/225000 [241:21:12<191:11:14,  6.36s/it]
 52%|█████▏    | 116796/225000 [241:21:16<164:01:22,  5.46s/it]
 52%|█████▏    | 116797/225000 [241:21:22<174:17:38,  5.80s/it]
 52%|█████▏    | 116798/225000 [241:21:29<179:36:40,  5.98s/it]
 52%|█████▏    | 116799/225000 [241:21:34<173:47:50,  5.78s/it]
 52%|█████▏    | 116800/225000 [241:21:39<164:39:05,  5.48s/it]
                                                               
{'loss': '0.8925', 'grad_norm': '1.107', 'learning_rate': '2.528e-05', 'epoch': '0.5191', 'num_input_tokens_seen': 402852688, 'train_runtime': '8.689e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 116800/225000 [241:21:39<164:39:05,  5.48s/it]
 52%|█████▏    | 116801/225000 [241:21:42<147:51:13,  4.92s/it]
 52%|█████▏    | 116802/225000 [241:21:47<145:13:32,  4.83s/it]
 52%|█████▏    | 116803/225000 [241:21:52<146:12:07,  4.86s/it]
 52%|█████▏    | 116804/225000 [241:21:56<141:04:49,  4.69s/it]
 52%|█████▏    | 116805/225000 [241:22:00<134:31:07,  4.48s/it]
 52%|█████▏    | 116806/225000 [241:22:13<206:22:08,  6.87s/it]
 52%|█████▏    | 116807/225000 [241:22:16<171:25:33,  5.70s/it]
 52%|█████▏    | 116808/225000 [241:22:19<155:14:59,  5.17s/it]
 52%|█████▏    | 116809/225000 [241:22:23<143:04:46,  4.76s/it]
 52%|█████▏    | 116810/225000 [241:22:28<140:36:26,  4.68s/it]
 52%|█████▏    | 116811/225000 [241:22:32<139:48:24,  4.65s/it]
 52%|█████▏    | 116812/225000 [241:22:43<191:38:00,  6.38s/it]
 52%|█████▏    | 116813/225000 [241:22:49<187:31:25,  6.24s/it]
 52%|█████▏    | 116814/225000 [241:22:52<157:40:34,  5.25s/it]
 52%|█████▏    | 116815/225000 [241:22:56<150:38:40,  5.01s/it]
 52%|█████▏    | 116816/225000 [241:23:00<140:00:56,  4.66s/it]
 52%|█████▏    | 116817/225000 [241:23:04<137:26:06,  4.57s/it]
 52%|█████▏    | 116818/225000 [241:23:16<199:47:07,  6.65s/it]
 52%|█████▏    | 116819/225000 [241:23:20<180:00:34,  5.99s/it]
 52%|█████▏    | 116820/225000 [241:23:24<157:44:37,  5.25s/it]
 52%|█████▏    | 116821/225000 [241:23:33<190:10:59,  6.33s/it]
 52%|█████▏    | 116822/225000 [241:23:36<167:36:30,  5.58s/it]
 52%|█████▏    | 116823/225000 [241:23:40<149:08:15,  4.96s/it]
 52%|█████▏    | 116824/225000 [241:23:43<135:21:41,  4.50s/it]
 52%|█████▏    | 116825/225000 [241:23:53<185:31:55,  6.17s/it]
 52%|█████▏    | 116826/225000 [241:23:57<159:24:36,  5.31s/it]
 52%|█████▏    | 116827/225000 [241:24:01<150:30:25,  5.01s/it]
 52%|█████▏    | 116828/225000 [241:24:05<144:04:16,  4.79s/it]
 52%|█████▏    | 116829/225000 [241:24:10<138:47:07,  4.62s/it]
 52%|█████▏    | 116830/225000 [241:24:14<136:46:09,  4.55s/it]
 52%|█████▏    | 116831/225000 [241:24:19<141:41:19,  4.72s/it]
 52%|█████▏    | 116832/225000 [241:24:24<141:14:49,  4.70s/it]
 52%|█████▏    | 116833/225000 [241:24:35<204:42:36,  6.81s/it]
 52%|█████▏    | 116834/225000 [241:24:40<182:16:29,  6.07s/it]
 52%|█████▏    | 116835/225000 [241:24:44<162:27:34,  5.41s/it]
 52%|█████▏    | 116836/225000 [241:24:48<152:51:27,  5.09s/it]
 52%|█████▏    | 116837/225000 [241:24:53<154:01:20,  5.13s/it]
 52%|█████▏    | 116838/225000 [241:24:58<149:14:01,  4.97s/it]
 52%|█████▏    | 116839/225000 [241:25:03<150:58:27,  5.02s/it]
 52%|█████▏    | 116840/225000 [241:25:08<146:49:47,  4.89s/it]
 52%|█████▏    | 116841/225000 [241:25:19<205:11:59,  6.83s/it]
 52%|█████▏    | 116842/225000 [241:25:27<214:37:51,  7.14s/it]
 52%|█████▏    | 116843/225000 [241:25:30<180:55:35,  6.02s/it]
 52%|█████▏    | 116844/225000 [241:25:38<200:32:00,  6.67s/it]
 52%|█████▏    | 116845/225000 [241:25:43<177:37:52,  5.91s/it]
 52%|█████▏    | 116846/225000 [241:25:46<159:54:46,  5.32s/it]
 52%|█████▏    | 116847/225000 [241:25:51<149:17:37,  4.97s/it]
 52%|█████▏    | 116848/225000 [241:25:55<147:45:15,  4.92s/it]
 52%|█████▏    | 116849/225000 [241:26:05<188:13:40,  6.27s/it]
 52%|█████▏    | 116850/225000 [241:26:09<168:38:19,  5.61s/it]
                                                               
{'loss': '0.8886', 'grad_norm': '1.034', 'learning_rate': '2.526e-05', 'epoch': '0.5193', 'num_input_tokens_seen': 402972936, 'train_runtime': '8.692e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 116850/225000 [241:26:09<168:38:19,  5.61s/it]
 52%|█████▏    | 116851/225000 [241:26:13<157:58:32,  5.26s/it]
 52%|█████▏    | 116852/225000 [241:26:17<147:54:46,  4.92s/it]
 52%|█████▏    | 116853/225000 [241:26:21<131:02:05,  4.36s/it]
 52%|█████▏    | 116854/225000 [241:26:31<181:49:32,  6.05s/it]
 52%|█████▏    | 116855/225000 [241:26:38<198:44:01,  6.62s/it]
 52%|█████▏    | 116856/225000 [241:26:42<174:43:42,  5.82s/it]
 52%|█████▏    | 116857/225000 [241:26:47<160:18:10,  5.34s/it]
 52%|█████▏    | 116858/225000 [241:26:53<171:37:37,  5.71s/it]
 52%|█████▏    | 116859/225000 [241:27:00<179:45:00,  5.98s/it]
 52%|█████▏    | 116860/225000 [241:27:03<155:05:21,  5.16s/it]
 52%|█████▏    | 116861/225000 [241:27:15<216:09:44,  7.20s/it]
 52%|█████▏    | 116862/225000 [241:27:19<188:31:20,  6.28s/it]
 52%|█████▏    | 116863/225000 [241:27:27<198:44:50,  6.62s/it]
 52%|█████▏    | 116864/225000 [241:27:30<166:33:24,  5.54s/it]
 52%|█████▏    | 116865/225000 [241:27:39<198:03:47,  6.59s/it]
 52%|█████▏    | 116866/225000 [241:27:42<171:36:49,  5.71s/it]
 52%|█████▏    | 116867/225000 [241:27:46<153:49:22,  5.12s/it]
 52%|█████▏    | 116868/225000 [241:27:49<138:29:17,  4.61s/it]
 52%|█████▏    | 116869/225000 [241:27:53<128:12:51,  4.27s/it]
 52%|█████▏    | 116870/225000 [241:28:04<188:18:29,  6.27s/it]
 52%|█████▏    | 116871/225000 [241:28:07<162:54:10,  5.42s/it]
 52%|█████▏    | 116872/225000 [241:28:12<152:26:33,  5.08s/it]
 52%|█████▏    | 116873/225000 [241:28:15<140:31:14,  4.68s/it]
 52%|█████▏    | 116874/225000 [241:28:19<134:24:07,  4.47s/it]
 52%|█████▏    | 116875/225000 [241:28:24<134:56:15,  4.49s/it]
 52%|█████▏    | 116876/225000 [241:28:28<134:10:51,  4.47s/it]
 52%|█████▏    | 116877/225000 [241:28:39<186:50:46,  6.22s/it]
 52%|█████▏    | 116878/225000 [241:28:43<170:41:54,  5.68s/it]
 52%|█████▏    | 116879/225000 [241:28:53<211:00:00,  7.03s/it]
 52%|█████▏    | 116880/225000 [241:28:58<191:26:00,  6.37s/it]
 52%|█████▏    | 116881/225000 [241:29:02<170:43:02,  5.68s/it]
 52%|█████▏    | 116882/225000 [241:29:06<154:26:39,  5.14s/it]
 52%|█████▏    | 116883/225000 [241:29:09<139:08:28,  4.63s/it]
 52%|█████▏    | 116884/225000 [241:29:14<135:58:49,  4.53s/it]
 52%|█████▏    | 116885/225000 [241:29:25<197:47:11,  6.59s/it]
 52%|█████▏    | 116886/225000 [241:29:30<183:42:16,  6.12s/it]
 52%|█████▏    | 116887/225000 [241:29:34<161:24:38,  5.37s/it]
 52%|█████▏    | 116888/225000 [241:29:38<148:04:28,  4.93s/it]
 52%|█████▏    | 116889/225000 [241:29:42<138:58:23,  4.63s/it]
 52%|█████▏    | 116890/225000 [241:29:46<133:06:20,  4.43s/it]
 52%|█████▏    | 116891/225000 [241:29:58<201:08:14,  6.70s/it]
 52%|█████▏    | 116892/225000 [241:30:04<195:45:03,  6.52s/it]
 52%|█████▏    | 116893/225000 [241:30:13<221:50:05,  7.39s/it]
 52%|█████▏    | 116894/225000 [241:30:17<188:25:50,  6.27s/it]
 52%|█████▏    | 116895/225000 [241:30:23<184:03:52,  6.13s/it]
 52%|█████▏    | 116896/225000 [241:30:27<172:14:45,  5.74s/it]
 52%|█████▏    | 116897/225000 [241:30:32<160:52:14,  5.36s/it]
 52%|█████▏    | 116898/225000 [241:30:41<196:38:36,  6.55s/it]
 52%|█████▏    | 116899/225000 [241:30:44<167:13:58,  5.57s/it]
 52%|█████▏    | 116900/225000 [241:30:48<148:56:46,  4.96s/it]
                                                               
{'loss': '0.8753', 'grad_norm': '1.222', 'learning_rate': '2.524e-05', 'epoch': '0.5196', 'num_input_tokens_seen': 403094696, 'train_runtime': '8.694e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 116900/225000 [241:30:48<148:56:46,  4.96s/it]
 52%|█████▏    | 116901/225000 [241:30:51<136:01:59,  4.53s/it]
 52%|█████▏    | 116902/225000 [241:31:02<191:10:09,  6.37s/it]
 52%|█████▏    | 116903/225000 [241:31:07<179:02:52,  5.96s/it]
 52%|█████▏    | 116904/225000 [241:31:12<171:32:40,  5.71s/it]
 52%|█████▏    | 116905/225000 [241:31:17<165:40:07,  5.52s/it]
 52%|█████▏    | 116906/225000 [241:31:22<160:02:31,  5.33s/it]
 52%|█████▏    | 116907/225000 [241:31:27<157:36:56,  5.25s/it]
 52%|█████▏    | 116908/225000 [241:31:32<149:29:05,  4.98s/it]
 52%|█████▏    | 116909/225000 [241:31:37<155:34:46,  5.18s/it]
 52%|█████▏    | 116910/225000 [241:31:45<174:24:46,  5.81s/it]
 52%|█████▏    | 116911/225000 [241:31:49<162:10:28,  5.40s/it]
 52%|█████▏    | 116912/225000 [241:31:53<150:09:13,  5.00s/it]
 52%|█████▏    | 116913/225000 [241:31:57<143:49:35,  4.79s/it]
 52%|█████▏    | 116914/225000 [241:32:03<153:14:37,  5.10s/it]
 52%|█████▏    | 116915/225000 [241:32:13<198:02:44,  6.60s/it]
 52%|█████▏    | 116916/225000 [241:32:17<174:46:46,  5.82s/it]
 52%|█████▏    | 116917/225000 [241:32:27<210:00:56,  7.00s/it]
 52%|█████▏    | 116918/225000 [241:32:31<185:52:33,  6.19s/it]
 52%|█████▏    | 116919/225000 [241:32:35<166:46:47,  5.56s/it]
 52%|█████▏    | 116920/225000 [241:32:40<155:23:13,  5.18s/it]
 52%|█████▏    | 116921/225000 [241:32:43<142:28:39,  4.75s/it]
 52%|█████▏    | 116922/225000 [241:32:54<197:45:52,  6.59s/it]
 52%|█████▏    | 116923/225000 [241:33:00<190:09:41,  6.33s/it]
 52%|█████▏    | 116924/225000 [241:33:04<169:31:39,  5.65s/it]
 52%|█████▏    | 116925/225000 [241:33:12<191:09:15,  6.37s/it]
 52%|█████▏    | 116926/225000 [241:33:17<176:21:02,  5.87s/it]
 52%|█████▏    | 116927/225000 [241:33:20<151:34:18,  5.05s/it]
 52%|█████▏    | 116928/225000 [241:33:25<148:45:13,  4.96s/it]
 52%|█████▏    | 116929/225000 [241:33:33<177:59:32,  5.93s/it]
 52%|█████▏    | 116930/225000 [241:33:39<179:05:15,  5.97s/it]
 52%|█████▏    | 116931/225000 [241:33:42<153:55:02,  5.13s/it]
 52%|█████▏    | 116932/225000 [241:33:45<134:44:39,  4.49s/it]
 52%|█████▏    | 116933/225000 [241:33:50<140:43:17,  4.69s/it]
 52%|█████▏    | 116934/225000 [241:33:59<173:28:27,  5.78s/it]
 52%|█████▏    | 116935/225000 [241:34:03<158:42:18,  5.29s/it]
 52%|█████▏    | 116936/225000 [241:34:09<170:25:22,  5.68s/it]
 52%|█████▏    | 116937/225000 [241:34:16<182:03:13,  6.06s/it]
 52%|█████▏    | 116938/225000 [241:34:20<159:24:50,  5.31s/it]
 52%|█████▏    | 116939/225000 [241:34:24<148:53:46,  4.96s/it]
 52%|█████▏    | 116940/225000 [241:34:28<140:48:34,  4.69s/it]
 52%|█████▏    | 116941/225000 [241:34:33<140:11:00,  4.67s/it]
 52%|█████▏    | 116942/225000 [241:34:45<204:20:43,  6.81s/it]
 52%|█████▏    | 116943/225000 [241:34:48<172:13:16,  5.74s/it]
 52%|█████▏    | 116944/225000 [241:34:51<147:54:19,  4.93s/it]
 52%|█████▏    | 116945/225000 [241:34:59<177:56:27,  5.93s/it]
 52%|█████▏    | 116946/225000 [241:35:02<154:41:24,  5.15s/it]
 52%|█████▏    | 116947/225000 [241:35:06<144:18:03,  4.81s/it]
 52%|█████▏    | 116948/225000 [241:35:13<162:40:19,  5.42s/it]
 52%|█████▏    | 116949/225000 [241:35:19<165:48:50,  5.52s/it]
 52%|█████▏    | 116950/225000 [241:35:23<148:38:51,  4.95s/it]
                                                               
{'loss': '0.8942', 'grad_norm': '1.176', 'learning_rate': '2.522e-05', 'epoch': '0.5198', 'num_input_tokens_seen': 403217480, 'train_runtime': '8.697e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 116950/225000 [241:35:23<148:38:51,  4.95s/it]
 52%|█████▏    | 116951/225000 [241:35:32<183:51:09,  6.13s/it]
 52%|█████▏    | 116952/225000 [241:35:38<184:16:08,  6.14s/it]
 52%|█████▏    | 116953/225000 [241:35:41<162:28:55,  5.41s/it]
 52%|█████▏    | 116954/225000 [241:35:45<146:35:21,  4.88s/it]
 52%|█████▏    | 116955/225000 [241:35:53<175:19:23,  5.84s/it]
 52%|█████▏    | 116956/225000 [241:35:59<172:46:11,  5.76s/it]
 52%|█████▏    | 116957/225000 [241:36:02<152:03:43,  5.07s/it]
 52%|█████▏    | 116958/225000 [241:36:12<194:43:27,  6.49s/it]
 52%|█████▏    | 116959/225000 [241:36:18<193:05:16,  6.43s/it]
 52%|█████▏    | 116960/225000 [241:36:22<169:55:19,  5.66s/it]
 52%|█████▏    | 116961/225000 [241:36:29<183:59:38,  6.13s/it]
 52%|█████▏    | 116962/225000 [241:36:35<180:23:01,  6.01s/it]
 52%|█████▏    | 116963/225000 [241:36:39<165:17:30,  5.51s/it]
 52%|█████▏    | 116964/225000 [241:36:44<156:20:46,  5.21s/it]
 52%|█████▏    | 116965/225000 [241:36:48<148:28:56,  4.95s/it]
 52%|█████▏    | 116966/225000 [241:36:54<151:17:46,  5.04s/it]
 52%|█████▏    | 116967/225000 [241:36:58<146:46:00,  4.89s/it]
 52%|█████▏    | 116968/225000 [241:37:03<147:05:12,  4.90s/it]
 52%|█████▏    | 116969/225000 [241:37:11<176:44:03,  5.89s/it]
 52%|█████▏    | 116970/225000 [241:37:20<199:57:30,  6.66s/it]
 52%|█████▏    | 116971/225000 [241:37:25<186:37:22,  6.22s/it]
 52%|█████▏    | 116972/225000 [241:37:30<173:17:29,  5.77s/it]
 52%|█████▏    | 116973/225000 [241:37:35<170:05:15,  5.67s/it]
 52%|█████▏    | 116974/225000 [241:37:44<198:58:16,  6.63s/it]
 52%|█████▏    | 116975/225000 [241:37:48<179:33:37,  5.98s/it]
 52%|█████▏    | 116976/225000 [241:37:53<166:32:55,  5.55s/it]
 52%|█████▏    | 116977/225000 [241:37:57<157:56:08,  5.26s/it]
 52%|█████▏    | 116978/225000 [241:38:02<151:45:42,  5.06s/it]
 52%|█████▏    | 116979/225000 [241:38:08<155:09:32,  5.17s/it]
 52%|█████▏    | 116980/225000 [241:38:19<214:47:44,  7.16s/it]
 52%|█████▏    | 116981/225000 [241:38:25<198:41:41,  6.62s/it]
 52%|█████▏    | 116982/225000 [241:38:28<171:34:02,  5.72s/it]
 52%|█████▏    | 116983/225000 [241:38:32<149:12:30,  4.97s/it]
 52%|█████▏    | 116984/225000 [241:38:35<139:09:11,  4.64s/it]
 52%|█████▏    | 116985/225000 [241:38:47<200:21:12,  6.68s/it]
 52%|█████▏    | 116986/225000 [241:38:50<167:41:50,  5.59s/it]
 52%|█████▏    | 116987/225000 [241:38:54<158:56:20,  5.30s/it]
 52%|█████▏    | 116988/225000 [241:38:59<151:31:36,  5.05s/it]
 52%|█████▏    | 116989/225000 [241:39:04<149:47:07,  4.99s/it]
 52%|█████▏    | 116990/225000 [241:39:08<143:38:01,  4.79s/it]
 52%|█████▏    | 116991/225000 [241:39:15<159:47:09,  5.33s/it]
 52%|█████▏    | 116992/225000 [241:39:25<200:50:13,  6.69s/it]
 52%|█████▏    | 116993/225000 [241:39:28<171:54:01,  5.73s/it]
 52%|█████▏    | 116994/225000 [241:39:33<165:49:26,  5.53s/it]
 52%|█████▏    | 116995/225000 [241:39:39<173:02:21,  5.77s/it]
 52%|█████▏    | 116996/225000 [241:39:45<167:13:11,  5.57s/it]
 52%|█████▏    | 116997/225000 [241:39:50<162:06:41,  5.40s/it]
 52%|█████▏    | 116998/225000 [241:39:59<201:02:09,  6.70s/it]
 52%|█████▏    | 116999/225000 [241:40:04<178:31:46,  5.95s/it]
 52%|█████▏    | 117000/225000 [241:40:08<166:58:57,  5.57s/it]
                                                               
{'loss': '0.8713', 'grad_norm': '1.135', 'learning_rate': '2.52e-05', 'epoch': '0.52', 'num_input_tokens_seen': 403338544, 'train_runtime': '8.7e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117000/225000 [241:40:08<166:58:57,  5.57s/it]
 52%|█████▏    | 117001/225000 [241:40:13<158:16:22,  5.28s/it]
 52%|█████▏    | 117002/225000 [241:40:18<154:04:24,  5.14s/it]
 52%|█████▏    | 117003/225000 [241:40:22<150:56:30,  5.03s/it]
 52%|█████▏    | 117004/225000 [241:40:27<147:45:50,  4.93s/it]
 52%|█████▏    | 117005/225000 [241:40:31<142:17:44,  4.74s/it]
 52%|█████▏    | 117006/225000 [241:40:35<134:57:54,  4.50s/it]
 52%|█████▏    | 117007/225000 [241:40:40<134:24:01,  4.48s/it]
 52%|█████▏    | 117008/225000 [241:40:44<130:11:09,  4.34s/it]
 52%|█████▏    | 117009/225000 [241:40:52<166:18:53,  5.54s/it]
 52%|█████▏    | 117010/225000 [241:41:03<211:11:14,  7.04s/it]
 52%|█████▏    | 117011/225000 [241:41:06<178:29:51,  5.95s/it]
 52%|█████▏    | 117012/225000 [241:41:16<212:15:43,  7.08s/it]
 52%|█████▏    | 117013/225000 [241:41:19<178:24:23,  5.95s/it]
 52%|█████▏    | 117014/225000 [241:41:23<162:46:18,  5.43s/it]
 52%|█████▏    | 117015/225000 [241:41:32<192:19:39,  6.41s/it]
 52%|█████▏    | 117016/225000 [241:41:35<162:37:17,  5.42s/it]
 52%|█████▏    | 117017/225000 [241:41:39<149:24:50,  4.98s/it]
 52%|█████▏    | 117018/225000 [241:41:48<187:34:59,  6.25s/it]
 52%|█████▏    | 117019/225000 [241:41:51<159:58:22,  5.33s/it]
 52%|█████▏    | 117020/225000 [241:42:00<186:02:21,  6.20s/it]
 52%|█████▏    | 117021/225000 [241:42:04<165:37:24,  5.52s/it]
 52%|█████▏    | 117022/225000 [241:42:08<151:04:30,  5.04s/it]
 52%|█████▏    | 117023/225000 [241:42:12<149:33:05,  4.99s/it]
 52%|█████▏    | 117024/225000 [241:42:17<145:56:39,  4.87s/it]
 52%|█████▏    | 117025/225000 [241:42:22<145:04:37,  4.84s/it]
 52%|█████▏    | 117026/225000 [241:42:26<137:23:00,  4.58s/it]
 52%|█████▏    | 117027/225000 [241:42:37<195:43:21,  6.53s/it]
 52%|█████▏    | 117028/225000 [241:42:42<185:37:50,  6.19s/it]
 52%|█████▏    | 117029/225000 [241:42:46<167:33:42,  5.59s/it]
 52%|█████▏    | 117030/225000 [241:42:51<158:27:04,  5.28s/it]
 52%|█████▏    | 117031/225000 [241:42:56<155:24:52,  5.18s/it]
 52%|█████▏    | 117032/225000 [241:43:01<156:50:42,  5.23s/it]
 52%|█████▏    | 117033/225000 [241:43:07<157:22:26,  5.25s/it]
 52%|█████▏    | 117034/225000 [241:43:12<161:51:58,  5.40s/it]
 52%|█████▏    | 117035/225000 [241:43:18<162:35:38,  5.42s/it]
 52%|█████▏    | 117036/225000 [241:43:26<188:30:12,  6.29s/it]
 52%|█████▏    | 117037/225000 [241:43:29<162:49:11,  5.43s/it]
 52%|█████▏    | 117038/225000 [241:43:33<146:03:10,  4.87s/it]
 52%|█████▏    | 117039/225000 [241:43:42<180:11:06,  6.01s/it]
 52%|█████▏    | 117040/225000 [241:43:47<170:58:30,  5.70s/it]
 52%|█████▏    | 117041/225000 [241:43:50<151:04:58,  5.04s/it]
 52%|█████▏    | 117042/225000 [241:43:54<137:58:13,  4.60s/it]
 52%|█████▏    | 117043/225000 [241:44:05<193:54:19,  6.47s/it]
 52%|█████▏    | 117044/225000 [241:44:10<182:34:52,  6.09s/it]
 52%|█████▏    | 117045/225000 [241:44:15<170:49:35,  5.70s/it]
 52%|█████▏    | 117046/225000 [241:44:19<161:32:33,  5.39s/it]
 52%|█████▏    | 117047/225000 [241:44:23<150:44:02,  5.03s/it]
 52%|█████▏    | 117048/225000 [241:44:28<149:12:30,  4.98s/it]
 52%|█████▏    | 117049/225000 [241:44:33<148:51:14,  4.96s/it]
 52%|█████▏    | 117050/225000 [241:44:38<147:03:53,  4.90s/it]
                                                               
{'loss': '0.899', 'grad_norm': '1.181', 'learning_rate': '2.518e-05', 'epoch': '0.5202', 'num_input_tokens_seen': 403457696, 'train_runtime': '8.703e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117050/225000 [241:44:38<147:03:53,  4.90s/it]
 52%|█████▏    | 117051/225000 [241:44:43<148:28:12,  4.95s/it]
 52%|█████▏    | 117052/225000 [241:44:48<145:18:12,  4.85s/it]
 52%|█████▏    | 117053/225000 [241:44:59<206:09:43,  6.88s/it]
 52%|█████▏    | 117054/225000 [241:45:05<193:09:03,  6.44s/it]
 52%|█████▏    | 117055/225000 [241:45:12<196:56:31,  6.57s/it]
 52%|█████▏    | 117056/225000 [241:45:15<168:34:12,  5.62s/it]
 52%|█████▏    | 117057/225000 [241:45:19<153:54:13,  5.13s/it]
 52%|█████▏    | 117058/225000 [241:45:23<140:51:13,  4.70s/it]
 52%|█████▏    | 117059/225000 [241:45:29<153:52:15,  5.13s/it]
 52%|█████▏    | 117060/225000 [241:45:37<181:15:22,  6.05s/it]
 52%|█████▏    | 117061/225000 [241:45:40<155:54:05,  5.20s/it]
 52%|█████▏    | 117062/225000 [241:45:48<180:29:12,  6.02s/it]
 52%|█████▏    | 117063/225000 [241:45:52<162:45:30,  5.43s/it]
 52%|█████▏    | 117064/225000 [241:45:55<142:08:43,  4.74s/it]
 52%|█████▏    | 117065/225000 [241:46:03<170:03:14,  5.67s/it]
 52%|█████▏    | 117066/225000 [241:46:09<167:57:34,  5.60s/it]
 52%|█████▏    | 117067/225000 [241:46:13<158:33:12,  5.29s/it]
 52%|█████▏    | 117068/225000 [241:46:18<152:11:11,  5.08s/it]
 52%|█████▏    | 117069/225000 [241:46:22<149:13:47,  4.98s/it]
 52%|█████▏    | 117070/225000 [241:46:27<148:10:52,  4.94s/it]
 52%|█████▏    | 117071/225000 [241:46:32<142:56:34,  4.77s/it]
 52%|█████▏    | 117072/225000 [241:46:37<143:53:39,  4.80s/it]
 52%|█████▏    | 117073/225000 [241:46:41<143:59:58,  4.80s/it]
 52%|█████▏    | 117074/225000 [241:46:52<194:11:15,  6.48s/it]
 52%|█████▏    | 117075/225000 [241:47:01<216:57:56,  7.24s/it]
 52%|█████▏    | 117076/225000 [241:47:04<183:12:37,  6.11s/it]
 52%|█████▏    | 117077/225000 [241:47:08<157:46:34,  5.26s/it]
 52%|█████▏    | 117078/225000 [241:47:12<149:00:10,  4.97s/it]
 52%|█████▏    | 117079/225000 [241:47:16<142:17:40,  4.75s/it]
 52%|█████▏    | 117080/225000 [241:47:28<207:04:31,  6.91s/it]
 52%|█████▏    | 117081/225000 [241:47:33<189:18:00,  6.31s/it]
 52%|█████▏    | 117082/225000 [241:47:37<168:32:44,  5.62s/it]
 52%|█████▏    | 117083/225000 [241:47:41<153:26:00,  5.12s/it]
 52%|█████▏    | 117084/225000 [241:47:53<218:57:58,  7.30s/it]
 52%|█████▏    | 117085/225000 [241:47:57<188:55:39,  6.30s/it]
 52%|█████▏    | 117086/225000 [241:48:01<168:05:40,  5.61s/it]
 52%|█████▏    | 117087/225000 [241:48:05<155:46:54,  5.20s/it]
 52%|█████▏    | 117088/225000 [241:48:10<148:17:43,  4.95s/it]
 52%|█████▏    | 117089/225000 [241:48:19<188:24:44,  6.29s/it]
 52%|█████▏    | 117090/225000 [241:48:23<165:38:39,  5.53s/it]
 52%|█████▏    | 117091/225000 [241:48:28<158:45:00,  5.30s/it]
 52%|█████▏    | 117092/225000 [241:48:31<143:54:27,  4.80s/it]
 52%|█████▏    | 117093/225000 [241:48:41<191:14:46,  6.38s/it]
 52%|█████▏    | 117094/225000 [241:48:47<184:10:02,  6.14s/it]
 52%|█████▏    | 117095/225000 [241:48:50<157:00:22,  5.24s/it]
 52%|█████▏    | 117096/225000 [241:48:58<177:07:12,  5.91s/it]
 52%|█████▏    | 117097/225000 [241:49:03<170:27:42,  5.69s/it]
 52%|█████▏    | 117098/225000 [241:49:07<152:31:29,  5.09s/it]
 52%|█████▏    | 117099/225000 [241:49:11<144:09:45,  4.81s/it]
 52%|█████▏    | 117100/225000 [241:49:15<137:53:40,  4.60s/it]
                                                               
{'loss': '0.8901', 'grad_norm': '1.327', 'learning_rate': '2.517e-05', 'epoch': '0.5204', 'num_input_tokens_seen': 403581144, 'train_runtime': '8.706e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117100/225000 [241:49:15<137:53:40,  4.60s/it]
 52%|█████▏    | 117101/225000 [241:49:19<132:30:32,  4.42s/it]
 52%|█████▏    | 117102/225000 [241:49:30<191:57:03,  6.40s/it]
 52%|█████▏    | 117103/225000 [241:49:35<179:54:12,  6.00s/it]
 52%|█████▏    | 117104/225000 [241:49:38<153:29:19,  5.12s/it]
 52%|█████▏    | 117105/225000 [241:49:45<170:30:57,  5.69s/it]
 52%|█████▏    | 117106/225000 [241:49:51<174:51:50,  5.83s/it]
 52%|█████▏    | 117107/225000 [241:49:54<150:56:56,  5.04s/it]
 52%|█████▏    | 117108/225000 [241:49:58<138:19:58,  4.62s/it]
 52%|█████▏    | 117109/225000 [241:50:02<135:21:37,  4.52s/it]
 52%|█████▏    | 117110/225000 [241:50:12<178:53:03,  5.97s/it]
 52%|█████▏    | 117111/225000 [241:50:15<155:58:22,  5.20s/it]
 52%|█████▏    | 117112/225000 [241:50:18<139:46:15,  4.66s/it]
 52%|█████▏    | 117113/225000 [241:50:26<162:50:35,  5.43s/it]
 52%|█████▏    | 117114/225000 [241:50:32<166:47:25,  5.57s/it]
 52%|█████▏    | 117115/225000 [241:50:36<158:02:02,  5.27s/it]
 52%|█████▏    | 117116/225000 [241:50:41<152:11:31,  5.08s/it]
 52%|█████▏    | 117117/225000 [241:50:48<175:13:00,  5.85s/it]
 52%|█████▏    | 117118/225000 [241:50:53<163:18:49,  5.45s/it]
 52%|█████▏    | 117119/225000 [241:50:57<150:49:46,  5.03s/it]
 52%|█████▏    | 117120/225000 [241:51:02<147:53:51,  4.94s/it]
 52%|█████▏    | 117121/225000 [241:51:06<141:18:15,  4.72s/it]
 52%|█████▏    | 117122/225000 [241:51:18<208:49:03,  6.97s/it]
 52%|█████▏    | 117123/225000 [241:51:23<190:15:38,  6.35s/it]
 52%|█████▏    | 117124/225000 [241:51:29<188:38:05,  6.30s/it]
 52%|█████▏    | 117125/225000 [241:51:33<165:22:11,  5.52s/it]
 52%|█████▏    | 117126/225000 [241:51:41<187:53:34,  6.27s/it]
 52%|█████▏    | 117127/225000 [241:51:44<161:15:45,  5.38s/it]
 52%|█████▏    | 117128/225000 [241:51:48<143:24:34,  4.79s/it]
 52%|█████▏    | 117129/225000 [241:52:00<215:50:19,  7.20s/it]
 52%|█████▏    | 117130/225000 [241:52:04<182:24:09,  6.09s/it]
 52%|█████▏    | 117131/225000 [241:52:08<165:36:57,  5.53s/it]
 52%|█████▏    | 117132/225000 [241:52:13<158:38:50,  5.29s/it]
 52%|█████▏    | 117133/225000 [241:52:18<156:20:05,  5.22s/it]
 52%|█████▏    | 117134/225000 [241:52:23<151:15:57,  5.05s/it]
 52%|█████▏    | 117135/225000 [241:52:28<154:53:44,  5.17s/it]
 52%|█████▏    | 117136/225000 [241:52:37<187:02:31,  6.24s/it]
 52%|█████▏    | 117137/225000 [241:52:46<212:12:41,  7.08s/it]
 52%|█████▏    | 117138/225000 [241:52:50<189:33:07,  6.33s/it]
 52%|█████▏    | 117139/225000 [241:52:55<172:52:52,  5.77s/it]
 52%|█████▏    | 117140/225000 [241:53:00<164:46:59,  5.50s/it]
 52%|█████▏    | 117141/225000 [241:53:04<157:11:39,  5.25s/it]
 52%|█████▏    | 117142/225000 [241:53:09<150:13:29,  5.01s/it]
 52%|█████▏    | 117143/225000 [241:53:14<155:09:32,  5.18s/it]
 52%|█████▏    | 117144/225000 [241:53:19<151:13:02,  5.05s/it]
 52%|█████▏    | 117145/225000 [241:53:24<144:50:16,  4.83s/it]
 52%|█████▏    | 117146/225000 [241:53:33<190:54:28,  6.37s/it]
 52%|█████▏    | 117147/225000 [241:53:44<224:05:19,  7.48s/it]
 52%|█████▏    | 117148/225000 [241:53:50<213:16:25,  7.12s/it]
 52%|█████▏    | 117149/225000 [241:53:57<217:13:45,  7.25s/it]
 52%|█████▏    | 117150/225000 [241:54:01<184:22:35,  6.15s/it]
                                                               
{'loss': '0.874', 'grad_norm': '1.211', 'learning_rate': '2.515e-05', 'epoch': '0.5207', 'num_input_tokens_seen': 403704864, 'train_runtime': '8.708e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117150/225000 [241:54:01<184:22:35,  6.15s/it]
 52%|█████▏    | 117151/225000 [241:54:05<168:59:30,  5.64s/it]
 52%|█████▏    | 117152/225000 [241:54:10<158:00:49,  5.27s/it]
 52%|█████▏    | 117153/225000 [241:54:20<198:47:00,  6.64s/it]
 52%|█████▏    | 117154/225000 [241:54:24<173:53:36,  5.80s/it]
 52%|█████▏    | 117155/225000 [241:54:28<158:56:39,  5.31s/it]
 52%|█████▏    | 117156/225000 [241:54:32<151:16:27,  5.05s/it]
 52%|█████▏    | 117157/225000 [241:54:36<143:16:11,  4.78s/it]
 52%|█████▏    | 117158/225000 [241:54:47<196:02:05,  6.54s/it]
 52%|█████▏    | 117159/225000 [241:54:51<174:44:59,  5.83s/it]
 52%|█████▏    | 117160/225000 [241:54:55<155:12:03,  5.18s/it]
 52%|█████▏    | 117161/225000 [241:54:59<149:45:07,  5.00s/it]
 52%|█████▏    | 117162/225000 [241:55:04<144:41:09,  4.83s/it]
 52%|█████▏    | 117163/225000 [241:55:09<145:08:01,  4.85s/it]
 52%|█████▏    | 117164/225000 [241:55:13<142:20:51,  4.75s/it]
 52%|█████▏    | 117165/225000 [241:55:18<146:51:28,  4.90s/it]
 52%|█████▏    | 117166/225000 [241:55:23<144:03:41,  4.81s/it]
 52%|█████▏    | 117167/225000 [241:55:35<205:54:34,  6.87s/it]
 52%|█████▏    | 117168/225000 [241:55:41<200:50:22,  6.71s/it]
 52%|█████▏    | 117169/225000 [241:55:46<182:39:46,  6.10s/it]
 52%|█████▏    | 117170/225000 [241:55:50<163:05:54,  5.45s/it]
 52%|█████▏    | 117171/225000 [241:55:54<154:01:04,  5.14s/it]
 52%|█████▏    | 117172/225000 [241:55:58<146:26:55,  4.89s/it]
 52%|█████▏    | 117173/225000 [241:56:09<196:41:27,  6.57s/it]
 52%|█████▏    | 117174/225000 [241:56:13<178:15:37,  5.95s/it]
 52%|█████▏    | 117175/225000 [241:56:17<157:47:33,  5.27s/it]
 52%|█████▏    | 117176/225000 [241:56:23<161:03:24,  5.38s/it]
 52%|█████▏    | 117177/225000 [241:56:31<190:26:18,  6.36s/it]
 52%|█████▏    | 117178/225000 [241:56:35<163:40:56,  5.47s/it]
 52%|█████▏    | 117179/225000 [241:56:38<146:24:51,  4.89s/it]
 52%|█████▏    | 117180/225000 [241:56:42<136:36:41,  4.56s/it]
 52%|█████▏    | 117181/225000 [241:56:46<132:38:02,  4.43s/it]
 52%|█████▏    | 117182/225000 [241:56:51<133:20:23,  4.45s/it]
 52%|█████▏    | 117183/225000 [241:57:00<176:43:32,  5.90s/it]
 52%|█████▏    | 117184/225000 [241:57:09<204:38:37,  6.83s/it]
 52%|█████▏    | 117185/225000 [241:57:13<181:31:20,  6.06s/it]
 52%|█████▏    | 117186/225000 [241:57:17<162:04:09,  5.41s/it]
 52%|█████▏    | 117187/225000 [241:57:21<145:36:50,  4.86s/it]
 52%|█████▏    | 117188/225000 [241:57:25<138:00:05,  4.61s/it]
 52%|█████▏    | 117189/225000 [241:57:29<134:39:44,  4.50s/it]
 52%|█████▏    | 117190/225000 [241:57:35<147:51:27,  4.94s/it]
 52%|█████▏    | 117191/225000 [241:57:46<206:10:27,  6.88s/it]
 52%|█████▏    | 117192/225000 [241:57:50<172:55:53,  5.77s/it]
 52%|█████▏    | 117193/225000 [241:57:58<197:33:52,  6.60s/it]
 52%|█████▏    | 117194/225000 [241:58:02<170:46:56,  5.70s/it]
 52%|█████▏    | 117195/225000 [241:58:06<154:08:12,  5.15s/it]
 52%|█████▏    | 117196/225000 [241:58:10<150:56:02,  5.04s/it]
 52%|█████▏    | 117197/225000 [241:58:15<148:23:18,  4.96s/it]
 52%|█████▏    | 117198/225000 [241:58:20<146:15:29,  4.88s/it]
 52%|█████▏    | 117199/225000 [241:58:27<164:15:26,  5.49s/it]
 52%|█████▏    | 117200/225000 [241:58:33<168:09:46,  5.62s/it]
                                                               
{'loss': '0.8823', 'grad_norm': '1.325', 'learning_rate': '2.513e-05', 'epoch': '0.5209', 'num_input_tokens_seen': 403822800, 'train_runtime': '8.711e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117200/225000 [241:58:33<168:09:46,  5.62s/it]
 52%|█████▏    | 117201/225000 [241:58:38<162:40:31,  5.43s/it]
 52%|█████▏    | 117202/225000 [241:58:43<159:08:21,  5.31s/it]
 52%|█████▏    | 117203/225000 [241:58:47<153:31:38,  5.13s/it]
 52%|█████▏    | 117204/225000 [241:58:55<175:39:11,  5.87s/it]
 52%|█████▏    | 117205/225000 [241:59:02<188:36:11,  6.30s/it]
 52%|█████▏    | 117206/225000 [241:59:06<165:02:55,  5.51s/it]
 52%|█████▏    | 117207/225000 [241:59:12<167:53:03,  5.61s/it]
 52%|█████▏    | 117208/225000 [241:59:21<196:23:09,  6.56s/it]
 52%|█████▏    | 117209/225000 [241:59:24<168:58:42,  5.64s/it]
 52%|█████▏    | 117210/225000 [241:59:33<200:23:41,  6.69s/it]
 52%|█████▏    | 117211/225000 [241:59:37<175:41:47,  5.87s/it]
 52%|█████▏    | 117212/225000 [241:59:41<158:20:26,  5.29s/it]
 52%|█████▏    | 117213/225000 [241:59:45<148:31:07,  4.96s/it]
 52%|█████▏    | 117214/225000 [241:59:49<139:40:01,  4.66s/it]
 52%|█████▏    | 117215/225000 [241:59:54<140:57:47,  4.71s/it]
 52%|█████▏    | 117216/225000 [242:00:05<194:58:18,  6.51s/it]
 52%|█████▏    | 117217/225000 [242:00:08<165:26:18,  5.53s/it]
 52%|█████▏    | 117218/225000 [242:00:11<146:20:21,  4.89s/it]
 52%|█████▏    | 117219/225000 [242:00:21<187:02:07,  6.25s/it]
 52%|█████▏    | 117220/225000 [242:00:24<160:20:48,  5.36s/it]
 52%|█████▏    | 117221/225000 [242:00:28<147:42:51,  4.93s/it]
 52%|█████▏    | 117222/225000 [242:00:32<142:57:35,  4.78s/it]
 52%|█████▏    | 117223/225000 [242:00:37<143:51:15,  4.81s/it]
 52%|█████▏    | 117224/225000 [242:00:42<140:20:18,  4.69s/it]
 52%|█████▏    | 117225/225000 [242:00:46<138:41:32,  4.63s/it]
 52%|█████▏    | 117226/225000 [242:00:57<192:04:47,  6.42s/it]
 52%|█████▏    | 117227/225000 [242:01:04<197:37:01,  6.60s/it]
 52%|█████▏    | 117228/225000 [242:01:09<184:31:51,  6.16s/it]
 52%|█████▏    | 117229/225000 [242:01:15<186:35:56,  6.23s/it]
 52%|█████▏    | 117230/225000 [242:01:19<167:36:35,  5.60s/it]
 52%|█████▏    | 117231/225000 [242:01:28<192:39:42,  6.44s/it]
 52%|█████▏    | 117232/225000 [242:01:31<164:34:23,  5.50s/it]
 52%|█████▏    | 117233/225000 [242:01:38<173:39:41,  5.80s/it]
 52%|█████▏    | 117234/225000 [242:01:41<152:04:13,  5.08s/it]
 52%|█████▏    | 117235/225000 [242:01:49<174:49:34,  5.84s/it]
 52%|█████▏    | 117236/225000 [242:01:52<156:25:35,  5.23s/it]
 52%|█████▏    | 117237/225000 [242:01:58<158:54:47,  5.31s/it]
 52%|█████▏    | 117238/225000 [242:02:06<181:53:04,  6.08s/it]
 52%|█████▏    | 117239/225000 [242:02:10<165:10:17,  5.52s/it]
 52%|█████▏    | 117240/225000 [242:02:15<156:57:30,  5.24s/it]
 52%|█████▏    | 117241/225000 [242:02:20<155:12:13,  5.19s/it]
 52%|█████▏    | 117242/225000 [242:02:30<202:49:45,  6.78s/it]
 52%|█████▏    | 117243/225000 [242:02:36<194:53:03,  6.51s/it]
 52%|█████▏    | 117244/225000 [242:02:40<170:37:01,  5.70s/it]
 52%|█████▏    | 117245/225000 [242:02:45<168:47:28,  5.64s/it]
 52%|█████▏    | 117246/225000 [242:02:51<167:09:14,  5.58s/it]
 52%|█████▏    | 117247/225000 [242:02:54<147:26:50,  4.93s/it]
 52%|█████▏    | 117248/225000 [242:02:57<131:27:33,  4.39s/it]
 52%|█████▏    | 117249/225000 [242:03:04<153:04:39,  5.11s/it]
 52%|█████▏    | 117250/225000 [242:03:14<198:20:32,  6.63s/it]
                                                               
{'loss': '0.8932', 'grad_norm': '1.111', 'learning_rate': '2.511e-05', 'epoch': '0.5211', 'num_input_tokens_seen': 403942568, 'train_runtime': '8.714e+05', 'train_tokens_per_second': '463.6'}

 52%|█████▏    | 117250/225000 [242:03:14<198:20:32,  6.63s/it]
 52%|█████▏    | 117251/225000 [242:03:18<174:20:44,  5.83s/it]
 52%|█████▏    | 117252/225000 [242:03:30<227:01:30,  7.59s/it]
 52%|█████▏    | 117253/225000 [242:03:33<188:15:50,  6.29s/it]
 52%|█████▏    | 117254/225000 [242:03:37<165:54:44,  5.54s/it]
 52%|█████▏    | 117255/225000 [242:03:42<156:38:53,  5.23s/it]
 52%|█████▏    | 117256/225000 [242:03:50<182:39:32,  6.10s/it]
 52%|█████▏    | 117257/225000 [242:03:53<159:41:51,  5.34s/it]
 52%|█████▏    | 117258/225000 [242:03:57<145:13:09,  4.85s/it]
 52%|█████▏    | 117259/225000 [242:04:01<142:14:32,  4.75s/it]
 52%|█████▏    | 117260/225000 [242:04:08<161:06:39,  5.38s/it]
 52%|█████▏    | 117261/225000 [242:04:19<208:14:16,  6.96s/it]
 52%|█████▏    | 117262/225000 [242:04:23<178:19:10,  5.96s/it]
 52%|█████▏    | 117263/225000 [242:04:31<198:29:21,  6.63s/it]
 52%|█████▏    | 117264/225000 [242:04:35<174:24:23,  5.83s/it]
 52%|█████▏    | 117265/225000 [242:04:39<155:55:26,  5.21s/it]
 52%|█████▏    | 117266/225000 [242:04:42<142:16:56,  4.75s/it]
 52%|█████▏    | 117267/225000 [242:04:53<200:29:33,  6.70s/it]
 52%|█████▏    | 117268/225000 [242:04:56<164:04:12,  5.48s/it]
 52%|█████▏    | 117269/225000 [242:04:59<143:32:17,  4.80s/it]
 52%|█████▏    | 117270/225000 [242:05:10<195:03:15,  6.52s/it]
 52%|█████▏    | 117271/225000 [242:05:14<176:51:00,  5.91s/it]
 52%|█████▏    | 117272/225000 [242:05:18<160:12:08,  5.35s/it]
 52%|█████▏    | 117273/225000 [242:05:22<144:14:10,  4.82s/it]
 52%|█████▏    | 117274/225000 [242:05:26<138:19:44,  4.62s/it]
 52%|█████▏    | 117275/225000 [242:05:32<150:45:52,  5.04s/it]
 52%|█████▏    | 117276/225000 [242:05:43<200:37:57,  6.70s/it]
 52%|█████▏    | 117277/225000 [242:05:46<170:30:44,  5.70s/it]
 52%|█████▏    | 117278/225000 [242:05:49<149:49:41,  5.01s/it]
 52%|█████▏    | 117279/225000 [242:05:58<185:39:15,  6.20s/it]
 52%|█████▏    | 117280/225000 [242:06:03<170:15:28,  5.69s/it]
 52%|█████▏    | 117281/225000 [242:06:08<161:49:07,  5.41s/it]
 52%|█████▏    | 117282/225000 [242:06:13<156:28:43,  5.23s/it]
 52%|█████▏    | 117283/225000 [242:06:17<146:07:13,  4.88s/it]
 52%|█████▏    | 117284/225000 [242:06:21<143:32:07,  4.80s/it]
 52%|█████▏    | 117285/225000 [242:06:26<140:22:50,  4.69s/it]
 52%|█████▏    | 117286/225000 [242:06:30<140:38:45,  4.70s/it]
 52%|█████▏    | 117287/225000 [242:06:37<154:44:09,  5.17s/it]
 52%|█████▏    | 117288/225000 [242:06:49<220:13:26,  7.36s/it]
 52%|█████▏    | 117289/225000 [242:06:53<193:29:47,  6.47s/it]
 52%|█████▏    | 117290/225000 [242:07:01<205:34:11,  6.87s/it]
 52%|█████▏    | 117291/225000 [242:07:10<221:44:32,  7.41s/it]
 52%|█████▏    | 117292/225000 [242:07:14<190:23:14,  6.36s/it]
 52%|█████▏    | 117293/225000 [242:07:19<180:47:48,  6.04s/it]
 52%|█████▏    | 117294/225000 [242:07:23<163:47:43,  5.47s/it]
 52%|█████▏    | 117295/225000 [242:07:32<195:40:12,  6.54s/it]
 52%|█████▏    | 117296/225000 [242:07:38<184:56:10,  6.18s/it]
 52%|█████▏    | 117297/225000 [242:07:48<225:25:59,  7.54s/it]
 52%|█████▏    | 117298/225000 [242:07:51<184:06:01,  6.15s/it]
 52%|█████▏    | 117299/225000 [242:07:58<192:08:25,  6.42s/it]
 52%|█████▏    | 117300/225000 [242:08:03<174:04:27,  5.82s/it]
                                                               
{'loss': '0.8811', 'grad_norm': '1.224', 'learning_rate': '2.509e-05', 'epoch': '0.5213', 'num_input_tokens_seen': 404066000, 'train_runtime': '8.717e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117300/225000 [242:08:03<174:04:27,  5.82s/it]
 52%|█████▏    | 117301/225000 [242:08:10<187:26:42,  6.27s/it]
 52%|█████▏    | 117302/225000 [242:08:14<163:10:51,  5.45s/it]
 52%|█████▏    | 117303/225000 [242:08:18<148:50:18,  4.98s/it]
 52%|█████▏    | 117304/225000 [242:08:24<159:40:00,  5.34s/it]
 52%|█████▏    | 117305/225000 [242:08:30<169:01:00,  5.65s/it]
 52%|█████▏    | 117306/225000 [242:08:35<159:53:25,  5.34s/it]
 52%|█████▏    | 117307/225000 [242:08:39<151:24:47,  5.06s/it]
 52%|█████▏    | 117308/225000 [242:08:44<145:43:19,  4.87s/it]
 52%|█████▏    | 117309/225000 [242:08:52<179:52:37,  6.01s/it]
 52%|█████▏    | 117310/225000 [242:09:00<191:36:03,  6.41s/it]
 52%|█████▏    | 117311/225000 [242:09:03<162:36:52,  5.44s/it]
 52%|█████▏    | 117312/225000 [242:09:17<242:03:29,  8.09s/it]
 52%|█████▏    | 117313/225000 [242:09:22<214:14:20,  7.16s/it]
 52%|█████▏    | 117314/225000 [242:09:31<228:15:39,  7.63s/it]
 52%|█████▏    | 117315/225000 [242:09:36<204:53:05,  6.85s/it]
 52%|█████▏    | 117316/225000 [242:09:39<171:06:48,  5.72s/it]
 52%|█████▏    | 117317/225000 [242:09:45<171:04:28,  5.72s/it]
 52%|█████▏    | 117318/225000 [242:09:52<182:27:49,  6.10s/it]
 52%|█████▏    | 117319/225000 [242:09:54<153:39:24,  5.14s/it]
 52%|█████▏    | 117320/225000 [242:10:01<165:58:24,  5.55s/it]
 52%|█████▏    | 117321/225000 [242:10:07<173:53:13,  5.81s/it]
 52%|█████▏    | 117322/225000 [242:10:10<147:12:39,  4.92s/it]
 52%|█████▏    | 117323/225000 [242:10:15<141:59:53,  4.75s/it]
 52%|█████▏    | 117324/225000 [242:10:19<141:21:41,  4.73s/it]
 52%|█████▏    | 117325/225000 [242:10:23<136:59:14,  4.58s/it]
 52%|█████▏    | 117326/225000 [242:10:28<140:10:35,  4.69s/it]
 52%|█████▏    | 117327/225000 [242:10:33<139:09:02,  4.65s/it]
 52%|█████▏    | 117328/225000 [242:10:37<136:06:53,  4.55s/it]
 52%|█████▏    | 117329/225000 [242:10:42<140:41:54,  4.70s/it]
 52%|█████▏    | 117330/225000 [242:10:53<196:21:42,  6.57s/it]
 52%|█████▏    | 117331/225000 [242:10:59<193:06:01,  6.46s/it]
 52%|█████▏    | 117332/225000 [242:11:03<169:26:48,  5.67s/it]
 52%|█████▏    | 117333/225000 [242:11:06<146:54:56,  4.91s/it]
 52%|█████▏    | 117334/225000 [242:11:11<140:28:26,  4.70s/it]
 52%|█████▏    | 117335/225000 [242:11:19<173:57:03,  5.82s/it]
 52%|█████▏    | 117336/225000 [242:11:24<165:34:36,  5.54s/it]
 52%|█████▏    | 117337/225000 [242:11:28<154:14:02,  5.16s/it]
 52%|█████▏    | 117338/225000 [242:11:33<154:34:26,  5.17s/it]
 52%|█████▏    | 117339/225000 [242:11:38<149:19:46,  4.99s/it]
 52%|█████▏    | 117340/225000 [242:11:47<189:05:14,  6.32s/it]
 52%|█████▏    | 117341/225000 [242:11:54<187:59:18,  6.29s/it]
 52%|█████▏    | 117342/225000 [242:11:58<174:51:24,  5.85s/it]
 52%|█████▏    | 117343/225000 [242:12:04<173:30:28,  5.80s/it]
 52%|█████▏    | 117344/225000 [242:12:10<171:46:45,  5.74s/it]
 52%|█████▏    | 117345/225000 [242:12:15<163:07:54,  5.46s/it]
 52%|█████▏    | 117346/225000 [242:12:26<220:39:36,  7.38s/it]
 52%|█████▏    | 117347/225000 [242:12:30<188:29:26,  6.30s/it]
 52%|█████▏    | 117348/225000 [242:12:34<163:52:19,  5.48s/it]
 52%|█████▏    | 117349/225000 [242:12:40<174:10:42,  5.82s/it]
 52%|█████▏    | 117350/225000 [242:12:47<177:19:18,  5.93s/it]
                                                               
{'loss': '0.8787', 'grad_norm': '1.039', 'learning_rate': '2.507e-05', 'epoch': '0.5216', 'num_input_tokens_seen': 404190896, 'train_runtime': '8.72e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117350/225000 [242:12:47<177:19:18,  5.93s/it]
 52%|█████▏    | 117351/225000 [242:12:50<155:01:13,  5.18s/it]
 52%|█████▏    | 117352/225000 [242:12:56<163:49:48,  5.48s/it]
 52%|█████▏    | 117353/225000 [242:13:00<153:34:43,  5.14s/it]
 52%|█████▏    | 117354/225000 [242:13:06<152:52:28,  5.11s/it]
 52%|█████▏    | 117355/225000 [242:13:10<148:52:19,  4.98s/it]
 52%|█████▏    | 117356/225000 [242:13:15<143:17:00,  4.79s/it]
 52%|█████▏    | 117357/225000 [242:13:20<144:52:43,  4.85s/it]
 52%|█████▏    | 117358/225000 [242:13:24<141:20:26,  4.73s/it]
 52%|█████▏    | 117359/225000 [242:13:29<139:33:40,  4.67s/it]
 52%|█████▏    | 117360/225000 [242:13:34<145:33:47,  4.87s/it]
 52%|█████▏    | 117361/225000 [242:13:39<147:22:09,  4.93s/it]
 52%|█████▏    | 117362/225000 [242:13:44<147:56:23,  4.95s/it]
 52%|█████▏    | 117363/225000 [242:13:54<193:24:06,  6.47s/it]
 52%|█████▏    | 117364/225000 [242:14:05<232:07:15,  7.76s/it]
 52%|█████▏    | 117365/225000 [242:14:09<198:49:22,  6.65s/it]
 52%|█████▏    | 117366/225000 [242:14:13<177:07:31,  5.92s/it]
 52%|█████▏    | 117367/225000 [242:14:17<159:21:37,  5.33s/it]
 52%|█████▏    | 117368/225000 [242:14:26<192:51:37,  6.45s/it]
 52%|█████▏    | 117369/225000 [242:14:31<177:06:33,  5.92s/it]
 52%|█████▏    | 117370/225000 [242:14:34<153:27:53,  5.13s/it]
 52%|█████▏    | 117371/225000 [242:14:37<137:09:08,  4.59s/it]
 52%|█████▏    | 117372/225000 [242:14:48<188:09:52,  6.29s/it]
 52%|█████▏    | 117373/225000 [242:14:52<174:55:04,  5.85s/it]
 52%|█████▏    | 117374/225000 [242:14:55<150:05:34,  5.02s/it]
 52%|█████▏    | 117375/225000 [242:15:04<182:40:24,  6.11s/it]
 52%|█████▏    | 117376/225000 [242:15:08<160:03:35,  5.35s/it]
 52%|█████▏    | 117377/225000 [242:15:11<143:48:50,  4.81s/it]
 52%|█████▏    | 117378/225000 [242:15:19<172:46:58,  5.78s/it]
 52%|█████▏    | 117379/225000 [242:15:26<177:40:01,  5.94s/it]
 52%|█████▏    | 117380/225000 [242:15:30<159:23:35,  5.33s/it]
 52%|█████▏    | 117381/225000 [242:15:36<173:08:39,  5.79s/it]
 52%|█████▏    | 117382/225000 [242:15:40<153:10:33,  5.12s/it]
 52%|█████▏    | 117383/225000 [242:15:44<139:54:10,  4.68s/it]
 52%|█████▏    | 117384/225000 [242:15:50<152:17:51,  5.09s/it]
 52%|█████▏    | 117385/225000 [242:15:59<192:32:02,  6.44s/it]
 52%|█████▏    | 117386/225000 [242:16:03<165:50:46,  5.55s/it]
 52%|█████▏    | 117387/225000 [242:16:06<149:07:50,  4.99s/it]
 52%|█████▏    | 117388/225000 [242:16:14<174:28:15,  5.84s/it]
 52%|█████▏    | 117389/225000 [242:16:18<151:42:36,  5.08s/it]
 52%|█████▏    | 117390/225000 [242:16:25<170:08:14,  5.69s/it]
 52%|█████▏    | 117391/225000 [242:16:31<175:14:10,  5.86s/it]
 52%|█████▏    | 117392/225000 [242:16:37<178:33:31,  5.97s/it]
 52%|█████▏    | 117393/225000 [242:16:41<163:11:24,  5.46s/it]
 52%|█████▏    | 117394/225000 [242:16:46<152:01:01,  5.09s/it]
 52%|█████▏    | 117395/225000 [242:16:51<156:02:08,  5.22s/it]
 52%|█████▏    | 117396/225000 [242:17:03<215:55:20,  7.22s/it]
 52%|█████▏    | 117397/225000 [242:17:07<182:41:18,  6.11s/it]
 52%|█████▏    | 117398/225000 [242:17:10<161:47:01,  5.41s/it]
 52%|█████▏    | 117399/225000 [242:17:16<162:15:26,  5.43s/it]
 52%|█████▏    | 117400/225000 [242:17:21<159:38:31,  5.34s/it]
                                                               
{'loss': '0.8756', 'grad_norm': '1.115', 'learning_rate': '2.506e-05', 'epoch': '0.5218', 'num_input_tokens_seen': 404312648, 'train_runtime': '8.722e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117400/225000 [242:17:21<159:38:31,  5.34s/it]
 52%|█████▏    | 117401/225000 [242:17:26<155:57:22,  5.22s/it]
 52%|█████▏    | 117402/225000 [242:17:30<149:17:55,  5.00s/it]
 52%|█████▏    | 117403/225000 [242:17:36<153:42:39,  5.14s/it]
 52%|█████▏    | 117404/225000 [242:17:47<205:47:40,  6.89s/it]
 52%|█████▏    | 117405/225000 [242:17:52<186:38:29,  6.24s/it]
 52%|█████▏    | 117406/225000 [242:17:56<171:13:06,  5.73s/it]
 52%|█████▏    | 117407/225000 [242:18:00<155:15:44,  5.19s/it]
 52%|█████▏    | 117408/225000 [242:18:05<152:31:51,  5.10s/it]
 52%|█████▏    | 117409/225000 [242:18:17<211:58:00,  7.09s/it]
 52%|█████▏    | 117410/225000 [242:18:22<195:55:22,  6.56s/it]
 52%|█████▏    | 117411/225000 [242:18:31<215:52:14,  7.22s/it]
 52%|█████▏    | 117412/225000 [242:18:34<180:58:38,  6.06s/it]
 52%|█████▏    | 117413/225000 [242:18:38<159:11:05,  5.33s/it]
 52%|█████▏    | 117414/225000 [242:18:47<193:58:06,  6.49s/it]
 52%|█████▏    | 117415/225000 [242:18:51<169:27:08,  5.67s/it]
 52%|█████▏    | 117416/225000 [242:18:55<153:33:21,  5.14s/it]
 52%|█████▏    | 117417/225000 [242:18:59<147:37:49,  4.94s/it]
 52%|█████▏    | 117418/225000 [242:19:07<177:45:36,  5.95s/it]
 52%|█████▏    | 117419/225000 [242:19:14<188:43:29,  6.32s/it]
 52%|█████▏    | 117420/225000 [242:19:18<159:55:03,  5.35s/it]
 52%|█████▏    | 117421/225000 [242:19:25<180:10:45,  6.03s/it]
 52%|█████▏    | 117422/225000 [242:19:28<155:12:01,  5.19s/it]
 52%|█████▏    | 117423/225000 [242:19:35<168:57:07,  5.65s/it]
 52%|█████▏    | 117424/225000 [242:19:40<163:50:13,  5.48s/it]
 52%|█████▏    | 117425/225000 [242:19:44<151:47:07,  5.08s/it]
 52%|█████▏    | 117426/225000 [242:19:54<196:23:00,  6.57s/it]
 52%|█████▏    | 117427/225000 [242:19:59<175:29:52,  5.87s/it]
 52%|█████▏    | 117428/225000 [242:20:03<157:22:03,  5.27s/it]
 52%|█████▏    | 117429/225000 [242:20:10<180:46:34,  6.05s/it]
 52%|█████▏    | 117430/225000 [242:20:14<157:06:49,  5.26s/it]
 52%|█████▏    | 117431/225000 [242:20:23<192:16:49,  6.44s/it]
 52%|█████▏    | 117432/225000 [242:20:26<165:37:25,  5.54s/it]
 52%|█████▏    | 117433/225000 [242:20:30<146:39:07,  4.91s/it]
 52%|█████▏    | 117434/225000 [242:20:34<137:56:39,  4.62s/it]
 52%|█████▏    | 117435/225000 [242:20:44<185:49:38,  6.22s/it]
 52%|█████▏    | 117436/225000 [242:20:47<162:33:27,  5.44s/it]
 52%|█████▏    | 117437/225000 [242:20:55<184:20:09,  6.17s/it]
 52%|█████▏    | 117438/225000 [242:20:59<160:45:35,  5.38s/it]
 52%|█████▏    | 117439/225000 [242:21:05<164:59:12,  5.52s/it]
 52%|█████▏    | 117440/225000 [242:21:09<155:43:13,  5.21s/it]
 52%|█████▏    | 117441/225000 [242:21:13<145:27:23,  4.87s/it]
 52%|█████▏    | 117442/225000 [242:21:20<166:39:16,  5.58s/it]
 52%|█████▏    | 117443/225000 [242:21:25<152:43:56,  5.11s/it]
 52%|█████▏    | 117444/225000 [242:21:32<178:10:59,  5.96s/it]
 52%|█████▏    | 117445/225000 [242:21:41<203:02:49,  6.80s/it]
 52%|█████▏    | 117446/225000 [242:21:45<172:58:31,  5.79s/it]
 52%|█████▏    | 117447/225000 [242:21:48<152:53:24,  5.12s/it]
 52%|█████▏    | 117448/225000 [242:22:01<220:27:07,  7.38s/it]
 52%|█████▏    | 117449/225000 [242:22:04<186:48:19,  6.25s/it]
 52%|█████▏    | 117450/225000 [242:22:08<159:06:31,  5.33s/it]
                                                               
{'loss': '0.8891', 'grad_norm': '1.07', 'learning_rate': '2.504e-05', 'epoch': '0.522', 'num_input_tokens_seen': 404434416, 'train_runtime': '8.725e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117450/225000 [242:22:08<159:06:31,  5.33s/it]
 52%|█████▏    | 117451/225000 [242:22:11<144:15:09,  4.83s/it]
 52%|█████▏    | 117452/225000 [242:22:15<135:46:30,  4.54s/it]
 52%|█████▏    | 117453/225000 [242:22:19<133:25:52,  4.47s/it]
 52%|█████▏    | 117454/225000 [242:22:31<197:28:10,  6.61s/it]
 52%|█████▏    | 117455/225000 [242:22:35<173:13:54,  5.80s/it]
 52%|█████▏    | 117456/225000 [242:22:41<175:23:41,  5.87s/it]
 52%|█████▏    | 117457/225000 [242:22:45<160:25:44,  5.37s/it]
 52%|█████▏    | 117458/225000 [242:22:49<146:45:11,  4.91s/it]
 52%|█████▏    | 117459/225000 [242:22:56<165:10:00,  5.53s/it]
 52%|█████▏    | 117460/225000 [242:23:04<187:32:14,  6.28s/it]
 52%|█████▏    | 117461/225000 [242:23:08<166:32:24,  5.58s/it]
 52%|█████▏    | 117462/225000 [242:23:12<153:53:17,  5.15s/it]
 52%|█████▏    | 117463/225000 [242:23:16<144:49:19,  4.85s/it]
 52%|█████▏    | 117464/225000 [242:23:21<140:42:54,  4.71s/it]
 52%|█████▏    | 117465/225000 [242:23:26<146:13:21,  4.90s/it]
 52%|█████▏    | 117466/225000 [242:23:37<204:49:43,  6.86s/it]
 52%|█████▏    | 117467/225000 [242:23:42<185:44:28,  6.22s/it]
 52%|█████▏    | 117468/225000 [242:23:46<168:44:24,  5.65s/it]
 52%|█████▏    | 117469/225000 [242:23:50<153:56:42,  5.15s/it]
 52%|█████▏    | 117470/225000 [242:23:55<152:37:14,  5.11s/it]
 52%|█████▏    | 117471/225000 [242:24:01<152:55:07,  5.12s/it]
 52%|█████▏    | 117472/225000 [242:24:05<149:18:28,  5.00s/it]
 52%|█████▏    | 117473/225000 [242:24:10<144:04:23,  4.82s/it]
 52%|█████▏    | 117474/225000 [242:24:17<161:19:47,  5.40s/it]
 52%|█████▏    | 117475/225000 [242:24:29<225:53:35,  7.56s/it]
 52%|█████▏    | 117476/225000 [242:24:36<215:38:25,  7.22s/it]
 52%|█████▏    | 117477/225000 [242:24:39<184:08:26,  6.17s/it]
 52%|█████▏    | 117478/225000 [242:24:48<208:59:49,  7.00s/it]
 52%|█████▏    | 117479/225000 [242:24:53<192:16:46,  6.44s/it]
 52%|█████▏    | 117480/225000 [242:24:58<177:08:11,  5.93s/it]
 52%|█████▏    | 117481/225000 [242:25:02<162:38:05,  5.45s/it]
 52%|█████▏    | 117482/225000 [242:25:07<152:25:29,  5.10s/it]
 52%|█████▏    | 117483/225000 [242:25:11<147:22:27,  4.93s/it]
 52%|█████▏    | 117484/225000 [242:25:16<147:32:23,  4.94s/it]
 52%|█████▏    | 117485/225000 [242:25:20<139:41:04,  4.68s/it]
 52%|█████▏    | 117486/225000 [242:25:25<139:23:59,  4.67s/it]
 52%|█████▏    | 117487/225000 [242:25:32<158:43:13,  5.31s/it]
 52%|█████▏    | 117488/225000 [242:25:43<211:01:57,  7.07s/it]
 52%|█████▏    | 117489/225000 [242:25:49<199:37:13,  6.68s/it]
 52%|█████▏    | 117490/225000 [242:25:52<173:52:37,  5.82s/it]
 52%|█████▏    | 117491/225000 [242:25:59<179:36:39,  6.01s/it]
 52%|█████▏    | 117492/225000 [242:26:05<177:19:41,  5.94s/it]
 52%|█████▏    | 117493/225000 [242:26:08<154:55:46,  5.19s/it]
 52%|█████▏    | 117494/225000 [242:26:16<182:46:32,  6.12s/it]
 52%|█████▏    | 117495/225000 [242:26:20<159:48:51,  5.35s/it]
 52%|█████▏    | 117496/225000 [242:26:25<154:53:57,  5.19s/it]
 52%|█████▏    | 117497/225000 [242:26:29<150:25:24,  5.04s/it]
 52%|█████▏    | 117498/225000 [242:26:34<145:51:38,  4.88s/it]
 52%|█████▏    | 117499/225000 [242:26:39<143:12:37,  4.80s/it]
 52%|█████▏    | 117500/225000 [242:26:43<140:41:21,  4.71s/it]
                                                               
{'loss': '0.8711', 'grad_norm': '1.139', 'learning_rate': '2.502e-05', 'epoch': '0.5222', 'num_input_tokens_seen': 404557728, 'train_runtime': '8.728e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117500/225000 [242:26:43<140:41:21,  4.71s/it]
 52%|█████▏    | 117501/225000 [242:26:55<203:31:37,  6.82s/it]
 52%|█████▏    | 117502/225000 [242:27:02<207:19:35,  6.94s/it]
 52%|█████▏    | 117503/225000 [242:27:06<176:10:01,  5.90s/it]
 52%|█████▏    | 117504/225000 [242:27:13<186:26:24,  6.24s/it]
 52%|█████▏    | 117505/225000 [242:27:18<176:38:47,  5.92s/it]
 52%|█████▏    | 117506/225000 [242:27:22<163:41:33,  5.48s/it]
 52%|█████▏    | 117507/225000 [242:27:27<159:57:51,  5.36s/it]
 52%|█████▏    | 117508/225000 [242:27:33<159:31:56,  5.34s/it]
 52%|█████▏    | 117509/225000 [242:27:38<158:35:49,  5.31s/it]
 52%|█████▏    | 117510/225000 [242:27:43<154:23:47,  5.17s/it]
 52%|█████▏    | 117511/225000 [242:27:50<171:50:06,  5.76s/it]
 52%|█████▏    | 117512/225000 [242:27:56<178:42:43,  5.99s/it]
 52%|█████▏    | 117513/225000 [242:28:01<171:27:26,  5.74s/it]
 52%|█████▏    | 117514/225000 [242:28:07<171:42:19,  5.75s/it]
 52%|█████▏    | 117515/225000 [242:28:11<154:03:41,  5.16s/it]
 52%|█████▏    | 117516/225000 [242:28:16<148:25:51,  4.97s/it]
 52%|█████▏    | 117517/225000 [242:28:28<211:13:37,  7.07s/it]
 52%|█████▏    | 117518/225000 [242:28:32<185:09:42,  6.20s/it]
 52%|█████▏    | 117519/225000 [242:28:35<161:14:40,  5.40s/it]
 52%|█████▏    | 117520/225000 [242:28:39<146:52:41,  4.92s/it]
 52%|█████▏    | 117521/225000 [242:28:43<141:28:57,  4.74s/it]
 52%|█████▏    | 117522/225000 [242:28:55<204:42:32,  6.86s/it]
 52%|█████▏    | 117523/225000 [242:29:00<183:13:25,  6.14s/it]
 52%|█████▏    | 117524/225000 [242:29:04<166:21:22,  5.57s/it]
 52%|█████▏    | 117525/225000 [242:29:08<153:00:39,  5.13s/it]
 52%|█████▏    | 117526/225000 [242:29:12<144:04:06,  4.83s/it]
 52%|█████▏    | 117527/225000 [242:29:21<179:26:36,  6.01s/it]
 52%|█████▏    | 117528/225000 [242:29:25<164:17:02,  5.50s/it]
 52%|█████▏    | 117529/225000 [242:29:29<151:14:52,  5.07s/it]
 52%|█████▏    | 117530/225000 [242:29:33<142:51:27,  4.79s/it]
 52%|█████▏    | 117531/225000 [242:29:42<174:06:22,  5.83s/it]
 52%|█████▏    | 117532/225000 [242:29:48<178:48:38,  5.99s/it]
 52%|█████▏    | 117533/225000 [242:29:59<222:17:44,  7.45s/it]
 52%|█████▏    | 117534/225000 [242:30:02<184:11:00,  6.17s/it]
 52%|█████▏    | 117535/225000 [242:30:06<164:06:22,  5.50s/it]
 52%|█████▏    | 117536/225000 [242:30:16<208:10:28,  6.97s/it]
 52%|█████▏    | 117537/225000 [242:30:20<176:21:39,  5.91s/it]
 52%|█████▏    | 117538/225000 [242:30:26<178:36:50,  5.98s/it]
 52%|█████▏    | 117539/225000 [242:30:30<159:01:41,  5.33s/it]
 52%|█████▏    | 117540/225000 [242:30:34<153:07:17,  5.13s/it]
 52%|█████▏    | 117541/225000 [242:30:39<145:10:10,  4.86s/it]
 52%|█████▏    | 117542/225000 [242:30:44<148:07:54,  4.96s/it]
 52%|█████▏    | 117543/225000 [242:30:49<147:00:34,  4.93s/it]
 52%|█████▏    | 117544/225000 [242:30:53<140:43:29,  4.71s/it]
 52%|█████▏    | 117545/225000 [242:30:58<142:12:28,  4.76s/it]
 52%|█████▏    | 117546/225000 [242:31:03<141:43:53,  4.75s/it]
 52%|█████▏    | 117547/225000 [242:31:07<143:12:57,  4.80s/it]
 52%|█████▏    | 117548/225000 [242:31:16<178:45:03,  5.99s/it]
 52%|█████▏    | 117549/225000 [242:31:24<198:31:58,  6.65s/it]
 52%|█████▏    | 117550/225000 [242:31:28<175:44:03,  5.89s/it]
                                                               
{'loss': '0.8844', 'grad_norm': '1.12', 'learning_rate': '2.5e-05', 'epoch': '0.5224', 'num_input_tokens_seen': 404680768, 'train_runtime': '8.731e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117550/225000 [242:31:28<175:44:03,  5.89s/it]
 52%|█████▏    | 117551/225000 [242:31:32<156:23:15,  5.24s/it]
 52%|█████▏    | 117552/225000 [242:31:37<148:02:48,  4.96s/it]
 52%|█████▏    | 117553/225000 [242:31:41<141:58:29,  4.76s/it]
 52%|█████▏    | 117554/225000 [242:31:47<156:45:14,  5.25s/it]
 52%|█████▏    | 117555/225000 [242:31:59<210:55:23,  7.07s/it]
 52%|█████▏    | 117556/225000 [242:32:02<181:21:14,  6.08s/it]
 52%|█████▏    | 117557/225000 [242:32:07<165:36:13,  5.55s/it]
 52%|█████▏    | 117558/225000 [242:32:11<158:00:54,  5.29s/it]
 52%|█████▏    | 117559/225000 [242:32:17<159:57:11,  5.36s/it]
 52%|█████▏    | 117560/225000 [242:32:22<155:57:05,  5.23s/it]
 52%|█████▏    | 117561/225000 [242:32:27<158:56:19,  5.33s/it]
 52%|█████▏    | 117562/225000 [242:32:31<147:28:51,  4.94s/it]
 52%|█████▏    | 117563/225000 [242:32:37<153:06:18,  5.13s/it]
 52%|█████▏    | 117564/225000 [242:32:41<147:34:56,  4.95s/it]
 52%|█████▏    | 117565/225000 [242:32:47<152:05:48,  5.10s/it]
 52%|█████▏    | 117566/225000 [242:32:52<152:25:37,  5.11s/it]
 52%|█████▏    | 117567/225000 [242:33:05<220:32:32,  7.39s/it]
 52%|█████▏    | 117568/225000 [242:33:12<222:14:23,  7.45s/it]
 52%|█████▏    | 117569/225000 [242:33:16<192:02:21,  6.44s/it]
 52%|█████▏    | 117570/225000 [242:33:20<169:37:11,  5.68s/it]
 52%|█████▏    | 117571/225000 [242:33:24<147:47:38,  4.95s/it]
 52%|█████▏    | 117572/225000 [242:33:30<162:55:58,  5.46s/it]
 52%|█████▏    | 117573/225000 [242:33:40<202:02:12,  6.77s/it]
 52%|█████▏    | 117574/225000 [242:33:43<170:14:01,  5.70s/it]
 52%|█████▏    | 117575/225000 [242:33:50<180:01:51,  6.03s/it]
 52%|█████▏    | 117576/225000 [242:33:54<164:29:45,  5.51s/it]
 52%|█████▏    | 117577/225000 [242:33:58<146:39:58,  4.92s/it]
 52%|█████▏    | 117578/225000 [242:34:06<172:02:01,  5.77s/it]
 52%|█████▏    | 117579/225000 [242:34:12<176:55:33,  5.93s/it]
 52%|█████▏    | 117580/225000 [242:34:19<183:06:21,  6.14s/it]
 52%|█████▏    | 117581/225000 [242:34:22<156:54:05,  5.26s/it]
 52%|█████▏    | 117582/225000 [242:34:25<142:24:29,  4.77s/it]
 52%|█████▏    | 117583/225000 [242:34:31<149:34:52,  5.01s/it]
 52%|█████▏    | 117584/225000 [242:34:42<202:49:03,  6.80s/it]
 52%|█████▏    | 117585/225000 [242:34:46<179:27:33,  6.01s/it]
 52%|█████▏    | 117586/225000 [242:34:51<170:04:34,  5.70s/it]
 52%|█████▏    | 117587/225000 [242:34:56<162:51:30,  5.46s/it]
 52%|█████▏    | 117588/225000 [242:35:02<164:39:40,  5.52s/it]
 52%|█████▏    | 117589/225000 [242:35:05<146:52:55,  4.92s/it]
 52%|█████▏    | 117590/225000 [242:35:14<178:25:28,  5.98s/it]
 52%|█████▏    | 117591/225000 [242:35:20<180:03:08,  6.03s/it]
 52%|█████▏    | 117592/225000 [242:35:24<162:05:03,  5.43s/it]
 52%|█████▏    | 117593/225000 [242:35:28<151:22:14,  5.07s/it]
 52%|█████▏    | 117594/225000 [242:35:32<143:21:13,  4.80s/it]
 52%|█████▏    | 117595/225000 [242:35:37<143:35:53,  4.81s/it]
 52%|█████▏    | 117596/225000 [242:35:41<135:37:37,  4.55s/it]
 52%|█████▏    | 117597/225000 [242:35:52<198:01:39,  6.64s/it]
 52%|█████▏    | 117598/225000 [242:35:57<176:10:31,  5.91s/it]
 52%|█████▏    | 117599/225000 [242:36:05<195:09:56,  6.54s/it]
 52%|█████▏    | 117600/225000 [242:36:09<172:19:12,  5.78s/it]
                                                               
{'loss': '0.8735', 'grad_norm': '0.9784', 'learning_rate': '2.498e-05', 'epoch': '0.5227', 'num_input_tokens_seen': 404801192, 'train_runtime': '8.734e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117600/225000 [242:36:09<172:19:12,  5.78s/it]
 52%|█████▏    | 117601/225000 [242:36:15<178:19:33,  5.98s/it]
 52%|█████▏    | 117602/225000 [242:36:19<158:32:46,  5.31s/it]
 52%|█████▏    | 117603/225000 [242:36:24<155:41:02,  5.22s/it]
 52%|█████▏    | 117604/225000 [242:36:29<150:42:09,  5.05s/it]
 52%|█████▏    | 117605/225000 [242:36:33<147:00:44,  4.93s/it]
 52%|█████▏    | 117606/225000 [242:36:39<151:15:06,  5.07s/it]
 52%|█████▏    | 117607/225000 [242:36:43<142:49:01,  4.79s/it]
 52%|█████▏    | 117608/225000 [242:36:47<136:05:36,  4.56s/it]
 52%|█████▏    | 117609/225000 [242:36:52<138:46:52,  4.65s/it]
 52%|█████▏    | 117610/225000 [242:36:57<144:51:13,  4.86s/it]
 52%|█████▏    | 117611/225000 [242:37:09<210:16:05,  7.05s/it]
 52%|█████▏    | 117612/225000 [242:37:14<190:41:42,  6.39s/it]
 52%|█████▏    | 117613/225000 [242:37:19<182:16:46,  6.11s/it]
 52%|█████▏    | 117614/225000 [242:37:27<193:57:17,  6.50s/it]
 52%|█████▏    | 117615/225000 [242:37:30<165:31:34,  5.55s/it]
 52%|█████▏    | 117616/225000 [242:37:35<158:24:33,  5.31s/it]
 52%|█████▏    | 117617/225000 [242:37:40<156:52:38,  5.26s/it]
 52%|█████▏    | 117618/225000 [242:37:45<154:03:17,  5.16s/it]
 52%|█████▏    | 117619/225000 [242:37:54<186:32:34,  6.25s/it]
 52%|█████▏    | 117620/225000 [242:37:57<157:58:10,  5.30s/it]
 52%|█████▏    | 117621/225000 [242:38:03<168:23:40,  5.65s/it]
 52%|█████▏    | 117622/225000 [242:38:09<166:31:58,  5.58s/it]
 52%|█████▏    | 117623/225000 [242:38:13<152:49:46,  5.12s/it]
 52%|█████▏    | 117624/225000 [242:38:22<189:07:34,  6.34s/it]
 52%|█████▏    | 117625/225000 [242:38:27<178:08:22,  5.97s/it]
 52%|█████▏    | 117626/225000 [242:38:30<152:11:28,  5.10s/it]
 52%|█████▏    | 117627/225000 [242:38:35<152:26:57,  5.11s/it]
 52%|█████▏    | 117628/225000 [242:38:47<208:56:42,  7.01s/it]
 52%|█████▏    | 117629/225000 [242:38:51<182:16:02,  6.11s/it]
 52%|█████▏    | 117630/225000 [242:38:59<202:43:33,  6.80s/it]
 52%|█████▏    | 117631/225000 [242:39:08<218:23:09,  7.32s/it]
 52%|█████▏    | 117632/225000 [242:39:14<211:02:54,  7.08s/it]
 52%|█████▏    | 117633/225000 [242:39:17<176:18:50,  5.91s/it]
 52%|█████▏    | 117634/225000 [242:39:21<153:45:10,  5.16s/it]
 52%|█████▏    | 117635/225000 [242:39:24<137:48:23,  4.62s/it]
 52%|█████▏    | 117636/225000 [242:39:35<193:35:33,  6.49s/it]
 52%|█████▏    | 117637/225000 [242:39:38<165:10:08,  5.54s/it]
 52%|█████▏    | 117638/225000 [242:39:45<178:33:51,  5.99s/it]
 52%|█████▏    | 117639/225000 [242:39:51<171:25:27,  5.75s/it]
 52%|█████▏    | 117640/225000 [242:39:55<155:46:13,  5.22s/it]
 52%|█████▏    | 117641/225000 [242:40:02<179:47:38,  6.03s/it]
 52%|█████▏    | 117642/225000 [242:40:09<186:11:59,  6.24s/it]
 52%|█████▏    | 117643/225000 [242:40:14<175:41:06,  5.89s/it]
 52%|█████▏    | 117644/225000 [242:40:20<175:55:59,  5.90s/it]
 52%|█████▏    | 117645/225000 [242:40:23<152:13:47,  5.10s/it]
 52%|█████▏    | 117646/225000 [242:40:30<165:29:58,  5.55s/it]
 52%|█████▏    | 117647/225000 [242:40:34<147:28:44,  4.95s/it]
 52%|█████▏    | 117648/225000 [242:40:38<140:37:34,  4.72s/it]
 52%|█████▏    | 117649/225000 [242:40:49<197:24:36,  6.62s/it]
 52%|█████▏    | 117650/225000 [242:40:53<177:49:21,  5.96s/it]
                                                               
{'loss': '0.8905', 'grad_norm': '1.117', 'learning_rate': '2.496e-05', 'epoch': '0.5229', 'num_input_tokens_seen': 404924184, 'train_runtime': '8.737e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117650/225000 [242:40:53<177:49:21,  5.96s/it]
 52%|█████▏    | 117651/225000 [242:40:59<178:20:41,  5.98s/it]
 52%|█████▏    | 117652/225000 [242:41:03<155:56:34,  5.23s/it]
 52%|█████▏    | 117653/225000 [242:41:07<144:01:33,  4.83s/it]
 52%|█████▏    | 117654/225000 [242:41:11<137:47:00,  4.62s/it]
 52%|█████▏    | 117655/225000 [242:41:16<140:44:40,  4.72s/it]
 52%|█████▏    | 117656/225000 [242:41:21<144:47:05,  4.86s/it]
 52%|█████▏    | 117657/225000 [242:41:25<140:15:05,  4.70s/it]
 52%|█████▏    | 117658/225000 [242:41:41<236:18:47,  7.93s/it]
 52%|█████▏    | 117659/225000 [242:41:51<256:03:07,  8.59s/it]
 52%|█████▏    | 117660/225000 [242:41:54<205:29:18,  6.89s/it]
 52%|█████▏    | 117661/225000 [242:41:58<184:34:12,  6.19s/it]
 52%|█████▏    | 117662/225000 [242:42:03<173:12:51,  5.81s/it]
 52%|█████▏    | 117663/225000 [242:42:08<162:35:57,  5.45s/it]
 52%|█████▏    | 117664/225000 [242:42:16<182:56:10,  6.14s/it]
 52%|█████▏    | 117665/225000 [242:42:19<161:19:02,  5.41s/it]
 52%|█████▏    | 117666/225000 [242:42:24<153:21:39,  5.14s/it]
 52%|█████▏    | 117667/225000 [242:42:28<146:03:07,  4.90s/it]
 52%|█████▏    | 117668/225000 [242:42:36<176:08:46,  5.91s/it]
 52%|█████▏    | 117669/225000 [242:42:42<172:47:31,  5.80s/it]
 52%|█████▏    | 117670/225000 [242:42:46<155:13:13,  5.21s/it]
 52%|█████▏    | 117671/225000 [242:42:51<152:01:12,  5.10s/it]
 52%|█████▏    | 117672/225000 [242:42:56<152:59:51,  5.13s/it]
 52%|█████▏    | 117673/225000 [242:43:00<146:55:27,  4.93s/it]
 52%|█████▏    | 117674/225000 [242:43:05<148:30:41,  4.98s/it]
 52%|█████▏    | 117675/225000 [242:43:11<151:15:09,  5.07s/it]
 52%|█████▏    | 117676/225000 [242:43:15<147:06:25,  4.93s/it]
 52%|█████▏    | 117677/225000 [242:43:21<149:47:14,  5.02s/it]
 52%|█████▏    | 117678/225000 [242:43:33<214:43:22,  7.20s/it]
 52%|█████▏    | 117679/225000 [242:43:37<188:21:45,  6.32s/it]
 52%|█████▏    | 117680/225000 [242:43:46<211:04:24,  7.08s/it]
 52%|█████▏    | 117681/225000 [242:43:49<175:23:35,  5.88s/it]
 52%|█████▏    | 117682/225000 [242:43:53<155:04:42,  5.20s/it]
 52%|█████▏    | 117683/225000 [242:43:57<144:31:26,  4.85s/it]
 52%|█████▏    | 117684/225000 [242:44:01<143:27:48,  4.81s/it]
 52%|█████▏    | 117685/225000 [242:44:13<206:39:37,  6.93s/it]
 52%|█████▏    | 117686/225000 [242:44:17<176:02:30,  5.91s/it]
 52%|█████▏    | 117687/225000 [242:44:21<163:58:23,  5.50s/it]
 52%|█████▏    | 117688/225000 [242:44:26<158:08:45,  5.31s/it]
 52%|█████▏    | 117689/225000 [242:44:31<151:43:24,  5.09s/it]
 52%|█████▏    | 117690/225000 [242:44:39<182:53:35,  6.14s/it]
 52%|█████▏    | 117691/225000 [242:44:43<159:35:11,  5.35s/it]
 52%|█████▏    | 117692/225000 [242:44:52<192:49:56,  6.47s/it]
 52%|█████▏    | 117693/225000 [242:44:56<172:28:25,  5.79s/it]
 52%|█████▏    | 117694/225000 [242:45:00<151:37:30,  5.09s/it]
 52%|█████▏    | 117695/225000 [242:45:03<135:12:28,  4.54s/it]
 52%|█████▏    | 117696/225000 [242:45:08<136:59:03,  4.60s/it]
 52%|█████▏    | 117697/225000 [242:45:17<177:43:55,  5.96s/it]
 52%|█████▏    | 117698/225000 [242:45:22<166:55:56,  5.60s/it]
 52%|█████▏    | 117699/225000 [242:45:26<156:36:45,  5.25s/it]
 52%|█████▏    | 117700/225000 [242:45:31<152:27:47,  5.12s/it]
                                                               
{'loss': '0.8873', 'grad_norm': '0.992', 'learning_rate': '2.495e-05', 'epoch': '0.5231', 'num_input_tokens_seen': 405046024, 'train_runtime': '8.739e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117700/225000 [242:45:31<152:27:47,  5.12s/it]
 52%|█████▏    | 117701/225000 [242:45:36<150:07:20,  5.04s/it]
 52%|█████▏    | 117702/225000 [242:45:40<146:02:53,  4.90s/it]
 52%|█████▏    | 117703/225000 [242:45:48<174:38:01,  5.86s/it]
 52%|█████▏    | 117704/225000 [242:45:56<192:37:01,  6.46s/it]
 52%|█████▏    | 117705/225000 [242:46:00<167:29:04,  5.62s/it]
 52%|█████▏    | 117706/225000 [242:46:04<152:23:27,  5.11s/it]
 52%|█████▏    | 117707/225000 [242:46:07<139:02:13,  4.67s/it]
 52%|█████▏    | 117708/225000 [242:46:14<158:32:02,  5.32s/it]
 52%|█████▏    | 117709/225000 [242:46:23<190:38:13,  6.40s/it]
 52%|█████▏    | 117710/225000 [242:46:27<165:20:14,  5.55s/it]
 52%|█████▏    | 117711/225000 [242:46:31<156:47:21,  5.26s/it]
 52%|█████▏    | 117712/225000 [242:46:41<195:45:34,  6.57s/it]
 52%|█████▏    | 117713/225000 [242:46:45<169:24:30,  5.68s/it]
 52%|█████▏    | 117714/225000 [242:46:53<191:18:24,  6.42s/it]
 52%|█████▏    | 117715/225000 [242:46:57<169:04:17,  5.67s/it]
 52%|█████▏    | 117716/225000 [242:47:00<152:52:24,  5.13s/it]
 52%|█████▏    | 117717/225000 [242:47:06<161:09:34,  5.41s/it]
 52%|█████▏    | 117718/225000 [242:47:16<196:57:30,  6.61s/it]
 52%|█████▏    | 117719/225000 [242:47:20<170:06:40,  5.71s/it]
 52%|█████▏    | 117720/225000 [242:47:24<155:26:47,  5.22s/it]
 52%|█████▏    | 117721/225000 [242:47:27<142:51:38,  4.79s/it]
 52%|█████▏    | 117722/225000 [242:47:31<135:41:50,  4.55s/it]
 52%|█████▏    | 117723/225000 [242:47:44<204:36:51,  6.87s/it]
 52%|█████▏    | 117724/225000 [242:47:49<194:14:14,  6.52s/it]
 52%|█████▏    | 117725/225000 [242:47:53<166:22:06,  5.58s/it]
 52%|█████▏    | 117726/225000 [242:47:57<154:26:45,  5.18s/it]
 52%|█████▏    | 117727/225000 [242:48:01<147:17:49,  4.94s/it]
 52%|█████▏    | 117728/225000 [242:48:06<143:47:08,  4.83s/it]
 52%|█████▏    | 117729/225000 [242:48:17<197:37:37,  6.63s/it]
 52%|█████▏    | 117730/225000 [242:48:21<175:16:24,  5.88s/it]
 52%|█████▏    | 117731/225000 [242:48:25<158:48:24,  5.33s/it]
 52%|█████▏    | 117732/225000 [242:48:32<171:26:03,  5.75s/it]
 52%|█████▏    | 117733/225000 [242:48:42<209:46:18,  7.04s/it]
 52%|█████▏    | 117734/225000 [242:48:48<206:35:22,  6.93s/it]
 52%|█████▏    | 117735/225000 [242:48:52<173:38:13,  5.83s/it]
 52%|█████▏    | 117736/225000 [242:48:55<154:26:11,  5.18s/it]
 52%|█████▏    | 117737/225000 [242:49:05<197:20:17,  6.62s/it]
 52%|█████▏    | 117738/225000 [242:49:09<167:02:59,  5.61s/it]
 52%|█████▏    | 117739/225000 [242:49:17<194:31:31,  6.53s/it]
 52%|█████▏    | 117740/225000 [242:49:21<171:32:23,  5.76s/it]
 52%|█████▏    | 117741/225000 [242:49:32<216:54:00,  7.28s/it]
 52%|█████▏    | 117742/225000 [242:49:36<183:34:03,  6.16s/it]
 52%|█████▏    | 117743/225000 [242:49:39<158:44:03,  5.33s/it]
 52%|█████▏    | 117744/225000 [242:49:49<196:23:32,  6.59s/it]
 52%|█████▏    | 117745/225000 [242:49:52<171:52:40,  5.77s/it]
 52%|█████▏    | 117746/225000 [242:49:58<172:29:23,  5.79s/it]
 52%|█████▏    | 117747/225000 [242:50:05<179:20:24,  6.02s/it]
 52%|█████▏    | 117748/225000 [242:50:15<212:45:36,  7.14s/it]
 52%|█████▏    | 117749/225000 [242:50:19<185:49:09,  6.24s/it]
 52%|█████▏    | 117750/225000 [242:50:24<174:39:48,  5.86s/it]
                                                               
{'loss': '0.8789', 'grad_norm': '0.9196', 'learning_rate': '2.493e-05', 'epoch': '0.5233', 'num_input_tokens_seen': 405172264, 'train_runtime': '8.742e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117750/225000 [242:50:24<174:39:48,  5.86s/it]
 52%|█████▏    | 117751/225000 [242:50:27<156:48:33,  5.26s/it]
 52%|█████▏    | 117752/225000 [242:50:32<147:18:07,  4.94s/it]
 52%|█████▏    | 117753/225000 [242:50:43<201:31:57,  6.76s/it]
 52%|█████▏    | 117754/225000 [242:50:46<172:26:59,  5.79s/it]
 52%|█████▏    | 117755/225000 [242:50:54<186:24:30,  6.26s/it]
 52%|█████▏    | 117756/225000 [242:50:57<160:31:03,  5.39s/it]
 52%|█████▏    | 117757/225000 [242:51:01<146:13:41,  4.91s/it]
 52%|█████▏    | 117758/225000 [242:51:08<163:12:07,  5.48s/it]
 52%|█████▏    | 117759/225000 [242:51:15<180:17:44,  6.05s/it]
 52%|█████▏    | 117760/225000 [242:51:20<169:47:35,  5.70s/it]
 52%|█████▏    | 117761/225000 [242:51:27<182:07:24,  6.11s/it]
 52%|█████▏    | 117762/225000 [242:51:31<166:38:31,  5.59s/it]
 52%|█████▏    | 117763/225000 [242:51:35<153:21:38,  5.15s/it]
 52%|█████▏    | 117764/225000 [242:51:40<144:56:27,  4.87s/it]
 52%|█████▏    | 117765/225000 [242:51:44<142:52:29,  4.80s/it]
 52%|█████▏    | 117766/225000 [242:51:51<156:52:49,  5.27s/it]
 52%|█████▏    | 117767/225000 [242:52:04<227:33:22,  7.64s/it]
 52%|█████▏    | 117768/225000 [242:52:08<193:31:28,  6.50s/it]
 52%|█████▏    | 117769/225000 [242:52:13<181:07:11,  6.08s/it]
 52%|█████▏    | 117770/225000 [242:52:16<160:07:14,  5.38s/it]
 52%|█████▏    | 117771/225000 [242:52:24<177:56:44,  5.97s/it]
 52%|█████▏    | 117772/225000 [242:52:29<167:45:44,  5.63s/it]
 52%|█████▏    | 117773/225000 [242:52:33<153:45:11,  5.16s/it]
 52%|█████▏    | 117774/225000 [242:52:37<148:15:59,  4.98s/it]
 52%|█████▏    | 117775/225000 [242:52:47<187:01:39,  6.28s/it]
 52%|█████▏    | 117776/225000 [242:52:50<161:58:13,  5.44s/it]
 52%|█████▏    | 117777/225000 [242:52:59<196:17:33,  6.59s/it]
 52%|█████▏    | 117778/225000 [242:53:02<165:36:17,  5.56s/it]
 52%|█████▏    | 117779/225000 [242:53:06<146:55:30,  4.93s/it]
 52%|█████▏    | 117780/225000 [242:53:09<134:40:50,  4.52s/it]
 52%|█████▏    | 117781/225000 [242:53:14<134:36:25,  4.52s/it]
 52%|█████▏    | 117782/225000 [242:53:24<180:38:50,  6.07s/it]
 52%|█████▏    | 117783/225000 [242:53:27<160:18:33,  5.38s/it]
 52%|█████▏    | 117784/225000 [242:53:36<186:50:47,  6.27s/it]
 52%|█████▏    | 117785/225000 [242:53:40<169:41:55,  5.70s/it]
 52%|█████▏    | 117786/225000 [242:53:47<179:06:21,  6.01s/it]
 52%|█████▏    | 117787/225000 [242:53:51<161:03:29,  5.41s/it]
 52%|█████▏    | 117788/225000 [242:53:55<145:59:13,  4.90s/it]
 52%|█████▏    | 117789/225000 [242:53:59<140:13:36,  4.71s/it]
 52%|█████▏    | 117790/225000 [242:54:05<154:30:25,  5.19s/it]
 52%|█████▏    | 117791/225000 [242:54:17<209:30:01,  7.03s/it]
 52%|█████▏    | 117792/225000 [242:54:20<175:13:16,  5.88s/it]
 52%|█████▏    | 117793/225000 [242:54:26<178:29:08,  5.99s/it]
 52%|█████▏    | 117794/225000 [242:54:31<169:40:03,  5.70s/it]
 52%|█████▏    | 117795/225000 [242:54:39<191:59:21,  6.45s/it]
 52%|█████▏    | 117796/225000 [242:54:44<174:59:56,  5.88s/it]
 52%|█████▏    | 117797/225000 [242:54:48<164:52:09,  5.54s/it]
 52%|█████▏    | 117798/225000 [242:54:53<151:13:45,  5.08s/it]
 52%|█████▏    | 117799/225000 [242:55:03<202:04:38,  6.79s/it]
 52%|█████▏    | 117800/225000 [242:55:08<185:48:45,  6.24s/it]
                                                               
{'loss': '0.8896', 'grad_norm': '1.184', 'learning_rate': '2.491e-05', 'epoch': '0.5236', 'num_input_tokens_seen': 405296600, 'train_runtime': '8.745e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117800/225000 [242:55:08<185:48:45,  6.24s/it]
 52%|█████▏    | 117801/225000 [242:55:16<196:53:58,  6.61s/it]
 52%|█████▏    | 117802/225000 [242:55:19<169:01:41,  5.68s/it]
 52%|█████▏    | 117803/225000 [242:55:23<149:41:52,  5.03s/it]
 52%|█████▏    | 117804/225000 [242:55:30<172:25:19,  5.79s/it]
 52%|█████▏    | 117805/225000 [242:55:36<169:44:11,  5.70s/it]
 52%|█████▏    | 117806/225000 [242:55:40<159:52:56,  5.37s/it]
 52%|█████▏    | 117807/225000 [242:55:45<150:32:23,  5.06s/it]
 52%|█████▏    | 117808/225000 [242:55:55<200:01:04,  6.72s/it]
 52%|█████▏    | 117809/225000 [242:56:00<178:09:19,  5.98s/it]
 52%|█████▏    | 117810/225000 [242:56:06<186:03:43,  6.25s/it]
 52%|█████▏    | 117811/225000 [242:56:11<171:37:45,  5.76s/it]
 52%|█████▏    | 117812/225000 [242:56:15<155:54:33,  5.24s/it]
 52%|█████▏    | 117813/225000 [242:56:24<189:55:28,  6.38s/it]
 52%|█████▏    | 117814/225000 [242:56:27<162:15:32,  5.45s/it]
 52%|█████▏    | 117815/225000 [242:56:32<150:40:46,  5.06s/it]
 52%|█████▏    | 117816/225000 [242:56:35<138:59:50,  4.67s/it]
 52%|█████▏    | 117817/225000 [242:56:39<134:00:11,  4.50s/it]
 52%|█████▏    | 117818/225000 [242:56:49<180:56:02,  6.08s/it]
 52%|█████▏    | 117819/225000 [242:56:56<184:55:26,  6.21s/it]
 52%|█████▏    | 117820/225000 [242:56:59<160:55:59,  5.41s/it]
 52%|█████▏    | 117821/225000 [242:57:07<185:32:33,  6.23s/it]
 52%|█████▏    | 117822/225000 [242:57:13<178:09:06,  5.98s/it]
 52%|█████▏    | 117823/225000 [242:57:17<161:20:14,  5.42s/it]
 52%|█████▏    | 117824/225000 [242:57:27<206:14:39,  6.93s/it]
 52%|█████▏    | 117825/225000 [242:57:30<171:28:52,  5.76s/it]
 52%|█████▏    | 117826/225000 [242:57:34<153:52:28,  5.17s/it]
 52%|█████▏    | 117827/225000 [242:57:42<178:25:21,  5.99s/it]
 52%|█████▏    | 117828/225000 [242:57:45<153:35:33,  5.16s/it]
 52%|█████▏    | 117829/225000 [242:57:49<143:53:07,  4.83s/it]
 52%|█████▏    | 117830/225000 [242:57:57<168:10:54,  5.65s/it]
 52%|█████▏    | 117831/225000 [242:58:00<148:20:06,  4.98s/it]
 52%|█████▏    | 117832/225000 [242:58:04<138:51:21,  4.66s/it]
 52%|█████▏    | 117833/225000 [242:58:08<133:48:57,  4.50s/it]
 52%|█████▏    | 117834/225000 [242:58:18<175:23:28,  5.89s/it]
 52%|█████▏    | 117835/225000 [242:58:24<178:43:40,  6.00s/it]
 52%|█████▏    | 117836/225000 [242:58:29<171:33:23,  5.76s/it]
 52%|█████▏    | 117837/225000 [242:58:36<180:58:39,  6.08s/it]
 52%|█████▏    | 117838/225000 [242:58:42<183:24:52,  6.16s/it]
 52%|█████▏    | 117839/225000 [242:58:47<169:33:08,  5.70s/it]
 52%|█████▏    | 117840/225000 [242:58:50<150:40:37,  5.06s/it]
 52%|█████▏    | 117841/225000 [242:58:55<144:27:13,  4.85s/it]
 52%|█████▏    | 117842/225000 [242:58:59<143:27:52,  4.82s/it]
 52%|█████▏    | 117843/225000 [242:59:04<137:48:12,  4.63s/it]
 52%|█████▏    | 117844/225000 [242:59:09<146:13:03,  4.91s/it]
 52%|█████▏    | 117845/225000 [242:59:16<163:40:10,  5.50s/it]
 52%|█████▏    | 117846/225000 [242:59:31<245:03:14,  8.23s/it]
 52%|█████▏    | 117847/225000 [242:59:34<205:28:34,  6.90s/it]
 52%|█████▏    | 117848/225000 [242:59:41<198:51:35,  6.68s/it]
 52%|█████▏    | 117849/225000 [242:59:44<171:26:02,  5.76s/it]
 52%|█████▏    | 117850/225000 [242:59:50<173:25:33,  5.83s/it]
                                                               
{'loss': '0.8659', 'grad_norm': '1.044', 'learning_rate': '2.489e-05', 'epoch': '0.5238', 'num_input_tokens_seen': 405423640, 'train_runtime': '8.748e+05', 'train_tokens_per_second': '463.5'}

 52%|█████▏    | 117850/225000 [242:59:50<173:25:33,  5.83s/it]
 52%|█████▏    | 117851/225000 [242:59:58<186:34:13,  6.27s/it]
 52%|█████▏    | 117852/225000 [243:00:03<180:52:50,  6.08s/it]
 52%|█████▏    | 117853/225000 [243:00:08<172:05:38,  5.78s/it]
 52%|█████▏    | 117854/225000 [243:00:11<146:53:37,  4.94s/it]
 52%|█████▏    | 117855/225000 [243:00:20<178:43:30,  6.01s/it]
 52%|█████▏    | 117856/225000 [243:00:24<163:26:51,  5.49s/it]
 52%|█████▏    | 117857/225000 [243:00:31<179:21:25,  6.03s/it]
 52%|█████▏    | 117858/225000 [243:00:35<158:41:01,  5.33s/it]
 52%|█████▏    | 117859/225000 [243:00:38<137:17:37,  4.61s/it]
 52%|█████▏    | 117860/225000 [243:00:46<167:54:55,  5.64s/it]
 52%|█████▏    | 117861/225000 [243:00:51<160:38:06,  5.40s/it]
 52%|█████▏    | 117862/225000 [243:00:55<154:08:34,  5.18s/it]
 52%|█████▏    | 117863/225000 [243:01:00<144:26:54,  4.85s/it]
 52%|█████▏    | 117864/225000 [243:01:04<138:25:50,  4.65s/it]
 52%|█████▏    | 117865/225000 [243:01:09<142:56:44,  4.80s/it]
 52%|█████▏    | 117866/225000 [243:01:13<140:50:49,  4.73s/it]
 52%|█████▏    | 117867/225000 [243:01:19<145:30:02,  4.89s/it]
 52%|█████▏    | 117868/225000 [243:01:24<151:04:28,  5.08s/it]
 52%|█████▏    | 117869/225000 [243:01:29<148:04:11,  4.98s/it]
 52%|█████▏    | 117870/225000 [243:01:34<144:41:40,  4.86s/it]
 52%|█████▏    | 117871/225000 [243:01:43<184:55:46,  6.21s/it]
 52%|█████▏    | 117872/225000 [243:01:53<221:18:10,  7.44s/it]
 52%|█████▏    | 117873/225000 [243:01:57<184:35:05,  6.20s/it]
 52%|█████▏    | 117874/225000 [243:02:01<170:03:57,  5.72s/it]
 52%|█████▏    | 117875/225000 [243:02:06<165:33:12,  5.56s/it]
 52%|█████▏    | 117876/225000 [243:02:14<185:25:36,  6.23s/it]
 52%|█████▏    | 117877/225000 [243:02:19<170:50:55,  5.74s/it]
 52%|█████▏    | 117878/225000 [243:02:23<157:25:03,  5.29s/it]
 52%|█████▏    | 117879/225000 [243:02:29<163:10:43,  5.48s/it]
 52%|█████▏    | 117880/225000 [243:02:33<151:11:20,  5.08s/it]
 52%|█████▏    | 117881/225000 [243:02:46<219:09:13,  7.37s/it]
 52%|█████▏    | 117882/225000 [243:02:51<196:30:43,  6.60s/it]
 52%|█████▏    | 117883/225000 [243:02:56<184:13:10,  6.19s/it]
 52%|█████▏    | 117884/225000 [243:03:00<161:57:28,  5.44s/it]
 52%|█████▏    | 117885/225000 [243:03:03<148:41:06,  5.00s/it]
 52%|█████▏    | 117886/225000 [243:03:08<141:38:56,  4.76s/it]
 52%|█████▏    | 117887/225000 [243:03:19<202:13:10,  6.80s/it]
 52%|█████▏    | 117888/225000 [243:03:25<189:06:19,  6.36s/it]
 52%|█████▏    | 117889/225000 [243:03:31<186:48:43,  6.28s/it]
 52%|█████▏    | 117890/225000 [243:03:36<179:21:24,  6.03s/it]
 52%|█████▏    | 117891/225000 [243:03:40<164:00:48,  5.51s/it]
 52%|█████▏    | 117892/225000 [243:03:46<161:54:27,  5.44s/it]
 52%|█████▏    | 117893/225000 [243:03:54<191:25:35,  6.43s/it]
 52%|█████▏    | 117894/225000 [243:03:58<169:21:26,  5.69s/it]
 52%|█████▏    | 117895/225000 [243:04:04<170:47:36,  5.74s/it]
 52%|█████▏    | 117896/225000 [243:04:11<180:47:33,  6.08s/it]
 52%|█████▏    | 117897/225000 [243:04:15<160:08:33,  5.38s/it]
 52%|█████▏    | 117898/225000 [243:04:19<148:01:39,  4.98s/it]
 52%|█████▏    | 117899/225000 [243:04:23<136:58:23,  4.60s/it]
 52%|█████▏    | 117900/225000 [243:04:27<136:48:08,  4.60s/it]
                                                               
{'loss': '0.8999', 'grad_norm': '1.06', 'learning_rate': '2.487e-05', 'epoch': '0.524', 'num_input_tokens_seen': 405542712, 'train_runtime': '8.751e+05', 'train_tokens_per_second': '463.4'}

 52%|█████▏    | 117900/225000 [243:04:27<136:48:08,  4.60s/it]
 52%|█████▏    | 117901/225000 [243:04:31<131:08:02,  4.41s/it]
 52%|█████▏    | 117902/225000 [243:04:36<131:55:21,  4.43s/it]
 52%|█████▏    | 117903/225000 [243:04:40<135:01:26,  4.54s/it]
 52%|█████▏    | 117904/225000 [243:04:50<183:30:13,  6.17s/it]
 52%|█████▏    | 117905/225000 [243:04:59<202:27:38,  6.81s/it]
 52%|█████▏    | 117906/225000 [243:05:04<184:58:08,  6.22s/it]
 52%|█████▏    | 117907/225000 [243:05:10<189:09:44,  6.36s/it]
 52%|█████▏    | 117908/225000 [243:05:14<165:07:57,  5.55s/it]
 52%|█████▏    | 117909/225000 [243:05:17<146:16:57,  4.92s/it]
 52%|█████▏    | 117910/225000 [243:05:26<175:24:50,  5.90s/it]
 52%|█████▏    | 117911/225000 [243:05:30<166:45:33,  5.61s/it]
 52%|█████▏    | 117912/225000 [243:05:36<166:06:06,  5.58s/it]
 52%|█████▏    | 117913/225000 [243:05:39<144:13:59,  4.85s/it]
 52%|█████▏    | 117914/225000 [243:05:47<171:38:15,  5.77s/it]
 52%|█████▏    | 117915/225000 [243:05:50<147:04:58,  4.94s/it]
 52%|█████▏    | 117916/225000 [243:05:54<141:11:37,  4.75s/it]
 52%|█████▏    | 117917/225000 [243:06:00<144:57:22,  4.87s/it]
 52%|█████▏    | 117918/225000 [243:06:09<187:46:18,  6.31s/it]
 52%|█████▏    | 117919/225000 [243:06:15<182:20:56,  6.13s/it]
 52%|█████▏    | 117920/225000 [243:06:19<166:15:41,  5.59s/it]
 52%|█████▏    | 117921/225000 [243:06:23<153:48:33,  5.17s/it]
 52%|█████▏    | 117922/225000 [243:06:28<146:01:20,  4.91s/it]
 52%|█████▏    | 117923/225000 [243:06:32<141:55:51,  4.77s/it]
 52%|█████▏    | 117924/225000 [243:06:44<205:43:21,  6.92s/it]
 52%|█████▏    | 117925/225000 [243:06:48<179:45:19,  6.04s/it]
 52%|█████▏    | 117926/225000 [243:06:52<162:40:27,  5.47s/it]
 52%|█████▏    | 117927/225000 [243:06:56<149:57:04,  5.04s/it]
 52%|█████▏    | 117928/225000 [243:07:08<210:48:40,  7.09s/it]
 52%|█████▏    | 117929/225000 [243:07:11<176:57:46,  5.95s/it]
 52%|█████▏    | 117930/225000 [243:07:16<167:46:44,  5.64s/it]
 52%|█████▏    | 117931/225000 [243:07:25<193:23:49,  6.50s/it]
 52%|█████▏    | 117932/225000 [243:07:29<170:17:54,  5.73s/it]
 52%|█████▏    | 117933/225000 [243:07:33<155:45:54,  5.24s/it]
 52%|█████▏    | 117934/225000 [243:07:37<141:17:38,  4.75s/it]
 52%|█████▏    | 117935/225000 [243:07:43<154:25:34,  5.19s/it]
 52%|█████▏    | 117936/225000 [243:07:54<205:47:24,  6.92s/it]
 52%|█████▏    | 117937/225000 [243:07:57<176:00:10,  5.92s/it]
 52%|█████▏    | 117938/225000 [243:08:06<204:55:36,  6.89s/it]
 52%|█████▏    | 117939/225000 [243:08:09<169:46:59,  5.71s/it]
 52%|█████▏    | 117940/225000 [243:08:16<180:05:24,  6.06s/it]
 52%|█████▏    | 117941/225000 [243:08:20<155:45:26,  5.24s/it]
 52%|█████▏    | 117942/225000 [243:08:28<185:03:10,  6.22s/it]
 52%|█████▏    | 117943/225000 [243:08:32<166:46:58,  5.61s/it]
 52%|█████▏    | 117944/225000 [243:08:37<160:42:32,  5.40s/it]
 52%|█████▏    | 117945/225000 [243:08:42<151:28:31,  5.09s/it]
 52%|█████▏    | 117946/225000 [243:08:46<144:36:19,  4.86s/it]
 52%|█████▏    | 117947/225000 [243:08:57<203:27:50,  6.84s/it]
 52%|█████▏    | 117948/225000 [243:09:01<179:23:34,  6.03s/it]
 52%|█████▏    | 117949/225000 [243:09:05<160:37:35,  5.40s/it]
 52%|█████▏    | 117950/225000 [243:09:10<154:34:35,  5.20s/it]
                                                               
{'loss': '0.8663', 'grad_norm': '1.168', 'learning_rate': '2.485e-05', 'epoch': '0.5242', 'num_input_tokens_seen': 405666200, 'train_runtime': '8.754e+05', 'train_tokens_per_second': '463.4'}

 52%|█████▏    | 117950/225000 [243:09:10<154:34:35,  5.20s/it]
 52%|█████▏    | 117951/225000 [243:09:17<172:46:40,  5.81s/it]
 52%|█████▏    | 117952/225000 [243:09:22<160:55:46,  5.41s/it]
 52%|█████▏    | 117953/225000 [243:09:27<156:24:03,  5.26s/it]
 52%|█████▏    | 117954/225000 [243:09:31<151:34:48,  5.10s/it]
 52%|█████▏    | 117955/225000 [243:09:37<153:04:28,  5.15s/it]
 52%|█████▏    | 117956/225000 [243:09:42<152:27:39,  5.13s/it]
 52%|█████▏    | 117957/225000 [243:09:47<148:41:05,  5.00s/it]
 52%|█████▏    | 117958/225000 [243:09:51<142:43:11,  4.80s/it]
 52%|█████▏    | 117959/225000 [243:09:56<141:31:11,  4.76s/it]
 52%|█████▏    | 117960/225000 [243:10:07<203:53:30,  6.86s/it]
 52%|█████▏    | 117961/225000 [243:10:13<192:59:50,  6.49s/it]
 52%|█████▏    | 117962/225000 [243:10:17<172:12:09,  5.79s/it]
 52%|█████▏    | 117963/225000 [243:10:23<170:54:02,  5.75s/it]
 52%|█████▏    | 117964/225000 [243:10:33<214:34:03,  7.22s/it]
 52%|█████▏    | 117965/225000 [243:10:37<182:36:50,  6.14s/it]
 52%|█████▏    | 117966/225000 [243:10:41<161:21:41,  5.43s/it]
 52%|█████▏    | 117967/225000 [243:10:50<198:36:20,  6.68s/it]
 52%|█████▏    | 117968/225000 [243:10:54<174:28:59,  5.87s/it]
 52%|█████▏    | 117969/225000 [243:10:58<158:14:13,  5.32s/it]
 52%|█████▏    | 117970/225000 [243:11:07<189:45:52,  6.38s/it]
 52%|█████▏    | 117971/225000 [243:11:11<165:32:10,  5.57s/it]
 52%|█████▏    | 117972/225000 [243:11:15<149:42:29,  5.04s/it]
 52%|█████▏    | 117973/225000 [243:11:19<140:25:41,  4.72s/it]
 52%|█████▏    | 117974/225000 [243:11:25<151:48:47,  5.11s/it]
 52%|█████▏    | 117975/225000 [243:11:35<196:13:57,  6.60s/it]
 52%|█████▏    | 117976/225000 [243:11:38<169:13:59,  5.69s/it]
 52%|█████▏    | 117977/225000 [243:11:46<184:05:49,  6.19s/it]
 52%|█████▏    | 117978/225000 [243:11:53<194:21:03,  6.54s/it]
 52%|█████▏    | 117979/225000 [243:11:56<166:10:25,  5.59s/it]
 52%|█████▏    | 117980/225000 [243:12:05<190:52:22,  6.42s/it]
 52%|█████▏    | 117981/225000 [243:12:08<160:37:09,  5.40s/it]
 52%|█████▏    | 117982/225000 [243:12:12<151:20:42,  5.09s/it]
 52%|█████▏    | 117983/225000 [243:12:16<140:27:21,  4.72s/it]
 52%|█████▏    | 117984/225000 [243:12:20<134:57:41,  4.54s/it]
 52%|█████▏    | 117985/225000 [243:12:23<123:30:21,  4.15s/it]
 52%|█████▏    | 117986/225000 [243:12:33<167:31:45,  5.64s/it]
 52%|█████▏    | 117987/225000 [243:12:39<173:27:40,  5.84s/it]
 52%|█████▏    | 117988/225000 [243:12:42<152:48:05,  5.14s/it]
 52%|█████▏    | 117989/225000 [243:12:50<178:22:55,  6.00s/it]
 52%|█████▏    | 117990/225000 [243:12:54<158:22:34,  5.33s/it]
 52%|█████▏    | 117991/225000 [243:12:58<148:42:17,  5.00s/it]
 52%|█████▏    | 117992/225000 [243:13:02<140:36:35,  4.73s/it]
 52%|█████▏    | 117993/225000 [243:13:07<140:51:18,  4.74s/it]
 52%|█████▏    | 117994/225000 [243:13:19<205:17:35,  6.91s/it]
 52%|█████▏    | 117995/225000 [243:13:23<177:17:02,  5.96s/it]
 52%|█████▏    | 117996/225000 [243:13:27<164:45:20,  5.54s/it]
 52%|█████▏    | 117997/225000 [243:13:32<155:03:44,  5.22s/it]
 52%|█████▏    | 117998/225000 [243:13:36<144:55:49,  4.88s/it]
 52%|█████▏    | 117999/225000 [243:13:45<179:32:47,  6.04s/it]
 52%|█████▏    | 118000/225000 [243:13:52<193:32:13,  6.51s/it]
                                                               
{'loss': '0.8877', 'grad_norm': '1.2', 'learning_rate': '2.484e-05', 'epoch': '0.5244', 'num_input_tokens_seen': 405789008, 'train_runtime': '8.756e+05', 'train_tokens_per_second': '463.4'}

 52%|█████▏    | 118000/225000 [243:13:52<193:32:13,  6.51s/it]
 52%|█████▏    | 118001/225000 [243:13:57<175:05:55,  5.89s/it]
 52%|█████▏    | 118002/225000 [243:14:01<161:25:56,  5.43s/it]
 52%|█████▏    | 118003/225000 [243:14:06<160:07:41,  5.39s/it]
 52%|█████▏    | 118004/225000 [243:14:12<161:54:18,  5.45s/it]
 52%|█████▏    | 118005/225000 [243:14:22<197:35:13,  6.65s/it]
 52%|█████▏    | 118006/225000 [243:14:26<174:44:09,  5.88s/it]
 52%|█████▏    | 118007/225000 [243:14:30<157:56:19,  5.31s/it]
 52%|█████▏    | 118008/225000 [243:14:34<146:33:14,  4.93s/it]
 52%|█████▏    | 118009/225000 [243:14:38<141:11:38,  4.75s/it]
 52%|█████▏    | 118010/225000 [243:14:49<197:21:53,  6.64s/it]
 52%|█████▏    | 118011/225000 [243:14:54<185:34:20,  6.24s/it]
 52%|█████▏    | 118012/225000 [243:14:59<168:05:34,  5.66s/it]
 52%|█████▏    | 118013/225000 [243:15:03<159:15:34,  5.36s/it]
 52%|█████▏    | 118014/225000 [243:15:07<146:31:04,  4.93s/it]
 52%|█████▏    | 118015/225000 [243:15:12<143:22:19,  4.82s/it]
 52%|█████▏    | 118016/225000 [243:15:19<163:45:12,  5.51s/it]
 52%|█████▏    | 118017/225000 [243:15:28<198:45:51,  6.69s/it]
 52%|█████▏    | 118018/225000 [243:15:34<191:25:40,  6.44s/it]
 52%|█████▏    | 118019/225000 [243:15:41<193:55:37,  6.53s/it]
 52%|█████▏    | 118020/225000 [243:15:51<225:25:28,  7.59s/it]
 52%|█████▏    | 118021/225000 [243:15:55<196:11:32,  6.60s/it]
 52%|█████▏    | 118022/225000 [243:16:01<188:37:50,  6.35s/it]
 52%|█████▏    | 118023/225000 [243:16:04<162:05:07,  5.45s/it]
 52%|█████▏    | 118024/225000 [243:16:15<206:23:32,  6.95s/it]
 52%|█████▏    | 118025/225000 [243:16:18<174:25:04,  5.87s/it]
 52%|█████▏    | 118026/225000 [243:16:23<168:34:17,  5.67s/it]
 52%|█████▏    | 118027/225000 [243:16:30<174:51:17,  5.88s/it]
 52%|█████▏    | 118028/225000 [243:16:33<154:17:20,  5.19s/it]
 52%|█████▏    | 118029/225000 [243:16:38<145:00:00,  4.88s/it]
 52%|█████▏    | 118030/225000 [243:16:42<140:38:36,  4.73s/it]
 52%|█████▏    | 118031/225000 [243:16:48<150:41:49,  5.07s/it]
 52%|█████▏    | 118032/225000 [243:16:55<170:30:02,  5.74s/it]
 52%|█████▏    | 118033/225000 [243:17:02<184:10:03,  6.20s/it]
 52%|█████▏    | 118034/225000 [243:17:06<157:38:54,  5.31s/it]
 52%|█████▏    | 118035/225000 [243:17:10<147:46:23,  4.97s/it]
 52%|█████▏    | 118036/225000 [243:17:13<135:23:37,  4.56s/it]
 52%|█████▏    | 118037/225000 [243:17:18<133:14:43,  4.48s/it]
 52%|█████▏    | 118038/225000 [243:17:22<134:51:55,  4.54s/it]
 52%|█████▏    | 118039/225000 [243:17:27<137:43:36,  4.64s/it]
 52%|█████▏    | 118040/225000 [243:17:32<137:18:10,  4.62s/it]
 52%|█████▏    | 118041/225000 [243:17:43<197:52:37,  6.66s/it]
 52%|█████▏    | 118042/225000 [243:17:49<187:19:14,  6.30s/it]
 52%|█████▏    | 118043/225000 [243:17:52<159:58:45,  5.38s/it]
 52%|█████▏    | 118044/225000 [243:18:01<188:34:55,  6.35s/it]
 52%|█████▏    | 118045/225000 [243:18:04<167:02:31,  5.62s/it]
 52%|█████▏    | 118046/225000 [243:18:09<160:14:14,  5.39s/it]
 52%|█████▏    | 118047/225000 [243:18:13<145:08:39,  4.89s/it]
 52%|█████▏    | 118048/225000 [243:18:19<155:44:45,  5.24s/it]
 52%|█████▏    | 118049/225000 [243:18:29<195:11:04,  6.57s/it]
 52%|█████▏    | 118050/225000 [243:18:32<170:06:02,  5.73s/it]
                                                               
{'loss': '0.8803', 'grad_norm': '1.239', 'learning_rate': '2.482e-05', 'epoch': '0.5247', 'num_input_tokens_seen': 405912320, 'train_runtime': '8.759e+05', 'train_tokens_per_second': '463.4'}

 52%|█████▏    | 118050/225000 [243:18:33<170:06:02,  5.73s/it]
 52%|█████▏    | 118051/225000 [243:18:41<192:36:55,  6.48s/it]
 52%|█████▏    | 118052/225000 [243:18:46<184:31:19,  6.21s/it]
 52%|█████▏    | 118053/225000 [243:18:58<231:30:59,  7.79s/it]
 52%|█████▏    | 118054/225000 [243:19:01<190:10:12,  6.40s/it]
 52%|█████▏    | 118055/225000 [243:19:04<164:17:41,  5.53s/it]
 52%|█████▏    | 118056/225000 [243:19:08<149:23:28,  5.03s/it]
 52%|█████▏    | 118057/225000 [243:19:14<155:46:02,  5.24s/it]
 52%|█████▏    | 118058/225000 [243:19:23<189:03:58,  6.36s/it]
 52%|█████▏    | 118059/225000 [243:19:27<165:19:20,  5.57s/it]
 52%|█████▏    | 118060/225000 [243:19:30<148:22:37,  4.99s/it]
 52%|█████▏    | 118061/225000 [243:19:41<196:27:53,  6.61s/it]
 52%|█████▏    | 118062/225000 [243:19:45<175:11:10,  5.90s/it]
 52%|█████▏    | 118063/225000 [243:19:50<163:31:03,  5.50s/it]
 52%|█████▏    | 118064/225000 [243:19:59<195:38:51,  6.59s/it]
 52%|█████▏    | 118065/225000 [243:20:04<180:00:02,  6.06s/it]
 52%|█████▏    | 118066/225000 [243:20:07<161:00:07,  5.42s/it]
 52%|█████▏    | 118067/225000 [243:20:12<156:04:04,  5.25s/it]
 52%|█████▏    | 118068/225000 [243:20:20<181:42:50,  6.12s/it]
 52%|█████▏    | 118069/225000 [243:20:24<161:29:07,  5.44s/it]
 52%|█████▏    | 118070/225000 [243:20:33<186:16:51,  6.27s/it]
 52%|█████▏    | 118071/225000 [243:20:37<174:25:02,  5.87s/it]
 52%|█████▏    | 118072/225000 [243:20:41<154:16:57,  5.19s/it]
 52%|█████▏    | 118073/225000 [243:20:45<143:58:18,  4.85s/it]
 52%|█████▏    | 118074/225000 [243:20:48<130:21:19,  4.39s/it]
 52%|█████▏    | 118075/225000 [243:20:53<135:10:47,  4.55s/it]
 52%|█████▏    | 118076/225000 [243:21:05<202:12:48,  6.81s/it]
 52%|█████▏    | 118077/225000 [243:21:08<166:29:38,  5.61s/it]
 52%|█████▏    | 118078/225000 [243:21:12<150:02:07,  5.05s/it]
 52%|█████▏    | 118079/225000 [243:21:16<143:25:53,  4.83s/it]
 52%|█████▏    | 118080/225000 [243:21:20<137:27:48,  4.63s/it]
 52%|█████▏    | 118081/225000 [243:21:25<137:07:37,  4.62s/it]
 52%|█████▏    | 118082/225000 [243:21:30<138:03:52,  4.65s/it]
 52%|█████▏    | 118083/225000 [243:21:34<135:17:34,  4.56s/it]
 52%|█████▏    | 118084/225000 [243:21:40<144:34:01,  4.87s/it]
 52%|█████▏    | 118085/225000 [243:21:53<215:07:26,  7.24s/it]
 52%|█████▏    | 118086/225000 [243:21:59<205:15:44,  6.91s/it]
 52%|█████▏    | 118087/225000 [243:22:02<173:33:22,  5.84s/it]
 52%|█████▏    | 118088/225000 [243:22:11<200:53:24,  6.76s/it]
 52%|█████▏    | 118089/225000 [243:22:16<188:21:39,  6.34s/it]
 52%|█████▏    | 118090/225000 [243:22:21<171:42:26,  5.78s/it]
 52%|█████▏    | 118091/225000 [243:22:26<167:24:15,  5.64s/it]
 52%|█████▏    | 118092/225000 [243:22:35<193:06:15,  6.50s/it]
 52%|█████▏    | 118093/225000 [243:22:39<177:42:33,  5.98s/it]
 52%|█████▏    | 118094/225000 [243:22:43<158:06:44,  5.32s/it]
 52%|█████▏    | 118095/225000 [243:22:47<142:39:46,  4.80s/it]
 52%|█████▏    | 118096/225000 [243:22:51<137:44:01,  4.64s/it]
 52%|█████▏    | 118097/225000 [243:23:08<244:14:53,  8.23s/it]
 52%|█████▏    | 118098/225000 [243:23:11<199:24:53,  6.72s/it]
 52%|█████▏    | 118099/225000 [243:23:17<194:26:15,  6.55s/it]
 52%|█████▏    | 118100/225000 [243:23:23<190:39:22,  6.42s/it]
                                                               
{'loss': '0.8782', 'grad_norm': '0.9851', 'learning_rate': '2.48e-05', 'epoch': '0.5249', 'num_input_tokens_seen': 406036784, 'train_runtime': '8.762e+05', 'train_tokens_per_second': '463.4'}

 52%|█████▏    | 118100/225000 [243:23:23<190:39:22,  6.42s/it]
 52%|█████▏    | 118101/225000 [243:23:26<157:56:27,  5.32s/it]
 52%|█████▏    | 118102/225000 [243:23:31<154:21:40,  5.20s/it]
 52%|█████▏    | 118103/225000 [243:23:40<186:28:23,  6.28s/it]
 52%|█████▏    | 118104/225000 [243:23:44<171:32:28,  5.78s/it]
 52%|█████▏    | 118105/225000 [243:23:48<156:20:14,  5.27s/it]
 52%|█████▏    | 118106/225000 [243:23:52<146:22:25,  4.93s/it]
 52%|█████▏    | 118107/225000 [243:23:57<143:25:33,  4.83s/it]
 52%|█████▏    | 118108/225000 [243:24:01<137:17:23,  4.62s/it]
 52%|█████▏    | 118109/225000 [243:24:13<203:50:11,  6.87s/it]
 52%|█████▏    | 118110/225000 [243:24:19<196:41:22,  6.62s/it]
 52%|█████▏    | 118111/225000 [243:24:23<170:14:19,  5.73s/it]
 52%|█████▏    | 118112/225000 [243:24:26<149:16:31,  5.03s/it]
 52%|█████▏    | 118113/225000 [243:24:30<135:50:56,  4.58s/it]
 52%|█████▏    | 118114/225000 [243:24:36<154:10:13,  5.19s/it]
 52%|█████▏    | 118115/225000 [243:24:49<221:04:09,  7.45s/it]
 52%|█████▏    | 118116/225000 [243:24:52<180:54:30,  6.09s/it]
 52%|█████▏    | 118117/225000 [243:24:59<186:13:31,  6.27s/it]
 52%|█████▏    | 118118/225000 [243:25:02<163:23:59,  5.50s/it]
 52%|█████▏    | 118119/225000 [243:25:09<173:42:55,  5.85s/it]
 52%|█████▏    | 118120/225000 [243:25:16<185:04:42,  6.23s/it]
 52%|█████▏    | 118121/225000 [243:25:20<166:04:33,  5.59s/it]
 52%|█████▏    | 118122/225000 [243:25:25<155:50:45,  5.25s/it]
 52%|█████▏    | 118123/225000 [243:25:29<147:15:56,  4.96s/it]
 52%|█████▏    | 118124/225000 [243:25:33<136:43:56,  4.61s/it]
 52%|█████▎    | 118125/225000 [243:25:37<132:40:34,  4.47s/it]
 53%|█████▎    | 118126/225000 [243:25:47<182:59:26,  6.16s/it]
 53%|█████▎    | 118127/225000 [243:25:55<197:54:19,  6.67s/it]
 53%|█████▎    | 118128/225000 [243:25:59<172:56:44,  5.83s/it]
 53%|█████▎    | 118129/225000 [243:26:03<155:58:24,  5.25s/it]
 53%|█████▎    | 118130/225000 [243:26:07<142:49:50,  4.81s/it]
 53%|█████▎    | 118131/225000 [243:26:11<136:20:21,  4.59s/it]
 53%|█████▎    | 118132/225000 [243:26:15<132:14:31,  4.45s/it]
 53%|█████▎    | 118133/225000 [243:26:23<166:26:49,  5.61s/it]
 53%|█████▎    | 118134/225000 [243:26:31<190:38:51,  6.42s/it]
 53%|█████▎    | 118135/225000 [243:26:36<173:53:02,  5.86s/it]
 53%|█████▎    | 118136/225000 [243:26:40<159:41:43,  5.38s/it]
 53%|█████▎    | 118137/225000 [243:26:45<150:25:25,  5.07s/it]
 53%|█████▎    | 118138/225000 [243:26:49<143:13:48,  4.83s/it]
 53%|█████▎    | 118139/225000 [243:26:53<141:05:19,  4.75s/it]
 53%|█████▎    | 118140/225000 [243:26:58<139:35:15,  4.70s/it]
 53%|█████▎    | 118141/225000 [243:27:02<136:28:18,  4.60s/it]
 53%|█████▎    | 118142/225000 [243:27:07<138:41:07,  4.67s/it]
 53%|█████▎    | 118143/225000 [243:27:16<180:19:21,  6.08s/it]
 53%|█████▎    | 118144/225000 [243:27:26<208:38:40,  7.03s/it]
 53%|█████▎    | 118145/225000 [243:27:28<169:49:27,  5.72s/it]
 53%|█████▎    | 118146/225000 [243:27:32<152:51:05,  5.15s/it]
 53%|█████▎    | 118147/225000 [243:27:37<150:46:36,  5.08s/it]
 53%|█████▎    | 118148/225000 [243:27:42<147:05:11,  4.96s/it]
 53%|█████▎    | 118149/225000 [243:27:46<142:04:31,  4.79s/it]
 53%|█████▎    | 118150/225000 [243:27:51<138:00:09,  4.65s/it]
                                                               
{'loss': '0.8763', 'grad_norm': '1.426', 'learning_rate': '2.478e-05', 'epoch': '0.5251', 'num_input_tokens_seen': 406157944, 'train_runtime': '8.765e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118150/225000 [243:27:51<138:00:09,  4.65s/it]
 53%|█████▎    | 118151/225000 [243:27:55<138:50:42,  4.68s/it]
 53%|█████▎    | 118152/225000 [243:28:08<211:52:25,  7.14s/it]
 53%|█████▎    | 118153/225000 [243:28:15<211:32:27,  7.13s/it]
 53%|█████▎    | 118154/225000 [243:28:19<180:29:27,  6.08s/it]
 53%|█████▎    | 118155/225000 [243:28:23<166:18:42,  5.60s/it]
 53%|█████▎    | 118156/225000 [243:28:27<149:49:26,  5.05s/it]
 53%|█████▎    | 118157/225000 [243:28:32<146:53:27,  4.95s/it]
 53%|█████▎    | 118158/225000 [243:28:36<140:08:24,  4.72s/it]
 53%|█████▎    | 118159/225000 [243:28:41<144:04:06,  4.85s/it]
 53%|█████▎    | 118160/225000 [243:28:50<183:22:43,  6.18s/it]
 53%|█████▎    | 118161/225000 [243:29:00<211:50:08,  7.14s/it]
 53%|█████▎    | 118162/225000 [243:29:11<243:26:33,  8.20s/it]
 53%|█████▎    | 118163/225000 [243:29:16<217:35:53,  7.33s/it]
 53%|█████▎    | 118164/225000 [243:29:20<185:26:55,  6.25s/it]
 53%|█████▎    | 118165/225000 [243:29:23<160:40:58,  5.41s/it]
 53%|█████▎    | 118166/225000 [243:29:33<202:17:31,  6.82s/it]
 53%|█████▎    | 118167/225000 [243:29:36<169:18:42,  5.71s/it]
 53%|█████▎    | 118168/225000 [243:29:45<196:09:25,  6.61s/it]
 53%|█████▎    | 118169/225000 [243:29:49<173:05:27,  5.83s/it]
 53%|█████▎    | 118170/225000 [243:29:54<161:56:55,  5.46s/it]
 53%|█████▎    | 118171/225000 [243:29:58<153:15:19,  5.16s/it]
 53%|█████▎    | 118172/225000 [243:30:03<148:49:43,  5.02s/it]
 53%|█████▎    | 118173/225000 [243:30:14<203:16:48,  6.85s/it]
 53%|█████▎    | 118174/225000 [243:30:26<250:16:26,  8.43s/it]
 53%|█████▎    | 118175/225000 [243:30:30<208:02:36,  7.01s/it]
 53%|█████▎    | 118176/225000 [243:30:34<184:45:15,  6.23s/it]
 53%|█████▎    | 118177/225000 [243:30:44<222:00:04,  7.48s/it]
 53%|█████▎    | 118178/225000 [243:30:48<184:45:57,  6.23s/it]
 53%|█████▎    | 118179/225000 [243:30:51<161:19:36,  5.44s/it]
 53%|█████▎    | 118180/225000 [243:31:03<219:11:53,  7.39s/it]
 53%|█████▎    | 118181/225000 [243:31:07<184:00:42,  6.20s/it]
 53%|█████▎    | 118182/225000 [243:31:11<165:41:19,  5.58s/it]
 53%|█████▎    | 118183/225000 [243:31:15<155:13:00,  5.23s/it]
 53%|█████▎    | 118184/225000 [243:31:28<223:00:28,  7.52s/it]
 53%|█████▎    | 118185/225000 [243:31:32<193:20:24,  6.52s/it]
 53%|█████▎    | 118186/225000 [243:31:37<176:38:28,  5.95s/it]
 53%|█████▎    | 118187/225000 [243:31:41<161:41:12,  5.45s/it]
 53%|█████▎    | 118188/225000 [243:31:46<152:07:52,  5.13s/it]
 53%|█████▎    | 118189/225000 [243:31:50<147:04:07,  4.96s/it]
 53%|█████▎    | 118190/225000 [243:32:00<192:40:28,  6.49s/it]
 53%|█████▎    | 118191/225000 [243:32:05<176:54:02,  5.96s/it]
 53%|█████▎    | 118192/225000 [243:32:12<183:18:52,  6.18s/it]
 53%|█████▎    | 118193/225000 [243:32:15<157:27:58,  5.31s/it]
 53%|█████▎    | 118194/225000 [243:32:23<181:30:21,  6.12s/it]
 53%|█████▎    | 118195/225000 [243:32:28<174:17:38,  5.87s/it]
 53%|█████▎    | 118196/225000 [243:32:32<154:08:09,  5.20s/it]
 53%|█████▎    | 118197/225000 [243:32:36<144:49:39,  4.88s/it]
 53%|█████▎    | 118198/225000 [243:32:40<135:23:56,  4.56s/it]
 53%|█████▎    | 118199/225000 [243:32:52<203:50:59,  6.87s/it]
 53%|█████▎    | 118200/225000 [243:32:57<182:11:46,  6.14s/it]
                                                               
{'loss': '0.8893', 'grad_norm': '1.127', 'learning_rate': '2.476e-05', 'epoch': '0.5253', 'num_input_tokens_seen': 406286360, 'train_runtime': '8.768e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118200/225000 [243:32:57<182:11:46,  6.14s/it]
 53%|█████▎    | 118201/225000 [243:33:02<178:07:40,  6.00s/it]
 53%|█████▎    | 118202/225000 [243:33:07<165:48:38,  5.59s/it]
 53%|█████▎    | 118203/225000 [243:33:11<155:34:09,  5.24s/it]
 53%|█████▎    | 118204/225000 [243:33:16<149:45:26,  5.05s/it]
 53%|█████▎    | 118205/225000 [243:33:20<143:34:27,  4.84s/it]
 53%|█████▎    | 118206/225000 [243:33:24<137:22:50,  4.63s/it]
 53%|█████▎    | 118207/225000 [243:33:36<195:23:54,  6.59s/it]
 53%|█████▎    | 118208/225000 [243:33:42<191:48:50,  6.47s/it]
 53%|█████▎    | 118209/225000 [243:33:50<205:45:25,  6.94s/it]
 53%|█████▎    | 118210/225000 [243:33:54<182:56:53,  6.17s/it]
 53%|█████▎    | 118211/225000 [243:33:57<157:23:57,  5.31s/it]
 53%|█████▎    | 118212/225000 [243:34:07<198:16:44,  6.68s/it]
 53%|█████▎    | 118213/225000 [243:34:10<163:41:31,  5.52s/it]
 53%|█████▎    | 118214/225000 [243:34:14<150:25:25,  5.07s/it]
 53%|█████▎    | 118215/225000 [243:34:19<149:50:57,  5.05s/it]
 53%|█████▎    | 118216/225000 [243:34:24<148:04:58,  4.99s/it]
 53%|█████▎    | 118217/225000 [243:34:29<148:25:42,  5.00s/it]
 53%|█████▎    | 118218/225000 [243:34:36<167:25:27,  5.64s/it]
 53%|█████▎    | 118219/225000 [243:34:41<160:31:09,  5.41s/it]
 53%|█████▎    | 118220/225000 [243:34:46<158:38:34,  5.35s/it]
 53%|█████▎    | 118221/225000 [243:34:51<152:25:25,  5.14s/it]
 53%|█████▎    | 118222/225000 [243:34:57<157:18:16,  5.30s/it]
 53%|█████▎    | 118223/225000 [243:35:01<153:17:33,  5.17s/it]
 53%|█████▎    | 118224/225000 [243:35:07<155:05:40,  5.23s/it]
 53%|█████▎    | 118225/225000 [243:35:12<155:46:47,  5.25s/it]
 53%|█████▎    | 118226/225000 [243:35:22<199:05:07,  6.71s/it]
 53%|█████▎    | 118227/225000 [243:35:27<184:36:07,  6.22s/it]
 53%|█████▎    | 118228/225000 [243:35:40<245:52:02,  8.29s/it]
 53%|█████▎    | 118229/225000 [243:35:44<203:37:44,  6.87s/it]
 53%|█████▎    | 118230/225000 [243:35:54<229:12:44,  7.73s/it]
 53%|█████▎    | 118231/225000 [243:35:57<190:08:51,  6.41s/it]
 53%|█████▎    | 118232/225000 [243:36:06<210:10:25,  7.09s/it]
 53%|█████▎    | 118233/225000 [243:36:09<177:22:52,  5.98s/it]
 53%|█████▎    | 118234/225000 [243:36:12<153:48:08,  5.19s/it]
 53%|█████▎    | 118235/225000 [243:36:21<183:18:11,  6.18s/it]
 53%|█████▎    | 118236/225000 [243:36:26<170:26:21,  5.75s/it]
 53%|█████▎    | 118237/225000 [243:36:30<156:34:57,  5.28s/it]
 53%|█████▎    | 118238/225000 [243:36:35<151:57:07,  5.12s/it]
 53%|█████▎    | 118239/225000 [243:36:39<144:57:53,  4.89s/it]
 53%|█████▎    | 118240/225000 [243:36:43<139:51:45,  4.72s/it]
 53%|█████▎    | 118241/225000 [243:36:48<141:37:35,  4.78s/it]
 53%|█████▎    | 118242/225000 [243:36:53<143:13:05,  4.83s/it]
 53%|█████▎    | 118243/225000 [243:37:03<184:21:15,  6.22s/it]
 53%|█████▎    | 118244/225000 [243:37:09<189:46:23,  6.40s/it]
 53%|█████▎    | 118245/225000 [243:37:14<169:26:00,  5.71s/it]
 53%|█████▎    | 118246/225000 [243:37:17<149:35:42,  5.04s/it]
 53%|█████▎    | 118247/225000 [243:37:26<188:25:35,  6.35s/it]
 53%|█████▎    | 118248/225000 [243:37:31<171:07:34,  5.77s/it]
 53%|█████▎    | 118249/225000 [243:37:37<174:59:09,  5.90s/it]
 53%|█████▎    | 118250/225000 [243:37:44<183:32:51,  6.19s/it]
                                                               
{'loss': '0.8813', 'grad_norm': '1.138', 'learning_rate': '2.475e-05', 'epoch': '0.5256', 'num_input_tokens_seen': 406411080, 'train_runtime': '8.771e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118250/225000 [243:37:44<183:32:51,  6.19s/it]
 53%|█████▎    | 118251/225000 [243:37:53<212:27:36,  7.17s/it]
 53%|█████▎    | 118252/225000 [243:37:57<180:03:05,  6.07s/it]
 53%|█████▎    | 118253/225000 [243:38:01<162:35:33,  5.48s/it]
 53%|█████▎    | 118254/225000 [243:38:05<148:54:39,  5.02s/it]
 53%|█████▎    | 118255/225000 [243:38:17<207:46:57,  7.01s/it]
 53%|█████▎    | 118256/225000 [243:38:21<187:48:06,  6.33s/it]
 53%|█████▎    | 118257/225000 [243:38:25<168:05:35,  5.67s/it]
 53%|█████▎    | 118258/225000 [243:38:29<149:51:05,  5.05s/it]
 53%|█████▎    | 118259/225000 [243:38:34<144:58:12,  4.89s/it]
 53%|█████▎    | 118260/225000 [243:38:49<240:41:10,  8.12s/it]
 53%|█████▎    | 118261/225000 [243:38:53<201:39:08,  6.80s/it]
 53%|█████▎    | 118262/225000 [243:38:57<174:03:03,  5.87s/it]
 53%|█████▎    | 118263/225000 [243:39:01<161:55:59,  5.46s/it]
 53%|█████▎    | 118264/225000 [243:39:06<153:27:45,  5.18s/it]
 53%|█████▎    | 118265/225000 [243:39:11<155:15:26,  5.24s/it]
 53%|█████▎    | 118266/225000 [243:39:16<152:48:25,  5.15s/it]
 53%|█████▎    | 118267/225000 [243:39:21<154:33:45,  5.21s/it]
 53%|█████▎    | 118268/225000 [243:39:27<160:58:43,  5.43s/it]
 53%|█████▎    | 118269/225000 [243:39:38<208:28:36,  7.03s/it]
 53%|█████▎    | 118270/225000 [243:39:41<175:14:43,  5.91s/it]
 53%|█████▎    | 118271/225000 [243:39:49<189:17:58,  6.39s/it]
 53%|█████▎    | 118272/225000 [243:39:55<184:53:33,  6.24s/it]
 53%|█████▎    | 118273/225000 [243:40:02<191:55:30,  6.47s/it]
 53%|█████▎    | 118274/225000 [243:40:06<173:26:53,  5.85s/it]
 53%|█████▎    | 118275/225000 [243:40:10<151:54:55,  5.12s/it]
 53%|█████▎    | 118276/225000 [243:40:16<161:50:53,  5.46s/it]
 53%|█████▎    | 118277/225000 [243:40:24<187:56:59,  6.34s/it]
 53%|█████▎    | 118278/225000 [243:40:27<158:43:05,  5.35s/it]
 53%|█████▎    | 118279/225000 [243:40:32<153:38:09,  5.18s/it]
 53%|█████▎    | 118280/225000 [243:40:37<154:50:35,  5.22s/it]
 53%|█████▎    | 118281/225000 [243:40:43<155:07:20,  5.23s/it]
 53%|█████▎    | 118282/225000 [243:40:47<150:34:17,  5.08s/it]
 53%|█████▎    | 118283/225000 [243:40:53<151:54:44,  5.12s/it]
 53%|█████▎    | 118284/225000 [243:40:58<152:10:54,  5.13s/it]
 53%|█████▎    | 118285/225000 [243:41:03<153:14:22,  5.17s/it]
 53%|█████▎    | 118286/225000 [243:41:08<150:49:49,  5.09s/it]
 53%|█████▎    | 118287/225000 [243:41:13<152:50:01,  5.16s/it]
 53%|█████▎    | 118288/225000 [243:41:25<209:55:44,  7.08s/it]
 53%|█████▎    | 118289/225000 [243:41:29<182:38:15,  6.16s/it]
 53%|█████▎    | 118290/225000 [243:41:33<161:01:41,  5.43s/it]
 53%|█████▎    | 118291/225000 [243:41:36<144:30:01,  4.87s/it]
 53%|█████▎    | 118292/225000 [243:41:46<191:40:22,  6.47s/it]
 53%|█████▎    | 118293/225000 [243:41:52<184:08:59,  6.21s/it]
 53%|█████▎    | 118294/225000 [243:41:58<185:51:10,  6.27s/it]
 53%|█████▎    | 118295/225000 [243:42:02<160:43:34,  5.42s/it]
 53%|█████▎    | 118296/225000 [243:42:05<145:49:03,  4.92s/it]
 53%|█████▎    | 118297/225000 [243:42:16<195:59:13,  6.61s/it]
 53%|█████▎    | 118298/225000 [243:42:21<180:44:06,  6.10s/it]
 53%|█████▎    | 118299/225000 [243:42:26<174:46:16,  5.90s/it]
 53%|█████▎    | 118300/225000 [243:42:31<166:32:01,  5.62s/it]
                                                               
{'loss': '0.8718', 'grad_norm': '1.081', 'learning_rate': '2.473e-05', 'epoch': '0.5258', 'num_input_tokens_seen': 406535368, 'train_runtime': '8.774e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118300/225000 [243:42:31<166:32:01,  5.62s/it]
 53%|█████▎    | 118301/225000 [243:42:36<158:10:21,  5.34s/it]
 53%|█████▎    | 118302/225000 [243:42:41<153:22:38,  5.17s/it]
 53%|█████▎    | 118303/225000 [243:42:46<151:44:14,  5.12s/it]
 53%|█████▎    | 118304/225000 [243:42:51<152:50:19,  5.16s/it]
 53%|█████▎    | 118305/225000 [243:42:56<151:27:41,  5.11s/it]
 53%|█████▎    | 118306/225000 [243:43:01<146:17:57,  4.94s/it]
 53%|█████▎    | 118307/225000 [243:43:06<149:14:22,  5.04s/it]
 53%|█████▎    | 118308/225000 [243:43:15<190:09:45,  6.42s/it]
 53%|█████▎    | 118309/225000 [243:43:22<194:29:15,  6.56s/it]
 53%|█████▎    | 118310/225000 [243:43:26<167:30:01,  5.65s/it]
 53%|█████▎    | 118311/225000 [243:43:33<178:30:20,  6.02s/it]
 53%|█████▎    | 118312/225000 [243:43:40<192:36:05,  6.50s/it]
 53%|█████▎    | 118313/225000 [243:43:45<172:16:20,  5.81s/it]
 53%|█████▎    | 118314/225000 [243:43:50<169:26:17,  5.72s/it]
 53%|█████▎    | 118315/225000 [243:43:58<190:55:07,  6.44s/it]
 53%|█████▎    | 118316/225000 [243:44:06<200:37:15,  6.77s/it]
 53%|█████▎    | 118317/225000 [243:44:09<169:39:47,  5.73s/it]
 53%|█████▎    | 118318/225000 [243:44:14<162:19:02,  5.48s/it]
 53%|█████▎    | 118319/225000 [243:44:20<170:00:12,  5.74s/it]
 53%|█████▎    | 118320/225000 [243:44:27<182:23:52,  6.16s/it]
 53%|█████▎    | 118321/225000 [243:44:31<156:54:51,  5.30s/it]
 53%|█████▎    | 118322/225000 [243:44:34<138:30:24,  4.67s/it]
 53%|█████▎    | 118323/225000 [243:44:42<171:27:21,  5.79s/it]
 53%|█████▎    | 118324/225000 [243:44:49<176:15:37,  5.95s/it]
 53%|█████▎    | 118325/225000 [243:44:55<182:50:30,  6.17s/it]
 53%|█████▎    | 118326/225000 [243:44:59<158:07:29,  5.34s/it]
 53%|█████▎    | 118327/225000 [243:45:07<181:07:53,  6.11s/it]
 53%|█████▎    | 118328/225000 [243:45:10<158:49:33,  5.36s/it]
 53%|█████▎    | 118329/225000 [243:45:14<143:55:22,  4.86s/it]
 53%|█████▎    | 118330/225000 [243:45:18<139:40:10,  4.71s/it]
 53%|█████▎    | 118331/225000 [243:45:30<204:07:28,  6.89s/it]
 53%|█████▎    | 118332/225000 [243:45:34<175:07:45,  5.91s/it]
 53%|█████▎    | 118333/225000 [243:45:37<151:35:02,  5.12s/it]
 53%|█████▎    | 118334/225000 [243:45:46<186:36:16,  6.30s/it]
 53%|█████▎    | 118335/225000 [243:45:51<169:00:34,  5.70s/it]
 53%|█████▎    | 118336/225000 [243:45:55<154:26:40,  5.21s/it]
 53%|█████▎    | 118337/225000 [243:45:58<137:27:00,  4.64s/it]
 53%|█████▎    | 118338/225000 [243:46:02<134:16:09,  4.53s/it]
 53%|█████▎    | 118339/225000 [243:46:07<133:04:18,  4.49s/it]
 53%|█████▎    | 118340/225000 [243:46:19<203:48:17,  6.88s/it]
 53%|█████▎    | 118341/225000 [243:46:23<178:49:30,  6.04s/it]
 53%|█████▎    | 118342/225000 [243:46:26<154:48:35,  5.23s/it]
 53%|█████▎    | 118343/225000 [243:46:35<184:13:55,  6.22s/it]
 53%|█████▎    | 118344/225000 [243:46:39<167:31:11,  5.65s/it]
 53%|█████▎    | 118345/225000 [243:46:44<158:11:19,  5.34s/it]
 53%|█████▎    | 118346/225000 [243:46:48<148:41:32,  5.02s/it]
 53%|█████▎    | 118347/225000 [243:46:52<141:51:43,  4.79s/it]
 53%|█████▎    | 118348/225000 [243:46:57<140:14:35,  4.73s/it]
 53%|█████▎    | 118349/225000 [243:47:02<140:27:37,  4.74s/it]
 53%|█████▎    | 118350/225000 [243:47:07<142:55:52,  4.82s/it]
                                                               
{'loss': '0.8753', 'grad_norm': '1.026', 'learning_rate': '2.471e-05', 'epoch': '0.526', 'num_input_tokens_seen': 406657464, 'train_runtime': '8.776e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118350/225000 [243:47:07<142:55:52,  4.82s/it]
 53%|█████▎    | 118351/225000 [243:47:12<142:59:49,  4.83s/it]
 53%|█████▎    | 118352/225000 [243:47:30<259:18:49,  8.75s/it]
 53%|█████▎    | 118353/225000 [243:47:34<217:19:33,  7.34s/it]
 53%|█████▎    | 118354/225000 [243:47:37<179:49:43,  6.07s/it]
 53%|█████▎    | 118355/225000 [243:47:43<184:46:25,  6.24s/it]
 53%|█████▎    | 118356/225000 [243:47:48<174:37:33,  5.89s/it]
 53%|█████▎    | 118357/225000 [243:47:54<168:18:45,  5.68s/it]
 53%|█████▎    | 118358/225000 [243:47:59<162:02:21,  5.47s/it]
 53%|█████▎    | 118359/225000 [243:48:03<155:13:43,  5.24s/it]
 53%|█████▎    | 118360/225000 [243:48:08<151:35:14,  5.12s/it]
 53%|█████▎    | 118361/225000 [243:48:12<144:03:02,  4.86s/it]
 53%|█████▎    | 118362/225000 [243:48:17<144:30:24,  4.88s/it]
 53%|█████▎    | 118363/225000 [243:48:23<148:23:34,  5.01s/it]
 53%|█████▎    | 118364/225000 [243:48:28<147:04:14,  4.97s/it]
 53%|█████▎    | 118365/225000 [243:48:33<149:38:33,  5.05s/it]
 53%|█████▎    | 118366/225000 [243:48:46<220:05:42,  7.43s/it]
 53%|█████▎    | 118367/225000 [243:48:49<185:42:10,  6.27s/it]
 53%|█████▎    | 118368/225000 [243:48:55<182:36:03,  6.16s/it]
 53%|█████▎    | 118369/225000 [243:49:02<190:42:34,  6.44s/it]
 53%|█████▎    | 118370/225000 [243:49:06<166:37:07,  5.63s/it]
 53%|█████▎    | 118371/225000 [243:49:10<148:42:04,  5.02s/it]
 53%|█████▎    | 118372/225000 [243:49:13<137:17:25,  4.64s/it]
 53%|█████▎    | 118373/225000 [243:49:25<200:49:02,  6.78s/it]
 53%|█████▎    | 118374/225000 [243:49:30<180:43:50,  6.10s/it]
 53%|█████▎    | 118375/225000 [243:49:34<162:30:07,  5.49s/it]
 53%|█████▎    | 118376/225000 [243:49:38<148:38:00,  5.02s/it]
 53%|█████▎    | 118377/225000 [243:49:42<142:01:43,  4.80s/it]
 53%|█████▎    | 118378/225000 [243:49:46<135:09:25,  4.56s/it]
 53%|█████▎    | 118379/225000 [243:49:56<183:16:42,  6.19s/it]
 53%|█████▎    | 118380/225000 [243:50:04<202:36:48,  6.84s/it]
 53%|█████▎    | 118381/225000 [243:50:08<171:13:57,  5.78s/it]
 53%|█████▎    | 118382/225000 [243:50:12<155:57:59,  5.27s/it]
 53%|█████▎    | 118383/225000 [243:50:15<142:08:13,  4.80s/it]
 53%|█████▎    | 118384/225000 [243:50:27<204:16:54,  6.90s/it]
 53%|█████▎    | 118385/225000 [243:50:31<172:52:13,  5.84s/it]
 53%|█████▎    | 118386/225000 [243:50:36<166:15:00,  5.61s/it]
 53%|█████▎    | 118387/225000 [243:50:40<157:50:37,  5.33s/it]
 53%|█████▎    | 118388/225000 [243:50:46<158:18:48,  5.35s/it]
 53%|█████▎    | 118389/225000 [243:50:51<153:30:07,  5.18s/it]
 53%|█████▎    | 118390/225000 [243:50:55<151:46:03,  5.12s/it]
 53%|█████▎    | 118391/225000 [243:51:01<151:31:26,  5.12s/it]
 53%|█████▎    | 118392/225000 [243:51:06<151:56:12,  5.13s/it]
 53%|█████▎    | 118393/225000 [243:51:10<148:01:17,  5.00s/it]
 53%|█████▎    | 118394/225000 [243:51:19<182:28:27,  6.16s/it]
 53%|█████▎    | 118395/225000 [243:51:30<222:22:24,  7.51s/it]
 53%|█████▎    | 118396/225000 [243:51:33<184:23:18,  6.23s/it]
 53%|█████▎    | 118397/225000 [243:51:37<161:18:14,  5.45s/it]
 53%|█████▎    | 118398/225000 [243:51:41<151:38:00,  5.12s/it]
 53%|█████▎    | 118399/225000 [243:51:46<147:15:22,  4.97s/it]
 53%|█████▎    | 118400/225000 [243:51:51<148:36:45,  5.02s/it]
                                                               
{'loss': '0.8829', 'grad_norm': '1.085', 'learning_rate': '2.469e-05', 'epoch': '0.5262', 'num_input_tokens_seen': 406779944, 'train_runtime': '8.779e+05', 'train_tokens_per_second': '463.4'}

 53%|█████▎    | 118400/225000 [243:51:51<148:36:45,  5.02s/it]
 53%|█████▎    | 118401/225000 [243:52:06<241:25:59,  8.15s/it]
 53%|█████▎    | 118402/225000 [243:52:11<206:36:13,  6.98s/it]
 53%|█████▎    | 118403/225000 [243:52:14<175:17:57,  5.92s/it]
 53%|█████▎    | 118404/225000 [243:52:26<232:18:38,  7.85s/it]
 53%|█████▎    | 118405/225000 [243:52:30<190:18:34,  6.43s/it]
 53%|█████▎    | 118406/225000 [243:52:34<170:18:08,  5.75s/it]
 53%|█████▎    | 118407/225000 [243:52:39<162:27:08,  5.49s/it]
 53%|█████▎    | 118408/225000 [243:52:43<149:27:06,  5.05s/it]
 53%|█████▎    | 118409/225000 [243:52:47<143:48:47,  4.86s/it]
 53%|█████▎    | 118410/225000 [243:52:51<139:08:29,  4.70s/it]
 53%|█████▎    | 118411/225000 [243:52:56<136:50:26,  4.62s/it]
 53%|█████▎    | 118412/225000 [243:53:07<196:43:23,  6.64s/it]
 53%|█████▎    | 118413/225000 [243:53:15<203:52:31,  6.89s/it]
 53%|█████▎    | 118414/225000 [243:53:18<170:09:40,  5.75s/it]
 53%|█████▎    | 118415/225000 [243:53:21<149:02:38,  5.03s/it]
 53%|█████▎    | 118416/225000 [243:53:25<143:16:17,  4.84s/it]
 53%|█████▎    | 118417/225000 [243:53:29<135:10:22,  4.57s/it]
 53%|█████▎    | 118418/225000 [243:53:37<165:00:37,  5.57s/it]
 53%|█████▎    | 118419/225000 [243:53:44<171:47:23,  5.80s/it]
 53%|█████▎    | 118420/225000 [243:53:48<157:24:33,  5.32s/it]
 53%|█████▎    | 118421/225000 [243:53:52<147:26:42,  4.98s/it]
 53%|█████▎    | 118422/225000 [243:53:56<139:47:44,  4.72s/it]
 53%|█████▎    | 118423/225000 [243:54:00<128:57:44,  4.36s/it]
 53%|█████▎    | 118424/225000 [243:54:12<196:04:39,  6.62s/it]
 53%|█████▎    | 118425/225000 [243:54:17<181:46:24,  6.14s/it]
 53%|█████▎    | 118426/225000 [243:54:20<160:31:40,  5.42s/it]
 53%|█████▎    | 118427/225000 [243:54:25<152:51:42,  5.16s/it]
 53%|█████▎    | 118428/225000 [243:54:34<189:48:46,  6.41s/it]
 53%|█████▎    | 118429/225000 [243:54:38<169:03:26,  5.71s/it]
 53%|█████▎    | 118430/225000 [243:54:42<153:57:55,  5.20s/it]
 53%|█████▎    | 118431/225000 [243:54:49<169:25:17,  5.72s/it]
 53%|█████▎    | 118432/225000 [243:54:59<200:47:58,  6.78s/it]
 53%|█████▎    | 118433/225000 [243:55:02<171:38:52,  5.80s/it]
 53%|█████▎    | 118434/225000 [243:55:10<191:25:41,  6.47s/it]
 53%|█████▎    | 118435/225000 [243:55:16<185:23:10,  6.26s/it]
 53%|█████▎    | 118436/225000 [243:55:20<171:00:50,  5.78s/it]
 53%|█████▎    | 118437/225000 [243:55:25<159:23:35,  5.38s/it]
 53%|█████▎    | 118438/225000 [243:55:30<155:58:22,  5.27s/it]
 53%|█████▎    | 118439/225000 [243:55:35<150:37:13,  5.09s/it]
 53%|█████▎    | 118440/225000 [243:55:39<144:28:59,  4.88s/it]
 53%|█████▎    | 118441/225000 [243:55:48<179:08:28,  6.05s/it]
 53%|█████▎    | 118442/225000 [243:55:56<194:33:47,  6.57s/it]
 53%|█████▎    | 118443/225000 [243:55:59<166:33:58,  5.63s/it]
 53%|█████▎    | 118444/225000 [243:56:04<160:22:18,  5.42s/it]
 53%|█████▎    | 118445/225000 [243:56:09<158:18:04,  5.35s/it]
 53%|█████▎    | 118446/225000 [243:56:14<158:38:53,  5.36s/it]
 53%|█████▎    | 118447/225000 [243:56:19<153:19:37,  5.18s/it]
 53%|█████▎    | 118448/225000 [243:56:24<152:50:58,  5.16s/it]
 53%|█████▎    | 118449/225000 [243:56:29<149:37:23,  5.06s/it]
 53%|█████▎    | 118450/225000 [243:56:34<144:31:53,  4.88s/it]
                                                               
{'loss': '0.8702', 'grad_norm': '1.155', 'learning_rate': '2.467e-05', 'epoch': '0.5264', 'num_input_tokens_seen': 406902552, 'train_runtime': '8.782e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118450/225000 [243:56:34<144:31:53,  4.88s/it]
 53%|█████▎    | 118451/225000 [243:56:39<149:35:30,  5.05s/it]
 53%|█████▎    | 118452/225000 [243:56:44<144:45:40,  4.89s/it]
 53%|█████▎    | 118453/225000 [243:56:49<147:43:48,  4.99s/it]
 53%|█████▎    | 118454/225000 [243:56:54<153:11:38,  5.18s/it]
 53%|█████▎    | 118455/225000 [243:57:06<208:00:13,  7.03s/it]
 53%|█████▎    | 118456/225000 [243:57:11<192:07:38,  6.49s/it]
 53%|█████▎    | 118457/225000 [243:57:17<185:54:29,  6.28s/it]
 53%|█████▎    | 118458/225000 [243:57:25<198:53:01,  6.72s/it]
 53%|█████▎    | 118459/225000 [243:57:28<166:52:34,  5.64s/it]
 53%|█████▎    | 118460/225000 [243:57:36<188:57:33,  6.38s/it]
 53%|█████▎    | 118461/225000 [243:57:42<188:47:40,  6.38s/it]
 53%|█████▎    | 118462/225000 [243:57:48<188:01:30,  6.35s/it]
 53%|█████▎    | 118463/225000 [243:57:53<168:04:28,  5.68s/it]
 53%|█████▎    | 118464/225000 [243:57:57<152:37:37,  5.16s/it]
 53%|█████▎    | 118465/225000 [243:58:01<143:54:14,  4.86s/it]
 53%|█████▎    | 118466/225000 [243:58:05<137:02:09,  4.63s/it]
 53%|█████▎    | 118467/225000 [243:58:16<193:36:55,  6.54s/it]
 53%|█████▎    | 118468/225000 [243:58:21<178:50:34,  6.04s/it]
 53%|█████▎    | 118469/225000 [243:58:28<191:52:49,  6.48s/it]
 53%|█████▎    | 118470/225000 [243:58:32<167:53:38,  5.67s/it]
 53%|█████▎    | 118471/225000 [243:58:36<148:52:35,  5.03s/it]
 53%|█████▎    | 118472/225000 [243:58:43<171:29:09,  5.80s/it]
 53%|█████▎    | 118473/225000 [243:58:48<166:50:05,  5.64s/it]
 53%|█████▎    | 118474/225000 [243:58:53<156:17:12,  5.28s/it]
 53%|█████▎    | 118475/225000 [243:58:57<150:07:49,  5.07s/it]
 53%|█████▎    | 118476/225000 [243:59:06<183:58:00,  6.22s/it]
 53%|█████▎    | 118477/225000 [243:59:12<182:04:09,  6.15s/it]
 53%|█████▎    | 118478/225000 [243:59:15<155:45:39,  5.26s/it]
 53%|█████▎    | 118479/225000 [243:59:19<143:30:36,  4.85s/it]
 53%|█████▎    | 118480/225000 [243:59:28<175:42:22,  5.94s/it]
 53%|█████▎    | 118481/225000 [243:59:31<151:44:42,  5.13s/it]
 53%|█████▎    | 118482/225000 [243:59:35<141:44:42,  4.79s/it]
 53%|█████▎    | 118483/225000 [243:59:46<196:05:04,  6.63s/it]
 53%|█████▎    | 118484/225000 [243:59:50<171:38:18,  5.80s/it]
 53%|█████▎    | 118485/225000 [243:59:54<157:08:43,  5.31s/it]
 53%|█████▎    | 118486/225000 [243:59:58<149:11:37,  5.04s/it]
 53%|█████▎    | 118487/225000 [244:00:08<189:45:38,  6.41s/it]
 53%|█████▎    | 118488/225000 [244:00:12<164:08:24,  5.55s/it]
 53%|█████▎    | 118489/225000 [244:00:15<144:17:55,  4.88s/it]
 53%|█████▎    | 118490/225000 [244:00:19<140:18:34,  4.74s/it]
 53%|█████▎    | 118491/225000 [244:00:25<144:24:34,  4.88s/it]
 53%|█████▎    | 118492/225000 [244:00:29<140:55:01,  4.76s/it]
 53%|█████▎    | 118493/225000 [244:00:40<192:07:52,  6.49s/it]
 53%|█████▎    | 118494/225000 [244:00:45<181:50:05,  6.15s/it]
 53%|█████▎    | 118495/225000 [244:00:50<173:11:10,  5.85s/it]
 53%|█████▎    | 118496/225000 [244:00:56<170:35:29,  5.77s/it]
 53%|█████▎    | 118497/225000 [244:01:01<165:00:56,  5.58s/it]
 53%|█████▎    | 118498/225000 [244:01:09<186:32:38,  6.31s/it]
 53%|█████▎    | 118499/225000 [244:01:17<199:30:26,  6.74s/it]
 53%|█████▎    | 118500/225000 [244:01:20<173:23:33,  5.86s/it]
                                                               
{'loss': '0.8637', 'grad_norm': '1.178', 'learning_rate': '2.465e-05', 'epoch': '0.5267', 'num_input_tokens_seen': 407025824, 'train_runtime': '8.785e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118500/225000 [244:01:20<173:23:33,  5.86s/it]
 53%|█████▎    | 118501/225000 [244:01:24<157:54:03,  5.34s/it]
 53%|█████▎    | 118502/225000 [244:01:29<149:40:08,  5.06s/it]
 53%|█████▎    | 118503/225000 [244:01:33<138:28:21,  4.68s/it]
 53%|█████▎    | 118504/225000 [244:01:37<135:59:35,  4.60s/it]
 53%|█████▎    | 118505/225000 [244:01:42<137:04:29,  4.63s/it]
 53%|█████▎    | 118506/225000 [244:01:54<205:58:33,  6.96s/it]
 53%|█████▎    | 118507/225000 [244:01:57<171:34:18,  5.80s/it]
 53%|█████▎    | 118508/225000 [244:02:00<146:59:03,  4.97s/it]
 53%|█████▎    | 118509/225000 [244:02:07<165:59:26,  5.61s/it]
 53%|█████▎    | 118510/225000 [244:02:11<149:24:30,  5.05s/it]
 53%|█████▎    | 118511/225000 [244:02:20<180:52:52,  6.11s/it]
 53%|█████▎    | 118512/225000 [244:02:23<159:05:12,  5.38s/it]
 53%|█████▎    | 118513/225000 [244:02:28<155:31:08,  5.26s/it]
 53%|█████▎    | 118514/225000 [244:02:34<156:03:49,  5.28s/it]
 53%|█████▎    | 118515/225000 [244:02:40<161:22:47,  5.46s/it]
 53%|█████▎    | 118516/225000 [244:02:44<155:04:25,  5.24s/it]
 53%|█████▎    | 118517/225000 [244:02:50<155:45:16,  5.27s/it]
 53%|█████▎    | 118518/225000 [244:03:01<207:54:49,  7.03s/it]
 53%|█████▎    | 118519/225000 [244:03:05<181:45:23,  6.14s/it]
 53%|█████▎    | 118520/225000 [244:03:09<166:51:30,  5.64s/it]
 53%|█████▎    | 118521/225000 [244:03:14<155:31:13,  5.26s/it]
 53%|█████▎    | 118522/225000 [244:03:18<146:17:58,  4.95s/it]
 53%|█████▎    | 118523/225000 [244:03:23<143:43:02,  4.86s/it]
 53%|█████▎    | 118524/225000 [244:03:35<209:13:17,  7.07s/it]
 53%|█████▎    | 118525/225000 [244:03:39<184:49:22,  6.25s/it]
 53%|█████▎    | 118526/225000 [244:03:45<179:19:41,  6.06s/it]
 53%|█████▎    | 118527/225000 [244:03:52<190:57:44,  6.46s/it]
 53%|█████▎    | 118528/225000 [244:03:58<181:34:06,  6.14s/it]
 53%|█████▎    | 118529/225000 [244:04:01<159:08:52,  5.38s/it]
 53%|█████▎    | 118530/225000 [244:04:05<144:37:00,  4.89s/it]
 53%|█████▎    | 118531/225000 [244:04:14<177:41:03,  6.01s/it]
 53%|█████▎    | 118532/225000 [244:04:18<166:43:34,  5.64s/it]
 53%|█████▎    | 118533/225000 [244:04:22<153:52:42,  5.20s/it]
 53%|█████▎    | 118534/225000 [244:04:27<146:01:02,  4.94s/it]
 53%|█████▎    | 118535/225000 [244:04:32<150:33:01,  5.09s/it]
 53%|█████▎    | 118536/225000 [244:04:37<148:10:21,  5.01s/it]
 53%|█████▎    | 118537/225000 [244:04:42<146:47:54,  4.96s/it]
 53%|█████▎    | 118538/225000 [244:04:47<148:33:21,  5.02s/it]
 53%|█████▎    | 118539/225000 [244:04:51<140:33:32,  4.75s/it]
 53%|█████▎    | 118540/225000 [244:04:56<144:27:45,  4.89s/it]
 53%|█████▎    | 118541/225000 [244:05:11<227:56:49,  7.71s/it]
 53%|█████▎    | 118542/225000 [244:05:14<192:34:40,  6.51s/it]
 53%|█████▎    | 118543/225000 [244:05:18<169:12:51,  5.72s/it]
 53%|█████▎    | 118544/225000 [244:05:22<154:32:11,  5.23s/it]
 53%|█████▎    | 118545/225000 [244:05:30<177:09:32,  5.99s/it]
 53%|█████▎    | 118546/225000 [244:05:40<208:56:36,  7.07s/it]
 53%|█████▎    | 118547/225000 [244:05:44<187:48:02,  6.35s/it]
 53%|█████▎    | 118548/225000 [244:05:49<172:13:35,  5.82s/it]
 53%|█████▎    | 118549/225000 [244:05:54<165:03:09,  5.58s/it]
 53%|█████▎    | 118550/225000 [244:05:59<157:21:20,  5.32s/it]
                                                               
{'loss': '0.8719', 'grad_norm': '1.103', 'learning_rate': '2.464e-05', 'epoch': '0.5269', 'num_input_tokens_seen': 407150088, 'train_runtime': '8.788e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118550/225000 [244:05:59<157:21:20,  5.32s/it]
 53%|█████▎    | 118551/225000 [244:06:04<154:33:35,  5.23s/it]
 53%|█████▎    | 118552/225000 [244:06:09<152:57:27,  5.17s/it]
 53%|█████▎    | 118553/225000 [244:06:13<147:27:26,  4.99s/it]
 53%|█████▎    | 118554/225000 [244:06:21<167:47:01,  5.67s/it]
 53%|█████▎    | 118555/225000 [244:06:33<230:44:16,  7.80s/it]
 53%|█████▎    | 118556/225000 [244:06:38<199:54:23,  6.76s/it]
 53%|█████▎    | 118557/225000 [244:06:46<213:04:58,  7.21s/it]
 53%|█████▎    | 118558/225000 [244:06:49<175:04:26,  5.92s/it]
 53%|█████▎    | 118559/225000 [244:06:54<170:56:13,  5.78s/it]
 53%|█████▎    | 118560/225000 [244:07:01<182:38:31,  6.18s/it]
 53%|█████▎    | 118561/225000 [244:07:06<164:45:31,  5.57s/it]
 53%|█████▎    | 118562/225000 [244:07:10<157:28:48,  5.33s/it]
 53%|█████▎    | 118563/225000 [244:07:18<182:27:24,  6.17s/it]
 53%|█████▎    | 118564/225000 [244:07:22<158:18:38,  5.35s/it]
 53%|█████▎    | 118565/225000 [244:07:29<177:28:52,  6.00s/it]
 53%|█████▎    | 118566/225000 [244:07:35<170:51:37,  5.78s/it]
 53%|█████▎    | 118567/225000 [244:07:38<152:23:32,  5.15s/it]
 53%|█████▎    | 118568/225000 [244:07:42<137:43:08,  4.66s/it]
 53%|█████▎    | 118569/225000 [244:07:46<134:35:32,  4.55s/it]
 53%|█████▎    | 118570/225000 [244:07:51<139:21:24,  4.71s/it]
 53%|█████▎    | 118571/225000 [244:08:02<191:17:57,  6.47s/it]
 53%|█████▎    | 118572/225000 [244:08:05<164:02:17,  5.55s/it]
 53%|█████▎    | 118573/225000 [244:08:09<145:14:59,  4.91s/it]
 53%|█████▎    | 118574/225000 [244:08:18<184:03:58,  6.23s/it]
 53%|█████▎    | 118575/225000 [244:08:21<159:47:48,  5.41s/it]
 53%|█████▎    | 118576/225000 [244:08:25<144:34:32,  4.89s/it]
 53%|█████▎    | 118577/225000 [244:08:36<195:07:22,  6.60s/it]
 53%|█████▎    | 118578/225000 [244:08:39<168:22:51,  5.70s/it]
 53%|█████▎    | 118579/225000 [244:08:44<159:42:05,  5.40s/it]
 53%|█████▎    | 118580/225000 [244:08:53<187:29:18,  6.34s/it]
 53%|█████▎    | 118581/225000 [244:08:56<162:10:11,  5.49s/it]
 53%|█████▎    | 118582/225000 [244:09:01<156:03:40,  5.28s/it]
 53%|█████▎    | 118583/225000 [244:09:06<152:15:19,  5.15s/it]
 53%|█████▎    | 118584/225000 [244:09:11<153:06:14,  5.18s/it]
 53%|█████▎    | 118585/225000 [244:09:15<144:39:16,  4.89s/it]
 53%|█████▎    | 118586/225000 [244:09:20<142:48:13,  4.83s/it]
 53%|█████▎    | 118587/225000 [244:09:25<147:53:40,  5.00s/it]
 53%|█████▎    | 118588/225000 [244:09:36<198:10:58,  6.70s/it]
 53%|█████▎    | 118589/225000 [244:09:41<187:05:20,  6.33s/it]
 53%|█████▎    | 118590/225000 [244:09:45<158:36:22,  5.37s/it]
 53%|█████▎    | 118591/225000 [244:09:48<146:11:23,  4.95s/it]
 53%|█████▎    | 118592/225000 [244:09:54<146:54:24,  4.97s/it]
 53%|█████▎    | 118593/225000 [244:10:05<201:39:12,  6.82s/it]
 53%|█████▎    | 118594/225000 [244:10:08<171:43:41,  5.81s/it]
 53%|█████▎    | 118595/225000 [244:10:12<155:37:53,  5.27s/it]
 53%|█████▎    | 118596/225000 [244:10:16<142:57:59,  4.84s/it]
 53%|█████▎    | 118597/225000 [244:10:21<143:51:43,  4.87s/it]
 53%|█████▎    | 118598/225000 [244:10:26<146:39:52,  4.96s/it]
 53%|█████▎    | 118599/225000 [244:10:31<147:50:21,  5.00s/it]
 53%|█████▎    | 118600/225000 [244:10:36<146:36:36,  4.96s/it]
                                                               
{'loss': '0.8645', 'grad_norm': '1.316', 'learning_rate': '2.462e-05', 'epoch': '0.5271', 'num_input_tokens_seen': 407273568, 'train_runtime': '8.79e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118600/225000 [244:10:36<146:36:36,  4.96s/it]
 53%|█████▎    | 118601/225000 [244:10:40<135:10:45,  4.57s/it]
 53%|█████▎    | 118602/225000 [244:10:52<201:27:37,  6.82s/it]
 53%|█████▎    | 118603/225000 [244:10:57<190:50:16,  6.46s/it]
 53%|█████▎    | 118604/225000 [244:11:01<166:00:27,  5.62s/it]
 53%|█████▎    | 118605/225000 [244:11:05<150:58:39,  5.11s/it]
 53%|█████▎    | 118606/225000 [244:11:13<175:23:43,  5.93s/it]
 53%|█████▎    | 118607/225000 [244:11:21<193:24:29,  6.54s/it]
 53%|█████▎    | 118608/225000 [244:11:25<171:12:27,  5.79s/it]
 53%|█████▎    | 118609/225000 [244:11:29<154:37:15,  5.23s/it]
 53%|█████▎    | 118610/225000 [244:11:33<143:22:02,  4.85s/it]
 53%|█████▎    | 118611/225000 [244:11:36<133:56:40,  4.53s/it]
 53%|█████▎    | 118612/225000 [244:11:44<164:35:03,  5.57s/it]
 53%|█████▎    | 118613/225000 [244:11:53<194:12:55,  6.57s/it]
 53%|█████▎    | 118614/225000 [244:12:00<191:47:24,  6.49s/it]
 53%|█████▎    | 118615/225000 [244:12:03<164:21:32,  5.56s/it]
 53%|█████▎    | 118616/225000 [244:12:09<170:12:01,  5.76s/it]
 53%|█████▎    | 118617/225000 [244:12:13<154:23:00,  5.22s/it]
 53%|█████▎    | 118618/225000 [244:12:22<182:40:31,  6.18s/it]
 53%|█████▎    | 118619/225000 [244:12:25<158:20:16,  5.36s/it]
 53%|█████▎    | 118620/225000 [244:12:29<145:39:10,  4.93s/it]
 53%|█████▎    | 118621/225000 [244:12:37<169:53:41,  5.75s/it]
 53%|█████▎    | 118622/225000 [244:12:43<175:26:45,  5.94s/it]
 53%|█████▎    | 118623/225000 [244:12:51<190:17:44,  6.44s/it]
 53%|█████▎    | 118624/225000 [244:12:55<166:56:54,  5.65s/it]
 53%|█████▎    | 118625/225000 [244:12:59<152:26:50,  5.16s/it]
 53%|█████▎    | 118626/225000 [244:13:03<142:47:11,  4.83s/it]
 53%|█████▎    | 118627/225000 [244:13:07<141:13:05,  4.78s/it]
 53%|█████▎    | 118628/225000 [244:13:13<149:00:39,  5.04s/it]
 53%|█████▎    | 118629/225000 [244:13:26<218:44:02,  7.40s/it]
 53%|█████▎    | 118630/225000 [244:13:30<185:44:05,  6.29s/it]
 53%|█████▎    | 118631/225000 [244:13:33<164:29:32,  5.57s/it]
 53%|█████▎    | 118632/225000 [244:13:42<192:04:50,  6.50s/it]
 53%|█████▎    | 118633/225000 [244:13:46<167:04:21,  5.65s/it]
 53%|█████▎    | 118634/225000 [244:13:54<190:09:18,  6.44s/it]
 53%|█████▎    | 118635/225000 [244:13:58<164:06:16,  5.55s/it]
 53%|█████▎    | 118636/225000 [244:14:01<149:19:14,  5.05s/it]
 53%|█████▎    | 118637/225000 [244:14:09<176:02:47,  5.96s/it]
 53%|█████▎    | 118638/225000 [244:14:15<172:12:45,  5.83s/it]
 53%|█████▎    | 118639/225000 [244:14:18<151:28:33,  5.13s/it]
 53%|█████▎    | 118640/225000 [244:14:22<136:48:10,  4.63s/it]
 53%|█████▎    | 118641/225000 [244:14:31<171:53:14,  5.82s/it]
 53%|█████▎    | 118642/225000 [244:14:37<180:58:26,  6.13s/it]
 53%|█████▎    | 118643/225000 [244:14:41<155:59:19,  5.28s/it]
 53%|█████▎    | 118644/225000 [244:14:44<140:05:46,  4.74s/it]
 53%|█████▎    | 118645/225000 [244:14:54<185:45:15,  6.29s/it]
 53%|█████▎    | 118646/225000 [244:14:57<159:46:56,  5.41s/it]
 53%|█████▎    | 118647/225000 [244:15:07<194:56:21,  6.60s/it]
 53%|█████▎    | 118648/225000 [244:15:11<172:20:50,  5.83s/it]
 53%|█████▎    | 118649/225000 [244:15:15<156:46:19,  5.31s/it]
 53%|█████▎    | 118650/225000 [244:15:20<155:31:06,  5.26s/it]
                                                               
{'loss': '0.8601', 'grad_norm': '1.002', 'learning_rate': '2.46e-05', 'epoch': '0.5273', 'num_input_tokens_seen': 407397952, 'train_runtime': '8.793e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118650/225000 [244:15:20<155:31:06,  5.26s/it]
 53%|█████▎    | 118651/225000 [244:15:26<157:14:41,  5.32s/it]
 53%|█████▎    | 118652/225000 [244:15:30<148:48:14,  5.04s/it]
 53%|█████▎    | 118653/225000 [244:15:34<143:32:29,  4.86s/it]
 53%|█████▎    | 118654/225000 [244:15:39<145:40:29,  4.93s/it]
 53%|█████▎    | 118655/225000 [244:15:44<144:20:40,  4.89s/it]
 53%|█████▎    | 118656/225000 [244:15:50<147:34:59,  5.00s/it]
 53%|█████▎    | 118657/225000 [244:15:54<146:13:36,  4.95s/it]
 53%|█████▎    | 118658/225000 [244:16:00<149:56:54,  5.08s/it]
 53%|█████▎    | 118659/225000 [244:16:09<183:39:09,  6.22s/it]
 53%|█████▎    | 118660/225000 [244:16:18<212:12:49,  7.18s/it]
 53%|█████▎    | 118661/225000 [244:16:21<179:10:28,  6.07s/it]
 53%|█████▎    | 118662/225000 [244:16:25<157:35:56,  5.34s/it]
 53%|█████▎    | 118663/225000 [244:16:30<149:21:32,  5.06s/it]
 53%|█████▎    | 118664/225000 [244:16:34<142:53:38,  4.84s/it]
 53%|█████▎    | 118665/225000 [244:16:44<193:04:57,  6.54s/it]
 53%|█████▎    | 118666/225000 [244:16:50<185:18:57,  6.27s/it]
 53%|█████▎    | 118667/225000 [244:16:55<173:48:59,  5.88s/it]
 53%|█████▎    | 118668/225000 [244:17:03<191:31:28,  6.48s/it]
 53%|█████▎    | 118669/225000 [244:17:08<182:09:16,  6.17s/it]
 53%|█████▎    | 118670/225000 [244:17:14<177:44:17,  6.02s/it]
 53%|█████▎    | 118671/225000 [244:17:18<159:40:43,  5.41s/it]
 53%|█████▎    | 118672/225000 [244:17:22<147:43:09,  5.00s/it]
 53%|█████▎    | 118673/225000 [244:17:26<137:32:46,  4.66s/it]
 53%|█████▎    | 118674/225000 [244:17:30<134:29:40,  4.55s/it]
 53%|█████▎    | 118675/225000 [244:17:34<128:58:31,  4.37s/it]
 53%|█████▎    | 118676/225000 [244:17:44<179:56:19,  6.09s/it]
 53%|█████▎    | 118677/225000 [244:17:52<194:46:39,  6.59s/it]
 53%|█████▎    | 118678/225000 [244:17:56<168:17:45,  5.70s/it]
 53%|█████▎    | 118679/225000 [244:18:02<173:36:28,  5.88s/it]
 53%|█████▎    | 118680/225000 [244:18:09<181:59:52,  6.16s/it]
 53%|█████▎    | 118681/225000 [244:18:12<155:07:58,  5.25s/it]
 53%|█████▎    | 118682/225000 [244:18:16<148:24:09,  5.03s/it]
 53%|█████▎    | 118683/225000 [244:18:20<137:11:14,  4.65s/it]
 53%|█████▎    | 118684/225000 [244:18:25<138:28:41,  4.69s/it]
 53%|█████▎    | 118685/225000 [244:18:29<136:18:39,  4.62s/it]
 53%|█████▎    | 118686/225000 [244:18:35<142:16:40,  4.82s/it]
 53%|█████▎    | 118687/225000 [244:18:40<149:40:00,  5.07s/it]
 53%|█████▎    | 118688/225000 [244:18:50<192:46:51,  6.53s/it]
 53%|█████▎    | 118689/225000 [244:18:55<173:40:24,  5.88s/it]
 53%|█████▎    | 118690/225000 [244:18:59<162:50:30,  5.51s/it]
 53%|█████▎    | 118691/225000 [244:19:07<183:29:17,  6.21s/it]
 53%|█████▎    | 118692/225000 [244:19:11<159:59:08,  5.42s/it]
 53%|█████▎    | 118693/225000 [244:19:19<187:45:59,  6.36s/it]
 53%|█████▎    | 118694/225000 [244:19:24<170:37:29,  5.78s/it]
 53%|█████▎    | 118695/225000 [244:19:31<182:33:28,  6.18s/it]
 53%|█████▎    | 118696/225000 [244:19:34<156:02:26,  5.28s/it]
 53%|█████▎    | 118697/225000 [244:19:42<178:04:41,  6.03s/it]
 53%|█████▎    | 118698/225000 [244:19:47<173:30:10,  5.88s/it]
 53%|█████▎    | 118699/225000 [244:19:52<167:40:43,  5.68s/it]
 53%|█████▎    | 118700/225000 [244:19:56<152:43:18,  5.17s/it]
                                                               
{'loss': '0.8675', 'grad_norm': '1.188', 'learning_rate': '2.458e-05', 'epoch': '0.5276', 'num_input_tokens_seen': 407520656, 'train_runtime': '8.796e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118700/225000 [244:19:56<152:43:18,  5.17s/it]
 53%|█████▎    | 118701/225000 [244:20:01<149:25:11,  5.06s/it]
 53%|█████▎    | 118702/225000 [244:20:06<148:56:38,  5.04s/it]
 53%|█████▎    | 118703/225000 [244:20:10<138:35:30,  4.69s/it]
 53%|█████▎    | 118704/225000 [244:20:15<138:12:05,  4.68s/it]
 53%|█████▎    | 118705/225000 [244:20:19<135:23:57,  4.59s/it]
 53%|█████▎    | 118706/225000 [244:20:30<189:12:06,  6.41s/it]
 53%|█████▎    | 118707/225000 [244:20:36<184:17:59,  6.24s/it]
 53%|█████▎    | 118708/225000 [244:20:42<182:03:47,  6.17s/it]
 53%|█████▎    | 118709/225000 [244:20:48<184:56:31,  6.26s/it]
 53%|█████▎    | 118710/225000 [244:20:54<182:06:07,  6.17s/it]
 53%|█████▎    | 118711/225000 [244:21:00<183:05:31,  6.20s/it]
 53%|█████▎    | 118712/225000 [244:21:04<160:38:12,  5.44s/it]
 53%|█████▎    | 118713/225000 [244:21:08<145:46:17,  4.94s/it]
 53%|█████▎    | 118714/225000 [244:21:11<134:49:20,  4.57s/it]
 53%|█████▎    | 118715/225000 [244:21:22<190:14:05,  6.44s/it]
 53%|█████▎    | 118716/225000 [244:21:26<165:29:25,  5.61s/it]
 53%|█████▎    | 118717/225000 [244:21:30<149:31:31,  5.06s/it]
 53%|█████▎    | 118718/225000 [244:21:38<174:12:29,  5.90s/it]
 53%|█████▎    | 118719/225000 [244:21:43<172:22:44,  5.84s/it]
 53%|█████▎    | 118720/225000 [244:21:47<156:47:31,  5.31s/it]
 53%|█████▎    | 118721/225000 [244:21:56<186:02:51,  6.30s/it]
 53%|█████▎    | 118722/225000 [244:22:01<175:53:06,  5.96s/it]
 53%|█████▎    | 118723/225000 [244:22:11<209:39:45,  7.10s/it]
 53%|█████▎    | 118724/225000 [244:22:14<175:07:37,  5.93s/it]
 53%|█████▎    | 118725/225000 [244:22:17<151:34:29,  5.13s/it]
 53%|█████▎    | 118726/225000 [244:22:22<150:21:28,  5.09s/it]
 53%|█████▎    | 118727/225000 [244:22:28<155:15:13,  5.26s/it]
 53%|█████▎    | 118728/225000 [244:22:34<157:59:05,  5.35s/it]
 53%|█████▎    | 118729/225000 [244:22:39<157:54:46,  5.35s/it]
 53%|█████▎    | 118730/225000 [244:22:47<182:09:58,  6.17s/it]
 53%|█████▎    | 118731/225000 [244:22:53<181:42:30,  6.16s/it]
 53%|█████▎    | 118732/225000 [244:22:57<163:08:00,  5.53s/it]
 53%|█████▎    | 118733/225000 [244:23:01<151:01:38,  5.12s/it]
 53%|█████▎    | 118734/225000 [244:23:06<144:19:21,  4.89s/it]
 53%|█████▎    | 118735/225000 [244:23:10<138:26:07,  4.69s/it]
 53%|█████▎    | 118736/225000 [244:23:17<159:21:24,  5.40s/it]
 53%|█████▎    | 118737/225000 [244:23:25<185:29:27,  6.28s/it]
 53%|█████▎    | 118738/225000 [244:23:29<159:24:39,  5.40s/it]
 53%|█████▎    | 118739/225000 [244:23:32<142:50:49,  4.84s/it]
 53%|█████▎    | 118740/225000 [244:23:42<187:55:17,  6.37s/it]
 53%|█████▎    | 118741/225000 [244:23:46<162:17:58,  5.50s/it]
 53%|█████▎    | 118742/225000 [244:23:51<161:43:01,  5.48s/it]
 53%|█████▎    | 118743/225000 [244:24:00<188:30:27,  6.39s/it]
 53%|█████▎    | 118744/225000 [244:24:06<192:40:54,  6.53s/it]
 53%|█████▎    | 118745/225000 [244:24:10<165:28:26,  5.61s/it]
 53%|█████▎    | 118746/225000 [244:24:14<150:17:51,  5.09s/it]
 53%|█████▎    | 118747/225000 [244:24:18<140:51:22,  4.77s/it]
 53%|█████▎    | 118748/225000 [244:24:23<148:47:48,  5.04s/it]
 53%|█████▎    | 118749/225000 [244:24:33<190:42:39,  6.46s/it]
 53%|█████▎    | 118750/225000 [244:24:37<165:47:25,  5.62s/it]
                                                               
{'loss': '0.8816', 'grad_norm': '1.07', 'learning_rate': '2.456e-05', 'epoch': '0.5278', 'num_input_tokens_seen': 407642512, 'train_runtime': '8.799e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118750/225000 [244:24:37<165:47:25,  5.62s/it]
 53%|█████▎    | 118751/225000 [244:24:45<187:41:09,  6.36s/it]
 53%|█████▎    | 118752/225000 [244:24:50<174:00:52,  5.90s/it]
 53%|█████▎    | 118753/225000 [244:24:54<159:52:16,  5.42s/it]
 53%|█████▎    | 118754/225000 [244:24:57<141:50:11,  4.81s/it]
 53%|█████▎    | 118755/225000 [244:25:02<134:52:28,  4.57s/it]
 53%|█████▎    | 118756/225000 [244:25:10<172:31:08,  5.85s/it]
 53%|█████▎    | 118757/225000 [244:25:17<176:18:45,  5.97s/it]
 53%|█████▎    | 118758/225000 [244:25:22<166:50:13,  5.65s/it]
 53%|█████▎    | 118759/225000 [244:25:27<163:23:03,  5.54s/it]
 53%|█████▎    | 118760/225000 [244:25:31<150:35:32,  5.10s/it]
 53%|█████▎    | 118761/225000 [244:25:35<140:56:02,  4.78s/it]
 53%|█████▎    | 118762/225000 [244:25:39<134:17:32,  4.55s/it]
 53%|█████▎    | 118763/225000 [244:25:51<200:54:14,  6.81s/it]
 53%|█████▎    | 118764/225000 [244:25:55<173:45:46,  5.89s/it]
 53%|█████▎    | 118765/225000 [244:25:58<153:29:16,  5.20s/it]
 53%|█████▎    | 118766/225000 [244:26:08<194:37:30,  6.60s/it]
 53%|█████▎    | 118767/225000 [244:26:12<172:24:30,  5.84s/it]
 53%|█████▎    | 118768/225000 [244:26:16<156:45:20,  5.31s/it]
 53%|█████▎    | 118769/225000 [244:26:21<147:30:01,  5.00s/it]
 53%|█████▎    | 118770/225000 [244:26:25<142:44:26,  4.84s/it]
 53%|█████▎    | 118771/225000 [244:26:36<194:44:32,  6.60s/it]
 53%|█████▎    | 118772/225000 [244:26:41<184:37:34,  6.26s/it]
 53%|█████▎    | 118773/225000 [244:26:45<165:16:30,  5.60s/it]
 53%|█████▎    | 118774/225000 [244:26:50<156:59:28,  5.32s/it]
 53%|█████▎    | 118775/225000 [244:26:55<151:00:01,  5.12s/it]
 53%|█████▎    | 118776/225000 [244:27:00<150:41:49,  5.11s/it]
 53%|█████▎    | 118777/225000 [244:27:04<146:53:09,  4.98s/it]
 53%|█████▎    | 118778/225000 [244:27:09<142:28:14,  4.83s/it]
 53%|█████▎    | 118779/225000 [244:27:14<145:33:07,  4.93s/it]
 53%|█████▎    | 118780/225000 [244:27:19<148:28:42,  5.03s/it]
 53%|█████▎    | 118781/225000 [244:27:28<178:13:13,  6.04s/it]
 53%|█████▎    | 118782/225000 [244:27:39<225:04:23,  7.63s/it]
 53%|█████▎    | 118783/225000 [244:27:45<207:47:57,  7.04s/it]
 53%|█████▎    | 118784/225000 [244:28:00<278:01:27,  9.42s/it]
 53%|█████▎    | 118785/225000 [244:28:03<221:58:57,  7.52s/it]
 53%|█████▎    | 118786/225000 [244:28:10<215:09:42,  7.29s/it]
 53%|█████▎    | 118787/225000 [244:28:14<186:18:06,  6.31s/it]
 53%|█████▎    | 118788/225000 [244:28:18<170:58:40,  5.80s/it]
 53%|█████▎    | 118789/225000 [244:28:22<156:24:14,  5.30s/it]
 53%|█████▎    | 118790/225000 [244:28:27<148:40:59,  5.04s/it]
 53%|█████▎    | 118791/225000 [244:28:37<198:33:24,  6.73s/it]
 53%|█████▎    | 118792/225000 [244:28:42<183:51:34,  6.23s/it]
 53%|█████▎    | 118793/225000 [244:28:49<189:22:21,  6.42s/it]
 53%|█████▎    | 118794/225000 [244:28:53<162:50:53,  5.52s/it]
 53%|█████▎    | 118795/225000 [244:28:59<172:32:22,  5.85s/it]
 53%|█████▎    | 118796/225000 [244:29:04<161:54:29,  5.49s/it]
 53%|█████▎    | 118797/225000 [244:29:08<148:25:23,  5.03s/it]
 53%|█████▎    | 118798/225000 [244:29:13<146:01:22,  4.95s/it]
 53%|█████▎    | 118799/225000 [244:29:17<144:13:52,  4.89s/it]
 53%|█████▎    | 118800/225000 [244:29:22<142:53:35,  4.84s/it]
                                                               
{'loss': '0.8707', 'grad_norm': '1.074', 'learning_rate': '2.454e-05', 'epoch': '0.528', 'num_input_tokens_seen': 407768488, 'train_runtime': '8.802e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118800/225000 [244:29:22<142:53:35,  4.84s/it]
 53%|█████▎    | 118801/225000 [244:29:27<141:19:55,  4.79s/it]
 53%|█████▎    | 118802/225000 [244:29:33<154:38:32,  5.24s/it]
 53%|█████▎    | 118803/225000 [244:29:46<222:50:34,  7.55s/it]
 53%|█████▎    | 118804/225000 [244:29:51<195:14:23,  6.62s/it]
 53%|█████▎    | 118805/225000 [244:29:55<176:53:53,  6.00s/it]
 53%|█████▎    | 118806/225000 [244:30:00<163:17:38,  5.54s/it]
 53%|█████▎    | 118807/225000 [244:30:03<149:01:15,  5.05s/it]
 53%|█████▎    | 118808/225000 [244:30:13<189:09:59,  6.41s/it]
 53%|█████▎    | 118809/225000 [244:30:18<174:48:03,  5.93s/it]
 53%|█████▎    | 118810/225000 [244:30:22<159:50:17,  5.42s/it]
 53%|█████▎    | 118811/225000 [244:30:29<176:44:28,  5.99s/it]
 53%|█████▎    | 118812/225000 [244:30:34<167:04:39,  5.66s/it]
 53%|█████▎    | 118813/225000 [244:30:41<175:59:27,  5.97s/it]
 53%|█████▎    | 118814/225000 [244:30:44<148:46:31,  5.04s/it]
 53%|█████▎    | 118815/225000 [244:30:48<136:52:54,  4.64s/it]
 53%|█████▎    | 118816/225000 [244:30:54<156:56:09,  5.32s/it]
 53%|█████▎    | 118817/225000 [244:31:01<170:03:25,  5.77s/it]
 53%|█████▎    | 118818/225000 [244:31:06<156:35:38,  5.31s/it]
 53%|█████▎    | 118819/225000 [244:31:10<147:41:44,  5.01s/it]
 53%|█████▎    | 118820/225000 [244:31:15<151:45:38,  5.15s/it]
 53%|█████▎    | 118821/225000 [244:31:27<211:12:41,  7.16s/it]
 53%|█████▎    | 118822/225000 [244:31:31<179:14:14,  6.08s/it]
 53%|█████▎    | 118823/225000 [244:31:34<157:52:32,  5.35s/it]
 53%|█████▎    | 118824/225000 [244:31:44<197:56:13,  6.71s/it]
 53%|█████▎    | 118825/225000 [244:31:48<171:09:57,  5.80s/it]
 53%|█████▎    | 118826/225000 [244:31:57<199:17:22,  6.76s/it]
 53%|█████▎    | 118827/225000 [244:32:00<166:24:57,  5.64s/it]
 53%|█████▎    | 118828/225000 [244:32:04<151:58:49,  5.15s/it]
 53%|█████▎    | 118829/225000 [244:32:08<142:50:05,  4.84s/it]
 53%|█████▎    | 118830/225000 [244:32:19<193:18:50,  6.55s/it]
 53%|█████▎    | 118831/225000 [244:32:24<182:10:13,  6.18s/it]
 53%|█████▎    | 118832/225000 [244:32:27<157:30:05,  5.34s/it]
 53%|█████▎    | 118833/225000 [244:32:33<156:14:59,  5.30s/it]
 53%|█████▎    | 118834/225000 [244:32:37<153:04:26,  5.19s/it]
 53%|█████▎    | 118835/225000 [244:32:42<150:56:27,  5.12s/it]
 53%|█████▎    | 118836/225000 [244:32:48<152:18:04,  5.16s/it]
 53%|█████▎    | 118837/225000 [244:32:53<150:29:04,  5.10s/it]
 53%|█████▎    | 118838/225000 [244:33:00<167:34:43,  5.68s/it]
 53%|█████▎    | 118839/225000 [244:33:10<210:13:17,  7.13s/it]
 53%|█████▎    | 118840/225000 [244:33:14<182:07:44,  6.18s/it]
 53%|█████▎    | 118841/225000 [244:33:22<194:22:04,  6.59s/it]
 53%|█████▎    | 118842/225000 [244:33:25<167:06:21,  5.67s/it]
 53%|█████▎    | 118843/225000 [244:33:29<146:11:18,  4.96s/it]
 53%|█████▎    | 118844/225000 [244:33:33<143:14:28,  4.86s/it]
 53%|█████▎    | 118845/225000 [244:33:38<140:57:11,  4.78s/it]
 53%|█████▎    | 118846/225000 [244:33:43<141:55:00,  4.81s/it]
 53%|█████▎    | 118847/225000 [244:33:49<152:03:34,  5.16s/it]
 53%|█████▎    | 118848/225000 [244:34:00<205:50:08,  6.98s/it]
 53%|█████▎    | 118849/225000 [244:34:04<182:54:06,  6.20s/it]
 53%|█████▎    | 118850/225000 [244:34:09<170:35:31,  5.79s/it]
                                                               
{'loss': '0.8761', 'grad_norm': '1.101', 'learning_rate': '2.453e-05', 'epoch': '0.5282', 'num_input_tokens_seen': 407891872, 'train_runtime': '8.804e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118850/225000 [244:34:09<170:35:31,  5.79s/it]
 53%|█████▎    | 118851/225000 [244:34:14<162:57:19,  5.53s/it]
 53%|█████▎    | 118852/225000 [244:34:19<159:40:24,  5.42s/it]
 53%|█████▎    | 118853/225000 [244:34:24<155:36:58,  5.28s/it]
 53%|█████▎    | 118854/225000 [244:34:29<149:47:04,  5.08s/it]
 53%|█████▎    | 118855/225000 [244:34:33<146:41:03,  4.97s/it]
 53%|█████▎    | 118856/225000 [244:34:39<150:18:01,  5.10s/it]
 53%|█████▎    | 118857/225000 [244:34:53<226:27:38,  7.68s/it]
 53%|█████▎    | 118858/225000 [244:34:56<189:57:39,  6.44s/it]
 53%|█████▎    | 118859/225000 [244:34:59<163:05:57,  5.53s/it]
 53%|█████▎    | 118860/225000 [244:35:10<210:19:49,  7.13s/it]
 53%|█████▎    | 118861/225000 [244:35:14<179:22:25,  6.08s/it]
 53%|█████▎    | 118862/225000 [244:35:17<156:11:15,  5.30s/it]
 53%|█████▎    | 118863/225000 [244:35:27<194:47:35,  6.61s/it]
 53%|█████▎    | 118864/225000 [244:35:30<162:08:57,  5.50s/it]
 53%|█████▎    | 118865/225000 [244:35:33<142:49:52,  4.84s/it]
 53%|█████▎    | 118866/225000 [244:35:38<143:40:06,  4.87s/it]
 53%|█████▎    | 118867/225000 [244:35:43<142:53:02,  4.85s/it]
 53%|█████▎    | 118868/225000 [244:35:52<178:12:15,  6.04s/it]
 53%|█████▎    | 118869/225000 [244:36:00<193:57:28,  6.58s/it]
 53%|█████▎    | 118870/225000 [244:36:05<184:26:34,  6.26s/it]
 53%|█████▎    | 118871/225000 [244:36:09<165:23:46,  5.61s/it]
 53%|█████▎    | 118872/225000 [244:36:13<149:17:45,  5.06s/it]
 53%|█████▎    | 118873/225000 [244:36:20<165:46:56,  5.62s/it]
 53%|█████▎    | 118874/225000 [244:36:24<147:46:55,  5.01s/it]
 53%|█████▎    | 118875/225000 [244:36:28<142:40:28,  4.84s/it]
 53%|█████▎    | 118876/225000 [244:36:33<140:51:03,  4.78s/it]
 53%|█████▎    | 118877/225000 [244:36:37<138:31:41,  4.70s/it]
 53%|█████▎    | 118878/225000 [244:36:49<197:18:50,  6.69s/it]
 53%|█████▎    | 118879/225000 [244:36:55<193:05:04,  6.55s/it]
 53%|█████▎    | 118880/225000 [244:36:58<164:29:13,  5.58s/it]
 53%|█████▎    | 118881/225000 [244:37:02<151:46:04,  5.15s/it]
 53%|█████▎    | 118882/225000 [244:37:10<179:11:42,  6.08s/it]
 53%|█████▎    | 118883/225000 [244:37:16<170:10:22,  5.77s/it]
 53%|█████▎    | 118884/225000 [244:37:19<148:04:46,  5.02s/it]
 53%|█████▎    | 118885/225000 [244:37:23<141:57:22,  4.82s/it]
 53%|█████▎    | 118886/225000 [244:37:33<186:50:38,  6.34s/it]
 53%|█████▎    | 118887/225000 [244:37:36<157:28:06,  5.34s/it]
 53%|█████▎    | 118888/225000 [244:37:44<176:43:11,  6.00s/it]
 53%|█████▎    | 118889/225000 [244:37:47<153:14:28,  5.20s/it]
 53%|█████▎    | 118890/225000 [244:37:52<155:50:55,  5.29s/it]
 53%|█████▎    | 118891/225000 [244:38:00<172:09:25,  5.84s/it]
 53%|█████▎    | 118892/225000 [244:38:03<154:13:26,  5.23s/it]
 53%|█████▎    | 118893/225000 [244:38:07<144:23:16,  4.90s/it]
 53%|█████▎    | 118894/225000 [244:38:13<147:58:25,  5.02s/it]
 53%|█████▎    | 118895/225000 [244:38:23<195:07:38,  6.62s/it]
 53%|█████▎    | 118896/225000 [244:38:27<169:37:16,  5.76s/it]
 53%|█████▎    | 118897/225000 [244:38:32<167:52:08,  5.70s/it]
 53%|█████▎    | 118898/225000 [244:38:39<175:08:21,  5.94s/it]
 53%|█████▎    | 118899/225000 [244:38:46<180:33:46,  6.13s/it]
 53%|█████▎    | 118900/225000 [244:38:50<168:38:53,  5.72s/it]
                                                               
{'loss': '0.8577', 'grad_norm': '1.017', 'learning_rate': '2.451e-05', 'epoch': '0.5284', 'num_input_tokens_seen': 408015488, 'train_runtime': '8.807e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118900/225000 [244:38:50<168:38:53,  5.72s/it]
 53%|█████▎    | 118901/225000 [244:38:55<157:39:35,  5.35s/it]
 53%|█████▎    | 118902/225000 [244:38:59<147:40:31,  5.01s/it]
 53%|█████▎    | 118903/225000 [244:39:04<146:05:53,  4.96s/it]
 53%|█████▎    | 118904/225000 [244:39:08<138:53:32,  4.71s/it]
 53%|█████▎    | 118905/225000 [244:39:12<137:07:42,  4.65s/it]
 53%|█████▎    | 118906/225000 [244:39:17<135:36:07,  4.60s/it]
 53%|█████▎    | 118907/225000 [244:39:27<186:28:27,  6.33s/it]
 53%|█████▎    | 118908/225000 [244:39:35<195:01:59,  6.62s/it]
 53%|█████▎    | 118909/225000 [244:39:38<164:30:22,  5.58s/it]
 53%|█████▎    | 118910/225000 [244:39:46<189:29:59,  6.43s/it]
 53%|█████▎    | 118911/225000 [244:39:50<167:06:22,  5.67s/it]
 53%|█████▎    | 118912/225000 [244:39:54<152:50:40,  5.19s/it]
 53%|█████▎    | 118913/225000 [244:39:58<140:46:34,  4.78s/it]
 53%|█████▎    | 118914/225000 [244:40:10<203:03:15,  6.89s/it]
 53%|█████▎    | 118915/225000 [244:40:13<173:10:29,  5.88s/it]
 53%|█████▎    | 118916/225000 [244:40:17<157:07:22,  5.33s/it]
 53%|█████▎    | 118917/225000 [244:40:21<144:36:23,  4.91s/it]
 53%|█████▎    | 118918/225000 [244:40:31<184:48:26,  6.27s/it]
 53%|█████▎    | 118919/225000 [244:40:36<180:17:09,  6.12s/it]
 53%|█████▎    | 118920/225000 [244:40:40<158:59:40,  5.40s/it]
 53%|█████▎    | 118921/225000 [244:40:49<187:24:31,  6.36s/it]
 53%|█████▎    | 118922/225000 [244:40:52<162:46:08,  5.52s/it]
 53%|█████▎    | 118923/225000 [244:40:58<167:22:42,  5.68s/it]
 53%|█████▎    | 118924/225000 [244:41:05<174:32:31,  5.92s/it]
 53%|█████▎    | 118925/225000 [244:41:08<147:18:14,  5.00s/it]
 53%|█████▎    | 118926/225000 [244:41:16<173:08:41,  5.88s/it]
 53%|█████▎    | 118927/225000 [244:41:19<150:57:43,  5.12s/it]
 53%|█████▎    | 118928/225000 [244:41:27<178:49:54,  6.07s/it]
 53%|█████▎    | 118929/225000 [244:41:32<165:36:24,  5.62s/it]
 53%|█████▎    | 118930/225000 [244:41:36<152:24:13,  5.17s/it]
 53%|█████▎    | 118931/225000 [244:41:42<155:49:42,  5.29s/it]
 53%|█████▎    | 118932/225000 [244:41:46<148:53:55,  5.05s/it]
 53%|█████▎    | 118933/225000 [244:41:51<146:31:41,  4.97s/it]
 53%|█████▎    | 118934/225000 [244:41:56<147:16:39,  5.00s/it]
 53%|█████▎    | 118935/225000 [244:42:01<151:20:22,  5.14s/it]
 53%|█████▎    | 118936/225000 [244:42:07<152:21:01,  5.17s/it]
 53%|█████▎    | 118937/225000 [244:42:21<233:53:10,  7.94s/it]
 53%|█████▎    | 118938/225000 [244:42:32<264:10:27,  8.97s/it]
 53%|█████▎    | 118939/225000 [244:42:36<215:43:55,  7.32s/it]
 53%|█████▎    | 118940/225000 [244:42:39<181:12:51,  6.15s/it]
 53%|█████▎    | 118941/225000 [244:42:46<182:03:52,  6.18s/it]
 53%|█████▎    | 118942/225000 [244:42:52<180:28:57,  6.13s/it]
 53%|█████▎    | 118943/225000 [244:42:55<155:48:12,  5.29s/it]
 53%|█████▎    | 118944/225000 [244:42:59<146:03:11,  4.96s/it]
 53%|█████▎    | 118945/225000 [244:43:03<139:07:30,  4.72s/it]
 53%|█████▎    | 118946/225000 [244:43:08<136:33:39,  4.64s/it]
 53%|█████▎    | 118947/225000 [244:43:12<135:09:36,  4.59s/it]
 53%|█████▎    | 118948/225000 [244:43:27<229:24:03,  7.79s/it]
 53%|█████▎    | 118949/225000 [244:43:31<194:13:19,  6.59s/it]
 53%|█████▎    | 118950/225000 [244:43:35<173:06:34,  5.88s/it]
                                                               
{'loss': '0.8687', 'grad_norm': '1.048', 'learning_rate': '2.449e-05', 'epoch': '0.5287', 'num_input_tokens_seen': 408141904, 'train_runtime': '8.81e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 118950/225000 [244:43:35<173:06:34,  5.88s/it]
 53%|█████▎    | 118951/225000 [244:43:46<217:04:33,  7.37s/it]
 53%|█████▎    | 118952/225000 [244:43:50<181:50:17,  6.17s/it]
 53%|█████▎    | 118953/225000 [244:43:58<202:22:47,  6.87s/it]
 53%|█████▎    | 118954/225000 [244:44:01<170:29:05,  5.79s/it]
 53%|█████▎    | 118955/225000 [244:44:06<159:28:44,  5.41s/it]
 53%|█████▎    | 118956/225000 [244:44:11<154:02:13,  5.23s/it]
 53%|█████▎    | 118957/225000 [244:44:15<145:13:06,  4.93s/it]
 53%|█████▎    | 118958/225000 [244:44:19<141:15:36,  4.80s/it]
 53%|█████▎    | 118959/225000 [244:44:31<203:02:01,  6.89s/it]
 53%|█████▎    | 118960/225000 [244:44:35<176:40:36,  6.00s/it]
 53%|█████▎    | 118961/225000 [244:44:39<155:37:27,  5.28s/it]
 53%|█████▎    | 118962/225000 [244:44:52<226:44:07,  7.70s/it]
 53%|█████▎    | 118963/225000 [244:44:55<188:30:15,  6.40s/it]
 53%|█████▎    | 118964/225000 [244:45:00<167:59:50,  5.70s/it]
 53%|█████▎    | 118965/225000 [244:45:03<150:40:22,  5.12s/it]
 53%|█████▎    | 118966/225000 [244:45:14<203:20:42,  6.90s/it]
 53%|█████▎    | 118967/225000 [244:45:18<173:37:23,  5.89s/it]
 53%|█████▎    | 118968/225000 [244:45:21<152:46:39,  5.19s/it]
 53%|█████▎    | 118969/225000 [244:45:30<180:23:00,  6.12s/it]
 53%|█████▎    | 118970/225000 [244:45:35<173:43:10,  5.90s/it]
 53%|█████▎    | 118971/225000 [244:45:42<182:29:11,  6.20s/it]
 53%|█████▎    | 118972/225000 [244:45:46<163:05:50,  5.54s/it]
 53%|█████▎    | 118973/225000 [244:45:50<150:52:33,  5.12s/it]
 53%|█████▎    | 118974/225000 [244:46:01<201:26:28,  6.84s/it]
 53%|█████▎    | 118975/225000 [244:46:05<173:37:34,  5.90s/it]
 53%|█████▎    | 118976/225000 [244:46:08<153:46:25,  5.22s/it]
 53%|█████▎    | 118977/225000 [244:46:15<162:44:55,  5.53s/it]
 53%|█████▎    | 118978/225000 [244:46:22<177:27:04,  6.03s/it]
 53%|█████▎    | 118979/225000 [244:46:25<155:33:07,  5.28s/it]
 53%|█████▎    | 118980/225000 [244:46:34<189:39:10,  6.44s/it]
 53%|█████▎    | 118981/225000 [244:46:38<162:11:00,  5.51s/it]
 53%|█████▎    | 118982/225000 [244:46:47<197:45:27,  6.72s/it]
 53%|█████▎    | 118983/225000 [244:46:52<175:15:53,  5.95s/it]
 53%|█████▎    | 118984/225000 [244:46:55<155:49:59,  5.29s/it]
 53%|█████▎    | 118985/225000 [244:47:00<148:09:27,  5.03s/it]
 53%|█████▎    | 118986/225000 [244:47:05<152:42:50,  5.19s/it]
 53%|█████▎    | 118987/225000 [244:47:11<153:24:06,  5.21s/it]
 53%|█████▎    | 118988/225000 [244:47:16<155:57:01,  5.30s/it]
 53%|█████▎    | 118989/225000 [244:47:21<152:54:09,  5.19s/it]
 53%|█████▎    | 118990/225000 [244:47:26<154:00:17,  5.23s/it]
 53%|█████▎    | 118991/225000 [244:47:32<154:18:32,  5.24s/it]
 53%|█████▎    | 118992/225000 [244:47:43<211:48:33,  7.19s/it]
 53%|█████▎    | 118993/225000 [244:47:49<200:32:24,  6.81s/it]
 53%|█████▎    | 118994/225000 [244:47:53<170:52:52,  5.80s/it]
 53%|█████▎    | 118995/225000 [244:47:58<166:24:34,  5.65s/it]
 53%|█████▎    | 118996/225000 [244:48:07<196:55:16,  6.69s/it]
 53%|█████▎    | 118997/225000 [244:48:11<172:18:17,  5.85s/it]
 53%|█████▎    | 118998/225000 [244:48:16<163:12:14,  5.54s/it]
 53%|█████▎    | 118999/225000 [244:48:21<163:58:00,  5.57s/it]
 53%|█████▎    | 119000/225000 [244:48:34<223:05:20,  7.58s/it]
                                                               
{'loss': '0.8792', 'grad_norm': '1.053', 'learning_rate': '2.447e-05', 'epoch': '0.5289', 'num_input_tokens_seen': 408272728, 'train_runtime': '8.813e+05', 'train_tokens_per_second': '463.3'}

 53%|█████▎    | 119000/225000 [244:48:34<223:05:20,  7.58s/it]
 53%|█████▎    | 119001/225000 [244:48:41<219:50:49,  7.47s/it]
 53%|█████▎    | 119002/225000 [244:48:46<195:43:05,  6.65s/it]
 53%|█████▎    | 119003/225000 [244:48:49<170:52:00,  5.80s/it]
 53%|█████▎    | 119004/225000 [244:48:53<149:35:46,  5.08s/it]
 53%|█████▎    | 119005/225000 [244:49:01<175:05:53,  5.95s/it]
 53%|█████▎    | 119006/225000 [244:49:06<168:13:25,  5.71s/it]
 53%|█████▎    | 119007/225000 [244:49:10<151:39:46,  5.15s/it]
 53%|█████▎    | 119008/225000 [244:49:19<183:01:03,  6.22s/it]
 53%|█████▎    | 119009/225000 [244:49:24<177:16:23,  6.02s/it]
 53%|█████▎    | 119010/225000 [244:49:28<159:13:30,  5.41s/it]
 53%|█████▎    | 119011/225000 [244:49:37<193:00:54,  6.56s/it]
 53%|█████▎    | 119012/225000 [244:49:41<166:24:04,  5.65s/it]
 53%|█████▎    | 119013/225000 [244:49:50<199:15:06,  6.77s/it]
 53%|█████▎    | 119014/225000 [244:49:56<192:23:39,  6.54s/it]
 53%|█████▎    | 119015/225000 [244:50:00<172:24:40,  5.86s/it]
 53%|█████▎    | 119016/225000 [244:50:04<154:17:11,  5.24s/it]
 53%|█████▎    | 119017/225000 [244:50:12<177:19:39,  6.02s/it]
 53%|█████▎    | 119018/225000 [244:50:19<188:50:29,  6.41s/it]
 53%|█████▎    | 119019/225000 [244:50:25<176:49:31,  6.01s/it]
 53%|█████▎    | 119020/225000 [244:50:30<175:30:00,  5.96s/it]
 53%|█████▎    | 119021/225000 [244:50:34<152:04:18,  5.17s/it]
 53%|█████▎    | 119022/225000 [244:50:43<192:01:11,  6.52s/it]
 53%|█████▎    | 119023/225000 [244:50:47<165:47:15,  5.63s/it]
 53%|█████▎    | 119024/225000 [244:50:50<145:53:40,  4.96s/it]
 53%|█████▎    | 119025/225000 [244:50:55<139:55:07,  4.75s/it]
 53%|█████▎    | 119026/225000 [244:50:59<135:37:14,  4.61s/it]
 53%|█████▎    | 119027/225000 [244:51:03<132:41:24,  4.51s/it]
 53%|█████▎    | 119028/225000 [244:51:12<174:59:19,  5.94s/it]
 53%|█████▎    | 119029/225000 [244:51:20<191:07:11,  6.49s/it]
 53%|█████▎    | 119030/225000 [244:51:25<177:58:31,  6.05s/it]
 53%|█████▎    | 119031/225000 [244:51:31<180:06:08,  6.12s/it]
 53%|█████▎    | 119032/225000 [244:51:44<234:15:36,  7.96s/it]
 53%|█████▎    | 119033/225000 [244:51:47<196:03:30,  6.66s/it]
 53%|█████▎    | 119034/225000 [244:51:58<231:10:33,  7.85s/it]
 53%|█████▎    | 119035/225000 [244:52:03<204:33:05,  6.95s/it]
 53%|█████▎    | 119036/225000 [244:52:07<179:49:42,  6.11s/it]
 53%|█████▎    | 119037/225000 [244:52:11<164:28:02,  5.59s/it]
 53%|█████▎    | 119038/225000 [244:52:15<150:00:29,  5.10s/it]
 53%|█████▎    | 119039/225000 [244:52:19<140:19:39,  4.77s/it]
 53%|█████▎    | 119040/225000 [244:52:30<191:00:45,  6.49s/it]
 53%|█████▎    | 119041/225000 [244:52:36<189:17:42,  6.43s/it]
 53%|█████▎    | 119042/225000 [244:52:41<174:55:05,  5.94s/it]
 53%|█████▎    | 119043/225000 [244:52:46<165:30:50,  5.62s/it]
 53%|█████▎    | 119044/225000 [244:52:51<159:07:18,  5.41s/it]
 53%|█████▎    | 119045/225000 [244:52:55<153:31:46,  5.22s/it]
 53%|█████▎    | 119046/225000 [244:53:01<152:42:25,  5.19s/it]
 53%|█████▎    | 119047/225000 [244:53:06<151:11:10,  5.14s/it]
 53%|█████▎    | 119048/225000 [244:53:11<149:26:29,  5.08s/it]
 53%|█████▎    | 119049/225000 [244:53:16<151:13:44,  5.14s/it]
 53%|█████▎    | 119050/225000 [244:53:21<154:11:10,  5.24s/it]
                                                               
{'loss': '0.8666', 'grad_norm': '1.061', 'learning_rate': '2.445e-05', 'epoch': '0.5291', 'num_input_tokens_seen': 408400400, 'train_runtime': '8.816e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119050/225000 [244:53:21<154:11:10,  5.24s/it]
 53%|█████▎    | 119051/225000 [244:53:26<148:03:36,  5.03s/it]
 53%|█████▎    | 119052/225000 [244:53:30<143:44:42,  4.88s/it]
 53%|█████▎    | 119053/225000 [244:53:37<162:47:27,  5.53s/it]
 53%|█████▎    | 119054/225000 [244:53:50<227:45:07,  7.74s/it]
 53%|█████▎    | 119055/225000 [244:53:55<198:46:23,  6.75s/it]
 53%|█████▎    | 119056/225000 [244:54:02<202:36:56,  6.88s/it]
 53%|█████▎    | 119057/225000 [244:54:06<180:05:17,  6.12s/it]
 53%|█████▎    | 119058/225000 [244:54:11<170:05:42,  5.78s/it]
 53%|█████▎    | 119059/225000 [244:54:17<164:59:45,  5.61s/it]
 53%|█████▎    | 119060/225000 [244:54:22<164:43:10,  5.60s/it]
 53%|█████▎    | 119061/225000 [244:54:29<180:00:24,  6.12s/it]
 53%|█████▎    | 119062/225000 [244:54:37<188:34:50,  6.41s/it]
 53%|█████▎    | 119063/225000 [244:54:41<168:55:47,  5.74s/it]
 53%|█████▎    | 119064/225000 [244:54:46<161:04:33,  5.47s/it]
 53%|█████▎    | 119065/225000 [244:54:52<171:33:22,  5.83s/it]
 53%|█████▎    | 119066/225000 [244:54:56<156:38:04,  5.32s/it]
 53%|█████▎    | 119067/225000 [244:55:01<148:10:59,  5.04s/it]
 53%|█████▎    | 119068/225000 [244:55:05<143:19:27,  4.87s/it]
 53%|█████▎    | 119069/225000 [244:55:17<202:26:54,  6.88s/it]
 53%|█████▎    | 119070/225000 [244:55:23<197:09:26,  6.70s/it]
 53%|█████▎    | 119071/225000 [244:55:27<172:28:09,  5.86s/it]
 53%|█████▎    | 119072/225000 [244:55:30<151:22:40,  5.14s/it]
 53%|█████▎    | 119073/225000 [244:55:38<171:40:42,  5.83s/it]
 53%|█████▎    | 119074/225000 [244:55:44<169:59:30,  5.78s/it]
 53%|█████▎    | 119075/225000 [244:55:47<150:07:41,  5.10s/it]
 53%|█████▎    | 119076/225000 [244:55:52<145:08:51,  4.93s/it]
 53%|█████▎    | 119077/225000 [244:56:02<190:28:09,  6.47s/it]
 53%|█████▎    | 119078/225000 [244:56:05<159:49:34,  5.43s/it]
 53%|█████▎    | 119079/225000 [244:56:09<148:44:52,  5.06s/it]
 53%|█████▎    | 119080/225000 [244:56:14<148:20:20,  5.04s/it]
 53%|█████▎    | 119081/225000 [244:56:20<157:25:37,  5.35s/it]
 53%|█████▎    | 119082/225000 [244:56:28<177:46:47,  6.04s/it]
 53%|█████▎    | 119083/225000 [244:56:31<152:24:05,  5.18s/it]
 53%|█████▎    | 119084/225000 [244:56:40<188:10:00,  6.40s/it]
 53%|█████▎    | 119085/225000 [244:56:44<163:07:09,  5.54s/it]
 53%|█████▎    | 119086/225000 [244:56:47<144:31:47,  4.91s/it]
 53%|█████▎    | 119087/225000 [244:57:00<216:32:42,  7.36s/it]
 53%|█████▎    | 119088/225000 [244:57:04<185:34:05,  6.31s/it]
 53%|█████▎    | 119089/225000 [244:57:13<206:52:43,  7.03s/it]
 53%|█████▎    | 119090/225000 [244:57:16<175:49:38,  5.98s/it]
 53%|█████▎    | 119091/225000 [244:57:20<161:27:55,  5.49s/it]
 53%|█████▎    | 119092/225000 [244:57:29<184:07:27,  6.26s/it]
 53%|█████▎    | 119093/225000 [244:57:32<159:21:32,  5.42s/it]
 53%|█████▎    | 119094/225000 [244:57:41<188:34:42,  6.41s/it]
 53%|█████▎    | 119095/225000 [244:57:44<164:50:07,  5.60s/it]
 53%|█████▎    | 119096/225000 [244:57:53<195:23:01,  6.64s/it]
 53%|█████▎    | 119097/225000 [244:57:58<178:08:28,  6.06s/it]
 53%|█████▎    | 119098/225000 [244:58:02<159:01:13,  5.41s/it]
 53%|█████▎    | 119099/225000 [244:58:07<153:28:06,  5.22s/it]
 53%|█████▎    | 119100/225000 [244:58:11<148:25:35,  5.05s/it]
                                                               
{'loss': '0.8753', 'grad_norm': '0.9837', 'learning_rate': '2.444e-05', 'epoch': '0.5293', 'num_input_tokens_seen': 408525344, 'train_runtime': '8.819e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119100/225000 [244:58:11<148:25:35,  5.05s/it]
 53%|█████▎    | 119101/225000 [244:58:15<139:06:20,  4.73s/it]
 53%|█████▎    | 119102/225000 [244:58:20<139:08:30,  4.73s/it]
 53%|█████▎    | 119103/225000 [244:58:27<153:49:29,  5.23s/it]
 53%|█████▎    | 119104/225000 [244:58:36<190:33:20,  6.48s/it]
 53%|█████▎    | 119105/225000 [244:58:42<187:44:42,  6.38s/it]
 53%|█████▎    | 119106/225000 [244:58:47<169:52:41,  5.78s/it]
 53%|█████▎    | 119107/225000 [244:58:51<159:16:56,  5.42s/it]
 53%|█████▎    | 119108/225000 [244:58:56<152:55:16,  5.20s/it]
 53%|█████▎    | 119109/225000 [244:59:01<151:00:19,  5.13s/it]
 53%|█████▎    | 119110/225000 [244:59:06<148:39:50,  5.05s/it]
 53%|█████▎    | 119111/225000 [244:59:10<144:20:58,  4.91s/it]
 53%|█████▎    | 119112/225000 [244:59:15<143:43:34,  4.89s/it]
 53%|█████▎    | 119113/225000 [244:59:21<157:16:57,  5.35s/it]
 53%|█████▎    | 119114/225000 [244:59:33<211:19:59,  7.19s/it]
 53%|█████▎    | 119115/225000 [244:59:37<180:10:43,  6.13s/it]
 53%|█████▎    | 119116/225000 [244:59:40<157:29:44,  5.35s/it]
 53%|█████▎    | 119117/225000 [244:59:51<210:21:34,  7.15s/it]
 53%|█████▎    | 119118/225000 [244:59:55<181:36:48,  6.17s/it]
 53%|█████▎    | 119119/225000 [244:59:59<157:38:54,  5.36s/it]
 53%|█████▎    | 119120/225000 [245:00:08<188:56:08,  6.42s/it]
 53%|█████▎    | 119121/225000 [245:00:11<162:06:51,  5.51s/it]
 53%|█████▎    | 119122/225000 [245:00:16<152:44:04,  5.19s/it]
 53%|█████▎    | 119123/225000 [245:00:20<143:56:42,  4.89s/it]
 53%|█████▎    | 119124/225000 [245:00:24<141:27:13,  4.81s/it]
 53%|█████▎    | 119125/225000 [245:00:30<150:54:00,  5.13s/it]
 53%|█████▎    | 119126/225000 [245:00:43<220:43:22,  7.51s/it]
 53%|█████▎    | 119127/225000 [245:00:49<200:20:30,  6.81s/it]
 53%|█████▎    | 119128/225000 [245:00:53<176:57:23,  6.02s/it]
 53%|█████▎    | 119129/225000 [245:00:59<181:52:07,  6.18s/it]
 53%|█████▎    | 119130/225000 [245:01:05<177:01:34,  6.02s/it]
 53%|█████▎    | 119131/225000 [245:01:08<154:58:38,  5.27s/it]
 53%|█████▎    | 119132/225000 [245:01:13<145:51:05,  4.96s/it]
 53%|█████▎    | 119133/225000 [245:01:17<136:27:38,  4.64s/it]
 53%|█████▎    | 119134/225000 [245:01:28<194:35:49,  6.62s/it]
 53%|█████▎    | 119135/225000 [245:01:31<169:04:10,  5.75s/it]
 53%|█████▎    | 119136/225000 [245:01:35<151:18:33,  5.15s/it]
 53%|█████▎    | 119137/225000 [245:01:40<144:38:13,  4.92s/it]
 53%|█████▎    | 119138/225000 [245:01:46<154:40:23,  5.26s/it]
 53%|█████▎    | 119139/225000 [245:01:57<207:36:50,  7.06s/it]
 53%|█████▎    | 119140/225000 [245:02:03<195:58:17,  6.66s/it]
 53%|█████▎    | 119141/225000 [245:02:06<166:52:35,  5.68s/it]
 53%|█████▎    | 119142/225000 [245:02:11<159:55:57,  5.44s/it]
 53%|█████▎    | 119143/225000 [245:02:21<204:02:05,  6.94s/it]
 53%|█████▎    | 119144/225000 [245:02:25<176:55:08,  6.02s/it]
 53%|█████▎    | 119145/225000 [245:02:30<162:07:12,  5.51s/it]
 53%|█████▎    | 119146/225000 [245:02:33<144:32:42,  4.92s/it]
 53%|█████▎    | 119147/225000 [245:02:44<196:02:24,  6.67s/it]
 53%|█████▎    | 119148/225000 [245:02:47<168:47:42,  5.74s/it]
 53%|█████▎    | 119149/225000 [245:02:53<168:22:50,  5.73s/it]
 53%|█████▎    | 119150/225000 [245:03:00<177:02:09,  6.02s/it]
                                                               
{'loss': '0.8658', 'grad_norm': '1.076', 'learning_rate': '2.442e-05', 'epoch': '0.5296', 'num_input_tokens_seen': 408650040, 'train_runtime': '8.822e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119150/225000 [245:03:00<177:02:09,  6.02s/it]
 53%|█████▎    | 119151/225000 [245:03:04<156:39:34,  5.33s/it]
 53%|█████▎    | 119152/225000 [245:03:07<142:30:56,  4.85s/it]
 53%|█████▎    | 119153/225000 [245:03:17<187:59:38,  6.39s/it]
 53%|█████▎    | 119154/225000 [245:03:21<162:31:42,  5.53s/it]
 53%|█████▎    | 119155/225000 [245:03:25<154:03:10,  5.24s/it]
 53%|█████▎    | 119156/225000 [245:03:30<147:46:39,  5.03s/it]
 53%|█████▎    | 119157/225000 [245:03:34<141:33:19,  4.81s/it]
 53%|█████▎    | 119158/225000 [245:03:38<134:00:35,  4.56s/it]
 53%|█████▎    | 119159/225000 [245:03:42<132:09:42,  4.50s/it]
 53%|█████▎    | 119160/225000 [245:03:49<147:56:10,  5.03s/it]
 53%|█████▎    | 119161/225000 [245:04:01<208:37:03,  7.10s/it]
 53%|█████▎    | 119162/225000 [245:04:04<174:15:20,  5.93s/it]
 53%|█████▎    | 119163/225000 [245:04:11<183:22:14,  6.24s/it]
 53%|█████▎    | 119164/225000 [245:04:15<165:00:28,  5.61s/it]
 53%|█████▎    | 119165/225000 [245:04:20<157:02:34,  5.34s/it]
 53%|█████▎    | 119166/225000 [245:04:24<151:38:11,  5.16s/it]
 53%|█████▎    | 119167/225000 [245:04:30<150:49:27,  5.13s/it]
 53%|█████▎    | 119168/225000 [245:04:34<147:43:59,  5.03s/it]
 53%|█████▎    | 119169/225000 [245:04:39<149:04:53,  5.07s/it]
 53%|█████▎    | 119170/225000 [245:04:44<142:39:12,  4.85s/it]
 53%|█████▎    | 119171/225000 [245:04:53<183:58:17,  6.26s/it]
 53%|█████▎    | 119172/225000 [245:05:06<244:00:35,  8.30s/it]
 53%|█████▎    | 119173/225000 [245:05:11<208:07:37,  7.08s/it]
 53%|█████▎    | 119174/225000 [245:05:15<180:21:32,  6.14s/it]
 53%|█████▎    | 119175/225000 [245:05:24<211:46:54,  7.20s/it]
 53%|█████▎    | 119176/225000 [245:05:27<175:26:14,  5.97s/it]
 53%|█████▎    | 119177/225000 [245:05:31<158:39:33,  5.40s/it]
 53%|█████▎    | 119178/225000 [245:05:35<146:55:20,  5.00s/it]
 53%|█████▎    | 119179/225000 [245:05:47<206:11:07,  7.01s/it]
 53%|█████▎    | 119180/225000 [245:05:50<171:54:21,  5.85s/it]
 53%|█████▎    | 119181/225000 [245:05:56<168:34:55,  5.74s/it]
 53%|█████▎    | 119182/225000 [245:06:02<174:50:33,  5.95s/it]
 53%|█████▎    | 119183/225000 [245:06:10<186:26:19,  6.34s/it]
 53%|█████▎    | 119184/225000 [245:06:15<178:34:23,  6.08s/it]
 53%|█████▎    | 119185/225000 [245:06:21<175:54:10,  5.98s/it]
 53%|█████▎    | 119186/225000 [245:06:26<169:46:55,  5.78s/it]
 53%|█████▎    | 119187/225000 [245:06:32<173:18:55,  5.90s/it]
 53%|█████▎    | 119188/225000 [245:06:36<154:01:55,  5.24s/it]
 53%|█████▎    | 119189/225000 [245:06:40<146:52:45,  5.00s/it]
 53%|█████▎    | 119190/225000 [245:06:50<192:00:03,  6.53s/it]
 53%|█████▎    | 119191/225000 [245:06:54<166:52:24,  5.68s/it]
 53%|█████▎    | 119192/225000 [245:06:58<151:40:38,  5.16s/it]
 53%|█████▎    | 119193/225000 [245:07:02<143:07:32,  4.87s/it]
 53%|█████▎    | 119194/225000 [245:07:07<139:54:58,  4.76s/it]
 53%|█████▎    | 119195/225000 [245:07:18<193:19:57,  6.58s/it]
 53%|█████▎    | 119196/225000 [245:07:25<196:06:15,  6.67s/it]
 53%|█████▎    | 119197/225000 [245:07:28<171:53:33,  5.85s/it]
 53%|█████▎    | 119198/225000 [245:07:33<157:05:26,  5.35s/it]
 53%|█████▎    | 119199/225000 [245:07:37<145:32:12,  4.95s/it]
 53%|█████▎    | 119200/225000 [245:07:48<205:37:32,  7.00s/it]
                                                               
{'loss': '0.8522', 'grad_norm': '1.093', 'learning_rate': '2.44e-05', 'epoch': '0.5298', 'num_input_tokens_seen': 408778712, 'train_runtime': '8.825e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119200/225000 [245:07:48<205:37:32,  7.00s/it]
 53%|█████▎    | 119201/225000 [245:07:56<208:33:22,  7.10s/it]
 53%|█████▎    | 119202/225000 [245:08:00<179:38:09,  6.11s/it]
 53%|█████▎    | 119203/225000 [245:08:04<166:57:27,  5.68s/it]
 53%|█████▎    | 119204/225000 [245:08:09<156:45:42,  5.33s/it]
 53%|█████▎    | 119205/225000 [245:08:14<153:19:05,  5.22s/it]
 53%|█████▎    | 119206/225000 [245:08:20<159:28:52,  5.43s/it]
 53%|█████▎    | 119207/225000 [245:08:33<225:40:33,  7.68s/it]
 53%|█████▎    | 119208/225000 [245:08:38<202:26:46,  6.89s/it]
 53%|█████▎    | 119209/225000 [245:08:41<174:22:18,  5.93s/it]
 53%|█████▎    | 119210/225000 [245:08:46<161:21:22,  5.49s/it]
 53%|█████▎    | 119211/225000 [245:08:50<148:14:46,  5.04s/it]
 53%|█████▎    | 119212/225000 [245:08:54<144:22:16,  4.91s/it]
 53%|█████▎    | 119213/225000 [245:08:59<143:45:14,  4.89s/it]
 53%|█████▎    | 119214/225000 [245:09:09<183:36:58,  6.25s/it]
 53%|█████▎    | 119215/225000 [245:09:17<203:42:42,  6.93s/it]
 53%|█████▎    | 119216/225000 [245:09:20<170:26:48,  5.80s/it]
 53%|█████▎    | 119217/225000 [245:09:29<192:28:39,  6.55s/it]
 53%|█████▎    | 119218/225000 [245:09:36<198:25:27,  6.75s/it]
 53%|█████▎    | 119219/225000 [245:09:41<188:03:36,  6.40s/it]
 53%|█████▎    | 119220/225000 [245:09:46<174:59:50,  5.96s/it]
 53%|█████▎    | 119221/225000 [245:09:51<159:58:05,  5.44s/it]
 53%|█████▎    | 119222/225000 [245:09:56<160:36:32,  5.47s/it]
 53%|█████▎    | 119223/225000 [245:10:01<158:28:41,  5.39s/it]
 53%|█████▎    | 119224/225000 [245:10:06<150:17:40,  5.12s/it]
 53%|█████▎    | 119225/225000 [245:10:11<153:26:09,  5.22s/it]
 53%|█████▎    | 119226/225000 [245:10:17<153:55:36,  5.24s/it]
 53%|█████▎    | 119227/225000 [245:10:21<150:52:32,  5.14s/it]
 53%|█████▎    | 119228/225000 [245:10:26<148:55:59,  5.07s/it]
 53%|█████▎    | 119229/225000 [245:10:32<150:53:57,  5.14s/it]
 53%|█████▎    | 119230/225000 [245:10:36<144:52:06,  4.93s/it]
 53%|█████▎    | 119231/225000 [245:10:40<137:04:49,  4.67s/it]
 53%|█████▎    | 119232/225000 [245:10:45<140:46:35,  4.79s/it]
 53%|█████▎    | 119233/225000 [245:10:51<148:12:03,  5.04s/it]
 53%|█████▎    | 119234/225000 [245:10:56<151:02:03,  5.14s/it]
 53%|█████▎    | 119235/225000 [245:11:08<207:16:45,  7.06s/it]
 53%|█████▎    | 119236/225000 [245:11:14<202:13:04,  6.88s/it]
 53%|█████▎    | 119237/225000 [245:11:18<170:31:06,  5.80s/it]
 53%|█████▎    | 119238/225000 [245:11:26<192:04:12,  6.54s/it]
 53%|█████▎    | 119239/225000 [245:11:29<165:50:49,  5.65s/it]
 53%|█████▎    | 119240/225000 [245:11:37<186:55:43,  6.36s/it]
 53%|█████▎    | 119241/225000 [245:11:41<164:31:39,  5.60s/it]
 53%|█████▎    | 119242/225000 [245:11:49<185:43:53,  6.32s/it]
 53%|█████▎    | 119243/225000 [245:11:52<158:30:00,  5.40s/it]
 53%|█████▎    | 119244/225000 [245:11:57<148:58:22,  5.07s/it]
 53%|█████▎    | 119245/225000 [245:12:05<176:57:00,  6.02s/it]
 53%|█████▎    | 119246/225000 [245:12:09<161:31:03,  5.50s/it]
 53%|█████▎    | 119247/225000 [245:12:14<151:56:38,  5.17s/it]
 53%|█████▎    | 119248/225000 [245:12:17<140:02:22,  4.77s/it]
 53%|█████▎    | 119249/225000 [245:12:22<134:55:49,  4.59s/it]
 53%|█████▎    | 119250/225000 [245:12:34<205:38:18,  7.00s/it]
                                                               
{'loss': '0.866', 'grad_norm': '0.9789', 'learning_rate': '2.438e-05', 'epoch': '0.53', 'num_input_tokens_seen': 408902840, 'train_runtime': '8.828e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119250/225000 [245:12:34<205:38:18,  7.00s/it]
 53%|█████▎    | 119251/225000 [245:12:37<170:29:43,  5.80s/it]
 53%|█████▎    | 119252/225000 [245:12:41<148:29:40,  5.06s/it]
 53%|█████▎    | 119253/225000 [245:12:48<166:56:07,  5.68s/it]
 53%|█████▎    | 119254/225000 [245:12:51<145:12:26,  4.94s/it]
 53%|█████▎    | 119255/225000 [245:12:57<153:30:12,  5.23s/it]
 53%|█████▎    | 119256/225000 [245:13:04<169:31:15,  5.77s/it]
 53%|█████▎    | 119257/225000 [245:13:07<148:57:59,  5.07s/it]
 53%|█████▎    | 119258/225000 [245:13:11<136:32:50,  4.65s/it]
 53%|█████▎    | 119259/225000 [245:13:16<139:38:37,  4.75s/it]
 53%|█████▎    | 119260/225000 [245:13:21<142:57:56,  4.87s/it]
 53%|█████▎    | 119261/225000 [245:13:26<143:23:43,  4.88s/it]
 53%|█████▎    | 119262/225000 [245:13:38<202:43:21,  6.90s/it]
 53%|█████▎    | 119263/225000 [245:13:44<197:40:13,  6.73s/it]
 53%|█████▎    | 119264/225000 [245:13:51<199:18:16,  6.79s/it]
 53%|█████▎    | 119265/225000 [245:13:54<167:11:57,  5.69s/it]
 53%|█████▎    | 119266/225000 [245:14:02<185:07:18,  6.30s/it]
 53%|█████▎    | 119267/225000 [245:14:06<167:08:27,  5.69s/it]
 53%|█████▎    | 119268/225000 [245:14:11<157:40:46,  5.37s/it]
 53%|█████▎    | 119269/225000 [245:14:15<151:41:42,  5.17s/it]
 53%|█████▎    | 119270/225000 [245:14:20<149:43:54,  5.10s/it]
 53%|█████▎    | 119271/225000 [245:14:25<148:19:37,  5.05s/it]
 53%|█████▎    | 119272/225000 [245:14:31<155:43:44,  5.30s/it]
 53%|█████▎    | 119273/225000 [245:14:37<157:08:15,  5.35s/it]
 53%|█████▎    | 119274/225000 [245:14:46<194:29:58,  6.62s/it]
 53%|█████▎    | 119275/225000 [245:14:54<209:06:28,  7.12s/it]
 53%|█████▎    | 119276/225000 [245:14:58<178:15:53,  6.07s/it]
 53%|█████▎    | 119277/225000 [245:15:08<210:51:44,  7.18s/it]
 53%|█████▎    | 119278/225000 [245:15:11<174:45:33,  5.95s/it]
 53%|█████▎    | 119279/225000 [245:15:17<175:00:34,  5.96s/it]
 53%|█████▎    | 119280/225000 [245:15:24<182:58:38,  6.23s/it]
 53%|█████▎    | 119281/225000 [245:15:28<168:59:13,  5.75s/it]
 53%|█████▎    | 119282/225000 [245:15:36<185:02:20,  6.30s/it]
 53%|█████▎    | 119283/225000 [245:15:40<166:52:43,  5.68s/it]
 53%|█████▎    | 119284/225000 [245:15:47<175:45:14,  5.99s/it]
 53%|█████▎    | 119285/225000 [245:15:52<164:51:18,  5.61s/it]
 53%|█████▎    | 119286/225000 [245:15:56<156:06:46,  5.32s/it]
 53%|█████▎    | 119287/225000 [245:16:01<148:52:46,  5.07s/it]
 53%|█████▎    | 119288/225000 [245:16:05<142:16:31,  4.85s/it]
 53%|█████▎    | 119289/225000 [245:16:15<185:25:31,  6.31s/it]
 53%|█████▎    | 119290/225000 [245:16:21<185:33:53,  6.32s/it]
 53%|█████▎    | 119291/225000 [245:16:25<162:51:58,  5.55s/it]
 53%|█████▎    | 119292/225000 [245:16:30<156:22:14,  5.33s/it]
 53%|█████▎    | 119293/225000 [245:16:34<146:08:39,  4.98s/it]
 53%|█████▎    | 119294/225000 [245:16:44<191:45:29,  6.53s/it]
 53%|█████▎    | 119295/225000 [245:16:50<189:31:31,  6.45s/it]
 53%|█████▎    | 119296/225000 [245:16:54<161:55:25,  5.51s/it]
 53%|█████▎    | 119297/225000 [245:16:57<144:57:08,  4.94s/it]
 53%|█████▎    | 119298/225000 [245:17:08<198:37:04,  6.76s/it]
 53%|█████▎    | 119299/225000 [245:17:12<169:04:49,  5.76s/it]
 53%|█████▎    | 119300/225000 [245:17:15<149:58:19,  5.11s/it]
                                                               
{'loss': '0.8641', 'grad_norm': '1.168', 'learning_rate': '2.436e-05', 'epoch': '0.5302', 'num_input_tokens_seen': 409028360, 'train_runtime': '8.83e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119300/225000 [245:17:15<149:58:19,  5.11s/it]
 53%|█████▎    | 119301/225000 [245:17:22<160:04:38,  5.45s/it]
 53%|█████▎    | 119302/225000 [245:17:29<178:13:59,  6.07s/it]
 53%|█████▎    | 119303/225000 [245:17:33<161:00:12,  5.48s/it]
 53%|█████▎    | 119304/225000 [245:17:38<152:52:18,  5.21s/it]
 53%|█████▎    | 119305/225000 [245:17:42<145:10:34,  4.94s/it]
 53%|█████▎    | 119306/225000 [245:17:46<136:39:48,  4.65s/it]
 53%|█████▎    | 119307/225000 [245:17:58<203:19:20,  6.93s/it]
 53%|█████▎    | 119308/225000 [245:18:01<170:45:33,  5.82s/it]
 53%|█████▎    | 119309/225000 [245:18:10<193:12:25,  6.58s/it]
 53%|█████▎    | 119310/225000 [245:18:15<177:22:12,  6.04s/it]
 53%|█████▎    | 119311/225000 [245:18:19<161:43:39,  5.51s/it]
 53%|█████▎    | 119312/225000 [245:18:23<149:49:08,  5.10s/it]
 53%|█████▎    | 119313/225000 [245:18:27<141:56:10,  4.83s/it]
 53%|█████▎    | 119314/225000 [245:18:31<130:36:24,  4.45s/it]
 53%|█████▎    | 119315/225000 [245:18:37<146:37:09,  4.99s/it]
 53%|█████▎    | 119316/225000 [245:18:46<183:55:48,  6.27s/it]
 53%|█████▎    | 119317/225000 [245:18:50<162:21:25,  5.53s/it]
 53%|█████▎    | 119318/225000 [245:18:55<159:36:42,  5.44s/it]
 53%|█████▎    | 119319/225000 [245:19:00<154:30:01,  5.26s/it]
 53%|█████▎    | 119320/225000 [245:19:05<149:28:35,  5.09s/it]
 53%|█████▎    | 119321/225000 [245:19:10<148:07:26,  5.05s/it]
 53%|█████▎    | 119322/225000 [245:19:15<149:36:16,  5.10s/it]
 53%|█████▎    | 119323/225000 [245:19:20<152:04:21,  5.18s/it]
 53%|█████▎    | 119324/225000 [245:19:26<153:18:48,  5.22s/it]
 53%|█████▎    | 119325/225000 [245:19:38<219:02:46,  7.46s/it]
 53%|█████▎    | 119326/225000 [245:19:45<213:19:01,  7.27s/it]
 53%|█████▎    | 119327/225000 [245:19:51<202:20:28,  6.89s/it]
 53%|█████▎    | 119328/225000 [245:19:57<188:38:26,  6.43s/it]
 53%|█████▎    | 119329/225000 [245:20:01<166:53:12,  5.69s/it]
 53%|█████▎    | 119330/225000 [245:20:09<187:33:58,  6.39s/it]
 53%|█████▎    | 119331/225000 [245:20:12<160:49:31,  5.48s/it]
 53%|█████▎    | 119332/225000 [245:20:17<160:36:13,  5.47s/it]
 53%|█████▎    | 119333/225000 [245:20:25<179:32:25,  6.12s/it]
 53%|█████▎    | 119334/225000 [245:20:28<154:07:20,  5.25s/it]
 53%|█████▎    | 119335/225000 [245:20:33<149:10:19,  5.08s/it]
 53%|█████▎    | 119336/225000 [245:20:37<142:25:06,  4.85s/it]
 53%|█████▎    | 119337/225000 [245:20:42<140:24:51,  4.78s/it]
 53%|█████▎    | 119338/225000 [245:20:46<137:55:00,  4.70s/it]
 53%|█████▎    | 119339/225000 [245:20:51<138:43:38,  4.73s/it]
 53%|█████▎    | 119340/225000 [245:20:56<137:26:18,  4.68s/it]
 53%|█████▎    | 119341/225000 [245:21:01<140:10:58,  4.78s/it]
 53%|█████▎    | 119342/225000 [245:21:06<140:11:42,  4.78s/it]
 53%|█████▎    | 119343/225000 [245:21:12<154:20:13,  5.26s/it]
 53%|█████▎    | 119344/225000 [245:21:24<215:49:02,  7.35s/it]
 53%|█████▎    | 119345/225000 [245:21:29<191:41:05,  6.53s/it]
 53%|█████▎    | 119346/225000 [245:21:34<183:42:40,  6.26s/it]
 53%|█████▎    | 119347/225000 [245:21:43<207:32:56,  7.07s/it]
 53%|█████▎    | 119348/225000 [245:21:50<203:22:28,  6.93s/it]
 53%|█████▎    | 119349/225000 [245:21:54<176:20:02,  6.01s/it]
 53%|█████▎    | 119350/225000 [245:21:57<155:27:06,  5.30s/it]
                                                               
{'loss': '0.8681', 'grad_norm': '1.334', 'learning_rate': '2.434e-05', 'epoch': '0.5304', 'num_input_tokens_seen': 409152032, 'train_runtime': '8.833e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119350/225000 [245:21:57<155:27:06,  5.30s/it]
 53%|█████▎    | 119351/225000 [245:22:02<149:33:46,  5.10s/it]
 53%|█████▎    | 119352/225000 [245:22:07<145:54:05,  4.97s/it]
 53%|█████▎    | 119353/225000 [245:22:16<179:53:32,  6.13s/it]
 53%|█████▎    | 119354/225000 [245:22:23<192:14:22,  6.55s/it]
 53%|█████▎    | 119355/225000 [245:22:27<172:29:27,  5.88s/it]
 53%|█████▎    | 119356/225000 [245:22:32<161:51:00,  5.52s/it]
 53%|█████▎    | 119357/225000 [245:22:36<151:55:29,  5.18s/it]
 53%|█████▎    | 119358/225000 [245:22:45<178:11:02,  6.07s/it]
 53%|█████▎    | 119359/225000 [245:22:57<231:42:08,  7.90s/it]
 53%|█████▎    | 119360/225000 [245:23:00<194:37:18,  6.63s/it]
 53%|█████▎    | 119361/225000 [245:23:04<166:29:36,  5.67s/it]
 53%|█████▎    | 119362/225000 [245:23:11<176:07:26,  6.00s/it]
 53%|█████▎    | 119363/225000 [245:23:18<187:13:05,  6.38s/it]
 53%|█████▎    | 119364/225000 [245:23:22<166:09:14,  5.66s/it]
 53%|█████▎    | 119365/225000 [245:23:30<186:27:20,  6.35s/it]
 53%|█████▎    | 119366/225000 [245:23:34<163:57:54,  5.59s/it]
 53%|█████▎    | 119367/225000 [245:23:38<155:58:00,  5.32s/it]
 53%|█████▎    | 119368/225000 [245:23:43<149:20:35,  5.09s/it]
 53%|█████▎    | 119369/225000 [245:23:47<138:06:13,  4.71s/it]
 53%|█████▎    | 119370/225000 [245:23:51<137:03:10,  4.67s/it]
 53%|█████▎    | 119371/225000 [245:23:59<166:12:01,  5.66s/it]
 53%|█████▎    | 119372/225000 [245:24:10<209:25:04,  7.14s/it]
 53%|█████▎    | 119373/225000 [245:24:14<180:29:36,  6.15s/it]
 53%|█████▎    | 119374/225000 [245:24:18<164:57:13,  5.62s/it]
 53%|█████▎    | 119375/225000 [245:24:22<152:31:14,  5.20s/it]
 53%|█████▎    | 119376/225000 [245:24:31<182:30:14,  6.22s/it]
 53%|█████▎    | 119377/225000 [245:24:39<194:45:43,  6.64s/it]
 53%|█████▎    | 119378/225000 [245:24:42<166:18:18,  5.67s/it]
 53%|█████▎    | 119379/225000 [245:24:50<188:37:57,  6.43s/it]
 53%|█████▎    | 119380/225000 [245:24:55<170:43:37,  5.82s/it]
 53%|█████▎    | 119381/225000 [245:24:59<160:39:49,  5.48s/it]
 53%|█████▎    | 119382/225000 [245:25:04<152:10:22,  5.19s/it]
 53%|█████▎    | 119383/225000 [245:25:08<147:08:40,  5.02s/it]
 53%|█████▎    | 119384/225000 [245:25:13<140:26:25,  4.79s/it]
 53%|█████▎    | 119385/225000 [245:25:17<136:19:53,  4.65s/it]
 53%|█████▎    | 119386/225000 [245:25:21<134:56:52,  4.60s/it]
 53%|█████▎    | 119387/225000 [245:25:35<210:09:06,  7.16s/it]
 53%|█████▎    | 119388/225000 [245:25:40<197:42:53,  6.74s/it]
 53%|█████▎    | 119389/225000 [245:25:44<173:37:02,  5.92s/it]
 53%|█████▎    | 119390/225000 [245:25:48<157:31:27,  5.37s/it]
 53%|█████▎    | 119391/225000 [245:25:52<145:15:32,  4.95s/it]
 53%|█████▎    | 119392/225000 [245:25:57<141:22:10,  4.82s/it]
 53%|█████▎    | 119393/225000 [245:26:09<204:20:56,  6.97s/it]
 53%|█████▎    | 119394/225000 [245:26:18<225:32:58,  7.69s/it]
 53%|█████▎    | 119395/225000 [245:26:26<225:18:14,  7.68s/it]
 53%|█████▎    | 119396/225000 [245:26:34<230:36:56,  7.86s/it]
 53%|█████▎    | 119397/225000 [245:26:38<197:26:21,  6.73s/it]
 53%|█████▎    | 119398/225000 [245:26:43<180:27:51,  6.15s/it]
 53%|█████▎    | 119399/225000 [245:26:48<165:06:31,  5.63s/it]
 53%|█████▎    | 119400/225000 [245:26:52<154:28:26,  5.27s/it]
                                                               
{'loss': '0.8604', 'grad_norm': '1.14', 'learning_rate': '2.433e-05', 'epoch': '0.5307', 'num_input_tokens_seen': 409278792, 'train_runtime': '8.836e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119400/225000 [245:26:52<154:28:26,  5.27s/it]
 53%|█████▎    | 119401/225000 [245:26:57<148:22:05,  5.06s/it]
 53%|█████▎    | 119402/225000 [245:27:01<141:34:37,  4.83s/it]
 53%|█████▎    | 119403/225000 [245:27:05<135:35:26,  4.62s/it]
 53%|█████▎    | 119404/225000 [245:27:13<166:06:16,  5.66s/it]
 53%|█████▎    | 119405/225000 [245:27:21<184:30:44,  6.29s/it]
 53%|█████▎    | 119406/225000 [245:27:27<187:40:40,  6.40s/it]
 53%|█████▎    | 119407/225000 [245:27:35<196:32:46,  6.70s/it]
 53%|█████▎    | 119408/225000 [245:27:39<172:00:10,  5.86s/it]
 53%|█████▎    | 119409/225000 [245:27:43<155:00:13,  5.28s/it]
 53%|█████▎    | 119410/225000 [245:27:48<152:56:47,  5.21s/it]
 53%|█████▎    | 119411/225000 [245:27:58<194:29:48,  6.63s/it]
 53%|█████▎    | 119412/225000 [245:28:01<162:57:50,  5.56s/it]
 53%|█████▎    | 119413/225000 [245:28:10<191:56:05,  6.54s/it]
 53%|█████▎    | 119414/225000 [245:28:13<161:19:41,  5.50s/it]
 53%|█████▎    | 119415/225000 [245:28:19<165:31:20,  5.64s/it]
 53%|█████▎    | 119416/225000 [245:28:24<167:31:40,  5.71s/it]
 53%|█████▎    | 119417/225000 [245:28:30<161:24:20,  5.50s/it]
 53%|█████▎    | 119418/225000 [245:28:34<151:20:12,  5.16s/it]
 53%|█████▎    | 119419/225000 [245:28:39<147:27:33,  5.03s/it]
 53%|█████▎    | 119420/225000 [245:28:43<144:42:57,  4.93s/it]
 53%|█████▎    | 119421/225000 [245:28:49<152:45:54,  5.21s/it]
 53%|█████▎    | 119422/225000 [245:29:00<206:19:39,  7.04s/it]
 53%|█████▎    | 119423/225000 [245:29:06<191:16:14,  6.52s/it]
 53%|█████▎    | 119424/225000 [245:29:10<168:45:12,  5.75s/it]
 53%|█████▎    | 119425/225000 [245:29:14<157:05:09,  5.36s/it]
 53%|█████▎    | 119426/225000 [245:29:18<145:46:50,  4.97s/it]
 53%|█████▎    | 119427/225000 [245:29:29<198:45:17,  6.78s/it]
 53%|█████▎    | 119428/225000 [245:29:33<169:00:23,  5.76s/it]
 53%|█████▎    | 119429/225000 [245:29:42<196:40:47,  6.71s/it]
 53%|█████▎    | 119430/225000 [245:29:45<169:45:03,  5.79s/it]
 53%|█████▎    | 119431/225000 [245:29:48<147:49:58,  5.04s/it]
 53%|█████▎    | 119432/225000 [245:29:52<136:31:30,  4.66s/it]
 53%|█████▎    | 119433/225000 [245:30:05<207:29:04,  7.08s/it]
 53%|█████▎    | 119434/225000 [245:30:10<191:29:48,  6.53s/it]
 53%|█████▎    | 119435/225000 [245:30:15<175:38:54,  5.99s/it]
 53%|█████▎    | 119436/225000 [245:30:20<164:09:13,  5.60s/it]
 53%|█████▎    | 119437/225000 [245:30:24<154:22:25,  5.26s/it]
 53%|█████▎    | 119438/225000 [245:30:29<151:15:39,  5.16s/it]
 53%|█████▎    | 119439/225000 [245:30:34<149:17:10,  5.09s/it]
 53%|█████▎    | 119440/225000 [245:30:39<150:15:57,  5.12s/it]
 53%|█████▎    | 119441/225000 [245:30:44<150:39:38,  5.14s/it]
 53%|█████▎    | 119442/225000 [245:30:59<235:46:15,  8.04s/it]
 53%|█████▎    | 119443/225000 [245:31:05<214:59:34,  7.33s/it]
 53%|█████▎    | 119444/225000 [245:31:09<183:56:51,  6.27s/it]
 53%|█████▎    | 119445/225000 [245:31:17<203:45:14,  6.95s/it]
 53%|█████▎    | 119446/225000 [245:31:20<171:06:11,  5.84s/it]
 53%|█████▎    | 119447/225000 [245:31:27<174:58:35,  5.97s/it]
 53%|█████▎    | 119448/225000 [245:31:34<189:48:05,  6.47s/it]
 53%|█████▎    | 119449/225000 [245:31:38<161:51:29,  5.52s/it]
 53%|█████▎    | 119450/225000 [245:31:43<164:40:32,  5.62s/it]
                                                               
{'loss': '0.8605', 'grad_norm': '1.113', 'learning_rate': '2.431e-05', 'epoch': '0.5309', 'num_input_tokens_seen': 409405160, 'train_runtime': '8.839e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119450/225000 [245:31:43<164:40:32,  5.62s/it]
 53%|█████▎    | 119451/225000 [245:31:50<170:10:38,  5.80s/it]
 53%|█████▎    | 119452/225000 [245:31:53<150:15:43,  5.13s/it]
 53%|█████▎    | 119453/225000 [245:31:59<153:18:29,  5.23s/it]
 53%|█████▎    | 119454/225000 [245:32:03<149:08:24,  5.09s/it]
 53%|█████▎    | 119455/225000 [245:32:09<150:53:33,  5.15s/it]
 53%|█████▎    | 119456/225000 [245:32:14<150:26:13,  5.13s/it]
 53%|█████▎    | 119457/225000 [245:32:22<179:48:37,  6.13s/it]
 53%|█████▎    | 119458/225000 [245:32:33<215:44:16,  7.36s/it]
 53%|█████▎    | 119459/225000 [245:32:37<188:29:26,  6.43s/it]
 53%|█████▎    | 119460/225000 [245:32:41<165:18:32,  5.64s/it]
 53%|█████▎    | 119461/225000 [245:32:45<154:25:33,  5.27s/it]
 53%|█████▎    | 119462/225000 [245:32:49<143:07:34,  4.88s/it]
 53%|█████▎    | 119463/225000 [245:33:01<206:00:49,  7.03s/it]
 53%|█████▎    | 119464/225000 [245:33:05<177:42:53,  6.06s/it]
 53%|█████▎    | 119465/225000 [245:33:12<190:24:51,  6.50s/it]
 53%|█████▎    | 119466/225000 [245:33:17<175:38:43,  5.99s/it]
 53%|█████▎    | 119467/225000 [245:33:22<161:50:22,  5.52s/it]
 53%|█████▎    | 119468/225000 [245:33:26<151:47:52,  5.18s/it]
 53%|█████▎    | 119469/225000 [245:33:31<147:59:26,  5.05s/it]
 53%|█████▎    | 119470/225000 [245:33:36<147:18:45,  5.03s/it]
 53%|█████▎    | 119471/225000 [245:33:40<140:13:07,  4.78s/it]
 53%|█████▎    | 119472/225000 [245:33:44<137:52:44,  4.70s/it]
 53%|█████▎    | 119473/225000 [245:33:53<173:49:16,  5.93s/it]
 53%|█████▎    | 119474/225000 [245:34:05<222:17:50,  7.58s/it]
 53%|█████▎    | 119475/225000 [245:34:09<193:01:34,  6.59s/it]
 53%|█████▎    | 119476/225000 [245:34:13<174:27:06,  5.95s/it]
 53%|█████▎    | 119477/225000 [245:34:18<158:43:34,  5.42s/it]
 53%|█████▎    | 119478/225000 [245:34:22<148:28:48,  5.07s/it]
 53%|█████▎    | 119479/225000 [245:34:27<150:09:17,  5.12s/it]
 53%|█████▎    | 119480/225000 [245:34:32<149:08:51,  5.09s/it]
 53%|█████▎    | 119481/225000 [245:34:38<153:22:54,  5.23s/it]
 53%|█████▎    | 119482/225000 [245:34:43<154:26:29,  5.27s/it]
 53%|█████▎    | 119483/225000 [245:34:48<150:56:58,  5.15s/it]
 53%|█████▎    | 119484/225000 [245:34:54<163:46:04,  5.59s/it]
 53%|█████▎    | 119485/225000 [245:35:07<222:20:35,  7.59s/it]
 53%|█████▎    | 119486/225000 [245:35:14<216:42:05,  7.39s/it]
 53%|█████▎    | 119487/225000 [245:35:17<183:12:47,  6.25s/it]
 53%|█████▎    | 119488/225000 [245:35:23<176:41:51,  6.03s/it]
 53%|█████▎    | 119489/225000 [245:35:32<201:27:53,  6.87s/it]
 53%|█████▎    | 119490/225000 [245:35:35<173:48:12,  5.93s/it]
 53%|█████▎    | 119491/225000 [245:35:41<174:35:47,  5.96s/it]
 53%|█████▎    | 119492/225000 [245:35:45<154:13:28,  5.26s/it]
 53%|█████▎    | 119493/225000 [245:35:59<227:20:02,  7.76s/it]
 53%|█████▎    | 119494/225000 [245:36:02<187:21:43,  6.39s/it]
 53%|█████▎    | 119495/225000 [245:36:05<161:21:59,  5.51s/it]
 53%|█████▎    | 119496/225000 [245:36:14<194:32:01,  6.64s/it]
 53%|█████▎    | 119497/225000 [245:36:18<167:00:28,  5.70s/it]
 53%|█████▎    | 119498/225000 [245:36:23<160:34:10,  5.48s/it]
 53%|█████▎    | 119499/225000 [245:36:31<182:06:35,  6.21s/it]
 53%|█████▎    | 119500/225000 [245:36:35<162:44:12,  5.55s/it]
                                                               
{'loss': '0.8657', 'grad_norm': '0.942', 'learning_rate': '2.429e-05', 'epoch': '0.5311', 'num_input_tokens_seen': 409533264, 'train_runtime': '8.842e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119500/225000 [245:36:35<162:44:12,  5.55s/it]
 53%|█████▎    | 119501/225000 [245:36:40<161:30:08,  5.51s/it]
 53%|█████▎    | 119502/225000 [245:36:46<159:21:41,  5.44s/it]
 53%|█████▎    | 119503/225000 [245:36:50<152:04:48,  5.19s/it]
 53%|█████▎    | 119504/225000 [245:36:56<154:26:57,  5.27s/it]
 53%|█████▎    | 119505/225000 [245:37:00<148:22:45,  5.06s/it]
 53%|█████▎    | 119506/225000 [245:37:05<144:57:19,  4.95s/it]
 53%|█████▎    | 119507/225000 [245:37:10<146:02:55,  4.98s/it]
 53%|█████▎    | 119508/225000 [245:37:15<147:37:18,  5.04s/it]
 53%|█████▎    | 119509/225000 [245:37:20<148:42:34,  5.07s/it]
 53%|█████▎    | 119510/225000 [245:37:25<145:48:45,  4.98s/it]
 53%|█████▎    | 119511/225000 [245:37:30<148:39:29,  5.07s/it]
 53%|█████▎    | 119512/225000 [245:37:42<210:42:00,  7.19s/it]
 53%|█████▎    | 119513/225000 [245:37:49<207:34:17,  7.08s/it]
 53%|█████▎    | 119514/225000 [245:37:54<188:40:24,  6.44s/it]
 53%|█████▎    | 119515/225000 [245:37:59<170:32:14,  5.82s/it]
 53%|█████▎    | 119516/225000 [245:38:03<153:58:38,  5.26s/it]
 53%|█████▎    | 119517/225000 [245:38:07<143:29:45,  4.90s/it]
 53%|█████▎    | 119518/225000 [245:38:18<196:29:58,  6.71s/it]
 53%|█████▎    | 119519/225000 [245:38:22<175:52:45,  6.00s/it]
 53%|█████▎    | 119520/225000 [245:38:31<202:58:01,  6.93s/it]
 53%|█████▎    | 119521/225000 [245:38:34<172:30:08,  5.89s/it]
 53%|█████▎    | 119522/225000 [245:38:41<181:27:52,  6.19s/it]
 53%|█████▎    | 119523/225000 [245:38:47<173:53:17,  5.93s/it]
 53%|█████▎    | 119524/225000 [245:38:50<149:15:08,  5.09s/it]
 53%|█████▎    | 119525/225000 [245:38:59<181:44:09,  6.20s/it]
 53%|█████▎    | 119526/225000 [245:39:02<156:57:06,  5.36s/it]
 53%|█████▎    | 119527/225000 [245:39:10<183:39:53,  6.27s/it]
 53%|█████▎    | 119528/225000 [245:39:14<157:22:30,  5.37s/it]
 53%|█████▎    | 119529/225000 [245:39:22<184:22:30,  6.29s/it]
 53%|█████▎    | 119530/225000 [245:39:26<159:35:41,  5.45s/it]
 53%|█████▎    | 119531/225000 [245:39:34<182:07:02,  6.22s/it]
 53%|█████▎    | 119532/225000 [245:39:37<156:56:50,  5.36s/it]
 53%|█████▎    | 119533/225000 [245:39:45<177:39:17,  6.06s/it]
 53%|█████▎    | 119534/225000 [245:39:48<152:53:11,  5.22s/it]
 53%|█████▎    | 119535/225000 [245:39:58<192:45:56,  6.58s/it]
 53%|█████▎    | 119536/225000 [245:40:01<168:02:19,  5.74s/it]
 53%|█████▎    | 119537/225000 [245:40:05<149:15:30,  5.09s/it]
 53%|█████▎    | 119538/225000 [245:40:09<141:44:15,  4.84s/it]
 53%|█████▎    | 119539/225000 [245:40:14<140:19:18,  4.79s/it]
 53%|█████▎    | 119540/225000 [245:40:19<140:18:12,  4.79s/it]
 53%|█████▎    | 119541/225000 [245:40:24<143:48:15,  4.91s/it]
 53%|█████▎    | 119542/225000 [245:40:35<199:45:13,  6.82s/it]
 53%|█████▎    | 119543/225000 [245:40:41<189:31:14,  6.47s/it]
 53%|█████▎    | 119544/225000 [245:40:45<167:52:52,  5.73s/it]
 53%|█████▎    | 119545/225000 [245:40:53<188:30:10,  6.44s/it]
 53%|█████▎    | 119546/225000 [245:40:56<160:58:56,  5.50s/it]
 53%|█████▎    | 119547/225000 [245:41:00<148:33:38,  5.07s/it]
 53%|█████▎    | 119548/225000 [245:41:11<196:49:47,  6.72s/it]
 53%|█████▎    | 119549/225000 [245:41:15<175:30:39,  5.99s/it]
 53%|█████▎    | 119550/225000 [245:41:19<159:19:28,  5.44s/it]
                                                               
{'loss': '0.8668', 'grad_norm': '1.239', 'learning_rate': '2.427e-05', 'epoch': '0.5313', 'num_input_tokens_seen': 409657416, 'train_runtime': '8.845e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119550/225000 [245:41:19<159:19:28,  5.44s/it]
 53%|█████▎    | 119551/225000 [245:41:24<151:23:56,  5.17s/it]
 53%|█████▎    | 119552/225000 [245:41:28<142:30:13,  4.87s/it]
 53%|█████▎    | 119553/225000 [245:41:38<191:27:35,  6.54s/it]
 53%|█████▎    | 119554/225000 [245:41:46<201:55:42,  6.89s/it]
 53%|█████▎    | 119555/225000 [245:41:50<172:20:26,  5.88s/it]
 53%|█████▎    | 119556/225000 [245:41:53<151:15:40,  5.16s/it]
 53%|█████▎    | 119557/225000 [245:42:04<200:32:46,  6.85s/it]
 53%|█████▎    | 119558/225000 [245:42:07<167:08:57,  5.71s/it]
 53%|█████▎    | 119559/225000 [245:42:15<188:11:58,  6.43s/it]
 53%|█████▎    | 119560/225000 [245:42:18<159:10:17,  5.43s/it]
 53%|█████▎    | 119561/225000 [245:42:25<170:11:33,  5.81s/it]
 53%|█████▎    | 119562/225000 [245:42:29<153:53:50,  5.25s/it]
 53%|█████▎    | 119563/225000 [245:42:33<143:11:12,  4.89s/it]
 53%|█████▎    | 119564/225000 [245:42:37<135:11:35,  4.62s/it]
 53%|█████▎    | 119565/225000 [245:42:49<198:32:15,  6.78s/it]
 53%|█████▎    | 119566/225000 [245:42:53<173:32:33,  5.93s/it]
 53%|█████▎    | 119567/225000 [245:42:57<161:03:50,  5.50s/it]
 53%|█████▎    | 119568/225000 [245:43:02<151:37:08,  5.18s/it]
 53%|█████▎    | 119569/225000 [245:43:05<139:01:17,  4.75s/it]
 53%|█████▎    | 119570/225000 [245:43:13<167:45:56,  5.73s/it]
 53%|█████▎    | 119571/225000 [245:43:20<178:45:09,  6.10s/it]
 53%|█████▎    | 119572/225000 [245:43:24<158:33:52,  5.41s/it]
 53%|█████▎    | 119573/225000 [245:43:28<146:16:59,  5.00s/it]
 53%|█████▎    | 119574/225000 [245:43:36<168:53:47,  5.77s/it]
 53%|█████▎    | 119575/225000 [245:43:43<179:19:57,  6.12s/it]
 53%|█████▎    | 119576/225000 [245:43:50<193:24:42,  6.60s/it]
 53%|█████▎    | 119577/225000 [245:43:54<171:17:21,  5.85s/it]
 53%|█████▎    | 119578/225000 [245:44:01<180:37:44,  6.17s/it]
 53%|█████▎    | 119579/225000 [245:44:07<176:44:23,  6.04s/it]
 53%|█████▎    | 119580/225000 [245:44:11<159:20:47,  5.44s/it]
 53%|█████▎    | 119581/225000 [245:44:15<144:05:33,  4.92s/it]
 53%|█████▎    | 119582/225000 [245:44:23<172:08:18,  5.88s/it]
 53%|█████▎    | 119583/225000 [245:44:29<175:03:43,  5.98s/it]
 53%|█████▎    | 119584/225000 [245:44:39<206:42:14,  7.06s/it]
 53%|█████▎    | 119585/225000 [245:44:42<173:57:53,  5.94s/it]
 53%|█████▎    | 119586/225000 [245:44:47<160:52:48,  5.49s/it]
 53%|█████▎    | 119587/225000 [245:44:51<151:56:12,  5.19s/it]
 53%|█████▎    | 119588/225000 [245:44:55<141:00:09,  4.82s/it]
 53%|█████▎    | 119589/225000 [245:44:59<137:01:16,  4.68s/it]
 53%|█████▎    | 119590/225000 [245:45:07<164:58:35,  5.63s/it]
 53%|█████▎    | 119591/225000 [245:45:18<207:47:52,  7.10s/it]
 53%|█████▎    | 119592/225000 [245:45:21<173:10:43,  5.91s/it]
 53%|█████▎    | 119593/225000 [245:45:25<155:30:52,  5.31s/it]
 53%|█████▎    | 119594/225000 [245:45:30<150:32:26,  5.14s/it]
 53%|█████▎    | 119595/225000 [245:45:34<148:21:44,  5.07s/it]
 53%|█████▎    | 119596/225000 [245:45:38<139:02:30,  4.75s/it]
 53%|█████▎    | 119597/225000 [245:45:48<184:01:00,  6.29s/it]
 53%|█████▎    | 119598/225000 [245:45:54<174:46:18,  5.97s/it]
 53%|█████▎    | 119599/225000 [245:45:58<158:06:44,  5.40s/it]
 53%|█████▎    | 119600/225000 [245:46:01<137:48:51,  4.71s/it]
                                                               
{'loss': '0.8585', 'grad_norm': '1.216', 'learning_rate': '2.425e-05', 'epoch': '0.5316', 'num_input_tokens_seen': 409782048, 'train_runtime': '8.848e+05', 'train_tokens_per_second': '463.2'}

 53%|█████▎    | 119600/225000 [245:46:01<137:48:51,  4.71s/it]
 53%|█████▎    | 119601/225000 [245:46:10<177:13:52,  6.05s/it]
 53%|█████▎    | 119602/225000 [245:46:17<184:51:16,  6.31s/it]
 53%|█████▎    | 119603/225000 [245:46:22<177:15:56,  6.05s/it]
 53%|█████▎    | 119604/225000 [245:46:30<189:59:20,  6.49s/it]
 53%|█████▎    | 119605/225000 [245:46:35<180:11:11,  6.15s/it]
 53%|█████▎    | 119606/225000 [245:46:40<172:05:48,  5.88s/it]
 53%|█████▎    | 119607/225000 [245:46:46<173:51:52,  5.94s/it]
 53%|█████▎    | 119608/225000 [245:46:51<162:22:00,  5.55s/it]
 53%|█████▎    | 119609/225000 [245:46:55<151:50:14,  5.19s/it]
 53%|█████▎    | 119610/225000 [245:47:00<146:41:58,  5.01s/it]
 53%|█████▎    | 119611/225000 [245:47:05<143:42:23,  4.91s/it]
 53%|█████▎    | 119612/225000 [245:47:09<138:23:58,  4.73s/it]
 53%|█████▎    | 119613/225000 [245:47:20<192:27:06,  6.57s/it]
 53%|█████▎    | 119614/225000 [245:47:28<203:05:13,  6.94s/it]
 53%|█████▎    | 119615/225000 [245:47:36<212:17:28,  7.25s/it]
 53%|█████▎    | 119616/225000 [245:47:40<186:10:49,  6.36s/it]
 53%|█████▎    | 119617/225000 [245:47:46<179:11:18,  6.12s/it]
 53%|█████▎    | 119618/225000 [245:47:53<192:11:35,  6.57s/it]
 53%|█████▎    | 119619/225000 [245:47:57<169:20:45,  5.79s/it]
 53%|█████▎    | 119620/225000 [245:48:00<147:13:03,  5.03s/it]
 53%|█████▎    | 119621/225000 [245:48:04<136:37:38,  4.67s/it]
 53%|█████▎    | 119622/225000 [245:48:13<174:11:20,  5.95s/it]
 53%|█████▎    | 119623/225000 [245:48:19<172:19:54,  5.89s/it]
 53%|█████▎    | 119624/225000 [245:48:22<148:50:53,  5.09s/it]
 53%|█████▎    | 119625/225000 [245:48:27<143:56:40,  4.92s/it]
 53%|█████▎    | 119626/225000 [245:48:38<201:40:25,  6.89s/it]
 53%|█████▎    | 119627/225000 [245:48:44<190:31:02,  6.51s/it]
 53%|█████▎    | 119628/225000 [245:48:48<170:19:32,  5.82s/it]
 53%|█████▎    | 119629/225000 [245:48:57<194:44:02,  6.65s/it]
 53%|█████▎    | 119630/225000 [245:49:02<186:46:25,  6.38s/it]
 53%|█████▎    | 119631/225000 [245:49:06<165:58:13,  5.67s/it]
 53%|█████▎    | 119632/225000 [245:49:11<155:24:17,  5.31s/it]
 53%|█████▎    | 119633/225000 [245:49:21<201:14:59,  6.88s/it]
 53%|█████▎    | 119634/225000 [245:49:25<174:56:46,  5.98s/it]
 53%|█████▎    | 119635/225000 [245:49:30<160:44:42,  5.49s/it]
 53%|█████▎    | 119636/225000 [245:49:34<152:20:24,  5.21s/it]
 53%|█████▎    | 119637/225000 [245:49:39<146:11:14,  4.99s/it]
 53%|█████▎    | 119638/225000 [245:49:43<141:50:25,  4.85s/it]
 53%|█████▎    | 119639/225000 [245:49:55<207:34:29,  7.09s/it]
 53%|█████▎    | 119640/225000 [245:49:59<180:30:40,  6.17s/it]
 53%|█████▎    | 119641/225000 [245:50:03<161:04:23,  5.50s/it]
 53%|█████▎    | 119642/225000 [245:50:07<146:40:09,  5.01s/it]
 53%|█████▎    | 119643/225000 [245:50:13<156:48:57,  5.36s/it]
 53%|█████▎    | 119644/225000 [245:50:25<212:21:12,  7.26s/it]
 53%|█████▎    | 119645/225000 [245:50:28<177:44:35,  6.07s/it]
 53%|█████▎    | 119646/225000 [245:50:34<178:00:50,  6.08s/it]
 53%|█████▎    | 119647/225000 [245:50:42<186:53:13,  6.39s/it]
 53%|█████▎    | 119648/225000 [245:50:47<174:53:00,  5.98s/it]
 53%|█████▎    | 119649/225000 [245:50:51<162:58:21,  5.57s/it]
 53%|█████▎    | 119650/225000 [245:50:56<155:17:57,  5.31s/it]
                                                               
{'loss': '0.8585', 'grad_norm': '1.115', 'learning_rate': '2.423e-05', 'epoch': '0.5318', 'num_input_tokens_seen': 409908416, 'train_runtime': '8.851e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119650/225000 [245:50:56<155:17:57,  5.31s/it]
 53%|█████▎    | 119651/225000 [245:51:01<149:11:10,  5.10s/it]
 53%|█████▎    | 119652/225000 [245:51:05<147:59:12,  5.06s/it]
 53%|█████▎    | 119653/225000 [245:51:10<140:55:29,  4.82s/it]
 53%|█████▎    | 119654/225000 [245:51:14<138:31:45,  4.73s/it]
 53%|█████▎    | 119655/225000 [245:51:29<225:47:03,  7.72s/it]
 53%|█████▎    | 119656/225000 [245:51:33<190:07:26,  6.50s/it]
 53%|█████▎    | 119657/225000 [245:51:38<181:26:46,  6.20s/it]
 53%|█████▎    | 119658/225000 [245:51:48<217:43:56,  7.44s/it]
 53%|█████▎    | 119659/225000 [245:51:52<180:04:50,  6.15s/it]
 53%|█████▎    | 119660/225000 [245:51:56<166:11:33,  5.68s/it]
 53%|█████▎    | 119661/225000 [245:52:01<155:45:25,  5.32s/it]
 53%|█████▎    | 119662/225000 [245:52:06<156:08:03,  5.34s/it]
 53%|█████▎    | 119663/225000 [245:52:14<177:11:25,  6.06s/it]
 53%|█████▎    | 119664/225000 [245:52:17<152:33:52,  5.21s/it]
 53%|█████▎    | 119665/225000 [245:52:26<185:23:51,  6.34s/it]
 53%|█████▎    | 119666/225000 [245:52:30<163:07:15,  5.57s/it]
 53%|█████▎    | 119667/225000 [245:52:33<145:20:39,  4.97s/it]
 53%|█████▎    | 119668/225000 [245:52:43<184:06:16,  6.29s/it]
 53%|█████▎    | 119669/225000 [245:52:47<167:41:57,  5.73s/it]
 53%|█████▎    | 119670/225000 [245:52:51<150:47:22,  5.15s/it]
 53%|█████▎    | 119671/225000 [245:52:56<146:01:10,  4.99s/it]
 53%|█████▎    | 119672/225000 [245:53:00<143:33:54,  4.91s/it]
 53%|█████▎    | 119673/225000 [245:53:07<161:44:55,  5.53s/it]
 53%|█████▎    | 119674/225000 [245:53:19<216:10:14,  7.39s/it]
 53%|█████▎    | 119675/225000 [245:53:23<186:18:53,  6.37s/it]
 53%|█████▎    | 119676/225000 [245:53:27<164:12:14,  5.61s/it]
 53%|█████▎    | 119677/225000 [245:53:31<151:58:07,  5.19s/it]
 53%|█████▎    | 119678/225000 [245:53:37<160:18:32,  5.48s/it]
 53%|█████▎    | 119679/225000 [245:53:47<201:51:41,  6.90s/it]
 53%|█████▎    | 119680/225000 [245:53:51<177:06:41,  6.05s/it]
 53%|█████▎    | 119681/225000 [245:53:56<161:43:55,  5.53s/it]
 53%|█████▎    | 119682/225000 [245:54:02<167:34:32,  5.73s/it]
 53%|█████▎    | 119683/225000 [245:54:11<193:38:16,  6.62s/it]
 53%|█████▎    | 119684/225000 [245:54:14<166:53:26,  5.70s/it]
 53%|█████▎    | 119685/225000 [245:54:18<150:07:02,  5.13s/it]
 53%|█████▎    | 119686/225000 [245:54:25<168:14:59,  5.75s/it]
 53%|█████▎    | 119687/225000 [245:54:32<176:17:59,  6.03s/it]
 53%|█████▎    | 119688/225000 [245:54:43<224:24:33,  7.67s/it]
 53%|█████▎    | 119689/225000 [245:54:47<185:57:15,  6.36s/it]
 53%|█████▎    | 119690/225000 [245:54:53<185:32:17,  6.34s/it]
 53%|█████▎    | 119691/225000 [245:54:58<176:15:51,  6.03s/it]
 53%|█████▎    | 119692/225000 [245:55:02<154:30:34,  5.28s/it]
 53%|█████▎    | 119693/225000 [245:55:06<146:23:48,  5.00s/it]
 53%|█████▎    | 119694/225000 [245:55:11<140:58:34,  4.82s/it]
 53%|█████▎    | 119695/225000 [245:55:14<132:49:50,  4.54s/it]
 53%|█████▎    | 119696/225000 [245:55:19<129:57:36,  4.44s/it]
 53%|█████▎    | 119697/225000 [245:55:23<133:19:52,  4.56s/it]
 53%|█████▎    | 119698/225000 [245:55:28<132:51:19,  4.54s/it]
 53%|█████▎    | 119699/225000 [245:55:37<172:56:28,  5.91s/it]
 53%|█████▎    | 119700/225000 [245:55:46<201:57:02,  6.90s/it]
                                                               
{'loss': '0.8537', 'grad_norm': '1.246', 'learning_rate': '2.422e-05', 'epoch': '0.532', 'num_input_tokens_seen': 410034512, 'train_runtime': '8.853e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119700/225000 [245:55:46<201:57:02,  6.90s/it]
 53%|█████▎    | 119701/225000 [245:55:50<172:53:08,  5.91s/it]
 53%|█████▎    | 119702/225000 [245:55:59<200:17:04,  6.85s/it]
 53%|█████▎    | 119703/225000 [245:56:03<177:43:48,  6.08s/it]
 53%|█████▎    | 119704/225000 [245:56:09<175:03:42,  5.99s/it]
 53%|█████▎    | 119705/225000 [245:56:12<149:42:01,  5.12s/it]
 53%|█████▎    | 119706/225000 [245:56:16<139:08:50,  4.76s/it]
 53%|█████▎    | 119707/225000 [245:56:24<163:55:13,  5.60s/it]
 53%|█████▎    | 119708/225000 [245:56:33<193:30:37,  6.62s/it]
 53%|█████▎    | 119709/225000 [245:56:37<174:05:15,  5.95s/it]
 53%|█████▎    | 119710/225000 [245:56:46<198:04:07,  6.77s/it]
 53%|█████▎    | 119711/225000 [245:56:49<167:17:18,  5.72s/it]
 53%|█████▎    | 119712/225000 [245:56:54<161:49:11,  5.53s/it]
 53%|█████▎    | 119713/225000 [245:56:59<156:20:49,  5.35s/it]
 53%|█████▎    | 119714/225000 [245:57:03<148:23:17,  5.07s/it]
 53%|█████▎    | 119715/225000 [245:57:08<144:44:07,  4.95s/it]
 53%|█████▎    | 119716/225000 [245:57:13<146:13:26,  5.00s/it]
 53%|█████▎    | 119717/225000 [245:57:18<147:06:19,  5.03s/it]
 53%|█████▎    | 119718/225000 [245:57:23<144:47:41,  4.95s/it]
 53%|█████▎    | 119719/225000 [245:57:28<145:30:21,  4.98s/it]
 53%|█████▎    | 119720/225000 [245:57:33<146:07:18,  5.00s/it]
 53%|█████▎    | 119721/225000 [245:57:41<171:20:16,  5.86s/it]
 53%|█████▎    | 119722/225000 [245:57:53<224:40:08,  7.68s/it]
 53%|█████▎    | 119723/225000 [245:57:57<189:40:36,  6.49s/it]
 53%|█████▎    | 119724/225000 [245:58:00<162:58:07,  5.57s/it]
 53%|█████▎    | 119725/225000 [245:58:10<202:50:27,  6.94s/it]
 53%|█████▎    | 119726/225000 [245:58:13<169:49:12,  5.81s/it]
 53%|█████▎    | 119727/225000 [245:58:22<194:35:42,  6.65s/it]
 53%|█████▎    | 119728/225000 [245:58:26<173:37:28,  5.94s/it]
 53%|█████▎    | 119729/225000 [245:58:31<167:15:59,  5.72s/it]
 53%|█████▎    | 119730/225000 [245:58:36<159:51:48,  5.47s/it]
 53%|█████▎    | 119731/225000 [245:58:41<156:11:59,  5.34s/it]
 53%|█████▎    | 119732/225000 [245:58:46<148:18:45,  5.07s/it]
 53%|█████▎    | 119733/225000 [245:58:51<145:04:47,  4.96s/it]
 53%|█████▎    | 119734/225000 [245:58:57<156:51:58,  5.36s/it]
 53%|█████▎    | 119735/225000 [245:59:07<201:46:50,  6.90s/it]
 53%|█████▎    | 119736/225000 [245:59:13<195:10:11,  6.67s/it]
 53%|█████▎    | 119737/225000 [245:59:18<172:14:10,  5.89s/it]
 53%|█████▎    | 119738/225000 [245:59:22<157:45:16,  5.40s/it]
 53%|█████▎    | 119739/225000 [245:59:32<196:47:01,  6.73s/it]
 53%|█████▎    | 119740/225000 [245:59:35<171:12:16,  5.86s/it]
 53%|█████▎    | 119741/225000 [245:59:39<153:47:04,  5.26s/it]
 53%|█████▎    | 119742/225000 [245:59:47<172:34:16,  5.90s/it]
 53%|█████▎    | 119743/225000 [245:59:54<183:58:22,  6.29s/it]
 53%|█████▎    | 119744/225000 [245:59:58<161:35:36,  5.53s/it]
 53%|█████▎    | 119745/225000 [246:00:02<150:06:07,  5.13s/it]
 53%|█████▎    | 119746/225000 [246:00:06<141:43:26,  4.85s/it]
 53%|█████▎    | 119747/225000 [246:00:10<136:11:46,  4.66s/it]
 53%|█████▎    | 119748/225000 [246:00:22<202:24:54,  6.92s/it]
 53%|█████▎    | 119749/225000 [246:00:26<170:39:33,  5.84s/it]
 53%|█████▎    | 119750/225000 [246:00:30<152:56:07,  5.23s/it]
                                                               
{'loss': '0.8718', 'grad_norm': '1.139', 'learning_rate': '2.42e-05', 'epoch': '0.5322', 'num_input_tokens_seen': 410158920, 'train_runtime': '8.856e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119750/225000 [246:00:30<152:56:07,  5.23s/it]
 53%|█████▎    | 119751/225000 [246:00:39<191:13:51,  6.54s/it]
 53%|█████▎    | 119752/225000 [246:00:43<164:08:27,  5.61s/it]
 53%|█████▎    | 119753/225000 [246:00:48<160:57:23,  5.51s/it]
 53%|█████▎    | 119754/225000 [246:00:57<195:11:17,  6.68s/it]
 53%|█████▎    | 119755/225000 [246:01:01<167:03:12,  5.71s/it]
 53%|█████▎    | 119756/225000 [246:01:07<170:41:08,  5.84s/it]
 53%|█████▎    | 119757/225000 [246:01:13<177:30:17,  6.07s/it]
 53%|█████▎    | 119758/225000 [246:01:17<153:34:11,  5.25s/it]
 53%|█████▎    | 119759/225000 [246:01:26<185:23:51,  6.34s/it]
 53%|█████▎    | 119760/225000 [246:01:29<159:00:08,  5.44s/it]
 53%|█████▎    | 119761/225000 [246:01:37<177:41:11,  6.08s/it]
 53%|█████▎    | 119762/225000 [246:01:40<152:02:55,  5.20s/it]
 53%|█████▎    | 119763/225000 [246:01:44<146:20:01,  5.01s/it]
 53%|█████▎    | 119764/225000 [246:01:49<147:24:23,  5.04s/it]
 53%|█████▎    | 119765/225000 [246:01:55<149:21:28,  5.11s/it]
 53%|█████▎    | 119766/225000 [246:01:59<143:06:42,  4.90s/it]
 53%|█████▎    | 119767/225000 [246:02:04<144:54:32,  4.96s/it]
 53%|█████▎    | 119768/225000 [246:02:09<145:31:28,  4.98s/it]
 53%|█████▎    | 119769/225000 [246:02:15<150:17:07,  5.14s/it]
 53%|█████▎    | 119770/225000 [246:02:20<151:14:09,  5.17s/it]
 53%|█████▎    | 119771/225000 [246:02:29<187:43:17,  6.42s/it]
 53%|█████▎    | 119772/225000 [246:02:36<189:46:15,  6.49s/it]
 53%|█████▎    | 119773/225000 [246:02:39<161:34:00,  5.53s/it]
 53%|█████▎    | 119774/225000 [246:02:47<180:47:36,  6.19s/it]
 53%|█████▎    | 119775/225000 [246:02:53<179:48:13,  6.15s/it]
 53%|█████▎    | 119776/225000 [246:02:57<157:10:11,  5.38s/it]
 53%|█████▎    | 119777/225000 [246:03:06<193:04:34,  6.61s/it]
 53%|█████▎    | 119778/225000 [246:03:09<164:38:07,  5.63s/it]
 53%|█████▎    | 119779/225000 [246:03:13<142:58:55,  4.89s/it]
 53%|█████▎    | 119780/225000 [246:03:23<189:58:14,  6.50s/it]
 53%|█████▎    | 119781/225000 [246:03:28<176:20:01,  6.03s/it]
 53%|█████▎    | 119782/225000 [246:03:32<163:34:11,  5.60s/it]
 53%|█████▎    | 119783/225000 [246:03:37<152:31:13,  5.22s/it]
 53%|█████▎    | 119784/225000 [246:03:42<152:31:18,  5.22s/it]
 53%|█████▎    | 119785/225000 [246:03:52<190:46:16,  6.53s/it]
 53%|█████▎    | 119786/225000 [246:03:55<165:49:15,  5.67s/it]
 53%|█████▎    | 119787/225000 [246:03:59<147:51:08,  5.06s/it]
 53%|█████▎    | 119788/225000 [246:04:06<167:18:20,  5.72s/it]
 53%|█████▎    | 119789/225000 [246:04:14<188:58:23,  6.47s/it]
 53%|█████▎    | 119790/225000 [246:04:18<164:18:29,  5.62s/it]
 53%|█████▎    | 119791/225000 [246:04:23<155:35:58,  5.32s/it]
 53%|█████▎    | 119792/225000 [246:04:31<184:37:13,  6.32s/it]
 53%|█████▎    | 119793/225000 [246:04:35<162:11:21,  5.55s/it]
 53%|█████▎    | 119794/225000 [246:04:39<148:14:03,  5.07s/it]
 53%|█████▎    | 119795/225000 [246:04:44<149:49:14,  5.13s/it]
 53%|█████▎    | 119796/225000 [246:04:54<192:46:41,  6.60s/it]
 53%|█████▎    | 119797/225000 [246:04:58<163:49:23,  5.61s/it]
 53%|█████▎    | 119798/225000 [246:05:05<176:02:37,  6.02s/it]
 53%|█████▎    | 119799/225000 [246:05:10<171:30:15,  5.87s/it]
 53%|█████▎    | 119800/225000 [246:05:16<172:58:24,  5.92s/it]
                                                               
{'loss': '0.8632', 'grad_norm': '1.208', 'learning_rate': '2.418e-05', 'epoch': '0.5324', 'num_input_tokens_seen': 410282448, 'train_runtime': '8.859e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119800/225000 [246:05:16<172:58:24,  5.92s/it]
 53%|█████▎    | 119801/225000 [246:05:20<154:24:40,  5.28s/it]
 53%|█████▎    | 119802/225000 [246:05:25<150:27:34,  5.15s/it]
 53%|█████▎    | 119803/225000 [246:05:30<151:32:58,  5.19s/it]
 53%|█████▎    | 119804/225000 [246:05:34<143:31:54,  4.91s/it]
 53%|█████▎    | 119805/225000 [246:05:39<144:08:08,  4.93s/it]
 53%|█████▎    | 119806/225000 [246:05:44<140:57:17,  4.82s/it]
 53%|█████▎    | 119807/225000 [246:05:48<139:11:27,  4.76s/it]
 53%|█████▎    | 119808/225000 [246:05:54<142:57:30,  4.89s/it]
 53%|█████▎    | 119809/225000 [246:05:58<140:38:04,  4.81s/it]
 53%|█████▎    | 119810/225000 [246:06:08<179:41:53,  6.15s/it]
 53%|█████▎    | 119811/225000 [246:06:18<215:38:46,  7.38s/it]
 53%|█████▎    | 119812/225000 [246:06:21<181:15:09,  6.20s/it]
 53%|█████▎    | 119813/225000 [246:06:25<160:47:42,  5.50s/it]
 53%|█████▎    | 119814/225000 [246:06:31<163:43:19,  5.60s/it]
 53%|█████▎    | 119815/225000 [246:06:41<199:51:02,  6.84s/it]
 53%|█████▎    | 119816/225000 [246:06:45<178:26:29,  6.11s/it]
 53%|█████▎    | 119817/225000 [246:06:49<161:04:32,  5.51s/it]
 53%|█████▎    | 119818/225000 [246:06:53<149:21:53,  5.11s/it]
 53%|█████▎    | 119819/225000 [246:07:01<171:41:53,  5.88s/it]
 53%|█████▎    | 119820/225000 [246:07:09<193:24:56,  6.62s/it]
 53%|█████▎    | 119821/225000 [246:07:14<180:08:23,  6.17s/it]
 53%|█████▎    | 119822/225000 [246:07:20<170:34:02,  5.84s/it]
 53%|█████▎    | 119823/225000 [246:07:24<160:43:34,  5.50s/it]
 53%|█████▎    | 119824/225000 [246:07:29<154:27:41,  5.29s/it]
 53%|█████▎    | 119825/225000 [246:07:34<149:39:46,  5.12s/it]
 53%|█████▎    | 119826/225000 [246:07:39<149:22:27,  5.11s/it]
 53%|█████▎    | 119827/225000 [246:07:50<199:45:31,  6.84s/it]
 53%|█████▎    | 119828/225000 [246:07:58<211:19:06,  7.23s/it]
 53%|█████▎    | 119829/225000 [246:08:02<184:15:53,  6.31s/it]
 53%|█████▎    | 119830/225000 [246:08:06<165:20:29,  5.66s/it]
 53%|█████▎    | 119831/225000 [246:08:12<169:04:32,  5.79s/it]
 53%|█████▎    | 119832/225000 [246:08:21<193:34:15,  6.63s/it]
 53%|█████▎    | 119833/225000 [246:08:28<196:06:26,  6.71s/it]
 53%|█████▎    | 119834/225000 [246:08:31<167:21:02,  5.73s/it]
 53%|█████▎    | 119835/225000 [246:08:40<192:35:32,  6.59s/it]
 53%|█████▎    | 119836/225000 [246:08:44<173:50:17,  5.95s/it]
 53%|█████▎    | 119837/225000 [246:08:52<187:16:58,  6.41s/it]
 53%|█████▎    | 119838/225000 [246:08:58<184:29:38,  6.32s/it]
 53%|█████▎    | 119839/225000 [246:09:04<183:29:56,  6.28s/it]
 53%|█████▎    | 119840/225000 [246:09:12<200:54:55,  6.88s/it]
 53%|█████▎    | 119841/225000 [246:09:16<169:54:40,  5.82s/it]
 53%|█████▎    | 119842/225000 [246:09:21<165:41:02,  5.67s/it]
 53%|█████▎    | 119843/225000 [246:09:26<161:45:55,  5.54s/it]
 53%|█████▎    | 119844/225000 [246:09:31<158:31:53,  5.43s/it]
 53%|█████▎    | 119845/225000 [246:09:37<156:55:04,  5.37s/it]
 53%|█████▎    | 119846/225000 [246:09:42<154:22:13,  5.28s/it]
 53%|█████▎    | 119847/225000 [246:09:47<154:00:29,  5.27s/it]
 53%|█████▎    | 119848/225000 [246:09:52<154:18:16,  5.28s/it]
 53%|█████▎    | 119849/225000 [246:09:57<150:33:19,  5.15s/it]
 53%|█████▎    | 119850/225000 [246:10:05<170:45:19,  5.85s/it]
                                                               
{'loss': '0.8695', 'grad_norm': '1.166', 'learning_rate': '2.416e-05', 'epoch': '0.5327', 'num_input_tokens_seen': 410407232, 'train_runtime': '8.862e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119850/225000 [246:10:05<170:45:19,  5.85s/it]
 53%|█████▎    | 119851/225000 [246:10:09<158:44:26,  5.43s/it]
 53%|█████▎    | 119852/225000 [246:10:15<160:29:09,  5.49s/it]
 53%|█████▎    | 119853/225000 [246:10:19<151:09:55,  5.18s/it]
 53%|█████▎    | 119854/225000 [246:10:32<214:12:33,  7.33s/it]
 53%|█████▎    | 119855/225000 [246:10:36<190:29:30,  6.52s/it]
 53%|█████▎    | 119856/225000 [246:10:41<171:56:15,  5.89s/it]
 53%|█████▎    | 119857/225000 [246:10:45<155:17:46,  5.32s/it]
 53%|█████▎    | 119858/225000 [246:10:49<151:28:10,  5.19s/it]
 53%|█████▎    | 119859/225000 [246:10:54<148:14:59,  5.08s/it]
 53%|█████▎    | 119860/225000 [246:10:59<144:12:47,  4.94s/it]
 53%|█████▎    | 119861/225000 [246:11:10<195:43:21,  6.70s/it]
 53%|█████▎    | 119862/225000 [246:11:15<183:20:43,  6.28s/it]
 53%|█████▎    | 119863/225000 [246:11:21<178:15:53,  6.10s/it]
 53%|█████▎    | 119864/225000 [246:11:26<171:10:05,  5.86s/it]
 53%|█████▎    | 119865/225000 [246:11:31<159:53:57,  5.48s/it]
 53%|█████▎    | 119866/225000 [246:11:35<152:47:38,  5.23s/it]
 53%|█████▎    | 119867/225000 [246:11:40<146:06:27,  5.00s/it]
 53%|█████▎    | 119868/225000 [246:11:44<144:32:21,  4.95s/it]
 53%|█████▎    | 119869/225000 [246:11:51<160:40:31,  5.50s/it]
 53%|█████▎    | 119870/225000 [246:12:04<223:19:35,  7.65s/it]
 53%|█████▎    | 119871/225000 [246:12:07<186:42:24,  6.39s/it]
 53%|█████▎    | 119872/225000 [246:12:13<181:43:38,  6.22s/it]
 53%|█████▎    | 119873/225000 [246:12:20<182:43:48,  6.26s/it]
 53%|█████▎    | 119874/225000 [246:12:24<167:52:34,  5.75s/it]
 53%|█████▎    | 119875/225000 [246:12:29<157:52:57,  5.41s/it]
 53%|█████▎    | 119876/225000 [246:12:33<148:41:18,  5.09s/it]
 53%|█████▎    | 119877/225000 [246:12:38<143:52:44,  4.93s/it]
 53%|█████▎    | 119878/225000 [246:12:42<141:23:42,  4.84s/it]
 53%|█████▎    | 119879/225000 [246:12:47<141:56:13,  4.86s/it]
 53%|█████▎    | 119880/225000 [246:12:56<181:05:12,  6.20s/it]
 53%|█████▎    | 119881/225000 [246:13:04<193:19:15,  6.62s/it]
 53%|█████▎    | 119882/225000 [246:13:10<189:41:02,  6.50s/it]
 53%|█████▎    | 119883/225000 [246:13:20<217:58:15,  7.46s/it]
 53%|█████▎    | 119884/225000 [246:13:23<182:45:44,  6.26s/it]
 53%|█████▎    | 119885/225000 [246:13:27<155:53:41,  5.34s/it]
 53%|█████▎    | 119886/225000 [246:13:34<171:27:49,  5.87s/it]
 53%|█████▎    | 119887/225000 [246:13:41<185:06:43,  6.34s/it]
 53%|█████▎    | 119888/225000 [246:13:46<167:37:44,  5.74s/it]
 53%|█████▎    | 119889/225000 [246:13:50<154:33:04,  5.29s/it]
 53%|█████▎    | 119890/225000 [246:13:54<144:40:56,  4.96s/it]
 53%|█████▎    | 119891/225000 [246:13:58<137:49:36,  4.72s/it]
 53%|█████▎    | 119892/225000 [246:14:08<180:22:42,  6.18s/it]
 53%|█████▎    | 119893/225000 [246:14:15<188:10:58,  6.45s/it]
 53%|█████▎    | 119894/225000 [246:14:18<160:06:26,  5.48s/it]
 53%|█████▎    | 119895/225000 [246:14:27<190:47:31,  6.53s/it]
 53%|█████▎    | 119896/225000 [246:14:31<169:31:04,  5.81s/it]
 53%|█████▎    | 119897/225000 [246:14:35<156:14:31,  5.35s/it]
 53%|█████▎    | 119898/225000 [246:14:46<201:27:27,  6.90s/it]
 53%|█████▎    | 119899/225000 [246:14:51<186:08:57,  6.38s/it]
 53%|█████▎    | 119900/225000 [246:14:56<175:27:42,  6.01s/it]
                                                               
{'loss': '0.8587', 'grad_norm': '1.038', 'learning_rate': '2.414e-05', 'epoch': '0.5329', 'num_input_tokens_seen': 410532552, 'train_runtime': '8.865e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119900/225000 [246:14:56<175:27:42,  6.01s/it]
 53%|█████▎    | 119901/225000 [246:15:00<154:42:31,  5.30s/it]
 53%|█████▎    | 119902/225000 [246:15:08<178:39:00,  6.12s/it]
 53%|█████▎    | 119903/225000 [246:15:12<159:26:07,  5.46s/it]
 53%|█████▎    | 119904/225000 [246:15:16<148:03:46,  5.07s/it]
 53%|█████▎    | 119905/225000 [246:15:27<198:00:12,  6.78s/it]
 53%|█████▎    | 119906/225000 [246:15:31<175:02:55,  6.00s/it]
 53%|█████▎    | 119907/225000 [246:15:38<183:49:47,  6.30s/it]
 53%|█████▎    | 119908/225000 [246:15:43<171:19:16,  5.87s/it]
 53%|█████▎    | 119909/225000 [246:15:48<164:29:09,  5.63s/it]
 53%|█████▎    | 119910/225000 [246:15:52<151:14:56,  5.18s/it]
 53%|█████▎    | 119911/225000 [246:15:57<149:42:33,  5.13s/it]
 53%|█████▎    | 119912/225000 [246:16:03<153:47:44,  5.27s/it]
 53%|█████▎    | 119913/225000 [246:16:12<191:06:36,  6.55s/it]
 53%|█████▎    | 119914/225000 [246:16:15<162:39:36,  5.57s/it]
 53%|█████▎    | 119915/225000 [246:16:19<146:42:43,  5.03s/it]
 53%|█████▎    | 119916/225000 [246:16:28<177:05:19,  6.07s/it]
 53%|█████▎    | 119917/225000 [246:16:33<169:04:01,  5.79s/it]
 53%|█████▎    | 119918/225000 [246:16:37<153:49:58,  5.27s/it]
 53%|█████▎    | 119919/225000 [246:16:41<140:23:33,  4.81s/it]
 53%|█████▎    | 119920/225000 [246:16:48<165:54:48,  5.68s/it]
 53%|█████▎    | 119921/225000 [246:16:57<188:57:27,  6.47s/it]
 53%|█████▎    | 119922/225000 [246:17:00<165:01:52,  5.65s/it]
 53%|█████▎    | 119923/225000 [246:17:04<146:31:16,  5.02s/it]
 53%|█████▎    | 119924/225000 [246:17:13<181:10:08,  6.21s/it]
 53%|█████▎    | 119925/225000 [246:17:16<154:54:56,  5.31s/it]
 53%|█████▎    | 119926/225000 [246:17:20<146:15:24,  5.01s/it]
 53%|█████▎    | 119927/225000 [246:17:25<142:19:27,  4.88s/it]
 53%|█████▎    | 119928/225000 [246:17:29<135:33:26,  4.64s/it]
 53%|█████▎    | 119929/225000 [246:17:33<132:15:34,  4.53s/it]
 53%|█████▎    | 119930/225000 [246:17:43<177:22:33,  6.08s/it]
 53%|█████▎    | 119931/225000 [246:17:49<175:21:51,  6.01s/it]
 53%|█████▎    | 119932/225000 [246:17:52<152:41:54,  5.23s/it]
 53%|█████▎    | 119933/225000 [246:18:01<179:57:17,  6.17s/it]
 53%|█████▎    | 119934/225000 [246:18:04<153:21:57,  5.25s/it]
 53%|█████▎    | 119935/225000 [246:18:10<165:21:56,  5.67s/it]
 53%|█████▎    | 119936/225000 [246:18:15<159:57:07,  5.48s/it]
 53%|█████▎    | 119937/225000 [246:18:20<147:07:09,  5.04s/it]
 53%|█████▎    | 119938/225000 [246:18:24<145:28:33,  4.98s/it]
 53%|█████▎    | 119939/225000 [246:18:29<140:15:11,  4.81s/it]
 53%|█████▎    | 119940/225000 [246:18:33<139:16:22,  4.77s/it]
 53%|█████▎    | 119941/225000 [246:18:38<136:57:31,  4.69s/it]
 53%|█████▎    | 119942/225000 [246:18:43<138:15:37,  4.74s/it]
 53%|█████▎    | 119943/225000 [246:18:47<137:53:51,  4.73s/it]
 53%|█████▎    | 119944/225000 [246:18:53<141:07:42,  4.84s/it]
 53%|█████▎    | 119945/225000 [246:19:03<193:18:42,  6.62s/it]
 53%|█████▎    | 119946/225000 [246:19:09<180:51:26,  6.20s/it]
 53%|█████▎    | 119947/225000 [246:19:15<186:09:52,  6.38s/it]
 53%|█████▎    | 119948/225000 [246:19:19<164:53:45,  5.65s/it]
 53%|█████▎    | 119949/225000 [246:19:23<150:51:55,  5.17s/it]
 53%|█████▎    | 119950/225000 [246:19:33<193:31:42,  6.63s/it]
                                                               
{'loss': '0.859', 'grad_norm': '1.179', 'learning_rate': '2.412e-05', 'epoch': '0.5331', 'num_input_tokens_seen': 410657624, 'train_runtime': '8.868e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 119950/225000 [246:19:33<193:31:42,  6.63s/it]
 53%|█████▎    | 119951/225000 [246:19:37<163:25:23,  5.60s/it]
 53%|█████▎    | 119952/225000 [246:19:42<159:10:14,  5.45s/it]
 53%|█████▎    | 119953/225000 [246:19:48<169:50:00,  5.82s/it]
 53%|█████▎    | 119954/225000 [246:19:52<147:20:19,  5.05s/it]
 53%|█████▎    | 119955/225000 [246:20:00<177:18:08,  6.08s/it]
 53%|█████▎    | 119956/225000 [246:20:04<158:49:30,  5.44s/it]
 53%|█████▎    | 119957/225000 [246:20:08<141:52:45,  4.86s/it]
 53%|█████▎    | 119958/225000 [246:20:11<131:59:24,  4.52s/it]
 53%|█████▎    | 119959/225000 [246:20:20<168:54:10,  5.79s/it]
 53%|█████▎    | 119960/225000 [246:20:23<147:42:13,  5.06s/it]
 53%|█████▎    | 119961/225000 [246:20:34<197:44:20,  6.78s/it]
 53%|█████▎    | 119962/225000 [246:20:38<170:08:40,  5.83s/it]
 53%|█████▎    | 119963/225000 [246:20:42<152:53:57,  5.24s/it]
 53%|█████▎    | 119964/225000 [246:20:51<191:25:29,  6.56s/it]
 53%|█████▎    | 119965/225000 [246:20:56<172:04:58,  5.90s/it]
 53%|█████▎    | 119966/225000 [246:20:59<149:04:43,  5.11s/it]
 53%|█████▎    | 119967/225000 [246:21:03<139:15:54,  4.77s/it]
 53%|█████▎    | 119968/225000 [246:21:07<132:15:16,  4.53s/it]
 53%|█████▎    | 119969/225000 [246:21:14<158:41:20,  5.44s/it]
 53%|█████▎    | 119970/225000 [246:21:21<169:57:14,  5.83s/it]
 53%|█████▎    | 119971/225000 [246:21:25<150:00:35,  5.14s/it]
 53%|█████▎    | 119972/225000 [246:21:29<139:27:35,  4.78s/it]
 53%|█████▎    | 119973/225000 [246:21:39<190:44:13,  6.54s/it]
 53%|█████▎    | 119974/225000 [246:21:43<163:53:20,  5.62s/it]
 53%|█████▎    | 119975/225000 [246:21:47<148:14:03,  5.08s/it]
 53%|█████▎    | 119976/225000 [246:21:50<136:56:52,  4.69s/it]
 53%|█████▎    | 119977/225000 [246:21:59<174:53:40,  6.00s/it]
 53%|█████▎    | 119978/225000 [246:22:06<183:26:14,  6.29s/it]
 53%|█████▎    | 119979/225000 [246:22:10<155:18:15,  5.32s/it]
 53%|█████▎    | 119980/225000 [246:22:14<146:14:17,  5.01s/it]
 53%|█████▎    | 119981/225000 [246:22:20<159:33:25,  5.47s/it]
 53%|█████▎    | 119982/225000 [246:22:24<141:22:05,  4.85s/it]
 53%|█████▎    | 119983/225000 [246:22:28<132:38:06,  4.55s/it]
 53%|█████▎    | 119984/225000 [246:22:31<126:00:36,  4.32s/it]
 53%|█████▎    | 119985/225000 [246:22:38<143:22:21,  4.91s/it]
 53%|█████▎    | 119986/225000 [246:22:47<185:37:36,  6.36s/it]
 53%|█████▎    | 119987/225000 [246:22:51<164:02:34,  5.62s/it]
 53%|█████▎    | 119988/225000 [246:22:56<152:12:29,  5.22s/it]
 53%|█████▎    | 119989/225000 [246:23:00<145:19:02,  4.98s/it]
 53%|█████▎    | 119990/225000 [246:23:08<172:21:25,  5.91s/it]
 53%|█████▎    | 119991/225000 [246:23:14<171:56:02,  5.89s/it]
 53%|█████▎    | 119992/225000 [246:23:19<164:23:56,  5.64s/it]
 53%|█████▎    | 119993/225000 [246:23:25<163:48:05,  5.62s/it]
 53%|█████▎    | 119994/225000 [246:23:34<193:44:04,  6.64s/it]
 53%|█████▎    | 119995/225000 [246:23:37<165:37:53,  5.68s/it]
 53%|█████▎    | 119996/225000 [246:23:45<190:10:26,  6.52s/it]
 53%|█████▎    | 119997/225000 [246:23:49<166:07:42,  5.70s/it]
 53%|█████▎    | 119998/225000 [246:23:54<155:24:08,  5.33s/it]
 53%|█████▎    | 119999/225000 [246:24:03<186:57:39,  6.41s/it]
 53%|█████▎    | 120000/225000 [246:24:09<188:51:41,  6.48s/it]
                                                               
{'loss': '0.8583', 'grad_norm': '1.201', 'learning_rate': '2.411e-05', 'epoch': '0.5333', 'num_input_tokens_seen': 410784080, 'train_runtime': '8.87e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 120000/225000 [246:24:09<188:51:41,  6.48s/it]
 53%|█████▎    | 120001/225000 [246:24:13<164:32:53,  5.64s/it]
 53%|█████▎    | 120002/225000 [246:24:17<149:41:28,  5.13s/it]
 53%|█████▎    | 120003/225000 [246:24:21<141:52:58,  4.86s/it]
 53%|█████▎    | 120004/225000 [246:24:26<143:04:48,  4.91s/it]
 53%|█████▎    | 120005/225000 [246:24:31<142:08:36,  4.87s/it]
 53%|█████▎    | 120006/225000 [246:24:39<165:29:25,  5.67s/it]
 53%|█████▎    | 120007/225000 [246:24:46<180:00:22,  6.17s/it]
 53%|█████▎    | 120008/225000 [246:24:53<192:38:38,  6.61s/it]
 53%|█████▎    | 120009/225000 [246:25:01<197:17:48,  6.77s/it]
 53%|█████▎    | 120010/225000 [246:25:05<180:45:35,  6.20s/it]
 53%|█████▎    | 120011/225000 [246:25:10<170:25:59,  5.84s/it]
 53%|█████▎    | 120012/225000 [246:25:15<159:01:38,  5.45s/it]
 53%|█████▎    | 120013/225000 [246:25:20<151:24:31,  5.19s/it]
 53%|█████▎    | 120014/225000 [246:25:24<143:31:21,  4.92s/it]
 53%|█████▎    | 120015/225000 [246:25:29<141:14:01,  4.84s/it]
 53%|█████▎    | 120016/225000 [246:25:34<142:56:46,  4.90s/it]
 53%|█████▎    | 120017/225000 [246:25:39<144:22:13,  4.95s/it]
 53%|█████▎    | 120018/225000 [246:25:44<144:59:01,  4.97s/it]
 53%|█████▎    | 120019/225000 [246:25:48<143:19:29,  4.91s/it]
 53%|█████▎    | 120020/225000 [246:26:01<206:05:25,  7.07s/it]
 53%|█████▎    | 120021/225000 [246:26:08<206:29:45,  7.08s/it]
 53%|█████▎    | 120022/225000 [246:26:12<185:52:54,  6.37s/it]
 53%|█████▎    | 120023/225000 [246:26:18<177:34:20,  6.09s/it]
 53%|█████▎    | 120024/225000 [246:26:23<171:12:57,  5.87s/it]
 53%|█████▎    | 120025/225000 [246:26:29<169:16:17,  5.80s/it]
 53%|█████▎    | 120026/225000 [246:26:34<161:17:14,  5.53s/it]
 53%|█████▎    | 120027/225000 [246:26:39<158:36:08,  5.44s/it]
 53%|█████▎    | 120028/225000 [246:26:44<154:15:10,  5.29s/it]
 53%|█████▎    | 120029/225000 [246:26:49<155:25:57,  5.33s/it]
 53%|█████▎    | 120030/225000 [246:26:55<157:32:42,  5.40s/it]
 53%|█████▎    | 120031/225000 [246:27:00<158:42:08,  5.44s/it]
 53%|█████▎    | 120032/225000 [246:27:10<190:34:52,  6.54s/it]
 53%|█████▎    | 120033/225000 [246:27:18<210:35:59,  7.22s/it]
 53%|█████▎    | 120034/225000 [246:27:24<192:36:18,  6.61s/it]
 53%|█████▎    | 120035/225000 [246:27:29<183:53:14,  6.31s/it]
 53%|█████▎    | 120036/225000 [246:27:34<174:40:47,  5.99s/it]
 53%|█████▎    | 120037/225000 [246:27:39<163:03:35,  5.59s/it]
 53%|█████▎    | 120038/225000 [246:27:44<153:57:57,  5.28s/it]
 53%|█████▎    | 120039/225000 [246:27:48<145:49:49,  5.00s/it]
 53%|█████▎    | 120040/225000 [246:27:52<141:50:13,  4.86s/it]
 53%|█████▎    | 120041/225000 [246:27:57<141:33:07,  4.86s/it]
 53%|█████▎    | 120042/225000 [246:28:05<168:22:17,  5.78s/it]
 53%|█████▎    | 120043/225000 [246:28:15<205:02:32,  7.03s/it]
 53%|█████▎    | 120044/225000 [246:28:19<172:49:50,  5.93s/it]
 53%|█████▎    | 120045/225000 [246:28:23<156:10:39,  5.36s/it]
 53%|█████▎    | 120046/225000 [246:28:27<146:50:17,  5.04s/it]
 53%|█████▎    | 120047/225000 [246:28:42<231:03:55,  7.93s/it]
 53%|█████▎    | 120048/225000 [246:28:45<188:52:36,  6.48s/it]
 53%|█████▎    | 120049/225000 [246:28:49<174:12:46,  5.98s/it]
 53%|█████▎    | 120050/225000 [246:28:54<164:43:14,  5.65s/it]
                                                               
{'loss': '0.8646', 'grad_norm': '0.9037', 'learning_rate': '2.409e-05', 'epoch': '0.5336', 'num_input_tokens_seen': 410910024, 'train_runtime': '8.873e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 120050/225000 [246:28:54<164:43:14,  5.65s/it]
 53%|█████▎    | 120051/225000 [246:28:59<154:28:30,  5.30s/it]
 53%|█████▎    | 120052/225000 [246:29:03<148:20:04,  5.09s/it]
 53%|█████▎    | 120053/225000 [246:29:08<146:57:42,  5.04s/it]
 53%|█████▎    | 120054/225000 [246:29:20<203:33:35,  6.98s/it]
 53%|█████▎    | 120055/225000 [246:29:25<185:23:28,  6.36s/it]
 53%|█████▎    | 120056/225000 [246:29:29<164:08:51,  5.63s/it]
 53%|█████▎    | 120057/225000 [246:29:37<184:17:53,  6.32s/it]
 53%|█████▎    | 120058/225000 [246:29:43<186:02:05,  6.38s/it]
 53%|█████▎    | 120059/225000 [246:29:47<161:33:22,  5.54s/it]
 53%|█████▎    | 120060/225000 [246:29:50<142:03:13,  4.87s/it]
 53%|█████▎    | 120061/225000 [246:29:55<143:50:20,  4.93s/it]
 53%|█████▎    | 120062/225000 [246:30:00<141:15:52,  4.85s/it]
 53%|█████▎    | 120063/225000 [246:30:05<142:41:24,  4.90s/it]
 53%|█████▎    | 120064/225000 [246:30:11<151:32:01,  5.20s/it]
 53%|█████▎    | 120065/225000 [246:30:16<153:25:36,  5.26s/it]
 53%|█████▎    | 120066/225000 [246:30:21<154:27:15,  5.30s/it]
 53%|█████▎    | 120067/225000 [246:30:28<163:49:12,  5.62s/it]
 53%|█████▎    | 120068/225000 [246:30:39<216:24:11,  7.42s/it]
 53%|█████▎    | 120069/225000 [246:30:44<191:35:19,  6.57s/it]
 53%|█████▎    | 120070/225000 [246:30:48<170:06:17,  5.84s/it]
 53%|█████▎    | 120071/225000 [246:30:52<155:31:43,  5.34s/it]
 53%|█████▎    | 120072/225000 [246:30:56<143:21:47,  4.92s/it]
 53%|█████▎    | 120073/225000 [246:31:08<206:57:04,  7.10s/it]
 53%|█████▎    | 120074/225000 [246:31:15<200:22:38,  6.87s/it]
 53%|█████▎    | 120075/225000 [246:31:19<173:50:33,  5.96s/it]
 53%|█████▎    | 120076/225000 [246:31:22<152:53:34,  5.25s/it]
 53%|█████▎    | 120077/225000 [246:31:26<142:16:00,  4.88s/it]
 53%|█████▎    | 120078/225000 [246:31:36<186:24:39,  6.40s/it]
 53%|█████▎    | 120079/225000 [246:31:43<189:30:36,  6.50s/it]
 53%|█████▎    | 120080/225000 [246:31:47<170:41:13,  5.86s/it]
 53%|█████▎    | 120081/225000 [246:31:52<158:08:11,  5.43s/it]
 53%|█████▎    | 120082/225000 [246:31:56<149:11:29,  5.12s/it]
 53%|█████▎    | 120083/225000 [246:32:04<171:24:09,  5.88s/it]
 53%|█████▎    | 120084/225000 [246:32:12<194:28:48,  6.67s/it]
 53%|█████▎    | 120085/225000 [246:32:16<168:01:32,  5.77s/it]
 53%|█████▎    | 120086/225000 [246:32:21<161:32:43,  5.54s/it]
 53%|█████▎    | 120087/225000 [246:32:26<161:12:28,  5.53s/it]
 53%|█████▎    | 120088/225000 [246:32:32<161:33:41,  5.54s/it]
 53%|█████▎    | 120089/225000 [246:32:37<156:34:38,  5.37s/it]
 53%|█████▎    | 120090/225000 [246:32:42<149:37:19,  5.13s/it]
 53%|█████▎    | 120091/225000 [246:32:47<150:03:41,  5.15s/it]
 53%|█████▎    | 120092/225000 [246:32:52<150:46:04,  5.17s/it]
 53%|█████▎    | 120093/225000 [246:32:57<152:04:05,  5.22s/it]
 53%|█████▎    | 120094/225000 [246:33:03<153:57:39,  5.28s/it]
 53%|█████▎    | 120095/225000 [246:33:08<156:53:41,  5.38s/it]
 53%|█████▎    | 120096/225000 [246:33:14<155:03:55,  5.32s/it]
 53%|█████▎    | 120097/225000 [246:33:24<203:55:58,  7.00s/it]
 53%|█████▎    | 120098/225000 [246:33:34<226:19:18,  7.77s/it]
 53%|█████▎    | 120099/225000 [246:33:37<188:31:33,  6.47s/it]
 53%|█████▎    | 120100/225000 [246:33:45<200:12:18,  6.87s/it]
                                                               
{'loss': '0.8633', 'grad_norm': '1.111', 'learning_rate': '2.407e-05', 'epoch': '0.5338', 'num_input_tokens_seen': 411036272, 'train_runtime': '8.876e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 120100/225000 [246:33:45<200:12:18,  6.87s/it]
 53%|█████▎    | 120101/225000 [246:33:49<174:00:14,  5.97s/it]
 53%|█████▎    | 120102/225000 [246:33:53<159:30:41,  5.47s/it]
 53%|█████▎    | 120103/225000 [246:33:57<146:06:20,  5.01s/it]
 53%|█████▎    | 120104/225000 [246:34:02<138:01:56,  4.74s/it]
 53%|█████▎    | 120105/225000 [246:34:07<145:31:09,  4.99s/it]
 53%|█████▎    | 120106/225000 [246:34:19<205:21:28,  7.05s/it]
 53%|█████▎    | 120107/225000 [246:34:24<185:59:26,  6.38s/it]
 53%|█████▎    | 120108/225000 [246:34:31<190:25:11,  6.54s/it]
 53%|█████▎    | 120109/225000 [246:34:35<173:31:04,  5.96s/it]
 53%|█████▎    | 120110/225000 [246:34:39<158:02:42,  5.42s/it]
 53%|█████▎    | 120111/225000 [246:34:44<152:33:54,  5.24s/it]
 53%|█████▎    | 120112/225000 [246:34:49<147:53:42,  5.08s/it]
 53%|█████▎    | 120113/225000 [246:34:54<150:30:22,  5.17s/it]
 53%|█████▎    | 120114/225000 [246:35:06<203:13:20,  6.98s/it]
 53%|█████▎    | 120115/225000 [246:35:09<173:41:04,  5.96s/it]
 53%|█████▎    | 120116/225000 [246:35:13<159:38:40,  5.48s/it]
 53%|█████▎    | 120117/225000 [246:35:18<151:21:21,  5.20s/it]
 53%|█████▎    | 120118/225000 [246:35:23<146:31:55,  5.03s/it]
 53%|█████▎    | 120119/225000 [246:35:28<146:22:45,  5.02s/it]
 53%|█████▎    | 120120/225000 [246:35:33<147:04:07,  5.05s/it]
 53%|█████▎    | 120121/225000 [246:35:46<214:07:25,  7.35s/it]
 53%|█████▎    | 120122/225000 [246:35:52<204:08:47,  7.01s/it]
 53%|█████▎    | 120123/225000 [246:35:55<173:47:25,  5.97s/it]
 53%|█████▎    | 120124/225000 [246:36:04<198:42:36,  6.82s/it]
 53%|█████▎    | 120125/225000 [246:36:08<172:58:30,  5.94s/it]
 53%|█████▎    | 120126/225000 [246:36:15<179:48:08,  6.17s/it]
 53%|█████▎    | 120127/225000 [246:36:18<159:18:40,  5.47s/it]
 53%|█████▎    | 120128/225000 [246:36:22<144:34:43,  4.96s/it]
 53%|█████▎    | 120129/225000 [246:36:32<183:06:54,  6.29s/it]
 53%|█████▎    | 120130/225000 [246:36:39<196:19:56,  6.74s/it]
 53%|█████▎    | 120131/225000 [246:36:44<176:28:11,  6.06s/it]
 53%|█████▎    | 120132/225000 [246:36:49<165:30:42,  5.68s/it]
 53%|█████▎    | 120133/225000 [246:36:53<150:52:43,  5.18s/it]
 53%|█████▎    | 120134/225000 [246:37:02<188:50:45,  6.48s/it]
 53%|█████▎    | 120135/225000 [246:37:08<182:49:41,  6.28s/it]
 53%|█████▎    | 120136/225000 [246:37:12<166:40:34,  5.72s/it]
 53%|█████▎    | 120137/225000 [246:37:20<182:11:00,  6.25s/it]
 53%|█████▎    | 120138/225000 [246:37:24<161:10:52,  5.53s/it]
 53%|█████▎    | 120139/225000 [246:37:32<182:16:11,  6.26s/it]
 53%|█████▎    | 120140/225000 [246:37:36<160:27:47,  5.51s/it]
 53%|█████▎    | 120141/225000 [246:37:40<147:26:27,  5.06s/it]
 53%|█████▎    | 120142/225000 [246:37:47<166:16:33,  5.71s/it]
 53%|█████▎    | 120143/225000 [246:37:53<168:54:03,  5.80s/it]
 53%|█████▎    | 120144/225000 [246:37:58<160:06:17,  5.50s/it]
 53%|█████▎    | 120145/225000 [246:38:03<159:06:49,  5.46s/it]
 53%|█████▎    | 120146/225000 [246:38:08<156:09:46,  5.36s/it]
 53%|█████▎    | 120147/225000 [246:38:14<157:39:33,  5.41s/it]
 53%|█████▎    | 120148/225000 [246:38:19<156:36:13,  5.38s/it]
 53%|█████▎    | 120149/225000 [246:38:24<152:33:38,  5.24s/it]
 53%|█████▎    | 120150/225000 [246:38:29<149:56:25,  5.15s/it]
                                                               
{'loss': '0.8587', 'grad_norm': '1.157', 'learning_rate': '2.405e-05', 'epoch': '0.534', 'num_input_tokens_seen': 411161944, 'train_runtime': '8.879e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 120150/225000 [246:38:29<149:56:25,  5.15s/it]
 53%|█████▎    | 120151/225000 [246:38:34<149:58:25,  5.15s/it]
 53%|█████▎    | 120152/225000 [246:38:40<154:37:10,  5.31s/it]
 53%|█████▎    | 120153/225000 [246:38:45<153:44:14,  5.28s/it]
 53%|█████▎    | 120154/225000 [246:38:50<155:48:55,  5.35s/it]
 53%|█████▎    | 120155/225000 [246:38:56<156:16:01,  5.37s/it]
 53%|█████▎    | 120156/225000 [246:39:07<206:43:52,  7.10s/it]
 53%|█████▎    | 120157/225000 [246:39:18<239:02:06,  8.21s/it]
 53%|█████▎    | 120158/225000 [246:39:21<196:07:38,  6.73s/it]
 53%|█████▎    | 120159/225000 [246:39:30<212:59:24,  7.31s/it]
 53%|█████▎    | 120160/225000 [246:39:33<181:43:33,  6.24s/it]
 53%|█████▎    | 120161/225000 [246:39:37<161:24:10,  5.54s/it]
 53%|█████▎    | 120162/225000 [246:39:48<205:54:51,  7.07s/it]
 53%|█████▎    | 120163/225000 [246:39:51<173:06:44,  5.94s/it]
 53%|█████▎    | 120164/225000 [246:39:55<152:07:29,  5.22s/it]
 53%|█████▎    | 120165/225000 [246:39:59<147:00:37,  5.05s/it]
 53%|█████▎    | 120166/225000 [246:40:04<146:10:14,  5.02s/it]
 53%|█████▎    | 120167/225000 [246:40:16<206:06:09,  7.08s/it]
 53%|█████▎    | 120168/225000 [246:40:21<182:40:56,  6.27s/it]
 53%|█████▎    | 120169/225000 [246:40:25<163:30:35,  5.62s/it]
 53%|█████▎    | 120170/225000 [246:40:32<181:32:01,  6.23s/it]
 53%|█████▎    | 120171/225000 [246:40:39<184:48:26,  6.35s/it]
 53%|█████▎    | 120172/225000 [246:40:43<164:28:05,  5.65s/it]
 53%|█████▎    | 120173/225000 [246:40:49<165:36:32,  5.69s/it]
 53%|█████▎    | 120174/225000 [246:40:58<195:15:33,  6.71s/it]
 53%|█████▎    | 120175/225000 [246:41:01<165:39:31,  5.69s/it]
 53%|█████▎    | 120176/225000 [246:41:06<162:05:38,  5.57s/it]
 53%|█████▎    | 120177/225000 [246:41:15<185:43:12,  6.38s/it]
 53%|█████▎    | 120178/225000 [246:41:20<171:56:17,  5.91s/it]
 53%|█████▎    | 120179/225000 [246:41:24<161:35:49,  5.55s/it]
 53%|█████▎    | 120180/225000 [246:41:29<152:18:38,  5.23s/it]
 53%|█████▎    | 120181/225000 [246:41:33<147:20:42,  5.06s/it]
 53%|█████▎    | 120182/225000 [246:41:38<142:10:32,  4.88s/it]
 53%|█████▎    | 120183/225000 [246:41:50<206:34:17,  7.09s/it]
 53%|█████▎    | 120184/225000 [246:41:55<188:22:52,  6.47s/it]
 53%|█████▎    | 120185/225000 [246:41:59<165:01:40,  5.67s/it]
 53%|█████▎    | 120186/225000 [246:42:03<146:57:26,  5.05s/it]
 53%|█████▎    | 120187/225000 [246:42:07<139:11:07,  4.78s/it]
 53%|█████▎    | 120188/225000 [246:42:17<184:47:05,  6.35s/it]
 53%|█████▎    | 120189/225000 [246:42:21<169:03:28,  5.81s/it]
 53%|█████▎    | 120190/225000 [246:42:25<154:58:11,  5.32s/it]
 53%|█████▎    | 120191/225000 [246:42:30<147:48:37,  5.08s/it]
 53%|█████▎    | 120192/225000 [246:42:34<141:21:03,  4.86s/it]
 53%|█████▎    | 120193/225000 [246:42:45<192:14:02,  6.60s/it]
 53%|█████▎    | 120194/225000 [246:42:50<179:53:02,  6.18s/it]
 53%|█████▎    | 120195/225000 [246:42:54<159:10:49,  5.47s/it]
 53%|█████▎    | 120196/225000 [246:43:00<166:22:46,  5.72s/it]
 53%|█████▎    | 120197/225000 [246:43:08<186:45:23,  6.42s/it]
 53%|█████▎    | 120198/225000 [246:43:13<171:35:22,  5.89s/it]
 53%|█████▎    | 120199/225000 [246:43:18<164:33:07,  5.65s/it]
 53%|█████▎    | 120200/225000 [246:43:22<147:44:05,  5.07s/it]
                                                               
{'loss': '0.8721', 'grad_norm': '1.21', 'learning_rate': '2.403e-05', 'epoch': '0.5342', 'num_input_tokens_seen': 411288776, 'train_runtime': '8.882e+05', 'train_tokens_per_second': '463.1'}

 53%|█████▎    | 120200/225000 [246:43:22<147:44:05,  5.07s/it]
 53%|█████▎    | 120201/225000 [246:43:32<194:16:59,  6.67s/it]
 53%|█████▎    | 120202/225000 [246:43:36<165:46:17,  5.69s/it]
 53%|█████▎    | 120203/225000 [246:43:45<197:32:33,  6.79s/it]
 53%|█████▎    | 120204/225000 [246:43:48<166:53:09,  5.73s/it]
 53%|█████▎    | 120205/225000 [246:43:56<186:24:42,  6.40s/it]
 53%|█████▎    | 120206/225000 [246:44:00<165:42:03,  5.69s/it]
 53%|█████▎    | 120207/225000 [246:44:05<157:36:51,  5.41s/it]
 53%|█████▎    | 120208/225000 [246:44:10<151:59:28,  5.22s/it]
 53%|█████▎    | 120209/225000 [246:44:15<150:04:01,  5.16s/it]
 53%|█████▎    | 120210/225000 [246:44:20<151:07:54,  5.19s/it]
 53%|█████▎    | 120211/225000 [246:44:25<148:39:02,  5.11s/it]
 53%|█████▎    | 120212/225000 [246:44:30<148:56:39,  5.12s/it]
 53%|█████▎    | 120213/225000 [246:44:35<148:43:54,  5.11s/it]
 53%|█████▎    | 120214/225000 [246:44:41<153:15:49,  5.27s/it]
 53%|█████▎    | 120215/225000 [246:44:46<154:01:53,  5.29s/it]
 53%|█████▎    | 120216/225000 [246:44:51<154:06:20,  5.29s/it]
 53%|█████▎    | 120217/225000 [246:44:57<152:39:09,  5.24s/it]
 53%|█████▎    | 120218/225000 [246:45:07<199:46:06,  6.86s/it]
 53%|█████▎    | 120219/225000 [246:45:16<219:15:50,  7.53s/it]
 53%|█████▎    | 120220/225000 [246:45:20<183:59:31,  6.32s/it]
 53%|█████▎    | 120221/225000 [246:45:24<164:59:59,  5.67s/it]
 53%|█████▎    | 120222/225000 [246:45:30<171:51:08,  5.90s/it]
 53%|█████▎    | 120223/225000 [246:45:41<210:41:34,  7.24s/it]
 53%|█████▎    | 120224/225000 [246:45:44<179:00:59,  6.15s/it]
 53%|█████▎    | 120225/225000 [246:45:48<155:15:30,  5.33s/it]
 53%|█████▎    | 120226/225000 [246:45:52<147:02:38,  5.05s/it]
 53%|█████▎    | 120227/225000 [246:45:58<150:47:29,  5.18s/it]
 53%|█████▎    | 120228/225000 [246:46:02<142:52:10,  4.91s/it]
 53%|█████▎    | 120229/225000 [246:46:07<144:56:06,  4.98s/it]
 53%|█████▎    | 120230/225000 [246:46:12<146:13:33,  5.02s/it]
 53%|█████▎    | 120231/225000 [246:46:19<162:14:52,  5.58s/it]
 53%|█████▎    | 120232/225000 [246:46:30<211:05:15,  7.25s/it]
 53%|█████▎    | 120233/225000 [246:46:35<188:28:24,  6.48s/it]
 53%|█████▎    | 120234/225000 [246:46:39<171:44:45,  5.90s/it]
 53%|█████▎    | 120235/225000 [246:46:43<152:35:06,  5.24s/it]
 53%|█████▎    | 120236/225000 [246:46:53<189:17:00,  6.50s/it]
 53%|█████▎    | 120237/225000 [246:46:57<174:10:59,  5.99s/it]
 53%|█████▎    | 120238/225000 [246:47:02<158:47:53,  5.46s/it]
 53%|█████▎    | 120239/225000 [246:47:06<148:27:27,  5.10s/it]
 53%|█████▎    | 120240/225000 [246:47:18<208:38:32,  7.17s/it]
 53%|█████▎    | 120241/225000 [246:47:21<176:04:02,  6.05s/it]
 53%|█████▎    | 120242/225000 [246:47:28<180:31:09,  6.20s/it]
 53%|█████▎    | 120243/225000 [246:47:33<168:58:38,  5.81s/it]
 53%|█████▎    | 120244/225000 [246:47:38<159:47:34,  5.49s/it]
 53%|█████▎    | 120245/225000 [246:47:42<152:55:12,  5.26s/it]
 53%|█████▎    | 120246/225000 [246:47:48<156:01:00,  5.36s/it]
 53%|█████▎    | 120247/225000 [246:47:58<201:06:01,  6.91s/it]
 53%|█████▎    | 120248/225000 [246:48:04<187:10:46,  6.43s/it]
 53%|█████▎    | 120249/225000 [246:48:08<165:32:51,  5.69s/it]
 53%|█████▎    | 120250/225000 [246:48:12<151:13:05,  5.20s/it]
                                                               
{'loss': '0.8631', 'grad_norm': '1.046', 'learning_rate': '2.402e-05', 'epoch': '0.5344', 'num_input_tokens_seen': 411414824, 'train_runtime': '8.885e+05', 'train_tokens_per_second': '463'}

 53%|█████▎    | 120250/225000 [246:48:12<151:13:05,  5.20s/it]
 53%|█████▎    | 120251/225000 [246:48:23<201:09:59,  6.91s/it]
 53%|█████▎    | 120252/225000 [246:48:26<171:09:31,  5.88s/it]
 53%|█████▎    | 120253/225000 [246:48:33<183:50:14,  6.32s/it]
 53%|█████▎    | 120254/225000 [246:48:37<161:53:36,  5.56s/it]
 53%|█████▎    | 120255/225000 [246:48:45<179:19:13,  6.16s/it]
 53%|█████▎    | 120256/225000 [246:48:51<177:10:14,  6.09s/it]
 53%|█████▎    | 120257/225000 [246:48:55<164:39:20,  5.66s/it]
 53%|█████▎    | 120258/225000 [246:49:00<155:14:16,  5.34s/it]
 53%|█████▎    | 120259/225000 [246:49:05<153:20:26,  5.27s/it]
 53%|█████▎    | 120260/225000 [246:49:10<147:46:00,  5.08s/it]
 53%|█████▎    | 120261/225000 [246:49:22<209:55:11,  7.22s/it]
 53%|█████▎    | 120262/225000 [246:49:26<182:40:22,  6.28s/it]
 53%|█████▎    | 120263/225000 [246:49:30<162:59:03,  5.60s/it]
 53%|█████▎    | 120264/225000 [246:49:34<152:39:47,  5.25s/it]
 53%|█████▎    | 120265/225000 [246:49:42<176:59:27,  6.08s/it]
 53%|█████▎    | 120266/225000 [246:49:49<182:18:12,  6.27s/it]
 53%|█████▎    | 120267/225000 [246:49:53<162:33:37,  5.59s/it]
 53%|█████▎    | 120268/225000 [246:49:57<148:16:35,  5.10s/it]
 53%|█████▎    | 120269/225000 [246:50:01<140:16:05,  4.82s/it]
 53%|█████▎    | 120270/225000 [246:50:06<136:40:11,  4.70s/it]
 53%|█████▎    | 120271/225000 [246:50:16<185:08:54,  6.36s/it]
 53%|█████▎    | 120272/225000 [246:50:23<187:29:28,  6.44s/it]
 53%|█████▎    | 120273/225000 [246:50:30<197:02:14,  6.77s/it]
 53%|█████▎    | 120274/225000 [246:50:35<179:06:58,  6.16s/it]
 53%|█████▎    | 120275/225000 [246:50:42<190:30:47,  6.55s/it]
 53%|█████▎    | 120276/225000 [246:50:46<165:15:32,  5.68s/it]
 53%|█████▎    | 120277/225000 [246:50:55<193:06:57,  6.64s/it]
 53%|█████▎    | 120278/225000 [246:50:58<166:54:01,  5.74s/it]
 53%|█████▎    | 120279/225000 [246:51:03<155:50:38,  5.36s/it]
 53%|█████▎    | 120280/225000 [246:51:07<144:12:23,  4.96s/it]
 53%|█████▎    | 120281/225000 [246:51:11<139:22:54,  4.79s/it]
 53%|█████▎    | 120282/225000 [246:51:16<140:44:53,  4.84s/it]
 53%|█████▎    | 120283/225000 [246:51:28<200:20:20,  6.89s/it]
 53%|█████▎    | 120284/225000 [246:51:35<204:32:36,  7.03s/it]
 53%|█████▎    | 120285/225000 [246:51:39<179:08:08,  6.16s/it]
 53%|█████▎    | 120286/225000 [246:51:47<190:10:30,  6.54s/it]
 53%|█████▎    | 120287/225000 [246:51:54<193:57:46,  6.67s/it]
 53%|█████▎    | 120288/225000 [246:51:58<175:59:11,  6.05s/it]
 53%|█████▎    | 120289/225000 [246:52:03<161:18:23,  5.55s/it]
 53%|█████▎    | 120290/225000 [246:52:07<148:47:23,  5.12s/it]
 53%|█████▎    | 120291/225000 [246:52:12<144:46:33,  4.98s/it]
 53%|█████▎    | 120292/225000 [246:52:16<141:06:30,  4.85s/it]
 53%|█████▎    | 120293/225000 [246:52:24<168:39:08,  5.80s/it]
 53%|█████▎    | 120294/225000 [246:52:38<235:55:57,  8.11s/it]
 53%|█████▎    | 120295/225000 [246:52:43<213:43:44,  7.35s/it]
 53%|█████▎    | 120296/225000 [246:52:48<192:32:53,  6.62s/it]
 53%|█████▎    | 120297/225000 [246:52:55<191:08:14,  6.57s/it]
 53%|█████▎    | 120298/225000 [246:53:01<193:09:02,  6.64s/it]
 53%|█████▎    | 120299/225000 [246:53:05<166:54:04,  5.74s/it]
 53%|█████▎    | 120300/225000 [246:53:12<179:34:22,  6.17s/it]
                                                               
{'loss': '0.8537', 'grad_norm': '1.073', 'learning_rate': '2.4e-05', 'epoch': '0.5347', 'num_input_tokens_seen': 411542632, 'train_runtime': '8.888e+05', 'train_tokens_per_second': '463'}

 53%|█████▎    | 120300/225000 [246:53:12<179:34:22,  6.17s/it]
 53%|█████▎    | 120301/225000 [246:53:19<185:04:59,  6.36s/it]
 53%|█████▎    | 120302/225000 [246:53:24<172:49:11,  5.94s/it]
 53%|█████▎    | 120303/225000 [246:53:29<167:28:55,  5.76s/it]
 53%|█████▎    | 120304/225000 [246:53:34<161:49:39,  5.56s/it]
 53%|█████▎    | 120305/225000 [246:53:39<149:36:07,  5.14s/it]
 53%|█████▎    | 120306/225000 [246:53:44<147:11:10,  5.06s/it]
 53%|█████▎    | 120307/225000 [246:53:49<147:06:04,  5.06s/it]
 53%|█████▎    | 120308/225000 [246:53:53<145:07:43,  4.99s/it]
 53%|█████▎    | 120309/225000 [246:53:58<144:01:51,  4.95s/it]
 53%|█████▎    | 120310/225000 [246:54:03<143:56:19,  4.95s/it]
 53%|█████▎    | 120311/225000 [246:54:09<147:22:30,  5.07s/it]
 53%|█████▎    | 120312/225000 [246:54:15<158:23:41,  5.45s/it]
 53%|█████▎    | 120313/225000 [246:54:27<220:38:08,  7.59s/it]
 53%|█████▎    | 120314/225000 [246:54:30<180:09:18,  6.20s/it]
 53%|█████▎    | 120315/225000 [246:54:38<194:32:36,  6.69s/it]
 53%|█████▎    | 120316/225000 [246:54:43<174:51:17,  6.01s/it]
 53%|█████▎    | 120317/225000 [246:54:47<161:45:37,  5.56s/it]
 53%|█████▎    | 120318/225000 [246:54:53<162:28:35,  5.59s/it]
 53%|█████▎    | 120319/225000 [246:55:01<183:19:42,  6.30s/it]
 53%|█████▎    | 120320/225000 [246:55:05<167:12:06,  5.75s/it]
 53%|█████▎    | 120321/225000 [246:55:11<163:34:10,  5.63s/it]
 53%|█████▎    | 120322/225000 [246:55:16<158:58:40,  5.47s/it]
 53%|█████▎    | 120323/225000 [246:55:21<157:56:37,  5.43s/it]
 53%|█████▎    | 120324/225000 [246:55:26<153:09:56,  5.27s/it]
 53%|█████▎    | 120325/225000 [246:55:31<154:10:05,  5.30s/it]
 53%|█████▎    | 120326/225000 [246:55:36<152:00:27,  5.23s/it]
 53%|█████▎    | 120327/225000 [246:55:41<147:09:35,  5.06s/it]
 53%|█████▎    | 120328/225000 [246:55:46<145:28:48,  5.00s/it]
 53%|█████▎    | 120329/225000 [246:55:51<146:40:37,  5.04s/it]
 53%|█████▎    | 120330/225000 [246:55:56<147:18:26,  5.07s/it]
 53%|█████▎    | 120331/225000 [246:56:01<141:07:26,  4.85s/it]
 53%|█████▎    | 120332/225000 [246:56:11<192:53:06,  6.63s/it]
 53%|█████▎    | 120333/225000 [246:56:21<222:11:52,  7.64s/it]
 53%|█████▎    | 120334/225000 [246:56:28<217:16:06,  7.47s/it]
 53%|█████▎    | 120335/225000 [246:56:33<191:41:35,  6.59s/it]
 53%|█████▎    | 120336/225000 [246:56:37<171:40:00,  5.90s/it]
 53%|█████▎    | 120337/225000 [246:56:42<158:32:00,  5.45s/it]
 53%|█████▎    | 120338/225000 [246:56:47<158:39:14,  5.46s/it]
 53%|█████▎    | 120339/225000 [246:56:59<219:02:19,  7.53s/it]
 53%|█████▎    | 120340/225000 [246:57:05<199:48:25,  6.87s/it]
 53%|█████▎    | 120341/225000 [246:57:09<173:12:11,  5.96s/it]
 53%|█████▎    | 120342/225000 [246:57:14<164:43:20,  5.67s/it]
 53%|█████▎    | 120343/225000 [246:57:23<201:17:56,  6.92s/it]
 53%|█████▎    | 120344/225000 [246:57:27<168:04:04,  5.78s/it]
 53%|█████▎    | 120345/225000 [246:57:34<179:54:05,  6.19s/it]
 53%|█████▎    | 120346/225000 [246:57:39<172:54:07,  5.95s/it]
 53%|█████▎    | 120347/225000 [246:57:47<188:49:06,  6.50s/it]
 53%|█████▎    | 120348/225000 [246:57:53<183:47:36,  6.32s/it]
 53%|█████▎    | 120349/225000 [246:57:57<169:15:17,  5.82s/it]
 53%|█████▎    | 120350/225000 [246:58:01<149:03:22,  5.13s/it]
                                                               
{'loss': '0.8616', 'grad_norm': '1.12', 'learning_rate': '2.398e-05', 'epoch': '0.5349', 'num_input_tokens_seen': 411668688, 'train_runtime': '8.891e+05', 'train_tokens_per_second': '463'}

 53%|█████▎    | 120350/225000 [246:58:01<149:03:22,  5.13s/it]
 53%|█████▎    | 120351/225000 [246:58:10<184:06:15,  6.33s/it]
 53%|█████▎    | 120352/225000 [246:58:20<212:39:28,  7.32s/it]
 53%|█████▎    | 120353/225000 [246:58:23<179:32:45,  6.18s/it]
 53%|█████▎    | 120354/225000 [246:58:29<172:03:45,  5.92s/it]
 53%|█████▎    | 120355/225000 [246:58:35<173:22:53,  5.96s/it]
 53%|█████▎    | 120356/225000 [246:58:40<169:53:07,  5.84s/it]
 53%|█████▎    | 120357/225000 [246:58:45<156:35:32,  5.39s/it]
 53%|█████▎    | 120358/225000 [246:58:49<145:40:44,  5.01s/it]
 53%|█████▎    | 120359/225000 [246:58:53<140:44:34,  4.84s/it]
 53%|█████▎    | 120360/225000 [246:58:58<140:25:52,  4.83s/it]
 53%|█████▎    | 120361/225000 [246:59:03<139:35:04,  4.80s/it]
 53%|█████▎    | 120362/225000 [246:59:09<152:54:11,  5.26s/it]
 53%|█████▎    | 120363/225000 [246:59:21<208:58:27,  7.19s/it]
 53%|█████▎    | 120364/225000 [246:59:24<175:27:05,  6.04s/it]
 53%|█████▎    | 120365/225000 [246:59:28<158:20:36,  5.45s/it]
 53%|█████▎    | 120366/225000 [246:59:33<153:28:42,  5.28s/it]
 53%|█████▎    | 120367/225000 [246:59:38<152:11:21,  5.24s/it]
 53%|█████▎    | 120368/225000 [246:59:43<145:30:05,  5.01s/it]
 53%|█████▎    | 120369/225000 [246:59:48<148:52:28,  5.12s/it]
 53%|█████▎    | 120370/225000 [246:59:53<149:02:40,  5.13s/it]
 53%|█████▎    | 120371/225000 [246:59:59<155:36:29,  5.35s/it]
 53%|█████▎    | 120372/225000 [247:00:10<208:35:10,  7.18s/it]
 53%|█████▎    | 120373/225000 [247:00:17<200:13:09,  6.89s/it]
 53%|█████▎    | 120374/225000 [247:00:20<171:14:17,  5.89s/it]
 54%|█████▎    | 120375/225000 [247:00:25<160:41:24,  5.53s/it]
 54%|█████▎    | 120376/225000 [247:00:29<149:39:49,  5.15s/it]
 54%|█████▎    | 120377/225000 [247:00:33<139:25:24,  4.80s/it]
 54%|█████▎    | 120378/225000 [247:00:45<203:34:19,  7.00s/it]
 54%|█████▎    | 120379/225000 [247:00:50<187:11:25,  6.44s/it]
 54%|█████▎    | 120380/225000 [247:00:55<169:16:23,  5.82s/it]
 54%|█████▎    | 120381/225000 [247:01:05<208:11:39,  7.16s/it]
 54%|█████▎    | 120382/225000 [247:01:11<199:47:47,  6.88s/it]
 54%|█████▎    | 120383/225000 [247:01:15<173:11:53,  5.96s/it]
 54%|█████▎    | 120384/225000 [247:01:23<188:02:37,  6.47s/it]
 54%|█████▎    | 120385/225000 [247:01:27<164:23:18,  5.66s/it]
 54%|█████▎    | 120386/225000 [247:01:31<156:58:14,  5.40s/it]
 54%|█████▎    | 120387/225000 [247:01:40<188:31:33,  6.49s/it]
 54%|█████▎    | 120388/225000 [247:01:44<164:10:44,  5.65s/it]
 54%|█████▎    | 120389/225000 [247:01:50<165:54:07,  5.71s/it]
 54%|█████▎    | 120390/225000 [247:01:58<184:52:01,  6.36s/it]
 54%|█████▎    | 120391/225000 [247:02:05<189:06:43,  6.51s/it]
 54%|█████▎    | 120392/225000 [247:02:10<176:21:20,  6.07s/it]
 54%|█████▎    | 120393/225000 [247:02:13<155:12:01,  5.34s/it]
 54%|█████▎    | 120394/225000 [247:02:23<195:47:18,  6.74s/it]
 54%|█████▎    | 120395/225000 [247:02:28<175:53:07,  6.05s/it]
 54%|█████▎    | 120396/225000 [247:02:32<158:55:33,  5.47s/it]
 54%|█████▎    | 120397/225000 [247:02:36<149:16:15,  5.14s/it]
 54%|█████▎    | 120398/225000 [247:02:48<204:22:48,  7.03s/it]
 54%|█████▎    | 120399/225000 [247:02:51<168:11:46,  5.79s/it]
 54%|█████▎    | 120400/225000 [247:02:58<180:07:57,  6.20s/it]
                                                               
{'loss': '0.8534', 'grad_norm': '0.9915', 'learning_rate': '2.396e-05', 'epoch': '0.5351', 'num_input_tokens_seen': 411796464, 'train_runtime': '8.894e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120400/225000 [247:02:58<180:07:57,  6.20s/it]
 54%|█████▎    | 120401/225000 [247:03:01<157:33:27,  5.42s/it]
 54%|█████▎    | 120402/225000 [247:03:09<177:17:49,  6.10s/it]
 54%|█████▎    | 120403/225000 [247:03:16<180:49:48,  6.22s/it]
 54%|█████▎    | 120404/225000 [247:03:21<170:17:17,  5.86s/it]
 54%|█████▎    | 120405/225000 [247:03:25<160:20:43,  5.52s/it]
 54%|█████▎    | 120406/225000 [247:03:30<149:35:05,  5.15s/it]
 54%|█████▎    | 120407/225000 [247:03:34<144:52:32,  4.99s/it]
 54%|█████▎    | 120408/225000 [247:03:39<144:32:49,  4.98s/it]
 54%|█████▎    | 120409/225000 [247:03:44<140:23:32,  4.83s/it]
 54%|█████▎    | 120410/225000 [247:03:52<168:37:22,  5.80s/it]
 54%|█████▎    | 120411/225000 [247:04:02<204:37:40,  7.04s/it]
 54%|█████▎    | 120412/225000 [247:04:06<179:28:55,  6.18s/it]
 54%|█████▎    | 120413/225000 [247:04:10<162:16:24,  5.59s/it]
 54%|█████▎    | 120414/225000 [247:04:13<143:43:40,  4.95s/it]
 54%|█████▎    | 120415/225000 [247:04:24<194:45:04,  6.70s/it]
 54%|█████▎    | 120416/225000 [247:04:28<165:57:11,  5.71s/it]
 54%|█████▎    | 120417/225000 [247:04:32<151:43:17,  5.22s/it]
 54%|█████▎    | 120418/225000 [247:04:36<147:14:37,  5.07s/it]
 54%|█████▎    | 120419/225000 [247:04:47<196:31:31,  6.77s/it]
 54%|█████▎    | 120420/225000 [247:04:52<178:47:30,  6.15s/it]
 54%|█████▎    | 120421/225000 [247:04:57<167:28:01,  5.76s/it]
 54%|█████▎    | 120422/225000 [247:05:01<155:28:27,  5.35s/it]
 54%|█████▎    | 120423/225000 [247:05:10<188:46:21,  6.50s/it]
 54%|█████▎    | 120424/225000 [247:05:17<194:00:18,  6.68s/it]
 54%|█████▎    | 120425/225000 [247:05:20<161:56:06,  5.57s/it]
 54%|█████▎    | 120426/225000 [247:05:28<182:13:07,  6.27s/it]
 54%|█████▎    | 120427/225000 [247:05:32<156:02:25,  5.37s/it]
 54%|█████▎    | 120428/225000 [247:05:41<189:31:09,  6.52s/it]
 54%|█████▎    | 120429/225000 [247:05:44<162:48:17,  5.60s/it]
 54%|█████▎    | 120430/225000 [247:05:48<146:48:40,  5.05s/it]
 54%|█████▎    | 120431/225000 [247:05:56<171:29:08,  5.90s/it]
 54%|█████▎    | 120432/225000 [247:06:00<159:18:48,  5.48s/it]
 54%|█████▎    | 120433/225000 [247:06:05<150:12:15,  5.17s/it]
 54%|█████▎    | 120434/225000 [247:06:09<145:18:21,  5.00s/it]
 54%|█████▎    | 120435/225000 [247:06:15<151:16:54,  5.21s/it]
 54%|█████▎    | 120436/225000 [247:06:19<142:35:44,  4.91s/it]
 54%|█████▎    | 120437/225000 [247:06:29<186:46:19,  6.43s/it]
 54%|█████▎    | 120438/225000 [247:06:38<203:14:01,  7.00s/it]
 54%|█████▎    | 120439/225000 [247:06:41<173:05:34,  5.96s/it]
 54%|█████▎    | 120440/225000 [247:06:49<186:55:53,  6.44s/it]
 54%|█████▎    | 120441/225000 [247:06:57<198:48:21,  6.84s/it]
 54%|█████▎    | 120442/225000 [247:07:01<175:46:49,  6.05s/it]
 54%|█████▎    | 120443/225000 [247:07:05<156:17:34,  5.38s/it]
 54%|█████▎    | 120444/225000 [247:07:09<147:17:24,  5.07s/it]
 54%|█████▎    | 120445/225000 [247:07:13<135:55:46,  4.68s/it]
 54%|█████▎    | 120446/225000 [247:07:22<173:26:32,  5.97s/it]
 54%|█████▎    | 120447/225000 [247:07:30<196:32:30,  6.77s/it]
 54%|█████▎    | 120448/225000 [247:07:35<178:36:03,  6.15s/it]
 54%|█████▎    | 120449/225000 [247:07:39<162:32:40,  5.60s/it]
 54%|█████▎    | 120450/225000 [247:07:43<150:06:07,  5.17s/it]
                                                               
{'loss': '0.8576', 'grad_norm': '1.061', 'learning_rate': '2.394e-05', 'epoch': '0.5353', 'num_input_tokens_seen': 411923768, 'train_runtime': '8.897e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120450/225000 [247:07:43<150:06:07,  5.17s/it]
 54%|█████▎    | 120451/225000 [247:07:54<194:37:05,  6.70s/it]
 54%|█████▎    | 120452/225000 [247:08:00<191:02:04,  6.58s/it]
 54%|█████▎    | 120453/225000 [247:08:04<164:12:53,  5.65s/it]
 54%|█████▎    | 120454/225000 [247:08:08<150:31:39,  5.18s/it]
 54%|█████▎    | 120455/225000 [247:08:12<142:23:46,  4.90s/it]
 54%|█████▎    | 120456/225000 [247:08:17<148:22:32,  5.11s/it]
 54%|█████▎    | 120457/225000 [247:08:30<211:00:28,  7.27s/it]
 54%|█████▎    | 120458/225000 [247:08:33<179:09:22,  6.17s/it]
 54%|█████▎    | 120459/225000 [247:08:42<199:15:29,  6.86s/it]
 54%|█████▎    | 120460/225000 [247:08:46<172:26:39,  5.94s/it]
 54%|█████▎    | 120461/225000 [247:08:50<159:49:45,  5.50s/it]
 54%|█████▎    | 120462/225000 [247:08:54<145:55:47,  5.03s/it]
 54%|█████▎    | 120463/225000 [247:08:58<138:19:50,  4.76s/it]
 54%|█████▎    | 120464/225000 [247:09:02<133:33:07,  4.60s/it]
 54%|█████▎    | 120465/225000 [247:09:10<155:25:45,  5.35s/it]
 54%|█████▎    | 120466/225000 [247:09:21<204:24:22,  7.04s/it]
 54%|█████▎    | 120467/225000 [247:09:24<170:47:37,  5.88s/it]
 54%|█████▎    | 120468/225000 [247:09:32<193:38:15,  6.67s/it]
 54%|█████▎    | 120469/225000 [247:09:36<166:38:35,  5.74s/it]
 54%|█████▎    | 120470/225000 [247:09:41<165:28:56,  5.70s/it]
 54%|█████▎    | 120471/225000 [247:09:50<191:47:08,  6.61s/it]
 54%|█████▎    | 120472/225000 [247:09:57<197:05:41,  6.79s/it]
 54%|█████▎    | 120473/225000 [247:10:01<168:08:17,  5.79s/it]
 54%|█████▎    | 120474/225000 [247:10:05<153:16:40,  5.28s/it]
 54%|█████▎    | 120475/225000 [247:10:17<208:50:53,  7.19s/it]
 54%|█████▎    | 120476/225000 [247:10:20<174:15:09,  6.00s/it]
 54%|█████▎    | 120477/225000 [247:10:26<178:17:27,  6.14s/it]
 54%|█████▎    | 120478/225000 [247:10:32<172:29:22,  5.94s/it]
 54%|█████▎    | 120479/225000 [247:10:35<153:19:32,  5.28s/it]
 54%|█████▎    | 120480/225000 [247:10:39<142:12:49,  4.90s/it]
 54%|█████▎    | 120481/225000 [247:10:43<130:32:51,  4.50s/it]
 54%|█████▎    | 120482/225000 [247:10:47<126:24:23,  4.35s/it]
 54%|█████▎    | 120483/225000 [247:10:52<128:51:55,  4.44s/it]
 54%|█████▎    | 120484/225000 [247:11:02<179:28:21,  6.18s/it]
 54%|█████▎    | 120485/225000 [247:11:06<164:51:51,  5.68s/it]
 54%|█████▎    | 120486/225000 [247:11:11<156:31:22,  5.39s/it]
 54%|█████▎    | 120487/225000 [247:11:16<153:16:59,  5.28s/it]
 54%|█████▎    | 120488/225000 [247:11:21<150:02:17,  5.17s/it]
 54%|█████▎    | 120489/225000 [247:11:26<146:01:01,  5.03s/it]
 54%|█████▎    | 120490/225000 [247:11:30<140:38:56,  4.84s/it]
 54%|█████▎    | 120491/225000 [247:11:36<149:11:39,  5.14s/it]
 54%|█████▎    | 120492/225000 [247:11:49<213:30:39,  7.35s/it]
 54%|█████▎    | 120493/225000 [247:11:53<187:32:57,  6.46s/it]
 54%|█████▎    | 120494/225000 [247:11:57<169:01:31,  5.82s/it]
 54%|█████▎    | 120495/225000 [247:12:06<194:30:32,  6.70s/it]
 54%|█████▎    | 120496/225000 [247:12:10<174:38:21,  6.02s/it]
 54%|█████▎    | 120497/225000 [247:12:15<161:52:16,  5.58s/it]
 54%|█████▎    | 120498/225000 [247:12:19<151:05:18,  5.20s/it]
 54%|█████▎    | 120499/225000 [247:12:31<209:49:07,  7.23s/it]
 54%|█████▎    | 120500/225000 [247:12:36<187:31:38,  6.46s/it]
                                                               
{'loss': '0.8544', 'grad_norm': '1.012', 'learning_rate': '2.392e-05', 'epoch': '0.5356', 'num_input_tokens_seen': 412048128, 'train_runtime': '8.9e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120500/225000 [247:12:36<187:31:38,  6.46s/it]
 54%|█████▎    | 120501/225000 [247:12:42<180:58:23,  6.23s/it]
 54%|█████▎    | 120502/225000 [247:12:47<177:05:38,  6.10s/it]
 54%|█████▎    | 120503/225000 [247:12:53<171:38:28,  5.91s/it]
 54%|█████▎    | 120504/225000 [247:12:57<159:00:38,  5.48s/it]
 54%|█████▎    | 120505/225000 [247:13:02<155:58:07,  5.37s/it]
 54%|█████▎    | 120506/225000 [247:13:08<154:03:04,  5.31s/it]
 54%|█████▎    | 120507/225000 [247:13:12<148:34:41,  5.12s/it]
 54%|█████▎    | 120508/225000 [247:13:20<171:42:07,  5.92s/it]
 54%|█████▎    | 120509/225000 [247:13:25<166:43:25,  5.74s/it]
 54%|█████▎    | 120510/225000 [247:13:29<150:40:48,  5.19s/it]
 54%|█████▎    | 120511/225000 [247:13:34<149:04:12,  5.14s/it]
 54%|█████▎    | 120512/225000 [247:13:39<146:52:46,  5.06s/it]
 54%|█████▎    | 120513/225000 [247:13:44<146:10:14,  5.04s/it]
 54%|█████▎    | 120514/225000 [247:13:49<148:24:14,  5.11s/it]
 54%|█████▎    | 120515/225000 [247:13:54<142:56:20,  4.92s/it]
 54%|█████▎    | 120516/225000 [247:13:59<147:11:33,  5.07s/it]
 54%|█████▎    | 120517/225000 [247:14:10<197:14:44,  6.80s/it]
 54%|█████▎    | 120518/225000 [247:14:19<211:27:50,  7.29s/it]
 54%|█████▎    | 120519/225000 [247:14:22<174:58:50,  6.03s/it]
 54%|█████▎    | 120520/225000 [247:14:29<188:31:37,  6.50s/it]
 54%|█████▎    | 120521/225000 [247:14:35<177:22:14,  6.11s/it]
 54%|█████▎    | 120522/225000 [247:14:39<166:56:08,  5.75s/it]
 54%|█████▎    | 120523/225000 [247:14:45<162:01:48,  5.58s/it]
 54%|█████▎    | 120524/225000 [247:14:50<157:00:46,  5.41s/it]
 54%|█████▎    | 120525/225000 [247:14:55<153:59:25,  5.31s/it]
 54%|█████▎    | 120526/225000 [247:15:00<151:12:40,  5.21s/it]
 54%|█████▎    | 120527/225000 [247:15:04<142:14:23,  4.90s/it]
 54%|█████▎    | 120528/225000 [247:15:09<147:22:35,  5.08s/it]
 54%|█████▎    | 120529/225000 [247:15:14<147:03:48,  5.07s/it]
 54%|█████▎    | 120530/225000 [247:15:25<193:00:17,  6.65s/it]
 54%|█████▎    | 120531/225000 [247:15:33<205:56:14,  7.10s/it]
 54%|█████▎    | 120532/225000 [247:15:36<173:47:10,  5.99s/it]
 54%|█████▎    | 120533/225000 [247:15:45<201:00:51,  6.93s/it]
 54%|█████▎    | 120534/225000 [247:15:49<168:06:22,  5.79s/it]
 54%|█████▎    | 120535/225000 [247:15:56<180:54:46,  6.23s/it]
 54%|█████▎    | 120536/225000 [247:16:00<158:56:17,  5.48s/it]
 54%|█████▎    | 120537/225000 [247:16:03<145:40:43,  5.02s/it]
 54%|█████▎    | 120538/225000 [247:16:15<202:13:49,  6.97s/it]
 54%|█████▎    | 120539/225000 [247:16:19<177:05:47,  6.10s/it]
 54%|█████▎    | 120540/225000 [247:16:23<158:38:05,  5.47s/it]
 54%|█████▎    | 120541/225000 [247:16:29<161:27:08,  5.56s/it]
 54%|█████▎    | 120542/225000 [247:16:39<199:02:57,  6.86s/it]
 54%|█████▎    | 120543/225000 [247:16:42<167:45:30,  5.78s/it]
 54%|█████▎    | 120544/225000 [247:16:46<149:03:52,  5.14s/it]
 54%|█████▎    | 120545/225000 [247:16:50<139:50:27,  4.82s/it]
 54%|█████▎    | 120546/225000 [247:16:59<179:05:54,  6.17s/it]
 54%|█████▎    | 120547/225000 [247:17:05<180:00:04,  6.20s/it]
 54%|█████▎    | 120548/225000 [247:17:09<160:00:40,  5.51s/it]
 54%|█████▎    | 120549/225000 [247:17:13<146:23:02,  5.05s/it]
 54%|█████▎    | 120550/225000 [247:17:17<137:40:39,  4.75s/it]
                                                               
{'loss': '0.859', 'grad_norm': '1.018', 'learning_rate': '2.391e-05', 'epoch': '0.5358', 'num_input_tokens_seen': 412174176, 'train_runtime': '8.902e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120550/225000 [247:17:17<137:40:39,  4.75s/it]
 54%|█████▎    | 120551/225000 [247:17:22<141:57:58,  4.89s/it]
 54%|█████▎    | 120552/225000 [247:17:33<188:55:41,  6.51s/it]
 54%|█████▎    | 120553/225000 [247:17:36<160:58:00,  5.55s/it]
 54%|█████▎    | 120554/225000 [247:17:46<195:53:33,  6.75s/it]
 54%|█████▎    | 120555/225000 [247:17:49<165:34:56,  5.71s/it]
 54%|█████▎    | 120556/225000 [247:17:52<147:31:55,  5.09s/it]
 54%|█████▎    | 120557/225000 [247:17:57<140:30:29,  4.84s/it]
 54%|█████▎    | 120558/225000 [247:18:01<134:51:40,  4.65s/it]
 54%|█████▎    | 120559/225000 [247:18:10<169:33:53,  5.84s/it]
 54%|█████▎    | 120560/225000 [247:18:18<195:13:16,  6.73s/it]
 54%|█████▎    | 120561/225000 [247:18:22<164:45:57,  5.68s/it]
 54%|█████▎    | 120562/225000 [247:18:31<194:21:06,  6.70s/it]
 54%|█████▎    | 120563/225000 [247:18:36<177:51:36,  6.13s/it]
 54%|█████▎    | 120564/225000 [247:18:40<163:57:42,  5.65s/it]
 54%|█████▎    | 120565/225000 [247:18:44<153:06:00,  5.28s/it]
 54%|█████▎    | 120566/225000 [247:18:49<148:04:56,  5.10s/it]
 54%|█████▎    | 120567/225000 [247:18:54<143:16:33,  4.94s/it]
 54%|█████▎    | 120568/225000 [247:19:03<177:15:00,  6.11s/it]
 54%|█████▎    | 120569/225000 [247:19:10<191:43:36,  6.61s/it]
 54%|█████▎    | 120570/225000 [247:19:14<168:33:24,  5.81s/it]
 54%|█████▎    | 120571/225000 [247:19:18<148:57:18,  5.13s/it]
 54%|█████▎    | 120572/225000 [247:19:28<190:13:04,  6.56s/it]
 54%|█████▎    | 120573/225000 [247:19:31<158:31:58,  5.47s/it]
 54%|█████▎    | 120574/225000 [247:19:38<174:56:52,  6.03s/it]
 54%|█████▎    | 120575/225000 [247:19:42<155:41:06,  5.37s/it]
 54%|█████▎    | 120576/225000 [247:19:49<174:43:15,  6.02s/it]
 54%|█████▎    | 120577/225000 [247:19:53<155:44:24,  5.37s/it]
 54%|█████▎    | 120578/225000 [247:20:01<175:01:43,  6.03s/it]
 54%|█████▎    | 120579/225000 [247:20:06<170:34:17,  5.88s/it]
 54%|█████▎    | 120580/225000 [247:20:13<181:38:09,  6.26s/it]
 54%|█████▎    | 120581/225000 [247:20:18<163:03:45,  5.62s/it]
 54%|█████▎    | 120582/225000 [247:20:23<159:44:43,  5.51s/it]
 54%|█████▎    | 120583/225000 [247:20:28<156:18:23,  5.39s/it]
 54%|█████▎    | 120584/225000 [247:20:33<150:34:08,  5.19s/it]
 54%|█████▎    | 120585/225000 [247:20:38<152:11:50,  5.25s/it]
 54%|█████▎    | 120586/225000 [247:20:43<152:20:08,  5.25s/it]
 54%|█████▎    | 120587/225000 [247:20:49<152:23:50,  5.25s/it]
 54%|█████▎    | 120588/225000 [247:20:54<154:36:06,  5.33s/it]
 54%|█████▎    | 120589/225000 [247:20:59<148:53:38,  5.13s/it]
 54%|█████▎    | 120590/225000 [247:21:04<148:18:57,  5.11s/it]
 54%|█████▎    | 120591/225000 [247:21:09<149:38:21,  5.16s/it]
 54%|█████▎    | 120592/225000 [247:21:14<149:14:16,  5.15s/it]
 54%|█████▎    | 120593/225000 [247:21:19<147:20:13,  5.08s/it]
 54%|█████▎    | 120594/225000 [247:21:26<159:51:17,  5.51s/it]
 54%|█████▎    | 120595/225000 [247:21:38<217:13:12,  7.49s/it]
 54%|█████▎    | 120596/225000 [247:21:41<182:59:47,  6.31s/it]
 54%|█████▎    | 120597/225000 [247:21:48<184:03:15,  6.35s/it]
 54%|█████▎    | 120598/225000 [247:21:52<165:00:42,  5.69s/it]
 54%|█████▎    | 120599/225000 [247:21:56<149:46:15,  5.16s/it]
 54%|█████▎    | 120600/225000 [247:22:00<143:13:46,  4.94s/it]
                                                               
{'loss': '0.8648', 'grad_norm': '1.051', 'learning_rate': '2.389e-05', 'epoch': '0.536', 'num_input_tokens_seen': 412300496, 'train_runtime': '8.905e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120600/225000 [247:22:00<143:13:46,  4.94s/it]
 54%|█████▎    | 120601/225000 [247:22:05<141:13:02,  4.87s/it]
 54%|█████▎    | 120602/225000 [247:22:15<187:07:08,  6.45s/it]
 54%|█████▎    | 120603/225000 [247:22:23<198:43:41,  6.85s/it]
 54%|█████▎    | 120604/225000 [247:22:26<169:52:14,  5.86s/it]
 54%|█████▎    | 120605/225000 [247:22:30<149:42:28,  5.16s/it]
 54%|█████▎    | 120606/225000 [247:22:34<142:31:09,  4.91s/it]
 54%|█████▎    | 120607/225000 [247:22:39<143:07:38,  4.94s/it]
 54%|█████▎    | 120608/225000 [247:22:44<143:24:37,  4.95s/it]
 54%|█████▎    | 120609/225000 [247:22:50<148:08:12,  5.11s/it]
 54%|█████▎    | 120610/225000 [247:22:55<148:08:47,  5.11s/it]
 54%|█████▎    | 120611/225000 [247:23:00<149:08:33,  5.14s/it]
 54%|█████▎    | 120612/225000 [247:23:07<165:52:50,  5.72s/it]
 54%|█████▎    | 120613/225000 [247:23:19<217:53:02,  7.51s/it]
 54%|█████▎    | 120614/225000 [247:23:22<182:44:00,  6.30s/it]
 54%|█████▎    | 120615/225000 [247:23:28<179:16:41,  6.18s/it]
 54%|█████▎    | 120616/225000 [247:23:37<200:05:11,  6.90s/it]
 54%|█████▎    | 120617/225000 [247:23:41<175:52:22,  6.07s/it]
 54%|█████▎    | 120618/225000 [247:23:45<157:38:44,  5.44s/it]
 54%|█████▎    | 120619/225000 [247:23:52<174:02:40,  6.00s/it]
 54%|█████▎    | 120620/225000 [247:23:59<180:37:49,  6.23s/it]
 54%|█████▎    | 120621/225000 [247:24:04<170:18:24,  5.87s/it]
 54%|█████▎    | 120622/225000 [247:24:13<196:30:01,  6.78s/it]
 54%|█████▎    | 120623/225000 [247:24:16<168:08:27,  5.80s/it]
 54%|█████▎    | 120624/225000 [247:24:21<159:46:48,  5.51s/it]
 54%|█████▎    | 120625/225000 [247:24:28<170:40:53,  5.89s/it]
 54%|█████▎    | 120626/225000 [247:24:32<158:11:11,  5.46s/it]
 54%|█████▎    | 120627/225000 [247:24:37<147:09:23,  5.08s/it]
 54%|█████▎    | 120628/225000 [247:24:41<139:12:39,  4.80s/it]
 54%|█████▎    | 120629/225000 [247:24:52<192:32:15,  6.64s/it]
 54%|█████▎    | 120630/225000 [247:24:56<175:29:49,  6.05s/it]
 54%|█████▎    | 120631/225000 [247:25:01<165:25:43,  5.71s/it]
 54%|█████▎    | 120632/225000 [247:25:06<156:05:41,  5.38s/it]
 54%|█████▎    | 120633/225000 [247:25:10<145:10:40,  5.01s/it]
 54%|█████▎    | 120634/225000 [247:25:14<138:26:08,  4.78s/it]
 54%|█████▎    | 120635/225000 [247:25:19<135:39:49,  4.68s/it]
 54%|█████▎    | 120636/225000 [247:25:29<186:54:35,  6.45s/it]
 54%|█████▎    | 120637/225000 [247:25:36<187:14:40,  6.46s/it]
 54%|█████▎    | 120638/225000 [247:25:40<164:00:18,  5.66s/it]
 54%|█████▎    | 120639/225000 [247:25:44<150:26:58,  5.19s/it]
 54%|█████▎    | 120640/225000 [247:25:49<153:41:33,  5.30s/it]
 54%|█████▎    | 120641/225000 [247:26:00<202:58:18,  7.00s/it]
 54%|█████▎    | 120642/225000 [247:26:04<171:05:58,  5.90s/it]
 54%|█████▎    | 120643/225000 [247:26:08<155:19:46,  5.36s/it]
 54%|█████▎    | 120644/225000 [247:26:12<145:47:44,  5.03s/it]
 54%|█████▎    | 120645/225000 [247:26:22<189:17:26,  6.53s/it]
 54%|█████▎    | 120646/225000 [247:26:26<167:35:55,  5.78s/it]
 54%|█████▎    | 120647/225000 [247:26:30<153:27:19,  5.29s/it]
 54%|█████▎    | 120648/225000 [247:26:35<146:52:08,  5.07s/it]
 54%|█████▎    | 120649/225000 [247:26:39<139:39:51,  4.82s/it]
 54%|█████▎    | 120650/225000 [247:26:51<199:19:01,  6.88s/it]
                                                               
{'loss': '0.8562', 'grad_norm': '0.9928', 'learning_rate': '2.387e-05', 'epoch': '0.5362', 'num_input_tokens_seen': 412426848, 'train_runtime': '8.908e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120650/225000 [247:26:51<199:19:01,  6.88s/it]
 54%|█████▎    | 120651/225000 [247:26:57<190:48:59,  6.58s/it]
 54%|█████▎    | 120652/225000 [247:27:00<160:05:36,  5.52s/it]
 54%|█████▎    | 120653/225000 [247:27:03<140:26:45,  4.85s/it]
 54%|█████▎    | 120654/225000 [247:27:13<188:36:25,  6.51s/it]
 54%|█████▎    | 120655/225000 [247:27:18<170:43:29,  5.89s/it]
 54%|█████▎    | 120656/225000 [247:27:22<160:00:22,  5.52s/it]
 54%|█████▎    | 120657/225000 [247:27:27<152:17:12,  5.25s/it]
 54%|█████▎    | 120658/225000 [247:27:31<143:33:11,  4.95s/it]
 54%|█████▎    | 120659/225000 [247:27:36<139:39:00,  4.82s/it]
 54%|█████▎    | 120660/225000 [247:27:41<141:54:28,  4.90s/it]
 54%|█████▎    | 120661/225000 [247:27:53<206:33:15,  7.13s/it]
 54%|█████▎    | 120662/225000 [247:27:59<193:18:51,  6.67s/it]
 54%|█████▎    | 120663/225000 [247:28:03<173:55:32,  6.00s/it]
 54%|█████▎    | 120664/225000 [247:28:07<151:47:30,  5.24s/it]
 54%|█████▎    | 120665/225000 [247:28:11<144:08:57,  4.97s/it]
 54%|█████▎    | 120666/225000 [247:28:17<149:36:31,  5.16s/it]
 54%|█████▎    | 120667/225000 [247:28:26<190:54:24,  6.59s/it]
 54%|█████▎    | 120668/225000 [247:28:30<163:43:51,  5.65s/it]
 54%|█████▎    | 120669/225000 [247:28:36<169:01:31,  5.83s/it]
 54%|█████▎    | 120670/225000 [247:28:43<175:10:28,  6.04s/it]
 54%|█████▎    | 120671/225000 [247:28:47<155:44:40,  5.37s/it]
 54%|█████▎    | 120672/225000 [247:28:51<148:59:01,  5.14s/it]
 54%|█████▎    | 120673/225000 [247:28:56<150:35:26,  5.20s/it]
 54%|█████▎    | 120674/225000 [247:29:02<150:47:24,  5.20s/it]
 54%|█████▎    | 120675/225000 [247:29:06<146:36:14,  5.06s/it]
 54%|█████▎    | 120676/225000 [247:29:12<149:26:28,  5.16s/it]
 54%|█████▎    | 120677/225000 [247:29:17<148:54:59,  5.14s/it]
 54%|█████▎    | 120678/225000 [247:29:22<151:33:09,  5.23s/it]
 54%|█████▎    | 120679/225000 [247:29:28<152:46:43,  5.27s/it]
 54%|█████▎    | 120680/225000 [247:29:33<156:09:34,  5.39s/it]
 54%|█████▎    | 120681/225000 [247:29:44<201:21:43,  6.95s/it]
 54%|█████▎    | 120682/225000 [247:29:53<216:16:48,  7.46s/it]
 54%|█████▎    | 120683/225000 [247:30:00<214:35:50,  7.41s/it]
 54%|█████▎    | 120684/225000 [247:30:04<189:11:29,  6.53s/it]
 54%|█████▎    | 120685/225000 [247:30:10<183:02:07,  6.32s/it]
 54%|█████▎    | 120686/225000 [247:30:14<160:19:42,  5.53s/it]
 54%|█████▎    | 120687/225000 [247:30:21<176:02:24,  6.08s/it]
 54%|█████▎    | 120688/225000 [247:30:25<158:16:47,  5.46s/it]
 54%|█████▎    | 120689/225000 [247:30:29<145:01:16,  5.01s/it]
 54%|█████▎    | 120690/225000 [247:30:40<192:33:04,  6.65s/it]
 54%|█████▎    | 120691/225000 [247:30:45<182:44:45,  6.31s/it]
 54%|█████▎    | 120692/225000 [247:30:49<158:15:32,  5.46s/it]
 54%|█████▎    | 120693/225000 [247:30:57<184:51:44,  6.38s/it]
 54%|█████▎    | 120694/225000 [247:31:01<164:55:50,  5.69s/it]
 54%|█████▎    | 120695/225000 [247:31:08<170:46:51,  5.89s/it]
 54%|█████▎    | 120696/225000 [247:31:16<189:55:45,  6.56s/it]
 54%|█████▎    | 120697/225000 [247:31:21<178:48:24,  6.17s/it]
 54%|█████▎    | 120698/225000 [247:31:25<156:34:06,  5.40s/it]
 54%|█████▎    | 120699/225000 [247:31:29<146:47:16,  5.07s/it]
 54%|█████▎    | 120700/225000 [247:31:40<197:01:59,  6.80s/it]
                                                               
{'loss': '0.8535', 'grad_norm': '0.9631', 'learning_rate': '2.385e-05', 'epoch': '0.5364', 'num_input_tokens_seen': 412554896, 'train_runtime': '8.911e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120700/225000 [247:31:40<197:01:59,  6.80s/it]
 54%|█████▎    | 120701/225000 [247:31:43<168:38:51,  5.82s/it]
 54%|█████▎    | 120702/225000 [247:31:49<166:45:07,  5.76s/it]
 54%|█████▎    | 120703/225000 [247:31:55<167:35:05,  5.78s/it]
 54%|█████▎    | 120704/225000 [247:32:00<161:52:16,  5.59s/it]
 54%|█████▎    | 120705/225000 [247:32:05<157:37:05,  5.44s/it]
 54%|█████▎    | 120706/225000 [247:32:10<157:07:31,  5.42s/it]
 54%|█████▎    | 120707/225000 [247:32:21<198:24:41,  6.85s/it]
 54%|█████▎    | 120708/225000 [247:32:24<165:05:20,  5.70s/it]
 54%|█████▎    | 120709/225000 [247:32:29<163:03:14,  5.63s/it]
 54%|█████▎    | 120710/225000 [247:32:33<151:22:31,  5.23s/it]
 54%|█████▎    | 120711/225000 [247:32:39<154:22:32,  5.33s/it]
 54%|█████▎    | 120712/225000 [247:32:45<160:09:43,  5.53s/it]
 54%|█████▎    | 120713/225000 [247:32:49<146:55:25,  5.07s/it]
 54%|█████▎    | 120714/225000 [247:32:53<135:47:17,  4.69s/it]
 54%|█████▎    | 120715/225000 [247:32:57<133:12:58,  4.60s/it]
 54%|█████▎    | 120716/225000 [247:33:03<140:52:59,  4.86s/it]
 54%|█████▎    | 120717/225000 [247:33:15<205:26:47,  7.09s/it]
 54%|█████▎    | 120718/225000 [247:33:18<173:57:40,  6.01s/it]
 54%|█████▎    | 120719/225000 [247:33:24<174:32:11,  6.03s/it]
 54%|█████▎    | 120720/225000 [247:33:28<153:27:55,  5.30s/it]
 54%|█████▎    | 120721/225000 [247:33:36<172:42:54,  5.96s/it]
 54%|█████▎    | 120722/225000 [247:33:39<149:59:22,  5.18s/it]
 54%|█████▎    | 120723/225000 [247:33:44<149:14:05,  5.15s/it]
 54%|█████▎    | 120724/225000 [247:33:52<173:14:59,  5.98s/it]
 54%|█████▎    | 120725/225000 [247:33:56<153:32:01,  5.30s/it]
 54%|█████▎    | 120726/225000 [247:34:03<174:58:43,  6.04s/it]
 54%|█████▎    | 120727/225000 [247:34:07<153:39:49,  5.31s/it]
 54%|█████▎    | 120728/225000 [247:34:15<173:19:00,  5.98s/it]
 54%|█████▎    | 120729/225000 [247:34:20<165:49:46,  5.73s/it]
 54%|█████▎    | 120730/225000 [247:34:24<151:12:53,  5.22s/it]
 54%|█████▎    | 120731/225000 [247:34:28<141:50:51,  4.90s/it]
 54%|█████▎    | 120732/225000 [247:34:32<137:44:27,  4.76s/it]
 54%|█████▎    | 120733/225000 [247:34:37<136:31:49,  4.71s/it]
 54%|█████▎    | 120734/225000 [247:34:41<134:30:29,  4.64s/it]
 54%|█████▎    | 120735/225000 [247:34:51<175:40:56,  6.07s/it]
 54%|█████▎    | 120736/225000 [247:34:59<195:53:25,  6.76s/it]
 54%|█████▎    | 120737/225000 [247:35:02<164:21:56,  5.68s/it]
 54%|█████▎    | 120738/225000 [247:35:06<146:56:13,  5.07s/it]
 54%|█████▎    | 120739/225000 [247:35:15<182:25:09,  6.30s/it]
 54%|█████▎    | 120740/225000 [247:35:19<163:37:24,  5.65s/it]
 54%|█████▎    | 120741/225000 [247:35:23<147:12:43,  5.08s/it]
 54%|█████▎    | 120742/225000 [247:35:33<187:47:45,  6.48s/it]
 54%|█████▎    | 120743/225000 [247:35:37<172:27:29,  5.95s/it]
 54%|█████▎    | 120744/225000 [247:35:41<154:11:01,  5.32s/it]
 54%|█████▎    | 120745/225000 [247:35:45<143:54:38,  4.97s/it]
 54%|█████▎    | 120746/225000 [247:35:50<141:11:39,  4.88s/it]
 54%|█████▎    | 120747/225000 [247:36:02<202:50:18,  7.00s/it]
 54%|█████▎    | 120748/225000 [247:36:05<169:27:51,  5.85s/it]
 54%|█████▎    | 120749/225000 [247:36:09<153:54:01,  5.31s/it]
 54%|█████▎    | 120750/225000 [247:36:14<145:56:30,  5.04s/it]
                                                               
{'loss': '0.8452', 'grad_norm': '1.289', 'learning_rate': '2.383e-05', 'epoch': '0.5367', 'num_input_tokens_seen': 412679520, 'train_runtime': '8.914e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120750/225000 [247:36:14<145:56:30,  5.04s/it]
 54%|█████▎    | 120751/225000 [247:36:18<141:33:28,  4.89s/it]
 54%|█████▎    | 120752/225000 [247:36:30<202:08:22,  6.98s/it]
 54%|█████▎    | 120753/225000 [247:36:34<179:12:16,  6.19s/it]
 54%|█████▎    | 120754/225000 [247:36:39<165:13:59,  5.71s/it]
 54%|█████▎    | 120755/225000 [247:36:44<156:15:54,  5.40s/it]
 54%|█████▎    | 120756/225000 [247:36:48<148:22:36,  5.12s/it]
 54%|█████▎    | 120757/225000 [247:36:53<143:30:58,  4.96s/it]
 54%|█████▎    | 120758/225000 [247:37:01<176:00:46,  6.08s/it]
 54%|█████▎    | 120759/225000 [247:37:10<195:38:52,  6.76s/it]
 54%|█████▎    | 120760/225000 [247:37:13<169:11:38,  5.84s/it]
 54%|█████▎    | 120761/225000 [247:37:22<188:45:23,  6.52s/it]
 54%|█████▎    | 120762/225000 [247:37:25<159:22:07,  5.50s/it]
 54%|█████▎    | 120763/225000 [247:37:33<179:45:37,  6.21s/it]
 54%|█████▎    | 120764/225000 [247:37:36<156:17:55,  5.40s/it]
 54%|█████▎    | 120765/225000 [247:37:40<146:49:39,  5.07s/it]
 54%|█████▎    | 120766/225000 [247:37:51<192:02:39,  6.63s/it]
 54%|█████▎    | 120767/225000 [247:37:55<168:36:54,  5.82s/it]
 54%|█████▎    | 120768/225000 [247:37:59<153:38:12,  5.31s/it]
 54%|█████▎    | 120769/225000 [247:38:03<141:02:25,  4.87s/it]
 54%|█████▎    | 120770/225000 [247:38:14<195:37:47,  6.76s/it]
 54%|█████▎    | 120771/225000 [247:38:18<170:54:58,  5.90s/it]
 54%|█████▎    | 120772/225000 [247:38:23<165:41:07,  5.72s/it]
 54%|█████▎    | 120773/225000 [247:38:30<176:24:09,  6.09s/it]
 54%|█████▎    | 120774/225000 [247:38:34<163:06:13,  5.63s/it]
 54%|█████▎    | 120775/225000 [247:38:39<153:28:49,  5.30s/it]
 54%|█████▎    | 120776/225000 [247:38:44<152:23:31,  5.26s/it]
 54%|█████▎    | 120777/225000 [247:38:49<149:42:45,  5.17s/it]
 54%|█████▎    | 120778/225000 [247:38:56<161:15:38,  5.57s/it]
 54%|█████▎    | 120779/225000 [247:39:06<207:10:17,  7.16s/it]
 54%|█████▎    | 120780/225000 [247:39:10<179:13:15,  6.19s/it]
 54%|█████▎    | 120781/225000 [247:39:14<160:26:39,  5.54s/it]
 54%|█████▎    | 120782/225000 [247:39:18<145:35:58,  5.03s/it]
 54%|█████▎    | 120783/225000 [247:39:29<191:23:10,  6.61s/it]
 54%|█████▎    | 120784/225000 [247:39:32<165:27:00,  5.72s/it]
 54%|█████▎    | 120785/225000 [247:39:36<147:58:31,  5.11s/it]
 54%|█████▎    | 120786/225000 [247:39:46<189:48:23,  6.56s/it]
 54%|█████▎    | 120787/225000 [247:39:49<163:51:49,  5.66s/it]
 54%|█████▎    | 120788/225000 [247:39:58<191:19:21,  6.61s/it]
 54%|█████▎    | 120789/225000 [247:40:02<166:44:08,  5.76s/it]
 54%|█████▎    | 120790/225000 [247:40:06<150:08:55,  5.19s/it]
 54%|█████▎    | 120791/225000 [247:40:10<139:23:32,  4.82s/it]
 54%|█████▎    | 120792/225000 [247:40:19<175:50:58,  6.07s/it]
 54%|█████▎    | 120793/225000 [247:40:26<185:36:59,  6.41s/it]
 54%|█████▎    | 120794/225000 [247:40:30<162:54:23,  5.63s/it]
 54%|█████▎    | 120795/225000 [247:40:38<185:40:26,  6.41s/it]
 54%|█████▎    | 120796/225000 [247:40:42<165:00:36,  5.70s/it]
 54%|█████▎    | 120797/225000 [247:40:46<151:05:24,  5.22s/it]
 54%|█████▎    | 120798/225000 [247:40:54<173:42:07,  6.00s/it]
 54%|█████▎    | 120799/225000 [247:41:00<170:26:52,  5.89s/it]
 54%|█████▎    | 120800/225000 [247:41:04<153:12:14,  5.29s/it]
                                                               
{'loss': '0.8584', 'grad_norm': '1.011', 'learning_rate': '2.382e-05', 'epoch': '0.5369', 'num_input_tokens_seen': 412808544, 'train_runtime': '8.917e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120800/225000 [247:41:04<153:12:14,  5.29s/it]
 54%|█████▎    | 120801/225000 [247:41:07<138:39:36,  4.79s/it]
 54%|█████▎    | 120802/225000 [247:41:15<165:05:30,  5.70s/it]
 54%|█████▎    | 120803/225000 [247:41:22<178:00:14,  6.15s/it]
 54%|█████▎    | 120804/225000 [247:41:27<166:23:22,  5.75s/it]
 54%|█████▎    | 120805/225000 [247:41:32<158:49:32,  5.49s/it]
 54%|█████▎    | 120806/225000 [247:41:37<153:16:38,  5.30s/it]
 54%|█████▎    | 120807/225000 [247:41:42<152:09:13,  5.26s/it]
 54%|█████▎    | 120808/225000 [247:41:47<151:14:13,  5.23s/it]
 54%|█████▎    | 120809/225000 [247:41:52<149:47:08,  5.18s/it]
 54%|█████▎    | 120810/225000 [247:41:57<146:43:36,  5.07s/it]
 54%|█████▎    | 120811/225000 [247:42:02<147:37:24,  5.10s/it]
 54%|█████▎    | 120812/225000 [247:42:07<148:12:20,  5.12s/it]
 54%|█████▎    | 120813/225000 [247:42:15<168:56:02,  5.84s/it]
 54%|█████▎    | 120814/225000 [247:42:25<203:32:01,  7.03s/it]
 54%|█████▎    | 120815/225000 [247:42:28<169:44:25,  5.87s/it]
 54%|█████▎    | 120816/225000 [247:42:36<187:32:12,  6.48s/it]
 54%|█████▎    | 120817/225000 [247:42:39<163:01:05,  5.63s/it]
 54%|█████▎    | 120818/225000 [247:42:47<182:09:49,  6.29s/it]
 54%|█████▎    | 120819/225000 [247:42:51<163:53:11,  5.66s/it]
 54%|█████▎    | 120820/225000 [247:42:55<150:05:01,  5.19s/it]
 54%|█████▎    | 120821/225000 [247:43:00<143:21:25,  4.95s/it]
 54%|█████▎    | 120822/225000 [247:43:08<175:38:57,  6.07s/it]
 54%|█████▎    | 120823/225000 [247:43:12<153:21:20,  5.30s/it]
 54%|█████▎    | 120824/225000 [247:43:22<191:29:29,  6.62s/it]
 54%|█████▎    | 120825/225000 [247:43:28<188:23:31,  6.51s/it]
 54%|█████▎    | 120826/225000 [247:43:33<176:45:06,  6.11s/it]
 54%|█████▎    | 120827/225000 [247:43:39<175:25:30,  6.06s/it]
 54%|█████▎    | 120828/225000 [247:43:43<159:15:15,  5.50s/it]
 54%|█████▎    | 120829/225000 [247:43:47<145:00:30,  5.01s/it]
 54%|█████▎    | 120830/225000 [247:43:53<153:25:13,  5.30s/it]
 54%|█████▎    | 120831/225000 [247:44:00<165:43:59,  5.73s/it]
 54%|█████▎    | 120832/225000 [247:44:04<152:39:17,  5.28s/it]
 54%|█████▎    | 120833/225000 [247:44:08<140:27:42,  4.85s/it]
 54%|█████▎    | 120834/225000 [247:44:12<135:17:24,  4.68s/it]
 54%|█████▎    | 120835/225000 [247:44:19<152:02:47,  5.25s/it]
 54%|█████▎    | 120836/225000 [247:44:28<189:14:00,  6.54s/it]
 54%|█████▎    | 120837/225000 [247:44:35<187:53:35,  6.49s/it]
 54%|█████▎    | 120838/225000 [247:44:38<160:59:07,  5.56s/it]
 54%|█████▎    | 120839/225000 [247:44:42<144:17:13,  4.99s/it]
 54%|█████▎    | 120840/225000 [247:44:51<177:37:19,  6.14s/it]
 54%|█████▎    | 120841/225000 [247:44:54<154:50:10,  5.35s/it]
 54%|█████▎    | 120842/225000 [247:45:01<171:01:31,  5.91s/it]
 54%|█████▎    | 120843/225000 [247:45:05<152:58:58,  5.29s/it]
 54%|█████▎    | 120844/225000 [247:45:16<197:46:07,  6.84s/it]
 54%|█████▎    | 120845/225000 [247:45:19<164:42:28,  5.69s/it]
 54%|█████▎    | 120846/225000 [247:45:26<175:53:29,  6.08s/it]
 54%|█████▎    | 120847/225000 [247:45:30<158:24:57,  5.48s/it]
 54%|█████▎    | 120848/225000 [247:45:33<142:26:14,  4.92s/it]
 54%|█████▎    | 120849/225000 [247:45:42<175:02:19,  6.05s/it]
 54%|█████▎    | 120850/225000 [247:45:46<154:25:36,  5.34s/it]
                                                               
{'loss': '0.856', 'grad_norm': '1.134', 'learning_rate': '2.38e-05', 'epoch': '0.5371', 'num_input_tokens_seen': 412935416, 'train_runtime': '8.919e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120850/225000 [247:45:46<154:25:36,  5.34s/it]
 54%|█████▎    | 120851/225000 [247:45:54<181:23:37,  6.27s/it]
 54%|█████▎    | 120852/225000 [247:45:59<167:15:10,  5.78s/it]
 54%|█████▎    | 120853/225000 [247:46:02<149:07:50,  5.15s/it]
 54%|█████▎    | 120854/225000 [247:46:09<163:26:09,  5.65s/it]
 54%|█████▎    | 120855/225000 [247:46:14<152:41:36,  5.28s/it]
 54%|█████▎    | 120856/225000 [247:46:22<178:15:36,  6.16s/it]
 54%|█████▎    | 120857/225000 [247:46:25<152:34:10,  5.27s/it]
 54%|█████▎    | 120858/225000 [247:46:30<149:58:00,  5.18s/it]
 54%|█████▎    | 120859/225000 [247:46:35<147:26:01,  5.10s/it]
 54%|█████▎    | 120860/225000 [247:46:40<143:11:16,  4.95s/it]
 54%|█████▎    | 120861/225000 [247:46:45<144:42:55,  5.00s/it]
 54%|█████▎    | 120862/225000 [247:46:49<143:10:16,  4.95s/it]
 54%|█████▎    | 120863/225000 [247:46:54<139:10:40,  4.81s/it]
 54%|█████▎    | 120864/225000 [247:46:59<138:13:37,  4.78s/it]
 54%|█████▎    | 120865/225000 [247:47:10<193:53:04,  6.70s/it]
 54%|█████▎    | 120866/225000 [247:47:15<183:29:50,  6.34s/it]
 54%|█████▎    | 120867/225000 [247:47:20<170:14:05,  5.89s/it]
 54%|█████▎    | 120868/225000 [247:47:25<159:20:10,  5.51s/it]
 54%|█████▎    | 120869/225000 [247:47:29<150:49:05,  5.21s/it]
 54%|█████▎    | 120870/225000 [247:47:34<147:49:39,  5.11s/it]
 54%|█████▎    | 120871/225000 [247:47:39<148:22:35,  5.13s/it]
 54%|█████▎    | 120872/225000 [247:47:44<144:39:50,  5.00s/it]
 54%|█████▎    | 120873/225000 [247:47:49<145:17:04,  5.02s/it]
 54%|█████▎    | 120874/225000 [247:48:02<214:19:05,  7.41s/it]
 54%|█████▎    | 120875/225000 [247:48:06<187:34:28,  6.49s/it]
 54%|█████▎    | 120876/225000 [247:48:10<162:25:34,  5.62s/it]
 54%|█████▎    | 120877/225000 [247:48:14<149:36:48,  5.17s/it]
 54%|█████▎    | 120878/225000 [247:48:27<215:39:08,  7.46s/it]
 54%|█████▎    | 120879/225000 [247:48:30<181:03:16,  6.26s/it]
 54%|█████▎    | 120880/225000 [247:48:35<162:13:59,  5.61s/it]
 54%|█████▎    | 120881/225000 [247:48:39<149:47:25,  5.18s/it]
 54%|█████▎    | 120882/225000 [247:48:50<205:17:55,  7.10s/it]
 54%|█████▎    | 120883/225000 [247:48:54<175:51:43,  6.08s/it]
 54%|█████▎    | 120884/225000 [247:48:58<158:36:37,  5.48s/it]
 54%|█████▎    | 120885/225000 [247:49:05<168:41:36,  5.83s/it]
 54%|█████▎    | 120886/225000 [247:49:10<162:09:02,  5.61s/it]
 54%|█████▎    | 120887/225000 [247:49:13<144:18:23,  4.99s/it]
 54%|█████▎    | 120888/225000 [247:49:22<179:41:30,  6.21s/it]
 54%|█████▎    | 120889/225000 [247:49:26<154:58:28,  5.36s/it]
 54%|█████▎    | 120890/225000 [247:49:35<186:42:21,  6.46s/it]
 54%|█████▎    | 120891/225000 [247:49:39<164:25:16,  5.69s/it]
 54%|█████▎    | 120892/225000 [247:49:43<149:31:13,  5.17s/it]
 54%|█████▎    | 120893/225000 [247:49:46<137:13:59,  4.75s/it]
 54%|█████▎    | 120894/225000 [247:49:53<155:32:56,  5.38s/it]
 54%|█████▎    | 120895/225000 [247:50:02<185:25:48,  6.41s/it]
 54%|█████▎    | 120896/225000 [247:50:06<164:01:32,  5.67s/it]
 54%|█████▎    | 120897/225000 [247:50:10<146:14:19,  5.06s/it]
 54%|█████▎    | 120898/225000 [247:50:20<187:57:15,  6.50s/it]
 54%|█████▎    | 120899/225000 [247:50:23<162:03:35,  5.60s/it]
 54%|█████▎    | 120900/225000 [247:50:27<145:10:38,  5.02s/it]
                                                               
{'loss': '0.8665', 'grad_norm': '1.158', 'learning_rate': '2.378e-05', 'epoch': '0.5373', 'num_input_tokens_seen': 413063192, 'train_runtime': '8.922e+05', 'train_tokens_per_second': '463'}

 54%|█████▎    | 120900/225000 [247:50:27<145:10:38,  5.02s/it]
 54%|█████▎    | 120901/225000 [247:50:31<138:37:18,  4.79s/it]
 54%|█████▎    | 120902/225000 [247:50:37<150:25:14,  5.20s/it]
 54%|█████▎    | 120903/225000 [247:50:45<172:41:00,  5.97s/it]
 54%|█████▎    | 120904/225000 [247:50:50<163:26:05,  5.65s/it]
 54%|█████▎    | 120905/225000 [247:50:55<159:28:05,  5.52s/it]
 54%|█████▎    | 120906/225000 [247:51:00<154:15:31,  5.33s/it]
 54%|█████▎    | 120907/225000 [247:51:05<155:40:23,  5.38s/it]
 54%|█████▎    | 120908/225000 [247:51:11<155:27:22,  5.38s/it]
 54%|█████▎    | 120909/225000 [247:51:16<152:35:20,  5.28s/it]
 54%|█████▎    | 120910/225000 [247:51:21<151:46:45,  5.25s/it]
 54%|█████▎    | 120911/225000 [247:51:26<150:30:48,  5.21s/it]
 54%|█████▎    | 120912/225000 [247:51:32<153:51:21,  5.32s/it]
 54%|█████▎    | 120913/225000 [247:51:37<152:01:09,  5.26s/it]
 54%|█████▎    | 120914/225000 [247:51:46<187:46:49,  6.49s/it]
 54%|█████▎    | 120915/225000 [247:51:53<193:53:47,  6.71s/it]
 54%|█████▎    | 120916/225000 [247:51:59<186:01:28,  6.43s/it]
 54%|█████▎    | 120917/225000 [247:52:04<170:59:30,  5.91s/it]
 54%|█████▎    | 120918/225000 [247:52:08<155:31:43,  5.38s/it]
 54%|█████▎    | 120919/225000 [247:52:12<141:37:11,  4.90s/it]
 54%|█████▎    | 120920/225000 [247:52:22<185:05:57,  6.40s/it]
 54%|█████▎    | 120921/225000 [247:52:26<170:41:28,  5.90s/it]
 54%|█████▎    | 120922/225000 [247:52:34<186:48:14,  6.46s/it]
 54%|█████▎    | 120923/225000 [247:52:37<158:40:27,  5.49s/it]
 54%|█████▎    | 120924/225000 [247:52:45<175:15:27,  6.06s/it]
 54%|█████▎    | 120925/225000 [247:52:51<171:54:48,  5.95s/it]
 54%|█████▎    | 120926/225000 [247:52:55<158:48:46,  5.49s/it]
 54%|█████▎    | 120927/225000 [247:52:59<149:09:44,  5.16s/it]
 54%|█████▎    | 120928/225000 [247:53:05<150:13:33,  5.20s/it]
 54%|█████▎    | 120929/225000 [247:53:09<146:50:59,  5.08s/it]
 54%|█████▎    | 120930/225000 [247:53:14<144:14:40,  4.99s/it]
 54%|█████▎    | 120931/225000 [247:53:19<142:41:30,  4.94s/it]
 54%|█████▎    | 120932/225000 [247:53:24<142:15:00,  4.92s/it]
 54%|█████▎    | 120933/225000 [247:53:29<142:58:32,  4.95s/it]
 54%|█████▎    | 120934/225000 [247:53:41<204:33:58,  7.08s/it]
 54%|█████▎    | 120935/225000 [247:53:45<179:27:10,  6.21s/it]
 54%|█████▎    | 120936/225000 [247:53:49<155:08:55,  5.37s/it]
 54%|█████▎    | 120937/225000 [247:54:02<223:37:11,  7.74s/it]
 54%|█████▍    | 120938/225000 [247:54:07<200:29:35,  6.94s/it]
 54%|█████▍    | 120939/225000 [247:54:13<189:40:06,  6.56s/it]
 54%|█████▍    | 120940/225000 [247:54:18<179:04:10,  6.19s/it]
 54%|█████▍    | 120941/225000 [247:54:23<169:20:28,  5.86s/it]
 54%|█████▍    | 120942/225000 [247:54:28<164:01:19,  5.67s/it]
 54%|█████▍    | 120943/225000 [247:54:33<158:00:17,  5.47s/it]
 54%|█████▍    | 120944/225000 [247:54:38<156:21:38,  5.41s/it]
 54%|█████▍    | 120945/225000 [247:54:44<153:32:28,  5.31s/it]
 54%|█████▍    | 120946/225000 [247:54:49<154:50:32,  5.36s/it]
 54%|█████▍    | 120947/225000 [247:54:55<158:38:11,  5.49s/it]
 54%|█████▍    | 120948/225000 [247:55:00<153:12:10,  5.30s/it]
 54%|█████▍    | 120949/225000 [247:55:05<152:29:37,  5.28s/it]
 54%|█████▍    | 120950/225000 [247:55:10<154:48:04,  5.36s/it]
                                                               
{'loss': '0.8514', 'grad_norm': '1.049', 'learning_rate': '2.376e-05', 'epoch': '0.5376', 'num_input_tokens_seen': 413190792, 'train_runtime': '8.925e+05', 'train_tokens_per_second': '463'}

 54%|█████▍    | 120950/225000 [247:55:10<154:48:04,  5.36s/it]
 54%|█████▍    | 120951/225000 [247:55:16<154:50:05,  5.36s/it]
 54%|█████▍    | 120952/225000 [247:55:23<168:52:14,  5.84s/it]
 54%|█████▍    | 120953/225000 [247:55:33<205:19:30,  7.10s/it]
 54%|█████▍    | 120954/225000 [247:55:37<182:05:48,  6.30s/it]
 54%|█████▍    | 120955/225000 [247:55:41<157:29:50,  5.45s/it]
 54%|█████▍    | 120956/225000 [247:55:51<196:01:09,  6.78s/it]
 54%|█████▍    | 120957/225000 [247:55:56<182:14:46,  6.31s/it]
 54%|█████▍    | 120958/225000 [247:56:00<161:42:50,  5.60s/it]
 54%|█████▍    | 120959/225000 [247:56:04<149:25:14,  5.17s/it]
 54%|█████▍    | 120960/225000 [247:56:08<138:01:20,  4.78s/it]
 54%|█████▍    | 120961/225000 [247:56:15<163:35:02,  5.66s/it]
 54%|█████▍    | 120962/225000 [247:56:23<180:31:09,  6.25s/it]
 54%|█████▍    | 120963/225000 [247:56:26<155:40:46,  5.39s/it]
 54%|█████▍    | 120964/225000 [247:56:34<175:42:42,  6.08s/it]
 54%|█████▍    | 120965/225000 [247:56:41<179:30:41,  6.21s/it]
 54%|█████▍    | 120966/225000 [247:56:48<191:41:42,  6.63s/it]
 54%|█████▍    | 120967/225000 [247:56:51<160:39:43,  5.56s/it]
 54%|█████▍    | 120968/225000 [247:56:59<178:04:58,  6.16s/it]
 54%|█████▍    | 120969/225000 [247:57:05<179:12:09,  6.20s/it]
 54%|█████▍    | 120970/225000 [247:57:09<161:46:42,  5.60s/it]
 54%|█████▍    | 120971/225000 [247:57:15<158:34:22,  5.49s/it]
 54%|█████▍    | 120972/225000 [247:57:19<149:02:58,  5.16s/it]
 54%|█████▍    | 120973/225000 [247:57:24<145:22:17,  5.03s/it]
 54%|█████▍    | 120974/225000 [247:57:28<140:08:45,  4.85s/it]
 54%|█████▍    | 120975/225000 [247:57:37<177:34:59,  6.15s/it]
 54%|█████▍    | 120976/225000 [247:57:44<184:27:08,  6.38s/it]
 54%|█████▍    | 120977/225000 [247:57:49<170:11:09,  5.89s/it]
 54%|█████▍    | 120978/225000 [247:57:54<158:07:58,  5.47s/it]
 54%|█████▍    | 120979/225000 [247:57:58<144:54:46,  5.02s/it]
 54%|█████▍    | 120980/225000 [247:58:02<137:59:14,  4.78s/it]
 54%|█████▍    | 120981/225000 [247:58:08<153:15:05,  5.30s/it]
 54%|█████▍    | 120982/225000 [247:58:19<197:03:23,  6.82s/it]
 54%|█████▍    | 120983/225000 [247:58:24<180:49:10,  6.26s/it]
 54%|█████▍    | 120984/225000 [247:58:29<176:34:33,  6.11s/it]
 54%|█████▍    | 120985/225000 [247:58:33<152:36:39,  5.28s/it]
 54%|█████▍    | 120986/225000 [247:58:42<185:26:46,  6.42s/it]
 54%|█████▍    | 120987/225000 [247:58:46<165:24:13,  5.72s/it]
 54%|█████▍    | 120988/225000 [247:58:50<155:30:15,  5.38s/it]
 54%|█████▍    | 120989/225000 [247:58:55<150:42:44,  5.22s/it]
 54%|█████▍    | 120990/225000 [247:59:00<146:55:19,  5.09s/it]
 54%|█████▍    | 120991/225000 [247:59:05<146:11:54,  5.06s/it]
 54%|█████▍    | 120992/225000 [247:59:10<141:36:11,  4.90s/it]
 54%|█████▍    | 120993/225000 [247:59:16<157:15:21,  5.44s/it]
 54%|█████▍    | 120994/225000 [247:59:28<210:08:55,  7.27s/it]
 54%|█████▍    | 120995/225000 [247:59:32<184:20:41,  6.38s/it]
 54%|█████▍    | 120996/225000 [247:59:36<163:10:06,  5.65s/it]
 54%|█████▍    | 120997/225000 [247:59:49<227:14:43,  7.87s/it]
 54%|█████▍    | 120998/225000 [247:59:53<191:00:01,  6.61s/it]
 54%|█████▍    | 120999/225000 [247:59:57<168:37:13,  5.84s/it]
 54%|█████▍    | 121000/225000 [248:00:04<175:56:12,  6.09s/it]
                                                               
{'loss': '0.8642', 'grad_norm': '1.042', 'learning_rate': '2.374e-05', 'epoch': '0.5378', 'num_input_tokens_seen': 413319544, 'train_runtime': '8.928e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121000/225000 [248:00:04<175:56:12,  6.09s/it]
 54%|█████▍    | 121001/225000 [248:00:11<189:15:14,  6.55s/it]
 54%|█████▍    | 121002/225000 [248:00:14<159:23:33,  5.52s/it]
 54%|█████▍    | 121003/225000 [248:00:19<153:14:56,  5.30s/it]
 54%|█████▍    | 121004/225000 [248:00:24<148:05:14,  5.13s/it]
 54%|█████▍    | 121005/225000 [248:00:28<142:44:04,  4.94s/it]
 54%|█████▍    | 121006/225000 [248:00:33<138:41:30,  4.80s/it]
 54%|█████▍    | 121007/225000 [248:00:37<135:10:10,  4.68s/it]
 54%|█████▍    | 121008/225000 [248:00:42<137:55:47,  4.77s/it]
 54%|█████▍    | 121009/225000 [248:00:53<192:17:57,  6.66s/it]
 54%|█████▍    | 121010/225000 [248:01:00<195:07:30,  6.75s/it]
 54%|█████▍    | 121011/225000 [248:01:04<171:09:37,  5.93s/it]
 54%|█████▍    | 121012/225000 [248:01:08<156:05:11,  5.40s/it]
 54%|█████▍    | 121013/225000 [248:01:20<208:51:53,  7.23s/it]
 54%|█████▍    | 121014/225000 [248:01:26<195:55:55,  6.78s/it]
 54%|█████▍    | 121015/225000 [248:01:31<187:10:17,  6.48s/it]
 54%|█████▍    | 121016/225000 [248:01:36<172:06:24,  5.96s/it]
 54%|█████▍    | 121017/225000 [248:01:41<159:45:03,  5.53s/it]
 54%|█████▍    | 121018/225000 [248:01:45<152:45:19,  5.29s/it]
 54%|█████▍    | 121019/225000 [248:01:50<148:49:15,  5.15s/it]
 54%|█████▍    | 121020/225000 [248:01:55<146:47:08,  5.08s/it]
 54%|█████▍    | 121021/225000 [248:02:06<198:41:26,  6.88s/it]
 54%|█████▍    | 121022/225000 [248:02:10<169:35:52,  5.87s/it]
 54%|█████▍    | 121023/225000 [248:02:14<156:29:46,  5.42s/it]
 54%|█████▍    | 121024/225000 [248:02:18<146:48:19,  5.08s/it]
 54%|█████▍    | 121025/225000 [248:02:25<164:32:20,  5.70s/it]
 54%|█████▍    | 121026/225000 [248:02:34<191:26:34,  6.63s/it]
 54%|█████▍    | 121027/225000 [248:02:38<168:23:01,  5.83s/it]
 54%|█████▍    | 121028/225000 [248:02:44<168:14:42,  5.83s/it]
 54%|█████▍    | 121029/225000 [248:02:51<178:42:20,  6.19s/it]
 54%|█████▍    | 121030/225000 [248:02:55<159:05:35,  5.51s/it]
 54%|█████▍    | 121031/225000 [248:03:00<153:42:19,  5.32s/it]
 54%|█████▍    | 121032/225000 [248:03:05<151:41:09,  5.25s/it]
 54%|█████▍    | 121033/225000 [248:03:10<151:12:33,  5.24s/it]
 54%|█████▍    | 121034/225000 [248:03:15<146:39:30,  5.08s/it]
 54%|█████▍    | 121035/225000 [248:03:21<153:00:35,  5.30s/it]
 54%|█████▍    | 121036/225000 [248:03:32<201:40:01,  6.98s/it]
 54%|█████▍    | 121037/225000 [248:03:36<179:35:27,  6.22s/it]
 54%|█████▍    | 121038/225000 [248:03:41<166:01:25,  5.75s/it]
 54%|█████▍    | 121039/225000 [248:03:45<156:21:49,  5.41s/it]
 54%|█████▍    | 121040/225000 [248:03:50<149:54:06,  5.19s/it]
 54%|█████▍    | 121041/225000 [248:03:54<138:09:44,  4.78s/it]
 54%|█████▍    | 121042/225000 [248:04:00<153:19:15,  5.31s/it]
 54%|█████▍    | 121043/225000 [248:04:12<204:01:29,  7.07s/it]
 54%|█████▍    | 121044/225000 [248:04:15<171:17:41,  5.93s/it]
 54%|█████▍    | 121045/225000 [248:04:19<156:09:49,  5.41s/it]
 54%|█████▍    | 121046/225000 [248:04:23<144:29:11,  5.00s/it]
 54%|█████▍    | 121047/225000 [248:04:34<197:43:31,  6.85s/it]
 54%|█████▍    | 121048/225000 [248:04:39<181:31:25,  6.29s/it]
 54%|█████▍    | 121049/225000 [248:04:44<168:46:33,  5.84s/it]
 54%|█████▍    | 121050/225000 [248:04:49<163:47:24,  5.67s/it]
                                                               
{'loss': '0.8623', 'grad_norm': '0.9964', 'learning_rate': '2.372e-05', 'epoch': '0.538', 'num_input_tokens_seen': 413446160, 'train_runtime': '8.931e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121050/225000 [248:04:49<163:47:24,  5.67s/it]
 54%|█████▍    | 121051/225000 [248:04:54<152:30:36,  5.28s/it]
 54%|█████▍    | 121052/225000 [248:04:59<152:37:42,  5.29s/it]
 54%|█████▍    | 121053/225000 [248:05:04<151:19:21,  5.24s/it]
 54%|█████▍    | 121054/225000 [248:05:09<145:45:37,  5.05s/it]
 54%|█████▍    | 121055/225000 [248:05:14<146:16:14,  5.07s/it]
 54%|█████▍    | 121056/225000 [248:05:20<155:07:39,  5.37s/it]
 54%|█████▍    | 121057/225000 [248:05:33<222:22:21,  7.70s/it]
 54%|█████▍    | 121058/225000 [248:05:38<196:15:56,  6.80s/it]
 54%|█████▍    | 121059/225000 [248:05:42<176:23:52,  6.11s/it]
 54%|█████▍    | 121060/225000 [248:05:46<154:03:45,  5.34s/it]
 54%|█████▍    | 121061/225000 [248:05:50<146:07:02,  5.06s/it]
 54%|█████▍    | 121062/225000 [248:05:57<163:24:44,  5.66s/it]
 54%|█████▍    | 121063/225000 [248:06:06<192:27:15,  6.67s/it]
 54%|█████▍    | 121064/225000 [248:06:11<171:53:54,  5.95s/it]
 54%|█████▍    | 121065/225000 [248:06:15<158:35:08,  5.49s/it]
 54%|█████▍    | 121066/225000 [248:06:19<150:19:37,  5.21s/it]
 54%|█████▍    | 121067/225000 [248:06:23<139:19:53,  4.83s/it]
 54%|█████▍    | 121068/225000 [248:06:28<137:58:58,  4.78s/it]
 54%|█████▍    | 121069/225000 [248:06:33<137:54:21,  4.78s/it]
 54%|█████▍    | 121070/225000 [248:06:37<135:53:33,  4.71s/it]
 54%|█████▍    | 121071/225000 [248:06:48<190:14:21,  6.59s/it]
 54%|█████▍    | 121072/225000 [248:06:54<183:20:54,  6.35s/it]
 54%|█████▍    | 121073/225000 [248:06:58<157:39:51,  5.46s/it]
 54%|█████▍    | 121074/225000 [248:07:08<198:46:43,  6.89s/it]
 54%|█████▍    | 121075/225000 [248:07:11<169:32:18,  5.87s/it]
 54%|█████▍    | 121076/225000 [248:07:18<173:50:27,  6.02s/it]
 54%|█████▍    | 121077/225000 [248:07:24<177:38:29,  6.15s/it]
 54%|█████▍    | 121078/225000 [248:07:28<160:39:15,  5.57s/it]
 54%|█████▍    | 121079/225000 [248:07:33<148:43:50,  5.15s/it]
 54%|█████▍    | 121080/225000 [248:07:37<142:43:52,  4.94s/it]
 54%|█████▍    | 121081/225000 [248:07:45<171:39:48,  5.95s/it]
 54%|█████▍    | 121082/225000 [248:07:53<188:42:11,  6.54s/it]
 54%|█████▍    | 121083/225000 [248:07:57<166:44:17,  5.78s/it]
 54%|█████▍    | 121084/225000 [248:08:01<153:35:08,  5.32s/it]
 54%|█████▍    | 121085/225000 [248:08:06<142:50:34,  4.95s/it]
 54%|█████▍    | 121086/225000 [248:08:10<140:05:05,  4.85s/it]
 54%|█████▍    | 121087/225000 [248:08:15<137:39:33,  4.77s/it]
 54%|█████▍    | 121088/225000 [248:08:24<176:28:50,  6.11s/it]
 54%|█████▍    | 121089/225000 [248:08:34<207:31:45,  7.19s/it]
 54%|█████▍    | 121090/225000 [248:08:37<173:08:07,  6.00s/it]
 54%|█████▍    | 121091/225000 [248:08:44<185:13:23,  6.42s/it]
 54%|█████▍    | 121092/225000 [248:08:48<162:00:38,  5.61s/it]
 54%|█████▍    | 121093/225000 [248:08:52<149:32:36,  5.18s/it]
 54%|█████▍    | 121094/225000 [248:08:56<135:21:45,  4.69s/it]
 54%|█████▍    | 121095/225000 [248:09:06<183:21:53,  6.35s/it]
 54%|█████▍    | 121096/225000 [248:09:10<162:54:31,  5.64s/it]
 54%|█████▍    | 121097/225000 [248:09:16<168:04:26,  5.82s/it]
 54%|█████▍    | 121098/225000 [248:09:20<146:37:48,  5.08s/it]
 54%|█████▍    | 121099/225000 [248:09:28<178:53:47,  6.20s/it]
 54%|█████▍    | 121100/225000 [248:09:33<165:08:49,  5.72s/it]
                                                               
{'loss': '0.8664', 'grad_norm': '0.8911', 'learning_rate': '2.371e-05', 'epoch': '0.5382', 'num_input_tokens_seen': 413573656, 'train_runtime': '8.934e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121100/225000 [248:09:33<165:08:49,  5.72s/it]
 54%|█████▍    | 121101/225000 [248:09:37<153:39:01,  5.32s/it]
 54%|█████▍    | 121102/225000 [248:09:42<146:36:18,  5.08s/it]
 54%|█████▍    | 121103/225000 [248:09:47<143:38:23,  4.98s/it]
 54%|█████▍    | 121104/225000 [248:09:51<138:29:01,  4.80s/it]
 54%|█████▍    | 121105/225000 [248:09:55<133:43:07,  4.63s/it]
 54%|█████▍    | 121106/225000 [248:10:07<194:44:26,  6.75s/it]
 54%|█████▍    | 121107/225000 [248:10:10<164:49:55,  5.71s/it]
 54%|█████▍    | 121108/225000 [248:10:18<178:41:30,  6.19s/it]
 54%|█████▍    | 121109/225000 [248:10:21<156:31:16,  5.42s/it]
 54%|█████▍    | 121110/225000 [248:10:27<163:07:27,  5.65s/it]
 54%|█████▍    | 121111/225000 [248:10:35<177:18:12,  6.14s/it]
 54%|█████▍    | 121112/225000 [248:10:39<157:54:00,  5.47s/it]
 54%|█████▍    | 121113/225000 [248:10:48<191:11:54,  6.63s/it]
 54%|█████▍    | 121114/225000 [248:10:52<170:55:34,  5.92s/it]
 54%|█████▍    | 121115/225000 [248:10:57<158:12:21,  5.48s/it]
 54%|█████▍    | 121116/225000 [248:11:01<145:34:02,  5.04s/it]
 54%|█████▍    | 121117/225000 [248:11:05<137:01:20,  4.75s/it]
 54%|█████▍    | 121118/225000 [248:11:09<130:45:18,  4.53s/it]
 54%|█████▍    | 121119/225000 [248:11:13<130:05:21,  4.51s/it]
 54%|█████▍    | 121120/225000 [248:11:20<146:40:46,  5.08s/it]
 54%|█████▍    | 121121/225000 [248:11:31<200:40:28,  6.95s/it]
 54%|█████▍    | 121122/225000 [248:11:36<187:30:36,  6.50s/it]
 54%|█████▍    | 121123/225000 [248:11:40<160:18:24,  5.56s/it]
 54%|█████▍    | 121124/225000 [248:11:46<170:14:08,  5.90s/it]
 54%|█████▍    | 121125/225000 [248:11:50<150:44:45,  5.22s/it]
 54%|█████▍    | 121126/225000 [248:11:54<139:25:28,  4.83s/it]
 54%|█████▍    | 121127/225000 [248:12:02<166:45:08,  5.78s/it]
 54%|█████▍    | 121128/225000 [248:12:09<179:21:29,  6.22s/it]
 54%|█████▍    | 121129/225000 [248:12:14<163:00:21,  5.65s/it]
 54%|█████▍    | 121130/225000 [248:12:18<149:32:10,  5.18s/it]
 54%|█████▍    | 121131/225000 [248:12:27<189:52:03,  6.58s/it]
 54%|█████▍    | 121132/225000 [248:12:32<172:14:17,  5.97s/it]
 54%|█████▍    | 121133/225000 [248:12:36<156:44:07,  5.43s/it]
 54%|█████▍    | 121134/225000 [248:12:41<150:12:41,  5.21s/it]
 54%|█████▍    | 121135/225000 [248:12:46<148:07:40,  5.13s/it]
 54%|█████▍    | 121136/225000 [248:12:57<198:12:04,  6.87s/it]
 54%|█████▍    | 121137/225000 [248:13:01<177:55:43,  6.17s/it]
 54%|█████▍    | 121138/225000 [248:13:07<177:56:46,  6.17s/it]
 54%|█████▍    | 121139/225000 [248:13:11<153:09:01,  5.31s/it]
 54%|█████▍    | 121140/225000 [248:13:17<161:01:51,  5.58s/it]
 54%|█████▍    | 121141/225000 [248:13:25<180:08:40,  6.24s/it]
 54%|█████▍    | 121142/225000 [248:13:28<158:21:14,  5.49s/it]
 54%|█████▍    | 121143/225000 [248:13:34<156:52:45,  5.44s/it]
 54%|█████▍    | 121144/225000 [248:13:39<155:25:18,  5.39s/it]
 54%|█████▍    | 121145/225000 [248:13:44<153:26:10,  5.32s/it]
 54%|█████▍    | 121146/225000 [248:13:49<150:30:07,  5.22s/it]
 54%|█████▍    | 121147/225000 [248:13:54<149:50:26,  5.19s/it]
 54%|█████▍    | 121148/225000 [248:14:00<150:38:15,  5.22s/it]
 54%|█████▍    | 121149/225000 [248:14:05<149:16:33,  5.17s/it]
 54%|█████▍    | 121150/225000 [248:14:16<204:17:10,  7.08s/it]
                                                               
{'loss': '0.8459', 'grad_norm': '0.9886', 'learning_rate': '2.369e-05', 'epoch': '0.5384', 'num_input_tokens_seen': 413699128, 'train_runtime': '8.937e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121150/225000 [248:14:16<204:17:10,  7.08s/it]
 54%|█████▍    | 121151/225000 [248:14:22<189:11:45,  6.56s/it]
 54%|█████▍    | 121152/225000 [248:14:26<169:05:43,  5.86s/it]
 54%|█████▍    | 121153/225000 [248:14:30<155:49:21,  5.40s/it]
 54%|█████▍    | 121154/225000 [248:14:40<190:30:39,  6.60s/it]
 54%|█████▍    | 121155/225000 [248:14:45<179:59:23,  6.24s/it]
 54%|█████▍    | 121156/225000 [248:14:48<156:53:33,  5.44s/it]
 54%|█████▍    | 121157/225000 [248:14:59<199:10:35,  6.90s/it]
 54%|█████▍    | 121158/225000 [248:15:03<173:42:47,  6.02s/it]
 54%|█████▍    | 121159/225000 [248:15:07<154:47:31,  5.37s/it]
 54%|█████▍    | 121160/225000 [248:15:10<139:46:43,  4.85s/it]
 54%|█████▍    | 121161/225000 [248:15:14<132:23:48,  4.59s/it]
 54%|█████▍    | 121162/225000 [248:15:19<129:39:46,  4.50s/it]
 54%|█████▍    | 121163/225000 [248:15:30<192:34:53,  6.68s/it]
 54%|█████▍    | 121164/225000 [248:15:33<162:04:18,  5.62s/it]
 54%|█████▍    | 121165/225000 [248:15:42<183:45:45,  6.37s/it]
 54%|█████▍    | 121166/225000 [248:15:46<163:43:49,  5.68s/it]
 54%|█████▍    | 121167/225000 [248:15:49<144:33:57,  5.01s/it]
 54%|█████▍    | 121168/225000 [248:15:53<136:07:26,  4.72s/it]
 54%|█████▍    | 121169/225000 [248:16:03<182:26:06,  6.33s/it]
 54%|█████▍    | 121170/225000 [248:16:07<159:15:43,  5.52s/it]
 54%|█████▍    | 121171/225000 [248:16:11<150:11:57,  5.21s/it]
 54%|█████▍    | 121172/225000 [248:16:20<182:01:49,  6.31s/it]
 54%|█████▍    | 121173/225000 [248:16:24<164:38:57,  5.71s/it]
 54%|█████▍    | 121174/225000 [248:16:29<156:15:31,  5.42s/it]
 54%|█████▍    | 121175/225000 [248:16:34<149:27:39,  5.18s/it]
 54%|█████▍    | 121176/225000 [248:16:38<141:19:55,  4.90s/it]
 54%|█████▍    | 121177/225000 [248:16:47<177:42:55,  6.16s/it]
 54%|█████▍    | 121178/225000 [248:16:55<195:04:00,  6.76s/it]
 54%|█████▍    | 121179/225000 [248:16:59<165:08:51,  5.73s/it]
 54%|█████▍    | 121180/225000 [248:17:08<191:55:49,  6.66s/it]
 54%|█████▍    | 121181/225000 [248:17:11<163:41:25,  5.68s/it]
 54%|█████▍    | 121182/225000 [248:17:15<152:05:47,  5.27s/it]
 54%|█████▍    | 121183/225000 [248:17:19<143:12:32,  4.97s/it]
 54%|█████▍    | 121184/225000 [248:17:30<187:16:53,  6.49s/it]
 54%|█████▍    | 121185/225000 [248:17:33<161:20:34,  5.59s/it]
 54%|█████▍    | 121186/225000 [248:17:40<177:07:23,  6.14s/it]
 54%|█████▍    | 121187/225000 [248:17:44<156:04:57,  5.41s/it]
 54%|█████▍    | 121188/225000 [248:17:50<157:10:22,  5.45s/it]
 54%|█████▍    | 121189/225000 [248:17:58<182:28:37,  6.33s/it]
 54%|█████▍    | 121190/225000 [248:18:02<159:24:36,  5.53s/it]
 54%|█████▍    | 121191/225000 [248:18:07<158:02:41,  5.48s/it]
 54%|█████▍    | 121192/225000 [248:18:12<154:02:01,  5.34s/it]
 54%|█████▍    | 121193/225000 [248:18:17<150:05:59,  5.21s/it]
 54%|█████▍    | 121194/225000 [248:18:23<153:33:19,  5.33s/it]
 54%|█████▍    | 121195/225000 [248:18:28<157:34:22,  5.46s/it]
 54%|█████▍    | 121196/225000 [248:18:34<155:50:50,  5.40s/it]
 54%|█████▍    | 121197/225000 [248:18:38<149:49:09,  5.20s/it]
 54%|█████▍    | 121198/225000 [248:18:44<154:25:17,  5.36s/it]
 54%|█████▍    | 121199/225000 [248:18:50<155:45:52,  5.40s/it]
 54%|█████▍    | 121200/225000 [248:18:57<172:27:13,  5.98s/it]
                                                               
{'loss': '0.8411', 'grad_norm': '0.9444', 'learning_rate': '2.367e-05', 'epoch': '0.5387', 'num_input_tokens_seen': 413828144, 'train_runtime': '8.939e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121200/225000 [248:18:57<172:27:13,  5.98s/it]
 54%|█████▍    | 121201/225000 [248:19:10<234:12:41,  8.12s/it]
 54%|█████▍    | 121202/225000 [248:19:14<197:15:05,  6.84s/it]
 54%|█████▍    | 121203/225000 [248:19:19<183:35:10,  6.37s/it]
 54%|█████▍    | 121204/225000 [248:19:25<176:00:30,  6.10s/it]
 54%|█████▍    | 121205/225000 [248:19:30<166:04:04,  5.76s/it]
 54%|█████▍    | 121206/225000 [248:19:35<161:00:14,  5.58s/it]
 54%|█████▍    | 121207/225000 [248:19:40<159:33:13,  5.53s/it]
 54%|█████▍    | 121208/225000 [248:19:46<158:56:45,  5.51s/it]
 54%|█████▍    | 121209/225000 [248:19:51<157:34:14,  5.47s/it]
 54%|█████▍    | 121210/225000 [248:19:56<155:34:20,  5.40s/it]
 54%|█████▍    | 121211/225000 [248:20:01<152:14:04,  5.28s/it]
 54%|█████▍    | 121212/225000 [248:20:06<151:06:40,  5.24s/it]
 54%|█████▍    | 121213/225000 [248:20:11<148:25:05,  5.15s/it]
 54%|█████▍    | 121214/225000 [248:20:16<146:36:03,  5.09s/it]
 54%|█████▍    | 121215/225000 [248:20:25<174:03:20,  6.04s/it]
 54%|█████▍    | 121216/225000 [248:20:36<217:48:45,  7.56s/it]
 54%|█████▍    | 121217/225000 [248:20:40<189:21:01,  6.57s/it]
 54%|█████▍    | 121218/225000 [248:20:44<168:46:28,  5.85s/it]
 54%|█████▍    | 121219/225000 [248:20:49<156:26:48,  5.43s/it]
 54%|█████▍    | 121220/225000 [248:20:54<158:41:39,  5.50s/it]
 54%|█████▍    | 121221/225000 [248:21:05<204:37:54,  7.10s/it]
 54%|█████▍    | 121222/225000 [248:21:09<177:10:28,  6.15s/it]
 54%|█████▍    | 121223/225000 [248:21:13<158:04:13,  5.48s/it]
 54%|█████▍    | 121224/225000 [248:21:23<198:20:32,  6.88s/it]
 54%|█████▍    | 121225/225000 [248:21:27<170:12:48,  5.90s/it]
 54%|█████▍    | 121226/225000 [248:21:30<151:02:20,  5.24s/it]
 54%|█████▍    | 121227/225000 [248:21:39<176:05:31,  6.11s/it]
 54%|█████▍    | 121228/225000 [248:21:46<186:48:23,  6.48s/it]
 54%|█████▍    | 121229/225000 [248:21:49<162:04:22,  5.62s/it]
 54%|█████▍    | 121230/225000 [248:21:56<171:52:49,  5.96s/it]
 54%|█████▍    | 121231/225000 [248:22:01<165:36:08,  5.75s/it]
 54%|█████▍    | 121232/225000 [248:22:05<144:32:47,  5.01s/it]
 54%|█████▍    | 121233/225000 [248:22:14<181:38:53,  6.30s/it]
 54%|█████▍    | 121234/225000 [248:22:17<155:32:48,  5.40s/it]
 54%|█████▍    | 121235/225000 [248:22:21<139:44:59,  4.85s/it]
 54%|█████▍    | 121236/225000 [248:22:25<131:33:36,  4.56s/it]
 54%|█████▍    | 121237/225000 [248:22:29<129:41:16,  4.50s/it]
 54%|█████▍    | 121238/225000 [248:22:34<130:51:27,  4.54s/it]
 54%|█████▍    | 121239/225000 [248:22:46<198:00:49,  6.87s/it]
 54%|█████▍    | 121240/225000 [248:22:50<171:31:55,  5.95s/it]
 54%|█████▍    | 121241/225000 [248:22:54<155:13:03,  5.39s/it]
 54%|█████▍    | 121242/225000 [248:22:59<148:18:21,  5.15s/it]
 54%|█████▍    | 121243/225000 [248:23:07<176:05:27,  6.11s/it]
 54%|█████▍    | 121244/225000 [248:23:11<156:01:34,  5.41s/it]
 54%|█████▍    | 121245/225000 [248:23:22<205:28:30,  7.13s/it]
 54%|█████▍    | 121246/225000 [248:23:27<189:00:39,  6.56s/it]
 54%|█████▍    | 121247/225000 [248:23:32<178:19:50,  6.19s/it]
 54%|█████▍    | 121248/225000 [248:23:37<166:03:43,  5.76s/it]
 54%|█████▍    | 121249/225000 [248:23:44<172:06:54,  5.97s/it]
 54%|█████▍    | 121250/225000 [248:23:47<153:18:36,  5.32s/it]
                                                               
{'loss': '0.8551', 'grad_norm': '1.005', 'learning_rate': '2.365e-05', 'epoch': '0.5389', 'num_input_tokens_seen': 413954792, 'train_runtime': '8.942e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121250/225000 [248:23:47<153:18:36,  5.32s/it]
 54%|█████▍    | 121251/225000 [248:23:53<157:14:30,  5.46s/it]
 54%|█████▍    | 121252/225000 [248:24:00<167:19:54,  5.81s/it]
 54%|█████▍    | 121253/225000 [248:24:03<148:27:41,  5.15s/it]
 54%|█████▍    | 121254/225000 [248:24:08<139:32:02,  4.84s/it]
 54%|█████▍    | 121255/225000 [248:24:12<132:10:52,  4.59s/it]
 54%|█████▍    | 121256/225000 [248:24:23<194:34:29,  6.75s/it]
 54%|█████▍    | 121257/225000 [248:24:27<167:24:30,  5.81s/it]
 54%|█████▍    | 121258/225000 [248:24:31<148:06:14,  5.14s/it]
 54%|█████▍    | 121259/225000 [248:24:35<140:19:41,  4.87s/it]
 54%|█████▍    | 121260/225000 [248:24:40<140:24:43,  4.87s/it]
 54%|█████▍    | 121261/225000 [248:24:44<139:16:19,  4.83s/it]
 54%|█████▍    | 121262/225000 [248:24:49<136:06:00,  4.72s/it]
 54%|█████▍    | 121263/225000 [248:24:54<139:27:07,  4.84s/it]
 54%|█████▍    | 121264/225000 [248:24:58<136:11:04,  4.73s/it]
 54%|█████▍    | 121265/225000 [248:25:09<183:06:00,  6.35s/it]
 54%|█████▍    | 121266/225000 [248:25:20<224:05:44,  7.78s/it]
 54%|█████▍    | 121267/225000 [248:25:24<192:57:45,  6.70s/it]
 54%|█████▍    | 121268/225000 [248:25:28<174:49:29,  6.07s/it]
 54%|█████▍    | 121269/225000 [248:25:33<162:46:36,  5.65s/it]
 54%|█████▍    | 121270/225000 [248:25:37<150:45:44,  5.23s/it]
 54%|█████▍    | 121271/225000 [248:25:42<143:01:03,  4.96s/it]
 54%|█████▍    | 121272/225000 [248:25:48<155:11:13,  5.39s/it]
 54%|█████▍    | 121273/225000 [248:25:59<199:09:17,  6.91s/it]
 54%|█████▍    | 121274/225000 [248:26:02<169:34:16,  5.89s/it]
 54%|█████▍    | 121275/225000 [248:26:07<159:28:56,  5.54s/it]
 54%|█████▍    | 121276/225000 [248:26:11<151:30:53,  5.26s/it]
 54%|█████▍    | 121277/225000 [248:26:16<142:38:18,  4.95s/it]
 54%|█████▍    | 121278/225000 [248:26:20<141:22:56,  4.91s/it]
 54%|█████▍    | 121279/225000 [248:26:25<135:31:56,  4.70s/it]
 54%|█████▍    | 121280/225000 [248:26:31<152:08:19,  5.28s/it]
 54%|█████▍    | 121281/225000 [248:26:44<212:54:20,  7.39s/it]
 54%|█████▍    | 121282/225000 [248:26:47<179:55:53,  6.25s/it]
 54%|█████▍    | 121283/225000 [248:26:51<158:27:05,  5.50s/it]
 54%|█████▍    | 121284/225000 [248:26:59<180:20:37,  6.26s/it]
 54%|█████▍    | 121285/225000 [248:27:05<180:17:34,  6.26s/it]
 54%|█████▍    | 121286/225000 [248:27:09<160:59:11,  5.59s/it]
 54%|█████▍    | 121287/225000 [248:27:17<181:52:35,  6.31s/it]
 54%|█████▍    | 121288/225000 [248:27:22<168:01:45,  5.83s/it]
 54%|█████▍    | 121289/225000 [248:27:25<143:49:50,  4.99s/it]
 54%|█████▍    | 121290/225000 [248:27:33<168:51:00,  5.86s/it]
 54%|█████▍    | 121291/225000 [248:27:38<162:18:21,  5.63s/it]
 54%|█████▍    | 121292/225000 [248:27:45<176:58:25,  6.14s/it]
 54%|█████▍    | 121293/225000 [248:27:49<156:37:57,  5.44s/it]
 54%|█████▍    | 121294/225000 [248:27:52<138:10:13,  4.80s/it]
 54%|█████▍    | 121295/225000 [248:27:57<138:11:57,  4.80s/it]
 54%|█████▍    | 121296/225000 [248:28:07<184:29:58,  6.40s/it]
 54%|█████▍    | 121297/225000 [248:28:11<160:07:41,  5.56s/it]
 54%|█████▍    | 121298/225000 [248:28:18<171:59:05,  5.97s/it]
 54%|█████▍    | 121299/225000 [248:28:24<175:21:21,  6.09s/it]
 54%|█████▍    | 121300/225000 [248:28:31<184:34:09,  6.41s/it]
                                                               
{'loss': '0.8516', 'grad_norm': '1.036', 'learning_rate': '2.363e-05', 'epoch': '0.5391', 'num_input_tokens_seen': 414081720, 'train_runtime': '8.945e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121300/225000 [248:28:31<184:34:09,  6.41s/it]
 54%|█████▍    | 121301/225000 [248:28:37<173:09:27,  6.01s/it]
 54%|█████▍    | 121302/225000 [248:28:41<164:05:53,  5.70s/it]
 54%|█████▍    | 121303/225000 [248:28:46<154:06:46,  5.35s/it]
 54%|█████▍    | 121304/225000 [248:28:51<147:11:55,  5.11s/it]
 54%|█████▍    | 121305/225000 [248:28:55<144:32:07,  5.02s/it]
 54%|█████▍    | 121306/225000 [248:29:00<139:24:43,  4.84s/it]
 54%|█████▍    | 121307/225000 [248:29:04<135:02:14,  4.69s/it]
 54%|█████▍    | 121308/225000 [248:29:09<134:15:06,  4.66s/it]
 54%|█████▍    | 121309/225000 [248:29:20<187:54:31,  6.52s/it]
 54%|█████▍    | 121310/225000 [248:29:27<193:32:22,  6.72s/it]
 54%|█████▍    | 121311/225000 [248:29:30<165:13:37,  5.74s/it]
 54%|█████▍    | 121312/225000 [248:29:34<150:24:53,  5.22s/it]
 54%|█████▍    | 121313/225000 [248:29:44<191:04:10,  6.63s/it]
 54%|█████▍    | 121314/225000 [248:29:48<166:11:38,  5.77s/it]
 54%|█████▍    | 121315/225000 [248:29:51<145:43:03,  5.06s/it]
 54%|█████▍    | 121316/225000 [248:30:03<205:09:37,  7.12s/it]
 54%|█████▍    | 121317/225000 [248:30:07<178:56:59,  6.21s/it]
 54%|█████▍    | 121318/225000 [248:30:12<161:59:46,  5.62s/it]
 54%|█████▍    | 121319/225000 [248:30:16<155:27:07,  5.40s/it]
 54%|█████▍    | 121320/225000 [248:30:21<149:09:06,  5.18s/it]
 54%|█████▍    | 121321/225000 [248:30:26<148:09:28,  5.14s/it]
 54%|█████▍    | 121322/225000 [248:30:31<147:13:22,  5.11s/it]
 54%|█████▍    | 121323/225000 [248:30:36<143:47:09,  4.99s/it]
 54%|█████▍    | 121324/225000 [248:30:40<138:53:38,  4.82s/it]
 54%|█████▍    | 121325/225000 [248:30:45<133:30:16,  4.64s/it]
 54%|█████▍    | 121326/225000 [248:30:54<176:46:02,  6.14s/it]
 54%|█████▍    | 121327/225000 [248:31:02<192:52:27,  6.70s/it]
 54%|█████▍    | 121328/225000 [248:31:06<166:57:54,  5.80s/it]
 54%|█████▍    | 121329/225000 [248:31:10<151:14:16,  5.25s/it]
 54%|█████▍    | 121330/225000 [248:31:21<201:23:15,  6.99s/it]
 54%|█████▍    | 121331/225000 [248:31:25<175:06:12,  6.08s/it]
 54%|█████▍    | 121332/225000 [248:31:29<160:54:02,  5.59s/it]
 54%|█████▍    | 121333/225000 [248:31:37<177:45:12,  6.17s/it]
 54%|█████▍    | 121334/225000 [248:31:40<153:29:47,  5.33s/it]
 54%|█████▍    | 121335/225000 [248:31:47<162:12:25,  5.63s/it]
 54%|█████▍    | 121336/225000 [248:31:52<161:55:59,  5.62s/it]
 54%|█████▍    | 121337/225000 [248:31:56<143:46:04,  4.99s/it]
 54%|█████▍    | 121338/225000 [248:32:02<158:11:58,  5.49s/it]
 54%|█████▍    | 121339/225000 [248:32:09<171:40:11,  5.96s/it]
 54%|█████▍    | 121340/225000 [248:32:13<152:09:34,  5.28s/it]
 54%|█████▍    | 121341/225000 [248:32:21<171:51:04,  5.97s/it]
 54%|█████▍    | 121342/225000 [248:32:27<175:29:26,  6.09s/it]
 54%|█████▍    | 121343/225000 [248:32:32<163:15:57,  5.67s/it]
 54%|█████▍    | 121344/225000 [248:32:36<149:03:06,  5.18s/it]
 54%|█████▍    | 121345/225000 [248:32:40<139:56:48,  4.86s/it]
 54%|█████▍    | 121346/225000 [248:32:44<133:09:22,  4.62s/it]
 54%|█████▍    | 121347/225000 [248:32:52<158:29:38,  5.50s/it]
 54%|█████▍    | 121348/225000 [248:32:59<177:27:12,  6.16s/it]
 54%|█████▍    | 121349/225000 [248:33:03<155:21:41,  5.40s/it]
 54%|█████▍    | 121350/225000 [248:33:11<178:58:14,  6.22s/it]
                                                               
{'loss': '0.8473', 'grad_norm': '0.9825', 'learning_rate': '2.361e-05', 'epoch': '0.5393', 'num_input_tokens_seen': 414208208, 'train_runtime': '8.948e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121350/225000 [248:33:11<178:58:14,  6.22s/it]
 54%|█████▍    | 121351/225000 [248:33:17<179:48:37,  6.25s/it]
 54%|█████▍    | 121352/225000 [248:33:21<158:27:09,  5.50s/it]
 54%|█████▍    | 121353/225000 [248:33:29<177:44:10,  6.17s/it]
 54%|█████▍    | 121354/225000 [248:33:34<168:04:33,  5.84s/it]
 54%|█████▍    | 121355/225000 [248:33:37<148:56:26,  5.17s/it]
 54%|█████▍    | 121356/225000 [248:33:42<139:11:06,  4.83s/it]
 54%|█████▍    | 121357/225000 [248:33:46<131:49:01,  4.58s/it]
 54%|█████▍    | 121358/225000 [248:33:57<191:43:52,  6.66s/it]
 54%|█████▍    | 121359/225000 [248:34:01<170:13:52,  5.91s/it]
 54%|█████▍    | 121360/225000 [248:34:07<169:25:02,  5.88s/it]
 54%|█████▍    | 121361/225000 [248:34:14<180:12:50,  6.26s/it]
 54%|█████▍    | 121362/225000 [248:34:18<163:26:09,  5.68s/it]
 54%|█████▍    | 121363/225000 [248:34:23<152:28:00,  5.30s/it]
 54%|█████▍    | 121364/225000 [248:34:31<175:59:12,  6.11s/it]
 54%|█████▍    | 121365/225000 [248:34:37<177:02:34,  6.15s/it]
 54%|█████▍    | 121366/225000 [248:34:43<177:20:16,  6.16s/it]
 54%|█████▍    | 121367/225000 [248:34:47<155:41:07,  5.41s/it]
 54%|█████▍    | 121368/225000 [248:34:51<144:55:45,  5.03s/it]
 54%|█████▍    | 121369/225000 [248:34:55<133:35:46,  4.64s/it]
 54%|█████▍    | 121370/225000 [248:35:05<182:04:23,  6.33s/it]
 54%|█████▍    | 121371/225000 [248:35:09<156:56:18,  5.45s/it]
 54%|█████▍    | 121372/225000 [248:35:12<141:16:30,  4.91s/it]
 54%|█████▍    | 121373/225000 [248:35:21<171:10:30,  5.95s/it]
 54%|█████▍    | 121374/225000 [248:35:28<186:16:39,  6.47s/it]
 54%|█████▍    | 121375/225000 [248:35:32<162:04:32,  5.63s/it]
 54%|█████▍    | 121376/225000 [248:35:39<175:09:22,  6.09s/it]
 54%|█████▍    | 121377/225000 [248:35:47<188:53:31,  6.56s/it]
 54%|█████▍    | 121378/225000 [248:35:50<160:55:09,  5.59s/it]
 54%|█████▍    | 121379/225000 [248:35:56<166:24:42,  5.78s/it]
 54%|█████▍    | 121380/225000 [248:36:03<171:01:07,  5.94s/it]
 54%|█████▍    | 121381/225000 [248:36:07<154:40:58,  5.37s/it]
 54%|█████▍    | 121382/225000 [248:36:11<145:12:32,  5.04s/it]
 54%|█████▍    | 121383/225000 [248:36:15<138:36:39,  4.82s/it]
 54%|█████▍    | 121384/225000 [248:36:20<139:58:42,  4.86s/it]
 54%|█████▍    | 121385/225000 [248:36:28<166:40:30,  5.79s/it]
 54%|█████▍    | 121386/225000 [248:36:36<183:52:44,  6.39s/it]
 54%|█████▍    | 121387/225000 [248:36:40<162:25:36,  5.64s/it]
 54%|█████▍    | 121388/225000 [248:36:44<152:51:53,  5.31s/it]
 54%|█████▍    | 121389/225000 [248:36:49<144:25:24,  5.02s/it]
 54%|█████▍    | 121390/225000 [248:36:58<181:17:58,  6.30s/it]
 54%|█████▍    | 121391/225000 [248:37:06<194:46:13,  6.77s/it]
 54%|█████▍    | 121392/225000 [248:37:10<172:59:27,  6.01s/it]
 54%|█████▍    | 121393/225000 [248:37:14<156:33:19,  5.44s/it]
 54%|█████▍    | 121394/225000 [248:37:25<199:12:46,  6.92s/it]
 54%|█████▍    | 121395/225000 [248:37:28<168:29:13,  5.85s/it]
 54%|█████▍    | 121396/225000 [248:37:32<155:51:32,  5.42s/it]
 54%|█████▍    | 121397/225000 [248:37:38<154:46:59,  5.38s/it]
 54%|█████▍    | 121398/225000 [248:37:43<151:40:48,  5.27s/it]
 54%|█████▍    | 121399/225000 [248:37:48<153:14:38,  5.33s/it]
 54%|█████▍    | 121400/225000 [248:37:53<152:40:17,  5.31s/it]
                                                               
{'loss': '0.8572', 'grad_norm': '0.9792', 'learning_rate': '2.36e-05', 'epoch': '0.5396', 'num_input_tokens_seen': 414337400, 'train_runtime': '8.951e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121400/225000 [248:37:53<152:40:17,  5.31s/it]
 54%|█████▍    | 121401/225000 [248:37:59<152:18:17,  5.29s/it]
 54%|█████▍    | 121402/225000 [248:38:04<152:48:21,  5.31s/it]
 54%|█████▍    | 121403/225000 [248:38:09<152:19:32,  5.29s/it]
 54%|█████▍    | 121404/225000 [248:38:14<150:34:00,  5.23s/it]
 54%|█████▍    | 121405/225000 [248:38:19<145:18:47,  5.05s/it]
 54%|█████▍    | 121406/225000 [248:38:31<204:58:44,  7.12s/it]
 54%|█████▍    | 121407/225000 [248:38:38<205:00:39,  7.12s/it]
 54%|█████▍    | 121408/225000 [248:38:41<172:27:02,  5.99s/it]
 54%|█████▍    | 121409/225000 [248:38:45<149:18:56,  5.19s/it]
 54%|█████▍    | 121410/225000 [248:38:55<192:03:28,  6.67s/it]
 54%|█████▍    | 121411/225000 [248:38:58<158:16:33,  5.50s/it]
 54%|█████▍    | 121412/225000 [248:39:01<141:39:18,  4.92s/it]
 54%|█████▍    | 121413/225000 [248:39:09<170:59:08,  5.94s/it]
 54%|█████▍    | 121414/225000 [248:39:14<157:14:37,  5.46s/it]
 54%|█████▍    | 121415/225000 [248:39:18<144:48:16,  5.03s/it]
 54%|█████▍    | 121416/225000 [248:39:22<137:09:13,  4.77s/it]
 54%|█████▍    | 121417/225000 [248:39:27<136:17:16,  4.74s/it]
 54%|█████▍    | 121418/225000 [248:39:35<165:53:09,  5.77s/it]
 54%|█████▍    | 121419/225000 [248:39:45<200:14:43,  6.96s/it]
 54%|█████▍    | 121420/225000 [248:39:48<172:38:00,  6.00s/it]
 54%|█████▍    | 121421/225000 [248:39:53<161:52:55,  5.63s/it]
 54%|█████▍    | 121422/225000 [248:39:57<150:50:05,  5.24s/it]
 54%|█████▍    | 121423/225000 [248:40:02<145:08:20,  5.04s/it]
 54%|█████▍    | 121424/225000 [248:40:07<142:12:18,  4.94s/it]
 54%|█████▍    | 121425/225000 [248:40:12<141:11:12,  4.91s/it]
 54%|█████▍    | 121426/225000 [248:40:23<194:54:56,  6.77s/it]
 54%|█████▍    | 121427/225000 [248:40:28<179:11:44,  6.23s/it]
 54%|█████▍    | 121428/225000 [248:40:31<156:14:52,  5.43s/it]
 54%|█████▍    | 121429/225000 [248:40:42<198:27:44,  6.90s/it]
 54%|█████▍    | 121430/225000 [248:40:45<170:25:09,  5.92s/it]
 54%|█████▍    | 121431/225000 [248:40:53<186:14:22,  6.47s/it]
 54%|█████▍    | 121432/225000 [248:40:58<177:53:34,  6.18s/it]
 54%|█████▍    | 121433/225000 [248:41:02<157:03:51,  5.46s/it]
 54%|█████▍    | 121434/225000 [248:41:10<180:13:10,  6.26s/it]
 54%|█████▍    | 121435/225000 [248:41:14<155:08:02,  5.39s/it]
 54%|█████▍    | 121436/225000 [248:41:21<172:57:16,  6.01s/it]
 54%|█████▍    | 121437/225000 [248:41:25<153:30:06,  5.34s/it]
 54%|█████▍    | 121438/225000 [248:41:30<148:56:17,  5.18s/it]
 54%|█████▍    | 121439/225000 [248:41:35<152:29:05,  5.30s/it]
 54%|█████▍    | 121440/225000 [248:41:43<172:29:42,  6.00s/it]
 54%|█████▍    | 121441/225000 [248:41:48<167:51:40,  5.84s/it]
 54%|█████▍    | 121442/225000 [248:41:52<149:32:19,  5.20s/it]
 54%|█████▍    | 121443/225000 [248:42:00<170:30:57,  5.93s/it]
 54%|█████▍    | 121444/225000 [248:42:07<179:10:58,  6.23s/it]
 54%|█████▍    | 121445/225000 [248:42:12<168:00:55,  5.84s/it]
 54%|█████▍    | 121446/225000 [248:42:17<161:03:26,  5.60s/it]
 54%|█████▍    | 121447/225000 [248:42:21<150:13:51,  5.22s/it]
 54%|█████▍    | 121448/225000 [248:42:26<151:59:42,  5.28s/it]
 54%|█████▍    | 121449/225000 [248:42:32<151:30:16,  5.27s/it]
 54%|█████▍    | 121450/225000 [248:42:36<147:00:35,  5.11s/it]
                                                               
{'loss': '0.8553', 'grad_norm': '1.136', 'learning_rate': '2.358e-05', 'epoch': '0.5398', 'num_input_tokens_seen': 414464976, 'train_runtime': '8.954e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121450/225000 [248:42:36<147:00:35,  5.11s/it]
 54%|█████▍    | 121451/225000 [248:42:42<149:54:25,  5.21s/it]
 54%|█████▍    | 121452/225000 [248:42:46<144:44:43,  5.03s/it]
 54%|█████▍    | 121453/225000 [248:42:51<140:47:04,  4.89s/it]
 54%|█████▍    | 121454/225000 [248:42:56<145:25:14,  5.06s/it]
 54%|█████▍    | 121455/225000 [248:43:02<147:50:01,  5.14s/it]
 54%|█████▍    | 121456/225000 [248:43:07<147:01:12,  5.11s/it]
 54%|█████▍    | 121457/225000 [248:43:12<143:43:57,  5.00s/it]
 54%|█████▍    | 121458/225000 [248:43:17<144:39:36,  5.03s/it]
 54%|█████▍    | 121459/225000 [248:43:22<145:58:09,  5.08s/it]
 54%|█████▍    | 121460/225000 [248:43:35<213:10:11,  7.41s/it]
 54%|█████▍    | 121461/225000 [248:43:42<214:40:39,  7.46s/it]
 54%|█████▍    | 121462/225000 [248:43:46<180:49:47,  6.29s/it]
 54%|█████▍    | 121463/225000 [248:43:50<162:57:13,  5.67s/it]
 54%|█████▍    | 121464/225000 [248:43:54<150:24:23,  5.23s/it]
 54%|█████▍    | 121465/225000 [248:43:58<138:58:25,  4.83s/it]
 54%|█████▍    | 121466/225000 [248:44:03<135:06:01,  4.70s/it]
 54%|█████▍    | 121467/225000 [248:44:11<165:08:49,  5.74s/it]
 54%|█████▍    | 121468/225000 [248:44:20<192:39:45,  6.70s/it]
 54%|█████▍    | 121469/225000 [248:44:25<179:16:34,  6.23s/it]
 54%|█████▍    | 121470/225000 [248:44:31<179:04:27,  6.23s/it]
 54%|█████▍    | 121471/225000 [248:44:38<188:40:29,  6.56s/it]
 54%|█████▍    | 121472/225000 [248:44:42<165:33:10,  5.76s/it]
 54%|█████▍    | 121473/225000 [248:44:47<158:23:51,  5.51s/it]
 54%|█████▍    | 121474/225000 [248:44:56<183:54:44,  6.40s/it]
 54%|█████▍    | 121475/225000 [248:44:59<156:04:53,  5.43s/it]
 54%|█████▍    | 121476/225000 [248:45:06<173:15:45,  6.03s/it]
 54%|█████▍    | 121477/225000 [248:45:09<148:10:25,  5.15s/it]
 54%|█████▍    | 121478/225000 [248:45:17<167:16:04,  5.82s/it]
 54%|█████▍    | 121479/225000 [248:45:21<149:50:09,  5.21s/it]
 54%|█████▍    | 121480/225000 [248:45:24<134:41:21,  4.68s/it]
 54%|█████▍    | 121481/225000 [248:45:29<140:50:38,  4.90s/it]
 54%|█████▍    | 121482/225000 [248:45:41<199:37:08,  6.94s/it]
 54%|█████▍    | 121483/225000 [248:45:45<170:54:06,  5.94s/it]
 54%|█████▍    | 121484/225000 [248:45:49<153:28:30,  5.34s/it]
 54%|█████▍    | 121485/225000 [248:45:53<149:09:38,  5.19s/it]
 54%|█████▍    | 121486/225000 [248:46:02<182:00:11,  6.33s/it]
 54%|█████▍    | 121487/225000 [248:46:09<186:40:54,  6.49s/it]
 54%|█████▍    | 121488/225000 [248:46:14<169:21:25,  5.89s/it]
 54%|█████▍    | 121489/225000 [248:46:18<157:37:50,  5.48s/it]
 54%|█████▍    | 121490/225000 [248:46:29<203:59:32,  7.09s/it]
 54%|█████▍    | 121491/225000 [248:46:33<176:49:34,  6.15s/it]
 54%|█████▍    | 121492/225000 [248:46:43<211:01:20,  7.34s/it]
 54%|█████▍    | 121493/225000 [248:46:47<176:15:47,  6.13s/it]
 54%|█████▍    | 121494/225000 [248:46:52<169:56:03,  5.91s/it]
 54%|█████▍    | 121495/225000 [248:46:58<169:21:47,  5.89s/it]
 54%|█████▍    | 121496/225000 [248:47:04<171:37:11,  5.97s/it]
 54%|█████▍    | 121497/225000 [248:47:08<157:51:01,  5.49s/it]
 54%|█████▍    | 121498/225000 [248:47:13<149:44:59,  5.21s/it]
 54%|█████▍    | 121499/225000 [248:47:17<138:29:21,  4.82s/it]
 54%|█████▍    | 121500/225000 [248:47:23<154:09:18,  5.36s/it]
                                                               
{'loss': '0.8591', 'grad_norm': '1.09', 'learning_rate': '2.356e-05', 'epoch': '0.54', 'num_input_tokens_seen': 414593048, 'train_runtime': '8.956e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121500/225000 [248:47:23<154:09:18,  5.36s/it]
 54%|█████▍    | 121501/225000 [248:47:28<145:55:50,  5.08s/it]
 54%|█████▍    | 121502/225000 [248:47:34<154:50:25,  5.39s/it]
 54%|█████▍    | 121503/225000 [248:47:38<143:48:21,  5.00s/it]
 54%|█████▍    | 121504/225000 [248:47:45<161:36:57,  5.62s/it]
 54%|█████▍    | 121505/225000 [248:47:54<187:04:41,  6.51s/it]
 54%|█████▍    | 121506/225000 [248:47:57<157:34:09,  5.48s/it]
 54%|█████▍    | 121507/225000 [248:48:03<166:21:42,  5.79s/it]
 54%|█████▍    | 121508/225000 [248:48:08<159:11:02,  5.54s/it]
 54%|█████▍    | 121509/225000 [248:48:13<152:09:42,  5.29s/it]
 54%|█████▍    | 121510/225000 [248:48:17<144:45:49,  5.04s/it]
 54%|█████▍    | 121511/225000 [248:48:22<141:46:10,  4.93s/it]
 54%|█████▍    | 121512/225000 [248:48:27<139:53:38,  4.87s/it]
 54%|█████▍    | 121513/225000 [248:48:39<205:01:18,  7.13s/it]
 54%|█████▍    | 121514/225000 [248:48:45<191:15:29,  6.65s/it]
 54%|█████▍    | 121515/225000 [248:48:48<165:24:39,  5.75s/it]
 54%|█████▍    | 121516/225000 [248:48:57<187:12:28,  6.51s/it]
 54%|█████▍    | 121517/225000 [248:49:03<184:34:12,  6.42s/it]
 54%|█████▍    | 121518/225000 [248:49:09<183:50:07,  6.40s/it]
 54%|█████▍    | 121519/225000 [248:49:14<165:53:10,  5.77s/it]
 54%|█████▍    | 121520/225000 [248:49:18<154:16:19,  5.37s/it]
 54%|█████▍    | 121521/225000 [248:49:25<165:41:56,  5.76s/it]
 54%|█████▍    | 121522/225000 [248:49:28<144:55:41,  5.04s/it]
 54%|█████▍    | 121523/225000 [248:49:38<184:44:39,  6.43s/it]
 54%|█████▍    | 121524/225000 [248:49:41<155:21:19,  5.40s/it]
 54%|█████▍    | 121525/225000 [248:49:48<170:14:35,  5.92s/it]
 54%|█████▍    | 121526/225000 [248:49:52<153:11:44,  5.33s/it]
 54%|█████▍    | 121527/225000 [248:49:56<142:28:34,  4.96s/it]
 54%|█████▍    | 121528/225000 [248:50:01<140:53:04,  4.90s/it]
 54%|█████▍    | 121529/225000 [248:50:12<194:03:28,  6.75s/it]
 54%|█████▍    | 121530/225000 [248:50:15<164:14:27,  5.71s/it]
 54%|█████▍    | 121531/225000 [248:50:22<176:04:49,  6.13s/it]
 54%|█████▍    | 121532/225000 [248:50:26<159:37:23,  5.55s/it]
 54%|█████▍    | 121533/225000 [248:50:31<149:28:38,  5.20s/it]
 54%|█████▍    | 121534/225000 [248:50:35<137:37:09,  4.79s/it]
 54%|█████▍    | 121535/225000 [248:50:42<160:57:26,  5.60s/it]
 54%|█████▍    | 121536/225000 [248:50:47<152:26:43,  5.30s/it]
 54%|█████▍    | 121537/225000 [248:50:52<149:59:35,  5.22s/it]
 54%|█████▍    | 121538/225000 [248:50:57<147:47:47,  5.14s/it]
 54%|█████▍    | 121539/225000 [248:51:01<144:10:16,  5.02s/it]
 54%|█████▍    | 121540/225000 [248:51:06<142:18:08,  4.95s/it]
 54%|█████▍    | 121541/225000 [248:51:12<147:42:02,  5.14s/it]
 54%|█████▍    | 121542/225000 [248:51:23<200:32:47,  6.98s/it]
 54%|█████▍    | 121543/225000 [248:51:28<187:02:02,  6.51s/it]
 54%|█████▍    | 121544/225000 [248:51:32<161:09:54,  5.61s/it]
 54%|█████▍    | 121545/225000 [248:51:41<189:05:43,  6.58s/it]
 54%|█████▍    | 121546/225000 [248:51:44<162:17:20,  5.65s/it]
 54%|█████▍    | 121547/225000 [248:51:50<160:54:39,  5.60s/it]
 54%|█████▍    | 121548/225000 [248:51:57<175:41:14,  6.11s/it]
 54%|█████▍    | 121549/225000 [248:52:01<153:32:52,  5.34s/it]
 54%|█████▍    | 121550/225000 [248:52:09<182:14:40,  6.34s/it]
                                                               
{'loss': '0.8465', 'grad_norm': '1.099', 'learning_rate': '2.354e-05', 'epoch': '0.5402', 'num_input_tokens_seen': 414720712, 'train_runtime': '8.959e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121550/225000 [248:52:09<182:14:40,  6.34s/it]
 54%|█████▍    | 121551/225000 [248:52:13<158:00:14,  5.50s/it]
 54%|█████▍    | 121552/225000 [248:52:19<165:18:01,  5.75s/it]
 54%|█████▍    | 121553/225000 [248:52:29<202:54:50,  7.06s/it]
 54%|█████▍    | 121554/225000 [248:52:32<168:01:12,  5.85s/it]
 54%|█████▍    | 121555/225000 [248:52:37<157:14:18,  5.47s/it]
 54%|█████▍    | 121556/225000 [248:52:42<152:55:43,  5.32s/it]
 54%|█████▍    | 121557/225000 [248:52:47<153:32:59,  5.34s/it]
 54%|█████▍    | 121558/225000 [248:52:52<148:10:35,  5.16s/it]
 54%|█████▍    | 121559/225000 [248:52:57<148:05:12,  5.15s/it]
 54%|█████▍    | 121560/225000 [248:53:02<147:46:26,  5.14s/it]
 54%|█████▍    | 121561/225000 [248:53:07<146:47:12,  5.11s/it]
 54%|█████▍    | 121562/225000 [248:53:13<148:25:43,  5.17s/it]
 54%|█████▍    | 121563/225000 [248:53:18<152:40:53,  5.31s/it]
 54%|█████▍    | 121564/225000 [248:53:26<178:24:52,  6.21s/it]
 54%|█████▍    | 121565/225000 [248:53:31<167:16:08,  5.82s/it]
 54%|█████▍    | 121566/225000 [248:53:36<157:29:32,  5.48s/it]
 54%|█████▍    | 121567/225000 [248:53:42<157:17:07,  5.47s/it]
 54%|█████▍    | 121568/225000 [248:53:47<156:10:33,  5.44s/it]
 54%|█████▍    | 121569/225000 [248:53:52<150:09:52,  5.23s/it]
 54%|█████▍    | 121570/225000 [248:53:57<150:54:03,  5.25s/it]
 54%|█████▍    | 121571/225000 [248:54:02<148:18:43,  5.16s/it]
 54%|█████▍    | 121572/225000 [248:54:07<150:04:14,  5.22s/it]
 54%|█████▍    | 121573/225000 [248:54:13<154:30:07,  5.38s/it]
 54%|█████▍    | 121574/225000 [248:54:18<153:54:54,  5.36s/it]
 54%|█████▍    | 121575/225000 [248:54:25<167:04:26,  5.82s/it]
 54%|█████▍    | 121576/225000 [248:54:37<218:57:02,  7.62s/it]
 54%|█████▍    | 121577/225000 [248:54:41<183:27:33,  6.39s/it]
 54%|█████▍    | 121578/225000 [248:54:44<161:56:14,  5.64s/it]
 54%|█████▍    | 121579/225000 [248:54:52<176:16:00,  6.14s/it]
 54%|█████▍    | 121580/225000 [248:54:59<186:10:01,  6.48s/it]
 54%|█████▍    | 121581/225000 [248:55:03<168:37:03,  5.87s/it]
 54%|█████▍    | 121582/225000 [248:55:07<151:34:30,  5.28s/it]
 54%|█████▍    | 121583/225000 [248:55:11<141:10:12,  4.91s/it]
 54%|█████▍    | 121584/225000 [248:55:16<138:31:43,  4.82s/it]
 54%|█████▍    | 121585/225000 [248:55:21<136:05:00,  4.74s/it]
 54%|█████▍    | 121586/225000 [248:55:31<184:28:56,  6.42s/it]
 54%|█████▍    | 121587/225000 [248:55:39<195:11:36,  6.80s/it]
 54%|█████▍    | 121588/225000 [248:55:42<166:09:51,  5.78s/it]
 54%|█████▍    | 121589/225000 [248:55:50<183:47:56,  6.40s/it]
 54%|█████▍    | 121590/225000 [248:55:55<169:35:19,  5.90s/it]
 54%|█████▍    | 121591/225000 [248:56:00<164:46:18,  5.74s/it]
 54%|█████▍    | 121592/225000 [248:56:05<160:04:53,  5.57s/it]
 54%|█████▍    | 121593/225000 [248:56:10<153:58:59,  5.36s/it]
 54%|█████▍    | 121594/225000 [248:56:15<150:46:17,  5.25s/it]
 54%|█████▍    | 121595/225000 [248:56:20<151:06:13,  5.26s/it]
 54%|█████▍    | 121596/225000 [248:56:25<148:32:26,  5.17s/it]
 54%|█████▍    | 121597/225000 [248:56:31<150:28:43,  5.24s/it]
 54%|█████▍    | 121598/225000 [248:56:40<187:17:07,  6.52s/it]
 54%|█████▍    | 121599/225000 [248:56:46<184:00:43,  6.41s/it]
 54%|█████▍    | 121600/225000 [248:56:50<163:10:01,  5.68s/it]
                                                               
{'loss': '0.8461', 'grad_norm': '1.039', 'learning_rate': '2.352e-05', 'epoch': '0.5404', 'num_input_tokens_seen': 414848216, 'train_runtime': '8.962e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121600/225000 [248:56:50<163:10:01,  5.68s/it]
 54%|█████▍    | 121601/225000 [248:56:59<192:19:36,  6.70s/it]
 54%|█████▍    | 121602/225000 [248:57:03<166:55:25,  5.81s/it]
 54%|█████▍    | 121603/225000 [248:57:11<187:06:57,  6.51s/it]
 54%|█████▍    | 121604/225000 [248:57:15<161:25:28,  5.62s/it]
 54%|█████▍    | 121605/225000 [248:57:24<192:25:05,  6.70s/it]
 54%|█████▍    | 121606/225000 [248:57:28<166:16:27,  5.79s/it]
 54%|█████▍    | 121607/225000 [248:57:35<182:52:00,  6.37s/it]
 54%|█████▍    | 121608/225000 [248:57:40<170:54:59,  5.95s/it]
 54%|█████▍    | 121609/225000 [248:57:45<162:32:48,  5.66s/it]
 54%|█████▍    | 121610/225000 [248:57:50<158:15:37,  5.51s/it]
 54%|█████▍    | 121611/225000 [248:57:55<152:49:40,  5.32s/it]
 54%|█████▍    | 121612/225000 [248:58:00<148:47:33,  5.18s/it]
 54%|█████▍    | 121613/225000 [248:58:05<148:20:50,  5.17s/it]
 54%|█████▍    | 121614/225000 [248:58:11<148:54:22,  5.19s/it]
 54%|█████▍    | 121615/225000 [248:58:16<148:11:58,  5.16s/it]
 54%|█████▍    | 121616/225000 [248:58:24<178:35:16,  6.22s/it]
 54%|█████▍    | 121617/225000 [248:58:33<198:07:04,  6.90s/it]
 54%|█████▍    | 121618/225000 [248:58:36<168:34:37,  5.87s/it]
 54%|█████▍    | 121619/225000 [248:58:41<159:06:57,  5.54s/it]
 54%|█████▍    | 121620/225000 [248:58:46<150:59:27,  5.26s/it]
 54%|█████▍    | 121621/225000 [248:58:51<153:42:55,  5.35s/it]
 54%|█████▍    | 121622/225000 [248:58:56<149:17:17,  5.20s/it]
 54%|█████▍    | 121623/225000 [248:59:02<151:52:15,  5.29s/it]
 54%|█████▍    | 121624/225000 [248:59:07<152:23:33,  5.31s/it]
 54%|█████▍    | 121625/225000 [248:59:19<209:38:57,  7.30s/it]
 54%|█████▍    | 121626/225000 [248:59:25<196:42:55,  6.85s/it]
 54%|█████▍    | 121627/225000 [248:59:32<201:47:41,  7.03s/it]
 54%|█████▍    | 121628/225000 [248:59:36<178:21:13,  6.21s/it]
 54%|█████▍    | 121629/225000 [248:59:41<163:03:11,  5.68s/it]
 54%|█████▍    | 121630/225000 [248:59:46<154:38:16,  5.39s/it]
 54%|█████▍    | 121631/225000 [248:59:50<144:47:52,  5.04s/it]
 54%|█████▍    | 121632/225000 [248:59:55<141:49:07,  4.94s/it]
 54%|█████▍    | 121633/225000 [248:59:59<138:52:35,  4.84s/it]
 54%|█████▍    | 121634/225000 [249:00:04<136:16:32,  4.75s/it]
 54%|█████▍    | 121635/225000 [249:00:15<194:30:21,  6.77s/it]
 54%|█████▍    | 121636/225000 [249:00:21<184:24:40,  6.42s/it]
 54%|█████▍    | 121637/225000 [249:00:25<163:59:08,  5.71s/it]
 54%|█████▍    | 121638/225000 [249:00:29<154:46:09,  5.39s/it]
 54%|█████▍    | 121639/225000 [249:00:34<148:05:50,  5.16s/it]
 54%|█████▍    | 121640/225000 [249:00:39<145:46:19,  5.08s/it]
 54%|█████▍    | 121641/225000 [249:00:43<137:51:44,  4.80s/it]
 54%|█████▍    | 121642/225000 [249:00:51<166:49:49,  5.81s/it]
 54%|█████▍    | 121643/225000 [249:01:01<202:29:26,  7.05s/it]
 54%|█████▍    | 121644/225000 [249:01:04<169:54:54,  5.92s/it]
 54%|█████▍    | 121645/225000 [249:01:09<154:34:44,  5.38s/it]
 54%|█████▍    | 121646/225000 [249:01:15<163:53:33,  5.71s/it]
 54%|█████▍    | 121647/225000 [249:01:22<174:38:23,  6.08s/it]
 54%|█████▍    | 121648/225000 [249:01:26<152:04:38,  5.30s/it]
 54%|█████▍    | 121649/225000 [249:01:33<170:58:26,  5.96s/it]
 54%|█████▍    | 121650/225000 [249:01:37<153:06:23,  5.33s/it]
                                                               
{'loss': '0.8469', 'grad_norm': '1.197', 'learning_rate': '2.35e-05', 'epoch': '0.5407', 'num_input_tokens_seen': 414976856, 'train_runtime': '8.965e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121650/225000 [249:01:37<153:06:23,  5.33s/it]
 54%|█████▍    | 121651/225000 [249:01:41<141:02:15,  4.91s/it]
 54%|█████▍    | 121652/225000 [249:01:51<187:57:17,  6.55s/it]
 54%|█████▍    | 121653/225000 [249:01:54<157:07:20,  5.47s/it]
 54%|█████▍    | 121654/225000 [249:01:58<145:35:28,  5.07s/it]
 54%|█████▍    | 121655/225000 [249:02:02<135:50:55,  4.73s/it]
 54%|█████▍    | 121656/225000 [249:02:07<133:37:32,  4.65s/it]
 54%|█████▍    | 121657/225000 [249:02:11<134:40:37,  4.69s/it]
 54%|█████▍    | 121658/225000 [249:02:16<137:16:28,  4.78s/it]
 54%|█████▍    | 121659/225000 [249:02:28<196:21:32,  6.84s/it]
 54%|█████▍    | 121660/225000 [249:02:33<177:21:26,  6.18s/it]
 54%|█████▍    | 121661/225000 [249:02:38<167:34:13,  5.84s/it]
 54%|█████▍    | 121662/225000 [249:02:42<157:08:50,  5.47s/it]
 54%|█████▍    | 121663/225000 [249:02:47<152:13:56,  5.30s/it]
 54%|█████▍    | 121664/225000 [249:02:52<146:37:06,  5.11s/it]
 54%|█████▍    | 121665/225000 [249:02:56<137:50:49,  4.80s/it]
 54%|█████▍    | 121666/225000 [249:03:01<137:53:55,  4.80s/it]
 54%|█████▍    | 121667/225000 [249:03:06<139:31:19,  4.86s/it]
 54%|█████▍    | 121668/225000 [249:03:11<141:27:24,  4.93s/it]
 54%|█████▍    | 121669/225000 [249:03:16<142:43:46,  4.97s/it]
 54%|█████▍    | 121670/225000 [249:03:28<204:42:38,  7.13s/it]
 54%|█████▍    | 121671/225000 [249:03:33<182:50:47,  6.37s/it]
 54%|█████▍    | 121672/225000 [249:03:41<195:46:43,  6.82s/it]
 54%|█████▍    | 121673/225000 [249:03:44<166:55:45,  5.82s/it]
 54%|█████▍    | 121674/225000 [249:03:49<158:54:47,  5.54s/it]
 54%|█████▍    | 121675/225000 [249:03:54<152:39:39,  5.32s/it]
 54%|█████▍    | 121676/225000 [249:03:58<145:22:37,  5.07s/it]
 54%|█████▍    | 121677/225000 [249:04:03<143:39:14,  5.01s/it]
 54%|█████▍    | 121678/225000 [249:04:08<143:19:58,  4.99s/it]
 54%|█████▍    | 121679/225000 [249:04:14<146:57:20,  5.12s/it]
 54%|█████▍    | 121680/225000 [249:04:18<144:31:38,  5.04s/it]
 54%|█████▍    | 121681/225000 [249:04:23<144:45:14,  5.04s/it]
 54%|█████▍    | 121682/225000 [249:04:29<147:18:20,  5.13s/it]
 54%|█████▍    | 121683/225000 [249:04:40<202:10:59,  7.04s/it]
 54%|█████▍    | 121684/225000 [249:04:48<210:06:23,  7.32s/it]
 54%|█████▍    | 121685/225000 [249:04:58<229:37:09,  8.00s/it]
 54%|█████▍    | 121686/225000 [249:05:01<190:37:42,  6.64s/it]
 54%|█████▍    | 121687/225000 [249:05:09<200:23:06,  6.98s/it]
 54%|█████▍    | 121688/225000 [249:05:14<180:39:01,  6.29s/it]
 54%|█████▍    | 121689/225000 [249:05:18<165:10:36,  5.76s/it]
 54%|█████▍    | 121690/225000 [249:05:22<150:28:36,  5.24s/it]
 54%|█████▍    | 121691/225000 [249:05:27<141:35:54,  4.93s/it]
 54%|█████▍    | 121692/225000 [249:05:32<143:11:20,  4.99s/it]
 54%|█████▍    | 121693/225000 [249:05:44<206:56:21,  7.21s/it]
 54%|█████▍    | 121694/225000 [249:05:48<177:06:41,  6.17s/it]
 54%|█████▍    | 121695/225000 [249:05:52<164:06:48,  5.72s/it]
 54%|█████▍    | 121696/225000 [249:06:01<186:33:12,  6.50s/it]
 54%|█████▍    | 121697/225000 [249:06:05<163:08:06,  5.69s/it]
 54%|█████▍    | 121698/225000 [249:06:13<183:21:01,  6.39s/it]
 54%|█████▍    | 121699/225000 [249:06:16<159:45:45,  5.57s/it]
 54%|█████▍    | 121700/225000 [249:06:21<151:35:50,  5.28s/it]
                                                               
{'loss': '0.8522', 'grad_norm': '1.011', 'learning_rate': '2.349e-05', 'epoch': '0.5409', 'num_input_tokens_seen': 415103008, 'train_runtime': '8.968e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121700/225000 [249:06:21<151:35:50,  5.28s/it]
 54%|█████▍    | 121701/225000 [249:06:26<148:22:48,  5.17s/it]
 54%|█████▍    | 121702/225000 [249:06:31<148:11:11,  5.16s/it]
 54%|█████▍    | 121703/225000 [249:06:36<143:13:10,  4.99s/it]
 54%|█████▍    | 121704/225000 [249:06:40<138:36:37,  4.83s/it]
 54%|█████▍    | 121705/225000 [249:06:53<205:54:21,  7.18s/it]
 54%|█████▍    | 121706/225000 [249:06:59<200:06:39,  6.97s/it]
 54%|█████▍    | 121707/225000 [249:07:04<181:10:13,  6.31s/it]
 54%|█████▍    | 121708/225000 [249:07:09<173:42:40,  6.05s/it]
 54%|█████▍    | 121709/225000 [249:07:14<159:24:41,  5.56s/it]
 54%|█████▍    | 121710/225000 [249:07:19<156:07:52,  5.44s/it]
 54%|█████▍    | 121711/225000 [249:07:24<152:11:42,  5.30s/it]
 54%|█████▍    | 121712/225000 [249:07:29<154:29:12,  5.38s/it]
 54%|█████▍    | 121713/225000 [249:07:35<153:00:15,  5.33s/it]
 54%|█████▍    | 121714/225000 [249:07:40<150:47:37,  5.26s/it]
 54%|█████▍    | 121715/225000 [249:07:45<153:22:02,  5.35s/it]
 54%|█████▍    | 121716/225000 [249:07:51<152:32:01,  5.32s/it]
 54%|█████▍    | 121717/225000 [249:07:55<148:37:35,  5.18s/it]
 54%|█████▍    | 121718/225000 [249:08:01<150:01:18,  5.23s/it]
 54%|█████▍    | 121719/225000 [249:08:06<150:35:36,  5.25s/it]
 54%|█████▍    | 121720/225000 [249:08:12<153:43:09,  5.36s/it]
 54%|█████▍    | 121721/225000 [249:08:16<147:58:06,  5.16s/it]
 54%|█████▍    | 121722/225000 [249:08:23<160:08:11,  5.58s/it]
 54%|█████▍    | 121723/225000 [249:08:28<159:33:57,  5.56s/it]
 54%|█████▍    | 121724/225000 [249:08:33<154:24:38,  5.38s/it]
 54%|█████▍    | 121725/225000 [249:08:39<158:25:17,  5.52s/it]
 54%|█████▍    | 121726/225000 [249:08:44<154:04:28,  5.37s/it]
 54%|█████▍    | 121727/225000 [249:08:50<152:44:44,  5.32s/it]
 54%|█████▍    | 121728/225000 [249:09:01<202:14:48,  7.05s/it]
 54%|█████▍    | 121729/225000 [249:09:08<207:33:15,  7.24s/it]
 54%|█████▍    | 121730/225000 [249:09:11<173:05:44,  6.03s/it]
 54%|█████▍    | 121731/225000 [249:09:17<172:04:09,  6.00s/it]
 54%|█████▍    | 121732/225000 [249:09:23<166:57:40,  5.82s/it]
 54%|█████▍    | 121733/225000 [249:09:31<187:39:20,  6.54s/it]
 54%|█████▍    | 121734/225000 [249:09:35<167:48:54,  5.85s/it]
 54%|█████▍    | 121735/225000 [249:09:40<155:39:16,  5.43s/it]
 54%|█████▍    | 121736/225000 [249:09:49<189:05:19,  6.59s/it]
 54%|█████▍    | 121737/225000 [249:09:53<169:16:35,  5.90s/it]
 54%|█████▍    | 121738/225000 [249:10:01<181:42:44,  6.34s/it]
 54%|█████▍    | 121739/225000 [249:10:05<164:13:33,  5.73s/it]
 54%|█████▍    | 121740/225000 [249:10:09<149:56:08,  5.23s/it]
 54%|█████▍    | 121741/225000 [249:10:13<141:42:04,  4.94s/it]
 54%|█████▍    | 121742/225000 [249:10:22<177:08:25,  6.18s/it]
 54%|█████▍    | 121743/225000 [249:10:30<186:13:58,  6.49s/it]
 54%|█████▍    | 121744/225000 [249:10:33<163:52:25,  5.71s/it]
 54%|█████▍    | 121745/225000 [249:10:37<148:37:37,  5.18s/it]
 54%|█████▍    | 121746/225000 [249:10:47<183:57:47,  6.41s/it]
 54%|█████▍    | 121747/225000 [249:10:50<157:39:58,  5.50s/it]
 54%|█████▍    | 121748/225000 [249:10:55<151:46:08,  5.29s/it]
 54%|█████▍    | 121749/225000 [249:11:00<151:36:57,  5.29s/it]
 54%|█████▍    | 121750/225000 [249:11:05<150:36:12,  5.25s/it]
                                                               
{'loss': '0.8521', 'grad_norm': '1.043', 'learning_rate': '2.347e-05', 'epoch': '0.5411', 'num_input_tokens_seen': 415231288, 'train_runtime': '8.971e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121750/225000 [249:11:05<150:36:12,  5.25s/it]
 54%|█████▍    | 121751/225000 [249:11:11<150:01:12,  5.23s/it]
 54%|█████▍    | 121752/225000 [249:11:16<148:11:16,  5.17s/it]
 54%|█████▍    | 121753/225000 [249:11:21<152:57:44,  5.33s/it]
 54%|█████▍    | 121754/225000 [249:11:27<156:09:27,  5.44s/it]
 54%|█████▍    | 121755/225000 [249:11:38<201:04:55,  7.01s/it]
 54%|█████▍    | 121756/225000 [249:11:42<180:32:45,  6.30s/it]
 54%|█████▍    | 121757/225000 [249:11:47<169:15:20,  5.90s/it]
 54%|█████▍    | 121758/225000 [249:11:51<154:35:00,  5.39s/it]
 54%|█████▍    | 121759/225000 [249:11:56<148:05:10,  5.16s/it]
 54%|█████▍    | 121760/225000 [249:12:02<157:48:51,  5.50s/it]
 54%|█████▍    | 121761/225000 [249:12:14<211:18:21,  7.37s/it]
 54%|█████▍    | 121762/225000 [249:12:18<177:53:03,  6.20s/it]
 54%|█████▍    | 121763/225000 [249:12:22<166:35:26,  5.81s/it]
 54%|█████▍    | 121764/225000 [249:12:27<156:45:48,  5.47s/it]
 54%|█████▍    | 121765/225000 [249:12:37<191:41:36,  6.68s/it]
 54%|█████▍    | 121766/225000 [249:12:40<159:06:31,  5.55s/it]
 54%|█████▍    | 121767/225000 [249:12:43<145:05:25,  5.06s/it]
 54%|█████▍    | 121768/225000 [249:12:48<138:58:51,  4.85s/it]
 54%|█████▍    | 121769/225000 [249:12:54<149:30:24,  5.21s/it]
 54%|█████▍    | 121770/225000 [249:13:04<187:29:31,  6.54s/it]
 54%|█████▍    | 121771/225000 [249:13:07<160:42:06,  5.60s/it]
 54%|█████▍    | 121772/225000 [249:13:10<139:45:32,  4.87s/it]
 54%|█████▍    | 121773/225000 [249:13:18<168:17:45,  5.87s/it]
 54%|█████▍    | 121774/225000 [249:13:24<165:00:02,  5.75s/it]
 54%|█████▍    | 121775/225000 [249:13:29<156:14:52,  5.45s/it]
 54%|█████▍    | 121776/225000 [249:13:36<171:19:20,  5.97s/it]
 54%|█████▍    | 121777/225000 [249:13:42<174:48:47,  6.10s/it]
 54%|█████▍    | 121778/225000 [249:13:46<158:08:33,  5.52s/it]
 54%|█████▍    | 121779/225000 [249:13:51<154:47:36,  5.40s/it]
 54%|█████▍    | 121780/225000 [249:13:56<146:14:22,  5.10s/it]
 54%|█████▍    | 121781/225000 [249:14:01<145:35:58,  5.08s/it]
 54%|█████▍    | 121782/225000 [249:14:12<194:53:15,  6.80s/it]
 54%|█████▍    | 121783/225000 [249:14:17<181:34:15,  6.33s/it]
 54%|█████▍    | 121784/225000 [249:14:21<165:32:24,  5.77s/it]
 54%|█████▍    | 121785/225000 [249:14:26<152:44:27,  5.33s/it]
 54%|█████▍    | 121786/225000 [249:14:30<143:30:13,  5.01s/it]
 54%|█████▍    | 121787/225000 [249:14:34<138:18:20,  4.82s/it]
 54%|█████▍    | 121788/225000 [249:14:45<191:15:00,  6.67s/it]
 54%|█████▍    | 121789/225000 [249:14:51<183:31:55,  6.40s/it]
 54%|█████▍    | 121790/225000 [249:14:56<172:37:13,  6.02s/it]
 54%|█████▍    | 121791/225000 [249:15:04<187:40:37,  6.55s/it]
 54%|█████▍    | 121792/225000 [249:15:08<165:02:52,  5.76s/it]
 54%|█████▍    | 121793/225000 [249:15:12<154:00:27,  5.37s/it]
 54%|█████▍    | 121794/225000 [249:15:17<143:57:38,  5.02s/it]
 54%|█████▍    | 121795/225000 [249:15:21<139:51:02,  4.88s/it]
 54%|█████▍    | 121796/225000 [249:15:32<194:26:40,  6.78s/it]
 54%|█████▍    | 121797/225000 [249:15:38<185:48:26,  6.48s/it]
 54%|█████▍    | 121798/225000 [249:15:42<160:34:34,  5.60s/it]
 54%|█████▍    | 121799/225000 [249:15:47<160:22:37,  5.59s/it]
 54%|█████▍    | 121800/225000 [249:15:52<153:25:06,  5.35s/it]
                                                               
{'loss': '0.8447', 'grad_norm': '1.222', 'learning_rate': '2.345e-05', 'epoch': '0.5413', 'num_input_tokens_seen': 415359968, 'train_runtime': '8.974e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121800/225000 [249:15:52<153:25:06,  5.35s/it]
 54%|█████▍    | 121801/225000 [249:15:57<148:38:31,  5.19s/it]
 54%|█████▍    | 121802/225000 [249:16:02<148:23:57,  5.18s/it]
 54%|█████▍    | 121803/225000 [249:16:08<152:00:23,  5.30s/it]
 54%|█████▍    | 121804/225000 [249:16:14<162:41:51,  5.68s/it]
 54%|█████▍    | 121805/225000 [249:16:21<170:16:13,  5.94s/it]
 54%|█████▍    | 121806/225000 [249:16:28<178:19:23,  6.22s/it]
 54%|█████▍    | 121807/225000 [249:16:33<171:00:54,  5.97s/it]
 54%|█████▍    | 121808/225000 [249:16:38<163:32:08,  5.71s/it]
 54%|█████▍    | 121809/225000 [249:16:43<159:54:47,  5.58s/it]
 54%|█████▍    | 121810/225000 [249:16:50<166:12:43,  5.80s/it]
 54%|█████▍    | 121811/225000 [249:16:55<164:37:18,  5.74s/it]
 54%|█████▍    | 121812/225000 [249:17:05<199:02:30,  6.94s/it]
 54%|█████▍    | 121813/225000 [249:17:09<170:29:11,  5.95s/it]
 54%|█████▍    | 121814/225000 [249:17:16<180:46:27,  6.31s/it]
 54%|█████▍    | 121815/225000 [249:17:21<168:07:45,  5.87s/it]
 54%|█████▍    | 121816/225000 [249:17:25<157:01:19,  5.48s/it]
 54%|█████▍    | 121817/225000 [249:17:29<144:28:07,  5.04s/it]
 54%|█████▍    | 121818/225000 [249:17:35<154:33:35,  5.39s/it]
 54%|█████▍    | 121819/225000 [249:17:45<194:20:37,  6.78s/it]
 54%|█████▍    | 121820/225000 [249:17:49<166:07:51,  5.80s/it]
 54%|█████▍    | 121821/225000 [249:17:57<185:47:46,  6.48s/it]
 54%|█████▍    | 121822/225000 [249:18:01<162:16:06,  5.66s/it]
 54%|█████▍    | 121823/225000 [249:18:09<183:41:15,  6.41s/it]
 54%|█████▍    | 121824/225000 [249:18:12<159:18:57,  5.56s/it]
 54%|█████▍    | 121825/225000 [249:18:16<139:55:00,  4.88s/it]
 54%|█████▍    | 121826/225000 [249:18:20<137:32:59,  4.80s/it]
 54%|█████▍    | 121827/225000 [249:18:25<138:12:54,  4.82s/it]
 54%|█████▍    | 121828/225000 [249:18:30<138:26:34,  4.83s/it]
 54%|█████▍    | 121829/225000 [249:18:35<141:46:09,  4.95s/it]
 54%|█████▍    | 121830/225000 [249:18:40<138:55:46,  4.85s/it]
 54%|█████▍    | 121831/225000 [249:18:45<142:01:59,  4.96s/it]
 54%|█████▍    | 121832/225000 [249:18:56<191:26:51,  6.68s/it]
 54%|█████▍    | 121833/225000 [249:19:03<197:41:49,  6.90s/it]
 54%|█████▍    | 121834/225000 [249:19:08<176:54:40,  6.17s/it]
 54%|█████▍    | 121835/225000 [249:19:12<162:12:43,  5.66s/it]
 54%|█████▍    | 121836/225000 [249:19:17<153:14:56,  5.35s/it]
 54%|█████▍    | 121837/225000 [249:19:21<144:29:14,  5.04s/it]
 54%|█████▍    | 121838/225000 [249:19:26<139:50:09,  4.88s/it]
 54%|█████▍    | 121839/225000 [249:19:37<193:18:14,  6.75s/it]
 54%|█████▍    | 121840/225000 [249:19:42<184:34:19,  6.44s/it]
 54%|█████▍    | 121841/225000 [249:19:46<160:04:23,  5.59s/it]
 54%|█████▍    | 121842/225000 [249:19:53<175:55:29,  6.14s/it]
 54%|█████▍    | 121843/225000 [249:19:58<160:20:58,  5.60s/it]
 54%|█████▍    | 121844/225000 [249:20:03<158:58:23,  5.55s/it]
 54%|█████▍    | 121845/225000 [249:20:09<158:16:11,  5.52s/it]
 54%|█████▍    | 121846/225000 [249:20:14<157:52:23,  5.51s/it]
 54%|█████▍    | 121847/225000 [249:20:23<188:50:20,  6.59s/it]
 54%|█████▍    | 121848/225000 [249:20:28<170:23:51,  5.95s/it]
 54%|█████▍    | 121849/225000 [249:20:32<156:37:03,  5.47s/it]
 54%|█████▍    | 121850/225000 [249:20:36<147:06:03,  5.13s/it]
                                                               
{'loss': '0.8623', 'grad_norm': '1.113', 'learning_rate': '2.343e-05', 'epoch': '0.5416', 'num_input_tokens_seen': 415487800, 'train_runtime': '8.976e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121850/225000 [249:20:36<147:06:03,  5.13s/it]
 54%|█████▍    | 121851/225000 [249:20:41<140:19:51,  4.90s/it]
 54%|█████▍    | 121852/225000 [249:20:48<159:05:24,  5.55s/it]
 54%|█████▍    | 121853/225000 [249:20:56<185:19:03,  6.47s/it]
 54%|█████▍    | 121854/225000 [249:21:00<158:53:45,  5.55s/it]
 54%|█████▍    | 121855/225000 [249:21:08<182:24:11,  6.37s/it]
 54%|█████▍    | 121856/225000 [249:21:11<156:22:36,  5.46s/it]
 54%|█████▍    | 121857/225000 [249:21:19<175:41:47,  6.13s/it]
 54%|█████▍    | 121858/225000 [249:21:23<156:31:46,  5.46s/it]
 54%|█████▍    | 121859/225000 [249:21:28<154:08:50,  5.38s/it]
 54%|█████▍    | 121860/225000 [249:21:33<147:12:18,  5.14s/it]
 54%|█████▍    | 121861/225000 [249:21:38<146:02:36,  5.10s/it]
 54%|█████▍    | 121862/225000 [249:21:43<142:27:08,  4.97s/it]
 54%|█████▍    | 121863/225000 [249:21:54<200:39:14,  7.00s/it]
 54%|█████▍    | 121864/225000 [249:21:58<168:49:19,  5.89s/it]
 54%|█████▍    | 121865/225000 [249:22:01<149:42:49,  5.23s/it]
 54%|█████▍    | 121866/225000 [249:22:05<135:35:53,  4.73s/it]
 54%|█████▍    | 121867/225000 [249:22:16<188:31:17,  6.58s/it]
 54%|█████▍    | 121868/225000 [249:22:21<173:11:15,  6.05s/it]
 54%|█████▍    | 121869/225000 [249:22:26<164:45:06,  5.75s/it]
 54%|█████▍    | 121870/225000 [249:22:30<155:17:03,  5.42s/it]
 54%|█████▍    | 121871/225000 [249:22:35<152:25:31,  5.32s/it]
 54%|█████▍    | 121872/225000 [249:22:40<147:32:49,  5.15s/it]
 54%|█████▍    | 121873/225000 [249:22:45<146:21:07,  5.11s/it]
 54%|█████▍    | 121874/225000 [249:22:50<145:03:31,  5.06s/it]
 54%|█████▍    | 121875/225000 [249:23:00<190:42:41,  6.66s/it]
 54%|█████▍    | 121876/225000 [249:23:07<188:27:12,  6.58s/it]
 54%|█████▍    | 121877/225000 [249:23:11<164:22:20,  5.74s/it]
 54%|█████▍    | 121878/225000 [249:23:15<153:59:48,  5.38s/it]
 54%|█████▍    | 121879/225000 [249:23:20<148:51:04,  5.20s/it]
 54%|█████▍    | 121880/225000 [249:23:28<172:14:25,  6.01s/it]
 54%|█████▍    | 121881/225000 [249:23:36<192:02:25,  6.70s/it]
 54%|█████▍    | 121882/225000 [249:23:39<159:50:40,  5.58s/it]
 54%|█████▍    | 121883/225000 [249:23:44<151:56:45,  5.30s/it]
 54%|█████▍    | 121884/225000 [249:23:48<146:01:15,  5.10s/it]
 54%|█████▍    | 121885/225000 [249:23:53<142:39:26,  4.98s/it]
 54%|█████▍    | 121886/225000 [249:24:02<177:44:22,  6.21s/it]
 54%|█████▍    | 121887/225000 [249:24:10<189:22:50,  6.61s/it]
 54%|█████▍    | 121888/225000 [249:24:13<164:08:04,  5.73s/it]
 54%|█████▍    | 121889/225000 [249:24:24<207:13:50,  7.24s/it]
 54%|█████▍    | 121890/225000 [249:24:28<179:27:09,  6.27s/it]
 54%|█████▍    | 121891/225000 [249:24:32<157:19:02,  5.49s/it]
 54%|█████▍    | 121892/225000 [249:24:39<169:07:03,  5.90s/it]
 54%|█████▍    | 121893/225000 [249:24:45<170:55:24,  5.97s/it]
 54%|█████▍    | 121894/225000 [249:24:49<158:35:47,  5.54s/it]
 54%|█████▍    | 121895/225000 [249:24:54<154:47:46,  5.40s/it]
 54%|█████▍    | 121896/225000 [249:24:59<151:10:49,  5.28s/it]
 54%|█████▍    | 121897/225000 [249:25:04<143:00:11,  4.99s/it]
 54%|█████▍    | 121898/225000 [249:25:09<143:51:27,  5.02s/it]
 54%|█████▍    | 121899/225000 [249:25:13<140:03:13,  4.89s/it]
 54%|█████▍    | 121900/225000 [249:25:21<162:29:59,  5.67s/it]
                                                               
{'loss': '0.841', 'grad_norm': '0.9793', 'learning_rate': '2.341e-05', 'epoch': '0.5418', 'num_input_tokens_seen': 415615440, 'train_runtime': '8.979e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121900/225000 [249:25:21<162:29:59,  5.67s/it]
 54%|█████▍    | 121901/225000 [249:25:30<195:05:37,  6.81s/it]
 54%|█████▍    | 121902/225000 [249:25:36<180:35:37,  6.31s/it]
 54%|█████▍    | 121903/225000 [249:25:41<172:48:09,  6.03s/it]
 54%|█████▍    | 121904/225000 [249:25:46<165:17:58,  5.77s/it]
 54%|█████▍    | 121905/225000 [249:25:51<158:37:23,  5.54s/it]
 54%|█████▍    | 121906/225000 [249:25:56<157:12:11,  5.49s/it]
 54%|█████▍    | 121907/225000 [249:26:02<154:53:17,  5.41s/it]
 54%|█████▍    | 121908/225000 [249:26:06<148:32:56,  5.19s/it]
 54%|█████▍    | 121909/225000 [249:26:12<150:16:54,  5.25s/it]
 54%|█████▍    | 121910/225000 [249:26:17<151:31:18,  5.29s/it]
 54%|█████▍    | 121911/225000 [249:26:22<150:37:47,  5.26s/it]
 54%|█████▍    | 121912/225000 [249:26:27<146:28:04,  5.11s/it]
 54%|█████▍    | 121913/225000 [249:26:31<139:22:25,  4.87s/it]
 54%|█████▍    | 121914/225000 [249:26:36<137:32:17,  4.80s/it]
 54%|█████▍    | 121915/225000 [249:26:42<143:24:45,  5.01s/it]
 54%|█████▍    | 121916/225000 [249:26:47<146:38:57,  5.12s/it]
 54%|█████▍    | 121917/225000 [249:26:55<171:17:46,  5.98s/it]
 54%|█████▍    | 121918/225000 [249:26:59<152:54:06,  5.34s/it]
 54%|█████▍    | 121919/225000 [249:27:04<150:38:37,  5.26s/it]
 54%|█████▍    | 121920/225000 [249:27:09<149:44:16,  5.23s/it]
 54%|█████▍    | 121921/225000 [249:27:14<150:20:40,  5.25s/it]
 54%|█████▍    | 121922/225000 [249:27:26<206:59:02,  7.23s/it]
 54%|█████▍    | 121923/225000 [249:27:32<197:27:49,  6.90s/it]
 54%|█████▍    | 121924/225000 [249:27:39<199:44:41,  6.98s/it]
 54%|█████▍    | 121925/225000 [249:27:44<175:59:43,  6.15s/it]
 54%|█████▍    | 121926/225000 [249:27:48<163:15:09,  5.70s/it]
 54%|█████▍    | 121927/225000 [249:27:52<150:14:41,  5.25s/it]
 54%|█████▍    | 121928/225000 [249:27:57<142:47:44,  4.99s/it]
 54%|█████▍    | 121929/225000 [249:28:02<141:54:36,  4.96s/it]
 54%|█████▍    | 121930/225000 [249:28:09<158:42:49,  5.54s/it]
 54%|█████▍    | 121931/225000 [249:28:20<207:39:03,  7.25s/it]
 54%|█████▍    | 121932/225000 [249:28:23<176:24:41,  6.16s/it]
 54%|█████▍    | 121933/225000 [249:28:27<152:06:36,  5.31s/it]
 54%|█████▍    | 121934/225000 [249:28:37<190:01:47,  6.64s/it]
 54%|█████▍    | 121935/225000 [249:28:40<163:59:41,  5.73s/it]
 54%|█████▍    | 121936/225000 [249:28:44<149:00:48,  5.21s/it]
 54%|█████▍    | 121937/225000 [249:28:53<180:52:49,  6.32s/it]
 54%|█████▍    | 121938/225000 [249:28:59<173:33:32,  6.06s/it]
 54%|█████▍    | 121939/225000 [249:29:04<167:25:50,  5.85s/it]
 54%|█████▍    | 121940/225000 [249:29:08<156:20:32,  5.46s/it]
 54%|█████▍    | 121941/225000 [249:29:13<151:29:02,  5.29s/it]
 54%|█████▍    | 121942/225000 [249:29:18<150:30:47,  5.26s/it]
 54%|█████▍    | 121943/225000 [249:29:24<152:54:32,  5.34s/it]
 54%|█████▍    | 121944/225000 [249:29:30<154:24:10,  5.39s/it]
 54%|█████▍    | 121945/225000 [249:29:35<151:29:37,  5.29s/it]
 54%|█████▍    | 121946/225000 [249:29:40<152:15:06,  5.32s/it]
 54%|█████▍    | 121947/225000 [249:29:45<153:24:52,  5.36s/it]
 54%|█████▍    | 121948/225000 [249:29:50<147:51:31,  5.17s/it]
 54%|█████▍    | 121949/225000 [249:29:55<146:04:21,  5.10s/it]
 54%|█████▍    | 121950/225000 [249:30:06<198:29:15,  6.93s/it]
                                                               
{'loss': '0.8537', 'grad_norm': '1.106', 'learning_rate': '2.34e-05', 'epoch': '0.542', 'num_input_tokens_seen': 415744144, 'train_runtime': '8.982e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 121950/225000 [249:30:06<198:29:15,  6.93s/it]
 54%|█████▍    | 121951/225000 [249:30:11<182:52:04,  6.39s/it]
 54%|█████▍    | 121952/225000 [249:30:15<162:43:39,  5.68s/it]
 54%|█████▍    | 121953/225000 [249:30:19<147:51:07,  5.17s/it]
 54%|█████▍    | 121954/225000 [249:30:29<184:14:33,  6.44s/it]
 54%|█████▍    | 121955/225000 [249:30:34<171:16:22,  5.98s/it]
 54%|█████▍    | 121956/225000 [249:30:37<148:22:07,  5.18s/it]
 54%|█████▍    | 121957/225000 [249:30:42<142:25:53,  4.98s/it]
 54%|█████▍    | 121958/225000 [249:30:47<142:13:29,  4.97s/it]
 54%|█████▍    | 121959/225000 [249:30:51<141:37:11,  4.95s/it]
 54%|█████▍    | 121960/225000 [249:30:56<142:15:07,  4.97s/it]
 54%|█████▍    | 121961/225000 [249:31:02<143:39:10,  5.02s/it]
 54%|█████▍    | 121962/225000 [249:31:06<141:44:20,  4.95s/it]
 54%|█████▍    | 121963/225000 [249:31:18<196:07:30,  6.85s/it]
 54%|█████▍    | 121964/225000 [249:31:24<193:16:18,  6.75s/it]
 54%|█████▍    | 121965/225000 [249:31:29<173:59:14,  6.08s/it]
 54%|█████▍    | 121966/225000 [249:31:33<163:06:44,  5.70s/it]
 54%|█████▍    | 121967/225000 [249:31:38<154:41:57,  5.41s/it]
 54%|█████▍    | 121968/225000 [249:31:43<148:07:25,  5.18s/it]
 54%|█████▍    | 121969/225000 [249:31:48<145:46:49,  5.09s/it]
 54%|█████▍    | 121970/225000 [249:31:53<146:50:42,  5.13s/it]
 54%|█████▍    | 121971/225000 [249:31:58<148:17:24,  5.18s/it]
 54%|█████▍    | 121972/225000 [249:32:03<148:03:00,  5.17s/it]
 54%|█████▍    | 121973/225000 [249:32:08<144:28:18,  5.05s/it]
 54%|█████▍    | 121974/225000 [249:32:13<146:21:45,  5.11s/it]
 54%|█████▍    | 121975/225000 [249:32:19<150:23:14,  5.25s/it]
 54%|█████▍    | 121976/225000 [249:32:27<176:20:04,  6.16s/it]
 54%|█████▍    | 121977/225000 [249:32:39<224:21:38,  7.84s/it]
 54%|█████▍    | 121978/225000 [249:32:43<194:23:45,  6.79s/it]
 54%|█████▍    | 121979/225000 [249:32:47<163:49:54,  5.72s/it]
 54%|█████▍    | 121980/225000 [249:32:57<204:35:03,  7.15s/it]
 54%|█████▍    | 121981/225000 [249:33:01<177:24:11,  6.20s/it]
 54%|█████▍    | 121982/225000 [249:33:05<157:13:34,  5.49s/it]
 54%|█████▍    | 121983/225000 [249:33:09<148:39:42,  5.20s/it]
 54%|█████▍    | 121984/225000 [249:33:14<145:53:21,  5.10s/it]
 54%|█████▍    | 121985/225000 [249:33:19<140:33:11,  4.91s/it]
 54%|█████▍    | 121986/225000 [249:33:24<144:37:46,  5.05s/it]
 54%|█████▍    | 121987/225000 [249:33:29<138:55:52,  4.86s/it]
 54%|█████▍    | 121988/225000 [249:33:40<194:46:21,  6.81s/it]
 54%|█████▍    | 121989/225000 [249:33:47<196:10:30,  6.86s/it]
 54%|█████▍    | 121990/225000 [249:33:51<173:39:34,  6.07s/it]
 54%|█████▍    | 121991/225000 [249:33:55<155:44:30,  5.44s/it]
 54%|█████▍    | 121992/225000 [249:34:03<177:52:04,  6.22s/it]
 54%|█████▍    | 121993/225000 [249:34:08<164:43:18,  5.76s/it]
 54%|█████▍    | 121994/225000 [249:34:13<162:13:47,  5.67s/it]
 54%|█████▍    | 121995/225000 [249:34:18<154:41:50,  5.41s/it]
 54%|█████▍    | 121996/225000 [249:34:23<153:20:21,  5.36s/it]
 54%|█████▍    | 121997/225000 [249:34:28<149:49:57,  5.24s/it]
 54%|█████▍    | 121998/225000 [249:34:34<149:52:11,  5.24s/it]
 54%|█████▍    | 121999/225000 [249:34:39<152:34:02,  5.33s/it]
 54%|█████▍    | 122000/225000 [249:34:44<150:14:43,  5.25s/it]
                                                               
{'loss': '0.8598', 'grad_norm': '1.103', 'learning_rate': '2.338e-05', 'epoch': '0.5422', 'num_input_tokens_seen': 415871792, 'train_runtime': '8.985e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 122000/225000 [249:34:44<150:14:43,  5.25s/it]
 54%|█████▍    | 122001/225000 [249:34:49<147:17:01,  5.15s/it]
 54%|█████▍    | 122002/225000 [249:35:00<196:36:40,  6.87s/it]
 54%|█████▍    | 122003/225000 [249:35:06<193:50:40,  6.78s/it]
 54%|█████▍    | 122004/225000 [249:35:11<176:07:50,  6.16s/it]
 54%|█████▍    | 122005/225000 [249:35:16<167:23:02,  5.85s/it]
 54%|█████▍    | 122006/225000 [249:35:21<154:52:55,  5.41s/it]
 54%|█████▍    | 122007/225000 [249:35:26<149:53:24,  5.24s/it]
 54%|█████▍    | 122008/225000 [249:35:30<142:58:26,  5.00s/it]
 54%|█████▍    | 122009/225000 [249:35:37<164:05:23,  5.74s/it]
 54%|█████▍    | 122010/225000 [249:35:48<205:47:34,  7.19s/it]
 54%|█████▍    | 122011/225000 [249:35:53<190:13:07,  6.65s/it]
 54%|█████▍    | 122012/225000 [249:35:59<177:53:25,  6.22s/it]
 54%|█████▍    | 122013/225000 [249:36:04<167:14:02,  5.85s/it]
 54%|█████▍    | 122014/225000 [249:36:09<163:38:06,  5.72s/it]
 54%|█████▍    | 122015/225000 [249:36:17<184:39:50,  6.46s/it]
 54%|█████▍    | 122016/225000 [249:36:24<183:59:53,  6.43s/it]
 54%|█████▍    | 122017/225000 [249:36:27<162:03:28,  5.67s/it]
 54%|█████▍    | 122018/225000 [249:36:32<151:20:07,  5.29s/it]
 54%|█████▍    | 122019/225000 [249:36:36<140:16:03,  4.90s/it]
 54%|█████▍    | 122020/225000 [249:36:45<178:52:15,  6.25s/it]
 54%|█████▍    | 122021/225000 [249:36:50<163:54:52,  5.73s/it]
 54%|█████▍    | 122022/225000 [249:36:54<149:28:54,  5.23s/it]
 54%|█████▍    | 122023/225000 [249:36:58<139:24:12,  4.87s/it]
 54%|█████▍    | 122024/225000 [249:37:02<135:40:41,  4.74s/it]
 54%|█████▍    | 122025/225000 [249:37:12<179:18:52,  6.27s/it]
 54%|█████▍    | 122026/225000 [249:37:19<180:21:38,  6.31s/it]
 54%|█████▍    | 122027/225000 [249:37:22<158:44:56,  5.55s/it]
 54%|█████▍    | 122028/225000 [249:37:31<186:49:53,  6.53s/it]
 54%|█████▍    | 122029/225000 [249:37:35<162:05:59,  5.67s/it]
 54%|█████▍    | 122030/225000 [249:37:39<145:18:04,  5.08s/it]
 54%|█████▍    | 122031/225000 [249:37:48<185:41:28,  6.49s/it]
 54%|█████▍    | 122032/225000 [249:37:52<160:56:13,  5.63s/it]
 54%|█████▍    | 122033/225000 [249:38:00<185:47:13,  6.50s/it]
 54%|█████▍    | 122034/225000 [249:38:04<162:14:31,  5.67s/it]
 54%|█████▍    | 122035/225000 [249:38:09<155:36:07,  5.44s/it]
 54%|█████▍    | 122036/225000 [249:38:14<147:08:04,  5.14s/it]
 54%|█████▍    | 122037/225000 [249:38:18<145:05:32,  5.07s/it]
 54%|█████▍    | 122038/225000 [249:38:23<144:43:35,  5.06s/it]
 54%|█████▍    | 122039/225000 [249:38:29<145:32:05,  5.09s/it]
 54%|█████▍    | 122040/225000 [249:38:34<145:12:59,  5.08s/it]
 54%|█████▍    | 122041/225000 [249:38:39<144:18:12,  5.05s/it]
 54%|█████▍    | 122042/225000 [249:38:49<185:36:52,  6.49s/it]
 54%|█████▍    | 122043/225000 [249:38:58<209:11:27,  7.31s/it]
 54%|█████▍    | 122044/225000 [249:39:02<181:54:04,  6.36s/it]
 54%|█████▍    | 122045/225000 [249:39:07<168:55:07,  5.91s/it]
 54%|█████▍    | 122046/225000 [249:39:11<157:28:10,  5.51s/it]
 54%|█████▍    | 122047/225000 [249:39:16<153:19:00,  5.36s/it]
 54%|█████▍    | 122048/225000 [249:39:22<152:57:05,  5.35s/it]
 54%|█████▍    | 122049/225000 [249:39:27<151:49:26,  5.31s/it]
 54%|█████▍    | 122050/225000 [249:39:32<148:08:53,  5.18s/it]
                                                               
{'loss': '0.862', 'grad_norm': '1.138', 'learning_rate': '2.336e-05', 'epoch': '0.5424', 'num_input_tokens_seen': 416000752, 'train_runtime': '8.988e+05', 'train_tokens_per_second': '462.9'}

 54%|█████▍    | 122050/225000 [249:39:32<148:08:53,  5.18s/it]
 54%|█████▍    | 122051/225000 [249:39:37<146:03:28,  5.11s/it]
 54%|█████▍    | 122052/225000 [249:39:42<146:24:03,  5.12s/it]
 54%|█████▍    | 122053/225000 [249:39:54<207:59:44,  7.27s/it]
 54%|█████▍    | 122054/225000 [249:39:59<191:04:18,  6.68s/it]
 54%|█████▍    | 122055/225000 [249:40:04<173:02:06,  6.05s/it]
 54%|█████▍    | 122056/225000 [249:40:09<163:39:45,  5.72s/it]
 54%|█████▍    | 122057/225000 [249:40:14<154:50:09,  5.41s/it]
 54%|█████▍    | 122058/225000 [249:40:19<151:18:24,  5.29s/it]
 54%|█████▍    | 122059/225000 [249:40:23<145:14:22,  5.08s/it]
 54%|█████▍    | 122060/225000 [249:40:28<143:02:35,  5.00s/it]
 54%|█████▍    | 122061/225000 [249:40:33<141:17:09,  4.94s/it]
 54%|█████▍    | 122062/225000 [249:40:38<144:34:24,  5.06s/it]
 54%|█████▍    | 122063/225000 [249:40:44<147:26:23,  5.16s/it]
 54%|█████▍    | 122064/225000 [249:40:48<144:29:33,  5.05s/it]
 54%|█████▍    | 122065/225000 [249:41:00<199:17:29,  6.97s/it]
 54%|█████▍    | 122066/225000 [249:41:06<196:11:25,  6.86s/it]
 54%|█████▍    | 122067/225000 [249:41:11<180:09:16,  6.30s/it]
 54%|█████▍    | 122068/225000 [249:41:17<171:45:12,  6.01s/it]
 54%|█████▍    | 122069/225000 [249:41:21<157:09:24,  5.50s/it]
 54%|█████▍    | 122070/225000 [249:41:26<150:23:50,  5.26s/it]
 54%|█████▍    | 122071/225000 [249:41:31<150:28:59,  5.26s/it]
 54%|█████▍    | 122072/225000 [249:41:36<144:41:33,  5.06s/it]
 54%|█████▍    | 122073/225000 [249:41:41<149:43:15,  5.24s/it]
 54%|█████▍    | 122074/225000 [249:41:46<149:22:41,  5.22s/it]
 54%|█████▍    | 122075/225000 [249:41:52<151:49:48,  5.31s/it]
 54%|█████▍    | 122076/225000 [249:41:57<151:27:53,  5.30s/it]
 54%|█████▍    | 122077/225000 [249:42:03<154:51:18,  5.42s/it]
 54%|█████▍    | 122078/225000 [249:42:14<204:55:23,  7.17s/it]
 54%|█████▍    | 122079/225000 [249:42:20<190:32:16,  6.66s/it]
 54%|█████▍    | 122080/225000 [249:42:28<206:12:27,  7.21s/it]
 54%|█████▍    | 122081/225000 [249:42:32<174:19:39,  6.10s/it]
 54%|█████▍    | 122082/225000 [249:42:38<171:58:22,  6.02s/it]
 54%|█████▍    | 122083/225000 [249:42:42<161:34:26,  5.65s/it]
 54%|█████▍    | 122084/225000 [249:42:47<153:50:39,  5.38s/it]
 54%|█████▍    | 122085/225000 [249:42:52<145:45:44,  5.10s/it]
 54%|█████▍    | 122086/225000 [249:42:56<142:49:06,  5.00s/it]
 54%|█████▍    | 122087/225000 [249:43:00<134:10:56,  4.69s/it]
 54%|█████▍    | 122088/225000 [249:43:05<134:16:37,  4.70s/it]
 54%|█████▍    | 122089/225000 [249:43:18<205:57:29,  7.20s/it]
 54%|█████▍    | 122090/225000 [249:43:23<187:41:42,  6.57s/it]
 54%|█████▍    | 122091/225000 [249:43:26<160:02:55,  5.60s/it]
 54%|█████▍    | 122092/225000 [249:43:35<184:24:45,  6.45s/it]
 54%|█████▍    | 122093/225000 [249:43:38<158:49:17,  5.56s/it]
 54%|█████▍    | 122094/225000 [249:43:43<149:33:36,  5.23s/it]
 54%|█████▍    | 122095/225000 [249:43:47<141:15:42,  4.94s/it]
 54%|█████▍    | 122096/225000 [249:43:54<160:07:20,  5.60s/it]
 54%|█████▍    | 122097/225000 [249:44:04<197:37:24,  6.91s/it]
 54%|█████▍    | 122098/225000 [249:44:08<171:35:07,  6.00s/it]
 54%|█████▍    | 122099/225000 [249:44:12<155:47:55,  5.45s/it]
 54%|█████▍    | 122100/225000 [249:44:21<181:26:44,  6.35s/it]
                                                               
{'loss': '0.8431', 'grad_norm': '1.088', 'learning_rate': '2.334e-05', 'epoch': '0.5427', 'num_input_tokens_seen': 416126712, 'train_runtime': '8.991e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122100/225000 [249:44:21<181:26:44,  6.35s/it]
 54%|█████▍    | 122101/225000 [249:44:24<157:44:32,  5.52s/it]
 54%|█████▍    | 122102/225000 [249:44:29<152:40:34,  5.34s/it]
 54%|█████▍    | 122103/225000 [249:44:34<147:55:35,  5.18s/it]
 54%|█████▍    | 122104/225000 [249:44:39<143:44:01,  5.03s/it]
 54%|█████▍    | 122105/225000 [249:44:44<143:30:40,  5.02s/it]
 54%|█████▍    | 122106/225000 [249:44:49<142:20:14,  4.98s/it]
 54%|█████▍    | 122107/225000 [249:44:53<138:29:28,  4.85s/it]
 54%|█████▍    | 122108/225000 [249:45:00<155:47:26,  5.45s/it]
 54%|█████▍    | 122109/225000 [249:45:11<205:53:41,  7.20s/it]
 54%|█████▍    | 122110/225000 [249:45:15<175:46:29,  6.15s/it]
 54%|█████▍    | 122111/225000 [249:45:21<175:14:20,  6.13s/it]
 54%|█████▍    | 122112/225000 [249:45:25<159:57:52,  5.60s/it]
 54%|█████▍    | 122113/225000 [249:45:30<154:55:38,  5.42s/it]
 54%|█████▍    | 122114/225000 [249:45:35<150:02:30,  5.25s/it]
 54%|█████▍    | 122115/225000 [249:45:40<145:57:02,  5.11s/it]
 54%|█████▍    | 122116/225000 [249:45:50<185:05:03,  6.48s/it]
 54%|█████▍    | 122117/225000 [249:45:57<190:31:43,  6.67s/it]
 54%|█████▍    | 122118/225000 [249:46:00<164:51:26,  5.77s/it]
 54%|█████▍    | 122119/225000 [249:46:04<144:23:40,  5.05s/it]
 54%|█████▍    | 122120/225000 [249:46:08<139:15:27,  4.87s/it]
 54%|█████▍    | 122121/225000 [249:46:17<171:56:52,  6.02s/it]
 54%|█████▍    | 122122/225000 [249:46:22<164:50:18,  5.77s/it]
 54%|█████▍    | 122123/225000 [249:46:26<149:29:09,  5.23s/it]
 54%|█████▍    | 122124/225000 [249:46:31<142:48:50,  5.00s/it]
 54%|█████▍    | 122125/225000 [249:46:35<140:08:18,  4.90s/it]
 54%|█████▍    | 122126/225000 [249:46:40<136:57:00,  4.79s/it]
 54%|█████▍    | 122127/225000 [249:46:52<203:22:45,  7.12s/it]
 54%|█████▍    | 122128/225000 [249:46:56<172:56:02,  6.05s/it]
 54%|█████▍    | 122129/225000 [249:47:00<157:47:39,  5.52s/it]
 54%|█████▍    | 122130/225000 [249:47:05<152:12:49,  5.33s/it]
 54%|█████▍    | 122131/225000 [249:47:10<148:25:35,  5.19s/it]
 54%|█████▍    | 122132/225000 [249:47:15<147:14:43,  5.15s/it]
 54%|█████▍    | 122133/225000 [249:47:20<142:42:46,  4.99s/it]
 54%|█████▍    | 122134/225000 [249:47:26<151:05:05,  5.29s/it]
 54%|█████▍    | 122135/225000 [249:47:37<201:12:23,  7.04s/it]
 54%|█████▍    | 122136/225000 [249:47:40<167:24:16,  5.86s/it]
 54%|█████▍    | 122137/225000 [249:47:44<152:38:13,  5.34s/it]
 54%|█████▍    | 122138/225000 [249:47:49<147:01:31,  5.15s/it]
 54%|█████▍    | 122139/225000 [249:47:53<143:33:02,  5.02s/it]
 54%|█████▍    | 122140/225000 [249:47:58<139:12:18,  4.87s/it]
 54%|█████▍    | 122141/225000 [249:48:03<137:50:15,  4.82s/it]
 54%|█████▍    | 122142/225000 [249:48:10<156:32:47,  5.48s/it]
 54%|█████▍    | 122143/225000 [249:48:20<196:44:23,  6.89s/it]
 54%|█████▍    | 122144/225000 [249:48:24<171:06:53,  5.99s/it]
 54%|█████▍    | 122145/225000 [249:48:29<161:06:41,  5.64s/it]
 54%|█████▍    | 122146/225000 [249:48:39<205:39:58,  7.20s/it]
 54%|█████▍    | 122147/225000 [249:48:42<170:15:54,  5.96s/it]
 54%|█████▍    | 122148/225000 [249:48:51<188:47:47,  6.61s/it]
 54%|█████▍    | 122149/225000 [249:48:56<174:55:36,  6.12s/it]
 54%|█████▍    | 122150/225000 [249:49:03<185:32:51,  6.49s/it]
                                                               
{'loss': '0.8584', 'grad_norm': '1.18', 'learning_rate': '2.332e-05', 'epoch': '0.5429', 'num_input_tokens_seen': 416253336, 'train_runtime': '8.993e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122150/225000 [249:49:03<185:32:51,  6.49s/it]
 54%|█████▍    | 122151/225000 [249:49:06<159:50:21,  5.59s/it]
 54%|█████▍    | 122152/225000 [249:49:10<142:32:42,  4.99s/it]
 54%|█████▍    | 122153/225000 [249:49:18<170:16:09,  5.96s/it]
 54%|█████▍    | 122154/225000 [249:49:23<159:16:18,  5.58s/it]
 54%|█████▍    | 122155/225000 [249:49:28<153:23:06,  5.37s/it]
 54%|█████▍    | 122156/225000 [249:49:32<145:52:21,  5.11s/it]
 54%|█████▍    | 122157/225000 [249:49:37<144:08:13,  5.05s/it]
 54%|█████▍    | 122158/225000 [249:49:42<141:29:54,  4.95s/it]
 54%|█████▍    | 122159/225000 [249:49:47<140:02:51,  4.90s/it]
 54%|█████▍    | 122160/225000 [249:49:52<143:30:46,  5.02s/it]
 54%|█████▍    | 122161/225000 [249:50:04<204:08:56,  7.15s/it]
 54%|█████▍    | 122162/225000 [249:50:11<198:47:01,  6.96s/it]
 54%|█████▍    | 122163/225000 [249:50:14<167:36:34,  5.87s/it]
 54%|█████▍    | 122164/225000 [249:50:22<183:43:49,  6.43s/it]
 54%|█████▍    | 122165/225000 [249:50:27<170:57:23,  5.98s/it]
 54%|█████▍    | 122166/225000 [249:50:32<163:06:00,  5.71s/it]
 54%|█████▍    | 122167/225000 [249:50:37<156:27:29,  5.48s/it]
 54%|█████▍    | 122168/225000 [249:50:42<151:43:55,  5.31s/it]
 54%|█████▍    | 122169/225000 [249:50:47<151:11:01,  5.29s/it]
 54%|█████▍    | 122170/225000 [249:50:52<150:00:27,  5.25s/it]
 54%|█████▍    | 122171/225000 [249:50:57<151:21:22,  5.30s/it]
 54%|█████▍    | 122172/225000 [249:51:03<154:36:34,  5.41s/it]
 54%|█████▍    | 122173/225000 [249:51:08<149:26:15,  5.23s/it]
 54%|█████▍    | 122174/225000 [249:51:14<154:45:42,  5.42s/it]
 54%|█████▍    | 122175/225000 [249:51:21<169:14:11,  5.93s/it]
 54%|█████▍    | 122176/225000 [249:51:26<162:46:47,  5.70s/it]
 54%|█████▍    | 122177/225000 [249:51:31<160:52:24,  5.63s/it]
 54%|█████▍    | 122178/225000 [249:51:37<159:22:22,  5.58s/it]
 54%|█████▍    | 122179/225000 [249:51:42<158:51:23,  5.56s/it]
 54%|█████▍    | 122180/225000 [249:51:48<158:39:15,  5.55s/it]
 54%|█████▍    | 122181/225000 [249:51:53<155:05:19,  5.43s/it]
 54%|█████▍    | 122182/225000 [249:51:58<149:28:58,  5.23s/it]
 54%|█████▍    | 122183/225000 [249:52:05<164:06:13,  5.75s/it]
 54%|█████▍    | 122184/225000 [249:52:15<197:42:08,  6.92s/it]
 54%|█████▍    | 122185/225000 [249:52:20<181:25:14,  6.35s/it]
 54%|█████▍    | 122186/225000 [249:52:25<172:58:49,  6.06s/it]
 54%|█████▍    | 122187/225000 [249:52:30<164:45:58,  5.77s/it]
 54%|█████▍    | 122188/225000 [249:52:35<161:09:35,  5.64s/it]
 54%|█████▍    | 122189/225000 [249:52:40<155:23:23,  5.44s/it]
 54%|█████▍    | 122190/225000 [249:52:45<151:53:01,  5.32s/it]
 54%|█████▍    | 122191/225000 [249:52:51<152:11:24,  5.33s/it]
 54%|█████▍    | 122192/225000 [249:52:56<153:41:06,  5.38s/it]
 54%|█████▍    | 122193/225000 [249:53:02<153:24:08,  5.37s/it]
 54%|█████▍    | 122194/225000 [249:53:07<155:16:56,  5.44s/it]
 54%|█████▍    | 122195/225000 [249:53:20<214:21:50,  7.51s/it]
 54%|█████▍    | 122196/225000 [249:53:25<195:13:23,  6.84s/it]
 54%|█████▍    | 122197/225000 [249:53:29<171:52:59,  6.02s/it]
 54%|█████▍    | 122198/225000 [249:53:33<158:16:06,  5.54s/it]
 54%|█████▍    | 122199/225000 [249:53:44<199:51:24,  7.00s/it]
 54%|█████▍    | 122200/225000 [249:53:49<186:10:55,  6.52s/it]
                                                               
{'loss': '0.8458', 'grad_norm': '1.021', 'learning_rate': '2.33e-05', 'epoch': '0.5431', 'num_input_tokens_seen': 416383400, 'train_runtime': '8.996e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122200/225000 [249:53:49<186:10:55,  6.52s/it]
 54%|█████▍    | 122201/225000 [249:53:52<159:13:51,  5.58s/it]
 54%|█████▍    | 122202/225000 [249:54:00<179:17:16,  6.28s/it]
 54%|█████▍    | 122203/225000 [249:54:04<155:29:54,  5.45s/it]
 54%|█████▍    | 122204/225000 [249:54:12<178:22:09,  6.25s/it]
 54%|█████▍    | 122205/225000 [249:54:16<156:02:04,  5.46s/it]
 54%|█████▍    | 122206/225000 [249:54:19<139:33:23,  4.89s/it]
 54%|█████▍    | 122207/225000 [249:54:23<132:41:08,  4.65s/it]
 54%|█████▍    | 122208/225000 [249:54:31<158:54:54,  5.57s/it]
 54%|█████▍    | 122209/225000 [249:54:36<152:41:33,  5.35s/it]
 54%|█████▍    | 122210/225000 [249:54:45<184:35:29,  6.46s/it]
 54%|█████▍    | 122211/225000 [249:54:49<161:53:33,  5.67s/it]
 54%|█████▍    | 122212/225000 [249:54:56<176:35:18,  6.18s/it]
 54%|█████▍    | 122213/225000 [249:55:00<155:57:41,  5.46s/it]
 54%|█████▍    | 122214/225000 [249:55:07<173:01:07,  6.06s/it]
 54%|█████▍    | 122215/225000 [249:55:13<166:44:43,  5.84s/it]
 54%|█████▍    | 122216/225000 [249:55:17<157:18:18,  5.51s/it]
 54%|█████▍    | 122217/225000 [249:55:22<146:05:03,  5.12s/it]
 54%|█████▍    | 122218/225000 [249:55:26<138:29:21,  4.85s/it]
 54%|█████▍    | 122219/225000 [249:55:38<198:47:27,  6.96s/it]
 54%|█████▍    | 122220/225000 [249:55:42<179:18:38,  6.28s/it]
 54%|█████▍    | 122221/225000 [249:55:46<152:04:33,  5.33s/it]
 54%|█████▍    | 122222/225000 [249:55:54<179:02:47,  6.27s/it]
 54%|█████▍    | 122223/225000 [249:55:58<159:49:01,  5.60s/it]
 54%|█████▍    | 122224/225000 [249:56:02<147:17:03,  5.16s/it]
 54%|█████▍    | 122225/225000 [249:56:06<136:26:11,  4.78s/it]
 54%|█████▍    | 122226/225000 [249:56:14<165:41:31,  5.80s/it]
 54%|█████▍    | 122227/225000 [249:56:21<171:51:56,  6.02s/it]
 54%|█████▍    | 122228/225000 [249:56:24<152:00:00,  5.32s/it]
 54%|█████▍    | 122229/225000 [249:56:32<168:23:10,  5.90s/it]
 54%|█████▍    | 122230/225000 [249:56:37<167:01:06,  5.85s/it]
 54%|█████▍    | 122231/225000 [249:56:41<149:20:39,  5.23s/it]
 54%|█████▍    | 122232/225000 [249:56:47<155:21:21,  5.44s/it]
 54%|█████▍    | 122233/225000 [249:56:54<164:38:25,  5.77s/it]
 54%|█████▍    | 122234/225000 [249:56:59<160:23:15,  5.62s/it]
 54%|█████▍    | 122235/225000 [249:57:06<175:33:50,  6.15s/it]
 54%|█████▍    | 122236/225000 [249:57:10<155:44:36,  5.46s/it]
 54%|█████▍    | 122237/225000 [249:57:16<156:11:16,  5.47s/it]
 54%|█████▍    | 122238/225000 [249:57:21<155:38:08,  5.45s/it]
 54%|█████▍    | 122239/225000 [249:57:26<152:52:09,  5.36s/it]
 54%|█████▍    | 122240/225000 [249:57:31<149:41:58,  5.24s/it]
 54%|█████▍    | 122241/225000 [249:57:36<148:04:27,  5.19s/it]
 54%|█████▍    | 122242/225000 [249:57:41<146:05:20,  5.12s/it]
 54%|█████▍    | 122243/225000 [249:57:47<149:03:36,  5.22s/it]
 54%|█████▍    | 122244/225000 [249:57:56<186:09:06,  6.52s/it]
 54%|█████▍    | 122245/225000 [249:58:04<200:13:40,  7.01s/it]
 54%|█████▍    | 122246/225000 [249:58:08<172:06:52,  6.03s/it]
 54%|█████▍    | 122247/225000 [249:58:16<185:18:58,  6.49s/it]
 54%|█████▍    | 122248/225000 [249:58:21<174:19:00,  6.11s/it]
 54%|█████▍    | 122249/225000 [249:58:24<150:08:23,  5.26s/it]
 54%|█████▍    | 122250/225000 [249:58:29<147:50:57,  5.18s/it]
                                                               
{'loss': '0.8397', 'grad_norm': '1.021', 'learning_rate': '2.329e-05', 'epoch': '0.5433', 'num_input_tokens_seen': 416511512, 'train_runtime': '8.999e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122250/225000 [249:58:29<147:50:57,  5.18s/it]
 54%|█████▍    | 122251/225000 [249:58:34<143:38:36,  5.03s/it]
 54%|█████▍    | 122252/225000 [249:58:39<140:40:51,  4.93s/it]
 54%|█████▍    | 122253/225000 [249:58:44<146:05:23,  5.12s/it]
 54%|█████▍    | 122254/225000 [249:58:56<203:32:43,  7.13s/it]
 54%|█████▍    | 122255/225000 [249:58:59<171:02:43,  5.99s/it]
 54%|█████▍    | 122256/225000 [249:59:03<149:31:52,  5.24s/it]
 54%|█████▍    | 122257/225000 [249:59:11<174:36:36,  6.12s/it]
 54%|█████▍    | 122258/225000 [249:59:15<158:04:51,  5.54s/it]
 54%|█████▍    | 122259/225000 [249:59:23<176:31:54,  6.19s/it]
 54%|█████▍    | 122260/225000 [249:59:28<165:45:43,  5.81s/it]
 54%|█████▍    | 122261/225000 [249:59:31<144:47:09,  5.07s/it]
 54%|█████▍    | 122262/225000 [249:59:35<135:18:09,  4.74s/it]
 54%|█████▍    | 122263/225000 [249:59:39<132:08:58,  4.63s/it]
 54%|█████▍    | 122264/225000 [249:59:44<132:58:37,  4.66s/it]
 54%|█████▍    | 122265/225000 [249:59:56<193:23:19,  6.78s/it]
 54%|█████▍    | 122266/225000 [250:00:00<168:34:30,  5.91s/it]
 54%|█████▍    | 122267/225000 [250:00:09<193:21:03,  6.78s/it]
 54%|█████▍    | 122268/225000 [250:00:12<166:46:57,  5.84s/it]
 54%|█████▍    | 122269/225000 [250:00:17<156:49:39,  5.50s/it]
 54%|█████▍    | 122270/225000 [250:00:22<149:45:56,  5.25s/it]
 54%|█████▍    | 122271/225000 [250:00:31<185:46:16,  6.51s/it]
 54%|█████▍    | 122272/225000 [250:00:35<167:45:30,  5.88s/it]
 54%|█████▍    | 122273/225000 [250:00:40<151:57:48,  5.33s/it]
 54%|█████▍    | 122274/225000 [250:00:44<142:46:04,  5.00s/it]
 54%|█████▍    | 122275/225000 [250:00:48<138:22:57,  4.85s/it]
 54%|█████▍    | 122276/225000 [250:00:52<132:57:45,  4.66s/it]
 54%|█████▍    | 122277/225000 [250:01:05<196:53:13,  6.90s/it]
 54%|█████▍    | 122278/225000 [250:01:09<179:14:55,  6.28s/it]
 54%|█████▍    | 122279/225000 [250:01:14<161:46:19,  5.67s/it]
 54%|█████▍    | 122280/225000 [250:01:19<154:33:45,  5.42s/it]
 54%|█████▍    | 122281/225000 [250:01:23<145:41:22,  5.11s/it]
 54%|█████▍    | 122282/225000 [250:01:28<146:14:32,  5.13s/it]
 54%|█████▍    | 122283/225000 [250:01:33<143:01:31,  5.01s/it]
 54%|█████▍    | 122284/225000 [250:01:38<142:20:36,  4.99s/it]
 54%|█████▍    | 122285/225000 [250:01:43<142:10:47,  4.98s/it]
 54%|█████▍    | 122286/225000 [250:01:48<146:08:41,  5.12s/it]
 54%|█████▍    | 122287/225000 [250:02:01<208:11:03,  7.30s/it]
 54%|█████▍    | 122288/225000 [250:02:07<205:00:07,  7.19s/it]
 54%|█████▍    | 122289/225000 [250:02:12<184:44:40,  6.48s/it]
 54%|█████▍    | 122290/225000 [250:02:19<185:06:58,  6.49s/it]
 54%|█████▍    | 122291/225000 [250:02:26<193:18:39,  6.78s/it]
 54%|█████▍    | 122292/225000 [250:02:29<161:45:55,  5.67s/it]
 54%|█████▍    | 122293/225000 [250:02:36<172:53:55,  6.06s/it]
 54%|█████▍    | 122294/225000 [250:02:42<168:52:13,  5.92s/it]
 54%|█████▍    | 122295/225000 [250:02:50<189:37:37,  6.65s/it]
 54%|█████▍    | 122296/225000 [250:02:54<163:42:40,  5.74s/it]
 54%|█████▍    | 122297/225000 [250:02:58<147:18:23,  5.16s/it]
 54%|█████▍    | 122298/225000 [250:03:07<183:35:38,  6.44s/it]
 54%|█████▍    | 122299/225000 [250:03:11<161:50:23,  5.67s/it]
 54%|█████▍    | 122300/225000 [250:03:16<155:06:52,  5.44s/it]
                                                               
{'loss': '0.8544', 'grad_norm': '1.095', 'learning_rate': '2.327e-05', 'epoch': '0.5436', 'num_input_tokens_seen': 416639232, 'train_runtime': '9.002e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122300/225000 [250:03:16<155:06:52,  5.44s/it]
 54%|█████▍    | 122301/225000 [250:03:21<155:23:00,  5.45s/it]
 54%|█████▍    | 122302/225000 [250:03:27<156:30:48,  5.49s/it]
 54%|█████▍    | 122303/225000 [250:03:38<201:33:49,  7.07s/it]
 54%|█████▍    | 122304/225000 [250:03:43<183:23:07,  6.43s/it]
 54%|█████▍    | 122305/225000 [250:03:49<180:06:13,  6.31s/it]
 54%|█████▍    | 122306/225000 [250:03:53<165:33:25,  5.80s/it]
 54%|█████▍    | 122307/225000 [250:03:58<158:38:50,  5.56s/it]
 54%|█████▍    | 122308/225000 [250:04:03<148:16:21,  5.20s/it]
 54%|█████▍    | 122309/225000 [250:04:08<146:58:55,  5.15s/it]
 54%|█████▍    | 122310/225000 [250:04:12<141:55:46,  4.98s/it]
 54%|█████▍    | 122311/225000 [250:04:17<140:13:03,  4.92s/it]
 54%|█████▍    | 122312/225000 [250:04:22<141:45:17,  4.97s/it]
 54%|█████▍    | 122313/225000 [250:04:27<143:33:49,  5.03s/it]
 54%|█████▍    | 122314/225000 [250:04:32<143:59:49,  5.05s/it]
 54%|█████▍    | 122315/225000 [250:04:42<186:19:33,  6.53s/it]
 54%|█████▍    | 122316/225000 [250:04:49<187:07:43,  6.56s/it]
 54%|█████▍    | 122317/225000 [250:04:53<168:13:22,  5.90s/it]
 54%|█████▍    | 122318/225000 [250:05:00<174:50:10,  6.13s/it]
 54%|█████▍    | 122319/225000 [250:05:07<183:35:38,  6.44s/it]
 54%|█████▍    | 122320/225000 [250:05:13<176:39:02,  6.19s/it]
 54%|█████▍    | 122321/225000 [250:05:19<177:26:14,  6.22s/it]
 54%|█████▍    | 122322/225000 [250:05:24<168:23:18,  5.90s/it]
 54%|█████▍    | 122323/225000 [250:05:28<148:55:19,  5.22s/it]
 54%|█████▍    | 122324/225000 [250:05:37<180:16:43,  6.32s/it]
 54%|█████▍    | 122325/225000 [250:05:41<161:36:14,  5.67s/it]
 54%|█████▍    | 122326/225000 [250:05:45<151:29:19,  5.31s/it]
 54%|█████▍    | 122327/225000 [250:05:49<141:27:30,  4.96s/it]
 54%|█████▍    | 122328/225000 [250:06:02<203:29:36,  7.14s/it]
 54%|█████▍    | 122329/225000 [250:06:06<180:02:24,  6.31s/it]
 54%|█████▍    | 122330/225000 [250:06:11<166:59:20,  5.86s/it]
 54%|█████▍    | 122331/225000 [250:06:16<157:04:36,  5.51s/it]
 54%|█████▍    | 122332/225000 [250:06:20<145:15:30,  5.09s/it]
 54%|█████▍    | 122333/225000 [250:06:25<142:38:56,  5.00s/it]
 54%|█████▍    | 122334/225000 [250:06:33<172:40:45,  6.06s/it]
 54%|█████▍    | 122335/225000 [250:06:41<191:36:36,  6.72s/it]
 54%|█████▍    | 122336/225000 [250:06:45<162:19:50,  5.69s/it]
 54%|█████▍    | 122337/225000 [250:06:52<174:41:26,  6.13s/it]
 54%|█████▍    | 122338/225000 [250:06:56<157:21:53,  5.52s/it]
 54%|█████▍    | 122339/225000 [250:07:00<144:03:17,  5.05s/it]
 54%|█████▍    | 122340/225000 [250:07:06<155:57:00,  5.47s/it]
 54%|█████▍    | 122341/225000 [250:07:15<186:36:56,  6.54s/it]
 54%|█████▍    | 122342/225000 [250:07:22<184:44:22,  6.48s/it]
 54%|█████▍    | 122343/225000 [250:07:26<163:54:34,  5.75s/it]
 54%|█████▍    | 122344/225000 [250:07:29<143:54:36,  5.05s/it]
 54%|█████▍    | 122345/225000 [250:07:38<177:49:29,  6.24s/it]
 54%|█████▍    | 122346/225000 [250:07:45<186:37:58,  6.55s/it]
 54%|█████▍    | 122347/225000 [250:07:49<163:54:07,  5.75s/it]
 54%|█████▍    | 122348/225000 [250:07:53<144:31:52,  5.07s/it]
 54%|█████▍    | 122349/225000 [250:07:57<135:18:26,  4.75s/it]
 54%|█████▍    | 122350/225000 [250:08:02<144:12:12,  5.06s/it]
                                                               
{'loss': '0.8483', 'grad_norm': '0.949', 'learning_rate': '2.325e-05', 'epoch': '0.5438', 'num_input_tokens_seen': 416770384, 'train_runtime': '9.005e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122350/225000 [250:08:02<144:12:12,  5.06s/it]
 54%|█████▍    | 122351/225000 [250:08:08<147:24:00,  5.17s/it]
 54%|█████▍    | 122352/225000 [250:08:13<149:31:02,  5.24s/it]
 54%|█████▍    | 122353/225000 [250:08:19<151:01:11,  5.30s/it]
 54%|█████▍    | 122354/225000 [250:08:24<149:31:53,  5.24s/it]
 54%|█████▍    | 122355/225000 [250:08:30<154:56:02,  5.43s/it]
 54%|█████▍    | 122356/225000 [250:08:36<160:36:41,  5.63s/it]
 54%|█████▍    | 122357/225000 [250:08:41<159:02:03,  5.58s/it]
 54%|█████▍    | 122358/225000 [250:08:47<159:57:56,  5.61s/it]
 54%|█████▍    | 122359/225000 [250:08:53<159:39:05,  5.60s/it]
 54%|█████▍    | 122360/225000 [250:08:58<156:05:34,  5.47s/it]
 54%|█████▍    | 122361/225000 [250:09:03<156:32:39,  5.49s/it]
 54%|█████▍    | 122362/225000 [250:09:08<154:03:45,  5.40s/it]
 54%|█████▍    | 122363/225000 [250:09:14<151:07:18,  5.30s/it]
 54%|█████▍    | 122364/225000 [250:09:22<179:41:22,  6.30s/it]
 54%|█████▍    | 122365/225000 [250:09:27<169:46:30,  5.95s/it]
 54%|█████▍    | 122366/225000 [250:09:32<162:09:04,  5.69s/it]
 54%|█████▍    | 122367/225000 [250:09:38<160:43:30,  5.64s/it]
 54%|█████▍    | 122368/225000 [250:09:44<160:53:37,  5.64s/it]
 54%|█████▍    | 122369/225000 [250:09:50<166:06:48,  5.83s/it]
 54%|█████▍    | 122370/225000 [250:10:02<220:57:51,  7.75s/it]
 54%|█████▍    | 122371/225000 [250:10:06<187:47:49,  6.59s/it]
 54%|█████▍    | 122372/225000 [250:10:10<163:03:10,  5.72s/it]
 54%|█████▍    | 122373/225000 [250:10:19<196:20:07,  6.89s/it]
 54%|█████▍    | 122374/225000 [250:10:22<165:23:35,  5.80s/it]
 54%|█████▍    | 122375/225000 [250:10:30<178:42:36,  6.27s/it]
 54%|█████▍    | 122376/225000 [250:10:33<154:36:28,  5.42s/it]
 54%|█████▍    | 122377/225000 [250:10:43<191:05:51,  6.70s/it]
 54%|█████▍    | 122378/225000 [250:10:46<158:17:29,  5.55s/it]
 54%|█████▍    | 122379/225000 [250:10:52<166:46:02,  5.85s/it]
 54%|█████▍    | 122380/225000 [250:10:58<160:41:36,  5.64s/it]
 54%|█████▍    | 122381/225000 [250:11:04<169:09:10,  5.93s/it]
 54%|█████▍    | 122382/225000 [250:11:08<152:54:15,  5.36s/it]
 54%|█████▍    | 122383/225000 [250:11:12<139:26:24,  4.89s/it]
 54%|█████▍    | 122384/225000 [250:11:16<133:51:50,  4.70s/it]
 54%|█████▍    | 122385/225000 [250:11:21<133:24:12,  4.68s/it]
 54%|█████▍    | 122386/225000 [250:11:31<182:54:11,  6.42s/it]
 54%|█████▍    | 122387/225000 [250:11:38<187:15:23,  6.57s/it]
 54%|█████▍    | 122388/225000 [250:11:42<162:13:26,  5.69s/it]
 54%|█████▍    | 122389/225000 [250:11:46<150:06:07,  5.27s/it]
 54%|█████▍    | 122390/225000 [250:11:51<142:53:03,  5.01s/it]
 54%|█████▍    | 122391/225000 [250:11:58<159:45:30,  5.61s/it]
 54%|█████▍    | 122392/225000 [250:12:06<184:39:53,  6.48s/it]
 54%|█████▍    | 122393/225000 [250:12:10<164:57:28,  5.79s/it]
 54%|█████▍    | 122394/225000 [250:12:16<166:08:25,  5.83s/it]
 54%|█████▍    | 122395/225000 [250:12:23<177:16:39,  6.22s/it]
 54%|█████▍    | 122396/225000 [250:12:28<161:44:04,  5.67s/it]
 54%|█████▍    | 122397/225000 [250:12:33<158:56:47,  5.58s/it]
 54%|█████▍    | 122398/225000 [250:12:38<155:57:22,  5.47s/it]
 54%|█████▍    | 122399/225000 [250:12:43<151:13:49,  5.31s/it]
 54%|█████▍    | 122400/225000 [250:12:54<194:48:42,  6.84s/it]
                                                               
{'loss': '0.8506', 'grad_norm': '1.015', 'learning_rate': '2.323e-05', 'epoch': '0.544', 'num_input_tokens_seen': 416898952, 'train_runtime': '9.008e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122400/225000 [250:12:54<194:48:42,  6.84s/it]
 54%|█████▍    | 122401/225000 [250:12:57<166:17:22,  5.83s/it]
 54%|█████▍    | 122402/225000 [250:13:02<162:01:53,  5.69s/it]
 54%|█████▍    | 122403/225000 [250:13:08<161:19:44,  5.66s/it]
 54%|█████▍    | 122404/225000 [250:13:13<153:36:11,  5.39s/it]
 54%|█████▍    | 122405/225000 [250:13:18<152:46:29,  5.36s/it]
 54%|█████▍    | 122406/225000 [250:13:24<153:15:27,  5.38s/it]
 54%|█████▍    | 122407/225000 [250:13:32<178:02:55,  6.25s/it]
 54%|█████▍    | 122408/225000 [250:13:38<173:27:33,  6.09s/it]
 54%|█████▍    | 122409/225000 [250:13:42<160:45:13,  5.64s/it]
 54%|█████▍    | 122410/225000 [250:13:47<151:39:17,  5.32s/it]
 54%|█████▍    | 122411/225000 [250:13:51<143:33:43,  5.04s/it]
 54%|█████▍    | 122412/225000 [250:14:01<187:54:56,  6.59s/it]
 54%|█████▍    | 122413/225000 [250:14:08<190:31:51,  6.69s/it]
 54%|█████▍    | 122414/225000 [250:14:12<167:35:38,  5.88s/it]
 54%|█████▍    | 122415/225000 [250:14:16<151:54:02,  5.33s/it]
 54%|█████▍    | 122416/225000 [250:14:21<143:27:28,  5.03s/it]
 54%|█████▍    | 122417/225000 [250:14:28<165:02:30,  5.79s/it]
 54%|█████▍    | 122418/225000 [250:14:36<182:37:31,  6.41s/it]
 54%|█████▍    | 122419/225000 [250:14:41<166:49:27,  5.85s/it]
 54%|█████▍    | 122420/225000 [250:14:45<153:34:51,  5.39s/it]
 54%|█████▍    | 122421/225000 [250:14:49<145:18:46,  5.10s/it]
 54%|█████▍    | 122422/225000 [250:14:54<139:20:48,  4.89s/it]
 54%|█████▍    | 122423/225000 [250:15:06<200:24:25,  7.03s/it]
 54%|█████▍    | 122424/225000 [250:15:09<170:15:05,  5.98s/it]
 54%|█████▍    | 122425/225000 [250:15:18<197:05:24,  6.92s/it]
 54%|█████▍    | 122426/225000 [250:15:22<169:11:12,  5.94s/it]
 54%|█████▍    | 122427/225000 [250:15:29<174:38:30,  6.13s/it]
 54%|█████▍    | 122428/225000 [250:15:34<168:25:40,  5.91s/it]
 54%|█████▍    | 122429/225000 [250:15:41<177:27:13,  6.23s/it]
 54%|█████▍    | 122430/225000 [250:15:45<155:19:32,  5.45s/it]
 54%|█████▍    | 122431/225000 [250:15:50<151:35:17,  5.32s/it]
 54%|█████▍    | 122432/225000 [250:15:54<144:19:34,  5.07s/it]
 54%|█████▍    | 122433/225000 [250:15:59<142:21:48,  5.00s/it]
 54%|█████▍    | 122434/225000 [250:16:04<138:52:28,  4.87s/it]
 54%|█████▍    | 122435/225000 [250:16:09<142:18:51,  5.00s/it]
 54%|█████▍    | 122436/225000 [250:16:14<143:53:53,  5.05s/it]
 54%|█████▍    | 122437/225000 [250:16:19<140:33:15,  4.93s/it]
 54%|█████▍    | 122438/225000 [250:16:24<144:37:56,  5.08s/it]
 54%|█████▍    | 122439/225000 [250:16:36<205:51:06,  7.23s/it]
 54%|█████▍    | 122440/225000 [250:16:42<196:56:19,  6.91s/it]
 54%|█████▍    | 122441/225000 [250:16:46<169:39:49,  5.96s/it]
 54%|█████▍    | 122442/225000 [250:16:51<160:37:38,  5.64s/it]
 54%|█████▍    | 122443/225000 [250:16:56<150:24:15,  5.28s/it]
 54%|█████▍    | 122444/225000 [250:17:05<187:20:56,  6.58s/it]
 54%|█████▍    | 122445/225000 [250:17:14<208:23:26,  7.32s/it]
 54%|█████▍    | 122446/225000 [250:17:18<179:31:15,  6.30s/it]
 54%|█████▍    | 122447/225000 [250:17:23<170:29:50,  5.99s/it]
 54%|█████▍    | 122448/225000 [250:17:29<163:33:26,  5.74s/it]
 54%|█████▍    | 122449/225000 [250:17:34<159:22:05,  5.59s/it]
 54%|█████▍    | 122450/225000 [250:17:39<155:38:50,  5.46s/it]
                                                               
{'loss': '0.842', 'grad_norm': '1.027', 'learning_rate': '2.321e-05', 'epoch': '0.5442', 'num_input_tokens_seen': 417028280, 'train_runtime': '9.011e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122450/225000 [250:17:39<155:38:50,  5.46s/it]
 54%|█████▍    | 122451/225000 [250:17:44<151:28:29,  5.32s/it]
 54%|█████▍    | 122452/225000 [250:17:49<148:37:06,  5.22s/it]
 54%|█████▍    | 122453/225000 [250:17:54<147:38:18,  5.18s/it]
 54%|█████▍    | 122454/225000 [250:17:59<148:03:17,  5.20s/it]
 54%|█████▍    | 122455/225000 [250:18:04<148:05:34,  5.20s/it]
 54%|█████▍    | 122456/225000 [250:18:09<144:53:01,  5.09s/it]
 54%|█████▍    | 122457/225000 [250:18:14<141:47:40,  4.98s/it]
 54%|█████▍    | 122458/225000 [250:18:19<143:19:16,  5.03s/it]
 54%|█████▍    | 122459/225000 [250:18:30<190:44:22,  6.70s/it]
 54%|█████▍    | 122460/225000 [250:18:36<189:59:06,  6.67s/it]
 54%|█████▍    | 122461/225000 [250:18:40<161:29:38,  5.67s/it]
 54%|█████▍    | 122462/225000 [250:18:49<190:54:17,  6.70s/it]
 54%|█████▍    | 122463/225000 [250:18:53<168:03:29,  5.90s/it]
 54%|█████▍    | 122464/225000 [250:18:57<156:00:14,  5.48s/it]
 54%|█████▍    | 122465/225000 [250:19:02<149:24:57,  5.25s/it]
 54%|█████▍    | 122466/225000 [250:19:06<140:05:18,  4.92s/it]
 54%|█████▍    | 122467/225000 [250:19:13<157:51:27,  5.54s/it]
 54%|█████▍    | 122468/225000 [250:19:24<207:10:31,  7.27s/it]
 54%|█████▍    | 122469/225000 [250:19:29<185:01:52,  6.50s/it]
 54%|█████▍    | 122470/225000 [250:19:33<161:37:08,  5.67s/it]
 54%|█████▍    | 122471/225000 [250:19:37<151:22:11,  5.31s/it]
 54%|█████▍    | 122472/225000 [250:19:42<141:34:05,  4.97s/it]
 54%|█████▍    | 122473/225000 [250:19:46<134:04:55,  4.71s/it]
 54%|█████▍    | 122474/225000 [250:19:50<132:42:57,  4.66s/it]
 54%|█████▍    | 122475/225000 [250:20:02<191:12:25,  6.71s/it]
 54%|█████▍    | 122476/225000 [250:20:08<191:41:25,  6.73s/it]
 54%|█████▍    | 122477/225000 [250:20:12<168:14:24,  5.91s/it]
 54%|█████▍    | 122478/225000 [250:20:17<154:33:05,  5.43s/it]
 54%|█████▍    | 122479/225000 [250:20:21<143:36:10,  5.04s/it]
 54%|█████▍    | 122480/225000 [250:20:26<142:52:32,  5.02s/it]
 54%|█████▍    | 122481/225000 [250:20:37<195:26:23,  6.86s/it]
 54%|█████▍    | 122482/225000 [250:20:42<182:00:44,  6.39s/it]
 54%|█████▍    | 122483/225000 [250:20:46<156:54:29,  5.51s/it]
 54%|█████▍    | 122484/225000 [250:20:56<194:45:37,  6.84s/it]
 54%|█████▍    | 122485/225000 [250:20:59<164:30:00,  5.78s/it]
 54%|█████▍    | 122486/225000 [250:21:04<158:15:27,  5.56s/it]
 54%|█████▍    | 122487/225000 [250:21:11<166:33:11,  5.85s/it]
 54%|█████▍    | 122488/225000 [250:21:14<148:12:13,  5.20s/it]
 54%|█████▍    | 122489/225000 [250:21:22<165:13:00,  5.80s/it]
 54%|█████▍    | 122490/225000 [250:21:29<177:09:10,  6.22s/it]
 54%|█████▍    | 122491/225000 [250:21:33<159:46:12,  5.61s/it]
 54%|█████▍    | 122492/225000 [250:21:37<145:28:37,  5.11s/it]
 54%|█████▍    | 122493/225000 [250:21:41<137:18:04,  4.82s/it]
 54%|█████▍    | 122494/225000 [250:21:45<131:52:54,  4.63s/it]
 54%|█████▍    | 122495/225000 [250:21:57<196:11:49,  6.89s/it]
 54%|█████▍    | 122496/225000 [250:22:01<164:38:58,  5.78s/it]
 54%|█████▍    | 122497/225000 [250:22:05<149:31:59,  5.25s/it]
 54%|█████▍    | 122498/225000 [250:22:08<136:31:06,  4.79s/it]
 54%|█████▍    | 122499/225000 [250:22:13<132:50:26,  4.67s/it]
 54%|█████▍    | 122500/225000 [250:22:23<179:23:37,  6.30s/it]
                                                               
{'loss': '0.8383', 'grad_norm': '0.9732', 'learning_rate': '2.32e-05', 'epoch': '0.5444', 'num_input_tokens_seen': 417155448, 'train_runtime': '9.013e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122500/225000 [250:22:23<179:23:37,  6.30s/it]
 54%|█████▍    | 122501/225000 [250:22:28<172:09:07,  6.05s/it]
 54%|█████▍    | 122502/225000 [250:22:32<154:11:10,  5.42s/it]
 54%|█████▍    | 122503/225000 [250:22:42<196:11:50,  6.89s/it]
 54%|█████▍    | 122504/225000 [250:22:47<172:05:51,  6.04s/it]
 54%|█████▍    | 122505/225000 [250:22:51<155:30:04,  5.46s/it]
 54%|█████▍    | 122506/225000 [250:22:55<145:12:46,  5.10s/it]
 54%|█████▍    | 122507/225000 [250:22:59<136:49:09,  4.81s/it]
 54%|█████▍    | 122508/225000 [250:23:06<155:16:40,  5.45s/it]
 54%|█████▍    | 122509/225000 [250:23:15<187:11:10,  6.57s/it]
 54%|█████▍    | 122510/225000 [250:23:18<158:41:19,  5.57s/it]
 54%|█████▍    | 122511/225000 [250:23:23<152:56:11,  5.37s/it]
 54%|█████▍    | 122512/225000 [250:23:32<179:03:06,  6.29s/it]
 54%|█████▍    | 122513/225000 [250:23:36<158:57:26,  5.58s/it]
 54%|█████▍    | 122514/225000 [250:23:40<148:24:54,  5.21s/it]
 54%|█████▍    | 122515/225000 [250:23:50<190:09:08,  6.68s/it]
 54%|█████▍    | 122516/225000 [250:23:55<171:02:54,  6.01s/it]
 54%|█████▍    | 122517/225000 [250:23:59<161:38:23,  5.68s/it]
 54%|█████▍    | 122518/225000 [250:24:04<152:25:02,  5.35s/it]
 54%|█████▍    | 122519/225000 [250:24:09<149:18:17,  5.24s/it]
 54%|█████▍    | 122520/225000 [250:24:14<143:50:39,  5.05s/it]
 54%|█████▍    | 122521/225000 [250:24:19<145:32:43,  5.11s/it]
 54%|█████▍    | 122522/225000 [250:24:24<142:12:05,  5.00s/it]
 54%|█████▍    | 122523/225000 [250:24:29<141:06:10,  4.96s/it]
 54%|█████▍    | 122524/225000 [250:24:34<141:30:31,  4.97s/it]
 54%|█████▍    | 122525/225000 [250:24:39<143:09:39,  5.03s/it]
 54%|█████▍    | 122526/225000 [250:24:44<145:34:21,  5.11s/it]
 54%|█████▍    | 122527/225000 [250:24:49<146:24:30,  5.14s/it]
 54%|█████▍    | 122528/225000 [250:24:55<148:46:28,  5.23s/it]
 54%|█████▍    | 122529/225000 [250:25:00<145:58:34,  5.13s/it]
 54%|█████▍    | 122530/225000 [250:25:05<146:29:45,  5.15s/it]
 54%|█████▍    | 122531/225000 [250:25:10<147:31:24,  5.18s/it]
 54%|█████▍    | 122532/225000 [250:25:21<193:22:19,  6.79s/it]
 54%|█████▍    | 122533/225000 [250:25:30<213:04:21,  7.49s/it]
 54%|█████▍    | 122534/225000 [250:25:33<175:15:05,  6.16s/it]
 54%|█████▍    | 122535/225000 [250:25:37<158:51:51,  5.58s/it]
 54%|█████▍    | 122536/225000 [250:25:41<149:54:36,  5.27s/it]
 54%|█████▍    | 122537/225000 [250:25:46<139:27:26,  4.90s/it]
 54%|█████▍    | 122538/225000 [250:25:50<134:21:26,  4.72s/it]
 54%|█████▍    | 122539/225000 [250:25:57<154:12:47,  5.42s/it]
 54%|█████▍    | 122540/225000 [250:26:08<206:56:46,  7.27s/it]
 54%|█████▍    | 122541/225000 [250:26:12<175:57:54,  6.18s/it]
 54%|█████▍    | 122542/225000 [250:26:16<157:22:52,  5.53s/it]
 54%|█████▍    | 122543/225000 [250:26:21<149:35:01,  5.26s/it]
 54%|█████▍    | 122544/225000 [250:26:31<192:07:29,  6.75s/it]
 54%|█████▍    | 122545/225000 [250:26:35<170:37:52,  6.00s/it]
 54%|█████▍    | 122546/225000 [250:26:39<152:42:04,  5.37s/it]
 54%|█████▍    | 122547/225000 [250:26:44<148:43:23,  5.23s/it]
 54%|█████▍    | 122548/225000 [250:26:49<146:06:23,  5.13s/it]
 54%|█████▍    | 122549/225000 [250:26:53<141:09:06,  4.96s/it]
 54%|█████▍    | 122550/225000 [250:26:58<141:14:55,  4.96s/it]
                                                               
{'loss': '0.8456', 'grad_norm': '1.166', 'learning_rate': '2.318e-05', 'epoch': '0.5447', 'num_input_tokens_seen': 417282336, 'train_runtime': '9.016e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122550/225000 [250:26:58<141:14:55,  4.96s/it]
 54%|█████▍    | 122551/225000 [250:27:04<142:06:30,  4.99s/it]
 54%|█████▍    | 122552/225000 [250:27:09<143:47:37,  5.05s/it]
 54%|█████▍    | 122553/225000 [250:27:19<192:19:09,  6.76s/it]
 54%|█████▍    | 122554/225000 [250:27:27<200:25:43,  7.04s/it]
 54%|█████▍    | 122555/225000 [250:27:30<165:53:18,  5.83s/it]
 54%|█████▍    | 122556/225000 [250:27:37<176:10:51,  6.19s/it]
 54%|█████▍    | 122557/225000 [250:27:45<186:38:58,  6.56s/it]
 54%|█████▍    | 122558/225000 [250:27:49<169:26:04,  5.95s/it]
 54%|█████▍    | 122559/225000 [250:27:53<155:39:55,  5.47s/it]
 54%|█████▍    | 122560/225000 [250:27:59<152:14:34,  5.35s/it]
 54%|█████▍    | 122561/225000 [250:28:08<185:15:57,  6.51s/it]
 54%|█████▍    | 122562/225000 [250:28:12<169:02:37,  5.94s/it]
 54%|█████▍    | 122563/225000 [250:28:18<164:20:32,  5.78s/it]
 54%|█████▍    | 122564/225000 [250:28:22<154:39:42,  5.44s/it]
 54%|█████▍    | 122565/225000 [250:28:27<150:08:15,  5.28s/it]
 54%|█████▍    | 122566/225000 [250:28:32<145:50:10,  5.13s/it]
 54%|█████▍    | 122567/225000 [250:28:37<145:54:40,  5.13s/it]
 54%|█████▍    | 122568/225000 [250:28:42<144:23:10,  5.07s/it]
 54%|█████▍    | 122569/225000 [250:28:47<142:01:28,  4.99s/it]
 54%|█████▍    | 122570/225000 [250:28:52<138:33:23,  4.87s/it]
 54%|█████▍    | 122571/225000 [250:28:56<136:59:20,  4.81s/it]
 54%|█████▍    | 122572/225000 [250:29:01<138:29:23,  4.87s/it]
 54%|█████▍    | 122573/225000 [250:29:06<140:25:21,  4.94s/it]
 54%|█████▍    | 122574/225000 [250:29:13<153:52:20,  5.41s/it]
 54%|█████▍    | 122575/225000 [250:29:26<217:07:38,  7.63s/it]
 54%|█████▍    | 122576/225000 [250:29:31<193:38:53,  6.81s/it]
 54%|█████▍    | 122577/225000 [250:29:38<198:02:58,  6.96s/it]
 54%|█████▍    | 122578/225000 [250:29:41<166:19:53,  5.85s/it]
 54%|█████▍    | 122579/225000 [250:29:48<172:10:12,  6.05s/it]
 54%|█████▍    | 122580/225000 [250:29:54<174:21:30,  6.13s/it]
 54%|█████▍    | 122581/225000 [250:29:57<149:22:31,  5.25s/it]
 54%|█████▍    | 122582/225000 [250:30:04<164:23:18,  5.78s/it]
 54%|█████▍    | 122583/225000 [250:30:10<167:11:44,  5.88s/it]
 54%|█████▍    | 122584/225000 [250:30:15<153:28:55,  5.40s/it]
 54%|█████▍    | 122585/225000 [250:30:18<135:32:57,  4.76s/it]
 54%|█████▍    | 122586/225000 [250:30:23<135:44:56,  4.77s/it]
 54%|█████▍    | 122587/225000 [250:30:28<140:42:46,  4.95s/it]
 54%|█████▍    | 122588/225000 [250:30:33<145:26:44,  5.11s/it]
 54%|█████▍    | 122589/225000 [250:30:39<145:59:13,  5.13s/it]
 54%|█████▍    | 122590/225000 [250:30:44<147:37:28,  5.19s/it]
 54%|█████▍    | 122591/225000 [250:30:49<145:59:06,  5.13s/it]
 54%|█████▍    | 122592/225000 [250:30:58<178:08:52,  6.26s/it]
 54%|█████▍    | 122593/225000 [250:31:04<173:48:03,  6.11s/it]
 54%|█████▍    | 122594/225000 [250:31:09<167:31:54,  5.89s/it]
 54%|█████▍    | 122595/225000 [250:31:14<158:06:49,  5.56s/it]
 54%|█████▍    | 122596/225000 [250:31:19<154:55:37,  5.45s/it]
 54%|█████▍    | 122597/225000 [250:31:24<154:07:08,  5.42s/it]
 54%|█████▍    | 122598/225000 [250:31:30<156:27:50,  5.50s/it]
 54%|█████▍    | 122599/225000 [250:31:36<157:05:36,  5.52s/it]
 54%|█████▍    | 122600/225000 [250:31:41<155:04:24,  5.45s/it]
                                                               
{'loss': '0.8476', 'grad_norm': '1.055', 'learning_rate': '2.316e-05', 'epoch': '0.5449', 'num_input_tokens_seen': 417409760, 'train_runtime': '9.019e+05', 'train_tokens_per_second': '462.8'}

 54%|█████▍    | 122600/225000 [250:31:41<155:04:24,  5.45s/it]
 54%|█████▍    | 122601/225000 [250:31:46<152:11:34,  5.35s/it]
 54%|█████▍    | 122602/225000 [250:31:51<149:23:41,  5.25s/it]
 54%|█████▍    | 122603/225000 [250:31:56<146:15:14,  5.14s/it]
 54%|█████▍    | 122604/225000 [250:32:04<172:36:00,  6.07s/it]
 54%|█████▍    | 122605/225000 [250:32:14<202:02:56,  7.10s/it]
 54%|█████▍    | 122606/225000 [250:32:17<171:29:07,  6.03s/it]
 54%|█████▍    | 122607/225000 [250:32:21<154:21:53,  5.43s/it]
 54%|█████▍    | 122608/225000 [250:32:26<145:30:03,  5.12s/it]
 54%|█████▍    | 122609/225000 [250:32:31<152:17:49,  5.35s/it]
 54%|█████▍    | 122610/225000 [250:32:41<189:37:11,  6.67s/it]
 54%|█████▍    | 122611/225000 [250:32:45<166:34:16,  5.86s/it]
 54%|█████▍    | 122612/225000 [250:32:51<168:33:41,  5.93s/it]
 54%|█████▍    | 122613/225000 [250:32:55<148:21:15,  5.22s/it]
 54%|█████▍    | 122614/225000 [250:33:03<172:05:19,  6.05s/it]
 54%|█████▍    | 122615/225000 [250:33:06<150:26:52,  5.29s/it]
 54%|█████▍    | 122616/225000 [250:33:12<150:53:02,  5.31s/it]
 54%|█████▍    | 122617/225000 [250:33:19<167:42:09,  5.90s/it]
 54%|█████▍    | 122618/225000 [250:33:22<147:27:52,  5.19s/it]
 54%|█████▍    | 122619/225000 [250:33:32<184:05:45,  6.47s/it]
 54%|█████▍    | 122620/225000 [250:33:36<160:27:39,  5.64s/it]
 54%|█████▍    | 122621/225000 [250:33:40<146:34:25,  5.15s/it]
 54%|█████▍    | 122622/225000 [250:33:43<133:59:03,  4.71s/it]
 54%|█████▍    | 122623/225000 [250:33:52<165:13:21,  5.81s/it]
 54%|█████▍    | 122624/225000 [250:33:57<164:00:04,  5.77s/it]
 55%|█████▍    | 122625/225000 [250:34:02<150:09:27,  5.28s/it]
 55%|█████▍    | 122626/225000 [250:34:09<171:53:57,  6.04s/it]
 55%|█████▍    | 122627/225000 [250:34:15<165:32:35,  5.82s/it]
 55%|█████▍    | 122628/225000 [250:34:22<181:44:41,  6.39s/it]
 55%|█████▍    | 122629/225000 [250:34:26<156:49:00,  5.51s/it]
 55%|█████▍    | 122630/225000 [250:34:32<158:20:12,  5.57s/it]
 55%|█████▍    | 122631/225000 [250:34:40<185:22:15,  6.52s/it]
 55%|█████▍    | 122632/225000 [250:34:45<167:28:45,  5.89s/it]
 55%|█████▍    | 122633/225000 [250:34:49<153:23:31,  5.39s/it]
 55%|█████▍    | 122634/225000 [250:34:53<140:45:49,  4.95s/it]
 55%|█████▍    | 122635/225000 [250:34:57<134:57:59,  4.75s/it]
 55%|█████▍    | 122636/225000 [250:35:01<129:51:40,  4.57s/it]
 55%|█████▍    | 122637/225000 [250:35:06<132:22:22,  4.66s/it]
 55%|█████▍    | 122638/225000 [250:35:12<146:29:27,  5.15s/it]
 55%|█████▍    | 122639/225000 [250:35:25<205:40:49,  7.23s/it]
 55%|█████▍    | 122640/225000 [250:35:28<171:39:08,  6.04s/it]
 55%|█████▍    | 122641/225000 [250:35:36<189:20:02,  6.66s/it]
 55%|█████▍    | 122642/225000 [250:35:40<168:13:38,  5.92s/it]
 55%|█████▍    | 122643/225000 [250:35:44<155:18:50,  5.46s/it]
 55%|█████▍    | 122644/225000 [250:35:49<145:36:40,  5.12s/it]
 55%|█████▍    | 122645/225000 [250:35:57<168:01:53,  5.91s/it]
 55%|█████▍    | 122646/225000 [250:36:05<186:44:50,  6.57s/it]
 55%|█████▍    | 122647/225000 [250:36:11<187:53:22,  6.61s/it]
 55%|█████▍    | 122648/225000 [250:36:15<160:37:49,  5.65s/it]
 55%|█████▍    | 122649/225000 [250:36:19<145:26:25,  5.12s/it]
 55%|█████▍    | 122650/225000 [250:36:27<169:58:56,  5.98s/it]
                                                               
{'loss': '0.8536', 'grad_norm': '1.061', 'learning_rate': '2.314e-05', 'epoch': '0.5451', 'num_input_tokens_seen': 417537976, 'train_runtime': '9.022e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122650/225000 [250:36:27<169:58:56,  5.98s/it]
 55%|█████▍    | 122651/225000 [250:36:32<166:47:27,  5.87s/it]
 55%|█████▍    | 122652/225000 [250:36:36<152:51:37,  5.38s/it]
 55%|█████▍    | 122653/225000 [250:36:41<149:02:34,  5.24s/it]
 55%|█████▍    | 122654/225000 [250:36:47<148:03:44,  5.21s/it]
 55%|█████▍    | 122655/225000 [250:36:51<144:00:09,  5.07s/it]
 55%|█████▍    | 122656/225000 [250:36:56<140:54:03,  4.96s/it]
 55%|█████▍    | 122657/225000 [250:37:01<140:57:23,  4.96s/it]
 55%|█████▍    | 122658/225000 [250:37:06<138:09:37,  4.86s/it]
 55%|█████▍    | 122659/225000 [250:37:10<137:33:34,  4.84s/it]
 55%|█████▍    | 122660/225000 [250:37:17<151:10:13,  5.32s/it]
 55%|█████▍    | 122661/225000 [250:37:29<206:58:48,  7.28s/it]
 55%|█████▍    | 122662/225000 [250:37:33<181:39:00,  6.39s/it]
 55%|█████▍    | 122663/225000 [250:37:37<161:57:52,  5.70s/it]
 55%|█████▍    | 122664/225000 [250:37:46<187:49:38,  6.61s/it]
 55%|█████▍    | 122665/225000 [250:37:49<161:11:43,  5.67s/it]
 55%|█████▍    | 122666/225000 [250:37:56<173:07:27,  6.09s/it]
 55%|█████▍    | 122667/225000 [250:38:02<170:16:15,  5.99s/it]
 55%|█████▍    | 122668/225000 [250:38:07<158:56:57,  5.59s/it]
 55%|█████▍    | 122669/225000 [250:38:11<150:50:05,  5.31s/it]
 55%|█████▍    | 122670/225000 [250:38:16<145:16:40,  5.11s/it]
 55%|█████▍    | 122671/225000 [250:38:21<143:47:36,  5.06s/it]
 55%|█████▍    | 122672/225000 [250:38:26<143:39:37,  5.05s/it]
 55%|█████▍    | 122673/225000 [250:38:33<157:40:37,  5.55s/it]
 55%|█████▍    | 122674/225000 [250:38:43<197:57:53,  6.96s/it]
 55%|█████▍    | 122675/225000 [250:38:46<167:39:02,  5.90s/it]
 55%|█████▍    | 122676/225000 [250:38:54<185:06:32,  6.51s/it]
 55%|█████▍    | 122677/225000 [250:38:59<167:21:13,  5.89s/it]
 55%|█████▍    | 122678/225000 [250:39:03<151:36:23,  5.33s/it]
 55%|█████▍    | 122679/225000 [250:39:11<177:55:10,  6.26s/it]
 55%|█████▍    | 122680/225000 [250:39:17<172:45:39,  6.08s/it]
 55%|█████▍    | 122681/225000 [250:39:22<160:49:03,  5.66s/it]
 55%|█████▍    | 122682/225000 [250:39:26<150:47:38,  5.31s/it]
 55%|█████▍    | 122683/225000 [250:39:30<139:28:07,  4.91s/it]
 55%|█████▍    | 122684/225000 [250:39:35<135:47:23,  4.78s/it]
 55%|█████▍    | 122685/225000 [250:39:45<183:00:20,  6.44s/it]
 55%|█████▍    | 122686/225000 [250:39:51<179:31:51,  6.32s/it]
 55%|█████▍    | 122687/225000 [250:39:58<189:36:47,  6.67s/it]
 55%|█████▍    | 122688/225000 [250:40:02<167:22:09,  5.89s/it]
 55%|█████▍    | 122689/225000 [250:40:09<175:10:39,  6.16s/it]
 55%|█████▍    | 122690/225000 [250:40:15<169:56:52,  5.98s/it]
 55%|█████▍    | 122691/225000 [250:40:18<148:33:13,  5.23s/it]
 55%|█████▍    | 122692/225000 [250:40:24<149:15:44,  5.25s/it]
 55%|█████▍    | 122693/225000 [250:40:33<182:44:18,  6.43s/it]
 55%|█████▍    | 122694/225000 [250:40:38<169:22:02,  5.96s/it]
 55%|█████▍    | 122695/225000 [250:40:42<158:30:17,  5.58s/it]
 55%|█████▍    | 122696/225000 [250:40:48<157:03:18,  5.53s/it]
 55%|█████▍    | 122697/225000 [250:40:53<151:57:38,  5.35s/it]
 55%|█████▍    | 122698/225000 [250:40:58<149:45:34,  5.27s/it]
 55%|█████▍    | 122699/225000 [250:41:03<147:24:57,  5.19s/it]
 55%|█████▍    | 122700/225000 [250:41:07<141:28:40,  4.98s/it]
                                                               
{'loss': '0.8484', 'grad_norm': '1.021', 'learning_rate': '2.312e-05', 'epoch': '0.5453', 'num_input_tokens_seen': 417666856, 'train_runtime': '9.025e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122700/225000 [250:41:07<141:28:40,  4.98s/it]
 55%|█████▍    | 122701/225000 [250:41:12<141:07:31,  4.97s/it]
 55%|█████▍    | 122702/225000 [250:41:17<144:12:12,  5.07s/it]
 55%|█████▍    | 122703/225000 [250:41:23<146:02:18,  5.14s/it]
 55%|█████▍    | 122704/225000 [250:41:28<145:24:17,  5.12s/it]
 55%|█████▍    | 122705/225000 [250:41:35<162:53:59,  5.73s/it]
 55%|█████▍    | 122706/225000 [250:41:40<159:33:19,  5.62s/it]
 55%|█████▍    | 122707/225000 [250:41:46<157:49:04,  5.55s/it]
 55%|█████▍    | 122708/225000 [250:41:50<147:59:23,  5.21s/it]
 55%|█████▍    | 122709/225000 [250:41:55<148:28:01,  5.23s/it]
 55%|█████▍    | 122710/225000 [250:42:01<149:40:02,  5.27s/it]
 55%|█████▍    | 122711/225000 [250:42:13<209:52:26,  7.39s/it]
 55%|█████▍    | 122712/225000 [250:42:20<206:29:26,  7.27s/it]
 55%|█████▍    | 122713/225000 [250:42:25<187:43:39,  6.61s/it]
 55%|█████▍    | 122714/225000 [250:42:31<177:05:50,  6.23s/it]
 55%|█████▍    | 122715/225000 [250:42:42<225:26:48,  7.93s/it]
 55%|█████▍    | 122716/225000 [250:42:47<193:08:13,  6.80s/it]
 55%|█████▍    | 122717/225000 [250:42:51<172:22:21,  6.07s/it]
 55%|█████▍    | 122718/225000 [250:42:56<161:31:47,  5.69s/it]
 55%|█████▍    | 122719/225000 [250:43:00<149:57:38,  5.28s/it]
 55%|█████▍    | 122720/225000 [250:43:05<148:35:56,  5.23s/it]
 55%|█████▍    | 122721/225000 [250:43:10<145:58:00,  5.14s/it]
 55%|█████▍    | 122722/225000 [250:43:15<140:38:34,  4.95s/it]
 55%|█████▍    | 122723/225000 [250:43:22<159:58:56,  5.63s/it]
 55%|█████▍    | 122724/225000 [250:43:33<210:13:18,  7.40s/it]
 55%|█████▍    | 122725/225000 [250:43:37<179:01:41,  6.30s/it]
 55%|█████▍    | 122726/225000 [250:43:46<200:47:47,  7.07s/it]
 55%|█████▍    | 122727/225000 [250:43:50<175:55:04,  6.19s/it]
 55%|█████▍    | 122728/225000 [250:43:54<159:16:06,  5.61s/it]
 55%|█████▍    | 122729/225000 [250:44:01<170:56:36,  6.02s/it]
 55%|█████▍    | 122730/225000 [250:44:06<162:05:56,  5.71s/it]
 55%|█████▍    | 122731/225000 [250:44:12<158:21:21,  5.57s/it]
 55%|█████▍    | 122732/225000 [250:44:17<155:46:31,  5.48s/it]
 55%|█████▍    | 122733/225000 [250:44:22<149:49:53,  5.27s/it]
 55%|█████▍    | 122734/225000 [250:44:26<146:08:25,  5.14s/it]
 55%|█████▍    | 122735/225000 [250:44:32<146:44:41,  5.17s/it]
 55%|█████▍    | 122736/225000 [250:44:37<146:33:29,  5.16s/it]
 55%|█████▍    | 122737/225000 [250:44:42<149:16:20,  5.25s/it]
 55%|█████▍    | 122738/225000 [250:44:47<148:09:35,  5.22s/it]
 55%|█████▍    | 122739/225000 [250:44:52<143:27:33,  5.05s/it]
 55%|█████▍    | 122740/225000 [250:44:56<137:39:44,  4.85s/it]
 55%|█████▍    | 122741/225000 [250:45:05<165:54:10,  5.84s/it]
 55%|█████▍    | 122742/225000 [250:45:17<222:20:41,  7.83s/it]
 55%|█████▍    | 122743/225000 [250:45:22<194:57:41,  6.86s/it]
 55%|█████▍    | 122744/225000 [250:45:27<177:44:35,  6.26s/it]
 55%|█████▍    | 122745/225000 [250:45:31<165:12:29,  5.82s/it]
 55%|█████▍    | 122746/225000 [250:45:35<148:58:37,  5.24s/it]
 55%|█████▍    | 122747/225000 [250:45:40<147:37:24,  5.20s/it]
 55%|█████▍    | 122748/225000 [250:45:45<142:38:20,  5.02s/it]
 55%|█████▍    | 122749/225000 [250:45:50<141:21:33,  4.98s/it]
 55%|█████▍    | 122750/225000 [250:45:55<144:44:33,  5.10s/it]
                                                               
{'loss': '0.8574', 'grad_norm': '1.056', 'learning_rate': '2.31e-05', 'epoch': '0.5456', 'num_input_tokens_seen': 417796680, 'train_runtime': '9.028e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122750/225000 [250:45:55<144:44:33,  5.10s/it]
 55%|█████▍    | 122751/225000 [250:46:00<143:40:01,  5.06s/it]
 55%|█████▍    | 122752/225000 [250:46:08<164:12:21,  5.78s/it]
 55%|█████▍    | 122753/225000 [250:46:19<210:59:52,  7.43s/it]
 55%|█████▍    | 122754/225000 [250:46:23<179:42:49,  6.33s/it]
 55%|█████▍    | 122755/225000 [250:46:28<168:32:39,  5.93s/it]
 55%|█████▍    | 122756/225000 [250:46:35<180:09:26,  6.34s/it]
 55%|█████▍    | 122757/225000 [250:46:39<156:53:58,  5.52s/it]
 55%|█████▍    | 122758/225000 [250:46:46<171:47:58,  6.05s/it]
 55%|█████▍    | 122759/225000 [250:46:52<168:43:50,  5.94s/it]
 55%|█████▍    | 122760/225000 [250:46:59<181:20:33,  6.39s/it]
 55%|█████▍    | 122761/225000 [250:47:03<157:26:27,  5.54s/it]
 55%|█████▍    | 122762/225000 [250:47:07<149:52:17,  5.28s/it]
 55%|█████▍    | 122763/225000 [250:47:12<143:02:50,  5.04s/it]
 55%|█████▍    | 122764/225000 [250:47:16<139:45:11,  4.92s/it]
 55%|█████▍    | 122765/225000 [250:47:21<138:48:01,  4.89s/it]
 55%|█████▍    | 122766/225000 [250:47:25<133:59:38,  4.72s/it]
 55%|█████▍    | 122767/225000 [250:47:38<196:52:07,  6.93s/it]
 55%|█████▍    | 122768/225000 [250:47:42<177:29:34,  6.25s/it]
 55%|█████▍    | 122769/225000 [250:47:50<190:55:55,  6.72s/it]
 55%|█████▍    | 122770/225000 [250:47:54<167:11:01,  5.89s/it]
 55%|█████▍    | 122771/225000 [250:47:58<147:17:12,  5.19s/it]
 55%|█████▍    | 122772/225000 [250:48:01<136:01:19,  4.79s/it]
 55%|█████▍    | 122773/225000 [250:48:11<176:06:58,  6.20s/it]
 55%|█████▍    | 122774/225000 [250:48:16<170:01:21,  5.99s/it]
 55%|█████▍    | 122775/225000 [250:48:26<196:43:42,  6.93s/it]
 55%|█████▍    | 122776/225000 [250:48:29<165:38:54,  5.83s/it]
 55%|█████▍    | 122777/225000 [250:48:32<146:13:46,  5.15s/it]
 55%|█████▍    | 122778/225000 [250:48:42<182:43:36,  6.44s/it]
 55%|█████▍    | 122779/225000 [250:48:46<163:36:01,  5.76s/it]
 55%|█████▍    | 122780/225000 [250:48:50<148:11:31,  5.22s/it]
 55%|█████▍    | 122781/225000 [250:48:57<164:46:02,  5.80s/it]
 55%|█████▍    | 122782/225000 [250:49:04<170:08:08,  5.99s/it]
 55%|█████▍    | 122783/225000 [250:49:10<174:34:13,  6.15s/it]
 55%|█████▍    | 122784/225000 [250:49:15<161:21:30,  5.68s/it]
 55%|█████▍    | 122785/225000 [250:49:19<154:07:39,  5.43s/it]
 55%|█████▍    | 122786/225000 [250:49:24<146:43:36,  5.17s/it]
 55%|█████▍    | 122787/225000 [250:49:29<144:32:23,  5.09s/it]
 55%|█████▍    | 122788/225000 [250:49:34<142:00:56,  5.00s/it]
 55%|█████▍    | 122789/225000 [250:49:39<144:44:53,  5.10s/it]
 55%|█████▍    | 122790/225000 [250:49:44<140:58:23,  4.97s/it]
 55%|█████▍    | 122791/225000 [250:49:49<141:55:43,  5.00s/it]
 55%|█████▍    | 122792/225000 [250:49:54<141:15:47,  4.98s/it]
 55%|█████▍    | 122793/225000 [250:50:06<205:04:01,  7.22s/it]
 55%|█████▍    | 122794/225000 [250:50:12<194:57:09,  6.87s/it]
 55%|█████▍    | 122795/225000 [250:50:20<199:46:33,  7.04s/it]
 55%|█████▍    | 122796/225000 [250:50:23<165:06:37,  5.82s/it]
 55%|█████▍    | 122797/225000 [250:50:27<155:42:51,  5.48s/it]
 55%|█████▍    | 122798/225000 [250:50:32<149:13:07,  5.26s/it]
 55%|█████▍    | 122799/225000 [250:50:37<142:24:37,  5.02s/it]
 55%|█████▍    | 122800/225000 [250:50:41<137:08:53,  4.83s/it]
                                                               
{'loss': '0.839', 'grad_norm': '1.045', 'learning_rate': '2.309e-05', 'epoch': '0.5458', 'num_input_tokens_seen': 417925720, 'train_runtime': '9.03e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122800/225000 [250:50:41<137:08:53,  4.83s/it]
 55%|█████▍    | 122801/225000 [250:50:46<141:13:05,  4.97s/it]
 55%|█████▍    | 122802/225000 [250:50:51<141:10:05,  4.97s/it]
 55%|█████▍    | 122803/225000 [250:50:56<142:20:28,  5.01s/it]
 55%|█████▍    | 122804/225000 [250:51:02<145:02:09,  5.11s/it]
 55%|█████▍    | 122805/225000 [250:51:07<146:30:45,  5.16s/it]
 55%|█████▍    | 122806/225000 [250:51:12<149:08:34,  5.25s/it]
 55%|█████▍    | 122807/225000 [250:51:17<146:14:33,  5.15s/it]
 55%|█████▍    | 122808/225000 [250:51:23<147:59:28,  5.21s/it]
 55%|█████▍    | 122809/225000 [250:51:32<184:32:54,  6.50s/it]
 55%|█████▍    | 122810/225000 [250:51:41<208:08:40,  7.33s/it]
 55%|█████▍    | 122811/225000 [250:51:46<185:46:25,  6.54s/it]
 55%|█████▍    | 122812/225000 [250:51:50<166:40:47,  5.87s/it]
 55%|█████▍    | 122813/225000 [250:51:56<160:44:04,  5.66s/it]
 55%|█████▍    | 122814/225000 [250:52:00<153:15:52,  5.40s/it]
 55%|█████▍    | 122815/225000 [250:52:05<147:51:33,  5.21s/it]
 55%|█████▍    | 122816/225000 [250:52:09<139:38:28,  4.92s/it]
 55%|█████▍    | 122817/225000 [250:52:20<185:25:09,  6.53s/it]
 55%|█████▍    | 122818/225000 [250:52:25<175:40:51,  6.19s/it]
 55%|█████▍    | 122819/225000 [250:52:32<184:59:19,  6.52s/it]
 55%|█████▍    | 122820/225000 [250:52:36<160:46:44,  5.66s/it]
 55%|█████▍    | 122821/225000 [250:52:42<161:31:38,  5.69s/it]
 55%|█████▍    | 122822/225000 [250:52:49<175:40:21,  6.19s/it]
 55%|█████▍    | 122823/225000 [250:52:53<152:09:44,  5.36s/it]
 55%|█████▍    | 122824/225000 [250:53:00<166:38:53,  5.87s/it]
 55%|█████▍    | 122825/225000 [250:53:03<147:15:21,  5.19s/it]
 55%|█████▍    | 122826/225000 [250:53:13<186:09:06,  6.56s/it]
 55%|█████▍    | 122827/225000 [250:53:17<166:21:11,  5.86s/it]
 55%|█████▍    | 122828/225000 [250:53:21<151:04:38,  5.32s/it]
 55%|█████▍    | 122829/225000 [250:53:25<139:53:28,  4.93s/it]
 55%|█████▍    | 122830/225000 [250:53:30<133:50:51,  4.72s/it]
 55%|█████▍    | 122831/225000 [250:53:40<184:09:17,  6.49s/it]
 55%|█████▍    | 122832/225000 [250:53:46<177:12:49,  6.24s/it]
 55%|█████▍    | 122833/225000 [250:53:49<152:02:04,  5.36s/it]
 55%|█████▍    | 122834/225000 [250:53:59<188:32:39,  6.64s/it]
 55%|█████▍    | 122835/225000 [250:54:04<174:51:09,  6.16s/it]
 55%|█████▍    | 122836/225000 [250:54:08<162:07:56,  5.71s/it]
 55%|█████▍    | 122837/225000 [250:54:13<149:05:30,  5.25s/it]
 55%|█████▍    | 122838/225000 [250:54:17<141:33:33,  4.99s/it]
 55%|█████▍    | 122839/225000 [250:54:21<134:01:56,  4.72s/it]
 55%|█████▍    | 122840/225000 [250:54:26<134:28:38,  4.74s/it]
 55%|█████▍    | 122841/225000 [250:54:37<189:47:44,  6.69s/it]
 55%|█████▍    | 122842/225000 [250:54:40<160:49:00,  5.67s/it]
 55%|█████▍    | 122843/225000 [250:54:45<154:17:04,  5.44s/it]
 55%|█████▍    | 122844/225000 [250:54:50<146:28:26,  5.16s/it]
 55%|█████▍    | 122845/225000 [250:54:55<148:40:35,  5.24s/it]
 55%|█████▍    | 122846/225000 [250:55:00<146:36:15,  5.17s/it]
 55%|█████▍    | 122847/225000 [250:55:06<150:18:43,  5.30s/it]
 55%|█████▍    | 122848/225000 [250:55:11<151:49:25,  5.35s/it]
 55%|█████▍    | 122849/225000 [250:55:17<153:01:51,  5.39s/it]
 55%|█████▍    | 122850/225000 [250:55:22<151:29:53,  5.34s/it]
                                                               
{'loss': '0.8501', 'grad_norm': '1.007', 'learning_rate': '2.307e-05', 'epoch': '0.546', 'num_input_tokens_seen': 418053392, 'train_runtime': '9.033e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122850/225000 [250:55:22<151:29:53,  5.34s/it]
 55%|█████▍    | 122851/225000 [250:55:28<154:13:52,  5.44s/it]
 55%|█████▍    | 122852/225000 [250:55:33<152:59:51,  5.39s/it]
 55%|█████▍    | 122853/225000 [250:55:39<155:40:15,  5.49s/it]
 55%|█████▍    | 122854/225000 [250:55:47<180:36:50,  6.37s/it]
 55%|█████▍    | 122855/225000 [250:55:57<209:48:52,  7.39s/it]
 55%|█████▍    | 122856/225000 [250:56:01<183:21:51,  6.46s/it]
 55%|█████▍    | 122857/225000 [250:56:05<163:11:37,  5.75s/it]
 55%|█████▍    | 122858/225000 [250:56:10<152:39:35,  5.38s/it]
 55%|█████▍    | 122859/225000 [250:56:15<148:38:52,  5.24s/it]
 55%|█████▍    | 122860/225000 [250:56:20<147:15:51,  5.19s/it]
 55%|█████▍    | 122861/225000 [250:56:25<146:29:59,  5.16s/it]
 55%|█████▍    | 122862/225000 [250:56:30<147:03:38,  5.18s/it]
 55%|█████▍    | 122863/225000 [250:56:41<199:30:31,  7.03s/it]
 55%|█████▍    | 122864/225000 [250:56:49<202:56:15,  7.15s/it]
 55%|█████▍    | 122865/225000 [250:56:52<172:43:30,  6.09s/it]
 55%|█████▍    | 122866/225000 [250:56:56<152:58:37,  5.39s/it]
 55%|█████▍    | 122867/225000 [250:57:07<200:46:09,  7.08s/it]
 55%|█████▍    | 122868/225000 [250:57:11<169:11:32,  5.96s/it]
 55%|█████▍    | 122869/225000 [250:57:18<180:49:58,  6.37s/it]
 55%|█████▍    | 122870/225000 [250:57:21<156:11:09,  5.51s/it]
 55%|█████▍    | 122871/225000 [250:57:26<148:26:56,  5.23s/it]
 55%|█████▍    | 122872/225000 [250:57:31<148:39:00,  5.24s/it]
 55%|█████▍    | 122873/225000 [250:57:36<146:38:19,  5.17s/it]
 55%|█████▍    | 122874/225000 [250:57:41<142:35:16,  5.03s/it]
 55%|█████▍    | 122875/225000 [250:57:47<148:30:35,  5.24s/it]
 55%|█████▍    | 122876/225000 [250:57:52<146:59:24,  5.18s/it]
 55%|█████▍    | 122877/225000 [250:57:57<147:23:35,  5.20s/it]
 55%|█████▍    | 122878/225000 [250:58:02<148:19:15,  5.23s/it]
 55%|█████▍    | 122879/225000 [250:58:15<212:23:51,  7.49s/it]
 55%|█████▍    | 122880/225000 [250:58:20<190:47:01,  6.73s/it]
 55%|█████▍    | 122881/225000 [250:58:24<166:51:17,  5.88s/it]
 55%|█████▍    | 122882/225000 [250:58:29<162:24:19,  5.73s/it]
 55%|█████▍    | 122883/225000 [250:58:39<195:12:34,  6.88s/it]
 55%|█████▍    | 122884/225000 [250:58:45<189:45:00,  6.69s/it]
 55%|█████▍    | 122885/225000 [250:58:50<174:29:10,  6.15s/it]
 55%|█████▍    | 122886/225000 [250:58:54<156:45:49,  5.53s/it]
 55%|█████▍    | 122887/225000 [250:59:02<173:39:17,  6.12s/it]
 55%|█████▍    | 122888/225000 [250:59:05<153:17:47,  5.40s/it]
 55%|█████▍    | 122889/225000 [250:59:14<184:23:52,  6.50s/it]
 55%|█████▍    | 122890/225000 [250:59:18<158:23:20,  5.58s/it]
 55%|█████▍    | 122891/225000 [250:59:27<187:38:54,  6.62s/it]
 55%|█████▍    | 122892/225000 [250:59:30<161:22:20,  5.69s/it]
 55%|█████▍    | 122893/225000 [250:59:34<144:00:46,  5.08s/it]
 55%|█████▍    | 122894/225000 [250:59:39<145:15:47,  5.12s/it]
 55%|█████▍    | 122895/225000 [250:59:48<175:45:11,  6.20s/it]
 55%|█████▍    | 122896/225000 [250:59:51<150:59:19,  5.32s/it]
 55%|█████▍    | 122897/225000 [250:59:58<166:26:14,  5.87s/it]
 55%|█████▍    | 122898/225000 [251:00:02<150:49:12,  5.32s/it]
 55%|█████▍    | 122899/225000 [251:00:07<145:04:07,  5.12s/it]
 55%|█████▍    | 122900/225000 [251:00:12<144:55:10,  5.11s/it]
                                                               
{'loss': '0.832', 'grad_norm': '0.9537', 'learning_rate': '2.305e-05', 'epoch': '0.5462', 'num_input_tokens_seen': 418183040, 'train_runtime': '9.036e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122900/225000 [251:00:12<144:55:10,  5.11s/it]
 55%|█████▍    | 122901/225000 [251:00:23<191:18:40,  6.75s/it]
 55%|█████▍    | 122902/225000 [251:00:29<187:01:21,  6.59s/it]
 55%|█████▍    | 122903/225000 [251:00:33<163:35:06,  5.77s/it]
 55%|█████▍    | 122904/225000 [251:00:36<145:55:34,  5.15s/it]
 55%|█████▍    | 122905/225000 [251:00:40<134:13:28,  4.73s/it]
 55%|█████▍    | 122906/225000 [251:00:52<190:43:57,  6.73s/it]
 55%|█████▍    | 122907/225000 [251:00:55<163:52:26,  5.78s/it]
 55%|█████▍    | 122908/225000 [251:01:03<184:17:56,  6.50s/it]
 55%|█████▍    | 122909/225000 [251:01:08<167:09:25,  5.89s/it]
 55%|█████▍    | 122910/225000 [251:01:11<147:24:58,  5.20s/it]
 55%|█████▍    | 122911/225000 [251:01:20<175:11:00,  6.18s/it]
 55%|█████▍    | 122912/225000 [251:01:24<157:14:50,  5.55s/it]
 55%|█████▍    | 122913/225000 [251:01:28<141:59:56,  5.01s/it]
 55%|█████▍    | 122914/225000 [251:01:32<136:07:41,  4.80s/it]
 55%|█████▍    | 122915/225000 [251:01:42<181:48:00,  6.41s/it]
 55%|█████▍    | 122916/225000 [251:01:46<162:57:29,  5.75s/it]
 55%|█████▍    | 122917/225000 [251:01:52<159:02:27,  5.61s/it]
 55%|█████▍    | 122918/225000 [251:01:56<151:39:25,  5.35s/it]
 55%|█████▍    | 122919/225000 [251:02:11<228:51:26,  8.07s/it]
 55%|█████▍    | 122920/225000 [251:02:14<187:16:44,  6.60s/it]
 55%|█████▍    | 122921/225000 [251:02:18<162:58:45,  5.75s/it]
 55%|█████▍    | 122922/225000 [251:02:23<159:26:50,  5.62s/it]
 55%|█████▍    | 122923/225000 [251:02:31<177:37:38,  6.26s/it]
 55%|█████▍    | 122924/225000 [251:02:35<162:09:10,  5.72s/it]
 55%|█████▍    | 122925/225000 [251:02:44<185:11:47,  6.53s/it]
 55%|█████▍    | 122926/225000 [251:02:47<157:55:30,  5.57s/it]
 55%|█████▍    | 122927/225000 [251:02:52<152:28:16,  5.38s/it]
 55%|█████▍    | 122928/225000 [251:02:57<150:43:32,  5.32s/it]
 55%|█████▍    | 122929/225000 [251:03:02<144:59:42,  5.11s/it]
 55%|█████▍    | 122930/225000 [251:03:07<144:39:46,  5.10s/it]
 55%|█████▍    | 122931/225000 [251:03:12<142:01:21,  5.01s/it]
 55%|█████▍    | 122932/225000 [251:03:17<143:27:44,  5.06s/it]
 55%|█████▍    | 122933/225000 [251:03:22<142:31:44,  5.03s/it]
 55%|█████▍    | 122934/225000 [251:03:27<144:04:24,  5.08s/it]
 55%|█████▍    | 122935/225000 [251:03:32<145:40:09,  5.14s/it]
 55%|█████▍    | 122936/225000 [251:03:44<205:08:24,  7.24s/it]
 55%|█████▍    | 122937/225000 [251:03:48<176:09:55,  6.21s/it]
 55%|█████▍    | 122938/225000 [251:03:52<158:20:13,  5.58s/it]
 55%|█████▍    | 122939/225000 [251:04:01<186:41:33,  6.59s/it]
 55%|█████▍    | 122940/225000 [251:04:05<166:19:30,  5.87s/it]
 55%|█████▍    | 122941/225000 [251:04:10<150:42:32,  5.32s/it]
 55%|█████▍    | 122942/225000 [251:04:14<141:51:45,  5.00s/it]
 55%|█████▍    | 122943/225000 [251:04:26<201:06:53,  7.09s/it]
 55%|█████▍    | 122944/225000 [251:04:30<173:28:39,  6.12s/it]
 55%|█████▍    | 122945/225000 [251:04:34<157:50:03,  5.57s/it]
 55%|█████▍    | 122946/225000 [251:04:42<176:10:15,  6.21s/it]
 55%|█████▍    | 122947/225000 [251:04:48<175:00:06,  6.17s/it]
 55%|█████▍    | 122948/225000 [251:04:57<200:05:35,  7.06s/it]
 55%|█████▍    | 122949/225000 [251:05:01<171:45:52,  6.06s/it]
 55%|█████▍    | 122950/225000 [251:05:04<149:14:29,  5.26s/it]
                                                               
{'loss': '0.8472', 'grad_norm': '1.089', 'learning_rate': '2.303e-05', 'epoch': '0.5464', 'num_input_tokens_seen': 418313168, 'train_runtime': '9.039e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 122950/225000 [251:05:04<149:14:29,  5.26s/it]
 55%|█████▍    | 122951/225000 [251:05:14<187:14:28,  6.61s/it]
 55%|█████▍    | 122952/225000 [251:05:17<157:44:37,  5.56s/it]
 55%|█████▍    | 122953/225000 [251:05:22<155:29:21,  5.49s/it]
 55%|█████▍    | 122954/225000 [251:05:30<173:46:20,  6.13s/it]
 55%|█████▍    | 122955/225000 [251:05:33<148:39:45,  5.24s/it]
 55%|█████▍    | 122956/225000 [251:05:38<143:16:39,  5.05s/it]
 55%|█████▍    | 122957/225000 [251:05:42<137:32:42,  4.85s/it]
 55%|█████▍    | 122958/225000 [251:05:46<127:51:03,  4.51s/it]
 55%|█████▍    | 122959/225000 [251:05:56<178:44:25,  6.31s/it]
 55%|█████▍    | 122960/225000 [251:06:01<163:46:52,  5.78s/it]
 55%|█████▍    | 122961/225000 [251:06:08<176:17:07,  6.22s/it]
 55%|█████▍    | 122962/225000 [251:06:15<182:32:09,  6.44s/it]
 55%|█████▍    | 122963/225000 [251:06:20<171:02:24,  6.03s/it]
 55%|█████▍    | 122964/225000 [251:06:25<160:21:20,  5.66s/it]
 55%|█████▍    | 122965/225000 [251:06:29<149:54:06,  5.29s/it]
 55%|█████▍    | 122966/225000 [251:06:34<147:37:43,  5.21s/it]
 55%|█████▍    | 122967/225000 [251:06:39<146:04:41,  5.15s/it]
 55%|█████▍    | 122968/225000 [251:06:44<143:06:11,  5.05s/it]
 55%|█████▍    | 122969/225000 [251:06:49<143:49:20,  5.07s/it]
 55%|█████▍    | 122970/225000 [251:06:54<142:29:57,  5.03s/it]
 55%|█████▍    | 122971/225000 [251:06:59<140:44:19,  4.97s/it]
 55%|█████▍    | 122972/225000 [251:07:10<197:02:21,  6.95s/it]
 55%|█████▍    | 122973/225000 [251:07:18<198:00:24,  6.99s/it]
 55%|█████▍    | 122974/225000 [251:07:21<169:17:44,  5.97s/it]
 55%|█████▍    | 122975/225000 [251:07:25<151:54:17,  5.36s/it]
 55%|█████▍    | 122976/225000 [251:07:37<203:57:29,  7.20s/it]
 55%|█████▍    | 122977/225000 [251:07:41<180:52:47,  6.38s/it]
 55%|█████▍    | 122978/225000 [251:07:45<164:15:39,  5.80s/it]
 55%|█████▍    | 122979/225000 [251:07:50<152:01:37,  5.36s/it]
 55%|█████▍    | 122980/225000 [251:07:55<147:34:24,  5.21s/it]
 55%|█████▍    | 122981/225000 [251:07:59<141:33:46,  5.00s/it]
 55%|█████▍    | 122982/225000 [251:08:08<177:13:59,  6.25s/it]
 55%|█████▍    | 122983/225000 [251:08:14<168:36:06,  5.95s/it]
 55%|█████▍    | 122984/225000 [251:08:18<152:11:38,  5.37s/it]
 55%|█████▍    | 122985/225000 [251:08:25<165:38:05,  5.85s/it]
 55%|█████▍    | 122986/225000 [251:08:32<180:42:51,  6.38s/it]
 55%|█████▍    | 122987/225000 [251:08:36<158:59:42,  5.61s/it]
 55%|█████▍    | 122988/225000 [251:08:44<180:41:11,  6.38s/it]
 55%|█████▍    | 122989/225000 [251:08:48<161:28:26,  5.70s/it]
 55%|█████▍    | 122990/225000 [251:08:55<168:44:49,  5.96s/it]
 55%|█████▍    | 122991/225000 [251:08:59<151:29:06,  5.35s/it]
 55%|█████▍    | 122992/225000 [251:09:02<137:25:13,  4.85s/it]
 55%|█████▍    | 122993/225000 [251:09:07<132:13:18,  4.67s/it]
 55%|█████▍    | 122994/225000 [251:09:19<192:38:12,  6.80s/it]
 55%|█████▍    | 122995/225000 [251:09:22<162:40:14,  5.74s/it]
 55%|█████▍    | 122996/225000 [251:09:30<179:32:07,  6.34s/it]
 55%|█████▍    | 122997/225000 [251:09:33<153:28:58,  5.42s/it]
 55%|█████▍    | 122998/225000 [251:09:37<144:31:26,  5.10s/it]
 55%|█████▍    | 122999/225000 [251:09:42<138:45:48,  4.90s/it]
 55%|█████▍    | 123000/225000 [251:09:47<142:00:14,  5.01s/it]
                                                               
{'loss': '0.8458', 'grad_norm': '1.028', 'learning_rate': '2.301e-05', 'epoch': '0.5467', 'num_input_tokens_seen': 418441776, 'train_runtime': '9.042e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123000/225000 [251:09:47<142:00:14,  5.01s/it]
 55%|█████▍    | 123001/225000 [251:09:52<142:27:47,  5.03s/it]
 55%|█████▍    | 123002/225000 [251:09:56<137:58:13,  4.87s/it]
 55%|█████▍    | 123003/225000 [251:10:01<137:35:26,  4.86s/it]
 55%|█████▍    | 123004/225000 [251:10:06<138:07:17,  4.88s/it]
 55%|█████▍    | 123005/225000 [251:10:13<155:23:19,  5.48s/it]
 55%|█████▍    | 123006/225000 [251:10:24<205:26:30,  7.25s/it]
 55%|█████▍    | 123007/225000 [251:10:28<176:02:19,  6.21s/it]
 55%|█████▍    | 123008/225000 [251:10:37<196:11:38,  6.93s/it]
 55%|█████▍    | 123009/225000 [251:10:40<166:19:46,  5.87s/it]
 55%|█████▍    | 123010/225000 [251:10:48<183:56:52,  6.49s/it]
 55%|█████▍    | 123011/225000 [251:10:54<182:04:21,  6.43s/it]
 55%|█████▍    | 123012/225000 [251:11:00<172:16:30,  6.08s/it]
 55%|█████▍    | 123013/225000 [251:11:04<159:22:12,  5.63s/it]
 55%|█████▍    | 123014/225000 [251:11:10<157:19:40,  5.55s/it]
 55%|█████▍    | 123015/225000 [251:11:15<155:08:19,  5.48s/it]
 55%|█████▍    | 123016/225000 [251:11:20<149:22:43,  5.27s/it]
 55%|█████▍    | 123017/225000 [251:11:24<143:12:15,  5.06s/it]
 55%|█████▍    | 123018/225000 [251:11:29<138:19:38,  4.88s/it]
 55%|█████▍    | 123019/225000 [251:11:34<141:15:10,  4.99s/it]
 55%|█████▍    | 123020/225000 [251:11:40<145:56:16,  5.15s/it]
 55%|█████▍    | 123021/225000 [251:11:45<148:31:35,  5.24s/it]
 55%|█████▍    | 123022/225000 [251:11:50<147:33:31,  5.21s/it]
 55%|█████▍    | 123023/225000 [251:11:55<144:59:28,  5.12s/it]
 55%|█████▍    | 123024/225000 [251:12:00<147:14:19,  5.20s/it]
 55%|█████▍    | 123025/225000 [251:12:06<146:58:41,  5.19s/it]
 55%|█████▍    | 123026/225000 [251:12:11<148:56:59,  5.26s/it]
 55%|█████▍    | 123027/225000 [251:12:16<148:18:51,  5.24s/it]
 55%|█████▍    | 123028/225000 [251:12:27<198:20:26,  7.00s/it]
 55%|█████▍    | 123029/225000 [251:12:34<191:28:03,  6.76s/it]
 55%|█████▍    | 123030/225000 [251:12:40<188:04:49,  6.64s/it]
 55%|█████▍    | 123031/225000 [251:12:46<179:51:55,  6.35s/it]
 55%|█████▍    | 123032/225000 [251:12:49<157:44:51,  5.57s/it]
 55%|█████▍    | 123033/225000 [251:12:57<173:06:25,  6.11s/it]
 55%|█████▍    | 123034/225000 [251:13:03<177:33:29,  6.27s/it]
 55%|█████▍    | 123035/225000 [251:13:07<157:56:28,  5.58s/it]
 55%|█████▍    | 123036/225000 [251:13:12<148:51:06,  5.26s/it]
 55%|█████▍    | 123037/225000 [251:13:16<141:50:45,  5.01s/it]
 55%|█████▍    | 123038/225000 [251:13:22<145:53:34,  5.15s/it]
 55%|█████▍    | 123039/225000 [251:13:27<149:30:50,  5.28s/it]
 55%|█████▍    | 123040/225000 [251:13:34<163:45:48,  5.78s/it]
 55%|█████▍    | 123041/225000 [251:13:41<173:16:37,  6.12s/it]
 55%|█████▍    | 123042/225000 [251:13:48<183:36:53,  6.48s/it]
 55%|█████▍    | 123043/225000 [251:13:54<174:33:08,  6.16s/it]
 55%|█████▍    | 123044/225000 [251:13:59<167:18:38,  5.91s/it]
 55%|█████▍    | 123045/225000 [251:14:04<158:58:34,  5.61s/it]
 55%|█████▍    | 123046/225000 [251:14:09<155:45:30,  5.50s/it]
 55%|█████▍    | 123047/225000 [251:14:14<151:11:03,  5.34s/it]
 55%|█████▍    | 123048/225000 [251:14:19<149:51:14,  5.29s/it]
 55%|█████▍    | 123049/225000 [251:14:24<146:38:34,  5.18s/it]
 55%|█████▍    | 123050/225000 [251:14:29<141:35:16,  5.00s/it]
                                                               
{'loss': '0.8263', 'grad_norm': '1.002', 'learning_rate': '2.3e-05', 'epoch': '0.5469', 'num_input_tokens_seen': 418569928, 'train_runtime': '9.045e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123050/225000 [251:14:29<141:35:16,  5.00s/it]
 55%|█████▍    | 123051/225000 [251:14:34<144:03:02,  5.09s/it]
 55%|█████▍    | 123052/225000 [251:14:39<141:33:15,  5.00s/it]
 55%|█████▍    | 123053/225000 [251:14:49<186:33:42,  6.59s/it]
 55%|█████▍    | 123054/225000 [251:14:57<197:08:53,  6.96s/it]
 55%|█████▍    | 123055/225000 [251:15:01<168:15:19,  5.94s/it]
 55%|█████▍    | 123056/225000 [251:15:09<189:15:27,  6.68s/it]
 55%|█████▍    | 123057/225000 [251:15:14<170:06:51,  6.01s/it]
 55%|█████▍    | 123058/225000 [251:15:17<150:42:08,  5.32s/it]
 55%|█████▍    | 123059/225000 [251:15:22<144:32:58,  5.10s/it]
 55%|█████▍    | 123060/225000 [251:15:26<139:04:38,  4.91s/it]
 55%|█████▍    | 123061/225000 [251:15:37<189:37:13,  6.70s/it]
 55%|█████▍    | 123062/225000 [251:15:43<177:51:35,  6.28s/it]
 55%|█████▍    | 123063/225000 [251:15:46<157:01:45,  5.55s/it]
 55%|█████▍    | 123064/225000 [251:15:56<192:38:16,  6.80s/it]
 55%|█████▍    | 123065/225000 [251:15:59<161:06:59,  5.69s/it]
 55%|█████▍    | 123066/225000 [251:16:02<139:01:40,  4.91s/it]
 55%|█████▍    | 123067/225000 [251:16:06<132:38:18,  4.68s/it]
 55%|█████▍    | 123068/225000 [251:16:13<150:27:47,  5.31s/it]
 55%|█████▍    | 123069/225000 [251:16:21<175:21:06,  6.19s/it]
 55%|█████▍    | 123070/225000 [251:16:28<175:43:28,  6.21s/it]
 55%|█████▍    | 123071/225000 [251:16:32<158:34:35,  5.60s/it]
 55%|█████▍    | 123072/225000 [251:16:37<153:02:44,  5.41s/it]
 55%|█████▍    | 123073/225000 [251:16:42<154:25:37,  5.45s/it]
 55%|█████▍    | 123074/225000 [251:16:47<150:39:30,  5.32s/it]
 55%|█████▍    | 123075/225000 [251:16:52<147:48:18,  5.22s/it]
 55%|█████▍    | 123076/225000 [251:16:57<146:10:57,  5.16s/it]
 55%|█████▍    | 123077/225000 [251:17:03<148:05:40,  5.23s/it]
 55%|█████▍    | 123078/225000 [251:17:10<160:48:08,  5.68s/it]
 55%|█████▍    | 123079/225000 [251:17:18<183:32:05,  6.48s/it]
 55%|█████▍    | 123080/225000 [251:17:22<160:52:32,  5.68s/it]
 55%|█████▍    | 123081/225000 [251:17:31<187:43:51,  6.63s/it]
 55%|█████▍    | 123082/225000 [251:17:39<199:27:21,  7.05s/it]
 55%|█████▍    | 123083/225000 [251:17:44<182:29:52,  6.45s/it]
 55%|█████▍    | 123084/225000 [251:17:49<177:15:47,  6.26s/it]
 55%|█████▍    | 123085/225000 [251:17:57<187:28:10,  6.62s/it]
 55%|█████▍    | 123086/225000 [251:18:00<159:43:10,  5.64s/it]
 55%|█████▍    | 123087/225000 [251:18:09<185:12:09,  6.54s/it]
 55%|█████▍    | 123088/225000 [251:18:13<163:50:53,  5.79s/it]
 55%|█████▍    | 123089/225000 [251:18:17<151:22:27,  5.35s/it]
 55%|█████▍    | 123090/225000 [251:18:21<139:57:34,  4.94s/it]
 55%|█████▍    | 123091/225000 [251:18:33<193:31:54,  6.84s/it]
 55%|█████▍    | 123092/225000 [251:18:36<168:08:03,  5.94s/it]
 55%|█████▍    | 123093/225000 [251:18:40<152:13:27,  5.38s/it]
 55%|█████▍    | 123094/225000 [251:18:46<154:12:09,  5.45s/it]
 55%|█████▍    | 123095/225000 [251:18:55<184:52:45,  6.53s/it]
 55%|█████▍    | 123096/225000 [251:19:00<166:42:16,  5.89s/it]
 55%|█████▍    | 123097/225000 [251:19:04<154:10:16,  5.45s/it]
 55%|█████▍    | 123098/225000 [251:19:09<148:25:04,  5.24s/it]
 55%|█████▍    | 123099/225000 [251:19:13<141:16:45,  4.99s/it]
 55%|█████▍    | 123100/225000 [251:19:18<144:13:40,  5.10s/it]
                                                               
{'loss': '0.8433', 'grad_norm': '1.053', 'learning_rate': '2.298e-05', 'epoch': '0.5471', 'num_input_tokens_seen': 418699408, 'train_runtime': '9.048e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123100/225000 [251:19:18<144:13:40,  5.10s/it]
 55%|█████▍    | 123101/225000 [251:19:30<200:49:01,  7.09s/it]
 55%|█████▍    | 123102/225000 [251:19:34<173:27:18,  6.13s/it]
 55%|█████▍    | 123103/225000 [251:19:39<164:34:40,  5.81s/it]
 55%|█████▍    | 123104/225000 [251:19:47<177:47:19,  6.28s/it]
 55%|█████▍    | 123105/225000 [251:19:50<156:01:06,  5.51s/it]
 55%|█████▍    | 123106/225000 [251:19:58<178:46:08,  6.32s/it]
 55%|█████▍    | 123107/225000 [251:20:04<170:41:30,  6.03s/it]
 55%|█████▍    | 123108/225000 [251:20:12<186:33:06,  6.59s/it]
 55%|█████▍    | 123109/225000 [251:20:15<162:31:12,  5.74s/it]
 55%|█████▍    | 123110/225000 [251:20:19<147:41:25,  5.22s/it]
 55%|█████▍    | 123111/225000 [251:20:29<182:49:17,  6.46s/it]
 55%|█████▍    | 123112/225000 [251:20:33<162:55:02,  5.76s/it]
 55%|█████▍    | 123113/225000 [251:20:36<143:10:59,  5.06s/it]
 55%|█████▍    | 123114/225000 [251:20:47<186:46:05,  6.60s/it]
 55%|█████▍    | 123115/225000 [251:20:51<171:58:00,  6.08s/it]
 55%|█████▍    | 123116/225000 [251:20:56<160:52:37,  5.68s/it]
 55%|█████▍    | 123117/225000 [251:21:01<150:57:04,  5.33s/it]
 55%|█████▍    | 123118/225000 [251:21:05<142:21:11,  5.03s/it]
 55%|█████▍    | 123119/225000 [251:21:10<139:00:40,  4.91s/it]
 55%|█████▍    | 123120/225000 [251:21:14<136:16:16,  4.82s/it]
 55%|█████▍    | 123121/225000 [251:21:20<143:10:19,  5.06s/it]
 55%|█████▍    | 123122/225000 [251:21:32<200:08:04,  7.07s/it]
 55%|█████▍    | 123123/225000 [251:21:36<173:35:12,  6.13s/it]
 55%|█████▍    | 123124/225000 [251:21:39<153:15:56,  5.42s/it]
 55%|█████▍    | 123125/225000 [251:21:44<145:11:12,  5.13s/it]
 55%|█████▍    | 123126/225000 [251:21:51<162:39:00,  5.75s/it]
 55%|█████▍    | 123127/225000 [251:22:00<193:27:23,  6.84s/it]
 55%|█████▍    | 123128/225000 [251:22:05<172:56:53,  6.11s/it]
 55%|█████▍    | 123129/225000 [251:22:10<161:42:14,  5.71s/it]
 55%|█████▍    | 123130/225000 [251:22:14<151:36:55,  5.36s/it]
 55%|█████▍    | 123131/225000 [251:22:18<136:36:41,  4.83s/it]
 55%|█████▍    | 123132/225000 [251:22:29<193:53:29,  6.85s/it]
 55%|█████▍    | 123133/225000 [251:22:33<170:29:01,  6.02s/it]
 55%|█████▍    | 123134/225000 [251:22:37<151:55:24,  5.37s/it]
 55%|█████▍    | 123135/225000 [251:22:42<143:51:33,  5.08s/it]
 55%|█████▍    | 123136/225000 [251:22:49<160:13:07,  5.66s/it]
 55%|█████▍    | 123137/225000 [251:22:56<177:35:42,  6.28s/it]
 55%|█████▍    | 123138/225000 [251:23:00<159:01:20,  5.62s/it]
 55%|█████▍    | 123139/225000 [251:23:05<152:28:54,  5.39s/it]
 55%|█████▍    | 123140/225000 [251:23:10<148:19:20,  5.24s/it]
 55%|█████▍    | 123141/225000 [251:23:15<144:29:11,  5.11s/it]
 55%|█████▍    | 123142/225000 [251:23:22<159:46:57,  5.65s/it]
 55%|█████▍    | 123143/225000 [251:23:32<194:19:15,  6.87s/it]
 55%|█████▍    | 123144/225000 [251:23:36<170:30:20,  6.03s/it]
 55%|█████▍    | 123145/225000 [251:23:40<154:33:37,  5.46s/it]
 55%|█████▍    | 123146/225000 [251:23:44<145:47:27,  5.15s/it]
 55%|█████▍    | 123147/225000 [251:23:50<149:08:03,  5.27s/it]
 55%|█████▍    | 123148/225000 [251:23:59<181:48:51,  6.43s/it]
 55%|█████▍    | 123149/225000 [251:24:03<160:52:27,  5.69s/it]
 55%|█████▍    | 123150/225000 [251:24:07<149:23:27,  5.28s/it]
                                                               
{'loss': '0.8616', 'grad_norm': '0.9944', 'learning_rate': '2.296e-05', 'epoch': '0.5473', 'num_input_tokens_seen': 418828392, 'train_runtime': '9.05e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123150/225000 [251:24:07<149:23:27,  5.28s/it]
 55%|█████▍    | 123151/225000 [251:24:11<140:18:04,  4.96s/it]
 55%|█████▍    | 123152/225000 [251:24:16<134:42:48,  4.76s/it]
 55%|█████▍    | 123153/225000 [251:24:27<193:52:34,  6.85s/it]
 55%|█████▍    | 123154/225000 [251:24:31<166:12:07,  5.87s/it]
 55%|█████▍    | 123155/225000 [251:24:37<168:36:10,  5.96s/it]
 55%|█████▍    | 123156/225000 [251:24:43<166:59:34,  5.90s/it]
 55%|█████▍    | 123157/225000 [251:24:47<152:15:11,  5.38s/it]
 55%|█████▍    | 123158/225000 [251:24:51<142:02:51,  5.02s/it]
 55%|█████▍    | 123159/225000 [251:25:00<172:34:37,  6.10s/it]
 55%|█████▍    | 123160/225000 [251:25:07<180:40:33,  6.39s/it]
 55%|█████▍    | 123161/225000 [251:25:10<156:16:16,  5.52s/it]
 55%|█████▍    | 123162/225000 [251:25:19<180:52:40,  6.39s/it]
 55%|█████▍    | 123163/225000 [251:25:22<154:42:37,  5.47s/it]
 55%|█████▍    | 123164/225000 [251:25:27<147:43:45,  5.22s/it]
 55%|█████▍    | 123165/225000 [251:25:31<140:45:03,  4.98s/it]
 55%|█████▍    | 123166/225000 [251:25:35<134:13:10,  4.74s/it]
 55%|█████▍    | 123167/225000 [251:25:48<200:21:51,  7.08s/it]
 55%|█████▍    | 123168/225000 [251:25:52<170:17:01,  6.02s/it]
 55%|█████▍    | 123169/225000 [251:25:56<154:29:37,  5.46s/it]
 55%|█████▍    | 123170/225000 [251:26:04<176:51:52,  6.25s/it]
 55%|█████▍    | 123171/225000 [251:26:08<162:19:07,  5.74s/it]
 55%|█████▍    | 123172/225000 [251:26:13<151:44:24,  5.36s/it]
 55%|█████▍    | 123173/225000 [251:26:17<142:42:37,  5.05s/it]
 55%|█████▍    | 123174/225000 [251:26:22<137:21:41,  4.86s/it]
 55%|█████▍    | 123175/225000 [251:26:27<141:46:15,  5.01s/it]
 55%|█████▍    | 123176/225000 [251:26:40<208:54:16,  7.39s/it]
 55%|█████▍    | 123177/225000 [251:26:44<177:34:00,  6.28s/it]
 55%|█████▍    | 123178/225000 [251:26:50<178:50:25,  6.32s/it]
 55%|█████▍    | 123179/225000 [251:26:53<153:30:16,  5.43s/it]
 55%|█████▍    | 123180/225000 [251:27:03<193:05:21,  6.83s/it]
 55%|█████▍    | 123181/225000 [251:27:07<166:11:50,  5.88s/it]
 55%|█████▍    | 123182/225000 [251:27:12<161:23:46,  5.71s/it]
 55%|█████▍    | 123183/225000 [251:27:22<191:18:37,  6.76s/it]
 55%|█████▍    | 123184/225000 [251:27:25<161:19:58,  5.70s/it]
 55%|█████▍    | 123185/225000 [251:27:31<167:08:59,  5.91s/it]
 55%|█████▍    | 123186/225000 [251:27:36<156:59:38,  5.55s/it]
 55%|█████▍    | 123187/225000 [251:27:40<147:14:37,  5.21s/it]
 55%|█████▍    | 123188/225000 [251:27:45<145:01:02,  5.13s/it]
 55%|█████▍    | 123189/225000 [251:27:50<143:42:42,  5.08s/it]
 55%|█████▍    | 123190/225000 [251:27:55<136:51:45,  4.84s/it]
 55%|█████▍    | 123191/225000 [251:27:59<133:55:36,  4.74s/it]
 55%|█████▍    | 123192/225000 [251:28:04<137:45:51,  4.87s/it]
 55%|█████▍    | 123193/225000 [251:28:13<172:55:03,  6.11s/it]
 55%|█████▍    | 123194/225000 [251:28:22<197:50:12,  7.00s/it]
 55%|█████▍    | 123195/225000 [251:28:27<178:23:08,  6.31s/it]
 55%|█████▍    | 123196/225000 [251:28:34<181:02:45,  6.40s/it]
 55%|█████▍    | 123197/225000 [251:28:40<182:15:11,  6.44s/it]
 55%|█████▍    | 123198/225000 [251:28:45<170:29:09,  6.03s/it]
 55%|█████▍    | 123199/225000 [251:28:52<179:24:40,  6.34s/it]
 55%|█████▍    | 123200/225000 [251:28:57<162:31:55,  5.75s/it]
                                                               
{'loss': '0.8384', 'grad_norm': '1.043', 'learning_rate': '2.294e-05', 'epoch': '0.5476', 'num_input_tokens_seen': 418956552, 'train_runtime': '9.053e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123200/225000 [251:28:57<162:31:55,  5.75s/it]
 55%|█████▍    | 123201/225000 [251:29:01<148:02:01,  5.24s/it]
 55%|█████▍    | 123202/225000 [251:29:05<138:25:04,  4.90s/it]
 55%|█████▍    | 123203/225000 [251:29:11<145:56:50,  5.16s/it]
 55%|█████▍    | 123204/225000 [251:29:20<184:34:09,  6.53s/it]
 55%|█████▍    | 123205/225000 [251:29:27<184:08:06,  6.51s/it]
 55%|█████▍    | 123206/225000 [251:29:31<168:00:55,  5.94s/it]
 55%|█████▍    | 123207/225000 [251:29:36<155:48:27,  5.51s/it]
 55%|█████▍    | 123208/225000 [251:29:44<178:17:12,  6.31s/it]
 55%|█████▍    | 123209/225000 [251:29:49<169:59:00,  6.01s/it]
 55%|█████▍    | 123210/225000 [251:29:54<160:06:14,  5.66s/it]
 55%|█████▍    | 123211/225000 [251:29:59<154:39:37,  5.47s/it]
 55%|█████▍    | 123212/225000 [251:30:05<154:27:17,  5.46s/it]
 55%|█████▍    | 123213/225000 [251:30:10<152:29:04,  5.39s/it]
 55%|█████▍    | 123214/225000 [251:30:15<151:30:16,  5.36s/it]
 55%|█████▍    | 123215/225000 [251:30:20<150:33:55,  5.33s/it]
 55%|█████▍    | 123216/225000 [251:30:26<151:28:42,  5.36s/it]
 55%|█████▍    | 123217/225000 [251:30:34<177:44:11,  6.29s/it]
 55%|█████▍    | 123218/225000 [251:30:41<183:06:44,  6.48s/it]
 55%|█████▍    | 123219/225000 [251:30:46<171:48:25,  6.08s/it]
 55%|█████▍    | 123220/225000 [251:30:51<159:49:25,  5.65s/it]
 55%|█████▍    | 123221/225000 [251:30:56<154:47:57,  5.48s/it]
 55%|█████▍    | 123222/225000 [251:31:05<183:43:54,  6.50s/it]
 55%|█████▍    | 123223/225000 [251:31:09<166:14:02,  5.88s/it]
 55%|█████▍    | 123224/225000 [251:31:14<156:13:53,  5.53s/it]
 55%|█████▍    | 123225/225000 [251:31:19<148:31:18,  5.25s/it]
 55%|█████▍    | 123226/225000 [251:31:23<143:32:00,  5.08s/it]
 55%|█████▍    | 123227/225000 [251:31:36<205:42:33,  7.28s/it]
 55%|█████▍    | 123228/225000 [251:31:40<175:27:22,  6.21s/it]
 55%|█████▍    | 123229/225000 [251:31:44<159:52:10,  5.66s/it]
 55%|█████▍    | 123230/225000 [251:31:48<150:36:17,  5.33s/it]
 55%|█████▍    | 123231/225000 [251:31:54<148:55:21,  5.27s/it]
 55%|█████▍    | 123232/225000 [251:31:58<143:31:21,  5.08s/it]
 55%|█████▍    | 123233/225000 [251:32:03<141:01:46,  4.99s/it]
 55%|█████▍    | 123234/225000 [251:32:08<140:18:24,  4.96s/it]
 55%|█████▍    | 123235/225000 [251:32:13<137:40:29,  4.87s/it]
 55%|█████▍    | 123236/225000 [251:32:18<139:30:01,  4.93s/it]
 55%|█████▍    | 123237/225000 [251:32:28<182:02:13,  6.44s/it]
 55%|█████▍    | 123238/225000 [251:32:36<197:48:30,  7.00s/it]
 55%|█████▍    | 123239/225000 [251:32:40<170:41:09,  6.04s/it]
 55%|█████▍    | 123240/225000 [251:32:45<166:38:49,  5.90s/it]
 55%|█████▍    | 123241/225000 [251:32:53<183:44:27,  6.50s/it]
 55%|█████▍    | 123242/225000 [251:32:56<156:20:06,  5.53s/it]
 55%|█████▍    | 123243/225000 [251:33:02<156:00:33,  5.52s/it]
 55%|█████▍    | 123244/225000 [251:33:07<150:08:01,  5.31s/it]
 55%|█████▍    | 123245/225000 [251:33:12<151:47:31,  5.37s/it]
 55%|█████▍    | 123246/225000 [251:33:17<149:56:43,  5.30s/it]
 55%|█████▍    | 123247/225000 [251:33:23<149:51:22,  5.30s/it]
 55%|█████▍    | 123248/225000 [251:33:28<150:34:23,  5.33s/it]
 55%|█████▍    | 123249/225000 [251:33:36<172:08:38,  6.09s/it]
 55%|█████▍    | 123250/225000 [251:33:41<160:35:32,  5.68s/it]
                                                               
{'loss': '0.8464', 'grad_norm': '1.14', 'learning_rate': '2.292e-05', 'epoch': '0.5478', 'num_input_tokens_seen': 419086768, 'train_runtime': '9.056e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123250/225000 [251:33:41<160:35:32,  5.68s/it]
 55%|█████▍    | 123251/225000 [251:33:46<154:43:01,  5.47s/it]
 55%|█████▍    | 123252/225000 [251:33:56<199:00:35,  7.04s/it]
 55%|█████▍    | 123253/225000 [251:34:02<186:14:02,  6.59s/it]
 55%|█████▍    | 123254/225000 [251:34:07<176:37:01,  6.25s/it]
 55%|█████▍    | 123255/225000 [251:34:14<177:22:13,  6.28s/it]
 55%|█████▍    | 123256/225000 [251:34:17<155:39:48,  5.51s/it]
 55%|█████▍    | 123257/225000 [251:34:26<180:24:38,  6.38s/it]
 55%|█████▍    | 123258/225000 [251:34:31<172:26:54,  6.10s/it]
 55%|█████▍    | 123259/225000 [251:34:36<157:44:55,  5.58s/it]
 55%|█████▍    | 123260/225000 [251:34:41<151:59:05,  5.38s/it]
 55%|█████▍    | 123261/225000 [251:34:45<146:08:00,  5.17s/it]
 55%|█████▍    | 123262/225000 [251:34:50<143:21:55,  5.07s/it]
 55%|█████▍    | 123263/225000 [251:34:54<136:29:29,  4.83s/it]
 55%|█████▍    | 123264/225000 [251:34:59<136:14:59,  4.82s/it]
 55%|█████▍    | 123265/225000 [251:35:06<153:44:47,  5.44s/it]
 55%|█████▍    | 123266/225000 [251:35:18<207:38:02,  7.35s/it]
 55%|█████▍    | 123267/225000 [251:35:22<178:36:06,  6.32s/it]
 55%|█████▍    | 123268/225000 [251:35:29<186:17:22,  6.59s/it]
 55%|█████▍    | 123269/225000 [251:35:33<159:58:34,  5.66s/it]
 55%|█████▍    | 123270/225000 [251:35:38<155:52:53,  5.52s/it]
 55%|█████▍    | 123271/225000 [251:35:43<152:54:32,  5.41s/it]
 55%|█████▍    | 123272/225000 [251:35:47<145:21:02,  5.14s/it]
 55%|█████▍    | 123273/225000 [251:35:52<145:13:58,  5.14s/it]
 55%|█████▍    | 123274/225000 [251:35:58<144:05:38,  5.10s/it]
 55%|█████▍    | 123275/225000 [251:36:03<144:48:40,  5.12s/it]
 55%|█████▍    | 123276/225000 [251:36:08<144:37:23,  5.12s/it]
 55%|█████▍    | 123277/225000 [251:36:13<141:26:09,  5.01s/it]
 55%|█████▍    | 123278/225000 [251:36:20<164:51:24,  5.83s/it]
 55%|█████▍    | 123279/225000 [251:36:33<221:16:03,  7.83s/it]
 55%|█████▍    | 123280/225000 [251:36:37<186:54:54,  6.62s/it]
 55%|█████▍    | 123281/225000 [251:36:41<166:49:24,  5.90s/it]
 55%|█████▍    | 123282/225000 [251:36:45<155:57:10,  5.52s/it]
 55%|█████▍    | 123283/225000 [251:36:50<147:29:05,  5.22s/it]
 55%|█████▍    | 123284/225000 [251:37:00<185:35:13,  6.57s/it]
 55%|█████▍    | 123285/225000 [251:37:07<194:05:26,  6.87s/it]
 55%|█████▍    | 123286/225000 [251:37:14<191:23:14,  6.77s/it]
 55%|█████▍    | 123287/225000 [251:37:17<161:57:24,  5.73s/it]
 55%|█████▍    | 123288/225000 [251:37:26<192:09:05,  6.80s/it]
 55%|█████▍    | 123289/225000 [251:37:30<162:15:03,  5.74s/it]
 55%|█████▍    | 123290/225000 [251:37:37<172:36:28,  6.11s/it]
 55%|█████▍    | 123291/225000 [251:37:41<154:27:48,  5.47s/it]
 55%|█████▍    | 123292/225000 [251:37:44<139:47:48,  4.95s/it]
 55%|█████▍    | 123293/225000 [251:37:48<129:08:38,  4.57s/it]
 55%|█████▍    | 123294/225000 [251:37:58<172:36:43,  6.11s/it]
 55%|█████▍    | 123295/225000 [251:38:03<167:20:30,  5.92s/it]
 55%|█████▍    | 123296/225000 [251:38:09<165:56:55,  5.87s/it]
 55%|█████▍    | 123297/225000 [251:38:12<144:17:20,  5.11s/it]
 55%|█████▍    | 123298/225000 [251:38:16<135:09:48,  4.78s/it]
 55%|█████▍    | 123299/225000 [251:38:26<180:11:55,  6.38s/it]
 55%|█████▍    | 123300/225000 [251:38:31<162:36:17,  5.76s/it]
                                                               
{'loss': '0.8266', 'grad_norm': '1.349', 'learning_rate': '2.29e-05', 'epoch': '0.548', 'num_input_tokens_seen': 419215120, 'train_runtime': '9.059e+05', 'train_tokens_per_second': '462.8'}

 55%|█████▍    | 123300/225000 [251:38:31<162:36:17,  5.76s/it]
 55%|█████▍    | 123301/225000 [251:38:36<155:19:31,  5.50s/it]
 55%|█████▍    | 123302/225000 [251:38:43<172:57:53,  6.12s/it]
 55%|█████▍    | 123303/225000 [251:38:47<152:39:33,  5.40s/it]
 55%|█████▍    | 123304/225000 [251:38:53<155:10:00,  5.49s/it]
 55%|█████▍    | 123305/225000 [251:39:00<171:20:18,  6.07s/it]
 55%|█████▍    | 123306/225000 [251:39:04<151:04:42,  5.35s/it]
 55%|█████▍    | 123307/225000 [251:39:08<140:03:35,  4.96s/it]
 55%|█████▍    | 123308/225000 [251:39:17<177:51:30,  6.30s/it]
 55%|█████▍    | 123309/225000 [251:39:24<185:41:15,  6.57s/it]
 55%|█████▍    | 123310/225000 [251:39:29<164:54:58,  5.84s/it]
 55%|█████▍    | 123311/225000 [251:39:33<153:28:24,  5.43s/it]
 55%|█████▍    | 123312/225000 [251:39:38<153:25:04,  5.43s/it]
 55%|█████▍    | 123313/225000 [251:39:44<154:20:32,  5.46s/it]
 55%|█████▍    | 123314/225000 [251:39:49<151:26:12,  5.36s/it]
 55%|█████▍    | 123315/225000 [251:39:54<148:01:58,  5.24s/it]
 55%|█████▍    | 123316/225000 [251:39:59<149:33:48,  5.30s/it]
 55%|█████▍    | 123317/225000 [251:40:04<141:07:10,  5.00s/it]
 55%|█████▍    | 123318/225000 [251:40:09<140:45:53,  4.98s/it]
 55%|█████▍    | 123319/225000 [251:40:16<158:30:32,  5.61s/it]
 55%|█████▍    | 123320/225000 [251:40:26<199:58:24,  7.08s/it]
 55%|█████▍    | 123321/225000 [251:40:30<174:46:31,  6.19s/it]
 55%|█████▍    | 123322/225000 [251:40:38<182:54:03,  6.48s/it]
 55%|█████▍    | 123323/225000 [251:40:43<171:53:39,  6.09s/it]
 55%|█████▍    | 123324/225000 [251:40:48<165:02:44,  5.84s/it]
 55%|█████▍    | 123325/225000 [251:40:53<158:57:33,  5.63s/it]
 55%|█████▍    | 123326/225000 [251:40:59<157:53:23,  5.59s/it]
 55%|█████▍    | 123327/225000 [251:41:04<154:34:27,  5.47s/it]
 55%|█████▍    | 123328/225000 [251:41:08<146:03:08,  5.17s/it]
 55%|█████▍    | 123329/225000 [251:41:14<147:48:06,  5.23s/it]
 55%|█████▍    | 123330/225000 [251:41:19<145:29:36,  5.15s/it]
 55%|█████▍    | 123331/225000 [251:41:24<143:27:55,  5.08s/it]
 55%|█████▍    | 123332/225000 [251:41:32<171:10:35,  6.06s/it]
 55%|█████▍    | 123333/225000 [251:41:42<202:09:19,  7.16s/it]
 55%|█████▍    | 123334/225000 [251:41:46<174:51:15,  6.19s/it]
 55%|█████▍    | 123335/225000 [251:41:50<158:40:43,  5.62s/it]
 55%|█████▍    | 123336/225000 [251:41:55<154:04:07,  5.46s/it]
 55%|█████▍    | 123337/225000 [251:42:00<149:21:10,  5.29s/it]
 55%|█████▍    | 123338/225000 [251:42:05<146:50:52,  5.20s/it]
 55%|█████▍    | 123339/225000 [251:42:10<144:24:23,  5.11s/it]
 55%|█████▍    | 123340/225000 [251:42:22<200:54:20,  7.11s/it]
 55%|█████▍    | 123341/225000 [251:42:28<192:35:43,  6.82s/it]
 55%|█████▍    | 123342/225000 [251:42:34<189:27:26,  6.71s/it]
 55%|█████▍    | 123343/225000 [251:42:38<164:12:55,  5.82s/it]
 55%|█████▍    | 123344/225000 [251:42:47<189:19:17,  6.70s/it]
 55%|█████▍    | 123345/225000 [251:42:50<162:44:33,  5.76s/it]
 55%|█████▍    | 123346/225000 [251:42:58<180:34:59,  6.40s/it]
 55%|█████▍    | 123347/225000 [251:43:01<151:00:21,  5.35s/it]
 55%|█████▍    | 123348/225000 [251:43:09<176:44:23,  6.26s/it]
 55%|█████▍    | 123349/225000 [251:43:12<150:07:13,  5.32s/it]
 55%|█████▍    | 123350/225000 [251:43:20<166:21:06,  5.89s/it]
                                                               
{'loss': '0.8315', 'grad_norm': '1.109', 'learning_rate': '2.289e-05', 'epoch': '0.5482', 'num_input_tokens_seen': 419343640, 'train_runtime': '9.062e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123350/225000 [251:43:20<166:21:06,  5.89s/it]
 55%|█████▍    | 123351/225000 [251:43:23<145:41:36,  5.16s/it]
 55%|█████▍    | 123352/225000 [251:43:27<133:51:07,  4.74s/it]
 55%|█████▍    | 123353/225000 [251:43:36<175:00:05,  6.20s/it]
 55%|█████▍    | 123354/225000 [251:43:41<156:55:25,  5.56s/it]
 55%|█████▍    | 123355/225000 [251:43:45<144:53:13,  5.13s/it]
 55%|█████▍    | 123356/225000 [251:43:49<139:47:09,  4.95s/it]
 55%|█████▍    | 123357/225000 [251:43:54<140:12:15,  4.97s/it]
 55%|█████▍    | 123358/225000 [251:44:06<199:14:47,  7.06s/it]
 55%|█████▍    | 123359/225000 [251:44:11<178:13:42,  6.31s/it]
 55%|█████▍    | 123360/225000 [251:44:16<170:29:17,  6.04s/it]
 55%|█████▍    | 123361/225000 [251:44:21<162:53:08,  5.77s/it]
 55%|█████▍    | 123362/225000 [251:44:26<157:26:13,  5.58s/it]
 55%|█████▍    | 123363/225000 [251:44:31<150:30:29,  5.33s/it]
 55%|█████▍    | 123364/225000 [251:44:37<151:06:05,  5.35s/it]
 55%|█████▍    | 123365/225000 [251:44:42<153:34:21,  5.44s/it]
 55%|█████▍    | 123366/225000 [251:44:47<149:01:08,  5.28s/it]
 55%|█████▍    | 123367/225000 [251:44:52<147:11:07,  5.21s/it]
 55%|█████▍    | 123368/225000 [251:44:57<147:56:31,  5.24s/it]
 55%|█████▍    | 123369/225000 [251:45:03<150:01:48,  5.31s/it]
 55%|█████▍    | 123370/225000 [251:45:08<151:35:21,  5.37s/it]
 55%|█████▍    | 123371/225000 [251:45:14<153:00:33,  5.42s/it]
 55%|█████▍    | 123372/225000 [251:45:26<207:38:21,  7.36s/it]
 55%|█████▍    | 123373/225000 [251:45:32<197:40:36,  7.00s/it]
 55%|█████▍    | 123374/225000 [251:45:39<193:45:54,  6.86s/it]
 55%|█████▍    | 123375/225000 [251:45:44<181:30:51,  6.43s/it]
 55%|█████▍    | 123376/225000 [251:45:49<170:12:40,  6.03s/it]
 55%|█████▍    | 123377/225000 [251:45:54<164:29:13,  5.83s/it]
 55%|█████▍    | 123378/225000 [251:45:59<157:39:23,  5.59s/it]
 55%|█████▍    | 123379/225000 [251:46:05<153:28:32,  5.44s/it]
 55%|█████▍    | 123380/225000 [251:46:09<143:54:12,  5.10s/it]
 55%|█████▍    | 123381/225000 [251:46:14<144:49:18,  5.13s/it]
 55%|█████▍    | 123382/225000 [251:46:18<138:18:21,  4.90s/it]
 55%|█████▍    | 123383/225000 [251:46:24<141:48:20,  5.02s/it]
 55%|█████▍    | 123384/225000 [251:46:32<169:35:46,  6.01s/it]
 55%|█████▍    | 123385/225000 [251:46:43<207:58:06,  7.37s/it]
 55%|█████▍    | 123386/225000 [251:46:47<184:27:53,  6.54s/it]
 55%|█████▍    | 123387/225000 [251:46:52<166:03:40,  5.88s/it]
 55%|█████▍    | 123388/225000 [251:46:56<153:41:08,  5.44s/it]
 55%|█████▍    | 123389/225000 [251:47:01<149:08:15,  5.28s/it]
 55%|█████▍    | 123390/225000 [251:47:06<143:51:51,  5.10s/it]
 55%|█████▍    | 123391/225000 [251:47:13<163:06:44,  5.78s/it]
 55%|█████▍    | 123392/225000 [251:47:25<216:02:21,  7.65s/it]
 55%|█████▍    | 123393/225000 [251:47:28<180:06:59,  6.38s/it]
 55%|█████▍    | 123394/225000 [251:47:32<157:59:04,  5.60s/it]
 55%|█████▍    | 123395/225000 [251:47:41<187:15:39,  6.63s/it]
 55%|█████▍    | 123396/225000 [251:47:46<171:54:22,  6.09s/it]
 55%|█████▍    | 123397/225000 [251:47:51<163:03:51,  5.78s/it]
 55%|█████▍    | 123398/225000 [251:47:57<160:22:33,  5.68s/it]
 55%|█████▍    | 123399/225000 [251:48:02<158:50:10,  5.63s/it]
 55%|█████▍    | 123400/225000 [251:48:07<155:40:34,  5.52s/it]
                                                               
{'loss': '0.8429', 'grad_norm': '1.052', 'learning_rate': '2.287e-05', 'epoch': '0.5484', 'num_input_tokens_seen': 419474008, 'train_runtime': '9.065e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123400/225000 [251:48:07<155:40:34,  5.52s/it]
 55%|█████▍    | 123401/225000 [251:48:13<157:59:27,  5.60s/it]
 55%|█████▍    | 123402/225000 [251:48:19<157:03:14,  5.57s/it]
 55%|█████▍    | 123403/225000 [251:48:24<156:18:13,  5.54s/it]
 55%|█████▍    | 123404/225000 [251:48:29<151:04:03,  5.35s/it]
 55%|█████▍    | 123405/225000 [251:48:34<149:45:32,  5.31s/it]
 55%|█████▍    | 123406/225000 [251:48:39<145:55:09,  5.17s/it]
 55%|█████▍    | 123407/225000 [251:48:44<144:28:42,  5.12s/it]
 55%|█████▍    | 123408/225000 [251:48:56<204:01:17,  7.23s/it]
 55%|█████▍    | 123409/225000 [251:49:02<194:59:45,  6.91s/it]
 55%|█████▍    | 123410/225000 [251:49:10<205:18:21,  7.28s/it]
 55%|█████▍    | 123411/225000 [251:49:14<174:01:22,  6.17s/it]
 55%|█████▍    | 123412/225000 [251:49:18<158:54:11,  5.63s/it]
 55%|█████▍    | 123413/225000 [251:49:27<182:47:52,  6.48s/it]
 55%|█████▍    | 123414/225000 [251:49:31<163:22:14,  5.79s/it]
 55%|█████▍    | 123415/225000 [251:49:35<151:28:29,  5.37s/it]
 55%|█████▍    | 123416/225000 [251:49:46<193:07:56,  6.84s/it]
 55%|█████▍    | 123417/225000 [251:49:49<162:58:13,  5.78s/it]
 55%|█████▍    | 123418/225000 [251:49:53<145:21:01,  5.15s/it]
 55%|█████▍    | 123419/225000 [251:49:57<137:25:54,  4.87s/it]
 55%|█████▍    | 123420/225000 [251:50:07<182:50:26,  6.48s/it]
 55%|█████▍    | 123421/225000 [251:50:14<181:52:26,  6.45s/it]
 55%|█████▍    | 123422/225000 [251:50:20<180:58:53,  6.41s/it]
 55%|█████▍    | 123423/225000 [251:50:26<178:12:19,  6.32s/it]
 55%|█████▍    | 123424/225000 [251:50:32<172:58:02,  6.13s/it]
 55%|█████▍    | 123425/225000 [251:50:38<171:05:34,  6.06s/it]
 55%|█████▍    | 123426/225000 [251:50:43<164:18:11,  5.82s/it]
 55%|█████▍    | 123427/225000 [251:50:48<156:16:11,  5.54s/it]
 55%|█████▍    | 123428/225000 [251:50:52<146:39:28,  5.20s/it]
 55%|█████▍    | 123429/225000 [251:50:57<145:06:09,  5.14s/it]
 55%|█████▍    | 123430/225000 [251:51:02<143:46:41,  5.10s/it]
 55%|█████▍    | 123431/225000 [251:51:07<142:03:50,  5.04s/it]
 55%|█████▍    | 123432/225000 [251:51:12<141:11:38,  5.00s/it]
 55%|█████▍    | 123433/225000 [251:51:24<200:18:21,  7.10s/it]
 55%|█████▍    | 123434/225000 [251:51:28<177:07:18,  6.28s/it]
 55%|█████▍    | 123435/225000 [251:51:35<181:26:35,  6.43s/it]
 55%|█████▍    | 123436/225000 [251:51:39<158:23:47,  5.61s/it]
 55%|█████▍    | 123437/225000 [251:51:46<172:27:04,  6.11s/it]
 55%|█████▍    | 123438/225000 [251:51:52<168:07:13,  5.96s/it]
 55%|█████▍    | 123439/225000 [251:51:57<160:42:49,  5.70s/it]
 55%|█████▍    | 123440/225000 [251:52:02<157:20:18,  5.58s/it]
 55%|█████▍    | 123441/225000 [251:52:08<158:27:29,  5.62s/it]
 55%|█████▍    | 123442/225000 [251:52:13<158:15:25,  5.61s/it]
 55%|█████▍    | 123443/225000 [251:52:19<155:30:25,  5.51s/it]
 55%|█████▍    | 123444/225000 [251:52:24<152:39:19,  5.41s/it]
 55%|█████▍    | 123445/225000 [251:52:29<149:15:59,  5.29s/it]
 55%|█████▍    | 123446/225000 [251:52:34<146:23:57,  5.19s/it]
 55%|█████▍    | 123447/225000 [251:52:39<144:03:21,  5.11s/it]
 55%|█████▍    | 123448/225000 [251:52:44<142:22:37,  5.05s/it]
 55%|█████▍    | 123449/225000 [251:52:53<179:02:16,  6.35s/it]
 55%|█████▍    | 123450/225000 [251:53:01<193:56:54,  6.88s/it]
                                                               
{'loss': '0.8325', 'grad_norm': '1.087', 'learning_rate': '2.285e-05', 'epoch': '0.5487', 'num_input_tokens_seen': 419604224, 'train_runtime': '9.068e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123450/225000 [251:53:01<193:56:54,  6.88s/it]
 55%|█████▍    | 123451/225000 [251:53:04<163:30:33,  5.80s/it]
 55%|█████▍    | 123452/225000 [251:53:10<158:21:10,  5.61s/it]
 55%|█████▍    | 123453/225000 [251:53:18<185:30:40,  6.58s/it]
 55%|█████▍    | 123454/225000 [251:53:23<166:49:28,  5.91s/it]
 55%|█████▍    | 123455/225000 [251:53:27<155:28:27,  5.51s/it]
 55%|█████▍    | 123456/225000 [251:53:32<147:10:32,  5.22s/it]
 55%|█████▍    | 123457/225000 [251:53:36<139:30:36,  4.95s/it]
 55%|█████▍    | 123458/225000 [251:53:47<192:33:40,  6.83s/it]
 55%|█████▍    | 123459/225000 [251:53:50<160:00:32,  5.67s/it]
 55%|█████▍    | 123460/225000 [251:53:57<166:23:03,  5.90s/it]
 55%|█████▍    | 123461/225000 [251:54:04<178:46:11,  6.34s/it]
 55%|█████▍    | 123462/225000 [251:54:07<151:22:56,  5.37s/it]
 55%|█████▍    | 123463/225000 [251:54:15<172:59:02,  6.13s/it]
 55%|█████▍    | 123464/225000 [251:54:20<161:06:39,  5.71s/it]
 55%|█████▍    | 123465/225000 [251:54:27<173:49:11,  6.16s/it]
 55%|█████▍    | 123466/225000 [251:54:31<152:50:02,  5.42s/it]
 55%|█████▍    | 123467/225000 [251:54:39<176:00:45,  6.24s/it]
 55%|█████▍    | 123468/225000 [251:54:42<151:38:26,  5.38s/it]
 55%|█████▍    | 123469/225000 [251:54:47<144:57:02,  5.14s/it]
 55%|█████▍    | 123470/225000 [251:54:57<189:48:43,  6.73s/it]
 55%|█████▍    | 123471/225000 [251:55:04<186:44:22,  6.62s/it]
 55%|█████▍    | 123472/225000 [251:55:10<182:53:47,  6.49s/it]
 55%|█████▍    | 123473/225000 [251:55:17<186:12:10,  6.60s/it]
 55%|█████▍    | 123474/225000 [251:55:20<159:00:32,  5.64s/it]
 55%|█████▍    | 123475/225000 [251:55:26<158:36:25,  5.62s/it]
 55%|█████▍    | 123476/225000 [251:55:31<155:24:05,  5.51s/it]
 55%|█████▍    | 123477/225000 [251:55:39<181:10:02,  6.42s/it]
 55%|█████▍    | 123478/225000 [251:55:43<159:21:17,  5.65s/it]
 55%|█████▍    | 123479/225000 [251:55:47<144:44:57,  5.13s/it]
 55%|█████▍    | 123480/225000 [251:55:57<179:29:56,  6.37s/it]
 55%|█████▍    | 123481/225000 [251:56:00<158:00:22,  5.60s/it]
 55%|█████▍    | 123482/225000 [251:56:04<143:01:12,  5.07s/it]
 55%|█████▍    | 123483/225000 [251:56:08<136:17:25,  4.83s/it]
 55%|█████▍    | 123484/225000 [251:56:13<135:29:47,  4.81s/it]
 55%|█████▍    | 123485/225000 [251:56:22<172:01:49,  6.10s/it]
 55%|█████▍    | 123486/225000 [251:56:31<190:02:31,  6.74s/it]
 55%|█████▍    | 123487/225000 [251:56:34<163:51:56,  5.81s/it]
 55%|█████▍    | 123488/225000 [251:56:41<172:45:33,  6.13s/it]
 55%|█████▍    | 123489/225000 [251:56:47<172:04:50,  6.10s/it]
 55%|█████▍    | 123490/225000 [251:56:51<153:51:12,  5.46s/it]
 55%|█████▍    | 123491/225000 [251:56:59<174:17:54,  6.18s/it]
 55%|█████▍    | 123492/225000 [251:57:04<162:39:40,  5.77s/it]
 55%|█████▍    | 123493/225000 [251:57:11<172:01:17,  6.10s/it]
 55%|█████▍    | 123494/225000 [251:57:15<154:32:28,  5.48s/it]
 55%|█████▍    | 123495/225000 [251:57:20<155:40:34,  5.52s/it]
 55%|█████▍    | 123496/225000 [251:57:26<155:56:16,  5.53s/it]
 55%|█████▍    | 123497/225000 [251:57:33<171:07:00,  6.07s/it]
 55%|█████▍    | 123498/225000 [251:57:40<177:55:28,  6.31s/it]
 55%|█████▍    | 123499/225000 [251:57:44<161:50:40,  5.74s/it]
 55%|█████▍    | 123500/225000 [251:57:49<152:58:06,  5.43s/it]
                                                               
{'loss': '0.8508', 'grad_norm': '1.133', 'learning_rate': '2.283e-05', 'epoch': '0.5489', 'num_input_tokens_seen': 419733720, 'train_runtime': '9.071e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123500/225000 [251:57:49<152:58:06,  5.43s/it]
 55%|█████▍    | 123501/225000 [251:57:54<144:29:23,  5.12s/it]
 55%|█████▍    | 123502/225000 [251:57:58<136:15:36,  4.83s/it]
 55%|█████▍    | 123503/225000 [251:58:08<183:40:21,  6.51s/it]
 55%|█████▍    | 123504/225000 [251:58:14<178:31:12,  6.33s/it]
 55%|█████▍    | 123505/225000 [251:58:18<160:56:00,  5.71s/it]
 55%|█████▍    | 123506/225000 [251:58:22<147:56:21,  5.25s/it]
 55%|█████▍    | 123507/225000 [251:58:27<140:18:40,  4.98s/it]
 55%|█████▍    | 123508/225000 [251:58:39<204:37:11,  7.26s/it]
 55%|█████▍    | 123509/225000 [251:58:45<190:45:24,  6.77s/it]
 55%|█████▍    | 123510/225000 [251:58:52<189:20:59,  6.72s/it]
 55%|█████▍    | 123511/225000 [251:58:56<170:41:08,  6.05s/it]
 55%|█████▍    | 123512/225000 [251:59:02<166:32:48,  5.91s/it]
 55%|█████▍    | 123513/225000 [251:59:07<162:34:09,  5.77s/it]
 55%|█████▍    | 123514/225000 [251:59:15<180:31:41,  6.40s/it]
 55%|█████▍    | 123515/225000 [251:59:19<160:40:29,  5.70s/it]
 55%|█████▍    | 123516/225000 [251:59:23<142:12:56,  5.04s/it]
 55%|█████▍    | 123517/225000 [251:59:32<178:44:09,  6.34s/it]
 55%|█████▍    | 123518/225000 [251:59:36<161:14:03,  5.72s/it]
 55%|█████▍    | 123519/225000 [251:59:40<141:02:19,  5.00s/it]
 55%|█████▍    | 123520/225000 [251:59:44<136:57:41,  4.86s/it]
 55%|█████▍    | 123521/225000 [251:59:50<142:19:01,  5.05s/it]
 55%|█████▍    | 123522/225000 [251:59:54<138:03:41,  4.90s/it]
 55%|█████▍    | 123523/225000 [251:59:59<138:55:57,  4.93s/it]
 55%|█████▍    | 123524/225000 [252:00:05<145:33:53,  5.16s/it]
 55%|█████▍    | 123525/225000 [252:00:17<207:24:07,  7.36s/it]
 55%|█████▍    | 123526/225000 [252:00:21<180:38:12,  6.41s/it]
 55%|█████▍    | 123527/225000 [252:00:27<172:50:58,  6.13s/it]
 55%|█████▍    | 123528/225000 [252:00:33<172:10:08,  6.11s/it]
 55%|█████▍    | 123529/225000 [252:00:37<153:11:49,  5.44s/it]
 55%|█████▍    | 123530/225000 [252:00:42<148:05:26,  5.25s/it]
 55%|█████▍    | 123531/225000 [252:00:47<146:25:23,  5.19s/it]
 55%|█████▍    | 123532/225000 [252:00:52<145:50:56,  5.17s/it]
 55%|█████▍    | 123533/225000 [252:00:57<143:46:14,  5.10s/it]
 55%|█████▍    | 123534/225000 [252:01:02<143:51:02,  5.10s/it]
 55%|█████▍    | 123535/225000 [252:01:07<146:32:00,  5.20s/it]
 55%|█████▍    | 123536/225000 [252:01:12<141:03:05,  5.00s/it]
 55%|█████▍    | 123537/225000 [252:01:17<140:20:35,  4.98s/it]
 55%|█████▍    | 123538/225000 [252:01:29<199:54:50,  7.09s/it]
 55%|█████▍    | 123539/225000 [252:01:36<198:00:07,  7.03s/it]
 55%|█████▍    | 123540/225000 [252:01:42<193:31:47,  6.87s/it]
 55%|█████▍    | 123541/225000 [252:01:46<168:16:38,  5.97s/it]
 55%|█████▍    | 123542/225000 [252:01:56<197:24:06,  7.00s/it]
 55%|█████▍    | 123543/225000 [252:01:59<169:22:09,  6.01s/it]
 55%|█████▍    | 123544/225000 [252:02:03<147:55:02,  5.25s/it]
 55%|█████▍    | 123545/225000 [252:02:09<156:39:17,  5.56s/it]
 55%|█████▍    | 123546/225000 [252:02:17<181:41:17,  6.45s/it]
 55%|█████▍    | 123547/225000 [252:02:22<163:02:40,  5.79s/it]
 55%|█████▍    | 123548/225000 [252:02:26<151:47:43,  5.39s/it]
 55%|█████▍    | 123549/225000 [252:02:31<147:28:35,  5.23s/it]
 55%|█████▍    | 123550/225000 [252:02:41<183:28:42,  6.51s/it]
                                                               
{'loss': '0.8338', 'grad_norm': '1.069', 'learning_rate': '2.281e-05', 'epoch': '0.5491', 'num_input_tokens_seen': 419861624, 'train_runtime': '9.074e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123550/225000 [252:02:41<183:28:42,  6.51s/it]
 55%|█████▍    | 123551/225000 [252:02:45<167:19:33,  5.94s/it]
 55%|█████▍    | 123552/225000 [252:02:50<156:05:10,  5.54s/it]
 55%|█████▍    | 123553/225000 [252:02:55<149:25:42,  5.30s/it]
 55%|█████▍    | 123554/225000 [252:02:59<141:04:12,  5.01s/it]
 55%|█████▍    | 123555/225000 [252:03:04<140:12:08,  4.98s/it]
 55%|█████▍    | 123556/225000 [252:03:08<138:01:28,  4.90s/it]
 55%|█████▍    | 123557/225000 [252:03:13<136:22:41,  4.84s/it]
 55%|█████▍    | 123558/225000 [252:03:22<174:28:08,  6.19s/it]
 55%|█████▍    | 123559/225000 [252:03:32<205:43:49,  7.30s/it]
 55%|█████▍    | 123560/225000 [252:03:36<172:10:27,  6.11s/it]
 55%|█████▍    | 123561/225000 [252:03:43<179:07:02,  6.36s/it]
 55%|█████▍    | 123562/225000 [252:03:46<154:38:44,  5.49s/it]
 55%|█████▍    | 123563/225000 [252:03:54<174:31:57,  6.19s/it]
 55%|█████▍    | 123564/225000 [252:03:59<164:29:18,  5.84s/it]
 55%|█████▍    | 123565/225000 [252:04:06<171:52:28,  6.10s/it]
 55%|█████▍    | 123566/225000 [252:04:10<154:02:00,  5.47s/it]
 55%|█████▍    | 123567/225000 [252:04:17<169:28:35,  6.01s/it]
 55%|█████▍    | 123568/225000 [252:04:22<163:57:36,  5.82s/it]
 55%|█████▍    | 123569/225000 [252:04:27<157:29:39,  5.59s/it]
 55%|█████▍    | 123570/225000 [252:04:33<155:07:18,  5.51s/it]
 55%|█████▍    | 123571/225000 [252:04:38<149:55:12,  5.32s/it]
 55%|█████▍    | 123572/225000 [252:04:43<147:03:10,  5.22s/it]
 55%|█████▍    | 123573/225000 [252:04:48<145:58:24,  5.18s/it]
 55%|█████▍    | 123574/225000 [252:04:53<149:26:07,  5.30s/it]
 55%|█████▍    | 123575/225000 [252:04:59<149:29:14,  5.31s/it]
 55%|█████▍    | 123576/225000 [252:05:03<141:16:04,  5.01s/it]
 55%|█████▍    | 123577/225000 [252:05:08<139:03:27,  4.94s/it]
 55%|█████▍    | 123578/225000 [252:05:13<144:43:03,  5.14s/it]
 55%|█████▍    | 123579/225000 [252:05:19<147:18:06,  5.23s/it]
 55%|█████▍    | 123580/225000 [252:05:24<149:04:44,  5.29s/it]
 55%|█████▍    | 123581/225000 [252:05:29<149:12:37,  5.30s/it]
 55%|█████▍    | 123582/225000 [252:05:42<206:54:12,  7.34s/it]
 55%|█████▍    | 123583/225000 [252:05:46<183:08:27,  6.50s/it]
 55%|█████▍    | 123584/225000 [252:05:50<162:50:22,  5.78s/it]
 55%|█████▍    | 123585/225000 [252:05:54<148:26:27,  5.27s/it]
 55%|█████▍    | 123586/225000 [252:05:59<142:57:09,  5.07s/it]
 55%|█████▍    | 123587/225000 [252:06:04<140:38:50,  4.99s/it]
 55%|█████▍    | 123588/225000 [252:06:09<140:56:36,  5.00s/it]
 55%|█████▍    | 123589/225000 [252:06:16<159:00:56,  5.64s/it]
 55%|█████▍    | 123590/225000 [252:06:25<186:10:55,  6.61s/it]
 55%|█████▍    | 123591/225000 [252:06:28<158:32:26,  5.63s/it]
 55%|█████▍    | 123592/225000 [252:06:38<193:47:40,  6.88s/it]
 55%|█████▍    | 123593/225000 [252:06:41<163:26:19,  5.80s/it]
 55%|█████▍    | 123594/225000 [252:06:48<169:18:24,  6.01s/it]
 55%|█████▍    | 123595/225000 [252:06:54<171:15:25,  6.08s/it]
 55%|█████▍    | 123596/225000 [252:06:59<162:31:59,  5.77s/it]
 55%|█████▍    | 123597/225000 [252:07:04<157:30:58,  5.59s/it]
 55%|█████▍    | 123598/225000 [252:07:09<155:11:22,  5.51s/it]
 55%|█████▍    | 123599/225000 [252:07:15<154:42:24,  5.49s/it]
 55%|█████▍    | 123600/225000 [252:07:20<150:51:49,  5.36s/it]
                                                               
{'loss': '0.8497', 'grad_norm': '1.002', 'learning_rate': '2.279e-05', 'epoch': '0.5493', 'num_input_tokens_seen': 419988648, 'train_runtime': '9.076e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123600/225000 [252:07:20<150:51:49,  5.36s/it]
 55%|█████▍    | 123601/225000 [252:07:25<151:46:26,  5.39s/it]
 55%|█████▍    | 123602/225000 [252:07:31<155:42:02,  5.53s/it]
 55%|█████▍    | 123603/225000 [252:07:36<153:37:55,  5.45s/it]
 55%|█████▍    | 123604/225000 [252:07:42<152:40:21,  5.42s/it]
 55%|█████▍    | 123605/225000 [252:07:47<149:36:37,  5.31s/it]
 55%|█████▍    | 123606/225000 [252:07:52<151:13:29,  5.37s/it]
 55%|█████▍    | 123607/225000 [252:08:01<175:52:10,  6.24s/it]
 55%|█████▍    | 123608/225000 [252:08:11<209:03:00,  7.42s/it]
 55%|█████▍    | 123609/225000 [252:08:15<181:11:22,  6.43s/it]
 55%|█████▍    | 123610/225000 [252:08:19<159:22:20,  5.66s/it]
 55%|█████▍    | 123611/225000 [252:08:28<189:30:47,  6.73s/it]
 55%|█████▍    | 123612/225000 [252:08:34<178:46:43,  6.35s/it]
 55%|█████▍    | 123613/225000 [252:08:41<188:49:48,  6.70s/it]
 55%|█████▍    | 123614/225000 [252:08:45<162:34:08,  5.77s/it]
 55%|█████▍    | 123615/225000 [252:08:51<163:32:12,  5.81s/it]
 55%|█████▍    | 123616/225000 [252:08:58<173:54:45,  6.18s/it]
 55%|█████▍    | 123617/225000 [252:09:05<182:09:40,  6.47s/it]
 55%|█████▍    | 123618/225000 [252:09:08<157:50:09,  5.60s/it]
 55%|█████▍    | 123619/225000 [252:09:12<142:05:29,  5.05s/it]
 55%|█████▍    | 123620/225000 [252:09:21<175:56:47,  6.25s/it]
 55%|█████▍    | 123621/225000 [252:09:25<152:57:25,  5.43s/it]
 55%|█████▍    | 123622/225000 [252:09:34<186:30:31,  6.62s/it]
 55%|█████▍    | 123623/225000 [252:09:37<158:48:28,  5.64s/it]
 55%|█████▍    | 123624/225000 [252:09:44<169:22:45,  6.01s/it]
 55%|█████▍    | 123625/225000 [252:09:49<157:15:58,  5.58s/it]
 55%|█████▍    | 123626/225000 [252:09:53<145:24:08,  5.16s/it]
 55%|█████▍    | 123627/225000 [252:09:57<138:28:08,  4.92s/it]
 55%|█████▍    | 123628/225000 [252:10:02<137:04:09,  4.87s/it]
 55%|█████▍    | 123629/225000 [252:10:13<187:22:05,  6.65s/it]
 55%|█████▍    | 123630/225000 [252:10:17<164:52:28,  5.86s/it]
 55%|█████▍    | 123631/225000 [252:10:21<150:19:35,  5.34s/it]
 55%|█████▍    | 123632/225000 [252:10:26<146:47:42,  5.21s/it]
 55%|█████▍    | 123633/225000 [252:10:38<202:53:20,  7.21s/it]
 55%|█████▍    | 123634/225000 [252:10:42<175:05:31,  6.22s/it]
 55%|█████▍    | 123635/225000 [252:10:47<165:09:12,  5.87s/it]
 55%|█████▍    | 123636/225000 [252:10:53<169:49:51,  6.03s/it]
 55%|█████▍    | 123637/225000 [252:10:57<149:13:12,  5.30s/it]
 55%|█████▍    | 123638/225000 [252:11:07<187:49:46,  6.67s/it]
 55%|█████▍    | 123639/225000 [252:11:11<171:12:43,  6.08s/it]
 55%|█████▍    | 123640/225000 [252:11:16<161:09:20,  5.72s/it]
 55%|█████▍    | 123641/225000 [252:11:21<152:22:23,  5.41s/it]
 55%|█████▍    | 123642/225000 [252:11:26<150:23:02,  5.34s/it]
 55%|█████▍    | 123643/225000 [252:11:31<143:41:25,  5.10s/it]
 55%|█████▍    | 123644/225000 [252:11:35<139:43:58,  4.96s/it]
 55%|█████▍    | 123645/225000 [252:11:40<139:08:03,  4.94s/it]
 55%|█████▍    | 123646/225000 [252:11:44<132:53:37,  4.72s/it]
 55%|█████▍    | 123647/225000 [252:11:49<131:06:55,  4.66s/it]
 55%|█████▍    | 123648/225000 [252:11:54<134:01:00,  4.76s/it]
 55%|█████▍    | 123649/225000 [252:11:59<135:08:01,  4.80s/it]
 55%|█████▍    | 123650/225000 [252:12:04<136:48:23,  4.86s/it]
                                                               
{'loss': '0.8429', 'grad_norm': '1.06', 'learning_rate': '2.278e-05', 'epoch': '0.5496', 'num_input_tokens_seen': 420117896, 'train_runtime': '9.079e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123650/225000 [252:12:04<136:48:23,  4.86s/it]
 55%|█████▍    | 123651/225000 [252:12:16<195:40:21,  6.95s/it]
 55%|█████▍    | 123652/225000 [252:12:23<195:49:47,  6.96s/it]
 55%|█████▍    | 123653/225000 [252:12:27<175:16:21,  6.23s/it]
 55%|█████▍    | 123654/225000 [252:12:31<158:03:08,  5.61s/it]
 55%|█████▍    | 123655/225000 [252:12:36<149:16:28,  5.30s/it]
 55%|█████▍    | 123656/225000 [252:12:41<144:48:42,  5.14s/it]
 55%|█████▍    | 123657/225000 [252:12:45<139:31:06,  4.96s/it]
 55%|█████▍    | 123658/225000 [252:12:53<166:22:52,  5.91s/it]
 55%|█████▍    | 123659/225000 [252:13:01<185:11:35,  6.58s/it]
 55%|█████▍    | 123660/225000 [252:13:05<158:08:41,  5.62s/it]
 55%|█████▍    | 123661/225000 [252:13:13<183:40:00,  6.52s/it]
 55%|█████▍    | 123662/225000 [252:13:17<160:08:13,  5.69s/it]
 55%|█████▍    | 123663/225000 [252:13:24<166:21:30,  5.91s/it]
 55%|█████▍    | 123664/225000 [252:13:30<172:27:51,  6.13s/it]
 55%|█████▍    | 123665/225000 [252:13:34<151:42:11,  5.39s/it]
 55%|█████▍    | 123666/225000 [252:13:43<182:03:11,  6.47s/it]
 55%|█████▍    | 123667/225000 [252:13:46<154:59:54,  5.51s/it]
 55%|█████▍    | 123668/225000 [252:13:52<153:40:16,  5.46s/it]
 55%|█████▍    | 123669/225000 [252:13:57<156:54:11,  5.57s/it]
 55%|█████▍    | 123670/225000 [252:14:05<173:01:26,  6.15s/it]
 55%|█████▍    | 123671/225000 [252:14:11<169:49:21,  6.03s/it]
 55%|█████▍    | 123672/225000 [252:14:17<173:37:14,  6.17s/it]
 55%|█████▍    | 123673/225000 [252:14:25<184:25:48,  6.55s/it]
 55%|█████▍    | 123674/225000 [252:14:28<157:57:13,  5.61s/it]
 55%|█████▍    | 123675/225000 [252:14:35<173:24:30,  6.16s/it]
 55%|█████▍    | 123676/225000 [252:14:39<155:13:01,  5.51s/it]
 55%|█████▍    | 123677/225000 [252:14:44<143:06:08,  5.08s/it]
 55%|█████▍    | 123678/225000 [252:14:48<134:47:51,  4.79s/it]
 55%|█████▍    | 123679/225000 [252:14:57<175:45:20,  6.24s/it]
 55%|█████▍    | 123680/225000 [252:15:01<155:52:08,  5.54s/it]
 55%|█████▍    | 123681/225000 [252:15:05<145:37:58,  5.17s/it]
 55%|█████▍    | 123682/225000 [252:15:09<134:57:18,  4.80s/it]
 55%|█████▍    | 123683/225000 [252:15:16<149:55:00,  5.33s/it]
 55%|█████▍    | 123684/225000 [252:15:26<188:57:13,  6.71s/it]
 55%|█████▍    | 123685/225000 [252:15:29<158:28:37,  5.63s/it]
 55%|█████▍    | 123686/225000 [252:15:37<181:32:01,  6.45s/it]
 55%|█████▍    | 123687/225000 [252:15:41<157:16:53,  5.59s/it]
 55%|█████▍    | 123688/225000 [252:15:45<142:17:46,  5.06s/it]
 55%|█████▍    | 123689/225000 [252:15:55<184:48:50,  6.57s/it]
 55%|█████▍    | 123690/225000 [252:15:59<160:55:14,  5.72s/it]
 55%|█████▍    | 123691/225000 [252:16:02<144:09:13,  5.12s/it]
 55%|█████▍    | 123692/225000 [252:16:09<153:58:03,  5.47s/it]
 55%|█████▍    | 123693/225000 [252:16:14<151:40:38,  5.39s/it]
 55%|█████▍    | 123694/225000 [252:16:18<139:27:32,  4.96s/it]
 55%|█████▍    | 123695/225000 [252:16:21<129:05:18,  4.59s/it]
 55%|█████▍    | 123696/225000 [252:16:26<128:49:29,  4.58s/it]
 55%|█████▍    | 123697/225000 [252:16:33<150:58:22,  5.37s/it]
 55%|█████▍    | 123698/225000 [252:16:42<182:00:51,  6.47s/it]
 55%|█████▍    | 123699/225000 [252:16:46<161:13:19,  5.73s/it]
 55%|█████▍    | 123700/225000 [252:16:50<145:14:20,  5.16s/it]
                                                               
{'loss': '0.8413', 'grad_norm': '1.019', 'learning_rate': '2.276e-05', 'epoch': '0.5498', 'num_input_tokens_seen': 420246296, 'train_runtime': '9.082e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▍    | 123700/225000 [252:16:50<145:14:20,  5.16s/it]
 55%|█████▍    | 123701/225000 [252:17:00<182:49:50,  6.50s/it]
 55%|█████▍    | 123702/225000 [252:17:04<167:50:50,  5.97s/it]
 55%|█████▍    | 123703/225000 [252:17:09<155:53:08,  5.54s/it]
 55%|█████▍    | 123704/225000 [252:17:14<147:02:22,  5.23s/it]
 55%|█████▍    | 123705/225000 [252:17:18<141:17:07,  5.02s/it]
 55%|█████▍    | 123706/225000 [252:17:22<134:06:27,  4.77s/it]
 55%|█████▍    | 123707/225000 [252:17:27<136:25:39,  4.85s/it]
 55%|█████▍    | 123708/225000 [252:17:36<167:42:50,  5.96s/it]
 55%|█████▍    | 123709/225000 [252:17:45<198:29:31,  7.05s/it]
 55%|█████▍    | 123710/225000 [252:17:49<170:36:43,  6.06s/it]
 55%|█████▍    | 123711/225000 [252:17:57<187:26:30,  6.66s/it]
 55%|█████▍    | 123712/225000 [252:18:01<163:56:27,  5.83s/it]
 55%|█████▍    | 123713/225000 [252:18:07<161:13:43,  5.73s/it]
 55%|█████▍    | 123714/225000 [252:18:15<180:29:45,  6.42s/it]
 55%|█████▍    | 123715/225000 [252:18:18<156:47:53,  5.57s/it]
 55%|█████▍    | 123716/225000 [252:18:22<144:00:51,  5.12s/it]
 55%|█████▍    | 123717/225000 [252:18:26<131:29:07,  4.67s/it]
 55%|█████▍    | 123718/225000 [252:18:36<179:29:46,  6.38s/it]
 55%|█████▍    | 123719/225000 [252:18:41<161:34:10,  5.74s/it]
 55%|█████▍    | 123720/225000 [252:18:45<146:53:11,  5.22s/it]
 55%|█████▍    | 123721/225000 [252:18:49<139:55:48,  4.97s/it]
 55%|█████▍    | 123722/225000 [252:18:54<143:13:39,  5.09s/it]
 55%|█████▍    | 123723/225000 [252:19:05<192:21:51,  6.84s/it]
 55%|█████▍    | 123724/225000 [252:19:09<164:22:04,  5.84s/it]
 55%|█████▍    | 123725/225000 [252:19:13<146:47:49,  5.22s/it]
 55%|█████▍    | 123726/225000 [252:19:22<181:14:29,  6.44s/it]
 55%|█████▍    | 123727/225000 [252:19:25<157:13:25,  5.59s/it]
 55%|█████▍    | 123728/225000 [252:19:29<137:38:00,  4.89s/it]
 55%|█████▍    | 123729/225000 [252:19:38<173:37:01,  6.17s/it]
 55%|█████▍    | 123730/225000 [252:19:42<156:32:52,  5.57s/it]
 55%|█████▍    | 123731/225000 [252:19:49<166:28:49,  5.92s/it]
 55%|█████▍    | 123732/225000 [252:19:52<141:03:33,  5.01s/it]
 55%|█████▍    | 123733/225000 [252:19:56<138:05:29,  4.91s/it]
 55%|█████▍    | 123734/225000 [252:20:01<136:14:19,  4.84s/it]
 55%|█████▍    | 123735/225000 [252:20:06<139:37:11,  4.96s/it]
 55%|█████▍    | 123736/225000 [252:20:12<146:05:33,  5.19s/it]
 55%|█████▍    | 123737/225000 [252:20:25<210:26:46,  7.48s/it]
 55%|█████▍    | 123738/225000 [252:20:29<180:08:19,  6.40s/it]
 55%|█████▍    | 123739/225000 [252:20:36<189:23:12,  6.73s/it]
 55%|█████▍    | 123740/225000 [252:20:41<170:40:13,  6.07s/it]
 55%|█████▍    | 123741/225000 [252:20:47<169:08:47,  6.01s/it]
 55%|█████▍    | 123742/225000 [252:20:52<162:17:13,  5.77s/it]
 55%|█████▍    | 123743/225000 [252:20:58<162:51:45,  5.79s/it]
 55%|█████▍    | 123744/225000 [252:21:04<166:45:37,  5.93s/it]
 55%|█████▍    | 123745/225000 [252:21:11<178:52:06,  6.36s/it]
 55%|█████▍    | 123746/225000 [252:21:15<156:02:17,  5.55s/it]
 55%|█████▍    | 123747/225000 [252:21:24<182:33:12,  6.49s/it]
 55%|█████▍    | 123748/225000 [252:21:28<164:01:33,  5.83s/it]
 55%|█████▍    | 123749/225000 [252:21:32<153:20:39,  5.45s/it]
 55%|█████▌    | 123750/225000 [252:21:37<148:22:23,  5.28s/it]
                                                               
{'loss': '0.8391', 'grad_norm': '1.126', 'learning_rate': '2.274e-05', 'epoch': '0.55', 'num_input_tokens_seen': 420375872, 'train_runtime': '9.085e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 123750/225000 [252:21:37<148:22:23,  5.28s/it]
 55%|█████▌    | 123751/225000 [252:21:42<144:29:06,  5.14s/it]
 55%|█████▌    | 123752/225000 [252:21:47<143:05:39,  5.09s/it]
 55%|█████▌    | 123753/225000 [252:21:52<141:04:18,  5.02s/it]
 55%|█████▌    | 123754/225000 [252:21:56<136:41:10,  4.86s/it]
 55%|█████▌    | 123755/225000 [252:22:01<137:33:21,  4.89s/it]
 55%|█████▌    | 123756/225000 [252:22:06<135:11:31,  4.81s/it]
 55%|█████▌    | 123757/225000 [252:22:13<153:38:01,  5.46s/it]
 55%|█████▌    | 123758/225000 [252:22:25<209:54:10,  7.46s/it]
 55%|█████▌    | 123759/225000 [252:22:29<177:48:33,  6.32s/it]
 55%|█████▌    | 123760/225000 [252:22:32<154:51:57,  5.51s/it]
 55%|█████▌    | 123761/225000 [252:22:37<143:10:04,  5.09s/it]
 55%|█████▌    | 123762/225000 [252:22:48<194:13:43,  6.91s/it]
 55%|█████▌    | 123763/225000 [252:22:52<172:18:07,  6.13s/it]
 55%|█████▌    | 123764/225000 [252:22:57<160:13:59,  5.70s/it]
 55%|█████▌    | 123765/225000 [252:23:01<150:15:58,  5.34s/it]
 55%|█████▌    | 123766/225000 [252:23:05<141:15:51,  5.02s/it]
 55%|█████▌    | 123767/225000 [252:23:10<134:50:11,  4.79s/it]
 55%|█████▌    | 123768/225000 [252:23:19<171:50:11,  6.11s/it]
 55%|█████▌    | 123769/225000 [252:23:27<186:23:40,  6.63s/it]
 55%|█████▌    | 123770/225000 [252:23:30<160:56:27,  5.72s/it]
 55%|█████▌    | 123771/225000 [252:23:35<149:05:11,  5.30s/it]
 55%|█████▌    | 123772/225000 [252:23:39<142:53:30,  5.08s/it]
 55%|█████▌    | 123773/225000 [252:23:44<140:17:52,  4.99s/it]
 55%|█████▌    | 123774/225000 [252:23:50<144:44:19,  5.15s/it]
 55%|█████▌    | 123775/225000 [252:24:01<198:12:45,  7.05s/it]
 55%|█████▌    | 123776/225000 [252:24:06<179:41:44,  6.39s/it]
 55%|█████▌    | 123777/225000 [252:24:11<164:55:57,  5.87s/it]
 55%|█████▌    | 123778/225000 [252:24:16<160:20:45,  5.70s/it]
 55%|█████▌    | 123779/225000 [252:24:21<152:08:26,  5.41s/it]
 55%|█████▌    | 123780/225000 [252:24:26<148:08:44,  5.27s/it]
 55%|█████▌    | 123781/225000 [252:24:30<145:34:41,  5.18s/it]
 55%|█████▌    | 123782/225000 [252:24:35<143:28:56,  5.10s/it]
 55%|█████▌    | 123783/225000 [252:24:40<141:00:34,  5.02s/it]
 55%|█████▌    | 123784/225000 [252:24:46<145:04:16,  5.16s/it]
 55%|█████▌    | 123785/225000 [252:24:58<206:18:48,  7.34s/it]
 55%|█████▌    | 123786/225000 [252:25:04<195:31:25,  6.95s/it]
 55%|█████▌    | 123787/225000 [252:25:12<198:50:21,  7.07s/it]
 55%|█████▌    | 123788/225000 [252:25:15<166:49:32,  5.93s/it]
 55%|█████▌    | 123789/225000 [252:25:22<176:06:10,  6.26s/it]
 55%|█████▌    | 123790/225000 [252:25:27<166:00:43,  5.90s/it]
 55%|█████▌    | 123791/225000 [252:25:31<150:40:47,  5.36s/it]
 55%|█████▌    | 123792/225000 [252:25:35<141:13:10,  5.02s/it]
 55%|█████▌    | 123793/225000 [252:25:45<178:51:51,  6.36s/it]
 55%|█████▌    | 123794/225000 [252:25:48<152:52:32,  5.44s/it]
 55%|█████▌    | 123795/225000 [252:25:56<171:23:34,  6.10s/it]
 55%|█████▌    | 123796/225000 [252:26:00<156:52:02,  5.58s/it]
 55%|█████▌    | 123797/225000 [252:26:05<147:46:41,  5.26s/it]
 55%|█████▌    | 123798/225000 [252:26:09<143:19:43,  5.10s/it]
 55%|█████▌    | 123799/225000 [252:26:13<135:02:39,  4.80s/it]
 55%|█████▌    | 123800/225000 [252:26:18<133:01:19,  4.73s/it]
                                                               
{'loss': '0.8508', 'grad_norm': '1.139', 'learning_rate': '2.272e-05', 'epoch': '0.5502', 'num_input_tokens_seen': 420504592, 'train_runtime': '9.088e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 123800/225000 [252:26:18<133:01:19,  4.73s/it]
 55%|█████▌    | 123801/225000 [252:26:22<130:46:39,  4.65s/it]
 55%|█████▌    | 123802/225000 [252:26:31<165:00:46,  5.87s/it]
 55%|█████▌    | 123803/225000 [252:26:39<182:59:53,  6.51s/it]
 55%|█████▌    | 123804/225000 [252:26:44<168:02:08,  5.98s/it]
 55%|█████▌    | 123805/225000 [252:26:50<173:13:18,  6.16s/it]
 55%|█████▌    | 123806/225000 [252:26:54<152:43:47,  5.43s/it]
 55%|█████▌    | 123807/225000 [252:27:03<178:48:38,  6.36s/it]
 55%|█████▌    | 123808/225000 [252:27:08<166:13:36,  5.91s/it]
 55%|█████▌    | 123809/225000 [252:27:12<153:59:18,  5.48s/it]
 55%|█████▌    | 123810/225000 [252:27:21<181:12:18,  6.45s/it]
 55%|█████▌    | 123811/225000 [252:27:25<162:40:47,  5.79s/it]
 55%|█████▌    | 123812/225000 [252:27:30<153:18:44,  5.45s/it]
 55%|█████▌    | 123813/225000 [252:27:34<145:31:13,  5.18s/it]
 55%|█████▌    | 123814/225000 [252:27:39<139:40:14,  4.97s/it]
 55%|█████▌    | 123815/225000 [252:27:43<138:03:51,  4.91s/it]
 55%|█████▌    | 123816/225000 [252:27:48<136:11:41,  4.85s/it]
 55%|█████▌    | 123817/225000 [252:27:59<190:20:03,  6.77s/it]
 55%|█████▌    | 123818/225000 [252:28:05<182:33:42,  6.50s/it]
 55%|█████▌    | 123819/225000 [252:28:10<168:04:31,  5.98s/it]
 55%|█████▌    | 123820/225000 [252:28:15<157:34:16,  5.61s/it]
 55%|█████▌    | 123821/225000 [252:28:19<149:46:25,  5.33s/it]
 55%|█████▌    | 123822/225000 [252:28:24<146:26:34,  5.21s/it]
 55%|█████▌    | 123823/225000 [252:28:34<184:57:01,  6.58s/it]
 55%|█████▌    | 123824/225000 [252:28:42<193:34:23,  6.89s/it]
 55%|█████▌    | 123825/225000 [252:28:45<162:55:35,  5.80s/it]
 55%|█████▌    | 123826/225000 [252:28:53<179:25:36,  6.38s/it]
 55%|█████▌    | 123827/225000 [252:28:57<160:51:45,  5.72s/it]
 55%|█████▌    | 123828/225000 [252:29:04<168:09:06,  5.98s/it]
 55%|█████▌    | 123829/225000 [252:29:09<167:39:30,  5.97s/it]
 55%|█████▌    | 123830/225000 [252:29:14<153:34:26,  5.46s/it]
 55%|█████▌    | 123831/225000 [252:29:18<146:57:28,  5.23s/it]
 55%|█████▌    | 123832/225000 [252:29:23<139:22:08,  4.96s/it]
 55%|█████▌    | 123833/225000 [252:29:33<184:30:09,  6.57s/it]
 55%|█████▌    | 123834/225000 [252:29:36<157:33:18,  5.61s/it]
 55%|█████▌    | 123835/225000 [252:29:44<176:25:02,  6.28s/it]
 55%|█████▌    | 123836/225000 [252:29:49<163:30:59,  5.82s/it]
 55%|█████▌    | 123837/225000 [252:29:54<152:30:13,  5.43s/it]
 55%|█████▌    | 123838/225000 [252:29:58<144:31:38,  5.14s/it]
 55%|█████▌    | 123839/225000 [252:30:02<136:04:31,  4.84s/it]
 55%|█████▌    | 123840/225000 [252:30:07<134:21:43,  4.78s/it]
 55%|█████▌    | 123841/225000 [252:30:12<135:20:05,  4.82s/it]
 55%|█████▌    | 123842/225000 [252:30:16<134:00:14,  4.77s/it]
 55%|█████▌    | 123843/225000 [252:30:21<133:07:45,  4.74s/it]
 55%|█████▌    | 123844/225000 [252:30:26<133:29:19,  4.75s/it]
 55%|█████▌    | 123845/225000 [252:30:38<194:14:47,  6.91s/it]
 55%|█████▌    | 123846/225000 [252:30:43<183:44:51,  6.54s/it]
 55%|█████▌    | 123847/225000 [252:30:50<181:06:09,  6.45s/it]
 55%|█████▌    | 123848/225000 [252:30:53<157:45:40,  5.61s/it]
 55%|█████▌    | 123849/225000 [252:30:57<143:31:07,  5.11s/it]
 55%|█████▌    | 123850/225000 [252:31:07<185:45:42,  6.61s/it]
                                                               
{'loss': '0.8266', 'grad_norm': '1.044', 'learning_rate': '2.27e-05', 'epoch': '0.5504', 'num_input_tokens_seen': 420634120, 'train_runtime': '9.091e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 123850/225000 [252:31:07<185:45:42,  6.61s/it]
 55%|█████▌    | 123851/225000 [252:31:13<178:10:13,  6.34s/it]
 55%|█████▌    | 123852/225000 [252:31:17<155:40:08,  5.54s/it]
 55%|█████▌    | 123853/225000 [252:31:20<139:01:47,  4.95s/it]
 55%|█████▌    | 123854/225000 [252:31:25<134:57:45,  4.80s/it]
 55%|█████▌    | 123855/225000 [252:31:29<132:56:17,  4.73s/it]
 55%|█████▌    | 123856/225000 [252:31:35<136:24:58,  4.86s/it]
 55%|█████▌    | 123857/225000 [252:31:39<137:21:42,  4.89s/it]
 55%|█████▌    | 123858/225000 [252:31:45<138:35:07,  4.93s/it]
 55%|█████▌    | 123859/225000 [252:31:49<138:53:38,  4.94s/it]
 55%|█████▌    | 123860/225000 [252:32:01<194:14:57,  6.91s/it]
 55%|█████▌    | 123861/225000 [252:32:07<189:43:11,  6.75s/it]
 55%|█████▌    | 123862/225000 [252:32:14<189:06:46,  6.73s/it]
 55%|█████▌    | 123863/225000 [252:32:18<163:08:45,  5.81s/it]
 55%|█████▌    | 123864/225000 [252:32:22<151:09:44,  5.38s/it]
 55%|█████▌    | 123865/225000 [252:32:27<145:50:57,  5.19s/it]
 55%|█████▌    | 123866/225000 [252:32:32<144:08:49,  5.13s/it]
 55%|█████▌    | 123867/225000 [252:32:36<138:43:38,  4.94s/it]
 55%|█████▌    | 123868/225000 [252:32:41<134:57:45,  4.80s/it]
 55%|█████▌    | 123869/225000 [252:32:53<199:01:53,  7.08s/it]
 55%|█████▌    | 123870/225000 [252:32:57<169:45:51,  6.04s/it]
 55%|█████▌    | 123871/225000 [252:33:03<170:20:50,  6.06s/it]
 55%|█████▌    | 123872/225000 [252:33:10<180:31:32,  6.43s/it]
 55%|█████▌    | 123873/225000 [252:33:14<158:19:12,  5.64s/it]
 55%|█████▌    | 123874/225000 [252:33:18<142:18:07,  5.07s/it]
 55%|█████▌    | 123875/225000 [252:33:26<167:58:28,  5.98s/it]
 55%|█████▌    | 123876/225000 [252:33:34<184:03:43,  6.55s/it]
 55%|█████▌    | 123877/225000 [252:33:39<172:17:11,  6.13s/it]
 55%|█████▌    | 123878/225000 [252:33:45<170:53:48,  6.08s/it]
 55%|█████▌    | 123879/225000 [252:33:48<144:59:54,  5.16s/it]
 55%|█████▌    | 123880/225000 [252:33:54<153:33:45,  5.47s/it]
 55%|█████▌    | 123881/225000 [252:34:02<172:18:55,  6.13s/it]
 55%|█████▌    | 123882/225000 [252:34:06<154:58:33,  5.52s/it]
 55%|█████▌    | 123883/225000 [252:34:10<143:35:17,  5.11s/it]
 55%|█████▌    | 123884/225000 [252:34:15<140:15:32,  4.99s/it]
 55%|█████▌    | 123885/225000 [252:34:19<132:26:56,  4.72s/it]
 55%|█████▌    | 123886/225000 [252:34:30<189:15:17,  6.74s/it]
 55%|█████▌    | 123887/225000 [252:34:34<167:43:48,  5.97s/it]
 55%|█████▌    | 123888/225000 [252:34:42<180:00:29,  6.41s/it]
 55%|█████▌    | 123889/225000 [252:34:47<166:47:54,  5.94s/it]
 55%|█████▌    | 123890/225000 [252:34:51<153:17:28,  5.46s/it]
 55%|█████▌    | 123891/225000 [252:34:55<144:08:50,  5.13s/it]
 55%|█████▌    | 123892/225000 [252:35:06<191:57:37,  6.83s/it]
 55%|█████▌    | 123893/225000 [252:35:14<201:44:04,  7.18s/it]
 55%|█████▌    | 123894/225000 [252:35:18<173:57:07,  6.19s/it]
 55%|█████▌    | 123895/225000 [252:35:22<154:40:58,  5.51s/it]
 55%|█████▌    | 123896/225000 [252:35:26<142:15:07,  5.07s/it]
 55%|█████▌    | 123897/225000 [252:35:36<185:23:31,  6.60s/it]
 55%|█████▌    | 123898/225000 [252:35:40<158:11:12,  5.63s/it]
 55%|█████▌    | 123899/225000 [252:35:49<189:44:25,  6.76s/it]
 55%|█████▌    | 123900/225000 [252:35:54<176:41:26,  6.29s/it]
                                                               
{'loss': '0.8468', 'grad_norm': '1.173', 'learning_rate': '2.269e-05', 'epoch': '0.5507', 'num_input_tokens_seen': 420763672, 'train_runtime': '9.094e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 123900/225000 [252:35:54<176:41:26,  6.29s/it]
 55%|█████▌    | 123901/225000 [252:35:58<157:43:24,  5.62s/it]
 55%|█████▌    | 123902/225000 [252:36:03<149:45:08,  5.33s/it]
 55%|█████▌    | 123903/225000 [252:36:07<141:23:47,  5.04s/it]
 55%|█████▌    | 123904/225000 [252:36:12<136:33:21,  4.86s/it]
 55%|█████▌    | 123905/225000 [252:36:18<145:18:52,  5.17s/it]
 55%|█████▌    | 123906/225000 [252:36:28<189:26:08,  6.75s/it]
 55%|█████▌    | 123907/225000 [252:36:34<181:26:17,  6.46s/it]
 55%|█████▌    | 123908/225000 [252:36:41<191:21:51,  6.81s/it]
 55%|█████▌    | 123909/225000 [252:36:47<182:00:12,  6.48s/it]
 55%|█████▌    | 123910/225000 [252:36:50<152:13:06,  5.42s/it]
 55%|█████▌    | 123911/225000 [252:36:58<176:39:09,  6.29s/it]
 55%|█████▌    | 123912/225000 [252:37:02<153:36:37,  5.47s/it]
 55%|█████▌    | 123913/225000 [252:37:06<140:47:26,  5.01s/it]
 55%|█████▌    | 123914/225000 [252:37:11<137:24:13,  4.89s/it]
 55%|█████▌    | 123915/225000 [252:37:15<134:09:13,  4.78s/it]
 55%|█████▌    | 123916/225000 [252:37:20<133:06:15,  4.74s/it]
 55%|█████▌    | 123917/225000 [252:37:25<135:56:45,  4.84s/it]
 55%|█████▌    | 123918/225000 [252:37:36<193:31:08,  6.89s/it]
 55%|█████▌    | 123919/225000 [252:37:40<163:00:03,  5.81s/it]
 55%|█████▌    | 123920/225000 [252:37:49<190:10:04,  6.77s/it]
 55%|█████▌    | 123921/225000 [252:37:53<171:28:10,  6.11s/it]
 55%|█████▌    | 123922/225000 [252:37:57<150:47:48,  5.37s/it]
 55%|█████▌    | 123923/225000 [252:38:02<146:34:56,  5.22s/it]
 55%|█████▌    | 123924/225000 [252:38:07<150:13:26,  5.35s/it]
 55%|█████▌    | 123925/225000 [252:38:13<154:42:52,  5.51s/it]
 55%|█████▌    | 123926/225000 [252:38:19<152:44:59,  5.44s/it]
 55%|█████▌    | 123927/225000 [252:38:24<148:55:12,  5.30s/it]
 55%|█████▌    | 123928/225000 [252:38:29<148:38:17,  5.29s/it]
 55%|█████▌    | 123929/225000 [252:38:34<150:31:20,  5.36s/it]
 55%|█████▌    | 123930/225000 [252:38:46<206:17:00,  7.35s/it]
 55%|█████▌    | 123931/225000 [252:38:50<176:05:32,  6.27s/it]
 55%|█████▌    | 123932/225000 [252:38:54<155:28:31,  5.54s/it]
 55%|█████▌    | 123933/225000 [252:38:58<140:58:28,  5.02s/it]
 55%|█████▌    | 123934/225000 [252:39:02<133:01:59,  4.74s/it]
 55%|█████▌    | 123935/225000 [252:39:06<131:06:02,  4.67s/it]
 55%|█████▌    | 123936/225000 [252:39:19<195:12:14,  6.95s/it]
 55%|█████▌    | 123937/225000 [252:39:24<178:51:36,  6.37s/it]
 55%|█████▌    | 123938/225000 [252:39:28<161:16:27,  5.74s/it]
 55%|█████▌    | 123939/225000 [252:39:32<144:03:46,  5.13s/it]
 55%|█████▌    | 123940/225000 [252:39:36<140:00:36,  4.99s/it]
 55%|█████▌    | 123941/225000 [252:39:47<184:14:37,  6.56s/it]
 55%|█████▌    | 123942/225000 [252:39:50<156:10:19,  5.56s/it]
 55%|█████▌    | 123943/225000 [252:39:57<173:11:36,  6.17s/it]
 55%|█████▌    | 123944/225000 [252:40:06<193:09:09,  6.88s/it]
 55%|█████▌    | 123945/225000 [252:40:10<173:33:17,  6.18s/it]
 55%|█████▌    | 123946/225000 [252:40:16<166:03:24,  5.92s/it]
 55%|█████▌    | 123947/225000 [252:40:20<150:58:56,  5.38s/it]
 55%|█████▌    | 123948/225000 [252:40:27<166:02:00,  5.91s/it]
 55%|█████▌    | 123949/225000 [252:40:33<168:05:23,  5.99s/it]
 55%|█████▌    | 123950/225000 [252:40:37<150:30:07,  5.36s/it]
                                                               
{'loss': '0.8512', 'grad_norm': '1.004', 'learning_rate': '2.267e-05', 'epoch': '0.5509', 'num_input_tokens_seen': 420894208, 'train_runtime': '9.096e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 123950/225000 [252:40:37<150:30:07,  5.36s/it]
 55%|█████▌    | 123951/225000 [252:40:41<141:52:43,  5.05s/it]
 55%|█████▌    | 123952/225000 [252:40:52<187:48:45,  6.69s/it]
 55%|█████▌    | 123953/225000 [252:40:56<161:21:58,  5.75s/it]
 55%|█████▌    | 123954/225000 [252:41:01<155:31:43,  5.54s/it]
 55%|█████▌    | 123955/225000 [252:41:05<150:18:42,  5.36s/it]
 55%|█████▌    | 123956/225000 [252:41:10<145:41:32,  5.19s/it]
 55%|█████▌    | 123957/225000 [252:41:15<142:00:10,  5.06s/it]
 55%|█████▌    | 123958/225000 [252:41:20<137:45:15,  4.91s/it]
 55%|█████▌    | 123959/225000 [252:41:24<136:08:56,  4.85s/it]
 55%|█████▌    | 123960/225000 [252:41:31<148:46:06,  5.30s/it]
 55%|█████▌    | 123961/225000 [252:41:44<212:31:27,  7.57s/it]
 55%|█████▌    | 123962/225000 [252:41:47<180:03:42,  6.42s/it]
 55%|█████▌    | 123963/225000 [252:41:52<166:30:00,  5.93s/it]
 55%|█████▌    | 123964/225000 [252:41:57<159:55:49,  5.70s/it]
 55%|█████▌    | 123965/225000 [252:42:02<154:46:53,  5.52s/it]
 55%|█████▌    | 123966/225000 [252:42:08<154:30:42,  5.51s/it]
 55%|█████▌    | 123967/225000 [252:42:13<151:09:10,  5.39s/it]
 55%|█████▌    | 123968/225000 [252:42:18<149:20:56,  5.32s/it]
 55%|█████▌    | 123969/225000 [252:42:23<144:30:58,  5.15s/it]
 55%|█████▌    | 123970/225000 [252:42:28<143:36:50,  5.12s/it]
 55%|█████▌    | 123971/225000 [252:42:33<142:15:05,  5.07s/it]
 55%|█████▌    | 123972/225000 [252:42:38<143:18:38,  5.11s/it]
 55%|█████▌    | 123973/225000 [252:42:50<201:42:38,  7.19s/it]
 55%|█████▌    | 123974/225000 [252:42:55<178:45:36,  6.37s/it]
 55%|█████▌    | 123975/225000 [252:43:02<187:02:46,  6.67s/it]
 55%|█████▌    | 123976/225000 [252:43:06<169:19:19,  6.03s/it]
 55%|█████▌    | 123977/225000 [252:43:15<192:10:19,  6.85s/it]
 55%|█████▌    | 123978/225000 [252:43:19<164:08:43,  5.85s/it]
 55%|█████▌    | 123979/225000 [252:43:28<191:08:38,  6.81s/it]
 55%|█████▌    | 123980/225000 [252:43:32<166:55:44,  5.95s/it]
 55%|█████▌    | 123981/225000 [252:43:36<150:23:26,  5.36s/it]
 55%|█████▌    | 123982/225000 [252:43:42<157:24:43,  5.61s/it]
 55%|█████▌    | 123983/225000 [252:43:50<181:20:34,  6.46s/it]
 55%|█████▌    | 123984/225000 [252:43:53<153:42:13,  5.48s/it]
 55%|█████▌    | 123985/225000 [252:44:02<175:35:52,  6.26s/it]
 55%|█████▌    | 123986/225000 [252:44:05<154:52:01,  5.52s/it]
 55%|█████▌    | 123987/225000 [252:44:14<181:01:55,  6.45s/it]
 55%|█████▌    | 123988/225000 [252:44:18<156:58:00,  5.59s/it]
 55%|█████▌    | 123989/225000 [252:44:22<150:07:51,  5.35s/it]
 55%|█████▌    | 123990/225000 [252:44:31<174:01:28,  6.20s/it]
 55%|█████▌    | 123991/225000 [252:44:36<166:25:14,  5.93s/it]
 55%|█████▌    | 123992/225000 [252:44:42<171:41:30,  6.12s/it]
 55%|█████▌    | 123993/225000 [252:44:46<152:39:33,  5.44s/it]
 55%|█████▌    | 123994/225000 [252:44:56<185:17:38,  6.60s/it]
 55%|█████▌    | 123995/225000 [252:45:00<168:39:12,  6.01s/it]
 55%|█████▌    | 123996/225000 [252:45:07<175:24:19,  6.25s/it]
 55%|█████▌    | 123997/225000 [252:45:11<156:30:00,  5.58s/it]
 55%|█████▌    | 123998/225000 [252:45:15<140:48:14,  5.02s/it]
 55%|█████▌    | 123999/225000 [252:45:19<135:38:41,  4.83s/it]
 55%|█████▌    | 124000/225000 [252:45:24<133:26:21,  4.76s/it]
                                                               
{'loss': '0.8362', 'grad_norm': '1.09', 'learning_rate': '2.265e-05', 'epoch': '0.5511', 'num_input_tokens_seen': 421024304, 'train_runtime': '9.099e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124000/225000 [252:45:24<133:26:21,  4.76s/it]
 55%|█████▌    | 124001/225000 [252:45:28<131:55:35,  4.70s/it]
 55%|█████▌    | 124002/225000 [252:45:41<196:37:54,  7.01s/it]
 55%|█████▌    | 124003/225000 [252:45:45<177:20:34,  6.32s/it]
 55%|█████▌    | 124004/225000 [252:45:52<175:33:10,  6.26s/it]
 55%|█████▌    | 124005/225000 [252:45:56<159:57:26,  5.70s/it]
 55%|█████▌    | 124006/225000 [252:46:00<145:34:59,  5.19s/it]
 55%|█████▌    | 124007/225000 [252:46:12<200:00:25,  7.13s/it]
 55%|█████▌    | 124008/225000 [252:46:17<184:52:07,  6.59s/it]
 55%|█████▌    | 124009/225000 [252:46:21<159:47:22,  5.70s/it]
 55%|█████▌    | 124010/225000 [252:46:29<181:43:13,  6.48s/it]
 55%|█████▌    | 124011/225000 [252:46:33<158:36:36,  5.65s/it]
 55%|█████▌    | 124012/225000 [252:46:36<143:10:29,  5.10s/it]
 55%|█████▌    | 124013/225000 [252:46:47<187:54:09,  6.70s/it]
 55%|█████▌    | 124014/225000 [252:46:51<165:17:34,  5.89s/it]
 55%|█████▌    | 124015/225000 [252:46:57<165:50:32,  5.91s/it]
 55%|█████▌    | 124016/225000 [252:47:02<163:01:16,  5.81s/it]
 55%|█████▌    | 124017/225000 [252:47:08<161:29:04,  5.76s/it]
 55%|█████▌    | 124018/225000 [252:47:11<141:59:55,  5.06s/it]
 55%|█████▌    | 124019/225000 [252:47:16<140:56:23,  5.02s/it]
 55%|█████▌    | 124020/225000 [252:47:21<139:11:44,  4.96s/it]
 55%|█████▌    | 124021/225000 [252:47:26<138:20:33,  4.93s/it]
 55%|█████▌    | 124022/225000 [252:47:31<139:25:17,  4.97s/it]
 55%|█████▌    | 124023/225000 [252:47:36<141:13:41,  5.04s/it]
 55%|█████▌    | 124024/225000 [252:47:41<141:42:17,  5.05s/it]
 55%|█████▌    | 124025/225000 [252:47:46<140:53:48,  5.02s/it]
 55%|█████▌    | 124026/225000 [252:47:51<137:28:49,  4.90s/it]
 55%|█████▌    | 124027/225000 [252:47:56<139:37:56,  4.98s/it]
 55%|█████▌    | 124028/225000 [252:48:00<133:27:10,  4.76s/it]
 55%|█████▌    | 124029/225000 [252:48:13<201:32:48,  7.19s/it]
 55%|█████▌    | 124030/225000 [252:48:19<186:17:03,  6.64s/it]
 55%|█████▌    | 124031/225000 [252:48:23<165:26:20,  5.90s/it]
 55%|█████▌    | 124032/225000 [252:48:31<185:46:49,  6.62s/it]
 55%|█████▌    | 124033/225000 [252:48:35<161:34:03,  5.76s/it]
 55%|█████▌    | 124034/225000 [252:48:40<153:13:39,  5.46s/it]
 55%|█████▌    | 124035/225000 [252:48:44<148:25:53,  5.29s/it]
 55%|█████▌    | 124036/225000 [252:48:50<149:47:47,  5.34s/it]
 55%|█████▌    | 124037/225000 [252:48:55<150:43:11,  5.37s/it]
 55%|█████▌    | 124038/225000 [252:49:00<146:37:11,  5.23s/it]
 55%|█████▌    | 124039/225000 [252:49:06<146:48:56,  5.24s/it]
 55%|█████▌    | 124040/225000 [252:49:11<147:38:50,  5.26s/it]
 55%|█████▌    | 124041/225000 [252:49:16<145:32:28,  5.19s/it]
 55%|█████▌    | 124042/225000 [252:49:21<148:43:39,  5.30s/it]
 55%|█████▌    | 124043/225000 [252:49:27<149:49:13,  5.34s/it]
 55%|█████▌    | 124044/225000 [252:49:32<151:54:06,  5.42s/it]
 55%|█████▌    | 124045/225000 [252:49:38<150:01:53,  5.35s/it]
 55%|█████▌    | 124046/225000 [252:49:45<163:30:04,  5.83s/it]
 55%|█████▌    | 124047/225000 [252:49:57<218:59:15,  7.81s/it]
 55%|█████▌    | 124048/225000 [252:50:02<193:23:24,  6.90s/it]
 55%|█████▌    | 124049/225000 [252:50:06<174:42:00,  6.23s/it]
 55%|█████▌    | 124050/225000 [252:50:11<158:04:10,  5.64s/it]
                                                               
{'loss': '0.8329', 'grad_norm': '1.111', 'learning_rate': '2.263e-05', 'epoch': '0.5513', 'num_input_tokens_seen': 421153664, 'train_runtime': '9.102e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124050/225000 [252:50:11<158:04:10,  5.64s/it]
 55%|█████▌    | 124051/225000 [252:50:16<155:39:33,  5.55s/it]
 55%|█████▌    | 124052/225000 [252:50:21<148:42:46,  5.30s/it]
 55%|█████▌    | 124053/225000 [252:50:26<145:52:01,  5.20s/it]
 55%|█████▌    | 124054/225000 [252:50:31<142:28:30,  5.08s/it]
 55%|█████▌    | 124055/225000 [252:50:41<185:16:15,  6.61s/it]
 55%|█████▌    | 124056/225000 [252:50:48<193:23:02,  6.90s/it]
 55%|█████▌    | 124057/225000 [252:50:52<167:21:26,  5.97s/it]
 55%|█████▌    | 124058/225000 [252:51:01<190:06:31,  6.78s/it]
 55%|█████▌    | 124059/225000 [252:51:04<160:21:13,  5.72s/it]
 55%|█████▌    | 124060/225000 [252:51:13<185:46:13,  6.63s/it]
 55%|█████▌    | 124061/225000 [252:51:16<159:06:22,  5.67s/it]
 55%|█████▌    | 124062/225000 [252:51:20<142:21:29,  5.08s/it]
 55%|█████▌    | 124063/225000 [252:51:24<132:04:33,  4.71s/it]
 55%|█████▌    | 124064/225000 [252:51:28<131:13:05,  4.68s/it]
 55%|█████▌    | 124065/225000 [252:51:40<185:51:49,  6.63s/it]
 55%|█████▌    | 124066/225000 [252:51:45<173:04:34,  6.17s/it]
 55%|█████▌    | 124067/225000 [252:51:49<156:11:08,  5.57s/it]
 55%|█████▌    | 124068/225000 [252:51:53<146:33:31,  5.23s/it]
 55%|█████▌    | 124069/225000 [252:51:58<140:36:46,  5.02s/it]
 55%|█████▌    | 124070/225000 [252:52:09<195:51:37,  6.99s/it]
 55%|█████▌    | 124071/225000 [252:52:13<164:57:35,  5.88s/it]
 55%|█████▌    | 124072/225000 [252:52:20<179:21:25,  6.40s/it]
 55%|█████▌    | 124073/225000 [252:52:24<160:19:54,  5.72s/it]
 55%|█████▌    | 124074/225000 [252:52:28<144:58:47,  5.17s/it]
 55%|█████▌    | 124075/225000 [252:52:32<136:05:50,  4.85s/it]
 55%|█████▌    | 124076/225000 [252:52:44<192:50:17,  6.88s/it]
 55%|█████▌    | 124077/225000 [252:52:49<176:29:20,  6.30s/it]
 55%|█████▌    | 124078/225000 [252:52:52<152:05:02,  5.43s/it]
 55%|█████▌    | 124079/225000 [252:53:00<171:07:36,  6.10s/it]
 55%|█████▌    | 124080/225000 [252:53:05<162:15:43,  5.79s/it]
 55%|█████▌    | 124081/225000 [252:53:09<149:24:54,  5.33s/it]
 55%|█████▌    | 124082/225000 [252:53:14<140:47:30,  5.02s/it]
 55%|█████▌    | 124083/225000 [252:53:18<132:55:12,  4.74s/it]
 55%|█████▌    | 124084/225000 [252:53:26<162:54:33,  5.81s/it]
 55%|█████▌    | 124085/225000 [252:53:33<176:04:42,  6.28s/it]
 55%|█████▌    | 124086/225000 [252:53:37<155:58:38,  5.56s/it]
 55%|█████▌    | 124087/225000 [252:53:42<146:10:55,  5.21s/it]
 55%|█████▌    | 124088/225000 [252:53:46<138:14:36,  4.93s/it]
 55%|█████▌    | 124089/225000 [252:53:52<149:15:53,  5.33s/it]
 55%|█████▌    | 124090/225000 [252:54:04<204:26:53,  7.29s/it]
 55%|█████▌    | 124091/225000 [252:54:07<171:02:51,  6.10s/it]
 55%|█████▌    | 124092/225000 [252:54:12<157:14:35,  5.61s/it]
 55%|█████▌    | 124093/225000 [252:54:16<148:34:09,  5.30s/it]
 55%|█████▌    | 124094/225000 [252:54:21<142:16:21,  5.08s/it]
 55%|█████▌    | 124095/225000 [252:54:26<138:52:51,  4.95s/it]
 55%|█████▌    | 124096/225000 [252:54:30<131:00:57,  4.67s/it]
 55%|█████▌    | 124097/225000 [252:54:34<131:54:02,  4.71s/it]
 55%|█████▌    | 124098/225000 [252:54:46<190:42:20,  6.80s/it]
 55%|█████▌    | 124099/225000 [252:54:52<182:30:29,  6.51s/it]
 55%|█████▌    | 124100/225000 [252:54:56<159:39:50,  5.70s/it]
                                                               
{'loss': '0.8434', 'grad_norm': '1.113', 'learning_rate': '2.261e-05', 'epoch': '0.5516', 'num_input_tokens_seen': 421282776, 'train_runtime': '9.105e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124100/225000 [252:54:56<159:39:50,  5.70s/it]
 55%|█████▌    | 124101/225000 [252:55:06<193:45:43,  6.91s/it]
 55%|█████▌    | 124102/225000 [252:55:11<180:08:39,  6.43s/it]
 55%|█████▌    | 124103/225000 [252:55:15<163:20:58,  5.83s/it]
 55%|█████▌    | 124104/225000 [252:55:20<150:45:18,  5.38s/it]
 55%|█████▌    | 124105/225000 [252:55:24<139:24:26,  4.97s/it]
 55%|█████▌    | 124106/225000 [252:55:28<137:48:26,  4.92s/it]
 55%|█████▌    | 124107/225000 [252:55:40<196:16:34,  7.00s/it]
 55%|█████▌    | 124108/225000 [252:55:44<172:21:15,  6.15s/it]
 55%|█████▌    | 124109/225000 [252:55:49<160:01:02,  5.71s/it]
 55%|█████▌    | 124110/225000 [252:55:54<151:09:24,  5.39s/it]
 55%|█████▌    | 124111/225000 [252:55:58<142:20:01,  5.08s/it]
 55%|█████▌    | 124112/225000 [252:56:03<139:40:53,  4.98s/it]
 55%|█████▌    | 124113/225000 [252:56:08<139:57:39,  4.99s/it]
 55%|█████▌    | 124114/225000 [252:56:13<137:03:52,  4.89s/it]
 55%|█████▌    | 124115/225000 [252:56:18<138:53:48,  4.96s/it]
 55%|█████▌    | 124116/225000 [252:56:23<139:18:51,  4.97s/it]
 55%|█████▌    | 124117/225000 [252:56:35<201:17:36,  7.18s/it]
 55%|█████▌    | 124118/225000 [252:56:38<168:06:14,  6.00s/it]
 55%|█████▌    | 124119/225000 [252:56:42<152:58:38,  5.46s/it]
 55%|█████▌    | 124120/225000 [252:56:47<148:06:28,  5.29s/it]
 55%|█████▌    | 124121/225000 [252:56:52<145:59:59,  5.21s/it]
 55%|█████▌    | 124122/225000 [252:57:00<167:02:29,  5.96s/it]
 55%|█████▌    | 124123/225000 [252:57:10<198:44:37,  7.09s/it]
 55%|█████▌    | 124124/225000 [252:57:14<170:17:04,  6.08s/it]
 55%|█████▌    | 124125/225000 [252:57:18<153:20:19,  5.47s/it]
 55%|█████▌    | 124126/225000 [252:57:26<182:07:12,  6.50s/it]
 55%|█████▌    | 124127/225000 [252:57:33<179:53:20,  6.42s/it]
 55%|█████▌    | 124128/225000 [252:57:37<158:17:47,  5.65s/it]
 55%|█████▌    | 124129/225000 [252:57:43<165:44:49,  5.92s/it]
 55%|█████▌    | 124130/225000 [252:57:46<144:13:25,  5.15s/it]
 55%|█████▌    | 124131/225000 [252:57:55<175:46:22,  6.27s/it]
 55%|█████▌    | 124132/225000 [252:57:59<152:43:34,  5.45s/it]
 55%|█████▌    | 124133/225000 [252:58:04<148:36:39,  5.30s/it]
 55%|█████▌    | 124134/225000 [252:58:13<177:37:14,  6.34s/it]
 55%|█████▌    | 124135/225000 [252:58:16<153:22:19,  5.47s/it]
 55%|█████▌    | 124136/225000 [252:58:20<139:03:56,  4.96s/it]
 55%|█████▌    | 124137/225000 [252:58:30<185:20:56,  6.62s/it]
 55%|█████▌    | 124138/225000 [252:58:34<163:51:15,  5.85s/it]
 55%|█████▌    | 124139/225000 [252:58:39<152:17:32,  5.44s/it]
 55%|█████▌    | 124140/225000 [252:58:43<145:21:24,  5.19s/it]
 55%|█████▌    | 124141/225000 [252:58:47<134:49:24,  4.81s/it]
 55%|█████▌    | 124142/225000 [252:58:52<133:09:28,  4.75s/it]
 55%|█████▌    | 124143/225000 [252:59:02<174:25:36,  6.23s/it]
 55%|█████▌    | 124144/225000 [252:59:09<182:37:09,  6.52s/it]
 55%|█████▌    | 124145/225000 [252:59:12<158:02:24,  5.64s/it]
 55%|█████▌    | 124146/225000 [252:59:16<141:31:18,  5.05s/it]
 55%|█████▌    | 124147/225000 [252:59:26<180:02:57,  6.43s/it]
 55%|█████▌    | 124148/225000 [252:59:30<160:07:44,  5.72s/it]
 55%|█████▌    | 124149/225000 [252:59:34<147:01:27,  5.25s/it]
 55%|█████▌    | 124150/225000 [252:59:38<138:02:22,  4.93s/it]
                                                               
{'loss': '0.8485', 'grad_norm': '1.092', 'learning_rate': '2.259e-05', 'epoch': '0.5518', 'num_input_tokens_seen': 421411232, 'train_runtime': '9.108e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124150/225000 [252:59:38<138:02:22,  4.93s/it]
 55%|█████▌    | 124151/225000 [252:59:46<159:01:13,  5.68s/it]
 55%|█████▌    | 124152/225000 [252:59:54<185:24:24,  6.62s/it]
 55%|█████▌    | 124153/225000 [252:59:58<160:02:34,  5.71s/it]
 55%|█████▌    | 124154/225000 [253:00:02<143:54:56,  5.14s/it]
 55%|█████▌    | 124155/225000 [253:00:06<135:42:59,  4.84s/it]
 55%|█████▌    | 124156/225000 [253:00:14<166:14:55,  5.93s/it]
 55%|█████▌    | 124157/225000 [253:00:21<171:26:37,  6.12s/it]
 55%|█████▌    | 124158/225000 [253:00:26<162:53:35,  5.82s/it]
 55%|█████▌    | 124159/225000 [253:00:35<190:17:42,  6.79s/it]
 55%|█████▌    | 124160/225000 [253:00:40<171:42:16,  6.13s/it]
 55%|█████▌    | 124161/225000 [253:00:45<160:44:21,  5.74s/it]
 55%|█████▌    | 124162/225000 [253:00:49<149:58:45,  5.35s/it]
 55%|█████▌    | 124163/225000 [253:00:54<144:08:59,  5.15s/it]
 55%|█████▌    | 124164/225000 [253:00:58<136:30:00,  4.87s/it]
 55%|█████▌    | 124165/225000 [253:01:02<131:28:01,  4.69s/it]
 55%|█████▌    | 124166/225000 [253:01:14<191:06:01,  6.82s/it]
 55%|█████▌    | 124167/225000 [253:01:20<180:40:58,  6.45s/it]
 55%|█████▌    | 124168/225000 [253:01:24<166:12:49,  5.93s/it]
 55%|█████▌    | 124169/225000 [253:01:29<159:31:58,  5.70s/it]
 55%|█████▌    | 124170/225000 [253:01:36<167:06:22,  5.97s/it]
 55%|█████▌    | 124171/225000 [253:01:42<165:41:18,  5.92s/it]
 55%|█████▌    | 124172/225000 [253:01:46<154:45:07,  5.53s/it]
 55%|█████▌    | 124173/225000 [253:01:51<144:47:48,  5.17s/it]
 55%|█████▌    | 124174/225000 [253:01:55<140:44:05,  5.02s/it]
 55%|█████▌    | 124175/225000 [253:02:01<141:16:37,  5.04s/it]
 55%|█████▌    | 124176/225000 [253:02:13<201:10:38,  7.18s/it]
 55%|█████▌    | 124177/225000 [253:02:16<169:02:07,  6.04s/it]
 55%|█████▌    | 124178/225000 [253:02:24<187:49:17,  6.71s/it]
 55%|█████▌    | 124179/225000 [253:02:31<184:29:32,  6.59s/it]
 55%|█████▌    | 124180/225000 [253:02:35<161:44:53,  5.78s/it]
 55%|█████▌    | 124181/225000 [253:02:38<144:19:03,  5.15s/it]
 55%|█████▌    | 124182/225000 [253:02:45<159:10:00,  5.68s/it]
 55%|█████▌    | 124183/225000 [253:02:52<164:57:58,  5.89s/it]
 55%|█████▌    | 124184/225000 [253:02:58<172:20:55,  6.15s/it]
 55%|█████▌    | 124185/225000 [253:03:03<161:19:40,  5.76s/it]
 55%|█████▌    | 124186/225000 [253:03:07<144:16:17,  5.15s/it]
 55%|█████▌    | 124187/225000 [253:03:11<133:55:52,  4.78s/it]
 55%|█████▌    | 124188/225000 [253:03:15<130:44:16,  4.67s/it]
 55%|█████▌    | 124189/225000 [253:03:23<156:02:35,  5.57s/it]
 55%|█████▌    | 124190/225000 [253:03:29<162:44:51,  5.81s/it]
 55%|█████▌    | 124191/225000 [253:03:33<147:22:34,  5.26s/it]
 55%|█████▌    | 124192/225000 [253:03:37<136:33:37,  4.88s/it]
 55%|█████▌    | 124193/225000 [253:03:42<132:05:30,  4.72s/it]
 55%|█████▌    | 124194/225000 [253:03:53<191:00:10,  6.82s/it]
 55%|█████▌    | 124195/225000 [253:03:57<166:32:10,  5.95s/it]
 55%|█████▌    | 124196/225000 [253:04:01<147:25:19,  5.26s/it]
 55%|█████▌    | 124197/225000 [253:04:11<186:19:46,  6.65s/it]
 55%|█████▌    | 124198/225000 [253:04:15<166:55:51,  5.96s/it]
 55%|█████▌    | 124199/225000 [253:04:19<151:13:51,  5.40s/it]
 55%|█████▌    | 124200/225000 [253:04:24<144:17:20,  5.15s/it]
                                                               
{'loss': '0.8331', 'grad_norm': '1.042', 'learning_rate': '2.258e-05', 'epoch': '0.552', 'num_input_tokens_seen': 421540440, 'train_runtime': '9.111e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124200/225000 [253:04:24<144:17:20,  5.15s/it]
 55%|█████▌    | 124201/225000 [253:04:30<149:52:14,  5.35s/it]
 55%|█████▌    | 124202/225000 [253:04:40<190:14:03,  6.79s/it]
 55%|█████▌    | 124203/225000 [253:04:43<162:49:33,  5.82s/it]
 55%|█████▌    | 124204/225000 [253:04:53<194:40:43,  6.95s/it]
 55%|█████▌    | 124205/225000 [253:04:57<168:54:57,  6.03s/it]
 55%|█████▌    | 124206/225000 [253:05:01<153:20:26,  5.48s/it]
 55%|█████▌    | 124207/225000 [253:05:05<143:51:35,  5.14s/it]
 55%|█████▌    | 124208/225000 [253:05:09<131:20:31,  4.69s/it]
 55%|█████▌    | 124209/225000 [253:05:21<193:19:14,  6.90s/it]
 55%|█████▌    | 124210/225000 [253:05:25<165:01:48,  5.89s/it]
 55%|█████▌    | 124211/225000 [253:05:33<184:47:12,  6.60s/it]
 55%|█████▌    | 124212/225000 [253:05:37<168:24:22,  6.02s/it]
 55%|█████▌    | 124213/225000 [253:05:41<149:00:12,  5.32s/it]
 55%|█████▌    | 124214/225000 [253:05:45<137:13:33,  4.90s/it]
 55%|█████▌    | 124215/225000 [253:05:56<185:27:18,  6.62s/it]
 55%|█████▌    | 124216/225000 [253:05:59<154:48:05,  5.53s/it]
 55%|█████▌    | 124217/225000 [253:06:02<135:15:52,  4.83s/it]
 55%|█████▌    | 124218/225000 [253:06:06<128:43:06,  4.60s/it]
 55%|█████▌    | 124219/225000 [253:06:10<126:59:03,  4.54s/it]
 55%|█████▌    | 124220/225000 [253:06:15<128:14:27,  4.58s/it]
 55%|█████▌    | 124221/225000 [253:06:20<129:09:56,  4.61s/it]
 55%|█████▌    | 124222/225000 [253:06:24<127:15:07,  4.55s/it]
 55%|█████▌    | 124223/225000 [253:06:32<156:50:16,  5.60s/it]
 55%|█████▌    | 124224/225000 [253:06:43<202:14:02,  7.22s/it]
 55%|█████▌    | 124225/225000 [253:06:47<170:34:55,  6.09s/it]
 55%|█████▌    | 124226/225000 [253:06:54<183:59:48,  6.57s/it]
 55%|█████▌    | 124227/225000 [253:06:58<157:29:35,  5.63s/it]
 55%|█████▌    | 124228/225000 [253:07:05<174:15:23,  6.23s/it]
 55%|█████▌    | 124229/225000 [253:07:09<151:24:54,  5.41s/it]
 55%|█████▌    | 124230/225000 [253:07:14<148:25:59,  5.30s/it]
 55%|█████▌    | 124231/225000 [253:07:19<145:27:35,  5.20s/it]
 55%|█████▌    | 124232/225000 [253:07:24<148:28:25,  5.30s/it]
 55%|█████▌    | 124233/225000 [253:07:30<149:37:34,  5.35s/it]
 55%|█████▌    | 124234/225000 [253:07:35<148:40:46,  5.31s/it]
 55%|█████▌    | 124235/225000 [253:07:41<150:21:55,  5.37s/it]
 55%|█████▌    | 124236/225000 [253:07:46<149:25:06,  5.34s/it]
 55%|█████▌    | 124237/225000 [253:07:51<146:23:10,  5.23s/it]
 55%|█████▌    | 124238/225000 [253:07:56<143:07:36,  5.11s/it]
 55%|█████▌    | 124239/225000 [253:08:08<201:17:54,  7.19s/it]
 55%|█████▌    | 124240/225000 [253:08:14<196:05:12,  7.01s/it]
 55%|█████▌    | 124241/225000 [253:08:18<164:45:16,  5.89s/it]
 55%|█████▌    | 124242/225000 [253:08:26<182:40:18,  6.53s/it]
 55%|█████▌    | 124243/225000 [253:08:29<158:01:08,  5.65s/it]
 55%|█████▌    | 124244/225000 [253:08:37<175:16:04,  6.26s/it]
 55%|█████▌    | 124245/225000 [253:08:40<152:11:59,  5.44s/it]
 55%|█████▌    | 124246/225000 [253:08:44<139:17:29,  4.98s/it]
 55%|█████▌    | 124247/225000 [253:08:54<181:30:41,  6.49s/it]
 55%|█████▌    | 124248/225000 [253:09:01<180:08:13,  6.44s/it]
 55%|█████▌    | 124249/225000 [253:09:05<159:40:38,  5.71s/it]
 55%|█████▌    | 124250/225000 [253:09:09<147:54:15,  5.28s/it]
                                                               
{'loss': '0.8316', 'grad_norm': '1.003', 'learning_rate': '2.256e-05', 'epoch': '0.5522', 'num_input_tokens_seen': 421669256, 'train_runtime': '9.113e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124250/225000 [253:09:09<147:54:15,  5.28s/it]
 55%|█████▌    | 124251/225000 [253:09:14<144:59:59,  5.18s/it]
 55%|█████▌    | 124252/225000 [253:09:24<190:02:38,  6.79s/it]
 55%|█████▌    | 124253/225000 [253:09:28<162:02:03,  5.79s/it]
 55%|█████▌    | 124254/225000 [253:09:36<179:08:45,  6.40s/it]
 55%|█████▌    | 124255/225000 [253:09:40<158:08:46,  5.65s/it]
 55%|█████▌    | 124256/225000 [253:09:44<145:43:31,  5.21s/it]
 55%|█████▌    | 124257/225000 [253:09:48<136:22:31,  4.87s/it]
 55%|█████▌    | 124258/225000 [253:09:52<131:30:55,  4.70s/it]
 55%|█████▌    | 124259/225000 [253:09:57<134:44:37,  4.82s/it]
 55%|█████▌    | 124260/225000 [253:10:09<191:14:57,  6.83s/it]
 55%|█████▌    | 124261/225000 [253:10:12<161:00:31,  5.75s/it]
 55%|█████▌    | 124262/225000 [253:10:20<181:27:23,  6.48s/it]
 55%|█████▌    | 124263/225000 [253:10:24<159:08:42,  5.69s/it]
 55%|█████▌    | 124264/225000 [253:10:29<152:41:39,  5.46s/it]
 55%|█████▌    | 124265/225000 [253:10:34<146:47:49,  5.25s/it]
 55%|█████▌    | 124266/225000 [253:10:39<143:42:37,  5.14s/it]
 55%|█████▌    | 124267/225000 [253:10:43<139:51:42,  5.00s/it]
 55%|█████▌    | 124268/225000 [253:10:48<135:17:14,  4.83s/it]
 55%|█████▌    | 124269/225000 [253:10:58<177:35:50,  6.35s/it]
 55%|█████▌    | 124270/225000 [253:11:06<192:06:04,  6.87s/it]
 55%|█████▌    | 124271/225000 [253:11:10<173:03:11,  6.18s/it]
 55%|█████▌    | 124272/225000 [253:11:15<157:49:01,  5.64s/it]
 55%|█████▌    | 124273/225000 [253:11:19<148:59:46,  5.33s/it]
 55%|█████▌    | 124274/225000 [253:11:24<141:30:43,  5.06s/it]
 55%|█████▌    | 124275/225000 [253:11:29<139:37:15,  4.99s/it]
 55%|█████▌    | 124276/225000 [253:11:33<132:16:46,  4.73s/it]
 55%|█████▌    | 124277/225000 [253:11:37<128:53:48,  4.61s/it]
 55%|█████▌    | 124278/225000 [253:11:45<159:23:44,  5.70s/it]
 55%|█████▌    | 124279/225000 [253:11:54<183:18:33,  6.55s/it]
 55%|█████▌    | 124280/225000 [253:11:58<161:33:55,  5.77s/it]
 55%|█████▌    | 124281/225000 [253:12:02<146:17:43,  5.23s/it]
 55%|█████▌    | 124282/225000 [253:12:13<199:23:43,  7.13s/it]
 55%|█████▌    | 124283/225000 [253:12:17<169:16:34,  6.05s/it]
 55%|█████▌    | 124284/225000 [253:12:20<148:06:51,  5.29s/it]
 55%|█████▌    | 124285/225000 [253:12:30<182:10:50,  6.51s/it]
 55%|█████▌    | 124286/225000 [253:12:34<161:01:32,  5.76s/it]
 55%|█████▌    | 124287/225000 [253:12:38<150:13:05,  5.37s/it]
 55%|█████▌    | 124288/225000 [253:12:42<140:56:21,  5.04s/it]
 55%|█████▌    | 124289/225000 [253:12:53<184:42:41,  6.60s/it]
 55%|█████▌    | 124290/225000 [253:12:58<172:21:37,  6.16s/it]
 55%|█████▌    | 124291/225000 [253:13:01<150:53:38,  5.39s/it]
 55%|█████▌    | 124292/225000 [253:13:10<179:44:44,  6.43s/it]
 55%|█████▌    | 124293/225000 [253:13:13<153:16:33,  5.48s/it]
 55%|█████▌    | 124294/225000 [253:13:21<170:52:11,  6.11s/it]
 55%|█████▌    | 124295/225000 [253:13:25<154:13:29,  5.51s/it]
 55%|█████▌    | 124296/225000 [253:13:29<140:34:28,  5.03s/it]
 55%|█████▌    | 124297/225000 [253:13:33<129:16:12,  4.62s/it]
 55%|█████▌    | 124298/225000 [253:13:43<176:13:30,  6.30s/it]
 55%|█████▌    | 124299/225000 [253:13:47<154:21:11,  5.52s/it]
 55%|█████▌    | 124300/225000 [253:13:51<140:49:07,  5.03s/it]
                                                               
{'loss': '0.8497', 'grad_norm': '1.129', 'learning_rate': '2.254e-05', 'epoch': '0.5524', 'num_input_tokens_seen': 421796864, 'train_runtime': '9.116e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124300/225000 [253:13:51<140:49:07,  5.03s/it]
 55%|█████▌    | 124301/225000 [253:13:54<130:42:06,  4.67s/it]
 55%|█████▌    | 124302/225000 [253:13:59<129:07:33,  4.62s/it]
 55%|█████▌    | 124303/225000 [253:14:11<193:59:03,  6.94s/it]
 55%|█████▌    | 124304/225000 [253:14:17<180:29:48,  6.45s/it]
 55%|█████▌    | 124305/225000 [253:14:23<184:00:11,  6.58s/it]
 55%|█████▌    | 124306/225000 [253:14:28<168:24:32,  6.02s/it]
 55%|█████▌    | 124307/225000 [253:14:35<176:11:13,  6.30s/it]
 55%|█████▌    | 124308/225000 [253:14:40<161:45:04,  5.78s/it]
 55%|█████▌    | 124309/225000 [253:14:44<149:29:16,  5.34s/it]
 55%|█████▌    | 124310/225000 [253:14:48<139:15:09,  4.98s/it]
 55%|█████▌    | 124311/225000 [253:14:53<137:05:18,  4.90s/it]
 55%|█████▌    | 124312/225000 [253:14:57<134:58:11,  4.83s/it]
 55%|█████▌    | 124313/225000 [253:15:03<141:23:09,  5.06s/it]
 55%|█████▌    | 124314/225000 [253:15:16<207:12:10,  7.41s/it]
 55%|█████▌    | 124315/225000 [253:15:21<183:43:02,  6.57s/it]
 55%|█████▌    | 124316/225000 [253:15:26<169:52:02,  6.07s/it]
 55%|█████▌    | 124317/225000 [253:15:30<157:52:31,  5.64s/it]
 55%|█████▌    | 124318/225000 [253:15:35<150:43:47,  5.39s/it]
 55%|█████▌    | 124319/225000 [253:15:40<147:44:35,  5.28s/it]
 55%|█████▌    | 124320/225000 [253:15:45<147:22:41,  5.27s/it]
 55%|█████▌    | 124321/225000 [253:15:50<144:44:05,  5.18s/it]
 55%|█████▌    | 124322/225000 [253:15:55<142:32:16,  5.10s/it]
 55%|█████▌    | 124323/225000 [253:16:00<142:38:06,  5.10s/it]
 55%|█████▌    | 124324/225000 [253:16:06<148:07:41,  5.30s/it]
 55%|█████▌    | 124325/225000 [253:16:18<207:39:46,  7.43s/it]
 55%|█████▌    | 124326/225000 [253:16:24<194:12:23,  6.94s/it]
 55%|█████▌    | 124327/225000 [253:16:28<169:50:27,  6.07s/it]
 55%|█████▌    | 124328/225000 [253:16:33<155:33:16,  5.56s/it]
 55%|█████▌    | 124329/225000 [253:16:37<147:04:23,  5.26s/it]
 55%|█████▌    | 124330/225000 [253:16:42<142:35:52,  5.10s/it]
 55%|█████▌    | 124331/225000 [253:16:48<151:12:57,  5.41s/it]
 55%|█████▌    | 124332/225000 [253:16:59<195:21:40,  6.99s/it]
 55%|█████▌    | 124333/225000 [253:17:03<171:50:50,  6.15s/it]
 55%|█████▌    | 124334/225000 [253:17:07<155:54:46,  5.58s/it]
 55%|█████▌    | 124335/225000 [253:17:11<142:49:20,  5.11s/it]
 55%|█████▌    | 124336/225000 [253:17:17<146:14:21,  5.23s/it]
 55%|█████▌    | 124337/225000 [253:17:26<182:29:25,  6.53s/it]
 55%|█████▌    | 124338/225000 [253:17:30<161:20:05,  5.77s/it]
 55%|█████▌    | 124339/225000 [253:17:34<143:18:29,  5.13s/it]
 55%|█████▌    | 124340/225000 [253:17:45<192:01:33,  6.87s/it]
 55%|█████▌    | 124341/225000 [253:17:48<163:02:59,  5.83s/it]
 55%|█████▌    | 124342/225000 [253:17:52<149:44:13,  5.36s/it]
 55%|█████▌    | 124343/225000 [253:17:56<134:33:40,  4.81s/it]
 55%|█████▌    | 124344/225000 [253:18:01<135:37:24,  4.85s/it]
 55%|█████▌    | 124345/225000 [253:18:12<191:22:49,  6.84s/it]
 55%|█████▌    | 124346/225000 [253:18:16<163:30:52,  5.85s/it]
 55%|█████▌    | 124347/225000 [253:18:23<170:53:09,  6.11s/it]
 55%|█████▌    | 124348/225000 [253:18:26<149:16:41,  5.34s/it]
 55%|█████▌    | 124349/225000 [253:18:36<188:46:38,  6.75s/it]
 55%|█████▌    | 124350/225000 [253:18:40<167:29:48,  5.99s/it]
                                                               
{'loss': '0.8318', 'grad_norm': '1.024', 'learning_rate': '2.252e-05', 'epoch': '0.5527', 'num_input_tokens_seen': 421925936, 'train_runtime': '9.119e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124350/225000 [253:18:40<167:29:48,  5.99s/it]
 55%|█████▌    | 124351/225000 [253:18:44<151:26:10,  5.42s/it]
 55%|█████▌    | 124352/225000 [253:18:49<142:33:01,  5.10s/it]
 55%|█████▌    | 124353/225000 [253:18:54<142:29:27,  5.10s/it]
 55%|█████▌    | 124354/225000 [253:18:59<142:59:40,  5.11s/it]
 55%|█████▌    | 124355/225000 [253:19:04<142:36:16,  5.10s/it]
 55%|█████▌    | 124356/225000 [253:19:09<141:40:05,  5.07s/it]
 55%|█████▌    | 124357/225000 [253:19:22<203:01:05,  7.26s/it]
 55%|█████▌    | 124358/225000 [253:19:27<186:09:29,  6.66s/it]
 55%|█████▌    | 124359/225000 [253:19:31<167:29:51,  5.99s/it]
 55%|█████▌    | 124360/225000 [253:19:36<154:53:31,  5.54s/it]
 55%|█████▌    | 124361/225000 [253:19:40<142:01:49,  5.08s/it]
 55%|█████▌    | 124362/225000 [253:19:48<168:22:39,  6.02s/it]
 55%|█████▌    | 124363/225000 [253:19:56<182:53:15,  6.54s/it]
 55%|█████▌    | 124364/225000 [253:19:59<157:40:19,  5.64s/it]
 55%|█████▌    | 124365/225000 [253:20:08<185:34:43,  6.64s/it]
 55%|█████▌    | 124366/225000 [253:20:14<174:45:17,  6.25s/it]
 55%|█████▌    | 124367/225000 [253:20:18<161:12:26,  5.77s/it]
 55%|█████▌    | 124368/225000 [253:20:25<168:22:02,  6.02s/it]
 55%|█████▌    | 124369/225000 [253:20:29<150:45:40,  5.39s/it]
 55%|█████▌    | 124370/225000 [253:20:32<135:41:42,  4.85s/it]
 55%|█████▌    | 124371/225000 [253:20:36<128:23:11,  4.59s/it]
 55%|█████▌    | 124372/225000 [253:20:41<128:24:04,  4.59s/it]
 55%|█████▌    | 124373/225000 [253:20:53<187:06:51,  6.69s/it]
 55%|█████▌    | 124374/225000 [253:20:56<162:46:31,  5.82s/it]
 55%|█████▌    | 124375/225000 [253:21:05<186:59:15,  6.69s/it]
 55%|█████▌    | 124376/225000 [253:21:09<162:43:37,  5.82s/it]
 55%|█████▌    | 124377/225000 [253:21:16<170:24:09,  6.10s/it]
 55%|█████▌    | 124378/225000 [253:21:21<161:49:25,  5.79s/it]
 55%|█████▌    | 124379/225000 [253:21:25<150:08:12,  5.37s/it]
 55%|█████▌    | 124380/225000 [253:21:32<159:54:04,  5.72s/it]
 55%|█████▌    | 124381/225000 [253:21:35<143:25:40,  5.13s/it]
 55%|█████▌    | 124382/225000 [253:21:41<146:02:42,  5.23s/it]
 55%|█████▌    | 124383/225000 [253:21:47<155:29:54,  5.56s/it]
 55%|█████▌    | 124384/225000 [253:21:51<143:19:58,  5.13s/it]
 55%|█████▌    | 124385/225000 [253:21:56<140:54:07,  5.04s/it]
 55%|█████▌    | 124386/225000 [253:22:06<179:41:08,  6.43s/it]
 55%|█████▌    | 124387/225000 [253:22:09<155:24:38,  5.56s/it]
 55%|█████▌    | 124388/225000 [253:22:15<154:04:12,  5.51s/it]
 55%|█████▌    | 124389/225000 [253:22:23<174:43:47,  6.25s/it]
 55%|█████▌    | 124390/225000 [253:22:26<152:10:13,  5.44s/it]
 55%|█████▌    | 124391/225000 [253:22:30<138:21:26,  4.95s/it]
 55%|█████▌    | 124392/225000 [253:22:35<136:20:47,  4.88s/it]
 55%|█████▌    | 124393/225000 [253:22:40<138:58:19,  4.97s/it]
 55%|█████▌    | 124394/225000 [253:22:45<138:55:34,  4.97s/it]
 55%|█████▌    | 124395/225000 [253:22:50<137:25:20,  4.92s/it]
 55%|█████▌    | 124396/225000 [253:22:55<139:03:00,  4.98s/it]
 55%|█████▌    | 124397/225000 [253:23:01<148:14:52,  5.30s/it]
 55%|█████▌    | 124398/225000 [253:23:13<207:55:40,  7.44s/it]
 55%|█████▌    | 124399/225000 [253:23:18<184:47:07,  6.61s/it]
 55%|█████▌    | 124400/225000 [253:23:22<164:22:49,  5.88s/it]
                                                               
{'loss': '0.8251', 'grad_norm': '1.127', 'learning_rate': '2.25e-05', 'epoch': '0.5529', 'num_input_tokens_seen': 422055456, 'train_runtime': '9.122e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124400/225000 [253:23:22<164:22:49,  5.88s/it]
 55%|█████▌    | 124401/225000 [253:23:26<149:07:07,  5.34s/it]
 55%|█████▌    | 124402/225000 [253:23:30<139:51:57,  5.01s/it]
 55%|█████▌    | 124403/225000 [253:23:38<158:48:33,  5.68s/it]
 55%|█████▌    | 124404/225000 [253:23:49<202:43:40,  7.25s/it]
 55%|█████▌    | 124405/225000 [253:23:53<180:50:05,  6.47s/it]
 55%|█████▌    | 124406/225000 [253:23:58<168:29:19,  6.03s/it]
 55%|█████▌    | 124407/225000 [253:24:02<152:05:52,  5.44s/it]
 55%|█████▌    | 124408/225000 [253:24:07<144:35:25,  5.17s/it]
 55%|█████▌    | 124409/225000 [253:24:11<139:30:02,  4.99s/it]
 55%|█████▌    | 124410/225000 [253:24:22<186:03:46,  6.66s/it]
 55%|█████▌    | 124411/225000 [253:24:26<164:47:37,  5.90s/it]
 55%|█████▌    | 124412/225000 [253:24:30<149:52:36,  5.36s/it]
 55%|█████▌    | 124413/225000 [253:24:35<141:49:20,  5.08s/it]
 55%|█████▌    | 124414/225000 [253:24:40<140:24:47,  5.03s/it]
 55%|█████▌    | 124415/225000 [253:24:44<138:08:18,  4.94s/it]
 55%|█████▌    | 124416/225000 [253:24:49<133:30:25,  4.78s/it]
 55%|█████▌    | 124417/225000 [253:24:57<159:48:07,  5.72s/it]
 55%|█████▌    | 124418/225000 [253:25:07<200:23:10,  7.17s/it]
 55%|█████▌    | 124419/225000 [253:25:11<168:48:19,  6.04s/it]
 55%|█████▌    | 124420/225000 [253:25:19<187:22:34,  6.71s/it]
 55%|█████▌    | 124421/225000 [253:25:23<165:15:05,  5.91s/it]
 55%|█████▌    | 124422/225000 [253:25:28<161:07:42,  5.77s/it]
 55%|█████▌    | 124423/225000 [253:25:36<179:34:26,  6.43s/it]
 55%|█████▌    | 124424/225000 [253:25:40<153:04:27,  5.48s/it]
 55%|█████▌    | 124425/225000 [253:25:44<143:36:08,  5.14s/it]
 55%|█████▌    | 124426/225000 [253:25:48<135:58:55,  4.87s/it]
 55%|█████▌    | 124427/225000 [253:25:54<147:15:16,  5.27s/it]
 55%|█████▌    | 124428/225000 [253:26:05<195:54:30,  7.01s/it]
 55%|█████▌    | 124429/225000 [253:26:09<164:43:00,  5.90s/it]
 55%|█████▌    | 124430/225000 [253:26:17<182:19:23,  6.53s/it]
 55%|█████▌    | 124431/225000 [253:26:20<155:44:34,  5.58s/it]
 55%|█████▌    | 124432/225000 [253:26:26<160:10:20,  5.73s/it]
 55%|█████▌    | 124433/225000 [253:26:34<177:38:07,  6.36s/it]
 55%|█████▌    | 124434/225000 [253:26:37<150:46:05,  5.40s/it]
 55%|█████▌    | 124435/225000 [253:26:45<175:04:51,  6.27s/it]
 55%|█████▌    | 124436/225000 [253:26:49<152:17:22,  5.45s/it]
 55%|█████▌    | 124437/225000 [253:26:53<137:56:19,  4.94s/it]
 55%|█████▌    | 124438/225000 [253:27:04<191:12:40,  6.85s/it]
 55%|█████▌    | 124439/225000 [253:27:08<169:16:57,  6.06s/it]
 55%|█████▌    | 124440/225000 [253:27:12<147:08:17,  5.27s/it]
 55%|█████▌    | 124441/225000 [253:27:19<166:14:43,  5.95s/it]
 55%|█████▌    | 124442/225000 [253:27:24<158:49:26,  5.69s/it]
 55%|█████▌    | 124443/225000 [253:27:31<163:53:39,  5.87s/it]
 55%|█████▌    | 124444/225000 [253:27:35<155:54:07,  5.58s/it]
 55%|█████▌    | 124445/225000 [253:27:40<144:22:49,  5.17s/it]
 55%|█████▌    | 124446/225000 [253:27:46<150:02:18,  5.37s/it]
 55%|█████▌    | 124447/225000 [253:27:50<145:38:33,  5.21s/it]
 55%|█████▌    | 124448/225000 [253:27:56<144:54:55,  5.19s/it]
 55%|█████▌    | 124449/225000 [253:28:01<145:33:47,  5.21s/it]
 55%|█████▌    | 124450/225000 [253:28:06<149:01:20,  5.34s/it]
                                                               
{'loss': '0.8372', 'grad_norm': '0.9992', 'learning_rate': '2.249e-05', 'epoch': '0.5531', 'num_input_tokens_seen': 422185000, 'train_runtime': '9.125e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124450/225000 [253:28:06<149:01:20,  5.34s/it]
 55%|█████▌    | 124451/225000 [253:28:12<151:55:03,  5.44s/it]
 55%|█████▌    | 124452/225000 [253:28:17<150:24:50,  5.39s/it]
 55%|█████▌    | 124453/225000 [253:28:23<150:25:27,  5.39s/it]
 55%|█████▌    | 124454/225000 [253:28:28<148:20:51,  5.31s/it]
 55%|█████▌    | 124455/225000 [253:28:33<148:17:32,  5.31s/it]
 55%|█████▌    | 124456/225000 [253:28:38<141:19:54,  5.06s/it]
 55%|█████▌    | 124457/225000 [253:28:43<146:12:38,  5.24s/it]
 55%|█████▌    | 124458/225000 [253:28:49<146:53:25,  5.26s/it]
 55%|█████▌    | 124459/225000 [253:28:54<148:37:19,  5.32s/it]
 55%|█████▌    | 124460/225000 [253:29:06<202:48:13,  7.26s/it]
 55%|█████▌    | 124461/225000 [253:29:12<196:21:48,  7.03s/it]
 55%|█████▌    | 124462/225000 [253:29:16<172:10:12,  6.16s/it]
 55%|█████▌    | 124463/225000 [253:29:21<159:51:10,  5.72s/it]
 55%|█████▌    | 124464/225000 [253:29:26<152:20:46,  5.46s/it]
 55%|█████▌    | 124465/225000 [253:29:31<145:30:49,  5.21s/it]
 55%|█████▌    | 124466/225000 [253:29:35<141:02:53,  5.05s/it]
 55%|█████▌    | 124467/225000 [253:29:41<145:33:14,  5.21s/it]
 55%|█████▌    | 124468/225000 [253:29:53<205:52:22,  7.37s/it]
 55%|█████▌    | 124469/225000 [253:29:57<173:50:55,  6.23s/it]
 55%|█████▌    | 124470/225000 [253:30:04<178:51:43,  6.41s/it]
 55%|█████▌    | 124471/225000 [253:30:07<151:44:54,  5.43s/it]
 55%|█████▌    | 124472/225000 [253:30:11<137:22:58,  4.92s/it]
 55%|█████▌    | 124473/225000 [253:30:18<161:32:50,  5.79s/it]
 55%|█████▌    | 124474/225000 [253:30:24<158:16:40,  5.67s/it]
 55%|█████▌    | 124475/225000 [253:30:28<145:51:05,  5.22s/it]
 55%|█████▌    | 124476/225000 [253:30:32<137:31:00,  4.92s/it]
 55%|█████▌    | 124477/225000 [253:30:37<132:27:21,  4.74s/it]
 55%|█████▌    | 124478/225000 [253:30:41<128:14:24,  4.59s/it]
 55%|█████▌    | 124479/225000 [253:30:51<175:37:52,  6.29s/it]
 55%|█████▌    | 124480/225000 [253:30:54<150:21:37,  5.38s/it]
 55%|█████▌    | 124481/225000 [253:30:59<142:15:26,  5.09s/it]
 55%|█████▌    | 124482/225000 [253:31:07<171:27:08,  6.14s/it]
 55%|█████▌    | 124483/225000 [253:31:13<170:47:05,  6.12s/it]
 55%|█████▌    | 124484/225000 [253:31:18<159:23:16,  5.71s/it]
 55%|█████▌    | 124485/225000 [253:31:24<158:52:04,  5.69s/it]
 55%|█████▌    | 124486/225000 [253:31:30<165:02:47,  5.91s/it]
 55%|█████▌    | 124487/225000 [253:31:34<149:49:31,  5.37s/it]
 55%|█████▌    | 124488/225000 [253:31:38<139:43:21,  5.00s/it]
 55%|█████▌    | 124489/225000 [253:31:47<165:38:05,  5.93s/it]
 55%|█████▌    | 124490/225000 [253:31:52<165:01:00,  5.91s/it]
 55%|█████▌    | 124491/225000 [253:31:56<147:52:45,  5.30s/it]
 55%|█████▌    | 124492/225000 [253:32:00<136:58:09,  4.91s/it]
 55%|█████▌    | 124493/225000 [253:32:11<185:07:30,  6.63s/it]
 55%|█████▌    | 124494/225000 [253:32:14<156:32:24,  5.61s/it]
 55%|█████▌    | 124495/225000 [253:32:23<183:06:30,  6.56s/it]
 55%|█████▌    | 124496/225000 [253:32:26<156:46:36,  5.62s/it]
 55%|█████▌    | 124497/225000 [253:32:34<174:31:01,  6.25s/it]
 55%|█████▌    | 124498/225000 [253:32:38<154:36:29,  5.54s/it]
 55%|█████▌    | 124499/225000 [253:32:45<171:26:57,  6.14s/it]
 55%|█████▌    | 124500/225000 [253:32:50<153:52:53,  5.51s/it]
                                                               
{'loss': '0.8295', 'grad_norm': '1.05', 'learning_rate': '2.247e-05', 'epoch': '0.5533', 'num_input_tokens_seen': 422313912, 'train_runtime': '9.128e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124500/225000 [253:32:50<153:52:53,  5.51s/it]
 55%|█████▌    | 124501/225000 [253:32:58<178:33:09,  6.40s/it]
 55%|█████▌    | 124502/225000 [253:33:02<154:37:09,  5.54s/it]
 55%|█████▌    | 124503/225000 [253:33:11<183:41:15,  6.58s/it]
 55%|█████▌    | 124504/225000 [253:33:14<157:47:37,  5.65s/it]
 55%|█████▌    | 124505/225000 [253:33:24<197:50:23,  7.09s/it]
 55%|█████▌    | 124506/225000 [253:33:30<184:27:37,  6.61s/it]
 55%|█████▌    | 124507/225000 [253:33:35<167:36:44,  6.00s/it]
 55%|█████▌    | 124508/225000 [253:33:40<159:18:30,  5.71s/it]
 55%|█████▌    | 124509/225000 [253:33:44<152:41:47,  5.47s/it]
 55%|█████▌    | 124510/225000 [253:33:50<153:07:24,  5.49s/it]
 55%|█████▌    | 124511/225000 [253:33:55<149:59:32,  5.37s/it]
 55%|█████▌    | 124512/225000 [253:34:00<143:44:22,  5.15s/it]
 55%|█████▌    | 124513/225000 [253:34:05<141:55:55,  5.08s/it]
 55%|█████▌    | 124514/225000 [253:34:10<142:58:42,  5.12s/it]
 55%|█████▌    | 124515/225000 [253:34:19<174:13:41,  6.24s/it]
 55%|█████▌    | 124516/225000 [253:34:25<178:00:31,  6.38s/it]
 55%|█████▌    | 124517/225000 [253:34:30<164:58:41,  5.91s/it]
 55%|█████▌    | 124518/225000 [253:34:35<154:42:54,  5.54s/it]
 55%|█████▌    | 124519/225000 [253:34:40<148:03:57,  5.30s/it]
 55%|█████▌    | 124520/225000 [253:34:44<140:38:00,  5.04s/it]
 55%|█████▌    | 124521/225000 [253:34:53<171:27:28,  6.14s/it]
 55%|█████▌    | 124522/225000 [253:35:01<187:15:45,  6.71s/it]
 55%|█████▌    | 124523/225000 [253:35:05<161:52:10,  5.80s/it]
 55%|█████▌    | 124524/225000 [253:35:13<180:28:51,  6.47s/it]
 55%|█████▌    | 124525/225000 [253:35:16<155:11:08,  5.56s/it]
 55%|█████▌    | 124526/225000 [253:35:24<177:28:53,  6.36s/it]
 55%|█████▌    | 124527/225000 [253:35:28<151:52:57,  5.44s/it]
 55%|█████▌    | 124528/225000 [253:35:34<161:46:11,  5.80s/it]
 55%|█████▌    | 124529/225000 [253:35:38<144:00:22,  5.16s/it]
 55%|█████▌    | 124530/225000 [253:35:42<139:46:35,  5.01s/it]
 55%|█████▌    | 124531/225000 [253:35:49<154:28:40,  5.54s/it]
 55%|█████▌    | 124532/225000 [253:35:58<183:40:24,  6.58s/it]
 55%|█████▌    | 124533/225000 [253:36:04<174:08:28,  6.24s/it]
 55%|█████▌    | 124534/225000 [253:36:07<148:50:12,  5.33s/it]
 55%|█████▌    | 124535/225000 [253:36:10<133:33:05,  4.79s/it]
 55%|█████▌    | 124536/225000 [253:36:20<169:40:02,  6.08s/it]
 55%|█████▌    | 124537/225000 [253:36:23<146:43:42,  5.26s/it]
 55%|█████▌    | 124538/225000 [253:36:27<141:11:17,  5.06s/it]
 55%|█████▌    | 124539/225000 [253:36:37<179:42:44,  6.44s/it]
 55%|█████▌    | 124540/225000 [253:36:41<154:23:13,  5.53s/it]
 55%|█████▌    | 124541/225000 [253:36:45<148:36:54,  5.33s/it]
 55%|█████▌    | 124542/225000 [253:36:51<147:47:16,  5.30s/it]
 55%|█████▌    | 124543/225000 [253:36:56<144:30:08,  5.18s/it]
 55%|█████▌    | 124544/225000 [253:37:00<138:40:43,  4.97s/it]
 55%|█████▌    | 124545/225000 [253:37:05<138:29:43,  4.96s/it]
 55%|█████▌    | 124546/225000 [253:37:10<140:10:44,  5.02s/it]
 55%|█████▌    | 124547/225000 [253:37:15<142:54:24,  5.12s/it]
 55%|█████▌    | 124548/225000 [253:37:21<143:23:50,  5.14s/it]
 55%|█████▌    | 124549/225000 [253:37:26<145:12:57,  5.20s/it]
 55%|█████▌    | 124550/225000 [253:37:31<147:12:50,  5.28s/it]
                                                               
{'loss': '0.8353', 'grad_norm': '1.095', 'learning_rate': '2.245e-05', 'epoch': '0.5536', 'num_input_tokens_seen': 422442712, 'train_runtime': '9.131e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124550/225000 [253:37:31<147:12:50,  5.28s/it]
 55%|█████▌    | 124551/225000 [253:37:42<192:22:26,  6.89s/it]
 55%|█████▌    | 124552/225000 [253:37:49<189:14:23,  6.78s/it]
 55%|█████▌    | 124553/225000 [253:37:53<166:04:31,  5.95s/it]
 55%|█████▌    | 124554/225000 [253:37:57<151:12:51,  5.42s/it]
 55%|█████▌    | 124555/225000 [253:38:01<141:35:10,  5.07s/it]
 55%|█████▌    | 124556/225000 [253:38:11<185:47:45,  6.66s/it]
 55%|█████▌    | 124557/225000 [253:38:15<163:37:17,  5.86s/it]
 55%|█████▌    | 124558/225000 [253:38:19<146:57:23,  5.27s/it]
 55%|█████▌    | 124559/225000 [253:38:27<167:17:57,  6.00s/it]
 55%|█████▌    | 124560/225000 [253:38:36<194:34:03,  6.97s/it]
 55%|█████▌    | 124561/225000 [253:38:41<174:32:15,  6.26s/it]
 55%|█████▌    | 124562/225000 [253:38:46<161:12:06,  5.78s/it]
 55%|█████▌    | 124563/225000 [253:38:51<154:38:09,  5.54s/it]
 55%|█████▌    | 124564/225000 [253:38:55<145:25:42,  5.21s/it]
 55%|█████▌    | 124565/225000 [253:39:01<153:18:11,  5.50s/it]
 55%|█████▌    | 124566/225000 [253:39:11<189:18:48,  6.79s/it]
 55%|█████▌    | 124567/225000 [253:39:15<167:16:40,  6.00s/it]
 55%|█████▌    | 124568/225000 [253:39:19<151:41:08,  5.44s/it]
 55%|█████▌    | 124569/225000 [253:39:24<142:39:03,  5.11s/it]
 55%|█████▌    | 124570/225000 [253:39:34<190:04:09,  6.81s/it]
 55%|█████▌    | 124571/225000 [253:39:37<158:26:33,  5.68s/it]
 55%|█████▌    | 124572/225000 [253:39:45<175:52:24,  6.30s/it]
 55%|█████▌    | 124573/225000 [253:39:49<154:30:05,  5.54s/it]
 55%|█████▌    | 124574/225000 [253:39:58<183:42:30,  6.59s/it]
 55%|█████▌    | 124575/225000 [253:40:01<157:58:09,  5.66s/it]
 55%|█████▌    | 124576/225000 [253:40:09<173:23:06,  6.22s/it]
 55%|█████▌    | 124577/225000 [253:40:12<149:08:52,  5.35s/it]
 55%|█████▌    | 124578/225000 [253:40:22<185:16:08,  6.64s/it]
 55%|█████▌    | 124579/225000 [253:40:25<158:17:57,  5.67s/it]
 55%|█████▌    | 124580/225000 [253:40:33<176:31:58,  6.33s/it]
 55%|█████▌    | 124581/225000 [253:40:37<155:39:27,  5.58s/it]
 55%|█████▌    | 124582/225000 [253:40:41<141:28:07,  5.07s/it]
 55%|█████▌    | 124583/225000 [253:40:45<136:31:52,  4.89s/it]
 55%|█████▌    | 124584/225000 [253:40:50<130:06:26,  4.66s/it]
 55%|█████▌    | 124585/225000 [253:40:59<174:27:24,  6.25s/it]
 55%|█████▌    | 124586/225000 [253:41:06<179:14:09,  6.43s/it]
 55%|█████▌    | 124587/225000 [253:41:10<155:17:26,  5.57s/it]
 55%|█████▌    | 124588/225000 [253:41:17<169:34:51,  6.08s/it]
 55%|█████▌    | 124589/225000 [253:41:21<155:08:04,  5.56s/it]
 55%|█████▌    | 124590/225000 [253:41:28<159:31:56,  5.72s/it]
 55%|█████▌    | 124591/225000 [253:41:33<152:57:08,  5.48s/it]
 55%|█████▌    | 124592/225000 [253:41:40<170:40:51,  6.12s/it]
 55%|█████▌    | 124593/225000 [253:41:46<164:55:32,  5.91s/it]
 55%|█████▌    | 124594/225000 [253:41:50<155:38:27,  5.58s/it]
 55%|█████▌    | 124595/225000 [253:41:55<151:53:36,  5.45s/it]
 55%|█████▌    | 124596/225000 [253:42:01<148:30:28,  5.32s/it]
 55%|█████▌    | 124597/225000 [253:42:06<147:17:35,  5.28s/it]
 55%|█████▌    | 124598/225000 [253:42:10<142:46:10,  5.12s/it]
 55%|█████▌    | 124599/225000 [253:42:16<142:56:43,  5.13s/it]
 55%|█████▌    | 124600/225000 [253:42:21<144:30:44,  5.18s/it]
                                                               
{'loss': '0.8306', 'grad_norm': '0.9546', 'learning_rate': '2.243e-05', 'epoch': '0.5538', 'num_input_tokens_seen': 422572448, 'train_runtime': '9.133e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124600/225000 [253:42:21<144:30:44,  5.18s/it]
 55%|█████▌    | 124601/225000 [253:42:26<146:08:35,  5.24s/it]
 55%|█████▌    | 124602/225000 [253:42:31<144:28:58,  5.18s/it]
 55%|█████▌    | 124603/225000 [253:42:37<149:59:51,  5.38s/it]
 55%|█████▌    | 124604/225000 [253:42:50<209:26:20,  7.51s/it]
 55%|█████▌    | 124605/225000 [253:42:54<180:19:29,  6.47s/it]
 55%|█████▌    | 124606/225000 [253:42:57<155:49:09,  5.59s/it]
 55%|█████▌    | 124607/225000 [253:43:02<151:13:26,  5.42s/it]
 55%|█████▌    | 124608/225000 [253:43:07<142:44:47,  5.12s/it]
 55%|█████▌    | 124609/225000 [253:43:18<193:13:30,  6.93s/it]
 55%|█████▌    | 124610/225000 [253:43:22<169:03:30,  6.06s/it]
 55%|█████▌    | 124611/225000 [253:43:28<171:16:55,  6.14s/it]
 55%|█████▌    | 124612/225000 [253:43:35<179:45:30,  6.45s/it]
 55%|█████▌    | 124613/225000 [253:43:39<156:14:22,  5.60s/it]
 55%|█████▌    | 124614/225000 [253:43:43<140:05:19,  5.02s/it]
 55%|█████▌    | 124615/225000 [253:43:47<132:00:24,  4.73s/it]
 55%|█████▌    | 124616/225000 [253:43:58<184:12:56,  6.61s/it]
 55%|█████▌    | 124617/225000 [253:44:02<166:02:28,  5.95s/it]
 55%|█████▌    | 124618/225000 [253:44:06<149:49:38,  5.37s/it]
 55%|█████▌    | 124619/225000 [253:44:10<140:02:32,  5.02s/it]
 55%|█████▌    | 124620/225000 [253:44:15<140:19:07,  5.03s/it]
 55%|█████▌    | 124621/225000 [253:44:27<198:58:07,  7.14s/it]
 55%|█████▌    | 124622/225000 [253:44:31<165:44:21,  5.94s/it]
 55%|█████▌    | 124623/225000 [253:44:34<145:59:19,  5.24s/it]
 55%|█████▌    | 124624/225000 [253:44:43<174:17:11,  6.25s/it]
 55%|█████▌    | 124625/225000 [253:44:48<166:20:58,  5.97s/it]
 55%|█████▌    | 124626/225000 [253:44:53<154:39:35,  5.55s/it]
 55%|█████▌    | 124627/225000 [253:44:57<141:42:27,  5.08s/it]
 55%|█████▌    | 124628/225000 [253:45:01<135:13:15,  4.85s/it]
 55%|█████▌    | 124629/225000 [253:45:06<132:31:51,  4.75s/it]
 55%|█████▌    | 124630/225000 [253:45:11<142:15:53,  5.10s/it]
 55%|█████▌    | 124631/225000 [253:45:23<193:07:43,  6.93s/it]
 55%|█████▌    | 124632/225000 [253:45:26<163:27:58,  5.86s/it]
 55%|█████▌    | 124633/225000 [253:45:35<189:22:52,  6.79s/it]
 55%|█████▌    | 124634/225000 [253:45:38<161:02:46,  5.78s/it]
 55%|█████▌    | 124635/225000 [253:45:46<172:47:54,  6.20s/it]
 55%|█████▌    | 124636/225000 [253:45:50<154:17:33,  5.53s/it]
 55%|█████▌    | 124637/225000 [253:45:54<141:21:56,  5.07s/it]
 55%|█████▌    | 124638/225000 [253:45:59<145:40:02,  5.23s/it]
 55%|█████▌    | 124639/225000 [253:46:08<173:37:00,  6.23s/it]
 55%|█████▌    | 124640/225000 [253:46:11<151:29:46,  5.43s/it]
 55%|█████▌    | 124641/225000 [253:46:18<165:50:53,  5.95s/it]
 55%|█████▌    | 124642/225000 [253:46:24<165:00:26,  5.92s/it]
 55%|█████▌    | 124643/225000 [253:46:28<148:08:24,  5.31s/it]
 55%|█████▌    | 124644/225000 [253:46:32<138:22:34,  4.96s/it]
 55%|█████▌    | 124645/225000 [253:46:37<132:08:32,  4.74s/it]
 55%|█████▌    | 124646/225000 [253:46:41<127:51:03,  4.59s/it]
 55%|█████▌    | 124647/225000 [253:46:45<128:31:46,  4.61s/it]
 55%|█████▌    | 124648/225000 [253:46:57<189:55:18,  6.81s/it]
 55%|█████▌    | 124649/225000 [253:47:02<168:05:03,  6.03s/it]
 55%|█████▌    | 124650/225000 [253:47:06<152:33:23,  5.47s/it]
                                                               
{'loss': '0.827', 'grad_norm': '1.055', 'learning_rate': '2.241e-05', 'epoch': '0.554', 'num_input_tokens_seen': 422699680, 'train_runtime': '9.136e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124650/225000 [253:47:06<152:33:23,  5.47s/it]
 55%|█████▌    | 124651/225000 [253:47:14<173:48:44,  6.24s/it]
 55%|█████▌    | 124652/225000 [253:47:21<186:16:13,  6.68s/it]
 55%|█████▌    | 124653/225000 [253:47:25<156:48:28,  5.63s/it]
 55%|█████▌    | 124654/225000 [253:47:29<144:44:04,  5.19s/it]
 55%|█████▌    | 124655/225000 [253:47:39<184:13:44,  6.61s/it]
 55%|█████▌    | 124656/225000 [253:47:42<160:13:03,  5.75s/it]
 55%|█████▌    | 124657/225000 [253:47:46<142:16:32,  5.10s/it]
 55%|█████▌    | 124658/225000 [253:47:55<176:34:42,  6.34s/it]
 55%|█████▌    | 124659/225000 [253:47:59<154:08:44,  5.53s/it]
 55%|█████▌    | 124660/225000 [253:48:06<168:38:46,  6.05s/it]
 55%|█████▌    | 124661/225000 [253:48:10<148:39:14,  5.33s/it]
 55%|█████▌    | 124662/225000 [253:48:17<165:49:17,  5.95s/it]
 55%|█████▌    | 124663/225000 [253:48:22<155:13:56,  5.57s/it]
 55%|█████▌    | 124664/225000 [253:48:25<138:02:25,  4.95s/it]
 55%|█████▌    | 124665/225000 [253:48:30<133:16:46,  4.78s/it]
 55%|█████▌    | 124666/225000 [253:48:34<132:20:29,  4.75s/it]
 55%|█████▌    | 124667/225000 [253:48:39<132:26:59,  4.75s/it]
 55%|█████▌    | 124668/225000 [253:48:44<134:38:36,  4.83s/it]
 55%|█████▌    | 124669/225000 [253:48:57<199:55:08,  7.17s/it]
 55%|█████▌    | 124670/225000 [253:49:02<186:35:22,  6.70s/it]
 55%|█████▌    | 124671/225000 [253:49:06<160:17:56,  5.75s/it]
 55%|█████▌    | 124672/225000 [253:49:15<185:35:30,  6.66s/it]
 55%|█████▌    | 124673/225000 [253:49:18<157:01:36,  5.63s/it]
 55%|█████▌    | 124674/225000 [253:49:27<185:22:41,  6.65s/it]
 55%|█████▌    | 124675/225000 [253:49:31<165:24:42,  5.94s/it]
 55%|█████▌    | 124676/225000 [253:49:35<148:41:03,  5.34s/it]
 55%|█████▌    | 124677/225000 [253:49:39<138:02:50,  4.95s/it]
 55%|█████▌    | 124678/225000 [253:49:48<169:21:01,  6.08s/it]
 55%|█████▌    | 124679/225000 [253:49:55<178:28:04,  6.40s/it]
 55%|█████▌    | 124680/225000 [253:49:59<154:36:27,  5.55s/it]
 55%|█████▌    | 124681/225000 [253:50:06<166:15:19,  5.97s/it]
 55%|█████▌    | 124682/225000 [253:50:13<174:08:37,  6.25s/it]
 55%|█████▌    | 124683/225000 [253:50:17<155:13:40,  5.57s/it]
 55%|█████▌    | 124684/225000 [253:50:21<146:03:40,  5.24s/it]
 55%|█████▌    | 124685/225000 [253:50:25<137:01:18,  4.92s/it]
 55%|█████▌    | 124686/225000 [253:50:38<200:27:30,  7.19s/it]
 55%|█████▌    | 124687/225000 [253:50:42<178:39:09,  6.41s/it]
 55%|█████▌    | 124688/225000 [253:50:49<180:05:10,  6.46s/it]
 55%|█████▌    | 124689/225000 [253:50:53<159:27:36,  5.72s/it]
 55%|█████▌    | 124690/225000 [253:50:56<140:33:30,  5.04s/it]
 55%|█████▌    | 124691/225000 [253:51:06<175:02:17,  6.28s/it]
 55%|█████▌    | 124692/225000 [253:51:09<153:27:49,  5.51s/it]
 55%|█████▌    | 124693/225000 [253:51:14<144:03:47,  5.17s/it]
 55%|█████▌    | 124694/225000 [253:51:23<182:31:37,  6.55s/it]
 55%|█████▌    | 124695/225000 [253:51:27<155:22:00,  5.58s/it]
 55%|█████▌    | 124696/225000 [253:51:30<139:14:03,  5.00s/it]
 55%|█████▌    | 124697/225000 [253:51:34<127:36:12,  4.58s/it]
 55%|█████▌    | 124698/225000 [253:51:45<178:51:06,  6.42s/it]
 55%|█████▌    | 124699/225000 [253:51:49<165:09:11,  5.93s/it]
 55%|█████▌    | 124700/225000 [253:51:54<156:02:13,  5.60s/it]
                                                               
{'loss': '0.8435', 'grad_norm': '0.9765', 'learning_rate': '2.24e-05', 'epoch': '0.5542', 'num_input_tokens_seen': 422828616, 'train_runtime': '9.139e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124700/225000 [253:51:54<156:02:13,  5.60s/it]
 55%|█████▌    | 124701/225000 [253:51:59<149:05:34,  5.35s/it]
 55%|█████▌    | 124702/225000 [253:52:03<139:07:16,  4.99s/it]
 55%|█████▌    | 124703/225000 [253:52:08<136:05:12,  4.88s/it]
 55%|█████▌    | 124704/225000 [253:52:13<135:05:25,  4.85s/it]
 55%|█████▌    | 124705/225000 [253:52:20<154:44:10,  5.55s/it]
 55%|█████▌    | 124706/225000 [253:52:32<206:35:19,  7.42s/it]
 55%|█████▌    | 124707/225000 [253:52:35<171:20:36,  6.15s/it]
 55%|█████▌    | 124708/225000 [253:52:43<186:37:15,  6.70s/it]
 55%|█████▌    | 124709/225000 [253:52:46<161:00:33,  5.78s/it]
 55%|█████▌    | 124710/225000 [253:52:52<155:54:36,  5.60s/it]
 55%|█████▌    | 124711/225000 [253:52:58<162:42:13,  5.84s/it]
 55%|█████▌    | 124712/225000 [253:53:04<162:10:51,  5.82s/it]
 55%|█████▌    | 124713/225000 [253:53:09<155:45:57,  5.59s/it]
 55%|█████▌    | 124714/225000 [253:53:13<144:13:01,  5.18s/it]
 55%|█████▌    | 124715/225000 [253:53:18<146:14:22,  5.25s/it]
 55%|█████▌    | 124716/225000 [253:53:27<176:38:37,  6.34s/it]
 55%|█████▌    | 124717/225000 [253:53:31<157:59:38,  5.67s/it]
 55%|█████▌    | 124718/225000 [253:53:36<147:13:47,  5.29s/it]
 55%|█████▌    | 124719/225000 [253:53:40<141:12:35,  5.07s/it]
 55%|█████▌    | 124720/225000 [253:53:50<178:38:21,  6.41s/it]
 55%|█████▌    | 124721/225000 [253:53:53<152:04:55,  5.46s/it]
 55%|█████▌    | 124722/225000 [253:53:58<143:34:55,  5.15s/it]
 55%|█████▌    | 124723/225000 [253:54:05<158:24:44,  5.69s/it]
 55%|█████▌    | 124724/225000 [253:54:08<143:51:27,  5.16s/it]
 55%|█████▌    | 124725/225000 [253:54:13<138:41:17,  4.98s/it]
 55%|█████▌    | 124726/225000 [253:54:18<134:59:19,  4.85s/it]
 55%|█████▌    | 124727/225000 [253:54:26<162:36:00,  5.84s/it]
 55%|█████▌    | 124728/225000 [253:54:33<172:48:00,  6.20s/it]
 55%|█████▌    | 124729/225000 [253:54:36<148:43:08,  5.34s/it]
 55%|█████▌    | 124730/225000 [253:54:44<169:39:02,  6.09s/it]
 55%|█████▌    | 124731/225000 [253:54:49<160:06:39,  5.75s/it]
 55%|█████▌    | 124732/225000 [253:54:54<152:17:32,  5.47s/it]
 55%|█████▌    | 124733/225000 [253:54:58<144:46:06,  5.20s/it]
 55%|█████▌    | 124734/225000 [253:55:03<139:25:09,  5.01s/it]
 55%|█████▌    | 124735/225000 [253:55:08<137:01:14,  4.92s/it]
 55%|█████▌    | 124736/225000 [253:55:12<135:13:58,  4.86s/it]
 55%|█████▌    | 124737/225000 [253:55:17<136:59:03,  4.92s/it]
 55%|█████▌    | 124738/225000 [253:55:22<138:01:27,  4.96s/it]
 55%|█████▌    | 124739/225000 [253:55:27<138:33:33,  4.98s/it]
 55%|█████▌    | 124740/225000 [253:55:39<197:51:16,  7.10s/it]
 55%|█████▌    | 124741/225000 [253:55:46<195:26:32,  7.02s/it]
 55%|█████▌    | 124742/225000 [253:55:50<164:16:48,  5.90s/it]
 55%|█████▌    | 124743/225000 [253:55:58<188:05:48,  6.75s/it]
 55%|█████▌    | 124744/225000 [253:56:02<163:49:17,  5.88s/it]
 55%|█████▌    | 124745/225000 [253:56:06<147:16:27,  5.29s/it]
 55%|█████▌    | 124746/225000 [253:56:17<192:17:07,  6.90s/it]
 55%|█████▌    | 124747/225000 [253:56:20<162:30:33,  5.84s/it]
 55%|█████▌    | 124748/225000 [253:56:24<144:28:32,  5.19s/it]
 55%|█████▌    | 124749/225000 [253:56:29<147:18:50,  5.29s/it]
 55%|█████▌    | 124750/225000 [253:56:37<171:53:32,  6.17s/it]
                                                               
{'loss': '0.8302', 'grad_norm': '0.9444', 'learning_rate': '2.238e-05', 'epoch': '0.5544', 'num_input_tokens_seen': 422955992, 'train_runtime': '9.142e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124750/225000 [253:56:37<171:53:32,  6.17s/it]
 55%|█████▌    | 124751/225000 [253:56:43<168:19:21,  6.04s/it]
 55%|█████▌    | 124752/225000 [253:56:50<176:05:30,  6.32s/it]
 55%|█████▌    | 124753/225000 [253:56:53<149:18:38,  5.36s/it]
 55%|█████▌    | 124754/225000 [253:57:02<172:58:01,  6.21s/it]
 55%|█████▌    | 124755/225000 [253:57:05<151:00:53,  5.42s/it]
 55%|█████▌    | 124756/225000 [253:57:10<146:52:32,  5.27s/it]
 55%|█████▌    | 124757/225000 [253:57:17<163:41:38,  5.88s/it]
 55%|█████▌    | 124758/225000 [253:57:21<145:19:52,  5.22s/it]
 55%|█████▌    | 124759/225000 [253:57:26<143:46:55,  5.16s/it]
 55%|█████▌    | 124760/225000 [253:57:31<144:01:40,  5.17s/it]
 55%|█████▌    | 124761/225000 [253:57:36<144:31:12,  5.19s/it]
 55%|█████▌    | 124762/225000 [253:57:42<143:13:07,  5.14s/it]
 55%|█████▌    | 124763/225000 [253:57:46<141:31:11,  5.08s/it]
 55%|█████▌    | 124764/225000 [253:57:51<137:11:11,  4.93s/it]
 55%|█████▌    | 124765/225000 [253:57:56<138:28:54,  4.97s/it]
 55%|█████▌    | 124766/225000 [253:58:01<141:54:45,  5.10s/it]
 55%|█████▌    | 124767/225000 [253:58:06<139:56:58,  5.03s/it]
 55%|█████▌    | 124768/225000 [253:58:14<159:08:02,  5.72s/it]
 55%|█████▌    | 124769/225000 [253:58:25<207:09:31,  7.44s/it]
 55%|█████▌    | 124770/225000 [253:58:29<179:34:35,  6.45s/it]
 55%|█████▌    | 124771/225000 [253:58:33<160:13:41,  5.76s/it]
 55%|█████▌    | 124772/225000 [253:58:39<155:52:59,  5.60s/it]
 55%|█████▌    | 124773/225000 [253:58:44<154:37:21,  5.55s/it]
 55%|█████▌    | 124774/225000 [253:58:49<149:20:56,  5.36s/it]
 55%|█████▌    | 124775/225000 [253:58:54<148:52:48,  5.35s/it]
 55%|█████▌    | 124776/225000 [253:59:00<148:30:40,  5.33s/it]
 55%|█████▌    | 124777/225000 [253:59:04<144:03:49,  5.17s/it]
 55%|█████▌    | 124778/225000 [253:59:09<140:24:08,  5.04s/it]
 55%|█████▌    | 124779/225000 [253:59:14<139:40:27,  5.02s/it]
 55%|█████▌    | 124780/225000 [253:59:19<139:16:19,  5.00s/it]
 55%|█████▌    | 124781/225000 [253:59:24<139:06:34,  5.00s/it]
 55%|█████▌    | 124782/225000 [253:59:33<172:45:54,  6.21s/it]
 55%|█████▌    | 124783/225000 [253:59:45<221:57:31,  7.97s/it]
 55%|█████▌    | 124784/225000 [253:59:49<186:38:51,  6.70s/it]
 55%|█████▌    | 124785/225000 [253:59:56<190:55:24,  6.86s/it]
 55%|█████▌    | 124786/225000 [254:00:00<165:20:14,  5.94s/it]
 55%|█████▌    | 124787/225000 [254:00:03<145:07:39,  5.21s/it]
 55%|█████▌    | 124788/225000 [254:00:15<194:57:44,  7.00s/it]
 55%|█████▌    | 124789/225000 [254:00:18<162:15:14,  5.83s/it]
 55%|█████▌    | 124790/225000 [254:00:22<152:07:55,  5.47s/it]
 55%|█████▌    | 124791/225000 [254:00:28<150:33:44,  5.41s/it]
 55%|█████▌    | 124792/225000 [254:00:33<152:13:06,  5.47s/it]
 55%|█████▌    | 124793/225000 [254:00:40<166:58:25,  6.00s/it]
 55%|█████▌    | 124794/225000 [254:00:46<161:49:29,  5.81s/it]
 55%|█████▌    | 124795/225000 [254:00:51<157:15:31,  5.65s/it]
 55%|█████▌    | 124796/225000 [254:00:56<151:44:47,  5.45s/it]
 55%|█████▌    | 124797/225000 [254:01:01<147:26:44,  5.30s/it]
 55%|█████▌    | 124798/225000 [254:01:07<149:18:46,  5.36s/it]
 55%|█████▌    | 124799/225000 [254:01:12<146:12:32,  5.25s/it]
 55%|█████▌    | 124800/225000 [254:01:17<146:02:18,  5.25s/it]
                                                               
{'loss': '0.83', 'grad_norm': '1.073', 'learning_rate': '2.236e-05', 'epoch': '0.5547', 'num_input_tokens_seen': 423085480, 'train_runtime': '9.145e+05', 'train_tokens_per_second': '462.7'}

 55%|█████▌    | 124800/225000 [254:01:17<146:02:18,  5.25s/it]
 55%|█████▌    | 124801/225000 [254:01:23<151:03:10,  5.43s/it]
 55%|█████▌    | 124802/225000 [254:01:28<149:26:50,  5.37s/it]
 55%|█████▌    | 124803/225000 [254:01:33<147:09:43,  5.29s/it]
 55%|█████▌    | 124804/225000 [254:01:38<147:16:55,  5.29s/it]
 55%|█████▌    | 124805/225000 [254:01:43<145:51:04,  5.24s/it]
 55%|█████▌    | 124806/225000 [254:01:56<204:34:09,  7.35s/it]
 55%|█████▌    | 124807/225000 [254:02:01<189:45:47,  6.82s/it]
 55%|█████▌    | 124808/225000 [254:02:06<169:22:37,  6.09s/it]
 55%|█████▌    | 124809/225000 [254:02:10<157:56:20,  5.67s/it]
 55%|█████▌    | 124810/225000 [254:02:15<148:42:57,  5.34s/it]
 55%|█████▌    | 124811/225000 [254:02:20<148:11:50,  5.33s/it]
 55%|█████▌    | 124812/225000 [254:02:26<151:30:18,  5.44s/it]
 55%|█████▌    | 124813/225000 [254:02:31<144:21:41,  5.19s/it]
 55%|█████▌    | 124814/225000 [254:02:42<196:44:22,  7.07s/it]
 55%|█████▌    | 124815/225000 [254:02:45<164:27:27,  5.91s/it]
 55%|█████▌    | 124816/225000 [254:02:53<183:44:06,  6.60s/it]
 55%|█████▌    | 124817/225000 [254:02:57<159:04:19,  5.72s/it]
 55%|█████▌    | 124818/225000 [254:03:05<180:53:23,  6.50s/it]
 55%|█████▌    | 124819/225000 [254:03:10<162:33:29,  5.84s/it]
 55%|█████▌    | 124820/225000 [254:03:14<151:22:14,  5.44s/it]
 55%|█████▌    | 124821/225000 [254:03:22<168:15:13,  6.05s/it]
 55%|█████▌    | 124822/225000 [254:03:25<148:12:10,  5.33s/it]
 55%|█████▌    | 124823/225000 [254:03:30<140:50:06,  5.06s/it]
 55%|█████▌    | 124824/225000 [254:03:35<142:48:33,  5.13s/it]
 55%|█████▌    | 124825/225000 [254:03:40<139:34:42,  5.02s/it]
 55%|█████▌    | 124826/225000 [254:03:44<137:13:58,  4.93s/it]
 55%|█████▌    | 124827/225000 [254:03:50<142:13:27,  5.11s/it]
 55%|█████▌    | 124828/225000 [254:03:55<141:21:41,  5.08s/it]
 55%|█████▌    | 124829/225000 [254:04:00<142:37:56,  5.13s/it]
 55%|█████▌    | 124830/225000 [254:04:05<140:55:08,  5.06s/it]
 55%|█████▌    | 124831/225000 [254:04:10<141:37:58,  5.09s/it]
 55%|█████▌    | 124832/225000 [254:04:15<141:27:38,  5.08s/it]
 55%|█████▌    | 124833/225000 [254:04:21<142:57:30,  5.14s/it]
 55%|█████▌    | 124834/225000 [254:04:33<201:09:45,  7.23s/it]
 55%|█████▌    | 124835/225000 [254:04:40<200:29:16,  7.21s/it]
 55%|█████▌    | 124836/225000 [254:04:43<167:50:36,  6.03s/it]
 55%|█████▌    | 124837/225000 [254:04:48<153:40:32,  5.52s/it]
 55%|█████▌    | 124838/225000 [254:04:52<142:35:32,  5.13s/it]
 55%|█████▌    | 124839/225000 [254:04:56<135:58:03,  4.89s/it]
 55%|█████▌    | 124840/225000 [254:05:07<186:32:23,  6.70s/it]
 55%|█████▌    | 124841/225000 [254:05:14<192:14:53,  6.91s/it]
 55%|█████▌    | 124842/225000 [254:05:18<165:58:59,  5.97s/it]
 55%|█████▌    | 124843/225000 [254:05:23<156:14:45,  5.62s/it]
 55%|█████▌    | 124844/225000 [254:05:32<181:35:12,  6.53s/it]
 55%|█████▌    | 124845/225000 [254:05:35<154:52:13,  5.57s/it]
 55%|█████▌    | 124846/225000 [254:05:42<169:06:49,  6.08s/it]
 55%|█████▌    | 124847/225000 [254:05:46<149:00:44,  5.36s/it]
 55%|█████▌    | 124848/225000 [254:05:52<157:28:21,  5.66s/it]
 55%|█████▌    | 124849/225000 [254:05:57<148:04:19,  5.32s/it]
 55%|█████▌    | 124850/225000 [254:06:02<143:51:01,  5.17s/it]
                                                               
{'loss': '0.8238', 'grad_norm': '1.026', 'learning_rate': '2.234e-05', 'epoch': '0.5549', 'num_input_tokens_seen': 423214048, 'train_runtime': '9.148e+05', 'train_tokens_per_second': '462.6'}

 55%|█████▌    | 124850/225000 [254:06:02<143:51:01,  5.17s/it]
 55%|█████▌    | 124851/225000 [254:06:06<140:33:57,  5.05s/it]
 55%|█████▌    | 124852/225000 [254:06:11<140:15:06,  5.04s/it]
 55%|█████▌    | 124853/225000 [254:06:16<138:52:30,  4.99s/it]
 55%|█████▌    | 124854/225000 [254:06:21<134:35:56,  4.84s/it]
 55%|█████▌    | 124855/225000 [254:06:29<162:09:12,  5.83s/it]
 55%|█████▌    | 124856/225000 [254:06:38<191:48:46,  6.90s/it]
 55%|█████▌    | 124857/225000 [254:06:42<161:38:33,  5.81s/it]
 55%|█████▌    | 124858/225000 [254:06:45<145:25:50,  5.23s/it]
 55%|█████▌    | 124859/225000 [254:06:50<139:31:49,  5.02s/it]
 55%|█████▌    | 124860/225000 [254:07:01<189:48:05,  6.82s/it]
 55%|█████▌    | 124861/225000 [254:07:08<194:04:11,  6.98s/it]
 55%|█████▌    | 124862/225000 [254:07:12<162:39:47,  5.85s/it]
 55%|█████▌    | 124863/225000 [254:07:19<175:21:31,  6.30s/it]
 55%|█████▌    | 124864/225000 [254:07:23<160:15:25,  5.76s/it]
 55%|█████▌    | 124865/225000 [254:07:28<153:00:50,  5.50s/it]
 55%|█████▌    | 124866/225000 [254:07:33<149:20:29,  5.37s/it]
 55%|█████▌    | 124867/225000 [254:07:38<144:10:51,  5.18s/it]
 55%|█████▌    | 124868/225000 [254:07:43<138:01:41,  4.96s/it]
 55%|█████▌    | 124869/225000 [254:07:47<136:41:37,  4.91s/it]
 55%|█████▌    | 124870/225000 [254:07:52<135:34:22,  4.87s/it]
 55%|█████▌    | 124871/225000 [254:07:57<133:55:21,  4.82s/it]
 55%|█████▌    | 124872/225000 [254:08:02<136:14:01,  4.90s/it]
 55%|█████▌    | 124873/225000 [254:08:07<135:26:44,  4.87s/it]
 55%|█████▌    | 124874/225000 [254:08:11<133:59:36,  4.82s/it]
 56%|█████▌    | 124875/225000 [254:08:17<136:47:25,  4.92s/it]
 56%|█████▌    | 124876/225000 [254:08:22<138:44:10,  4.99s/it]
 56%|█████▌    | 124877/225000 [254:08:28<149:46:14,  5.39s/it]
 56%|█████▌    | 124878/225000 [254:08:37<179:03:07,  6.44s/it]
 56%|█████▌    | 124879/225000 [254:08:45<193:18:07,  6.95s/it]
 56%|█████▌    | 124880/225000 [254:08:49<167:18:24,  6.02s/it]
 56%|█████▌    | 124881/225000 [254:08:53<148:17:49,  5.33s/it]
 56%|█████▌    | 124882/225000 [254:09:01<177:04:57,  6.37s/it]
 56%|█████▌    | 124883/225000 [254:09:09<183:21:51,  6.59s/it]
 56%|█████▌    | 124884/225000 [254:09:12<153:54:18,  5.53s/it]
 56%|█████▌    | 124885/225000 [254:09:21<186:17:13,  6.70s/it]
 56%|█████▌    | 124886/225000 [254:09:24<158:14:27,  5.69s/it]
 56%|█████▌    | 124887/225000 [254:09:32<175:47:31,  6.32s/it]
 56%|█████▌    | 124888/225000 [254:09:36<158:47:46,  5.71s/it]
 56%|█████▌    | 124889/225000 [254:09:41<148:00:23,  5.32s/it]
 56%|█████▌    | 124890/225000 [254:09:45<137:13:28,  4.93s/it]
 56%|█████▌    | 124891/225000 [254:09:49<130:34:59,  4.70s/it]
 56%|█████▌    | 124892/225000 [254:09:54<128:55:24,  4.64s/it]
 56%|█████▌    | 124893/225000 [254:09:58<130:07:00,  4.68s/it]
 56%|█████▌    | 124894/225000 [254:10:08<170:21:04,  6.13s/it]
 56%|█████▌    | 124895/225000 [254:10:17<192:41:30,  6.93s/it]
 56%|█████▌    | 124896/225000 [254:10:22<177:58:08,  6.40s/it]
 56%|█████▌    | 124897/225000 [254:10:26<162:45:47,  5.85s/it]
 56%|█████▌    | 124898/225000 [254:10:31<151:48:08,  5.46s/it]
 56%|█████▌    | 124899/225000 [254:10:36<145:10:50,  5.22s/it]
 56%|█████▌    | 124900/225000 [254:10:40<139:34:45,  5.02s/it]
                                                               
{'loss': '0.8386', 'grad_norm': '1.089', 'learning_rate': '2.232e-05', 'epoch': '0.5551', 'num_input_tokens_seen': 423342224, 'train_runtime': '9.15e+05', 'train_tokens_per_second': '462.6'}

 56%|█████▌    | 124900/225000 [254:10:40<139:34:45,  5.02s/it]
 56%|█████▌    | 124901/225000 [254:10:45<136:45:25,  4.92s/it]
 56%|█████▌    | 124902/225000 [254:10:50<135:42:55,  4.88s/it]
 56%|█████▌    | 124903/225000 [254:11:02<196:59:40,  7.08s/it]
 56%|█████▌    | 124904/225000 [254:11:06<169:35:51,  6.10s/it]
 56%|█████▌    | 124905/225000 [254:11:14<191:03:41,  6.87s/it]
 56%|█████▌    | 124906/225000 [254:11:20<178:22:17,  6.42s/it]
 56%|█████▌    | 124907/225000 [254:11:25<171:48:48,  6.18s/it]
 56%|█████▌    | 124908/225000 [254:11:32<172:39:32,  6.21s/it]
 56%|█████▌    | 124909/225000 [254:11:36<157:42:04,  5.67s/it]
 56%|█████▌    | 124910/225000 [254:11:42<159:54:46,  5.75s/it]
 56%|█████▌    | 124911/225000 [254:11:47<157:10:27,  5.65s/it]
 56%|█████▌    | 124912/225000 [254:11:53<160:26:39,  5.77s/it]
 56%|█████▌    | 124913/225000 [254:11:57<141:47:14,  5.10s/it]
 56%|█████▌    | 124914/225000 [254:12:05<170:06:07,  6.12s/it]
 56%|█████▌    | 124915/225000 [254:12:09<145:17:12,  5.23s/it]
 56%|█████▌    | 124916/225000 [254:12:12<133:31:32,  4.80s/it]
 56%|█████▌    | 124917/225000 [254:12:17<134:45:58,  4.85s/it]
 56%|█████▌    | 124918/225000 [254:12:22<136:13:19,  4.90s/it]
 56%|█████▌    | 124919/225000 [254:12:27<134:14:07,  4.83s/it]
 56%|█████▌    | 124920/225000 [254:12:33<140:44:58,  5.06s/it]
 56%|█████▌    | 124921/225000 [254:12:38<139:45:48,  5.03s/it]
 56%|█████▌    | 124922/225000 [254:12:43<142:09:41,  5.11s/it]
 56%|█████▌    | 124923/225000 [254:12:47<137:42:47,  4.95s/it]
 56%|█████▌    | 124924/225000 [254:12:53<140:13:40,  5.04s/it]
 56%|█████▌    | 124925/225000 [254:12:59<150:33:18,  5.42s/it]
 56%|█████▌    | 124926/225000 [254:13:11<206:39:04,  7.43s/it]
 56%|█████▌    | 124927/225000 [254:13:15<172:53:28,  6.22s/it]
 56%|█████▌    | 124928/225000 [254:13:22<180:38:05,  6.50s/it]
 56%|█████▌    | 124929/225000 [254:13:25<156:10:42,  5.62s/it]
 56%|█████▌    | 124930/225000 [254:13:33<171:08:37,  6.16s/it]
 56%|█████▌    | 124931/225000 [254:13:39<172:46:21,  6.22s/it]
 56%|█████▌    | 124932/225000 [254:13:43<155:12:35,  5.58s/it]
 56%|█████▌    | 124933/225000 [254:13:47<139:37:35,  5.02s/it]
 56%|█████▌    | 124934/225000 [254:13:58<189:06:32,  6.80s/it]
 56%|█████▌    | 124935/225000 [254:14:01<163:00:54,  5.86s/it]
 56%|█████▌    | 124936/225000 [254:14:05<143:27:08,  5.16s/it]
 56%|█████▌    | 124937/225000 [254:14:10<138:06:05,  4.97s/it]
 56%|█████▌    | 124938/225000 [254:14:19<179:56:37,  6.47s/it]
 56%|█████▌    | 124939/225000 [254:14:25<169:09:53,  6.09s/it]
 56%|█████▌    | 124940/225000 [254:14:28<146:43:29,  5.28s/it]
 56%|█████▌    | 124941/225000 [254:14:34<153:18:42,  5.52s/it]
 56%|█████▌    | 124942/225000 [254:14:42<171:08:49,  6.16s/it]
 56%|█████▌    | 124943/225000 [254:14:49<179:02:22,  6.44s/it]
 56%|█████▌    | 124944/225000 [254:14:56<180:26:39,  6.49s/it]
 56%|█████▌    | 124945/225000 [254:14:59<158:32:18,  5.70s/it]
 56%|█████▌    | 124946/225000 [254:15:04<148:29:03,  5.34s/it]
 56%|█████▌    | 124947/225000 [254:15:12<174:40:05,  6.28s/it]
 56%|█████▌    | 124948/225000 [254:15:16<149:43:38,  5.39s/it]
 56%|█████▌    | 124949/225000 [254:15:21<151:44:40,  5.46s/it]
 56%|█████▌    | 124950/225000 [254:15:29<170:35:16,  6.14s/it]
                                                               
{'loss': '0.8301', 'grad_norm': '1.043', 'learning_rate': '2.23e-05', 'epoch': '0.5553', 'num_input_tokens_seen': 423471496, 'train_runtime': '9.153e+05', 'train_tokens_per_second': '462.6'}

 56%|█████▌    | 124950/225000 [254:15:29<170:35:16,  6.14s/it]
 56%|█████▌    | 124951/225000 [254:15:33<156:46:26,  5.64s/it]
 56%|█████▌    | 124952/225000 [254:15:38<148:32:48,  5.35s/it]
 56%|█████▌    | 124953/225000 [254:15:43<145:33:24,  5.24s/it]
 56%|█████▌    | 124954/225000 [254:15:47<138:00:43,  4.97s/it]
 56%|█████▌    | 124955/225000 [254:15:52<136:14:41,  4.90s/it]
 56%|█████▌    | 124956/225000 [254:15:57<133:07:15,  4.79s/it]
 56%|█████▌    | 124957/225000 [254:16:01<128:00:13,  4.61s/it]
 56%|█████▌    | 124958/225000 [254:16:10<163:50:19,  5.90s/it]
 56%|█████▌    | 124959/225000 [254:16:19<195:16:20,  7.03s/it]
 56%|█████▌    | 124960/225000 [254:16:24<170:22:36,  6.13s/it]
 56%|█████▌    | 124961/225000 [254:16:27<151:26:19,  5.45s/it]
 56%|█████▌    | 124962/225000 [254:16:35<165:33:09,  5.96s/it]
 56%|█████▌    | 124963/225000 [254:16:41<171:44:45,  6.18s/it]
 56%|█████▌    | 124964/225000 [254:16:45<154:37:07,  5.56s/it]
 56%|█████▌    | 124965/225000 [254:16:49<142:12:40,  5.12s/it]
 56%|█████▌    | 124966/225000 [254:16:54<137:03:15,  4.93s/it]
 56%|█████▌    | 124967/225000 [254:16:58<131:56:52,  4.75s/it]
 56%|█████▌    | 124968/225000 [254:17:03<130:31:51,  4.70s/it]
 56%|█████▌    | 124969/225000 [254:17:14<180:39:32,  6.50s/it]
 56%|█████▌    | 124970/225000 [254:17:19<173:28:38,  6.24s/it]
 56%|█████▌    | 124971/225000 [254:17:23<150:30:56,  5.42s/it]
 56%|█████▌    | 124972/225000 [254:17:33<187:27:05,  6.75s/it]
 56%|█████▌    | 124973/225000 [254:17:38<174:17:27,  6.27s/it]
 56%|█████▌    | 124974/225000 [254:17:42<156:17:05,  5.62s/it]
 56%|█████▌    | 124975/225000 [254:17:46<146:41:12,  5.28s/it]
 56%|█████▌    | 124976/225000 [254:17:51<141:40:05,  5.10s/it]
 56%|█████▌    | 124977/225000 [254:17:55<133:34:10,  4.81s/it]
 56%|█████▌    | 124978/225000 [254:18:00<132:15:17,  4.76s/it]
 56%|█████▌    | 124979/225000 [254:18:08<163:00:46,  5.87s/it]
 56%|█████▌    | 124980/225000 [254:18:19<203:40:55,  7.33s/it]
 56%|█████▌    | 124981/225000 [254:18:22<171:32:27,  6.17s/it]
 56%|█████▌    | 124982/225000 [254:18:27<160:54:26,  5.79s/it]
 56%|█████▌    | 124983/225000 [254:18:32<151:57:00,  5.47s/it]
 56%|█████▌    | 124984/225000 [254:18:37<146:22:07,  5.27s/it]
 56%|█████▌    | 124985/225000 [254:18:42<145:17:32,  5.23s/it]
 56%|█████▌    | 124986/225000 [254:18:47<143:54:15,  5.18s/it]
 56%|█████▌    | 124987/225000 [254:18:52<142:00:56,  5.11s/it]
 56%|█████▌    | 124988/225000 [254:18:57<138:12:33,  4.97s/it]
 56%|█████▌    | 124989/225000 [254:19:02<138:11:03,  4.97s/it]
 56%|█████▌    | 124990/225000 [254:19:06<135:08:19,  4.86s/it]
 56%|█████▌    | 124991/225000 [254:19:11<132:11:50,  4.76s/it]
 56%|█████▌    | 124992/225000 [254:19:23<197:28:01,  7.11s/it]
 56%|█████▌    | 124993/225000 [254:19:31<203:33:01,  7.33s/it]
 56%|█████▌    | 124994/225000 [254:19:35<171:22:38,  6.17s/it]
 56%|█████▌    | 124995/225000 [254:19:39<153:40:44,  5.53s/it]
 56%|█████▌    | 124996/225000 [254:19:49<191:39:43,  6.90s/it]
 56%|█████▌    | 124997/225000 [254:19:52<158:28:44,  5.71s/it]
 56%|█████▌    | 124998/225000 [254:19:57<159:18:57,  5.74s/it]
 56%|█████▌    | 124999/225000 [254:20:05<170:13:27,  6.13s/it]
 56%|█████▌    | 125000/225000 [254:20:09<154:39:46,  5.57s/it]
                                                               
{'loss': '0.8303', 'grad_norm': '1.057', 'learning_rate': '2.229e-05', 'epoch': '0.5556', 'num_input_tokens_seen': 423599272, 'train_runtime': '9.156e+05', 'train_tokens_per_second': '462.6'}

 56%|█████▌    | 125000/225000 [254:20:09<154:39:46,  5.57s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                               
{'eval_loss': '1.342', 'eval_runtime': '429.4', 'eval_samples_per_second': '1.164', 'eval_steps_per_second': '0.582', 'epoch': '0.5556', 'num_input_tokens_seen': 423599272}

 56%|█████▌    | 125000/225000 [254:27:18<154:39:46,  5.57s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-125000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 56%|█████▌    | 125001/225000 [254:27:26<3753:47:15, 135.14s/it]
 56%|█████▌    | 125002/225000 [254:27:29<2654:19:10, 95.56s/it] 
 56%|█████▌    | 125003/225000 [254:27:34<1893:32:28, 68.17s/it]
 56%|█████▌    | 125004/225000 [254:27:38<1363:49:19, 49.10s/it]
 56%|█████▌    | 125005/225000 [254:27:43<994:23:32, 35.80s/it] 
 56%|█████▌    | 125006/225000 [254:27:48<734:13:15, 26.43s/it]
 56%|█████▌    | 125007/225000 [254:27:55<578:20:06, 20.82s/it]
 56%|█████▌    | 125008/225000 [254:28:05<481:39:49, 17.34s/it]
 56%|█████▌    | 125009/225000 [254:28:08<369:26:42, 13.30s/it]
 56%|█████▌    | 125010/225000 [254:28:14<301:13:53, 10.85s/it]
 56%|█████▌    | 125011/225000 [254:28:18<248:05:16,  8.93s/it]
 56%|█████▌    | 125012/225000 [254:28:25<234:45:14,  8.45s/it]
 56%|█████▌    | 125013/225000 [254:28:31<213:13:06,  7.68s/it]
 56%|█████▌    | 125014/225000 [254:28:38<202:58:37,  7.31s/it]
 56%|█████▌    | 125015/225000 [254:28:42<176:15:22,  6.35s/it]
 56%|█████▌    | 125016/225000 [254:28:46<158:00:13,  5.69s/it]
 56%|█████▌    | 125017/225000 [254:28:50<144:45:31,  5.21s/it]
 56%|█████▌    | 125018/225000 [254:28:58<167:59:52,  6.05s/it]
 56%|█████▌    | 125019/225000 [254:29:05<171:33:57,  6.18s/it]
 56%|█████▌    | 125020/225000 [254:29:08<148:39:16,  5.35s/it]
 56%|█████▌    | 125021/225000 [254:29:18<183:49:14,  6.62s/it]
 56%|█████▌    | 125022/225000 [254:29:21<155:46:19,  5.61s/it]
 56%|█████▌    | 125023/225000 [254:29:28<168:26:07,  6.07s/it]
 56%|█████▌    | 125024/225000 [254:29:32<149:59:33,  5.40s/it]
 56%|█████▌    | 125025/225000 [254:29:36<138:28:56,  4.99s/it]
 56%|█████▌    | 125026/225000 [254:29:45<171:05:07,  6.16s/it]
 56%|█████▌    | 125027/225000 [254:29:48<149:35:09,  5.39s/it]
 56%|█████▌    | 125028/225000 [254:29:53<142:20:07,  5.13s/it]
 56%|█████▌    | 125029/225000 [254:29:57<136:33:08,  4.92s/it]
 56%|█████▌    | 125030/225000 [254:30:07<174:17:14,  6.28s/it]
 56%|█████▌    | 125031/225000 [254:30:13<177:17:42,  6.38s/it]
 56%|█████▌    | 125032/225000 [254:30:17<156:35:49,  5.64s/it]
 56%|█████▌    | 125033/225000 [254:30:26<178:46:26,  6.44s/it]
 56%|█████▌    | 125034/225000 [254:30:29<154:04:27,  5.55s/it]
 56%|█████▌    | 125035/225000 [254:30:38<181:08:14,  6.52s/it]
 56%|█████▌    | 125036/225000 [254:30:41<154:51:05,  5.58s/it]
 56%|█████▌    | 125037/225000 [254:30:47<155:44:24,  5.61s/it]
 56%|█████▌    | 125038/225000 [254:30:53<162:01:56,  5.84s/it]
 56%|█████▌    | 125039/225000 [254:31:00<170:01:29,  6.12s/it]
 56%|█████▌    | 125040/225000 [254:31:07<179:14:35,  6.46s/it]
 56%|█████▌    | 125041/225000 [254:31:11<153:33:26,  5.53s/it]
 56%|█████▌    | 125042/225000 [254:31:19<174:29:14,  6.28s/it]
 56%|█████▌    | 125043/225000 [254:31:22<153:04:48,  5.51s/it]
 56%|█████▌    | 125044/225000 [254:31:29<163:07:18,  5.87s/it]
 56%|█████▌    | 125045/225000 [254:31:35<164:09:11,  5.91s/it]
 56%|█████▌    | 125046/225000 [254:31:39<144:20:05,  5.20s/it]
 56%|█████▌    | 125047/225000 [254:31:44<145:09:25,  5.23s/it]
 56%|█████▌    | 125048/225000 [254:31:52<170:41:48,  6.15s/it]
 56%|█████▌    | 125049/225000 [254:31:57<158:26:27,  5.71s/it]
 56%|█████▌    | 125050/225000 [254:32:04<168:04:37,  6.05s/it]
                                                               
{'loss': '0.8335', 'grad_norm': '1.111', 'learning_rate': '2.227e-05', 'epoch': '0.5558', 'num_input_tokens_seen': 423728432, 'train_runtime': '9.163e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125050/225000 [254:32:04<168:04:37,  6.05s/it]
 56%|█████▌    | 125051/225000 [254:32:10<168:04:49,  6.05s/it]
 56%|█████▌    | 125052/225000 [254:32:13<147:25:08,  5.31s/it]
 56%|█████▌    | 125053/225000 [254:32:21<164:38:50,  5.93s/it]
 56%|█████▌    | 125054/225000 [254:32:25<147:23:45,  5.31s/it]
 56%|█████▌    | 125055/225000 [254:32:29<139:42:41,  5.03s/it]
 56%|█████▌    | 125056/225000 [254:32:41<195:32:54,  7.04s/it]
 56%|█████▌    | 125057/225000 [254:32:44<165:35:55,  5.96s/it]
 56%|█████▌    | 125058/225000 [254:32:52<184:49:59,  6.66s/it]
 56%|█████▌    | 125059/225000 [254:32:56<155:59:47,  5.62s/it]
 56%|█████▌    | 125060/225000 [254:33:00<146:32:23,  5.28s/it]
 56%|█████▌    | 125061/225000 [254:33:05<144:48:39,  5.22s/it]
 56%|█████▌    | 125062/225000 [254:33:10<141:19:53,  5.09s/it]
 56%|█████▌    | 125063/225000 [254:33:15<139:01:42,  5.01s/it]
 56%|█████▌    | 125064/225000 [254:33:20<140:40:50,  5.07s/it]
 56%|█████▌    | 125065/225000 [254:33:24<135:39:48,  4.89s/it]
 56%|█████▌    | 125066/225000 [254:33:34<175:38:00,  6.33s/it]
 56%|█████▌    | 125067/225000 [254:33:41<183:54:16,  6.63s/it]
 56%|█████▌    | 125068/225000 [254:33:48<182:13:48,  6.56s/it]
 56%|█████▌    | 125069/225000 [254:33:52<157:33:26,  5.68s/it]
 56%|█████▌    | 125070/225000 [254:34:01<187:46:58,  6.76s/it]
 56%|█████▌    | 125071/225000 [254:34:05<162:05:37,  5.84s/it]
 56%|█████▌    | 125072/225000 [254:34:13<186:21:10,  6.71s/it]
 56%|█████▌    | 125073/225000 [254:34:17<159:45:41,  5.76s/it]
 56%|█████▌    | 125074/225000 [254:34:24<168:09:42,  6.06s/it]
 56%|█████▌    | 125075/225000 [254:34:27<147:34:15,  5.32s/it]
 56%|█████▌    | 125076/225000 [254:34:31<138:23:31,  4.99s/it]
 56%|█████▌    | 125077/225000 [254:34:42<186:22:16,  6.71s/it]
 56%|█████▌    | 125078/225000 [254:34:45<156:49:31,  5.65s/it]
 56%|█████▌    | 125079/225000 [254:34:51<160:19:22,  5.78s/it]
 56%|█████▌    | 125080/225000 [254:34:56<154:19:32,  5.56s/it]
 56%|█████▌    | 125081/225000 [254:35:01<147:33:20,  5.32s/it]
 56%|█████▌    | 125082/225000 [254:35:06<142:11:46,  5.12s/it]
 56%|█████▌    | 125083/225000 [254:35:11<142:04:59,  5.12s/it]
 56%|█████▌    | 125084/225000 [254:35:16<140:17:46,  5.05s/it]
 56%|█████▌    | 125085/225000 [254:35:20<134:23:05,  4.84s/it]
 56%|█████▌    | 125086/225000 [254:35:25<134:18:43,  4.84s/it]
 56%|█████▌    | 125087/225000 [254:35:31<141:06:26,  5.08s/it]
 56%|█████▌    | 125088/225000 [254:35:42<192:41:23,  6.94s/it]
 56%|█████▌    | 125089/225000 [254:35:47<177:48:40,  6.41s/it]
 56%|█████▌    | 125090/225000 [254:35:53<174:13:34,  6.28s/it]
 56%|█████▌    | 125091/225000 [254:36:00<178:24:03,  6.43s/it]
 56%|█████▌    | 125092/225000 [254:36:05<164:16:04,  5.92s/it]
 56%|█████▌    | 125093/225000 [254:36:09<152:22:19,  5.49s/it]
 56%|█████▌    | 125094/225000 [254:36:19<185:20:44,  6.68s/it]
 56%|█████▌    | 125095/225000 [254:36:22<160:15:54,  5.78s/it]
 56%|█████▌    | 125096/225000 [254:36:31<188:41:26,  6.80s/it]
 56%|█████▌    | 125097/225000 [254:36:34<157:54:25,  5.69s/it]
 56%|█████▌    | 125098/225000 [254:36:42<169:07:26,  6.09s/it]
 56%|█████▌    | 125099/225000 [254:36:45<151:26:57,  5.46s/it]
 56%|█████▌    | 125100/225000 [254:36:51<153:54:19,  5.55s/it]
                                                               
{'loss': '0.8306', 'grad_norm': '1.06', 'learning_rate': '2.225e-05', 'epoch': '0.556', 'num_input_tokens_seen': 423859448, 'train_runtime': '9.166e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125100/225000 [254:36:51<153:54:19,  5.55s/it]
 56%|█████▌    | 125101/225000 [254:36:59<168:24:42,  6.07s/it]
 56%|█████▌    | 125102/225000 [254:37:03<152:05:36,  5.48s/it]
 56%|█████▌    | 125103/225000 [254:37:07<141:45:42,  5.11s/it]
 56%|█████▌    | 125104/225000 [254:37:18<188:05:13,  6.78s/it]
 56%|█████▌    | 125105/225000 [254:37:25<194:43:07,  7.02s/it]
 56%|█████▌    | 125106/225000 [254:37:29<169:09:43,  6.10s/it]
 56%|█████▌    | 125107/225000 [254:37:36<178:02:29,  6.42s/it]
 56%|█████▌    | 125108/225000 [254:37:40<153:38:50,  5.54s/it]
 56%|█████▌    | 125109/225000 [254:37:49<184:24:19,  6.65s/it]
 56%|█████▌    | 125110/225000 [254:37:53<162:07:46,  5.84s/it]
 56%|█████▌    | 125111/225000 [254:37:57<144:46:22,  5.22s/it]
 56%|█████▌    | 125112/225000 [254:38:02<141:39:05,  5.11s/it]
 56%|█████▌    | 125113/225000 [254:38:10<173:08:31,  6.24s/it]
 56%|█████▌    | 125114/225000 [254:38:14<147:44:17,  5.32s/it]
 56%|█████▌    | 125115/225000 [254:38:21<165:31:57,  5.97s/it]
 56%|█████▌    | 125116/225000 [254:38:25<148:54:50,  5.37s/it]
 56%|█████▌    | 125117/225000 [254:38:29<136:18:59,  4.91s/it]
 56%|█████▌    | 125118/225000 [254:38:38<169:30:04,  6.11s/it]
 56%|█████▌    | 125119/225000 [254:38:43<158:06:34,  5.70s/it]
 56%|█████▌    | 125120/225000 [254:38:48<153:23:26,  5.53s/it]
 56%|█████▌    | 125121/225000 [254:38:52<147:04:15,  5.30s/it]
 56%|█████▌    | 125122/225000 [254:38:58<145:20:00,  5.24s/it]
 56%|█████▌    | 125123/225000 [254:39:03<143:50:15,  5.18s/it]
 56%|█████▌    | 125124/225000 [254:39:07<140:31:01,  5.06s/it]
 56%|█████▌    | 125125/225000 [254:39:13<141:42:18,  5.11s/it]
 56%|█████▌    | 125126/225000 [254:39:20<158:48:54,  5.72s/it]
 56%|█████▌    | 125127/225000 [254:39:27<169:35:28,  6.11s/it]
 56%|█████▌    | 125128/225000 [254:39:32<161:43:18,  5.83s/it]
 56%|█████▌    | 125129/225000 [254:39:37<153:33:07,  5.54s/it]
 56%|█████▌    | 125130/225000 [254:39:42<149:28:28,  5.39s/it]
 56%|█████▌    | 125131/225000 [254:39:49<162:07:38,  5.84s/it]
 56%|█████▌    | 125132/225000 [254:40:00<206:01:43,  7.43s/it]
 56%|█████▌    | 125133/225000 [254:40:04<175:56:24,  6.34s/it]
 56%|█████▌    | 125134/225000 [254:40:09<168:59:30,  6.09s/it]
 56%|█████▌    | 125135/225000 [254:40:18<191:55:57,  6.92s/it]
 56%|█████▌    | 125136/225000 [254:40:21<162:01:38,  5.84s/it]
 56%|█████▌    | 125137/225000 [254:40:29<179:09:35,  6.46s/it]
 56%|█████▌    | 125138/225000 [254:40:34<169:05:59,  6.10s/it]
 56%|█████▌    | 125139/225000 [254:40:42<180:29:13,  6.51s/it]
 56%|█████▌    | 125140/225000 [254:40:45<152:15:37,  5.49s/it]
 56%|█████▌    | 125141/225000 [254:40:49<136:51:45,  4.93s/it]
 56%|█████▌    | 125142/225000 [254:40:58<171:32:21,  6.18s/it]
 56%|█████▌    | 125143/225000 [254:41:02<154:45:05,  5.58s/it]
 56%|█████▌    | 125144/225000 [254:41:07<146:25:27,  5.28s/it]
 56%|█████▌    | 125145/225000 [254:41:11<136:08:30,  4.91s/it]
 56%|█████▌    | 125146/225000 [254:41:15<133:51:02,  4.83s/it]
 56%|█████▌    | 125147/225000 [254:41:20<130:57:35,  4.72s/it]
 56%|█████▌    | 125148/225000 [254:41:29<171:43:31,  6.19s/it]
 56%|█████▌    | 125149/225000 [254:41:38<195:07:28,  7.03s/it]
 56%|█████▌    | 125150/225000 [254:41:43<174:03:54,  6.28s/it]
                                                               
{'loss': '0.8378', 'grad_norm': '1.126', 'learning_rate': '2.223e-05', 'epoch': '0.5562', 'num_input_tokens_seen': 423989560, 'train_runtime': '9.169e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125150/225000 [254:41:43<174:03:54,  6.28s/it]
 56%|█████▌    | 125151/225000 [254:41:49<170:25:40,  6.14s/it]
 56%|█████▌    | 125152/225000 [254:41:54<166:59:57,  6.02s/it]
 56%|█████▌    | 125153/225000 [254:42:00<159:44:49,  5.76s/it]
 56%|█████▌    | 125154/225000 [254:42:04<149:37:22,  5.39s/it]
 56%|█████▌    | 125155/225000 [254:42:09<147:51:00,  5.33s/it]
 56%|█████▌    | 125156/225000 [254:42:15<147:33:16,  5.32s/it]
 56%|█████▌    | 125157/225000 [254:42:20<144:29:11,  5.21s/it]
 56%|█████▌    | 125158/225000 [254:42:25<144:46:03,  5.22s/it]
 56%|█████▌    | 125159/225000 [254:42:30<142:26:30,  5.14s/it]
 56%|█████▌    | 125160/225000 [254:42:35<140:16:00,  5.06s/it]
 56%|█████▌    | 125161/225000 [254:42:40<141:15:50,  5.09s/it]
 56%|█████▌    | 125162/225000 [254:42:52<201:24:48,  7.26s/it]
 56%|█████▌    | 125163/225000 [254:42:56<175:50:06,  6.34s/it]
 56%|█████▌    | 125164/225000 [254:43:03<182:36:16,  6.58s/it]
 56%|█████▌    | 125165/225000 [254:43:07<158:59:00,  5.73s/it]
 56%|█████▌    | 125166/225000 [254:43:11<143:20:49,  5.17s/it]
 56%|█████▌    | 125167/225000 [254:43:20<178:08:55,  6.42s/it]
 56%|█████▌    | 125168/225000 [254:43:25<162:48:25,  5.87s/it]
 56%|█████▌    | 125169/225000 [254:43:31<160:33:40,  5.79s/it]
 56%|█████▌    | 125170/225000 [254:43:35<147:21:00,  5.31s/it]
 56%|█████▌    | 125171/225000 [254:43:39<139:13:24,  5.02s/it]
 56%|█████▌    | 125172/225000 [254:43:51<193:59:48,  7.00s/it]
 56%|█████▌    | 125173/225000 [254:43:54<167:11:22,  6.03s/it]
 56%|█████▌    | 125174/225000 [254:43:59<153:18:44,  5.53s/it]
 56%|█████▌    | 125175/225000 [254:44:08<186:13:19,  6.72s/it]
 56%|█████▌    | 125176/225000 [254:44:13<167:18:17,  6.03s/it]
 56%|█████▌    | 125177/225000 [254:44:17<152:51:08,  5.51s/it]
 56%|█████▌    | 125178/225000 [254:44:21<143:46:01,  5.18s/it]
 56%|█████▌    | 125179/225000 [254:44:26<134:31:40,  4.85s/it]
 56%|█████▌    | 125180/225000 [254:44:30<131:30:32,  4.74s/it]
 56%|█████▌    | 125181/225000 [254:44:34<127:55:43,  4.61s/it]
 56%|█████▌    | 125182/225000 [254:44:40<132:28:18,  4.78s/it]
 56%|█████▌    | 125183/225000 [254:44:52<195:29:18,  7.05s/it]
 56%|█████▌    | 125184/225000 [254:44:56<173:31:14,  6.26s/it]
 56%|█████▌    | 125185/225000 [254:45:04<184:07:09,  6.64s/it]
 56%|█████▌    | 125186/225000 [254:45:07<158:28:15,  5.72s/it]
 56%|█████▌    | 125187/225000 [254:45:12<146:56:50,  5.30s/it]
 56%|█████▌    | 125188/225000 [254:45:16<137:44:06,  4.97s/it]
 56%|█████▌    | 125189/225000 [254:45:26<183:37:58,  6.62s/it]
 56%|█████▌    | 125190/225000 [254:45:30<156:05:17,  5.63s/it]
 56%|█████▌    | 125191/225000 [254:45:39<186:37:52,  6.73s/it]
 56%|█████▌    | 125192/225000 [254:45:45<180:32:21,  6.51s/it]
 56%|█████▌    | 125193/225000 [254:45:51<174:01:02,  6.28s/it]
 56%|█████▌    | 125194/225000 [254:45:54<148:30:40,  5.36s/it]
 56%|█████▌    | 125195/225000 [254:45:59<147:24:28,  5.32s/it]
 56%|█████▌    | 125196/225000 [254:46:06<157:51:04,  5.69s/it]
 56%|█████▌    | 125197/225000 [254:46:12<166:20:10,  6.00s/it]
 56%|█████▌    | 125198/225000 [254:46:16<148:42:51,  5.36s/it]
 56%|█████▌    | 125199/225000 [254:46:25<179:36:35,  6.48s/it]
 56%|█████▌    | 125200/225000 [254:46:30<160:24:53,  5.79s/it]
                                                               
{'loss': '0.8331', 'grad_norm': '1.204', 'learning_rate': '2.221e-05', 'epoch': '0.5564', 'num_input_tokens_seen': 424118728, 'train_runtime': '9.172e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125200/225000 [254:46:30<160:24:53,  5.79s/it]
 56%|█████▌    | 125201/225000 [254:46:34<147:42:19,  5.33s/it]
 56%|█████▌    | 125202/225000 [254:46:38<141:15:14,  5.10s/it]
 56%|█████▌    | 125203/225000 [254:46:43<133:55:59,  4.83s/it]
 56%|█████▌    | 125204/225000 [254:46:53<183:48:40,  6.63s/it]
 56%|█████▌    | 125205/225000 [254:46:57<161:55:16,  5.84s/it]
 56%|█████▌    | 125206/225000 [254:47:02<147:32:44,  5.32s/it]
 56%|█████▌    | 125207/225000 [254:47:06<138:09:05,  4.98s/it]
 56%|█████▌    | 125208/225000 [254:47:15<176:59:43,  6.39s/it]
 56%|█████▌    | 125209/225000 [254:47:20<161:15:26,  5.82s/it]
 56%|█████▌    | 125210/225000 [254:47:28<182:54:06,  6.60s/it]
 56%|█████▌    | 125211/225000 [254:47:33<168:11:10,  6.07s/it]
 56%|█████▌    | 125212/225000 [254:47:38<157:29:29,  5.68s/it]
 56%|█████▌    | 125213/225000 [254:47:43<151:37:08,  5.47s/it]
 56%|█████▌    | 125214/225000 [254:47:47<144:05:48,  5.20s/it]
 56%|█████▌    | 125215/225000 [254:47:52<138:57:58,  5.01s/it]
 56%|█████▌    | 125216/225000 [254:47:56<133:54:02,  4.83s/it]
 56%|█████▌    | 125217/225000 [254:48:07<181:17:49,  6.54s/it]
 56%|█████▌    | 125218/225000 [254:48:15<191:34:27,  6.91s/it]
 56%|█████▌    | 125219/225000 [254:48:20<179:31:57,  6.48s/it]
 56%|█████▌    | 125220/225000 [254:48:26<175:38:46,  6.34s/it]
 56%|█████▌    | 125221/225000 [254:48:32<170:20:42,  6.15s/it]
 56%|█████▌    | 125222/225000 [254:48:40<182:28:45,  6.58s/it]
 56%|█████▌    | 125223/225000 [254:48:44<167:48:00,  6.05s/it]
 56%|█████▌    | 125224/225000 [254:48:48<147:59:12,  5.34s/it]
 56%|█████▌    | 125225/225000 [254:48:52<139:17:40,  5.03s/it]
 56%|█████▌    | 125226/225000 [254:49:01<169:37:16,  6.12s/it]
 56%|█████▌    | 125227/225000 [254:49:05<155:17:06,  5.60s/it]
 56%|█████▌    | 125228/225000 [254:49:11<151:07:11,  5.45s/it]
 56%|█████▌    | 125229/225000 [254:49:15<144:34:28,  5.22s/it]
 56%|█████▌    | 125230/225000 [254:49:20<143:15:05,  5.17s/it]
 56%|█████▌    | 125231/225000 [254:49:25<142:25:13,  5.14s/it]
 56%|█████▌    | 125232/225000 [254:49:30<139:47:47,  5.04s/it]
 56%|█████▌    | 125233/225000 [254:49:35<139:17:06,  5.03s/it]
 56%|█████▌    | 125234/225000 [254:49:40<140:32:14,  5.07s/it]
 56%|█████▌    | 125235/225000 [254:49:45<137:57:54,  4.98s/it]
 56%|█████▌    | 125236/225000 [254:49:50<138:54:27,  5.01s/it]
 56%|█████▌    | 125237/225000 [254:49:55<141:17:31,  5.10s/it]
 56%|█████▌    | 125238/225000 [254:50:02<156:25:08,  5.64s/it]
 56%|█████▌    | 125239/225000 [254:50:15<215:44:01,  7.79s/it]
 56%|█████▌    | 125240/225000 [254:50:19<183:36:12,  6.63s/it]
 56%|█████▌    | 125241/225000 [254:50:27<193:56:02,  7.00s/it]
 56%|█████▌    | 125242/225000 [254:50:31<172:56:47,  6.24s/it]
 56%|█████▌    | 125243/225000 [254:50:36<155:48:33,  5.62s/it]
 56%|█████▌    | 125244/225000 [254:50:40<144:20:33,  5.21s/it]
 56%|█████▌    | 125245/225000 [254:50:46<153:01:59,  5.52s/it]
 56%|█████▌    | 125246/225000 [254:50:55<180:43:30,  6.52s/it]
 56%|█████▌    | 125247/225000 [254:50:59<160:11:00,  5.78s/it]
 56%|█████▌    | 125248/225000 [254:51:05<160:34:57,  5.80s/it]
 56%|█████▌    | 125249/225000 [254:51:09<144:15:42,  5.21s/it]
 56%|█████▌    | 125250/225000 [254:51:13<137:07:39,  4.95s/it]
                                                               
{'loss': '0.8296', 'grad_norm': '1.187', 'learning_rate': '2.22e-05', 'epoch': '0.5567', 'num_input_tokens_seen': 424248000, 'train_runtime': '9.175e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125250/225000 [254:51:13<137:07:39,  4.95s/it]
 56%|█████▌    | 125251/225000 [254:51:20<151:46:14,  5.48s/it]
 56%|█████▌    | 125252/225000 [254:51:30<188:53:20,  6.82s/it]
 56%|█████▌    | 125253/225000 [254:51:35<179:49:24,  6.49s/it]
 56%|█████▌    | 125254/225000 [254:51:39<156:14:47,  5.64s/it]
 56%|█████▌    | 125255/225000 [254:51:42<137:35:08,  4.97s/it]
 56%|█████▌    | 125256/225000 [254:51:51<165:56:35,  5.99s/it]
 56%|█████▌    | 125257/225000 [254:51:56<161:01:13,  5.81s/it]
 56%|█████▌    | 125258/225000 [254:52:02<160:37:48,  5.80s/it]
 56%|█████▌    | 125259/225000 [254:52:06<143:46:39,  5.19s/it]
 56%|█████▌    | 125260/225000 [254:52:10<133:56:32,  4.83s/it]
 56%|█████▌    | 125261/225000 [254:52:21<183:19:31,  6.62s/it]
 56%|█████▌    | 125262/225000 [254:52:25<164:06:43,  5.92s/it]
 56%|█████▌    | 125263/225000 [254:52:29<147:10:12,  5.31s/it]
 56%|█████▌    | 125264/225000 [254:52:36<163:18:29,  5.89s/it]
 56%|█████▌    | 125265/225000 [254:52:44<180:27:20,  6.51s/it]
 56%|█████▌    | 125266/225000 [254:52:49<170:18:38,  6.15s/it]
 56%|█████▌    | 125267/225000 [254:52:54<162:04:19,  5.85s/it]
 56%|█████▌    | 125268/225000 [254:53:06<210:09:44,  7.59s/it]
 56%|█████▌    | 125269/225000 [254:53:09<173:57:17,  6.28s/it]
 56%|█████▌    | 125270/225000 [254:53:14<165:21:29,  5.97s/it]
 56%|█████▌    | 125271/225000 [254:53:22<174:23:27,  6.30s/it]
 56%|█████▌    | 125272/225000 [254:53:25<151:49:41,  5.48s/it]
 56%|█████▌    | 125273/225000 [254:53:31<151:29:00,  5.47s/it]
 56%|█████▌    | 125274/225000 [254:53:36<150:42:00,  5.44s/it]
 56%|█████▌    | 125275/225000 [254:53:41<147:52:58,  5.34s/it]
 56%|█████▌    | 125276/225000 [254:53:46<144:39:04,  5.22s/it]
 56%|█████▌    | 125277/225000 [254:53:51<145:48:29,  5.26s/it]
 56%|█████▌    | 125278/225000 [254:53:56<144:58:33,  5.23s/it]
 56%|█████▌    | 125279/225000 [254:54:01<138:44:15,  5.01s/it]
 56%|█████▌    | 125280/225000 [254:54:06<141:07:19,  5.09s/it]
 56%|█████▌    | 125281/225000 [254:54:14<166:18:57,  6.00s/it]
 56%|█████▌    | 125282/225000 [254:54:20<160:33:56,  5.80s/it]
 56%|█████▌    | 125283/225000 [254:54:25<155:55:33,  5.63s/it]
 56%|█████▌    | 125284/225000 [254:54:30<149:55:23,  5.41s/it]
 56%|█████▌    | 125285/225000 [254:54:37<162:58:30,  5.88s/it]
 56%|█████▌    | 125286/225000 [254:54:46<191:16:47,  6.91s/it]
 56%|█████▌    | 125287/225000 [254:54:51<172:41:17,  6.23s/it]
 56%|█████▌    | 125288/225000 [254:54:56<167:24:13,  6.04s/it]
 56%|█████▌    | 125289/225000 [254:55:02<163:32:55,  5.90s/it]
 56%|█████▌    | 125290/225000 [254:55:08<164:24:03,  5.94s/it]
 56%|█████▌    | 125291/225000 [254:55:12<148:29:02,  5.36s/it]
 56%|█████▌    | 125292/225000 [254:55:17<142:27:39,  5.14s/it]
 56%|█████▌    | 125293/225000 [254:55:22<141:09:28,  5.10s/it]
 56%|█████▌    | 125294/225000 [254:55:27<142:40:38,  5.15s/it]
 56%|█████▌    | 125295/225000 [254:55:32<141:39:22,  5.11s/it]
 56%|█████▌    | 125296/225000 [254:55:37<140:12:29,  5.06s/it]
 56%|█████▌    | 125297/225000 [254:55:42<143:52:15,  5.19s/it]
 56%|█████▌    | 125298/225000 [254:55:47<140:45:52,  5.08s/it]
 56%|█████▌    | 125299/225000 [254:55:52<138:35:30,  5.00s/it]
 56%|█████▌    | 125300/225000 [254:55:59<157:50:11,  5.70s/it]
                                                               
{'loss': '0.8348', 'grad_norm': '0.9831', 'learning_rate': '2.218e-05', 'epoch': '0.5569', 'num_input_tokens_seen': 424377784, 'train_runtime': '9.178e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125300/225000 [254:55:59<157:50:11,  5.70s/it]
 56%|█████▌    | 125301/225000 [254:56:10<199:23:33,  7.20s/it]
 56%|█████▌    | 125302/225000 [254:56:14<175:54:40,  6.35s/it]
 56%|█████▌    | 125303/225000 [254:56:18<154:07:37,  5.57s/it]
 56%|█████▌    | 125304/225000 [254:56:29<194:53:35,  7.04s/it]
 56%|█████▌    | 125305/225000 [254:56:32<165:30:07,  5.98s/it]
 56%|█████▌    | 125306/225000 [254:56:40<178:05:47,  6.43s/it]
 56%|█████▌    | 125307/225000 [254:56:45<165:23:49,  5.97s/it]
 56%|█████▌    | 125308/225000 [254:56:48<148:44:36,  5.37s/it]
 56%|█████▌    | 125309/225000 [254:56:57<174:32:41,  6.30s/it]
 56%|█████▌    | 125310/225000 [254:57:04<177:18:38,  6.40s/it]
 56%|█████▌    | 125311/225000 [254:57:08<156:58:57,  5.67s/it]
 56%|█████▌    | 125312/225000 [254:57:12<145:32:29,  5.26s/it]
 56%|█████▌    | 125313/225000 [254:57:17<145:25:56,  5.25s/it]
 56%|█████▌    | 125314/225000 [254:57:22<143:49:23,  5.19s/it]
 56%|█████▌    | 125315/225000 [254:57:27<143:43:05,  5.19s/it]
 56%|█████▌    | 125316/225000 [254:57:32<142:27:05,  5.14s/it]
 56%|█████▌    | 125317/225000 [254:57:37<140:58:59,  5.09s/it]
 56%|█████▌    | 125318/225000 [254:57:42<139:44:06,  5.05s/it]
 56%|█████▌    | 125319/225000 [254:57:47<140:46:27,  5.08s/it]
 56%|█████▌    | 125320/225000 [254:57:52<138:23:21,  5.00s/it]
 56%|█████▌    | 125321/225000 [254:57:57<138:44:44,  5.01s/it]
 56%|█████▌    | 125322/225000 [254:58:09<196:53:06,  7.11s/it]
 56%|█████▌    | 125323/225000 [254:58:16<193:04:13,  6.97s/it]
 56%|█████▌    | 125324/225000 [254:58:21<174:47:21,  6.31s/it]
 56%|█████▌    | 125325/225000 [254:58:26<162:16:44,  5.86s/it]
 56%|█████▌    | 125326/225000 [254:58:30<152:39:04,  5.51s/it]
 56%|█████▌    | 125327/225000 [254:58:35<144:18:10,  5.21s/it]
 56%|█████▌    | 125328/225000 [254:58:39<139:55:43,  5.05s/it]
 56%|█████▌    | 125329/225000 [254:58:50<189:14:50,  6.84s/it]
 56%|█████▌    | 125330/225000 [254:58:55<171:52:45,  6.21s/it]
 56%|█████▌    | 125331/225000 [254:59:00<159:11:11,  5.75s/it]
 56%|█████▌    | 125332/225000 [254:59:04<149:16:00,  5.39s/it]
 56%|█████▌    | 125333/225000 [254:59:09<145:04:15,  5.24s/it]
 56%|█████▌    | 125334/225000 [254:59:14<137:26:40,  4.96s/it]
 56%|█████▌    | 125335/225000 [254:59:18<136:39:53,  4.94s/it]
 56%|█████▌    | 125336/225000 [254:59:24<141:58:27,  5.13s/it]
 56%|█████▌    | 125337/225000 [254:59:37<208:53:17,  7.55s/it]
 56%|█████▌    | 125338/225000 [254:59:41<177:24:33,  6.41s/it]
 56%|█████▌    | 125339/225000 [254:59:45<157:26:24,  5.69s/it]
 56%|█████▌    | 125340/225000 [254:59:50<154:36:08,  5.58s/it]
 56%|█████▌    | 125341/225000 [254:59:55<147:11:57,  5.32s/it]
 56%|█████▌    | 125342/225000 [255:00:00<147:03:52,  5.31s/it]
 56%|█████▌    | 125343/225000 [255:00:05<142:38:50,  5.15s/it]
 56%|█████▌    | 125344/225000 [255:00:10<143:19:20,  5.18s/it]
 56%|█████▌    | 125345/225000 [255:00:16<144:25:27,  5.22s/it]
 56%|█████▌    | 125346/225000 [255:00:21<144:36:01,  5.22s/it]
 56%|█████▌    | 125347/225000 [255:00:26<146:29:38,  5.29s/it]
 56%|█████▌    | 125348/225000 [255:00:31<144:35:22,  5.22s/it]
 56%|█████▌    | 125349/225000 [255:00:37<144:35:22,  5.22s/it]
 56%|█████▌    | 125350/225000 [255:00:45<173:30:41,  6.27s/it]
                                                               
{'loss': '0.8264', 'grad_norm': '1.018', 'learning_rate': '2.216e-05', 'epoch': '0.5571', 'num_input_tokens_seen': 424506448, 'train_runtime': '9.18e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125350/225000 [255:00:45<173:30:41,  6.27s/it]
 56%|█████▌    | 125351/225000 [255:00:56<212:16:43,  7.67s/it]
 56%|█████▌    | 125352/225000 [255:01:00<177:08:28,  6.40s/it]
 56%|█████▌    | 125353/225000 [255:01:04<162:25:58,  5.87s/it]
 56%|█████▌    | 125354/225000 [255:01:09<151:16:47,  5.47s/it]
 56%|█████▌    | 125355/225000 [255:01:16<162:49:28,  5.88s/it]
 56%|█████▌    | 125356/225000 [255:01:26<199:39:03,  7.21s/it]
 56%|█████▌    | 125357/225000 [255:01:30<169:33:44,  6.13s/it]
 56%|█████▌    | 125358/225000 [255:01:34<152:47:57,  5.52s/it]
 56%|█████▌    | 125359/225000 [255:01:40<159:33:58,  5.77s/it]
 56%|█████▌    | 125360/225000 [255:01:49<189:07:33,  6.83s/it]
 56%|█████▌    | 125361/225000 [255:01:54<169:47:53,  6.13s/it]
 56%|█████▌    | 125362/225000 [255:01:59<158:13:26,  5.72s/it]
 56%|█████▌    | 125363/225000 [255:02:03<145:52:03,  5.27s/it]
 56%|█████▌    | 125364/225000 [255:02:07<137:12:18,  4.96s/it]
 56%|█████▌    | 125365/225000 [255:02:12<135:58:40,  4.91s/it]
 56%|█████▌    | 125366/225000 [255:02:24<197:55:42,  7.15s/it]
 56%|█████▌    | 125367/225000 [255:02:27<164:05:08,  5.93s/it]
 56%|█████▌    | 125368/225000 [255:02:35<179:42:56,  6.49s/it]
 56%|█████▌    | 125369/225000 [255:02:39<157:31:48,  5.69s/it]
 56%|█████▌    | 125370/225000 [255:02:46<166:46:09,  6.03s/it]
 56%|█████▌    | 125371/225000 [255:02:54<184:46:35,  6.68s/it]
 56%|█████▌    | 125372/225000 [255:02:58<162:22:55,  5.87s/it]
 56%|█████▌    | 125373/225000 [255:03:04<166:39:44,  6.02s/it]
 56%|█████▌    | 125374/225000 [255:03:12<180:38:39,  6.53s/it]
 56%|█████▌    | 125375/225000 [255:03:16<155:51:38,  5.63s/it]
 56%|█████▌    | 125376/225000 [255:03:20<142:18:02,  5.14s/it]
 56%|█████▌    | 125377/225000 [255:03:24<138:54:54,  5.02s/it]
 56%|█████▌    | 125378/225000 [255:03:29<135:54:00,  4.91s/it]
 56%|█████▌    | 125379/225000 [255:03:34<137:27:54,  4.97s/it]
 56%|█████▌    | 125380/225000 [255:03:39<133:23:04,  4.82s/it]
 56%|█████▌    | 125381/225000 [255:03:44<134:52:10,  4.87s/it]
 56%|█████▌    | 125382/225000 [255:03:55<189:15:03,  6.84s/it]
 56%|█████▌    | 125383/225000 [255:04:02<192:35:16,  6.96s/it]
 56%|█████▌    | 125384/225000 [255:04:05<160:32:21,  5.80s/it]
 56%|█████▌    | 125385/225000 [255:04:13<177:54:35,  6.43s/it]
 56%|█████▌    | 125386/225000 [255:04:17<156:22:06,  5.65s/it]
 56%|█████▌    | 125387/225000 [255:04:24<170:39:20,  6.17s/it]
 56%|█████▌    | 125388/225000 [255:04:29<159:05:10,  5.75s/it]
 56%|█████▌    | 125389/225000 [255:04:33<145:18:34,  5.25s/it]
 56%|█████▌    | 125390/225000 [255:04:43<186:10:27,  6.73s/it]
 56%|█████▌    | 125391/225000 [255:04:48<163:40:20,  5.92s/it]
 56%|█████▌    | 125392/225000 [255:04:52<151:20:55,  5.47s/it]
 56%|█████▌    | 125393/225000 [255:04:56<141:19:03,  5.11s/it]
 56%|█████▌    | 125394/225000 [255:05:02<147:13:23,  5.32s/it]
 56%|█████▌    | 125395/225000 [255:05:12<184:33:22,  6.67s/it]
 56%|█████▌    | 125396/225000 [255:05:16<160:02:30,  5.78s/it]
 56%|█████▌    | 125397/225000 [255:05:23<172:24:35,  6.23s/it]
 56%|█████▌    | 125398/225000 [255:05:28<166:02:23,  6.00s/it]
 56%|█████▌    | 125399/225000 [255:05:36<177:55:24,  6.43s/it]
 56%|█████▌    | 125400/225000 [255:05:39<152:04:45,  5.50s/it]
                                                               
{'loss': '0.8309', 'grad_norm': '1.147', 'learning_rate': '2.214e-05', 'epoch': '0.5573', 'num_input_tokens_seen': 424635736, 'train_runtime': '9.183e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125400/225000 [255:05:39<152:04:45,  5.50s/it]
 56%|█████▌    | 125401/225000 [255:05:43<143:19:40,  5.18s/it]
 56%|█████▌    | 125402/225000 [255:05:48<134:47:16,  4.87s/it]
 56%|█████▌    | 125403/225000 [255:05:52<130:09:51,  4.70s/it]
 56%|█████▌    | 125404/225000 [255:05:57<129:19:01,  4.67s/it]
 56%|█████▌    | 125405/225000 [255:06:10<198:31:55,  7.18s/it]
 56%|█████▌    | 125406/225000 [255:06:14<174:26:59,  6.31s/it]
 56%|█████▌    | 125407/225000 [255:06:19<166:28:09,  6.02s/it]
 56%|█████▌    | 125408/225000 [255:06:25<162:58:30,  5.89s/it]
 56%|█████▌    | 125409/225000 [255:06:30<160:10:17,  5.79s/it]
 56%|█████▌    | 125410/225000 [255:06:35<154:51:48,  5.60s/it]
 56%|█████▌    | 125411/225000 [255:06:45<187:43:21,  6.79s/it]
 56%|█████▌    | 125412/225000 [255:06:49<161:54:51,  5.85s/it]
 56%|█████▌    | 125413/225000 [255:06:54<152:57:21,  5.53s/it]
 56%|█████▌    | 125414/225000 [255:06:58<147:15:02,  5.32s/it]
 56%|█████▌    | 125415/225000 [255:07:03<144:08:51,  5.21s/it]
 56%|█████▌    | 125416/225000 [255:07:08<140:06:34,  5.07s/it]
 56%|█████▌    | 125417/225000 [255:07:13<138:40:59,  5.01s/it]
 56%|█████▌    | 125418/225000 [255:07:23<180:53:20,  6.54s/it]
 56%|█████▌    | 125419/225000 [255:07:32<200:07:59,  7.24s/it]
 56%|█████▌    | 125420/225000 [255:07:39<195:59:09,  7.09s/it]
 56%|█████▌    | 125421/225000 [255:07:42<162:54:31,  5.89s/it]
 56%|█████▌    | 125422/225000 [255:07:49<175:24:23,  6.34s/it]
 56%|█████▌    | 125423/225000 [255:07:56<177:10:25,  6.41s/it]
 56%|█████▌    | 125424/225000 [255:08:00<160:22:36,  5.80s/it]
 56%|█████▌    | 125425/225000 [255:08:04<145:15:58,  5.25s/it]
 56%|█████▌    | 125426/225000 [255:08:13<178:23:40,  6.45s/it]
 56%|█████▌    | 125427/225000 [255:08:18<160:49:05,  5.81s/it]
 56%|█████▌    | 125428/225000 [255:08:22<147:50:38,  5.35s/it]
 56%|█████▌    | 125429/225000 [255:08:26<135:48:42,  4.91s/it]
 56%|█████▌    | 125430/225000 [255:08:30<133:55:56,  4.84s/it]
 56%|█████▌    | 125431/225000 [255:08:43<198:25:36,  7.17s/it]
 56%|█████▌    | 125432/225000 [255:08:47<169:33:34,  6.13s/it]
 56%|█████▌    | 125433/225000 [255:08:50<149:35:03,  5.41s/it]
 56%|█████▌    | 125434/225000 [255:08:55<144:51:17,  5.24s/it]
 56%|█████▌    | 125435/225000 [255:09:00<140:40:40,  5.09s/it]
 56%|█████▌    | 125436/225000 [255:09:05<135:39:40,  4.91s/it]
 56%|█████▌    | 125437/225000 [255:09:10<138:48:05,  5.02s/it]
 56%|█████▌    | 125438/225000 [255:09:15<141:26:11,  5.11s/it]
 56%|█████▌    | 125439/225000 [255:09:20<138:42:51,  5.02s/it]
 56%|█████▌    | 125440/225000 [255:09:25<137:20:02,  4.97s/it]
 56%|█████▌    | 125441/225000 [255:09:36<189:52:58,  6.87s/it]
 56%|█████▌    | 125442/225000 [255:09:45<205:02:48,  7.41s/it]
 56%|█████▌    | 125443/225000 [255:09:48<174:30:41,  6.31s/it]
 56%|█████▌    | 125444/225000 [255:09:53<155:33:37,  5.63s/it]
 56%|█████▌    | 125445/225000 [255:09:59<162:16:21,  5.87s/it]
 56%|█████▌    | 125446/225000 [255:10:08<186:20:23,  6.74s/it]
 56%|█████▌    | 125447/225000 [255:10:14<186:29:53,  6.74s/it]
 56%|█████▌    | 125448/225000 [255:10:19<170:22:52,  6.16s/it]
 56%|█████▌    | 125449/225000 [255:10:27<181:13:42,  6.55s/it]
 56%|█████▌    | 125450/225000 [255:10:31<166:03:48,  6.01s/it]
                                                               
{'loss': '0.8225', 'grad_norm': '1.013', 'learning_rate': '2.212e-05', 'epoch': '0.5576', 'num_input_tokens_seen': 424765464, 'train_runtime': '9.186e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125450/225000 [255:10:31<166:03:48,  6.01s/it]
 56%|█████▌    | 125451/225000 [255:10:36<155:29:46,  5.62s/it]
 56%|█████▌    | 125452/225000 [255:10:41<146:31:46,  5.30s/it]
 56%|█████▌    | 125453/225000 [255:10:45<141:47:20,  5.13s/it]
 56%|█████▌    | 125454/225000 [255:10:50<136:34:08,  4.94s/it]
 56%|█████▌    | 125455/225000 [255:10:55<135:26:36,  4.90s/it]
 56%|█████▌    | 125456/225000 [255:11:03<161:09:56,  5.83s/it]
 56%|█████▌    | 125457/225000 [255:11:11<181:38:41,  6.57s/it]
 56%|█████▌    | 125458/225000 [255:11:15<159:52:34,  5.78s/it]
 56%|█████▌    | 125459/225000 [255:11:21<159:24:42,  5.77s/it]
 56%|█████▌    | 125460/225000 [255:11:29<181:09:34,  6.55s/it]
 56%|█████▌    | 125461/225000 [255:11:33<155:06:49,  5.61s/it]
 56%|█████▌    | 125462/225000 [255:11:37<148:21:20,  5.37s/it]
 56%|█████▌    | 125463/225000 [255:11:43<147:00:53,  5.32s/it]
 56%|█████▌    | 125464/225000 [255:11:48<144:01:53,  5.21s/it]
 56%|█████▌    | 125465/225000 [255:11:52<140:49:35,  5.09s/it]
 56%|█████▌    | 125466/225000 [255:11:58<143:36:42,  5.19s/it]
 56%|█████▌    | 125467/225000 [255:12:03<141:51:32,  5.13s/it]
 56%|█████▌    | 125468/225000 [255:12:08<139:39:38,  5.05s/it]
 56%|█████▌    | 125469/225000 [255:12:13<138:56:11,  5.03s/it]
 56%|█████▌    | 125470/225000 [255:12:18<138:47:38,  5.02s/it]
 56%|█████▌    | 125471/225000 [255:12:30<197:53:12,  7.16s/it]
 56%|█████▌    | 125472/225000 [255:12:36<194:16:09,  7.03s/it]
 56%|█████▌    | 125473/225000 [255:12:40<166:01:28,  6.01s/it]
 56%|█████▌    | 125474/225000 [255:12:45<153:52:24,  5.57s/it]
 56%|█████▌    | 125475/225000 [255:12:53<180:33:33,  6.53s/it]
 56%|█████▌    | 125476/225000 [255:13:00<184:03:52,  6.66s/it]
 56%|█████▌    | 125477/225000 [255:13:04<158:15:06,  5.72s/it]
 56%|█████▌    | 125478/225000 [255:13:08<142:49:14,  5.17s/it]
 56%|█████▌    | 125479/225000 [255:13:18<183:29:16,  6.64s/it]
 56%|█████▌    | 125480/225000 [255:13:22<162:26:06,  5.88s/it]
 56%|█████▌    | 125481/225000 [255:13:26<145:43:18,  5.27s/it]
 56%|█████▌    | 125482/225000 [255:13:29<127:29:50,  4.61s/it]
 56%|█████▌    | 125483/225000 [255:13:35<141:55:42,  5.13s/it]
 56%|█████▌    | 125484/225000 [255:13:46<190:22:18,  6.89s/it]
 56%|█████▌    | 125485/225000 [255:13:49<160:44:02,  5.81s/it]
 56%|█████▌    | 125486/225000 [255:13:57<174:38:52,  6.32s/it]
 56%|█████▌    | 125487/225000 [255:14:00<151:00:35,  5.46s/it]
 56%|█████▌    | 125488/225000 [255:14:08<167:31:38,  6.06s/it]
 56%|█████▌    | 125489/225000 [255:14:12<151:27:35,  5.48s/it]
 56%|█████▌    | 125490/225000 [255:14:17<151:03:04,  5.46s/it]
 56%|█████▌    | 125491/225000 [255:14:27<182:33:25,  6.60s/it]
 56%|█████▌    | 125492/225000 [255:14:31<161:01:40,  5.83s/it]
 56%|█████▌    | 125493/225000 [255:14:35<146:47:03,  5.31s/it]
 56%|█████▌    | 125494/225000 [255:14:45<186:50:04,  6.76s/it]
 56%|█████▌    | 125495/225000 [255:14:48<159:12:34,  5.76s/it]
 56%|█████▌    | 125496/225000 [255:14:56<177:11:47,  6.41s/it]
 56%|█████▌    | 125497/225000 [255:15:00<157:12:43,  5.69s/it]
 56%|█████▌    | 125498/225000 [255:15:05<145:36:06,  5.27s/it]
 56%|█████▌    | 125499/225000 [255:15:10<144:14:29,  5.22s/it]
 56%|█████▌    | 125500/225000 [255:15:18<167:11:04,  6.05s/it]
                                                               
{'loss': '0.8373', 'grad_norm': '0.9586', 'learning_rate': '2.21e-05', 'epoch': '0.5578', 'num_input_tokens_seen': 424895288, 'train_runtime': '9.189e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125500/225000 [255:15:18<167:11:04,  6.05s/it]
 56%|█████▌    | 125501/225000 [255:15:22<154:14:47,  5.58s/it]
 56%|█████▌    | 125502/225000 [255:15:27<150:04:20,  5.43s/it]
 56%|█████▌    | 125503/225000 [255:15:32<147:07:34,  5.32s/it]
 56%|█████▌    | 125504/225000 [255:15:37<142:31:00,  5.16s/it]
 56%|█████▌    | 125505/225000 [255:15:42<143:14:34,  5.18s/it]
 56%|█████▌    | 125506/225000 [255:15:47<141:09:28,  5.11s/it]
 56%|█████▌    | 125507/225000 [255:15:52<139:48:29,  5.06s/it]
 56%|█████▌    | 125508/225000 [255:15:57<141:03:46,  5.10s/it]
 56%|█████▌    | 125509/225000 [255:16:09<195:18:57,  7.07s/it]
 56%|█████▌    | 125510/225000 [255:16:16<193:50:54,  7.01s/it]
 56%|█████▌    | 125511/225000 [255:16:20<168:03:55,  6.08s/it]
 56%|█████▌    | 125512/225000 [255:16:24<149:41:22,  5.42s/it]
 56%|█████▌    | 125513/225000 [255:16:34<192:56:26,  6.98s/it]
 56%|█████▌    | 125514/225000 [255:16:38<163:53:41,  5.93s/it]
 56%|█████▌    | 125515/225000 [255:16:47<186:09:17,  6.74s/it]
 56%|█████▌    | 125516/225000 [255:16:51<170:29:22,  6.17s/it]
 56%|█████▌    | 125517/225000 [255:16:56<161:34:46,  5.85s/it]
 56%|█████▌    | 125518/225000 [255:17:01<154:49:45,  5.60s/it]
 56%|█████▌    | 125519/225000 [255:17:07<151:07:55,  5.47s/it]
 56%|█████▌    | 125520/225000 [255:17:12<148:26:17,  5.37s/it]
 56%|█████▌    | 125521/225000 [255:17:17<147:16:13,  5.33s/it]
 56%|█████▌    | 125522/225000 [255:17:22<144:14:28,  5.22s/it]
 56%|█████▌    | 125523/225000 [255:17:27<144:48:21,  5.24s/it]
 56%|█████▌    | 125524/225000 [255:17:32<141:06:27,  5.11s/it]
 56%|█████▌    | 125525/225000 [255:17:38<144:51:32,  5.24s/it]
 56%|█████▌    | 125526/225000 [255:17:43<147:45:11,  5.35s/it]
 56%|█████▌    | 125527/225000 [255:17:50<156:16:57,  5.66s/it]
 56%|█████▌    | 125528/225000 [255:18:03<219:07:43,  7.93s/it]
 56%|█████▌    | 125529/225000 [255:18:07<190:58:04,  6.91s/it]
 56%|█████▌    | 125530/225000 [255:18:11<163:40:17,  5.92s/it]
 56%|█████▌    | 125531/225000 [255:18:20<190:14:14,  6.89s/it]
 56%|█████▌    | 125532/225000 [255:18:25<169:42:01,  6.14s/it]
 56%|█████▌    | 125533/225000 [255:18:31<172:58:39,  6.26s/it]
 56%|█████▌    | 125534/225000 [255:18:35<152:47:43,  5.53s/it]
 56%|█████▌    | 125535/225000 [255:18:40<148:10:38,  5.36s/it]
 56%|█████▌    | 125536/225000 [255:18:44<142:14:42,  5.15s/it]
 56%|█████▌    | 125537/225000 [255:18:49<138:20:33,  5.01s/it]
 56%|█████▌    | 125538/225000 [255:18:54<137:11:10,  4.97s/it]
 56%|█████▌    | 125539/225000 [255:18:58<132:36:50,  4.80s/it]
 56%|█████▌    | 125540/225000 [255:19:09<177:56:36,  6.44s/it]
 56%|█████▌    | 125541/225000 [255:19:18<203:47:45,  7.38s/it]
 56%|█████▌    | 125542/225000 [255:19:22<171:10:30,  6.20s/it]
 56%|█████▌    | 125543/225000 [255:19:25<150:28:22,  5.45s/it]
 56%|█████▌    | 125544/225000 [255:19:35<183:59:19,  6.66s/it]
 56%|█████▌    | 125545/225000 [255:19:41<176:56:50,  6.41s/it]
 56%|█████▌    | 125546/225000 [255:19:46<165:42:35,  6.00s/it]
 56%|█████▌    | 125547/225000 [255:19:50<153:18:23,  5.55s/it]
 56%|█████▌    | 125548/225000 [255:19:58<172:45:22,  6.25s/it]
 56%|█████▌    | 125549/225000 [255:20:02<150:54:59,  5.46s/it]
 56%|█████▌    | 125550/225000 [255:20:10<174:47:56,  6.33s/it]
                                                               
{'loss': '0.8336', 'grad_norm': '1.032', 'learning_rate': '2.209e-05', 'epoch': '0.558', 'num_input_tokens_seen': 425026560, 'train_runtime': '9.192e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125550/225000 [255:20:10<174:47:56,  6.33s/it]
 56%|█████▌    | 125551/225000 [255:20:14<156:51:05,  5.68s/it]
 56%|█████▌    | 125552/225000 [255:20:18<142:35:31,  5.16s/it]
 56%|█████▌    | 125553/225000 [255:20:23<141:10:32,  5.11s/it]
 56%|█████▌    | 125554/225000 [255:20:28<141:01:09,  5.10s/it]
 56%|█████▌    | 125555/225000 [255:20:33<137:39:01,  4.98s/it]
 56%|█████▌    | 125556/225000 [255:20:45<194:43:28,  7.05s/it]
 56%|█████▌    | 125557/225000 [255:20:49<170:25:57,  6.17s/it]
 56%|█████▌    | 125558/225000 [255:20:53<151:50:07,  5.50s/it]
 56%|█████▌    | 125559/225000 [255:20:57<139:17:46,  5.04s/it]
 56%|█████▌    | 125560/225000 [255:21:03<146:11:34,  5.29s/it]
 56%|█████▌    | 125561/225000 [255:21:15<199:42:40,  7.23s/it]
 56%|█████▌    | 125562/225000 [255:21:18<169:46:02,  6.15s/it]
 56%|█████▌    | 125563/225000 [255:21:23<154:53:47,  5.61s/it]
 56%|█████▌    | 125564/225000 [255:21:33<192:38:29,  6.97s/it]
 56%|█████▌    | 125565/225000 [255:21:36<160:22:30,  5.81s/it]
 56%|█████▌    | 125566/225000 [255:21:43<173:28:35,  6.28s/it]
 56%|█████▌    | 125567/225000 [255:21:48<158:07:21,  5.72s/it]
 56%|█████▌    | 125568/225000 [255:21:52<146:24:36,  5.30s/it]
 56%|█████▌    | 125569/225000 [255:21:56<140:07:43,  5.07s/it]
 56%|█████▌    | 125570/225000 [255:22:01<133:45:48,  4.84s/it]
 56%|█████▌    | 125571/225000 [255:22:07<145:07:22,  5.25s/it]
 56%|█████▌    | 125572/225000 [255:22:19<198:44:26,  7.20s/it]
 56%|█████▌    | 125573/225000 [255:22:22<166:05:18,  6.01s/it]
 56%|█████▌    | 125574/225000 [255:22:29<172:15:02,  6.24s/it]
 56%|█████▌    | 125575/225000 [255:22:35<172:07:11,  6.23s/it]
 56%|█████▌    | 125576/225000 [255:22:40<159:22:54,  5.77s/it]
 56%|█████▌    | 125577/225000 [255:22:44<149:39:48,  5.42s/it]
 56%|█████▌    | 125578/225000 [255:22:49<142:43:28,  5.17s/it]
 56%|█████▌    | 125579/225000 [255:22:54<139:40:33,  5.06s/it]
 56%|█████▌    | 125580/225000 [255:23:01<159:17:24,  5.77s/it]
 56%|█████▌    | 125581/225000 [255:23:10<184:59:03,  6.70s/it]
 56%|█████▌    | 125582/225000 [255:23:14<160:16:50,  5.80s/it]
 56%|█████▌    | 125583/225000 [255:23:18<150:54:59,  5.46s/it]
 56%|█████▌    | 125584/225000 [255:23:23<143:38:29,  5.20s/it]
 56%|█████▌    | 125585/225000 [255:23:34<191:49:36,  6.95s/it]
 56%|█████▌    | 125586/225000 [255:23:39<172:39:41,  6.25s/it]
 56%|█████▌    | 125587/225000 [255:23:44<167:46:49,  6.08s/it]
 56%|█████▌    | 125588/225000 [255:23:51<177:50:42,  6.44s/it]
 56%|█████▌    | 125589/225000 [255:23:56<161:31:43,  5.85s/it]
 56%|█████▌    | 125590/225000 [255:24:01<150:58:42,  5.47s/it]
 56%|█████▌    | 125591/225000 [255:24:05<141:51:12,  5.14s/it]
 56%|█████▌    | 125592/225000 [255:24:09<135:24:12,  4.90s/it]
 56%|█████▌    | 125593/225000 [255:24:14<130:14:19,  4.72s/it]
 56%|█████▌    | 125594/225000 [255:24:18<130:58:02,  4.74s/it]
 56%|█████▌    | 125595/225000 [255:24:23<129:23:31,  4.69s/it]
 56%|█████▌    | 125596/225000 [255:24:28<129:43:17,  4.70s/it]
 56%|█████▌    | 125597/225000 [255:24:33<133:52:39,  4.85s/it]
 56%|█████▌    | 125598/225000 [255:24:40<156:58:13,  5.68s/it]
 56%|█████▌    | 125599/225000 [255:24:53<214:52:04,  7.78s/it]
 56%|█████▌    | 125600/225000 [255:24:57<178:54:22,  6.48s/it]
                                                               
{'loss': '0.8298', 'grad_norm': '1.142', 'learning_rate': '2.207e-05', 'epoch': '0.5582', 'num_input_tokens_seen': 425155696, 'train_runtime': '9.195e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125600/225000 [255:24:57<178:54:22,  6.48s/it]
 56%|█████▌    | 125601/225000 [255:25:04<187:15:49,  6.78s/it]
 56%|█████▌    | 125602/225000 [255:25:08<161:04:59,  5.83s/it]
 56%|█████▌    | 125603/225000 [255:25:12<146:03:38,  5.29s/it]
 56%|█████▌    | 125604/225000 [255:25:24<203:07:08,  7.36s/it]
 56%|█████▌    | 125605/225000 [255:25:27<170:43:40,  6.18s/it]
 56%|█████▌    | 125606/225000 [255:25:35<185:16:59,  6.71s/it]
 56%|█████▌    | 125607/225000 [255:25:39<158:43:30,  5.75s/it]
 56%|█████▌    | 125608/225000 [255:25:45<164:48:48,  5.97s/it]
 56%|█████▌    | 125609/225000 [255:25:52<174:46:02,  6.33s/it]
 56%|█████▌    | 125610/225000 [255:25:56<150:17:41,  5.44s/it]
 56%|█████▌    | 125611/225000 [255:26:01<144:16:21,  5.23s/it]
 56%|█████▌    | 125612/225000 [255:26:10<181:32:38,  6.58s/it]
 56%|█████▌    | 125613/225000 [255:26:15<167:52:36,  6.08s/it]
 56%|█████▌    | 125614/225000 [255:26:20<155:08:53,  5.62s/it]
 56%|█████▌    | 125615/225000 [255:26:24<148:07:55,  5.37s/it]
 56%|█████▌    | 125616/225000 [255:26:29<142:59:27,  5.18s/it]
 56%|█████▌    | 125617/225000 [255:26:34<141:44:34,  5.13s/it]
 56%|█████▌    | 125618/225000 [255:26:45<189:28:44,  6.86s/it]
 56%|█████▌    | 125619/225000 [255:26:49<167:30:58,  6.07s/it]
 56%|█████▌    | 125620/225000 [255:26:54<155:10:39,  5.62s/it]
 56%|█████▌    | 125621/225000 [255:26:58<145:02:58,  5.25s/it]
 56%|█████▌    | 125622/225000 [255:27:03<138:00:31,  5.00s/it]
 56%|█████▌    | 125623/225000 [255:27:14<193:45:02,  7.02s/it]
 56%|█████▌    | 125624/225000 [255:27:18<164:31:14,  5.96s/it]
 56%|█████▌    | 125625/225000 [255:27:27<189:51:03,  6.88s/it]
 56%|█████▌    | 125626/225000 [255:27:31<167:17:27,  6.06s/it]
 56%|█████▌    | 125627/225000 [255:27:36<154:05:26,  5.58s/it]
 56%|█████▌    | 125628/225000 [255:27:40<143:12:18,  5.19s/it]
 56%|█████▌    | 125629/225000 [255:27:44<135:14:14,  4.90s/it]
 56%|█████▌    | 125630/225000 [255:27:49<133:04:03,  4.82s/it]
 56%|█████▌    | 125631/225000 [255:27:58<171:00:36,  6.20s/it]
 56%|█████▌    | 125632/225000 [255:28:07<189:27:08,  6.86s/it]
 56%|█████▌    | 125633/225000 [255:28:11<166:00:58,  6.01s/it]
 56%|█████▌    | 125634/225000 [255:28:14<148:29:27,  5.38s/it]
 56%|█████▌    | 125635/225000 [255:28:19<138:09:12,  5.01s/it]
 56%|█████▌    | 125636/225000 [255:28:26<157:17:05,  5.70s/it]
 56%|█████▌    | 125637/225000 [255:28:35<182:38:15,  6.62s/it]
 56%|█████▌    | 125638/225000 [255:28:40<169:13:46,  6.13s/it]
 56%|█████▌    | 125639/225000 [255:28:45<160:37:39,  5.82s/it]
 56%|█████▌    | 125640/225000 [255:28:50<152:22:41,  5.52s/it]
 56%|█████▌    | 125641/225000 [255:28:55<151:17:30,  5.48s/it]
 56%|█████▌    | 125642/225000 [255:29:00<149:44:43,  5.43s/it]
 56%|█████▌    | 125643/225000 [255:29:05<145:16:12,  5.26s/it]
 56%|█████▌    | 125644/225000 [255:29:10<144:23:04,  5.23s/it]
 56%|█████▌    | 125645/225000 [255:29:15<142:18:48,  5.16s/it]
 56%|█████▌    | 125646/225000 [255:29:27<194:05:26,  7.03s/it]
 56%|█████▌    | 125647/225000 [255:29:34<193:22:48,  7.01s/it]
 56%|█████▌    | 125648/225000 [255:29:37<161:59:22,  5.87s/it]
 56%|█████▌    | 125649/225000 [255:29:47<198:19:23,  7.19s/it]
 56%|█████▌    | 125650/225000 [255:29:51<167:38:15,  6.07s/it]
                                                               
{'loss': '0.8317', 'grad_norm': '1.159', 'learning_rate': '2.205e-05', 'epoch': '0.5584', 'num_input_tokens_seen': 425284952, 'train_runtime': '9.198e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125650/225000 [255:29:51<167:38:15,  6.07s/it]
 56%|█████▌    | 125651/225000 [255:29:59<183:50:02,  6.66s/it]
 56%|█████▌    | 125652/225000 [255:30:03<160:46:17,  5.83s/it]
 56%|█████▌    | 125653/225000 [255:30:11<179:57:08,  6.52s/it]
 56%|█████▌    | 125654/225000 [255:30:15<160:24:45,  5.81s/it]
 56%|█████▌    | 125655/225000 [255:30:19<144:39:30,  5.24s/it]
 56%|█████▌    | 125656/225000 [255:30:28<180:41:54,  6.55s/it]
 56%|█████▌    | 125657/225000 [255:30:32<155:00:26,  5.62s/it]
 56%|█████▌    | 125658/225000 [255:30:35<138:22:22,  5.01s/it]
 56%|█████▌    | 125659/225000 [255:30:44<168:47:31,  6.12s/it]
 56%|█████▌    | 125660/225000 [255:30:49<159:42:49,  5.79s/it]
 56%|█████▌    | 125661/225000 [255:30:55<159:52:52,  5.79s/it]
 56%|█████▌    | 125662/225000 [255:31:00<156:24:13,  5.67s/it]
 56%|█████▌    | 125663/225000 [255:31:07<161:48:26,  5.86s/it]
 56%|█████▌    | 125664/225000 [255:31:14<176:30:46,  6.40s/it]
 56%|█████▌    | 125665/225000 [255:31:18<157:40:06,  5.71s/it]
 56%|█████▌    | 125666/225000 [255:31:23<149:58:47,  5.44s/it]
 56%|█████▌    | 125667/225000 [255:31:27<139:40:02,  5.06s/it]
 56%|█████▌    | 125668/225000 [255:31:32<134:54:31,  4.89s/it]
 56%|█████▌    | 125669/225000 [255:31:43<184:39:16,  6.69s/it]
 56%|█████▌    | 125670/225000 [255:31:47<162:35:59,  5.89s/it]
 56%|█████▌    | 125671/225000 [255:31:53<164:12:24,  5.95s/it]
 56%|█████▌    | 125672/225000 [255:32:00<176:46:31,  6.41s/it]
 56%|█████▌    | 125673/225000 [255:32:04<154:22:41,  5.60s/it]
 56%|█████▌    | 125674/225000 [255:32:09<145:03:36,  5.26s/it]
 56%|█████▌    | 125675/225000 [255:32:12<133:49:55,  4.85s/it]
 56%|█████▌    | 125676/225000 [255:32:22<175:46:28,  6.37s/it]
 56%|█████▌    | 125677/225000 [255:32:26<155:49:16,  5.65s/it]
 56%|█████▌    | 125678/225000 [255:32:30<141:30:13,  5.13s/it]
 56%|█████▌    | 125679/225000 [255:32:40<178:18:49,  6.46s/it]
 56%|█████▌    | 125680/225000 [255:32:43<152:48:52,  5.54s/it]
 56%|█████▌    | 125681/225000 [255:32:48<145:07:00,  5.26s/it]
 56%|█████▌    | 125682/225000 [255:32:52<139:13:39,  5.05s/it]
 56%|█████▌    | 125683/225000 [255:33:03<189:36:53,  6.87s/it]
 56%|█████▌    | 125684/225000 [255:33:07<163:35:06,  5.93s/it]
 56%|█████▌    | 125685/225000 [255:33:13<165:30:36,  6.00s/it]
 56%|█████▌    | 125686/225000 [255:33:21<176:01:22,  6.38s/it]
 56%|█████▌    | 125687/225000 [255:33:25<162:58:08,  5.91s/it]
 56%|█████▌    | 125688/225000 [255:33:32<170:50:44,  6.19s/it]
 56%|█████▌    | 125689/225000 [255:33:40<183:36:13,  6.66s/it]
 56%|█████▌    | 125690/225000 [255:33:46<174:19:11,  6.32s/it]
 56%|█████▌    | 125691/225000 [255:33:50<160:46:44,  5.83s/it]
 56%|█████▌    | 125692/225000 [255:33:55<148:35:00,  5.39s/it]
 56%|█████▌    | 125693/225000 [255:33:59<142:31:18,  5.17s/it]
 56%|█████▌    | 125694/225000 [255:34:09<178:46:42,  6.48s/it]
 56%|█████▌    | 125695/225000 [255:34:15<179:39:49,  6.51s/it]
 56%|█████▌    | 125696/225000 [255:34:20<167:58:04,  6.09s/it]
 56%|█████▌    | 125697/225000 [255:34:27<174:14:52,  6.32s/it]
 56%|█████▌    | 125698/225000 [255:34:31<153:30:00,  5.56s/it]
 56%|█████▌    | 125699/225000 [255:34:40<179:34:10,  6.51s/it]
 56%|█████▌    | 125700/225000 [255:34:45<171:32:33,  6.22s/it]
                                                               
{'loss': '0.839', 'grad_norm': '1.057', 'learning_rate': '2.203e-05', 'epoch': '0.5587', 'num_input_tokens_seen': 425415784, 'train_runtime': '9.201e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125700/225000 [255:34:45<171:32:33,  6.22s/it]
 56%|█████▌    | 125701/225000 [255:34:50<157:19:25,  5.70s/it]
 56%|█████▌    | 125702/225000 [255:34:54<148:10:01,  5.37s/it]
 56%|█████▌    | 125703/225000 [255:34:59<140:24:45,  5.09s/it]
 56%|█████▌    | 125704/225000 [255:35:04<139:11:30,  5.05s/it]
 56%|█████▌    | 125705/225000 [255:35:09<137:59:28,  5.00s/it]
 56%|█████▌    | 125706/225000 [255:35:20<185:49:07,  6.74s/it]
 56%|█████▌    | 125707/225000 [255:35:27<192:36:21,  6.98s/it]
 56%|█████▌    | 125708/225000 [255:35:31<164:17:16,  5.96s/it]
 56%|█████▌    | 125709/225000 [255:35:38<173:00:15,  6.27s/it]
 56%|█████▌    | 125710/225000 [255:35:43<165:21:18,  6.00s/it]
 56%|█████▌    | 125711/225000 [255:35:49<162:31:16,  5.89s/it]
 56%|█████▌    | 125712/225000 [255:35:55<169:10:37,  6.13s/it]
 56%|█████▌    | 125713/225000 [255:36:02<176:04:05,  6.38s/it]
 56%|█████▌    | 125714/225000 [255:36:06<150:45:38,  5.47s/it]
 56%|█████▌    | 125715/225000 [255:36:11<150:23:43,  5.45s/it]
 56%|█████▌    | 125716/225000 [255:36:16<144:08:42,  5.23s/it]
 56%|█████▌    | 125717/225000 [255:36:21<142:33:25,  5.17s/it]
 56%|█████▌    | 125718/225000 [255:36:33<202:15:58,  7.33s/it]
 56%|█████▌    | 125719/225000 [255:36:37<173:14:20,  6.28s/it]
 56%|█████▌    | 125720/225000 [255:36:42<160:28:59,  5.82s/it]
 56%|█████▌    | 125721/225000 [255:36:50<177:59:30,  6.45s/it]
 56%|█████▌    | 125722/225000 [255:36:53<155:50:33,  5.65s/it]
 56%|█████▌    | 125723/225000 [255:36:58<142:40:22,  5.17s/it]
 56%|█████▌    | 125724/225000 [255:37:08<182:17:37,  6.61s/it]
 56%|█████▌    | 125725/225000 [255:37:11<160:27:49,  5.82s/it]
 56%|█████▌    | 125726/225000 [255:37:15<142:34:59,  5.17s/it]
 56%|█████▌    | 125727/225000 [255:37:25<185:29:23,  6.73s/it]
 56%|█████▌    | 125728/225000 [255:37:29<159:02:53,  5.77s/it]
 56%|█████▌    | 125729/225000 [255:37:33<143:29:54,  5.20s/it]
 56%|█████▌    | 125730/225000 [255:37:38<141:02:33,  5.11s/it]
 56%|█████▌    | 125731/225000 [255:37:43<142:01:11,  5.15s/it]
 56%|█████▌    | 125732/225000 [255:37:49<145:45:32,  5.29s/it]
 56%|█████▌    | 125733/225000 [255:37:58<175:33:21,  6.37s/it]
 56%|█████▌    | 125734/225000 [255:38:04<172:58:42,  6.27s/it]
 56%|█████▌    | 125735/225000 [255:38:11<183:32:18,  6.66s/it]
 56%|█████▌    | 125736/225000 [255:38:15<158:22:42,  5.74s/it]
 56%|█████▌    | 125737/225000 [255:38:19<148:15:14,  5.38s/it]
 56%|█████▌    | 125738/225000 [255:38:28<177:01:13,  6.42s/it]
 56%|█████▌    | 125739/225000 [255:38:32<157:36:24,  5.72s/it]
 56%|█████▌    | 125740/225000 [255:38:36<142:54:50,  5.18s/it]
 56%|█████▌    | 125741/225000 [255:38:46<178:21:41,  6.47s/it]
 56%|█████▌    | 125742/225000 [255:38:52<178:08:55,  6.46s/it]
 56%|█████▌    | 125743/225000 [255:38:56<160:28:38,  5.82s/it]
 56%|█████▌    | 125744/225000 [255:39:01<147:52:43,  5.36s/it]
 56%|█████▌    | 125745/225000 [255:39:09<175:32:23,  6.37s/it]
 56%|█████▌    | 125746/225000 [255:39:17<186:48:14,  6.78s/it]
 56%|█████▌    | 125747/225000 [255:39:21<161:50:20,  5.87s/it]
 56%|█████▌    | 125748/225000 [255:39:30<192:11:06,  6.97s/it]
 56%|█████▌    | 125749/225000 [255:39:34<166:41:00,  6.05s/it]
 56%|█████▌    | 125750/225000 [255:39:38<148:18:19,  5.38s/it]
                                                               
{'loss': '0.8123', 'grad_norm': '1.177', 'learning_rate': '2.201e-05', 'epoch': '0.5589', 'num_input_tokens_seen': 425544736, 'train_runtime': '9.204e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125750/225000 [255:39:38<148:18:19,  5.38s/it]
 56%|█████▌    | 125751/225000 [255:39:49<190:03:45,  6.89s/it]
 56%|█████▌    | 125752/225000 [255:39:57<202:32:10,  7.35s/it]
 56%|█████▌    | 125753/225000 [255:40:00<166:05:50,  6.02s/it]
 56%|█████▌    | 125754/225000 [255:40:05<159:10:32,  5.77s/it]
 56%|█████▌    | 125755/225000 [255:40:10<153:17:24,  5.56s/it]
 56%|█████▌    | 125756/225000 [255:40:15<150:19:38,  5.45s/it]
 56%|█████▌    | 125757/225000 [255:40:20<147:14:42,  5.34s/it]
 56%|█████▌    | 125758/225000 [255:40:26<146:33:47,  5.32s/it]
 56%|█████▌    | 125759/225000 [255:40:31<145:50:20,  5.29s/it]
 56%|█████▌    | 125760/225000 [255:40:36<142:14:44,  5.16s/it]
 56%|█████▌    | 125761/225000 [255:40:41<145:07:58,  5.26s/it]
 56%|█████▌    | 125762/225000 [255:40:48<153:08:25,  5.56s/it]
 56%|█████▌    | 125763/225000 [255:40:53<151:18:09,  5.49s/it]
 56%|█████▌    | 125764/225000 [255:40:58<150:32:20,  5.46s/it]
 56%|█████▌    | 125765/225000 [255:41:04<149:06:14,  5.41s/it]
 56%|█████▌    | 125766/225000 [255:41:11<169:05:52,  6.13s/it]
 56%|█████▌    | 125767/225000 [255:41:22<203:50:48,  7.40s/it]
 56%|█████▌    | 125768/225000 [255:41:26<179:25:19,  6.51s/it]
 56%|█████▌    | 125769/225000 [255:41:30<161:30:55,  5.86s/it]
 56%|█████▌    | 125770/225000 [255:41:35<149:43:24,  5.43s/it]
 56%|█████▌    | 125771/225000 [255:41:44<178:58:00,  6.49s/it]
 56%|█████▌    | 125772/225000 [255:41:50<179:29:56,  6.51s/it]
 56%|█████▌    | 125773/225000 [255:41:56<167:55:52,  6.09s/it]
 56%|█████▌    | 125774/225000 [255:42:01<162:28:35,  5.89s/it]
 56%|█████▌    | 125775/225000 [255:42:06<158:41:15,  5.76s/it]
 56%|█████▌    | 125776/225000 [255:42:12<153:45:22,  5.58s/it]
 56%|█████▌    | 125777/225000 [255:42:17<151:17:03,  5.49s/it]
 56%|█████▌    | 125778/225000 [255:42:22<148:24:18,  5.38s/it]
 56%|█████▌    | 125779/225000 [255:42:31<178:36:54,  6.48s/it]
 56%|█████▌    | 125780/225000 [255:42:35<156:42:03,  5.69s/it]
 56%|█████▌    | 125781/225000 [255:42:43<173:00:39,  6.28s/it]
 56%|█████▌    | 125782/225000 [255:42:48<168:35:39,  6.12s/it]
 56%|█████▌    | 125783/225000 [255:42:54<168:43:39,  6.12s/it]
 56%|█████▌    | 125784/225000 [255:42:59<157:18:39,  5.71s/it]
 56%|█████▌    | 125785/225000 [255:43:03<145:14:56,  5.27s/it]
 56%|█████▌    | 125786/225000 [255:43:08<140:23:34,  5.09s/it]
 56%|█████▌    | 125787/225000 [255:43:13<136:19:34,  4.95s/it]
 56%|█████▌    | 125788/225000 [255:43:17<133:05:10,  4.83s/it]
 56%|█████▌    | 125789/225000 [255:43:22<132:39:34,  4.81s/it]
 56%|█████▌    | 125790/225000 [255:43:31<167:02:08,  6.06s/it]
 56%|█████▌    | 125791/225000 [255:43:42<211:37:26,  7.68s/it]
 56%|█████▌    | 125792/225000 [255:43:46<174:56:41,  6.35s/it]
 56%|█████▌    | 125793/225000 [255:43:53<180:52:12,  6.56s/it]
 56%|█████▌    | 125794/225000 [255:43:59<179:41:46,  6.52s/it]
 56%|█████▌    | 125795/225000 [255:44:03<154:07:30,  5.59s/it]
 56%|█████▌    | 125796/225000 [255:44:10<166:48:31,  6.05s/it]
 56%|█████▌    | 125797/225000 [255:44:14<151:39:26,  5.50s/it]
 56%|█████▌    | 125798/225000 [255:44:18<140:08:07,  5.09s/it]
 56%|█████▌    | 125799/225000 [255:44:22<131:56:28,  4.79s/it]
 56%|█████▌    | 125800/225000 [255:44:34<187:16:20,  6.80s/it]
                                                               
{'loss': '0.8271', 'grad_norm': '1.234', 'learning_rate': '2.2e-05', 'epoch': '0.5591', 'num_input_tokens_seen': 425674584, 'train_runtime': '9.207e+05', 'train_tokens_per_second': '462.4'}

 56%|█████▌    | 125800/225000 [255:44:34<187:16:20,  6.80s/it]
 56%|█████▌    | 125801/225000 [255:44:37<159:15:52,  5.78s/it]
 56%|█████▌    | 125802/225000 [255:44:41<143:43:56,  5.22s/it]
 56%|█████▌    | 125803/225000 [255:44:45<135:05:29,  4.90s/it]
 56%|█████▌    | 125804/225000 [255:44:55<173:10:20,  6.28s/it]
 56%|█████▌    | 125805/225000 [255:45:01<173:56:05,  6.31s/it]
 56%|█████▌    | 125806/225000 [255:45:05<151:16:10,  5.49s/it]
 56%|█████▌    | 125807/225000 [255:45:08<133:50:53,  4.86s/it]
 56%|█████▌    | 125808/225000 [255:45:18<179:00:07,  6.50s/it]
 56%|█████▌    | 125809/225000 [255:45:22<152:38:02,  5.54s/it]
 56%|█████▌    | 125810/225000 [255:45:31<181:20:34,  6.58s/it]
 56%|█████▌    | 125811/225000 [255:45:34<151:45:21,  5.51s/it]
 56%|█████▌    | 125812/225000 [255:45:38<145:12:36,  5.27s/it]
 56%|█████▌    | 125813/225000 [255:45:48<184:34:30,  6.70s/it]
 56%|█████▌    | 125814/225000 [255:45:52<158:19:02,  5.75s/it]
 56%|█████▌    | 125815/225000 [255:45:56<142:32:40,  5.17s/it]
 56%|█████▌    | 125816/225000 [255:46:05<176:34:21,  6.41s/it]
 56%|█████▌    | 125817/225000 [255:46:11<170:07:47,  6.18s/it]
 56%|█████▌    | 125818/225000 [255:46:15<157:16:03,  5.71s/it]
 56%|█████▌    | 125819/225000 [255:46:21<154:30:13,  5.61s/it]
 56%|█████▌    | 125820/225000 [255:46:26<154:10:10,  5.60s/it]
 56%|█████▌    | 125821/225000 [255:46:31<149:24:46,  5.42s/it]
 56%|█████▌    | 125822/225000 [255:46:36<146:00:16,  5.30s/it]
 56%|█████▌    | 125823/225000 [255:46:42<147:28:21,  5.35s/it]
 56%|█████▌    | 125824/225000 [255:46:47<146:54:55,  5.33s/it]
 56%|█████▌    | 125825/225000 [255:46:53<148:46:41,  5.40s/it]
 56%|█████▌    | 125826/225000 [255:46:58<146:36:53,  5.32s/it]
 56%|█████▌    | 125827/225000 [255:47:03<145:10:33,  5.27s/it]
 56%|█████▌    | 125828/225000 [255:47:08<142:50:59,  5.19s/it]
 56%|█████▌    | 125829/225000 [255:47:13<143:26:54,  5.21s/it]
 56%|█████▌    | 125830/225000 [255:47:20<160:50:49,  5.84s/it]
 56%|█████▌    | 125831/225000 [255:47:33<213:40:29,  7.76s/it]
 56%|█████▌    | 125832/225000 [255:47:36<180:06:51,  6.54s/it]
 56%|█████▌    | 125833/225000 [255:47:40<158:45:42,  5.76s/it]
 56%|█████▌    | 125834/225000 [255:47:51<203:36:32,  7.39s/it]
 56%|█████▌    | 125835/225000 [255:47:56<178:32:26,  6.48s/it]
 56%|█████▌    | 125836/225000 [255:48:00<158:27:12,  5.75s/it]
 56%|█████▌    | 125837/225000 [255:48:04<147:34:10,  5.36s/it]
 56%|█████▌    | 125838/225000 [255:48:12<169:52:36,  6.17s/it]
 56%|█████▌    | 125839/225000 [255:48:20<177:48:58,  6.46s/it]
 56%|█████▌    | 125840/225000 [255:48:24<158:35:38,  5.76s/it]
 56%|█████▌    | 125841/225000 [255:48:27<141:56:23,  5.15s/it]
 56%|█████▌    | 125842/225000 [255:48:32<133:54:55,  4.86s/it]
 56%|█████▌    | 125843/225000 [255:48:43<184:13:40,  6.69s/it]
 56%|█████▌    | 125844/225000 [255:48:48<171:03:10,  6.21s/it]
 56%|█████▌    | 125845/225000 [255:48:52<152:52:02,  5.55s/it]
 56%|█████▌    | 125846/225000 [255:49:01<184:26:37,  6.70s/it]
 56%|█████▌    | 125847/225000 [255:49:06<170:19:25,  6.18s/it]
 56%|█████▌    | 125848/225000 [255:49:11<161:14:12,  5.85s/it]
 56%|█████▌    | 125849/225000 [255:49:16<152:17:31,  5.53s/it]
 56%|█████▌    | 125850/225000 [255:49:20<143:56:08,  5.23s/it]
                                                               
{'loss': '0.8242', 'grad_norm': '1.201', 'learning_rate': '2.198e-05', 'epoch': '0.5593', 'num_input_tokens_seen': 425803160, 'train_runtime': '9.21e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 125850/225000 [255:49:20<143:56:08,  5.23s/it]
 56%|█████▌    | 125851/225000 [255:49:26<144:46:18,  5.26s/it]
 56%|█████▌    | 125852/225000 [255:49:31<142:50:12,  5.19s/it]
 56%|█████▌    | 125853/225000 [255:49:35<139:09:44,  5.05s/it]
 56%|█████▌    | 125854/225000 [255:49:48<199:52:29,  7.26s/it]
 56%|█████▌    | 125855/225000 [255:49:53<182:05:10,  6.61s/it]
 56%|█████▌    | 125856/225000 [255:49:56<156:37:46,  5.69s/it]
 56%|█████▌    | 125857/225000 [255:50:01<143:32:02,  5.21s/it]
 56%|█████▌    | 125858/225000 [255:50:05<134:16:23,  4.88s/it]
 56%|█████▌    | 125859/225000 [255:50:13<161:34:57,  5.87s/it]
 56%|█████▌    | 125860/225000 [255:50:21<183:15:32,  6.65s/it]
 56%|█████▌    | 125861/225000 [255:50:25<158:45:05,  5.76s/it]
 56%|█████▌    | 125862/225000 [255:50:30<152:25:40,  5.54s/it]
 56%|█████▌    | 125863/225000 [255:50:35<148:33:26,  5.39s/it]
 56%|█████▌    | 125864/225000 [255:50:40<145:46:06,  5.29s/it]
 56%|█████▌    | 125865/225000 [255:50:45<143:06:54,  5.20s/it]
 56%|█████▌    | 125866/225000 [255:50:56<191:03:05,  6.94s/it]
 56%|█████▌    | 125867/225000 [255:51:03<186:25:05,  6.77s/it]
 56%|█████▌    | 125868/225000 [255:51:08<175:58:27,  6.39s/it]
 56%|█████▌    | 125869/225000 [255:51:14<175:48:51,  6.38s/it]
 56%|█████▌    | 125870/225000 [255:51:18<152:05:02,  5.52s/it]
 56%|█████▌    | 125871/225000 [255:51:22<143:31:49,  5.21s/it]
 56%|█████▌    | 125872/225000 [255:51:32<181:41:18,  6.60s/it]
 56%|█████▌    | 125873/225000 [255:51:40<187:35:43,  6.81s/it]
 56%|█████▌    | 125874/225000 [255:51:43<162:36:34,  5.91s/it]
 56%|█████▌    | 125875/225000 [255:51:49<161:28:36,  5.86s/it]
 56%|█████▌    | 125876/225000 [255:51:59<191:50:45,  6.97s/it]
 56%|█████▌    | 125877/225000 [255:52:04<179:21:29,  6.51s/it]
 56%|█████▌    | 125878/225000 [255:52:07<153:39:17,  5.58s/it]
 56%|█████▌    | 125879/225000 [255:52:13<151:49:56,  5.51s/it]
 56%|█████▌    | 125880/225000 [255:52:20<167:00:45,  6.07s/it]
 56%|█████▌    | 125881/225000 [255:52:24<148:25:17,  5.39s/it]
 56%|█████▌    | 125882/225000 [255:52:28<134:58:10,  4.90s/it]
 56%|█████▌    | 125883/225000 [255:52:32<127:55:26,  4.65s/it]
 56%|█████▌    | 125884/225000 [255:52:38<142:00:30,  5.16s/it]
 56%|█████▌    | 125885/225000 [255:52:47<171:59:21,  6.25s/it]
 56%|█████▌    | 125886/225000 [255:52:51<156:00:36,  5.67s/it]
 56%|█████▌    | 125887/225000 [255:52:56<148:27:56,  5.39s/it]
 56%|█████▌    | 125888/225000 [255:53:00<140:27:31,  5.10s/it]
 56%|█████▌    | 125889/225000 [255:53:05<137:36:10,  5.00s/it]
 56%|█████▌    | 125890/225000 [255:53:10<135:34:41,  4.92s/it]
 56%|█████▌    | 125891/225000 [255:53:16<147:48:05,  5.37s/it]
 56%|█████▌    | 125892/225000 [255:53:29<209:18:15,  7.60s/it]
 56%|█████▌    | 125893/225000 [255:53:34<185:39:04,  6.74s/it]
 56%|█████▌    | 125894/225000 [255:53:38<165:27:46,  6.01s/it]
 56%|█████▌    | 125895/225000 [255:53:42<148:59:50,  5.41s/it]
 56%|█████▌    | 125896/225000 [255:53:47<144:57:09,  5.27s/it]
 56%|█████▌    | 125897/225000 [255:54:00<207:10:56,  7.53s/it]
 56%|█████▌    | 125898/225000 [255:54:04<176:27:09,  6.41s/it]
 56%|█████▌    | 125899/225000 [255:54:08<157:36:05,  5.73s/it]
 56%|█████▌    | 125900/225000 [255:54:12<144:56:03,  5.27s/it]
                                                               
{'loss': '0.832', 'grad_norm': '1.036', 'learning_rate': '2.196e-05', 'epoch': '0.5596', 'num_input_tokens_seen': 425932016, 'train_runtime': '9.213e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 125900/225000 [255:54:12<144:56:03,  5.27s/it]
 56%|█████▌    | 125901/225000 [255:54:17<138:05:15,  5.02s/it]
 56%|█████▌    | 125902/225000 [255:54:28<195:01:07,  7.08s/it]
 56%|█████▌    | 125903/225000 [255:54:34<181:32:11,  6.59s/it]
 56%|█████▌    | 125904/225000 [255:54:38<157:18:47,  5.71s/it]
 56%|█████▌    | 125905/225000 [255:54:47<185:53:22,  6.75s/it]
 56%|█████▌    | 125906/225000 [255:54:51<166:59:47,  6.07s/it]
 56%|█████▌    | 125907/225000 [255:54:59<183:02:10,  6.65s/it]
 56%|█████▌    | 125908/225000 [255:55:02<151:12:12,  5.49s/it]
 56%|█████▌    | 125909/225000 [255:55:10<175:45:00,  6.39s/it]
 56%|█████▌    | 125910/225000 [255:55:14<149:51:09,  5.44s/it]
 56%|█████▌    | 125911/225000 [255:55:17<136:09:13,  4.95s/it]
 56%|█████▌    | 125912/225000 [255:55:23<144:49:43,  5.26s/it]
 56%|█████▌    | 125913/225000 [255:55:31<161:01:03,  5.85s/it]
 56%|█████▌    | 125914/225000 [255:55:34<142:23:26,  5.17s/it]
 56%|█████▌    | 125915/225000 [255:55:39<140:10:29,  5.09s/it]
 56%|█████▌    | 125916/225000 [255:55:44<139:23:03,  5.06s/it]
 56%|█████▌    | 125917/225000 [255:55:49<140:03:40,  5.09s/it]
 56%|█████▌    | 125918/225000 [255:55:54<138:06:40,  5.02s/it]
 56%|█████▌    | 125919/225000 [255:55:59<135:52:41,  4.94s/it]
 56%|█████▌    | 125920/225000 [255:56:04<135:30:21,  4.92s/it]
 56%|█████▌    | 125921/225000 [255:56:09<136:27:51,  4.96s/it]
 56%|█████▌    | 125922/225000 [255:56:19<180:52:29,  6.57s/it]
 56%|█████▌    | 125923/225000 [255:56:28<202:14:14,  7.35s/it]
 56%|█████▌    | 125924/225000 [255:56:32<173:18:34,  6.30s/it]
 56%|█████▌    | 125925/225000 [255:56:36<153:13:43,  5.57s/it]
 56%|█████▌    | 125926/225000 [255:56:41<148:12:46,  5.39s/it]
 56%|█████▌    | 125927/225000 [255:56:46<142:28:57,  5.18s/it]
 56%|█████▌    | 125928/225000 [255:56:56<187:38:10,  6.82s/it]
 56%|█████▌    | 125929/225000 [255:57:05<198:44:50,  7.22s/it]
 56%|█████▌    | 125930/225000 [255:57:09<171:53:11,  6.25s/it]
 56%|█████▌    | 125931/225000 [255:57:12<149:43:10,  5.44s/it]
 56%|█████▌    | 125932/225000 [255:57:18<157:28:30,  5.72s/it]
 56%|█████▌    | 125933/225000 [255:57:27<176:48:57,  6.43s/it]
 56%|█████▌    | 125934/225000 [255:57:31<161:27:08,  5.87s/it]
 56%|█████▌    | 125935/225000 [255:57:36<150:53:22,  5.48s/it]
 56%|█████▌    | 125936/225000 [255:57:40<145:03:43,  5.27s/it]
 56%|█████▌    | 125937/225000 [255:57:45<140:31:06,  5.11s/it]
 56%|█████▌    | 125938/225000 [255:57:50<139:00:40,  5.05s/it]
 56%|█████▌    | 125939/225000 [255:57:56<144:18:40,  5.24s/it]
 56%|█████▌    | 125940/225000 [255:58:09<207:03:45,  7.52s/it]
 56%|█████▌    | 125941/225000 [255:58:12<176:24:19,  6.41s/it]
 56%|█████▌    | 125942/225000 [255:58:16<150:37:26,  5.47s/it]
 56%|█████▌    | 125943/225000 [255:58:26<186:48:11,  6.79s/it]
 56%|█████▌    | 125944/225000 [255:58:30<167:10:30,  6.08s/it]
 56%|█████▌    | 125945/225000 [255:58:34<149:45:33,  5.44s/it]
 56%|█████▌    | 125946/225000 [255:58:39<143:27:18,  5.21s/it]
 56%|█████▌    | 125947/225000 [255:58:43<139:01:44,  5.05s/it]
 56%|█████▌    | 125948/225000 [255:58:47<131:19:33,  4.77s/it]
 56%|█████▌    | 125949/225000 [255:58:55<158:05:02,  5.75s/it]
 56%|█████▌    | 125950/225000 [255:59:05<191:23:23,  6.96s/it]
                                                               
{'loss': '0.8334', 'grad_norm': '0.986', 'learning_rate': '2.194e-05', 'epoch': '0.5598', 'num_input_tokens_seen': 426060448, 'train_runtime': '9.215e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 125950/225000 [255:59:05<191:23:23,  6.96s/it]
 56%|█████▌    | 125951/225000 [255:59:09<162:03:49,  5.89s/it]
 56%|█████▌    | 125952/225000 [255:59:14<155:49:23,  5.66s/it]
 56%|█████▌    | 125953/225000 [255:59:19<151:04:56,  5.49s/it]
 56%|█████▌    | 125954/225000 [255:59:24<144:33:49,  5.25s/it]
 56%|█████▌    | 125955/225000 [255:59:29<147:37:44,  5.37s/it]
 56%|█████▌    | 125956/225000 [255:59:34<143:33:02,  5.22s/it]
 56%|█████▌    | 125957/225000 [255:59:39<140:02:41,  5.09s/it]
 56%|█████▌    | 125958/225000 [255:59:50<188:51:16,  6.86s/it]
 56%|█████▌    | 125959/225000 [255:59:57<193:23:31,  7.03s/it]
 56%|█████▌    | 125960/225000 [256:00:01<168:20:54,  6.12s/it]
 56%|█████▌    | 125961/225000 [256:00:05<151:49:04,  5.52s/it]
 56%|█████▌    | 125962/225000 [256:00:16<190:55:04,  6.94s/it]
 56%|█████▌    | 125963/225000 [256:00:22<187:19:56,  6.81s/it]
 56%|█████▌    | 125964/225000 [256:00:26<159:42:08,  5.81s/it]
 56%|█████▌    | 125965/225000 [256:00:35<192:39:19,  7.00s/it]
 56%|█████▌    | 125966/225000 [256:00:39<164:00:39,  5.96s/it]
 56%|█████▌    | 125967/225000 [256:00:43<151:02:02,  5.49s/it]
 56%|█████▌    | 125968/225000 [256:00:48<146:17:59,  5.32s/it]
 56%|█████▌    | 125969/225000 [256:00:53<141:42:13,  5.15s/it]
 56%|█████▌    | 125970/225000 [256:00:58<140:05:58,  5.09s/it]
 56%|█████▌    | 125971/225000 [256:01:02<134:58:01,  4.91s/it]
 56%|█████▌    | 125972/225000 [256:01:08<137:34:07,  5.00s/it]
 56%|█████▌    | 125973/225000 [256:01:13<138:44:07,  5.04s/it]
 56%|█████▌    | 125974/225000 [256:01:20<154:07:30,  5.60s/it]
 56%|█████▌    | 125975/225000 [256:01:33<215:48:42,  7.85s/it]
 56%|█████▌    | 125976/225000 [256:01:37<182:35:51,  6.64s/it]
 56%|█████▌    | 125977/225000 [256:01:40<158:32:37,  5.76s/it]
 56%|█████▌    | 125978/225000 [256:01:45<145:27:17,  5.29s/it]
 56%|█████▌    | 125979/225000 [256:01:49<142:02:28,  5.16s/it]
 56%|█████▌    | 125980/225000 [256:01:55<143:49:50,  5.23s/it]
 56%|█████▌    | 125981/225000 [256:02:00<141:04:26,  5.13s/it]
 56%|█████▌    | 125982/225000 [256:02:05<141:04:03,  5.13s/it]
 56%|█████▌    | 125983/225000 [256:02:10<140:05:43,  5.09s/it]
 56%|█████▌    | 125984/225000 [256:02:21<191:14:58,  6.95s/it]
 56%|█████▌    | 125985/225000 [256:02:28<190:42:58,  6.93s/it]
 56%|█████▌    | 125986/225000 [256:02:32<169:29:15,  6.16s/it]
 56%|█████▌    | 125987/225000 [256:02:37<154:14:11,  5.61s/it]
 56%|█████▌    | 125988/225000 [256:02:41<142:44:32,  5.19s/it]
 56%|█████▌    | 125989/225000 [256:02:46<143:05:41,  5.20s/it]
 56%|█████▌    | 125990/225000 [256:02:59<207:15:19,  7.54s/it]
 56%|█████▌    | 125991/225000 [256:03:03<178:44:49,  6.50s/it]
 56%|█████▌    | 125992/225000 [256:03:10<178:34:49,  6.49s/it]
 56%|█████▌    | 125993/225000 [256:03:13<154:44:07,  5.63s/it]
 56%|█████▌    | 125994/225000 [256:03:23<188:17:45,  6.85s/it]
 56%|█████▌    | 125995/225000 [256:03:27<168:12:52,  6.12s/it]
 56%|█████▌    | 125996/225000 [256:03:32<154:28:50,  5.62s/it]
 56%|█████▌    | 125997/225000 [256:03:37<146:57:44,  5.34s/it]
 56%|█████▌    | 125998/225000 [256:03:41<138:12:59,  5.03s/it]
 56%|█████▌    | 125999/225000 [256:03:45<134:18:44,  4.88s/it]
 56%|█████▌    | 126000/225000 [256:03:50<135:19:26,  4.92s/it]
                                                               
{'loss': '0.8335', 'grad_norm': '1.06', 'learning_rate': '2.192e-05', 'epoch': '0.56', 'num_input_tokens_seen': 426189824, 'train_runtime': '9.218e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126000/225000 [256:03:50<135:19:26,  4.92s/it]
 56%|█████▌    | 126001/225000 [256:03:57<149:08:50,  5.42s/it]
 56%|█████▌    | 126002/225000 [256:04:07<190:43:17,  6.94s/it]
 56%|█████▌    | 126003/225000 [256:04:12<169:31:35,  6.16s/it]
 56%|█████▌    | 126004/225000 [256:04:20<182:33:08,  6.64s/it]
 56%|█████▌    | 126005/225000 [256:04:24<160:23:37,  5.83s/it]
 56%|█████▌    | 126006/225000 [256:04:28<149:06:20,  5.42s/it]
 56%|█████▌    | 126007/225000 [256:04:32<141:13:02,  5.14s/it]
 56%|█████▌    | 126008/225000 [256:04:37<136:26:33,  4.96s/it]
 56%|█████▌    | 126009/225000 [256:04:48<187:13:51,  6.81s/it]
 56%|█████▌    | 126010/225000 [256:04:54<175:57:32,  6.40s/it]
 56%|█████▌    | 126011/225000 [256:04:58<157:34:39,  5.73s/it]
 56%|█████▌    | 126012/225000 [256:05:02<147:28:36,  5.36s/it]
 56%|█████▌    | 126013/225000 [256:05:06<136:55:12,  4.98s/it]
 56%|█████▌    | 126014/225000 [256:05:11<131:53:29,  4.80s/it]
 56%|█████▌    | 126015/225000 [256:05:19<163:33:33,  5.95s/it]
 56%|█████▌    | 126016/225000 [256:05:28<188:02:39,  6.84s/it]
 56%|█████▌    | 126017/225000 [256:05:32<160:39:26,  5.84s/it]
 56%|█████▌    | 126018/225000 [256:05:38<165:34:52,  6.02s/it]
 56%|█████▌    | 126019/225000 [256:05:46<176:08:22,  6.41s/it]
 56%|█████▌    | 126020/225000 [256:05:52<178:47:26,  6.50s/it]
 56%|█████▌    | 126021/225000 [256:05:58<174:02:33,  6.33s/it]
 56%|█████▌    | 126022/225000 [256:06:02<152:06:08,  5.53s/it]
 56%|█████▌    | 126023/225000 [256:06:10<173:43:44,  6.32s/it]
 56%|█████▌    | 126024/225000 [256:06:14<151:30:04,  5.51s/it]
 56%|█████▌    | 126025/225000 [256:06:22<173:35:44,  6.31s/it]
 56%|█████▌    | 126026/225000 [256:06:25<146:52:25,  5.34s/it]
 56%|█████▌    | 126027/225000 [256:06:33<166:02:25,  6.04s/it]
 56%|█████▌    | 126028/225000 [256:06:37<152:08:53,  5.53s/it]
 56%|█████▌    | 126029/225000 [256:06:45<175:16:51,  6.38s/it]
 56%|█████▌    | 126030/225000 [256:06:51<167:39:59,  6.10s/it]
 56%|█████▌    | 126031/225000 [256:06:54<148:36:25,  5.41s/it]
 56%|█████▌    | 126032/225000 [256:06:58<135:53:36,  4.94s/it]
 56%|█████▌    | 126033/225000 [256:07:02<129:07:23,  4.70s/it]
 56%|█████▌    | 126034/225000 [256:07:13<176:13:31,  6.41s/it]
 56%|█████▌    | 126035/225000 [256:07:18<168:17:14,  6.12s/it]
 56%|█████▌    | 126036/225000 [256:07:26<178:39:41,  6.50s/it]
 56%|█████▌    | 126037/225000 [256:07:31<166:07:23,  6.04s/it]
 56%|█████▌    | 126038/225000 [256:07:35<154:59:16,  5.64s/it]
 56%|█████▌    | 126039/225000 [256:07:40<146:21:51,  5.32s/it]
 56%|█████▌    | 126040/225000 [256:07:44<138:54:04,  5.05s/it]
 56%|█████▌    | 126041/225000 [256:07:48<131:02:37,  4.77s/it]
 56%|█████▌    | 126042/225000 [256:08:00<189:44:40,  6.90s/it]
 56%|█████▌    | 126043/225000 [256:08:05<167:01:15,  6.08s/it]
 56%|█████▌    | 126044/225000 [256:08:11<173:08:12,  6.30s/it]
 56%|█████▌    | 126045/225000 [256:08:15<149:39:25,  5.44s/it]
 56%|█████▌    | 126046/225000 [256:08:23<173:53:14,  6.33s/it]
 56%|█████▌    | 126047/225000 [256:08:27<149:58:28,  5.46s/it]
 56%|█████▌    | 126048/225000 [256:08:35<173:02:07,  6.30s/it]
 56%|█████▌    | 126049/225000 [256:08:39<153:55:37,  5.60s/it]
 56%|█████▌    | 126050/225000 [256:08:48<182:56:23,  6.66s/it]
                                                               
{'loss': '0.8324', 'grad_norm': '1.151', 'learning_rate': '2.191e-05', 'epoch': '0.5602', 'num_input_tokens_seen': 426318864, 'train_runtime': '9.221e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126050/225000 [256:08:48<182:56:23,  6.66s/it]
 56%|█████▌    | 126051/225000 [256:08:52<162:34:37,  5.91s/it]
 56%|█████▌    | 126052/225000 [256:08:56<143:28:14,  5.22s/it]
 56%|█████▌    | 126053/225000 [256:09:00<137:08:03,  4.99s/it]
 56%|█████▌    | 126054/225000 [256:09:11<184:59:51,  6.73s/it]
 56%|█████▌    | 126055/225000 [256:09:14<157:50:52,  5.74s/it]
 56%|█████▌    | 126056/225000 [256:09:23<177:20:05,  6.45s/it]
 56%|█████▌    | 126057/225000 [256:09:26<156:12:23,  5.68s/it]
 56%|█████▌    | 126058/225000 [256:09:30<138:53:41,  5.05s/it]
 56%|█████▌    | 126059/225000 [256:09:39<170:53:20,  6.22s/it]
 56%|█████▌    | 126060/225000 [256:09:43<150:21:33,  5.47s/it]
 56%|█████▌    | 126061/225000 [256:09:52<185:49:22,  6.76s/it]
 56%|█████▌    | 126062/225000 [256:09:56<157:41:34,  5.74s/it]
 56%|█████▌    | 126063/225000 [256:10:04<180:15:22,  6.56s/it]
 56%|█████▌    | 126064/225000 [256:10:08<155:57:14,  5.67s/it]
 56%|█████▌    | 126065/225000 [256:10:11<138:28:50,  5.04s/it]
 56%|█████▌    | 126066/225000 [256:10:16<136:11:02,  4.96s/it]
 56%|█████▌    | 126067/225000 [256:10:21<133:12:03,  4.85s/it]
 56%|█████▌    | 126068/225000 [256:10:26<133:34:05,  4.86s/it]
 56%|█████▌    | 126069/225000 [256:10:37<190:46:32,  6.94s/it]
 56%|█████▌    | 126070/225000 [256:10:42<172:05:29,  6.26s/it]
 56%|█████▌    | 126071/225000 [256:10:49<178:59:02,  6.51s/it]
 56%|█████▌    | 126072/225000 [256:10:54<167:04:00,  6.08s/it]
 56%|█████▌    | 126073/225000 [256:11:01<175:10:17,  6.37s/it]
 56%|█████▌    | 126074/225000 [256:11:09<182:14:19,  6.63s/it]
 56%|█████▌    | 126075/225000 [256:11:12<159:26:25,  5.80s/it]
 56%|█████▌    | 126076/225000 [256:11:16<142:09:54,  5.17s/it]
 56%|█████▌    | 126077/225000 [256:11:26<182:44:42,  6.65s/it]
 56%|█████▌    | 126078/225000 [256:11:30<161:32:15,  5.88s/it]
 56%|█████▌    | 126079/225000 [256:11:34<144:59:48,  5.28s/it]
 56%|█████▌    | 126080/225000 [256:11:43<174:52:54,  6.36s/it]
 56%|█████▌    | 126081/225000 [256:11:49<171:03:50,  6.23s/it]
 56%|█████▌    | 126082/225000 [256:11:54<160:41:37,  5.85s/it]
 56%|█████▌    | 126083/225000 [256:11:59<155:00:22,  5.64s/it]
 56%|█████▌    | 126084/225000 [256:12:06<167:10:57,  6.08s/it]
 56%|█████▌    | 126085/225000 [256:12:11<158:09:58,  5.76s/it]
 56%|█████▌    | 126086/225000 [256:12:17<154:32:21,  5.62s/it]
 56%|█████▌    | 126087/225000 [256:12:22<153:28:34,  5.59s/it]
 56%|█████▌    | 126088/225000 [256:12:30<173:26:11,  6.31s/it]
 56%|█████▌    | 126089/225000 [256:12:35<160:48:05,  5.85s/it]
 56%|█████▌    | 126090/225000 [256:12:39<146:16:42,  5.32s/it]
 56%|█████▌    | 126091/225000 [256:12:43<138:57:25,  5.06s/it]
 56%|█████▌    | 126092/225000 [256:12:48<135:17:49,  4.92s/it]
 56%|█████▌    | 126093/225000 [256:12:55<155:40:48,  5.67s/it]
 56%|█████▌    | 126094/225000 [256:13:06<200:12:22,  7.29s/it]
 56%|█████▌    | 126095/225000 [256:13:10<170:08:11,  6.19s/it]
 56%|█████▌    | 126096/225000 [256:13:18<183:21:54,  6.67s/it]
 56%|█████▌    | 126097/225000 [256:13:21<157:14:48,  5.72s/it]
 56%|█████▌    | 126098/225000 [256:13:29<175:00:59,  6.37s/it]
 56%|█████▌    | 126099/225000 [256:13:32<148:41:12,  5.41s/it]
 56%|█████▌    | 126100/225000 [256:13:41<173:31:44,  6.32s/it]
                                                               
{'loss': '0.8277', 'grad_norm': '1.115', 'learning_rate': '2.189e-05', 'epoch': '0.5604', 'num_input_tokens_seen': 426448016, 'train_runtime': '9.224e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126100/225000 [256:13:41<173:31:44,  6.32s/it]
 56%|█████▌    | 126101/225000 [256:13:45<155:03:38,  5.64s/it]
 56%|█████▌    | 126102/225000 [256:13:49<142:58:00,  5.20s/it]
 56%|█████▌    | 126103/225000 [256:13:53<133:38:52,  4.86s/it]
 56%|█████▌    | 126104/225000 [256:14:03<172:51:27,  6.29s/it]
 56%|█████▌    | 126105/225000 [256:14:08<164:28:38,  5.99s/it]
 56%|█████▌    | 126106/225000 [256:14:12<150:13:50,  5.47s/it]
 56%|█████▌    | 126107/225000 [256:14:16<138:31:34,  5.04s/it]
 56%|█████▌    | 126108/225000 [256:14:23<153:32:42,  5.59s/it]
 56%|█████▌    | 126109/225000 [256:14:31<172:16:56,  6.27s/it]
 56%|█████▌    | 126110/225000 [256:14:36<161:15:22,  5.87s/it]
 56%|█████▌    | 126111/225000 [256:14:41<153:53:05,  5.60s/it]
 56%|█████▌    | 126112/225000 [256:14:46<148:38:38,  5.41s/it]
 56%|█████▌    | 126113/225000 [256:14:51<144:03:38,  5.24s/it]
 56%|█████▌    | 126114/225000 [256:14:56<139:49:09,  5.09s/it]
 56%|█████▌    | 126115/225000 [256:15:01<140:59:50,  5.13s/it]
 56%|█████▌    | 126116/225000 [256:15:11<178:44:05,  6.51s/it]
 56%|█████▌    | 126117/225000 [256:15:20<200:51:04,  7.31s/it]
 56%|█████▌    | 126118/225000 [256:15:24<176:14:13,  6.42s/it]
 56%|█████▌    | 126119/225000 [256:15:31<179:00:44,  6.52s/it]
 56%|█████▌    | 126120/225000 [256:15:38<180:55:32,  6.59s/it]
 56%|█████▌    | 126121/225000 [256:15:41<156:06:35,  5.68s/it]
 56%|█████▌    | 126122/225000 [256:15:46<148:08:25,  5.39s/it]
 56%|█████▌    | 126123/225000 [256:15:51<143:58:02,  5.24s/it]
 56%|█████▌    | 126124/225000 [256:15:56<143:22:29,  5.22s/it]
 56%|█████▌    | 126125/225000 [256:16:01<145:30:51,  5.30s/it]
 56%|█████▌    | 126126/225000 [256:16:06<143:01:45,  5.21s/it]
 56%|█████▌    | 126127/225000 [256:16:12<142:43:39,  5.20s/it]
 56%|█████▌    | 126128/225000 [256:16:17<143:10:25,  5.21s/it]
 56%|█████▌    | 126129/225000 [256:16:24<159:40:51,  5.81s/it]
 56%|█████▌    | 126130/225000 [256:16:35<199:30:49,  7.26s/it]
 56%|█████▌    | 126131/225000 [256:16:38<165:48:52,  6.04s/it]
 56%|█████▌    | 126132/225000 [256:16:42<148:22:20,  5.40s/it]
 56%|█████▌    | 126133/225000 [256:16:47<144:31:44,  5.26s/it]
 56%|█████▌    | 126134/225000 [256:16:52<144:24:26,  5.26s/it]
 56%|█████▌    | 126135/225000 [256:16:57<141:38:10,  5.16s/it]
 56%|█████▌    | 126136/225000 [256:17:03<145:58:26,  5.32s/it]
 56%|█████▌    | 126137/225000 [256:17:08<144:25:06,  5.26s/it]
 56%|█████▌    | 126138/225000 [256:17:13<143:55:27,  5.24s/it]
 56%|█████▌    | 126139/225000 [256:17:19<147:17:34,  5.36s/it]
 56%|█████▌    | 126140/225000 [256:17:31<206:45:47,  7.53s/it]
 56%|█████▌    | 126141/225000 [256:17:38<197:29:17,  7.19s/it]
 56%|█████▌    | 126142/225000 [256:17:45<200:44:13,  7.31s/it]
 56%|█████▌    | 126143/225000 [256:17:49<169:10:52,  6.16s/it]
 56%|█████▌    | 126144/225000 [256:17:57<190:07:51,  6.92s/it]
 56%|█████▌    | 126145/225000 [256:18:01<161:16:34,  5.87s/it]
 56%|█████▌    | 126146/225000 [256:18:04<142:36:57,  5.19s/it]
 56%|█████▌    | 126147/225000 [256:18:15<186:54:10,  6.81s/it]
 56%|█████▌    | 126148/225000 [256:18:19<164:28:26,  5.99s/it]
 56%|█████▌    | 126149/225000 [256:18:23<144:43:00,  5.27s/it]
 56%|█████▌    | 126150/225000 [256:18:27<140:18:50,  5.11s/it]
                                                               
{'loss': '0.829', 'grad_norm': '1.097', 'learning_rate': '2.187e-05', 'epoch': '0.5607', 'num_input_tokens_seen': 426577752, 'train_runtime': '9.227e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126150/225000 [256:18:27<140:18:50,  5.11s/it]
 56%|█████▌    | 126151/225000 [256:18:32<136:41:54,  4.98s/it]
 56%|█████▌    | 126152/225000 [256:18:37<139:23:30,  5.08s/it]
 56%|█████▌    | 126153/225000 [256:18:42<139:03:10,  5.06s/it]
 56%|█████▌    | 126154/225000 [256:18:54<190:04:00,  6.92s/it]
 56%|█████▌    | 126155/225000 [256:19:01<191:32:11,  6.98s/it]
 56%|█████▌    | 126156/225000 [256:19:05<168:40:29,  6.14s/it]
 56%|█████▌    | 126157/225000 [256:19:09<153:33:57,  5.59s/it]
 56%|█████▌    | 126158/225000 [256:19:19<189:36:48,  6.91s/it]
 56%|█████▌    | 126159/225000 [256:19:24<169:25:42,  6.17s/it]
 56%|█████▌    | 126160/225000 [256:19:32<184:41:57,  6.73s/it]
 56%|█████▌    | 126161/225000 [256:19:36<160:56:12,  5.86s/it]
 56%|█████▌    | 126162/225000 [256:19:43<177:30:44,  6.47s/it]
 56%|█████▌    | 126163/225000 [256:19:48<162:48:57,  5.93s/it]
 56%|█████▌    | 126164/225000 [256:19:56<180:55:57,  6.59s/it]
 56%|█████▌    | 126165/225000 [256:20:00<159:41:42,  5.82s/it]
 56%|█████▌    | 126166/225000 [256:20:08<176:43:22,  6.44s/it]
 56%|█████▌    | 126167/225000 [256:20:12<153:52:04,  5.60s/it]
 56%|█████▌    | 126168/225000 [256:20:18<155:57:00,  5.68s/it]
 56%|█████▌    | 126169/225000 [256:20:22<146:00:09,  5.32s/it]
 56%|█████▌    | 126170/225000 [256:20:27<141:21:02,  5.15s/it]
 56%|█████▌    | 126171/225000 [256:20:32<142:20:57,  5.19s/it]
 56%|█████▌    | 126172/225000 [256:20:37<142:55:54,  5.21s/it]
 56%|█████▌    | 126173/225000 [256:20:42<139:24:11,  5.08s/it]
 56%|█████▌    | 126174/225000 [256:20:48<141:55:22,  5.17s/it]
 56%|█████▌    | 126175/225000 [256:20:54<150:10:56,  5.47s/it]
 56%|█████▌    | 126176/225000 [256:21:06<209:12:41,  7.62s/it]
 56%|█████▌    | 126177/225000 [256:21:13<201:20:43,  7.33s/it]
 56%|█████▌    | 126178/225000 [256:21:16<169:22:18,  6.17s/it]
 56%|█████▌    | 126179/225000 [256:21:24<178:43:29,  6.51s/it]
 56%|█████▌    | 126180/225000 [256:21:30<174:25:15,  6.35s/it]
 56%|█████▌    | 126181/225000 [256:21:35<165:42:17,  6.04s/it]
 56%|█████▌    | 126182/225000 [256:21:40<158:09:45,  5.76s/it]
 56%|█████▌    | 126183/225000 [256:21:44<140:27:36,  5.12s/it]
 56%|█████▌    | 126184/225000 [256:21:53<176:56:38,  6.45s/it]
 56%|█████▌    | 126185/225000 [256:21:57<151:44:39,  5.53s/it]
 56%|█████▌    | 126186/225000 [256:22:01<145:32:00,  5.30s/it]
 56%|█████▌    | 126187/225000 [256:22:06<142:39:09,  5.20s/it]
 56%|█████▌    | 126188/225000 [256:22:12<145:33:26,  5.30s/it]
 56%|█████▌    | 126189/225000 [256:22:17<144:26:19,  5.26s/it]
 56%|█████▌    | 126190/225000 [256:22:22<142:12:47,  5.18s/it]
 56%|█████▌    | 126191/225000 [256:22:34<194:00:56,  7.07s/it]
 56%|█████▌    | 126192/225000 [256:22:40<186:58:43,  6.81s/it]
 56%|█████▌    | 126193/225000 [256:22:44<164:15:59,  5.98s/it]
 56%|█████▌    | 126194/225000 [256:22:52<183:13:24,  6.68s/it]
 56%|█████▌    | 126195/225000 [256:22:56<157:05:11,  5.72s/it]
 56%|█████▌    | 126196/225000 [256:23:04<179:29:45,  6.54s/it]
 56%|█████▌    | 126197/225000 [256:23:08<159:00:12,  5.79s/it]
 56%|█████▌    | 126198/225000 [256:23:12<145:55:57,  5.32s/it]
 56%|█████▌    | 126199/225000 [256:23:20<163:29:28,  5.96s/it]
 56%|█████▌    | 126200/225000 [256:23:28<184:00:07,  6.70s/it]
                                                               
{'loss': '0.8296', 'grad_norm': '1.101', 'learning_rate': '2.185e-05', 'epoch': '0.5609', 'num_input_tokens_seen': 426707960, 'train_runtime': '9.23e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126200/225000 [256:23:28<184:00:07,  6.70s/it]
 56%|█████▌    | 126201/225000 [256:23:32<159:29:41,  5.81s/it]
 56%|█████▌    | 126202/225000 [256:23:37<155:11:01,  5.65s/it]
 56%|█████▌    | 126203/225000 [256:23:45<173:06:57,  6.31s/it]
 56%|█████▌    | 126204/225000 [256:23:49<153:09:45,  5.58s/it]
 56%|█████▌    | 126205/225000 [256:23:56<164:37:35,  6.00s/it]
 56%|█████▌    | 126206/225000 [256:23:59<143:18:22,  5.22s/it]
 56%|█████▌    | 126207/225000 [256:24:08<168:03:40,  6.12s/it]
 56%|█████▌    | 126208/225000 [256:24:12<151:13:19,  5.51s/it]
 56%|█████▌    | 126209/225000 [256:24:16<139:32:31,  5.08s/it]
 56%|█████▌    | 126210/225000 [256:24:20<135:28:58,  4.94s/it]
 56%|█████▌    | 126211/225000 [256:24:30<175:05:20,  6.38s/it]
 56%|█████▌    | 126212/225000 [256:24:35<158:36:30,  5.78s/it]
 56%|█████▌    | 126213/225000 [256:24:39<146:16:55,  5.33s/it]
 56%|█████▌    | 126214/225000 [256:24:43<138:23:23,  5.04s/it]
 56%|█████▌    | 126215/225000 [256:24:48<133:21:51,  4.86s/it]
 56%|█████▌    | 126216/225000 [256:24:58<179:08:14,  6.53s/it]
 56%|█████▌    | 126217/225000 [256:25:05<184:56:17,  6.74s/it]
 56%|█████▌    | 126218/225000 [256:25:09<159:15:37,  5.80s/it]
 56%|█████▌    | 126219/225000 [256:25:19<191:54:41,  6.99s/it]
 56%|█████▌    | 126220/225000 [256:25:22<160:30:36,  5.85s/it]
 56%|█████▌    | 126221/225000 [256:25:30<179:22:08,  6.54s/it]
 56%|█████▌    | 126222/225000 [256:25:33<153:18:30,  5.59s/it]
 56%|█████▌    | 126223/225000 [256:25:41<169:18:36,  6.17s/it]
 56%|█████▌    | 126224/225000 [256:25:45<153:10:42,  5.58s/it]
 56%|█████▌    | 126225/225000 [256:25:49<141:48:07,  5.17s/it]
 56%|█████▌    | 126226/225000 [256:25:54<136:36:48,  4.98s/it]
 56%|█████▌    | 126227/225000 [256:25:58<133:32:52,  4.87s/it]
 56%|█████▌    | 126228/225000 [256:26:08<173:47:24,  6.33s/it]
 56%|█████▌    | 126229/225000 [256:26:17<194:56:37,  7.11s/it]
 56%|█████▌    | 126230/225000 [256:26:20<160:32:13,  5.85s/it]
 56%|█████▌    | 126231/225000 [256:26:24<146:24:23,  5.34s/it]
 56%|█████▌    | 126232/225000 [256:26:33<173:57:00,  6.34s/it]
 56%|█████▌    | 126233/225000 [256:26:37<154:33:58,  5.63s/it]
 56%|█████▌    | 126234/225000 [256:26:41<141:09:26,  5.15s/it]
 56%|█████▌    | 126235/225000 [256:26:45<131:09:44,  4.78s/it]
 56%|█████▌    | 126236/225000 [256:26:51<141:39:28,  5.16s/it]
 56%|█████▌    | 126237/225000 [256:27:02<189:26:12,  6.91s/it]
 56%|█████▌    | 126238/225000 [256:27:05<161:13:29,  5.88s/it]
 56%|█████▌    | 126239/225000 [256:27:14<188:55:11,  6.89s/it]
 56%|█████▌    | 126240/225000 [256:27:18<158:45:13,  5.79s/it]
 56%|█████▌    | 126241/225000 [256:27:26<177:54:15,  6.49s/it]
 56%|█████▌    | 126242/225000 [256:27:29<154:31:03,  5.63s/it]
 56%|█████▌    | 126243/225000 [256:27:38<179:09:36,  6.53s/it]
 56%|█████▌    | 126244/225000 [256:27:42<160:22:10,  5.85s/it]
 56%|█████▌    | 126245/225000 [256:27:48<156:01:25,  5.69s/it]
 56%|█████▌    | 126246/225000 [256:27:53<151:58:40,  5.54s/it]
 56%|█████▌    | 126247/225000 [256:27:58<147:51:58,  5.39s/it]
 56%|█████▌    | 126248/225000 [256:28:03<145:18:46,  5.30s/it]
 56%|█████▌    | 126249/225000 [256:28:07<138:01:06,  5.03s/it]
 56%|█████▌    | 126250/225000 [256:28:13<139:55:12,  5.10s/it]
                                                               
{'loss': '0.8227', 'grad_norm': '1.002', 'learning_rate': '2.183e-05', 'epoch': '0.5611', 'num_input_tokens_seen': 426836976, 'train_runtime': '9.233e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126250/225000 [256:28:13<139:55:12,  5.10s/it]
 56%|█████▌    | 126251/225000 [256:28:18<143:04:50,  5.22s/it]
 56%|█████▌    | 126252/225000 [256:28:23<143:38:15,  5.24s/it]
 56%|█████▌    | 126253/225000 [256:28:29<142:18:07,  5.19s/it]
 56%|█████▌    | 126254/225000 [256:28:34<144:03:18,  5.25s/it]
 56%|█████▌    | 126255/225000 [256:28:40<147:11:08,  5.37s/it]
 56%|█████▌    | 126256/225000 [256:28:45<149:19:15,  5.44s/it]
 56%|█████▌    | 126257/225000 [256:28:58<206:17:20,  7.52s/it]
 56%|█████▌    | 126258/225000 [256:29:03<192:50:07,  7.03s/it]
 56%|█████▌    | 126259/225000 [256:29:11<193:59:56,  7.07s/it]
 56%|█████▌    | 126260/225000 [256:29:17<189:28:37,  6.91s/it]
 56%|█████▌    | 126261/225000 [256:29:21<162:03:28,  5.91s/it]
 56%|█████▌    | 126262/225000 [256:29:30<188:54:01,  6.89s/it]
 56%|█████▌    | 126263/225000 [256:29:33<158:47:39,  5.79s/it]
 56%|█████▌    | 126264/225000 [256:29:41<174:41:52,  6.37s/it]
 56%|█████▌    | 126265/225000 [256:29:46<165:46:00,  6.04s/it]
 56%|█████▌    | 126266/225000 [256:29:49<141:01:07,  5.14s/it]
 56%|█████▌    | 126267/225000 [256:29:59<179:22:27,  6.54s/it]
 56%|█████▌    | 126268/225000 [256:30:02<154:30:24,  5.63s/it]
 56%|█████▌    | 126269/225000 [256:30:10<170:18:10,  6.21s/it]
 56%|█████▌    | 126270/225000 [256:30:14<149:31:47,  5.45s/it]
 56%|█████▌    | 126271/225000 [256:30:20<154:57:05,  5.65s/it]
 56%|█████▌    | 126272/225000 [256:30:28<174:16:24,  6.35s/it]
 56%|█████▌    | 126273/225000 [256:30:33<166:45:24,  6.08s/it]
 56%|█████▌    | 126274/225000 [256:30:39<162:49:51,  5.94s/it]
 56%|█████▌    | 126275/225000 [256:30:45<163:38:07,  5.97s/it]
 56%|█████▌    | 126276/225000 [256:30:49<150:36:43,  5.49s/it]
 56%|█████▌    | 126277/225000 [256:30:54<143:41:04,  5.24s/it]
 56%|█████▌    | 126278/225000 [256:30:59<139:13:35,  5.08s/it]
 56%|█████▌    | 126279/225000 [256:31:03<136:33:51,  4.98s/it]
 56%|█████▌    | 126280/225000 [256:31:08<136:20:29,  4.97s/it]
 56%|█████▌    | 126281/225000 [256:31:13<135:07:01,  4.93s/it]
 56%|█████▌    | 126282/225000 [256:31:18<134:49:01,  4.92s/it]
 56%|█████▌    | 126283/225000 [256:31:23<138:27:52,  5.05s/it]
 56%|█████▌    | 126284/225000 [256:31:36<197:49:45,  7.21s/it]
 56%|█████▌    | 126285/225000 [256:31:41<182:23:55,  6.65s/it]
 56%|█████▌    | 126286/225000 [256:31:48<184:02:24,  6.71s/it]
 56%|█████▌    | 126287/225000 [256:31:54<182:21:13,  6.65s/it]
 56%|█████▌    | 126288/225000 [256:31:58<160:46:21,  5.86s/it]
 56%|█████▌    | 126289/225000 [256:32:03<147:20:47,  5.37s/it]
 56%|█████▌    | 126290/225000 [256:32:07<136:22:45,  4.97s/it]
 56%|█████▌    | 126291/225000 [256:32:11<129:46:56,  4.73s/it]
 56%|█████▌    | 126292/225000 [256:32:16<132:00:51,  4.81s/it]
 56%|█████▌    | 126293/225000 [256:32:21<133:37:00,  4.87s/it]
 56%|█████▌    | 126294/225000 [256:32:26<136:12:35,  4.97s/it]
 56%|█████▌    | 126295/225000 [256:32:31<140:10:35,  5.11s/it]
 56%|█████▌    | 126296/225000 [256:32:37<141:23:36,  5.16s/it]
 56%|█████▌    | 126297/225000 [256:32:43<151:48:29,  5.54s/it]
 56%|█████▌    | 126298/225000 [256:32:56<212:31:26,  7.75s/it]
 56%|█████▌    | 126299/225000 [256:33:00<181:37:00,  6.62s/it]
 56%|█████▌    | 126300/225000 [256:33:04<159:51:24,  5.83s/it]
                                                               
{'loss': '0.8258', 'grad_norm': '1.075', 'learning_rate': '2.181e-05', 'epoch': '0.5613', 'num_input_tokens_seen': 426966456, 'train_runtime': '9.236e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126300/225000 [256:33:04<159:51:24,  5.83s/it]
 56%|█████▌    | 126301/225000 [256:33:11<171:55:46,  6.27s/it]
 56%|█████▌    | 126302/225000 [256:33:17<166:29:45,  6.07s/it]
 56%|█████▌    | 126303/225000 [256:33:23<169:09:56,  6.17s/it]
 56%|█████▌    | 126304/225000 [256:33:31<177:55:50,  6.49s/it]
 56%|█████▌    | 126305/225000 [256:33:35<164:15:29,  5.99s/it]
 56%|█████▌    | 126306/225000 [256:33:42<167:20:53,  6.10s/it]
 56%|█████▌    | 126307/225000 [256:33:50<188:27:38,  6.87s/it]
 56%|█████▌    | 126308/225000 [256:33:54<163:19:05,  5.96s/it]
 56%|█████▌    | 126309/225000 [256:33:57<140:07:14,  5.11s/it]
 56%|█████▌    | 126310/225000 [256:34:03<145:08:17,  5.29s/it]
 56%|█████▌    | 126311/225000 [256:34:13<180:20:18,  6.58s/it]
 56%|█████▌    | 126312/225000 [256:34:16<154:44:11,  5.64s/it]
 56%|█████▌    | 126313/225000 [256:34:20<141:12:36,  5.15s/it]
 56%|█████▌    | 126314/225000 [256:34:28<165:26:11,  6.04s/it]
 56%|█████▌    | 126315/225000 [256:34:33<158:10:42,  5.77s/it]
 56%|█████▌    | 126316/225000 [256:34:37<140:54:57,  5.14s/it]
 56%|█████▌    | 126317/225000 [256:34:47<180:54:24,  6.60s/it]
 56%|█████▌    | 126318/225000 [256:34:50<154:20:00,  5.63s/it]
 56%|█████▌    | 126319/225000 [256:34:56<152:28:26,  5.56s/it]
 56%|█████▌    | 126320/225000 [256:35:03<163:21:32,  5.96s/it]
 56%|█████▌    | 126321/225000 [256:35:06<144:37:39,  5.28s/it]
 56%|█████▌    | 126322/225000 [256:35:11<139:47:44,  5.10s/it]
 56%|█████▌    | 126323/225000 [256:35:17<146:38:33,  5.35s/it]
 56%|█████▌    | 126324/225000 [256:35:25<167:13:22,  6.10s/it]
 56%|█████▌    | 126325/225000 [256:35:29<149:34:59,  5.46s/it]
 56%|█████▌    | 126326/225000 [256:35:33<140:28:23,  5.12s/it]
 56%|█████▌    | 126327/225000 [256:35:38<135:54:12,  4.96s/it]
 56%|█████▌    | 126328/225000 [256:35:47<172:02:29,  6.28s/it]
 56%|█████▌    | 126329/225000 [256:35:54<178:38:13,  6.52s/it]
 56%|█████▌    | 126330/225000 [256:35:57<151:46:13,  5.54s/it]
 56%|█████▌    | 126331/225000 [256:36:05<166:12:23,  6.06s/it]
 56%|█████▌    | 126332/225000 [256:36:12<174:54:10,  6.38s/it]
 56%|█████▌    | 126333/225000 [256:36:16<154:53:30,  5.65s/it]
 56%|█████▌    | 126334/225000 [256:36:20<141:55:55,  5.18s/it]
 56%|█████▌    | 126335/225000 [256:36:24<135:13:26,  4.93s/it]
 56%|█████▌    | 126336/225000 [256:36:35<186:23:16,  6.80s/it]
 56%|█████▌    | 126337/225000 [256:36:40<171:21:17,  6.25s/it]
 56%|█████▌    | 126338/225000 [256:36:44<149:05:04,  5.44s/it]
 56%|█████▌    | 126339/225000 [256:36:53<175:34:52,  6.41s/it]
 56%|█████▌    | 126340/225000 [256:36:56<150:53:50,  5.51s/it]
 56%|█████▌    | 126341/225000 [256:37:01<150:09:45,  5.48s/it]
 56%|█████▌    | 126342/225000 [256:37:10<174:27:56,  6.37s/it]
 56%|█████▌    | 126343/225000 [256:37:13<149:40:31,  5.46s/it]
 56%|█████▌    | 126344/225000 [256:37:22<180:35:00,  6.59s/it]
 56%|█████▌    | 126345/225000 [256:37:27<161:35:16,  5.90s/it]
 56%|█████▌    | 126346/225000 [256:37:31<148:11:36,  5.41s/it]
 56%|█████▌    | 126347/225000 [256:37:35<140:32:57,  5.13s/it]
 56%|█████▌    | 126348/225000 [256:37:40<137:13:13,  5.01s/it]
 56%|█████▌    | 126349/225000 [256:37:45<133:11:53,  4.86s/it]
 56%|█████▌    | 126350/225000 [256:37:49<131:13:29,  4.79s/it]
                                                               
{'loss': '0.8392', 'grad_norm': '1.048', 'learning_rate': '2.18e-05', 'epoch': '0.5616', 'num_input_tokens_seen': 427094296, 'train_runtime': '9.239e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126350/225000 [256:37:49<131:13:29,  4.79s/it]
 56%|█████▌    | 126351/225000 [256:37:56<143:25:46,  5.23s/it]
 56%|█████▌    | 126352/225000 [256:38:08<202:53:34,  7.40s/it]
 56%|█████▌    | 126353/225000 [256:38:14<187:13:54,  6.83s/it]
 56%|█████▌    | 126354/225000 [256:38:17<161:50:15,  5.91s/it]
 56%|█████▌    | 126355/225000 [256:38:27<189:10:35,  6.90s/it]
 56%|█████▌    | 126356/225000 [256:38:30<158:58:27,  5.80s/it]
 56%|█████▌    | 126357/225000 [256:38:35<157:06:50,  5.73s/it]
 56%|█████▌    | 126358/225000 [256:38:41<153:16:45,  5.59s/it]
 56%|█████▌    | 126359/225000 [256:38:46<150:31:32,  5.49s/it]
 56%|█████▌    | 126360/225000 [256:38:55<180:10:47,  6.58s/it]
 56%|█████▌    | 126361/225000 [256:38:59<158:37:42,  5.79s/it]
 56%|█████▌    | 126362/225000 [256:39:04<149:03:50,  5.44s/it]
 56%|█████▌    | 126363/225000 [256:39:10<159:54:16,  5.84s/it]
 56%|█████▌    | 126364/225000 [256:39:18<174:19:16,  6.36s/it]
 56%|█████▌    | 126365/225000 [256:39:26<185:12:38,  6.76s/it]
 56%|█████▌    | 126366/225000 [256:39:29<154:32:58,  5.64s/it]
 56%|█████▌    | 126367/225000 [256:39:37<180:34:58,  6.59s/it]
 56%|█████▌    | 126368/225000 [256:39:41<156:41:50,  5.72s/it]
 56%|█████▌    | 126369/225000 [256:39:46<149:11:11,  5.45s/it]
 56%|█████▌    | 126370/225000 [256:39:53<166:23:46,  6.07s/it]
 56%|█████▌    | 126371/225000 [256:39:57<144:58:48,  5.29s/it]
 56%|█████▌    | 126372/225000 [256:40:04<162:18:27,  5.92s/it]
 56%|█████▌    | 126373/225000 [256:40:10<160:22:49,  5.85s/it]
 56%|█████▌    | 126374/225000 [256:40:13<139:46:54,  5.10s/it]
 56%|█████▌    | 126375/225000 [256:40:17<128:44:04,  4.70s/it]
 56%|█████▌    | 126376/225000 [256:40:28<178:51:32,  6.53s/it]
 56%|█████▌    | 126377/225000 [256:40:31<154:16:10,  5.63s/it]
 56%|█████▌    | 126378/225000 [256:40:36<146:45:01,  5.36s/it]
 56%|█████▌    | 126379/225000 [256:40:45<173:45:28,  6.34s/it]
 56%|█████▌    | 126380/225000 [256:40:49<155:44:19,  5.69s/it]
 56%|█████▌    | 126381/225000 [256:40:56<166:29:04,  6.08s/it]
 56%|█████▌    | 126382/225000 [256:41:02<169:11:32,  6.18s/it]
 56%|█████▌    | 126383/225000 [256:41:06<150:48:54,  5.51s/it]
 56%|█████▌    | 126384/225000 [256:41:15<177:47:29,  6.49s/it]
 56%|█████▌    | 126385/225000 [256:41:19<156:55:10,  5.73s/it]
 56%|█████▌    | 126386/225000 [256:41:28<180:08:53,  6.58s/it]
 56%|█████▌    | 126387/225000 [256:41:32<158:11:23,  5.77s/it]
 56%|█████▌    | 126388/225000 [256:41:39<175:50:47,  6.42s/it]
 56%|█████▌    | 126389/225000 [256:41:43<152:05:24,  5.55s/it]
 56%|█████▌    | 126390/225000 [256:41:47<137:03:03,  5.00s/it]
 56%|█████▌    | 126391/225000 [256:41:51<133:02:09,  4.86s/it]
 56%|█████▌    | 126392/225000 [256:41:56<129:46:50,  4.74s/it]
 56%|█████▌    | 126393/225000 [256:42:01<134:45:08,  4.92s/it]
 56%|█████▌    | 126394/225000 [256:42:13<191:21:35,  6.99s/it]
 56%|█████▌    | 126395/225000 [256:42:17<166:08:11,  6.07s/it]
 56%|█████▌    | 126396/225000 [256:42:24<173:31:58,  6.34s/it]
 56%|█████▌    | 126397/225000 [256:42:30<171:02:26,  6.24s/it]
 56%|█████▌    | 126398/225000 [256:42:36<167:48:46,  6.13s/it]
 56%|█████▌    | 126399/225000 [256:42:43<180:08:36,  6.58s/it]
 56%|█████▌    | 126400/225000 [256:42:47<153:31:26,  5.61s/it]
                                                               
{'loss': '0.8162', 'grad_norm': '0.9268', 'learning_rate': '2.178e-05', 'epoch': '0.5618', 'num_input_tokens_seen': 427223640, 'train_runtime': '9.242e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126400/225000 [256:42:47<153:31:26,  5.61s/it]
 56%|█████▌    | 126401/225000 [256:42:51<147:48:03,  5.40s/it]
 56%|█████▌    | 126402/225000 [256:42:57<145:37:15,  5.32s/it]
 56%|█████▌    | 126403/225000 [256:43:01<141:55:03,  5.18s/it]
 56%|█████▌    | 126404/225000 [256:43:07<142:26:49,  5.20s/it]
 56%|█████▌    | 126405/225000 [256:43:12<141:10:37,  5.15s/it]
 56%|█████▌    | 126406/225000 [256:43:17<141:40:57,  5.17s/it]
 56%|█████▌    | 126407/225000 [256:43:22<138:15:12,  5.05s/it]
 56%|█████▌    | 126408/225000 [256:43:26<134:26:46,  4.91s/it]
 56%|█████▌    | 126409/225000 [256:43:32<137:34:40,  5.02s/it]
 56%|█████▌    | 126410/225000 [256:43:45<202:32:32,  7.40s/it]
 56%|█████▌    | 126411/225000 [256:43:52<206:03:02,  7.52s/it]
 56%|█████▌    | 126412/225000 [256:43:59<202:35:11,  7.40s/it]
 56%|█████▌    | 126413/225000 [256:44:04<181:34:52,  6.63s/it]
 56%|█████▌    | 126414/225000 [256:44:09<167:42:06,  6.12s/it]
 56%|█████▌    | 126415/225000 [256:44:14<158:19:56,  5.78s/it]
 56%|█████▌    | 126416/225000 [256:44:19<150:25:11,  5.49s/it]
 56%|█████▌    | 126417/225000 [256:44:24<148:28:01,  5.42s/it]
 56%|█████▌    | 126418/225000 [256:44:29<141:32:46,  5.17s/it]
 56%|█████▌    | 126419/225000 [256:44:36<156:34:56,  5.72s/it]
 56%|█████▌    | 126420/225000 [256:44:48<208:07:05,  7.60s/it]
 56%|█████▌    | 126421/225000 [256:44:51<174:26:05,  6.37s/it]
 56%|█████▌    | 126422/225000 [256:44:55<150:21:45,  5.49s/it]
 56%|█████▌    | 126423/225000 [256:45:06<195:43:21,  7.15s/it]
 56%|█████▌    | 126424/225000 [256:45:10<167:12:49,  6.11s/it]
 56%|█████▌    | 126425/225000 [256:45:13<147:14:09,  5.38s/it]
 56%|█████▌    | 126426/225000 [256:45:21<165:33:04,  6.05s/it]
 56%|█████▌    | 126427/225000 [256:45:28<177:35:55,  6.49s/it]
 56%|█████▌    | 126428/225000 [256:45:32<153:36:28,  5.61s/it]
 56%|█████▌    | 126429/225000 [256:45:40<177:26:14,  6.48s/it]
 56%|█████▌    | 126430/225000 [256:45:44<150:35:22,  5.50s/it]
 56%|█████▌    | 126431/225000 [256:45:52<171:32:10,  6.26s/it]
 56%|█████▌    | 126432/225000 [256:45:57<163:08:25,  5.96s/it]
 56%|█████▌    | 126433/225000 [256:46:01<149:34:02,  5.46s/it]
 56%|█████▌    | 126434/225000 [256:46:06<146:47:08,  5.36s/it]
 56%|█████▌    | 126435/225000 [256:46:12<149:15:41,  5.45s/it]
 56%|█████▌    | 126436/225000 [256:46:17<144:24:58,  5.27s/it]
 56%|█████▌    | 126437/225000 [256:46:24<161:29:14,  5.90s/it]
 56%|█████▌    | 126438/225000 [256:46:30<159:31:49,  5.83s/it]
 56%|█████▌    | 126439/225000 [256:46:35<153:09:28,  5.59s/it]
 56%|█████▌    | 126440/225000 [256:46:43<173:31:17,  6.34s/it]
 56%|█████▌    | 126441/225000 [256:46:47<153:23:27,  5.60s/it]
 56%|█████▌    | 126442/225000 [256:46:52<147:48:40,  5.40s/it]
 56%|█████▌    | 126443/225000 [256:46:57<148:08:18,  5.41s/it]
 56%|█████▌    | 126444/225000 [256:47:03<148:04:40,  5.41s/it]
 56%|█████▌    | 126445/225000 [256:47:08<146:51:59,  5.36s/it]
 56%|█████▌    | 126446/225000 [256:47:13<146:32:30,  5.35s/it]
 56%|█████▌    | 126447/225000 [256:47:24<187:52:03,  6.86s/it]
 56%|█████▌    | 126448/225000 [256:47:31<196:13:27,  7.17s/it]
 56%|█████▌    | 126449/225000 [256:47:35<163:23:06,  5.97s/it]
 56%|█████▌    | 126450/225000 [256:47:42<177:37:16,  6.49s/it]
                                                               
{'loss': '0.8208', 'grad_norm': '0.9574', 'learning_rate': '2.176e-05', 'epoch': '0.562', 'num_input_tokens_seen': 427353904, 'train_runtime': '9.245e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126450/225000 [256:47:42<177:37:16,  6.49s/it]
 56%|█████▌    | 126451/225000 [256:47:46<150:16:57,  5.49s/it]
 56%|█████▌    | 126452/225000 [256:47:54<176:56:13,  6.46s/it]
 56%|█████▌    | 126453/225000 [256:47:58<152:27:19,  5.57s/it]
 56%|█████▌    | 126454/225000 [256:48:08<189:59:21,  6.94s/it]
 56%|█████▌    | 126455/225000 [256:48:12<166:27:03,  6.08s/it]
 56%|█████▌    | 126456/225000 [256:48:16<145:53:29,  5.33s/it]
 56%|█████▌    | 126457/225000 [256:48:20<136:49:03,  5.00s/it]
 56%|█████▌    | 126458/225000 [256:48:30<179:47:41,  6.57s/it]
 56%|█████▌    | 126459/225000 [256:48:36<172:59:18,  6.32s/it]
 56%|█████▌    | 126460/225000 [256:48:43<179:57:17,  6.57s/it]
 56%|█████▌    | 126461/225000 [256:48:47<158:44:37,  5.80s/it]
 56%|█████▌    | 126462/225000 [256:48:51<145:17:29,  5.31s/it]
 56%|█████▌    | 126463/225000 [256:48:55<135:11:08,  4.94s/it]
 56%|█████▌    | 126464/225000 [256:49:03<161:04:59,  5.89s/it]
 56%|█████▌    | 126465/225000 [256:49:11<174:57:33,  6.39s/it]
 56%|█████▌    | 126466/225000 [256:49:15<156:52:36,  5.73s/it]
 56%|█████▌    | 126467/225000 [256:49:23<177:54:31,  6.50s/it]
 56%|█████▌    | 126468/225000 [256:49:27<152:06:49,  5.56s/it]
 56%|█████▌    | 126469/225000 [256:49:31<142:02:02,  5.19s/it]
 56%|█████▌    | 126470/225000 [256:49:35<136:17:44,  4.98s/it]
 56%|█████▌    | 126471/225000 [256:49:40<132:50:26,  4.85s/it]
 56%|█████▌    | 126472/225000 [256:49:51<180:16:03,  6.59s/it]
 56%|█████▌    | 126473/225000 [256:49:58<183:45:20,  6.71s/it]
 56%|█████▌    | 126474/225000 [256:50:01<157:42:59,  5.76s/it]
 56%|█████▌    | 126475/225000 [256:50:05<141:32:23,  5.17s/it]
 56%|█████▌    | 126476/225000 [256:50:15<181:57:20,  6.65s/it]
 56%|█████▌    | 126477/225000 [256:50:19<155:57:55,  5.70s/it]
 56%|█████▌    | 126478/225000 [256:50:24<152:58:04,  5.59s/it]
 56%|█████▌    | 126479/225000 [256:50:33<177:49:36,  6.50s/it]
 56%|█████▌    | 126480/225000 [256:50:38<169:21:22,  6.19s/it]
 56%|█████▌    | 126481/225000 [256:50:45<173:24:48,  6.34s/it]
 56%|█████▌    | 126482/225000 [256:50:49<154:06:28,  5.63s/it]
 56%|█████▌    | 126483/225000 [256:50:52<138:44:23,  5.07s/it]
 56%|█████▌    | 126484/225000 [256:51:02<178:32:03,  6.52s/it]
 56%|█████▌    | 126485/225000 [256:51:07<164:40:05,  6.02s/it]
 56%|█████▌    | 126486/225000 [256:51:11<148:01:56,  5.41s/it]
 56%|█████▌    | 126487/225000 [256:51:15<137:58:46,  5.04s/it]
 56%|█████▌    | 126488/225000 [256:51:20<131:42:59,  4.81s/it]
 56%|█████▌    | 126489/225000 [256:51:30<174:20:10,  6.37s/it]
 56%|█████▌    | 126490/225000 [256:51:35<168:02:32,  6.14s/it]
 56%|█████▌    | 126491/225000 [256:51:41<165:40:23,  6.05s/it]
 56%|█████▌    | 126492/225000 [256:51:47<165:35:29,  6.05s/it]
 56%|█████▌    | 126493/225000 [256:51:55<180:17:29,  6.59s/it]
 56%|█████▌    | 126494/225000 [256:52:03<190:15:05,  6.95s/it]
 56%|█████▌    | 126495/225000 [256:52:09<186:42:46,  6.82s/it]
 56%|█████▌    | 126496/225000 [256:52:13<159:46:53,  5.84s/it]
 56%|█████▌    | 126497/225000 [256:52:23<192:34:50,  7.04s/it]
 56%|█████▌    | 126498/225000 [256:52:27<168:41:10,  6.17s/it]
 56%|█████▌    | 126499/225000 [256:52:31<149:28:44,  5.46s/it]
 56%|█████▌    | 126500/225000 [256:52:35<143:37:07,  5.25s/it]
                                                               
{'loss': '0.8272', 'grad_norm': '1.047', 'learning_rate': '2.174e-05', 'epoch': '0.5622', 'num_input_tokens_seen': 427483744, 'train_runtime': '9.248e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126500/225000 [256:52:35<143:37:07,  5.25s/it]
 56%|█████▌    | 126501/225000 [256:52:40<139:09:58,  5.09s/it]
 56%|█████▌    | 126502/225000 [256:52:45<134:28:55,  4.92s/it]
 56%|█████▌    | 126503/225000 [256:52:50<140:42:11,  5.14s/it]
 56%|█████▌    | 126504/225000 [256:53:02<196:20:47,  7.18s/it]
 56%|█████▌    | 126505/225000 [256:53:07<178:18:47,  6.52s/it]
 56%|█████▌    | 126506/225000 [256:53:12<161:15:01,  5.89s/it]
 56%|█████▌    | 126507/225000 [256:53:19<173:30:00,  6.34s/it]
 56%|█████▌    | 126508/225000 [256:53:23<151:21:26,  5.53s/it]
 56%|█████▌    | 126509/225000 [256:53:34<195:51:01,  7.16s/it]
 56%|█████▌    | 126510/225000 [256:53:37<166:23:16,  6.08s/it]
 56%|█████▌    | 126511/225000 [256:53:45<181:01:34,  6.62s/it]
 56%|█████▌    | 126512/225000 [256:53:49<155:46:19,  5.69s/it]
 56%|█████▌    | 126513/225000 [256:53:54<157:21:14,  5.75s/it]
 56%|█████▌    | 126514/225000 [256:53:58<142:29:47,  5.21s/it]
 56%|█████▌    | 126515/225000 [256:54:02<133:21:52,  4.87s/it]
 56%|█████▌    | 126516/225000 [256:54:13<182:54:38,  6.69s/it]
 56%|█████▌    | 126517/225000 [256:54:19<170:55:34,  6.25s/it]
 56%|█████▌    | 126518/225000 [256:54:22<150:17:04,  5.49s/it]
 56%|█████▌    | 126519/225000 [256:54:29<157:04:31,  5.74s/it]
 56%|█████▌    | 126520/225000 [256:54:36<168:41:25,  6.17s/it]
 56%|█████▌    | 126521/225000 [256:54:39<146:34:32,  5.36s/it]
 56%|█████▌    | 126522/225000 [256:54:44<141:58:46,  5.19s/it]
 56%|█████▌    | 126523/225000 [256:54:49<139:31:11,  5.10s/it]
 56%|█████▌    | 126524/225000 [256:54:54<137:24:36,  5.02s/it]
 56%|█████▌    | 126525/225000 [256:54:59<136:18:10,  4.98s/it]
 56%|█████▌    | 126526/225000 [256:55:04<135:06:15,  4.94s/it]
 56%|█████▌    | 126527/225000 [256:55:09<135:50:31,  4.97s/it]
 56%|█████▌    | 126528/225000 [256:55:14<136:38:18,  5.00s/it]
 56%|█████▌    | 126529/225000 [256:55:24<178:35:11,  6.53s/it]
 56%|█████▌    | 126530/225000 [256:55:33<201:32:48,  7.37s/it]
 56%|█████▌    | 126531/225000 [256:55:36<168:25:42,  6.16s/it]
 56%|█████▌    | 126532/225000 [256:55:42<161:28:56,  5.90s/it]
 56%|█████▌    | 126533/225000 [256:55:52<199:15:27,  7.28s/it]
 56%|█████▌    | 126534/225000 [256:55:55<165:17:47,  6.04s/it]
 56%|█████▌    | 126535/225000 [256:56:03<178:37:42,  6.53s/it]
 56%|█████▌    | 126536/225000 [256:56:07<153:40:12,  5.62s/it]
 56%|█████▌    | 126537/225000 [256:56:16<183:31:25,  6.71s/it]
 56%|█████▌    | 126538/225000 [256:56:19<157:37:55,  5.76s/it]
 56%|█████▌    | 126539/225000 [256:56:28<181:19:38,  6.63s/it]
 56%|█████▌    | 126540/225000 [256:56:31<153:35:32,  5.62s/it]
 56%|█████▌    | 126541/225000 [256:56:39<172:18:24,  6.30s/it]
 56%|█████▌    | 126542/225000 [256:56:44<164:07:16,  6.00s/it]
 56%|█████▌    | 126543/225000 [256:56:49<154:25:22,  5.65s/it]
 56%|█████▌    | 126544/225000 [256:56:54<146:32:49,  5.36s/it]
 56%|█████▌    | 126545/225000 [256:56:59<140:00:03,  5.12s/it]
 56%|█████▌    | 126546/225000 [256:57:10<193:46:01,  7.09s/it]
 56%|█████▌    | 126547/225000 [256:57:14<170:15:15,  6.23s/it]
 56%|█████▌    | 126548/225000 [256:57:21<170:38:53,  6.24s/it]
 56%|█████▌    | 126549/225000 [256:57:26<160:02:28,  5.85s/it]
 56%|█████▌    | 126550/225000 [256:57:29<139:47:51,  5.11s/it]
                                                               
{'loss': '0.8285', 'grad_norm': '1.216', 'learning_rate': '2.172e-05', 'epoch': '0.5624', 'num_input_tokens_seen': 427613824, 'train_runtime': '9.25e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▌    | 126550/225000 [256:57:29<139:47:51,  5.11s/it]
 56%|█████▌    | 126551/225000 [256:57:39<181:34:51,  6.64s/it]
 56%|█████▌    | 126552/225000 [256:57:44<166:09:05,  6.08s/it]
 56%|█████▌    | 126553/225000 [256:57:50<167:33:30,  6.13s/it]
 56%|█████▌    | 126554/225000 [256:57:54<145:15:50,  5.31s/it]
 56%|█████▌    | 126555/225000 [256:57:57<132:25:30,  4.84s/it]
 56%|█████▌    | 126556/225000 [256:58:06<163:25:52,  5.98s/it]
 56%|█████▌    | 126557/225000 [256:58:13<169:07:17,  6.18s/it]
 56%|█████▌    | 126558/225000 [256:58:16<145:21:20,  5.32s/it]
 56%|█████▌    | 126559/225000 [256:58:26<185:54:01,  6.80s/it]
 56%|█████▌    | 126560/225000 [256:58:30<157:06:03,  5.75s/it]
 56%|█████▌    | 126561/225000 [256:58:33<140:06:00,  5.12s/it]
 56%|█████▌    | 126562/225000 [256:58:37<131:38:00,  4.81s/it]
 56%|█████▋    | 126563/225000 [256:58:42<134:48:18,  4.93s/it]
 56%|█████▋    | 126564/225000 [256:58:47<134:26:56,  4.92s/it]
 56%|█████▋    | 126565/225000 [256:58:59<186:16:12,  6.81s/it]
 56%|█████▋    | 126566/225000 [256:59:03<166:45:57,  6.10s/it]
 56%|█████▋    | 126567/225000 [256:59:07<151:47:18,  5.55s/it]
 56%|█████▋    | 126568/225000 [256:59:12<145:04:45,  5.31s/it]
 56%|█████▋    | 126569/225000 [256:59:23<193:43:10,  7.09s/it]
 56%|█████▋    | 126570/225000 [256:59:27<167:19:38,  6.12s/it]
 56%|█████▋    | 126571/225000 [256:59:31<148:32:14,  5.43s/it]
 56%|█████▋    | 126572/225000 [256:59:39<171:49:19,  6.28s/it]
 56%|█████▋    | 126573/225000 [256:59:45<166:33:35,  6.09s/it]
 56%|█████▋    | 126574/225000 [256:59:49<148:19:12,  5.42s/it]
 56%|█████▋    | 126575/225000 [256:59:55<151:30:37,  5.54s/it]
 56%|█████▋    | 126576/225000 [257:00:03<176:52:17,  6.47s/it]
 56%|█████▋    | 126577/225000 [257:00:07<156:26:42,  5.72s/it]
 56%|█████▋    | 126578/225000 [257:00:12<144:54:36,  5.30s/it]
 56%|█████▋    | 126579/225000 [257:00:16<135:27:21,  4.95s/it]
 56%|█████▋    | 126580/225000 [257:00:25<175:14:29,  6.41s/it]
 56%|█████▋    | 126581/225000 [257:00:31<166:01:40,  6.07s/it]
 56%|█████▋    | 126582/225000 [257:00:39<181:53:26,  6.65s/it]
 56%|█████▋    | 126583/225000 [257:00:44<166:21:19,  6.09s/it]
 56%|█████▋    | 126584/225000 [257:00:48<155:58:58,  5.71s/it]
 56%|█████▋    | 126585/225000 [257:00:53<149:46:17,  5.48s/it]
 56%|█████▋    | 126586/225000 [257:00:58<144:37:49,  5.29s/it]
 56%|█████▋    | 126587/225000 [257:01:04<148:04:10,  5.42s/it]
 56%|█████▋    | 126588/225000 [257:01:16<206:46:39,  7.56s/it]
 56%|█████▋    | 126589/225000 [257:01:20<175:23:58,  6.42s/it]
 56%|█████▋    | 126590/225000 [257:01:24<154:31:53,  5.65s/it]
 56%|█████▋    | 126591/225000 [257:01:34<193:31:03,  7.08s/it]
 56%|█████▋    | 126592/225000 [257:01:39<174:12:34,  6.37s/it]
 56%|█████▋    | 126593/225000 [257:01:44<159:14:40,  5.83s/it]
 56%|█████▋    | 126594/225000 [257:01:48<150:20:36,  5.50s/it]
 56%|█████▋    | 126595/225000 [257:01:53<141:34:47,  5.18s/it]
 56%|█████▋    | 126596/225000 [257:01:57<136:23:33,  4.99s/it]
 56%|█████▋    | 126597/225000 [257:02:02<134:31:36,  4.92s/it]
 56%|█████▋    | 126598/225000 [257:02:12<172:08:17,  6.30s/it]
 56%|█████▋    | 126599/225000 [257:02:22<202:47:35,  7.42s/it]
 56%|█████▋    | 126600/225000 [257:02:25<169:13:04,  6.19s/it]
                                                               
{'loss': '0.8209', 'grad_norm': '0.9641', 'learning_rate': '2.171e-05', 'epoch': '0.5627', 'num_input_tokens_seen': 427743184, 'train_runtime': '9.253e+05', 'train_tokens_per_second': '462.3'}

 56%|█████▋    | 126600/225000 [257:02:25<169:13:04,  6.19s/it]
 56%|█████▋    | 126601/225000 [257:02:33<182:57:03,  6.69s/it]
 56%|█████▋    | 126602/225000 [257:02:36<155:51:59,  5.70s/it]
 56%|█████▋    | 126603/225000 [257:02:42<153:47:38,  5.63s/it]
 56%|█████▋    | 126604/225000 [257:02:51<184:24:18,  6.75s/it]
 56%|█████▋    | 126605/225000 [257:02:55<158:53:55,  5.81s/it]
 56%|█████▋    | 126606/225000 [257:03:04<186:40:09,  6.83s/it]
 56%|█████▋    | 126607/225000 [257:03:07<156:18:00,  5.72s/it]
 56%|█████▋    | 126608/225000 [257:03:15<175:16:30,  6.41s/it]
 56%|█████▋    | 126609/225000 [257:03:20<160:00:04,  5.85s/it]
 56%|█████▋    | 126610/225000 [257:03:24<146:00:55,  5.34s/it]
 56%|█████▋    | 126611/225000 [257:03:28<137:01:53,  5.01s/it]
 56%|█████▋    | 126612/225000 [257:03:38<173:19:08,  6.34s/it]
 56%|█████▋    | 126613/225000 [257:03:43<162:11:47,  5.93s/it]
 56%|█████▋    | 126614/225000 [257:03:49<168:45:45,  6.18s/it]
 56%|█████▋    | 126615/225000 [257:03:54<160:18:15,  5.87s/it]
 56%|█████▋    | 126616/225000 [257:04:02<176:52:16,  6.47s/it]
 56%|█████▋    | 126617/225000 [257:04:07<164:40:14,  6.03s/it]
 56%|█████▋    | 126618/225000 [257:04:11<145:15:35,  5.32s/it]
 56%|█████▋    | 126619/225000 [257:04:15<134:36:55,  4.93s/it]
 56%|█████▋    | 126620/225000 [257:04:20<131:41:56,  4.82s/it]
 56%|█████▋    | 126621/225000 [257:04:30<176:56:56,  6.48s/it]
 56%|█████▋    | 126622/225000 [257:04:36<171:50:38,  6.29s/it]
 56%|█████▋    | 126623/225000 [257:04:41<163:00:21,  5.97s/it]
 56%|█████▋    | 126624/225000 [257:04:46<155:07:09,  5.68s/it]
 56%|█████▋    | 126625/225000 [257:04:51<149:51:37,  5.48s/it]
 56%|█████▋    | 126626/225000 [257:04:56<148:44:25,  5.44s/it]
 56%|█████▋    | 126627/225000 [257:05:01<145:03:57,  5.31s/it]
 56%|█████▋    | 126628/225000 [257:05:06<141:11:36,  5.17s/it]
 56%|█████▋    | 126629/225000 [257:05:11<140:34:04,  5.14s/it]
 56%|█████▋    | 126630/225000 [257:05:16<138:09:58,  5.06s/it]
 56%|█████▋    | 126631/225000 [257:05:25<173:51:16,  6.36s/it]
 56%|█████▋    | 126632/225000 [257:05:36<211:18:35,  7.73s/it]
 56%|█████▋    | 126633/225000 [257:05:40<181:26:14,  6.64s/it]
 56%|█████▋    | 126634/225000 [257:05:44<158:48:06,  5.81s/it]
 56%|█████▋    | 126635/225000 [257:05:51<162:41:37,  5.95s/it]
 56%|█████▋    | 126636/225000 [257:06:00<188:00:55,  6.88s/it]
 56%|█████▋    | 126637/225000 [257:06:05<172:58:36,  6.33s/it]
 56%|█████▋    | 126638/225000 [257:06:10<163:53:01,  6.00s/it]
 56%|█████▋    | 126639/225000 [257:06:15<153:01:34,  5.60s/it]
 56%|█████▋    | 126640/225000 [257:06:20<151:25:16,  5.54s/it]
 56%|█████▋    | 126641/225000 [257:06:25<144:45:39,  5.30s/it]
 56%|█████▋    | 126642/225000 [257:06:30<146:30:11,  5.36s/it]
 56%|█████▋    | 126643/225000 [257:06:35<144:06:28,  5.27s/it]
 56%|█████▋    | 126644/225000 [257:06:41<147:01:11,  5.38s/it]
 56%|█████▋    | 126645/225000 [257:06:46<147:59:58,  5.42s/it]
 56%|█████▋    | 126646/225000 [257:06:51<144:06:30,  5.27s/it]
 56%|█████▋    | 126647/225000 [257:07:00<174:42:56,  6.40s/it]
 56%|█████▋    | 126648/225000 [257:07:12<214:01:19,  7.83s/it]
 56%|█████▋    | 126649/225000 [257:07:15<177:35:46,  6.50s/it]
 56%|█████▋    | 126650/225000 [257:07:21<173:50:51,  6.36s/it]
                                                               
{'loss': '0.8304', 'grad_norm': '0.9946', 'learning_rate': '2.169e-05', 'epoch': '0.5629', 'num_input_tokens_seen': 427873312, 'train_runtime': '9.256e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126650/225000 [257:07:21<173:50:51,  6.36s/it]
 56%|█████▋    | 126651/225000 [257:07:30<197:07:39,  7.22s/it]
 56%|█████▋    | 126652/225000 [257:07:34<168:20:26,  6.16s/it]
 56%|█████▋    | 126653/225000 [257:07:38<148:14:20,  5.43s/it]
 56%|█████▋    | 126654/225000 [257:07:49<195:30:50,  7.16s/it]
 56%|█████▋    | 126655/225000 [257:07:52<162:33:38,  5.95s/it]
 56%|█████▋    | 126656/225000 [257:07:55<142:02:18,  5.20s/it]
 56%|█████▋    | 126657/225000 [257:08:04<167:34:43,  6.13s/it]
 56%|█████▋    | 126658/225000 [257:08:08<148:50:22,  5.45s/it]
 56%|█████▋    | 126659/225000 [257:08:12<137:28:49,  5.03s/it]
 56%|█████▋    | 126660/225000 [257:08:21<175:42:27,  6.43s/it]
 56%|█████▋    | 126661/225000 [257:08:26<163:33:20,  5.99s/it]
 56%|█████▋    | 126662/225000 [257:08:30<142:50:48,  5.23s/it]
 56%|█████▋    | 126663/225000 [257:08:34<133:28:47,  4.89s/it]
 56%|█████▋    | 126664/225000 [257:08:39<138:28:57,  5.07s/it]
 56%|█████▋    | 126665/225000 [257:08:52<200:19:17,  7.33s/it]
 56%|█████▋    | 126666/225000 [257:08:55<168:50:27,  6.18s/it]
 56%|█████▋    | 126667/225000 [257:09:03<177:59:01,  6.52s/it]
 56%|█████▋    | 126668/225000 [257:09:07<158:08:12,  5.79s/it]
 56%|█████▋    | 126669/225000 [257:09:11<140:59:30,  5.16s/it]
 56%|█████▋    | 126670/225000 [257:09:15<135:52:28,  4.97s/it]
 56%|█████▋    | 126671/225000 [257:09:26<182:52:17,  6.70s/it]
 56%|█████▋    | 126672/225000 [257:09:31<166:45:43,  6.11s/it]
 56%|█████▋    | 126673/225000 [257:09:35<152:45:37,  5.59s/it]
 56%|█████▋    | 126674/225000 [257:09:39<141:27:03,  5.18s/it]
 56%|█████▋    | 126675/225000 [257:09:47<165:34:40,  6.06s/it]
 56%|█████▋    | 126676/225000 [257:09:54<170:21:11,  6.24s/it]
 56%|█████▋    | 126677/225000 [257:09:58<153:54:15,  5.64s/it]
 56%|█████▋    | 126678/225000 [257:10:03<148:24:41,  5.43s/it]
 56%|█████▋    | 126679/225000 [257:10:08<145:42:07,  5.33s/it]
 56%|█████▋    | 126680/225000 [257:10:13<138:20:50,  5.07s/it]
 56%|█████▋    | 126681/225000 [257:10:18<137:58:09,  5.05s/it]
 56%|█████▋    | 126682/225000 [257:10:23<139:24:28,  5.10s/it]
 56%|█████▋    | 126683/225000 [257:10:28<140:15:42,  5.14s/it]
 56%|█████▋    | 126684/225000 [257:10:34<145:40:13,  5.33s/it]
 56%|█████▋    | 126685/225000 [257:10:46<202:27:06,  7.41s/it]
 56%|█████▋    | 126686/225000 [257:10:53<195:57:34,  7.18s/it]
 56%|█████▋    | 126687/225000 [257:10:57<170:04:18,  6.23s/it]
 56%|█████▋    | 126688/225000 [257:11:02<164:46:26,  6.03s/it]
 56%|█████▋    | 126689/225000 [257:11:10<181:23:58,  6.64s/it]
 56%|█████▋    | 126690/225000 [257:11:13<151:24:44,  5.54s/it]
 56%|█████▋    | 126691/225000 [257:11:23<187:53:10,  6.88s/it]
 56%|█████▋    | 126692/225000 [257:11:29<177:00:10,  6.48s/it]
 56%|█████▋    | 126693/225000 [257:11:34<163:49:54,  6.00s/it]
 56%|█████▋    | 126694/225000 [257:11:39<159:04:28,  5.83s/it]
 56%|█████▋    | 126695/225000 [257:11:45<156:33:33,  5.73s/it]
 56%|█████▋    | 126696/225000 [257:11:50<153:57:54,  5.64s/it]
 56%|█████▋    | 126697/225000 [257:11:55<148:50:41,  5.45s/it]
 56%|█████▋    | 126698/225000 [257:12:00<146:26:39,  5.36s/it]
 56%|█████▋    | 126699/225000 [257:12:06<147:21:36,  5.40s/it]
 56%|█████▋    | 126700/225000 [257:12:11<143:43:05,  5.26s/it]
                                                               
{'loss': '0.8275', 'grad_norm': '1.069', 'learning_rate': '2.167e-05', 'epoch': '0.5631', 'num_input_tokens_seen': 428002416, 'train_runtime': '9.259e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126700/225000 [257:12:11<143:43:05,  5.26s/it]
 56%|█████▋    | 126701/225000 [257:12:16<142:04:45,  5.20s/it]
 56%|█████▋    | 126702/225000 [257:12:21<143:43:17,  5.26s/it]
 56%|█████▋    | 126703/225000 [257:12:26<141:05:43,  5.17s/it]
 56%|█████▋    | 126704/225000 [257:12:31<138:20:44,  5.07s/it]
 56%|█████▋    | 126705/225000 [257:12:36<140:27:28,  5.14s/it]
 56%|█████▋    | 126706/225000 [257:12:43<151:47:49,  5.56s/it]
 56%|█████▋    | 126707/225000 [257:12:56<211:21:54,  7.74s/it]
 56%|█████▋    | 126708/225000 [257:13:01<194:37:28,  7.13s/it]
 56%|█████▋    | 126709/225000 [257:13:10<202:33:04,  7.42s/it]
 56%|█████▋    | 126710/225000 [257:13:15<184:30:32,  6.76s/it]
 56%|█████▋    | 126711/225000 [257:13:23<193:17:30,  7.08s/it]
 56%|█████▋    | 126712/225000 [257:13:28<178:20:47,  6.53s/it]
 56%|█████▋    | 126713/225000 [257:13:33<167:26:16,  6.13s/it]
 56%|█████▋    | 126714/225000 [257:13:38<158:07:41,  5.79s/it]
 56%|█████▋    | 126715/225000 [257:13:43<153:37:38,  5.63s/it]
 56%|█████▋    | 126716/225000 [257:13:48<147:18:18,  5.40s/it]
 56%|█████▋    | 126717/225000 [257:13:53<140:44:14,  5.16s/it]
 56%|█████▋    | 126718/225000 [257:13:58<139:49:39,  5.12s/it]
 56%|█████▋    | 126719/225000 [257:14:03<138:58:23,  5.09s/it]
 56%|█████▋    | 126720/225000 [257:14:08<136:47:35,  5.01s/it]
 56%|█████▋    | 126721/225000 [257:14:20<196:23:46,  7.19s/it]
 56%|█████▋    | 126722/225000 [257:14:28<207:21:34,  7.60s/it]
 56%|█████▋    | 126723/225000 [257:14:32<171:46:05,  6.29s/it]
 56%|█████▋    | 126724/225000 [257:14:41<193:14:47,  7.08s/it]
 56%|█████▋    | 126725/225000 [257:14:44<162:48:30,  5.96s/it]
 56%|█████▋    | 126726/225000 [257:14:48<144:19:38,  5.29s/it]
 56%|█████▋    | 126727/225000 [257:14:56<169:57:29,  6.23s/it]
 56%|█████▋    | 126728/225000 [257:15:00<152:01:14,  5.57s/it]
 56%|█████▋    | 126729/225000 [257:15:05<142:20:26,  5.21s/it]
 56%|█████▋    | 126730/225000 [257:15:09<134:47:35,  4.94s/it]
 56%|█████▋    | 126731/225000 [257:15:20<185:27:39,  6.79s/it]
 56%|█████▋    | 126732/225000 [257:15:25<175:10:38,  6.42s/it]
 56%|█████▋    | 126733/225000 [257:15:29<151:29:28,  5.55s/it]
 56%|█████▋    | 126734/225000 [257:15:35<153:03:19,  5.61s/it]
 56%|█████▋    | 126735/225000 [257:15:43<175:04:26,  6.41s/it]
 56%|█████▋    | 126736/225000 [257:15:47<157:22:39,  5.77s/it]
 56%|█████▋    | 126737/225000 [257:15:53<157:23:59,  5.77s/it]
 56%|█████▋    | 126738/225000 [257:16:00<163:20:53,  5.98s/it]
 56%|█████▋    | 126739/225000 [257:16:06<167:35:12,  6.14s/it]
 56%|█████▋    | 126740/225000 [257:16:10<152:00:38,  5.57s/it]
 56%|█████▋    | 126741/225000 [257:16:14<140:06:29,  5.13s/it]
 56%|█████▋    | 126742/225000 [257:16:18<129:59:38,  4.76s/it]
 56%|█████▋    | 126743/225000 [257:16:23<128:32:40,  4.71s/it]
 56%|█████▋    | 126744/225000 [257:16:33<176:10:48,  6.46s/it]
 56%|█████▋    | 126745/225000 [257:16:42<190:42:53,  6.99s/it]
 56%|█████▋    | 126746/225000 [257:16:45<158:26:56,  5.81s/it]
 56%|█████▋    | 126747/225000 [257:16:53<181:28:32,  6.65s/it]
 56%|█████▋    | 126748/225000 [257:16:57<158:15:09,  5.80s/it]
 56%|█████▋    | 126749/225000 [257:17:01<141:46:49,  5.19s/it]
 56%|█████▋    | 126750/225000 [257:17:05<134:45:07,  4.94s/it]
                                                               
{'loss': '0.8304', 'grad_norm': '1.025', 'learning_rate': '2.165e-05', 'epoch': '0.5633', 'num_input_tokens_seen': 428131600, 'train_runtime': '9.262e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126750/225000 [257:17:05<134:45:07,  4.94s/it]
 56%|█████▋    | 126751/225000 [257:17:13<155:50:05,  5.71s/it]
 56%|█████▋    | 126752/225000 [257:17:21<175:08:03,  6.42s/it]
 56%|█████▋    | 126753/225000 [257:17:27<171:33:49,  6.29s/it]
 56%|█████▋    | 126754/225000 [257:17:31<153:25:15,  5.62s/it]
 56%|█████▋    | 126755/225000 [257:17:35<141:25:50,  5.18s/it]
 56%|█████▋    | 126756/225000 [257:17:39<135:11:28,  4.95s/it]
 56%|█████▋    | 126757/225000 [257:17:51<185:36:28,  6.80s/it]
 56%|█████▋    | 126758/225000 [257:17:55<164:44:58,  6.04s/it]
 56%|█████▋    | 126759/225000 [257:17:59<148:53:56,  5.46s/it]
 56%|█████▋    | 126760/225000 [257:18:03<141:06:26,  5.17s/it]
 56%|█████▋    | 126761/225000 [257:18:10<156:11:45,  5.72s/it]
 56%|█████▋    | 126762/225000 [257:18:21<197:35:51,  7.24s/it]
 56%|█████▋    | 126763/225000 [257:18:25<171:18:53,  6.28s/it]
 56%|█████▋    | 126764/225000 [257:18:29<151:44:10,  5.56s/it]
 56%|█████▋    | 126765/225000 [257:18:39<186:17:00,  6.83s/it]
 56%|█████▋    | 126766/225000 [257:18:43<163:40:34,  6.00s/it]
 56%|█████▋    | 126767/225000 [257:18:47<148:27:39,  5.44s/it]
 56%|█████▋    | 126768/225000 [257:18:56<176:56:14,  6.48s/it]
 56%|█████▋    | 126769/225000 [257:19:01<161:00:05,  5.90s/it]
 56%|█████▋    | 126770/225000 [257:19:05<147:02:43,  5.39s/it]
 56%|█████▋    | 126771/225000 [257:19:12<159:17:37,  5.84s/it]
 56%|█████▋    | 126772/225000 [257:19:20<175:53:42,  6.45s/it]
 56%|█████▋    | 126773/225000 [257:19:26<178:06:21,  6.53s/it]
 56%|█████▋    | 126774/225000 [257:19:32<169:05:15,  6.20s/it]
 56%|█████▋    | 126775/225000 [257:19:36<156:50:54,  5.75s/it]
 56%|█████▋    | 126776/225000 [257:19:41<149:53:58,  5.49s/it]
 56%|█████▋    | 126777/225000 [257:19:46<145:25:51,  5.33s/it]
 56%|█████▋    | 126778/225000 [257:19:51<138:25:42,  5.07s/it]
 56%|█████▋    | 126779/225000 [257:19:56<140:05:47,  5.13s/it]
 56%|█████▋    | 126780/225000 [257:20:05<174:08:49,  6.38s/it]
 56%|█████▋    | 126781/225000 [257:20:15<202:34:20,  7.42s/it]
 56%|█████▋    | 126782/225000 [257:20:21<187:46:42,  6.88s/it]
 56%|█████▋    | 126783/225000 [257:20:26<170:55:21,  6.26s/it]
 56%|█████▋    | 126784/225000 [257:20:29<149:27:22,  5.48s/it]
 56%|█████▋    | 126785/225000 [257:20:38<174:41:51,  6.40s/it]
 56%|█████▋    | 126786/225000 [257:20:41<150:29:08,  5.52s/it]
 56%|█████▋    | 126787/225000 [257:20:50<175:15:19,  6.42s/it]
 56%|█████▋    | 126788/225000 [257:20:54<157:23:45,  5.77s/it]
 56%|█████▋    | 126789/225000 [257:20:58<142:16:22,  5.22s/it]
 56%|█████▋    | 126790/225000 [257:21:07<170:59:14,  6.27s/it]
 56%|█████▋    | 126791/225000 [257:21:10<150:44:14,  5.53s/it]
 56%|█████▋    | 126792/225000 [257:21:19<176:34:29,  6.47s/it]
 56%|█████▋    | 126793/225000 [257:21:22<150:30:57,  5.52s/it]
 56%|█████▋    | 126794/225000 [257:21:26<134:48:16,  4.94s/it]
 56%|█████▋    | 126795/225000 [257:21:37<183:21:35,  6.72s/it]
 56%|█████▋    | 126796/225000 [257:21:40<157:25:47,  5.77s/it]
 56%|█████▋    | 126797/225000 [257:21:48<172:07:16,  6.31s/it]
 56%|█████▋    | 126798/225000 [257:21:53<161:21:10,  5.92s/it]
 56%|█████▋    | 126799/225000 [257:21:57<142:52:49,  5.24s/it]
 56%|█████▋    | 126800/225000 [257:22:06<176:07:36,  6.46s/it]
                                                               
{'loss': '0.8156', 'grad_norm': '1.153', 'learning_rate': '2.163e-05', 'epoch': '0.5636', 'num_input_tokens_seen': 428260960, 'train_runtime': '9.265e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126800/225000 [257:22:06<176:07:36,  6.46s/it]
 56%|█████▋    | 126801/225000 [257:22:10<156:03:57,  5.72s/it]
 56%|█████▋    | 126802/225000 [257:22:19<181:29:58,  6.65s/it]
 56%|█████▋    | 126803/225000 [257:22:22<156:17:51,  5.73s/it]
 56%|█████▋    | 126804/225000 [257:22:30<175:17:08,  6.43s/it]
 56%|█████▋    | 126805/225000 [257:22:37<176:57:46,  6.49s/it]
 56%|█████▋    | 126806/225000 [257:22:42<160:18:54,  5.88s/it]
 56%|█████▋    | 126807/225000 [257:22:46<144:51:30,  5.31s/it]
 56%|█████▋    | 126808/225000 [257:22:50<138:06:25,  5.06s/it]
 56%|█████▋    | 126809/225000 [257:22:57<154:18:32,  5.66s/it]
 56%|█████▋    | 126810/225000 [257:23:05<175:50:01,  6.45s/it]
 56%|█████▋    | 126811/225000 [257:23:15<205:08:00,  7.52s/it]
 56%|█████▋    | 126812/225000 [257:23:19<173:31:24,  6.36s/it]
 56%|█████▋    | 126813/225000 [257:23:23<155:37:24,  5.71s/it]
 56%|█████▋    | 126814/225000 [257:23:30<167:32:23,  6.14s/it]
 56%|█████▋    | 126815/225000 [257:23:34<146:48:09,  5.38s/it]
 56%|█████▋    | 126816/225000 [257:23:43<179:31:07,  6.58s/it]
 56%|█████▋    | 126817/225000 [257:23:49<170:46:56,  6.26s/it]
 56%|█████▋    | 126818/225000 [257:23:54<157:44:31,  5.78s/it]
 56%|█████▋    | 126819/225000 [257:23:58<145:53:28,  5.35s/it]
 56%|█████▋    | 126820/225000 [257:24:02<136:41:28,  5.01s/it]
 56%|█████▋    | 126821/225000 [257:24:10<157:06:51,  5.76s/it]
 56%|█████▋    | 126822/225000 [257:24:19<184:07:55,  6.75s/it]
 56%|█████▋    | 126823/225000 [257:24:22<155:40:35,  5.71s/it]
 56%|█████▋    | 126824/225000 [257:24:30<176:10:47,  6.46s/it]
 56%|█████▋    | 126825/225000 [257:24:34<152:11:52,  5.58s/it]
 56%|█████▋    | 126826/225000 [257:24:39<151:05:16,  5.54s/it]
 56%|█████▋    | 126827/225000 [257:24:46<164:22:52,  6.03s/it]
 56%|█████▋    | 126828/225000 [257:24:50<148:33:21,  5.45s/it]
 56%|█████▋    | 126829/225000 [257:24:54<137:02:38,  5.03s/it]
 56%|█████▋    | 126830/225000 [257:25:05<184:13:41,  6.76s/it]
 56%|█████▋    | 126831/225000 [257:25:09<158:53:09,  5.83s/it]
 56%|█████▋    | 126832/225000 [257:25:14<151:58:29,  5.57s/it]
 56%|█████▋    | 126833/225000 [257:25:18<138:34:12,  5.08s/it]
 56%|█████▋    | 126834/225000 [257:25:22<127:59:14,  4.69s/it]
 56%|█████▋    | 126835/225000 [257:25:26<126:16:47,  4.63s/it]
 56%|█████▋    | 126836/225000 [257:25:37<179:19:13,  6.58s/it]
 56%|█████▋    | 126837/225000 [257:25:42<163:02:24,  5.98s/it]
 56%|█████▋    | 126838/225000 [257:25:46<151:44:35,  5.57s/it]
 56%|█████▋    | 126839/225000 [257:25:51<145:30:27,  5.34s/it]
 56%|█████▋    | 126840/225000 [257:25:56<138:00:27,  5.06s/it]
 56%|█████▋    | 126841/225000 [257:26:04<167:57:05,  6.16s/it]
 56%|█████▋    | 126842/225000 [257:26:14<196:50:15,  7.22s/it]
 56%|█████▋    | 126843/225000 [257:26:18<172:34:27,  6.33s/it]
 56%|█████▋    | 126844/225000 [257:26:22<151:27:24,  5.55s/it]
 56%|█████▋    | 126845/225000 [257:26:26<139:38:30,  5.12s/it]
 56%|█████▋    | 126846/225000 [257:26:30<130:54:07,  4.80s/it]
 56%|█████▋    | 126847/225000 [257:26:43<196:05:50,  7.19s/it]
 56%|█████▋    | 126848/225000 [257:26:49<186:23:59,  6.84s/it]
 56%|█████▋    | 126849/225000 [257:26:53<161:09:22,  5.91s/it]
 56%|█████▋    | 126850/225000 [257:26:57<147:08:13,  5.40s/it]
                                                               
{'loss': '0.8219', 'grad_norm': '1.098', 'learning_rate': '2.162e-05', 'epoch': '0.5638', 'num_input_tokens_seen': 428390432, 'train_runtime': '9.268e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126850/225000 [257:26:57<147:08:13,  5.40s/it]
 56%|█████▋    | 126851/225000 [257:27:01<136:29:08,  5.01s/it]
 56%|█████▋    | 126852/225000 [257:27:14<199:15:53,  7.31s/it]
 56%|█████▋    | 126853/225000 [257:27:19<184:05:11,  6.75s/it]
 56%|█████▋    | 126854/225000 [257:27:24<166:16:49,  6.10s/it]
 56%|█████▋    | 126855/225000 [257:27:29<161:47:47,  5.93s/it]
 56%|█████▋    | 126856/225000 [257:27:38<183:53:39,  6.75s/it]
 56%|█████▋    | 126857/225000 [257:27:42<159:02:51,  5.83s/it]
 56%|█████▋    | 126858/225000 [257:27:52<194:25:22,  7.13s/it]
 56%|█████▋    | 126859/225000 [257:27:55<163:24:50,  5.99s/it]
 56%|█████▋    | 126860/225000 [257:27:59<144:01:44,  5.28s/it]
 56%|█████▋    | 126861/225000 [257:28:09<187:21:18,  6.87s/it]
 56%|█████▋    | 126862/225000 [257:28:13<157:50:39,  5.79s/it]
 56%|█████▋    | 126863/225000 [257:28:18<152:34:13,  5.60s/it]
 56%|█████▋    | 126864/225000 [257:28:26<178:19:20,  6.54s/it]
 56%|█████▋    | 126865/225000 [257:28:31<165:35:17,  6.07s/it]
 56%|█████▋    | 126866/225000 [257:28:36<152:35:44,  5.60s/it]
 56%|█████▋    | 126867/225000 [257:28:41<148:16:16,  5.44s/it]
 56%|█████▋    | 126868/225000 [257:28:46<143:03:04,  5.25s/it]
 56%|█████▋    | 126869/225000 [257:28:51<141:23:20,  5.19s/it]
 56%|█████▋    | 126870/225000 [257:28:56<140:32:29,  5.16s/it]
 56%|█████▋    | 126871/225000 [257:29:01<137:13:44,  5.03s/it]
 56%|█████▋    | 126872/225000 [257:29:12<187:10:08,  6.87s/it]
 56%|█████▋    | 126873/225000 [257:29:20<201:11:16,  7.38s/it]
 56%|█████▋    | 126874/225000 [257:29:25<175:55:20,  6.45s/it]
 56%|█████▋    | 126875/225000 [257:29:30<167:52:52,  6.16s/it]
 56%|█████▋    | 126876/225000 [257:29:35<153:36:14,  5.64s/it]
 56%|█████▋    | 126877/225000 [257:29:39<147:03:12,  5.40s/it]
 56%|█████▋    | 126878/225000 [257:29:44<141:17:35,  5.18s/it]
 56%|█████▋    | 126879/225000 [257:29:49<139:26:41,  5.12s/it]
 56%|█████▋    | 126880/225000 [257:29:54<138:02:30,  5.06s/it]
 56%|█████▋    | 126881/225000 [257:29:59<134:39:40,  4.94s/it]
 56%|█████▋    | 126882/225000 [257:30:04<135:37:31,  4.98s/it]
 56%|█████▋    | 126883/225000 [257:30:09<140:42:13,  5.16s/it]
 56%|█████▋    | 126884/225000 [257:30:22<201:26:43,  7.39s/it]
 56%|█████▋    | 126885/225000 [257:30:29<196:12:07,  7.20s/it]
 56%|█████▋    | 126886/225000 [257:30:33<171:32:04,  6.29s/it]
 56%|█████▋    | 126887/225000 [257:30:40<176:07:01,  6.46s/it]
 56%|█████▋    | 126888/225000 [257:30:46<176:58:20,  6.49s/it]
 56%|█████▋    | 126889/225000 [257:30:50<150:59:20,  5.54s/it]
 56%|█████▋    | 126890/225000 [257:30:53<132:56:11,  4.88s/it]
 56%|█████▋    | 126891/225000 [257:31:02<170:39:16,  6.26s/it]
 56%|█████▋    | 126892/225000 [257:31:07<155:05:02,  5.69s/it]
 56%|█████▋    | 126893/225000 [257:31:11<144:25:31,  5.30s/it]
 56%|█████▋    | 126894/225000 [257:31:15<131:54:29,  4.84s/it]
 56%|█████▋    | 126895/225000 [257:31:25<175:50:04,  6.45s/it]
 56%|█████▋    | 126896/225000 [257:31:30<161:46:34,  5.94s/it]
 56%|█████▋    | 126897/225000 [257:31:34<149:19:30,  5.48s/it]
 56%|█████▋    | 126898/225000 [257:31:39<140:36:41,  5.16s/it]
 56%|█████▋    | 126899/225000 [257:31:43<135:08:53,  4.96s/it]
 56%|█████▋    | 126900/225000 [257:31:56<196:25:09,  7.21s/it]
                                                               
{'loss': '0.8335', 'grad_norm': '1.074', 'learning_rate': '2.16e-05', 'epoch': '0.564', 'num_input_tokens_seen': 428518432, 'train_runtime': '9.271e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126900/225000 [257:31:56<196:25:09,  7.21s/it]
 56%|█████▋    | 126901/225000 [257:32:01<183:17:35,  6.73s/it]
 56%|█████▋    | 126902/225000 [257:32:06<163:27:12,  6.00s/it]
 56%|█████▋    | 126903/225000 [257:32:12<170:17:39,  6.25s/it]
 56%|█████▋    | 126904/225000 [257:32:18<168:54:31,  6.20s/it]
 56%|█████▋    | 126905/225000 [257:32:23<152:43:26,  5.60s/it]
 56%|█████▋    | 126906/225000 [257:32:27<139:51:40,  5.13s/it]
 56%|█████▋    | 126907/225000 [257:32:38<188:10:59,  6.91s/it]
 56%|█████▋    | 126908/225000 [257:32:42<168:59:46,  6.20s/it]
 56%|█████▋    | 126909/225000 [257:32:49<173:55:40,  6.38s/it]
 56%|█████▋    | 126910/225000 [257:32:54<160:53:28,  5.90s/it]
 56%|█████▋    | 126911/225000 [257:32:59<152:01:16,  5.58s/it]
 56%|█████▋    | 126912/225000 [257:33:03<143:48:52,  5.28s/it]
 56%|█████▋    | 126913/225000 [257:33:08<140:20:59,  5.15s/it]
 56%|█████▋    | 126914/225000 [257:33:13<134:34:23,  4.94s/it]
 56%|█████▋    | 126915/225000 [257:33:18<137:56:46,  5.06s/it]
 56%|█████▋    | 126916/225000 [257:33:27<170:18:37,  6.25s/it]
 56%|█████▋    | 126917/225000 [257:33:35<186:26:41,  6.84s/it]
 56%|█████▋    | 126918/225000 [257:33:39<159:14:45,  5.84s/it]
 56%|█████▋    | 126919/225000 [257:33:47<179:15:52,  6.58s/it]
 56%|█████▋    | 126920/225000 [257:33:50<151:28:33,  5.56s/it]
 56%|█████▋    | 126921/225000 [257:33:58<171:37:26,  6.30s/it]
 56%|█████▋    | 126922/225000 [257:34:01<146:54:55,  5.39s/it]
 56%|█████▋    | 126923/225000 [257:34:06<135:55:27,  4.99s/it]
 56%|█████▋    | 126924/225000 [257:34:11<136:23:36,  5.01s/it]
 56%|█████▋    | 126925/225000 [257:34:16<135:44:54,  4.98s/it]
 56%|█████▋    | 126926/225000 [257:34:21<140:59:20,  5.18s/it]
 56%|█████▋    | 126927/225000 [257:34:26<141:34:14,  5.20s/it]
 56%|█████▋    | 126928/225000 [257:34:32<143:25:13,  5.26s/it]
 56%|█████▋    | 126929/225000 [257:34:37<144:21:27,  5.30s/it]
 56%|█████▋    | 126930/225000 [257:34:42<138:57:51,  5.10s/it]
 56%|█████▋    | 126931/225000 [257:34:47<142:06:57,  5.22s/it]
 56%|█████▋    | 126932/225000 [257:34:53<143:30:32,  5.27s/it]
 56%|█████▋    | 126933/225000 [257:34:58<140:03:37,  5.14s/it]
 56%|█████▋    | 126934/225000 [257:35:09<191:34:07,  7.03s/it]
 56%|█████▋    | 126935/225000 [257:35:17<197:17:51,  7.24s/it]
 56%|█████▋    | 126936/225000 [257:35:20<168:40:47,  6.19s/it]
 56%|█████▋    | 126937/225000 [257:35:29<187:18:40,  6.88s/it]
 56%|█████▋    | 126938/225000 [257:35:34<172:56:51,  6.35s/it]
 56%|█████▋    | 126939/225000 [257:35:38<153:54:42,  5.65s/it]
 56%|█████▋    | 126940/225000 [257:35:42<142:18:52,  5.22s/it]
 56%|█████▋    | 126941/225000 [257:35:46<133:09:22,  4.89s/it]
 56%|█████▋    | 126942/225000 [257:35:51<133:11:11,  4.89s/it]
 56%|█████▋    | 126943/225000 [257:35:56<132:49:18,  4.88s/it]
 56%|█████▋    | 126944/225000 [257:36:03<148:33:51,  5.45s/it]
 56%|█████▋    | 126945/225000 [257:36:15<204:36:10,  7.51s/it]
 56%|█████▋    | 126946/225000 [257:36:20<181:24:57,  6.66s/it]
 56%|█████▋    | 126947/225000 [257:36:24<163:44:56,  6.01s/it]
 56%|█████▋    | 126948/225000 [257:36:29<151:33:53,  5.56s/it]
 56%|█████▋    | 126949/225000 [257:36:36<163:54:40,  6.02s/it]
 56%|█████▋    | 126950/225000 [257:36:46<196:06:44,  7.20s/it]
                                                               
{'loss': '0.8221', 'grad_norm': '1.033', 'learning_rate': '2.158e-05', 'epoch': '0.5642', 'num_input_tokens_seen': 428646824, 'train_runtime': '9.274e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 126950/225000 [257:36:46<196:06:44,  7.20s/it]
 56%|█████▋    | 126951/225000 [257:36:50<173:44:55,  6.38s/it]
 56%|█████▋    | 126952/225000 [257:36:56<168:43:25,  6.19s/it]
 56%|█████▋    | 126953/225000 [257:37:00<147:38:15,  5.42s/it]
 56%|█████▋    | 126954/225000 [257:37:04<137:54:26,  5.06s/it]
 56%|█████▋    | 126955/225000 [257:37:08<132:29:16,  4.86s/it]
 56%|█████▋    | 126956/225000 [257:37:13<131:49:18,  4.84s/it]
 56%|█████▋    | 126957/225000 [257:37:22<167:26:06,  6.15s/it]
 56%|█████▋    | 126958/225000 [257:37:32<193:31:42,  7.11s/it]
 56%|█████▋    | 126959/225000 [257:37:35<163:25:33,  6.00s/it]
 56%|█████▋    | 126960/225000 [257:37:41<161:21:04,  5.92s/it]
 56%|█████▋    | 126961/225000 [257:37:47<166:23:10,  6.11s/it]
 56%|█████▋    | 126962/225000 [257:37:51<141:46:57,  5.21s/it]
 56%|█████▋    | 126963/225000 [257:38:01<184:34:57,  6.78s/it]
 56%|█████▋    | 126964/225000 [257:38:05<161:54:44,  5.95s/it]
 56%|█████▋    | 126965/225000 [257:38:10<150:41:56,  5.53s/it]
 56%|█████▋    | 126966/225000 [257:38:14<140:25:28,  5.16s/it]
 56%|█████▋    | 126967/225000 [257:38:21<152:47:49,  5.61s/it]
 56%|█████▋    | 126968/225000 [257:38:28<171:25:30,  6.30s/it]
 56%|█████▋    | 126969/225000 [257:38:35<177:15:45,  6.51s/it]
 56%|█████▋    | 126970/225000 [257:38:40<157:17:21,  5.78s/it]
 56%|█████▋    | 126971/225000 [257:38:43<139:36:40,  5.13s/it]
 56%|█████▋    | 126972/225000 [257:38:54<189:25:53,  6.96s/it]
 56%|█████▋    | 126973/225000 [257:38:59<166:44:08,  6.12s/it]
 56%|█████▋    | 126974/225000 [257:39:04<161:57:36,  5.95s/it]
 56%|█████▋    | 126975/225000 [257:39:08<145:19:15,  5.34s/it]
 56%|█████▋    | 126976/225000 [257:39:13<140:25:59,  5.16s/it]
 56%|█████▋    | 126977/225000 [257:39:18<139:34:18,  5.13s/it]
 56%|█████▋    | 126978/225000 [257:39:23<141:17:01,  5.19s/it]
 56%|█████▋    | 126979/225000 [257:39:28<141:48:30,  5.21s/it]
 56%|█████▋    | 126980/225000 [257:39:34<142:48:54,  5.25s/it]
 56%|█████▋    | 126981/225000 [257:39:42<167:19:14,  6.15s/it]
 56%|█████▋    | 126982/225000 [257:39:52<201:49:41,  7.41s/it]
 56%|█████▋    | 126983/225000 [257:39:57<179:06:56,  6.58s/it]
 56%|█████▋    | 126984/225000 [257:40:03<176:39:59,  6.49s/it]
 56%|█████▋    | 126985/225000 [257:40:11<186:59:51,  6.87s/it]
 56%|█████▋    | 126986/225000 [257:40:14<156:45:36,  5.76s/it]
 56%|█████▋    | 126987/225000 [257:40:22<173:59:13,  6.39s/it]
 56%|█████▋    | 126988/225000 [257:40:26<152:07:54,  5.59s/it]
 56%|█████▋    | 126989/225000 [257:40:35<181:22:29,  6.66s/it]
 56%|█████▋    | 126990/225000 [257:40:38<154:18:59,  5.67s/it]
 56%|█████▋    | 126991/225000 [257:40:46<173:25:10,  6.37s/it]
 56%|█████▋    | 126992/225000 [257:40:50<152:27:38,  5.60s/it]
 56%|█████▋    | 126993/225000 [257:40:54<141:55:06,  5.21s/it]
 56%|█████▋    | 126994/225000 [257:41:03<168:31:18,  6.19s/it]
 56%|█████▋    | 126995/225000 [257:41:10<178:12:45,  6.55s/it]
 56%|█████▋    | 126996/225000 [257:41:14<151:38:34,  5.57s/it]
 56%|█████▋    | 126997/225000 [257:41:23<184:07:15,  6.76s/it]
 56%|█████▋    | 126998/225000 [257:41:29<173:24:53,  6.37s/it]
 56%|█████▋    | 126999/225000 [257:41:32<150:29:10,  5.53s/it]
 56%|█████▋    | 127000/225000 [257:41:36<136:04:07,  5.00s/it]
                                                               
{'loss': '0.8249', 'grad_norm': '1.052', 'learning_rate': '2.156e-05', 'epoch': '0.5644', 'num_input_tokens_seen': 428775640, 'train_runtime': '9.277e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 127000/225000 [257:41:36<136:04:07,  5.00s/it]
 56%|█████▋    | 127001/225000 [257:41:40<129:36:43,  4.76s/it]
 56%|█████▋    | 127002/225000 [257:41:45<132:17:20,  4.86s/it]
 56%|█████▋    | 127003/225000 [257:41:50<129:38:29,  4.76s/it]
 56%|█████▋    | 127004/225000 [257:41:55<131:54:34,  4.85s/it]
 56%|█████▋    | 127005/225000 [257:41:59<130:02:45,  4.78s/it]
 56%|█████▋    | 127006/225000 [257:42:04<131:21:14,  4.83s/it]
 56%|█████▋    | 127007/225000 [257:42:09<132:22:03,  4.86s/it]
 56%|█████▋    | 127008/225000 [257:42:14<131:40:34,  4.84s/it]
 56%|█████▋    | 127009/225000 [257:42:21<153:03:57,  5.62s/it]
 56%|█████▋    | 127010/225000 [257:42:34<208:24:32,  7.66s/it]
 56%|█████▋    | 127011/225000 [257:42:37<172:10:28,  6.33s/it]
 56%|█████▋    | 127012/225000 [257:42:42<158:39:44,  5.83s/it]
 56%|█████▋    | 127013/225000 [257:42:47<150:16:09,  5.52s/it]
 56%|█████▋    | 127014/225000 [257:42:51<143:13:24,  5.26s/it]
 56%|█████▋    | 127015/225000 [257:42:56<140:13:14,  5.15s/it]
 56%|█████▋    | 127016/225000 [257:43:10<210:11:19,  7.72s/it]
 56%|█████▋    | 127017/225000 [257:43:15<187:18:27,  6.88s/it]
 56%|█████▋    | 127018/225000 [257:43:20<171:33:50,  6.30s/it]
 56%|█████▋    | 127019/225000 [257:43:25<163:09:25,  5.99s/it]
 56%|█████▋    | 127020/225000 [257:43:30<152:14:40,  5.59s/it]
 56%|█████▋    | 127021/225000 [257:43:34<145:44:50,  5.36s/it]
 56%|█████▋    | 127022/225000 [257:43:40<143:50:26,  5.29s/it]
 56%|█████▋    | 127023/225000 [257:43:46<152:41:11,  5.61s/it]
 56%|█████▋    | 127024/225000 [257:43:58<207:44:59,  7.63s/it]
 56%|█████▋    | 127025/225000 [257:44:04<192:35:52,  7.08s/it]
 56%|█████▋    | 127026/225000 [257:44:11<188:38:59,  6.93s/it]
 56%|█████▋    | 127027/225000 [257:44:16<173:18:02,  6.37s/it]
 56%|█████▋    | 127028/225000 [257:44:21<162:03:12,  5.95s/it]
 56%|█████▋    | 127029/225000 [257:44:26<153:22:31,  5.64s/it]
 56%|█████▋    | 127030/225000 [257:44:31<149:38:42,  5.50s/it]
 56%|█████▋    | 127031/225000 [257:44:36<147:13:45,  5.41s/it]
 56%|█████▋    | 127032/225000 [257:44:41<142:06:40,  5.22s/it]
 56%|█████▋    | 127033/225000 [257:44:47<146:56:59,  5.40s/it]
 56%|█████▋    | 127034/225000 [257:45:00<209:30:40,  7.70s/it]
 56%|█████▋    | 127035/225000 [257:45:03<176:11:27,  6.47s/it]
 56%|█████▋    | 127036/225000 [257:45:08<163:33:16,  6.01s/it]
 56%|█████▋    | 127037/225000 [257:45:16<174:28:37,  6.41s/it]
 56%|█████▋    | 127038/225000 [257:45:22<174:43:14,  6.42s/it]
 56%|█████▋    | 127039/225000 [257:45:26<151:48:16,  5.58s/it]
 56%|█████▋    | 127040/225000 [257:45:36<193:58:15,  7.13s/it]
 56%|█████▋    | 127041/225000 [257:45:40<164:01:00,  6.03s/it]
 56%|█████▋    | 127042/225000 [257:45:44<148:26:20,  5.46s/it]
 56%|█████▋    | 127043/225000 [257:45:52<169:02:54,  6.21s/it]
 56%|█████▋    | 127044/225000 [257:45:59<176:03:35,  6.47s/it]
 56%|█████▋    | 127045/225000 [257:46:03<155:13:59,  5.71s/it]
 56%|█████▋    | 127046/225000 [257:46:06<137:05:53,  5.04s/it]
 56%|█████▋    | 127047/225000 [257:46:15<170:07:43,  6.25s/it]
 56%|█████▋    | 127048/225000 [257:46:20<155:19:36,  5.71s/it]
 56%|█████▋    | 127049/225000 [257:46:27<163:33:59,  6.01s/it]
 56%|█████▋    | 127050/225000 [257:46:31<146:28:36,  5.38s/it]
                                                               
{'loss': '0.8295', 'grad_norm': '1.222', 'learning_rate': '2.154e-05', 'epoch': '0.5647', 'num_input_tokens_seen': 428906184, 'train_runtime': '9.28e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 127050/225000 [257:46:31<146:28:36,  5.38s/it]
 56%|█████▋    | 127051/225000 [257:46:35<137:54:17,  5.07s/it]
 56%|█████▋    | 127052/225000 [257:46:46<187:30:19,  6.89s/it]
 56%|█████▋    | 127053/225000 [257:46:49<156:10:24,  5.74s/it]
 56%|█████▋    | 127054/225000 [257:46:53<140:13:10,  5.15s/it]
 56%|█████▋    | 127055/225000 [257:46:57<133:16:26,  4.90s/it]
 56%|█████▋    | 127056/225000 [257:47:07<174:14:50,  6.40s/it]
 56%|█████▋    | 127057/225000 [257:47:13<170:28:39,  6.27s/it]
 56%|█████▋    | 127058/225000 [257:47:17<153:55:46,  5.66s/it]
 56%|█████▋    | 127059/225000 [257:47:22<145:00:24,  5.33s/it]
 56%|█████▋    | 127060/225000 [257:47:26<134:49:31,  4.96s/it]
 56%|█████▋    | 127061/225000 [257:47:31<135:13:20,  4.97s/it]
 56%|█████▋    | 127062/225000 [257:47:36<135:11:48,  4.97s/it]
 56%|█████▋    | 127063/225000 [257:47:47<184:31:51,  6.78s/it]
 56%|█████▋    | 127064/225000 [257:47:53<180:51:00,  6.65s/it]
 56%|█████▋    | 127065/225000 [257:47:58<162:31:20,  5.97s/it]
 56%|█████▋    | 127066/225000 [257:48:02<146:18:12,  5.38s/it]
 56%|█████▋    | 127067/225000 [257:48:06<138:26:09,  5.09s/it]
 56%|█████▋    | 127068/225000 [257:48:10<131:35:43,  4.84s/it]
 56%|█████▋    | 127069/225000 [257:48:20<169:13:30,  6.22s/it]
 56%|█████▋    | 127070/225000 [257:48:26<168:08:13,  6.18s/it]
 56%|█████▋    | 127071/225000 [257:48:31<157:24:04,  5.79s/it]
 56%|█████▋    | 127072/225000 [257:48:37<160:16:07,  5.89s/it]
 56%|█████▋    | 127073/225000 [257:48:45<180:22:46,  6.63s/it]
 56%|█████▋    | 127074/225000 [257:48:50<167:20:35,  6.15s/it]
 56%|█████▋    | 127075/225000 [257:48:56<166:25:12,  6.12s/it]
 56%|█████▋    | 127076/225000 [257:49:01<154:29:05,  5.68s/it]
 56%|█████▋    | 127077/225000 [257:49:05<145:07:10,  5.34s/it]
 56%|█████▋    | 127078/225000 [257:49:10<137:13:30,  5.04s/it]
 56%|█████▋    | 127079/225000 [257:49:20<180:43:24,  6.64s/it]
 56%|█████▋    | 127080/225000 [257:49:26<176:22:57,  6.48s/it]
 56%|█████▋    | 127081/225000 [257:49:31<158:12:54,  5.82s/it]
 56%|█████▋    | 127082/225000 [257:49:40<187:32:16,  6.89s/it]
 56%|█████▋    | 127083/225000 [257:49:43<158:26:58,  5.83s/it]
 56%|█████▋    | 127084/225000 [257:49:52<179:29:22,  6.60s/it]
 56%|█████▋    | 127085/225000 [257:49:55<153:26:01,  5.64s/it]
 56%|█████▋    | 127086/225000 [257:50:04<178:36:55,  6.57s/it]
 56%|█████▋    | 127087/225000 [257:50:08<157:38:21,  5.80s/it]
 56%|█████▋    | 127088/225000 [257:50:15<165:46:48,  6.10s/it]
 56%|█████▋    | 127089/225000 [257:50:18<144:52:30,  5.33s/it]
 56%|█████▋    | 127090/225000 [257:50:28<183:11:18,  6.74s/it]
 56%|█████▋    | 127091/225000 [257:50:32<158:02:28,  5.81s/it]
 56%|█████▋    | 127092/225000 [257:50:36<141:39:05,  5.21s/it]
 56%|█████▋    | 127093/225000 [257:50:46<181:14:58,  6.66s/it]
 56%|█████▋    | 127094/225000 [257:50:49<155:33:11,  5.72s/it]
 56%|█████▋    | 127095/225000 [257:50:54<149:37:27,  5.50s/it]
 56%|█████▋    | 127096/225000 [257:51:02<168:19:48,  6.19s/it]
 56%|█████▋    | 127097/225000 [257:51:05<146:32:30,  5.39s/it]
 56%|█████▋    | 127098/225000 [257:51:13<162:45:33,  5.98s/it]
 56%|█████▋    | 127099/225000 [257:51:17<150:21:55,  5.53s/it]
 56%|█████▋    | 127100/225000 [257:51:22<144:35:58,  5.32s/it]
                                                               
{'loss': '0.8151', 'grad_norm': '1.025', 'learning_rate': '2.153e-05', 'epoch': '0.5649', 'num_input_tokens_seen': 429034120, 'train_runtime': '9.283e+05', 'train_tokens_per_second': '462.2'}

 56%|█████▋    | 127100/225000 [257:51:22<144:35:58,  5.32s/it]
 56%|█████▋    | 127101/225000 [257:51:27<138:49:49,  5.11s/it]
 56%|█████▋    | 127102/225000 [257:51:31<133:45:43,  4.92s/it]
 56%|█████▋    | 127103/225000 [257:51:43<192:17:02,  7.07s/it]
 56%|█████▋    | 127104/225000 [257:51:49<184:43:54,  6.79s/it]
 56%|█████▋    | 127105/225000 [257:51:54<163:04:58,  6.00s/it]
 56%|█████▋    | 127106/225000 [257:51:59<156:17:49,  5.75s/it]
 56%|█████▋    | 127107/225000 [257:52:06<170:12:04,  6.26s/it]
 56%|█████▋    | 127108/225000 [257:52:14<180:34:22,  6.64s/it]
 56%|█████▋    | 127109/225000 [257:52:18<157:26:31,  5.79s/it]
 56%|█████▋    | 127110/225000 [257:52:23<155:48:59,  5.73s/it]
 56%|█████▋    | 127111/225000 [257:52:29<154:34:08,  5.68s/it]
 56%|█████▋    | 127112/225000 [257:52:34<152:29:48,  5.61s/it]
 56%|█████▋    | 127113/225000 [257:52:43<178:12:12,  6.55s/it]
 56%|█████▋    | 127114/225000 [257:52:47<154:53:24,  5.70s/it]
 56%|█████▋    | 127115/225000 [257:52:51<144:41:20,  5.32s/it]
 56%|█████▋    | 127116/225000 [257:52:56<142:24:56,  5.24s/it]
 56%|█████▋    | 127117/225000 [257:53:01<136:35:54,  5.02s/it]
 56%|█████▋    | 127118/225000 [257:53:09<165:38:10,  6.09s/it]
 56%|█████▋    | 127119/225000 [257:53:20<205:02:20,  7.54s/it]
 56%|█████▋    | 127120/225000 [257:53:25<180:29:25,  6.64s/it]
 56%|█████▋    | 127121/225000 [257:53:30<166:48:27,  6.14s/it]
 56%|█████▋    | 127122/225000 [257:53:34<152:48:09,  5.62s/it]
 56%|█████▋    | 127123/225000 [257:53:39<144:15:51,  5.31s/it]
 56%|█████▋    | 127124/225000 [257:53:50<194:23:14,  7.15s/it]
 56%|█████▋    | 127125/225000 [257:53:55<178:15:55,  6.56s/it]
 57%|█████▋    | 127126/225000 [257:54:00<163:35:56,  6.02s/it]
 57%|█████▋    | 127127/225000 [257:54:04<150:32:25,  5.54s/it]
 57%|█████▋    | 127128/225000 [257:54:09<144:01:03,  5.30s/it]
 57%|█████▋    | 127129/225000 [257:54:14<140:46:45,  5.18s/it]
 57%|█████▋    | 127130/225000 [257:54:24<176:24:35,  6.49s/it]
 57%|█████▋    | 127131/225000 [257:54:31<179:58:14,  6.62s/it]
 57%|█████▋    | 127132/225000 [257:54:34<156:44:08,  5.77s/it]
 57%|█████▋    | 127133/225000 [257:54:43<177:42:32,  6.54s/it]
 57%|█████▋    | 127134/225000 [257:54:48<165:09:53,  6.08s/it]
 57%|█████▋    | 127135/225000 [257:54:52<154:11:28,  5.67s/it]
 57%|█████▋    | 127136/225000 [257:54:57<149:35:23,  5.50s/it]
 57%|█████▋    | 127137/225000 [257:55:02<144:16:26,  5.31s/it]
 57%|█████▋    | 127138/225000 [257:55:08<143:26:21,  5.28s/it]
 57%|█████▋    | 127139/225000 [257:55:18<186:50:40,  6.87s/it]
 57%|█████▋    | 127140/225000 [257:55:22<163:37:25,  6.02s/it]
 57%|█████▋    | 127141/225000 [257:55:31<188:06:43,  6.92s/it]
 57%|█████▋    | 127142/225000 [257:55:35<158:36:29,  5.83s/it]
 57%|█████▋    | 127143/225000 [257:55:41<167:46:46,  6.17s/it]
 57%|█████▋    | 127144/225000 [257:55:46<151:41:30,  5.58s/it]
 57%|█████▋    | 127145/225000 [257:55:50<139:47:18,  5.14s/it]
 57%|█████▋    | 127146/225000 [257:55:58<163:02:20,  6.00s/it]
 57%|█████▋    | 127147/225000 [257:56:06<182:19:26,  6.71s/it]
 57%|█████▋    | 127148/225000 [257:56:10<158:51:47,  5.84s/it]
 57%|█████▋    | 127149/225000 [257:56:17<171:24:00,  6.31s/it]
 57%|█████▋    | 127150/225000 [257:56:23<167:06:10,  6.15s/it]
                                                               
{'loss': '0.8346', 'grad_norm': '1.039', 'learning_rate': '2.151e-05', 'epoch': '0.5651', 'num_input_tokens_seen': 429164056, 'train_runtime': '9.286e+05', 'train_tokens_per_second': '462.2'}

 57%|█████▋    | 127150/225000 [257:56:23<167:06:10,  6.15s/it]
 57%|█████▋    | 127151/225000 [257:56:27<146:30:50,  5.39s/it]
 57%|█████▋    | 127152/225000 [257:56:35<173:47:41,  6.39s/it]
 57%|█████▋    | 127153/225000 [257:56:39<152:56:48,  5.63s/it]
 57%|█████▋    | 127154/225000 [257:56:43<139:26:20,  5.13s/it]
 57%|█████▋    | 127155/225000 [257:56:50<155:23:31,  5.72s/it]
 57%|█████▋    | 127156/225000 [257:56:59<177:34:37,  6.53s/it]
 57%|█████▋    | 127157/225000 [257:57:03<157:27:03,  5.79s/it]
 57%|█████▋    | 127158/225000 [257:57:13<195:23:18,  7.19s/it]
 57%|█████▋    | 127159/225000 [257:57:17<163:42:20,  6.02s/it]
 57%|█████▋    | 127160/225000 [257:57:22<155:06:58,  5.71s/it]
 57%|█████▋    | 127161/225000 [257:57:29<170:00:20,  6.26s/it]
 57%|█████▋    | 127162/225000 [257:57:33<150:47:00,  5.55s/it]
 57%|█████▋    | 127163/225000 [257:57:37<140:43:34,  5.18s/it]
 57%|█████▋    | 127164/225000 [257:57:42<139:23:26,  5.13s/it]
 57%|█████▋    | 127165/225000 [257:57:53<184:45:35,  6.80s/it]
 57%|█████▋    | 127166/225000 [257:57:57<161:43:59,  5.95s/it]
 57%|█████▋    | 127167/225000 [257:58:01<142:52:31,  5.26s/it]
 57%|█████▋    | 127168/225000 [257:58:10<173:47:00,  6.39s/it]
 57%|█████▋    | 127169/225000 [257:58:16<174:16:18,  6.41s/it]
 57%|█████▋    | 127170/225000 [257:58:24<185:25:19,  6.82s/it]
 57%|█████▋    | 127171/225000 [257:58:27<158:26:44,  5.83s/it]
 57%|█████▋    | 127172/225000 [257:58:32<146:12:36,  5.38s/it]
 57%|█████▋    | 127173/225000 [257:58:36<133:07:01,  4.90s/it]
 57%|█████▋    | 127174/225000 [257:58:46<174:03:50,  6.41s/it]
 57%|█████▋    | 127175/225000 [257:58:50<158:42:54,  5.84s/it]
 57%|█████▋    | 127176/225000 [257:58:55<152:45:11,  5.62s/it]
 57%|█████▋    | 127177/225000 [257:59:00<145:51:25,  5.37s/it]
 57%|█████▋    | 127178/225000 [257:59:05<140:00:40,  5.15s/it]
 57%|█████▋    | 127179/225000 [257:59:09<133:24:32,  4.91s/it]
 57%|█████▋    | 127180/225000 [257:59:16<149:10:09,  5.49s/it]
 57%|█████▋    | 127181/225000 [257:59:27<198:01:35,  7.29s/it]
 57%|█████▋    | 127182/225000 [257:59:32<176:23:23,  6.49s/it]
 57%|█████▋    | 127183/225000 [257:59:36<159:35:11,  5.87s/it]
 57%|█████▋    | 127184/225000 [257:59:41<149:39:18,  5.51s/it]
 57%|█████▋    | 127185/225000 [257:59:45<140:16:22,  5.16s/it]
 57%|█████▋    | 127186/225000 [257:59:51<144:10:49,  5.31s/it]
 57%|█████▋    | 127187/225000 [258:00:03<195:59:58,  7.21s/it]
 57%|█████▋    | 127188/225000 [258:00:08<177:31:23,  6.53s/it]
 57%|█████▋    | 127189/225000 [258:00:15<183:14:12,  6.74s/it]
 57%|█████▋    | 127190/225000 [258:00:18<156:28:31,  5.76s/it]
 57%|█████▋    | 127191/225000 [258:00:24<159:59:48,  5.89s/it]
 57%|█████▋    | 127192/225000 [258:00:32<170:23:24,  6.27s/it]
 57%|█████▋    | 127193/225000 [258:00:36<155:04:59,  5.71s/it]
 57%|█████▋    | 127194/225000 [258:00:41<146:41:16,  5.40s/it]
 57%|█████▋    | 127195/225000 [258:00:45<138:40:57,  5.10s/it]
 57%|█████▋    | 127196/225000 [258:00:50<133:00:14,  4.90s/it]
 57%|█████▋    | 127197/225000 [258:01:01<184:31:53,  6.79s/it]
 57%|█████▋    | 127198/225000 [258:01:06<168:32:49,  6.20s/it]
 57%|█████▋    | 127199/225000 [258:01:09<149:45:34,  5.51s/it]
 57%|█████▋    | 127200/225000 [258:01:20<191:00:40,  7.03s/it]
                                                               
{'loss': '0.8136', 'grad_norm': '0.9654', 'learning_rate': '2.149e-05', 'epoch': '0.5653', 'num_input_tokens_seen': 429294568, 'train_runtime': '9.289e+05', 'train_tokens_per_second': '462.2'}

 57%|█████▋    | 127200/225000 [258:01:20<191:00:40,  7.03s/it]
 57%|█████▋    | 127201/225000 [258:01:24<166:36:29,  6.13s/it]
 57%|█████▋    | 127202/225000 [258:01:28<148:03:34,  5.45s/it]
 57%|█████▋    | 127203/225000 [258:01:32<134:02:44,  4.93s/it]
 57%|█████▋    | 127204/225000 [258:01:36<126:14:07,  4.65s/it]
 57%|█████▋    | 127205/225000 [258:01:46<175:47:01,  6.47s/it]
 57%|█████▋    | 127206/225000 [258:01:52<169:22:56,  6.24s/it]
 57%|█████▋    | 127207/225000 [258:01:56<152:16:31,  5.61s/it]
 57%|█████▋    | 127208/225000 [258:02:01<144:07:20,  5.31s/it]
 57%|█████▋    | 127209/225000 [258:02:07<151:14:16,  5.57s/it]
 57%|█████▋    | 127210/225000 [258:02:19<201:26:48,  7.42s/it]
 57%|█████▋    | 127211/225000 [258:02:22<169:07:13,  6.23s/it]
 57%|█████▋    | 127212/225000 [258:02:26<152:53:15,  5.63s/it]
 57%|█████▋    | 127213/225000 [258:02:35<173:18:30,  6.38s/it]
 57%|█████▋    | 127214/225000 [258:02:42<178:15:02,  6.56s/it]
 57%|█████▋    | 127215/225000 [258:02:47<168:37:53,  6.21s/it]
 57%|█████▋    | 127216/225000 [258:02:51<151:42:57,  5.59s/it]
 57%|█████▋    | 127217/225000 [258:02:55<140:56:47,  5.19s/it]
 57%|█████▋    | 127218/225000 [258:03:06<182:45:05,  6.73s/it]
 57%|█████▋    | 127219/225000 [258:03:11<175:19:15,  6.45s/it]
 57%|█████▋    | 127220/225000 [258:03:16<156:07:08,  5.75s/it]
 57%|█████▋    | 127221/225000 [258:03:20<148:21:09,  5.46s/it]
 57%|█████▋    | 127222/225000 [258:03:25<139:51:58,  5.15s/it]
 57%|█████▋    | 127223/225000 [258:03:29<134:09:07,  4.94s/it]
 57%|█████▋    | 127224/225000 [258:03:35<143:49:42,  5.30s/it]
 57%|█████▋    | 127225/225000 [258:03:47<198:36:40,  7.31s/it]
 57%|█████▋    | 127226/225000 [258:03:51<169:45:33,  6.25s/it]
 57%|█████▋    | 127227/225000 [258:03:55<150:10:08,  5.53s/it]
 57%|█████▋    | 127228/225000 [258:03:59<138:34:44,  5.10s/it]
 57%|█████▋    | 127229/225000 [258:04:04<138:11:44,  5.09s/it]
 57%|█████▋    | 127230/225000 [258:04:08<132:09:51,  4.87s/it]
 57%|█████▋    | 127231/225000 [258:04:15<148:02:31,  5.45s/it]
 57%|█████▋    | 127232/225000 [258:04:28<210:48:59,  7.76s/it]
 57%|█████▋    | 127233/225000 [258:04:33<183:50:10,  6.77s/it]
 57%|█████▋    | 127234/225000 [258:04:37<161:43:21,  5.96s/it]
 57%|█████▋    | 127235/225000 [258:04:42<150:55:09,  5.56s/it]
 57%|█████▋    | 127236/225000 [258:04:46<142:14:52,  5.24s/it]
 57%|█████▋    | 127237/225000 [258:04:53<156:00:22,  5.74s/it]
 57%|█████▋    | 127238/225000 [258:05:05<206:36:47,  7.61s/it]
 57%|█████▋    | 127239/225000 [258:05:09<181:16:29,  6.68s/it]
 57%|█████▋    | 127240/225000 [258:05:14<161:18:52,  5.94s/it]
 57%|█████▋    | 127241/225000 [258:05:18<149:03:21,  5.49s/it]
 57%|█████▋    | 127242/225000 [258:05:23<141:26:12,  5.21s/it]
 57%|█████▋    | 127243/225000 [258:05:27<137:52:34,  5.08s/it]
 57%|█████▋    | 127244/225000 [258:05:38<182:22:05,  6.72s/it]
 57%|█████▋    | 127245/225000 [258:05:45<182:52:30,  6.73s/it]
 57%|█████▋    | 127246/225000 [258:05:49<159:01:56,  5.86s/it]
 57%|█████▋    | 127247/225000 [258:05:57<180:15:52,  6.64s/it]
 57%|█████▋    | 127248/225000 [258:06:01<161:22:15,  5.94s/it]
 57%|█████▋    | 127249/225000 [258:06:05<144:01:22,  5.30s/it]
 57%|█████▋    | 127250/225000 [258:06:10<137:16:55,  5.06s/it]
                                                               
{'loss': '0.8208', 'grad_norm': '1.031', 'learning_rate': '2.147e-05', 'epoch': '0.5656', 'num_input_tokens_seen': 429424000, 'train_runtime': '9.292e+05', 'train_tokens_per_second': '462.2'}

 57%|█████▋    | 127250/225000 [258:06:10<137:16:55,  5.06s/it]
 57%|█████▋    | 127251/225000 [258:06:21<189:33:20,  6.98s/it]
 57%|█████▋    | 127252/225000 [258:06:25<162:26:50,  5.98s/it]
 57%|█████▋    | 127253/225000 [258:06:29<146:31:12,  5.40s/it]
 57%|█████▋    | 127254/225000 [258:06:36<165:09:47,  6.08s/it]
 57%|█████▋    | 127255/225000 [258:06:43<170:27:17,  6.28s/it]
 57%|█████▋    | 127256/225000 [258:06:47<151:35:06,  5.58s/it]
 57%|█████▋    | 127257/225000 [258:06:56<174:34:17,  6.43s/it]
 57%|█████▋    | 127258/225000 [258:07:00<157:05:43,  5.79s/it]
 57%|█████▋    | 127259/225000 [258:07:05<149:02:12,  5.49s/it]
 57%|█████▋    | 127260/225000 [258:07:10<145:48:16,  5.37s/it]
 57%|█████▋    | 127261/225000 [258:07:15<143:12:43,  5.27s/it]
 57%|█████▋    | 127262/225000 [258:07:20<146:02:55,  5.38s/it]
 57%|█████▋    | 127263/225000 [258:07:26<145:13:35,  5.35s/it]
 57%|█████▋    | 127264/225000 [258:07:31<143:32:07,  5.29s/it]
 57%|█████▋    | 127265/225000 [258:07:36<144:07:55,  5.31s/it]
 57%|█████▋    | 127266/225000 [258:07:41<143:17:09,  5.28s/it]
 57%|█████▋    | 127267/225000 [258:07:54<202:31:12,  7.46s/it]
 57%|█████▋    | 127268/225000 [258:07:58<171:56:06,  6.33s/it]
 57%|█████▋    | 127269/225000 [258:08:06<186:42:38,  6.88s/it]
 57%|█████▋    | 127270/225000 [258:08:10<163:01:07,  6.00s/it]
 57%|█████▋    | 127271/225000 [258:08:14<144:32:45,  5.32s/it]
 57%|█████▋    | 127272/225000 [258:08:25<192:53:25,  7.11s/it]
 57%|█████▋    | 127273/225000 [258:08:29<165:27:41,  6.10s/it]
 57%|█████▋    | 127274/225000 [258:08:32<144:07:55,  5.31s/it]
 57%|█████▋    | 127275/225000 [258:08:37<139:29:08,  5.14s/it]
 57%|█████▋    | 127276/225000 [258:08:42<139:19:24,  5.13s/it]
 57%|█████▋    | 127277/225000 [258:08:47<140:14:17,  5.17s/it]
 57%|█████▋    | 127278/225000 [258:08:52<140:02:31,  5.16s/it]
 57%|█████▋    | 127279/225000 [258:08:58<141:17:09,  5.20s/it]
 57%|█████▋    | 127280/225000 [258:09:03<141:03:42,  5.20s/it]
 57%|█████▋    | 127281/225000 [258:09:08<139:22:20,  5.13s/it]
 57%|█████▋    | 127282/225000 [258:09:13<137:41:50,  5.07s/it]
 57%|█████▋    | 127283/225000 [258:09:25<199:15:03,  7.34s/it]
 57%|█████▋    | 127284/225000 [258:09:32<194:41:24,  7.17s/it]
 57%|█████▋    | 127285/225000 [258:09:36<172:22:37,  6.35s/it]
 57%|█████▋    | 127286/225000 [258:09:41<155:31:10,  5.73s/it]
 57%|█████▋    | 127287/225000 [258:09:45<144:31:01,  5.32s/it]
 57%|█████▋    | 127288/225000 [258:09:50<137:49:36,  5.08s/it]
 57%|█████▋    | 127289/225000 [258:09:54<132:45:40,  4.89s/it]
 57%|█████▋    | 127290/225000 [258:10:05<179:49:10,  6.63s/it]
 57%|█████▋    | 127291/225000 [258:10:13<194:35:57,  7.17s/it]
 57%|█████▋    | 127292/225000 [258:10:16<162:07:43,  5.97s/it]
 57%|█████▋    | 127293/225000 [258:10:26<190:21:15,  7.01s/it]
 57%|█████▋    | 127294/225000 [258:10:30<164:15:05,  6.05s/it]
 57%|█████▋    | 127295/225000 [258:10:34<146:50:17,  5.41s/it]
 57%|█████▋    | 127296/225000 [258:10:44<187:56:43,  6.93s/it]
 57%|█████▋    | 127297/225000 [258:10:47<159:52:19,  5.89s/it]
 57%|█████▋    | 127298/225000 [258:10:54<165:25:56,  6.10s/it]
 57%|█████▋    | 127299/225000 [258:10:59<158:31:47,  5.84s/it]
 57%|█████▋    | 127300/225000 [258:11:03<138:57:28,  5.12s/it]
                                                               
{'loss': '0.8184', 'grad_norm': '1.136', 'learning_rate': '2.145e-05', 'epoch': '0.5658', 'num_input_tokens_seen': 429553512, 'train_runtime': '9.295e+05', 'train_tokens_per_second': '462.2'}

 57%|█████▋    | 127300/225000 [258:11:03<138:57:28,  5.12s/it]
 57%|█████▋    | 127301/225000 [258:11:13<180:41:42,  6.66s/it]
 57%|█████▋    | 127302/225000 [258:11:16<152:29:45,  5.62s/it]
 57%|█████▋    | 127303/225000 [258:11:23<162:35:21,  5.99s/it]
 57%|█████▋    | 127304/225000 [258:11:26<140:02:10,  5.16s/it]
 57%|█████▋    | 127305/225000 [258:11:35<165:38:40,  6.10s/it]
 57%|█████▋    | 127306/225000 [258:11:38<139:53:53,  5.16s/it]
 57%|█████▋    | 127307/225000 [258:11:42<138:07:43,  5.09s/it]
 57%|█████▋    | 127308/225000 [258:11:50<156:44:22,  5.78s/it]
 57%|█████▋    | 127309/225000 [258:11:54<144:32:20,  5.33s/it]
 57%|█████▋    | 127310/225000 [258:12:01<157:57:23,  5.82s/it]
 57%|█████▋    | 127311/225000 [258:12:05<141:48:31,  5.23s/it]
 57%|█████▋    | 127312/225000 [258:12:15<179:11:36,  6.60s/it]
 57%|█████▋    | 127313/225000 [258:12:19<156:15:41,  5.76s/it]
 57%|█████▋    | 127314/225000 [258:12:27<175:32:49,  6.47s/it]
 57%|█████▋    | 127315/225000 [258:12:30<153:14:10,  5.65s/it]
 57%|█████▋    | 127316/225000 [258:12:35<147:39:36,  5.44s/it]
 57%|█████▋    | 127317/225000 [258:12:40<144:52:06,  5.34s/it]
 57%|█████▋    | 127318/225000 [258:12:46<144:35:25,  5.33s/it]
 57%|█████▋    | 127319/225000 [258:12:51<142:52:40,  5.27s/it]
 57%|█████▋    | 127320/225000 [258:12:56<139:33:23,  5.14s/it]
 57%|█████▋    | 127321/225000 [258:13:01<138:19:47,  5.10s/it]
 57%|█████▋    | 127322/225000 [258:13:06<137:14:13,  5.06s/it]
 57%|█████▋    | 127323/225000 [258:13:12<145:50:38,  5.38s/it]
 57%|█████▋    | 127324/225000 [258:13:25<207:50:07,  7.66s/it]
 57%|█████▋    | 127325/225000 [258:13:30<185:16:46,  6.83s/it]
 57%|█████▋    | 127326/225000 [258:13:34<167:29:31,  6.17s/it]
 57%|█████▋    | 127327/225000 [258:13:39<155:35:26,  5.73s/it]
 57%|█████▋    | 127328/225000 [258:13:44<147:04:09,  5.42s/it]
 57%|█████▋    | 127329/225000 [258:13:49<143:05:23,  5.27s/it]
 57%|█████▋    | 127330/225000 [258:13:54<140:59:55,  5.20s/it]
 57%|█████▋    | 127331/225000 [258:13:59<139:19:07,  5.14s/it]
 57%|█████▋    | 127332/225000 [258:14:04<137:02:38,  5.05s/it]
 57%|█████▋    | 127333/225000 [258:14:16<193:41:00,  7.14s/it]
 57%|█████▋    | 127334/225000 [258:14:23<195:36:14,  7.21s/it]
 57%|█████▋    | 127335/225000 [258:14:26<165:19:03,  6.09s/it]
 57%|█████▋    | 127336/225000 [258:14:31<149:39:03,  5.52s/it]
 57%|█████▋    | 127337/225000 [258:14:39<170:54:15,  6.30s/it]
 57%|█████▋    | 127338/225000 [258:14:43<152:36:18,  5.63s/it]
 57%|█████▋    | 127339/225000 [258:14:47<139:17:39,  5.13s/it]
 57%|█████▋    | 127340/225000 [258:14:57<181:02:12,  6.67s/it]
 57%|█████▋    | 127341/225000 [258:15:02<171:00:47,  6.30s/it]
 57%|█████▋    | 127342/225000 [258:15:10<180:10:05,  6.64s/it]
 57%|█████▋    | 127343/225000 [258:15:15<169:03:01,  6.23s/it]
 57%|█████▋    | 127344/225000 [258:15:21<168:28:09,  6.21s/it]
 57%|█████▋    | 127345/225000 [258:15:26<158:53:08,  5.86s/it]
 57%|█████▋    | 127346/225000 [258:15:31<149:27:01,  5.51s/it]
 57%|█████▋    | 127347/225000 [258:15:35<140:40:21,  5.19s/it]
 57%|█████▋    | 127348/225000 [258:15:40<133:17:13,  4.91s/it]
 57%|█████▋    | 127349/225000 [258:15:44<131:13:11,  4.84s/it]
 57%|█████▋    | 127350/225000 [258:15:52<151:39:15,  5.59s/it]
                                                               
{'loss': '0.8345', 'grad_norm': '1.095', 'learning_rate': '2.144e-05', 'epoch': '0.566', 'num_input_tokens_seen': 429681960, 'train_runtime': '9.298e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127350/225000 [258:15:52<151:39:15,  5.59s/it]
 57%|█████▋    | 127351/225000 [258:16:04<202:41:00,  7.47s/it]
 57%|█████▋    | 127352/225000 [258:16:07<170:43:16,  6.29s/it]
 57%|█████▋    | 127353/225000 [258:16:11<149:00:55,  5.49s/it]
 57%|█████▋    | 127354/225000 [258:16:15<138:50:23,  5.12s/it]
 57%|█████▋    | 127355/225000 [258:16:21<145:31:51,  5.37s/it]
 57%|█████▋    | 127356/225000 [258:16:27<149:01:53,  5.49s/it]
 57%|█████▋    | 127357/225000 [258:16:32<147:36:00,  5.44s/it]
 57%|█████▋    | 127358/225000 [258:16:38<150:33:41,  5.55s/it]
 57%|█████▋    | 127359/225000 [258:16:48<186:58:36,  6.89s/it]
 57%|█████▋    | 127360/225000 [258:16:52<165:43:02,  6.11s/it]
 57%|█████▋    | 127361/225000 [258:16:57<151:00:40,  5.57s/it]
 57%|█████▋    | 127362/225000 [258:17:00<137:58:24,  5.09s/it]
 57%|█████▋    | 127363/225000 [258:17:04<129:07:36,  4.76s/it]
 57%|█████▋    | 127364/225000 [258:17:16<183:56:07,  6.78s/it]
 57%|█████▋    | 127365/225000 [258:17:20<158:20:38,  5.84s/it]
 57%|█████▋    | 127366/225000 [258:17:31<204:43:06,  7.55s/it]
 57%|█████▋    | 127367/225000 [258:17:34<169:54:40,  6.27s/it]
 57%|█████▋    | 127368/225000 [258:17:42<180:58:00,  6.67s/it]
 57%|█████▋    | 127369/225000 [258:17:46<155:44:36,  5.74s/it]
 57%|█████▋    | 127370/225000 [258:17:54<174:53:42,  6.45s/it]
 57%|█████▋    | 127371/225000 [258:17:57<151:34:52,  5.59s/it]
 57%|█████▋    | 127372/225000 [258:18:01<134:46:43,  4.97s/it]
 57%|█████▋    | 127373/225000 [258:18:12<183:27:31,  6.77s/it]
 57%|█████▋    | 127374/225000 [258:18:16<161:15:50,  5.95s/it]
 57%|█████▋    | 127375/225000 [258:18:21<157:17:23,  5.80s/it]
 57%|█████▋    | 127376/225000 [258:18:26<151:06:24,  5.57s/it]
 57%|█████▋    | 127377/225000 [258:18:33<157:02:50,  5.79s/it]
 57%|█████▋    | 127378/225000 [258:18:36<141:30:55,  5.22s/it]
 57%|█████▋    | 127379/225000 [258:18:41<133:07:02,  4.91s/it]
 57%|█████▋    | 127380/225000 [258:18:48<153:31:33,  5.66s/it]
 57%|█████▋    | 127381/225000 [258:18:58<185:57:38,  6.86s/it]
 57%|█████▋    | 127382/225000 [258:19:01<155:54:45,  5.75s/it]
 57%|█████▋    | 127383/225000 [258:19:10<184:34:27,  6.81s/it]
 57%|█████▋    | 127384/225000 [258:19:14<158:52:26,  5.86s/it]
 57%|█████▋    | 127385/225000 [258:19:18<144:18:24,  5.32s/it]
 57%|█████▋    | 127386/225000 [258:19:22<135:59:51,  5.02s/it]
 57%|█████▋    | 127387/225000 [258:19:35<195:17:32,  7.20s/it]
 57%|█████▋    | 127388/225000 [258:19:39<172:53:38,  6.38s/it]
 57%|█████▋    | 127389/225000 [258:19:45<170:40:49,  6.29s/it]
 57%|█████▋    | 127390/225000 [258:19:50<162:26:38,  5.99s/it]
 57%|█████▋    | 127391/225000 [258:19:56<161:40:24,  5.96s/it]
 57%|█████▋    | 127392/225000 [258:20:01<150:34:15,  5.55s/it]
 57%|█████▋    | 127393/225000 [258:20:06<150:10:51,  5.54s/it]
 57%|█████▋    | 127394/225000 [258:20:12<148:21:22,  5.47s/it]
 57%|█████▋    | 127395/225000 [258:20:17<150:07:38,  5.54s/it]
 57%|█████▋    | 127396/225000 [258:20:22<146:36:54,  5.41s/it]
 57%|█████▋    | 127397/225000 [258:20:31<175:36:32,  6.48s/it]
 57%|█████▋    | 127398/225000 [258:20:37<166:09:23,  6.13s/it]
 57%|█████▋    | 127399/225000 [258:20:41<150:59:26,  5.57s/it]
 57%|█████▋    | 127400/225000 [258:20:45<141:01:13,  5.20s/it]
                                                               
{'loss': '0.8264', 'grad_norm': '1.023', 'learning_rate': '2.142e-05', 'epoch': '0.5662', 'num_input_tokens_seen': 429810560, 'train_runtime': '9.3e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127400/225000 [258:20:45<141:01:13,  5.20s/it]
 57%|█████▋    | 127401/225000 [258:20:50<135:00:30,  4.98s/it]
 57%|█████▋    | 127402/225000 [258:20:54<128:26:26,  4.74s/it]
 57%|█████▋    | 127403/225000 [258:21:03<164:02:59,  6.05s/it]
 57%|█████▋    | 127404/225000 [258:21:10<167:30:59,  6.18s/it]
 57%|█████▋    | 127405/225000 [258:21:13<147:17:41,  5.43s/it]
 57%|█████▋    | 127406/225000 [258:21:18<143:01:34,  5.28s/it]
 57%|█████▋    | 127407/225000 [258:21:23<137:31:45,  5.07s/it]
 57%|█████▋    | 127408/225000 [258:21:32<174:15:09,  6.43s/it]
 57%|█████▋    | 127409/225000 [258:21:41<188:50:33,  6.97s/it]
 57%|█████▋    | 127410/225000 [258:21:44<159:28:43,  5.88s/it]
 57%|█████▋    | 127411/225000 [258:21:48<141:59:29,  5.24s/it]
 57%|█████▋    | 127412/225000 [258:21:56<166:58:22,  6.16s/it]
 57%|█████▋    | 127413/225000 [258:22:03<171:50:02,  6.34s/it]
 57%|█████▋    | 127414/225000 [258:22:07<152:19:07,  5.62s/it]
 57%|█████▋    | 127415/225000 [258:22:13<161:18:36,  5.95s/it]
 57%|█████▋    | 127416/225000 [258:22:22<178:41:44,  6.59s/it]
 57%|█████▋    | 127417/225000 [258:22:25<154:18:22,  5.69s/it]
 57%|█████▋    | 127418/225000 [258:22:30<148:00:45,  5.46s/it]
 57%|█████▋    | 127419/225000 [258:22:35<140:39:38,  5.19s/it]
 57%|█████▋    | 127420/225000 [258:22:40<141:04:03,  5.20s/it]
 57%|█████▋    | 127421/225000 [258:22:45<140:03:24,  5.17s/it]
 57%|█████▋    | 127422/225000 [258:22:50<136:47:37,  5.05s/it]
 57%|█████▋    | 127423/225000 [258:22:54<133:30:06,  4.93s/it]
 57%|█████▋    | 127424/225000 [258:22:59<133:11:48,  4.91s/it]
 57%|█████▋    | 127425/225000 [258:23:10<178:01:14,  6.57s/it]
 57%|█████▋    | 127426/225000 [258:23:18<195:11:20,  7.20s/it]
 57%|█████▋    | 127427/225000 [258:23:23<170:59:11,  6.31s/it]
 57%|█████▋    | 127428/225000 [258:23:29<171:27:29,  6.33s/it]
 57%|█████▋    | 127429/225000 [258:23:37<186:46:27,  6.89s/it]
 57%|█████▋    | 127430/225000 [258:23:43<178:14:38,  6.58s/it]
 57%|█████▋    | 127431/225000 [258:23:48<167:31:25,  6.18s/it]
 57%|█████▋    | 127432/225000 [258:23:52<149:31:57,  5.52s/it]
 57%|█████▋    | 127433/225000 [258:24:00<171:59:50,  6.35s/it]
 57%|█████▋    | 127434/225000 [258:24:04<147:21:36,  5.44s/it]
 57%|█████▋    | 127435/225000 [258:24:10<157:37:32,  5.82s/it]
 57%|█████▋    | 127436/225000 [258:24:18<175:14:00,  6.47s/it]
 57%|█████▋    | 127437/225000 [258:24:23<158:13:20,  5.84s/it]
 57%|█████▋    | 127438/225000 [258:24:27<148:30:25,  5.48s/it]
 57%|█████▋    | 127439/225000 [258:24:32<140:10:07,  5.17s/it]
 57%|█████▋    | 127440/225000 [258:24:38<150:35:56,  5.56s/it]
 57%|█████▋    | 127441/225000 [258:24:47<173:50:46,  6.42s/it]
 57%|█████▋    | 127442/225000 [258:24:51<152:02:13,  5.61s/it]
 57%|█████▋    | 127443/225000 [258:24:57<157:32:40,  5.81s/it]
 57%|█████▋    | 127444/225000 [258:25:03<164:20:16,  6.06s/it]
 57%|█████▋    | 127445/225000 [258:25:08<151:46:35,  5.60s/it]
 57%|█████▋    | 127446/225000 [258:25:12<139:49:46,  5.16s/it]
 57%|█████▋    | 127447/225000 [258:25:16<133:14:46,  4.92s/it]
 57%|█████▋    | 127448/225000 [258:25:21<128:05:04,  4.73s/it]
 57%|█████▋    | 127449/225000 [258:25:28<149:38:48,  5.52s/it]
 57%|█████▋    | 127450/225000 [258:25:39<194:17:29,  7.17s/it]
                                                               
{'loss': '0.8247', 'grad_norm': '1.206', 'learning_rate': '2.14e-05', 'epoch': '0.5664', 'num_input_tokens_seen': 429940656, 'train_runtime': '9.303e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127450/225000 [258:25:39<194:17:29,  7.17s/it]
 57%|█████▋    | 127451/225000 [258:25:43<166:19:26,  6.14s/it]
 57%|█████▋    | 127452/225000 [258:25:48<157:23:04,  5.81s/it]
 57%|█████▋    | 127453/225000 [258:25:52<144:24:33,  5.33s/it]
 57%|█████▋    | 127454/225000 [258:25:57<141:30:21,  5.22s/it]
 57%|█████▋    | 127455/225000 [258:26:02<139:11:45,  5.14s/it]
 57%|█████▋    | 127456/225000 [258:26:13<189:23:11,  6.99s/it]
 57%|█████▋    | 127457/225000 [258:26:18<172:27:50,  6.37s/it]
 57%|█████▋    | 127458/225000 [258:26:25<178:47:55,  6.60s/it]
 57%|█████▋    | 127459/225000 [258:26:31<168:47:31,  6.23s/it]
 57%|█████▋    | 127460/225000 [258:26:36<160:15:38,  5.91s/it]
 57%|█████▋    | 127461/225000 [258:26:43<165:36:49,  6.11s/it]
 57%|█████▋    | 127462/225000 [258:26:46<142:28:40,  5.26s/it]
 57%|█████▋    | 127463/225000 [258:26:53<157:40:52,  5.82s/it]
 57%|█████▋    | 127464/225000 [258:26:59<156:55:53,  5.79s/it]
 57%|█████▋    | 127465/225000 [258:27:05<160:34:59,  5.93s/it]
 57%|█████▋    | 127466/225000 [258:27:10<154:19:51,  5.70s/it]
 57%|█████▋    | 127467/225000 [258:27:18<172:23:52,  6.36s/it]
 57%|█████▋    | 127468/225000 [258:27:22<156:28:48,  5.78s/it]
 57%|█████▋    | 127469/225000 [258:27:27<144:35:18,  5.34s/it]
 57%|█████▋    | 127470/225000 [258:27:31<136:17:52,  5.03s/it]
 57%|█████▋    | 127471/225000 [258:27:35<128:30:16,  4.74s/it]
 57%|█████▋    | 127472/225000 [258:27:40<127:48:51,  4.72s/it]
 57%|█████▋    | 127473/225000 [258:27:52<192:37:44,  7.11s/it]
 57%|█████▋    | 127474/225000 [258:27:56<160:26:07,  5.92s/it]
 57%|█████▋    | 127475/225000 [258:28:04<177:09:46,  6.54s/it]
 57%|█████▋    | 127476/225000 [258:28:07<152:45:09,  5.64s/it]
 57%|█████▋    | 127477/225000 [258:28:11<141:13:29,  5.21s/it]
 57%|█████▋    | 127478/225000 [258:28:16<138:08:18,  5.10s/it]
 57%|█████▋    | 127479/225000 [258:28:21<136:08:28,  5.03s/it]
 57%|█████▋    | 127480/225000 [258:28:26<132:48:03,  4.90s/it]
 57%|█████▋    | 127481/225000 [258:28:36<177:38:44,  6.56s/it]
 57%|█████▋    | 127482/225000 [258:28:44<185:12:05,  6.84s/it]
 57%|█████▋    | 127483/225000 [258:28:49<176:21:26,  6.51s/it]
 57%|█████▋    | 127484/225000 [258:28:55<169:24:53,  6.25s/it]
 57%|█████▋    | 127485/225000 [258:28:59<154:33:38,  5.71s/it]
 57%|█████▋    | 127486/225000 [258:29:04<144:21:06,  5.33s/it]
 57%|█████▋    | 127487/225000 [258:29:08<137:34:40,  5.08s/it]
 57%|█████▋    | 127488/225000 [258:29:20<190:39:05,  7.04s/it]
 57%|█████▋    | 127489/225000 [258:29:24<165:36:04,  6.11s/it]
 57%|█████▋    | 127490/225000 [258:29:32<180:21:14,  6.66s/it]
 57%|█████▋    | 127491/225000 [258:29:35<152:11:16,  5.62s/it]
 57%|█████▋    | 127492/225000 [258:29:39<141:58:48,  5.24s/it]
 57%|█████▋    | 127493/225000 [258:29:44<136:05:23,  5.02s/it]
 57%|█████▋    | 127494/225000 [258:29:55<183:42:37,  6.78s/it]
 57%|█████▋    | 127495/225000 [258:30:00<171:53:57,  6.35s/it]
 57%|█████▋    | 127496/225000 [258:30:08<185:36:24,  6.85s/it]
 57%|█████▋    | 127497/225000 [258:30:11<156:43:39,  5.79s/it]
 57%|█████▋    | 127498/225000 [258:30:15<141:14:53,  5.22s/it]
 57%|█████▋    | 127499/225000 [258:30:24<168:30:58,  6.22s/it]
 57%|█████▋    | 127500/225000 [258:30:28<150:24:39,  5.55s/it]
                                                               
{'loss': '0.8213', 'grad_norm': '1.119', 'learning_rate': '2.138e-05', 'epoch': '0.5667', 'num_input_tokens_seen': 430067904, 'train_runtime': '9.306e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127500/225000 [258:30:28<150:24:39,  5.55s/it]
 57%|█████▋    | 127501/225000 [258:30:32<138:04:36,  5.10s/it]
 57%|█████▋    | 127502/225000 [258:30:38<143:47:33,  5.31s/it]
 57%|█████▋    | 127503/225000 [258:30:48<180:58:58,  6.68s/it]
 57%|█████▋    | 127504/225000 [258:30:50<149:59:53,  5.54s/it]
 57%|█████▋    | 127505/225000 [258:30:57<161:38:40,  5.97s/it]
 57%|█████▋    | 127506/225000 [258:31:01<144:28:38,  5.33s/it]
 57%|█████▋    | 127507/225000 [258:31:05<132:28:30,  4.89s/it]
 57%|█████▋    | 127508/225000 [258:31:13<156:20:44,  5.77s/it]
 57%|█████▋    | 127509/225000 [258:31:21<170:57:33,  6.31s/it]
 57%|█████▋    | 127510/225000 [258:31:24<147:30:49,  5.45s/it]
 57%|█████▋    | 127511/225000 [258:31:33<177:13:00,  6.54s/it]
 57%|█████▋    | 127512/225000 [258:31:37<152:54:39,  5.65s/it]
 57%|█████▋    | 127513/225000 [258:31:40<137:28:15,  5.08s/it]
 57%|█████▋    | 127514/225000 [258:31:44<127:26:18,  4.71s/it]
 57%|█████▋    | 127515/225000 [258:31:55<177:36:01,  6.56s/it]
 57%|█████▋    | 127516/225000 [258:31:59<156:31:38,  5.78s/it]
 57%|█████▋    | 127517/225000 [258:32:03<143:04:09,  5.28s/it]
 57%|█████▋    | 127518/225000 [258:32:09<146:35:19,  5.41s/it]
 57%|█████▋    | 127519/225000 [258:32:17<165:00:00,  6.09s/it]
 57%|█████▋    | 127520/225000 [258:32:20<146:45:34,  5.42s/it]
 57%|█████▋    | 127521/225000 [258:32:25<139:44:57,  5.16s/it]
 57%|█████▋    | 127522/225000 [258:32:29<134:26:09,  4.96s/it]
 57%|█████▋    | 127523/225000 [258:32:34<131:06:25,  4.84s/it]
 57%|█████▋    | 127524/225000 [258:32:39<129:42:07,  4.79s/it]
 57%|█████▋    | 127525/225000 [258:32:43<129:39:15,  4.79s/it]
 57%|█████▋    | 127526/225000 [258:32:52<161:47:17,  5.98s/it]
 57%|█████▋    | 127527/225000 [258:33:02<196:32:23,  7.26s/it]
 57%|█████▋    | 127528/225000 [258:33:06<164:03:15,  6.06s/it]
 57%|█████▋    | 127529/225000 [258:33:17<203:15:24,  7.51s/it]
 57%|█████▋    | 127530/225000 [258:33:20<168:51:25,  6.24s/it]
 57%|█████▋    | 127531/225000 [258:33:27<175:51:23,  6.50s/it]
 57%|█████▋    | 127532/225000 [258:33:31<151:59:55,  5.61s/it]
 57%|█████▋    | 127533/225000 [258:33:36<152:38:03,  5.64s/it]
 57%|█████▋    | 127534/225000 [258:33:45<174:55:10,  6.46s/it]
 57%|█████▋    | 127535/225000 [258:33:48<152:18:10,  5.63s/it]
 57%|█████▋    | 127536/225000 [258:33:52<138:03:42,  5.10s/it]
 57%|█████▋    | 127537/225000 [258:33:56<128:10:43,  4.73s/it]
 57%|█████▋    | 127538/225000 [258:34:02<137:07:14,  5.06s/it]
 57%|█████▋    | 127539/225000 [258:34:13<186:24:58,  6.89s/it]
 57%|█████▋    | 127540/225000 [258:34:16<158:10:50,  5.84s/it]
 57%|█████▋    | 127541/225000 [258:34:24<171:00:34,  6.32s/it]
 57%|█████▋    | 127542/225000 [258:34:29<160:30:46,  5.93s/it]
 57%|█████▋    | 127543/225000 [258:34:34<155:38:21,  5.75s/it]
 57%|█████▋    | 127544/225000 [258:34:39<149:40:37,  5.53s/it]
 57%|█████▋    | 127545/225000 [258:34:44<145:48:59,  5.39s/it]
 57%|█████▋    | 127546/225000 [258:34:49<143:46:57,  5.31s/it]
 57%|█████▋    | 127547/225000 [258:34:55<145:36:03,  5.38s/it]
 57%|█████▋    | 127548/225000 [258:35:01<148:17:31,  5.48s/it]
 57%|█████▋    | 127549/225000 [258:35:06<143:51:49,  5.31s/it]
 57%|█████▋    | 127550/225000 [258:35:11<141:56:28,  5.24s/it]
                                                               
{'loss': '0.8114', 'grad_norm': '0.9799', 'learning_rate': '2.136e-05', 'epoch': '0.5669', 'num_input_tokens_seen': 430196312, 'train_runtime': '9.309e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127550/225000 [258:35:11<141:56:28,  5.24s/it]
 57%|█████▋    | 127551/225000 [258:35:16<143:21:37,  5.30s/it]
 57%|█████▋    | 127552/225000 [258:35:21<142:02:13,  5.25s/it]
 57%|█████▋    | 127553/225000 [258:35:31<179:39:34,  6.64s/it]
 57%|█████▋    | 127554/225000 [258:35:40<201:27:28,  7.44s/it]
 57%|█████▋    | 127555/225000 [258:35:45<175:08:33,  6.47s/it]
 57%|█████▋    | 127556/225000 [258:35:53<194:17:55,  7.18s/it]
 57%|█████▋    | 127557/225000 [258:35:57<165:27:23,  6.11s/it]
 57%|█████▋    | 127558/225000 [258:36:05<177:32:00,  6.56s/it]
 57%|█████▋    | 127559/225000 [258:36:10<164:47:03,  6.09s/it]
 57%|█████▋    | 127560/225000 [258:36:16<165:18:07,  6.11s/it]
 57%|█████▋    | 127561/225000 [258:36:19<142:54:27,  5.28s/it]
 57%|█████▋    | 127562/225000 [258:36:27<162:34:31,  6.01s/it]
 57%|█████▋    | 127563/225000 [258:36:31<147:52:12,  5.46s/it]
 57%|█████▋    | 127564/225000 [258:36:40<178:19:52,  6.59s/it]
 57%|█████▋    | 127565/225000 [258:36:44<153:30:23,  5.67s/it]
 57%|█████▋    | 127566/225000 [258:36:48<138:36:52,  5.12s/it]
 57%|█████▋    | 127567/225000 [258:36:52<135:11:31,  5.00s/it]
 57%|█████▋    | 127568/225000 [258:37:04<188:02:29,  6.95s/it]
 57%|█████▋    | 127569/225000 [258:37:07<158:19:32,  5.85s/it]
 57%|█████▋    | 127570/225000 [258:37:11<143:27:49,  5.30s/it]
 57%|█████▋    | 127571/225000 [258:37:15<134:08:33,  4.96s/it]
 57%|█████▋    | 127572/225000 [258:37:28<195:36:55,  7.23s/it]
 57%|█████▋    | 127573/225000 [258:37:32<166:36:59,  6.16s/it]
 57%|█████▋    | 127574/225000 [258:37:36<155:57:41,  5.76s/it]
 57%|█████▋    | 127575/225000 [258:37:45<180:42:20,  6.68s/it]
 57%|█████▋    | 127576/225000 [258:37:48<152:35:37,  5.64s/it]
 57%|█████▋    | 127577/225000 [258:37:56<164:37:00,  6.08s/it]
 57%|█████▋    | 127578/225000 [258:38:00<154:35:13,  5.71s/it]
 57%|█████▋    | 127579/225000 [258:38:07<163:19:24,  6.04s/it]
 57%|█████▋    | 127580/225000 [258:38:11<143:59:19,  5.32s/it]
 57%|█████▋    | 127581/225000 [258:38:18<160:17:07,  5.92s/it]
 57%|█████▋    | 127582/225000 [258:38:24<162:39:24,  6.01s/it]
 57%|█████▋    | 127583/225000 [258:38:30<158:34:58,  5.86s/it]
 57%|█████▋    | 127584/225000 [258:38:36<160:17:41,  5.92s/it]
 57%|█████▋    | 127585/225000 [258:38:39<140:32:53,  5.19s/it]
 57%|█████▋    | 127586/225000 [258:38:44<135:26:35,  5.01s/it]
 57%|█████▋    | 127587/225000 [258:38:49<133:45:50,  4.94s/it]
 57%|█████▋    | 127588/225000 [258:38:53<130:02:48,  4.81s/it]
 57%|█████▋    | 127589/225000 [258:39:01<153:55:10,  5.69s/it]
 57%|█████▋    | 127590/225000 [258:39:11<185:36:55,  6.86s/it]
 57%|█████▋    | 127591/225000 [258:39:14<155:39:48,  5.75s/it]
 57%|█████▋    | 127592/225000 [258:39:22<174:37:20,  6.45s/it]
 57%|█████▋    | 127593/225000 [258:39:26<152:49:01,  5.65s/it]
 57%|█████▋    | 127594/225000 [258:39:31<151:02:31,  5.58s/it]
 57%|█████▋    | 127595/225000 [258:39:38<162:52:41,  6.02s/it]
 57%|█████▋    | 127596/225000 [258:39:42<144:37:49,  5.35s/it]
 57%|█████▋    | 127597/225000 [258:39:46<132:38:30,  4.90s/it]
 57%|█████▋    | 127598/225000 [258:39:57<186:13:37,  6.88s/it]
 57%|█████▋    | 127599/225000 [258:40:01<159:42:40,  5.90s/it]
 57%|█████▋    | 127600/225000 [258:40:09<176:21:47,  6.52s/it]
                                                               
{'loss': '0.8242', 'grad_norm': '1.083', 'learning_rate': '2.134e-05', 'epoch': '0.5671', 'num_input_tokens_seen': 430324944, 'train_runtime': '9.312e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127600/225000 [258:40:09<176:21:47,  6.52s/it]
 57%|█████▋    | 127601/225000 [258:40:13<155:17:49,  5.74s/it]
 57%|█████▋    | 127602/225000 [258:40:20<168:21:30,  6.22s/it]
 57%|█████▋    | 127603/225000 [258:40:27<177:00:51,  6.54s/it]
 57%|█████▋    | 127604/225000 [258:40:32<158:52:34,  5.87s/it]
 57%|█████▋    | 127605/225000 [258:40:37<151:35:38,  5.60s/it]
 57%|█████▋    | 127606/225000 [258:40:41<144:45:05,  5.35s/it]
 57%|█████▋    | 127607/225000 [258:40:46<139:29:31,  5.16s/it]
 57%|█████▋    | 127608/225000 [258:40:51<139:01:58,  5.14s/it]
 57%|█████▋    | 127609/225000 [258:40:56<132:38:08,  4.90s/it]
 57%|█████▋    | 127610/225000 [258:41:00<131:46:00,  4.87s/it]
 57%|█████▋    | 127611/225000 [258:41:13<194:04:50,  7.17s/it]
 57%|█████▋    | 127612/225000 [258:41:16<163:29:55,  6.04s/it]
 57%|█████▋    | 127613/225000 [258:41:26<190:05:57,  7.03s/it]
 57%|█████▋    | 127614/225000 [258:41:30<166:14:29,  6.15s/it]
 57%|█████▋    | 127615/225000 [258:41:34<148:59:28,  5.51s/it]
 57%|█████▋    | 127616/225000 [258:41:38<136:02:42,  5.03s/it]
 57%|█████▋    | 127617/225000 [258:41:49<185:02:32,  6.84s/it]
 57%|█████▋    | 127618/225000 [258:41:53<166:12:38,  6.14s/it]
 57%|█████▋    | 127619/225000 [258:42:01<177:02:59,  6.55s/it]
 57%|█████▋    | 127620/225000 [258:42:05<162:19:21,  6.00s/it]
 57%|█████▋    | 127621/225000 [258:42:11<154:31:41,  5.71s/it]
 57%|█████▋    | 127622/225000 [258:42:15<147:45:31,  5.46s/it]
 57%|█████▋    | 127623/225000 [258:42:20<144:20:50,  5.34s/it]
 57%|█████▋    | 127624/225000 [258:42:26<143:00:00,  5.29s/it]
 57%|█████▋    | 127625/225000 [258:42:31<142:58:24,  5.29s/it]
 57%|█████▋    | 127626/225000 [258:42:36<142:49:14,  5.28s/it]
 57%|█████▋    | 127627/225000 [258:42:41<138:35:53,  5.12s/it]
 57%|█████▋    | 127628/225000 [258:42:54<199:49:55,  7.39s/it]
 57%|█████▋    | 127629/225000 [258:42:59<183:45:00,  6.79s/it]
 57%|█████▋    | 127630/225000 [258:43:06<187:19:01,  6.93s/it]
 57%|█████▋    | 127631/225000 [258:43:10<160:21:33,  5.93s/it]
 57%|█████▋    | 127632/225000 [258:43:16<159:23:52,  5.89s/it]
 57%|█████▋    | 127633/225000 [258:43:23<168:47:01,  6.24s/it]
 57%|█████▋    | 127634/225000 [258:43:27<156:53:06,  5.80s/it]
 57%|█████▋    | 127635/225000 [258:43:32<149:00:55,  5.51s/it]
 57%|█████▋    | 127636/225000 [258:43:37<144:49:35,  5.35s/it]
 57%|█████▋    | 127637/225000 [258:43:42<141:07:55,  5.22s/it]
 57%|█████▋    | 127638/225000 [258:43:47<138:37:34,  5.13s/it]
 57%|█████▋    | 127639/225000 [258:44:00<198:52:01,  7.35s/it]
 57%|█████▋    | 127640/225000 [258:44:04<172:54:54,  6.39s/it]
 57%|█████▋    | 127641/225000 [258:44:08<151:45:25,  5.61s/it]
 57%|█████▋    | 127642/225000 [258:44:12<139:03:06,  5.14s/it]
 57%|█████▋    | 127643/225000 [258:44:23<193:02:28,  7.14s/it]
 57%|█████▋    | 127644/225000 [258:44:27<167:30:28,  6.19s/it]
 57%|█████▋    | 127645/225000 [258:44:35<178:43:55,  6.61s/it]
 57%|█████▋    | 127646/225000 [258:44:39<159:26:02,  5.90s/it]
 57%|█████▋    | 127647/225000 [258:44:46<168:21:30,  6.23s/it]
 57%|█████▋    | 127648/225000 [258:44:52<162:00:09,  5.99s/it]
 57%|█████▋    | 127649/225000 [258:44:56<149:38:21,  5.53s/it]
 57%|█████▋    | 127650/225000 [258:45:00<138:56:56,  5.14s/it]
                                                               
{'loss': '0.8294', 'grad_norm': '1.039', 'learning_rate': '2.133e-05', 'epoch': '0.5673', 'num_input_tokens_seen': 430454928, 'train_runtime': '9.315e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127650/225000 [258:45:00<138:56:56,  5.14s/it]
 57%|█████▋    | 127651/225000 [258:45:05<132:09:07,  4.89s/it]
 57%|█████▋    | 127652/225000 [258:45:14<171:08:30,  6.33s/it]
 57%|█████▋    | 127653/225000 [258:45:21<174:23:50,  6.45s/it]
 57%|█████▋    | 127654/225000 [258:45:25<150:49:54,  5.58s/it]
 57%|█████▋    | 127655/225000 [258:45:30<148:13:42,  5.48s/it]
 57%|█████▋    | 127656/225000 [258:45:35<144:38:24,  5.35s/it]
 57%|█████▋    | 127657/225000 [258:45:39<138:04:25,  5.11s/it]
 57%|█████▋    | 127658/225000 [258:45:44<136:49:48,  5.06s/it]
 57%|█████▋    | 127659/225000 [258:45:49<135:27:09,  5.01s/it]
 57%|█████▋    | 127660/225000 [258:45:55<136:57:25,  5.07s/it]
 57%|█████▋    | 127661/225000 [258:46:05<177:00:39,  6.55s/it]
 57%|█████▋    | 127662/225000 [258:46:13<193:29:17,  7.16s/it]
 57%|█████▋    | 127663/225000 [258:46:17<167:05:32,  6.18s/it]
 57%|█████▋    | 127664/225000 [258:46:22<159:27:17,  5.90s/it]
 57%|█████▋    | 127665/225000 [258:46:32<192:30:01,  7.12s/it]
 57%|█████▋    | 127666/225000 [258:46:37<171:22:27,  6.34s/it]
 57%|█████▋    | 127667/225000 [258:46:41<153:53:11,  5.69s/it]
 57%|█████▋    | 127668/225000 [258:46:45<140:53:42,  5.21s/it]
 57%|█████▋    | 127669/225000 [258:46:49<133:52:58,  4.95s/it]
 57%|█████▋    | 127670/225000 [258:47:01<191:36:29,  7.09s/it]
 57%|█████▋    | 127671/225000 [258:47:06<170:39:42,  6.31s/it]
 57%|█████▋    | 127672/225000 [258:47:11<158:53:59,  5.88s/it]
 57%|█████▋    | 127673/225000 [258:47:20<185:05:47,  6.85s/it]
 57%|█████▋    | 127674/225000 [258:47:23<156:09:41,  5.78s/it]
 57%|█████▋    | 127675/225000 [258:47:32<180:05:54,  6.66s/it]
 57%|█████▋    | 127676/225000 [258:47:35<154:22:11,  5.71s/it]
 57%|█████▋    | 127677/225000 [258:47:43<167:24:52,  6.19s/it]
 57%|█████▋    | 127678/225000 [258:47:47<148:59:38,  5.51s/it]
 57%|█████▋    | 127679/225000 [258:47:51<140:25:22,  5.19s/it]
 57%|█████▋    | 127680/225000 [258:47:55<130:39:37,  4.83s/it]
 57%|█████▋    | 127681/225000 [258:48:06<183:59:11,  6.81s/it]
 57%|█████▋    | 127682/225000 [258:48:10<157:28:00,  5.83s/it]
 57%|█████▋    | 127683/225000 [258:48:20<189:42:19,  7.02s/it]
 57%|█████▋    | 127684/225000 [258:48:23<159:14:04,  5.89s/it]
 57%|█████▋    | 127685/225000 [258:48:30<169:39:22,  6.28s/it]
 57%|█████▋    | 127686/225000 [258:48:34<152:04:14,  5.63s/it]
 57%|█████▋    | 127687/225000 [258:48:39<142:50:24,  5.28s/it]
 57%|█████▋    | 127688/225000 [258:48:47<163:29:07,  6.05s/it]
 57%|█████▋    | 127689/225000 [258:48:52<155:49:39,  5.76s/it]
 57%|█████▋    | 127690/225000 [258:48:59<170:57:42,  6.32s/it]
 57%|█████▋    | 127691/225000 [258:49:05<165:35:30,  6.13s/it]
 57%|█████▋    | 127692/225000 [258:49:11<161:23:39,  5.97s/it]
 57%|█████▋    | 127693/225000 [258:49:15<144:44:59,  5.36s/it]
 57%|█████▋    | 127694/225000 [258:49:22<162:17:54,  6.00s/it]
 57%|█████▋    | 127695/225000 [258:49:28<158:55:20,  5.88s/it]
 57%|█████▋    | 127696/225000 [258:49:32<146:04:06,  5.40s/it]
 57%|█████▋    | 127697/225000 [258:49:36<136:14:07,  5.04s/it]
 57%|█████▋    | 127698/225000 [258:49:40<128:31:37,  4.76s/it]
 57%|█████▋    | 127699/225000 [258:49:53<190:50:37,  7.06s/it]
 57%|█████▋    | 127700/225000 [258:49:57<166:34:53,  6.16s/it]
                                                               
{'loss': '0.8119', 'grad_norm': '0.9998', 'learning_rate': '2.131e-05', 'epoch': '0.5676', 'num_input_tokens_seen': 430584656, 'train_runtime': '9.318e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127700/225000 [258:49:57<166:34:53,  6.16s/it]
 57%|█████▋    | 127701/225000 [258:50:04<177:37:49,  6.57s/it]
 57%|█████▋    | 127702/225000 [258:50:09<161:01:21,  5.96s/it]
 57%|█████▋    | 127703/225000 [258:50:12<139:38:16,  5.17s/it]
 57%|█████▋    | 127704/225000 [258:50:23<183:51:36,  6.80s/it]
 57%|█████▋    | 127705/225000 [258:50:27<160:59:04,  5.96s/it]
 57%|█████▋    | 127706/225000 [258:50:31<143:05:50,  5.29s/it]
 57%|█████▋    | 127707/225000 [258:50:34<127:16:24,  4.71s/it]
 57%|█████▋    | 127708/225000 [258:50:45<177:19:17,  6.56s/it]
 57%|█████▋    | 127709/225000 [258:50:49<155:09:02,  5.74s/it]
 57%|█████▋    | 127710/225000 [258:50:58<188:51:37,  6.99s/it]
 57%|█████▋    | 127711/225000 [258:51:02<163:56:05,  6.07s/it]
 57%|█████▋    | 127712/225000 [258:51:11<187:26:14,  6.94s/it]
 57%|█████▋    | 127713/225000 [258:51:17<173:41:15,  6.43s/it]
 57%|█████▋    | 127714/225000 [258:51:22<163:02:35,  6.03s/it]
 57%|█████▋    | 127715/225000 [258:51:27<158:07:49,  5.85s/it]
 57%|█████▋    | 127716/225000 [258:51:32<149:42:42,  5.54s/it]
 57%|█████▋    | 127717/225000 [258:51:37<144:55:08,  5.36s/it]
 57%|█████▋    | 127718/225000 [258:51:42<143:27:45,  5.31s/it]
 57%|█████▋    | 127719/225000 [258:51:47<140:46:37,  5.21s/it]
 57%|█████▋    | 127720/225000 [258:51:52<139:11:31,  5.15s/it]
 57%|█████▋    | 127721/225000 [258:51:57<137:07:59,  5.07s/it]
 57%|█████▋    | 127722/225000 [258:52:02<139:15:14,  5.15s/it]
 57%|█████▋    | 127723/225000 [258:52:12<174:10:24,  6.45s/it]
 57%|█████▋    | 127724/225000 [258:52:21<194:41:52,  7.21s/it]
 57%|█████▋    | 127725/225000 [258:52:24<162:29:44,  6.01s/it]
 57%|█████▋    | 127726/225000 [258:52:35<199:44:43,  7.39s/it]
 57%|█████▋    | 127727/225000 [258:52:39<173:04:27,  6.41s/it]
 57%|█████▋    | 127728/225000 [258:52:43<154:46:49,  5.73s/it]
 57%|█████▋    | 127729/225000 [258:52:48<146:39:01,  5.43s/it]
 57%|█████▋    | 127730/225000 [258:52:52<136:48:24,  5.06s/it]
 57%|█████▋    | 127731/225000 [258:52:56<130:27:28,  4.83s/it]
 57%|█████▋    | 127732/225000 [258:53:08<188:16:15,  6.97s/it]
 57%|█████▋    | 127733/225000 [258:53:15<184:23:56,  6.82s/it]
 57%|█████▋    | 127734/225000 [258:53:18<156:47:23,  5.80s/it]
 57%|█████▋    | 127735/225000 [258:53:27<181:40:09,  6.72s/it]
 57%|█████▋    | 127736/225000 [258:53:32<169:35:50,  6.28s/it]
 57%|█████▋    | 127737/225000 [258:53:35<145:56:04,  5.40s/it]
 57%|█████▋    | 127738/225000 [258:53:44<173:59:33,  6.44s/it]
 57%|█████▋    | 127739/225000 [258:53:48<152:27:28,  5.64s/it]
 57%|█████▋    | 127740/225000 [258:53:53<150:41:22,  5.58s/it]
 57%|█████▋    | 127741/225000 [258:54:03<181:55:39,  6.73s/it]
 57%|█████▋    | 127742/225000 [258:54:06<154:27:33,  5.72s/it]
 57%|█████▋    | 127743/225000 [258:54:11<147:47:27,  5.47s/it]
 57%|█████▋    | 127744/225000 [258:54:16<145:41:32,  5.39s/it]
 57%|█████▋    | 127745/225000 [258:54:29<201:55:07,  7.47s/it]
 57%|█████▋    | 127746/225000 [258:54:35<194:52:44,  7.21s/it]
 57%|█████▋    | 127747/225000 [258:54:40<171:14:30,  6.34s/it]
 57%|█████▋    | 127748/225000 [258:54:44<153:12:45,  5.67s/it]
 57%|█████▋    | 127749/225000 [258:54:50<158:53:00,  5.88s/it]
 57%|█████▋    | 127750/225000 [258:54:55<149:02:26,  5.52s/it]
                                                               
{'loss': '0.813', 'grad_norm': '1.106', 'learning_rate': '2.129e-05', 'epoch': '0.5678', 'num_input_tokens_seen': 430715176, 'train_runtime': '9.321e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127750/225000 [258:54:55<149:02:26,  5.52s/it]
 57%|█████▋    | 127751/225000 [258:54:59<138:05:57,  5.11s/it]
 57%|█████▋    | 127752/225000 [258:55:11<197:27:32,  7.31s/it]
 57%|█████▋    | 127753/225000 [258:55:15<167:31:54,  6.20s/it]
 57%|█████▋    | 127754/225000 [258:55:18<145:44:13,  5.40s/it]
 57%|█████▋    | 127755/225000 [258:55:28<177:45:36,  6.58s/it]
 57%|█████▋    | 127756/225000 [258:55:31<153:41:22,  5.69s/it]
 57%|█████▋    | 127757/225000 [258:55:35<132:37:38,  4.91s/it]
 57%|█████▋    | 127758/225000 [258:55:39<126:02:15,  4.67s/it]
 57%|█████▋    | 127759/225000 [258:55:48<162:03:00,  6.00s/it]
 57%|█████▋    | 127760/225000 [258:55:55<173:16:11,  6.41s/it]
 57%|█████▋    | 127761/225000 [258:55:59<151:18:37,  5.60s/it]
 57%|█████▋    | 127762/225000 [258:56:08<182:52:01,  6.77s/it]
 57%|█████▋    | 127763/225000 [258:56:12<155:11:23,  5.75s/it]
 57%|█████▋    | 127764/225000 [258:56:16<144:16:59,  5.34s/it]
 57%|█████▋    | 127765/225000 [258:56:26<179:34:56,  6.65s/it]
 57%|█████▋    | 127766/225000 [258:56:30<159:18:59,  5.90s/it]
 57%|█████▋    | 127767/225000 [258:56:34<147:05:47,  5.45s/it]
 57%|█████▋    | 127768/225000 [258:56:38<136:40:30,  5.06s/it]
 57%|█████▋    | 127769/225000 [258:56:49<184:49:03,  6.84s/it]
 57%|█████▋    | 127770/225000 [258:56:53<158:59:15,  5.89s/it]
 57%|█████▋    | 127771/225000 [258:57:00<164:32:23,  6.09s/it]
 57%|█████▋    | 127772/225000 [258:57:04<151:39:35,  5.62s/it]
 57%|█████▋    | 127773/225000 [258:57:09<146:50:42,  5.44s/it]
 57%|█████▋    | 127774/225000 [258:57:14<142:32:21,  5.28s/it]
 57%|█████▋    | 127775/225000 [258:57:19<138:15:50,  5.12s/it]
 57%|█████▋    | 127776/225000 [258:57:24<138:54:57,  5.14s/it]
 57%|█████▋    | 127777/225000 [258:57:29<137:43:08,  5.10s/it]
 57%|█████▋    | 127778/225000 [258:57:34<138:44:04,  5.14s/it]
 57%|█████▋    | 127779/225000 [258:57:39<135:56:02,  5.03s/it]
 57%|█████▋    | 127780/225000 [258:57:44<135:47:37,  5.03s/it]
 57%|█████▋    | 127781/225000 [258:57:57<198:41:17,  7.36s/it]
 57%|█████▋    | 127782/225000 [258:58:03<191:14:36,  7.08s/it]
 57%|█████▋    | 127783/225000 [258:58:08<169:10:44,  6.26s/it]
 57%|█████▋    | 127784/225000 [258:58:12<151:59:20,  5.63s/it]
 57%|█████▋    | 127785/225000 [258:58:20<175:09:28,  6.49s/it]
 57%|█████▋    | 127786/225000 [258:58:29<193:45:27,  7.18s/it]
 57%|█████▋    | 127787/225000 [258:58:33<167:22:22,  6.20s/it]
 57%|█████▋    | 127788/225000 [258:58:41<181:08:09,  6.71s/it]
 57%|█████▋    | 127789/225000 [258:58:44<155:10:02,  5.75s/it]
 57%|█████▋    | 127790/225000 [258:58:49<142:02:11,  5.26s/it]
 57%|█████▋    | 127791/225000 [258:58:58<175:41:44,  6.51s/it]
 57%|█████▋    | 127792/225000 [258:59:04<172:01:06,  6.37s/it]
 57%|█████▋    | 127793/225000 [258:59:08<152:52:09,  5.66s/it]
 57%|█████▋    | 127794/225000 [258:59:14<154:00:47,  5.70s/it]
 57%|█████▋    | 127795/225000 [258:59:23<178:31:09,  6.61s/it]
 57%|█████▋    | 127796/225000 [258:59:29<179:19:12,  6.64s/it]
 57%|█████▋    | 127797/225000 [258:59:33<153:44:50,  5.69s/it]
 57%|█████▋    | 127798/225000 [258:59:36<135:31:09,  5.02s/it]
 57%|█████▋    | 127799/225000 [258:59:47<181:54:55,  6.74s/it]
 57%|█████▋    | 127800/225000 [258:59:53<177:18:52,  6.57s/it]
                                                               
{'loss': '0.8268', 'grad_norm': '0.9549', 'learning_rate': '2.127e-05', 'epoch': '0.568', 'num_input_tokens_seen': 430845040, 'train_runtime': '9.324e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127800/225000 [258:59:53<177:18:52,  6.57s/it]
 57%|█████▋    | 127801/225000 [258:59:57<157:54:52,  5.85s/it]
 57%|█████▋    | 127802/225000 [259:00:01<141:01:37,  5.22s/it]
 57%|█████▋    | 127803/225000 [259:00:05<131:58:48,  4.89s/it]
 57%|█████▋    | 127804/225000 [259:00:09<125:56:41,  4.66s/it]
 57%|█████▋    | 127805/225000 [259:00:20<173:02:07,  6.41s/it]
 57%|█████▋    | 127806/225000 [259:00:26<170:01:59,  6.30s/it]
 57%|█████▋    | 127807/225000 [259:00:29<148:11:42,  5.49s/it]
 57%|█████▋    | 127808/225000 [259:00:33<133:20:33,  4.94s/it]
 57%|█████▋    | 127809/225000 [259:00:44<185:43:05,  6.88s/it]
 57%|█████▋    | 127810/225000 [259:00:49<166:27:54,  6.17s/it]
 57%|█████▋    | 127811/225000 [259:00:53<152:52:48,  5.66s/it]
 57%|█████▋    | 127812/225000 [259:00:58<140:57:04,  5.22s/it]
 57%|█████▋    | 127813/225000 [259:01:02<134:33:04,  4.98s/it]
 57%|█████▋    | 127814/225000 [259:01:07<130:16:49,  4.83s/it]
 57%|█████▋    | 127815/225000 [259:01:12<135:09:37,  5.01s/it]
 57%|█████▋    | 127816/225000 [259:01:24<195:41:46,  7.25s/it]
 57%|█████▋    | 127817/225000 [259:01:30<178:45:07,  6.62s/it]
 57%|█████▋    | 127818/225000 [259:01:38<193:01:51,  7.15s/it]
 57%|█████▋    | 127819/225000 [259:01:41<160:52:59,  5.96s/it]
 57%|█████▋    | 127820/225000 [259:01:48<167:00:22,  6.19s/it]
 57%|█████▋    | 127821/225000 [259:01:52<149:59:10,  5.56s/it]
 57%|█████▋    | 127822/225000 [259:01:56<137:38:29,  5.10s/it]
 57%|█████▋    | 127823/225000 [259:02:00<128:38:34,  4.77s/it]
 57%|█████▋    | 127824/225000 [259:02:09<164:02:42,  6.08s/it]
 57%|█████▋    | 127825/225000 [259:02:15<161:45:37,  5.99s/it]
 57%|█████▋    | 127826/225000 [259:02:21<161:13:52,  5.97s/it]
 57%|█████▋    | 127827/225000 [259:02:28<174:03:44,  6.45s/it]
 57%|█████▋    | 127828/225000 [259:02:33<159:26:28,  5.91s/it]
 57%|█████▋    | 127829/225000 [259:02:37<140:34:27,  5.21s/it]
 57%|█████▋    | 127830/225000 [259:02:47<178:58:13,  6.63s/it]
 57%|█████▋    | 127831/225000 [259:02:50<152:27:05,  5.65s/it]
 57%|█████▋    | 127832/225000 [259:02:58<171:29:34,  6.35s/it]
 57%|█████▋    | 127833/225000 [259:03:01<147:09:55,  5.45s/it]
 57%|█████▋    | 127834/225000 [259:03:06<141:27:38,  5.24s/it]
 57%|█████▋    | 127835/225000 [259:03:11<141:16:01,  5.23s/it]
 57%|█████▋    | 127836/225000 [259:03:16<136:41:15,  5.06s/it]
 57%|█████▋    | 127837/225000 [259:03:21<137:58:59,  5.11s/it]
 57%|█████▋    | 127838/225000 [259:03:26<135:17:52,  5.01s/it]
 57%|█████▋    | 127839/225000 [259:03:31<135:14:07,  5.01s/it]
 57%|█████▋    | 127840/225000 [259:03:36<137:51:05,  5.11s/it]
 57%|█████▋    | 127841/225000 [259:03:41<137:22:40,  5.09s/it]
 57%|█████▋    | 127842/225000 [259:03:52<180:44:05,  6.70s/it]
 57%|█████▋    | 127843/225000 [259:04:01<203:40:55,  7.55s/it]
 57%|█████▋    | 127844/225000 [259:04:04<168:22:38,  6.24s/it]
 57%|█████▋    | 127845/225000 [259:04:10<159:03:14,  5.89s/it]
 57%|█████▋    | 127846/225000 [259:04:20<195:54:32,  7.26s/it]
 57%|█████▋    | 127847/225000 [259:04:23<162:36:47,  6.03s/it]
 57%|█████▋    | 127848/225000 [259:04:27<146:55:42,  5.44s/it]
 57%|█████▋    | 127849/225000 [259:04:33<146:04:04,  5.41s/it]
 57%|█████▋    | 127850/225000 [259:04:38<143:34:31,  5.32s/it]
                                                               
{'loss': '0.8186', 'grad_norm': '1.108', 'learning_rate': '2.125e-05', 'epoch': '0.5682', 'num_input_tokens_seen': 430973264, 'train_runtime': '9.327e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127850/225000 [259:04:38<143:34:31,  5.32s/it]
 57%|█████▋    | 127851/225000 [259:04:43<140:59:37,  5.22s/it]
 57%|█████▋    | 127852/225000 [259:04:48<140:12:48,  5.20s/it]
 57%|█████▋    | 127853/225000 [259:04:53<138:10:19,  5.12s/it]
 57%|█████▋    | 127854/225000 [259:04:58<136:14:55,  5.05s/it]
 57%|█████▋    | 127855/225000 [259:05:06<166:39:10,  6.18s/it]
 57%|█████▋    | 127856/225000 [259:05:17<204:52:31,  7.59s/it]
 57%|█████▋    | 127857/225000 [259:05:21<170:50:22,  6.33s/it]
 57%|█████▋    | 127858/225000 [259:05:28<181:59:27,  6.74s/it]
 57%|█████▋    | 127859/225000 [259:05:34<170:27:23,  6.32s/it]
 57%|█████▋    | 127860/225000 [259:05:37<148:03:28,  5.49s/it]
 57%|█████▋    | 127861/225000 [259:05:48<189:12:04,  7.01s/it]
 57%|█████▋    | 127862/225000 [259:05:51<161:38:57,  5.99s/it]
 57%|█████▋    | 127863/225000 [259:05:55<138:36:08,  5.14s/it]
 57%|█████▋    | 127864/225000 [259:05:58<127:34:20,  4.73s/it]
 57%|█████▋    | 127865/225000 [259:06:09<176:08:46,  6.53s/it]
 57%|█████▋    | 127866/225000 [259:06:13<150:44:35,  5.59s/it]
 57%|█████▋    | 127867/225000 [259:06:17<139:29:56,  5.17s/it]
 57%|█████▋    | 127868/225000 [259:06:21<132:35:29,  4.91s/it]
 57%|█████▋    | 127869/225000 [259:06:25<128:16:07,  4.75s/it]
 57%|█████▋    | 127870/225000 [259:06:30<123:36:15,  4.58s/it]
 57%|█████▋    | 127871/225000 [259:06:40<173:35:50,  6.43s/it]
 57%|█████▋    | 127872/225000 [259:06:50<203:07:42,  7.53s/it]
 57%|█████▋    | 127873/225000 [259:06:54<169:08:46,  6.27s/it]
 57%|█████▋    | 127874/225000 [259:07:02<181:46:24,  6.74s/it]
 57%|█████▋    | 127875/225000 [259:07:06<163:42:05,  6.07s/it]
 57%|█████▋    | 127876/225000 [259:07:10<145:36:17,  5.40s/it]
 57%|█████▋    | 127877/225000 [259:07:14<138:17:35,  5.13s/it]
 57%|█████▋    | 127878/225000 [259:07:19<133:26:19,  4.95s/it]
 57%|█████▋    | 127879/225000 [259:07:31<190:00:37,  7.04s/it]
 57%|█████▋    | 127880/225000 [259:07:37<185:18:11,  6.87s/it]
 57%|█████▋    | 127881/225000 [259:07:41<156:31:12,  5.80s/it]
 57%|█████▋    | 127882/225000 [259:07:48<172:23:22,  6.39s/it]
 57%|█████▋    | 127883/225000 [259:07:52<147:19:35,  5.46s/it]
 57%|█████▋    | 127884/225000 [259:08:01<179:14:02,  6.64s/it]
 57%|█████▋    | 127885/225000 [259:08:05<158:43:58,  5.88s/it]
 57%|█████▋    | 127886/225000 [259:08:09<141:17:57,  5.24s/it]
 57%|█████▋    | 127887/225000 [259:08:13<129:07:07,  4.79s/it]
 57%|█████▋    | 127888/225000 [259:08:17<127:13:14,  4.72s/it]
 57%|█████▋    | 127889/225000 [259:08:22<124:09:03,  4.60s/it]
 57%|█████▋    | 127890/225000 [259:08:29<149:24:57,  5.54s/it]
 57%|█████▋    | 127891/225000 [259:08:41<201:57:06,  7.49s/it]
 57%|█████▋    | 127892/225000 [259:08:45<171:58:28,  6.38s/it]
 57%|█████▋    | 127893/225000 [259:08:54<189:38:57,  7.03s/it]
 57%|█████▋    | 127894/225000 [259:08:57<160:50:16,  5.96s/it]
 57%|█████▋    | 127895/225000 [259:09:05<173:24:21,  6.43s/it]
 57%|█████▋    | 127896/225000 [259:09:08<151:38:07,  5.62s/it]
 57%|█████▋    | 127897/225000 [259:09:20<200:29:21,  7.43s/it]
 57%|█████▋    | 127898/225000 [259:09:24<170:07:13,  6.31s/it]
 57%|█████▋    | 127899/225000 [259:09:28<150:18:33,  5.57s/it]
 57%|█████▋    | 127900/225000 [259:09:31<135:51:55,  5.04s/it]
                                                               
{'loss': '0.8215', 'grad_norm': '1.014', 'learning_rate': '2.124e-05', 'epoch': '0.5684', 'num_input_tokens_seen': 431102312, 'train_runtime': '9.33e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127900/225000 [259:09:31<135:51:55,  5.04s/it]
 57%|█████▋    | 127901/225000 [259:09:36<133:26:15,  4.95s/it]
 57%|█████▋    | 127902/225000 [259:09:42<137:07:11,  5.08s/it]
 57%|█████▋    | 127903/225000 [259:09:46<134:57:53,  5.00s/it]
 57%|█████▋    | 127904/225000 [259:09:51<135:44:19,  5.03s/it]
 57%|█████▋    | 127905/225000 [259:09:56<134:56:28,  5.00s/it]
 57%|█████▋    | 127906/225000 [259:10:02<139:54:11,  5.19s/it]
 57%|█████▋    | 127907/225000 [259:10:07<140:00:55,  5.19s/it]
 57%|█████▋    | 127908/225000 [259:10:13<143:55:46,  5.34s/it]
 57%|█████▋    | 127909/225000 [259:10:18<142:40:26,  5.29s/it]
 57%|█████▋    | 127910/225000 [259:10:24<145:04:29,  5.38s/it]
 57%|█████▋    | 127911/225000 [259:10:36<200:06:04,  7.42s/it]
 57%|█████▋    | 127912/225000 [259:10:44<206:31:39,  7.66s/it]
 57%|█████▋    | 127913/225000 [259:10:49<181:48:00,  6.74s/it]
 57%|█████▋    | 127914/225000 [259:10:53<162:07:07,  6.01s/it]
 57%|█████▋    | 127915/225000 [259:10:58<151:42:11,  5.63s/it]
 57%|█████▋    | 127916/225000 [259:11:02<143:21:45,  5.32s/it]
 57%|█████▋    | 127917/225000 [259:11:07<137:50:52,  5.11s/it]
 57%|█████▋    | 127918/225000 [259:11:19<194:14:26,  7.20s/it]
 57%|█████▋    | 127919/225000 [259:11:24<175:39:39,  6.51s/it]
 57%|█████▋    | 127920/225000 [259:11:28<153:08:26,  5.68s/it]
 57%|█████▋    | 127921/225000 [259:11:37<179:53:42,  6.67s/it]
 57%|█████▋    | 127922/225000 [259:11:40<152:37:21,  5.66s/it]
 57%|█████▋    | 127923/225000 [259:11:48<168:30:16,  6.25s/it]
 57%|█████▋    | 127924/225000 [259:11:55<181:08:09,  6.72s/it]
 57%|█████▋    | 127925/225000 [259:11:59<153:52:10,  5.71s/it]
 57%|█████▋    | 127926/225000 [259:12:07<172:16:04,  6.39s/it]
 57%|█████▋    | 127927/225000 [259:12:11<155:42:37,  5.77s/it]
 57%|█████▋    | 127928/225000 [259:12:15<142:40:13,  5.29s/it]
 57%|█████▋    | 127929/225000 [259:12:19<130:36:29,  4.84s/it]
 57%|█████▋    | 127930/225000 [259:12:27<153:39:19,  5.70s/it]
 57%|█████▋    | 127931/225000 [259:12:35<174:28:59,  6.47s/it]
 57%|█████▋    | 127932/225000 [259:12:41<170:33:33,  6.33s/it]
 57%|█████▋    | 127933/225000 [259:12:46<162:41:42,  6.03s/it]
 57%|█████▋    | 127934/225000 [259:12:52<156:18:38,  5.80s/it]
 57%|█████▋    | 127935/225000 [259:12:59<170:58:18,  6.34s/it]
 57%|█████▋    | 127936/225000 [259:13:03<150:17:19,  5.57s/it]
 57%|█████▋    | 127937/225000 [259:13:08<146:20:06,  5.43s/it]
 57%|█████▋    | 127938/225000 [259:13:18<179:04:56,  6.64s/it]
 57%|█████▋    | 127939/225000 [259:13:22<163:36:49,  6.07s/it]
 57%|█████▋    | 127940/225000 [259:13:29<167:20:47,  6.21s/it]
 57%|█████▋    | 127941/225000 [259:13:33<152:18:04,  5.65s/it]
 57%|█████▋    | 127942/225000 [259:13:39<155:03:39,  5.75s/it]
 57%|█████▋    | 127943/225000 [259:13:44<146:17:21,  5.43s/it]
 57%|█████▋    | 127944/225000 [259:13:49<141:33:42,  5.25s/it]
 57%|█████▋    | 127945/225000 [259:13:54<139:03:54,  5.16s/it]
 57%|█████▋    | 127946/225000 [259:13:58<135:50:34,  5.04s/it]
 57%|█████▋    | 127947/225000 [259:14:04<143:01:18,  5.31s/it]
 57%|█████▋    | 127948/225000 [259:14:17<204:36:11,  7.59s/it]
 57%|█████▋    | 127949/225000 [259:14:23<192:57:06,  7.16s/it]
 57%|█████▋    | 127950/225000 [259:14:28<174:20:06,  6.47s/it]
                                                               
{'loss': '0.825', 'grad_norm': '1.072', 'learning_rate': '2.122e-05', 'epoch': '0.5687', 'num_input_tokens_seen': 431231960, 'train_runtime': '9.333e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 127950/225000 [259:14:28<174:20:06,  6.47s/it]
 57%|█████▋    | 127951/225000 [259:14:35<177:40:53,  6.59s/it]
 57%|█████▋    | 127952/225000 [259:14:40<165:09:36,  6.13s/it]
 57%|█████▋    | 127953/225000 [259:14:45<152:10:48,  5.65s/it]
 57%|█████▋    | 127954/225000 [259:14:49<143:29:44,  5.32s/it]
 57%|█████▋    | 127955/225000 [259:14:59<179:28:46,  6.66s/it]
 57%|█████▋    | 127956/225000 [259:15:05<177:11:04,  6.57s/it]
 57%|█████▋    | 127957/225000 [259:15:09<153:56:45,  5.71s/it]
 57%|█████▋    | 127958/225000 [259:15:17<175:21:21,  6.51s/it]
 57%|█████▋    | 127959/225000 [259:15:22<156:06:14,  5.79s/it]
 57%|█████▋    | 127960/225000 [259:15:30<174:09:48,  6.46s/it]
 57%|█████▋    | 127961/225000 [259:15:34<154:16:05,  5.72s/it]
 57%|█████▋    | 127962/225000 [259:15:37<138:00:53,  5.12s/it]
 57%|█████▋    | 127963/225000 [259:15:42<138:41:28,  5.15s/it]
 57%|█████▋    | 127964/225000 [259:15:47<131:33:20,  4.88s/it]
 57%|█████▋    | 127965/225000 [259:15:57<175:22:24,  6.51s/it]
 57%|█████▋    | 127966/225000 [259:16:02<166:05:31,  6.16s/it]
 57%|█████▋    | 127967/225000 [259:16:06<145:05:44,  5.38s/it]
 57%|█████▋    | 127968/225000 [259:16:15<176:07:12,  6.53s/it]
 57%|█████▋    | 127969/225000 [259:16:20<161:26:36,  5.99s/it]
 57%|█████▋    | 127970/225000 [259:16:27<171:08:16,  6.35s/it]
 57%|█████▋    | 127971/225000 [259:16:32<159:44:24,  5.93s/it]
 57%|█████▋    | 127972/225000 [259:16:37<153:48:38,  5.71s/it]
 57%|█████▋    | 127973/225000 [259:16:43<151:13:24,  5.61s/it]
 57%|█████▋    | 127974/225000 [259:16:48<148:06:21,  5.50s/it]
 57%|█████▋    | 127975/225000 [259:16:53<146:27:12,  5.43s/it]
 57%|█████▋    | 127976/225000 [259:16:58<140:28:25,  5.21s/it]
 57%|█████▋    | 127977/225000 [259:17:03<140:28:56,  5.21s/it]
 57%|█████▋    | 127978/225000 [259:17:08<140:48:55,  5.22s/it]
 57%|█████▋    | 127979/225000 [259:17:13<138:43:31,  5.15s/it]
 57%|█████▋    | 127980/225000 [259:17:19<139:52:11,  5.19s/it]
 57%|█████▋    | 127981/225000 [259:17:24<138:37:51,  5.14s/it]
 57%|█████▋    | 127982/225000 [259:17:30<153:07:34,  5.68s/it]
 57%|█████▋    | 127983/225000 [259:17:43<208:42:17,  7.74s/it]
 57%|█████▋    | 127984/225000 [259:17:49<192:50:26,  7.16s/it]
 57%|█████▋    | 127985/225000 [259:17:53<165:39:15,  6.15s/it]
 57%|█████▋    | 127986/225000 [259:18:03<199:11:18,  7.39s/it]
 57%|█████▋    | 127987/225000 [259:18:07<169:23:24,  6.29s/it]
 57%|█████▋    | 127988/225000 [259:18:10<143:34:33,  5.33s/it]
 57%|█████▋    | 127989/225000 [259:18:21<189:46:39,  7.04s/it]
 57%|█████▋    | 127990/225000 [259:18:25<164:34:45,  6.11s/it]
 57%|█████▋    | 127991/225000 [259:18:30<154:26:24,  5.73s/it]
 57%|█████▋    | 127992/225000 [259:18:38<173:50:01,  6.45s/it]
 57%|█████▋    | 127993/225000 [259:18:41<147:44:22,  5.48s/it]
 57%|█████▋    | 127994/225000 [259:18:50<174:34:20,  6.48s/it]
 57%|█████▋    | 127995/225000 [259:18:53<148:20:50,  5.51s/it]
 57%|█████▋    | 127996/225000 [259:19:01<168:15:05,  6.24s/it]
 57%|█████▋    | 127997/225000 [259:19:07<165:22:08,  6.14s/it]
 57%|█████▋    | 127998/225000 [259:19:13<162:29:14,  6.03s/it]
 57%|█████▋    | 127999/225000 [259:19:17<146:51:40,  5.45s/it]
 57%|█████▋    | 128000/225000 [259:19:21<135:51:11,  5.04s/it]
                                                               
{'loss': '0.8116', 'grad_norm': '1.007', 'learning_rate': '2.12e-05', 'epoch': '0.5689', 'num_input_tokens_seen': 431362248, 'train_runtime': '9.336e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 128000/225000 [259:19:21<135:51:11,  5.04s/it]
 57%|█████▋    | 128001/225000 [259:19:32<185:27:04,  6.88s/it]
 57%|█████▋    | 128002/225000 [259:19:35<157:40:14,  5.85s/it]
 57%|█████▋    | 128003/225000 [259:19:39<140:17:17,  5.21s/it]
 57%|█████▋    | 128004/225000 [259:19:50<187:52:12,  6.97s/it]
 57%|█████▋    | 128005/225000 [259:19:54<159:39:23,  5.93s/it]
 57%|█████▋    | 128006/225000 [259:19:59<155:41:09,  5.78s/it]
 57%|█████▋    | 128007/225000 [259:20:06<165:57:02,  6.16s/it]
 57%|█████▋    | 128008/225000 [259:20:10<149:59:35,  5.57s/it]
 57%|█████▋    | 128009/225000 [259:20:18<163:44:42,  6.08s/it]
 57%|█████▋    | 128010/225000 [259:20:21<140:04:52,  5.20s/it]
 57%|█████▋    | 128011/225000 [259:20:25<130:44:43,  4.85s/it]
 57%|█████▋    | 128012/225000 [259:20:29<124:39:56,  4.63s/it]
 57%|█████▋    | 128013/225000 [259:20:41<188:27:03,  6.99s/it]
 57%|█████▋    | 128014/225000 [259:20:45<163:37:50,  6.07s/it]
 57%|█████▋    | 128015/225000 [259:20:49<147:41:05,  5.48s/it]
 57%|█████▋    | 128016/225000 [259:20:57<165:57:16,  6.16s/it]
 57%|█████▋    | 128017/225000 [259:21:05<176:02:06,  6.53s/it]
 57%|█████▋    | 128018/225000 [259:21:11<171:40:43,  6.37s/it]
 57%|█████▋    | 128019/225000 [259:21:17<169:10:51,  6.28s/it]
 57%|█████▋    | 128020/225000 [259:21:20<148:24:57,  5.51s/it]
 57%|█████▋    | 128021/225000 [259:21:27<159:02:01,  5.90s/it]
 57%|█████▋    | 128022/225000 [259:21:34<163:12:23,  6.06s/it]
 57%|█████▋    | 128023/225000 [259:21:37<140:30:52,  5.22s/it]
 57%|█████▋    | 128024/225000 [259:21:48<187:02:07,  6.94s/it]
 57%|█████▋    | 128025/225000 [259:21:51<159:25:24,  5.92s/it]
 57%|█████▋    | 128026/225000 [259:21:59<171:11:40,  6.36s/it]
 57%|█████▋    | 128027/225000 [259:22:03<151:02:31,  5.61s/it]
 57%|█████▋    | 128028/225000 [259:22:08<150:40:11,  5.59s/it]
 57%|█████▋    | 128029/225000 [259:22:16<171:34:19,  6.37s/it]
 57%|█████▋    | 128030/225000 [259:22:20<146:43:09,  5.45s/it]
 57%|█████▋    | 128031/225000 [259:22:28<171:19:59,  6.36s/it]
 57%|█████▋    | 128032/225000 [259:22:31<146:44:28,  5.45s/it]
 57%|█████▋    | 128033/225000 [259:22:37<148:52:30,  5.53s/it]
 57%|█████▋    | 128034/225000 [259:22:45<168:17:50,  6.25s/it]
 57%|█████▋    | 128035/225000 [259:22:50<157:40:13,  5.85s/it]
 57%|█████▋    | 128036/225000 [259:22:54<146:22:36,  5.43s/it]
 57%|█████▋    | 128037/225000 [259:22:59<137:19:31,  5.10s/it]
 57%|█████▋    | 128038/225000 [259:23:03<132:57:11,  4.94s/it]
 57%|█████▋    | 128039/225000 [259:23:08<130:17:41,  4.84s/it]
 57%|█████▋    | 128040/225000 [259:23:14<141:59:12,  5.27s/it]
 57%|█████▋    | 128041/225000 [259:23:27<204:14:34,  7.58s/it]
 57%|█████▋    | 128042/225000 [259:23:31<170:38:21,  6.34s/it]
 57%|█████▋    | 128043/225000 [259:23:38<175:02:12,  6.50s/it]
 57%|█████▋    | 128044/225000 [259:23:44<174:31:19,  6.48s/it]
 57%|█████▋    | 128045/225000 [259:23:48<151:10:56,  5.61s/it]
 57%|█████▋    | 128046/225000 [259:23:52<139:09:22,  5.17s/it]
 57%|█████▋    | 128047/225000 [259:23:56<136:23:05,  5.06s/it]
 57%|█████▋    | 128048/225000 [259:24:02<140:51:55,  5.23s/it]
 57%|█████▋    | 128049/225000 [259:24:08<144:12:21,  5.35s/it]
 57%|█████▋    | 128050/225000 [259:24:13<144:37:12,  5.37s/it]
                                                               
{'loss': '0.8191', 'grad_norm': '0.9906', 'learning_rate': '2.118e-05', 'epoch': '0.5691', 'num_input_tokens_seen': 431492528, 'train_runtime': '9.339e+05', 'train_tokens_per_second': '462.1'}

 57%|█████▋    | 128050/225000 [259:24:13<144:37:12,  5.37s/it]
 57%|█████▋    | 128051/225000 [259:24:19<144:56:45,  5.38s/it]
 57%|█████▋    | 128052/225000 [259:24:24<148:41:44,  5.52s/it]
 57%|█████▋    | 128053/225000 [259:24:29<143:57:19,  5.35s/it]
 57%|█████▋    | 128054/225000 [259:24:35<145:34:26,  5.41s/it]
 57%|█████▋    | 128055/225000 [259:24:48<205:54:53,  7.65s/it]
 57%|█████▋    | 128056/225000 [259:24:55<199:22:11,  7.40s/it]
 57%|█████▋    | 128057/225000 [259:24:59<170:57:47,  6.35s/it]
 57%|█████▋    | 128058/225000 [259:25:06<178:19:56,  6.62s/it]
 57%|█████▋    | 128059/225000 [259:25:13<182:05:35,  6.76s/it]
 57%|█████▋    | 128060/225000 [259:25:17<161:23:07,  5.99s/it]
 57%|█████▋    | 128061/225000 [259:25:21<144:54:26,  5.38s/it]
 57%|█████▋    | 128062/225000 [259:25:25<134:49:32,  5.01s/it]
 57%|█████▋    | 128063/225000 [259:25:37<193:27:02,  7.18s/it]
 57%|█████▋    | 128064/225000 [259:25:41<163:57:09,  6.09s/it]
 57%|█████▋    | 128065/225000 [259:25:46<153:15:10,  5.69s/it]
 57%|█████▋    | 128066/225000 [259:25:55<181:23:28,  6.74s/it]
 57%|█████▋    | 128067/225000 [259:26:02<183:57:48,  6.83s/it]
 57%|█████▋    | 128068/225000 [259:26:06<162:40:33,  6.04s/it]
 57%|█████▋    | 128069/225000 [259:26:10<144:00:13,  5.35s/it]
 57%|█████▋    | 128070/225000 [259:26:20<179:26:23,  6.66s/it]
 57%|█████▋    | 128071/225000 [259:26:26<175:55:52,  6.53s/it]
 57%|█████▋    | 128072/225000 [259:26:29<150:35:05,  5.59s/it]
 57%|█████▋    | 128073/225000 [259:26:36<162:24:04,  6.03s/it]
 57%|█████▋    | 128074/225000 [259:26:40<143:13:56,  5.32s/it]
 57%|█████▋    | 128075/225000 [259:26:49<171:19:30,  6.36s/it]
 57%|█████▋    | 128076/225000 [259:26:53<150:59:42,  5.61s/it]
 57%|█████▋    | 128077/225000 [259:26:57<139:30:16,  5.18s/it]
 57%|█████▋    | 128078/225000 [259:27:01<131:28:58,  4.88s/it]
 57%|█████▋    | 128079/225000 [259:27:12<184:24:51,  6.85s/it]
 57%|█████▋    | 128080/225000 [259:27:16<157:11:49,  5.84s/it]
 57%|█████▋    | 128081/225000 [259:27:20<141:09:02,  5.24s/it]
 57%|█████▋    | 128082/225000 [259:27:30<181:35:09,  6.74s/it]
 57%|█████▋    | 128083/225000 [259:27:35<166:00:03,  6.17s/it]
 57%|█████▋    | 128084/225000 [259:27:40<156:28:25,  5.81s/it]
 57%|█████▋    | 128085/225000 [259:27:45<150:03:20,  5.57s/it]
 57%|█████▋    | 128086/225000 [259:27:50<146:50:45,  5.45s/it]
 57%|█████▋    | 128087/225000 [259:27:55<142:51:15,  5.31s/it]
 57%|█████▋    | 128088/225000 [259:27:59<135:48:00,  5.04s/it]
 57%|█████▋    | 128089/225000 [259:28:04<133:15:42,  4.95s/it]
 57%|█████▋    | 128090/225000 [259:28:12<153:22:35,  5.70s/it]
 57%|█████▋    | 128091/225000 [259:28:24<210:14:31,  7.81s/it]
 57%|█████▋    | 128092/225000 [259:28:28<175:12:06,  6.51s/it]
 57%|█████▋    | 128093/225000 [259:28:35<185:12:14,  6.88s/it]
 57%|█████▋    | 128094/225000 [259:28:39<161:09:42,  5.99s/it]
 57%|█████▋    | 128095/225000 [259:28:43<143:21:35,  5.33s/it]
 57%|█████▋    | 128096/225000 [259:28:54<188:58:26,  7.02s/it]
 57%|█████▋    | 128097/225000 [259:28:58<159:29:39,  5.93s/it]
 57%|█████▋    | 128098/225000 [259:29:05<170:25:47,  6.33s/it]
 57%|█████▋    | 128099/225000 [259:29:08<146:29:02,  5.44s/it]
 57%|█████▋    | 128100/225000 [259:29:16<168:28:58,  6.26s/it]
                                                               
{'loss': '0.8238', 'grad_norm': '1.042', 'learning_rate': '2.116e-05', 'epoch': '0.5693', 'num_input_tokens_seen': 431622104, 'train_runtime': '9.342e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128100/225000 [259:29:16<168:28:58,  6.26s/it]
 57%|█████▋    | 128101/225000 [259:29:19<141:51:27,  5.27s/it]
 57%|█████▋    | 128102/225000 [259:29:24<136:15:48,  5.06s/it]
 57%|█████▋    | 128103/225000 [259:29:29<135:57:49,  5.05s/it]
 57%|█████▋    | 128104/225000 [259:29:34<134:06:08,  4.98s/it]
 57%|█████▋    | 128105/225000 [259:29:38<130:17:32,  4.84s/it]
 57%|█████▋    | 128106/225000 [259:29:44<134:06:40,  4.98s/it]
 57%|█████▋    | 128107/225000 [259:29:48<132:26:05,  4.92s/it]
 57%|█████▋    | 128108/225000 [259:29:55<142:44:59,  5.30s/it]
 57%|█████▋    | 128109/225000 [259:30:07<201:56:07,  7.50s/it]
 57%|█████▋    | 128110/225000 [259:30:13<188:32:32,  7.01s/it]
 57%|█████▋    | 128111/225000 [259:30:19<183:19:43,  6.81s/it]
 57%|█████▋    | 128112/225000 [259:30:25<171:14:59,  6.36s/it]
 57%|█████▋    | 128113/225000 [259:30:29<154:31:11,  5.74s/it]
 57%|█████▋    | 128114/225000 [259:30:33<141:50:24,  5.27s/it]
 57%|█████▋    | 128115/225000 [259:30:38<138:56:42,  5.16s/it]
 57%|█████▋    | 128116/225000 [259:30:46<163:44:36,  6.08s/it]
 57%|█████▋    | 128117/225000 [259:30:56<190:14:30,  7.07s/it]
 57%|█████▋    | 128118/225000 [259:30:59<160:15:00,  5.95s/it]
 57%|█████▋    | 128119/225000 [259:31:07<176:03:48,  6.54s/it]
 57%|█████▋    | 128120/225000 [259:31:11<155:28:21,  5.78s/it]
 57%|█████▋    | 128121/225000 [259:31:15<144:05:51,  5.35s/it]
 57%|█████▋    | 128122/225000 [259:31:23<161:12:33,  5.99s/it]
 57%|█████▋    | 128123/225000 [259:31:30<171:20:58,  6.37s/it]
 57%|█████▋    | 128124/225000 [259:31:34<150:16:41,  5.58s/it]
 57%|█████▋    | 128125/225000 [259:31:38<137:08:00,  5.10s/it]
 57%|█████▋    | 128126/225000 [259:31:47<170:19:49,  6.33s/it]
 57%|█████▋    | 128127/225000 [259:31:51<150:06:06,  5.58s/it]
 57%|█████▋    | 128128/225000 [259:31:55<139:08:06,  5.17s/it]
 57%|█████▋    | 128129/225000 [259:31:59<130:13:39,  4.84s/it]
 57%|█████▋    | 128130/225000 [259:32:10<183:33:12,  6.82s/it]
 57%|█████▋    | 128131/225000 [259:32:15<161:13:33,  5.99s/it]
 57%|█████▋    | 128132/225000 [259:32:21<166:24:02,  6.18s/it]
 57%|█████▋    | 128133/225000 [259:32:27<160:27:07,  5.96s/it]
 57%|█████▋    | 128134/225000 [259:32:30<140:52:34,  5.24s/it]
 57%|█████▋    | 128135/225000 [259:32:34<131:22:53,  4.88s/it]
 57%|█████▋    | 128136/225000 [259:32:41<146:54:25,  5.46s/it]
 57%|█████▋    | 128137/225000 [259:32:46<146:37:24,  5.45s/it]
 57%|█████▋    | 128138/225000 [259:32:56<181:52:18,  6.76s/it]
 57%|█████▋    | 128139/225000 [259:33:00<157:35:06,  5.86s/it]
 57%|█████▋    | 128140/225000 [259:33:04<141:51:41,  5.27s/it]
 57%|█████▋    | 128141/225000 [259:33:08<129:24:06,  4.81s/it]
 57%|█████▋    | 128142/225000 [259:33:20<187:59:00,  6.99s/it]
 57%|█████▋    | 128143/225000 [259:33:24<163:41:11,  6.08s/it]
 57%|█████▋    | 128144/225000 [259:33:30<165:06:05,  6.14s/it]
 57%|█████▋    | 128145/225000 [259:33:35<152:57:27,  5.69s/it]
 57%|█████▋    | 128146/225000 [259:33:39<142:24:24,  5.29s/it]
 57%|█████▋    | 128147/225000 [259:33:44<136:49:05,  5.09s/it]
 57%|█████▋    | 128148/225000 [259:33:48<135:07:41,  5.02s/it]
 57%|█████▋    | 128149/225000 [259:33:53<132:14:16,  4.92s/it]
 57%|█████▋    | 128150/225000 [259:34:05<191:07:11,  7.10s/it]
                                                               
{'loss': '0.8268', 'grad_norm': '1.196', 'learning_rate': '2.115e-05', 'epoch': '0.5696', 'num_input_tokens_seen': 431751360, 'train_runtime': '9.344e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128150/225000 [259:34:05<191:07:11,  7.10s/it]
 57%|█████▋    | 128151/225000 [259:34:11<177:28:17,  6.60s/it]
 57%|█████▋    | 128152/225000 [259:34:15<157:04:17,  5.84s/it]
 57%|█████▋    | 128153/225000 [259:34:19<142:44:00,  5.31s/it]
 57%|█████▋    | 128154/225000 [259:34:27<164:56:49,  6.13s/it]
 57%|█████▋    | 128155/225000 [259:34:35<181:38:58,  6.75s/it]
 57%|█████▋    | 128156/225000 [259:34:41<171:48:14,  6.39s/it]
 57%|█████▋    | 128157/225000 [259:34:46<161:15:31,  5.99s/it]
 57%|█████▋    | 128158/225000 [259:34:49<139:51:26,  5.20s/it]
 57%|█████▋    | 128159/225000 [259:34:53<128:46:12,  4.79s/it]
 57%|█████▋    | 128160/225000 [259:34:58<128:54:23,  4.79s/it]
 57%|█████▋    | 128161/225000 [259:35:03<132:06:24,  4.91s/it]
 57%|█████▋    | 128162/225000 [259:35:08<136:06:04,  5.06s/it]
 57%|█████▋    | 128163/225000 [259:35:13<134:44:51,  5.01s/it]
 57%|█████▋    | 128164/225000 [259:35:18<137:03:45,  5.10s/it]
 57%|█████▋    | 128165/225000 [259:35:31<195:14:40,  7.26s/it]
 57%|█████▋    | 128166/225000 [259:35:37<183:37:13,  6.83s/it]
 57%|█████▋    | 128167/225000 [259:35:41<166:05:30,  6.17s/it]
 57%|█████▋    | 128168/225000 [259:35:46<155:27:25,  5.78s/it]
 57%|█████▋    | 128169/225000 [259:35:51<149:12:26,  5.55s/it]
 57%|█████▋    | 128170/225000 [259:35:56<141:51:26,  5.27s/it]
 57%|█████▋    | 128171/225000 [259:36:01<141:56:09,  5.28s/it]
 57%|█████▋    | 128172/225000 [259:36:14<201:40:34,  7.50s/it]
 57%|█████▋    | 128173/225000 [259:36:19<182:16:00,  6.78s/it]
 57%|█████▋    | 128174/225000 [259:36:22<156:36:47,  5.82s/it]
 57%|█████▋    | 128175/225000 [259:36:28<153:19:46,  5.70s/it]
 57%|█████▋    | 128176/225000 [259:36:33<149:42:36,  5.57s/it]
 57%|█████▋    | 128177/225000 [259:36:44<192:21:46,  7.15s/it]
 57%|█████▋    | 128178/225000 [259:36:48<166:31:10,  6.19s/it]
 57%|█████▋    | 128179/225000 [259:36:57<188:32:45,  7.01s/it]
 57%|█████▋    | 128180/225000 [259:37:02<175:19:31,  6.52s/it]
 57%|█████▋    | 128181/225000 [259:37:07<158:03:29,  5.88s/it]
 57%|█████▋    | 128182/225000 [259:37:11<149:29:33,  5.56s/it]
 57%|█████▋    | 128183/225000 [259:37:16<140:08:18,  5.21s/it]
 57%|█████▋    | 128184/225000 [259:37:20<135:00:46,  5.02s/it]
 57%|█████▋    | 128185/225000 [259:37:33<196:35:47,  7.31s/it]
 57%|█████▋    | 128186/225000 [259:37:37<166:21:14,  6.19s/it]
 57%|█████▋    | 128187/225000 [259:37:44<172:22:08,  6.41s/it]
 57%|█████▋    | 128188/225000 [259:37:50<169:56:49,  6.32s/it]
 57%|█████▋    | 128189/225000 [259:37:57<182:07:25,  6.77s/it]
 57%|█████▋    | 128190/225000 [259:38:02<166:37:56,  6.20s/it]
 57%|█████▋    | 128191/225000 [259:38:06<149:22:28,  5.55s/it]
 57%|█████▋    | 128192/225000 [259:38:11<141:28:34,  5.26s/it]
 57%|█████▋    | 128193/225000 [259:38:15<135:51:38,  5.05s/it]
 57%|█████▋    | 128194/225000 [259:38:28<193:37:39,  7.20s/it]
 57%|█████▋    | 128195/225000 [259:38:31<161:38:13,  6.01s/it]
 57%|█████▋    | 128196/225000 [259:38:36<152:33:15,  5.67s/it]
 57%|█████▋    | 128197/225000 [259:38:41<148:20:40,  5.52s/it]
 57%|█████▋    | 128198/225000 [259:38:50<175:03:42,  6.51s/it]
 57%|█████▋    | 128199/225000 [259:38:55<164:02:53,  6.10s/it]
 57%|█████▋    | 128200/225000 [259:38:59<149:24:21,  5.56s/it]
                                                               
{'loss': '0.8282', 'grad_norm': '1.066', 'learning_rate': '2.113e-05', 'epoch': '0.5698', 'num_input_tokens_seen': 431880560, 'train_runtime': '9.347e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128200/225000 [259:38:59<149:24:21,  5.56s/it]
 57%|█████▋    | 128201/225000 [259:39:03<138:11:01,  5.14s/it]
 57%|█████▋    | 128202/225000 [259:39:12<166:30:30,  6.19s/it]
 57%|█████▋    | 128203/225000 [259:39:22<199:52:37,  7.43s/it]
 57%|█████▋    | 128204/225000 [259:39:26<165:59:45,  6.17s/it]
 57%|█████▋    | 128205/225000 [259:39:34<181:41:47,  6.76s/it]
 57%|█████▋    | 128206/225000 [259:39:39<166:17:52,  6.19s/it]
 57%|█████▋    | 128207/225000 [259:39:44<162:26:06,  6.04s/it]
 57%|█████▋    | 128208/225000 [259:39:50<157:32:38,  5.86s/it]
 57%|█████▋    | 128209/225000 [259:39:54<141:52:03,  5.28s/it]
 57%|█████▋    | 128210/225000 [259:39:59<145:06:04,  5.40s/it]
 57%|█████▋    | 128211/225000 [259:40:07<160:38:45,  5.98s/it]
 57%|█████▋    | 128212/225000 [259:40:13<165:45:57,  6.17s/it]
 57%|█████▋    | 128213/225000 [259:40:17<149:50:12,  5.57s/it]
 57%|█████▋    | 128214/225000 [259:40:22<141:23:04,  5.26s/it]
 57%|█████▋    | 128215/225000 [259:40:26<132:47:06,  4.94s/it]
 57%|█████▋    | 128216/225000 [259:40:31<127:58:37,  4.76s/it]
 57%|█████▋    | 128217/225000 [259:40:41<174:50:17,  6.50s/it]
 57%|█████▋    | 128218/225000 [259:40:48<174:17:08,  6.48s/it]
 57%|█████▋    | 128219/225000 [259:40:52<161:44:51,  6.02s/it]
 57%|█████▋    | 128220/225000 [259:40:58<155:31:12,  5.79s/it]
 57%|█████▋    | 128221/225000 [259:41:03<154:49:39,  5.76s/it]
 57%|█████▋    | 128222/225000 [259:41:09<151:51:52,  5.65s/it]
 57%|█████▋    | 128223/225000 [259:41:15<156:20:43,  5.82s/it]
 57%|█████▋    | 128224/225000 [259:41:22<162:19:15,  6.04s/it]
 57%|█████▋    | 128225/225000 [259:41:26<149:26:01,  5.56s/it]
 57%|█████▋    | 128226/225000 [259:41:31<148:27:57,  5.52s/it]
 57%|█████▋    | 128227/225000 [259:41:41<184:03:55,  6.85s/it]
 57%|█████▋    | 128228/225000 [259:41:45<158:28:01,  5.90s/it]
 57%|█████▋    | 128229/225000 [259:41:49<142:30:29,  5.30s/it]
 57%|█████▋    | 128230/225000 [259:41:55<144:30:29,  5.38s/it]
 57%|█████▋    | 128231/225000 [259:42:00<143:32:01,  5.34s/it]
 57%|█████▋    | 128232/225000 [259:42:05<143:28:27,  5.34s/it]
 57%|█████▋    | 128233/225000 [259:42:10<143:51:43,  5.35s/it]
 57%|█████▋    | 128234/225000 [259:42:16<143:03:03,  5.32s/it]
 57%|█████▋    | 128235/225000 [259:42:21<142:57:07,  5.32s/it]
 57%|█████▋    | 128236/225000 [259:42:26<139:32:05,  5.19s/it]
 57%|█████▋    | 128237/225000 [259:42:32<143:19:02,  5.33s/it]
 57%|█████▋    | 128238/225000 [259:42:37<141:56:51,  5.28s/it]
 57%|█████▋    | 128239/225000 [259:42:45<164:47:32,  6.13s/it]
 57%|█████▋    | 128240/225000 [259:42:57<216:20:24,  8.05s/it]
 57%|█████▋    | 128241/225000 [259:43:01<177:33:21,  6.61s/it]
 57%|█████▋    | 128242/225000 [259:43:05<157:10:10,  5.85s/it]
 57%|█████▋    | 128243/225000 [259:43:09<147:11:32,  5.48s/it]
 57%|█████▋    | 128244/225000 [259:43:14<139:43:36,  5.20s/it]
 57%|█████▋    | 128245/225000 [259:43:23<172:51:44,  6.43s/it]
 57%|█████▋    | 128246/225000 [259:43:30<179:42:38,  6.69s/it]
 57%|█████▋    | 128247/225000 [259:43:35<159:47:13,  5.95s/it]
 57%|█████▋    | 128248/225000 [259:43:39<146:58:03,  5.47s/it]
 57%|█████▋    | 128249/225000 [259:43:45<152:03:27,  5.66s/it]
 57%|█████▋    | 128250/225000 [259:43:55<189:52:15,  7.06s/it]
                                                               
{'loss': '0.8271', 'grad_norm': '1.073', 'learning_rate': '2.111e-05', 'epoch': '0.57', 'num_input_tokens_seen': 432009800, 'train_runtime': '9.35e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128250/225000 [259:43:55<189:52:15,  7.06s/it]
 57%|█████▋    | 128251/225000 [259:44:02<184:30:34,  6.87s/it]
 57%|█████▋    | 128252/225000 [259:44:05<157:26:16,  5.86s/it]
 57%|█████▋    | 128253/225000 [259:44:09<141:06:45,  5.25s/it]
 57%|█████▋    | 128254/225000 [259:44:18<167:11:55,  6.22s/it]
 57%|█████▋    | 128255/225000 [259:44:24<166:48:43,  6.21s/it]
 57%|█████▋    | 128256/225000 [259:44:31<172:12:17,  6.41s/it]
 57%|█████▋    | 128257/225000 [259:44:35<152:28:37,  5.67s/it]
 57%|█████▋    | 128258/225000 [259:44:39<140:16:15,  5.22s/it]
 57%|█████▋    | 128259/225000 [259:44:50<183:54:31,  6.84s/it]
 57%|█████▋    | 128260/225000 [259:44:54<162:04:22,  6.03s/it]
 57%|█████▋    | 128261/225000 [259:44:58<146:48:02,  5.46s/it]
 57%|█████▋    | 128262/225000 [259:45:02<134:33:33,  5.01s/it]
 57%|█████▋    | 128263/225000 [259:45:08<141:00:21,  5.25s/it]
 57%|█████▋    | 128264/225000 [259:45:17<178:54:06,  6.66s/it]
 57%|█████▋    | 128265/225000 [259:45:21<152:19:40,  5.67s/it]
 57%|█████▋    | 128266/225000 [259:45:30<177:47:38,  6.62s/it]
 57%|█████▋    | 128267/225000 [259:45:33<153:47:45,  5.72s/it]
 57%|█████▋    | 128268/225000 [259:45:42<174:28:23,  6.49s/it]
 57%|█████▋    | 128269/225000 [259:45:45<149:14:09,  5.55s/it]
 57%|█████▋    | 128270/225000 [259:45:53<173:10:39,  6.45s/it]
 57%|█████▋    | 128271/225000 [259:45:58<157:34:54,  5.86s/it]
 57%|█████▋    | 128272/225000 [259:46:06<178:29:30,  6.64s/it]
 57%|█████▋    | 128273/225000 [259:46:11<165:04:39,  6.14s/it]
 57%|█████▋    | 128274/225000 [259:46:16<152:11:39,  5.66s/it]
 57%|█████▋    | 128275/225000 [259:46:21<148:19:22,  5.52s/it]
 57%|█████▋    | 128276/225000 [259:46:26<144:45:33,  5.39s/it]
 57%|█████▋    | 128277/225000 [259:46:32<143:40:51,  5.35s/it]
 57%|█████▋    | 128278/225000 [259:46:37<141:54:06,  5.28s/it]
 57%|█████▋    | 128279/225000 [259:46:42<143:12:50,  5.33s/it]
 57%|█████▋    | 128280/225000 [259:46:48<144:42:23,  5.39s/it]
 57%|█████▋    | 128281/225000 [259:46:52<136:58:03,  5.10s/it]
 57%|█████▋    | 128282/225000 [259:46:58<140:13:40,  5.22s/it]
 57%|█████▋    | 128283/225000 [259:47:03<143:37:49,  5.35s/it]
 57%|█████▋    | 128284/225000 [259:47:15<197:28:44,  7.35s/it]
 57%|█████▋    | 128285/225000 [259:47:19<167:07:38,  6.22s/it]
 57%|█████▋    | 128286/225000 [259:47:26<175:42:22,  6.54s/it]
 57%|█████▋    | 128287/225000 [259:47:30<153:24:42,  5.71s/it]
 57%|█████▋    | 128288/225000 [259:47:36<159:06:07,  5.92s/it]
 57%|█████▋    | 128289/225000 [259:47:43<168:10:15,  6.26s/it]
 57%|█████▋    | 128290/225000 [259:47:47<150:35:38,  5.61s/it]
 57%|█████▋    | 128291/225000 [259:47:51<137:18:59,  5.11s/it]
 57%|█████▋    | 128292/225000 [259:47:56<132:21:44,  4.93s/it]
 57%|█████▋    | 128293/225000 [259:48:00<129:10:15,  4.81s/it]
 57%|█████▋    | 128294/225000 [259:48:05<127:59:35,  4.76s/it]
 57%|█████▋    | 128295/225000 [259:48:17<184:48:26,  6.88s/it]
 57%|█████▋    | 128296/225000 [259:48:25<196:00:59,  7.30s/it]
 57%|█████▋    | 128297/225000 [259:48:29<165:01:36,  6.14s/it]
 57%|█████▋    | 128298/225000 [259:48:37<182:30:45,  6.79s/it]
 57%|█████▋    | 128299/225000 [259:48:42<171:31:34,  6.39s/it]
 57%|█████▋    | 128300/225000 [259:48:48<163:49:04,  6.10s/it]
                                                               
{'loss': '0.8115', 'grad_norm': '1.014', 'learning_rate': '2.109e-05', 'epoch': '0.5702', 'num_input_tokens_seen': 432137640, 'train_runtime': '9.353e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128300/225000 [259:48:48<163:49:04,  6.10s/it]
 57%|█████▋    | 128301/225000 [259:48:53<155:53:09,  5.80s/it]
 57%|█████▋    | 128302/225000 [259:48:58<153:38:41,  5.72s/it]
 57%|█████▋    | 128303/225000 [259:49:08<186:10:21,  6.93s/it]
 57%|█████▋    | 128304/225000 [259:49:12<162:16:05,  6.04s/it]
 57%|█████▋    | 128305/225000 [259:49:16<143:54:45,  5.36s/it]
 57%|█████▋    | 128306/225000 [259:49:28<199:16:19,  7.42s/it]
 57%|█████▋    | 128307/225000 [259:49:32<169:21:55,  6.31s/it]
 57%|█████▋    | 128308/225000 [259:49:36<150:40:18,  5.61s/it]
 57%|█████▋    | 128309/225000 [259:49:42<154:09:53,  5.74s/it]
 57%|█████▋    | 128310/225000 [259:49:50<174:16:21,  6.49s/it]
 57%|█████▋    | 128311/225000 [259:49:56<171:55:54,  6.40s/it]
 57%|█████▋    | 128312/225000 [259:50:01<161:46:46,  6.02s/it]
 57%|█████▋    | 128313/225000 [259:50:08<164:51:21,  6.14s/it]
 57%|█████▋    | 128314/225000 [259:50:13<159:39:52,  5.94s/it]
 57%|█████▋    | 128315/225000 [259:50:18<152:08:05,  5.66s/it]
 57%|█████▋    | 128316/225000 [259:50:24<150:14:25,  5.59s/it]
 57%|█████▋    | 128317/225000 [259:50:31<164:07:38,  6.11s/it]
 57%|█████▋    | 128318/225000 [259:50:36<156:41:14,  5.83s/it]
 57%|█████▋    | 128319/225000 [259:50:41<151:18:24,  5.63s/it]
 57%|█████▋    | 128320/225000 [259:50:49<167:17:41,  6.23s/it]
 57%|█████▋    | 128321/225000 [259:50:56<171:30:40,  6.39s/it]
 57%|█████▋    | 128322/225000 [259:50:59<148:46:51,  5.54s/it]
 57%|█████▋    | 128323/225000 [259:51:07<163:20:33,  6.08s/it]
 57%|█████▋    | 128324/225000 [259:51:13<163:52:19,  6.10s/it]
 57%|█████▋    | 128325/225000 [259:51:20<173:06:27,  6.45s/it]
 57%|█████▋    | 128326/225000 [259:51:24<151:39:59,  5.65s/it]
 57%|█████▋    | 128327/225000 [259:51:28<138:25:17,  5.15s/it]
 57%|█████▋    | 128328/225000 [259:51:32<132:41:46,  4.94s/it]
 57%|█████▋    | 128329/225000 [259:51:37<126:27:50,  4.71s/it]
 57%|█████▋    | 128330/225000 [259:51:49<186:34:24,  6.95s/it]
 57%|█████▋    | 128331/225000 [259:51:54<169:29:34,  6.31s/it]
 57%|█████▋    | 128332/225000 [259:51:58<156:02:38,  5.81s/it]
 57%|█████▋    | 128333/225000 [259:52:03<148:42:25,  5.54s/it]
 57%|█████▋    | 128334/225000 [259:52:08<144:27:17,  5.38s/it]
 57%|█████▋    | 128335/225000 [259:52:13<142:07:38,  5.29s/it]
 57%|█████▋    | 128336/225000 [259:52:18<138:21:02,  5.15s/it]
 57%|█████▋    | 128337/225000 [259:52:23<138:52:06,  5.17s/it]
 57%|█████▋    | 128338/225000 [259:52:30<151:02:22,  5.63s/it]
 57%|█████▋    | 128339/225000 [259:52:42<205:05:38,  7.64s/it]
 57%|█████▋    | 128340/225000 [259:52:47<185:25:44,  6.91s/it]
 57%|█████▋    | 128341/225000 [259:52:51<159:27:18,  5.94s/it]
 57%|█████▋    | 128342/225000 [259:52:55<143:09:51,  5.33s/it]
 57%|█████▋    | 128343/225000 [259:53:06<192:01:13,  7.15s/it]
 57%|█████▋    | 128344/225000 [259:53:11<169:09:32,  6.30s/it]
 57%|█████▋    | 128345/225000 [259:53:15<149:35:45,  5.57s/it]
 57%|█████▋    | 128346/225000 [259:53:24<179:52:36,  6.70s/it]
 57%|█████▋    | 128347/225000 [259:53:29<165:08:10,  6.15s/it]
 57%|█████▋    | 128348/225000 [259:53:34<161:25:00,  6.01s/it]
 57%|█████▋    | 128349/225000 [259:53:40<157:07:35,  5.85s/it]
 57%|█████▋    | 128350/225000 [259:53:45<152:05:48,  5.67s/it]
                                                               
{'loss': '0.8294', 'grad_norm': '1.024', 'learning_rate': '2.107e-05', 'epoch': '0.5704', 'num_input_tokens_seen': 432268408, 'train_runtime': '9.356e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128350/225000 [259:53:45<152:05:48,  5.67s/it]
 57%|█████▋    | 128351/225000 [259:53:50<145:05:36,  5.40s/it]
 57%|█████▋    | 128352/225000 [259:53:55<140:21:58,  5.23s/it]
 57%|█████▋    | 128353/225000 [259:54:00<137:49:29,  5.13s/it]
 57%|█████▋    | 128354/225000 [259:54:06<144:45:57,  5.39s/it]
 57%|█████▋    | 128355/225000 [259:54:18<197:32:55,  7.36s/it]
 57%|█████▋    | 128356/225000 [259:54:22<172:41:34,  6.43s/it]
 57%|█████▋    | 128357/225000 [259:54:26<154:17:47,  5.75s/it]
 57%|█████▋    | 128358/225000 [259:54:30<139:25:51,  5.19s/it]
 57%|█████▋    | 128359/225000 [259:54:34<126:10:31,  4.70s/it]
 57%|█████▋    | 128360/225000 [259:54:46<185:32:37,  6.91s/it]
 57%|█████▋    | 128361/225000 [259:54:50<165:28:33,  6.16s/it]
 57%|█████▋    | 128362/225000 [259:54:59<190:39:57,  7.10s/it]
 57%|█████▋    | 128363/225000 [259:55:02<158:41:06,  5.91s/it]
 57%|█████▋    | 128364/225000 [259:55:06<139:05:07,  5.18s/it]
 57%|█████▋    | 128365/225000 [259:55:14<162:20:02,  6.05s/it]
 57%|█████▋    | 128366/225000 [259:55:17<141:51:35,  5.28s/it]
 57%|█████▋    | 128367/225000 [259:55:27<176:38:37,  6.58s/it]
 57%|█████▋    | 128368/225000 [259:55:32<166:58:59,  6.22s/it]
 57%|█████▋    | 128369/225000 [259:55:36<147:26:55,  5.49s/it]
 57%|█████▋    | 128370/225000 [259:55:40<134:30:18,  5.01s/it]
 57%|█████▋    | 128371/225000 [259:55:51<182:11:22,  6.79s/it]
 57%|█████▋    | 128372/225000 [259:55:56<163:18:08,  6.08s/it]
 57%|█████▋    | 128373/225000 [259:56:00<147:53:00,  5.51s/it]
 57%|█████▋    | 128374/225000 [259:56:04<137:25:42,  5.12s/it]
 57%|█████▋    | 128375/225000 [259:56:10<142:18:52,  5.30s/it]
 57%|█████▋    | 128376/225000 [259:56:22<196:18:48,  7.31s/it]
 57%|█████▋    | 128377/225000 [259:56:26<170:05:58,  6.34s/it]
 57%|█████▋    | 128378/225000 [259:56:30<149:47:36,  5.58s/it]
 57%|█████▋    | 128379/225000 [259:56:33<136:15:45,  5.08s/it]
 57%|█████▋    | 128380/225000 [259:56:46<197:07:36,  7.34s/it]
 57%|█████▋    | 128381/225000 [259:56:49<165:11:03,  6.15s/it]
 57%|█████▋    | 128382/225000 [259:56:53<147:52:14,  5.51s/it]
 57%|█████▋    | 128383/225000 [259:56:57<134:36:30,  5.02s/it]
 57%|█████▋    | 128384/225000 [259:57:06<166:40:23,  6.21s/it]
 57%|█████▋    | 128385/225000 [259:57:10<144:21:20,  5.38s/it]
 57%|█████▋    | 128386/225000 [259:57:18<165:13:31,  6.16s/it]
 57%|█████▋    | 128387/225000 [259:57:22<149:13:35,  5.56s/it]
 57%|█████▋    | 128388/225000 [259:57:26<135:26:25,  5.05s/it]
 57%|█████▋    | 128389/225000 [259:57:31<136:48:31,  5.10s/it]
 57%|█████▋    | 128390/225000 [259:57:36<138:09:24,  5.15s/it]
 57%|█████▋    | 128391/225000 [259:57:41<138:10:03,  5.15s/it]
 57%|█████▋    | 128392/225000 [259:57:46<137:56:05,  5.14s/it]
 57%|█████▋    | 128393/225000 [259:57:51<136:12:58,  5.08s/it]
 57%|█████▋    | 128394/225000 [259:57:57<138:38:47,  5.17s/it]
 57%|█████▋    | 128395/225000 [259:58:02<137:23:41,  5.12s/it]
 57%|█████▋    | 128396/225000 [259:58:07<137:42:46,  5.13s/it]
 57%|█████▋    | 128397/225000 [259:58:16<172:01:22,  6.41s/it]
 57%|█████▋    | 128398/225000 [259:58:28<212:01:27,  7.90s/it]
 57%|█████▋    | 128399/225000 [259:58:31<173:13:49,  6.46s/it]
 57%|█████▋    | 128400/225000 [259:58:35<155:03:50,  5.78s/it]
                                                               
{'loss': '0.8181', 'grad_norm': '1.066', 'learning_rate': '2.106e-05', 'epoch': '0.5707', 'num_input_tokens_seen': 432397456, 'train_runtime': '9.359e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128400/225000 [259:58:35<155:03:50,  5.78s/it]
 57%|█████▋    | 128401/225000 [259:58:39<144:05:47,  5.37s/it]
 57%|█████▋    | 128402/225000 [259:58:50<189:35:54,  7.07s/it]
 57%|█████▋    | 128403/225000 [259:58:56<175:21:40,  6.54s/it]
 57%|█████▋    | 128404/225000 [259:58:59<152:43:05,  5.69s/it]
 57%|█████▋    | 128405/225000 [259:59:09<182:07:17,  6.79s/it]
 57%|█████▋    | 128406/225000 [259:59:13<164:52:49,  6.14s/it]
 57%|█████▋    | 128407/225000 [259:59:19<161:49:11,  6.03s/it]
 57%|█████▋    | 128408/225000 [259:59:23<144:43:43,  5.39s/it]
 57%|█████▋    | 128409/225000 [259:59:32<170:48:06,  6.37s/it]
 57%|█████▋    | 128410/225000 [259:59:37<161:58:15,  6.04s/it]
 57%|█████▋    | 128411/225000 [259:59:41<146:50:44,  5.47s/it]
 57%|█████▋    | 128412/225000 [259:59:51<181:51:36,  6.78s/it]
 57%|█████▋    | 128413/225000 [259:59:55<156:46:42,  5.84s/it]
 57%|█████▋    | 128414/225000 [260:00:03<173:52:18,  6.48s/it]
 57%|█████▋    | 128415/225000 [260:00:09<172:18:59,  6.42s/it]
 57%|█████▋    | 128416/225000 [260:00:13<152:42:54,  5.69s/it]
 57%|█████▋    | 128417/225000 [260:00:21<169:41:37,  6.33s/it]
 57%|█████▋    | 128418/225000 [260:00:24<146:01:48,  5.44s/it]
 57%|█████▋    | 128419/225000 [260:00:33<171:05:39,  6.38s/it]
 57%|█████▋    | 128420/225000 [260:00:36<150:31:13,  5.61s/it]
 57%|█████▋    | 128421/225000 [260:00:40<133:04:24,  4.96s/it]
 57%|█████▋    | 128422/225000 [260:00:49<165:41:13,  6.18s/it]
 57%|█████▋    | 128423/225000 [260:00:53<145:33:02,  5.43s/it]
 57%|█████▋    | 128424/225000 [260:00:57<140:04:35,  5.22s/it]
 57%|█████▋    | 128425/225000 [260:01:04<154:28:05,  5.76s/it]
 57%|█████▋    | 128426/225000 [260:01:09<144:58:56,  5.40s/it]
 57%|█████▋    | 128427/225000 [260:01:14<138:20:57,  5.16s/it]
 57%|█████▋    | 128428/225000 [260:01:18<131:55:51,  4.92s/it]
 57%|█████▋    | 128429/225000 [260:01:25<147:54:55,  5.51s/it]
 57%|█████▋    | 128430/225000 [260:01:36<195:01:38,  7.27s/it]
 57%|█████▋    | 128431/225000 [260:01:39<162:15:32,  6.05s/it]
 57%|█████▋    | 128432/225000 [260:01:48<180:38:44,  6.73s/it]
 57%|█████▋    | 128433/225000 [260:01:52<157:32:05,  5.87s/it]
 57%|█████▋    | 128434/225000 [260:01:58<165:36:47,  6.17s/it]
 57%|█████▋    | 128435/225000 [260:02:05<170:39:27,  6.36s/it]
 57%|█████▋    | 128436/225000 [260:02:10<155:20:43,  5.79s/it]
 57%|█████▋    | 128437/225000 [260:02:15<154:13:37,  5.75s/it]
 57%|█████▋    | 128438/225000 [260:02:23<173:15:26,  6.46s/it]
 57%|█████▋    | 128439/225000 [260:02:29<166:57:12,  6.22s/it]
 57%|█████▋    | 128440/225000 [260:02:35<163:02:55,  6.08s/it]
 57%|█████▋    | 128441/225000 [260:02:43<175:47:38,  6.55s/it]
 57%|█████▋    | 128442/225000 [260:02:46<149:55:19,  5.59s/it]
 57%|█████▋    | 128443/225000 [260:02:54<173:58:43,  6.49s/it]
 57%|█████▋    | 128444/225000 [260:03:01<174:10:12,  6.49s/it]
 57%|█████▋    | 128445/225000 [260:03:06<159:11:44,  5.94s/it]
 57%|█████▋    | 128446/225000 [260:03:10<145:58:05,  5.44s/it]
 57%|█████▋    | 128447/225000 [260:03:14<135:48:41,  5.06s/it]
 57%|█████▋    | 128448/225000 [260:03:25<184:53:39,  6.89s/it]
 57%|█████▋    | 128449/225000 [260:03:31<178:39:22,  6.66s/it]
 57%|█████▋    | 128450/225000 [260:03:35<156:19:01,  5.83s/it]
                                                               
{'loss': '0.8162', 'grad_norm': '1.032', 'learning_rate': '2.104e-05', 'epoch': '0.5709', 'num_input_tokens_seen': 432527800, 'train_runtime': '9.362e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128450/225000 [260:03:35<156:19:01,  5.83s/it]
 57%|█████▋    | 128451/225000 [260:03:42<163:01:53,  6.08s/it]
 57%|█████▋    | 128452/225000 [260:03:50<182:19:35,  6.80s/it]
 57%|█████▋    | 128453/225000 [260:03:54<158:57:30,  5.93s/it]
 57%|█████▋    | 128454/225000 [260:04:03<177:44:00,  6.63s/it]
 57%|█████▋    | 128455/225000 [260:04:08<170:41:52,  6.37s/it]
 57%|█████▋    | 128456/225000 [260:04:14<162:16:29,  6.05s/it]
 57%|█████▋    | 128457/225000 [260:04:18<148:23:07,  5.53s/it]
 57%|█████▋    | 128458/225000 [260:04:22<136:41:33,  5.10s/it]
 57%|█████▋    | 128459/225000 [260:04:26<130:15:43,  4.86s/it]
 57%|█████▋    | 128460/225000 [260:04:38<186:49:20,  6.97s/it]
 57%|█████▋    | 128461/225000 [260:04:42<163:32:23,  6.10s/it]
 57%|█████▋    | 128462/225000 [260:04:46<148:15:41,  5.53s/it]
 57%|█████▋    | 128463/225000 [260:04:56<181:29:56,  6.77s/it]
 57%|█████▋    | 128464/225000 [260:05:01<169:30:09,  6.32s/it]
 57%|█████▋    | 128465/225000 [260:05:05<150:41:26,  5.62s/it]
 57%|█████▋    | 128466/225000 [260:05:12<156:29:53,  5.84s/it]
 57%|█████▋    | 128467/225000 [260:05:18<162:09:34,  6.05s/it]
 57%|█████▋    | 128468/225000 [260:05:22<145:58:56,  5.44s/it]
 57%|█████▋    | 128469/225000 [260:05:30<167:30:32,  6.25s/it]
 57%|█████▋    | 128470/225000 [260:05:34<143:26:55,  5.35s/it]
 57%|█████▋    | 128471/225000 [260:05:39<139:05:30,  5.19s/it]
 57%|█████▋    | 128472/225000 [260:05:47<166:02:15,  6.19s/it]
 57%|█████▋    | 128473/225000 [260:05:54<174:33:24,  6.51s/it]
 57%|█████▋    | 128474/225000 [260:05:58<151:24:15,  5.65s/it]
 57%|█████▋    | 128475/225000 [260:06:04<157:01:41,  5.86s/it]
 57%|█████▋    | 128476/225000 [260:06:11<161:44:40,  6.03s/it]
 57%|█████▋    | 128477/225000 [260:06:15<149:33:58,  5.58s/it]
 57%|█████▋    | 128478/225000 [260:06:20<140:36:38,  5.24s/it]
 57%|█████▋    | 128479/225000 [260:06:29<174:55:43,  6.52s/it]
 57%|█████▋    | 128480/225000 [260:06:33<156:16:24,  5.83s/it]
 57%|█████▋    | 128481/225000 [260:06:41<170:15:39,  6.35s/it]
 57%|█████▋    | 128482/225000 [260:06:49<186:04:41,  6.94s/it]
 57%|█████▋    | 128483/225000 [260:06:52<155:42:10,  5.81s/it]
 57%|█████▋    | 128484/225000 [260:07:01<174:28:02,  6.51s/it]
 57%|█████▋    | 128485/225000 [260:07:04<150:00:02,  5.60s/it]
 57%|█████▋    | 128486/225000 [260:07:10<153:26:08,  5.72s/it]
 57%|█████▋    | 128487/225000 [260:07:17<166:18:10,  6.20s/it]
 57%|█████▋    | 128488/225000 [260:07:22<151:47:34,  5.66s/it]
 57%|█████▋    | 128489/225000 [260:07:28<156:58:15,  5.86s/it]
 57%|█████▋    | 128490/225000 [260:07:32<143:22:47,  5.35s/it]
 57%|█████▋    | 128491/225000 [260:07:36<133:04:29,  4.96s/it]
 57%|█████▋    | 128492/225000 [260:07:42<136:14:53,  5.08s/it]
 57%|█████▋    | 128493/225000 [260:07:47<134:34:32,  5.02s/it]
 57%|█████▋    | 128494/225000 [260:07:52<135:20:52,  5.05s/it]
 57%|█████▋    | 128495/225000 [260:08:03<189:30:46,  7.07s/it]
 57%|█████▋    | 128496/225000 [260:08:07<163:32:53,  6.10s/it]
 57%|█████▋    | 128497/225000 [260:08:11<143:32:59,  5.36s/it]
 57%|█████▋    | 128498/225000 [260:08:21<180:57:50,  6.75s/it]
 57%|█████▋    | 128499/225000 [260:08:25<160:57:59,  6.00s/it]
 57%|█████▋    | 128500/225000 [260:08:30<149:34:06,  5.58s/it]
                                                               
{'loss': '0.8107', 'grad_norm': '0.9923', 'learning_rate': '2.102e-05', 'epoch': '0.5711', 'num_input_tokens_seen': 432657792, 'train_runtime': '9.365e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128500/225000 [260:08:30<149:34:06,  5.58s/it]
 57%|█████▋    | 128501/225000 [260:08:35<146:41:36,  5.47s/it]
 57%|█████▋    | 128502/225000 [260:08:44<176:45:39,  6.59s/it]
 57%|█████▋    | 128503/225000 [260:08:49<159:46:00,  5.96s/it]
 57%|█████▋    | 128504/225000 [260:08:57<177:15:20,  6.61s/it]
 57%|█████▋    | 128505/225000 [260:09:01<155:48:38,  5.81s/it]
 57%|█████▋    | 128506/225000 [260:09:05<142:49:50,  5.33s/it]
 57%|█████▋    | 128507/225000 [260:09:09<134:57:30,  5.04s/it]
 57%|█████▋    | 128508/225000 [260:09:21<184:45:22,  6.89s/it]
 57%|█████▋    | 128509/225000 [260:09:27<178:58:39,  6.68s/it]
 57%|█████▋    | 128510/225000 [260:09:30<154:43:44,  5.77s/it]
 57%|█████▋    | 128511/225000 [260:09:35<142:13:22,  5.31s/it]
 57%|█████▋    | 128512/225000 [260:09:39<136:38:35,  5.10s/it]
 57%|█████▋    | 128513/225000 [260:09:43<128:48:20,  4.81s/it]
 57%|█████▋    | 128514/225000 [260:09:51<153:34:24,  5.73s/it]
 57%|█████▋    | 128515/225000 [260:10:01<182:35:16,  6.81s/it]
 57%|█████▋    | 128516/225000 [260:10:08<190:59:06,  7.13s/it]
 57%|█████▋    | 128517/225000 [260:10:13<166:14:13,  6.20s/it]
 57%|█████▋    | 128518/225000 [260:10:17<151:25:11,  5.65s/it]
 57%|█████▋    | 128519/225000 [260:10:21<138:51:37,  5.18s/it]
 57%|█████▋    | 128520/225000 [260:10:26<135:20:04,  5.05s/it]
 57%|█████▋    | 128521/225000 [260:10:31<135:16:46,  5.05s/it]
 57%|█████▋    | 128522/225000 [260:10:36<134:06:12,  5.00s/it]
 57%|█████▋    | 128523/225000 [260:10:43<149:32:22,  5.58s/it]
 57%|█████▋    | 128524/225000 [260:10:56<211:28:35,  7.89s/it]
 57%|█████▋    | 128525/225000 [260:10:59<174:17:57,  6.50s/it]
 57%|█████▋    | 128526/225000 [260:11:03<149:58:29,  5.60s/it]
 57%|█████▋    | 128527/225000 [260:11:12<182:31:02,  6.81s/it]
 57%|█████▋    | 128528/225000 [260:11:16<154:43:38,  5.77s/it]
 57%|█████▋    | 128529/225000 [260:11:23<166:56:58,  6.23s/it]
 57%|█████▋    | 128530/225000 [260:11:29<166:36:57,  6.22s/it]
 57%|█████▋    | 128531/225000 [260:11:33<146:30:34,  5.47s/it]
 57%|█████▋    | 128532/225000 [260:11:38<140:41:27,  5.25s/it]
 57%|█████▋    | 128533/225000 [260:11:42<135:28:48,  5.06s/it]
 57%|█████▋    | 128534/225000 [260:11:47<132:30:39,  4.95s/it]
 57%|█████▋    | 128535/225000 [260:11:51<127:15:54,  4.75s/it]
 57%|█████▋    | 128536/225000 [260:12:01<169:49:23,  6.34s/it]
 57%|█████▋    | 128537/225000 [260:12:07<167:26:18,  6.25s/it]
 57%|█████▋    | 128538/225000 [260:12:11<149:25:28,  5.58s/it]
 57%|█████▋    | 128539/225000 [260:12:20<178:24:58,  6.66s/it]
 57%|█████▋    | 128540/225000 [260:12:26<167:22:30,  6.25s/it]
 57%|█████▋    | 128541/225000 [260:12:33<173:09:59,  6.46s/it]
 57%|█████▋    | 128542/225000 [260:12:36<149:32:29,  5.58s/it]
 57%|█████▋    | 128543/225000 [260:12:43<162:27:05,  6.06s/it]
 57%|█████▋    | 128544/225000 [260:12:48<148:11:01,  5.53s/it]
 57%|█████▋    | 128545/225000 [260:12:52<141:19:52,  5.27s/it]
 57%|█████▋    | 128546/225000 [260:12:57<133:39:28,  4.99s/it]
 57%|█████▋    | 128547/225000 [260:13:01<130:34:02,  4.87s/it]
 57%|█████▋    | 128548/225000 [260:13:06<128:36:17,  4.80s/it]
 57%|█████▋    | 128549/225000 [260:13:12<135:32:38,  5.06s/it]
 57%|█████▋    | 128550/225000 [260:13:23<185:41:26,  6.93s/it]
                                                               
{'loss': '0.8183', 'grad_norm': '1.037', 'learning_rate': '2.1e-05', 'epoch': '0.5713', 'num_input_tokens_seen': 432786240, 'train_runtime': '9.368e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128550/225000 [260:13:23<185:41:26,  6.93s/it]
 57%|█████▋    | 128551/225000 [260:13:26<158:21:35,  5.91s/it]
 57%|█████▋    | 128552/225000 [260:13:30<140:17:50,  5.24s/it]
 57%|█████▋    | 128553/225000 [260:13:41<182:56:51,  6.83s/it]
 57%|█████▋    | 128554/225000 [260:13:47<181:45:08,  6.78s/it]
 57%|█████▋    | 128555/225000 [260:13:51<156:42:27,  5.85s/it]
 57%|█████▋    | 128556/225000 [260:14:00<185:25:37,  6.92s/it]
 57%|█████▋    | 128557/225000 [260:14:04<158:16:59,  5.91s/it]
 57%|█████▋    | 128558/225000 [260:14:09<150:40:53,  5.62s/it]
 57%|█████▋    | 128559/225000 [260:14:19<185:16:51,  6.92s/it]
 57%|█████▋    | 128560/225000 [260:14:23<165:37:40,  6.18s/it]
 57%|█████▋    | 128561/225000 [260:14:28<152:17:53,  5.69s/it]
 57%|█████▋    | 128562/225000 [260:14:32<142:29:24,  5.32s/it]
 57%|█████▋    | 128563/225000 [260:14:37<139:52:43,  5.22s/it]
 57%|█████▋    | 128564/225000 [260:14:50<198:49:46,  7.42s/it]
 57%|█████▋    | 128565/225000 [260:14:54<169:22:58,  6.32s/it]
 57%|█████▋    | 128566/225000 [260:14:57<148:00:47,  5.53s/it]
 57%|█████▋    | 128567/225000 [260:15:05<163:33:48,  6.11s/it]
 57%|█████▋    | 128568/225000 [260:15:12<171:23:49,  6.40s/it]
 57%|█████▋    | 128569/225000 [260:15:15<147:21:04,  5.50s/it]
 57%|█████▋    | 128570/225000 [260:15:21<152:56:36,  5.71s/it]
 57%|█████▋    | 128571/225000 [260:15:27<155:04:00,  5.79s/it]
 57%|█████▋    | 128572/225000 [260:15:31<141:21:53,  5.28s/it]
 57%|█████▋    | 128573/225000 [260:15:36<138:26:50,  5.17s/it]
 57%|█████▋    | 128574/225000 [260:15:46<173:20:41,  6.47s/it]
 57%|█████▋    | 128575/225000 [260:15:50<155:14:20,  5.80s/it]
 57%|█████▋    | 128576/225000 [260:15:55<145:21:34,  5.43s/it]
 57%|█████▋    | 128577/225000 [260:15:59<140:44:26,  5.25s/it]
 57%|█████▋    | 128578/225000 [260:16:04<134:42:57,  5.03s/it]
 57%|█████▋    | 128579/225000 [260:16:08<130:10:06,  4.86s/it]
 57%|█████▋    | 128580/225000 [260:16:18<165:00:44,  6.16s/it]
 57%|█████▋    | 128581/225000 [260:16:28<194:42:12,  7.27s/it]
 57%|█████▋    | 128582/225000 [260:16:31<166:06:07,  6.20s/it]
 57%|█████▋    | 128583/225000 [260:16:40<190:18:19,  7.11s/it]
 57%|█████▋    | 128584/225000 [260:16:44<165:09:39,  6.17s/it]
 57%|█████▋    | 128585/225000 [260:16:48<144:26:56,  5.39s/it]
 57%|█████▋    | 128586/225000 [260:16:54<152:49:29,  5.71s/it]
 57%|█████▋    | 128587/225000 [260:17:03<171:45:58,  6.41s/it]
 57%|█████▋    | 128588/225000 [260:17:06<147:49:58,  5.52s/it]
 57%|█████▋    | 128589/225000 [260:17:11<147:35:40,  5.51s/it]
 57%|█████▋    | 128590/225000 [260:17:19<160:29:09,  5.99s/it]
 57%|█████▋    | 128591/225000 [260:17:23<151:40:09,  5.66s/it]
 57%|█████▋    | 128592/225000 [260:17:31<166:43:39,  6.23s/it]
 57%|█████▋    | 128593/225000 [260:17:35<145:31:10,  5.43s/it]
 57%|█████▋    | 128594/225000 [260:17:40<145:30:54,  5.43s/it]
 57%|█████▋    | 128595/225000 [260:17:48<165:52:32,  6.19s/it]
 57%|█████▋    | 128596/225000 [260:17:54<165:06:28,  6.17s/it]
 57%|█████▋    | 128597/225000 [260:18:00<161:34:08,  6.03s/it]
 57%|█████▋    | 128598/225000 [260:18:06<163:59:17,  6.12s/it]
 57%|█████▋    | 128599/225000 [260:18:09<141:24:32,  5.28s/it]
 57%|█████▋    | 128600/225000 [260:18:15<145:07:05,  5.42s/it]
                                                               
{'loss': '0.8262', 'grad_norm': '1.037', 'learning_rate': '2.098e-05', 'epoch': '0.5716', 'num_input_tokens_seen': 432915904, 'train_runtime': '9.371e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128600/225000 [260:18:15<145:07:05,  5.42s/it]
 57%|█████▋    | 128601/225000 [260:18:24<172:02:06,  6.42s/it]
 57%|█████▋    | 128602/225000 [260:18:29<157:06:29,  5.87s/it]
 57%|█████▋    | 128603/225000 [260:18:33<147:19:06,  5.50s/it]
 57%|█████▋    | 128604/225000 [260:18:37<137:00:08,  5.12s/it]
 57%|█████▋    | 128605/225000 [260:18:43<139:01:32,  5.19s/it]
 57%|█████▋    | 128606/225000 [260:18:54<184:16:43,  6.88s/it]
 57%|█████▋    | 128607/225000 [260:18:58<165:40:05,  6.19s/it]
 57%|█████▋    | 128608/225000 [260:19:03<152:20:24,  5.69s/it]
 57%|█████▋    | 128609/225000 [260:19:07<143:41:04,  5.37s/it]
 57%|█████▋    | 128610/225000 [260:19:12<135:45:39,  5.07s/it]
 57%|█████▋    | 128611/225000 [260:19:23<188:09:49,  7.03s/it]
 57%|█████▋    | 128612/225000 [260:19:28<171:31:21,  6.41s/it]
 57%|█████▋    | 128613/225000 [260:19:33<157:32:29,  5.88s/it]
 57%|█████▋    | 128614/225000 [260:19:38<148:33:22,  5.55s/it]
 57%|█████▋    | 128615/225000 [260:19:42<141:15:14,  5.28s/it]
 57%|█████▋    | 128616/225000 [260:19:47<135:37:07,  5.07s/it]
 57%|█████▋    | 128617/225000 [260:19:53<146:06:05,  5.46s/it]
 57%|█████▋    | 128618/225000 [260:20:06<202:43:44,  7.57s/it]
 57%|█████▋    | 128619/225000 [260:20:10<178:34:19,  6.67s/it]
 57%|█████▋    | 128620/225000 [260:20:14<151:59:17,  5.68s/it]
 57%|█████▋    | 128621/225000 [260:20:23<178:32:38,  6.67s/it]
 57%|█████▋    | 128622/225000 [260:20:26<154:54:21,  5.79s/it]
 57%|█████▋    | 128623/225000 [260:20:34<171:13:43,  6.40s/it]
 57%|█████▋    | 128624/225000 [260:20:38<152:07:21,  5.68s/it]
 57%|█████▋    | 128625/225000 [260:20:43<144:23:06,  5.39s/it]
 57%|█████▋    | 128626/225000 [260:20:48<139:34:23,  5.21s/it]
 57%|█████▋    | 128627/225000 [260:20:54<151:36:02,  5.66s/it]
 57%|█████▋    | 128628/225000 [260:21:03<177:50:39,  6.64s/it]
 57%|█████▋    | 128629/225000 [260:21:08<160:02:49,  5.98s/it]
 57%|█████▋    | 128630/225000 [260:21:12<146:44:53,  5.48s/it]
 57%|█████▋    | 128631/225000 [260:21:22<183:49:41,  6.87s/it]
 57%|█████▋    | 128632/225000 [260:21:27<171:04:39,  6.39s/it]
 57%|█████▋    | 128633/225000 [260:21:32<153:39:48,  5.74s/it]
 57%|█████▋    | 128634/225000 [260:21:36<145:03:41,  5.42s/it]
 57%|█████▋    | 128635/225000 [260:21:41<136:58:04,  5.12s/it]
 57%|█████▋    | 128636/225000 [260:21:45<127:33:38,  4.77s/it]
 57%|█████▋    | 128637/225000 [260:21:55<175:00:06,  6.54s/it]
 57%|█████▋    | 128638/225000 [260:22:04<189:36:37,  7.08s/it]
 57%|█████▋    | 128639/225000 [260:22:08<170:08:03,  6.36s/it]
 57%|█████▋    | 128640/225000 [260:22:14<160:51:14,  6.01s/it]
 57%|█████▋    | 128641/225000 [260:22:19<154:28:19,  5.77s/it]
 57%|█████▋    | 128642/225000 [260:22:24<147:25:05,  5.51s/it]
 57%|█████▋    | 128643/225000 [260:22:28<138:54:43,  5.19s/it]
 57%|█████▋    | 128644/225000 [260:22:34<141:36:53,  5.29s/it]
 57%|█████▋    | 128645/225000 [260:22:38<136:26:17,  5.10s/it]
 57%|█████▋    | 128646/225000 [260:22:43<136:19:21,  5.09s/it]
 57%|█████▋    | 128647/225000 [260:22:49<137:46:37,  5.15s/it]
 57%|█████▋    | 128648/225000 [260:23:00<189:28:58,  7.08s/it]
 57%|█████▋    | 128649/225000 [260:23:07<184:42:14,  6.90s/it]
 57%|█████▋    | 128650/225000 [260:23:11<162:24:55,  6.07s/it]
                                                               
{'loss': '0.8139', 'grad_norm': '1.243', 'learning_rate': '2.097e-05', 'epoch': '0.5718', 'num_input_tokens_seen': 433046296, 'train_runtime': '9.374e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128650/225000 [260:23:11<162:24:55,  6.07s/it]
 57%|█████▋    | 128651/225000 [260:23:15<143:22:17,  5.36s/it]
 57%|█████▋    | 128652/225000 [260:23:19<134:57:31,  5.04s/it]
 57%|█████▋    | 128653/225000 [260:23:23<130:25:10,  4.87s/it]
 57%|█████▋    | 128654/225000 [260:23:29<133:15:39,  4.98s/it]
 57%|█████▋    | 128655/225000 [260:23:34<135:50:01,  5.08s/it]
 57%|█████▋    | 128656/225000 [260:23:40<147:02:02,  5.49s/it]
 57%|█████▋    | 128657/225000 [260:23:52<192:13:39,  7.18s/it]
 57%|█████▋    | 128658/225000 [260:23:57<176:11:28,  6.58s/it]
 57%|█████▋    | 128659/225000 [260:24:00<149:24:48,  5.58s/it]
 57%|█████▋    | 128660/225000 [260:24:09<179:59:02,  6.73s/it]
 57%|█████▋    | 128661/225000 [260:24:13<154:20:01,  5.77s/it]
 57%|█████▋    | 128662/225000 [260:24:20<165:42:28,  6.19s/it]
 57%|█████▋    | 128663/225000 [260:24:27<172:40:25,  6.45s/it]
 57%|█████▋    | 128664/225000 [260:24:31<154:46:34,  5.78s/it]
 57%|█████▋    | 128665/225000 [260:24:35<139:35:07,  5.22s/it]
 57%|█████▋    | 128666/225000 [260:24:40<136:00:00,  5.08s/it]
 57%|█████▋    | 128667/225000 [260:24:45<135:50:03,  5.08s/it]
 57%|█████▋    | 128668/225000 [260:24:51<139:14:26,  5.20s/it]
 57%|█████▋    | 128669/225000 [260:24:56<141:15:34,  5.28s/it]
 57%|█████▋    | 128670/225000 [260:25:01<139:59:52,  5.23s/it]
 57%|█████▋    | 128671/225000 [260:25:06<139:15:17,  5.20s/it]
 57%|█████▋    | 128672/225000 [260:25:12<140:59:52,  5.27s/it]
 57%|█████▋    | 128673/225000 [260:25:17<139:11:36,  5.20s/it]
 57%|█████▋    | 128674/225000 [260:25:29<191:59:38,  7.18s/it]
 57%|█████▋    | 128675/225000 [260:25:38<212:01:54,  7.92s/it]
 57%|█████▋    | 128676/225000 [260:25:42<177:04:33,  6.62s/it]
 57%|█████▋    | 128677/225000 [260:25:50<189:28:51,  7.08s/it]
 57%|█████▋    | 128678/225000 [260:25:54<162:35:51,  6.08s/it]
 57%|█████▋    | 128679/225000 [260:25:57<143:23:34,  5.36s/it]
 57%|█████▋    | 128680/225000 [260:26:01<131:07:22,  4.90s/it]
 57%|█████▋    | 128681/225000 [260:26:09<153:37:50,  5.74s/it]
 57%|█████▋    | 128682/225000 [260:26:18<177:22:50,  6.63s/it]
 57%|█████▋    | 128683/225000 [260:26:21<152:55:24,  5.72s/it]
 57%|█████▋    | 128684/225000 [260:26:25<137:51:40,  5.15s/it]
 57%|█████▋    | 128685/225000 [260:26:35<175:13:21,  6.55s/it]
 57%|█████▋    | 128686/225000 [260:26:39<152:22:02,  5.70s/it]
 57%|█████▋    | 128687/225000 [260:26:47<173:11:26,  6.47s/it]
 57%|█████▋    | 128688/225000 [260:26:52<161:09:04,  6.02s/it]
 57%|█████▋    | 128689/225000 [260:26:56<150:31:15,  5.63s/it]
 57%|█████▋    | 128690/225000 [260:27:02<146:19:03,  5.47s/it]
 57%|█████▋    | 128691/225000 [260:27:06<139:39:47,  5.22s/it]
 57%|█████▋    | 128692/225000 [260:27:11<137:04:58,  5.12s/it]
 57%|█████▋    | 128693/225000 [260:27:16<135:19:19,  5.06s/it]
 57%|█████▋    | 128694/225000 [260:27:21<131:03:54,  4.90s/it]
 57%|█████▋    | 128695/225000 [260:27:26<133:38:31,  5.00s/it]
 57%|█████▋    | 128696/225000 [260:27:36<171:56:36,  6.43s/it]
 57%|█████▋    | 128697/225000 [260:27:45<198:32:21,  7.42s/it]
 57%|█████▋    | 128698/225000 [260:27:49<170:24:58,  6.37s/it]
 57%|█████▋    | 128699/225000 [260:27:53<151:53:11,  5.68s/it]
 57%|█████▋    | 128700/225000 [260:27:58<140:17:13,  5.24s/it]
                                                               
{'loss': '0.8273', 'grad_norm': '1.082', 'learning_rate': '2.095e-05', 'epoch': '0.572', 'num_input_tokens_seen': 433175240, 'train_runtime': '9.377e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128700/225000 [260:27:58<140:17:13,  5.24s/it]
 57%|█████▋    | 128701/225000 [260:28:09<191:16:18,  7.15s/it]
 57%|█████▋    | 128702/225000 [260:28:13<163:29:58,  6.11s/it]
 57%|█████▋    | 128703/225000 [260:28:20<173:39:09,  6.49s/it]
 57%|█████▋    | 128704/225000 [260:28:27<172:33:17,  6.45s/it]
 57%|█████▋    | 128705/225000 [260:28:33<169:10:31,  6.32s/it]
 57%|█████▋    | 128706/225000 [260:28:40<175:20:38,  6.56s/it]
 57%|█████▋    | 128707/225000 [260:28:43<151:31:27,  5.66s/it]
 57%|█████▋    | 128708/225000 [260:28:47<138:19:35,  5.17s/it]
 57%|█████▋    | 128709/225000 [260:28:51<129:28:14,  4.84s/it]
 57%|█████▋    | 128710/225000 [260:29:03<182:20:40,  6.82s/it]
 57%|█████▋    | 128711/225000 [260:29:07<159:13:42,  5.95s/it]
 57%|█████▋    | 128712/225000 [260:29:10<139:56:17,  5.23s/it]
 57%|█████▋    | 128713/225000 [260:29:15<132:37:46,  4.96s/it]
 57%|█████▋    | 128714/225000 [260:29:22<154:48:33,  5.79s/it]
 57%|█████▋    | 128715/225000 [260:29:33<196:10:03,  7.33s/it]
 57%|█████▋    | 128716/225000 [260:29:37<170:25:48,  6.37s/it]
 57%|█████▋    | 128717/225000 [260:29:41<149:07:16,  5.58s/it]
 57%|█████▋    | 128718/225000 [260:29:46<140:00:26,  5.23s/it]
 57%|█████▋    | 128719/225000 [260:29:50<137:17:42,  5.13s/it]
 57%|█████▋    | 128720/225000 [260:29:55<130:35:57,  4.88s/it]
 57%|█████▋    | 128721/225000 [260:30:00<134:07:25,  5.02s/it]
 57%|█████▋    | 128722/225000 [260:30:12<185:57:23,  6.95s/it]
 57%|█████▋    | 128723/225000 [260:30:18<184:51:48,  6.91s/it]
 57%|█████▋    | 128724/225000 [260:30:22<157:39:30,  5.90s/it]
 57%|█████▋    | 128725/225000 [260:30:30<177:51:51,  6.65s/it]
 57%|█████▋    | 128726/225000 [260:30:34<156:52:10,  5.87s/it]
 57%|█████▋    | 128727/225000 [260:30:38<141:37:55,  5.30s/it]
 57%|█████▋    | 128728/225000 [260:30:43<134:41:24,  5.04s/it]
 57%|█████▋    | 128729/225000 [260:30:47<130:05:11,  4.86s/it]
 57%|█████▋    | 128730/225000 [260:30:52<129:25:13,  4.84s/it]
 57%|█████▋    | 128731/225000 [260:31:04<190:15:22,  7.11s/it]
 57%|█████▋    | 128732/225000 [260:31:11<182:40:41,  6.83s/it]
 57%|█████▋    | 128733/225000 [260:31:14<156:57:28,  5.87s/it]
 57%|█████▋    | 128734/225000 [260:31:24<188:09:07,  7.04s/it]
 57%|█████▋    | 128735/225000 [260:31:27<159:04:43,  5.95s/it]
 57%|█████▋    | 128736/225000 [260:31:32<147:35:49,  5.52s/it]
 57%|█████▋    | 128737/225000 [260:31:38<148:38:26,  5.56s/it]
 57%|█████▋    | 128738/225000 [260:31:42<141:09:18,  5.28s/it]
 57%|█████▋    | 128739/225000 [260:31:47<138:26:12,  5.18s/it]
 57%|█████▋    | 128740/225000 [260:31:52<138:45:43,  5.19s/it]
 57%|█████▋    | 128741/225000 [260:31:58<140:39:22,  5.26s/it]
 57%|█████▋    | 128742/225000 [260:32:03<141:04:03,  5.28s/it]
 57%|█████▋    | 128743/225000 [260:32:08<141:50:53,  5.31s/it]
 57%|█████▋    | 128744/225000 [260:32:13<138:51:30,  5.19s/it]
 57%|█████▋    | 128745/225000 [260:32:19<141:09:05,  5.28s/it]
 57%|█████▋    | 128746/225000 [260:32:30<186:26:18,  6.97s/it]
 57%|█████▋    | 128747/225000 [260:32:37<187:54:16,  7.03s/it]
 57%|█████▋    | 128748/225000 [260:32:40<156:12:27,  5.84s/it]
 57%|█████▋    | 128749/225000 [260:32:45<147:41:35,  5.52s/it]
 57%|█████▋    | 128750/225000 [260:32:50<145:40:53,  5.45s/it]
                                                               
{'loss': '0.8284', 'grad_norm': '0.984', 'learning_rate': '2.093e-05', 'epoch': '0.5722', 'num_input_tokens_seen': 433304456, 'train_runtime': '9.38e+05', 'train_tokens_per_second': '462'}

 57%|█████▋    | 128750/225000 [260:32:50<145:40:53,  5.45s/it]
 57%|█████▋    | 128751/225000 [260:32:54<134:32:57,  5.03s/it]
 57%|█████▋    | 128752/225000 [260:32:59<136:26:16,  5.10s/it]
 57%|█████▋    | 128753/225000 [260:33:04<134:36:09,  5.03s/it]
 57%|█████▋    | 128754/225000 [260:33:10<138:40:43,  5.19s/it]
 57%|█████▋    | 128755/225000 [260:33:23<201:21:33,  7.53s/it]
 57%|█████▋    | 128756/225000 [260:33:29<191:03:47,  7.15s/it]
 57%|█████▋    | 128757/225000 [260:33:35<181:57:47,  6.81s/it]
 57%|█████▋    | 128758/225000 [260:33:42<183:16:19,  6.86s/it]
 57%|█████▋    | 128759/225000 [260:33:47<172:20:24,  6.45s/it]
 57%|█████▋    | 128760/225000 [260:33:55<182:29:01,  6.83s/it]
 57%|█████▋    | 128761/225000 [260:33:59<156:14:41,  5.84s/it]
 57%|█████▋    | 128762/225000 [260:34:06<168:27:49,  6.30s/it]
 57%|█████▋    | 128763/225000 [260:34:12<163:33:08,  6.12s/it]
 57%|█████▋    | 128764/225000 [260:34:16<147:34:44,  5.52s/it]
 57%|█████▋    | 128765/225000 [260:34:21<146:17:01,  5.47s/it]
 57%|█████▋    | 128766/225000 [260:34:30<171:43:25,  6.42s/it]
 57%|█████▋    | 128767/225000 [260:34:34<155:55:55,  5.83s/it]
 57%|█████▋    | 128768/225000 [260:34:41<161:02:34,  6.02s/it]
 57%|█████▋    | 128769/225000 [260:34:44<140:46:56,  5.27s/it]
 57%|█████▋    | 128770/225000 [260:34:51<155:19:47,  5.81s/it]
 57%|█████▋    | 128771/225000 [260:34:59<168:26:27,  6.30s/it]
 57%|█████▋    | 128772/225000 [260:35:04<156:35:07,  5.86s/it]
 57%|█████▋    | 128773/225000 [260:35:08<144:55:09,  5.42s/it]
 57%|█████▋    | 128774/225000 [260:35:13<138:47:19,  5.19s/it]
 57%|█████▋    | 128775/225000 [260:35:18<135:34:05,  5.07s/it]
 57%|█████▋    | 128776/225000 [260:35:25<157:12:47,  5.88s/it]
 57%|█████▋    | 128777/225000 [260:35:35<188:15:01,  7.04s/it]
 57%|█████▋    | 128778/225000 [260:35:39<166:02:51,  6.21s/it]
 57%|█████▋    | 128779/225000 [260:35:45<157:51:02,  5.91s/it]
 57%|█████▋    | 128780/225000 [260:35:49<149:33:51,  5.60s/it]
 57%|█████▋    | 128781/225000 [260:35:55<149:36:59,  5.60s/it]
 57%|█████▋    | 128782/225000 [260:36:00<146:05:17,  5.47s/it]
 57%|█████▋    | 128783/225000 [260:36:05<143:38:32,  5.37s/it]
 57%|█████▋    | 128784/225000 [260:36:10<141:21:05,  5.29s/it]
 57%|█████▋    | 128785/225000 [260:36:16<143:44:06,  5.38s/it]
 57%|█████▋    | 128786/225000 [260:36:22<144:47:46,  5.42s/it]
 57%|█████▋    | 128787/225000 [260:36:38<229:41:47,  8.59s/it]
 57%|█████▋    | 128788/225000 [260:36:44<209:45:08,  7.85s/it]
 57%|█████▋    | 128789/225000 [260:36:47<176:36:37,  6.61s/it]
 57%|█████▋    | 128790/225000 [260:36:54<178:39:13,  6.68s/it]
 57%|█████▋    | 128791/225000 [260:37:02<187:01:21,  7.00s/it]
 57%|█████▋    | 128792/225000 [260:37:06<167:03:19,  6.25s/it]
 57%|█████▋    | 128793/225000 [260:37:11<154:19:03,  5.77s/it]
 57%|█████▋    | 128794/225000 [260:37:16<145:53:18,  5.46s/it]
 57%|█████▋    | 128795/225000 [260:37:21<140:56:59,  5.27s/it]
 57%|█████▋    | 128796/225000 [260:37:25<135:07:06,  5.06s/it]
 57%|█████▋    | 128797/225000 [260:37:34<164:29:32,  6.16s/it]
 57%|█████▋    | 128798/225000 [260:37:45<201:17:57,  7.53s/it]
 57%|█████▋    | 128799/225000 [260:37:49<178:31:22,  6.68s/it]
 57%|█████▋    | 128800/225000 [260:37:56<173:54:06,  6.51s/it]
                                                               
{'loss': '0.8144', 'grad_norm': '1.067', 'learning_rate': '2.091e-05', 'epoch': '0.5724', 'num_input_tokens_seen': 433433704, 'train_runtime': '9.383e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 128800/225000 [260:37:56<173:54:06,  6.51s/it]
 57%|█████▋    | 128801/225000 [260:38:00<156:55:41,  5.87s/it]
 57%|█████▋    | 128802/225000 [260:38:04<144:26:13,  5.41s/it]
 57%|█████▋    | 128803/225000 [260:38:09<136:19:06,  5.10s/it]
 57%|█████▋    | 128804/225000 [260:38:13<133:49:23,  5.01s/it]
 57%|█████▋    | 128805/225000 [260:38:22<161:48:57,  6.06s/it]
 57%|█████▋    | 128806/225000 [260:38:31<188:55:45,  7.07s/it]
 57%|█████▋    | 128807/225000 [260:38:35<163:36:53,  6.12s/it]
 57%|█████▋    | 128808/225000 [260:38:42<166:14:15,  6.22s/it]
 57%|█████▋    | 128809/225000 [260:38:50<180:42:28,  6.76s/it]
 57%|█████▋    | 128810/225000 [260:38:56<176:35:53,  6.61s/it]
 57%|█████▋    | 128811/225000 [260:39:00<152:06:07,  5.69s/it]
 57%|█████▋    | 128812/225000 [260:39:09<178:58:46,  6.70s/it]
 57%|█████▋    | 128813/225000 [260:39:13<161:10:54,  6.03s/it]
 57%|█████▋    | 128814/225000 [260:39:17<145:29:28,  5.45s/it]
 57%|█████▋    | 128815/225000 [260:39:21<135:39:31,  5.08s/it]
 57%|█████▋    | 128816/225000 [260:39:25<127:07:33,  4.76s/it]
 57%|█████▋    | 128817/225000 [260:39:36<178:11:08,  6.67s/it]
 57%|█████▋    | 128818/225000 [260:39:40<155:00:35,  5.80s/it]
 57%|█████▋    | 128819/225000 [260:39:44<140:12:07,  5.25s/it]
 57%|█████▋    | 128820/225000 [260:39:54<174:29:50,  6.53s/it]
 57%|█████▋    | 128821/225000 [260:39:57<147:03:44,  5.50s/it]
 57%|█████▋    | 128822/225000 [260:40:06<173:51:02,  6.51s/it]
 57%|█████▋    | 128823/225000 [260:40:10<154:20:52,  5.78s/it]
 57%|█████▋    | 128824/225000 [260:40:13<135:37:22,  5.08s/it]
 57%|█████▋    | 128825/225000 [260:40:24<183:24:21,  6.87s/it]
 57%|█████▋    | 128826/225000 [260:40:28<155:56:57,  5.84s/it]
 57%|█████▋    | 128827/225000 [260:40:31<139:26:45,  5.22s/it]
 57%|█████▋    | 128828/225000 [260:40:41<177:45:02,  6.65s/it]
 57%|█████▋    | 128829/225000 [260:40:45<156:19:33,  5.85s/it]
 57%|█████▋    | 128830/225000 [260:40:50<143:17:58,  5.36s/it]
 57%|█████▋    | 128831/225000 [260:40:55<141:25:52,  5.29s/it]
 57%|█████▋    | 128832/225000 [260:41:06<191:31:15,  7.17s/it]
 57%|█████▋    | 128833/225000 [260:41:10<163:14:37,  6.11s/it]
 57%|█████▋    | 128834/225000 [260:41:19<182:24:18,  6.83s/it]
 57%|█████▋    | 128835/225000 [260:41:22<157:40:17,  5.90s/it]
 57%|█████▋    | 128836/225000 [260:41:30<175:57:00,  6.59s/it]
 57%|█████▋    | 128837/225000 [260:41:34<151:02:38,  5.65s/it]
 57%|█████▋    | 128838/225000 [260:41:43<178:34:51,  6.69s/it]
 57%|█████▋    | 128839/225000 [260:41:47<153:02:20,  5.73s/it]
 57%|█████▋    | 128840/225000 [260:41:55<173:02:33,  6.48s/it]
 57%|█████▋    | 128841/225000 [260:41:59<153:00:23,  5.73s/it]
 57%|█████▋    | 128842/225000 [260:42:07<173:05:33,  6.48s/it]
 57%|█████▋    | 128843/225000 [260:42:14<178:17:39,  6.68s/it]
 57%|█████▋    | 128844/225000 [260:42:18<155:05:22,  5.81s/it]
 57%|█████▋    | 128845/225000 [260:42:28<186:13:33,  6.97s/it]
 57%|█████▋    | 128846/225000 [260:42:32<164:56:18,  6.18s/it]
 57%|█████▋    | 128847/225000 [260:42:37<154:57:26,  5.80s/it]
 57%|█████▋    | 128848/225000 [260:42:41<144:43:32,  5.42s/it]
 57%|█████▋    | 128849/225000 [260:42:46<139:54:01,  5.24s/it]
 57%|█████▋    | 128850/225000 [260:42:50<132:56:05,  4.98s/it]
                                                               
{'loss': '0.8207', 'grad_norm': '1.086', 'learning_rate': '2.089e-05', 'epoch': '0.5727', 'num_input_tokens_seen': 433563320, 'train_runtime': '9.386e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 128850/225000 [260:42:51<132:56:05,  4.98s/it]
 57%|█████▋    | 128851/225000 [260:42:56<134:40:22,  5.04s/it]
 57%|█████▋    | 128852/225000 [260:43:09<198:07:49,  7.42s/it]
 57%|█████▋    | 128853/225000 [260:43:12<163:04:33,  6.11s/it]
 57%|█████▋    | 128854/225000 [260:43:19<173:28:20,  6.50s/it]
 57%|█████▋    | 128855/225000 [260:43:23<151:41:14,  5.68s/it]
 57%|█████▋    | 128856/225000 [260:43:27<140:45:28,  5.27s/it]
 57%|█████▋    | 128857/225000 [260:43:37<173:25:36,  6.49s/it]
 57%|█████▋    | 128858/225000 [260:43:40<148:55:01,  5.58s/it]
 57%|█████▋    | 128859/225000 [260:43:52<202:33:27,  7.58s/it]
 57%|█████▋    | 128860/225000 [260:43:57<178:08:18,  6.67s/it]
 57%|█████▋    | 128861/225000 [260:44:01<162:15:24,  6.08s/it]
 57%|█████▋    | 128862/225000 [260:44:06<149:55:24,  5.61s/it]
 57%|█████▋    | 128863/225000 [260:44:11<141:05:06,  5.28s/it]
 57%|█████▋    | 128864/225000 [260:44:17<149:33:41,  5.60s/it]
 57%|█████▋    | 128865/225000 [260:44:27<183:49:37,  6.88s/it]
 57%|█████▋    | 128866/225000 [260:44:30<154:44:39,  5.79s/it]
 57%|█████▋    | 128867/225000 [260:44:38<176:05:03,  6.59s/it]
 57%|█████▋    | 128868/225000 [260:44:42<151:21:45,  5.67s/it]
 57%|█████▋    | 128869/225000 [260:44:48<156:45:58,  5.87s/it]
 57%|█████▋    | 128870/225000 [260:44:55<160:18:26,  6.00s/it]
 57%|█████▋    | 128871/225000 [260:44:58<142:13:37,  5.33s/it]
 57%|█████▋    | 128872/225000 [260:45:03<136:08:07,  5.10s/it]
 57%|█████▋    | 128873/225000 [260:45:14<187:25:29,  7.02s/it]
 57%|█████▋    | 128874/225000 [260:45:18<160:03:08,  5.99s/it]
 57%|█████▋    | 128875/225000 [260:45:22<144:06:34,  5.40s/it]
 57%|█████▋    | 128876/225000 [260:45:32<179:32:49,  6.72s/it]
 57%|█████▋    | 128877/225000 [260:45:36<162:07:45,  6.07s/it]
 57%|█████▋    | 128878/225000 [260:45:41<147:59:29,  5.54s/it]
 57%|█████▋    | 128879/225000 [260:45:45<137:39:17,  5.16s/it]
 57%|█████▋    | 128880/225000 [260:45:54<172:27:28,  6.46s/it]
 57%|█████▋    | 128881/225000 [260:45:59<155:34:07,  5.83s/it]
 57%|█████▋    | 128882/225000 [260:46:07<172:13:06,  6.45s/it]
 57%|█████▋    | 128883/225000 [260:46:12<160:18:34,  6.00s/it]
 57%|█████▋    | 128884/225000 [260:46:18<161:22:25,  6.04s/it]
 57%|█████▋    | 128885/225000 [260:46:22<146:30:31,  5.49s/it]
 57%|█████▋    | 128886/225000 [260:46:26<131:55:26,  4.94s/it]
 57%|█████▋    | 128887/225000 [260:46:32<140:25:22,  5.26s/it]
 57%|█████▋    | 128888/225000 [260:46:42<179:58:17,  6.74s/it]
 57%|█████▋    | 128889/225000 [260:46:46<160:22:26,  6.01s/it]
 57%|█████▋    | 128890/225000 [260:46:50<144:58:06,  5.43s/it]
 57%|█████▋    | 128891/225000 [260:47:00<179:13:14,  6.71s/it]
 57%|█████▋    | 128892/225000 [260:47:03<150:29:19,  5.64s/it]
 57%|█████▋    | 128893/225000 [260:47:11<169:51:06,  6.36s/it]
 57%|█████▋    | 128894/225000 [260:47:15<147:26:31,  5.52s/it]
 57%|█████▋    | 128895/225000 [260:47:20<143:32:45,  5.38s/it]
 57%|█████▋    | 128896/225000 [260:47:24<138:07:39,  5.17s/it]
 57%|█████▋    | 128897/225000 [260:47:29<136:23:33,  5.11s/it]
 57%|█████▋    | 128898/225000 [260:47:40<179:58:00,  6.74s/it]
 57%|█████▋    | 128899/225000 [260:47:45<168:09:16,  6.30s/it]
 57%|█████▋    | 128900/225000 [260:47:52<172:47:34,  6.47s/it]
                                                               
{'loss': '0.8222', 'grad_norm': '1.224', 'learning_rate': '2.088e-05', 'epoch': '0.5729', 'num_input_tokens_seen': 433693960, 'train_runtime': '9.389e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 128900/225000 [260:47:52<172:47:34,  6.47s/it]
 57%|█████▋    | 128901/225000 [260:47:57<158:05:02,  5.92s/it]
 57%|█████▋    | 128902/225000 [260:48:03<162:52:48,  6.10s/it]
 57%|█████▋    | 128903/225000 [260:48:06<137:53:08,  5.17s/it]
 57%|█████▋    | 128904/225000 [260:48:15<169:35:45,  6.35s/it]
 57%|█████▋    | 128905/225000 [260:48:19<148:55:29,  5.58s/it]
 57%|█████▋    | 128906/225000 [260:48:23<138:59:20,  5.21s/it]
 57%|█████▋    | 128907/225000 [260:48:34<185:12:16,  6.94s/it]
 57%|█████▋    | 128908/225000 [260:48:41<180:57:42,  6.78s/it]
 57%|█████▋    | 128909/225000 [260:48:45<159:33:28,  5.98s/it]
 57%|█████▋    | 128910/225000 [260:48:49<145:49:38,  5.46s/it]
 57%|█████▋    | 128911/225000 [260:48:53<132:01:12,  4.95s/it]
 57%|█████▋    | 128912/225000 [260:49:02<164:58:51,  6.18s/it]
 57%|█████▋    | 128913/225000 [260:49:07<151:15:16,  5.67s/it]
 57%|█████▋    | 128914/225000 [260:49:11<144:21:01,  5.41s/it]
 57%|█████▋    | 128915/225000 [260:49:16<137:59:52,  5.17s/it]
 57%|█████▋    | 128916/225000 [260:49:21<135:04:37,  5.06s/it]
 57%|█████▋    | 128917/225000 [260:49:29<159:33:54,  5.98s/it]
 57%|█████▋    | 128918/225000 [260:49:40<203:36:42,  7.63s/it]
 57%|█████▋    | 128919/225000 [260:49:44<170:11:14,  6.38s/it]
 57%|█████▋    | 128920/225000 [260:49:53<189:20:00,  7.09s/it]
 57%|█████▋    | 128921/225000 [260:49:57<167:14:59,  6.27s/it]
 57%|█████▋    | 128922/225000 [260:50:02<155:07:57,  5.81s/it]
 57%|█████▋    | 128923/225000 [260:50:06<141:54:30,  5.32s/it]
 57%|█████▋    | 128924/225000 [260:50:10<136:52:00,  5.13s/it]
 57%|█████▋    | 128925/225000 [260:50:15<135:36:00,  5.08s/it]
 57%|█████▋    | 128926/225000 [260:50:20<132:18:17,  4.96s/it]
 57%|█████▋    | 128927/225000 [260:50:25<130:32:25,  4.89s/it]
 57%|█████▋    | 128928/225000 [260:50:38<194:07:32,  7.27s/it]
 57%|█████▋    | 128929/225000 [260:50:44<185:38:06,  6.96s/it]
 57%|█████▋    | 128930/225000 [260:50:47<157:34:46,  5.90s/it]
 57%|█████▋    | 128931/225000 [260:50:58<193:17:54,  7.24s/it]
 57%|█████▋    | 128932/225000 [260:51:03<175:05:44,  6.56s/it]
 57%|█████▋    | 128933/225000 [260:51:08<162:43:50,  6.10s/it]
 57%|█████▋    | 128934/225000 [260:51:12<150:19:51,  5.63s/it]
 57%|█████▋    | 128935/225000 [260:51:17<143:47:11,  5.39s/it]
 57%|█████▋    | 128936/225000 [260:51:22<138:03:43,  5.17s/it]
 57%|█████▋    | 128937/225000 [260:51:27<136:57:36,  5.13s/it]
 57%|█████▋    | 128938/225000 [260:51:32<138:28:28,  5.19s/it]
 57%|█████▋    | 128939/225000 [260:51:37<134:03:23,  5.02s/it]
 57%|█████▋    | 128940/225000 [260:51:42<134:39:12,  5.05s/it]
 57%|█████▋    | 128941/225000 [260:51:47<135:45:45,  5.09s/it]
 57%|█████▋    | 128942/225000 [260:51:59<190:10:22,  7.13s/it]
 57%|█████▋    | 128943/225000 [260:52:06<187:47:18,  7.04s/it]
 57%|█████▋    | 128944/225000 [260:52:13<191:38:05,  7.18s/it]
 57%|█████▋    | 128945/225000 [260:52:18<173:27:47,  6.50s/it]
 57%|█████▋    | 128946/225000 [260:52:22<149:52:32,  5.62s/it]
 57%|█████▋    | 128947/225000 [260:52:31<182:18:29,  6.83s/it]
 57%|█████▋    | 128948/225000 [260:52:36<167:06:10,  6.26s/it]
 57%|█████▋    | 128949/225000 [260:52:42<158:46:45,  5.95s/it]
 57%|█████▋    | 128950/225000 [260:52:47<151:30:26,  5.68s/it]
                                                               
{'loss': '0.8188', 'grad_norm': '1.322', 'learning_rate': '2.086e-05', 'epoch': '0.5731', 'num_input_tokens_seen': 433823120, 'train_runtime': '9.392e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 128950/225000 [260:52:47<151:30:26,  5.68s/it]
 57%|█████▋    | 128951/225000 [260:52:51<143:19:11,  5.37s/it]
 57%|█████▋    | 128952/225000 [260:52:56<140:25:09,  5.26s/it]
 57%|█████▋    | 128953/225000 [260:53:01<139:39:32,  5.23s/it]
 57%|█████▋    | 128954/225000 [260:53:07<139:11:25,  5.22s/it]
 57%|█████▋    | 128955/225000 [260:53:12<137:45:19,  5.16s/it]
 57%|█████▋    | 128956/225000 [260:53:17<137:38:04,  5.16s/it]
 57%|█████▋    | 128957/225000 [260:53:21<133:07:21,  4.99s/it]
 57%|█████▋    | 128958/225000 [260:53:29<157:42:11,  5.91s/it]
 57%|█████▋    | 128959/225000 [260:53:42<210:47:12,  7.90s/it]
 57%|█████▋    | 128960/225000 [260:53:46<182:09:29,  6.83s/it]
 57%|█████▋    | 128961/225000 [260:53:55<195:10:05,  7.32s/it]
 57%|█████▋    | 128962/225000 [260:53:59<167:09:55,  6.27s/it]
 57%|█████▋    | 128963/225000 [260:54:02<145:31:43,  5.46s/it]
 57%|█████▋    | 128964/225000 [260:54:14<196:53:36,  7.38s/it]
 57%|█████▋    | 128965/225000 [260:54:19<176:34:23,  6.62s/it]
 57%|█████▋    | 128966/225000 [260:54:24<160:51:12,  6.03s/it]
 57%|█████▋    | 128967/225000 [260:54:28<149:37:22,  5.61s/it]
 57%|█████▋    | 128968/225000 [260:54:33<140:52:47,  5.28s/it]
 57%|█████▋    | 128969/225000 [260:54:37<136:35:22,  5.12s/it]
 57%|█████▋    | 128970/225000 [260:54:42<135:51:57,  5.09s/it]
 57%|█████▋    | 128971/225000 [260:54:56<201:12:42,  7.54s/it]
 57%|█████▋    | 128972/225000 [260:54:59<170:16:46,  6.38s/it]
 57%|█████▋    | 128973/225000 [260:55:03<148:29:33,  5.57s/it]
 57%|█████▋    | 128974/225000 [260:55:13<182:34:01,  6.84s/it]
 57%|█████▋    | 128975/225000 [260:55:17<164:14:01,  6.16s/it]
 57%|█████▋    | 128976/225000 [260:55:22<154:08:34,  5.78s/it]
 57%|█████▋    | 128977/225000 [260:55:28<153:18:56,  5.75s/it]
 57%|█████▋    | 128978/225000 [260:55:33<144:12:57,  5.41s/it]
 57%|█████▋    | 128979/225000 [260:55:38<140:01:09,  5.25s/it]
 57%|█████▋    | 128980/225000 [260:55:42<137:18:24,  5.15s/it]
 57%|█████▋    | 128981/225000 [260:55:48<139:24:21,  5.23s/it]
 57%|█████▋    | 128982/225000 [260:55:52<135:01:10,  5.06s/it]
 57%|█████▋    | 128983/225000 [260:56:01<160:14:23,  6.01s/it]
 57%|█████▋    | 128984/225000 [260:56:13<212:21:00,  7.96s/it]
 57%|█████▋    | 128985/225000 [260:56:16<171:56:08,  6.45s/it]
 57%|█████▋    | 128986/225000 [260:56:23<177:33:58,  6.66s/it]
 57%|█████▋    | 128987/225000 [260:56:27<151:28:07,  5.68s/it]
 57%|█████▋    | 128988/225000 [260:56:34<166:51:31,  6.26s/it]
 57%|█████▋    | 128989/225000 [260:56:38<149:55:35,  5.62s/it]
 57%|█████▋    | 128990/225000 [260:56:43<137:40:24,  5.16s/it]
 57%|█████▋    | 128991/225000 [260:56:51<166:48:44,  6.25s/it]
 57%|█████▋    | 128992/225000 [260:56:59<175:11:15,  6.57s/it]
 57%|█████▋    | 128993/225000 [260:57:06<181:50:37,  6.82s/it]
 57%|█████▋    | 128994/225000 [260:57:10<160:14:58,  6.01s/it]
 57%|█████▋    | 128995/225000 [260:57:14<143:18:38,  5.37s/it]
 57%|█████▋    | 128996/225000 [260:57:23<170:57:23,  6.41s/it]
 57%|█████▋    | 128997/225000 [260:57:26<147:31:25,  5.53s/it]
 57%|█████▋    | 128998/225000 [260:57:33<157:38:37,  5.91s/it]
 57%|█████▋    | 128999/225000 [260:57:39<158:22:13,  5.94s/it]
 57%|█████▋    | 129000/225000 [260:57:46<169:09:36,  6.34s/it]
                                                               
{'loss': '0.8191', 'grad_norm': '1.042', 'learning_rate': '2.084e-05', 'epoch': '0.5733', 'num_input_tokens_seen': 433953152, 'train_runtime': '9.395e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129000/225000 [260:57:46<169:09:36,  6.34s/it]
 57%|█████▋    | 129001/225000 [260:57:51<151:14:32,  5.67s/it]
 57%|█████▋    | 129002/225000 [260:57:55<139:17:54,  5.22s/it]
 57%|█████▋    | 129003/225000 [260:57:59<131:22:38,  4.93s/it]
 57%|█████▋    | 129004/225000 [260:58:07<153:26:43,  5.75s/it]
 57%|█████▋    | 129005/225000 [260:58:16<182:44:29,  6.85s/it]
 57%|█████▋    | 129006/225000 [260:58:20<156:26:12,  5.87s/it]
 57%|█████▋    | 129007/225000 [260:58:29<180:47:36,  6.78s/it]
 57%|█████▋    | 129008/225000 [260:58:33<163:23:35,  6.13s/it]
 57%|█████▋    | 129009/225000 [260:58:40<166:21:14,  6.24s/it]
 57%|█████▋    | 129010/225000 [260:58:43<144:21:51,  5.41s/it]
 57%|█████▋    | 129011/225000 [260:58:56<200:16:49,  7.51s/it]
 57%|█████▋    | 129012/225000 [260:58:59<168:59:38,  6.34s/it]
 57%|█████▋    | 129013/225000 [260:59:03<145:11:40,  5.45s/it]
 57%|█████▋    | 129014/225000 [260:59:11<166:46:40,  6.26s/it]
 57%|█████▋    | 129015/225000 [260:59:15<149:59:42,  5.63s/it]
 57%|█████▋    | 129016/225000 [260:59:19<142:21:56,  5.34s/it]
 57%|█████▋    | 129017/225000 [260:59:24<138:30:11,  5.19s/it]
 57%|█████▋    | 129018/225000 [260:59:29<137:08:35,  5.14s/it]
 57%|█████▋    | 129019/225000 [260:59:34<133:56:58,  5.02s/it]
 57%|█████▋    | 129020/225000 [260:59:46<191:56:09,  7.20s/it]
 57%|█████▋    | 129021/225000 [260:59:53<186:50:37,  7.01s/it]
 57%|█████▋    | 129022/225000 [260:59:57<166:05:29,  6.23s/it]
 57%|█████▋    | 129023/225000 [261:00:02<149:34:39,  5.61s/it]
 57%|█████▋    | 129024/225000 [261:00:06<138:19:48,  5.19s/it]
 57%|█████▋    | 129025/225000 [261:00:12<150:25:46,  5.64s/it]
 57%|█████▋    | 129026/225000 [261:00:23<191:32:22,  7.18s/it]
 57%|█████▋    | 129027/225000 [261:00:27<162:29:56,  6.10s/it]
 57%|█████▋    | 129028/225000 [261:00:30<142:41:20,  5.35s/it]
 57%|█████▋    | 129029/225000 [261:00:41<181:51:44,  6.82s/it]
 57%|█████▋    | 129030/225000 [261:00:44<155:27:52,  5.83s/it]
 57%|█████▋    | 129031/225000 [261:00:52<172:15:34,  6.46s/it]
 57%|█████▋    | 129032/225000 [261:00:56<151:35:49,  5.69s/it]
 57%|█████▋    | 129033/225000 [261:01:01<149:29:26,  5.61s/it]
 57%|█████▋    | 129034/225000 [261:01:05<134:11:38,  5.03s/it]
 57%|█████▋    | 129035/225000 [261:01:15<172:37:48,  6.48s/it]
 57%|█████▋    | 129036/225000 [261:01:19<151:08:50,  5.67s/it]
 57%|█████▋    | 129037/225000 [261:01:23<138:37:14,  5.20s/it]
 57%|█████▋    | 129038/225000 [261:01:32<170:35:58,  6.40s/it]
 57%|█████▋    | 129039/225000 [261:01:39<176:45:38,  6.63s/it]
 57%|█████▋    | 129040/225000 [261:01:43<155:53:16,  5.85s/it]
 57%|█████▋    | 129041/225000 [261:01:48<149:26:26,  5.61s/it]
 57%|█████▋    | 129042/225000 [261:01:59<190:15:02,  7.14s/it]
 57%|█████▋    | 129043/225000 [261:02:04<170:49:21,  6.41s/it]
 57%|█████▋    | 129044/225000 [261:02:08<155:28:50,  5.83s/it]
 57%|█████▋    | 129045/225000 [261:02:13<143:38:02,  5.39s/it]
 57%|█████▋    | 129046/225000 [261:02:17<135:55:42,  5.10s/it]
 57%|█████▋    | 129047/225000 [261:02:21<131:34:43,  4.94s/it]
 57%|█████▋    | 129048/225000 [261:02:29<149:35:15,  5.61s/it]
 57%|█████▋    | 129049/225000 [261:02:41<204:31:35,  7.67s/it]
 57%|█████▋    | 129050/225000 [261:02:46<179:41:48,  6.74s/it]
                                                               
{'loss': '0.8157', 'grad_norm': '0.9198', 'learning_rate': '2.082e-05', 'epoch': '0.5736', 'num_input_tokens_seen': 434084160, 'train_runtime': '9.398e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129050/225000 [261:02:46<179:41:48,  6.74s/it]
 57%|█████▋    | 129051/225000 [261:02:49<154:15:20,  5.79s/it]
 57%|█████▋    | 129052/225000 [261:02:53<140:42:04,  5.28s/it]
 57%|█████▋    | 129053/225000 [261:02:58<132:01:11,  4.95s/it]
 57%|█████▋    | 129054/225000 [261:03:09<184:01:34,  6.90s/it]
 57%|█████▋    | 129055/225000 [261:03:14<171:50:36,  6.45s/it]
 57%|█████▋    | 129056/225000 [261:03:19<160:07:25,  6.01s/it]
 57%|█████▋    | 129057/225000 [261:03:24<145:42:22,  5.47s/it]
 57%|█████▋    | 129058/225000 [261:03:28<138:04:12,  5.18s/it]
 57%|█████▋    | 129059/225000 [261:03:33<137:00:55,  5.14s/it]
 57%|█████▋    | 129060/225000 [261:03:38<133:18:47,  5.00s/it]
 57%|█████▋    | 129061/225000 [261:03:50<190:05:13,  7.13s/it]
 57%|█████▋    | 129062/225000 [261:03:56<183:15:49,  6.88s/it]
 57%|█████▋    | 129063/225000 [261:04:01<169:29:20,  6.36s/it]
 57%|█████▋    | 129064/225000 [261:04:08<170:18:30,  6.39s/it]
 57%|█████▋    | 129065/225000 [261:04:13<156:43:34,  5.88s/it]
 57%|█████▋    | 129066/225000 [261:04:17<147:21:49,  5.53s/it]
 57%|█████▋    | 129067/225000 [261:04:22<140:30:35,  5.27s/it]
 57%|█████▋    | 129068/225000 [261:04:27<136:42:23,  5.13s/it]
 57%|█████▋    | 129069/225000 [261:04:31<130:04:36,  4.88s/it]
 57%|█████▋    | 129070/225000 [261:04:36<127:10:59,  4.77s/it]
 57%|█████▋    | 129071/225000 [261:04:48<189:05:17,  7.10s/it]
 57%|█████▋    | 129072/225000 [261:04:52<164:13:32,  6.16s/it]
 57%|█████▋    | 129073/225000 [261:04:56<144:44:02,  5.43s/it]
 57%|█████▋    | 129074/225000 [261:05:06<181:57:47,  6.83s/it]
 57%|█████▋    | 129075/225000 [261:05:10<157:54:02,  5.93s/it]
 57%|█████▋    | 129076/225000 [261:05:14<141:29:11,  5.31s/it]
 57%|█████▋    | 129077/225000 [261:05:18<130:56:55,  4.91s/it]
 57%|█████▋    | 129078/225000 [261:05:31<197:14:06,  7.40s/it]
 57%|█████▋    | 129079/225000 [261:05:34<167:06:15,  6.27s/it]
 57%|█████▋    | 129080/225000 [261:05:41<171:32:28,  6.44s/it]
 57%|█████▋    | 129081/225000 [261:05:46<159:55:54,  6.00s/it]
 57%|█████▋    | 129082/225000 [261:05:56<187:24:45,  7.03s/it]
 57%|█████▋    | 129083/225000 [261:05:59<158:17:31,  5.94s/it]
 57%|█████▋    | 129084/225000 [261:06:05<158:36:10,  5.95s/it]
 57%|█████▋    | 129085/225000 [261:06:12<163:32:02,  6.14s/it]
 57%|█████▋    | 129086/225000 [261:06:19<172:01:24,  6.46s/it]
 57%|█████▋    | 129087/225000 [261:06:23<151:04:41,  5.67s/it]
 57%|█████▋    | 129088/225000 [261:06:26<136:09:47,  5.11s/it]
 57%|█████▋    | 129089/225000 [261:06:30<127:01:04,  4.77s/it]
 57%|█████▋    | 129090/225000 [261:06:34<121:02:12,  4.54s/it]
 57%|█████▋    | 129091/225000 [261:06:46<175:00:25,  6.57s/it]
 57%|█████▋    | 129092/225000 [261:06:49<151:50:57,  5.70s/it]
 57%|█████▋    | 129093/225000 [261:06:54<140:43:11,  5.28s/it]
 57%|█████▋    | 129094/225000 [261:07:03<171:47:27,  6.45s/it]
 57%|█████▋    | 129095/225000 [261:07:07<155:33:21,  5.84s/it]
 57%|█████▋    | 129096/225000 [261:07:12<144:37:20,  5.43s/it]
 57%|█████▋    | 129097/225000 [261:07:16<139:17:38,  5.23s/it]
 57%|█████▋    | 129098/225000 [261:07:22<142:58:40,  5.37s/it]
 57%|█████▋    | 129099/225000 [261:07:35<200:25:09,  7.52s/it]
 57%|█████▋    | 129100/225000 [261:07:38<165:56:23,  6.23s/it]
                                                               
{'loss': '0.8173', 'grad_norm': '1.051', 'learning_rate': '2.08e-05', 'epoch': '0.5738', 'num_input_tokens_seen': 434213432, 'train_runtime': '9.401e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129100/225000 [261:07:38<165:56:23,  6.23s/it]
 57%|█████▋    | 129101/225000 [261:07:46<178:57:02,  6.72s/it]
 57%|█████▋    | 129102/225000 [261:07:49<152:18:23,  5.72s/it]
 57%|█████▋    | 129103/225000 [261:07:57<169:33:12,  6.37s/it]
 57%|█████▋    | 129104/225000 [261:08:01<153:12:44,  5.75s/it]
 57%|█████▋    | 129105/225000 [261:08:05<139:23:37,  5.23s/it]
 57%|█████▋    | 129106/225000 [261:08:17<187:42:31,  7.05s/it]
 57%|█████▋    | 129107/225000 [261:08:21<162:13:32,  6.09s/it]
 57%|█████▋    | 129108/225000 [261:08:25<147:45:27,  5.55s/it]
 57%|█████▋    | 129109/225000 [261:08:29<139:35:22,  5.24s/it]
 57%|█████▋    | 129110/225000 [261:08:39<173:07:10,  6.50s/it]
 57%|█████▋    | 129111/225000 [261:08:46<180:23:56,  6.77s/it]
 57%|█████▋    | 129112/225000 [261:08:51<167:38:25,  6.29s/it]
 57%|█████▋    | 129113/225000 [261:08:56<150:33:15,  5.65s/it]
 57%|█████▋    | 129114/225000 [261:08:59<133:15:12,  5.00s/it]
 57%|█████▋    | 129115/225000 [261:09:10<182:59:13,  6.87s/it]
 57%|█████▋    | 129116/225000 [261:09:14<158:12:51,  5.94s/it]
 57%|█████▋    | 129117/225000 [261:09:18<140:35:46,  5.28s/it]
 57%|█████▋    | 129118/225000 [261:09:25<159:35:13,  5.99s/it]
 57%|█████▋    | 129119/225000 [261:09:33<172:50:20,  6.49s/it]
 57%|█████▋    | 129120/225000 [261:09:37<152:15:54,  5.72s/it]
 57%|█████▋    | 129121/225000 [261:09:42<147:11:32,  5.53s/it]
 57%|█████▋    | 129122/225000 [261:09:52<179:45:59,  6.75s/it]
 57%|█████▋    | 129123/225000 [261:09:56<160:18:36,  6.02s/it]
 57%|█████▋    | 129124/225000 [261:10:01<149:13:09,  5.60s/it]
 57%|█████▋    | 129125/225000 [261:10:04<135:31:46,  5.09s/it]
 57%|█████▋    | 129126/225000 [261:10:15<179:58:26,  6.76s/it]
 57%|█████▋    | 129127/225000 [261:10:20<166:23:07,  6.25s/it]
 57%|█████▋    | 129128/225000 [261:10:28<178:45:45,  6.71s/it]
 57%|█████▋    | 129129/225000 [261:10:32<153:42:33,  5.77s/it]
 57%|█████▋    | 129130/225000 [261:10:40<170:55:26,  6.42s/it]
 57%|█████▋    | 129131/225000 [261:10:44<153:01:05,  5.75s/it]
 57%|█████▋    | 129132/225000 [261:10:48<140:26:53,  5.27s/it]
 57%|█████▋    | 129133/225000 [261:10:53<136:12:33,  5.11s/it]
 57%|█████▋    | 129134/225000 [261:11:04<185:19:40,  6.96s/it]
 57%|█████▋    | 129135/225000 [261:11:07<156:26:41,  5.87s/it]
 57%|█████▋    | 129136/225000 [261:11:11<138:30:00,  5.20s/it]
 57%|█████▋    | 129137/225000 [261:11:21<181:34:37,  6.82s/it]
 57%|█████▋    | 129138/225000 [261:11:25<155:29:49,  5.84s/it]
 57%|█████▋    | 129139/225000 [261:11:33<169:40:55,  6.37s/it]
 57%|█████▋    | 129140/225000 [261:11:36<149:48:05,  5.63s/it]
 57%|█████▋    | 129141/225000 [261:11:40<135:07:31,  5.07s/it]
 57%|█████▋    | 129142/225000 [261:11:45<130:26:57,  4.90s/it]
 57%|█████▋    | 129143/225000 [261:11:49<127:43:46,  4.80s/it]
 57%|█████▋    | 129144/225000 [261:11:54<126:15:06,  4.74s/it]
 57%|█████▋    | 129145/225000 [261:12:06<186:54:04,  7.02s/it]
 57%|█████▋    | 129146/225000 [261:12:10<162:56:52,  6.12s/it]
 57%|█████▋    | 129147/225000 [261:12:15<147:45:47,  5.55s/it]
 57%|█████▋    | 129148/225000 [261:12:19<135:46:52,  5.10s/it]
 57%|█████▋    | 129149/225000 [261:12:30<188:21:45,  7.07s/it]
 57%|█████▋    | 129150/225000 [261:12:36<175:11:58,  6.58s/it]
                                                               
{'loss': '0.8305', 'grad_norm': '1.152', 'learning_rate': '2.079e-05', 'epoch': '0.574', 'num_input_tokens_seen': 434343240, 'train_runtime': '9.404e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129150/225000 [261:12:36<175:11:58,  6.58s/it]
 57%|█████▋    | 129151/225000 [261:12:40<155:24:55,  5.84s/it]
 57%|█████▋    | 129152/225000 [261:12:44<141:38:57,  5.32s/it]
 57%|█████▋    | 129153/225000 [261:12:49<140:05:35,  5.26s/it]
 57%|█████▋    | 129154/225000 [261:13:02<201:38:08,  7.57s/it]
 57%|█████▋    | 129155/225000 [261:13:06<173:32:25,  6.52s/it]
 57%|█████▋    | 129156/225000 [261:13:13<175:29:54,  6.59s/it]
 57%|█████▋    | 129157/225000 [261:13:17<159:16:30,  5.98s/it]
 57%|█████▋    | 129158/225000 [261:13:21<142:23:07,  5.35s/it]
 57%|█████▋    | 129159/225000 [261:13:26<134:18:54,  5.05s/it]
 57%|█████▋    | 129160/225000 [261:13:33<156:22:27,  5.87s/it]
 57%|█████▋    | 129161/225000 [261:13:41<170:14:58,  6.40s/it]
 57%|█████▋    | 129162/225000 [261:13:49<181:00:34,  6.80s/it]
 57%|█████▋    | 129163/225000 [261:13:52<152:35:29,  5.73s/it]
 57%|█████▋    | 129164/225000 [261:14:00<171:02:49,  6.43s/it]
 57%|█████▋    | 129165/225000 [261:14:08<182:20:57,  6.85s/it]
 57%|█████▋    | 129166/225000 [261:14:11<155:43:01,  5.85s/it]
 57%|█████▋    | 129167/225000 [261:14:20<180:31:19,  6.78s/it]
 57%|█████▋    | 129168/225000 [261:14:24<159:05:24,  5.98s/it]
 57%|█████▋    | 129169/225000 [261:14:28<143:52:04,  5.40s/it]
 57%|█████▋    | 129170/225000 [261:14:33<135:45:59,  5.10s/it]
 57%|█████▋    | 129171/225000 [261:14:40<150:22:18,  5.65s/it]
 57%|█████▋    | 129172/225000 [261:14:48<171:21:47,  6.44s/it]
 57%|█████▋    | 129173/225000 [261:14:53<155:30:15,  5.84s/it]
 57%|█████▋    | 129174/225000 [261:14:59<163:42:18,  6.15s/it]
 57%|█████▋    | 129175/225000 [261:15:07<177:22:09,  6.66s/it]
 57%|█████▋    | 129176/225000 [261:15:11<154:35:09,  5.81s/it]
 57%|█████▋    | 129177/225000 [261:15:15<136:27:38,  5.13s/it]
 57%|█████▋    | 129178/225000 [261:15:21<149:57:13,  5.63s/it]
 57%|█████▋    | 129179/225000 [261:15:30<169:50:22,  6.38s/it]
 57%|█████▋    | 129180/225000 [261:15:34<156:02:46,  5.86s/it]
 57%|█████▋    | 129181/225000 [261:15:39<145:19:15,  5.46s/it]
 57%|█████▋    | 129182/225000 [261:15:43<136:25:07,  5.13s/it]
 57%|█████▋    | 129183/225000 [261:15:47<130:45:31,  4.91s/it]
 57%|█████▋    | 129184/225000 [261:15:52<128:05:13,  4.81s/it]
 57%|█████▋    | 129185/225000 [261:16:04<184:28:37,  6.93s/it]
 57%|█████▋    | 129186/225000 [261:16:10<175:03:24,  6.58s/it]
 57%|█████▋    | 129187/225000 [261:16:13<147:36:07,  5.55s/it]
 57%|█████▋    | 129188/225000 [261:16:22<175:49:27,  6.61s/it]
 57%|█████▋    | 129189/225000 [261:16:25<150:46:32,  5.67s/it]
 57%|█████▋    | 129190/225000 [261:16:33<169:20:43,  6.36s/it]
 57%|█████▋    | 129191/225000 [261:16:38<152:17:08,  5.72s/it]
 57%|█████▋    | 129192/225000 [261:16:41<136:35:56,  5.13s/it]
 57%|█████▋    | 129193/225000 [261:16:52<179:51:36,  6.76s/it]
 57%|█████▋    | 129194/225000 [261:16:55<153:33:03,  5.77s/it]
 57%|█████▋    | 129195/225000 [261:17:03<171:29:42,  6.44s/it]
 57%|█████▋    | 129196/225000 [261:17:07<152:16:56,  5.72s/it]
 57%|█████▋    | 129197/225000 [261:17:14<158:02:53,  5.94s/it]
 57%|█████▋    | 129198/225000 [261:17:19<151:46:43,  5.70s/it]
 57%|█████▋    | 129199/225000 [261:17:23<136:14:43,  5.12s/it]
 57%|█████▋    | 129200/225000 [261:17:33<174:18:14,  6.55s/it]
                                                               
{'loss': '0.8245', 'grad_norm': '1.234', 'learning_rate': '2.077e-05', 'epoch': '0.5742', 'num_input_tokens_seen': 434472696, 'train_runtime': '9.407e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129200/225000 [261:17:33<174:18:14,  6.55s/it]
 57%|█████▋    | 129201/225000 [261:17:36<150:05:19,  5.64s/it]
 57%|█████▋    | 129202/225000 [261:17:44<163:30:52,  6.14s/it]
 57%|█████▋    | 129203/225000 [261:17:49<155:31:01,  5.84s/it]
 57%|█████▋    | 129204/225000 [261:17:52<139:25:32,  5.24s/it]
 57%|█████▋    | 129205/225000 [261:18:01<167:02:14,  6.28s/it]
 57%|█████▋    | 129206/225000 [261:18:05<151:17:54,  5.69s/it]
 57%|█████▋    | 129207/225000 [261:18:10<143:40:11,  5.40s/it]
 57%|█████▋    | 129208/225000 [261:18:15<141:53:31,  5.33s/it]
 57%|█████▋    | 129209/225000 [261:18:21<143:51:48,  5.41s/it]
 57%|█████▋    | 129210/225000 [261:18:26<144:42:36,  5.44s/it]
 57%|█████▋    | 129211/225000 [261:18:36<179:02:04,  6.73s/it]
 57%|█████▋    | 129212/225000 [261:18:41<165:08:41,  6.21s/it]
 57%|█████▋    | 129213/225000 [261:18:49<177:18:45,  6.66s/it]
 57%|█████▋    | 129214/225000 [261:18:54<164:46:20,  6.19s/it]
 57%|█████▋    | 129215/225000 [261:18:58<149:26:55,  5.62s/it]
 57%|█████▋    | 129216/225000 [261:19:04<153:02:26,  5.75s/it]
 57%|█████▋    | 129217/225000 [261:19:13<178:38:27,  6.71s/it]
 57%|█████▋    | 129218/225000 [261:19:17<154:20:56,  5.80s/it]
 57%|█████▋    | 129219/225000 [261:19:20<135:41:52,  5.10s/it]
 57%|█████▋    | 129220/225000 [261:19:30<173:58:39,  6.54s/it]
 57%|█████▋    | 129221/225000 [261:19:35<162:14:34,  6.10s/it]
 57%|█████▋    | 129222/225000 [261:19:40<152:12:10,  5.72s/it]
 57%|█████▋    | 129223/225000 [261:19:45<146:34:27,  5.51s/it]
 57%|█████▋    | 129224/225000 [261:19:50<143:57:36,  5.41s/it]
 57%|█████▋    | 129225/225000 [261:19:56<142:51:18,  5.37s/it]
 57%|█████▋    | 129226/225000 [261:20:02<147:55:51,  5.56s/it]
 57%|█████▋    | 129227/225000 [261:20:07<143:23:49,  5.39s/it]
 57%|█████▋    | 129228/225000 [261:20:12<139:43:35,  5.25s/it]
 57%|█████▋    | 129229/225000 [261:20:17<137:12:32,  5.16s/it]
 57%|█████▋    | 129230/225000 [261:20:22<136:32:14,  5.13s/it]
 57%|█████▋    | 129231/225000 [261:20:28<147:24:27,  5.54s/it]
 57%|█████▋    | 129232/225000 [261:20:40<201:12:17,  7.56s/it]
 57%|█████▋    | 129233/225000 [261:20:49<208:19:33,  7.83s/it]
 57%|█████▋    | 129234/225000 [261:20:53<180:39:36,  6.79s/it]
 57%|█████▋    | 129235/225000 [261:20:58<160:00:21,  6.01s/it]
 57%|█████▋    | 129236/225000 [261:21:08<192:37:08,  7.24s/it]
 57%|█████▋    | 129237/225000 [261:21:12<167:48:30,  6.31s/it]
 57%|█████▋    | 129238/225000 [261:21:15<145:24:27,  5.47s/it]
 57%|█████▋    | 129239/225000 [261:21:25<182:11:08,  6.85s/it]
 57%|█████▋    | 129240/225000 [261:21:29<156:01:13,  5.87s/it]
 57%|█████▋    | 129241/225000 [261:21:34<152:38:47,  5.74s/it]
 57%|█████▋    | 129242/225000 [261:21:39<145:45:08,  5.48s/it]
 57%|█████▋    | 129243/225000 [261:21:45<144:53:37,  5.45s/it]
 57%|█████▋    | 129244/225000 [261:21:50<142:02:35,  5.34s/it]
 57%|█████▋    | 129245/225000 [261:21:55<142:40:32,  5.36s/it]
 57%|█████▋    | 129246/225000 [261:22:00<141:54:27,  5.34s/it]
 57%|█████▋    | 129247/225000 [261:22:12<195:03:03,  7.33s/it]
 57%|█████▋    | 129248/225000 [261:22:18<180:08:25,  6.77s/it]
 57%|█████▋    | 129249/225000 [261:22:21<153:50:56,  5.78s/it]
 57%|█████▋    | 129250/225000 [261:22:29<171:05:40,  6.43s/it]
                                                               
{'loss': '0.8272', 'grad_norm': '1.036', 'learning_rate': '2.075e-05', 'epoch': '0.5744', 'num_input_tokens_seen': 434603104, 'train_runtime': '9.409e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129250/225000 [261:22:29<171:05:40,  6.43s/it]
 57%|█████▋    | 129251/225000 [261:22:32<144:54:30,  5.45s/it]
 57%|█████▋    | 129252/225000 [261:22:41<167:28:04,  6.30s/it]
 57%|█████▋    | 129253/225000 [261:22:45<152:44:51,  5.74s/it]
 57%|█████▋    | 129254/225000 [261:22:53<168:39:25,  6.34s/it]
 57%|█████▋    | 129255/225000 [261:22:56<145:35:17,  5.47s/it]
 57%|█████▋    | 129256/225000 [261:23:04<161:45:17,  6.08s/it]
 57%|█████▋    | 129257/225000 [261:23:08<150:18:50,  5.65s/it]
 57%|█████▋    | 129258/225000 [261:23:17<172:42:22,  6.49s/it]
 57%|█████▋    | 129259/225000 [261:23:20<147:29:38,  5.55s/it]
 57%|█████▋    | 129260/225000 [261:23:29<174:05:47,  6.55s/it]
 57%|█████▋    | 129261/225000 [261:23:33<151:03:52,  5.68s/it]
 57%|█████▋    | 129262/225000 [261:23:37<135:26:20,  5.09s/it]
 57%|█████▋    | 129263/225000 [261:23:42<141:50:17,  5.33s/it]
 57%|█████▋    | 129264/225000 [261:23:52<176:39:32,  6.64s/it]
 57%|█████▋    | 129265/225000 [261:23:58<167:36:31,  6.30s/it]
 57%|█████▋    | 129266/225000 [261:24:03<159:22:19,  5.99s/it]
 57%|█████▋    | 129267/225000 [261:24:12<182:39:25,  6.87s/it]
 57%|█████▋    | 129268/225000 [261:24:15<151:52:31,  5.71s/it]
 57%|█████▋    | 129269/225000 [261:24:22<163:00:16,  6.13s/it]
 57%|█████▋    | 129270/225000 [261:24:25<139:34:36,  5.25s/it]
 57%|█████▋    | 129271/225000 [261:24:35<176:54:16,  6.65s/it]
 57%|█████▋    | 129272/225000 [261:24:39<159:16:42,  5.99s/it]
 57%|█████▋    | 129273/225000 [261:24:43<143:17:25,  5.39s/it]
 57%|█████▋    | 129274/225000 [261:24:48<136:36:18,  5.14s/it]
 57%|█████▋    | 129275/225000 [261:24:53<132:24:35,  4.98s/it]
 57%|█████▋    | 129276/225000 [261:24:57<128:40:42,  4.84s/it]
 57%|█████▋    | 129277/225000 [261:25:08<176:55:33,  6.65s/it]
 57%|█████▋    | 129278/225000 [261:25:14<168:22:23,  6.33s/it]
 57%|█████▋    | 129279/225000 [261:25:22<183:37:09,  6.91s/it]
 57%|█████▋    | 129280/225000 [261:25:26<164:06:27,  6.17s/it]
 57%|█████▋    | 129281/225000 [261:25:30<147:59:31,  5.57s/it]
 57%|█████▋    | 129282/225000 [261:25:35<139:10:39,  5.23s/it]
 57%|█████▋    | 129283/225000 [261:25:39<130:57:08,  4.93s/it]
 57%|█████▋    | 129284/225000 [261:25:46<147:20:30,  5.54s/it]
 57%|█████▋    | 129285/225000 [261:25:54<167:17:31,  6.29s/it]
 57%|█████▋    | 129286/225000 [261:25:58<146:26:38,  5.51s/it]
 57%|█████▋    | 129287/225000 [261:26:07<177:37:46,  6.68s/it]
 57%|█████▋    | 129288/225000 [261:26:13<169:17:55,  6.37s/it]
 57%|█████▋    | 129289/225000 [261:26:20<172:23:38,  6.48s/it]
 57%|█████▋    | 129290/225000 [261:26:24<155:18:33,  5.84s/it]
 57%|█████▋    | 129291/225000 [261:26:29<147:29:45,  5.55s/it]
 57%|█████▋    | 129292/225000 [261:26:33<138:45:44,  5.22s/it]
 57%|█████▋    | 129293/225000 [261:26:38<136:30:42,  5.13s/it]
 57%|█████▋    | 129294/225000 [261:26:43<134:46:20,  5.07s/it]
 57%|█████▋    | 129295/225000 [261:26:55<192:09:05,  7.23s/it]
 57%|█████▋    | 129296/225000 [261:27:02<190:20:29,  7.16s/it]
 57%|█████▋    | 129297/225000 [261:27:08<174:34:37,  6.57s/it]
 57%|█████▋    | 129298/225000 [261:27:12<159:48:54,  6.01s/it]
 57%|█████▋    | 129299/225000 [261:27:17<152:25:16,  5.73s/it]
 57%|█████▋    | 129300/225000 [261:27:22<142:09:02,  5.35s/it]
                                                               
{'loss': '0.8275', 'grad_norm': '1.059', 'learning_rate': '2.073e-05', 'epoch': '0.5747', 'num_input_tokens_seen': 434732128, 'train_runtime': '9.412e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129300/225000 [261:27:22<142:09:02,  5.35s/it]
 57%|█████▋    | 129301/225000 [261:27:27<139:29:28,  5.25s/it]
 57%|█████▋    | 129302/225000 [261:27:35<161:35:59,  6.08s/it]
 57%|█████▋    | 129303/225000 [261:27:46<203:56:28,  7.67s/it]
 57%|█████▋    | 129304/225000 [261:27:50<172:56:46,  6.51s/it]
 57%|█████▋    | 129305/225000 [261:27:54<151:45:10,  5.71s/it]
 57%|█████▋    | 129306/225000 [261:27:58<137:36:33,  5.18s/it]
 57%|█████▋    | 129307/225000 [261:28:10<196:19:11,  7.39s/it]
 57%|█████▋    | 129308/225000 [261:28:15<170:22:01,  6.41s/it]
 57%|█████▋    | 129309/225000 [261:28:20<164:59:13,  6.21s/it]
 57%|█████▋    | 129310/225000 [261:28:26<159:20:15,  5.99s/it]
 57%|█████▋    | 129311/225000 [261:28:32<164:18:08,  6.18s/it]
 57%|█████▋    | 129312/225000 [261:28:39<164:49:52,  6.20s/it]
 57%|█████▋    | 129313/225000 [261:28:44<157:42:48,  5.93s/it]
 57%|█████▋    | 129314/225000 [261:28:51<167:24:44,  6.30s/it]
 57%|█████▋    | 129315/225000 [261:28:55<150:12:36,  5.65s/it]
 57%|█████▋    | 129316/225000 [261:28:59<134:39:16,  5.07s/it]
 57%|█████▋    | 129317/225000 [261:29:04<133:11:32,  5.01s/it]
 57%|█████▋    | 129318/225000 [261:29:12<161:21:35,  6.07s/it]
 57%|█████▋    | 129319/225000 [261:29:20<172:32:46,  6.49s/it]
 57%|█████▋    | 129320/225000 [261:29:23<148:47:22,  5.60s/it]
 57%|█████▋    | 129321/225000 [261:29:31<165:19:22,  6.22s/it]
 57%|█████▋    | 129322/225000 [261:29:36<153:24:04,  5.77s/it]
 57%|█████▋    | 129323/225000 [261:29:40<144:23:40,  5.43s/it]
 57%|█████▋    | 129324/225000 [261:29:45<136:03:26,  5.12s/it]
 57%|█████▋    | 129325/225000 [261:29:49<132:03:31,  4.97s/it]
 57%|█████▋    | 129326/225000 [261:29:54<132:08:52,  4.97s/it]
 57%|█████▋    | 129327/225000 [261:30:03<157:45:59,  5.94s/it]
 57%|█████▋    | 129328/225000 [261:30:12<182:45:23,  6.88s/it]
 57%|█████▋    | 129329/225000 [261:30:19<187:14:57,  7.05s/it]
 57%|█████▋    | 129330/225000 [261:30:23<159:55:52,  6.02s/it]
 57%|█████▋    | 129331/225000 [261:30:26<140:04:11,  5.27s/it]
 57%|█████▋    | 129332/225000 [261:30:36<176:32:36,  6.64s/it]
 57%|█████▋    | 129333/225000 [261:30:40<155:35:51,  5.86s/it]
 57%|█████▋    | 129334/225000 [261:30:45<149:47:43,  5.64s/it]
 57%|█████▋    | 129335/225000 [261:30:53<170:14:22,  6.41s/it]
 57%|█████▋    | 129336/225000 [261:30:59<163:04:53,  6.14s/it]
 57%|█████▋    | 129337/225000 [261:31:04<154:39:36,  5.82s/it]
 57%|█████▋    | 129338/225000 [261:31:09<150:43:08,  5.67s/it]
 57%|█████▋    | 129339/225000 [261:31:15<147:02:21,  5.53s/it]
 57%|█████▋    | 129340/225000 [261:31:24<175:33:34,  6.61s/it]
 57%|█████▋    | 129341/225000 [261:31:28<158:10:04,  5.95s/it]
 57%|█████▋    | 129342/225000 [261:31:32<144:27:04,  5.44s/it]
 57%|█████▋    | 129343/225000 [261:31:37<137:03:58,  5.16s/it]
 57%|█████▋    | 129344/225000 [261:31:49<190:02:24,  7.15s/it]
 57%|█████▋    | 129345/225000 [261:31:54<177:06:22,  6.67s/it]
 57%|█████▋    | 129346/225000 [261:31:59<162:26:33,  6.11s/it]
 57%|█████▋    | 129347/225000 [261:32:04<152:15:22,  5.73s/it]
 57%|█████▋    | 129348/225000 [261:32:10<159:28:31,  6.00s/it]
 57%|█████▋    | 129349/225000 [261:32:15<149:22:31,  5.62s/it]
 57%|█████▋    | 129350/225000 [261:32:21<147:12:35,  5.54s/it]
                                                               
{'loss': '0.8144', 'grad_norm': '1.027', 'learning_rate': '2.071e-05', 'epoch': '0.5749', 'num_input_tokens_seen': 434861264, 'train_runtime': '9.415e+05', 'train_tokens_per_second': '461.9'}

 57%|█████▋    | 129350/225000 [261:32:21<147:12:35,  5.54s/it]
 57%|█████▋    | 129351/225000 [261:32:25<141:22:06,  5.32s/it]
 57%|█████▋    | 129352/225000 [261:32:30<138:39:55,  5.22s/it]
 57%|█████▋    | 129353/225000 [261:32:35<136:19:59,  5.13s/it]
 57%|█████▋    | 129354/225000 [261:32:40<132:53:30,  5.00s/it]
 57%|█████▋    | 129355/225000 [261:32:52<190:10:49,  7.16s/it]
 57%|█████▋    | 129356/225000 [261:32:59<186:23:30,  7.02s/it]
 57%|█████▋    | 129357/225000 [261:33:03<161:23:55,  6.08s/it]
 57%|█████▋    | 129358/225000 [261:33:07<145:41:48,  5.48s/it]
 57%|█████▋    | 129359/225000 [261:33:15<169:47:32,  6.39s/it]
 57%|█████▋    | 129360/225000 [261:33:22<173:26:43,  6.53s/it]
 57%|█████▋    | 129361/225000 [261:33:27<159:19:16,  6.00s/it]
 57%|█████▋    | 129362/225000 [261:33:32<150:59:00,  5.68s/it]
 57%|█████▋    | 129363/225000 [261:33:37<146:02:42,  5.50s/it]
 57%|█████▋    | 129364/225000 [261:33:42<141:10:38,  5.31s/it]
 57%|█████▋    | 129365/225000 [261:33:46<135:21:38,  5.10s/it]
 57%|█████▋    | 129366/225000 [261:33:54<159:22:13,  6.00s/it]
 57%|█████▋    | 129367/225000 [261:34:04<188:56:13,  7.11s/it]
 57%|█████▋    | 129368/225000 [261:34:09<173:35:53,  6.53s/it]
 57%|█████▋    | 129369/225000 [261:34:16<172:56:24,  6.51s/it]
 57%|█████▋    | 129370/225000 [261:34:19<149:25:12,  5.62s/it]
 57%|█████▋    | 129371/225000 [261:34:29<180:49:09,  6.81s/it]
 57%|█████▋    | 129372/225000 [261:34:32<153:30:14,  5.78s/it]
 57%|█████▋    | 129373/225000 [261:34:41<178:46:47,  6.73s/it]
 57%|█████▋    | 129374/225000 [261:34:46<162:06:15,  6.10s/it]
 57%|█████▊    | 129375/225000 [261:34:50<147:39:24,  5.56s/it]
 58%|█████▊    | 129376/225000 [261:34:55<141:01:41,  5.31s/it]
 58%|█████▊    | 129377/225000 [261:35:03<166:40:30,  6.27s/it]
 58%|█████▊    | 129378/225000 [261:35:08<156:01:35,  5.87s/it]
 58%|█████▊    | 129379/225000 [261:35:13<146:33:57,  5.52s/it]
 58%|█████▊    | 129380/225000 [261:35:18<140:38:24,  5.29s/it]
 58%|█████▊    | 129381/225000 [261:35:23<141:16:03,  5.32s/it]
 58%|█████▊    | 129382/225000 [261:35:28<140:13:34,  5.28s/it]
 58%|█████▊    | 129383/225000 [261:35:34<140:10:21,  5.28s/it]
 58%|█████▊    | 129384/225000 [261:35:38<136:10:16,  5.13s/it]
 58%|█████▊    | 129385/225000 [261:35:44<136:03:40,  5.12s/it]
 58%|█████▊    | 129386/225000 [261:35:53<167:51:41,  6.32s/it]
 58%|█████▊    | 129387/225000 [261:35:59<166:34:01,  6.27s/it]
 58%|█████▊    | 129388/225000 [261:36:04<158:03:29,  5.95s/it]
 58%|█████▊    | 129389/225000 [261:36:09<153:39:47,  5.79s/it]
 58%|█████▊    | 129390/225000 [261:36:15<149:14:17,  5.62s/it]
 58%|█████▊    | 129391/225000 [261:36:20<146:24:35,  5.51s/it]
 58%|█████▊    | 129392/225000 [261:36:25<144:31:44,  5.44s/it]
 58%|█████▊    | 129393/225000 [261:36:30<142:41:27,  5.37s/it]
 58%|█████▊    | 129394/225000 [261:36:42<194:24:17,  7.32s/it]
 58%|█████▊    | 129395/225000 [261:36:50<197:24:31,  7.43s/it]
 58%|█████▊    | 129396/225000 [261:36:53<163:04:06,  6.14s/it]
 58%|█████▊    | 129397/225000 [261:37:03<192:26:08,  7.25s/it]
 58%|█████▊    | 129398/225000 [261:37:06<162:03:50,  6.10s/it]
 58%|█████▊    | 129399/225000 [261:37:10<141:51:28,  5.34s/it]
 58%|█████▊    | 129400/225000 [261:37:19<167:57:46,  6.32s/it]
                                                               
{'loss': '0.8182', 'grad_norm': '1.082', 'learning_rate': '2.07e-05', 'epoch': '0.5751', 'num_input_tokens_seen': 434989688, 'train_runtime': '9.418e+05', 'train_tokens_per_second': '461.9'}

 58%|█████▊    | 129400/225000 [261:37:19<167:57:46,  6.32s/it]
 58%|█████▊    | 129401/225000 [261:37:24<164:27:19,  6.19s/it]
 58%|█████▊    | 129402/225000 [261:37:28<141:57:42,  5.35s/it]
 58%|█████▊    | 129403/225000 [261:37:37<175:34:28,  6.61s/it]
 58%|█████▊    | 129404/225000 [261:37:42<156:40:41,  5.90s/it]
 58%|█████▊    | 129405/225000 [261:37:46<142:42:59,  5.37s/it]
 58%|█████▊    | 129406/225000 [261:37:50<132:16:38,  4.98s/it]
 58%|█████▊    | 129407/225000 [261:37:55<132:42:22,  5.00s/it]
 58%|█████▊    | 129408/225000 [261:38:06<179:29:05,  6.76s/it]
 58%|█████▊    | 129409/225000 [261:38:10<155:33:19,  5.86s/it]
 58%|█████▊    | 129410/225000 [261:38:18<176:03:56,  6.63s/it]
 58%|█████▊    | 129411/225000 [261:38:23<161:35:42,  6.09s/it]
 58%|█████▊    | 129412/225000 [261:38:27<147:32:21,  5.56s/it]
 58%|█████▊    | 129413/225000 [261:38:32<139:24:15,  5.25s/it]
 58%|█████▊    | 129414/225000 [261:38:37<139:32:19,  5.26s/it]
 58%|█████▊    | 129415/225000 [261:38:44<154:51:22,  5.83s/it]
 58%|█████▊    | 129416/225000 [261:38:55<197:56:18,  7.45s/it]
 58%|█████▊    | 129417/225000 [261:38:59<165:46:29,  6.24s/it]
 58%|█████▊    | 129418/225000 [261:39:07<181:21:28,  6.83s/it]
 58%|█████▊    | 129419/225000 [261:39:12<165:30:45,  6.23s/it]
 58%|█████▊    | 129420/225000 [261:39:16<149:23:59,  5.63s/it]
 58%|█████▊    | 129421/225000 [261:39:21<141:11:52,  5.32s/it]
 58%|█████▊    | 129422/225000 [261:39:25<134:10:49,  5.05s/it]
 58%|█████▊    | 129423/225000 [261:39:29<127:05:07,  4.79s/it]
 58%|█████▊    | 129424/225000 [261:39:40<176:42:52,  6.66s/it]
 58%|█████▊    | 129425/225000 [261:39:45<162:18:02,  6.11s/it]
 58%|█████▊    | 129426/225000 [261:39:50<156:30:52,  5.90s/it]
 58%|█████▊    | 129427/225000 [261:39:59<176:45:27,  6.66s/it]
 58%|█████▊    | 129428/225000 [261:40:02<151:37:13,  5.71s/it]
 58%|█████▊    | 129429/225000 [261:40:06<136:37:39,  5.15s/it]
 58%|█████▊    | 129430/225000 [261:40:15<162:11:56,  6.11s/it]
 58%|█████▊    | 129431/225000 [261:40:21<161:49:41,  6.10s/it]
 58%|█████▊    | 129432/225000 [261:40:28<168:03:09,  6.33s/it]
 58%|█████▊    | 129433/225000 [261:40:34<165:35:23,  6.24s/it]
 58%|█████▊    | 129434/225000 [261:40:40<170:06:47,  6.41s/it]
 58%|█████▊    | 129435/225000 [261:40:44<151:25:44,  5.70s/it]
 58%|█████▊    | 129436/225000 [261:40:48<135:22:47,  5.10s/it]
 58%|█████▊    | 129437/225000 [261:40:55<152:42:47,  5.75s/it]
 58%|█████▊    | 129438/225000 [261:41:03<166:24:32,  6.27s/it]
 58%|█████▊    | 129439/225000 [261:41:07<148:21:40,  5.59s/it]
 58%|█████▊    | 129440/225000 [261:41:11<137:17:29,  5.17s/it]
 58%|█████▊    | 129441/225000 [261:41:17<144:35:15,  5.45s/it]
 58%|█████▊    | 129442/225000 [261:41:28<185:55:52,  7.00s/it]
 58%|█████▊    | 129443/225000 [261:41:31<157:47:05,  5.94s/it]
 58%|█████▊    | 129444/225000 [261:41:40<177:32:01,  6.69s/it]
 58%|█████▊    | 129445/225000 [261:41:46<175:02:40,  6.59s/it]
 58%|█████▊    | 129446/225000 [261:41:49<149:21:10,  5.63s/it]
 58%|█████▊    | 129447/225000 [261:41:54<142:30:23,  5.37s/it]
 58%|█████▊    | 129448/225000 [261:42:00<143:38:37,  5.41s/it]
 58%|█████▊    | 129449/225000 [261:42:04<138:48:40,  5.23s/it]
 58%|█████▊    | 129450/225000 [261:42:10<141:20:02,  5.32s/it]
                                                               
{'loss': '0.8092', 'grad_norm': '0.9447', 'learning_rate': '2.068e-05', 'epoch': '0.5753', 'num_input_tokens_seen': 435118240, 'train_runtime': '9.421e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129450/225000 [261:42:10<141:20:02,  5.32s/it]
 58%|█████▊    | 129451/225000 [261:42:15<142:10:32,  5.36s/it]
 58%|█████▊    | 129452/225000 [261:42:21<141:08:45,  5.32s/it]
 58%|█████▊    | 129453/225000 [261:42:26<141:00:43,  5.31s/it]
 58%|█████▊    | 129454/225000 [261:42:31<138:10:15,  5.21s/it]
 58%|█████▊    | 129455/225000 [261:42:36<139:09:38,  5.24s/it]
 58%|█████▊    | 129456/225000 [261:42:44<160:23:55,  6.04s/it]
 58%|█████▊    | 129457/225000 [261:42:56<206:30:18,  7.78s/it]
 58%|█████▊    | 129458/225000 [261:43:00<175:21:49,  6.61s/it]
 58%|█████▊    | 129459/225000 [261:43:04<153:59:55,  5.80s/it]
 58%|█████▊    | 129460/225000 [261:43:10<157:01:59,  5.92s/it]
 58%|█████▊    | 129461/225000 [261:43:19<181:58:54,  6.86s/it]
 58%|█████▊    | 129462/225000 [261:43:23<154:50:32,  5.83s/it]
 58%|█████▊    | 129463/225000 [261:43:31<178:13:58,  6.72s/it]
 58%|█████▊    | 129464/225000 [261:43:36<163:21:45,  6.16s/it]
 58%|█████▊    | 129465/225000 [261:43:40<147:37:51,  5.56s/it]
 58%|█████▊    | 129466/225000 [261:43:45<137:30:34,  5.18s/it]
 58%|█████▊    | 129467/225000 [261:43:49<128:18:52,  4.84s/it]
 58%|█████▊    | 129468/225000 [261:43:53<124:32:08,  4.69s/it]
 58%|█████▊    | 129469/225000 [261:44:04<176:38:37,  6.66s/it]
 58%|█████▊    | 129470/225000 [261:44:11<177:33:21,  6.69s/it]
 58%|█████▊    | 129471/225000 [261:44:18<182:33:17,  6.88s/it]
 58%|█████▊    | 129472/225000 [261:44:22<154:42:46,  5.83s/it]
 58%|█████▊    | 129473/225000 [261:44:30<172:13:05,  6.49s/it]
 58%|█████▊    | 129474/225000 [261:44:33<149:51:34,  5.65s/it]
 58%|█████▊    | 129475/225000 [261:44:38<138:19:57,  5.21s/it]
 58%|█████▊    | 129476/225000 [261:44:42<134:37:38,  5.07s/it]
 58%|█████▊    | 129477/225000 [261:44:51<163:28:45,  6.16s/it]
 58%|█████▊    | 129478/225000 [261:45:01<192:12:17,  7.24s/it]
 58%|█████▊    | 129479/225000 [261:45:05<167:19:11,  6.31s/it]
 58%|█████▊    | 129480/225000 [261:45:09<150:35:01,  5.68s/it]
 58%|█████▊    | 129481/225000 [261:45:19<180:04:12,  6.79s/it]
 58%|█████▊    | 129482/225000 [261:45:23<162:52:42,  6.14s/it]
 58%|█████▊    | 129483/225000 [261:45:28<149:31:35,  5.64s/it]
 58%|█████▊    | 129484/225000 [261:45:33<145:07:41,  5.47s/it]
 58%|█████▊    | 129485/225000 [261:45:37<138:28:42,  5.22s/it]
 58%|█████▊    | 129486/225000 [261:45:42<132:10:02,  4.98s/it]
 58%|█████▊    | 129487/225000 [261:45:49<152:13:02,  5.74s/it]
 58%|█████▊    | 129488/225000 [261:46:00<189:58:13,  7.16s/it]
 58%|█████▊    | 129489/225000 [261:46:03<159:48:57,  6.02s/it]
 58%|█████▊    | 129490/225000 [261:46:12<179:00:40,  6.75s/it]
 58%|█████▊    | 129491/225000 [261:46:15<155:17:00,  5.85s/it]
 58%|█████▊    | 129492/225000 [261:46:19<137:02:42,  5.17s/it]
 58%|█████▊    | 129493/225000 [261:46:30<186:13:10,  7.02s/it]
 58%|█████▊    | 129494/225000 [261:46:33<154:41:24,  5.83s/it]
 58%|█████▊    | 129495/225000 [261:46:41<165:51:47,  6.25s/it]
 58%|█████▊    | 129496/225000 [261:46:44<145:29:07,  5.48s/it]
 58%|█████▊    | 129497/225000 [261:46:52<165:25:49,  6.24s/it]
 58%|█████▊    | 129498/225000 [261:46:59<169:41:45,  6.40s/it]
 58%|█████▊    | 129499/225000 [261:47:02<146:01:19,  5.50s/it]
 58%|█████▊    | 129500/225000 [261:47:09<158:33:39,  5.98s/it]
                                                               
{'loss': '0.8274', 'grad_norm': '1.046', 'learning_rate': '2.066e-05', 'epoch': '0.5756', 'num_input_tokens_seen': 435249600, 'train_runtime': '9.424e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129500/225000 [261:47:09<158:33:39,  5.98s/it]
 58%|█████▊    | 129501/225000 [261:47:17<168:28:09,  6.35s/it]
 58%|█████▊    | 129502/225000 [261:47:21<153:27:27,  5.78s/it]
 58%|█████▊    | 129503/225000 [261:47:25<140:53:02,  5.31s/it]
 58%|█████▊    | 129504/225000 [261:47:30<132:52:15,  5.01s/it]
 58%|█████▊    | 129505/225000 [261:47:40<173:17:10,  6.53s/it]
 58%|█████▊    | 129506/225000 [261:47:45<164:20:22,  6.20s/it]
 58%|█████▊    | 129507/225000 [261:47:51<158:57:23,  5.99s/it]
 58%|█████▊    | 129508/225000 [261:48:00<182:10:19,  6.87s/it]
 58%|█████▊    | 129509/225000 [261:48:03<157:09:39,  5.92s/it]
 58%|█████▊    | 129510/225000 [261:48:07<140:37:32,  5.30s/it]
 58%|█████▊    | 129511/225000 [261:48:17<177:05:00,  6.68s/it]
 58%|█████▊    | 129512/225000 [261:48:21<154:56:48,  5.84s/it]
 58%|█████▊    | 129513/225000 [261:48:25<139:45:20,  5.27s/it]
 58%|█████▊    | 129514/225000 [261:48:29<129:28:57,  4.88s/it]
 58%|█████▊    | 129515/225000 [261:48:39<169:59:16,  6.41s/it]
 58%|█████▊    | 129516/225000 [261:48:46<179:33:47,  6.77s/it]
 58%|█████▊    | 129517/225000 [261:48:50<156:02:03,  5.88s/it]
 58%|█████▊    | 129518/225000 [261:48:59<179:47:44,  6.78s/it]
 58%|█████▊    | 129519/225000 [261:49:02<152:29:03,  5.75s/it]
 58%|█████▊    | 129520/225000 [261:49:07<146:28:19,  5.52s/it]
 58%|█████▊    | 129521/225000 [261:49:13<143:37:11,  5.42s/it]
 58%|█████▊    | 129522/225000 [261:49:18<142:02:47,  5.36s/it]
 58%|█████▊    | 129523/225000 [261:49:23<136:32:00,  5.15s/it]
 58%|█████▊    | 129524/225000 [261:49:28<135:51:30,  5.12s/it]
 58%|█████▊    | 129525/225000 [261:49:39<186:25:22,  7.03s/it]
 58%|█████▊    | 129526/225000 [261:49:47<196:03:02,  7.39s/it]
 58%|█████▊    | 129527/225000 [261:49:52<170:52:47,  6.44s/it]
 58%|█████▊    | 129528/225000 [261:49:56<152:18:46,  5.74s/it]
 58%|█████▊    | 129529/225000 [261:50:03<165:25:42,  6.24s/it]
 58%|█████▊    | 129530/225000 [261:50:11<178:34:19,  6.73s/it]
 58%|█████▊    | 129531/225000 [261:50:15<158:38:28,  5.98s/it]
 58%|█████▊    | 129532/225000 [261:50:20<149:45:26,  5.65s/it]
 58%|█████▊    | 129533/225000 [261:50:25<141:38:19,  5.34s/it]
 58%|█████▊    | 129534/225000 [261:50:37<196:40:04,  7.42s/it]
 58%|█████▊    | 129535/225000 [261:50:42<174:29:24,  6.58s/it]
 58%|█████▊    | 129536/225000 [261:50:47<162:29:16,  6.13s/it]
 58%|█████▊    | 129537/225000 [261:50:53<166:56:40,  6.30s/it]
 58%|█████▊    | 129538/225000 [261:50:58<154:10:08,  5.81s/it]
 58%|█████▊    | 129539/225000 [261:51:03<145:31:15,  5.49s/it]
 58%|█████▊    | 129540/225000 [261:51:07<138:28:00,  5.22s/it]
 58%|█████▊    | 129541/225000 [261:51:12<133:54:35,  5.05s/it]
 58%|█████▊    | 129542/225000 [261:51:17<131:49:59,  4.97s/it]
 58%|█████▊    | 129543/225000 [261:51:28<181:01:50,  6.83s/it]
 58%|█████▊    | 129544/225000 [261:51:37<196:11:32,  7.40s/it]
 58%|█████▊    | 129545/225000 [261:51:40<162:39:59,  6.13s/it]
 58%|█████▊    | 129546/225000 [261:51:48<181:43:43,  6.85s/it]
 58%|█████▊    | 129547/225000 [261:51:52<155:23:35,  5.86s/it]
 58%|█████▊    | 129548/225000 [261:52:00<174:52:01,  6.60s/it]
 58%|█████▊    | 129549/225000 [261:52:05<158:32:37,  5.98s/it]
 58%|█████▊    | 129550/225000 [261:52:09<146:08:53,  5.51s/it]
                                                               
{'loss': '0.8216', 'grad_norm': '1.06', 'learning_rate': '2.064e-05', 'epoch': '0.5758', 'num_input_tokens_seen': 435380056, 'train_runtime': '9.427e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129550/225000 [261:52:09<146:08:53,  5.51s/it]
 58%|█████▊    | 129551/225000 [261:52:14<140:45:30,  5.31s/it]
 58%|█████▊    | 129552/225000 [261:52:19<135:23:05,  5.11s/it]
 58%|█████▊    | 129553/225000 [261:52:24<134:28:52,  5.07s/it]
 58%|█████▊    | 129554/225000 [261:52:28<129:56:42,  4.90s/it]
 58%|█████▊    | 129555/225000 [261:52:40<187:10:21,  7.06s/it]
 58%|█████▊    | 129556/225000 [261:52:46<176:43:05,  6.67s/it]
 58%|█████▊    | 129557/225000 [261:52:52<170:50:57,  6.44s/it]
 58%|█████▊    | 129558/225000 [261:52:59<173:54:47,  6.56s/it]
 58%|█████▊    | 129559/225000 [261:53:03<155:15:42,  5.86s/it]
 58%|█████▊    | 129560/225000 [261:53:13<185:21:58,  6.99s/it]
 58%|█████▊    | 129561/225000 [261:53:16<155:36:08,  5.87s/it]
 58%|█████▊    | 129562/225000 [261:53:23<166:33:28,  6.28s/it]
 58%|█████▊    | 129563/225000 [261:53:28<154:55:13,  5.84s/it]
 58%|█████▊    | 129564/225000 [261:53:32<143:58:21,  5.43s/it]
 58%|█████▊    | 129565/225000 [261:53:37<136:15:15,  5.14s/it]
 58%|█████▊    | 129566/225000 [261:53:47<173:33:08,  6.55s/it]
 58%|█████▊    | 129567/225000 [261:53:51<152:45:34,  5.76s/it]
 58%|█████▊    | 129568/225000 [261:53:56<152:22:14,  5.75s/it]
 58%|█████▊    | 129569/225000 [261:54:06<180:28:55,  6.81s/it]
 58%|█████▊    | 129570/225000 [261:54:09<154:08:45,  5.82s/it]
 58%|█████▊    | 129571/225000 [261:54:17<171:47:56,  6.48s/it]
 58%|█████▊    | 129572/225000 [261:54:21<148:40:31,  5.61s/it]
 58%|█████▊    | 129573/225000 [261:54:30<176:27:20,  6.66s/it]
 58%|█████▊    | 129574/225000 [261:54:33<151:56:30,  5.73s/it]
 58%|█████▊    | 129575/225000 [261:54:42<173:06:27,  6.53s/it]
 58%|█████▊    | 129576/225000 [261:54:46<158:37:26,  5.98s/it]
 58%|█████▊    | 129577/225000 [261:54:51<148:47:47,  5.61s/it]
 58%|█████▊    | 129578/225000 [261:54:56<143:45:42,  5.42s/it]
 58%|█████▊    | 129579/225000 [261:55:01<137:39:08,  5.19s/it]
 58%|█████▊    | 129580/225000 [261:55:06<133:48:50,  5.05s/it]
 58%|█████▊    | 129581/225000 [261:55:10<131:24:00,  4.96s/it]
 58%|█████▊    | 129582/225000 [261:55:20<168:16:42,  6.35s/it]
 58%|█████▊    | 129583/225000 [261:55:30<198:43:26,  7.50s/it]
 58%|█████▊    | 129584/225000 [261:55:34<168:21:51,  6.35s/it]
 58%|█████▊    | 129585/225000 [261:55:38<150:17:03,  5.67s/it]
 58%|█████▊    | 129586/225000 [261:55:48<186:50:36,  7.05s/it]
 58%|█████▊    | 129587/225000 [261:55:51<157:31:21,  5.94s/it]
 58%|█████▊    | 129588/225000 [261:56:00<177:41:32,  6.70s/it]
 58%|█████▊    | 129589/225000 [261:56:04<155:44:24,  5.88s/it]
 58%|█████▊    | 129590/225000 [261:56:11<168:34:24,  6.36s/it]
 58%|█████▊    | 129591/225000 [261:56:16<152:39:32,  5.76s/it]
 58%|█████▊    | 129592/225000 [261:56:23<163:36:39,  6.17s/it]
 58%|█████▊    | 129593/225000 [261:56:26<143:02:28,  5.40s/it]
 58%|█████▊    | 129594/225000 [261:56:37<182:22:47,  6.88s/it]
 58%|█████▊    | 129595/225000 [261:56:41<157:26:47,  5.94s/it]
 58%|█████▊    | 129596/225000 [261:56:45<149:25:19,  5.64s/it]
 58%|█████▊    | 129597/225000 [261:56:51<145:22:38,  5.49s/it]
 58%|█████▊    | 129598/225000 [261:56:56<145:45:33,  5.50s/it]
 58%|█████▊    | 129599/225000 [261:57:01<144:19:07,  5.45s/it]
 58%|█████▊    | 129600/225000 [261:57:06<137:33:15,  5.19s/it]
                                                               
{'loss': '0.8166', 'grad_norm': '1.027', 'learning_rate': '2.062e-05', 'epoch': '0.576', 'num_input_tokens_seen': 435509720, 'train_runtime': '9.43e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129600/225000 [261:57:06<137:33:15,  5.19s/it]
 58%|█████▊    | 129601/225000 [261:57:11<137:24:29,  5.19s/it]
 58%|█████▊    | 129602/225000 [261:57:16<136:44:56,  5.16s/it]
 58%|█████▊    | 129603/225000 [261:57:21<135:25:45,  5.11s/it]
 58%|█████▊    | 129604/225000 [261:57:30<166:45:50,  6.29s/it]
 58%|█████▊    | 129605/225000 [261:57:41<201:08:23,  7.59s/it]
 58%|█████▊    | 129606/225000 [261:57:44<167:30:32,  6.32s/it]
 58%|█████▊    | 129607/225000 [261:57:50<160:52:06,  6.07s/it]
 58%|█████▊    | 129608/225000 [261:57:55<153:11:41,  5.78s/it]
 58%|█████▊    | 129609/225000 [261:58:00<148:52:12,  5.62s/it]
 58%|█████▊    | 129610/225000 [261:58:06<147:01:01,  5.55s/it]
 58%|█████▊    | 129611/225000 [261:58:11<144:14:09,  5.44s/it]
 58%|█████▊    | 129612/225000 [261:58:16<142:30:04,  5.38s/it]
 58%|█████▊    | 129613/225000 [261:58:21<140:33:16,  5.30s/it]
 58%|█████▊    | 129614/225000 [261:58:26<139:43:47,  5.27s/it]
 58%|█████▊    | 129615/225000 [261:58:32<141:55:52,  5.36s/it]
 58%|█████▊    | 129616/225000 [261:58:42<178:21:02,  6.73s/it]
 58%|█████▊    | 129617/225000 [261:58:54<220:47:39,  8.33s/it]
 58%|█████▊    | 129618/225000 [261:58:57<182:54:43,  6.90s/it]
 58%|█████▊    | 129619/225000 [261:59:01<156:22:46,  5.90s/it]
 58%|█████▊    | 129620/225000 [261:59:11<190:00:33,  7.17s/it]
 58%|█████▊    | 129621/225000 [261:59:14<158:07:55,  5.97s/it]
 58%|█████▊    | 129622/225000 [261:59:18<143:04:53,  5.40s/it]
 58%|█████▊    | 129623/225000 [261:59:23<133:57:40,  5.06s/it]
 58%|█████▊    | 129624/225000 [261:59:33<178:23:08,  6.73s/it]
 58%|█████▊    | 129625/225000 [261:59:37<156:10:17,  5.89s/it]
 58%|█████▊    | 129626/225000 [261:59:45<169:32:59,  6.40s/it]
 58%|█████▊    | 129627/225000 [261:59:49<148:48:25,  5.62s/it]
 58%|█████▊    | 129628/225000 [261:59:52<133:27:34,  5.04s/it]
 58%|█████▊    | 129629/225000 [262:00:02<174:01:14,  6.57s/it]
 58%|█████▊    | 129630/225000 [262:00:07<158:39:29,  5.99s/it]
 58%|█████▊    | 129631/225000 [262:00:15<170:15:19,  6.43s/it]
 58%|█████▊    | 129632/225000 [262:00:18<148:28:03,  5.60s/it]
 58%|█████▊    | 129633/225000 [262:00:27<171:54:05,  6.49s/it]
 58%|█████▊    | 129634/225000 [262:00:31<155:12:14,  5.86s/it]
 58%|█████▊    | 129635/225000 [262:00:35<139:59:44,  5.28s/it]
 58%|█████▊    | 129636/225000 [262:00:40<133:28:47,  5.04s/it]
 58%|█████▊    | 129637/225000 [262:00:44<127:10:38,  4.80s/it]
 58%|█████▊    | 129638/225000 [262:00:49<126:47:28,  4.79s/it]
 58%|█████▊    | 129639/225000 [262:00:56<144:54:38,  5.47s/it]
 58%|█████▊    | 129640/225000 [262:01:08<199:00:06,  7.51s/it]
 58%|█████▊    | 129641/225000 [262:01:12<169:03:06,  6.38s/it]
 58%|█████▊    | 129642/225000 [262:01:16<149:54:28,  5.66s/it]
 58%|█████▊    | 129643/225000 [262:01:19<135:25:25,  5.11s/it]
 58%|█████▊    | 129644/225000 [262:01:24<134:20:00,  5.07s/it]
 58%|█████▊    | 129645/225000 [262:01:29<130:58:50,  4.94s/it]
 58%|█████▊    | 129646/225000 [262:01:35<137:41:40,  5.20s/it]
 58%|█████▊    | 129647/225000 [262:01:48<200:41:28,  7.58s/it]
 58%|█████▊    | 129648/225000 [262:01:52<169:05:45,  6.38s/it]
 58%|█████▊    | 129649/225000 [262:01:58<170:31:58,  6.44s/it]
 58%|█████▊    | 129650/225000 [262:02:04<165:30:51,  6.25s/it]
                                                               
{'loss': '0.8167', 'grad_norm': '1.006', 'learning_rate': '2.061e-05', 'epoch': '0.5762', 'num_input_tokens_seen': 435639480, 'train_runtime': '9.433e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129650/225000 [262:02:04<165:30:51,  6.25s/it]
 58%|█████▊    | 129651/225000 [262:02:09<156:51:10,  5.92s/it]
 58%|█████▊    | 129652/225000 [262:02:17<171:40:46,  6.48s/it]
 58%|█████▊    | 129653/225000 [262:02:21<151:16:53,  5.71s/it]
 58%|█████▊    | 129654/225000 [262:02:30<178:52:49,  6.75s/it]
 58%|█████▊    | 129655/225000 [262:02:33<152:17:03,  5.75s/it]
 58%|█████▊    | 129656/225000 [262:02:41<168:48:55,  6.37s/it]
 58%|█████▊    | 129657/225000 [262:02:47<163:17:21,  6.17s/it]
 58%|█████▊    | 129658/225000 [262:02:54<172:05:49,  6.50s/it]
 58%|█████▊    | 129659/225000 [262:02:57<143:19:46,  5.41s/it]
 58%|█████▊    | 129660/225000 [262:03:06<172:15:59,  6.50s/it]
 58%|█████▊    | 129661/225000 [262:03:10<150:16:34,  5.67s/it]
 58%|█████▊    | 129662/225000 [262:03:13<129:42:58,  4.90s/it]
 58%|█████▊    | 129663/225000 [262:03:22<162:14:42,  6.13s/it]
 58%|█████▊    | 129664/225000 [262:03:26<145:44:34,  5.50s/it]
 58%|█████▊    | 129665/225000 [262:03:30<136:10:48,  5.14s/it]
 58%|█████▊    | 129666/225000 [262:03:42<185:27:33,  7.00s/it]
 58%|█████▊    | 129667/225000 [262:03:46<165:15:00,  6.24s/it]
 58%|█████▊    | 129668/225000 [262:03:50<149:01:42,  5.63s/it]
 58%|█████▊    | 129669/225000 [262:03:54<136:31:12,  5.16s/it]
 58%|█████▊    | 129670/225000 [262:04:01<146:44:34,  5.54s/it]
 58%|█████▊    | 129671/225000 [262:04:12<193:18:37,  7.30s/it]
 58%|█████▊    | 129672/225000 [262:04:15<161:03:35,  6.08s/it]
 58%|█████▊    | 129673/225000 [262:04:23<175:59:31,  6.65s/it]
 58%|█████▊    | 129674/225000 [262:04:28<156:14:41,  5.90s/it]
 58%|█████▊    | 129675/225000 [262:04:32<147:03:39,  5.55s/it]
 58%|█████▊    | 129676/225000 [262:04:37<137:25:52,  5.19s/it]
 58%|█████▊    | 129677/225000 [262:04:41<132:07:37,  4.99s/it]
 58%|█████▊    | 129678/225000 [262:04:54<190:45:18,  7.20s/it]
 58%|█████▊    | 129679/225000 [262:04:58<167:48:29,  6.34s/it]
 58%|█████▊    | 129680/225000 [262:05:03<155:21:25,  5.87s/it]
 58%|█████▊    | 129681/225000 [262:05:10<169:29:23,  6.40s/it]
 58%|█████▊    | 129682/225000 [262:05:15<156:12:36,  5.90s/it]
 58%|█████▊    | 129683/225000 [262:05:20<146:43:39,  5.54s/it]
 58%|█████▊    | 129684/225000 [262:05:24<136:41:48,  5.16s/it]
 58%|█████▊    | 129685/225000 [262:05:28<127:20:11,  4.81s/it]
 58%|█████▊    | 129686/225000 [262:05:33<127:45:13,  4.83s/it]
 58%|█████▊    | 129687/225000 [262:05:45<188:42:31,  7.13s/it]
 58%|█████▊    | 129688/225000 [262:05:49<162:43:48,  6.15s/it]
 58%|█████▊    | 129689/225000 [262:05:56<166:00:24,  6.27s/it]
 58%|█████▊    | 129690/225000 [262:06:02<163:07:27,  6.16s/it]
 58%|█████▊    | 129691/225000 [262:06:10<179:09:39,  6.77s/it]
 58%|█████▊    | 129692/225000 [262:06:14<161:03:46,  6.08s/it]
 58%|█████▊    | 129693/225000 [262:06:18<145:34:15,  5.50s/it]
 58%|█████▊    | 129694/225000 [262:06:23<137:44:07,  5.20s/it]
 58%|█████▊    | 129695/225000 [262:06:27<131:31:48,  4.97s/it]
 58%|█████▊    | 129696/225000 [262:06:32<125:13:03,  4.73s/it]
 58%|█████▊    | 129697/225000 [262:06:43<177:39:01,  6.71s/it]
 58%|█████▊    | 129698/225000 [262:06:48<167:30:48,  6.33s/it]
 58%|█████▊    | 129699/225000 [262:06:52<146:38:55,  5.54s/it]
 58%|█████▊    | 129700/225000 [262:07:01<172:17:50,  6.51s/it]
                                                               
{'loss': '0.8246', 'grad_norm': '0.9956', 'learning_rate': '2.059e-05', 'epoch': '0.5764', 'num_input_tokens_seen': 435768672, 'train_runtime': '9.436e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129700/225000 [262:07:01<172:17:50,  6.51s/it]
 58%|█████▊    | 129701/225000 [262:07:04<149:16:42,  5.64s/it]
 58%|█████▊    | 129702/225000 [262:07:14<176:42:12,  6.68s/it]
 58%|█████▊    | 129703/225000 [262:07:18<156:03:34,  5.90s/it]
 58%|█████▊    | 129704/225000 [262:07:22<140:48:32,  5.32s/it]
 58%|█████▊    | 129705/225000 [262:07:30<164:18:20,  6.21s/it]
 58%|█████▊    | 129706/225000 [262:07:35<157:59:53,  5.97s/it]
 58%|█████▊    | 129707/225000 [262:07:38<135:52:16,  5.13s/it]
 58%|█████▊    | 129708/225000 [262:07:46<157:38:11,  5.96s/it]
 58%|█████▊    | 129709/225000 [262:07:54<169:48:03,  6.41s/it]
 58%|█████▊    | 129710/225000 [262:07:57<146:44:38,  5.54s/it]
 58%|█████▊    | 129711/225000 [262:08:05<163:53:49,  6.19s/it]
 58%|█████▊    | 129712/225000 [262:08:13<175:34:46,  6.63s/it]
 58%|█████▊    | 129713/225000 [262:08:17<154:10:59,  5.83s/it]
 58%|█████▊    | 129714/225000 [262:08:20<136:01:45,  5.14s/it]
 58%|█████▊    | 129715/225000 [262:08:29<164:18:14,  6.21s/it]
 58%|█████▊    | 129716/225000 [262:08:32<139:55:49,  5.29s/it]
 58%|█████▊    | 129717/225000 [262:08:40<161:26:42,  6.10s/it]
 58%|█████▊    | 129718/225000 [262:08:45<148:43:24,  5.62s/it]
 58%|█████▊    | 129719/225000 [262:08:49<138:42:06,  5.24s/it]
 58%|█████▊    | 129720/225000 [262:08:54<134:53:26,  5.10s/it]
 58%|█████▊    | 129721/225000 [262:08:59<133:23:34,  5.04s/it]
 58%|█████▊    | 129722/225000 [262:09:03<131:50:08,  4.98s/it]
 58%|█████▊    | 129723/225000 [262:09:08<129:04:05,  4.88s/it]
 58%|█████▊    | 129724/225000 [262:09:13<130:37:27,  4.94s/it]
 58%|█████▊    | 129725/225000 [262:09:18<130:44:05,  4.94s/it]
 58%|█████▊    | 129726/225000 [262:09:31<191:28:52,  7.24s/it]
 58%|█████▊    | 129727/225000 [262:09:37<186:48:04,  7.06s/it]
 58%|█████▊    | 129728/225000 [262:09:41<161:17:15,  6.09s/it]
 58%|█████▊    | 129729/225000 [262:09:46<148:42:16,  5.62s/it]
 58%|█████▊    | 129730/225000 [262:09:50<136:32:09,  5.16s/it]
 58%|█████▊    | 129731/225000 [262:10:03<197:39:59,  7.47s/it]
 58%|█████▊    | 129732/225000 [262:10:07<171:39:40,  6.49s/it]
 58%|█████▊    | 129733/225000 [262:10:11<150:21:18,  5.68s/it]
 58%|█████▊    | 129734/225000 [262:10:19<170:51:24,  6.46s/it]
 58%|█████▊    | 129735/225000 [262:10:23<149:00:08,  5.63s/it]
 58%|█████▊    | 129736/225000 [262:10:30<167:22:15,  6.32s/it]
 58%|█████▊    | 129737/225000 [262:10:35<156:47:33,  5.93s/it]
 58%|█████▊    | 129738/225000 [262:10:43<166:54:47,  6.31s/it]
 58%|█████▊    | 129739/225000 [262:10:50<178:13:55,  6.74s/it]
 58%|█████▊    | 129740/225000 [262:10:54<151:25:33,  5.72s/it]
 58%|█████▊    | 129741/225000 [262:11:04<184:23:47,  6.97s/it]
 58%|█████▊    | 129742/225000 [262:11:07<156:10:42,  5.90s/it]
 58%|█████▊    | 129743/225000 [262:11:14<163:46:00,  6.19s/it]
 58%|█████▊    | 129744/225000 [262:11:20<160:39:01,  6.07s/it]
 58%|█████▊    | 129745/225000 [262:11:24<147:49:20,  5.59s/it]
 58%|█████▊    | 129746/225000 [262:11:29<140:33:44,  5.31s/it]
 58%|█████▊    | 129747/225000 [262:11:33<132:38:37,  5.01s/it]
 58%|█████▊    | 129748/225000 [262:11:38<131:40:12,  4.98s/it]
 58%|█████▊    | 129749/225000 [262:11:43<134:12:49,  5.07s/it]
 58%|█████▊    | 129750/225000 [262:11:55<185:25:40,  7.01s/it]
                                                               
{'loss': '0.803', 'grad_norm': '1.037', 'learning_rate': '2.057e-05', 'epoch': '0.5767', 'num_input_tokens_seen': 435898336, 'train_runtime': '9.439e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129750/225000 [262:11:55<185:25:40,  7.01s/it]
 58%|█████▊    | 129751/225000 [262:11:59<164:49:10,  6.23s/it]
 58%|█████▊    | 129752/225000 [262:12:06<170:17:54,  6.44s/it]
 58%|█████▊    | 129753/225000 [262:12:11<154:13:40,  5.83s/it]
 58%|█████▊    | 129754/225000 [262:12:15<142:12:33,  5.38s/it]
 58%|█████▊    | 129755/225000 [262:12:19<128:53:24,  4.87s/it]
 58%|█████▊    | 129756/225000 [262:12:25<143:21:03,  5.42s/it]
 58%|█████▊    | 129757/225000 [262:12:38<198:35:48,  7.51s/it]
 58%|█████▊    | 129758/225000 [262:12:41<165:39:02,  6.26s/it]
 58%|█████▊    | 129759/225000 [262:12:46<156:31:42,  5.92s/it]
 58%|█████▊    | 129760/225000 [262:12:53<163:16:33,  6.17s/it]
 58%|█████▊    | 129761/225000 [262:12:59<163:47:32,  6.19s/it]
 58%|█████▊    | 129762/225000 [262:13:03<141:54:31,  5.36s/it]
 58%|█████▊    | 129763/225000 [262:13:06<127:59:41,  4.84s/it]
 58%|█████▊    | 129764/225000 [262:13:16<168:15:22,  6.36s/it]
 58%|█████▊    | 129765/225000 [262:13:21<152:55:30,  5.78s/it]
 58%|█████▊    | 129766/225000 [262:13:25<140:16:32,  5.30s/it]
 58%|█████▊    | 129767/225000 [262:13:29<132:29:46,  5.01s/it]
 58%|█████▊    | 129768/225000 [262:13:41<191:26:59,  7.24s/it]
 58%|█████▊    | 129769/225000 [262:13:46<172:27:16,  6.52s/it]
 58%|█████▊    | 129770/225000 [262:13:51<154:32:33,  5.84s/it]
 58%|█████▊    | 129771/225000 [262:13:54<137:37:14,  5.20s/it]
 58%|█████▊    | 129772/225000 [262:14:03<167:33:42,  6.33s/it]
 58%|█████▊    | 129773/225000 [262:14:11<176:42:56,  6.68s/it]
 58%|█████▊    | 129774/225000 [262:14:18<181:44:04,  6.87s/it]
 58%|█████▊    | 129775/225000 [262:14:22<158:32:22,  5.99s/it]
 58%|█████▊    | 129776/225000 [262:14:26<145:29:36,  5.50s/it]
 58%|█████▊    | 129777/225000 [262:14:31<137:35:07,  5.20s/it]
 58%|█████▊    | 129778/225000 [262:14:40<170:44:41,  6.46s/it]
 58%|█████▊    | 129779/225000 [262:14:49<189:25:17,  7.16s/it]
 58%|█████▊    | 129780/225000 [262:14:52<159:29:22,  6.03s/it]
 58%|█████▊    | 129781/225000 [262:14:57<145:32:45,  5.50s/it]
 58%|█████▊    | 129782/225000 [262:15:05<167:57:41,  6.35s/it]
 58%|█████▊    | 129783/225000 [262:15:10<156:11:15,  5.91s/it]
 58%|█████▊    | 129784/225000 [262:15:15<145:43:24,  5.51s/it]
 58%|█████▊    | 129785/225000 [262:15:19<136:57:25,  5.18s/it]
 58%|█████▊    | 129786/225000 [262:15:24<134:36:35,  5.09s/it]
 58%|█████▊    | 129787/225000 [262:15:34<177:29:40,  6.71s/it]
 58%|█████▊    | 129788/225000 [262:15:42<188:30:26,  7.13s/it]
 58%|█████▊    | 129789/225000 [262:15:49<187:06:30,  7.07s/it]
 58%|█████▊    | 129790/225000 [262:15:54<165:37:42,  6.26s/it]
 58%|█████▊    | 129791/225000 [262:15:57<143:59:02,  5.44s/it]
 58%|█████▊    | 129792/225000 [262:16:06<169:07:50,  6.40s/it]
 58%|█████▊    | 129793/225000 [262:16:10<149:42:05,  5.66s/it]
 58%|█████▊    | 129794/225000 [262:16:14<136:35:35,  5.16s/it]
 58%|█████▊    | 129795/225000 [262:16:18<129:14:20,  4.89s/it]
 58%|█████▊    | 129796/225000 [262:16:30<182:25:39,  6.90s/it]
 58%|█████▊    | 129797/225000 [262:16:34<159:14:04,  6.02s/it]
 58%|█████▊    | 129798/225000 [262:16:38<144:25:30,  5.46s/it]
 58%|█████▊    | 129799/225000 [262:16:46<169:47:51,  6.42s/it]
 58%|█████▊    | 129800/225000 [262:16:50<146:48:05,  5.55s/it]
                                                               
{'loss': '0.8162', 'grad_norm': '1.025', 'learning_rate': '2.055e-05', 'epoch': '0.5769', 'num_input_tokens_seen': 436027488, 'train_runtime': '9.442e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129800/225000 [262:16:50<146:48:05,  5.55s/it]
 58%|█████▊    | 129801/225000 [262:16:58<165:54:32,  6.27s/it]
 58%|█████▊    | 129802/225000 [262:17:03<157:31:19,  5.96s/it]
 58%|█████▊    | 129803/225000 [262:17:07<142:33:37,  5.39s/it]
 58%|█████▊    | 129804/225000 [262:17:12<137:50:24,  5.21s/it]
 58%|█████▊    | 129805/225000 [262:17:16<130:57:03,  4.95s/it]
 58%|█████▊    | 129806/225000 [262:17:21<129:14:27,  4.89s/it]
 58%|█████▊    | 129807/225000 [262:17:27<141:07:20,  5.34s/it]
 58%|█████▊    | 129808/225000 [262:17:39<189:10:55,  7.15s/it]
 58%|█████▊    | 129809/225000 [262:17:45<181:22:21,  6.86s/it]
 58%|█████▊    | 129810/225000 [262:17:53<193:48:41,  7.33s/it]
 58%|█████▊    | 129811/225000 [262:17:58<167:49:11,  6.35s/it]
 58%|█████▊    | 129812/225000 [262:18:02<155:15:16,  5.87s/it]
 58%|█████▊    | 129813/225000 [262:18:08<152:25:54,  5.77s/it]
 58%|█████▊    | 129814/225000 [262:18:13<148:55:03,  5.63s/it]
 58%|█████▊    | 129815/225000 [262:18:19<149:48:53,  5.67s/it]
 58%|█████▊    | 129816/225000 [262:18:24<146:33:40,  5.54s/it]
 58%|█████▊    | 129817/225000 [262:18:30<148:49:13,  5.63s/it]
 58%|█████▊    | 129818/225000 [262:18:35<147:12:37,  5.57s/it]
 58%|█████▊    | 129819/225000 [262:18:41<144:14:06,  5.46s/it]
 58%|█████▊    | 129820/225000 [262:18:46<144:16:20,  5.46s/it]
 58%|█████▊    | 129821/225000 [262:18:52<144:18:31,  5.46s/it]
 58%|█████▊    | 129822/225000 [262:18:57<140:51:08,  5.33s/it]
 58%|█████▊    | 129823/225000 [262:19:01<137:57:29,  5.22s/it]
 58%|█████▊    | 129824/225000 [262:19:14<194:02:58,  7.34s/it]
 58%|█████▊    | 129825/225000 [262:19:23<208:37:36,  7.89s/it]
 58%|█████▊    | 129826/225000 [262:19:27<178:50:30,  6.76s/it]
 58%|█████▊    | 129827/225000 [262:19:31<155:11:34,  5.87s/it]
 58%|█████▊    | 129828/225000 [262:19:41<190:51:20,  7.22s/it]
 58%|█████▊    | 129829/225000 [262:19:46<169:23:32,  6.41s/it]
 58%|█████▊    | 129830/225000 [262:19:50<155:18:41,  5.87s/it]
 58%|█████▊    | 129831/225000 [262:19:55<145:44:33,  5.51s/it]
 58%|█████▊    | 129832/225000 [262:20:00<138:36:33,  5.24s/it]
 58%|█████▊    | 129833/225000 [262:20:10<182:19:57,  6.90s/it]
 58%|█████▊    | 129834/225000 [262:20:14<158:39:45,  6.00s/it]
 58%|█████▊    | 129835/225000 [262:20:18<141:37:39,  5.36s/it]
 58%|█████▊    | 129836/225000 [262:20:27<170:59:00,  6.47s/it]
 58%|█████▊    | 129837/225000 [262:20:34<177:02:44,  6.70s/it]
 58%|█████▊    | 129838/225000 [262:20:38<154:55:07,  5.86s/it]
 58%|█████▊    | 129839/225000 [262:20:48<184:58:29,  7.00s/it]
 58%|█████▊    | 129840/225000 [262:20:52<157:00:08,  5.94s/it]
 58%|█████▊    | 129841/225000 [262:20:55<136:00:25,  5.15s/it]
 58%|█████▊    | 129842/225000 [262:21:05<174:46:13,  6.61s/it]
 58%|█████▊    | 129843/225000 [262:21:08<150:48:00,  5.71s/it]
 58%|█████▊    | 129844/225000 [262:21:15<159:37:32,  6.04s/it]
 58%|█████▊    | 129845/225000 [262:21:19<140:27:18,  5.31s/it]
 58%|█████▊    | 129846/225000 [262:21:27<161:47:27,  6.12s/it]
 58%|█████▊    | 129847/225000 [262:21:33<161:39:27,  6.12s/it]
 58%|█████▊    | 129848/225000 [262:21:36<140:25:55,  5.31s/it]
 58%|█████▊    | 129849/225000 [262:21:45<167:57:34,  6.35s/it]
 58%|█████▊    | 129850/225000 [262:21:50<158:03:35,  5.98s/it]
                                                               
{'loss': '0.8151', 'grad_norm': '0.9719', 'learning_rate': '2.053e-05', 'epoch': '0.5771', 'num_input_tokens_seen': 436158496, 'train_runtime': '9.445e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129850/225000 [262:21:50<158:03:35,  5.98s/it]
 58%|█████▊    | 129851/225000 [262:21:54<141:02:42,  5.34s/it]
 58%|█████▊    | 129852/225000 [262:21:59<139:36:17,  5.28s/it]
 58%|█████▊    | 129853/225000 [262:22:08<170:03:20,  6.43s/it]
 58%|█████▊    | 129854/225000 [262:22:13<153:43:01,  5.82s/it]
 58%|█████▊    | 129855/225000 [262:22:17<140:59:35,  5.33s/it]
 58%|█████▊    | 129856/225000 [262:22:21<133:31:45,  5.05s/it]
 58%|█████▊    | 129857/225000 [262:22:26<127:45:12,  4.83s/it]
 58%|█████▊    | 129858/225000 [262:22:32<135:18:19,  5.12s/it]
 58%|█████▊    | 129859/225000 [262:22:44<191:35:24,  7.25s/it]
 58%|█████▊    | 129860/225000 [262:22:47<158:35:26,  6.00s/it]
 58%|█████▊    | 129861/225000 [262:22:51<141:03:08,  5.34s/it]
 58%|█████▊    | 129862/225000 [262:22:55<133:57:44,  5.07s/it]
 58%|█████▊    | 129863/225000 [262:22:59<127:22:26,  4.82s/it]
 58%|█████▊    | 129864/225000 [262:23:11<182:03:02,  6.89s/it]
 58%|█████▊    | 129865/225000 [262:23:18<180:30:00,  6.83s/it]
 58%|█████▊    | 129866/225000 [262:23:21<153:19:13,  5.80s/it]
 58%|█████▊    | 129867/225000 [262:23:30<175:43:35,  6.65s/it]
 58%|█████▊    | 129868/225000 [262:23:33<151:17:40,  5.73s/it]
 58%|█████▊    | 129869/225000 [262:23:41<169:14:33,  6.40s/it]
 58%|█████▊    | 129870/225000 [262:23:45<147:23:29,  5.58s/it]
 58%|█████▊    | 129871/225000 [262:23:52<156:55:54,  5.94s/it]
 58%|█████▊    | 129872/225000 [262:23:58<158:40:05,  6.00s/it]
 58%|█████▊    | 129873/225000 [262:24:01<138:23:21,  5.24s/it]
 58%|█████▊    | 129874/225000 [262:24:05<127:04:04,  4.81s/it]
 58%|█████▊    | 129875/225000 [262:24:16<173:09:27,  6.55s/it]
 58%|█████▊    | 129876/225000 [262:24:19<149:41:30,  5.67s/it]
 58%|█████▊    | 129877/225000 [262:24:28<174:57:21,  6.62s/it]
 58%|█████▊    | 129878/225000 [262:24:32<149:33:26,  5.66s/it]
 58%|█████▊    | 129879/225000 [262:24:37<145:31:13,  5.51s/it]
 58%|█████▊    | 129880/225000 [262:24:42<143:11:43,  5.42s/it]
 58%|█████▊    | 129881/225000 [262:24:48<144:19:24,  5.46s/it]
 58%|█████▊    | 129882/225000 [262:24:52<140:10:07,  5.31s/it]
 58%|█████▊    | 129883/225000 [262:24:58<140:12:56,  5.31s/it]
 58%|█████▊    | 129884/225000 [262:25:03<139:26:08,  5.28s/it]
 58%|█████▊    | 129885/225000 [262:25:08<135:51:00,  5.14s/it]
 58%|█████▊    | 129886/225000 [262:25:13<138:19:44,  5.24s/it]
 58%|█████▊    | 129887/225000 [262:25:19<143:21:44,  5.43s/it]
 58%|█████▊    | 129888/225000 [262:25:32<202:07:33,  7.65s/it]
 58%|█████▊    | 129889/225000 [262:25:39<193:18:43,  7.32s/it]
 58%|█████▊    | 129890/225000 [262:25:42<164:38:10,  6.23s/it]
 58%|█████▊    | 129891/225000 [262:25:49<167:57:38,  6.36s/it]
 58%|█████▊    | 129892/225000 [262:25:55<165:55:02,  6.28s/it]
 58%|█████▊    | 129893/225000 [262:25:59<144:08:26,  5.46s/it]
 58%|█████▊    | 129894/225000 [262:26:08<176:22:32,  6.68s/it]
 58%|█████▊    | 129895/225000 [262:26:12<151:17:17,  5.73s/it]
 58%|█████▊    | 129896/225000 [262:26:20<173:13:08,  6.56s/it]
 58%|█████▊    | 129897/225000 [262:26:24<155:09:19,  5.87s/it]
 58%|█████▊    | 129898/225000 [262:26:31<162:50:41,  6.16s/it]
 58%|█████▊    | 129899/225000 [262:26:35<143:32:22,  5.43s/it]
 58%|█████▊    | 129900/225000 [262:26:43<162:36:27,  6.16s/it]
                                                               
{'loss': '0.8127', 'grad_norm': '1.034', 'learning_rate': '2.052e-05', 'epoch': '0.5773', 'num_input_tokens_seen': 436287008, 'train_runtime': '9.448e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129900/225000 [262:26:43<162:36:27,  6.16s/it]
 58%|█████▊    | 129901/225000 [262:26:49<166:40:12,  6.31s/it]
 58%|█████▊    | 129902/225000 [262:26:53<148:24:33,  5.62s/it]
 58%|█████▊    | 129903/225000 [262:27:02<169:40:34,  6.42s/it]
 58%|█████▊    | 129904/225000 [262:27:05<143:58:50,  5.45s/it]
 58%|█████▊    | 129905/225000 [262:27:10<137:22:42,  5.20s/it]
 58%|█████▊    | 129906/225000 [262:27:17<152:37:31,  5.78s/it]
 58%|█████▊    | 129907/225000 [262:27:23<154:23:03,  5.84s/it]
 58%|█████▊    | 129908/225000 [262:27:27<142:43:33,  5.40s/it]
 58%|█████▊    | 129909/225000 [262:27:32<135:57:15,  5.15s/it]
 58%|█████▊    | 129910/225000 [262:27:36<131:59:46,  5.00s/it]
 58%|█████▊    | 129911/225000 [262:27:41<130:46:31,  4.95s/it]
 58%|█████▊    | 129912/225000 [262:27:54<192:08:02,  7.27s/it]
 58%|█████▊    | 129913/225000 [262:27:59<173:14:57,  6.56s/it]
 58%|█████▊    | 129914/225000 [262:28:02<149:59:48,  5.68s/it]
 58%|█████▊    | 129915/225000 [262:28:12<184:30:47,  6.99s/it]
 58%|█████▊    | 129916/225000 [262:28:16<156:54:18,  5.94s/it]
 58%|█████▊    | 129917/225000 [262:28:21<153:07:40,  5.80s/it]
 58%|█████▊    | 129918/225000 [262:28:28<160:18:32,  6.07s/it]
 58%|█████▊    | 129919/225000 [262:28:32<145:52:24,  5.52s/it]
 58%|█████▊    | 129920/225000 [262:28:37<139:25:51,  5.28s/it]
 58%|█████▊    | 129921/225000 [262:28:42<134:55:36,  5.11s/it]
 58%|█████▊    | 129922/225000 [262:28:46<132:47:22,  5.03s/it]
 58%|█████▊    | 129923/225000 [262:28:51<128:40:48,  4.87s/it]
 58%|█████▊    | 129924/225000 [262:28:56<130:12:59,  4.93s/it]
 58%|█████▊    | 129925/225000 [262:29:04<151:28:44,  5.74s/it]
 58%|█████▊    | 129926/225000 [262:29:15<193:04:01,  7.31s/it]
 58%|█████▊    | 129927/225000 [262:29:18<159:23:07,  6.04s/it]
 58%|█████▊    | 129928/225000 [262:29:28<194:02:56,  7.35s/it]
 58%|█████▊    | 129929/225000 [262:29:32<168:44:01,  6.39s/it]
 58%|█████▊    | 129930/225000 [262:29:36<150:35:36,  5.70s/it]
 58%|█████▊    | 129931/225000 [262:29:40<136:56:23,  5.19s/it]
 58%|█████▊    | 129932/225000 [262:29:48<156:23:07,  5.92s/it]
 58%|█████▊    | 129933/225000 [262:29:57<178:23:25,  6.76s/it]
 58%|█████▊    | 129934/225000 [262:30:03<178:36:20,  6.76s/it]
 58%|█████▊    | 129935/225000 [262:30:08<163:49:37,  6.20s/it]
 58%|█████▊    | 129936/225000 [262:30:13<150:05:28,  5.68s/it]
 58%|█████▊    | 129937/225000 [262:30:22<175:10:41,  6.63s/it]
 58%|█████▊    | 129938/225000 [262:30:26<156:26:20,  5.92s/it]
 58%|█████▊    | 129939/225000 [262:30:30<138:45:29,  5.25s/it]
 58%|█████▊    | 129940/225000 [262:30:34<132:30:27,  5.02s/it]
 58%|█████▊    | 129941/225000 [262:30:39<130:00:42,  4.92s/it]
 58%|█████▊    | 129942/225000 [262:30:44<131:25:35,  4.98s/it]
 58%|█████▊    | 129943/225000 [262:30:50<143:11:21,  5.42s/it]
 58%|█████▊    | 129944/225000 [262:31:02<194:27:18,  7.36s/it]
 58%|█████▊    | 129945/225000 [262:31:07<170:08:13,  6.44s/it]
 58%|█████▊    | 129946/225000 [262:31:11<151:36:06,  5.74s/it]
 58%|█████▊    | 129947/225000 [262:31:15<137:51:02,  5.22s/it]
 58%|█████▊    | 129948/225000 [262:31:26<186:53:46,  7.08s/it]
 58%|█████▊    | 129949/225000 [262:31:30<162:24:19,  6.15s/it]
 58%|█████▊    | 129950/225000 [262:31:37<168:32:53,  6.38s/it]
                                                               
{'loss': '0.8067', 'grad_norm': '1.034', 'learning_rate': '2.05e-05', 'epoch': '0.5776', 'num_input_tokens_seen': 436417848, 'train_runtime': '9.451e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 129950/225000 [262:31:37<168:32:53,  6.38s/it]
 58%|█████▊    | 129951/225000 [262:31:43<166:19:54,  6.30s/it]
 58%|█████▊    | 129952/225000 [262:31:47<144:52:44,  5.49s/it]
 58%|█████▊    | 129953/225000 [262:31:55<165:17:01,  6.26s/it]
 58%|█████▊    | 129954/225000 [262:31:59<151:04:55,  5.72s/it]
 58%|█████▊    | 129955/225000 [262:32:04<139:32:22,  5.29s/it]
 58%|█████▊    | 129956/225000 [262:32:08<135:56:33,  5.15s/it]
 58%|█████▊    | 129957/225000 [262:32:15<150:27:08,  5.70s/it]
 58%|█████▊    | 129958/225000 [262:32:27<196:44:38,  7.45s/it]
 58%|█████▊    | 129959/225000 [262:32:30<162:56:55,  6.17s/it]
 58%|█████▊    | 129960/225000 [262:32:37<172:13:35,  6.52s/it]
 58%|█████▊    | 129961/225000 [262:32:42<159:19:20,  6.04s/it]
 58%|█████▊    | 129962/225000 [262:32:47<152:26:10,  5.77s/it]
 58%|█████▊    | 129963/225000 [262:32:53<147:47:54,  5.60s/it]
 58%|█████▊    | 129964/225000 [262:32:58<143:56:26,  5.45s/it]
 58%|█████▊    | 129965/225000 [262:33:03<143:02:55,  5.42s/it]
 58%|█████▊    | 129966/225000 [262:33:09<145:33:26,  5.51s/it]
 58%|█████▊    | 129967/225000 [262:33:14<144:18:47,  5.47s/it]
 58%|█████▊    | 129968/225000 [262:33:19<142:13:47,  5.39s/it]
 58%|█████▊    | 129969/225000 [262:33:30<184:54:51,  7.00s/it]
 58%|█████▊    | 129970/225000 [262:33:34<163:35:06,  6.20s/it]
 58%|█████▊    | 129971/225000 [262:33:39<148:31:18,  5.63s/it]
 58%|█████▊    | 129972/225000 [262:33:43<140:06:36,  5.31s/it]
 58%|█████▊    | 129973/225000 [262:33:53<175:30:47,  6.65s/it]
 58%|█████▊    | 129974/225000 [262:33:59<168:33:44,  6.39s/it]
 58%|█████▊    | 129975/225000 [262:34:04<156:50:57,  5.94s/it]
 58%|█████▊    | 129976/225000 [262:34:11<168:25:54,  6.38s/it]
 58%|█████▊    | 129977/225000 [262:34:15<151:24:36,  5.74s/it]
 58%|█████▊    | 129978/225000 [262:34:22<154:11:16,  5.84s/it]
 58%|█████▊    | 129979/225000 [262:34:31<184:48:07,  7.00s/it]
 58%|█████▊    | 129980/225000 [262:34:35<159:30:46,  6.04s/it]
 58%|█████▊    | 129981/225000 [262:34:39<142:01:11,  5.38s/it]
 58%|█████▊    | 129982/225000 [262:34:43<131:18:16,  4.97s/it]
 58%|█████▊    | 129983/225000 [262:34:55<184:17:45,  6.98s/it]
 58%|█████▊    | 129984/225000 [262:34:58<157:02:37,  5.95s/it]
 58%|█████▊    | 129985/225000 [262:35:03<150:04:15,  5.69s/it]
 58%|█████▊    | 129986/225000 [262:35:08<146:14:15,  5.54s/it]
 58%|█████▊    | 129987/225000 [262:35:13<142:06:39,  5.38s/it]
 58%|█████▊    | 129988/225000 [262:35:18<139:13:39,  5.28s/it]
 58%|█████▊    | 129989/225000 [262:35:24<138:10:57,  5.24s/it]
 58%|█████▊    | 129990/225000 [262:35:28<133:36:11,  5.06s/it]
 58%|█████▊    | 129991/225000 [262:35:34<136:36:39,  5.18s/it]
 58%|█████▊    | 129992/225000 [262:35:38<131:10:53,  4.97s/it]
 58%|█████▊    | 129993/225000 [262:35:44<139:12:42,  5.28s/it]
 58%|█████▊    | 129994/225000 [262:35:57<195:42:52,  7.42s/it]
 58%|█████▊    | 129995/225000 [262:36:01<175:14:16,  6.64s/it]
 58%|█████▊    | 129996/225000 [262:36:10<190:23:37,  7.21s/it]
 58%|█████▊    | 129997/225000 [262:36:13<160:31:57,  6.08s/it]
 58%|█████▊    | 129998/225000 [262:36:18<148:31:14,  5.63s/it]
 58%|█████▊    | 129999/225000 [262:36:23<140:31:16,  5.32s/it]
 58%|█████▊    | 130000/225000 [262:36:27<135:19:25,  5.13s/it]
                                                               
{'loss': '0.8178', 'grad_norm': '1.036', 'learning_rate': '2.048e-05', 'epoch': '0.5778', 'num_input_tokens_seen': 436547416, 'train_runtime': '9.454e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 130000/225000 [262:36:27<135:19:25,  5.13s/it]
 58%|█████▊    | 130001/225000 [262:36:32<130:21:30,  4.94s/it]
 58%|█████▊    | 130002/225000 [262:36:43<182:37:26,  6.92s/it]
 58%|█████▊    | 130003/225000 [262:36:48<168:15:08,  6.38s/it]
 58%|█████▊    | 130004/225000 [262:36:56<175:45:07,  6.66s/it]
 58%|█████▊    | 130005/225000 [262:37:00<159:30:31,  6.04s/it]
 58%|█████▊    | 130006/225000 [262:37:05<149:47:13,  5.68s/it]
 58%|█████▊    | 130007/225000 [262:37:10<141:33:21,  5.36s/it]
 58%|█████▊    | 130008/225000 [262:37:14<133:05:01,  5.04s/it]
 58%|█████▊    | 130009/225000 [262:37:20<140:19:07,  5.32s/it]
 58%|█████▊    | 130010/225000 [262:37:31<186:50:21,  7.08s/it]
 58%|█████▊    | 130011/225000 [262:37:35<158:58:03,  6.02s/it]
 58%|█████▊    | 130012/225000 [262:37:42<169:51:02,  6.44s/it]
 58%|█████▊    | 130013/225000 [262:37:48<168:54:07,  6.40s/it]
 58%|█████▊    | 130014/225000 [262:37:54<161:57:14,  6.14s/it]
 58%|█████▊    | 130015/225000 [262:38:01<171:48:29,  6.51s/it]
 58%|█████▊    | 130016/225000 [262:38:05<146:22:22,  5.55s/it]
 58%|█████▊    | 130017/225000 [262:38:09<137:05:28,  5.20s/it]
 58%|█████▊    | 130018/225000 [262:38:13<130:11:26,  4.93s/it]
 58%|█████▊    | 130019/225000 [262:38:26<190:59:11,  7.24s/it]
 58%|█████▊    | 130020/225000 [262:38:29<160:46:05,  6.09s/it]
 58%|█████▊    | 130021/225000 [262:38:38<176:40:36,  6.70s/it]
 58%|█████▊    | 130022/225000 [262:38:43<163:18:01,  6.19s/it]
 58%|█████▊    | 130023/225000 [262:38:47<152:20:32,  5.77s/it]
 58%|█████▊    | 130024/225000 [262:38:52<143:06:28,  5.42s/it]
 58%|█████▊    | 130025/225000 [262:38:56<135:02:42,  5.12s/it]
 58%|█████▊    | 130026/225000 [262:39:01<133:36:01,  5.06s/it]
 58%|█████▊    | 130027/225000 [262:39:06<133:42:04,  5.07s/it]
 58%|█████▊    | 130028/225000 [262:39:11<131:10:06,  4.97s/it]
 58%|█████▊    | 130029/225000 [262:39:16<133:11:04,  5.05s/it]
 58%|█████▊    | 130030/225000 [262:39:28<188:44:30,  7.15s/it]
 58%|█████▊    | 130031/225000 [262:39:33<172:06:41,  6.52s/it]
 58%|█████▊    | 130032/225000 [262:39:43<192:12:46,  7.29s/it]
 58%|█████▊    | 130033/225000 [262:39:46<165:34:58,  6.28s/it]
 58%|█████▊    | 130034/225000 [262:39:50<147:03:56,  5.58s/it]
 58%|█████▊    | 130035/225000 [262:40:03<199:20:29,  7.56s/it]
 58%|█████▊    | 130036/225000 [262:40:07<172:43:30,  6.55s/it]
 58%|█████▊    | 130037/225000 [262:40:14<176:27:54,  6.69s/it]
 58%|█████▊    | 130038/225000 [262:40:19<160:46:21,  6.09s/it]
 58%|█████▊    | 130039/225000 [262:40:23<151:29:15,  5.74s/it]
 58%|█████▊    | 130040/225000 [262:40:29<152:10:24,  5.77s/it]
 58%|█████▊    | 130041/225000 [262:40:35<148:21:43,  5.62s/it]
 58%|█████▊    | 130042/225000 [262:40:39<142:07:54,  5.39s/it]
 58%|█████▊    | 130043/225000 [262:40:45<142:22:20,  5.40s/it]
 58%|█████▊    | 130044/225000 [262:40:50<136:54:54,  5.19s/it]
 58%|█████▊    | 130045/225000 [262:40:54<134:51:49,  5.11s/it]
 58%|█████▊    | 130046/225000 [262:41:00<136:26:00,  5.17s/it]
 58%|█████▊    | 130047/225000 [262:41:05<134:09:41,  5.09s/it]
 58%|█████▊    | 130048/225000 [262:41:10<136:36:22,  5.18s/it]
 58%|█████▊    | 130049/225000 [262:41:16<139:17:32,  5.28s/it]
 58%|█████▊    | 130050/225000 [262:41:20<135:53:16,  5.15s/it]
                                                               
{'loss': '0.8198', 'grad_norm': '1.062', 'learning_rate': '2.046e-05', 'epoch': '0.578', 'num_input_tokens_seen': 436677304, 'train_runtime': '9.457e+05', 'train_tokens_per_second': '461.8'}

 58%|█████▊    | 130050/225000 [262:41:20<135:53:16,  5.15s/it]
 58%|█████▊    | 130051/225000 [262:41:34<202:24:47,  7.67s/it]
 58%|█████▊    | 130052/225000 [262:41:41<200:56:47,  7.62s/it]
 58%|█████▊    | 130053/225000 [262:41:45<171:58:00,  6.52s/it]
 58%|█████▊    | 130054/225000 [262:41:50<154:20:47,  5.85s/it]
 58%|█████▊    | 130055/225000 [262:42:01<196:31:31,  7.45s/it]
 58%|█████▊    | 130056/225000 [262:42:05<168:25:44,  6.39s/it]
 58%|█████▊    | 130057/225000 [262:42:08<145:36:07,  5.52s/it]
 58%|█████▊    | 130058/225000 [262:42:12<133:08:14,  5.05s/it]
 58%|█████▊    | 130059/225000 [262:42:24<183:35:33,  6.96s/it]
 58%|█████▊    | 130060/225000 [262:42:27<154:20:12,  5.85s/it]
 58%|█████▊    | 130061/225000 [262:42:36<177:49:23,  6.74s/it]
 58%|█████▊    | 130062/225000 [262:42:40<158:45:18,  6.02s/it]
 58%|█████▊    | 130063/225000 [262:42:46<154:12:04,  5.85s/it]
 58%|█████▊    | 130064/225000 [262:42:51<148:59:05,  5.65s/it]
 58%|█████▊    | 130065/225000 [262:42:56<143:34:28,  5.44s/it]
 58%|█████▊    | 130066/225000 [262:43:01<141:42:18,  5.37s/it]
 58%|█████▊    | 130067/225000 [262:43:06<141:05:01,  5.35s/it]
 58%|█████▊    | 130068/225000 [262:43:12<140:55:14,  5.34s/it]
 58%|█████▊    | 130069/225000 [262:43:17<141:39:27,  5.37s/it]
 58%|█████▊    | 130070/225000 [262:43:22<140:28:35,  5.33s/it]
 58%|█████▊    | 130071/225000 [262:43:27<140:09:02,  5.31s/it]
 58%|█████▊    | 130072/225000 [262:43:38<178:08:40,  6.76s/it]
 58%|█████▊    | 130073/225000 [262:43:47<203:14:00,  7.71s/it]
 58%|█████▊    | 130074/225000 [262:43:55<198:50:55,  7.54s/it]
 58%|█████▊    | 130075/225000 [262:44:01<188:41:00,  7.16s/it]
 58%|█████▊    | 130076/225000 [262:44:05<164:07:40,  6.22s/it]
 58%|█████▊    | 130077/225000 [262:44:09<150:04:13,  5.69s/it]
 58%|█████▊    | 130078/225000 [262:44:14<144:17:58,  5.47s/it]
 58%|█████▊    | 130079/225000 [262:44:19<136:41:18,  5.18s/it]
 58%|█████▊    | 130080/225000 [262:44:29<176:12:23,  6.68s/it]
 58%|█████▊    | 130081/225000 [262:44:35<172:08:39,  6.53s/it]
 58%|█████▊    | 130082/225000 [262:44:40<158:31:56,  6.01s/it]
 58%|█████▊    | 130083/225000 [262:44:45<147:52:13,  5.61s/it]
 58%|█████▊    | 130084/225000 [262:44:50<142:24:07,  5.40s/it]
 58%|█████▊    | 130085/225000 [262:44:55<139:04:44,  5.28s/it]
 58%|█████▊    | 130086/225000 [262:45:00<137:14:30,  5.21s/it]
 58%|█████▊    | 130087/225000 [262:45:05<137:03:59,  5.20s/it]
 58%|█████▊    | 130088/225000 [262:45:10<133:52:48,  5.08s/it]
 58%|█████▊    | 130089/225000 [262:45:20<175:58:59,  6.68s/it]
 58%|█████▊    | 130090/225000 [262:45:30<201:29:12,  7.64s/it]
 58%|█████▊    | 130091/225000 [262:45:36<190:52:41,  7.24s/it]
 58%|█████▊    | 130092/225000 [262:45:41<167:54:09,  6.37s/it]
 58%|█████▊    | 130093/225000 [262:45:45<156:32:08,  5.94s/it]
 58%|█████▊    | 130094/225000 [262:45:50<148:37:23,  5.64s/it]
 58%|█████▊    | 130095/225000 [262:45:55<143:39:35,  5.45s/it]
 58%|█████▊    | 130096/225000 [262:46:00<137:37:19,  5.22s/it]
 58%|█████▊    | 130097/225000 [262:46:09<167:31:06,  6.35s/it]
 58%|█████▊    | 130098/225000 [262:46:18<184:57:17,  7.02s/it]
 58%|█████▊    | 130099/225000 [262:46:21<158:55:55,  6.03s/it]
 58%|█████▊    | 130100/225000 [262:46:28<164:14:26,  6.23s/it]
                                                               
{'loss': '0.8209', 'grad_norm': '1.034', 'learning_rate': '2.044e-05', 'epoch': '0.5782', 'num_input_tokens_seen': 436808248, 'train_runtime': '9.46e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130100/225000 [262:46:28<164:14:26,  6.23s/it]
 58%|█████▊    | 130101/225000 [262:46:35<166:50:06,  6.33s/it]
 58%|█████▊    | 130102/225000 [262:46:38<144:02:00,  5.46s/it]
 58%|█████▊    | 130103/225000 [262:46:42<133:30:10,  5.06s/it]
 58%|█████▊    | 130104/225000 [262:46:48<138:05:38,  5.24s/it]
 58%|█████▊    | 130105/225000 [262:47:01<199:01:55,  7.55s/it]
 58%|█████▊    | 130106/225000 [262:47:05<171:49:40,  6.52s/it]
 58%|█████▊    | 130107/225000 [262:47:09<155:52:29,  5.91s/it]
 58%|█████▊    | 130108/225000 [262:47:14<142:06:40,  5.39s/it]
 58%|█████▊    | 130109/225000 [262:47:18<133:30:59,  5.07s/it]
 58%|█████▊    | 130110/225000 [262:47:29<179:01:34,  6.79s/it]
 58%|█████▊    | 130111/225000 [262:47:35<176:59:13,  6.71s/it]
 58%|█████▊    | 130112/225000 [262:47:39<152:59:52,  5.80s/it]
 58%|█████▊    | 130113/225000 [262:47:49<185:33:13,  7.04s/it]
 58%|█████▊    | 130114/225000 [262:47:52<154:22:29,  5.86s/it]
 58%|█████▊    | 130115/225000 [262:48:01<178:10:32,  6.76s/it]
 58%|█████▊    | 130116/225000 [262:48:05<154:51:34,  5.88s/it]
 58%|█████▊    | 130117/225000 [262:48:08<135:45:45,  5.15s/it]
 58%|█████▊    | 130118/225000 [262:48:16<158:17:38,  6.01s/it]
 58%|█████▊    | 130119/225000 [262:48:23<163:11:47,  6.19s/it]
 58%|█████▊    | 130120/225000 [262:48:26<140:27:37,  5.33s/it]
 58%|█████▊    | 130121/225000 [262:48:31<135:04:36,  5.13s/it]
 58%|█████▊    | 130122/225000 [262:48:36<132:24:41,  5.02s/it]
 58%|█████▊    | 130123/225000 [262:48:46<172:43:06,  6.55s/it]
 58%|█████▊    | 130124/225000 [262:48:53<182:35:17,  6.93s/it]
 58%|█████▊    | 130125/225000 [262:48:57<159:00:25,  6.03s/it]
 58%|█████▊    | 130126/225000 [262:49:06<179:52:12,  6.83s/it]
 58%|█████▊    | 130127/225000 [262:49:10<157:40:25,  5.98s/it]
 58%|█████▊    | 130128/225000 [262:49:17<163:12:19,  6.19s/it]
 58%|█████▊    | 130129/225000 [262:49:20<142:50:43,  5.42s/it]
 58%|█████▊    | 130130/225000 [262:49:28<157:43:13,  5.98s/it]
 58%|█████▊    | 130131/225000 [262:49:36<172:40:29,  6.55s/it]
 58%|█████▊    | 130132/225000 [262:49:39<151:59:05,  5.77s/it]
 58%|█████▊    | 130133/225000 [262:49:43<137:37:36,  5.22s/it]
 58%|█████▊    | 130134/225000 [262:49:51<156:18:38,  5.93s/it]
 58%|█████▊    | 130135/225000 [262:49:58<161:04:12,  6.11s/it]
 58%|█████▊    | 130136/225000 [262:50:04<165:47:10,  6.29s/it]
 58%|█████▊    | 130137/225000 [262:50:12<175:44:22,  6.67s/it]
 58%|█████▊    | 130138/225000 [262:50:15<149:15:54,  5.66s/it]
 58%|█████▊    | 130139/225000 [262:50:25<181:05:49,  6.87s/it]
 58%|█████▊    | 130140/225000 [262:50:30<169:25:49,  6.43s/it]
 58%|█████▊    | 130141/225000 [262:50:35<159:53:46,  6.07s/it]
 58%|█████▊    | 130142/225000 [262:50:40<150:43:36,  5.72s/it]
 58%|█████▊    | 130143/225000 [262:50:46<146:33:57,  5.56s/it]
 58%|█████▊    | 130144/225000 [262:50:51<145:32:06,  5.52s/it]
 58%|█████▊    | 130145/225000 [262:50:56<145:19:17,  5.52s/it]
 58%|█████▊    | 130146/225000 [262:51:02<144:12:56,  5.47s/it]
 58%|█████▊    | 130147/225000 [262:51:07<141:04:53,  5.35s/it]
 58%|█████▊    | 130148/225000 [262:51:12<138:23:17,  5.25s/it]
 58%|█████▊    | 130149/225000 [262:51:18<147:59:45,  5.62s/it]
 58%|█████▊    | 130150/225000 [262:51:31<200:49:11,  7.62s/it]
                                                               
{'loss': '0.8367', 'grad_norm': '1.128', 'learning_rate': '2.043e-05', 'epoch': '0.5784', 'num_input_tokens_seen': 436938000, 'train_runtime': '9.463e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130150/225000 [262:51:31<200:49:11,  7.62s/it]
 58%|█████▊    | 130151/225000 [262:51:35<171:22:16,  6.50s/it]
 58%|█████▊    | 130152/225000 [262:51:39<150:44:39,  5.72s/it]
 58%|█████▊    | 130153/225000 [262:51:45<153:03:23,  5.81s/it]
 58%|█████▊    | 130154/225000 [262:51:54<180:42:43,  6.86s/it]
 58%|█████▊    | 130155/225000 [262:51:58<160:58:52,  6.11s/it]
 58%|█████▊    | 130156/225000 [262:52:04<155:43:56,  5.91s/it]
 58%|█████▊    | 130157/225000 [262:52:09<151:59:56,  5.77s/it]
 58%|█████▊    | 130158/225000 [262:52:14<145:01:17,  5.50s/it]
 58%|█████▊    | 130159/225000 [262:52:19<140:46:14,  5.34s/it]
 58%|█████▊    | 130160/225000 [262:52:24<137:58:12,  5.24s/it]
 58%|█████▊    | 130161/225000 [262:52:29<140:19:59,  5.33s/it]
 58%|█████▊    | 130162/225000 [262:52:41<186:00:36,  7.06s/it]
 58%|█████▊    | 130163/225000 [262:52:48<185:44:54,  7.05s/it]
 58%|█████▊    | 130164/225000 [262:52:51<156:16:18,  5.93s/it]
 58%|█████▊    | 130165/225000 [262:53:00<181:46:44,  6.90s/it]
 58%|█████▊    | 130166/225000 [262:53:05<162:40:48,  6.18s/it]
 58%|█████▊    | 130167/225000 [262:53:08<139:35:24,  5.30s/it]
 58%|█████▊    | 130168/225000 [262:53:16<162:55:27,  6.18s/it]
 58%|█████▊    | 130169/225000 [262:53:23<166:55:51,  6.34s/it]
 58%|█████▊    | 130170/225000 [262:53:28<160:58:38,  6.11s/it]
 58%|█████▊    | 130171/225000 [262:53:34<154:16:36,  5.86s/it]
 58%|█████▊    | 130172/225000 [262:53:39<153:00:43,  5.81s/it]
 58%|█████▊    | 130173/225000 [262:53:44<146:55:57,  5.58s/it]
 58%|█████▊    | 130174/225000 [262:53:50<144:11:04,  5.47s/it]
 58%|█████▊    | 130175/225000 [262:53:55<144:54:43,  5.50s/it]
 58%|█████▊    | 130176/225000 [262:54:01<145:10:25,  5.51s/it]
 58%|█████▊    | 130177/225000 [262:54:05<138:15:56,  5.25s/it]
 58%|█████▊    | 130178/225000 [262:54:11<138:15:04,  5.25s/it]
 58%|█████▊    | 130179/225000 [262:54:17<147:00:25,  5.58s/it]
 58%|█████▊    | 130180/225000 [262:54:23<153:21:48,  5.82s/it]
 58%|█████▊    | 130181/225000 [262:54:36<209:11:28,  7.94s/it]
 58%|█████▊    | 130182/225000 [262:54:42<193:07:51,  7.33s/it]
 58%|█████▊    | 130183/225000 [262:54:47<174:56:44,  6.64s/it]
 58%|█████▊    | 130184/225000 [262:54:52<158:53:34,  6.03s/it]
 58%|█████▊    | 130185/225000 [262:54:56<146:17:15,  5.55s/it]
 58%|█████▊    | 130186/225000 [262:55:07<186:48:11,  7.09s/it]
 58%|█████▊    | 130187/225000 [262:55:10<157:22:12,  5.98s/it]
 58%|█████▊    | 130188/225000 [262:55:15<147:39:26,  5.61s/it]
 58%|█████▊    | 130189/225000 [262:55:24<171:04:50,  6.50s/it]
 58%|█████▊    | 130190/225000 [262:55:29<159:47:50,  6.07s/it]
 58%|█████▊    | 130191/225000 [262:55:33<146:02:59,  5.55s/it]
 58%|█████▊    | 130192/225000 [262:55:38<138:45:35,  5.27s/it]
 58%|█████▊    | 130193/225000 [262:55:43<139:01:32,  5.28s/it]
 58%|█████▊    | 130194/225000 [262:55:48<140:52:41,  5.35s/it]
 58%|█████▊    | 130195/225000 [262:55:53<138:27:51,  5.26s/it]
 58%|█████▊    | 130196/225000 [262:56:03<173:34:42,  6.59s/it]
 58%|█████▊    | 130197/225000 [262:56:12<194:01:57,  7.37s/it]
 58%|█████▊    | 130198/225000 [262:56:16<163:37:19,  6.21s/it]
 58%|█████▊    | 130199/225000 [262:56:23<167:57:44,  6.38s/it]
 58%|█████▊    | 130200/225000 [262:56:28<156:27:20,  5.94s/it]
                                                               
{'loss': '0.8187', 'grad_norm': '1.02', 'learning_rate': '2.041e-05', 'epoch': '0.5787', 'num_input_tokens_seen': 437067264, 'train_runtime': '9.466e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130200/225000 [262:56:28<156:27:20,  5.94s/it]
 58%|█████▊    | 130201/225000 [262:56:32<148:31:57,  5.64s/it]
 58%|█████▊    | 130202/225000 [262:56:37<139:56:26,  5.31s/it]
 58%|█████▊    | 130203/225000 [262:56:42<135:59:04,  5.16s/it]
 58%|█████▊    | 130204/225000 [262:56:47<135:03:05,  5.13s/it]
 58%|█████▊    | 130205/225000 [262:56:52<135:01:10,  5.13s/it]
 58%|█████▊    | 130206/225000 [262:56:57<131:42:17,  5.00s/it]
 58%|█████▊    | 130207/225000 [262:57:02<134:59:46,  5.13s/it]
 58%|█████▊    | 130208/225000 [262:57:14<189:52:59,  7.21s/it]
 58%|█████▊    | 130209/225000 [262:57:22<193:53:06,  7.36s/it]
 58%|█████▊    | 130210/225000 [262:57:28<182:35:08,  6.93s/it]
 58%|█████▊    | 130211/225000 [262:57:33<168:42:15,  6.41s/it]
 58%|█████▊    | 130212/225000 [262:57:36<145:18:39,  5.52s/it]
 58%|█████▊    | 130213/225000 [262:57:43<150:29:46,  5.72s/it]
 58%|█████▊    | 130214/225000 [262:57:51<174:42:37,  6.64s/it]
 58%|█████▊    | 130215/225000 [262:57:55<150:47:21,  5.73s/it]
 58%|█████▊    | 130216/225000 [262:58:02<162:01:36,  6.15s/it]
 58%|█████▊    | 130217/225000 [262:58:08<156:27:31,  5.94s/it]
 58%|█████▊    | 130218/225000 [262:58:14<161:20:35,  6.13s/it]
 58%|█████▊    | 130219/225000 [262:58:22<174:04:13,  6.61s/it]
 58%|█████▊    | 130220/225000 [262:58:26<157:26:19,  5.98s/it]
 58%|█████▊    | 130221/225000 [262:58:34<166:56:57,  6.34s/it]
 58%|█████▊    | 130222/225000 [262:58:38<148:04:21,  5.62s/it]
 58%|█████▊    | 130223/225000 [262:58:44<155:40:13,  5.91s/it]
 58%|█████▊    | 130224/225000 [262:58:48<136:24:45,  5.18s/it]
 58%|█████▊    | 130225/225000 [262:58:52<129:29:47,  4.92s/it]
 58%|█████▊    | 130226/225000 [262:58:56<121:44:45,  4.62s/it]
 58%|█████▊    | 130227/225000 [262:59:01<125:08:51,  4.75s/it]
 58%|█████▊    | 130228/225000 [262:59:13<184:13:53,  7.00s/it]
 58%|█████▊    | 130229/225000 [262:59:19<173:45:37,  6.60s/it]
 58%|█████▊    | 130230/225000 [262:59:23<151:11:30,  5.74s/it]
 58%|█████▊    | 130231/225000 [262:59:27<143:36:04,  5.45s/it]
 58%|█████▊    | 130232/225000 [262:59:32<133:36:22,  5.08s/it]
 58%|█████▊    | 130233/225000 [262:59:38<140:32:56,  5.34s/it]
 58%|█████▊    | 130234/225000 [262:59:49<188:12:45,  7.15s/it]
 58%|█████▊    | 130235/225000 [262:59:55<177:52:31,  6.76s/it]
 58%|█████▊    | 130236/225000 [262:59:59<158:53:28,  6.04s/it]
 58%|█████▊    | 130237/225000 [263:00:04<146:10:03,  5.55s/it]
 58%|█████▊    | 130238/225000 [263:00:08<135:21:40,  5.14s/it]
 58%|█████▊    | 130239/225000 [263:00:12<130:16:09,  4.95s/it]
 58%|█████▊    | 130240/225000 [263:00:24<187:40:00,  7.13s/it]
 58%|█████▊    | 130241/225000 [263:00:29<171:27:29,  6.51s/it]
 58%|█████▊    | 130242/225000 [263:00:37<175:27:06,  6.67s/it]
 58%|█████▊    | 130243/225000 [263:00:42<168:19:42,  6.40s/it]
 58%|█████▊    | 130244/225000 [263:00:47<154:56:11,  5.89s/it]
 58%|█████▊    | 130245/225000 [263:00:52<148:31:06,  5.64s/it]
 58%|█████▊    | 130246/225000 [263:00:57<139:20:23,  5.29s/it]
 58%|█████▊    | 130247/225000 [263:01:02<139:38:18,  5.31s/it]
 58%|█████▊    | 130248/225000 [263:01:07<139:03:51,  5.28s/it]
 58%|█████▊    | 130249/225000 [263:01:12<136:34:58,  5.19s/it]
 58%|█████▊    | 130250/225000 [263:01:18<139:46:55,  5.31s/it]
                                                               
{'loss': '0.8245', 'grad_norm': '1.019', 'learning_rate': '2.039e-05', 'epoch': '0.5789', 'num_input_tokens_seen': 437195880, 'train_runtime': '9.469e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130250/225000 [263:01:18<139:46:55,  5.31s/it]
 58%|█████▊    | 130251/225000 [263:01:23<139:17:56,  5.29s/it]
 58%|█████▊    | 130252/225000 [263:01:28<137:34:07,  5.23s/it]
 58%|█████▊    | 130253/225000 [263:01:39<181:57:09,  6.91s/it]
 58%|█████▊    | 130254/225000 [263:01:45<176:45:12,  6.72s/it]
 58%|█████▊    | 130255/225000 [263:01:52<181:55:32,  6.91s/it]
 58%|█████▊    | 130256/225000 [263:01:57<161:58:24,  6.15s/it]
 58%|█████▊    | 130257/225000 [263:02:01<147:08:43,  5.59s/it]
 58%|█████▊    | 130258/225000 [263:02:06<139:41:12,  5.31s/it]
 58%|█████▊    | 130259/225000 [263:02:10<130:32:01,  4.96s/it]
 58%|█████▊    | 130260/225000 [263:02:23<191:24:02,  7.27s/it]
 58%|█████▊    | 130261/225000 [263:02:27<167:46:00,  6.37s/it]
 58%|█████▊    | 130262/225000 [263:02:35<180:05:46,  6.84s/it]
 58%|█████▊    | 130263/225000 [263:02:39<162:54:44,  6.19s/it]
 58%|█████▊    | 130264/225000 [263:02:44<153:08:17,  5.82s/it]
 58%|█████▊    | 130265/225000 [263:02:50<149:26:54,  5.68s/it]
 58%|█████▊    | 130266/225000 [263:02:55<143:08:09,  5.44s/it]
 58%|█████▊    | 130267/225000 [263:02:59<137:50:11,  5.24s/it]
 58%|█████▊    | 130268/225000 [263:03:04<136:38:14,  5.19s/it]
 58%|█████▊    | 130269/225000 [263:03:09<134:20:59,  5.11s/it]
 58%|█████▊    | 130270/225000 [263:03:14<134:01:27,  5.09s/it]
 58%|█████▊    | 130271/225000 [263:03:23<160:38:22,  6.10s/it]
 58%|█████▊    | 130272/225000 [263:03:35<206:19:08,  7.84s/it]
 58%|█████▊    | 130273/225000 [263:03:38<172:17:05,  6.55s/it]
 58%|█████▊    | 130274/225000 [263:03:46<184:20:28,  7.01s/it]
 58%|█████▊    | 130275/225000 [263:03:50<154:09:18,  5.86s/it]
 58%|█████▊    | 130276/225000 [263:03:57<163:39:04,  6.22s/it]
 58%|█████▊    | 130277/225000 [263:04:01<151:59:03,  5.78s/it]
 58%|█████▊    | 130278/225000 [263:04:07<147:34:56,  5.61s/it]
 58%|█████▊    | 130279/225000 [263:04:14<161:44:56,  6.15s/it]
 58%|█████▊    | 130280/225000 [263:04:19<151:26:35,  5.76s/it]
 58%|█████▊    | 130281/225000 [263:04:24<146:27:37,  5.57s/it]
 58%|█████▊    | 130282/225000 [263:04:29<139:09:55,  5.29s/it]
 58%|█████▊    | 130283/225000 [263:04:34<137:04:57,  5.21s/it]
 58%|█████▊    | 130284/225000 [263:04:39<137:34:37,  5.23s/it]
 58%|█████▊    | 130285/225000 [263:04:44<137:44:57,  5.24s/it]
 58%|█████▊    | 130286/225000 [263:04:49<137:36:29,  5.23s/it]
 58%|█████▊    | 130287/225000 [263:04:56<147:10:07,  5.59s/it]
 58%|█████▊    | 130288/225000 [263:05:08<195:46:16,  7.44s/it]
 58%|█████▊    | 130289/225000 [263:05:12<168:21:05,  6.40s/it]
 58%|█████▊    | 130290/225000 [263:05:15<147:06:40,  5.59s/it]
 58%|█████▊    | 130291/225000 [263:05:25<181:45:52,  6.91s/it]
 58%|█████▊    | 130292/225000 [263:05:29<154:38:21,  5.88s/it]
 58%|█████▊    | 130293/225000 [263:05:37<171:08:12,  6.51s/it]
 58%|█████▊    | 130294/225000 [263:05:41<153:14:43,  5.83s/it]
 58%|█████▊    | 130295/225000 [263:05:45<141:28:31,  5.38s/it]
 58%|█████▊    | 130296/225000 [263:05:49<132:13:47,  5.03s/it]
 58%|█████▊    | 130297/225000 [263:06:00<178:57:29,  6.80s/it]
 58%|█████▊    | 130298/225000 [263:06:05<161:41:28,  6.15s/it]
 58%|█████▊    | 130299/225000 [263:06:10<149:24:25,  5.68s/it]
 58%|█████▊    | 130300/225000 [263:06:14<141:13:07,  5.37s/it]
                                                               
{'loss': '0.8154', 'grad_norm': '1.11', 'learning_rate': '2.037e-05', 'epoch': '0.5791', 'num_input_tokens_seen': 437325760, 'train_runtime': '9.472e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130300/225000 [263:06:14<141:13:07,  5.37s/it]
 58%|█████▊    | 130301/225000 [263:06:25<180:54:55,  6.88s/it]
 58%|█████▊    | 130302/225000 [263:06:30<165:25:29,  6.29s/it]
 58%|█████▊    | 130303/225000 [263:06:38<179:49:28,  6.84s/it]
 58%|█████▊    | 130304/225000 [263:06:42<158:03:50,  6.01s/it]
 58%|█████▊    | 130305/225000 [263:06:49<169:44:25,  6.45s/it]
 58%|█████▊    | 130306/225000 [263:06:53<151:58:43,  5.78s/it]
 58%|█████▊    | 130307/225000 [263:06:57<138:07:48,  5.25s/it]
 58%|█████▊    | 130308/225000 [263:07:02<128:15:47,  4.88s/it]
 58%|█████▊    | 130309/225000 [263:07:13<176:36:05,  6.71s/it]
 58%|█████▊    | 130310/225000 [263:07:17<156:43:45,  5.96s/it]
 58%|█████▊    | 130311/225000 [263:07:22<150:32:08,  5.72s/it]
 58%|█████▊    | 130312/225000 [263:07:27<147:17:12,  5.60s/it]
 58%|█████▊    | 130313/225000 [263:07:33<145:12:19,  5.52s/it]
 58%|█████▊    | 130314/225000 [263:07:38<144:34:14,  5.50s/it]
 58%|█████▊    | 130315/225000 [263:07:43<140:43:39,  5.35s/it]
 58%|█████▊    | 130316/225000 [263:07:48<135:23:48,  5.15s/it]
 58%|█████▊    | 130317/225000 [263:07:53<135:23:00,  5.15s/it]
 58%|█████▊    | 130318/225000 [263:07:58<134:25:44,  5.11s/it]
 58%|█████▊    | 130319/225000 [263:08:09<181:25:46,  6.90s/it]
 58%|█████▊    | 130320/225000 [263:08:16<184:16:38,  7.01s/it]
 58%|█████▊    | 130321/225000 [263:08:20<158:44:10,  6.04s/it]
 58%|█████▊    | 130322/225000 [263:08:30<193:58:10,  7.38s/it]
 58%|█████▊    | 130323/225000 [263:08:34<164:47:55,  6.27s/it]
 58%|█████▊    | 130324/225000 [263:08:38<145:14:31,  5.52s/it]
 58%|█████▊    | 130325/225000 [263:08:48<180:43:48,  6.87s/it]
 58%|█████▊    | 130326/225000 [263:08:51<153:39:07,  5.84s/it]
 58%|█████▊    | 130327/225000 [263:09:00<175:42:59,  6.68s/it]
 58%|█████▊    | 130328/225000 [263:09:04<151:47:10,  5.77s/it]
 58%|█████▊    | 130329/225000 [263:09:11<164:58:36,  6.27s/it]
 58%|█████▊    | 130330/225000 [263:09:15<143:56:55,  5.47s/it]
 58%|█████▊    | 130331/225000 [263:09:23<167:54:19,  6.38s/it]
 58%|█████▊    | 130332/225000 [263:09:29<160:10:46,  6.09s/it]
 58%|█████▊    | 130333/225000 [263:09:33<144:00:53,  5.48s/it]
 58%|█████▊    | 130334/225000 [263:09:37<132:01:27,  5.02s/it]
 58%|█████▊    | 130335/225000 [263:09:47<176:49:11,  6.72s/it]
 58%|█████▊    | 130336/225000 [263:09:51<153:14:33,  5.83s/it]
 58%|█████▊    | 130337/225000 [263:09:55<140:59:08,  5.36s/it]
 58%|█████▊    | 130338/225000 [263:10:04<165:57:22,  6.31s/it]
 58%|█████▊    | 130339/225000 [263:10:12<178:33:17,  6.79s/it]
 58%|█████▊    | 130340/225000 [263:10:16<158:40:38,  6.03s/it]
 58%|█████▊    | 130341/225000 [263:10:21<149:25:22,  5.68s/it]
 58%|█████▊    | 130342/225000 [263:10:25<140:24:15,  5.34s/it]
 58%|█████▊    | 130343/225000 [263:10:30<134:00:19,  5.10s/it]
 58%|█████▊    | 130344/225000 [263:10:34<128:44:19,  4.90s/it]
 58%|█████▊    | 130345/225000 [263:10:46<185:29:36,  7.05s/it]
 58%|█████▊    | 130346/225000 [263:10:51<165:18:51,  6.29s/it]
 58%|█████▊    | 130347/225000 [263:10:55<148:16:55,  5.64s/it]
 58%|█████▊    | 130348/225000 [263:11:02<156:16:46,  5.94s/it]
 58%|█████▊    | 130349/225000 [263:11:10<171:27:23,  6.52s/it]
 58%|█████▊    | 130350/225000 [263:11:13<147:39:31,  5.62s/it]
                                                               
{'loss': '0.8141', 'grad_norm': '1.181', 'learning_rate': '2.035e-05', 'epoch': '0.5793', 'num_input_tokens_seen': 437454856, 'train_runtime': '9.475e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130350/225000 [263:11:13<147:39:31,  5.62s/it]
 58%|█████▊    | 130351/225000 [263:11:17<136:43:50,  5.20s/it]
 58%|█████▊    | 130352/225000 [263:11:26<166:50:14,  6.35s/it]
 58%|█████▊    | 130353/225000 [263:11:32<160:13:43,  6.09s/it]
 58%|█████▊    | 130354/225000 [263:11:38<163:53:28,  6.23s/it]
 58%|█████▊    | 130355/225000 [263:11:46<177:58:27,  6.77s/it]
 58%|█████▊    | 130356/225000 [263:11:52<166:05:51,  6.32s/it]
 58%|█████▊    | 130357/225000 [263:11:55<143:58:28,  5.48s/it]
 58%|█████▊    | 130358/225000 [263:12:05<178:32:19,  6.79s/it]
 58%|█████▊    | 130359/225000 [263:12:09<154:55:25,  5.89s/it]
 58%|█████▊    | 130360/225000 [263:12:16<160:45:11,  6.11s/it]
 58%|█████▊    | 130361/225000 [263:12:20<146:11:04,  5.56s/it]
 58%|█████▊    | 130362/225000 [263:12:24<133:17:40,  5.07s/it]
 58%|█████▊    | 130363/225000 [263:12:31<150:09:45,  5.71s/it]
 58%|█████▊    | 130364/225000 [263:12:38<160:20:22,  6.10s/it]
 58%|█████▊    | 130365/225000 [263:12:42<147:28:35,  5.61s/it]
 58%|█████▊    | 130366/225000 [263:12:50<160:05:54,  6.09s/it]
 58%|█████▊    | 130367/225000 [263:12:54<143:25:27,  5.46s/it]
 58%|█████▊    | 130368/225000 [263:12:58<137:29:51,  5.23s/it]
 58%|█████▊    | 130369/225000 [263:13:09<178:57:32,  6.81s/it]
 58%|█████▊    | 130370/225000 [263:13:12<152:34:43,  5.80s/it]
 58%|█████▊    | 130371/225000 [263:13:16<134:41:55,  5.12s/it]
 58%|█████▊    | 130372/225000 [263:13:27<182:10:10,  6.93s/it]
 58%|█████▊    | 130373/225000 [263:13:31<156:35:15,  5.96s/it]
 58%|█████▊    | 130374/225000 [263:13:34<136:13:15,  5.18s/it]
 58%|█████▊    | 130375/225000 [263:13:45<183:11:43,  6.97s/it]
 58%|█████▊    | 130376/225000 [263:13:49<160:29:50,  6.11s/it]
 58%|█████▊    | 130377/225000 [263:13:56<168:49:16,  6.42s/it]
 58%|█████▊    | 130378/225000 [263:14:00<145:28:10,  5.53s/it]
 58%|█████▊    | 130379/225000 [263:14:09<173:59:12,  6.62s/it]
 58%|█████▊    | 130380/225000 [263:14:13<155:44:28,  5.93s/it]
 58%|█████▊    | 130381/225000 [263:14:17<141:44:40,  5.39s/it]
 58%|█████▊    | 130382/225000 [263:14:22<131:17:53,  5.00s/it]
 58%|█████▊    | 130383/225000 [263:14:26<126:32:54,  4.81s/it]
 58%|█████▊    | 130384/225000 [263:14:30<123:31:57,  4.70s/it]
 58%|█████▊    | 130385/225000 [263:14:40<165:30:17,  6.30s/it]
 58%|█████▊    | 130386/225000 [263:14:50<188:29:27,  7.17s/it]
 58%|█████▊    | 130387/225000 [263:14:55<172:11:12,  6.55s/it]
 58%|█████▊    | 130388/225000 [263:15:00<162:55:05,  6.20s/it]
 58%|█████▊    | 130389/225000 [263:15:04<142:13:08,  5.41s/it]
 58%|█████▊    | 130390/225000 [263:15:13<170:41:15,  6.49s/it]
 58%|█████▊    | 130391/225000 [263:15:19<165:54:20,  6.31s/it]
 58%|█████▊    | 130392/225000 [263:15:24<155:31:34,  5.92s/it]
 58%|█████▊    | 130393/225000 [263:15:28<146:07:01,  5.56s/it]
 58%|█████▊    | 130394/225000 [263:15:39<188:10:10,  7.16s/it]
 58%|█████▊    | 130395/225000 [263:15:43<165:00:09,  6.28s/it]
 58%|█████▊    | 130396/225000 [263:15:47<144:46:26,  5.51s/it]
 58%|█████▊    | 130397/225000 [263:15:56<169:39:41,  6.46s/it]
 58%|█████▊    | 130398/225000 [263:16:00<149:02:46,  5.67s/it]
 58%|█████▊    | 130399/225000 [263:16:03<132:53:08,  5.06s/it]
 58%|█████▊    | 130400/225000 [263:16:13<170:46:39,  6.50s/it]
                                                               
{'loss': '0.811', 'grad_norm': '1.073', 'learning_rate': '2.034e-05', 'epoch': '0.5796', 'num_input_tokens_seen': 437585032, 'train_runtime': '9.478e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130400/225000 [263:16:13<170:46:39,  6.50s/it]
 58%|█████▊    | 130401/225000 [263:16:17<147:48:22,  5.62s/it]
 58%|█████▊    | 130402/225000 [263:16:26<179:31:39,  6.83s/it]
 58%|█████▊    | 130403/225000 [263:16:30<154:00:56,  5.86s/it]
 58%|█████▊    | 130404/225000 [263:16:38<171:24:54,  6.52s/it]
 58%|█████▊    | 130405/225000 [263:16:42<150:27:15,  5.73s/it]
 58%|█████▊    | 130406/225000 [263:16:47<148:29:12,  5.65s/it]
 58%|█████▊    | 130407/225000 [263:16:55<163:22:22,  6.22s/it]
 58%|█████▊    | 130408/225000 [263:16:58<140:31:23,  5.35s/it]
 58%|█████▊    | 130409/225000 [263:17:07<166:31:03,  6.34s/it]
 58%|█████▊    | 130410/225000 [263:17:10<144:57:24,  5.52s/it]
 58%|█████▊    | 130411/225000 [263:17:15<134:16:56,  5.11s/it]
 58%|█████▊    | 130412/225000 [263:17:19<125:53:43,  4.79s/it]
 58%|█████▊    | 130413/225000 [263:17:29<172:45:51,  6.58s/it]
 58%|█████▊    | 130414/225000 [263:17:35<167:27:32,  6.37s/it]
 58%|█████▊    | 130415/225000 [263:17:40<151:50:13,  5.78s/it]
 58%|█████▊    | 130416/225000 [263:17:44<139:56:16,  5.33s/it]
 58%|█████▊    | 130417/225000 [263:17:48<133:42:26,  5.09s/it]
 58%|█████▊    | 130418/225000 [263:17:57<158:08:05,  6.02s/it]
 58%|█████▊    | 130419/225000 [263:18:06<181:12:29,  6.90s/it]
 58%|█████▊    | 130420/225000 [263:18:09<152:52:20,  5.82s/it]
 58%|█████▊    | 130421/225000 [263:18:18<177:44:55,  6.77s/it]
 58%|█████▊    | 130422/225000 [263:18:24<169:29:38,  6.45s/it]
 58%|█████▊    | 130423/225000 [263:18:28<155:06:45,  5.90s/it]
 58%|█████▊    | 130424/225000 [263:18:32<137:25:02,  5.23s/it]
 58%|█████▊    | 130425/225000 [263:18:42<177:41:57,  6.76s/it]
 58%|█████▊    | 130426/225000 [263:18:45<149:39:36,  5.70s/it]
 58%|█████▊    | 130427/225000 [263:18:50<144:08:50,  5.49s/it]
 58%|█████▊    | 130428/225000 [263:18:56<144:13:02,  5.49s/it]
 58%|█████▊    | 130429/225000 [263:19:01<139:43:11,  5.32s/it]
 58%|█████▊    | 130430/225000 [263:19:06<138:45:31,  5.28s/it]
 58%|█████▊    | 130431/225000 [263:19:11<138:22:10,  5.27s/it]
 58%|█████▊    | 130432/225000 [263:19:17<139:12:12,  5.30s/it]
 58%|█████▊    | 130433/225000 [263:19:22<137:22:23,  5.23s/it]
 58%|█████▊    | 130434/225000 [263:19:34<189:02:48,  7.20s/it]
 58%|█████▊    | 130435/225000 [263:19:42<195:27:17,  7.44s/it]
 58%|█████▊    | 130436/225000 [263:19:46<168:12:50,  6.40s/it]
 58%|█████▊    | 130437/225000 [263:19:53<178:22:41,  6.79s/it]
 58%|█████▊    | 130438/225000 [263:20:00<178:49:58,  6.81s/it]
 58%|█████▊    | 130439/225000 [263:20:04<156:56:01,  5.97s/it]
 58%|█████▊    | 130440/225000 [263:20:10<154:18:15,  5.87s/it]
 58%|█████▊    | 130441/225000 [263:20:20<185:19:31,  7.06s/it]
 58%|█████▊    | 130442/225000 [263:20:23<157:29:48,  6.00s/it]
 58%|█████▊    | 130443/225000 [263:20:33<186:28:52,  7.10s/it]
 58%|█████▊    | 130444/225000 [263:20:36<157:23:54,  5.99s/it]
 58%|█████▊    | 130445/225000 [263:20:43<167:46:41,  6.39s/it]
 58%|█████▊    | 130446/225000 [263:20:50<169:58:34,  6.47s/it]
 58%|█████▊    | 130447/225000 [263:20:54<153:25:31,  5.84s/it]
 58%|█████▊    | 130448/225000 [263:20:59<145:14:14,  5.53s/it]
 58%|█████▊    | 130449/225000 [263:21:04<136:20:00,  5.19s/it]
 58%|█████▊    | 130450/225000 [263:21:08<133:16:21,  5.07s/it]
                                                               
{'loss': '0.8092', 'grad_norm': '0.9832', 'learning_rate': '2.032e-05', 'epoch': '0.5798', 'num_input_tokens_seen': 437715312, 'train_runtime': '9.481e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130450/225000 [263:21:09<133:16:21,  5.07s/it]
 58%|█████▊    | 130451/225000 [263:21:19<176:02:26,  6.70s/it]
 58%|█████▊    | 130452/225000 [263:21:27<185:07:55,  7.05s/it]
 58%|█████▊    | 130453/225000 [263:21:32<170:36:51,  6.50s/it]
 58%|█████▊    | 130454/225000 [263:21:37<157:05:53,  5.98s/it]
 58%|█████▊    | 130455/225000 [263:21:42<151:51:30,  5.78s/it]
 58%|█████▊    | 130456/225000 [263:21:48<148:50:17,  5.67s/it]
 58%|█████▊    | 130457/225000 [263:21:53<145:24:18,  5.54s/it]
 58%|█████▊    | 130458/225000 [263:22:02<171:40:39,  6.54s/it]
 58%|█████▊    | 130459/225000 [263:22:07<163:22:34,  6.22s/it]
 58%|█████▊    | 130460/225000 [263:22:11<145:50:34,  5.55s/it]
 58%|█████▊    | 130461/225000 [263:22:15<135:23:54,  5.16s/it]
 58%|█████▊    | 130462/225000 [263:22:26<175:37:02,  6.69s/it]
 58%|█████▊    | 130463/225000 [263:22:31<167:10:13,  6.37s/it]
 58%|█████▊    | 130464/225000 [263:22:36<152:23:19,  5.80s/it]
 58%|█████▊    | 130465/225000 [263:22:44<168:59:05,  6.44s/it]
 58%|█████▊    | 130466/225000 [263:22:47<145:57:46,  5.56s/it]
 58%|█████▊    | 130467/225000 [263:22:51<131:29:15,  5.01s/it]
 58%|█████▊    | 130468/225000 [263:22:55<120:33:57,  4.59s/it]
 58%|█████▊    | 130469/225000 [263:23:07<180:02:25,  6.86s/it]
 58%|█████▊    | 130470/225000 [263:23:11<158:08:05,  6.02s/it]
 58%|█████▊    | 130471/225000 [263:23:15<144:01:58,  5.49s/it]
 58%|█████▊    | 130472/225000 [263:23:19<134:44:30,  5.13s/it]
 58%|█████▊    | 130473/225000 [263:23:24<130:24:10,  4.97s/it]
 58%|█████▊    | 130474/225000 [263:23:28<124:07:56,  4.73s/it]
 58%|█████▊    | 130475/225000 [263:23:33<122:21:26,  4.66s/it]
 58%|█████▊    | 130476/225000 [263:23:43<169:54:06,  6.47s/it]
 58%|█████▊    | 130477/225000 [263:23:51<180:59:20,  6.89s/it]
 58%|█████▊    | 130478/225000 [263:23:54<150:22:21,  5.73s/it]
 58%|█████▊    | 130479/225000 [263:24:02<169:36:14,  6.46s/it]
 58%|█████▊    | 130480/225000 [263:24:06<149:56:04,  5.71s/it]
 58%|█████▊    | 130481/225000 [263:24:10<135:02:22,  5.14s/it]
 58%|█████▊    | 130482/225000 [263:24:22<189:53:16,  7.23s/it]
 58%|█████▊    | 130483/225000 [263:24:25<158:08:59,  6.02s/it]
 58%|█████▊    | 130484/225000 [263:24:34<181:37:03,  6.92s/it]
 58%|█████▊    | 130485/225000 [263:24:38<152:46:25,  5.82s/it]
 58%|█████▊    | 130486/225000 [263:24:46<175:50:37,  6.70s/it]
 58%|█████▊    | 130487/225000 [263:24:51<156:14:37,  5.95s/it]
 58%|█████▊    | 130488/225000 [263:24:55<140:30:19,  5.35s/it]
 58%|█████▊    | 130489/225000 [263:24:59<130:54:38,  4.99s/it]
 58%|█████▊    | 130490/225000 [263:25:03<126:26:15,  4.82s/it]
 58%|█████▊    | 130491/225000 [263:25:07<122:29:38,  4.67s/it]
 58%|█████▊    | 130492/225000 [263:25:20<185:41:17,  7.07s/it]
 58%|█████▊    | 130493/225000 [263:25:24<163:03:21,  6.21s/it]
 58%|█████▊    | 130494/225000 [263:25:34<192:31:59,  7.33s/it]
 58%|█████▊    | 130495/225000 [263:25:38<161:38:31,  6.16s/it]
 58%|█████▊    | 130496/225000 [263:25:44<162:14:09,  6.18s/it]
 58%|█████▊    | 130497/225000 [263:25:52<175:02:59,  6.67s/it]
 58%|█████▊    | 130498/225000 [263:25:55<150:13:13,  5.72s/it]
 58%|█████▊    | 130499/225000 [263:26:05<178:44:40,  6.81s/it]
 58%|█████▊    | 130500/225000 [263:26:08<153:13:25,  5.84s/it]
                                                               
{'loss': '0.7993', 'grad_norm': '1.048', 'learning_rate': '2.03e-05', 'epoch': '0.58', 'num_input_tokens_seen': 437844824, 'train_runtime': '9.484e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130500/225000 [263:26:08<153:13:25,  5.84s/it]
 58%|█████▊    | 130501/225000 [263:26:16<167:13:31,  6.37s/it]
 58%|█████▊    | 130502/225000 [263:26:20<150:29:12,  5.73s/it]
 58%|█████▊    | 130503/225000 [263:26:24<137:49:07,  5.25s/it]
 58%|█████▊    | 130504/225000 [263:26:28<128:18:00,  4.89s/it]
 58%|█████▊    | 130505/225000 [263:26:38<163:48:03,  6.24s/it]
 58%|█████▊    | 130506/225000 [263:26:43<159:48:04,  6.09s/it]
 58%|█████▊    | 130507/225000 [263:26:50<165:33:36,  6.31s/it]
 58%|█████▊    | 130508/225000 [263:26:56<161:09:37,  6.14s/it]
 58%|█████▊    | 130509/225000 [263:27:04<173:32:16,  6.61s/it]
 58%|█████▊    | 130510/225000 [263:27:08<159:13:38,  6.07s/it]
 58%|█████▊    | 130511/225000 [263:27:13<148:13:38,  5.65s/it]
 58%|█████▊    | 130512/225000 [263:27:18<139:10:00,  5.30s/it]
 58%|█████▊    | 130513/225000 [263:27:22<132:23:04,  5.04s/it]
 58%|█████▊    | 130514/225000 [263:27:26<125:21:01,  4.78s/it]
 58%|█████▊    | 130515/225000 [263:27:31<123:30:24,  4.71s/it]
 58%|█████▊    | 130516/225000 [263:27:39<149:09:58,  5.68s/it]
 58%|█████▊    | 130517/225000 [263:27:49<184:55:01,  7.05s/it]
 58%|█████▊    | 130518/225000 [263:27:53<162:26:18,  6.19s/it]
 58%|█████▊    | 130519/225000 [263:27:57<147:37:00,  5.62s/it]
 58%|█████▊    | 130520/225000 [263:28:06<172:00:40,  6.55s/it]
 58%|█████▊    | 130521/225000 [263:28:14<182:18:01,  6.95s/it]
 58%|█████▊    | 130522/225000 [263:28:19<164:28:45,  6.27s/it]
 58%|█████▊    | 130523/225000 [263:28:23<150:46:23,  5.75s/it]
 58%|█████▊    | 130524/225000 [263:28:27<139:42:07,  5.32s/it]
 58%|█████▊    | 130525/225000 [263:28:34<151:07:22,  5.76s/it]
 58%|█████▊    | 130526/225000 [263:28:43<178:07:02,  6.79s/it]
 58%|█████▊    | 130527/225000 [263:28:47<150:36:09,  5.74s/it]
 58%|█████▊    | 130528/225000 [263:28:53<157:08:32,  5.99s/it]
 58%|█████▊    | 130529/225000 [263:29:00<159:11:10,  6.07s/it]
 58%|█████▊    | 130530/225000 [263:29:03<140:38:52,  5.36s/it]
 58%|█████▊    | 130531/225000 [263:29:07<127:48:51,  4.87s/it]
 58%|█████▊    | 130532/225000 [263:29:18<177:01:42,  6.75s/it]
 58%|█████▊    | 130533/225000 [263:29:22<153:01:15,  5.83s/it]
 58%|█████▊    | 130534/225000 [263:29:26<142:30:29,  5.43s/it]
 58%|█████▊    | 130535/225000 [263:29:34<158:45:26,  6.05s/it]
 58%|█████▊    | 130536/225000 [263:29:41<165:50:23,  6.32s/it]
 58%|█████▊    | 130537/225000 [263:29:46<154:03:13,  5.87s/it]
 58%|█████▊    | 130538/225000 [263:29:49<135:41:01,  5.17s/it]
 58%|█████▊    | 130539/225000 [263:29:54<130:08:00,  4.96s/it]
 58%|█████▊    | 130540/225000 [263:29:58<125:08:19,  4.77s/it]
 58%|█████▊    | 130541/225000 [263:30:07<161:37:27,  6.16s/it]
 58%|█████▊    | 130542/225000 [263:30:15<170:46:29,  6.51s/it]
 58%|█████▊    | 130543/225000 [263:30:18<149:17:53,  5.69s/it]
 58%|█████▊    | 130544/225000 [263:30:25<156:33:54,  5.97s/it]
 58%|█████▊    | 130545/225000 [263:30:33<173:13:10,  6.60s/it]
 58%|█████▊    | 130546/225000 [263:30:38<157:55:09,  6.02s/it]
 58%|█████▊    | 130547/225000 [263:30:42<146:07:26,  5.57s/it]
 58%|█████▊    | 130548/225000 [263:30:47<140:00:52,  5.34s/it]
 58%|█████▊    | 130549/225000 [263:30:51<132:17:52,  5.04s/it]
 58%|█████▊    | 130550/225000 [263:31:05<195:31:09,  7.45s/it]
                                                               
{'loss': '0.8249', 'grad_norm': '1.112', 'learning_rate': '2.028e-05', 'epoch': '0.5802', 'num_input_tokens_seen': 437973744, 'train_runtime': '9.487e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130550/225000 [263:31:05<195:31:09,  7.45s/it]
 58%|█████▊    | 130551/225000 [263:31:08<167:44:09,  6.39s/it]
 58%|█████▊    | 130552/225000 [263:31:12<146:49:00,  5.60s/it]
 58%|█████▊    | 130553/225000 [263:31:20<165:23:16,  6.30s/it]
 58%|█████▊    | 130554/225000 [263:31:26<165:36:18,  6.31s/it]
 58%|█████▊    | 130555/225000 [263:31:31<152:29:30,  5.81s/it]
 58%|█████▊    | 130556/225000 [263:31:39<167:15:05,  6.38s/it]
 58%|█████▊    | 130557/225000 [263:31:43<151:46:11,  5.79s/it]
 58%|█████▊    | 130558/225000 [263:31:48<143:10:37,  5.46s/it]
 58%|█████▊    | 130559/225000 [263:31:52<133:42:35,  5.10s/it]
 58%|█████▊    | 130560/225000 [263:31:56<127:50:11,  4.87s/it]
 58%|█████▊    | 130561/225000 [263:32:06<165:47:14,  6.32s/it]
 58%|█████▊    | 130562/225000 [263:32:19<219:03:35,  8.35s/it]
 58%|█████▊    | 130563/225000 [263:32:23<185:20:15,  7.07s/it]
 58%|█████▊    | 130564/225000 [263:32:29<173:24:16,  6.61s/it]
 58%|█████▊    | 130565/225000 [263:32:34<164:02:11,  6.25s/it]
 58%|█████▊    | 130566/225000 [263:32:40<158:10:15,  6.03s/it]
 58%|█████▊    | 130567/225000 [263:32:45<151:49:12,  5.79s/it]
 58%|█████▊    | 130568/225000 [263:32:50<144:31:24,  5.51s/it]
 58%|█████▊    | 130569/225000 [263:32:55<141:50:25,  5.41s/it]
 58%|█████▊    | 130570/225000 [263:33:00<137:31:35,  5.24s/it]
 58%|█████▊    | 130571/225000 [263:33:05<136:35:12,  5.21s/it]
 58%|█████▊    | 130572/225000 [263:33:10<136:41:39,  5.21s/it]
 58%|█████▊    | 130573/225000 [263:33:19<167:48:15,  6.40s/it]
 58%|█████▊    | 130574/225000 [263:33:24<157:07:00,  5.99s/it]
 58%|█████▊    | 130575/225000 [263:33:30<152:57:24,  5.83s/it]
 58%|█████▊    | 130576/225000 [263:33:35<150:21:34,  5.73s/it]
 58%|█████▊    | 130577/225000 [263:33:41<146:04:45,  5.57s/it]
 58%|█████▊    | 130578/225000 [263:33:46<143:55:49,  5.49s/it]
 58%|█████▊    | 130579/225000 [263:33:51<141:42:02,  5.40s/it]
 58%|█████▊    | 130580/225000 [263:34:01<180:33:52,  6.88s/it]
 58%|█████▊    | 130581/225000 [263:34:09<187:52:22,  7.16s/it]
 58%|█████▊    | 130582/225000 [263:34:14<168:21:59,  6.42s/it]
 58%|█████▊    | 130583/225000 [263:34:19<158:47:34,  6.05s/it]
 58%|█████▊    | 130584/225000 [263:34:24<148:05:39,  5.65s/it]
 58%|█████▊    | 130585/225000 [263:34:29<141:42:21,  5.40s/it]
 58%|█████▊    | 130586/225000 [263:34:34<138:14:35,  5.27s/it]
 58%|█████▊    | 130587/225000 [263:34:39<136:52:51,  5.22s/it]
 58%|█████▊    | 130588/225000 [263:34:44<137:08:38,  5.23s/it]
 58%|█████▊    | 130589/225000 [263:34:50<139:23:16,  5.32s/it]
 58%|█████▊    | 130590/225000 [263:35:02<194:54:47,  7.43s/it]
 58%|█████▊    | 130591/225000 [263:35:07<179:47:00,  6.86s/it]
 58%|█████▊    | 130592/225000 [263:35:13<168:34:03,  6.43s/it]
 58%|█████▊    | 130593/225000 [263:35:18<155:26:27,  5.93s/it]
 58%|█████▊    | 130594/225000 [263:35:22<146:00:08,  5.57s/it]
 58%|█████▊    | 130595/225000 [263:35:33<187:37:42,  7.15s/it]
 58%|█████▊    | 130596/225000 [263:35:39<178:34:32,  6.81s/it]
 58%|█████▊    | 130597/225000 [263:35:44<162:29:30,  6.20s/it]
 58%|█████▊    | 130598/225000 [263:35:49<151:02:59,  5.76s/it]
 58%|█████▊    | 130599/225000 [263:35:54<143:30:39,  5.47s/it]
 58%|█████▊    | 130600/225000 [263:35:59<140:10:05,  5.35s/it]
                                                               
{'loss': '0.8206', 'grad_norm': '0.9661', 'learning_rate': '2.027e-05', 'epoch': '0.5804', 'num_input_tokens_seen': 438105456, 'train_runtime': '9.49e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130600/225000 [263:35:59<140:10:05,  5.35s/it]
 58%|█████▊    | 130601/225000 [263:36:03<133:50:26,  5.10s/it]
 58%|█████▊    | 130602/225000 [263:36:11<157:34:30,  6.01s/it]
 58%|█████▊    | 130603/225000 [263:36:21<184:32:50,  7.04s/it]
 58%|█████▊    | 130604/225000 [263:36:25<163:48:36,  6.25s/it]
 58%|█████▊    | 130605/225000 [263:36:30<150:33:49,  5.74s/it]
 58%|█████▊    | 130606/225000 [263:36:34<137:58:03,  5.26s/it]
 58%|█████▊    | 130607/225000 [263:36:41<157:13:10,  6.00s/it]
 58%|█████▊    | 130608/225000 [263:36:53<200:54:30,  7.66s/it]
 58%|█████▊    | 130609/225000 [263:36:56<165:08:06,  6.30s/it]
 58%|█████▊    | 130610/225000 [263:37:01<154:47:57,  5.90s/it]
 58%|█████▊    | 130611/225000 [263:37:06<149:15:14,  5.69s/it]
 58%|█████▊    | 130612/225000 [263:37:14<165:53:29,  6.33s/it]
 58%|█████▊    | 130613/225000 [263:37:20<158:55:59,  6.06s/it]
 58%|█████▊    | 130614/225000 [263:37:26<159:57:05,  6.10s/it]
 58%|█████▊    | 130615/225000 [263:37:29<140:47:03,  5.37s/it]
 58%|█████▊    | 130616/225000 [263:37:34<131:49:19,  5.03s/it]
 58%|█████▊    | 130617/225000 [263:37:38<127:21:27,  4.86s/it]
 58%|█████▊    | 130618/225000 [263:37:42<119:45:27,  4.57s/it]
 58%|█████▊    | 130619/225000 [263:37:54<178:25:06,  6.81s/it]
 58%|█████▊    | 130620/225000 [263:37:59<164:13:31,  6.26s/it]
 58%|█████▊    | 130621/225000 [263:38:03<146:05:34,  5.57s/it]
 58%|█████▊    | 130622/225000 [263:38:07<133:16:09,  5.08s/it]
 58%|█████▊    | 130623/225000 [263:38:17<175:13:17,  6.68s/it]
 58%|█████▊    | 130624/225000 [263:38:21<152:28:44,  5.82s/it]
 58%|█████▊    | 130625/225000 [263:38:25<136:40:07,  5.21s/it]
 58%|█████▊    | 130626/225000 [263:38:31<146:07:26,  5.57s/it]
 58%|█████▊    | 130627/225000 [263:38:40<169:46:57,  6.48s/it]
 58%|█████▊    | 130628/225000 [263:38:44<147:31:15,  5.63s/it]
 58%|█████▊    | 130629/225000 [263:38:47<132:11:59,  5.04s/it]
 58%|█████▊    | 130630/225000 [263:38:57<172:18:01,  6.57s/it]
 58%|█████▊    | 130631/225000 [263:39:01<150:24:32,  5.74s/it]
 58%|█████▊    | 130632/225000 [263:39:07<151:19:39,  5.77s/it]
 58%|█████▊    | 130633/225000 [263:39:15<171:25:02,  6.54s/it]
 58%|█████▊    | 130634/225000 [263:39:19<148:22:53,  5.66s/it]
 58%|█████▊    | 130635/225000 [263:39:28<172:52:32,  6.60s/it]
 58%|█████▊    | 130636/225000 [263:39:31<147:49:42,  5.64s/it]
 58%|█████▊    | 130637/225000 [263:39:40<175:27:25,  6.69s/it]
 58%|█████▊    | 130638/225000 [263:39:44<151:46:48,  5.79s/it]
 58%|█████▊    | 130639/225000 [263:39:50<151:13:15,  5.77s/it]
 58%|█████▊    | 130640/225000 [263:39:57<166:05:59,  6.34s/it]
 58%|█████▊    | 130641/225000 [263:40:03<162:51:24,  6.21s/it]
 58%|█████▊    | 130642/225000 [263:40:09<161:27:00,  6.16s/it]
 58%|█████▊    | 130643/225000 [263:40:14<151:55:44,  5.80s/it]
 58%|█████▊    | 130644/225000 [263:40:19<146:52:27,  5.60s/it]
 58%|█████▊    | 130645/225000 [263:40:25<146:15:09,  5.58s/it]
 58%|█████▊    | 130646/225000 [263:40:30<144:42:23,  5.52s/it]
 58%|█████▊    | 130647/225000 [263:40:35<139:49:02,  5.33s/it]
 58%|█████▊    | 130648/225000 [263:40:41<139:21:29,  5.32s/it]
 58%|█████▊    | 130649/225000 [263:40:46<136:39:22,  5.21s/it]
 58%|█████▊    | 130650/225000 [263:40:51<136:20:40,  5.20s/it]
                                                               
{'loss': '0.8162', 'grad_norm': '1.085', 'learning_rate': '2.025e-05', 'epoch': '0.5807', 'num_input_tokens_seen': 438234616, 'train_runtime': '9.493e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130650/225000 [263:40:51<136:20:40,  5.20s/it]
 58%|█████▊    | 130651/225000 [263:40:56<134:02:16,  5.11s/it]
 58%|█████▊    | 130652/225000 [263:41:01<138:13:38,  5.27s/it]
 58%|█████▊    | 130653/225000 [263:41:10<167:23:20,  6.39s/it]
 58%|█████▊    | 130654/225000 [263:41:20<195:49:36,  7.47s/it]
 58%|█████▊    | 130655/225000 [263:41:24<162:41:36,  6.21s/it]
 58%|█████▊    | 130656/225000 [263:41:27<144:09:29,  5.50s/it]
 58%|█████▊    | 130657/225000 [263:41:40<197:27:50,  7.53s/it]
 58%|█████▊    | 130658/225000 [263:41:43<162:36:14,  6.20s/it]
 58%|█████▊    | 130659/225000 [263:41:48<152:41:40,  5.83s/it]
 58%|█████▊    | 130660/225000 [263:41:52<143:45:46,  5.49s/it]
 58%|█████▊    | 130661/225000 [263:41:57<135:47:43,  5.18s/it]
 58%|█████▊    | 130662/225000 [263:42:03<142:27:09,  5.44s/it]
 58%|█████▊    | 130663/225000 [263:42:15<193:51:09,  7.40s/it]
 58%|█████▊    | 130664/225000 [263:42:19<171:37:30,  6.55s/it]
 58%|█████▊    | 130665/225000 [263:42:23<147:47:59,  5.64s/it]
 58%|█████▊    | 130666/225000 [263:42:31<165:20:34,  6.31s/it]
 58%|█████▊    | 130667/225000 [263:42:38<175:23:49,  6.69s/it]
 58%|█████▊    | 130668/225000 [263:42:43<155:21:08,  5.93s/it]
 58%|█████▊    | 130669/225000 [263:42:46<139:36:06,  5.33s/it]
 58%|█████▊    | 130670/225000 [263:42:51<131:00:43,  5.00s/it]
 58%|█████▊    | 130671/225000 [263:43:03<190:42:23,  7.28s/it]
 58%|█████▊    | 130672/225000 [263:43:07<165:31:49,  6.32s/it]
 58%|█████▊    | 130673/225000 [263:43:12<150:30:06,  5.74s/it]
 58%|█████▊    | 130674/225000 [263:43:19<163:51:30,  6.25s/it]
 58%|█████▊    | 130675/225000 [263:43:26<168:15:22,  6.42s/it]
 58%|█████▊    | 130676/225000 [263:43:32<165:48:19,  6.33s/it]
 58%|█████▊    | 130677/225000 [263:43:38<161:08:16,  6.15s/it]
 58%|█████▊    | 130678/225000 [263:43:41<140:29:31,  5.36s/it]
 58%|█████▊    | 130679/225000 [263:43:50<168:01:08,  6.41s/it]
 58%|█████▊    | 130680/225000 [263:43:54<148:41:59,  5.68s/it]
 58%|█████▊    | 130681/225000 [263:43:59<138:13:08,  5.28s/it]
 58%|█████▊    | 130682/225000 [263:44:03<130:10:11,  4.97s/it]
 58%|█████▊    | 130683/225000 [263:44:11<154:51:15,  5.91s/it]
 58%|█████▊    | 130684/225000 [263:44:21<184:39:47,  7.05s/it]
 58%|█████▊    | 130685/225000 [263:44:25<161:58:12,  6.18s/it]
 58%|█████▊    | 130686/225000 [263:44:29<146:34:50,  5.60s/it]
 58%|█████▊    | 130687/225000 [263:44:39<179:06:19,  6.84s/it]
 58%|█████▊    | 130688/225000 [263:44:44<162:54:34,  6.22s/it]
 58%|█████▊    | 130689/225000 [263:44:48<149:50:38,  5.72s/it]
 58%|█████▊    | 130690/225000 [263:44:53<139:34:13,  5.33s/it]
 58%|█████▊    | 130691/225000 [263:44:57<133:21:40,  5.09s/it]
 58%|█████▊    | 130692/225000 [263:45:05<157:21:23,  6.01s/it]
 58%|█████▊    | 130693/225000 [263:45:14<176:54:36,  6.75s/it]
 58%|█████▊    | 130694/225000 [263:45:18<154:00:01,  5.88s/it]
 58%|█████▊    | 130695/225000 [263:45:23<153:02:16,  5.84s/it]
 58%|█████▊    | 130696/225000 [263:45:31<169:06:59,  6.46s/it]
 58%|█████▊    | 130697/225000 [263:45:35<150:24:27,  5.74s/it]
 58%|█████▊    | 130698/225000 [263:45:40<140:40:48,  5.37s/it]
 58%|█████▊    | 130699/225000 [263:45:44<131:30:41,  5.02s/it]
 58%|█████▊    | 130700/225000 [263:45:49<129:35:44,  4.95s/it]
                                                               
{'loss': '0.8228', 'grad_norm': '1.02', 'learning_rate': '2.023e-05', 'epoch': '0.5809', 'num_input_tokens_seen': 438365720, 'train_runtime': '9.495e+05', 'train_tokens_per_second': '461.7'}

 58%|█████▊    | 130700/225000 [263:45:49<129:35:44,  4.95s/it]
 58%|█████▊    | 130701/225000 [263:45:54<131:52:55,  5.03s/it]
 58%|█████▊    | 130702/225000 [263:45:59<134:55:53,  5.15s/it]
 58%|█████▊    | 130703/225000 [263:46:04<133:35:32,  5.10s/it]
 58%|█████▊    | 130704/225000 [263:46:10<136:44:29,  5.22s/it]
 58%|█████▊    | 130705/225000 [263:46:15<139:46:06,  5.34s/it]
 58%|█████▊    | 130706/225000 [263:46:24<166:55:47,  6.37s/it]
 58%|█████▊    | 130707/225000 [263:46:36<205:29:59,  7.85s/it]
 58%|█████▊    | 130708/225000 [263:46:40<182:08:02,  6.95s/it]
 58%|█████▊    | 130709/225000 [263:46:46<167:43:04,  6.40s/it]
 58%|█████▊    | 130710/225000 [263:46:51<157:34:34,  6.02s/it]
 58%|█████▊    | 130711/225000 [263:46:55<146:27:16,  5.59s/it]
 58%|█████▊    | 130712/225000 [263:47:00<143:14:58,  5.47s/it]
 58%|█████▊    | 130713/225000 [263:47:06<140:42:43,  5.37s/it]
 58%|█████▊    | 130714/225000 [263:47:11<138:18:36,  5.28s/it]
 58%|█████▊    | 130715/225000 [263:47:18<155:10:19,  5.92s/it]
 58%|█████▊    | 130716/225000 [263:47:30<202:38:26,  7.74s/it]
 58%|█████▊    | 130717/225000 [263:47:34<172:23:55,  6.58s/it]
 58%|█████▊    | 130718/225000 [263:47:38<152:18:49,  5.82s/it]
 58%|█████▊    | 130719/225000 [263:47:45<162:56:38,  6.22s/it]
 58%|█████▊    | 130720/225000 [263:47:55<187:50:39,  7.17s/it]
 58%|█████▊    | 130721/225000 [263:47:58<160:37:01,  6.13s/it]
 58%|█████▊    | 130722/225000 [263:48:08<191:59:28,  7.33s/it]
 58%|█████▊    | 130723/225000 [263:48:12<163:09:05,  6.23s/it]
 58%|█████▊    | 130724/225000 [263:48:20<173:36:20,  6.63s/it]
 58%|█████▊    | 130725/225000 [263:48:24<159:19:58,  6.08s/it]
 58%|█████▊    | 130726/225000 [263:48:28<142:28:42,  5.44s/it]
 58%|█████▊    | 130727/225000 [263:48:32<129:43:36,  4.95s/it]
 58%|█████▊    | 130728/225000 [263:48:39<146:08:00,  5.58s/it]
 58%|█████▊    | 130729/225000 [263:48:48<171:56:28,  6.57s/it]
 58%|█████▊    | 130730/225000 [263:48:53<162:20:00,  6.20s/it]
 58%|█████▊    | 130731/225000 [263:49:01<173:51:31,  6.64s/it]
 58%|█████▊    | 130732/225000 [263:49:05<149:55:15,  5.73s/it]
 58%|█████▊    | 130733/225000 [263:49:14<177:13:01,  6.77s/it]
 58%|█████▊    | 130734/225000 [263:49:18<160:07:40,  6.12s/it]
 58%|█████▊    | 130735/225000 [263:49:23<146:15:15,  5.59s/it]
 58%|█████▊    | 130736/225000 [263:49:27<137:27:28,  5.25s/it]
 58%|█████▊    | 130737/225000 [263:49:32<132:04:37,  5.04s/it]
 58%|█████▊    | 130738/225000 [263:49:36<126:23:18,  4.83s/it]
 58%|█████▊    | 130739/225000 [263:49:42<134:54:33,  5.15s/it]
 58%|█████▊    | 130740/225000 [263:49:54<191:40:52,  7.32s/it]
 58%|█████▊    | 130741/225000 [263:49:59<169:56:32,  6.49s/it]
 58%|█████▊    | 130742/225000 [263:50:03<153:15:37,  5.85s/it]
 58%|█████▊    | 130743/225000 [263:50:08<142:21:23,  5.44s/it]
 58%|█████▊    | 130744/225000 [263:50:13<136:21:00,  5.21s/it]
 58%|█████▊    | 130745/225000 [263:50:19<146:04:46,  5.58s/it]
 58%|█████▊    | 130746/225000 [263:50:30<185:16:52,  7.08s/it]
 58%|█████▊    | 130747/225000 [263:50:35<170:19:36,  6.51s/it]
 58%|█████▊    | 130748/225000 [263:50:39<153:21:14,  5.86s/it]
 58%|█████▊    | 130749/225000 [263:50:44<142:49:01,  5.46s/it]
 58%|█████▊    | 130750/225000 [263:50:53<175:13:41,  6.69s/it]
                                                               
{'loss': '0.8144', 'grad_norm': '1.211', 'learning_rate': '2.021e-05', 'epoch': '0.5811', 'num_input_tokens_seen': 438496296, 'train_runtime': '9.499e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 130750/225000 [263:50:53<175:13:41,  6.69s/it]
 58%|█████▊    | 130751/225000 [263:50:57<153:30:51,  5.86s/it]
 58%|█████▊    | 130752/225000 [263:51:01<140:11:59,  5.36s/it]
 58%|█████▊    | 130753/225000 [263:51:05<131:04:00,  5.01s/it]
 58%|█████▊    | 130754/225000 [263:51:17<185:25:53,  7.08s/it]
 58%|█████▊    | 130755/225000 [263:51:22<163:19:12,  6.24s/it]
 58%|█████▊    | 130756/225000 [263:51:26<147:46:45,  5.64s/it]
 58%|█████▊    | 130757/225000 [263:51:32<148:20:40,  5.67s/it]
 58%|█████▊    | 130758/225000 [263:51:40<170:06:37,  6.50s/it]
 58%|█████▊    | 130759/225000 [263:51:44<151:26:50,  5.79s/it]
 58%|█████▊    | 130760/225000 [263:51:48<137:29:16,  5.25s/it]
 58%|█████▊    | 130761/225000 [263:51:52<128:42:04,  4.92s/it]
 58%|█████▊    | 130762/225000 [263:52:03<172:46:17,  6.60s/it]
 58%|█████▊    | 130763/225000 [263:52:10<174:03:12,  6.65s/it]
 58%|█████▊    | 130764/225000 [263:52:15<162:32:08,  6.21s/it]
 58%|█████▊    | 130765/225000 [263:52:21<161:33:56,  6.17s/it]
 58%|█████▊    | 130766/225000 [263:52:24<140:29:44,  5.37s/it]
 58%|█████▊    | 130767/225000 [263:52:32<158:57:22,  6.07s/it]
 58%|█████▊    | 130768/225000 [263:52:36<141:28:29,  5.40s/it]
 58%|█████▊    | 130769/225000 [263:52:46<180:57:25,  6.91s/it]
 58%|█████▊    | 130770/225000 [263:52:50<153:14:15,  5.85s/it]
 58%|█████▊    | 130771/225000 [263:52:58<171:12:51,  6.54s/it]
 58%|█████▊    | 130772/225000 [263:53:01<147:51:48,  5.65s/it]
 58%|█████▊    | 130773/225000 [263:53:06<140:44:12,  5.38s/it]
 58%|█████▊    | 130774/225000 [263:53:11<136:44:43,  5.22s/it]
 58%|█████▊    | 130775/225000 [263:53:16<131:47:03,  5.04s/it]
 58%|█████▊    | 130776/225000 [263:53:21<133:32:17,  5.10s/it]
 58%|█████▊    | 130777/225000 [263:53:26<129:47:02,  4.96s/it]
 58%|█████▊    | 130778/225000 [263:53:31<130:42:45,  4.99s/it]
 58%|█████▊    | 130779/225000 [263:53:39<159:37:41,  6.10s/it]
 58%|█████▊    | 130780/225000 [263:53:49<190:33:44,  7.28s/it]
 58%|█████▊    | 130781/225000 [263:53:53<164:34:22,  6.29s/it]
 58%|█████▊    | 130782/225000 [263:53:58<148:59:21,  5.69s/it]
 58%|█████▊    | 130783/225000 [263:54:02<135:57:05,  5.19s/it]
 58%|█████▊    | 130784/225000 [263:54:06<130:52:08,  5.00s/it]
 58%|█████▊    | 130785/225000 [263:54:18<185:07:02,  7.07s/it]
 58%|█████▊    | 130786/225000 [263:54:23<170:21:48,  6.51s/it]
 58%|█████▊    | 130787/225000 [263:54:27<149:21:04,  5.71s/it]
 58%|█████▊    | 130788/225000 [263:54:31<135:44:42,  5.19s/it]
 58%|█████▊    | 130789/225000 [263:54:41<174:43:23,  6.68s/it]
 58%|█████▊    | 130790/225000 [263:54:49<186:38:08,  7.13s/it]
 58%|█████▊    | 130791/225000 [263:54:53<162:14:14,  6.20s/it]
 58%|█████▊    | 130792/225000 [263:54:58<145:37:56,  5.57s/it]
 58%|█████▊    | 130793/225000 [263:55:01<132:31:45,  5.06s/it]
 58%|█████▊    | 130794/225000 [263:55:12<178:29:53,  6.82s/it]
 58%|█████▊    | 130795/225000 [263:55:18<165:57:02,  6.34s/it]
 58%|█████▊    | 130796/225000 [263:55:25<176:14:01,  6.73s/it]
 58%|█████▊    | 130797/225000 [263:55:28<148:52:12,  5.69s/it]
 58%|█████▊    | 130798/225000 [263:55:38<177:13:36,  6.77s/it]
 58%|█████▊    | 130799/225000 [263:55:41<149:03:16,  5.70s/it]
 58%|█████▊    | 130800/225000 [263:55:44<131:56:42,  5.04s/it]
                                                               
{'loss': '0.8102', 'grad_norm': '1.164', 'learning_rate': '2.019e-05', 'epoch': '0.5813', 'num_input_tokens_seen': 438625560, 'train_runtime': '9.501e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 130800/225000 [263:55:44<131:56:42,  5.04s/it]
 58%|█████▊    | 130801/225000 [263:55:55<171:46:18,  6.56s/it]
 58%|█████▊    | 130802/225000 [263:55:58<149:41:51,  5.72s/it]
 58%|█████▊    | 130803/225000 [263:56:02<136:22:14,  5.21s/it]
 58%|█████▊    | 130804/225000 [263:56:12<171:39:38,  6.56s/it]
 58%|█████▊    | 130805/225000 [263:56:19<172:50:25,  6.61s/it]
 58%|█████▊    | 130806/225000 [263:56:23<150:25:09,  5.75s/it]
 58%|█████▊    | 130807/225000 [263:56:26<136:00:30,  5.20s/it]
 58%|█████▊    | 130808/225000 [263:56:31<129:25:21,  4.95s/it]
 58%|█████▊    | 130809/225000 [263:56:42<176:45:21,  6.76s/it]
 58%|█████▊    | 130810/225000 [263:56:45<149:13:16,  5.70s/it]
 58%|█████▊    | 130811/225000 [263:56:53<166:51:59,  6.38s/it]
 58%|█████▊    | 130812/225000 [263:56:58<159:30:27,  6.10s/it]
 58%|█████▊    | 130813/225000 [263:57:04<157:58:20,  6.04s/it]
 58%|█████▊    | 130814/225000 [263:57:12<170:38:32,  6.52s/it]
 58%|█████▊    | 130815/225000 [263:57:17<160:19:31,  6.13s/it]
 58%|█████▊    | 130816/225000 [263:57:21<142:14:12,  5.44s/it]
 58%|█████▊    | 130817/225000 [263:57:25<128:13:16,  4.90s/it]
 58%|█████▊    | 130818/225000 [263:57:35<172:03:32,  6.58s/it]
 58%|█████▊    | 130819/225000 [263:57:39<154:05:05,  5.89s/it]
 58%|█████▊    | 130820/225000 [263:57:45<148:03:46,  5.66s/it]
 58%|█████▊    | 130821/225000 [263:57:50<144:43:55,  5.53s/it]
 58%|█████▊    | 130822/225000 [263:57:55<141:38:00,  5.41s/it]
 58%|█████▊    | 130823/225000 [263:58:00<142:25:04,  5.44s/it]
 58%|█████▊    | 130824/225000 [263:58:06<141:08:10,  5.40s/it]
 58%|█████▊    | 130825/225000 [263:58:11<142:37:17,  5.45s/it]
 58%|█████▊    | 130826/225000 [263:58:17<143:26:44,  5.48s/it]
 58%|█████▊    | 130827/225000 [263:58:22<141:02:31,  5.39s/it]
 58%|█████▊    | 130828/225000 [263:58:27<138:14:20,  5.28s/it]
 58%|█████▊    | 130829/225000 [263:58:32<137:33:22,  5.26s/it]
 58%|█████▊    | 130830/225000 [263:58:37<135:45:58,  5.19s/it]
 58%|█████▊    | 130831/225000 [263:58:48<180:57:53,  6.92s/it]
 58%|█████▊    | 130832/225000 [263:58:55<182:54:20,  6.99s/it]
 58%|█████▊    | 130833/225000 [263:58:59<155:17:58,  5.94s/it]
 58%|█████▊    | 130834/225000 [263:59:09<185:20:41,  7.09s/it]
 58%|█████▊    | 130835/225000 [263:59:15<178:21:08,  6.82s/it]
 58%|█████▊    | 130836/225000 [263:59:18<152:58:18,  5.85s/it]
 58%|█████▊    | 130837/225000 [263:59:27<174:27:41,  6.67s/it]
 58%|█████▊    | 130838/225000 [263:59:31<153:46:16,  5.88s/it]
 58%|█████▊    | 130839/225000 [263:59:35<140:25:08,  5.37s/it]
 58%|█████▊    | 130840/225000 [263:59:39<130:16:52,  4.98s/it]
 58%|█████▊    | 130841/225000 [263:59:51<180:59:04,  6.92s/it]
 58%|█████▊    | 130842/225000 [263:59:55<156:44:55,  5.99s/it]
 58%|█████▊    | 130843/225000 [263:59:59<141:57:17,  5.43s/it]
 58%|█████▊    | 130844/225000 [264:00:03<136:43:10,  5.23s/it]
 58%|█████▊    | 130845/225000 [264:00:16<190:06:04,  7.27s/it]
 58%|█████▊    | 130846/225000 [264:00:19<159:56:13,  6.12s/it]
 58%|█████▊    | 130847/225000 [264:00:23<142:24:06,  5.44s/it]
 58%|█████▊    | 130848/225000 [264:00:29<146:13:01,  5.59s/it]
 58%|█████▊    | 130849/225000 [264:00:39<179:30:51,  6.86s/it]
 58%|█████▊    | 130850/225000 [264:00:42<154:07:41,  5.89s/it]
                                                               
{'loss': '0.8072', 'grad_norm': '1.179', 'learning_rate': '2.018e-05', 'epoch': '0.5816', 'num_input_tokens_seen': 438754120, 'train_runtime': '9.504e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 130850/225000 [264:00:42<154:07:41,  5.89s/it]
 58%|█████▊    | 130851/225000 [264:00:46<140:56:50,  5.39s/it]
 58%|█████▊    | 130852/225000 [264:00:55<169:17:39,  6.47s/it]
 58%|█████▊    | 130853/225000 [264:00:59<145:42:19,  5.57s/it]
 58%|█████▊    | 130854/225000 [264:01:07<165:23:50,  6.32s/it]
 58%|█████▊    | 130855/225000 [264:01:12<153:59:32,  5.89s/it]
 58%|█████▊    | 130856/225000 [264:01:16<142:24:21,  5.45s/it]
 58%|█████▊    | 130857/225000 [264:01:21<134:06:41,  5.13s/it]
 58%|█████▊    | 130858/225000 [264:01:25<131:39:58,  5.03s/it]
 58%|█████▊    | 130859/225000 [264:01:30<127:24:15,  4.87s/it]
 58%|█████▊    | 130860/225000 [264:01:42<183:49:48,  7.03s/it]
 58%|█████▊    | 130861/225000 [264:01:48<174:22:10,  6.67s/it]
 58%|█████▊    | 130862/225000 [264:01:56<183:08:37,  7.00s/it]
 58%|█████▊    | 130863/225000 [264:02:00<159:59:26,  6.12s/it]
 58%|█████▊    | 130864/225000 [264:02:03<141:42:27,  5.42s/it]
 58%|█████▊    | 130865/225000 [264:02:08<133:50:34,  5.12s/it]
 58%|█████▊    | 130866/225000 [264:02:14<143:04:10,  5.47s/it]
 58%|█████▊    | 130867/225000 [264:02:25<188:41:27,  7.22s/it]
 58%|█████▊    | 130868/225000 [264:02:29<159:25:19,  6.10s/it]
 58%|█████▊    | 130869/225000 [264:02:35<162:51:10,  6.23s/it]
 58%|█████▊    | 130870/225000 [264:02:40<148:46:50,  5.69s/it]
 58%|█████▊    | 130871/225000 [264:02:48<165:11:12,  6.32s/it]
 58%|█████▊    | 130872/225000 [264:02:52<145:48:15,  5.58s/it]
 58%|█████▊    | 130873/225000 [264:02:55<129:56:46,  4.97s/it]
 58%|█████▊    | 130874/225000 [264:03:05<168:19:50,  6.44s/it]
 58%|█████▊    | 130875/225000 [264:03:10<154:03:12,  5.89s/it]
 58%|█████▊    | 130876/225000 [264:03:14<145:10:39,  5.55s/it]
 58%|█████▊    | 130877/225000 [264:03:19<138:59:54,  5.32s/it]
 58%|█████▊    | 130878/225000 [264:03:24<136:46:39,  5.23s/it]
 58%|█████▊    | 130879/225000 [264:03:29<134:54:25,  5.16s/it]
 58%|█████▊    | 130880/225000 [264:03:34<135:13:58,  5.17s/it]
 58%|█████▊    | 130881/225000 [264:03:39<131:56:33,  5.05s/it]
 58%|█████▊    | 130882/225000 [264:03:44<131:44:28,  5.04s/it]
 58%|█████▊    | 130883/225000 [264:03:57<192:38:26,  7.37s/it]
 58%|█████▊    | 130884/225000 [264:04:02<177:01:21,  6.77s/it]
 58%|█████▊    | 130885/225000 [264:04:06<154:12:50,  5.90s/it]
 58%|█████▊    | 130886/225000 [264:04:12<153:43:40,  5.88s/it]
 58%|█████▊    | 130887/225000 [264:04:21<177:27:39,  6.79s/it]
 58%|█████▊    | 130888/225000 [264:04:24<150:53:10,  5.77s/it]
 58%|█████▊    | 130889/225000 [264:04:28<133:25:29,  5.10s/it]
 58%|█████▊    | 130890/225000 [264:04:37<165:59:37,  6.35s/it]
 58%|█████▊    | 130891/225000 [264:04:43<160:42:31,  6.15s/it]
 58%|█████▊    | 130892/225000 [264:04:48<154:50:23,  5.92s/it]
 58%|█████▊    | 130893/225000 [264:04:55<164:45:57,  6.30s/it]
 58%|█████▊    | 130894/225000 [264:05:00<154:42:26,  5.92s/it]
 58%|█████▊    | 130895/225000 [264:05:05<145:28:25,  5.57s/it]
 58%|█████▊    | 130896/225000 [264:05:10<142:11:38,  5.44s/it]
 58%|█████▊    | 130897/225000 [264:05:15<139:03:16,  5.32s/it]
 58%|█████▊    | 130898/225000 [264:05:20<133:24:40,  5.10s/it]
 58%|█████▊    | 130899/225000 [264:05:25<131:49:11,  5.04s/it]
 58%|█████▊    | 130900/225000 [264:05:30<132:12:51,  5.06s/it]
                                                               
{'loss': '0.8031', 'grad_norm': '1.044', 'learning_rate': '2.016e-05', 'epoch': '0.5818', 'num_input_tokens_seen': 438883424, 'train_runtime': '9.507e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 130900/225000 [264:05:30<132:12:51,  5.06s/it]
 58%|█████▊    | 130901/225000 [264:05:35<133:09:03,  5.09s/it]
 58%|█████▊    | 130902/225000 [264:05:41<138:24:09,  5.30s/it]
 58%|█████▊    | 130903/225000 [264:05:53<193:38:59,  7.41s/it]
 58%|█████▊    | 130904/225000 [264:06:00<185:22:22,  7.09s/it]
 58%|█████▊    | 130905/225000 [264:06:06<177:16:10,  6.78s/it]
 58%|█████▊    | 130906/225000 [264:06:12<172:02:02,  6.58s/it]
 58%|█████▊    | 130907/225000 [264:06:17<162:27:36,  6.22s/it]
 58%|█████▊    | 130908/225000 [264:06:22<154:28:54,  5.91s/it]
 58%|█████▊    | 130909/225000 [264:06:28<151:05:33,  5.78s/it]
 58%|█████▊    | 130910/225000 [264:06:33<144:05:11,  5.51s/it]
 58%|█████▊    | 130911/225000 [264:06:38<141:07:25,  5.40s/it]
 58%|█████▊    | 130912/225000 [264:06:49<183:08:29,  7.01s/it]
 58%|█████▊    | 130913/225000 [264:06:53<166:24:52,  6.37s/it]
 58%|█████▊    | 130914/225000 [264:07:01<174:19:07,  6.67s/it]
 58%|█████▊    | 130915/225000 [264:07:07<171:56:14,  6.58s/it]
 58%|█████▊    | 130916/225000 [264:07:15<178:29:54,  6.83s/it]
 58%|█████▊    | 130917/225000 [264:07:18<150:15:19,  5.75s/it]
 58%|█████▊    | 130918/225000 [264:07:22<140:17:53,  5.37s/it]
 58%|█████▊    | 130919/225000 [264:07:31<164:16:02,  6.29s/it]
 58%|█████▊    | 130920/225000 [264:07:35<148:13:49,  5.67s/it]
 58%|█████▊    | 130921/225000 [264:07:40<141:33:44,  5.42s/it]
 58%|█████▊    | 130922/225000 [264:07:44<132:54:23,  5.09s/it]
 58%|█████▊    | 130923/225000 [264:07:48<127:04:41,  4.86s/it]
 58%|█████▊    | 130924/225000 [264:08:01<184:42:09,  7.07s/it]
 58%|█████▊    | 130925/225000 [264:08:06<170:50:29,  6.54s/it]
 58%|█████▊    | 130926/225000 [264:08:13<171:57:35,  6.58s/it]
 58%|█████▊    | 130927/225000 [264:08:18<160:14:27,  6.13s/it]
 58%|█████▊    | 130928/225000 [264:08:21<140:31:57,  5.38s/it]
 58%|█████▊    | 130929/225000 [264:08:29<160:40:58,  6.15s/it]
 58%|█████▊    | 130930/225000 [264:08:33<142:21:34,  5.45s/it]
 58%|█████▊    | 130931/225000 [264:08:37<130:21:52,  4.99s/it]
 58%|█████▊    | 130932/225000 [264:08:49<183:31:13,  7.02s/it]
 58%|█████▊    | 130933/225000 [264:08:52<155:06:59,  5.94s/it]
 58%|█████▊    | 130934/225000 [264:08:56<137:45:36,  5.27s/it]
 58%|█████▊    | 130935/225000 [264:09:00<128:44:10,  4.93s/it]
 58%|█████▊    | 130936/225000 [264:09:04<122:06:59,  4.67s/it]
 58%|█████▊    | 130937/225000 [264:09:11<136:26:37,  5.22s/it]
 58%|█████▊    | 130938/225000 [264:09:22<181:10:46,  6.93s/it]
 58%|█████▊    | 130939/225000 [264:09:26<160:44:10,  6.15s/it]
 58%|█████▊    | 130940/225000 [264:09:30<144:06:45,  5.52s/it]
 58%|█████▊    | 130941/225000 [264:09:34<135:45:36,  5.20s/it]
 58%|█████▊    | 130942/225000 [264:09:39<133:47:35,  5.12s/it]
 58%|█████▊    | 130943/225000 [264:09:52<190:17:29,  7.28s/it]
 58%|█████▊    | 130944/225000 [264:09:56<165:13:59,  6.32s/it]
 58%|█████▊    | 130945/225000 [264:10:03<169:09:22,  6.47s/it]
 58%|█████▊    | 130946/225000 [264:10:08<159:22:07,  6.10s/it]
 58%|█████▊    | 130947/225000 [264:10:13<153:01:36,  5.86s/it]
 58%|█████▊    | 130948/225000 [264:10:18<148:47:00,  5.69s/it]
 58%|█████▊    | 130949/225000 [264:10:23<142:54:44,  5.47s/it]
 58%|█████▊    | 130950/225000 [264:10:29<143:23:00,  5.49s/it]
                                                               
{'loss': '0.83', 'grad_norm': '0.9908', 'learning_rate': '2.014e-05', 'epoch': '0.582', 'num_input_tokens_seen': 439013720, 'train_runtime': '9.51e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 130950/225000 [264:10:29<143:23:00,  5.49s/it]
 58%|█████▊    | 130951/225000 [264:10:33<136:22:34,  5.22s/it]
 58%|█████▊    | 130952/225000 [264:10:38<134:32:06,  5.15s/it]
 58%|█████▊    | 130953/225000 [264:10:44<137:48:23,  5.28s/it]
 58%|█████▊    | 130954/225000 [264:10:49<136:04:02,  5.21s/it]
 58%|█████▊    | 130955/225000 [264:10:54<136:18:14,  5.22s/it]
 58%|█████▊    | 130956/225000 [264:11:04<170:10:19,  6.51s/it]
 58%|█████▊    | 130957/225000 [264:11:15<203:42:05,  7.80s/it]
 58%|█████▊    | 130958/225000 [264:11:19<173:51:18,  6.66s/it]
 58%|█████▊    | 130959/225000 [264:11:24<161:20:12,  6.18s/it]
 58%|█████▊    | 130960/225000 [264:11:29<151:41:56,  5.81s/it]
 58%|█████▊    | 130961/225000 [264:11:34<148:53:58,  5.70s/it]
 58%|█████▊    | 130962/225000 [264:11:39<146:06:42,  5.59s/it]
 58%|█████▊    | 130963/225000 [264:11:45<145:50:46,  5.58s/it]
 58%|█████▊    | 130964/225000 [264:11:50<142:46:07,  5.47s/it]
 58%|█████▊    | 130965/225000 [264:11:55<140:08:50,  5.37s/it]
 58%|█████▊    | 130966/225000 [264:12:01<139:21:28,  5.34s/it]
 58%|█████▊    | 130967/225000 [264:12:06<136:44:01,  5.23s/it]
 58%|█████▊    | 130968/225000 [264:12:16<175:44:03,  6.73s/it]
 58%|█████▊    | 130969/225000 [264:12:25<196:48:07,  7.53s/it]
 58%|█████▊    | 130970/225000 [264:12:29<164:38:41,  6.30s/it]
 58%|█████▊    | 130971/225000 [264:12:33<147:57:05,  5.66s/it]
 58%|█████▊    | 130972/225000 [264:12:44<187:59:47,  7.20s/it]
 58%|█████▊    | 130973/225000 [264:12:50<181:43:23,  6.96s/it]
 58%|█████▊    | 130974/225000 [264:12:53<154:51:25,  5.93s/it]
 58%|█████▊    | 130975/225000 [264:13:03<182:57:58,  7.01s/it]
 58%|█████▊    | 130976/225000 [264:13:07<158:35:39,  6.07s/it]
 58%|█████▊    | 130977/225000 [264:13:11<141:03:11,  5.40s/it]
 58%|█████▊    | 130978/225000 [264:13:20<175:11:55,  6.71s/it]
 58%|█████▊    | 130979/225000 [264:13:27<174:11:43,  6.67s/it]
 58%|█████▊    | 130980/225000 [264:13:31<156:18:24,  5.98s/it]
 58%|█████▊    | 130981/225000 [264:13:36<142:18:42,  5.45s/it]
 58%|█████▊    | 130982/225000 [264:13:40<132:04:07,  5.06s/it]
 58%|█████▊    | 130983/225000 [264:13:50<175:07:33,  6.71s/it]
 58%|█████▊    | 130984/225000 [264:13:56<165:12:24,  6.33s/it]
 58%|█████▊    | 130985/225000 [264:14:03<168:17:35,  6.44s/it]
 58%|█████▊    | 130986/225000 [264:14:07<151:31:41,  5.80s/it]
 58%|█████▊    | 130987/225000 [264:14:11<141:43:05,  5.43s/it]
 58%|█████▊    | 130988/225000 [264:14:16<134:24:57,  5.15s/it]
 58%|█████▊    | 130989/225000 [264:14:22<139:09:48,  5.33s/it]
 58%|█████▊    | 130990/225000 [264:14:32<181:52:05,  6.96s/it]
 58%|█████▊    | 130991/225000 [264:14:36<152:01:38,  5.82s/it]
 58%|█████▊    | 130992/225000 [264:14:40<138:56:03,  5.32s/it]
 58%|█████▊    | 130993/225000 [264:14:49<171:36:34,  6.57s/it]
 58%|█████▊    | 130994/225000 [264:14:55<164:29:33,  6.30s/it]
 58%|█████▊    | 130995/225000 [264:14:58<141:56:28,  5.44s/it]
 58%|█████▊    | 130996/225000 [264:15:08<172:10:56,  6.59s/it]
 58%|█████▊    | 130997/225000 [264:15:11<146:19:20,  5.60s/it]
 58%|█████▊    | 130998/225000 [264:15:14<129:28:32,  4.96s/it]
 58%|█████▊    | 130999/225000 [264:15:20<133:36:36,  5.12s/it]
 58%|█████▊    | 131000/225000 [264:15:25<135:21:32,  5.18s/it]
                                                               
{'loss': '0.8183', 'grad_norm': '1.219', 'learning_rate': '2.012e-05', 'epoch': '0.5822', 'num_input_tokens_seen': 439144416, 'train_runtime': '9.513e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131000/225000 [264:15:25<135:21:32,  5.18s/it]
 58%|█████▊    | 131001/225000 [264:15:30<135:25:25,  5.19s/it]
 58%|█████▊    | 131002/225000 [264:15:36<137:21:59,  5.26s/it]
 58%|█████▊    | 131003/225000 [264:15:41<133:40:43,  5.12s/it]
 58%|█████▊    | 131004/225000 [264:15:46<132:46:56,  5.09s/it]
 58%|█████▊    | 131005/225000 [264:15:51<134:20:54,  5.15s/it]
 58%|█████▊    | 131006/225000 [264:15:56<135:36:25,  5.19s/it]
 58%|█████▊    | 131007/225000 [264:16:07<181:47:52,  6.96s/it]
 58%|█████▊    | 131008/225000 [264:16:16<193:25:45,  7.41s/it]
 58%|█████▊    | 131009/225000 [264:16:20<165:54:56,  6.35s/it]
 58%|█████▊    | 131010/225000 [264:16:28<180:11:04,  6.90s/it]
 58%|█████▊    | 131011/225000 [264:16:34<172:51:10,  6.62s/it]
 58%|█████▊    | 131012/225000 [264:16:38<150:42:19,  5.77s/it]
 58%|█████▊    | 131013/225000 [264:16:41<135:49:05,  5.20s/it]
 58%|█████▊    | 131014/225000 [264:16:52<175:55:48,  6.74s/it]
 58%|█████▊    | 131015/225000 [264:16:57<165:34:15,  6.34s/it]
 58%|█████▊    | 131016/225000 [264:17:00<141:01:40,  5.40s/it]
 58%|█████▊    | 131017/225000 [264:17:09<165:13:31,  6.33s/it]
 58%|█████▊    | 131018/225000 [264:17:14<154:48:30,  5.93s/it]
 58%|█████▊    | 131019/225000 [264:17:21<161:41:17,  6.19s/it]
 58%|█████▊    | 131020/225000 [264:17:24<140:05:08,  5.37s/it]
 58%|█████▊    | 131021/225000 [264:17:33<166:19:52,  6.37s/it]
 58%|█████▊    | 131022/225000 [264:17:37<146:00:48,  5.59s/it]
 58%|█████▊    | 131023/225000 [264:17:45<170:21:18,  6.53s/it]
 58%|█████▊    | 131024/225000 [264:17:51<163:42:42,  6.27s/it]
 58%|█████▊    | 131025/225000 [264:17:58<168:42:54,  6.46s/it]
 58%|█████▊    | 131026/225000 [264:18:01<146:01:06,  5.59s/it]
 58%|█████▊    | 131027/225000 [264:18:07<142:01:22,  5.44s/it]
 58%|█████▊    | 131028/225000 [264:18:12<141:19:24,  5.41s/it]
 58%|█████▊    | 131029/225000 [264:18:17<137:52:10,  5.28s/it]
 58%|█████▊    | 131030/225000 [264:18:22<134:59:17,  5.17s/it]
 58%|█████▊    | 131031/225000 [264:18:27<137:33:37,  5.27s/it]
 58%|█████▊    | 131032/225000 [264:18:33<137:27:42,  5.27s/it]
 58%|█████▊    | 131033/225000 [264:18:38<137:45:17,  5.28s/it]
 58%|█████▊    | 131034/225000 [264:18:48<175:46:08,  6.73s/it]
 58%|█████▊    | 131035/225000 [264:18:57<193:13:01,  7.40s/it]
 58%|█████▊    | 131036/225000 [264:19:01<163:43:06,  6.27s/it]
 58%|█████▊    | 131037/225000 [264:19:09<182:45:49,  7.00s/it]
 58%|█████▊    | 131038/225000 [264:19:12<152:57:53,  5.86s/it]
 58%|█████▊    | 131039/225000 [264:19:20<166:17:51,  6.37s/it]
 58%|█████▊    | 131040/225000 [264:19:27<173:43:34,  6.66s/it]
 58%|█████▊    | 131041/225000 [264:19:31<150:17:01,  5.76s/it]
 58%|█████▊    | 131042/225000 [264:19:35<139:34:33,  5.35s/it]
 58%|█████▊    | 131043/225000 [264:19:40<131:05:58,  5.02s/it]
 58%|█████▊    | 131044/225000 [264:19:48<158:17:51,  6.07s/it]
 58%|█████▊    | 131045/225000 [264:19:56<169:24:24,  6.49s/it]
 58%|█████▊    | 131046/225000 [264:19:59<145:44:41,  5.58s/it]
 58%|█████▊    | 131047/225000 [264:20:03<131:00:11,  5.02s/it]
 58%|█████▊    | 131048/225000 [264:20:13<172:23:40,  6.61s/it]
 58%|█████▊    | 131049/225000 [264:20:17<153:47:37,  5.89s/it]
 58%|█████▊    | 131050/225000 [264:20:21<135:31:28,  5.19s/it]
                                                               
{'loss': '0.8226', 'grad_norm': '1.132', 'learning_rate': '2.01e-05', 'epoch': '0.5824', 'num_input_tokens_seen': 439274664, 'train_runtime': '9.516e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131050/225000 [264:20:21<135:31:28,  5.19s/it]
 58%|█████▊    | 131051/225000 [264:20:28<151:08:40,  5.79s/it]
 58%|█████▊    | 131052/225000 [264:20:36<167:20:53,  6.41s/it]
 58%|█████▊    | 131053/225000 [264:20:42<163:53:15,  6.28s/it]
 58%|█████▊    | 131054/225000 [264:20:49<169:08:27,  6.48s/it]
 58%|█████▊    | 131055/225000 [264:20:53<146:58:44,  5.63s/it]
 58%|█████▊    | 131056/225000 [264:21:03<185:08:16,  7.09s/it]
 58%|█████▊    | 131057/225000 [264:21:06<155:27:12,  5.96s/it]
 58%|█████▊    | 131058/225000 [264:21:10<139:08:20,  5.33s/it]
 58%|█████▊    | 131059/225000 [264:21:21<181:05:59,  6.94s/it]
 58%|█████▊    | 131060/225000 [264:21:26<164:23:01,  6.30s/it]
 58%|█████▊    | 131061/225000 [264:21:31<153:11:30,  5.87s/it]
 58%|█████▊    | 131062/225000 [264:21:35<145:02:04,  5.56s/it]
 58%|█████▊    | 131063/225000 [264:21:40<138:11:29,  5.30s/it]
 58%|█████▊    | 131064/225000 [264:21:45<134:08:34,  5.14s/it]
 58%|█████▊    | 131065/225000 [264:21:50<132:52:07,  5.09s/it]
 58%|█████▊    | 131066/225000 [264:21:55<133:00:55,  5.10s/it]
 58%|█████▊    | 131067/225000 [264:22:01<142:26:32,  5.46s/it]
 58%|█████▊    | 131068/225000 [264:22:14<202:06:01,  7.75s/it]
 58%|█████▊    | 131069/225000 [264:22:18<173:25:51,  6.65s/it]
 58%|█████▊    | 131070/225000 [264:22:22<152:28:04,  5.84s/it]
 58%|█████▊    | 131071/225000 [264:22:27<141:20:06,  5.42s/it]
 58%|█████▊    | 131072/225000 [264:22:31<133:04:25,  5.10s/it]
 58%|█████▊    | 131073/225000 [264:22:42<174:23:21,  6.68s/it]
 58%|█████▊    | 131074/225000 [264:22:48<171:36:01,  6.58s/it]
 58%|█████▊    | 131075/225000 [264:22:53<160:13:07,  6.14s/it]
 58%|█████▊    | 131076/225000 [264:23:00<168:24:55,  6.46s/it]
 58%|█████▊    | 131077/225000 [264:23:05<157:17:46,  6.03s/it]
 58%|█████▊    | 131078/225000 [264:23:13<171:21:53,  6.57s/it]
 58%|█████▊    | 131079/225000 [264:23:18<158:15:54,  6.07s/it]
 58%|█████▊    | 131080/225000 [264:23:22<145:27:13,  5.58s/it]
 58%|█████▊    | 131081/225000 [264:23:27<135:12:45,  5.18s/it]
 58%|█████▊    | 131082/225000 [264:23:34<152:57:44,  5.86s/it]
 58%|█████▊    | 131083/225000 [264:23:45<191:08:39,  7.33s/it]
 58%|█████▊    | 131084/225000 [264:23:48<161:24:23,  6.19s/it]
 58%|█████▊    | 131085/225000 [264:23:52<140:08:35,  5.37s/it]
 58%|█████▊    | 131086/225000 [264:24:03<181:48:23,  6.97s/it]
 58%|█████▊    | 131087/225000 [264:24:07<159:15:01,  6.10s/it]
 58%|█████▊    | 131088/225000 [264:24:10<141:36:48,  5.43s/it]
 58%|█████▊    | 131089/225000 [264:24:16<141:12:21,  5.41s/it]
 58%|█████▊    | 131090/225000 [264:24:21<141:13:33,  5.41s/it]
 58%|█████▊    | 131091/225000 [264:24:27<140:25:59,  5.38s/it]
 58%|█████▊    | 131092/225000 [264:24:32<140:53:05,  5.40s/it]
 58%|█████▊    | 131093/225000 [264:24:41<168:32:29,  6.46s/it]
 58%|█████▊    | 131094/225000 [264:24:46<158:05:43,  6.06s/it]
 58%|█████▊    | 131095/225000 [264:24:50<142:26:06,  5.46s/it]
 58%|█████▊    | 131096/225000 [264:24:57<151:19:13,  5.80s/it]
 58%|█████▊    | 131097/225000 [264:25:06<178:07:13,  6.83s/it]
 58%|█████▊    | 131098/225000 [264:25:09<150:17:08,  5.76s/it]
 58%|█████▊    | 131099/225000 [264:25:14<140:00:42,  5.37s/it]
 58%|█████▊    | 131100/225000 [264:25:22<164:29:24,  6.31s/it]
                                                               
{'loss': '0.8244', 'grad_norm': '1.117', 'learning_rate': '2.009e-05', 'epoch': '0.5827', 'num_input_tokens_seen': 439404424, 'train_runtime': '9.519e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131100/225000 [264:25:22<164:29:24,  6.31s/it]
 58%|█████▊    | 131101/225000 [264:25:26<145:55:17,  5.59s/it]
 58%|█████▊    | 131102/225000 [264:25:32<146:46:46,  5.63s/it]
 58%|█████▊    | 131103/225000 [264:25:39<156:36:54,  6.00s/it]
 58%|█████▊    | 131104/225000 [264:25:43<139:55:55,  5.37s/it]
 58%|█████▊    | 131105/225000 [264:25:52<174:06:17,  6.68s/it]
 58%|█████▊    | 131106/225000 [264:25:57<159:16:02,  6.11s/it]
 58%|█████▊    | 131107/225000 [264:26:02<147:15:22,  5.65s/it]
 58%|█████▊    | 131108/225000 [264:26:06<139:07:39,  5.33s/it]
 58%|█████▊    | 131109/225000 [264:26:11<136:13:13,  5.22s/it]
 58%|█████▊    | 131110/225000 [264:26:16<130:29:18,  5.00s/it]
 58%|█████▊    | 131111/225000 [264:26:25<164:35:52,  6.31s/it]
 58%|█████▊    | 131112/225000 [264:26:33<179:20:11,  6.88s/it]
 58%|█████▊    | 131113/225000 [264:26:37<152:56:44,  5.86s/it]
 58%|█████▊    | 131114/225000 [264:26:46<177:04:11,  6.79s/it]
 58%|█████▊    | 131115/225000 [264:26:49<151:48:23,  5.82s/it]
 58%|█████▊    | 131116/225000 [264:26:57<169:11:25,  6.49s/it]
 58%|█████▊    | 131117/225000 [264:27:03<162:59:46,  6.25s/it]
 58%|█████▊    | 131118/225000 [264:27:08<156:22:53,  6.00s/it]
 58%|█████▊    | 131119/225000 [264:27:16<171:30:15,  6.58s/it]
 58%|█████▊    | 131120/225000 [264:27:19<143:29:17,  5.50s/it]
 58%|█████▊    | 131121/225000 [264:27:25<147:19:31,  5.65s/it]
 58%|█████▊    | 131122/225000 [264:27:33<162:55:39,  6.25s/it]
 58%|█████▊    | 131123/225000 [264:27:36<138:42:38,  5.32s/it]
 58%|█████▊    | 131124/225000 [264:27:41<137:22:13,  5.27s/it]
 58%|█████▊    | 131125/225000 [264:27:49<157:28:47,  6.04s/it]
 58%|█████▊    | 131126/225000 [264:27:53<144:07:27,  5.53s/it]
 58%|█████▊    | 131127/225000 [264:27:58<136:57:57,  5.25s/it]
 58%|█████▊    | 131128/225000 [264:28:02<129:42:49,  4.97s/it]
 58%|█████▊    | 131129/225000 [264:28:11<154:58:20,  5.94s/it]
 58%|█████▊    | 131130/225000 [264:28:19<170:39:17,  6.54s/it]
 58%|█████▊    | 131131/225000 [264:28:23<157:42:01,  6.05s/it]
 58%|█████▊    | 131132/225000 [264:28:29<150:39:07,  5.78s/it]
 58%|█████▊    | 131133/225000 [264:28:33<143:03:38,  5.49s/it]
 58%|█████▊    | 131134/225000 [264:28:39<140:18:54,  5.38s/it]
 58%|█████▊    | 131135/225000 [264:28:44<138:47:37,  5.32s/it]
 58%|█████▊    | 131136/225000 [264:28:49<137:48:57,  5.29s/it]
 58%|█████▊    | 131137/225000 [264:28:54<137:16:18,  5.26s/it]
 58%|█████▊    | 131138/225000 [264:29:02<154:53:54,  5.94s/it]
 58%|█████▊    | 131139/225000 [264:29:14<201:52:48,  7.74s/it]
 58%|█████▊    | 131140/225000 [264:29:17<168:41:53,  6.47s/it]
 58%|█████▊    | 131141/225000 [264:29:25<180:06:49,  6.91s/it]
 58%|█████▊    | 131142/225000 [264:29:28<150:52:54,  5.79s/it]
 58%|█████▊    | 131143/225000 [264:29:32<137:13:03,  5.26s/it]
 58%|█████▊    | 131144/225000 [264:29:36<127:16:58,  4.88s/it]
 58%|█████▊    | 131145/225000 [264:29:44<146:08:22,  5.61s/it]
 58%|█████▊    | 131146/225000 [264:29:54<183:35:34,  7.04s/it]
 58%|█████▊    | 131147/225000 [264:29:57<155:52:05,  5.98s/it]
 58%|█████▊    | 131148/225000 [264:30:06<172:29:51,  6.62s/it]
 58%|█████▊    | 131149/225000 [264:30:09<145:42:31,  5.59s/it]
 58%|█████▊    | 131150/225000 [264:30:17<163:20:52,  6.27s/it]
                                                               
{'loss': '0.8052', 'grad_norm': '1.067', 'learning_rate': '2.007e-05', 'epoch': '0.5829', 'num_input_tokens_seen': 439533688, 'train_runtime': '9.522e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131150/225000 [264:30:17<163:20:52,  6.27s/it]
 58%|█████▊    | 131151/225000 [264:30:20<140:52:39,  5.40s/it]
 58%|█████▊    | 131152/225000 [264:30:29<166:19:34,  6.38s/it]
 58%|█████▊    | 131153/225000 [264:30:32<144:27:01,  5.54s/it]
 58%|█████▊    | 131154/225000 [264:30:36<133:02:42,  5.10s/it]
 58%|█████▊    | 131155/225000 [264:30:41<126:33:25,  4.85s/it]
 58%|█████▊    | 131156/225000 [264:30:52<174:19:52,  6.69s/it]
 58%|█████▊    | 131157/225000 [264:30:56<154:15:20,  5.92s/it]
 58%|█████▊    | 131158/225000 [264:31:00<142:36:25,  5.47s/it]
 58%|█████▊    | 131159/225000 [264:31:04<131:03:17,  5.03s/it]
 58%|█████▊    | 131160/225000 [264:31:09<131:07:25,  5.03s/it]
 58%|█████▊    | 131161/225000 [264:31:14<128:07:43,  4.92s/it]
 58%|█████▊    | 131162/225000 [264:31:19<131:17:56,  5.04s/it]
 58%|█████▊    | 131163/225000 [264:31:26<149:56:22,  5.75s/it]
 58%|█████▊    | 131164/225000 [264:31:39<203:39:11,  7.81s/it]
 58%|█████▊    | 131165/225000 [264:31:43<169:05:51,  6.49s/it]
 58%|█████▊    | 131166/225000 [264:31:50<179:38:01,  6.89s/it]
 58%|█████▊    | 131167/225000 [264:31:56<168:09:08,  6.45s/it]
 58%|█████▊    | 131168/225000 [264:31:59<145:58:20,  5.60s/it]
 58%|█████▊    | 131169/225000 [264:32:07<158:59:07,  6.10s/it]
 58%|█████▊    | 131170/225000 [264:32:15<176:36:48,  6.78s/it]
 58%|█████▊    | 131171/225000 [264:32:21<172:35:16,  6.62s/it]
 58%|█████▊    | 131172/225000 [264:32:26<154:16:28,  5.92s/it]
 58%|█████▊    | 131173/225000 [264:32:30<139:46:15,  5.36s/it]
 58%|█████▊    | 131174/225000 [264:32:34<135:03:19,  5.18s/it]
 58%|█████▊    | 131175/225000 [264:32:46<182:09:07,  6.99s/it]
 58%|█████▊    | 131176/225000 [264:32:49<152:16:57,  5.84s/it]
 58%|█████▊    | 131177/225000 [264:32:54<147:57:57,  5.68s/it]
 58%|█████▊    | 131178/225000 [264:33:01<159:34:53,  6.12s/it]
 58%|█████▊    | 131179/225000 [264:33:05<139:19:44,  5.35s/it]
 58%|█████▊    | 131180/225000 [264:33:14<171:24:37,  6.58s/it]
 58%|█████▊    | 131181/225000 [264:33:18<147:27:57,  5.66s/it]
 58%|█████▊    | 131182/225000 [264:33:25<156:29:05,  6.00s/it]
 58%|█████▊    | 131183/225000 [264:33:31<161:12:49,  6.19s/it]
 58%|█████▊    | 131184/225000 [264:33:36<150:24:02,  5.77s/it]
 58%|█████▊    | 131185/225000 [264:33:41<143:28:49,  5.51s/it]
 58%|█████▊    | 131186/225000 [264:33:46<139:21:37,  5.35s/it]
 58%|█████▊    | 131187/225000 [264:33:51<135:30:02,  5.20s/it]
 58%|█████▊    | 131188/225000 [264:33:56<137:44:47,  5.29s/it]
 58%|█████▊    | 131189/225000 [264:34:01<132:14:27,  5.07s/it]
 58%|█████▊    | 131190/225000 [264:34:06<132:40:42,  5.09s/it]
 58%|█████▊    | 131191/225000 [264:34:18<185:08:19,  7.10s/it]
 58%|█████▊    | 131192/225000 [264:34:23<173:36:26,  6.66s/it]
 58%|█████▊    | 131193/225000 [264:34:30<177:21:21,  6.81s/it]
 58%|█████▊    | 131194/225000 [264:34:37<174:42:59,  6.71s/it]
 58%|█████▊    | 131195/225000 [264:34:40<149:17:44,  5.73s/it]
 58%|█████▊    | 131196/225000 [264:34:48<167:53:44,  6.44s/it]
 58%|█████▊    | 131197/225000 [264:34:54<161:24:25,  6.19s/it]
 58%|█████▊    | 131198/225000 [264:34:58<139:59:05,  5.37s/it]
 58%|█████▊    | 131199/225000 [264:35:07<168:39:17,  6.47s/it]
 58%|█████▊    | 131200/225000 [264:35:10<148:17:59,  5.69s/it]
                                                               
{'loss': '0.8168', 'grad_norm': '1.015', 'learning_rate': '2.005e-05', 'epoch': '0.5831', 'num_input_tokens_seen': 439663264, 'train_runtime': '9.525e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131200/225000 [264:35:10<148:17:59,  5.69s/it]
 58%|█████▊    | 131201/225000 [264:35:15<136:48:17,  5.25s/it]
 58%|█████▊    | 131202/225000 [264:35:22<151:09:49,  5.80s/it]
 58%|█████▊    | 131203/225000 [264:35:28<157:19:20,  6.04s/it]
 58%|█████▊    | 131204/225000 [264:35:32<138:56:42,  5.33s/it]
 58%|█████▊    | 131205/225000 [264:35:36<129:20:02,  4.96s/it]
 58%|█████▊    | 131206/225000 [264:35:47<179:00:24,  6.87s/it]
 58%|█████▊    | 131207/225000 [264:35:51<154:58:49,  5.95s/it]
 58%|█████▊    | 131208/225000 [264:35:56<149:15:54,  5.73s/it]
 58%|█████▊    | 131209/225000 [264:36:05<173:35:43,  6.66s/it]
 58%|█████▊    | 131210/225000 [264:36:09<147:45:38,  5.67s/it]
 58%|█████▊    | 131211/225000 [264:36:16<164:07:29,  6.30s/it]
 58%|█████▊    | 131212/225000 [264:36:23<167:49:40,  6.44s/it]
 58%|█████▊    | 131213/225000 [264:36:28<157:48:23,  6.06s/it]
 58%|█████▊    | 131214/225000 [264:36:33<143:38:40,  5.51s/it]
 58%|█████▊    | 131215/225000 [264:36:42<170:57:42,  6.56s/it]
 58%|█████▊    | 131216/225000 [264:36:46<155:18:02,  5.96s/it]
 58%|█████▊    | 131217/225000 [264:36:51<144:17:54,  5.54s/it]
 58%|█████▊    | 131218/225000 [264:36:56<138:48:09,  5.33s/it]
 58%|█████▊    | 131219/225000 [264:37:00<131:00:44,  5.03s/it]
 58%|█████▊    | 131220/225000 [264:37:05<129:50:05,  4.98s/it]
 58%|█████▊    | 131221/225000 [264:37:16<182:15:22,  7.00s/it]
 58%|█████▊    | 131222/225000 [264:37:20<158:03:34,  6.07s/it]
 58%|█████▊    | 131223/225000 [264:37:27<165:37:45,  6.36s/it]
 58%|█████▊    | 131224/225000 [264:37:31<147:03:37,  5.65s/it]
 58%|█████▊    | 131225/225000 [264:37:35<131:08:07,  5.03s/it]
 58%|█████▊    | 131226/225000 [264:37:42<147:19:57,  5.66s/it]
 58%|█████▊    | 131227/225000 [264:37:51<174:19:28,  6.69s/it]
 58%|█████▊    | 131228/225000 [264:37:55<150:20:39,  5.77s/it]
 58%|█████▊    | 131229/225000 [264:37:59<138:07:26,  5.30s/it]
 58%|█████▊    | 131230/225000 [264:38:03<131:22:59,  5.04s/it]
 58%|█████▊    | 131231/225000 [264:38:08<126:54:48,  4.87s/it]
 58%|█████▊    | 131232/225000 [264:38:18<171:22:16,  6.58s/it]
 58%|█████▊    | 131233/225000 [264:38:25<171:22:24,  6.58s/it]
 58%|█████▊    | 131234/225000 [264:38:29<147:13:09,  5.65s/it]
 58%|█████▊    | 131235/225000 [264:38:37<169:42:34,  6.52s/it]
 58%|█████▊    | 131236/225000 [264:38:41<147:00:37,  5.64s/it]
 58%|█████▊    | 131237/225000 [264:38:45<136:27:29,  5.24s/it]
 58%|█████▊    | 131238/225000 [264:38:50<132:31:31,  5.09s/it]
 58%|█████▊    | 131239/225000 [264:38:55<130:27:10,  5.01s/it]
 58%|█████▊    | 131240/225000 [264:39:05<170:04:40,  6.53s/it]
 58%|█████▊    | 131241/225000 [264:39:14<188:36:52,  7.24s/it]
 58%|█████▊    | 131242/225000 [264:39:20<183:37:21,  7.05s/it]
 58%|█████▊    | 131243/225000 [264:39:25<163:45:16,  6.29s/it]
 58%|█████▊    | 131244/225000 [264:39:29<150:25:56,  5.78s/it]
 58%|█████▊    | 131245/225000 [264:39:34<143:06:51,  5.50s/it]
 58%|█████▊    | 131246/225000 [264:39:39<138:20:35,  5.31s/it]
 58%|█████▊    | 131247/225000 [264:39:44<135:16:40,  5.19s/it]
 58%|█████▊    | 131248/225000 [264:39:55<183:03:23,  7.03s/it]
 58%|█████▊    | 131249/225000 [264:40:03<186:07:21,  7.15s/it]
 58%|█████▊    | 131250/225000 [264:40:07<167:34:36,  6.43s/it]
                                                               
{'loss': '0.8119', 'grad_norm': '1.169', 'learning_rate': '2.003e-05', 'epoch': '0.5833', 'num_input_tokens_seen': 439792944, 'train_runtime': '9.528e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131250/225000 [264:40:07<167:34:36,  6.43s/it]
 58%|█████▊    | 131251/225000 [264:40:12<156:32:33,  6.01s/it]
 58%|█████▊    | 131252/225000 [264:40:17<147:53:20,  5.68s/it]
 58%|█████▊    | 131253/225000 [264:40:23<145:33:53,  5.59s/it]
 58%|█████▊    | 131254/225000 [264:40:28<142:59:27,  5.49s/it]
 58%|█████▊    | 131255/225000 [264:40:33<138:12:08,  5.31s/it]
 58%|█████▊    | 131256/225000 [264:40:38<133:53:19,  5.14s/it]
 58%|█████▊    | 131257/225000 [264:40:43<134:01:31,  5.15s/it]
 58%|█████▊    | 131258/225000 [264:40:48<133:43:15,  5.14s/it]
 58%|█████▊    | 131259/225000 [264:40:53<134:45:05,  5.17s/it]
 58%|█████▊    | 131260/225000 [264:41:01<155:50:50,  5.99s/it]
 58%|█████▊    | 131261/225000 [264:41:12<195:46:15,  7.52s/it]
 58%|█████▊    | 131262/225000 [264:41:16<166:42:29,  6.40s/it]
 58%|█████▊    | 131263/225000 [264:41:23<172:45:21,  6.63s/it]
 58%|█████▊    | 131264/225000 [264:41:27<150:11:49,  5.77s/it]
 58%|█████▊    | 131265/225000 [264:41:30<132:58:20,  5.11s/it]
 58%|█████▊    | 131266/225000 [264:41:42<181:55:43,  6.99s/it]
 58%|█████▊    | 131267/225000 [264:41:46<160:55:52,  6.18s/it]
 58%|█████▊    | 131268/225000 [264:41:50<146:22:59,  5.62s/it]
 58%|█████▊    | 131269/225000 [264:42:00<177:11:35,  6.81s/it]
 58%|█████▊    | 131270/225000 [264:42:07<179:22:13,  6.89s/it]
 58%|█████▊    | 131271/225000 [264:42:11<152:40:18,  5.86s/it]
 58%|█████▊    | 131272/225000 [264:42:18<164:07:50,  6.30s/it]
 58%|█████▊    | 131273/225000 [264:42:23<155:36:51,  5.98s/it]
 58%|█████▊    | 131274/225000 [264:42:27<142:48:39,  5.49s/it]
 58%|█████▊    | 131275/225000 [264:42:32<136:42:46,  5.25s/it]
 58%|█████▊    | 131276/225000 [264:42:36<129:47:32,  4.99s/it]
 58%|█████▊    | 131277/225000 [264:42:41<126:38:05,  4.86s/it]
 58%|█████▊    | 131278/225000 [264:42:53<184:04:07,  7.07s/it]
 58%|█████▊    | 131279/225000 [264:42:58<166:34:27,  6.40s/it]
 58%|█████▊    | 131280/225000 [264:43:01<142:19:00,  5.47s/it]
 58%|█████▊    | 131281/225000 [264:43:11<177:39:57,  6.82s/it]
 58%|█████▊    | 131282/225000 [264:43:16<159:46:17,  6.14s/it]
 58%|█████▊    | 131283/225000 [264:43:20<145:56:36,  5.61s/it]
 58%|█████▊    | 131284/225000 [264:43:25<137:16:58,  5.27s/it]
 58%|█████▊    | 131285/225000 [264:43:29<131:56:36,  5.07s/it]
 58%|█████▊    | 131286/225000 [264:43:41<186:23:09,  7.16s/it]
 58%|█████▊    | 131287/225000 [264:43:45<161:38:13,  6.21s/it]
 58%|█████▊    | 131288/225000 [264:43:49<140:18:21,  5.39s/it]
 58%|█████▊    | 131289/225000 [264:43:53<134:02:28,  5.15s/it]
 58%|█████▊    | 131290/225000 [264:44:03<171:12:19,  6.58s/it]
 58%|█████▊    | 131291/225000 [264:44:07<146:27:50,  5.63s/it]
 58%|█████▊    | 131292/225000 [264:44:14<161:53:23,  6.22s/it]
 58%|█████▊    | 131293/225000 [264:44:18<143:39:27,  5.52s/it]
 58%|█████▊    | 131294/225000 [264:44:24<144:48:20,  5.56s/it]
 58%|█████▊    | 131295/225000 [264:44:33<173:51:00,  6.68s/it]
 58%|█████▊    | 131296/225000 [264:44:37<150:53:06,  5.80s/it]
 58%|█████▊    | 131297/225000 [264:44:44<162:58:25,  6.26s/it]
 58%|█████▊    | 131298/225000 [264:44:48<145:22:12,  5.59s/it]
 58%|█████▊    | 131299/225000 [264:44:57<168:54:12,  6.49s/it]
 58%|█████▊    | 131300/225000 [264:45:00<142:32:31,  5.48s/it]
                                                               
{'loss': '0.8197', 'grad_norm': '1.248', 'learning_rate': '2.001e-05', 'epoch': '0.5836', 'num_input_tokens_seen': 439923176, 'train_runtime': '9.531e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131300/225000 [264:45:00<142:32:31,  5.48s/it]
 58%|█████▊    | 131301/225000 [264:45:08<160:00:09,  6.15s/it]
 58%|█████▊    | 131302/225000 [264:45:12<146:06:02,  5.61s/it]
 58%|█████▊    | 131303/225000 [264:45:16<133:03:52,  5.11s/it]
 58%|█████▊    | 131304/225000 [264:45:21<133:12:39,  5.12s/it]
 58%|█████▊    | 131305/225000 [264:45:34<191:44:51,  7.37s/it]
 58%|█████▊    | 131306/225000 [264:45:37<159:47:34,  6.14s/it]
 58%|█████▊    | 131307/225000 [264:45:42<147:10:04,  5.65s/it]
 58%|█████▊    | 131308/225000 [264:45:46<140:33:58,  5.40s/it]
 58%|█████▊    | 131309/225000 [264:45:51<137:44:43,  5.29s/it]
 58%|█████▊    | 131310/225000 [264:45:57<137:00:56,  5.26s/it]
 58%|█████▊    | 131311/225000 [264:46:01<133:48:28,  5.14s/it]
 58%|█████▊    | 131312/225000 [264:46:07<135:20:52,  5.20s/it]
 58%|█████▊    | 131313/225000 [264:46:13<141:44:41,  5.45s/it]
 58%|█████▊    | 131314/225000 [264:46:25<195:27:31,  7.51s/it]
 58%|█████▊    | 131315/225000 [264:46:31<178:43:36,  6.87s/it]
 58%|█████▊    | 131316/225000 [264:46:34<152:55:05,  5.88s/it]
 58%|█████▊    | 131317/225000 [264:46:43<174:32:39,  6.71s/it]
 58%|█████▊    | 131318/225000 [264:46:47<159:11:04,  6.12s/it]
 58%|█████▊    | 131319/225000 [264:46:53<151:25:54,  5.82s/it]
 58%|█████▊    | 131320/225000 [264:46:58<147:19:39,  5.66s/it]
 58%|█████▊    | 131321/225000 [264:47:03<140:47:09,  5.41s/it]
 58%|█████▊    | 131322/225000 [264:47:08<139:11:47,  5.35s/it]
 58%|█████▊    | 131323/225000 [264:47:13<133:36:22,  5.13s/it]
 58%|█████▊    | 131324/225000 [264:47:18<135:46:03,  5.22s/it]
 58%|█████▊    | 131325/225000 [264:47:22<130:10:13,  5.00s/it]
 58%|█████▊    | 131326/225000 [264:47:32<163:33:26,  6.29s/it]
 58%|█████▊    | 131327/225000 [264:47:41<188:50:44,  7.26s/it]
 58%|█████▊    | 131328/225000 [264:47:44<155:24:39,  5.97s/it]
 58%|█████▊    | 131329/225000 [264:47:52<170:25:00,  6.55s/it]
 58%|█████▊    | 131330/225000 [264:47:57<155:34:54,  5.98s/it]
 58%|█████▊    | 131331/225000 [264:48:01<144:48:48,  5.57s/it]
 58%|█████▊    | 131332/225000 [264:48:06<135:42:26,  5.22s/it]
 58%|█████▊    | 131333/225000 [264:48:10<130:46:17,  5.03s/it]
 58%|█████▊    | 131334/225000 [264:48:15<125:39:15,  4.83s/it]
 58%|█████▊    | 131335/225000 [264:48:19<120:37:41,  4.64s/it]
 58%|█████▊    | 131336/225000 [264:48:26<141:17:04,  5.43s/it]
 58%|█████▊    | 131337/225000 [264:48:38<190:21:27,  7.32s/it]
 58%|█████▊    | 131338/225000 [264:48:42<162:24:42,  6.24s/it]
 58%|█████▊    | 131339/225000 [264:48:46<144:43:36,  5.56s/it]
 58%|█████▊    | 131340/225000 [264:48:56<183:02:40,  7.04s/it]
 58%|█████▊    | 131341/225000 [264:49:00<158:26:20,  6.09s/it]
 58%|█████▊    | 131342/225000 [264:49:04<145:31:23,  5.59s/it]
 58%|█████▊    | 131343/225000 [264:49:09<141:03:50,  5.42s/it]
 58%|█████▊    | 131344/225000 [264:49:15<140:35:29,  5.40s/it]
 58%|█████▊    | 131345/225000 [264:49:20<138:59:53,  5.34s/it]
 58%|█████▊    | 131346/225000 [264:49:25<138:33:05,  5.33s/it]
 58%|█████▊    | 131347/225000 [264:49:31<139:34:08,  5.36s/it]
 58%|█████▊    | 131348/225000 [264:49:36<137:53:42,  5.30s/it]
 58%|█████▊    | 131349/225000 [264:49:41<137:00:58,  5.27s/it]
 58%|█████▊    | 131350/225000 [264:49:53<190:26:48,  7.32s/it]
                                                               
{'loss': '0.8135', 'grad_norm': '1.148', 'learning_rate': '2e-05', 'epoch': '0.5838', 'num_input_tokens_seen': 440051864, 'train_runtime': '9.534e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131350/225000 [264:49:53<190:26:48,  7.32s/it]
 58%|█████▊    | 131351/225000 [264:50:00<184:31:40,  7.09s/it]
 58%|█████▊    | 131352/225000 [264:50:06<181:23:35,  6.97s/it]
 58%|█████▊    | 131353/225000 [264:50:10<154:31:14,  5.94s/it]
 58%|█████▊    | 131354/225000 [264:50:14<139:30:20,  5.36s/it]
 58%|█████▊    | 131355/225000 [264:50:21<154:22:51,  5.93s/it]
 58%|█████▊    | 131356/225000 [264:50:29<168:26:49,  6.48s/it]
 58%|█████▊    | 131357/225000 [264:50:33<148:48:55,  5.72s/it]
 58%|█████▊    | 131358/225000 [264:50:41<166:20:06,  6.39s/it]
 58%|█████▊    | 131359/225000 [264:50:46<159:32:00,  6.13s/it]
 58%|█████▊    | 131360/225000 [264:50:54<167:33:13,  6.44s/it]
 58%|█████▊    | 131361/225000 [264:50:58<148:26:33,  5.71s/it]
 58%|█████▊    | 131362/225000 [264:51:03<147:41:11,  5.68s/it]
 58%|█████▊    | 131363/225000 [264:51:10<153:47:47,  5.91s/it]
 58%|█████▊    | 131364/225000 [264:51:15<147:10:51,  5.66s/it]
 58%|█████▊    | 131365/225000 [264:51:20<144:45:48,  5.57s/it]
 58%|█████▊    | 131366/225000 [264:51:25<142:33:49,  5.48s/it]
 58%|█████▊    | 131367/225000 [264:51:31<140:57:33,  5.42s/it]
 58%|█████▊    | 131368/225000 [264:51:36<137:50:42,  5.30s/it]
 58%|█████▊    | 131369/225000 [264:51:41<136:52:15,  5.26s/it]
 58%|█████▊    | 131370/225000 [264:51:46<138:34:54,  5.33s/it]
 58%|█████▊    | 131371/225000 [264:51:58<186:39:04,  7.18s/it]
 58%|█████▊    | 131372/225000 [264:52:07<200:02:21,  7.69s/it]
 58%|█████▊    | 131373/225000 [264:52:10<166:40:42,  6.41s/it]
 58%|█████▊    | 131374/225000 [264:52:14<148:05:54,  5.69s/it]
 58%|█████▊    | 131375/225000 [264:52:23<171:17:56,  6.59s/it]
 58%|█████▊    | 131376/225000 [264:52:30<176:29:01,  6.79s/it]
 58%|█████▊    | 131377/225000 [264:52:34<156:35:13,  6.02s/it]
 58%|█████▊    | 131378/225000 [264:52:39<144:10:34,  5.54s/it]
 58%|█████▊    | 131379/225000 [264:52:43<137:21:30,  5.28s/it]
 58%|█████▊    | 131380/225000 [264:52:48<132:10:11,  5.08s/it]
 58%|█████▊    | 131381/225000 [264:52:58<173:39:50,  6.68s/it]
 58%|█████▊    | 131382/225000 [264:53:06<177:46:46,  6.84s/it]
 58%|█████▊    | 131383/225000 [264:53:09<152:33:12,  5.87s/it]
 58%|█████▊    | 131384/225000 [264:53:17<169:14:54,  6.51s/it]
 58%|█████▊    | 131385/225000 [264:53:24<170:44:04,  6.57s/it]
 58%|█████▊    | 131386/225000 [264:53:28<150:24:46,  5.78s/it]
 58%|█████▊    | 131387/225000 [264:53:34<148:57:39,  5.73s/it]
 58%|█████▊    | 131388/225000 [264:53:42<171:59:23,  6.61s/it]
 58%|█████▊    | 131389/225000 [264:53:46<147:38:39,  5.68s/it]
 58%|█████▊    | 131390/225000 [264:53:49<131:25:32,  5.05s/it]
 58%|█████▊    | 131391/225000 [264:53:54<128:22:02,  4.94s/it]
 58%|█████▊    | 131392/225000 [264:53:58<122:33:41,  4.71s/it]
 58%|█████▊    | 131393/225000 [264:54:07<151:11:05,  5.81s/it]
 58%|█████▊    | 131394/225000 [264:54:17<188:28:33,  7.25s/it]
 58%|█████▊    | 131395/225000 [264:54:21<159:03:21,  6.12s/it]
 58%|█████▊    | 131396/225000 [264:54:24<140:27:34,  5.40s/it]
 58%|█████▊    | 131397/225000 [264:54:33<162:13:39,  6.24s/it]
 58%|█████▊    | 131398/225000 [264:54:39<162:13:30,  6.24s/it]
 58%|█████▊    | 131399/225000 [264:54:43<146:46:51,  5.65s/it]
 58%|█████▊    | 131400/225000 [264:54:53<180:41:07,  6.95s/it]
                                                               
{'loss': '0.8161', 'grad_norm': '1.045', 'learning_rate': '1.998e-05', 'epoch': '0.584', 'num_input_tokens_seen': 440181712, 'train_runtime': '9.537e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131400/225000 [264:54:53<180:41:07,  6.95s/it]
 58%|█████▊    | 131401/225000 [264:54:57<153:30:27,  5.90s/it]
 58%|█████▊    | 131402/225000 [264:55:01<145:26:35,  5.59s/it]
 58%|█████▊    | 131403/225000 [264:55:08<151:42:13,  5.83s/it]
 58%|█████▊    | 131404/225000 [264:55:11<133:54:50,  5.15s/it]
 58%|█████▊    | 131405/225000 [264:55:20<164:00:01,  6.31s/it]
 58%|█████▊    | 131406/225000 [264:55:24<143:09:03,  5.51s/it]
 58%|█████▊    | 131407/225000 [264:55:30<144:38:00,  5.56s/it]
 58%|█████▊    | 131408/225000 [264:55:36<150:32:33,  5.79s/it]
 58%|█████▊    | 131409/225000 [264:55:40<140:00:18,  5.39s/it]
 58%|█████▊    | 131410/225000 [264:55:45<130:50:27,  5.03s/it]
 58%|█████▊    | 131411/225000 [264:55:56<180:34:13,  6.95s/it]
 58%|█████▊    | 131412/225000 [264:56:01<162:39:11,  6.26s/it]
 58%|█████▊    | 131413/225000 [264:56:09<175:12:50,  6.74s/it]
 58%|█████▊    | 131414/225000 [264:56:13<157:52:35,  6.07s/it]
 58%|█████▊    | 131415/225000 [264:56:18<148:20:48,  5.71s/it]
 58%|█████▊    | 131416/225000 [264:56:23<141:43:32,  5.45s/it]
 58%|█████▊    | 131417/225000 [264:56:28<140:13:34,  5.39s/it]
 58%|█████▊    | 131418/225000 [264:56:33<138:45:13,  5.34s/it]
 58%|█████▊    | 131419/225000 [264:56:38<134:24:35,  5.17s/it]
 58%|█████▊    | 131420/225000 [264:56:43<133:19:12,  5.13s/it]
 58%|█████▊    | 131421/225000 [264:56:49<136:17:28,  5.24s/it]
 58%|█████▊    | 131422/225000 [264:56:54<135:34:03,  5.22s/it]
 58%|█████▊    | 131423/225000 [264:56:58<131:39:00,  5.06s/it]
 58%|█████▊    | 131424/225000 [264:57:04<134:13:52,  5.16s/it]
 58%|█████▊    | 131425/225000 [264:57:16<190:42:22,  7.34s/it]
 58%|█████▊    | 131426/225000 [264:57:24<196:30:01,  7.56s/it]
 58%|█████▊    | 131427/225000 [264:57:29<173:27:49,  6.67s/it]
 58%|█████▊    | 131428/225000 [264:57:33<149:25:56,  5.75s/it]
 58%|█████▊    | 131429/225000 [264:57:37<139:17:35,  5.36s/it]
 58%|█████▊    | 131430/225000 [264:57:42<133:30:21,  5.14s/it]
 58%|█████▊    | 131431/225000 [264:57:46<130:19:55,  5.01s/it]
 58%|█████▊    | 131432/225000 [264:57:52<132:09:44,  5.08s/it]
 58%|█████▊    | 131433/225000 [264:57:57<131:10:41,  5.05s/it]
 58%|█████▊    | 131434/225000 [264:58:02<131:39:40,  5.07s/it]
 58%|█████▊    | 131435/225000 [264:58:13<181:42:23,  6.99s/it]
 58%|█████▊    | 131436/225000 [264:58:17<161:11:16,  6.20s/it]
 58%|█████▊    | 131437/225000 [264:58:22<146:34:26,  5.64s/it]
 58%|█████▊    | 131438/225000 [264:58:26<137:38:18,  5.30s/it]
 58%|█████▊    | 131439/225000 [264:58:36<173:47:04,  6.69s/it]
 58%|█████▊    | 131440/225000 [264:58:43<177:19:41,  6.82s/it]
 58%|█████▊    | 131441/225000 [264:58:48<162:07:41,  6.24s/it]
 58%|█████▊    | 131442/225000 [264:58:53<151:49:51,  5.84s/it]
 58%|█████▊    | 131443/225000 [264:58:58<140:22:03,  5.40s/it]
 58%|█████▊    | 131444/225000 [264:59:03<137:53:25,  5.31s/it]
 58%|█████▊    | 131445/225000 [264:59:07<131:30:27,  5.06s/it]
 58%|█████▊    | 131446/225000 [264:59:12<127:50:24,  4.92s/it]
 58%|█████▊    | 131447/225000 [264:59:17<131:55:27,  5.08s/it]
 58%|█████▊    | 131448/225000 [264:59:29<185:24:17,  7.13s/it]
 58%|█████▊    | 131449/225000 [264:59:36<184:10:27,  7.09s/it]
 58%|█████▊    | 131450/225000 [264:59:40<159:39:20,  6.14s/it]
                                                               
{'loss': '0.8102', 'grad_norm': '1.022', 'learning_rate': '1.996e-05', 'epoch': '0.5842', 'num_input_tokens_seen': 440311656, 'train_runtime': '9.54e+05', 'train_tokens_per_second': '461.6'}

 58%|█████▊    | 131450/225000 [264:59:40<159:39:20,  6.14s/it]
 58%|█████▊    | 131451/225000 [264:59:44<145:03:46,  5.58s/it]
 58%|█████▊    | 131452/225000 [264:59:54<179:19:43,  6.90s/it]
 58%|█████▊    | 131453/225000 [265:00:01<174:28:43,  6.71s/it]
 58%|█████▊    | 131454/225000 [265:00:04<151:40:34,  5.84s/it]
 58%|█████▊    | 131455/225000 [265:00:08<135:48:23,  5.23s/it]
 58%|█████▊    | 131456/225000 [265:00:13<129:15:00,  4.97s/it]
 58%|█████▊    | 131457/225000 [265:00:17<127:34:22,  4.91s/it]
 58%|█████▊    | 131458/225000 [265:00:30<191:55:31,  7.39s/it]
 58%|█████▊    | 131459/225000 [265:00:35<170:40:14,  6.57s/it]
 58%|█████▊    | 131460/225000 [265:00:39<150:08:18,  5.78s/it]
 58%|█████▊    | 131461/225000 [265:00:44<141:53:20,  5.46s/it]
 58%|█████▊    | 131462/225000 [265:00:52<166:25:15,  6.41s/it]
 58%|█████▊    | 131463/225000 [265:00:56<148:12:40,  5.70s/it]
 58%|█████▊    | 131464/225000 [265:01:07<182:55:44,  7.04s/it]
 58%|█████▊    | 131465/225000 [265:01:10<153:35:17,  5.91s/it]
 58%|█████▊    | 131466/225000 [265:01:16<158:19:49,  6.09s/it]
 58%|█████▊    | 131467/225000 [265:01:21<145:28:26,  5.60s/it]
 58%|█████▊    | 131468/225000 [265:01:29<162:02:07,  6.24s/it]
 58%|█████▊    | 131469/225000 [265:01:34<158:48:39,  6.11s/it]
 58%|█████▊    | 131470/225000 [265:01:39<147:17:34,  5.67s/it]
 58%|█████▊    | 131471/225000 [265:01:44<140:50:32,  5.42s/it]
 58%|█████▊    | 131472/225000 [265:01:49<136:29:43,  5.25s/it]
 58%|█████▊    | 131473/225000 [265:01:54<133:26:10,  5.14s/it]
 58%|█████▊    | 131474/225000 [265:01:58<129:21:14,  4.98s/it]
 58%|█████▊    | 131475/225000 [265:02:04<138:33:48,  5.33s/it]
 58%|█████▊    | 131476/225000 [265:02:16<190:15:30,  7.32s/it]
 58%|█████▊    | 131477/225000 [265:02:20<158:09:43,  6.09s/it]
 58%|█████▊    | 131478/225000 [265:02:28<176:18:50,  6.79s/it]
 58%|█████▊    | 131479/225000 [265:02:32<157:25:13,  6.06s/it]
 58%|█████▊    | 131480/225000 [265:02:37<143:51:35,  5.54s/it]
 58%|█████▊    | 131481/225000 [265:02:42<138:52:17,  5.35s/it]
 58%|█████▊    | 131482/225000 [265:02:47<137:09:01,  5.28s/it]
 58%|█████▊    | 131483/225000 [265:02:52<137:15:59,  5.28s/it]
 58%|█████▊    | 131484/225000 [265:02:57<137:25:26,  5.29s/it]
 58%|█████▊    | 131485/225000 [265:03:03<140:11:18,  5.40s/it]
 58%|█████▊    | 131486/225000 [265:03:08<140:16:27,  5.40s/it]
 58%|█████▊    | 131487/225000 [265:03:14<139:09:48,  5.36s/it]
 58%|█████▊    | 131488/225000 [265:03:25<188:45:04,  7.27s/it]
 58%|█████▊    | 131489/225000 [265:03:33<189:51:07,  7.31s/it]
 58%|█████▊    | 131490/225000 [265:03:36<161:14:21,  6.21s/it]
 58%|█████▊    | 131491/225000 [265:03:43<165:39:13,  6.38s/it]
 58%|█████▊    | 131492/225000 [265:03:51<179:44:44,  6.92s/it]
 58%|█████▊    | 131493/225000 [265:03:55<152:38:57,  5.88s/it]
 58%|█████▊    | 131494/225000 [265:04:03<170:22:36,  6.56s/it]
 58%|█████▊    | 131495/225000 [265:04:06<146:35:24,  5.64s/it]
 58%|█████▊    | 131496/225000 [265:04:13<153:47:34,  5.92s/it]
 58%|█████▊    | 131497/225000 [265:04:19<151:33:56,  5.84s/it]
 58%|█████▊    | 131498/225000 [265:04:24<147:47:17,  5.69s/it]
 58%|█████▊    | 131499/225000 [265:04:32<169:36:51,  6.53s/it]
 58%|█████▊    | 131500/225000 [265:04:36<147:05:50,  5.66s/it]
                                                               
{'loss': '0.8159', 'grad_norm': '0.9706', 'learning_rate': '1.994e-05', 'epoch': '0.5844', 'num_input_tokens_seen': 440441792, 'train_runtime': '9.543e+05', 'train_tokens_per_second': '461.5'}

 58%|█████▊    | 131500/225000 [265:04:36<147:05:50,  5.66s/it]
 58%|█████▊    | 131501/225000 [265:04:41<138:35:06,  5.34s/it]
 58%|█████▊    | 131502/225000 [265:04:45<134:05:59,  5.16s/it]
 58%|█████▊    | 131503/225000 [265:04:50<132:39:15,  5.11s/it]
 58%|█████▊    | 131504/225000 [265:05:01<177:32:25,  6.84s/it]
 58%|█████▊    | 131505/225000 [265:05:08<176:39:23,  6.80s/it]
 58%|█████▊    | 131506/225000 [265:05:16<185:17:10,  7.13s/it]
 58%|█████▊    | 131507/225000 [265:05:19<155:17:46,  5.98s/it]
 58%|█████▊    | 131508/225000 [265:05:24<148:43:53,  5.73s/it]
 58%|█████▊    | 131509/225000 [265:05:31<155:34:13,  5.99s/it]
 58%|█████▊    | 131510/225000 [265:05:36<145:46:24,  5.61s/it]
 58%|█████▊    | 131511/225000 [265:05:40<138:58:57,  5.35s/it]
 58%|█████▊    | 131512/225000 [265:05:45<136:24:38,  5.25s/it]
 58%|█████▊    | 131513/225000 [265:05:50<134:33:22,  5.18s/it]
 58%|█████▊    | 131514/225000 [265:05:55<131:13:56,  5.05s/it]
 58%|█████▊    | 131515/225000 [265:06:00<129:57:22,  5.00s/it]
 58%|█████▊    | 131516/225000 [265:06:12<185:04:14,  7.13s/it]
 58%|█████▊    | 131517/225000 [265:06:18<177:12:53,  6.82s/it]
 58%|█████▊    | 131518/225000 [265:06:22<150:36:05,  5.80s/it]
 58%|█████▊    | 131519/225000 [265:06:31<176:23:24,  6.79s/it]
 58%|█████▊    | 131520/225000 [265:06:35<156:13:00,  6.02s/it]
 58%|█████▊    | 131521/225000 [265:06:40<146:47:26,  5.65s/it]
 58%|█████▊    | 131522/225000 [265:06:48<166:20:30,  6.41s/it]
 58%|█████▊    | 131523/225000 [265:06:56<177:22:54,  6.83s/it]
 58%|█████▊    | 131524/225000 [265:06:59<150:27:12,  5.79s/it]
 58%|█████▊    | 131525/225000 [265:07:07<162:49:25,  6.27s/it]
 58%|█████▊    | 131526/225000 [265:07:11<146:21:37,  5.64s/it]
 58%|█████▊    | 131527/225000 [265:07:15<136:59:07,  5.28s/it]
 58%|█████▊    | 131528/225000 [265:07:20<131:17:28,  5.06s/it]
 58%|█████▊    | 131529/225000 [265:07:26<138:39:52,  5.34s/it]
 58%|█████▊    | 131530/225000 [265:07:36<181:21:36,  6.99s/it]
 58%|█████▊    | 131531/225000 [265:07:40<152:18:29,  5.87s/it]
 58%|█████▊    | 131532/225000 [265:07:45<145:47:04,  5.62s/it]
 58%|█████▊    | 131533/225000 [265:07:53<168:45:56,  6.50s/it]
 58%|█████▊    | 131534/225000 [265:07:59<160:10:41,  6.17s/it]
 58%|█████▊    | 131535/225000 [265:08:03<147:08:42,  5.67s/it]
 58%|█████▊    | 131536/225000 [265:08:07<133:08:35,  5.13s/it]
 58%|█████▊    | 131537/225000 [265:08:13<138:05:13,  5.32s/it]
 58%|█████▊    | 131538/225000 [265:08:23<178:47:24,  6.89s/it]
 58%|█████▊    | 131539/225000 [265:08:27<152:28:28,  5.87s/it]
 58%|█████▊    | 131540/225000 [265:08:34<160:13:09,  6.17s/it]
 58%|█████▊    | 131541/225000 [265:08:40<159:29:30,  6.14s/it]
 58%|█████▊    | 131542/225000 [265:08:44<141:24:36,  5.45s/it]
 58%|█████▊    | 131543/225000 [265:08:48<130:48:30,  5.04s/it]
 58%|█████▊    | 131544/225000 [265:08:56<152:49:54,  5.89s/it]
 58%|█████▊    | 131545/225000 [265:09:02<159:22:52,  6.14s/it]
 58%|█████▊    | 131546/225000 [265:09:06<138:53:27,  5.35s/it]
 58%|█████▊    | 131547/225000 [265:09:13<151:41:56,  5.84s/it]
 58%|█████▊    | 131548/225000 [265:09:21<166:42:20,  6.42s/it]
 58%|█████▊    | 131549/225000 [265:09:24<143:55:52,  5.54s/it]
 58%|█████▊    | 131550/225000 [265:09:33<165:54:01,  6.39s/it]
                                                               
{'loss': '0.8053', 'grad_norm': '0.9973', 'learning_rate': '1.992e-05', 'epoch': '0.5847', 'num_input_tokens_seen': 440570960, 'train_runtime': '9.546e+05', 'train_tokens_per_second': '461.5'}

 58%|█████▊    | 131550/225000 [265:09:33<165:54:01,  6.39s/it]
 58%|█████▊    | 131551/225000 [265:09:36<140:07:57,  5.40s/it]
 58%|█████▊    | 131552/225000 [265:09:45<174:02:17,  6.70s/it]
 58%|█████▊    | 131553/225000 [265:09:49<151:10:45,  5.82s/it]
 58%|█████▊    | 131554/225000 [265:09:53<136:56:24,  5.28s/it]
 58%|█████▊    | 131555/225000 [265:10:03<171:10:20,  6.59s/it]
 58%|█████▊    | 131556/225000 [265:10:06<146:53:59,  5.66s/it]
 58%|█████▊    | 131557/225000 [265:10:15<173:42:28,  6.69s/it]
 58%|█████▊    | 131558/225000 [265:10:19<149:25:40,  5.76s/it]
 58%|█████▊    | 131559/225000 [265:10:23<132:28:22,  5.10s/it]
 58%|█████▊    | 131560/225000 [265:10:30<147:39:14,  5.69s/it]
 58%|█████▊    | 131561/225000 [265:10:38<172:07:53,  6.63s/it]
 58%|█████▊    | 131562/225000 [265:10:42<148:46:21,  5.73s/it]
 58%|█████▊    | 131563/225000 [265:10:46<135:44:01,  5.23s/it]
 58%|█████▊    | 131564/225000 [265:10:57<177:37:41,  6.84s/it]
 58%|█████▊    | 131565/225000 [265:11:00<152:13:33,  5.87s/it]
 58%|█████▊    | 131566/225000 [265:11:04<135:46:31,  5.23s/it]
 58%|█████▊    | 131567/225000 [265:11:14<172:16:50,  6.64s/it]
 58%|█████▊    | 131568/225000 [265:11:17<146:38:45,  5.65s/it]
 58%|█████▊    | 131569/225000 [265:11:25<163:00:04,  6.28s/it]
 58%|█████▊    | 131570/225000 [265:11:29<143:50:10,  5.54s/it]
 58%|█████▊    | 131571/225000 [265:11:36<152:45:11,  5.89s/it]
 58%|█████▊    | 131572/225000 [265:11:43<165:17:50,  6.37s/it]
 58%|█████▊    | 131573/225000 [265:11:48<156:18:28,  6.02s/it]
 58%|█████▊    | 131574/225000 [265:11:53<146:23:32,  5.64s/it]
 58%|█████▊    | 131575/225000 [265:11:58<141:58:57,  5.47s/it]
 58%|█████▊    | 131576/225000 [265:12:03<135:15:32,  5.21s/it]
 58%|█████▊    | 131577/225000 [265:12:07<128:27:18,  4.95s/it]
 58%|█████▊    | 131578/225000 [265:12:12<129:38:51,  5.00s/it]
 58%|█████▊    | 131579/225000 [265:12:17<128:40:48,  4.96s/it]
 58%|█████▊    | 131580/225000 [265:12:25<153:27:23,  5.91s/it]
 58%|█████▊    | 131581/225000 [265:12:38<203:53:03,  7.86s/it]
 58%|█████▊    | 131582/225000 [265:12:41<166:43:29,  6.42s/it]
 58%|█████▊    | 131583/225000 [265:12:45<147:06:03,  5.67s/it]
 58%|█████▊    | 131584/225000 [265:12:48<133:29:24,  5.14s/it]
 58%|█████▊    | 131585/225000 [265:12:55<145:04:36,  5.59s/it]
 58%|█████▊    | 131586/225000 [265:13:06<189:02:49,  7.29s/it]
 58%|█████▊    | 131587/225000 [265:13:10<162:08:45,  6.25s/it]
 58%|█████▊    | 131588/225000 [265:13:19<181:00:48,  6.98s/it]
 58%|█████▊    | 131589/225000 [265:13:23<156:04:33,  6.02s/it]
 58%|█████▊    | 131590/225000 [265:13:30<168:16:32,  6.49s/it]
 58%|█████▊    | 131591/225000 [265:13:34<148:06:04,  5.71s/it]
 58%|█████▊    | 131592/225000 [265:13:38<137:37:12,  5.30s/it]
 58%|█████▊    | 131593/225000 [265:13:43<128:49:41,  4.97s/it]
 58%|█████▊    | 131594/225000 [265:13:54<181:03:29,  6.98s/it]
 58%|█████▊    | 131595/225000 [265:13:59<161:13:11,  6.21s/it]
 58%|█████▊    | 131596/225000 [265:14:03<144:28:02,  5.57s/it]
 58%|█████▊    | 131597/225000 [265:14:12<169:13:41,  6.52s/it]
 58%|█████▊    | 131598/225000 [265:14:15<145:23:17,  5.60s/it]
 58%|█████▊    | 131599/225000 [265:14:19<135:12:32,  5.21s/it]
 58%|█████▊    | 131600/225000 [265:14:24<135:04:43,  5.21s/it]
                                                               
{'loss': '0.8118', 'grad_norm': '1.205', 'learning_rate': '1.991e-05', 'epoch': '0.5849', 'num_input_tokens_seen': 440700096, 'train_runtime': '9.549e+05', 'train_tokens_per_second': '461.5'}

 58%|█████▊    | 131600/225000 [265:14:24<135:04:43,  5.21s/it]
 58%|█████▊    | 131601/225000 [265:14:34<165:48:10,  6.39s/it]
 58%|█████▊    | 131602/225000 [265:14:40<165:39:52,  6.39s/it]
 58%|█████▊    | 131603/225000 [265:14:44<149:45:27,  5.77s/it]
 58%|█████▊    | 131604/225000 [265:14:49<137:04:25,  5.28s/it]
 58%|█████▊    | 131605/225000 [265:14:53<130:48:25,  5.04s/it]
 58%|█████▊    | 131606/225000 [265:15:05<188:21:18,  7.26s/it]
 58%|█████▊    | 131607/225000 [265:15:11<172:52:37,  6.66s/it]
 58%|█████▊    | 131608/225000 [265:15:15<150:38:51,  5.81s/it]
 58%|█████▊    | 131609/225000 [265:15:19<137:17:47,  5.29s/it]
 58%|█████▊    | 131610/225000 [265:15:28<167:06:23,  6.44s/it]
 58%|█████▊    | 131611/225000 [265:15:37<193:04:32,  7.44s/it]
 58%|█████▊    | 131612/225000 [265:15:41<159:26:42,  6.15s/it]
 58%|█████▊    | 131613/225000 [265:15:46<150:14:26,  5.79s/it]
 58%|█████▊    | 131614/225000 [265:15:54<170:54:32,  6.59s/it]
 58%|█████▊    | 131615/225000 [265:16:01<170:15:45,  6.56s/it]
 58%|█████▊    | 131616/225000 [265:16:06<162:44:31,  6.27s/it]
 58%|█████▊    | 131617/225000 [265:16:14<172:48:02,  6.66s/it]
 58%|█████▊    | 131618/225000 [265:16:18<151:50:53,  5.85s/it]
 58%|█████▊    | 131619/225000 [265:16:22<140:14:23,  5.41s/it]
 58%|█████▊    | 131620/225000 [265:16:26<131:00:22,  5.05s/it]
 58%|█████▊    | 131621/225000 [265:16:31<128:13:51,  4.94s/it]
 58%|█████▊    | 131622/225000 [265:16:43<181:12:15,  6.99s/it]
 58%|█████▊    | 131623/225000 [265:16:47<159:30:15,  6.15s/it]
 58%|█████▊    | 131624/225000 [265:16:52<152:43:59,  5.89s/it]
 58%|█████▊    | 131625/225000 [265:16:58<149:51:28,  5.78s/it]
 59%|█████▊    | 131626/225000 [265:17:06<169:37:30,  6.54s/it]
 59%|█████▊    | 131627/225000 [265:17:12<163:12:19,  6.29s/it]
 59%|█████▊    | 131628/225000 [265:17:16<143:38:27,  5.54s/it]
 59%|█████▊    | 131629/225000 [265:17:24<163:33:39,  6.31s/it]
 59%|█████▊    | 131630/225000 [265:17:30<165:36:38,  6.39s/it]
 59%|█████▊    | 131631/225000 [265:17:35<150:46:02,  5.81s/it]
 59%|█████▊    | 131632/225000 [265:17:39<140:53:37,  5.43s/it]
 59%|█████▊    | 131633/225000 [265:17:43<127:22:39,  4.91s/it]
 59%|█████▊    | 131634/225000 [265:17:48<126:56:39,  4.89s/it]
 59%|█████▊    | 131635/225000 [265:18:00<186:16:48,  7.18s/it]
 59%|█████▊    | 131636/225000 [265:18:04<160:13:09,  6.18s/it]
 59%|█████▊    | 131637/225000 [265:18:08<141:09:07,  5.44s/it]
 59%|█████▊    | 131638/225000 [265:18:17<173:44:56,  6.70s/it]
 59%|█████▊    | 131639/225000 [265:18:21<148:28:22,  5.73s/it]
 59%|█████▊    | 131640/225000 [265:18:29<166:23:17,  6.42s/it]
 59%|█████▊    | 131641/225000 [265:18:32<143:48:40,  5.55s/it]
 59%|█████▊    | 131642/225000 [265:18:40<159:32:36,  6.15s/it]
 59%|█████▊    | 131643/225000 [265:18:45<149:39:52,  5.77s/it]
 59%|█████▊    | 131644/225000 [265:18:54<172:02:30,  6.63s/it]
 59%|█████▊    | 131645/225000 [265:18:58<158:32:29,  6.11s/it]
 59%|█████▊    | 131646/225000 [265:19:03<146:06:56,  5.63s/it]
 59%|█████▊    | 131647/225000 [265:19:07<137:04:21,  5.29s/it]
 59%|█████▊    | 131648/225000 [265:19:12<133:37:11,  5.15s/it]
 59%|█████▊    | 131649/225000 [265:19:17<129:31:26,  4.99s/it]
 59%|█████▊    | 131650/225000 [265:19:23<136:04:31,  5.25s/it]
                                                               
{'loss': '0.8104', 'grad_norm': '1.008', 'learning_rate': '1.989e-05', 'epoch': '0.5851', 'num_input_tokens_seen': 440829672, 'train_runtime': '9.552e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131650/225000 [265:19:23<136:04:31,  5.25s/it]
 59%|█████▊    | 131651/225000 [265:19:35<191:05:10,  7.37s/it]
 59%|█████▊    | 131652/225000 [265:19:41<181:19:49,  6.99s/it]
 59%|█████▊    | 131653/225000 [265:19:45<153:05:43,  5.90s/it]
 59%|█████▊    | 131654/225000 [265:19:54<182:11:32,  7.03s/it]
 59%|█████▊    | 131655/225000 [265:19:58<156:51:43,  6.05s/it]
 59%|█████▊    | 131656/225000 [265:20:05<162:36:40,  6.27s/it]
 59%|█████▊    | 131657/225000 [265:20:08<142:21:58,  5.49s/it]
 59%|█████▊    | 131658/225000 [265:20:19<179:10:43,  6.91s/it]
 59%|█████▊    | 131659/225000 [265:20:22<150:42:23,  5.81s/it]
 59%|█████▊    | 131660/225000 [265:20:27<141:32:12,  5.46s/it]
 59%|█████▊    | 131661/225000 [265:20:34<160:50:42,  6.20s/it]
 59%|█████▊    | 131662/225000 [265:20:39<144:18:30,  5.57s/it]
 59%|█████▊    | 131663/225000 [265:20:44<143:40:57,  5.54s/it]
 59%|█████▊    | 131664/225000 [265:20:49<141:11:21,  5.45s/it]
 59%|█████▊    | 131665/225000 [265:20:54<139:19:15,  5.37s/it]
 59%|█████▊    | 131666/225000 [265:21:00<137:34:32,  5.31s/it]
 59%|█████▊    | 131667/225000 [265:21:05<135:54:40,  5.24s/it]
 59%|█████▊    | 131668/225000 [265:21:10<137:24:21,  5.30s/it]
 59%|█████▊    | 131669/225000 [265:21:15<137:03:21,  5.29s/it]
 59%|█████▊    | 131670/225000 [265:21:24<160:04:30,  6.17s/it]
 59%|█████▊    | 131671/225000 [265:21:37<212:15:14,  8.19s/it]
 59%|█████▊    | 131672/225000 [265:21:42<195:02:42,  7.52s/it]
 59%|█████▊    | 131673/225000 [265:21:47<173:11:18,  6.68s/it]
 59%|█████▊    | 131674/225000 [265:21:52<157:58:44,  6.09s/it]
 59%|█████▊    | 131675/225000 [265:21:56<145:30:26,  5.61s/it]
 59%|█████▊    | 131676/225000 [265:22:01<137:25:58,  5.30s/it]
 59%|█████▊    | 131677/225000 [265:22:12<180:35:26,  6.97s/it]
 59%|█████▊    | 131678/225000 [265:22:18<176:09:06,  6.80s/it]
 59%|█████▊    | 131679/225000 [265:22:23<160:05:33,  6.18s/it]
 59%|█████▊    | 131680/225000 [265:22:28<148:55:14,  5.74s/it]
 59%|█████▊    | 131681/225000 [265:22:32<139:29:19,  5.38s/it]
 59%|█████▊    | 131682/225000 [265:22:40<159:11:10,  6.14s/it]
 59%|█████▊    | 131683/225000 [265:22:51<197:44:29,  7.63s/it]
 59%|█████▊    | 131684/225000 [265:22:55<164:50:24,  6.36s/it]
 59%|█████▊    | 131685/225000 [265:23:02<170:52:32,  6.59s/it]
 59%|█████▊    | 131686/225000 [265:23:07<163:13:53,  6.30s/it]
 59%|█████▊    | 131687/225000 [265:23:13<154:05:09,  5.94s/it]
 59%|█████▊    | 131688/225000 [265:23:19<154:25:56,  5.96s/it]
 59%|█████▊    | 131689/225000 [265:23:25<161:31:13,  6.23s/it]
 59%|█████▊    | 131690/225000 [265:23:31<153:18:06,  5.91s/it]
 59%|█████▊    | 131691/225000 [265:23:35<142:56:01,  5.51s/it]
 59%|█████▊    | 131692/225000 [265:23:41<143:56:24,  5.55s/it]
 59%|█████▊    | 131693/225000 [265:23:46<141:39:39,  5.47s/it]
 59%|█████▊    | 131694/225000 [265:23:52<142:19:12,  5.49s/it]
 59%|█████▊    | 131695/225000 [265:24:01<171:31:48,  6.62s/it]
 59%|█████▊    | 131696/225000 [265:24:05<154:30:46,  5.96s/it]
 59%|█████▊    | 131697/225000 [265:24:09<139:07:35,  5.37s/it]
 59%|█████▊    | 131698/225000 [265:24:14<135:33:39,  5.23s/it]
 59%|█████▊    | 131699/225000 [265:24:24<171:45:05,  6.63s/it]
 59%|█████▊    | 131700/225000 [265:24:28<152:24:22,  5.88s/it]
                                                               
{'loss': '0.8163', 'grad_norm': '1.03', 'learning_rate': '1.987e-05', 'epoch': '0.5853', 'num_input_tokens_seen': 440959520, 'train_runtime': '9.555e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131700/225000 [265:24:28<152:24:22,  5.88s/it]
 59%|█████▊    | 131701/225000 [265:24:34<151:19:10,  5.84s/it]
 59%|█████▊    | 131702/225000 [265:24:42<169:21:55,  6.54s/it]
 59%|█████▊    | 131703/225000 [265:24:46<146:57:38,  5.67s/it]
 59%|█████▊    | 131704/225000 [265:24:54<166:26:26,  6.42s/it]
 59%|█████▊    | 131705/225000 [265:24:58<145:48:09,  5.63s/it]
 59%|█████▊    | 131706/225000 [265:25:06<169:23:21,  6.54s/it]
 59%|█████▊    | 131707/225000 [265:25:11<154:36:36,  5.97s/it]
 59%|█████▊    | 131708/225000 [265:25:15<143:10:41,  5.53s/it]
 59%|█████▊    | 131709/225000 [265:25:20<135:42:47,  5.24s/it]
 59%|█████▊    | 131710/225000 [265:25:25<133:20:22,  5.15s/it]
 59%|█████▊    | 131711/225000 [265:25:36<181:17:13,  7.00s/it]
 59%|█████▊    | 131712/225000 [265:25:42<169:19:41,  6.53s/it]
 59%|█████▊    | 131713/225000 [265:25:48<165:03:40,  6.37s/it]
 59%|█████▊    | 131714/225000 [265:25:54<164:18:25,  6.34s/it]
 59%|█████▊    | 131715/225000 [265:25:58<145:26:05,  5.61s/it]
 59%|█████▊    | 131716/225000 [265:26:02<131:44:48,  5.08s/it]
 59%|█████▊    | 131717/225000 [265:26:09<147:23:30,  5.69s/it]
 59%|█████▊    | 131718/225000 [265:26:18<176:17:09,  6.80s/it]
 59%|█████▊    | 131719/225000 [265:26:22<148:41:09,  5.74s/it]
 59%|█████▊    | 131720/225000 [265:26:30<170:16:51,  6.57s/it]
 59%|█████▊    | 131721/225000 [265:26:34<150:06:28,  5.79s/it]
 59%|█████▊    | 131722/225000 [265:26:39<146:05:26,  5.64s/it]
 59%|█████▊    | 131723/225000 [265:26:44<142:39:43,  5.51s/it]
 59%|█████▊    | 131724/225000 [265:26:50<143:33:49,  5.54s/it]
 59%|█████▊    | 131725/225000 [265:26:55<141:25:55,  5.46s/it]
 59%|█████▊    | 131726/225000 [265:27:02<146:40:36,  5.66s/it]
 59%|█████▊    | 131727/225000 [265:27:10<170:14:13,  6.57s/it]
 59%|█████▊    | 131728/225000 [265:27:15<155:16:22,  5.99s/it]
 59%|█████▊    | 131729/225000 [265:27:20<144:59:53,  5.60s/it]
 59%|█████▊    | 131730/225000 [265:27:24<134:04:27,  5.17s/it]
 59%|█████▊    | 131731/225000 [265:27:35<183:04:15,  7.07s/it]
 59%|█████▊    | 131732/225000 [265:27:40<168:34:09,  6.51s/it]
 59%|█████▊    | 131733/225000 [265:27:45<155:38:54,  6.01s/it]
 59%|█████▊    | 131734/225000 [265:27:50<143:31:18,  5.54s/it]
 59%|█████▊    | 131735/225000 [265:27:55<138:42:42,  5.35s/it]
 59%|█████▊    | 131736/225000 [265:27:59<134:02:03,  5.17s/it]
 59%|█████▊    | 131737/225000 [265:28:08<157:52:08,  6.09s/it]
 59%|█████▊    | 131738/225000 [265:28:17<186:27:37,  7.20s/it]
 59%|█████▊    | 131739/225000 [265:28:22<166:54:55,  6.44s/it]
 59%|█████▊    | 131740/225000 [265:28:26<150:20:26,  5.80s/it]
 59%|█████▊    | 131741/225000 [265:28:31<137:58:35,  5.33s/it]
 59%|█████▊    | 131742/225000 [265:28:37<149:29:16,  5.77s/it]
 59%|█████▊    | 131743/225000 [265:28:48<187:04:15,  7.22s/it]
 59%|█████▊    | 131744/225000 [265:28:51<157:52:59,  6.09s/it]
 59%|█████▊    | 131745/225000 [265:29:01<186:38:33,  7.21s/it]
 59%|█████▊    | 131746/225000 [265:29:05<158:18:59,  6.11s/it]
 59%|█████▊    | 131747/225000 [265:29:11<160:22:16,  6.19s/it]
 59%|█████▊    | 131748/225000 [265:29:18<165:11:03,  6.38s/it]
 59%|█████▊    | 131749/225000 [265:29:23<152:12:22,  5.88s/it]
 59%|█████▊    | 131750/225000 [265:29:28<150:27:59,  5.81s/it]
                                                               
{'loss': '0.8172', 'grad_norm': '1.077', 'learning_rate': '1.985e-05', 'epoch': '0.5856', 'num_input_tokens_seen': 441087936, 'train_runtime': '9.558e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131750/225000 [265:29:28<150:27:59,  5.81s/it]
 59%|█████▊    | 131751/225000 [265:29:34<147:43:47,  5.70s/it]
 59%|█████▊    | 131752/225000 [265:29:39<143:03:27,  5.52s/it]
 59%|█████▊    | 131753/225000 [265:29:44<138:42:00,  5.35s/it]
 59%|█████▊    | 131754/225000 [265:29:49<136:28:35,  5.27s/it]
 59%|█████▊    | 131755/225000 [265:29:54<134:36:19,  5.20s/it]
 59%|█████▊    | 131756/225000 [265:29:59<129:46:50,  5.01s/it]
 59%|█████▊    | 131757/225000 [265:30:06<145:53:38,  5.63s/it]
 59%|█████▊    | 131758/225000 [265:30:11<142:44:32,  5.51s/it]
 59%|█████▊    | 131759/225000 [265:30:16<142:24:00,  5.50s/it]
 59%|█████▊    | 131760/225000 [265:30:26<172:10:38,  6.65s/it]
 59%|█████▊    | 131761/225000 [265:30:36<203:12:24,  7.85s/it]
 59%|█████▊    | 131762/225000 [265:30:40<169:08:41,  6.53s/it]
 59%|█████▊    | 131763/225000 [265:30:46<169:10:57,  6.53s/it]
 59%|█████▊    | 131764/225000 [265:30:54<174:41:48,  6.75s/it]
 59%|█████▊    | 131765/225000 [265:30:58<157:15:00,  6.07s/it]
 59%|█████▊    | 131766/225000 [265:31:02<143:51:42,  5.55s/it]
 59%|█████▊    | 131767/225000 [265:31:11<163:43:05,  6.32s/it]
 59%|█████▊    | 131768/225000 [265:31:18<175:44:17,  6.79s/it]
 59%|█████▊    | 131769/225000 [265:31:22<149:09:55,  5.76s/it]
 59%|█████▊    | 131770/225000 [265:31:29<159:08:57,  6.15s/it]
 59%|█████▊    | 131771/225000 [265:31:35<163:09:23,  6.30s/it]
 59%|█████▊    | 131772/225000 [265:31:40<146:06:20,  5.64s/it]
 59%|█████▊    | 131773/225000 [265:31:45<144:16:56,  5.57s/it]
 59%|█████▊    | 131774/225000 [265:31:50<142:57:16,  5.52s/it]
 59%|█████▊    | 131775/225000 [265:31:56<140:43:38,  5.43s/it]
 59%|█████▊    | 131776/225000 [265:32:00<133:47:19,  5.17s/it]
 59%|█████▊    | 131777/225000 [265:32:06<135:38:20,  5.24s/it]
 59%|█████▊    | 131778/225000 [265:32:14<158:21:22,  6.12s/it]
 59%|█████▊    | 131779/225000 [265:32:26<208:46:55,  8.06s/it]
 59%|█████▊    | 131780/225000 [265:32:30<177:40:21,  6.86s/it]
 59%|█████▊    | 131781/225000 [265:32:34<154:40:50,  5.97s/it]
 59%|█████▊    | 131782/225000 [265:32:41<163:42:40,  6.32s/it]
 59%|█████▊    | 131783/225000 [265:32:49<171:31:51,  6.62s/it]
 59%|█████▊    | 131784/225000 [265:32:53<153:23:11,  5.92s/it]
 59%|█████▊    | 131785/225000 [265:32:57<138:35:13,  5.35s/it]
 59%|█████▊    | 131786/225000 [265:33:01<128:53:12,  4.98s/it]
 59%|█████▊    | 131787/225000 [265:33:14<187:35:45,  7.25s/it]
 59%|█████▊    | 131788/225000 [265:33:17<156:41:55,  6.05s/it]
 59%|█████▊    | 131789/225000 [265:33:25<172:29:32,  6.66s/it]
 59%|█████▊    | 131790/225000 [265:33:29<147:36:37,  5.70s/it]
 59%|█████▊    | 131791/225000 [265:33:38<180:07:34,  6.96s/it]
 59%|█████▊    | 131792/225000 [265:33:43<161:06:44,  6.22s/it]
 59%|█████▊    | 131793/225000 [265:33:47<145:36:48,  5.62s/it]
 59%|█████▊    | 131794/225000 [265:33:51<134:32:39,  5.20s/it]
 59%|█████▊    | 131795/225000 [265:33:59<151:14:09,  5.84s/it]
 59%|█████▊    | 131796/225000 [265:34:10<192:52:34,  7.45s/it]
 59%|█████▊    | 131797/225000 [265:34:13<160:39:32,  6.21s/it]
 59%|█████▊    | 131798/225000 [265:34:21<170:19:32,  6.58s/it]
 59%|█████▊    | 131799/225000 [265:34:26<164:15:53,  6.34s/it]
 59%|█████▊    | 131800/225000 [265:34:33<163:46:53,  6.33s/it]
                                                               
{'loss': '0.8139', 'grad_norm': '1.081', 'learning_rate': '1.984e-05', 'epoch': '0.5858', 'num_input_tokens_seen': 441219080, 'train_runtime': '9.561e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131800/225000 [265:34:33<163:46:53,  6.33s/it]
 59%|█████▊    | 131801/225000 [265:34:39<160:01:05,  6.18s/it]
 59%|█████▊    | 131802/225000 [265:34:42<141:27:36,  5.46s/it]
 59%|█████▊    | 131803/225000 [265:34:50<159:16:50,  6.15s/it]
 59%|█████▊    | 131804/225000 [265:34:54<142:04:16,  5.49s/it]
 59%|█████▊    | 131805/225000 [265:34:58<130:15:31,  5.03s/it]
 59%|█████▊    | 131806/225000 [265:35:07<158:33:18,  6.12s/it]
 59%|█████▊    | 131807/225000 [265:35:14<166:44:42,  6.44s/it]
 59%|█████▊    | 131808/225000 [265:35:18<149:56:51,  5.79s/it]
 59%|█████▊    | 131809/225000 [265:35:22<136:12:45,  5.26s/it]
 59%|█████▊    | 131810/225000 [265:35:33<178:52:23,  6.91s/it]
 59%|█████▊    | 131811/225000 [265:35:38<167:03:58,  6.45s/it]
 59%|█████▊    | 131812/225000 [265:35:42<146:42:21,  5.67s/it]
 59%|█████▊    | 131813/225000 [265:35:47<138:43:07,  5.36s/it]
 59%|█████▊    | 131814/225000 [265:35:58<184:05:01,  7.11s/it]
 59%|█████▊    | 131815/225000 [265:36:02<156:22:29,  6.04s/it]
 59%|█████▊    | 131816/225000 [265:36:06<147:53:54,  5.71s/it]
 59%|█████▊    | 131817/225000 [265:36:11<138:22:23,  5.35s/it]
 59%|█████▊    | 131818/225000 [265:36:16<135:09:08,  5.22s/it]
 59%|█████▊    | 131819/225000 [265:36:21<134:35:33,  5.20s/it]
 59%|█████▊    | 131820/225000 [265:36:26<132:32:34,  5.12s/it]
 59%|█████▊    | 131821/225000 [265:36:31<133:19:02,  5.15s/it]
 59%|█████▊    | 131822/225000 [265:36:41<167:56:35,  6.49s/it]
 59%|█████▊    | 131823/225000 [265:36:51<196:18:28,  7.58s/it]
 59%|█████▊    | 131824/225000 [265:36:54<162:51:16,  6.29s/it]
 59%|█████▊    | 131825/225000 [265:37:02<172:56:44,  6.68s/it]
 59%|█████▊    | 131826/225000 [265:37:05<149:15:36,  5.77s/it]
 59%|█████▊    | 131827/225000 [265:37:14<171:19:11,  6.62s/it]
 59%|█████▊    | 131828/225000 [265:37:18<147:33:05,  5.70s/it]
 59%|█████▊    | 131829/225000 [265:37:24<154:23:42,  5.97s/it]
 59%|█████▊    | 131830/225000 [265:37:32<166:38:31,  6.44s/it]
 59%|█████▊    | 131831/225000 [265:37:36<152:33:46,  5.89s/it]
 59%|█████▊    | 131832/225000 [265:37:41<140:54:00,  5.44s/it]
 59%|█████▊    | 131833/225000 [265:37:46<137:15:25,  5.30s/it]
 59%|█████▊    | 131834/225000 [265:37:51<134:17:17,  5.19s/it]
 59%|█████▊    | 131835/225000 [265:37:55<128:14:20,  4.96s/it]
 59%|█████▊    | 131836/225000 [265:38:00<127:30:09,  4.93s/it]
 59%|█████▊    | 131837/225000 [265:38:06<134:54:09,  5.21s/it]
 59%|█████▊    | 131838/225000 [265:38:18<190:28:11,  7.36s/it]
 59%|█████▊    | 131839/225000 [265:38:23<167:59:34,  6.49s/it]
 59%|█████▊    | 131840/225000 [265:38:30<178:01:17,  6.88s/it]
 59%|█████▊    | 131841/225000 [265:38:37<178:58:10,  6.92s/it]
 59%|█████▊    | 131842/225000 [265:38:45<181:57:51,  7.03s/it]
 59%|█████▊    | 131843/225000 [265:38:49<160:54:18,  6.22s/it]
 59%|█████▊    | 131844/225000 [265:38:58<183:19:36,  7.08s/it]
 59%|█████▊    | 131845/225000 [265:39:01<153:50:36,  5.95s/it]
 59%|█████▊    | 131846/225000 [265:39:09<164:52:07,  6.37s/it]
 59%|█████▊    | 131847/225000 [265:39:14<154:01:04,  5.95s/it]
 59%|█████▊    | 131848/225000 [265:39:17<136:10:11,  5.26s/it]
 59%|█████▊    | 131849/225000 [265:39:21<123:30:53,  4.77s/it]
 59%|█████▊    | 131850/225000 [265:39:34<185:00:30,  7.15s/it]
                                                               
{'loss': '0.8083', 'grad_norm': '0.8982', 'learning_rate': '1.982e-05', 'epoch': '0.586', 'num_input_tokens_seen': 441348856, 'train_runtime': '9.564e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131850/225000 [265:39:34<185:00:30,  7.15s/it]
 59%|█████▊    | 131851/225000 [265:39:38<160:30:25,  6.20s/it]
 59%|█████▊    | 131852/225000 [265:39:41<139:44:47,  5.40s/it]
 59%|█████▊    | 131853/225000 [265:39:50<164:51:25,  6.37s/it]
 59%|█████▊    | 131854/225000 [265:39:57<170:11:38,  6.58s/it]
 59%|█████▊    | 131855/225000 [265:40:01<150:47:47,  5.83s/it]
 59%|█████▊    | 131856/225000 [265:40:05<139:04:16,  5.38s/it]
 59%|█████▊    | 131857/225000 [265:40:09<128:43:23,  4.98s/it]
 59%|█████▊    | 131858/225000 [265:40:14<123:50:56,  4.79s/it]
 59%|█████▊    | 131859/225000 [265:40:18<118:06:32,  4.57s/it]
 59%|█████▊    | 131860/225000 [265:40:29<167:36:56,  6.48s/it]
 59%|█████▊    | 131861/225000 [265:40:35<165:25:57,  6.39s/it]
 59%|█████▊    | 131862/225000 [265:40:39<146:13:56,  5.65s/it]
 59%|█████▊    | 131863/225000 [265:40:43<132:36:29,  5.13s/it]
 59%|█████▊    | 131864/225000 [265:40:48<133:33:38,  5.16s/it]
 59%|█████▊    | 131865/225000 [265:40:53<132:38:09,  5.13s/it]
 59%|█████▊    | 131866/225000 [265:40:58<132:12:43,  5.11s/it]
 59%|█████▊    | 131867/225000 [265:41:03<130:51:34,  5.06s/it]
 59%|█████▊    | 131868/225000 [265:41:08<132:53:05,  5.14s/it]
 59%|█████▊    | 131869/225000 [265:41:15<141:30:32,  5.47s/it]
 59%|█████▊    | 131870/225000 [265:41:27<194:34:31,  7.52s/it]
 59%|█████▊    | 131871/225000 [265:41:32<174:22:43,  6.74s/it]
 59%|█████▊    | 131872/225000 [265:41:40<188:29:24,  7.29s/it]
 59%|█████▊    | 131873/225000 [265:41:44<156:40:52,  6.06s/it]
 59%|█████▊    | 131874/225000 [265:41:52<174:27:32,  6.74s/it]
 59%|█████▊    | 131875/225000 [265:41:57<164:14:57,  6.35s/it]
 59%|█████▊    | 131876/225000 [265:42:03<155:42:18,  6.02s/it]
 59%|█████▊    | 131877/225000 [265:42:08<148:05:02,  5.72s/it]
 59%|█████▊    | 131878/225000 [265:42:12<139:02:30,  5.38s/it]
 59%|█████▊    | 131879/225000 [265:42:17<135:56:05,  5.26s/it]
 59%|█████▊    | 131880/225000 [265:42:23<136:32:19,  5.28s/it]
 59%|█████▊    | 131881/225000 [265:42:28<134:33:28,  5.20s/it]
 59%|█████▊    | 131882/225000 [265:42:33<135:01:35,  5.22s/it]
 59%|█████▊    | 131883/225000 [265:42:38<135:44:03,  5.25s/it]
 59%|█████▊    | 131884/225000 [265:42:45<147:42:34,  5.71s/it]
 59%|█████▊    | 131885/225000 [265:42:57<199:27:28,  7.71s/it]
 59%|█████▊    | 131886/225000 [265:43:04<190:43:16,  7.37s/it]
 59%|█████▊    | 131887/225000 [265:43:08<161:23:21,  6.24s/it]
 59%|█████▊    | 131888/225000 [265:43:14<161:28:37,  6.24s/it]
 59%|█████▊    | 131889/225000 [265:43:21<171:13:36,  6.62s/it]
 59%|█████▊    | 131890/225000 [265:43:28<172:58:23,  6.69s/it]
 59%|█████▊    | 131891/225000 [265:43:33<157:38:06,  6.09s/it]
 59%|█████▊    | 131892/225000 [265:43:37<139:59:41,  5.41s/it]
 59%|█████▊    | 131893/225000 [265:43:42<142:11:06,  5.50s/it]
 59%|█████▊    | 131894/225000 [265:43:51<163:59:28,  6.34s/it]
 59%|█████▊    | 131895/225000 [265:43:55<151:43:55,  5.87s/it]
 59%|█████▊    | 131896/225000 [265:44:00<143:46:59,  5.56s/it]
 59%|█████▊    | 131897/225000 [265:44:05<136:06:55,  5.26s/it]
 59%|█████▊    | 131898/225000 [265:44:10<132:05:11,  5.11s/it]
 59%|█████▊    | 131899/225000 [265:44:16<141:46:51,  5.48s/it]
 59%|█████▊    | 131900/225000 [265:44:27<184:50:22,  7.15s/it]
                                                               
{'loss': '0.8048', 'grad_norm': '1.129', 'learning_rate': '1.98e-05', 'epoch': '0.5862', 'num_input_tokens_seen': 441478120, 'train_runtime': '9.567e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131900/225000 [265:44:27<184:50:22,  7.15s/it]
 59%|█████▊    | 131901/225000 [265:44:30<154:42:40,  5.98s/it]
 59%|█████▊    | 131902/225000 [265:44:34<137:04:36,  5.30s/it]
 59%|█████▊    | 131903/225000 [265:44:39<134:35:28,  5.20s/it]
 59%|█████▊    | 131904/225000 [265:44:44<130:18:01,  5.04s/it]
 59%|█████▊    | 131905/225000 [265:44:48<127:52:54,  4.95s/it]
 59%|█████▊    | 131906/225000 [265:44:53<125:04:36,  4.84s/it]
 59%|█████▊    | 131907/225000 [265:44:58<126:22:48,  4.89s/it]
 59%|█████▊    | 131908/225000 [265:45:07<162:31:42,  6.29s/it]
 59%|█████▊    | 131909/225000 [265:45:16<183:13:53,  7.09s/it]
 59%|█████▊    | 131910/225000 [265:45:20<153:29:49,  5.94s/it]
 59%|█████▊    | 131911/225000 [265:45:26<158:02:56,  6.11s/it]
 59%|█████▊    | 131912/225000 [265:45:34<168:53:24,  6.53s/it]
 59%|█████▊    | 131913/225000 [265:45:39<159:48:50,  6.18s/it]
 59%|█████▊    | 131914/225000 [265:45:45<160:00:47,  6.19s/it]
 59%|█████▊    | 131915/225000 [265:45:50<148:50:44,  5.76s/it]
 59%|█████▊    | 131916/225000 [265:45:56<148:03:35,  5.73s/it]
 59%|█████▊    | 131917/225000 [265:46:01<146:46:59,  5.68s/it]
 59%|█████▊    | 131918/225000 [265:46:07<144:01:03,  5.57s/it]
 59%|█████▊    | 131919/225000 [265:46:12<140:16:42,  5.43s/it]
 59%|█████▊    | 131920/225000 [265:46:17<141:33:23,  5.47s/it]
 59%|█████▊    | 131921/225000 [265:46:22<138:51:10,  5.37s/it]
 59%|█████▊    | 131922/225000 [265:46:28<139:21:06,  5.39s/it]
 59%|█████▊    | 131923/225000 [265:46:33<139:38:17,  5.40s/it]
 59%|█████▊    | 131924/225000 [265:46:39<140:33:09,  5.44s/it]
 59%|█████▊    | 131925/225000 [265:46:44<141:11:41,  5.46s/it]
 59%|█████▊    | 131926/225000 [265:46:51<149:31:55,  5.78s/it]
 59%|█████▊    | 131927/225000 [265:47:02<190:02:57,  7.35s/it]
 59%|█████▊    | 131928/225000 [265:47:06<169:44:05,  6.57s/it]
 59%|█████▊    | 131929/225000 [265:47:11<155:22:27,  6.01s/it]
 59%|█████▊    | 131930/225000 [265:47:15<140:47:23,  5.45s/it]
 59%|█████▊    | 131931/225000 [265:47:26<178:23:14,  6.90s/it]
 59%|█████▊    | 131932/225000 [265:47:33<185:47:16,  7.19s/it]
 59%|█████▊    | 131933/225000 [265:47:39<170:07:04,  6.58s/it]
 59%|█████▊    | 131934/225000 [265:47:45<171:43:48,  6.64s/it]
 59%|█████▊    | 131935/225000 [265:47:52<167:36:54,  6.48s/it]
 59%|█████▊    | 131936/225000 [265:47:58<164:53:16,  6.38s/it]
 59%|█████▊    | 131937/225000 [265:48:02<150:32:38,  5.82s/it]
 59%|█████▊    | 131938/225000 [265:48:07<140:58:45,  5.45s/it]
 59%|█████▊    | 131939/225000 [265:48:11<129:39:58,  5.02s/it]
 59%|█████▊    | 131940/225000 [265:48:23<182:42:36,  7.07s/it]
 59%|█████▊    | 131941/225000 [265:48:27<162:58:45,  6.30s/it]
 59%|█████▊    | 131942/225000 [265:48:34<165:36:37,  6.41s/it]
 59%|█████▊    | 131943/225000 [265:48:41<170:13:24,  6.59s/it]
 59%|█████▊    | 131944/225000 [265:48:44<143:07:54,  5.54s/it]
 59%|█████▊    | 131945/225000 [265:48:49<138:15:34,  5.35s/it]
 59%|█████▊    | 131946/225000 [265:48:54<138:12:32,  5.35s/it]
 59%|█████▊    | 131947/225000 [265:48:59<134:26:15,  5.20s/it]
 59%|█████▊    | 131948/225000 [265:49:04<136:06:24,  5.27s/it]
 59%|█████▊    | 131949/225000 [265:49:10<136:08:02,  5.27s/it]
 59%|█████▊    | 131950/225000 [265:49:15<135:51:18,  5.26s/it]
                                                               
{'loss': '0.8036', 'grad_norm': '1.055', 'learning_rate': '1.978e-05', 'epoch': '0.5864', 'num_input_tokens_seen': 441606816, 'train_runtime': '9.57e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 131950/225000 [265:49:15<135:51:18,  5.26s/it]
 59%|█████▊    | 131951/225000 [265:49:20<132:20:36,  5.12s/it]
 59%|█████▊    | 131952/225000 [265:49:32<186:16:14,  7.21s/it]
 59%|█████▊    | 131953/225000 [265:49:38<179:20:37,  6.94s/it]
 59%|█████▊    | 131954/225000 [265:49:43<166:59:27,  6.46s/it]
 59%|█████▊    | 131955/225000 [265:49:51<173:05:41,  6.70s/it]
 59%|█████▊    | 131956/225000 [265:49:54<147:34:44,  5.71s/it]
 59%|█████▊    | 131957/225000 [265:50:02<167:48:11,  6.49s/it]
 59%|█████▊    | 131958/225000 [265:50:06<148:06:56,  5.73s/it]
 59%|█████▊    | 131959/225000 [265:50:13<152:11:34,  5.89s/it]
 59%|█████▊    | 131960/225000 [265:50:20<160:40:02,  6.22s/it]
 59%|█████▊    | 131961/225000 [265:50:23<139:05:00,  5.38s/it]
 59%|█████▊    | 131962/225000 [265:50:34<179:55:14,  6.96s/it]
 59%|█████▊    | 131963/225000 [265:50:37<148:26:22,  5.74s/it]
 59%|█████▊    | 131964/225000 [265:50:42<142:06:41,  5.50s/it]
 59%|█████▊    | 131965/225000 [265:50:47<140:05:24,  5.42s/it]
 59%|█████▊    | 131966/225000 [265:50:52<136:23:24,  5.28s/it]
 59%|█████▊    | 131967/225000 [265:50:57<133:49:29,  5.18s/it]
 59%|█████▊    | 131968/225000 [265:51:02<132:50:40,  5.14s/it]
 59%|█████▊    | 131969/225000 [265:51:07<134:37:11,  5.21s/it]
 59%|█████▊    | 131970/225000 [265:51:12<135:51:03,  5.26s/it]
 59%|█████▊    | 131971/225000 [265:51:17<133:43:05,  5.17s/it]
 59%|█████▊    | 131972/225000 [265:51:23<135:19:50,  5.24s/it]
 59%|█████▊    | 131973/225000 [265:51:34<184:16:35,  7.13s/it]
 59%|█████▊    | 131974/225000 [265:51:41<177:01:02,  6.85s/it]
 59%|█████▊    | 131975/225000 [265:51:44<153:12:59,  5.93s/it]
 59%|█████▊    | 131976/225000 [265:51:54<182:32:10,  7.06s/it]
 59%|█████▊    | 131977/225000 [265:51:57<153:42:28,  5.95s/it]
 59%|█████▊    | 131978/225000 [265:52:04<162:16:18,  6.28s/it]
 59%|█████▊    | 131979/225000 [265:52:08<141:14:31,  5.47s/it]
 59%|█████▊    | 131980/225000 [265:52:14<147:04:16,  5.69s/it]
 59%|█████▊    | 131981/225000 [265:52:22<160:12:00,  6.20s/it]
 59%|█████▊    | 131982/225000 [265:52:27<150:06:00,  5.81s/it]
 59%|█████▊    | 131983/225000 [265:52:31<142:14:57,  5.51s/it]
 59%|█████▊    | 131984/225000 [265:52:36<132:08:13,  5.11s/it]
 59%|█████▊    | 131985/225000 [265:52:41<131:02:44,  5.07s/it]
 59%|█████▊    | 131986/225000 [265:52:45<127:39:36,  4.94s/it]
 59%|█████▊    | 131987/225000 [265:52:58<187:24:33,  7.25s/it]
 59%|█████▊    | 131988/225000 [265:53:01<156:30:18,  6.06s/it]
 59%|█████▊    | 131989/225000 [265:53:09<173:59:45,  6.73s/it]
 59%|█████▊    | 131990/225000 [265:53:14<155:48:54,  6.03s/it]
 59%|█████▊    | 131991/225000 [265:53:18<140:54:05,  5.45s/it]
 59%|█████▊    | 131992/225000 [265:53:22<127:36:22,  4.94s/it]
 59%|█████▊    | 131993/225000 [265:53:31<161:11:26,  6.24s/it]
 59%|█████▊    | 131994/225000 [265:53:37<156:52:10,  6.07s/it]
 59%|█████▊    | 131995/225000 [265:53:41<146:39:05,  5.68s/it]
 59%|█████▊    | 131996/225000 [265:53:46<138:59:24,  5.38s/it]
 59%|█████▊    | 131997/225000 [265:53:51<135:00:24,  5.23s/it]
 59%|█████▊    | 131998/225000 [265:53:55<126:36:40,  4.90s/it]
 59%|█████▊    | 131999/225000 [265:54:02<139:28:14,  5.40s/it]
 59%|█████▊    | 132000/225000 [265:54:12<181:34:35,  7.03s/it]
                                                               
{'loss': '0.8106', 'grad_norm': '1.088', 'learning_rate': '1.976e-05', 'epoch': '0.5867', 'num_input_tokens_seen': 441735848, 'train_runtime': '9.573e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 132000/225000 [265:54:12<181:34:35,  7.03s/it]
 59%|█████▊    | 132001/225000 [265:54:16<154:05:35,  5.96s/it]
 59%|█████▊    | 132002/225000 [265:54:21<144:47:54,  5.61s/it]
 59%|█████▊    | 132003/225000 [265:54:25<137:24:29,  5.32s/it]
 59%|█████▊    | 132004/225000 [265:54:30<131:30:11,  5.09s/it]
 59%|█████▊    | 132005/225000 [265:54:35<130:40:19,  5.06s/it]
 59%|█████▊    | 132006/225000 [265:54:40<131:03:06,  5.07s/it]
 59%|█████▊    | 132007/225000 [265:54:52<181:41:51,  7.03s/it]
 59%|█████▊    | 132008/225000 [265:54:58<175:46:09,  6.80s/it]
 59%|█████▊    | 132009/225000 [265:55:03<163:21:33,  6.32s/it]
 59%|█████▊    | 132010/225000 [265:55:07<141:32:04,  5.48s/it]
 59%|█████▊    | 132011/225000 [265:55:17<180:25:36,  6.99s/it]
 59%|█████▊    | 132012/225000 [265:55:21<153:10:04,  5.93s/it]
 59%|█████▊    | 132013/225000 [265:55:25<142:48:51,  5.53s/it]
 59%|█████▊    | 132014/225000 [265:55:32<156:15:57,  6.05s/it]
 59%|█████▊    | 132015/225000 [265:55:36<140:55:46,  5.46s/it]
 59%|█████▊    | 132016/225000 [265:55:41<135:38:42,  5.25s/it]
 59%|█████▊    | 132017/225000 [265:55:52<177:36:05,  6.88s/it]
 59%|█████▊    | 132018/225000 [265:55:56<153:15:58,  5.93s/it]
 59%|█████▊    | 132019/225000 [265:56:04<171:45:53,  6.65s/it]
 59%|█████▊    | 132020/225000 [265:56:07<145:41:12,  5.64s/it]
 59%|█████▊    | 132021/225000 [265:56:12<138:55:43,  5.38s/it]
 59%|█████▊    | 132022/225000 [265:56:19<155:01:27,  6.00s/it]
 59%|█████▊    | 132023/225000 [265:56:24<146:33:14,  5.67s/it]
 59%|█████▊    | 132024/225000 [265:56:31<154:46:35,  5.99s/it]
 59%|█████▊    | 132025/225000 [265:56:38<162:46:34,  6.30s/it]
 59%|█████▊    | 132026/225000 [265:56:42<140:15:34,  5.43s/it]
 59%|█████▊    | 132027/225000 [265:56:51<174:31:04,  6.76s/it]
 59%|█████▊    | 132028/225000 [265:56:55<151:22:24,  5.86s/it]
 59%|█████▊    | 132029/225000 [265:57:04<175:59:35,  6.81s/it]
 59%|█████▊    | 132030/225000 [265:57:09<159:47:24,  6.19s/it]
 59%|█████▊    | 132031/225000 [265:57:14<147:41:16,  5.72s/it]
 59%|█████▊    | 132032/225000 [265:57:18<137:32:31,  5.33s/it]
 59%|█████▊    | 132033/225000 [265:57:23<132:33:57,  5.13s/it]
 59%|█████▊    | 132034/225000 [265:57:27<128:41:33,  4.98s/it]
 59%|█████▊    | 132035/225000 [265:57:32<126:13:39,  4.89s/it]
 59%|█████▊    | 132036/225000 [265:57:37<127:44:19,  4.95s/it]
 59%|█████▊    | 132037/225000 [265:57:49<185:19:38,  7.18s/it]
 59%|█████▊    | 132038/225000 [265:57:55<171:07:16,  6.63s/it]
 59%|█████▊    | 132039/225000 [265:58:00<162:41:43,  6.30s/it]
 59%|█████▊    | 132040/225000 [265:58:08<170:17:43,  6.59s/it]
 59%|█████▊    | 132041/225000 [265:58:15<173:16:58,  6.71s/it]
 59%|█████▊    | 132042/225000 [265:58:20<160:55:18,  6.23s/it]
 59%|█████▊    | 132043/225000 [265:58:23<140:30:36,  5.44s/it]
 59%|█████▊    | 132044/225000 [265:58:31<154:57:41,  6.00s/it]
 59%|█████▊    | 132045/225000 [265:58:34<136:09:11,  5.27s/it]
 59%|█████▊    | 132046/225000 [265:58:43<164:51:04,  6.38s/it]
 59%|█████▊    | 132047/225000 [265:58:47<142:47:40,  5.53s/it]
 59%|█████▊    | 132048/225000 [265:58:51<134:54:57,  5.23s/it]
 59%|█████▊    | 132049/225000 [265:58:56<132:37:13,  5.14s/it]
 59%|█████▊    | 132050/225000 [265:59:00<126:32:01,  4.90s/it]
                                                               
{'loss': '0.8085', 'grad_norm': '1.149', 'learning_rate': '1.975e-05', 'epoch': '0.5869', 'num_input_tokens_seen': 441866184, 'train_runtime': '9.575e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 132050/225000 [265:59:00<126:32:01,  4.90s/it]
 59%|█████▊    | 132051/225000 [265:59:05<123:04:44,  4.77s/it]
 59%|█████▊    | 132052/225000 [265:59:13<150:56:47,  5.85s/it]
 59%|█████▊    | 132053/225000 [265:59:25<199:22:28,  7.72s/it]
 59%|█████▊    | 132054/225000 [265:59:29<168:57:06,  6.54s/it]
 59%|█████▊    | 132055/225000 [265:59:33<149:15:32,  5.78s/it]
 59%|█████▊    | 132056/225000 [265:59:38<142:00:39,  5.50s/it]
 59%|█████▊    | 132057/225000 [265:59:43<140:01:34,  5.42s/it]
 59%|█████▊    | 132058/225000 [265:59:48<134:31:10,  5.21s/it]
 59%|█████▊    | 132059/225000 [265:59:53<134:15:52,  5.20s/it]
 59%|█████▊    | 132060/225000 [265:59:58<134:24:38,  5.21s/it]
 59%|█████▊    | 132061/225000 [266:00:03<131:50:40,  5.11s/it]
 59%|█████▊    | 132062/225000 [266:00:09<135:04:03,  5.23s/it]
 59%|█████▊    | 132063/225000 [266:00:14<137:04:51,  5.31s/it]
 59%|█████▊    | 132064/225000 [266:00:19<134:11:17,  5.20s/it]
 59%|█████▊    | 132065/225000 [266:00:25<139:24:56,  5.40s/it]
 59%|█████▊    | 132066/225000 [266:00:31<145:48:16,  5.65s/it]
 59%|█████▊    | 132067/225000 [266:00:37<148:27:11,  5.75s/it]
 59%|█████▊    | 132068/225000 [266:00:50<204:19:04,  7.91s/it]
 59%|█████▊    | 132069/225000 [266:00:55<183:42:34,  7.12s/it]
 59%|█████▊    | 132070/225000 [266:01:02<177:17:17,  6.87s/it]
 59%|█████▊    | 132071/225000 [266:01:07<163:18:11,  6.33s/it]
 59%|█████▊    | 132072/225000 [266:01:11<144:47:28,  5.61s/it]
 59%|█████▊    | 132073/225000 [266:01:15<131:21:00,  5.09s/it]
 59%|█████▊    | 132074/225000 [266:01:25<170:58:14,  6.62s/it]
 59%|█████▊    | 132075/225000 [266:01:31<168:52:12,  6.54s/it]
 59%|█████▊    | 132076/225000 [266:01:35<143:47:22,  5.57s/it]
 59%|█████▊    | 132077/225000 [266:01:44<171:10:35,  6.63s/it]
 59%|█████▊    | 132078/225000 [266:01:47<146:03:35,  5.66s/it]
 59%|█████▊    | 132079/225000 [266:01:50<129:00:25,  5.00s/it]
 59%|█████▊    | 132080/225000 [266:02:00<160:25:17,  6.22s/it]
 59%|█████▊    | 132081/225000 [266:02:03<141:23:15,  5.48s/it]
 59%|█████▊    | 132082/225000 [266:02:07<127:11:26,  4.93s/it]
 59%|█████▊    | 132083/225000 [266:02:11<123:54:57,  4.80s/it]
 59%|█████▊    | 132084/225000 [266:02:16<125:15:19,  4.85s/it]
 59%|█████▊    | 132085/225000 [266:02:21<126:31:19,  4.90s/it]
 59%|█████▊    | 132086/225000 [266:02:27<128:53:24,  4.99s/it]
 59%|█████▊    | 132087/225000 [266:02:31<127:05:43,  4.92s/it]
 59%|█████▊    | 132088/225000 [266:02:36<127:35:26,  4.94s/it]
 59%|█████▊    | 132089/225000 [266:02:49<189:05:45,  7.33s/it]
 59%|█████▊    | 132090/225000 [266:02:55<175:56:23,  6.82s/it]
 59%|█████▊    | 132091/225000 [266:03:03<183:19:25,  7.10s/it]
 59%|█████▊    | 132092/225000 [266:03:07<163:41:02,  6.34s/it]
 59%|█████▊    | 132093/225000 [266:03:12<150:39:53,  5.84s/it]
 59%|█████▊    | 132094/225000 [266:03:17<144:20:13,  5.59s/it]
 59%|█████▊    | 132095/225000 [266:03:21<134:43:48,  5.22s/it]
 59%|█████▊    | 132096/225000 [266:03:27<139:00:13,  5.39s/it]
 59%|█████▊    | 132097/225000 [266:03:38<178:52:35,  6.93s/it]
 59%|█████▊    | 132098/225000 [266:03:41<151:22:49,  5.87s/it]
 59%|█████▊    | 132099/225000 [266:03:49<164:23:18,  6.37s/it]
 59%|█████▊    | 132100/225000 [266:03:53<147:04:46,  5.70s/it]
                                                               
{'loss': '0.8012', 'grad_norm': '1.041', 'learning_rate': '1.973e-05', 'epoch': '0.5871', 'num_input_tokens_seen': 441994640, 'train_runtime': '9.578e+05', 'train_tokens_per_second': '461.5'}

 59%|█████▊    | 132100/225000 [266:03:53<147:04:46,  5.70s/it]
 59%|█████▊    | 132101/225000 [266:03:57<133:35:57,  5.18s/it]
 59%|█████▊    | 132102/225000 [266:04:01<130:38:59,  5.06s/it]
 59%|█████▊    | 132103/225000 [266:04:06<125:42:48,  4.87s/it]
 59%|█████▊    | 132104/225000 [266:04:10<122:53:35,  4.76s/it]
 59%|█████▊    | 132105/225000 [266:04:22<178:30:01,  6.92s/it]
 59%|█████▊    | 132106/225000 [266:04:29<179:33:47,  6.96s/it]
 59%|█████▊    | 132107/225000 [266:04:36<179:28:08,  6.96s/it]
 59%|█████▊    | 132108/225000 [266:04:41<159:01:56,  6.16s/it]
 59%|█████▊    | 132109/225000 [266:04:44<138:08:32,  5.35s/it]
 59%|█████▊    | 132110/225000 [266:04:53<168:08:38,  6.52s/it]
 59%|█████▊    | 132111/225000 [266:04:56<142:06:13,  5.51s/it]
 59%|█████▊    | 132112/225000 [266:05:05<162:27:59,  6.30s/it]
 59%|█████▊    | 132113/225000 [266:05:10<152:32:36,  5.91s/it]
 59%|█████▊    | 132114/225000 [266:05:16<160:10:01,  6.21s/it]
 59%|█████▊    | 132115/225000 [266:05:21<144:42:50,  5.61s/it]
 59%|█████▊    | 132116/225000 [266:05:25<133:36:45,  5.18s/it]
 59%|█████▊    | 132117/225000 [266:05:34<164:26:21,  6.37s/it]
 59%|█████▊    | 132118/225000 [266:05:40<158:18:34,  6.14s/it]
 59%|█████▊    | 132119/225000 [266:05:47<167:46:38,  6.50s/it]
 59%|█████▊    | 132120/225000 [266:05:50<143:53:26,  5.58s/it]
 59%|█████▊    | 132121/225000 [266:05:59<166:49:14,  6.47s/it]
 59%|█████▊    | 132122/225000 [266:06:04<153:46:49,  5.96s/it]
 59%|█████▊    | 132123/225000 [266:06:08<142:49:17,  5.54s/it]
 59%|█████▊    | 132124/225000 [266:06:13<133:04:32,  5.16s/it]
 59%|█████▊    | 132125/225000 [266:06:17<130:18:36,  5.05s/it]
 59%|█████▊    | 132126/225000 [266:06:22<127:05:00,  4.93s/it]
 59%|█████▊    | 132127/225000 [266:06:31<159:30:38,  6.18s/it]
 59%|█████▊    | 132128/225000 [266:06:38<168:53:50,  6.55s/it]
 59%|█████▊    | 132129/225000 [266:06:42<144:21:33,  5.60s/it]
 59%|█████▊    | 132130/225000 [266:06:50<161:48:59,  6.27s/it]
 59%|█████▊    | 132131/225000 [266:06:56<163:22:16,  6.33s/it]
 59%|█████▊    | 132132/225000 [266:07:01<152:00:21,  5.89s/it]
 59%|█████▊    | 132133/225000 [266:07:07<151:41:59,  5.88s/it]
 59%|█████▊    | 132134/225000 [266:07:12<143:46:20,  5.57s/it]
 59%|█████▊    | 132135/225000 [266:07:16<137:09:28,  5.32s/it]
 59%|█████▊    | 132136/225000 [266:07:21<130:59:19,  5.08s/it]
 59%|█████▊    | 132137/225000 [266:07:25<125:59:06,  4.88s/it]
 59%|█████▊    | 132138/225000 [266:07:30<124:52:26,  4.84s/it]
 59%|█████▊    | 132139/225000 [266:07:38<146:04:26,  5.66s/it]
 59%|█████▊    | 132140/225000 [266:07:48<184:04:42,  7.14s/it]
 59%|█████▊    | 132141/225000 [266:07:52<154:55:01,  6.01s/it]
 59%|█████▊    | 132142/225000 [266:07:56<138:20:44,  5.36s/it]
 59%|█████▊    | 132143/225000 [266:08:04<159:08:17,  6.17s/it]
 59%|█████▊    | 132144/225000 [266:08:11<170:13:03,  6.60s/it]
 59%|█████▊    | 132145/225000 [266:08:15<144:36:27,  5.61s/it]
 59%|█████▊    | 132146/225000 [266:08:18<131:44:39,  5.11s/it]
 59%|█████▊    | 132147/225000 [266:08:29<175:46:49,  6.82s/it]
 59%|█████▊    | 132148/225000 [266:08:33<155:51:25,  6.04s/it]
 59%|█████▊    | 132149/225000 [266:08:37<136:04:52,  5.28s/it]
 59%|█████▊    | 132150/225000 [266:08:47<173:20:45,  6.72s/it]
                                                               
{'loss': '0.8082', 'grad_norm': '1.187', 'learning_rate': '1.971e-05', 'epoch': '0.5873', 'num_input_tokens_seen': 442123576, 'train_runtime': '9.581e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▊    | 132150/225000 [266:08:47<173:20:45,  6.72s/it]
 59%|█████▊    | 132151/225000 [266:08:50<145:20:16,  5.64s/it]
 59%|█████▊    | 132152/225000 [266:08:55<137:34:56,  5.33s/it]
 59%|█████▊    | 132153/225000 [266:09:00<136:53:24,  5.31s/it]
 59%|█████▊    | 132154/225000 [266:09:05<133:21:51,  5.17s/it]
 59%|█████▊    | 132155/225000 [266:09:10<133:12:03,  5.16s/it]
 59%|█████▊    | 132156/225000 [266:09:15<128:13:54,  4.97s/it]
 59%|█████▊    | 132157/225000 [266:09:24<159:36:09,  6.19s/it]
 59%|█████▊    | 132158/225000 [266:09:32<179:06:13,  6.94s/it]
 59%|█████▊    | 132159/225000 [266:09:36<152:55:59,  5.93s/it]
 59%|█████▊    | 132160/225000 [266:09:41<144:51:57,  5.62s/it]
 59%|█████▊    | 132161/225000 [266:09:46<142:02:17,  5.51s/it]
 59%|█████▊    | 132162/225000 [266:09:51<140:44:05,  5.46s/it]
 59%|█████▊    | 132163/225000 [266:09:56<133:29:50,  5.18s/it]
 59%|█████▊    | 132164/225000 [266:10:01<135:01:47,  5.24s/it]
 59%|█████▊    | 132165/225000 [266:10:13<187:52:28,  7.29s/it]
 59%|█████▊    | 132166/225000 [266:10:19<177:40:32,  6.89s/it]
 59%|█████▊    | 132167/225000 [266:10:23<153:21:13,  5.95s/it]
 59%|█████▊    | 132168/225000 [266:10:29<152:04:02,  5.90s/it]
 59%|█████▊    | 132169/225000 [266:10:36<163:36:03,  6.34s/it]
 59%|█████▊    | 132170/225000 [266:10:40<145:32:08,  5.64s/it]
 59%|█████▊    | 132171/225000 [266:10:44<132:31:46,  5.14s/it]
 59%|█████▊    | 132172/225000 [266:10:57<190:18:03,  7.38s/it]
 59%|█████▊    | 132173/225000 [266:11:01<167:00:49,  6.48s/it]
 59%|█████▊    | 132174/225000 [266:11:05<150:01:31,  5.82s/it]
 59%|█████▊    | 132175/225000 [266:11:10<136:35:41,  5.30s/it]
 59%|█████▊    | 132176/225000 [266:11:14<129:49:27,  5.03s/it]
 59%|█████▊    | 132177/225000 [266:11:20<137:50:30,  5.35s/it]
 59%|█████▊    | 132178/225000 [266:11:30<174:20:13,  6.76s/it]
 59%|█████▊    | 132179/225000 [266:11:34<154:17:28,  5.98s/it]
 59%|█████▊    | 132180/225000 [266:11:39<143:29:00,  5.56s/it]
 59%|█████▊    | 132181/225000 [266:11:43<136:09:50,  5.28s/it]
 59%|█████▊    | 132182/225000 [266:11:48<130:10:28,  5.05s/it]
 59%|█████▊    | 132183/225000 [266:11:57<158:04:37,  6.13s/it]
 59%|█████▊    | 132184/225000 [266:12:05<176:07:08,  6.83s/it]
 59%|█████▊    | 132185/225000 [266:12:12<175:40:36,  6.81s/it]
 59%|█████▊    | 132186/225000 [266:12:16<154:40:10,  6.00s/it]
 59%|█████▊    | 132187/225000 [266:12:20<138:09:39,  5.36s/it]
 59%|█████▉    | 132188/225000 [266:12:24<132:47:23,  5.15s/it]
 59%|█████▉    | 132189/225000 [266:12:34<165:20:03,  6.41s/it]
 59%|█████▉    | 132190/225000 [266:12:39<157:09:44,  6.10s/it]
 59%|█████▉    | 132191/225000 [266:12:47<166:45:03,  6.47s/it]
 59%|█████▉    | 132192/225000 [266:12:52<155:27:36,  6.03s/it]
 59%|█████▉    | 132193/225000 [266:12:57<148:08:04,  5.75s/it]
 59%|█████▉    | 132194/225000 [266:13:02<143:47:15,  5.58s/it]
 59%|█████▉    | 132195/225000 [266:13:06<136:12:04,  5.28s/it]
 59%|█████▉    | 132196/225000 [266:13:11<132:12:28,  5.13s/it]
 59%|█████▉    | 132197/225000 [266:13:16<131:52:49,  5.12s/it]
 59%|█████▉    | 132198/225000 [266:13:21<129:30:40,  5.02s/it]
 59%|█████▉    | 132199/225000 [266:13:29<149:45:02,  5.81s/it]
 59%|█████▉    | 132200/225000 [266:13:41<200:04:43,  7.76s/it]
                                                               
{'loss': '0.8138', 'grad_norm': '1.109', 'learning_rate': '1.969e-05', 'epoch': '0.5876', 'num_input_tokens_seen': 442253496, 'train_runtime': '9.584e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132200/225000 [266:13:41<200:04:43,  7.76s/it]
 59%|█████▉    | 132201/225000 [266:13:45<168:48:46,  6.55s/it]
 59%|█████▉    | 132202/225000 [266:13:49<149:53:55,  5.82s/it]
 59%|█████▉    | 132203/225000 [266:13:53<139:53:20,  5.43s/it]
 59%|█████▉    | 132204/225000 [266:14:00<152:31:52,  5.92s/it]
 59%|█████▉    | 132205/225000 [266:14:10<181:02:07,  7.02s/it]
 59%|█████▉    | 132206/225000 [266:14:14<157:34:33,  6.11s/it]
 59%|█████▉    | 132207/225000 [266:14:18<140:21:52,  5.45s/it]
 59%|█████▉    | 132208/225000 [266:14:29<182:07:31,  7.07s/it]
 59%|█████▉    | 132209/225000 [266:14:33<156:44:31,  6.08s/it]
 59%|█████▉    | 132210/225000 [266:14:36<139:25:25,  5.41s/it]
 59%|█████▉    | 132211/225000 [266:14:46<168:57:06,  6.55s/it]
 59%|█████▉    | 132212/225000 [266:14:49<146:47:10,  5.70s/it]
 59%|█████▉    | 132213/225000 [266:14:55<146:06:14,  5.67s/it]
 59%|█████▉    | 132214/225000 [266:15:02<153:22:20,  5.95s/it]
 59%|█████▉    | 132215/225000 [266:15:06<139:01:38,  5.39s/it]
 59%|█████▉    | 132216/225000 [266:15:12<145:16:46,  5.64s/it]
 59%|█████▉    | 132217/225000 [266:15:21<173:10:32,  6.72s/it]
 59%|█████▉    | 132218/225000 [266:15:26<156:29:10,  6.07s/it]
 59%|█████▉    | 132219/225000 [266:15:30<142:56:12,  5.55s/it]
 59%|█████▉    | 132220/225000 [266:15:35<135:49:25,  5.27s/it]
 59%|█████▉    | 132221/225000 [266:15:39<128:04:18,  4.97s/it]
 59%|█████▉    | 132222/225000 [266:15:49<165:00:02,  6.40s/it]
 59%|█████▉    | 132223/225000 [266:15:56<170:31:54,  6.62s/it]
 59%|█████▉    | 132224/225000 [266:15:59<147:55:43,  5.74s/it]
 59%|█████▉    | 132225/225000 [266:16:09<176:22:07,  6.84s/it]
 59%|█████▉    | 132226/225000 [266:16:12<151:35:48,  5.88s/it]
 59%|█████▉    | 132227/225000 [266:16:21<173:29:58,  6.73s/it]
 59%|█████▉    | 132228/225000 [266:16:24<146:18:46,  5.68s/it]
 59%|█████▉    | 132229/225000 [266:16:32<162:59:49,  6.33s/it]
 59%|█████▉    | 132230/225000 [266:16:36<140:40:47,  5.46s/it]
 59%|█████▉    | 132231/225000 [266:16:44<166:01:32,  6.44s/it]
 59%|█████▉    | 132232/225000 [266:16:49<149:02:58,  5.78s/it]
 59%|█████▉    | 132233/225000 [266:16:53<140:01:12,  5.43s/it]
 59%|█████▉    | 132234/225000 [266:16:58<132:16:53,  5.13s/it]
 59%|█████▉    | 132235/225000 [266:17:02<127:11:32,  4.94s/it]
 59%|█████▉    | 132236/225000 [266:17:07<126:23:46,  4.91s/it]
 59%|█████▉    | 132237/225000 [266:17:17<169:08:23,  6.56s/it]
 59%|█████▉    | 132238/225000 [266:17:22<149:52:08,  5.82s/it]
 59%|█████▉    | 132239/225000 [266:17:26<139:58:11,  5.43s/it]
 59%|█████▉    | 132240/225000 [266:17:36<176:12:42,  6.84s/it]
 59%|█████▉    | 132241/225000 [266:17:41<157:46:56,  6.12s/it]
 59%|█████▉    | 132242/225000 [266:17:45<145:06:10,  5.63s/it]
 59%|█████▉    | 132243/225000 [266:17:50<135:39:59,  5.27s/it]
 59%|█████▉    | 132244/225000 [266:17:58<162:55:32,  6.32s/it]
 59%|█████▉    | 132245/225000 [266:18:05<165:43:38,  6.43s/it]
 59%|█████▉    | 132246/225000 [266:18:08<142:07:56,  5.52s/it]
 59%|█████▉    | 132247/225000 [266:18:18<172:53:13,  6.71s/it]
 59%|█████▉    | 132248/225000 [266:18:22<150:55:12,  5.86s/it]
 59%|█████▉    | 132249/225000 [266:18:25<134:22:31,  5.22s/it]
 59%|█████▉    | 132250/225000 [266:18:36<175:08:59,  6.80s/it]
                                                               
{'loss': '0.8102', 'grad_norm': '1.168', 'learning_rate': '1.967e-05', 'epoch': '0.5878', 'num_input_tokens_seen': 442382312, 'train_runtime': '9.587e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132250/225000 [266:18:36<175:08:59,  6.80s/it]
 59%|█████▉    | 132251/225000 [266:18:39<146:47:42,  5.70s/it]
 59%|█████▉    | 132252/225000 [266:18:47<164:48:10,  6.40s/it]
 59%|█████▉    | 132253/225000 [266:18:50<140:17:22,  5.45s/it]
 59%|█████▉    | 132254/225000 [266:18:58<158:23:28,  6.15s/it]
 59%|█████▉    | 132255/225000 [266:19:03<147:24:17,  5.72s/it]
 59%|█████▉    | 132256/225000 [266:19:10<155:37:05,  6.04s/it]
 59%|█████▉    | 132257/225000 [266:19:13<133:26:53,  5.18s/it]
 59%|█████▉    | 132258/225000 [266:19:17<127:47:54,  4.96s/it]
 59%|█████▉    | 132259/225000 [266:19:22<125:40:25,  4.88s/it]
 59%|█████▉    | 132260/225000 [266:19:26<121:16:14,  4.71s/it]
 59%|█████▉    | 132261/225000 [266:19:31<120:56:11,  4.69s/it]
 59%|█████▉    | 132262/225000 [266:19:37<129:27:11,  5.03s/it]
 59%|█████▉    | 132263/225000 [266:19:48<181:45:24,  7.06s/it]
 59%|█████▉    | 132264/225000 [266:19:55<174:09:05,  6.76s/it]
 59%|█████▉    | 132265/225000 [266:19:59<157:16:01,  6.11s/it]
 59%|█████▉    | 132266/225000 [266:20:07<172:56:33,  6.71s/it]
 59%|█████▉    | 132267/225000 [266:20:11<152:12:53,  5.91s/it]
 59%|█████▉    | 132268/225000 [266:20:15<136:29:24,  5.30s/it]
 59%|█████▉    | 132269/225000 [266:20:27<187:13:12,  7.27s/it]
 59%|█████▉    | 132270/225000 [266:20:30<156:25:27,  6.07s/it]
 59%|█████▉    | 132271/225000 [266:20:37<158:38:59,  6.16s/it]
 59%|█████▉    | 132272/225000 [266:20:41<145:57:51,  5.67s/it]
 59%|█████▉    | 132273/225000 [266:20:51<179:36:16,  6.97s/it]
 59%|█████▉    | 132274/225000 [266:20:57<171:41:37,  6.67s/it]
 59%|█████▉    | 132275/225000 [266:21:01<146:22:02,  5.68s/it]
 59%|█████▉    | 132276/225000 [266:21:08<162:20:38,  6.30s/it]
 59%|█████▉    | 132277/225000 [266:21:11<136:36:21,  5.30s/it]
 59%|█████▉    | 132278/225000 [266:21:16<132:49:27,  5.16s/it]
 59%|█████▉    | 132279/225000 [266:21:21<130:22:49,  5.06s/it]
 59%|█████▉    | 132280/225000 [266:21:26<126:47:43,  4.92s/it]
 59%|█████▉    | 132281/225000 [266:21:30<126:25:36,  4.91s/it]
 59%|█████▉    | 132282/225000 [266:21:35<125:42:16,  4.88s/it]
 59%|█████▉    | 132283/225000 [266:21:40<126:05:22,  4.90s/it]
 59%|█████▉    | 132284/225000 [266:21:52<176:54:11,  6.87s/it]
 59%|█████▉    | 132285/225000 [266:21:58<174:07:15,  6.76s/it]
 59%|█████▉    | 132286/225000 [266:22:05<174:40:46,  6.78s/it]
 59%|█████▉    | 132287/225000 [266:22:10<158:26:44,  6.15s/it]
 59%|█████▉    | 132288/225000 [266:22:14<145:02:55,  5.63s/it]
 59%|█████▉    | 132289/225000 [266:22:19<138:17:03,  5.37s/it]
 59%|█████▉    | 132290/225000 [266:22:24<133:53:22,  5.20s/it]
 59%|█████▉    | 132291/225000 [266:22:28<129:19:09,  5.02s/it]
 59%|█████▉    | 132292/225000 [266:22:33<126:59:59,  4.93s/it]
 59%|█████▉    | 132293/225000 [266:22:38<125:10:49,  4.86s/it]
 59%|█████▉    | 132294/225000 [266:22:43<125:31:45,  4.87s/it]
 59%|█████▉    | 132295/225000 [266:22:53<165:55:12,  6.44s/it]
 59%|█████▉    | 132296/225000 [266:23:03<196:17:40,  7.62s/it]
 59%|█████▉    | 132297/225000 [266:23:06<163:20:11,  6.34s/it]
 59%|█████▉    | 132298/225000 [266:23:14<176:25:31,  6.85s/it]
 59%|█████▉    | 132299/225000 [266:23:19<161:30:34,  6.27s/it]
 59%|█████▉    | 132300/225000 [266:23:24<150:56:01,  5.86s/it]
                                                               
{'loss': '0.8122', 'grad_norm': '1.13', 'learning_rate': '1.966e-05', 'epoch': '0.588', 'num_input_tokens_seen': 442511888, 'train_runtime': '9.59e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132300/225000 [266:23:24<150:56:01,  5.86s/it]
 59%|█████▉    | 132301/225000 [266:23:29<138:48:55,  5.39s/it]
 59%|█████▉    | 132302/225000 [266:23:34<135:30:11,  5.26s/it]
 59%|█████▉    | 132303/225000 [266:23:39<134:39:05,  5.23s/it]
 59%|█████▉    | 132304/225000 [266:23:46<147:29:59,  5.73s/it]
 59%|█████▉    | 132305/225000 [266:23:58<200:40:52,  7.79s/it]
 59%|█████▉    | 132306/225000 [266:24:02<166:17:40,  6.46s/it]
 59%|█████▉    | 132307/225000 [266:24:06<152:54:36,  5.94s/it]
 59%|█████▉    | 132308/225000 [266:24:11<144:18:08,  5.60s/it]
 59%|█████▉    | 132309/225000 [266:24:15<133:45:25,  5.19s/it]
 59%|█████▉    | 132310/225000 [266:24:20<130:50:30,  5.08s/it]
 59%|█████▉    | 132311/225000 [266:24:31<172:15:43,  6.69s/it]
 59%|█████▉    | 132312/225000 [266:24:39<187:34:14,  7.29s/it]
 59%|█████▉    | 132313/225000 [266:24:46<180:22:20,  7.01s/it]
 59%|█████▉    | 132314/225000 [266:24:49<153:36:55,  5.97s/it]
 59%|█████▉    | 132315/225000 [266:24:53<141:08:24,  5.48s/it]
 59%|█████▉    | 132316/225000 [266:25:02<162:49:33,  6.32s/it]
 59%|█████▉    | 132317/225000 [266:25:06<143:07:12,  5.56s/it]
 59%|█████▉    | 132318/225000 [266:25:15<174:51:22,  6.79s/it]
 59%|█████▉    | 132319/225000 [266:25:18<147:33:41,  5.73s/it]
 59%|█████▉    | 132320/225000 [266:25:24<145:02:30,  5.63s/it]
 59%|█████▉    | 132321/225000 [266:25:30<148:45:33,  5.78s/it]
 59%|█████▉    | 132322/225000 [266:25:37<160:04:25,  6.22s/it]
 59%|█████▉    | 132323/225000 [266:25:41<142:30:20,  5.54s/it]
 59%|█████▉    | 132324/225000 [266:25:46<136:43:29,  5.31s/it]
 59%|█████▉    | 132325/225000 [266:25:56<173:02:06,  6.72s/it]
 59%|█████▉    | 132326/225000 [266:26:00<149:10:29,  5.79s/it]
 59%|█████▉    | 132327/225000 [266:26:04<137:14:49,  5.33s/it]
 59%|█████▉    | 132328/225000 [266:26:14<172:54:48,  6.72s/it]
 59%|█████▉    | 132329/225000 [266:26:17<148:50:28,  5.78s/it]
 59%|█████▉    | 132330/225000 [266:26:24<154:24:15,  6.00s/it]
 59%|█████▉    | 132331/225000 [266:26:29<143:46:13,  5.59s/it]
 59%|█████▉    | 132332/225000 [266:26:37<165:12:50,  6.42s/it]
 59%|█████▉    | 132333/225000 [266:26:40<140:50:02,  5.47s/it]
 59%|█████▉    | 132334/225000 [266:26:48<158:47:56,  6.17s/it]
 59%|█████▉    | 132335/225000 [266:26:51<136:38:02,  5.31s/it]
 59%|█████▉    | 132336/225000 [266:26:59<156:57:13,  6.10s/it]
 59%|█████▉    | 132337/225000 [266:27:02<134:40:06,  5.23s/it]
 59%|█████▉    | 132338/225000 [266:27:11<160:19:52,  6.23s/it]
 59%|█████▉    | 132339/225000 [266:27:16<152:03:09,  5.91s/it]
 59%|█████▉    | 132340/225000 [266:27:20<135:33:26,  5.27s/it]
 59%|█████▉    | 132341/225000 [266:27:24<125:24:11,  4.87s/it]
 59%|█████▉    | 132342/225000 [266:27:34<163:14:04,  6.34s/it]
 59%|█████▉    | 132343/225000 [266:27:39<153:21:22,  5.96s/it]
 59%|█████▉    | 132344/225000 [266:27:43<141:49:07,  5.51s/it]
 59%|█████▉    | 132345/225000 [266:27:48<137:36:39,  5.35s/it]
 59%|█████▉    | 132346/225000 [266:27:53<135:57:10,  5.28s/it]
 59%|█████▉    | 132347/225000 [266:27:58<131:51:48,  5.12s/it]
 59%|█████▉    | 132348/225000 [266:28:04<136:04:15,  5.29s/it]
 59%|█████▉    | 132349/225000 [266:28:15<182:13:03,  7.08s/it]
 59%|█████▉    | 132350/225000 [266:28:18<154:38:29,  6.01s/it]
                                                               
{'loss': '0.8129', 'grad_norm': '1.084', 'learning_rate': '1.964e-05', 'epoch': '0.5882', 'num_input_tokens_seen': 442642216, 'train_runtime': '9.593e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132350/225000 [266:28:18<154:38:29,  6.01s/it]
 59%|█████▉    | 132351/225000 [266:28:26<168:30:54,  6.55s/it]
 59%|█████▉    | 132352/225000 [266:28:31<156:07:21,  6.07s/it]
 59%|█████▉    | 132353/225000 [266:28:38<164:47:58,  6.40s/it]
 59%|█████▉    | 132354/225000 [266:28:42<141:35:08,  5.50s/it]
 59%|█████▉    | 132355/225000 [266:28:46<130:10:14,  5.06s/it]
 59%|█████▉    | 132356/225000 [266:28:50<127:07:50,  4.94s/it]
 59%|█████▉    | 132357/225000 [266:29:01<173:12:52,  6.73s/it]
 59%|█████▉    | 132358/225000 [266:29:06<159:32:07,  6.20s/it]
 59%|█████▉    | 132359/225000 [266:29:10<138:44:14,  5.39s/it]
 59%|█████▉    | 132360/225000 [266:29:20<172:00:24,  6.68s/it]
 59%|█████▉    | 132361/225000 [266:29:24<151:45:25,  5.90s/it]
 59%|█████▉    | 132362/225000 [266:29:28<138:16:03,  5.37s/it]
 59%|█████▉    | 132363/225000 [266:29:36<160:40:34,  6.24s/it]
 59%|█████▉    | 132364/225000 [266:29:43<162:45:03,  6.32s/it]
 59%|█████▉    | 132365/225000 [266:29:49<166:47:48,  6.48s/it]
 59%|█████▉    | 132366/225000 [266:29:53<146:38:27,  5.70s/it]
 59%|█████▉    | 132367/225000 [266:29:57<133:54:26,  5.20s/it]
 59%|█████▉    | 132368/225000 [266:30:02<126:16:30,  4.91s/it]
 59%|█████▉    | 132369/225000 [266:30:06<120:08:08,  4.67s/it]
 59%|█████▉    | 132370/225000 [266:30:17<169:43:35,  6.60s/it]
 59%|█████▉    | 132371/225000 [266:30:20<145:58:09,  5.67s/it]
 59%|█████▉    | 132372/225000 [266:30:28<161:38:29,  6.28s/it]
 59%|█████▉    | 132373/225000 [266:30:32<147:27:35,  5.73s/it]
 59%|█████▉    | 132374/225000 [266:30:39<153:50:58,  5.98s/it]
 59%|█████▉    | 132375/225000 [266:30:44<144:17:52,  5.61s/it]
 59%|█████▉    | 132376/225000 [266:30:51<158:42:55,  6.17s/it]
 59%|█████▉    | 132377/225000 [266:30:55<141:10:10,  5.49s/it]
 59%|█████▉    | 132378/225000 [266:31:05<173:42:33,  6.75s/it]
 59%|█████▉    | 132379/225000 [266:31:08<146:29:15,  5.69s/it]
 59%|█████▉    | 132380/225000 [266:31:15<153:41:09,  5.97s/it]
 59%|█████▉    | 132381/225000 [266:31:20<147:47:48,  5.74s/it]
 59%|█████▉    | 132382/225000 [266:31:25<140:03:09,  5.44s/it]
 59%|█████▉    | 132383/225000 [266:31:29<134:21:14,  5.22s/it]
 59%|█████▉    | 132384/225000 [266:31:34<129:16:36,  5.03s/it]
 59%|█████▉    | 132385/225000 [266:31:39<126:36:26,  4.92s/it]
 59%|█████▉    | 132386/225000 [266:31:43<124:21:02,  4.83s/it]
 59%|█████▉    | 132387/225000 [266:31:54<170:04:24,  6.61s/it]
 59%|█████▉    | 132388/225000 [266:32:01<175:52:39,  6.84s/it]
 59%|█████▉    | 132389/225000 [266:32:05<149:17:55,  5.80s/it]
 59%|█████▉    | 132390/225000 [266:32:14<175:34:15,  6.82s/it]
 59%|█████▉    | 132391/225000 [266:32:21<174:13:00,  6.77s/it]
 59%|█████▉    | 132392/225000 [266:32:25<153:51:51,  5.98s/it]
 59%|█████▉    | 132393/225000 [266:32:29<138:46:43,  5.39s/it]
 59%|█████▉    | 132394/225000 [266:32:34<141:35:59,  5.50s/it]
 59%|█████▉    | 132395/225000 [266:32:43<167:27:32,  6.51s/it]
 59%|█████▉    | 132396/225000 [266:32:47<146:28:41,  5.69s/it]
 59%|█████▉    | 132397/225000 [266:32:51<136:08:08,  5.29s/it]
 59%|█████▉    | 132398/225000 [266:32:57<137:28:53,  5.34s/it]
 59%|█████▉    | 132399/225000 [266:33:02<137:45:32,  5.36s/it]
 59%|█████▉    | 132400/225000 [266:33:08<138:04:02,  5.37s/it]
                                                               
{'loss': '0.8147', 'grad_norm': '1.08', 'learning_rate': '1.962e-05', 'epoch': '0.5884', 'num_input_tokens_seen': 442771592, 'train_runtime': '9.596e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132400/225000 [266:33:08<138:04:02,  5.37s/it]
 59%|█████▉    | 132401/225000 [266:33:13<135:58:45,  5.29s/it]
 59%|█████▉    | 132402/225000 [266:33:18<137:25:41,  5.34s/it]
 59%|█████▉    | 132403/225000 [266:33:23<135:58:47,  5.29s/it]
 59%|█████▉    | 132404/225000 [266:33:29<135:09:22,  5.25s/it]
 59%|█████▉    | 132405/225000 [266:33:34<133:51:48,  5.20s/it]
 59%|█████▉    | 132406/225000 [266:33:40<141:42:30,  5.51s/it]
 59%|█████▉    | 132407/225000 [266:33:52<195:04:42,  7.58s/it]
 59%|█████▉    | 132408/225000 [266:33:59<186:52:08,  7.27s/it]
 59%|█████▉    | 132409/225000 [266:34:02<156:17:25,  6.08s/it]
 59%|█████▉    | 132410/225000 [266:34:10<168:14:27,  6.54s/it]
 59%|█████▉    | 132411/225000 [266:34:13<143:58:26,  5.60s/it]
 59%|█████▉    | 132412/225000 [266:34:21<164:06:40,  6.38s/it]
 59%|█████▉    | 132413/225000 [266:34:25<144:04:04,  5.60s/it]
 59%|█████▉    | 132414/225000 [266:34:34<171:53:12,  6.68s/it]
 59%|█████▉    | 132415/225000 [266:34:38<148:11:46,  5.76s/it]
 59%|█████▉    | 132416/225000 [266:34:45<156:43:34,  6.09s/it]
 59%|█████▉    | 132417/225000 [266:34:49<139:11:37,  5.41s/it]
 59%|█████▉    | 132418/225000 [266:34:53<126:50:36,  4.93s/it]
 59%|█████▉    | 132419/225000 [266:34:58<133:03:38,  5.17s/it]
 59%|█████▉    | 132420/225000 [266:35:08<166:59:53,  6.49s/it]
 59%|█████▉    | 132421/225000 [266:35:12<145:51:19,  5.67s/it]
 59%|█████▉    | 132422/225000 [266:35:18<148:22:46,  5.77s/it]
 59%|█████▉    | 132423/225000 [266:35:24<156:48:39,  6.10s/it]
 59%|█████▉    | 132424/225000 [266:35:28<139:24:29,  5.42s/it]
 59%|█████▉    | 132425/225000 [266:35:35<145:50:07,  5.67s/it]
 59%|█████▉    | 132426/225000 [266:35:43<167:47:27,  6.53s/it]
 59%|█████▉    | 132427/225000 [266:35:51<176:26:13,  6.86s/it]
 59%|█████▉    | 132428/225000 [266:35:54<149:16:56,  5.81s/it]
 59%|█████▉    | 132429/225000 [266:36:02<164:37:01,  6.40s/it]
 59%|█████▉    | 132430/225000 [266:36:05<143:01:18,  5.56s/it]
 59%|█████▉    | 132431/225000 [266:36:09<129:30:19,  5.04s/it]
 59%|█████▉    | 132432/225000 [266:36:18<158:15:59,  6.16s/it]
 59%|█████▉    | 132433/225000 [266:36:23<151:22:59,  5.89s/it]
 59%|█████▉    | 132434/225000 [266:36:27<137:43:26,  5.36s/it]
 59%|█████▉    | 132435/225000 [266:36:33<138:20:38,  5.38s/it]
 59%|█████▉    | 132436/225000 [266:36:42<166:20:29,  6.47s/it]
 59%|█████▉    | 132437/225000 [266:36:46<150:39:03,  5.86s/it]
 59%|█████▉    | 132438/225000 [266:36:51<143:22:36,  5.58s/it]
 59%|█████▉    | 132439/225000 [266:36:56<138:30:45,  5.39s/it]
 59%|█████▉    | 132440/225000 [266:37:01<134:13:35,  5.22s/it]
 59%|█████▉    | 132441/225000 [266:37:06<132:42:18,  5.16s/it]
 59%|█████▉    | 132442/225000 [266:37:11<128:09:08,  4.98s/it]
 59%|█████▉    | 132443/225000 [266:37:20<161:51:06,  6.30s/it]
 59%|█████▉    | 132444/225000 [266:37:29<181:59:29,  7.08s/it]
 59%|█████▉    | 132445/225000 [266:37:32<154:22:43,  6.00s/it]
 59%|█████▉    | 132446/225000 [266:37:40<169:34:55,  6.60s/it]
 59%|█████▉    | 132447/225000 [266:37:44<144:39:50,  5.63s/it]
 59%|█████▉    | 132448/225000 [266:37:53<172:21:50,  6.70s/it]
 59%|█████▉    | 132449/225000 [266:37:57<154:01:29,  5.99s/it]
 59%|█████▉    | 132450/225000 [266:38:05<164:38:58,  6.40s/it]
                                                               
{'loss': '0.8181', 'grad_norm': '1.008', 'learning_rate': '1.96e-05', 'epoch': '0.5887', 'num_input_tokens_seen': 442901520, 'train_runtime': '9.599e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132450/225000 [266:38:05<164:38:58,  6.40s/it]
 59%|█████▉    | 132451/225000 [266:38:08<140:32:19,  5.47s/it]
 59%|█████▉    | 132452/225000 [266:38:14<148:08:05,  5.76s/it]
 59%|█████▉    | 132453/225000 [266:38:20<144:00:13,  5.60s/it]
 59%|█████▉    | 132454/225000 [266:38:24<131:37:58,  5.12s/it]
 59%|█████▉    | 132455/225000 [266:38:29<130:32:34,  5.08s/it]
 59%|█████▉    | 132456/225000 [266:38:34<132:58:37,  5.17s/it]
 59%|█████▉    | 132457/225000 [266:38:43<163:42:19,  6.37s/it]
 59%|█████▉    | 132458/225000 [266:38:49<157:21:31,  6.12s/it]
 59%|█████▉    | 132459/225000 [266:38:52<137:19:10,  5.34s/it]
 59%|█████▉    | 132460/225000 [266:38:56<124:45:19,  4.85s/it]
 59%|█████▉    | 132461/225000 [266:39:07<172:01:37,  6.69s/it]
 59%|█████▉    | 132462/225000 [266:39:10<146:03:24,  5.68s/it]
 59%|█████▉    | 132463/225000 [266:39:14<132:31:53,  5.16s/it]
 59%|█████▉    | 132464/225000 [266:39:19<132:21:33,  5.15s/it]
 59%|█████▉    | 132465/225000 [266:39:30<177:19:27,  6.90s/it]
 59%|█████▉    | 132466/225000 [266:39:34<150:13:40,  5.84s/it]
 59%|█████▉    | 132467/225000 [266:39:38<141:57:33,  5.52s/it]
 59%|█████▉    | 132468/225000 [266:39:43<136:46:22,  5.32s/it]
 59%|█████▉    | 132469/225000 [266:39:48<129:45:55,  5.05s/it]
 59%|█████▉    | 132470/225000 [266:39:52<126:46:51,  4.93s/it]
 59%|█████▉    | 132471/225000 [266:40:00<147:59:40,  5.76s/it]
 59%|█████▉    | 132472/225000 [266:40:11<192:23:49,  7.49s/it]
 59%|█████▉    | 132473/225000 [266:40:15<158:50:48,  6.18s/it]
 59%|█████▉    | 132474/225000 [266:40:21<163:25:19,  6.36s/it]
 59%|█████▉    | 132475/225000 [266:40:27<159:27:50,  6.20s/it]
 59%|█████▉    | 132476/225000 [266:40:34<163:48:07,  6.37s/it]
 59%|█████▉    | 132477/225000 [266:40:39<155:31:42,  6.05s/it]
 59%|█████▉    | 132478/225000 [266:40:45<151:02:22,  5.88s/it]
 59%|█████▉    | 132479/225000 [266:40:49<138:27:38,  5.39s/it]
 59%|█████▉    | 132480/225000 [266:40:56<149:42:51,  5.83s/it]
 59%|█████▉    | 132481/225000 [266:41:04<164:00:27,  6.38s/it]
 59%|█████▉    | 132482/225000 [266:41:07<141:33:33,  5.51s/it]
 59%|█████▉    | 132483/225000 [266:41:14<155:12:12,  6.04s/it]
 59%|█████▉    | 132484/225000 [266:41:18<135:48:25,  5.28s/it]
 59%|█████▉    | 132485/225000 [266:41:23<131:17:13,  5.11s/it]
 59%|█████▉    | 132486/225000 [266:41:27<127:01:21,  4.94s/it]
 59%|█████▉    | 132487/225000 [266:41:32<125:09:47,  4.87s/it]
 59%|█████▉    | 132488/225000 [266:41:40<154:21:25,  6.01s/it]
 59%|█████▉    | 132489/225000 [266:41:51<186:01:23,  7.24s/it]
 59%|█████▉    | 132490/225000 [266:41:54<156:19:11,  6.08s/it]
 59%|█████▉    | 132491/225000 [266:42:02<170:08:22,  6.62s/it]
 59%|█████▉    | 132492/225000 [266:42:07<157:29:04,  6.13s/it]
 59%|█████▉    | 132493/225000 [266:42:12<149:13:09,  5.81s/it]
 59%|█████▉    | 132494/225000 [266:42:17<145:14:28,  5.65s/it]
 59%|█████▉    | 132495/225000 [266:42:22<142:09:32,  5.53s/it]
 59%|█████▉    | 132496/225000 [266:42:27<138:02:59,  5.37s/it]
 59%|█████▉    | 132497/225000 [266:42:33<139:23:35,  5.42s/it]
 59%|█████▉    | 132498/225000 [266:42:39<140:36:06,  5.47s/it]
 59%|█████▉    | 132499/225000 [266:42:44<139:15:59,  5.42s/it]
 59%|█████▉    | 132500/225000 [266:42:49<140:34:36,  5.47s/it]
                                                               
{'loss': '0.81', 'grad_norm': '1.119', 'learning_rate': '1.959e-05', 'epoch': '0.5889', 'num_input_tokens_seen': 443030248, 'train_runtime': '9.602e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132500/225000 [266:42:49<140:34:36,  5.47s/it]
 59%|█████▉    | 132501/225000 [266:42:55<139:23:06,  5.42s/it]
 59%|█████▉    | 132502/225000 [266:43:00<139:49:44,  5.44s/it]
 59%|█████▉    | 132503/225000 [266:43:06<140:26:45,  5.47s/it]
 59%|█████▉    | 132504/225000 [266:43:11<136:50:30,  5.33s/it]
 59%|█████▉    | 132505/225000 [266:43:16<135:50:24,  5.29s/it]
 59%|█████▉    | 132506/225000 [266:43:21<133:50:52,  5.21s/it]
 59%|█████▉    | 132507/225000 [266:43:27<139:48:11,  5.44s/it]
 59%|█████▉    | 132508/225000 [266:43:39<189:08:39,  7.36s/it]
 59%|█████▉    | 132509/225000 [266:43:44<173:46:52,  6.76s/it]
 59%|█████▉    | 132510/225000 [266:43:49<156:47:38,  6.10s/it]
 59%|█████▉    | 132511/225000 [266:43:53<146:02:54,  5.68s/it]
 59%|█████▉    | 132512/225000 [266:43:59<142:02:42,  5.53s/it]
 59%|█████▉    | 132513/225000 [266:44:04<137:25:20,  5.35s/it]
 59%|█████▉    | 132514/225000 [266:44:09<137:29:10,  5.35s/it]
 59%|█████▉    | 132515/225000 [266:44:14<133:43:37,  5.21s/it]
 59%|█████▉    | 132516/225000 [266:44:19<133:07:16,  5.18s/it]
 59%|█████▉    | 132517/225000 [266:44:31<189:20:57,  7.37s/it]
 59%|█████▉    | 132518/225000 [266:44:36<165:41:00,  6.45s/it]
 59%|█████▉    | 132519/225000 [266:44:40<148:30:11,  5.78s/it]
 59%|█████▉    | 132520/225000 [266:44:45<144:30:52,  5.63s/it]
 59%|█████▉    | 132521/225000 [266:44:50<141:31:53,  5.51s/it]
 59%|█████▉    | 132522/225000 [266:44:56<141:47:13,  5.52s/it]
 59%|█████▉    | 132523/225000 [266:45:01<139:55:19,  5.45s/it]
 59%|█████▉    | 132524/225000 [266:45:06<135:42:37,  5.28s/it]
 59%|█████▉    | 132525/225000 [266:45:12<136:52:21,  5.33s/it]
 59%|█████▉    | 132526/225000 [266:45:16<134:14:37,  5.23s/it]
 59%|█████▉    | 132527/225000 [266:45:21<132:19:08,  5.15s/it]
 59%|█████▉    | 132528/225000 [266:45:33<184:10:22,  7.17s/it]
 59%|█████▉    | 132529/225000 [266:45:42<194:56:01,  7.59s/it]
 59%|█████▉    | 132530/225000 [266:45:45<161:17:48,  6.28s/it]
 59%|█████▉    | 132531/225000 [266:45:53<172:31:22,  6.72s/it]
 59%|█████▉    | 132532/225000 [266:45:57<154:55:43,  6.03s/it]
 59%|█████▉    | 132533/225000 [266:46:02<142:31:33,  5.55s/it]
 59%|█████▉    | 132534/225000 [266:46:06<132:03:59,  5.14s/it]
 59%|█████▉    | 132535/225000 [266:46:10<125:44:23,  4.90s/it]
 59%|█████▉    | 132536/225000 [266:46:22<179:28:30,  6.99s/it]
 59%|█████▉    | 132537/225000 [266:46:27<161:23:14,  6.28s/it]
 59%|█████▉    | 132538/225000 [266:46:31<145:50:49,  5.68s/it]
 59%|█████▉    | 132539/225000 [266:46:35<133:46:23,  5.21s/it]
 59%|█████▉    | 132540/225000 [266:46:40<128:07:44,  4.99s/it]
 59%|█████▉    | 132541/225000 [266:46:48<154:02:13,  6.00s/it]
 59%|█████▉    | 132542/225000 [266:46:56<168:47:56,  6.57s/it]
 59%|█████▉    | 132543/225000 [266:47:00<146:45:33,  5.71s/it]
 59%|█████▉    | 132544/225000 [266:47:04<135:07:55,  5.26s/it]
 59%|█████▉    | 132545/225000 [266:47:08<128:49:37,  5.02s/it]
 59%|█████▉    | 132546/225000 [266:47:15<142:19:53,  5.54s/it]
 59%|█████▉    | 132547/225000 [266:47:26<181:30:42,  7.07s/it]
 59%|█████▉    | 132548/225000 [266:47:29<154:12:19,  6.00s/it]
 59%|█████▉    | 132549/225000 [266:47:36<161:05:56,  6.27s/it]
 59%|█████▉    | 132550/225000 [266:47:42<157:03:57,  6.12s/it]
                                                               
{'loss': '0.824', 'grad_norm': '1.12', 'learning_rate': '1.957e-05', 'epoch': '0.5891', 'num_input_tokens_seen': 443160544, 'train_runtime': '9.605e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132550/225000 [266:47:42<157:03:57,  6.12s/it]
 59%|█████▉    | 132551/225000 [266:47:45<137:26:09,  5.35s/it]
 59%|█████▉    | 132552/225000 [266:47:54<166:25:56,  6.48s/it]
 59%|█████▉    | 132553/225000 [266:48:00<160:47:44,  6.26s/it]
 59%|█████▉    | 132554/225000 [266:48:06<157:25:26,  6.13s/it]
 59%|█████▉    | 132555/225000 [266:48:13<161:54:51,  6.31s/it]
 59%|█████▉    | 132556/225000 [266:48:17<145:56:30,  5.68s/it]
 59%|█████▉    | 132557/225000 [266:48:22<140:09:57,  5.46s/it]
 59%|█████▉    | 132558/225000 [266:48:30<158:11:02,  6.16s/it]
 59%|█████▉    | 132559/225000 [266:48:37<165:24:31,  6.44s/it]
 59%|█████▉    | 132560/225000 [266:48:40<142:11:43,  5.54s/it]
 59%|█████▉    | 132561/225000 [266:48:48<156:31:03,  6.10s/it]
 59%|█████▉    | 132562/225000 [266:48:52<142:16:30,  5.54s/it]
 59%|█████▉    | 132563/225000 [266:48:56<133:24:19,  5.20s/it]
 59%|█████▉    | 132564/225000 [266:49:01<127:48:36,  4.98s/it]
 59%|█████▉    | 132565/225000 [266:49:10<162:14:11,  6.32s/it]
 59%|█████▉    | 132566/225000 [266:49:19<178:17:10,  6.94s/it]
 59%|█████▉    | 132567/225000 [266:49:25<177:28:51,  6.91s/it]
 59%|█████▉    | 132568/225000 [266:49:29<151:15:46,  5.89s/it]
 59%|█████▉    | 132569/225000 [266:49:32<132:35:32,  5.16s/it]
 59%|█████▉    | 132570/225000 [266:49:41<161:33:50,  6.29s/it]
 59%|█████▉    | 132571/225000 [266:49:45<140:42:29,  5.48s/it]
 59%|█████▉    | 132572/225000 [266:49:54<171:35:43,  6.68s/it]
 59%|█████▉    | 132573/225000 [266:49:58<145:02:48,  5.65s/it]
 59%|█████▉    | 132574/225000 [266:50:02<132:52:55,  5.18s/it]
 59%|█████▉    | 132575/225000 [266:50:10<160:24:25,  6.25s/it]
 59%|█████▉    | 132576/225000 [266:50:14<141:53:26,  5.53s/it]
 59%|█████▉    | 132577/225000 [266:50:18<130:31:12,  5.08s/it]
 59%|█████▉    | 132578/225000 [266:50:23<123:34:51,  4.81s/it]
 59%|█████▉    | 132579/225000 [266:50:34<173:37:25,  6.76s/it]
 59%|█████▉    | 132580/225000 [266:50:40<166:32:13,  6.49s/it]
 59%|█████▉    | 132581/225000 [266:50:45<154:11:45,  6.01s/it]
 59%|█████▉    | 132582/225000 [266:50:49<142:51:08,  5.56s/it]
 59%|█████▉    | 132583/225000 [266:50:54<134:48:21,  5.25s/it]
 59%|█████▉    | 132584/225000 [266:50:58<130:51:40,  5.10s/it]
 59%|█████▉    | 132585/225000 [266:51:03<125:13:09,  4.88s/it]
 59%|█████▉    | 132586/225000 [266:51:07<122:57:24,  4.79s/it]
 59%|█████▉    | 132587/225000 [266:51:12<123:20:41,  4.80s/it]
 59%|█████▉    | 132588/225000 [266:51:20<143:52:52,  5.61s/it]
 59%|█████▉    | 132589/225000 [266:51:33<200:07:00,  7.80s/it]
 59%|█████▉    | 132590/225000 [266:51:37<175:56:46,  6.85s/it]
 59%|█████▉    | 132591/225000 [266:51:42<159:46:57,  6.22s/it]
 59%|█████▉    | 132592/225000 [266:51:46<146:01:25,  5.69s/it]
 59%|█████▉    | 132593/225000 [266:51:51<138:48:04,  5.41s/it]
 59%|█████▉    | 132594/225000 [266:51:55<129:38:29,  5.05s/it]
 59%|█████▉    | 132595/225000 [266:52:00<126:40:39,  4.94s/it]
 59%|█████▉    | 132596/225000 [266:52:08<149:46:39,  5.84s/it]
 59%|█████▉    | 132597/225000 [266:52:19<188:04:25,  7.33s/it]
 59%|█████▉    | 132598/225000 [266:52:23<161:02:31,  6.27s/it]
 59%|█████▉    | 132599/225000 [266:52:28<157:18:02,  6.13s/it]
 59%|█████▉    | 132600/225000 [266:52:37<177:01:37,  6.90s/it]
                                                               
{'loss': '0.8096', 'grad_norm': '1.04', 'learning_rate': '1.955e-05', 'epoch': '0.5893', 'num_input_tokens_seen': 443290656, 'train_runtime': '9.608e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132600/225000 [266:52:37<177:01:37,  6.90s/it]
 59%|█████▉    | 132601/225000 [266:52:41<150:56:14,  5.88s/it]
 59%|█████▉    | 132602/225000 [266:52:44<135:10:05,  5.27s/it]
 59%|█████▉    | 132603/225000 [266:52:54<169:37:32,  6.61s/it]
 59%|█████▉    | 132604/225000 [266:52:58<147:38:47,  5.75s/it]
 59%|█████▉    | 132605/225000 [266:53:03<145:31:45,  5.67s/it]
 59%|█████▉    | 132606/225000 [266:53:12<171:44:49,  6.69s/it]
 59%|█████▉    | 132607/225000 [266:53:17<153:26:44,  5.98s/it]
 59%|█████▉    | 132608/225000 [266:53:24<161:24:08,  6.29s/it]
 59%|█████▉    | 132609/225000 [266:53:28<148:25:57,  5.78s/it]
 59%|█████▉    | 132610/225000 [266:53:34<144:00:00,  5.61s/it]
 59%|█████▉    | 132611/225000 [266:53:39<141:22:21,  5.51s/it]
 59%|█████▉    | 132612/225000 [266:53:44<139:21:37,  5.43s/it]
 59%|█████▉    | 132613/225000 [266:53:49<137:59:28,  5.38s/it]
 59%|█████▉    | 132614/225000 [266:53:55<136:54:27,  5.33s/it]
 59%|█████▉    | 132615/225000 [266:54:00<135:18:00,  5.27s/it]
 59%|█████▉    | 132616/225000 [266:54:04<130:46:06,  5.10s/it]
 59%|█████▉    | 132617/225000 [266:54:10<132:10:42,  5.15s/it]
 59%|█████▉    | 132618/225000 [266:54:16<139:55:45,  5.45s/it]
 59%|█████▉    | 132619/225000 [266:54:27<184:29:38,  7.19s/it]
 59%|█████▉    | 132620/225000 [266:54:34<182:08:07,  7.10s/it]
 59%|█████▉    | 132621/225000 [266:54:38<158:55:46,  6.19s/it]
 59%|█████▉    | 132622/225000 [266:54:42<141:34:45,  5.52s/it]
 59%|█████▉    | 132623/225000 [266:54:53<186:20:03,  7.26s/it]
 59%|█████▉    | 132624/225000 [266:54:57<155:03:13,  6.04s/it]
 59%|█████▉    | 132625/225000 [266:55:04<166:02:59,  6.47s/it]
 59%|█████▉    | 132626/225000 [266:55:08<145:06:30,  5.66s/it]
 59%|█████▉    | 132627/225000 [266:55:13<141:00:40,  5.50s/it]
 59%|█████▉    | 132628/225000 [266:55:17<133:01:50,  5.18s/it]
 59%|█████▉    | 132629/225000 [266:55:22<128:13:30,  5.00s/it]
 59%|█████▉    | 132630/225000 [266:55:27<128:07:30,  4.99s/it]
 59%|█████▉    | 132631/225000 [266:55:32<128:10:55,  5.00s/it]
 59%|█████▉    | 132632/225000 [266:55:36<124:14:48,  4.84s/it]
 59%|█████▉    | 132633/225000 [266:55:43<139:35:30,  5.44s/it]
 59%|█████▉    | 132634/225000 [266:55:56<193:10:28,  7.53s/it]
 59%|█████▉    | 132635/225000 [266:55:59<162:20:13,  6.33s/it]
 59%|█████▉    | 132636/225000 [266:56:04<150:05:29,  5.85s/it]
 59%|█████▉    | 132637/225000 [266:56:09<144:22:15,  5.63s/it]
 59%|█████▉    | 132638/225000 [266:56:14<137:52:53,  5.37s/it]
 59%|█████▉    | 132639/225000 [266:56:18<132:14:35,  5.15s/it]
 59%|█████▉    | 132640/225000 [266:56:23<126:26:58,  4.93s/it]
 59%|█████▉    | 132641/225000 [266:56:29<138:17:52,  5.39s/it]
 59%|█████▉    | 132642/225000 [266:56:41<184:25:10,  7.19s/it]
 59%|█████▉    | 132643/225000 [266:56:45<163:03:59,  6.36s/it]
 59%|█████▉    | 132644/225000 [266:56:50<151:02:55,  5.89s/it]
 59%|█████▉    | 132645/225000 [266:56:57<159:41:20,  6.22s/it]
 59%|█████▉    | 132646/225000 [266:57:01<145:48:10,  5.68s/it]
 59%|█████▉    | 132647/225000 [266:57:06<135:26:43,  5.28s/it]
 59%|█████▉    | 132648/225000 [266:57:16<175:08:36,  6.83s/it]
 59%|█████▉    | 132649/225000 [266:57:21<159:54:58,  6.23s/it]
 59%|█████▉    | 132650/225000 [266:57:27<156:07:40,  6.09s/it]
                                                               
{'loss': '0.8068', 'grad_norm': '1.018', 'learning_rate': '1.953e-05', 'epoch': '0.5896', 'num_input_tokens_seen': 443420560, 'train_runtime': '9.61e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132650/225000 [266:57:27<156:07:40,  6.09s/it]
 59%|█████▉    | 132651/225000 [266:57:34<162:14:32,  6.32s/it]
 59%|█████▉    | 132652/225000 [266:57:40<159:50:01,  6.23s/it]
 59%|█████▉    | 132653/225000 [266:57:46<164:43:20,  6.42s/it]
 59%|█████▉    | 132654/225000 [266:57:50<142:32:00,  5.56s/it]
 59%|█████▉    | 132655/225000 [266:57:57<155:05:47,  6.05s/it]
 59%|█████▉    | 132656/225000 [266:58:02<144:40:18,  5.64s/it]
 59%|█████▉    | 132657/225000 [266:58:07<138:09:07,  5.39s/it]
 59%|█████▉    | 132658/225000 [266:58:11<131:03:22,  5.11s/it]
 59%|█████▉    | 132659/225000 [266:58:16<130:09:56,  5.07s/it]
 59%|█████▉    | 132660/225000 [266:58:21<129:23:43,  5.04s/it]
 59%|█████▉    | 132661/225000 [266:58:26<129:12:47,  5.04s/it]
 59%|█████▉    | 132662/225000 [266:58:31<129:34:45,  5.05s/it]
 59%|█████▉    | 132663/225000 [266:58:37<134:19:30,  5.24s/it]
 59%|█████▉    | 132664/225000 [266:58:50<194:39:47,  7.59s/it]
 59%|█████▉    | 132665/225000 [266:58:55<171:30:47,  6.69s/it]
 59%|█████▉    | 132666/225000 [266:59:01<173:27:36,  6.76s/it]
 59%|█████▉    | 132667/225000 [266:59:04<144:35:27,  5.64s/it]
 59%|█████▉    | 132668/225000 [266:59:13<167:16:08,  6.52s/it]
 59%|█████▉    | 132669/225000 [266:59:18<154:20:02,  6.02s/it]
 59%|█████▉    | 132670/225000 [266:59:23<145:16:13,  5.66s/it]
 59%|█████▉    | 132671/225000 [266:59:28<140:57:44,  5.50s/it]
 59%|█████▉    | 132672/225000 [266:59:32<133:48:01,  5.22s/it]
 59%|█████▉    | 132673/225000 [266:59:37<132:37:10,  5.17s/it]
 59%|█████▉    | 132674/225000 [266:59:42<125:36:40,  4.90s/it]
 59%|█████▉    | 132675/225000 [266:59:46<124:09:07,  4.84s/it]
 59%|█████▉    | 132676/225000 [266:59:52<126:07:22,  4.92s/it]
 59%|█████▉    | 132677/225000 [266:59:58<137:40:46,  5.37s/it]
 59%|█████▉    | 132678/225000 [267:00:11<196:18:51,  7.66s/it]
 59%|█████▉    | 132679/225000 [267:00:17<185:55:18,  7.25s/it]
 59%|█████▉    | 132680/225000 [267:00:21<161:46:20,  6.31s/it]
 59%|█████▉    | 132681/225000 [267:00:25<143:18:18,  5.59s/it]
 59%|█████▉    | 132682/225000 [267:00:35<173:00:36,  6.75s/it]
 59%|█████▉    | 132683/225000 [267:00:42<173:30:05,  6.77s/it]
 59%|█████▉    | 132684/225000 [267:00:46<157:20:21,  6.14s/it]
 59%|█████▉    | 132685/225000 [267:00:51<145:49:43,  5.69s/it]
 59%|█████▉    | 132686/225000 [267:00:56<138:05:29,  5.39s/it]
 59%|█████▉    | 132687/225000 [267:01:00<133:35:51,  5.21s/it]
 59%|█████▉    | 132688/225000 [267:01:05<129:28:38,  5.05s/it]
 59%|█████▉    | 132689/225000 [267:01:10<127:31:36,  4.97s/it]
 59%|█████▉    | 132690/225000 [267:01:23<189:27:37,  7.39s/it]
 59%|█████▉    | 132691/225000 [267:01:28<172:24:33,  6.72s/it]
 59%|█████▉    | 132692/225000 [267:01:34<163:07:16,  6.36s/it]
 59%|█████▉    | 132693/225000 [267:01:40<161:34:44,  6.30s/it]
 59%|█████▉    | 132694/225000 [267:01:44<143:37:31,  5.60s/it]
 59%|█████▉    | 132695/225000 [267:01:48<134:01:13,  5.23s/it]
 59%|█████▉    | 132696/225000 [267:02:00<184:21:46,  7.19s/it]
 59%|█████▉    | 132697/225000 [267:02:03<156:04:48,  6.09s/it]
 59%|█████▉    | 132698/225000 [267:02:11<167:32:18,  6.53s/it]
 59%|█████▉    | 132699/225000 [267:02:15<145:48:48,  5.69s/it]
 59%|█████▉    | 132700/225000 [267:02:19<134:06:04,  5.23s/it]
                                                               
{'loss': '0.804', 'grad_norm': '0.9903', 'learning_rate': '1.951e-05', 'epoch': '0.5898', 'num_input_tokens_seen': 443551320, 'train_runtime': '9.613e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132700/225000 [267:02:19<134:06:04,  5.23s/it]
 59%|█████▉    | 132701/225000 [267:02:23<125:45:32,  4.91s/it]
 59%|█████▉    | 132702/225000 [267:02:31<152:02:40,  5.93s/it]
 59%|█████▉    | 132703/225000 [267:02:40<174:43:50,  6.82s/it]
 59%|█████▉    | 132704/225000 [267:02:44<153:54:59,  6.00s/it]
 59%|█████▉    | 132705/225000 [267:02:48<137:27:44,  5.36s/it]
 59%|█████▉    | 132706/225000 [267:02:52<128:00:27,  4.99s/it]
 59%|█████▉    | 132707/225000 [267:03:00<150:59:23,  5.89s/it]
 59%|█████▉    | 132708/225000 [267:03:09<171:05:45,  6.67s/it]
 59%|█████▉    | 132709/225000 [267:03:12<146:09:21,  5.70s/it]
 59%|█████▉    | 132710/225000 [267:03:18<150:40:07,  5.88s/it]
 59%|█████▉    | 132711/225000 [267:03:26<160:49:45,  6.27s/it]
 59%|█████▉    | 132712/225000 [267:03:30<143:03:37,  5.58s/it]
 59%|█████▉    | 132713/225000 [267:03:38<161:44:20,  6.31s/it]
 59%|█████▉    | 132714/225000 [267:03:42<145:31:40,  5.68s/it]
 59%|█████▉    | 132715/225000 [267:03:49<153:57:57,  6.01s/it]
 59%|█████▉    | 132716/225000 [267:03:53<139:48:35,  5.45s/it]
 59%|█████▉    | 132717/225000 [267:04:01<161:13:17,  6.29s/it]
 59%|█████▉    | 132718/225000 [267:04:04<136:46:34,  5.34s/it]
 59%|█████▉    | 132719/225000 [267:04:12<154:31:34,  6.03s/it]
 59%|█████▉    | 132720/225000 [267:04:15<134:00:22,  5.23s/it]
 59%|█████▉    | 132721/225000 [267:04:20<132:30:25,  5.17s/it]
 59%|█████▉    | 132722/225000 [267:04:25<129:16:23,  5.04s/it]
 59%|█████▉    | 132723/225000 [267:04:30<129:23:10,  5.05s/it]
 59%|█████▉    | 132724/225000 [267:04:35<129:25:26,  5.05s/it]
 59%|█████▉    | 132725/225000 [267:04:40<130:54:17,  5.11s/it]
 59%|█████▉    | 132726/225000 [267:04:45<130:14:24,  5.08s/it]
 59%|█████▉    | 132727/225000 [267:04:51<131:55:16,  5.15s/it]
 59%|█████▉    | 132728/225000 [267:04:56<132:47:22,  5.18s/it]
 59%|█████▉    | 132729/225000 [267:05:05<160:41:23,  6.27s/it]
 59%|█████▉    | 132730/225000 [267:05:17<204:03:50,  7.96s/it]
 59%|█████▉    | 132731/225000 [267:05:20<169:07:00,  6.60s/it]
 59%|█████▉    | 132732/225000 [267:05:27<169:24:07,  6.61s/it]
 59%|█████▉    | 132733/225000 [267:05:30<148:55:04,  5.81s/it]
 59%|█████▉    | 132734/225000 [267:05:35<136:51:06,  5.34s/it]
 59%|█████▉    | 132735/225000 [267:05:43<160:24:35,  6.26s/it]
 59%|█████▉    | 132736/225000 [267:05:50<164:19:51,  6.41s/it]
 59%|█████▉    | 132737/225000 [267:05:54<143:26:45,  5.60s/it]
 59%|█████▉    | 132738/225000 [267:05:59<138:19:14,  5.40s/it]
 59%|█████▉    | 132739/225000 [267:06:09<175:17:08,  6.84s/it]
 59%|█████▉    | 132740/225000 [267:06:12<147:38:33,  5.76s/it]
 59%|█████▉    | 132741/225000 [267:06:21<172:45:50,  6.74s/it]
 59%|█████▉    | 132742/225000 [267:06:24<143:20:59,  5.59s/it]
 59%|█████▉    | 132743/225000 [267:06:29<141:56:52,  5.54s/it]
 59%|█████▉    | 132744/225000 [267:06:35<142:32:10,  5.56s/it]
 59%|█████▉    | 132745/225000 [267:06:40<137:40:10,  5.37s/it]
 59%|█████▉    | 132746/225000 [267:06:45<135:05:49,  5.27s/it]
 59%|█████▉    | 132747/225000 [267:06:50<134:11:56,  5.24s/it]
 59%|█████▉    | 132748/225000 [267:06:55<134:38:16,  5.25s/it]
 59%|█████▉    | 132749/225000 [267:07:01<133:43:46,  5.22s/it]
 59%|█████▉    | 132750/225000 [267:07:06<138:17:37,  5.40s/it]
                                                               
{'loss': '0.8068', 'grad_norm': '1.022', 'learning_rate': '1.95e-05', 'epoch': '0.59', 'num_input_tokens_seen': 443680672, 'train_runtime': '9.616e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132750/225000 [267:07:06<138:17:37,  5.40s/it]
 59%|█████▉    | 132751/225000 [267:07:12<137:31:10,  5.37s/it]
 59%|█████▉    | 132752/225000 [267:07:17<133:47:26,  5.22s/it]
 59%|█████▉    | 132753/225000 [267:07:21<130:13:12,  5.08s/it]
 59%|█████▉    | 132754/225000 [267:07:27<132:37:40,  5.18s/it]
 59%|█████▉    | 132755/225000 [267:07:32<131:45:44,  5.14s/it]
 59%|█████▉    | 132756/225000 [267:07:37<134:52:23,  5.26s/it]
 59%|█████▉    | 132757/225000 [267:07:42<133:26:38,  5.21s/it]
 59%|█████▉    | 132758/225000 [267:07:48<135:00:50,  5.27s/it]
 59%|█████▉    | 132759/225000 [267:07:55<149:37:07,  5.84s/it]
 59%|█████▉    | 132760/225000 [267:08:08<202:30:18,  7.90s/it]
 59%|█████▉    | 132761/225000 [267:08:13<180:53:56,  7.06s/it]
 59%|█████▉    | 132762/225000 [267:08:20<180:40:33,  7.05s/it]
 59%|█████▉    | 132763/225000 [267:08:23<151:19:49,  5.91s/it]
 59%|█████▉    | 132764/225000 [267:08:32<174:18:13,  6.80s/it]
 59%|█████▉    | 132765/225000 [267:08:38<170:31:06,  6.66s/it]
 59%|█████▉    | 132766/225000 [267:08:42<146:39:22,  5.72s/it]
 59%|█████▉    | 132767/225000 [267:08:51<172:25:52,  6.73s/it]
 59%|█████▉    | 132768/225000 [267:08:55<149:16:44,  5.83s/it]
 59%|█████▉    | 132769/225000 [267:09:02<164:20:39,  6.41s/it]
 59%|█████▉    | 132770/225000 [267:09:06<140:23:38,  5.48s/it]
 59%|█████▉    | 132771/225000 [267:09:14<160:56:53,  6.28s/it]
 59%|█████▉    | 132772/225000 [267:09:17<136:22:50,  5.32s/it]
 59%|█████▉    | 132773/225000 [267:09:24<150:52:35,  5.89s/it]
 59%|█████▉    | 132774/225000 [267:09:28<138:39:13,  5.41s/it]
 59%|█████▉    | 132775/225000 [267:09:33<131:06:31,  5.12s/it]
 59%|█████▉    | 132776/225000 [267:09:37<123:32:03,  4.82s/it]
 59%|█████▉    | 132777/225000 [267:09:45<146:45:39,  5.73s/it]
 59%|█████▉    | 132778/225000 [267:09:54<170:11:18,  6.64s/it]
 59%|█████▉    | 132779/225000 [267:09:57<148:42:30,  5.81s/it]
 59%|█████▉    | 132780/225000 [267:10:01<134:57:05,  5.27s/it]
 59%|█████▉    | 132781/225000 [267:10:06<126:33:18,  4.94s/it]
 59%|█████▉    | 132782/225000 [267:10:14<149:45:23,  5.85s/it]
 59%|█████▉    | 132783/225000 [267:10:21<161:28:57,  6.30s/it]
 59%|█████▉    | 132784/225000 [267:10:25<145:24:38,  5.68s/it]
 59%|█████▉    | 132785/225000 [267:10:30<135:35:16,  5.29s/it]
 59%|█████▉    | 132786/225000 [267:10:42<189:51:30,  7.41s/it]
 59%|█████▉    | 132787/225000 [267:10:45<159:06:08,  6.21s/it]
 59%|█████▉    | 132788/225000 [267:10:53<173:02:59,  6.76s/it]
 59%|█████▉    | 132789/225000 [267:10:58<153:58:40,  6.01s/it]
 59%|█████▉    | 132790/225000 [267:11:02<140:21:00,  5.48s/it]
 59%|█████▉    | 132791/225000 [267:11:06<129:20:09,  5.05s/it]
 59%|█████▉    | 132792/225000 [267:11:17<176:22:10,  6.89s/it]
 59%|█████▉    | 132793/225000 [267:11:22<164:06:20,  6.41s/it]
 59%|█████▉    | 132794/225000 [267:11:26<145:39:45,  5.69s/it]
 59%|█████▉    | 132795/225000 [267:11:30<133:36:02,  5.22s/it]
 59%|█████▉    | 132796/225000 [267:11:34<123:04:54,  4.81s/it]
 59%|█████▉    | 132797/225000 [267:11:41<136:17:37,  5.32s/it]
 59%|█████▉    | 132798/225000 [267:11:51<169:34:10,  6.62s/it]
 59%|█████▉    | 132799/225000 [267:11:54<141:40:47,  5.53s/it]
 59%|█████▉    | 132800/225000 [267:12:02<163:12:55,  6.37s/it]
                                                               
{'loss': '0.8015', 'grad_norm': '0.9863', 'learning_rate': '1.948e-05', 'epoch': '0.5902', 'num_input_tokens_seen': 443809624, 'train_runtime': '9.619e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132800/225000 [267:12:02<163:12:55,  6.37s/it]
 59%|█████▉    | 132801/225000 [267:12:06<143:22:59,  5.60s/it]
 59%|█████▉    | 132802/225000 [267:12:09<127:05:08,  4.96s/it]
 59%|█████▉    | 132803/225000 [267:12:16<142:34:17,  5.57s/it]
 59%|█████▉    | 132804/225000 [267:12:23<155:17:30,  6.06s/it]
 59%|█████▉    | 132805/225000 [267:12:27<138:50:38,  5.42s/it]
 59%|█████▉    | 132806/225000 [267:12:31<127:55:48,  5.00s/it]
 59%|█████▉    | 132807/225000 [267:12:36<128:16:35,  5.01s/it]
 59%|█████▉    | 132808/225000 [267:12:46<168:15:24,  6.57s/it]
 59%|█████▉    | 132809/225000 [267:12:50<144:38:08,  5.65s/it]
 59%|█████▉    | 132810/225000 [267:12:58<163:40:44,  6.39s/it]
 59%|█████▉    | 132811/225000 [267:13:02<145:22:18,  5.68s/it]
 59%|█████▉    | 132812/225000 [267:13:08<145:07:40,  5.67s/it]
 59%|█████▉    | 132813/225000 [267:13:12<137:13:29,  5.36s/it]
 59%|█████▉    | 132814/225000 [267:13:18<136:38:07,  5.34s/it]
 59%|█████▉    | 132815/225000 [267:13:28<176:24:13,  6.89s/it]
 59%|█████▉    | 132816/225000 [267:13:32<154:27:14,  6.03s/it]
 59%|█████▉    | 132817/225000 [267:13:37<147:02:25,  5.74s/it]
 59%|█████▉    | 132818/225000 [267:13:43<145:01:08,  5.66s/it]
 59%|█████▉    | 132819/225000 [267:13:48<138:36:43,  5.41s/it]
 59%|█████▉    | 132820/225000 [267:13:52<133:36:20,  5.22s/it]
 59%|█████▉    | 132821/225000 [267:13:57<132:49:05,  5.19s/it]
 59%|█████▉    | 132822/225000 [267:14:03<134:21:20,  5.25s/it]
 59%|█████▉    | 132823/225000 [267:14:08<134:43:01,  5.26s/it]
 59%|█████▉    | 132824/225000 [267:14:13<134:39:12,  5.26s/it]
 59%|█████▉    | 132825/225000 [267:14:18<133:05:39,  5.20s/it]
 59%|█████▉    | 132826/225000 [267:14:24<136:30:59,  5.33s/it]
 59%|█████▉    | 132827/225000 [267:14:37<193:13:06,  7.55s/it]
 59%|█████▉    | 132828/225000 [267:14:41<170:02:11,  6.64s/it]
 59%|█████▉    | 132829/225000 [267:14:49<181:08:14,  7.07s/it]
 59%|█████▉    | 132830/225000 [267:14:54<165:13:49,  6.45s/it]
 59%|█████▉    | 132831/225000 [267:15:01<164:10:49,  6.41s/it]
 59%|█████▉    | 132832/225000 [267:15:07<161:47:20,  6.32s/it]
 59%|█████▉    | 132833/225000 [267:15:14<168:27:17,  6.58s/it]
 59%|█████▉    | 132834/225000 [267:15:18<146:11:41,  5.71s/it]
 59%|█████▉    | 132835/225000 [267:15:25<160:32:09,  6.27s/it]
 59%|█████▉    | 132836/225000 [267:15:31<153:11:51,  5.98s/it]
 59%|█████▉    | 132837/225000 [267:15:36<147:58:58,  5.78s/it]
 59%|█████▉    | 132838/225000 [267:15:41<143:02:50,  5.59s/it]
 59%|█████▉    | 132839/225000 [267:15:48<154:50:40,  6.05s/it]
 59%|█████▉    | 132840/225000 [267:15:55<157:38:10,  6.16s/it]
 59%|█████▉    | 132841/225000 [267:16:00<149:23:31,  5.84s/it]
 59%|█████▉    | 132842/225000 [267:16:05<144:21:19,  5.64s/it]
 59%|█████▉    | 132843/225000 [267:16:10<140:37:39,  5.49s/it]
 59%|█████▉    | 132844/225000 [267:16:19<166:18:27,  6.50s/it]
 59%|█████▉    | 132845/225000 [267:16:24<154:36:05,  6.04s/it]
 59%|█████▉    | 132846/225000 [267:16:30<154:17:52,  6.03s/it]
 59%|█████▉    | 132847/225000 [267:16:34<139:03:40,  5.43s/it]
 59%|█████▉    | 132848/225000 [267:16:41<152:09:36,  5.94s/it]
 59%|█████▉    | 132849/225000 [267:16:49<166:30:05,  6.50s/it]
 59%|█████▉    | 132850/225000 [267:16:54<157:34:46,  6.16s/it]
                                                               
{'loss': '0.8101', 'grad_norm': '1.082', 'learning_rate': '1.946e-05', 'epoch': '0.5904', 'num_input_tokens_seen': 443940088, 'train_runtime': '9.622e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132850/225000 [267:16:54<157:34:46,  6.16s/it]
 59%|█████▉    | 132851/225000 [267:16:59<147:43:45,  5.77s/it]
 59%|█████▉    | 132852/225000 [267:17:04<143:22:30,  5.60s/it]
 59%|█████▉    | 132853/225000 [267:17:09<139:22:43,  5.45s/it]
 59%|█████▉    | 132854/225000 [267:17:15<137:57:45,  5.39s/it]
 59%|█████▉    | 132855/225000 [267:17:20<135:53:22,  5.31s/it]
 59%|█████▉    | 132856/225000 [267:17:25<135:49:31,  5.31s/it]
 59%|█████▉    | 132857/225000 [267:17:30<136:29:18,  5.33s/it]
 59%|█████▉    | 132858/225000 [267:17:36<136:32:04,  5.33s/it]
 59%|█████▉    | 132859/225000 [267:17:46<177:26:14,  6.93s/it]
 59%|█████▉    | 132860/225000 [267:17:55<189:13:49,  7.39s/it]
 59%|█████▉    | 132861/225000 [267:17:59<163:51:21,  6.40s/it]
 59%|█████▉    | 132862/225000 [267:18:09<190:31:17,  7.44s/it]
 59%|█████▉    | 132863/225000 [267:18:13<167:46:10,  6.56s/it]
 59%|█████▉    | 132864/225000 [267:18:17<148:30:21,  5.80s/it]
 59%|█████▉    | 132865/225000 [267:18:21<134:06:35,  5.24s/it]
 59%|█████▉    | 132866/225000 [267:18:32<174:38:21,  6.82s/it]
 59%|█████▉    | 132867/225000 [267:18:37<159:38:36,  6.24s/it]
 59%|█████▉    | 132868/225000 [267:18:42<153:40:39,  6.00s/it]
 59%|█████▉    | 132869/225000 [267:18:50<169:28:43,  6.62s/it]
 59%|█████▉    | 132870/225000 [267:18:55<151:39:31,  5.93s/it]
 59%|█████▉    | 132871/225000 [267:19:02<162:59:30,  6.37s/it]
 59%|█████▉    | 132872/225000 [267:19:05<138:36:15,  5.42s/it]
 59%|█████▉    | 132873/225000 [267:19:10<135:26:35,  5.29s/it]
 59%|█████▉    | 132874/225000 [267:19:18<155:55:18,  6.09s/it]
 59%|█████▉    | 132875/225000 [267:19:22<137:23:31,  5.37s/it]
 59%|█████▉    | 132876/225000 [267:19:30<161:49:09,  6.32s/it]
 59%|█████▉    | 132877/225000 [267:19:33<137:20:27,  5.37s/it]
 59%|█████▉    | 132878/225000 [267:19:40<147:53:46,  5.78s/it]
 59%|█████▉    | 132879/225000 [267:19:44<130:34:14,  5.10s/it]
 59%|█████▉    | 132880/225000 [267:19:47<118:35:24,  4.63s/it]
 59%|█████▉    | 132881/225000 [267:19:57<156:47:34,  6.13s/it]
 59%|█████▉    | 132882/225000 [267:20:02<148:59:31,  5.82s/it]
 59%|█████▉    | 132883/225000 [267:20:10<168:57:34,  6.60s/it]
 59%|█████▉    | 132884/225000 [267:20:14<147:26:05,  5.76s/it]
 59%|█████▉    | 132885/225000 [267:20:19<141:03:27,  5.51s/it]
 59%|█████▉    | 132886/225000 [267:20:29<175:36:55,  6.86s/it]
 59%|█████▉    | 132887/225000 [267:20:34<160:38:50,  6.28s/it]
 59%|█████▉    | 132888/225000 [267:20:39<150:25:38,  5.88s/it]
 59%|█████▉    | 132889/225000 [267:20:44<142:45:26,  5.58s/it]
 59%|█████▉    | 132890/225000 [267:20:49<139:08:50,  5.44s/it]
 59%|█████▉    | 132891/225000 [267:20:53<130:29:18,  5.10s/it]
 59%|█████▉    | 132892/225000 [267:20:58<129:25:19,  5.06s/it]
 59%|█████▉    | 132893/225000 [267:21:03<126:12:17,  4.93s/it]
 59%|█████▉    | 132894/225000 [267:21:08<125:35:55,  4.91s/it]
 59%|█████▉    | 132895/225000 [267:21:13<128:31:13,  5.02s/it]
 59%|█████▉    | 132896/225000 [267:21:22<160:28:51,  6.27s/it]
 59%|█████▉    | 132897/225000 [267:21:33<191:07:03,  7.47s/it]
 59%|█████▉    | 132898/225000 [267:21:36<160:52:46,  6.29s/it]
 59%|█████▉    | 132899/225000 [267:21:41<149:17:25,  5.84s/it]
 59%|█████▉    | 132900/225000 [267:21:46<142:44:56,  5.58s/it]
                                                               
{'loss': '0.8041', 'grad_norm': '1.143', 'learning_rate': '1.944e-05', 'epoch': '0.5907', 'num_input_tokens_seen': 444069688, 'train_runtime': '9.625e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132900/225000 [267:21:46<142:44:56,  5.58s/it]
 59%|█████▉    | 132901/225000 [267:21:51<137:56:59,  5.39s/it]
 59%|█████▉    | 132902/225000 [267:21:55<132:44:29,  5.19s/it]
 59%|█████▉    | 132903/225000 [267:22:08<185:46:12,  7.26s/it]
 59%|█████▉    | 132904/225000 [267:22:13<175:27:50,  6.86s/it]
 59%|█████▉    | 132905/225000 [267:22:21<179:58:41,  7.04s/it]
 59%|█████▉    | 132906/225000 [267:22:26<162:42:58,  6.36s/it]
 59%|█████▉    | 132907/225000 [267:22:31<152:24:03,  5.96s/it]
 59%|█████▉    | 132908/225000 [267:22:36<145:02:24,  5.67s/it]
 59%|█████▉    | 132909/225000 [267:22:41<141:51:09,  5.55s/it]
 59%|█████▉    | 132910/225000 [267:22:45<131:09:01,  5.13s/it]
 59%|█████▉    | 132911/225000 [267:22:50<130:36:43,  5.11s/it]
 59%|█████▉    | 132912/225000 [267:22:56<132:57:04,  5.20s/it]
 59%|█████▉    | 132913/225000 [267:23:01<133:07:21,  5.20s/it]
 59%|█████▉    | 132914/225000 [267:23:07<140:00:34,  5.47s/it]
 59%|█████▉    | 132915/225000 [267:23:18<184:54:32,  7.23s/it]
 59%|█████▉    | 132916/225000 [267:23:24<173:56:14,  6.80s/it]
 59%|█████▉    | 132917/225000 [267:23:31<172:28:57,  6.74s/it]
 59%|█████▉    | 132918/225000 [267:23:36<161:11:16,  6.30s/it]
 59%|█████▉    | 132919/225000 [267:23:40<144:47:20,  5.66s/it]
 59%|█████▉    | 132920/225000 [267:23:44<132:10:42,  5.17s/it]
 59%|█████▉    | 132921/225000 [267:23:53<163:23:51,  6.39s/it]
 59%|█████▉    | 132922/225000 [267:24:00<164:53:14,  6.45s/it]
 59%|█████▉    | 132923/225000 [267:24:04<144:56:07,  5.67s/it]
 59%|█████▉    | 132924/225000 [267:24:13<175:42:18,  6.87s/it]
 59%|█████▉    | 132925/225000 [267:24:18<160:50:34,  6.29s/it]
 59%|█████▉    | 132926/225000 [267:24:24<158:19:36,  6.19s/it]
 59%|█████▉    | 132927/225000 [267:24:33<173:40:40,  6.79s/it]
 59%|█████▉    | 132928/225000 [267:24:36<148:22:58,  5.80s/it]
 59%|█████▉    | 132929/225000 [267:24:44<164:46:47,  6.44s/it]
 59%|█████▉    | 132930/225000 [267:24:49<156:36:57,  6.12s/it]
 59%|█████▉    | 132931/225000 [267:24:53<141:08:38,  5.52s/it]
 59%|█████▉    | 132932/225000 [267:24:58<131:02:48,  5.12s/it]
 59%|█████▉    | 132933/225000 [267:25:08<170:06:02,  6.65s/it]
 59%|█████▉    | 132934/225000 [267:25:13<159:55:17,  6.25s/it]
 59%|█████▉    | 132935/225000 [267:25:17<139:58:37,  5.47s/it]
 59%|█████▉    | 132936/225000 [267:25:27<176:39:33,  6.91s/it]
 59%|█████▉    | 132937/225000 [267:25:31<155:44:25,  6.09s/it]
 59%|█████▉    | 132938/225000 [267:25:36<143:27:18,  5.61s/it]
 59%|█████▉    | 132939/225000 [267:25:40<132:17:55,  5.17s/it]
 59%|█████▉    | 132940/225000 [267:25:44<126:14:50,  4.94s/it]
 59%|█████▉    | 132941/225000 [267:25:49<123:26:49,  4.83s/it]
 59%|█████▉    | 132942/225000 [267:26:02<186:26:46,  7.29s/it]
 59%|█████▉    | 132943/225000 [267:26:06<164:53:13,  6.45s/it]
 59%|█████▉    | 132944/225000 [267:26:11<148:34:24,  5.81s/it]
 59%|█████▉    | 132945/225000 [267:26:15<136:26:39,  5.34s/it]
 59%|█████▉    | 132946/225000 [267:26:19<128:04:14,  5.01s/it]
 59%|█████▉    | 132947/225000 [267:26:26<144:17:34,  5.64s/it]
 59%|█████▉    | 132948/225000 [267:26:39<197:02:10,  7.71s/it]
 59%|█████▉    | 132949/225000 [267:26:43<170:46:28,  6.68s/it]
 59%|█████▉    | 132950/225000 [267:26:47<152:04:38,  5.95s/it]
                                                               
{'loss': '0.799', 'grad_norm': '1.039', 'learning_rate': '1.943e-05', 'epoch': '0.5909', 'num_input_tokens_seen': 444199256, 'train_runtime': '9.628e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 132950/225000 [267:26:47<152:04:38,  5.95s/it]
 59%|█████▉    | 132951/225000 [267:26:51<135:34:07,  5.30s/it]
 59%|█████▉    | 132952/225000 [267:26:55<125:55:19,  4.92s/it]
 59%|█████▉    | 132953/225000 [267:27:06<169:25:41,  6.63s/it]
 59%|█████▉    | 132954/225000 [267:27:11<157:21:38,  6.15s/it]
 59%|█████▉    | 132955/225000 [267:27:15<138:07:27,  5.40s/it]
 59%|█████▉    | 132956/225000 [267:27:23<164:11:18,  6.42s/it]
 59%|█████▉    | 132957/225000 [267:27:27<143:33:38,  5.61s/it]
 59%|█████▉    | 132958/225000 [267:27:35<165:01:24,  6.45s/it]
 59%|█████▉    | 132959/225000 [267:27:42<162:04:00,  6.34s/it]
 59%|█████▉    | 132960/225000 [267:27:46<146:16:15,  5.72s/it]
 59%|█████▉    | 132961/225000 [267:27:50<135:45:45,  5.31s/it]
 59%|█████▉    | 132962/225000 [267:27:55<132:06:13,  5.17s/it]
 59%|█████▉    | 132963/225000 [267:27:59<125:49:25,  4.92s/it]
 59%|█████▉    | 132964/225000 [267:28:04<124:10:39,  4.86s/it]
 59%|█████▉    | 132965/225000 [267:28:12<151:01:01,  5.91s/it]
 59%|█████▉    | 132966/225000 [267:28:21<174:54:34,  6.84s/it]
 59%|█████▉    | 132967/225000 [267:28:25<151:39:18,  5.93s/it]
 59%|█████▉    | 132968/225000 [267:28:35<182:12:49,  7.13s/it]
 59%|█████▉    | 132969/225000 [267:28:39<160:23:45,  6.27s/it]
 59%|█████▉    | 132970/225000 [267:28:44<144:23:57,  5.65s/it]
 59%|█████▉    | 132971/225000 [267:28:49<143:46:21,  5.62s/it]
 59%|█████▉    | 132972/225000 [267:28:56<155:29:00,  6.08s/it]
 59%|█████▉    | 132973/225000 [267:29:00<136:17:41,  5.33s/it]
 59%|█████▉    | 132974/225000 [267:29:09<168:17:56,  6.58s/it]
 59%|█████▉    | 132975/225000 [267:29:14<151:39:38,  5.93s/it]
 59%|█████▉    | 132976/225000 [267:29:21<161:24:28,  6.31s/it]
 59%|█████▉    | 132977/225000 [267:29:24<137:16:33,  5.37s/it]
 59%|█████▉    | 132978/225000 [267:29:33<160:48:24,  6.29s/it]
 59%|█████▉    | 132979/225000 [267:29:37<143:46:59,  5.63s/it]
 59%|█████▉    | 132980/225000 [267:29:41<132:05:02,  5.17s/it]
 59%|█████▉    | 132981/225000 [267:29:45<127:39:40,  4.99s/it]
 59%|█████▉    | 132982/225000 [267:29:56<168:44:35,  6.60s/it]
 59%|█████▉    | 132983/225000 [267:30:00<153:04:24,  5.99s/it]
 59%|█████▉    | 132984/225000 [267:30:05<142:13:05,  5.56s/it]
 59%|█████▉    | 132985/225000 [267:30:09<131:32:26,  5.15s/it]
 59%|█████▉    | 132986/225000 [267:30:14<128:47:43,  5.04s/it]
 59%|█████▉    | 132987/225000 [267:30:19<126:17:41,  4.94s/it]
 59%|█████▉    | 132988/225000 [267:30:31<182:09:18,  7.13s/it]
 59%|█████▉    | 132989/225000 [267:30:36<171:02:53,  6.69s/it]
 59%|█████▉    | 132990/225000 [267:30:40<146:57:56,  5.75s/it]
 59%|█████▉    | 132991/225000 [267:30:48<160:41:31,  6.29s/it]
 59%|█████▉    | 132992/225000 [267:30:54<160:52:51,  6.29s/it]
 59%|█████▉    | 132993/225000 [267:30:58<144:14:51,  5.64s/it]
 59%|█████▉    | 132994/225000 [267:31:03<137:12:06,  5.37s/it]
 59%|█████▉    | 132995/225000 [267:31:12<167:32:19,  6.56s/it]
 59%|█████▉    | 132996/225000 [267:31:16<150:21:21,  5.88s/it]
 59%|█████▉    | 132997/225000 [267:31:21<138:32:17,  5.42s/it]
 59%|█████▉    | 132998/225000 [267:31:25<133:31:34,  5.22s/it]
 59%|█████▉    | 132999/225000 [267:31:30<126:34:01,  4.95s/it]
 59%|█████▉    | 133000/225000 [267:31:37<141:07:15,  5.52s/it]
                                                               
{'loss': '0.7998', 'grad_norm': '1.16', 'learning_rate': '1.941e-05', 'epoch': '0.5911', 'num_input_tokens_seen': 444328464, 'train_runtime': '9.631e+05', 'train_tokens_per_second': '461.4'}

 59%|█████▉    | 133000/225000 [267:31:37<141:07:15,  5.52s/it]
 59%|█████▉    | 133001/225000 [267:31:49<190:43:12,  7.46s/it]
 59%|█████▉    | 133002/225000 [267:31:52<158:57:55,  6.22s/it]
 59%|█████▉    | 133003/225000 [267:31:55<137:07:39,  5.37s/it]
 59%|█████▉    | 133004/225000 [267:32:04<161:46:40,  6.33s/it]
 59%|█████▉    | 133005/225000 [267:32:09<149:25:23,  5.85s/it]
 59%|█████▉    | 133006/225000 [267:32:15<152:41:42,  5.98s/it]
 59%|█████▉    | 133007/225000 [267:32:20<142:14:40,  5.57s/it]
 59%|█████▉    | 133008/225000 [267:32:24<135:45:24,  5.31s/it]
 59%|█████▉    | 133009/225000 [267:32:29<132:58:55,  5.20s/it]
 59%|█████▉    | 133010/225000 [267:32:34<128:12:18,  5.02s/it]
 59%|█████▉    | 133011/225000 [267:32:39<126:12:10,  4.94s/it]
 59%|█████▉    | 133012/225000 [267:32:44<132:55:33,  5.20s/it]
 59%|█████▉    | 133013/225000 [267:32:56<184:38:50,  7.23s/it]
 59%|█████▉    | 133014/225000 [267:33:02<172:23:44,  6.75s/it]
 59%|█████▉    | 133015/225000 [267:33:07<163:13:30,  6.39s/it]
 59%|█████▉    | 133016/225000 [267:33:14<162:56:40,  6.38s/it]
 59%|█████▉    | 133017/225000 [267:33:18<143:01:31,  5.60s/it]
 59%|█████▉    | 133018/225000 [267:33:22<132:35:12,  5.19s/it]
 59%|█████▉    | 133019/225000 [267:33:33<176:58:55,  6.93s/it]
 59%|█████▉    | 133020/225000 [267:33:36<149:33:05,  5.85s/it]
 59%|█████▉    | 133021/225000 [267:33:44<163:06:32,  6.38s/it]
 59%|█████▉    | 133022/225000 [267:33:48<146:57:48,  5.75s/it]
 59%|█████▉    | 133023/225000 [267:33:52<134:02:13,  5.25s/it]
 59%|█████▉    | 133024/225000 [267:33:56<123:16:27,  4.83s/it]
 59%|█████▉    | 133025/225000 [267:34:05<155:09:46,  6.07s/it]
 59%|█████▉    | 133026/225000 [267:34:12<162:16:22,  6.35s/it]
 59%|█████▉    | 133027/225000 [267:34:19<164:40:40,  6.45s/it]
 59%|█████▉    | 133028/225000 [267:34:23<149:23:17,  5.85s/it]
 59%|█████▉    | 133029/225000 [267:34:27<137:22:09,  5.38s/it]
 59%|█████▉    | 133030/225000 [267:34:31<126:00:30,  4.93s/it]
 59%|█████▉    | 133031/225000 [267:34:38<138:02:12,  5.40s/it]
 59%|█████▉    | 133032/225000 [267:34:49<181:07:57,  7.09s/it]
 59%|█████▉    | 133033/225000 [267:34:53<158:43:40,  6.21s/it]
 59%|█████▉    | 133034/225000 [267:34:57<144:55:50,  5.67s/it]
 59%|█████▉    | 133035/225000 [267:35:02<134:11:44,  5.25s/it]
 59%|█████▉    | 133036/225000 [267:35:14<188:36:16,  7.38s/it]
 59%|█████▉    | 133037/225000 [267:35:17<157:51:36,  6.18s/it]
 59%|█████▉    | 133038/225000 [267:35:24<163:08:06,  6.39s/it]
 59%|█████▉    | 133039/225000 [267:35:28<141:06:54,  5.52s/it]
 59%|█████▉    | 133040/225000 [267:35:32<132:12:09,  5.18s/it]
 59%|█████▉    | 133041/225000 [267:35:38<141:07:07,  5.52s/it]
 59%|█████▉    | 133042/225000 [267:35:48<174:19:25,  6.82s/it]
 59%|█████▉    | 133043/225000 [267:35:52<147:56:09,  5.79s/it]
 59%|█████▉    | 133044/225000 [267:36:01<172:09:35,  6.74s/it]
 59%|█████▉    | 133045/225000 [267:36:04<146:33:15,  5.74s/it]
 59%|█████▉    | 133046/225000 [267:36:13<173:17:50,  6.78s/it]
 59%|█████▉    | 133047/225000 [267:36:18<157:15:22,  6.16s/it]
 59%|█████▉    | 133048/225000 [267:36:23<147:35:22,  5.78s/it]
 59%|█████▉    | 133049/225000 [267:36:27<136:02:29,  5.33s/it]
 59%|█████▉    | 133050/225000 [267:36:32<131:59:36,  5.17s/it]
                                                               
{'loss': '0.8119', 'grad_norm': '0.9907', 'learning_rate': '1.939e-05', 'epoch': '0.5913', 'num_input_tokens_seen': 444458000, 'train_runtime': '9.634e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133050/225000 [267:36:32<131:59:36,  5.17s/it]
 59%|█████▉    | 133051/225000 [267:36:37<129:18:23,  5.06s/it]
 59%|█████▉    | 133052/225000 [267:36:41<124:12:17,  4.86s/it]
 59%|█████▉    | 133053/225000 [267:36:46<123:04:25,  4.82s/it]
 59%|█████▉    | 133054/225000 [267:36:55<156:19:20,  6.12s/it]
 59%|█████▉    | 133055/225000 [267:37:04<178:09:35,  6.98s/it]
 59%|█████▉    | 133056/225000 [267:37:09<165:35:36,  6.48s/it]
 59%|█████▉    | 133057/225000 [267:37:14<149:52:06,  5.87s/it]
 59%|█████▉    | 133058/225000 [267:37:18<139:50:02,  5.48s/it]
 59%|█████▉    | 133059/225000 [267:37:23<134:18:17,  5.26s/it]
 59%|█████▉    | 133060/225000 [267:37:28<131:51:11,  5.16s/it]
 59%|█████▉    | 133061/225000 [267:37:33<133:08:15,  5.21s/it]
 59%|█████▉    | 133062/225000 [267:37:39<138:33:54,  5.43s/it]
 59%|█████▉    | 133063/225000 [267:37:52<196:34:41,  7.70s/it]
 59%|█████▉    | 133064/225000 [267:37:57<171:39:07,  6.72s/it]
 59%|█████▉    | 133065/225000 [267:38:01<153:58:01,  6.03s/it]
 59%|█████▉    | 133066/225000 [267:38:06<143:23:39,  5.62s/it]
 59%|█████▉    | 133067/225000 [267:38:11<138:04:07,  5.41s/it]
 59%|█████▉    | 133068/225000 [267:38:15<132:40:40,  5.20s/it]
 59%|█████▉    | 133069/225000 [267:38:21<138:24:36,  5.42s/it]
 59%|█████▉    | 133070/225000 [267:38:34<191:14:26,  7.49s/it]
 59%|█████▉    | 133071/225000 [267:38:38<168:56:16,  6.62s/it]
 59%|█████▉    | 133072/225000 [267:38:42<144:48:01,  5.67s/it]
 59%|█████▉    | 133073/225000 [267:38:50<166:05:45,  6.50s/it]
 59%|█████▉    | 133074/225000 [267:38:53<139:03:16,  5.45s/it]
 59%|█████▉    | 133075/225000 [267:39:00<150:09:09,  5.88s/it]
 59%|█████▉    | 133076/225000 [267:39:04<132:55:08,  5.21s/it]
 59%|█████▉    | 133077/225000 [267:39:13<161:02:21,  6.31s/it]
 59%|█████▉    | 133078/225000 [267:39:16<142:15:22,  5.57s/it]
 59%|█████▉    | 133079/225000 [267:39:25<162:40:45,  6.37s/it]
 59%|█████▉    | 133080/225000 [267:39:28<141:43:26,  5.55s/it]
 59%|█████▉    | 133081/225000 [267:39:35<148:30:59,  5.82s/it]
 59%|█████▉    | 133082/225000 [267:39:42<158:09:29,  6.19s/it]
 59%|█████▉    | 133083/225000 [267:39:46<143:22:19,  5.62s/it]
 59%|█████▉    | 133084/225000 [267:39:50<132:10:52,  5.18s/it]
 59%|█████▉    | 133085/225000 [267:39:55<129:54:00,  5.09s/it]
 59%|█████▉    | 133086/225000 [267:40:07<181:38:05,  7.11s/it]
 59%|█████▉    | 133087/225000 [267:40:10<153:29:55,  6.01s/it]
 59%|█████▉    | 133088/225000 [267:40:15<145:01:25,  5.68s/it]
 59%|█████▉    | 133089/225000 [267:40:20<138:20:37,  5.42s/it]
 59%|█████▉    | 133090/225000 [267:40:26<141:57:48,  5.56s/it]
 59%|█████▉    | 133091/225000 [267:40:34<158:45:07,  6.22s/it]
 59%|█████▉    | 133092/225000 [267:40:40<162:27:10,  6.36s/it]
 59%|█████▉    | 133093/225000 [267:40:47<163:47:46,  6.42s/it]
 59%|█████▉    | 133094/225000 [267:40:51<142:33:25,  5.58s/it]
 59%|█████▉    | 133095/225000 [267:41:00<173:19:53,  6.79s/it]
 59%|█████▉    | 133096/225000 [267:41:03<144:05:13,  5.64s/it]
 59%|█████▉    | 133097/225000 [267:41:11<157:53:16,  6.18s/it]
 59%|█████▉    | 133098/225000 [267:41:15<144:28:12,  5.66s/it]
 59%|█████▉    | 133099/225000 [267:41:19<133:07:03,  5.21s/it]
 59%|█████▉    | 133100/225000 [267:41:27<149:15:17,  5.85s/it]
                                                               
{'loss': '0.7867', 'grad_norm': '1.04', 'learning_rate': '1.937e-05', 'epoch': '0.5916', 'num_input_tokens_seen': 444586888, 'train_runtime': '9.637e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133100/225000 [267:41:27<149:15:17,  5.85s/it]
 59%|█████▉    | 133101/225000 [267:41:36<173:48:09,  6.81s/it]
 59%|█████▉    | 133102/225000 [267:41:43<175:23:13,  6.87s/it]
 59%|█████▉    | 133103/225000 [267:41:46<150:27:20,  5.89s/it]
 59%|█████▉    | 133104/225000 [267:41:53<160:18:28,  6.28s/it]
 59%|█████▉    | 133105/225000 [267:41:57<138:17:49,  5.42s/it]
 59%|█████▉    | 133106/225000 [267:42:06<164:04:59,  6.43s/it]
 59%|█████▉    | 133107/225000 [267:42:09<143:45:18,  5.63s/it]
 59%|█████▉    | 133108/225000 [267:42:15<141:46:32,  5.55s/it]
 59%|█████▉    | 133109/225000 [267:42:22<152:08:43,  5.96s/it]
 59%|█████▉    | 133110/225000 [267:42:27<149:49:09,  5.87s/it]
 59%|█████▉    | 133111/225000 [267:42:37<178:19:55,  6.99s/it]
 59%|█████▉    | 133112/225000 [267:42:40<151:55:24,  5.95s/it]
 59%|█████▉    | 133113/225000 [267:42:44<134:41:37,  5.28s/it]
 59%|█████▉    | 133114/225000 [267:42:52<155:55:05,  6.11s/it]
 59%|█████▉    | 133115/225000 [267:43:01<179:25:37,  7.03s/it]
 59%|█████▉    | 133116/225000 [267:43:05<156:59:54,  6.15s/it]
 59%|█████▉    | 133117/225000 [267:43:09<139:54:53,  5.48s/it]
 59%|█████▉    | 133118/225000 [267:43:17<156:56:59,  6.15s/it]
 59%|█████▉    | 133119/225000 [267:43:23<155:44:44,  6.10s/it]
 59%|█████▉    | 133120/225000 [267:43:27<135:43:07,  5.32s/it]
 59%|█████▉    | 133121/225000 [267:43:34<149:56:00,  5.87s/it]
 59%|█████▉    | 133122/225000 [267:43:41<163:31:00,  6.41s/it]
 59%|█████▉    | 133123/225000 [267:43:45<145:25:21,  5.70s/it]
 59%|█████▉    | 133124/225000 [267:43:49<130:41:12,  5.12s/it]
 59%|█████▉    | 133125/225000 [267:43:56<146:04:04,  5.72s/it]
 59%|█████▉    | 133126/225000 [267:44:05<166:13:58,  6.51s/it]
 59%|█████▉    | 133127/225000 [267:44:11<166:04:45,  6.51s/it]
 59%|█████▉    | 133128/225000 [267:44:17<162:50:49,  6.38s/it]
 59%|█████▉    | 133129/225000 [267:44:22<152:40:21,  5.98s/it]
 59%|█████▉    | 133130/225000 [267:44:26<137:01:50,  5.37s/it]
 59%|█████▉    | 133131/225000 [267:44:37<174:19:10,  6.83s/it]
 59%|█████▉    | 133132/225000 [267:44:40<149:57:12,  5.88s/it]
 59%|█████▉    | 133133/225000 [267:44:49<171:42:01,  6.73s/it]
 59%|█████▉    | 133134/225000 [267:44:52<146:22:29,  5.74s/it]
 59%|█████▉    | 133135/225000 [267:44:59<151:55:07,  5.95s/it]
 59%|█████▉    | 133136/225000 [267:45:06<161:20:19,  6.32s/it]
 59%|█████▉    | 133137/225000 [267:45:10<141:46:40,  5.56s/it]
 59%|█████▉    | 133138/225000 [267:45:13<126:35:10,  4.96s/it]
 59%|█████▉    | 133139/225000 [267:45:19<130:06:12,  5.10s/it]
 59%|█████▉    | 133140/225000 [267:45:27<156:29:15,  6.13s/it]
 59%|█████▉    | 133141/225000 [267:45:33<152:13:40,  5.97s/it]
 59%|█████▉    | 133142/225000 [267:45:41<168:43:13,  6.61s/it]
 59%|█████▉    | 133143/225000 [267:45:45<152:48:27,  5.99s/it]
 59%|█████▉    | 133144/225000 [267:45:50<142:27:38,  5.58s/it]
 59%|█████▉    | 133145/225000 [267:45:55<139:19:28,  5.46s/it]
 59%|█████▉    | 133146/225000 [267:46:00<134:50:24,  5.28s/it]
 59%|█████▉    | 133147/225000 [267:46:05<131:38:05,  5.16s/it]
 59%|█████▉    | 133148/225000 [267:46:10<131:22:19,  5.15s/it]
 59%|█████▉    | 133149/225000 [267:46:22<184:02:19,  7.21s/it]
 59%|█████▉    | 133150/225000 [267:46:27<165:31:57,  6.49s/it]
                                                               
{'loss': '0.8042', 'grad_norm': '0.9524', 'learning_rate': '1.935e-05', 'epoch': '0.5918', 'num_input_tokens_seen': 444717040, 'train_runtime': '9.64e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133150/225000 [267:46:27<165:31:57,  6.49s/it]
 59%|█████▉    | 133151/225000 [267:46:31<143:42:36,  5.63s/it]
 59%|█████▉    | 133152/225000 [267:46:38<156:06:58,  6.12s/it]
 59%|█████▉    | 133153/225000 [267:46:45<166:47:51,  6.54s/it]
 59%|█████▉    | 133154/225000 [267:46:51<162:27:27,  6.37s/it]
 59%|█████▉    | 133155/225000 [267:46:58<162:32:35,  6.37s/it]
 59%|█████▉    | 133156/225000 [267:47:02<145:34:08,  5.71s/it]
 59%|█████▉    | 133157/225000 [267:47:06<135:04:04,  5.29s/it]
 59%|█████▉    | 133158/225000 [267:47:16<171:03:07,  6.70s/it]
 59%|█████▉    | 133159/225000 [267:47:23<168:19:49,  6.60s/it]
 59%|█████▉    | 133160/225000 [267:47:28<160:27:53,  6.29s/it]
 59%|█████▉    | 133161/225000 [267:47:33<149:06:10,  5.84s/it]
 59%|█████▉    | 133162/225000 [267:47:41<162:38:08,  6.38s/it]
 59%|█████▉    | 133163/225000 [267:47:44<142:22:27,  5.58s/it]
 59%|█████▉    | 133164/225000 [267:47:50<140:48:09,  5.52s/it]
 59%|█████▉    | 133165/225000 [267:48:01<186:59:52,  7.33s/it]
 59%|█████▉    | 133166/225000 [267:48:05<160:46:53,  6.30s/it]
 59%|█████▉    | 133167/225000 [267:48:09<141:00:00,  5.53s/it]
 59%|█████▉    | 133168/225000 [267:48:13<127:39:31,  5.00s/it]
 59%|█████▉    | 133169/225000 [267:48:23<168:05:00,  6.59s/it]
 59%|█████▉    | 133170/225000 [267:48:26<143:38:04,  5.63s/it]
 59%|█████▉    | 133171/225000 [267:48:31<134:03:16,  5.26s/it]
 59%|█████▉    | 133172/225000 [267:48:35<128:15:02,  5.03s/it]
 59%|█████▉    | 133173/225000 [267:48:40<124:15:34,  4.87s/it]
 59%|█████▉    | 133174/225000 [267:48:52<177:44:33,  6.97s/it]
 59%|█████▉    | 133175/225000 [267:48:56<160:48:44,  6.30s/it]
 59%|█████▉    | 133176/225000 [267:49:01<148:32:54,  5.82s/it]
 59%|█████▉    | 133177/225000 [267:49:09<163:28:00,  6.41s/it]
 59%|█████▉    | 133178/225000 [267:49:13<148:17:01,  5.81s/it]
 59%|█████▉    | 133179/225000 [267:49:17<135:52:53,  5.33s/it]
 59%|█████▉    | 133180/225000 [267:49:22<131:14:07,  5.15s/it]
 59%|█████▉    | 133181/225000 [267:49:27<125:58:52,  4.94s/it]
 59%|█████▉    | 133182/225000 [267:49:38<173:09:13,  6.79s/it]
 59%|█████▉    | 133183/225000 [267:49:41<147:44:34,  5.79s/it]
 59%|█████▉    | 133184/225000 [267:49:50<168:03:25,  6.59s/it]
 59%|█████▉    | 133185/225000 [267:49:53<142:14:00,  5.58s/it]
 59%|█████▉    | 133186/225000 [267:49:58<138:04:51,  5.41s/it]
 59%|█████▉    | 133187/225000 [267:50:05<151:23:19,  5.94s/it]
 59%|█████▉    | 133188/225000 [267:50:09<134:57:23,  5.29s/it]
 59%|█████▉    | 133189/225000 [267:50:13<126:42:26,  4.97s/it]
 59%|█████▉    | 133190/225000 [267:50:25<183:34:05,  7.20s/it]
 59%|█████▉    | 133191/225000 [267:50:29<157:21:49,  6.17s/it]
 59%|█████▉    | 133192/225000 [267:50:34<144:41:09,  5.67s/it]
 59%|█████▉    | 133193/225000 [267:50:39<142:40:15,  5.59s/it]
 59%|█████▉    | 133194/225000 [267:50:45<141:12:18,  5.54s/it]
 59%|█████▉    | 133195/225000 [267:50:50<138:07:41,  5.42s/it]
 59%|█████▉    | 133196/225000 [267:50:55<134:16:35,  5.27s/it]
 59%|█████▉    | 133197/225000 [267:51:00<135:28:47,  5.31s/it]
 59%|█████▉    | 133198/225000 [267:51:05<134:43:32,  5.28s/it]
 59%|█████▉    | 133199/225000 [267:51:11<135:50:19,  5.33s/it]
 59%|█████▉    | 133200/225000 [267:51:16<134:12:02,  5.26s/it]
                                                               
{'loss': '0.8142', 'grad_norm': '1.003', 'learning_rate': '1.934e-05', 'epoch': '0.592', 'num_input_tokens_seen': 444845968, 'train_runtime': '9.643e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133200/225000 [267:51:16<134:12:02,  5.26s/it]
 59%|█████▉    | 133201/225000 [267:51:22<138:06:05,  5.42s/it]
 59%|█████▉    | 133202/225000 [267:51:27<137:39:18,  5.40s/it]
 59%|█████▉    | 133203/225000 [267:51:33<144:30:31,  5.67s/it]
 59%|█████▉    | 133204/225000 [267:51:45<190:55:53,  7.49s/it]
 59%|█████▉    | 133205/225000 [267:51:48<161:04:37,  6.32s/it]
 59%|█████▉    | 133206/225000 [267:51:54<155:17:01,  6.09s/it]
 59%|█████▉    | 133207/225000 [267:52:03<178:11:13,  6.99s/it]
 59%|█████▉    | 133208/225000 [267:52:07<152:11:32,  5.97s/it]
 59%|█████▉    | 133209/225000 [267:52:12<149:19:46,  5.86s/it]
 59%|█████▉    | 133210/225000 [267:52:20<166:50:38,  6.54s/it]
 59%|█████▉    | 133211/225000 [267:52:26<156:43:31,  6.15s/it]
 59%|█████▉    | 133212/225000 [267:52:30<144:01:00,  5.65s/it]
 59%|█████▉    | 133213/225000 [267:52:35<137:38:52,  5.40s/it]
 59%|█████▉    | 133214/225000 [267:52:40<135:39:08,  5.32s/it]
 59%|█████▉    | 133215/225000 [267:52:45<129:28:56,  5.08s/it]
 59%|█████▉    | 133216/225000 [267:52:50<130:35:52,  5.12s/it]
 59%|█████▉    | 133217/225000 [267:53:02<187:56:37,  7.37s/it]
 59%|█████▉    | 133218/225000 [267:53:06<158:20:26,  6.21s/it]
 59%|█████▉    | 133219/225000 [267:53:11<145:38:03,  5.71s/it]
 59%|█████▉    | 133220/225000 [267:53:15<137:05:30,  5.38s/it]
 59%|█████▉    | 133221/225000 [267:53:20<132:06:18,  5.18s/it]
 59%|█████▉    | 133222/225000 [267:53:28<158:02:51,  6.20s/it]
 59%|█████▉    | 133223/225000 [267:53:37<179:36:02,  7.04s/it]
 59%|█████▉    | 133224/225000 [267:53:42<164:06:39,  6.44s/it]
 59%|█████▉    | 133225/225000 [267:53:49<161:13:38,  6.32s/it]
 59%|█████▉    | 133226/225000 [267:53:54<156:37:12,  6.14s/it]
 59%|█████▉    | 133227/225000 [267:53:59<146:52:30,  5.76s/it]
 59%|█████▉    | 133228/225000 [267:54:04<142:20:30,  5.58s/it]
 59%|█████▉    | 133229/225000 [267:54:09<136:50:44,  5.37s/it]
 59%|█████▉    | 133230/225000 [267:54:14<134:05:22,  5.26s/it]
 59%|█████▉    | 133231/225000 [267:54:19<131:20:57,  5.15s/it]
 59%|█████▉    | 133232/225000 [267:54:24<130:49:16,  5.13s/it]
 59%|█████▉    | 133233/225000 [267:54:29<130:27:27,  5.12s/it]
 59%|█████▉    | 133234/225000 [267:54:34<130:49:34,  5.13s/it]
 59%|█████▉    | 133235/225000 [267:54:40<131:28:25,  5.16s/it]
 59%|█████▉    | 133236/225000 [267:54:50<175:10:37,  6.87s/it]
 59%|█████▉    | 133237/225000 [267:54:59<189:33:56,  7.44s/it]
 59%|█████▉    | 133238/225000 [267:55:03<160:05:16,  6.28s/it]
 59%|█████▉    | 133239/225000 [267:55:07<144:13:39,  5.66s/it]
 59%|█████▉    | 133240/225000 [267:55:13<146:48:17,  5.76s/it]
 59%|█████▉    | 133241/225000 [267:55:23<179:32:22,  7.04s/it]
 59%|█████▉    | 133242/225000 [267:55:27<158:17:53,  6.21s/it]
 59%|█████▉    | 133243/225000 [267:55:32<147:54:39,  5.80s/it]
 59%|█████▉    | 133244/225000 [267:55:37<141:56:30,  5.57s/it]
 59%|█████▉    | 133245/225000 [267:55:42<138:22:13,  5.43s/it]
 59%|█████▉    | 133246/225000 [267:55:47<131:55:52,  5.18s/it]
 59%|█████▉    | 133247/225000 [267:55:52<130:51:58,  5.13s/it]
 59%|█████▉    | 133248/225000 [267:55:57<130:03:21,  5.10s/it]
 59%|█████▉    | 133249/225000 [267:56:02<129:17:47,  5.07s/it]
 59%|█████▉    | 133250/225000 [267:56:13<171:23:30,  6.72s/it]
                                                               
{'loss': '0.8168', 'grad_norm': '1.111', 'learning_rate': '1.932e-05', 'epoch': '0.5922', 'num_input_tokens_seen': 444975728, 'train_runtime': '9.646e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133250/225000 [267:56:13<171:23:30,  6.72s/it]
 59%|█████▉    | 133251/225000 [267:56:22<189:53:33,  7.45s/it]
 59%|█████▉    | 133252/225000 [267:56:25<158:07:44,  6.20s/it]
 59%|█████▉    | 133253/225000 [267:56:34<176:04:30,  6.91s/it]
 59%|█████▉    | 133254/225000 [267:56:37<153:01:21,  6.00s/it]
 59%|█████▉    | 133255/225000 [267:56:42<140:49:00,  5.53s/it]
 59%|█████▉    | 133256/225000 [267:56:50<164:32:06,  6.46s/it]
 59%|█████▉    | 133257/225000 [267:56:55<146:43:06,  5.76s/it]
 59%|█████▉    | 133258/225000 [267:56:58<130:54:48,  5.14s/it]
 59%|█████▉    | 133259/225000 [267:57:03<124:25:23,  4.88s/it]
 59%|█████▉    | 133260/225000 [267:57:15<183:01:56,  7.18s/it]
 59%|█████▉    | 133261/225000 [267:57:20<162:12:56,  6.37s/it]
 59%|█████▉    | 133262/225000 [267:57:25<156:32:15,  6.14s/it]
 59%|█████▉    | 133263/225000 [267:57:33<165:22:49,  6.49s/it]
 59%|█████▉    | 133264/225000 [267:57:36<145:42:45,  5.72s/it]
 59%|█████▉    | 133265/225000 [267:57:41<139:43:22,  5.48s/it]
 59%|█████▉    | 133266/225000 [267:57:46<132:10:12,  5.19s/it]
 59%|█████▉    | 133267/225000 [267:57:51<128:48:05,  5.05s/it]
 59%|█████▉    | 133268/225000 [267:57:55<126:16:19,  4.96s/it]
 59%|█████▉    | 133269/225000 [267:58:03<150:45:57,  5.92s/it]
 59%|█████▉    | 133270/225000 [267:58:13<178:28:21,  7.00s/it]
 59%|█████▉    | 133271/225000 [267:58:17<152:31:15,  5.99s/it]
 59%|█████▉    | 133272/225000 [267:58:20<135:51:53,  5.33s/it]
 59%|█████▉    | 133273/225000 [267:58:31<173:08:26,  6.80s/it]
 59%|█████▉    | 133274/225000 [267:58:35<152:19:00,  5.98s/it]
 59%|█████▉    | 133275/225000 [267:58:39<136:46:34,  5.37s/it]
 59%|█████▉    | 133276/225000 [267:58:48<164:34:22,  6.46s/it]
 59%|█████▉    | 133277/225000 [267:58:51<140:48:18,  5.53s/it]
 59%|█████▉    | 133278/225000 [267:58:59<158:25:19,  6.22s/it]
 59%|█████▉    | 133279/225000 [267:59:04<151:24:27,  5.94s/it]
 59%|█████▉    | 133280/225000 [267:59:09<141:22:33,  5.55s/it]
 59%|█████▉    | 133281/225000 [267:59:13<134:17:44,  5.27s/it]
 59%|█████▉    | 133282/225000 [267:59:18<128:01:52,  5.03s/it]
 59%|█████▉    | 133283/225000 [267:59:23<128:18:52,  5.04s/it]
 59%|█████▉    | 133284/225000 [267:59:28<131:01:26,  5.14s/it]
 59%|█████▉    | 133285/225000 [267:59:36<152:06:34,  5.97s/it]
 59%|█████▉    | 133286/225000 [267:59:45<169:48:23,  6.67s/it]
 59%|█████▉    | 133287/225000 [267:59:49<149:57:43,  5.89s/it]
 59%|█████▉    | 133288/225000 [267:59:57<166:52:02,  6.55s/it]
 59%|█████▉    | 133289/225000 [268:00:02<155:36:16,  6.11s/it]
 59%|█████▉    | 133290/225000 [268:00:07<147:39:09,  5.80s/it]
 59%|█████▉    | 133291/225000 [268:00:13<150:08:05,  5.89s/it]
 59%|█████▉    | 133292/225000 [268:00:21<163:32:14,  6.42s/it]
 59%|█████▉    | 133293/225000 [268:00:26<156:28:52,  6.14s/it]
 59%|█████▉    | 133294/225000 [268:00:31<147:53:34,  5.81s/it]
 59%|█████▉    | 133295/225000 [268:00:36<144:15:03,  5.66s/it]
 59%|█████▉    | 133296/225000 [268:00:41<138:17:20,  5.43s/it]
 59%|█████▉    | 133297/225000 [268:00:46<133:16:49,  5.23s/it]
 59%|█████▉    | 133298/225000 [268:00:51<128:55:57,  5.06s/it]
 59%|█████▉    | 133299/225000 [268:00:55<125:48:44,  4.94s/it]
 59%|█████▉    | 133300/225000 [268:01:00<125:07:03,  4.91s/it]
                                                               
{'loss': '0.807', 'grad_norm': '1.099', 'learning_rate': '1.93e-05', 'epoch': '0.5924', 'num_input_tokens_seen': 445105640, 'train_runtime': '9.649e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133300/225000 [268:01:00<125:07:03,  4.91s/it]
 59%|█████▉    | 133301/225000 [268:01:06<133:57:33,  5.26s/it]
 59%|█████▉    | 133302/225000 [268:01:19<192:46:33,  7.57s/it]
 59%|█████▉    | 133303/225000 [268:01:24<169:49:04,  6.67s/it]
 59%|█████▉    | 133304/225000 [268:01:28<148:42:04,  5.84s/it]
 59%|█████▉    | 133305/225000 [268:01:37<176:16:07,  6.92s/it]
 59%|█████▉    | 133306/225000 [268:01:43<164:11:37,  6.45s/it]
 59%|█████▉    | 133307/225000 [268:01:47<148:26:30,  5.83s/it]
 59%|█████▉    | 133308/225000 [268:01:51<135:12:51,  5.31s/it]
 59%|█████▉    | 133309/225000 [268:02:04<191:40:29,  7.53s/it]
 59%|█████▉    | 133310/225000 [268:02:08<169:51:45,  6.67s/it]
 59%|█████▉    | 133311/225000 [268:02:13<152:02:59,  5.97s/it]
 59%|█████▉    | 133312/225000 [268:02:17<141:05:31,  5.54s/it]
 59%|█████▉    | 133313/225000 [268:02:22<135:36:54,  5.32s/it]
 59%|█████▉    | 133314/225000 [268:02:26<127:45:56,  5.02s/it]
 59%|█████▉    | 133315/225000 [268:02:31<121:53:56,  4.79s/it]
 59%|█████▉    | 133316/225000 [268:02:35<122:04:03,  4.79s/it]
 59%|█████▉    | 133317/225000 [268:02:45<158:06:41,  6.21s/it]
 59%|█████▉    | 133318/225000 [268:02:52<168:04:47,  6.60s/it]
 59%|█████▉    | 133319/225000 [268:02:58<162:03:54,  6.36s/it]
 59%|█████▉    | 133320/225000 [268:03:04<159:00:01,  6.24s/it]
 59%|█████▉    | 133321/225000 [268:03:10<151:31:31,  5.95s/it]
 59%|█████▉    | 133322/225000 [268:03:18<169:13:05,  6.64s/it]
 59%|█████▉    | 133323/225000 [268:03:21<145:16:09,  5.70s/it]
 59%|█████▉    | 133324/225000 [268:03:30<168:43:09,  6.63s/it]
 59%|█████▉    | 133325/225000 [268:03:35<158:44:01,  6.23s/it]
 59%|█████▉    | 133326/225000 [268:03:40<147:04:13,  5.78s/it]
 59%|█████▉    | 133327/225000 [268:03:45<137:21:56,  5.39s/it]
 59%|█████▉    | 133328/225000 [268:03:49<131:23:33,  5.16s/it]
 59%|█████▉    | 133329/225000 [268:03:54<132:11:15,  5.19s/it]
 59%|█████▉    | 133330/225000 [268:03:59<130:50:58,  5.14s/it]
 59%|█████▉    | 133331/225000 [268:04:05<131:31:49,  5.17s/it]
 59%|█████▉    | 133332/225000 [268:04:12<149:38:21,  5.88s/it]
 59%|█████▉    | 133333/225000 [268:04:24<191:01:57,  7.50s/it]
 59%|█████▉    | 133334/225000 [268:04:27<158:57:44,  6.24s/it]
 59%|█████▉    | 133335/225000 [268:04:31<142:33:41,  5.60s/it]
 59%|█████▉    | 133336/225000 [268:04:36<135:31:47,  5.32s/it]
 59%|█████▉    | 133337/225000 [268:04:40<127:58:54,  5.03s/it]
 59%|█████▉    | 133338/225000 [268:04:44<123:26:27,  4.85s/it]
 59%|█████▉    | 133339/225000 [268:04:54<160:32:52,  6.31s/it]
 59%|█████▉    | 133340/225000 [268:05:02<174:54:36,  6.87s/it]
 59%|█████▉    | 133341/225000 [268:05:09<176:10:06,  6.92s/it]
 59%|█████▉    | 133342/225000 [268:05:13<149:48:08,  5.88s/it]
 59%|█████▉    | 133343/225000 [268:05:17<134:41:44,  5.29s/it]
 59%|█████▉    | 133344/225000 [268:05:26<164:00:30,  6.44s/it]
 59%|█████▉    | 133345/225000 [268:05:30<146:04:03,  5.74s/it]
 59%|█████▉    | 133346/225000 [268:05:35<138:31:43,  5.44s/it]
 59%|█████▉    | 133347/225000 [268:05:46<184:05:01,  7.23s/it]
 59%|█████▉    | 133348/225000 [268:05:50<158:45:48,  6.24s/it]
 59%|█████▉    | 133349/225000 [268:05:57<166:29:30,  6.54s/it]
 59%|█████▉    | 133350/225000 [268:06:03<163:30:53,  6.42s/it]
                                                               
{'loss': '0.8219', 'grad_norm': '1.136', 'learning_rate': '1.928e-05', 'epoch': '0.5927', 'num_input_tokens_seen': 445236144, 'train_runtime': '9.652e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133350/225000 [268:06:03<163:30:53,  6.42s/it]
 59%|█████▉    | 133351/225000 [268:06:07<142:49:24,  5.61s/it]
 59%|█████▉    | 133352/225000 [268:06:12<139:29:08,  5.48s/it]
 59%|█████▉    | 133353/225000 [268:06:18<139:40:44,  5.49s/it]
 59%|█████▉    | 133354/225000 [268:06:24<141:41:44,  5.57s/it]
 59%|█████▉    | 133355/225000 [268:06:29<140:48:34,  5.53s/it]
 59%|█████▉    | 133356/225000 [268:06:35<142:15:00,  5.59s/it]
 59%|█████▉    | 133357/225000 [268:06:40<139:01:01,  5.46s/it]
 59%|█████▉    | 133358/225000 [268:06:45<138:11:29,  5.43s/it]
 59%|█████▉    | 133359/225000 [268:06:53<152:43:19,  6.00s/it]
 59%|█████▉    | 133360/225000 [268:07:03<184:35:34,  7.25s/it]
 59%|█████▉    | 133361/225000 [268:07:06<157:00:53,  6.17s/it]
 59%|█████▉    | 133362/225000 [268:07:11<145:07:30,  5.70s/it]
 59%|█████▉    | 133363/225000 [268:07:21<178:57:53,  7.03s/it]
 59%|█████▉    | 133364/225000 [268:07:25<152:21:02,  5.99s/it]
 59%|█████▉    | 133365/225000 [268:07:32<164:51:19,  6.48s/it]
 59%|█████▉    | 133366/225000 [268:07:40<170:51:18,  6.71s/it]
 59%|█████▉    | 133367/225000 [268:07:44<156:27:06,  6.15s/it]
 59%|█████▉    | 133368/225000 [268:07:49<141:26:43,  5.56s/it]
 59%|█████▉    | 133369/225000 [268:07:52<127:08:59,  5.00s/it]
 59%|█████▉    | 133370/225000 [268:07:59<141:26:23,  5.56s/it]
 59%|█████▉    | 133371/225000 [268:08:06<154:29:58,  6.07s/it]
 59%|█████▉    | 133372/225000 [268:08:10<134:08:58,  5.27s/it]
 59%|█████▉    | 133373/225000 [268:08:15<133:35:05,  5.25s/it]
 59%|█████▉    | 133374/225000 [268:08:20<129:52:40,  5.10s/it]
 59%|█████▉    | 133375/225000 [268:08:26<142:35:30,  5.60s/it]
 59%|█████▉    | 133376/225000 [268:08:38<189:52:12,  7.46s/it]
 59%|█████▉    | 133377/225000 [268:08:42<159:16:41,  6.26s/it]
 59%|█████▉    | 133378/225000 [268:08:48<155:44:48,  6.12s/it]
 59%|█████▉    | 133379/225000 [268:08:54<158:09:08,  6.21s/it]
 59%|█████▉    | 133380/225000 [268:08:58<145:10:35,  5.70s/it]
 59%|█████▉    | 133381/225000 [268:09:03<133:23:16,  5.24s/it]
 59%|█████▉    | 133382/225000 [268:09:07<124:46:44,  4.90s/it]
 59%|█████▉    | 133383/225000 [268:09:15<152:46:01,  6.00s/it]
 59%|█████▉    | 133384/225000 [268:09:23<167:06:44,  6.57s/it]
 59%|█████▉    | 133385/225000 [268:09:27<147:19:35,  5.79s/it]
 59%|█████▉    | 133386/225000 [268:09:31<131:07:16,  5.15s/it]
 59%|█████▉    | 133387/225000 [268:09:35<123:44:51,  4.86s/it]
 59%|█████▉    | 133388/225000 [268:09:42<139:20:22,  5.48s/it]
 59%|█████▉    | 133389/225000 [268:09:54<191:54:52,  7.54s/it]
 59%|█████▉    | 133390/225000 [268:09:58<163:21:27,  6.42s/it]
 59%|█████▉    | 133391/225000 [268:10:02<146:30:30,  5.76s/it]
 59%|█████▉    | 133392/225000 [268:10:13<182:09:56,  7.16s/it]
 59%|█████▉    | 133393/225000 [268:10:16<154:56:37,  6.09s/it]
 59%|█████▉    | 133394/225000 [268:10:24<168:50:45,  6.64s/it]
 59%|█████▉    | 133395/225000 [268:10:28<148:59:23,  5.86s/it]
 59%|█████▉    | 133396/225000 [268:10:32<134:17:31,  5.28s/it]
 59%|█████▉    | 133397/225000 [268:10:36<126:13:24,  4.96s/it]
 59%|█████▉    | 133398/225000 [268:10:43<139:22:39,  5.48s/it]
 59%|█████▉    | 133399/225000 [268:10:51<157:37:57,  6.20s/it]
 59%|█████▉    | 133400/225000 [268:11:00<175:22:57,  6.89s/it]
                                                               
{'loss': '0.8105', 'grad_norm': '0.968', 'learning_rate': '1.927e-05', 'epoch': '0.5929', 'num_input_tokens_seen': 445367096, 'train_runtime': '9.655e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133400/225000 [268:11:00<175:22:57,  6.89s/it]
 59%|█████▉    | 133401/225000 [268:11:03<150:29:18,  5.91s/it]
 59%|█████▉    | 133402/225000 [268:11:06<130:48:07,  5.14s/it]
 59%|█████▉    | 133403/225000 [268:11:16<160:27:56,  6.31s/it]
 59%|█████▉    | 133404/225000 [268:11:19<139:35:32,  5.49s/it]
 59%|█████▉    | 133405/225000 [268:11:27<161:24:51,  6.34s/it]
 59%|█████▉    | 133406/225000 [268:11:33<152:20:56,  5.99s/it]
 59%|█████▉    | 133407/225000 [268:11:40<166:01:14,  6.53s/it]
 59%|█████▉    | 133408/225000 [268:11:44<141:54:11,  5.58s/it]
 59%|█████▉    | 133409/225000 [268:11:48<132:53:25,  5.22s/it]
 59%|█████▉    | 133410/225000 [268:11:53<128:44:24,  5.06s/it]
 59%|█████▉    | 133411/225000 [268:11:57<124:47:37,  4.91s/it]
 59%|█████▉    | 133412/225000 [268:12:05<147:27:38,  5.80s/it]
 59%|█████▉    | 133413/225000 [268:12:13<166:17:18,  6.54s/it]
 59%|█████▉    | 133414/225000 [268:12:17<145:16:32,  5.71s/it]
 59%|█████▉    | 133415/225000 [268:12:24<151:57:42,  5.97s/it]
 59%|█████▉    | 133416/225000 [268:12:32<169:10:05,  6.65s/it]
 59%|█████▉    | 133417/225000 [268:12:36<151:32:00,  5.96s/it]
 59%|█████▉    | 133418/225000 [268:12:41<140:18:57,  5.52s/it]
 59%|█████▉    | 133419/225000 [268:12:45<132:18:17,  5.20s/it]
 59%|█████▉    | 133420/225000 [268:12:50<129:34:52,  5.09s/it]
 59%|█████▉    | 133421/225000 [268:12:55<126:54:48,  4.99s/it]
 59%|█████▉    | 133422/225000 [268:13:07<184:30:04,  7.25s/it]
 59%|█████▉    | 133423/225000 [268:13:13<172:14:17,  6.77s/it]
 59%|█████▉    | 133424/225000 [268:13:18<156:03:53,  6.14s/it]
 59%|█████▉    | 133425/225000 [268:13:22<142:23:40,  5.60s/it]
 59%|█████▉    | 133426/225000 [268:13:27<139:38:31,  5.49s/it]
 59%|█████▉    | 133427/225000 [268:13:38<181:16:05,  7.13s/it]
 59%|█████▉    | 133428/225000 [268:13:43<159:50:43,  6.28s/it]
 59%|█████▉    | 133429/225000 [268:13:47<144:17:45,  5.67s/it]
 59%|█████▉    | 133430/225000 [268:13:52<136:16:11,  5.36s/it]
 59%|█████▉    | 133431/225000 [268:13:56<131:30:31,  5.17s/it]
 59%|█████▉    | 133432/225000 [268:14:01<129:03:45,  5.07s/it]
 59%|█████▉    | 133433/225000 [268:14:11<162:49:55,  6.40s/it]
 59%|█████▉    | 133434/225000 [268:14:18<172:02:42,  6.76s/it]
 59%|█████▉    | 133435/225000 [268:14:22<146:04:46,  5.74s/it]
 59%|█████▉    | 133436/225000 [268:14:31<170:34:40,  6.71s/it]
 59%|█████▉    | 133437/225000 [268:14:35<156:48:08,  6.17s/it]
 59%|█████▉    | 133438/225000 [268:14:41<149:24:53,  5.87s/it]
 59%|█████▉    | 133439/225000 [268:14:45<141:51:09,  5.58s/it]
 59%|█████▉    | 133440/225000 [268:14:51<139:01:40,  5.47s/it]
 59%|█████▉    | 133441/225000 [268:14:55<132:58:48,  5.23s/it]
 59%|█████▉    | 133442/225000 [268:15:01<136:54:08,  5.38s/it]
 59%|█████▉    | 133443/225000 [268:15:13<186:01:37,  7.31s/it]
 59%|█████▉    | 133444/225000 [268:15:16<156:33:37,  6.16s/it]
 59%|█████▉    | 133445/225000 [268:15:22<149:48:17,  5.89s/it]
 59%|█████▉    | 133446/225000 [268:15:30<170:49:07,  6.72s/it]
 59%|█████▉    | 133447/225000 [268:15:35<157:23:47,  6.19s/it]
 59%|█████▉    | 133448/225000 [268:15:43<169:04:14,  6.65s/it]
 59%|█████▉    | 133449/225000 [268:15:48<154:07:48,  6.06s/it]
 59%|█████▉    | 133450/225000 [268:15:53<146:58:12,  5.78s/it]
                                                               
{'loss': '0.8144', 'grad_norm': '1.159', 'learning_rate': '1.925e-05', 'epoch': '0.5931', 'num_input_tokens_seen': 445497632, 'train_runtime': '9.658e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133450/225000 [268:15:53<146:58:12,  5.78s/it]
 59%|█████▉    | 133451/225000 [268:16:01<163:53:11,  6.44s/it]
 59%|█████▉    | 133452/225000 [268:16:05<150:23:25,  5.91s/it]
 59%|█████▉    | 133453/225000 [268:16:10<141:37:03,  5.57s/it]
 59%|█████▉    | 133454/225000 [268:16:14<131:37:23,  5.18s/it]
 59%|█████▉    | 133455/225000 [268:16:19<123:02:52,  4.84s/it]
 59%|█████▉    | 133456/225000 [268:16:30<172:32:30,  6.79s/it]
 59%|█████▉    | 133457/225000 [268:16:35<161:54:12,  6.37s/it]
 59%|█████▉    | 133458/225000 [268:16:39<138:31:38,  5.45s/it]
 59%|█████▉    | 133459/225000 [268:16:47<160:53:49,  6.33s/it]
 59%|█████▉    | 133460/225000 [268:16:51<144:40:33,  5.69s/it]
 59%|█████▉    | 133461/225000 [268:16:56<135:59:54,  5.35s/it]
 59%|█████▉    | 133462/225000 [268:17:00<129:15:23,  5.08s/it]
 59%|█████▉    | 133463/225000 [268:17:04<123:26:05,  4.85s/it]
 59%|█████▉    | 133464/225000 [268:17:17<182:22:36,  7.17s/it]
 59%|█████▉    | 133465/225000 [268:17:22<165:40:03,  6.52s/it]
 59%|█████▉    | 133466/225000 [268:17:27<152:01:07,  5.98s/it]
 59%|█████▉    | 133467/225000 [268:17:31<139:57:15,  5.50s/it]
 59%|█████▉    | 133468/225000 [268:17:36<135:48:13,  5.34s/it]
 59%|█████▉    | 133469/225000 [268:17:41<133:25:21,  5.25s/it]
 59%|█████▉    | 133470/225000 [268:17:46<130:24:40,  5.13s/it]
 59%|█████▉    | 133471/225000 [268:17:51<132:03:33,  5.19s/it]
 59%|█████▉    | 133472/225000 [268:18:01<166:18:53,  6.54s/it]
 59%|█████▉    | 133473/225000 [268:18:09<174:14:08,  6.85s/it]
 59%|█████▉    | 133474/225000 [268:18:16<175:24:06,  6.90s/it]
 59%|█████▉    | 133475/225000 [268:18:22<169:59:56,  6.69s/it]
 59%|█████▉    | 133476/225000 [268:18:26<148:18:41,  5.83s/it]
 59%|█████▉    | 133477/225000 [268:18:32<148:31:52,  5.84s/it]
 59%|█████▉    | 133478/225000 [268:18:40<169:26:09,  6.66s/it]
 59%|█████▉    | 133479/225000 [268:18:44<147:06:31,  5.79s/it]
 59%|█████▉    | 133480/225000 [268:18:52<163:18:35,  6.42s/it]
 59%|█████▉    | 133481/225000 [268:18:56<146:09:56,  5.75s/it]
 59%|█████▉    | 133482/225000 [268:19:00<134:44:06,  5.30s/it]
 59%|█████▉    | 133483/225000 [268:19:04<126:39:39,  4.98s/it]
 59%|█████▉    | 133484/225000 [268:19:10<130:46:39,  5.14s/it]
 59%|█████▉    | 133485/225000 [268:19:22<179:57:10,  7.08s/it]
 59%|█████▉    | 133486/225000 [268:19:26<161:15:34,  6.34s/it]
 59%|█████▉    | 133487/225000 [268:19:30<140:13:51,  5.52s/it]
 59%|█████▉    | 133488/225000 [268:19:34<132:42:02,  5.22s/it]
 59%|█████▉    | 133489/225000 [268:19:40<134:43:31,  5.30s/it]
 59%|█████▉    | 133490/225000 [268:19:45<134:19:07,  5.28s/it]
 59%|█████▉    | 133491/225000 [268:19:51<136:58:37,  5.39s/it]
 59%|█████▉    | 133492/225000 [268:19:56<135:40:32,  5.34s/it]
 59%|█████▉    | 133493/225000 [268:20:01<135:38:16,  5.34s/it]
 59%|█████▉    | 133494/225000 [268:20:07<142:09:16,  5.59s/it]
 59%|█████▉    | 133495/225000 [268:20:20<199:08:02,  7.83s/it]
 59%|█████▉    | 133496/225000 [268:20:25<172:01:22,  6.77s/it]
 59%|█████▉    | 133497/225000 [268:20:31<170:45:53,  6.72s/it]
 59%|█████▉    | 133498/225000 [268:20:36<153:59:50,  6.06s/it]
 59%|█████▉    | 133499/225000 [268:20:40<140:48:00,  5.54s/it]
 59%|█████▉    | 133500/225000 [268:20:45<135:42:54,  5.34s/it]
                                                               
{'loss': '0.8094', 'grad_norm': '1.053', 'learning_rate': '1.923e-05', 'epoch': '0.5933', 'num_input_tokens_seen': 445627392, 'train_runtime': '9.66e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133500/225000 [268:20:45<135:42:54,  5.34s/it]
 59%|█████▉    | 133501/225000 [268:20:49<127:46:36,  5.03s/it]
 59%|█████▉    | 133502/225000 [268:20:57<144:01:59,  5.67s/it]
 59%|█████▉    | 133503/225000 [268:21:08<191:45:36,  7.54s/it]
 59%|█████▉    | 133504/225000 [268:21:12<163:11:36,  6.42s/it]
 59%|█████▉    | 133505/225000 [268:21:16<144:01:48,  5.67s/it]
 59%|█████▉    | 133506/225000 [268:21:28<188:34:25,  7.42s/it]
 59%|█████▉    | 133507/225000 [268:21:33<171:35:36,  6.75s/it]
 59%|█████▉    | 133508/225000 [268:21:38<159:46:15,  6.29s/it]
 59%|█████▉    | 133509/225000 [268:21:42<140:02:38,  5.51s/it]
 59%|█████▉    | 133510/225000 [268:21:52<178:43:15,  7.03s/it]
 59%|█████▉    | 133511/225000 [268:21:56<151:54:55,  5.98s/it]
 59%|█████▉    | 133512/225000 [268:22:00<134:26:44,  5.29s/it]
 59%|█████▉    | 133513/225000 [268:22:09<165:20:24,  6.51s/it]
 59%|█████▉    | 133514/225000 [268:22:12<142:28:40,  5.61s/it]
 59%|█████▉    | 133515/225000 [268:22:17<131:47:25,  5.19s/it]
 59%|█████▉    | 133516/225000 [268:22:21<125:29:23,  4.94s/it]
 59%|█████▉    | 133517/225000 [268:22:31<161:18:08,  6.35s/it]
 59%|█████▉    | 133518/225000 [268:22:35<148:58:22,  5.86s/it]
 59%|█████▉    | 133519/225000 [268:22:44<168:28:03,  6.63s/it]
 59%|█████▉    | 133520/225000 [268:22:47<144:03:39,  5.67s/it]
 59%|█████▉    | 133521/225000 [268:22:51<129:41:33,  5.10s/it]
 59%|█████▉    | 133522/225000 [268:22:55<124:11:54,  4.89s/it]
 59%|█████▉    | 133523/225000 [268:23:00<123:48:53,  4.87s/it]
 59%|█████▉    | 133524/225000 [268:23:10<161:28:37,  6.35s/it]
 59%|█████▉    | 133525/225000 [268:23:18<176:40:21,  6.95s/it]
 59%|█████▉    | 133526/225000 [268:23:22<152:37:33,  6.01s/it]
 59%|█████▉    | 133527/225000 [268:23:28<148:23:25,  5.84s/it]
 59%|█████▉    | 133528/225000 [268:23:33<144:47:41,  5.70s/it]
 59%|█████▉    | 133529/225000 [268:23:39<144:21:58,  5.68s/it]
 59%|█████▉    | 133530/225000 [268:23:44<140:29:51,  5.53s/it]
 59%|█████▉    | 133531/225000 [268:23:49<138:16:10,  5.44s/it]
 59%|█████▉    | 133532/225000 [268:23:54<137:20:22,  5.41s/it]
 59%|█████▉    | 133533/225000 [268:24:00<137:56:38,  5.43s/it]
 59%|█████▉    | 133534/225000 [268:24:05<138:17:39,  5.44s/it]
 59%|█████▉    | 133535/225000 [268:24:11<138:25:28,  5.45s/it]
 59%|█████▉    | 133536/225000 [268:24:18<148:39:40,  5.85s/it]
 59%|█████▉    | 133537/225000 [268:24:29<190:30:51,  7.50s/it]
 59%|█████▉    | 133538/225000 [268:24:33<162:25:10,  6.39s/it]
 59%|█████▉    | 133539/225000 [268:24:40<170:56:01,  6.73s/it]
 59%|█████▉    | 133540/225000 [268:24:44<147:52:40,  5.82s/it]
 59%|█████▉    | 133541/225000 [268:24:53<175:14:31,  6.90s/it]
 59%|█████▉    | 133542/225000 [268:25:00<172:43:01,  6.80s/it]
 59%|█████▉    | 133543/225000 [268:25:05<156:10:48,  6.15s/it]
 59%|█████▉    | 133544/225000 [268:25:08<135:25:13,  5.33s/it]
 59%|█████▉    | 133545/225000 [268:25:16<155:01:14,  6.10s/it]
 59%|█████▉    | 133546/225000 [268:25:23<164:18:41,  6.47s/it]
 59%|█████▉    | 133547/225000 [268:25:28<154:25:10,  6.08s/it]
 59%|█████▉    | 133548/225000 [268:25:32<137:36:17,  5.42s/it]
 59%|█████▉    | 133549/225000 [268:25:38<138:35:21,  5.46s/it]
 59%|█████▉    | 133550/225000 [268:25:46<162:53:21,  6.41s/it]
                                                               
{'loss': '0.8126', 'grad_norm': '1.174', 'learning_rate': '1.921e-05', 'epoch': '0.5936', 'num_input_tokens_seen': 445756432, 'train_runtime': '9.663e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133550/225000 [268:25:46<162:53:21,  6.41s/it]
 59%|█████▉    | 133551/225000 [268:25:51<146:37:55,  5.77s/it]
 59%|█████▉    | 133552/225000 [268:25:55<136:31:25,  5.37s/it]
 59%|█████▉    | 133553/225000 [268:26:00<131:41:12,  5.18s/it]
 59%|█████▉    | 133554/225000 [268:26:05<129:47:14,  5.11s/it]
 59%|█████▉    | 133555/225000 [268:26:13<151:04:33,  5.95s/it]
 59%|█████▉    | 133556/225000 [268:26:23<184:02:06,  7.25s/it]
 59%|█████▉    | 133557/225000 [268:26:27<157:30:42,  6.20s/it]
 59%|█████▉    | 133558/225000 [268:26:30<138:35:10,  5.46s/it]
 59%|█████▉    | 133559/225000 [268:26:42<184:55:30,  7.28s/it]
 59%|█████▉    | 133560/225000 [268:26:46<156:22:55,  6.16s/it]
 59%|█████▉    | 133561/225000 [268:26:50<139:48:16,  5.50s/it]
 59%|█████▉    | 133562/225000 [268:26:56<144:40:59,  5.70s/it]
 59%|█████▉    | 133563/225000 [268:27:05<169:21:42,  6.67s/it]
 59%|█████▉    | 133564/225000 [268:27:08<146:50:15,  5.78s/it]
 59%|█████▉    | 133565/225000 [268:27:17<170:09:47,  6.70s/it]
 59%|█████▉    | 133566/225000 [268:27:21<150:02:40,  5.91s/it]
 59%|█████▉    | 133567/225000 [268:27:25<136:41:17,  5.38s/it]
 59%|█████▉    | 133568/225000 [268:27:35<167:08:09,  6.58s/it]
 59%|█████▉    | 133569/225000 [268:27:38<142:37:42,  5.62s/it]
 59%|█████▉    | 133570/225000 [268:27:42<131:10:20,  5.16s/it]
 59%|█████▉    | 133571/225000 [268:27:53<171:58:28,  6.77s/it]
 59%|█████▉    | 133572/225000 [268:27:56<148:20:15,  5.84s/it]
 59%|█████▉    | 133573/225000 [268:28:00<129:05:12,  5.08s/it]
 59%|█████▉    | 133574/225000 [268:28:10<168:28:42,  6.63s/it]
 59%|█████▉    | 133575/225000 [268:28:14<148:18:09,  5.84s/it]
 59%|█████▉    | 133576/225000 [268:28:18<133:56:08,  5.27s/it]
 59%|█████▉    | 133577/225000 [268:28:27<166:36:30,  6.56s/it]
 59%|█████▉    | 133578/225000 [268:28:33<159:09:41,  6.27s/it]
 59%|█████▉    | 133579/225000 [268:28:37<144:31:11,  5.69s/it]
 59%|█████▉    | 133580/225000 [268:28:42<133:07:28,  5.24s/it]
 59%|█████▉    | 133581/225000 [268:28:46<123:35:57,  4.87s/it]
 59%|█████▉    | 133582/225000 [268:28:56<163:40:57,  6.45s/it]
 59%|█████▉    | 133583/225000 [268:29:02<163:51:28,  6.45s/it]
 59%|█████▉    | 133584/225000 [268:29:06<141:42:50,  5.58s/it]
 59%|█████▉    | 133585/225000 [268:29:14<162:39:20,  6.41s/it]
 59%|█████▉    | 133586/225000 [268:29:21<165:00:51,  6.50s/it]
 59%|█████▉    | 133587/225000 [268:29:24<141:42:44,  5.58s/it]
 59%|█████▉    | 133588/225000 [268:29:32<158:53:28,  6.26s/it]
 59%|█████▉    | 133589/225000 [268:29:36<141:25:54,  5.57s/it]
 59%|█████▉    | 133590/225000 [268:29:41<133:00:56,  5.24s/it]
 59%|█████▉    | 133591/225000 [268:29:45<126:08:03,  4.97s/it]
 59%|█████▉    | 133592/225000 [268:29:56<174:02:25,  6.85s/it]
 59%|█████▉    | 133593/225000 [268:30:01<158:22:02,  6.24s/it]
 59%|█████▉    | 133594/225000 [268:30:09<174:45:38,  6.88s/it]
 59%|█████▉    | 133595/225000 [268:30:13<147:46:14,  5.82s/it]
 59%|█████▉    | 133596/225000 [268:30:21<166:38:39,  6.56s/it]
 59%|█████▉    | 133597/225000 [268:30:24<141:59:23,  5.59s/it]
 59%|█████▉    | 133598/225000 [268:30:33<163:24:06,  6.44s/it]
 59%|█████▉    | 133599/225000 [268:30:36<140:09:21,  5.52s/it]
 59%|█████▉    | 133600/225000 [268:30:42<140:41:07,  5.54s/it]
                                                               
{'loss': '0.7989', 'grad_norm': '1.055', 'learning_rate': '1.919e-05', 'epoch': '0.5938', 'num_input_tokens_seen': 445887888, 'train_runtime': '9.666e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133600/225000 [268:30:42<140:41:07,  5.54s/it]
 59%|█████▉    | 133601/225000 [268:30:49<153:11:20,  6.03s/it]
 59%|█████▉    | 133602/225000 [268:30:52<134:59:29,  5.32s/it]
 59%|█████▉    | 133603/225000 [268:30:57<132:33:26,  5.22s/it]
 59%|█████▉    | 133604/225000 [268:31:03<134:14:44,  5.29s/it]
 59%|█████▉    | 133605/225000 [268:31:11<156:53:50,  6.18s/it]
 59%|█████▉    | 133606/225000 [268:31:16<149:09:35,  5.88s/it]
 59%|█████▉    | 133607/225000 [268:31:25<174:11:52,  6.86s/it]
 59%|█████▉    | 133608/225000 [268:31:29<145:39:46,  5.74s/it]
 59%|█████▉    | 133609/225000 [268:31:36<156:10:21,  6.15s/it]
 59%|█████▉    | 133610/225000 [268:31:39<135:41:02,  5.34s/it]
 59%|█████▉    | 133611/225000 [268:31:45<139:06:05,  5.48s/it]
 59%|█████▉    | 133612/225000 [268:31:52<147:24:43,  5.81s/it]
 59%|█████▉    | 133613/225000 [268:31:59<161:37:44,  6.37s/it]
 59%|█████▉    | 133614/225000 [268:32:06<164:59:50,  6.50s/it]
 59%|█████▉    | 133615/225000 [268:32:10<149:06:18,  5.87s/it]
 59%|█████▉    | 133616/225000 [268:32:15<137:09:02,  5.40s/it]
 59%|█████▉    | 133617/225000 [268:32:24<170:01:10,  6.70s/it]
 59%|█████▉    | 133618/225000 [268:32:28<143:33:34,  5.66s/it]
 59%|█████▉    | 133619/225000 [268:32:35<158:07:12,  6.23s/it]
 59%|█████▉    | 133620/225000 [268:32:41<153:20:02,  6.04s/it]
 59%|█████▉    | 133621/225000 [268:32:47<153:48:41,  6.06s/it]
 59%|█████▉    | 133622/225000 [268:32:52<145:18:17,  5.72s/it]
 59%|█████▉    | 133623/225000 [268:32:55<127:28:22,  5.02s/it]
 59%|█████▉    | 133624/225000 [268:33:05<161:30:49,  6.36s/it]
 59%|█████▉    | 133625/225000 [268:33:09<143:27:19,  5.65s/it]
 59%|█████▉    | 133626/225000 [268:33:12<128:21:16,  5.06s/it]
 59%|█████▉    | 133627/225000 [268:33:21<154:31:24,  6.09s/it]
 59%|█████▉    | 133628/225000 [268:33:28<158:37:16,  6.25s/it]
 59%|█████▉    | 133629/225000 [268:33:31<138:04:01,  5.44s/it]
 59%|█████▉    | 133630/225000 [268:33:36<133:28:04,  5.26s/it]
 59%|█████▉    | 133631/225000 [268:33:41<134:17:31,  5.29s/it]
 59%|█████▉    | 133632/225000 [268:33:47<134:43:46,  5.31s/it]
 59%|█████▉    | 133633/225000 [268:33:52<134:28:14,  5.30s/it]
 59%|█████▉    | 133634/225000 [268:33:57<134:49:27,  5.31s/it]
 59%|█████▉    | 133635/225000 [268:34:02<133:57:45,  5.28s/it]
 59%|█████▉    | 133636/225000 [268:34:16<193:20:39,  7.62s/it]
 59%|█████▉    | 133637/225000 [268:34:19<165:10:16,  6.51s/it]
 59%|█████▉    | 133638/225000 [268:34:23<143:05:11,  5.64s/it]
 59%|█████▉    | 133639/225000 [268:34:33<175:59:47,  6.93s/it]
 59%|█████▉    | 133640/225000 [268:34:37<151:50:28,  5.98s/it]
 59%|█████▉    | 133641/225000 [268:34:46<174:16:04,  6.87s/it]
 59%|█████▉    | 133642/225000 [268:34:49<148:04:02,  5.83s/it]
 59%|█████▉    | 133643/225000 [268:34:57<166:16:04,  6.55s/it]
 59%|█████▉    | 133644/225000 [268:35:01<144:17:09,  5.69s/it]
 59%|█████▉    | 133645/225000 [268:35:08<157:30:35,  6.21s/it]
 59%|█████▉    | 133646/225000 [268:35:14<155:20:45,  6.12s/it]
 59%|█████▉    | 133647/225000 [268:35:23<170:39:31,  6.73s/it]
 59%|█████▉    | 133648/225000 [268:35:28<161:40:07,  6.37s/it]
 59%|█████▉    | 133649/225000 [268:35:33<148:18:20,  5.84s/it]
 59%|█████▉    | 133650/225000 [268:35:37<135:36:57,  5.34s/it]
                                                               
{'loss': '0.7928', 'grad_norm': '1.163', 'learning_rate': '1.918e-05', 'epoch': '0.594', 'num_input_tokens_seen': 446018560, 'train_runtime': '9.669e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133650/225000 [268:35:37<135:36:57,  5.34s/it]
 59%|█████▉    | 133651/225000 [268:35:42<132:25:48,  5.22s/it]
 59%|█████▉    | 133652/225000 [268:35:52<167:45:44,  6.61s/it]
 59%|█████▉    | 133653/225000 [268:35:55<145:27:01,  5.73s/it]
 59%|█████▉    | 133654/225000 [268:35:59<130:39:46,  5.15s/it]
 59%|█████▉    | 133655/225000 [268:36:03<123:57:53,  4.89s/it]
 59%|█████▉    | 133656/225000 [268:36:15<177:36:13,  7.00s/it]
 59%|█████▉    | 133657/225000 [268:36:20<158:55:09,  6.26s/it]
 59%|█████▉    | 133658/225000 [268:36:28<175:39:33,  6.92s/it]
 59%|█████▉    | 133659/225000 [268:36:32<148:19:19,  5.85s/it]
 59%|█████▉    | 133660/225000 [268:36:35<132:40:21,  5.23s/it]
 59%|█████▉    | 133661/225000 [268:36:43<152:50:46,  6.02s/it]
 59%|█████▉    | 133662/225000 [268:36:50<156:51:29,  6.18s/it]
 59%|█████▉    | 133663/225000 [268:36:53<136:04:32,  5.36s/it]
 59%|█████▉    | 133664/225000 [268:37:01<152:31:10,  6.01s/it]
 59%|█████▉    | 133665/225000 [268:37:04<132:44:06,  5.23s/it]
 59%|█████▉    | 133666/225000 [268:37:13<155:53:54,  6.14s/it]
 59%|█████▉    | 133667/225000 [268:37:16<137:18:59,  5.41s/it]
 59%|█████▉    | 133668/225000 [268:37:25<162:57:45,  6.42s/it]
 59%|█████▉    | 133669/225000 [268:37:30<148:45:13,  5.86s/it]
 59%|█████▉    | 133670/225000 [268:37:34<140:22:38,  5.53s/it]
 59%|█████▉    | 133671/225000 [268:37:39<133:34:31,  5.27s/it]
 59%|█████▉    | 133672/225000 [268:37:44<130:10:04,  5.13s/it]
 59%|█████▉    | 133673/225000 [268:37:48<125:04:50,  4.93s/it]
 59%|█████▉    | 133674/225000 [268:37:53<126:53:59,  5.00s/it]
 59%|█████▉    | 133675/225000 [268:38:06<182:38:03,  7.20s/it]
 59%|█████▉    | 133676/225000 [268:38:11<167:27:00,  6.60s/it]
 59%|█████▉    | 133677/225000 [268:38:16<158:25:45,  6.25s/it]
 59%|█████▉    | 133678/225000 [268:38:22<153:14:10,  6.04s/it]
 59%|█████▉    | 133679/225000 [268:38:30<165:13:57,  6.51s/it]
 59%|█████▉    | 133680/225000 [268:38:33<139:59:08,  5.52s/it]
 59%|█████▉    | 133681/225000 [268:38:41<158:45:33,  6.26s/it]
 59%|█████▉    | 133682/225000 [268:38:45<141:34:04,  5.58s/it]
 59%|█████▉    | 133683/225000 [268:38:49<128:25:42,  5.06s/it]
 59%|█████▉    | 133684/225000 [268:38:53<126:51:11,  5.00s/it]
 59%|█████▉    | 133685/225000 [268:39:05<174:32:10,  6.88s/it]
 59%|█████▉    | 133686/225000 [268:39:08<150:42:49,  5.94s/it]
 59%|█████▉    | 133687/225000 [268:39:12<132:40:11,  5.23s/it]
 59%|█████▉    | 133688/225000 [268:39:19<145:42:23,  5.74s/it]
 59%|█████▉    | 133689/225000 [268:39:27<166:21:25,  6.56s/it]
 59%|█████▉    | 133690/225000 [268:39:32<150:10:59,  5.92s/it]
 59%|█████▉    | 133691/225000 [268:39:36<138:41:10,  5.47s/it]
 59%|█████▉    | 133692/225000 [268:39:41<131:11:48,  5.17s/it]
 59%|█████▉    | 133693/225000 [268:39:45<126:27:22,  4.99s/it]
 59%|█████▉    | 133694/225000 [268:39:54<152:41:22,  6.02s/it]
 59%|█████▉    | 133695/225000 [268:40:04<180:59:59,  7.14s/it]
 59%|█████▉    | 133696/225000 [268:40:08<160:35:22,  6.33s/it]
 59%|█████▉    | 133697/225000 [268:40:13<148:47:59,  5.87s/it]
 59%|█████▉    | 133698/225000 [268:40:17<133:59:46,  5.28s/it]
 59%|█████▉    | 133699/225000 [268:40:21<127:26:30,  5.03s/it]
 59%|█████▉    | 133700/225000 [268:40:25<122:12:24,  4.82s/it]
                                                               
{'loss': '0.8101', 'grad_norm': '1.094', 'learning_rate': '1.916e-05', 'epoch': '0.5942', 'num_input_tokens_seen': 446148304, 'train_runtime': '9.672e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133700/225000 [268:40:25<122:12:24,  4.82s/it]
 59%|█████▉    | 133701/225000 [268:40:38<179:55:54,  7.09s/it]
 59%|█████▉    | 133702/225000 [268:40:42<158:02:23,  6.23s/it]
 59%|█████▉    | 133703/225000 [268:40:46<143:55:18,  5.68s/it]
 59%|█████▉    | 133704/225000 [268:40:56<170:37:17,  6.73s/it]
 59%|█████▉    | 133705/225000 [268:41:00<151:26:50,  5.97s/it]
 59%|█████▉    | 133706/225000 [268:41:08<164:27:04,  6.48s/it]
 59%|█████▉    | 133707/225000 [268:41:11<143:10:17,  5.65s/it]
 59%|█████▉    | 133708/225000 [268:41:21<175:48:34,  6.93s/it]
 59%|█████▉    | 133709/225000 [268:41:24<147:58:45,  5.84s/it]
 59%|█████▉    | 133710/225000 [268:41:31<152:25:40,  6.01s/it]
 59%|█████▉    | 133711/225000 [268:41:36<147:57:32,  5.83s/it]
 59%|█████▉    | 133712/225000 [268:41:44<163:58:26,  6.47s/it]
 59%|█████▉    | 133713/225000 [268:41:49<148:55:38,  5.87s/it]
 59%|█████▉    | 133714/225000 [268:41:53<139:05:57,  5.49s/it]
 59%|█████▉    | 133715/225000 [268:41:57<129:26:37,  5.10s/it]
 59%|█████▉    | 133716/225000 [268:42:02<124:45:59,  4.92s/it]
 59%|█████▉    | 133717/225000 [268:42:14<175:51:22,  6.94s/it]
 59%|█████▉    | 133718/225000 [268:42:19<166:34:25,  6.57s/it]
 59%|█████▉    | 133719/225000 [268:42:24<150:01:34,  5.92s/it]
 59%|█████▉    | 133720/225000 [268:42:28<141:19:54,  5.57s/it]
 59%|█████▉    | 133721/225000 [268:42:33<133:42:29,  5.27s/it]
 59%|█████▉    | 133722/225000 [268:42:38<129:43:00,  5.12s/it]
 59%|█████▉    | 133723/225000 [268:42:42<125:01:43,  4.93s/it]
 59%|█████▉    | 133724/225000 [268:42:54<173:30:15,  6.84s/it]
 59%|█████▉    | 133725/225000 [268:42:58<155:47:27,  6.14s/it]
 59%|█████▉    | 133726/225000 [268:43:02<135:31:21,  5.35s/it]
 59%|█████▉    | 133727/225000 [268:43:10<162:09:42,  6.40s/it]
 59%|█████▉    | 133728/225000 [268:43:14<143:09:21,  5.65s/it]
 59%|█████▉    | 133729/225000 [268:43:18<129:35:53,  5.11s/it]
 59%|█████▉    | 133730/225000 [268:43:29<170:42:50,  6.73s/it]
 59%|█████▉    | 133731/225000 [268:43:33<148:12:41,  5.85s/it]
 59%|█████▉    | 133732/225000 [268:43:36<128:15:36,  5.06s/it]
 59%|█████▉    | 133733/225000 [268:43:45<161:57:01,  6.39s/it]
 59%|█████▉    | 133734/225000 [268:43:49<138:19:27,  5.46s/it]
 59%|█████▉    | 133735/225000 [268:43:56<152:57:00,  6.03s/it]
 59%|█████▉    | 133736/225000 [268:44:01<143:43:53,  5.67s/it]
 59%|█████▉    | 133737/225000 [268:44:07<147:17:15,  5.81s/it]
 59%|█████▉    | 133738/225000 [268:44:10<130:30:29,  5.15s/it]
 59%|█████▉    | 133739/225000 [268:44:19<153:16:45,  6.05s/it]
 59%|█████▉    | 133740/225000 [268:44:23<142:18:01,  5.61s/it]
 59%|█████▉    | 133741/225000 [268:44:32<166:35:43,  6.57s/it]
 59%|█████▉    | 133742/225000 [268:44:35<139:39:35,  5.51s/it]
 59%|█████▉    | 133743/225000 [268:44:42<148:01:44,  5.84s/it]
 59%|█████▉    | 133744/225000 [268:44:46<139:17:37,  5.50s/it]
 59%|█████▉    | 133745/225000 [268:44:51<135:37:12,  5.35s/it]
 59%|█████▉    | 133746/225000 [268:45:00<160:26:03,  6.33s/it]
 59%|█████▉    | 133747/225000 [268:45:04<144:07:23,  5.69s/it]
 59%|█████▉    | 133748/225000 [268:45:08<132:06:29,  5.21s/it]
 59%|█████▉    | 133749/225000 [268:45:15<145:33:26,  5.74s/it]
 59%|█████▉    | 133750/225000 [268:45:24<169:16:10,  6.68s/it]
                                                               
{'loss': '0.8106', 'grad_norm': '0.9053', 'learning_rate': '1.914e-05', 'epoch': '0.5944', 'num_input_tokens_seen': 446277736, 'train_runtime': '9.675e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133750/225000 [268:45:24<169:16:10,  6.68s/it]
 59%|█████▉    | 133751/225000 [268:45:27<142:47:18,  5.63s/it]
 59%|█████▉    | 133752/225000 [268:45:36<165:30:29,  6.53s/it]
 59%|█████▉    | 133753/225000 [268:45:40<143:37:07,  5.67s/it]
 59%|█████▉    | 133754/225000 [268:45:45<138:29:31,  5.46s/it]
 59%|█████▉    | 133755/225000 [268:45:50<138:28:16,  5.46s/it]
 59%|█████▉    | 133756/225000 [268:45:55<137:59:35,  5.44s/it]
 59%|█████▉    | 133757/225000 [268:46:01<137:41:06,  5.43s/it]
 59%|█████▉    | 133758/225000 [268:46:08<152:05:13,  6.00s/it]
 59%|█████▉    | 133759/225000 [268:46:13<139:49:30,  5.52s/it]
 59%|█████▉    | 133760/225000 [268:46:17<128:34:48,  5.07s/it]
 59%|█████▉    | 133761/225000 [268:46:22<128:23:53,  5.07s/it]
 59%|█████▉    | 133762/225000 [268:46:34<180:32:07,  7.12s/it]
 59%|█████▉    | 133763/225000 [268:46:37<153:50:45,  6.07s/it]
 59%|█████▉    | 133764/225000 [268:46:41<135:41:08,  5.35s/it]
 59%|█████▉    | 133765/225000 [268:46:51<169:43:16,  6.70s/it]
 59%|█████▉    | 133766/225000 [268:46:54<146:56:08,  5.80s/it]
 59%|█████▉    | 133767/225000 [268:47:03<170:10:33,  6.72s/it]
 59%|█████▉    | 133768/225000 [268:47:07<147:04:30,  5.80s/it]
 59%|█████▉    | 133769/225000 [268:47:13<145:35:11,  5.74s/it]
 59%|█████▉    | 133770/225000 [268:47:20<156:04:47,  6.16s/it]
 59%|█████▉    | 133771/225000 [268:47:24<142:06:33,  5.61s/it]
 59%|█████▉    | 133772/225000 [268:47:28<133:21:46,  5.26s/it]
 59%|█████▉    | 133773/225000 [268:47:33<127:17:22,  5.02s/it]
 59%|█████▉    | 133774/225000 [268:47:37<122:25:55,  4.83s/it]
 59%|█████▉    | 133775/225000 [268:47:45<144:56:05,  5.72s/it]
 59%|█████▉    | 133776/225000 [268:47:54<172:39:10,  6.81s/it]
 59%|█████▉    | 133777/225000 [268:47:58<145:19:14,  5.73s/it]
 59%|█████▉    | 133778/225000 [268:48:06<162:07:33,  6.40s/it]
 59%|█████▉    | 133779/225000 [268:48:10<150:50:00,  5.95s/it]
 59%|█████▉    | 133780/225000 [268:48:15<139:41:08,  5.51s/it]
 59%|█████▉    | 133781/225000 [268:48:20<133:58:17,  5.29s/it]
 59%|█████▉    | 133782/225000 [268:48:25<130:17:23,  5.14s/it]
 59%|█████▉    | 133783/225000 [268:48:29<123:35:00,  4.88s/it]
 59%|█████▉    | 133784/225000 [268:48:34<126:25:54,  4.99s/it]
 59%|█████▉    | 133785/225000 [268:48:39<125:29:39,  4.95s/it]
 59%|█████▉    | 133786/225000 [268:48:44<123:37:20,  4.88s/it]
 59%|█████▉    | 133787/225000 [268:48:50<137:22:12,  5.42s/it]
 59%|█████▉    | 133788/225000 [268:49:02<186:46:15,  7.37s/it]
 59%|█████▉    | 133789/225000 [268:49:06<160:11:32,  6.32s/it]
 59%|█████▉    | 133790/225000 [268:49:10<143:57:52,  5.68s/it]
 59%|█████▉    | 133791/225000 [268:49:22<192:34:09,  7.60s/it]
 59%|█████▉    | 133792/225000 [268:49:26<160:21:48,  6.33s/it]
 59%|█████▉    | 133793/225000 [268:49:31<150:30:33,  5.94s/it]
 59%|█████▉    | 133794/225000 [268:49:36<144:01:29,  5.68s/it]
 59%|█████▉    | 133795/225000 [268:49:41<137:44:21,  5.44s/it]
 59%|█████▉    | 133796/225000 [268:49:46<136:09:56,  5.37s/it]
 59%|█████▉    | 133797/225000 [268:49:51<133:13:28,  5.26s/it]
 59%|█████▉    | 133798/225000 [268:49:56<130:23:03,  5.15s/it]
 59%|█████▉    | 133799/225000 [268:50:01<131:14:28,  5.18s/it]
 59%|█████▉    | 133800/225000 [268:50:06<131:19:25,  5.18s/it]
                                                               
{'loss': '0.8216', 'grad_norm': '1.055', 'learning_rate': '1.912e-05', 'epoch': '0.5947', 'num_input_tokens_seen': 446407712, 'train_runtime': '9.678e+05', 'train_tokens_per_second': '461.3'}

 59%|█████▉    | 133800/225000 [268:50:06<131:19:25,  5.18s/it]
 59%|█████▉    | 133801/225000 [268:50:15<159:51:39,  6.31s/it]
 59%|█████▉    | 133802/225000 [268:50:28<209:22:23,  8.26s/it]
 59%|█████▉    | 133803/225000 [268:50:31<171:47:27,  6.78s/it]
 59%|█████▉    | 133804/225000 [268:50:39<179:30:54,  7.09s/it]
 59%|█████▉    | 133805/225000 [268:50:44<161:57:08,  6.39s/it]
 59%|█████▉    | 133806/225000 [268:50:48<147:44:19,  5.83s/it]
 59%|█████▉    | 133807/225000 [268:50:53<140:25:08,  5.54s/it]
 59%|█████▉    | 133808/225000 [268:50:58<134:28:49,  5.31s/it]
 59%|█████▉    | 133809/225000 [268:51:03<130:29:07,  5.15s/it]
 59%|█████▉    | 133810/225000 [268:51:08<127:06:03,  5.02s/it]
 59%|█████▉    | 133811/225000 [268:51:19<174:21:42,  6.88s/it]
 59%|█████▉    | 133812/225000 [268:51:26<180:11:58,  7.11s/it]
 59%|█████▉    | 133813/225000 [268:51:30<155:17:18,  6.13s/it]
 59%|█████▉    | 133814/225000 [268:51:34<136:32:58,  5.39s/it]
 59%|█████▉    | 133815/225000 [268:51:43<163:12:19,  6.44s/it]
 59%|█████▉    | 133816/225000 [268:51:49<158:57:46,  6.28s/it]
 59%|█████▉    | 133817/225000 [268:51:55<157:04:44,  6.20s/it]
 59%|█████▉    | 133818/225000 [268:52:00<152:59:17,  6.04s/it]
 59%|█████▉    | 133819/225000 [268:52:05<143:52:27,  5.68s/it]
 59%|█████▉    | 133820/225000 [268:52:13<161:13:53,  6.37s/it]
 59%|█████▉    | 133821/225000 [268:52:18<147:34:01,  5.83s/it]
 59%|█████▉    | 133822/225000 [268:52:22<133:56:01,  5.29s/it]
 59%|█████▉    | 133823/225000 [268:52:27<129:28:15,  5.11s/it]
 59%|█████▉    | 133824/225000 [268:52:31<125:27:16,  4.95s/it]
 59%|█████▉    | 133825/225000 [268:52:36<123:56:00,  4.89s/it]
 59%|█████▉    | 133826/225000 [268:52:41<127:26:33,  5.03s/it]
 59%|█████▉    | 133827/225000 [268:52:54<184:15:49,  7.28s/it]
 59%|█████▉    | 133828/225000 [268:52:59<167:10:50,  6.60s/it]
 59%|█████▉    | 133829/225000 [268:53:02<143:55:13,  5.68s/it]
 59%|█████▉    | 133830/225000 [268:53:10<156:20:34,  6.17s/it]
 59%|█████▉    | 133831/225000 [268:53:14<145:02:58,  5.73s/it]
 59%|█████▉    | 133832/225000 [268:53:19<137:53:34,  5.45s/it]
 59%|█████▉    | 133833/225000 [268:53:26<150:54:01,  5.96s/it]
 59%|█████▉    | 133834/225000 [268:53:35<170:48:49,  6.75s/it]
 59%|█████▉    | 133835/225000 [268:53:39<152:17:08,  6.01s/it]
 59%|█████▉    | 133836/225000 [268:53:43<138:15:11,  5.46s/it]
 59%|█████▉    | 133837/225000 [268:53:48<129:30:05,  5.11s/it]
 59%|█████▉    | 133838/225000 [268:53:57<162:22:23,  6.41s/it]
 59%|█████▉    | 133839/225000 [268:54:03<157:37:53,  6.22s/it]
 59%|█████▉    | 133840/225000 [268:54:08<149:44:54,  5.91s/it]
 59%|█████▉    | 133841/225000 [268:54:12<138:00:47,  5.45s/it]
 59%|█████▉    | 133842/225000 [268:54:22<167:47:00,  6.63s/it]
 59%|█████▉    | 133843/225000 [268:54:26<150:20:52,  5.94s/it]
 59%|█████▉    | 133844/225000 [268:54:30<135:41:10,  5.36s/it]
 59%|█████▉    | 133845/225000 [268:54:35<128:57:22,  5.09s/it]
 59%|█████▉    | 133846/225000 [268:54:39<126:25:17,  4.99s/it]
 59%|█████▉    | 133847/225000 [268:54:49<165:08:30,  6.52s/it]
 59%|█████▉    | 133848/225000 [268:54:56<167:24:45,  6.61s/it]
 59%|█████▉    | 133849/225000 [268:55:00<148:46:46,  5.88s/it]
 59%|█████▉    | 133850/225000 [268:55:07<154:26:38,  6.10s/it]
                                                               
{'loss': '0.8036', 'grad_norm': '1.044', 'learning_rate': '1.911e-05', 'epoch': '0.5949', 'num_input_tokens_seen': 446537104, 'train_runtime': '9.681e+05', 'train_tokens_per_second': '461.2'}

 59%|█████▉    | 133850/225000 [268:55:07<154:26:38,  6.10s/it]
 59%|█████▉    | 133851/225000 [268:55:12<143:16:08,  5.66s/it]
 59%|█████▉    | 133852/225000 [268:55:16<132:03:19,  5.22s/it]
 59%|█████▉    | 133853/225000 [268:55:20<124:41:42,  4.93s/it]
 59%|█████▉    | 133854/225000 [268:55:25<121:00:46,  4.78s/it]
 59%|█████▉    | 133855/225000 [268:55:33<147:45:28,  5.84s/it]
 59%|█████▉    | 133856/225000 [268:55:44<187:53:19,  7.42s/it]
 59%|█████▉    | 133857/225000 [268:55:49<171:41:59,  6.78s/it]
 59%|█████▉    | 133858/225000 [268:55:54<157:11:49,  6.21s/it]
 59%|█████▉    | 133859/225000 [268:55:59<147:55:39,  5.84s/it]
 59%|█████▉    | 133860/225000 [268:56:04<138:25:44,  5.47s/it]
 59%|█████▉    | 133861/225000 [268:56:09<135:48:45,  5.36s/it]
 59%|█████▉    | 133862/225000 [268:56:13<130:24:46,  5.15s/it]
 59%|█████▉    | 133863/225000 [268:56:18<127:50:44,  5.05s/it]
 59%|█████▉    | 133864/225000 [268:56:23<124:45:05,  4.93s/it]
 59%|█████▉    | 133865/225000 [268:56:36<184:53:57,  7.30s/it]
 59%|█████▉    | 133866/225000 [268:56:40<164:51:38,  6.51s/it]
 59%|█████▉    | 133867/225000 [268:56:47<161:26:32,  6.38s/it]
 59%|█████▉    | 133868/225000 [268:56:52<153:35:17,  6.07s/it]
 59%|█████▉    | 133869/225000 [268:56:56<137:41:56,  5.44s/it]
 59%|█████▉    | 133870/225000 [268:57:01<134:33:24,  5.32s/it]
 59%|█████▉    | 133871/225000 [268:57:06<131:40:41,  5.20s/it]
 59%|█████▉    | 133872/225000 [268:57:11<132:15:29,  5.22s/it]
 59%|█████▉    | 133873/225000 [268:57:21<165:05:48,  6.52s/it]
 59%|█████▉    | 133874/225000 [268:57:26<155:55:18,  6.16s/it]
 60%|█████▉    | 133875/225000 [268:57:31<150:17:44,  5.94s/it]
 60%|█████▉    | 133876/225000 [268:57:36<142:28:44,  5.63s/it]
 60%|█████▉    | 133877/225000 [268:57:41<138:07:53,  5.46s/it]
 60%|█████▉    | 133878/225000 [268:57:46<130:25:53,  5.15s/it]
 60%|█████▉    | 133879/225000 [268:57:51<129:32:48,  5.12s/it]
 60%|█████▉    | 133880/225000 [268:57:56<128:44:22,  5.09s/it]
 60%|█████▉    | 133881/225000 [268:58:02<133:17:42,  5.27s/it]
 60%|█████▉    | 133882/225000 [268:58:09<151:48:20,  6.00s/it]
 60%|█████▉    | 133883/225000 [268:58:21<198:27:27,  7.84s/it]
 60%|█████▉    | 133884/225000 [268:58:26<174:04:46,  6.88s/it]
 60%|█████▉    | 133885/225000 [268:58:33<171:29:42,  6.78s/it]
 60%|█████▉    | 133886/225000 [268:58:39<165:47:32,  6.55s/it]
 60%|█████▉    | 133887/225000 [268:58:43<151:29:17,  5.99s/it]
 60%|█████▉    | 133888/225000 [268:58:48<141:43:16,  5.60s/it]
 60%|█████▉    | 133889/225000 [268:58:52<130:22:25,  5.15s/it]
 60%|█████▉    | 133890/225000 [268:58:58<139:10:26,  5.50s/it]
 60%|█████▉    | 133891/225000 [268:59:11<190:30:00,  7.53s/it]
 60%|█████▉    | 133892/225000 [268:59:14<160:56:24,  6.36s/it]
 60%|█████▉    | 133893/225000 [268:59:20<152:44:14,  6.04s/it]
 60%|█████▉    | 133894/225000 [268:59:26<158:10:32,  6.25s/it]
 60%|█████▉    | 133895/225000 [268:59:33<161:03:57,  6.36s/it]
 60%|█████▉    | 133896/225000 [268:59:37<147:25:43,  5.83s/it]
 60%|█████▉    | 133897/225000 [268:59:42<140:22:28,  5.55s/it]
 60%|█████▉    | 133898/225000 [268:59:47<135:39:37,  5.36s/it]
 60%|█████▉    | 133899/225000 [268:59:52<132:45:31,  5.25s/it]
 60%|█████▉    | 133900/225000 [268:59:58<132:51:11,  5.25s/it]
                                                               
{'loss': '0.806', 'grad_norm': '1.129', 'learning_rate': '1.909e-05', 'epoch': '0.5951', 'num_input_tokens_seen': 446666520, 'train_runtime': '9.684e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 133900/225000 [268:59:58<132:51:11,  5.25s/it]
 60%|█████▉    | 133901/225000 [269:00:02<128:27:16,  5.08s/it]
 60%|█████▉    | 133902/225000 [269:00:07<127:03:55,  5.02s/it]
 60%|█████▉    | 133903/225000 [269:00:12<128:05:30,  5.06s/it]
 60%|█████▉    | 133904/225000 [269:00:21<155:26:56,  6.14s/it]
 60%|█████▉    | 133905/225000 [269:00:28<164:23:05,  6.50s/it]
 60%|█████▉    | 133906/225000 [269:00:32<144:00:21,  5.69s/it]
 60%|█████▉    | 133907/225000 [269:00:40<163:24:01,  6.46s/it]
 60%|█████▉    | 133908/225000 [269:00:48<171:45:36,  6.79s/it]
 60%|█████▉    | 133909/225000 [269:00:52<150:52:34,  5.96s/it]
 60%|█████▉    | 133910/225000 [269:00:56<136:48:57,  5.41s/it]
 60%|█████▉    | 133911/225000 [269:01:01<133:44:00,  5.29s/it]
 60%|█████▉    | 133912/225000 [269:01:13<182:56:26,  7.23s/it]
 60%|█████▉    | 133913/225000 [269:01:16<153:19:48,  6.06s/it]
 60%|█████▉    | 133914/225000 [269:01:23<158:28:28,  6.26s/it]
 60%|█████▉    | 133915/225000 [269:01:26<138:08:28,  5.46s/it]
 60%|█████▉    | 133916/225000 [269:01:32<142:17:56,  5.62s/it]
 60%|█████▉    | 133917/225000 [269:01:39<152:23:06,  6.02s/it]
 60%|█████▉    | 133918/225000 [269:01:43<132:59:54,  5.26s/it]
 60%|█████▉    | 133919/225000 [269:01:52<164:46:35,  6.51s/it]
 60%|█████▉    | 133920/225000 [269:01:56<139:59:45,  5.53s/it]
 60%|█████▉    | 133921/225000 [269:01:59<125:38:25,  4.97s/it]
 60%|█████▉    | 133922/225000 [269:02:06<141:09:09,  5.58s/it]
 60%|█████▉    | 133923/225000 [269:02:11<137:11:40,  5.42s/it]
 60%|█████▉    | 133924/225000 [269:02:15<126:14:47,  4.99s/it]
 60%|█████▉    | 133925/225000 [269:02:20<122:54:56,  4.86s/it]
 60%|█████▉    | 133926/225000 [269:02:25<124:30:06,  4.92s/it]
 60%|█████▉    | 133927/225000 [269:02:30<123:05:43,  4.87s/it]
 60%|█████▉    | 133928/225000 [269:02:42<179:50:53,  7.11s/it]
 60%|█████▉    | 133929/225000 [269:02:47<160:53:12,  6.36s/it]
 60%|█████▉    | 133930/225000 [269:02:55<176:45:41,  6.99s/it]
 60%|█████▉    | 133931/225000 [269:02:59<150:43:16,  5.96s/it]
 60%|█████▉    | 133932/225000 [269:03:09<181:35:13,  7.18s/it]
 60%|█████▉    | 133933/225000 [269:03:12<152:19:46,  6.02s/it]
 60%|█████▉    | 133934/225000 [269:03:17<144:52:26,  5.73s/it]
 60%|█████▉    | 133935/225000 [269:03:25<164:43:10,  6.51s/it]
 60%|█████▉    | 133936/225000 [269:03:29<144:53:50,  5.73s/it]
 60%|█████▉    | 133937/225000 [269:03:36<151:23:50,  5.99s/it]
 60%|█████▉    | 133938/225000 [269:03:44<169:45:38,  6.71s/it]
 60%|█████▉    | 133939/225000 [269:03:48<144:45:26,  5.72s/it]
 60%|█████▉    | 133940/225000 [269:03:52<132:58:31,  5.26s/it]
 60%|█████▉    | 133941/225000 [269:04:00<159:00:18,  6.29s/it]
 60%|█████▉    | 133942/225000 [269:04:04<139:00:14,  5.50s/it]
 60%|█████▉    | 133943/225000 [269:04:10<142:06:10,  5.62s/it]
 60%|█████▉    | 133944/225000 [269:04:17<151:51:29,  6.00s/it]
 60%|█████▉    | 133945/225000 [269:04:21<139:35:17,  5.52s/it]
 60%|█████▉    | 133946/225000 [269:04:25<129:24:14,  5.12s/it]
 60%|█████▉    | 133947/225000 [269:04:29<119:16:14,  4.72s/it]
 60%|█████▉    | 133948/225000 [269:04:38<149:33:51,  5.91s/it]
 60%|█████▉    | 133949/225000 [269:04:45<157:50:10,  6.24s/it]
 60%|█████▉    | 133950/225000 [269:04:49<140:18:45,  5.55s/it]
                                                               
{'loss': '0.8152', 'grad_norm': '1.034', 'learning_rate': '1.907e-05', 'epoch': '0.5953', 'num_input_tokens_seen': 446795600, 'train_runtime': '9.687e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 133950/225000 [269:04:49<140:18:45,  5.55s/it]
 60%|█████▉    | 133951/225000 [269:04:56<148:51:41,  5.89s/it]
 60%|█████▉    | 133952/225000 [269:05:04<165:17:24,  6.54s/it]
 60%|█████▉    | 133953/225000 [269:05:10<166:01:01,  6.56s/it]
 60%|█████▉    | 133954/225000 [269:05:15<150:59:41,  5.97s/it]
 60%|█████▉    | 133955/225000 [269:05:19<139:04:31,  5.50s/it]
 60%|█████▉    | 133956/225000 [269:05:24<131:10:32,  5.19s/it]
 60%|█████▉    | 133957/225000 [269:05:28<127:38:18,  5.05s/it]
 60%|█████▉    | 133958/225000 [269:05:33<122:52:20,  4.86s/it]
 60%|█████▉    | 133959/225000 [269:05:37<120:51:59,  4.78s/it]
 60%|█████▉    | 133960/225000 [269:05:42<123:00:13,  4.86s/it]
 60%|█████▉    | 133961/225000 [269:05:51<152:46:35,  6.04s/it]
 60%|█████▉    | 133962/225000 [269:06:03<192:09:45,  7.60s/it]
 60%|█████▉    | 133963/225000 [269:06:06<159:56:16,  6.32s/it]
 60%|█████▉    | 133964/225000 [269:06:14<172:42:59,  6.83s/it]
 60%|█████▉    | 133965/225000 [269:06:20<166:13:47,  6.57s/it]
 60%|█████▉    | 133966/225000 [269:06:24<150:57:15,  5.97s/it]
 60%|█████▉    | 133967/225000 [269:06:30<151:14:28,  5.98s/it]
 60%|█████▉    | 133968/225000 [269:06:37<154:46:47,  6.12s/it]
 60%|█████▉    | 133969/225000 [269:06:42<147:02:55,  5.82s/it]
 60%|█████▉    | 133970/225000 [269:06:50<162:08:16,  6.41s/it]
 60%|█████▉    | 133971/225000 [269:06:53<138:34:53,  5.48s/it]
 60%|█████▉    | 133972/225000 [269:06:58<131:53:55,  5.22s/it]
 60%|█████▉    | 133973/225000 [269:07:02<127:44:11,  5.05s/it]
 60%|█████▉    | 133974/225000 [269:07:07<128:07:59,  5.07s/it]
 60%|█████▉    | 133975/225000 [269:07:12<126:38:58,  5.01s/it]
 60%|█████▉    | 133976/225000 [269:07:17<123:42:43,  4.89s/it]
 60%|█████▉    | 133977/225000 [269:07:30<182:41:52,  7.23s/it]
 60%|█████▉    | 133978/225000 [269:07:34<159:09:15,  6.29s/it]
 60%|█████▉    | 133979/225000 [269:07:42<172:19:45,  6.82s/it]
 60%|█████▉    | 133980/225000 [269:07:45<144:21:51,  5.71s/it]
 60%|█████▉    | 133981/225000 [269:07:49<134:07:57,  5.31s/it]
 60%|█████▉    | 133982/225000 [269:07:53<125:03:51,  4.95s/it]
 60%|█████▉    | 133983/225000 [269:07:58<122:31:52,  4.85s/it]
 60%|█████▉    | 133984/225000 [269:08:03<121:36:55,  4.81s/it]
 60%|█████▉    | 133985/225000 [269:08:08<125:32:10,  4.97s/it]
 60%|█████▉    | 133986/225000 [269:08:18<162:11:35,  6.42s/it]
 60%|█████▉    | 133987/225000 [269:08:27<186:07:26,  7.36s/it]
 60%|█████▉    | 133988/225000 [269:08:32<162:42:24,  6.44s/it]
 60%|█████▉    | 133989/225000 [269:08:36<144:04:51,  5.70s/it]
 60%|█████▉    | 133990/225000 [269:08:41<138:47:59,  5.49s/it]
 60%|█████▉    | 133991/225000 [269:08:50<166:48:59,  6.60s/it]
 60%|█████▉    | 133992/225000 [269:08:54<147:03:47,  5.82s/it]
 60%|█████▉    | 133993/225000 [269:08:59<141:15:09,  5.59s/it]
 60%|█████▉    | 133994/225000 [269:09:07<163:58:44,  6.49s/it]
 60%|█████▉    | 133995/225000 [269:09:11<144:47:39,  5.73s/it]
 60%|█████▉    | 133996/225000 [269:09:16<137:15:53,  5.43s/it]
 60%|█████▉    | 133997/225000 [269:09:21<130:57:32,  5.18s/it]
 60%|█████▉    | 133998/225000 [269:09:25<125:53:37,  4.98s/it]
 60%|█████▉    | 133999/225000 [269:09:37<174:53:37,  6.92s/it]
 60%|█████▉    | 134000/225000 [269:09:41<153:44:45,  6.08s/it]
                                                               
{'loss': '0.805', 'grad_norm': '0.996', 'learning_rate': '1.905e-05', 'epoch': '0.5956', 'num_input_tokens_seen': 446925832, 'train_runtime': '9.69e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134000/225000 [269:09:41<153:44:45,  6.08s/it]
 60%|█████▉    | 134001/225000 [269:09:45<136:10:49,  5.39s/it]
 60%|█████▉    | 134002/225000 [269:09:54<167:26:34,  6.62s/it]
 60%|█████▉    | 134003/225000 [269:09:57<141:26:28,  5.60s/it]
 60%|█████▉    | 134004/225000 [269:10:06<165:10:19,  6.53s/it]
 60%|█████▉    | 134005/225000 [269:10:10<142:02:31,  5.62s/it]
 60%|█████▉    | 134006/225000 [269:10:18<160:40:35,  6.36s/it]
 60%|█████▉    | 134007/225000 [269:10:22<145:29:36,  5.76s/it]
 60%|█████▉    | 134008/225000 [269:10:26<132:34:47,  5.25s/it]
 60%|█████▉    | 134009/225000 [269:10:30<122:58:16,  4.87s/it]
 60%|█████▉    | 134010/225000 [269:10:35<121:37:24,  4.81s/it]
 60%|█████▉    | 134011/225000 [269:10:47<180:19:07,  7.13s/it]
 60%|█████▉    | 134012/225000 [269:10:51<151:08:51,  5.98s/it]
 60%|█████▉    | 134013/225000 [269:10:59<171:43:38,  6.79s/it]
 60%|█████▉    | 134014/225000 [269:11:03<148:05:15,  5.86s/it]
 60%|█████▉    | 134015/225000 [269:11:07<132:10:38,  5.23s/it]
 60%|█████▉    | 134016/225000 [269:11:13<140:49:19,  5.57s/it]
 60%|█████▉    | 134017/225000 [269:11:22<167:12:04,  6.62s/it]
 60%|█████▉    | 134018/225000 [269:11:27<156:27:31,  6.19s/it]
 60%|█████▉    | 134019/225000 [269:11:32<147:10:09,  5.82s/it]
 60%|█████▉    | 134020/225000 [269:11:38<145:22:42,  5.75s/it]
 60%|█████▉    | 134021/225000 [269:11:43<139:35:20,  5.52s/it]
 60%|█████▉    | 134022/225000 [269:11:48<138:13:50,  5.47s/it]
 60%|█████▉    | 134023/225000 [269:11:53<132:26:30,  5.24s/it]
 60%|█████▉    | 134024/225000 [269:11:58<134:50:04,  5.34s/it]
 60%|█████▉    | 134025/225000 [269:12:04<134:05:51,  5.31s/it]
 60%|█████▉    | 134026/225000 [269:12:09<135:37:38,  5.37s/it]
 60%|█████▉    | 134027/225000 [269:12:14<133:13:41,  5.27s/it]
 60%|█████▉    | 134028/225000 [269:12:19<130:46:00,  5.17s/it]
 60%|█████▉    | 134029/225000 [269:12:24<131:45:26,  5.21s/it]
 60%|█████▉    | 134030/225000 [269:12:30<134:37:18,  5.33s/it]
 60%|█████▉    | 134031/225000 [269:12:39<164:07:13,  6.49s/it]
 60%|█████▉    | 134032/225000 [269:12:51<201:24:11,  7.97s/it]
 60%|█████▉    | 134033/225000 [269:12:55<172:11:16,  6.81s/it]
 60%|█████▉    | 134034/225000 [269:12:59<154:05:43,  6.10s/it]
 60%|█████▉    | 134035/225000 [269:13:03<137:10:35,  5.43s/it]
 60%|█████▉    | 134036/225000 [269:13:07<125:02:05,  4.95s/it]
 60%|█████▉    | 134037/225000 [269:13:17<162:13:50,  6.42s/it]
 60%|█████▉    | 134038/225000 [269:13:23<161:02:10,  6.37s/it]
 60%|█████▉    | 134039/225000 [269:13:29<155:08:38,  6.14s/it]
 60%|█████▉    | 134040/225000 [269:13:33<143:02:39,  5.66s/it]
 60%|█████▉    | 134041/225000 [269:13:39<141:27:41,  5.60s/it]
 60%|█████▉    | 134042/225000 [269:13:44<137:33:11,  5.44s/it]
 60%|█████▉    | 134043/225000 [269:13:49<135:40:00,  5.37s/it]
 60%|█████▉    | 134044/225000 [269:13:54<133:49:39,  5.30s/it]
 60%|█████▉    | 134045/225000 [269:13:59<131:35:55,  5.21s/it]
 60%|█████▉    | 134046/225000 [269:14:09<165:00:43,  6.53s/it]
 60%|█████▉    | 134047/225000 [269:14:17<175:48:49,  6.96s/it]
 60%|█████▉    | 134048/225000 [269:14:20<149:53:36,  5.93s/it]
 60%|█████▉    | 134049/225000 [269:14:28<166:04:08,  6.57s/it]
 60%|█████▉    | 134050/225000 [269:14:33<150:21:51,  5.95s/it]
                                                               
{'loss': '0.7935', 'grad_norm': '1.163', 'learning_rate': '1.903e-05', 'epoch': '0.5958', 'num_input_tokens_seen': 447056552, 'train_runtime': '9.693e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134050/225000 [269:14:33<150:21:51,  5.95s/it]
 60%|█████▉    | 134051/225000 [269:14:37<139:44:15,  5.53s/it]
 60%|█████▉    | 134052/225000 [269:14:42<134:23:17,  5.32s/it]
 60%|█████▉    | 134053/225000 [269:14:47<130:04:53,  5.15s/it]
 60%|█████▉    | 134054/225000 [269:14:51<122:05:31,  4.83s/it]
 60%|█████▉    | 134055/225000 [269:15:01<160:54:18,  6.37s/it]
 60%|█████▉    | 134056/225000 [269:15:08<169:11:50,  6.70s/it]
 60%|█████▉    | 134057/225000 [269:15:13<150:22:44,  5.95s/it]
 60%|█████▉    | 134058/225000 [269:15:20<161:42:17,  6.40s/it]
 60%|█████▉    | 134059/225000 [269:15:24<146:46:33,  5.81s/it]
 60%|█████▉    | 134060/225000 [269:15:29<136:34:53,  5.41s/it]
 60%|█████▉    | 134061/225000 [269:15:33<129:29:56,  5.13s/it]
 60%|█████▉    | 134062/225000 [269:15:38<125:49:23,  4.98s/it]
 60%|█████▉    | 134063/225000 [269:15:51<182:27:24,  7.22s/it]
 60%|█████▉    | 134064/225000 [269:15:55<158:05:28,  6.26s/it]
 60%|█████▉    | 134065/225000 [269:15:58<138:05:52,  5.47s/it]
 60%|█████▉    | 134066/225000 [269:16:08<171:14:15,  6.78s/it]
 60%|█████▉    | 134067/225000 [269:16:12<146:49:40,  5.81s/it]
 60%|█████▉    | 134068/225000 [269:16:16<133:24:19,  5.28s/it]
 60%|█████▉    | 134069/225000 [269:16:24<160:47:23,  6.37s/it]
 60%|█████▉    | 134070/225000 [269:16:31<163:28:28,  6.47s/it]
 60%|█████▉    | 134071/225000 [269:16:35<142:27:15,  5.64s/it]
 60%|█████▉    | 134072/225000 [269:16:44<166:46:33,  6.60s/it]
 60%|█████▉    | 134073/225000 [269:16:48<147:40:09,  5.85s/it]
 60%|█████▉    | 134074/225000 [269:16:55<161:09:11,  6.38s/it]
 60%|█████▉    | 134075/225000 [269:17:00<147:04:10,  5.82s/it]
 60%|█████▉    | 134076/225000 [269:17:04<132:44:50,  5.26s/it]
 60%|█████▉    | 134077/225000 [269:17:12<156:22:07,  6.19s/it]
 60%|█████▉    | 134078/225000 [269:17:18<155:39:31,  6.16s/it]
 60%|█████▉    | 134079/225000 [269:17:25<159:39:35,  6.32s/it]
 60%|█████▉    | 134080/225000 [269:17:30<152:00:50,  6.02s/it]
 60%|█████▉    | 134081/225000 [269:17:35<144:58:34,  5.74s/it]
 60%|█████▉    | 134082/225000 [269:17:41<141:52:12,  5.62s/it]
 60%|█████▉    | 134083/225000 [269:17:46<139:05:30,  5.51s/it]
 60%|█████▉    | 134084/225000 [269:17:51<137:52:16,  5.46s/it]
 60%|█████▉    | 134085/225000 [269:17:56<132:40:42,  5.25s/it]
 60%|█████▉    | 134086/225000 [269:18:02<135:06:17,  5.35s/it]
 60%|█████▉    | 134087/225000 [269:18:07<137:28:26,  5.44s/it]
 60%|█████▉    | 134088/225000 [269:18:13<137:20:59,  5.44s/it]
 60%|█████▉    | 134089/225000 [269:18:18<132:04:43,  5.23s/it]
 60%|█████▉    | 134090/225000 [269:18:23<133:22:42,  5.28s/it]
 60%|█████▉    | 134091/225000 [269:18:30<146:58:14,  5.82s/it]
 60%|█████▉    | 134092/225000 [269:18:35<141:26:03,  5.60s/it]
 60%|█████▉    | 134093/225000 [269:18:40<138:16:45,  5.48s/it]
 60%|█████▉    | 134094/225000 [269:18:46<136:29:29,  5.41s/it]
 60%|█████▉    | 134095/225000 [269:18:51<136:10:00,  5.39s/it]
 60%|█████▉    | 134096/225000 [269:18:57<138:32:07,  5.49s/it]
 60%|█████▉    | 134097/225000 [269:19:02<136:36:42,  5.41s/it]
 60%|█████▉    | 134098/225000 [269:19:15<192:15:47,  7.61s/it]
 60%|█████▉    | 134099/225000 [269:19:20<175:18:06,  6.94s/it]
 60%|█████▉    | 134100/225000 [269:19:26<168:24:26,  6.67s/it]
                                                               
{'loss': '0.8061', 'grad_norm': '0.9836', 'learning_rate': '1.902e-05', 'epoch': '0.596', 'num_input_tokens_seen': 447187352, 'train_runtime': '9.696e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134100/225000 [269:19:26<168:24:26,  6.67s/it]
 60%|█████▉    | 134101/225000 [269:19:31<156:02:30,  6.18s/it]
 60%|█████▉    | 134102/225000 [269:19:36<148:25:07,  5.88s/it]
 60%|█████▉    | 134103/225000 [269:19:41<141:28:07,  5.60s/it]
 60%|█████▉    | 134104/225000 [269:19:46<137:11:16,  5.43s/it]
 60%|█████▉    | 134105/225000 [269:19:51<135:28:43,  5.37s/it]
 60%|█████▉    | 134106/225000 [269:20:02<177:39:00,  7.04s/it]
 60%|█████▉    | 134107/225000 [269:20:06<149:06:49,  5.91s/it]
 60%|█████▉    | 134108/225000 [269:20:14<165:48:33,  6.57s/it]
 60%|█████▉    | 134109/225000 [269:20:20<161:29:07,  6.40s/it]
 60%|█████▉    | 134110/225000 [269:20:23<139:40:44,  5.53s/it]
 60%|█████▉    | 134111/225000 [269:20:33<174:31:19,  6.91s/it]
 60%|█████▉    | 134112/225000 [269:20:38<154:58:34,  6.14s/it]
 60%|█████▉    | 134113/225000 [269:20:42<139:46:00,  5.54s/it]
 60%|█████▉    | 134114/225000 [269:20:46<126:23:36,  5.01s/it]
 60%|█████▉    | 134115/225000 [269:20:50<122:41:50,  4.86s/it]
 60%|█████▉    | 134116/225000 [269:21:02<175:38:36,  6.96s/it]
 60%|█████▉    | 134117/225000 [269:21:07<157:41:47,  6.25s/it]
 60%|█████▉    | 134118/225000 [269:21:11<147:05:49,  5.83s/it]
 60%|█████▉    | 134119/225000 [269:21:16<140:36:22,  5.57s/it]
 60%|█████▉    | 134120/225000 [269:21:21<135:04:08,  5.35s/it]
 60%|█████▉    | 134121/225000 [269:21:26<128:26:16,  5.09s/it]
 60%|█████▉    | 134122/225000 [269:21:31<132:58:08,  5.27s/it]
 60%|█████▉    | 134123/225000 [269:21:37<135:24:44,  5.36s/it]
 60%|█████▉    | 134124/225000 [269:21:43<138:32:45,  5.49s/it]
 60%|█████▉    | 134125/225000 [269:21:49<142:03:14,  5.63s/it]
 60%|█████▉    | 134126/225000 [269:21:54<138:04:38,  5.47s/it]
 60%|█████▉    | 134127/225000 [269:22:06<189:23:01,  7.50s/it]
 60%|█████▉    | 134128/225000 [269:22:11<172:13:00,  6.82s/it]
 60%|█████▉    | 134129/225000 [269:22:16<159:28:19,  6.32s/it]
 60%|█████▉    | 134130/225000 [269:22:22<152:21:23,  6.04s/it]
 60%|█████▉    | 134131/225000 [269:22:27<146:21:32,  5.80s/it]
 60%|█████▉    | 134132/225000 [269:22:32<142:15:09,  5.64s/it]
 60%|█████▉    | 134133/225000 [269:22:41<168:10:34,  6.66s/it]
 60%|█████▉    | 134134/225000 [269:22:45<146:14:05,  5.79s/it]
 60%|█████▉    | 134135/225000 [269:22:52<151:28:22,  6.00s/it]
 60%|█████▉    | 134136/225000 [269:22:59<163:00:05,  6.46s/it]
 60%|█████▉    | 134137/225000 [269:23:04<148:01:51,  5.87s/it]
 60%|█████▉    | 134138/225000 [269:23:08<137:35:34,  5.45s/it]
 60%|█████▉    | 134139/225000 [269:23:13<130:28:04,  5.17s/it]
 60%|█████▉    | 134140/225000 [269:23:17<125:24:29,  4.97s/it]
 60%|█████▉    | 134141/225000 [269:23:29<174:16:01,  6.90s/it]
 60%|█████▉    | 134142/225000 [269:23:33<156:36:05,  6.20s/it]
 60%|█████▉    | 134143/225000 [269:23:38<143:14:36,  5.68s/it]
 60%|█████▉    | 134144/225000 [269:23:42<134:49:31,  5.34s/it]
 60%|█████▉    | 134145/225000 [269:23:47<127:34:20,  5.05s/it]
 60%|█████▉    | 134146/225000 [269:23:55<149:43:22,  5.93s/it]
 60%|█████▉    | 134147/225000 [269:24:04<173:41:49,  6.88s/it]
 60%|█████▉    | 134148/225000 [269:24:10<167:59:02,  6.66s/it]
 60%|█████▉    | 134149/225000 [269:24:14<149:26:02,  5.92s/it]
 60%|█████▉    | 134150/225000 [269:24:18<137:32:51,  5.45s/it]
                                                               
{'loss': '0.7974', 'grad_norm': '1.044', 'learning_rate': '1.9e-05', 'epoch': '0.5962', 'num_input_tokens_seen': 447316408, 'train_runtime': '9.699e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134150/225000 [269:24:18<137:32:51,  5.45s/it]
 60%|█████▉    | 134151/225000 [269:24:23<131:29:38,  5.21s/it]
 60%|█████▉    | 134152/225000 [269:24:33<170:19:53,  6.75s/it]
 60%|█████▉    | 134153/225000 [269:24:38<155:45:45,  6.17s/it]
 60%|█████▉    | 134154/225000 [269:24:43<146:00:44,  5.79s/it]
 60%|█████▉    | 134155/225000 [269:24:48<140:36:11,  5.57s/it]
 60%|█████▉    | 134156/225000 [269:24:53<137:37:11,  5.45s/it]
 60%|█████▉    | 134157/225000 [269:24:59<136:25:45,  5.41s/it]
 60%|█████▉    | 134158/225000 [269:25:04<136:22:40,  5.40s/it]
 60%|█████▉    | 134159/225000 [269:25:09<135:42:56,  5.38s/it]
 60%|█████▉    | 134160/225000 [269:25:14<134:10:46,  5.32s/it]
 60%|█████▉    | 134161/225000 [269:25:21<140:36:55,  5.57s/it]
 60%|█████▉    | 134162/225000 [269:25:33<195:42:51,  7.76s/it]
 60%|█████▉    | 134163/225000 [269:25:38<169:53:45,  6.73s/it]
 60%|█████▉    | 134164/225000 [269:25:42<154:12:30,  6.11s/it]
 60%|█████▉    | 134165/225000 [269:25:47<145:14:21,  5.76s/it]
 60%|█████▉    | 134166/225000 [269:25:52<139:08:48,  5.51s/it]
 60%|█████▉    | 134167/225000 [269:25:56<127:47:50,  5.07s/it]
 60%|█████▉    | 134168/225000 [269:26:01<127:07:18,  5.04s/it]
 60%|█████▉    | 134169/225000 [269:26:06<125:50:44,  4.99s/it]
 60%|█████▉    | 134170/225000 [269:26:16<159:48:11,  6.33s/it]
 60%|█████▉    | 134171/225000 [269:26:24<173:32:51,  6.88s/it]
 60%|█████▉    | 134172/225000 [269:26:28<149:46:49,  5.94s/it]
 60%|█████▉    | 134173/225000 [269:26:31<130:44:42,  5.18s/it]
 60%|█████▉    | 134174/225000 [269:26:42<173:04:54,  6.86s/it]
 60%|█████▉    | 134175/225000 [269:26:46<151:16:24,  6.00s/it]
 60%|█████▉    | 134176/225000 [269:26:50<136:57:25,  5.43s/it]
 60%|█████▉    | 134177/225000 [269:26:58<158:02:37,  6.26s/it]
 60%|█████▉    | 134178/225000 [269:27:03<150:36:41,  5.97s/it]
 60%|█████▉    | 134179/225000 [269:27:10<152:22:41,  6.04s/it]
 60%|█████▉    | 134180/225000 [269:27:13<133:35:25,  5.30s/it]
 60%|█████▉    | 134181/225000 [269:27:17<125:03:24,  4.96s/it]
 60%|█████▉    | 134182/225000 [269:27:22<119:52:06,  4.75s/it]
 60%|█████▉    | 134183/225000 [269:27:26<115:41:04,  4.59s/it]
 60%|█████▉    | 134184/225000 [269:27:34<139:49:34,  5.54s/it]
 60%|█████▉    | 134185/225000 [269:27:43<170:17:54,  6.75s/it]
 60%|█████▉    | 134186/225000 [269:27:48<153:48:02,  6.10s/it]
 60%|█████▉    | 134187/225000 [269:27:52<140:15:59,  5.56s/it]
 60%|█████▉    | 134188/225000 [269:27:57<133:19:07,  5.29s/it]
 60%|█████▉    | 134189/225000 [269:28:08<178:23:18,  7.07s/it]
 60%|█████▉    | 134190/225000 [269:28:13<160:31:18,  6.36s/it]
 60%|█████▉    | 134191/225000 [269:28:17<148:43:56,  5.90s/it]
 60%|█████▉    | 134192/225000 [269:28:22<135:33:27,  5.37s/it]
 60%|█████▉    | 134193/225000 [269:28:26<129:49:31,  5.15s/it]
 60%|█████▉    | 134194/225000 [269:28:37<173:59:12,  6.90s/it]
 60%|█████▉    | 134195/225000 [269:28:42<157:16:39,  6.24s/it]
 60%|█████▉    | 134196/225000 [269:28:45<136:33:02,  5.41s/it]
 60%|█████▉    | 134197/225000 [269:28:54<161:45:24,  6.41s/it]
 60%|█████▉    | 134198/225000 [269:28:59<147:05:45,  5.83s/it]
 60%|█████▉    | 134199/225000 [269:29:03<135:26:44,  5.37s/it]
 60%|█████▉    | 134200/225000 [269:29:07<125:21:12,  4.97s/it]
                                                               
{'loss': '0.8042', 'grad_norm': '1.076', 'learning_rate': '1.898e-05', 'epoch': '0.5964', 'num_input_tokens_seen': 447445192, 'train_runtime': '9.701e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134200/225000 [269:29:07<125:21:12,  4.97s/it]
 60%|█████▉    | 134201/225000 [269:29:11<121:17:02,  4.81s/it]
 60%|█████▉    | 134202/225000 [269:29:19<140:50:57,  5.58s/it]
 60%|█████▉    | 134203/225000 [269:29:30<185:58:06,  7.37s/it]
 60%|█████▉    | 134204/225000 [269:29:34<155:29:55,  6.17s/it]
 60%|█████▉    | 134205/225000 [269:29:37<135:44:53,  5.38s/it]
 60%|█████▉    | 134206/225000 [269:29:46<162:08:38,  6.43s/it]
 60%|█████▉    | 134207/225000 [269:29:53<169:26:07,  6.72s/it]
 60%|█████▉    | 134208/225000 [269:29:57<148:56:16,  5.91s/it]
 60%|█████▉    | 134209/225000 [269:30:01<134:55:23,  5.35s/it]
 60%|█████▉    | 134210/225000 [269:30:12<176:46:53,  7.01s/it]
 60%|█████▉    | 134211/225000 [269:30:16<153:24:48,  6.08s/it]
 60%|█████▉    | 134212/225000 [269:30:24<162:58:23,  6.46s/it]
 60%|█████▉    | 134213/225000 [269:30:28<145:02:19,  5.75s/it]
 60%|█████▉    | 134214/225000 [269:30:32<130:09:07,  5.16s/it]
 60%|█████▉    | 134215/225000 [269:30:36<125:06:57,  4.96s/it]
 60%|█████▉    | 134216/225000 [269:30:41<121:40:25,  4.82s/it]
 60%|█████▉    | 134217/225000 [269:30:45<118:58:12,  4.72s/it]
 60%|█████▉    | 134218/225000 [269:30:56<166:19:00,  6.60s/it]
 60%|█████▉    | 134219/225000 [269:31:03<167:59:15,  6.66s/it]
 60%|█████▉    | 134220/225000 [269:31:10<174:00:52,  6.90s/it]
 60%|█████▉    | 134221/225000 [269:31:14<147:26:03,  5.85s/it]
 60%|█████▉    | 134222/225000 [269:31:21<159:33:23,  6.33s/it]
 60%|█████▉    | 134223/225000 [269:31:27<153:16:46,  6.08s/it]
 60%|█████▉    | 134224/225000 [269:31:32<152:11:16,  6.04s/it]
 60%|█████▉    | 134225/225000 [269:31:36<133:15:11,  5.28s/it]
 60%|█████▉    | 134226/225000 [269:31:40<124:23:58,  4.93s/it]
 60%|█████▉    | 134227/225000 [269:31:45<120:30:03,  4.78s/it]
 60%|█████▉    | 134228/225000 [269:31:52<137:25:37,  5.45s/it]
 60%|█████▉    | 134229/225000 [269:32:01<170:33:48,  6.76s/it]
 60%|█████▉    | 134230/225000 [269:32:06<153:18:47,  6.08s/it]
 60%|█████▉    | 134231/225000 [269:32:11<142:25:29,  5.65s/it]
 60%|█████▉    | 134232/225000 [269:32:15<136:53:20,  5.43s/it]
 60%|█████▉    | 134233/225000 [269:32:20<127:09:28,  5.04s/it]
 60%|█████▉    | 134234/225000 [269:32:25<130:21:16,  5.17s/it]
 60%|█████▉    | 134235/225000 [269:32:30<131:43:20,  5.22s/it]
 60%|█████▉    | 134236/225000 [269:32:37<138:33:02,  5.50s/it]
 60%|█████▉    | 134237/225000 [269:32:49<192:56:27,  7.65s/it]
 60%|█████▉    | 134238/225000 [269:32:53<161:05:52,  6.39s/it]
 60%|█████▉    | 134239/225000 [269:33:00<165:32:01,  6.57s/it]
 60%|█████▉    | 134240/225000 [269:33:04<148:31:34,  5.89s/it]
 60%|█████▉    | 134241/225000 [269:33:08<132:15:13,  5.25s/it]
 60%|█████▉    | 134242/225000 [269:33:19<176:43:30,  7.01s/it]
 60%|█████▉    | 134243/225000 [269:33:23<152:48:34,  6.06s/it]
 60%|█████▉    | 134244/225000 [269:33:27<141:40:40,  5.62s/it]
 60%|█████▉    | 134245/225000 [269:33:36<164:06:48,  6.51s/it]
 60%|█████▉    | 134246/225000 [269:33:40<145:43:43,  5.78s/it]
 60%|█████▉    | 134247/225000 [269:33:45<138:42:20,  5.50s/it]
 60%|█████▉    | 134248/225000 [269:33:49<131:36:18,  5.22s/it]
 60%|█████▉    | 134249/225000 [269:33:57<149:37:44,  5.94s/it]
 60%|█████▉    | 134250/225000 [269:34:06<171:36:06,  6.81s/it]
                                                               
{'loss': '0.8074', 'grad_norm': '1.09', 'learning_rate': '1.896e-05', 'epoch': '0.5967', 'num_input_tokens_seen': 447575640, 'train_runtime': '9.704e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134250/225000 [269:34:06<171:36:06,  6.81s/it]
 60%|█████▉    | 134251/225000 [269:34:09<147:11:25,  5.84s/it]
 60%|█████▉    | 134252/225000 [269:34:18<169:52:07,  6.74s/it]
 60%|█████▉    | 134253/225000 [269:34:22<143:56:25,  5.71s/it]
 60%|█████▉    | 134254/225000 [269:34:26<131:28:12,  5.22s/it]
 60%|█████▉    | 134255/225000 [269:34:34<155:16:41,  6.16s/it]
 60%|█████▉    | 134256/225000 [269:34:38<138:02:37,  5.48s/it]
 60%|█████▉    | 134257/225000 [269:34:42<127:18:20,  5.05s/it]
 60%|█████▉    | 134258/225000 [269:34:53<173:32:09,  6.88s/it]
 60%|█████▉    | 134259/225000 [269:34:57<153:28:40,  6.09s/it]
 60%|█████▉    | 134260/225000 [269:35:02<141:57:29,  5.63s/it]
 60%|█████▉    | 134261/225000 [269:35:07<136:16:41,  5.41s/it]
 60%|█████▉    | 134262/225000 [269:35:11<130:00:38,  5.16s/it]
 60%|█████▉    | 134263/225000 [269:35:16<127:00:57,  5.04s/it]
 60%|█████▉    | 134264/225000 [269:35:24<151:32:41,  6.01s/it]
 60%|█████▉    | 134265/225000 [269:35:33<169:54:06,  6.74s/it]
 60%|█████▉    | 134266/225000 [269:35:36<145:29:41,  5.77s/it]
 60%|█████▉    | 134267/225000 [269:35:46<174:33:39,  6.93s/it]
 60%|█████▉    | 134268/225000 [269:35:51<163:18:01,  6.48s/it]
 60%|█████▉    | 134269/225000 [269:35:58<164:42:42,  6.54s/it]
 60%|█████▉    | 134270/225000 [269:36:01<138:20:45,  5.49s/it]
 60%|█████▉    | 134271/225000 [269:36:09<159:55:00,  6.35s/it]
 60%|█████▉    | 134272/225000 [269:36:13<135:39:05,  5.38s/it]
 60%|█████▉    | 134273/225000 [269:36:22<164:10:15,  6.51s/it]
 60%|█████▉    | 134274/225000 [269:36:26<144:16:06,  5.72s/it]
 60%|█████▉    | 134275/225000 [269:36:34<164:39:39,  6.53s/it]
 60%|█████▉    | 134276/225000 [269:36:39<152:09:49,  6.04s/it]
 60%|█████▉    | 134277/225000 [269:36:46<159:18:31,  6.32s/it]
 60%|█████▉    | 134278/225000 [269:36:50<145:19:14,  5.77s/it]
 60%|█████▉    | 134279/225000 [269:36:54<128:58:46,  5.12s/it]
 60%|█████▉    | 134280/225000 [269:36:58<119:48:24,  4.75s/it]
 60%|█████▉    | 134281/225000 [269:37:08<161:11:45,  6.40s/it]
 60%|█████▉    | 134282/225000 [269:37:12<143:11:11,  5.68s/it]
 60%|█████▉    | 134283/225000 [269:37:16<128:22:51,  5.09s/it]
 60%|█████▉    | 134284/225000 [269:37:27<170:49:19,  6.78s/it]
 60%|█████▉    | 134285/225000 [269:37:31<152:36:03,  6.06s/it]
 60%|█████▉    | 134286/225000 [269:37:34<133:01:57,  5.28s/it]
 60%|█████▉    | 134287/225000 [269:37:44<162:31:46,  6.45s/it]
 60%|█████▉    | 134288/225000 [269:37:48<146:01:48,  5.80s/it]
 60%|█████▉    | 134289/225000 [269:37:52<132:35:19,  5.26s/it]
 60%|█████▉    | 134290/225000 [269:37:56<122:38:28,  4.87s/it]
 60%|█████▉    | 134291/225000 [269:38:01<124:19:35,  4.93s/it]
 60%|█████▉    | 134292/225000 [269:38:13<176:26:13,  7.00s/it]
 60%|█████▉    | 134293/225000 [269:38:16<148:59:50,  5.91s/it]
 60%|█████▉    | 134294/225000 [269:38:23<154:36:25,  6.14s/it]
 60%|█████▉    | 134295/225000 [269:38:29<152:09:58,  6.04s/it]
 60%|█████▉    | 134296/225000 [269:38:33<142:39:02,  5.66s/it]
 60%|█████▉    | 134297/225000 [269:38:38<135:17:51,  5.37s/it]
 60%|█████▉    | 134298/225000 [269:38:43<133:05:32,  5.28s/it]
 60%|█████▉    | 134299/225000 [269:38:48<130:19:16,  5.17s/it]
 60%|█████▉    | 134300/225000 [269:38:53<128:38:37,  5.11s/it]
                                                               
{'loss': '0.7991', 'grad_norm': '1.081', 'learning_rate': '1.895e-05', 'epoch': '0.5969', 'num_input_tokens_seen': 447705672, 'train_runtime': '9.707e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134300/225000 [269:38:53<128:38:37,  5.11s/it]
 60%|█████▉    | 134301/225000 [269:38:58<124:40:17,  4.95s/it]
 60%|█████▉    | 134302/225000 [269:39:03<130:42:31,  5.19s/it]
 60%|█████▉    | 134303/225000 [269:39:15<180:02:50,  7.15s/it]
 60%|█████▉    | 134304/225000 [269:39:20<165:43:31,  6.58s/it]
 60%|█████▉    | 134305/225000 [269:39:24<146:38:36,  5.82s/it]
 60%|█████▉    | 134306/225000 [269:39:29<135:03:49,  5.36s/it]
 60%|█████▉    | 134307/225000 [269:39:39<172:59:56,  6.87s/it]
 60%|█████▉    | 134308/225000 [269:39:43<155:03:30,  6.16s/it]
 60%|█████▉    | 134309/225000 [269:39:52<170:50:11,  6.78s/it]
 60%|█████▉    | 134310/225000 [269:39:55<144:52:35,  5.75s/it]
 60%|█████▉    | 134311/225000 [269:39:59<128:26:19,  5.10s/it]
 60%|█████▉    | 134312/225000 [269:40:08<162:54:13,  6.47s/it]
 60%|█████▉    | 134313/225000 [269:40:12<145:15:56,  5.77s/it]
 60%|█████▉    | 134314/225000 [269:40:17<133:18:09,  5.29s/it]
 60%|█████▉    | 134315/225000 [269:40:21<124:02:10,  4.92s/it]
 60%|█████▉    | 134316/225000 [269:40:29<148:50:13,  5.91s/it]
 60%|█████▉    | 134317/225000 [269:40:36<159:20:39,  6.33s/it]
 60%|█████▉    | 134318/225000 [269:40:40<137:46:34,  5.47s/it]
 60%|█████▉    | 134319/225000 [269:40:49<163:43:40,  6.50s/it]
 60%|█████▉    | 134320/225000 [269:40:54<152:52:49,  6.07s/it]
 60%|█████▉    | 134321/225000 [269:40:58<136:46:58,  5.43s/it]
 60%|█████▉    | 134322/225000 [269:41:01<124:25:10,  4.94s/it]
 60%|█████▉    | 134323/225000 [269:41:12<166:55:58,  6.63s/it]
 60%|█████▉    | 134324/225000 [269:41:15<142:05:18,  5.64s/it]
 60%|█████▉    | 134325/225000 [269:41:23<157:39:09,  6.26s/it]
 60%|█████▉    | 134326/225000 [269:41:29<158:26:58,  6.29s/it]
 60%|█████▉    | 134327/225000 [269:41:34<144:58:45,  5.76s/it]
 60%|█████▉    | 134328/225000 [269:41:38<132:15:17,  5.25s/it]
 60%|█████▉    | 134329/225000 [269:41:42<123:57:37,  4.92s/it]
 60%|█████▉    | 134330/225000 [269:41:47<121:30:48,  4.82s/it]
 60%|█████▉    | 134331/225000 [269:41:56<152:19:14,  6.05s/it]
 60%|█████▉    | 134332/225000 [269:42:04<170:25:17,  6.77s/it]
 60%|█████▉    | 134333/225000 [269:42:08<146:45:12,  5.83s/it]
 60%|█████▉    | 134334/225000 [269:42:15<156:16:11,  6.20s/it]
 60%|█████▉    | 134335/225000 [269:42:24<177:40:55,  7.06s/it]
 60%|█████▉    | 134336/225000 [269:42:28<153:23:44,  6.09s/it]
 60%|█████▉    | 134337/225000 [269:42:31<135:59:36,  5.40s/it]
 60%|█████▉    | 134338/225000 [269:42:38<141:28:16,  5.62s/it]
 60%|█████▉    | 134339/225000 [269:42:47<171:00:02,  6.79s/it]
 60%|█████▉    | 134340/225000 [269:42:51<147:29:07,  5.86s/it]
 60%|█████▉    | 134341/225000 [269:42:57<147:09:06,  5.84s/it]
 60%|█████▉    | 134342/225000 [269:43:06<170:44:07,  6.78s/it]
 60%|█████▉    | 134343/225000 [269:43:10<152:56:03,  6.07s/it]
 60%|█████▉    | 134344/225000 [269:43:15<141:50:46,  5.63s/it]
 60%|█████▉    | 134345/225000 [269:43:19<134:21:09,  5.34s/it]
 60%|█████▉    | 134346/225000 [269:43:24<128:21:26,  5.10s/it]
 60%|█████▉    | 134347/225000 [269:43:28<124:31:39,  4.95s/it]
 60%|█████▉    | 134348/225000 [269:43:33<121:36:36,  4.83s/it]
 60%|█████▉    | 134349/225000 [269:43:38<122:12:08,  4.85s/it]
 60%|█████▉    | 134350/225000 [269:43:50<176:05:44,  6.99s/it]
                                                               
{'loss': '0.7949', 'grad_norm': '1.067', 'learning_rate': '1.893e-05', 'epoch': '0.5971', 'num_input_tokens_seen': 447835016, 'train_runtime': '9.71e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134350/225000 [269:43:50<176:05:44,  6.99s/it]
 60%|█████▉    | 134351/225000 [269:43:55<164:14:23,  6.52s/it]
 60%|█████▉    | 134352/225000 [269:43:59<145:54:24,  5.79s/it]
 60%|█████▉    | 134353/225000 [269:44:06<150:43:51,  5.99s/it]
 60%|█████▉    | 134354/225000 [269:44:12<155:11:18,  6.16s/it]
 60%|█████▉    | 134355/225000 [269:44:17<142:30:19,  5.66s/it]
 60%|█████▉    | 134356/225000 [269:44:21<132:25:39,  5.26s/it]
 60%|█████▉    | 134357/225000 [269:44:26<128:36:23,  5.11s/it]
 60%|█████▉    | 134358/225000 [269:44:30<123:47:40,  4.92s/it]
 60%|█████▉    | 134359/225000 [269:44:42<171:10:39,  6.80s/it]
 60%|█████▉    | 134360/225000 [269:44:47<159:14:41,  6.32s/it]
 60%|█████▉    | 134361/225000 [269:44:50<138:30:51,  5.50s/it]
 60%|█████▉    | 134362/225000 [269:45:00<173:35:56,  6.90s/it]
 60%|█████▉    | 134363/225000 [269:45:04<148:04:47,  5.88s/it]
 60%|█████▉    | 134364/225000 [269:45:12<162:23:06,  6.45s/it]
 60%|█████▉    | 134365/225000 [269:45:17<150:31:47,  5.98s/it]
 60%|█████▉    | 134366/225000 [269:45:20<132:25:37,  5.26s/it]
 60%|█████▉    | 134367/225000 [269:45:28<152:45:40,  6.07s/it]
 60%|█████▉    | 134368/225000 [269:45:34<150:18:45,  5.97s/it]
 60%|█████▉    | 134369/225000 [269:45:41<158:21:13,  6.29s/it]
 60%|█████▉    | 134370/225000 [269:45:46<148:05:21,  5.88s/it]
 60%|█████▉    | 134371/225000 [269:45:49<129:19:00,  5.14s/it]
 60%|█████▉    | 134372/225000 [269:45:54<123:10:39,  4.89s/it]
 60%|█████▉    | 134373/225000 [269:45:58<119:30:49,  4.75s/it]
 60%|█████▉    | 134374/225000 [269:46:02<115:25:09,  4.58s/it]
 60%|█████▉    | 134375/225000 [269:46:14<168:51:32,  6.71s/it]
 60%|█████▉    | 134376/225000 [269:46:17<143:51:56,  5.71s/it]
 60%|█████▉    | 134377/225000 [269:46:23<142:45:56,  5.67s/it]
 60%|█████▉    | 134378/225000 [269:46:28<142:16:17,  5.65s/it]
 60%|█████▉    | 134379/225000 [269:46:36<157:37:59,  6.26s/it]
 60%|█████▉    | 134380/225000 [269:46:41<144:08:58,  5.73s/it]
 60%|█████▉    | 134381/225000 [269:46:45<136:04:30,  5.41s/it]
 60%|█████▉    | 134382/225000 [269:46:50<127:28:44,  5.06s/it]
 60%|█████▉    | 134383/225000 [269:47:01<172:01:35,  6.83s/it]
 60%|█████▉    | 134384/225000 [269:47:05<155:28:17,  6.18s/it]
 60%|█████▉    | 134385/225000 [269:47:11<149:27:44,  5.94s/it]
 60%|█████▉    | 134386/225000 [269:47:19<166:26:47,  6.61s/it]
 60%|█████▉    | 134387/225000 [269:47:22<139:58:41,  5.56s/it]
 60%|█████▉    | 134388/225000 [269:47:28<143:38:51,  5.71s/it]
 60%|█████▉    | 134389/225000 [269:47:34<149:23:35,  5.94s/it]
 60%|█████▉    | 134390/225000 [269:47:42<164:03:31,  6.52s/it]
 60%|█████▉    | 134391/225000 [269:47:46<140:30:01,  5.58s/it]
 60%|█████▉    | 134392/225000 [269:47:51<139:25:26,  5.54s/it]
 60%|█████▉    | 134393/225000 [269:47:59<154:53:35,  6.15s/it]
 60%|█████▉    | 134394/225000 [269:48:03<141:27:27,  5.62s/it]
 60%|█████▉    | 134395/225000 [269:48:08<133:59:44,  5.32s/it]
 60%|█████▉    | 134396/225000 [269:48:12<127:16:16,  5.06s/it]
 60%|█████▉    | 134397/225000 [269:48:18<133:42:21,  5.31s/it]
 60%|█████▉    | 134398/225000 [269:48:28<169:50:47,  6.75s/it]
 60%|█████▉    | 134399/225000 [269:48:32<146:20:23,  5.81s/it]
 60%|█████▉    | 134400/225000 [269:48:35<128:59:15,  5.13s/it]
                                                               
{'loss': '0.8062', 'grad_norm': '1', 'learning_rate': '1.891e-05', 'epoch': '0.5973', 'num_input_tokens_seen': 447964424, 'train_runtime': '9.713e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134400/225000 [269:48:35<128:59:15,  5.13s/it]
 60%|█████▉    | 134401/225000 [269:48:47<176:50:52,  7.03s/it]
 60%|█████▉    | 134402/225000 [269:48:51<152:57:35,  6.08s/it]
 60%|█████▉    | 134403/225000 [269:48:55<136:43:43,  5.43s/it]
 60%|█████▉    | 134404/225000 [269:49:03<156:54:06,  6.23s/it]
 60%|█████▉    | 134405/225000 [269:49:08<151:21:07,  6.01s/it]
 60%|█████▉    | 134406/225000 [269:49:13<145:27:43,  5.78s/it]
 60%|█████▉    | 134407/225000 [269:49:18<139:48:18,  5.56s/it]
 60%|█████▉    | 134408/225000 [269:49:24<139:29:33,  5.54s/it]
 60%|█████▉    | 134409/225000 [269:49:29<134:21:09,  5.34s/it]
 60%|█████▉    | 134410/225000 [269:49:34<135:09:37,  5.37s/it]
 60%|█████▉    | 134411/225000 [269:49:39<132:10:51,  5.25s/it]
 60%|█████▉    | 134412/225000 [269:49:44<130:22:32,  5.18s/it]
 60%|█████▉    | 134413/225000 [269:49:50<132:29:02,  5.27s/it]
 60%|█████▉    | 134414/225000 [269:50:01<179:03:33,  7.12s/it]
 60%|█████▉    | 134415/225000 [269:50:09<184:42:48,  7.34s/it]
 60%|█████▉    | 134416/225000 [269:50:13<156:47:24,  6.23s/it]
 60%|█████▉    | 134417/225000 [269:50:17<146:26:18,  5.82s/it]
 60%|█████▉    | 134418/225000 [269:50:22<140:19:28,  5.58s/it]
 60%|█████▉    | 134419/225000 [269:50:28<137:46:28,  5.48s/it]
 60%|█████▉    | 134420/225000 [269:50:33<134:02:48,  5.33s/it]
 60%|█████▉    | 134421/225000 [269:50:38<134:24:34,  5.34s/it]
 60%|█████▉    | 134422/225000 [269:50:43<134:04:48,  5.33s/it]
 60%|█████▉    | 134423/225000 [269:50:49<132:56:36,  5.28s/it]
 60%|█████▉    | 134424/225000 [269:50:54<131:22:05,  5.22s/it]
 60%|█████▉    | 134425/225000 [269:51:04<168:14:02,  6.69s/it]
 60%|█████▉    | 134426/225000 [269:51:14<193:15:19,  7.68s/it]
 60%|█████▉    | 134427/225000 [269:51:17<161:40:26,  6.43s/it]
 60%|█████▉    | 134428/225000 [269:51:23<157:31:16,  6.26s/it]
 60%|█████▉    | 134429/225000 [269:51:28<149:47:07,  5.95s/it]
 60%|█████▉    | 134430/225000 [269:51:35<153:51:51,  6.12s/it]
 60%|█████▉    | 134431/225000 [269:51:41<154:57:33,  6.16s/it]
 60%|█████▉    | 134432/225000 [269:51:46<144:13:36,  5.73s/it]
 60%|█████▉    | 134433/225000 [269:51:50<135:02:33,  5.37s/it]
 60%|█████▉    | 134434/225000 [269:51:55<128:11:13,  5.10s/it]
 60%|█████▉    | 134435/225000 [269:51:59<124:55:55,  4.97s/it]
 60%|█████▉    | 134436/225000 [269:52:12<179:14:58,  7.13s/it]
 60%|█████▉    | 134437/225000 [269:52:16<158:05:51,  6.28s/it]
 60%|█████▉    | 134438/225000 [269:52:22<156:42:30,  6.23s/it]
 60%|█████▉    | 134439/225000 [269:52:28<158:06:39,  6.29s/it]
 60%|█████▉    | 134440/225000 [269:52:34<150:59:48,  6.00s/it]
 60%|█████▉    | 134441/225000 [269:52:37<132:19:59,  5.26s/it]
 60%|█████▉    | 134442/225000 [269:52:47<162:50:01,  6.47s/it]
 60%|█████▉    | 134443/225000 [269:52:50<139:24:57,  5.54s/it]
 60%|█████▉    | 134444/225000 [269:52:57<150:21:35,  5.98s/it]
 60%|█████▉    | 134445/225000 [269:53:03<148:55:55,  5.92s/it]
 60%|█████▉    | 134446/225000 [269:53:06<130:51:20,  5.20s/it]
 60%|█████▉    | 134447/225000 [269:53:16<167:28:48,  6.66s/it]
 60%|█████▉    | 134448/225000 [269:53:20<143:02:22,  5.69s/it]
 60%|█████▉    | 134449/225000 [269:53:24<133:30:03,  5.31s/it]
 60%|█████▉    | 134450/225000 [269:53:32<153:11:44,  6.09s/it]
                                                               
{'loss': '0.7914', 'grad_norm': '1.145', 'learning_rate': '1.889e-05', 'epoch': '0.5976', 'num_input_tokens_seen': 448094728, 'train_runtime': '9.716e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134450/225000 [269:53:32<153:11:44,  6.09s/it]
 60%|█████▉    | 134451/225000 [269:53:37<143:33:06,  5.71s/it]
 60%|█████▉    | 134452/225000 [269:53:41<134:42:59,  5.36s/it]
 60%|█████▉    | 134453/225000 [269:53:46<130:06:04,  5.17s/it]
 60%|█████▉    | 134454/225000 [269:53:51<124:12:06,  4.94s/it]
 60%|█████▉    | 134455/225000 [269:53:55<122:06:41,  4.86s/it]
 60%|█████▉    | 134456/225000 [269:54:00<123:00:45,  4.89s/it]
 60%|█████▉    | 134457/225000 [269:54:07<133:23:51,  5.30s/it]
 60%|█████▉    | 134458/225000 [269:54:19<185:40:10,  7.38s/it]
 60%|█████▉    | 134459/225000 [269:54:23<163:11:42,  6.49s/it]
 60%|█████▉    | 134460/225000 [269:54:30<168:31:51,  6.70s/it]
 60%|█████▉    | 134461/225000 [269:54:35<151:22:01,  6.02s/it]
 60%|█████▉    | 134462/225000 [269:54:39<137:43:55,  5.48s/it]
 60%|█████▉    | 134463/225000 [269:54:44<131:35:10,  5.23s/it]
 60%|█████▉    | 134464/225000 [269:54:48<125:36:50,  4.99s/it]
 60%|█████▉    | 134465/225000 [269:54:54<132:33:11,  5.27s/it]
 60%|█████▉    | 134466/225000 [269:55:05<173:25:38,  6.90s/it]
 60%|█████▉    | 134467/225000 [269:55:08<148:06:56,  5.89s/it]
 60%|█████▉    | 134468/225000 [269:55:17<167:10:36,  6.65s/it]
 60%|█████▉    | 134469/225000 [269:55:25<178:04:12,  7.08s/it]
 60%|█████▉    | 134470/225000 [269:55:28<150:51:19,  6.00s/it]
 60%|█████▉    | 134471/225000 [269:55:34<148:37:27,  5.91s/it]
 60%|█████▉    | 134472/225000 [269:55:42<164:53:04,  6.56s/it]
 60%|█████▉    | 134473/225000 [269:55:46<141:59:22,  5.65s/it]
 60%|█████▉    | 134474/225000 [269:55:54<164:17:51,  6.53s/it]
 60%|█████▉    | 134475/225000 [269:55:58<145:09:03,  5.77s/it]
 60%|█████▉    | 134476/225000 [269:56:03<134:58:30,  5.37s/it]
 60%|█████▉    | 134477/225000 [269:56:06<124:21:15,  4.95s/it]
 60%|█████▉    | 134478/225000 [269:56:14<144:15:10,  5.74s/it]
 60%|█████▉    | 134479/225000 [269:56:22<162:22:56,  6.46s/it]
 60%|█████▉    | 134480/225000 [269:56:26<141:31:14,  5.63s/it]
 60%|█████▉    | 134481/225000 [269:56:30<131:24:45,  5.23s/it]
 60%|█████▉    | 134482/225000 [269:56:34<123:51:00,  4.93s/it]
 60%|█████▉    | 134483/225000 [269:56:43<147:53:33,  5.88s/it]
 60%|█████▉    | 134484/225000 [269:56:50<160:54:14,  6.40s/it]
 60%|█████▉    | 134485/225000 [269:56:54<143:23:12,  5.70s/it]
 60%|█████▉    | 134486/225000 [269:56:58<130:31:44,  5.19s/it]
 60%|█████▉    | 134487/225000 [269:57:03<123:57:51,  4.93s/it]
 60%|█████▉    | 134488/225000 [269:57:07<119:46:36,  4.76s/it]
 60%|█████▉    | 134489/225000 [269:57:19<173:04:51,  6.88s/it]
 60%|█████▉    | 134490/225000 [269:57:23<154:10:28,  6.13s/it]
 60%|█████▉    | 134491/225000 [269:57:27<135:04:26,  5.37s/it]
 60%|█████▉    | 134492/225000 [269:57:33<138:27:50,  5.51s/it]
 60%|█████▉    | 134493/225000 [269:57:41<162:50:30,  6.48s/it]
 60%|█████▉    | 134494/225000 [269:57:46<149:25:06,  5.94s/it]
 60%|█████▉    | 134495/225000 [269:57:55<169:34:04,  6.74s/it]
 60%|█████▉    | 134496/225000 [269:57:58<147:19:17,  5.86s/it]
 60%|█████▉    | 134497/225000 [269:58:02<131:01:34,  5.21s/it]
 60%|█████▉    | 134498/225000 [269:58:10<148:49:17,  5.92s/it]
 60%|█████▉    | 134499/225000 [269:58:16<153:48:43,  6.12s/it]
 60%|█████▉    | 134500/225000 [269:58:23<161:02:44,  6.41s/it]
                                                               
{'loss': '0.8231', 'grad_norm': '0.9988', 'learning_rate': '1.888e-05', 'epoch': '0.5978', 'num_input_tokens_seen': 448225624, 'train_runtime': '9.719e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134500/225000 [269:58:23<161:02:44,  6.41s/it]
 60%|█████▉    | 134501/225000 [269:58:28<150:34:24,  5.99s/it]
 60%|█████▉    | 134502/225000 [269:58:33<142:15:42,  5.66s/it]
 60%|█████▉    | 134503/225000 [269:58:43<172:17:57,  6.85s/it]
 60%|█████▉    | 134504/225000 [269:58:46<146:06:04,  5.81s/it]
 60%|█████▉    | 134505/225000 [269:58:54<161:04:59,  6.41s/it]
 60%|█████▉    | 134506/225000 [269:58:58<141:29:22,  5.63s/it]
 60%|█████▉    | 134507/225000 [269:59:07<165:06:47,  6.57s/it]
 60%|█████▉    | 134508/225000 [269:59:10<142:45:06,  5.68s/it]
 60%|█████▉    | 134509/225000 [269:59:19<166:51:45,  6.64s/it]
 60%|█████▉    | 134510/225000 [269:59:22<141:33:12,  5.63s/it]
 60%|█████▉    | 134511/225000 [269:59:30<155:49:33,  6.20s/it]
 60%|█████▉    | 134512/225000 [269:59:34<137:27:36,  5.47s/it]
 60%|█████▉    | 134513/225000 [269:59:37<124:49:01,  4.97s/it]
 60%|█████▉    | 134514/225000 [269:59:45<141:51:29,  5.64s/it]
 60%|█████▉    | 134515/225000 [269:59:52<153:36:25,  6.11s/it]
 60%|█████▉    | 134516/225000 [269:59:55<133:04:14,  5.29s/it]
 60%|█████▉    | 134517/225000 [269:59:59<122:25:09,  4.87s/it]
 60%|█████▉    | 134518/225000 [270:00:06<134:30:39,  5.35s/it]
 60%|█████▉    | 134519/225000 [270:00:13<151:49:00,  6.04s/it]
 60%|█████▉    | 134520/225000 [270:00:17<135:41:20,  5.40s/it]
 60%|█████▉    | 134521/225000 [270:00:22<127:37:31,  5.08s/it]
 60%|█████▉    | 134522/225000 [270:00:26<121:30:55,  4.83s/it]
 60%|█████▉    | 134523/225000 [270:00:31<121:58:38,  4.85s/it]
 60%|█████▉    | 134524/225000 [270:00:36<125:22:12,  4.99s/it]
 60%|█████▉    | 134525/225000 [270:00:41<128:27:28,  5.11s/it]
 60%|█████▉    | 134526/225000 [270:00:47<128:31:44,  5.11s/it]
 60%|█████▉    | 134527/225000 [270:00:51<127:29:42,  5.07s/it]
 60%|█████▉    | 134528/225000 [270:00:57<130:36:28,  5.20s/it]
 60%|█████▉    | 134529/225000 [270:01:05<152:50:04,  6.08s/it]
 60%|█████▉    | 134530/225000 [270:01:17<199:55:47,  7.96s/it]
 60%|█████▉    | 134531/225000 [270:01:22<170:53:24,  6.80s/it]
 60%|█████▉    | 134532/225000 [270:01:27<162:28:35,  6.47s/it]
 60%|█████▉    | 134533/225000 [270:01:35<174:08:44,  6.93s/it]
 60%|█████▉    | 134534/225000 [270:01:40<157:13:09,  6.26s/it]
 60%|█████▉    | 134535/225000 [270:01:45<148:04:37,  5.89s/it]
 60%|█████▉    | 134536/225000 [270:01:50<139:37:43,  5.56s/it]
 60%|█████▉    | 134537/225000 [270:01:55<134:44:31,  5.36s/it]
 60%|█████▉    | 134538/225000 [270:02:00<132:12:52,  5.26s/it]
 60%|█████▉    | 134539/225000 [270:02:05<132:24:35,  5.27s/it]
 60%|█████▉    | 134540/225000 [270:02:13<156:31:07,  6.23s/it]
 60%|█████▉    | 134541/225000 [270:02:24<190:15:58,  7.57s/it]
 60%|█████▉    | 134542/225000 [270:02:29<171:19:15,  6.82s/it]
 60%|█████▉    | 134543/225000 [270:02:34<154:25:05,  6.15s/it]
 60%|█████▉    | 134544/225000 [270:02:39<143:49:38,  5.72s/it]
 60%|█████▉    | 134545/225000 [270:02:44<139:07:22,  5.54s/it]
 60%|█████▉    | 134546/225000 [270:02:49<137:11:03,  5.46s/it]
 60%|█████▉    | 134547/225000 [270:02:53<128:35:41,  5.12s/it]
 60%|█████▉    | 134548/225000 [270:02:58<128:40:32,  5.12s/it]
 60%|█████▉    | 134549/225000 [270:03:03<123:58:14,  4.93s/it]
 60%|█████▉    | 134550/225000 [270:03:12<156:13:56,  6.22s/it]
                                                               
{'loss': '0.7978', 'grad_norm': '1.15', 'learning_rate': '1.886e-05', 'epoch': '0.598', 'num_input_tokens_seen': 448356720, 'train_runtime': '9.722e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134550/225000 [270:03:12<156:13:56,  6.22s/it]
 60%|█████▉    | 134551/225000 [270:03:21<179:22:46,  7.14s/it]
 60%|█████▉    | 134552/225000 [270:03:28<178:03:57,  7.09s/it]
 60%|█████▉    | 134553/225000 [270:03:32<152:13:11,  6.06s/it]
 60%|█████▉    | 134554/225000 [270:03:36<137:25:32,  5.47s/it]
 60%|█████▉    | 134555/225000 [270:03:46<173:44:38,  6.92s/it]
 60%|█████▉    | 134556/225000 [270:03:50<152:11:07,  6.06s/it]
 60%|█████▉    | 134557/225000 [270:03:54<136:09:59,  5.42s/it]
 60%|█████▉    | 134558/225000 [270:03:58<123:14:05,  4.91s/it]
 60%|█████▉    | 134559/225000 [270:04:10<173:47:28,  6.92s/it]
 60%|█████▉    | 134560/225000 [270:04:14<150:50:16,  6.00s/it]
 60%|█████▉    | 134561/225000 [270:04:17<133:41:42,  5.32s/it]
 60%|█████▉    | 134562/225000 [270:04:22<128:21:34,  5.11s/it]
 60%|█████▉    | 134563/225000 [270:04:31<160:30:56,  6.39s/it]
 60%|█████▉    | 134564/225000 [270:04:35<140:41:39,  5.60s/it]
 60%|█████▉    | 134565/225000 [270:04:41<140:23:36,  5.59s/it]
 60%|█████▉    | 134566/225000 [270:04:50<166:54:47,  6.64s/it]
 60%|█████▉    | 134567/225000 [270:04:54<150:48:02,  6.00s/it]
 60%|█████▉    | 134568/225000 [270:05:01<159:45:45,  6.36s/it]
 60%|█████▉    | 134569/225000 [270:05:05<137:35:53,  5.48s/it]
 60%|█████▉    | 134570/225000 [270:05:12<147:40:27,  5.88s/it]
 60%|█████▉    | 134571/225000 [270:05:18<153:32:31,  6.11s/it]
 60%|█████▉    | 134572/225000 [270:05:23<141:14:17,  5.62s/it]
 60%|█████▉    | 134573/225000 [270:05:27<129:04:14,  5.14s/it]
 60%|█████▉    | 134574/225000 [270:05:31<125:04:57,  4.98s/it]
 60%|█████▉    | 134575/225000 [270:05:44<181:01:28,  7.21s/it]
 60%|█████▉    | 134576/225000 [270:05:47<151:34:04,  6.03s/it]
 60%|█████▉    | 134577/225000 [270:05:52<143:12:05,  5.70s/it]
 60%|█████▉    | 134578/225000 [270:06:01<170:21:19,  6.78s/it]
 60%|█████▉    | 134579/225000 [270:06:05<143:41:47,  5.72s/it]
 60%|█████▉    | 134580/225000 [270:06:13<163:57:30,  6.53s/it]
 60%|█████▉    | 134581/225000 [270:06:18<154:31:24,  6.15s/it]
 60%|█████▉    | 134582/225000 [270:06:24<153:51:29,  6.13s/it]
 60%|█████▉    | 134583/225000 [270:06:28<133:58:58,  5.33s/it]
 60%|█████▉    | 134584/225000 [270:06:32<127:56:49,  5.09s/it]
 60%|█████▉    | 134585/225000 [270:06:41<154:28:56,  6.15s/it]
 60%|█████▉    | 134586/225000 [270:06:44<133:56:03,  5.33s/it]
 60%|█████▉    | 134587/225000 [270:06:51<144:40:59,  5.76s/it]
 60%|█████▉    | 134588/225000 [270:06:56<134:22:51,  5.35s/it]
 60%|█████▉    | 134589/225000 [270:07:00<130:02:02,  5.18s/it]
 60%|█████▉    | 134590/225000 [270:07:05<126:26:54,  5.04s/it]
 60%|█████▉    | 134591/225000 [270:07:10<126:24:26,  5.03s/it]
 60%|█████▉    | 134592/225000 [270:07:15<125:09:39,  4.98s/it]
 60%|█████▉    | 134593/225000 [270:07:20<123:48:53,  4.93s/it]
 60%|█████▉    | 134594/225000 [270:07:25<122:56:07,  4.90s/it]
 60%|█████▉    | 134595/225000 [270:07:29<121:18:25,  4.83s/it]
 60%|█████▉    | 134596/225000 [270:07:40<169:40:43,  6.76s/it]
 60%|█████▉    | 134597/225000 [270:07:50<194:29:40,  7.75s/it]
 60%|█████▉    | 134598/225000 [270:07:55<168:35:34,  6.71s/it]
 60%|█████▉    | 134599/225000 [270:07:59<149:34:27,  5.96s/it]
 60%|█████▉    | 134600/225000 [270:08:03<137:25:21,  5.47s/it]
                                                               
{'loss': '0.7987', 'grad_norm': '1.099', 'learning_rate': '1.884e-05', 'epoch': '0.5982', 'num_input_tokens_seen': 448486344, 'train_runtime': '9.725e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134600/225000 [270:08:03<137:25:21,  5.47s/it]
 60%|█████▉    | 134601/225000 [270:08:07<126:43:53,  5.05s/it]
 60%|█████▉    | 134602/225000 [270:08:16<155:54:40,  6.21s/it]
 60%|█████▉    | 134603/225000 [270:08:25<171:30:56,  6.83s/it]
 60%|█████▉    | 134604/225000 [270:08:28<146:58:14,  5.85s/it]
 60%|█████▉    | 134605/225000 [270:08:38<174:48:26,  6.96s/it]
 60%|█████▉    | 134606/225000 [270:08:41<147:21:21,  5.87s/it]
 60%|█████▉    | 134607/225000 [270:08:51<179:17:10,  7.14s/it]
 60%|█████▉    | 134608/225000 [270:08:56<159:01:53,  6.33s/it]
 60%|█████▉    | 134609/225000 [270:09:00<144:55:07,  5.77s/it]
 60%|█████▉    | 134610/225000 [270:09:04<132:43:36,  5.29s/it]
 60%|█████▉    | 134611/225000 [270:09:08<124:25:26,  4.96s/it]
 60%|█████▉    | 134612/225000 [270:09:14<132:45:53,  5.29s/it]
 60%|█████▉    | 134613/225000 [270:09:24<166:17:30,  6.62s/it]
 60%|█████▉    | 134614/225000 [270:09:28<142:38:13,  5.68s/it]
 60%|█████▉    | 134615/225000 [270:09:32<132:41:25,  5.29s/it]
 60%|█████▉    | 134616/225000 [270:09:37<127:12:47,  5.07s/it]
 60%|█████▉    | 134617/225000 [270:09:41<122:09:24,  4.87s/it]
 60%|█████▉    | 134618/225000 [270:09:53<174:31:35,  6.95s/it]
 60%|█████▉    | 134619/225000 [270:09:58<163:27:16,  6.51s/it]
 60%|█████▉    | 134620/225000 [270:10:03<146:33:32,  5.84s/it]
 60%|█████▉    | 134621/225000 [270:10:11<165:25:52,  6.59s/it]
 60%|█████▉    | 134622/225000 [270:10:17<159:13:46,  6.34s/it]
 60%|█████▉    | 134623/225000 [270:10:21<142:55:19,  5.69s/it]
 60%|█████▉    | 134624/225000 [270:10:27<145:37:59,  5.80s/it]
 60%|█████▉    | 134625/225000 [270:10:33<146:08:04,  5.82s/it]
 60%|█████▉    | 134626/225000 [270:10:36<129:09:25,  5.14s/it]
 60%|█████▉    | 134627/225000 [270:10:45<154:34:16,  6.16s/it]
 60%|█████▉    | 134628/225000 [270:10:49<137:32:07,  5.48s/it]
 60%|█████▉    | 134629/225000 [270:10:53<127:49:14,  5.09s/it]
 60%|█████▉    | 134630/225000 [270:10:57<123:34:42,  4.92s/it]
 60%|█████▉    | 134631/225000 [270:11:02<119:26:50,  4.76s/it]
 60%|█████▉    | 134632/225000 [270:11:06<116:26:38,  4.64s/it]
 60%|█████▉    | 134633/225000 [270:11:16<154:40:22,  6.16s/it]
 60%|█████▉    | 134634/225000 [270:11:24<167:52:08,  6.69s/it]
 60%|█████▉    | 134635/225000 [270:11:27<143:03:42,  5.70s/it]
 60%|█████▉    | 134636/225000 [270:11:37<173:22:34,  6.91s/it]
 60%|█████▉    | 134637/225000 [270:11:40<146:18:59,  5.83s/it]
 60%|█████▉    | 134638/225000 [270:11:48<159:09:46,  6.34s/it]
 60%|█████▉    | 134639/225000 [270:11:52<140:59:01,  5.62s/it]
 60%|█████▉    | 134640/225000 [270:11:57<134:52:07,  5.37s/it]
 60%|█████▉    | 134641/225000 [270:12:02<132:37:03,  5.28s/it]
 60%|█████▉    | 134642/225000 [270:12:06<128:56:44,  5.14s/it]
 60%|█████▉    | 134643/225000 [270:12:11<124:24:42,  4.96s/it]
 60%|█████▉    | 134644/225000 [270:12:16<123:51:52,  4.94s/it]
 60%|█████▉    | 134645/225000 [270:12:21<122:49:19,  4.89s/it]
 60%|█████▉    | 134646/225000 [270:12:27<135:21:24,  5.39s/it]
 60%|█████▉    | 134647/225000 [270:12:40<192:26:52,  7.67s/it]
 60%|█████▉    | 134648/225000 [270:12:45<169:42:12,  6.76s/it]
 60%|█████▉    | 134649/225000 [270:12:50<159:45:39,  6.37s/it]
 60%|█████▉    | 134650/225000 [270:12:58<167:25:19,  6.67s/it]
                                                               
{'loss': '0.81', 'grad_norm': '1.044', 'learning_rate': '1.882e-05', 'epoch': '0.5984', 'num_input_tokens_seen': 448616368, 'train_runtime': '9.728e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134650/225000 [270:12:58<167:25:19,  6.67s/it]
 60%|█████▉    | 134651/225000 [270:13:02<146:44:55,  5.85s/it]
 60%|█████▉    | 134652/225000 [270:13:05<132:15:33,  5.27s/it]
 60%|█████▉    | 134653/225000 [270:13:16<173:52:04,  6.93s/it]
 60%|█████▉    | 134654/225000 [270:13:20<151:57:17,  6.05s/it]
 60%|█████▉    | 134655/225000 [270:13:25<138:17:08,  5.51s/it]
 60%|█████▉    | 134656/225000 [270:13:29<130:11:52,  5.19s/it]
 60%|█████▉    | 134657/225000 [270:13:37<154:49:31,  6.17s/it]
 60%|█████▉    | 134658/225000 [270:13:45<164:50:54,  6.57s/it]
 60%|█████▉    | 134659/225000 [270:13:49<144:38:16,  5.76s/it]
 60%|█████▉    | 134660/225000 [270:13:58<167:34:01,  6.68s/it]
 60%|█████▉    | 134661/225000 [270:14:01<141:59:04,  5.66s/it]
 60%|█████▉    | 134662/225000 [270:14:09<157:49:06,  6.29s/it]
 60%|█████▉    | 134663/225000 [270:14:13<146:12:05,  5.83s/it]
 60%|█████▉    | 134664/225000 [270:14:17<128:50:46,  5.13s/it]
 60%|█████▉    | 134665/225000 [270:14:26<161:03:01,  6.42s/it]
 60%|█████▉    | 134666/225000 [270:14:30<137:28:30,  5.48s/it]
 60%|█████▉    | 134667/225000 [270:14:36<146:49:13,  5.85s/it]
 60%|█████▉    | 134668/225000 [270:14:43<149:12:51,  5.95s/it]
 60%|█████▉    | 134669/225000 [270:14:46<129:29:55,  5.16s/it]
 60%|█████▉    | 134670/225000 [270:14:55<158:36:21,  6.32s/it]
 60%|█████▉    | 134671/225000 [270:14:59<142:08:37,  5.67s/it]
 60%|█████▉    | 134672/225000 [270:15:04<135:15:07,  5.39s/it]
 60%|█████▉    | 134673/225000 [270:15:09<130:39:26,  5.21s/it]
 60%|█████▉    | 134674/225000 [270:15:14<129:07:10,  5.15s/it]
 60%|█████▉    | 134675/225000 [270:15:18<126:56:21,  5.06s/it]
 60%|█████▉    | 134676/225000 [270:15:22<117:23:14,  4.68s/it]
 60%|█████▉    | 134677/225000 [270:15:27<119:32:12,  4.76s/it]
 60%|█████▉    | 134678/225000 [270:15:40<180:18:46,  7.19s/it]
 60%|█████▉    | 134679/225000 [270:15:45<162:50:51,  6.49s/it]
 60%|█████▉    | 134680/225000 [270:15:51<160:50:34,  6.41s/it]
 60%|█████▉    | 134681/225000 [270:15:57<159:38:03,  6.36s/it]
 60%|█████▉    | 134682/225000 [270:16:01<140:31:52,  5.60s/it]
 60%|█████▉    | 134683/225000 [270:16:05<127:00:11,  5.06s/it]
 60%|█████▉    | 134684/225000 [270:16:09<123:05:18,  4.91s/it]
 60%|█████▉    | 134685/225000 [270:16:17<146:06:39,  5.82s/it]
 60%|█████▉    | 134686/225000 [270:16:24<152:07:16,  6.06s/it]
 60%|█████▉    | 134687/225000 [270:16:30<153:18:06,  6.11s/it]
 60%|█████▉    | 134688/225000 [270:16:36<151:33:17,  6.04s/it]
 60%|█████▉    | 134689/225000 [270:16:44<162:27:35,  6.48s/it]
 60%|█████▉    | 134690/225000 [270:16:47<142:01:22,  5.66s/it]
 60%|█████▉    | 134691/225000 [270:16:52<132:43:07,  5.29s/it]
 60%|█████▉    | 134692/225000 [270:16:56<124:27:31,  4.96s/it]
 60%|█████▉    | 134693/225000 [270:17:00<119:35:51,  4.77s/it]
 60%|█████▉    | 134694/225000 [270:17:11<160:24:43,  6.39s/it]
 60%|█████▉    | 134695/225000 [270:17:18<167:29:03,  6.68s/it]
 60%|█████▉    | 134696/225000 [270:17:22<148:59:14,  5.94s/it]
 60%|█████▉    | 134697/225000 [270:17:27<137:46:10,  5.49s/it]
 60%|█████▉    | 134698/225000 [270:17:31<129:47:38,  5.17s/it]
 60%|█████▉    | 134699/225000 [270:17:36<129:38:33,  5.17s/it]
 60%|█████▉    | 134700/225000 [270:17:48<183:16:54,  7.31s/it]
                                                               
{'loss': '0.808', 'grad_norm': '1.057', 'learning_rate': '1.88e-05', 'epoch': '0.5987', 'num_input_tokens_seen': 448745880, 'train_runtime': '9.731e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134700/225000 [270:17:48<183:16:54,  7.31s/it]
 60%|█████▉    | 134701/225000 [270:17:53<161:50:48,  6.45s/it]
 60%|█████▉    | 134702/225000 [270:17:58<149:08:48,  5.95s/it]
 60%|█████▉    | 134703/225000 [270:18:02<139:49:01,  5.57s/it]
 60%|█████▉    | 134704/225000 [270:18:07<131:02:29,  5.22s/it]
 60%|█████▉    | 134705/225000 [270:18:11<123:21:48,  4.92s/it]
 60%|█████▉    | 134706/225000 [270:18:22<172:13:06,  6.87s/it]
 60%|█████▉    | 134707/225000 [270:18:27<154:50:38,  6.17s/it]
 60%|█████▉    | 134708/225000 [270:18:33<150:28:03,  6.00s/it]
 60%|█████▉    | 134709/225000 [270:18:40<159:24:09,  6.36s/it]
 60%|█████▉    | 134710/225000 [270:18:43<139:16:09,  5.55s/it]
 60%|█████▉    | 134711/225000 [270:18:47<125:43:04,  5.01s/it]
 60%|█████▉    | 134712/225000 [270:18:58<165:59:35,  6.62s/it]
 60%|█████▉    | 134713/225000 [270:19:02<146:39:54,  5.85s/it]
 60%|█████▉    | 134714/225000 [270:19:05<131:31:34,  5.24s/it]
 60%|█████▉    | 134715/225000 [270:19:10<123:29:24,  4.92s/it]
 60%|█████▉    | 134716/225000 [270:19:16<135:40:33,  5.41s/it]
 60%|█████▉    | 134717/225000 [270:19:25<162:03:38,  6.46s/it]
 60%|█████▉    | 134718/225000 [270:19:30<149:02:33,  5.94s/it]
 60%|█████▉    | 134719/225000 [270:19:34<138:34:06,  5.53s/it]
 60%|█████▉    | 134720/225000 [270:19:39<129:43:31,  5.17s/it]
 60%|█████▉    | 134721/225000 [270:19:43<124:56:14,  4.98s/it]
 60%|█████▉    | 134722/225000 [270:19:47<118:28:10,  4.72s/it]
 60%|█████▉    | 134723/225000 [270:20:00<179:46:08,  7.17s/it]
 60%|█████▉    | 134724/225000 [270:20:05<162:22:44,  6.48s/it]
 60%|█████▉    | 134725/225000 [270:20:09<140:00:25,  5.58s/it]
 60%|█████▉    | 134726/225000 [270:20:18<166:21:04,  6.63s/it]
 60%|█████▉    | 134727/225000 [270:20:22<152:17:34,  6.07s/it]
 60%|█████▉    | 134728/225000 [270:20:29<157:26:21,  6.28s/it]
 60%|█████▉    | 134729/225000 [270:20:32<134:33:29,  5.37s/it]
 60%|█████▉    | 134730/225000 [270:20:37<126:14:38,  5.03s/it]
 60%|█████▉    | 134731/225000 [270:20:49<178:07:38,  7.10s/it]
 60%|█████▉    | 134732/225000 [270:20:52<149:40:02,  5.97s/it]
 60%|█████▉    | 134733/225000 [270:20:59<155:25:19,  6.20s/it]
 60%|█████▉    | 134734/225000 [270:21:02<135:52:52,  5.42s/it]
 60%|█████▉    | 134735/225000 [270:21:12<167:44:31,  6.69s/it]
 60%|█████▉    | 134736/225000 [270:21:16<148:48:16,  5.93s/it]
 60%|█████▉    | 134737/225000 [270:21:20<134:53:49,  5.38s/it]
 60%|█████▉    | 134738/225000 [270:21:26<135:04:46,  5.39s/it]
 60%|█████▉    | 134739/225000 [270:21:36<171:59:44,  6.86s/it]
 60%|█████▉    | 134740/225000 [270:21:39<147:28:28,  5.88s/it]
 60%|█████▉    | 134741/225000 [270:21:48<163:54:46,  6.54s/it]
 60%|█████▉    | 134742/225000 [270:21:52<146:59:48,  5.86s/it]
 60%|█████▉    | 134743/225000 [270:21:56<137:21:24,  5.48s/it]
 60%|█████▉    | 134744/225000 [270:22:01<126:52:19,  5.06s/it]
 60%|█████▉    | 134745/225000 [270:22:06<130:20:45,  5.20s/it]
 60%|█████▉    | 134746/225000 [270:22:17<176:49:48,  7.05s/it]
 60%|█████▉    | 134747/225000 [270:22:21<150:07:19,  5.99s/it]
 60%|█████▉    | 134748/225000 [270:22:29<164:39:39,  6.57s/it]
 60%|█████▉    | 134749/225000 [270:22:32<141:14:20,  5.63s/it]
 60%|█████▉    | 134750/225000 [270:22:39<147:09:05,  5.87s/it]
                                                               
{'loss': '0.8113', 'grad_norm': '1.191', 'learning_rate': '1.879e-05', 'epoch': '0.5989', 'num_input_tokens_seen': 448875912, 'train_runtime': '9.734e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134750/225000 [270:22:39<147:09:05,  5.87s/it]
 60%|█████▉    | 134751/225000 [270:22:47<162:43:59,  6.49s/it]
 60%|█████▉    | 134752/225000 [270:22:51<143:52:26,  5.74s/it]
 60%|█████▉    | 134753/225000 [270:22:55<130:07:53,  5.19s/it]
 60%|█████▉    | 134754/225000 [270:22:59<126:39:28,  5.05s/it]
 60%|█████▉    | 134755/225000 [270:23:04<124:42:10,  4.97s/it]
 60%|█████▉    | 134756/225000 [270:23:09<122:46:22,  4.90s/it]
 60%|█████▉    | 134757/225000 [270:23:14<121:41:24,  4.85s/it]
 60%|█████▉    | 134758/225000 [270:23:26<175:34:27,  7.00s/it]
 60%|█████▉    | 134759/225000 [270:23:31<161:58:58,  6.46s/it]
 60%|█████▉    | 134760/225000 [270:23:35<147:24:10,  5.88s/it]
 60%|█████▉    | 134761/225000 [270:23:43<161:21:24,  6.44s/it]
 60%|█████▉    | 134762/225000 [270:23:46<137:43:23,  5.49s/it]
 60%|█████▉    | 134763/225000 [270:23:55<162:47:53,  6.49s/it]
 60%|█████▉    | 134764/225000 [270:23:59<139:10:45,  5.55s/it]
 60%|█████▉    | 134765/225000 [270:24:03<131:04:10,  5.23s/it]
 60%|█████▉    | 134766/225000 [270:24:10<147:50:33,  5.90s/it]
 60%|█████▉    | 134767/225000 [270:24:14<133:26:59,  5.32s/it]
 60%|█████▉    | 134768/225000 [270:24:20<132:27:03,  5.28s/it]
 60%|█████▉    | 134769/225000 [270:24:24<129:26:38,  5.16s/it]
 60%|█████▉    | 134770/225000 [270:24:29<127:51:03,  5.10s/it]
 60%|█████▉    | 134771/225000 [270:24:34<125:36:20,  5.01s/it]
 60%|█████▉    | 134772/225000 [270:24:39<122:39:22,  4.89s/it]
 60%|█████▉    | 134773/225000 [270:24:46<137:24:56,  5.48s/it]
 60%|█████▉    | 134774/225000 [270:24:57<180:59:40,  7.22s/it]
 60%|█████▉    | 134775/225000 [270:25:00<150:58:30,  6.02s/it]
 60%|█████▉    | 134776/225000 [270:25:07<153:46:41,  6.14s/it]
 60%|█████▉    | 134777/225000 [270:25:12<146:12:23,  5.83s/it]
 60%|█████▉    | 134778/225000 [270:25:21<168:07:03,  6.71s/it]
 60%|█████▉    | 134779/225000 [270:25:25<149:49:08,  5.98s/it]
 60%|█████▉    | 134780/225000 [270:25:29<135:38:16,  5.41s/it]
 60%|█████▉    | 134781/225000 [270:25:33<128:09:54,  5.11s/it]
 60%|█████▉    | 134782/225000 [270:25:44<171:54:15,  6.86s/it]
 60%|█████▉    | 134783/225000 [270:25:48<149:15:46,  5.96s/it]
 60%|█████▉    | 134784/225000 [270:25:57<170:51:45,  6.82s/it]
 60%|█████▉    | 134785/225000 [270:26:03<163:36:36,  6.53s/it]
 60%|█████▉    | 134786/225000 [270:26:08<151:25:25,  6.04s/it]
 60%|█████▉    | 134787/225000 [270:26:12<136:15:28,  5.44s/it]
 60%|█████▉    | 134788/225000 [270:26:15<121:59:32,  4.87s/it]
 60%|█████▉    | 134789/225000 [270:26:25<159:44:05,  6.37s/it]
 60%|█████▉    | 134790/225000 [270:26:29<144:03:40,  5.75s/it]
 60%|█████▉    | 134791/225000 [270:26:34<136:13:50,  5.44s/it]
 60%|█████▉    | 134792/225000 [270:26:39<133:05:38,  5.31s/it]
 60%|█████▉    | 134793/225000 [270:26:44<132:06:46,  5.27s/it]
 60%|█████▉    | 134794/225000 [270:26:49<129:48:06,  5.18s/it]
 60%|█████▉    | 134795/225000 [270:26:54<127:42:04,  5.10s/it]
 60%|█████▉    | 134796/225000 [270:26:59<126:20:12,  5.04s/it]
 60%|█████▉    | 134797/225000 [270:27:10<173:53:50,  6.94s/it]
 60%|█████▉    | 134798/225000 [270:27:15<153:57:00,  6.14s/it]
 60%|█████▉    | 134799/225000 [270:27:20<143:42:13,  5.74s/it]
 60%|█████▉    | 134800/225000 [270:27:28<163:58:04,  6.54s/it]
                                                               
{'loss': '0.8125', 'grad_norm': '1.043', 'learning_rate': '1.877e-05', 'epoch': '0.5991', 'num_input_tokens_seen': 449004864, 'train_runtime': '9.736e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134800/225000 [270:27:28<163:58:04,  6.54s/it]
 60%|█████▉    | 134801/225000 [270:27:32<148:19:16,  5.92s/it]
 60%|█████▉    | 134802/225000 [270:27:37<137:07:14,  5.47s/it]
 60%|█████▉    | 134803/225000 [270:27:41<128:26:03,  5.13s/it]
 60%|█████▉    | 134804/225000 [270:27:46<123:04:43,  4.91s/it]
 60%|█████▉    | 134805/225000 [270:27:50<119:24:36,  4.77s/it]
 60%|█████▉    | 134806/225000 [270:27:55<119:39:42,  4.78s/it]
 60%|█████▉    | 134807/225000 [270:28:07<174:40:16,  6.97s/it]
 60%|█████▉    | 134808/225000 [270:28:14<175:28:12,  7.00s/it]
 60%|█████▉    | 134809/225000 [270:28:18<153:44:31,  6.14s/it]
 60%|█████▉    | 134810/225000 [270:28:25<162:15:46,  6.48s/it]
 60%|█████▉    | 134811/225000 [270:28:30<145:33:44,  5.81s/it]
 60%|█████▉    | 134812/225000 [270:28:34<135:36:07,  5.41s/it]
 60%|█████▉    | 134813/225000 [270:28:39<132:51:51,  5.30s/it]
 60%|█████▉    | 134814/225000 [270:28:44<128:12:23,  5.12s/it]
 60%|█████▉    | 134815/225000 [270:28:48<121:59:35,  4.87s/it]
 60%|█████▉    | 134816/225000 [270:28:59<164:37:52,  6.57s/it]
 60%|█████▉    | 134817/225000 [270:29:07<177:46:35,  7.10s/it]
 60%|█████▉    | 134818/225000 [270:29:11<152:41:26,  6.10s/it]
 60%|█████▉    | 134819/225000 [270:29:20<174:19:07,  6.96s/it]
 60%|█████▉    | 134820/225000 [270:29:23<148:27:29,  5.93s/it]
 60%|█████▉    | 134821/225000 [270:29:31<160:00:10,  6.39s/it]
 60%|█████▉    | 134822/225000 [270:29:36<148:16:06,  5.92s/it]
 60%|█████▉    | 134823/225000 [270:29:39<131:09:05,  5.24s/it]
 60%|█████▉    | 134824/225000 [270:29:47<150:55:51,  6.03s/it]
 60%|█████▉    | 134825/225000 [270:29:52<141:15:12,  5.64s/it]
 60%|█████▉    | 134826/225000 [270:29:57<135:47:11,  5.42s/it]
 60%|█████▉    | 134827/225000 [270:30:01<129:15:59,  5.16s/it]
 60%|█████▉    | 134828/225000 [270:30:06<125:32:30,  5.01s/it]
 60%|█████▉    | 134829/225000 [270:30:11<123:33:39,  4.93s/it]
 60%|█████▉    | 134830/225000 [270:30:16<124:37:53,  4.98s/it]
 60%|█████▉    | 134831/225000 [270:30:20<122:53:54,  4.91s/it]
 60%|█████▉    | 134832/225000 [270:30:33<181:01:53,  7.23s/it]
 60%|█████▉    | 134833/225000 [270:30:39<168:38:10,  6.73s/it]
 60%|█████▉    | 134834/225000 [270:30:42<145:14:39,  5.80s/it]
 60%|█████▉    | 134835/225000 [270:30:46<130:44:42,  5.22s/it]
 60%|█████▉    | 134836/225000 [270:30:51<124:54:09,  4.99s/it]
 60%|█████▉    | 134837/225000 [270:31:00<159:29:26,  6.37s/it]
 60%|█████▉    | 134838/225000 [270:31:07<161:12:04,  6.44s/it]
 60%|█████▉    | 134839/225000 [270:31:11<143:11:55,  5.72s/it]
 60%|█████▉    | 134840/225000 [270:31:15<133:02:05,  5.31s/it]
 60%|█████▉    | 134841/225000 [270:31:21<137:20:19,  5.48s/it]
 60%|█████▉    | 134842/225000 [270:31:31<168:20:56,  6.72s/it]
 60%|█████▉    | 134843/225000 [270:31:36<156:53:30,  6.26s/it]
 60%|█████▉    | 134844/225000 [270:31:41<145:34:40,  5.81s/it]
 60%|█████▉    | 134845/225000 [270:31:46<138:52:14,  5.55s/it]
 60%|█████▉    | 134846/225000 [270:31:51<135:02:25,  5.39s/it]
 60%|█████▉    | 134847/225000 [270:31:55<127:59:55,  5.11s/it]
 60%|█████▉    | 134848/225000 [270:32:00<125:40:02,  5.02s/it]
 60%|█████▉    | 134849/225000 [270:32:05<125:17:57,  5.00s/it]
 60%|█████▉    | 134850/225000 [270:32:10<126:52:44,  5.07s/it]
                                                               
{'loss': '0.7875', 'grad_norm': '1.122', 'learning_rate': '1.875e-05', 'epoch': '0.5993', 'num_input_tokens_seen': 449133440, 'train_runtime': '9.739e+05', 'train_tokens_per_second': '461.2'}

 60%|█████▉    | 134850/225000 [270:32:10<126:52:44,  5.07s/it]
 60%|█████▉    | 134851/225000 [270:32:23<183:21:24,  7.32s/it]
 60%|█████▉    | 134852/225000 [270:32:30<183:23:13,  7.32s/it]
 60%|█████▉    | 134853/225000 [270:32:34<157:00:26,  6.27s/it]
 60%|█████▉    | 134854/225000 [270:32:37<137:28:33,  5.49s/it]
 60%|█████▉    | 134855/225000 [270:32:45<151:28:38,  6.05s/it]
 60%|█████▉    | 134856/225000 [270:32:53<164:09:00,  6.56s/it]
 60%|█████▉    | 134857/225000 [270:32:58<153:20:49,  6.12s/it]
 60%|█████▉    | 134858/225000 [270:33:04<155:51:08,  6.22s/it]
 60%|█████▉    | 134859/225000 [270:33:08<139:14:20,  5.56s/it]
 60%|█████▉    | 134860/225000 [270:33:15<150:20:04,  6.00s/it]
 60%|█████▉    | 134861/225000 [270:33:21<148:35:03,  5.93s/it]
 60%|█████▉    | 134862/225000 [270:33:27<151:17:50,  6.04s/it]
 60%|█████▉    | 134863/225000 [270:33:30<129:40:22,  5.18s/it]
 60%|█████▉    | 134864/225000 [270:33:38<147:36:28,  5.90s/it]
 60%|█████▉    | 134865/225000 [270:33:43<142:05:58,  5.68s/it]
 60%|█████▉    | 134866/225000 [270:33:47<129:29:05,  5.17s/it]
 60%|█████▉    | 134867/225000 [270:33:54<139:48:31,  5.58s/it]
 60%|█████▉    | 134868/225000 [270:34:01<152:09:24,  6.08s/it]
 60%|█████▉    | 134869/225000 [270:34:04<132:44:07,  5.30s/it]
 60%|█████▉    | 134870/225000 [270:34:09<128:48:31,  5.14s/it]
 60%|█████▉    | 134871/225000 [270:34:17<151:12:02,  6.04s/it]
 60%|█████▉    | 134872/225000 [270:34:21<133:33:31,  5.33s/it]
 60%|█████▉    | 134873/225000 [270:34:29<149:54:22,  5.99s/it]
 60%|█████▉    | 134874/225000 [270:34:32<131:18:28,  5.24s/it]
 60%|█████▉    | 134875/225000 [270:34:41<156:56:33,  6.27s/it]
 60%|█████▉    | 134876/225000 [270:34:45<142:52:04,  5.71s/it]
 60%|█████▉    | 134877/225000 [270:34:50<133:16:45,  5.32s/it]
 60%|█████▉    | 134878/225000 [270:34:54<125:40:01,  5.02s/it]
 60%|█████▉    | 134879/225000 [270:35:05<169:04:35,  6.75s/it]
 60%|█████▉    | 134880/225000 [270:35:08<143:46:16,  5.74s/it]
 60%|█████▉    | 134881/225000 [270:35:14<143:16:59,  5.72s/it]
 60%|█████▉    | 134882/225000 [270:35:21<153:33:36,  6.13s/it]
 60%|█████▉    | 134883/225000 [270:35:28<158:51:39,  6.35s/it]
 60%|█████▉    | 134884/225000 [270:35:31<137:42:46,  5.50s/it]
 60%|█████▉    | 134885/225000 [270:35:41<174:00:29,  6.95s/it]
 60%|█████▉    | 134886/225000 [270:35:45<144:31:49,  5.77s/it]
 60%|█████▉    | 134887/225000 [270:35:48<128:49:05,  5.15s/it]
 60%|█████▉    | 134888/225000 [270:35:53<123:51:32,  4.95s/it]
 60%|█████▉    | 134889/225000 [270:35:57<119:42:22,  4.78s/it]
 60%|█████▉    | 134890/225000 [270:36:08<168:04:50,  6.72s/it]
 60%|█████▉    | 134891/225000 [270:36:12<144:10:01,  5.76s/it]
 60%|█████▉    | 134892/225000 [270:36:16<130:35:35,  5.22s/it]
 60%|█████▉    | 134893/225000 [270:36:25<158:03:40,  6.31s/it]
 60%|█████▉    | 134894/225000 [270:36:31<157:42:39,  6.30s/it]
 60%|█████▉    | 134895/225000 [270:36:35<139:28:04,  5.57s/it]
 60%|█████▉    | 134896/225000 [270:36:39<125:35:03,  5.02s/it]
 60%|█████▉    | 134897/225000 [270:36:50<174:55:55,  6.99s/it]
 60%|█████▉    | 134898/225000 [270:36:54<152:30:25,  6.09s/it]
 60%|█████▉    | 134899/225000 [270:36:58<136:28:28,  5.45s/it]
 60%|█████▉    | 134900/225000 [270:37:03<129:22:22,  5.17s/it]
                                                               
{'loss': '0.7995', 'grad_norm': '1.01', 'learning_rate': '1.873e-05', 'epoch': '0.5996', 'num_input_tokens_seen': 449262832, 'train_runtime': '9.742e+05', 'train_tokens_per_second': '461.1'}

 60%|█████▉    | 134900/225000 [270:37:03<129:22:22,  5.17s/it]
 60%|█████▉    | 134901/225000 [270:37:14<174:41:27,  6.98s/it]
 60%|█████▉    | 134902/225000 [270:37:17<148:25:53,  5.93s/it]
 60%|█████▉    | 134903/225000 [270:37:21<134:01:25,  5.36s/it]
 60%|█████▉    | 134904/225000 [270:37:26<129:43:10,  5.18s/it]
 60%|█████▉    | 134905/225000 [270:37:31<127:55:37,  5.11s/it]
 60%|█████▉    | 134906/225000 [270:37:36<126:43:04,  5.06s/it]
 60%|█████▉    | 134907/225000 [270:37:43<143:32:35,  5.74s/it]
 60%|█████▉    | 134908/225000 [270:37:55<188:20:34,  7.53s/it]
 60%|█████▉    | 134909/225000 [270:37:58<155:44:56,  6.22s/it]
 60%|█████▉    | 134910/225000 [270:38:06<168:38:44,  6.74s/it]
 60%|█████▉    | 134911/225000 [270:38:10<146:53:02,  5.87s/it]
 60%|█████▉    | 134912/225000 [270:38:14<135:31:35,  5.42s/it]
 60%|█████▉    | 134913/225000 [270:38:22<150:42:50,  6.02s/it]
 60%|█████▉    | 134914/225000 [270:38:27<145:37:35,  5.82s/it]
 60%|█████▉    | 134915/225000 [270:38:31<133:39:07,  5.34s/it]
 60%|█████▉    | 134916/225000 [270:38:40<158:01:14,  6.31s/it]
 60%|█████▉    | 134917/225000 [270:38:46<157:42:42,  6.30s/it]
 60%|█████▉    | 134918/225000 [270:38:52<152:40:12,  6.10s/it]
 60%|█████▉    | 134919/225000 [270:38:55<132:57:05,  5.31s/it]
 60%|█████▉    | 134920/225000 [270:39:04<155:04:37,  6.20s/it]
 60%|█████▉    | 134921/225000 [270:39:07<133:24:47,  5.33s/it]
 60%|█████▉    | 134922/225000 [270:39:15<157:30:19,  6.29s/it]
 60%|█████▉    | 134923/225000 [270:39:19<137:00:16,  5.48s/it]
 60%|█████▉    | 134924/225000 [270:39:28<163:59:14,  6.55s/it]
 60%|█████▉    | 134925/225000 [270:39:31<139:18:57,  5.57s/it]
 60%|█████▉    | 134926/225000 [270:39:40<164:45:48,  6.59s/it]
 60%|█████▉    | 134927/225000 [270:39:44<140:15:17,  5.61s/it]
 60%|█████▉    | 134928/225000 [270:39:49<136:35:37,  5.46s/it]
 60%|█████▉    | 134929/225000 [270:39:54<136:36:30,  5.46s/it]
 60%|█████▉    | 134930/225000 [270:39:59<135:14:56,  5.41s/it]
 60%|█████▉    | 134931/225000 [270:40:05<133:30:34,  5.34s/it]
 60%|█████▉    | 134932/225000 [270:40:10<131:36:39,  5.26s/it]
 60%|█████▉    | 134933/225000 [270:40:15<132:18:37,  5.29s/it]
 60%|█████▉    | 134934/225000 [270:40:20<128:05:05,  5.12s/it]
 60%|█████▉    | 134935/225000 [270:40:25<127:23:57,  5.09s/it]
 60%|█████▉    | 134936/225000 [270:40:30<126:27:12,  5.05s/it]
 60%|█████▉    | 134937/225000 [270:40:35<129:50:34,  5.19s/it]
 60%|█████▉    | 134938/225000 [270:40:43<152:00:15,  6.08s/it]
 60%|█████▉    | 134939/225000 [270:40:56<198:17:12,  7.93s/it]
 60%|█████▉    | 134940/225000 [270:40:59<161:02:04,  6.44s/it]
 60%|█████▉    | 134941/225000 [270:41:05<160:09:34,  6.40s/it]
 60%|█████▉    | 134942/225000 [270:41:09<146:00:02,  5.84s/it]
 60%|█████▉    | 134943/225000 [270:41:14<136:52:44,  5.47s/it]
 60%|█████▉    | 134944/225000 [270:41:18<127:39:26,  5.10s/it]
 60%|█████▉    | 134945/225000 [270:41:23<122:42:30,  4.91s/it]
 60%|█████▉    | 134946/225000 [270:41:27<120:04:17,  4.80s/it]
 60%|█████▉    | 134947/225000 [270:41:38<160:32:23,  6.42s/it]
 60%|█████▉    | 134948/225000 [270:41:45<165:23:52,  6.61s/it]
 60%|█████▉    | 134949/225000 [270:41:49<151:46:26,  6.07s/it]
 60%|█████▉    | 134950/225000 [270:41:54<140:53:36,  5.63s/it]
                                                               
{'loss': '0.8027', 'grad_norm': '1.061', 'learning_rate': '1.872e-05', 'epoch': '0.5998', 'num_input_tokens_seen': 449392104, 'train_runtime': '9.745e+05', 'train_tokens_per_second': '461.1'}

 60%|█████▉    | 134950/225000 [270:41:54<140:53:36,  5.63s/it]
 60%|█████▉    | 134951/225000 [270:41:59<134:34:12,  5.38s/it]
 60%|█████▉    | 134952/225000 [270:42:03<129:07:02,  5.16s/it]
 60%|█████▉    | 134953/225000 [270:42:09<128:38:45,  5.14s/it]
 60%|█████▉    | 134954/225000 [270:42:14<130:11:40,  5.21s/it]
 60%|█████▉    | 134955/225000 [270:42:27<186:03:52,  7.44s/it]
 60%|█████▉    | 134956/225000 [270:42:33<178:00:02,  7.12s/it]
 60%|█████▉    | 134957/225000 [270:42:38<162:26:07,  6.49s/it]
 60%|█████▉    | 134958/225000 [270:42:44<158:16:19,  6.33s/it]
 60%|█████▉    | 134959/225000 [270:42:48<138:47:47,  5.55s/it]
 60%|█████▉    | 134960/225000 [270:42:52<126:37:52,  5.06s/it]
 60%|█████▉    | 134961/225000 [270:43:02<170:08:25,  6.80s/it]
 60%|█████▉    | 134962/225000 [270:43:06<145:41:17,  5.83s/it]
 60%|█████▉    | 134963/225000 [270:43:10<131:57:40,  5.28s/it]
 60%|█████▉    | 134964/225000 [270:43:14<123:11:42,  4.93s/it]
 60%|█████▉    | 134965/225000 [270:43:26<175:54:04,  7.03s/it]
 60%|█████▉    | 134966/225000 [270:43:31<157:21:19,  6.29s/it]
 60%|█████▉    | 134967/225000 [270:43:35<142:31:26,  5.70s/it]
 60%|█████▉    | 134968/225000 [270:43:39<134:13:43,  5.37s/it]
 60%|█████▉    | 134969/225000 [270:43:44<126:13:06,  5.05s/it]
 60%|█████▉    | 134970/225000 [270:43:48<121:20:28,  4.85s/it]
 60%|█████▉    | 134971/225000 [270:43:56<141:43:56,  5.67s/it]
 60%|█████▉    | 134972/225000 [270:44:08<189:09:26,  7.56s/it]
 60%|█████▉    | 134973/225000 [270:44:11<157:20:47,  6.29s/it]
 60%|█████▉    | 134974/225000 [270:44:15<139:15:51,  5.57s/it]
 60%|█████▉    | 134975/225000 [270:44:23<154:58:48,  6.20s/it]
 60%|█████▉    | 134976/225000 [270:44:27<142:50:02,  5.71s/it]
 60%|█████▉    | 134977/225000 [270:44:32<134:50:06,  5.39s/it]
 60%|█████▉    | 134978/225000 [270:44:36<128:00:13,  5.12s/it]
 60%|█████▉    | 134979/225000 [270:44:45<156:11:30,  6.25s/it]
 60%|█████▉    | 134980/225000 [270:44:54<177:13:27,  7.09s/it]
 60%|█████▉    | 134981/225000 [270:44:59<156:11:41,  6.25s/it]
 60%|█████▉    | 134982/225000 [270:45:04<153:33:13,  6.14s/it]
 60%|█████▉    | 134983/225000 [270:45:08<135:25:22,  5.42s/it]
 60%|█████▉    | 134984/225000 [270:45:18<169:16:28,  6.77s/it]
 60%|█████▉    | 134985/225000 [270:45:22<144:43:55,  5.79s/it]
 60%|█████▉    | 134986/225000 [270:45:29<159:01:45,  6.36s/it]
 60%|█████▉    | 134987/225000 [270:45:34<143:51:50,  5.75s/it]
 60%|█████▉    | 134988/225000 [270:45:38<132:43:31,  5.31s/it]
 60%|█████▉    | 134989/225000 [270:45:42<122:41:24,  4.91s/it]
 60%|█████▉    | 134990/225000 [270:45:46<118:39:54,  4.75s/it]
 60%|█████▉    | 134991/225000 [270:45:58<170:51:28,  6.83s/it]
 60%|█████▉    | 134992/225000 [270:46:03<158:14:42,  6.33s/it]
 60%|█████▉    | 134993/225000 [270:46:09<151:47:25,  6.07s/it]
 60%|█████▉    | 134994/225000 [270:46:13<139:24:02,  5.58s/it]
 60%|█████▉    | 134995/225000 [270:46:22<167:02:11,  6.68s/it]
 60%|█████▉    | 134996/225000 [270:46:26<148:39:16,  5.95s/it]
 60%|█████▉    | 134997/225000 [270:46:34<160:49:05,  6.43s/it]
 60%|█████▉    | 134998/225000 [270:46:37<137:50:29,  5.51s/it]
 60%|█████▉    | 134999/225000 [270:46:46<164:10:19,  6.57s/it]
 60%|██████    | 135000/225000 [270:46:50<143:35:25,  5.74s/it]
                                                               
{'loss': '0.7995', 'grad_norm': '1.155', 'learning_rate': '1.87e-05', 'epoch': '0.6', 'num_input_tokens_seen': 449521824, 'train_runtime': '9.748e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135000/225000 [270:46:50<143:35:25,  5.74s/it]
 60%|██████    | 135001/225000 [270:46:55<134:32:37,  5.38s/it]
 60%|██████    | 135002/225000 [270:47:01<143:19:54,  5.73s/it]
 60%|██████    | 135003/225000 [270:47:07<144:13:22,  5.77s/it]
 60%|██████    | 135004/225000 [270:47:12<133:42:04,  5.35s/it]
 60%|██████    | 135005/225000 [270:47:19<151:37:32,  6.07s/it]
 60%|██████    | 135006/225000 [270:47:23<132:32:10,  5.30s/it]
 60%|██████    | 135007/225000 [270:47:30<147:43:01,  5.91s/it]
 60%|██████    | 135008/225000 [270:47:37<154:10:31,  6.17s/it]
 60%|██████    | 135009/225000 [270:47:41<135:54:36,  5.44s/it]
 60%|██████    | 135010/225000 [270:47:45<126:08:52,  5.05s/it]
 60%|██████    | 135011/225000 [270:47:54<154:44:26,  6.19s/it]
 60%|██████    | 135012/225000 [270:47:58<139:14:57,  5.57s/it]
 60%|██████    | 135013/225000 [270:48:06<159:11:56,  6.37s/it]
 60%|██████    | 135014/225000 [270:48:09<137:21:45,  5.50s/it]
 60%|██████    | 135015/225000 [270:48:14<133:55:06,  5.36s/it]
 60%|██████    | 135016/225000 [270:48:19<131:08:48,  5.25s/it]
 60%|██████    | 135017/225000 [270:48:25<129:40:18,  5.19s/it]
 60%|██████    | 135018/225000 [270:48:30<131:10:28,  5.25s/it]
 60%|██████    | 135019/225000 [270:48:35<128:27:16,  5.14s/it]
 60%|██████    | 135020/225000 [270:48:41<136:13:57,  5.45s/it]
 60%|██████    | 135021/225000 [270:48:52<174:46:55,  6.99s/it]
 60%|██████    | 135022/225000 [270:48:56<158:01:39,  6.32s/it]
 60%|██████    | 135023/225000 [270:49:02<155:11:21,  6.21s/it]
 60%|██████    | 135024/225000 [270:49:06<136:59:14,  5.48s/it]
 60%|██████    | 135025/225000 [270:49:14<156:51:58,  6.28s/it]
 60%|██████    | 135026/225000 [270:49:23<177:14:42,  7.09s/it]
 60%|██████    | 135027/225000 [270:49:28<159:04:29,  6.36s/it]
 60%|██████    | 135028/225000 [270:49:33<148:27:41,  5.94s/it]
 60%|██████    | 135029/225000 [270:49:38<139:34:30,  5.58s/it]
 60%|██████    | 135030/225000 [270:49:43<135:00:27,  5.40s/it]
 60%|██████    | 135031/225000 [270:49:47<129:34:13,  5.18s/it]
 60%|██████    | 135032/225000 [270:49:52<128:32:48,  5.14s/it]
 60%|██████    | 135033/225000 [270:49:57<127:10:58,  5.09s/it]
 60%|██████    | 135034/225000 [270:50:06<154:50:03,  6.20s/it]
 60%|██████    | 135035/225000 [270:50:14<171:11:47,  6.85s/it]
 60%|██████    | 135036/225000 [270:50:18<149:39:39,  5.99s/it]
 60%|██████    | 135037/225000 [270:50:23<137:15:26,  5.49s/it]
 60%|██████    | 135038/225000 [270:50:33<173:18:10,  6.94s/it]
 60%|██████    | 135039/225000 [270:50:36<146:58:09,  5.88s/it]
 60%|██████    | 135040/225000 [270:50:43<154:34:01,  6.19s/it]
 60%|██████    | 135041/225000 [270:50:48<141:18:17,  5.65s/it]
 60%|██████    | 135042/225000 [270:50:52<134:27:34,  5.38s/it]
 60%|██████    | 135043/225000 [270:50:57<130:27:08,  5.22s/it]
 60%|██████    | 135044/225000 [270:51:03<130:32:16,  5.22s/it]
 60%|██████    | 135045/225000 [270:51:07<126:51:51,  5.08s/it]
 60%|██████    | 135046/225000 [270:51:12<124:42:21,  4.99s/it]
 60%|██████    | 135047/225000 [270:51:17<124:51:52,  5.00s/it]
 60%|██████    | 135048/225000 [270:51:22<121:31:12,  4.86s/it]
 60%|██████    | 135049/225000 [270:51:27<122:46:48,  4.91s/it]
 60%|██████    | 135050/225000 [270:51:39<180:35:48,  7.23s/it]
                                                               
{'loss': '0.7969', 'grad_norm': '1.15', 'learning_rate': '1.868e-05', 'epoch': '0.6002', 'num_input_tokens_seen': 449650928, 'train_runtime': '9.751e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135050/225000 [270:51:39<180:35:48,  7.23s/it]
 60%|██████    | 135051/225000 [270:51:44<160:58:14,  6.44s/it]
 60%|██████    | 135052/225000 [270:51:48<141:08:20,  5.65s/it]
 60%|██████    | 135053/225000 [270:51:52<132:00:09,  5.28s/it]
 60%|██████    | 135054/225000 [270:52:01<159:24:54,  6.38s/it]
 60%|██████    | 135055/225000 [270:52:10<181:02:58,  7.25s/it]
 60%|██████    | 135056/225000 [270:52:14<153:23:59,  6.14s/it]
 60%|██████    | 135057/225000 [270:52:22<167:11:15,  6.69s/it]
 60%|██████    | 135058/225000 [270:52:26<149:30:29,  5.98s/it]
 60%|██████    | 135059/225000 [270:52:31<143:02:05,  5.73s/it]
 60%|██████    | 135060/225000 [270:52:38<153:05:21,  6.13s/it]
 60%|██████    | 135061/225000 [270:52:42<137:16:32,  5.49s/it]
 60%|██████    | 135062/225000 [270:52:47<128:56:54,  5.16s/it]
 60%|██████    | 135063/225000 [270:52:52<126:06:42,  5.05s/it]
 60%|██████    | 135064/225000 [270:52:57<126:13:06,  5.05s/it]
 60%|██████    | 135065/225000 [270:53:08<173:15:39,  6.94s/it]
 60%|██████    | 135066/225000 [270:53:12<153:36:13,  6.15s/it]
 60%|██████    | 135067/225000 [270:53:17<142:34:01,  5.71s/it]
 60%|██████    | 135068/225000 [270:53:22<134:36:01,  5.39s/it]
 60%|██████    | 135069/225000 [270:53:26<130:18:58,  5.22s/it]
 60%|██████    | 135070/225000 [270:53:31<127:12:17,  5.09s/it]
 60%|██████    | 135071/225000 [270:53:36<124:36:57,  4.99s/it]
 60%|██████    | 135072/225000 [270:53:41<123:44:15,  4.95s/it]
 60%|██████    | 135073/225000 [270:53:46<122:03:49,  4.89s/it]
 60%|██████    | 135074/225000 [270:53:51<122:43:16,  4.91s/it]
 60%|██████    | 135075/225000 [270:53:57<135:15:19,  5.41s/it]
 60%|██████    | 135076/225000 [270:54:08<179:57:52,  7.20s/it]
 60%|██████    | 135077/225000 [270:54:13<159:49:05,  6.40s/it]
 60%|██████    | 135078/225000 [270:54:17<144:46:26,  5.80s/it]
 60%|██████    | 135079/225000 [270:54:26<168:32:58,  6.75s/it]
 60%|██████    | 135080/225000 [270:54:33<167:31:11,  6.71s/it]
 60%|██████    | 135081/225000 [270:54:37<147:06:13,  5.89s/it]
 60%|██████    | 135082/225000 [270:54:44<157:57:17,  6.32s/it]
 60%|██████    | 135083/225000 [270:54:48<136:35:52,  5.47s/it]
 60%|██████    | 135084/225000 [270:54:57<166:43:39,  6.68s/it]
 60%|██████    | 135085/225000 [270:55:01<142:49:46,  5.72s/it]
 60%|██████    | 135086/225000 [270:55:04<127:11:04,  5.09s/it]
 60%|██████    | 135087/225000 [270:55:08<117:42:24,  4.71s/it]
 60%|██████    | 135088/225000 [270:55:13<117:56:03,  4.72s/it]
 60%|██████    | 135089/225000 [270:55:24<161:45:36,  6.48s/it]
 60%|██████    | 135090/225000 [270:55:28<147:40:15,  5.91s/it]
 60%|██████    | 135091/225000 [270:55:33<141:53:06,  5.68s/it]
 60%|██████    | 135092/225000 [270:55:38<138:06:02,  5.53s/it]
 60%|██████    | 135093/225000 [270:55:46<151:23:31,  6.06s/it]
 60%|██████    | 135094/225000 [270:55:53<159:20:41,  6.38s/it]
 60%|██████    | 135095/225000 [270:56:00<164:04:59,  6.57s/it]
 60%|██████    | 135096/225000 [270:56:04<146:04:22,  5.85s/it]
 60%|██████    | 135097/225000 [270:56:08<128:52:09,  5.16s/it]
 60%|██████    | 135098/225000 [270:56:16<153:21:38,  6.14s/it]
 60%|██████    | 135099/225000 [270:56:20<137:33:22,  5.51s/it]
 60%|██████    | 135100/225000 [270:56:23<121:35:57,  4.87s/it]
                                                               
{'loss': '0.8014', 'grad_norm': '1.043', 'learning_rate': '1.866e-05', 'epoch': '0.6004', 'num_input_tokens_seen': 449780168, 'train_runtime': '9.754e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135100/225000 [270:56:23<121:35:57,  4.87s/it]
 60%|██████    | 135101/225000 [270:56:32<151:14:17,  6.06s/it]
 60%|██████    | 135102/225000 [270:56:38<145:32:27,  5.83s/it]
 60%|██████    | 135103/225000 [270:56:41<129:54:58,  5.20s/it]
 60%|██████    | 135104/225000 [270:56:46<124:43:23,  4.99s/it]
 60%|██████    | 135105/225000 [270:56:56<165:42:05,  6.64s/it]
 60%|██████    | 135106/225000 [270:57:00<144:12:58,  5.78s/it]
 60%|██████    | 135107/225000 [270:57:04<132:46:49,  5.32s/it]
 60%|██████    | 135108/225000 [270:57:08<124:25:15,  4.98s/it]
 60%|██████    | 135109/225000 [270:57:19<167:24:58,  6.70s/it]
 60%|██████    | 135110/225000 [270:57:24<151:40:53,  6.07s/it]
 60%|██████    | 135111/225000 [270:57:29<144:32:57,  5.79s/it]
 60%|██████    | 135112/225000 [270:57:34<139:14:34,  5.58s/it]
 60%|██████    | 135113/225000 [270:57:39<136:30:44,  5.47s/it]
 60%|██████    | 135114/225000 [270:57:44<132:27:11,  5.30s/it]
 60%|██████    | 135115/225000 [270:57:49<129:00:14,  5.17s/it]
 60%|██████    | 135116/225000 [270:57:54<129:35:39,  5.19s/it]
 60%|██████    | 135117/225000 [270:57:59<128:57:35,  5.17s/it]
 60%|██████    | 135118/225000 [270:58:04<126:07:50,  5.05s/it]
 60%|██████    | 135119/225000 [270:58:14<162:31:59,  6.51s/it]
 60%|██████    | 135120/225000 [270:58:21<165:16:39,  6.62s/it]
 60%|██████    | 135121/225000 [270:58:26<151:41:10,  6.08s/it]
 60%|██████    | 135122/225000 [270:58:31<142:59:05,  5.73s/it]
 60%|██████    | 135123/225000 [270:58:36<140:05:54,  5.61s/it]
 60%|██████    | 135124/225000 [270:58:41<135:46:36,  5.44s/it]
 60%|██████    | 135125/225000 [270:58:47<137:42:53,  5.52s/it]
 60%|██████    | 135126/225000 [270:58:52<132:53:42,  5.32s/it]
 60%|██████    | 135127/225000 [270:58:57<133:30:22,  5.35s/it]
 60%|██████    | 135128/225000 [270:59:08<179:05:28,  7.17s/it]
 60%|██████    | 135129/225000 [270:59:15<178:05:41,  7.13s/it]
 60%|██████    | 135130/225000 [270:59:19<150:14:21,  6.02s/it]
 60%|██████    | 135131/225000 [270:59:27<169:10:18,  6.78s/it]
 60%|██████    | 135132/225000 [270:59:31<146:24:16,  5.86s/it]
 60%|██████    | 135133/225000 [270:59:35<130:46:37,  5.24s/it]
 60%|██████    | 135134/225000 [270:59:46<170:45:55,  6.84s/it]
 60%|██████    | 135135/225000 [270:59:49<142:57:57,  5.73s/it]
 60%|██████    | 135136/225000 [270:59:57<163:50:56,  6.56s/it]
 60%|██████    | 135137/225000 [271:00:01<142:01:26,  5.69s/it]
 60%|██████    | 135138/225000 [271:00:07<148:23:30,  5.94s/it]
 60%|██████    | 135139/225000 [271:00:11<132:03:32,  5.29s/it]
 60%|██████    | 135140/225000 [271:00:15<118:39:45,  4.75s/it]
 60%|██████    | 135141/225000 [271:00:26<168:25:54,  6.75s/it]
 60%|██████    | 135142/225000 [271:00:31<151:42:59,  6.08s/it]
 60%|██████    | 135143/225000 [271:00:34<132:23:20,  5.30s/it]
 60%|██████    | 135144/225000 [271:00:42<152:02:03,  6.09s/it]
 60%|██████    | 135145/225000 [271:00:48<154:05:02,  6.17s/it]
 60%|██████    | 135146/225000 [271:00:52<138:53:45,  5.56s/it]
 60%|██████    | 135147/225000 [271:01:00<151:30:31,  6.07s/it]
 60%|██████    | 135148/225000 [271:01:04<135:42:01,  5.44s/it]
 60%|██████    | 135149/225000 [271:01:08<128:36:58,  5.15s/it]
 60%|██████    | 135150/225000 [271:01:12<121:19:39,  4.86s/it]
                                                               
{'loss': '0.7994', 'grad_norm': '1.113', 'learning_rate': '1.865e-05', 'epoch': '0.6007', 'num_input_tokens_seen': 449909560, 'train_runtime': '9.757e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135150/225000 [271:01:12<121:19:39,  4.86s/it]
 60%|██████    | 135151/225000 [271:01:17<119:07:27,  4.77s/it]
 60%|██████    | 135152/225000 [271:01:22<123:39:08,  4.95s/it]
 60%|██████    | 135153/225000 [271:01:32<160:20:42,  6.42s/it]
 60%|██████    | 135154/225000 [271:01:37<147:35:50,  5.91s/it]
 60%|██████    | 135155/225000 [271:01:42<140:39:10,  5.64s/it]
 60%|██████    | 135156/225000 [271:01:46<131:15:39,  5.26s/it]
 60%|██████    | 135157/225000 [271:01:51<127:43:52,  5.12s/it]
 60%|██████    | 135158/225000 [271:01:56<123:16:00,  4.94s/it]
 60%|██████    | 135159/225000 [271:02:03<140:43:54,  5.64s/it]
 60%|██████    | 135160/225000 [271:02:14<182:49:40,  7.33s/it]
 60%|██████    | 135161/225000 [271:02:17<152:40:55,  6.12s/it]
 60%|██████    | 135162/225000 [271:02:25<161:43:43,  6.48s/it]
 60%|██████    | 135163/225000 [271:02:30<151:46:50,  6.08s/it]
 60%|██████    | 135164/225000 [271:02:35<143:12:55,  5.74s/it]
 60%|██████    | 135165/225000 [271:02:41<148:22:51,  5.95s/it]
 60%|██████    | 135166/225000 [271:02:46<138:50:25,  5.56s/it]
 60%|██████    | 135167/225000 [271:02:51<135:58:53,  5.45s/it]
 60%|██████    | 135168/225000 [271:02:59<156:45:42,  6.28s/it]
 60%|██████    | 135169/225000 [271:03:05<150:40:25,  6.04s/it]
 60%|██████    | 135170/225000 [271:03:11<150:46:08,  6.04s/it]
 60%|██████    | 135171/225000 [271:03:18<158:31:13,  6.35s/it]
 60%|██████    | 135172/225000 [271:03:24<157:05:24,  6.30s/it]
 60%|██████    | 135173/225000 [271:03:28<137:06:47,  5.50s/it]
 60%|██████    | 135174/225000 [271:03:34<142:49:33,  5.72s/it]
 60%|██████    | 135175/225000 [271:03:43<166:25:52,  6.67s/it]
 60%|██████    | 135176/225000 [271:03:46<141:14:00,  5.66s/it]
 60%|██████    | 135177/225000 [271:03:51<131:52:26,  5.29s/it]
 60%|██████    | 135178/225000 [271:03:55<124:20:56,  4.98s/it]
 60%|██████    | 135179/225000 [271:03:59<121:50:27,  4.88s/it]
 60%|██████    | 135180/225000 [271:04:11<169:06:18,  6.78s/it]
 60%|██████    | 135181/225000 [271:04:14<142:59:49,  5.73s/it]
 60%|██████    | 135182/225000 [271:04:22<162:08:41,  6.50s/it]
 60%|██████    | 135183/225000 [271:04:26<140:51:35,  5.65s/it]
 60%|██████    | 135184/225000 [271:04:35<170:08:23,  6.82s/it]
 60%|██████    | 135185/225000 [271:04:40<153:31:46,  6.15s/it]
 60%|██████    | 135186/225000 [271:04:45<142:43:04,  5.72s/it]
 60%|██████    | 135187/225000 [271:04:50<136:43:19,  5.48s/it]
 60%|██████    | 135188/225000 [271:04:54<130:15:47,  5.22s/it]
 60%|██████    | 135189/225000 [271:04:59<126:24:30,  5.07s/it]
 60%|██████    | 135190/225000 [271:05:04<123:45:20,  4.96s/it]
 60%|██████    | 135191/225000 [271:05:08<121:27:42,  4.87s/it]
 60%|██████    | 135192/225000 [271:05:19<166:31:21,  6.68s/it]
 60%|██████    | 135193/225000 [271:05:26<166:09:42,  6.66s/it]
 60%|██████    | 135194/225000 [271:05:30<143:15:37,  5.74s/it]
 60%|██████    | 135195/225000 [271:05:34<134:03:55,  5.37s/it]
 60%|██████    | 135196/225000 [271:05:39<129:03:52,  5.17s/it]
 60%|██████    | 135197/225000 [271:05:44<127:01:45,  5.09s/it]
 60%|██████    | 135198/225000 [271:05:48<123:23:06,  4.95s/it]
 60%|██████    | 135199/225000 [271:05:56<143:04:33,  5.74s/it]
 60%|██████    | 135200/225000 [271:06:05<165:06:01,  6.62s/it]
                                                               
{'loss': '0.7967', 'grad_norm': '1.103', 'learning_rate': '1.863e-05', 'epoch': '0.6009', 'num_input_tokens_seen': 450040032, 'train_runtime': '9.76e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135200/225000 [271:06:05<165:06:01,  6.62s/it]
 60%|██████    | 135201/225000 [271:06:08<142:30:56,  5.71s/it]
 60%|██████    | 135202/225000 [271:06:12<131:36:50,  5.28s/it]
 60%|██████    | 135203/225000 [271:06:24<176:48:18,  7.09s/it]
 60%|██████    | 135204/225000 [271:06:27<150:16:32,  6.02s/it]
 60%|██████    | 135205/225000 [271:06:35<162:49:46,  6.53s/it]
 60%|██████    | 135206/225000 [271:06:38<140:19:50,  5.63s/it]
 60%|██████    | 135207/225000 [271:06:48<170:20:19,  6.83s/it]
 60%|██████    | 135208/225000 [271:06:51<142:16:12,  5.70s/it]
 60%|██████    | 135209/225000 [271:06:57<143:47:28,  5.77s/it]
 60%|██████    | 135210/225000 [271:07:04<150:32:08,  6.04s/it]
 60%|██████    | 135211/225000 [271:07:09<143:39:12,  5.76s/it]
 60%|██████    | 135212/225000 [271:07:14<135:37:49,  5.44s/it]
 60%|██████    | 135213/225000 [271:07:19<132:37:39,  5.32s/it]
 60%|██████    | 135214/225000 [271:07:23<127:25:52,  5.11s/it]
 60%|██████    | 135215/225000 [271:07:28<125:04:53,  5.02s/it]
 60%|██████    | 135216/225000 [271:07:33<124:42:53,  5.00s/it]
 60%|██████    | 135217/225000 [271:07:42<155:03:44,  6.22s/it]
 60%|██████    | 135218/225000 [271:07:54<200:16:01,  8.03s/it]
 60%|██████    | 135219/225000 [271:07:58<165:40:23,  6.64s/it]
 60%|██████    | 135220/225000 [271:08:04<162:10:09,  6.50s/it]
 60%|██████    | 135221/225000 [271:08:10<161:48:33,  6.49s/it]
 60%|██████    | 135222/225000 [271:08:18<168:45:34,  6.77s/it]
 60%|██████    | 135223/225000 [271:08:21<145:22:22,  5.83s/it]
 60%|██████    | 135224/225000 [271:08:27<142:15:18,  5.70s/it]
 60%|██████    | 135225/225000 [271:08:34<153:21:37,  6.15s/it]
 60%|██████    | 135226/225000 [271:08:37<131:37:36,  5.28s/it]
 60%|██████    | 135227/225000 [271:08:47<162:53:10,  6.53s/it]
 60%|██████    | 135228/225000 [271:08:50<139:04:47,  5.58s/it]
 60%|██████    | 135229/225000 [271:08:54<126:12:56,  5.06s/it]
 60%|██████    | 135230/225000 [271:09:03<160:08:46,  6.42s/it]
 60%|██████    | 135231/225000 [271:09:08<146:45:50,  5.89s/it]
 60%|██████    | 135232/225000 [271:09:14<145:24:11,  5.83s/it]
 60%|██████    | 135233/225000 [271:09:21<154:33:58,  6.20s/it]
 60%|██████    | 135234/225000 [271:09:26<145:05:35,  5.82s/it]
 60%|██████    | 135235/225000 [271:09:31<139:00:38,  5.57s/it]
 60%|██████    | 135236/225000 [271:09:38<149:51:26,  6.01s/it]
 60%|██████    | 135237/225000 [271:09:43<143:17:49,  5.75s/it]
 60%|██████    | 135238/225000 [271:09:48<140:49:02,  5.65s/it]
 60%|██████    | 135239/225000 [271:09:53<136:38:18,  5.48s/it]
 60%|██████    | 135240/225000 [271:09:59<135:48:27,  5.45s/it]
 60%|██████    | 135241/225000 [271:10:04<133:28:44,  5.35s/it]
 60%|██████    | 135242/225000 [271:10:09<130:42:58,  5.24s/it]
 60%|██████    | 135243/225000 [271:10:14<131:27:43,  5.27s/it]
 60%|██████    | 135244/225000 [271:10:19<129:36:44,  5.20s/it]
 60%|██████    | 135245/225000 [271:10:24<128:41:14,  5.16s/it]
 60%|██████    | 135246/225000 [271:10:30<129:30:36,  5.19s/it]
 60%|██████    | 135247/225000 [271:10:35<127:07:07,  5.10s/it]
 60%|██████    | 135248/225000 [271:10:39<124:58:48,  5.01s/it]
 60%|██████    | 135249/225000 [271:10:45<128:46:38,  5.17s/it]
 60%|██████    | 135250/225000 [271:10:57<183:03:54,  7.34s/it]
                                                               
{'loss': '0.8002', 'grad_norm': '0.9476', 'learning_rate': '1.861e-05', 'epoch': '0.6011', 'num_input_tokens_seen': 450169992, 'train_runtime': '9.763e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135250/225000 [271:10:57<183:03:54,  7.34s/it]
 60%|██████    | 135251/225000 [271:11:01<158:53:56,  6.37s/it]
 60%|██████    | 135252/225000 [271:11:11<182:01:37,  7.30s/it]
 60%|██████    | 135253/225000 [271:11:15<159:39:27,  6.40s/it]
 60%|██████    | 135254/225000 [271:11:19<140:49:09,  5.65s/it]
 60%|██████    | 135255/225000 [271:11:24<133:17:32,  5.35s/it]
 60%|██████    | 135256/225000 [271:11:28<121:30:36,  4.87s/it]
 60%|██████    | 135257/225000 [271:11:40<175:02:07,  7.02s/it]
 60%|██████    | 135258/225000 [271:11:44<156:48:08,  6.29s/it]
 60%|██████    | 135259/225000 [271:11:48<135:19:57,  5.43s/it]
 60%|██████    | 135260/225000 [271:11:56<156:08:56,  6.26s/it]
 60%|██████    | 135261/225000 [271:12:00<144:22:22,  5.79s/it]
 60%|██████    | 135262/225000 [271:12:04<131:05:12,  5.26s/it]
 60%|██████    | 135263/225000 [271:12:09<123:44:38,  4.96s/it]
 60%|██████    | 135264/225000 [271:12:14<122:33:31,  4.92s/it]
 60%|██████    | 135265/225000 [271:12:19<124:32:19,  5.00s/it]
 60%|██████    | 135266/225000 [271:12:29<166:27:10,  6.68s/it]
 60%|██████    | 135267/225000 [271:12:33<142:36:15,  5.72s/it]
 60%|██████    | 135268/225000 [271:12:36<126:32:59,  5.08s/it]
 60%|██████    | 135269/225000 [271:12:45<155:08:12,  6.22s/it]
 60%|██████    | 135270/225000 [271:12:50<140:24:58,  5.63s/it]
 60%|██████    | 135271/225000 [271:12:58<159:33:02,  6.40s/it]
 60%|██████    | 135272/225000 [271:13:05<162:57:55,  6.54s/it]
 60%|██████    | 135273/225000 [271:13:08<142:02:32,  5.70s/it]
 60%|██████    | 135274/225000 [271:13:14<143:17:32,  5.75s/it]
 60%|██████    | 135275/225000 [271:13:23<167:05:01,  6.70s/it]
 60%|██████    | 135276/225000 [271:13:27<147:40:14,  5.93s/it]
 60%|██████    | 135277/225000 [271:13:31<133:15:33,  5.35s/it]
 60%|██████    | 135278/225000 [271:13:42<175:26:26,  7.04s/it]
 60%|██████    | 135279/225000 [271:13:46<152:14:57,  6.11s/it]
 60%|██████    | 135280/225000 [271:13:54<162:00:55,  6.50s/it]
 60%|██████    | 135281/225000 [271:13:57<140:37:22,  5.64s/it]
 60%|██████    | 135282/225000 [271:14:07<168:09:13,  6.75s/it]
 60%|██████    | 135283/225000 [271:14:10<144:27:12,  5.80s/it]
 60%|██████    | 135284/225000 [271:14:14<128:23:49,  5.15s/it]
 60%|██████    | 135285/225000 [271:14:24<163:47:32,  6.57s/it]
 60%|██████    | 135286/225000 [271:14:27<137:46:56,  5.53s/it]
 60%|██████    | 135287/225000 [271:14:34<153:54:43,  6.18s/it]
 60%|██████    | 135288/225000 [271:14:39<143:19:10,  5.75s/it]
 60%|██████    | 135289/225000 [271:14:44<135:09:35,  5.42s/it]
 60%|██████    | 135290/225000 [271:14:49<131:01:08,  5.26s/it]
 60%|██████    | 135291/225000 [271:14:54<129:29:33,  5.20s/it]
 60%|██████    | 135292/225000 [271:14:59<127:48:24,  5.13s/it]
 60%|██████    | 135293/225000 [271:15:03<124:30:14,  5.00s/it]
 60%|██████    | 135294/225000 [271:15:16<179:49:54,  7.22s/it]
 60%|██████    | 135295/225000 [271:15:20<158:38:03,  6.37s/it]
 60%|██████    | 135296/225000 [271:15:24<142:35:36,  5.72s/it]
 60%|██████    | 135297/225000 [271:15:29<131:25:49,  5.27s/it]
 60%|██████    | 135298/225000 [271:15:33<125:45:37,  5.05s/it]
 60%|██████    | 135299/225000 [271:15:43<159:34:09,  6.40s/it]
 60%|██████    | 135300/225000 [271:15:51<170:48:39,  6.86s/it]
                                                               
{'loss': '0.8142', 'grad_norm': '1.078', 'learning_rate': '1.859e-05', 'epoch': '0.6013', 'num_input_tokens_seen': 450300352, 'train_runtime': '9.766e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135300/225000 [271:15:51<170:48:39,  6.86s/it]
 60%|██████    | 135301/225000 [271:15:54<146:25:54,  5.88s/it]
 60%|██████    | 135302/225000 [271:15:58<131:43:11,  5.29s/it]
 60%|██████    | 135303/225000 [271:16:07<156:07:33,  6.27s/it]
 60%|██████    | 135304/225000 [271:16:14<160:06:14,  6.43s/it]
 60%|██████    | 135305/225000 [271:16:17<139:24:00,  5.59s/it]
 60%|██████    | 135306/225000 [271:16:21<125:48:10,  5.05s/it]
 60%|██████    | 135307/225000 [271:16:32<167:56:12,  6.74s/it]
 60%|██████    | 135308/225000 [271:16:35<143:58:38,  5.78s/it]
 60%|██████    | 135309/225000 [271:16:43<158:09:33,  6.35s/it]
 60%|██████    | 135310/225000 [271:16:46<135:30:13,  5.44s/it]
 60%|██████    | 135311/225000 [271:16:50<121:33:21,  4.88s/it]
 60%|██████    | 135312/225000 [271:16:54<119:52:21,  4.81s/it]
 60%|██████    | 135313/225000 [271:16:59<121:47:15,  4.89s/it]
 60%|██████    | 135314/225000 [271:17:05<123:26:27,  4.95s/it]
 60%|██████    | 135315/225000 [271:17:15<166:42:53,  6.69s/it]
 60%|██████    | 135316/225000 [271:17:22<167:04:34,  6.71s/it]
 60%|██████    | 135317/225000 [271:17:25<141:35:53,  5.68s/it]
 60%|██████    | 135318/225000 [271:17:34<162:25:51,  6.52s/it]
 60%|██████    | 135319/225000 [271:17:38<145:31:31,  5.84s/it]
 60%|██████    | 135320/225000 [271:17:42<133:08:06,  5.34s/it]
 60%|██████    | 135321/225000 [271:17:47<124:52:01,  5.01s/it]
 60%|██████    | 135322/225000 [271:17:52<126:24:51,  5.07s/it]
 60%|██████    | 135323/225000 [271:18:03<172:25:22,  6.92s/it]
 60%|██████    | 135324/225000 [271:18:06<145:45:15,  5.85s/it]
 60%|██████    | 135325/225000 [271:18:13<152:29:26,  6.12s/it]
 60%|██████    | 135326/225000 [271:18:17<138:24:08,  5.56s/it]
 60%|██████    | 135327/225000 [271:18:26<162:00:14,  6.50s/it]
 60%|██████    | 135328/225000 [271:18:30<139:36:10,  5.60s/it]
 60%|██████    | 135329/225000 [271:18:34<131:39:48,  5.29s/it]
 60%|██████    | 135330/225000 [271:18:42<152:29:58,  6.12s/it]
 60%|██████    | 135331/225000 [271:18:46<137:34:20,  5.52s/it]
 60%|██████    | 135332/225000 [271:18:54<150:56:39,  6.06s/it]
 60%|██████    | 135333/225000 [271:18:57<132:46:08,  5.33s/it]
 60%|██████    | 135334/225000 [271:19:07<165:12:50,  6.63s/it]
 60%|██████    | 135335/225000 [271:19:11<149:36:21,  6.01s/it]
 60%|██████    | 135336/225000 [271:19:17<144:15:01,  5.79s/it]
 60%|██████    | 135337/225000 [271:19:25<159:22:00,  6.40s/it]
 60%|██████    | 135338/225000 [271:19:29<147:05:15,  5.91s/it]
 60%|██████    | 135339/225000 [271:19:34<136:48:17,  5.49s/it]
 60%|██████    | 135340/225000 [271:19:38<130:25:53,  5.24s/it]
 60%|██████    | 135341/225000 [271:19:43<123:19:27,  4.95s/it]
 60%|██████    | 135342/225000 [271:19:49<134:02:11,  5.38s/it]
 60%|██████    | 135343/225000 [271:20:00<176:17:19,  7.08s/it]
 60%|██████    | 135344/225000 [271:20:05<155:55:50,  6.26s/it]
 60%|██████    | 135345/225000 [271:20:09<139:28:31,  5.60s/it]
 60%|██████    | 135346/225000 [271:20:13<130:07:20,  5.22s/it]
 60%|██████    | 135347/225000 [271:20:17<123:59:20,  4.98s/it]
 60%|██████    | 135348/225000 [271:20:23<125:53:28,  5.06s/it]
 60%|██████    | 135349/225000 [271:20:34<176:02:35,  7.07s/it]
 60%|██████    | 135350/225000 [271:20:38<149:24:47,  6.00s/it]
                                                               
{'loss': '0.8168', 'grad_norm': '1.022', 'learning_rate': '1.857e-05', 'epoch': '0.6016', 'num_input_tokens_seen': 450429544, 'train_runtime': '9.768e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135350/225000 [271:20:38<149:24:47,  6.00s/it]
 60%|██████    | 135351/225000 [271:20:44<150:57:27,  6.06s/it]
 60%|██████    | 135352/225000 [271:20:47<128:44:55,  5.17s/it]
 60%|██████    | 135353/225000 [271:20:54<141:34:45,  5.69s/it]
 60%|██████    | 135354/225000 [271:21:02<155:05:34,  6.23s/it]
 60%|██████    | 135355/225000 [271:21:06<145:39:45,  5.85s/it]
 60%|██████    | 135356/225000 [271:21:14<157:32:26,  6.33s/it]
 60%|██████    | 135357/225000 [271:21:21<163:51:00,  6.58s/it]
 60%|██████    | 135358/225000 [271:21:24<138:49:36,  5.58s/it]
 60%|██████    | 135359/225000 [271:21:32<156:13:15,  6.27s/it]
 60%|██████    | 135360/225000 [271:21:36<133:53:29,  5.38s/it]
 60%|██████    | 135361/225000 [271:21:43<151:11:20,  6.07s/it]
 60%|██████    | 135362/225000 [271:21:49<147:11:45,  5.91s/it]
 60%|██████    | 135363/225000 [271:21:53<135:42:24,  5.45s/it]
 60%|██████    | 135364/225000 [271:21:58<130:53:20,  5.26s/it]
 60%|██████    | 135365/225000 [271:22:03<126:25:31,  5.08s/it]
 60%|██████    | 135366/225000 [271:22:07<123:59:35,  4.98s/it]
 60%|██████    | 135367/225000 [271:22:15<140:29:35,  5.64s/it]
 60%|██████    | 135368/225000 [271:22:26<184:57:01,  7.43s/it]
 60%|██████    | 135369/225000 [271:22:30<155:01:14,  6.23s/it]
 60%|██████    | 135370/225000 [271:22:36<159:13:07,  6.40s/it]
 60%|██████    | 135371/225000 [271:22:41<143:42:10,  5.77s/it]
 60%|██████    | 135372/225000 [271:22:45<130:13:59,  5.23s/it]
 60%|██████    | 135373/225000 [271:22:55<167:56:59,  6.75s/it]
 60%|██████    | 135374/225000 [271:22:58<140:46:39,  5.65s/it]
 60%|██████    | 135375/225000 [271:23:05<153:31:17,  6.17s/it]
 60%|██████    | 135376/225000 [271:23:09<137:25:44,  5.52s/it]
 60%|██████    | 135377/225000 [271:23:17<152:40:51,  6.13s/it]
 60%|██████    | 135378/225000 [271:23:21<137:17:29,  5.51s/it]
 60%|██████    | 135379/225000 [271:23:26<134:05:21,  5.39s/it]
 60%|██████    | 135380/225000 [271:23:31<131:10:23,  5.27s/it]
 60%|██████    | 135381/225000 [271:23:36<129:16:18,  5.19s/it]
 60%|██████    | 135382/225000 [271:23:41<128:03:27,  5.14s/it]
 60%|██████    | 135383/225000 [271:23:46<123:51:00,  4.98s/it]
 60%|██████    | 135384/225000 [271:23:51<122:58:58,  4.94s/it]
 60%|██████    | 135385/225000 [271:23:55<121:10:25,  4.87s/it]
 60%|██████    | 135386/225000 [271:24:07<171:25:43,  6.89s/it]
 60%|██████    | 135387/225000 [271:24:12<159:03:12,  6.39s/it]
 60%|██████    | 135388/225000 [271:24:19<163:57:39,  6.59s/it]
 60%|██████    | 135389/225000 [271:24:26<163:28:28,  6.57s/it]
 60%|██████    | 135390/225000 [271:24:30<148:43:25,  5.97s/it]
 60%|██████    | 135391/225000 [271:24:34<130:10:39,  5.23s/it]
 60%|██████    | 135392/225000 [271:24:42<154:16:00,  6.20s/it]
 60%|██████    | 135393/225000 [271:24:47<141:53:09,  5.70s/it]
 60%|██████    | 135394/225000 [271:24:51<134:42:27,  5.41s/it]
 60%|██████    | 135395/225000 [271:24:56<125:40:45,  5.05s/it]
 60%|██████    | 135396/225000 [271:25:00<123:10:20,  4.95s/it]
 60%|██████    | 135397/225000 [271:25:12<176:00:17,  7.07s/it]
 60%|██████    | 135398/225000 [271:25:18<165:58:17,  6.67s/it]
 60%|██████    | 135399/225000 [271:25:23<150:18:56,  6.04s/it]
 60%|██████    | 135400/225000 [271:25:27<136:23:07,  5.48s/it]
                                                               
{'loss': '0.7951', 'grad_norm': '1.218', 'learning_rate': '1.856e-05', 'epoch': '0.6018', 'num_input_tokens_seen': 450558240, 'train_runtime': '9.771e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135400/225000 [271:25:27<136:23:07,  5.48s/it]
 60%|██████    | 135401/225000 [271:25:31<128:56:59,  5.18s/it]
 60%|██████    | 135402/225000 [271:25:36<125:01:37,  5.02s/it]
 60%|██████    | 135403/225000 [271:25:41<121:59:04,  4.90s/it]
 60%|██████    | 135404/225000 [271:25:45<117:39:46,  4.73s/it]
 60%|██████    | 135405/225000 [271:25:58<178:45:45,  7.18s/it]
 60%|██████    | 135406/225000 [271:26:02<158:53:43,  6.38s/it]
 60%|██████    | 135407/225000 [271:26:11<171:42:50,  6.90s/it]
 60%|██████    | 135408/225000 [271:26:14<143:59:07,  5.79s/it]
 60%|██████    | 135409/225000 [271:26:19<138:36:13,  5.57s/it]
 60%|██████    | 135410/225000 [271:26:24<136:12:08,  5.47s/it]
 60%|██████    | 135411/225000 [271:26:29<130:58:57,  5.26s/it]
 60%|██████    | 135412/225000 [271:26:34<128:37:54,  5.17s/it]
 60%|██████    | 135413/225000 [271:26:39<129:15:19,  5.19s/it]
 60%|██████    | 135414/225000 [271:26:44<125:19:37,  5.04s/it]
 60%|██████    | 135415/225000 [271:26:49<124:38:50,  5.01s/it]
 60%|██████    | 135416/225000 [271:26:54<125:50:06,  5.06s/it]
 60%|██████    | 135417/225000 [271:26:59<125:26:42,  5.04s/it]
 60%|██████    | 135418/225000 [271:27:10<172:52:04,  6.95s/it]
 60%|██████    | 135419/225000 [271:27:19<183:28:39,  7.37s/it]
 60%|██████    | 135420/225000 [271:27:22<155:19:40,  6.24s/it]
 60%|██████    | 135421/225000 [271:27:30<167:52:31,  6.75s/it]
 60%|██████    | 135422/225000 [271:27:37<168:04:02,  6.75s/it]
 60%|██████    | 135423/225000 [271:27:41<151:45:24,  6.10s/it]
 60%|██████    | 135424/225000 [271:27:46<140:40:09,  5.65s/it]
 60%|██████    | 135425/225000 [271:27:51<133:42:09,  5.37s/it]
 60%|██████    | 135426/225000 [271:27:55<126:30:06,  5.08s/it]
 60%|██████    | 135427/225000 [271:28:00<122:13:03,  4.91s/it]
 60%|██████    | 135428/225000 [271:28:05<125:29:44,  5.04s/it]
 60%|██████    | 135429/225000 [271:28:18<183:55:35,  7.39s/it]
 60%|██████    | 135430/225000 [271:28:22<162:01:52,  6.51s/it]
 60%|██████    | 135431/225000 [271:28:26<142:34:34,  5.73s/it]
 60%|██████    | 135432/225000 [271:28:31<131:30:58,  5.29s/it]
 60%|██████    | 135433/225000 [271:28:35<127:05:33,  5.11s/it]
 60%|██████    | 135434/225000 [271:28:40<124:44:05,  5.01s/it]
 60%|██████    | 135435/225000 [271:28:45<121:13:37,  4.87s/it]
 60%|██████    | 135436/225000 [271:28:56<168:22:30,  6.77s/it]
 60%|██████    | 135437/225000 [271:29:03<170:48:25,  6.87s/it]
 60%|██████    | 135438/225000 [271:29:06<146:53:34,  5.90s/it]
 60%|██████    | 135439/225000 [271:29:15<165:07:41,  6.64s/it]
 60%|██████    | 135440/225000 [271:29:19<143:58:46,  5.79s/it]
 60%|██████    | 135441/225000 [271:29:23<132:52:37,  5.34s/it]
 60%|██████    | 135442/225000 [271:29:27<123:42:32,  4.97s/it]
 60%|██████    | 135443/225000 [271:29:38<170:57:34,  6.87s/it]
 60%|██████    | 135444/225000 [271:29:44<158:14:47,  6.36s/it]
 60%|██████    | 135445/225000 [271:29:47<138:01:54,  5.55s/it]
 60%|██████    | 135446/225000 [271:29:51<125:14:51,  5.03s/it]
 60%|██████    | 135447/225000 [271:30:00<156:21:40,  6.29s/it]
 60%|██████    | 135448/225000 [271:30:06<151:30:28,  6.09s/it]
 60%|██████    | 135449/225000 [271:30:14<165:13:50,  6.64s/it]
 60%|██████    | 135450/225000 [271:30:18<148:36:34,  5.97s/it]
                                                               
{'loss': '0.801', 'grad_norm': '1.008', 'learning_rate': '1.854e-05', 'epoch': '0.602', 'num_input_tokens_seen': 450688064, 'train_runtime': '9.774e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135450/225000 [271:30:18<148:36:34,  5.97s/it]
 60%|██████    | 135451/225000 [271:30:23<137:22:02,  5.52s/it]
 60%|██████    | 135452/225000 [271:30:27<126:04:26,  5.07s/it]
 60%|██████    | 135453/225000 [271:30:31<120:59:35,  4.86s/it]
 60%|██████    | 135454/225000 [271:30:41<160:45:51,  6.46s/it]
 60%|██████    | 135455/225000 [271:30:48<161:56:27,  6.51s/it]
 60%|██████    | 135456/225000 [271:30:52<147:05:23,  5.91s/it]
 60%|██████    | 135457/225000 [271:31:00<161:20:53,  6.49s/it]
 60%|██████    | 135458/225000 [271:31:04<139:49:49,  5.62s/it]
 60%|██████    | 135459/225000 [271:31:12<160:19:55,  6.45s/it]
 60%|██████    | 135460/225000 [271:31:16<139:20:42,  5.60s/it]
 60%|██████    | 135461/225000 [271:31:20<125:10:55,  5.03s/it]
 60%|██████    | 135462/225000 [271:31:26<138:09:38,  5.55s/it]
 60%|██████    | 135463/225000 [271:31:35<160:14:55,  6.44s/it]
 60%|██████    | 135464/225000 [271:31:39<142:09:53,  5.72s/it]
 60%|██████    | 135465/225000 [271:31:43<133:28:45,  5.37s/it]
 60%|██████    | 135466/225000 [271:31:48<128:26:32,  5.16s/it]
 60%|██████    | 135467/225000 [271:31:58<164:14:14,  6.60s/it]
 60%|██████    | 135468/225000 [271:32:04<163:05:36,  6.56s/it]
 60%|██████    | 135469/225000 [271:32:09<144:18:43,  5.80s/it]
 60%|██████    | 135470/225000 [271:32:13<131:21:12,  5.28s/it]
 60%|██████    | 135471/225000 [271:32:20<144:52:56,  5.83s/it]
 60%|██████    | 135472/225000 [271:32:24<130:38:38,  5.25s/it]
 60%|██████    | 135473/225000 [271:32:28<123:10:22,  4.95s/it]
 60%|██████    | 135474/225000 [271:32:32<117:49:19,  4.74s/it]
 60%|██████    | 135475/225000 [271:32:43<163:48:17,  6.59s/it]
 60%|██████    | 135476/225000 [271:32:49<159:05:19,  6.40s/it]
 60%|██████    | 135477/225000 [271:32:54<147:29:37,  5.93s/it]
 60%|██████    | 135478/225000 [271:33:01<153:21:33,  6.17s/it]
 60%|██████    | 135479/225000 [271:33:07<155:56:34,  6.27s/it]
 60%|██████    | 135480/225000 [271:33:13<152:01:18,  6.11s/it]
 60%|██████    | 135481/225000 [271:33:18<146:53:48,  5.91s/it]
 60%|██████    | 135482/225000 [271:33:25<154:33:16,  6.22s/it]
 60%|██████    | 135483/225000 [271:33:31<153:47:38,  6.18s/it]
 60%|██████    | 135484/225000 [271:33:39<168:59:44,  6.80s/it]
 60%|██████    | 135485/225000 [271:33:44<148:44:05,  5.98s/it]
 60%|██████    | 135486/225000 [271:33:47<132:28:05,  5.33s/it]
 60%|██████    | 135487/225000 [271:33:51<121:07:30,  4.87s/it]
 60%|██████    | 135488/225000 [271:34:00<153:55:05,  6.19s/it]
 60%|██████    | 135489/225000 [271:34:06<150:08:06,  6.04s/it]
 60%|██████    | 135490/225000 [271:34:14<162:34:02,  6.54s/it]
 60%|██████    | 135491/225000 [271:34:18<141:58:48,  5.71s/it]
 60%|██████    | 135492/225000 [271:34:24<146:33:02,  5.89s/it]
 60%|██████    | 135493/225000 [271:34:31<153:10:29,  6.16s/it]
 60%|██████    | 135494/225000 [271:34:34<135:24:00,  5.45s/it]
 60%|██████    | 135495/225000 [271:34:44<163:24:25,  6.57s/it]
 60%|██████    | 135496/225000 [271:34:47<139:33:29,  5.61s/it]
 60%|██████    | 135497/225000 [271:34:55<158:23:22,  6.37s/it]
 60%|██████    | 135498/225000 [271:34:59<137:18:30,  5.52s/it]
 60%|██████    | 135499/225000 [271:35:06<152:29:18,  6.13s/it]
 60%|██████    | 135500/225000 [271:35:12<147:05:19,  5.92s/it]
                                                               
{'loss': '0.7953', 'grad_norm': '1.058', 'learning_rate': '1.852e-05', 'epoch': '0.6022', 'num_input_tokens_seen': 450818120, 'train_runtime': '9.777e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135500/225000 [271:35:12<147:05:19,  5.92s/it]
 60%|██████    | 135501/225000 [271:35:17<140:52:14,  5.67s/it]
 60%|██████    | 135502/225000 [271:35:22<137:35:40,  5.53s/it]
 60%|██████    | 135503/225000 [271:35:28<137:31:08,  5.53s/it]
 60%|██████    | 135504/225000 [271:35:33<134:15:24,  5.40s/it]
 60%|██████    | 135505/225000 [271:35:39<141:28:34,  5.69s/it]
 60%|██████    | 135506/225000 [271:35:49<170:36:04,  6.86s/it]
 60%|██████    | 135507/225000 [271:35:52<148:13:50,  5.96s/it]
 60%|██████    | 135508/225000 [271:35:57<134:42:26,  5.42s/it]
 60%|██████    | 135509/225000 [271:36:01<125:36:58,  5.05s/it]
 60%|██████    | 135510/225000 [271:36:05<120:53:08,  4.86s/it]
 60%|██████    | 135511/225000 [271:36:10<121:57:10,  4.91s/it]
 60%|██████    | 135512/225000 [271:36:15<122:04:19,  4.91s/it]
 60%|██████    | 135513/225000 [271:36:28<178:24:33,  7.18s/it]
 60%|██████    | 135514/225000 [271:36:33<164:16:01,  6.61s/it]
 60%|██████    | 135515/225000 [271:36:40<170:56:23,  6.88s/it]
 60%|██████    | 135516/225000 [271:36:45<150:32:37,  6.06s/it]
 60%|██████    | 135517/225000 [271:36:49<135:59:09,  5.47s/it]
 60%|██████    | 135518/225000 [271:36:53<124:14:16,  5.00s/it]
 60%|██████    | 135519/225000 [271:36:57<118:13:20,  4.76s/it]
 60%|██████    | 135520/225000 [271:37:05<145:43:02,  5.86s/it]
 60%|██████    | 135521/225000 [271:37:14<167:56:02,  6.76s/it]
 60%|██████    | 135522/225000 [271:37:20<165:28:22,  6.66s/it]
 60%|██████    | 135523/225000 [271:37:24<141:25:14,  5.69s/it]
 60%|██████    | 135524/225000 [271:37:29<135:23:45,  5.45s/it]
 60%|██████    | 135525/225000 [271:37:34<134:53:23,  5.43s/it]
 60%|██████    | 135526/225000 [271:37:39<133:16:04,  5.36s/it]
 60%|██████    | 135527/225000 [271:37:45<132:15:17,  5.32s/it]
 60%|██████    | 135528/225000 [271:37:49<128:09:43,  5.16s/it]
 60%|██████    | 135529/225000 [271:37:54<124:37:05,  5.01s/it]
 60%|██████    | 135530/225000 [271:37:59<127:40:20,  5.14s/it]
 60%|██████    | 135531/225000 [271:38:07<143:08:47,  5.76s/it]
 60%|██████    | 135532/225000 [271:38:18<180:54:37,  7.28s/it]
 60%|██████    | 135533/225000 [271:38:21<154:19:23,  6.21s/it]
 60%|██████    | 135534/225000 [271:38:32<187:12:56,  7.53s/it]
 60%|██████    | 135535/225000 [271:38:35<155:36:39,  6.26s/it]
 60%|██████    | 135536/225000 [271:38:39<136:47:37,  5.50s/it]
 60%|██████    | 135537/225000 [271:38:43<127:34:38,  5.13s/it]
 60%|██████    | 135538/225000 [271:38:53<164:21:54,  6.61s/it]
 60%|██████    | 135539/225000 [271:38:57<143:22:04,  5.77s/it]
 60%|██████    | 135540/225000 [271:39:01<129:14:58,  5.20s/it]
 60%|██████    | 135541/225000 [271:39:11<168:06:34,  6.77s/it]
 60%|██████    | 135542/225000 [271:39:15<146:05:30,  5.88s/it]
 60%|██████    | 135543/225000 [271:39:19<132:18:28,  5.32s/it]
 60%|██████    | 135544/225000 [271:39:28<159:32:55,  6.42s/it]
 60%|██████    | 135545/225000 [271:39:33<150:09:35,  6.04s/it]
 60%|██████    | 135546/225000 [271:39:39<145:11:36,  5.84s/it]
 60%|██████    | 135547/225000 [271:39:46<153:25:56,  6.17s/it]
 60%|██████    | 135548/225000 [271:39:50<142:49:02,  5.75s/it]
 60%|██████    | 135549/225000 [271:39:56<139:18:10,  5.61s/it]
 60%|██████    | 135550/225000 [271:40:01<137:15:18,  5.52s/it]
                                                               
{'loss': '0.805', 'grad_norm': '0.9993', 'learning_rate': '1.85e-05', 'epoch': '0.6024', 'num_input_tokens_seen': 450948296, 'train_runtime': '9.78e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135550/225000 [271:40:01<137:15:18,  5.52s/it]
 60%|██████    | 135551/225000 [271:40:06<133:39:29,  5.38s/it]
 60%|██████    | 135552/225000 [271:40:11<128:55:41,  5.19s/it]
 60%|██████    | 135553/225000 [271:40:16<129:33:27,  5.21s/it]
 60%|██████    | 135554/225000 [271:40:22<133:01:08,  5.35s/it]
 60%|██████    | 135555/225000 [271:40:29<150:29:20,  6.06s/it]
 60%|██████    | 135556/225000 [271:40:40<185:32:57,  7.47s/it]
 60%|██████    | 135557/225000 [271:40:43<152:34:03,  6.14s/it]
 60%|██████    | 135558/225000 [271:40:51<165:52:26,  6.68s/it]
 60%|██████    | 135559/225000 [271:40:55<144:16:09,  5.81s/it]
 60%|██████    | 135560/225000 [271:41:03<163:15:32,  6.57s/it]
 60%|██████    | 135561/225000 [271:41:07<142:22:44,  5.73s/it]
 60%|██████    | 135562/225000 [271:41:14<154:55:08,  6.24s/it]
 60%|██████    | 135563/225000 [271:41:18<137:01:01,  5.52s/it]
 60%|██████    | 135564/225000 [271:41:22<122:42:57,  4.94s/it]
 60%|██████    | 135565/225000 [271:41:31<156:57:55,  6.32s/it]
 60%|██████    | 135566/225000 [271:41:38<160:45:01,  6.47s/it]
 60%|██████    | 135567/225000 [271:41:42<137:50:39,  5.55s/it]
 60%|██████    | 135568/225000 [271:41:50<158:26:36,  6.38s/it]
 60%|██████    | 135569/225000 [271:41:54<140:09:29,  5.64s/it]
 60%|██████    | 135570/225000 [271:42:01<151:41:11,  6.11s/it]
 60%|██████    | 135571/225000 [271:42:05<133:10:47,  5.36s/it]
 60%|██████    | 135572/225000 [271:42:14<161:56:23,  6.52s/it]
 60%|██████    | 135573/225000 [271:42:18<143:30:10,  5.78s/it]
 60%|██████    | 135574/225000 [271:42:22<130:25:31,  5.25s/it]
 60%|██████    | 135575/225000 [271:42:27<126:36:25,  5.10s/it]
 60%|██████    | 135576/225000 [271:42:32<125:08:42,  5.04s/it]
 60%|██████    | 135577/225000 [271:42:37<125:33:14,  5.05s/it]
 60%|██████    | 135578/225000 [271:42:42<127:42:17,  5.14s/it]
 60%|██████    | 135579/225000 [271:42:47<128:14:54,  5.16s/it]
 60%|██████    | 135580/225000 [271:42:52<127:20:53,  5.13s/it]
 60%|██████    | 135581/225000 [271:42:58<130:02:11,  5.24s/it]
 60%|██████    | 135582/225000 [271:43:11<185:39:07,  7.47s/it]
 60%|██████    | 135583/225000 [271:43:17<174:56:18,  7.04s/it]
 60%|██████    | 135584/225000 [271:43:20<150:11:07,  6.05s/it]
 60%|██████    | 135585/225000 [271:43:25<143:10:01,  5.76s/it]
 60%|██████    | 135586/225000 [271:43:35<174:27:14,  7.02s/it]
 60%|██████    | 135587/225000 [271:43:41<165:52:12,  6.68s/it]
 60%|██████    | 135588/225000 [271:43:45<143:35:06,  5.78s/it]
 60%|██████    | 135589/225000 [271:43:48<127:21:44,  5.13s/it]
 60%|██████    | 135590/225000 [271:43:58<156:28:09,  6.30s/it]
 60%|██████    | 135591/225000 [271:44:04<155:25:37,  6.26s/it]
 60%|██████    | 135592/225000 [271:44:08<144:28:10,  5.82s/it]
 60%|██████    | 135593/225000 [271:44:15<150:07:40,  6.04s/it]
 60%|██████    | 135594/225000 [271:44:19<131:46:29,  5.31s/it]
 60%|██████    | 135595/225000 [271:44:28<158:45:25,  6.39s/it]
 60%|██████    | 135596/225000 [271:44:31<133:50:09,  5.39s/it]
 60%|██████    | 135597/225000 [271:44:37<140:57:59,  5.68s/it]
 60%|██████    | 135598/225000 [271:44:41<128:40:37,  5.18s/it]
 60%|██████    | 135599/225000 [271:44:48<144:30:51,  5.82s/it]
 60%|██████    | 135600/225000 [271:44:52<125:34:30,  5.06s/it]
                                                               
{'loss': '0.8026', 'grad_norm': '1.14', 'learning_rate': '1.849e-05', 'epoch': '0.6027', 'num_input_tokens_seen': 451078496, 'train_runtime': '9.783e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135600/225000 [271:44:52<125:34:30,  5.06s/it]
 60%|██████    | 135601/225000 [271:45:01<158:21:58,  6.38s/it]
 60%|██████    | 135602/225000 [271:45:05<139:26:27,  5.62s/it]
 60%|██████    | 135603/225000 [271:45:09<127:55:58,  5.15s/it]
 60%|██████    | 135604/225000 [271:45:13<119:29:40,  4.81s/it]
 60%|██████    | 135605/225000 [271:45:17<117:12:10,  4.72s/it]
 60%|██████    | 135606/225000 [271:45:26<145:28:07,  5.86s/it]
 60%|██████    | 135607/225000 [271:45:36<174:51:09,  7.04s/it]
 60%|██████    | 135608/225000 [271:45:40<157:17:57,  6.33s/it]
 60%|██████    | 135609/225000 [271:45:46<148:28:47,  5.98s/it]
 60%|██████    | 135610/225000 [271:45:51<141:57:07,  5.72s/it]
 60%|██████    | 135611/225000 [271:45:56<138:18:43,  5.57s/it]
 60%|██████    | 135612/225000 [271:46:01<133:24:18,  5.37s/it]
 60%|██████    | 135613/225000 [271:46:06<130:29:33,  5.26s/it]
 60%|██████    | 135614/225000 [271:46:11<131:09:39,  5.28s/it]
 60%|██████    | 135615/225000 [271:46:16<130:56:02,  5.27s/it]
 60%|██████    | 135616/225000 [271:46:26<162:16:38,  6.54s/it]
 60%|██████    | 135617/225000 [271:46:38<200:09:07,  8.06s/it]
 60%|██████    | 135618/225000 [271:46:41<166:05:58,  6.69s/it]
 60%|██████    | 135619/225000 [271:46:48<168:23:00,  6.78s/it]
 60%|██████    | 135620/225000 [271:46:53<154:25:51,  6.22s/it]
 60%|██████    | 135621/225000 [271:46:59<155:01:03,  6.24s/it]
 60%|██████    | 135622/225000 [271:47:03<136:30:11,  5.50s/it]
 60%|██████    | 135623/225000 [271:47:08<135:33:08,  5.46s/it]
 60%|██████    | 135624/225000 [271:47:13<133:03:06,  5.36s/it]
 60%|██████    | 135625/225000 [271:47:21<145:51:49,  5.88s/it]
 60%|██████    | 135626/225000 [271:47:26<141:43:20,  5.71s/it]
 60%|██████    | 135627/225000 [271:47:33<150:41:24,  6.07s/it]
 60%|██████    | 135628/225000 [271:47:37<134:47:17,  5.43s/it]
 60%|██████    | 135629/225000 [271:47:45<152:19:19,  6.14s/it]
 60%|██████    | 135630/225000 [271:47:50<145:40:13,  5.87s/it]
 60%|██████    | 135631/225000 [271:47:53<125:40:37,  5.06s/it]
 60%|██████    | 135632/225000 [271:47:58<126:30:16,  5.10s/it]
 60%|██████    | 135633/225000 [271:48:04<129:11:03,  5.20s/it]
 60%|██████    | 135634/225000 [271:48:13<164:03:11,  6.61s/it]
 60%|██████    | 135635/225000 [271:48:18<147:50:15,  5.96s/it]
 60%|██████    | 135636/225000 [271:48:25<156:27:46,  6.30s/it]
 60%|██████    | 135637/225000 [271:48:28<132:46:24,  5.35s/it]
 60%|██████    | 135638/225000 [271:48:31<118:01:05,  4.75s/it]
 60%|██████    | 135639/225000 [271:48:36<116:01:00,  4.67s/it]
 60%|██████    | 135640/225000 [271:48:47<160:05:30,  6.45s/it]
 60%|██████    | 135641/225000 [271:48:50<138:34:50,  5.58s/it]
 60%|██████    | 135642/225000 [271:48:55<135:14:53,  5.45s/it]
 60%|██████    | 135643/225000 [271:49:01<138:22:52,  5.58s/it]
 60%|██████    | 135644/225000 [271:49:06<135:12:46,  5.45s/it]
 60%|██████    | 135645/225000 [271:49:12<133:39:09,  5.38s/it]
 60%|██████    | 135646/225000 [271:49:17<132:33:21,  5.34s/it]
 60%|██████    | 135647/225000 [271:49:22<129:45:34,  5.23s/it]
 60%|██████    | 135648/225000 [271:49:27<127:27:44,  5.14s/it]
 60%|██████    | 135649/225000 [271:49:32<129:13:23,  5.21s/it]
 60%|██████    | 135650/225000 [271:49:37<128:35:02,  5.18s/it]
                                                               
{'loss': '0.8018', 'grad_norm': '1.131', 'learning_rate': '1.847e-05', 'epoch': '0.6029', 'num_input_tokens_seen': 451207864, 'train_runtime': '9.786e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135650/225000 [271:49:37<128:35:02,  5.18s/it]
 60%|██████    | 135651/225000 [271:49:43<131:47:11,  5.31s/it]
 60%|██████    | 135652/225000 [271:49:48<129:17:40,  5.21s/it]
 60%|██████    | 135653/225000 [271:49:53<130:47:57,  5.27s/it]
 60%|██████    | 135654/225000 [271:50:00<139:31:03,  5.62s/it]
 60%|██████    | 135655/225000 [271:50:12<193:20:33,  7.79s/it]
 60%|██████    | 135656/225000 [271:50:16<164:40:56,  6.64s/it]
 60%|██████    | 135657/225000 [271:50:20<141:24:45,  5.70s/it]
 60%|██████    | 135658/225000 [271:50:30<172:29:20,  6.95s/it]
 60%|██████    | 135659/225000 [271:50:34<152:18:19,  6.14s/it]
 60%|██████    | 135660/225000 [271:50:38<136:32:25,  5.50s/it]
 60%|██████    | 135661/225000 [271:50:42<125:49:50,  5.07s/it]
 60%|██████    | 135662/225000 [271:50:49<141:55:12,  5.72s/it]
 60%|██████    | 135663/225000 [271:50:59<174:38:07,  7.04s/it]
 60%|██████    | 135664/225000 [271:51:03<146:00:38,  5.88s/it]
 60%|██████    | 135665/225000 [271:51:11<161:00:26,  6.49s/it]
 60%|██████    | 135666/225000 [271:51:15<146:51:49,  5.92s/it]
 60%|██████    | 135667/225000 [271:51:23<159:10:08,  6.41s/it]
 60%|██████    | 135668/225000 [271:51:26<137:50:49,  5.56s/it]
 60%|██████    | 135669/225000 [271:51:31<132:19:59,  5.33s/it]
 60%|██████    | 135670/225000 [271:51:36<132:47:19,  5.35s/it]
 60%|██████    | 135671/225000 [271:51:41<128:49:12,  5.19s/it]
 60%|██████    | 135672/225000 [271:51:46<128:15:39,  5.17s/it]
 60%|██████    | 135673/225000 [271:51:51<125:51:32,  5.07s/it]
 60%|██████    | 135674/225000 [271:51:56<126:10:12,  5.08s/it]
 60%|██████    | 135675/225000 [271:52:01<124:02:55,  5.00s/it]
 60%|██████    | 135676/225000 [271:52:07<128:26:23,  5.18s/it]
 60%|██████    | 135677/225000 [271:52:14<143:59:21,  5.80s/it]
 60%|██████    | 135678/225000 [271:52:24<171:59:21,  6.93s/it]
 60%|██████    | 135679/225000 [271:52:27<146:54:15,  5.92s/it]
 60%|██████    | 135680/225000 [271:52:31<130:16:15,  5.25s/it]
 60%|██████    | 135681/225000 [271:52:42<174:36:54,  7.04s/it]
 60%|██████    | 135682/225000 [271:52:47<155:59:59,  6.29s/it]
 60%|██████    | 135683/225000 [271:52:51<142:01:51,  5.72s/it]
 60%|██████    | 135684/225000 [271:52:56<133:24:59,  5.38s/it]
 60%|██████    | 135685/225000 [271:53:00<127:30:59,  5.14s/it]
 60%|██████    | 135686/225000 [271:53:05<123:50:54,  4.99s/it]
 60%|██████    | 135687/225000 [271:53:09<120:18:57,  4.85s/it]
 60%|██████    | 135688/225000 [271:53:22<180:02:05,  7.26s/it]
 60%|██████    | 135689/225000 [271:53:26<155:34:11,  6.27s/it]
 60%|██████    | 135690/225000 [271:53:35<171:55:11,  6.93s/it]
 60%|██████    | 135691/225000 [271:53:39<152:13:45,  6.14s/it]
 60%|██████    | 135692/225000 [271:53:43<136:47:58,  5.51s/it]
 60%|██████    | 135693/225000 [271:53:50<146:46:58,  5.92s/it]
 60%|██████    | 135694/225000 [271:53:58<160:53:14,  6.49s/it]
 60%|██████    | 135695/225000 [271:54:01<137:59:12,  5.56s/it]
 60%|██████    | 135696/225000 [271:54:06<135:07:19,  5.45s/it]
 60%|██████    | 135697/225000 [271:54:11<132:59:46,  5.36s/it]
 60%|██████    | 135698/225000 [271:54:17<132:11:20,  5.33s/it]
 60%|██████    | 135699/225000 [271:54:22<132:32:18,  5.34s/it]
 60%|██████    | 135700/225000 [271:54:27<132:19:17,  5.33s/it]
                                                               
{'loss': '0.7976', 'grad_norm': '1.169', 'learning_rate': '1.845e-05', 'epoch': '0.6031', 'num_input_tokens_seen': 451337152, 'train_runtime': '9.789e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135700/225000 [271:54:27<132:19:17,  5.33s/it]
 60%|██████    | 135701/225000 [271:54:33<135:02:48,  5.44s/it]
 60%|██████    | 135702/225000 [271:54:40<146:45:01,  5.92s/it]
 60%|██████    | 135703/225000 [271:54:51<180:54:53,  7.29s/it]
 60%|██████    | 135704/225000 [271:54:54<153:46:18,  6.20s/it]
 60%|██████    | 135705/225000 [271:54:59<145:03:45,  5.85s/it]
 60%|██████    | 135706/225000 [271:55:04<138:47:23,  5.60s/it]
 60%|██████    | 135707/225000 [271:55:10<138:51:30,  5.60s/it]
 60%|██████    | 135708/225000 [271:55:15<137:34:47,  5.55s/it]
 60%|██████    | 135709/225000 [271:55:21<137:01:19,  5.52s/it]
 60%|██████    | 135710/225000 [271:55:29<159:08:03,  6.42s/it]
 60%|██████    | 135711/225000 [271:55:36<160:04:10,  6.45s/it]
 60%|██████    | 135712/225000 [271:55:40<141:26:27,  5.70s/it]
 60%|██████    | 135713/225000 [271:55:48<162:39:11,  6.56s/it]
 60%|██████    | 135714/225000 [271:55:53<146:23:00,  5.90s/it]
 60%|██████    | 135715/225000 [271:55:57<132:39:42,  5.35s/it]
 60%|██████    | 135716/225000 [271:56:01<127:51:22,  5.16s/it]
 60%|██████    | 135717/225000 [271:56:06<121:33:46,  4.90s/it]
 60%|██████    | 135718/225000 [271:56:13<142:32:36,  5.75s/it]
 60%|██████    | 135719/225000 [271:56:23<169:09:12,  6.82s/it]
 60%|██████    | 135720/225000 [271:56:26<143:47:19,  5.80s/it]
 60%|██████    | 135721/225000 [271:56:30<126:27:36,  5.10s/it]
 60%|██████    | 135722/225000 [271:56:34<120:30:35,  4.86s/it]
 60%|██████    | 135723/225000 [271:56:43<154:15:27,  6.22s/it]
 60%|██████    | 135724/225000 [271:56:50<155:35:12,  6.27s/it]
 60%|██████    | 135725/225000 [271:56:56<154:36:35,  6.23s/it]
 60%|██████    | 135726/225000 [271:57:01<145:05:53,  5.85s/it]
 60%|██████    | 135727/225000 [271:57:08<158:06:01,  6.38s/it]
 60%|██████    | 135728/225000 [271:57:12<137:46:22,  5.56s/it]
 60%|██████    | 135729/225000 [271:57:16<125:15:40,  5.05s/it]
 60%|██████    | 135730/225000 [271:57:24<146:39:30,  5.91s/it]
 60%|██████    | 135731/225000 [271:57:31<158:17:25,  6.38s/it]
 60%|██████    | 135732/225000 [271:57:35<140:58:57,  5.69s/it]
 60%|██████    | 135733/225000 [271:57:39<128:10:40,  5.17s/it]
 60%|██████    | 135734/225000 [271:57:48<154:02:15,  6.21s/it]
 60%|██████    | 135735/225000 [271:57:53<145:41:56,  5.88s/it]
 60%|██████    | 135736/225000 [271:57:58<141:35:57,  5.71s/it]
 60%|██████    | 135737/225000 [271:58:03<136:42:06,  5.51s/it]
 60%|██████    | 135738/225000 [271:58:10<147:05:56,  5.93s/it]
 60%|██████    | 135739/225000 [271:58:16<143:17:23,  5.78s/it]
 60%|██████    | 135740/225000 [271:58:23<150:50:55,  6.08s/it]
 60%|██████    | 135741/225000 [271:58:29<155:02:32,  6.25s/it]
 60%|██████    | 135742/225000 [271:58:33<135:24:24,  5.46s/it]
 60%|██████    | 135743/225000 [271:58:37<123:23:46,  4.98s/it]
 60%|██████    | 135744/225000 [271:58:49<174:21:53,  7.03s/it]
 60%|██████    | 135745/225000 [271:58:52<145:02:54,  5.85s/it]
 60%|██████    | 135746/225000 [271:58:56<136:12:14,  5.49s/it]
 60%|██████    | 135747/225000 [271:59:01<131:21:13,  5.30s/it]
 60%|██████    | 135748/225000 [271:59:06<130:43:26,  5.27s/it]
 60%|██████    | 135749/225000 [271:59:12<131:24:32,  5.30s/it]
 60%|██████    | 135750/225000 [271:59:17<132:15:06,  5.33s/it]
                                                               
{'loss': '0.7918', 'grad_norm': '1.144', 'learning_rate': '1.843e-05', 'epoch': '0.6033', 'num_input_tokens_seen': 451464976, 'train_runtime': '9.792e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135750/225000 [271:59:17<132:15:06,  5.33s/it]
 60%|██████    | 135751/225000 [271:59:22<130:06:43,  5.25s/it]
 60%|██████    | 135752/225000 [271:59:27<129:08:35,  5.21s/it]
 60%|██████    | 135753/225000 [271:59:32<124:52:30,  5.04s/it]
 60%|██████    | 135754/225000 [271:59:37<125:54:14,  5.08s/it]
 60%|██████    | 135755/225000 [271:59:42<127:27:32,  5.14s/it]
 60%|██████    | 135756/225000 [271:59:48<128:26:52,  5.18s/it]
 60%|██████    | 135757/225000 [272:00:00<185:23:08,  7.48s/it]
 60%|██████    | 135758/225000 [272:00:06<167:29:05,  6.76s/it]
 60%|██████    | 135759/225000 [272:00:14<179:02:49,  7.22s/it]
 60%|██████    | 135760/225000 [272:00:19<162:16:22,  6.55s/it]
 60%|██████    | 135761/225000 [272:00:23<140:50:11,  5.68s/it]
 60%|██████    | 135762/225000 [272:00:28<141:20:39,  5.70s/it]
 60%|██████    | 135763/225000 [272:00:37<163:00:34,  6.58s/it]
 60%|██████    | 135764/225000 [272:00:43<163:13:21,  6.58s/it]
 60%|██████    | 135765/225000 [272:00:48<147:36:57,  5.96s/it]
 60%|██████    | 135766/225000 [272:00:52<131:51:24,  5.32s/it]
 60%|██████    | 135767/225000 [272:00:55<119:15:11,  4.81s/it]
 60%|██████    | 135768/225000 [272:01:05<155:52:09,  6.29s/it]
 60%|██████    | 135769/225000 [272:01:09<140:46:42,  5.68s/it]
 60%|██████    | 135770/225000 [272:01:14<129:59:09,  5.24s/it]
 60%|██████    | 135771/225000 [272:01:18<121:45:38,  4.91s/it]
 60%|██████    | 135772/225000 [272:01:29<171:52:27,  6.93s/it]
 60%|██████    | 135773/225000 [272:01:33<146:33:03,  5.91s/it]
 60%|██████    | 135774/225000 [272:01:40<157:38:36,  6.36s/it]
 60%|██████    | 135775/225000 [272:01:45<143:30:49,  5.79s/it]
 60%|██████    | 135776/225000 [272:01:49<133:19:47,  5.38s/it]
 60%|██████    | 135777/225000 [272:01:54<126:51:57,  5.12s/it]
 60%|██████    | 135778/225000 [272:02:00<134:54:01,  5.44s/it]
 60%|██████    | 135779/225000 [272:02:10<167:22:11,  6.75s/it]
 60%|██████    | 135780/225000 [272:02:13<143:27:47,  5.79s/it]
 60%|██████    | 135781/225000 [272:02:22<162:54:06,  6.57s/it]
 60%|██████    | 135782/225000 [272:02:25<141:24:22,  5.71s/it]
 60%|██████    | 135783/225000 [272:02:30<131:22:16,  5.30s/it]
 60%|██████    | 135784/225000 [272:02:34<124:27:11,  5.02s/it]
 60%|██████    | 135785/225000 [272:02:45<166:16:39,  6.71s/it]
 60%|██████    | 135786/225000 [272:02:51<164:36:03,  6.64s/it]
 60%|██████    | 135787/225000 [272:02:55<142:34:07,  5.75s/it]
 60%|██████    | 135788/225000 [272:02:59<131:23:24,  5.30s/it]
 60%|██████    | 135789/225000 [272:03:10<172:00:56,  6.94s/it]
 60%|██████    | 135790/225000 [272:03:14<146:53:03,  5.93s/it]
 60%|██████    | 135791/225000 [272:03:22<166:14:03,  6.71s/it]
 60%|██████    | 135792/225000 [272:03:25<139:22:23,  5.62s/it]
 60%|██████    | 135793/225000 [272:03:29<123:02:35,  4.97s/it]
 60%|██████    | 135794/225000 [272:03:35<137:13:39,  5.54s/it]
 60%|██████    | 135795/225000 [272:03:43<150:07:58,  6.06s/it]
 60%|██████    | 135796/225000 [272:03:47<133:59:50,  5.41s/it]
 60%|██████    | 135797/225000 [272:03:51<123:55:06,  5.00s/it]
 60%|██████    | 135798/225000 [272:04:01<166:21:49,  6.71s/it]
 60%|██████    | 135799/225000 [272:04:05<145:21:12,  5.87s/it]
 60%|██████    | 135800/225000 [272:04:09<131:40:41,  5.31s/it]
                                                               
{'loss': '0.8001', 'grad_norm': '1.298', 'learning_rate': '1.842e-05', 'epoch': '0.6036', 'num_input_tokens_seen': 451594584, 'train_runtime': '9.794e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135800/225000 [272:04:09<131:40:41,  5.31s/it]
 60%|██████    | 135801/225000 [272:04:14<129:10:04,  5.21s/it]
 60%|██████    | 135802/225000 [272:04:26<177:03:48,  7.15s/it]
 60%|██████    | 135803/225000 [272:04:29<146:53:28,  5.93s/it]
 60%|██████    | 135804/225000 [272:04:34<140:26:50,  5.67s/it]
 60%|██████    | 135805/225000 [272:04:41<151:19:10,  6.11s/it]
 60%|██████    | 135806/225000 [272:04:47<145:22:52,  5.87s/it]
 60%|██████    | 135807/225000 [272:04:54<156:52:42,  6.33s/it]
 60%|██████    | 135808/225000 [272:04:57<134:52:08,  5.44s/it]
 60%|██████    | 135809/225000 [272:05:01<122:55:11,  4.96s/it]
 60%|██████    | 135810/225000 [272:05:09<147:02:34,  5.94s/it]
 60%|██████    | 135811/225000 [272:05:15<144:32:12,  5.83s/it]
 60%|██████    | 135812/225000 [272:05:19<132:15:12,  5.34s/it]
 60%|██████    | 135813/225000 [272:05:24<126:34:43,  5.11s/it]
 60%|██████    | 135814/225000 [272:05:28<119:53:25,  4.84s/it]
 60%|██████    | 135815/225000 [272:05:38<160:22:06,  6.47s/it]
 60%|██████    | 135816/225000 [272:05:42<137:19:03,  5.54s/it]
 60%|██████    | 135817/225000 [272:05:46<125:36:11,  5.07s/it]
 60%|██████    | 135818/225000 [272:05:54<148:34:16,  6.00s/it]
 60%|██████    | 135819/225000 [272:06:01<158:03:40,  6.38s/it]
 60%|██████    | 135820/225000 [272:06:05<138:27:41,  5.59s/it]
 60%|██████    | 135821/225000 [272:06:14<169:16:30,  6.83s/it]
 60%|██████    | 135822/225000 [272:06:18<141:29:37,  5.71s/it]
 60%|██████    | 135823/225000 [272:06:27<171:35:16,  6.93s/it]
 60%|██████    | 135824/225000 [272:06:31<144:18:33,  5.83s/it]
 60%|██████    | 135825/225000 [272:06:36<140:30:40,  5.67s/it]
 60%|██████    | 135826/225000 [272:06:43<149:15:23,  6.03s/it]
 60%|██████    | 135827/225000 [272:06:47<135:48:56,  5.48s/it]
 60%|██████    | 135828/225000 [272:06:52<130:36:33,  5.27s/it]
 60%|██████    | 135829/225000 [272:06:57<127:41:33,  5.16s/it]
 60%|██████    | 135830/225000 [272:07:01<125:12:50,  5.06s/it]
 60%|██████    | 135831/225000 [272:07:07<127:20:01,  5.14s/it]
 60%|██████    | 135832/225000 [272:07:11<123:59:00,  5.01s/it]
 60%|██████    | 135833/225000 [272:07:16<124:05:57,  5.01s/it]
 60%|██████    | 135834/225000 [272:07:27<167:39:47,  6.77s/it]
 60%|██████    | 135835/225000 [272:07:34<166:20:56,  6.72s/it]
 60%|██████    | 135836/225000 [272:07:39<156:32:31,  6.32s/it]
 60%|██████    | 135837/225000 [272:07:45<148:54:43,  6.01s/it]
 60%|██████    | 135838/225000 [272:07:53<167:05:18,  6.75s/it]
 60%|██████    | 135839/225000 [272:07:57<142:41:08,  5.76s/it]
 60%|██████    | 135840/225000 [272:08:01<131:39:54,  5.32s/it]
 60%|██████    | 135841/225000 [272:08:05<125:46:02,  5.08s/it]
 60%|██████    | 135842/225000 [272:08:10<125:58:40,  5.09s/it]
 60%|██████    | 135843/225000 [272:08:16<127:46:27,  5.16s/it]
 60%|██████    | 135844/225000 [272:08:21<126:56:18,  5.13s/it]
 60%|██████    | 135845/225000 [272:08:26<127:53:33,  5.16s/it]
 60%|██████    | 135846/225000 [272:08:38<176:29:14,  7.13s/it]
 60%|██████    | 135847/225000 [272:08:44<172:46:46,  6.98s/it]
 60%|██████    | 135848/225000 [272:08:48<146:53:50,  5.93s/it]
 60%|██████    | 135849/225000 [272:08:57<167:09:23,  6.75s/it]
 60%|██████    | 135850/225000 [272:09:00<143:22:53,  5.79s/it]
                                                               
{'loss': '0.798', 'grad_norm': '1.174', 'learning_rate': '1.84e-05', 'epoch': '0.6038', 'num_input_tokens_seen': 451724856, 'train_runtime': '9.797e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135850/225000 [272:09:00<143:22:53,  5.79s/it]
 60%|██████    | 135851/225000 [272:09:08<162:00:24,  6.54s/it]
 60%|██████    | 135852/225000 [272:09:13<150:40:52,  6.08s/it]
 60%|██████    | 135853/225000 [272:09:17<134:50:15,  5.45s/it]
 60%|██████    | 135854/225000 [272:09:26<158:16:49,  6.39s/it]
 60%|██████    | 135855/225000 [272:09:29<136:01:39,  5.49s/it]
 60%|██████    | 135856/225000 [272:09:33<123:23:57,  4.98s/it]
 60%|██████    | 135857/225000 [272:09:45<170:34:18,  6.89s/it]
 60%|██████    | 135858/225000 [272:09:48<144:42:13,  5.84s/it]
 60%|██████    | 135859/225000 [272:09:55<155:08:31,  6.27s/it]
 60%|██████    | 135860/225000 [272:09:59<138:33:36,  5.60s/it]
 60%|██████    | 135861/225000 [272:10:04<128:53:55,  5.21s/it]
 60%|██████    | 135862/225000 [272:10:09<134:30:45,  5.43s/it]
 60%|██████    | 135863/225000 [272:10:18<155:50:47,  6.29s/it]
 60%|██████    | 135864/225000 [272:10:21<134:47:08,  5.44s/it]
 60%|██████    | 135865/225000 [272:10:27<134:52:56,  5.45s/it]
 60%|██████    | 135866/225000 [272:10:35<156:41:03,  6.33s/it]
 60%|██████    | 135867/225000 [272:10:38<133:27:33,  5.39s/it]
 60%|██████    | 135868/225000 [272:10:46<149:52:31,  6.05s/it]
 60%|██████    | 135869/225000 [272:10:49<131:37:02,  5.32s/it]
 60%|██████    | 135870/225000 [272:10:55<135:46:12,  5.48s/it]
 60%|██████    | 135871/225000 [272:11:05<166:21:48,  6.72s/it]
 60%|██████    | 135872/225000 [272:11:08<141:47:56,  5.73s/it]
 60%|██████    | 135873/225000 [272:11:17<161:02:26,  6.50s/it]
 60%|██████    | 135874/225000 [272:11:21<143:49:24,  5.81s/it]
 60%|██████    | 135875/225000 [272:11:27<142:43:23,  5.76s/it]
 60%|██████    | 135876/225000 [272:11:33<145:06:57,  5.86s/it]
 60%|██████    | 135877/225000 [272:11:36<129:34:02,  5.23s/it]
 60%|██████    | 135878/225000 [272:11:41<123:02:15,  4.97s/it]
 60%|██████    | 135879/225000 [272:11:51<164:16:19,  6.64s/it]
 60%|██████    | 135880/225000 [272:11:55<139:47:59,  5.65s/it]
 60%|██████    | 135881/225000 [272:11:59<131:45:05,  5.32s/it]
 60%|██████    | 135882/225000 [272:12:03<123:25:46,  4.99s/it]
 60%|██████    | 135883/225000 [272:12:08<123:08:24,  4.97s/it]
 60%|██████    | 135884/225000 [272:12:13<122:02:55,  4.93s/it]
 60%|██████    | 135885/225000 [272:12:18<119:18:54,  4.82s/it]
 60%|██████    | 135886/225000 [272:12:26<148:36:43,  6.00s/it]
 60%|██████    | 135887/225000 [272:12:35<168:03:07,  6.79s/it]
 60%|██████    | 135888/225000 [272:12:38<141:39:07,  5.72s/it]
 60%|██████    | 135889/225000 [272:12:44<141:59:19,  5.74s/it]
 60%|██████    | 135890/225000 [272:12:51<148:43:40,  6.01s/it]
 60%|██████    | 135891/225000 [272:12:57<152:03:07,  6.14s/it]
 60%|██████    | 135892/225000 [272:13:04<154:36:55,  6.25s/it]
 60%|██████    | 135893/225000 [272:13:07<131:59:54,  5.33s/it]
 60%|██████    | 135894/225000 [272:13:15<153:48:41,  6.21s/it]
 60%|██████    | 135895/225000 [272:13:19<135:04:54,  5.46s/it]
 60%|██████    | 135896/225000 [272:13:28<160:11:14,  6.47s/it]
 60%|██████    | 135897/225000 [272:13:33<149:41:00,  6.05s/it]
 60%|██████    | 135898/225000 [272:13:38<141:26:05,  5.71s/it]
 60%|██████    | 135899/225000 [272:13:43<134:50:54,  5.45s/it]
 60%|██████    | 135900/225000 [272:13:48<133:01:23,  5.37s/it]
                                                               
{'loss': '0.8043', 'grad_norm': '1.113', 'learning_rate': '1.838e-05', 'epoch': '0.604', 'num_input_tokens_seen': 451854872, 'train_runtime': '9.8e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135900/225000 [272:13:48<133:01:23,  5.37s/it]
 60%|██████    | 135901/225000 [272:13:53<130:03:57,  5.26s/it]
 60%|██████    | 135902/225000 [272:13:58<126:58:44,  5.13s/it]
 60%|██████    | 135903/225000 [272:14:03<126:36:28,  5.12s/it]
 60%|██████    | 135904/225000 [272:14:07<123:28:58,  4.99s/it]
 60%|██████    | 135905/225000 [272:14:16<151:23:28,  6.12s/it]
 60%|██████    | 135906/225000 [272:14:25<172:21:30,  6.96s/it]
 60%|██████    | 135907/225000 [272:14:29<148:38:11,  6.01s/it]
 60%|██████    | 135908/225000 [272:14:36<159:49:37,  6.46s/it]
 60%|██████    | 135909/225000 [272:14:41<148:06:41,  5.98s/it]
 60%|██████    | 135910/225000 [272:14:46<136:24:01,  5.51s/it]
 60%|██████    | 135911/225000 [272:14:51<133:01:02,  5.38s/it]
 60%|██████    | 135912/225000 [272:14:56<129:40:44,  5.24s/it]
 60%|██████    | 135913/225000 [272:15:00<122:09:14,  4.94s/it]
 60%|██████    | 135914/225000 [272:15:07<141:30:44,  5.72s/it]
 60%|██████    | 135915/225000 [272:15:18<174:59:59,  7.07s/it]
 60%|██████    | 135916/225000 [272:15:22<158:02:33,  6.39s/it]
 60%|██████    | 135917/225000 [272:15:27<145:48:08,  5.89s/it]
 60%|██████    | 135918/225000 [272:15:31<127:45:39,  5.16s/it]
 60%|██████    | 135919/225000 [272:15:41<167:25:09,  6.77s/it]
 60%|██████    | 135920/225000 [272:15:45<149:49:21,  6.05s/it]
 60%|██████    | 135921/225000 [272:15:53<159:09:43,  6.43s/it]
 60%|██████    | 135922/225000 [272:15:58<147:52:39,  5.98s/it]
 60%|██████    | 135923/225000 [272:16:02<138:53:27,  5.61s/it]
 60%|██████    | 135924/225000 [272:16:07<134:53:32,  5.45s/it]
 60%|██████    | 135925/225000 [272:16:12<128:51:15,  5.21s/it]
 60%|██████    | 135926/225000 [272:16:17<123:14:31,  4.98s/it]
 60%|██████    | 135927/225000 [272:16:22<123:58:38,  5.01s/it]
 60%|██████    | 135928/225000 [272:16:27<123:29:02,  4.99s/it]
 60%|██████    | 135929/225000 [272:16:38<169:48:03,  6.86s/it]
 60%|██████    | 135930/225000 [272:16:44<165:55:37,  6.71s/it]
 60%|██████    | 135931/225000 [272:16:48<143:32:23,  5.80s/it]
 60%|██████    | 135932/225000 [272:16:55<155:20:36,  6.28s/it]
 60%|██████    | 135933/225000 [272:17:01<150:44:21,  6.09s/it]
 60%|██████    | 135934/225000 [272:17:09<163:53:40,  6.62s/it]
 60%|██████    | 135935/225000 [272:17:12<139:30:26,  5.64s/it]
 60%|██████    | 135936/225000 [272:17:21<164:56:25,  6.67s/it]
 60%|██████    | 135937/225000 [272:17:25<141:35:21,  5.72s/it]
 60%|██████    | 135938/225000 [272:17:30<139:44:22,  5.65s/it]
 60%|██████    | 135939/225000 [272:17:35<135:47:48,  5.49s/it]
 60%|██████    | 135940/225000 [272:17:41<134:22:19,  5.43s/it]
 60%|██████    | 135941/225000 [272:17:46<135:03:33,  5.46s/it]
 60%|██████    | 135942/225000 [272:17:52<137:21:37,  5.55s/it]
 60%|██████    | 135943/225000 [272:17:58<137:50:59,  5.57s/it]
 60%|██████    | 135944/225000 [272:18:03<135:35:59,  5.48s/it]
 60%|██████    | 135945/225000 [272:18:08<135:03:50,  5.46s/it]
 60%|██████    | 135946/225000 [272:18:16<155:53:08,  6.30s/it]
 60%|██████    | 135947/225000 [272:18:27<188:48:56,  7.63s/it]
 60%|██████    | 135948/225000 [272:18:31<160:38:00,  6.49s/it]
 60%|██████    | 135949/225000 [272:18:39<173:31:57,  7.02s/it]
 60%|██████    | 135950/225000 [272:18:43<152:29:28,  6.16s/it]
                                                               
{'loss': '0.7977', 'grad_norm': '1.149', 'learning_rate': '1.836e-05', 'epoch': '0.6042', 'num_input_tokens_seen': 451985336, 'train_runtime': '9.803e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 135950/225000 [272:18:43<152:29:28,  6.16s/it]
 60%|██████    | 135951/225000 [272:18:51<160:17:40,  6.48s/it]
 60%|██████    | 135952/225000 [272:18:54<135:39:22,  5.48s/it]
 60%|██████    | 135953/225000 [272:18:58<125:37:30,  5.08s/it]
 60%|██████    | 135954/225000 [272:19:03<125:48:52,  5.09s/it]
 60%|██████    | 135955/225000 [272:19:08<124:48:38,  5.05s/it]
 60%|██████    | 135956/225000 [272:19:13<124:31:18,  5.03s/it]
 60%|██████    | 135957/225000 [272:19:18<124:44:32,  5.04s/it]
 60%|██████    | 135958/225000 [272:19:23<124:21:48,  5.03s/it]
 60%|██████    | 135959/225000 [272:19:30<135:11:29,  5.47s/it]
 60%|██████    | 135960/225000 [272:19:42<186:02:22,  7.52s/it]
 60%|██████    | 135961/225000 [272:19:46<158:41:13,  6.42s/it]
 60%|██████    | 135962/225000 [272:19:49<138:16:41,  5.59s/it]
 60%|██████    | 135963/225000 [272:19:53<126:12:05,  5.10s/it]
 60%|██████    | 135964/225000 [272:20:02<149:38:14,  6.05s/it]
 60%|██████    | 135965/225000 [272:20:09<162:31:23,  6.57s/it]
 60%|██████    | 135966/225000 [272:20:17<170:53:50,  6.91s/it]
 60%|██████    | 135967/225000 [272:20:22<157:21:11,  6.36s/it]
 60%|██████    | 135968/225000 [272:20:26<136:19:59,  5.51s/it]
 60%|██████    | 135969/225000 [272:20:36<168:30:36,  6.81s/it]
 60%|██████    | 135970/225000 [272:20:39<143:34:50,  5.81s/it]
 60%|██████    | 135971/225000 [272:20:44<134:53:07,  5.45s/it]
 60%|██████    | 135972/225000 [272:20:54<168:42:20,  6.82s/it]
 60%|██████    | 135973/225000 [272:20:57<142:26:43,  5.76s/it]
 60%|██████    | 135974/225000 [272:21:01<126:46:59,  5.13s/it]
 60%|██████    | 135975/225000 [272:21:06<127:41:56,  5.16s/it]
 60%|██████    | 135976/225000 [272:21:11<131:06:04,  5.30s/it]
 60%|██████    | 135977/225000 [272:21:17<130:18:25,  5.27s/it]
 60%|██████    | 135978/225000 [272:21:22<130:14:09,  5.27s/it]
 60%|██████    | 135979/225000 [272:21:27<129:05:43,  5.22s/it]
 60%|██████    | 135980/225000 [272:21:32<128:08:35,  5.18s/it]
 60%|██████    | 135981/225000 [272:21:38<130:35:14,  5.28s/it]
 60%|██████    | 135982/225000 [272:21:43<131:26:40,  5.32s/it]
 60%|██████    | 135983/225000 [272:21:50<143:37:25,  5.81s/it]
 60%|██████    | 135984/225000 [272:21:56<142:44:43,  5.77s/it]
 60%|██████    | 135985/225000 [272:22:01<138:30:24,  5.60s/it]
 60%|██████    | 135986/225000 [272:22:06<135:43:35,  5.49s/it]
 60%|██████    | 135987/225000 [272:22:13<143:24:56,  5.80s/it]
 60%|██████    | 135988/225000 [272:22:25<195:29:08,  7.91s/it]
 60%|██████    | 135989/225000 [272:22:32<184:37:15,  7.47s/it]
 60%|██████    | 135990/225000 [272:22:35<155:36:47,  6.29s/it]
 60%|██████    | 135991/225000 [272:22:45<181:55:20,  7.36s/it]
 60%|██████    | 135992/225000 [272:22:50<161:00:46,  6.51s/it]
 60%|██████    | 135993/225000 [272:22:59<177:11:24,  7.17s/it]
 60%|██████    | 135994/225000 [272:23:02<153:24:13,  6.20s/it]
 60%|██████    | 135995/225000 [272:23:06<134:25:57,  5.44s/it]
 60%|██████    | 135996/225000 [272:23:11<126:54:54,  5.13s/it]
 60%|██████    | 135997/225000 [272:23:15<123:26:18,  4.99s/it]
 60%|██████    | 135998/225000 [272:23:20<121:34:50,  4.92s/it]
 60%|██████    | 135999/225000 [272:23:25<122:13:11,  4.94s/it]
 60%|██████    | 136000/225000 [272:23:30<125:58:03,  5.10s/it]
                                                               
{'loss': '0.7968', 'grad_norm': '1.076', 'learning_rate': '1.835e-05', 'epoch': '0.6044', 'num_input_tokens_seen': 452116688, 'train_runtime': '9.806e+05', 'train_tokens_per_second': '461.1'}

 60%|██████    | 136000/225000 [272:23:30<125:58:03,  5.10s/it]
 60%|██████    | 136001/225000 [272:23:42<175:32:09,  7.10s/it]
 60%|██████    | 136002/225000 [272:23:50<179:52:17,  7.28s/it]
 60%|██████    | 136003/225000 [272:23:54<158:29:37,  6.41s/it]
 60%|██████    | 136004/225000 [272:23:59<143:08:28,  5.79s/it]
 60%|██████    | 136005/225000 [272:24:03<132:37:20,  5.36s/it]
 60%|██████    | 136006/225000 [272:24:08<127:48:07,  5.17s/it]
 60%|██████    | 136007/225000 [272:24:12<123:18:33,  4.99s/it]
 60%|██████    | 136008/225000 [272:24:17<120:06:19,  4.86s/it]
 60%|██████    | 136009/225000 [272:24:21<117:12:30,  4.74s/it]
 60%|██████    | 136010/225000 [272:24:32<164:31:02,  6.66s/it]
 60%|██████    | 136011/225000 [272:24:40<174:13:57,  7.05s/it]
 60%|██████    | 136012/225000 [272:24:45<154:31:41,  6.25s/it]
 60%|██████    | 136013/225000 [272:24:52<161:31:46,  6.53s/it]
 60%|██████    | 136014/225000 [272:24:56<139:17:11,  5.63s/it]
 60%|██████    | 136015/225000 [272:25:05<171:24:20,  6.93s/it]
 60%|██████    | 136016/225000 [272:25:09<146:30:14,  5.93s/it]
 60%|██████    | 136017/225000 [272:25:16<157:36:35,  6.38s/it]
 60%|██████    | 136018/225000 [272:25:21<146:44:28,  5.94s/it]
 60%|██████    | 136019/225000 [272:25:25<132:02:37,  5.34s/it]
 60%|██████    | 136020/225000 [272:25:30<129:10:19,  5.23s/it]
 60%|██████    | 136021/225000 [272:25:42<175:47:31,  7.11s/it]
 60%|██████    | 136022/225000 [272:25:45<149:15:44,  6.04s/it]
 60%|██████    | 136023/225000 [272:25:49<132:04:17,  5.34s/it]
 60%|██████    | 136024/225000 [272:26:01<178:14:36,  7.21s/it]
 60%|██████    | 136025/225000 [272:26:04<148:24:49,  6.00s/it]
 60%|██████    | 136026/225000 [272:26:11<158:36:42,  6.42s/it]
 60%|██████    | 136027/225000 [272:26:15<135:55:26,  5.50s/it]
 60%|██████    | 136028/225000 [272:26:21<141:45:01,  5.74s/it]
 60%|██████    | 136029/225000 [272:26:29<156:45:17,  6.34s/it]
 60%|██████    | 136030/225000 [272:26:33<141:51:23,  5.74s/it]
 60%|██████    | 136031/225000 [272:26:37<131:24:07,  5.32s/it]
 60%|██████    | 136032/225000 [272:26:42<124:46:15,  5.05s/it]
 60%|██████    | 136033/225000 [272:26:46<122:39:16,  4.96s/it]
 60%|██████    | 136034/225000 [272:26:51<118:10:21,  4.78s/it]
 60%|██████    | 136035/225000 [272:27:02<165:45:46,  6.71s/it]
 60%|██████    | 136036/225000 [272:27:07<156:23:03,  6.33s/it]
 60%|██████    | 136037/225000 [272:27:13<152:17:34,  6.16s/it]
 60%|██████    | 136038/225000 [272:27:21<162:50:57,  6.59s/it]
 60%|██████    | 136039/225000 [272:27:25<144:54:28,  5.86s/it]
 60%|██████    | 136040/225000 [272:27:29<131:30:56,  5.32s/it]
 60%|██████    | 136041/225000 [272:27:36<144:14:17,  5.84s/it]
 60%|██████    | 136042/225000 [272:27:45<167:34:59,  6.78s/it]
 60%|██████    | 136043/225000 [272:27:49<144:35:53,  5.85s/it]
 60%|██████    | 136044/225000 [272:27:55<144:32:01,  5.85s/it]
 60%|██████    | 136045/225000 [272:28:05<176:12:56,  7.13s/it]
 60%|██████    | 136046/225000 [272:28:08<145:35:35,  5.89s/it]
 60%|██████    | 136047/225000 [272:28:16<163:37:58,  6.62s/it]
 60%|██████    | 136048/225000 [272:28:20<141:38:50,  5.73s/it]
 60%|██████    | 136049/225000 [272:28:25<135:24:49,  5.48s/it]
 60%|██████    | 136050/225000 [272:28:29<129:12:36,  5.23s/it]
                                                               
{'loss': '0.7998', 'grad_norm': '1.197', 'learning_rate': '1.833e-05', 'epoch': '0.6047', 'num_input_tokens_seen': 452246904, 'train_runtime': '9.809e+05', 'train_tokens_per_second': '461'}

 60%|██████    | 136050/225000 [272:28:29<129:12:36,  5.23s/it]
 60%|██████    | 136051/225000 [272:28:34<127:24:47,  5.16s/it]
 60%|██████    | 136052/225000 [272:28:39<124:53:13,  5.05s/it]
 60%|██████    | 136053/225000 [272:28:46<135:24:20,  5.48s/it]
 60%|██████    | 136054/225000 [272:28:57<180:32:10,  7.31s/it]
 60%|██████    | 136055/225000 [272:29:00<150:41:34,  6.10s/it]
 60%|██████    | 136056/225000 [272:29:08<161:19:29,  6.53s/it]
 60%|██████    | 136057/225000 [272:29:12<143:38:44,  5.81s/it]
 60%|██████    | 136058/225000 [272:29:22<172:00:01,  6.96s/it]
 60%|██████    | 136059/225000 [272:29:25<145:43:18,  5.90s/it]
 60%|██████    | 136060/225000 [272:29:31<148:43:45,  6.02s/it]
 60%|██████    | 136061/225000 [272:29:39<163:19:42,  6.61s/it]
 60%|██████    | 136062/225000 [272:29:43<137:39:57,  5.57s/it]
 60%|██████    | 136063/225000 [272:29:51<158:24:31,  6.41s/it]
 60%|██████    | 136064/225000 [272:29:55<137:31:05,  5.57s/it]
 60%|██████    | 136065/225000 [272:30:00<133:12:48,  5.39s/it]
 60%|██████    | 136066/225000 [272:30:04<129:10:06,  5.23s/it]
 60%|██████    | 136067/225000 [272:30:09<126:55:40,  5.14s/it]
 60%|██████    | 136068/225000 [272:30:14<126:57:41,  5.14s/it]
 60%|██████    | 136069/225000 [272:30:19<125:29:30,  5.08s/it]
 60%|██████    | 136070/225000 [272:30:24<120:35:01,  4.88s/it]
 60%|██████    | 136071/225000 [272:30:32<146:59:17,  5.95s/it]
 60%|██████    | 136072/225000 [272:30:42<178:08:28,  7.21s/it]
 60%|██████    | 136073/225000 [272:30:46<148:53:23,  6.03s/it]
 60%|██████    | 136074/225000 [272:30:52<154:52:54,  6.27s/it]
 60%|██████    | 136075/225000 [272:31:00<163:05:33,  6.60s/it]
 60%|██████    | 136076/225000 [272:31:06<159:39:39,  6.46s/it]
 60%|██████    | 136077/225000 [272:31:11<147:39:35,  5.98s/it]
 60%|██████    | 136078/225000 [272:31:15<136:36:47,  5.53s/it]
 60%|██████    | 136079/225000 [272:31:20<130:39:09,  5.29s/it]
 60%|██████    | 136080/225000 [272:31:25<124:35:22,  5.04s/it]
 60%|██████    | 136081/225000 [272:31:29<122:34:58,  4.96s/it]
 60%|██████    | 136082/225000 [272:31:34<120:19:50,  4.87s/it]
 60%|██████    | 136083/225000 [272:31:46<171:09:11,  6.93s/it]
 60%|██████    | 136084/225000 [272:31:52<168:47:56,  6.83s/it]
 60%|██████    | 136085/225000 [272:31:56<148:51:35,  6.03s/it]
 60%|██████    | 136086/225000 [272:32:01<136:26:28,  5.52s/it]
 60%|██████    | 136087/225000 [272:32:05<126:20:22,  5.12s/it]
 60%|██████    | 136088/225000 [272:32:15<161:51:08,  6.55s/it]
 60%|██████    | 136089/225000 [272:32:22<167:33:13,  6.78s/it]
 60%|██████    | 136090/225000 [272:32:26<146:44:11,  5.94s/it]
 60%|██████    | 136091/225000 [272:32:30<134:36:58,  5.45s/it]
 60%|██████    | 136092/225000 [272:32:37<141:40:20,  5.74s/it]
 60%|██████    | 136093/225000 [272:32:45<158:41:00,  6.43s/it]
 60%|██████    | 136094/225000 [272:32:49<141:04:54,  5.71s/it]
 60%|██████    | 136095/225000 [272:32:54<135:39:08,  5.49s/it]
 60%|██████    | 136096/225000 [272:33:05<176:05:55,  7.13s/it]
 60%|██████    | 136097/225000 [272:33:09<151:19:03,  6.13s/it]
 60%|██████    | 136098/225000 [272:33:12<133:39:43,  5.41s/it]
 60%|██████    | 136099/225000 [272:33:17<125:30:54,  5.08s/it]
 60%|██████    | 136100/225000 [272:33:30<182:54:38,  7.41s/it]
                                                               
{'loss': '0.8026', 'grad_norm': '1.11', 'learning_rate': '1.831e-05', 'epoch': '0.6049', 'num_input_tokens_seen': 452377360, 'train_runtime': '9.812e+05', 'train_tokens_per_second': '461'}

 60%|██████    | 136100/225000 [272:33:30<182:54:38,  7.41s/it]
 60%|██████    | 136101/225000 [272:33:33<152:23:40,  6.17s/it]
 60%|██████    | 136102/225000 [272:33:41<165:27:47,  6.70s/it]
 60%|██████    | 136103/225000 [272:33:44<141:40:07,  5.74s/it]
 60%|██████    | 136104/225000 [272:33:49<137:49:30,  5.58s/it]
 60%|██████    | 136105/225000 [272:33:56<141:36:19,  5.73s/it]
 60%|██████    | 136106/225000 [272:34:02<148:24:42,  6.01s/it]
 60%|██████    | 136107/225000 [272:34:09<154:39:28,  6.26s/it]
 60%|██████    | 136108/225000 [272:34:16<156:13:48,  6.33s/it]
 60%|██████    | 136109/225000 [272:34:20<139:38:06,  5.66s/it]
 60%|██████    | 136110/225000 [272:34:24<129:46:11,  5.26s/it]
 60%|██████    | 136111/225000 [272:34:28<121:26:01,  4.92s/it]
 60%|██████    | 136112/225000 [272:34:32<114:28:33,  4.64s/it]
 60%|██████    | 136113/225000 [272:34:43<161:53:15,  6.56s/it]
 60%|██████    | 136114/225000 [272:34:47<140:40:38,  5.70s/it]
 60%|██████    | 136115/225000 [272:34:55<162:31:00,  6.58s/it]
 60%|██████    | 136116/225000 [272:35:00<147:28:07,  5.97s/it]
 60%|██████    | 136117/225000 [272:35:07<152:00:52,  6.16s/it]
 60%|██████    | 136118/225000 [272:35:11<138:16:17,  5.60s/it]
 60%|██████    | 136119/225000 [272:35:15<128:54:17,  5.22s/it]
 60%|██████    | 136120/225000 [272:35:20<126:24:57,  5.12s/it]
 60%|██████    | 136121/225000 [272:35:25<123:08:20,  4.99s/it]
 60%|██████    | 136122/225000 [272:35:30<122:20:27,  4.96s/it]
 60%|██████    | 136123/225000 [272:35:38<145:23:47,  5.89s/it]
 60%|██████    | 136124/225000 [272:35:49<185:24:07,  7.51s/it]
 60%|██████    | 136125/225000 [272:35:53<160:32:10,  6.50s/it]
 61%|██████    | 136126/225000 [272:36:00<164:17:23,  6.65s/it]
 61%|██████    | 136127/225000 [272:36:04<143:03:43,  5.80s/it]
 61%|██████    | 136128/225000 [272:36:08<128:07:02,  5.19s/it]
 61%|██████    | 136129/225000 [272:36:15<146:54:01,  5.95s/it]
 61%|██████    | 136130/225000 [272:36:24<163:41:24,  6.63s/it]
 61%|██████    | 136131/225000 [272:36:27<138:58:27,  5.63s/it]
 61%|██████    | 136132/225000 [272:36:31<126:23:11,  5.12s/it]
 61%|██████    | 136133/225000 [272:36:42<170:32:26,  6.91s/it]
 61%|██████    | 136134/225000 [272:36:46<145:14:43,  5.88s/it]
 61%|██████    | 136135/225000 [272:36:50<133:02:07,  5.39s/it]
 61%|██████    | 136136/225000 [272:36:55<131:52:16,  5.34s/it]
 61%|██████    | 136137/225000 [272:37:00<130:04:05,  5.27s/it]
 61%|██████    | 136138/225000 [272:37:05<128:02:49,  5.19s/it]
 61%|██████    | 136139/225000 [272:37:10<127:36:35,  5.17s/it]
 61%|██████    | 136140/225000 [272:37:15<127:50:21,  5.18s/it]
 61%|██████    | 136141/225000 [272:37:21<129:22:03,  5.24s/it]
 61%|██████    | 136142/225000 [272:37:29<153:22:54,  6.21s/it]
 61%|██████    | 136143/225000 [272:37:34<143:34:32,  5.82s/it]
 61%|██████    | 136144/225000 [272:37:40<143:02:41,  5.80s/it]
 61%|██████    | 136145/225000 [272:37:45<139:05:50,  5.64s/it]
 61%|██████    | 136146/225000 [272:37:51<139:22:32,  5.65s/it]
 61%|██████    | 136147/225000 [272:37:56<135:42:55,  5.50s/it]
 61%|██████    | 136148/225000 [272:38:02<136:43:59,  5.54s/it]
 61%|██████    | 136149/225000 [272:38:11<164:37:58,  6.67s/it]
 61%|██████    | 136150/225000 [272:38:23<201:51:11,  8.18s/it]
                                                               
{'loss': '0.7969', 'grad_norm': '0.976', 'learning_rate': '1.829e-05', 'epoch': '0.6051', 'num_input_tokens_seen': 452507312, 'train_runtime': '9.815e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136150/225000 [272:38:23<201:51:11,  8.18s/it]
 61%|██████    | 136151/225000 [272:38:26<165:05:47,  6.69s/it]
 61%|██████    | 136152/225000 [272:38:30<145:39:05,  5.90s/it]
 61%|██████    | 136153/225000 [272:38:34<132:21:31,  5.36s/it]
 61%|██████    | 136154/225000 [272:38:46<182:41:52,  7.40s/it]
 61%|██████    | 136155/225000 [272:38:51<162:52:34,  6.60s/it]
 61%|██████    | 136156/225000 [272:38:55<144:16:41,  5.85s/it]
 61%|██████    | 136157/225000 [272:39:04<167:58:52,  6.81s/it]
 61%|██████    | 136158/225000 [272:39:08<143:47:09,  5.83s/it]
 61%|██████    | 136159/225000 [272:39:14<149:26:20,  6.06s/it]
 61%|██████    | 136160/225000 [272:39:21<152:46:43,  6.19s/it]
 61%|██████    | 136161/225000 [272:39:26<143:16:25,  5.81s/it]
 61%|██████    | 136162/225000 [272:39:30<134:51:13,  5.46s/it]
 61%|██████    | 136163/225000 [272:39:35<127:28:37,  5.17s/it]
 61%|██████    | 136164/225000 [272:39:40<124:58:54,  5.06s/it]
 61%|██████    | 136165/225000 [272:39:45<124:16:27,  5.04s/it]
 61%|██████    | 136166/225000 [272:39:57<177:48:31,  7.21s/it]
 61%|██████    | 136167/225000 [272:40:02<163:59:02,  6.65s/it]
 61%|██████    | 136168/225000 [272:40:09<165:56:03,  6.72s/it]
 61%|██████    | 136169/225000 [272:40:13<147:46:30,  5.99s/it]
 61%|██████    | 136170/225000 [272:40:19<142:37:04,  5.78s/it]
 61%|██████    | 136171/225000 [272:40:25<145:48:23,  5.91s/it]
 61%|██████    | 136172/225000 [272:40:32<152:09:01,  6.17s/it]
 61%|██████    | 136173/225000 [272:40:38<150:20:23,  6.09s/it]
 61%|██████    | 136174/225000 [272:40:43<147:47:20,  5.99s/it]
 61%|██████    | 136175/225000 [272:40:47<132:08:14,  5.36s/it]
 61%|██████    | 136176/225000 [272:40:51<122:59:58,  4.99s/it]
 61%|██████    | 136177/225000 [272:40:57<130:30:41,  5.29s/it]
 61%|██████    | 136178/225000 [272:41:07<160:13:54,  6.49s/it]
 61%|██████    | 136179/225000 [272:41:10<137:23:02,  5.57s/it]
 61%|██████    | 136180/225000 [272:41:14<128:42:47,  5.22s/it]
 61%|██████    | 136181/225000 [272:41:19<123:02:16,  4.99s/it]
 61%|██████    | 136182/225000 [272:41:23<118:34:17,  4.81s/it]
 61%|██████    | 136183/225000 [272:41:30<133:33:45,  5.41s/it]
 61%|██████    | 136184/225000 [272:41:40<168:09:28,  6.82s/it]
 61%|██████    | 136185/225000 [272:41:44<149:48:23,  6.07s/it]
 61%|██████    | 136186/225000 [272:41:49<136:56:43,  5.55s/it]
 61%|██████    | 136187/225000 [272:41:53<127:19:58,  5.16s/it]
 61%|██████    | 136188/225000 [272:41:58<123:40:37,  5.01s/it]
 61%|██████    | 136189/225000 [272:42:04<133:32:46,  5.41s/it]
 61%|██████    | 136190/225000 [272:42:16<180:09:04,  7.30s/it]
 61%|██████    | 136191/225000 [272:42:21<166:33:56,  6.75s/it]
 61%|██████    | 136192/225000 [272:42:26<150:18:07,  6.09s/it]
 61%|██████    | 136193/225000 [272:42:34<163:07:34,  6.61s/it]
 61%|██████    | 136194/225000 [272:42:37<140:19:51,  5.69s/it]
 61%|██████    | 136195/225000 [272:42:47<171:35:57,  6.96s/it]
 61%|██████    | 136196/225000 [272:42:51<147:55:10,  6.00s/it]
 61%|██████    | 136197/225000 [272:42:59<166:55:23,  6.77s/it]
 61%|██████    | 136198/225000 [272:43:06<165:49:21,  6.72s/it]
 61%|██████    | 136199/225000 [272:43:10<143:19:23,  5.81s/it]
 61%|██████    | 136200/225000 [272:43:14<134:42:23,  5.46s/it]
                                                               
{'loss': '0.8107', 'grad_norm': '1.141', 'learning_rate': '1.828e-05', 'epoch': '0.6053', 'num_input_tokens_seen': 452637168, 'train_runtime': '9.818e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136200/225000 [272:43:14<134:42:23,  5.46s/it]
 61%|██████    | 136201/225000 [272:43:22<147:53:37,  6.00s/it]
 61%|██████    | 136202/225000 [272:43:26<132:46:20,  5.38s/it]
 61%|██████    | 136203/225000 [272:43:29<121:42:08,  4.93s/it]
 61%|██████    | 136204/225000 [272:43:36<132:16:24,  5.36s/it]
 61%|██████    | 136205/225000 [272:43:46<165:31:07,  6.71s/it]
 61%|██████    | 136206/225000 [272:43:49<139:50:44,  5.67s/it]
 61%|██████    | 136207/225000 [272:43:56<152:55:30,  6.20s/it]
 61%|██████    | 136208/225000 [272:44:01<138:53:08,  5.63s/it]
 61%|██████    | 136209/225000 [272:44:09<159:26:34,  6.46s/it]
 61%|██████    | 136210/225000 [272:44:13<141:18:46,  5.73s/it]
 61%|██████    | 136211/225000 [272:44:17<129:55:17,  5.27s/it]
 61%|██████    | 136212/225000 [272:44:27<160:06:21,  6.49s/it]
 61%|██████    | 136213/225000 [272:44:33<156:37:58,  6.35s/it]
 61%|██████    | 136214/225000 [272:44:37<141:55:33,  5.75s/it]
 61%|██████    | 136215/225000 [272:44:41<130:21:41,  5.29s/it]
 61%|██████    | 136216/225000 [272:44:45<122:59:41,  4.99s/it]
 61%|██████    | 136217/225000 [272:44:50<119:48:41,  4.86s/it]
 61%|██████    | 136218/225000 [272:45:02<174:21:43,  7.07s/it]
 61%|██████    | 136219/225000 [272:45:06<147:00:04,  5.96s/it]
 61%|██████    | 136220/225000 [272:45:14<162:13:32,  6.58s/it]
 61%|██████    | 136221/225000 [272:45:18<146:40:54,  5.95s/it]
 61%|██████    | 136222/225000 [272:45:23<138:36:21,  5.62s/it]
 61%|██████    | 136223/225000 [272:45:28<132:25:21,  5.37s/it]
 61%|██████    | 136224/225000 [272:45:32<127:23:46,  5.17s/it]
 61%|██████    | 136225/225000 [272:45:37<125:22:32,  5.08s/it]
 61%|██████    | 136226/225000 [272:45:42<123:28:53,  5.01s/it]
 61%|██████    | 136227/225000 [272:45:54<172:38:08,  7.00s/it]
 61%|██████    | 136228/225000 [272:46:01<176:43:16,  7.17s/it]
 61%|██████    | 136229/225000 [272:46:05<148:18:13,  6.01s/it]
 61%|██████    | 136230/225000 [272:46:14<175:39:13,  7.12s/it]
 61%|██████    | 136231/225000 [272:46:19<154:47:11,  6.28s/it]
 61%|██████    | 136232/225000 [272:46:23<137:15:10,  5.57s/it]
 61%|██████    | 136233/225000 [272:46:27<127:52:00,  5.19s/it]
 61%|██████    | 136234/225000 [272:46:32<130:32:01,  5.29s/it]
 61%|██████    | 136235/225000 [272:46:45<181:56:51,  7.38s/it]
 61%|██████    | 136236/225000 [272:46:48<152:20:19,  6.18s/it]
 61%|██████    | 136237/225000 [272:46:53<140:53:37,  5.71s/it]
 61%|██████    | 136238/225000 [272:46:59<144:05:16,  5.84s/it]
 61%|██████    | 136239/225000 [272:47:06<154:05:51,  6.25s/it]
 61%|██████    | 136240/225000 [272:47:10<136:50:11,  5.55s/it]
 61%|██████    | 136241/225000 [272:47:17<151:10:02,  6.13s/it]
 61%|██████    | 136242/225000 [272:47:23<145:51:06,  5.92s/it]
 61%|██████    | 136243/225000 [272:47:27<130:23:04,  5.29s/it]
 61%|██████    | 136244/225000 [272:47:38<176:01:52,  7.14s/it]
 61%|██████    | 136245/225000 [272:47:43<157:01:46,  6.37s/it]
 61%|██████    | 136246/225000 [272:47:47<143:03:38,  5.80s/it]
 61%|██████    | 136247/225000 [272:47:51<131:08:52,  5.32s/it]
 61%|██████    | 136248/225000 [272:47:56<124:10:18,  5.04s/it]
 61%|██████    | 136249/225000 [272:48:01<122:08:13,  4.95s/it]
 61%|██████    | 136250/225000 [272:48:05<122:11:44,  4.96s/it]
                                                               
{'loss': '0.8061', 'grad_norm': '1.152', 'learning_rate': '1.826e-05', 'epoch': '0.6056', 'num_input_tokens_seen': 452765976, 'train_runtime': '9.821e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136250/225000 [272:48:06<122:11:44,  4.96s/it]
 61%|██████    | 136251/225000 [272:48:13<137:34:09,  5.58s/it]
 61%|██████    | 136252/225000 [272:48:24<181:23:18,  7.36s/it]
 61%|██████    | 136253/225000 [272:48:28<156:20:06,  6.34s/it]
 61%|██████    | 136254/225000 [272:48:32<136:59:45,  5.56s/it]
 61%|██████    | 136255/225000 [272:48:38<144:20:32,  5.86s/it]
 61%|██████    | 136256/225000 [272:48:48<172:04:06,  6.98s/it]
 61%|██████    | 136257/225000 [272:48:52<150:26:31,  6.10s/it]
 61%|██████    | 136258/225000 [272:48:56<135:48:28,  5.51s/it]
 61%|██████    | 136259/225000 [272:49:03<149:49:03,  6.08s/it]
 61%|██████    | 136260/225000 [272:49:10<154:23:20,  6.26s/it]
 61%|██████    | 136261/225000 [272:49:13<131:20:29,  5.33s/it]
 61%|██████    | 136262/225000 [272:49:23<167:01:38,  6.78s/it]
 61%|██████    | 136263/225000 [272:49:27<145:09:18,  5.89s/it]
 61%|██████    | 136264/225000 [272:49:31<130:08:51,  5.28s/it]
 61%|██████    | 136265/225000 [272:49:38<140:57:47,  5.72s/it]
 61%|██████    | 136266/225000 [272:49:44<146:13:04,  5.93s/it]
 61%|██████    | 136267/225000 [272:49:50<142:26:23,  5.78s/it]
 61%|██████    | 136268/225000 [272:49:58<162:05:44,  6.58s/it]
 61%|██████    | 136269/225000 [272:50:02<138:57:18,  5.64s/it]
 61%|██████    | 136270/225000 [272:50:10<158:13:44,  6.42s/it]
 61%|██████    | 136271/225000 [272:50:16<152:21:21,  6.18s/it]
 61%|██████    | 136272/225000 [272:50:20<137:25:13,  5.58s/it]
 61%|██████    | 136273/225000 [272:50:24<131:50:07,  5.35s/it]
 61%|██████    | 136274/225000 [272:50:29<125:26:56,  5.09s/it]
 61%|██████    | 136275/225000 [272:50:34<122:25:01,  4.97s/it]
 61%|██████    | 136276/225000 [272:50:39<127:36:53,  5.18s/it]
 61%|██████    | 136277/225000 [272:50:52<180:32:40,  7.33s/it]
 61%|██████    | 136278/225000 [272:50:57<162:59:39,  6.61s/it]
 61%|██████    | 136279/225000 [272:51:02<151:40:38,  6.15s/it]
 61%|██████    | 136280/225000 [272:51:06<140:44:14,  5.71s/it]
 61%|██████    | 136281/225000 [272:51:11<135:47:42,  5.51s/it]
 61%|██████    | 136282/225000 [272:51:16<127:28:14,  5.17s/it]
 61%|██████    | 136283/225000 [272:51:21<125:11:02,  5.08s/it]
 61%|██████    | 136284/225000 [272:51:25<121:18:50,  4.92s/it]
 61%|██████    | 136285/225000 [272:51:33<140:18:13,  5.69s/it]
 61%|██████    | 136286/225000 [272:51:44<181:27:15,  7.36s/it]
 61%|██████    | 136287/225000 [272:51:48<154:10:36,  6.26s/it]
 61%|██████    | 136288/225000 [272:51:52<140:08:10,  5.69s/it]
 61%|██████    | 136289/225000 [272:51:58<140:16:47,  5.69s/it]
 61%|██████    | 136290/225000 [272:52:03<134:53:03,  5.47s/it]
 61%|██████    | 136291/225000 [272:52:12<160:11:24,  6.50s/it]
 61%|██████    | 136292/225000 [272:52:16<145:49:33,  5.92s/it]
 61%|██████    | 136293/225000 [272:52:21<138:17:29,  5.61s/it]
 61%|██████    | 136294/225000 [272:52:26<132:34:56,  5.38s/it]
 61%|██████    | 136295/225000 [272:52:30<125:43:06,  5.10s/it]
 61%|██████    | 136296/225000 [272:52:35<123:52:28,  5.03s/it]
 61%|██████    | 136297/225000 [272:52:40<123:23:45,  5.01s/it]
 61%|██████    | 136298/225000 [272:52:47<135:17:25,  5.49s/it]
 61%|██████    | 136299/225000 [272:52:58<180:38:59,  7.33s/it]
 61%|██████    | 136300/225000 [272:53:02<156:25:21,  6.35s/it]
                                                               
{'loss': '0.8121', 'grad_norm': '1.059', 'learning_rate': '1.824e-05', 'epoch': '0.6058', 'num_input_tokens_seen': 452895040, 'train_runtime': '9.824e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136300/225000 [272:53:02<156:25:21,  6.35s/it]
 61%|██████    | 136301/225000 [272:53:11<174:13:47,  7.07s/it]
 61%|██████    | 136302/225000 [272:53:15<150:31:54,  6.11s/it]
 61%|██████    | 136303/225000 [272:53:19<132:53:32,  5.39s/it]
 61%|██████    | 136304/225000 [272:53:30<175:34:04,  7.13s/it]
 61%|██████    | 136305/225000 [272:53:34<150:23:56,  6.10s/it]
 61%|██████    | 136306/225000 [272:53:38<134:11:58,  5.45s/it]
 61%|██████    | 136307/225000 [272:53:44<143:28:12,  5.82s/it]
 61%|██████    | 136308/225000 [272:53:53<163:04:55,  6.62s/it]
 61%|██████    | 136309/225000 [272:53:56<141:36:52,  5.75s/it]
 61%|██████    | 136310/225000 [272:54:00<127:51:44,  5.19s/it]
 61%|██████    | 136311/225000 [272:54:11<168:43:47,  6.85s/it]
 61%|██████    | 136312/225000 [272:54:15<144:04:24,  5.85s/it]
 61%|██████    | 136313/225000 [272:54:18<128:46:36,  5.23s/it]
 61%|██████    | 136314/225000 [272:54:22<120:30:41,  4.89s/it]
 61%|██████    | 136315/225000 [272:54:27<117:28:45,  4.77s/it]
 61%|██████    | 136316/225000 [272:54:32<117:31:11,  4.77s/it]
 61%|██████    | 136317/225000 [272:54:37<119:28:52,  4.85s/it]
 61%|██████    | 136318/225000 [272:54:49<174:33:42,  7.09s/it]
 61%|██████    | 136319/225000 [272:54:56<170:40:23,  6.93s/it]
 61%|██████    | 136320/225000 [272:55:00<152:19:45,  6.18s/it]
 61%|██████    | 136321/225000 [272:55:04<138:25:53,  5.62s/it]
 61%|██████    | 136322/225000 [272:55:09<131:23:46,  5.33s/it]
 61%|██████    | 136323/225000 [272:55:16<146:21:16,  5.94s/it]
 61%|██████    | 136324/225000 [272:55:25<162:19:07,  6.59s/it]
 61%|██████    | 136325/225000 [272:55:28<139:23:48,  5.66s/it]
 61%|██████    | 136326/225000 [272:55:36<156:10:14,  6.34s/it]
 61%|██████    | 136327/225000 [272:55:41<148:26:22,  6.03s/it]
 61%|██████    | 136328/225000 [272:55:45<129:24:50,  5.25s/it]
 61%|██████    | 136329/225000 [272:55:55<166:37:22,  6.76s/it]
 61%|██████    | 136330/225000 [272:55:58<139:30:01,  5.66s/it]
 61%|██████    | 136331/225000 [272:56:05<146:43:01,  5.96s/it]
 61%|██████    | 136332/225000 [272:56:08<129:42:06,  5.27s/it]
 61%|██████    | 136333/225000 [272:56:16<145:59:08,  5.93s/it]
 61%|██████    | 136334/225000 [272:56:23<158:50:00,  6.45s/it]
 61%|██████    | 136335/225000 [272:56:27<136:32:27,  5.54s/it]
 61%|██████    | 136336/225000 [272:56:35<158:16:34,  6.43s/it]
 61%|██████    | 136337/225000 [272:56:39<134:35:46,  5.47s/it]
 61%|██████    | 136338/225000 [272:56:46<146:31:09,  5.95s/it]
 61%|██████    | 136339/225000 [272:56:51<144:09:28,  5.85s/it]
 61%|██████    | 136340/225000 [272:56:56<135:40:09,  5.51s/it]
 61%|██████    | 136341/225000 [272:57:01<129:00:55,  5.24s/it]
 61%|██████    | 136342/225000 [272:57:06<127:47:11,  5.19s/it]
 61%|██████    | 136343/225000 [272:57:10<122:58:54,  4.99s/it]
 61%|██████    | 136344/225000 [272:57:15<124:01:45,  5.04s/it]
 61%|██████    | 136345/225000 [272:57:20<122:08:40,  4.96s/it]
 61%|██████    | 136346/225000 [272:57:25<123:59:10,  5.03s/it]
 61%|██████    | 136347/225000 [272:57:32<135:52:16,  5.52s/it]
 61%|██████    | 136348/225000 [272:57:44<186:07:48,  7.56s/it]
 61%|██████    | 136349/225000 [272:57:50<174:35:58,  7.09s/it]
 61%|██████    | 136350/225000 [272:57:54<150:44:53,  6.12s/it]
                                                               
{'loss': '0.8106', 'grad_norm': '0.9673', 'learning_rate': '1.822e-05', 'epoch': '0.606', 'num_input_tokens_seen': 453024032, 'train_runtime': '9.827e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136350/225000 [272:57:54<150:44:53,  6.12s/it]
 61%|██████    | 136351/225000 [272:58:02<166:33:32,  6.76s/it]
 61%|██████    | 136352/225000 [272:58:08<158:53:55,  6.45s/it]
 61%|██████    | 136353/225000 [272:58:12<138:48:34,  5.64s/it]
 61%|██████    | 136354/225000 [272:58:16<127:19:02,  5.17s/it]
 61%|██████    | 136355/225000 [272:58:27<171:45:56,  6.98s/it]
 61%|██████    | 136356/225000 [272:58:31<147:31:23,  5.99s/it]
 61%|██████    | 136357/225000 [272:58:35<133:23:42,  5.42s/it]
 61%|██████    | 136358/225000 [272:58:44<160:15:09,  6.51s/it]
 61%|██████    | 136359/225000 [272:58:48<138:27:36,  5.62s/it]
 61%|██████    | 136360/225000 [272:58:52<126:40:19,  5.14s/it]
 61%|██████    | 136361/225000 [272:58:59<143:18:19,  5.82s/it]
 61%|██████    | 136362/225000 [272:59:09<173:13:06,  7.04s/it]
 61%|██████    | 136363/225000 [272:59:12<142:53:07,  5.80s/it]
 61%|██████    | 136364/225000 [272:59:20<158:52:16,  6.45s/it]
 61%|██████    | 136365/225000 [272:59:23<134:52:25,  5.48s/it]
 61%|██████    | 136366/225000 [272:59:32<158:15:44,  6.43s/it]
 61%|██████    | 136367/225000 [272:59:37<150:29:11,  6.11s/it]
 61%|██████    | 136368/225000 [272:59:44<155:01:23,  6.30s/it]
 61%|██████    | 136369/225000 [272:59:48<141:00:35,  5.73s/it]
 61%|██████    | 136370/225000 [272:59:52<129:31:46,  5.26s/it]
 61%|██████    | 136371/225000 [273:00:00<148:16:39,  6.02s/it]
 61%|██████    | 136372/225000 [273:00:05<140:55:27,  5.72s/it]
 61%|██████    | 136373/225000 [273:00:09<127:15:21,  5.17s/it]
 61%|██████    | 136374/225000 [273:00:13<121:07:41,  4.92s/it]
 61%|██████    | 136375/225000 [273:00:26<174:52:51,  7.10s/it]
 61%|██████    | 136376/225000 [273:00:29<146:24:11,  5.95s/it]
 61%|██████    | 136377/225000 [273:00:37<160:34:41,  6.52s/it]
 61%|██████    | 136378/225000 [273:00:40<140:27:24,  5.71s/it]
 61%|██████    | 136379/225000 [273:00:49<159:04:13,  6.46s/it]
 61%|██████    | 136380/225000 [273:00:53<142:47:09,  5.80s/it]
 61%|██████    | 136381/225000 [273:00:57<132:16:44,  5.37s/it]
 61%|██████    | 136382/225000 [273:01:01<121:42:31,  4.94s/it]
 61%|██████    | 136383/225000 [273:01:10<150:44:27,  6.12s/it]
 61%|██████    | 136384/225000 [273:01:18<160:09:57,  6.51s/it]
 61%|██████    | 136385/225000 [273:01:22<141:31:15,  5.75s/it]
 61%|██████    | 136386/225000 [273:01:26<131:01:02,  5.32s/it]
 61%|██████    | 136387/225000 [273:01:30<123:48:39,  5.03s/it]
 61%|██████    | 136388/225000 [273:01:39<149:59:28,  6.09s/it]
 61%|██████    | 136389/225000 [273:01:46<161:57:25,  6.58s/it]
 61%|██████    | 136390/225000 [273:01:51<143:21:19,  5.82s/it]
 61%|██████    | 136391/225000 [273:01:55<130:36:02,  5.31s/it]
 61%|██████    | 136392/225000 [273:01:58<118:29:24,  4.81s/it]
 61%|██████    | 136393/225000 [273:02:05<132:24:48,  5.38s/it]
 61%|██████    | 136394/225000 [273:02:15<165:37:11,  6.73s/it]
 61%|██████    | 136395/225000 [273:02:19<144:34:37,  5.87s/it]
 61%|██████    | 136396/225000 [273:02:24<141:24:43,  5.75s/it]
 61%|██████    | 136397/225000 [273:02:30<139:53:24,  5.68s/it]
 61%|██████    | 136398/225000 [273:02:35<138:09:07,  5.61s/it]
 61%|██████    | 136399/225000 [273:02:41<136:58:53,  5.57s/it]
 61%|██████    | 136400/225000 [273:02:45<129:20:07,  5.26s/it]
                                                               
{'loss': '0.8055', 'grad_norm': '1.096', 'learning_rate': '1.821e-05', 'epoch': '0.6062', 'num_input_tokens_seen': 453153144, 'train_runtime': '9.83e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136400/225000 [273:02:45<129:20:07,  5.26s/it]
 61%|██████    | 136401/225000 [273:02:51<130:39:10,  5.31s/it]
 61%|██████    | 136402/225000 [273:02:56<128:00:36,  5.20s/it]
 61%|██████    | 136403/225000 [273:03:01<130:39:40,  5.31s/it]
 61%|██████    | 136404/225000 [273:03:07<131:39:43,  5.35s/it]
 61%|██████    | 136405/225000 [273:03:18<178:09:42,  7.24s/it]
 61%|██████    | 136406/225000 [273:03:26<181:54:57,  7.39s/it]
 61%|██████    | 136407/225000 [273:03:29<152:59:25,  6.22s/it]
 61%|██████    | 136408/225000 [273:03:34<141:25:10,  5.75s/it]
 61%|██████    | 136409/225000 [273:03:39<138:40:23,  5.64s/it]
 61%|██████    | 136410/225000 [273:03:45<135:43:23,  5.52s/it]
 61%|██████    | 136411/225000 [273:03:50<132:29:36,  5.38s/it]
 61%|██████    | 136412/225000 [273:03:55<132:55:38,  5.40s/it]
 61%|██████    | 136413/225000 [273:04:01<134:15:14,  5.46s/it]
 61%|██████    | 136414/225000 [273:04:06<130:46:21,  5.31s/it]
 61%|██████    | 136415/225000 [273:04:11<132:12:05,  5.37s/it]
 61%|██████    | 136416/225000 [273:04:22<172:55:39,  7.03s/it]
 61%|██████    | 136417/225000 [273:04:30<177:55:59,  7.23s/it]
 61%|██████    | 136418/225000 [273:04:34<156:53:00,  6.38s/it]
 61%|██████    | 136419/225000 [273:04:38<139:36:11,  5.67s/it]
 61%|██████    | 136420/225000 [273:04:49<176:27:14,  7.17s/it]
 61%|██████    | 136421/225000 [273:04:56<172:43:02,  7.02s/it]
 61%|██████    | 136422/225000 [273:04:59<146:20:07,  5.95s/it]
 61%|██████    | 136423/225000 [273:05:04<136:21:24,  5.54s/it]
 61%|██████    | 136424/225000 [273:05:15<178:38:42,  7.26s/it]
 61%|██████    | 136425/225000 [273:05:20<161:24:53,  6.56s/it]
 61%|██████    | 136426/225000 [273:05:25<150:19:10,  6.11s/it]
 61%|██████    | 136427/225000 [273:05:30<141:25:16,  5.75s/it]
 61%|██████    | 136428/225000 [273:05:35<138:21:54,  5.62s/it]
 61%|██████    | 136429/225000 [273:05:40<132:50:33,  5.40s/it]
 61%|██████    | 136430/225000 [273:05:45<129:03:05,  5.25s/it]
 61%|██████    | 136431/225000 [273:05:56<169:38:30,  6.90s/it]
 61%|██████    | 136432/225000 [273:06:03<171:07:32,  6.96s/it]
 61%|██████    | 136433/225000 [273:06:07<147:46:37,  6.01s/it]
 61%|██████    | 136434/225000 [273:06:11<134:05:45,  5.45s/it]
 61%|██████    | 136435/225000 [273:06:16<129:19:20,  5.26s/it]
 61%|██████    | 136436/225000 [273:06:28<179:02:31,  7.28s/it]
 61%|██████    | 136437/225000 [273:06:30<146:19:15,  5.95s/it]
 61%|██████    | 136438/225000 [273:06:38<158:41:17,  6.45s/it]
 61%|██████    | 136439/225000 [273:06:44<157:03:56,  6.38s/it]
 61%|██████    | 136440/225000 [273:06:49<143:55:17,  5.85s/it]
 61%|██████    | 136441/225000 [273:06:53<132:54:22,  5.40s/it]
 61%|██████    | 136442/225000 [273:06:58<126:10:34,  5.13s/it]
 61%|██████    | 136443/225000 [273:07:02<122:03:31,  4.96s/it]
 61%|██████    | 136444/225000 [273:07:15<179:17:00,  7.29s/it]
 61%|██████    | 136445/225000 [273:07:20<162:51:35,  6.62s/it]
 61%|██████    | 136446/225000 [273:07:25<151:47:10,  6.17s/it]
 61%|██████    | 136447/225000 [273:07:30<142:04:41,  5.78s/it]
 61%|██████    | 136448/225000 [273:07:35<134:37:10,  5.47s/it]
 61%|██████    | 136449/225000 [273:07:43<157:19:08,  6.40s/it]
 61%|██████    | 136450/225000 [273:07:53<180:40:03,  7.35s/it]
                                                               
{'loss': '0.8026', 'grad_norm': '1.113', 'learning_rate': '1.819e-05', 'epoch': '0.6064', 'num_input_tokens_seen': 453284144, 'train_runtime': '9.833e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136450/225000 [273:07:53<180:40:03,  7.35s/it]
 61%|██████    | 136451/225000 [273:07:56<152:47:01,  6.21s/it]
 61%|██████    | 136452/225000 [273:08:03<158:33:59,  6.45s/it]
 61%|██████    | 136453/225000 [273:08:10<161:57:29,  6.58s/it]
 61%|██████    | 136454/225000 [273:08:14<140:25:35,  5.71s/it]
 61%|██████    | 136455/225000 [273:08:24<173:22:06,  7.05s/it]
 61%|██████    | 136456/225000 [273:08:28<146:50:52,  5.97s/it]
 61%|██████    | 136457/225000 [273:08:36<165:24:36,  6.73s/it]
 61%|██████    | 136458/225000 [273:08:39<140:21:20,  5.71s/it]
 61%|██████    | 136459/225000 [273:08:47<156:08:03,  6.35s/it]
 61%|██████    | 136460/225000 [273:08:51<135:33:20,  5.51s/it]
 61%|██████    | 136461/225000 [273:08:55<123:32:11,  5.02s/it]
 61%|██████    | 136462/225000 [273:09:05<162:18:13,  6.60s/it]
 61%|██████    | 136463/225000 [273:09:09<142:31:45,  5.80s/it]
 61%|██████    | 136464/225000 [273:09:12<125:09:35,  5.09s/it]
 61%|██████    | 136465/225000 [273:09:23<165:03:57,  6.71s/it]
 61%|██████    | 136466/225000 [273:09:28<156:57:09,  6.38s/it]
 61%|██████    | 136467/225000 [273:09:34<146:59:07,  5.98s/it]
 61%|██████    | 136468/225000 [273:09:39<141:06:07,  5.74s/it]
 61%|██████    | 136469/225000 [273:09:44<138:01:46,  5.61s/it]
 61%|██████    | 136470/225000 [273:09:50<138:34:05,  5.63s/it]
 61%|██████    | 136471/225000 [273:09:55<138:33:40,  5.63s/it]
 61%|██████    | 136472/225000 [273:10:01<137:12:26,  5.58s/it]
 61%|██████    | 136473/225000 [273:10:06<133:29:31,  5.43s/it]
 61%|██████    | 136474/225000 [273:10:13<148:23:09,  6.03s/it]
 61%|██████    | 136475/225000 [273:10:22<171:06:11,  6.96s/it]
 61%|██████    | 136476/225000 [273:10:26<146:11:40,  5.95s/it]
 61%|██████    | 136477/225000 [273:10:30<134:10:40,  5.46s/it]
 61%|██████    | 136478/225000 [273:10:35<126:08:00,  5.13s/it]
 61%|██████    | 136479/225000 [273:10:39<121:45:30,  4.95s/it]
 61%|██████    | 136480/225000 [273:10:48<147:12:09,  5.99s/it]
 61%|██████    | 136481/225000 [273:10:58<177:58:58,  7.24s/it]
 61%|██████    | 136482/225000 [273:11:01<151:11:43,  6.15s/it]
 61%|██████    | 136483/225000 [273:11:10<168:38:31,  6.86s/it]
 61%|██████    | 136484/225000 [273:11:15<153:04:46,  6.23s/it]
 61%|██████    | 136485/225000 [273:11:19<135:26:19,  5.51s/it]
 61%|██████    | 136486/225000 [273:11:24<133:25:31,  5.43s/it]
 61%|██████    | 136487/225000 [273:11:30<137:24:27,  5.59s/it]
 61%|██████    | 136488/225000 [273:11:35<132:12:33,  5.38s/it]
 61%|██████    | 136489/225000 [273:11:40<134:49:04,  5.48s/it]
 61%|██████    | 136490/225000 [273:11:45<131:56:06,  5.37s/it]
 61%|██████    | 136491/225000 [273:11:51<130:07:08,  5.29s/it]
 61%|██████    | 136492/225000 [273:11:57<140:06:44,  5.70s/it]
 61%|██████    | 136493/225000 [273:12:10<190:06:06,  7.73s/it]
 61%|██████    | 136494/225000 [273:12:13<158:20:55,  6.44s/it]
 61%|██████    | 136495/225000 [273:12:21<169:51:35,  6.91s/it]
 61%|██████    | 136496/225000 [273:12:25<147:33:41,  6.00s/it]
 61%|██████    | 136497/225000 [273:12:29<131:57:49,  5.37s/it]
 61%|██████    | 136498/225000 [273:12:41<180:58:13,  7.36s/it]
 61%|██████    | 136499/225000 [273:12:45<153:24:16,  6.24s/it]
 61%|██████    | 136500/225000 [273:12:54<174:29:33,  7.10s/it]
                                                               
{'loss': '0.7994', 'grad_norm': '1.061', 'learning_rate': '1.817e-05', 'epoch': '0.6067', 'num_input_tokens_seen': 453414752, 'train_runtime': '9.836e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136500/225000 [273:12:54<174:29:33,  7.10s/it]
 61%|██████    | 136501/225000 [273:12:57<148:09:15,  6.03s/it]
 61%|██████    | 136502/225000 [273:13:01<134:25:03,  5.47s/it]
 61%|██████    | 136503/225000 [273:13:09<148:39:02,  6.05s/it]
 61%|██████    | 136504/225000 [273:13:17<166:28:01,  6.77s/it]
 61%|██████    | 136505/225000 [273:13:21<142:30:09,  5.80s/it]
 61%|██████    | 136506/225000 [273:13:30<170:27:06,  6.93s/it]
 61%|██████    | 136507/225000 [273:13:34<144:04:32,  5.86s/it]
 61%|██████    | 136508/225000 [273:13:39<143:00:06,  5.82s/it]
 61%|██████    | 136509/225000 [273:13:46<147:31:22,  6.00s/it]
 61%|██████    | 136510/225000 [273:13:54<161:37:47,  6.58s/it]
 61%|██████    | 136511/225000 [273:13:58<148:24:51,  6.04s/it]
 61%|██████    | 136512/225000 [273:14:03<135:49:53,  5.53s/it]
 61%|██████    | 136513/225000 [273:14:07<128:11:53,  5.22s/it]
 61%|██████    | 136514/225000 [273:14:12<125:21:48,  5.10s/it]
 61%|██████    | 136515/225000 [273:14:17<122:16:11,  4.97s/it]
 61%|██████    | 136516/225000 [273:14:22<120:25:05,  4.90s/it]
 61%|██████    | 136517/225000 [273:14:34<174:39:59,  7.11s/it]
 61%|██████    | 136518/225000 [273:14:40<170:30:59,  6.94s/it]
 61%|██████    | 136519/225000 [273:14:44<150:04:12,  6.11s/it]
 61%|██████    | 136520/225000 [273:14:48<134:32:06,  5.47s/it]
 61%|██████    | 136521/225000 [273:14:52<123:41:03,  5.03s/it]
 61%|██████    | 136522/225000 [273:14:58<123:46:49,  5.04s/it]
 61%|██████    | 136523/225000 [273:15:03<127:08:19,  5.17s/it]
 61%|██████    | 136524/225000 [273:15:08<129:04:28,  5.25s/it]
 61%|██████    | 136525/225000 [273:15:14<129:12:44,  5.26s/it]
 61%|██████    | 136526/225000 [273:15:19<129:10:30,  5.26s/it]
 61%|██████    | 136527/225000 [273:15:26<144:18:29,  5.87s/it]
 61%|██████    | 136528/225000 [273:15:38<186:07:54,  7.57s/it]
 61%|██████    | 136529/225000 [273:15:42<157:47:48,  6.42s/it]
 61%|██████    | 136530/225000 [273:15:50<173:26:08,  7.06s/it]
 61%|██████    | 136531/225000 [273:15:54<148:55:06,  6.06s/it]
 61%|██████    | 136532/225000 [273:16:03<168:38:48,  6.86s/it]
 61%|██████    | 136533/225000 [273:16:07<153:50:25,  6.26s/it]
 61%|██████    | 136534/225000 [273:16:11<134:47:49,  5.49s/it]
 61%|██████    | 136535/225000 [273:16:22<171:19:02,  6.97s/it]
 61%|██████    | 136536/225000 [273:16:25<146:09:03,  5.95s/it]
 61%|██████    | 136537/225000 [273:16:33<161:58:49,  6.59s/it]
 61%|██████    | 136538/225000 [273:16:38<147:21:14,  6.00s/it]
 61%|██████    | 136539/225000 [273:16:45<158:30:21,  6.45s/it]
 61%|██████    | 136540/225000 [273:16:49<136:12:17,  5.54s/it]
 61%|██████    | 136541/225000 [273:16:57<159:35:45,  6.50s/it]
 61%|██████    | 136542/225000 [273:17:01<137:49:01,  5.61s/it]
 61%|██████    | 136543/225000 [273:17:08<148:49:55,  6.06s/it]
 61%|██████    | 136544/225000 [273:17:15<151:24:07,  6.16s/it]
 61%|██████    | 136545/225000 [273:17:19<140:37:52,  5.72s/it]
 61%|██████    | 136546/225000 [273:17:24<131:23:01,  5.35s/it]
 61%|██████    | 136547/225000 [273:17:29<128:27:02,  5.23s/it]
 61%|██████    | 136548/225000 [273:17:34<127:52:09,  5.20s/it]
 61%|██████    | 136549/225000 [273:17:39<128:09:24,  5.22s/it]
 61%|██████    | 136550/225000 [273:17:44<126:12:48,  5.14s/it]
                                                               
{'loss': '0.8046', 'grad_norm': '1.069', 'learning_rate': '1.815e-05', 'epoch': '0.6069', 'num_input_tokens_seen': 453544080, 'train_runtime': '9.839e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136550/225000 [273:17:44<126:12:48,  5.14s/it]
 61%|██████    | 136551/225000 [273:17:49<123:52:39,  5.04s/it]
 61%|██████    | 136552/225000 [273:18:01<176:48:24,  7.20s/it]
 61%|██████    | 136553/225000 [273:18:07<167:18:24,  6.81s/it]
 61%|██████    | 136554/225000 [273:18:11<146:20:04,  5.96s/it]
 61%|██████    | 136555/225000 [273:18:15<133:18:56,  5.43s/it]
 61%|██████    | 136556/225000 [273:18:24<157:41:43,  6.42s/it]
 61%|██████    | 136557/225000 [273:18:29<151:17:41,  6.16s/it]
 61%|██████    | 136558/225000 [273:18:33<132:09:36,  5.38s/it]
 61%|██████    | 136559/225000 [273:18:41<149:19:06,  6.08s/it]
 61%|██████    | 136560/225000 [273:18:45<139:39:14,  5.68s/it]
 61%|██████    | 136561/225000 [273:18:50<128:19:05,  5.22s/it]
 61%|██████    | 136562/225000 [273:18:54<124:30:56,  5.07s/it]
 61%|██████    | 136563/225000 [273:19:06<175:36:29,  7.15s/it]
 61%|██████    | 136564/225000 [273:19:12<166:33:02,  6.78s/it]
 61%|██████    | 136565/225000 [273:19:19<167:47:41,  6.83s/it]
 61%|██████    | 136566/225000 [273:19:24<152:15:33,  6.20s/it]
 61%|██████    | 136567/225000 [273:19:28<135:12:42,  5.50s/it]
 61%|██████    | 136568/225000 [273:19:32<127:36:19,  5.19s/it]
 61%|██████    | 136569/225000 [273:19:43<168:27:51,  6.86s/it]
 61%|██████    | 136570/225000 [273:19:48<153:48:28,  6.26s/it]
 61%|██████    | 136571/225000 [273:19:53<143:26:14,  5.84s/it]
 61%|██████    | 136572/225000 [273:19:57<135:13:40,  5.51s/it]
 61%|██████    | 136573/225000 [273:20:03<132:48:04,  5.41s/it]
 61%|██████    | 136574/225000 [273:20:07<127:39:32,  5.20s/it]
 61%|██████    | 136575/225000 [273:20:12<126:44:50,  5.16s/it]
 61%|██████    | 136576/225000 [273:20:19<137:54:17,  5.61s/it]
 61%|██████    | 136577/225000 [273:20:31<184:13:52,  7.50s/it]
 61%|██████    | 136578/225000 [273:20:37<173:16:46,  7.05s/it]
 61%|██████    | 136579/225000 [273:20:40<145:11:22,  5.91s/it]
 61%|██████    | 136580/225000 [273:20:49<168:58:44,  6.88s/it]
 61%|██████    | 136581/225000 [273:20:54<149:56:33,  6.10s/it]
 61%|██████    | 136582/225000 [273:20:58<134:04:18,  5.46s/it]
 61%|██████    | 136583/225000 [273:21:03<132:09:04,  5.38s/it]
 61%|██████    | 136584/225000 [273:21:13<168:37:42,  6.87s/it]
 61%|██████    | 136585/225000 [273:21:16<140:53:43,  5.74s/it]
 61%|██████    | 136586/225000 [273:21:25<160:31:29,  6.54s/it]
 61%|██████    | 136587/225000 [273:21:30<150:13:18,  6.12s/it]
 61%|██████    | 136588/225000 [273:21:35<144:06:56,  5.87s/it]
 61%|██████    | 136589/225000 [273:21:40<138:41:29,  5.65s/it]
 61%|██████    | 136590/225000 [273:21:46<137:16:28,  5.59s/it]
 61%|██████    | 136591/225000 [273:21:51<134:18:02,  5.47s/it]
 61%|██████    | 136592/225000 [273:21:56<133:39:11,  5.44s/it]
 61%|██████    | 136593/225000 [273:22:01<129:18:46,  5.27s/it]
 61%|██████    | 136594/225000 [273:22:06<128:54:44,  5.25s/it]
 61%|██████    | 136595/225000 [273:22:11<128:33:41,  5.24s/it]
 61%|██████    | 136596/225000 [273:22:20<155:46:25,  6.34s/it]
 61%|██████    | 136597/225000 [273:22:31<190:27:03,  7.76s/it]
 61%|██████    | 136598/225000 [273:22:35<160:54:55,  6.55s/it]
 61%|██████    | 136599/225000 [273:22:39<140:09:07,  5.71s/it]
 61%|██████    | 136600/225000 [273:22:49<172:28:02,  7.02s/it]
                                                               
{'loss': '0.8031', 'grad_norm': '1.103', 'learning_rate': '1.814e-05', 'epoch': '0.6071', 'num_input_tokens_seen': 453674376, 'train_runtime': '9.842e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136600/225000 [273:22:49<172:28:02,  7.02s/it]
 61%|██████    | 136601/225000 [273:22:53<146:54:36,  5.98s/it]
 61%|██████    | 136602/225000 [273:22:56<129:26:14,  5.27s/it]
 61%|██████    | 136603/225000 [273:23:02<131:39:25,  5.36s/it]
 61%|██████    | 136604/225000 [273:23:11<162:43:13,  6.63s/it]
 61%|██████    | 136605/225000 [273:23:15<139:24:06,  5.68s/it]
 61%|██████    | 136606/225000 [273:23:23<157:42:28,  6.42s/it]
 61%|██████    | 136607/225000 [273:23:28<148:45:46,  6.06s/it]
 61%|██████    | 136608/225000 [273:23:35<151:46:04,  6.18s/it]
 61%|██████    | 136609/225000 [273:23:40<142:19:33,  5.80s/it]
 61%|██████    | 136610/225000 [273:23:46<149:49:06,  6.10s/it]
 61%|██████    | 136611/225000 [273:23:51<141:58:18,  5.78s/it]
 61%|██████    | 136612/225000 [273:23:57<141:33:41,  5.77s/it]
 61%|██████    | 136613/225000 [273:24:04<151:51:27,  6.19s/it]
 61%|██████    | 136614/225000 [273:24:12<160:15:45,  6.53s/it]
 61%|██████    | 136615/225000 [273:24:15<140:00:29,  5.70s/it]
 61%|██████    | 136616/225000 [273:24:19<126:15:12,  5.14s/it]
 61%|██████    | 136617/225000 [273:24:30<164:48:12,  6.71s/it]
 61%|██████    | 136618/225000 [273:24:34<150:33:46,  6.13s/it]
 61%|██████    | 136619/225000 [273:24:38<130:41:43,  5.32s/it]
 61%|██████    | 136620/225000 [273:24:43<131:27:04,  5.35s/it]
 61%|██████    | 136621/225000 [273:24:49<136:09:53,  5.55s/it]
 61%|██████    | 136622/225000 [273:25:00<173:06:35,  7.05s/it]
 61%|██████    | 136623/225000 [273:25:04<155:02:36,  6.32s/it]
 61%|██████    | 136624/225000 [273:25:09<139:47:23,  5.69s/it]
 61%|██████    | 136625/225000 [273:25:13<131:41:26,  5.36s/it]
 61%|██████    | 136626/225000 [273:25:21<150:58:07,  6.15s/it]
 61%|██████    | 136627/225000 [273:25:29<166:34:44,  6.79s/it]
 61%|██████    | 136628/225000 [273:25:33<141:50:10,  5.78s/it]
 61%|██████    | 136629/225000 [273:25:40<154:57:19,  6.31s/it]
 61%|██████    | 136630/225000 [273:25:44<136:18:14,  5.55s/it]
 61%|██████    | 136631/225000 [273:25:54<167:06:24,  6.81s/it]
 61%|██████    | 136632/225000 [273:25:57<139:45:48,  5.69s/it]
 61%|██████    | 136633/225000 [273:26:03<144:46:53,  5.90s/it]
 61%|██████    | 136634/225000 [273:26:08<137:12:34,  5.59s/it]
 61%|██████    | 136635/225000 [273:26:13<130:25:35,  5.31s/it]
 61%|██████    | 136636/225000 [273:26:18<128:45:12,  5.25s/it]
 61%|██████    | 136637/225000 [273:26:23<128:08:12,  5.22s/it]
 61%|██████    | 136638/225000 [273:26:29<129:03:46,  5.26s/it]
 61%|██████    | 136639/225000 [273:26:33<124:16:40,  5.06s/it]
 61%|██████    | 136640/225000 [273:26:38<122:30:17,  4.99s/it]
 61%|██████    | 136641/225000 [273:26:50<175:34:43,  7.15s/it]
 61%|██████    | 136642/225000 [273:26:57<170:37:42,  6.95s/it]
 61%|██████    | 136643/225000 [273:27:01<149:13:31,  6.08s/it]
 61%|██████    | 136644/225000 [273:27:06<141:02:07,  5.75s/it]
 61%|██████    | 136645/225000 [273:27:18<187:22:16,  7.63s/it]
 61%|██████    | 136646/225000 [273:27:21<155:42:21,  6.34s/it]
 61%|██████    | 136647/225000 [273:27:29<170:24:33,  6.94s/it]
 61%|██████    | 136648/225000 [273:27:33<145:48:00,  5.94s/it]
 61%|██████    | 136649/225000 [273:27:40<152:59:18,  6.23s/it]
 61%|██████    | 136650/225000 [273:27:46<153:37:46,  6.26s/it]
                                                               
{'loss': '0.7988', 'grad_norm': '1.015', 'learning_rate': '1.812e-05', 'epoch': '0.6073', 'num_input_tokens_seen': 453804824, 'train_runtime': '9.845e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136650/225000 [273:27:46<153:37:46,  6.26s/it]
 61%|██████    | 136651/225000 [273:27:50<138:34:56,  5.65s/it]
 61%|██████    | 136652/225000 [273:27:54<123:15:57,  5.02s/it]
 61%|██████    | 136653/225000 [273:28:01<139:28:38,  5.68s/it]
 61%|██████    | 136654/225000 [273:28:12<172:58:37,  7.05s/it]
 61%|██████    | 136655/225000 [273:28:15<148:26:32,  6.05s/it]
 61%|██████    | 136656/225000 [273:28:20<135:59:14,  5.54s/it]
 61%|██████    | 136657/225000 [273:28:24<126:22:56,  5.15s/it]
 61%|██████    | 136658/225000 [273:28:36<178:17:04,  7.27s/it]
 61%|██████    | 136659/225000 [273:28:39<149:41:37,  6.10s/it]
 61%|██████    | 136660/225000 [273:28:43<133:24:41,  5.44s/it]
 61%|██████    | 136661/225000 [273:28:52<159:30:23,  6.50s/it]
 61%|██████    | 136662/225000 [273:28:56<138:12:55,  5.63s/it]
 61%|██████    | 136663/225000 [273:29:01<137:18:06,  5.60s/it]
 61%|██████    | 136664/225000 [273:29:07<135:46:46,  5.53s/it]
 61%|██████    | 136665/225000 [273:29:12<132:36:41,  5.40s/it]
 61%|██████    | 136666/225000 [273:29:17<132:30:23,  5.40s/it]
 61%|██████    | 136667/225000 [273:29:23<133:31:27,  5.44s/it]
 61%|██████    | 136668/225000 [273:29:29<135:45:10,  5.53s/it]
 61%|██████    | 136669/225000 [273:29:34<132:42:01,  5.41s/it]
 61%|██████    | 136670/225000 [273:29:41<148:07:01,  6.04s/it]
 61%|██████    | 136671/225000 [273:29:51<174:28:41,  7.11s/it]
 61%|██████    | 136672/225000 [273:29:55<153:23:18,  6.25s/it]
 61%|██████    | 136673/225000 [273:29:59<139:33:21,  5.69s/it]
 61%|██████    | 136674/225000 [273:30:07<149:48:11,  6.11s/it]
 61%|██████    | 136675/225000 [273:30:17<180:48:33,  7.37s/it]
 61%|██████    | 136676/225000 [273:30:23<170:14:58,  6.94s/it]
 61%|██████    | 136677/225000 [273:30:26<146:20:15,  5.96s/it]
 61%|██████    | 136678/225000 [273:30:36<173:14:23,  7.06s/it]
 61%|██████    | 136679/225000 [273:30:41<154:55:42,  6.31s/it]
 61%|██████    | 136680/225000 [273:30:46<147:38:40,  6.02s/it]
 61%|██████    | 136681/225000 [273:30:51<139:51:24,  5.70s/it]
 61%|██████    | 136682/225000 [273:30:56<134:07:43,  5.47s/it]
 61%|██████    | 136683/225000 [273:31:01<129:31:17,  5.28s/it]
 61%|██████    | 136684/225000 [273:31:05<125:59:19,  5.14s/it]
 61%|██████    | 136685/225000 [273:31:10<124:52:48,  5.09s/it]
 61%|██████    | 136686/225000 [273:31:18<144:01:30,  5.87s/it]
 61%|██████    | 136687/225000 [273:31:29<178:26:12,  7.27s/it]
 61%|██████    | 136688/225000 [273:31:32<151:16:04,  6.17s/it]
 61%|██████    | 136689/225000 [273:31:41<168:01:32,  6.85s/it]
 61%|██████    | 136690/225000 [273:31:46<153:17:44,  6.25s/it]
 61%|██████    | 136691/225000 [273:31:50<141:46:46,  5.78s/it]
 61%|██████    | 136692/225000 [273:31:55<132:28:11,  5.40s/it]
 61%|██████    | 136693/225000 [273:31:59<125:28:28,  5.12s/it]
 61%|██████    | 136694/225000 [273:32:04<121:07:12,  4.94s/it]
 61%|██████    | 136695/225000 [273:32:08<116:10:05,  4.74s/it]
 61%|██████    | 136696/225000 [273:32:17<144:04:05,  5.87s/it]
 61%|██████    | 136697/225000 [273:32:27<178:27:14,  7.28s/it]
 61%|██████    | 136698/225000 [273:32:30<149:23:40,  6.09s/it]
 61%|██████    | 136699/225000 [273:32:38<163:47:24,  6.68s/it]
 61%|██████    | 136700/225000 [273:32:42<142:50:27,  5.82s/it]
                                                               
{'loss': '0.8085', 'grad_norm': '1.163', 'learning_rate': '1.81e-05', 'epoch': '0.6076', 'num_input_tokens_seen': 453935128, 'train_runtime': '9.848e+05', 'train_tokens_per_second': '461'}

 61%|██████    | 136700/225000 [273:32:42<142:50:27,  5.82s/it]
 61%|██████    | 136701/225000 [273:32:48<143:38:29,  5.86s/it]
 61%|██████    | 136702/225000 [273:32:57<165:02:17,  6.73s/it]
 61%|██████    | 136703/225000 [273:33:02<152:29:04,  6.22s/it]
 61%|██████    | 136704/225000 [273:33:10<161:47:28,  6.60s/it]
 61%|██████    | 136705/225000 [273:33:14<143:30:00,  5.85s/it]
 61%|██████    | 136706/225000 [273:33:18<132:21:29,  5.40s/it]
 61%|██████    | 136707/225000 [273:33:23<132:14:20,  5.39s/it]
 61%|██████    | 136708/225000 [273:33:35<177:04:21,  7.22s/it]
 61%|██████    | 136709/225000 [273:33:38<147:31:58,  6.02s/it]
 61%|██████    | 136710/225000 [273:33:44<144:13:16,  5.88s/it]
 61%|██████    | 136711/225000 [273:33:49<139:42:40,  5.70s/it]
 61%|██████    | 136712/225000 [273:33:54<138:35:40,  5.65s/it]
 61%|██████    | 136713/225000 [273:34:04<164:51:26,  6.72s/it]
 61%|██████    | 136714/225000 [273:34:08<145:02:24,  5.91s/it]
 61%|██████    | 136715/225000 [273:34:16<162:46:51,  6.64s/it]
 61%|██████    | 136716/225000 [273:34:20<141:58:00,  5.79s/it]
 61%|██████    | 136717/225000 [273:34:27<156:01:25,  6.36s/it]
 61%|██████    | 136718/225000 [273:34:32<144:31:18,  5.89s/it]
 61%|██████    | 136719/225000 [273:34:38<143:28:06,  5.85s/it]
 61%|██████    | 136720/225000 [273:34:46<158:29:00,  6.46s/it]
 61%|██████    | 136721/225000 [273:34:52<154:33:32,  6.30s/it]
 61%|██████    | 136722/225000 [273:34:55<133:48:56,  5.46s/it]
 61%|██████    | 136723/225000 [273:34:59<119:24:52,  4.87s/it]
 61%|██████    | 136724/225000 [273:35:08<151:54:23,  6.19s/it]
 61%|██████    | 136725/225000 [273:35:11<130:41:40,  5.33s/it]
 61%|██████    | 136726/225000 [273:35:20<154:09:32,  6.29s/it]
 61%|██████    | 136727/225000 [273:35:23<131:25:21,  5.36s/it]
 61%|██████    | 136728/225000 [273:35:28<130:20:20,  5.32s/it]
 61%|██████    | 136729/225000 [273:35:34<129:31:13,  5.28s/it]
 61%|██████    | 136730/225000 [273:35:39<129:00:43,  5.26s/it]
 61%|██████    | 136731/225000 [273:35:44<130:28:21,  5.32s/it]
 61%|██████    | 136732/225000 [273:35:50<133:01:21,  5.43s/it]
 61%|██████    | 136733/225000 [273:35:55<133:27:56,  5.44s/it]
 61%|██████    | 136734/225000 [273:36:01<132:19:07,  5.40s/it]
 61%|██████    | 136735/225000 [273:36:06<134:16:33,  5.48s/it]
 61%|██████    | 136736/225000 [273:36:12<135:05:56,  5.51s/it]
 61%|██████    | 136737/225000 [273:36:18<135:50:49,  5.54s/it]
 61%|██████    | 136738/225000 [273:36:30<189:48:38,  7.74s/it]
 61%|██████    | 136739/225000 [273:36:39<199:16:37,  8.13s/it]
 61%|██████    | 136740/225000 [273:36:43<164:07:10,  6.69s/it]
 61%|██████    | 136741/225000 [273:36:51<176:28:04,  7.20s/it]
 61%|██████    | 136742/225000 [273:36:55<147:55:06,  6.03s/it]
 61%|██████    | 136743/225000 [273:37:02<155:47:18,  6.35s/it]
 61%|██████    | 136744/225000 [273:37:07<147:54:21,  6.03s/it]
 61%|██████    | 136745/225000 [273:37:13<145:37:41,  5.94s/it]
 61%|██████    | 136746/225000 [273:37:17<132:11:59,  5.39s/it]
 61%|██████    | 136747/225000 [273:37:24<146:42:34,  5.98s/it]
 61%|██████    | 136748/225000 [273:37:28<130:43:06,  5.33s/it]
 61%|██████    | 136749/225000 [273:37:35<144:05:25,  5.88s/it]
 61%|██████    | 136750/225000 [273:37:40<137:33:34,  5.61s/it]
                                                               
{'loss': '0.7958', 'grad_norm': '1.06', 'learning_rate': '1.808e-05', 'epoch': '0.6078', 'num_input_tokens_seen': 454063712, 'train_runtime': '9.851e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 136750/225000 [273:37:40<137:33:34,  5.61s/it]
 61%|██████    | 136751/225000 [273:37:44<128:22:37,  5.24s/it]
 61%|██████    | 136752/225000 [273:37:49<122:18:54,  4.99s/it]
 61%|██████    | 136753/225000 [273:37:53<118:44:30,  4.84s/it]
 61%|██████    | 136754/225000 [273:38:05<171:53:17,  7.01s/it]
 61%|██████    | 136755/225000 [273:38:12<170:35:08,  6.96s/it]
 61%|██████    | 136756/225000 [273:38:16<146:17:17,  5.97s/it]
 61%|██████    | 136757/225000 [273:38:22<150:26:58,  6.14s/it]
 61%|██████    | 136758/225000 [273:38:30<159:58:30,  6.53s/it]
 61%|██████    | 136759/225000 [273:38:34<143:18:24,  5.85s/it]
 61%|██████    | 136760/225000 [273:38:38<130:11:49,  5.31s/it]
 61%|██████    | 136761/225000 [273:38:42<120:28:41,  4.92s/it]
 61%|██████    | 136762/225000 [273:38:53<160:38:53,  6.55s/it]
 61%|██████    | 136763/225000 [273:38:57<148:12:20,  6.05s/it]
 61%|██████    | 136764/225000 [273:39:05<156:16:18,  6.38s/it]
 61%|██████    | 136765/225000 [273:39:10<149:20:44,  6.09s/it]
 61%|██████    | 136766/225000 [273:39:16<146:15:57,  5.97s/it]
 61%|██████    | 136767/225000 [273:39:21<145:00:41,  5.92s/it]
 61%|██████    | 136768/225000 [273:39:26<131:58:38,  5.38s/it]
 61%|██████    | 136769/225000 [273:39:30<127:27:24,  5.20s/it]
 61%|██████    | 136770/225000 [273:39:40<157:27:29,  6.42s/it]
 61%|██████    | 136771/225000 [273:39:44<143:19:30,  5.85s/it]
 61%|██████    | 136772/225000 [273:39:48<130:26:52,  5.32s/it]
 61%|██████    | 136773/225000 [273:39:53<124:32:06,  5.08s/it]
 61%|██████    | 136774/225000 [273:39:57<120:50:45,  4.93s/it]
 61%|██████    | 136775/225000 [273:40:02<116:29:41,  4.75s/it]
 61%|██████    | 136776/225000 [273:40:06<114:37:01,  4.68s/it]
 61%|██████    | 136777/225000 [273:40:11<112:37:03,  4.60s/it]
 61%|██████    | 136778/225000 [273:40:22<165:58:33,  6.77s/it]
 61%|██████    | 136779/225000 [273:40:28<158:10:38,  6.45s/it]
 61%|██████    | 136780/225000 [273:40:33<142:42:06,  5.82s/it]
 61%|██████    | 136781/225000 [273:40:40<157:46:03,  6.44s/it]
 61%|██████    | 136782/225000 [273:40:45<141:42:04,  5.78s/it]
 61%|██████    | 136783/225000 [273:40:52<153:27:42,  6.26s/it]
 61%|██████    | 136784/225000 [273:41:01<171:39:07,  7.00s/it]
 61%|██████    | 136785/225000 [273:41:05<153:28:22,  6.26s/it]
 61%|██████    | 136786/225000 [273:41:10<144:03:24,  5.88s/it]
 61%|██████    | 136787/225000 [273:41:15<138:14:43,  5.64s/it]
 61%|██████    | 136788/225000 [273:41:20<131:22:54,  5.36s/it]
 61%|██████    | 136789/225000 [273:41:25<130:35:54,  5.33s/it]
 61%|██████    | 136790/225000 [273:41:31<130:45:39,  5.34s/it]
 61%|██████    | 136791/225000 [273:41:35<126:06:53,  5.15s/it]
 61%|██████    | 136792/225000 [273:41:41<127:04:19,  5.19s/it]
 61%|██████    | 136793/225000 [273:41:51<168:19:53,  6.87s/it]
 61%|██████    | 136794/225000 [273:41:56<147:34:26,  6.02s/it]
 61%|██████    | 136795/225000 [273:41:59<131:54:23,  5.38s/it]
 61%|██████    | 136796/225000 [273:42:05<129:58:51,  5.31s/it]
 61%|██████    | 136797/225000 [273:42:15<171:28:34,  7.00s/it]
 61%|██████    | 136798/225000 [273:42:19<148:00:53,  6.04s/it]
 61%|██████    | 136799/225000 [273:42:23<130:18:20,  5.32s/it]
 61%|██████    | 136800/225000 [273:42:34<173:52:01,  7.10s/it]
                                                               
{'loss': '0.7973', 'grad_norm': '1.048', 'learning_rate': '1.806e-05', 'epoch': '0.608', 'num_input_tokens_seen': 454192808, 'train_runtime': '9.854e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 136800/225000 [273:42:34<173:52:01,  7.10s/it]
 61%|██████    | 136801/225000 [273:42:37<146:00:31,  5.96s/it]
 61%|██████    | 136802/225000 [273:42:47<172:10:21,  7.03s/it]
 61%|██████    | 136803/225000 [273:42:50<145:22:56,  5.93s/it]
 61%|██████    | 136804/225000 [273:42:57<153:07:16,  6.25s/it]
 61%|██████    | 136805/225000 [273:43:01<134:20:59,  5.48s/it]
 61%|██████    | 136806/225000 [273:43:10<157:27:15,  6.43s/it]
 61%|██████    | 136807/225000 [273:43:14<142:22:50,  5.81s/it]
 61%|██████    | 136808/225000 [273:43:18<130:49:14,  5.34s/it]
 61%|██████    | 136809/225000 [273:43:23<123:41:22,  5.05s/it]
 61%|██████    | 136810/225000 [273:43:27<120:43:17,  4.93s/it]
 61%|██████    | 136811/225000 [273:43:34<131:35:41,  5.37s/it]
 61%|██████    | 136812/225000 [273:43:43<162:26:04,  6.63s/it]
 61%|██████    | 136813/225000 [273:43:47<143:27:04,  5.86s/it]
 61%|██████    | 136814/225000 [273:43:51<128:24:13,  5.24s/it]
 61%|██████    | 136815/225000 [273:44:01<163:49:28,  6.69s/it]
 61%|██████    | 136816/225000 [273:44:05<142:43:01,  5.83s/it]
 61%|██████    | 136817/225000 [273:44:09<128:18:51,  5.24s/it]
 61%|██████    | 136818/225000 [273:44:19<166:21:08,  6.79s/it]
 61%|██████    | 136819/225000 [273:44:23<142:24:53,  5.81s/it]
 61%|██████    | 136820/225000 [273:44:28<139:21:18,  5.69s/it]
 61%|██████    | 136821/225000 [273:44:33<136:07:47,  5.56s/it]
 61%|██████    | 136822/225000 [273:44:42<154:51:50,  6.32s/it]
 61%|██████    | 136823/225000 [273:44:47<149:51:45,  6.12s/it]
 61%|██████    | 136824/225000 [273:44:52<137:17:57,  5.61s/it]
 61%|██████    | 136825/225000 [273:44:56<128:40:55,  5.25s/it]
 61%|██████    | 136826/225000 [273:45:05<157:34:21,  6.43s/it]
 61%|██████    | 136827/225000 [273:45:12<157:57:41,  6.45s/it]
 61%|██████    | 136828/225000 [273:45:15<137:21:23,  5.61s/it]
 61%|██████    | 136829/225000 [273:45:24<156:01:40,  6.37s/it]
 61%|██████    | 136830/225000 [273:45:31<160:40:07,  6.56s/it]
 61%|██████    | 136831/225000 [273:45:36<150:47:33,  6.16s/it]
 61%|██████    | 136832/225000 [273:45:40<137:59:48,  5.63s/it]
 61%|██████    | 136833/225000 [273:45:45<131:45:58,  5.38s/it]
 61%|██████    | 136834/225000 [273:45:49<124:32:00,  5.08s/it]
 61%|██████    | 136835/225000 [273:45:55<127:31:16,  5.21s/it]
 61%|██████    | 136836/225000 [273:46:00<123:44:39,  5.05s/it]
 61%|██████    | 136837/225000 [273:46:04<122:26:23,  5.00s/it]
 61%|██████    | 136838/225000 [273:46:10<123:09:03,  5.03s/it]
 61%|██████    | 136839/225000 [273:46:19<154:01:45,  6.29s/it]
 61%|██████    | 136840/225000 [273:46:29<183:36:20,  7.50s/it]
 61%|██████    | 136841/225000 [273:46:32<153:37:54,  6.27s/it]
 61%|██████    | 136842/225000 [273:46:41<170:20:52,  6.96s/it]
 61%|██████    | 136843/225000 [273:46:45<145:04:14,  5.92s/it]
 61%|██████    | 136844/225000 [273:46:55<176:32:28,  7.21s/it]
 61%|██████    | 136845/225000 [273:46:59<151:11:51,  6.17s/it]
 61%|██████    | 136846/225000 [273:47:03<135:27:20,  5.53s/it]
 61%|██████    | 136847/225000 [273:47:10<148:14:04,  6.05s/it]
 61%|██████    | 136848/225000 [273:47:17<160:01:23,  6.54s/it]
 61%|██████    | 136849/225000 [273:47:22<142:00:28,  5.80s/it]
 61%|██████    | 136850/225000 [273:47:26<131:32:08,  5.37s/it]
                                                               
{'loss': '0.8137', 'grad_norm': '1.095', 'learning_rate': '1.805e-05', 'epoch': '0.6082', 'num_input_tokens_seen': 454322608, 'train_runtime': '9.856e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 136850/225000 [273:47:26<131:32:08,  5.37s/it]
 61%|██████    | 136851/225000 [273:47:31<125:58:52,  5.15s/it]
 61%|██████    | 136852/225000 [273:47:39<146:53:24,  6.00s/it]
 61%|██████    | 136853/225000 [273:47:48<169:06:44,  6.91s/it]
 61%|██████    | 136854/225000 [273:47:52<149:52:09,  6.12s/it]
 61%|██████    | 136855/225000 [273:47:58<149:40:13,  6.11s/it]
 61%|██████    | 136856/225000 [273:48:02<133:30:55,  5.45s/it]
 61%|██████    | 136857/225000 [273:48:06<124:06:10,  5.07s/it]
 61%|██████    | 136858/225000 [273:48:17<168:05:28,  6.87s/it]
 61%|██████    | 136859/225000 [273:48:21<145:06:39,  5.93s/it]
 61%|██████    | 136860/225000 [273:48:25<129:07:08,  5.27s/it]
 61%|██████    | 136861/225000 [273:48:32<144:25:37,  5.90s/it]
 61%|██████    | 136862/225000 [273:48:39<154:58:37,  6.33s/it]
 61%|██████    | 136863/225000 [273:48:43<139:21:46,  5.69s/it]
 61%|██████    | 136864/225000 [273:48:51<154:03:03,  6.29s/it]
 61%|██████    | 136865/225000 [273:48:57<151:17:02,  6.18s/it]
 61%|██████    | 136866/225000 [273:49:04<155:26:46,  6.35s/it]
 61%|██████    | 136867/225000 [273:49:08<140:24:04,  5.74s/it]
 61%|██████    | 136868/225000 [273:49:13<132:12:47,  5.40s/it]
 61%|██████    | 136869/225000 [273:49:18<134:33:11,  5.50s/it]
 61%|██████    | 136870/225000 [273:49:30<181:20:13,  7.41s/it]
 61%|██████    | 136871/225000 [273:49:35<160:54:14,  6.57s/it]
 61%|██████    | 136872/225000 [273:49:39<140:35:36,  5.74s/it]
 61%|██████    | 136873/225000 [273:49:43<128:02:09,  5.23s/it]
 61%|██████    | 136874/225000 [273:49:47<122:33:39,  5.01s/it]
 61%|██████    | 136875/225000 [273:49:53<125:43:11,  5.14s/it]
 61%|██████    | 136876/225000 [273:49:59<131:45:13,  5.38s/it]
 61%|██████    | 136877/225000 [273:50:11<183:03:48,  7.48s/it]
 61%|██████    | 136878/225000 [273:50:14<152:56:57,  6.25s/it]
 61%|██████    | 136879/225000 [273:50:20<151:30:31,  6.19s/it]
 61%|██████    | 136880/225000 [273:50:28<160:06:13,  6.54s/it]
 61%|██████    | 136881/225000 [273:50:31<137:21:56,  5.61s/it]
 61%|██████    | 136882/225000 [273:50:40<160:03:28,  6.54s/it]
 61%|██████    | 136883/225000 [273:50:43<135:07:36,  5.52s/it]
 61%|██████    | 136884/225000 [273:50:50<143:51:23,  5.88s/it]
 61%|██████    | 136885/225000 [273:50:54<129:23:46,  5.29s/it]
 61%|██████    | 136886/225000 [273:51:04<163:11:38,  6.67s/it]
 61%|██████    | 136887/225000 [273:51:08<143:27:05,  5.86s/it]
 61%|██████    | 136888/225000 [273:51:11<127:07:59,  5.19s/it]
 61%|██████    | 136889/225000 [273:51:18<135:03:55,  5.52s/it]
 61%|██████    | 136890/225000 [273:51:28<169:32:39,  6.93s/it]
 61%|██████    | 136891/225000 [273:51:33<154:57:27,  6.33s/it]
 61%|██████    | 136892/225000 [273:51:38<150:13:36,  6.14s/it]
 61%|██████    | 136893/225000 [273:51:43<141:50:20,  5.80s/it]
 61%|██████    | 136894/225000 [273:51:51<154:01:13,  6.29s/it]
 61%|██████    | 136895/225000 [273:51:56<146:18:58,  5.98s/it]
 61%|██████    | 136896/225000 [273:52:01<136:51:42,  5.59s/it]
 61%|██████    | 136897/225000 [273:52:05<129:51:39,  5.31s/it]
 61%|██████    | 136898/225000 [273:52:10<127:42:24,  5.22s/it]
 61%|██████    | 136899/225000 [273:52:15<122:38:38,  5.01s/it]
 61%|██████    | 136900/225000 [273:52:19<119:02:15,  4.86s/it]
                                                               
{'loss': '0.7964', 'grad_norm': '1.073', 'learning_rate': '1.803e-05', 'epoch': '0.6084', 'num_input_tokens_seen': 454453856, 'train_runtime': '9.859e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 136900/225000 [273:52:19<119:02:15,  4.86s/it]
 61%|██████    | 136901/225000 [273:52:24<120:03:37,  4.91s/it]
 61%|██████    | 136902/225000 [273:52:29<119:06:30,  4.87s/it]
 61%|██████    | 136903/225000 [273:52:35<127:46:35,  5.22s/it]
 61%|██████    | 136904/225000 [273:52:47<172:39:05,  7.06s/it]
 61%|██████    | 136905/225000 [273:52:50<147:45:07,  6.04s/it]
 61%|██████    | 136906/225000 [273:52:56<142:48:04,  5.84s/it]
 61%|██████    | 136907/225000 [273:53:01<136:15:05,  5.57s/it]
 61%|██████    | 136908/225000 [273:53:06<136:14:07,  5.57s/it]
 61%|██████    | 136909/225000 [273:53:12<136:31:35,  5.58s/it]
 61%|██████    | 136910/225000 [273:53:17<136:14:25,  5.57s/it]
 61%|██████    | 136911/225000 [273:53:23<133:46:19,  5.47s/it]
 61%|██████    | 136912/225000 [273:53:35<182:17:03,  7.45s/it]
 61%|██████    | 136913/225000 [273:53:39<159:12:02,  6.51s/it]
 61%|██████    | 136914/225000 [273:53:43<141:50:59,  5.80s/it]
 61%|██████    | 136915/225000 [273:53:48<135:33:53,  5.54s/it]
 61%|██████    | 136916/225000 [273:53:53<130:22:09,  5.33s/it]
 61%|██████    | 136917/225000 [273:53:58<129:31:25,  5.29s/it]
 61%|██████    | 136918/225000 [273:54:09<168:06:42,  6.87s/it]
 61%|██████    | 136919/225000 [273:54:16<173:01:11,  7.07s/it]
 61%|██████    | 136920/225000 [273:54:21<159:06:06,  6.50s/it]
 61%|██████    | 136921/225000 [273:54:27<153:20:44,  6.27s/it]
 61%|██████    | 136922/225000 [273:54:33<148:31:02,  6.07s/it]
 61%|██████    | 136923/225000 [273:54:43<177:33:46,  7.26s/it]
 61%|██████    | 136924/225000 [273:54:46<147:13:24,  6.02s/it]
 61%|██████    | 136925/225000 [273:54:55<171:47:53,  7.02s/it]
 61%|██████    | 136926/225000 [273:55:00<153:50:39,  6.29s/it]
 61%|██████    | 136927/225000 [273:55:07<157:19:37,  6.43s/it]
 61%|██████    | 136928/225000 [273:55:14<168:46:31,  6.90s/it]
 61%|██████    | 136929/225000 [273:55:18<146:59:15,  6.01s/it]
 61%|██████    | 136930/225000 [273:55:22<131:32:17,  5.38s/it]
 61%|██████    | 136931/225000 [273:55:27<122:56:47,  5.03s/it]
 61%|██████    | 136932/225000 [273:55:37<166:05:52,  6.79s/it]
 61%|██████    | 136933/225000 [273:55:41<142:36:50,  5.83s/it]
 61%|██████    | 136934/225000 [273:55:46<133:31:38,  5.46s/it]
 61%|██████    | 136935/225000 [273:55:53<149:49:12,  6.12s/it]
 61%|██████    | 136936/225000 [273:55:57<131:38:19,  5.38s/it]
 61%|██████    | 136937/225000 [273:56:06<157:12:55,  6.43s/it]
 61%|██████    | 136938/225000 [273:56:11<147:16:51,  6.02s/it]
 61%|██████    | 136939/225000 [273:56:16<137:07:35,  5.61s/it]
 61%|██████    | 136940/225000 [273:56:20<128:14:05,  5.24s/it]
 61%|██████    | 136941/225000 [273:56:24<120:39:24,  4.93s/it]
 61%|██████    | 136942/225000 [273:56:33<152:54:12,  6.25s/it]
 61%|██████    | 136943/225000 [273:56:40<155:03:23,  6.34s/it]
 61%|██████    | 136944/225000 [273:56:46<156:02:03,  6.38s/it]
 61%|██████    | 136945/225000 [273:56:52<151:49:38,  6.21s/it]
 61%|██████    | 136946/225000 [273:56:57<142:08:41,  5.81s/it]
 61%|██████    | 136947/225000 [273:57:03<141:40:52,  5.79s/it]
 61%|██████    | 136948/225000 [273:57:06<122:29:10,  5.01s/it]
 61%|██████    | 136949/225000 [273:57:15<152:45:35,  6.25s/it]
 61%|██████    | 136950/225000 [273:57:19<131:30:31,  5.38s/it]
                                                               
{'loss': '0.7988', 'grad_norm': '1.149', 'learning_rate': '1.801e-05', 'epoch': '0.6087', 'num_input_tokens_seen': 454584040, 'train_runtime': '9.862e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 136950/225000 [273:57:19<131:30:31,  5.38s/it]
 61%|██████    | 136951/225000 [273:57:24<134:32:43,  5.50s/it]
 61%|██████    | 136952/225000 [273:57:32<151:54:15,  6.21s/it]
 61%|██████    | 136953/225000 [273:57:39<152:56:38,  6.25s/it]
 61%|██████    | 136954/225000 [273:57:43<137:25:51,  5.62s/it]
 61%|██████    | 136955/225000 [273:57:48<134:37:14,  5.50s/it]
 61%|██████    | 136956/225000 [273:57:53<133:11:01,  5.45s/it]
 61%|██████    | 136957/225000 [273:57:58<131:22:36,  5.37s/it]
 61%|██████    | 136958/225000 [273:58:04<130:31:20,  5.34s/it]
 61%|██████    | 136959/225000 [273:58:09<128:54:14,  5.27s/it]
 61%|██████    | 136960/225000 [273:58:14<128:49:53,  5.27s/it]
 61%|██████    | 136961/225000 [273:58:19<128:49:46,  5.27s/it]
 61%|██████    | 136962/225000 [273:58:25<129:35:13,  5.30s/it]
 61%|██████    | 136963/225000 [273:58:30<129:04:22,  5.28s/it]
 61%|██████    | 136964/225000 [273:58:35<129:48:44,  5.31s/it]
 61%|██████    | 136965/225000 [273:58:47<177:32:32,  7.26s/it]
 61%|██████    | 136966/225000 [273:58:56<188:28:37,  7.71s/it]
 61%|██████    | 136967/225000 [273:58:59<157:03:49,  6.42s/it]
 61%|██████    | 136968/225000 [273:59:05<151:32:22,  6.20s/it]
 61%|██████    | 136969/225000 [273:59:14<174:28:24,  7.14s/it]
 61%|██████    | 136970/225000 [273:59:18<146:21:51,  5.99s/it]
 61%|██████    | 136971/225000 [273:59:26<164:21:32,  6.72s/it]
 61%|██████    | 136972/225000 [273:59:30<147:21:45,  6.03s/it]
 61%|██████    | 136973/225000 [273:59:35<137:20:15,  5.62s/it]
 61%|██████    | 136974/225000 [273:59:40<133:46:21,  5.47s/it]
 61%|██████    | 136975/225000 [273:59:46<132:07:48,  5.40s/it]
 61%|██████    | 136976/225000 [273:59:50<128:10:06,  5.24s/it]
 61%|██████    | 136977/225000 [273:59:55<127:02:13,  5.20s/it]
 61%|██████    | 136978/225000 [274:00:00<125:43:35,  5.14s/it]
 61%|██████    | 136979/225000 [274:00:05<124:36:09,  5.10s/it]
 61%|██████    | 136980/225000 [274:00:11<125:54:50,  5.15s/it]
 61%|██████    | 136981/225000 [274:00:16<124:30:35,  5.09s/it]
 61%|██████    | 136982/225000 [274:00:24<148:02:18,  6.05s/it]
 61%|██████    | 136983/225000 [274:00:35<185:42:15,  7.60s/it]
 61%|██████    | 136984/225000 [274:00:39<154:13:45,  6.31s/it]
 61%|██████    | 136985/225000 [274:00:47<171:05:39,  7.00s/it]
 61%|██████    | 136986/225000 [274:00:51<148:44:18,  6.08s/it]
 61%|██████    | 136987/225000 [274:00:55<133:32:43,  5.46s/it]
 61%|██████    | 136988/225000 [274:00:59<124:56:53,  5.11s/it]
 61%|██████    | 136989/225000 [274:01:04<120:11:40,  4.92s/it]
 61%|██████    | 136990/225000 [274:01:15<164:52:55,  6.74s/it]
 61%|██████    | 136991/225000 [274:01:20<155:58:50,  6.38s/it]
 61%|██████    | 136992/225000 [274:01:24<138:47:47,  5.68s/it]
 61%|██████    | 136993/225000 [274:01:33<163:31:33,  6.69s/it]
 61%|██████    | 136994/225000 [274:01:37<142:18:35,  5.82s/it]
 61%|██████    | 136995/225000 [274:01:41<126:56:35,  5.19s/it]
 61%|██████    | 136996/225000 [274:01:53<175:16:57,  7.17s/it]
 61%|██████    | 136997/225000 [274:01:56<147:27:02,  6.03s/it]
 61%|██████    | 136998/225000 [274:02:03<154:12:44,  6.31s/it]
 61%|██████    | 136999/225000 [274:02:07<133:03:54,  5.44s/it]
 61%|██████    | 137000/225000 [274:02:15<158:23:13,  6.48s/it]
                                                               
{'loss': '0.797', 'grad_norm': '1.012', 'learning_rate': '1.799e-05', 'epoch': '0.6089', 'num_input_tokens_seen': 454713976, 'train_runtime': '9.865e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137000/225000 [274:02:15<158:23:13,  6.48s/it]
 61%|██████    | 137001/225000 [274:02:19<138:15:06,  5.66s/it]
 61%|██████    | 137002/225000 [274:02:26<149:44:24,  6.13s/it]
 61%|██████    | 137003/225000 [274:02:30<133:16:04,  5.45s/it]
 61%|██████    | 137004/225000 [274:02:34<124:11:47,  5.08s/it]
 61%|██████    | 137005/225000 [274:02:41<134:41:26,  5.51s/it]
 61%|██████    | 137006/225000 [274:02:50<161:42:18,  6.62s/it]
 61%|██████    | 137007/225000 [274:02:53<136:53:50,  5.60s/it]
 61%|██████    | 137008/225000 [274:03:02<159:47:25,  6.54s/it]
 61%|██████    | 137009/225000 [274:03:07<144:25:49,  5.91s/it]
 61%|██████    | 137010/225000 [274:03:11<135:07:44,  5.53s/it]
 61%|██████    | 137011/225000 [274:03:16<128:12:02,  5.25s/it]
 61%|██████    | 137012/225000 [274:03:20<123:47:43,  5.07s/it]
 61%|██████    | 137013/225000 [274:03:25<123:02:39,  5.03s/it]
 61%|██████    | 137014/225000 [274:03:30<121:16:20,  4.96s/it]
 61%|██████    | 137015/225000 [274:03:35<121:56:31,  4.99s/it]
 61%|██████    | 137016/225000 [274:03:47<171:32:34,  7.02s/it]
 61%|██████    | 137017/225000 [274:03:51<152:47:41,  6.25s/it]
 61%|██████    | 137018/225000 [274:04:00<168:40:45,  6.90s/it]
 61%|██████    | 137019/225000 [274:04:03<143:06:43,  5.86s/it]
 61%|██████    | 137020/225000 [274:04:10<148:13:20,  6.07s/it]
 61%|██████    | 137021/225000 [274:04:18<162:14:04,  6.64s/it]
 61%|██████    | 137022/225000 [274:04:23<148:20:24,  6.07s/it]
 61%|██████    | 137023/225000 [274:04:28<141:16:17,  5.78s/it]
 61%|██████    | 137024/225000 [274:04:32<131:17:42,  5.37s/it]
 61%|██████    | 137025/225000 [274:04:37<127:15:07,  5.21s/it]
 61%|██████    | 137026/225000 [274:04:42<125:08:20,  5.12s/it]
 61%|██████    | 137027/225000 [274:04:47<123:06:35,  5.04s/it]
 61%|██████    | 137028/225000 [274:04:53<134:50:55,  5.52s/it]
 61%|██████    | 137029/225000 [274:05:05<183:30:54,  7.51s/it]
 61%|██████    | 137030/225000 [274:05:10<162:22:07,  6.64s/it]
 61%|██████    | 137031/225000 [274:05:15<146:49:21,  6.01s/it]
 61%|██████    | 137032/225000 [274:05:19<135:20:52,  5.54s/it]
 61%|██████    | 137033/225000 [274:05:24<128:02:58,  5.24s/it]
 61%|██████    | 137034/225000 [274:05:28<122:03:54,  5.00s/it]
 61%|██████    | 137035/225000 [274:05:35<139:21:32,  5.70s/it]
 61%|██████    | 137036/225000 [274:05:47<178:58:01,  7.32s/it]
 61%|██████    | 137037/225000 [274:05:50<150:14:13,  6.15s/it]
 61%|██████    | 137038/225000 [274:05:55<140:16:07,  5.74s/it]
 61%|██████    | 137039/225000 [274:06:00<135:29:42,  5.55s/it]
 61%|██████    | 137040/225000 [274:06:05<132:03:17,  5.40s/it]
 61%|██████    | 137041/225000 [274:06:10<128:38:29,  5.27s/it]
 61%|██████    | 137042/225000 [274:06:15<127:07:56,  5.20s/it]
 61%|██████    | 137043/225000 [274:06:20<127:34:17,  5.22s/it]
 61%|██████    | 137044/225000 [274:06:25<125:16:35,  5.13s/it]
 61%|██████    | 137045/225000 [274:06:32<138:42:59,  5.68s/it]
 61%|██████    | 137046/225000 [274:06:45<193:59:05,  7.94s/it]
 61%|██████    | 137047/225000 [274:06:49<161:30:03,  6.61s/it]
 61%|██████    | 137048/225000 [274:06:52<139:32:27,  5.71s/it]
 61%|██████    | 137049/225000 [274:06:59<149:22:20,  6.11s/it]
 61%|██████    | 137050/225000 [274:07:04<140:53:28,  5.77s/it]
                                                               
{'loss': '0.8147', 'grad_norm': '1.221', 'learning_rate': '1.798e-05', 'epoch': '0.6091', 'num_input_tokens_seen': 454843408, 'train_runtime': '9.868e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137050/225000 [274:07:04<140:53:28,  5.77s/it]
 61%|██████    | 137051/225000 [274:07:10<137:44:11,  5.64s/it]
 61%|██████    | 137052/225000 [274:07:15<137:59:33,  5.65s/it]
 61%|██████    | 137053/225000 [274:07:20<133:10:17,  5.45s/it]
 61%|██████    | 137054/225000 [274:07:26<134:04:26,  5.49s/it]
 61%|██████    | 137055/225000 [274:07:31<132:05:55,  5.41s/it]
 61%|██████    | 137056/225000 [274:07:37<132:14:33,  5.41s/it]
 61%|██████    | 137057/225000 [274:07:42<130:41:43,  5.35s/it]
 61%|██████    | 137058/225000 [274:07:47<128:31:44,  5.26s/it]
 61%|██████    | 137059/225000 [274:07:53<136:33:26,  5.59s/it]
 61%|██████    | 137060/225000 [274:08:06<187:00:18,  7.66s/it]
 61%|██████    | 137061/225000 [274:08:13<183:56:11,  7.53s/it]
 61%|██████    | 137062/225000 [274:08:17<157:48:49,  6.46s/it]
 61%|██████    | 137063/225000 [274:08:21<140:25:20,  5.75s/it]
 61%|██████    | 137064/225000 [274:08:30<167:57:22,  6.88s/it]
 61%|██████    | 137065/225000 [274:08:35<150:39:37,  6.17s/it]
 61%|██████    | 137066/225000 [274:08:42<156:57:55,  6.43s/it]
 61%|██████    | 137067/225000 [274:08:48<156:27:20,  6.41s/it]
 61%|██████    | 137068/225000 [274:08:52<134:01:37,  5.49s/it]
 61%|██████    | 137069/225000 [274:08:59<147:24:06,  6.03s/it]
 61%|██████    | 137070/225000 [274:09:03<135:58:38,  5.57s/it]
 61%|██████    | 137071/225000 [274:09:08<126:53:36,  5.20s/it]
 61%|██████    | 137072/225000 [274:09:12<121:33:40,  4.98s/it]
 61%|██████    | 137073/225000 [274:09:24<173:21:22,  7.10s/it]
 61%|██████    | 137074/225000 [274:09:29<152:27:53,  6.24s/it]
 61%|██████    | 137075/225000 [274:09:33<139:23:37,  5.71s/it]
 61%|██████    | 137076/225000 [274:09:38<131:40:25,  5.39s/it]
 61%|██████    | 137077/225000 [274:09:42<125:56:30,  5.16s/it]
 61%|██████    | 137078/225000 [274:09:47<120:26:39,  4.93s/it]
 61%|██████    | 137079/225000 [274:09:52<119:45:46,  4.90s/it]
 61%|██████    | 137080/225000 [274:09:56<119:57:04,  4.91s/it]
 61%|██████    | 137081/225000 [274:10:08<165:08:08,  6.76s/it]
 61%|██████    | 137082/225000 [274:10:13<155:38:00,  6.37s/it]
 61%|██████    | 137083/225000 [274:10:17<135:58:30,  5.57s/it]
 61%|██████    | 137084/225000 [274:10:21<128:36:24,  5.27s/it]
 61%|██████    | 137085/225000 [274:10:33<175:40:29,  7.19s/it]
 61%|██████    | 137086/225000 [274:10:36<148:48:37,  6.09s/it]
 61%|██████    | 137087/225000 [274:10:40<130:36:54,  5.35s/it]
 61%|██████    | 137088/225000 [274:10:50<163:44:42,  6.71s/it]
 61%|██████    | 137089/225000 [274:10:54<140:58:15,  5.77s/it]
 61%|██████    | 137090/225000 [274:10:57<126:39:00,  5.19s/it]
 61%|██████    | 137091/225000 [274:11:09<170:23:19,  6.98s/it]
 61%|██████    | 137092/225000 [274:11:12<145:40:42,  5.97s/it]
 61%|██████    | 137093/225000 [274:11:18<145:37:49,  5.96s/it]
 61%|██████    | 137094/225000 [274:11:23<140:07:20,  5.74s/it]
 61%|██████    | 137095/225000 [274:11:28<129:25:14,  5.30s/it]
 61%|██████    | 137096/225000 [274:11:33<127:35:08,  5.23s/it]
 61%|██████    | 137097/225000 [274:11:37<123:41:23,  5.07s/it]
 61%|██████    | 137098/225000 [274:11:43<125:47:22,  5.15s/it]
 61%|██████    | 137099/225000 [274:11:48<125:38:16,  5.15s/it]
 61%|██████    | 137100/225000 [274:11:53<126:27:19,  5.18s/it]
                                                               
{'loss': '0.7931', 'grad_norm': '0.9833', 'learning_rate': '1.796e-05', 'epoch': '0.6093', 'num_input_tokens_seen': 454973696, 'train_runtime': '9.871e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137100/225000 [274:11:53<126:27:19,  5.18s/it]
 61%|██████    | 137101/225000 [274:11:58<126:31:18,  5.18s/it]
 61%|██████    | 137102/225000 [274:12:03<125:53:45,  5.16s/it]
 61%|██████    | 137103/225000 [274:12:12<153:31:43,  6.29s/it]
 61%|██████    | 137104/225000 [274:12:23<183:10:56,  7.50s/it]
 61%|██████    | 137105/225000 [274:12:26<153:55:54,  6.30s/it]
 61%|██████    | 137106/225000 [274:12:33<157:51:14,  6.47s/it]
 61%|██████    | 137107/225000 [274:12:37<141:01:06,  5.78s/it]
 61%|██████    | 137108/225000 [274:12:42<131:15:06,  5.38s/it]
 61%|██████    | 137109/225000 [274:12:51<162:36:33,  6.66s/it]
 61%|██████    | 137110/225000 [274:12:57<158:13:15,  6.48s/it]
 61%|██████    | 137111/225000 [274:13:01<139:08:44,  5.70s/it]
 61%|██████    | 137112/225000 [274:13:10<165:30:28,  6.78s/it]
 61%|██████    | 137113/225000 [274:13:15<146:38:15,  6.01s/it]
 61%|██████    | 137114/225000 [274:13:19<132:20:21,  5.42s/it]
 61%|██████    | 137115/225000 [274:13:27<155:01:54,  6.35s/it]
 61%|██████    | 137116/225000 [274:13:31<132:25:18,  5.42s/it]
 61%|██████    | 137117/225000 [274:13:34<118:29:14,  4.85s/it]
 61%|██████    | 137118/225000 [274:13:41<133:25:05,  5.47s/it]
 61%|██████    | 137119/225000 [274:13:49<153:18:44,  6.28s/it]
 61%|██████    | 137120/225000 [274:13:53<136:15:34,  5.58s/it]
 61%|██████    | 137121/225000 [274:13:57<123:25:56,  5.06s/it]
 61%|██████    | 137122/225000 [274:14:07<162:52:11,  6.67s/it]
 61%|██████    | 137123/225000 [274:14:12<145:35:10,  5.96s/it]
 61%|██████    | 137124/225000 [274:14:16<131:21:26,  5.38s/it]
 61%|██████    | 137125/225000 [274:14:23<146:49:34,  6.02s/it]
 61%|██████    | 137126/225000 [274:14:31<158:07:36,  6.48s/it]
 61%|██████    | 137127/225000 [274:14:35<139:24:09,  5.71s/it]
 61%|██████    | 137128/225000 [274:14:39<128:10:01,  5.25s/it]
 61%|██████    | 137129/225000 [274:14:50<171:36:39,  7.03s/it]
 61%|██████    | 137130/225000 [274:14:54<152:33:53,  6.25s/it]
 61%|██████    | 137131/225000 [274:15:01<153:16:24,  6.28s/it]
 61%|██████    | 137132/225000 [274:15:05<134:49:01,  5.52s/it]
 61%|██████    | 137133/225000 [274:15:08<123:13:11,  5.05s/it]
 61%|██████    | 137134/225000 [274:15:19<164:18:29,  6.73s/it]
 61%|██████    | 137135/225000 [274:15:23<140:58:12,  5.78s/it]
 61%|██████    | 137136/225000 [274:15:28<140:06:59,  5.74s/it]
 61%|██████    | 137137/225000 [274:15:34<139:49:51,  5.73s/it]
 61%|██████    | 137138/225000 [274:15:38<125:58:58,  5.16s/it]
 61%|██████    | 137139/225000 [274:15:47<156:32:10,  6.41s/it]
 61%|██████    | 137140/225000 [274:15:52<147:54:13,  6.06s/it]
 61%|██████    | 137141/225000 [274:15:56<131:53:31,  5.40s/it]
 61%|██████    | 137142/225000 [274:16:00<121:03:25,  4.96s/it]
 61%|██████    | 137143/225000 [274:16:09<148:23:38,  6.08s/it]
 61%|██████    | 137144/225000 [274:16:15<148:07:50,  6.07s/it]
 61%|██████    | 137145/225000 [274:16:19<131:04:59,  5.37s/it]
 61%|██████    | 137146/225000 [274:16:26<145:00:31,  5.94s/it]
 61%|██████    | 137147/225000 [274:16:32<146:13:46,  5.99s/it]
 61%|██████    | 137148/225000 [274:16:36<129:53:02,  5.32s/it]
 61%|██████    | 137149/225000 [274:16:41<129:32:08,  5.31s/it]
 61%|██████    | 137150/225000 [274:16:46<128:48:41,  5.28s/it]
                                                               
{'loss': '0.8019', 'grad_norm': '1.136', 'learning_rate': '1.794e-05', 'epoch': '0.6096', 'num_input_tokens_seen': 455104488, 'train_runtime': '9.874e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137150/225000 [274:16:46<128:48:41,  5.28s/it]
 61%|██████    | 137151/225000 [274:16:51<127:37:41,  5.23s/it]
 61%|██████    | 137152/225000 [274:16:57<130:06:38,  5.33s/it]
 61%|██████    | 137153/225000 [274:17:03<131:33:52,  5.39s/it]
 61%|██████    | 137154/225000 [274:17:08<129:58:16,  5.33s/it]
 61%|██████    | 137155/225000 [274:17:13<131:38:18,  5.39s/it]
 61%|██████    | 137156/225000 [274:17:19<133:10:01,  5.46s/it]
 61%|██████    | 137157/225000 [274:17:28<158:24:22,  6.49s/it]
 61%|██████    | 137158/225000 [274:17:38<184:19:28,  7.55s/it]
 61%|██████    | 137159/225000 [274:17:41<153:16:49,  6.28s/it]
 61%|██████    | 137160/225000 [274:17:49<163:38:15,  6.71s/it]
 61%|██████    | 137161/225000 [274:17:54<149:51:59,  6.14s/it]
 61%|██████    | 137162/225000 [274:17:58<139:32:31,  5.72s/it]
 61%|██████    | 137163/225000 [274:18:03<131:05:22,  5.37s/it]
 61%|██████    | 137164/225000 [274:18:08<128:49:21,  5.28s/it]
 61%|██████    | 137165/225000 [274:18:13<127:13:55,  5.21s/it]
 61%|██████    | 137166/225000 [274:18:20<137:36:19,  5.64s/it]
 61%|██████    | 137167/225000 [274:18:31<180:21:11,  7.39s/it]
 61%|██████    | 137168/225000 [274:18:34<149:37:34,  6.13s/it]
 61%|██████    | 137169/225000 [274:18:44<172:25:28,  7.07s/it]
 61%|██████    | 137170/225000 [274:18:47<143:27:51,  5.88s/it]
 61%|██████    | 137171/225000 [274:18:51<132:09:19,  5.42s/it]
 61%|██████    | 137172/225000 [274:18:56<125:54:34,  5.16s/it]
 61%|██████    | 137173/225000 [274:19:00<120:52:01,  4.95s/it]
 61%|██████    | 137174/225000 [274:19:05<121:38:49,  4.99s/it]
 61%|██████    | 137175/225000 [274:19:10<119:35:18,  4.90s/it]
 61%|██████    | 137176/225000 [274:19:15<120:12:52,  4.93s/it]
 61%|██████    | 137177/225000 [274:19:25<156:27:08,  6.41s/it]
 61%|██████    | 137178/225000 [274:19:34<178:35:08,  7.32s/it]
 61%|██████    | 137179/225000 [274:19:40<168:53:37,  6.92s/it]
 61%|██████    | 137180/225000 [274:19:44<146:32:22,  6.01s/it]
 61%|██████    | 137181/225000 [274:19:50<149:01:08,  6.11s/it]
 61%|██████    | 137182/225000 [274:19:59<169:13:19,  6.94s/it]
 61%|██████    | 137183/225000 [274:20:03<143:00:37,  5.86s/it]
 61%|██████    | 137184/225000 [274:20:10<150:44:18,  6.18s/it]
 61%|██████    | 137185/225000 [274:20:16<153:34:19,  6.30s/it]
 61%|██████    | 137186/225000 [274:20:19<131:00:19,  5.37s/it]
 61%|██████    | 137187/225000 [274:20:25<131:43:21,  5.40s/it]
 61%|██████    | 137188/225000 [274:20:31<135:17:08,  5.55s/it]
 61%|██████    | 137189/225000 [274:20:36<133:29:56,  5.47s/it]
 61%|██████    | 137190/225000 [274:20:41<133:17:20,  5.46s/it]
 61%|██████    | 137191/225000 [274:20:47<135:09:44,  5.54s/it]
 61%|██████    | 137192/225000 [274:20:58<171:55:53,  7.05s/it]
 61%|██████    | 137193/225000 [274:21:02<147:48:39,  6.06s/it]
 61%|██████    | 137194/225000 [274:21:05<131:01:17,  5.37s/it]
 61%|██████    | 137195/225000 [274:21:16<166:52:51,  6.84s/it]
 61%|██████    | 137196/225000 [274:21:20<146:32:58,  6.01s/it]
 61%|██████    | 137197/225000 [274:21:24<132:14:17,  5.42s/it]
 61%|██████    | 137198/225000 [274:21:29<130:50:43,  5.36s/it]
 61%|██████    | 137199/225000 [274:21:34<130:07:03,  5.34s/it]
 61%|██████    | 137200/225000 [274:21:40<130:23:40,  5.35s/it]
                                                               
{'loss': '0.8049', 'grad_norm': '1.08', 'learning_rate': '1.792e-05', 'epoch': '0.6098', 'num_input_tokens_seen': 455235352, 'train_runtime': '9.877e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137200/225000 [274:21:40<130:23:40,  5.35s/it]
 61%|██████    | 137201/225000 [274:21:45<131:50:47,  5.41s/it]
 61%|██████    | 137202/225000 [274:21:51<132:11:36,  5.42s/it]
 61%|██████    | 137203/225000 [274:21:56<133:48:14,  5.49s/it]
 61%|██████    | 137204/225000 [274:22:02<134:43:30,  5.52s/it]
 61%|██████    | 137205/225000 [274:22:07<133:58:06,  5.49s/it]
 61%|██████    | 137206/225000 [274:22:13<134:22:48,  5.51s/it]
 61%|██████    | 137207/225000 [274:22:18<134:57:21,  5.53s/it]
 61%|██████    | 137208/225000 [274:22:26<151:51:38,  6.23s/it]
 61%|██████    | 137209/225000 [274:22:36<181:00:55,  7.42s/it]
 61%|██████    | 137210/225000 [274:22:40<155:36:04,  6.38s/it]
 61%|██████    | 137211/225000 [274:22:44<137:14:39,  5.63s/it]
 61%|██████    | 137212/225000 [274:22:56<179:24:21,  7.36s/it]
 61%|██████    | 137213/225000 [274:22:59<153:52:48,  6.31s/it]
 61%|██████    | 137214/225000 [274:23:03<136:24:15,  5.59s/it]
 61%|██████    | 137215/225000 [274:23:11<154:06:54,  6.32s/it]
 61%|██████    | 137216/225000 [274:23:18<155:11:49,  6.36s/it]
 61%|██████    | 137217/225000 [274:23:22<136:40:16,  5.60s/it]
 61%|██████    | 137218/225000 [274:23:26<124:44:23,  5.12s/it]
 61%|██████    | 137219/225000 [274:23:30<117:23:50,  4.81s/it]
 61%|██████    | 137220/225000 [274:23:36<125:58:39,  5.17s/it]
 61%|██████    | 137221/225000 [274:23:47<170:08:01,  6.98s/it]
 61%|██████    | 137222/225000 [274:23:51<146:25:50,  6.01s/it]
 61%|██████    | 137223/225000 [274:23:54<129:14:58,  5.30s/it]
 61%|██████    | 137224/225000 [274:23:58<118:15:34,  4.85s/it]
 61%|██████    | 137225/225000 [274:24:09<158:32:32,  6.50s/it]
 61%|██████    | 137226/225000 [274:24:14<149:20:37,  6.13s/it]
 61%|██████    | 137227/225000 [274:24:20<148:36:45,  6.10s/it]
 61%|██████    | 137228/225000 [274:24:26<147:27:00,  6.05s/it]
 61%|██████    | 137229/225000 [274:24:30<135:36:20,  5.56s/it]
 61%|██████    | 137230/225000 [274:24:34<122:31:54,  5.03s/it]
 61%|██████    | 137231/225000 [274:24:40<127:50:48,  5.24s/it]
 61%|██████    | 137232/225000 [274:24:45<129:26:41,  5.31s/it]
 61%|██████    | 137233/225000 [274:24:51<131:39:41,  5.40s/it]
 61%|██████    | 137234/225000 [274:24:57<134:16:19,  5.51s/it]
 61%|██████    | 137235/225000 [274:25:05<155:51:08,  6.39s/it]
 61%|██████    | 137236/225000 [274:25:11<156:17:30,  6.41s/it]
 61%|██████    | 137237/225000 [274:25:15<135:50:49,  5.57s/it]
 61%|██████    | 137238/225000 [274:25:20<132:11:07,  5.42s/it]
 61%|██████    | 137239/225000 [274:25:28<148:32:42,  6.09s/it]
 61%|██████    | 137240/225000 [274:25:33<144:43:31,  5.94s/it]
 61%|██████    | 137241/225000 [274:25:39<141:40:40,  5.81s/it]
 61%|██████    | 137242/225000 [274:25:46<152:21:46,  6.25s/it]
 61%|██████    | 137243/225000 [274:25:50<132:34:37,  5.44s/it]
 61%|██████    | 137244/225000 [274:25:55<133:48:53,  5.49s/it]
 61%|██████    | 137245/225000 [274:26:04<155:31:17,  6.38s/it]
 61%|██████    | 137246/225000 [274:26:08<142:53:31,  5.86s/it]
 61%|██████    | 137247/225000 [274:26:13<131:22:32,  5.39s/it]
 61%|██████    | 137248/225000 [274:26:18<128:23:14,  5.27s/it]
 61%|██████    | 137249/225000 [274:26:23<125:05:50,  5.13s/it]
 61%|██████    | 137250/225000 [274:26:28<125:48:43,  5.16s/it]
                                                               
{'loss': '0.7904', 'grad_norm': '1.134', 'learning_rate': '1.791e-05', 'epoch': '0.61', 'num_input_tokens_seen': 455366488, 'train_runtime': '9.88e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137250/225000 [274:26:28<125:48:43,  5.16s/it]
 61%|██████    | 137251/225000 [274:26:33<125:17:52,  5.14s/it]
 61%|██████    | 137252/225000 [274:26:38<122:43:41,  5.04s/it]
 61%|██████    | 137253/225000 [274:26:48<162:25:05,  6.66s/it]
 61%|██████    | 137254/225000 [274:26:55<166:34:31,  6.83s/it]
 61%|██████    | 137255/225000 [274:26:59<141:40:26,  5.81s/it]
 61%|██████    | 137256/225000 [274:27:07<162:34:54,  6.67s/it]
 61%|██████    | 137257/225000 [274:27:11<139:42:00,  5.73s/it]
 61%|██████    | 137258/225000 [274:27:15<129:05:06,  5.30s/it]
 61%|██████    | 137259/225000 [274:27:20<122:19:10,  5.02s/it]
 61%|██████    | 137260/225000 [274:27:26<131:03:14,  5.38s/it]
 61%|██████    | 137261/225000 [274:27:37<172:11:15,  7.06s/it]
 61%|██████    | 137262/225000 [274:27:40<146:07:55,  6.00s/it]
 61%|██████    | 137263/225000 [274:27:44<128:18:41,  5.26s/it]
 61%|██████    | 137264/225000 [274:27:53<157:22:12,  6.46s/it]
 61%|██████    | 137265/225000 [274:27:56<134:30:46,  5.52s/it]
 61%|██████    | 137266/225000 [274:28:01<125:25:25,  5.15s/it]
 61%|██████    | 137267/225000 [274:28:10<155:03:59,  6.36s/it]
 61%|██████    | 137268/225000 [274:28:14<135:35:53,  5.56s/it]
 61%|██████    | 137269/225000 [274:28:18<125:57:21,  5.17s/it]
 61%|██████    | 137270/225000 [274:28:22<116:48:31,  4.79s/it]
 61%|██████    | 137271/225000 [274:28:34<171:13:52,  7.03s/it]
 61%|██████    | 137272/225000 [274:28:39<154:05:53,  6.32s/it]
 61%|██████    | 137273/225000 [274:28:43<142:14:15,  5.84s/it]
 61%|██████    | 137274/225000 [274:28:48<134:00:46,  5.50s/it]
 61%|██████    | 137275/225000 [274:28:53<126:57:46,  5.21s/it]
 61%|██████    | 137276/225000 [274:28:57<122:25:49,  5.02s/it]
 61%|██████    | 137277/225000 [274:29:02<120:11:51,  4.93s/it]
 61%|██████    | 137278/225000 [274:29:14<169:30:53,  6.96s/it]
 61%|██████    | 137279/225000 [274:29:18<147:21:45,  6.05s/it]
 61%|██████    | 137280/225000 [274:29:26<166:20:14,  6.83s/it]
 61%|██████    | 137281/225000 [274:29:30<144:37:11,  5.94s/it]
 61%|██████    | 137282/225000 [274:29:38<160:08:21,  6.57s/it]
 61%|██████    | 137283/225000 [274:29:43<144:12:25,  5.92s/it]
 61%|██████    | 137284/225000 [274:29:47<131:11:03,  5.38s/it]
 61%|██████    | 137285/225000 [274:29:52<131:09:11,  5.38s/it]
 61%|██████    | 137286/225000 [274:29:58<131:46:56,  5.41s/it]
 61%|██████    | 137287/225000 [274:30:02<128:29:18,  5.27s/it]
 61%|██████    | 137288/225000 [274:30:08<128:23:24,  5.27s/it]
 61%|██████    | 137289/225000 [274:30:12<124:14:19,  5.10s/it]
 61%|██████    | 137290/225000 [274:30:18<125:12:28,  5.14s/it]
 61%|██████    | 137291/225000 [274:30:24<133:10:21,  5.47s/it]
 61%|██████    | 137292/225000 [274:30:36<181:36:39,  7.45s/it]
 61%|██████    | 137293/225000 [274:30:41<166:54:11,  6.85s/it]
 61%|██████    | 137294/225000 [274:30:45<144:30:38,  5.93s/it]
 61%|██████    | 137295/225000 [274:30:51<140:34:16,  5.77s/it]
 61%|██████    | 137296/225000 [274:30:59<158:00:55,  6.49s/it]
 61%|██████    | 137297/225000 [274:31:02<137:25:58,  5.64s/it]
 61%|██████    | 137298/225000 [274:31:11<161:25:27,  6.63s/it]
 61%|██████    | 137299/225000 [274:31:15<139:21:27,  5.72s/it]
 61%|██████    | 137300/225000 [274:31:25<167:54:52,  6.89s/it]
                                                               
{'loss': '0.8104', 'grad_norm': '1.165', 'learning_rate': '1.789e-05', 'epoch': '0.6102', 'num_input_tokens_seen': 455496248, 'train_runtime': '9.883e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137300/225000 [274:31:25<167:54:52,  6.89s/it]
 61%|██████    | 137301/225000 [274:31:29<151:11:11,  6.21s/it]
 61%|██████    | 137302/225000 [274:31:34<137:22:00,  5.64s/it]
 61%|██████    | 137303/225000 [274:31:38<130:00:23,  5.34s/it]
 61%|██████    | 137304/225000 [274:31:43<123:32:44,  5.07s/it]
 61%|██████    | 137305/225000 [274:31:49<131:11:41,  5.39s/it]
 61%|██████    | 137306/225000 [274:31:59<170:01:46,  6.98s/it]
 61%|██████    | 137307/225000 [274:32:03<145:25:59,  5.97s/it]
 61%|██████    | 137308/225000 [274:32:11<160:26:42,  6.59s/it]
 61%|██████    | 137309/225000 [274:32:17<152:20:58,  6.25s/it]
 61%|██████    | 137310/225000 [274:32:20<135:11:04,  5.55s/it]
 61%|██████    | 137311/225000 [274:32:25<128:33:27,  5.28s/it]
 61%|██████    | 137312/225000 [274:32:30<126:30:19,  5.19s/it]
 61%|██████    | 137313/225000 [274:32:35<127:08:07,  5.22s/it]
 61%|██████    | 137314/225000 [274:32:40<124:39:53,  5.12s/it]
 61%|██████    | 137315/225000 [274:32:45<124:25:43,  5.11s/it]
 61%|██████    | 137316/225000 [274:32:50<123:00:36,  5.05s/it]
 61%|██████    | 137317/225000 [274:33:03<179:52:20,  7.39s/it]
 61%|██████    | 137318/225000 [274:33:09<166:08:08,  6.82s/it]
 61%|██████    | 137319/225000 [274:33:13<144:59:02,  5.95s/it]
 61%|██████    | 137320/225000 [274:33:17<132:06:47,  5.42s/it]
 61%|██████    | 137321/225000 [274:33:23<137:29:20,  5.65s/it]
 61%|██████    | 137322/225000 [274:33:34<176:47:52,  7.26s/it]
 61%|██████    | 137323/225000 [274:33:38<157:13:09,  6.46s/it]
 61%|██████    | 137324/225000 [274:33:43<143:15:53,  5.88s/it]
 61%|██████    | 137325/225000 [274:33:48<134:26:52,  5.52s/it]
 61%|██████    | 137326/225000 [274:33:52<126:07:45,  5.18s/it]
 61%|██████    | 137327/225000 [274:33:57<123:23:17,  5.07s/it]
 61%|██████    | 137328/225000 [274:34:01<117:17:23,  4.82s/it]
 61%|██████    | 137329/225000 [274:34:13<167:36:24,  6.88s/it]
 61%|██████    | 137330/225000 [274:34:17<147:14:45,  6.05s/it]
 61%|██████    | 137331/225000 [274:34:21<131:25:29,  5.40s/it]
 61%|██████    | 137332/225000 [274:34:28<146:43:10,  6.02s/it]
 61%|██████    | 137333/225000 [274:34:37<168:07:38,  6.90s/it]
 61%|██████    | 137334/225000 [274:34:41<144:19:34,  5.93s/it]
 61%|██████    | 137335/225000 [274:34:45<130:32:09,  5.36s/it]
 61%|██████    | 137336/225000 [274:34:50<131:49:10,  5.41s/it]
 61%|██████    | 137337/225000 [274:35:01<171:18:51,  7.04s/it]
 61%|██████    | 137338/225000 [274:35:05<147:34:36,  6.06s/it]
 61%|██████    | 137339/225000 [274:35:10<138:15:48,  5.68s/it]
 61%|██████    | 137340/225000 [274:35:15<132:55:48,  5.46s/it]
 61%|██████    | 137341/225000 [274:35:20<130:32:02,  5.36s/it]
 61%|██████    | 137342/225000 [274:35:25<127:49:37,  5.25s/it]
 61%|██████    | 137343/225000 [274:35:30<125:26:11,  5.15s/it]
 61%|██████    | 137344/225000 [274:35:35<122:51:42,  5.05s/it]
 61%|██████    | 137345/225000 [274:35:40<122:48:55,  5.04s/it]
 61%|██████    | 137346/225000 [274:35:45<121:08:40,  4.98s/it]
 61%|██████    | 137347/225000 [274:35:56<165:21:26,  6.79s/it]
 61%|██████    | 137348/225000 [274:36:02<164:01:34,  6.74s/it]
 61%|██████    | 137349/225000 [274:36:08<155:28:59,  6.39s/it]
 61%|██████    | 137350/225000 [274:36:15<160:51:37,  6.61s/it]
                                                               
{'loss': '0.797', 'grad_norm': '0.9889', 'learning_rate': '1.787e-05', 'epoch': '0.6104', 'num_input_tokens_seen': 455627080, 'train_runtime': '9.886e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137350/225000 [274:36:15<160:51:37,  6.61s/it]
 61%|██████    | 137351/225000 [274:36:19<140:51:19,  5.79s/it]
 61%|██████    | 137352/225000 [274:36:28<163:29:13,  6.71s/it]
 61%|██████    | 137353/225000 [274:36:32<145:35:59,  5.98s/it]
 61%|██████    | 137354/225000 [274:36:36<134:15:05,  5.51s/it]
 61%|██████    | 137355/225000 [274:36:41<124:52:20,  5.13s/it]
 61%|██████    | 137356/225000 [274:36:45<121:50:16,  5.00s/it]
 61%|██████    | 137357/225000 [274:36:57<171:03:29,  7.03s/it]
 61%|██████    | 137358/225000 [274:37:03<161:29:28,  6.63s/it]
 61%|██████    | 137359/225000 [274:37:10<169:26:09,  6.96s/it]
 61%|██████    | 137360/225000 [274:37:14<142:01:25,  5.83s/it]
 61%|██████    | 137361/225000 [274:37:21<152:32:45,  6.27s/it]
 61%|██████    | 137362/225000 [274:37:25<133:20:35,  5.48s/it]
 61%|██████    | 137363/225000 [274:37:29<123:44:58,  5.08s/it]
 61%|██████    | 137364/225000 [274:37:33<114:53:52,  4.72s/it]
 61%|██████    | 137365/225000 [274:37:44<163:43:14,  6.73s/it]
 61%|██████    | 137366/225000 [274:37:48<142:11:57,  5.84s/it]
 61%|██████    | 137367/225000 [274:37:52<132:12:21,  5.43s/it]
 61%|██████    | 137368/225000 [274:38:03<174:40:36,  7.18s/it]
 61%|██████    | 137369/225000 [274:38:07<148:16:20,  6.09s/it]
 61%|██████    | 137370/225000 [274:38:13<145:02:31,  5.96s/it]
 61%|██████    | 137371/225000 [274:38:17<135:21:16,  5.56s/it]
 61%|██████    | 137372/225000 [274:38:21<122:22:50,  5.03s/it]
 61%|██████    | 137373/225000 [274:38:28<134:39:44,  5.53s/it]
 61%|██████    | 137374/225000 [274:38:32<126:56:41,  5.22s/it]
 61%|██████    | 137375/225000 [274:38:38<127:14:35,  5.23s/it]
 61%|██████    | 137376/225000 [274:38:42<120:30:12,  4.95s/it]
 61%|██████    | 137377/225000 [274:38:47<121:32:46,  4.99s/it]
 61%|██████    | 137378/225000 [274:38:52<121:26:01,  4.99s/it]
 61%|██████    | 137379/225000 [274:38:57<121:28:21,  4.99s/it]
 61%|██████    | 137380/225000 [274:39:02<120:55:42,  4.97s/it]
 61%|██████    | 137381/225000 [274:39:07<121:23:49,  4.99s/it]
 61%|██████    | 137382/225000 [274:39:12<122:09:55,  5.02s/it]
 61%|██████    | 137383/225000 [274:39:17<122:21:20,  5.03s/it]
 61%|██████    | 137384/225000 [274:39:26<153:33:11,  6.31s/it]
 61%|██████    | 137385/225000 [274:39:37<189:06:09,  7.77s/it]
 61%|██████    | 137386/225000 [274:39:41<155:09:02,  6.38s/it]
 61%|██████    | 137387/225000 [274:39:50<173:46:27,  7.14s/it]
 61%|██████    | 137388/225000 [274:39:53<145:33:33,  5.98s/it]
 61%|██████    | 137389/225000 [274:39:57<133:06:53,  5.47s/it]
 61%|██████    | 137390/225000 [274:40:02<126:00:51,  5.18s/it]
 61%|██████    | 137391/225000 [274:40:08<135:00:45,  5.55s/it]
 61%|██████    | 137392/225000 [274:40:18<169:39:28,  6.97s/it]
 61%|██████    | 137393/225000 [274:40:22<146:05:35,  6.00s/it]
 61%|██████    | 137394/225000 [274:40:26<129:07:50,  5.31s/it]
 61%|██████    | 137395/225000 [274:40:36<165:49:41,  6.81s/it]
 61%|██████    | 137396/225000 [274:40:39<140:05:30,  5.76s/it]
 61%|██████    | 137397/225000 [274:40:48<159:11:33,  6.54s/it]
 61%|██████    | 137398/225000 [274:40:51<137:41:56,  5.66s/it]
 61%|██████    | 137399/225000 [274:41:00<156:53:15,  6.45s/it]
 61%|██████    | 137400/225000 [274:41:05<149:44:44,  6.15s/it]
                                                               
{'loss': '0.7983', 'grad_norm': '1.004', 'learning_rate': '1.785e-05', 'epoch': '0.6107', 'num_input_tokens_seen': 455757376, 'train_runtime': '9.889e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137400/225000 [274:41:05<149:44:44,  6.15s/it]
 61%|██████    | 137401/225000 [274:41:11<145:58:27,  6.00s/it]
 61%|██████    | 137402/225000 [274:41:16<141:06:20,  5.80s/it]
 61%|██████    | 137403/225000 [274:41:22<140:01:32,  5.75s/it]
 61%|██████    | 137404/225000 [274:41:27<137:50:12,  5.66s/it]
 61%|██████    | 137405/225000 [274:41:33<138:28:23,  5.69s/it]
 61%|██████    | 137406/225000 [274:41:40<149:02:58,  6.13s/it]
 61%|██████    | 137407/225000 [274:41:45<141:40:39,  5.82s/it]
 61%|██████    | 137408/225000 [274:41:50<131:20:49,  5.40s/it]
 61%|██████    | 137409/225000 [274:41:55<130:11:56,  5.35s/it]
 61%|██████    | 137410/225000 [274:42:00<125:46:49,  5.17s/it]
 61%|██████    | 137411/225000 [274:42:05<124:41:27,  5.12s/it]
 61%|██████    | 137412/225000 [274:42:09<121:49:23,  5.01s/it]
 61%|██████    | 137413/225000 [274:42:14<120:46:49,  4.96s/it]
 61%|██████    | 137414/225000 [274:42:19<122:02:13,  5.02s/it]
 61%|██████    | 137415/225000 [274:42:24<122:58:55,  5.05s/it]
 61%|██████    | 137416/225000 [274:42:29<118:52:10,  4.89s/it]
 61%|██████    | 137417/225000 [274:42:40<164:15:30,  6.75s/it]
 61%|██████    | 137418/225000 [274:42:49<178:48:23,  7.35s/it]
 61%|██████    | 137419/225000 [274:42:52<149:58:40,  6.16s/it]
 61%|██████    | 137420/225000 [274:42:58<143:59:29,  5.92s/it]
 61%|██████    | 137421/225000 [274:43:05<157:02:59,  6.46s/it]
 61%|██████    | 137422/225000 [274:43:11<152:50:49,  6.28s/it]
 61%|██████    | 137423/225000 [274:43:17<149:12:41,  6.13s/it]
 61%|██████    | 137424/225000 [274:43:21<131:29:47,  5.41s/it]
 61%|██████    | 137425/225000 [274:43:25<121:28:14,  4.99s/it]
 61%|██████    | 137426/225000 [274:43:35<160:52:33,  6.61s/it]
 61%|██████    | 137427/225000 [274:43:40<149:10:02,  6.13s/it]
 61%|██████    | 137428/225000 [274:43:47<156:49:56,  6.45s/it]
 61%|██████    | 137429/225000 [274:43:53<150:51:19,  6.20s/it]
 61%|██████    | 137430/225000 [274:44:00<156:44:31,  6.44s/it]
 61%|██████    | 137431/225000 [274:44:05<148:13:25,  6.09s/it]
 61%|██████    | 137432/225000 [274:44:09<129:42:38,  5.33s/it]
 61%|██████    | 137433/225000 [274:44:18<155:05:49,  6.38s/it]
 61%|██████    | 137434/225000 [274:44:21<135:27:55,  5.57s/it]
 61%|██████    | 137435/225000 [274:44:26<129:59:52,  5.34s/it]
 61%|██████    | 137436/225000 [274:44:34<149:00:38,  6.13s/it]
 61%|██████    | 137437/225000 [274:44:42<159:47:22,  6.57s/it]
 61%|██████    | 137438/225000 [274:44:46<143:08:53,  5.89s/it]
 61%|██████    | 137439/225000 [274:44:50<129:35:39,  5.33s/it]
 61%|██████    | 137440/225000 [274:44:54<120:18:59,  4.95s/it]
 61%|██████    | 137441/225000 [274:45:04<156:11:51,  6.42s/it]
 61%|██████    | 137442/225000 [274:45:08<137:59:51,  5.67s/it]
 61%|██████    | 137443/225000 [274:45:15<146:01:05,  6.00s/it]
 61%|██████    | 137444/225000 [274:45:18<128:28:22,  5.28s/it]
 61%|██████    | 137445/225000 [274:45:22<120:56:05,  4.97s/it]
 61%|██████    | 137446/225000 [274:45:27<116:06:15,  4.77s/it]
 61%|██████    | 137447/225000 [274:45:37<155:05:21,  6.38s/it]
 61%|██████    | 137448/225000 [274:45:43<152:19:29,  6.26s/it]
 61%|██████    | 137449/225000 [274:45:47<135:42:48,  5.58s/it]
 61%|██████    | 137450/225000 [274:45:56<161:46:39,  6.65s/it]
                                                               
{'loss': '0.812', 'grad_norm': '1.065', 'learning_rate': '1.784e-05', 'epoch': '0.6109', 'num_input_tokens_seen': 455888104, 'train_runtime': '9.892e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137450/225000 [274:45:56<161:46:39,  6.65s/it]
 61%|██████    | 137451/225000 [274:46:02<160:44:36,  6.61s/it]
 61%|██████    | 137452/225000 [274:46:06<139:21:22,  5.73s/it]
 61%|██████    | 137453/225000 [274:46:15<160:56:28,  6.62s/it]
 61%|██████    | 137454/225000 [274:46:20<147:47:45,  6.08s/it]
 61%|██████    | 137455/225000 [274:46:24<138:31:15,  5.70s/it]
 61%|██████    | 137456/225000 [274:46:29<129:06:16,  5.31s/it]
 61%|██████    | 137457/225000 [274:46:33<124:03:16,  5.10s/it]
 61%|██████    | 137458/225000 [274:46:38<117:41:07,  4.84s/it]
 61%|██████    | 137459/225000 [274:46:43<117:38:52,  4.84s/it]
 61%|██████    | 137460/225000 [274:46:48<122:09:06,  5.02s/it]
 61%|██████    | 137461/225000 [274:47:00<175:03:09,  7.20s/it]
 61%|██████    | 137462/225000 [274:47:06<167:04:31,  6.87s/it]
 61%|██████    | 137463/225000 [274:47:10<141:42:44,  5.83s/it]
 61%|██████    | 137464/225000 [274:47:18<163:02:21,  6.71s/it]
 61%|██████    | 137465/225000 [274:47:22<138:44:25,  5.71s/it]
 61%|██████    | 137466/225000 [274:47:31<166:39:59,  6.85s/it]
 61%|██████    | 137467/225000 [274:47:34<138:03:43,  5.68s/it]
 61%|██████    | 137468/225000 [274:47:41<145:06:58,  5.97s/it]
 61%|██████    | 137469/225000 [274:47:46<140:42:27,  5.79s/it]
 61%|██████    | 137470/225000 [274:47:51<133:28:36,  5.49s/it]
 61%|██████    | 137471/225000 [274:47:56<125:37:29,  5.17s/it]
 61%|██████    | 137472/225000 [274:48:00<121:32:26,  5.00s/it]
 61%|██████    | 137473/225000 [274:48:05<120:36:05,  4.96s/it]
 61%|██████    | 137474/225000 [274:48:09<116:59:11,  4.81s/it]
 61%|██████    | 137475/225000 [274:48:19<147:55:05,  6.08s/it]
 61%|██████    | 137476/225000 [274:48:27<165:53:39,  6.82s/it]
 61%|██████    | 137477/225000 [274:48:31<143:53:44,  5.92s/it]
 61%|██████    | 137478/225000 [274:48:36<137:08:55,  5.64s/it]
 61%|██████    | 137479/225000 [274:48:45<161:27:38,  6.64s/it]
 61%|██████    | 137480/225000 [274:48:48<138:47:10,  5.71s/it]
 61%|██████    | 137481/225000 [274:48:58<165:46:54,  6.82s/it]
 61%|██████    | 137482/225000 [274:49:01<141:59:14,  5.84s/it]
 61%|██████    | 137483/225000 [274:49:07<139:43:01,  5.75s/it]
 61%|██████    | 137484/225000 [274:49:16<165:09:50,  6.79s/it]
 61%|██████    | 137485/225000 [274:49:20<145:06:49,  5.97s/it]
 61%|██████    | 137486/225000 [274:49:26<144:38:00,  5.95s/it]
 61%|██████    | 137487/225000 [274:49:32<146:11:14,  6.01s/it]
 61%|██████    | 137488/225000 [274:49:39<152:10:08,  6.26s/it]
 61%|██████    | 137489/225000 [274:49:44<140:27:02,  5.78s/it]
 61%|██████    | 137490/225000 [274:49:48<132:13:36,  5.44s/it]
 61%|██████    | 137491/225000 [274:49:53<127:46:52,  5.26s/it]
 61%|██████    | 137492/225000 [274:49:58<122:54:23,  5.06s/it]
 61%|██████    | 137493/225000 [274:50:03<122:17:50,  5.03s/it]
 61%|██████    | 137494/225000 [274:50:12<151:41:08,  6.24s/it]
 61%|██████    | 137495/225000 [274:50:22<177:37:44,  7.31s/it]
 61%|██████    | 137496/225000 [274:50:26<158:19:13,  6.51s/it]
 61%|██████    | 137497/225000 [274:50:32<150:46:14,  6.20s/it]
 61%|██████    | 137498/225000 [274:50:36<134:03:58,  5.52s/it]
 61%|██████    | 137499/225000 [274:50:43<145:09:27,  5.97s/it]
 61%|██████    | 137500/225000 [274:50:50<155:04:27,  6.38s/it]
                                                               
{'loss': '0.7995', 'grad_norm': '1.081', 'learning_rate': '1.782e-05', 'epoch': '0.6111', 'num_input_tokens_seen': 456017936, 'train_runtime': '9.895e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137500/225000 [274:50:50<155:04:27,  6.38s/it]
 61%|██████    | 137501/225000 [274:50:53<130:42:06,  5.38s/it]
 61%|██████    | 137502/225000 [274:50:58<129:03:12,  5.31s/it]
 61%|██████    | 137503/225000 [274:51:08<160:53:51,  6.62s/it]
 61%|██████    | 137504/225000 [274:51:11<136:33:46,  5.62s/it]
 61%|██████    | 137505/225000 [274:51:15<120:16:52,  4.95s/it]
 61%|██████    | 137506/225000 [274:51:24<156:19:12,  6.43s/it]
 61%|██████    | 137507/225000 [274:51:30<148:30:01,  6.11s/it]
 61%|██████    | 137508/225000 [274:51:34<135:36:03,  5.58s/it]
 61%|██████    | 137509/225000 [274:51:39<129:41:30,  5.34s/it]
 61%|██████    | 137510/225000 [274:51:44<124:41:56,  5.13s/it]
 61%|██████    | 137511/225000 [274:51:49<123:57:55,  5.10s/it]
 61%|██████    | 137512/225000 [274:51:53<118:58:07,  4.90s/it]
 61%|██████    | 137513/225000 [274:51:58<119:44:50,  4.93s/it]
 61%|██████    | 137514/225000 [274:52:03<121:12:31,  4.99s/it]
 61%|██████    | 137515/225000 [274:52:09<128:53:21,  5.30s/it]
 61%|██████    | 137516/225000 [274:52:21<172:27:27,  7.10s/it]
 61%|██████    | 137517/225000 [274:52:28<174:26:02,  7.18s/it]
 61%|██████    | 137518/225000 [274:52:32<149:42:39,  6.16s/it]
 61%|██████    | 137519/225000 [274:52:36<133:43:38,  5.50s/it]
 61%|██████    | 137520/225000 [274:52:46<165:44:34,  6.82s/it]
 61%|██████    | 137521/225000 [274:52:49<141:24:27,  5.82s/it]
 61%|██████    | 137522/225000 [274:52:53<131:18:26,  5.40s/it]
 61%|██████    | 137523/225000 [274:53:04<170:05:18,  7.00s/it]
 61%|██████    | 137524/225000 [274:53:08<146:58:33,  6.05s/it]
 61%|██████    | 137525/225000 [274:53:15<151:13:00,  6.22s/it]
 61%|██████    | 137526/225000 [274:53:21<155:00:56,  6.38s/it]
 61%|██████    | 137527/225000 [274:53:28<157:25:47,  6.48s/it]
 61%|██████    | 137528/225000 [274:53:34<150:33:59,  6.20s/it]
 61%|██████    | 137529/225000 [274:53:41<156:16:25,  6.43s/it]
 61%|██████    | 137530/225000 [274:53:45<139:11:56,  5.73s/it]
 61%|██████    | 137531/225000 [274:53:49<127:19:48,  5.24s/it]
 61%|██████    | 137532/225000 [274:53:53<119:39:35,  4.92s/it]
 61%|██████    | 137533/225000 [274:54:01<141:25:19,  5.82s/it]
 61%|██████    | 137534/225000 [274:54:09<160:51:02,  6.62s/it]
 61%|██████    | 137535/225000 [274:54:13<139:57:27,  5.76s/it]
 61%|██████    | 137536/225000 [274:54:19<137:22:55,  5.65s/it]
 61%|██████    | 137537/225000 [274:54:24<133:28:48,  5.49s/it]
 61%|██████    | 137538/225000 [274:54:29<130:05:17,  5.35s/it]
 61%|██████    | 137539/225000 [274:54:34<128:33:35,  5.29s/it]
 61%|██████    | 137540/225000 [274:54:39<129:21:14,  5.32s/it]
 61%|██████    | 137541/225000 [274:54:44<128:51:57,  5.30s/it]
 61%|██████    | 137542/225000 [274:54:50<127:55:01,  5.27s/it]
 61%|██████    | 137543/225000 [274:54:55<125:32:21,  5.17s/it]
 61%|██████    | 137544/225000 [274:55:05<162:20:14,  6.68s/it]
 61%|██████    | 137545/225000 [274:55:13<172:24:05,  7.10s/it]
 61%|██████    | 137546/225000 [274:55:16<144:52:52,  5.96s/it]
 61%|██████    | 137547/225000 [274:55:24<155:18:58,  6.39s/it]
 61%|██████    | 137548/225000 [274:55:29<149:02:01,  6.14s/it]
 61%|██████    | 137549/225000 [274:55:33<133:24:15,  5.49s/it]
 61%|██████    | 137550/225000 [274:55:38<125:30:37,  5.17s/it]
                                                               
{'loss': '0.7987', 'grad_norm': '1.024', 'learning_rate': '1.78e-05', 'epoch': '0.6113', 'num_input_tokens_seen': 456147136, 'train_runtime': '9.897e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137550/225000 [274:55:38<125:30:37,  5.17s/it]
 61%|██████    | 137551/225000 [274:55:45<139:06:41,  5.73s/it]
 61%|██████    | 137552/225000 [274:55:55<170:29:02,  7.02s/it]
 61%|██████    | 137553/225000 [274:55:58<143:40:26,  5.91s/it]
 61%|██████    | 137554/225000 [274:56:06<157:48:47,  6.50s/it]
 61%|██████    | 137555/225000 [274:56:10<141:57:29,  5.84s/it]
 61%|██████    | 137556/225000 [274:56:17<152:26:43,  6.28s/it]
 61%|██████    | 137557/225000 [274:56:20<129:07:14,  5.32s/it]
 61%|██████    | 137558/225000 [274:56:29<154:46:02,  6.37s/it]
 61%|██████    | 137559/225000 [274:56:33<133:56:42,  5.51s/it]
 61%|██████    | 137560/225000 [274:56:37<123:01:44,  5.07s/it]
 61%|██████    | 137561/225000 [274:56:41<119:26:12,  4.92s/it]
 61%|██████    | 137562/225000 [274:56:46<115:55:21,  4.77s/it]
 61%|██████    | 137563/225000 [274:56:51<116:57:12,  4.82s/it]
 61%|██████    | 137564/225000 [274:57:01<155:58:36,  6.42s/it]
 61%|██████    | 137565/225000 [274:57:09<168:47:14,  6.95s/it]
 61%|██████    | 137566/225000 [274:57:14<153:45:00,  6.33s/it]
 61%|██████    | 137567/225000 [274:57:19<144:52:46,  5.97s/it]
 61%|██████    | 137568/225000 [274:57:25<141:50:31,  5.84s/it]
 61%|██████    | 137569/225000 [274:57:29<133:27:56,  5.50s/it]
 61%|██████    | 137570/225000 [274:57:34<127:37:24,  5.26s/it]
 61%|██████    | 137571/225000 [274:57:39<128:10:40,  5.28s/it]
 61%|██████    | 137572/225000 [274:57:45<130:41:55,  5.38s/it]
 61%|██████    | 137573/225000 [274:57:50<128:38:33,  5.30s/it]
 61%|██████    | 137574/225000 [274:57:55<127:26:03,  5.25s/it]
 61%|██████    | 137575/225000 [274:58:00<126:37:26,  5.21s/it]
 61%|██████    | 137576/225000 [274:58:06<126:25:59,  5.21s/it]
 61%|██████    | 137577/225000 [274:58:11<126:38:18,  5.21s/it]
 61%|██████    | 137578/225000 [274:58:23<176:10:15,  7.25s/it]
 61%|██████    | 137579/225000 [274:58:31<185:49:45,  7.65s/it]
 61%|██████    | 137580/225000 [274:58:35<156:18:54,  6.44s/it]
 61%|██████    | 137581/225000 [274:58:43<170:28:01,  7.02s/it]
 61%|██████    | 137582/225000 [274:58:47<144:30:34,  5.95s/it]
 61%|██████    | 137583/225000 [274:58:55<162:35:10,  6.70s/it]
 61%|██████    | 137584/225000 [274:58:59<139:11:22,  5.73s/it]
 61%|██████    | 137585/225000 [274:59:07<159:35:42,  6.57s/it]
 61%|██████    | 137586/225000 [274:59:11<136:23:39,  5.62s/it]
 61%|██████    | 137587/225000 [274:59:20<160:35:47,  6.61s/it]
 61%|██████    | 137588/225000 [274:59:24<147:42:52,  6.08s/it]
 61%|██████    | 137589/225000 [274:59:29<137:22:46,  5.66s/it]
 61%|██████    | 137590/225000 [274:59:34<130:49:58,  5.39s/it]
 61%|██████    | 137591/225000 [274:59:39<127:00:13,  5.23s/it]
 61%|██████    | 137592/225000 [274:59:43<122:14:55,  5.03s/it]
 61%|██████    | 137593/225000 [274:59:48<120:12:48,  4.95s/it]
 61%|██████    | 137594/225000 [274:59:53<120:36:17,  4.97s/it]
 61%|██████    | 137595/225000 [274:59:58<118:15:17,  4.87s/it]
 61%|██████    | 137596/225000 [275:00:05<138:17:47,  5.70s/it]
 61%|██████    | 137597/225000 [275:00:16<171:10:49,  7.05s/it]
 61%|██████    | 137598/225000 [275:00:20<148:37:58,  6.12s/it]
 61%|██████    | 137599/225000 [275:00:24<133:23:14,  5.49s/it]
 61%|██████    | 137600/225000 [275:00:34<169:46:25,  6.99s/it]
                                                               
{'loss': '0.7989', 'grad_norm': '0.9905', 'learning_rate': '1.778e-05', 'epoch': '0.6116', 'num_input_tokens_seen': 456277912, 'train_runtime': '9.9e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137600/225000 [275:00:34<169:46:25,  6.99s/it]
 61%|██████    | 137601/225000 [275:00:38<147:55:10,  6.09s/it]
 61%|██████    | 137602/225000 [275:00:42<130:40:05,  5.38s/it]
 61%|██████    | 137603/225000 [275:00:46<120:03:19,  4.95s/it]
 61%|██████    | 137604/225000 [275:00:57<166:15:58,  6.85s/it]
 61%|██████    | 137605/225000 [275:01:00<140:34:30,  5.79s/it]
 61%|██████    | 137606/225000 [275:01:04<126:26:56,  5.21s/it]
 61%|██████    | 137607/225000 [275:01:14<162:02:39,  6.68s/it]
 61%|██████    | 137608/225000 [275:01:18<138:49:43,  5.72s/it]
 61%|██████    | 137609/225000 [275:01:22<127:07:57,  5.24s/it]
 61%|██████    | 137610/225000 [275:01:26<117:58:22,  4.86s/it]
 61%|██████    | 137611/225000 [275:01:38<172:27:13,  7.10s/it]
 61%|██████    | 137612/225000 [275:01:41<144:51:50,  5.97s/it]
 61%|██████    | 137613/225000 [275:01:45<127:51:56,  5.27s/it]
 61%|██████    | 137614/225000 [275:01:49<119:43:03,  4.93s/it]
 61%|██████    | 137615/225000 [275:01:58<151:01:06,  6.22s/it]
 61%|██████    | 137616/225000 [275:02:04<148:43:13,  6.13s/it]
 61%|██████    | 137617/225000 [275:02:10<147:57:01,  6.10s/it]
 61%|██████    | 137618/225000 [275:02:15<133:56:02,  5.52s/it]
 61%|██████    | 137619/225000 [275:02:19<125:04:33,  5.15s/it]
 61%|██████    | 137620/225000 [275:02:24<123:45:28,  5.10s/it]
 61%|██████    | 137621/225000 [275:02:35<168:43:14,  6.95s/it]
 61%|██████    | 137622/225000 [275:02:39<143:14:00,  5.90s/it]
 61%|██████    | 137623/225000 [275:02:43<133:02:54,  5.48s/it]
 61%|██████    | 137624/225000 [275:02:48<130:39:26,  5.38s/it]
 61%|██████    | 137625/225000 [275:02:53<128:31:01,  5.30s/it]
 61%|██████    | 137626/225000 [275:02:58<126:35:29,  5.22s/it]
 61%|██████    | 137627/225000 [275:03:04<127:24:28,  5.25s/it]
 61%|██████    | 137628/225000 [275:03:09<125:14:42,  5.16s/it]
 61%|██████    | 137629/225000 [275:03:13<122:37:29,  5.05s/it]
 61%|██████    | 137630/225000 [275:03:25<166:42:04,  6.87s/it]
 61%|██████    | 137631/225000 [275:03:32<168:43:13,  6.95s/it]
 61%|██████    | 137632/225000 [275:03:36<152:47:50,  6.30s/it]
 61%|██████    | 137633/225000 [275:03:44<158:48:31,  6.54s/it]
 61%|██████    | 137634/225000 [275:03:48<142:16:31,  5.86s/it]
 61%|██████    | 137635/225000 [275:03:52<132:08:35,  5.45s/it]
 61%|██████    | 137636/225000 [275:03:57<126:59:12,  5.23s/it]
 61%|██████    | 137637/225000 [275:04:02<124:23:51,  5.13s/it]
 61%|██████    | 137638/225000 [275:04:15<182:20:07,  7.51s/it]
 61%|██████    | 137639/225000 [275:04:19<155:43:04,  6.42s/it]
 61%|██████    | 137640/225000 [275:04:24<142:42:27,  5.88s/it]
 61%|██████    | 137641/225000 [275:04:29<141:10:41,  5.82s/it]
 61%|██████    | 137642/225000 [275:04:35<140:28:34,  5.79s/it]
 61%|██████    | 137643/225000 [275:04:40<137:34:45,  5.67s/it]
 61%|██████    | 137644/225000 [275:04:47<147:57:41,  6.10s/it]
 61%|██████    | 137645/225000 [275:04:55<159:46:27,  6.58s/it]
 61%|██████    | 137646/225000 [275:04:59<142:16:43,  5.86s/it]
 61%|██████    | 137647/225000 [275:05:04<133:26:21,  5.50s/it]
 61%|██████    | 137648/225000 [275:05:14<165:04:13,  6.80s/it]
 61%|██████    | 137649/225000 [275:05:17<142:25:08,  5.87s/it]
 61%|██████    | 137650/225000 [275:05:24<143:59:28,  5.93s/it]
                                                               
{'loss': '0.8048', 'grad_norm': '1.111', 'learning_rate': '1.777e-05', 'epoch': '0.6118', 'num_input_tokens_seen': 456409736, 'train_runtime': '9.903e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137650/225000 [275:05:24<143:59:28,  5.93s/it]
 61%|██████    | 137651/225000 [275:05:32<161:31:46,  6.66s/it]
 61%|██████    | 137652/225000 [275:05:36<140:10:07,  5.78s/it]
 61%|██████    | 137653/225000 [275:05:41<134:06:40,  5.53s/it]
 61%|██████    | 137654/225000 [275:05:50<160:13:49,  6.60s/it]
 61%|██████    | 137655/225000 [275:05:55<147:24:02,  6.08s/it]
 61%|██████    | 137656/225000 [275:06:00<140:34:28,  5.79s/it]
 61%|██████    | 137657/225000 [275:06:05<136:16:46,  5.62s/it]
 61%|██████    | 137658/225000 [275:06:10<132:51:47,  5.48s/it]
 61%|██████    | 137659/225000 [275:06:15<126:56:46,  5.23s/it]
 61%|██████    | 137660/225000 [275:06:20<124:09:37,  5.12s/it]
 61%|██████    | 137661/225000 [275:06:25<123:09:41,  5.08s/it]
 61%|██████    | 137662/225000 [275:06:30<125:02:42,  5.15s/it]
 61%|██████    | 137663/225000 [275:06:35<123:26:12,  5.09s/it]
 61%|██████    | 137664/225000 [275:06:42<136:51:32,  5.64s/it]
 61%|██████    | 137665/225000 [275:06:55<189:18:12,  7.80s/it]
 61%|██████    | 137666/225000 [275:06:58<159:50:51,  6.59s/it]
 61%|██████    | 137667/225000 [275:07:02<140:04:31,  5.77s/it]
 61%|██████    | 137668/225000 [275:07:11<159:20:43,  6.57s/it]
 61%|██████    | 137669/225000 [275:07:15<144:24:15,  5.95s/it]
 61%|██████    | 137670/225000 [275:07:23<156:16:16,  6.44s/it]
 61%|██████    | 137671/225000 [275:07:27<138:18:43,  5.70s/it]
 61%|██████    | 137672/225000 [275:07:32<137:39:45,  5.67s/it]
 61%|██████    | 137673/225000 [275:07:39<148:05:42,  6.11s/it]
 61%|██████    | 137674/225000 [275:07:45<147:02:59,  6.06s/it]
 61%|██████    | 137675/225000 [275:07:51<142:08:42,  5.86s/it]
 61%|██████    | 137676/225000 [275:07:55<126:48:09,  5.23s/it]
 61%|██████    | 137677/225000 [275:07:59<124:39:09,  5.14s/it]
 61%|██████    | 137678/225000 [275:08:05<128:18:13,  5.29s/it]
 61%|██████    | 137679/225000 [275:08:10<128:36:47,  5.30s/it]
 61%|██████    | 137680/225000 [275:08:16<131:31:17,  5.42s/it]
 61%|██████    | 137681/225000 [275:08:21<128:24:59,  5.29s/it]
 61%|██████    | 137682/225000 [275:08:26<125:47:47,  5.19s/it]
 61%|██████    | 137683/225000 [275:08:32<130:05:31,  5.36s/it]
 61%|██████    | 137684/225000 [275:08:37<131:13:14,  5.41s/it]
 61%|██████    | 137685/225000 [275:08:42<127:45:05,  5.27s/it]
 61%|██████    | 137686/225000 [275:08:48<129:29:30,  5.34s/it]
 61%|██████    | 137687/225000 [275:08:57<155:59:10,  6.43s/it]
 61%|██████    | 137688/225000 [275:09:08<191:26:35,  7.89s/it]
 61%|██████    | 137689/225000 [275:09:12<161:09:40,  6.64s/it]
 61%|██████    | 137690/225000 [275:09:19<162:43:51,  6.71s/it]
 61%|██████    | 137691/225000 [275:09:23<142:09:47,  5.86s/it]
 61%|██████    | 137692/225000 [275:09:27<128:28:41,  5.30s/it]
 61%|██████    | 137693/225000 [275:09:31<119:22:36,  4.92s/it]
 61%|██████    | 137694/225000 [275:09:41<158:06:13,  6.52s/it]
 61%|██████    | 137695/225000 [275:09:47<155:28:34,  6.41s/it]
 61%|██████    | 137696/225000 [275:09:51<138:26:51,  5.71s/it]
 61%|██████    | 137697/225000 [275:09:59<151:56:34,  6.27s/it]
 61%|██████    | 137698/225000 [275:10:05<154:10:40,  6.36s/it]
 61%|██████    | 137699/225000 [275:10:10<144:55:03,  5.98s/it]
 61%|██████    | 137700/225000 [275:10:18<157:05:37,  6.48s/it]
                                                               
{'loss': '0.8085', 'grad_norm': '1.09', 'learning_rate': '1.775e-05', 'epoch': '0.612', 'num_input_tokens_seen': 456538952, 'train_runtime': '9.906e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137700/225000 [275:10:18<157:05:37,  6.48s/it]
 61%|██████    | 137701/225000 [275:10:22<136:31:56,  5.63s/it]
 61%|██████    | 137702/225000 [275:10:29<148:28:33,  6.12s/it]
 61%|██████    | 137703/225000 [275:10:35<149:39:27,  6.17s/it]
 61%|██████    | 137704/225000 [275:10:39<132:40:56,  5.47s/it]
 61%|██████    | 137705/225000 [275:10:43<121:44:14,  5.02s/it]
 61%|██████    | 137706/225000 [275:10:49<126:51:07,  5.23s/it]
 61%|██████    | 137707/225000 [275:10:55<137:13:20,  5.66s/it]
 61%|██████    | 137708/225000 [275:11:06<170:58:11,  7.05s/it]
 61%|██████    | 137709/225000 [275:11:09<147:24:07,  6.08s/it]
 61%|██████    | 137710/225000 [275:11:18<166:37:22,  6.87s/it]
 61%|██████    | 137711/225000 [275:11:22<143:25:57,  5.92s/it]
 61%|██████    | 137712/225000 [275:11:26<128:46:46,  5.31s/it]
 61%|██████    | 137713/225000 [275:11:35<161:12:48,  6.65s/it]
 61%|██████    | 137714/225000 [275:11:39<140:17:29,  5.79s/it]
 61%|██████    | 137715/225000 [275:11:43<124:45:23,  5.15s/it]
 61%|██████    | 137716/225000 [275:11:49<130:23:13,  5.38s/it]
 61%|██████    | 137717/225000 [275:11:58<158:08:31,  6.52s/it]
 61%|██████    | 137718/225000 [275:12:03<148:54:03,  6.14s/it]
 61%|██████    | 137719/225000 [275:12:10<156:08:11,  6.44s/it]
 61%|██████    | 137720/225000 [275:12:15<143:12:54,  5.91s/it]
 61%|██████    | 137721/225000 [275:12:20<138:12:18,  5.70s/it]
 61%|██████    | 137722/225000 [275:12:26<134:47:12,  5.56s/it]
 61%|██████    | 137723/225000 [275:12:31<130:52:26,  5.40s/it]
 61%|██████    | 137724/225000 [275:12:35<125:03:56,  5.16s/it]
 61%|██████    | 137725/225000 [275:12:40<124:18:18,  5.13s/it]
 61%|██████    | 137726/225000 [275:12:45<122:23:26,  5.05s/it]
 61%|██████    | 137727/225000 [275:12:50<122:51:58,  5.07s/it]
 61%|██████    | 137728/225000 [275:12:55<123:12:03,  5.08s/it]
 61%|██████    | 137729/225000 [275:13:00<122:33:28,  5.06s/it]
 61%|██████    | 137730/225000 [275:13:06<123:42:06,  5.10s/it]
 61%|██████    | 137731/225000 [275:13:16<164:06:23,  6.77s/it]
 61%|██████    | 137732/225000 [275:13:24<168:41:48,  6.96s/it]
 61%|██████    | 137733/225000 [275:13:28<153:35:19,  6.34s/it]
 61%|██████    | 137734/225000 [275:13:34<145:43:02,  6.01s/it]
 61%|██████    | 137735/225000 [275:13:39<139:48:26,  5.77s/it]
 61%|██████    | 137736/225000 [275:13:44<134:03:14,  5.53s/it]
 61%|██████    | 137737/225000 [275:13:49<130:03:48,  5.37s/it]
 61%|██████    | 137738/225000 [275:14:02<183:56:30,  7.59s/it]
 61%|██████    | 137739/225000 [275:14:06<158:40:27,  6.55s/it]
 61%|██████    | 137740/225000 [275:14:10<144:56:58,  5.98s/it]
 61%|██████    | 137741/225000 [275:14:15<136:06:23,  5.62s/it]
 61%|██████    | 137742/225000 [275:14:20<128:37:13,  5.31s/it]
 61%|██████    | 137743/225000 [275:14:24<124:22:51,  5.13s/it]
 61%|██████    | 137744/225000 [275:14:29<122:20:46,  5.05s/it]
 61%|██████    | 137745/225000 [275:14:39<154:32:44,  6.38s/it]
 61%|██████    | 137746/225000 [275:14:48<172:56:44,  7.14s/it]
 61%|██████    | 137747/225000 [275:14:52<152:15:54,  6.28s/it]
 61%|██████    | 137748/225000 [275:14:57<142:59:59,  5.90s/it]
 61%|██████    | 137749/225000 [275:15:04<154:05:06,  6.36s/it]
 61%|██████    | 137750/225000 [275:15:09<143:10:59,  5.91s/it]
                                                               
{'loss': '0.7969', 'grad_norm': '0.9715', 'learning_rate': '1.773e-05', 'epoch': '0.6122', 'num_input_tokens_seen': 456669592, 'train_runtime': '9.909e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137750/225000 [275:15:09<143:10:59,  5.91s/it]
 61%|██████    | 137751/225000 [275:15:14<132:16:36,  5.46s/it]
 61%|██████    | 137752/225000 [275:15:22<149:57:50,  6.19s/it]
 61%|██████    | 137753/225000 [275:15:28<153:54:54,  6.35s/it]
 61%|██████    | 137754/225000 [275:15:35<152:45:40,  6.30s/it]
 61%|██████    | 137755/225000 [275:15:41<153:29:24,  6.33s/it]
 61%|██████    | 137756/225000 [275:15:45<136:32:07,  5.63s/it]
 61%|██████    | 137757/225000 [275:15:49<124:16:08,  5.13s/it]
 61%|██████    | 137758/225000 [275:15:59<157:04:59,  6.48s/it]
 61%|██████    | 137759/225000 [275:16:03<145:45:40,  6.01s/it]
 61%|██████    | 137760/225000 [275:16:10<150:25:10,  6.21s/it]
 61%|██████    | 137761/225000 [275:16:15<142:25:43,  5.88s/it]
 61%|██████    | 137762/225000 [275:16:20<137:35:09,  5.68s/it]
 61%|██████    | 137763/225000 [275:16:25<132:35:37,  5.47s/it]
 61%|██████    | 137764/225000 [275:16:30<128:28:14,  5.30s/it]
 61%|██████    | 137765/225000 [275:16:35<126:54:45,  5.24s/it]
 61%|██████    | 137766/225000 [275:16:41<128:14:06,  5.29s/it]
 61%|██████    | 137767/225000 [275:16:46<128:43:26,  5.31s/it]
 61%|██████    | 137768/225000 [275:16:51<124:54:42,  5.16s/it]
 61%|██████    | 137769/225000 [275:16:56<124:31:21,  5.14s/it]
 61%|██████    | 137770/225000 [275:17:01<122:31:33,  5.06s/it]
 61%|██████    | 137771/225000 [275:17:06<123:41:53,  5.11s/it]
 61%|██████    | 137772/225000 [275:17:12<125:51:20,  5.19s/it]
 61%|██████    | 137773/225000 [275:17:17<127:09:36,  5.25s/it]
 61%|██████    | 137774/225000 [275:17:22<128:19:17,  5.30s/it]
 61%|██████    | 137775/225000 [275:17:28<129:00:33,  5.32s/it]
 61%|██████    | 137776/225000 [275:17:33<129:18:51,  5.34s/it]
 61%|██████    | 137777/225000 [275:17:38<127:51:29,  5.28s/it]
 61%|██████    | 137778/225000 [275:17:46<144:11:34,  5.95s/it]
 61%|██████    | 137779/225000 [275:17:59<194:20:37,  8.02s/it]
 61%|██████    | 137780/225000 [275:18:02<162:22:09,  6.70s/it]
 61%|██████    | 137781/225000 [275:18:09<166:08:43,  6.86s/it]
 61%|██████    | 137782/225000 [275:18:13<140:10:33,  5.79s/it]
 61%|██████    | 137783/225000 [275:18:21<157:04:18,  6.48s/it]
 61%|██████    | 137784/225000 [275:18:24<134:26:47,  5.55s/it]
 61%|██████    | 137785/225000 [275:18:33<155:21:52,  6.41s/it]
 61%|██████    | 137786/225000 [275:18:36<131:59:24,  5.45s/it]
 61%|██████    | 137787/225000 [275:18:41<130:59:43,  5.41s/it]
 61%|██████    | 137788/225000 [275:18:49<150:07:28,  6.20s/it]
 61%|██████    | 137789/225000 [275:18:53<132:47:42,  5.48s/it]
 61%|██████    | 137790/225000 [275:19:00<143:34:08,  5.93s/it]
 61%|██████    | 137791/225000 [275:19:07<150:46:10,  6.22s/it]
 61%|██████    | 137792/225000 [275:19:12<145:32:05,  6.01s/it]
 61%|██████    | 137793/225000 [275:19:17<133:00:32,  5.49s/it]
 61%|██████    | 137794/225000 [275:19:23<139:21:32,  5.75s/it]
 61%|██████    | 137795/225000 [275:19:28<135:25:40,  5.59s/it]
 61%|██████    | 137796/225000 [275:19:36<151:25:17,  6.25s/it]
 61%|██████    | 137797/225000 [275:19:39<130:51:49,  5.40s/it]
 61%|██████    | 137798/225000 [275:19:46<140:37:31,  5.81s/it]
 61%|██████    | 137799/225000 [275:19:50<129:04:04,  5.33s/it]
 61%|██████    | 137800/225000 [275:19:58<142:26:24,  5.88s/it]
                                                               
{'loss': '0.7951', 'grad_norm': '1.018', 'learning_rate': '1.771e-05', 'epoch': '0.6124', 'num_input_tokens_seen': 456799640, 'train_runtime': '9.912e+05', 'train_tokens_per_second': '460.9'}

 61%|██████    | 137800/225000 [275:19:58<142:26:24,  5.88s/it]
 61%|██████    | 137801/225000 [275:20:01<123:19:37,  5.09s/it]
 61%|██████    | 137802/225000 [275:20:10<152:59:25,  6.32s/it]
 61%|██████    | 137803/225000 [275:20:13<130:56:11,  5.41s/it]
 61%|██████    | 137804/225000 [275:20:20<143:38:06,  5.93s/it]
 61%|██████    | 137805/225000 [275:20:25<135:30:47,  5.59s/it]
 61%|██████    | 137806/225000 [275:20:29<121:04:07,  5.00s/it]
 61%|██████    | 137807/225000 [275:20:37<141:50:42,  5.86s/it]
 61%|██████    | 137808/225000 [275:20:42<136:51:23,  5.65s/it]
 61%|██████    | 137809/225000 [275:20:48<138:24:21,  5.71s/it]
 61%|██████    | 137810/225000 [275:20:54<144:12:24,  5.95s/it]
 61%|██████    | 137811/225000 [275:20:59<132:04:22,  5.45s/it]
 61%|██████    | 137812/225000 [275:21:03<125:28:19,  5.18s/it]
 61%|██████▏   | 137813/225000 [275:21:08<122:49:37,  5.07s/it]
 61%|██████▏   | 137814/225000 [275:21:13<119:52:22,  4.95s/it]
 61%|██████▏   | 137815/225000 [275:21:23<156:30:21,  6.46s/it]
 61%|██████▏   | 137816/225000 [275:21:31<167:51:43,  6.93s/it]
 61%|██████▏   | 137817/225000 [275:21:34<138:55:18,  5.74s/it]
 61%|██████▏   | 137818/225000 [275:21:43<162:51:10,  6.72s/it]
 61%|██████▏   | 137819/225000 [275:21:46<138:34:40,  5.72s/it]
 61%|██████▏   | 137820/225000 [275:21:51<131:50:45,  5.44s/it]
 61%|██████▏   | 137821/225000 [275:21:58<143:58:19,  5.95s/it]
 61%|██████▏   | 137822/225000 [275:22:02<129:48:33,  5.36s/it]
 61%|██████▏   | 137823/225000 [275:22:06<123:29:27,  5.10s/it]
 61%|██████▏   | 137824/225000 [275:22:15<150:32:36,  6.22s/it]
 61%|██████▏   | 137825/225000 [275:22:19<130:24:00,  5.39s/it]
 61%|██████▏   | 137826/225000 [275:22:23<122:34:39,  5.06s/it]
 61%|██████▏   | 137827/225000 [275:22:32<149:48:35,  6.19s/it]
 61%|██████▏   | 137828/225000 [275:22:39<156:30:42,  6.46s/it]
 61%|██████▏   | 137829/225000 [275:22:42<134:33:52,  5.56s/it]
 61%|██████▏   | 137830/225000 [275:22:52<164:04:56,  6.78s/it]
 61%|██████▏   | 137831/225000 [275:22:57<149:09:15,  6.16s/it]
 61%|██████▏   | 137832/225000 [275:23:01<134:49:05,  5.57s/it]
 61%|██████▏   | 137833/225000 [275:23:06<134:38:08,  5.56s/it]
 61%|██████▏   | 137834/225000 [275:23:12<133:52:27,  5.53s/it]
 61%|██████▏   | 137835/225000 [275:23:18<141:19:19,  5.84s/it]
 61%|██████▏   | 137836/225000 [275:23:25<147:21:25,  6.09s/it]
 61%|██████▏   | 137837/225000 [275:23:29<134:58:14,  5.57s/it]
 61%|██████▏   | 137838/225000 [275:23:34<124:51:55,  5.16s/it]
 61%|██████▏   | 137839/225000 [275:23:38<120:13:40,  4.97s/it]
 61%|██████▏   | 137840/225000 [275:23:42<115:15:12,  4.76s/it]
 61%|██████▏   | 137841/225000 [275:23:51<144:26:07,  5.97s/it]
 61%|██████▏   | 137842/225000 [275:24:00<162:08:10,  6.70s/it]
 61%|██████▏   | 137843/225000 [275:24:03<137:20:07,  5.67s/it]
 61%|██████▏   | 137844/225000 [275:24:11<153:54:36,  6.36s/it]
 61%|██████▏   | 137845/225000 [275:24:16<144:36:10,  5.97s/it]
 61%|██████▏   | 137846/225000 [275:24:20<130:40:39,  5.40s/it]
 61%|██████▏   | 137847/225000 [275:24:27<143:21:08,  5.92s/it]
 61%|██████▏   | 137848/225000 [275:24:31<127:36:33,  5.27s/it]
 61%|██████▏   | 137849/225000 [275:24:41<159:27:32,  6.59s/it]
 61%|██████▏   | 137850/225000 [275:24:44<135:06:52,  5.58s/it]
                                                               
{'loss': '0.7975', 'grad_norm': '1.025', 'learning_rate': '1.77e-05', 'epoch': '0.6127', 'num_input_tokens_seen': 456927968, 'train_runtime': '9.915e+05', 'train_tokens_per_second': '460.9'}

 61%|██████▏   | 137850/225000 [275:24:44<135:06:52,  5.58s/it]
 61%|██████▏   | 137851/225000 [275:24:51<147:53:22,  6.11s/it]
 61%|██████▏   | 137852/225000 [275:24:56<142:10:52,  5.87s/it]
 61%|██████▏   | 137853/225000 [275:25:01<133:25:53,  5.51s/it]
 61%|██████▏   | 137854/225000 [275:25:06<126:50:16,  5.24s/it]
 61%|██████▏   | 137855/225000 [275:25:10<122:55:10,  5.08s/it]
 61%|██████▏   | 137856/225000 [275:25:15<118:09:12,  4.88s/it]
 61%|██████▏   | 137857/225000 [275:25:20<118:51:52,  4.91s/it]
 61%|██████▏   | 137858/225000 [275:25:25<121:42:14,  5.03s/it]
 61%|██████▏   | 137859/225000 [275:25:30<122:42:33,  5.07s/it]
 61%|██████▏   | 137860/225000 [275:25:35<121:31:59,  5.02s/it]
 61%|██████▏   | 137861/225000 [275:25:47<173:39:00,  7.17s/it]
 61%|██████▏   | 137862/225000 [275:25:53<161:14:30,  6.66s/it]
 61%|██████▏   | 137863/225000 [275:25:58<147:28:00,  6.09s/it]
 61%|██████▏   | 137864/225000 [275:26:03<140:32:59,  5.81s/it]
 61%|██████▏   | 137865/225000 [275:26:08<134:08:44,  5.54s/it]
 61%|██████▏   | 137866/225000 [275:26:13<129:56:37,  5.37s/it]
 61%|██████▏   | 137867/225000 [275:26:25<177:24:34,  7.33s/it]
 61%|██████▏   | 137868/225000 [275:26:28<149:48:39,  6.19s/it]
 61%|██████▏   | 137869/225000 [275:26:37<168:07:26,  6.95s/it]
 61%|██████▏   | 137870/225000 [275:26:41<150:14:44,  6.21s/it]
 61%|██████▏   | 137871/225000 [275:26:45<134:37:52,  5.56s/it]
 61%|██████▏   | 137872/225000 [275:26:51<135:48:22,  5.61s/it]
 61%|██████▏   | 137873/225000 [275:27:02<173:23:11,  7.16s/it]
 61%|██████▏   | 137874/225000 [275:27:06<150:22:13,  6.21s/it]
 61%|██████▏   | 137875/225000 [275:27:10<136:55:28,  5.66s/it]
 61%|██████▏   | 137876/225000 [275:27:14<123:06:39,  5.09s/it]
 61%|██████▏   | 137877/225000 [275:27:24<162:12:59,  6.70s/it]
 61%|██████▏   | 137878/225000 [275:27:30<156:52:24,  6.48s/it]
 61%|██████▏   | 137879/225000 [275:27:34<135:54:16,  5.62s/it]
 61%|██████▏   | 137880/225000 [275:27:43<161:06:52,  6.66s/it]
 61%|██████▏   | 137881/225000 [275:27:47<138:36:52,  5.73s/it]
 61%|██████▏   | 137882/225000 [275:27:53<146:41:55,  6.06s/it]
 61%|██████▏   | 137883/225000 [275:27:57<128:53:50,  5.33s/it]
 61%|██████▏   | 137884/225000 [275:28:06<157:23:02,  6.50s/it]
 61%|██████▏   | 137885/225000 [275:28:09<130:55:41,  5.41s/it]
 61%|██████▏   | 137886/225000 [275:28:15<136:07:46,  5.63s/it]
 61%|██████▏   | 137887/225000 [275:28:21<133:04:54,  5.50s/it]
 61%|██████▏   | 137888/225000 [275:28:28<146:54:08,  6.07s/it]
 61%|██████▏   | 137889/225000 [275:28:33<141:06:12,  5.83s/it]
 61%|██████▏   | 137890/225000 [275:28:42<160:20:32,  6.63s/it]
 61%|██████▏   | 137891/225000 [275:28:45<139:28:46,  5.76s/it]
 61%|██████▏   | 137892/225000 [275:28:49<125:13:48,  5.18s/it]
 61%|██████▏   | 137893/225000 [275:28:57<146:47:23,  6.07s/it]
 61%|██████▏   | 137894/225000 [275:29:04<149:21:23,  6.17s/it]
 61%|██████▏   | 137895/225000 [275:29:07<130:26:04,  5.39s/it]
 61%|██████▏   | 137896/225000 [275:29:16<157:28:32,  6.51s/it]
 61%|██████▏   | 137897/225000 [275:29:21<142:40:11,  5.90s/it]
 61%|██████▏   | 137898/225000 [275:29:25<131:56:16,  5.45s/it]
 61%|██████▏   | 137899/225000 [275:29:29<121:12:09,  5.01s/it]
 61%|██████▏   | 137900/225000 [275:29:34<117:21:29,  4.85s/it]
                                                               
{'loss': '0.7987', 'grad_norm': '1.257', 'learning_rate': '1.768e-05', 'epoch': '0.6129', 'num_input_tokens_seen': 457057880, 'train_runtime': '9.918e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 137900/225000 [275:29:34<117:21:29,  4.85s/it]
 61%|██████▏   | 137901/225000 [275:29:45<165:00:54,  6.82s/it]
 61%|██████▏   | 137902/225000 [275:29:51<159:43:06,  6.60s/it]
 61%|██████▏   | 137903/225000 [275:29:56<145:08:23,  6.00s/it]
 61%|██████▏   | 137904/225000 [275:30:02<143:53:50,  5.95s/it]
 61%|██████▏   | 137905/225000 [275:30:06<130:17:59,  5.39s/it]
 61%|██████▏   | 137906/225000 [275:30:10<122:09:24,  5.05s/it]
 61%|██████▏   | 137907/225000 [275:30:15<123:16:36,  5.10s/it]
 61%|██████▏   | 137908/225000 [275:30:20<121:39:53,  5.03s/it]
 61%|██████▏   | 137909/225000 [275:30:26<124:18:00,  5.14s/it]
 61%|██████▏   | 137910/225000 [275:30:31<123:42:31,  5.11s/it]
 61%|██████▏   | 137911/225000 [275:30:36<126:02:38,  5.21s/it]
 61%|██████▏   | 137912/225000 [275:30:48<172:32:30,  7.13s/it]
 61%|██████▏   | 137913/225000 [275:30:52<153:33:26,  6.35s/it]
 61%|██████▏   | 137914/225000 [275:30:59<154:21:39,  6.38s/it]
 61%|██████▏   | 137915/225000 [275:31:06<160:33:18,  6.64s/it]
 61%|██████▏   | 137916/225000 [275:31:10<142:20:12,  5.88s/it]
 61%|██████▏   | 137917/225000 [275:31:14<129:41:50,  5.36s/it]
 61%|██████▏   | 137918/225000 [275:31:21<142:22:05,  5.89s/it]
 61%|██████▏   | 137919/225000 [275:31:28<150:32:09,  6.22s/it]
 61%|██████▏   | 137920/225000 [275:31:32<130:33:33,  5.40s/it]
 61%|██████▏   | 137921/225000 [275:31:38<135:02:40,  5.58s/it]
 61%|██████▏   | 137922/225000 [275:31:44<141:49:36,  5.86s/it]
 61%|██████▏   | 137923/225000 [275:31:50<138:59:11,  5.75s/it]
 61%|██████▏   | 137924/225000 [275:31:58<155:01:03,  6.41s/it]
 61%|██████▏   | 137925/225000 [275:32:03<144:21:06,  5.97s/it]
 61%|██████▏   | 137926/225000 [275:32:09<144:40:02,  5.98s/it]
 61%|██████▏   | 137927/225000 [275:32:13<135:09:24,  5.59s/it]
 61%|██████▏   | 137928/225000 [275:32:18<130:36:00,  5.40s/it]
 61%|██████▏   | 137929/225000 [275:32:23<126:54:40,  5.25s/it]
 61%|██████▏   | 137930/225000 [275:32:28<123:53:19,  5.12s/it]
 61%|██████▏   | 137931/225000 [275:32:33<121:24:26,  5.02s/it]
 61%|██████▏   | 137932/225000 [275:32:37<117:46:22,  4.87s/it]
 61%|██████▏   | 137933/225000 [275:32:42<118:07:32,  4.88s/it]
 61%|██████▏   | 137934/225000 [275:32:54<171:15:33,  7.08s/it]
 61%|██████▏   | 137935/225000 [275:33:00<157:24:04,  6.51s/it]
 61%|██████▏   | 137936/225000 [275:33:07<165:47:03,  6.85s/it]
 61%|██████▏   | 137937/225000 [275:33:11<144:10:16,  5.96s/it]
 61%|██████▏   | 137938/225000 [275:33:16<132:49:21,  5.49s/it]
 61%|██████▏   | 137939/225000 [275:33:19<119:58:30,  4.96s/it]
 61%|██████▏   | 137940/225000 [275:33:30<164:14:29,  6.79s/it]
 61%|██████▏   | 137941/225000 [275:33:34<141:19:34,  5.84s/it]
 61%|██████▏   | 137942/225000 [275:33:41<149:19:27,  6.17s/it]
 61%|██████▏   | 137943/225000 [275:33:45<134:42:21,  5.57s/it]
 61%|██████▏   | 137944/225000 [275:33:49<126:08:13,  5.22s/it]
 61%|██████▏   | 137945/225000 [275:33:54<121:10:33,  5.01s/it]
 61%|██████▏   | 137946/225000 [275:33:59<123:30:30,  5.11s/it]
 61%|██████▏   | 137947/225000 [275:34:11<170:50:03,  7.06s/it]
 61%|██████▏   | 137948/225000 [275:34:14<142:18:48,  5.89s/it]
 61%|██████▏   | 137949/225000 [275:34:23<166:31:10,  6.89s/it]
 61%|██████▏   | 137950/225000 [275:34:27<140:52:56,  5.83s/it]
                                                               
{'loss': '0.7992', 'grad_norm': '1.065', 'learning_rate': '1.766e-05', 'epoch': '0.6131', 'num_input_tokens_seen': 457185944, 'train_runtime': '9.921e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 137950/225000 [275:34:27<140:52:56,  5.83s/it]
 61%|██████▏   | 137951/225000 [275:34:34<151:52:59,  6.28s/it]
 61%|██████▏   | 137952/225000 [275:34:38<137:21:21,  5.68s/it]
 61%|██████▏   | 137953/225000 [275:34:44<134:12:05,  5.55s/it]
 61%|██████▏   | 137954/225000 [275:34:49<132:26:26,  5.48s/it]
 61%|██████▏   | 137955/225000 [275:34:54<130:42:56,  5.41s/it]
 61%|██████▏   | 137956/225000 [275:34:59<126:31:29,  5.23s/it]
 61%|██████▏   | 137957/225000 [275:35:04<128:10:01,  5.30s/it]
 61%|██████▏   | 137958/225000 [275:35:10<127:43:34,  5.28s/it]
 61%|██████▏   | 137959/225000 [275:35:15<129:50:06,  5.37s/it]
 61%|██████▏   | 137960/225000 [275:35:21<129:50:34,  5.37s/it]
 61%|██████▏   | 137961/225000 [275:35:26<128:13:55,  5.30s/it]
 61%|██████▏   | 137962/225000 [275:35:31<128:35:03,  5.32s/it]
 61%|██████▏   | 137963/225000 [275:35:36<128:26:30,  5.31s/it]
 61%|██████▏   | 137964/225000 [275:35:44<148:41:11,  6.15s/it]
 61%|██████▏   | 137965/225000 [275:35:56<188:32:51,  7.80s/it]
 61%|██████▏   | 137966/225000 [275:36:00<158:45:11,  6.57s/it]
 61%|██████▏   | 137967/225000 [275:36:06<155:02:18,  6.41s/it]
 61%|██████▏   | 137968/225000 [275:36:13<158:30:29,  6.56s/it]
 61%|██████▏   | 137969/225000 [275:36:16<136:05:53,  5.63s/it]
 61%|██████▏   | 137970/225000 [275:36:26<163:33:41,  6.77s/it]
 61%|██████▏   | 137971/225000 [275:36:29<141:06:03,  5.84s/it]
 61%|██████▏   | 137972/225000 [275:36:33<126:02:58,  5.21s/it]
 61%|██████▏   | 137973/225000 [275:36:41<146:46:06,  6.07s/it]
 61%|██████▏   | 137974/225000 [275:36:45<129:12:05,  5.34s/it]
 61%|██████▏   | 137975/225000 [275:36:52<142:20:16,  5.89s/it]
 61%|██████▏   | 137976/225000 [275:36:58<143:13:33,  5.92s/it]
 61%|██████▏   | 137977/225000 [275:37:01<125:08:57,  5.18s/it]
 61%|██████▏   | 137978/225000 [275:37:06<118:52:21,  4.92s/it]
 61%|██████▏   | 137979/225000 [275:37:10<114:59:02,  4.76s/it]
 61%|██████▏   | 137980/225000 [275:37:21<161:00:29,  6.66s/it]
 61%|██████▏   | 137981/225000 [275:37:26<145:49:03,  6.03s/it]
 61%|██████▏   | 137982/225000 [275:37:30<135:15:47,  5.60s/it]
 61%|██████▏   | 137983/225000 [275:37:35<127:42:26,  5.28s/it]
 61%|██████▏   | 137984/225000 [275:37:40<123:31:53,  5.11s/it]
 61%|██████▏   | 137985/225000 [275:37:47<143:32:24,  5.94s/it]
 61%|██████▏   | 137986/225000 [275:37:56<161:41:00,  6.69s/it]
 61%|██████▏   | 137987/225000 [275:38:00<142:33:48,  5.90s/it]
 61%|██████▏   | 137988/225000 [275:38:04<129:30:55,  5.36s/it]
 61%|██████▏   | 137989/225000 [275:38:11<144:34:46,  5.98s/it]
 61%|██████▏   | 137990/225000 [275:38:20<165:17:05,  6.84s/it]
 61%|██████▏   | 137991/225000 [275:38:25<148:12:08,  6.13s/it]
 61%|██████▏   | 137992/225000 [275:38:29<136:27:54,  5.65s/it]
 61%|██████▏   | 137993/225000 [275:38:34<128:00:53,  5.30s/it]
 61%|██████▏   | 137994/225000 [275:38:38<122:00:53,  5.05s/it]
 61%|██████▏   | 137995/225000 [275:38:49<163:22:57,  6.76s/it]
 61%|██████▏   | 137996/225000 [275:38:55<156:19:06,  6.47s/it]
 61%|██████▏   | 137997/225000 [275:38:59<136:24:19,  5.64s/it]
 61%|██████▏   | 137998/225000 [275:39:06<149:34:36,  6.19s/it]
 61%|██████▏   | 137999/225000 [275:39:10<131:35:10,  5.44s/it]
 61%|██████▏   | 138000/225000 [275:39:17<142:31:01,  5.90s/it]
                                                               
{'loss': '0.8062', 'grad_norm': '1.118', 'learning_rate': '1.764e-05', 'epoch': '0.6133', 'num_input_tokens_seen': 457316464, 'train_runtime': '9.924e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138000/225000 [275:39:17<142:31:01,  5.90s/it]
 61%|██████▏   | 138001/225000 [275:39:23<146:08:58,  6.05s/it]
 61%|██████▏   | 138002/225000 [275:39:31<162:35:59,  6.73s/it]
 61%|██████▏   | 138003/225000 [275:39:35<138:00:15,  5.71s/it]
 61%|██████▏   | 138004/225000 [275:39:38<122:42:42,  5.08s/it]
 61%|██████▏   | 138005/225000 [275:39:47<150:52:08,  6.24s/it]
 61%|██████▏   | 138006/225000 [275:39:51<132:35:55,  5.49s/it]
 61%|██████▏   | 138007/225000 [275:39:54<118:06:28,  4.89s/it]
 61%|██████▏   | 138008/225000 [275:39:58<111:02:05,  4.59s/it]
 61%|██████▏   | 138009/225000 [275:40:09<157:06:56,  6.50s/it]
 61%|██████▏   | 138010/225000 [275:40:13<139:57:10,  5.79s/it]
 61%|██████▏   | 138011/225000 [275:40:17<124:48:13,  5.16s/it]
 61%|██████▏   | 138012/225000 [275:40:26<149:04:20,  6.17s/it]
 61%|██████▏   | 138013/225000 [275:40:31<141:12:06,  5.84s/it]
 61%|██████▏   | 138014/225000 [275:40:35<129:19:42,  5.35s/it]
 61%|██████▏   | 138015/225000 [275:40:39<120:21:11,  4.98s/it]
 61%|██████▏   | 138016/225000 [275:40:44<119:43:07,  4.95s/it]
 61%|██████▏   | 138017/225000 [275:40:55<164:49:55,  6.82s/it]
 61%|██████▏   | 138018/225000 [275:40:59<145:35:15,  6.03s/it]
 61%|██████▏   | 138019/225000 [275:41:03<131:06:26,  5.43s/it]
 61%|██████▏   | 138020/225000 [275:41:09<129:49:06,  5.37s/it]
 61%|██████▏   | 138021/225000 [275:41:18<156:28:02,  6.48s/it]
 61%|██████▏   | 138022/225000 [275:41:21<136:03:26,  5.63s/it]
 61%|██████▏   | 138023/225000 [275:41:25<120:11:28,  4.97s/it]
 61%|██████▏   | 138024/225000 [275:41:28<109:13:42,  4.52s/it]
 61%|██████▏   | 138025/225000 [275:41:37<138:09:02,  5.72s/it]
 61%|██████▏   | 138026/225000 [275:41:43<145:10:54,  6.01s/it]
 61%|██████▏   | 138027/225000 [275:41:47<126:30:46,  5.24s/it]
 61%|██████▏   | 138028/225000 [275:41:54<143:51:29,  5.95s/it]
 61%|██████▏   | 138029/225000 [275:42:01<146:16:04,  6.05s/it]
 61%|██████▏   | 138030/225000 [275:42:04<128:20:09,  5.31s/it]
 61%|██████▏   | 138031/225000 [275:42:13<155:39:15,  6.44s/it]
 61%|██████▏   | 138032/225000 [275:42:17<134:42:54,  5.58s/it]
 61%|██████▏   | 138033/225000 [275:42:21<123:34:18,  5.12s/it]
 61%|██████▏   | 138034/225000 [275:42:26<121:34:05,  5.03s/it]
 61%|██████▏   | 138035/225000 [275:42:36<157:51:08,  6.53s/it]
 61%|██████▏   | 138036/225000 [275:42:40<142:24:16,  5.90s/it]
 61%|██████▏   | 138037/225000 [275:42:48<153:01:39,  6.33s/it]
 61%|██████▏   | 138038/225000 [275:42:54<153:40:08,  6.36s/it]
 61%|██████▏   | 138039/225000 [275:42:58<136:37:34,  5.66s/it]
 61%|██████▏   | 138040/225000 [275:43:02<124:54:15,  5.17s/it]
 61%|██████▏   | 138041/225000 [275:43:06<118:37:32,  4.91s/it]
 61%|██████▏   | 138042/225000 [275:43:18<163:17:27,  6.76s/it]
 61%|██████▏   | 138043/225000 [275:43:21<142:57:15,  5.92s/it]
 61%|██████▏   | 138044/225000 [275:43:25<128:24:55,  5.32s/it]
 61%|██████▏   | 138045/225000 [275:43:29<117:35:50,  4.87s/it]
 61%|██████▏   | 138046/225000 [275:43:34<117:08:51,  4.85s/it]
 61%|██████▏   | 138047/225000 [275:43:45<158:44:06,  6.57s/it]
 61%|██████▏   | 138048/225000 [275:43:48<139:06:09,  5.76s/it]
 61%|██████▏   | 138049/225000 [275:43:53<128:50:00,  5.33s/it]
 61%|██████▏   | 138050/225000 [275:44:03<161:55:21,  6.70s/it]
                                                               
{'loss': '0.8', 'grad_norm': '1.001', 'learning_rate': '1.763e-05', 'epoch': '0.6136', 'num_input_tokens_seen': 457445048, 'train_runtime': '9.926e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138050/225000 [275:44:03<161:55:21,  6.70s/it]
 61%|██████▏   | 138051/225000 [275:44:06<138:15:59,  5.72s/it]
 61%|██████▏   | 138052/225000 [275:44:15<157:16:37,  6.51s/it]
 61%|██████▏   | 138053/225000 [275:44:18<138:02:01,  5.72s/it]
 61%|██████▏   | 138054/225000 [275:44:22<124:56:58,  5.17s/it]
 61%|██████▏   | 138055/225000 [275:44:26<116:07:37,  4.81s/it]
 61%|██████▏   | 138056/225000 [275:44:36<152:58:52,  6.33s/it]
 61%|██████▏   | 138057/225000 [275:44:41<138:55:40,  5.75s/it]
 61%|██████▏   | 138058/225000 [275:44:45<131:09:37,  5.43s/it]
 61%|██████▏   | 138059/225000 [275:44:49<119:06:06,  4.93s/it]
 61%|██████▏   | 138060/225000 [275:44:54<116:28:17,  4.82s/it]
 61%|██████▏   | 138061/225000 [275:45:01<133:27:05,  5.53s/it]
 61%|██████▏   | 138062/225000 [275:45:11<169:49:34,  7.03s/it]
 61%|██████▏   | 138063/225000 [275:45:16<153:22:37,  6.35s/it]
 61%|██████▏   | 138064/225000 [275:45:20<139:18:34,  5.77s/it]
 61%|██████▏   | 138065/225000 [275:45:24<125:45:41,  5.21s/it]
 61%|██████▏   | 138066/225000 [275:45:36<171:09:45,  7.09s/it]
 61%|██████▏   | 138067/225000 [275:45:39<146:03:33,  6.05s/it]
 61%|██████▏   | 138068/225000 [275:45:43<127:25:15,  5.28s/it]
 61%|██████▏   | 138069/225000 [275:45:50<139:04:04,  5.76s/it]
 61%|██████▏   | 138070/225000 [275:45:57<152:10:42,  6.30s/it]
 61%|██████▏   | 138071/225000 [275:46:01<132:03:09,  5.47s/it]
 61%|██████▏   | 138072/225000 [275:46:10<161:01:13,  6.67s/it]
 61%|██████▏   | 138073/225000 [275:46:13<135:00:11,  5.59s/it]
 61%|██████▏   | 138074/225000 [275:46:20<141:05:24,  5.84s/it]
 61%|██████▏   | 138075/225000 [275:46:23<121:50:47,  5.05s/it]
 61%|██████▏   | 138076/225000 [275:46:32<147:47:06,  6.12s/it]
 61%|██████▏   | 138077/225000 [275:46:35<127:14:25,  5.27s/it]
 61%|██████▏   | 138078/225000 [275:46:41<133:07:44,  5.51s/it]
 61%|██████▏   | 138079/225000 [275:46:48<142:04:22,  5.88s/it]
 61%|██████▏   | 138080/225000 [275:46:52<133:16:57,  5.52s/it]
 61%|██████▏   | 138081/225000 [275:46:57<128:12:42,  5.31s/it]
 61%|██████▏   | 138082/225000 [275:47:02<126:15:18,  5.23s/it]
 61%|██████▏   | 138083/225000 [275:47:08<127:05:05,  5.26s/it]
 61%|██████▏   | 138084/225000 [275:47:12<122:58:52,  5.09s/it]
 61%|██████▏   | 138085/225000 [275:47:17<121:08:56,  5.02s/it]
 61%|██████▏   | 138086/225000 [275:47:22<121:33:06,  5.03s/it]
 61%|██████▏   | 138087/225000 [275:47:27<116:38:25,  4.83s/it]
 61%|██████▏   | 138088/225000 [275:47:35<138:42:23,  5.75s/it]
 61%|██████▏   | 138089/225000 [275:47:46<182:57:34,  7.58s/it]
 61%|██████▏   | 138090/225000 [275:47:50<154:21:18,  6.39s/it]
 61%|██████▏   | 138091/225000 [275:47:53<132:37:05,  5.49s/it]
 61%|██████▏   | 138092/225000 [275:47:58<125:52:31,  5.21s/it]
 61%|██████▏   | 138093/225000 [275:48:03<125:30:43,  5.20s/it]
 61%|██████▏   | 138094/225000 [275:48:08<125:08:34,  5.18s/it]
 61%|██████▏   | 138095/225000 [275:48:13<123:18:56,  5.11s/it]
 61%|██████▏   | 138096/225000 [275:48:18<124:39:24,  5.16s/it]
 61%|██████▏   | 138097/225000 [275:48:24<123:54:52,  5.13s/it]
 61%|██████▏   | 138098/225000 [275:48:34<159:36:10,  6.61s/it]
 61%|██████▏   | 138099/225000 [275:48:41<164:38:04,  6.82s/it]
 61%|██████▏   | 138100/225000 [275:48:44<139:39:01,  5.79s/it]
                                                               
{'loss': '0.8007', 'grad_norm': '1.128', 'learning_rate': '1.761e-05', 'epoch': '0.6138', 'num_input_tokens_seen': 457574912, 'train_runtime': '9.929e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138100/225000 [275:48:44<139:39:01,  5.79s/it]
 61%|██████▏   | 138101/225000 [275:48:54<166:32:05,  6.90s/it]
 61%|██████▏   | 138102/225000 [275:48:58<145:11:18,  6.01s/it]
 61%|██████▏   | 138103/225000 [275:49:02<129:38:00,  5.37s/it]
 61%|██████▏   | 138104/225000 [275:49:06<121:47:30,  5.05s/it]
 61%|██████▏   | 138105/225000 [275:49:17<165:51:15,  6.87s/it]
 61%|██████▏   | 138106/225000 [275:49:20<138:37:40,  5.74s/it]
 61%|██████▏   | 138107/225000 [275:49:29<159:50:42,  6.62s/it]
 61%|██████▏   | 138108/225000 [275:49:32<136:11:25,  5.64s/it]
 61%|██████▏   | 138109/225000 [275:49:40<150:08:14,  6.22s/it]
 61%|██████▏   | 138110/225000 [275:49:44<134:31:55,  5.57s/it]
 61%|██████▏   | 138111/225000 [275:49:52<150:27:31,  6.23s/it]
 61%|██████▏   | 138112/225000 [275:49:56<140:14:05,  5.81s/it]
 61%|██████▏   | 138113/225000 [275:50:02<137:19:40,  5.69s/it]
 61%|██████▏   | 138114/225000 [275:50:06<129:51:50,  5.38s/it]
 61%|██████▏   | 138115/225000 [275:50:12<127:25:14,  5.28s/it]
 61%|██████▏   | 138116/225000 [275:50:17<126:34:19,  5.24s/it]
 61%|██████▏   | 138117/225000 [275:50:21<123:32:24,  5.12s/it]
 61%|██████▏   | 138118/225000 [275:50:27<124:12:49,  5.15s/it]
 61%|██████▏   | 138119/225000 [275:50:32<125:11:47,  5.19s/it]
 61%|██████▏   | 138120/225000 [275:50:37<122:44:44,  5.09s/it]
 61%|██████▏   | 138121/225000 [275:50:42<121:35:43,  5.04s/it]
 61%|██████▏   | 138122/225000 [275:50:46<118:05:07,  4.89s/it]
 61%|██████▏   | 138123/225000 [275:50:51<117:59:03,  4.89s/it]
 61%|██████▏   | 138124/225000 [275:50:57<121:14:19,  5.02s/it]
 61%|██████▏   | 138125/225000 [275:51:09<175:20:16,  7.27s/it]
 61%|██████▏   | 138126/225000 [275:51:15<166:37:18,  6.90s/it]
 61%|██████▏   | 138127/225000 [275:51:19<146:24:28,  6.07s/it]
 61%|██████▏   | 138128/225000 [275:51:23<132:41:42,  5.50s/it]
 61%|██████▏   | 138129/225000 [275:51:27<121:52:00,  5.05s/it]
 61%|██████▏   | 138130/225000 [275:51:40<173:04:50,  7.17s/it]
 61%|██████▏   | 138131/225000 [275:51:44<157:15:56,  6.52s/it]
 61%|██████▏   | 138132/225000 [275:51:49<144:40:23,  6.00s/it]
 61%|██████▏   | 138133/225000 [275:51:54<137:30:59,  5.70s/it]
 61%|██████▏   | 138134/225000 [275:51:59<129:15:41,  5.36s/it]
 61%|██████▏   | 138135/225000 [275:52:04<126:17:47,  5.23s/it]
 61%|██████▏   | 138136/225000 [275:52:09<123:48:37,  5.13s/it]
 61%|██████▏   | 138137/225000 [275:52:14<121:59:30,  5.06s/it]
 61%|██████▏   | 138138/225000 [275:52:24<157:48:48,  6.54s/it]
 61%|██████▏   | 138139/225000 [275:52:31<167:47:30,  6.95s/it]
 61%|██████▏   | 138140/225000 [275:52:35<141:47:58,  5.88s/it]
 61%|██████▏   | 138141/225000 [275:52:42<153:45:48,  6.37s/it]
 61%|██████▏   | 138142/225000 [275:52:47<142:42:55,  5.92s/it]
 61%|██████▏   | 138143/225000 [275:52:51<126:23:33,  5.24s/it]
 61%|██████▏   | 138144/225000 [275:52:55<118:33:58,  4.91s/it]
 61%|██████▏   | 138145/225000 [275:53:05<152:24:15,  6.32s/it]
 61%|██████▏   | 138146/225000 [275:53:11<150:39:18,  6.24s/it]
 61%|██████▏   | 138147/225000 [275:53:14<130:03:14,  5.39s/it]
 61%|██████▏   | 138148/225000 [275:53:19<129:36:25,  5.37s/it]
 61%|██████▏   | 138149/225000 [275:53:24<125:05:26,  5.19s/it]
 61%|██████▏   | 138150/225000 [275:53:30<127:30:20,  5.29s/it]
                                                               
{'loss': '0.8025', 'grad_norm': '1.045', 'learning_rate': '1.759e-05', 'epoch': '0.614', 'num_input_tokens_seen': 457704640, 'train_runtime': '9.932e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138150/225000 [275:53:30<127:30:20,  5.29s/it]
 61%|██████▏   | 138151/225000 [275:53:35<125:21:59,  5.20s/it]
 61%|██████▏   | 138152/225000 [275:53:40<122:42:03,  5.09s/it]
 61%|██████▏   | 138153/225000 [275:53:45<123:35:28,  5.12s/it]
 61%|██████▏   | 138154/225000 [275:53:50<124:32:06,  5.16s/it]
 61%|██████▏   | 138155/225000 [275:53:55<126:55:08,  5.26s/it]
 61%|██████▏   | 138156/225000 [275:54:03<142:11:53,  5.89s/it]
 61%|██████▏   | 138157/225000 [275:54:14<182:28:46,  7.56s/it]
 61%|██████▏   | 138158/225000 [275:54:18<150:52:31,  6.25s/it]
 61%|██████▏   | 138159/225000 [275:54:26<169:29:30,  7.03s/it]
 61%|██████▏   | 138160/225000 [275:54:30<145:08:33,  6.02s/it]
 61%|██████▏   | 138161/225000 [275:54:34<129:56:26,  5.39s/it]
 61%|██████▏   | 138162/225000 [275:54:44<163:11:07,  6.77s/it]
 61%|██████▏   | 138163/225000 [275:54:48<141:43:39,  5.88s/it]
 61%|██████▏   | 138164/225000 [275:54:52<129:43:07,  5.38s/it]
 61%|██████▏   | 138165/225000 [275:54:57<125:19:09,  5.20s/it]
 61%|██████▏   | 138166/225000 [275:55:02<123:06:08,  5.10s/it]
 61%|██████▏   | 138167/225000 [275:55:07<125:34:12,  5.21s/it]
 61%|██████▏   | 138168/225000 [275:55:12<127:11:13,  5.27s/it]
 61%|██████▏   | 138169/225000 [275:55:18<126:01:31,  5.22s/it]
 61%|██████▏   | 138170/225000 [275:55:22<122:30:50,  5.08s/it]
 61%|██████▏   | 138171/225000 [275:55:29<130:46:34,  5.42s/it]
 61%|██████▏   | 138172/225000 [275:55:38<159:59:37,  6.63s/it]
 61%|██████▏   | 138173/225000 [275:55:43<145:22:36,  6.03s/it]
 61%|██████▏   | 138174/225000 [275:55:48<137:43:40,  5.71s/it]
 61%|██████▏   | 138175/225000 [275:55:53<133:14:50,  5.52s/it]
 61%|██████▏   | 138176/225000 [275:55:58<132:34:43,  5.50s/it]
 61%|██████▏   | 138177/225000 [275:56:04<134:59:49,  5.60s/it]
 61%|██████▏   | 138178/225000 [275:56:09<130:50:55,  5.43s/it]
 61%|██████▏   | 138179/225000 [275:56:14<126:23:05,  5.24s/it]
 61%|██████▏   | 138180/225000 [275:56:19<123:33:37,  5.12s/it]
 61%|██████▏   | 138181/225000 [275:56:24<125:55:17,  5.22s/it]
 61%|██████▏   | 138182/225000 [275:56:29<123:43:16,  5.13s/it]
 61%|██████▏   | 138183/225000 [275:56:34<122:51:45,  5.09s/it]
 61%|██████▏   | 138184/225000 [275:56:39<123:35:28,  5.12s/it]
 61%|██████▏   | 138185/225000 [275:56:44<119:46:18,  4.97s/it]
 61%|██████▏   | 138186/225000 [275:56:53<151:36:22,  6.29s/it]
 61%|██████▏   | 138187/225000 [275:57:00<157:43:57,  6.54s/it]
 61%|██████▏   | 138188/225000 [275:57:04<140:22:33,  5.82s/it]
 61%|██████▏   | 138189/225000 [275:57:13<162:16:35,  6.73s/it]
 61%|██████▏   | 138190/225000 [275:57:19<156:27:04,  6.49s/it]
 61%|██████▏   | 138191/225000 [275:57:22<133:11:52,  5.52s/it]
 61%|██████▏   | 138192/225000 [275:57:32<160:28:01,  6.65s/it]
 61%|██████▏   | 138193/225000 [275:57:35<134:53:15,  5.59s/it]
 61%|██████▏   | 138194/225000 [275:57:42<144:27:42,  5.99s/it]
 61%|██████▏   | 138195/225000 [275:57:47<138:02:49,  5.73s/it]
 61%|██████▏   | 138196/225000 [275:57:52<133:35:58,  5.54s/it]
 61%|██████▏   | 138197/225000 [275:57:57<127:57:57,  5.31s/it]
 61%|██████▏   | 138198/225000 [275:58:02<124:25:33,  5.16s/it]
 61%|██████▏   | 138199/225000 [275:58:06<120:06:25,  4.98s/it]
 61%|██████▏   | 138200/225000 [275:58:11<121:08:40,  5.02s/it]
                                                               
{'loss': '0.7924', 'grad_norm': '1.084', 'learning_rate': '1.758e-05', 'epoch': '0.6142', 'num_input_tokens_seen': 457833760, 'train_runtime': '9.935e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138200/225000 [275:58:11<121:08:40,  5.02s/it]
 61%|██████▏   | 138201/225000 [275:58:19<138:22:25,  5.74s/it]
 61%|██████▏   | 138202/225000 [275:58:29<169:25:44,  7.03s/it]
 61%|██████▏   | 138203/225000 [275:58:33<147:28:43,  6.12s/it]
 61%|██████▏   | 138204/225000 [275:58:37<132:39:15,  5.50s/it]
 61%|██████▏   | 138205/225000 [275:58:41<121:28:58,  5.04s/it]
 61%|██████▏   | 138206/225000 [275:58:52<164:16:51,  6.81s/it]
 61%|██████▏   | 138207/225000 [275:58:56<145:36:43,  6.04s/it]
 61%|██████▏   | 138208/225000 [275:59:00<132:33:12,  5.50s/it]
 61%|██████▏   | 138209/225000 [275:59:05<124:36:16,  5.17s/it]
 61%|██████▏   | 138210/225000 [275:59:09<120:21:36,  4.99s/it]
 61%|██████▏   | 138211/225000 [275:59:14<118:35:19,  4.92s/it]
 61%|██████▏   | 138212/225000 [275:59:18<115:24:40,  4.79s/it]
 61%|██████▏   | 138213/225000 [275:59:27<145:39:47,  6.04s/it]
 61%|██████▏   | 138214/225000 [275:59:37<173:55:33,  7.21s/it]
 61%|██████▏   | 138215/225000 [275:59:41<145:18:30,  6.03s/it]
 61%|██████▏   | 138216/225000 [275:59:45<136:44:50,  5.67s/it]
 61%|██████▏   | 138217/225000 [275:59:50<127:40:25,  5.30s/it]
 61%|██████▏   | 138218/225000 [275:59:54<121:39:52,  5.05s/it]
 61%|██████▏   | 138219/225000 [276:00:01<136:30:07,  5.66s/it]
 61%|██████▏   | 138220/225000 [276:00:11<168:15:24,  6.98s/it]
 61%|██████▏   | 138221/225000 [276:00:16<147:57:21,  6.14s/it]
 61%|██████▏   | 138222/225000 [276:00:23<160:12:42,  6.65s/it]
 61%|██████▏   | 138223/225000 [276:00:29<150:28:31,  6.24s/it]
 61%|██████▏   | 138224/225000 [276:00:32<132:23:43,  5.49s/it]
 61%|██████▏   | 138225/225000 [276:00:39<138:04:29,  5.73s/it]
 61%|██████▏   | 138226/225000 [276:00:44<132:59:49,  5.52s/it]
 61%|██████▏   | 138227/225000 [276:00:51<143:15:09,  5.94s/it]
 61%|██████▏   | 138228/225000 [276:00:54<126:45:40,  5.26s/it]
 61%|██████▏   | 138229/225000 [276:00:59<121:19:27,  5.03s/it]
 61%|██████▏   | 138230/225000 [276:01:04<119:57:50,  4.98s/it]
 61%|██████▏   | 138231/225000 [276:01:08<118:17:00,  4.91s/it]
 61%|██████▏   | 138232/225000 [276:01:13<116:16:08,  4.82s/it]
 61%|██████▏   | 138233/225000 [276:01:19<124:24:58,  5.16s/it]
 61%|██████▏   | 138234/225000 [276:01:31<172:46:51,  7.17s/it]
 61%|██████▏   | 138235/225000 [276:01:36<158:45:58,  6.59s/it]
 61%|██████▏   | 138236/225000 [276:01:41<149:13:52,  6.19s/it]
 61%|██████▏   | 138237/225000 [276:01:47<141:23:55,  5.87s/it]
 61%|██████▏   | 138238/225000 [276:01:52<138:46:30,  5.76s/it]
 61%|██████▏   | 138239/225000 [276:01:57<134:31:18,  5.58s/it]
 61%|██████▏   | 138240/225000 [276:02:03<133:52:45,  5.56s/it]
 61%|██████▏   | 138241/225000 [276:02:08<131:12:45,  5.44s/it]
 61%|██████▏   | 138242/225000 [276:02:13<130:45:29,  5.43s/it]
 61%|██████▏   | 138243/225000 [276:02:19<132:53:15,  5.51s/it]
 61%|██████▏   | 138244/225000 [276:02:24<131:35:32,  5.46s/it]
 61%|██████▏   | 138245/225000 [276:02:30<131:15:15,  5.45s/it]
 61%|██████▏   | 138246/225000 [276:02:35<130:52:30,  5.43s/it]
 61%|██████▏   | 138247/225000 [276:02:40<129:45:20,  5.38s/it]
 61%|██████▏   | 138248/225000 [276:02:46<129:36:46,  5.38s/it]
 61%|██████▏   | 138249/225000 [276:02:51<127:32:42,  5.29s/it]
 61%|██████▏   | 138250/225000 [276:02:56<129:11:53,  5.36s/it]
                                                               
{'loss': '0.8082', 'grad_norm': '1.165', 'learning_rate': '1.756e-05', 'epoch': '0.6144', 'num_input_tokens_seen': 457963704, 'train_runtime': '9.938e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138250/225000 [276:02:56<129:11:53,  5.36s/it]
 61%|██████▏   | 138251/225000 [276:03:08<172:55:53,  7.18s/it]
 61%|██████▏   | 138252/225000 [276:03:13<156:59:39,  6.52s/it]
 61%|██████▏   | 138253/225000 [276:03:20<161:21:17,  6.70s/it]
 61%|██████▏   | 138254/225000 [276:03:23<135:42:33,  5.63s/it]
 61%|██████▏   | 138255/225000 [276:03:32<156:50:28,  6.51s/it]
 61%|██████▏   | 138256/225000 [276:03:35<135:39:07,  5.63s/it]
 61%|██████▏   | 138257/225000 [276:03:40<129:29:04,  5.37s/it]
 61%|██████▏   | 138258/225000 [276:03:49<157:17:58,  6.53s/it]
 61%|██████▏   | 138259/225000 [276:03:54<143:00:38,  5.94s/it]
 61%|██████▏   | 138260/225000 [276:04:01<152:19:27,  6.32s/it]
 61%|██████▏   | 138261/225000 [276:04:05<133:50:28,  5.55s/it]
 61%|██████▏   | 138262/225000 [276:04:08<117:56:24,  4.90s/it]
 61%|██████▏   | 138263/225000 [276:04:18<155:51:29,  6.47s/it]
 61%|██████▏   | 138264/225000 [276:04:22<135:15:01,  5.61s/it]
 61%|██████▏   | 138265/225000 [276:04:27<133:31:41,  5.54s/it]
 61%|██████▏   | 138266/225000 [276:04:35<152:45:46,  6.34s/it]
 61%|██████▏   | 138267/225000 [276:04:39<131:48:21,  5.47s/it]
 61%|██████▏   | 138268/225000 [276:04:44<132:59:56,  5.52s/it]
 61%|██████▏   | 138269/225000 [276:04:52<146:16:55,  6.07s/it]
 61%|██████▏   | 138270/225000 [276:04:56<129:58:25,  5.39s/it]
 61%|██████▏   | 138271/225000 [276:05:00<119:01:46,  4.94s/it]
 61%|██████▏   | 138272/225000 [276:05:09<150:37:05,  6.25s/it]
 61%|██████▏   | 138273/225000 [276:05:13<132:25:33,  5.50s/it]
 61%|██████▏   | 138274/225000 [276:05:22<158:25:05,  6.58s/it]
 61%|██████▏   | 138275/225000 [276:05:25<135:41:25,  5.63s/it]
 61%|██████▏   | 138276/225000 [276:05:32<145:59:29,  6.06s/it]
 61%|██████▏   | 138277/225000 [276:05:36<129:07:04,  5.36s/it]
 61%|██████▏   | 138278/225000 [276:05:41<124:03:58,  5.15s/it]
 61%|██████▏   | 138279/225000 [276:05:49<148:25:21,  6.16s/it]
 61%|██████▏   | 138280/225000 [276:05:53<134:25:37,  5.58s/it]
 61%|██████▏   | 138281/225000 [276:05:57<124:13:01,  5.16s/it]
 61%|██████▏   | 138282/225000 [276:06:02<118:06:43,  4.90s/it]
 61%|██████▏   | 138283/225000 [276:06:12<154:34:16,  6.42s/it]
 61%|██████▏   | 138284/225000 [276:06:19<161:54:32,  6.72s/it]
 61%|██████▏   | 138285/225000 [276:06:25<158:29:12,  6.58s/it]
 61%|██████▏   | 138286/225000 [276:06:32<161:03:03,  6.69s/it]
 61%|██████▏   | 138287/225000 [276:06:37<147:22:34,  6.12s/it]
 61%|██████▏   | 138288/225000 [276:06:42<136:36:21,  5.67s/it]
 61%|██████▏   | 138289/225000 [276:06:47<133:48:09,  5.56s/it]
 61%|██████▏   | 138290/225000 [276:06:52<128:50:14,  5.35s/it]
 61%|██████▏   | 138291/225000 [276:06:57<125:33:51,  5.21s/it]
 61%|██████▏   | 138292/225000 [276:07:01<121:43:48,  5.05s/it]
 61%|██████▏   | 138293/225000 [276:07:06<120:34:08,  5.01s/it]
 61%|██████▏   | 138294/225000 [276:07:12<125:23:53,  5.21s/it]
 61%|██████▏   | 138295/225000 [276:07:24<173:35:03,  7.21s/it]
 61%|██████▏   | 138296/225000 [276:07:28<151:04:54,  6.27s/it]
 61%|██████▏   | 138297/225000 [276:07:34<145:43:56,  6.05s/it]
 61%|██████▏   | 138298/225000 [276:07:39<139:18:22,  5.78s/it]
 61%|██████▏   | 138299/225000 [276:07:44<136:26:26,  5.67s/it]
 61%|██████▏   | 138300/225000 [276:07:53<160:25:19,  6.66s/it]
                                                               
{'loss': '0.8014', 'grad_norm': '1.067', 'learning_rate': '1.754e-05', 'epoch': '0.6147', 'num_input_tokens_seen': 458093408, 'train_runtime': '9.941e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138300/225000 [276:07:53<160:25:19,  6.66s/it]
 61%|██████▏   | 138301/225000 [276:07:58<146:35:10,  6.09s/it]
 61%|██████▏   | 138302/225000 [276:08:05<155:18:15,  6.45s/it]
 61%|██████▏   | 138303/225000 [276:08:09<136:59:17,  5.69s/it]
 61%|██████▏   | 138304/225000 [276:08:14<131:21:10,  5.45s/it]
 61%|██████▏   | 138305/225000 [276:08:24<162:47:07,  6.76s/it]
 61%|██████▏   | 138306/225000 [276:08:28<147:23:04,  6.12s/it]
 61%|██████▏   | 138307/225000 [276:08:33<136:20:59,  5.66s/it]
 61%|██████▏   | 138308/225000 [276:08:37<126:18:15,  5.24s/it]
 61%|██████▏   | 138309/225000 [276:08:42<120:59:46,  5.02s/it]
 61%|██████▏   | 138310/225000 [276:08:47<120:25:27,  5.00s/it]
 61%|██████▏   | 138311/225000 [276:08:58<168:28:55,  7.00s/it]
 61%|██████▏   | 138312/225000 [276:09:02<145:41:33,  6.05s/it]
 61%|██████▏   | 138313/225000 [276:09:11<165:55:02,  6.89s/it]
 61%|██████▏   | 138314/225000 [276:09:15<147:24:40,  6.12s/it]
 61%|██████▏   | 138315/225000 [276:09:23<161:04:01,  6.69s/it]
 61%|██████▏   | 138316/225000 [276:09:27<138:40:49,  5.76s/it]
 61%|██████▏   | 138317/225000 [276:09:31<125:12:09,  5.20s/it]
 61%|██████▏   | 138318/225000 [276:09:41<158:41:04,  6.59s/it]
 61%|██████▏   | 138319/225000 [276:09:45<140:19:08,  5.83s/it]
 61%|██████▏   | 138320/225000 [276:09:49<125:55:18,  5.23s/it]
 61%|██████▏   | 138321/225000 [276:09:58<158:20:18,  6.58s/it]
 61%|██████▏   | 138322/225000 [276:10:02<137:52:37,  5.73s/it]
 61%|██████▏   | 138323/225000 [276:10:06<125:16:00,  5.20s/it]
 61%|██████▏   | 138324/225000 [276:10:17<163:45:56,  6.80s/it]
 61%|██████▏   | 138325/225000 [276:10:20<142:10:08,  5.90s/it]
 61%|██████▏   | 138326/225000 [276:10:28<152:58:48,  6.35s/it]
 61%|██████▏   | 138327/225000 [276:10:32<140:22:13,  5.83s/it]
 61%|██████▏   | 138328/225000 [276:10:37<131:15:32,  5.45s/it]
 61%|██████▏   | 138329/225000 [276:10:41<123:31:34,  5.13s/it]
 61%|██████▏   | 138330/225000 [276:10:46<117:27:29,  4.88s/it]
 61%|██████▏   | 138331/225000 [276:10:51<118:27:00,  4.92s/it]
 61%|██████▏   | 138332/225000 [276:10:55<117:42:28,  4.89s/it]
 61%|██████▏   | 138333/225000 [276:11:03<134:13:07,  5.58s/it]
 61%|██████▏   | 138334/225000 [276:11:15<182:58:19,  7.60s/it]
 61%|██████▏   | 138335/225000 [276:11:19<154:00:47,  6.40s/it]
 61%|██████▏   | 138336/225000 [276:11:22<132:46:18,  5.52s/it]
 61%|██████▏   | 138337/225000 [276:11:26<123:51:00,  5.14s/it]
 61%|██████▏   | 138338/225000 [276:11:31<121:39:13,  5.05s/it]
 61%|██████▏   | 138339/225000 [276:11:36<120:46:51,  5.02s/it]
 61%|██████▏   | 138340/225000 [276:11:48<168:36:24,  7.00s/it]
 61%|██████▏   | 138341/225000 [276:11:54<164:14:08,  6.82s/it]
 61%|██████▏   | 138342/225000 [276:11:58<140:38:35,  5.84s/it]
 61%|██████▏   | 138343/225000 [276:12:07<165:12:45,  6.86s/it]
 61%|██████▏   | 138344/225000 [276:12:11<147:57:18,  6.15s/it]
 61%|██████▏   | 138345/225000 [276:12:15<128:26:33,  5.34s/it]
 61%|██████▏   | 138346/225000 [276:12:19<117:44:16,  4.89s/it]
 61%|██████▏   | 138347/225000 [276:12:25<130:21:25,  5.42s/it]
 61%|██████▏   | 138348/225000 [276:12:36<164:56:06,  6.85s/it]
 61%|██████▏   | 138349/225000 [276:12:39<140:52:24,  5.85s/it]
 61%|██████▏   | 138350/225000 [276:12:43<127:21:13,  5.29s/it]
                                                               
{'loss': '0.7895', 'grad_norm': '1.028', 'learning_rate': '1.752e-05', 'epoch': '0.6149', 'num_input_tokens_seen': 458222960, 'train_runtime': '9.944e+05', 'train_tokens_per_second': '460.8'}

 61%|██████▏   | 138350/225000 [276:12:43<127:21:13,  5.29s/it]
 61%|██████▏   | 138351/225000 [276:12:51<147:24:47,  6.12s/it]
 61%|██████▏   | 138352/225000 [276:13:00<165:42:53,  6.89s/it]
 61%|██████▏   | 138353/225000 [276:13:04<147:08:35,  6.11s/it]
 61%|██████▏   | 138354/225000 [276:13:09<137:27:01,  5.71s/it]
 61%|██████▏   | 138355/225000 [276:13:13<128:54:02,  5.36s/it]
 61%|██████▏   | 138356/225000 [276:13:18<122:49:02,  5.10s/it]
 61%|██████▏   | 138357/225000 [276:13:26<145:01:47,  6.03s/it]
 61%|██████▏   | 138358/225000 [276:13:37<177:10:21,  7.36s/it]
 61%|██████▏   | 138359/225000 [276:13:40<148:43:22,  6.18s/it]
 61%|██████▏   | 138360/225000 [276:13:49<166:02:05,  6.90s/it]
 61%|██████▏   | 138361/225000 [276:13:53<145:45:08,  6.06s/it]
 61%|██████▏   | 138362/225000 [276:13:57<131:19:22,  5.46s/it]
 61%|██████▏   | 138363/225000 [276:14:03<134:39:37,  5.60s/it]
 61%|██████▏   | 138364/225000 [276:14:13<170:39:38,  7.09s/it]
 61%|██████▏   | 138365/225000 [276:14:17<148:55:39,  6.19s/it]
 61%|██████▏   | 138366/225000 [276:14:23<144:25:38,  6.00s/it]
 61%|██████▏   | 138367/225000 [276:14:27<130:01:21,  5.40s/it]
 61%|██████▏   | 138368/225000 [276:14:30<116:45:37,  4.85s/it]
 61%|██████▏   | 138369/225000 [276:14:35<116:41:01,  4.85s/it]
 61%|██████▏   | 138370/225000 [276:14:47<167:13:04,  6.95s/it]
 61%|██████▏   | 138371/225000 [276:14:50<139:54:53,  5.81s/it]
 61%|██████▏   | 138372/225000 [276:14:55<134:58:55,  5.61s/it]
 61%|██████▏   | 138373/225000 [276:15:01<134:40:49,  5.60s/it]
 61%|██████▏   | 138374/225000 [276:15:06<132:43:38,  5.52s/it]
 62%|██████▏   | 138375/225000 [276:15:12<130:41:20,  5.43s/it]
 62%|██████▏   | 138376/225000 [276:15:17<133:01:04,  5.53s/it]
 62%|██████▏   | 138377/225000 [276:15:22<130:34:55,  5.43s/it]
 62%|██████▏   | 138378/225000 [276:15:27<125:04:40,  5.20s/it]
 62%|██████▏   | 138379/225000 [276:15:32<125:10:25,  5.20s/it]
 62%|██████▏   | 138380/225000 [276:15:38<125:15:59,  5.21s/it]
 62%|██████▏   | 138381/225000 [276:15:43<129:07:06,  5.37s/it]
 62%|██████▏   | 138382/225000 [276:15:55<174:53:47,  7.27s/it]
 62%|██████▏   | 138383/225000 [276:16:00<159:18:56,  6.62s/it]
 62%|██████▏   | 138384/225000 [276:16:05<147:09:42,  6.12s/it]
 62%|██████▏   | 138385/225000 [276:16:10<141:55:59,  5.90s/it]
 62%|██████▏   | 138386/225000 [276:16:16<139:23:52,  5.79s/it]
 62%|██████▏   | 138387/225000 [276:16:21<135:27:35,  5.63s/it]
 62%|██████▏   | 138388/225000 [276:16:27<135:07:36,  5.62s/it]
 62%|██████▏   | 138389/225000 [276:16:32<133:24:35,  5.55s/it]
 62%|██████▏   | 138390/225000 [276:16:37<129:21:41,  5.38s/it]
 62%|██████▏   | 138391/225000 [276:16:43<128:55:39,  5.36s/it]
 62%|██████▏   | 138392/225000 [276:16:48<128:21:43,  5.34s/it]
 62%|██████▏   | 138393/225000 [276:17:00<180:41:06,  7.51s/it]
 62%|██████▏   | 138394/225000 [276:17:08<178:38:28,  7.43s/it]
 62%|██████▏   | 138395/225000 [276:17:12<153:24:21,  6.38s/it]
 62%|██████▏   | 138396/225000 [276:17:15<135:20:40,  5.63s/it]
 62%|██████▏   | 138397/225000 [276:17:22<145:06:46,  6.03s/it]
 62%|██████▏   | 138398/225000 [276:17:29<146:00:07,  6.07s/it]
 62%|██████▏   | 138399/225000 [276:17:32<130:31:57,  5.43s/it]
 62%|██████▏   | 138400/225000 [276:17:37<120:37:17,  5.01s/it]
                                                               
{'loss': '0.792', 'grad_norm': '1.071', 'learning_rate': '1.751e-05', 'epoch': '0.6151', 'num_input_tokens_seen': 458353192, 'train_runtime': '9.947e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138400/225000 [276:17:37<120:37:17,  5.01s/it]
 62%|██████▏   | 138401/225000 [276:17:49<173:25:46,  7.21s/it]
 62%|██████▏   | 138402/225000 [276:17:53<149:48:02,  6.23s/it]
 62%|██████▏   | 138403/225000 [276:17:57<131:59:58,  5.49s/it]
 62%|██████▏   | 138404/225000 [276:18:03<139:28:40,  5.80s/it]
 62%|██████▏   | 138405/225000 [276:18:07<129:07:49,  5.37s/it]
 62%|██████▏   | 138406/225000 [276:18:12<120:33:18,  5.01s/it]
 62%|██████▏   | 138407/225000 [276:18:16<114:55:54,  4.78s/it]
 62%|██████▏   | 138408/225000 [276:18:27<163:05:32,  6.78s/it]
 62%|██████▏   | 138409/225000 [276:18:35<170:12:30,  7.08s/it]
 62%|██████▏   | 138410/225000 [276:18:38<141:32:22,  5.88s/it]
 62%|██████▏   | 138411/225000 [276:18:45<151:42:54,  6.31s/it]
 62%|██████▏   | 138412/225000 [276:18:52<151:28:40,  6.30s/it]
 62%|██████▏   | 138413/225000 [276:18:56<135:05:16,  5.62s/it]
 62%|██████▏   | 138414/225000 [276:19:00<126:38:05,  5.27s/it]
 62%|██████▏   | 138415/225000 [276:19:05<122:18:28,  5.09s/it]
 62%|██████▏   | 138416/225000 [276:19:09<116:02:43,  4.82s/it]
 62%|██████▏   | 138417/225000 [276:19:14<114:06:51,  4.74s/it]
 62%|██████▏   | 138418/225000 [276:19:25<162:46:17,  6.77s/it]
 62%|██████▏   | 138419/225000 [276:19:31<157:48:25,  6.56s/it]
 62%|██████▏   | 138420/225000 [276:19:35<137:53:43,  5.73s/it]
 62%|██████▏   | 138421/225000 [276:19:39<128:14:25,  5.33s/it]
 62%|██████▏   | 138422/225000 [276:19:44<124:07:21,  5.16s/it]
 62%|██████▏   | 138423/225000 [276:19:55<161:18:54,  6.71s/it]
 62%|██████▏   | 138424/225000 [276:20:00<152:25:49,  6.34s/it]
 62%|██████▏   | 138425/225000 [276:20:07<156:07:28,  6.49s/it]
 62%|██████▏   | 138426/225000 [276:20:11<136:07:47,  5.66s/it]
 62%|██████▏   | 138427/225000 [276:20:15<126:27:34,  5.26s/it]
 62%|██████▏   | 138428/225000 [276:20:23<146:25:16,  6.09s/it]
 62%|██████▏   | 138429/225000 [276:20:26<127:13:02,  5.29s/it]
 62%|██████▏   | 138430/225000 [276:20:34<146:14:57,  6.08s/it]
 62%|██████▏   | 138431/225000 [276:20:38<132:10:31,  5.50s/it]
 62%|██████▏   | 138432/225000 [276:20:47<156:22:24,  6.50s/it]
 62%|██████▏   | 138433/225000 [276:20:51<137:20:36,  5.71s/it]
 62%|██████▏   | 138434/225000 [276:20:55<122:20:20,  5.09s/it]
 62%|██████▏   | 138435/225000 [276:21:03<142:21:40,  5.92s/it]
 62%|██████▏   | 138436/225000 [276:21:10<152:58:16,  6.36s/it]
 62%|██████▏   | 138437/225000 [276:21:14<135:08:43,  5.62s/it]
 62%|██████▏   | 138438/225000 [276:21:18<122:39:51,  5.10s/it]
 62%|██████▏   | 138439/225000 [276:21:28<162:15:52,  6.75s/it]
 62%|██████▏   | 138440/225000 [276:21:32<142:33:45,  5.93s/it]
 62%|██████▏   | 138441/225000 [276:21:37<130:13:14,  5.42s/it]
 62%|██████▏   | 138442/225000 [276:21:41<121:40:42,  5.06s/it]
 62%|██████▏   | 138443/225000 [276:21:51<157:19:34,  6.54s/it]
 62%|██████▏   | 138444/225000 [276:21:56<144:38:25,  6.02s/it]
 62%|██████▏   | 138445/225000 [276:22:02<147:27:44,  6.13s/it]
 62%|██████▏   | 138446/225000 [276:22:08<144:51:51,  6.03s/it]
 62%|██████▏   | 138447/225000 [276:22:12<130:15:24,  5.42s/it]
 62%|██████▏   | 138448/225000 [276:22:20<153:14:06,  6.37s/it]
 62%|██████▏   | 138449/225000 [276:22:24<134:17:12,  5.59s/it]
 62%|██████▏   | 138450/225000 [276:22:33<159:10:35,  6.62s/it]
                                                               
{'loss': '0.8054', 'grad_norm': '0.9842', 'learning_rate': '1.749e-05', 'epoch': '0.6153', 'num_input_tokens_seen': 458483456, 'train_runtime': '9.95e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138450/225000 [276:22:33<159:10:35,  6.62s/it]
 62%|██████▏   | 138451/225000 [276:22:38<143:31:09,  5.97s/it]
 62%|██████▏   | 138452/225000 [276:22:43<136:32:50,  5.68s/it]
 62%|██████▏   | 138453/225000 [276:22:48<131:03:10,  5.45s/it]
 62%|██████▏   | 138454/225000 [276:22:53<127:38:02,  5.31s/it]
 62%|██████▏   | 138455/225000 [276:22:58<126:17:10,  5.25s/it]
 62%|██████▏   | 138456/225000 [276:23:03<126:02:26,  5.24s/it]
 62%|██████▏   | 138457/225000 [276:23:08<123:07:28,  5.12s/it]
 62%|██████▏   | 138458/225000 [276:23:13<122:04:09,  5.08s/it]
 62%|██████▏   | 138459/225000 [276:23:17<118:37:26,  4.93s/it]
 62%|██████▏   | 138460/225000 [276:23:30<174:10:22,  7.25s/it]
 62%|██████▏   | 138461/225000 [276:23:35<158:26:45,  6.59s/it]
 62%|██████▏   | 138462/225000 [276:23:39<139:27:24,  5.80s/it]
 62%|██████▏   | 138463/225000 [276:23:49<171:42:00,  7.14s/it]
 62%|██████▏   | 138464/225000 [276:23:54<152:46:29,  6.36s/it]
 62%|██████▏   | 138465/225000 [276:23:58<136:38:41,  5.68s/it]
 62%|██████▏   | 138466/225000 [276:24:02<127:13:04,  5.29s/it]
 62%|██████▏   | 138467/225000 [276:24:14<173:43:25,  7.23s/it]
 62%|██████▏   | 138468/225000 [276:24:19<159:46:28,  6.65s/it]
 62%|██████▏   | 138469/225000 [276:24:23<140:16:14,  5.84s/it]
 62%|██████▏   | 138470/225000 [276:24:28<129:08:10,  5.37s/it]
 62%|██████▏   | 138471/225000 [276:24:33<132:47:41,  5.52s/it]
 62%|██████▏   | 138472/225000 [276:24:44<167:33:52,  6.97s/it]
 62%|██████▏   | 138473/225000 [276:24:47<141:37:19,  5.89s/it]
 62%|██████▏   | 138474/225000 [276:24:52<131:54:28,  5.49s/it]
 62%|██████▏   | 138475/225000 [276:24:56<126:04:27,  5.25s/it]
 62%|██████▏   | 138476/225000 [276:25:01<119:21:58,  4.97s/it]
 62%|██████▏   | 138477/225000 [276:25:10<150:11:13,  6.25s/it]
 62%|██████▏   | 138478/225000 [276:25:17<155:07:47,  6.45s/it]
 62%|██████▏   | 138479/225000 [276:25:22<142:18:30,  5.92s/it]
 62%|██████▏   | 138480/225000 [276:25:26<135:07:18,  5.62s/it]
 62%|██████▏   | 138481/225000 [276:25:31<126:37:22,  5.27s/it]
 62%|██████▏   | 138482/225000 [276:25:43<176:27:26,  7.34s/it]
 62%|██████▏   | 138483/225000 [276:25:46<146:23:13,  6.09s/it]
 62%|██████▏   | 138484/225000 [276:25:52<142:18:49,  5.92s/it]
 62%|██████▏   | 138485/225000 [276:25:56<133:30:44,  5.56s/it]
 62%|██████▏   | 138486/225000 [276:26:02<130:55:57,  5.45s/it]
 62%|██████▏   | 138487/225000 [276:26:07<129:11:50,  5.38s/it]
 62%|██████▏   | 138488/225000 [276:26:12<128:30:16,  5.35s/it]
 62%|██████▏   | 138489/225000 [276:26:17<123:50:57,  5.15s/it]
 62%|██████▏   | 138490/225000 [276:26:22<124:31:23,  5.18s/it]
 62%|██████▏   | 138491/225000 [276:26:28<127:59:27,  5.33s/it]
 62%|██████▏   | 138492/225000 [276:26:33<130:29:17,  5.43s/it]
 62%|██████▏   | 138493/225000 [276:26:39<128:43:06,  5.36s/it]
 62%|██████▏   | 138494/225000 [276:26:44<127:53:47,  5.32s/it]
 62%|██████▏   | 138495/225000 [276:26:49<126:34:52,  5.27s/it]
 62%|██████▏   | 138496/225000 [276:27:01<174:31:47,  7.26s/it]
 62%|██████▏   | 138497/225000 [276:27:08<174:32:02,  7.26s/it]
 62%|██████▏   | 138498/225000 [276:27:14<164:44:18,  6.86s/it]
 62%|██████▏   | 138499/225000 [276:27:20<154:35:46,  6.43s/it]
 62%|██████▏   | 138500/225000 [276:27:24<141:43:31,  5.90s/it]
                                                               
{'loss': '0.8059', 'grad_norm': '1.189', 'learning_rate': '1.747e-05', 'epoch': '0.6156', 'num_input_tokens_seen': 458614464, 'train_runtime': '9.952e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138500/225000 [276:27:24<141:43:31,  5.90s/it]
 62%|██████▏   | 138501/225000 [276:27:29<131:52:55,  5.49s/it]
 62%|██████▏   | 138502/225000 [276:27:33<121:32:55,  5.06s/it]
 62%|██████▏   | 138503/225000 [276:27:40<137:44:50,  5.73s/it]
 62%|██████▏   | 138504/225000 [276:27:52<178:50:49,  7.44s/it]
 62%|██████▏   | 138505/225000 [276:27:55<150:23:43,  6.26s/it]
 62%|██████▏   | 138506/225000 [276:28:05<174:38:11,  7.27s/it]
 62%|██████▏   | 138507/225000 [276:28:08<145:14:12,  6.05s/it]
 62%|██████▏   | 138508/225000 [276:28:16<157:10:14,  6.54s/it]
 62%|██████▏   | 138509/225000 [276:28:19<136:32:33,  5.68s/it]
 62%|██████▏   | 138510/225000 [276:28:23<123:56:42,  5.16s/it]
 62%|██████▏   | 138511/225000 [276:28:28<118:18:25,  4.92s/it]
 62%|██████▏   | 138512/225000 [276:28:39<164:55:28,  6.86s/it]
 62%|██████▏   | 138513/225000 [276:28:44<153:22:15,  6.38s/it]
 62%|██████▏   | 138514/225000 [276:28:48<133:03:08,  5.54s/it]
 62%|██████▏   | 138515/225000 [276:28:53<127:36:42,  5.31s/it]
 62%|██████▏   | 138516/225000 [276:28:57<124:01:04,  5.16s/it]
 62%|██████▏   | 138517/225000 [276:29:02<122:52:28,  5.11s/it]
 62%|██████▏   | 138518/225000 [276:29:07<123:05:45,  5.12s/it]
 62%|██████▏   | 138519/225000 [276:29:18<163:45:41,  6.82s/it]
 62%|██████▏   | 138520/225000 [276:29:24<154:50:33,  6.45s/it]
 62%|██████▏   | 138521/225000 [276:29:30<155:01:30,  6.45s/it]
 62%|██████▏   | 138522/225000 [276:29:37<159:45:54,  6.65s/it]
 62%|██████▏   | 138523/225000 [276:29:41<137:05:50,  5.71s/it]
 62%|██████▏   | 138524/225000 [276:29:46<135:30:02,  5.64s/it]
 62%|██████▏   | 138525/225000 [276:29:55<157:39:03,  6.56s/it]
 62%|██████▏   | 138526/225000 [276:29:59<139:58:47,  5.83s/it]
 62%|██████▏   | 138527/225000 [276:30:03<126:15:52,  5.26s/it]
 62%|██████▏   | 138528/225000 [276:30:09<131:31:29,  5.48s/it]
 62%|██████▏   | 138529/225000 [276:30:18<154:27:43,  6.43s/it]
 62%|██████▏   | 138530/225000 [276:30:21<134:36:23,  5.60s/it]
 62%|██████▏   | 138531/225000 [276:30:31<162:00:12,  6.74s/it]
 62%|██████▏   | 138532/225000 [276:30:35<141:36:31,  5.90s/it]
 62%|██████▏   | 138533/225000 [276:30:39<125:59:04,  5.25s/it]
 62%|██████▏   | 138534/225000 [276:30:42<116:03:43,  4.83s/it]
 62%|██████▏   | 138535/225000 [276:30:47<112:50:45,  4.70s/it]
 62%|██████▏   | 138536/225000 [276:30:51<109:36:54,  4.56s/it]
 62%|██████▏   | 138537/225000 [276:30:56<112:02:31,  4.67s/it]
 62%|██████▏   | 138538/225000 [276:31:03<131:07:58,  5.46s/it]
 62%|██████▏   | 138539/225000 [276:31:15<179:58:08,  7.49s/it]
 62%|██████▏   | 138540/225000 [276:31:19<152:50:44,  6.36s/it]
 62%|██████▏   | 138541/225000 [276:31:23<132:55:56,  5.54s/it]
 62%|██████▏   | 138542/225000 [276:31:32<157:38:18,  6.56s/it]
 62%|██████▏   | 138543/225000 [276:31:35<134:23:01,  5.60s/it]
 62%|██████▏   | 138544/225000 [276:31:45<163:12:21,  6.80s/it]
 62%|██████▏   | 138545/225000 [276:31:48<141:11:42,  5.88s/it]
 62%|██████▏   | 138546/225000 [276:31:52<124:35:49,  5.19s/it]
 62%|██████▏   | 138547/225000 [276:31:59<140:19:43,  5.84s/it]
 62%|██████▏   | 138548/225000 [276:32:09<165:35:15,  6.90s/it]
 62%|██████▏   | 138549/225000 [276:32:13<147:42:18,  6.15s/it]
 62%|██████▏   | 138550/225000 [276:32:18<134:57:58,  5.62s/it]
                                                               
{'loss': '0.7971', 'grad_norm': '1.095', 'learning_rate': '1.745e-05', 'epoch': '0.6158', 'num_input_tokens_seen': 458742808, 'train_runtime': '9.955e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138550/225000 [276:32:18<134:57:58,  5.62s/it]
 62%|██████▏   | 138551/225000 [276:32:24<142:53:30,  5.95s/it]
 62%|██████▏   | 138552/225000 [276:32:33<164:52:52,  6.87s/it]
 62%|██████▏   | 138553/225000 [276:32:37<141:57:00,  5.91s/it]
 62%|██████▏   | 138554/225000 [276:32:44<152:26:53,  6.35s/it]
 62%|██████▏   | 138555/225000 [276:32:48<135:35:23,  5.65s/it]
 62%|██████▏   | 138556/225000 [276:32:52<120:16:53,  5.01s/it]
 62%|██████▏   | 138557/225000 [276:33:01<147:43:52,  6.15s/it]
 62%|██████▏   | 138558/225000 [276:33:06<144:16:09,  6.01s/it]
 62%|██████▏   | 138559/225000 [276:33:10<128:12:13,  5.34s/it]
 62%|██████▏   | 138560/225000 [276:33:14<119:26:54,  4.97s/it]
 62%|██████▏   | 138561/225000 [276:33:25<157:49:25,  6.57s/it]
 62%|██████▏   | 138562/225000 [276:33:29<144:34:11,  6.02s/it]
 62%|██████▏   | 138563/225000 [276:33:34<135:58:04,  5.66s/it]
 62%|██████▏   | 138564/225000 [276:33:39<129:06:59,  5.38s/it]
 62%|██████▏   | 138565/225000 [276:33:43<121:18:45,  5.05s/it]
 62%|██████▏   | 138566/225000 [276:33:49<129:05:42,  5.38s/it]
 62%|██████▏   | 138567/225000 [276:34:00<168:04:23,  7.00s/it]
 62%|██████▏   | 138568/225000 [276:34:05<155:10:27,  6.46s/it]
 62%|██████▏   | 138569/225000 [276:34:11<147:12:04,  6.13s/it]
 62%|██████▏   | 138570/225000 [276:34:16<141:53:26,  5.91s/it]
 62%|██████▏   | 138571/225000 [276:34:21<133:22:52,  5.56s/it]
 62%|██████▏   | 138572/225000 [276:34:26<130:10:39,  5.42s/it]
 62%|██████▏   | 138573/225000 [276:34:31<129:01:01,  5.37s/it]
 62%|██████▏   | 138574/225000 [276:34:36<124:34:49,  5.19s/it]
 62%|██████▏   | 138575/225000 [276:34:41<121:03:01,  5.04s/it]
 62%|██████▏   | 138576/225000 [276:34:53<176:47:07,  7.36s/it]
 62%|██████▏   | 138577/225000 [276:34:59<167:31:56,  6.98s/it]
 62%|██████▏   | 138578/225000 [276:35:04<151:00:39,  6.29s/it]
 62%|██████▏   | 138579/225000 [276:35:09<139:31:25,  5.81s/it]
 62%|██████▏   | 138580/225000 [276:35:13<131:14:54,  5.47s/it]
 62%|██████▏   | 138581/225000 [276:35:18<122:47:35,  5.12s/it]
 62%|██████▏   | 138582/225000 [276:35:23<121:07:42,  5.05s/it]
 62%|██████▏   | 138583/225000 [276:35:32<154:22:23,  6.43s/it]
 62%|██████▏   | 138584/225000 [276:35:40<166:25:03,  6.93s/it]
 62%|██████▏   | 138585/225000 [276:35:44<142:37:45,  5.94s/it]
 62%|██████▏   | 138586/225000 [276:35:54<170:08:42,  7.09s/it]
 62%|██████▏   | 138587/225000 [276:36:01<170:44:05,  7.11s/it]
 62%|██████▏   | 138588/225000 [276:36:06<156:19:35,  6.51s/it]
 62%|██████▏   | 138589/225000 [276:36:11<145:01:43,  6.04s/it]
 62%|██████▏   | 138590/225000 [276:36:16<139:58:52,  5.83s/it]
 62%|██████▏   | 138591/225000 [276:36:21<132:16:24,  5.51s/it]
 62%|██████▏   | 138592/225000 [276:36:26<130:00:54,  5.42s/it]
 62%|██████▏   | 138593/225000 [276:36:31<126:26:50,  5.27s/it]
 62%|██████▏   | 138594/225000 [276:36:36<125:28:19,  5.23s/it]
 62%|██████▏   | 138595/225000 [276:36:41<123:30:11,  5.15s/it]
 62%|██████▏   | 138596/225000 [276:36:46<122:58:42,  5.12s/it]
 62%|██████▏   | 138597/225000 [276:36:52<123:52:30,  5.16s/it]
 62%|██████▏   | 138598/225000 [276:36:59<140:41:10,  5.86s/it]
 62%|██████▏   | 138599/225000 [276:37:10<179:44:05,  7.49s/it]
 62%|██████▏   | 138600/225000 [276:37:15<160:41:21,  6.70s/it]
                                                               
{'loss': '0.7901', 'grad_norm': '1.069', 'learning_rate': '1.744e-05', 'epoch': '0.616', 'num_input_tokens_seen': 458873008, 'train_runtime': '9.958e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138600/225000 [276:37:15<160:41:21,  6.70s/it]
 62%|██████▏   | 138601/225000 [276:37:20<144:03:54,  6.00s/it]
 62%|██████▏   | 138602/225000 [276:37:24<133:40:34,  5.57s/it]
 62%|██████▏   | 138603/225000 [276:37:29<125:56:54,  5.25s/it]
 62%|██████▏   | 138604/225000 [276:37:35<135:57:38,  5.67s/it]
 62%|██████▏   | 138605/225000 [276:37:46<171:17:21,  7.14s/it]
 62%|██████▏   | 138606/225000 [276:37:50<146:30:53,  6.11s/it]
 62%|██████▏   | 138607/225000 [276:37:55<139:46:58,  5.82s/it]
 62%|██████▏   | 138608/225000 [276:38:00<136:18:16,  5.68s/it]
 62%|██████▏   | 138609/225000 [276:38:05<127:04:10,  5.30s/it]
 62%|██████▏   | 138610/225000 [276:38:09<123:16:20,  5.14s/it]
 62%|██████▏   | 138611/225000 [276:38:14<121:59:46,  5.08s/it]
 62%|██████▏   | 138612/225000 [276:38:19<122:22:54,  5.10s/it]
 62%|██████▏   | 138613/225000 [276:38:25<123:19:48,  5.14s/it]
 62%|██████▏   | 138614/225000 [276:38:30<122:42:38,  5.11s/it]
 62%|██████▏   | 138615/225000 [276:38:35<123:08:39,  5.13s/it]
 62%|██████▏   | 138616/225000 [276:38:40<123:19:37,  5.14s/it]
 62%|██████▏   | 138617/225000 [276:38:45<120:42:52,  5.03s/it]
 62%|██████▏   | 138618/225000 [276:38:58<177:55:42,  7.42s/it]
 62%|██████▏   | 138619/225000 [276:39:07<190:06:48,  7.92s/it]
 62%|██████▏   | 138620/225000 [276:39:10<158:45:41,  6.62s/it]
 62%|██████▏   | 138621/225000 [276:39:18<165:57:36,  6.92s/it]
 62%|██████▏   | 138622/225000 [276:39:22<144:00:40,  6.00s/it]
 62%|██████▏   | 138623/225000 [276:39:31<162:46:09,  6.78s/it]
 62%|██████▏   | 138624/225000 [276:39:36<152:02:03,  6.34s/it]
 62%|██████▏   | 138625/225000 [276:39:41<146:34:28,  6.11s/it]
 62%|██████▏   | 138626/225000 [276:39:46<138:07:31,  5.76s/it]
 62%|██████▏   | 138627/225000 [276:39:52<134:19:18,  5.60s/it]
 62%|██████▏   | 138628/225000 [276:39:57<130:48:52,  5.45s/it]
 62%|██████▏   | 138629/225000 [276:40:02<127:51:03,  5.33s/it]
 62%|██████▏   | 138630/225000 [276:40:06<121:21:33,  5.06s/it]
 62%|██████▏   | 138631/225000 [276:40:11<123:10:13,  5.13s/it]
 62%|██████▏   | 138632/225000 [276:40:16<121:38:28,  5.07s/it]
 62%|██████▏   | 138633/225000 [276:40:27<158:02:35,  6.59s/it]
 62%|██████▏   | 138634/225000 [276:40:36<179:46:47,  7.49s/it]
 62%|██████▏   | 138635/225000 [276:40:40<151:20:05,  6.31s/it]
 62%|██████▏   | 138636/225000 [276:40:49<169:36:08,  7.07s/it]
 62%|██████▏   | 138637/225000 [276:40:53<147:41:14,  6.16s/it]
 62%|██████▏   | 138638/225000 [276:40:57<134:06:12,  5.59s/it]
 62%|██████▏   | 138639/225000 [276:41:01<124:46:49,  5.20s/it]
 62%|██████▏   | 138640/225000 [276:41:10<152:56:21,  6.38s/it]
 62%|██████▏   | 138641/225000 [276:41:17<157:05:38,  6.55s/it]
 62%|██████▏   | 138642/225000 [276:41:21<136:43:11,  5.70s/it]
 62%|██████▏   | 138643/225000 [276:41:30<161:13:08,  6.72s/it]
 62%|██████▏   | 138644/225000 [276:41:36<154:34:48,  6.44s/it]
 62%|██████▏   | 138645/225000 [276:41:43<162:04:47,  6.76s/it]
 62%|██████▏   | 138646/225000 [276:41:47<140:15:52,  5.85s/it]
 62%|██████▏   | 138647/225000 [276:41:55<152:46:20,  6.37s/it]
 62%|██████▏   | 138648/225000 [276:41:58<130:06:29,  5.42s/it]
 62%|██████▏   | 138649/225000 [276:42:07<156:14:16,  6.51s/it]
 62%|██████▏   | 138650/225000 [276:42:11<135:48:02,  5.66s/it]
                                                               
{'loss': '0.804', 'grad_norm': '1.189', 'learning_rate': '1.742e-05', 'epoch': '0.6162', 'num_input_tokens_seen': 459001968, 'train_runtime': '9.961e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138650/225000 [276:42:11<135:48:02,  5.66s/it]
 62%|██████▏   | 138651/225000 [276:42:14<122:51:26,  5.12s/it]
 62%|██████▏   | 138652/225000 [276:42:22<142:54:10,  5.96s/it]
 62%|██████▏   | 138653/225000 [276:42:29<148:34:53,  6.19s/it]
 62%|██████▏   | 138654/225000 [276:42:35<148:18:00,  6.18s/it]
 62%|██████▏   | 138655/225000 [276:42:41<144:06:10,  6.01s/it]
 62%|██████▏   | 138656/225000 [276:42:46<138:36:00,  5.78s/it]
 62%|██████▏   | 138657/225000 [276:42:54<153:00:24,  6.38s/it]
 62%|██████▏   | 138658/225000 [276:42:58<139:05:53,  5.80s/it]
 62%|██████▏   | 138659/225000 [276:43:03<129:27:33,  5.40s/it]
 62%|██████▏   | 138660/225000 [276:43:07<119:50:03,  5.00s/it]
 62%|██████▏   | 138661/225000 [276:43:12<117:54:42,  4.92s/it]
 62%|██████▏   | 138662/225000 [276:43:16<116:27:07,  4.86s/it]
 62%|██████▏   | 138663/225000 [276:43:21<114:57:22,  4.79s/it]
 62%|██████▏   | 138664/225000 [276:43:34<172:03:03,  7.17s/it]
 62%|██████▏   | 138665/225000 [276:43:40<162:43:01,  6.78s/it]
 62%|██████▏   | 138666/225000 [276:43:43<141:11:56,  5.89s/it]
 62%|██████▏   | 138667/225000 [276:43:48<130:56:19,  5.46s/it]
 62%|██████▏   | 138668/225000 [276:43:53<126:37:36,  5.28s/it]
 62%|██████▏   | 138669/225000 [276:43:57<122:30:13,  5.11s/it]
 62%|██████▏   | 138670/225000 [276:44:06<147:44:15,  6.16s/it]
 62%|██████▏   | 138671/225000 [276:44:18<188:15:06,  7.85s/it]
 62%|██████▏   | 138672/225000 [276:44:21<157:39:12,  6.57s/it]
 62%|██████▏   | 138673/225000 [276:44:25<135:18:26,  5.64s/it]
 62%|██████▏   | 138674/225000 [276:44:34<157:42:52,  6.58s/it]
 62%|██████▏   | 138675/225000 [276:44:40<157:31:09,  6.57s/it]
 62%|██████▏   | 138676/225000 [276:44:44<140:58:54,  5.88s/it]
 62%|██████▏   | 138677/225000 [276:44:49<129:35:21,  5.40s/it]
 62%|██████▏   | 138678/225000 [276:44:53<119:50:22,  5.00s/it]
 62%|██████▏   | 138679/225000 [276:45:03<158:46:20,  6.62s/it]
 62%|██████▏   | 138680/225000 [276:45:08<145:06:05,  6.05s/it]
 62%|██████▏   | 138681/225000 [276:45:13<137:33:42,  5.74s/it]
 62%|██████▏   | 138682/225000 [276:45:18<132:32:18,  5.53s/it]
 62%|██████▏   | 138683/225000 [276:45:23<126:23:37,  5.27s/it]
 62%|██████▏   | 138684/225000 [276:45:28<126:57:06,  5.29s/it]
 62%|██████▏   | 138685/225000 [276:45:33<126:18:01,  5.27s/it]
 62%|██████▏   | 138686/225000 [276:45:39<127:50:42,  5.33s/it]
 62%|██████▏   | 138687/225000 [276:45:44<127:13:38,  5.31s/it]
 62%|██████▏   | 138688/225000 [276:45:49<128:56:37,  5.38s/it]
 62%|██████▏   | 138689/225000 [276:45:55<127:48:05,  5.33s/it]
 62%|██████▏   | 138690/225000 [276:46:00<129:59:43,  5.42s/it]
 62%|██████▏   | 138691/225000 [276:46:06<130:52:06,  5.46s/it]
 62%|██████▏   | 138692/225000 [276:46:18<179:19:37,  7.48s/it]
 62%|██████▏   | 138693/225000 [276:46:26<180:30:39,  7.53s/it]
 62%|██████▏   | 138694/225000 [276:46:34<186:09:09,  7.76s/it]
 62%|██████▏   | 138695/225000 [276:46:38<156:38:39,  6.53s/it]
 62%|██████▏   | 138696/225000 [276:46:46<167:42:37,  7.00s/it]
 62%|██████▏   | 138697/225000 [276:46:50<149:26:59,  6.23s/it]
 62%|██████▏   | 138698/225000 [276:46:54<135:16:47,  5.64s/it]
 62%|██████▏   | 138699/225000 [276:46:58<123:14:44,  5.14s/it]
 62%|██████▏   | 138700/225000 [276:47:03<117:29:46,  4.90s/it]
                                                               
{'loss': '0.8059', 'grad_norm': '1.125', 'learning_rate': '1.74e-05', 'epoch': '0.6164', 'num_input_tokens_seen': 459131608, 'train_runtime': '9.964e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138700/225000 [276:47:03<117:29:46,  4.90s/it]
 62%|██████▏   | 138701/225000 [276:47:15<173:02:57,  7.22s/it]
 62%|██████▏   | 138702/225000 [276:47:20<156:09:40,  6.51s/it]
 62%|██████▏   | 138703/225000 [276:47:24<135:21:51,  5.65s/it]
 62%|██████▏   | 138704/225000 [276:47:33<163:16:29,  6.81s/it]
 62%|██████▏   | 138705/225000 [276:47:37<142:01:04,  5.92s/it]
 62%|██████▏   | 138706/225000 [276:47:41<129:31:00,  5.40s/it]
 62%|██████▏   | 138707/225000 [276:47:45<117:42:31,  4.91s/it]
 62%|██████▏   | 138708/225000 [276:47:53<141:31:15,  5.90s/it]
 62%|██████▏   | 138709/225000 [276:48:02<157:46:03,  6.58s/it]
 62%|██████▏   | 138710/225000 [276:48:06<142:43:49,  5.95s/it]
 62%|██████▏   | 138711/225000 [276:48:11<136:35:26,  5.70s/it]
 62%|██████▏   | 138712/225000 [276:48:16<129:32:16,  5.40s/it]
 62%|██████▏   | 138713/225000 [276:48:20<119:54:30,  5.00s/it]
 62%|██████▏   | 138714/225000 [276:48:25<118:22:49,  4.94s/it]
 62%|██████▏   | 138715/225000 [276:48:35<153:15:10,  6.39s/it]
 62%|██████▏   | 138716/225000 [276:48:42<161:00:47,  6.72s/it]
 62%|██████▏   | 138717/225000 [276:48:45<133:18:32,  5.56s/it]
 62%|██████▏   | 138718/225000 [276:48:50<130:14:33,  5.43s/it]
 62%|██████▏   | 138719/225000 [276:48:57<144:53:56,  6.05s/it]
 62%|██████▏   | 138720/225000 [276:49:01<128:54:56,  5.38s/it]
 62%|██████▏   | 138721/225000 [276:49:10<154:24:32,  6.44s/it]
 62%|██████▏   | 138722/225000 [276:49:14<132:46:55,  5.54s/it]
 62%|██████▏   | 138723/225000 [276:49:18<121:28:29,  5.07s/it]
 62%|██████▏   | 138724/225000 [276:49:28<161:30:44,  6.74s/it]
 62%|██████▏   | 138725/225000 [276:49:32<137:40:02,  5.74s/it]
 62%|██████▏   | 138726/225000 [276:49:36<128:15:31,  5.35s/it]
 62%|██████▏   | 138727/225000 [276:49:41<123:04:08,  5.14s/it]
 62%|██████▏   | 138728/225000 [276:49:45<117:49:50,  4.92s/it]
 62%|██████▏   | 138729/225000 [276:49:53<141:15:40,  5.89s/it]
 62%|██████▏   | 138730/225000 [276:50:04<172:26:49,  7.20s/it]
 62%|██████▏   | 138731/225000 [276:50:07<145:03:14,  6.05s/it]
 62%|██████▏   | 138732/225000 [276:50:15<157:01:44,  6.55s/it]
 62%|██████▏   | 138733/225000 [276:50:18<135:47:10,  5.67s/it]
 62%|██████▏   | 138734/225000 [276:50:25<142:31:28,  5.95s/it]
 62%|██████▏   | 138735/225000 [276:50:31<146:39:41,  6.12s/it]
 62%|██████▏   | 138736/225000 [276:50:35<130:27:03,  5.44s/it]
 62%|██████▏   | 138737/225000 [276:50:45<162:03:39,  6.76s/it]
 62%|██████▏   | 138738/225000 [276:50:48<137:19:59,  5.73s/it]
 62%|██████▏   | 138739/225000 [276:50:57<156:58:23,  6.55s/it]
 62%|██████▏   | 138740/225000 [276:51:00<132:34:27,  5.53s/it]
 62%|██████▏   | 138741/225000 [276:51:07<145:59:29,  6.09s/it]
 62%|██████▏   | 138742/225000 [276:51:12<137:47:54,  5.75s/it]
 62%|██████▏   | 138743/225000 [276:51:16<124:50:23,  5.21s/it]
 62%|██████▏   | 138744/225000 [276:51:26<156:46:12,  6.54s/it]
 62%|██████▏   | 138745/225000 [276:51:29<133:28:23,  5.57s/it]
 62%|██████▏   | 138746/225000 [276:51:34<127:51:13,  5.34s/it]
 62%|██████▏   | 138747/225000 [276:51:43<156:35:01,  6.54s/it]
 62%|██████▏   | 138748/225000 [276:51:48<145:01:19,  6.05s/it]
 62%|██████▏   | 138749/225000 [276:51:55<146:15:29,  6.10s/it]
 62%|██████▏   | 138750/225000 [276:52:01<149:59:37,  6.26s/it]
                                                               
{'loss': '0.7996', 'grad_norm': '1.101', 'learning_rate': '1.738e-05', 'epoch': '0.6167', 'num_input_tokens_seen': 459260656, 'train_runtime': '9.967e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138750/225000 [276:52:01<149:59:37,  6.26s/it]
 62%|██████▏   | 138751/225000 [276:52:07<148:07:53,  6.18s/it]
 62%|██████▏   | 138752/225000 [276:52:15<159:36:53,  6.66s/it]
 62%|██████▏   | 138753/225000 [276:52:18<134:37:29,  5.62s/it]
 62%|██████▏   | 138754/225000 [276:52:22<122:00:08,  5.09s/it]
 62%|██████▏   | 138755/225000 [276:52:28<126:35:56,  5.28s/it]
 62%|██████▏   | 138756/225000 [276:52:36<149:24:05,  6.24s/it]
 62%|██████▏   | 138757/225000 [276:52:40<130:16:28,  5.44s/it]
 62%|██████▏   | 138758/225000 [276:52:44<118:00:48,  4.93s/it]
 62%|██████▏   | 138759/225000 [276:52:54<161:22:59,  6.74s/it]
 62%|██████▏   | 138760/225000 [276:52:58<136:54:38,  5.72s/it]
 62%|██████▏   | 138761/225000 [276:53:01<120:16:09,  5.02s/it]
 62%|██████▏   | 138762/225000 [276:53:09<143:24:48,  5.99s/it]
 62%|██████▏   | 138763/225000 [276:53:17<156:07:03,  6.52s/it]
 62%|██████▏   | 138764/225000 [276:53:22<144:13:54,  6.02s/it]
 62%|██████▏   | 138765/225000 [276:53:26<127:40:15,  5.33s/it]
 62%|██████▏   | 138766/225000 [276:53:31<123:58:54,  5.18s/it]
 62%|██████▏   | 138767/225000 [276:53:42<171:18:01,  7.15s/it]
 62%|██████▏   | 138768/225000 [276:53:46<144:20:11,  6.03s/it]
 62%|██████▏   | 138769/225000 [276:53:49<125:30:20,  5.24s/it]
 62%|██████▏   | 138770/225000 [276:53:59<160:36:27,  6.71s/it]
 62%|██████▏   | 138771/225000 [276:54:03<139:53:38,  5.84s/it]
 62%|██████▏   | 138772/225000 [276:54:07<128:47:12,  5.38s/it]
 62%|██████▏   | 138773/225000 [276:54:12<119:53:28,  5.01s/it]
 62%|██████▏   | 138774/225000 [276:54:24<173:44:53,  7.25s/it]
 62%|██████▏   | 138775/225000 [276:54:29<154:18:58,  6.44s/it]
 62%|██████▏   | 138776/225000 [276:54:34<145:29:51,  6.07s/it]
 62%|██████▏   | 138777/225000 [276:54:39<137:35:09,  5.74s/it]
 62%|██████▏   | 138778/225000 [276:54:43<128:58:07,  5.38s/it]
 62%|██████▏   | 138779/225000 [276:54:48<124:24:38,  5.19s/it]
 62%|██████▏   | 138780/225000 [276:54:53<123:55:11,  5.17s/it]
 62%|██████▏   | 138781/225000 [276:54:58<118:14:45,  4.94s/it]
 62%|██████▏   | 138782/225000 [276:55:02<116:29:28,  4.86s/it]
 62%|██████▏   | 138783/225000 [276:55:07<114:24:46,  4.78s/it]
 62%|██████▏   | 138784/225000 [276:55:19<164:34:16,  6.87s/it]
 62%|██████▏   | 138785/225000 [276:55:25<164:05:41,  6.85s/it]
 62%|██████▏   | 138786/225000 [276:55:29<140:20:46,  5.86s/it]
 62%|██████▏   | 138787/225000 [276:55:38<161:13:05,  6.73s/it]
 62%|██████▏   | 138788/225000 [276:55:44<156:05:50,  6.52s/it]
 62%|██████▏   | 138789/225000 [276:55:50<155:05:46,  6.48s/it]
 62%|██████▏   | 138790/225000 [276:55:54<137:47:10,  5.75s/it]
 62%|██████▏   | 138791/225000 [276:55:59<133:53:43,  5.59s/it]
 62%|██████▏   | 138792/225000 [276:56:05<132:57:35,  5.55s/it]
 62%|██████▏   | 138793/225000 [276:56:10<130:39:31,  5.46s/it]
 62%|██████▏   | 138794/225000 [276:56:15<128:24:38,  5.36s/it]
 62%|██████▏   | 138795/225000 [276:56:20<122:39:55,  5.12s/it]
 62%|██████▏   | 138796/225000 [276:56:25<124:02:43,  5.18s/it]
 62%|██████▏   | 138797/225000 [276:56:30<122:21:52,  5.11s/it]
 62%|██████▏   | 138798/225000 [276:56:40<157:02:27,  6.56s/it]
 62%|██████▏   | 138799/225000 [276:56:51<191:04:17,  7.98s/it]
 62%|██████▏   | 138800/225000 [276:56:55<160:46:26,  6.71s/it]
                                                               
{'loss': '0.7869', 'grad_norm': '1.072', 'learning_rate': '1.737e-05', 'epoch': '0.6169', 'num_input_tokens_seen': 459390304, 'train_runtime': '9.97e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138800/225000 [276:56:55<160:46:26,  6.71s/it]
 62%|██████▏   | 138801/225000 [276:57:00<144:32:49,  6.04s/it]
 62%|██████▏   | 138802/225000 [276:57:04<130:01:26,  5.43s/it]
 62%|██████▏   | 138803/225000 [276:57:10<134:31:35,  5.62s/it]
 62%|██████▏   | 138804/225000 [276:57:20<169:37:31,  7.08s/it]
 62%|██████▏   | 138805/225000 [276:57:25<150:42:31,  6.29s/it]
 62%|██████▏   | 138806/225000 [276:57:29<140:34:23,  5.87s/it]
 62%|██████▏   | 138807/225000 [276:57:34<134:18:03,  5.61s/it]
 62%|██████▏   | 138808/225000 [276:57:40<131:48:41,  5.51s/it]
 62%|██████▏   | 138809/225000 [276:57:45<128:41:08,  5.37s/it]
 62%|██████▏   | 138810/225000 [276:57:50<126:10:34,  5.27s/it]
 62%|██████▏   | 138811/225000 [276:57:55<122:27:24,  5.11s/it]
 62%|██████▏   | 138812/225000 [276:58:04<156:26:54,  6.53s/it]
 62%|██████▏   | 138813/225000 [276:58:13<173:48:08,  7.26s/it]
 62%|██████▏   | 138814/225000 [276:58:17<147:43:55,  6.17s/it]
 62%|██████▏   | 138815/225000 [276:58:24<153:58:29,  6.43s/it]
 62%|██████▏   | 138816/225000 [276:58:30<148:24:33,  6.20s/it]
 62%|██████▏   | 138817/225000 [276:58:34<131:40:20,  5.50s/it]
 62%|██████▏   | 138818/225000 [276:58:37<119:30:55,  4.99s/it]
 62%|██████▏   | 138819/225000 [276:58:47<153:48:08,  6.42s/it]
 62%|██████▏   | 138820/225000 [276:58:51<138:57:01,  5.80s/it]
 62%|██████▏   | 138821/225000 [276:58:59<147:57:46,  6.18s/it]
 62%|██████▏   | 138822/225000 [276:59:02<128:43:49,  5.38s/it]
 62%|██████▏   | 138823/225000 [276:59:11<154:46:06,  6.47s/it]
 62%|██████▏   | 138824/225000 [276:59:14<133:02:35,  5.56s/it]
 62%|██████▏   | 138825/225000 [276:59:22<144:52:50,  6.05s/it]
 62%|██████▏   | 138826/225000 [276:59:26<134:05:47,  5.60s/it]
 62%|██████▏   | 138827/225000 [276:59:31<126:15:32,  5.27s/it]
 62%|██████▏   | 138828/225000 [276:59:36<122:40:07,  5.12s/it]
 62%|██████▏   | 138829/225000 [276:59:41<124:00:33,  5.18s/it]
 62%|██████▏   | 138830/225000 [276:59:52<163:48:27,  6.84s/it]
 62%|██████▏   | 138831/225000 [276:59:55<142:25:45,  5.95s/it]
 62%|██████▏   | 138832/225000 [277:00:05<168:54:09,  7.06s/it]
 62%|██████▏   | 138833/225000 [277:00:09<143:21:11,  5.99s/it]
 62%|██████▏   | 138834/225000 [277:00:13<131:28:27,  5.49s/it]
 62%|██████▏   | 138835/225000 [277:00:17<124:29:33,  5.20s/it]
 62%|██████▏   | 138836/225000 [277:00:27<154:23:11,  6.45s/it]
 62%|██████▏   | 138837/225000 [277:00:34<157:10:50,  6.57s/it]
 62%|██████▏   | 138838/225000 [277:00:38<141:03:53,  5.89s/it]
 62%|██████▏   | 138839/225000 [277:00:43<134:14:25,  5.61s/it]
 62%|██████▏   | 138840/225000 [277:00:48<133:47:15,  5.59s/it]
 62%|██████▏   | 138841/225000 [277:00:59<168:41:50,  7.05s/it]
 62%|██████▏   | 138842/225000 [277:01:02<140:23:57,  5.87s/it]
 62%|██████▏   | 138843/225000 [277:01:11<161:18:04,  6.74s/it]
 62%|██████▏   | 138844/225000 [277:01:15<142:01:45,  5.93s/it]
 62%|██████▏   | 138845/225000 [277:01:19<127:34:24,  5.33s/it]
 62%|██████▏   | 138846/225000 [277:01:22<115:18:50,  4.82s/it]
 62%|██████▏   | 138847/225000 [277:01:34<164:04:22,  6.86s/it]
 62%|██████▏   | 138848/225000 [277:01:37<138:01:21,  5.77s/it]
 62%|██████▏   | 138849/225000 [277:01:45<153:52:30,  6.43s/it]
 62%|██████▏   | 138850/225000 [277:01:51<146:54:08,  6.14s/it]
                                                               
{'loss': '0.797', 'grad_norm': '0.9828', 'learning_rate': '1.735e-05', 'epoch': '0.6171', 'num_input_tokens_seen': 459520512, 'train_runtime': '9.973e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138850/225000 [277:01:51<146:54:08,  6.14s/it]
 62%|██████▏   | 138851/225000 [277:01:54<129:49:55,  5.43s/it]
 62%|██████▏   | 138852/225000 [277:02:04<159:18:50,  6.66s/it]
 62%|██████▏   | 138853/225000 [277:02:08<138:30:21,  5.79s/it]
 62%|██████▏   | 138854/225000 [277:02:16<158:57:42,  6.64s/it]
 62%|██████▏   | 138855/225000 [277:02:20<134:31:20,  5.62s/it]
 62%|██████▏   | 138856/225000 [277:02:28<154:08:15,  6.44s/it]
 62%|██████▏   | 138857/225000 [277:02:32<137:02:54,  5.73s/it]
 62%|██████▏   | 138858/225000 [277:02:36<126:15:24,  5.28s/it]
 62%|██████▏   | 138859/225000 [277:02:45<148:01:18,  6.19s/it]
 62%|██████▏   | 138860/225000 [277:02:52<155:13:14,  6.49s/it]
 62%|██████▏   | 138861/225000 [277:02:55<135:32:59,  5.67s/it]
 62%|██████▏   | 138862/225000 [277:02:59<122:33:39,  5.12s/it]
 62%|██████▏   | 138863/225000 [277:03:08<148:49:11,  6.22s/it]
 62%|██████▏   | 138864/225000 [277:03:14<147:03:48,  6.15s/it]
 62%|██████▏   | 138865/225000 [277:03:17<126:19:04,  5.28s/it]
 62%|██████▏   | 138866/225000 [277:03:24<134:46:43,  5.63s/it]
 62%|██████▏   | 138867/225000 [277:03:32<154:59:02,  6.48s/it]
 62%|██████▏   | 138868/225000 [277:03:38<151:43:18,  6.34s/it]
 62%|██████▏   | 138869/225000 [277:03:44<148:43:34,  6.22s/it]
 62%|██████▏   | 138870/225000 [277:03:50<142:50:53,  5.97s/it]
 62%|██████▏   | 138871/225000 [277:03:55<136:39:54,  5.71s/it]
 62%|██████▏   | 138872/225000 [277:04:00<133:55:30,  5.60s/it]
 62%|██████▏   | 138873/225000 [277:04:05<130:04:33,  5.44s/it]
 62%|██████▏   | 138874/225000 [277:04:10<128:44:07,  5.38s/it]
 62%|██████▏   | 138875/225000 [277:04:16<132:07:45,  5.52s/it]
 62%|██████▏   | 138876/225000 [277:04:25<159:04:13,  6.65s/it]
 62%|██████▏   | 138877/225000 [277:04:32<159:26:44,  6.66s/it]
 62%|██████▏   | 138878/225000 [277:04:37<145:29:18,  6.08s/it]
 62%|██████▏   | 138879/225000 [277:04:41<130:41:50,  5.46s/it]
 62%|██████▏   | 138880/225000 [277:04:45<121:25:35,  5.08s/it]
 62%|██████▏   | 138881/225000 [277:04:50<122:14:17,  5.11s/it]
 62%|██████▏   | 138882/225000 [277:05:03<174:50:56,  7.31s/it]
 62%|██████▏   | 138883/225000 [277:05:08<161:41:31,  6.76s/it]
 62%|██████▏   | 138884/225000 [277:05:12<138:12:55,  5.78s/it]
 62%|██████▏   | 138885/225000 [277:05:16<126:14:22,  5.28s/it]
 62%|██████▏   | 138886/225000 [277:05:21<123:40:08,  5.17s/it]
 62%|██████▏   | 138887/225000 [277:05:26<121:51:36,  5.09s/it]
 62%|██████▏   | 138888/225000 [277:05:31<120:50:11,  5.05s/it]
 62%|██████▏   | 138889/225000 [277:05:36<125:49:08,  5.26s/it]
 62%|██████▏   | 138890/225000 [277:05:49<181:28:08,  7.59s/it]
 62%|██████▏   | 138891/225000 [277:05:55<169:24:26,  7.08s/it]
 62%|██████▏   | 138892/225000 [277:06:03<170:49:14,  7.14s/it]
 62%|██████▏   | 138893/225000 [277:06:06<145:06:27,  6.07s/it]
 62%|██████▏   | 138894/225000 [277:06:10<132:05:27,  5.52s/it]
 62%|██████▏   | 138895/225000 [277:06:15<126:35:23,  5.29s/it]
 62%|██████▏   | 138896/225000 [277:06:20<122:55:05,  5.14s/it]
 62%|██████▏   | 138897/225000 [277:06:32<172:23:32,  7.21s/it]
 62%|██████▏   | 138898/225000 [277:06:37<159:20:13,  6.66s/it]
 62%|██████▏   | 138899/225000 [277:06:41<136:25:24,  5.70s/it]
 62%|██████▏   | 138900/225000 [277:06:45<127:36:37,  5.34s/it]
                                                               
{'loss': '0.7976', 'grad_norm': '1.125', 'learning_rate': '1.733e-05', 'epoch': '0.6173', 'num_input_tokens_seen': 459651392, 'train_runtime': '9.976e+05', 'train_tokens_per_second': '460.8'}

 62%|██████▏   | 138900/225000 [277:06:45<127:36:37,  5.34s/it]
 62%|██████▏   | 138901/225000 [277:06:54<148:46:02,  6.22s/it]
 62%|██████▏   | 138902/225000 [277:07:02<167:42:53,  7.01s/it]
 62%|██████▏   | 138903/225000 [277:07:06<140:43:47,  5.88s/it]
 62%|██████▏   | 138904/225000 [277:07:14<155:19:36,  6.49s/it]
 62%|██████▏   | 138905/225000 [277:07:19<149:11:49,  6.24s/it]
 62%|██████▏   | 138906/225000 [277:07:27<156:47:17,  6.56s/it]
 62%|██████▏   | 138907/225000 [277:07:33<159:27:53,  6.67s/it]
 62%|██████▏   | 138908/225000 [277:07:37<140:19:55,  5.87s/it]
 62%|██████▏   | 138909/225000 [277:07:41<123:00:08,  5.14s/it]
 62%|██████▏   | 138910/225000 [277:07:45<115:11:54,  4.82s/it]
 62%|██████▏   | 138911/225000 [277:07:50<114:39:52,  4.79s/it]
 62%|██████▏   | 138912/225000 [277:07:54<113:16:53,  4.74s/it]
 62%|██████▏   | 138913/225000 [277:08:06<163:15:35,  6.83s/it]
 62%|██████▏   | 138914/225000 [277:08:13<161:54:51,  6.77s/it]
 62%|██████▏   | 138915/225000 [277:08:17<142:49:00,  5.97s/it]
 62%|██████▏   | 138916/225000 [277:08:26<168:47:16,  7.06s/it]
 62%|██████▏   | 138917/225000 [277:08:30<143:32:16,  6.00s/it]
 62%|██████▏   | 138918/225000 [277:08:37<148:45:47,  6.22s/it]
 62%|██████▏   | 138919/225000 [277:08:42<145:40:00,  6.09s/it]
 62%|██████▏   | 138920/225000 [277:08:49<146:37:31,  6.13s/it]
 62%|██████▏   | 138921/225000 [277:08:54<140:50:41,  5.89s/it]
 62%|██████▏   | 138922/225000 [277:09:01<148:32:13,  6.21s/it]
 62%|██████▏   | 138923/225000 [277:09:07<149:58:07,  6.27s/it]
 62%|██████▏   | 138924/225000 [277:09:11<133:15:50,  5.57s/it]
 62%|██████▏   | 138925/225000 [277:09:16<124:09:20,  5.19s/it]
 62%|██████▏   | 138926/225000 [277:09:21<128:37:48,  5.38s/it]
 62%|██████▏   | 138927/225000 [277:09:32<166:09:56,  6.95s/it]
 62%|██████▏   | 138928/225000 [277:09:36<143:03:01,  5.98s/it]
 62%|██████▏   | 138929/225000 [277:09:43<154:04:25,  6.44s/it]
 62%|██████▏   | 138930/225000 [277:09:48<139:52:34,  5.85s/it]
 62%|██████▏   | 138931/225000 [277:09:55<150:15:34,  6.28s/it]
 62%|██████▏   | 138932/225000 [277:09:59<133:39:53,  5.59s/it]
 62%|██████▏   | 138933/225000 [277:10:06<142:16:42,  5.95s/it]
 62%|██████▏   | 138934/225000 [277:10:10<129:12:29,  5.40s/it]
 62%|██████▏   | 138935/225000 [277:10:14<120:28:31,  5.04s/it]
 62%|██████▏   | 138936/225000 [277:10:18<112:45:57,  4.72s/it]
 62%|██████▏   | 138937/225000 [277:10:23<114:19:52,  4.78s/it]
 62%|██████▏   | 138938/225000 [277:10:28<116:29:16,  4.87s/it]
 62%|██████▏   | 138939/225000 [277:10:33<118:18:28,  4.95s/it]
 62%|██████▏   | 138940/225000 [277:10:44<157:00:32,  6.57s/it]
 62%|██████▏   | 138941/225000 [277:10:52<168:15:53,  7.04s/it]
 62%|██████▏   | 138942/225000 [277:10:59<167:55:04,  7.02s/it]
 62%|██████▏   | 138943/225000 [277:11:06<167:54:41,  7.02s/it]
 62%|██████▏   | 138944/225000 [277:11:11<152:27:25,  6.38s/it]
 62%|██████▏   | 138945/225000 [277:11:16<143:36:40,  6.01s/it]
 62%|██████▏   | 138946/225000 [277:11:21<135:55:18,  5.69s/it]
 62%|██████▏   | 138947/225000 [277:11:26<132:29:24,  5.54s/it]
 62%|██████▏   | 138948/225000 [277:11:31<131:53:48,  5.52s/it]
 62%|██████▏   | 138949/225000 [277:11:36<128:33:25,  5.38s/it]
 62%|██████▏   | 138950/225000 [277:11:42<129:06:42,  5.40s/it]
                                                               
{'loss': '0.7859', 'grad_norm': '1.05', 'learning_rate': '1.731e-05', 'epoch': '0.6176', 'num_input_tokens_seen': 459781048, 'train_runtime': '9.979e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 138950/225000 [277:11:42<129:06:42,  5.40s/it]
 62%|██████▏   | 138951/225000 [277:11:47<128:11:11,  5.36s/it]
 62%|██████▏   | 138952/225000 [277:11:53<128:41:16,  5.38s/it]
 62%|██████▏   | 138953/225000 [277:12:05<176:16:14,  7.37s/it]
 62%|██████▏   | 138954/225000 [277:12:11<170:19:34,  7.13s/it]
 62%|██████▏   | 138955/225000 [277:12:15<148:35:37,  6.22s/it]
 62%|██████▏   | 138956/225000 [277:12:20<138:10:44,  5.78s/it]
 62%|██████▏   | 138957/225000 [277:12:28<153:55:24,  6.44s/it]
 62%|██████▏   | 138958/225000 [277:12:32<136:28:49,  5.71s/it]
 62%|██████▏   | 138959/225000 [277:12:36<124:38:58,  5.22s/it]
 62%|██████▏   | 138960/225000 [277:12:47<165:05:56,  6.91s/it]
 62%|██████▏   | 138961/225000 [277:12:52<148:50:52,  6.23s/it]
 62%|██████▏   | 138962/225000 [277:12:56<139:46:39,  5.85s/it]
 62%|██████▏   | 138963/225000 [277:13:01<131:21:55,  5.50s/it]
 62%|██████▏   | 138964/225000 [277:13:06<125:29:33,  5.25s/it]
 62%|██████▏   | 138965/225000 [277:13:11<122:30:46,  5.13s/it]
 62%|██████▏   | 138966/225000 [277:13:18<140:54:18,  5.90s/it]
 62%|██████▏   | 138967/225000 [277:13:29<174:59:44,  7.32s/it]
 62%|██████▏   | 138968/225000 [277:13:32<146:04:35,  6.11s/it]
 62%|██████▏   | 138969/225000 [277:13:41<164:39:14,  6.89s/it]
 62%|██████▏   | 138970/225000 [277:13:46<149:27:27,  6.25s/it]
 62%|██████▏   | 138971/225000 [277:13:51<141:43:53,  5.93s/it]
 62%|██████▏   | 138972/225000 [277:13:56<134:25:05,  5.62s/it]
 62%|██████▏   | 138973/225000 [277:14:00<126:36:43,  5.30s/it]
 62%|██████▏   | 138974/225000 [277:14:05<121:21:33,  5.08s/it]
 62%|██████▏   | 138975/225000 [277:14:09<115:30:48,  4.83s/it]
 62%|██████▏   | 138976/225000 [277:14:14<118:28:43,  4.96s/it]
 62%|██████▏   | 138977/225000 [277:14:27<175:05:43,  7.33s/it]
 62%|██████▏   | 138978/225000 [277:14:33<160:17:25,  6.71s/it]
 62%|██████▏   | 138979/225000 [277:14:36<140:07:21,  5.86s/it]
 62%|██████▏   | 138980/225000 [277:14:40<125:25:23,  5.25s/it]
 62%|██████▏   | 138981/225000 [277:14:44<116:00:34,  4.86s/it]
 62%|██████▏   | 138982/225000 [277:14:56<168:00:02,  7.03s/it]
 62%|██████▏   | 138983/225000 [277:15:02<156:32:37,  6.55s/it]
 62%|██████▏   | 138984/225000 [277:15:10<166:17:22,  6.96s/it]
 62%|██████▏   | 138985/225000 [277:15:13<143:40:50,  6.01s/it]
 62%|██████▏   | 138986/225000 [277:15:17<128:54:58,  5.40s/it]
 62%|██████▏   | 138987/225000 [277:15:21<118:03:38,  4.94s/it]
 62%|██████▏   | 138988/225000 [277:15:32<162:38:38,  6.81s/it]
 62%|██████▏   | 138989/225000 [277:15:37<145:03:15,  6.07s/it]
 62%|██████▏   | 138990/225000 [277:15:43<142:16:29,  5.95s/it]
 62%|██████▏   | 138991/225000 [277:15:53<175:34:06,  7.35s/it]
 62%|██████▏   | 138992/225000 [277:15:57<152:29:37,  6.38s/it]
 62%|██████▏   | 138993/225000 [277:16:01<132:15:41,  5.54s/it]
 62%|██████▏   | 138994/225000 [277:16:05<121:16:09,  5.08s/it]
 62%|██████▏   | 138995/225000 [277:16:15<157:50:37,  6.61s/it]
 62%|██████▏   | 138996/225000 [277:16:20<144:44:42,  6.06s/it]
 62%|██████▏   | 138997/225000 [277:16:27<154:30:49,  6.47s/it]
 62%|██████▏   | 138998/225000 [277:16:32<139:49:25,  5.85s/it]
 62%|██████▏   | 138999/225000 [277:16:35<125:29:11,  5.25s/it]
 62%|██████▏   | 139000/225000 [277:16:40<121:49:27,  5.10s/it]
                                                               
{'loss': '0.7882', 'grad_norm': '1.077', 'learning_rate': '1.73e-05', 'epoch': '0.6178', 'num_input_tokens_seen': 459911536, 'train_runtime': '9.982e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139000/225000 [277:16:40<121:49:27,  5.10s/it]
 62%|██████▏   | 139001/225000 [277:16:45<116:19:59,  4.87s/it]
 62%|██████▏   | 139002/225000 [277:16:49<112:24:22,  4.71s/it]
 62%|██████▏   | 139003/225000 [277:16:59<154:32:42,  6.47s/it]
 62%|██████▏   | 139004/225000 [277:17:08<168:35:57,  7.06s/it]
 62%|██████▏   | 139005/225000 [277:17:12<144:02:11,  6.03s/it]
 62%|██████▏   | 139006/225000 [277:17:21<169:38:10,  7.10s/it]
 62%|██████▏   | 139007/225000 [277:17:25<145:43:21,  6.10s/it]
 62%|██████▏   | 139008/225000 [277:17:30<140:28:15,  5.88s/it]
 62%|██████▏   | 139009/225000 [277:17:39<163:56:30,  6.86s/it]
 62%|██████▏   | 139010/225000 [277:17:44<150:32:36,  6.30s/it]
 62%|██████▏   | 139011/225000 [277:17:49<141:29:41,  5.92s/it]
 62%|██████▏   | 139012/225000 [277:17:54<131:37:40,  5.51s/it]
 62%|██████▏   | 139013/225000 [277:17:59<129:28:05,  5.42s/it]
 62%|██████▏   | 139014/225000 [277:18:04<127:40:59,  5.35s/it]
 62%|██████▏   | 139015/225000 [277:18:09<124:10:58,  5.20s/it]
 62%|██████▏   | 139016/225000 [277:18:15<126:09:56,  5.28s/it]
 62%|██████▏   | 139017/225000 [277:18:20<123:18:00,  5.16s/it]
 62%|██████▏   | 139018/225000 [277:18:24<120:24:28,  5.04s/it]
 62%|██████▏   | 139019/225000 [277:18:29<120:11:45,  5.03s/it]
 62%|██████▏   | 139020/225000 [277:18:41<169:06:49,  7.08s/it]
 62%|██████▏   | 139021/225000 [277:18:49<171:18:16,  7.17s/it]
 62%|██████▏   | 139022/225000 [277:18:52<142:45:46,  5.98s/it]
 62%|██████▏   | 139023/225000 [277:19:01<166:16:55,  6.96s/it]
 62%|██████▏   | 139024/225000 [277:19:06<149:17:32,  6.25s/it]
 62%|██████▏   | 139025/225000 [277:19:10<135:07:11,  5.66s/it]
 62%|██████▏   | 139026/225000 [277:19:15<129:24:20,  5.42s/it]
 62%|██████▏   | 139027/225000 [277:19:19<123:54:02,  5.19s/it]
 62%|██████▏   | 139028/225000 [277:19:24<122:43:47,  5.14s/it]
 62%|██████▏   | 139029/225000 [277:19:30<122:25:02,  5.13s/it]
 62%|██████▏   | 139030/225000 [277:19:41<169:14:43,  7.09s/it]
 62%|██████▏   | 139031/225000 [277:19:48<165:51:54,  6.95s/it]
 62%|██████▏   | 139032/225000 [277:19:51<140:15:55,  5.87s/it]
 62%|██████▏   | 139033/225000 [277:19:55<124:42:49,  5.22s/it]
 62%|██████▏   | 139034/225000 [277:20:05<161:03:51,  6.74s/it]
 62%|██████▏   | 139035/225000 [277:20:10<147:18:03,  6.17s/it]
 62%|██████▏   | 139036/225000 [277:20:15<135:47:37,  5.69s/it]
 62%|██████▏   | 139037/225000 [277:20:19<129:19:36,  5.42s/it]
 62%|██████▏   | 139038/225000 [277:20:24<125:37:52,  5.26s/it]
 62%|██████▏   | 139039/225000 [277:20:29<121:39:36,  5.10s/it]
 62%|██████▏   | 139040/225000 [277:20:40<167:03:52,  7.00s/it]
 62%|██████▏   | 139041/225000 [277:20:47<161:26:42,  6.76s/it]
 62%|██████▏   | 139042/225000 [277:20:50<139:39:10,  5.85s/it]
 62%|██████▏   | 139043/225000 [277:20:54<125:58:10,  5.28s/it]
 62%|██████▏   | 139044/225000 [277:21:00<128:17:22,  5.37s/it]
 62%|██████▏   | 139045/225000 [277:21:10<162:22:18,  6.80s/it]
 62%|██████▏   | 139046/225000 [277:21:13<137:16:35,  5.75s/it]
 62%|██████▏   | 139047/225000 [277:21:22<158:09:11,  6.62s/it]
 62%|██████▏   | 139048/225000 [277:21:26<138:35:36,  5.80s/it]
 62%|██████▏   | 139049/225000 [277:21:30<124:31:10,  5.22s/it]
 62%|██████▏   | 139050/225000 [277:21:39<157:08:34,  6.58s/it]
                                                               
{'loss': '0.7971', 'grad_norm': '1.1', 'learning_rate': '1.728e-05', 'epoch': '0.618', 'num_input_tokens_seen': 460040376, 'train_runtime': '9.985e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139050/225000 [277:21:39<157:08:34,  6.58s/it]
 62%|██████▏   | 139051/225000 [277:21:44<139:01:32,  5.82s/it]
 62%|██████▏   | 139052/225000 [277:21:48<127:35:18,  5.34s/it]
 62%|██████▏   | 139053/225000 [277:21:55<140:58:10,  5.90s/it]
 62%|██████▏   | 139054/225000 [277:22:04<162:17:03,  6.80s/it]
 62%|██████▏   | 139055/225000 [277:22:07<139:07:53,  5.83s/it]
 62%|██████▏   | 139056/225000 [277:22:18<172:59:48,  7.25s/it]
 62%|██████▏   | 139057/225000 [277:22:21<144:51:56,  6.07s/it]
 62%|██████▏   | 139058/225000 [277:22:26<138:34:09,  5.80s/it]
 62%|██████▏   | 139059/225000 [277:22:31<132:33:08,  5.55s/it]
 62%|██████▏   | 139060/225000 [277:22:36<128:21:20,  5.38s/it]
 62%|██████▏   | 139061/225000 [277:22:41<126:03:58,  5.28s/it]
 62%|██████▏   | 139062/225000 [277:22:47<127:01:19,  5.32s/it]
 62%|██████▏   | 139063/225000 [277:22:57<162:32:05,  6.81s/it]
 62%|██████▏   | 139064/225000 [277:23:02<150:46:50,  6.32s/it]
 62%|██████▏   | 139065/225000 [277:23:07<136:16:59,  5.71s/it]
 62%|██████▏   | 139066/225000 [277:23:12<131:14:52,  5.50s/it]
 62%|██████▏   | 139067/225000 [277:23:17<132:48:06,  5.56s/it]
 62%|██████▏   | 139068/225000 [277:23:22<129:08:55,  5.41s/it]
 62%|██████▏   | 139069/225000 [277:23:28<129:28:47,  5.42s/it]
 62%|██████▏   | 139070/225000 [277:23:33<130:02:40,  5.45s/it]
 62%|██████▏   | 139071/225000 [277:23:39<130:52:02,  5.48s/it]
 62%|██████▏   | 139072/225000 [277:23:45<132:41:15,  5.56s/it]
 62%|██████▏   | 139073/225000 [277:23:58<185:15:09,  7.76s/it]
 62%|██████▏   | 139074/225000 [277:24:04<174:51:27,  7.33s/it]
 62%|██████▏   | 139075/225000 [277:24:07<147:58:09,  6.20s/it]
 62%|██████▏   | 139076/225000 [277:24:12<135:47:14,  5.69s/it]
 62%|██████▏   | 139077/225000 [277:24:16<126:03:20,  5.28s/it]
 62%|██████▏   | 139078/225000 [277:24:29<180:40:01,  7.57s/it]
 62%|██████▏   | 139079/225000 [277:24:34<157:50:06,  6.61s/it]
 62%|██████▏   | 139080/225000 [277:24:38<139:16:47,  5.84s/it]
 62%|██████▏   | 139081/225000 [277:24:42<129:39:45,  5.43s/it]
 62%|██████▏   | 139082/225000 [277:24:46<120:15:36,  5.04s/it]
 62%|██████▏   | 139083/225000 [277:24:53<136:26:26,  5.72s/it]
 62%|██████▏   | 139084/225000 [277:25:03<166:50:01,  6.99s/it]
 62%|██████▏   | 139085/225000 [277:25:07<142:49:57,  5.98s/it]
 62%|██████▏   | 139086/225000 [277:25:12<136:34:13,  5.72s/it]
 62%|██████▏   | 139087/225000 [277:25:20<149:15:45,  6.25s/it]
 62%|██████▏   | 139088/225000 [277:25:23<130:12:59,  5.46s/it]
 62%|██████▏   | 139089/225000 [277:25:31<148:26:51,  6.22s/it]
 62%|██████▏   | 139090/225000 [277:25:37<142:17:03,  5.96s/it]
 62%|██████▏   | 139091/225000 [277:25:43<147:19:36,  6.17s/it]
 62%|██████▏   | 139092/225000 [277:25:50<152:04:42,  6.37s/it]
 62%|██████▏   | 139093/225000 [277:25:54<131:23:37,  5.51s/it]
 62%|██████▏   | 139094/225000 [277:26:03<160:27:59,  6.72s/it]
 62%|██████▏   | 139095/225000 [277:26:06<134:42:51,  5.65s/it]
 62%|██████▏   | 139096/225000 [277:26:12<133:35:09,  5.60s/it]
 62%|██████▏   | 139097/225000 [277:26:19<147:59:19,  6.20s/it]
 62%|██████▏   | 139098/225000 [277:26:24<137:01:24,  5.74s/it]
 62%|██████▏   | 139099/225000 [277:26:30<140:06:54,  5.87s/it]
 62%|██████▏   | 139100/225000 [277:26:34<124:06:50,  5.20s/it]
                                                               
{'loss': '0.8047', 'grad_norm': '1.12', 'learning_rate': '1.726e-05', 'epoch': '0.6182', 'num_input_tokens_seen': 460169808, 'train_runtime': '9.988e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139100/225000 [277:26:34<124:06:50,  5.20s/it]
 62%|██████▏   | 139101/225000 [277:26:39<125:04:58,  5.24s/it]
 62%|██████▏   | 139102/225000 [277:26:44<122:46:05,  5.15s/it]
 62%|██████▏   | 139103/225000 [277:26:49<123:51:02,  5.19s/it]
 62%|██████▏   | 139104/225000 [277:27:02<173:31:53,  7.27s/it]
 62%|██████▏   | 139105/225000 [277:27:06<155:48:03,  6.53s/it]
 62%|██████▏   | 139106/225000 [277:27:14<160:56:58,  6.75s/it]
 62%|██████▏   | 139107/225000 [277:27:20<161:36:04,  6.77s/it]
 62%|██████▏   | 139108/225000 [277:27:25<144:29:40,  6.06s/it]
 62%|██████▏   | 139109/225000 [277:27:29<130:41:36,  5.48s/it]
 62%|██████▏   | 139110/225000 [277:27:34<124:37:03,  5.22s/it]
 62%|██████▏   | 139111/225000 [277:27:40<133:32:32,  5.60s/it]
 62%|██████▏   | 139112/225000 [277:27:50<167:46:27,  7.03s/it]
 62%|██████▏   | 139113/225000 [277:27:54<143:17:19,  6.01s/it]
 62%|██████▏   | 139114/225000 [277:28:03<163:29:30,  6.85s/it]
 62%|██████▏   | 139115/225000 [277:28:08<154:24:33,  6.47s/it]
 62%|██████▏   | 139116/225000 [277:28:13<140:20:18,  5.88s/it]
 62%|██████▏   | 139117/225000 [277:28:19<142:57:45,  5.99s/it]
 62%|██████▏   | 139118/225000 [277:28:23<130:15:15,  5.46s/it]
 62%|██████▏   | 139119/225000 [277:28:28<121:26:40,  5.09s/it]
 62%|██████▏   | 139120/225000 [277:28:32<115:38:07,  4.85s/it]
 62%|██████▏   | 139121/225000 [277:28:42<155:35:08,  6.52s/it]
 62%|██████▏   | 139122/225000 [277:28:49<159:04:33,  6.67s/it]
 62%|██████▏   | 139123/225000 [277:28:53<137:57:47,  5.78s/it]
 62%|██████▏   | 139124/225000 [277:29:02<161:55:03,  6.79s/it]
 62%|██████▏   | 139125/225000 [277:29:06<139:57:30,  5.87s/it]
 62%|██████▏   | 139126/225000 [277:29:13<149:33:11,  6.27s/it]
 62%|██████▏   | 139127/225000 [277:29:17<130:21:26,  5.46s/it]
 62%|██████▏   | 139128/225000 [277:29:24<143:26:01,  6.01s/it]
 62%|██████▏   | 139129/225000 [277:29:30<142:22:29,  5.97s/it]
 62%|██████▏   | 139130/225000 [277:29:34<128:56:50,  5.41s/it]
 62%|██████▏   | 139131/225000 [277:29:43<157:45:52,  6.61s/it]
 62%|██████▏   | 139132/225000 [277:29:49<150:24:12,  6.31s/it]
 62%|██████▏   | 139133/225000 [277:29:53<132:19:05,  5.55s/it]
 62%|██████▏   | 139134/225000 [277:29:58<133:09:22,  5.58s/it]
 62%|██████▏   | 139135/225000 [277:30:09<166:14:04,  6.97s/it]
 62%|██████▏   | 139136/225000 [277:30:14<154:28:30,  6.48s/it]
 62%|██████▏   | 139137/225000 [277:30:19<147:14:49,  6.17s/it]
 62%|██████▏   | 139138/225000 [277:30:25<141:16:50,  5.92s/it]
 62%|██████▏   | 139139/225000 [277:30:30<136:27:31,  5.72s/it]
 62%|██████▏   | 139140/225000 [277:30:36<136:07:23,  5.71s/it]
 62%|██████▏   | 139141/225000 [277:30:41<134:24:14,  5.64s/it]
 62%|██████▏   | 139142/225000 [277:30:47<132:50:31,  5.57s/it]
 62%|██████▏   | 139143/225000 [277:30:52<128:00:10,  5.37s/it]
 62%|██████▏   | 139144/225000 [277:30:57<127:28:30,  5.35s/it]
 62%|██████▏   | 139145/225000 [277:31:02<122:43:26,  5.15s/it]
 62%|██████▏   | 139146/225000 [277:31:07<125:36:30,  5.27s/it]
 62%|██████▏   | 139147/225000 [277:31:12<126:40:41,  5.31s/it]
 62%|██████▏   | 139148/225000 [277:31:19<132:39:44,  5.56s/it]
 62%|██████▏   | 139149/225000 [277:31:31<182:18:30,  7.64s/it]
 62%|██████▏   | 139150/225000 [277:31:37<169:34:40,  7.11s/it]
                                                               
{'loss': '0.791', 'grad_norm': '1.035', 'learning_rate': '1.724e-05', 'epoch': '0.6184', 'num_input_tokens_seen': 460299960, 'train_runtime': '9.991e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139150/225000 [277:31:37<169:34:40,  7.11s/it]
 62%|██████▏   | 139151/225000 [277:31:42<156:19:52,  6.56s/it]
 62%|██████▏   | 139152/225000 [277:31:48<148:14:11,  6.22s/it]
 62%|██████▏   | 139153/225000 [277:31:52<137:00:04,  5.75s/it]
 62%|██████▏   | 139154/225000 [277:31:57<132:47:55,  5.57s/it]
 62%|██████▏   | 139155/225000 [277:32:03<131:58:22,  5.53s/it]
 62%|██████▏   | 139156/225000 [277:32:09<132:14:15,  5.55s/it]
 62%|██████▏   | 139157/225000 [277:32:13<127:51:54,  5.36s/it]
 62%|██████▏   | 139158/225000 [277:32:19<127:59:39,  5.37s/it]
 62%|██████▏   | 139159/225000 [277:32:24<126:53:40,  5.32s/it]
 62%|██████▏   | 139160/225000 [277:32:32<145:08:16,  6.09s/it]
 62%|██████▏   | 139161/225000 [277:32:44<190:11:30,  7.98s/it]
 62%|██████▏   | 139162/225000 [277:32:50<175:48:00,  7.37s/it]
 62%|██████▏   | 139163/225000 [277:32:54<152:50:47,  6.41s/it]
 62%|██████▏   | 139164/225000 [277:32:59<140:38:45,  5.90s/it]
 62%|██████▏   | 139165/225000 [277:33:04<131:54:29,  5.53s/it]
 62%|██████▏   | 139166/225000 [277:33:08<123:09:47,  5.17s/it]
 62%|██████▏   | 139167/225000 [277:33:18<160:26:01,  6.73s/it]
 62%|██████▏   | 139168/225000 [277:33:26<167:17:40,  7.02s/it]
 62%|██████▏   | 139169/225000 [277:33:30<147:06:49,  6.17s/it]
 62%|██████▏   | 139170/225000 [277:33:35<134:03:21,  5.62s/it]
 62%|██████▏   | 139171/225000 [277:33:39<124:49:25,  5.24s/it]
 62%|██████▏   | 139172/225000 [277:33:51<170:27:49,  7.15s/it]
 62%|██████▏   | 139173/225000 [277:33:56<157:16:29,  6.60s/it]
 62%|██████▏   | 139174/225000 [277:33:59<134:20:10,  5.63s/it]
 62%|██████▏   | 139175/225000 [277:34:09<162:55:57,  6.83s/it]
 62%|██████▏   | 139176/225000 [277:34:13<145:06:02,  6.09s/it]
 62%|██████▏   | 139177/225000 [277:34:18<134:25:30,  5.64s/it]
 62%|██████▏   | 139178/225000 [277:34:22<125:47:19,  5.28s/it]
 62%|██████▏   | 139179/225000 [277:34:27<117:38:17,  4.93s/it]
 62%|██████▏   | 139180/225000 [277:34:37<155:14:22,  6.51s/it]
 62%|██████▏   | 139181/225000 [277:34:45<166:38:07,  6.99s/it]
 62%|██████▏   | 139182/225000 [277:34:48<142:03:54,  5.96s/it]
 62%|██████▏   | 139183/225000 [277:34:57<160:06:03,  6.72s/it]
 62%|██████▏   | 139184/225000 [277:35:00<137:04:21,  5.75s/it]
 62%|██████▏   | 139185/225000 [277:35:10<168:29:59,  7.07s/it]
 62%|██████▏   | 139186/225000 [277:35:16<156:08:17,  6.55s/it]
 62%|██████▏   | 139187/225000 [277:35:21<148:04:14,  6.21s/it]
 62%|██████▏   | 139188/225000 [277:35:27<144:53:12,  6.08s/it]
 62%|██████▏   | 139189/225000 [277:35:34<153:19:46,  6.43s/it]
 62%|██████▏   | 139190/225000 [277:35:41<153:03:38,  6.42s/it]
 62%|██████▏   | 139191/225000 [277:35:45<137:26:31,  5.77s/it]
 62%|██████▏   | 139192/225000 [277:35:49<125:58:42,  5.29s/it]
 62%|██████▏   | 139193/225000 [277:35:53<118:39:58,  4.98s/it]
 62%|██████▏   | 139194/225000 [277:35:58<115:17:52,  4.84s/it]
 62%|██████▏   | 139195/225000 [277:36:10<170:17:36,  7.14s/it]
 62%|██████▏   | 139196/225000 [277:36:16<159:15:33,  6.68s/it]
 62%|██████▏   | 139197/225000 [277:36:20<137:56:23,  5.79s/it]
 62%|██████▏   | 139198/225000 [277:36:23<122:57:42,  5.16s/it]
 62%|██████▏   | 139199/225000 [277:36:36<173:25:33,  7.28s/it]
 62%|██████▏   | 139200/225000 [277:36:40<150:43:16,  6.32s/it]
                                                               
{'loss': '0.7965', 'grad_norm': '1.078', 'learning_rate': '1.723e-05', 'epoch': '0.6187', 'num_input_tokens_seen': 460429720, 'train_runtime': '9.994e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139200/225000 [277:36:40<150:43:16,  6.32s/it]
 62%|██████▏   | 139201/225000 [277:36:47<156:02:14,  6.55s/it]
 62%|██████▏   | 139202/225000 [277:36:50<135:17:07,  5.68s/it]
 62%|██████▏   | 139203/225000 [277:36:58<149:14:44,  6.26s/it]
 62%|██████▏   | 139204/225000 [277:37:02<134:36:32,  5.65s/it]
 62%|██████▏   | 139205/225000 [277:37:10<148:22:01,  6.23s/it]
 62%|██████▏   | 139206/225000 [277:37:14<130:31:39,  5.48s/it]
 62%|██████▏   | 139207/225000 [277:37:18<124:05:51,  5.21s/it]
 62%|██████▏   | 139208/225000 [277:37:27<150:10:34,  6.30s/it]
 62%|██████▏   | 139209/225000 [277:37:33<147:08:55,  6.17s/it]
 62%|██████▏   | 139210/225000 [277:37:38<140:28:32,  5.89s/it]
 62%|██████▏   | 139211/225000 [277:37:43<131:07:53,  5.50s/it]
 62%|██████▏   | 139212/225000 [277:37:48<132:05:20,  5.54s/it]
 62%|██████▏   | 139213/225000 [277:37:53<128:32:25,  5.39s/it]
 62%|██████▏   | 139214/225000 [277:37:58<125:51:26,  5.28s/it]
 62%|██████▏   | 139215/225000 [277:38:03<123:04:12,  5.16s/it]
 62%|██████▏   | 139216/225000 [277:38:08<122:44:42,  5.15s/it]
 62%|██████▏   | 139217/225000 [277:38:14<126:03:45,  5.29s/it]
 62%|██████▏   | 139218/225000 [277:38:19<126:08:21,  5.29s/it]
 62%|██████▏   | 139219/225000 [277:38:25<128:59:36,  5.41s/it]
 62%|██████▏   | 139220/225000 [277:38:31<130:10:00,  5.46s/it]
 62%|██████▏   | 139221/225000 [277:38:36<131:10:23,  5.51s/it]
 62%|██████▏   | 139222/225000 [277:38:41<129:13:53,  5.42s/it]
 62%|██████▏   | 139223/225000 [277:38:47<129:56:41,  5.45s/it]
 62%|██████▏   | 139224/225000 [277:38:57<165:01:16,  6.93s/it]
 62%|██████▏   | 139225/225000 [277:39:07<187:30:46,  7.87s/it]
 62%|██████▏   | 139226/225000 [277:39:11<159:47:43,  6.71s/it]
 62%|██████▏   | 139227/225000 [277:39:20<172:36:55,  7.24s/it]
 62%|██████▏   | 139228/225000 [277:39:25<156:04:20,  6.55s/it]
 62%|██████▏   | 139229/225000 [277:39:30<143:29:15,  6.02s/it]
 62%|██████▏   | 139230/225000 [277:39:34<133:49:57,  5.62s/it]
 62%|██████▏   | 139231/225000 [277:39:38<122:01:57,  5.12s/it]
 62%|██████▏   | 139232/225000 [277:39:43<117:39:05,  4.94s/it]
 62%|██████▏   | 139233/225000 [277:39:56<173:36:28,  7.29s/it]
 62%|██████▏   | 139234/225000 [277:39:59<143:48:13,  6.04s/it]
 62%|██████▏   | 139235/225000 [277:40:05<143:02:02,  6.00s/it]
 62%|██████▏   | 139236/225000 [277:40:12<150:09:37,  6.30s/it]
 62%|██████▏   | 139237/225000 [277:40:15<129:54:54,  5.45s/it]
 62%|██████▏   | 139238/225000 [277:40:24<153:03:31,  6.42s/it]
 62%|██████▏   | 139239/225000 [277:40:28<139:39:32,  5.86s/it]
 62%|██████▏   | 139240/225000 [277:40:32<122:02:31,  5.12s/it]
 62%|██████▏   | 139241/225000 [277:40:42<155:43:35,  6.54s/it]
 62%|██████▏   | 139242/225000 [277:40:45<131:57:56,  5.54s/it]
 62%|██████▏   | 139243/225000 [277:40:54<160:21:15,  6.73s/it]
 62%|██████▏   | 139244/225000 [277:40:58<135:47:15,  5.70s/it]
 62%|██████▏   | 139245/225000 [277:41:05<151:19:53,  6.35s/it]
 62%|██████▏   | 139246/225000 [277:41:11<146:20:39,  6.14s/it]
 62%|██████▏   | 139247/225000 [277:41:18<149:11:50,  6.26s/it]
 62%|██████▏   | 139248/225000 [277:41:23<141:21:55,  5.93s/it]
 62%|██████▏   | 139249/225000 [277:41:27<131:57:39,  5.54s/it]
 62%|██████▏   | 139250/225000 [277:41:33<133:46:41,  5.62s/it]
                                                               
{'loss': '0.8095', 'grad_norm': '0.9926', 'learning_rate': '1.721e-05', 'epoch': '0.6189', 'num_input_tokens_seen': 460558688, 'train_runtime': '9.997e+05', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139250/225000 [277:41:33<133:46:41,  5.62s/it]
 62%|██████▏   | 139251/225000 [277:41:38<129:24:28,  5.43s/it]
 62%|██████▏   | 139252/225000 [277:41:44<131:36:29,  5.53s/it]
 62%|██████▏   | 139253/225000 [277:41:52<149:37:22,  6.28s/it]
 62%|██████▏   | 139254/225000 [277:41:56<132:28:39,  5.56s/it]
 62%|██████▏   | 139255/225000 [277:42:05<158:46:18,  6.67s/it]
 62%|██████▏   | 139256/225000 [277:42:09<138:32:50,  5.82s/it]
 62%|██████▏   | 139257/225000 [277:42:13<123:27:44,  5.18s/it]
 62%|██████▏   | 139258/225000 [277:42:21<143:47:36,  6.04s/it]
 62%|██████▏   | 139259/225000 [277:42:28<153:25:51,  6.44s/it]
 62%|██████▏   | 139260/225000 [277:42:32<133:52:59,  5.62s/it]
 62%|██████▏   | 139261/225000 [277:42:36<121:21:34,  5.10s/it]
 62%|██████▏   | 139262/225000 [277:42:41<121:00:57,  5.08s/it]
 62%|██████▏   | 139263/225000 [277:42:52<168:25:43,  7.07s/it]
 62%|██████▏   | 139264/225000 [277:42:56<141:43:37,  5.95s/it]
 62%|██████▏   | 139265/225000 [277:43:00<133:14:10,  5.59s/it]
 62%|██████▏   | 139266/225000 [277:43:05<127:31:37,  5.35s/it]
 62%|██████▏   | 139267/225000 [277:43:10<123:49:16,  5.20s/it]
 62%|██████▏   | 139268/225000 [277:43:16<125:11:18,  5.26s/it]
 62%|██████▏   | 139269/225000 [277:43:20<120:54:48,  5.08s/it]
 62%|██████▏   | 139270/225000 [277:43:25<120:50:35,  5.07s/it]
 62%|██████▏   | 139271/225000 [277:43:38<172:20:34,  7.24s/it]
 62%|██████▏   | 139272/225000 [277:43:44<163:53:05,  6.88s/it]
 62%|██████▏   | 139273/225000 [277:43:50<160:45:09,  6.75s/it]
 62%|██████▏   | 139274/225000 [277:43:57<162:47:52,  6.84s/it]
 62%|██████▏   | 139275/225000 [277:44:01<140:49:34,  5.91s/it]
 62%|██████▏   | 139276/225000 [277:44:05<128:11:52,  5.38s/it]
 62%|██████▏   | 139277/225000 [277:44:14<157:30:33,  6.61s/it]
 62%|██████▏   | 139278/225000 [277:44:20<147:28:16,  6.19s/it]
 62%|██████▏   | 139279/225000 [277:44:24<130:42:40,  5.49s/it]
 62%|██████▏   | 139280/225000 [277:44:29<127:40:27,  5.36s/it]
 62%|██████▏   | 139281/225000 [277:44:38<159:43:49,  6.71s/it]
 62%|██████▏   | 139282/225000 [277:44:43<142:35:17,  5.99s/it]
 62%|██████▏   | 139283/225000 [277:44:46<125:51:12,  5.29s/it]
 62%|██████▏   | 139284/225000 [277:44:51<118:28:51,  4.98s/it]
 62%|██████▏   | 139285/225000 [277:44:59<145:53:09,  6.13s/it]
 62%|██████▏   | 139286/225000 [277:45:07<157:43:53,  6.62s/it]
 62%|██████▏   | 139287/225000 [277:45:12<141:10:51,  5.93s/it]
 62%|██████▏   | 139288/225000 [277:45:16<133:45:10,  5.62s/it]
 62%|██████▏   | 139289/225000 [277:45:21<125:48:09,  5.28s/it]
 62%|██████▏   | 139290/225000 [277:45:25<119:50:31,  5.03s/it]
 62%|██████▏   | 139291/225000 [277:45:30<118:17:28,  4.97s/it]
 62%|██████▏   | 139292/225000 [277:45:39<143:25:43,  6.02s/it]
 62%|██████▏   | 139293/225000 [277:45:49<170:39:39,  7.17s/it]
 62%|██████▏   | 139294/225000 [277:45:53<149:40:32,  6.29s/it]
 62%|██████▏   | 139295/225000 [277:45:57<134:35:18,  5.65s/it]
 62%|██████▏   | 139296/225000 [277:46:02<127:40:44,  5.36s/it]
 62%|██████▏   | 139297/225000 [277:46:09<139:29:22,  5.86s/it]
 62%|██████▏   | 139298/225000 [277:46:17<159:27:31,  6.70s/it]
 62%|██████▏   | 139299/225000 [277:46:21<135:01:13,  5.67s/it]
 62%|██████▏   | 139300/225000 [277:46:28<147:37:54,  6.20s/it]
                                                               
{'loss': '0.8', 'grad_norm': '1.3', 'learning_rate': '1.719e-05', 'epoch': '0.6191', 'num_input_tokens_seen': 460688136, 'train_runtime': '1e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139300/225000 [277:46:28<147:37:54,  6.20s/it]
 62%|██████▏   | 139301/225000 [277:46:31<127:23:36,  5.35s/it]
 62%|██████▏   | 139302/225000 [277:46:41<159:32:09,  6.70s/it]
 62%|██████▏   | 139303/225000 [277:46:45<139:03:15,  5.84s/it]
 62%|██████▏   | 139304/225000 [277:46:53<156:32:47,  6.58s/it]
 62%|██████▏   | 139305/225000 [277:46:58<143:57:43,  6.05s/it]
 62%|██████▏   | 139306/225000 [277:47:03<133:03:33,  5.59s/it]
 62%|██████▏   | 139307/225000 [277:47:07<122:56:22,  5.16s/it]
 62%|██████▏   | 139308/225000 [277:47:13<127:47:46,  5.37s/it]
 62%|██████▏   | 139309/225000 [277:47:25<177:26:05,  7.45s/it]
 62%|██████▏   | 139310/225000 [277:47:29<152:11:02,  6.39s/it]
 62%|██████▏   | 139311/225000 [277:47:34<145:06:25,  6.10s/it]
 62%|██████▏   | 139312/225000 [277:47:38<131:05:21,  5.51s/it]
 62%|██████▏   | 139313/225000 [277:47:44<128:55:45,  5.42s/it]
 62%|██████▏   | 139314/225000 [277:47:49<125:26:38,  5.27s/it]
 62%|██████▏   | 139315/225000 [277:47:54<125:05:42,  5.26s/it]
 62%|██████▏   | 139316/225000 [277:47:59<122:47:56,  5.16s/it]
 62%|██████▏   | 139317/225000 [277:48:04<122:51:25,  5.16s/it]
 62%|██████▏   | 139318/225000 [277:48:09<119:40:57,  5.03s/it]
 62%|██████▏   | 139319/225000 [277:48:13<118:05:32,  4.96s/it]
 62%|██████▏   | 139320/225000 [277:48:19<119:11:22,  5.01s/it]
 62%|██████▏   | 139321/225000 [277:48:24<120:03:46,  5.04s/it]
 62%|██████▏   | 139322/225000 [277:48:29<122:23:22,  5.14s/it]
 62%|██████▏   | 139323/225000 [277:48:40<166:24:30,  6.99s/it]
 62%|██████▏   | 139324/225000 [277:48:50<185:20:04,  7.79s/it]
 62%|██████▏   | 139325/225000 [277:48:53<151:03:58,  6.35s/it]
 62%|██████▏   | 139326/225000 [277:49:00<156:03:59,  6.56s/it]
 62%|██████▏   | 139327/225000 [277:49:07<156:21:53,  6.57s/it]
 62%|██████▏   | 139328/225000 [277:49:11<141:45:51,  5.96s/it]
 62%|██████▏   | 139329/225000 [277:49:18<148:44:58,  6.25s/it]
 62%|██████▏   | 139330/225000 [277:49:24<145:58:05,  6.13s/it]
 62%|██████▏   | 139331/225000 [277:49:31<151:09:23,  6.35s/it]
 62%|██████▏   | 139332/225000 [277:49:34<130:21:37,  5.48s/it]
 62%|██████▏   | 139333/225000 [277:49:41<142:30:36,  5.99s/it]
 62%|██████▏   | 139334/225000 [277:49:46<129:03:13,  5.42s/it]
 62%|██████▏   | 139335/225000 [277:49:51<129:33:53,  5.44s/it]
 62%|██████▏   | 139336/225000 [277:49:57<131:36:57,  5.53s/it]
 62%|██████▏   | 139337/225000 [277:50:05<151:37:15,  6.37s/it]
 62%|██████▏   | 139338/225000 [277:50:09<131:42:48,  5.54s/it]
 62%|██████▏   | 139339/225000 [277:50:13<125:51:20,  5.29s/it]
 62%|██████▏   | 139340/225000 [277:50:24<163:35:01,  6.87s/it]
 62%|██████▏   | 139341/225000 [277:50:28<142:26:33,  5.99s/it]
 62%|██████▏   | 139342/225000 [277:50:35<146:43:36,  6.17s/it]
 62%|██████▏   | 139343/225000 [277:50:38<126:37:55,  5.32s/it]
 62%|██████▏   | 139344/225000 [277:50:43<123:39:54,  5.20s/it]
 62%|██████▏   | 139345/225000 [277:50:52<150:17:42,  6.32s/it]
 62%|██████▏   | 139346/225000 [277:50:56<133:49:37,  5.62s/it]
 62%|██████▏   | 139347/225000 [277:50:59<118:54:33,  5.00s/it]
 62%|██████▏   | 139348/225000 [277:51:04<117:41:34,  4.95s/it]
 62%|██████▏   | 139349/225000 [277:51:11<131:09:52,  5.51s/it]
 62%|██████▏   | 139350/225000 [277:51:22<173:07:10,  7.28s/it]
                                                               
{'loss': '0.7968', 'grad_norm': '1.164', 'learning_rate': '1.718e-05', 'epoch': '0.6193', 'num_input_tokens_seen': 460818176, 'train_runtime': '1e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139350/225000 [277:51:22<173:07:10,  7.28s/it]
 62%|██████▏   | 139351/225000 [277:51:26<147:40:57,  6.21s/it]
 62%|██████▏   | 139352/225000 [277:51:30<128:54:12,  5.42s/it]
 62%|██████▏   | 139353/225000 [277:51:34<119:34:28,  5.03s/it]
 62%|██████▏   | 139354/225000 [277:51:44<157:58:09,  6.64s/it]
 62%|██████▏   | 139355/225000 [277:51:48<141:03:04,  5.93s/it]
 62%|██████▏   | 139356/225000 [277:51:56<154:36:43,  6.50s/it]
 62%|██████▏   | 139357/225000 [277:52:02<153:02:10,  6.43s/it]
 62%|██████▏   | 139358/225000 [277:52:07<137:38:02,  5.79s/it]
 62%|██████▏   | 139359/225000 [277:52:12<134:46:54,  5.67s/it]
 62%|██████▏   | 139360/225000 [277:52:17<130:36:03,  5.49s/it]
 62%|██████▏   | 139361/225000 [277:52:22<128:52:44,  5.42s/it]
 62%|██████▏   | 139362/225000 [277:52:28<128:19:36,  5.39s/it]
 62%|██████▏   | 139363/225000 [277:52:33<128:00:13,  5.38s/it]
 62%|██████▏   | 139364/225000 [277:52:38<126:11:21,  5.30s/it]
 62%|██████▏   | 139365/225000 [277:52:49<165:53:43,  6.97s/it]
 62%|██████▏   | 139366/225000 [277:52:55<157:10:33,  6.61s/it]
 62%|██████▏   | 139367/225000 [277:53:00<143:26:10,  6.03s/it]
 62%|██████▏   | 139368/225000 [277:53:06<142:44:43,  6.00s/it]
 62%|██████▏   | 139369/225000 [277:53:12<146:04:42,  6.14s/it]
 62%|██████▏   | 139370/225000 [277:53:18<146:43:13,  6.17s/it]
 62%|██████▏   | 139371/225000 [277:53:24<141:12:06,  5.94s/it]
 62%|██████▏   | 139372/225000 [277:53:27<124:41:26,  5.24s/it]
 62%|██████▏   | 139373/225000 [277:53:36<153:20:40,  6.45s/it]
 62%|██████▏   | 139374/225000 [277:53:41<141:08:50,  5.93s/it]
 62%|██████▏   | 139375/225000 [277:53:46<132:46:48,  5.58s/it]
 62%|██████▏   | 139376/225000 [277:53:51<126:52:18,  5.33s/it]
 62%|██████▏   | 139377/225000 [277:53:55<118:56:26,  5.00s/it]
 62%|██████▏   | 139378/225000 [277:53:59<112:04:07,  4.71s/it]
 62%|██████▏   | 139379/225000 [277:54:05<121:00:50,  5.09s/it]
 62%|██████▏   | 139380/225000 [277:54:15<157:26:03,  6.62s/it]
 62%|██████▏   | 139381/225000 [277:54:19<134:38:10,  5.66s/it]
 62%|██████▏   | 139382/225000 [277:54:28<163:45:15,  6.89s/it]
 62%|██████▏   | 139383/225000 [277:54:32<139:48:37,  5.88s/it]
 62%|██████▏   | 139384/225000 [277:54:37<134:52:45,  5.67s/it]
 62%|██████▏   | 139385/225000 [277:54:45<149:44:48,  6.30s/it]
 62%|██████▏   | 139386/225000 [277:54:49<136:51:04,  5.75s/it]
 62%|██████▏   | 139387/225000 [277:54:54<131:57:29,  5.55s/it]
 62%|██████▏   | 139388/225000 [277:54:59<123:57:23,  5.21s/it]
 62%|██████▏   | 139389/225000 [277:55:03<119:17:34,  5.02s/it]
 62%|██████▏   | 139390/225000 [277:55:08<119:39:07,  5.03s/it]
 62%|██████▏   | 139391/225000 [277:55:16<139:21:32,  5.86s/it]
 62%|██████▏   | 139392/225000 [277:55:25<163:28:35,  6.87s/it]
 62%|██████▏   | 139393/225000 [277:55:29<141:28:08,  5.95s/it]
 62%|██████▏   | 139394/225000 [277:55:38<159:12:28,  6.70s/it]
 62%|██████▏   | 139395/225000 [277:55:42<138:57:02,  5.84s/it]
 62%|██████▏   | 139396/225000 [277:55:45<125:05:07,  5.26s/it]
 62%|██████▏   | 139397/225000 [277:55:50<122:24:26,  5.15s/it]
 62%|██████▏   | 139398/225000 [277:56:01<160:16:46,  6.74s/it]
 62%|██████▏   | 139399/225000 [277:56:07<154:20:01,  6.49s/it]
 62%|██████▏   | 139400/225000 [277:56:11<137:14:15,  5.77s/it]
                                                               
{'loss': '0.7989', 'grad_norm': '1.057', 'learning_rate': '1.716e-05', 'epoch': '0.6196', 'num_input_tokens_seen': 460948704, 'train_runtime': '1.001e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139400/225000 [277:56:11<137:14:15,  5.77s/it]
 62%|██████▏   | 139401/225000 [277:56:14<122:08:47,  5.14s/it]
 62%|██████▏   | 139402/225000 [277:56:19<114:51:17,  4.83s/it]
 62%|██████▏   | 139403/225000 [277:56:30<162:26:58,  6.83s/it]
 62%|██████▏   | 139404/225000 [277:56:34<139:20:20,  5.86s/it]
 62%|██████▏   | 139405/225000 [277:56:37<124:40:54,  5.24s/it]
 62%|██████▏   | 139406/225000 [277:56:44<137:01:44,  5.76s/it]
 62%|██████▏   | 139407/225000 [277:56:51<143:54:46,  6.05s/it]
 62%|██████▏   | 139408/225000 [277:56:59<154:14:23,  6.49s/it]
 62%|██████▏   | 139409/225000 [277:57:02<131:16:23,  5.52s/it]
 62%|██████▏   | 139410/225000 [277:57:10<146:11:59,  6.15s/it]
 62%|██████▏   | 139411/225000 [277:57:14<131:22:33,  5.53s/it]
 62%|██████▏   | 139412/225000 [277:57:18<122:01:47,  5.13s/it]
 62%|██████▏   | 139413/225000 [277:57:24<128:37:30,  5.41s/it]
 62%|██████▏   | 139414/225000 [277:57:33<151:31:57,  6.37s/it]
 62%|██████▏   | 139415/225000 [277:57:36<133:25:15,  5.61s/it]
 62%|██████▏   | 139416/225000 [277:57:46<161:45:56,  6.80s/it]
 62%|██████▏   | 139417/225000 [277:57:50<139:45:55,  5.88s/it]
 62%|██████▏   | 139418/225000 [277:57:58<159:29:46,  6.71s/it]
 62%|██████▏   | 139419/225000 [277:58:02<135:54:35,  5.72s/it]
 62%|██████▏   | 139420/225000 [277:58:09<149:11:37,  6.28s/it]
 62%|██████▏   | 139421/225000 [277:58:13<131:57:39,  5.55s/it]
 62%|██████▏   | 139422/225000 [277:58:17<123:16:47,  5.19s/it]
 62%|██████▏   | 139423/225000 [277:58:21<114:23:06,  4.81s/it]
 62%|██████▏   | 139424/225000 [277:58:25<108:24:18,  4.56s/it]
 62%|██████▏   | 139425/225000 [277:58:36<154:55:10,  6.52s/it]
 62%|██████▏   | 139426/225000 [277:58:40<135:53:29,  5.72s/it]
 62%|██████▏   | 139427/225000 [277:58:44<122:34:00,  5.16s/it]
 62%|██████▏   | 139428/225000 [277:58:56<172:11:02,  7.24s/it]
 62%|██████▏   | 139429/225000 [277:59:01<153:34:15,  6.46s/it]
 62%|██████▏   | 139430/225000 [277:59:06<140:59:42,  5.93s/it]
 62%|██████▏   | 139431/225000 [277:59:10<131:58:21,  5.55s/it]
 62%|██████▏   | 139432/225000 [277:59:14<122:27:52,  5.15s/it]
 62%|██████▏   | 139433/225000 [277:59:19<119:39:08,  5.03s/it]
 62%|██████▏   | 139434/225000 [277:59:29<152:40:26,  6.42s/it]
 62%|██████▏   | 139435/225000 [277:59:35<148:48:48,  6.26s/it]
 62%|██████▏   | 139436/225000 [277:59:40<144:20:41,  6.07s/it]
 62%|██████▏   | 139437/225000 [277:59:44<129:51:32,  5.46s/it]
 62%|██████▏   | 139438/225000 [277:59:49<121:12:40,  5.10s/it]
 62%|██████▏   | 139439/225000 [278:00:01<169:21:28,  7.13s/it]
 62%|██████▏   | 139440/225000 [278:00:04<144:53:33,  6.10s/it]
 62%|██████▏   | 139441/225000 [278:00:12<158:54:47,  6.69s/it]
 62%|██████▏   | 139442/225000 [278:00:16<138:34:36,  5.83s/it]
 62%|██████▏   | 139443/225000 [278:00:20<123:01:48,  5.18s/it]
 62%|██████▏   | 139444/225000 [278:00:26<129:07:31,  5.43s/it]
 62%|██████▏   | 139445/225000 [278:00:35<155:44:43,  6.55s/it]
 62%|██████▏   | 139446/225000 [278:00:42<155:29:15,  6.54s/it]
 62%|██████▏   | 139447/225000 [278:00:45<134:23:17,  5.65s/it]
 62%|██████▏   | 139448/225000 [278:00:50<131:45:02,  5.54s/it]
 62%|██████▏   | 139449/225000 [278:00:57<141:53:20,  5.97s/it]
 62%|██████▏   | 139450/225000 [278:01:01<123:58:15,  5.22s/it]
                                                               
{'loss': '0.8019', 'grad_norm': '1.28', 'learning_rate': '1.714e-05', 'epoch': '0.6198', 'num_input_tokens_seen': 461078392, 'train_runtime': '1.001e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139450/225000 [278:01:01<123:58:15,  5.22s/it]
 62%|██████▏   | 139451/225000 [278:01:10<153:39:05,  6.47s/it]
 62%|██████▏   | 139452/225000 [278:01:14<136:59:14,  5.76s/it]
 62%|██████▏   | 139453/225000 [278:01:19<126:39:42,  5.33s/it]
 62%|██████▏   | 139454/225000 [278:01:23<117:26:00,  4.94s/it]
 62%|██████▏   | 139455/225000 [278:01:28<119:42:14,  5.04s/it]
 62%|██████▏   | 139456/225000 [278:01:37<146:06:18,  6.15s/it]
 62%|██████▏   | 139457/225000 [278:01:43<148:10:29,  6.24s/it]
 62%|██████▏   | 139458/225000 [278:01:49<142:14:54,  5.99s/it]
 62%|██████▏   | 139459/225000 [278:01:53<131:11:57,  5.52s/it]
 62%|██████▏   | 139460/225000 [278:02:01<150:03:53,  6.32s/it]
 62%|██████▏   | 139461/225000 [278:02:06<141:26:21,  5.95s/it]
 62%|██████▏   | 139462/225000 [278:02:11<134:32:20,  5.66s/it]
 62%|██████▏   | 139463/225000 [278:02:16<130:16:19,  5.48s/it]
 62%|██████▏   | 139464/225000 [278:02:21<126:35:40,  5.33s/it]
 62%|██████▏   | 139465/225000 [278:02:26<125:04:41,  5.26s/it]
 62%|██████▏   | 139466/225000 [278:02:31<123:43:51,  5.21s/it]
 62%|██████▏   | 139467/225000 [278:02:37<124:29:25,  5.24s/it]
 62%|██████▏   | 139468/225000 [278:02:41<118:17:47,  4.98s/it]
 62%|██████▏   | 139469/225000 [278:02:46<118:57:10,  5.01s/it]
 62%|██████▏   | 139470/225000 [278:02:51<119:56:10,  5.05s/it]
 62%|██████▏   | 139471/225000 [278:02:57<120:52:44,  5.09s/it]
 62%|██████▏   | 139472/225000 [278:03:03<128:44:24,  5.42s/it]
 62%|██████▏   | 139473/225000 [278:03:15<177:26:42,  7.47s/it]
 62%|██████▏   | 139474/225000 [278:03:19<155:03:45,  6.53s/it]
 62%|██████▏   | 139475/225000 [278:03:24<138:56:35,  5.85s/it]
 62%|██████▏   | 139476/225000 [278:03:28<126:36:53,  5.33s/it]
 62%|██████▏   | 139477/225000 [278:03:32<120:58:42,  5.09s/it]
 62%|██████▏   | 139478/225000 [278:03:37<118:42:58,  5.00s/it]
 62%|██████▏   | 139479/225000 [278:03:49<169:22:46,  7.13s/it]
 62%|██████▏   | 139480/225000 [278:03:55<158:58:50,  6.69s/it]
 62%|██████▏   | 139481/225000 [278:04:00<151:12:47,  6.37s/it]
 62%|██████▏   | 139482/225000 [278:04:06<142:38:57,  6.01s/it]
 62%|██████▏   | 139483/225000 [278:04:14<160:47:24,  6.77s/it]
 62%|██████▏   | 139484/225000 [278:04:18<141:02:31,  5.94s/it]
 62%|██████▏   | 139485/225000 [278:04:22<125:11:27,  5.27s/it]
 62%|██████▏   | 139486/225000 [278:04:30<145:34:00,  6.13s/it]
 62%|██████▏   | 139487/225000 [278:04:37<154:20:54,  6.50s/it]
 62%|██████▏   | 139488/225000 [278:04:41<133:50:56,  5.63s/it]
 62%|██████▏   | 139489/225000 [278:04:45<125:23:28,  5.28s/it]
 62%|██████▏   | 139490/225000 [278:04:55<159:25:03,  6.71s/it]
 62%|██████▏   | 139491/225000 [278:04:59<136:08:06,  5.73s/it]
 62%|██████▏   | 139492/225000 [278:05:06<148:12:37,  6.24s/it]
 62%|██████▏   | 139493/225000 [278:05:12<143:41:42,  6.05s/it]
 62%|██████▏   | 139494/225000 [278:05:17<134:28:19,  5.66s/it]
 62%|██████▏   | 139495/225000 [278:05:23<138:55:13,  5.85s/it]
 62%|██████▏   | 139496/225000 [278:05:26<121:08:25,  5.10s/it]
 62%|██████▏   | 139497/225000 [278:05:36<154:19:30,  6.50s/it]
 62%|██████▏   | 139498/225000 [278:05:39<130:53:20,  5.51s/it]
 62%|██████▏   | 139499/225000 [278:05:46<143:04:07,  6.02s/it]
 62%|██████▏   | 139500/225000 [278:05:51<134:03:01,  5.64s/it]
                                                               
{'loss': '0.7948', 'grad_norm': '1.01', 'learning_rate': '1.712e-05', 'epoch': '0.62', 'num_input_tokens_seen': 461208808, 'train_runtime': '1.001e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139500/225000 [278:05:51<134:03:01,  5.64s/it]
 62%|██████▏   | 139501/225000 [278:05:55<123:24:03,  5.20s/it]
 62%|██████▏   | 139502/225000 [278:06:00<117:21:45,  4.94s/it]
 62%|██████▏   | 139503/225000 [278:06:04<113:56:22,  4.80s/it]
 62%|██████▏   | 139504/225000 [278:06:09<115:31:42,  4.86s/it]
 62%|██████▏   | 139505/225000 [278:06:21<165:00:05,  6.95s/it]
 62%|██████▏   | 139506/225000 [278:06:25<142:17:10,  5.99s/it]
 62%|██████▏   | 139507/225000 [278:06:30<134:59:16,  5.68s/it]
 62%|██████▏   | 139508/225000 [278:06:39<158:19:24,  6.67s/it]
 62%|██████▏   | 139509/225000 [278:06:43<139:25:56,  5.87s/it]
 62%|██████▏   | 139510/225000 [278:06:49<143:07:22,  6.03s/it]
 62%|██████▏   | 139511/225000 [278:06:56<150:23:55,  6.33s/it]
 62%|██████▏   | 139512/225000 [278:07:00<131:36:01,  5.54s/it]
 62%|██████▏   | 139513/225000 [278:07:04<118:47:02,  5.00s/it]
 62%|██████▏   | 139514/225000 [278:07:13<146:55:28,  6.19s/it]
 62%|██████▏   | 139515/225000 [278:07:18<140:59:42,  5.94s/it]
 62%|██████▏   | 139516/225000 [278:07:25<146:10:02,  6.16s/it]
 62%|██████▏   | 139517/225000 [278:07:30<138:04:12,  5.81s/it]
 62%|██████▏   | 139518/225000 [278:07:35<135:30:12,  5.71s/it]
 62%|██████▏   | 139519/225000 [278:07:40<133:23:40,  5.62s/it]
 62%|██████▏   | 139520/225000 [278:07:48<148:09:44,  6.24s/it]
 62%|██████▏   | 139521/225000 [278:07:53<135:19:10,  5.70s/it]
 62%|██████▏   | 139522/225000 [278:07:57<125:29:56,  5.29s/it]
 62%|██████▏   | 139523/225000 [278:08:01<118:04:11,  4.97s/it]
 62%|██████▏   | 139524/225000 [278:08:06<114:10:05,  4.81s/it]
 62%|██████▏   | 139525/225000 [278:08:19<175:23:59,  7.39s/it]
 62%|██████▏   | 139526/225000 [278:08:24<161:26:48,  6.80s/it]
 62%|██████▏   | 139527/225000 [278:08:28<138:55:26,  5.85s/it]
 62%|██████▏   | 139528/225000 [278:08:37<160:48:38,  6.77s/it]
 62%|██████▏   | 139529/225000 [278:08:43<157:40:15,  6.64s/it]
 62%|██████▏   | 139530/225000 [278:08:47<139:37:26,  5.88s/it]
 62%|██████▏   | 139531/225000 [278:08:51<124:11:52,  5.23s/it]
 62%|██████▏   | 139532/225000 [278:08:55<117:32:41,  4.95s/it]
 62%|██████▏   | 139533/225000 [278:09:00<111:31:57,  4.70s/it]
 62%|██████▏   | 139534/225000 [278:09:08<139:49:34,  5.89s/it]
 62%|██████▏   | 139535/225000 [278:09:17<159:34:42,  6.72s/it]
 62%|██████▏   | 139536/225000 [278:09:21<143:14:55,  6.03s/it]
 62%|██████▏   | 139537/225000 [278:09:26<131:56:00,  5.56s/it]
 62%|██████▏   | 139538/225000 [278:09:30<124:29:22,  5.24s/it]
 62%|██████▏   | 139539/225000 [278:09:40<155:16:03,  6.54s/it]
 62%|██████▏   | 139540/225000 [278:09:46<153:58:16,  6.49s/it]
 62%|██████▏   | 139541/225000 [278:09:50<136:07:05,  5.73s/it]
 62%|██████▏   | 139542/225000 [278:09:54<122:43:09,  5.17s/it]
 62%|██████▏   | 139543/225000 [278:10:03<152:48:54,  6.44s/it]
 62%|██████▏   | 139544/225000 [278:10:09<143:45:32,  6.06s/it]
 62%|██████▏   | 139545/225000 [278:10:12<128:12:29,  5.40s/it]
 62%|██████▏   | 139546/225000 [278:10:17<120:19:27,  5.07s/it]
 62%|██████▏   | 139547/225000 [278:10:21<114:44:55,  4.83s/it]
 62%|██████▏   | 139548/225000 [278:10:31<148:41:08,  6.26s/it]
 62%|██████▏   | 139549/225000 [278:10:38<158:23:35,  6.67s/it]
 62%|██████▏   | 139550/225000 [278:10:42<134:18:46,  5.66s/it]
                                                               
{'loss': '0.8072', 'grad_norm': '1.261', 'learning_rate': '1.711e-05', 'epoch': '0.6202', 'num_input_tokens_seen': 461338456, 'train_runtime': '1.001e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139550/225000 [278:10:42<134:18:46,  5.66s/it]
 62%|██████▏   | 139551/225000 [278:10:50<152:17:31,  6.42s/it]
 62%|██████▏   | 139552/225000 [278:10:53<132:47:59,  5.59s/it]
 62%|██████▏   | 139553/225000 [278:11:01<148:32:08,  6.26s/it]
 62%|██████▏   | 139554/225000 [278:11:05<127:34:53,  5.38s/it]
 62%|██████▏   | 139555/225000 [278:11:13<146:48:46,  6.19s/it]
 62%|██████▏   | 139556/225000 [278:11:18<143:17:24,  6.04s/it]
 62%|██████▏   | 139557/225000 [278:11:26<154:24:42,  6.51s/it]
 62%|██████▏   | 139558/225000 [278:11:30<135:56:57,  5.73s/it]
 62%|██████▏   | 139559/225000 [278:11:35<131:50:53,  5.56s/it]
 62%|██████▏   | 139560/225000 [278:11:40<131:10:40,  5.53s/it]
 62%|██████▏   | 139561/225000 [278:11:47<138:57:37,  5.86s/it]
 62%|██████▏   | 139562/225000 [278:11:54<146:42:21,  6.18s/it]
 62%|██████▏   | 139563/225000 [278:11:58<130:59:05,  5.52s/it]
 62%|██████▏   | 139564/225000 [278:12:04<134:49:27,  5.68s/it]
 62%|██████▏   | 139565/225000 [278:12:11<141:29:58,  5.96s/it]
 62%|██████▏   | 139566/225000 [278:12:14<122:47:51,  5.17s/it]
 62%|██████▏   | 139567/225000 [278:12:24<156:38:12,  6.60s/it]
 62%|██████▏   | 139568/225000 [278:12:29<145:04:51,  6.11s/it]
 62%|██████▏   | 139569/225000 [278:12:34<136:30:57,  5.75s/it]
 62%|██████▏   | 139570/225000 [278:12:38<127:26:38,  5.37s/it]
 62%|██████▏   | 139571/225000 [278:12:43<124:29:23,  5.25s/it]
 62%|██████▏   | 139572/225000 [278:12:48<119:50:13,  5.05s/it]
 62%|██████▏   | 139573/225000 [278:12:53<119:22:05,  5.03s/it]
 62%|██████▏   | 139574/225000 [278:12:58<118:12:33,  4.98s/it]
 62%|██████▏   | 139575/225000 [278:13:08<159:10:19,  6.71s/it]
 62%|██████▏   | 139576/225000 [278:13:15<157:39:23,  6.64s/it]
 62%|██████▏   | 139577/225000 [278:13:19<136:29:23,  5.75s/it]
 62%|██████▏   | 139578/225000 [278:13:29<166:27:24,  7.02s/it]
 62%|██████▏   | 139579/225000 [278:13:32<141:52:09,  5.98s/it]
 62%|██████▏   | 139580/225000 [278:13:40<158:14:56,  6.67s/it]
 62%|██████▏   | 139581/225000 [278:13:44<136:24:05,  5.75s/it]
 62%|██████▏   | 139582/225000 [278:13:48<125:50:12,  5.30s/it]
 62%|██████▏   | 139583/225000 [278:13:55<136:17:59,  5.74s/it]
 62%|██████▏   | 139584/225000 [278:14:02<145:48:10,  6.15s/it]
 62%|██████▏   | 139585/225000 [278:14:09<151:26:00,  6.38s/it]
 62%|██████▏   | 139586/225000 [278:14:13<130:51:34,  5.52s/it]
 62%|██████▏   | 139587/225000 [278:14:21<148:25:29,  6.26s/it]
 62%|██████▏   | 139588/225000 [278:14:25<136:25:07,  5.75s/it]
 62%|██████▏   | 139589/225000 [278:14:29<120:54:55,  5.10s/it]
 62%|██████▏   | 139590/225000 [278:14:33<114:43:32,  4.84s/it]
 62%|██████▏   | 139591/225000 [278:14:39<123:00:56,  5.19s/it]
 62%|██████▏   | 139592/225000 [278:14:51<175:30:17,  7.40s/it]
 62%|██████▏   | 139593/225000 [278:14:55<145:58:18,  6.15s/it]
 62%|██████▏   | 139594/225000 [278:14:59<134:55:30,  5.69s/it]
 62%|██████▏   | 139595/225000 [278:15:07<149:50:24,  6.32s/it]
 62%|██████▏   | 139596/225000 [278:15:10<127:27:23,  5.37s/it]
 62%|██████▏   | 139597/225000 [278:15:16<129:12:35,  5.45s/it]
 62%|██████▏   | 139598/225000 [278:15:24<146:47:53,  6.19s/it]
 62%|██████▏   | 139599/225000 [278:15:27<127:03:29,  5.36s/it]
 62%|██████▏   | 139600/225000 [278:15:36<153:15:13,  6.46s/it]
                                                               
{'loss': '0.7888', 'grad_norm': '1.129', 'learning_rate': '1.709e-05', 'epoch': '0.6204', 'num_input_tokens_seen': 461468792, 'train_runtime': '1.002e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139600/225000 [278:15:36<153:15:13,  6.46s/it]
 62%|██████▏   | 139601/225000 [278:15:40<132:47:16,  5.60s/it]
 62%|██████▏   | 139602/225000 [278:15:48<148:47:11,  6.27s/it]
 62%|██████▏   | 139603/225000 [278:15:53<141:47:02,  5.98s/it]
 62%|██████▏   | 139604/225000 [278:15:59<143:27:11,  6.05s/it]
 62%|██████▏   | 139605/225000 [278:16:03<129:26:41,  5.46s/it]
 62%|██████▏   | 139606/225000 [278:16:08<120:56:28,  5.10s/it]
 62%|██████▏   | 139607/225000 [278:16:12<114:10:11,  4.81s/it]
 62%|██████▏   | 139608/225000 [278:16:23<162:30:55,  6.85s/it]
 62%|██████▏   | 139609/225000 [278:16:28<145:08:41,  6.12s/it]
 62%|██████▏   | 139610/225000 [278:16:34<146:51:54,  6.19s/it]
 62%|██████▏   | 139611/225000 [278:16:39<138:15:47,  5.83s/it]
 62%|██████▏   | 139612/225000 [278:16:44<130:50:08,  5.52s/it]
 62%|██████▏   | 139613/225000 [278:16:49<126:54:20,  5.35s/it]
 62%|██████▏   | 139614/225000 [278:16:53<117:31:54,  4.96s/it]
 62%|██████▏   | 139615/225000 [278:16:58<116:25:55,  4.91s/it]
 62%|██████▏   | 139616/225000 [278:17:02<114:36:21,  4.83s/it]
 62%|██████▏   | 139617/225000 [278:17:07<115:32:03,  4.87s/it]
 62%|██████▏   | 139618/225000 [278:17:12<114:41:01,  4.84s/it]
 62%|██████▏   | 139619/225000 [278:17:23<157:49:45,  6.65s/it]
 62%|██████▏   | 139620/225000 [278:17:30<162:04:19,  6.83s/it]
 62%|██████▏   | 139621/225000 [278:17:34<140:08:14,  5.91s/it]
 62%|██████▏   | 139622/225000 [278:17:42<156:04:34,  6.58s/it]
 62%|██████▏   | 139623/225000 [278:17:48<153:29:26,  6.47s/it]
 62%|██████▏   | 139624/225000 [278:17:52<135:17:03,  5.70s/it]
 62%|██████▏   | 139625/225000 [278:17:58<134:24:47,  5.67s/it]
 62%|██████▏   | 139626/225000 [278:18:03<132:43:29,  5.60s/it]
 62%|██████▏   | 139627/225000 [278:18:09<131:04:27,  5.53s/it]
 62%|██████▏   | 139628/225000 [278:18:14<133:32:20,  5.63s/it]
 62%|██████▏   | 139629/225000 [278:18:20<132:28:47,  5.59s/it]
 62%|██████▏   | 139630/225000 [278:18:25<131:40:20,  5.55s/it]
 62%|██████▏   | 139631/225000 [278:18:31<132:13:57,  5.58s/it]
 62%|██████▏   | 139632/225000 [278:18:36<130:34:31,  5.51s/it]
 62%|██████▏   | 139633/225000 [278:18:42<131:23:13,  5.54s/it]
 62%|██████▏   | 139634/225000 [278:18:48<133:43:43,  5.64s/it]
 62%|██████▏   | 139635/225000 [278:18:53<132:44:05,  5.60s/it]
 62%|██████▏   | 139636/225000 [278:19:05<176:07:14,  7.43s/it]
 62%|██████▏   | 139637/225000 [278:19:11<168:57:18,  7.13s/it]
 62%|██████▏   | 139638/225000 [278:19:20<179:32:14,  7.57s/it]
 62%|██████▏   | 139639/225000 [278:19:24<152:06:14,  6.41s/it]
 62%|██████▏   | 139640/225000 [278:19:29<141:33:34,  5.97s/it]
 62%|██████▏   | 139641/225000 [278:19:34<136:05:30,  5.74s/it]
 62%|██████▏   | 139642/225000 [278:19:39<128:16:56,  5.41s/it]
 62%|██████▏   | 139643/225000 [278:19:44<125:35:44,  5.30s/it]
 62%|██████▏   | 139644/225000 [278:19:49<122:54:39,  5.18s/it]
 62%|██████▏   | 139645/225000 [278:19:53<120:32:31,  5.08s/it]
 62%|██████▏   | 139646/225000 [278:19:58<117:50:56,  4.97s/it]
 62%|██████▏   | 139647/225000 [278:20:06<135:33:31,  5.72s/it]
 62%|██████▏   | 139648/225000 [278:20:18<180:21:57,  7.61s/it]
 62%|██████▏   | 139649/225000 [278:20:22<156:13:07,  6.59s/it]
 62%|██████▏   | 139650/225000 [278:20:26<139:20:38,  5.88s/it]
                                                               
{'loss': '0.7965', 'grad_norm': '1.157', 'learning_rate': '1.707e-05', 'epoch': '0.6207', 'num_input_tokens_seen': 461598272, 'train_runtime': '1.002e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139650/225000 [278:20:26<139:20:38,  5.88s/it]
 62%|██████▏   | 139651/225000 [278:20:31<131:17:19,  5.54s/it]
 62%|██████▏   | 139652/225000 [278:20:35<122:57:24,  5.19s/it]
 62%|██████▏   | 139653/225000 [278:20:40<120:27:42,  5.08s/it]
 62%|██████▏   | 139654/225000 [278:20:45<119:39:30,  5.05s/it]
 62%|██████▏   | 139655/225000 [278:20:56<159:27:24,  6.73s/it]
 62%|██████▏   | 139656/225000 [278:21:01<152:41:22,  6.44s/it]
 62%|██████▏   | 139657/225000 [278:21:05<132:41:45,  5.60s/it]
 62%|██████▏   | 139658/225000 [278:21:15<166:18:26,  7.02s/it]
 62%|██████▏   | 139659/225000 [278:21:19<146:10:40,  6.17s/it]
 62%|██████▏   | 139660/225000 [278:21:26<151:27:36,  6.39s/it]
 62%|██████▏   | 139661/225000 [278:21:30<132:21:20,  5.58s/it]
 62%|██████▏   | 139662/225000 [278:21:39<156:48:31,  6.62s/it]
 62%|██████▏   | 139663/225000 [278:21:43<139:26:03,  5.88s/it]
 62%|██████▏   | 139664/225000 [278:21:49<136:58:47,  5.78s/it]
 62%|██████▏   | 139665/225000 [278:21:53<124:06:21,  5.24s/it]
 62%|██████▏   | 139666/225000 [278:21:57<114:16:35,  4.82s/it]
 62%|██████▏   | 139667/225000 [278:22:04<135:38:08,  5.72s/it]
 62%|██████▏   | 139668/225000 [278:22:14<159:26:18,  6.73s/it]
 62%|██████▏   | 139669/225000 [278:22:17<138:30:49,  5.84s/it]
 62%|██████▏   | 139670/225000 [278:22:21<122:49:49,  5.18s/it]
 62%|██████▏   | 139671/225000 [278:22:26<120:18:35,  5.08s/it]
 62%|██████▏   | 139672/225000 [278:22:30<117:49:52,  4.97s/it]
 62%|██████▏   | 139673/225000 [278:22:40<148:00:43,  6.24s/it]
 62%|██████▏   | 139674/225000 [278:22:48<160:41:19,  6.78s/it]
 62%|██████▏   | 139675/225000 [278:22:54<155:38:25,  6.57s/it]
 62%|██████▏   | 139676/225000 [278:23:00<155:41:23,  6.57s/it]
 62%|██████▏   | 139677/225000 [278:23:04<138:17:05,  5.83s/it]
 62%|██████▏   | 139678/225000 [278:23:09<127:31:02,  5.38s/it]
 62%|██████▏   | 139679/225000 [278:23:13<120:36:35,  5.09s/it]
 62%|██████▏   | 139680/225000 [278:23:18<115:25:04,  4.87s/it]
 62%|██████▏   | 139681/225000 [278:23:22<110:43:46,  4.67s/it]
 62%|██████▏   | 139682/225000 [278:23:26<108:26:10,  4.58s/it]
 62%|██████▏   | 139683/225000 [278:23:34<134:56:00,  5.69s/it]
 62%|██████▏   | 139684/225000 [278:23:44<158:59:18,  6.71s/it]
 62%|██████▏   | 139685/225000 [278:23:48<141:03:23,  5.95s/it]
 62%|██████▏   | 139686/225000 [278:23:52<128:56:15,  5.44s/it]
 62%|██████▏   | 139687/225000 [278:23:57<127:25:50,  5.38s/it]
 62%|██████▏   | 139688/225000 [278:24:07<160:49:01,  6.79s/it]
 62%|██████▏   | 139689/225000 [278:24:11<137:11:31,  5.79s/it]
 62%|██████▏   | 139690/225000 [278:24:18<151:10:56,  6.38s/it]
 62%|██████▏   | 139691/225000 [278:24:22<134:06:40,  5.66s/it]
 62%|██████▏   | 139692/225000 [278:24:27<123:02:32,  5.19s/it]
 62%|██████▏   | 139693/225000 [278:24:35<143:41:38,  6.06s/it]
 62%|██████▏   | 139694/225000 [278:24:41<149:06:43,  6.29s/it]
 62%|██████▏   | 139695/225000 [278:24:45<130:01:47,  5.49s/it]
 62%|██████▏   | 139696/225000 [278:24:50<128:42:06,  5.43s/it]
 62%|██████▏   | 139697/225000 [278:25:02<170:26:45,  7.19s/it]
 62%|██████▏   | 139698/225000 [278:25:05<141:12:25,  5.96s/it]
 62%|██████▏   | 139699/225000 [278:25:09<128:04:02,  5.40s/it]
 62%|██████▏   | 139700/225000 [278:25:13<119:32:39,  5.05s/it]
                                                               
{'loss': '0.7981', 'grad_norm': '1.168', 'learning_rate': '1.705e-05', 'epoch': '0.6209', 'num_input_tokens_seen': 461727496, 'train_runtime': '1.002e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139700/225000 [278:25:13<119:32:39,  5.05s/it]
 62%|██████▏   | 139701/225000 [278:25:24<162:11:22,  6.85s/it]
 62%|██████▏   | 139702/225000 [278:25:28<138:05:08,  5.83s/it]
 62%|██████▏   | 139703/225000 [278:25:38<169:10:47,  7.14s/it]
 62%|██████▏   | 139704/225000 [278:25:41<142:35:14,  6.02s/it]
 62%|██████▏   | 139705/225000 [278:25:47<139:22:35,  5.88s/it]
 62%|██████▏   | 139706/225000 [278:25:54<149:48:52,  6.32s/it]
 62%|██████▏   | 139707/225000 [278:25:58<132:20:44,  5.59s/it]
 62%|██████▏   | 139708/225000 [278:26:02<120:52:33,  5.10s/it]
 62%|██████▏   | 139709/225000 [278:26:13<163:41:52,  6.91s/it]
 62%|██████▏   | 139710/225000 [278:26:16<138:39:51,  5.85s/it]
 62%|██████▏   | 139711/225000 [278:26:20<121:18:13,  5.12s/it]
 62%|██████▏   | 139712/225000 [278:26:25<119:06:02,  5.03s/it]
 62%|██████▏   | 139713/225000 [278:26:29<115:32:01,  4.88s/it]
 62%|██████▏   | 139714/225000 [278:26:34<117:20:30,  4.95s/it]
 62%|██████▏   | 139715/225000 [278:26:39<117:39:04,  4.97s/it]
 62%|██████▏   | 139716/225000 [278:26:46<129:09:47,  5.45s/it]
 62%|██████▏   | 139717/225000 [278:26:58<177:48:41,  7.51s/it]
 62%|██████▏   | 139718/225000 [278:27:02<148:35:59,  6.27s/it]
 62%|██████▏   | 139719/225000 [278:27:06<133:45:55,  5.65s/it]
 62%|██████▏   | 139720/225000 [278:27:13<145:45:10,  6.15s/it]
 62%|██████▏   | 139721/225000 [278:27:17<132:08:28,  5.58s/it]
 62%|██████▏   | 139722/225000 [278:27:25<149:21:42,  6.31s/it]
 62%|██████▏   | 139723/225000 [278:27:29<130:21:06,  5.50s/it]
 62%|██████▏   | 139724/225000 [278:27:33<117:26:58,  4.96s/it]
 62%|██████▏   | 139725/225000 [278:27:43<154:41:12,  6.53s/it]
 62%|██████▏   | 139726/225000 [278:27:47<137:01:30,  5.78s/it]
 62%|██████▏   | 139727/225000 [278:27:52<129:18:48,  5.46s/it]
 62%|██████▏   | 139728/225000 [278:27:57<126:18:46,  5.33s/it]
 62%|██████▏   | 139729/225000 [278:28:02<124:26:12,  5.25s/it]
 62%|██████▏   | 139730/225000 [278:28:07<122:29:14,  5.17s/it]
 62%|██████▏   | 139731/225000 [278:28:12<123:21:26,  5.21s/it]
 62%|██████▏   | 139732/225000 [278:28:17<125:15:14,  5.29s/it]
 62%|██████▏   | 139733/225000 [278:28:23<125:48:24,  5.31s/it]
 62%|██████▏   | 139734/225000 [278:28:30<139:37:17,  5.89s/it]
 62%|██████▏   | 139735/225000 [278:28:40<165:07:58,  6.97s/it]
 62%|██████▏   | 139736/225000 [278:28:45<154:21:38,  6.52s/it]
 62%|██████▏   | 139737/225000 [278:28:50<145:41:00,  6.15s/it]
 62%|██████▏   | 139738/225000 [278:28:56<139:10:42,  5.88s/it]
 62%|██████▏   | 139739/225000 [278:29:01<133:31:17,  5.64s/it]
 62%|██████▏   | 139740/225000 [278:29:06<130:00:13,  5.49s/it]
 62%|██████▏   | 139741/225000 [278:29:11<127:29:21,  5.38s/it]
 62%|██████▏   | 139742/225000 [278:29:16<127:23:46,  5.38s/it]
 62%|██████▏   | 139743/225000 [278:29:22<128:55:33,  5.44s/it]
 62%|██████▏   | 139744/225000 [278:29:27<129:07:35,  5.45s/it]
 62%|██████▏   | 139745/225000 [278:29:32<125:36:06,  5.30s/it]
 62%|██████▏   | 139746/225000 [278:29:38<127:18:11,  5.38s/it]
 62%|██████▏   | 139747/225000 [278:29:46<143:32:38,  6.06s/it]
 62%|██████▏   | 139748/225000 [278:29:51<139:00:31,  5.87s/it]
 62%|██████▏   | 139749/225000 [278:29:56<134:27:57,  5.68s/it]
 62%|██████▏   | 139750/225000 [278:30:02<134:27:42,  5.68s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.057', 'learning_rate': '1.704e-05', 'epoch': '0.6211', 'num_input_tokens_seen': 461857536, 'train_runtime': '1.003e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139750/225000 [278:30:02<134:27:42,  5.68s/it]
 62%|██████▏   | 139751/225000 [278:30:07<131:21:12,  5.55s/it]
 62%|██████▏   | 139752/225000 [278:30:13<132:18:51,  5.59s/it]
 62%|██████▏   | 139753/225000 [278:30:18<131:49:45,  5.57s/it]
 62%|██████▏   | 139754/225000 [278:30:24<130:03:08,  5.49s/it]
 62%|██████▏   | 139755/225000 [278:30:29<129:43:08,  5.48s/it]
 62%|██████▏   | 139756/225000 [278:30:40<167:05:48,  7.06s/it]
 62%|██████▏   | 139757/225000 [278:30:51<194:34:44,  8.22s/it]
 62%|██████▏   | 139758/225000 [278:30:54<159:54:43,  6.75s/it]
 62%|██████▏   | 139759/225000 [278:30:59<150:07:39,  6.34s/it]
 62%|██████▏   | 139760/225000 [278:31:05<142:39:56,  6.03s/it]
 62%|██████▏   | 139761/225000 [278:31:10<140:00:14,  5.91s/it]
 62%|██████▏   | 139762/225000 [278:31:16<139:02:54,  5.87s/it]
 62%|██████▏   | 139763/225000 [278:31:24<156:21:28,  6.60s/it]
 62%|██████▏   | 139764/225000 [278:31:29<142:57:52,  6.04s/it]
 62%|██████▏   | 139765/225000 [278:31:34<133:28:49,  5.64s/it]
 62%|██████▏   | 139766/225000 [278:31:38<124:43:17,  5.27s/it]
 62%|██████▏   | 139767/225000 [278:31:44<129:44:25,  5.48s/it]
 62%|██████▏   | 139768/225000 [278:31:55<167:10:27,  7.06s/it]
 62%|██████▏   | 139769/225000 [278:31:59<147:36:02,  6.23s/it]
 62%|██████▏   | 139770/225000 [278:32:04<133:29:26,  5.64s/it]
 62%|██████▏   | 139771/225000 [278:32:08<123:47:33,  5.23s/it]
 62%|██████▏   | 139772/225000 [278:32:12<117:44:05,  4.97s/it]
 62%|██████▏   | 139773/225000 [278:32:18<125:11:20,  5.29s/it]
 62%|██████▏   | 139774/225000 [278:32:30<171:34:32,  7.25s/it]
 62%|██████▏   | 139775/225000 [278:32:35<152:38:07,  6.45s/it]
 62%|██████▏   | 139776/225000 [278:32:39<139:57:40,  5.91s/it]
 62%|██████▏   | 139777/225000 [278:32:44<131:14:54,  5.54s/it]
 62%|██████▏   | 139778/225000 [278:32:49<123:49:42,  5.23s/it]
 62%|██████▏   | 139779/225000 [278:32:53<121:37:56,  5.14s/it]
 62%|██████▏   | 139780/225000 [278:32:58<117:13:43,  4.95s/it]
 62%|██████▏   | 139781/225000 [278:33:11<171:59:23,  7.27s/it]
 62%|██████▏   | 139782/225000 [278:33:15<150:18:04,  6.35s/it]
 62%|██████▏   | 139783/225000 [278:33:19<136:42:12,  5.78s/it]
 62%|██████▏   | 139784/225000 [278:33:24<126:54:29,  5.36s/it]
 62%|██████▏   | 139785/225000 [278:33:33<157:06:11,  6.64s/it]
 62%|██████▏   | 139786/225000 [278:33:40<159:34:08,  6.74s/it]
 62%|██████▏   | 139787/225000 [278:33:44<140:20:23,  5.93s/it]
 62%|██████▏   | 139788/225000 [278:33:49<129:34:21,  5.47s/it]
 62%|██████▏   | 139789/225000 [278:33:58<157:05:24,  6.64s/it]
 62%|██████▏   | 139790/225000 [278:34:02<140:20:39,  5.93s/it]
 62%|██████▏   | 139791/225000 [278:34:11<159:11:28,  6.73s/it]
 62%|██████▏   | 139792/225000 [278:34:14<135:01:48,  5.70s/it]
 62%|██████▏   | 139793/225000 [278:34:22<149:59:58,  6.34s/it]
 62%|██████▏   | 139794/225000 [278:34:25<127:43:25,  5.40s/it]
 62%|██████▏   | 139795/225000 [278:34:33<143:14:26,  6.05s/it]
 62%|██████▏   | 139796/225000 [278:34:37<129:07:52,  5.46s/it]
 62%|██████▏   | 139797/225000 [278:34:41<116:17:47,  4.91s/it]
 62%|██████▏   | 139798/225000 [278:34:50<148:13:51,  6.26s/it]
 62%|██████▏   | 139799/225000 [278:34:53<127:21:20,  5.38s/it]
 62%|██████▏   | 139800/225000 [278:35:02<148:12:05,  6.26s/it]
                                                               
{'loss': '0.7944', 'grad_norm': '1.167', 'learning_rate': '1.702e-05', 'epoch': '0.6213', 'num_input_tokens_seen': 461987520, 'train_runtime': '1.003e+06', 'train_tokens_per_second': '460.7'}

 62%|██████▏   | 139800/225000 [278:35:02<148:12:05,  6.26s/it]
 62%|██████▏   | 139801/225000 [278:35:05<127:30:23,  5.39s/it]
 62%|██████▏   | 139802/225000 [278:35:11<130:59:02,  5.53s/it]
 62%|██████▏   | 139803/225000 [278:35:17<134:47:06,  5.70s/it]
 62%|██████▏   | 139804/225000 [278:35:20<117:58:51,  4.99s/it]
 62%|██████▏   | 139805/225000 [278:35:25<114:48:31,  4.85s/it]
 62%|██████▏   | 139806/225000 [278:35:29<113:20:11,  4.79s/it]
 62%|██████▏   | 139807/225000 [278:35:34<114:17:33,  4.83s/it]
 62%|██████▏   | 139808/225000 [278:35:43<140:29:33,  5.94s/it]
 62%|██████▏   | 139809/225000 [278:35:54<175:32:26,  7.42s/it]
 62%|██████▏   | 139810/225000 [278:35:57<143:56:34,  6.08s/it]
 62%|██████▏   | 139811/225000 [278:36:01<129:11:25,  5.46s/it]
 62%|██████▏   | 139812/225000 [278:36:08<142:48:14,  6.03s/it]
 62%|██████▏   | 139813/225000 [278:36:12<129:13:48,  5.46s/it]
 62%|██████▏   | 139814/225000 [278:36:16<119:11:37,  5.04s/it]
 62%|██████▏   | 139815/225000 [278:36:22<126:58:36,  5.37s/it]
 62%|██████▏   | 139816/225000 [278:36:30<144:48:47,  6.12s/it]
 62%|██████▏   | 139817/225000 [278:36:35<134:18:44,  5.68s/it]
 62%|██████▏   | 139818/225000 [278:36:40<127:57:09,  5.41s/it]
 62%|██████▏   | 139819/225000 [278:36:45<125:42:56,  5.31s/it]
 62%|██████▏   | 139820/225000 [278:36:49<120:36:58,  5.10s/it]
 62%|██████▏   | 139821/225000 [278:36:54<119:05:13,  5.03s/it]
 62%|██████▏   | 139822/225000 [278:37:01<133:40:01,  5.65s/it]
 62%|██████▏   | 139823/225000 [278:37:12<168:16:52,  7.11s/it]
 62%|██████▏   | 139824/225000 [278:37:15<141:15:57,  5.97s/it]
 62%|██████▏   | 139825/225000 [278:37:25<166:51:01,  7.05s/it]
 62%|██████▏   | 139826/225000 [278:37:28<139:58:53,  5.92s/it]
 62%|██████▏   | 139827/225000 [278:37:32<126:49:45,  5.36s/it]
 62%|██████▏   | 139828/225000 [278:37:36<117:30:17,  4.97s/it]
 62%|██████▏   | 139829/225000 [278:37:45<142:20:02,  6.02s/it]
 62%|██████▏   | 139830/225000 [278:37:51<147:59:58,  6.26s/it]
 62%|██████▏   | 139831/225000 [278:37:55<130:21:50,  5.51s/it]
 62%|██████▏   | 139832/225000 [278:38:03<147:22:30,  6.23s/it]
 62%|██████▏   | 139833/225000 [278:38:07<130:15:22,  5.51s/it]
 62%|██████▏   | 139834/225000 [278:38:11<119:46:12,  5.06s/it]
 62%|██████▏   | 139835/225000 [278:38:16<117:00:19,  4.95s/it]
 62%|██████▏   | 139836/225000 [278:38:28<167:02:50,  7.06s/it]
 62%|██████▏   | 139837/225000 [278:38:31<140:27:14,  5.94s/it]
 62%|██████▏   | 139838/225000 [278:38:39<156:11:27,  6.60s/it]
 62%|██████▏   | 139839/225000 [278:38:42<133:17:24,  5.63s/it]
 62%|██████▏   | 139840/225000 [278:38:46<118:29:23,  5.01s/it]
 62%|██████▏   | 139841/225000 [278:38:54<141:02:35,  5.96s/it]
 62%|██████▏   | 139842/225000 [278:38:58<128:02:12,  5.41s/it]
 62%|██████▏   | 139843/225000 [278:39:06<141:26:41,  5.98s/it]
 62%|██████▏   | 139844/225000 [278:39:13<148:18:45,  6.27s/it]
 62%|██████▏   | 139845/225000 [278:39:17<134:41:58,  5.69s/it]
 62%|██████▏   | 139846/225000 [278:39:24<141:00:12,  5.96s/it]
 62%|██████▏   | 139847/225000 [278:39:28<131:21:13,  5.55s/it]
 62%|██████▏   | 139848/225000 [278:39:33<127:35:35,  5.39s/it]
 62%|██████▏   | 139849/225000 [278:39:38<127:01:01,  5.37s/it]
 62%|██████▏   | 139850/225000 [278:39:44<125:41:27,  5.31s/it]
                                                               
{'loss': '0.7898', 'grad_norm': '0.9982', 'learning_rate': '1.7e-05', 'epoch': '0.6216', 'num_input_tokens_seen': 462116320, 'train_runtime': '1.003e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 139850/225000 [278:39:44<125:41:27,  5.31s/it]
 62%|██████▏   | 139851/225000 [278:39:49<125:23:54,  5.30s/it]
 62%|██████▏   | 139852/225000 [278:39:53<118:52:28,  5.03s/it]
 62%|██████▏   | 139853/225000 [278:39:58<116:25:30,  4.92s/it]
 62%|██████▏   | 139854/225000 [278:40:03<117:28:18,  4.97s/it]
 62%|██████▏   | 139855/225000 [278:40:08<116:43:53,  4.94s/it]
 62%|██████▏   | 139856/225000 [278:40:13<119:07:01,  5.04s/it]
 62%|██████▏   | 139857/225000 [278:40:25<164:57:34,  6.97s/it]
 62%|██████▏   | 139858/225000 [278:40:33<172:15:07,  7.28s/it]
 62%|██████▏   | 139859/225000 [278:40:39<168:46:39,  7.14s/it]
 62%|██████▏   | 139860/225000 [278:40:43<144:10:57,  6.10s/it]
 62%|██████▏   | 139861/225000 [278:40:47<127:12:58,  5.38s/it]
 62%|██████▏   | 139862/225000 [278:40:52<124:18:21,  5.26s/it]
 62%|██████▏   | 139863/225000 [278:40:57<122:05:26,  5.16s/it]
 62%|██████▏   | 139864/225000 [278:41:02<122:20:47,  5.17s/it]
 62%|██████▏   | 139865/225000 [278:41:07<121:12:24,  5.13s/it]
 62%|██████▏   | 139866/225000 [278:41:13<124:18:15,  5.26s/it]
 62%|██████▏   | 139867/225000 [278:41:18<123:06:40,  5.21s/it]
 62%|██████▏   | 139868/225000 [278:41:23<126:25:57,  5.35s/it]
 62%|██████▏   | 139869/225000 [278:41:35<171:47:03,  7.26s/it]
 62%|██████▏   | 139870/225000 [278:41:39<145:34:37,  6.16s/it]
 62%|██████▏   | 139871/225000 [278:41:42<125:31:05,  5.31s/it]
 62%|██████▏   | 139872/225000 [278:41:51<153:16:20,  6.48s/it]
 62%|██████▏   | 139873/225000 [278:41:55<133:02:33,  5.63s/it]
 62%|██████▏   | 139874/225000 [278:42:04<156:34:34,  6.62s/it]
 62%|██████▏   | 139875/225000 [278:42:08<137:16:20,  5.81s/it]
 62%|██████▏   | 139876/225000 [278:42:12<125:09:45,  5.29s/it]
 62%|██████▏   | 139877/225000 [278:42:17<122:14:05,  5.17s/it]
 62%|██████▏   | 139878/225000 [278:42:22<121:55:02,  5.16s/it]
 62%|██████▏   | 139879/225000 [278:42:27<120:39:27,  5.10s/it]
 62%|██████▏   | 139880/225000 [278:42:32<125:13:04,  5.30s/it]
 62%|██████▏   | 139881/225000 [278:42:37<122:59:13,  5.20s/it]
 62%|██████▏   | 139882/225000 [278:42:43<124:02:16,  5.25s/it]
 62%|██████▏   | 139883/225000 [278:42:48<121:36:52,  5.14s/it]
 62%|██████▏   | 139884/225000 [278:43:00<170:39:58,  7.22s/it]
 62%|██████▏   | 139885/225000 [278:43:05<157:15:58,  6.65s/it]
 62%|██████▏   | 139886/225000 [278:43:10<141:48:34,  6.00s/it]
 62%|██████▏   | 139887/225000 [278:43:15<134:56:12,  5.71s/it]
 62%|██████▏   | 139888/225000 [278:43:20<130:37:07,  5.52s/it]
 62%|██████▏   | 139889/225000 [278:43:25<128:46:24,  5.45s/it]
 62%|██████▏   | 139890/225000 [278:43:30<126:04:04,  5.33s/it]
 62%|██████▏   | 139891/225000 [278:43:34<116:51:01,  4.94s/it]
 62%|██████▏   | 139892/225000 [278:43:39<116:43:16,  4.94s/it]
 62%|██████▏   | 139893/225000 [278:43:51<170:24:37,  7.21s/it]
 62%|██████▏   | 139894/225000 [278:43:58<164:07:33,  6.94s/it]
 62%|██████▏   | 139895/225000 [278:44:01<138:14:53,  5.85s/it]
 62%|██████▏   | 139896/225000 [278:44:09<155:56:32,  6.60s/it]
 62%|██████▏   | 139897/225000 [278:44:14<138:08:10,  5.84s/it]
 62%|██████▏   | 139898/225000 [278:44:18<125:42:35,  5.32s/it]
 62%|██████▏   | 139899/225000 [278:44:30<172:42:36,  7.31s/it]
 62%|██████▏   | 139900/225000 [278:44:35<161:29:12,  6.83s/it]
                                                               
{'loss': '0.7969', 'grad_norm': '1.173', 'learning_rate': '1.698e-05', 'epoch': '0.6218', 'num_input_tokens_seen': 462245952, 'train_runtime': '1.003e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 139900/225000 [278:44:35<161:29:12,  6.83s/it]
 62%|██████▏   | 139901/225000 [278:44:39<138:17:25,  5.85s/it]
 62%|██████▏   | 139902/225000 [278:44:43<128:02:23,  5.42s/it]
 62%|██████▏   | 139903/225000 [278:44:50<137:50:04,  5.83s/it]
 62%|██████▏   | 139904/225000 [278:44:58<153:09:23,  6.48s/it]
 62%|██████▏   | 139905/225000 [278:45:02<138:21:49,  5.85s/it]
 62%|██████▏   | 139906/225000 [278:45:07<127:38:16,  5.40s/it]
 62%|██████▏   | 139907/225000 [278:45:11<122:31:51,  5.18s/it]
 62%|██████▏   | 139908/225000 [278:45:16<117:56:11,  4.99s/it]
 62%|██████▏   | 139909/225000 [278:45:21<117:21:33,  4.97s/it]
 62%|██████▏   | 139910/225000 [278:45:33<166:53:31,  7.06s/it]
 62%|██████▏   | 139911/225000 [278:45:36<141:34:36,  5.99s/it]
 62%|██████▏   | 139912/225000 [278:45:44<153:25:05,  6.49s/it]
 62%|██████▏   | 139913/225000 [278:45:48<137:57:21,  5.84s/it]
 62%|██████▏   | 139914/225000 [278:45:53<127:15:21,  5.38s/it]
 62%|██████▏   | 139915/225000 [278:45:57<118:19:06,  5.01s/it]
 62%|██████▏   | 139916/225000 [278:46:04<135:12:07,  5.72s/it]
 62%|██████▏   | 139917/225000 [278:46:11<142:35:18,  6.03s/it]
 62%|██████▏   | 139918/225000 [278:46:18<153:13:26,  6.48s/it]
 62%|██████▏   | 139919/225000 [278:46:23<141:18:02,  5.98s/it]
 62%|██████▏   | 139920/225000 [278:46:28<134:13:12,  5.68s/it]
 62%|██████▏   | 139921/225000 [278:46:33<125:59:52,  5.33s/it]
 62%|██████▏   | 139922/225000 [278:46:38<123:35:51,  5.23s/it]
 62%|██████▏   | 139923/225000 [278:46:43<121:25:59,  5.14s/it]
 62%|██████▏   | 139924/225000 [278:46:48<120:49:09,  5.11s/it]
 62%|██████▏   | 139925/225000 [278:46:53<122:38:16,  5.19s/it]
 62%|██████▏   | 139926/225000 [278:46:58<121:55:28,  5.16s/it]
 62%|██████▏   | 139927/225000 [278:47:03<119:12:03,  5.04s/it]
 62%|██████▏   | 139928/225000 [278:47:12<147:54:51,  6.26s/it]
 62%|██████▏   | 139929/225000 [278:47:20<163:12:44,  6.91s/it]
 62%|██████▏   | 139930/225000 [278:47:24<142:36:11,  6.03s/it]
 62%|██████▏   | 139931/225000 [278:47:29<135:01:01,  5.71s/it]
 62%|██████▏   | 139932/225000 [278:47:34<127:31:59,  5.40s/it]
 62%|██████▏   | 139933/225000 [278:47:39<121:11:40,  5.13s/it]
 62%|██████▏   | 139934/225000 [278:47:44<120:47:53,  5.11s/it]
 62%|██████▏   | 139935/225000 [278:47:48<118:05:54,  5.00s/it]
 62%|██████▏   | 139936/225000 [278:47:57<141:32:39,  5.99s/it]
 62%|██████▏   | 139937/225000 [278:48:08<176:03:54,  7.45s/it]
 62%|██████▏   | 139938/225000 [278:48:11<149:45:11,  6.34s/it]
 62%|██████▏   | 139939/225000 [278:48:16<134:58:05,  5.71s/it]
 62%|██████▏   | 139940/225000 [278:48:20<125:08:17,  5.30s/it]
 62%|██████▏   | 139941/225000 [278:48:32<175:28:19,  7.43s/it]
 62%|██████▏   | 139942/225000 [278:48:36<150:34:54,  6.37s/it]
 62%|██████▏   | 139943/225000 [278:48:40<132:46:59,  5.62s/it]
 62%|██████▏   | 139944/225000 [278:48:44<121:07:29,  5.13s/it]
 62%|██████▏   | 139945/225000 [278:48:49<121:16:55,  5.13s/it]
 62%|██████▏   | 139946/225000 [278:48:54<120:48:29,  5.11s/it]
 62%|██████▏   | 139947/225000 [278:48:59<119:46:29,  5.07s/it]
 62%|██████▏   | 139948/225000 [278:49:11<165:56:26,  7.02s/it]
 62%|██████▏   | 139949/225000 [278:49:16<150:34:49,  6.37s/it]
 62%|██████▏   | 139950/225000 [278:49:20<135:43:49,  5.75s/it]
                                                               
{'loss': '0.7985', 'grad_norm': '1.12', 'learning_rate': '1.697e-05', 'epoch': '0.622', 'num_input_tokens_seen': 462375728, 'train_runtime': '1.004e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 139950/225000 [278:49:20<135:43:49,  5.75s/it]
 62%|██████▏   | 139951/225000 [278:49:25<129:23:02,  5.48s/it]
 62%|██████▏   | 139952/225000 [278:49:30<125:22:00,  5.31s/it]
 62%|██████▏   | 139953/225000 [278:49:34<120:50:04,  5.11s/it]
 62%|██████▏   | 139954/225000 [278:49:42<135:31:06,  5.74s/it]
 62%|██████▏   | 139955/225000 [278:49:52<167:22:32,  7.09s/it]
 62%|██████▏   | 139956/225000 [278:49:57<154:09:08,  6.53s/it]
 62%|██████▏   | 139957/225000 [278:50:03<147:14:37,  6.23s/it]
 62%|██████▏   | 139958/225000 [278:50:06<128:09:51,  5.43s/it]
 62%|██████▏   | 139959/225000 [278:50:13<139:17:48,  5.90s/it]
 62%|██████▏   | 139960/225000 [278:50:19<141:58:12,  6.01s/it]
 62%|██████▏   | 139961/225000 [278:50:28<157:17:02,  6.66s/it]
 62%|██████▏   | 139962/225000 [278:50:31<135:45:32,  5.75s/it]
 62%|██████▏   | 139963/225000 [278:50:35<122:14:05,  5.17s/it]
 62%|██████▏   | 139964/225000 [278:50:41<128:23:48,  5.44s/it]
 62%|██████▏   | 139965/225000 [278:50:49<147:57:58,  6.26s/it]
 62%|██████▏   | 139966/225000 [278:50:53<127:05:35,  5.38s/it]
 62%|██████▏   | 139967/225000 [278:50:59<137:29:23,  5.82s/it]
 62%|██████▏   | 139968/225000 [278:51:06<145:07:54,  6.14s/it]
 62%|██████▏   | 139969/225000 [278:51:11<131:38:49,  5.57s/it]
 62%|██████▏   | 139970/225000 [278:51:19<150:27:15,  6.37s/it]
 62%|██████▏   | 139971/225000 [278:51:22<131:01:45,  5.55s/it]
 62%|██████▏   | 139972/225000 [278:51:31<152:04:05,  6.44s/it]
 62%|██████▏   | 139973/225000 [278:51:34<131:08:26,  5.55s/it]
 62%|██████▏   | 139974/225000 [278:51:42<146:07:24,  6.19s/it]
 62%|██████▏   | 139975/225000 [278:51:46<133:21:30,  5.65s/it]
 62%|██████▏   | 139976/225000 [278:51:51<122:16:49,  5.18s/it]
 62%|██████▏   | 139977/225000 [278:51:55<117:09:53,  4.96s/it]
 62%|██████▏   | 139978/225000 [278:51:59<112:48:16,  4.78s/it]
 62%|██████▏   | 139979/225000 [278:52:04<110:13:41,  4.67s/it]
 62%|██████▏   | 139980/225000 [278:52:11<131:19:16,  5.56s/it]
 62%|██████▏   | 139981/225000 [278:52:21<160:42:05,  6.80s/it]
 62%|██████▏   | 139982/225000 [278:52:25<137:08:13,  5.81s/it]
 62%|██████▏   | 139983/225000 [278:52:32<146:39:02,  6.21s/it]
 62%|██████▏   | 139984/225000 [278:52:36<131:44:28,  5.58s/it]
 62%|██████▏   | 139985/225000 [278:52:40<121:32:12,  5.15s/it]
 62%|██████▏   | 139986/225000 [278:52:48<139:43:53,  5.92s/it]
 62%|██████▏   | 139987/225000 [278:52:54<139:10:47,  5.89s/it]
 62%|██████▏   | 139988/225000 [278:52:57<123:33:40,  5.23s/it]
 62%|██████▏   | 139989/225000 [278:53:06<148:11:24,  6.28s/it]
 62%|██████▏   | 139990/225000 [278:53:13<152:13:37,  6.45s/it]
 62%|██████▏   | 139991/225000 [278:53:19<148:20:02,  6.28s/it]
 62%|██████▏   | 139992/225000 [278:53:22<130:01:56,  5.51s/it]
 62%|██████▏   | 139993/225000 [278:53:27<124:17:01,  5.26s/it]
 62%|██████▏   | 139994/225000 [278:53:32<120:21:41,  5.10s/it]
 62%|██████▏   | 139995/225000 [278:53:37<118:10:38,  5.00s/it]
 62%|██████▏   | 139996/225000 [278:53:41<116:36:37,  4.94s/it]
 62%|██████▏   | 139997/225000 [278:53:49<139:16:24,  5.90s/it]
 62%|██████▏   | 139998/225000 [278:54:01<176:41:22,  7.48s/it]
 62%|██████▏   | 139999/225000 [278:54:04<147:41:59,  6.26s/it]
 62%|██████▏   | 140000/225000 [278:54:12<158:31:33,  6.71s/it]
                                                               
{'loss': '0.796', 'grad_norm': '1.014', 'learning_rate': '1.695e-05', 'epoch': '0.6222', 'num_input_tokens_seen': 462505536, 'train_runtime': '1.004e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140000/225000 [278:54:12<158:31:33,  6.71s/it]
 62%|██████▏   | 140001/225000 [278:54:17<144:36:13,  6.12s/it]
 62%|██████▏   | 140002/225000 [278:54:21<134:48:55,  5.71s/it]
 62%|██████▏   | 140003/225000 [278:54:26<128:39:49,  5.45s/it]
 62%|██████▏   | 140004/225000 [278:54:31<124:02:07,  5.25s/it]
 62%|██████▏   | 140005/225000 [278:54:36<120:14:22,  5.09s/it]
 62%|██████▏   | 140006/225000 [278:54:41<119:52:37,  5.08s/it]
 62%|██████▏   | 140007/225000 [278:54:45<116:05:36,  4.92s/it]
 62%|██████▏   | 140008/225000 [278:54:57<163:57:07,  6.94s/it]
 62%|██████▏   | 140009/225000 [278:55:02<153:49:54,  6.52s/it]
 62%|██████▏   | 140010/225000 [278:55:09<155:59:56,  6.61s/it]
 62%|██████▏   | 140011/225000 [278:55:13<136:48:14,  5.79s/it]
 62%|██████▏   | 140012/225000 [278:55:18<128:10:52,  5.43s/it]
 62%|██████▏   | 140013/225000 [278:55:22<118:23:28,  5.01s/it]
 62%|██████▏   | 140014/225000 [278:55:33<162:50:26,  6.90s/it]
 62%|██████▏   | 140015/225000 [278:55:37<143:13:46,  6.07s/it]
 62%|██████▏   | 140016/225000 [278:55:41<129:41:59,  5.49s/it]
 62%|██████▏   | 140017/225000 [278:55:45<118:51:28,  5.03s/it]
 62%|██████▏   | 140018/225000 [278:55:49<110:09:39,  4.67s/it]
 62%|██████▏   | 140019/225000 [278:56:01<157:56:52,  6.69s/it]
 62%|██████▏   | 140020/225000 [278:56:04<136:15:07,  5.77s/it]
 62%|██████▏   | 140021/225000 [278:56:08<125:29:19,  5.32s/it]
 62%|██████▏   | 140022/225000 [278:56:17<149:59:37,  6.35s/it]
 62%|██████▏   | 140023/225000 [278:56:21<135:00:15,  5.72s/it]
 62%|██████▏   | 140024/225000 [278:56:26<126:23:51,  5.35s/it]
 62%|██████▏   | 140025/225000 [278:56:30<120:15:15,  5.09s/it]
 62%|██████▏   | 140026/225000 [278:56:35<114:51:57,  4.87s/it]
 62%|██████▏   | 140027/225000 [278:56:39<113:03:22,  4.79s/it]
 62%|██████▏   | 140028/225000 [278:56:50<153:45:37,  6.51s/it]
 62%|██████▏   | 140029/225000 [278:56:56<147:51:44,  6.26s/it]
 62%|██████▏   | 140030/225000 [278:57:03<152:45:22,  6.47s/it]
 62%|██████▏   | 140031/225000 [278:57:09<152:10:03,  6.45s/it]
 62%|██████▏   | 140032/225000 [278:57:13<131:41:28,  5.58s/it]
 62%|██████▏   | 140033/225000 [278:57:18<131:16:41,  5.56s/it]
 62%|██████▏   | 140034/225000 [278:57:23<128:52:53,  5.46s/it]
 62%|██████▏   | 140035/225000 [278:57:29<128:03:16,  5.43s/it]
 62%|██████▏   | 140036/225000 [278:57:34<126:47:16,  5.37s/it]
 62%|██████▏   | 140037/225000 [278:57:39<126:04:37,  5.34s/it]
 62%|██████▏   | 140038/225000 [278:57:44<125:02:32,  5.30s/it]
 62%|██████▏   | 140039/225000 [278:57:50<125:36:04,  5.32s/it]
 62%|██████▏   | 140040/225000 [278:57:55<125:04:13,  5.30s/it]
 62%|██████▏   | 140041/225000 [278:58:00<125:32:26,  5.32s/it]
 62%|██████▏   | 140042/225000 [278:58:05<122:01:43,  5.17s/it]
 62%|██████▏   | 140043/225000 [278:58:18<174:06:22,  7.38s/it]
 62%|██████▏   | 140044/225000 [278:58:25<172:01:03,  7.29s/it]
 62%|██████▏   | 140045/225000 [278:58:28<144:55:12,  6.14s/it]
 62%|██████▏   | 140046/225000 [278:58:35<148:05:52,  6.28s/it]
 62%|██████▏   | 140047/225000 [278:58:41<149:23:59,  6.33s/it]
 62%|██████▏   | 140048/225000 [278:58:45<131:16:29,  5.56s/it]
 62%|██████▏   | 140049/225000 [278:58:52<141:37:50,  6.00s/it]
 62%|██████▏   | 140050/225000 [278:58:59<147:21:38,  6.24s/it]
                                                               
{'loss': '0.8035', 'grad_norm': '0.9779', 'learning_rate': '1.693e-05', 'epoch': '0.6224', 'num_input_tokens_seen': 462635488, 'train_runtime': '1.004e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140050/225000 [278:58:59<147:21:38,  6.24s/it]
 62%|██████▏   | 140051/225000 [278:59:03<128:54:58,  5.46s/it]
 62%|██████▏   | 140052/225000 [278:59:12<160:08:17,  6.79s/it]
 62%|██████▏   | 140053/225000 [278:59:16<137:49:40,  5.84s/it]
 62%|██████▏   | 140054/225000 [278:59:25<156:56:58,  6.65s/it]
 62%|██████▏   | 140055/225000 [278:59:29<138:34:00,  5.87s/it]
 62%|██████▏   | 140056/225000 [278:59:33<125:35:51,  5.32s/it]
 62%|██████▏   | 140057/225000 [278:59:40<141:08:58,  5.98s/it]
 62%|██████▏   | 140058/225000 [278:59:47<147:33:01,  6.25s/it]
 62%|██████▏   | 140059/225000 [278:59:51<129:53:39,  5.51s/it]
 62%|██████▏   | 140060/225000 [279:00:00<155:06:03,  6.57s/it]
 62%|██████▏   | 140061/225000 [279:00:04<138:07:43,  5.85s/it]
 62%|██████▏   | 140062/225000 [279:00:09<130:22:24,  5.53s/it]
 62%|██████▏   | 140063/225000 [279:00:14<126:58:10,  5.38s/it]
 62%|██████▏   | 140064/225000 [279:00:18<121:27:27,  5.15s/it]
 62%|██████▏   | 140065/225000 [279:00:23<116:49:27,  4.95s/it]
 62%|██████▏   | 140066/225000 [279:00:28<115:01:14,  4.88s/it]
 62%|██████▏   | 140067/225000 [279:00:32<113:53:15,  4.83s/it]
 62%|██████▏   | 140068/225000 [279:00:42<148:55:55,  6.31s/it]
 62%|██████▏   | 140069/225000 [279:00:51<165:37:49,  7.02s/it]
 62%|██████▏   | 140070/225000 [279:00:55<146:42:18,  6.22s/it]
 62%|██████▏   | 140071/225000 [279:00:59<132:55:44,  5.63s/it]
 62%|██████▏   | 140072/225000 [279:01:04<124:16:13,  5.27s/it]
 62%|██████▏   | 140073/225000 [279:01:09<120:30:23,  5.11s/it]
 62%|██████▏   | 140074/225000 [279:01:18<148:02:46,  6.28s/it]
 62%|██████▏   | 140075/225000 [279:01:23<143:20:12,  6.08s/it]
 62%|██████▏   | 140076/225000 [279:01:28<131:43:09,  5.58s/it]
 62%|██████▏   | 140077/225000 [279:01:32<124:35:45,  5.28s/it]
 62%|██████▏   | 140078/225000 [279:01:36<115:28:26,  4.90s/it]
 62%|██████▏   | 140079/225000 [279:01:45<142:12:45,  6.03s/it]
 62%|██████▏   | 140080/225000 [279:01:55<170:01:59,  7.21s/it]
 62%|██████▏   | 140081/225000 [279:01:59<150:51:47,  6.40s/it]
 62%|██████▏   | 140082/225000 [279:02:04<136:47:00,  5.80s/it]
 62%|██████▏   | 140083/225000 [279:02:08<129:06:19,  5.47s/it]
 62%|██████▏   | 140084/225000 [279:02:13<123:16:56,  5.23s/it]
 62%|██████▏   | 140085/225000 [279:02:18<118:43:30,  5.03s/it]
 62%|██████▏   | 140086/225000 [279:02:23<120:01:58,  5.09s/it]
 62%|██████▏   | 140087/225000 [279:02:29<125:29:58,  5.32s/it]
 62%|██████▏   | 140088/225000 [279:02:42<178:00:32,  7.55s/it]
 62%|██████▏   | 140089/225000 [279:02:45<151:13:21,  6.41s/it]
 62%|██████▏   | 140090/225000 [279:02:49<130:21:35,  5.53s/it]
 62%|██████▏   | 140091/225000 [279:02:59<160:33:17,  6.81s/it]
 62%|██████▏   | 140092/225000 [279:03:03<142:57:57,  6.06s/it]
 62%|██████▏   | 140093/225000 [279:03:07<129:18:26,  5.48s/it]
 62%|██████▏   | 140094/225000 [279:03:12<123:28:29,  5.24s/it]
 62%|██████▏   | 140095/225000 [279:03:16<116:51:41,  4.95s/it]
 62%|██████▏   | 140096/225000 [279:03:20<112:35:38,  4.77s/it]
 62%|██████▏   | 140097/225000 [279:03:29<142:53:02,  6.06s/it]
 62%|██████▏   | 140098/225000 [279:03:38<159:27:32,  6.76s/it]
 62%|██████▏   | 140099/225000 [279:03:41<133:18:12,  5.65s/it]
 62%|██████▏   | 140100/225000 [279:03:46<130:08:38,  5.52s/it]
                                                               
{'loss': '0.7948', 'grad_norm': '1.206', 'learning_rate': '1.692e-05', 'epoch': '0.6227', 'num_input_tokens_seen': 462765200, 'train_runtime': '1.005e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140100/225000 [279:03:46<130:08:38,  5.52s/it]
 62%|██████▏   | 140101/225000 [279:03:51<129:19:19,  5.48s/it]
 62%|██████▏   | 140102/225000 [279:03:56<125:46:39,  5.33s/it]
 62%|██████▏   | 140103/225000 [279:04:02<127:18:53,  5.40s/it]
 62%|██████▏   | 140104/225000 [279:04:07<126:18:14,  5.36s/it]
 62%|██████▏   | 140105/225000 [279:04:13<125:46:35,  5.33s/it]
 62%|██████▏   | 140106/225000 [279:04:17<122:42:19,  5.20s/it]
 62%|██████▏   | 140107/225000 [279:04:22<121:04:22,  5.13s/it]
 62%|██████▏   | 140108/225000 [279:04:27<118:12:48,  5.01s/it]
 62%|██████▏   | 140109/225000 [279:04:32<119:40:27,  5.08s/it]
 62%|██████▏   | 140110/225000 [279:04:38<121:08:46,  5.14s/it]
 62%|██████▏   | 140111/225000 [279:04:43<122:49:31,  5.21s/it]
 62%|██████▏   | 140112/225000 [279:04:48<121:04:51,  5.13s/it]
 62%|██████▏   | 140113/225000 [279:04:53<122:54:15,  5.21s/it]
 62%|██████▏   | 140114/225000 [279:05:05<169:45:56,  7.20s/it]
 62%|██████▏   | 140115/225000 [279:05:11<161:30:47,  6.85s/it]
 62%|██████▏   | 140116/225000 [279:05:15<138:15:52,  5.86s/it]
 62%|██████▏   | 140117/225000 [279:05:24<163:22:05,  6.93s/it]
 62%|██████▏   | 140118/225000 [279:05:29<149:13:45,  6.33s/it]
 62%|██████▏   | 140119/225000 [279:05:33<132:31:21,  5.62s/it]
 62%|██████▏   | 140120/225000 [279:05:37<118:41:32,  5.03s/it]
 62%|██████▏   | 140121/225000 [279:05:44<137:15:30,  5.82s/it]
 62%|██████▏   | 140122/225000 [279:05:51<146:09:06,  6.20s/it]
 62%|██████▏   | 140123/225000 [279:05:55<127:37:36,  5.41s/it]
 62%|██████▏   | 140124/225000 [279:05:59<116:53:01,  4.96s/it]
 62%|██████▏   | 140125/225000 [279:06:07<137:00:00,  5.81s/it]
 62%|██████▏   | 140126/225000 [279:06:13<142:24:59,  6.04s/it]
 62%|██████▏   | 140127/225000 [279:06:19<137:22:51,  5.83s/it]
 62%|██████▏   | 140128/225000 [279:06:23<130:01:53,  5.52s/it]
 62%|██████▏   | 140129/225000 [279:06:29<127:06:49,  5.39s/it]
 62%|██████▏   | 140130/225000 [279:06:34<125:01:49,  5.30s/it]
 62%|██████▏   | 140131/225000 [279:06:38<120:51:37,  5.13s/it]
 62%|██████▏   | 140132/225000 [279:06:43<118:59:12,  5.05s/it]
 62%|██████▏   | 140133/225000 [279:06:48<119:40:22,  5.08s/it]
 62%|██████▏   | 140134/225000 [279:06:56<139:00:32,  5.90s/it]
 62%|██████▏   | 140135/225000 [279:07:07<176:37:14,  7.49s/it]
 62%|██████▏   | 140136/225000 [279:07:11<145:35:14,  6.18s/it]
 62%|██████▏   | 140137/225000 [279:07:15<134:15:39,  5.70s/it]
 62%|██████▏   | 140138/225000 [279:07:20<126:15:32,  5.36s/it]
 62%|██████▏   | 140139/225000 [279:07:24<120:38:21,  5.12s/it]
 62%|██████▏   | 140140/225000 [279:07:29<118:18:26,  5.02s/it]
 62%|██████▏   | 140141/225000 [279:07:39<154:55:56,  6.57s/it]
 62%|██████▏   | 140142/225000 [279:07:49<178:38:42,  7.58s/it]
 62%|██████▏   | 140143/225000 [279:07:53<155:12:03,  6.58s/it]
 62%|██████▏   | 140144/225000 [279:07:58<138:26:10,  5.87s/it]
 62%|██████▏   | 140145/225000 [279:08:02<126:53:09,  5.38s/it]
 62%|██████▏   | 140146/225000 [279:08:06<118:10:44,  5.01s/it]
 62%|██████▏   | 140147/225000 [279:08:15<143:30:36,  6.09s/it]
 62%|██████▏   | 140148/225000 [279:08:22<149:27:47,  6.34s/it]
 62%|██████▏   | 140149/225000 [279:08:27<139:54:21,  5.94s/it]
 62%|██████▏   | 140150/225000 [279:08:32<133:48:51,  5.68s/it]
                                                               
{'loss': '0.7819', 'grad_norm': '1.089', 'learning_rate': '1.69e-05', 'epoch': '0.6229', 'num_input_tokens_seen': 462895376, 'train_runtime': '1.005e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140150/225000 [279:08:32<133:48:51,  5.68s/it]
 62%|██████▏   | 140151/225000 [279:08:37<134:12:57,  5.69s/it]
 62%|██████▏   | 140152/225000 [279:08:48<166:36:53,  7.07s/it]
 62%|██████▏   | 140153/225000 [279:08:51<139:50:13,  5.93s/it]
 62%|██████▏   | 140154/225000 [279:08:58<147:33:34,  6.26s/it]
 62%|██████▏   | 140155/225000 [279:09:01<126:42:32,  5.38s/it]
 62%|██████▏   | 140156/225000 [279:09:10<149:44:43,  6.35s/it]
 62%|██████▏   | 140157/225000 [279:09:13<129:47:51,  5.51s/it]
 62%|██████▏   | 140158/225000 [279:09:23<159:55:17,  6.79s/it]
 62%|██████▏   | 140159/225000 [279:09:28<145:37:54,  6.18s/it]
 62%|██████▏   | 140160/225000 [279:09:34<146:02:55,  6.20s/it]
 62%|██████▏   | 140161/225000 [279:09:38<130:45:03,  5.55s/it]
 62%|██████▏   | 140162/225000 [279:09:43<124:27:42,  5.28s/it]
 62%|██████▏   | 140163/225000 [279:09:48<120:51:33,  5.13s/it]
 62%|██████▏   | 140164/225000 [279:09:58<156:24:24,  6.64s/it]
 62%|██████▏   | 140165/225000 [279:10:01<134:14:47,  5.70s/it]
 62%|██████▏   | 140166/225000 [279:10:10<154:24:47,  6.55s/it]
 62%|██████▏   | 140167/225000 [279:10:13<131:11:23,  5.57s/it]
 62%|██████▏   | 140168/225000 [279:10:21<147:45:44,  6.27s/it]
 62%|██████▏   | 140169/225000 [279:10:24<127:44:16,  5.42s/it]
 62%|██████▏   | 140170/225000 [279:10:32<145:27:14,  6.17s/it]
 62%|██████▏   | 140171/225000 [279:10:36<128:58:15,  5.47s/it]
 62%|██████▏   | 140172/225000 [279:10:43<135:41:59,  5.76s/it]
 62%|██████▏   | 140173/225000 [279:10:51<152:03:01,  6.45s/it]
 62%|██████▏   | 140174/225000 [279:10:54<128:40:30,  5.46s/it]
 62%|██████▏   | 140175/225000 [279:11:03<155:36:14,  6.60s/it]
 62%|██████▏   | 140176/225000 [279:11:07<139:26:04,  5.92s/it]
 62%|██████▏   | 140177/225000 [279:11:14<146:21:14,  6.21s/it]
 62%|██████▏   | 140178/225000 [279:11:20<142:14:48,  6.04s/it]
 62%|██████▏   | 140179/225000 [279:11:25<132:26:05,  5.62s/it]
 62%|██████▏   | 140180/225000 [279:11:30<129:12:00,  5.48s/it]
 62%|██████▏   | 140181/225000 [279:11:35<125:41:21,  5.33s/it]
 62%|██████▏   | 140182/225000 [279:11:40<123:33:04,  5.24s/it]
 62%|██████▏   | 140183/225000 [279:11:45<122:50:52,  5.21s/it]
 62%|██████▏   | 140184/225000 [279:11:50<124:00:11,  5.26s/it]
 62%|██████▏   | 140185/225000 [279:11:55<122:01:39,  5.18s/it]
 62%|██████▏   | 140186/225000 [279:12:00<119:03:28,  5.05s/it]
 62%|██████▏   | 140187/225000 [279:12:05<119:13:22,  5.06s/it]
 62%|██████▏   | 140188/225000 [279:12:16<163:30:42,  6.94s/it]
 62%|██████▏   | 140189/225000 [279:12:21<143:55:33,  6.11s/it]
 62%|██████▏   | 140190/225000 [279:12:25<130:14:07,  5.53s/it]
 62%|██████▏   | 140191/225000 [279:12:29<123:25:35,  5.24s/it]
 62%|██████▏   | 140192/225000 [279:12:34<117:46:18,  5.00s/it]
 62%|██████▏   | 140193/225000 [279:12:47<173:54:54,  7.38s/it]
 62%|██████▏   | 140194/225000 [279:12:51<148:20:53,  6.30s/it]
 62%|██████▏   | 140195/225000 [279:12:55<132:30:44,  5.63s/it]
 62%|██████▏   | 140196/225000 [279:12:59<120:57:29,  5.13s/it]
 62%|██████▏   | 140197/225000 [279:13:08<150:13:05,  6.38s/it]
 62%|██████▏   | 140198/225000 [279:13:12<135:12:03,  5.74s/it]
 62%|██████▏   | 140199/225000 [279:13:16<124:31:38,  5.29s/it]
 62%|██████▏   | 140200/225000 [279:13:21<118:08:04,  5.02s/it]
                                                               
{'loss': '0.7862', 'grad_norm': '0.9905', 'learning_rate': '1.688e-05', 'epoch': '0.6231', 'num_input_tokens_seen': 463025024, 'train_runtime': '1.005e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140200/225000 [279:13:21<118:08:04,  5.02s/it]
 62%|██████▏   | 140201/225000 [279:13:29<142:50:17,  6.06s/it]
 62%|██████▏   | 140202/225000 [279:13:38<159:50:56,  6.79s/it]
 62%|██████▏   | 140203/225000 [279:13:41<137:35:24,  5.84s/it]
 62%|██████▏   | 140204/225000 [279:13:46<130:52:08,  5.56s/it]
 62%|██████▏   | 140205/225000 [279:13:52<130:08:09,  5.52s/it]
 62%|██████▏   | 140206/225000 [279:13:57<127:04:02,  5.39s/it]
 62%|██████▏   | 140207/225000 [279:14:02<126:03:19,  5.35s/it]
 62%|██████▏   | 140208/225000 [279:14:08<127:39:44,  5.42s/it]
 62%|██████▏   | 140209/225000 [279:14:13<126:24:18,  5.37s/it]
 62%|██████▏   | 140210/225000 [279:14:18<127:35:55,  5.42s/it]
 62%|██████▏   | 140211/225000 [279:14:24<126:46:28,  5.38s/it]
 62%|██████▏   | 140212/225000 [279:14:29<126:46:20,  5.38s/it]
 62%|██████▏   | 140213/225000 [279:14:34<124:19:24,  5.28s/it]
 62%|██████▏   | 140214/225000 [279:14:43<149:07:20,  6.33s/it]
 62%|██████▏   | 140215/225000 [279:14:54<186:02:26,  7.90s/it]
 62%|██████▏   | 140216/225000 [279:14:58<157:15:16,  6.68s/it]
 62%|██████▏   | 140217/225000 [279:15:06<167:22:38,  7.11s/it]
 62%|██████▏   | 140218/225000 [279:15:11<146:45:38,  6.23s/it]
 62%|██████▏   | 140219/225000 [279:15:15<134:11:36,  5.70s/it]
 62%|██████▏   | 140220/225000 [279:15:28<183:15:27,  7.78s/it]
 62%|██████▏   | 140221/225000 [279:15:32<157:17:58,  6.68s/it]
 62%|██████▏   | 140222/225000 [279:15:36<139:26:08,  5.92s/it]
 62%|██████▏   | 140223/225000 [279:15:40<125:58:43,  5.35s/it]
 62%|██████▏   | 140224/225000 [279:15:44<117:49:36,  5.00s/it]
 62%|██████▏   | 140225/225000 [279:15:49<114:31:55,  4.86s/it]
 62%|██████▏   | 140226/225000 [279:16:00<156:53:10,  6.66s/it]
 62%|██████▏   | 140227/225000 [279:16:07<159:52:45,  6.79s/it]
 62%|██████▏   | 140228/225000 [279:16:11<141:31:58,  6.01s/it]
 62%|██████▏   | 140229/225000 [279:16:15<126:02:14,  5.35s/it]
 62%|██████▏   | 140230/225000 [279:16:18<114:42:24,  4.87s/it]
 62%|██████▏   | 140231/225000 [279:16:29<156:05:10,  6.63s/it]
 62%|██████▏   | 140232/225000 [279:16:34<142:25:02,  6.05s/it]
 62%|██████▏   | 140233/225000 [279:16:38<132:43:27,  5.64s/it]
 62%|██████▏   | 140234/225000 [279:16:43<123:53:15,  5.26s/it]
 62%|██████▏   | 140235/225000 [279:16:48<122:26:42,  5.20s/it]
 62%|██████▏   | 140236/225000 [279:16:53<120:02:42,  5.10s/it]
 62%|██████▏   | 140237/225000 [279:17:05<173:19:50,  7.36s/it]
 62%|██████▏   | 140238/225000 [279:17:10<153:27:46,  6.52s/it]
 62%|██████▏   | 140239/225000 [279:17:19<173:15:13,  7.36s/it]
 62%|██████▏   | 140240/225000 [279:17:22<143:48:43,  6.11s/it]
 62%|██████▏   | 140241/225000 [279:17:29<149:15:41,  6.34s/it]
 62%|██████▏   | 140242/225000 [279:17:35<142:06:02,  6.04s/it]
 62%|██████▏   | 140243/225000 [279:17:40<137:00:51,  5.82s/it]
 62%|██████▏   | 140244/225000 [279:17:44<121:21:32,  5.15s/it]
 62%|██████▏   | 140245/225000 [279:17:51<134:46:07,  5.72s/it]
 62%|██████▏   | 140246/225000 [279:17:58<142:58:48,  6.07s/it]
 62%|██████▏   | 140247/225000 [279:18:01<124:53:10,  5.30s/it]
 62%|██████▏   | 140248/225000 [279:18:11<154:22:12,  6.56s/it]
 62%|██████▏   | 140249/225000 [279:18:15<139:02:32,  5.91s/it]
 62%|██████▏   | 140250/225000 [279:18:21<142:38:44,  6.06s/it]
                                                               
{'loss': '0.7934', 'grad_norm': '1.191', 'learning_rate': '1.686e-05', 'epoch': '0.6233', 'num_input_tokens_seen': 463155568, 'train_runtime': '1.006e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140250/225000 [279:18:21<142:38:44,  6.06s/it]
 62%|██████▏   | 140251/225000 [279:18:26<132:41:42,  5.64s/it]
 62%|██████▏   | 140252/225000 [279:18:30<119:04:11,  5.06s/it]
 62%|██████▏   | 140253/225000 [279:18:34<113:34:43,  4.82s/it]
 62%|██████▏   | 140254/225000 [279:18:41<129:09:32,  5.49s/it]
 62%|██████▏   | 140255/225000 [279:18:52<168:18:16,  7.15s/it]
 62%|██████▏   | 140256/225000 [279:18:56<146:49:49,  6.24s/it]
 62%|██████▏   | 140257/225000 [279:19:01<133:34:42,  5.67s/it]
 62%|██████▏   | 140258/225000 [279:19:07<136:56:28,  5.82s/it]
 62%|██████▏   | 140259/225000 [279:19:16<160:23:55,  6.81s/it]
 62%|██████▏   | 140260/225000 [279:19:20<138:50:21,  5.90s/it]
 62%|██████▏   | 140261/225000 [279:19:24<126:06:51,  5.36s/it]
 62%|██████▏   | 140262/225000 [279:19:31<142:37:46,  6.06s/it]
 62%|██████▏   | 140263/225000 [279:19:39<155:54:24,  6.62s/it]
 62%|██████▏   | 140264/225000 [279:19:44<140:57:26,  5.99s/it]
 62%|██████▏   | 140265/225000 [279:19:50<144:02:11,  6.12s/it]
 62%|██████▏   | 140266/225000 [279:19:54<124:03:35,  5.27s/it]
 62%|██████▏   | 140267/225000 [279:20:00<133:58:12,  5.69s/it]
 62%|██████▏   | 140268/225000 [279:20:07<140:30:35,  5.97s/it]
 62%|██████▏   | 140269/225000 [279:20:10<122:24:07,  5.20s/it]
 62%|██████▏   | 140270/225000 [279:20:18<143:04:29,  6.08s/it]
 62%|██████▏   | 140271/225000 [279:20:22<126:40:53,  5.38s/it]
 62%|██████▏   | 140272/225000 [279:20:27<121:50:34,  5.18s/it]
 62%|██████▏   | 140273/225000 [279:20:32<123:39:44,  5.25s/it]
 62%|██████▏   | 140274/225000 [279:20:37<120:07:12,  5.10s/it]
 62%|██████▏   | 140275/225000 [279:20:42<117:57:53,  5.01s/it]
 62%|██████▏   | 140276/225000 [279:20:50<139:43:14,  5.94s/it]
 62%|██████▏   | 140277/225000 [279:21:01<174:29:53,  7.41s/it]
 62%|██████▏   | 140278/225000 [279:21:06<158:03:02,  6.72s/it]
 62%|██████▏   | 140279/225000 [279:21:10<142:01:36,  6.04s/it]
 62%|██████▏   | 140280/225000 [279:21:15<132:48:12,  5.64s/it]
 62%|██████▏   | 140281/225000 [279:21:20<126:53:58,  5.39s/it]
 62%|██████▏   | 140282/225000 [279:21:25<124:48:22,  5.30s/it]
 62%|██████▏   | 140283/225000 [279:21:29<119:19:55,  5.07s/it]
 62%|██████▏   | 140284/225000 [279:21:34<118:13:05,  5.02s/it]
 62%|██████▏   | 140285/225000 [279:21:44<147:50:35,  6.28s/it]
 62%|██████▏   | 140286/225000 [279:21:51<158:12:37,  6.72s/it]
 62%|██████▏   | 140287/225000 [279:21:55<139:42:56,  5.94s/it]
 62%|██████▏   | 140288/225000 [279:22:01<135:07:30,  5.74s/it]
 62%|██████▏   | 140289/225000 [279:22:10<156:52:56,  6.67s/it]
 62%|██████▏   | 140290/225000 [279:22:14<141:14:17,  6.00s/it]
 62%|██████▏   | 140291/225000 [279:22:18<129:21:22,  5.50s/it]
 62%|██████▏   | 140292/225000 [279:22:23<121:45:35,  5.17s/it]
 62%|██████▏   | 140293/225000 [279:22:28<122:19:03,  5.20s/it]
 62%|██████▏   | 140294/225000 [279:22:38<158:43:02,  6.75s/it]
 62%|██████▏   | 140295/225000 [279:22:42<134:42:09,  5.72s/it]
 62%|██████▏   | 140296/225000 [279:22:51<161:38:39,  6.87s/it]
 62%|██████▏   | 140297/225000 [279:22:54<136:09:00,  5.79s/it]
 62%|██████▏   | 140298/225000 [279:22:59<126:49:20,  5.39s/it]
 62%|██████▏   | 140299/225000 [279:23:03<120:20:37,  5.11s/it]
 62%|██████▏   | 140300/225000 [279:23:08<116:14:23,  4.94s/it]
                                                               
{'loss': '0.7966', 'grad_norm': '1.022', 'learning_rate': '1.685e-05', 'epoch': '0.6236', 'num_input_tokens_seen': 463284792, 'train_runtime': '1.006e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140300/225000 [279:23:08<116:14:23,  4.94s/it]
 62%|██████▏   | 140301/225000 [279:23:20<165:56:19,  7.05s/it]
 62%|██████▏   | 140302/225000 [279:23:23<139:26:22,  5.93s/it]
 62%|██████▏   | 140303/225000 [279:23:32<160:37:10,  6.83s/it]
 62%|██████▏   | 140304/225000 [279:23:36<137:05:00,  5.83s/it]
 62%|██████▏   | 140305/225000 [279:23:44<152:51:26,  6.50s/it]
 62%|██████▏   | 140306/225000 [279:23:47<132:17:29,  5.62s/it]
 62%|██████▏   | 140307/225000 [279:23:51<119:52:28,  5.10s/it]
 62%|██████▏   | 140308/225000 [279:23:58<129:56:07,  5.52s/it]
 62%|██████▏   | 140309/225000 [279:24:08<161:22:22,  6.86s/it]
 62%|██████▏   | 140310/225000 [279:24:12<146:03:42,  6.21s/it]
 62%|██████▏   | 140311/225000 [279:24:17<137:23:56,  5.84s/it]
 62%|██████▏   | 140312/225000 [279:24:22<128:19:51,  5.46s/it]
 62%|██████▏   | 140313/225000 [279:24:27<122:58:52,  5.23s/it]
 62%|██████▏   | 140314/225000 [279:24:31<118:35:33,  5.04s/it]
 62%|██████▏   | 140315/225000 [279:24:36<115:46:38,  4.92s/it]
 62%|██████▏   | 140316/225000 [279:24:48<169:43:14,  7.21s/it]
 62%|██████▏   | 140317/225000 [279:24:53<149:16:39,  6.35s/it]
 62%|██████▏   | 140318/225000 [279:24:57<133:55:10,  5.69s/it]
 62%|██████▏   | 140319/225000 [279:25:06<160:36:30,  6.83s/it]
 62%|██████▏   | 140320/225000 [279:25:11<143:01:23,  6.08s/it]
 62%|██████▏   | 140321/225000 [279:25:16<136:05:01,  5.79s/it]
 62%|██████▏   | 140322/225000 [279:25:20<127:11:27,  5.41s/it]
 62%|██████▏   | 140323/225000 [279:25:25<123:00:28,  5.23s/it]
 62%|██████▏   | 140324/225000 [279:25:30<119:36:12,  5.08s/it]
 62%|██████▏   | 140325/225000 [279:25:36<127:26:23,  5.42s/it]
 62%|██████▏   | 140326/225000 [279:25:48<174:31:37,  7.42s/it]
 62%|██████▏   | 140327/225000 [279:25:51<145:13:51,  6.17s/it]
 62%|██████▏   | 140328/225000 [279:25:57<143:34:56,  6.10s/it]
 62%|██████▏   | 140329/225000 [279:26:04<149:57:33,  6.38s/it]
 62%|██████▏   | 140330/225000 [279:26:08<131:45:40,  5.60s/it]
 62%|██████▏   | 140331/225000 [279:26:14<129:54:36,  5.52s/it]
 62%|██████▏   | 140332/225000 [279:26:22<151:17:08,  6.43s/it]
 62%|██████▏   | 140333/225000 [279:26:27<138:37:59,  5.89s/it]
 62%|██████▏   | 140334/225000 [279:26:31<126:54:02,  5.40s/it]
 62%|██████▏   | 140335/225000 [279:26:35<120:04:04,  5.11s/it]
 62%|██████▏   | 140336/225000 [279:26:40<115:22:32,  4.91s/it]
 62%|██████▏   | 140337/225000 [279:26:45<116:16:19,  4.94s/it]
 62%|██████▏   | 140338/225000 [279:26:57<163:57:15,  6.97s/it]
 62%|██████▏   | 140339/225000 [279:27:00<140:53:27,  5.99s/it]
 62%|██████▏   | 140340/225000 [279:27:07<147:19:41,  6.26s/it]
 62%|██████▏   | 140341/225000 [279:27:14<148:01:23,  6.29s/it]
 62%|██████▏   | 140342/225000 [279:27:17<128:15:05,  5.45s/it]
 62%|██████▏   | 140343/225000 [279:27:26<155:29:30,  6.61s/it]
 62%|██████▏   | 140344/225000 [279:27:30<132:30:48,  5.64s/it]
 62%|██████▏   | 140345/225000 [279:27:38<152:24:37,  6.48s/it]
 62%|██████▏   | 140346/225000 [279:27:42<130:31:44,  5.55s/it]
 62%|██████▏   | 140347/225000 [279:27:46<121:35:20,  5.17s/it]
 62%|██████▏   | 140348/225000 [279:27:51<118:18:16,  5.03s/it]
 62%|██████▏   | 140349/225000 [279:27:56<118:56:05,  5.06s/it]
 62%|██████▏   | 140350/225000 [279:28:01<119:02:24,  5.06s/it]
                                                               
{'loss': '0.8067', 'grad_norm': '1.104', 'learning_rate': '1.683e-05', 'epoch': '0.6238', 'num_input_tokens_seen': 463416160, 'train_runtime': '1.006e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140350/225000 [279:28:01<119:02:24,  5.06s/it]
 62%|██████▏   | 140351/225000 [279:28:06<119:14:07,  5.07s/it]
 62%|██████▏   | 140352/225000 [279:28:11<119:43:02,  5.09s/it]
 62%|██████▏   | 140353/225000 [279:28:16<121:27:36,  5.17s/it]
 62%|██████▏   | 140354/225000 [279:28:22<122:20:06,  5.20s/it]
 62%|██████▏   | 140355/225000 [279:28:26<116:26:14,  4.95s/it]
 62%|██████▏   | 140356/225000 [279:28:37<156:13:56,  6.64s/it]
 62%|██████▏   | 140357/225000 [279:28:45<170:43:03,  7.26s/it]
 62%|██████▏   | 140358/225000 [279:28:49<144:44:29,  6.16s/it]
 62%|██████▏   | 140359/225000 [279:28:58<165:47:33,  7.05s/it]
 62%|██████▏   | 140360/225000 [279:29:02<143:27:57,  6.10s/it]
 62%|██████▏   | 140361/225000 [279:29:06<129:11:29,  5.49s/it]
 62%|██████▏   | 140362/225000 [279:29:11<125:08:17,  5.32s/it]
 62%|██████▏   | 140363/225000 [279:29:22<163:35:09,  6.96s/it]
 62%|██████▏   | 140364/225000 [279:29:25<135:48:11,  5.78s/it]
 62%|██████▏   | 140365/225000 [279:29:30<132:20:11,  5.63s/it]
 62%|██████▏   | 140366/225000 [279:29:39<153:09:58,  6.52s/it]
 62%|██████▏   | 140367/225000 [279:29:45<153:42:09,  6.54s/it]
 62%|██████▏   | 140368/225000 [279:29:49<134:33:22,  5.72s/it]
 62%|██████▏   | 140369/225000 [279:29:53<122:21:03,  5.20s/it]
 62%|██████▏   | 140370/225000 [279:29:58<118:49:21,  5.05s/it]
 62%|██████▏   | 140371/225000 [279:30:02<116:22:04,  4.95s/it]
 62%|██████▏   | 140372/225000 [279:30:09<128:43:11,  5.48s/it]
 62%|██████▏   | 140373/225000 [279:30:22<179:00:55,  7.62s/it]
 62%|██████▏   | 140374/225000 [279:30:27<160:31:35,  6.83s/it]
 62%|██████▏   | 140375/225000 [279:30:31<145:59:31,  6.21s/it]
 62%|██████▏   | 140376/225000 [279:30:36<134:22:54,  5.72s/it]
 62%|██████▏   | 140377/225000 [279:30:40<124:42:41,  5.31s/it]
 62%|██████▏   | 140378/225000 [279:30:45<119:13:13,  5.07s/it]
 62%|██████▏   | 140379/225000 [279:30:50<119:13:33,  5.07s/it]
 62%|██████▏   | 140380/225000 [279:31:02<171:32:03,  7.30s/it]
 62%|██████▏   | 140381/225000 [279:31:06<143:31:33,  6.11s/it]
 62%|██████▏   | 140382/225000 [279:31:09<126:03:23,  5.36s/it]
 62%|██████▏   | 140383/225000 [279:31:19<157:58:14,  6.72s/it]
 62%|██████▏   | 140384/225000 [279:31:22<129:44:52,  5.52s/it]
 62%|██████▏   | 140385/225000 [279:31:26<118:58:53,  5.06s/it]
 62%|██████▏   | 140386/225000 [279:31:31<121:52:25,  5.19s/it]
 62%|██████▏   | 140387/225000 [279:31:37<123:01:08,  5.23s/it]
 62%|██████▏   | 140388/225000 [279:31:42<123:54:56,  5.27s/it]
 62%|██████▏   | 140389/225000 [279:31:48<125:05:57,  5.32s/it]
 62%|██████▏   | 140390/225000 [279:31:53<127:20:32,  5.42s/it]
 62%|██████▏   | 140391/225000 [279:31:58<126:04:43,  5.36s/it]
 62%|██████▏   | 140392/225000 [279:32:04<125:18:19,  5.33s/it]
 62%|██████▏   | 140393/225000 [279:32:16<177:21:22,  7.55s/it]
 62%|██████▏   | 140394/225000 [279:32:21<158:09:10,  6.73s/it]
 62%|██████▏   | 140395/225000 [279:32:25<136:14:23,  5.80s/it]
 62%|██████▏   | 140396/225000 [279:32:30<129:14:39,  5.50s/it]
 62%|██████▏   | 140397/225000 [279:32:35<126:18:39,  5.37s/it]
 62%|██████▏   | 140398/225000 [279:32:39<120:16:17,  5.12s/it]
 62%|██████▏   | 140399/225000 [279:32:48<149:04:48,  6.34s/it]
 62%|██████▏   | 140400/225000 [279:32:58<168:43:44,  7.18s/it]
                                                               
{'loss': '0.7893', 'grad_norm': '1.03', 'learning_rate': '1.681e-05', 'epoch': '0.624', 'num_input_tokens_seen': 463547512, 'train_runtime': '1.006e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140400/225000 [279:32:58<168:43:44,  7.18s/it]
 62%|██████▏   | 140401/225000 [279:33:01<143:11:01,  6.09s/it]
 62%|██████▏   | 140402/225000 [279:33:07<141:10:00,  6.01s/it]
 62%|██████▏   | 140403/225000 [279:33:14<150:07:34,  6.39s/it]
 62%|██████▏   | 140404/225000 [279:33:18<128:13:01,  5.46s/it]
 62%|██████▏   | 140405/225000 [279:33:26<146:21:36,  6.23s/it]
 62%|██████▏   | 140406/225000 [279:33:32<150:52:30,  6.42s/it]
 62%|██████▏   | 140407/225000 [279:33:36<132:34:20,  5.64s/it]
 62%|██████▏   | 140408/225000 [279:33:40<121:16:23,  5.16s/it]
 62%|██████▏   | 140409/225000 [279:33:50<152:11:18,  6.48s/it]
 62%|██████▏   | 140410/225000 [279:33:53<129:41:06,  5.52s/it]
 62%|██████▏   | 140411/225000 [279:33:57<117:38:23,  5.01s/it]
 62%|██████▏   | 140412/225000 [279:34:06<147:24:43,  6.27s/it]
 62%|██████▏   | 140413/225000 [279:34:13<148:42:08,  6.33s/it]
 62%|██████▏   | 140414/225000 [279:34:17<133:10:35,  5.67s/it]
 62%|██████▏   | 140415/225000 [279:34:26<159:45:08,  6.80s/it]
 62%|██████▏   | 140416/225000 [279:34:29<134:49:58,  5.74s/it]
 62%|██████▏   | 140417/225000 [279:34:34<127:38:22,  5.43s/it]
 62%|██████▏   | 140418/225000 [279:34:40<128:32:30,  5.47s/it]
 62%|██████▏   | 140419/225000 [279:34:45<127:04:29,  5.41s/it]
 62%|██████▏   | 140420/225000 [279:34:54<151:37:01,  6.45s/it]
 62%|██████▏   | 140421/225000 [279:34:57<130:48:02,  5.57s/it]
 62%|██████▏   | 140422/225000 [279:35:06<152:14:10,  6.48s/it]
 62%|██████▏   | 140423/225000 [279:35:10<134:46:07,  5.74s/it]
 62%|██████▏   | 140424/225000 [279:35:14<121:59:54,  5.19s/it]
 62%|██████▏   | 140425/225000 [279:35:24<158:58:23,  6.77s/it]
 62%|██████▏   | 140426/225000 [279:35:28<133:25:10,  5.68s/it]
 62%|██████▏   | 140427/225000 [279:35:35<147:14:33,  6.27s/it]
 62%|██████▏   | 140428/225000 [279:35:39<132:44:21,  5.65s/it]
 62%|██████▏   | 140429/225000 [279:35:43<120:19:15,  5.12s/it]
 62%|██████▏   | 140430/225000 [279:35:47<114:03:09,  4.86s/it]
 62%|██████▏   | 140431/225000 [279:35:58<152:52:24,  6.51s/it]
 62%|██████▏   | 140432/225000 [279:36:03<140:58:08,  6.00s/it]
 62%|██████▏   | 140433/225000 [279:36:07<130:58:16,  5.58s/it]
 62%|██████▏   | 140434/225000 [279:36:12<128:07:05,  5.45s/it]
 62%|██████▏   | 140435/225000 [279:36:17<124:10:44,  5.29s/it]
 62%|██████▏   | 140436/225000 [279:36:22<117:55:34,  5.02s/it]
 62%|██████▏   | 140437/225000 [279:36:26<112:46:42,  4.80s/it]
 62%|██████▏   | 140438/225000 [279:36:31<111:27:30,  4.75s/it]
 62%|██████▏   | 140439/225000 [279:36:39<137:34:20,  5.86s/it]
 62%|██████▏   | 140440/225000 [279:36:48<158:02:27,  6.73s/it]
 62%|██████▏   | 140441/225000 [279:36:52<143:32:39,  6.11s/it]
 62%|██████▏   | 140442/225000 [279:36:58<138:26:40,  5.89s/it]
 62%|██████▏   | 140443/225000 [279:37:03<131:07:06,  5.58s/it]
 62%|██████▏   | 140444/225000 [279:37:08<128:48:19,  5.48s/it]
 62%|██████▏   | 140445/225000 [279:37:13<125:56:12,  5.36s/it]
 62%|██████▏   | 140446/225000 [279:37:18<124:50:10,  5.32s/it]
 62%|██████▏   | 140447/225000 [279:37:24<128:19:44,  5.46s/it]
 62%|██████▏   | 140448/225000 [279:37:29<124:39:58,  5.31s/it]
 62%|██████▏   | 140449/225000 [279:37:34<123:31:54,  5.26s/it]
 62%|██████▏   | 140450/225000 [279:37:40<124:24:17,  5.30s/it]
                                                               
{'loss': '0.7828', 'grad_norm': '1.048', 'learning_rate': '1.679e-05', 'epoch': '0.6242', 'num_input_tokens_seen': 463677984, 'train_runtime': '1.007e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140450/225000 [279:37:40<124:24:17,  5.30s/it]
 62%|██████▏   | 140451/225000 [279:37:45<122:39:29,  5.22s/it]
 62%|██████▏   | 140452/225000 [279:37:50<124:33:25,  5.30s/it]
 62%|██████▏   | 140453/225000 [279:37:58<145:08:07,  6.18s/it]
 62%|██████▏   | 140454/225000 [279:38:11<193:39:44,  8.25s/it]
 62%|██████▏   | 140455/225000 [279:38:15<163:29:55,  6.96s/it]
 62%|██████▏   | 140456/225000 [279:38:20<144:31:46,  6.15s/it]
 62%|██████▏   | 140457/225000 [279:38:28<157:56:14,  6.73s/it]
 62%|██████▏   | 140458/225000 [279:38:31<133:45:15,  5.70s/it]
 62%|██████▏   | 140459/225000 [279:38:37<135:44:36,  5.78s/it]
 62%|██████▏   | 140460/225000 [279:38:43<140:54:31,  6.00s/it]
 62%|██████▏   | 140461/225000 [279:38:49<138:48:43,  5.91s/it]
 62%|██████▏   | 140462/225000 [279:38:57<149:14:22,  6.36s/it]
 62%|██████▏   | 140463/225000 [279:39:01<135:20:52,  5.76s/it]
 62%|██████▏   | 140464/225000 [279:39:05<123:25:24,  5.26s/it]
 62%|██████▏   | 140465/225000 [279:39:09<117:31:31,  5.00s/it]
 62%|██████▏   | 140466/225000 [279:39:14<114:39:47,  4.88s/it]
 62%|██████▏   | 140467/225000 [279:39:25<159:15:16,  6.78s/it]
 62%|██████▏   | 140468/225000 [279:39:30<141:36:16,  6.03s/it]
 62%|██████▏   | 140469/225000 [279:39:35<138:52:34,  5.91s/it]
 62%|██████▏   | 140470/225000 [279:39:41<141:17:01,  6.02s/it]
 62%|██████▏   | 140471/225000 [279:39:46<129:47:24,  5.53s/it]
 62%|██████▏   | 140472/225000 [279:39:50<123:15:29,  5.25s/it]
 62%|██████▏   | 140473/225000 [279:39:55<118:05:38,  5.03s/it]
 62%|██████▏   | 140474/225000 [279:39:59<112:42:39,  4.80s/it]
 62%|██████▏   | 140475/225000 [279:40:05<117:14:49,  4.99s/it]
 62%|██████▏   | 140476/225000 [279:40:17<172:16:19,  7.34s/it]
 62%|██████▏   | 140477/225000 [279:40:21<147:00:30,  6.26s/it]
 62%|██████▏   | 140478/225000 [279:40:25<129:58:59,  5.54s/it]
 62%|██████▏   | 140479/225000 [279:40:29<118:38:04,  5.05s/it]
 62%|██████▏   | 140480/225000 [279:40:41<165:33:09,  7.05s/it]
 62%|██████▏   | 140481/225000 [279:40:45<143:10:11,  6.10s/it]
 62%|██████▏   | 140482/225000 [279:40:48<127:12:47,  5.42s/it]
 62%|██████▏   | 140483/225000 [279:40:52<114:37:49,  4.88s/it]
 62%|██████▏   | 140484/225000 [279:41:02<148:00:59,  6.30s/it]
 62%|██████▏   | 140485/225000 [279:41:07<140:01:44,  5.96s/it]
 62%|██████▏   | 140486/225000 [279:41:11<127:36:26,  5.44s/it]
 62%|██████▏   | 140487/225000 [279:41:17<128:08:17,  5.46s/it]
 62%|██████▏   | 140488/225000 [279:41:27<160:34:55,  6.84s/it]
 62%|██████▏   | 140489/225000 [279:41:31<144:27:31,  6.15s/it]
 62%|██████▏   | 140490/225000 [279:41:36<137:32:53,  5.86s/it]
 62%|██████▏   | 140491/225000 [279:41:44<147:45:07,  6.29s/it]
 62%|██████▏   | 140492/225000 [279:41:47<125:20:31,  5.34s/it]
 62%|██████▏   | 140493/225000 [279:41:50<113:33:18,  4.84s/it]
 62%|██████▏   | 140494/225000 [279:41:55<110:09:27,  4.69s/it]
 62%|██████▏   | 140495/225000 [279:42:05<152:07:35,  6.48s/it]
 62%|██████▏   | 140496/225000 [279:42:10<138:58:20,  5.92s/it]
 62%|██████▏   | 140497/225000 [279:42:15<129:50:06,  5.53s/it]
 62%|██████▏   | 140498/225000 [279:42:22<143:23:19,  6.11s/it]
 62%|██████▏   | 140499/225000 [279:42:27<134:14:37,  5.72s/it]
 62%|██████▏   | 140500/225000 [279:42:35<148:52:56,  6.34s/it]
                                                               
{'loss': '0.8021', 'grad_norm': '0.9555', 'learning_rate': '1.678e-05', 'epoch': '0.6244', 'num_input_tokens_seen': 463808184, 'train_runtime': '1.007e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140500/225000 [279:42:35<148:52:56,  6.34s/it]
 62%|██████▏   | 140501/225000 [279:42:39<137:12:00,  5.85s/it]
 62%|██████▏   | 140502/225000 [279:42:45<133:36:50,  5.69s/it]
 62%|██████▏   | 140503/225000 [279:42:50<131:06:26,  5.59s/it]
 62%|██████▏   | 140504/225000 [279:42:56<130:17:42,  5.55s/it]
 62%|██████▏   | 140505/225000 [279:43:01<129:03:05,  5.50s/it]
 62%|██████▏   | 140506/225000 [279:43:11<158:18:30,  6.74s/it]
 62%|██████▏   | 140507/225000 [279:43:15<139:57:00,  5.96s/it]
 62%|██████▏   | 140508/225000 [279:43:19<129:18:40,  5.51s/it]
 62%|██████▏   | 140509/225000 [279:43:29<161:07:45,  6.87s/it]
 62%|██████▏   | 140510/225000 [279:43:33<137:15:56,  5.85s/it]
 62%|██████▏   | 140511/225000 [279:43:37<126:30:59,  5.39s/it]
 62%|██████▏   | 140512/225000 [279:43:42<123:21:10,  5.26s/it]
 62%|██████▏   | 140513/225000 [279:43:46<117:41:54,  5.02s/it]
 62%|██████▏   | 140514/225000 [279:43:58<166:56:43,  7.11s/it]
 62%|██████▏   | 140515/225000 [279:44:02<144:21:01,  6.15s/it]
 62%|██████▏   | 140516/225000 [279:44:06<128:02:01,  5.46s/it]
 62%|██████▏   | 140517/225000 [279:44:12<130:52:17,  5.58s/it]
 62%|██████▏   | 140518/225000 [279:44:21<153:23:35,  6.54s/it]
 62%|██████▏   | 140519/225000 [279:44:24<133:36:06,  5.69s/it]
 62%|██████▏   | 140520/225000 [279:44:29<123:02:56,  5.24s/it]
 62%|██████▏   | 140521/225000 [279:44:36<136:55:04,  5.83s/it]
 62%|██████▏   | 140522/225000 [279:44:44<150:04:13,  6.40s/it]
 62%|██████▏   | 140523/225000 [279:44:48<136:56:04,  5.84s/it]
 62%|██████▏   | 140524/225000 [279:44:53<126:50:26,  5.41s/it]
 62%|██████▏   | 140525/225000 [279:44:57<121:54:06,  5.19s/it]
 62%|██████▏   | 140526/225000 [279:45:02<117:38:48,  5.01s/it]
 62%|██████▏   | 140527/225000 [279:45:07<116:48:00,  4.98s/it]
 62%|██████▏   | 140528/225000 [279:45:11<114:55:54,  4.90s/it]
 62%|██████▏   | 140529/225000 [279:45:18<126:54:32,  5.41s/it]
 62%|██████▏   | 140530/225000 [279:45:29<164:09:24,  7.00s/it]
 62%|██████▏   | 140531/225000 [279:45:33<148:09:15,  6.31s/it]
 62%|██████▏   | 140532/225000 [279:45:38<134:45:09,  5.74s/it]
 62%|██████▏   | 140533/225000 [279:45:42<126:25:12,  5.39s/it]
 62%|██████▏   | 140534/225000 [279:45:47<119:49:37,  5.11s/it]
 62%|██████▏   | 140535/225000 [279:45:51<114:56:33,  4.90s/it]
 62%|██████▏   | 140536/225000 [279:46:04<166:32:13,  7.10s/it]
 62%|██████▏   | 140537/225000 [279:46:09<151:41:59,  6.47s/it]
 62%|██████▏   | 140538/225000 [279:46:13<140:23:04,  5.98s/it]
 62%|██████▏   | 140539/225000 [279:46:18<132:02:53,  5.63s/it]
 62%|██████▏   | 140540/225000 [279:46:23<125:30:28,  5.35s/it]
 62%|██████▏   | 140541/225000 [279:46:28<122:29:59,  5.22s/it]
 62%|██████▏   | 140542/225000 [279:46:33<120:30:18,  5.14s/it]
 62%|██████▏   | 140543/225000 [279:46:37<117:10:19,  4.99s/it]
 62%|██████▏   | 140544/225000 [279:46:50<169:08:00,  7.21s/it]
 62%|██████▏   | 140545/225000 [279:46:55<155:35:57,  6.63s/it]
 62%|██████▏   | 140546/225000 [279:47:00<146:12:46,  6.23s/it]
 62%|██████▏   | 140547/225000 [279:47:05<135:16:15,  5.77s/it]
 62%|██████▏   | 140548/225000 [279:47:10<127:45:09,  5.45s/it]
 62%|██████▏   | 140549/225000 [279:47:15<124:48:09,  5.32s/it]
 62%|██████▏   | 140550/225000 [279:47:20<123:56:42,  5.28s/it]
                                                               
{'loss': '0.7958', 'grad_norm': '0.9817', 'learning_rate': '1.676e-05', 'epoch': '0.6247', 'num_input_tokens_seen': 463938368, 'train_runtime': '1.007e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140550/225000 [279:47:20<123:56:42,  5.28s/it]
 62%|██████▏   | 140551/225000 [279:47:25<124:00:19,  5.29s/it]
 62%|██████▏   | 140552/225000 [279:47:31<123:55:05,  5.28s/it]
 62%|██████▏   | 140553/225000 [279:47:36<122:46:20,  5.23s/it]
 62%|██████▏   | 140554/225000 [279:47:41<122:05:45,  5.21s/it]
 62%|██████▏   | 140555/225000 [279:47:46<120:47:28,  5.15s/it]
 62%|██████▏   | 140556/225000 [279:47:51<120:50:06,  5.15s/it]
 62%|██████▏   | 140557/225000 [279:47:56<120:43:24,  5.15s/it]
 62%|██████▏   | 140558/225000 [279:48:01<121:12:07,  5.17s/it]
 62%|██████▏   | 140559/225000 [279:48:12<161:58:05,  6.91s/it]
 62%|██████▏   | 140560/225000 [279:48:22<178:53:48,  7.63s/it]
 62%|██████▏   | 140561/225000 [279:48:26<152:49:42,  6.52s/it]
 62%|██████▏   | 140562/225000 [279:48:33<162:37:57,  6.93s/it]
 62%|██████▏   | 140563/225000 [279:48:39<155:35:40,  6.63s/it]
 62%|██████▏   | 140564/225000 [279:48:43<133:05:42,  5.67s/it]
 62%|██████▏   | 140565/225000 [279:48:50<145:54:11,  6.22s/it]
 62%|██████▏   | 140566/225000 [279:48:55<135:56:57,  5.80s/it]
 62%|██████▏   | 140567/225000 [279:49:00<128:17:16,  5.47s/it]
 62%|██████▏   | 140568/225000 [279:49:05<123:11:08,  5.25s/it]
 62%|██████▏   | 140569/225000 [279:49:09<117:56:37,  5.03s/it]
 62%|██████▏   | 140570/225000 [279:49:16<134:36:42,  5.74s/it]
 62%|██████▏   | 140571/225000 [279:49:27<171:57:16,  7.33s/it]
 62%|██████▏   | 140572/225000 [279:49:32<149:30:08,  6.37s/it]
 62%|██████▏   | 140573/225000 [279:49:36<135:25:06,  5.77s/it]
 62%|██████▏   | 140574/225000 [279:49:40<123:16:25,  5.26s/it]
 62%|██████▏   | 140575/225000 [279:49:45<118:11:01,  5.04s/it]
 62%|██████▏   | 140576/225000 [279:49:49<116:16:34,  4.96s/it]
 62%|██████▏   | 140577/225000 [279:49:58<145:18:51,  6.20s/it]
 62%|██████▏   | 140578/225000 [279:50:06<154:14:48,  6.58s/it]
 62%|██████▏   | 140579/225000 [279:50:10<136:10:49,  5.81s/it]
 62%|██████▏   | 140580/225000 [279:50:14<123:05:40,  5.25s/it]
 62%|██████▏   | 140581/225000 [279:50:18<115:49:41,  4.94s/it]
 62%|██████▏   | 140582/225000 [279:50:27<141:02:42,  6.01s/it]
 62%|██████▏   | 140583/225000 [279:50:34<153:17:43,  6.54s/it]
 62%|██████▏   | 140584/225000 [279:50:38<132:44:03,  5.66s/it]
 62%|██████▏   | 140585/225000 [279:50:42<120:18:02,  5.13s/it]
 62%|██████▏   | 140586/225000 [279:50:52<156:51:00,  6.69s/it]
 62%|██████▏   | 140587/225000 [279:50:56<138:19:39,  5.90s/it]
 62%|██████▏   | 140588/225000 [279:51:01<127:07:48,  5.42s/it]
 62%|██████▏   | 140589/225000 [279:51:05<117:40:23,  5.02s/it]
 62%|██████▏   | 140590/225000 [279:51:14<149:18:29,  6.37s/it]
 62%|██████▏   | 140591/225000 [279:51:21<152:05:06,  6.49s/it]
 62%|██████▏   | 140592/225000 [279:51:24<130:08:54,  5.55s/it]
 62%|██████▏   | 140593/225000 [279:51:33<150:20:08,  6.41s/it]
 62%|██████▏   | 140594/225000 [279:51:36<127:20:25,  5.43s/it]
 62%|██████▏   | 140595/225000 [279:51:43<140:51:49,  6.01s/it]
 62%|██████▏   | 140596/225000 [279:51:50<149:09:22,  6.36s/it]
 62%|██████▏   | 140597/225000 [279:51:54<129:50:15,  5.54s/it]
 62%|██████▏   | 140598/225000 [279:52:01<142:17:13,  6.07s/it]
 62%|██████▏   | 140599/225000 [279:52:06<130:02:29,  5.55s/it]
 62%|██████▏   | 140600/225000 [279:52:11<125:42:55,  5.36s/it]
                                                               
{'loss': '0.7913', 'grad_norm': '1.158', 'learning_rate': '1.674e-05', 'epoch': '0.6249', 'num_input_tokens_seen': 464068352, 'train_runtime': '1.008e+06', 'train_tokens_per_second': '460.6'}

 62%|██████▏   | 140600/225000 [279:52:11<125:42:55,  5.36s/it]
 62%|██████▏   | 140601/225000 [279:52:15<121:58:53,  5.20s/it]
 62%|██████▏   | 140602/225000 [279:52:20<119:03:46,  5.08s/it]
 62%|██████▏   | 140603/225000 [279:52:25<114:44:39,  4.89s/it]
 62%|██████▏   | 140604/225000 [279:52:29<114:10:28,  4.87s/it]
 62%|██████▏   | 140605/225000 [279:52:34<115:13:12,  4.91s/it]
 62%|██████▏   | 140606/225000 [279:52:41<127:48:08,  5.45s/it]
 62%|██████▏   | 140607/225000 [279:52:53<174:02:01,  7.42s/it]
 62%|██████▏   | 140608/225000 [279:52:56<144:38:11,  6.17s/it]
 62%|██████▏   | 140609/225000 [279:53:01<135:37:47,  5.79s/it]
 62%|██████▏   | 140610/225000 [279:53:06<129:21:50,  5.52s/it]
 62%|██████▏   | 140611/225000 [279:53:11<122:20:21,  5.22s/it]
 62%|██████▏   | 140612/225000 [279:53:15<116:34:36,  4.97s/it]
 62%|██████▏   | 140613/225000 [279:53:20<114:34:22,  4.89s/it]
 62%|██████▏   | 140614/225000 [279:53:28<140:51:54,  6.01s/it]
 62%|██████▏   | 140615/225000 [279:53:35<147:14:36,  6.28s/it]
 62%|██████▏   | 140616/225000 [279:53:40<132:51:29,  5.67s/it]
 62%|██████▏   | 140617/225000 [279:53:49<155:36:03,  6.64s/it]
 62%|██████▏   | 140618/225000 [279:53:52<132:56:09,  5.67s/it]
 62%|██████▏   | 140619/225000 [279:53:56<121:32:09,  5.19s/it]
 62%|██████▏   | 140620/225000 [279:54:07<160:28:08,  6.85s/it]
 62%|██████▏   | 140621/225000 [279:54:10<137:11:36,  5.85s/it]
 62%|██████▏   | 140622/225000 [279:54:14<122:20:35,  5.22s/it]
 62%|██████▏   | 140623/225000 [279:54:19<121:25:55,  5.18s/it]
 62%|██████▏   | 140624/225000 [279:54:25<123:53:32,  5.29s/it]
 62%|██████▎   | 140625/225000 [279:54:31<128:13:39,  5.47s/it]
 63%|██████▎   | 140626/225000 [279:54:36<127:47:26,  5.45s/it]
 63%|██████▎   | 140627/225000 [279:54:41<127:33:00,  5.44s/it]
 63%|██████▎   | 140628/225000 [279:54:47<127:13:53,  5.43s/it]
 63%|██████▎   | 140629/225000 [279:54:52<127:48:43,  5.45s/it]
 63%|██████▎   | 140630/225000 [279:54:58<126:50:37,  5.41s/it]
 63%|██████▎   | 140631/225000 [279:55:03<126:13:51,  5.39s/it]
 63%|██████▎   | 140632/225000 [279:55:09<127:49:27,  5.45s/it]
 63%|██████▎   | 140633/225000 [279:55:14<128:51:50,  5.50s/it]
 63%|██████▎   | 140634/225000 [279:55:22<147:01:52,  6.27s/it]
 63%|██████▎   | 140635/225000 [279:55:32<174:13:06,  7.43s/it]
 63%|██████▎   | 140636/225000 [279:55:36<148:33:00,  6.34s/it]
 63%|██████▎   | 140637/225000 [279:55:45<168:08:28,  7.18s/it]
 63%|██████▎   | 140638/225000 [279:55:48<140:24:08,  5.99s/it]
 63%|██████▎   | 140639/225000 [279:55:54<135:46:48,  5.79s/it]
 63%|██████▎   | 140640/225000 [279:56:00<137:39:51,  5.87s/it]
 63%|██████▎   | 140641/225000 [279:56:03<121:36:41,  5.19s/it]
 63%|██████▎   | 140642/225000 [279:56:08<117:28:28,  5.01s/it]
 63%|██████▎   | 140643/225000 [279:56:13<115:42:56,  4.94s/it]
 63%|██████▎   | 140644/225000 [279:56:18<115:30:19,  4.93s/it]
 63%|██████▎   | 140645/225000 [279:56:22<113:10:37,  4.83s/it]
 63%|██████▎   | 140646/225000 [279:56:30<134:54:13,  5.76s/it]
 63%|██████▎   | 140647/225000 [279:56:41<170:48:36,  7.29s/it]
 63%|██████▎   | 140648/225000 [279:56:45<143:54:38,  6.14s/it]
 63%|██████▎   | 140649/225000 [279:56:50<137:26:16,  5.87s/it]
 63%|██████▎   | 140650/225000 [279:56:58<152:10:21,  6.49s/it]
                                                               
{'loss': '0.7977', 'grad_norm': '1.043', 'learning_rate': '1.673e-05', 'epoch': '0.6251', 'num_input_tokens_seen': 464197784, 'train_runtime': '1.008e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140650/225000 [279:56:58<152:10:21,  6.49s/it]
 63%|██████▎   | 140651/225000 [279:57:02<137:00:54,  5.85s/it]
 63%|██████▎   | 140652/225000 [279:57:06<126:28:52,  5.40s/it]
 63%|██████▎   | 140653/225000 [279:57:11<121:47:07,  5.20s/it]
 63%|██████▎   | 140654/225000 [279:57:16<115:56:47,  4.95s/it]
 63%|██████▎   | 140655/225000 [279:57:27<159:03:56,  6.79s/it]
 63%|██████▎   | 140656/225000 [279:57:32<151:54:12,  6.48s/it]
 63%|██████▎   | 140657/225000 [279:57:36<129:29:19,  5.53s/it]
 63%|██████▎   | 140658/225000 [279:57:44<147:53:51,  6.31s/it]
 63%|██████▎   | 140659/225000 [279:57:47<125:29:22,  5.36s/it]
 63%|██████▎   | 140660/225000 [279:57:51<117:23:18,  5.01s/it]
 63%|██████▎   | 140661/225000 [279:57:59<135:05:41,  5.77s/it]
 63%|██████▎   | 140662/225000 [279:58:03<123:25:57,  5.27s/it]
 63%|██████▎   | 140663/225000 [279:58:08<120:44:13,  5.15s/it]
 63%|██████▎   | 140664/225000 [279:58:13<120:17:36,  5.13s/it]
 63%|██████▎   | 140665/225000 [279:58:17<115:53:46,  4.95s/it]
 63%|██████▎   | 140666/225000 [279:58:23<118:10:26,  5.04s/it]
 63%|██████▎   | 140667/225000 [279:58:27<115:58:11,  4.95s/it]
 63%|██████▎   | 140668/225000 [279:58:32<114:59:41,  4.91s/it]
 63%|██████▎   | 140669/225000 [279:58:38<120:01:12,  5.12s/it]
 63%|██████▎   | 140670/225000 [279:58:43<120:02:46,  5.12s/it]
 63%|██████▎   | 140671/225000 [279:58:48<120:06:55,  5.13s/it]
 63%|██████▎   | 140672/225000 [279:58:56<142:20:03,  6.08s/it]
 63%|██████▎   | 140673/225000 [279:59:08<180:05:42,  7.69s/it]
 63%|██████▎   | 140674/225000 [279:59:11<150:14:16,  6.41s/it]
 63%|██████▎   | 140675/225000 [279:59:15<134:32:28,  5.74s/it]
 63%|██████▎   | 140676/225000 [279:59:21<130:44:48,  5.58s/it]
 63%|██████▎   | 140677/225000 [279:59:26<128:20:59,  5.48s/it]
 63%|██████▎   | 140678/225000 [279:59:31<127:00:56,  5.42s/it]
 63%|██████▎   | 140679/225000 [279:59:37<129:23:10,  5.52s/it]
 63%|██████▎   | 140680/225000 [279:59:42<129:57:02,  5.55s/it]
 63%|██████▎   | 140681/225000 [279:59:47<125:29:46,  5.36s/it]
 63%|██████▎   | 140682/225000 [279:59:53<125:25:42,  5.36s/it]
 63%|██████▎   | 140683/225000 [279:59:58<126:14:34,  5.39s/it]
 63%|██████▎   | 140684/225000 [280:00:04<127:22:30,  5.44s/it]
 63%|██████▎   | 140685/225000 [280:00:09<128:25:28,  5.48s/it]
 63%|██████▎   | 140686/225000 [280:00:15<126:20:20,  5.39s/it]
 63%|██████▎   | 140687/225000 [280:00:20<126:37:22,  5.41s/it]
 63%|██████▎   | 140688/225000 [280:00:32<170:14:40,  7.27s/it]
 63%|██████▎   | 140689/225000 [280:00:38<165:49:21,  7.08s/it]
 63%|██████▎   | 140690/225000 [280:00:46<170:46:35,  7.29s/it]
 63%|██████▎   | 140691/225000 [280:00:49<143:44:10,  6.14s/it]
 63%|██████▎   | 140692/225000 [280:00:55<142:40:37,  6.09s/it]
 63%|██████▎   | 140693/225000 [280:01:03<153:37:07,  6.56s/it]
 63%|██████▎   | 140694/225000 [280:01:06<131:03:32,  5.60s/it]
 63%|██████▎   | 140695/225000 [280:01:14<147:13:22,  6.29s/it]
 63%|██████▎   | 140696/225000 [280:01:18<130:37:03,  5.58s/it]
 63%|██████▎   | 140697/225000 [280:01:22<119:44:43,  5.11s/it]
 63%|██████▎   | 140698/225000 [280:01:27<117:05:25,  5.00s/it]
 63%|██████▎   | 140699/225000 [280:01:34<134:06:41,  5.73s/it]
 63%|██████▎   | 140700/225000 [280:01:43<151:26:29,  6.47s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '0.9895', 'learning_rate': '1.671e-05', 'epoch': '0.6253', 'num_input_tokens_seen': 464326896, 'train_runtime': '1.008e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140700/225000 [280:01:43<151:26:29,  6.47s/it]
 63%|██████▎   | 140701/225000 [280:01:49<148:34:38,  6.35s/it]
 63%|██████▎   | 140702/225000 [280:01:52<128:54:03,  5.50s/it]
 63%|██████▎   | 140703/225000 [280:02:01<152:48:53,  6.53s/it]
 63%|██████▎   | 140704/225000 [280:02:06<140:30:25,  6.00s/it]
 63%|██████▎   | 140705/225000 [280:02:10<129:10:16,  5.52s/it]
 63%|██████▎   | 140706/225000 [280:02:15<123:29:52,  5.27s/it]
 63%|██████▎   | 140707/225000 [280:02:19<116:55:12,  4.99s/it]
 63%|██████▎   | 140708/225000 [280:02:24<116:19:54,  4.97s/it]
 63%|██████▎   | 140709/225000 [280:02:29<112:25:26,  4.80s/it]
 63%|██████▎   | 140710/225000 [280:02:33<107:18:11,  4.58s/it]
 63%|██████▎   | 140711/225000 [280:02:45<159:01:40,  6.79s/it]
 63%|██████▎   | 140712/225000 [280:02:51<158:33:31,  6.77s/it]
 63%|██████▎   | 140713/225000 [280:02:56<146:12:53,  6.25s/it]
 63%|██████▎   | 140714/225000 [280:03:01<137:22:56,  5.87s/it]
 63%|██████▎   | 140715/225000 [280:03:07<132:48:09,  5.67s/it]
 63%|██████▎   | 140716/225000 [280:03:12<128:04:50,  5.47s/it]
 63%|██████▎   | 140717/225000 [280:03:17<124:09:14,  5.30s/it]
 63%|██████▎   | 140718/225000 [280:03:21<121:11:23,  5.18s/it]
 63%|██████▎   | 140719/225000 [280:03:26<117:54:37,  5.04s/it]
 63%|██████▎   | 140720/225000 [280:03:31<117:18:38,  5.01s/it]
 63%|██████▎   | 140721/225000 [280:03:43<162:22:45,  6.94s/it]
 63%|██████▎   | 140722/225000 [280:03:47<147:01:20,  6.28s/it]
 63%|██████▎   | 140723/225000 [280:03:54<150:42:46,  6.44s/it]
 63%|██████▎   | 140724/225000 [280:03:58<130:02:45,  5.56s/it]
 63%|██████▎   | 140725/225000 [280:04:02<125:09:17,  5.35s/it]
 63%|██████▎   | 140726/225000 [280:04:08<124:19:35,  5.31s/it]
 63%|██████▎   | 140727/225000 [280:04:13<122:41:26,  5.24s/it]
 63%|██████▎   | 140728/225000 [280:04:18<119:37:12,  5.11s/it]
 63%|██████▎   | 140729/225000 [280:04:22<117:32:44,  5.02s/it]
 63%|██████▎   | 140730/225000 [280:04:28<119:23:40,  5.10s/it]
 63%|██████▎   | 140731/225000 [280:04:33<118:22:48,  5.06s/it]
 63%|██████▎   | 140732/225000 [280:04:38<119:11:14,  5.09s/it]
 63%|██████▎   | 140733/225000 [280:04:43<121:00:47,  5.17s/it]
 63%|██████▎   | 140734/225000 [280:04:48<119:42:30,  5.11s/it]
 63%|██████▎   | 140735/225000 [280:04:53<120:20:44,  5.14s/it]
 63%|██████▎   | 140736/225000 [280:04:59<122:29:53,  5.23s/it]
 63%|██████▎   | 140737/225000 [280:05:04<124:55:24,  5.34s/it]
 63%|██████▎   | 140738/225000 [280:05:16<168:28:28,  7.20s/it]
 63%|██████▎   | 140739/225000 [280:05:25<183:52:56,  7.86s/it]
 63%|██████▎   | 140740/225000 [280:05:29<151:57:15,  6.49s/it]
 63%|██████▎   | 140741/225000 [280:05:35<150:16:36,  6.42s/it]
 63%|██████▎   | 140742/225000 [280:05:40<143:40:34,  6.14s/it]
 63%|██████▎   | 140743/225000 [280:05:45<134:35:20,  5.75s/it]
 63%|██████▎   | 140744/225000 [280:05:54<154:15:41,  6.59s/it]
 63%|██████▎   | 140745/225000 [280:05:58<138:26:35,  5.92s/it]
 63%|██████▎   | 140746/225000 [280:06:02<127:04:37,  5.43s/it]
 63%|██████▎   | 140747/225000 [280:06:13<161:21:37,  6.89s/it]
 63%|██████▎   | 140748/225000 [280:06:17<142:17:32,  6.08s/it]
 63%|██████▎   | 140749/225000 [280:06:21<127:36:36,  5.45s/it]
 63%|██████▎   | 140750/225000 [280:06:30<155:44:42,  6.65s/it]
                                                               
{'loss': '0.7895', 'grad_norm': '0.9738', 'learning_rate': '1.669e-05', 'epoch': '0.6256', 'num_input_tokens_seen': 464456768, 'train_runtime': '1.008e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140750/225000 [280:06:30<155:44:42,  6.65s/it]
 63%|██████▎   | 140751/225000 [280:06:34<135:08:58,  5.78s/it]
 63%|██████▎   | 140752/225000 [280:06:38<121:23:19,  5.19s/it]
 63%|██████▎   | 140753/225000 [280:06:47<152:43:54,  6.53s/it]
 63%|██████▎   | 140754/225000 [280:06:52<135:56:34,  5.81s/it]
 63%|██████▎   | 140755/225000 [280:06:56<129:03:11,  5.51s/it]
 63%|██████▎   | 140756/225000 [280:07:02<132:36:20,  5.67s/it]
 63%|██████▎   | 140757/225000 [280:07:06<119:51:13,  5.12s/it]
 63%|██████▎   | 140758/225000 [280:07:11<117:46:17,  5.03s/it]
 63%|██████▎   | 140759/225000 [280:07:22<156:30:28,  6.69s/it]
 63%|██████▎   | 140760/225000 [280:07:25<135:01:42,  5.77s/it]
 63%|██████▎   | 140761/225000 [280:07:29<119:05:31,  5.09s/it]
 63%|██████▎   | 140762/225000 [280:07:37<142:38:02,  6.10s/it]
 63%|██████▎   | 140763/225000 [280:07:42<130:55:34,  5.60s/it]
 63%|██████▎   | 140764/225000 [280:07:46<118:34:39,  5.07s/it]
 63%|██████▎   | 140765/225000 [280:07:56<157:46:54,  6.74s/it]
 63%|██████▎   | 140766/225000 [280:08:00<138:07:37,  5.90s/it]
 63%|██████▎   | 140767/225000 [280:08:04<124:55:01,  5.34s/it]
 63%|██████▎   | 140768/225000 [280:08:13<147:50:11,  6.32s/it]
 63%|██████▎   | 140769/225000 [280:08:19<144:00:56,  6.16s/it]
 63%|██████▎   | 140770/225000 [280:08:23<129:39:17,  5.54s/it]
 63%|██████▎   | 140771/225000 [280:08:28<128:12:41,  5.48s/it]
 63%|██████▎   | 140772/225000 [280:08:33<124:00:20,  5.30s/it]
 63%|██████▎   | 140773/225000 [280:08:38<123:41:15,  5.29s/it]
 63%|██████▎   | 140774/225000 [280:08:44<124:33:10,  5.32s/it]
 63%|██████▎   | 140775/225000 [280:08:49<124:13:36,  5.31s/it]
 63%|██████▎   | 140776/225000 [280:08:54<125:02:56,  5.34s/it]
 63%|██████▎   | 140777/225000 [280:08:59<121:34:34,  5.20s/it]
 63%|██████▎   | 140778/225000 [280:09:05<123:19:09,  5.27s/it]
 63%|██████▎   | 140779/225000 [280:09:09<120:21:20,  5.14s/it]
 63%|██████▎   | 140780/225000 [280:09:15<122:00:13,  5.22s/it]
 63%|██████▎   | 140781/225000 [280:09:20<120:36:59,  5.16s/it]
 63%|██████▎   | 140782/225000 [280:09:25<120:47:16,  5.16s/it]
 63%|██████▎   | 140783/225000 [280:09:30<120:53:15,  5.17s/it]
 63%|██████▎   | 140784/225000 [280:09:35<121:57:29,  5.21s/it]
 63%|██████▎   | 140785/225000 [280:09:41<122:23:48,  5.23s/it]
 63%|██████▎   | 140786/225000 [280:09:46<120:56:56,  5.17s/it]
 63%|██████▎   | 140787/225000 [280:09:51<124:45:47,  5.33s/it]
 63%|██████▎   | 140788/225000 [280:09:57<128:44:07,  5.50s/it]
 63%|██████▎   | 140789/225000 [280:10:10<181:13:46,  7.75s/it]
 63%|██████▎   | 140790/225000 [280:10:16<168:22:00,  7.20s/it]
 63%|██████▎   | 140791/225000 [280:10:24<169:42:40,  7.26s/it]
 63%|██████▎   | 140792/225000 [280:10:29<155:37:24,  6.65s/it]
 63%|██████▎   | 140793/225000 [280:10:36<156:34:15,  6.69s/it]
 63%|██████▎   | 140794/225000 [280:10:41<149:17:45,  6.38s/it]
 63%|██████▎   | 140795/225000 [280:10:48<151:49:27,  6.49s/it]
 63%|██████▎   | 140796/225000 [280:10:52<134:14:22,  5.74s/it]
 63%|██████▎   | 140797/225000 [280:10:56<120:16:47,  5.14s/it]
 63%|██████▎   | 140798/225000 [280:11:07<160:18:57,  6.85s/it]
 63%|██████▎   | 140799/225000 [280:11:10<133:32:54,  5.71s/it]
 63%|██████▎   | 140800/225000 [280:11:14<125:32:02,  5.37s/it]
                                                               
{'loss': '0.7849', 'grad_norm': '1.107', 'learning_rate': '1.667e-05', 'epoch': '0.6258', 'num_input_tokens_seen': 464587424, 'train_runtime': '1.009e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140800/225000 [280:11:14<125:32:02,  5.37s/it]
 63%|██████▎   | 140801/225000 [280:11:19<122:34:30,  5.24s/it]
 63%|██████▎   | 140802/225000 [280:11:24<121:23:38,  5.19s/it]
 63%|██████▎   | 140803/225000 [280:11:29<118:43:03,  5.08s/it]
 63%|██████▎   | 140804/225000 [280:11:34<117:16:22,  5.01s/it]
 63%|██████▎   | 140805/225000 [280:11:44<150:52:21,  6.45s/it]
 63%|██████▎   | 140806/225000 [280:11:52<160:49:07,  6.88s/it]
 63%|██████▎   | 140807/225000 [280:11:56<141:31:17,  6.05s/it]
 63%|██████▎   | 140808/225000 [280:12:00<131:25:21,  5.62s/it]
 63%|██████▎   | 140809/225000 [280:12:05<124:40:00,  5.33s/it]
 63%|██████▎   | 140810/225000 [280:12:13<143:08:49,  6.12s/it]
 63%|██████▎   | 140811/225000 [280:12:23<171:03:36,  7.31s/it]
 63%|██████▎   | 140812/225000 [280:12:27<145:45:47,  6.23s/it]
 63%|██████▎   | 140813/225000 [280:12:31<129:50:25,  5.55s/it]
 63%|██████▎   | 140814/225000 [280:12:39<146:54:54,  6.28s/it]
 63%|██████▎   | 140815/225000 [280:12:43<132:30:55,  5.67s/it]
 63%|██████▎   | 140816/225000 [280:12:47<121:08:38,  5.18s/it]
 63%|██████▎   | 140817/225000 [280:12:57<151:10:19,  6.46s/it]
 63%|██████▎   | 140818/225000 [280:13:03<149:48:29,  6.41s/it]
 63%|██████▎   | 140819/225000 [280:13:06<130:53:15,  5.60s/it]
 63%|██████▎   | 140820/225000 [280:13:13<140:17:18,  6.00s/it]
 63%|██████▎   | 140821/225000 [280:13:21<148:06:11,  6.33s/it]
 63%|██████▎   | 140822/225000 [280:13:24<128:26:19,  5.49s/it]
 63%|██████▎   | 140823/225000 [280:13:28<118:18:16,  5.06s/it]
 63%|██████▎   | 140824/225000 [280:13:34<126:51:21,  5.43s/it]
 63%|██████▎   | 140825/225000 [280:13:44<157:26:03,  6.73s/it]
 63%|██████▎   | 140826/225000 [280:13:49<141:22:50,  6.05s/it]
 63%|██████▎   | 140827/225000 [280:13:54<136:29:09,  5.84s/it]
 63%|██████▎   | 140828/225000 [280:13:59<131:14:40,  5.61s/it]
 63%|██████▎   | 140829/225000 [280:14:04<127:18:42,  5.45s/it]
 63%|██████▎   | 140830/225000 [280:14:10<128:52:26,  5.51s/it]
 63%|██████▎   | 140831/225000 [280:14:15<127:56:48,  5.47s/it]
 63%|██████▎   | 140832/225000 [280:14:20<124:48:49,  5.34s/it]
 63%|██████▎   | 140833/225000 [280:14:26<126:19:26,  5.40s/it]
 63%|██████▎   | 140834/225000 [280:14:31<123:14:36,  5.27s/it]
 63%|██████▎   | 140835/225000 [280:14:36<122:05:46,  5.22s/it]
 63%|██████▎   | 140836/225000 [280:14:41<122:49:36,  5.25s/it]
 63%|██████▎   | 140837/225000 [280:14:46<122:47:15,  5.25s/it]
 63%|██████▎   | 140838/225000 [280:14:51<119:36:23,  5.12s/it]
 63%|██████▎   | 140839/225000 [280:14:56<117:35:59,  5.03s/it]
 63%|██████▎   | 140840/225000 [280:15:04<137:09:43,  5.87s/it]
 63%|██████▎   | 140841/225000 [280:15:13<162:24:00,  6.95s/it]
 63%|██████▎   | 140842/225000 [280:15:18<143:29:42,  6.14s/it]
 63%|██████▎   | 140843/225000 [280:15:22<129:20:57,  5.53s/it]
 63%|██████▎   | 140844/225000 [280:15:29<138:40:42,  5.93s/it]
 63%|██████▎   | 140845/225000 [280:15:38<160:58:25,  6.89s/it]
 63%|██████▎   | 140846/225000 [280:15:41<134:47:51,  5.77s/it]
 63%|██████▎   | 140847/225000 [280:15:45<124:47:05,  5.34s/it]
 63%|██████▎   | 140848/225000 [280:15:51<125:06:55,  5.35s/it]
 63%|██████▎   | 140849/225000 [280:15:56<122:43:28,  5.25s/it]
 63%|██████▎   | 140850/225000 [280:16:00<118:55:06,  5.09s/it]
                                                               
{'loss': '0.7788', 'grad_norm': '1.166', 'learning_rate': '1.666e-05', 'epoch': '0.626', 'num_input_tokens_seen': 464717760, 'train_runtime': '1.009e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140850/225000 [280:16:00<118:55:06,  5.09s/it]
 63%|██████▎   | 140851/225000 [280:16:06<121:11:24,  5.18s/it]
 63%|██████▎   | 140852/225000 [280:16:11<121:31:34,  5.20s/it]
 63%|██████▎   | 140853/225000 [280:16:16<121:05:53,  5.18s/it]
 63%|██████▎   | 140854/225000 [280:16:24<138:38:07,  5.93s/it]
 63%|██████▎   | 140855/225000 [280:16:36<180:20:09,  7.72s/it]
 63%|██████▎   | 140856/225000 [280:16:39<152:05:17,  6.51s/it]
 63%|██████▎   | 140857/225000 [280:16:43<129:44:40,  5.55s/it]
 63%|██████▎   | 140858/225000 [280:16:52<157:49:48,  6.75s/it]
 63%|██████▎   | 140859/225000 [280:16:57<146:59:54,  6.29s/it]
 63%|██████▎   | 140860/225000 [280:17:01<131:28:03,  5.62s/it]
 63%|██████▎   | 140861/225000 [280:17:05<120:08:06,  5.14s/it]
 63%|██████▎   | 140862/225000 [280:17:15<150:58:31,  6.46s/it]
 63%|██████▎   | 140863/225000 [280:17:20<139:56:28,  5.99s/it]
 63%|██████▎   | 140864/225000 [280:17:25<132:33:44,  5.67s/it]
 63%|██████▎   | 140865/225000 [280:17:29<124:04:43,  5.31s/it]
 63%|██████▎   | 140866/225000 [280:17:34<119:07:27,  5.10s/it]
 63%|██████▎   | 140867/225000 [280:17:38<115:30:25,  4.94s/it]
 63%|██████▎   | 140868/225000 [280:17:43<114:19:37,  4.89s/it]
 63%|██████▎   | 140869/225000 [280:17:50<127:18:13,  5.45s/it]
 63%|██████▎   | 140870/225000 [280:18:02<170:44:48,  7.31s/it]
 63%|██████▎   | 140871/225000 [280:18:05<142:09:10,  6.08s/it]
 63%|██████▎   | 140872/225000 [280:18:09<127:36:46,  5.46s/it]
 63%|██████▎   | 140873/225000 [280:18:13<121:04:34,  5.18s/it]
 63%|██████▎   | 140874/225000 [280:18:18<116:26:56,  4.98s/it]
 63%|██████▎   | 140875/225000 [280:18:25<133:22:35,  5.71s/it]
 63%|██████▎   | 140876/225000 [280:18:36<169:48:41,  7.27s/it]
 63%|██████▎   | 140877/225000 [280:18:39<141:39:37,  6.06s/it]
 63%|██████▎   | 140878/225000 [280:18:43<126:06:28,  5.40s/it]
 63%|██████▎   | 140879/225000 [280:18:53<155:05:57,  6.64s/it]
 63%|██████▎   | 140880/225000 [280:18:56<132:20:57,  5.66s/it]
 63%|██████▎   | 140881/225000 [280:19:01<125:03:19,  5.35s/it]
 63%|██████▎   | 140882/225000 [280:19:06<121:34:49,  5.20s/it]
 63%|██████▎   | 140883/225000 [280:19:10<118:22:38,  5.07s/it]
 63%|██████▎   | 140884/225000 [280:19:15<114:18:28,  4.89s/it]
 63%|██████▎   | 140885/225000 [280:19:24<145:46:00,  6.24s/it]
 63%|██████▎   | 140886/225000 [280:19:34<167:00:11,  7.15s/it]
 63%|██████▎   | 140887/225000 [280:19:38<148:58:47,  6.38s/it]
 63%|██████▎   | 140888/225000 [280:19:43<136:10:55,  5.83s/it]
 63%|██████▎   | 140889/225000 [280:19:47<124:58:02,  5.35s/it]
 63%|██████▎   | 140890/225000 [280:19:52<120:23:44,  5.15s/it]
 63%|██████▎   | 140891/225000 [280:20:03<163:58:10,  7.02s/it]
 63%|██████▎   | 140892/225000 [280:20:07<139:59:24,  5.99s/it]
 63%|██████▎   | 140893/225000 [280:20:15<158:18:42,  6.78s/it]
 63%|██████▎   | 140894/225000 [280:20:20<143:47:44,  6.15s/it]
 63%|██████▎   | 140895/225000 [280:20:24<131:21:26,  5.62s/it]
 63%|██████▎   | 140896/225000 [280:20:30<129:39:23,  5.55s/it]
 63%|██████▎   | 140897/225000 [280:20:35<127:20:42,  5.45s/it]
 63%|██████▎   | 140898/225000 [280:20:40<126:15:10,  5.40s/it]
 63%|██████▎   | 140899/225000 [280:20:45<124:12:32,  5.32s/it]
 63%|██████▎   | 140900/225000 [280:20:50<122:39:41,  5.25s/it]
                                                               
{'loss': '0.7835', 'grad_norm': '1.016', 'learning_rate': '1.664e-05', 'epoch': '0.6262', 'num_input_tokens_seen': 464847856, 'train_runtime': '1.009e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140900/225000 [280:20:50<122:39:41,  5.25s/it]
 63%|██████▎   | 140901/225000 [280:20:57<131:24:51,  5.63s/it]
 63%|██████▎   | 140902/225000 [280:21:06<154:37:19,  6.62s/it]
 63%|██████▎   | 140903/225000 [280:21:10<134:24:04,  5.75s/it]
 63%|██████▎   | 140904/225000 [280:21:13<121:00:57,  5.18s/it]
 63%|██████▎   | 140905/225000 [280:21:24<155:52:07,  6.67s/it]
 63%|██████▎   | 140906/225000 [280:21:28<139:47:36,  5.98s/it]
 63%|██████▎   | 140907/225000 [280:21:32<128:26:30,  5.50s/it]
 63%|██████▎   | 140908/225000 [280:21:36<118:09:00,  5.06s/it]
 63%|██████▎   | 140909/225000 [280:21:47<154:32:00,  6.62s/it]
 63%|██████▎   | 140910/225000 [280:21:53<153:01:13,  6.55s/it]
 63%|██████▎   | 140911/225000 [280:21:57<132:49:10,  5.69s/it]
 63%|██████▎   | 140912/225000 [280:22:02<132:25:53,  5.67s/it]
 63%|██████▎   | 140913/225000 [280:22:10<144:38:33,  6.19s/it]
 63%|██████▎   | 140914/225000 [280:22:14<133:10:18,  5.70s/it]
 63%|██████▎   | 140915/225000 [280:22:22<148:52:22,  6.37s/it]
 63%|██████▎   | 140916/225000 [280:22:27<138:54:08,  5.95s/it]
 63%|██████▎   | 140917/225000 [280:22:33<136:40:59,  5.85s/it]
 63%|██████▎   | 140918/225000 [280:22:41<149:45:40,  6.41s/it]
 63%|██████▎   | 140919/225000 [280:22:44<132:30:58,  5.67s/it]
 63%|██████▎   | 140920/225000 [280:22:48<119:43:57,  5.13s/it]
 63%|██████▎   | 140921/225000 [280:22:53<115:21:19,  4.94s/it]
 63%|██████▎   | 140922/225000 [280:23:02<141:35:15,  6.06s/it]
 63%|██████▎   | 140923/225000 [280:23:05<125:17:19,  5.36s/it]
 63%|██████▎   | 140924/225000 [280:23:09<114:51:53,  4.92s/it]
 63%|██████▎   | 140925/225000 [280:23:18<143:12:36,  6.13s/it]
 63%|██████▎   | 140926/225000 [280:23:22<127:56:36,  5.48s/it]
 63%|██████▎   | 140927/225000 [280:23:27<125:12:48,  5.36s/it]
 63%|██████▎   | 140928/225000 [280:23:37<155:54:08,  6.68s/it]
 63%|██████▎   | 140929/225000 [280:23:40<133:00:54,  5.70s/it]
 63%|██████▎   | 140930/225000 [280:23:45<128:26:36,  5.50s/it]
 63%|██████▎   | 140931/225000 [280:23:51<126:30:43,  5.42s/it]
 63%|██████▎   | 140932/225000 [280:23:56<128:33:44,  5.51s/it]
 63%|██████▎   | 140933/225000 [280:24:01<126:24:29,  5.41s/it]
 63%|██████▎   | 140934/225000 [280:24:07<127:20:08,  5.45s/it]
 63%|██████▎   | 140935/225000 [280:24:13<129:56:48,  5.56s/it]
 63%|██████▎   | 140936/225000 [280:24:22<156:24:12,  6.70s/it]
 63%|██████▎   | 140937/225000 [280:24:29<159:41:52,  6.84s/it]
 63%|██████▎   | 140938/225000 [280:24:34<146:49:38,  6.29s/it]
 63%|██████▎   | 140939/225000 [280:24:40<139:08:45,  5.96s/it]
 63%|██████▎   | 140940/225000 [280:24:45<136:09:29,  5.83s/it]
 63%|██████▎   | 140941/225000 [280:24:50<131:37:25,  5.64s/it]
 63%|██████▎   | 140942/225000 [280:24:56<131:03:40,  5.61s/it]
 63%|██████▎   | 140943/225000 [280:25:01<129:16:24,  5.54s/it]
 63%|██████▎   | 140944/225000 [280:25:06<127:50:04,  5.47s/it]
 63%|██████▎   | 140945/225000 [280:25:12<126:39:34,  5.42s/it]
 63%|██████▎   | 140946/225000 [280:25:21<154:28:02,  6.62s/it]
 63%|██████▎   | 140947/225000 [280:25:27<145:42:24,  6.24s/it]
 63%|██████▎   | 140948/225000 [280:25:32<140:53:51,  6.03s/it]
 63%|██████▎   | 140949/225000 [280:25:37<134:56:40,  5.78s/it]
 63%|██████▎   | 140950/225000 [280:25:42<129:10:57,  5.53s/it]
                                                               
{'loss': '0.7984', 'grad_norm': '1.097', 'learning_rate': '1.662e-05', 'epoch': '0.6264', 'num_input_tokens_seen': 464977400, 'train_runtime': '1.01e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 140950/225000 [280:25:42<129:10:57,  5.53s/it]
 63%|██████▎   | 140951/225000 [280:25:49<138:36:47,  5.94s/it]
 63%|██████▎   | 140952/225000 [280:25:56<142:59:55,  6.13s/it]
 63%|██████▎   | 140953/225000 [280:25:59<125:31:07,  5.38s/it]
 63%|██████▎   | 140954/225000 [280:26:08<146:10:42,  6.26s/it]
 63%|██████▎   | 140955/225000 [280:26:13<136:34:30,  5.85s/it]
 63%|██████▎   | 140956/225000 [280:26:17<128:39:46,  5.51s/it]
 63%|██████▎   | 140957/225000 [280:26:22<126:14:13,  5.41s/it]
 63%|██████▎   | 140958/225000 [280:26:27<122:46:28,  5.26s/it]
 63%|██████▎   | 140959/225000 [280:26:32<120:29:34,  5.16s/it]
 63%|██████▎   | 140960/225000 [280:26:37<120:00:20,  5.14s/it]
 63%|██████▎   | 140961/225000 [280:26:42<119:45:03,  5.13s/it]
 63%|██████▎   | 140962/225000 [280:26:47<116:50:56,  5.01s/it]
 63%|██████▎   | 140963/225000 [280:26:59<164:38:27,  7.05s/it]
 63%|██████▎   | 140964/225000 [280:27:06<164:23:04,  7.04s/it]
 63%|██████▎   | 140965/225000 [280:27:12<154:41:01,  6.63s/it]
 63%|██████▎   | 140966/225000 [280:27:19<160:31:07,  6.88s/it]
 63%|██████▎   | 140967/225000 [280:27:23<140:55:39,  6.04s/it]
 63%|██████▎   | 140968/225000 [280:27:27<125:46:48,  5.39s/it]
 63%|██████▎   | 140969/225000 [280:27:31<118:05:31,  5.06s/it]
 63%|██████▎   | 140970/225000 [280:27:37<119:56:03,  5.14s/it]
 63%|██████▎   | 140971/225000 [280:27:46<148:38:51,  6.37s/it]
 63%|██████▎   | 140972/225000 [280:27:49<128:41:50,  5.51s/it]
 63%|██████▎   | 140973/225000 [280:27:56<134:16:50,  5.75s/it]
 63%|██████▎   | 140974/225000 [280:28:04<149:23:47,  6.40s/it]
 63%|██████▎   | 140975/225000 [280:28:07<129:25:17,  5.54s/it]
 63%|██████▎   | 140976/225000 [280:28:17<155:47:09,  6.67s/it]
 63%|██████▎   | 140977/225000 [280:28:20<135:33:30,  5.81s/it]
 63%|██████▎   | 140978/225000 [280:28:24<120:33:27,  5.17s/it]
 63%|██████▎   | 140979/225000 [280:28:33<150:14:56,  6.44s/it]
 63%|██████▎   | 140980/225000 [280:28:36<126:18:10,  5.41s/it]
 63%|██████▎   | 140981/225000 [280:28:45<147:04:20,  6.30s/it]
 63%|██████▎   | 140982/225000 [280:28:49<129:36:13,  5.55s/it]
 63%|██████▎   | 140983/225000 [280:28:55<136:56:58,  5.87s/it]
 63%|██████▎   | 140984/225000 [280:29:00<130:02:19,  5.57s/it]
 63%|██████▎   | 140985/225000 [280:29:05<125:30:22,  5.38s/it]
 63%|██████▎   | 140986/225000 [280:29:10<121:22:41,  5.20s/it]
 63%|██████▎   | 140987/225000 [280:29:15<121:31:59,  5.21s/it]
 63%|██████▎   | 140988/225000 [280:29:20<121:17:12,  5.20s/it]
 63%|██████▎   | 140989/225000 [280:29:25<119:52:45,  5.14s/it]
 63%|██████▎   | 140990/225000 [280:29:30<118:26:28,  5.08s/it]
 63%|██████▎   | 140991/225000 [280:29:35<114:25:45,  4.90s/it]
 63%|██████▎   | 140992/225000 [280:29:40<115:22:52,  4.94s/it]
 63%|██████▎   | 140993/225000 [280:29:45<116:26:43,  4.99s/it]
 63%|██████▎   | 140994/225000 [280:29:55<153:13:33,  6.57s/it]
 63%|██████▎   | 140995/225000 [280:30:04<169:54:53,  7.28s/it]
 63%|██████▎   | 140996/225000 [280:30:08<148:51:29,  6.38s/it]
 63%|██████▎   | 140997/225000 [280:30:13<137:44:21,  5.90s/it]
 63%|██████▎   | 140998/225000 [280:30:18<128:26:43,  5.50s/it]
 63%|██████▎   | 140999/225000 [280:30:22<122:31:38,  5.25s/it]
 63%|██████▎   | 141000/225000 [280:30:27<118:59:57,  5.10s/it]
                                                               
{'loss': '0.7888', 'grad_norm': '1.072', 'learning_rate': '1.661e-05', 'epoch': '0.6267', 'num_input_tokens_seen': 465107384, 'train_runtime': '1.01e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141000/225000 [280:30:27<118:59:57,  5.10s/it]
 63%|██████▎   | 141001/225000 [280:30:37<152:17:56,  6.53s/it]
 63%|██████▎   | 141002/225000 [280:30:44<158:14:48,  6.78s/it]
 63%|██████▎   | 141003/225000 [280:30:48<136:09:46,  5.84s/it]
 63%|██████▎   | 141004/225000 [280:30:52<123:48:45,  5.31s/it]
 63%|██████▎   | 141005/225000 [280:31:02<159:08:42,  6.82s/it]
 63%|██████▎   | 141006/225000 [280:31:06<137:30:44,  5.89s/it]
 63%|██████▎   | 141007/225000 [280:31:10<123:37:53,  5.30s/it]
 63%|██████▎   | 141008/225000 [280:31:14<112:51:44,  4.84s/it]
 63%|██████▎   | 141009/225000 [280:31:24<151:24:59,  6.49s/it]
 63%|██████▎   | 141010/225000 [280:31:28<136:28:06,  5.85s/it]
 63%|██████▎   | 141011/225000 [280:31:33<126:02:06,  5.40s/it]
 63%|██████▎   | 141012/225000 [280:31:37<118:55:19,  5.10s/it]
 63%|██████▎   | 141013/225000 [280:31:41<112:31:29,  4.82s/it]
 63%|██████▎   | 141014/225000 [280:31:50<138:57:49,  5.96s/it]
 63%|██████▎   | 141015/225000 [280:31:59<161:10:07,  6.91s/it]
 63%|██████▎   | 141016/225000 [280:32:04<147:14:54,  6.31s/it]
 63%|██████▎   | 141017/225000 [280:32:09<136:06:59,  5.83s/it]
 63%|██████▎   | 141018/225000 [280:32:14<129:47:45,  5.56s/it]
 63%|██████▎   | 141019/225000 [280:32:19<125:06:09,  5.36s/it]
 63%|██████▎   | 141020/225000 [280:32:23<121:21:47,  5.20s/it]
 63%|██████▎   | 141021/225000 [280:32:28<120:06:35,  5.15s/it]
 63%|██████▎   | 141022/225000 [280:32:38<153:30:43,  6.58s/it]
 63%|██████▎   | 141023/225000 [280:32:47<168:02:37,  7.20s/it]
 63%|██████▎   | 141024/225000 [280:32:50<141:26:00,  6.06s/it]
 63%|██████▎   | 141025/225000 [280:32:59<156:04:10,  6.69s/it]
 63%|██████▎   | 141026/225000 [280:33:03<142:17:32,  6.10s/it]
 63%|██████▎   | 141027/225000 [280:33:07<125:13:36,  5.37s/it]
 63%|██████▎   | 141028/225000 [280:33:13<126:53:28,  5.44s/it]
 63%|██████▎   | 141029/225000 [280:33:22<152:40:24,  6.55s/it]
 63%|██████▎   | 141030/225000 [280:33:25<133:38:08,  5.73s/it]
 63%|██████▎   | 141031/225000 [280:33:30<122:42:18,  5.26s/it]
 63%|██████▎   | 141032/225000 [280:33:34<117:27:26,  5.04s/it]
 63%|██████▎   | 141033/225000 [280:33:38<112:00:12,  4.80s/it]
 63%|██████▎   | 141034/225000 [280:33:51<165:24:40,  7.09s/it]
 63%|██████▎   | 141035/225000 [280:33:57<155:48:16,  6.68s/it]
 63%|██████▎   | 141036/225000 [280:34:01<137:52:37,  5.91s/it]
 63%|██████▎   | 141037/225000 [280:34:04<122:24:05,  5.25s/it]
 63%|██████▎   | 141038/225000 [280:34:13<146:06:28,  6.26s/it]
 63%|██████▎   | 141039/225000 [280:34:16<126:33:51,  5.43s/it]
 63%|██████▎   | 141040/225000 [280:34:22<123:52:09,  5.31s/it]
 63%|██████▎   | 141041/225000 [280:34:29<135:45:24,  5.82s/it]
 63%|██████▎   | 141042/225000 [280:34:33<129:20:54,  5.55s/it]
 63%|██████▎   | 141043/225000 [280:34:38<122:01:40,  5.23s/it]
 63%|██████▎   | 141044/225000 [280:34:43<120:20:39,  5.16s/it]
 63%|██████▎   | 141045/225000 [280:34:47<115:15:03,  4.94s/it]
 63%|██████▎   | 141046/225000 [280:34:52<111:01:51,  4.76s/it]
 63%|██████▎   | 141047/225000 [280:34:56<108:44:09,  4.66s/it]
 63%|██████▎   | 141048/225000 [280:35:01<110:42:10,  4.75s/it]
 63%|██████▎   | 141049/225000 [280:35:06<109:57:10,  4.72s/it]
 63%|██████▎   | 141050/225000 [280:35:13<130:18:34,  5.59s/it]
                                                               
{'loss': '0.7953', 'grad_norm': '1.074', 'learning_rate': '1.659e-05', 'epoch': '0.6269', 'num_input_tokens_seen': 465236576, 'train_runtime': '1.01e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141050/225000 [280:35:13<130:18:34,  5.59s/it]
 63%|██████▎   | 141051/225000 [280:35:25<173:57:23,  7.46s/it]
 63%|██████▎   | 141052/225000 [280:35:29<149:40:04,  6.42s/it]
 63%|██████▎   | 141053/225000 [280:35:33<130:31:26,  5.60s/it]
 63%|██████▎   | 141054/225000 [280:35:38<126:57:19,  5.44s/it]
 63%|██████▎   | 141055/225000 [280:35:47<155:34:34,  6.67s/it]
 63%|██████▎   | 141056/225000 [280:35:51<132:58:14,  5.70s/it]
 63%|██████▎   | 141057/225000 [280:35:56<126:33:07,  5.43s/it]
 63%|██████▎   | 141058/225000 [280:36:04<148:34:37,  6.37s/it]
 63%|██████▎   | 141059/225000 [280:36:08<129:35:18,  5.56s/it]
 63%|██████▎   | 141060/225000 [280:36:17<153:06:27,  6.57s/it]
 63%|██████▎   | 141061/225000 [280:36:21<134:26:59,  5.77s/it]
 63%|██████▎   | 141062/225000 [280:36:26<129:18:07,  5.55s/it]
 63%|██████▎   | 141063/225000 [280:36:31<129:04:28,  5.54s/it]
 63%|██████▎   | 141064/225000 [280:36:37<128:08:02,  5.50s/it]
 63%|██████▎   | 141065/225000 [280:36:42<129:49:45,  5.57s/it]
 63%|██████▎   | 141066/225000 [280:36:48<130:04:38,  5.58s/it]
 63%|██████▎   | 141067/225000 [280:36:54<129:22:00,  5.55s/it]
 63%|██████▎   | 141068/225000 [280:36:59<129:08:05,  5.54s/it]
 63%|██████▎   | 141069/225000 [280:37:04<126:50:18,  5.44s/it]
 63%|██████▎   | 141070/225000 [280:37:09<125:02:39,  5.36s/it]
 63%|██████▎   | 141071/225000 [280:37:15<123:11:54,  5.28s/it]
 63%|██████▎   | 141072/225000 [280:37:20<122:32:11,  5.26s/it]
 63%|██████▎   | 141073/225000 [280:37:25<124:05:46,  5.32s/it]
 63%|██████▎   | 141074/225000 [280:37:31<124:48:26,  5.35s/it]
 63%|██████▎   | 141075/225000 [280:37:36<122:13:46,  5.24s/it]
 63%|██████▎   | 141076/225000 [280:37:48<170:58:11,  7.33s/it]
 63%|██████▎   | 141077/225000 [280:37:55<167:32:01,  7.19s/it]
 63%|██████▎   | 141078/225000 [280:37:59<146:29:13,  6.28s/it]
 63%|██████▎   | 141079/225000 [280:38:04<137:30:28,  5.90s/it]
 63%|██████▎   | 141080/225000 [280:38:09<133:32:31,  5.73s/it]
 63%|██████▎   | 141081/225000 [280:38:14<129:54:18,  5.57s/it]
 63%|██████▎   | 141082/225000 [280:38:19<123:51:55,  5.31s/it]
 63%|██████▎   | 141083/225000 [280:38:24<120:38:29,  5.18s/it]
 63%|██████▎   | 141084/225000 [280:38:29<119:43:42,  5.14s/it]
 63%|██████▎   | 141085/225000 [280:38:34<121:32:08,  5.21s/it]
 63%|██████▎   | 141086/225000 [280:38:39<119:32:32,  5.13s/it]
 63%|██████▎   | 141087/225000 [280:38:49<154:11:51,  6.62s/it]
 63%|██████▎   | 141088/225000 [280:38:58<171:03:22,  7.34s/it]
 63%|██████▎   | 141089/225000 [280:39:02<146:40:04,  6.29s/it]
 63%|██████▎   | 141090/225000 [280:39:07<136:45:08,  5.87s/it]
 63%|██████▎   | 141091/225000 [280:39:12<131:29:43,  5.64s/it]
 63%|██████▎   | 141092/225000 [280:39:17<128:15:37,  5.50s/it]
 63%|██████▎   | 141093/225000 [280:39:23<127:04:55,  5.45s/it]
 63%|██████▎   | 141094/225000 [280:39:28<124:05:50,  5.32s/it]
 63%|██████▎   | 141095/225000 [280:39:32<118:58:48,  5.10s/it]
 63%|██████▎   | 141096/225000 [280:39:37<117:07:13,  5.03s/it]
 63%|██████▎   | 141097/225000 [280:39:42<115:54:07,  4.97s/it]
 63%|██████▎   | 141098/225000 [280:39:52<152:01:37,  6.52s/it]
 63%|██████▎   | 141099/225000 [280:40:03<178:14:57,  7.65s/it]
 63%|██████▎   | 141100/225000 [280:40:07<153:18:53,  6.58s/it]
                                                               
{'loss': '0.7991', 'grad_norm': '1.02', 'learning_rate': '1.657e-05', 'epoch': '0.6271', 'num_input_tokens_seen': 465367136, 'train_runtime': '1.01e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141100/225000 [280:40:07<153:18:53,  6.58s/it]
 63%|██████▎   | 141101/225000 [280:40:11<141:37:55,  6.08s/it]
 63%|██████▎   | 141102/225000 [280:40:17<134:28:08,  5.77s/it]
 63%|██████▎   | 141103/225000 [280:40:22<130:14:50,  5.59s/it]
 63%|██████▎   | 141104/225000 [280:40:26<124:01:23,  5.32s/it]
 63%|██████▎   | 141105/225000 [280:40:32<123:18:56,  5.29s/it]
 63%|██████▎   | 141106/225000 [280:40:36<119:53:10,  5.14s/it]
 63%|██████▎   | 141107/225000 [280:40:42<120:02:32,  5.15s/it]
 63%|██████▎   | 141108/225000 [280:40:47<120:44:21,  5.18s/it]
 63%|██████▎   | 141109/225000 [280:41:00<174:29:08,  7.49s/it]
 63%|██████▎   | 141110/225000 [280:41:06<164:00:02,  7.04s/it]
 63%|██████▎   | 141111/225000 [280:41:12<160:15:21,  6.88s/it]
 63%|██████▎   | 141112/225000 [280:41:16<137:02:53,  5.88s/it]
 63%|██████▎   | 141113/225000 [280:41:19<121:46:37,  5.23s/it]
 63%|██████▎   | 141114/225000 [280:41:29<154:01:48,  6.61s/it]
 63%|██████▎   | 141115/225000 [280:41:33<136:55:27,  5.88s/it]
 63%|██████▎   | 141116/225000 [280:41:38<126:29:31,  5.43s/it]
 63%|██████▎   | 141117/225000 [280:41:46<145:50:33,  6.26s/it]
 63%|██████▎   | 141118/225000 [280:41:54<155:39:09,  6.68s/it]
 63%|██████▎   | 141119/225000 [280:41:57<133:34:36,  5.73s/it]
 63%|██████▎   | 141120/225000 [280:42:05<149:52:31,  6.43s/it]
 63%|██████▎   | 141121/225000 [280:42:09<131:17:03,  5.63s/it]
 63%|██████▎   | 141122/225000 [280:42:13<119:20:03,  5.12s/it]
 63%|██████▎   | 141123/225000 [280:42:21<140:47:20,  6.04s/it]
 63%|██████▎   | 141124/225000 [280:42:29<151:09:36,  6.49s/it]
 63%|██████▎   | 141125/225000 [280:42:33<137:44:31,  5.91s/it]
 63%|██████▎   | 141126/225000 [280:42:38<128:26:55,  5.51s/it]
 63%|██████▎   | 141127/225000 [280:42:42<121:25:00,  5.21s/it]
 63%|██████▎   | 141128/225000 [280:42:47<117:26:05,  5.04s/it]
 63%|██████▎   | 141129/225000 [280:42:52<113:37:30,  4.88s/it]
 63%|██████▎   | 141130/225000 [280:43:04<164:11:17,  7.05s/it]
 63%|██████▎   | 141131/225000 [280:43:07<138:33:15,  5.95s/it]
 63%|██████▎   | 141132/225000 [280:43:12<134:26:50,  5.77s/it]
 63%|██████▎   | 141133/225000 [280:43:19<139:30:18,  5.99s/it]
 63%|██████▎   | 141134/225000 [280:43:23<123:18:38,  5.29s/it]
 63%|██████▎   | 141135/225000 [280:43:34<163:43:36,  7.03s/it]
 63%|██████▎   | 141136/225000 [280:43:37<137:02:57,  5.88s/it]
 63%|██████▎   | 141137/225000 [280:43:45<152:36:22,  6.55s/it]
 63%|██████▎   | 141138/225000 [280:43:48<129:33:09,  5.56s/it]
 63%|██████▎   | 141139/225000 [280:43:53<127:23:10,  5.47s/it]
 63%|██████▎   | 141140/225000 [280:43:58<123:47:46,  5.31s/it]
 63%|██████▎   | 141141/225000 [280:44:03<120:58:50,  5.19s/it]
 63%|██████▎   | 141142/225000 [280:44:09<122:14:17,  5.25s/it]
 63%|██████▎   | 141143/225000 [280:44:14<122:07:57,  5.24s/it]
 63%|██████▎   | 141144/225000 [280:44:19<124:00:17,  5.32s/it]
 63%|██████▎   | 141145/225000 [280:44:24<121:56:53,  5.24s/it]
 63%|██████▎   | 141146/225000 [280:44:30<121:43:38,  5.23s/it]
 63%|██████▎   | 141147/225000 [280:44:36<129:21:37,  5.55s/it]
 63%|██████▎   | 141148/225000 [280:44:49<182:04:18,  7.82s/it]
 63%|██████▎   | 141149/225000 [280:44:53<155:59:55,  6.70s/it]
 63%|██████▎   | 141150/225000 [280:44:57<138:04:47,  5.93s/it]
                                                               
{'loss': '0.7789', 'grad_norm': '1.012', 'learning_rate': '1.655e-05', 'epoch': '0.6273', 'num_input_tokens_seen': 465497496, 'train_runtime': '1.011e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141150/225000 [280:44:57<138:04:47,  5.93s/it]
 63%|██████▎   | 141151/225000 [280:45:02<129:31:22,  5.56s/it]
 63%|██████▎   | 141152/225000 [280:45:06<119:47:32,  5.14s/it]
 63%|██████▎   | 141153/225000 [280:45:11<115:07:57,  4.94s/it]
 63%|██████▎   | 141154/225000 [280:45:18<131:48:32,  5.66s/it]
 63%|██████▎   | 141155/225000 [280:45:28<162:11:23,  6.96s/it]
 63%|██████▎   | 141156/225000 [280:45:32<144:25:43,  6.20s/it]
 63%|██████▎   | 141157/225000 [280:45:36<129:44:43,  5.57s/it]
 63%|██████▎   | 141158/225000 [280:45:41<121:14:58,  5.21s/it]
 63%|██████▎   | 141159/225000 [280:45:45<112:39:45,  4.84s/it]
 63%|██████▎   | 141160/225000 [280:45:54<142:40:06,  6.13s/it]
 63%|██████▎   | 141161/225000 [280:46:02<153:10:57,  6.58s/it]
 63%|██████▎   | 141162/225000 [280:46:09<159:47:15,  6.86s/it]
 63%|██████▎   | 141163/225000 [280:46:12<133:37:57,  5.74s/it]
 63%|██████▎   | 141164/225000 [280:46:17<128:21:55,  5.51s/it]
 63%|██████▎   | 141165/225000 [280:46:26<150:48:19,  6.48s/it]
 63%|██████▎   | 141166/225000 [280:46:31<138:42:10,  5.96s/it]
 63%|██████▎   | 141167/225000 [280:46:35<128:10:17,  5.50s/it]
 63%|██████▎   | 141168/225000 [280:46:40<122:04:45,  5.24s/it]
 63%|██████▎   | 141169/225000 [280:46:44<115:50:32,  4.97s/it]
 63%|██████▎   | 141170/225000 [280:46:49<115:30:54,  4.96s/it]
 63%|██████▎   | 141171/225000 [280:46:54<118:25:57,  5.09s/it]
 63%|██████▎   | 141172/225000 [280:47:08<175:09:35,  7.52s/it]
 63%|██████▎   | 141173/225000 [280:47:11<149:28:24,  6.42s/it]
 63%|██████▎   | 141174/225000 [280:47:18<153:37:12,  6.60s/it]
 63%|██████▎   | 141175/225000 [280:47:23<140:00:19,  6.01s/it]
 63%|██████▎   | 141176/225000 [280:47:28<129:11:45,  5.55s/it]
 63%|██████▎   | 141177/225000 [280:47:32<122:54:35,  5.28s/it]
 63%|██████▎   | 141178/225000 [280:47:37<117:43:20,  5.06s/it]
 63%|██████▎   | 141179/225000 [280:47:42<117:23:39,  5.04s/it]
 63%|██████▎   | 141180/225000 [280:47:54<167:52:57,  7.21s/it]
 63%|██████▎   | 141181/225000 [280:47:58<148:13:18,  6.37s/it]
 63%|██████▎   | 141182/225000 [280:48:02<128:07:48,  5.50s/it]
 63%|██████▎   | 141183/225000 [280:48:11<149:36:07,  6.43s/it]
 63%|██████▎   | 141184/225000 [280:48:15<133:24:33,  5.73s/it]
 63%|██████▎   | 141185/225000 [280:48:19<122:04:16,  5.24s/it]
 63%|██████▎   | 141186/225000 [280:48:23<113:36:29,  4.88s/it]
 63%|██████▎   | 141187/225000 [280:48:34<155:24:17,  6.68s/it]
 63%|██████▎   | 141188/225000 [280:48:37<134:54:04,  5.79s/it]
 63%|██████▎   | 141189/225000 [280:48:42<126:05:53,  5.42s/it]
 63%|██████▎   | 141190/225000 [280:48:46<115:06:42,  4.94s/it]
 63%|██████▎   | 141191/225000 [280:48:50<109:35:10,  4.71s/it]
 63%|██████▎   | 141192/225000 [280:48:55<109:20:41,  4.70s/it]
 63%|██████▎   | 141193/225000 [280:49:03<135:23:36,  5.82s/it]
 63%|██████▎   | 141194/225000 [280:49:11<151:06:41,  6.49s/it]
 63%|██████▎   | 141195/225000 [280:49:15<135:35:21,  5.82s/it]
 63%|██████▎   | 141196/225000 [280:49:19<122:46:03,  5.27s/it]
 63%|██████▎   | 141197/225000 [280:49:24<117:25:02,  5.04s/it]
 63%|██████▎   | 141198/225000 [280:49:28<110:21:09,  4.74s/it]
 63%|██████▎   | 141199/225000 [280:49:38<147:35:23,  6.34s/it]
 63%|██████▎   | 141200/225000 [280:49:44<148:37:03,  6.38s/it]
                                                               
{'loss': '0.7936', 'grad_norm': '1.28', 'learning_rate': '1.654e-05', 'epoch': '0.6276', 'num_input_tokens_seen': 465625616, 'train_runtime': '1.011e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141200/225000 [280:49:44<148:37:03,  6.38s/it]
 63%|██████▎   | 141201/225000 [280:49:48<132:22:54,  5.69s/it]
 63%|██████▎   | 141202/225000 [280:49:53<120:40:52,  5.18s/it]
 63%|██████▎   | 141203/225000 [280:49:57<114:53:55,  4.94s/it]
 63%|██████▎   | 141204/225000 [280:50:06<146:47:50,  6.31s/it]
 63%|██████▎   | 141205/225000 [280:50:13<148:30:55,  6.38s/it]
 63%|██████▎   | 141206/225000 [280:50:17<129:06:36,  5.55s/it]
 63%|██████▎   | 141207/225000 [280:50:23<138:35:47,  5.95s/it]
 63%|██████▎   | 141208/225000 [280:50:31<149:29:34,  6.42s/it]
 63%|██████▎   | 141209/225000 [280:50:34<129:09:47,  5.55s/it]
 63%|██████▎   | 141210/225000 [280:50:43<152:32:08,  6.55s/it]
 63%|██████▎   | 141211/225000 [280:50:49<148:33:19,  6.38s/it]
 63%|██████▎   | 141212/225000 [280:50:55<144:29:41,  6.21s/it]
 63%|██████▎   | 141213/225000 [280:51:00<131:59:14,  5.67s/it]
 63%|██████▎   | 141214/225000 [280:51:08<152:42:02,  6.56s/it]
 63%|██████▎   | 141215/225000 [280:51:12<130:25:30,  5.60s/it]
 63%|██████▎   | 141216/225000 [280:51:15<117:55:02,  5.07s/it]
 63%|██████▎   | 141217/225000 [280:51:24<145:52:46,  6.27s/it]
 63%|██████▎   | 141218/225000 [280:51:28<128:36:17,  5.53s/it]
 63%|██████▎   | 141219/225000 [280:51:32<119:19:27,  5.13s/it]
 63%|██████▎   | 141220/225000 [280:51:38<119:21:53,  5.13s/it]
 63%|██████▎   | 141221/225000 [280:51:43<123:30:37,  5.31s/it]
 63%|██████▎   | 141222/225000 [280:51:53<150:44:06,  6.48s/it]
 63%|██████▎   | 141223/225000 [280:51:57<134:10:14,  5.77s/it]
 63%|██████▎   | 141224/225000 [280:52:05<151:51:52,  6.53s/it]
 63%|██████▎   | 141225/225000 [280:52:09<131:25:05,  5.65s/it]
 63%|██████▎   | 141226/225000 [280:52:13<125:16:39,  5.38s/it]
 63%|██████▎   | 141227/225000 [280:52:18<122:53:38,  5.28s/it]
 63%|██████▎   | 141228/225000 [280:52:23<119:53:21,  5.15s/it]
 63%|██████▎   | 141229/225000 [280:52:28<119:44:22,  5.15s/it]
 63%|██████▎   | 141230/225000 [280:52:33<114:44:50,  4.93s/it]
 63%|██████▎   | 141231/225000 [280:52:38<115:29:44,  4.96s/it]
 63%|██████▎   | 141232/225000 [280:52:43<119:02:13,  5.12s/it]
 63%|██████▎   | 141233/225000 [280:52:48<118:20:22,  5.09s/it]
 63%|██████▎   | 141234/225000 [280:52:53<119:19:58,  5.13s/it]
 63%|██████▎   | 141235/225000 [280:53:06<171:59:26,  7.39s/it]
 63%|██████▎   | 141236/225000 [280:53:12<162:57:11,  7.00s/it]
 63%|██████▎   | 141237/225000 [280:53:16<137:39:45,  5.92s/it]
 63%|██████▎   | 141238/225000 [280:53:26<166:04:19,  7.14s/it]
 63%|██████▎   | 141239/225000 [280:53:32<157:43:39,  6.78s/it]
 63%|██████▎   | 141240/225000 [280:53:36<142:32:11,  6.13s/it]
 63%|██████▎   | 141241/225000 [280:53:39<122:17:40,  5.26s/it]
 63%|██████▎   | 141242/225000 [280:53:46<134:32:20,  5.78s/it]
 63%|██████▎   | 141243/225000 [280:53:53<137:39:05,  5.92s/it]
 63%|██████▎   | 141244/225000 [280:53:56<119:55:00,  5.15s/it]
 63%|██████▎   | 141245/225000 [280:54:05<144:01:31,  6.19s/it]
 63%|██████▎   | 141246/225000 [280:54:09<129:19:32,  5.56s/it]
 63%|██████▎   | 141247/225000 [280:54:12<116:55:30,  5.03s/it]
 63%|██████▎   | 141248/225000 [280:54:17<111:15:37,  4.78s/it]
 63%|██████▎   | 141249/225000 [280:54:26<145:20:15,  6.25s/it]
 63%|██████▎   | 141250/225000 [280:54:34<155:27:30,  6.68s/it]
                                                               
{'loss': '0.7962', 'grad_norm': '1.095', 'learning_rate': '1.652e-05', 'epoch': '0.6278', 'num_input_tokens_seen': 465755752, 'train_runtime': '1.011e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141250/225000 [280:54:34<155:27:30,  6.68s/it]
 63%|██████▎   | 141251/225000 [280:54:38<134:12:20,  5.77s/it]
 63%|██████▎   | 141252/225000 [280:54:42<121:35:56,  5.23s/it]
 63%|██████▎   | 141253/225000 [280:54:51<151:55:49,  6.53s/it]
 63%|██████▎   | 141254/225000 [280:54:55<134:29:57,  5.78s/it]
 63%|██████▎   | 141255/225000 [280:54:59<120:41:42,  5.19s/it]
 63%|██████▎   | 141256/225000 [280:55:09<152:09:28,  6.54s/it]
 63%|██████▎   | 141257/225000 [280:55:13<137:27:35,  5.91s/it]
 63%|██████▎   | 141258/225000 [280:55:20<143:24:39,  6.17s/it]
 63%|██████▎   | 141259/225000 [280:55:24<127:02:36,  5.46s/it]
 63%|██████▎   | 141260/225000 [280:55:31<141:30:59,  6.08s/it]
 63%|██████▎   | 141261/225000 [280:55:36<132:32:39,  5.70s/it]
 63%|██████▎   | 141262/225000 [280:55:43<142:12:39,  6.11s/it]
 63%|██████▎   | 141263/225000 [280:55:47<124:16:03,  5.34s/it]
 63%|██████▎   | 141264/225000 [280:55:51<116:36:10,  5.01s/it]
 63%|██████▎   | 141265/225000 [280:55:58<132:38:46,  5.70s/it]
 63%|██████▎   | 141266/225000 [280:56:06<144:53:30,  6.23s/it]
 63%|██████▎   | 141267/225000 [280:56:11<135:07:38,  5.81s/it]
 63%|██████▎   | 141268/225000 [280:56:17<137:59:54,  5.93s/it]
 63%|██████▎   | 141269/225000 [280:56:20<119:15:02,  5.13s/it]
 63%|██████▎   | 141270/225000 [280:56:29<147:11:38,  6.33s/it]
 63%|██████▎   | 141271/225000 [280:56:33<126:13:41,  5.43s/it]
 63%|██████▎   | 141272/225000 [280:56:37<118:34:24,  5.10s/it]
 63%|██████▎   | 141273/225000 [280:56:42<115:31:21,  4.97s/it]
 63%|██████▎   | 141274/225000 [280:56:46<113:38:34,  4.89s/it]
 63%|██████▎   | 141275/225000 [280:56:51<109:57:00,  4.73s/it]
 63%|██████▎   | 141276/225000 [280:56:58<127:07:17,  5.47s/it]
 63%|██████▎   | 141277/225000 [280:57:07<155:57:45,  6.71s/it]
 63%|██████▎   | 141278/225000 [280:57:14<153:00:33,  6.58s/it]
 63%|██████▎   | 141279/225000 [280:57:17<133:56:48,  5.76s/it]
 63%|██████▎   | 141280/225000 [280:57:23<131:35:41,  5.66s/it]
 63%|██████▎   | 141281/225000 [280:57:31<147:45:10,  6.35s/it]
 63%|██████▎   | 141282/225000 [280:57:35<130:36:47,  5.62s/it]
 63%|██████▎   | 141283/225000 [280:57:42<143:59:08,  6.19s/it]
 63%|██████▎   | 141284/225000 [280:57:46<126:33:02,  5.44s/it]
 63%|██████▎   | 141285/225000 [280:57:54<141:57:33,  6.10s/it]
 63%|██████▎   | 141286/225000 [280:57:58<130:39:22,  5.62s/it]
 63%|██████▎   | 141287/225000 [280:58:04<133:52:52,  5.76s/it]
 63%|██████▎   | 141288/225000 [280:58:11<142:40:00,  6.14s/it]
 63%|██████▎   | 141289/225000 [280:58:15<125:21:50,  5.39s/it]
 63%|██████▎   | 141290/225000 [280:58:23<144:29:35,  6.21s/it]
 63%|██████▎   | 141291/225000 [280:58:28<136:27:01,  5.87s/it]
 63%|██████▎   | 141292/225000 [280:58:32<121:29:25,  5.22s/it]
 63%|██████▎   | 141293/225000 [280:58:37<119:39:30,  5.15s/it]
 63%|██████▎   | 141294/225000 [280:58:45<141:27:25,  6.08s/it]
 63%|██████▎   | 141295/225000 [280:58:49<124:44:21,  5.36s/it]
 63%|██████▎   | 141296/225000 [280:58:53<117:13:48,  5.04s/it]
 63%|██████▎   | 141297/225000 [280:59:02<145:36:48,  6.26s/it]
 63%|██████▎   | 141298/225000 [280:59:06<125:33:05,  5.40s/it]
 63%|██████▎   | 141299/225000 [280:59:12<131:40:00,  5.66s/it]
 63%|██████▎   | 141300/225000 [280:59:17<130:44:44,  5.62s/it]
                                                               
{'loss': '0.8037', 'grad_norm': '1.058', 'learning_rate': '1.65e-05', 'epoch': '0.628', 'num_input_tokens_seen': 465885208, 'train_runtime': '1.012e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141300/225000 [280:59:17<130:44:44,  5.62s/it]
 63%|██████▎   | 141301/225000 [280:59:25<143:59:37,  6.19s/it]
 63%|██████▎   | 141302/225000 [280:59:28<124:48:57,  5.37s/it]
 63%|██████▎   | 141303/225000 [280:59:35<132:50:39,  5.71s/it]
 63%|██████▎   | 141304/225000 [280:59:40<130:49:26,  5.63s/it]
 63%|██████▎   | 141305/225000 [280:59:49<149:50:15,  6.45s/it]
 63%|██████▎   | 141306/225000 [280:59:53<136:46:21,  5.88s/it]
 63%|██████▎   | 141307/225000 [280:59:57<125:50:57,  5.41s/it]
 63%|██████▎   | 141308/225000 [281:00:02<120:16:40,  5.17s/it]
 63%|██████▎   | 141309/225000 [281:00:07<116:15:45,  5.00s/it]
 63%|██████▎   | 141310/225000 [281:00:11<113:18:35,  4.87s/it]
 63%|██████▎   | 141311/225000 [281:00:16<109:42:07,  4.72s/it]
 63%|██████▎   | 141312/225000 [281:00:20<108:28:20,  4.67s/it]
 63%|██████▎   | 141313/225000 [281:00:30<143:36:35,  6.18s/it]
 63%|██████▎   | 141314/225000 [281:00:38<157:27:54,  6.77s/it]
 63%|██████▎   | 141315/225000 [281:00:41<134:06:32,  5.77s/it]
 63%|██████▎   | 141316/225000 [281:00:50<150:32:05,  6.48s/it]
 63%|██████▎   | 141317/225000 [281:00:53<128:56:25,  5.55s/it]
 63%|██████▎   | 141318/225000 [281:01:00<136:30:26,  5.87s/it]
 63%|██████▎   | 141319/225000 [281:01:06<137:49:17,  5.93s/it]
 63%|██████▎   | 141320/225000 [281:01:09<122:57:14,  5.29s/it]
 63%|██████▎   | 141321/225000 [281:01:13<112:23:40,  4.84s/it]
 63%|██████▎   | 141322/225000 [281:01:20<124:31:30,  5.36s/it]
 63%|██████▎   | 141323/225000 [281:01:27<139:57:38,  6.02s/it]
 63%|██████▎   | 141324/225000 [281:01:33<135:38:55,  5.84s/it]
 63%|██████▎   | 141325/225000 [281:01:38<133:16:18,  5.73s/it]
 63%|██████▎   | 141326/225000 [281:01:43<127:07:59,  5.47s/it]
 63%|██████▎   | 141327/225000 [281:01:48<124:31:06,  5.36s/it]
 63%|██████▎   | 141328/225000 [281:01:54<124:04:09,  5.34s/it]
 63%|██████▎   | 141329/225000 [281:01:59<122:40:00,  5.28s/it]
 63%|██████▎   | 141330/225000 [281:02:04<123:31:28,  5.31s/it]
 63%|██████▎   | 141331/225000 [281:02:10<125:48:36,  5.41s/it]
 63%|██████▎   | 141332/225000 [281:02:15<127:55:04,  5.50s/it]
 63%|██████▎   | 141333/225000 [281:02:28<175:06:06,  7.53s/it]
 63%|██████▎   | 141334/225000 [281:02:32<152:48:21,  6.57s/it]
 63%|██████▎   | 141335/225000 [281:02:36<132:27:46,  5.70s/it]
 63%|██████▎   | 141336/225000 [281:02:45<159:58:58,  6.88s/it]
 63%|██████▎   | 141337/225000 [281:02:49<134:51:03,  5.80s/it]
 63%|██████▎   | 141338/225000 [281:02:57<151:40:49,  6.53s/it]
 63%|██████▎   | 141339/225000 [281:03:00<130:23:15,  5.61s/it]
 63%|██████▎   | 141340/225000 [281:03:07<139:20:48,  6.00s/it]
 63%|██████▎   | 141341/225000 [281:03:14<142:38:12,  6.14s/it]
 63%|██████▎   | 141342/225000 [281:03:19<134:53:30,  5.80s/it]
 63%|██████▎   | 141343/225000 [281:03:25<136:21:17,  5.87s/it]
 63%|██████▎   | 141344/225000 [281:03:29<122:40:31,  5.28s/it]
 63%|██████▎   | 141345/225000 [281:03:33<113:20:10,  4.88s/it]
 63%|██████▎   | 141346/225000 [281:03:44<160:07:37,  6.89s/it]
 63%|██████▎   | 141347/225000 [281:03:49<147:48:33,  6.36s/it]
 63%|██████▎   | 141348/225000 [281:03:54<138:34:39,  5.96s/it]
 63%|██████▎   | 141349/225000 [281:03:59<130:04:57,  5.60s/it]
 63%|██████▎   | 141350/225000 [281:04:04<126:26:30,  5.44s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '0.9979', 'learning_rate': '1.649e-05', 'epoch': '0.6282', 'num_input_tokens_seen': 466014576, 'train_runtime': '1.012e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141350/225000 [281:04:04<126:26:30,  5.44s/it]
 63%|██████▎   | 141351/225000 [281:04:09<121:53:24,  5.25s/it]
 63%|██████▎   | 141352/225000 [281:04:14<123:48:29,  5.33s/it]
 63%|██████▎   | 141353/225000 [281:04:19<120:56:41,  5.21s/it]
 63%|██████▎   | 141354/225000 [281:04:25<122:05:01,  5.25s/it]
 63%|██████▎   | 141355/225000 [281:04:30<121:22:51,  5.22s/it]
 63%|██████▎   | 141356/225000 [281:04:35<121:36:34,  5.23s/it]
 63%|██████▎   | 141357/225000 [281:04:40<121:46:12,  5.24s/it]
 63%|██████▎   | 141358/225000 [281:04:45<120:44:45,  5.20s/it]
 63%|██████▎   | 141359/225000 [281:04:50<116:47:43,  5.03s/it]
 63%|██████▎   | 141360/225000 [281:04:56<119:46:12,  5.16s/it]
 63%|██████▎   | 141361/225000 [281:05:01<121:32:35,  5.23s/it]
 63%|██████▎   | 141362/225000 [281:05:13<166:34:53,  7.17s/it]
 63%|██████▎   | 141363/225000 [281:05:19<161:01:18,  6.93s/it]
 63%|██████▎   | 141364/225000 [281:05:26<159:25:10,  6.86s/it]
 63%|██████▎   | 141365/225000 [281:05:30<141:58:21,  6.11s/it]
 63%|██████▎   | 141366/225000 [281:05:35<131:18:28,  5.65s/it]
 63%|██████▎   | 141367/225000 [281:05:39<121:08:54,  5.21s/it]
 63%|██████▎   | 141368/225000 [281:05:43<114:10:21,  4.91s/it]
 63%|██████▎   | 141369/225000 [281:05:47<110:16:28,  4.75s/it]
 63%|██████▎   | 141370/225000 [281:05:59<159:30:41,  6.87s/it]
 63%|██████▎   | 141371/225000 [281:06:03<135:45:05,  5.84s/it]
 63%|██████▎   | 141372/225000 [281:06:09<138:23:45,  5.96s/it]
 63%|██████▎   | 141373/225000 [281:06:15<141:10:44,  6.08s/it]
 63%|██████▎   | 141374/225000 [281:06:22<144:56:08,  6.24s/it]
 63%|██████▎   | 141375/225000 [281:06:25<126:01:26,  5.43s/it]
 63%|██████▎   | 141376/225000 [281:06:29<112:45:00,  4.85s/it]
 63%|██████▎   | 141377/225000 [281:06:34<113:12:05,  4.87s/it]
 63%|██████▎   | 141378/225000 [281:06:39<115:51:49,  4.99s/it]
 63%|██████▎   | 141379/225000 [281:06:44<116:20:11,  5.01s/it]
 63%|██████▎   | 141380/225000 [281:06:50<119:28:32,  5.14s/it]
 63%|██████▎   | 141381/225000 [281:06:55<122:03:53,  5.26s/it]
 63%|██████▎   | 141382/225000 [281:07:01<123:28:17,  5.32s/it]
 63%|██████▎   | 141383/225000 [281:07:06<124:37:04,  5.37s/it]
 63%|██████▎   | 141384/225000 [281:07:12<125:16:57,  5.39s/it]
 63%|██████▎   | 141385/225000 [281:07:17<124:00:50,  5.34s/it]
 63%|██████▎   | 141386/225000 [281:07:29<169:26:40,  7.30s/it]
 63%|██████▎   | 141387/225000 [281:07:34<155:13:05,  6.68s/it]
 63%|██████▎   | 141388/225000 [281:07:38<134:24:52,  5.79s/it]
 63%|██████▎   | 141389/225000 [281:07:42<125:47:51,  5.42s/it]
 63%|██████▎   | 141390/225000 [281:07:54<170:54:06,  7.36s/it]
 63%|██████▎   | 141391/225000 [281:07:58<145:30:00,  6.26s/it]
 63%|██████▎   | 141392/225000 [281:08:02<128:46:17,  5.54s/it]
 63%|██████▎   | 141393/225000 [281:08:11<156:29:58,  6.74s/it]
 63%|██████▎   | 141394/225000 [281:08:15<139:15:51,  6.00s/it]
 63%|██████▎   | 141395/225000 [281:08:20<128:21:04,  5.53s/it]
 63%|██████▎   | 141396/225000 [281:08:25<125:12:47,  5.39s/it]
 63%|██████▎   | 141397/225000 [281:08:29<119:04:39,  5.13s/it]
 63%|██████▎   | 141398/225000 [281:08:34<113:55:58,  4.91s/it]
 63%|██████▎   | 141399/225000 [281:08:39<113:48:02,  4.90s/it]
 63%|██████▎   | 141400/225000 [281:08:50<155:18:47,  6.69s/it]
                                                               
{'loss': '0.8016', 'grad_norm': '1.108', 'learning_rate': '1.647e-05', 'epoch': '0.6284', 'num_input_tokens_seen': 466143920, 'train_runtime': '1.012e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141400/225000 [281:08:50<155:18:47,  6.69s/it]
 63%|██████▎   | 141401/225000 [281:08:57<162:03:18,  6.98s/it]
 63%|██████▎   | 141402/225000 [281:09:01<140:54:44,  6.07s/it]
 63%|██████▎   | 141403/225000 [281:09:05<128:27:13,  5.53s/it]
 63%|██████▎   | 141404/225000 [281:09:10<118:16:13,  5.09s/it]
 63%|██████▎   | 141405/225000 [281:09:16<126:49:48,  5.46s/it]
 63%|██████▎   | 141406/225000 [281:09:26<156:37:31,  6.75s/it]
 63%|██████▎   | 141407/225000 [281:09:32<153:38:01,  6.62s/it]
 63%|██████▎   | 141408/225000 [281:09:36<133:38:37,  5.76s/it]
 63%|██████▎   | 141409/225000 [281:09:43<142:33:27,  6.14s/it]
 63%|██████▎   | 141410/225000 [281:09:46<123:17:34,  5.31s/it]
 63%|██████▎   | 141411/225000 [281:09:54<141:22:58,  6.09s/it]
 63%|██████▎   | 141412/225000 [281:09:59<135:49:27,  5.85s/it]
 63%|██████▎   | 141413/225000 [281:10:04<125:06:24,  5.39s/it]
 63%|██████▎   | 141414/225000 [281:10:08<118:36:39,  5.11s/it]
 63%|██████▎   | 141415/225000 [281:10:13<115:04:16,  4.96s/it]
 63%|██████▎   | 141416/225000 [281:10:17<111:07:47,  4.79s/it]
 63%|██████▎   | 141417/225000 [281:10:29<164:45:37,  7.10s/it]
 63%|██████▎   | 141418/225000 [281:10:34<147:39:50,  6.36s/it]
 63%|██████▎   | 141419/225000 [281:10:39<139:24:51,  6.00s/it]
 63%|██████▎   | 141420/225000 [281:10:44<130:45:54,  5.63s/it]
 63%|██████▎   | 141421/225000 [281:10:49<127:43:54,  5.50s/it]
 63%|██████▎   | 141422/225000 [281:10:55<127:07:37,  5.48s/it]
 63%|██████▎   | 141423/225000 [281:11:00<123:11:05,  5.31s/it]
 63%|██████▎   | 141424/225000 [281:11:04<119:10:44,  5.13s/it]
 63%|██████▎   | 141425/225000 [281:11:09<119:11:33,  5.13s/it]
 63%|██████▎   | 141426/225000 [281:11:14<116:02:33,  5.00s/it]
 63%|██████▎   | 141427/225000 [281:11:19<115:47:25,  4.99s/it]
 63%|██████▎   | 141428/225000 [281:11:24<117:37:17,  5.07s/it]
 63%|██████▎   | 141429/225000 [281:11:31<131:42:16,  5.67s/it]
 63%|██████▎   | 141430/225000 [281:11:43<170:03:47,  7.33s/it]
 63%|██████▎   | 141431/225000 [281:11:46<140:48:07,  6.07s/it]
 63%|██████▎   | 141432/225000 [281:11:51<133:14:49,  5.74s/it]
 63%|██████▎   | 141433/225000 [281:11:56<130:06:19,  5.60s/it]
 63%|██████▎   | 141434/225000 [281:12:01<127:21:50,  5.49s/it]
 63%|██████▎   | 141435/225000 [281:12:06<119:46:56,  5.16s/it]
 63%|██████▎   | 141436/225000 [281:12:11<119:31:32,  5.15s/it]
 63%|██████▎   | 141437/225000 [281:12:16<120:30:09,  5.19s/it]
 63%|██████▎   | 141438/225000 [281:12:29<172:08:03,  7.42s/it]
 63%|██████▎   | 141439/225000 [281:12:34<158:13:46,  6.82s/it]
 63%|██████▎   | 141440/225000 [281:12:37<131:57:16,  5.68s/it]
 63%|██████▎   | 141441/225000 [281:12:46<156:16:37,  6.73s/it]
 63%|██████▎   | 141442/225000 [281:12:50<134:31:50,  5.80s/it]
 63%|██████▎   | 141443/225000 [281:12:59<156:35:15,  6.75s/it]
 63%|██████▎   | 141444/225000 [281:13:02<132:45:32,  5.72s/it]
 63%|██████▎   | 141445/225000 [281:13:10<149:37:23,  6.45s/it]
 63%|██████▎   | 141446/225000 [281:13:14<128:36:29,  5.54s/it]
 63%|██████▎   | 141447/225000 [281:13:17<113:49:47,  4.90s/it]
 63%|██████▎   | 141448/225000 [281:13:26<137:56:30,  5.94s/it]
 63%|██████▎   | 141449/225000 [281:13:33<146:01:41,  6.29s/it]
 63%|██████▎   | 141450/225000 [281:13:39<148:53:05,  6.42s/it]
                                                               
{'loss': '0.7759', 'grad_norm': '1.051', 'learning_rate': '1.645e-05', 'epoch': '0.6287', 'num_input_tokens_seen': 466272816, 'train_runtime': '1.012e+06', 'train_tokens_per_second': '460.6'}

 63%|██████▎   | 141450/225000 [281:13:39<148:53:05,  6.42s/it]
 63%|██████▎   | 141451/225000 [281:13:44<134:20:24,  5.79s/it]
 63%|██████▎   | 141452/225000 [281:13:48<122:39:29,  5.29s/it]
 63%|██████▎   | 141453/225000 [281:13:52<116:37:48,  5.03s/it]
 63%|██████▎   | 141454/225000 [281:13:57<114:53:47,  4.95s/it]
 63%|██████▎   | 141455/225000 [281:14:04<131:15:08,  5.66s/it]
 63%|██████▎   | 141456/225000 [281:14:14<161:15:57,  6.95s/it]
 63%|██████▎   | 141457/225000 [281:14:19<147:45:52,  6.37s/it]
 63%|██████▎   | 141458/225000 [281:14:24<133:33:31,  5.76s/it]
 63%|██████▎   | 141459/225000 [281:14:28<126:29:10,  5.45s/it]
 63%|██████▎   | 141460/225000 [281:14:33<120:31:54,  5.19s/it]
 63%|██████▎   | 141461/225000 [281:14:38<116:58:31,  5.04s/it]
 63%|██████▎   | 141462/225000 [281:14:42<114:08:27,  4.92s/it]
 63%|██████▎   | 141463/225000 [281:14:48<117:27:06,  5.06s/it]
 63%|██████▎   | 141464/225000 [281:14:53<117:27:33,  5.06s/it]
 63%|██████▎   | 141465/225000 [281:14:58<118:36:33,  5.11s/it]
 63%|██████▎   | 141466/225000 [281:15:09<160:48:43,  6.93s/it]
 63%|██████▎   | 141467/225000 [281:15:17<165:02:01,  7.11s/it]
 63%|██████▎   | 141468/225000 [281:15:20<139:27:08,  6.01s/it]
 63%|██████▎   | 141469/225000 [281:15:27<149:06:29,  6.43s/it]
 63%|██████▎   | 141470/225000 [281:15:34<147:41:23,  6.37s/it]
 63%|██████▎   | 141471/225000 [281:15:37<129:23:09,  5.58s/it]
 63%|██████▎   | 141472/225000 [281:15:46<151:05:45,  6.51s/it]
 63%|██████▎   | 141473/225000 [281:15:52<145:26:52,  6.27s/it]
 63%|██████▎   | 141474/225000 [281:15:56<128:14:24,  5.53s/it]
 63%|██████▎   | 141475/225000 [281:16:00<117:13:49,  5.05s/it]
 63%|██████▎   | 141476/225000 [281:16:07<136:00:57,  5.86s/it]
 63%|██████▎   | 141477/225000 [281:16:13<134:38:04,  5.80s/it]
 63%|██████▎   | 141478/225000 [281:16:18<127:44:12,  5.51s/it]
 63%|██████▎   | 141479/225000 [281:16:23<123:36:41,  5.33s/it]
 63%|██████▎   | 141480/225000 [281:16:28<120:18:30,  5.19s/it]
 63%|██████▎   | 141481/225000 [281:16:32<118:10:10,  5.09s/it]
 63%|██████▎   | 141482/225000 [281:16:37<114:45:22,  4.95s/it]
 63%|██████▎   | 141483/225000 [281:16:50<169:41:33,  7.31s/it]
 63%|██████▎   | 141484/225000 [281:16:54<146:59:39,  6.34s/it]
 63%|██████▎   | 141485/225000 [281:17:02<157:42:32,  6.80s/it]
 63%|██████▎   | 141486/225000 [281:17:08<150:23:48,  6.48s/it]
 63%|██████▎   | 141487/225000 [281:17:11<128:50:25,  5.55s/it]
 63%|██████▎   | 141488/225000 [281:17:21<159:24:17,  6.87s/it]
 63%|██████▎   | 141489/225000 [281:17:24<133:40:57,  5.76s/it]
 63%|██████▎   | 141490/225000 [281:17:32<147:18:50,  6.35s/it]
 63%|██████▎   | 141491/225000 [281:17:35<128:28:23,  5.54s/it]
 63%|██████▎   | 141492/225000 [281:17:39<118:05:04,  5.09s/it]
 63%|██████▎   | 141493/225000 [281:17:44<115:58:41,  5.00s/it]
 63%|██████▎   | 141494/225000 [281:17:50<117:53:52,  5.08s/it]
 63%|██████▎   | 141495/225000 [281:17:55<117:57:56,  5.09s/it]
 63%|██████▎   | 141496/225000 [281:18:00<117:41:38,  5.07s/it]
 63%|██████▎   | 141497/225000 [281:18:05<118:58:14,  5.13s/it]
 63%|██████▎   | 141498/225000 [281:18:13<136:50:36,  5.90s/it]
 63%|██████▎   | 141499/225000 [281:18:22<160:43:58,  6.93s/it]
 63%|██████▎   | 141500/225000 [281:18:30<166:34:02,  7.18s/it]
                                                               
{'loss': '0.7913', 'grad_norm': '1.19', 'learning_rate': '1.643e-05', 'epoch': '0.6289', 'num_input_tokens_seen': 466403096, 'train_runtime': '1.013e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141500/225000 [281:18:30<166:34:02,  7.18s/it]
 63%|██████▎   | 141501/225000 [281:18:33<140:59:46,  6.08s/it]
 63%|██████▎   | 141502/225000 [281:18:36<120:17:04,  5.19s/it]
 63%|██████▎   | 141503/225000 [281:18:45<142:56:46,  6.16s/it]
 63%|██████▎   | 141504/225000 [281:18:51<139:56:53,  6.03s/it]
 63%|██████▎   | 141505/225000 [281:18:54<124:48:03,  5.38s/it]
 63%|██████▎   | 141506/225000 [281:19:02<140:39:36,  6.06s/it]
 63%|██████▎   | 141507/225000 [281:19:08<141:17:31,  6.09s/it]
 63%|██████▎   | 141508/225000 [281:19:12<123:44:43,  5.34s/it]
 63%|██████▎   | 141509/225000 [281:19:15<112:24:41,  4.85s/it]
 63%|██████▎   | 141510/225000 [281:19:25<145:36:44,  6.28s/it]
 63%|██████▎   | 141511/225000 [281:19:30<137:32:38,  5.93s/it]
 63%|██████▎   | 141512/225000 [281:19:34<124:28:59,  5.37s/it]
 63%|██████▎   | 141513/225000 [281:19:38<114:45:21,  4.95s/it]
 63%|██████▎   | 141514/225000 [281:19:43<112:17:12,  4.84s/it]
 63%|██████▎   | 141515/225000 [281:19:53<152:45:48,  6.59s/it]
 63%|██████▎   | 141516/225000 [281:20:00<153:33:02,  6.62s/it]
 63%|██████▎   | 141517/225000 [281:20:05<141:20:41,  6.10s/it]
 63%|██████▎   | 141518/225000 [281:20:11<138:47:51,  5.99s/it]
 63%|██████▎   | 141519/225000 [281:20:19<153:36:18,  6.62s/it]
 63%|██████▎   | 141520/225000 [281:20:26<153:27:18,  6.62s/it]
 63%|██████▎   | 141521/225000 [281:20:30<138:22:02,  5.97s/it]
 63%|██████▎   | 141522/225000 [281:20:35<129:14:56,  5.57s/it]
 63%|██████▎   | 141523/225000 [281:20:39<121:57:33,  5.26s/it]
 63%|██████▎   | 141524/225000 [281:20:43<115:06:16,  4.96s/it]
 63%|██████▎   | 141525/225000 [281:20:48<111:54:39,  4.83s/it]
 63%|██████▎   | 141526/225000 [281:20:58<145:25:14,  6.27s/it]
 63%|██████▎   | 141527/225000 [281:21:05<156:40:13,  6.76s/it]
 63%|██████▎   | 141528/225000 [281:21:11<148:00:09,  6.38s/it]
 63%|██████▎   | 141529/225000 [281:21:14<127:45:23,  5.51s/it]
 63%|██████▎   | 141530/225000 [281:21:24<153:26:38,  6.62s/it]
 63%|██████▎   | 141531/225000 [281:21:28<134:45:39,  5.81s/it]
 63%|██████▎   | 141532/225000 [281:21:31<118:03:50,  5.09s/it]
 63%|██████▎   | 141533/225000 [281:21:41<151:49:25,  6.55s/it]
 63%|██████▎   | 141534/225000 [281:21:45<137:41:14,  5.94s/it]
 63%|██████▎   | 141535/225000 [281:21:53<147:34:05,  6.36s/it]
 63%|██████▎   | 141536/225000 [281:21:56<127:47:39,  5.51s/it]
 63%|██████▎   | 141537/225000 [281:22:00<116:40:06,  5.03s/it]
 63%|██████▎   | 141538/225000 [281:22:11<158:29:12,  6.84s/it]
 63%|██████▎   | 141539/225000 [281:22:16<140:37:56,  6.07s/it]
 63%|██████▎   | 141540/225000 [281:22:20<126:53:44,  5.47s/it]
 63%|██████▎   | 141541/225000 [281:22:24<121:18:15,  5.23s/it]
 63%|██████▎   | 141542/225000 [281:22:28<112:20:05,  4.85s/it]
 63%|██████▎   | 141543/225000 [281:22:37<141:33:56,  6.11s/it]
 63%|██████▎   | 141544/225000 [281:22:44<145:53:19,  6.29s/it]
 63%|██████▎   | 141545/225000 [281:22:49<137:43:26,  5.94s/it]
 63%|██████▎   | 141546/225000 [281:22:54<129:02:02,  5.57s/it]
 63%|██████▎   | 141547/225000 [281:22:59<124:00:40,  5.35s/it]
 63%|██████▎   | 141548/225000 [281:23:04<121:17:18,  5.23s/it]
 63%|██████▎   | 141549/225000 [281:23:09<119:09:30,  5.14s/it]
 63%|██████▎   | 141550/225000 [281:23:14<119:13:53,  5.14s/it]
                                                               
{'loss': '0.7934', 'grad_norm': '1.087', 'learning_rate': '1.642e-05', 'epoch': '0.6291', 'num_input_tokens_seen': 466533608, 'train_runtime': '1.013e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141550/225000 [281:23:14<119:13:53,  5.14s/it]
 63%|██████▎   | 141551/225000 [281:23:19<117:58:00,  5.09s/it]
 63%|██████▎   | 141552/225000 [281:23:31<166:12:34,  7.17s/it]
 63%|██████▎   | 141553/225000 [281:23:37<160:19:41,  6.92s/it]
 63%|██████▎   | 141554/225000 [281:23:42<143:56:46,  6.21s/it]
 63%|██████▎   | 141555/225000 [281:23:48<142:14:19,  6.14s/it]
 63%|██████▎   | 141556/225000 [281:23:52<130:26:42,  5.63s/it]
 63%|██████▎   | 141557/225000 [281:23:56<122:20:26,  5.28s/it]
 63%|██████▎   | 141558/225000 [281:24:01<117:47:15,  5.08s/it]
 63%|██████▎   | 141559/225000 [281:24:06<113:41:55,  4.91s/it]
 63%|██████▎   | 141560/225000 [281:24:11<117:29:15,  5.07s/it]
 63%|██████▎   | 141561/225000 [281:24:23<165:29:37,  7.14s/it]
 63%|██████▎   | 141562/225000 [281:24:27<142:37:49,  6.15s/it]
 63%|██████▎   | 141563/225000 [281:24:31<126:32:09,  5.46s/it]
 63%|██████▎   | 141564/225000 [281:24:36<125:00:34,  5.39s/it]
 63%|██████▎   | 141565/225000 [281:24:41<119:37:51,  5.16s/it]
 63%|██████▎   | 141566/225000 [281:24:46<120:38:41,  5.21s/it]
 63%|██████▎   | 141567/225000 [281:24:51<117:33:18,  5.07s/it]
 63%|██████▎   | 141568/225000 [281:24:55<114:38:47,  4.95s/it]
 63%|██████▎   | 141569/225000 [281:25:00<112:31:27,  4.86s/it]
 63%|██████▎   | 141570/225000 [281:25:05<116:22:08,  5.02s/it]
 63%|██████▎   | 141571/225000 [281:25:10<117:00:54,  5.05s/it]
 63%|██████▎   | 141572/225000 [281:25:16<119:22:27,  5.15s/it]
 63%|██████▎   | 141573/225000 [281:25:23<133:53:38,  5.78s/it]
 63%|██████▎   | 141574/225000 [281:25:28<127:00:11,  5.48s/it]
 63%|██████▎   | 141575/225000 [281:25:33<126:45:52,  5.47s/it]
 63%|██████▎   | 141576/225000 [281:25:38<122:37:10,  5.29s/it]
 63%|██████▎   | 141577/225000 [281:25:43<119:04:30,  5.14s/it]
 63%|██████▎   | 141578/225000 [281:25:52<146:51:19,  6.34s/it]
 63%|██████▎   | 141579/225000 [281:26:02<168:43:14,  7.28s/it]
 63%|██████▎   | 141580/225000 [281:26:05<143:40:35,  6.20s/it]
 63%|██████▎   | 141581/225000 [281:26:09<126:46:51,  5.47s/it]
 63%|██████▎   | 141582/225000 [281:26:18<151:22:14,  6.53s/it]
 63%|██████▎   | 141583/225000 [281:26:24<147:16:27,  6.36s/it]
 63%|██████▎   | 141584/225000 [281:26:31<150:00:57,  6.47s/it]
 63%|██████▎   | 141585/225000 [281:26:36<138:12:13,  5.96s/it]
 63%|██████▎   | 141586/225000 [281:26:43<150:12:05,  6.48s/it]
 63%|██████▎   | 141587/225000 [281:26:47<133:00:54,  5.74s/it]
 63%|██████▎   | 141588/225000 [281:26:51<120:29:20,  5.20s/it]
 63%|██████▎   | 141589/225000 [281:26:55<112:32:55,  4.86s/it]
 63%|██████▎   | 141590/225000 [281:26:59<108:30:46,  4.68s/it]
 63%|██████▎   | 141591/225000 [281:27:07<126:22:19,  5.45s/it]
 63%|██████▎   | 141592/225000 [281:27:17<158:57:31,  6.86s/it]
 63%|██████▎   | 141593/225000 [281:27:20<136:04:52,  5.87s/it]
 63%|██████▎   | 141594/225000 [281:27:26<134:28:11,  5.80s/it]
 63%|██████▎   | 141595/225000 [281:27:32<136:55:29,  5.91s/it]
 63%|██████▎   | 141596/225000 [281:27:38<137:18:13,  5.93s/it]
 63%|██████▎   | 141597/225000 [281:27:45<141:04:30,  6.09s/it]
 63%|██████▎   | 141598/225000 [281:27:51<145:21:42,  6.27s/it]
 63%|██████▎   | 141599/225000 [281:27:55<129:05:54,  5.57s/it]
 63%|██████▎   | 141600/225000 [281:27:59<117:14:57,  5.06s/it]
                                                               
{'loss': '0.7939', 'grad_norm': '1.102', 'learning_rate': '1.64e-05', 'epoch': '0.6293', 'num_input_tokens_seen': 466662712, 'train_runtime': '1.013e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141600/225000 [281:27:59<117:14:57,  5.06s/it]
 63%|██████▎   | 141601/225000 [281:28:03<111:35:33,  4.82s/it]
 63%|██████▎   | 141602/225000 [281:28:10<123:03:37,  5.31s/it]
 63%|██████▎   | 141603/225000 [281:28:18<143:42:20,  6.20s/it]
 63%|██████▎   | 141604/225000 [281:28:22<125:15:09,  5.41s/it]
 63%|██████▎   | 141605/225000 [281:28:30<145:59:41,  6.30s/it]
 63%|██████▎   | 141606/225000 [281:28:36<142:03:07,  6.13s/it]
 63%|██████▎   | 141607/225000 [281:28:41<132:12:55,  5.71s/it]
 63%|██████▎   | 141608/225000 [281:28:45<124:13:54,  5.36s/it]
 63%|██████▎   | 141609/225000 [281:28:50<119:10:12,  5.14s/it]
 63%|██████▎   | 141610/225000 [281:28:55<117:31:16,  5.07s/it]
 63%|██████▎   | 141611/225000 [281:28:59<113:42:10,  4.91s/it]
 63%|██████▎   | 141612/225000 [281:29:04<115:57:42,  5.01s/it]
 63%|██████▎   | 141613/225000 [281:29:12<134:11:17,  5.79s/it]
 63%|██████▎   | 141614/225000 [281:29:24<173:53:40,  7.51s/it]
 63%|██████▎   | 141615/225000 [281:29:27<145:36:49,  6.29s/it]
 63%|██████▎   | 141616/225000 [281:29:31<126:08:14,  5.45s/it]
 63%|██████▎   | 141617/225000 [281:29:38<143:04:18,  6.18s/it]
 63%|██████▎   | 141618/225000 [281:29:43<128:55:40,  5.57s/it]
 63%|██████▎   | 141619/225000 [281:29:47<120:10:45,  5.19s/it]
 63%|██████▎   | 141620/225000 [281:29:51<113:26:33,  4.90s/it]
 63%|██████▎   | 141621/225000 [281:29:57<121:57:19,  5.27s/it]
 63%|██████▎   | 141622/225000 [281:30:09<164:42:50,  7.11s/it]
 63%|██████▎   | 141623/225000 [281:30:12<137:10:54,  5.92s/it]
 63%|██████▎   | 141624/225000 [281:30:15<120:25:49,  5.20s/it]
 63%|██████▎   | 141625/225000 [281:30:25<149:33:32,  6.46s/it]
 63%|██████▎   | 141626/225000 [281:30:28<127:59:20,  5.53s/it]
 63%|██████▎   | 141627/225000 [281:30:34<132:22:58,  5.72s/it]
 63%|██████▎   | 141628/225000 [281:30:41<142:13:00,  6.14s/it]
 63%|██████▎   | 141629/225000 [281:30:45<128:19:10,  5.54s/it]
 63%|██████▎   | 141630/225000 [281:30:50<121:01:46,  5.23s/it]
 63%|██████▎   | 141631/225000 [281:30:55<116:39:22,  5.04s/it]
 63%|██████▎   | 141632/225000 [281:31:01<123:26:58,  5.33s/it]
 63%|██████▎   | 141633/225000 [281:31:10<153:21:00,  6.62s/it]
 63%|██████▎   | 141634/225000 [281:31:14<133:20:29,  5.76s/it]
 63%|██████▎   | 141635/225000 [281:31:18<118:14:09,  5.11s/it]
 63%|██████▎   | 141636/225000 [281:31:28<154:45:06,  6.68s/it]
 63%|██████▎   | 141637/225000 [281:31:32<139:01:28,  6.00s/it]
 63%|██████▎   | 141638/225000 [281:31:39<145:52:13,  6.30s/it]
 63%|██████▎   | 141639/225000 [281:31:44<134:31:54,  5.81s/it]
 63%|██████▎   | 141640/225000 [281:31:48<123:13:16,  5.32s/it]
 63%|██████▎   | 141641/225000 [281:31:53<116:44:32,  5.04s/it]
 63%|██████▎   | 141642/225000 [281:31:57<112:31:40,  4.86s/it]
 63%|██████▎   | 141643/225000 [281:32:01<109:17:40,  4.72s/it]
 63%|██████▎   | 141644/225000 [281:32:07<115:57:19,  5.01s/it]
 63%|██████▎   | 141645/225000 [281:32:12<115:45:48,  5.00s/it]
 63%|██████▎   | 141646/225000 [281:32:17<114:33:15,  4.95s/it]
 63%|██████▎   | 141647/225000 [281:32:22<116:18:07,  5.02s/it]
 63%|██████▎   | 141648/225000 [281:32:34<165:33:20,  7.15s/it]
 63%|██████▎   | 141649/225000 [281:32:40<158:17:53,  6.84s/it]
 63%|██████▎   | 141650/225000 [281:32:45<145:53:09,  6.30s/it]
                                                               
{'loss': '0.7945', 'grad_norm': '1.06', 'learning_rate': '1.638e-05', 'epoch': '0.6296', 'num_input_tokens_seen': 466792184, 'train_runtime': '1.014e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141650/225000 [281:32:45<145:53:09,  6.30s/it]
 63%|██████▎   | 141651/225000 [281:32:52<146:41:47,  6.34s/it]
 63%|██████▎   | 141652/225000 [281:32:57<137:12:00,  5.93s/it]
 63%|██████▎   | 141653/225000 [281:33:04<145:28:25,  6.28s/it]
 63%|██████▎   | 141654/225000 [281:33:08<130:10:37,  5.62s/it]
 63%|██████▎   | 141655/225000 [281:33:12<117:53:36,  5.09s/it]
 63%|██████▎   | 141656/225000 [281:33:17<119:53:40,  5.18s/it]
 63%|██████▎   | 141657/225000 [281:33:26<144:59:41,  6.26s/it]
 63%|██████▎   | 141658/225000 [281:33:32<144:02:08,  6.22s/it]
 63%|██████▎   | 141659/225000 [281:33:37<132:11:21,  5.71s/it]
 63%|██████▎   | 141660/225000 [281:33:41<121:39:55,  5.26s/it]
 63%|██████▎   | 141661/225000 [281:33:46<119:48:32,  5.18s/it]
 63%|██████▎   | 141662/225000 [281:33:50<113:16:37,  4.89s/it]
 63%|██████▎   | 141663/225000 [281:33:55<114:16:27,  4.94s/it]
 63%|██████▎   | 141664/225000 [281:34:00<115:21:14,  4.98s/it]
 63%|██████▎   | 141665/225000 [281:34:06<119:02:45,  5.14s/it]
 63%|██████▎   | 141666/225000 [281:34:18<169:32:07,  7.32s/it]
 63%|██████▎   | 141667/225000 [281:34:22<143:54:53,  6.22s/it]
 63%|██████▎   | 141668/225000 [281:34:28<143:50:17,  6.21s/it]
 63%|██████▎   | 141669/225000 [281:34:35<148:47:36,  6.43s/it]
 63%|██████▎   | 141670/225000 [281:34:38<128:09:01,  5.54s/it]
 63%|██████▎   | 141671/225000 [281:34:47<149:45:03,  6.47s/it]
 63%|██████▎   | 141672/225000 [281:34:52<137:48:04,  5.95s/it]
 63%|██████▎   | 141673/225000 [281:34:56<125:38:35,  5.43s/it]
 63%|██████▎   | 141674/225000 [281:35:00<117:52:09,  5.09s/it]
 63%|██████▎   | 141675/225000 [281:35:06<119:46:12,  5.17s/it]
 63%|██████▎   | 141676/225000 [281:35:15<151:19:32,  6.54s/it]
 63%|██████▎   | 141677/225000 [281:35:19<133:33:48,  5.77s/it]
 63%|██████▎   | 141678/225000 [281:35:29<159:00:11,  6.87s/it]
 63%|██████▎   | 141679/225000 [281:35:32<136:47:10,  5.91s/it]
 63%|██████▎   | 141680/225000 [281:35:41<153:07:13,  6.62s/it]
 63%|██████▎   | 141681/225000 [281:35:45<138:31:36,  5.99s/it]
 63%|██████▎   | 141682/225000 [281:35:49<120:47:30,  5.22s/it]
 63%|██████▎   | 141683/225000 [281:35:56<134:08:11,  5.80s/it]
 63%|██████▎   | 141684/225000 [281:35:59<118:39:14,  5.13s/it]
 63%|██████▎   | 141685/225000 [281:36:09<153:31:00,  6.63s/it]
 63%|██████▎   | 141686/225000 [281:36:13<135:21:18,  5.85s/it]
 63%|██████▎   | 141687/225000 [281:36:17<120:29:54,  5.21s/it]
 63%|██████▎   | 141688/225000 [281:36:22<115:57:43,  5.01s/it]
 63%|██████▎   | 141689/225000 [281:36:26<113:27:39,  4.90s/it]
 63%|██████▎   | 141690/225000 [281:36:38<158:24:19,  6.85s/it]
 63%|██████▎   | 141691/225000 [281:36:41<134:54:22,  5.83s/it]
 63%|██████▎   | 141692/225000 [281:36:45<123:37:58,  5.34s/it]
 63%|██████▎   | 141693/225000 [281:36:50<117:33:20,  5.08s/it]
 63%|██████▎   | 141694/225000 [281:36:54<112:47:36,  4.87s/it]
 63%|██████▎   | 141695/225000 [281:37:07<164:48:49,  7.12s/it]
 63%|██████▎   | 141696/225000 [281:37:10<138:12:50,  5.97s/it]
 63%|██████▎   | 141697/225000 [281:37:18<150:18:37,  6.50s/it]
 63%|██████▎   | 141698/225000 [281:37:22<133:15:06,  5.76s/it]
 63%|██████▎   | 141699/225000 [281:37:27<129:24:43,  5.59s/it]
 63%|██████▎   | 141700/225000 [281:37:32<124:32:19,  5.38s/it]
                                                               
{'loss': '0.7984', 'grad_norm': '1.091', 'learning_rate': '1.637e-05', 'epoch': '0.6298', 'num_input_tokens_seen': 466922288, 'train_runtime': '1.014e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141700/225000 [281:37:32<124:32:19,  5.38s/it]
 63%|██████▎   | 141701/225000 [281:37:37<122:46:41,  5.31s/it]
 63%|██████▎   | 141702/225000 [281:37:42<122:33:35,  5.30s/it]
 63%|██████▎   | 141703/225000 [281:37:47<122:13:54,  5.28s/it]
 63%|██████▎   | 141704/225000 [281:37:52<117:16:18,  5.07s/it]
 63%|██████▎   | 141705/225000 [281:37:57<115:17:32,  4.98s/it]
 63%|██████▎   | 141706/225000 [281:38:02<114:12:12,  4.94s/it]
 63%|██████▎   | 141707/225000 [281:38:07<117:41:50,  5.09s/it]
 63%|██████▎   | 141708/225000 [281:38:12<117:38:51,  5.08s/it]
 63%|██████▎   | 141709/225000 [281:38:24<163:07:36,  7.05s/it]
 63%|██████▎   | 141710/225000 [281:38:30<158:16:26,  6.84s/it]
 63%|██████▎   | 141711/225000 [281:38:34<138:13:08,  5.97s/it]
 63%|██████▎   | 141712/225000 [281:38:38<125:46:12,  5.44s/it]
 63%|██████▎   | 141713/225000 [281:38:50<172:03:31,  7.44s/it]
 63%|██████▎   | 141714/225000 [281:38:54<147:58:39,  6.40s/it]
 63%|██████▎   | 141715/225000 [281:38:58<128:47:53,  5.57s/it]
 63%|██████▎   | 141716/225000 [281:39:02<121:27:36,  5.25s/it]
 63%|██████▎   | 141717/225000 [281:39:07<116:17:26,  5.03s/it]
 63%|██████▎   | 141718/225000 [281:39:18<155:45:39,  6.73s/it]
 63%|██████▎   | 141719/225000 [281:39:21<134:35:53,  5.82s/it]
 63%|██████▎   | 141720/225000 [281:39:30<151:34:49,  6.55s/it]
 63%|██████▎   | 141721/225000 [281:39:34<134:20:42,  5.81s/it]
 63%|██████▎   | 141722/225000 [281:39:37<120:36:55,  5.21s/it]
 63%|██████▎   | 141723/225000 [281:39:41<111:14:40,  4.81s/it]
 63%|██████▎   | 141724/225000 [281:39:48<127:04:34,  5.49s/it]
 63%|██████▎   | 141725/225000 [281:39:57<147:28:40,  6.38s/it]
 63%|██████▎   | 141726/225000 [281:40:01<129:36:19,  5.60s/it]
 63%|██████▎   | 141727/225000 [281:40:10<152:20:50,  6.59s/it]
 63%|██████▎   | 141728/225000 [281:40:16<150:19:23,  6.50s/it]
 63%|██████▎   | 141729/225000 [281:40:20<132:13:11,  5.72s/it]
 63%|██████▎   | 141730/225000 [281:40:24<122:44:02,  5.31s/it]
 63%|██████▎   | 141731/225000 [281:40:34<156:46:21,  6.78s/it]
 63%|██████▎   | 141732/225000 [281:40:38<133:17:35,  5.76s/it]
 63%|██████▎   | 141733/225000 [281:40:45<144:29:22,  6.25s/it]
 63%|██████▎   | 141734/225000 [281:40:49<126:02:46,  5.45s/it]
 63%|██████▎   | 141735/225000 [281:40:57<142:58:44,  6.18s/it]
 63%|██████▎   | 141736/225000 [281:41:00<123:25:21,  5.34s/it]
 63%|██████▎   | 141737/225000 [281:41:08<143:52:23,  6.22s/it]
 63%|██████▎   | 141738/225000 [281:41:12<125:07:29,  5.41s/it]
 63%|██████▎   | 141739/225000 [281:41:16<116:55:28,  5.06s/it]
 63%|██████▎   | 141740/225000 [281:41:20<112:00:30,  4.84s/it]
 63%|██████▎   | 141741/225000 [281:41:25<110:17:31,  4.77s/it]
 63%|██████▎   | 141742/225000 [281:41:37<161:26:04,  6.98s/it]
 63%|██████▎   | 141743/225000 [281:41:43<154:19:21,  6.67s/it]
 63%|██████▎   | 141744/225000 [281:41:48<141:59:56,  6.14s/it]
 63%|██████▎   | 141745/225000 [281:41:53<134:05:32,  5.80s/it]
 63%|██████▎   | 141746/225000 [281:41:57<122:55:57,  5.32s/it]
 63%|██████▎   | 141747/225000 [281:42:02<121:34:35,  5.26s/it]
 63%|██████▎   | 141748/225000 [281:42:07<120:07:56,  5.19s/it]
 63%|██████▎   | 141749/225000 [281:42:12<119:36:03,  5.17s/it]
 63%|██████▎   | 141750/225000 [281:42:17<119:17:10,  5.16s/it]
                                                               
{'loss': '0.7864', 'grad_norm': '1.137', 'learning_rate': '1.635e-05', 'epoch': '0.63', 'num_input_tokens_seen': 467052688, 'train_runtime': '1.014e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141750/225000 [281:42:18<119:17:10,  5.16s/it]
 63%|██████▎   | 141751/225000 [281:42:22<116:59:12,  5.06s/it]
 63%|██████▎   | 141752/225000 [281:42:27<113:23:59,  4.90s/it]
 63%|██████▎   | 141753/225000 [281:42:33<125:09:24,  5.41s/it]
 63%|██████▎   | 141754/225000 [281:42:47<179:22:36,  7.76s/it]
 63%|██████▎   | 141755/225000 [281:42:51<153:46:41,  6.65s/it]
 63%|██████▎   | 141756/225000 [281:42:58<156:00:31,  6.75s/it]
 63%|██████▎   | 141757/225000 [281:43:04<155:17:53,  6.72s/it]
 63%|██████▎   | 141758/225000 [281:43:08<133:09:56,  5.76s/it]
 63%|██████▎   | 141759/225000 [281:43:14<132:01:54,  5.71s/it]
 63%|██████▎   | 141760/225000 [281:43:21<142:52:36,  6.18s/it]
 63%|██████▎   | 141761/225000 [281:43:26<134:48:57,  5.83s/it]
 63%|██████▎   | 141762/225000 [281:43:31<128:07:53,  5.54s/it]
 63%|██████▎   | 141763/225000 [281:43:35<117:49:26,  5.10s/it]
 63%|██████▎   | 141764/225000 [281:43:39<111:10:51,  4.81s/it]
 63%|██████▎   | 141765/225000 [281:43:44<111:29:47,  4.82s/it]
 63%|██████▎   | 141766/225000 [281:43:56<166:40:21,  7.21s/it]
 63%|██████▎   | 141767/225000 [281:44:00<142:04:27,  6.15s/it]
 63%|██████▎   | 141768/225000 [281:44:04<124:46:25,  5.40s/it]
 63%|██████▎   | 141769/225000 [281:44:13<153:44:07,  6.65s/it]
 63%|██████▎   | 141770/225000 [281:44:20<151:40:43,  6.56s/it]
 63%|██████▎   | 141771/225000 [281:44:23<128:46:12,  5.57s/it]
 63%|██████▎   | 141772/225000 [281:44:27<120:29:47,  5.21s/it]
 63%|██████▎   | 141773/225000 [281:44:32<117:36:10,  5.09s/it]
 63%|██████▎   | 141774/225000 [281:44:37<117:14:16,  5.07s/it]
 63%|██████▎   | 141775/225000 [281:44:42<117:29:51,  5.08s/it]
 63%|██████▎   | 141776/225000 [281:44:55<170:45:28,  7.39s/it]
 63%|██████▎   | 141777/225000 [281:44:59<146:46:23,  6.35s/it]
 63%|██████▎   | 141778/225000 [281:45:03<131:44:09,  5.70s/it]
 63%|██████▎   | 141779/225000 [281:45:08<125:26:19,  5.43s/it]
 63%|██████▎   | 141780/225000 [281:45:18<156:43:54,  6.78s/it]
 63%|██████▎   | 141781/225000 [281:45:22<136:23:25,  5.90s/it]
 63%|██████▎   | 141782/225000 [281:45:31<159:51:14,  6.92s/it]
 63%|██████▎   | 141783/225000 [281:45:35<136:07:34,  5.89s/it]
 63%|██████▎   | 141784/225000 [281:45:42<146:19:28,  6.33s/it]
 63%|██████▎   | 141785/225000 [281:45:47<138:01:08,  5.97s/it]
 63%|██████▎   | 141786/225000 [281:45:52<128:43:40,  5.57s/it]
 63%|██████▎   | 141787/225000 [281:45:56<117:31:19,  5.08s/it]
 63%|██████▎   | 141788/225000 [281:46:00<113:47:40,  4.92s/it]
 63%|██████▎   | 141789/225000 [281:46:05<110:18:16,  4.77s/it]
 63%|██████▎   | 141790/225000 [281:46:15<147:01:58,  6.36s/it]
 63%|██████▎   | 141791/225000 [281:46:21<146:25:42,  6.34s/it]
 63%|██████▎   | 141792/225000 [281:46:25<133:43:38,  5.79s/it]
 63%|██████▎   | 141793/225000 [281:46:29<121:49:07,  5.27s/it]
 63%|██████▎   | 141794/225000 [281:46:34<115:49:54,  5.01s/it]
 63%|██████▎   | 141795/225000 [281:46:38<112:02:54,  4.85s/it]
 63%|██████▎   | 141796/225000 [281:46:46<132:07:12,  5.72s/it]
 63%|██████▎   | 141797/225000 [281:46:57<168:30:28,  7.29s/it]
 63%|██████▎   | 141798/225000 [281:47:00<140:12:07,  6.07s/it]
 63%|██████▎   | 141799/225000 [281:47:09<157:08:35,  6.80s/it]
 63%|██████▎   | 141800/225000 [281:47:12<133:11:27,  5.76s/it]
                                                               
{'loss': '0.7992', 'grad_norm': '1.122', 'learning_rate': '1.633e-05', 'epoch': '0.6302', 'num_input_tokens_seen': 467181720, 'train_runtime': '1.014e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141800/225000 [281:47:12<133:11:27,  5.76s/it]
 63%|██████▎   | 141801/225000 [281:47:16<118:01:19,  5.11s/it]
 63%|██████▎   | 141802/225000 [281:47:26<152:17:49,  6.59s/it]
 63%|██████▎   | 141803/225000 [281:47:31<140:39:55,  6.09s/it]
 63%|██████▎   | 141804/225000 [281:47:37<139:20:48,  6.03s/it]
 63%|██████▎   | 141805/225000 [281:47:40<124:45:14,  5.40s/it]
 63%|██████▎   | 141806/225000 [281:47:45<116:10:09,  5.03s/it]
 63%|██████▎   | 141807/225000 [281:47:54<143:21:33,  6.20s/it]
 63%|██████▎   | 141808/225000 [281:48:01<150:14:22,  6.50s/it]
 63%|██████▎   | 141809/225000 [281:48:05<136:56:35,  5.93s/it]
 63%|██████▎   | 141810/225000 [281:48:13<146:35:01,  6.34s/it]
 63%|██████▎   | 141811/225000 [281:48:16<123:49:09,  5.36s/it]
 63%|██████▎   | 141812/225000 [281:48:21<125:17:32,  5.42s/it]
 63%|██████▎   | 141813/225000 [281:48:27<126:09:03,  5.46s/it]
 63%|██████▎   | 141814/225000 [281:48:32<124:17:28,  5.38s/it]
 63%|██████▎   | 141815/225000 [281:48:38<125:14:30,  5.42s/it]
 63%|██████▎   | 141816/225000 [281:48:48<159:29:58,  6.90s/it]
 63%|██████▎   | 141817/225000 [281:48:54<154:42:42,  6.70s/it]
 63%|██████▎   | 141818/225000 [281:48:57<129:49:05,  5.62s/it]
 63%|██████▎   | 141819/225000 [281:49:05<146:29:50,  6.34s/it]
 63%|██████▎   | 141820/225000 [281:49:09<129:07:46,  5.59s/it]
 63%|██████▎   | 141821/225000 [281:49:16<135:25:02,  5.86s/it]
 63%|██████▎   | 141822/225000 [281:49:22<141:34:26,  6.13s/it]
 63%|██████▎   | 141823/225000 [281:49:28<135:06:40,  5.85s/it]
 63%|██████▎   | 141824/225000 [281:49:32<128:18:42,  5.55s/it]
 63%|██████▎   | 141825/225000 [281:49:37<124:05:29,  5.37s/it]
 63%|██████▎   | 141826/225000 [281:49:42<122:17:02,  5.29s/it]
 63%|██████▎   | 141827/225000 [281:49:48<120:25:43,  5.21s/it]
 63%|██████▎   | 141828/225000 [281:49:53<119:01:22,  5.15s/it]
 63%|██████▎   | 141829/225000 [281:49:58<119:26:02,  5.17s/it]
 63%|██████▎   | 141830/225000 [281:50:05<131:23:51,  5.69s/it]
 63%|██████▎   | 141831/225000 [281:50:17<177:44:35,  7.69s/it]
 63%|██████▎   | 141832/225000 [281:50:21<149:21:42,  6.47s/it]
 63%|██████▎   | 141833/225000 [281:50:30<166:52:33,  7.22s/it]
 63%|██████▎   | 141834/225000 [281:50:33<140:07:55,  6.07s/it]
 63%|██████▎   | 141835/225000 [281:50:39<141:27:43,  6.12s/it]
 63%|██████▎   | 141836/225000 [281:50:47<151:30:36,  6.56s/it]
 63%|██████▎   | 141837/225000 [281:50:50<130:48:52,  5.66s/it]
 63%|██████▎   | 141838/225000 [281:50:54<117:33:03,  5.09s/it]
 63%|██████▎   | 141839/225000 [281:51:05<155:41:16,  6.74s/it]
 63%|██████▎   | 141840/225000 [281:51:08<134:08:01,  5.81s/it]
 63%|██████▎   | 141841/225000 [281:51:12<120:07:55,  5.20s/it]
 63%|██████▎   | 141842/225000 [281:51:16<109:52:05,  4.76s/it]
 63%|██████▎   | 141843/225000 [281:51:23<123:22:11,  5.34s/it]
 63%|██████▎   | 141844/225000 [281:51:32<148:31:12,  6.43s/it]
 63%|██████▎   | 141845/225000 [281:51:36<135:38:49,  5.87s/it]
 63%|██████▎   | 141846/225000 [281:51:40<124:44:36,  5.40s/it]
 63%|██████▎   | 141847/225000 [281:51:45<118:43:54,  5.14s/it]
 63%|██████▎   | 141848/225000 [281:51:50<116:44:40,  5.05s/it]
 63%|██████▎   | 141849/225000 [281:51:54<114:12:39,  4.94s/it]
 63%|██████▎   | 141850/225000 [281:51:59<112:24:10,  4.87s/it]
                                                               
{'loss': '0.7917', 'grad_norm': '1.148', 'learning_rate': '1.631e-05', 'epoch': '0.6304', 'num_input_tokens_seen': 467311376, 'train_runtime': '1.015e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141850/225000 [281:51:59<112:24:10,  4.87s/it]
 63%|██████▎   | 141851/225000 [281:52:03<108:46:32,  4.71s/it]
 63%|██████▎   | 141852/225000 [281:52:08<108:43:22,  4.71s/it]
 63%|██████▎   | 141853/225000 [281:52:17<136:06:33,  5.89s/it]
 63%|██████▎   | 141854/225000 [281:52:27<168:40:11,  7.30s/it]
 63%|██████▎   | 141855/225000 [281:52:31<145:04:15,  6.28s/it]
 63%|██████▎   | 141856/225000 [281:52:36<135:24:22,  5.86s/it]
 63%|██████▎   | 141857/225000 [281:52:47<168:54:23,  7.31s/it]
 63%|██████▎   | 141858/225000 [281:52:51<143:43:53,  6.22s/it]
 63%|██████▎   | 141859/225000 [281:52:54<124:32:29,  5.39s/it]
 63%|██████▎   | 141860/225000 [281:53:03<146:09:58,  6.33s/it]
 63%|██████▎   | 141861/225000 [281:53:09<149:29:25,  6.47s/it]
 63%|██████▎   | 141862/225000 [281:53:14<134:02:26,  5.80s/it]
 63%|██████▎   | 141863/225000 [281:53:19<128:30:15,  5.56s/it]
 63%|██████▎   | 141864/225000 [281:53:24<126:08:37,  5.46s/it]
 63%|██████▎   | 141865/225000 [281:53:29<126:00:57,  5.46s/it]
 63%|██████▎   | 141866/225000 [281:53:34<123:15:58,  5.34s/it]
 63%|██████▎   | 141867/225000 [281:53:39<121:12:43,  5.25s/it]
 63%|██████▎   | 141868/225000 [281:53:45<120:53:55,  5.24s/it]
 63%|██████▎   | 141869/225000 [281:53:50<121:25:57,  5.26s/it]
 63%|██████▎   | 141870/225000 [281:53:55<117:12:55,  5.08s/it]
 63%|██████▎   | 141871/225000 [281:54:00<118:20:15,  5.12s/it]
 63%|██████▎   | 141872/225000 [281:54:12<170:17:41,  7.37s/it]
 63%|██████▎   | 141873/225000 [281:54:19<162:13:07,  7.03s/it]
 63%|██████▎   | 141874/225000 [281:54:27<168:32:54,  7.30s/it]
 63%|██████▎   | 141875/225000 [281:54:30<145:03:57,  6.28s/it]
 63%|██████▎   | 141876/225000 [281:54:34<126:37:05,  5.48s/it]
 63%|██████▎   | 141877/225000 [281:54:43<151:22:11,  6.56s/it]
 63%|██████▎   | 141878/225000 [281:54:48<142:17:01,  6.16s/it]
 63%|██████▎   | 141879/225000 [281:54:53<133:26:29,  5.78s/it]
 63%|██████▎   | 141880/225000 [281:54:58<129:06:30,  5.59s/it]
 63%|██████▎   | 141881/225000 [281:55:03<123:37:48,  5.35s/it]
 63%|██████▎   | 141882/225000 [281:55:08<121:06:48,  5.25s/it]
 63%|██████▎   | 141883/225000 [281:55:13<117:59:28,  5.11s/it]
 63%|██████▎   | 141884/225000 [281:55:18<116:02:41,  5.03s/it]
 63%|██████▎   | 141885/225000 [281:55:23<116:14:59,  5.04s/it]
 63%|██████▎   | 141886/225000 [281:55:28<117:49:32,  5.10s/it]
 63%|██████▎   | 141887/225000 [281:55:33<115:42:59,  5.01s/it]
 63%|██████▎   | 141888/225000 [281:55:38<114:47:36,  4.97s/it]
 63%|██████▎   | 141889/225000 [281:55:50<165:32:18,  7.17s/it]
 63%|██████▎   | 141890/225000 [281:55:59<175:18:21,  7.59s/it]
 63%|██████▎   | 141891/225000 [281:56:04<156:20:53,  6.77s/it]
 63%|██████▎   | 141892/225000 [281:56:10<152:02:09,  6.59s/it]
 63%|██████▎   | 141893/225000 [281:56:14<133:58:58,  5.80s/it]
 63%|██████▎   | 141894/225000 [281:56:23<156:57:30,  6.80s/it]
 63%|██████▎   | 141895/225000 [281:56:26<134:55:58,  5.85s/it]
 63%|██████▎   | 141896/225000 [281:56:31<122:33:13,  5.31s/it]
 63%|██████▎   | 141897/225000 [281:56:35<119:52:37,  5.19s/it]
 63%|██████▎   | 141898/225000 [281:56:40<116:05:47,  5.03s/it]
 63%|██████▎   | 141899/225000 [281:56:45<115:01:08,  4.98s/it]
 63%|██████▎   | 141900/225000 [281:56:50<113:24:32,  4.91s/it]
                                                               
{'loss': '0.7922', 'grad_norm': '1.187', 'learning_rate': '1.63e-05', 'epoch': '0.6307', 'num_input_tokens_seen': 467439832, 'train_runtime': '1.015e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141900/225000 [281:56:50<113:24:32,  4.91s/it]
 63%|██████▎   | 141901/225000 [281:56:55<115:01:19,  4.98s/it]
 63%|██████▎   | 141902/225000 [281:57:04<141:08:29,  6.11s/it]
 63%|██████▎   | 141903/225000 [281:57:15<174:58:56,  7.58s/it]
 63%|██████▎   | 141904/225000 [281:57:19<151:00:05,  6.54s/it]
 63%|██████▎   | 141905/225000 [281:57:23<136:11:12,  5.90s/it]
 63%|██████▎   | 141906/225000 [281:57:28<126:18:44,  5.47s/it]
 63%|██████▎   | 141907/225000 [281:57:38<160:54:42,  6.97s/it]
 63%|██████▎   | 141908/225000 [281:57:43<147:09:43,  6.38s/it]
 63%|██████▎   | 141909/225000 [281:57:49<142:18:07,  6.17s/it]
 63%|██████▎   | 141910/225000 [281:57:55<142:18:11,  6.17s/it]
 63%|██████▎   | 141911/225000 [281:57:59<124:36:15,  5.40s/it]
 63%|██████▎   | 141912/225000 [281:58:08<152:34:07,  6.61s/it]
 63%|██████▎   | 141913/225000 [281:58:11<128:58:25,  5.59s/it]
 63%|██████▎   | 141914/225000 [281:58:15<114:39:22,  4.97s/it]
 63%|██████▎   | 141915/225000 [281:58:24<147:09:30,  6.38s/it]
 63%|██████▎   | 141916/225000 [281:58:29<136:28:18,  5.91s/it]
 63%|██████▎   | 141917/225000 [281:58:34<128:03:34,  5.55s/it]
 63%|██████▎   | 141918/225000 [281:58:39<123:50:33,  5.37s/it]
 63%|██████▎   | 141919/225000 [281:58:44<123:25:55,  5.35s/it]
 63%|██████▎   | 141920/225000 [281:58:49<120:55:48,  5.24s/it]
 63%|██████▎   | 141921/225000 [281:58:54<118:22:02,  5.13s/it]
 63%|██████▎   | 141922/225000 [281:58:59<114:49:19,  4.98s/it]
 63%|██████▎   | 141923/225000 [281:59:05<126:56:32,  5.50s/it]
 63%|██████▎   | 141924/225000 [281:59:18<174:49:29,  7.58s/it]
 63%|██████▎   | 141925/225000 [281:59:22<154:50:34,  6.71s/it]
 63%|██████▎   | 141926/225000 [281:59:27<140:37:32,  6.09s/it]
 63%|██████▎   | 141927/225000 [281:59:32<129:24:29,  5.61s/it]
 63%|██████▎   | 141928/225000 [281:59:37<128:20:01,  5.56s/it]
 63%|██████▎   | 141929/225000 [281:59:42<123:05:23,  5.33s/it]
 63%|██████▎   | 141930/225000 [281:59:54<171:30:59,  7.43s/it]
 63%|██████▎   | 141931/225000 [281:59:59<150:45:57,  6.53s/it]
 63%|██████▎   | 141932/225000 [282:00:02<132:00:44,  5.72s/it]
 63%|██████▎   | 141933/225000 [282:00:06<116:19:42,  5.04s/it]
 63%|██████▎   | 141934/225000 [282:00:17<161:39:13,  7.01s/it]
 63%|██████▎   | 141935/225000 [282:00:21<135:25:18,  5.87s/it]
 63%|██████▎   | 141936/225000 [282:00:25<127:59:11,  5.55s/it]
 63%|██████▎   | 141937/225000 [282:00:31<128:57:20,  5.59s/it]
 63%|██████▎   | 141938/225000 [282:00:37<132:15:38,  5.73s/it]
 63%|██████▎   | 141939/225000 [282:00:42<126:32:27,  5.48s/it]
 63%|██████▎   | 141940/225000 [282:00:47<124:56:28,  5.42s/it]
 63%|██████▎   | 141941/225000 [282:00:56<144:11:41,  6.25s/it]
 63%|██████▎   | 141942/225000 [282:01:00<133:41:11,  5.79s/it]
 63%|██████▎   | 141943/225000 [282:01:07<137:56:56,  5.98s/it]
 63%|██████▎   | 141944/225000 [282:01:11<124:06:14,  5.38s/it]
 63%|██████▎   | 141945/225000 [282:01:15<115:57:39,  5.03s/it]
 63%|██████▎   | 141946/225000 [282:01:22<128:46:11,  5.58s/it]
 63%|██████▎   | 141947/225000 [282:01:32<158:55:30,  6.89s/it]
 63%|██████▎   | 141948/225000 [282:01:35<133:35:42,  5.79s/it]
 63%|██████▎   | 141949/225000 [282:01:43<149:01:45,  6.46s/it]
 63%|██████▎   | 141950/225000 [282:01:47<131:19:59,  5.69s/it]
                                                               
{'loss': '0.8007', 'grad_norm': '1.079', 'learning_rate': '1.628e-05', 'epoch': '0.6309', 'num_input_tokens_seen': 467569688, 'train_runtime': '1.015e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 141950/225000 [282:01:47<131:19:59,  5.69s/it]
 63%|██████▎   | 141951/225000 [282:01:51<120:52:53,  5.24s/it]
 63%|██████▎   | 141952/225000 [282:01:59<137:48:18,  5.97s/it]
 63%|██████▎   | 141953/225000 [282:02:03<126:12:10,  5.47s/it]
 63%|██████▎   | 141954/225000 [282:02:07<117:54:47,  5.11s/it]
 63%|██████▎   | 141955/225000 [282:02:11<110:58:09,  4.81s/it]
 63%|██████▎   | 141956/225000 [282:02:18<121:38:42,  5.27s/it]
 63%|██████▎   | 141957/225000 [282:02:28<155:39:07,  6.75s/it]
 63%|██████▎   | 141958/225000 [282:02:32<139:18:01,  6.04s/it]
 63%|██████▎   | 141959/225000 [282:02:36<124:39:35,  5.40s/it]
 63%|██████▎   | 141960/225000 [282:02:40<114:16:36,  4.95s/it]
 63%|██████▎   | 141961/225000 [282:02:47<129:38:38,  5.62s/it]
 63%|██████▎   | 141962/225000 [282:02:55<146:32:23,  6.35s/it]
 63%|██████▎   | 141963/225000 [282:02:59<130:18:14,  5.65s/it]
 63%|██████▎   | 141964/225000 [282:03:04<119:50:51,  5.20s/it]
 63%|██████▎   | 141965/225000 [282:03:08<111:59:17,  4.86s/it]
 63%|██████▎   | 141966/225000 [282:03:18<149:42:22,  6.49s/it]
 63%|██████▎   | 141967/225000 [282:03:21<126:54:10,  5.50s/it]
 63%|██████▎   | 141968/225000 [282:03:27<132:36:26,  5.75s/it]
 63%|██████▎   | 141969/225000 [282:03:34<139:30:48,  6.05s/it]
 63%|██████▎   | 141970/225000 [282:03:40<138:47:26,  6.02s/it]
 63%|██████▎   | 141971/225000 [282:03:45<130:22:58,  5.65s/it]
 63%|██████▎   | 141972/225000 [282:03:48<115:14:49,  5.00s/it]
 63%|██████▎   | 141973/225000 [282:03:53<113:17:23,  4.91s/it]
 63%|██████▎   | 141974/225000 [282:03:58<113:34:45,  4.92s/it]
 63%|██████▎   | 141975/225000 [282:04:02<106:57:11,  4.64s/it]
 63%|██████▎   | 141976/225000 [282:04:07<107:43:14,  4.67s/it]
 63%|██████▎   | 141977/225000 [282:04:12<111:34:56,  4.84s/it]
 63%|██████▎   | 141978/225000 [282:04:20<131:13:55,  5.69s/it]
 63%|██████▎   | 141979/225000 [282:04:33<181:49:31,  7.88s/it]
 63%|██████▎   | 141980/225000 [282:04:38<160:47:44,  6.97s/it]
 63%|██████▎   | 141981/225000 [282:04:42<145:15:43,  6.30s/it]
 63%|██████▎   | 141982/225000 [282:04:47<135:29:57,  5.88s/it]
 63%|██████▎   | 141983/225000 [282:04:52<127:12:43,  5.52s/it]
 63%|██████▎   | 141984/225000 [282:04:56<120:40:08,  5.23s/it]
 63%|██████▎   | 141985/225000 [282:05:01<118:36:48,  5.14s/it]
 63%|██████▎   | 141986/225000 [282:05:14<173:13:08,  7.51s/it]
 63%|██████▎   | 141987/225000 [282:05:19<150:03:59,  6.51s/it]
 63%|██████▎   | 141988/225000 [282:05:22<129:36:40,  5.62s/it]
 63%|██████▎   | 141989/225000 [282:05:32<156:12:48,  6.77s/it]
 63%|██████▎   | 141990/225000 [282:05:35<135:05:53,  5.86s/it]
 63%|██████▎   | 141991/225000 [282:05:43<151:03:16,  6.55s/it]
 63%|██████▎   | 141992/225000 [282:05:47<132:06:06,  5.73s/it]
 63%|██████▎   | 141993/225000 [282:05:53<133:22:59,  5.78s/it]
 63%|██████▎   | 141994/225000 [282:05:59<136:15:47,  5.91s/it]
 63%|██████▎   | 141995/225000 [282:06:03<123:11:03,  5.34s/it]
 63%|██████▎   | 141996/225000 [282:06:08<114:32:29,  4.97s/it]
 63%|██████▎   | 141997/225000 [282:06:16<137:18:34,  5.96s/it]
 63%|██████▎   | 141998/225000 [282:06:23<146:52:17,  6.37s/it]
 63%|██████▎   | 141999/225000 [282:06:26<125:25:32,  5.44s/it]
 63%|██████▎   | 142000/225000 [282:06:33<135:25:15,  5.87s/it]
                                                               
{'loss': '0.7856', 'grad_norm': '1.177', 'learning_rate': '1.626e-05', 'epoch': '0.6311', 'num_input_tokens_seen': 467697888, 'train_runtime': '1.016e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142000/225000 [282:06:33<135:25:15,  5.87s/it]
 63%|██████▎   | 142001/225000 [282:06:40<144:13:55,  6.26s/it]
 63%|██████▎   | 142002/225000 [282:06:44<128:42:13,  5.58s/it]
 63%|██████▎   | 142003/225000 [282:06:52<145:55:16,  6.33s/it]
 63%|██████▎   | 142004/225000 [282:06:57<133:18:31,  5.78s/it]
 63%|██████▎   | 142005/225000 [282:07:04<143:30:29,  6.22s/it]
 63%|██████▎   | 142006/225000 [282:07:09<136:22:25,  5.92s/it]
 63%|██████▎   | 142007/225000 [282:07:15<131:03:04,  5.68s/it]
 63%|██████▎   | 142008/225000 [282:07:20<129:49:52,  5.63s/it]
 63%|██████▎   | 142009/225000 [282:07:26<129:02:13,  5.60s/it]
 63%|██████▎   | 142010/225000 [282:07:31<127:35:13,  5.53s/it]
 63%|██████▎   | 142011/225000 [282:07:36<125:03:44,  5.43s/it]
 63%|██████▎   | 142012/225000 [282:07:42<129:41:02,  5.63s/it]
 63%|██████▎   | 142013/225000 [282:07:50<144:10:32,  6.25s/it]
 63%|██████▎   | 142014/225000 [282:07:55<134:17:12,  5.83s/it]
 63%|██████▎   | 142015/225000 [282:08:02<141:48:21,  6.15s/it]
 63%|██████▎   | 142016/225000 [282:08:06<129:00:42,  5.60s/it]
 63%|██████▎   | 142017/225000 [282:08:11<123:36:58,  5.36s/it]
 63%|██████▎   | 142018/225000 [282:08:17<126:07:40,  5.47s/it]
 63%|██████▎   | 142019/225000 [282:08:22<125:42:50,  5.45s/it]
 63%|██████▎   | 142020/225000 [282:08:27<121:33:21,  5.27s/it]
 63%|██████▎   | 142021/225000 [282:08:33<124:16:14,  5.39s/it]
 63%|██████▎   | 142022/225000 [282:08:38<125:35:24,  5.45s/it]
 63%|██████▎   | 142023/225000 [282:08:43<123:43:23,  5.37s/it]
 63%|██████▎   | 142024/225000 [282:08:48<121:26:38,  5.27s/it]
 63%|██████▎   | 142025/225000 [282:08:53<116:38:01,  5.06s/it]
 63%|██████▎   | 142026/225000 [282:08:58<117:15:14,  5.09s/it]
 63%|██████▎   | 142027/225000 [282:09:03<117:17:31,  5.09s/it]
 63%|██████▎   | 142028/225000 [282:09:08<118:15:44,  5.13s/it]
 63%|██████▎   | 142029/225000 [282:09:14<118:28:24,  5.14s/it]
 63%|██████▎   | 142030/225000 [282:09:19<118:28:05,  5.14s/it]
 63%|██████▎   | 142031/225000 [282:09:24<120:14:57,  5.22s/it]
 63%|██████▎   | 142032/225000 [282:09:34<153:59:31,  6.68s/it]
 63%|██████▎   | 142033/225000 [282:09:39<144:45:50,  6.28s/it]
 63%|██████▎   | 142034/225000 [282:09:45<141:15:31,  6.13s/it]
 63%|██████▎   | 142035/225000 [282:09:51<136:33:30,  5.93s/it]
 63%|██████▎   | 142036/225000 [282:09:56<132:19:38,  5.74s/it]
 63%|██████▎   | 142037/225000 [282:10:01<128:46:40,  5.59s/it]
 63%|██████▎   | 142038/225000 [282:10:07<127:53:49,  5.55s/it]
 63%|██████▎   | 142039/225000 [282:10:12<125:18:35,  5.44s/it]
 63%|██████▎   | 142040/225000 [282:10:22<157:39:50,  6.84s/it]
 63%|██████▎   | 142041/225000 [282:10:28<152:14:07,  6.61s/it]
 63%|██████▎   | 142042/225000 [282:10:32<130:18:59,  5.66s/it]
 63%|██████▎   | 142043/225000 [282:10:37<127:02:22,  5.51s/it]
 63%|██████▎   | 142044/225000 [282:10:42<127:21:36,  5.53s/it]
 63%|██████▎   | 142045/225000 [282:10:51<149:28:46,  6.49s/it]
 63%|██████▎   | 142046/225000 [282:10:55<130:47:10,  5.68s/it]
 63%|██████▎   | 142047/225000 [282:10:58<116:40:14,  5.06s/it]
 63%|██████▎   | 142048/225000 [282:11:07<138:25:01,  6.01s/it]
 63%|██████▎   | 142049/225000 [282:11:12<137:22:57,  5.96s/it]
 63%|██████▎   | 142050/225000 [282:11:16<119:25:50,  5.18s/it]
                                                               
{'loss': '0.7893', 'grad_norm': '1.219', 'learning_rate': '1.625e-05', 'epoch': '0.6313', 'num_input_tokens_seen': 467827672, 'train_runtime': '1.016e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142050/225000 [282:11:16<119:25:50,  5.18s/it]
 63%|██████▎   | 142051/225000 [282:11:25<147:37:43,  6.41s/it]
 63%|██████▎   | 142052/225000 [282:11:29<127:00:48,  5.51s/it]
 63%|██████▎   | 142053/225000 [282:11:32<112:31:43,  4.88s/it]
 63%|██████▎   | 142054/225000 [282:11:42<146:56:08,  6.38s/it]
 63%|██████▎   | 142055/225000 [282:11:46<129:46:16,  5.63s/it]
 63%|██████▎   | 142056/225000 [282:11:51<130:52:41,  5.68s/it]
 63%|██████▎   | 142057/225000 [282:11:57<127:14:38,  5.52s/it]
 63%|██████▎   | 142058/225000 [282:12:04<143:18:44,  6.22s/it]
 63%|██████▎   | 142059/225000 [282:12:08<123:45:31,  5.37s/it]
 63%|██████▎   | 142060/225000 [282:12:12<115:10:15,  5.00s/it]
 63%|██████▎   | 142061/225000 [282:12:18<120:21:28,  5.22s/it]
 63%|██████▎   | 142062/225000 [282:12:23<120:27:40,  5.23s/it]
 63%|██████▎   | 142063/225000 [282:12:29<124:48:48,  5.42s/it]
 63%|██████▎   | 142064/225000 [282:12:37<141:41:36,  6.15s/it]
 63%|██████▎   | 142065/225000 [282:12:42<132:17:34,  5.74s/it]
 63%|██████▎   | 142066/225000 [282:12:45<119:42:00,  5.20s/it]
 63%|██████▎   | 142067/225000 [282:12:55<150:40:11,  6.54s/it]
 63%|██████▎   | 142068/225000 [282:12:59<131:07:02,  5.69s/it]
 63%|██████▎   | 142069/225000 [282:13:05<137:06:28,  5.95s/it]
 63%|██████▎   | 142070/225000 [282:13:10<131:18:54,  5.70s/it]
 63%|██████▎   | 142071/225000 [282:13:14<116:23:00,  5.05s/it]
 63%|██████▎   | 142072/225000 [282:13:18<107:45:40,  4.68s/it]
 63%|██████▎   | 142073/225000 [282:13:27<139:41:35,  6.06s/it]
 63%|██████▎   | 142074/225000 [282:13:35<149:45:03,  6.50s/it]
 63%|██████▎   | 142075/225000 [282:13:39<138:19:57,  6.01s/it]
 63%|██████▎   | 142076/225000 [282:13:45<133:55:58,  5.81s/it]
 63%|██████▎   | 142077/225000 [282:13:50<128:26:34,  5.58s/it]
 63%|██████▎   | 142078/225000 [282:13:54<121:40:42,  5.28s/it]
 63%|██████▎   | 142079/225000 [282:14:00<122:51:34,  5.33s/it]
 63%|██████▎   | 142080/225000 [282:14:05<123:05:27,  5.34s/it]
 63%|██████▎   | 142081/225000 [282:14:10<121:59:16,  5.30s/it]
 63%|██████▎   | 142082/225000 [282:14:16<121:31:15,  5.28s/it]
 63%|██████▎   | 142083/225000 [282:14:21<121:15:22,  5.26s/it]
 63%|██████▎   | 142084/225000 [282:14:26<120:22:17,  5.23s/it]
 63%|██████▎   | 142085/225000 [282:14:32<122:20:36,  5.31s/it]
 63%|██████▎   | 142086/225000 [282:14:37<123:12:24,  5.35s/it]
 63%|██████▎   | 142087/225000 [282:14:42<123:26:57,  5.36s/it]
 63%|██████▎   | 142088/225000 [282:14:48<122:22:11,  5.31s/it]
 63%|██████▎   | 142089/225000 [282:14:53<122:27:25,  5.32s/it]
 63%|██████▎   | 142090/225000 [282:15:05<171:57:50,  7.47s/it]
 63%|██████▎   | 142091/225000 [282:15:13<171:12:17,  7.43s/it]
 63%|██████▎   | 142092/225000 [282:15:18<153:39:59,  6.67s/it]
 63%|██████▎   | 142093/225000 [282:15:22<139:11:03,  6.04s/it]
 63%|██████▎   | 142094/225000 [282:15:27<133:10:32,  5.78s/it]
 63%|██████▎   | 142095/225000 [282:15:32<125:11:57,  5.44s/it]
 63%|██████▎   | 142096/225000 [282:15:36<115:41:36,  5.02s/it]
 63%|██████▎   | 142097/225000 [282:15:41<116:57:40,  5.08s/it]
 63%|██████▎   | 142098/225000 [282:15:54<169:29:55,  7.36s/it]
 63%|██████▎   | 142099/225000 [282:15:59<152:56:21,  6.64s/it]
 63%|██████▎   | 142100/225000 [282:16:04<142:38:01,  6.19s/it]
                                                               
{'loss': '0.7893', 'grad_norm': '1.028', 'learning_rate': '1.623e-05', 'epoch': '0.6316', 'num_input_tokens_seen': 467957336, 'train_runtime': '1.016e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142100/225000 [282:16:04<142:38:01,  6.19s/it]
 63%|██████▎   | 142101/225000 [282:16:09<132:54:58,  5.77s/it]
 63%|██████▎   | 142102/225000 [282:16:14<126:56:13,  5.51s/it]
 63%|██████▎   | 142103/225000 [282:16:19<121:19:10,  5.27s/it]
 63%|██████▎   | 142104/225000 [282:16:24<120:26:20,  5.23s/it]
 63%|██████▎   | 142105/225000 [282:16:31<134:46:51,  5.85s/it]
 63%|██████▎   | 142106/225000 [282:16:42<167:43:59,  7.28s/it]
 63%|██████▎   | 142107/225000 [282:16:45<141:21:54,  6.14s/it]
 63%|██████▎   | 142108/225000 [282:16:54<162:40:51,  7.07s/it]
 63%|██████▎   | 142109/225000 [282:17:00<152:56:33,  6.64s/it]
 63%|██████▎   | 142110/225000 [282:17:03<129:31:42,  5.63s/it]
 63%|██████▎   | 142111/225000 [282:17:12<152:40:24,  6.63s/it]
 63%|██████▎   | 142112/225000 [282:17:16<133:28:08,  5.80s/it]
 63%|██████▎   | 142113/225000 [282:17:20<123:15:43,  5.35s/it]
 63%|██████▎   | 142114/225000 [282:17:25<115:02:49,  5.00s/it]
 63%|██████▎   | 142115/225000 [282:17:29<109:04:19,  4.74s/it]
 63%|██████▎   | 142116/225000 [282:17:40<153:10:07,  6.65s/it]
 63%|██████▎   | 142117/225000 [282:17:44<135:00:39,  5.86s/it]
 63%|██████▎   | 142118/225000 [282:17:48<122:27:05,  5.32s/it]
 63%|██████▎   | 142119/225000 [282:17:53<121:03:40,  5.26s/it]
 63%|██████▎   | 142120/225000 [282:17:58<119:38:53,  5.20s/it]
 63%|██████▎   | 142121/225000 [282:18:03<119:15:34,  5.18s/it]
 63%|██████▎   | 142122/225000 [282:18:08<117:19:55,  5.10s/it]
 63%|██████▎   | 142123/225000 [282:18:13<114:54:24,  4.99s/it]
 63%|██████▎   | 142124/225000 [282:18:18<119:25:36,  5.19s/it]
 63%|██████▎   | 142125/225000 [282:18:24<121:23:03,  5.27s/it]
 63%|██████▎   | 142126/225000 [282:18:29<121:26:04,  5.28s/it]
 63%|██████▎   | 142127/225000 [282:18:37<140:38:42,  6.11s/it]
 63%|██████▎   | 142128/225000 [282:18:47<166:26:00,  7.23s/it]
 63%|██████▎   | 142129/225000 [282:18:51<144:49:42,  6.29s/it]
 63%|██████▎   | 142130/225000 [282:18:56<133:33:07,  5.80s/it]
 63%|██████▎   | 142131/225000 [282:19:03<142:31:03,  6.19s/it]
 63%|██████▎   | 142132/225000 [282:19:12<161:12:32,  7.00s/it]
 63%|██████▎   | 142133/225000 [282:19:16<138:37:41,  6.02s/it]
 63%|██████▎   | 142134/225000 [282:19:20<124:30:09,  5.41s/it]
 63%|██████▎   | 142135/225000 [282:19:23<114:05:15,  4.96s/it]
 63%|██████▎   | 142136/225000 [282:19:34<151:44:58,  6.59s/it]
 63%|██████▎   | 142137/225000 [282:19:39<144:23:49,  6.27s/it]
 63%|██████▎   | 142138/225000 [282:19:47<151:50:31,  6.60s/it]
 63%|██████▎   | 142139/225000 [282:19:51<135:23:27,  5.88s/it]
 63%|██████▎   | 142140/225000 [282:19:55<123:14:15,  5.35s/it]
 63%|██████▎   | 142141/225000 [282:19:59<114:22:11,  4.97s/it]
 63%|██████▎   | 142142/225000 [282:20:08<143:31:25,  6.24s/it]
 63%|██████▎   | 142143/225000 [282:20:15<143:23:45,  6.23s/it]
 63%|██████▎   | 142144/225000 [282:20:19<128:10:55,  5.57s/it]
 63%|██████▎   | 142145/225000 [282:20:23<117:41:19,  5.11s/it]
 63%|██████▎   | 142146/225000 [282:20:30<133:08:36,  5.79s/it]
 63%|██████▎   | 142147/225000 [282:20:38<148:57:37,  6.47s/it]
 63%|██████▎   | 142148/225000 [282:20:43<138:37:01,  6.02s/it]
 63%|██████▎   | 142149/225000 [282:20:48<133:40:19,  5.81s/it]
 63%|██████▎   | 142150/225000 [282:20:54<129:56:22,  5.65s/it]
                                                               
{'loss': '0.7873', 'grad_norm': '1.023', 'learning_rate': '1.621e-05', 'epoch': '0.6318', 'num_input_tokens_seen': 468087064, 'train_runtime': '1.016e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142150/225000 [282:20:54<129:56:22,  5.65s/it]
 63%|██████▎   | 142151/225000 [282:20:59<126:30:37,  5.50s/it]
 63%|██████▎   | 142152/225000 [282:21:04<123:45:07,  5.38s/it]
 63%|██████▎   | 142153/225000 [282:21:09<122:12:28,  5.31s/it]
 63%|██████▎   | 142154/225000 [282:21:14<118:17:57,  5.14s/it]
 63%|██████▎   | 142155/225000 [282:21:19<117:28:07,  5.10s/it]
 63%|██████▎   | 142156/225000 [282:21:24<117:34:08,  5.11s/it]
 63%|██████▎   | 142157/225000 [282:21:28<113:17:32,  4.92s/it]
 63%|██████▎   | 142158/225000 [282:21:33<111:39:48,  4.85s/it]
 63%|██████▎   | 142159/225000 [282:21:38<114:17:01,  4.97s/it]
 63%|██████▎   | 142160/225000 [282:21:50<160:20:51,  6.97s/it]
 63%|██████▎   | 142161/225000 [282:21:56<156:26:07,  6.80s/it]
 63%|██████▎   | 142162/225000 [282:22:02<149:07:18,  6.48s/it]
 63%|██████▎   | 142163/225000 [282:22:07<139:36:12,  6.07s/it]
 63%|██████▎   | 142164/225000 [282:22:12<131:42:42,  5.72s/it]
 63%|██████▎   | 142165/225000 [282:22:17<127:47:59,  5.55s/it]
 63%|██████▎   | 142166/225000 [282:22:22<124:40:46,  5.42s/it]
 63%|██████▎   | 142167/225000 [282:22:27<121:42:59,  5.29s/it]
 63%|██████▎   | 142168/225000 [282:22:33<120:41:00,  5.25s/it]
 63%|██████▎   | 142169/225000 [282:22:38<121:47:55,  5.29s/it]
 63%|██████▎   | 142170/225000 [282:22:47<144:55:40,  6.30s/it]
 63%|██████▎   | 142171/225000 [282:22:57<171:13:25,  7.44s/it]
 63%|██████▎   | 142172/225000 [282:23:00<146:00:27,  6.35s/it]
 63%|██████▎   | 142173/225000 [282:23:04<129:37:56,  5.63s/it]
 63%|██████▎   | 142174/225000 [282:23:09<123:49:06,  5.38s/it]
 63%|██████▎   | 142175/225000 [282:23:14<119:26:38,  5.19s/it]
 63%|██████▎   | 142176/225000 [282:23:19<118:56:11,  5.17s/it]
 63%|██████▎   | 142177/225000 [282:23:24<117:54:23,  5.12s/it]
 63%|██████▎   | 142178/225000 [282:23:29<116:20:30,  5.06s/it]
 63%|██████▎   | 142179/225000 [282:23:34<119:04:25,  5.18s/it]
 63%|██████▎   | 142180/225000 [282:23:47<170:13:44,  7.40s/it]
 63%|██████▎   | 142181/225000 [282:23:51<147:43:10,  6.42s/it]
 63%|██████▎   | 142182/225000 [282:23:57<146:09:54,  6.35s/it]
 63%|██████▎   | 142183/225000 [282:24:03<141:49:07,  6.16s/it]
 63%|██████▎   | 142184/225000 [282:24:07<124:18:23,  5.40s/it]
 63%|██████▎   | 142185/225000 [282:24:15<146:44:51,  6.38s/it]
 63%|██████▎   | 142186/225000 [282:24:19<125:46:07,  5.47s/it]
 63%|██████▎   | 142187/225000 [282:24:26<134:52:45,  5.86s/it]
 63%|██████▎   | 142188/225000 [282:24:30<122:20:02,  5.32s/it]
 63%|██████▎   | 142189/225000 [282:24:34<113:28:39,  4.93s/it]
 63%|██████▎   | 142190/225000 [282:24:43<146:23:09,  6.36s/it]
 63%|██████▎   | 142191/225000 [282:24:47<125:58:05,  5.48s/it]
 63%|██████▎   | 142192/225000 [282:24:54<140:31:36,  6.11s/it]
 63%|██████▎   | 142193/225000 [282:24:58<120:44:49,  5.25s/it]
 63%|██████▎   | 142194/225000 [282:25:09<160:07:09,  6.96s/it]
 63%|██████▎   | 142195/225000 [282:25:12<134:23:15,  5.84s/it]
 63%|██████▎   | 142196/225000 [282:25:16<123:09:26,  5.35s/it]
 63%|██████▎   | 142197/225000 [282:25:20<116:17:10,  5.06s/it]
 63%|██████▎   | 142198/225000 [282:25:25<110:57:15,  4.82s/it]
 63%|██████▎   | 142199/225000 [282:25:37<163:01:20,  7.09s/it]
 63%|██████▎   | 142200/225000 [282:25:42<149:30:14,  6.50s/it]
                                                               
{'loss': '0.7984', 'grad_norm': '1.221', 'learning_rate': '1.619e-05', 'epoch': '0.632', 'num_input_tokens_seen': 468216936, 'train_runtime': '1.017e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142200/225000 [282:25:42<149:30:14,  6.50s/it]
 63%|██████▎   | 142201/225000 [282:25:46<130:05:57,  5.66s/it]
 63%|██████▎   | 142202/225000 [282:25:54<145:23:49,  6.32s/it]
 63%|██████▎   | 142203/225000 [282:26:02<156:25:56,  6.80s/it]
 63%|██████▎   | 142204/225000 [282:26:05<132:43:26,  5.77s/it]
 63%|██████▎   | 142205/225000 [282:26:12<140:07:02,  6.09s/it]
 63%|██████▎   | 142206/225000 [282:26:16<126:51:59,  5.52s/it]
 63%|██████▎   | 142207/225000 [282:26:20<117:31:57,  5.11s/it]
 63%|██████▎   | 142208/225000 [282:26:24<110:37:57,  4.81s/it]
 63%|██████▎   | 142209/225000 [282:26:34<142:20:54,  6.19s/it]
 63%|██████▎   | 142210/225000 [282:26:40<141:40:19,  6.16s/it]
 63%|██████▎   | 142211/225000 [282:26:44<125:10:56,  5.44s/it]
 63%|██████▎   | 142212/225000 [282:26:47<112:38:14,  4.90s/it]
 63%|██████▎   | 142213/225000 [282:26:52<115:26:05,  5.02s/it]
 63%|██████▎   | 142214/225000 [282:27:03<155:20:14,  6.75s/it]
 63%|██████▎   | 142215/225000 [282:27:07<134:17:58,  5.84s/it]
 63%|██████▎   | 142216/225000 [282:27:11<118:45:42,  5.16s/it]
 63%|██████▎   | 142217/225000 [282:27:15<113:51:57,  4.95s/it]
 63%|██████▎   | 142218/225000 [282:27:19<110:49:35,  4.82s/it]
 63%|██████▎   | 142219/225000 [282:27:24<107:09:37,  4.66s/it]
 63%|██████▎   | 142220/225000 [282:27:35<153:23:19,  6.67s/it]
 63%|██████▎   | 142221/225000 [282:27:42<152:48:27,  6.65s/it]
 63%|██████▎   | 142222/225000 [282:27:46<137:02:12,  5.96s/it]
 63%|██████▎   | 142223/225000 [282:27:50<125:21:03,  5.45s/it]
 63%|██████▎   | 142224/225000 [282:27:55<118:16:54,  5.14s/it]
 63%|██████▎   | 142225/225000 [282:28:00<119:14:12,  5.19s/it]
 63%|██████▎   | 142226/225000 [282:28:12<165:25:59,  7.20s/it]
 63%|██████▎   | 142227/225000 [282:28:15<140:06:50,  6.09s/it]
 63%|██████▎   | 142228/225000 [282:28:19<124:11:24,  5.40s/it]
 63%|██████▎   | 142229/225000 [282:28:31<166:14:31,  7.23s/it]
 63%|██████▎   | 142230/225000 [282:28:34<139:23:19,  6.06s/it]
 63%|██████▎   | 142231/225000 [282:28:42<151:25:53,  6.59s/it]
 63%|██████▎   | 142232/225000 [282:28:45<130:37:56,  5.68s/it]
 63%|██████▎   | 142233/225000 [282:28:53<142:58:37,  6.22s/it]
 63%|██████▎   | 142234/225000 [282:28:57<129:29:27,  5.63s/it]
 63%|██████▎   | 142235/225000 [282:29:05<144:19:20,  6.28s/it]
 63%|██████▎   | 142236/225000 [282:29:09<129:25:50,  5.63s/it]
 63%|██████▎   | 142237/225000 [282:29:13<120:14:54,  5.23s/it]
 63%|██████▎   | 142238/225000 [282:29:18<113:25:45,  4.93s/it]
 63%|██████▎   | 142239/225000 [282:29:22<109:37:15,  4.77s/it]
 63%|██████▎   | 142240/225000 [282:29:34<156:41:28,  6.82s/it]
 63%|██████▎   | 142241/225000 [282:29:38<140:42:09,  6.12s/it]
 63%|██████▎   | 142242/225000 [282:29:44<139:45:44,  6.08s/it]
 63%|██████▎   | 142243/225000 [282:29:48<126:43:48,  5.51s/it]
 63%|██████▎   | 142244/225000 [282:29:53<120:45:09,  5.25s/it]
 63%|██████▎   | 142245/225000 [282:29:57<113:59:56,  4.96s/it]
 63%|██████▎   | 142246/225000 [282:30:07<149:19:03,  6.50s/it]
 63%|██████▎   | 142247/225000 [282:30:13<146:55:57,  6.39s/it]
 63%|██████▎   | 142248/225000 [282:30:17<126:57:49,  5.52s/it]
 63%|██████▎   | 142249/225000 [282:30:21<117:42:50,  5.12s/it]
 63%|██████▎   | 142250/225000 [282:30:25<110:53:53,  4.82s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.144', 'learning_rate': '1.618e-05', 'epoch': '0.6322', 'num_input_tokens_seen': 468345704, 'train_runtime': '1.017e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142250/225000 [282:30:25<110:53:53,  4.82s/it]
 63%|██████▎   | 142251/225000 [282:30:36<153:53:53,  6.70s/it]
 63%|██████▎   | 142252/225000 [282:30:40<131:14:52,  5.71s/it]
 63%|██████▎   | 142253/225000 [282:30:47<139:00:09,  6.05s/it]
 63%|██████▎   | 142254/225000 [282:30:53<143:18:58,  6.24s/it]
 63%|██████▎   | 142255/225000 [282:30:57<124:03:02,  5.40s/it]
 63%|██████▎   | 142256/225000 [282:31:04<139:29:32,  6.07s/it]
 63%|██████▎   | 142257/225000 [282:31:08<119:40:18,  5.21s/it]
 63%|██████▎   | 142258/225000 [282:31:15<135:39:28,  5.90s/it]
 63%|██████▎   | 142259/225000 [282:31:19<125:11:16,  5.45s/it]
 63%|██████▎   | 142260/225000 [282:31:25<125:15:34,  5.45s/it]
 63%|██████▎   | 142261/225000 [282:31:30<125:47:09,  5.47s/it]
 63%|██████▎   | 142262/225000 [282:31:38<139:43:46,  6.08s/it]
 63%|██████▎   | 142263/225000 [282:31:43<129:47:30,  5.65s/it]
 63%|██████▎   | 142264/225000 [282:31:47<123:16:00,  5.36s/it]
 63%|██████▎   | 142265/225000 [282:31:52<118:30:04,  5.16s/it]
 63%|██████▎   | 142266/225000 [282:31:56<113:01:23,  4.92s/it]
 63%|██████▎   | 142267/225000 [282:32:01<111:56:39,  4.87s/it]
 63%|██████▎   | 142268/225000 [282:32:06<110:18:10,  4.80s/it]
 63%|██████▎   | 142269/225000 [282:32:16<145:38:59,  6.34s/it]
 63%|██████▎   | 142270/225000 [282:32:25<163:43:30,  7.12s/it]
 63%|██████▎   | 142271/225000 [282:32:29<144:45:39,  6.30s/it]
 63%|██████▎   | 142272/225000 [282:32:33<129:05:54,  5.62s/it]
 63%|██████▎   | 142273/225000 [282:32:37<121:09:16,  5.27s/it]
 63%|██████▎   | 142274/225000 [282:32:42<114:53:24,  5.00s/it]
 63%|██████▎   | 142275/225000 [282:32:50<138:45:26,  6.04s/it]
 63%|██████▎   | 142276/225000 [282:32:59<159:52:09,  6.96s/it]
 63%|██████▎   | 142277/225000 [282:33:03<136:38:11,  5.95s/it]
 63%|██████▎   | 142278/225000 [282:33:08<130:50:29,  5.69s/it]
 63%|██████▎   | 142279/225000 [282:33:13<125:50:24,  5.48s/it]
 63%|██████▎   | 142280/225000 [282:33:18<123:52:23,  5.39s/it]
 63%|██████▎   | 142281/225000 [282:33:24<123:44:43,  5.39s/it]
 63%|██████▎   | 142282/225000 [282:33:32<143:40:19,  6.25s/it]
 63%|██████▎   | 142283/225000 [282:33:38<145:09:22,  6.32s/it]
 63%|██████▎   | 142284/225000 [282:33:42<126:43:57,  5.52s/it]
 63%|██████▎   | 142285/225000 [282:33:53<165:23:11,  7.20s/it]
 63%|██████▎   | 142286/225000 [282:33:57<139:52:07,  6.09s/it]
 63%|██████▎   | 142287/225000 [282:34:01<130:30:42,  5.68s/it]
 63%|██████▎   | 142288/225000 [282:34:10<152:06:09,  6.62s/it]
 63%|██████▎   | 142289/225000 [282:34:14<134:28:54,  5.85s/it]
 63%|██████▎   | 142290/225000 [282:34:19<127:20:38,  5.54s/it]
 63%|██████▎   | 142291/225000 [282:34:23<119:17:18,  5.19s/it]
 63%|██████▎   | 142292/225000 [282:34:28<113:41:21,  4.95s/it]
 63%|██████▎   | 142293/225000 [282:34:32<111:05:31,  4.84s/it]
 63%|██████▎   | 142294/225000 [282:34:43<150:26:37,  6.55s/it]
 63%|██████▎   | 142295/225000 [282:34:48<142:26:35,  6.20s/it]
 63%|██████▎   | 142296/225000 [282:34:52<127:03:16,  5.53s/it]
 63%|██████▎   | 142297/225000 [282:34:57<120:38:55,  5.25s/it]
 63%|██████▎   | 142298/225000 [282:35:01<116:04:45,  5.05s/it]
 63%|██████▎   | 142299/225000 [282:35:06<113:06:10,  4.92s/it]
 63%|██████▎   | 142300/225000 [282:35:18<160:05:08,  6.97s/it]
                                                               
{'loss': '0.7912', 'grad_norm': '0.9823', 'learning_rate': '1.616e-05', 'epoch': '0.6324', 'num_input_tokens_seen': 468475872, 'train_runtime': '1.017e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142300/225000 [282:35:18<160:05:08,  6.97s/it]
 63%|██████▎   | 142301/225000 [282:35:21<135:56:41,  5.92s/it]
 63%|██████▎   | 142302/225000 [282:35:25<119:46:29,  5.21s/it]
 63%|██████▎   | 142303/225000 [282:35:34<145:48:28,  6.35s/it]
 63%|██████▎   | 142304/225000 [282:35:40<142:24:20,  6.20s/it]
 63%|██████▎   | 142305/225000 [282:35:44<126:14:32,  5.50s/it]
 63%|██████▎   | 142306/225000 [282:35:48<116:29:11,  5.07s/it]
 63%|██████▎   | 142307/225000 [282:35:55<129:11:17,  5.62s/it]
 63%|██████▎   | 142308/225000 [282:36:02<142:14:52,  6.19s/it]
 63%|██████▎   | 142309/225000 [282:36:05<121:32:00,  5.29s/it]
 63%|██████▎   | 142310/225000 [282:36:13<141:15:47,  6.15s/it]
 63%|██████▎   | 142311/225000 [282:36:17<123:09:50,  5.36s/it]
 63%|██████▎   | 142312/225000 [282:36:21<112:11:21,  4.88s/it]
 63%|██████▎   | 142313/225000 [282:36:29<137:43:05,  6.00s/it]
 63%|██████▎   | 142314/225000 [282:36:35<134:30:43,  5.86s/it]
 63%|██████▎   | 142315/225000 [282:36:39<121:54:56,  5.31s/it]
 63%|██████▎   | 142316/225000 [282:36:43<113:25:14,  4.94s/it]
 63%|██████▎   | 142317/225000 [282:36:53<151:55:10,  6.61s/it]
 63%|██████▎   | 142318/225000 [282:36:57<133:00:30,  5.79s/it]
 63%|██████▎   | 142319/225000 [282:37:01<119:33:54,  5.21s/it]
 63%|██████▎   | 142320/225000 [282:37:06<118:28:03,  5.16s/it]
 63%|██████▎   | 142321/225000 [282:37:12<123:47:46,  5.39s/it]
 63%|██████▎   | 142322/225000 [282:37:17<122:59:55,  5.36s/it]
 63%|██████▎   | 142323/225000 [282:37:22<117:56:41,  5.14s/it]
 63%|██████▎   | 142324/225000 [282:37:27<115:32:10,  5.03s/it]
 63%|██████▎   | 142325/225000 [282:37:32<116:00:08,  5.05s/it]
 63%|██████▎   | 142326/225000 [282:37:37<119:01:26,  5.18s/it]
 63%|██████▎   | 142327/225000 [282:37:42<117:18:12,  5.11s/it]
 63%|██████▎   | 142328/225000 [282:37:48<118:28:18,  5.16s/it]
 63%|██████▎   | 142329/225000 [282:37:53<118:44:41,  5.17s/it]
 63%|██████▎   | 142330/225000 [282:37:58<118:07:59,  5.14s/it]
 63%|██████▎   | 142331/225000 [282:38:03<119:01:41,  5.18s/it]
 63%|██████▎   | 142332/225000 [282:38:15<165:04:19,  7.19s/it]
 63%|██████▎   | 142333/225000 [282:38:23<169:28:47,  7.38s/it]
 63%|██████▎   | 142334/225000 [282:38:27<146:59:47,  6.40s/it]
 63%|██████▎   | 142335/225000 [282:38:36<164:39:34,  7.17s/it]
 63%|██████▎   | 142336/225000 [282:38:40<143:55:04,  6.27s/it]
 63%|██████▎   | 142337/225000 [282:38:44<130:00:58,  5.66s/it]
 63%|██████▎   | 142338/225000 [282:38:48<119:21:12,  5.20s/it]
 63%|██████▎   | 142339/225000 [282:38:53<116:00:27,  5.05s/it]
 63%|██████▎   | 142340/225000 [282:38:58<114:12:53,  4.97s/it]
 63%|██████▎   | 142341/225000 [282:39:03<117:35:36,  5.12s/it]
 63%|██████▎   | 142342/225000 [282:39:08<116:01:24,  5.05s/it]
 63%|██████▎   | 142343/225000 [282:39:14<117:16:34,  5.11s/it]
 63%|██████▎   | 142344/225000 [282:39:19<118:55:47,  5.18s/it]
 63%|██████▎   | 142345/225000 [282:39:27<141:26:49,  6.16s/it]
 63%|██████▎   | 142346/225000 [282:39:37<167:32:12,  7.30s/it]
 63%|██████▎   | 142347/225000 [282:39:41<139:41:03,  6.08s/it]
 63%|██████▎   | 142348/225000 [282:39:50<160:58:23,  7.01s/it]
 63%|██████▎   | 142349/225000 [282:39:53<132:50:07,  5.79s/it]
 63%|██████▎   | 142350/225000 [282:40:00<146:26:08,  6.38s/it]
                                                               
{'loss': '0.7775', 'grad_norm': '1.17', 'learning_rate': '1.614e-05', 'epoch': '0.6327', 'num_input_tokens_seen': 468603600, 'train_runtime': '1.018e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142350/225000 [282:40:00<146:26:08,  6.38s/it]
 63%|██████▎   | 142351/225000 [282:40:04<127:33:33,  5.56s/it]
 63%|██████▎   | 142352/225000 [282:40:09<123:56:42,  5.40s/it]
 63%|██████▎   | 142353/225000 [282:40:14<122:37:14,  5.34s/it]
 63%|██████▎   | 142354/225000 [282:40:19<119:09:19,  5.19s/it]
 63%|██████▎   | 142355/225000 [282:40:24<120:15:30,  5.24s/it]
 63%|██████▎   | 142356/225000 [282:40:30<119:20:56,  5.20s/it]
 63%|██████▎   | 142357/225000 [282:40:35<121:05:08,  5.27s/it]
 63%|██████▎   | 142358/225000 [282:40:40<121:31:47,  5.29s/it]
 63%|██████▎   | 142359/225000 [282:40:50<152:23:25,  6.64s/it]
 63%|██████▎   | 142360/225000 [282:40:53<129:20:11,  5.63s/it]
 63%|██████▎   | 142361/225000 [282:41:03<155:01:59,  6.75s/it]
 63%|██████▎   | 142362/225000 [282:41:07<140:34:39,  6.12s/it]
 63%|██████▎   | 142363/225000 [282:41:11<126:03:57,  5.49s/it]
 63%|██████▎   | 142364/225000 [282:41:16<121:11:38,  5.28s/it]
 63%|██████▎   | 142365/225000 [282:41:21<117:43:02,  5.13s/it]
 63%|██████▎   | 142366/225000 [282:41:25<112:44:20,  4.91s/it]
 63%|██████▎   | 142367/225000 [282:41:30<110:46:29,  4.83s/it]
 63%|██████▎   | 142368/225000 [282:41:34<107:52:07,  4.70s/it]
 63%|██████▎   | 142369/225000 [282:41:43<131:25:37,  5.73s/it]
 63%|██████▎   | 142370/225000 [282:41:55<175:53:33,  7.66s/it]
 63%|██████▎   | 142371/225000 [282:41:59<151:12:24,  6.59s/it]
 63%|██████▎   | 142372/225000 [282:42:03<133:32:15,  5.82s/it]
 63%|██████▎   | 142373/225000 [282:42:12<158:42:55,  6.92s/it]
 63%|██████▎   | 142374/225000 [282:42:17<144:11:37,  6.28s/it]
 63%|██████▎   | 142375/225000 [282:42:22<133:33:18,  5.82s/it]
 63%|██████▎   | 142376/225000 [282:42:26<122:15:45,  5.33s/it]
 63%|██████▎   | 142377/225000 [282:42:30<112:20:16,  4.89s/it]
 63%|██████▎   | 142378/225000 [282:42:34<108:33:24,  4.73s/it]
 63%|██████▎   | 142379/225000 [282:42:45<151:07:01,  6.58s/it]
 63%|██████▎   | 142380/225000 [282:42:49<129:54:33,  5.66s/it]
 63%|██████▎   | 142381/225000 [282:42:55<130:49:50,  5.70s/it]
 63%|██████▎   | 142382/225000 [282:43:03<148:31:32,  6.47s/it]
 63%|██████▎   | 142383/225000 [282:43:07<136:04:01,  5.93s/it]
 63%|██████▎   | 142384/225000 [282:43:14<140:37:12,  6.13s/it]
 63%|██████▎   | 142385/225000 [282:43:22<149:57:36,  6.53s/it]
 63%|██████▎   | 142386/225000 [282:43:26<134:30:34,  5.86s/it]
 63%|██████▎   | 142387/225000 [282:43:30<120:54:36,  5.27s/it]
 63%|██████▎   | 142388/225000 [282:43:34<112:57:00,  4.92s/it]
 63%|██████▎   | 142389/225000 [282:43:38<108:21:57,  4.72s/it]
 63%|██████▎   | 142390/225000 [282:43:45<122:42:51,  5.35s/it]
 63%|██████▎   | 142391/225000 [282:43:55<158:18:12,  6.90s/it]
 63%|██████▎   | 142392/225000 [282:43:59<132:07:22,  5.76s/it]
 63%|██████▎   | 142393/225000 [282:44:06<145:37:25,  6.35s/it]
 63%|██████▎   | 142394/225000 [282:44:10<126:19:50,  5.51s/it]
 63%|██████▎   | 142395/225000 [282:44:17<137:46:01,  6.00s/it]
 63%|██████▎   | 142396/225000 [282:44:21<122:12:55,  5.33s/it]
 63%|██████▎   | 142397/225000 [282:44:25<112:33:15,  4.91s/it]
 63%|██████▎   | 142398/225000 [282:44:34<143:39:33,  6.26s/it]
 63%|██████▎   | 142399/225000 [282:44:38<125:08:31,  5.45s/it]
 63%|██████▎   | 142400/225000 [282:44:41<114:06:20,  4.97s/it]
                                                               
{'loss': '0.7851', 'grad_norm': '0.9211', 'learning_rate': '1.613e-05', 'epoch': '0.6329', 'num_input_tokens_seen': 468732736, 'train_runtime': '1.018e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142400/225000 [282:44:41<114:06:20,  4.97s/it]
 63%|██████▎   | 142401/225000 [282:44:52<153:02:14,  6.67s/it]
 63%|██████▎   | 142402/225000 [282:44:56<132:21:14,  5.77s/it]
 63%|██████▎   | 142403/225000 [282:45:01<125:43:19,  5.48s/it]
 63%|██████▎   | 142404/225000 [282:45:06<123:19:17,  5.38s/it]
 63%|██████▎   | 142405/225000 [282:45:11<120:43:04,  5.26s/it]
 63%|██████▎   | 142406/225000 [282:45:16<119:34:34,  5.21s/it]
 63%|██████▎   | 142407/225000 [282:45:21<118:28:59,  5.16s/it]
 63%|██████▎   | 142408/225000 [282:45:30<147:05:42,  6.41s/it]
 63%|██████▎   | 142409/225000 [282:45:37<148:31:01,  6.47s/it]
 63%|██████▎   | 142410/225000 [282:45:41<133:21:43,  5.81s/it]
 63%|██████▎   | 142411/225000 [282:45:46<126:24:13,  5.51s/it]
 63%|██████▎   | 142412/225000 [282:45:53<140:22:11,  6.12s/it]
 63%|██████▎   | 142413/225000 [282:45:58<130:23:48,  5.68s/it]
 63%|██████▎   | 142414/225000 [282:46:03<123:32:00,  5.38s/it]
 63%|██████▎   | 142415/225000 [282:46:07<118:01:14,  5.14s/it]
 63%|██████▎   | 142416/225000 [282:46:19<161:28:27,  7.04s/it]
 63%|██████▎   | 142417/225000 [282:46:22<134:48:47,  5.88s/it]
 63%|██████▎   | 142418/225000 [282:46:27<132:26:29,  5.77s/it]
 63%|██████▎   | 142419/225000 [282:46:33<133:45:24,  5.83s/it]
 63%|██████▎   | 142420/225000 [282:46:42<149:11:10,  6.50s/it]
 63%|██████▎   | 142421/225000 [282:46:46<136:05:41,  5.93s/it]
 63%|██████▎   | 142422/225000 [282:46:51<127:23:25,  5.55s/it]
 63%|██████▎   | 142423/225000 [282:46:55<120:17:21,  5.24s/it]
 63%|██████▎   | 142424/225000 [282:47:00<116:24:14,  5.07s/it]
 63%|██████▎   | 142425/225000 [282:47:05<114:10:38,  4.98s/it]
 63%|██████▎   | 142426/225000 [282:47:09<107:50:46,  4.70s/it]
 63%|██████▎   | 142427/225000 [282:47:21<157:08:18,  6.85s/it]
 63%|██████▎   | 142428/225000 [282:47:26<145:50:47,  6.36s/it]
 63%|██████▎   | 142429/225000 [282:47:30<129:40:17,  5.65s/it]
 63%|██████▎   | 142430/225000 [282:47:34<121:30:08,  5.30s/it]
 63%|██████▎   | 142431/225000 [282:47:45<159:36:33,  6.96s/it]
 63%|██████▎   | 142432/225000 [282:47:50<147:34:48,  6.43s/it]
 63%|██████▎   | 142433/225000 [282:47:54<130:31:24,  5.69s/it]
 63%|██████▎   | 142434/225000 [282:47:59<121:49:38,  5.31s/it]
 63%|██████▎   | 142435/225000 [282:48:04<123:58:12,  5.41s/it]
 63%|██████▎   | 142436/225000 [282:48:10<122:28:06,  5.34s/it]
 63%|██████▎   | 142437/225000 [282:48:15<123:45:06,  5.40s/it]
 63%|██████▎   | 142438/225000 [282:48:21<126:24:52,  5.51s/it]
 63%|██████▎   | 142439/225000 [282:48:26<123:27:22,  5.38s/it]
 63%|██████▎   | 142440/225000 [282:48:31<117:51:01,  5.14s/it]
 63%|██████▎   | 142441/225000 [282:48:36<117:37:21,  5.13s/it]
 63%|██████▎   | 142442/225000 [282:48:41<118:25:56,  5.16s/it]
 63%|██████▎   | 142443/225000 [282:48:46<120:43:04,  5.26s/it]
 63%|██████▎   | 142444/225000 [282:48:52<121:04:27,  5.28s/it]
 63%|██████▎   | 142445/225000 [282:49:02<152:49:09,  6.66s/it]
 63%|██████▎   | 142446/225000 [282:49:12<178:15:37,  7.77s/it]
 63%|██████▎   | 142447/225000 [282:49:16<150:07:06,  6.55s/it]
 63%|██████▎   | 142448/225000 [282:49:24<161:07:20,  7.03s/it]
 63%|██████▎   | 142449/225000 [282:49:28<139:44:28,  6.09s/it]
 63%|██████▎   | 142450/225000 [282:49:35<146:13:58,  6.38s/it]
                                                               
{'loss': '0.7883', 'grad_norm': '0.9852', 'learning_rate': '1.611e-05', 'epoch': '0.6331', 'num_input_tokens_seen': 468864352, 'train_runtime': '1.018e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142450/225000 [282:49:35<146:13:58,  6.38s/it]
 63%|██████▎   | 142451/225000 [282:49:40<140:28:53,  6.13s/it]
 63%|██████▎   | 142452/225000 [282:49:44<122:21:04,  5.34s/it]
 63%|██████▎   | 142453/225000 [282:49:51<137:08:30,  5.98s/it]
 63%|██████▎   | 142454/225000 [282:49:57<132:57:44,  5.80s/it]
 63%|██████▎   | 142455/225000 [282:50:01<124:53:09,  5.45s/it]
 63%|██████▎   | 142456/225000 [282:50:06<119:57:19,  5.23s/it]
 63%|██████▎   | 142457/225000 [282:50:11<116:45:50,  5.09s/it]
 63%|██████▎   | 142458/225000 [282:50:15<113:08:37,  4.93s/it]
 63%|██████▎   | 142459/225000 [282:50:20<108:34:21,  4.74s/it]
 63%|██████▎   | 142460/225000 [282:50:32<162:16:52,  7.08s/it]
 63%|██████▎   | 142461/225000 [282:50:36<141:56:25,  6.19s/it]
 63%|██████▎   | 142462/225000 [282:50:40<122:41:37,  5.35s/it]
 63%|██████▎   | 142463/225000 [282:50:49<152:30:41,  6.65s/it]
 63%|██████▎   | 142464/225000 [282:50:53<130:06:33,  5.68s/it]
 63%|██████▎   | 142465/225000 [282:51:00<143:09:59,  6.24s/it]
 63%|██████▎   | 142466/225000 [282:51:03<121:51:46,  5.32s/it]
 63%|██████▎   | 142467/225000 [282:51:09<122:25:30,  5.34s/it]
 63%|██████▎   | 142468/225000 [282:51:14<120:04:37,  5.24s/it]
 63%|██████▎   | 142469/225000 [282:51:19<117:31:17,  5.13s/it]
 63%|██████▎   | 142470/225000 [282:51:23<113:25:36,  4.95s/it]
 63%|██████▎   | 142471/225000 [282:51:28<111:49:27,  4.88s/it]
 63%|██████▎   | 142472/225000 [282:51:33<113:47:27,  4.96s/it]
 63%|██████▎   | 142473/225000 [282:51:38<115:18:38,  5.03s/it]
 63%|██████▎   | 142474/225000 [282:51:44<117:40:44,  5.13s/it]
 63%|██████▎   | 142475/225000 [282:51:53<143:20:42,  6.25s/it]
 63%|██████▎   | 142476/225000 [282:52:04<178:51:18,  7.80s/it]
 63%|██████▎   | 142477/225000 [282:52:08<149:42:01,  6.53s/it]
 63%|██████▎   | 142478/225000 [282:52:11<128:19:00,  5.60s/it]
 63%|██████▎   | 142479/225000 [282:52:21<157:44:39,  6.88s/it]
 63%|██████▎   | 142480/225000 [282:52:24<134:26:49,  5.87s/it]
 63%|██████▎   | 142481/225000 [282:52:33<153:11:11,  6.68s/it]
 63%|██████▎   | 142482/225000 [282:52:36<131:15:32,  5.73s/it]
 63%|██████▎   | 142483/225000 [282:52:43<134:33:06,  5.87s/it]
 63%|██████▎   | 142484/225000 [282:52:49<135:26:01,  5.91s/it]
 63%|██████▎   | 142485/225000 [282:52:56<142:36:02,  6.22s/it]
 63%|██████▎   | 142486/225000 [282:53:02<143:52:07,  6.28s/it]
 63%|██████▎   | 142487/225000 [282:53:05<123:42:07,  5.40s/it]
 63%|██████▎   | 142488/225000 [282:53:10<121:03:12,  5.28s/it]
 63%|██████▎   | 142489/225000 [282:53:15<116:12:24,  5.07s/it]
 63%|██████▎   | 142490/225000 [282:53:20<114:23:14,  4.99s/it]
 63%|██████▎   | 142491/225000 [282:53:25<114:55:07,  5.01s/it]
 63%|██████▎   | 142492/225000 [282:53:30<114:40:21,  5.00s/it]
 63%|██████▎   | 142493/225000 [282:53:35<113:11:24,  4.94s/it]
 63%|██████▎   | 142494/225000 [282:53:40<114:03:26,  4.98s/it]
 63%|██████▎   | 142495/225000 [282:53:45<116:26:26,  5.08s/it]
 63%|██████▎   | 142496/225000 [282:53:57<164:22:54,  7.17s/it]
 63%|██████▎   | 142497/225000 [282:54:01<140:17:24,  6.12s/it]
 63%|██████▎   | 142498/225000 [282:54:08<145:07:47,  6.33s/it]
 63%|██████▎   | 142499/225000 [282:54:12<133:25:59,  5.82s/it]
 63%|██████▎   | 142500/225000 [282:54:17<129:01:44,  5.63s/it]
                                                               
{'loss': '0.7899', 'grad_norm': '1.972', 'learning_rate': '1.609e-05', 'epoch': '0.6333', 'num_input_tokens_seen': 468993112, 'train_runtime': '1.018e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142500/225000 [282:54:17<129:01:44,  5.63s/it]
 63%|██████▎   | 142501/225000 [282:54:22<122:05:09,  5.33s/it]
 63%|██████▎   | 142502/225000 [282:54:26<116:25:26,  5.08s/it]
 63%|██████▎   | 142503/225000 [282:54:31<111:12:51,  4.85s/it]
 63%|██████▎   | 142504/225000 [282:54:41<147:18:49,  6.43s/it]
 63%|██████▎   | 142505/225000 [282:54:46<135:59:11,  5.93s/it]
 63%|██████▎   | 142506/225000 [282:54:51<133:48:54,  5.84s/it]
 63%|██████▎   | 142507/225000 [282:55:01<157:18:32,  6.86s/it]
 63%|██████▎   | 142508/225000 [282:55:04<131:49:40,  5.75s/it]
 63%|██████▎   | 142509/225000 [282:55:11<139:28:54,  6.09s/it]
 63%|██████▎   | 142510/225000 [282:55:15<126:01:30,  5.50s/it]
 63%|██████▎   | 142511/225000 [282:55:19<119:35:49,  5.22s/it]
 63%|██████▎   | 142512/225000 [282:55:24<117:10:34,  5.11s/it]
 63%|██████▎   | 142513/225000 [282:55:29<114:39:43,  5.00s/it]
 63%|██████▎   | 142514/225000 [282:55:35<124:46:58,  5.45s/it]
 63%|██████▎   | 142515/225000 [282:55:48<171:58:08,  7.51s/it]
 63%|██████▎   | 142516/225000 [282:55:52<152:10:46,  6.64s/it]
 63%|██████▎   | 142517/225000 [282:55:57<137:54:46,  6.02s/it]
 63%|██████▎   | 142518/225000 [282:56:02<128:43:28,  5.62s/it]
 63%|██████▎   | 142519/225000 [282:56:06<119:56:53,  5.24s/it]
 63%|██████▎   | 142520/225000 [282:56:11<117:37:13,  5.13s/it]
 63%|██████▎   | 142521/225000 [282:56:16<117:49:14,  5.14s/it]
 63%|██████▎   | 142522/225000 [282:56:20<112:41:50,  4.92s/it]
 63%|██████▎   | 142523/225000 [282:56:25<113:34:40,  4.96s/it]
 63%|██████▎   | 142524/225000 [282:56:30<109:35:13,  4.78s/it]
 63%|██████▎   | 142525/225000 [282:56:35<109:21:05,  4.77s/it]
 63%|██████▎   | 142526/225000 [282:56:40<113:03:07,  4.93s/it]
 63%|██████▎   | 142527/225000 [282:56:44<108:55:53,  4.75s/it]
 63%|██████▎   | 142528/225000 [282:56:55<148:25:58,  6.48s/it]
 63%|██████▎   | 142529/225000 [282:57:02<153:10:27,  6.69s/it]
 63%|██████▎   | 142530/225000 [282:57:09<155:52:48,  6.80s/it]
 63%|██████▎   | 142531/225000 [282:57:12<131:27:12,  5.74s/it]
 63%|██████▎   | 142532/225000 [282:57:17<123:36:45,  5.40s/it]
 63%|██████▎   | 142533/225000 [282:57:22<121:13:07,  5.29s/it]
 63%|██████▎   | 142534/225000 [282:57:27<119:28:25,  5.22s/it]
 63%|██████▎   | 142535/225000 [282:57:32<117:01:54,  5.11s/it]
 63%|██████▎   | 142536/225000 [282:57:39<130:11:41,  5.68s/it]
 63%|██████▎   | 142537/225000 [282:57:49<162:05:11,  7.08s/it]
 63%|██████▎   | 142538/225000 [282:57:52<136:56:40,  5.98s/it]
 63%|██████▎   | 142539/225000 [282:57:57<125:18:54,  5.47s/it]
 63%|██████▎   | 142540/225000 [282:58:05<145:14:56,  6.34s/it]
 63%|██████▎   | 142541/225000 [282:58:14<160:57:02,  7.03s/it]
 63%|██████▎   | 142542/225000 [282:58:17<137:58:25,  6.02s/it]
 63%|██████▎   | 142543/225000 [282:58:21<120:56:11,  5.28s/it]
 63%|██████▎   | 142544/225000 [282:58:30<146:56:11,  6.42s/it]
 63%|██████▎   | 142545/225000 [282:58:34<128:10:17,  5.60s/it]
 63%|██████▎   | 142546/225000 [282:58:38<117:51:29,  5.15s/it]
 63%|██████▎   | 142547/225000 [282:58:42<110:17:22,  4.82s/it]
 63%|██████▎   | 142548/225000 [282:58:46<108:07:27,  4.72s/it]
 63%|██████▎   | 142549/225000 [282:58:57<150:30:11,  6.57s/it]
 63%|██████▎   | 142550/225000 [282:59:03<147:48:14,  6.45s/it]
                                                               
{'loss': '0.7815', 'grad_norm': '1.123', 'learning_rate': '1.607e-05', 'epoch': '0.6336', 'num_input_tokens_seen': 469122216, 'train_runtime': '1.019e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142550/225000 [282:59:03<147:48:14,  6.45s/it]
 63%|██████▎   | 142551/225000 [282:59:07<130:03:19,  5.68s/it]
 63%|██████▎   | 142552/225000 [282:59:11<116:48:35,  5.10s/it]
 63%|██████▎   | 142553/225000 [282:59:22<156:35:06,  6.84s/it]
 63%|██████▎   | 142554/225000 [282:59:26<134:17:57,  5.86s/it]
 63%|██████▎   | 142555/225000 [282:59:29<118:53:52,  5.19s/it]
 63%|██████▎   | 142556/225000 [282:59:39<152:21:14,  6.65s/it]
 63%|██████▎   | 142557/225000 [282:59:42<128:54:26,  5.63s/it]
 63%|██████▎   | 142558/225000 [282:59:47<121:17:20,  5.30s/it]
 63%|██████▎   | 142559/225000 [282:59:52<118:48:12,  5.19s/it]
 63%|██████▎   | 142560/225000 [282:59:57<114:41:26,  5.01s/it]
 63%|██████▎   | 142561/225000 [283:00:03<127:06:17,  5.55s/it]
 63%|██████▎   | 142562/225000 [283:00:12<148:58:36,  6.51s/it]
 63%|██████▎   | 142563/225000 [283:00:16<131:38:47,  5.75s/it]
 63%|██████▎   | 142564/225000 [283:00:20<118:30:08,  5.18s/it]
 63%|██████▎   | 142565/225000 [283:00:29<146:39:14,  6.40s/it]
 63%|██████▎   | 142566/225000 [283:00:35<143:26:19,  6.26s/it]
 63%|██████▎   | 142567/225000 [283:00:40<133:40:39,  5.84s/it]
 63%|██████▎   | 142568/225000 [283:00:47<143:58:24,  6.29s/it]
 63%|██████▎   | 142569/225000 [283:00:51<127:50:56,  5.58s/it]
 63%|██████▎   | 142570/225000 [283:00:57<126:52:40,  5.54s/it]
 63%|██████▎   | 142571/225000 [283:01:06<154:12:37,  6.73s/it]
 63%|██████▎   | 142572/225000 [283:01:10<132:25:53,  5.78s/it]
 63%|██████▎   | 142573/225000 [283:01:15<125:33:21,  5.48s/it]
 63%|██████▎   | 142574/225000 [283:01:23<148:16:14,  6.48s/it]
 63%|██████▎   | 142575/225000 [283:01:27<126:02:15,  5.50s/it]
 63%|██████▎   | 142576/225000 [283:01:34<139:37:08,  6.10s/it]
 63%|██████▎   | 142577/225000 [283:01:38<122:22:11,  5.34s/it]
 63%|██████▎   | 142578/225000 [283:01:44<132:43:27,  5.80s/it]
 63%|██████▎   | 142579/225000 [283:01:48<115:34:54,  5.05s/it]
 63%|██████▎   | 142580/225000 [283:01:54<126:05:45,  5.51s/it]
 63%|██████▎   | 142581/225000 [283:02:00<127:58:36,  5.59s/it]
 63%|██████▎   | 142582/225000 [283:02:04<115:05:27,  5.03s/it]
 63%|██████▎   | 142583/225000 [283:02:13<141:04:48,  6.16s/it]
 63%|██████▎   | 142584/225000 [283:02:17<130:13:19,  5.69s/it]
 63%|██████▎   | 142585/225000 [283:02:23<130:15:02,  5.69s/it]
 63%|██████▎   | 142586/225000 [283:02:28<124:53:06,  5.46s/it]
 63%|██████▎   | 142587/225000 [283:02:33<124:12:19,  5.43s/it]
 63%|██████▎   | 142588/225000 [283:02:38<122:36:59,  5.36s/it]
 63%|██████▎   | 142589/225000 [283:02:43<119:44:04,  5.23s/it]
 63%|██████▎   | 142590/225000 [283:02:49<121:08:26,  5.29s/it]
 63%|██████▎   | 142591/225000 [283:02:54<123:13:44,  5.38s/it]
 63%|██████▎   | 142592/225000 [283:03:00<122:16:52,  5.34s/it]
 63%|██████▎   | 142593/225000 [283:03:04<119:08:47,  5.20s/it]
 63%|██████▎   | 142594/225000 [283:03:11<130:15:04,  5.69s/it]
 63%|██████▎   | 142595/225000 [283:03:17<130:25:13,  5.70s/it]
 63%|██████▎   | 142596/225000 [283:03:22<127:57:27,  5.59s/it]
 63%|██████▎   | 142597/225000 [283:03:27<123:01:06,  5.37s/it]
 63%|██████▎   | 142598/225000 [283:03:32<119:38:15,  5.23s/it]
 63%|██████▎   | 142599/225000 [283:03:37<120:36:43,  5.27s/it]
 63%|██████▎   | 142600/225000 [283:03:42<118:47:12,  5.19s/it]
                                                               
{'loss': '0.778', 'grad_norm': '1.111', 'learning_rate': '1.606e-05', 'epoch': '0.6338', 'num_input_tokens_seen': 469251888, 'train_runtime': '1.019e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142600/225000 [283:03:42<118:47:12,  5.19s/it]
 63%|██████▎   | 142601/225000 [283:03:48<120:19:35,  5.26s/it]
 63%|██████▎   | 142602/225000 [283:03:56<140:18:26,  6.13s/it]
 63%|██████▎   | 142603/225000 [283:04:06<168:33:52,  7.36s/it]
 63%|██████▎   | 142604/225000 [283:04:09<139:21:40,  6.09s/it]
 63%|██████▎   | 142605/225000 [283:04:16<142:34:43,  6.23s/it]
 63%|██████▎   | 142606/225000 [283:04:22<140:10:51,  6.12s/it]
 63%|██████▎   | 142607/225000 [283:04:25<119:45:37,  5.23s/it]
 63%|██████▎   | 142608/225000 [283:04:34<146:29:13,  6.40s/it]
 63%|██████▎   | 142609/225000 [283:04:37<122:32:23,  5.35s/it]
 63%|██████▎   | 142610/225000 [283:04:45<136:56:19,  5.98s/it]
 63%|██████▎   | 142611/225000 [283:04:49<128:19:47,  5.61s/it]
 63%|██████▎   | 142612/225000 [283:04:54<119:49:24,  5.24s/it]
 63%|██████▎   | 142613/225000 [283:04:58<115:20:16,  5.04s/it]
 63%|██████▎   | 142614/225000 [283:05:03<112:03:54,  4.90s/it]
 63%|██████▎   | 142615/225000 [283:05:08<111:26:47,  4.87s/it]
 63%|██████▎   | 142616/225000 [283:05:12<110:23:32,  4.82s/it]
 63%|██████▎   | 142617/225000 [283:05:22<146:25:34,  6.40s/it]
 63%|██████▎   | 142618/225000 [283:05:30<156:17:22,  6.83s/it]
 63%|██████▎   | 142619/225000 [283:05:34<138:28:10,  6.05s/it]
 63%|██████▎   | 142620/225000 [283:05:38<123:46:11,  5.41s/it]
 63%|██████▎   | 142621/225000 [283:05:49<157:15:46,  6.87s/it]
 63%|██████▎   | 142622/225000 [283:05:52<136:03:13,  5.95s/it]
 63%|██████▎   | 142623/225000 [283:05:56<122:46:23,  5.37s/it]
 63%|██████▎   | 142624/225000 [283:06:02<126:48:27,  5.54s/it]
 63%|██████▎   | 142625/225000 [283:06:10<141:45:00,  6.19s/it]
 63%|██████▎   | 142626/225000 [283:06:15<131:07:41,  5.73s/it]
 63%|██████▎   | 142627/225000 [283:06:20<128:36:42,  5.62s/it]
 63%|██████▎   | 142628/225000 [283:06:25<123:35:29,  5.40s/it]
 63%|██████▎   | 142629/225000 [283:06:30<121:40:10,  5.32s/it]
 63%|██████▎   | 142630/225000 [283:06:35<117:47:31,  5.15s/it]
 63%|██████▎   | 142631/225000 [283:06:40<118:14:26,  5.17s/it]
 63%|██████▎   | 142632/225000 [283:06:45<118:21:36,  5.17s/it]
 63%|██████▎   | 142633/225000 [283:06:51<119:10:02,  5.21s/it]
 63%|██████▎   | 142634/225000 [283:06:56<118:59:12,  5.20s/it]
 63%|██████▎   | 142635/225000 [283:07:01<117:46:44,  5.15s/it]
 63%|██████▎   | 142636/225000 [283:07:09<137:14:47,  6.00s/it]
 63%|██████▎   | 142637/225000 [283:07:21<176:52:45,  7.73s/it]
 63%|██████▎   | 142638/225000 [283:07:25<152:32:02,  6.67s/it]
 63%|██████▎   | 142639/225000 [283:07:31<152:05:30,  6.65s/it]
 63%|██████▎   | 142640/225000 [283:07:35<129:26:02,  5.66s/it]
 63%|██████▎   | 142641/225000 [283:07:43<145:56:27,  6.38s/it]
 63%|██████▎   | 142642/225000 [283:07:47<129:34:46,  5.66s/it]
 63%|██████▎   | 142643/225000 [283:07:53<130:35:45,  5.71s/it]
 63%|██████▎   | 142644/225000 [283:08:01<147:39:20,  6.45s/it]
 63%|██████▎   | 142645/225000 [283:08:06<142:23:44,  6.22s/it]
 63%|██████▎   | 142646/225000 [283:08:12<135:30:57,  5.92s/it]
 63%|██████▎   | 142647/225000 [283:08:19<148:14:02,  6.48s/it]
 63%|██████▎   | 142648/225000 [283:08:26<147:45:42,  6.46s/it]
 63%|██████▎   | 142649/225000 [283:08:29<126:56:19,  5.55s/it]
 63%|██████▎   | 142650/225000 [283:08:33<116:24:46,  5.09s/it]
                                                               
{'loss': '0.7901', 'grad_norm': '1.025', 'learning_rate': '1.604e-05', 'epoch': '0.634', 'num_input_tokens_seen': 469382000, 'train_runtime': '1.019e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142650/225000 [283:08:33<116:24:46,  5.09s/it]
 63%|██████▎   | 142651/225000 [283:08:44<157:30:24,  6.89s/it]
 63%|██████▎   | 142652/225000 [283:08:48<133:19:02,  5.83s/it]
 63%|██████▎   | 142653/225000 [283:08:55<146:13:21,  6.39s/it]
 63%|██████▎   | 142654/225000 [283:09:00<133:08:32,  5.82s/it]
 63%|██████▎   | 142655/225000 [283:09:05<126:37:31,  5.54s/it]
 63%|██████▎   | 142656/225000 [283:09:09<120:16:02,  5.26s/it]
 63%|██████▎   | 142657/225000 [283:09:14<117:27:38,  5.14s/it]
 63%|██████▎   | 142658/225000 [283:09:19<114:19:19,  5.00s/it]
 63%|██████▎   | 142659/225000 [283:09:24<112:53:54,  4.94s/it]
 63%|██████▎   | 142660/225000 [283:09:28<110:42:08,  4.84s/it]
 63%|██████▎   | 142661/225000 [283:09:39<150:06:06,  6.56s/it]
 63%|██████▎   | 142662/225000 [283:09:49<171:41:09,  7.51s/it]
 63%|██████▎   | 142663/225000 [283:09:52<141:43:57,  6.20s/it]
 63%|██████▎   | 142664/225000 [283:09:59<150:01:09,  6.56s/it]
 63%|██████▎   | 142665/225000 [283:10:03<129:09:10,  5.65s/it]
 63%|██████▎   | 142666/225000 [283:10:07<121:51:25,  5.33s/it]
 63%|██████▎   | 142667/225000 [283:10:13<122:00:47,  5.34s/it]
 63%|██████▎   | 142668/225000 [283:10:18<123:25:31,  5.40s/it]
 63%|██████▎   | 142669/225000 [283:10:23<122:16:25,  5.35s/it]
 63%|██████▎   | 142670/225000 [283:10:28<117:55:21,  5.16s/it]
 63%|██████▎   | 142671/225000 [283:10:33<117:39:36,  5.14s/it]
 63%|██████▎   | 142672/225000 [283:10:38<118:18:54,  5.17s/it]
 63%|██████▎   | 142673/225000 [283:10:44<120:33:39,  5.27s/it]
 63%|██████▎   | 142674/225000 [283:10:49<120:57:01,  5.29s/it]
 63%|██████▎   | 142675/225000 [283:11:00<158:58:18,  6.95s/it]
 63%|██████▎   | 142676/225000 [283:11:07<155:26:28,  6.80s/it]
 63%|██████▎   | 142677/225000 [283:11:10<134:44:52,  5.89s/it]
 63%|██████▎   | 142678/225000 [283:11:21<165:16:18,  7.23s/it]
 63%|██████▎   | 142679/225000 [283:11:24<140:36:14,  6.15s/it]
 63%|██████▎   | 142680/225000 [283:11:28<124:22:16,  5.44s/it]
 63%|██████▎   | 142681/225000 [283:11:34<124:23:38,  5.44s/it]
 63%|██████▎   | 142682/225000 [283:11:44<156:05:47,  6.83s/it]
 63%|██████▎   | 142683/225000 [283:11:48<137:58:25,  6.03s/it]
 63%|██████▎   | 142684/225000 [283:11:54<139:45:46,  6.11s/it]
 63%|██████▎   | 142685/225000 [283:12:01<142:13:54,  6.22s/it]
 63%|██████▎   | 142686/225000 [283:12:04<124:45:36,  5.46s/it]
 63%|██████▎   | 142687/225000 [283:12:12<144:10:23,  6.31s/it]
 63%|██████▎   | 142688/225000 [283:12:18<137:49:35,  6.03s/it]
 63%|██████▎   | 142689/225000 [283:12:25<148:14:34,  6.48s/it]
 63%|██████▎   | 142690/225000 [283:12:29<126:29:15,  5.53s/it]
 63%|██████▎   | 142691/225000 [283:12:37<142:15:34,  6.22s/it]
 63%|██████▎   | 142692/225000 [283:12:40<124:09:22,  5.43s/it]
 63%|██████▎   | 142693/225000 [283:12:46<128:25:07,  5.62s/it]
 63%|██████▎   | 142694/225000 [283:12:52<131:01:20,  5.73s/it]
 63%|██████▎   | 142695/225000 [283:12:57<121:27:41,  5.31s/it]
 63%|██████▎   | 142696/225000 [283:13:00<110:59:05,  4.85s/it]
 63%|██████▎   | 142697/225000 [283:13:05<107:50:20,  4.72s/it]
 63%|██████▎   | 142698/225000 [283:13:09<107:08:31,  4.69s/it]
 63%|██████▎   | 142699/225000 [283:13:21<152:29:28,  6.67s/it]
 63%|██████▎   | 142700/225000 [283:13:24<132:36:19,  5.80s/it]
                                                               
{'loss': '0.79', 'grad_norm': '1.094', 'learning_rate': '1.602e-05', 'epoch': '0.6342', 'num_input_tokens_seen': 469511320, 'train_runtime': '1.02e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142700/225000 [283:13:24<132:36:19,  5.80s/it]
 63%|██████▎   | 142701/225000 [283:13:30<127:54:57,  5.60s/it]
 63%|██████▎   | 142702/225000 [283:13:35<125:39:04,  5.50s/it]
 63%|██████▎   | 142703/225000 [283:13:40<125:56:07,  5.51s/it]
 63%|██████▎   | 142704/225000 [283:13:52<168:37:03,  7.38s/it]
 63%|██████▎   | 142705/225000 [283:13:55<141:24:44,  6.19s/it]
 63%|██████▎   | 142706/225000 [283:14:03<150:25:41,  6.58s/it]
 63%|██████▎   | 142707/225000 [283:14:08<139:51:08,  6.12s/it]
 63%|██████▎   | 142708/225000 [283:14:12<122:17:52,  5.35s/it]
 63%|██████▎   | 142709/225000 [283:14:19<133:27:39,  5.84s/it]
 63%|██████▎   | 142710/225000 [283:14:26<143:08:19,  6.26s/it]
 63%|██████▎   | 142711/225000 [283:14:33<148:51:52,  6.51s/it]
 63%|██████▎   | 142712/225000 [283:14:37<131:01:22,  5.73s/it]
 63%|██████▎   | 142713/225000 [283:14:46<151:31:25,  6.63s/it]
 63%|██████▎   | 142714/225000 [283:14:50<133:24:46,  5.84s/it]
 63%|██████▎   | 142715/225000 [283:14:53<119:10:47,  5.21s/it]
 63%|██████▎   | 142716/225000 [283:15:00<131:27:20,  5.75s/it]
 63%|██████▎   | 142717/225000 [283:15:07<135:52:05,  5.94s/it]
 63%|██████▎   | 142718/225000 [283:15:11<123:31:45,  5.40s/it]
 63%|██████▎   | 142719/225000 [283:15:15<115:58:33,  5.07s/it]
 63%|██████▎   | 142720/225000 [283:15:19<109:33:20,  4.79s/it]
 63%|██████▎   | 142721/225000 [283:15:30<147:01:39,  6.43s/it]
 63%|██████▎   | 142722/225000 [283:15:36<146:36:57,  6.42s/it]
 63%|██████▎   | 142723/225000 [283:15:39<126:37:50,  5.54s/it]
 63%|██████▎   | 142724/225000 [283:15:44<117:28:05,  5.14s/it]
 63%|██████▎   | 142725/225000 [283:15:52<140:27:36,  6.15s/it]
 63%|██████▎   | 142726/225000 [283:15:56<128:24:56,  5.62s/it]
 63%|██████▎   | 142727/225000 [283:16:05<145:15:44,  6.36s/it]
 63%|██████▎   | 142728/225000 [283:16:08<128:08:00,  5.61s/it]
 63%|██████▎   | 142729/225000 [283:16:12<117:35:13,  5.15s/it]
 63%|██████▎   | 142730/225000 [283:16:24<161:21:08,  7.06s/it]
 63%|██████▎   | 142731/225000 [283:16:27<136:03:45,  5.95s/it]
 63%|██████▎   | 142732/225000 [283:16:34<140:29:42,  6.15s/it]
 63%|██████▎   | 142733/225000 [283:16:39<133:43:02,  5.85s/it]
 63%|██████▎   | 142734/225000 [283:16:44<128:27:46,  5.62s/it]
 63%|██████▎   | 142735/225000 [283:16:49<122:10:12,  5.35s/it]
 63%|██████▎   | 142736/225000 [283:16:58<145:59:55,  6.39s/it]
 63%|██████▎   | 142737/225000 [283:17:01<126:24:42,  5.53s/it]
 63%|██████▎   | 142738/225000 [283:17:05<113:54:22,  4.98s/it]
 63%|██████▎   | 142739/225000 [283:17:14<140:48:32,  6.16s/it]
 63%|██████▎   | 142740/225000 [283:17:18<123:32:41,  5.41s/it]
 63%|██████▎   | 142741/225000 [283:17:21<112:58:36,  4.94s/it]
 63%|██████▎   | 142742/225000 [283:17:32<149:32:34,  6.54s/it]
 63%|██████▎   | 142743/225000 [283:17:35<128:33:29,  5.63s/it]
 63%|██████▎   | 142744/225000 [283:17:40<125:41:44,  5.50s/it]
 63%|██████▎   | 142745/225000 [283:17:45<123:03:24,  5.39s/it]
 63%|██████▎   | 142746/225000 [283:17:51<121:54:49,  5.34s/it]
 63%|██████▎   | 142747/225000 [283:17:56<121:43:58,  5.33s/it]
 63%|██████▎   | 142748/225000 [283:18:01<118:56:57,  5.21s/it]
 63%|██████▎   | 142749/225000 [283:18:06<117:35:05,  5.15s/it]
 63%|██████▎   | 142750/225000 [283:18:12<121:19:36,  5.31s/it]
                                                               
{'loss': '0.7832', 'grad_norm': '0.9545', 'learning_rate': '1.601e-05', 'epoch': '0.6344', 'num_input_tokens_seen': 469640120, 'train_runtime': '1.02e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142750/225000 [283:18:12<121:19:36,  5.31s/it]
 63%|██████▎   | 142751/225000 [283:18:19<134:49:51,  5.90s/it]
 63%|██████▎   | 142752/225000 [283:18:27<151:27:17,  6.63s/it]
 63%|██████▎   | 142753/225000 [283:18:33<144:08:52,  6.31s/it]
 63%|██████▎   | 142754/225000 [283:18:38<139:10:06,  6.09s/it]
 63%|██████▎   | 142755/225000 [283:18:47<154:24:35,  6.76s/it]
 63%|██████▎   | 142756/225000 [283:18:51<134:21:07,  5.88s/it]
 63%|██████▎   | 142757/225000 [283:18:55<122:55:28,  5.38s/it]
 63%|██████▎   | 142758/225000 [283:18:59<114:25:30,  5.01s/it]
 63%|██████▎   | 142759/225000 [283:19:10<159:18:26,  6.97s/it]
 63%|██████▎   | 142760/225000 [283:19:14<138:00:02,  6.04s/it]
 63%|██████▎   | 142761/225000 [283:19:21<144:57:29,  6.35s/it]
 63%|██████▎   | 142762/225000 [283:19:25<126:43:57,  5.55s/it]
 63%|██████▎   | 142763/225000 [283:19:30<120:17:23,  5.27s/it]
 63%|██████▎   | 142764/225000 [283:19:35<118:09:38,  5.17s/it]
 63%|██████▎   | 142765/225000 [283:19:41<128:15:55,  5.62s/it]
 63%|██████▎   | 142766/225000 [283:19:52<165:32:49,  7.25s/it]
 63%|██████▎   | 142767/225000 [283:19:56<140:05:07,  6.13s/it]
 63%|██████▎   | 142768/225000 [283:20:02<140:53:37,  6.17s/it]
 63%|██████▎   | 142769/225000 [283:20:09<144:26:12,  6.32s/it]
 63%|██████▎   | 142770/225000 [283:20:14<136:08:55,  5.96s/it]
 63%|██████▎   | 142771/225000 [283:20:18<124:10:27,  5.44s/it]
 63%|██████▎   | 142772/225000 [283:20:22<113:41:11,  4.98s/it]
 63%|██████▎   | 142773/225000 [283:20:33<154:23:45,  6.76s/it]
 63%|██████▎   | 142774/225000 [283:20:37<137:58:04,  6.04s/it]
 63%|██████▎   | 142775/225000 [283:20:42<125:40:18,  5.50s/it]
 63%|██████▎   | 142776/225000 [283:20:46<117:10:59,  5.13s/it]
 63%|██████▎   | 142777/225000 [283:20:55<147:21:18,  6.45s/it]
 63%|██████▎   | 142778/225000 [283:21:04<159:46:22,  7.00s/it]
 63%|██████▎   | 142779/225000 [283:21:07<135:53:33,  5.95s/it]
 63%|██████▎   | 142780/225000 [283:21:12<125:33:15,  5.50s/it]
 63%|██████▎   | 142781/225000 [283:21:17<121:55:13,  5.34s/it]
 63%|██████▎   | 142782/225000 [283:21:22<121:52:54,  5.34s/it]
 63%|██████▎   | 142783/225000 [283:21:26<115:04:23,  5.04s/it]
 63%|██████▎   | 142784/225000 [283:21:31<116:21:31,  5.10s/it]
 63%|██████▎   | 142785/225000 [283:21:44<169:16:38,  7.41s/it]
 63%|██████▎   | 142786/225000 [283:21:49<152:25:02,  6.67s/it]
 63%|██████▎   | 142787/225000 [283:21:56<151:36:40,  6.64s/it]
 63%|██████▎   | 142788/225000 [283:22:01<143:20:32,  6.28s/it]
 63%|██████▎   | 142789/225000 [283:22:10<158:51:14,  6.96s/it]
 63%|██████▎   | 142790/225000 [283:22:13<133:58:36,  5.87s/it]
 63%|██████▎   | 142791/225000 [283:22:19<132:12:32,  5.79s/it]
 63%|██████▎   | 142792/225000 [283:22:24<130:06:06,  5.70s/it]
 63%|██████▎   | 142793/225000 [283:22:29<127:23:54,  5.58s/it]
 63%|██████▎   | 142794/225000 [283:22:35<123:49:46,  5.42s/it]
 63%|██████▎   | 142795/225000 [283:22:40<125:17:15,  5.49s/it]
 63%|██████▎   | 142796/225000 [283:22:47<137:54:54,  6.04s/it]
 63%|██████▎   | 142797/225000 [283:22:54<142:45:07,  6.25s/it]
 63%|██████▎   | 142798/225000 [283:22:58<128:20:57,  5.62s/it]
 63%|██████▎   | 142799/225000 [283:23:03<119:25:47,  5.23s/it]
 63%|██████▎   | 142800/225000 [283:23:07<115:17:30,  5.05s/it]
                                                               
{'loss': '0.7992', 'grad_norm': '1.151', 'learning_rate': '1.599e-05', 'epoch': '0.6347', 'num_input_tokens_seen': 469770568, 'train_runtime': '1.02e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142800/225000 [283:23:07<115:17:30,  5.05s/it]
 63%|██████▎   | 142801/225000 [283:23:18<156:46:55,  6.87s/it]
 63%|██████▎   | 142802/225000 [283:23:22<134:19:53,  5.88s/it]
 63%|██████▎   | 142803/225000 [283:23:27<131:21:15,  5.75s/it]
 63%|██████▎   | 142804/225000 [283:23:34<136:31:47,  5.98s/it]
 63%|██████▎   | 142805/225000 [283:23:38<123:35:36,  5.41s/it]
 63%|██████▎   | 142806/225000 [283:23:42<116:42:41,  5.11s/it]
 63%|██████▎   | 142807/225000 [283:23:47<110:58:21,  4.86s/it]
 63%|██████▎   | 142808/225000 [283:23:51<108:24:47,  4.75s/it]
 63%|██████▎   | 142809/225000 [283:24:02<151:41:12,  6.64s/it]
 63%|██████▎   | 142810/225000 [283:24:08<146:54:04,  6.43s/it]
 63%|██████▎   | 142811/225000 [283:24:13<135:13:54,  5.92s/it]
 63%|██████▎   | 142812/225000 [283:24:18<129:48:24,  5.69s/it]
 63%|██████▎   | 142813/225000 [283:24:24<130:50:25,  5.73s/it]
 63%|██████▎   | 142814/225000 [283:24:29<125:55:14,  5.52s/it]
 63%|██████▎   | 142815/225000 [283:24:34<123:06:05,  5.39s/it]
 63%|██████▎   | 142816/225000 [283:24:40<123:41:24,  5.42s/it]
 63%|██████▎   | 142817/225000 [283:24:44<117:58:10,  5.17s/it]
 63%|██████▎   | 142818/225000 [283:24:49<118:14:33,  5.18s/it]
 63%|██████▎   | 142819/225000 [283:24:54<114:26:19,  5.01s/it]
 63%|██████▎   | 142820/225000 [283:24:59<117:09:37,  5.13s/it]
 63%|██████▎   | 142821/225000 [283:25:05<117:35:09,  5.15s/it]
 63%|██████▎   | 142822/225000 [283:25:15<151:50:18,  6.65s/it]
 63%|██████▎   | 142823/225000 [283:25:24<169:41:08,  7.43s/it]
 63%|██████▎   | 142824/225000 [283:25:27<142:43:19,  6.25s/it]
 63%|██████▎   | 142825/225000 [283:25:36<161:12:48,  7.06s/it]
 63%|██████▎   | 142826/225000 [283:25:41<143:09:40,  6.27s/it]
 63%|██████▎   | 142827/225000 [283:25:44<122:33:56,  5.37s/it]
 63%|██████▎   | 142828/225000 [283:25:55<161:33:52,  7.08s/it]
 63%|██████▎   | 142829/225000 [283:25:59<136:15:37,  5.97s/it]
 63%|██████▎   | 142830/225000 [283:26:03<125:50:40,  5.51s/it]
 63%|██████▎   | 142831/225000 [283:26:07<118:20:11,  5.18s/it]
 63%|██████▎   | 142832/225000 [283:26:12<115:09:36,  5.05s/it]
 63%|██████▎   | 142833/225000 [283:26:17<111:04:53,  4.87s/it]
 63%|██████▎   | 142834/225000 [283:26:29<162:23:29,  7.11s/it]
 63%|██████▎   | 142835/225000 [283:26:33<141:15:15,  6.19s/it]
 63%|██████▎   | 142836/225000 [283:26:37<123:52:48,  5.43s/it]
 63%|██████▎   | 142837/225000 [283:26:41<114:47:45,  5.03s/it]
 63%|██████▎   | 142838/225000 [283:26:45<111:23:03,  4.88s/it]
 63%|██████▎   | 142839/225000 [283:26:57<159:02:49,  6.97s/it]
 63%|██████▎   | 142840/225000 [283:27:02<146:24:16,  6.42s/it]
 63%|██████▎   | 142841/225000 [283:27:11<162:07:42,  7.10s/it]
 63%|██████▎   | 142842/225000 [283:27:15<137:42:12,  6.03s/it]
 63%|██████▎   | 142843/225000 [283:27:19<124:52:34,  5.47s/it]
 63%|██████▎   | 142844/225000 [283:27:27<141:39:25,  6.21s/it]
 63%|██████▎   | 142845/225000 [283:27:30<122:52:47,  5.38s/it]
 63%|██████▎   | 142846/225000 [283:27:34<111:42:08,  4.89s/it]
 63%|██████▎   | 142847/225000 [283:27:43<143:08:13,  6.27s/it]
 63%|██████▎   | 142848/225000 [283:27:50<146:11:22,  6.41s/it]
 63%|██████▎   | 142849/225000 [283:27:56<142:24:57,  6.24s/it]
 63%|██████▎   | 142850/225000 [283:28:00<129:30:31,  5.68s/it]
                                                               
{'loss': '0.7888', 'grad_norm': '1.101', 'learning_rate': '1.597e-05', 'epoch': '0.6349', 'num_input_tokens_seen': 469900968, 'train_runtime': '1.02e+06', 'train_tokens_per_second': '460.5'}

 63%|██████▎   | 142850/225000 [283:28:00<129:30:31,  5.68s/it]
 63%|██████▎   | 142851/225000 [283:28:04<119:57:42,  5.26s/it]
 63%|██████▎   | 142852/225000 [283:28:09<115:53:01,  5.08s/it]
 63%|██████▎   | 142853/225000 [283:28:13<109:10:43,  4.78s/it]
 63%|██████▎   | 142854/225000 [283:28:18<108:23:06,  4.75s/it]
 63%|██████▎   | 142855/225000 [283:28:24<117:13:33,  5.14s/it]
 63%|██████▎   | 142856/225000 [283:28:36<161:32:02,  7.08s/it]
 63%|██████▎   | 142857/225000 [283:28:39<136:21:51,  5.98s/it]
 63%|██████▎   | 142858/225000 [283:28:46<143:39:19,  6.30s/it]
 63%|██████▎   | 142859/225000 [283:28:49<123:15:42,  5.40s/it]
 63%|██████▎   | 142860/225000 [283:28:58<146:11:35,  6.41s/it]
 63%|██████▎   | 142861/225000 [283:29:04<145:52:17,  6.39s/it]
 63%|██████▎   | 142862/225000 [283:29:09<136:21:36,  5.98s/it]
 63%|██████▎   | 142863/225000 [283:29:15<132:42:46,  5.82s/it]
 63%|██████▎   | 142864/225000 [283:29:20<128:42:10,  5.64s/it]
 63%|██████▎   | 142865/225000 [283:29:26<126:56:38,  5.56s/it]
 63%|██████▎   | 142866/225000 [283:29:31<125:04:43,  5.48s/it]
 63%|██████▎   | 142867/225000 [283:29:36<122:23:42,  5.36s/it]
 63%|██████▎   | 142868/225000 [283:29:41<120:51:24,  5.30s/it]
 63%|██████▎   | 142869/225000 [283:29:46<121:36:11,  5.33s/it]
 63%|██████▎   | 142870/225000 [283:29:52<120:48:08,  5.30s/it]
 63%|██████▎   | 142871/225000 [283:29:56<117:26:47,  5.15s/it]
 63%|██████▎   | 142872/225000 [283:30:02<117:50:20,  5.17s/it]
 63%|██████▎   | 142873/225000 [283:30:13<162:35:51,  7.13s/it]
 63%|██████▎   | 142874/225000 [283:30:20<161:49:09,  7.09s/it]
 64%|██████▎   | 142875/225000 [283:30:24<139:53:07,  6.13s/it]
 64%|██████▎   | 142876/225000 [283:30:31<146:43:23,  6.43s/it]
 64%|██████▎   | 142877/225000 [283:30:35<127:54:28,  5.61s/it]
 64%|██████▎   | 142878/225000 [283:30:45<160:39:41,  7.04s/it]
 64%|██████▎   | 142879/225000 [283:30:49<137:57:16,  6.05s/it]
 64%|██████▎   | 142880/225000 [283:30:54<130:19:15,  5.71s/it]
 64%|██████▎   | 142881/225000 [283:31:02<142:31:31,  6.25s/it]
 64%|██████▎   | 142882/225000 [283:31:10<155:15:12,  6.81s/it]
 64%|██████▎   | 142883/225000 [283:31:13<131:29:20,  5.76s/it]
 64%|██████▎   | 142884/225000 [283:31:20<138:14:59,  6.06s/it]
 64%|██████▎   | 142885/225000 [283:31:25<130:23:07,  5.72s/it]
 64%|██████▎   | 142886/225000 [283:31:29<121:41:02,  5.33s/it]
 64%|██████▎   | 142887/225000 [283:31:34<117:37:57,  5.16s/it]
 64%|██████▎   | 142888/225000 [283:31:39<116:08:11,  5.09s/it]
 64%|██████▎   | 142889/225000 [283:31:44<113:19:07,  4.97s/it]
 64%|██████▎   | 142890/225000 [283:31:48<111:36:24,  4.89s/it]
 64%|██████▎   | 142891/225000 [283:31:53<110:11:29,  4.83s/it]
 64%|██████▎   | 142892/225000 [283:31:58<111:45:58,  4.90s/it]
 64%|██████▎   | 142893/225000 [283:32:03<114:27:36,  5.02s/it]
 64%|██████▎   | 142894/225000 [283:32:16<163:56:27,  7.19s/it]
 64%|██████▎   | 142895/225000 [283:32:20<142:07:29,  6.23s/it]
 64%|██████▎   | 142896/225000 [283:32:23<125:35:50,  5.51s/it]
 64%|██████▎   | 142897/225000 [283:32:31<142:00:23,  6.23s/it]
 64%|██████▎   | 142898/225000 [283:32:39<154:25:14,  6.77s/it]
 64%|██████▎   | 142899/225000 [283:32:43<131:07:56,  5.75s/it]
 64%|██████▎   | 142900/225000 [283:32:51<150:06:30,  6.58s/it]
                                                               
{'loss': '0.7896', 'grad_norm': '1.13', 'learning_rate': '1.596e-05', 'epoch': '0.6351', 'num_input_tokens_seen': 470030760, 'train_runtime': '1.021e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 142900/225000 [283:32:51<150:06:30,  6.58s/it]
 64%|██████▎   | 142901/225000 [283:32:57<147:15:06,  6.46s/it]
 64%|██████▎   | 142902/225000 [283:33:01<126:49:52,  5.56s/it]
 64%|██████▎   | 142903/225000 [283:33:08<141:03:14,  6.19s/it]
 64%|██████▎   | 142904/225000 [283:33:12<123:04:19,  5.40s/it]
 64%|██████▎   | 142905/225000 [283:33:20<141:37:54,  6.21s/it]
 64%|██████▎   | 142906/225000 [283:33:25<132:07:28,  5.79s/it]
 64%|██████▎   | 142907/225000 [283:33:30<125:50:13,  5.52s/it]
 64%|██████▎   | 142908/225000 [283:33:35<124:00:03,  5.44s/it]
 64%|██████▎   | 142909/225000 [283:33:41<124:21:11,  5.45s/it]
 64%|██████▎   | 142910/225000 [283:33:46<123:13:21,  5.40s/it]
 64%|██████▎   | 142911/225000 [283:33:51<121:53:48,  5.35s/it]
 64%|██████▎   | 142912/225000 [283:33:57<122:51:30,  5.39s/it]
 64%|██████▎   | 142913/225000 [283:34:02<120:28:40,  5.28s/it]
 64%|██████▎   | 142914/225000 [283:34:07<121:47:46,  5.34s/it]
 64%|██████▎   | 142915/225000 [283:34:12<119:47:14,  5.25s/it]
 64%|██████▎   | 142916/225000 [283:34:17<118:43:40,  5.21s/it]
 64%|██████▎   | 142917/225000 [283:34:23<121:28:50,  5.33s/it]
 64%|██████▎   | 142918/225000 [283:34:28<121:47:46,  5.34s/it]
 64%|██████▎   | 142919/225000 [283:34:34<121:34:33,  5.33s/it]
 64%|██████▎   | 142920/225000 [283:34:45<163:31:26,  7.17s/it]
 64%|██████▎   | 142921/225000 [283:34:49<144:59:27,  6.36s/it]
 64%|██████▎   | 142922/225000 [283:34:53<128:45:55,  5.65s/it]
 64%|██████▎   | 142923/225000 [283:34:57<116:22:34,  5.10s/it]
 64%|██████▎   | 142924/225000 [283:35:08<153:06:52,  6.72s/it]
 64%|██████▎   | 142925/225000 [283:35:15<153:49:58,  6.75s/it]
 64%|██████▎   | 142926/225000 [283:35:20<145:29:11,  6.38s/it]
 64%|██████▎   | 142927/225000 [283:35:24<126:55:18,  5.57s/it]
 64%|██████▎   | 142928/225000 [283:35:29<121:09:55,  5.31s/it]
 64%|██████▎   | 142929/225000 [283:35:33<115:59:35,  5.09s/it]
 64%|██████▎   | 142930/225000 [283:35:38<113:56:48,  5.00s/it]
 64%|██████▎   | 142931/225000 [283:35:43<116:37:44,  5.12s/it]
 64%|██████▎   | 142932/225000 [283:35:55<162:05:58,  7.11s/it]
 64%|██████▎   | 142933/225000 [283:35:59<141:42:24,  6.22s/it]
 64%|██████▎   | 142934/225000 [283:36:03<126:17:51,  5.54s/it]
 64%|██████▎   | 142935/225000 [283:36:07<116:34:40,  5.11s/it]
 64%|██████▎   | 142936/225000 [283:36:12<116:05:13,  5.09s/it]
 64%|██████▎   | 142937/225000 [283:36:23<153:23:42,  6.73s/it]
 64%|██████▎   | 142938/225000 [283:36:26<130:49:21,  5.74s/it]
 64%|██████▎   | 142939/225000 [283:36:34<143:19:36,  6.29s/it]
 64%|██████▎   | 142940/225000 [283:36:37<123:20:19,  5.41s/it]
 64%|██████▎   | 142941/225000 [283:36:46<147:16:15,  6.46s/it]
 64%|██████▎   | 142942/225000 [283:36:50<131:18:52,  5.76s/it]
 64%|██████▎   | 142943/225000 [283:36:55<123:30:56,  5.42s/it]
 64%|██████▎   | 142944/225000 [283:37:00<119:05:27,  5.22s/it]
 64%|██████▎   | 142945/225000 [283:37:05<117:08:47,  5.14s/it]
 64%|██████▎   | 142946/225000 [283:37:09<114:10:55,  5.01s/it]
 64%|██████▎   | 142947/225000 [283:37:14<110:59:40,  4.87s/it]
 64%|██████▎   | 142948/225000 [283:37:19<112:20:04,  4.93s/it]
 64%|██████▎   | 142949/225000 [283:37:23<109:11:11,  4.79s/it]
 64%|██████▎   | 142950/225000 [283:37:30<123:11:32,  5.41s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.122', 'learning_rate': '1.594e-05', 'epoch': '0.6353', 'num_input_tokens_seen': 470160200, 'train_runtime': '1.021e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 142950/225000 [283:37:30<123:11:32,  5.41s/it]
 64%|██████▎   | 142951/225000 [283:37:42<166:25:12,  7.30s/it]
 64%|██████▎   | 142952/225000 [283:37:46<146:11:02,  6.41s/it]
 64%|██████▎   | 142953/225000 [283:37:51<131:28:34,  5.77s/it]
 64%|██████▎   | 142954/225000 [283:37:55<122:40:00,  5.38s/it]
 64%|██████▎   | 142955/225000 [283:38:04<149:13:37,  6.55s/it]
 64%|██████▎   | 142956/225000 [283:38:10<141:13:14,  6.20s/it]
 64%|██████▎   | 142957/225000 [283:38:13<122:43:50,  5.39s/it]
 64%|██████▎   | 142958/225000 [283:38:22<145:09:21,  6.37s/it]
 64%|██████▎   | 142959/225000 [283:38:26<131:29:10,  5.77s/it]
 64%|██████▎   | 142960/225000 [283:38:34<144:00:35,  6.32s/it]
 64%|██████▎   | 142961/225000 [283:38:39<137:27:56,  6.03s/it]
 64%|██████▎   | 142962/225000 [283:38:44<129:56:46,  5.70s/it]
 64%|██████▎   | 142963/225000 [283:38:49<126:30:17,  5.55s/it]
 64%|██████▎   | 142964/225000 [283:38:54<120:01:27,  5.27s/it]
 64%|██████▎   | 142965/225000 [283:38:59<119:05:58,  5.23s/it]
 64%|██████▎   | 142966/225000 [283:39:04<117:26:38,  5.15s/it]
 64%|██████▎   | 142967/225000 [283:39:09<117:48:45,  5.17s/it]
 64%|██████▎   | 142968/225000 [283:39:14<117:12:44,  5.14s/it]
 64%|██████▎   | 142969/225000 [283:39:19<112:12:41,  4.92s/it]
 64%|██████▎   | 142970/225000 [283:39:24<113:29:22,  4.98s/it]
 64%|██████▎   | 142971/225000 [283:39:29<116:30:26,  5.11s/it]
 64%|██████▎   | 142972/225000 [283:39:34<115:41:39,  5.08s/it]
 64%|██████▎   | 142973/225000 [283:39:39<114:36:37,  5.03s/it]
 64%|██████▎   | 142974/225000 [283:39:49<148:33:17,  6.52s/it]
 64%|██████▎   | 142975/225000 [283:39:56<153:54:43,  6.76s/it]
 64%|██████▎   | 142976/225000 [283:40:00<130:44:45,  5.74s/it]
 64%|██████▎   | 142977/225000 [283:40:09<154:31:38,  6.78s/it]
 64%|██████▎   | 142978/225000 [283:40:13<136:28:30,  5.99s/it]
 64%|██████▎   | 142979/225000 [283:40:17<123:20:32,  5.41s/it]
 64%|██████▎   | 142980/225000 [283:40:26<144:26:56,  6.34s/it]
 64%|██████▎   | 142981/225000 [283:40:31<136:16:34,  5.98s/it]
 64%|██████▎   | 142982/225000 [283:40:38<146:25:39,  6.43s/it]
 64%|██████▎   | 142983/225000 [283:40:42<125:25:36,  5.51s/it]
 64%|██████▎   | 142984/225000 [283:40:50<142:07:54,  6.24s/it]
 64%|██████▎   | 142985/225000 [283:40:53<121:38:22,  5.34s/it]
 64%|██████▎   | 142986/225000 [283:41:01<141:08:59,  6.20s/it]
 64%|██████▎   | 142987/225000 [283:41:04<120:00:42,  5.27s/it]
 64%|██████▎   | 142988/225000 [283:41:11<129:34:56,  5.69s/it]
 64%|██████▎   | 142989/225000 [283:41:14<114:17:08,  5.02s/it]
 64%|██████▎   | 142990/225000 [283:41:20<116:55:03,  5.13s/it]
 64%|██████▎   | 142991/225000 [283:41:26<128:12:26,  5.63s/it]
 64%|██████▎   | 142992/225000 [283:41:32<126:49:40,  5.57s/it]
 64%|██████▎   | 142993/225000 [283:41:39<138:42:07,  6.09s/it]
 64%|██████▎   | 142994/225000 [283:41:43<122:28:33,  5.38s/it]
 64%|██████▎   | 142995/225000 [283:41:51<138:17:12,  6.07s/it]
 64%|██████▎   | 142996/225000 [283:41:55<124:07:21,  5.45s/it]
 64%|██████▎   | 142997/225000 [283:41:58<113:11:34,  4.97s/it]
 64%|██████▎   | 142998/225000 [283:42:03<111:08:41,  4.88s/it]
 64%|██████▎   | 142999/225000 [283:42:09<121:07:15,  5.32s/it]
 64%|██████▎   | 143000/225000 [283:42:21<162:19:30,  7.13s/it]
                                                               
{'loss': '0.7835', 'grad_norm': '1.115', 'learning_rate': '1.592e-05', 'epoch': '0.6356', 'num_input_tokens_seen': 470289480, 'train_runtime': '1.021e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143000/225000 [283:42:21<162:19:30,  7.13s/it]
 64%|██████▎   | 143001/225000 [283:42:24<137:55:38,  6.06s/it]
 64%|██████▎   | 143002/225000 [283:42:32<146:30:49,  6.43s/it]
 64%|██████▎   | 143003/225000 [283:42:37<139:58:28,  6.15s/it]
 64%|██████▎   | 143004/225000 [283:42:40<120:08:52,  5.28s/it]
 64%|██████▎   | 143005/225000 [283:42:47<128:03:00,  5.62s/it]
 64%|██████▎   | 143006/225000 [283:42:52<125:23:45,  5.51s/it]
 64%|██████▎   | 143007/225000 [283:42:59<133:03:13,  5.84s/it]
 64%|██████▎   | 143008/225000 [283:43:03<122:05:41,  5.36s/it]
 64%|██████▎   | 143009/225000 [283:43:08<120:07:03,  5.27s/it]
 64%|██████▎   | 143010/225000 [283:43:13<118:41:17,  5.21s/it]
 64%|██████▎   | 143011/225000 [283:43:18<116:15:00,  5.10s/it]
 64%|██████▎   | 143012/225000 [283:43:23<115:28:00,  5.07s/it]
 64%|██████▎   | 143013/225000 [283:43:29<118:59:11,  5.22s/it]
 64%|██████▎   | 143014/225000 [283:43:33<116:16:07,  5.11s/it]
 64%|██████▎   | 143015/225000 [283:43:39<119:10:49,  5.23s/it]
 64%|██████▎   | 143016/225000 [283:43:44<119:12:32,  5.23s/it]
 64%|██████▎   | 143017/225000 [283:43:49<115:23:59,  5.07s/it]
 64%|██████▎   | 143018/225000 [283:43:54<114:07:36,  5.01s/it]
 64%|██████▎   | 143019/225000 [283:44:03<146:37:51,  6.44s/it]
 64%|██████▎   | 143020/225000 [283:44:12<164:05:31,  7.21s/it]
 64%|██████▎   | 143021/225000 [283:44:18<151:05:37,  6.64s/it]
 64%|██████▎   | 143022/225000 [283:44:23<142:23:35,  6.25s/it]
 64%|██████▎   | 143023/225000 [283:44:28<135:18:36,  5.94s/it]
 64%|██████▎   | 143024/225000 [283:44:33<129:30:48,  5.69s/it]
 64%|██████▎   | 143025/225000 [283:44:39<126:52:48,  5.57s/it]
 64%|██████▎   | 143026/225000 [283:44:43<121:31:25,  5.34s/it]
 64%|██████▎   | 143027/225000 [283:44:49<123:08:22,  5.41s/it]
 64%|██████▎   | 143028/225000 [283:44:54<120:45:33,  5.30s/it]
 64%|██████▎   | 143029/225000 [283:45:00<121:52:33,  5.35s/it]
 64%|██████▎   | 143030/225000 [283:45:05<123:04:56,  5.41s/it]
 64%|██████▎   | 143031/225000 [283:45:10<122:45:21,  5.39s/it]
 64%|██████▎   | 143032/225000 [283:45:16<123:30:46,  5.42s/it]
 64%|██████▎   | 143033/225000 [283:45:26<155:46:35,  6.84s/it]
 64%|██████▎   | 143034/225000 [283:45:35<166:36:49,  7.32s/it]
 64%|██████▎   | 143035/225000 [283:45:38<137:55:52,  6.06s/it]
 64%|██████▎   | 143036/225000 [283:45:46<155:23:02,  6.82s/it]
 64%|██████▎   | 143037/225000 [283:45:50<130:54:50,  5.75s/it]
 64%|██████▎   | 143038/225000 [283:45:56<134:46:09,  5.92s/it]
 64%|██████▎   | 143039/225000 [283:46:01<129:06:52,  5.67s/it]
 64%|██████▎   | 143040/225000 [283:46:07<129:09:04,  5.67s/it]
 64%|██████▎   | 143041/225000 [283:46:16<154:10:52,  6.77s/it]
 64%|██████▎   | 143042/225000 [283:46:20<137:23:33,  6.03s/it]
 64%|██████▎   | 143043/225000 [283:46:24<122:00:01,  5.36s/it]
 64%|██████▎   | 143044/225000 [283:46:28<115:44:28,  5.08s/it]
 64%|██████▎   | 143045/225000 [283:46:33<111:48:49,  4.91s/it]
 64%|██████▎   | 143046/225000 [283:46:44<155:37:08,  6.84s/it]
 64%|██████▎   | 143047/225000 [283:46:49<138:23:20,  6.08s/it]
 64%|██████▎   | 143048/225000 [283:46:53<124:18:11,  5.46s/it]
 64%|██████▎   | 143049/225000 [283:46:56<113:11:57,  4.97s/it]
 64%|██████▎   | 143050/225000 [283:47:05<136:33:36,  6.00s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.075', 'learning_rate': '1.59e-05', 'epoch': '0.6358', 'num_input_tokens_seen': 470418840, 'train_runtime': '1.022e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143050/225000 [283:47:05<136:33:36,  6.00s/it]
 64%|██████▎   | 143051/225000 [283:47:13<151:38:05,  6.66s/it]
 64%|██████▎   | 143052/225000 [283:47:18<136:50:20,  6.01s/it]
 64%|██████▎   | 143053/225000 [283:47:22<125:15:45,  5.50s/it]
 64%|██████▎   | 143054/225000 [283:47:26<118:07:38,  5.19s/it]
 64%|██████▎   | 143055/225000 [283:47:31<113:42:04,  5.00s/it]
 64%|██████▎   | 143056/225000 [283:47:38<130:17:31,  5.72s/it]
 64%|██████▎   | 143057/225000 [283:47:48<155:00:25,  6.81s/it]
 64%|██████▎   | 143058/225000 [283:47:51<131:12:35,  5.76s/it]
 64%|██████▎   | 143059/225000 [283:47:57<134:50:45,  5.92s/it]
 64%|██████▎   | 143060/225000 [283:48:05<149:40:06,  6.58s/it]
 64%|██████▎   | 143061/225000 [283:48:09<127:27:41,  5.60s/it]
 64%|██████▎   | 143062/225000 [283:48:16<140:28:51,  6.17s/it]
 64%|██████▎   | 143063/225000 [283:48:21<128:50:21,  5.66s/it]
 64%|██████▎   | 143064/225000 [283:48:25<118:54:30,  5.22s/it]
 64%|██████▎   | 143065/225000 [283:48:30<115:50:30,  5.09s/it]
 64%|██████▎   | 143066/225000 [283:48:34<111:36:21,  4.90s/it]
 64%|██████▎   | 143067/225000 [283:48:39<109:12:02,  4.80s/it]
 64%|██████▎   | 143068/225000 [283:48:49<146:20:39,  6.43s/it]
 64%|██████▎   | 143069/225000 [283:48:57<155:21:10,  6.83s/it]
 64%|██████▎   | 143070/225000 [283:49:00<132:51:34,  5.84s/it]
 64%|██████▎   | 143071/225000 [283:49:04<117:30:15,  5.16s/it]
 64%|██████▎   | 143072/225000 [283:49:12<139:50:55,  6.15s/it]
 64%|██████▎   | 143073/225000 [283:49:17<132:32:43,  5.82s/it]
 64%|██████▎   | 143074/225000 [283:49:21<119:31:10,  5.25s/it]
 64%|██████▎   | 143075/225000 [283:49:26<113:13:30,  4.98s/it]
 64%|██████▎   | 143076/225000 [283:49:30<107:48:28,  4.74s/it]
 64%|██████▎   | 143077/225000 [283:49:36<120:52:45,  5.31s/it]
 64%|██████▎   | 143078/225000 [283:49:46<150:48:27,  6.63s/it]
 64%|██████▎   | 143079/225000 [283:49:50<131:24:15,  5.77s/it]
 64%|██████▎   | 143080/225000 [283:49:54<118:49:03,  5.22s/it]
 64%|██████▎   | 143081/225000 [283:49:59<115:27:13,  5.07s/it]
 64%|██████▎   | 143082/225000 [283:50:02<107:13:24,  4.71s/it]
 64%|██████▎   | 143083/225000 [283:50:12<140:05:40,  6.16s/it]
 64%|██████▎   | 143084/225000 [283:50:18<136:58:59,  6.02s/it]
 64%|██████▎   | 143085/225000 [283:50:24<140:37:48,  6.18s/it]
 64%|██████▎   | 143086/225000 [283:50:30<136:51:57,  6.02s/it]
 64%|██████▎   | 143087/225000 [283:50:34<122:53:49,  5.40s/it]
 64%|██████▎   | 143088/225000 [283:50:40<128:47:53,  5.66s/it]
 64%|██████▎   | 143089/225000 [283:50:47<137:10:07,  6.03s/it]
 64%|██████▎   | 143090/225000 [283:50:51<125:10:41,  5.50s/it]
 64%|██████▎   | 143091/225000 [283:50:58<132:11:41,  5.81s/it]
 64%|██████▎   | 143092/225000 [283:51:05<141:58:33,  6.24s/it]
 64%|██████▎   | 143093/225000 [283:51:09<124:53:36,  5.49s/it]
 64%|██████▎   | 143094/225000 [283:51:12<112:41:31,  4.95s/it]
 64%|██████▎   | 143095/225000 [283:51:22<143:58:23,  6.33s/it]
 64%|██████▎   | 143096/225000 [283:51:27<133:41:44,  5.88s/it]
 64%|██████▎   | 143097/225000 [283:51:33<138:58:38,  6.11s/it]
 64%|██████▎   | 143098/225000 [283:51:39<133:13:12,  5.86s/it]
 64%|██████▎   | 143099/225000 [283:51:43<123:02:36,  5.41s/it]
 64%|██████▎   | 143100/225000 [283:51:49<128:55:09,  5.67s/it]
                                                               
{'loss': '0.7791', 'grad_norm': '1.18', 'learning_rate': '1.589e-05', 'epoch': '0.636', 'num_input_tokens_seen': 470548232, 'train_runtime': '1.022e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143100/225000 [283:51:49<128:55:09,  5.67s/it]
 64%|██████▎   | 143101/225000 [283:51:53<115:35:11,  5.08s/it]
 64%|██████▎   | 143102/225000 [283:52:01<136:08:39,  5.98s/it]
 64%|██████▎   | 143103/225000 [283:52:05<124:35:08,  5.48s/it]
 64%|██████▎   | 143104/225000 [283:52:14<142:22:44,  6.26s/it]
 64%|██████▎   | 143105/225000 [283:52:17<125:51:37,  5.53s/it]
 64%|██████▎   | 143106/225000 [283:52:21<115:27:50,  5.08s/it]
 64%|██████▎   | 143107/225000 [283:52:25<107:47:29,  4.74s/it]
 64%|██████▎   | 143108/225000 [283:52:35<139:17:21,  6.12s/it]
 64%|██████▎   | 143109/225000 [283:52:39<127:09:13,  5.59s/it]
 64%|██████▎   | 143110/225000 [283:52:43<114:55:19,  5.05s/it]
 64%|██████▎   | 143111/225000 [283:52:47<109:10:16,  4.80s/it]
 64%|██████▎   | 143112/225000 [283:52:58<151:47:46,  6.67s/it]
 64%|██████▎   | 143113/225000 [283:53:02<135:22:35,  5.95s/it]
 64%|██████▎   | 143114/225000 [283:53:06<123:03:23,  5.41s/it]
 64%|██████▎   | 143115/225000 [283:53:11<116:36:41,  5.13s/it]
 64%|██████▎   | 143116/225000 [283:53:16<114:58:28,  5.05s/it]
 64%|██████▎   | 143117/225000 [283:53:21<113:41:13,  5.00s/it]
 64%|██████▎   | 143118/225000 [283:53:26<112:56:05,  4.97s/it]
 64%|██████▎   | 143119/225000 [283:53:37<159:15:45,  7.00s/it]
 64%|██████▎   | 143120/225000 [283:53:42<143:13:45,  6.30s/it]
 64%|██████▎   | 143121/225000 [283:53:49<148:10:36,  6.51s/it]
 64%|██████▎   | 143122/225000 [283:53:53<129:15:35,  5.68s/it]
 64%|██████▎   | 143123/225000 [283:53:58<122:55:52,  5.41s/it]
 64%|██████▎   | 143124/225000 [283:54:02<116:11:23,  5.11s/it]
 64%|██████▎   | 143125/225000 [283:54:07<112:38:03,  4.95s/it]
 64%|██████▎   | 143126/225000 [283:54:18<157:37:37,  6.93s/it]
 64%|██████▎   | 143127/225000 [283:54:22<134:55:04,  5.93s/it]
 64%|██████▎   | 143128/225000 [283:54:25<118:12:06,  5.20s/it]
 64%|██████▎   | 143129/225000 [283:54:34<145:31:55,  6.40s/it]
 64%|██████▎   | 143130/225000 [283:54:39<130:41:36,  5.75s/it]
 64%|██████▎   | 143131/225000 [283:54:42<116:31:09,  5.12s/it]
 64%|██████▎   | 143132/225000 [283:54:47<113:22:07,  4.99s/it]
 64%|██████▎   | 143133/225000 [283:54:52<112:42:20,  4.96s/it]
 64%|██████▎   | 143134/225000 [283:54:57<112:25:08,  4.94s/it]
 64%|██████▎   | 143135/225000 [283:55:07<147:38:26,  6.49s/it]
 64%|██████▎   | 143136/225000 [283:55:14<153:24:57,  6.75s/it]
 64%|██████▎   | 143137/225000 [283:55:18<131:13:48,  5.77s/it]
 64%|██████▎   | 143138/225000 [283:55:26<146:23:03,  6.44s/it]
 64%|██████▎   | 143139/225000 [283:55:30<131:55:53,  5.80s/it]
 64%|██████▎   | 143140/225000 [283:55:34<122:25:43,  5.38s/it]
 64%|██████▎   | 143141/225000 [283:55:39<114:32:41,  5.04s/it]
 64%|██████▎   | 143142/225000 [283:55:43<110:04:00,  4.84s/it]
 64%|██████▎   | 143143/225000 [283:55:54<154:26:07,  6.79s/it]
 64%|██████▎   | 143144/225000 [283:55:58<133:46:11,  5.88s/it]
 64%|██████▎   | 143145/225000 [283:56:02<122:32:39,  5.39s/it]
 64%|██████▎   | 143146/225000 [283:56:07<115:55:37,  5.10s/it]
 64%|██████▎   | 143147/225000 [283:56:13<126:45:45,  5.58s/it]
 64%|██████▎   | 143148/225000 [283:56:23<153:17:53,  6.74s/it]
 64%|██████▎   | 143149/225000 [283:56:26<129:41:08,  5.70s/it]
 64%|██████▎   | 143150/225000 [283:56:35<150:41:59,  6.63s/it]
                                                               
{'loss': '0.7811', 'grad_norm': '1.158', 'learning_rate': '1.587e-05', 'epoch': '0.6362', 'num_input_tokens_seen': 470677400, 'train_runtime': '1.022e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143150/225000 [283:56:35<150:41:59,  6.63s/it]
 64%|██████▎   | 143151/225000 [283:56:39<132:12:52,  5.82s/it]
 64%|██████▎   | 143152/225000 [283:56:43<119:50:20,  5.27s/it]
 64%|██████▎   | 143153/225000 [283:56:47<112:25:43,  4.95s/it]
 64%|██████▎   | 143154/225000 [283:56:51<106:31:53,  4.69s/it]
 64%|██████▎   | 143155/225000 [283:57:02<149:23:24,  6.57s/it]
 64%|██████▎   | 143156/225000 [283:57:09<148:27:03,  6.53s/it]
 64%|██████▎   | 143157/225000 [283:57:13<134:07:09,  5.90s/it]
 64%|██████▎   | 143158/225000 [283:57:17<123:36:54,  5.44s/it]
 64%|██████▎   | 143159/225000 [283:57:22<115:27:54,  5.08s/it]
 64%|██████▎   | 143160/225000 [283:57:26<110:27:00,  4.86s/it]
 64%|██████▎   | 143161/225000 [283:57:31<108:39:47,  4.78s/it]
 64%|██████▎   | 143162/225000 [283:57:35<105:31:53,  4.64s/it]
 64%|██████▎   | 143163/225000 [283:57:41<114:57:47,  5.06s/it]
 64%|██████▎   | 143164/225000 [283:57:54<168:48:13,  7.43s/it]
 64%|██████▎   | 143165/225000 [283:57:57<139:59:47,  6.16s/it]
 64%|██████▎   | 143166/225000 [283:58:03<137:14:59,  6.04s/it]
 64%|██████▎   | 143167/225000 [283:58:09<137:30:38,  6.05s/it]
 64%|██████▎   | 143168/225000 [283:58:17<151:42:39,  6.67s/it]
 64%|██████▎   | 143169/225000 [283:58:24<151:00:39,  6.64s/it]
 64%|██████▎   | 143170/225000 [283:58:27<129:10:05,  5.68s/it]
 64%|██████▎   | 143171/225000 [283:58:33<132:28:29,  5.83s/it]
 64%|██████▎   | 143172/225000 [283:58:39<133:45:31,  5.88s/it]
 64%|██████▎   | 143173/225000 [283:58:43<118:50:58,  5.23s/it]
 64%|██████▎   | 143174/225000 [283:58:47<109:01:58,  4.80s/it]
 64%|██████▎   | 143175/225000 [283:58:52<111:52:21,  4.92s/it]
 64%|██████▎   | 143176/225000 [283:58:57<111:17:04,  4.90s/it]
 64%|██████▎   | 143177/225000 [283:59:02<114:32:43,  5.04s/it]
 64%|██████▎   | 143178/225000 [283:59:07<115:09:30,  5.07s/it]
 64%|██████▎   | 143179/225000 [283:59:12<116:13:38,  5.11s/it]
 64%|██████▎   | 143180/225000 [283:59:18<119:46:07,  5.27s/it]
 64%|██████▎   | 143181/225000 [283:59:23<117:58:57,  5.19s/it]
 64%|██████▎   | 143182/225000 [283:59:28<118:41:45,  5.22s/it]
 64%|██████▎   | 143183/225000 [283:59:41<168:17:36,  7.41s/it]
 64%|██████▎   | 143184/225000 [283:59:48<164:14:13,  7.23s/it]
 64%|██████▎   | 143185/225000 [283:59:52<141:07:03,  6.21s/it]
 64%|██████▎   | 143186/225000 [283:59:57<138:52:48,  6.11s/it]
 64%|██████▎   | 143187/225000 [284:00:04<141:47:04,  6.24s/it]
 64%|██████▎   | 143188/225000 [284:00:09<130:23:24,  5.74s/it]
 64%|██████▎   | 143189/225000 [284:00:13<120:37:55,  5.31s/it]
 64%|██████▎   | 143190/225000 [284:00:17<113:16:13,  4.98s/it]
 64%|██████▎   | 143191/225000 [284:00:29<162:42:01,  7.16s/it]
 64%|██████▎   | 143192/225000 [284:00:33<136:40:37,  6.01s/it]
 64%|██████▎   | 143193/225000 [284:00:40<146:18:45,  6.44s/it]
 64%|██████▎   | 143194/225000 [284:00:45<133:57:31,  5.90s/it]
 64%|██████▎   | 143195/225000 [284:00:49<123:58:03,  5.46s/it]
 64%|██████▎   | 143196/225000 [284:00:54<120:13:08,  5.29s/it]
 64%|██████▎   | 143197/225000 [284:01:04<153:27:25,  6.75s/it]
 64%|██████▎   | 143198/225000 [284:01:08<130:52:05,  5.76s/it]
 64%|██████▎   | 143199/225000 [284:01:12<118:06:46,  5.20s/it]
 64%|██████▎   | 143200/225000 [284:01:15<109:07:58,  4.80s/it]
                                                               
{'loss': '0.7921', 'grad_norm': '1.052', 'learning_rate': '1.585e-05', 'epoch': '0.6364', 'num_input_tokens_seen': 470806744, 'train_runtime': '1.022e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143200/225000 [284:01:15<109:07:58,  4.80s/it]
 64%|██████▎   | 143201/225000 [284:01:27<157:02:36,  6.91s/it]
 64%|██████▎   | 143202/225000 [284:01:31<132:11:32,  5.82s/it]
 64%|██████▎   | 143203/225000 [284:01:38<142:55:31,  6.29s/it]
 64%|██████▎   | 143204/225000 [284:01:41<122:11:20,  5.38s/it]
 64%|██████▎   | 143205/225000 [284:01:47<128:08:25,  5.64s/it]
 64%|██████▎   | 143206/225000 [284:01:53<125:22:42,  5.52s/it]
 64%|██████▎   | 143207/225000 [284:01:59<131:05:37,  5.77s/it]
 64%|██████▎   | 143208/225000 [284:02:03<121:17:29,  5.34s/it]
 64%|██████▎   | 143209/225000 [284:02:08<114:11:17,  5.03s/it]
 64%|██████▎   | 143210/225000 [284:02:12<110:44:09,  4.87s/it]
 64%|██████▎   | 143211/225000 [284:02:17<109:04:54,  4.80s/it]
 64%|██████▎   | 143212/225000 [284:02:29<156:58:58,  6.91s/it]
 64%|██████▎   | 143213/225000 [284:02:32<135:03:33,  5.94s/it]
 64%|██████▎   | 143214/225000 [284:02:36<119:31:10,  5.26s/it]
 64%|██████▎   | 143215/225000 [284:02:45<147:52:11,  6.51s/it]
 64%|██████▎   | 143216/225000 [284:02:50<138:12:20,  6.08s/it]
 64%|██████▎   | 143217/225000 [284:02:54<123:03:07,  5.42s/it]
 64%|██████▎   | 143218/225000 [284:02:59<114:39:44,  5.05s/it]
 64%|██████▎   | 143219/225000 [284:03:10<157:52:14,  6.95s/it]
 64%|██████▎   | 143220/225000 [284:03:14<135:19:31,  5.96s/it]
 64%|██████▎   | 143221/225000 [284:03:17<120:24:13,  5.30s/it]
 64%|██████▎   | 143222/225000 [284:03:27<150:17:46,  6.62s/it]
 64%|██████▎   | 143223/225000 [284:03:31<129:40:12,  5.71s/it]
 64%|██████▎   | 143224/225000 [284:03:38<140:57:13,  6.21s/it]
 64%|██████▎   | 143225/225000 [284:03:43<135:29:33,  5.96s/it]
 64%|██████▎   | 143226/225000 [284:03:48<127:08:02,  5.60s/it]
 64%|██████▎   | 143227/225000 [284:03:53<123:15:06,  5.43s/it]
 64%|██████▎   | 143228/225000 [284:03:58<120:34:01,  5.31s/it]
 64%|██████▎   | 143229/225000 [284:04:03<116:29:06,  5.13s/it]
 64%|██████▎   | 143230/225000 [284:04:08<113:15:50,  4.99s/it]
 64%|██████▎   | 143231/225000 [284:04:13<116:55:18,  5.15s/it]
 64%|██████▎   | 143232/225000 [284:04:18<116:55:55,  5.15s/it]
 64%|██████▎   | 143233/225000 [284:04:23<117:47:56,  5.19s/it]
 64%|██████▎   | 143234/225000 [284:04:29<118:56:42,  5.24s/it]
 64%|██████▎   | 143235/225000 [284:04:33<114:32:39,  5.04s/it]
 64%|██████▎   | 143236/225000 [284:04:38<113:11:26,  4.98s/it]
 64%|██████▎   | 143237/225000 [284:04:44<115:38:19,  5.09s/it]
 64%|██████▎   | 143238/225000 [284:04:48<113:26:32,  4.99s/it]
 64%|██████▎   | 143239/225000 [284:04:53<113:30:23,  5.00s/it]
 64%|██████▎   | 143240/225000 [284:04:59<116:53:46,  5.15s/it]
 64%|██████▎   | 143241/225000 [284:05:04<118:44:39,  5.23s/it]
 64%|██████▎   | 143242/225000 [284:05:09<118:14:11,  5.21s/it]
 64%|██████▎   | 143243/225000 [284:05:15<120:48:07,  5.32s/it]
 64%|██████▎   | 143244/225000 [284:05:25<149:50:33,  6.60s/it]
 64%|██████▎   | 143245/225000 [284:05:36<181:23:00,  7.99s/it]
 64%|██████▎   | 143246/225000 [284:05:39<149:24:36,  6.58s/it]
 64%|██████▎   | 143247/225000 [284:05:43<129:59:42,  5.72s/it]
 64%|██████▎   | 143248/225000 [284:05:51<145:31:52,  6.41s/it]
 64%|██████▎   | 143249/225000 [284:05:57<140:46:06,  6.20s/it]
 64%|██████▎   | 143250/225000 [284:06:01<126:35:16,  5.57s/it]
                                                               
{'loss': '0.7853', 'grad_norm': '1.002', 'learning_rate': '1.584e-05', 'epoch': '0.6367', 'num_input_tokens_seen': 470935976, 'train_runtime': '1.023e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143250/225000 [284:06:01<126:35:16,  5.57s/it]
 64%|██████▎   | 143251/225000 [284:06:10<153:16:23,  6.75s/it]
 64%|██████▎   | 143252/225000 [284:06:14<130:46:43,  5.76s/it]
 64%|██████▎   | 143253/225000 [284:06:19<126:52:06,  5.59s/it]
 64%|██████▎   | 143254/225000 [284:06:26<135:45:50,  5.98s/it]
 64%|██████▎   | 143255/225000 [284:06:33<143:56:18,  6.34s/it]
 64%|██████▎   | 143256/225000 [284:06:38<132:27:14,  5.83s/it]
 64%|██████▎   | 143257/225000 [284:06:41<114:53:19,  5.06s/it]
 64%|██████▎   | 143258/225000 [284:06:46<112:42:06,  4.96s/it]
 64%|██████▎   | 143259/225000 [284:06:52<125:31:13,  5.53s/it]
 64%|██████▎   | 143260/225000 [284:06:57<118:50:24,  5.23s/it]
 64%|██████▎   | 143261/225000 [284:07:02<115:16:45,  5.08s/it]
 64%|██████▎   | 143262/225000 [284:07:06<111:17:38,  4.90s/it]
 64%|██████▎   | 143263/225000 [284:07:11<110:55:37,  4.89s/it]
 64%|██████▎   | 143264/225000 [284:07:16<110:02:11,  4.85s/it]
 64%|██████▎   | 143265/225000 [284:07:28<161:48:39,  7.13s/it]
 64%|██████▎   | 143266/225000 [284:07:32<139:19:10,  6.14s/it]
 64%|██████▎   | 143267/225000 [284:07:37<129:10:13,  5.69s/it]
 64%|██████▎   | 143268/225000 [284:07:45<144:41:08,  6.37s/it]
 64%|██████▎   | 143269/225000 [284:07:48<121:56:24,  5.37s/it]
 64%|██████▎   | 143270/225000 [284:07:55<136:38:00,  6.02s/it]
 64%|██████▎   | 143271/225000 [284:08:01<137:48:09,  6.07s/it]
 64%|██████▎   | 143272/225000 [284:08:06<129:32:04,  5.71s/it]
 64%|██████▎   | 143273/225000 [284:08:11<124:24:11,  5.48s/it]
 64%|██████▎   | 143274/225000 [284:08:16<119:49:39,  5.28s/it]
 64%|██████▎   | 143275/225000 [284:08:21<118:17:12,  5.21s/it]
 64%|██████▎   | 143276/225000 [284:08:26<116:44:47,  5.14s/it]
 64%|██████▎   | 143277/225000 [284:08:32<120:01:05,  5.29s/it]
 64%|██████▎   | 143278/225000 [284:08:36<116:22:41,  5.13s/it]
 64%|██████▎   | 143279/225000 [284:08:41<114:37:58,  5.05s/it]
 64%|██████▎   | 143280/225000 [284:08:47<117:36:28,  5.18s/it]
 64%|██████▎   | 143281/225000 [284:08:52<120:05:54,  5.29s/it]
 64%|██████▎   | 143282/225000 [284:08:58<119:30:59,  5.27s/it]
 64%|██████▎   | 143283/225000 [284:09:03<119:40:58,  5.27s/it]
 64%|██████▎   | 143284/225000 [284:09:08<118:52:49,  5.24s/it]
 64%|██████▎   | 143285/225000 [284:09:19<155:57:42,  6.87s/it]
 64%|██████▎   | 143286/225000 [284:09:26<160:09:37,  7.06s/it]
 64%|██████▎   | 143287/225000 [284:09:33<159:35:20,  7.03s/it]
 64%|██████▎   | 143288/225000 [284:09:38<141:38:47,  6.24s/it]
 64%|██████▎   | 143289/225000 [284:09:44<143:04:13,  6.30s/it]
 64%|██████▎   | 143290/225000 [284:09:49<131:47:04,  5.81s/it]
 64%|██████▎   | 143291/225000 [284:09:55<138:44:40,  6.11s/it]
 64%|██████▎   | 143292/225000 [284:10:01<134:31:58,  5.93s/it]
 64%|██████▎   | 143293/225000 [284:10:06<129:18:10,  5.70s/it]
 64%|██████▎   | 143294/225000 [284:10:11<124:53:35,  5.50s/it]
 64%|██████▎   | 143295/225000 [284:10:19<140:27:36,  6.19s/it]
 64%|██████▎   | 143296/225000 [284:10:24<135:48:56,  5.98s/it]
 64%|██████▎   | 143297/225000 [284:10:30<132:18:54,  5.83s/it]
 64%|██████▎   | 143298/225000 [284:10:35<127:45:40,  5.63s/it]
 64%|██████▎   | 143299/225000 [284:10:39<116:26:12,  5.13s/it]
 64%|██████▎   | 143300/225000 [284:10:43<108:37:25,  4.79s/it]
                                                               
{'loss': '0.7883', 'grad_norm': '1.079', 'learning_rate': '1.582e-05', 'epoch': '0.6369', 'num_input_tokens_seen': 471066408, 'train_runtime': '1.023e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143300/225000 [284:10:43<108:37:25,  4.79s/it]
 64%|██████▎   | 143301/225000 [284:10:48<109:03:42,  4.81s/it]
 64%|██████▎   | 143302/225000 [284:10:58<142:26:23,  6.28s/it]
 64%|██████▎   | 143303/225000 [284:11:01<121:04:35,  5.34s/it]
 64%|██████▎   | 143304/225000 [284:11:08<131:10:18,  5.78s/it]
 64%|██████▎   | 143305/225000 [284:11:13<129:01:12,  5.69s/it]
 64%|██████▎   | 143306/225000 [284:11:19<133:46:15,  5.89s/it]
 64%|██████▎   | 143307/225000 [284:11:23<116:34:50,  5.14s/it]
 64%|██████▎   | 143308/225000 [284:11:30<130:35:45,  5.76s/it]
 64%|██████▎   | 143309/225000 [284:11:37<137:14:21,  6.05s/it]
 64%|██████▎   | 143310/225000 [284:11:40<119:18:20,  5.26s/it]
 64%|██████▎   | 143311/225000 [284:11:49<147:14:41,  6.49s/it]
 64%|██████▎   | 143312/225000 [284:11:53<126:41:43,  5.58s/it]
 64%|██████▎   | 143313/225000 [284:11:58<125:03:08,  5.51s/it]
 64%|██████▎   | 143314/225000 [284:12:06<143:00:30,  6.30s/it]
 64%|██████▎   | 143315/225000 [284:12:09<120:25:19,  5.31s/it]
 64%|██████▎   | 143316/225000 [284:12:17<133:55:30,  5.90s/it]
 64%|██████▎   | 143317/225000 [284:12:20<118:21:21,  5.22s/it]
 64%|██████▎   | 143318/225000 [284:12:25<112:19:49,  4.95s/it]
 64%|██████▎   | 143319/225000 [284:12:30<116:45:42,  5.15s/it]
 64%|██████▎   | 143320/225000 [284:12:36<118:37:33,  5.23s/it]
 64%|██████▎   | 143321/225000 [284:12:41<119:49:21,  5.28s/it]
 64%|██████▎   | 143322/225000 [284:12:46<118:00:24,  5.20s/it]
 64%|██████▎   | 143323/225000 [284:12:51<117:11:03,  5.17s/it]
 64%|██████▎   | 143324/225000 [284:12:56<118:19:12,  5.22s/it]
 64%|██████▎   | 143325/225000 [284:13:02<119:20:14,  5.26s/it]
 64%|██████▎   | 143326/225000 [284:13:07<119:24:11,  5.26s/it]
 64%|██████▎   | 143327/225000 [284:13:13<122:47:03,  5.41s/it]
 64%|██████▎   | 143328/225000 [284:13:18<122:27:25,  5.40s/it]
 64%|██████▎   | 143329/225000 [284:13:27<143:26:50,  6.32s/it]
 64%|██████▎   | 143330/225000 [284:13:32<136:20:22,  6.01s/it]
 64%|██████▎   | 143331/225000 [284:13:37<131:10:16,  5.78s/it]
 64%|██████▎   | 143332/225000 [284:13:47<159:57:02,  7.05s/it]
 64%|██████▎   | 143333/225000 [284:13:52<141:32:09,  6.24s/it]
 64%|██████▎   | 143334/225000 [284:13:55<124:20:05,  5.48s/it]
 64%|██████▎   | 143335/225000 [284:13:59<114:10:59,  5.03s/it]
 64%|██████▎   | 143336/225000 [284:14:06<126:46:28,  5.59s/it]
 64%|██████▎   | 143337/225000 [284:14:15<151:59:57,  6.70s/it]
 64%|██████▎   | 143338/225000 [284:14:19<130:51:31,  5.77s/it]
 64%|██████▎   | 143339/225000 [284:14:27<144:47:13,  6.38s/it]
 64%|██████▎   | 143340/225000 [284:14:32<133:02:02,  5.86s/it]
 64%|██████▎   | 143341/225000 [284:14:37<126:43:33,  5.59s/it]
 64%|██████▎   | 143342/225000 [284:14:42<123:16:47,  5.43s/it]
 64%|██████▎   | 143343/225000 [284:14:46<116:46:09,  5.15s/it]
 64%|██████▎   | 143344/225000 [284:14:51<113:09:54,  4.99s/it]
 64%|██████▎   | 143345/225000 [284:14:55<108:32:48,  4.79s/it]
 64%|██████▎   | 143346/225000 [284:15:00<109:31:38,  4.83s/it]
 64%|██████▎   | 143347/225000 [284:15:04<107:30:39,  4.74s/it]
 64%|██████▎   | 143348/225000 [284:15:09<106:59:55,  4.72s/it]
 64%|██████▎   | 143349/225000 [284:15:19<140:24:47,  6.19s/it]
 64%|██████▎   | 143350/225000 [284:15:28<160:12:13,  7.06s/it]
                                                               
{'loss': '0.7867', 'grad_norm': '1.028', 'learning_rate': '1.58e-05', 'epoch': '0.6371', 'num_input_tokens_seen': 471193824, 'train_runtime': '1.023e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143350/225000 [284:15:28<160:12:13,  7.06s/it]
 64%|██████▎   | 143351/225000 [284:15:31<134:44:57,  5.94s/it]
 64%|██████▎   | 143352/225000 [284:15:40<153:27:38,  6.77s/it]
 64%|██████▎   | 143353/225000 [284:15:44<132:59:46,  5.86s/it]
 64%|██████▎   | 143354/225000 [284:15:48<123:20:09,  5.44s/it]
 64%|██████▎   | 143355/225000 [284:15:54<129:47:03,  5.72s/it]
 64%|██████▎   | 143356/225000 [284:16:03<149:34:11,  6.60s/it]
 64%|██████▎   | 143357/225000 [284:16:07<132:12:48,  5.83s/it]
 64%|██████▎   | 143358/225000 [284:16:11<118:16:50,  5.22s/it]
 64%|██████▎   | 143359/225000 [284:16:16<116:51:50,  5.15s/it]
 64%|██████▎   | 143360/225000 [284:16:25<146:56:38,  6.48s/it]
 64%|██████▎   | 143361/225000 [284:16:29<128:26:41,  5.66s/it]
 64%|██████▎   | 143362/225000 [284:16:36<132:30:15,  5.84s/it]
 64%|██████▎   | 143363/225000 [284:16:42<138:08:13,  6.09s/it]
 64%|██████▎   | 143364/225000 [284:16:47<126:24:28,  5.57s/it]
 64%|██████▎   | 143365/225000 [284:16:52<123:03:51,  5.43s/it]
 64%|██████▎   | 143366/225000 [284:16:57<122:34:05,  5.41s/it]
 64%|██████▎   | 143367/225000 [284:17:02<118:48:14,  5.24s/it]
 64%|██████▎   | 143368/225000 [284:17:06<113:34:52,  5.01s/it]
 64%|██████▎   | 143369/225000 [284:17:11<113:49:04,  5.02s/it]
 64%|██████▎   | 143370/225000 [284:17:16<111:52:37,  4.93s/it]
 64%|██████▎   | 143371/225000 [284:17:21<113:33:00,  5.01s/it]
 64%|██████▎   | 143372/225000 [284:17:26<112:57:44,  4.98s/it]
 64%|██████▎   | 143373/225000 [284:17:31<113:57:32,  5.03s/it]
 64%|██████▎   | 143374/225000 [284:17:40<138:40:33,  6.12s/it]
 64%|██████▎   | 143375/225000 [284:17:46<135:42:58,  5.99s/it]
 64%|██████▎   | 143376/225000 [284:17:51<128:00:43,  5.65s/it]
 64%|██████▎   | 143377/225000 [284:17:56<126:27:26,  5.58s/it]
 64%|██████▎   | 143378/225000 [284:18:01<124:31:42,  5.49s/it]
 64%|██████▎   | 143379/225000 [284:18:07<124:59:24,  5.51s/it]
 64%|██████▎   | 143380/225000 [284:18:12<122:58:27,  5.42s/it]
 64%|██████▎   | 143381/225000 [284:18:17<120:03:53,  5.30s/it]
 64%|██████▎   | 143382/225000 [284:18:22<117:49:06,  5.20s/it]
 64%|██████▎   | 143383/225000 [284:18:27<118:46:20,  5.24s/it]
 64%|██████▎   | 143384/225000 [284:18:33<120:42:41,  5.32s/it]
 64%|██████▎   | 143385/225000 [284:18:46<172:39:49,  7.62s/it]
 64%|██████▎   | 143386/225000 [284:18:54<175:07:05,  7.72s/it]
 64%|██████▎   | 143387/225000 [284:18:57<147:19:29,  6.50s/it]
 64%|██████▎   | 143388/225000 [284:19:01<127:21:52,  5.62s/it]
 64%|██████▎   | 143389/225000 [284:19:11<159:20:26,  7.03s/it]
 64%|██████▎   | 143390/225000 [284:19:14<131:34:19,  5.80s/it]
 64%|██████▎   | 143391/225000 [284:19:18<118:24:28,  5.22s/it]
 64%|██████▎   | 143392/225000 [284:19:22<112:10:20,  4.95s/it]
 64%|██████▎   | 143393/225000 [284:19:33<151:31:00,  6.68s/it]
 64%|██████▎   | 143394/225000 [284:19:39<145:12:51,  6.41s/it]
 64%|██████▎   | 143395/225000 [284:19:43<130:32:28,  5.76s/it]
 64%|██████▎   | 143396/225000 [284:19:48<124:19:28,  5.48s/it]
 64%|██████▎   | 143397/225000 [284:19:55<134:02:39,  5.91s/it]
 64%|██████▎   | 143398/225000 [284:19:58<116:44:30,  5.15s/it]
 64%|██████▎   | 143399/225000 [284:20:07<143:44:52,  6.34s/it]
 64%|██████▎   | 143400/225000 [284:20:12<131:09:19,  5.79s/it]
                                                               
{'loss': '0.7909', 'grad_norm': '1.117', 'learning_rate': '1.579e-05', 'epoch': '0.6373', 'num_input_tokens_seen': 471323872, 'train_runtime': '1.024e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▎   | 143400/225000 [284:20:12<131:09:19,  5.79s/it]
 64%|██████▎   | 143401/225000 [284:20:17<125:57:07,  5.56s/it]
 64%|██████▎   | 143402/225000 [284:20:21<118:55:49,  5.25s/it]
 64%|██████▎   | 143403/225000 [284:20:31<147:37:58,  6.51s/it]
 64%|██████▎   | 143404/225000 [284:20:35<128:27:22,  5.67s/it]
 64%|██████▎   | 143405/225000 [284:20:38<115:22:08,  5.09s/it]
 64%|██████▎   | 143406/225000 [284:20:42<108:24:38,  4.78s/it]
 64%|██████▎   | 143407/225000 [284:20:53<146:03:03,  6.44s/it]
 64%|██████▎   | 143408/225000 [284:20:57<131:20:52,  5.80s/it]
 64%|██████▎   | 143409/225000 [284:21:01<119:10:11,  5.26s/it]
 64%|██████▎   | 143410/225000 [284:21:05<110:07:57,  4.86s/it]
 64%|██████▎   | 143411/225000 [284:21:11<121:39:31,  5.37s/it]
 64%|██████▎   | 143412/225000 [284:21:22<159:46:52,  7.05s/it]
 64%|██████▎   | 143413/225000 [284:21:26<132:37:22,  5.85s/it]
 64%|██████▎   | 143414/225000 [284:21:33<142:57:58,  6.31s/it]
 64%|██████▎   | 143415/225000 [284:21:37<126:34:05,  5.58s/it]
 64%|██████▎   | 143416/225000 [284:21:41<115:18:35,  5.09s/it]
 64%|██████▎   | 143417/225000 [284:21:50<142:28:16,  6.29s/it]
 64%|██████▎   | 143418/225000 [284:21:55<132:47:49,  5.86s/it]
 64%|██████▎   | 143419/225000 [284:22:00<125:59:41,  5.56s/it]
 64%|██████▎   | 143420/225000 [284:22:04<120:20:14,  5.31s/it]
 64%|██████▎   | 143421/225000 [284:22:09<117:00:20,  5.16s/it]
 64%|██████▎   | 143422/225000 [284:22:14<113:35:02,  5.01s/it]
 64%|██████▎   | 143423/225000 [284:22:18<111:15:54,  4.91s/it]
 64%|██████▎   | 143424/225000 [284:22:23<108:23:41,  4.78s/it]
 64%|██████▎   | 143425/225000 [284:22:28<109:48:29,  4.85s/it]
 64%|██████▎   | 143426/225000 [284:22:33<111:12:20,  4.91s/it]
 64%|██████▎   | 143427/225000 [284:22:42<141:56:59,  6.26s/it]
 64%|██████▎   | 143428/225000 [284:22:52<164:12:31,  7.25s/it]
 64%|██████▎   | 143429/225000 [284:22:55<134:37:21,  5.94s/it]
 64%|██████▎   | 143430/225000 [284:22:59<121:09:24,  5.35s/it]
 64%|██████▎   | 143431/225000 [284:23:06<136:23:06,  6.02s/it]
 64%|██████▎   | 143432/225000 [284:23:12<136:28:19,  6.02s/it]
 64%|██████▎   | 143433/225000 [284:23:16<119:54:32,  5.29s/it]
 64%|██████▎   | 143434/225000 [284:23:24<140:09:53,  6.19s/it]
 64%|██████▎   | 143435/225000 [284:23:29<131:44:47,  5.81s/it]
 64%|██████▎   | 143436/225000 [284:23:33<117:11:13,  5.17s/it]
 64%|██████▎   | 143437/225000 [284:23:42<141:26:37,  6.24s/it]
 64%|██████▍   | 143438/225000 [284:23:46<126:02:15,  5.56s/it]
 64%|██████▍   | 143439/225000 [284:23:49<113:46:42,  5.02s/it]
 64%|██████▍   | 143440/225000 [284:23:59<146:06:57,  6.45s/it]
 64%|██████▍   | 143441/225000 [284:24:02<124:43:34,  5.51s/it]
 64%|██████▍   | 143442/225000 [284:24:07<121:04:25,  5.34s/it]
 64%|██████▍   | 143443/225000 [284:24:14<129:11:08,  5.70s/it]
 64%|██████▍   | 143444/225000 [284:24:17<112:44:06,  4.98s/it]
 64%|██████▍   | 143445/225000 [284:24:24<122:08:10,  5.39s/it]
 64%|██████▍   | 143446/225000 [284:24:29<123:32:51,  5.45s/it]
 64%|██████▍   | 143447/225000 [284:24:34<117:17:59,  5.18s/it]
 64%|██████▍   | 143448/225000 [284:24:38<112:48:28,  4.98s/it]
 64%|██████▍   | 143449/225000 [284:24:43<108:53:59,  4.81s/it]
 64%|██████▍   | 143450/225000 [284:24:47<107:16:28,  4.74s/it]
                                                               
{'loss': '0.7877', 'grad_norm': '1.921', 'learning_rate': '1.577e-05', 'epoch': '0.6376', 'num_input_tokens_seen': 471453600, 'train_runtime': '1.024e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▍   | 143450/225000 [284:24:47<107:16:28,  4.74s/it]
 64%|██████▍   | 143451/225000 [284:24:55<129:44:37,  5.73s/it]
 64%|██████▍   | 143452/225000 [284:25:05<157:38:17,  6.96s/it]
 64%|██████▍   | 143453/225000 [284:25:08<132:30:35,  5.85s/it]
 64%|██████▍   | 143454/225000 [284:25:16<142:17:25,  6.28s/it]
 64%|██████▍   | 143455/225000 [284:25:20<126:53:40,  5.60s/it]
 64%|██████▍   | 143456/225000 [284:25:24<116:00:07,  5.12s/it]
 64%|██████▍   | 143457/225000 [284:25:34<151:38:15,  6.69s/it]
 64%|██████▍   | 143458/225000 [284:25:38<135:06:33,  5.96s/it]
 64%|██████▍   | 143459/225000 [284:25:42<121:12:07,  5.35s/it]
 64%|██████▍   | 143460/225000 [284:25:46<113:35:48,  5.02s/it]
 64%|██████▍   | 143461/225000 [284:25:51<109:46:05,  4.85s/it]
 64%|██████▍   | 143462/225000 [284:25:55<105:51:09,  4.67s/it]
 64%|██████▍   | 143463/225000 [284:26:06<146:33:50,  6.47s/it]
 64%|██████▍   | 143464/225000 [284:26:13<154:28:26,  6.82s/it]
 64%|██████▍   | 143465/225000 [284:26:18<139:04:35,  6.14s/it]
 64%|██████▍   | 143466/225000 [284:26:23<129:59:35,  5.74s/it]
 64%|██████▍   | 143467/225000 [284:26:27<121:32:03,  5.37s/it]
 64%|██████▍   | 143468/225000 [284:26:32<117:06:59,  5.17s/it]
 64%|██████▍   | 143469/225000 [284:26:36<111:02:20,  4.90s/it]
 64%|██████▍   | 143470/225000 [284:26:41<109:44:56,  4.85s/it]
 64%|██████▍   | 143471/225000 [284:26:49<127:46:47,  5.64s/it]
 64%|██████▍   | 143472/225000 [284:27:00<169:20:59,  7.48s/it]
 64%|██████▍   | 143473/225000 [284:27:04<144:34:51,  6.38s/it]
 64%|██████▍   | 143474/225000 [284:27:08<125:30:59,  5.54s/it]
 64%|██████▍   | 143475/225000 [284:27:18<156:03:14,  6.89s/it]
 64%|██████▍   | 143476/225000 [284:27:21<132:54:52,  5.87s/it]
 64%|██████▍   | 143477/225000 [284:27:28<138:02:36,  6.10s/it]
 64%|██████▍   | 143478/225000 [284:27:32<125:31:38,  5.54s/it]
 64%|██████▍   | 143479/225000 [284:27:37<123:08:45,  5.44s/it]
 64%|██████▍   | 143480/225000 [284:27:42<119:45:01,  5.29s/it]
 64%|██████▍   | 143481/225000 [284:27:47<118:50:22,  5.25s/it]
 64%|██████▍   | 143482/225000 [284:27:53<120:42:43,  5.33s/it]
 64%|██████▍   | 143483/225000 [284:27:58<117:16:09,  5.18s/it]
 64%|██████▍   | 143484/225000 [284:28:03<115:14:19,  5.09s/it]
 64%|██████▍   | 143485/225000 [284:28:08<117:19:27,  5.18s/it]
 64%|██████▍   | 143486/225000 [284:28:13<119:04:34,  5.26s/it]
 64%|██████▍   | 143487/225000 [284:28:18<117:37:18,  5.19s/it]
 64%|██████▍   | 143488/225000 [284:28:24<116:32:13,  5.15s/it]
 64%|██████▍   | 143489/225000 [284:28:29<117:02:24,  5.17s/it]
 64%|██████▍   | 143490/225000 [284:28:34<118:22:13,  5.23s/it]
 64%|██████▍   | 143491/225000 [284:28:42<133:16:50,  5.89s/it]
 64%|██████▍   | 143492/225000 [284:28:53<174:14:33,  7.70s/it]
 64%|██████▍   | 143493/225000 [284:28:59<160:28:34,  7.09s/it]
 64%|██████▍   | 143494/225000 [284:29:03<137:24:34,  6.07s/it]
 64%|██████▍   | 143495/225000 [284:29:12<157:30:48,  6.96s/it]
 64%|██████▍   | 143496/225000 [284:29:16<140:59:41,  6.23s/it]
 64%|██████▍   | 143497/225000 [284:29:21<126:57:55,  5.61s/it]
 64%|██████▍   | 143498/225000 [284:29:25<118:32:06,  5.24s/it]
 64%|██████▍   | 143499/225000 [284:29:29<113:10:35,  5.00s/it]
 64%|██████▍   | 143500/225000 [284:29:34<110:28:17,  4.88s/it]
                                                               
{'loss': '0.7751', 'grad_norm': '1.021', 'learning_rate': '1.575e-05', 'epoch': '0.6378', 'num_input_tokens_seen': 471582592, 'train_runtime': '1.024e+06', 'train_tokens_per_second': '460.5'}

 64%|██████▍   | 143500/225000 [284:29:34<110:28:17,  4.88s/it]
 64%|██████▍   | 143501/225000 [284:29:45<151:32:35,  6.69s/it]
 64%|██████▍   | 143502/225000 [284:29:52<151:29:02,  6.69s/it]
 64%|██████▍   | 143503/225000 [284:29:55<128:02:47,  5.66s/it]
 64%|██████▍   | 143504/225000 [284:30:01<132:57:09,  5.87s/it]
 64%|██████▍   | 143505/225000 [284:30:09<145:41:55,  6.44s/it]
 64%|██████▍   | 143506/225000 [284:30:12<124:25:35,  5.50s/it]
 64%|██████▍   | 143507/225000 [284:30:19<133:42:06,  5.91s/it]
 64%|██████▍   | 143508/225000 [284:30:27<146:31:56,  6.47s/it]
 64%|██████▍   | 143509/225000 [284:30:30<125:20:00,  5.54s/it]
 64%|██████▍   | 143510/225000 [284:30:34<115:44:43,  5.11s/it]
 64%|██████▍   | 143511/225000 [284:30:42<131:33:45,  5.81s/it]
 64%|██████▍   | 143512/225000 [284:30:47<126:07:03,  5.57s/it]
 64%|██████▍   | 143513/225000 [284:30:53<127:53:19,  5.65s/it]
 64%|██████▍   | 143514/225000 [284:31:00<136:32:33,  6.03s/it]
 64%|██████▍   | 143515/225000 [284:31:03<117:19:31,  5.18s/it]
 64%|██████▍   | 143516/225000 [284:31:11<138:18:44,  6.11s/it]
 64%|██████▍   | 143517/225000 [284:31:14<117:08:10,  5.18s/it]
 64%|██████▍   | 143518/225000 [284:31:21<130:12:10,  5.75s/it]
 64%|██████▍   | 143519/225000 [284:31:25<116:16:40,  5.14s/it]
 64%|██████▍   | 143520/225000 [284:31:30<116:24:39,  5.14s/it]
 64%|██████▍   | 143521/225000 [284:31:38<137:49:04,  6.09s/it]
 64%|██████▍   | 143522/225000 [284:31:42<123:56:05,  5.48s/it]
 64%|██████▍   | 143523/225000 [284:31:47<115:30:55,  5.10s/it]
 64%|██████▍   | 143524/225000 [284:31:57<154:59:02,  6.85s/it]
 64%|██████▍   | 143525/225000 [284:32:01<134:01:29,  5.92s/it]
 64%|██████▍   | 143526/225000 [284:32:05<118:21:52,  5.23s/it]
 64%|██████▍   | 143527/225000 [284:32:12<128:08:07,  5.66s/it]
 64%|██████▍   | 143528/225000 [284:32:17<124:33:21,  5.50s/it]
 64%|██████▍   | 143529/225000 [284:32:23<128:41:11,  5.69s/it]
 64%|██████▍   | 143530/225000 [284:32:27<119:49:57,  5.30s/it]
 64%|██████▍   | 143531/225000 [284:32:31<113:12:49,  5.00s/it]
 64%|██████▍   | 143532/225000 [284:32:36<109:41:44,  4.85s/it]
 64%|██████▍   | 143533/225000 [284:32:45<137:13:52,  6.06s/it]
 64%|██████▍   | 143534/225000 [284:32:51<138:14:28,  6.11s/it]
 64%|██████▍   | 143535/225000 [284:32:57<137:38:32,  6.08s/it]
 64%|██████▍   | 143536/225000 [284:33:02<127:57:45,  5.65s/it]
 64%|██████▍   | 143537/225000 [284:33:06<115:06:31,  5.09s/it]
 64%|██████▍   | 143538/225000 [284:33:15<144:38:35,  6.39s/it]
 64%|██████▍   | 143539/225000 [284:33:19<128:41:40,  5.69s/it]
 64%|██████▍   | 143540/225000 [284:33:23<113:56:11,  5.04s/it]
 64%|██████▍   | 143541/225000 [284:33:29<124:58:09,  5.52s/it]
 64%|██████▍   | 143542/225000 [284:33:36<133:04:36,  5.88s/it]
 64%|██████▍   | 143543/225000 [284:33:41<126:28:01,  5.59s/it]
 64%|██████▍   | 143544/225000 [284:33:48<139:57:57,  6.19s/it]
 64%|██████▍   | 143545/225000 [284:33:52<121:02:41,  5.35s/it]
 64%|██████▍   | 143546/225000 [284:34:00<138:58:43,  6.14s/it]
 64%|██████▍   | 143547/225000 [284:34:03<121:12:32,  5.36s/it]
 64%|██████▍   | 143548/225000 [284:34:07<109:23:10,  4.83s/it]
 64%|██████▍   | 143549/225000 [284:34:17<145:39:38,  6.44s/it]
 64%|██████▍   | 143550/225000 [284:34:21<126:39:56,  5.60s/it]
                                                               
{'loss': '0.7804', 'grad_norm': '1.167', 'learning_rate': '1.573e-05', 'epoch': '0.638', 'num_input_tokens_seen': 471711928, 'train_runtime': '1.024e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143550/225000 [284:34:21<126:39:56,  5.60s/it]
 64%|██████▍   | 143551/225000 [284:34:24<112:37:19,  4.98s/it]
 64%|██████▍   | 143552/225000 [284:34:33<140:15:31,  6.20s/it]
 64%|██████▍   | 143553/225000 [284:34:37<126:28:05,  5.59s/it]
 64%|██████▍   | 143554/225000 [284:34:42<116:41:22,  5.16s/it]
 64%|██████▍   | 143555/225000 [284:34:46<109:17:12,  4.83s/it]
 64%|██████▍   | 143556/225000 [284:34:50<104:50:14,  4.63s/it]
 64%|██████▍   | 143557/225000 [284:35:01<151:02:06,  6.68s/it]
 64%|██████▍   | 143558/225000 [284:35:08<152:37:45,  6.75s/it]
 64%|██████▍   | 143559/225000 [284:35:13<139:15:20,  6.16s/it]
 64%|██████▍   | 143560/225000 [284:35:18<129:26:03,  5.72s/it]
 64%|██████▍   | 143561/225000 [284:35:23<126:00:22,  5.57s/it]
 64%|██████▍   | 143562/225000 [284:35:27<117:41:46,  5.20s/it]
 64%|██████▍   | 143563/225000 [284:35:34<127:50:00,  5.65s/it]
 64%|██████▍   | 143564/225000 [284:35:39<121:00:34,  5.35s/it]
 64%|██████▍   | 143565/225000 [284:35:43<117:37:07,  5.20s/it]
 64%|██████▍   | 143566/225000 [284:35:48<114:47:25,  5.07s/it]
 64%|██████▍   | 143567/225000 [284:35:53<109:43:05,  4.85s/it]
 64%|██████▍   | 143568/225000 [284:35:58<111:22:47,  4.92s/it]
 64%|██████▍   | 143569/225000 [284:36:11<166:08:10,  7.34s/it]
 64%|██████▍   | 143570/225000 [284:36:16<154:27:33,  6.83s/it]
 64%|██████▍   | 143571/225000 [284:36:20<136:19:38,  6.03s/it]
 64%|██████▍   | 143572/225000 [284:36:25<126:43:27,  5.60s/it]
 64%|██████▍   | 143573/225000 [284:36:30<119:28:17,  5.28s/it]
 64%|██████▍   | 143574/225000 [284:36:34<115:21:49,  5.10s/it]
 64%|██████▍   | 143575/225000 [284:36:39<113:20:24,  5.01s/it]
 64%|██████▍   | 143576/225000 [284:36:44<111:41:37,  4.94s/it]
 64%|██████▍   | 143577/225000 [284:36:52<135:56:23,  6.01s/it]
 64%|██████▍   | 143578/225000 [284:37:00<146:28:08,  6.48s/it]
 64%|██████▍   | 143579/225000 [284:37:03<125:37:28,  5.55s/it]
 64%|██████▍   | 143580/225000 [284:37:12<149:47:50,  6.62s/it]
 64%|██████▍   | 143581/225000 [284:37:16<127:18:13,  5.63s/it]
 64%|██████▍   | 143582/225000 [284:37:20<115:00:05,  5.08s/it]
 64%|██████▍   | 143583/225000 [284:37:24<113:07:19,  5.00s/it]
 64%|██████▍   | 143584/225000 [284:37:29<112:01:29,  4.95s/it]
 64%|██████▍   | 143585/225000 [284:37:34<113:05:25,  5.00s/it]
 64%|██████▍   | 143586/225000 [284:37:40<115:55:12,  5.13s/it]
 64%|██████▍   | 143587/225000 [284:37:45<115:13:30,  5.10s/it]
 64%|██████▍   | 143588/225000 [284:37:50<115:02:35,  5.09s/it]
 64%|██████▍   | 143589/225000 [284:37:55<115:35:19,  5.11s/it]
 64%|██████▍   | 143590/225000 [284:38:00<116:58:37,  5.17s/it]
 64%|██████▍   | 143591/225000 [284:38:05<116:14:57,  5.14s/it]
 64%|██████▍   | 143592/225000 [284:38:17<161:28:29,  7.14s/it]
 64%|██████▍   | 143593/225000 [284:38:25<163:18:00,  7.22s/it]
 64%|██████▍   | 143594/225000 [284:38:29<142:50:53,  6.32s/it]
 64%|██████▍   | 143595/225000 [284:38:33<128:01:07,  5.66s/it]
 64%|██████▍   | 143596/225000 [284:38:44<165:08:59,  7.30s/it]
 64%|██████▍   | 143597/225000 [284:38:48<140:50:46,  6.23s/it]
 64%|██████▍   | 143598/225000 [284:38:55<147:35:06,  6.53s/it]
 64%|██████▍   | 143599/225000 [284:38:59<127:00:10,  5.62s/it]
 64%|██████▍   | 143600/225000 [284:39:02<113:30:47,  5.02s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.068', 'learning_rate': '1.572e-05', 'epoch': '0.6382', 'num_input_tokens_seen': 471840976, 'train_runtime': '1.025e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143600/225000 [284:39:02<113:30:47,  5.02s/it]
 64%|██████▍   | 143601/225000 [284:39:11<139:37:00,  6.17s/it]
 64%|██████▍   | 143602/225000 [284:39:16<132:54:07,  5.88s/it]
 64%|██████▍   | 143603/225000 [284:39:21<124:25:12,  5.50s/it]
 64%|██████▍   | 143604/225000 [284:39:25<115:37:57,  5.11s/it]
 64%|██████▍   | 143605/225000 [284:39:35<148:00:30,  6.55s/it]
 64%|██████▍   | 143606/225000 [284:39:41<147:11:54,  6.51s/it]
 64%|██████▍   | 143607/225000 [284:39:45<126:47:03,  5.61s/it]
 64%|██████▍   | 143608/225000 [284:39:53<144:07:19,  6.37s/it]
 64%|██████▍   | 143609/225000 [284:39:57<127:07:21,  5.62s/it]
 64%|██████▍   | 143610/225000 [284:40:01<114:31:00,  5.07s/it]
 64%|██████▍   | 143611/225000 [284:40:11<149:47:18,  6.63s/it]
 64%|██████▍   | 143612/225000 [284:40:15<131:43:41,  5.83s/it]
 64%|██████▍   | 143613/225000 [284:40:20<124:02:32,  5.49s/it]
 64%|██████▍   | 143614/225000 [284:40:24<115:40:56,  5.12s/it]
 64%|██████▍   | 143615/225000 [284:40:28<109:12:27,  4.83s/it]
 64%|██████▍   | 143616/225000 [284:40:33<107:40:08,  4.76s/it]
 64%|██████▍   | 143617/225000 [284:40:37<103:48:39,  4.59s/it]
 64%|██████▍   | 143618/225000 [284:40:41<102:03:58,  4.51s/it]
 64%|██████▍   | 143619/225000 [284:40:54<158:26:55,  7.01s/it]
 64%|██████▍   | 143620/225000 [284:40:58<137:23:30,  6.08s/it]
 64%|██████▍   | 143621/225000 [284:41:02<123:16:23,  5.45s/it]
 64%|██████▍   | 143622/225000 [284:41:11<149:47:44,  6.63s/it]
 64%|██████▍   | 143623/225000 [284:41:15<131:41:00,  5.83s/it]
 64%|██████▍   | 143624/225000 [284:41:23<143:02:10,  6.33s/it]
 64%|██████▍   | 143625/225000 [284:41:28<135:47:28,  6.01s/it]
 64%|██████▍   | 143626/225000 [284:41:36<147:51:25,  6.54s/it]
 64%|██████▍   | 143627/225000 [284:41:40<130:26:01,  5.77s/it]
 64%|██████▍   | 143628/225000 [284:41:44<119:48:20,  5.30s/it]
 64%|██████▍   | 143629/225000 [284:41:48<113:07:24,  5.00s/it]
 64%|██████▍   | 143630/225000 [284:41:53<109:18:56,  4.84s/it]
 64%|██████▍   | 143631/225000 [284:42:02<142:59:48,  6.33s/it]
 64%|██████▍   | 143632/225000 [284:42:06<127:19:50,  5.63s/it]
 64%|██████▍   | 143633/225000 [284:42:10<116:08:25,  5.14s/it]
 64%|██████▍   | 143634/225000 [284:42:19<139:34:13,  6.18s/it]
 64%|██████▍   | 143635/225000 [284:42:24<131:34:07,  5.82s/it]
 64%|██████▍   | 143636/225000 [284:42:27<114:23:53,  5.06s/it]
 64%|██████▍   | 143637/225000 [284:42:31<107:09:02,  4.74s/it]
 64%|██████▍   | 143638/225000 [284:42:41<139:06:59,  6.16s/it]
 64%|██████▍   | 143639/225000 [284:42:46<130:03:45,  5.75s/it]
 64%|██████▍   | 143640/225000 [284:42:49<115:56:44,  5.13s/it]
 64%|██████▍   | 143641/225000 [284:42:59<146:24:31,  6.48s/it]
 64%|██████▍   | 143642/225000 [284:43:02<125:55:35,  5.57s/it]
 64%|██████▍   | 143643/225000 [284:43:07<116:24:33,  5.15s/it]
 64%|██████▍   | 143644/225000 [284:43:15<140:01:55,  6.20s/it]
 64%|██████▍   | 143645/225000 [284:43:19<126:22:58,  5.59s/it]
 64%|██████▍   | 143646/225000 [284:43:24<118:13:28,  5.23s/it]
 64%|██████▍   | 143647/225000 [284:43:28<111:32:11,  4.94s/it]
 64%|██████▍   | 143648/225000 [284:43:32<108:02:13,  4.78s/it]
 64%|██████▍   | 143649/225000 [284:43:37<107:21:36,  4.75s/it]
 64%|██████▍   | 143650/225000 [284:43:48<149:03:29,  6.60s/it]
                                                               
{'loss': '0.7873', 'grad_norm': '1.233', 'learning_rate': '1.57e-05', 'epoch': '0.6384', 'num_input_tokens_seen': 471971696, 'train_runtime': '1.025e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143650/225000 [284:43:48<149:03:29,  6.60s/it]
 64%|██████▍   | 143651/225000 [284:43:54<145:14:36,  6.43s/it]
 64%|██████▍   | 143652/225000 [284:43:58<130:22:16,  5.77s/it]
 64%|██████▍   | 143653/225000 [284:44:03<121:26:09,  5.37s/it]
 64%|██████▍   | 143654/225000 [284:44:07<114:13:59,  5.06s/it]
 64%|██████▍   | 143655/225000 [284:44:19<159:02:01,  7.04s/it]
 64%|██████▍   | 143656/225000 [284:44:22<132:57:10,  5.88s/it]
 64%|██████▍   | 143657/225000 [284:44:30<148:09:33,  6.56s/it]
 64%|██████▍   | 143658/225000 [284:44:35<135:03:32,  5.98s/it]
 64%|██████▍   | 143659/225000 [284:44:39<121:00:05,  5.36s/it]
 64%|██████▍   | 143660/225000 [284:44:43<112:05:19,  4.96s/it]
 64%|██████▍   | 143661/225000 [284:44:47<109:00:07,  4.82s/it]
 64%|██████▍   | 143662/225000 [284:44:57<145:57:54,  6.46s/it]
 64%|██████▍   | 143663/225000 [284:45:03<143:07:50,  6.34s/it]
 64%|██████▍   | 143664/225000 [284:45:07<126:50:39,  5.61s/it]
 64%|██████▍   | 143665/225000 [284:45:15<139:59:34,  6.20s/it]
 64%|██████▍   | 143666/225000 [284:45:21<136:17:35,  6.03s/it]
 64%|██████▍   | 143667/225000 [284:45:25<124:48:06,  5.52s/it]
 64%|██████▍   | 143668/225000 [284:45:29<118:35:23,  5.25s/it]
 64%|██████▍   | 143669/225000 [284:45:34<110:53:46,  4.91s/it]
 64%|██████▍   | 143670/225000 [284:45:43<144:28:43,  6.40s/it]
 64%|██████▍   | 143671/225000 [284:45:47<124:26:11,  5.51s/it]
 64%|██████▍   | 143672/225000 [284:45:52<119:44:47,  5.30s/it]
 64%|██████▍   | 143673/225000 [284:45:57<117:03:21,  5.18s/it]
 64%|██████▍   | 143674/225000 [284:46:02<119:33:18,  5.29s/it]
 64%|██████▍   | 143675/225000 [284:46:08<126:38:54,  5.61s/it]
 64%|██████▍   | 143676/225000 [284:46:16<141:28:29,  6.26s/it]
 64%|██████▍   | 143677/225000 [284:46:22<140:02:00,  6.20s/it]
 64%|██████▍   | 143678/225000 [284:46:28<137:33:40,  6.09s/it]
 64%|██████▍   | 143679/225000 [284:46:32<121:06:18,  5.36s/it]
 64%|██████▍   | 143680/225000 [284:46:36<112:34:29,  4.98s/it]
 64%|██████▍   | 143681/225000 [284:46:43<128:50:33,  5.70s/it]
 64%|██████▍   | 143682/225000 [284:46:52<149:03:04,  6.60s/it]
 64%|██████▍   | 143683/225000 [284:46:55<127:25:46,  5.64s/it]
 64%|██████▍   | 143684/225000 [284:46:59<113:51:23,  5.04s/it]
 64%|██████▍   | 143685/225000 [284:47:04<114:25:51,  5.07s/it]
 64%|██████▍   | 143686/225000 [284:47:09<113:53:35,  5.04s/it]
 64%|██████▍   | 143687/225000 [284:47:14<115:19:09,  5.11s/it]
 64%|██████▍   | 143688/225000 [284:47:20<116:38:03,  5.16s/it]
 64%|██████▍   | 143689/225000 [284:47:25<117:08:44,  5.19s/it]
 64%|██████▍   | 143690/225000 [284:47:30<114:57:13,  5.09s/it]
 64%|██████▍   | 143691/225000 [284:47:35<113:23:29,  5.02s/it]
 64%|██████▍   | 143692/225000 [284:47:48<166:19:56,  7.36s/it]
 64%|██████▍   | 143693/225000 [284:47:52<148:24:51,  6.57s/it]
 64%|██████▍   | 143694/225000 [284:47:58<140:55:17,  6.24s/it]
 64%|██████▍   | 143695/225000 [284:48:03<134:53:23,  5.97s/it]
 64%|██████▍   | 143696/225000 [284:48:12<152:36:56,  6.76s/it]
 64%|██████▍   | 143697/225000 [284:48:16<133:06:22,  5.89s/it]
 64%|██████▍   | 143698/225000 [284:48:20<123:16:50,  5.46s/it]
 64%|██████▍   | 143699/225000 [284:48:24<116:38:29,  5.16s/it]
 64%|██████▍   | 143700/225000 [284:48:29<112:55:03,  5.00s/it]
                                                               
{'loss': '0.7843', 'grad_norm': '1.122', 'learning_rate': '1.568e-05', 'epoch': '0.6387', 'num_input_tokens_seen': 472103816, 'train_runtime': '1.025e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143700/225000 [284:48:29<112:55:03,  5.00s/it]
 64%|██████▍   | 143701/225000 [284:48:37<135:15:00,  5.99s/it]
 64%|██████▍   | 143702/225000 [284:48:43<131:30:10,  5.82s/it]
 64%|██████▍   | 143703/225000 [284:48:46<115:30:56,  5.12s/it]
 64%|██████▍   | 143704/225000 [284:48:56<145:47:47,  6.46s/it]
 64%|██████▍   | 143705/225000 [284:49:03<148:04:50,  6.56s/it]
 64%|██████▍   | 143706/225000 [284:49:07<133:39:35,  5.92s/it]
 64%|██████▍   | 143707/225000 [284:49:11<117:41:42,  5.21s/it]
 64%|██████▍   | 143708/225000 [284:49:19<137:39:29,  6.10s/it]
 64%|██████▍   | 143709/225000 [284:49:23<123:14:37,  5.46s/it]
 64%|██████▍   | 143710/225000 [284:49:26<111:26:38,  4.94s/it]
 64%|██████▍   | 143711/225000 [284:49:35<135:24:05,  6.00s/it]
 64%|██████▍   | 143712/225000 [284:49:39<119:14:59,  5.28s/it]
 64%|██████▍   | 143713/225000 [284:49:47<140:12:24,  6.21s/it]
 64%|██████▍   | 143714/225000 [284:49:52<134:59:51,  5.98s/it]
 64%|██████▍   | 143715/225000 [284:49:56<120:55:17,  5.36s/it]
 64%|██████▍   | 143716/225000 [284:50:00<112:08:55,  4.97s/it]
 64%|██████▍   | 143717/225000 [284:50:04<106:24:10,  4.71s/it]
 64%|██████▍   | 143718/225000 [284:50:09<108:40:42,  4.81s/it]
 64%|██████▍   | 143719/225000 [284:50:20<149:15:13,  6.61s/it]
 64%|██████▍   | 143720/225000 [284:50:24<132:30:46,  5.87s/it]
 64%|██████▍   | 143721/225000 [284:50:29<123:08:12,  5.45s/it]
 64%|██████▍   | 143722/225000 [284:50:33<116:25:54,  5.16s/it]
 64%|██████▍   | 143723/225000 [284:50:38<113:12:47,  5.01s/it]
 64%|██████▍   | 143724/225000 [284:50:43<115:59:35,  5.14s/it]
 64%|██████▍   | 143725/225000 [284:50:55<160:16:55,  7.10s/it]
 64%|██████▍   | 143726/225000 [284:50:58<133:09:53,  5.90s/it]
 64%|██████▍   | 143727/225000 [284:51:02<117:48:18,  5.22s/it]
 64%|██████▍   | 143728/225000 [284:51:10<139:09:15,  6.16s/it]
 64%|██████▍   | 143729/225000 [284:51:15<126:17:21,  5.59s/it]
 64%|██████▍   | 143730/225000 [284:51:22<138:40:26,  6.14s/it]
 64%|██████▍   | 143731/225000 [284:51:27<131:50:12,  5.84s/it]
 64%|██████▍   | 143732/225000 [284:51:32<125:46:08,  5.57s/it]
 64%|██████▍   | 143733/225000 [284:51:37<119:46:37,  5.31s/it]
 64%|██████▍   | 143734/225000 [284:51:41<112:17:25,  4.97s/it]
 64%|██████▍   | 143735/225000 [284:51:46<112:11:58,  4.97s/it]
 64%|██████▍   | 143736/225000 [284:51:51<110:20:50,  4.89s/it]
 64%|██████▍   | 143737/225000 [284:51:55<106:45:04,  4.73s/it]
 64%|██████▍   | 143738/225000 [284:52:00<108:10:09,  4.79s/it]
 64%|██████▍   | 143739/225000 [284:52:13<162:46:59,  7.21s/it]
 64%|██████▍   | 143740/225000 [284:52:16<139:12:44,  6.17s/it]
 64%|██████▍   | 143741/225000 [284:52:20<122:31:09,  5.43s/it]
 64%|██████▍   | 143742/225000 [284:52:24<112:40:52,  4.99s/it]
 64%|██████▍   | 143743/225000 [284:52:31<126:02:47,  5.58s/it]
 64%|██████▍   | 143744/225000 [284:52:39<142:10:58,  6.30s/it]
 64%|██████▍   | 143745/225000 [284:52:43<126:30:46,  5.61s/it]
 64%|██████▍   | 143746/225000 [284:52:47<116:59:48,  5.18s/it]
 64%|██████▍   | 143747/225000 [284:52:51<109:42:57,  4.86s/it]
 64%|██████▍   | 143748/225000 [284:52:59<126:07:45,  5.59s/it]
 64%|██████▍   | 143749/225000 [284:53:08<149:06:33,  6.61s/it]
 64%|██████▍   | 143750/225000 [284:53:11<130:27:14,  5.78s/it]
                                                               
{'loss': '0.7774', 'grad_norm': '1.13', 'learning_rate': '1.567e-05', 'epoch': '0.6389', 'num_input_tokens_seen': 472233176, 'train_runtime': '1.026e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143750/225000 [284:53:11<130:27:14,  5.78s/it]
 64%|██████▍   | 143751/225000 [284:53:20<147:40:39,  6.54s/it]
 64%|██████▍   | 143752/225000 [284:53:24<131:23:33,  5.82s/it]
 64%|██████▍   | 143753/225000 [284:53:30<134:19:35,  5.95s/it]
 64%|██████▍   | 143754/225000 [284:53:35<124:53:21,  5.53s/it]
 64%|██████▍   | 143755/225000 [284:53:39<119:25:23,  5.29s/it]
 64%|██████▍   | 143756/225000 [284:53:48<144:09:09,  6.39s/it]
 64%|██████▍   | 143757/225000 [284:53:52<124:05:39,  5.50s/it]
 64%|██████▍   | 143758/225000 [284:54:01<150:11:02,  6.65s/it]
 64%|██████▍   | 143759/225000 [284:54:05<132:26:34,  5.87s/it]
 64%|██████▍   | 143760/225000 [284:54:09<120:09:03,  5.32s/it]
 64%|██████▍   | 143761/225000 [284:54:13<111:37:31,  4.95s/it]
 64%|██████▍   | 143762/225000 [284:54:23<140:38:53,  6.23s/it]
 64%|██████▍   | 143763/225000 [284:54:27<127:50:48,  5.67s/it]
 64%|██████▍   | 143764/225000 [284:54:31<119:30:01,  5.30s/it]
 64%|██████▍   | 143765/225000 [284:54:36<111:42:08,  4.95s/it]
 64%|██████▍   | 143766/225000 [284:54:40<110:37:22,  4.90s/it]
 64%|██████▍   | 143767/225000 [284:54:45<108:28:24,  4.81s/it]
 64%|██████▍   | 143768/225000 [284:54:50<108:14:28,  4.80s/it]
 64%|██████▍   | 143769/225000 [284:55:02<157:12:41,  6.97s/it]
 64%|██████▍   | 143770/225000 [284:55:07<145:06:30,  6.43s/it]
 64%|██████▍   | 143771/225000 [284:55:11<130:07:18,  5.77s/it]
 64%|██████▍   | 143772/225000 [284:55:16<123:34:19,  5.48s/it]
 64%|██████▍   | 143773/225000 [284:55:25<149:17:44,  6.62s/it]
 64%|██████▍   | 143774/225000 [284:55:29<128:30:21,  5.70s/it]
 64%|██████▍   | 143775/225000 [284:55:36<138:25:13,  6.13s/it]
 64%|██████▍   | 143776/225000 [284:55:40<124:24:10,  5.51s/it]
 64%|██████▍   | 143777/225000 [284:55:44<115:11:11,  5.11s/it]
 64%|██████▍   | 143778/225000 [284:55:51<128:07:27,  5.68s/it]
 64%|██████▍   | 143779/225000 [284:55:59<141:36:48,  6.28s/it]
 64%|██████▍   | 143780/225000 [284:56:02<122:23:19,  5.42s/it]
 64%|██████▍   | 143781/225000 [284:56:10<138:42:07,  6.15s/it]
 64%|██████▍   | 143782/225000 [284:56:14<124:25:27,  5.52s/it]
 64%|██████▍   | 143783/225000 [284:56:18<114:16:39,  5.07s/it]
 64%|██████▍   | 143784/225000 [284:56:22<106:43:31,  4.73s/it]
 64%|██████▍   | 143785/225000 [284:56:26<102:38:21,  4.55s/it]
 64%|██████▍   | 143786/225000 [284:56:32<112:16:53,  4.98s/it]
 64%|██████▍   | 143787/225000 [284:56:43<153:31:13,  6.81s/it]
 64%|██████▍   | 143788/225000 [284:56:46<128:18:49,  5.69s/it]
 64%|██████▍   | 143789/225000 [284:56:54<142:37:02,  6.32s/it]
 64%|██████▍   | 143790/225000 [284:56:58<122:49:06,  5.44s/it]
 64%|██████▍   | 143791/225000 [284:57:06<142:05:19,  6.30s/it]
 64%|██████▍   | 143792/225000 [284:57:10<130:48:24,  5.80s/it]
 64%|██████▍   | 143793/225000 [284:57:15<119:38:22,  5.30s/it]
 64%|██████▍   | 143794/225000 [284:57:19<113:09:58,  5.02s/it]
 64%|██████▍   | 143795/225000 [284:57:23<109:36:30,  4.86s/it]
 64%|██████▍   | 143796/225000 [284:57:35<155:01:48,  6.87s/it]
 64%|██████▍   | 143797/225000 [284:57:39<133:15:29,  5.91s/it]
 64%|██████▍   | 143798/225000 [284:57:42<117:07:21,  5.19s/it]
 64%|██████▍   | 143799/225000 [284:57:46<107:57:20,  4.79s/it]
 64%|██████▍   | 143800/225000 [284:57:56<144:40:42,  6.41s/it]
                                                               
{'loss': '0.7827', 'grad_norm': '1.048', 'learning_rate': '1.565e-05', 'epoch': '0.6391', 'num_input_tokens_seen': 472363800, 'train_runtime': '1.026e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143800/225000 [284:57:56<144:40:42,  6.41s/it]
 64%|██████▍   | 143801/225000 [284:58:01<134:36:15,  5.97s/it]
 64%|██████▍   | 143802/225000 [284:58:06<126:27:17,  5.61s/it]
 64%|██████▍   | 143803/225000 [284:58:11<121:08:50,  5.37s/it]
 64%|██████▍   | 143804/225000 [284:58:15<113:41:05,  5.04s/it]
 64%|██████▍   | 143805/225000 [284:58:20<111:08:09,  4.93s/it]
 64%|██████▍   | 143806/225000 [284:58:25<110:45:15,  4.91s/it]
 64%|██████▍   | 143807/225000 [284:58:29<109:54:48,  4.87s/it]
 64%|██████▍   | 143808/225000 [284:58:42<162:22:12,  7.20s/it]
 64%|██████▍   | 143809/225000 [284:58:48<151:48:05,  6.73s/it]
 64%|██████▍   | 143810/225000 [284:58:52<133:55:55,  5.94s/it]
 64%|██████▍   | 143811/225000 [284:58:56<124:41:45,  5.53s/it]
 64%|██████▍   | 143812/225000 [284:59:01<116:30:17,  5.17s/it]
 64%|██████▍   | 143813/225000 [284:59:06<120:01:17,  5.32s/it]
 64%|██████▍   | 143814/225000 [284:59:16<147:29:03,  6.54s/it]
 64%|██████▍   | 143815/225000 [284:59:19<126:08:24,  5.59s/it]
 64%|██████▍   | 143816/225000 [284:59:24<122:56:18,  5.45s/it]
 64%|██████▍   | 143817/225000 [284:59:31<132:45:49,  5.89s/it]
 64%|██████▍   | 143818/225000 [284:59:37<134:40:27,  5.97s/it]
 64%|██████▍   | 143819/225000 [284:59:43<132:08:12,  5.86s/it]
 64%|██████▍   | 143820/225000 [284:59:51<150:29:53,  6.67s/it]
 64%|██████▍   | 143821/225000 [284:59:55<128:32:56,  5.70s/it]
 64%|██████▍   | 143822/225000 [285:00:03<145:56:08,  6.47s/it]
 64%|██████▍   | 143823/225000 [285:00:07<130:35:05,  5.79s/it]
 64%|██████▍   | 143824/225000 [285:00:13<133:17:42,  5.91s/it]
 64%|██████▍   | 143825/225000 [285:00:18<121:21:55,  5.38s/it]
 64%|██████▍   | 143826/225000 [285:00:22<111:58:01,  4.97s/it]
 64%|██████▍   | 143827/225000 [285:00:26<105:43:24,  4.69s/it]
 64%|██████▍   | 143828/225000 [285:00:30<103:11:55,  4.58s/it]
 64%|██████▍   | 143829/225000 [285:00:38<125:38:03,  5.57s/it]
 64%|██████▍   | 143830/225000 [285:00:48<154:53:41,  6.87s/it]
 64%|██████▍   | 143831/225000 [285:00:51<131:54:19,  5.85s/it]
 64%|██████▍   | 143832/225000 [285:00:55<118:02:04,  5.24s/it]
 64%|██████▍   | 143833/225000 [285:01:00<117:54:03,  5.23s/it]
 64%|██████▍   | 143834/225000 [285:01:10<148:12:14,  6.57s/it]
 64%|██████▍   | 143835/225000 [285:01:14<131:07:08,  5.82s/it]
 64%|██████▍   | 143836/225000 [285:01:18<118:07:06,  5.24s/it]
 64%|██████▍   | 143837/225000 [285:01:22<110:04:08,  4.88s/it]
 64%|██████▍   | 143838/225000 [285:01:32<147:17:54,  6.53s/it]
 64%|██████▍   | 143839/225000 [285:01:36<124:56:06,  5.54s/it]
 64%|██████▍   | 143840/225000 [285:01:40<114:45:33,  5.09s/it]
 64%|██████▍   | 143841/225000 [285:01:45<116:03:46,  5.15s/it]
 64%|██████▍   | 143842/225000 [285:01:54<142:44:52,  6.33s/it]
 64%|██████▍   | 143843/225000 [285:01:58<126:30:06,  5.61s/it]
 64%|██████▍   | 143844/225000 [285:02:02<118:44:45,  5.27s/it]
 64%|██████▍   | 143845/225000 [285:02:12<147:13:45,  6.53s/it]
 64%|██████▍   | 143846/225000 [285:02:16<129:01:07,  5.72s/it]
 64%|██████▍   | 143847/225000 [285:02:19<113:56:30,  5.05s/it]
 64%|██████▍   | 143848/225000 [285:02:29<144:50:39,  6.43s/it]
 64%|██████▍   | 143849/225000 [285:02:32<125:07:22,  5.55s/it]
 64%|██████▍   | 143850/225000 [285:02:37<118:15:35,  5.25s/it]
                                                               
{'loss': '0.7759', 'grad_norm': '1.009', 'learning_rate': '1.563e-05', 'epoch': '0.6393', 'num_input_tokens_seen': 472493728, 'train_runtime': '1.026e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143850/225000 [285:02:37<118:15:35,  5.25s/it]
 64%|██████▍   | 143851/225000 [285:02:41<113:23:12,  5.03s/it]
 64%|██████▍   | 143852/225000 [285:02:46<109:14:18,  4.85s/it]
 64%|██████▍   | 143853/225000 [285:02:51<108:38:14,  4.82s/it]
 64%|██████▍   | 143854/225000 [285:03:00<141:37:12,  6.28s/it]
 64%|██████▍   | 143855/225000 [285:03:08<149:05:54,  6.61s/it]
 64%|██████▍   | 143856/225000 [285:03:12<135:46:03,  6.02s/it]
 64%|██████▍   | 143857/225000 [285:03:17<127:03:01,  5.64s/it]
 64%|██████▍   | 143858/225000 [285:03:22<123:15:06,  5.47s/it]
 64%|██████▍   | 143859/225000 [285:03:27<116:49:42,  5.18s/it]
 64%|██████▍   | 143860/225000 [285:03:31<114:25:53,  5.08s/it]
 64%|██████▍   | 143861/225000 [285:03:36<112:37:47,  5.00s/it]
 64%|██████▍   | 143862/225000 [285:03:41<110:29:34,  4.90s/it]
 64%|██████▍   | 143863/225000 [285:03:52<151:34:03,  6.72s/it]
 64%|██████▍   | 143864/225000 [285:03:58<148:03:46,  6.57s/it]
 64%|██████▍   | 143865/225000 [285:04:02<131:40:39,  5.84s/it]
 64%|██████▍   | 143866/225000 [285:04:07<120:53:50,  5.36s/it]
 64%|██████▍   | 143867/225000 [285:04:16<149:50:39,  6.65s/it]
 64%|██████▍   | 143868/225000 [285:04:20<129:11:28,  5.73s/it]
 64%|██████▍   | 143869/225000 [285:04:27<140:25:55,  6.23s/it]
 64%|██████▍   | 143870/225000 [285:04:32<131:41:45,  5.84s/it]
 64%|██████▍   | 143871/225000 [285:04:37<127:09:14,  5.64s/it]
 64%|██████▍   | 143872/225000 [285:04:42<121:32:03,  5.39s/it]
 64%|██████▍   | 143873/225000 [285:04:47<118:58:13,  5.28s/it]
 64%|██████▍   | 143874/225000 [285:04:52<114:33:53,  5.08s/it]
 64%|██████▍   | 143875/225000 [285:05:03<155:11:11,  6.89s/it]
 64%|██████▍   | 143876/225000 [285:05:08<144:03:06,  6.39s/it]
 64%|██████▍   | 143877/225000 [285:05:12<124:58:28,  5.55s/it]
 64%|██████▍   | 143878/225000 [285:05:17<123:52:31,  5.50s/it]
 64%|██████▍   | 143879/225000 [285:05:22<119:49:53,  5.32s/it]
 64%|██████▍   | 143880/225000 [285:05:27<117:21:54,  5.21s/it]
 64%|██████▍   | 143881/225000 [285:05:32<116:11:01,  5.16s/it]
 64%|██████▍   | 143882/225000 [285:05:37<115:45:00,  5.14s/it]
 64%|██████▍   | 143883/225000 [285:05:42<114:27:20,  5.08s/it]
 64%|██████▍   | 143884/225000 [285:05:53<156:06:03,  6.93s/it]
 64%|██████▍   | 143885/225000 [285:05:58<141:26:17,  6.28s/it]
 64%|██████▍   | 143886/225000 [285:06:03<132:16:49,  5.87s/it]
 64%|██████▍   | 143887/225000 [285:06:08<124:52:28,  5.54s/it]
 64%|██████▍   | 143888/225000 [285:06:12<119:08:05,  5.29s/it]
 64%|██████▍   | 143889/225000 [285:06:17<116:24:32,  5.17s/it]
 64%|██████▍   | 143890/225000 [285:06:22<116:51:59,  5.19s/it]
 64%|██████▍   | 143891/225000 [285:06:27<114:23:26,  5.08s/it]
 64%|██████▍   | 143892/225000 [285:06:32<113:38:33,  5.04s/it]
 64%|██████▍   | 143893/225000 [285:06:40<133:45:34,  5.94s/it]
 64%|██████▍   | 143894/225000 [285:06:51<166:04:28,  7.37s/it]
 64%|██████▍   | 143895/225000 [285:06:56<152:56:00,  6.79s/it]
 64%|██████▍   | 143896/225000 [285:07:02<142:52:28,  6.34s/it]
 64%|██████▍   | 143897/225000 [285:07:07<136:55:06,  6.08s/it]
 64%|██████▍   | 143898/225000 [285:07:12<129:26:38,  5.75s/it]
 64%|██████▍   | 143899/225000 [285:07:17<126:18:06,  5.61s/it]
 64%|██████▍   | 143900/225000 [285:07:23<126:57:38,  5.64s/it]
                                                               
{'loss': '0.7844', 'grad_norm': '1.058', 'learning_rate': '1.562e-05', 'epoch': '0.6396', 'num_input_tokens_seen': 472623320, 'train_runtime': '1.026e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143900/225000 [285:07:23<126:57:38,  5.64s/it]
 64%|██████▍   | 143901/225000 [285:07:29<125:35:02,  5.57s/it]
 64%|██████▍   | 143902/225000 [285:07:34<124:43:10,  5.54s/it]
 64%|██████▍   | 143903/225000 [285:07:39<123:41:08,  5.49s/it]
 64%|██████▍   | 143904/225000 [285:07:45<124:22:51,  5.52s/it]
 64%|██████▍   | 143905/225000 [285:07:51<124:52:27,  5.54s/it]
 64%|██████▍   | 143906/225000 [285:07:56<124:06:40,  5.51s/it]
 64%|██████▍   | 143907/225000 [285:08:02<124:21:27,  5.52s/it]
 64%|██████▍   | 143908/225000 [285:08:07<124:52:06,  5.54s/it]
 64%|██████▍   | 143909/225000 [285:08:13<124:17:36,  5.52s/it]
 64%|██████▍   | 143910/225000 [285:08:18<123:02:29,  5.46s/it]
 64%|██████▍   | 143911/225000 [285:08:29<161:21:47,  7.16s/it]
 64%|██████▍   | 143912/225000 [285:08:32<134:46:14,  5.98s/it]
 64%|██████▍   | 143913/225000 [285:08:39<141:24:50,  6.28s/it]
 64%|██████▍   | 143914/225000 [285:08:44<129:10:44,  5.74s/it]
 64%|██████▍   | 143915/225000 [285:08:48<121:21:52,  5.39s/it]
 64%|██████▍   | 143916/225000 [285:08:53<115:53:10,  5.15s/it]
 64%|██████▍   | 143917/225000 [285:08:58<114:52:20,  5.10s/it]
 64%|██████▍   | 143918/225000 [285:09:02<110:05:44,  4.89s/it]
 64%|██████▍   | 143919/225000 [285:09:07<106:02:10,  4.71s/it]
 64%|██████▍   | 143920/225000 [285:09:11<105:27:11,  4.68s/it]
 64%|██████▍   | 143921/225000 [285:09:24<159:15:06,  7.07s/it]
 64%|██████▍   | 143922/225000 [285:09:29<143:01:35,  6.35s/it]
 64%|██████▍   | 143923/225000 [285:09:32<122:50:23,  5.45s/it]
 64%|██████▍   | 143924/225000 [285:09:36<114:17:29,  5.07s/it]
 64%|██████▍   | 143925/225000 [285:09:41<111:51:56,  4.97s/it]
 64%|██████▍   | 143926/225000 [285:09:46<111:27:05,  4.95s/it]
 64%|██████▍   | 143927/225000 [285:09:50<109:19:33,  4.85s/it]
 64%|██████▍   | 143928/225000 [285:10:03<161:42:21,  7.18s/it]
 64%|██████▍   | 143929/225000 [285:10:08<148:53:50,  6.61s/it]
 64%|██████▍   | 143930/225000 [285:10:16<155:03:15,  6.89s/it]
 64%|██████▍   | 143931/225000 [285:10:20<136:29:20,  6.06s/it]
 64%|██████▍   | 143932/225000 [285:10:24<123:04:01,  5.47s/it]
 64%|██████▍   | 143933/225000 [285:10:28<111:50:58,  4.97s/it]
 64%|██████▍   | 143934/225000 [285:10:38<146:26:03,  6.50s/it]
 64%|██████▍   | 143935/225000 [285:10:41<125:51:09,  5.59s/it]
 64%|██████▍   | 143936/225000 [285:10:50<147:05:05,  6.53s/it]
 64%|██████▍   | 143937/225000 [285:10:53<125:25:57,  5.57s/it]
 64%|██████▍   | 143938/225000 [285:11:01<138:10:13,  6.14s/it]
 64%|██████▍   | 143939/225000 [285:11:07<135:39:33,  6.02s/it]
 64%|██████▍   | 143940/225000 [285:11:11<127:55:40,  5.68s/it]
 64%|██████▍   | 143941/225000 [285:11:17<126:06:34,  5.60s/it]
 64%|██████▍   | 143942/225000 [285:11:22<124:23:09,  5.52s/it]
 64%|██████▍   | 143943/225000 [285:11:28<124:24:49,  5.53s/it]
 64%|██████▍   | 143944/225000 [285:11:33<122:54:38,  5.46s/it]
 64%|██████▍   | 143945/225000 [285:11:39<123:16:30,  5.48s/it]
 64%|██████▍   | 143946/225000 [285:11:43<119:33:34,  5.31s/it]
 64%|██████▍   | 143947/225000 [285:11:49<120:02:03,  5.33s/it]
 64%|██████▍   | 143948/225000 [285:11:54<120:54:07,  5.37s/it]
 64%|██████▍   | 143949/225000 [285:12:00<120:12:06,  5.34s/it]
 64%|██████▍   | 143950/225000 [285:12:05<119:43:47,  5.32s/it]
                                                               
{'loss': '0.7829', 'grad_norm': '1.112', 'learning_rate': '1.56e-05', 'epoch': '0.6398', 'num_input_tokens_seen': 472753304, 'train_runtime': '1.027e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 143950/225000 [285:12:05<119:43:47,  5.32s/it]
 64%|██████▍   | 143951/225000 [285:12:10<118:16:54,  5.25s/it]
 64%|██████▍   | 143952/225000 [285:12:16<125:07:52,  5.56s/it]
 64%|██████▍   | 143953/225000 [285:12:25<147:33:13,  6.55s/it]
 64%|██████▍   | 143954/225000 [285:12:33<157:40:09,  7.00s/it]
 64%|██████▍   | 143955/225000 [285:12:37<138:44:24,  6.16s/it]
 64%|██████▍   | 143956/225000 [285:12:43<135:36:13,  6.02s/it]
 64%|██████▍   | 143957/225000 [285:12:46<116:25:21,  5.17s/it]
 64%|██████▍   | 143958/225000 [285:12:56<144:37:48,  6.42s/it]
 64%|██████▍   | 143959/225000 [285:12:59<127:10:19,  5.65s/it]
 64%|██████▍   | 143960/225000 [285:13:07<137:58:45,  6.13s/it]
 64%|██████▍   | 143961/225000 [285:13:11<128:32:17,  5.71s/it]
 64%|██████▍   | 143962/225000 [285:13:15<115:20:22,  5.12s/it]
 64%|██████▍   | 143963/225000 [285:13:25<146:56:27,  6.53s/it]
 64%|██████▍   | 143964/225000 [285:13:29<129:38:19,  5.76s/it]
 64%|██████▍   | 143965/225000 [285:13:33<116:42:32,  5.18s/it]
 64%|██████▍   | 143966/225000 [285:13:37<109:25:35,  4.86s/it]
 64%|██████▍   | 143967/225000 [285:13:47<143:05:38,  6.36s/it]
 64%|██████▍   | 143968/225000 [285:13:50<122:38:42,  5.45s/it]
 64%|██████▍   | 143969/225000 [285:13:57<131:29:18,  5.84s/it]
 64%|██████▍   | 143970/225000 [285:14:00<114:47:14,  5.10s/it]
 64%|██████▍   | 143971/225000 [285:14:08<130:14:40,  5.79s/it]
 64%|██████▍   | 143972/225000 [285:14:12<120:03:26,  5.33s/it]
 64%|██████▍   | 143973/225000 [285:14:16<114:10:23,  5.07s/it]
 64%|██████▍   | 143974/225000 [285:14:23<123:00:57,  5.47s/it]
 64%|██████▍   | 143975/225000 [285:14:33<153:11:37,  6.81s/it]
 64%|██████▍   | 143976/225000 [285:14:36<130:20:07,  5.79s/it]
 64%|██████▍   | 143977/225000 [285:14:40<117:22:09,  5.21s/it]
 64%|██████▍   | 143978/225000 [285:14:44<112:00:49,  4.98s/it]
 64%|██████▍   | 143979/225000 [285:14:49<106:26:15,  4.73s/it]
 64%|██████▍   | 143980/225000 [285:14:57<134:44:22,  5.99s/it]
 64%|██████▍   | 143981/225000 [285:15:04<136:53:34,  6.08s/it]
 64%|██████▍   | 143982/225000 [285:15:07<119:15:26,  5.30s/it]
 64%|██████▍   | 143983/225000 [285:15:11<110:17:37,  4.90s/it]
 64%|██████▍   | 143984/225000 [285:15:20<137:17:19,  6.10s/it]
 64%|██████▍   | 143985/225000 [285:15:26<138:57:35,  6.17s/it]
 64%|██████▍   | 143986/225000 [285:15:31<125:10:18,  5.56s/it]
 64%|██████▍   | 143987/225000 [285:15:34<114:00:36,  5.07s/it]
 64%|██████▍   | 143988/225000 [285:15:39<107:00:43,  4.76s/it]
 64%|██████▍   | 143989/225000 [285:15:49<147:10:21,  6.54s/it]
 64%|██████▍   | 143990/225000 [285:15:53<128:24:52,  5.71s/it]
 64%|██████▍   | 143991/225000 [285:15:59<130:49:45,  5.81s/it]
 64%|██████▍   | 143992/225000 [285:16:07<143:37:06,  6.38s/it]
 64%|██████▍   | 143993/225000 [285:16:11<128:15:01,  5.70s/it]
 64%|██████▍   | 143994/225000 [285:16:15<118:40:07,  5.27s/it]
 64%|██████▍   | 143995/225000 [285:16:19<111:10:32,  4.94s/it]
 64%|██████▍   | 143996/225000 [285:16:24<108:21:34,  4.82s/it]
 64%|██████▍   | 143997/225000 [285:16:32<133:15:09,  5.92s/it]
 64%|██████▍   | 143998/225000 [285:16:40<147:30:27,  6.56s/it]
 64%|██████▍   | 143999/225000 [285:16:44<126:47:50,  5.64s/it]
 64%|██████▍   | 144000/225000 [285:16:52<141:50:08,  6.30s/it]
                                                               
{'loss': '0.7952', 'grad_norm': '1.157', 'learning_rate': '1.558e-05', 'epoch': '0.64', 'num_input_tokens_seen': 472882968, 'train_runtime': '1.027e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144000/225000 [285:16:52<141:50:08,  6.30s/it]
 64%|██████▍   | 144001/225000 [285:16:55<122:51:51,  5.46s/it]
 64%|██████▍   | 144002/225000 [285:17:04<142:16:43,  6.32s/it]
 64%|██████▍   | 144003/225000 [285:17:07<124:23:49,  5.53s/it]
 64%|██████▍   | 144004/225000 [285:17:11<110:05:35,  4.89s/it]
 64%|██████▍   | 144005/225000 [285:17:18<124:57:23,  5.55s/it]
 64%|██████▍   | 144006/225000 [285:17:25<137:02:10,  6.09s/it]
 64%|██████▍   | 144007/225000 [285:17:30<128:16:38,  5.70s/it]
 64%|██████▍   | 144008/225000 [285:17:35<122:40:54,  5.45s/it]
 64%|██████▍   | 144009/225000 [285:17:40<119:07:01,  5.29s/it]
 64%|██████▍   | 144010/225000 [285:17:44<115:03:56,  5.11s/it]
 64%|██████▍   | 144011/225000 [285:17:49<114:29:24,  5.09s/it]
 64%|██████▍   | 144012/225000 [285:17:54<111:37:22,  4.96s/it]
 64%|██████▍   | 144013/225000 [285:17:59<111:20:48,  4.95s/it]
 64%|██████▍   | 144014/225000 [285:18:04<110:30:02,  4.91s/it]
 64%|██████▍   | 144015/225000 [285:18:09<110:53:12,  4.93s/it]
 64%|██████▍   | 144016/225000 [285:18:14<112:50:46,  5.02s/it]
 64%|██████▍   | 144017/225000 [285:18:20<117:04:29,  5.20s/it]
 64%|██████▍   | 144018/225000 [285:18:25<117:56:30,  5.24s/it]
 64%|██████▍   | 144019/225000 [285:18:30<118:03:46,  5.25s/it]
 64%|██████▍   | 144020/225000 [285:18:35<118:07:39,  5.25s/it]
 64%|██████▍   | 144021/225000 [285:18:48<166:02:05,  7.38s/it]
 64%|██████▍   | 144022/225000 [285:18:56<173:39:03,  7.72s/it]
 64%|██████▍   | 144023/225000 [285:19:01<154:10:37,  6.85s/it]
 64%|██████▍   | 144024/225000 [285:19:06<142:25:10,  6.33s/it]
 64%|██████▍   | 144025/225000 [285:19:11<130:05:53,  5.78s/it]
 64%|██████▍   | 144026/225000 [285:19:16<124:18:12,  5.53s/it]
 64%|██████▍   | 144027/225000 [285:19:21<122:23:46,  5.44s/it]
 64%|██████▍   | 144028/225000 [285:19:26<117:06:31,  5.21s/it]
 64%|██████▍   | 144029/225000 [285:19:30<112:55:55,  5.02s/it]
 64%|██████▍   | 144030/225000 [285:19:35<109:18:39,  4.86s/it]
 64%|██████▍   | 144031/225000 [285:19:47<161:37:18,  7.19s/it]
 64%|██████▍   | 144032/225000 [285:19:53<150:18:37,  6.68s/it]
 64%|██████▍   | 144033/225000 [285:19:59<146:30:29,  6.51s/it]
 64%|██████▍   | 144034/225000 [285:20:03<128:28:11,  5.71s/it]
 64%|██████▍   | 144035/225000 [285:20:06<114:23:40,  5.09s/it]
 64%|██████▍   | 144036/225000 [285:20:17<154:06:50,  6.85s/it]
 64%|██████▍   | 144037/225000 [285:20:21<130:14:30,  5.79s/it]
 64%|██████▍   | 144038/225000 [285:20:30<151:26:03,  6.73s/it]
 64%|██████▍   | 144039/225000 [285:20:33<130:01:55,  5.78s/it]
 64%|██████▍   | 144040/225000 [285:20:42<148:39:49,  6.61s/it]
 64%|██████▍   | 144041/225000 [285:20:45<128:47:53,  5.73s/it]
 64%|██████▍   | 144042/225000 [285:20:49<113:15:54,  5.04s/it]
 64%|██████▍   | 144043/225000 [285:20:53<105:11:33,  4.68s/it]
 64%|██████▍   | 144044/225000 [285:20:57<103:22:41,  4.60s/it]
 64%|██████▍   | 144045/225000 [285:21:09<151:10:21,  6.72s/it]
 64%|██████▍   | 144046/225000 [285:21:12<128:28:48,  5.71s/it]
 64%|██████▍   | 144047/225000 [285:21:16<117:03:20,  5.21s/it]
 64%|██████▍   | 144048/225000 [285:21:20<110:39:09,  4.92s/it]
 64%|██████▍   | 144049/225000 [285:21:31<147:06:32,  6.54s/it]
 64%|██████▍   | 144050/225000 [285:21:37<146:11:13,  6.50s/it]
                                                               
{'loss': '0.7871', 'grad_norm': '1.117', 'learning_rate': '1.557e-05', 'epoch': '0.6402', 'num_input_tokens_seen': 473011392, 'train_runtime': '1.027e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144050/225000 [285:21:37<146:11:13,  6.50s/it]
 64%|██████▍   | 144051/225000 [285:21:41<129:24:44,  5.76s/it]
 64%|██████▍   | 144052/225000 [285:21:46<121:55:29,  5.42s/it]
 64%|██████▍   | 144053/225000 [285:21:50<116:04:07,  5.16s/it]
 64%|██████▍   | 144054/225000 [285:21:55<114:49:26,  5.11s/it]
 64%|██████▍   | 144055/225000 [285:22:00<112:06:39,  4.99s/it]
 64%|██████▍   | 144056/225000 [285:22:05<110:03:45,  4.90s/it]
 64%|██████▍   | 144057/225000 [285:22:09<107:15:24,  4.77s/it]
 64%|██████▍   | 144058/225000 [285:22:21<151:47:34,  6.75s/it]
 64%|██████▍   | 144059/225000 [285:22:27<150:03:27,  6.67s/it]
 64%|██████▍   | 144060/225000 [285:22:32<140:31:31,  6.25s/it]
 64%|██████▍   | 144061/225000 [285:22:37<133:05:26,  5.92s/it]
 64%|██████▍   | 144062/225000 [285:22:42<125:06:21,  5.56s/it]
 64%|██████▍   | 144063/225000 [285:22:48<123:47:53,  5.51s/it]
 64%|██████▍   | 144064/225000 [285:22:53<120:43:57,  5.37s/it]
 64%|██████▍   | 144065/225000 [285:22:58<117:47:37,  5.24s/it]
 64%|██████▍   | 144066/225000 [285:23:02<114:30:30,  5.09s/it]
 64%|██████▍   | 144067/225000 [285:23:07<114:17:26,  5.08s/it]
 64%|██████▍   | 144068/225000 [285:23:13<115:29:28,  5.14s/it]
 64%|██████▍   | 144069/225000 [285:23:18<114:59:52,  5.12s/it]
 64%|██████▍   | 144070/225000 [285:23:31<167:00:02,  7.43s/it]
 64%|██████▍   | 144071/225000 [285:23:36<152:01:27,  6.76s/it]
 64%|██████▍   | 144072/225000 [285:23:42<147:01:36,  6.54s/it]
 64%|██████▍   | 144073/225000 [285:23:46<131:04:44,  5.83s/it]
 64%|██████▍   | 144074/225000 [285:23:50<120:29:19,  5.36s/it]
 64%|██████▍   | 144075/225000 [285:24:01<157:08:38,  6.99s/it]
 64%|██████▍   | 144076/225000 [285:24:04<132:05:19,  5.88s/it]
 64%|██████▍   | 144077/225000 [285:24:08<117:27:04,  5.23s/it]
 64%|██████▍   | 144078/225000 [285:24:12<112:30:42,  5.01s/it]
 64%|██████▍   | 144079/225000 [285:24:17<108:55:36,  4.85s/it]
 64%|██████▍   | 144080/225000 [285:24:21<104:24:50,  4.65s/it]
 64%|██████▍   | 144081/225000 [285:24:32<149:22:35,  6.65s/it]
 64%|██████▍   | 144082/225000 [285:24:39<147:28:02,  6.56s/it]
 64%|██████▍   | 144083/225000 [285:24:43<131:19:13,  5.84s/it]
 64%|██████▍   | 144084/225000 [285:24:47<118:13:50,  5.26s/it]
 64%|██████▍   | 144085/225000 [285:24:51<110:13:24,  4.90s/it]
 64%|██████▍   | 144086/225000 [285:24:55<107:25:04,  4.78s/it]
 64%|██████▍   | 144087/225000 [285:25:02<116:50:54,  5.20s/it]
 64%|██████▍   | 144088/225000 [285:25:13<158:36:42,  7.06s/it]
 64%|██████▍   | 144089/225000 [285:25:18<144:47:59,  6.44s/it]
 64%|██████▍   | 144090/225000 [285:25:23<137:03:39,  6.10s/it]
 64%|██████▍   | 144091/225000 [285:25:28<130:35:11,  5.81s/it]
 64%|██████▍   | 144092/225000 [285:25:34<125:46:37,  5.60s/it]
 64%|██████▍   | 144093/225000 [285:25:39<122:33:55,  5.45s/it]
 64%|██████▍   | 144094/225000 [285:25:44<122:21:32,  5.44s/it]
 64%|██████▍   | 144095/225000 [285:25:49<119:07:18,  5.30s/it]
 64%|██████▍   | 144096/225000 [285:25:54<118:30:01,  5.27s/it]
 64%|██████▍   | 144097/225000 [285:25:59<118:17:55,  5.26s/it]
 64%|██████▍   | 144098/225000 [285:26:04<113:53:31,  5.07s/it]
 64%|██████▍   | 144099/225000 [285:26:09<114:38:42,  5.10s/it]
 64%|██████▍   | 144100/225000 [285:26:22<167:09:09,  7.44s/it]
                                                               
{'loss': '0.7982', 'grad_norm': '1.08', 'learning_rate': '1.555e-05', 'epoch': '0.6404', 'num_input_tokens_seen': 473141384, 'train_runtime': '1.028e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144100/225000 [285:26:22<167:09:09,  7.44s/it]
 64%|██████▍   | 144101/225000 [285:26:28<154:33:50,  6.88s/it]
 64%|██████▍   | 144102/225000 [285:26:34<150:10:31,  6.68s/it]
 64%|██████▍   | 144103/225000 [285:26:38<133:54:23,  5.96s/it]
 64%|██████▍   | 144104/225000 [285:26:43<126:27:02,  5.63s/it]
 64%|██████▍   | 144105/225000 [285:26:48<119:11:06,  5.30s/it]
 64%|██████▍   | 144106/225000 [285:26:53<117:30:44,  5.23s/it]
 64%|██████▍   | 144107/225000 [285:26:57<114:01:57,  5.07s/it]
 64%|██████▍   | 144108/225000 [285:27:05<129:15:35,  5.75s/it]
 64%|██████▍   | 144109/225000 [285:27:14<153:18:44,  6.82s/it]
 64%|██████▍   | 144110/225000 [285:27:18<132:44:33,  5.91s/it]
 64%|██████▍   | 144111/225000 [285:27:27<152:37:24,  6.79s/it]
 64%|██████▍   | 144112/225000 [285:27:30<126:29:00,  5.63s/it]
 64%|██████▍   | 144113/225000 [285:27:34<116:46:06,  5.20s/it]
 64%|██████▍   | 144114/225000 [285:27:38<111:06:16,  4.94s/it]
 64%|██████▍   | 144115/225000 [285:27:49<150:50:33,  6.71s/it]
 64%|██████▍   | 144116/225000 [285:27:53<131:11:05,  5.84s/it]
 64%|██████▍   | 144117/225000 [285:27:59<133:46:33,  5.95s/it]
 64%|██████▍   | 144118/225000 [285:28:05<135:15:51,  6.02s/it]
 64%|██████▍   | 144119/225000 [285:28:12<137:50:27,  6.14s/it]
 64%|██████▍   | 144120/225000 [285:28:16<123:03:31,  5.48s/it]
 64%|██████▍   | 144121/225000 [285:28:20<117:33:35,  5.23s/it]
 64%|██████▍   | 144122/225000 [285:28:24<110:33:38,  4.92s/it]
 64%|██████▍   | 144123/225000 [285:28:30<113:24:15,  5.05s/it]
 64%|██████▍   | 144124/225000 [285:28:40<146:57:30,  6.54s/it]
 64%|██████▍   | 144125/225000 [285:28:43<128:01:33,  5.70s/it]
 64%|██████▍   | 144126/225000 [285:28:50<132:25:36,  5.89s/it]
 64%|██████▍   | 144127/225000 [285:28:58<146:02:01,  6.50s/it]
 64%|██████▍   | 144128/225000 [285:29:02<133:19:07,  5.93s/it]
 64%|██████▍   | 144129/225000 [285:29:07<125:07:15,  5.57s/it]
 64%|██████▍   | 144130/225000 [285:29:12<118:33:50,  5.28s/it]
 64%|██████▍   | 144131/225000 [285:29:17<116:03:52,  5.17s/it]
 64%|██████▍   | 144132/225000 [285:29:21<112:14:55,  5.00s/it]
 64%|██████▍   | 144133/225000 [285:29:34<162:40:26,  7.24s/it]
 64%|██████▍   | 144134/225000 [285:29:38<140:34:42,  6.26s/it]
 64%|██████▍   | 144135/225000 [285:29:44<141:50:12,  6.31s/it]
 64%|██████▍   | 144136/225000 [285:29:48<124:04:36,  5.52s/it]
 64%|██████▍   | 144137/225000 [285:29:52<115:15:17,  5.13s/it]
 64%|██████▍   | 144138/225000 [285:30:01<139:43:30,  6.22s/it]
 64%|██████▍   | 144139/225000 [285:30:05<128:37:36,  5.73s/it]
 64%|██████▍   | 144140/225000 [285:30:13<143:02:27,  6.37s/it]
 64%|██████▍   | 144141/225000 [285:30:18<132:42:30,  5.91s/it]
 64%|██████▍   | 144142/225000 [285:30:23<129:01:39,  5.74s/it]
 64%|██████▍   | 144143/225000 [285:30:28<124:18:57,  5.53s/it]
 64%|██████▍   | 144144/225000 [285:30:37<144:00:59,  6.41s/it]
 64%|██████▍   | 144145/225000 [285:30:41<125:17:50,  5.58s/it]
 64%|██████▍   | 144146/225000 [285:30:44<110:26:47,  4.92s/it]
 64%|██████▍   | 144147/225000 [285:30:52<132:52:37,  5.92s/it]
 64%|██████▍   | 144148/225000 [285:30:59<142:03:55,  6.33s/it]
 64%|██████▍   | 144149/225000 [285:31:03<123:59:45,  5.52s/it]
 64%|██████▍   | 144150/225000 [285:31:11<139:56:13,  6.23s/it]
                                                               
{'loss': '0.7911', 'grad_norm': '1.064', 'learning_rate': '1.553e-05', 'epoch': '0.6407', 'num_input_tokens_seen': 473271776, 'train_runtime': '1.028e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144150/225000 [285:31:11<139:56:13,  6.23s/it]
 64%|██████▍   | 144151/225000 [285:31:16<129:06:19,  5.75s/it]
 64%|██████▍   | 144152/225000 [285:31:20<122:37:04,  5.46s/it]
 64%|██████▍   | 144153/225000 [285:31:25<119:29:04,  5.32s/it]
 64%|██████▍   | 144154/225000 [285:31:30<113:39:02,  5.06s/it]
 64%|██████▍   | 144155/225000 [285:31:34<109:27:04,  4.87s/it]
 64%|██████▍   | 144156/225000 [285:31:39<108:23:02,  4.83s/it]
 64%|██████▍   | 144157/225000 [285:31:44<107:25:55,  4.78s/it]
 64%|██████▍   | 144158/225000 [285:31:49<111:55:13,  4.98s/it]
 64%|██████▍   | 144159/225000 [285:32:00<154:23:01,  6.87s/it]
 64%|██████▍   | 144160/225000 [285:32:05<137:31:26,  6.12s/it]
 64%|██████▍   | 144161/225000 [285:32:09<125:26:10,  5.59s/it]
 64%|██████▍   | 144162/225000 [285:32:13<115:54:20,  5.16s/it]
 64%|██████▍   | 144163/225000 [285:32:18<110:34:03,  4.92s/it]
 64%|██████▍   | 144164/225000 [285:32:26<133:07:21,  5.93s/it]
 64%|██████▍   | 144165/225000 [285:32:35<156:38:00,  6.98s/it]
 64%|██████▍   | 144166/225000 [285:32:40<137:52:58,  6.14s/it]
 64%|██████▍   | 144167/225000 [285:32:43<122:44:20,  5.47s/it]
 64%|██████▍   | 144168/225000 [285:32:48<114:09:48,  5.08s/it]
 64%|██████▍   | 144169/225000 [285:32:58<151:12:55,  6.73s/it]
 64%|██████▍   | 144170/225000 [285:33:02<132:20:57,  5.89s/it]
 64%|██████▍   | 144171/225000 [285:33:07<125:25:39,  5.59s/it]
 64%|██████▍   | 144172/225000 [285:33:12<120:04:38,  5.35s/it]
 64%|██████▍   | 144173/225000 [285:33:16<113:47:22,  5.07s/it]
 64%|██████▍   | 144174/225000 [285:33:21<114:39:48,  5.11s/it]
 64%|██████▍   | 144175/225000 [285:33:26<113:01:50,  5.03s/it]
 64%|██████▍   | 144176/225000 [285:33:31<112:45:35,  5.02s/it]
 64%|██████▍   | 144177/225000 [285:33:36<112:18:45,  5.00s/it]
 64%|██████▍   | 144178/225000 [285:33:47<151:42:00,  6.76s/it]
 64%|██████▍   | 144179/225000 [285:33:52<140:30:31,  6.26s/it]
 64%|██████▍   | 144180/225000 [285:33:58<136:04:42,  6.06s/it]
 64%|██████▍   | 144181/225000 [285:34:05<145:10:37,  6.47s/it]
 64%|██████▍   | 144182/225000 [285:34:13<151:50:28,  6.76s/it]
 64%|██████▍   | 144183/225000 [285:34:16<129:26:21,  5.77s/it]
 64%|██████▍   | 144184/225000 [285:34:20<115:28:58,  5.14s/it]
 64%|██████▍   | 144185/225000 [285:34:30<147:04:26,  6.55s/it]
 64%|██████▍   | 144186/225000 [285:34:33<126:25:23,  5.63s/it]
 64%|██████▍   | 144187/225000 [285:34:40<138:00:52,  6.15s/it]
 64%|██████▍   | 144188/225000 [285:34:44<122:17:52,  5.45s/it]
 64%|██████▍   | 144189/225000 [285:34:48<109:49:01,  4.89s/it]
 64%|██████▍   | 144190/225000 [285:34:58<143:51:28,  6.41s/it]
 64%|██████▍   | 144191/225000 [285:35:01<124:56:49,  5.57s/it]
 64%|██████▍   | 144192/225000 [285:35:09<137:02:52,  6.11s/it]
 64%|██████▍   | 144193/225000 [285:35:12<119:25:41,  5.32s/it]
 64%|██████▍   | 144194/225000 [285:35:17<113:54:10,  5.07s/it]
 64%|██████▍   | 144195/225000 [285:35:21<110:42:03,  4.93s/it]
 64%|██████▍   | 144196/225000 [285:35:26<111:22:46,  4.96s/it]
 64%|██████▍   | 144197/225000 [285:35:32<113:51:26,  5.07s/it]
 64%|██████▍   | 144198/225000 [285:35:37<114:38:26,  5.11s/it]
 64%|██████▍   | 144199/225000 [285:35:42<112:52:00,  5.03s/it]
 64%|██████▍   | 144200/225000 [285:35:47<111:11:35,  4.95s/it]
                                                               
{'loss': '0.7997', 'grad_norm': '1.017', 'learning_rate': '1.551e-05', 'epoch': '0.6409', 'num_input_tokens_seen': 473401904, 'train_runtime': '1.028e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144200/225000 [285:35:47<111:11:35,  4.95s/it]
 64%|██████▍   | 144201/225000 [285:35:53<124:05:14,  5.53s/it]
 64%|██████▍   | 144202/225000 [285:36:05<163:25:05,  7.28s/it]
 64%|██████▍   | 144203/225000 [285:36:09<145:20:31,  6.48s/it]
 64%|██████▍   | 144204/225000 [285:36:14<133:59:15,  5.97s/it]
 64%|██████▍   | 144205/225000 [285:36:19<124:53:02,  5.56s/it]
 64%|██████▍   | 144206/225000 [285:36:24<120:23:15,  5.36s/it]
 64%|██████▍   | 144207/225000 [285:36:29<117:15:12,  5.22s/it]
 64%|██████▍   | 144208/225000 [285:36:33<114:46:58,  5.11s/it]
 64%|██████▍   | 144209/225000 [285:36:38<113:36:40,  5.06s/it]
 64%|██████▍   | 144210/225000 [285:36:43<113:06:27,  5.04s/it]
 64%|██████▍   | 144211/225000 [285:36:48<110:24:39,  4.92s/it]
 64%|██████▍   | 144212/225000 [285:36:55<123:16:08,  5.49s/it]
 64%|██████▍   | 144213/225000 [285:37:08<174:10:55,  7.76s/it]
 64%|██████▍   | 144214/225000 [285:37:11<144:22:43,  6.43s/it]
 64%|██████▍   | 144215/225000 [285:37:17<139:12:02,  6.20s/it]
 64%|██████▍   | 144216/225000 [285:37:24<142:51:42,  6.37s/it]
 64%|██████▍   | 144217/225000 [285:37:28<131:18:28,  5.85s/it]
 64%|██████▍   | 144218/225000 [285:37:33<124:49:38,  5.56s/it]
 64%|██████▍   | 144219/225000 [285:37:38<119:38:13,  5.33s/it]
 64%|██████▍   | 144220/225000 [285:37:42<112:42:49,  5.02s/it]
 64%|██████▍   | 144221/225000 [285:37:47<110:41:04,  4.93s/it]
 64%|██████▍   | 144222/225000 [285:37:52<111:50:24,  4.98s/it]
 64%|██████▍   | 144223/225000 [285:38:04<159:20:53,  7.10s/it]
 64%|██████▍   | 144224/225000 [285:38:08<134:31:46,  6.00s/it]
 64%|██████▍   | 144225/225000 [285:38:12<124:43:31,  5.56s/it]
 64%|██████▍   | 144226/225000 [285:38:17<122:07:28,  5.44s/it]
 64%|██████▍   | 144227/225000 [285:38:22<118:14:06,  5.27s/it]
 64%|██████▍   | 144228/225000 [285:38:27<113:39:56,  5.07s/it]
 64%|██████▍   | 144229/225000 [285:38:32<114:32:32,  5.11s/it]
 64%|██████▍   | 144230/225000 [285:38:37<115:51:42,  5.16s/it]
 64%|██████▍   | 144231/225000 [285:38:43<116:53:10,  5.21s/it]
 64%|██████▍   | 144232/225000 [285:38:47<114:27:15,  5.10s/it]
 64%|██████▍   | 144233/225000 [285:38:52<112:11:09,  5.00s/it]
 64%|██████▍   | 144234/225000 [285:39:04<159:16:59,  7.10s/it]
 64%|██████▍   | 144235/225000 [285:39:11<157:28:15,  7.02s/it]
 64%|██████▍   | 144236/225000 [285:39:18<154:23:20,  6.88s/it]
 64%|██████▍   | 144237/225000 [285:39:21<131:52:35,  5.88s/it]
 64%|██████▍   | 144238/225000 [285:39:26<126:44:29,  5.65s/it]
 64%|██████▍   | 144239/225000 [285:39:31<124:26:13,  5.55s/it]
 64%|██████▍   | 144240/225000 [285:39:36<120:46:29,  5.38s/it]
 64%|██████▍   | 144241/225000 [285:39:42<118:35:20,  5.29s/it]
 64%|██████▍   | 144242/225000 [285:39:47<117:09:57,  5.22s/it]
 64%|██████▍   | 144243/225000 [285:39:52<115:59:26,  5.17s/it]
 64%|██████▍   | 144244/225000 [285:39:57<115:23:38,  5.14s/it]
 64%|██████▍   | 144245/225000 [285:40:02<116:18:38,  5.19s/it]
 64%|██████▍   | 144246/225000 [285:40:07<116:04:49,  5.17s/it]
 64%|██████▍   | 144247/225000 [285:40:12<114:58:14,  5.13s/it]
 64%|██████▍   | 144248/225000 [285:40:17<115:29:44,  5.15s/it]
 64%|██████▍   | 144249/225000 [285:40:30<164:31:42,  7.33s/it]
 64%|██████▍   | 144250/225000 [285:40:36<155:09:03,  6.92s/it]
                                                               
{'loss': '0.7932', 'grad_norm': '1.105', 'learning_rate': '1.55e-05', 'epoch': '0.6411', 'num_input_tokens_seen': 473532280, 'train_runtime': '1.028e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144250/225000 [285:40:36<155:09:03,  6.92s/it]
 64%|██████▍   | 144251/225000 [285:40:40<134:07:37,  5.98s/it]
 64%|██████▍   | 144252/225000 [285:40:44<125:48:10,  5.61s/it]
 64%|██████▍   | 144253/225000 [285:40:49<122:46:39,  5.47s/it]
 64%|██████▍   | 144254/225000 [285:40:54<115:22:42,  5.14s/it]
 64%|██████▍   | 144255/225000 [285:40:59<115:10:42,  5.14s/it]
 64%|██████▍   | 144256/225000 [285:41:04<112:52:40,  5.03s/it]
 64%|██████▍   | 144257/225000 [285:41:09<113:32:56,  5.06s/it]
 64%|██████▍   | 144258/225000 [285:41:14<112:58:17,  5.04s/it]
 64%|██████▍   | 144259/225000 [285:41:24<146:08:32,  6.52s/it]
 64%|██████▍   | 144260/225000 [285:41:31<153:54:11,  6.86s/it]
 64%|██████▍   | 144261/225000 [285:41:36<138:49:45,  6.19s/it]
 64%|██████▍   | 144262/225000 [285:41:40<124:42:53,  5.56s/it]
 64%|██████▍   | 144263/225000 [285:41:45<116:26:02,  5.19s/it]
 64%|██████▍   | 144264/225000 [285:41:48<107:41:15,  4.80s/it]
 64%|██████▍   | 144265/225000 [285:41:53<104:55:40,  4.68s/it]
 64%|██████▍   | 144266/225000 [285:42:04<150:50:11,  6.73s/it]
 64%|██████▍   | 144267/225000 [285:42:11<151:21:44,  6.75s/it]
 64%|██████▍   | 144268/225000 [285:42:16<139:32:04,  6.22s/it]
 64%|██████▍   | 144269/225000 [285:42:21<128:55:41,  5.75s/it]
 64%|██████▍   | 144270/225000 [285:42:26<123:32:31,  5.51s/it]
 64%|██████▍   | 144271/225000 [285:42:31<119:49:50,  5.34s/it]
 64%|██████▍   | 144272/225000 [285:42:36<117:43:08,  5.25s/it]
 64%|██████▍   | 144273/225000 [285:42:45<142:19:28,  6.35s/it]
 64%|██████▍   | 144274/225000 [285:42:51<143:50:24,  6.41s/it]
 64%|██████▍   | 144275/225000 [285:42:55<126:18:20,  5.63s/it]
 64%|██████▍   | 144276/225000 [285:42:59<117:27:49,  5.24s/it]
 64%|██████▍   | 144277/225000 [285:43:08<138:27:57,  6.18s/it]
 64%|██████▍   | 144278/225000 [285:43:15<143:15:14,  6.39s/it]
 64%|██████▍   | 144279/225000 [285:43:20<134:53:07,  6.02s/it]
 64%|██████▍   | 144280/225000 [285:43:24<122:41:15,  5.47s/it]
 64%|██████▍   | 144281/225000 [285:43:28<114:25:49,  5.10s/it]
 64%|██████▍   | 144282/225000 [285:43:33<109:29:41,  4.88s/it]
 64%|██████▍   | 144283/225000 [285:43:37<103:49:10,  4.63s/it]
 64%|██████▍   | 144284/225000 [285:43:42<109:43:55,  4.89s/it]
 64%|██████▍   | 144285/225000 [285:43:55<160:42:17,  7.17s/it]
 64%|██████▍   | 144286/225000 [285:43:59<140:19:22,  6.26s/it]
 64%|██████▍   | 144287/225000 [285:44:04<133:31:25,  5.96s/it]
 64%|██████▍   | 144288/225000 [285:44:09<126:51:23,  5.66s/it]
 64%|██████▍   | 144289/225000 [285:44:14<121:23:51,  5.41s/it]
 64%|██████▍   | 144290/225000 [285:44:19<120:11:24,  5.36s/it]
 64%|██████▍   | 144291/225000 [285:44:24<116:10:43,  5.18s/it]
 64%|██████▍   | 144292/225000 [285:44:29<114:11:20,  5.09s/it]
 64%|██████▍   | 144293/225000 [285:44:34<113:04:49,  5.04s/it]
 64%|██████▍   | 144294/225000 [285:44:44<148:27:00,  6.62s/it]
 64%|██████▍   | 144295/225000 [285:44:50<147:57:35,  6.60s/it]
 64%|██████▍   | 144296/225000 [285:44:56<144:28:24,  6.44s/it]
 64%|██████▍   | 144297/225000 [285:45:03<143:24:35,  6.40s/it]
 64%|██████▍   | 144298/225000 [285:45:07<129:48:09,  5.79s/it]
 64%|██████▍   | 144299/225000 [285:45:15<141:11:57,  6.30s/it]
 64%|██████▍   | 144300/225000 [285:45:18<124:48:59,  5.57s/it]
                                                               
{'loss': '0.7877', 'grad_norm': '1.116', 'learning_rate': '1.548e-05', 'epoch': '0.6413', 'num_input_tokens_seen': 473662824, 'train_runtime': '1.029e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144300/225000 [285:45:18<124:48:59,  5.57s/it]
 64%|██████▍   | 144301/225000 [285:45:22<111:40:29,  4.98s/it]
 64%|██████▍   | 144302/225000 [285:45:27<110:00:42,  4.91s/it]
 64%|██████▍   | 144303/225000 [285:45:32<110:18:36,  4.92s/it]
 64%|██████▍   | 144304/225000 [285:45:37<111:59:02,  5.00s/it]
 64%|██████▍   | 144305/225000 [285:45:42<115:13:49,  5.14s/it]
 64%|██████▍   | 144306/225000 [285:45:48<117:35:03,  5.25s/it]
 64%|██████▍   | 144307/225000 [285:45:53<118:51:26,  5.30s/it]
 64%|██████▍   | 144308/225000 [285:45:58<116:31:44,  5.20s/it]
 64%|██████▍   | 144309/225000 [285:46:03<116:01:35,  5.18s/it]
 64%|██████▍   | 144310/225000 [285:46:09<116:08:50,  5.18s/it]
 64%|██████▍   | 144311/225000 [285:46:13<113:14:20,  5.05s/it]
 64%|██████▍   | 144312/225000 [285:46:18<113:32:52,  5.07s/it]
 64%|██████▍   | 144313/225000 [285:46:24<116:54:58,  5.22s/it]
 64%|██████▍   | 144314/225000 [285:46:29<118:18:19,  5.28s/it]
 64%|██████▍   | 144315/225000 [285:46:38<141:43:58,  6.32s/it]
 64%|██████▍   | 144316/225000 [285:46:49<174:02:38,  7.77s/it]
 64%|██████▍   | 144317/225000 [285:46:53<143:38:36,  6.41s/it]
 64%|██████▍   | 144318/225000 [285:46:59<141:53:33,  6.33s/it]
 64%|██████▍   | 144319/225000 [285:47:02<124:03:52,  5.54s/it]
 64%|██████▍   | 144320/225000 [285:47:11<141:46:58,  6.33s/it]
 64%|██████▍   | 144321/225000 [285:47:15<126:05:39,  5.63s/it]
 64%|██████▍   | 144322/225000 [285:47:19<115:52:45,  5.17s/it]
 64%|██████▍   | 144323/225000 [285:47:27<138:21:53,  6.17s/it]
 64%|██████▍   | 144324/225000 [285:47:33<135:52:59,  6.06s/it]
 64%|██████▍   | 144325/225000 [285:47:41<149:29:17,  6.67s/it]
 64%|██████▍   | 144326/225000 [285:47:45<128:58:19,  5.76s/it]
 64%|██████▍   | 144327/225000 [285:47:48<114:40:12,  5.12s/it]
 64%|██████▍   | 144328/225000 [285:47:53<111:34:42,  4.98s/it]
 64%|██████▍   | 144329/225000 [285:47:58<108:45:45,  4.85s/it]
 64%|██████▍   | 144330/225000 [285:48:02<106:56:47,  4.77s/it]
 64%|██████▍   | 144331/225000 [285:48:09<117:31:05,  5.24s/it]
 64%|██████▍   | 144332/225000 [285:48:20<161:48:24,  7.22s/it]
 64%|██████▍   | 144333/225000 [285:48:24<136:55:25,  6.11s/it]
 64%|██████▍   | 144334/225000 [285:48:28<126:40:04,  5.65s/it]
 64%|██████▍   | 144335/225000 [285:48:36<139:21:59,  6.22s/it]
 64%|██████▍   | 144336/225000 [285:48:41<128:49:21,  5.75s/it]
 64%|██████▍   | 144337/225000 [285:48:47<133:01:45,  5.94s/it]
 64%|██████▍   | 144338/225000 [285:48:51<121:07:43,  5.41s/it]
 64%|██████▍   | 144339/225000 [285:49:00<141:00:37,  6.29s/it]
 64%|██████▍   | 144340/225000 [285:49:04<128:59:10,  5.76s/it]
 64%|██████▍   | 144341/225000 [285:49:08<118:53:06,  5.31s/it]
 64%|██████▍   | 144342/225000 [285:49:13<111:43:25,  4.99s/it]
 64%|██████▍   | 144343/225000 [285:49:17<106:23:37,  4.75s/it]
 64%|██████▍   | 144344/225000 [285:49:28<152:09:39,  6.79s/it]
 64%|██████▍   | 144345/225000 [285:49:33<135:15:24,  6.04s/it]
 64%|██████▍   | 144346/225000 [285:49:36<120:50:29,  5.39s/it]
 64%|██████▍   | 144347/225000 [285:49:42<118:32:33,  5.29s/it]
 64%|██████▍   | 144348/225000 [285:49:47<118:42:10,  5.30s/it]
 64%|██████▍   | 144349/225000 [285:49:52<117:51:54,  5.26s/it]
 64%|██████▍   | 144350/225000 [285:49:57<117:37:31,  5.25s/it]
                                                               
{'loss': '0.7968', 'grad_norm': '1.14', 'learning_rate': '1.546e-05', 'epoch': '0.6416', 'num_input_tokens_seen': 473793160, 'train_runtime': '1.029e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144350/225000 [285:49:57<117:37:31,  5.25s/it]
 64%|██████▍   | 144351/225000 [285:50:02<114:12:34,  5.10s/it]
 64%|██████▍   | 144352/225000 [285:50:07<114:10:49,  5.10s/it]
 64%|██████▍   | 144353/225000 [285:50:12<115:12:49,  5.14s/it]
 64%|██████▍   | 144354/225000 [285:50:17<111:39:53,  4.98s/it]
 64%|██████▍   | 144355/225000 [285:50:22<112:49:40,  5.04s/it]
 64%|██████▍   | 144356/225000 [285:50:34<160:23:48,  7.16s/it]
 64%|██████▍   | 144357/225000 [285:50:40<148:52:26,  6.65s/it]
 64%|██████▍   | 144358/225000 [285:50:44<132:39:55,  5.92s/it]
 64%|██████▍   | 144359/225000 [285:50:49<125:25:06,  5.60s/it]
 64%|██████▍   | 144360/225000 [285:50:54<121:31:36,  5.43s/it]
 64%|██████▍   | 144361/225000 [285:50:58<115:55:25,  5.18s/it]
 64%|██████▍   | 144362/225000 [285:51:03<114:28:38,  5.11s/it]
 64%|██████▍   | 144363/225000 [285:51:09<115:28:25,  5.16s/it]
 64%|██████▍   | 144364/225000 [285:51:14<115:04:20,  5.14s/it]
 64%|██████▍   | 144365/225000 [285:51:19<115:32:52,  5.16s/it]
 64%|██████▍   | 144366/225000 [285:51:32<166:21:55,  7.43s/it]
 64%|██████▍   | 144367/225000 [285:51:35<139:38:40,  6.23s/it]
 64%|██████▍   | 144368/225000 [285:51:43<148:42:19,  6.64s/it]
 64%|██████▍   | 144369/225000 [285:51:50<153:19:04,  6.85s/it]
 64%|██████▍   | 144370/225000 [285:51:55<144:00:21,  6.43s/it]
 64%|██████▍   | 144371/225000 [285:52:00<132:22:00,  5.91s/it]
 64%|██████▍   | 144372/225000 [285:52:04<119:49:41,  5.35s/it]
 64%|██████▍   | 144373/225000 [285:52:13<146:11:14,  6.53s/it]
 64%|██████▍   | 144374/225000 [285:52:17<125:15:30,  5.59s/it]
 64%|██████▍   | 144375/225000 [285:52:26<151:52:07,  6.78s/it]
 64%|██████▍   | 144376/225000 [285:52:30<127:41:06,  5.70s/it]
 64%|██████▍   | 144377/225000 [285:52:36<135:09:39,  6.04s/it]
 64%|██████▍   | 144378/225000 [285:52:41<124:00:46,  5.54s/it]
 64%|██████▍   | 144379/225000 [285:52:45<114:56:40,  5.13s/it]
 64%|██████▍   | 144380/225000 [285:52:49<109:24:12,  4.89s/it]
 64%|██████▍   | 144381/225000 [285:52:53<104:54:29,  4.68s/it]
 64%|██████▍   | 144382/225000 [285:53:05<147:58:10,  6.61s/it]
 64%|██████▍   | 144383/225000 [285:53:09<136:28:10,  6.09s/it]
 64%|██████▍   | 144384/225000 [285:53:14<125:56:18,  5.62s/it]
 64%|██████▍   | 144385/225000 [285:53:19<118:41:39,  5.30s/it]
 64%|██████▍   | 144386/225000 [285:53:23<115:08:37,  5.14s/it]
 64%|██████▍   | 144387/225000 [285:53:28<111:14:13,  4.97s/it]
 64%|██████▍   | 144388/225000 [285:53:33<109:27:23,  4.89s/it]
 64%|██████▍   | 144389/225000 [285:53:37<104:03:50,  4.65s/it]
 64%|██████▍   | 144390/225000 [285:53:43<114:06:52,  5.10s/it]
 64%|██████▍   | 144391/225000 [285:53:54<156:23:53,  6.98s/it]
 64%|██████▍   | 144392/225000 [285:53:58<131:53:45,  5.89s/it]
 64%|██████▍   | 144393/225000 [285:54:05<141:46:51,  6.33s/it]
 64%|██████▍   | 144394/225000 [285:54:09<127:23:08,  5.69s/it]
 64%|██████▍   | 144395/225000 [285:54:13<115:23:56,  5.15s/it]
 64%|██████▍   | 144396/225000 [285:54:17<108:34:29,  4.85s/it]
 64%|██████▍   | 144397/225000 [285:54:26<136:04:51,  6.08s/it]
 64%|██████▍   | 144398/225000 [285:54:33<143:13:59,  6.40s/it]
 64%|██████▍   | 144399/225000 [285:54:37<126:24:22,  5.65s/it]
 64%|██████▍   | 144400/225000 [285:54:41<117:16:41,  5.24s/it]
                                                               
{'loss': '0.7674', 'grad_norm': '1.09', 'learning_rate': '1.545e-05', 'epoch': '0.6418', 'num_input_tokens_seen': 473923256, 'train_runtime': '1.029e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144400/225000 [285:54:41<117:16:41,  5.24s/it]
 64%|██████▍   | 144401/225000 [285:54:46<110:29:40,  4.94s/it]
 64%|██████▍   | 144402/225000 [285:54:50<107:01:35,  4.78s/it]
 64%|██████▍   | 144403/225000 [285:54:55<104:54:17,  4.69s/it]
 64%|██████▍   | 144404/225000 [285:55:03<131:34:41,  5.88s/it]
 64%|██████▍   | 144405/225000 [285:55:13<157:42:40,  7.04s/it]
 64%|██████▍   | 144406/225000 [285:55:16<132:32:49,  5.92s/it]
 64%|██████▍   | 144407/225000 [285:55:21<124:19:58,  5.55s/it]
 64%|██████▍   | 144408/225000 [285:55:25<116:25:03,  5.20s/it]
 64%|██████▍   | 144409/225000 [285:55:29<109:33:29,  4.89s/it]
 64%|██████▍   | 144410/225000 [285:55:34<106:23:26,  4.75s/it]
 64%|██████▍   | 144411/225000 [285:55:40<118:20:55,  5.29s/it]
 64%|██████▍   | 144412/225000 [285:55:53<167:59:04,  7.50s/it]
 64%|██████▍   | 144413/225000 [285:55:57<140:51:07,  6.29s/it]
 64%|██████▍   | 144414/225000 [285:56:01<126:07:22,  5.63s/it]
 64%|██████▍   | 144415/225000 [285:56:06<122:16:42,  5.46s/it]
 64%|██████▍   | 144416/225000 [285:56:17<158:17:35,  7.07s/it]
 64%|██████▍   | 144417/225000 [285:56:20<137:01:05,  6.12s/it]
 64%|██████▍   | 144418/225000 [285:56:24<119:59:53,  5.36s/it]
 64%|██████▍   | 144419/225000 [285:56:28<112:14:03,  5.01s/it]
 64%|██████▍   | 144420/225000 [285:56:32<106:49:24,  4.77s/it]
 64%|██████▍   | 144421/225000 [285:56:37<104:00:01,  4.65s/it]
 64%|██████▍   | 144422/225000 [285:56:46<133:41:55,  5.97s/it]
 64%|██████▍   | 144423/225000 [285:56:52<131:22:36,  5.87s/it]
 64%|██████▍   | 144424/225000 [285:56:56<124:13:26,  5.55s/it]
 64%|██████▍   | 144425/225000 [285:57:01<120:53:14,  5.40s/it]
 64%|██████▍   | 144426/225000 [285:57:06<117:10:41,  5.24s/it]
 64%|██████▍   | 144427/225000 [285:57:11<114:53:20,  5.13s/it]
 64%|██████▍   | 144428/225000 [285:57:15<109:19:43,  4.88s/it]
 64%|██████▍   | 144429/225000 [285:57:20<109:33:04,  4.89s/it]
 64%|██████▍   | 144430/225000 [285:57:26<112:25:32,  5.02s/it]
 64%|██████▍   | 144431/225000 [285:57:31<111:40:00,  4.99s/it]
 64%|██████▍   | 144432/225000 [285:57:37<123:27:26,  5.52s/it]
 64%|██████▍   | 144433/225000 [285:57:49<163:18:35,  7.30s/it]
 64%|██████▍   | 144434/225000 [285:57:52<136:25:01,  6.10s/it]
 64%|██████▍   | 144435/225000 [285:57:57<131:16:06,  5.87s/it]
 64%|██████▍   | 144436/225000 [285:58:05<139:56:00,  6.25s/it]
 64%|██████▍   | 144437/225000 [285:58:11<141:59:57,  6.35s/it]
 64%|██████▍   | 144438/225000 [285:58:16<129:34:45,  5.79s/it]
 64%|██████▍   | 144439/225000 [285:58:20<117:33:45,  5.25s/it]
 64%|██████▍   | 144440/225000 [285:58:24<111:45:37,  4.99s/it]
 64%|██████▍   | 144441/225000 [285:58:30<118:34:27,  5.30s/it]
 64%|██████▍   | 144442/225000 [285:58:40<151:46:07,  6.78s/it]
 64%|██████▍   | 144443/225000 [285:58:44<129:25:39,  5.78s/it]
 64%|██████▍   | 144444/225000 [285:58:48<121:58:26,  5.45s/it]
 64%|██████▍   | 144445/225000 [285:58:53<115:54:08,  5.18s/it]
 64%|██████▍   | 144446/225000 [285:58:57<111:35:18,  4.99s/it]
 64%|██████▍   | 144447/225000 [285:59:02<108:55:43,  4.87s/it]
 64%|██████▍   | 144448/225000 [285:59:11<134:26:37,  6.01s/it]
 64%|██████▍   | 144449/225000 [285:59:20<152:53:39,  6.83s/it]
 64%|██████▍   | 144450/225000 [285:59:24<137:34:18,  6.15s/it]
                                                               
{'loss': '0.8035', 'grad_norm': '1.053', 'learning_rate': '1.543e-05', 'epoch': '0.642', 'num_input_tokens_seen': 474051096, 'train_runtime': '1.03e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144450/225000 [285:59:24<137:34:18,  6.15s/it]
 64%|██████▍   | 144451/225000 [285:59:28<123:50:13,  5.53s/it]
 64%|██████▍   | 144452/225000 [285:59:33<117:10:58,  5.24s/it]
 64%|██████▍   | 144453/225000 [285:59:37<110:19:50,  4.93s/it]
 64%|██████▍   | 144454/225000 [285:59:41<105:22:12,  4.71s/it]
 64%|██████▍   | 144455/225000 [285:59:53<154:33:56,  6.91s/it]
 64%|██████▍   | 144456/225000 [285:59:57<134:59:22,  6.03s/it]
 64%|██████▍   | 144457/225000 [286:00:04<143:44:46,  6.42s/it]
 64%|██████▍   | 144458/225000 [286:00:09<127:43:11,  5.71s/it]
 64%|██████▍   | 144459/225000 [286:00:13<117:45:32,  5.26s/it]
 64%|██████▍   | 144460/225000 [286:00:20<134:22:26,  6.01s/it]
 64%|██████▍   | 144461/225000 [286:00:25<127:11:52,  5.69s/it]
 64%|██████▍   | 144462/225000 [286:00:32<131:02:27,  5.86s/it]
 64%|██████▍   | 144463/225000 [286:00:36<123:57:07,  5.54s/it]
 64%|██████▍   | 144464/225000 [286:00:42<123:52:06,  5.54s/it]
 64%|██████▍   | 144465/225000 [286:00:47<121:16:32,  5.42s/it]
 64%|██████▍   | 144466/225000 [286:00:52<117:34:06,  5.26s/it]
 64%|██████▍   | 144467/225000 [286:00:57<118:01:23,  5.28s/it]
 64%|██████▍   | 144468/225000 [286:01:03<119:41:50,  5.35s/it]
 64%|██████▍   | 144469/225000 [286:01:08<117:28:00,  5.25s/it]
 64%|██████▍   | 144470/225000 [286:01:13<117:43:59,  5.26s/it]
 64%|██████▍   | 144471/225000 [286:01:20<125:27:15,  5.61s/it]
 64%|██████▍   | 144472/225000 [286:01:28<142:57:11,  6.39s/it]
 64%|██████▍   | 144473/225000 [286:01:32<128:34:53,  5.75s/it]
 64%|██████▍   | 144474/225000 [286:01:41<149:26:12,  6.68s/it]
 64%|██████▍   | 144475/225000 [286:01:44<128:02:57,  5.72s/it]
 64%|██████▍   | 144476/225000 [286:01:53<146:45:07,  6.56s/it]
 64%|██████▍   | 144477/225000 [286:01:57<128:21:30,  5.74s/it]
 64%|██████▍   | 144478/225000 [286:02:01<118:26:24,  5.30s/it]
 64%|██████▍   | 144479/225000 [286:02:06<115:59:23,  5.19s/it]
 64%|██████▍   | 144480/225000 [286:02:11<113:56:23,  5.09s/it]
 64%|██████▍   | 144481/225000 [286:02:16<111:51:58,  5.00s/it]
 64%|██████▍   | 144482/225000 [286:02:20<110:28:55,  4.94s/it]
 64%|██████▍   | 144483/225000 [286:02:28<125:38:20,  5.62s/it]
 64%|██████▍   | 144484/225000 [286:02:37<151:51:57,  6.79s/it]
 64%|██████▍   | 144485/225000 [286:02:42<137:46:48,  6.16s/it]
 64%|██████▍   | 144486/225000 [286:02:46<127:09:52,  5.69s/it]
 64%|██████▍   | 144487/225000 [286:02:51<119:45:07,  5.35s/it]
 64%|██████▍   | 144488/225000 [286:02:56<116:17:47,  5.20s/it]
 64%|██████▍   | 144489/225000 [286:03:06<149:47:56,  6.70s/it]
 64%|██████▍   | 144490/225000 [286:03:12<147:45:32,  6.61s/it]
 64%|██████▍   | 144491/225000 [286:03:16<127:32:21,  5.70s/it]
 64%|██████▍   | 144492/225000 [286:03:20<116:25:17,  5.21s/it]
 64%|██████▍   | 144493/225000 [286:03:24<110:45:18,  4.95s/it]
 64%|██████▍   | 144494/225000 [286:03:37<160:25:57,  7.17s/it]
 64%|██████▍   | 144495/225000 [286:03:41<141:51:42,  6.34s/it]
 64%|██████▍   | 144496/225000 [286:03:46<130:00:24,  5.81s/it]
 64%|██████▍   | 144497/225000 [286:03:51<123:33:44,  5.53s/it]
 64%|██████▍   | 144498/225000 [286:03:55<117:08:50,  5.24s/it]
 64%|██████▍   | 144499/225000 [286:04:00<115:55:31,  5.18s/it]
 64%|██████▍   | 144500/225000 [286:04:05<115:11:52,  5.15s/it]
                                                               
{'loss': '0.7899', 'grad_norm': '1.115', 'learning_rate': '1.541e-05', 'epoch': '0.6422', 'num_input_tokens_seen': 474181272, 'train_runtime': '1.03e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144500/225000 [286:04:05<115:11:52,  5.15s/it]
 64%|██████▍   | 144501/225000 [286:04:10<110:11:14,  4.93s/it]
 64%|██████▍   | 144502/225000 [286:04:21<152:32:47,  6.82s/it]
 64%|██████▍   | 144503/225000 [286:04:25<136:48:08,  6.12s/it]
 64%|██████▍   | 144504/225000 [286:04:29<123:01:24,  5.50s/it]
 64%|██████▍   | 144505/225000 [286:04:33<112:58:25,  5.05s/it]
 64%|██████▍   | 144506/225000 [286:04:37<105:04:15,  4.70s/it]
 64%|██████▍   | 144507/225000 [286:04:48<146:56:13,  6.57s/it]
 64%|██████▍   | 144508/225000 [286:04:52<125:29:09,  5.61s/it]
 64%|██████▍   | 144509/225000 [286:04:56<117:02:55,  5.24s/it]
 64%|██████▍   | 144510/225000 [286:05:01<112:01:35,  5.01s/it]
 64%|██████▍   | 144511/225000 [286:05:05<108:01:29,  4.83s/it]
 64%|██████▍   | 144512/225000 [286:05:15<141:51:32,  6.34s/it]
 64%|██████▍   | 144513/225000 [286:05:21<141:54:12,  6.35s/it]
 64%|██████▍   | 144514/225000 [286:05:25<126:58:22,  5.68s/it]
 64%|██████▍   | 144515/225000 [286:05:29<116:41:55,  5.22s/it]
 64%|██████▍   | 144516/225000 [286:05:33<108:30:04,  4.85s/it]
 64%|██████▍   | 144517/225000 [286:05:38<106:14:00,  4.75s/it]
 64%|██████▍   | 144518/225000 [286:05:43<107:12:05,  4.80s/it]
 64%|██████▍   | 144519/225000 [286:05:48<108:23:07,  4.85s/it]
 64%|██████▍   | 144520/225000 [286:05:53<110:53:28,  4.96s/it]
 64%|██████▍   | 144521/225000 [286:06:02<135:12:53,  6.05s/it]
 64%|██████▍   | 144522/225000 [286:06:10<153:15:41,  6.86s/it]
 64%|██████▍   | 144523/225000 [286:06:16<144:02:22,  6.44s/it]
 64%|██████▍   | 144524/225000 [286:06:21<132:44:16,  5.94s/it]
 64%|██████▍   | 144525/225000 [286:06:25<125:34:18,  5.62s/it]
 64%|██████▍   | 144526/225000 [286:06:30<115:43:19,  5.18s/it]
 64%|██████▍   | 144527/225000 [286:06:35<114:27:21,  5.12s/it]
 64%|██████▍   | 144528/225000 [286:06:40<114:58:30,  5.14s/it]
 64%|██████▍   | 144529/225000 [286:06:45<115:52:27,  5.18s/it]
 64%|██████▍   | 144530/225000 [286:06:50<115:59:35,  5.19s/it]
 64%|██████▍   | 144531/225000 [286:07:00<145:24:35,  6.51s/it]
 64%|██████▍   | 144532/225000 [286:07:09<166:03:48,  7.43s/it]
 64%|██████▍   | 144533/225000 [286:07:13<141:08:22,  6.31s/it]
 64%|██████▍   | 144534/225000 [286:07:20<142:03:43,  6.36s/it]
 64%|██████▍   | 144535/225000 [286:07:26<144:32:15,  6.47s/it]
 64%|██████▍   | 144536/225000 [286:07:34<152:28:29,  6.82s/it]
 64%|██████▍   | 144537/225000 [286:07:38<130:38:48,  5.85s/it]
 64%|██████▍   | 144538/225000 [286:07:41<116:00:48,  5.19s/it]
 64%|██████▍   | 144539/225000 [286:07:51<144:38:51,  6.47s/it]
 64%|██████▍   | 144540/225000 [286:07:55<132:56:37,  5.95s/it]
 64%|██████▍   | 144541/225000 [286:08:00<124:27:34,  5.57s/it]
 64%|██████▍   | 144542/225000 [286:08:04<115:39:00,  5.17s/it]
 64%|██████▍   | 144543/225000 [286:08:09<111:38:50,  5.00s/it]
 64%|██████▍   | 144544/225000 [286:08:13<107:03:50,  4.79s/it]
 64%|██████▍   | 144545/225000 [286:08:26<160:47:37,  7.19s/it]
 64%|██████▍   | 144546/225000 [286:08:31<142:35:04,  6.38s/it]
 64%|██████▍   | 144547/225000 [286:08:35<129:13:05,  5.78s/it]
 64%|██████▍   | 144548/225000 [286:08:39<116:10:22,  5.20s/it]
 64%|██████▍   | 144549/225000 [286:08:43<107:37:09,  4.82s/it]
 64%|██████▍   | 144550/225000 [286:08:50<122:43:07,  5.49s/it]
                                                               
{'loss': '0.7847', 'grad_norm': '1.107', 'learning_rate': '1.54e-05', 'epoch': '0.6424', 'num_input_tokens_seen': 474311864, 'train_runtime': '1.03e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144550/225000 [286:08:50<122:43:07,  5.49s/it]
 64%|██████▍   | 144551/225000 [286:08:59<148:44:58,  6.66s/it]
 64%|██████▍   | 144552/225000 [286:09:03<130:09:12,  5.82s/it]
 64%|██████▍   | 144553/225000 [286:09:08<121:57:00,  5.46s/it]
 64%|██████▍   | 144554/225000 [286:09:12<115:12:33,  5.16s/it]
 64%|██████▍   | 144555/225000 [286:09:17<111:47:29,  5.00s/it]
 64%|██████▍   | 144556/225000 [286:09:21<108:29:08,  4.85s/it]
 64%|██████▍   | 144557/225000 [286:09:26<108:04:41,  4.84s/it]
 64%|██████▍   | 144558/225000 [286:09:31<107:51:10,  4.83s/it]
 64%|██████▍   | 144559/225000 [286:09:43<154:59:31,  6.94s/it]
 64%|██████▍   | 144560/225000 [286:09:47<139:00:51,  6.22s/it]
 64%|██████▍   | 144561/225000 [286:09:52<127:35:16,  5.71s/it]
 64%|██████▍   | 144562/225000 [286:10:00<147:14:21,  6.59s/it]
 64%|██████▍   | 144563/225000 [286:10:04<127:14:43,  5.69s/it]
 64%|██████▍   | 144564/225000 [286:10:12<140:05:01,  6.27s/it]
 64%|██████▍   | 144565/225000 [286:10:18<141:36:46,  6.34s/it]
 64%|██████▍   | 144566/225000 [286:10:21<121:29:10,  5.44s/it]
 64%|██████▍   | 144567/225000 [286:10:30<143:10:32,  6.41s/it]
 64%|██████▍   | 144568/225000 [286:10:34<124:02:55,  5.55s/it]
 64%|██████▍   | 144569/225000 [286:10:37<112:04:31,  5.02s/it]
 64%|██████▍   | 144570/225000 [286:10:46<135:04:21,  6.05s/it]
 64%|██████▍   | 144571/225000 [286:10:50<124:20:07,  5.57s/it]
 64%|██████▍   | 144572/225000 [286:10:57<133:09:34,  5.96s/it]
 64%|██████▍   | 144573/225000 [286:11:01<119:35:35,  5.35s/it]
 64%|██████▍   | 144574/225000 [286:11:05<111:40:59,  5.00s/it]
 64%|██████▍   | 144575/225000 [286:11:14<134:45:35,  6.03s/it]
 64%|██████▍   | 144576/225000 [286:11:21<143:41:07,  6.43s/it]
 64%|██████▍   | 144577/225000 [286:11:24<121:19:09,  5.43s/it]
 64%|██████▍   | 144578/225000 [286:11:33<140:57:09,  6.31s/it]
 64%|██████▍   | 144579/225000 [286:11:36<122:04:41,  5.46s/it]
 64%|██████▍   | 144580/225000 [286:11:41<119:06:21,  5.33s/it]
 64%|██████▍   | 144581/225000 [286:11:50<141:30:58,  6.34s/it]
 64%|██████▍   | 144582/225000 [286:11:55<134:09:50,  6.01s/it]
 64%|██████▍   | 144583/225000 [286:12:00<127:30:49,  5.71s/it]
 64%|██████▍   | 144584/225000 [286:12:05<122:14:32,  5.47s/it]
 64%|██████▍   | 144585/225000 [286:12:09<116:11:36,  5.20s/it]
 64%|██████▍   | 144586/225000 [286:12:15<115:01:53,  5.15s/it]
 64%|██████▍   | 144587/225000 [286:12:20<114:41:27,  5.13s/it]
 64%|██████▍   | 144588/225000 [286:12:25<114:30:30,  5.13s/it]
 64%|██████▍   | 144589/225000 [286:12:30<113:05:28,  5.06s/it]
 64%|██████▍   | 144590/225000 [286:12:35<112:08:37,  5.02s/it]
 64%|██████▍   | 144591/225000 [286:12:40<113:03:51,  5.06s/it]
 64%|██████▍   | 144592/225000 [286:12:44<109:42:19,  4.91s/it]
 64%|██████▍   | 144593/225000 [286:12:49<108:21:57,  4.85s/it]
 64%|██████▍   | 144594/225000 [286:12:54<107:44:33,  4.82s/it]
 64%|██████▍   | 144595/225000 [286:13:04<144:37:25,  6.48s/it]
 64%|██████▍   | 144596/225000 [286:13:12<155:07:09,  6.95s/it]
 64%|██████▍   | 144597/225000 [286:13:15<129:39:09,  5.81s/it]
 64%|██████▍   | 144598/225000 [286:13:24<147:26:58,  6.60s/it]
 64%|██████▍   | 144599/225000 [286:13:27<127:18:02,  5.70s/it]
 64%|██████▍   | 144600/225000 [286:13:36<147:25:29,  6.60s/it]
                                                               
{'loss': '0.7952', 'grad_norm': '1.187', 'learning_rate': '1.538e-05', 'epoch': '0.6427', 'num_input_tokens_seen': 474440592, 'train_runtime': '1.03e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144600/225000 [286:13:36<147:25:29,  6.60s/it]
 64%|██████▍   | 144601/225000 [286:13:40<128:30:34,  5.75s/it]
 64%|██████▍   | 144602/225000 [286:13:43<113:59:46,  5.10s/it]
 64%|██████▍   | 144603/225000 [286:13:48<112:36:28,  5.04s/it]
 64%|██████▍   | 144604/225000 [286:13:53<110:12:35,  4.94s/it]
 64%|██████▍   | 144605/225000 [286:13:58<111:56:48,  5.01s/it]
 64%|██████▍   | 144606/225000 [286:14:03<110:29:14,  4.95s/it]
 64%|██████▍   | 144607/225000 [286:14:12<137:59:58,  6.18s/it]
 64%|██████▍   | 144608/225000 [286:14:19<140:27:03,  6.29s/it]
 64%|██████▍   | 144609/225000 [286:14:26<148:33:50,  6.65s/it]
 64%|██████▍   | 144610/225000 [286:14:32<140:28:46,  6.29s/it]
 64%|██████▍   | 144611/225000 [286:14:37<133:39:54,  5.99s/it]
 64%|██████▍   | 144612/225000 [286:14:43<137:06:15,  6.14s/it]
 64%|██████▍   | 144613/225000 [286:14:50<141:12:25,  6.32s/it]
 64%|██████▍   | 144614/225000 [286:14:54<124:21:26,  5.57s/it]
 64%|██████▍   | 144615/225000 [286:14:57<110:09:17,  4.93s/it]
 64%|██████▍   | 144616/225000 [286:15:07<145:08:52,  6.50s/it]
 64%|██████▍   | 144617/225000 [286:15:14<142:09:32,  6.37s/it]
 64%|██████▍   | 144618/225000 [286:15:17<123:03:35,  5.51s/it]
 64%|██████▍   | 144619/225000 [286:15:21<109:37:51,  4.91s/it]
 64%|██████▍   | 144620/225000 [286:15:25<103:36:51,  4.64s/it]
 64%|██████▍   | 144621/225000 [286:15:36<146:57:28,  6.58s/it]
 64%|██████▍   | 144622/225000 [286:15:40<128:44:53,  5.77s/it]
 64%|██████▍   | 144623/225000 [286:15:43<116:37:25,  5.22s/it]
 64%|██████▍   | 144624/225000 [286:15:47<106:12:27,  4.76s/it]
 64%|██████▍   | 144625/225000 [286:15:58<145:34:50,  6.52s/it]
 64%|██████▍   | 144626/225000 [286:16:04<141:37:35,  6.34s/it]
 64%|██████▍   | 144627/225000 [286:16:09<133:00:43,  5.96s/it]
 64%|██████▍   | 144628/225000 [286:16:13<121:29:42,  5.44s/it]
 64%|██████▍   | 144629/225000 [286:16:18<117:34:12,  5.27s/it]
 64%|██████▍   | 144630/225000 [286:16:23<116:07:40,  5.20s/it]
 64%|██████▍   | 144631/225000 [286:16:28<112:07:53,  5.02s/it]
 64%|██████▍   | 144632/225000 [286:16:33<112:01:26,  5.02s/it]
 64%|██████▍   | 144633/225000 [286:16:38<112:49:54,  5.05s/it]
 64%|██████▍   | 144634/225000 [286:16:50<162:54:08,  7.30s/it]
 64%|██████▍   | 144635/225000 [286:16:55<143:05:04,  6.41s/it]
 64%|██████▍   | 144636/225000 [286:16:58<124:03:53,  5.56s/it]
 64%|██████▍   | 144637/225000 [286:17:06<139:34:16,  6.25s/it]
 64%|██████▍   | 144638/225000 [286:17:11<128:51:28,  5.77s/it]
 64%|██████▍   | 144639/225000 [286:17:19<143:00:17,  6.41s/it]
 64%|██████▍   | 144640/225000 [286:17:22<124:32:13,  5.58s/it]
 64%|██████▍   | 144641/225000 [286:17:30<140:03:24,  6.27s/it]
 64%|██████▍   | 144642/225000 [286:17:33<121:00:01,  5.42s/it]
 64%|██████▍   | 144643/225000 [286:17:40<130:56:48,  5.87s/it]
 64%|██████▍   | 144644/225000 [286:17:46<128:35:11,  5.76s/it]
 64%|██████▍   | 144645/225000 [286:17:50<118:25:20,  5.31s/it]
 64%|██████▍   | 144646/225000 [286:17:54<110:39:01,  4.96s/it]
 64%|██████▍   | 144647/225000 [286:17:59<108:24:49,  4.86s/it]
 64%|██████▍   | 144648/225000 [286:18:07<127:32:12,  5.71s/it]
 64%|██████▍   | 144649/225000 [286:18:15<148:26:10,  6.65s/it]
 64%|██████▍   | 144650/225000 [286:18:22<150:32:39,  6.74s/it]
                                                               
{'loss': '0.7771', 'grad_norm': '1.132', 'learning_rate': '1.536e-05', 'epoch': '0.6429', 'num_input_tokens_seen': 474571456, 'train_runtime': '1.031e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144650/225000 [286:18:22<150:32:39,  6.74s/it]
 64%|██████▍   | 144651/225000 [286:18:26<127:27:40,  5.71s/it]
 64%|██████▍   | 144652/225000 [286:18:29<113:04:48,  5.07s/it]
 64%|██████▍   | 144653/225000 [286:18:33<106:49:21,  4.79s/it]
 64%|██████▍   | 144654/225000 [286:18:38<103:10:30,  4.62s/it]
 64%|██████▍   | 144655/225000 [286:18:48<138:08:10,  6.19s/it]
 64%|██████▍   | 144656/225000 [286:18:53<130:46:32,  5.86s/it]
 64%|██████▍   | 144657/225000 [286:18:57<122:56:56,  5.51s/it]
 64%|██████▍   | 144658/225000 [286:19:02<114:53:26,  5.15s/it]
 64%|██████▍   | 144659/225000 [286:19:06<111:55:13,  5.02s/it]
 64%|██████▍   | 144660/225000 [286:19:11<111:04:17,  4.98s/it]
 64%|██████▍   | 144661/225000 [286:19:16<111:30:58,  5.00s/it]
 64%|██████▍   | 144662/225000 [286:19:21<109:09:42,  4.89s/it]
 64%|██████▍   | 144663/225000 [286:19:32<151:53:33,  6.81s/it]
 64%|██████▍   | 144664/225000 [286:19:39<149:09:16,  6.68s/it]
 64%|██████▍   | 144665/225000 [286:19:42<126:59:29,  5.69s/it]
 64%|██████▍   | 144666/225000 [286:19:49<135:23:15,  6.07s/it]
 64%|██████▍   | 144667/225000 [286:19:53<120:23:17,  5.40s/it]
 64%|██████▍   | 144668/225000 [286:19:57<111:07:44,  4.98s/it]
 64%|██████▍   | 144669/225000 [286:20:06<140:45:58,  6.31s/it]
 64%|██████▍   | 144670/225000 [286:20:11<129:17:16,  5.79s/it]
 64%|██████▍   | 144671/225000 [286:20:15<119:04:53,  5.34s/it]
 64%|██████▍   | 144672/225000 [286:20:19<113:17:42,  5.08s/it]
 64%|██████▍   | 144673/225000 [286:20:30<151:43:35,  6.80s/it]
 64%|██████▍   | 144674/225000 [286:20:34<128:40:28,  5.77s/it]
 64%|██████▍   | 144675/225000 [286:20:39<125:28:50,  5.62s/it]
 64%|██████▍   | 144676/225000 [286:20:45<129:44:31,  5.81s/it]
 64%|██████▍   | 144677/225000 [286:20:50<122:08:16,  5.47s/it]
 64%|██████▍   | 144678/225000 [286:20:55<116:32:54,  5.22s/it]
 64%|██████▍   | 144679/225000 [286:20:59<109:33:00,  4.91s/it]
 64%|██████▍   | 144680/225000 [286:21:03<107:08:13,  4.80s/it]
 64%|██████▍   | 144681/225000 [286:21:08<106:47:29,  4.79s/it]
 64%|██████▍   | 144682/225000 [286:21:20<153:08:32,  6.86s/it]
 64%|██████▍   | 144683/225000 [286:21:24<138:00:31,  6.19s/it]
 64%|██████▍   | 144684/225000 [286:21:30<135:33:56,  6.08s/it]
 64%|██████▍   | 144685/225000 [286:21:37<140:18:42,  6.29s/it]
 64%|██████▍   | 144686/225000 [286:21:41<127:52:43,  5.73s/it]
 64%|██████▍   | 144687/225000 [286:21:46<121:00:17,  5.42s/it]
 64%|██████▍   | 144688/225000 [286:21:51<116:51:56,  5.24s/it]
 64%|██████▍   | 144689/225000 [286:21:58<132:30:29,  5.94s/it]
 64%|██████▍   | 144690/225000 [286:22:07<148:45:44,  6.67s/it]
 64%|██████▍   | 144691/225000 [286:22:10<128:19:26,  5.75s/it]
 64%|██████▍   | 144692/225000 [286:22:18<140:31:28,  6.30s/it]
 64%|██████▍   | 144693/225000 [286:22:22<122:02:17,  5.47s/it]
 64%|██████▍   | 144694/225000 [286:22:30<140:44:28,  6.31s/it]
 64%|██████▍   | 144695/225000 [286:22:34<126:18:31,  5.66s/it]
 64%|██████▍   | 144696/225000 [286:22:39<119:47:21,  5.37s/it]
 64%|██████▍   | 144697/225000 [286:22:44<122:18:38,  5.48s/it]
 64%|██████▍   | 144698/225000 [286:22:53<140:19:00,  6.29s/it]
 64%|██████▍   | 144699/225000 [286:22:57<125:25:16,  5.62s/it]
 64%|██████▍   | 144700/225000 [286:23:02<120:37:47,  5.41s/it]
                                                               
{'loss': '0.7915', 'grad_norm': '1.084', 'learning_rate': '1.535e-05', 'epoch': '0.6431', 'num_input_tokens_seen': 474702312, 'train_runtime': '1.031e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144700/225000 [286:23:02<120:37:47,  5.41s/it]
 64%|██████▍   | 144701/225000 [286:23:11<147:47:04,  6.63s/it]
 64%|██████▍   | 144702/225000 [286:23:14<126:19:49,  5.66s/it]
 64%|██████▍   | 144703/225000 [286:23:23<144:39:50,  6.49s/it]
 64%|██████▍   | 144704/225000 [286:23:26<124:05:23,  5.56s/it]
 64%|██████▍   | 144705/225000 [286:23:34<137:59:24,  6.19s/it]
 64%|██████▍   | 144706/225000 [286:23:38<121:27:43,  5.45s/it]
 64%|██████▍   | 144707/225000 [286:23:42<115:41:05,  5.19s/it]
 64%|██████▍   | 144708/225000 [286:23:47<111:32:18,  5.00s/it]
 64%|██████▍   | 144709/225000 [286:23:52<111:27:00,  5.00s/it]
 64%|██████▍   | 144710/225000 [286:23:59<129:58:06,  5.83s/it]
 64%|██████▍   | 144711/225000 [286:24:09<156:45:28,  7.03s/it]
 64%|██████▍   | 144712/225000 [286:24:14<140:07:17,  6.28s/it]
 64%|██████▍   | 144713/225000 [286:24:18<128:04:08,  5.74s/it]
 64%|██████▍   | 144714/225000 [286:24:23<121:09:49,  5.43s/it]
 64%|██████▍   | 144715/225000 [286:24:28<115:40:21,  5.19s/it]
 64%|██████▍   | 144716/225000 [286:24:32<110:29:34,  4.95s/it]
 64%|██████▍   | 144717/225000 [286:24:42<144:15:50,  6.47s/it]
 64%|██████▍   | 144718/225000 [286:24:47<136:49:48,  6.14s/it]
 64%|██████▍   | 144719/225000 [286:24:56<150:28:57,  6.75s/it]
 64%|██████▍   | 144720/225000 [286:25:02<148:16:59,  6.65s/it]
 64%|██████▍   | 144721/225000 [286:25:07<135:39:12,  6.08s/it]
 64%|██████▍   | 144722/225000 [286:25:13<133:39:13,  5.99s/it]
 64%|██████▍   | 144723/225000 [286:25:18<126:46:21,  5.69s/it]
 64%|██████▍   | 144724/225000 [286:25:21<113:07:58,  5.07s/it]
 64%|██████▍   | 144725/225000 [286:25:30<137:32:36,  6.17s/it]
 64%|██████▍   | 144726/225000 [286:25:34<125:10:13,  5.61s/it]
 64%|██████▍   | 144727/225000 [286:25:38<111:25:24,  5.00s/it]
 64%|██████▍   | 144728/225000 [286:25:43<113:37:19,  5.10s/it]
 64%|██████▍   | 144729/225000 [286:25:50<125:09:17,  5.61s/it]
 64%|██████▍   | 144730/225000 [286:25:54<111:34:47,  5.00s/it]
 64%|██████▍   | 144731/225000 [286:26:02<136:37:20,  6.13s/it]
 64%|██████▍   | 144732/225000 [286:26:07<124:30:37,  5.58s/it]
 64%|██████▍   | 144733/225000 [286:26:12<120:43:27,  5.41s/it]
 64%|██████▍   | 144734/225000 [286:26:17<118:48:33,  5.33s/it]
 64%|██████▍   | 144735/225000 [286:26:21<114:31:53,  5.14s/it]
 64%|██████▍   | 144736/225000 [286:26:26<110:11:45,  4.94s/it]
 64%|██████▍   | 144737/225000 [286:26:31<109:07:44,  4.89s/it]
 64%|██████▍   | 144738/225000 [286:26:36<109:09:37,  4.90s/it]
 64%|██████▍   | 144739/225000 [286:26:46<147:16:13,  6.61s/it]
 64%|██████▍   | 144740/225000 [286:26:52<143:43:28,  6.45s/it]
 64%|██████▍   | 144741/225000 [286:26:56<126:20:12,  5.67s/it]
 64%|██████▍   | 144742/225000 [286:27:00<116:09:50,  5.21s/it]
 64%|██████▍   | 144743/225000 [286:27:05<111:55:04,  5.02s/it]
 64%|██████▍   | 144744/225000 [286:27:09<107:41:02,  4.83s/it]
 64%|██████▍   | 144745/225000 [286:27:15<114:05:31,  5.12s/it]
 64%|██████▍   | 144746/225000 [286:27:25<144:43:59,  6.49s/it]
 64%|██████▍   | 144747/225000 [286:27:29<128:37:01,  5.77s/it]
 64%|██████▍   | 144748/225000 [286:27:33<120:17:25,  5.40s/it]
 64%|██████▍   | 144749/225000 [286:27:43<147:55:31,  6.64s/it]
 64%|██████▍   | 144750/225000 [286:27:49<143:29:01,  6.44s/it]
                                                               
{'loss': '0.7835', 'grad_norm': '1.089', 'learning_rate': '1.533e-05', 'epoch': '0.6433', 'num_input_tokens_seen': 474833880, 'train_runtime': '1.031e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144750/225000 [286:27:49<143:29:01,  6.44s/it]
 64%|██████▍   | 144751/225000 [286:27:53<126:58:08,  5.70s/it]
 64%|██████▍   | 144752/225000 [286:27:57<115:02:08,  5.16s/it]
 64%|██████▍   | 144753/225000 [286:28:07<152:40:31,  6.85s/it]
 64%|██████▍   | 144754/225000 [286:28:11<128:38:50,  5.77s/it]
 64%|██████▍   | 144755/225000 [286:28:18<140:34:57,  6.31s/it]
 64%|██████▍   | 144756/225000 [286:28:23<133:02:02,  5.97s/it]
 64%|██████▍   | 144757/225000 [286:28:29<128:06:53,  5.75s/it]
 64%|██████▍   | 144758/225000 [286:28:34<122:30:55,  5.50s/it]
 64%|██████▍   | 144759/225000 [286:28:39<120:54:50,  5.42s/it]
 64%|██████▍   | 144760/225000 [286:28:44<117:20:36,  5.26s/it]
 64%|██████▍   | 144761/225000 [286:28:49<115:28:23,  5.18s/it]
 64%|██████▍   | 144762/225000 [286:28:54<115:46:13,  5.19s/it]
 64%|██████▍   | 144763/225000 [286:28:59<117:17:27,  5.26s/it]
 64%|██████▍   | 144764/225000 [286:29:05<116:47:43,  5.24s/it]
 64%|██████▍   | 144765/225000 [286:29:10<115:34:55,  5.19s/it]
 64%|██████▍   | 144766/225000 [286:29:15<115:11:41,  5.17s/it]
 64%|██████▍   | 144767/225000 [286:29:20<117:11:21,  5.26s/it]
 64%|██████▍   | 144768/225000 [286:29:25<116:01:29,  5.21s/it]
 64%|██████▍   | 144769/225000 [286:29:31<119:31:19,  5.36s/it]
 64%|██████▍   | 144770/225000 [286:29:36<118:34:36,  5.32s/it]
 64%|██████▍   | 144771/225000 [286:29:47<155:49:44,  6.99s/it]
 64%|██████▍   | 144772/225000 [286:29:56<168:04:54,  7.54s/it]
 64%|██████▍   | 144773/225000 [286:30:00<144:30:51,  6.48s/it]
 64%|██████▍   | 144774/225000 [286:30:05<136:36:23,  6.13s/it]
 64%|██████▍   | 144775/225000 [286:30:11<133:09:11,  5.98s/it]
 64%|██████▍   | 144776/225000 [286:30:17<130:27:03,  5.85s/it]
 64%|██████▍   | 144777/225000 [286:30:25<148:50:36,  6.68s/it]
 64%|██████▍   | 144778/225000 [286:30:29<129:46:09,  5.82s/it]
 64%|██████▍   | 144779/225000 [286:30:32<114:38:57,  5.15s/it]
 64%|██████▍   | 144780/225000 [286:30:42<144:25:38,  6.48s/it]
 64%|██████▍   | 144781/225000 [286:30:45<119:38:57,  5.37s/it]
 64%|██████▍   | 144782/225000 [286:30:49<111:45:44,  5.02s/it]
 64%|██████▍   | 144783/225000 [286:30:54<111:28:30,  5.00s/it]
 64%|██████▍   | 144784/225000 [286:30:59<111:09:42,  4.99s/it]
 64%|██████▍   | 144785/225000 [286:31:04<112:41:27,  5.06s/it]
 64%|██████▍   | 144786/225000 [286:31:09<113:21:03,  5.09s/it]
 64%|██████▍   | 144787/225000 [286:31:15<114:41:09,  5.15s/it]
 64%|██████▍   | 144788/225000 [286:31:20<115:02:06,  5.16s/it]
 64%|██████▍   | 144789/225000 [286:31:25<113:25:18,  5.09s/it]
 64%|██████▍   | 144790/225000 [286:31:30<113:15:24,  5.08s/it]
 64%|██████▍   | 144791/225000 [286:31:35<112:45:49,  5.06s/it]
 64%|██████▍   | 144792/225000 [286:31:40<112:23:43,  5.04s/it]
 64%|██████▍   | 144793/225000 [286:31:45<115:19:53,  5.18s/it]
 64%|██████▍   | 144794/225000 [286:31:51<116:05:35,  5.21s/it]
 64%|██████▍   | 144795/225000 [286:31:56<118:24:17,  5.31s/it]
 64%|██████▍   | 144796/225000 [286:32:02<119:36:22,  5.37s/it]
 64%|██████▍   | 144797/225000 [286:32:08<126:49:18,  5.69s/it]
 64%|██████▍   | 144798/225000 [286:32:13<124:17:41,  5.58s/it]
 64%|██████▍   | 144799/225000 [286:32:19<122:59:32,  5.52s/it]
 64%|██████▍   | 144800/225000 [286:32:24<121:52:06,  5.47s/it]
                                                               
{'loss': '0.7943', 'grad_norm': '1.145', 'learning_rate': '1.531e-05', 'epoch': '0.6436', 'num_input_tokens_seen': 474962952, 'train_runtime': '1.032e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144800/225000 [286:32:24<121:52:06,  5.47s/it]
 64%|██████▍   | 144801/225000 [286:32:29<119:08:11,  5.35s/it]
 64%|██████▍   | 144802/225000 [286:32:35<119:38:54,  5.37s/it]
 64%|██████▍   | 144803/225000 [286:32:42<134:10:44,  6.02s/it]
 64%|██████▍   | 144804/225000 [286:32:53<168:43:03,  7.57s/it]
 64%|██████▍   | 144805/225000 [286:32:57<142:35:22,  6.40s/it]
 64%|██████▍   | 144806/225000 [286:33:01<125:24:36,  5.63s/it]
 64%|██████▍   | 144807/225000 [286:33:12<159:28:28,  7.16s/it]
 64%|██████▍   | 144808/225000 [286:33:15<136:53:52,  6.15s/it]
 64%|██████▍   | 144809/225000 [286:33:19<120:49:38,  5.42s/it]
 64%|██████▍   | 144810/225000 [286:33:29<149:31:12,  6.71s/it]
 64%|██████▍   | 144811/225000 [286:33:35<143:07:24,  6.43s/it]
 64%|██████▍   | 144812/225000 [286:33:39<127:51:03,  5.74s/it]
 64%|██████▍   | 144813/225000 [286:33:43<117:46:26,  5.29s/it]
 64%|██████▍   | 144814/225000 [286:33:47<111:30:24,  5.01s/it]
 64%|██████▍   | 144815/225000 [286:33:53<116:40:32,  5.24s/it]
 64%|██████▍   | 144816/225000 [286:34:03<145:15:35,  6.52s/it]
 64%|██████▍   | 144817/225000 [286:34:06<122:49:32,  5.51s/it]
 64%|██████▍   | 144818/225000 [286:34:14<139:51:56,  6.28s/it]
 64%|██████▍   | 144819/225000 [286:34:17<119:17:02,  5.36s/it]
 64%|██████▍   | 144820/225000 [286:34:25<138:11:12,  6.20s/it]
 64%|██████▍   | 144821/225000 [286:34:29<122:39:57,  5.51s/it]
 64%|██████▍   | 144822/225000 [286:34:33<114:00:16,  5.12s/it]
 64%|██████▍   | 144823/225000 [286:34:37<105:22:39,  4.73s/it]
 64%|██████▍   | 144824/225000 [286:34:46<135:52:31,  6.10s/it]
 64%|██████▍   | 144825/225000 [286:34:51<125:58:37,  5.66s/it]
 64%|██████▍   | 144826/225000 [286:34:56<120:52:10,  5.43s/it]
 64%|██████▍   | 144827/225000 [286:35:01<116:45:21,  5.24s/it]
 64%|██████▍   | 144828/225000 [286:35:06<115:13:26,  5.17s/it]
 64%|██████▍   | 144829/225000 [286:35:11<112:36:49,  5.06s/it]
 64%|██████▍   | 144830/225000 [286:35:15<107:27:28,  4.83s/it]
 64%|██████▍   | 144831/225000 [286:35:19<105:52:43,  4.75s/it]
 64%|██████▍   | 144832/225000 [286:35:29<138:33:03,  6.22s/it]
 64%|██████▍   | 144833/225000 [286:35:38<154:33:55,  6.94s/it]
 64%|██████▍   | 144834/225000 [286:35:41<130:31:51,  5.86s/it]
 64%|██████▍   | 144835/225000 [286:35:46<125:48:12,  5.65s/it]
 64%|██████▍   | 144836/225000 [286:35:51<119:14:39,  5.36s/it]
 64%|██████▍   | 144837/225000 [286:35:56<119:51:12,  5.38s/it]
 64%|██████▍   | 144838/225000 [286:36:02<118:32:28,  5.32s/it]
 64%|██████▍   | 144839/225000 [286:36:07<117:35:43,  5.28s/it]
 64%|██████▍   | 144840/225000 [286:36:12<117:20:25,  5.27s/it]
 64%|██████▍   | 144841/225000 [286:36:17<116:32:28,  5.23s/it]
 64%|██████▍   | 144842/225000 [286:36:23<118:28:34,  5.32s/it]
 64%|██████▍   | 144843/225000 [286:36:28<116:36:51,  5.24s/it]
 64%|██████▍   | 144844/225000 [286:36:33<114:53:38,  5.16s/it]
 64%|██████▍   | 144845/225000 [286:36:38<115:44:09,  5.20s/it]
 64%|██████▍   | 144846/225000 [286:36:44<120:07:37,  5.40s/it]
 64%|██████▍   | 144847/225000 [286:36:49<120:01:28,  5.39s/it]
 64%|██████▍   | 144848/225000 [286:36:55<120:43:03,  5.42s/it]
 64%|██████▍   | 144849/225000 [286:36:59<115:33:28,  5.19s/it]
 64%|██████▍   | 144850/225000 [286:37:09<144:06:13,  6.47s/it]
                                                               
{'loss': '0.7917', 'grad_norm': '1.004', 'learning_rate': '1.53e-05', 'epoch': '0.6438', 'num_input_tokens_seen': 475093096, 'train_runtime': '1.032e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144850/225000 [286:37:09<144:06:13,  6.47s/it]
 64%|██████▍   | 144851/225000 [286:37:19<169:26:08,  7.61s/it]
 64%|██████▍   | 144852/225000 [286:37:22<141:34:12,  6.36s/it]
 64%|██████▍   | 144853/225000 [286:37:31<158:34:46,  7.12s/it]
 64%|██████▍   | 144854/225000 [286:37:35<135:26:21,  6.08s/it]
 64%|██████▍   | 144855/225000 [286:37:42<138:41:14,  6.23s/it]
 64%|██████▍   | 144856/225000 [286:37:48<137:18:06,  6.17s/it]
 64%|██████▍   | 144857/225000 [286:37:53<129:51:52,  5.83s/it]
 64%|██████▍   | 144858/225000 [286:37:57<119:25:46,  5.36s/it]
 64%|██████▍   | 144859/225000 [286:38:01<111:56:59,  5.03s/it]
 64%|██████▍   | 144860/225000 [286:38:06<107:24:44,  4.83s/it]
 64%|██████▍   | 144861/225000 [286:38:16<142:36:18,  6.41s/it]
 64%|██████▍   | 144862/225000 [286:38:21<138:01:11,  6.20s/it]
 64%|██████▍   | 144863/225000 [286:38:25<121:38:05,  5.46s/it]
 64%|██████▍   | 144864/225000 [286:38:30<117:58:29,  5.30s/it]
 64%|██████▍   | 144865/225000 [286:38:40<148:57:08,  6.69s/it]
 64%|██████▍   | 144866/225000 [286:38:44<129:19:30,  5.81s/it]
 64%|██████▍   | 144867/225000 [286:38:48<119:52:54,  5.39s/it]
 64%|██████▍   | 144868/225000 [286:38:52<112:32:01,  5.06s/it]
 64%|██████▍   | 144869/225000 [286:39:01<135:29:02,  6.09s/it]
 64%|██████▍   | 144870/225000 [286:39:07<137:56:19,  6.20s/it]
 64%|██████▍   | 144871/225000 [286:39:12<124:44:54,  5.60s/it]
 64%|██████▍   | 144872/225000 [286:39:16<118:18:15,  5.32s/it]
 64%|██████▍   | 144873/225000 [286:39:21<111:32:25,  5.01s/it]
 64%|██████▍   | 144874/225000 [286:39:25<105:07:34,  4.72s/it]
 64%|██████▍   | 144875/225000 [286:39:37<156:58:38,  7.05s/it]
 64%|██████▍   | 144876/225000 [286:39:41<135:57:40,  6.11s/it]
 64%|██████▍   | 144877/225000 [286:39:45<119:40:26,  5.38s/it]
 64%|██████▍   | 144878/225000 [286:39:51<123:50:41,  5.56s/it]
 64%|██████▍   | 144879/225000 [286:39:59<140:04:45,  6.29s/it]
 64%|██████▍   | 144880/225000 [286:40:02<120:41:01,  5.42s/it]
 64%|██████▍   | 144881/225000 [286:40:06<108:42:15,  4.88s/it]
 64%|██████▍   | 144882/225000 [286:40:15<138:22:58,  6.22s/it]
 64%|██████▍   | 144883/225000 [286:40:19<121:03:00,  5.44s/it]
 64%|██████▍   | 144884/225000 [286:40:23<113:10:52,  5.09s/it]
 64%|██████▍   | 144885/225000 [286:40:30<124:00:37,  5.57s/it]
 64%|██████▍   | 144886/225000 [286:40:33<112:14:02,  5.04s/it]
 64%|██████▍   | 144887/225000 [286:40:42<133:57:05,  6.02s/it]
 64%|██████▍   | 144888/225000 [286:40:49<143:13:28,  6.44s/it]
 64%|██████▍   | 144889/225000 [286:40:53<129:22:32,  5.81s/it]
 64%|██████▍   | 144890/225000 [286:40:57<115:17:39,  5.18s/it]
 64%|██████▍   | 144891/225000 [286:41:01<105:56:33,  4.76s/it]
 64%|██████▍   | 144892/225000 [286:41:05<103:07:21,  4.63s/it]
 64%|██████▍   | 144893/225000 [286:41:16<142:08:21,  6.39s/it]
 64%|██████▍   | 144894/225000 [286:41:20<125:09:03,  5.62s/it]
 64%|██████▍   | 144895/225000 [286:41:23<111:03:17,  4.99s/it]
 64%|██████▍   | 144896/225000 [286:41:33<144:38:58,  6.50s/it]
 64%|██████▍   | 144897/225000 [286:41:37<125:58:22,  5.66s/it]
 64%|██████▍   | 144898/225000 [286:41:42<119:31:32,  5.37s/it]
 64%|██████▍   | 144899/225000 [286:41:51<143:29:44,  6.45s/it]
 64%|██████▍   | 144900/225000 [286:41:54<125:21:43,  5.63s/it]
                                                               
{'loss': '0.7967', 'grad_norm': '0.9848', 'learning_rate': '1.528e-05', 'epoch': '0.644', 'num_input_tokens_seen': 475222864, 'train_runtime': '1.032e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144900/225000 [286:41:54<125:21:43,  5.63s/it]
 64%|██████▍   | 144901/225000 [286:42:01<131:15:31,  5.90s/it]
 64%|██████▍   | 144902/225000 [286:42:07<131:49:21,  5.92s/it]
 64%|██████▍   | 144903/225000 [286:42:10<115:37:53,  5.20s/it]
 64%|██████▍   | 144904/225000 [286:42:20<145:43:27,  6.55s/it]
 64%|██████▍   | 144905/225000 [286:42:24<128:51:41,  5.79s/it]
 64%|██████▍   | 144906/225000 [286:42:29<122:50:48,  5.52s/it]
 64%|██████▍   | 144907/225000 [286:42:34<121:51:51,  5.48s/it]
 64%|██████▍   | 144908/225000 [286:42:39<119:49:05,  5.39s/it]
 64%|██████▍   | 144909/225000 [286:42:44<114:25:27,  5.14s/it]
 64%|██████▍   | 144910/225000 [286:42:49<111:43:07,  5.02s/it]
 64%|██████▍   | 144911/225000 [286:42:55<119:02:22,  5.35s/it]
 64%|██████▍   | 144912/225000 [286:43:07<161:13:25,  7.25s/it]
 64%|██████▍   | 144913/225000 [286:43:10<136:27:52,  6.13s/it]
 64%|██████▍   | 144914/225000 [286:43:14<119:48:39,  5.39s/it]
 64%|██████▍   | 144915/225000 [286:43:18<110:33:26,  4.97s/it]
 64%|██████▍   | 144916/225000 [286:43:24<119:58:28,  5.39s/it]
 64%|██████▍   | 144917/225000 [286:43:34<151:02:54,  6.79s/it]
 64%|██████▍   | 144918/225000 [286:43:38<130:04:31,  5.85s/it]
 64%|██████▍   | 144919/225000 [286:43:46<143:23:41,  6.45s/it]
 64%|██████▍   | 144920/225000 [286:43:50<130:18:05,  5.86s/it]
 64%|██████▍   | 144921/225000 [286:43:56<132:25:18,  5.95s/it]
 64%|██████▍   | 144922/225000 [286:44:01<123:47:27,  5.57s/it]
 64%|██████▍   | 144923/225000 [286:44:06<117:26:22,  5.28s/it]
 64%|██████▍   | 144924/225000 [286:44:10<112:42:07,  5.07s/it]
 64%|██████▍   | 144925/225000 [286:44:15<109:14:00,  4.91s/it]
 64%|██████▍   | 144926/225000 [286:44:19<105:38:44,  4.75s/it]
 64%|██████▍   | 144927/225000 [286:44:30<148:44:20,  6.69s/it]
 64%|██████▍   | 144928/225000 [286:44:37<146:15:00,  6.58s/it]
 64%|██████▍   | 144929/225000 [286:44:41<131:09:55,  5.90s/it]
 64%|██████▍   | 144930/225000 [286:44:45<121:19:52,  5.46s/it]
 64%|██████▍   | 144931/225000 [286:44:50<114:12:44,  5.14s/it]
 64%|██████▍   | 144932/225000 [286:44:54<109:53:00,  4.94s/it]
 64%|██████▍   | 144933/225000 [286:45:02<131:30:06,  5.91s/it]
 64%|██████▍   | 144934/225000 [286:45:10<144:19:23,  6.49s/it]
 64%|██████▍   | 144935/225000 [286:45:14<128:25:48,  5.77s/it]
 64%|██████▍   | 144936/225000 [286:45:18<114:04:30,  5.13s/it]
 64%|██████▍   | 144937/225000 [286:45:24<119:37:40,  5.38s/it]
 64%|██████▍   | 144938/225000 [286:45:33<145:11:05,  6.53s/it]
 64%|██████▍   | 144939/225000 [286:45:37<124:40:06,  5.61s/it]
 64%|██████▍   | 144940/225000 [286:45:45<142:59:04,  6.43s/it]
 64%|██████▍   | 144941/225000 [286:45:49<126:12:41,  5.68s/it]
 64%|██████▍   | 144942/225000 [286:45:52<112:02:28,  5.04s/it]
 64%|██████▍   | 144943/225000 [286:45:57<112:28:48,  5.06s/it]
 64%|██████▍   | 144944/225000 [286:46:03<116:40:26,  5.25s/it]
 64%|██████▍   | 144945/225000 [286:46:08<114:24:10,  5.14s/it]
 64%|██████▍   | 144946/225000 [286:46:13<110:01:21,  4.95s/it]
 64%|██████▍   | 144947/225000 [286:46:18<114:17:09,  5.14s/it]
 64%|██████▍   | 144948/225000 [286:46:23<115:07:16,  5.18s/it]
 64%|██████▍   | 144949/225000 [286:46:28<113:55:38,  5.12s/it]
 64%|██████▍   | 144950/225000 [286:46:34<114:47:27,  5.16s/it]
                                                               
{'loss': '0.7863', 'grad_norm': '1.102', 'learning_rate': '1.526e-05', 'epoch': '0.6442', 'num_input_tokens_seen': 475354504, 'train_runtime': '1.032e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 144950/225000 [286:46:34<114:47:27,  5.16s/it]
 64%|██████▍   | 144951/225000 [286:46:45<155:31:25,  6.99s/it]
 64%|██████▍   | 144952/225000 [286:46:54<167:31:56,  7.53s/it]
 64%|██████▍   | 144953/225000 [286:46:59<152:45:22,  6.87s/it]
 64%|██████▍   | 144954/225000 [286:47:03<135:25:18,  6.09s/it]
 64%|██████▍   | 144955/225000 [286:47:08<122:53:52,  5.53s/it]
 64%|██████▍   | 144956/225000 [286:47:16<143:51:49,  6.47s/it]
 64%|██████▍   | 144957/225000 [286:47:20<123:40:36,  5.56s/it]
 64%|██████▍   | 144958/225000 [286:47:28<144:00:07,  6.48s/it]
 64%|██████▍   | 144959/225000 [286:47:33<133:00:03,  5.98s/it]
 64%|██████▍   | 144960/225000 [286:47:38<122:56:30,  5.53s/it]
 64%|██████▍   | 144961/225000 [286:47:42<118:24:04,  5.33s/it]
 64%|██████▍   | 144962/225000 [286:47:47<115:43:56,  5.21s/it]
 64%|██████▍   | 144963/225000 [286:47:52<113:33:57,  5.11s/it]
 64%|██████▍   | 144964/225000 [286:47:57<114:00:46,  5.13s/it]
 64%|██████▍   | 144965/225000 [286:48:03<114:32:22,  5.15s/it]
 64%|██████▍   | 144966/225000 [286:48:07<112:22:23,  5.05s/it]
 64%|██████▍   | 144967/225000 [286:48:12<109:39:01,  4.93s/it]
 64%|██████▍   | 144968/225000 [286:48:17<109:52:45,  4.94s/it]
 64%|██████▍   | 144969/225000 [286:48:22<110:03:12,  4.95s/it]
 64%|██████▍   | 144970/225000 [286:48:27<109:57:52,  4.95s/it]
 64%|██████▍   | 144971/225000 [286:48:38<150:19:12,  6.76s/it]
 64%|██████▍   | 144972/225000 [286:48:47<166:51:16,  7.51s/it]
 64%|██████▍   | 144973/225000 [286:48:51<141:58:47,  6.39s/it]
 64%|██████▍   | 144974/225000 [286:48:55<127:00:11,  5.71s/it]
 64%|██████▍   | 144975/225000 [286:49:00<124:47:52,  5.61s/it]
 64%|██████▍   | 144976/225000 [286:49:06<121:45:05,  5.48s/it]
 64%|██████▍   | 144977/225000 [286:49:11<120:38:47,  5.43s/it]
 64%|██████▍   | 144978/225000 [286:49:17<122:15:24,  5.50s/it]
 64%|██████▍   | 144979/225000 [286:49:22<119:46:07,  5.39s/it]
 64%|██████▍   | 144980/225000 [286:49:27<122:03:08,  5.49s/it]
 64%|██████▍   | 144981/225000 [286:49:33<122:29:50,  5.51s/it]
 64%|██████▍   | 144982/225000 [286:49:38<120:55:37,  5.44s/it]
 64%|██████▍   | 144983/225000 [286:49:43<117:06:38,  5.27s/it]
 64%|██████▍   | 144984/225000 [286:49:49<119:46:32,  5.39s/it]
 64%|██████▍   | 144985/225000 [286:49:54<115:49:33,  5.21s/it]
 64%|██████▍   | 144986/225000 [286:49:59<114:15:50,  5.14s/it]
 64%|██████▍   | 144987/225000 [286:50:04<115:58:17,  5.22s/it]
 64%|██████▍   | 144988/225000 [286:50:09<116:35:36,  5.25s/it]
 64%|██████▍   | 144989/225000 [286:50:18<136:43:23,  6.15s/it]
 64%|██████▍   | 144990/225000 [286:50:29<171:55:41,  7.74s/it]
 64%|██████▍   | 144991/225000 [286:50:33<146:53:30,  6.61s/it]
 64%|██████▍   | 144992/225000 [286:50:39<140:34:12,  6.33s/it]
 64%|██████▍   | 144993/225000 [286:50:45<141:41:47,  6.38s/it]
 64%|██████▍   | 144994/225000 [286:50:49<127:11:25,  5.72s/it]
 64%|██████▍   | 144995/225000 [286:50:54<121:57:58,  5.49s/it]
 64%|██████▍   | 144996/225000 [286:50:59<115:55:07,  5.22s/it]
 64%|██████▍   | 144997/225000 [286:51:04<114:27:17,  5.15s/it]
 64%|██████▍   | 144998/225000 [286:51:09<112:06:16,  5.04s/it]
 64%|██████▍   | 144999/225000 [286:51:14<110:54:26,  4.99s/it]
 64%|██████▍   | 145000/225000 [286:51:18<109:22:59,  4.92s/it]
                                                               
{'loss': '0.7965', 'grad_norm': '1.179', 'learning_rate': '1.524e-05', 'epoch': '0.6444', 'num_input_tokens_seen': 475484672, 'train_runtime': '1.033e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 145000/225000 [286:51:18<109:22:59,  4.92s/it]
 64%|██████▍   | 145001/225000 [286:51:25<118:48:54,  5.35s/it]
 64%|██████▍   | 145002/225000 [286:51:37<162:41:54,  7.32s/it]
 64%|██████▍   | 145003/225000 [286:51:42<150:13:57,  6.76s/it]
 64%|██████▍   | 145004/225000 [286:51:48<145:11:22,  6.53s/it]
 64%|██████▍   | 145005/225000 [286:51:53<134:17:16,  6.04s/it]
 64%|██████▍   | 145006/225000 [286:51:57<118:34:42,  5.34s/it]
 64%|██████▍   | 145007/225000 [286:52:04<134:14:30,  6.04s/it]
 64%|██████▍   | 145008/225000 [286:52:11<139:36:37,  6.28s/it]
 64%|██████▍   | 145009/225000 [286:52:15<120:52:20,  5.44s/it]
 64%|██████▍   | 145010/225000 [286:52:23<139:30:35,  6.28s/it]
 64%|██████▍   | 145011/225000 [286:52:26<120:02:46,  5.40s/it]
 64%|██████▍   | 145012/225000 [286:52:31<116:40:26,  5.25s/it]
 64%|██████▍   | 145013/225000 [286:52:36<115:13:33,  5.19s/it]
 64%|██████▍   | 145014/225000 [286:52:41<113:11:14,  5.09s/it]
 64%|██████▍   | 145015/225000 [286:52:46<114:36:55,  5.16s/it]
 64%|██████▍   | 145016/225000 [286:52:51<112:10:37,  5.05s/it]
 64%|██████▍   | 145017/225000 [286:52:56<113:32:44,  5.11s/it]
 64%|██████▍   | 145018/225000 [286:53:02<115:05:04,  5.18s/it]
 64%|██████▍   | 145019/225000 [286:53:07<114:54:14,  5.17s/it]
 64%|██████▍   | 145020/225000 [286:53:18<155:11:59,  6.99s/it]
 64%|██████▍   | 145021/225000 [286:53:25<153:26:00,  6.91s/it]
 64%|██████▍   | 145022/225000 [286:53:29<132:08:14,  5.95s/it]
 64%|██████▍   | 145023/225000 [286:53:32<116:50:27,  5.26s/it]
 64%|██████▍   | 145024/225000 [286:53:36<107:14:23,  4.83s/it]
 64%|██████▍   | 145025/225000 [286:53:47<149:44:30,  6.74s/it]
 64%|██████▍   | 145026/225000 [286:53:51<128:23:59,  5.78s/it]
 64%|██████▍   | 145027/225000 [286:53:57<133:30:13,  6.01s/it]
 64%|██████▍   | 145028/225000 [286:54:02<126:45:44,  5.71s/it]
 64%|██████▍   | 145029/225000 [286:54:07<118:40:48,  5.34s/it]
 64%|██████▍   | 145030/225000 [286:54:12<116:24:36,  5.24s/it]
 64%|██████▍   | 145031/225000 [286:54:16<111:10:55,  5.01s/it]
 64%|██████▍   | 145032/225000 [286:54:21<109:41:40,  4.94s/it]
 64%|██████▍   | 145033/225000 [286:54:26<108:17:41,  4.88s/it]
 64%|██████▍   | 145034/225000 [286:54:31<107:56:13,  4.86s/it]
 64%|██████▍   | 145035/225000 [286:54:35<108:10:07,  4.87s/it]
 64%|██████▍   | 145036/225000 [286:54:41<111:01:19,  5.00s/it]
 64%|██████▍   | 145037/225000 [286:54:46<114:37:11,  5.16s/it]
 64%|██████▍   | 145038/225000 [286:54:57<148:49:45,  6.70s/it]
 64%|██████▍   | 145039/225000 [286:55:02<139:04:43,  6.26s/it]
 64%|██████▍   | 145040/225000 [286:55:10<148:42:38,  6.70s/it]
 64%|██████▍   | 145041/225000 [286:55:14<134:25:08,  6.05s/it]
 64%|██████▍   | 145042/225000 [286:55:19<124:39:11,  5.61s/it]
 64%|██████▍   | 145043/225000 [286:55:23<117:09:24,  5.27s/it]
 64%|██████▍   | 145044/225000 [286:55:28<111:39:07,  5.03s/it]
 64%|██████▍   | 145045/225000 [286:55:32<109:18:58,  4.92s/it]
 64%|██████▍   | 145046/225000 [286:55:42<138:13:43,  6.22s/it]
 64%|██████▍   | 145047/225000 [286:55:50<151:54:26,  6.84s/it]
 64%|██████▍   | 145048/225000 [286:55:53<127:26:46,  5.74s/it]
 64%|██████▍   | 145049/225000 [286:56:00<136:47:35,  6.16s/it]
 64%|██████▍   | 145050/225000 [286:56:04<118:26:47,  5.33s/it]
                                                               
{'loss': '0.803', 'grad_norm': '1.077', 'learning_rate': '1.523e-05', 'epoch': '0.6447', 'num_input_tokens_seen': 475615184, 'train_runtime': '1.033e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 145050/225000 [286:56:04<118:26:47,  5.33s/it]
 64%|██████▍   | 145051/225000 [286:56:12<139:23:27,  6.28s/it]
 64%|██████▍   | 145052/225000 [286:56:15<119:56:22,  5.40s/it]
 64%|██████▍   | 145053/225000 [286:56:19<110:07:42,  4.96s/it]
 64%|██████▍   | 145054/225000 [286:56:27<131:04:28,  5.90s/it]
 64%|██████▍   | 145055/225000 [286:56:34<132:32:55,  5.97s/it]
 64%|██████▍   | 145056/225000 [286:56:40<133:37:51,  6.02s/it]
 64%|██████▍   | 145057/225000 [286:56:44<123:15:13,  5.55s/it]
 64%|██████▍   | 145058/225000 [286:56:50<126:24:45,  5.69s/it]
 64%|██████▍   | 145059/225000 [286:56:57<135:27:57,  6.10s/it]
 64%|██████▍   | 145060/225000 [286:57:01<118:16:39,  5.33s/it]
 64%|██████▍   | 145061/225000 [286:57:08<129:32:15,  5.83s/it]
 64%|██████▍   | 145062/225000 [286:57:14<129:47:20,  5.85s/it]
 64%|██████▍   | 145063/225000 [286:57:19<124:01:49,  5.59s/it]
 64%|██████▍   | 145064/225000 [286:57:23<113:24:36,  5.11s/it]
 64%|██████▍   | 145065/225000 [286:57:27<109:01:30,  4.91s/it]
 64%|██████▍   | 145066/225000 [286:57:32<107:00:23,  4.82s/it]
 64%|██████▍   | 145067/225000 [286:57:36<106:02:18,  4.78s/it]
 64%|██████▍   | 145068/225000 [286:57:47<143:43:20,  6.47s/it]
 64%|██████▍   | 145069/225000 [286:57:53<139:13:46,  6.27s/it]
 64%|██████▍   | 145070/225000 [286:57:56<121:36:40,  5.48s/it]
 64%|██████▍   | 145071/225000 [286:58:01<120:13:28,  5.41s/it]
 64%|██████▍   | 145072/225000 [286:58:07<119:01:48,  5.36s/it]
 64%|██████▍   | 145073/225000 [286:58:11<114:46:08,  5.17s/it]
 64%|██████▍   | 145074/225000 [286:58:16<112:08:07,  5.05s/it]
 64%|██████▍   | 145075/225000 [286:58:22<115:58:48,  5.22s/it]
 64%|██████▍   | 145076/225000 [286:58:27<118:41:47,  5.35s/it]
 64%|██████▍   | 145077/225000 [286:58:32<116:44:02,  5.26s/it]
 64%|██████▍   | 145078/225000 [286:58:38<117:36:45,  5.30s/it]
 64%|██████▍   | 145079/225000 [286:58:43<117:37:07,  5.30s/it]
 64%|██████▍   | 145080/225000 [286:58:49<119:14:19,  5.37s/it]
 64%|██████▍   | 145081/225000 [286:58:54<118:17:20,  5.33s/it]
 64%|██████▍   | 145082/225000 [286:58:59<114:28:11,  5.16s/it]
 64%|██████▍   | 145083/225000 [286:59:04<114:00:42,  5.14s/it]
 64%|██████▍   | 145084/225000 [286:59:09<114:42:49,  5.17s/it]
 64%|██████▍   | 145085/225000 [286:59:17<136:08:49,  6.13s/it]
 64%|██████▍   | 145086/225000 [286:59:30<176:19:51,  7.94s/it]
 64%|██████▍   | 145087/225000 [286:59:33<145:44:01,  6.57s/it]
 64%|██████▍   | 145088/225000 [286:59:39<140:18:20,  6.32s/it]
 64%|██████▍   | 145089/225000 [286:59:45<140:30:28,  6.33s/it]
 64%|██████▍   | 145090/225000 [286:59:50<128:16:09,  5.78s/it]
 64%|██████▍   | 145091/225000 [286:59:54<118:39:18,  5.35s/it]
 64%|██████▍   | 145092/225000 [286:59:58<113:10:51,  5.10s/it]
 64%|██████▍   | 145093/225000 [287:00:03<108:16:50,  4.88s/it]
 64%|██████▍   | 145094/225000 [287:00:14<148:45:37,  6.70s/it]
 64%|██████▍   | 145095/225000 [287:00:21<150:03:11,  6.76s/it]
 64%|██████▍   | 145096/225000 [287:00:25<135:19:56,  6.10s/it]
 64%|██████▍   | 145097/225000 [287:00:30<127:45:40,  5.76s/it]
 64%|██████▍   | 145098/225000 [287:00:35<121:53:12,  5.49s/it]
 64%|██████▍   | 145099/225000 [287:00:40<118:34:37,  5.34s/it]
 64%|██████▍   | 145100/225000 [287:00:45<116:25:08,  5.25s/it]
                                                               
{'loss': '0.7814', 'grad_norm': '1.135', 'learning_rate': '1.521e-05', 'epoch': '0.6449', 'num_input_tokens_seen': 475745472, 'train_runtime': '1.033e+06', 'train_tokens_per_second': '460.4'}

 64%|██████▍   | 145100/225000 [287:00:45<116:25:08,  5.25s/it]
 64%|██████▍   | 145101/225000 [287:00:50<115:10:11,  5.19s/it]
 64%|██████▍   | 145102/225000 [287:00:55<113:40:30,  5.12s/it]
 64%|██████▍   | 145103/225000 [287:01:00<113:26:36,  5.11s/it]
 64%|██████▍   | 145104/225000 [287:01:10<142:59:53,  6.44s/it]
 64%|██████▍   | 145105/225000 [287:01:19<165:38:59,  7.46s/it]
 64%|██████▍   | 145106/225000 [287:01:23<137:44:44,  6.21s/it]
 64%|██████▍   | 145107/225000 [287:01:28<132:23:45,  5.97s/it]
 64%|██████▍   | 145108/225000 [287:01:35<138:47:16,  6.25s/it]
 64%|██████▍   | 145109/225000 [287:01:39<120:33:50,  5.43s/it]
 64%|██████▍   | 145110/225000 [287:01:42<108:46:01,  4.90s/it]
 64%|██████▍   | 145111/225000 [287:01:53<149:39:47,  6.74s/it]
 64%|██████▍   | 145112/225000 [287:01:57<129:55:20,  5.85s/it]
 64%|██████▍   | 145113/225000 [287:02:04<135:18:16,  6.10s/it]
 64%|██████▍   | 145114/225000 [287:02:08<120:34:47,  5.43s/it]
 64%|██████▍   | 145115/225000 [287:02:12<112:39:16,  5.08s/it]
 64%|██████▍   | 145116/225000 [287:02:19<125:41:21,  5.66s/it]
 64%|██████▍   | 145117/225000 [287:02:24<122:22:18,  5.51s/it]
 64%|██████▍   | 145118/225000 [287:02:29<121:06:28,  5.46s/it]
 64%|██████▍   | 145119/225000 [287:02:34<118:03:45,  5.32s/it]
 64%|██████▍   | 145120/225000 [287:02:40<117:33:00,  5.30s/it]
 64%|██████▍   | 145121/225000 [287:02:45<115:49:26,  5.22s/it]
 64%|██████▍   | 145122/225000 [287:02:50<114:18:38,  5.15s/it]
 64%|██████▍   | 145123/225000 [287:02:55<113:58:12,  5.14s/it]
 64%|██████▍   | 145124/225000 [287:02:59<110:18:28,  4.97s/it]
 64%|██████▍   | 145125/225000 [287:03:05<114:37:04,  5.17s/it]
 65%|██████▍   | 145126/225000 [287:03:10<113:57:52,  5.14s/it]
 65%|██████▍   | 145127/225000 [287:03:15<114:04:04,  5.14s/it]
 65%|██████▍   | 145128/225000 [287:03:20<113:11:48,  5.10s/it]
 65%|██████▍   | 145129/225000 [287:03:28<131:08:09,  5.91s/it]
 65%|██████▍   | 145130/225000 [287:03:38<160:07:22,  7.22s/it]
 65%|██████▍   | 145131/225000 [287:03:42<135:02:38,  6.09s/it]
 65%|██████▍   | 145132/225000 [287:03:52<161:51:55,  7.30s/it]
 65%|██████▍   | 145133/225000 [287:03:55<136:27:41,  6.15s/it]
 65%|██████▍   | 145134/225000 [287:04:02<137:59:06,  6.22s/it]
 65%|██████▍   | 145135/225000 [287:04:07<130:26:34,  5.88s/it]
 65%|██████▍   | 145136/225000 [287:04:11<122:29:54,  5.52s/it]
 65%|██████▍   | 145137/225000 [287:04:16<118:09:19,  5.33s/it]
 65%|██████▍   | 145138/225000 [287:04:21<115:29:30,  5.21s/it]
 65%|██████▍   | 145139/225000 [287:04:26<114:54:37,  5.18s/it]
 65%|██████▍   | 145140/225000 [287:04:31<114:08:07,  5.15s/it]
 65%|██████▍   | 145141/225000 [287:04:37<113:37:36,  5.12s/it]
 65%|██████▍   | 145142/225000 [287:04:41<111:31:05,  5.03s/it]
 65%|██████▍   | 145143/225000 [287:04:47<112:25:07,  5.07s/it]
 65%|██████▍   | 145144/225000 [287:04:51<107:29:35,  4.85s/it]
 65%|██████▍   | 145145/225000 [287:04:56<110:01:43,  4.96s/it]
 65%|██████▍   | 145146/225000 [287:05:01<113:09:07,  5.10s/it]
 65%|██████▍   | 145147/225000 [287:05:07<113:10:42,  5.10s/it]
 65%|██████▍   | 145148/225000 [287:05:12<116:41:57,  5.26s/it]
 65%|██████▍   | 145149/225000 [287:05:21<138:29:29,  6.24s/it]
 65%|██████▍   | 145150/225000 [287:05:28<148:01:17,  6.67s/it]
                                                               
{'loss': '0.7941', 'grad_norm': '1.133', 'learning_rate': '1.519e-05', 'epoch': '0.6451', 'num_input_tokens_seen': 475874536, 'train_runtime': '1.034e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145150/225000 [287:05:28<148:01:17,  6.67s/it]
 65%|██████▍   | 145151/225000 [287:05:32<128:03:05,  5.77s/it]
 65%|██████▍   | 145152/225000 [287:05:38<129:44:06,  5.85s/it]
 65%|██████▍   | 145153/225000 [287:05:46<142:14:32,  6.41s/it]
 65%|██████▍   | 145154/225000 [287:05:49<119:28:19,  5.39s/it]
 65%|██████▍   | 145155/225000 [287:05:57<139:41:32,  6.30s/it]
 65%|██████▍   | 145156/225000 [287:06:03<135:29:53,  6.11s/it]
 65%|██████▍   | 145157/225000 [287:06:09<132:46:42,  5.99s/it]
 65%|██████▍   | 145158/225000 [287:06:13<121:12:42,  5.47s/it]
 65%|██████▍   | 145159/225000 [287:06:17<111:49:15,  5.04s/it]
 65%|██████▍   | 145160/225000 [287:06:23<117:00:10,  5.28s/it]
 65%|██████▍   | 145161/225000 [287:06:32<145:09:44,  6.55s/it]
 65%|██████▍   | 145162/225000 [287:06:36<124:32:43,  5.62s/it]
 65%|██████▍   | 145163/225000 [287:06:39<109:08:14,  4.92s/it]
 65%|██████▍   | 145164/225000 [287:06:47<128:08:34,  5.78s/it]
 65%|██████▍   | 145165/225000 [287:06:51<120:28:39,  5.43s/it]
 65%|██████▍   | 145166/225000 [287:06:56<115:29:35,  5.21s/it]
 65%|██████▍   | 145167/225000 [287:07:01<111:27:27,  5.03s/it]
 65%|██████▍   | 145168/225000 [287:07:05<108:39:09,  4.90s/it]
 65%|██████▍   | 145169/225000 [287:07:10<109:40:47,  4.95s/it]
 65%|██████▍   | 145170/225000 [287:07:15<110:27:47,  4.98s/it]
 65%|██████▍   | 145171/225000 [287:07:22<124:05:59,  5.60s/it]
 65%|██████▍   | 145172/225000 [287:07:35<168:42:24,  7.61s/it]
 65%|██████▍   | 145173/225000 [287:07:38<140:16:11,  6.33s/it]
 65%|██████▍   | 145174/225000 [287:07:45<146:16:13,  6.60s/it]
 65%|██████▍   | 145175/225000 [287:07:50<130:08:00,  5.87s/it]
 65%|██████▍   | 145176/225000 [287:07:53<116:27:37,  5.25s/it]
 65%|██████▍   | 145177/225000 [287:07:57<107:48:23,  4.86s/it]
 65%|██████▍   | 145178/225000 [287:08:05<128:16:12,  5.79s/it]
 65%|██████▍   | 145179/225000 [287:08:13<138:53:45,  6.26s/it]
 65%|██████▍   | 145180/225000 [287:08:18<133:50:14,  6.04s/it]
 65%|██████▍   | 145181/225000 [287:08:24<135:03:10,  6.09s/it]
 65%|██████▍   | 145182/225000 [287:08:28<118:13:26,  5.33s/it]
 65%|██████▍   | 145183/225000 [287:08:34<121:20:25,  5.47s/it]
 65%|██████▍   | 145184/225000 [287:08:41<135:41:32,  6.12s/it]
 65%|██████▍   | 145185/225000 [287:08:48<141:52:59,  6.40s/it]
 65%|██████▍   | 145186/225000 [287:08:52<120:27:49,  5.43s/it]
 65%|██████▍   | 145187/225000 [287:08:58<128:06:31,  5.78s/it]
 65%|██████▍   | 145188/225000 [287:09:03<124:21:54,  5.61s/it]
 65%|██████▍   | 145189/225000 [287:09:08<120:16:46,  5.43s/it]
 65%|██████▍   | 145190/225000 [287:09:13<117:12:30,  5.29s/it]
 65%|██████▍   | 145191/225000 [287:09:18<114:53:13,  5.18s/it]
 65%|██████▍   | 145192/225000 [287:09:23<112:00:27,  5.05s/it]
 65%|██████▍   | 145193/225000 [287:09:28<110:56:59,  5.00s/it]
 65%|██████▍   | 145194/225000 [287:09:33<109:19:22,  4.93s/it]
 65%|██████▍   | 145195/225000 [287:09:37<105:35:23,  4.76s/it]
 65%|██████▍   | 145196/225000 [287:09:43<114:32:57,  5.17s/it]
 65%|██████▍   | 145197/225000 [287:09:54<153:31:55,  6.93s/it]
 65%|██████▍   | 145198/225000 [287:09:57<128:40:53,  5.81s/it]
 65%|██████▍   | 145199/225000 [287:10:01<115:29:42,  5.21s/it]
 65%|██████▍   | 145200/225000 [287:10:11<145:36:54,  6.57s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '1.086', 'learning_rate': '1.518e-05', 'epoch': '0.6453', 'num_input_tokens_seen': 476004152, 'train_runtime': '1.034e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145200/225000 [287:10:11<145:36:54,  6.57s/it]
 65%|██████▍   | 145201/225000 [287:10:16<132:51:58,  5.99s/it]
 65%|██████▍   | 145202/225000 [287:10:19<115:48:19,  5.22s/it]
 65%|██████▍   | 145203/225000 [287:10:27<136:24:13,  6.15s/it]
 65%|██████▍   | 145204/225000 [287:10:31<121:09:06,  5.47s/it]
 65%|██████▍   | 145205/225000 [287:10:38<132:29:19,  5.98s/it]
 65%|██████▍   | 145206/225000 [287:10:42<114:23:58,  5.16s/it]
 65%|██████▍   | 145207/225000 [287:10:49<128:32:43,  5.80s/it]
 65%|██████▍   | 145208/225000 [287:10:53<118:00:23,  5.32s/it]
 65%|██████▍   | 145209/225000 [287:10:57<110:47:19,  5.00s/it]
 65%|██████▍   | 145210/225000 [287:11:02<107:34:26,  4.85s/it]
 65%|██████▍   | 145211/225000 [287:11:07<108:09:09,  4.88s/it]
 65%|██████▍   | 145212/225000 [287:11:18<151:34:18,  6.84s/it]
 65%|██████▍   | 145213/225000 [287:11:21<125:51:05,  5.68s/it]
 65%|██████▍   | 145214/225000 [287:11:29<138:57:14,  6.27s/it]
 65%|██████▍   | 145215/225000 [287:11:32<120:04:49,  5.42s/it]
 65%|██████▍   | 145216/225000 [287:11:37<116:43:55,  5.27s/it]
 65%|██████▍   | 145217/225000 [287:11:42<111:17:05,  5.02s/it]
 65%|██████▍   | 145218/225000 [287:11:46<110:08:27,  4.97s/it]
 65%|██████▍   | 145219/225000 [287:11:51<107:16:28,  4.84s/it]
 65%|██████▍   | 145220/225000 [287:11:56<108:05:50,  4.88s/it]
 65%|██████▍   | 145221/225000 [287:12:00<105:18:44,  4.75s/it]
 65%|██████▍   | 145222/225000 [287:12:13<156:40:47,  7.07s/it]
 65%|██████▍   | 145223/225000 [287:12:16<132:04:27,  5.96s/it]
 65%|██████▍   | 145224/225000 [287:12:21<120:55:55,  5.46s/it]
 65%|██████▍   | 145225/225000 [287:12:25<113:20:57,  5.12s/it]
 65%|██████▍   | 145226/225000 [287:12:33<136:08:24,  6.14s/it]
 65%|██████▍   | 145227/225000 [287:12:41<147:44:59,  6.67s/it]
 65%|██████▍   | 145228/225000 [287:12:46<133:50:46,  6.04s/it]
 65%|██████▍   | 145229/225000 [287:12:51<125:30:39,  5.66s/it]
 65%|██████▍   | 145230/225000 [287:12:55<118:50:14,  5.36s/it]
 65%|██████▍   | 145231/225000 [287:13:00<114:56:55,  5.19s/it]
 65%|██████▍   | 145232/225000 [287:13:05<111:04:38,  5.01s/it]
 65%|██████▍   | 145233/225000 [287:13:10<110:33:20,  4.99s/it]
 65%|██████▍   | 145234/225000 [287:13:14<107:39:27,  4.86s/it]
 65%|██████▍   | 145235/225000 [287:13:20<110:46:58,  5.00s/it]
 65%|██████▍   | 145236/225000 [287:13:31<154:22:51,  6.97s/it]
 65%|██████▍   | 145237/225000 [287:13:37<149:54:21,  6.77s/it]
 65%|██████▍   | 145238/225000 [287:13:41<131:49:00,  5.95s/it]
 65%|██████▍   | 145239/225000 [287:13:46<119:16:42,  5.38s/it]
 65%|██████▍   | 145240/225000 [287:13:53<130:33:12,  5.89s/it]
 65%|██████▍   | 145241/225000 [287:14:01<145:19:38,  6.56s/it]
 65%|██████▍   | 145242/225000 [287:14:04<123:55:48,  5.59s/it]
 65%|██████▍   | 145243/225000 [287:14:08<111:52:13,  5.05s/it]
 65%|██████▍   | 145244/225000 [287:14:12<107:30:18,  4.85s/it]
 65%|██████▍   | 145245/225000 [287:14:23<146:55:16,  6.63s/it]
 65%|██████▍   | 145246/225000 [287:14:28<133:34:52,  6.03s/it]
 65%|██████▍   | 145247/225000 [287:14:35<140:39:01,  6.35s/it]
 65%|██████▍   | 145248/225000 [287:14:39<125:05:53,  5.65s/it]
 65%|██████▍   | 145249/225000 [287:14:46<133:21:12,  6.02s/it]
 65%|██████▍   | 145250/225000 [287:14:49<116:05:05,  5.24s/it]
                                                               
{'loss': '0.7837', 'grad_norm': '1.266', 'learning_rate': '1.516e-05', 'epoch': '0.6456', 'num_input_tokens_seen': 476133776, 'train_runtime': '1.034e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145250/225000 [287:14:49<116:05:05,  5.24s/it]
 65%|██████▍   | 145251/225000 [287:14:55<122:00:59,  5.51s/it]
 65%|██████▍   | 145252/225000 [287:15:01<126:39:50,  5.72s/it]
 65%|██████▍   | 145253/225000 [287:15:06<116:02:04,  5.24s/it]
 65%|██████▍   | 145254/225000 [287:15:10<113:34:44,  5.13s/it]
 65%|██████▍   | 145255/225000 [287:15:14<106:54:57,  4.83s/it]
 65%|██████▍   | 145256/225000 [287:15:19<106:09:32,  4.79s/it]
 65%|██████▍   | 145257/225000 [287:15:24<105:57:43,  4.78s/it]
 65%|██████▍   | 145258/225000 [287:15:29<105:25:24,  4.76s/it]
 65%|██████▍   | 145259/225000 [287:15:35<117:11:38,  5.29s/it]
 65%|██████▍   | 145260/225000 [287:15:47<162:39:10,  7.34s/it]
 65%|██████▍   | 145261/225000 [287:15:51<137:57:38,  6.23s/it]
 65%|██████▍   | 145262/225000 [287:15:55<125:48:06,  5.68s/it]
 65%|██████▍   | 145263/225000 [287:16:02<133:41:24,  6.04s/it]
 65%|██████▍   | 145264/225000 [287:16:07<122:50:34,  5.55s/it]
 65%|██████▍   | 145265/225000 [287:16:11<114:47:40,  5.18s/it]
 65%|██████▍   | 145266/225000 [287:16:15<107:40:52,  4.86s/it]
 65%|██████▍   | 145267/225000 [287:16:22<121:52:21,  5.50s/it]
 65%|██████▍   | 145268/225000 [287:16:30<139:00:38,  6.28s/it]
 65%|██████▍   | 145269/225000 [287:16:35<129:34:09,  5.85s/it]
 65%|██████▍   | 145270/225000 [287:16:40<123:49:41,  5.59s/it]
 65%|██████▍   | 145271/225000 [287:16:45<119:56:52,  5.42s/it]
 65%|██████▍   | 145272/225000 [287:16:50<117:52:36,  5.32s/it]
 65%|██████▍   | 145273/225000 [287:16:55<112:47:00,  5.09s/it]
 65%|██████▍   | 145274/225000 [287:16:59<107:55:34,  4.87s/it]
 65%|██████▍   | 145275/225000 [287:17:04<110:52:33,  5.01s/it]
 65%|██████▍   | 145276/225000 [287:17:09<110:06:54,  4.97s/it]
 65%|██████▍   | 145277/225000 [287:17:14<110:20:23,  4.98s/it]
 65%|██████▍   | 145278/225000 [287:17:19<110:01:10,  4.97s/it]
 65%|██████▍   | 145279/225000 [287:17:32<164:01:33,  7.41s/it]
 65%|██████▍   | 145280/225000 [287:17:37<147:17:13,  6.65s/it]
 65%|██████▍   | 145281/225000 [287:17:45<152:29:00,  6.89s/it]
 65%|██████▍   | 145282/225000 [287:17:49<137:59:48,  6.23s/it]
 65%|██████▍   | 145283/225000 [287:17:53<122:24:02,  5.53s/it]
 65%|██████▍   | 145284/225000 [287:18:02<143:27:03,  6.48s/it]
 65%|██████▍   | 145285/225000 [287:18:05<123:28:47,  5.58s/it]
 65%|██████▍   | 145286/225000 [287:18:13<137:19:11,  6.20s/it]
 65%|██████▍   | 145287/225000 [287:18:18<126:02:30,  5.69s/it]
 65%|██████▍   | 145288/225000 [287:18:23<123:02:01,  5.56s/it]
 65%|██████▍   | 145289/225000 [287:18:30<132:14:01,  5.97s/it]
 65%|██████▍   | 145290/225000 [287:18:33<114:08:23,  5.15s/it]
 65%|██████▍   | 145291/225000 [287:18:37<105:56:46,  4.78s/it]
 65%|██████▍   | 145292/225000 [287:18:42<110:47:30,  5.00s/it]
 65%|██████▍   | 145293/225000 [287:18:52<143:46:12,  6.49s/it]
 65%|██████▍   | 145294/225000 [287:18:56<127:49:01,  5.77s/it]
 65%|██████▍   | 145295/225000 [287:19:00<115:42:12,  5.23s/it]
 65%|██████▍   | 145296/225000 [287:19:11<148:16:41,  6.70s/it]
 65%|██████▍   | 145297/225000 [287:19:14<128:28:06,  5.80s/it]
 65%|██████▍   | 145298/225000 [287:19:22<142:35:45,  6.44s/it]
 65%|██████▍   | 145299/225000 [287:19:26<122:40:00,  5.54s/it]
 65%|██████▍   | 145300/225000 [287:19:33<137:02:09,  6.19s/it]
                                                               
{'loss': '0.7832', 'grad_norm': '1.268', 'learning_rate': '1.514e-05', 'epoch': '0.6458', 'num_input_tokens_seen': 476263552, 'train_runtime': '1.034e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145300/225000 [287:19:33<137:02:09,  6.19s/it]
 65%|██████▍   | 145301/225000 [287:19:37<119:40:40,  5.41s/it]
 65%|██████▍   | 145302/225000 [287:19:41<109:32:49,  4.95s/it]
 65%|██████▍   | 145303/225000 [287:19:51<142:17:15,  6.43s/it]
 65%|██████▍   | 145304/225000 [287:19:54<124:16:09,  5.61s/it]
 65%|██████▍   | 145305/225000 [287:19:58<113:39:47,  5.13s/it]
 65%|██████▍   | 145306/225000 [287:20:06<129:17:26,  5.84s/it]
 65%|██████▍   | 145307/225000 [287:20:10<115:38:36,  5.22s/it]
 65%|██████▍   | 145308/225000 [287:20:14<107:10:27,  4.84s/it]
 65%|██████▍   | 145309/225000 [287:20:23<135:58:36,  6.14s/it]
 65%|██████▍   | 145310/225000 [287:20:27<120:25:21,  5.44s/it]
 65%|██████▍   | 145311/225000 [287:20:31<113:01:50,  5.11s/it]
 65%|██████▍   | 145312/225000 [287:20:35<109:03:44,  4.93s/it]
 65%|██████▍   | 145313/225000 [287:20:46<145:19:46,  6.57s/it]
 65%|██████▍   | 145314/225000 [287:20:53<147:09:27,  6.65s/it]
 65%|██████▍   | 145315/225000 [287:20:56<127:10:26,  5.75s/it]
 65%|██████▍   | 145316/225000 [287:21:04<143:09:36,  6.47s/it]
 65%|██████▍   | 145317/225000 [287:21:08<126:40:56,  5.72s/it]
 65%|██████▍   | 145318/225000 [287:21:12<114:05:24,  5.15s/it]
 65%|██████▍   | 145319/225000 [287:21:21<135:51:02,  6.14s/it]
 65%|██████▍   | 145320/225000 [287:21:25<121:35:54,  5.49s/it]
 65%|██████▍   | 145321/225000 [287:21:29<113:22:10,  5.12s/it]
 65%|██████▍   | 145322/225000 [287:21:33<106:01:29,  4.79s/it]
 65%|██████▍   | 145323/225000 [287:21:37<103:54:44,  4.70s/it]
 65%|██████▍   | 145324/225000 [287:21:47<136:50:14,  6.18s/it]
 65%|██████▍   | 145325/225000 [287:21:51<120:04:56,  5.43s/it]
 65%|██████▍   | 145326/225000 [287:21:55<109:43:54,  4.96s/it]
 65%|██████▍   | 145327/225000 [287:21:59<105:00:48,  4.74s/it]
 65%|██████▍   | 145328/225000 [287:22:09<141:45:25,  6.41s/it]
 65%|██████▍   | 145329/225000 [287:22:13<126:21:53,  5.71s/it]
 65%|██████▍   | 145330/225000 [287:22:18<117:20:53,  5.30s/it]
 65%|██████▍   | 145331/225000 [287:22:25<132:59:08,  6.01s/it]
 65%|██████▍   | 145332/225000 [287:22:29<117:04:18,  5.29s/it]
 65%|██████▍   | 145333/225000 [287:22:34<118:15:20,  5.34s/it]
 65%|██████▍   | 145334/225000 [287:22:43<137:57:48,  6.23s/it]
 65%|██████▍   | 145335/225000 [287:22:47<122:23:00,  5.53s/it]
 65%|██████▍   | 145336/225000 [287:22:51<115:12:16,  5.21s/it]
 65%|██████▍   | 145337/225000 [287:22:56<111:24:55,  5.03s/it]
 65%|██████▍   | 145338/225000 [287:23:00<106:28:27,  4.81s/it]
 65%|██████▍   | 145339/225000 [287:23:11<146:50:51,  6.64s/it]
 65%|██████▍   | 145340/225000 [287:23:17<144:53:35,  6.55s/it]
 65%|██████▍   | 145341/225000 [287:23:20<123:00:07,  5.56s/it]
 65%|██████▍   | 145342/225000 [287:23:29<143:58:15,  6.51s/it]
 65%|██████▍   | 145343/225000 [287:23:33<124:47:39,  5.64s/it]
 65%|██████▍   | 145344/225000 [287:23:38<121:00:07,  5.47s/it]
 65%|██████▍   | 145345/225000 [287:23:43<116:53:59,  5.28s/it]
 65%|██████▍   | 145346/225000 [287:23:48<114:36:36,  5.18s/it]
 65%|██████▍   | 145347/225000 [287:23:53<114:58:36,  5.20s/it]
 65%|██████▍   | 145348/225000 [287:23:58<114:45:32,  5.19s/it]
 65%|██████▍   | 145349/225000 [287:24:03<113:52:12,  5.15s/it]
 65%|██████▍   | 145350/225000 [287:24:08<112:43:42,  5.10s/it]
                                                               
{'loss': '0.779', 'grad_norm': '1.073', 'learning_rate': '1.513e-05', 'epoch': '0.646', 'num_input_tokens_seen': 476394400, 'train_runtime': '1.035e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145350/225000 [287:24:08<112:43:42,  5.10s/it]
 65%|██████▍   | 145351/225000 [287:24:16<131:53:05,  5.96s/it]
 65%|██████▍   | 145352/225000 [287:24:27<162:08:00,  7.33s/it]
 65%|██████▍   | 145353/225000 [287:24:30<137:57:22,  6.24s/it]
 65%|██████▍   | 145354/225000 [287:24:35<128:47:46,  5.82s/it]
 65%|██████▍   | 145355/225000 [287:24:43<145:59:00,  6.60s/it]
 65%|██████▍   | 145356/225000 [287:24:47<125:18:02,  5.66s/it]
 65%|██████▍   | 145357/225000 [287:24:50<110:27:32,  4.99s/it]
 65%|██████▍   | 145358/225000 [287:25:00<143:24:09,  6.48s/it]
 65%|██████▍   | 145359/225000 [287:25:04<125:39:45,  5.68s/it]
 65%|██████▍   | 145360/225000 [287:25:08<112:48:27,  5.10s/it]
 65%|██████▍   | 145361/225000 [287:25:18<148:59:49,  6.74s/it]
 65%|██████▍   | 145362/225000 [287:25:22<125:18:13,  5.66s/it]
 65%|██████▍   | 145363/225000 [287:25:28<130:57:04,  5.92s/it]
 65%|██████▍   | 145364/225000 [287:25:33<125:00:57,  5.65s/it]
 65%|██████▍   | 145365/225000 [287:25:37<112:48:49,  5.10s/it]
 65%|██████▍   | 145366/225000 [287:25:44<128:14:41,  5.80s/it]
 65%|██████▍   | 145367/225000 [287:25:50<129:35:37,  5.86s/it]
 65%|██████▍   | 145368/225000 [287:25:54<115:14:23,  5.21s/it]
 65%|██████▍   | 145369/225000 [287:26:02<130:58:29,  5.92s/it]
 65%|██████▍   | 145370/225000 [287:26:06<117:57:31,  5.33s/it]
 65%|██████▍   | 145371/225000 [287:26:10<113:32:41,  5.13s/it]
 65%|██████▍   | 145372/225000 [287:26:15<112:15:48,  5.08s/it]
 65%|██████▍   | 145373/225000 [287:26:20<108:44:48,  4.92s/it]
 65%|██████▍   | 145374/225000 [287:26:25<108:46:09,  4.92s/it]
 65%|██████▍   | 145375/225000 [287:26:30<109:09:27,  4.94s/it]
 65%|██████▍   | 145376/225000 [287:26:35<110:01:49,  4.97s/it]
 65%|██████▍   | 145377/225000 [287:26:40<111:58:49,  5.06s/it]
 65%|██████▍   | 145378/225000 [287:26:52<161:01:58,  7.28s/it]
 65%|██████▍   | 145379/225000 [287:26:57<143:00:14,  6.47s/it]
 65%|██████▍   | 145380/225000 [287:27:02<130:28:34,  5.90s/it]
 65%|██████▍   | 145381/225000 [287:27:06<119:06:21,  5.39s/it]
 65%|██████▍   | 145382/225000 [287:27:10<112:30:42,  5.09s/it]
 65%|██████▍   | 145383/225000 [287:27:15<108:09:15,  4.89s/it]
 65%|██████▍   | 145384/225000 [287:27:23<129:45:59,  5.87s/it]
 65%|██████▍   | 145385/225000 [287:27:30<141:06:51,  6.38s/it]
 65%|██████▍   | 145386/225000 [287:27:34<125:15:52,  5.66s/it]
 65%|██████▍   | 145387/225000 [287:27:39<117:07:16,  5.30s/it]
 65%|██████▍   | 145388/225000 [287:27:44<116:09:50,  5.25s/it]
 65%|██████▍   | 145389/225000 [287:27:49<115:00:19,  5.20s/it]
 65%|██████▍   | 145390/225000 [287:27:54<116:14:55,  5.26s/it]
 65%|██████▍   | 145391/225000 [287:27:59<115:04:21,  5.20s/it]
 65%|██████▍   | 145392/225000 [287:28:05<115:39:07,  5.23s/it]
 65%|██████▍   | 145393/225000 [287:28:10<116:24:42,  5.26s/it]
 65%|██████▍   | 145394/225000 [287:28:20<150:27:30,  6.80s/it]
 65%|██████▍   | 145395/225000 [287:28:30<166:57:01,  7.55s/it]
 65%|██████▍   | 145396/225000 [287:28:35<149:02:59,  6.74s/it]
 65%|██████▍   | 145397/225000 [287:28:39<135:36:14,  6.13s/it]
 65%|██████▍   | 145398/225000 [287:28:44<122:56:37,  5.56s/it]
 65%|██████▍   | 145399/225000 [287:28:48<116:56:27,  5.29s/it]
 65%|██████▍   | 145400/225000 [287:28:53<112:04:44,  5.07s/it]
                                                               
{'loss': '0.7727', 'grad_norm': '1.065', 'learning_rate': '1.511e-05', 'epoch': '0.6462', 'num_input_tokens_seen': 476523672, 'train_runtime': '1.035e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145400/225000 [287:28:53<112:04:44,  5.07s/it]
 65%|██████▍   | 145401/225000 [287:29:02<139:14:32,  6.30s/it]
 65%|██████▍   | 145402/225000 [287:29:10<152:44:23,  6.91s/it]
 65%|██████▍   | 145403/225000 [287:29:14<128:26:04,  5.81s/it]
 65%|██████▍   | 145404/225000 [287:29:19<127:58:52,  5.79s/it]
 65%|██████▍   | 145405/225000 [287:29:26<134:34:22,  6.09s/it]
 65%|██████▍   | 145406/225000 [287:29:31<127:31:23,  5.77s/it]
 65%|██████▍   | 145407/225000 [287:29:40<146:09:01,  6.61s/it]
 65%|██████▍   | 145408/225000 [287:29:43<126:22:19,  5.72s/it]
 65%|██████▍   | 145409/225000 [287:29:47<113:34:59,  5.14s/it]
 65%|██████▍   | 145410/225000 [287:29:54<125:12:32,  5.66s/it]
 65%|██████▍   | 145411/225000 [287:30:00<128:31:59,  5.81s/it]
 65%|██████▍   | 145412/225000 [287:30:08<143:33:23,  6.49s/it]
 65%|██████▍   | 145413/225000 [287:30:12<123:21:21,  5.58s/it]
 65%|██████▍   | 145414/225000 [287:30:15<109:38:25,  4.96s/it]
 65%|██████▍   | 145415/225000 [287:30:24<134:22:01,  6.08s/it]
 65%|██████▍   | 145416/225000 [287:30:28<118:24:50,  5.36s/it]
 65%|██████▍   | 145417/225000 [287:30:33<120:16:02,  5.44s/it]
 65%|██████▍   | 145418/225000 [287:30:39<121:49:49,  5.51s/it]
 65%|██████▍   | 145419/225000 [287:30:43<113:21:57,  5.13s/it]
 65%|██████▍   | 145420/225000 [287:30:52<138:32:25,  6.27s/it]
 65%|██████▍   | 145421/225000 [287:30:57<133:16:21,  6.03s/it]
 65%|██████▍   | 145422/225000 [287:31:04<138:26:18,  6.26s/it]
 65%|██████▍   | 145423/225000 [287:31:08<121:32:16,  5.50s/it]
 65%|██████▍   | 145424/225000 [287:31:12<111:06:21,  5.03s/it]
 65%|██████▍   | 145425/225000 [287:31:16<104:15:42,  4.72s/it]
 65%|██████▍   | 145426/225000 [287:31:22<116:11:14,  5.26s/it]
 65%|██████▍   | 145427/225000 [287:31:31<138:40:44,  6.27s/it]
 65%|██████▍   | 145428/225000 [287:31:35<121:26:21,  5.49s/it]
 65%|██████▍   | 145429/225000 [287:31:44<147:23:42,  6.67s/it]
 65%|██████▍   | 145430/225000 [287:31:48<131:27:02,  5.95s/it]
 65%|██████▍   | 145431/225000 [287:31:52<118:30:50,  5.36s/it]
 65%|██████▍   | 145432/225000 [287:31:57<111:17:07,  5.04s/it]
 65%|██████▍   | 145433/225000 [287:32:01<107:36:41,  4.87s/it]
 65%|██████▍   | 145434/225000 [287:32:07<116:44:06,  5.28s/it]
 65%|██████▍   | 145435/225000 [287:32:18<153:23:40,  6.94s/it]
 65%|██████▍   | 145436/225000 [287:32:22<132:11:47,  5.98s/it]
 65%|██████▍   | 145437/225000 [287:32:26<118:16:06,  5.35s/it]
 65%|██████▍   | 145438/225000 [287:32:30<110:54:36,  5.02s/it]
 65%|██████▍   | 145439/225000 [287:32:40<143:38:01,  6.50s/it]
 65%|██████▍   | 145440/225000 [287:32:44<129:25:19,  5.86s/it]
 65%|██████▍   | 145441/225000 [287:32:49<118:17:08,  5.35s/it]
 65%|██████▍   | 145442/225000 [287:32:53<109:08:37,  4.94s/it]
 65%|██████▍   | 145443/225000 [287:32:57<106:17:45,  4.81s/it]
 65%|██████▍   | 145444/225000 [287:33:03<110:36:45,  5.01s/it]
 65%|██████▍   | 145445/225000 [287:33:15<159:10:16,  7.20s/it]
 65%|██████▍   | 145446/225000 [287:33:19<139:50:14,  6.33s/it]
 65%|██████▍   | 145447/225000 [287:33:23<125:38:36,  5.69s/it]
 65%|██████▍   | 145448/225000 [287:33:28<116:59:28,  5.29s/it]
 65%|██████▍   | 145449/225000 [287:33:33<119:10:45,  5.39s/it]
 65%|██████▍   | 145450/225000 [287:33:42<139:04:13,  6.29s/it]
                                                               
{'loss': '0.7854', 'grad_norm': '1.058', 'learning_rate': '1.509e-05', 'epoch': '0.6464', 'num_input_tokens_seen': 476654664, 'train_runtime': '1.035e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145450/225000 [287:33:42<139:04:13,  6.29s/it]
 65%|██████▍   | 145451/225000 [287:33:49<148:22:31,  6.71s/it]
 65%|██████▍   | 145452/225000 [287:33:53<130:29:01,  5.91s/it]
 65%|██████▍   | 145453/225000 [287:34:00<133:19:57,  6.03s/it]
 65%|██████▍   | 145454/225000 [287:34:03<115:02:57,  5.21s/it]
 65%|██████▍   | 145455/225000 [287:34:12<137:13:08,  6.21s/it]
 65%|██████▍   | 145456/225000 [287:34:16<123:40:02,  5.60s/it]
 65%|██████▍   | 145457/225000 [287:34:20<114:27:34,  5.18s/it]
 65%|██████▍   | 145458/225000 [287:34:24<106:44:42,  4.83s/it]
 65%|██████▍   | 145459/225000 [287:34:32<126:57:57,  5.75s/it]
 65%|██████▍   | 145460/225000 [287:34:38<132:37:29,  6.00s/it]
 65%|██████▍   | 145461/225000 [287:34:46<143:16:08,  6.48s/it]
 65%|██████▍   | 145462/225000 [287:34:50<128:11:48,  5.80s/it]
 65%|██████▍   | 145463/225000 [287:34:56<124:29:43,  5.63s/it]
 65%|██████▍   | 145464/225000 [287:35:03<139:48:33,  6.33s/it]
 65%|██████▍   | 145465/225000 [287:35:08<124:47:48,  5.65s/it]
 65%|██████▍   | 145466/225000 [287:35:11<111:32:26,  5.05s/it]
 65%|██████▍   | 145467/225000 [287:35:15<103:56:01,  4.70s/it]
 65%|██████▍   | 145468/225000 [287:35:25<135:06:26,  6.12s/it]
 65%|██████▍   | 145469/225000 [287:35:30<133:20:15,  6.04s/it]
 65%|██████▍   | 145470/225000 [287:35:35<120:54:05,  5.47s/it]
 65%|██████▍   | 145471/225000 [287:35:39<112:02:59,  5.07s/it]
 65%|██████▍   | 145472/225000 [287:35:43<105:14:40,  4.76s/it]
 65%|██████▍   | 145473/225000 [287:35:50<120:41:47,  5.46s/it]
 65%|██████▍   | 145474/225000 [287:36:00<153:37:17,  6.95s/it]
 65%|██████▍   | 145475/225000 [287:36:04<130:37:37,  5.91s/it]
 65%|██████▍   | 145476/225000 [287:36:10<136:11:48,  6.17s/it]
 65%|██████▍   | 145477/225000 [287:36:17<141:06:39,  6.39s/it]
 65%|██████▍   | 145478/225000 [287:36:22<130:36:21,  5.91s/it]
 65%|██████▍   | 145479/225000 [287:36:27<123:41:40,  5.60s/it]
 65%|██████▍   | 145480/225000 [287:36:32<118:49:28,  5.38s/it]
 65%|██████▍   | 145481/225000 [287:36:37<116:39:06,  5.28s/it]
 65%|██████▍   | 145482/225000 [287:36:42<114:22:53,  5.18s/it]
 65%|██████▍   | 145483/225000 [287:36:47<115:06:39,  5.21s/it]
 65%|██████▍   | 145484/225000 [287:36:52<112:37:58,  5.10s/it]
 65%|██████▍   | 145485/225000 [287:36:57<112:34:38,  5.10s/it]
 65%|██████▍   | 145486/225000 [287:37:02<109:01:32,  4.94s/it]
 65%|██████▍   | 145487/225000 [287:37:07<109:50:43,  4.97s/it]
 65%|██████▍   | 145488/225000 [287:37:12<109:49:54,  4.97s/it]
 65%|██████▍   | 145489/225000 [287:37:17<110:40:38,  5.01s/it]
 65%|██████▍   | 145490/225000 [287:37:29<159:08:07,  7.21s/it]
 65%|██████▍   | 145491/225000 [287:37:34<143:34:15,  6.50s/it]
 65%|██████▍   | 145492/225000 [287:37:39<131:13:47,  5.94s/it]
 65%|██████▍   | 145493/225000 [287:37:43<123:52:16,  5.61s/it]
 65%|██████▍   | 145494/225000 [287:37:49<120:24:18,  5.45s/it]
 65%|██████▍   | 145495/225000 [287:37:54<118:37:43,  5.37s/it]
 65%|██████▍   | 145496/225000 [287:37:58<112:35:47,  5.10s/it]
 65%|██████▍   | 145497/225000 [287:38:03<110:29:44,  5.00s/it]
 65%|██████▍   | 145498/225000 [287:38:08<109:52:44,  4.98s/it]
 65%|██████▍   | 145499/225000 [287:38:16<128:50:53,  5.83s/it]
 65%|██████▍   | 145500/225000 [287:38:26<156:20:15,  7.08s/it]
                                                               
{'loss': '0.7934', 'grad_norm': '1.005', 'learning_rate': '1.508e-05', 'epoch': '0.6467', 'num_input_tokens_seen': 476784112, 'train_runtime': '1.036e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145500/225000 [287:38:26<156:20:15,  7.08s/it]
 65%|██████▍   | 145501/225000 [287:38:30<138:06:06,  6.25s/it]
 65%|██████▍   | 145502/225000 [287:38:35<127:55:48,  5.79s/it]
 65%|██████▍   | 145503/225000 [287:38:39<119:11:15,  5.40s/it]
 65%|██████▍   | 145504/225000 [287:38:44<113:31:21,  5.14s/it]
 65%|██████▍   | 145505/225000 [287:38:52<130:48:15,  5.92s/it]
 65%|██████▍   | 145506/225000 [287:39:01<154:16:39,  6.99s/it]
 65%|██████▍   | 145507/225000 [287:39:05<132:28:40,  6.00s/it]
 65%|██████▍   | 145508/225000 [287:39:09<121:01:17,  5.48s/it]
 65%|██████▍   | 145509/225000 [287:39:13<108:37:38,  4.92s/it]
 65%|██████▍   | 145510/225000 [287:39:23<146:17:13,  6.63s/it]
 65%|██████▍   | 145511/225000 [287:39:27<128:47:56,  5.83s/it]
 65%|██████▍   | 145512/225000 [287:39:31<117:24:39,  5.32s/it]
 65%|██████▍   | 145513/225000 [287:39:35<109:01:03,  4.94s/it]
 65%|██████▍   | 145514/225000 [287:39:46<148:48:26,  6.74s/it]
 65%|██████▍   | 145515/225000 [287:39:50<127:00:05,  5.75s/it]
 65%|██████▍   | 145516/225000 [287:39:54<116:13:13,  5.26s/it]
 65%|██████▍   | 145517/225000 [287:40:02<132:55:50,  6.02s/it]
 65%|██████▍   | 145518/225000 [287:40:05<117:17:13,  5.31s/it]
 65%|██████▍   | 145519/225000 [287:40:09<108:09:43,  4.90s/it]
 65%|██████▍   | 145520/225000 [287:40:18<132:20:39,  5.99s/it]
 65%|██████▍   | 145521/225000 [287:40:23<129:11:21,  5.85s/it]
 65%|██████▍   | 145522/225000 [287:40:27<114:44:06,  5.20s/it]
 65%|██████▍   | 145523/225000 [287:40:32<111:24:55,  5.05s/it]
 65%|██████▍   | 145524/225000 [287:40:41<140:51:04,  6.38s/it]
 65%|██████▍   | 145525/225000 [287:40:45<122:54:48,  5.57s/it]
 65%|██████▍   | 145526/225000 [287:40:51<126:29:24,  5.73s/it]
 65%|██████▍   | 145527/225000 [287:40:58<132:18:51,  5.99s/it]
 65%|██████▍   | 145528/225000 [287:41:01<114:08:24,  5.17s/it]
 65%|██████▍   | 145529/225000 [287:41:08<128:55:40,  5.84s/it]
 65%|██████▍   | 145530/225000 [287:41:13<121:54:46,  5.52s/it]
 65%|██████▍   | 145531/225000 [287:41:19<123:06:41,  5.58s/it]
 65%|██████▍   | 145532/225000 [287:41:27<140:39:16,  6.37s/it]
 65%|██████▍   | 145533/225000 [287:41:31<125:10:19,  5.67s/it]
 65%|██████▍   | 145534/225000 [287:41:35<113:23:00,  5.14s/it]
 65%|██████▍   | 145535/225000 [287:41:44<139:38:45,  6.33s/it]
 65%|██████▍   | 145536/225000 [287:41:47<117:11:52,  5.31s/it]
 65%|██████▍   | 145537/225000 [287:41:55<134:19:10,  6.09s/it]
 65%|██████▍   | 145538/225000 [287:41:59<123:21:45,  5.59s/it]
 65%|██████▍   | 145539/225000 [287:42:04<115:21:00,  5.23s/it]
 65%|██████▍   | 145540/225000 [287:42:08<107:26:31,  4.87s/it]
 65%|██████▍   | 145541/225000 [287:42:12<101:50:56,  4.61s/it]
 65%|██████▍   | 145542/225000 [287:42:22<141:37:02,  6.42s/it]
 65%|██████▍   | 145543/225000 [287:42:27<127:21:29,  5.77s/it]
 65%|██████▍   | 145544/225000 [287:42:31<120:26:43,  5.46s/it]
 65%|██████▍   | 145545/225000 [287:42:37<119:36:14,  5.42s/it]
 65%|██████▍   | 145546/225000 [287:42:42<119:23:21,  5.41s/it]
 65%|██████▍   | 145547/225000 [287:42:47<117:35:13,  5.33s/it]
 65%|██████▍   | 145548/225000 [287:42:53<119:47:11,  5.43s/it]
 65%|██████▍   | 145549/225000 [287:43:00<129:12:13,  5.85s/it]
 65%|██████▍   | 145550/225000 [287:43:04<120:05:53,  5.44s/it]
                                                               
{'loss': '0.7949', 'grad_norm': '1.008', 'learning_rate': '1.506e-05', 'epoch': '0.6469', 'num_input_tokens_seen': 476913176, 'train_runtime': '1.036e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145550/225000 [287:43:04<120:05:53,  5.44s/it]
 65%|██████▍   | 145551/225000 [287:43:09<114:01:00,  5.17s/it]
 65%|██████▍   | 145552/225000 [287:43:13<110:37:32,  5.01s/it]
 65%|██████▍   | 145553/225000 [287:43:25<156:45:53,  7.10s/it]
 65%|██████▍   | 145554/225000 [287:43:29<135:22:45,  6.13s/it]
 65%|██████▍   | 145555/225000 [287:43:33<119:18:53,  5.41s/it]
 65%|██████▍   | 145556/225000 [287:43:37<108:08:26,  4.90s/it]
 65%|██████▍   | 145557/225000 [287:43:46<139:20:48,  6.31s/it]
 65%|██████▍   | 145558/225000 [287:43:52<134:44:54,  6.11s/it]
 65%|██████▍   | 145559/225000 [287:43:58<134:10:10,  6.08s/it]
 65%|██████▍   | 145560/225000 [287:44:01<118:08:54,  5.35s/it]
 65%|██████▍   | 145561/225000 [287:44:11<147:59:59,  6.71s/it]
 65%|██████▍   | 145562/225000 [287:44:15<129:48:05,  5.88s/it]
 65%|██████▍   | 145563/225000 [287:44:20<121:42:38,  5.52s/it]
 65%|██████▍   | 145564/225000 [287:44:24<114:46:11,  5.20s/it]
 65%|██████▍   | 145565/225000 [287:44:29<108:40:36,  4.93s/it]
 65%|██████▍   | 145566/225000 [287:44:38<137:06:52,  6.21s/it]
 65%|██████▍   | 145567/225000 [287:44:41<117:11:17,  5.31s/it]
 65%|██████▍   | 145568/225000 [287:44:48<130:50:54,  5.93s/it]
 65%|██████▍   | 145569/225000 [287:44:52<114:41:46,  5.20s/it]
 65%|██████▍   | 145570/225000 [287:45:01<138:30:36,  6.28s/it]
 65%|██████▍   | 145571/225000 [287:45:04<119:36:16,  5.42s/it]
 65%|██████▍   | 145572/225000 [287:45:08<108:19:33,  4.91s/it]
 65%|██████▍   | 145573/225000 [287:45:12<105:12:34,  4.77s/it]
 65%|██████▍   | 145574/225000 [287:45:17<104:45:01,  4.75s/it]
 65%|██████▍   | 145575/225000 [287:45:22<105:42:08,  4.79s/it]
 65%|██████▍   | 145576/225000 [287:45:33<148:53:39,  6.75s/it]
 65%|██████▍   | 145577/225000 [287:45:40<150:22:57,  6.82s/it]
 65%|██████▍   | 145578/225000 [287:45:44<129:27:01,  5.87s/it]
 65%|██████▍   | 145579/225000 [287:45:51<140:58:40,  6.39s/it]
 65%|██████▍   | 145580/225000 [287:45:55<122:43:51,  5.56s/it]
 65%|██████▍   | 145581/225000 [287:45:59<110:30:32,  5.01s/it]
 65%|██████▍   | 145582/225000 [287:46:11<156:04:25,  7.07s/it]
 65%|██████▍   | 145583/225000 [287:46:14<129:49:27,  5.88s/it]
 65%|██████▍   | 145584/225000 [287:46:18<119:05:01,  5.40s/it]
 65%|██████▍   | 145585/225000 [287:46:23<115:58:39,  5.26s/it]
 65%|██████▍   | 145586/225000 [287:46:28<112:13:00,  5.09s/it]
 65%|██████▍   | 145587/225000 [287:46:40<159:06:03,  7.21s/it]
 65%|██████▍   | 145588/225000 [287:46:44<135:16:32,  6.13s/it]
 65%|██████▍   | 145589/225000 [287:46:47<116:21:43,  5.28s/it]
 65%|██████▍   | 145590/225000 [287:46:54<132:19:39,  6.00s/it]
 65%|██████▍   | 145591/225000 [287:47:01<138:15:44,  6.27s/it]
 65%|██████▍   | 145592/225000 [287:47:06<125:22:58,  5.68s/it]
 65%|██████▍   | 145593/225000 [287:47:10<115:46:38,  5.25s/it]
 65%|██████▍   | 145594/225000 [287:47:14<110:20:40,  5.00s/it]
 65%|██████▍   | 145595/225000 [287:47:19<108:15:01,  4.91s/it]
 65%|██████▍   | 145596/225000 [287:47:29<142:31:45,  6.46s/it]
 65%|██████▍   | 145597/225000 [287:47:36<147:42:28,  6.70s/it]
 65%|██████▍   | 145598/225000 [287:47:40<127:43:09,  5.79s/it]
 65%|██████▍   | 145599/225000 [287:47:45<119:33:00,  5.42s/it]
 65%|██████▍   | 145600/225000 [287:47:49<114:00:05,  5.17s/it]
                                                               
{'loss': '0.7879', 'grad_norm': '1.047', 'learning_rate': '1.504e-05', 'epoch': '0.6471', 'num_input_tokens_seen': 477042544, 'train_runtime': '1.036e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145600/225000 [287:47:49<114:00:05,  5.17s/it]
 65%|██████▍   | 145601/225000 [287:47:54<112:21:09,  5.09s/it]
 65%|██████▍   | 145602/225000 [287:47:59<110:11:21,  5.00s/it]
 65%|██████▍   | 145603/225000 [287:48:12<161:13:39,  7.31s/it]
 65%|██████▍   | 145604/225000 [287:48:15<136:16:05,  6.18s/it]
 65%|██████▍   | 145605/225000 [287:48:22<142:08:12,  6.44s/it]
 65%|██████▍   | 145606/225000 [287:48:27<134:09:32,  6.08s/it]
 65%|██████▍   | 145607/225000 [287:48:32<127:00:34,  5.76s/it]
 65%|██████▍   | 145608/225000 [287:48:38<123:45:45,  5.61s/it]
 65%|██████▍   | 145609/225000 [287:48:42<117:35:32,  5.33s/it]
 65%|██████▍   | 145610/225000 [287:48:48<116:36:42,  5.29s/it]
 65%|██████▍   | 145611/225000 [287:48:53<117:28:17,  5.33s/it]
 65%|██████▍   | 145612/225000 [287:48:58<113:43:39,  5.16s/it]
 65%|██████▍   | 145613/225000 [287:49:03<114:01:21,  5.17s/it]
 65%|██████▍   | 145614/225000 [287:49:08<115:20:03,  5.23s/it]
 65%|██████▍   | 145615/225000 [287:49:13<112:47:35,  5.12s/it]
 65%|██████▍   | 145616/225000 [287:49:18<111:27:11,  5.05s/it]
 65%|██████▍   | 145617/225000 [287:49:23<110:31:42,  5.01s/it]
 65%|██████▍   | 145618/225000 [287:49:28<112:56:45,  5.12s/it]
 65%|██████▍   | 145619/225000 [287:49:34<113:58:40,  5.17s/it]
 65%|██████▍   | 145620/225000 [287:49:43<141:15:51,  6.41s/it]
 65%|██████▍   | 145621/225000 [287:49:53<165:18:53,  7.50s/it]
 65%|██████▍   | 145622/225000 [287:49:57<139:23:57,  6.32s/it]
 65%|██████▍   | 145623/225000 [287:50:01<126:01:28,  5.72s/it]
 65%|██████▍   | 145624/225000 [287:50:06<120:07:09,  5.45s/it]
 65%|██████▍   | 145625/225000 [287:50:17<159:01:59,  7.21s/it]
 65%|██████▍   | 145626/225000 [287:50:22<143:06:28,  6.49s/it]
 65%|██████▍   | 145627/225000 [287:50:27<133:17:11,  6.05s/it]
 65%|██████▍   | 145628/225000 [287:50:31<120:57:28,  5.49s/it]
 65%|██████▍   | 145629/225000 [287:50:35<113:50:46,  5.16s/it]
 65%|██████▍   | 145630/225000 [287:50:39<106:25:57,  4.83s/it]
 65%|██████▍   | 145631/225000 [287:50:52<157:34:04,  7.15s/it]
 65%|██████▍   | 145632/225000 [287:50:56<138:00:20,  6.26s/it]
 65%|██████▍   | 145633/225000 [287:51:01<130:01:30,  5.90s/it]
 65%|██████▍   | 145634/225000 [287:51:09<142:00:35,  6.44s/it]
 65%|██████▍   | 145635/225000 [287:51:14<131:52:06,  5.98s/it]
 65%|██████▍   | 145636/225000 [287:51:22<146:41:59,  6.65s/it]
 65%|██████▍   | 145637/225000 [287:51:26<129:07:34,  5.86s/it]
 65%|██████▍   | 145638/225000 [287:51:34<140:21:28,  6.37s/it]
 65%|██████▍   | 145639/225000 [287:51:38<129:20:29,  5.87s/it]
 65%|██████▍   | 145640/225000 [287:51:43<121:03:31,  5.49s/it]
 65%|██████▍   | 145641/225000 [287:51:47<114:33:44,  5.20s/it]
 65%|██████▍   | 145642/225000 [287:51:52<108:14:28,  4.91s/it]
 65%|██████▍   | 145643/225000 [287:51:56<105:59:25,  4.81s/it]
 65%|██████▍   | 145644/225000 [287:52:01<103:58:35,  4.72s/it]
 65%|██████▍   | 145645/225000 [287:52:05<102:25:15,  4.65s/it]
 65%|██████▍   | 145646/225000 [287:52:12<118:53:59,  5.39s/it]
 65%|██████▍   | 145647/225000 [287:52:22<147:34:29,  6.70s/it]
 65%|██████▍   | 145648/225000 [287:52:27<132:40:59,  6.02s/it]
 65%|██████▍   | 145649/225000 [287:52:34<144:18:30,  6.55s/it]
 65%|██████▍   | 145650/225000 [287:52:38<122:40:21,  5.57s/it]
                                                               
{'loss': '0.7908', 'grad_norm': '1.084', 'learning_rate': '1.503e-05', 'epoch': '0.6473', 'num_input_tokens_seen': 477172456, 'train_runtime': '1.036e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145650/225000 [287:52:38<122:40:21,  5.57s/it]
 65%|██████▍   | 145651/225000 [287:52:45<132:16:00,  6.00s/it]
 65%|██████▍   | 145652/225000 [287:52:49<122:10:32,  5.54s/it]
 65%|██████▍   | 145653/225000 [287:52:57<139:49:20,  6.34s/it]
 65%|██████▍   | 145654/225000 [287:53:02<129:23:15,  5.87s/it]
 65%|██████▍   | 145655/225000 [287:53:07<121:59:54,  5.54s/it]
 65%|██████▍   | 145656/225000 [287:53:12<117:52:43,  5.35s/it]
 65%|██████▍   | 145657/225000 [287:53:16<113:21:41,  5.14s/it]
 65%|██████▍   | 145658/225000 [287:53:21<107:00:25,  4.86s/it]
 65%|██████▍   | 145659/225000 [287:53:26<108:08:00,  4.91s/it]
 65%|██████▍   | 145660/225000 [287:53:30<107:40:01,  4.89s/it]
 65%|██████▍   | 145661/225000 [287:53:36<108:55:41,  4.94s/it]
 65%|██████▍   | 145662/225000 [287:53:40<108:41:46,  4.93s/it]
 65%|██████▍   | 145663/225000 [287:53:53<161:12:47,  7.32s/it]
 65%|██████▍   | 145664/225000 [287:53:58<146:41:37,  6.66s/it]
 65%|██████▍   | 145665/225000 [287:54:05<147:03:22,  6.67s/it]
 65%|██████▍   | 145666/225000 [287:54:09<127:44:17,  5.80s/it]
 65%|██████▍   | 145667/225000 [287:54:13<113:26:08,  5.15s/it]
 65%|██████▍   | 145668/225000 [287:54:18<113:09:46,  5.14s/it]
 65%|██████▍   | 145669/225000 [287:54:26<133:20:58,  6.05s/it]
 65%|██████▍   | 145670/225000 [287:54:30<120:37:14,  5.47s/it]
 65%|██████▍   | 145671/225000 [287:54:34<114:07:53,  5.18s/it]
 65%|██████▍   | 145672/225000 [287:54:39<110:37:10,  5.02s/it]
 65%|██████▍   | 145673/225000 [287:54:44<107:27:13,  4.88s/it]
 65%|██████▍   | 145674/225000 [287:54:48<106:00:46,  4.81s/it]
 65%|██████▍   | 145675/225000 [287:54:55<120:54:31,  5.49s/it]
 65%|██████▍   | 145676/225000 [287:55:06<154:36:12,  7.02s/it]
 65%|██████▍   | 145677/225000 [287:55:09<130:05:25,  5.90s/it]
 65%|██████▍   | 145678/225000 [287:55:17<145:15:14,  6.59s/it]
 65%|██████▍   | 145679/225000 [287:55:22<131:55:04,  5.99s/it]
 65%|██████▍   | 145680/225000 [287:55:28<128:44:16,  5.84s/it]
 65%|██████▍   | 145681/225000 [287:55:34<133:37:27,  6.06s/it]
 65%|██████▍   | 145682/225000 [287:55:38<122:05:00,  5.54s/it]
 65%|██████▍   | 145683/225000 [287:55:42<111:28:45,  5.06s/it]
 65%|██████▍   | 145684/225000 [287:55:47<105:16:17,  4.78s/it]
 65%|██████▍   | 145685/225000 [287:55:51<102:14:50,  4.64s/it]
 65%|██████▍   | 145686/225000 [287:55:59<124:58:14,  5.67s/it]
 65%|██████▍   | 145687/225000 [287:56:06<134:55:02,  6.12s/it]
 65%|██████▍   | 145688/225000 [287:56:10<119:59:15,  5.45s/it]
 65%|██████▍   | 145689/225000 [287:56:15<116:47:46,  5.30s/it]
 65%|██████▍   | 145690/225000 [287:56:20<116:18:40,  5.28s/it]
 65%|██████▍   | 145691/225000 [287:56:25<115:40:55,  5.25s/it]
 65%|██████▍   | 145692/225000 [287:56:31<116:48:06,  5.30s/it]
 65%|██████▍   | 145693/225000 [287:56:36<113:41:44,  5.16s/it]
 65%|██████▍   | 145694/225000 [287:56:41<114:42:46,  5.21s/it]
 65%|██████▍   | 145695/225000 [287:56:50<137:03:06,  6.22s/it]
 65%|██████▍   | 145696/225000 [287:56:59<156:22:27,  7.10s/it]
 65%|██████▍   | 145697/225000 [287:57:03<137:30:36,  6.24s/it]
 65%|██████▍   | 145698/225000 [287:57:07<126:32:32,  5.74s/it]
 65%|██████▍   | 145699/225000 [287:57:12<118:16:55,  5.37s/it]
 65%|██████▍   | 145700/225000 [287:57:16<111:19:18,  5.05s/it]
                                                               
{'loss': '0.7851', 'grad_norm': '1.064', 'learning_rate': '1.501e-05', 'epoch': '0.6476', 'num_input_tokens_seen': 477303368, 'train_runtime': '1.037e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145700/225000 [287:57:16<111:19:18,  5.05s/it]
 65%|██████▍   | 145701/225000 [287:57:21<106:59:27,  4.86s/it]
 65%|██████▍   | 145702/225000 [287:57:25<105:12:55,  4.78s/it]
 65%|██████▍   | 145703/225000 [287:57:30<102:41:01,  4.66s/it]
 65%|██████▍   | 145704/225000 [287:57:36<112:40:19,  5.12s/it]
 65%|██████▍   | 145705/225000 [287:57:48<157:02:53,  7.13s/it]
 65%|██████▍   | 145706/225000 [287:57:51<133:02:31,  6.04s/it]
 65%|██████▍   | 145707/225000 [287:57:55<119:44:18,  5.44s/it]
 65%|██████▍   | 145708/225000 [287:58:03<138:25:25,  6.28s/it]
 65%|██████▍   | 145709/225000 [287:58:11<149:39:27,  6.79s/it]
 65%|██████▍   | 145710/225000 [287:58:17<140:49:20,  6.39s/it]
 65%|██████▍   | 145711/225000 [287:58:21<127:55:03,  5.81s/it]
 65%|██████▍   | 145712/225000 [287:58:28<135:27:07,  6.15s/it]
 65%|██████▍   | 145713/225000 [287:58:32<116:46:27,  5.30s/it]
 65%|██████▍   | 145714/225000 [287:58:39<128:10:58,  5.82s/it]
 65%|██████▍   | 145715/225000 [287:58:44<126:39:36,  5.75s/it]
 65%|██████▍   | 145716/225000 [287:58:51<133:41:30,  6.07s/it]
 65%|██████▍   | 145717/225000 [287:58:55<122:34:27,  5.57s/it]
 65%|██████▍   | 145718/225000 [287:59:00<113:52:11,  5.17s/it]
 65%|██████▍   | 145719/225000 [287:59:04<108:51:37,  4.94s/it]
 65%|██████▍   | 145720/225000 [287:59:09<106:59:34,  4.86s/it]
 65%|██████▍   | 145721/225000 [287:59:13<105:12:25,  4.78s/it]
 65%|██████▍   | 145722/225000 [287:59:18<104:58:04,  4.77s/it]
 65%|██████▍   | 145723/225000 [287:59:23<104:59:40,  4.77s/it]
 65%|██████▍   | 145724/225000 [287:59:35<152:19:52,  6.92s/it]
 65%|██████▍   | 145725/225000 [287:59:39<137:28:38,  6.24s/it]
 65%|██████▍   | 145726/225000 [287:59:46<139:19:15,  6.33s/it]
 65%|██████▍   | 145727/225000 [287:59:50<125:37:11,  5.70s/it]
 65%|██████▍   | 145728/225000 [287:59:58<138:40:46,  6.30s/it]
 65%|██████▍   | 145729/225000 [288:00:01<118:21:35,  5.38s/it]
 65%|██████▍   | 145730/225000 [288:00:09<137:24:43,  6.24s/it]
 65%|██████▍   | 145731/225000 [288:00:13<118:13:55,  5.37s/it]
 65%|██████▍   | 145732/225000 [288:00:19<124:02:15,  5.63s/it]
 65%|██████▍   | 145733/225000 [288:00:25<128:30:11,  5.84s/it]
 65%|██████▍   | 145734/225000 [288:00:29<115:39:21,  5.25s/it]
 65%|██████▍   | 145735/225000 [288:00:33<107:29:33,  4.88s/it]
 65%|██████▍   | 145736/225000 [288:00:38<108:29:25,  4.93s/it]
 65%|██████▍   | 145737/225000 [288:00:44<110:56:19,  5.04s/it]
 65%|██████▍   | 145738/225000 [288:00:49<111:57:54,  5.09s/it]
 65%|██████▍   | 145739/225000 [288:00:54<110:43:44,  5.03s/it]
 65%|██████▍   | 145740/225000 [288:00:59<112:25:56,  5.11s/it]
 65%|██████▍   | 145741/225000 [288:01:04<114:39:39,  5.21s/it]
 65%|██████▍   | 145742/225000 [288:01:10<116:03:37,  5.27s/it]
 65%|██████▍   | 145743/225000 [288:01:17<130:09:10,  5.91s/it]
 65%|██████▍   | 145744/225000 [288:01:29<166:38:30,  7.57s/it]
 65%|██████▍   | 145745/225000 [288:01:33<145:51:18,  6.63s/it]
 65%|██████▍   | 145746/225000 [288:01:37<127:18:39,  5.78s/it]
 65%|██████▍   | 145747/225000 [288:01:46<148:44:53,  6.76s/it]
 65%|██████▍   | 145748/225000 [288:01:49<125:36:55,  5.71s/it]
 65%|██████▍   | 145749/225000 [288:01:56<133:29:18,  6.06s/it]
 65%|██████▍   | 145750/225000 [288:02:03<136:29:13,  6.20s/it]
                                                               
{'loss': '0.7878', 'grad_norm': '1.052', 'learning_rate': '1.499e-05', 'epoch': '0.6478', 'num_input_tokens_seen': 477433352, 'train_runtime': '1.037e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145750/225000 [288:02:03<136:29:13,  6.20s/it]
 65%|██████▍   | 145751/225000 [288:02:07<122:00:10,  5.54s/it]
 65%|██████▍   | 145752/225000 [288:02:10<110:59:52,  5.04s/it]
 65%|██████▍   | 145753/225000 [288:02:18<127:59:29,  5.81s/it]
 65%|██████▍   | 145754/225000 [288:02:23<125:04:38,  5.68s/it]
 65%|██████▍   | 145755/225000 [288:02:28<117:37:45,  5.34s/it]
 65%|██████▍   | 145756/225000 [288:02:32<111:12:06,  5.05s/it]
 65%|██████▍   | 145757/225000 [288:02:37<109:46:23,  4.99s/it]
 65%|██████▍   | 145758/225000 [288:02:42<110:11:29,  5.01s/it]
 65%|██████▍   | 145759/225000 [288:02:47<106:45:53,  4.85s/it]
 65%|██████▍   | 145760/225000 [288:02:53<116:51:14,  5.31s/it]
 65%|██████▍   | 145761/225000 [288:03:04<154:38:25,  7.03s/it]
 65%|██████▍   | 145762/225000 [288:03:08<131:49:30,  5.99s/it]
 65%|██████▍   | 145763/225000 [288:03:15<142:28:06,  6.47s/it]
 65%|██████▍   | 145764/225000 [288:03:20<128:10:31,  5.82s/it]
 65%|██████▍   | 145765/225000 [288:03:24<118:32:39,  5.39s/it]
 65%|██████▍   | 145766/225000 [288:03:28<111:03:57,  5.05s/it]
 65%|██████▍   | 145767/225000 [288:03:33<106:09:59,  4.82s/it]
 65%|██████▍   | 145768/225000 [288:03:40<121:05:55,  5.50s/it]
 65%|██████▍   | 145769/225000 [288:03:52<166:54:55,  7.58s/it]
 65%|██████▍   | 145770/225000 [288:03:56<142:31:18,  6.48s/it]
 65%|██████▍   | 145771/225000 [288:04:00<126:36:07,  5.75s/it]
 65%|██████▍   | 145772/225000 [288:04:04<113:08:09,  5.14s/it]
 65%|██████▍   | 145773/225000 [288:04:14<144:00:19,  6.54s/it]
 65%|██████▍   | 145774/225000 [288:04:18<129:46:30,  5.90s/it]
 65%|██████▍   | 145775/225000 [288:04:22<116:39:53,  5.30s/it]
 65%|██████▍   | 145776/225000 [288:04:27<116:24:16,  5.29s/it]
 65%|██████▍   | 145777/225000 [288:04:35<134:37:42,  6.12s/it]
 65%|██████▍   | 145778/225000 [288:04:39<118:01:35,  5.36s/it]
 65%|██████▍   | 145779/225000 [288:04:42<106:32:44,  4.84s/it]
 65%|██████▍   | 145780/225000 [288:04:47<106:03:33,  4.82s/it]
 65%|██████▍   | 145781/225000 [288:04:52<105:50:45,  4.81s/it]
 65%|██████▍   | 145782/225000 [288:04:57<107:53:22,  4.90s/it]
 65%|██████▍   | 145783/225000 [288:05:02<110:58:20,  5.04s/it]
 65%|██████▍   | 145784/225000 [288:05:07<108:25:54,  4.93s/it]
 65%|██████▍   | 145785/225000 [288:05:12<107:35:43,  4.89s/it]
 65%|██████▍   | 145786/225000 [288:05:17<110:32:19,  5.02s/it]
 65%|██████▍   | 145787/225000 [288:05:22<111:33:28,  5.07s/it]
 65%|██████▍   | 145788/225000 [288:05:34<154:26:43,  7.02s/it]
 65%|██████▍   | 145789/225000 [288:05:40<147:12:34,  6.69s/it]
 65%|██████▍   | 145790/225000 [288:05:44<128:11:03,  5.83s/it]
 65%|██████▍   | 145791/225000 [288:05:48<115:29:28,  5.25s/it]
 65%|██████▍   | 145792/225000 [288:05:58<151:51:29,  6.90s/it]
 65%|██████▍   | 145793/225000 [288:06:02<130:46:55,  5.94s/it]
 65%|██████▍   | 145794/225000 [288:06:08<127:28:00,  5.79s/it]
 65%|██████▍   | 145795/225000 [288:06:16<143:02:36,  6.50s/it]
 65%|██████▍   | 145796/225000 [288:06:19<122:59:39,  5.59s/it]
 65%|██████▍   | 145797/225000 [288:06:23<114:22:21,  5.20s/it]
 65%|██████▍   | 145798/225000 [288:06:33<142:35:14,  6.48s/it]
 65%|██████▍   | 145799/225000 [288:06:36<123:10:47,  5.60s/it]
 65%|██████▍   | 145800/225000 [288:06:45<141:58:03,  6.45s/it]
                                                               
{'loss': '0.7833', 'grad_norm': '1.173', 'learning_rate': '1.498e-05', 'epoch': '0.648', 'num_input_tokens_seen': 477563056, 'train_runtime': '1.037e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145800/225000 [288:06:45<141:58:03,  6.45s/it]
 65%|██████▍   | 145801/225000 [288:06:48<122:32:07,  5.57s/it]
 65%|██████▍   | 145802/225000 [288:06:56<138:25:10,  6.29s/it]
 65%|██████▍   | 145803/225000 [288:07:00<120:13:14,  5.46s/it]
 65%|██████▍   | 145804/225000 [288:07:06<126:26:09,  5.75s/it]
 65%|██████▍   | 145805/225000 [288:07:10<114:15:55,  5.19s/it]
 65%|██████▍   | 145806/225000 [288:07:14<106:29:24,  4.84s/it]
 65%|██████▍   | 145807/225000 [288:07:23<132:27:08,  6.02s/it]
 65%|██████▍   | 145808/225000 [288:07:29<130:37:08,  5.94s/it]
 65%|██████▍   | 145809/225000 [288:07:33<116:42:42,  5.31s/it]
 65%|██████▍   | 145810/225000 [288:07:39<123:17:57,  5.61s/it]
 65%|██████▍   | 145811/225000 [288:07:46<132:13:43,  6.01s/it]
 65%|██████▍   | 145812/225000 [288:07:49<113:35:08,  5.16s/it]
 65%|██████▍   | 145813/225000 [288:07:53<105:05:22,  4.78s/it]
 65%|██████▍   | 145814/225000 [288:08:01<127:30:14,  5.80s/it]
 65%|██████▍   | 145815/225000 [288:08:07<129:03:10,  5.87s/it]
 65%|██████▍   | 145816/225000 [288:08:13<127:19:02,  5.79s/it]
 65%|██████▍   | 145817/225000 [288:08:19<127:55:03,  5.82s/it]
 65%|██████▍   | 145818/225000 [288:08:22<113:36:29,  5.17s/it]
 65%|██████▍   | 145819/225000 [288:08:32<144:11:43,  6.56s/it]
 65%|██████▍   | 145820/225000 [288:08:36<126:37:01,  5.76s/it]
 65%|██████▍   | 145821/225000 [288:08:43<135:14:23,  6.15s/it]
 65%|██████▍   | 145822/225000 [288:08:47<118:42:07,  5.40s/it]
 65%|██████▍   | 145823/225000 [288:08:54<131:17:34,  5.97s/it]
 65%|██████▍   | 145824/225000 [288:08:59<127:53:04,  5.81s/it]
 65%|██████▍   | 145825/225000 [288:09:05<124:25:01,  5.66s/it]
 65%|██████▍   | 145826/225000 [288:09:11<131:55:16,  6.00s/it]
 65%|██████▍   | 145827/225000 [288:09:15<113:57:05,  5.18s/it]
 65%|██████▍   | 145828/225000 [288:09:20<114:22:10,  5.20s/it]
 65%|██████▍   | 145829/225000 [288:09:25<114:40:01,  5.21s/it]
 65%|██████▍   | 145830/225000 [288:09:31<115:56:02,  5.27s/it]
 65%|██████▍   | 145831/225000 [288:09:40<142:18:43,  6.47s/it]
 65%|██████▍   | 145832/225000 [288:09:44<127:02:36,  5.78s/it]
 65%|██████▍   | 145833/225000 [288:09:48<116:12:04,  5.28s/it]
 65%|██████▍   | 145834/225000 [288:09:52<109:08:24,  4.96s/it]
 65%|██████▍   | 145835/225000 [288:09:57<106:36:55,  4.85s/it]
 65%|██████▍   | 145836/225000 [288:10:04<121:49:19,  5.54s/it]
 65%|██████▍   | 145837/225000 [288:10:13<140:51:07,  6.41s/it]
 65%|██████▍   | 145838/225000 [288:10:17<124:22:59,  5.66s/it]
 65%|██████▍   | 145839/225000 [288:10:26<150:22:55,  6.84s/it]
 65%|██████▍   | 145840/225000 [288:10:32<144:33:47,  6.57s/it]
 65%|██████▍   | 145841/225000 [288:10:37<133:54:52,  6.09s/it]
 65%|██████▍   | 145842/225000 [288:10:42<126:19:37,  5.75s/it]
 65%|██████▍   | 145843/225000 [288:10:46<117:38:27,  5.35s/it]
 65%|██████▍   | 145844/225000 [288:10:51<114:24:06,  5.20s/it]
 65%|██████▍   | 145845/225000 [288:10:56<114:02:54,  5.19s/it]
 65%|██████▍   | 145846/225000 [288:11:01<110:09:16,  5.01s/it]
 65%|██████▍   | 145847/225000 [288:11:06<107:15:40,  4.88s/it]
 65%|██████▍   | 145848/225000 [288:11:10<107:13:40,  4.88s/it]
 65%|██████▍   | 145849/225000 [288:11:15<106:52:25,  4.86s/it]
 65%|██████▍   | 145850/225000 [288:11:20<107:03:49,  4.87s/it]
                                                               
{'loss': '0.7847', 'grad_norm': '0.9937', 'learning_rate': '1.496e-05', 'epoch': '0.6482', 'num_input_tokens_seen': 477692968, 'train_runtime': '1.037e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145850/225000 [288:11:20<107:03:49,  4.87s/it]
 65%|██████▍   | 145851/225000 [288:11:26<113:36:26,  5.17s/it]
 65%|██████▍   | 145852/225000 [288:11:38<157:35:55,  7.17s/it]
 65%|██████▍   | 145853/225000 [288:11:43<143:50:31,  6.54s/it]
 65%|██████▍   | 145854/225000 [288:11:51<151:03:52,  6.87s/it]
 65%|██████▍   | 145855/225000 [288:11:54<128:12:59,  5.83s/it]
 65%|██████▍   | 145856/225000 [288:11:58<114:38:37,  5.21s/it]
 65%|██████▍   | 145857/225000 [288:12:09<156:26:55,  7.12s/it]
 65%|██████▍   | 145858/225000 [288:12:13<135:51:15,  6.18s/it]
 65%|██████▍   | 145859/225000 [288:12:17<117:38:14,  5.35s/it]
 65%|██████▍   | 145860/225000 [288:12:23<126:29:29,  5.75s/it]
 65%|██████▍   | 145861/225000 [288:12:30<133:40:31,  6.08s/it]
 65%|██████▍   | 145862/225000 [288:12:34<116:39:11,  5.31s/it]
 65%|██████▍   | 145863/225000 [288:12:43<143:46:45,  6.54s/it]
 65%|██████▍   | 145864/225000 [288:12:46<121:28:23,  5.53s/it]
 65%|██████▍   | 145865/225000 [288:12:51<118:57:38,  5.41s/it]
 65%|██████▍   | 145866/225000 [288:12:57<116:31:16,  5.30s/it]
 65%|██████▍   | 145867/225000 [288:13:02<114:23:02,  5.20s/it]
 65%|██████▍   | 145868/225000 [288:13:06<111:37:12,  5.08s/it]
 65%|██████▍   | 145869/225000 [288:13:12<114:39:29,  5.22s/it]
 65%|██████▍   | 145870/225000 [288:13:17<112:03:30,  5.10s/it]
 65%|██████▍   | 145871/225000 [288:13:22<110:25:01,  5.02s/it]
 65%|██████▍   | 145872/225000 [288:13:29<125:07:30,  5.69s/it]
 65%|██████▍   | 145873/225000 [288:13:36<133:37:58,  6.08s/it]
 65%|██████▍   | 145874/225000 [288:13:41<128:15:59,  5.84s/it]
 65%|██████▍   | 145875/225000 [288:13:49<144:21:05,  6.57s/it]
 65%|██████▍   | 145876/225000 [288:13:53<127:01:14,  5.78s/it]
 65%|██████▍   | 145877/225000 [288:13:58<119:17:58,  5.43s/it]
 65%|██████▍   | 145878/225000 [288:14:03<118:16:58,  5.38s/it]
 65%|██████▍   | 145879/225000 [288:14:13<145:13:12,  6.61s/it]
 65%|██████▍   | 145880/225000 [288:14:16<123:39:39,  5.63s/it]
 65%|██████▍   | 145881/225000 [288:14:24<142:03:23,  6.46s/it]
 65%|██████▍   | 145882/225000 [288:14:29<127:16:26,  5.79s/it]
 65%|██████▍   | 145883/225000 [288:14:36<135:20:22,  6.16s/it]
 65%|██████▍   | 145884/225000 [288:14:40<122:53:24,  5.59s/it]
 65%|██████▍   | 145885/225000 [288:14:45<122:29:55,  5.57s/it]
 65%|██████▍   | 145886/225000 [288:14:53<132:54:28,  6.05s/it]
 65%|██████▍   | 145887/225000 [288:14:57<124:54:34,  5.68s/it]
 65%|██████▍   | 145888/225000 [288:15:04<133:15:28,  6.06s/it]
 65%|██████▍   | 145889/225000 [288:15:10<130:54:10,  5.96s/it]
 65%|██████▍   | 145890/225000 [288:15:14<117:19:58,  5.34s/it]
 65%|██████▍   | 145891/225000 [288:15:23<142:09:52,  6.47s/it]
 65%|██████▍   | 145892/225000 [288:15:27<123:11:30,  5.61s/it]
 65%|██████▍   | 145893/225000 [288:15:34<136:14:06,  6.20s/it]
 65%|██████▍   | 145894/225000 [288:15:38<120:07:24,  5.47s/it]
 65%|██████▍   | 145895/225000 [288:15:42<108:25:51,  4.93s/it]
 65%|██████▍   | 145896/225000 [288:15:47<108:58:39,  4.96s/it]
 65%|██████▍   | 145897/225000 [288:15:52<110:19:01,  5.02s/it]
 65%|██████▍   | 145898/225000 [288:15:57<110:57:13,  5.05s/it]
 65%|██████▍   | 145899/225000 [288:16:06<137:43:50,  6.27s/it]
 65%|██████▍   | 145900/225000 [288:16:12<133:19:24,  6.07s/it]
                                                               
{'loss': '0.7939', 'grad_norm': '1.061', 'learning_rate': '1.494e-05', 'epoch': '0.6484', 'num_input_tokens_seen': 477823304, 'train_runtime': '1.038e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145900/225000 [288:16:12<133:19:24,  6.07s/it]
 65%|██████▍   | 145901/225000 [288:16:18<135:58:17,  6.19s/it]
 65%|██████▍   | 145902/225000 [288:16:23<125:24:36,  5.71s/it]
 65%|██████▍   | 145903/225000 [288:16:26<110:48:02,  5.04s/it]
 65%|██████▍   | 145904/225000 [288:16:37<147:52:47,  6.73s/it]
 65%|██████▍   | 145905/225000 [288:16:40<126:32:30,  5.76s/it]
 65%|██████▍   | 145906/225000 [288:16:44<113:27:44,  5.16s/it]
 65%|██████▍   | 145907/225000 [288:16:49<111:03:55,  5.06s/it]
 65%|██████▍   | 145908/225000 [288:16:54<111:26:34,  5.07s/it]
 65%|██████▍   | 145909/225000 [288:16:59<109:31:35,  4.99s/it]
 65%|██████▍   | 145910/225000 [288:17:04<111:51:39,  5.09s/it]
 65%|██████▍   | 145911/225000 [288:17:14<142:51:46,  6.50s/it]
 65%|██████▍   | 145912/225000 [288:17:22<154:55:16,  7.05s/it]
 65%|██████▍   | 145913/225000 [288:17:26<132:03:31,  6.01s/it]
 65%|██████▍   | 145914/225000 [288:17:31<126:56:42,  5.78s/it]
 65%|██████▍   | 145915/225000 [288:17:40<146:51:45,  6.69s/it]
 65%|██████▍   | 145916/225000 [288:17:44<132:30:48,  6.03s/it]
 65%|██████▍   | 145917/225000 [288:17:50<126:12:35,  5.75s/it]
 65%|██████▍   | 145918/225000 [288:17:54<119:05:26,  5.42s/it]
 65%|██████▍   | 145919/225000 [288:17:59<114:24:43,  5.21s/it]
 65%|██████▍   | 145920/225000 [288:18:06<126:26:22,  5.76s/it]
 65%|██████▍   | 145921/225000 [288:18:17<160:49:31,  7.32s/it]
 65%|██████▍   | 145922/225000 [288:18:21<138:14:50,  6.29s/it]
 65%|██████▍   | 145923/225000 [288:18:25<124:44:23,  5.68s/it]
 65%|██████▍   | 145924/225000 [288:18:29<114:50:53,  5.23s/it]
 65%|██████▍   | 145925/225000 [288:18:39<145:34:09,  6.63s/it]
 65%|██████▍   | 145926/225000 [288:18:43<130:15:54,  5.93s/it]
 65%|██████▍   | 145927/225000 [288:18:51<141:55:10,  6.46s/it]
 65%|██████▍   | 145928/225000 [288:18:55<123:45:20,  5.63s/it]
 65%|██████▍   | 145929/225000 [288:19:03<138:49:06,  6.32s/it]
 65%|██████▍   | 145930/225000 [288:19:08<131:36:29,  5.99s/it]
 65%|██████▍   | 145931/225000 [288:19:11<115:27:11,  5.26s/it]
 65%|██████▍   | 145932/225000 [288:19:16<111:20:51,  5.07s/it]
 65%|██████▍   | 145933/225000 [288:19:22<113:20:06,  5.16s/it]
 65%|██████▍   | 145934/225000 [288:19:27<114:35:06,  5.22s/it]
 65%|██████▍   | 145935/225000 [288:19:32<112:37:02,  5.13s/it]
 65%|██████▍   | 145936/225000 [288:19:37<115:01:33,  5.24s/it]
 65%|██████▍   | 145937/225000 [288:19:43<117:11:18,  5.34s/it]
 65%|██████▍   | 145938/225000 [288:19:49<119:38:57,  5.45s/it]
 65%|██████▍   | 145939/225000 [288:19:54<119:24:41,  5.44s/it]
 65%|██████▍   | 145940/225000 [288:19:59<119:42:16,  5.45s/it]
 65%|██████▍   | 145941/225000 [288:20:05<118:52:42,  5.41s/it]
 65%|██████▍   | 145942/225000 [288:20:10<117:22:55,  5.35s/it]
 65%|██████▍   | 145943/225000 [288:20:15<118:10:35,  5.38s/it]
 65%|██████▍   | 145944/225000 [288:20:27<159:07:22,  7.25s/it]
 65%|██████▍   | 145945/225000 [288:20:33<148:16:38,  6.75s/it]
 65%|██████▍   | 145946/225000 [288:20:39<145:35:16,  6.63s/it]
 65%|██████▍   | 145947/225000 [288:20:43<127:31:04,  5.81s/it]
 65%|██████▍   | 145948/225000 [288:20:48<122:08:48,  5.56s/it]
 65%|██████▍   | 145949/225000 [288:20:53<120:33:20,  5.49s/it]
 65%|██████▍   | 145950/225000 [288:20:59<120:39:55,  5.50s/it]
                                                               
{'loss': '0.781', 'grad_norm': '1.028', 'learning_rate': '1.493e-05', 'epoch': '0.6487', 'num_input_tokens_seen': 477954600, 'train_runtime': '1.038e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 145950/225000 [288:20:59<120:39:55,  5.50s/it]
 65%|██████▍   | 145951/225000 [288:21:04<120:09:18,  5.47s/it]
 65%|██████▍   | 145952/225000 [288:21:09<119:45:23,  5.45s/it]
 65%|██████▍   | 145953/225000 [288:21:14<115:37:47,  5.27s/it]
 65%|██████▍   | 145954/225000 [288:21:19<113:00:53,  5.15s/it]
 65%|██████▍   | 145955/225000 [288:21:24<110:56:11,  5.05s/it]
 65%|██████▍   | 145956/225000 [288:21:29<113:39:01,  5.18s/it]
 65%|██████▍   | 145957/225000 [288:21:35<112:49:00,  5.14s/it]
 65%|██████▍   | 145958/225000 [288:21:40<115:08:47,  5.24s/it]
 65%|██████▍   | 145959/225000 [288:21:51<154:16:53,  7.03s/it]
 65%|██████▍   | 145960/225000 [288:21:59<160:10:17,  7.30s/it]
 65%|██████▍   | 145961/225000 [288:22:03<135:12:41,  6.16s/it]
 65%|██████▍   | 145962/225000 [288:22:10<141:38:42,  6.45s/it]
 65%|██████▍   | 145963/225000 [288:22:15<132:08:46,  6.02s/it]
 65%|██████▍   | 145964/225000 [288:22:20<123:33:49,  5.63s/it]
 65%|██████▍   | 145965/225000 [288:22:24<116:30:59,  5.31s/it]
 65%|██████▍   | 145966/225000 [288:22:29<112:42:49,  5.13s/it]
 65%|██████▍   | 145967/225000 [288:22:39<148:37:54,  6.77s/it]
 65%|██████▍   | 145968/225000 [288:22:43<129:44:02,  5.91s/it]
 65%|██████▍   | 145969/225000 [288:22:47<116:35:30,  5.31s/it]
 65%|██████▍   | 145970/225000 [288:22:51<108:43:50,  4.95s/it]
 65%|██████▍   | 145971/225000 [288:23:01<142:44:09,  6.50s/it]
 65%|██████▍   | 145972/225000 [288:23:05<124:27:11,  5.67s/it]
 65%|██████▍   | 145973/225000 [288:23:10<117:22:11,  5.35s/it]
 65%|██████▍   | 145974/225000 [288:23:20<147:55:39,  6.74s/it]
 65%|██████▍   | 145975/225000 [288:23:23<126:20:03,  5.76s/it]
 65%|██████▍   | 145976/225000 [288:23:31<142:48:02,  6.51s/it]
 65%|██████▍   | 145977/225000 [288:23:35<122:31:00,  5.58s/it]
 65%|██████▍   | 145978/225000 [288:23:39<110:34:57,  5.04s/it]
 65%|██████▍   | 145979/225000 [288:23:43<106:10:24,  4.84s/it]
 65%|██████▍   | 145980/225000 [288:23:48<104:06:22,  4.74s/it]
 65%|██████▍   | 145981/225000 [288:23:52<101:40:21,  4.63s/it]
 65%|██████▍   | 145982/225000 [288:23:59<120:15:01,  5.48s/it]
 65%|██████▍   | 145983/225000 [288:24:09<146:23:22,  6.67s/it]
 65%|██████▍   | 145984/225000 [288:24:14<136:48:11,  6.23s/it]
 65%|██████▍   | 145985/225000 [288:24:19<128:24:23,  5.85s/it]
 65%|██████▍   | 145986/225000 [288:24:23<118:44:36,  5.41s/it]
 65%|██████▍   | 145987/225000 [288:24:28<115:30:00,  5.26s/it]
 65%|██████▍   | 145988/225000 [288:24:33<110:21:05,  5.03s/it]
 65%|██████▍   | 145989/225000 [288:24:38<111:15:16,  5.07s/it]
 65%|██████▍   | 145990/225000 [288:24:44<115:06:24,  5.24s/it]
 65%|██████▍   | 145991/225000 [288:24:49<112:56:45,  5.15s/it]
 65%|██████▍   | 145992/225000 [288:25:01<159:53:27,  7.29s/it]
 65%|██████▍   | 145993/225000 [288:25:07<150:05:13,  6.84s/it]
 65%|██████▍   | 145994/225000 [288:25:10<130:12:26,  5.93s/it]
 65%|██████▍   | 145995/225000 [288:25:15<119:32:01,  5.45s/it]
 65%|██████▍   | 145996/225000 [288:25:19<112:49:27,  5.14s/it]
 65%|██████▍   | 145997/225000 [288:25:23<106:38:08,  4.86s/it]
 65%|██████▍   | 145998/225000 [288:25:31<126:19:53,  5.76s/it]
 65%|██████▍   | 145999/225000 [288:25:41<151:49:54,  6.92s/it]
 65%|██████▍   | 146000/225000 [288:25:45<132:14:01,  6.03s/it]
                                                               
{'loss': '0.7799', 'grad_norm': '1.141', 'learning_rate': '1.491e-05', 'epoch': '0.6489', 'num_input_tokens_seen': 478083784, 'train_runtime': '1.038e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 146000/225000 [288:25:45<132:14:01,  6.03s/it]
 65%|██████▍   | 146001/225000 [288:25:51<135:35:40,  6.18s/it]
 65%|██████▍   | 146002/225000 [288:25:57<135:05:21,  6.16s/it]
 65%|██████▍   | 146003/225000 [288:26:01<119:31:00,  5.45s/it]
 65%|██████▍   | 146004/225000 [288:26:11<146:01:05,  6.65s/it]
 65%|██████▍   | 146005/225000 [288:26:15<131:15:47,  5.98s/it]
 65%|██████▍   | 146006/225000 [288:26:19<120:46:48,  5.50s/it]
 65%|██████▍   | 146007/225000 [288:26:24<114:12:01,  5.20s/it]
 65%|██████▍   | 146008/225000 [288:26:28<108:00:47,  4.92s/it]
 65%|██████▍   | 146009/225000 [288:26:33<105:21:52,  4.80s/it]
 65%|██████▍   | 146010/225000 [288:26:37<103:29:37,  4.72s/it]
 65%|██████▍   | 146011/225000 [288:26:42<104:02:33,  4.74s/it]
 65%|██████▍   | 146012/225000 [288:26:47<103:47:58,  4.73s/it]
 65%|██████▍   | 146013/225000 [288:26:59<153:33:22,  7.00s/it]
 65%|██████▍   | 146014/225000 [288:27:05<147:48:45,  6.74s/it]
 65%|██████▍   | 146015/225000 [288:27:10<132:19:31,  6.03s/it]
 65%|██████▍   | 146016/225000 [288:27:15<128:24:40,  5.85s/it]
 65%|██████▍   | 146017/225000 [288:27:21<127:12:30,  5.80s/it]
 65%|██████▍   | 146018/225000 [288:27:26<125:30:42,  5.72s/it]
 65%|██████▍   | 146019/225000 [288:27:34<136:00:42,  6.20s/it]
 65%|██████▍   | 146020/225000 [288:27:37<118:47:35,  5.41s/it]
 65%|██████▍   | 146021/225000 [288:27:47<145:21:09,  6.63s/it]
 65%|██████▍   | 146022/225000 [288:27:50<122:55:04,  5.60s/it]
 65%|██████▍   | 146023/225000 [288:27:55<121:35:59,  5.54s/it]
 65%|██████▍   | 146024/225000 [288:28:01<124:13:31,  5.66s/it]
 65%|██████▍   | 146025/225000 [288:28:07<126:34:20,  5.77s/it]
 65%|██████▍   | 146026/225000 [288:28:13<124:39:03,  5.68s/it]
 65%|██████▍   | 146027/225000 [288:28:17<117:58:20,  5.38s/it]
 65%|██████▍   | 146028/225000 [288:28:22<113:32:50,  5.18s/it]
 65%|██████▍   | 146029/225000 [288:28:27<109:38:31,  5.00s/it]
 65%|██████▍   | 146030/225000 [288:28:31<104:24:54,  4.76s/it]
 65%|██████▍   | 146031/225000 [288:28:38<118:13:04,  5.39s/it]
 65%|██████▍   | 146032/225000 [288:28:49<158:12:17,  7.21s/it]
 65%|██████▍   | 146033/225000 [288:28:52<132:46:02,  6.05s/it]
 65%|██████▍   | 146034/225000 [288:29:01<150:41:18,  6.87s/it]
 65%|██████▍   | 146035/225000 [288:29:05<129:35:27,  5.91s/it]
 65%|██████▍   | 146036/225000 [288:29:13<142:55:57,  6.52s/it]
 65%|██████▍   | 146037/225000 [288:29:17<129:22:25,  5.90s/it]
 65%|██████▍   | 146038/225000 [288:29:22<121:03:14,  5.52s/it]
 65%|██████▍   | 146039/225000 [288:29:26<113:18:24,  5.17s/it]
 65%|██████▍   | 146040/225000 [288:29:30<105:56:23,  4.83s/it]
 65%|██████▍   | 146041/225000 [288:29:35<104:32:27,  4.77s/it]
 65%|██████▍   | 146042/225000 [288:29:39<100:39:50,  4.59s/it]
 65%|██████▍   | 146043/225000 [288:29:44<105:40:25,  4.82s/it]
 65%|██████▍   | 146044/225000 [288:29:57<158:59:18,  7.25s/it]
 65%|██████▍   | 146045/225000 [288:30:01<134:18:44,  6.12s/it]
 65%|██████▍   | 146046/225000 [288:30:05<122:13:44,  5.57s/it]
 65%|██████▍   | 146047/225000 [288:30:13<136:38:50,  6.23s/it]
 65%|██████▍   | 146048/225000 [288:30:16<117:27:57,  5.36s/it]
 65%|██████▍   | 146049/225000 [288:30:23<124:47:14,  5.69s/it]
 65%|██████▍   | 146050/225000 [288:30:29<131:48:19,  6.01s/it]
                                                               
{'loss': '0.7845', 'grad_norm': '1.153', 'learning_rate': '1.489e-05', 'epoch': '0.6491', 'num_input_tokens_seen': 478213576, 'train_runtime': '1.039e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 146050/225000 [288:30:29<131:48:19,  6.01s/it]
 65%|██████▍   | 146051/225000 [288:30:33<118:21:05,  5.40s/it]
 65%|██████▍   | 146052/225000 [288:30:40<124:53:25,  5.69s/it]
 65%|██████▍   | 146053/225000 [288:30:47<136:45:05,  6.24s/it]
 65%|██████▍   | 146054/225000 [288:30:50<114:29:31,  5.22s/it]
 65%|██████▍   | 146055/225000 [288:30:58<130:53:46,  5.97s/it]
 65%|██████▍   | 146056/225000 [288:31:01<114:35:57,  5.23s/it]
 65%|██████▍   | 146057/225000 [288:31:10<136:34:53,  6.23s/it]
 65%|██████▍   | 146058/225000 [288:31:13<117:31:42,  5.36s/it]
 65%|██████▍   | 146059/225000 [288:31:21<135:41:43,  6.19s/it]
 65%|██████▍   | 146060/225000 [288:31:26<122:08:49,  5.57s/it]
 65%|██████▍   | 146061/225000 [288:31:30<113:12:44,  5.16s/it]
 65%|██████▍   | 146062/225000 [288:31:34<107:50:07,  4.92s/it]
 65%|██████▍   | 146063/225000 [288:31:41<121:13:47,  5.53s/it]
 65%|██████▍   | 146064/225000 [288:31:48<128:42:35,  5.87s/it]
 65%|██████▍   | 146065/225000 [288:31:52<121:14:00,  5.53s/it]
 65%|██████▍   | 146066/225000 [288:31:57<116:37:12,  5.32s/it]
 65%|██████▍   | 146067/225000 [288:32:02<114:29:43,  5.22s/it]
 65%|██████▍   | 146068/225000 [288:32:07<110:32:17,  5.04s/it]
 65%|██████▍   | 146069/225000 [288:32:12<109:06:20,  4.98s/it]
 65%|██████▍   | 146070/225000 [288:32:17<110:02:21,  5.02s/it]
 65%|██████▍   | 146071/225000 [288:32:22<110:18:45,  5.03s/it]
 65%|██████▍   | 146072/225000 [288:32:32<143:07:00,  6.53s/it]
 65%|██████▍   | 146073/225000 [288:32:37<134:37:54,  6.14s/it]
 65%|██████▍   | 146074/225000 [288:32:42<124:37:37,  5.68s/it]
 65%|██████▍   | 146075/225000 [288:32:46<116:33:42,  5.32s/it]
 65%|██████▍   | 146076/225000 [288:32:51<112:01:52,  5.11s/it]
 65%|██████▍   | 146077/225000 [288:32:56<108:56:01,  4.97s/it]
 65%|██████▍   | 146078/225000 [288:33:08<157:11:04,  7.17s/it]
 65%|██████▍   | 146079/225000 [288:33:13<143:22:34,  6.54s/it]
 65%|██████▍   | 146080/225000 [288:33:18<134:09:04,  6.12s/it]
 65%|██████▍   | 146081/225000 [288:33:24<130:29:48,  5.95s/it]
 65%|██████▍   | 146082/225000 [288:33:29<126:30:53,  5.77s/it]
 65%|██████▍   | 146083/225000 [288:33:36<134:55:47,  6.16s/it]
 65%|██████▍   | 146084/225000 [288:33:43<138:55:05,  6.34s/it]
 65%|██████▍   | 146085/225000 [288:33:50<141:56:33,  6.48s/it]
 65%|██████▍   | 146086/225000 [288:33:55<135:13:11,  6.17s/it]
 65%|██████▍   | 146087/225000 [288:34:00<124:38:16,  5.69s/it]
 65%|██████▍   | 146088/225000 [288:34:05<121:08:46,  5.53s/it]
 65%|██████▍   | 146089/225000 [288:34:10<118:14:27,  5.39s/it]
 65%|██████▍   | 146090/225000 [288:34:15<114:37:32,  5.23s/it]
 65%|██████▍   | 146091/225000 [288:34:20<113:36:41,  5.18s/it]
 65%|██████▍   | 146092/225000 [288:34:25<112:56:46,  5.15s/it]
 65%|██████▍   | 146093/225000 [288:34:29<109:51:31,  5.01s/it]
 65%|██████▍   | 146094/225000 [288:34:35<110:28:43,  5.04s/it]
 65%|██████▍   | 146095/225000 [288:34:47<158:26:31,  7.23s/it]
 65%|██████▍   | 146096/225000 [288:34:50<132:51:08,  6.06s/it]
 65%|██████▍   | 146097/225000 [288:34:58<141:51:45,  6.47s/it]
 65%|██████▍   | 146098/225000 [288:35:02<130:35:50,  5.96s/it]
 65%|██████▍   | 146099/225000 [288:35:07<120:36:18,  5.50s/it]
 65%|██████▍   | 146100/225000 [288:35:12<115:04:38,  5.25s/it]
                                                               
{'loss': '0.7731', 'grad_norm': '1.079', 'learning_rate': '1.488e-05', 'epoch': '0.6493', 'num_input_tokens_seen': 478343800, 'train_runtime': '1.039e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 146100/225000 [288:35:12<115:04:38,  5.25s/it]
 65%|██████▍   | 146101/225000 [288:35:16<109:47:08,  5.01s/it]
 65%|██████▍   | 146102/225000 [288:35:21<108:50:54,  4.97s/it]
 65%|██████▍   | 146103/225000 [288:35:26<110:07:22,  5.02s/it]
 65%|██████▍   | 146104/225000 [288:35:31<111:20:50,  5.08s/it]
 65%|██████▍   | 146105/225000 [288:35:36<110:38:38,  5.05s/it]
 65%|██████▍   | 146106/225000 [288:35:44<129:28:56,  5.91s/it]
 65%|██████▍   | 146107/225000 [288:35:53<149:34:52,  6.83s/it]
 65%|██████▍   | 146108/225000 [288:35:57<132:21:23,  6.04s/it]
 65%|██████▍   | 146109/225000 [288:36:02<120:55:28,  5.52s/it]
 65%|██████▍   | 146110/225000 [288:36:06<114:24:27,  5.22s/it]
 65%|██████▍   | 146111/225000 [288:36:17<150:47:36,  6.88s/it]
 65%|██████▍   | 146112/225000 [288:36:21<132:09:55,  6.03s/it]
 65%|██████▍   | 146113/225000 [288:36:25<117:56:19,  5.38s/it]
 65%|██████▍   | 146114/225000 [288:36:32<131:46:19,  6.01s/it]
 65%|██████▍   | 146115/225000 [288:36:40<142:56:28,  6.52s/it]
 65%|██████▍   | 146116/225000 [288:36:45<129:43:42,  5.92s/it]
 65%|██████▍   | 146117/225000 [288:36:49<121:36:45,  5.55s/it]
 65%|██████▍   | 146118/225000 [288:36:53<110:28:36,  5.04s/it]
 65%|██████▍   | 146119/225000 [288:37:03<145:54:10,  6.66s/it]
 65%|██████▍   | 146120/225000 [288:37:08<131:39:46,  6.01s/it]
 65%|██████▍   | 146121/225000 [288:37:12<121:46:00,  5.56s/it]
 65%|██████▍   | 146122/225000 [288:37:17<115:53:33,  5.29s/it]
 65%|██████▍   | 146123/225000 [288:37:21<109:08:32,  4.98s/it]
 65%|██████▍   | 146124/225000 [288:37:26<104:13:59,  4.76s/it]
 65%|██████▍   | 146125/225000 [288:37:33<118:08:13,  5.39s/it]
 65%|██████▍   | 146126/225000 [288:37:43<148:41:36,  6.79s/it]
 65%|██████▍   | 146127/225000 [288:37:48<141:18:53,  6.45s/it]
 65%|██████▍   | 146128/225000 [288:37:52<122:18:40,  5.58s/it]
 65%|██████▍   | 146129/225000 [288:37:59<131:31:21,  6.00s/it]
 65%|██████▍   | 146130/225000 [288:38:03<119:47:27,  5.47s/it]
 65%|██████▍   | 146131/225000 [288:38:08<113:35:14,  5.18s/it]
 65%|██████▍   | 146132/225000 [288:38:12<109:05:29,  4.98s/it]
 65%|██████▍   | 146133/225000 [288:38:18<116:19:10,  5.31s/it]
 65%|██████▍   | 146134/225000 [288:38:30<156:54:20,  7.16s/it]
 65%|██████▍   | 146135/225000 [288:38:33<132:20:14,  6.04s/it]
 65%|██████▍   | 146136/225000 [288:38:38<126:30:14,  5.77s/it]
 65%|██████▍   | 146137/225000 [288:38:43<122:50:06,  5.61s/it]
 65%|██████▍   | 146138/225000 [288:38:49<119:50:08,  5.47s/it]
 65%|██████▍   | 146139/225000 [288:38:53<115:18:54,  5.26s/it]
 65%|██████▍   | 146140/225000 [288:38:59<115:10:49,  5.26s/it]
 65%|██████▍   | 146141/225000 [288:39:03<112:38:38,  5.14s/it]
 65%|██████▍   | 146142/225000 [288:39:09<113:35:27,  5.19s/it]
 65%|██████▍   | 146143/225000 [288:39:14<114:19:49,  5.22s/it]
 65%|██████▍   | 146144/225000 [288:39:19<114:36:40,  5.23s/it]
 65%|██████▍   | 146145/225000 [288:39:25<117:26:28,  5.36s/it]
 65%|██████▍   | 146146/225000 [288:39:31<119:09:24,  5.44s/it]
 65%|██████▍   | 146147/225000 [288:39:36<116:49:22,  5.33s/it]
 65%|██████▍   | 146148/225000 [288:39:41<116:21:12,  5.31s/it]
 65%|██████▍   | 146149/225000 [288:39:46<117:12:49,  5.35s/it]
 65%|██████▍   | 146150/225000 [288:39:51<114:40:56,  5.24s/it]
                                                               
{'loss': '0.7973', 'grad_norm': '1.155', 'learning_rate': '1.486e-05', 'epoch': '0.6496', 'num_input_tokens_seen': 478473736, 'train_runtime': '1.039e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 146150/225000 [288:39:51<114:40:56,  5.24s/it]
 65%|██████▍   | 146151/225000 [288:39:56<113:15:26,  5.17s/it]
 65%|██████▍   | 146152/225000 [288:40:09<163:34:59,  7.47s/it]
 65%|██████▍   | 146153/225000 [288:40:15<154:11:27,  7.04s/it]
 65%|██████▍   | 146154/225000 [288:40:19<136:06:23,  6.21s/it]
 65%|██████▍   | 146155/225000 [288:40:25<131:03:31,  5.98s/it]
 65%|██████▍   | 146156/225000 [288:40:30<124:49:58,  5.70s/it]
 65%|██████▍   | 146157/225000 [288:40:35<122:44:40,  5.60s/it]
 65%|██████▍   | 146158/225000 [288:40:40<119:16:56,  5.45s/it]
 65%|██████▍   | 146159/225000 [288:40:45<115:20:53,  5.27s/it]
 65%|██████▍   | 146160/225000 [288:40:51<115:34:58,  5.28s/it]
 65%|██████▍   | 146161/225000 [288:40:55<112:28:47,  5.14s/it]
 65%|██████▍   | 146162/225000 [288:41:01<112:40:16,  5.14s/it]
 65%|██████▍   | 146163/225000 [288:41:06<113:34:01,  5.19s/it]
 65%|██████▍   | 146164/225000 [288:41:11<112:29:22,  5.14s/it]
 65%|██████▍   | 146165/225000 [288:41:22<152:28:02,  6.96s/it]
 65%|██████▍   | 146166/225000 [288:41:29<154:55:23,  7.07s/it]
 65%|██████▍   | 146167/225000 [288:41:33<132:29:32,  6.05s/it]
 65%|██████▍   | 146168/225000 [288:41:39<131:10:40,  5.99s/it]
 65%|██████▍   | 146169/225000 [288:41:47<142:36:33,  6.51s/it]
 65%|██████▍   | 146170/225000 [288:41:50<121:43:33,  5.56s/it]
 65%|██████▍   | 146171/225000 [288:41:55<118:24:30,  5.41s/it]
 65%|██████▍   | 146172/225000 [288:42:00<118:22:45,  5.41s/it]
 65%|██████▍   | 146173/225000 [288:42:06<117:54:57,  5.39s/it]
 65%|██████▍   | 146174/225000 [288:42:11<117:53:25,  5.38s/it]
 65%|██████▍   | 146175/225000 [288:42:22<154:35:00,  7.06s/it]
 65%|██████▍   | 146176/225000 [288:42:26<135:47:47,  6.20s/it]
 65%|██████▍   | 146177/225000 [288:42:34<143:20:00,  6.55s/it]
 65%|██████▍   | 146178/225000 [288:42:39<132:41:14,  6.06s/it]
 65%|██████▍   | 146179/225000 [288:42:44<125:56:03,  5.75s/it]
 65%|██████▍   | 146180/225000 [288:42:48<119:07:55,  5.44s/it]
 65%|██████▍   | 146181/225000 [288:42:53<114:55:32,  5.25s/it]
 65%|██████▍   | 146182/225000 [288:42:58<111:31:21,  5.09s/it]
 65%|██████▍   | 146183/225000 [288:43:03<111:56:07,  5.11s/it]
 65%|██████▍   | 146184/225000 [288:43:08<110:40:54,  5.06s/it]
 65%|██████▍   | 146185/225000 [288:43:13<108:53:36,  4.97s/it]
 65%|██████▍   | 146186/225000 [288:43:18<109:22:40,  5.00s/it]
 65%|██████▍   | 146187/225000 [288:43:23<110:49:09,  5.06s/it]
 65%|██████▍   | 146188/225000 [288:43:28<112:21:00,  5.13s/it]
 65%|██████▍   | 146189/225000 [288:43:39<149:14:55,  6.82s/it]
 65%|██████▍   | 146190/225000 [288:43:47<157:50:34,  7.21s/it]
 65%|██████▍   | 146191/225000 [288:43:52<143:39:40,  6.56s/it]
 65%|██████▍   | 146192/225000 [288:43:57<130:54:38,  5.98s/it]
 65%|██████▍   | 146193/225000 [288:44:02<122:18:28,  5.59s/it]
 65%|██████▍   | 146194/225000 [288:44:06<118:09:55,  5.40s/it]
 65%|██████▍   | 146195/225000 [288:44:11<113:36:18,  5.19s/it]
 65%|██████▍   | 146196/225000 [288:44:16<109:42:09,  5.01s/it]
 65%|██████▍   | 146197/225000 [288:44:21<107:47:57,  4.92s/it]
 65%|██████▍   | 146198/225000 [288:44:25<104:40:45,  4.78s/it]
 65%|██████▍   | 146199/225000 [288:44:30<106:26:19,  4.86s/it]
 65%|██████▍   | 146200/225000 [288:44:35<106:17:03,  4.86s/it]
                                                               
{'loss': '0.7921', 'grad_norm': '1.122', 'learning_rate': '1.484e-05', 'epoch': '0.6498', 'num_input_tokens_seen': 478602728, 'train_runtime': '1.039e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▍   | 146200/225000 [288:44:35<106:17:03,  4.86s/it]
 65%|██████▍   | 146201/225000 [288:44:47<154:55:47,  7.08s/it]
 65%|██████▍   | 146202/225000 [288:44:52<137:48:56,  6.30s/it]
 65%|██████▍   | 146203/225000 [288:44:55<118:50:39,  5.43s/it]
 65%|██████▍   | 146204/225000 [288:45:02<132:15:01,  6.04s/it]
 65%|██████▍   | 146205/225000 [288:45:09<138:23:24,  6.32s/it]
 65%|██████▍   | 146206/225000 [288:45:13<117:46:22,  5.38s/it]
 65%|██████▍   | 146207/225000 [288:45:21<135:56:30,  6.21s/it]
 65%|██████▍   | 146208/225000 [288:45:25<120:06:10,  5.49s/it]
 65%|██████▍   | 146209/225000 [288:45:28<109:04:40,  4.98s/it]
 65%|██████▍   | 146210/225000 [288:45:39<143:32:57,  6.56s/it]
 65%|██████▍   | 146211/225000 [288:45:42<121:27:43,  5.55s/it]
 65%|██████▍   | 146212/225000 [288:45:47<116:52:12,  5.34s/it]
 65%|██████▍   | 146213/225000 [288:45:51<111:34:54,  5.10s/it]
 65%|██████▍   | 146214/225000 [288:46:00<136:16:58,  6.23s/it]
 65%|██████▍   | 146215/225000 [288:46:06<136:04:00,  6.22s/it]
 65%|██████▍   | 146216/225000 [288:46:10<118:14:55,  5.40s/it]
 65%|██████▍   | 146217/225000 [288:46:17<132:17:59,  6.05s/it]
 65%|██████▍   | 146218/225000 [288:46:21<116:28:30,  5.32s/it]
 65%|██████▍   | 146219/225000 [288:46:26<115:07:26,  5.26s/it]
 65%|██████▍   | 146220/225000 [288:46:32<117:45:17,  5.38s/it]
 65%|██████▍   | 146221/225000 [288:46:37<119:08:53,  5.44s/it]
 65%|██████▍   | 146222/225000 [288:46:43<119:58:29,  5.48s/it]
 65%|██████▍   | 146223/225000 [288:46:48<120:20:33,  5.50s/it]
 65%|██████▍   | 146224/225000 [288:46:53<115:32:02,  5.28s/it]
 65%|██████▍   | 146225/225000 [288:47:05<155:12:47,  7.09s/it]
 65%|██████▍   | 146226/225000 [288:47:08<130:07:24,  5.95s/it]
 65%|██████▍   | 146227/225000 [288:47:12<116:30:43,  5.32s/it]
 65%|██████▍   | 146228/225000 [288:47:19<131:52:51,  6.03s/it]
 65%|██████▍   | 146229/225000 [288:47:24<122:33:20,  5.60s/it]
 65%|██████▍   | 146230/225000 [288:47:29<118:09:58,  5.40s/it]
 65%|██████▍   | 146231/225000 [288:47:34<113:54:25,  5.21s/it]
 65%|██████▍   | 146232/225000 [288:47:39<112:11:36,  5.13s/it]
 65%|██████▍   | 146233/225000 [288:47:44<111:05:22,  5.08s/it]
 65%|██████▍   | 146234/225000 [288:47:48<108:17:17,  4.95s/it]
 65%|██████▍   | 146235/225000 [288:47:53<105:45:34,  4.83s/it]
 65%|██████▍   | 146236/225000 [288:47:57<104:52:32,  4.79s/it]
 65%|██████▍   | 146237/225000 [288:48:08<143:59:19,  6.58s/it]
 65%|██████▍   | 146238/225000 [288:48:15<143:02:11,  6.54s/it]
 65%|██████▍   | 146239/225000 [288:48:18<121:51:36,  5.57s/it]
 65%|██████▍   | 146240/225000 [288:48:26<139:33:26,  6.38s/it]
 65%|██████▍   | 146241/225000 [288:48:32<133:31:37,  6.10s/it]
 65%|██████▍   | 146242/225000 [288:48:37<128:17:47,  5.86s/it]
 65%|██████▍   | 146243/225000 [288:48:46<147:50:12,  6.76s/it]
 65%|██████▍   | 146244/225000 [288:48:49<122:48:08,  5.61s/it]
 65%|██████▍   | 146245/225000 [288:48:54<123:17:59,  5.64s/it]
 65%|██████▍   | 146246/225000 [288:49:02<137:12:07,  6.27s/it]
 65%|██████▍   | 146247/225000 [288:49:06<120:53:42,  5.53s/it]
 65%|██████▍   | 146248/225000 [288:49:10<110:55:33,  5.07s/it]
 65%|██████▍   | 146249/225000 [288:49:21<148:29:29,  6.79s/it]
 65%|██████▌   | 146250/225000 [288:49:25<129:53:01,  5.94s/it]
                                                               
{'loss': '0.7934', 'grad_norm': '1.087', 'learning_rate': '1.483e-05', 'epoch': '0.65', 'num_input_tokens_seen': 478733392, 'train_runtime': '1.04e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146250/225000 [288:49:25<129:53:01,  5.94s/it]
 65%|██████▌   | 146251/225000 [288:49:28<113:59:21,  5.21s/it]
 65%|██████▌   | 146252/225000 [288:49:37<138:30:00,  6.33s/it]
 65%|██████▌   | 146253/225000 [288:49:41<124:12:00,  5.68s/it]
 65%|██████▌   | 146254/225000 [288:49:45<113:19:35,  5.18s/it]
 65%|██████▌   | 146255/225000 [288:49:50<108:54:33,  4.98s/it]
 65%|██████▌   | 146256/225000 [288:49:54<106:28:42,  4.87s/it]
 65%|██████▌   | 146257/225000 [288:50:06<151:16:04,  6.92s/it]
 65%|██████▌   | 146258/225000 [288:50:10<132:44:02,  6.07s/it]
 65%|██████▌   | 146259/225000 [288:50:14<118:45:57,  5.43s/it]
 65%|██████▌   | 146260/225000 [288:50:19<115:48:38,  5.29s/it]
 65%|██████▌   | 146261/225000 [288:50:28<139:12:27,  6.36s/it]
 65%|██████▌   | 146262/225000 [288:50:32<120:00:48,  5.49s/it]
 65%|██████▌   | 146263/225000 [288:50:39<131:36:07,  6.02s/it]
 65%|██████▌   | 146264/225000 [288:50:42<115:27:20,  5.28s/it]
 65%|██████▌   | 146265/225000 [288:50:52<143:21:56,  6.56s/it]
 65%|██████▌   | 146266/225000 [288:50:55<123:58:21,  5.67s/it]
 65%|██████▌   | 146267/225000 [288:51:02<129:06:11,  5.90s/it]
 65%|██████▌   | 146268/225000 [288:51:06<116:37:58,  5.33s/it]
 65%|██████▌   | 146269/225000 [288:51:13<126:41:11,  5.79s/it]
 65%|██████▌   | 146270/225000 [288:51:18<123:27:36,  5.65s/it]
 65%|██████▌   | 146271/225000 [288:51:24<122:53:28,  5.62s/it]
 65%|██████▌   | 146272/225000 [288:51:32<137:54:29,  6.31s/it]
 65%|██████▌   | 146273/225000 [288:51:35<121:22:47,  5.55s/it]
 65%|██████▌   | 146274/225000 [288:51:39<110:47:57,  5.07s/it]
 65%|██████▌   | 146275/225000 [288:51:48<135:07:54,  6.18s/it]
 65%|██████▌   | 146276/225000 [288:51:53<129:30:53,  5.92s/it]
 65%|██████▌   | 146277/225000 [288:51:57<113:56:22,  5.21s/it]
 65%|██████▌   | 146278/225000 [288:52:01<105:57:56,  4.85s/it]
 65%|██████▌   | 146279/225000 [288:52:06<107:05:16,  4.90s/it]
 65%|██████▌   | 146280/225000 [288:52:11<110:00:07,  5.03s/it]
 65%|██████▌   | 146281/225000 [288:52:17<111:55:25,  5.12s/it]
 65%|██████▌   | 146282/225000 [288:52:22<111:19:55,  5.09s/it]
 65%|██████▌   | 146283/225000 [288:52:27<112:12:53,  5.13s/it]
 65%|██████▌   | 146284/225000 [288:52:32<113:18:44,  5.18s/it]
 65%|██████▌   | 146285/225000 [288:52:38<116:05:55,  5.31s/it]
 65%|██████▌   | 146286/225000 [288:52:43<115:31:46,  5.28s/it]
 65%|██████▌   | 146287/225000 [288:52:48<116:52:34,  5.35s/it]
 65%|██████▌   | 146288/225000 [288:52:53<112:54:37,  5.16s/it]
 65%|██████▌   | 146289/225000 [288:52:58<113:42:11,  5.20s/it]
 65%|██████▌   | 146290/225000 [288:53:04<114:34:07,  5.24s/it]
 65%|██████▌   | 146291/225000 [288:53:09<112:04:47,  5.13s/it]
 65%|██████▌   | 146292/225000 [288:53:14<114:56:09,  5.26s/it]
 65%|██████▌   | 146293/225000 [288:53:21<123:07:42,  5.63s/it]
 65%|██████▌   | 146294/225000 [288:53:34<170:00:13,  7.78s/it]
 65%|██████▌   | 146295/225000 [288:53:37<141:35:37,  6.48s/it]
 65%|██████▌   | 146296/225000 [288:53:41<126:58:58,  5.81s/it]
 65%|██████▌   | 146297/225000 [288:53:46<122:58:55,  5.63s/it]
 65%|██████▌   | 146298/225000 [288:53:51<116:13:44,  5.32s/it]
 65%|██████▌   | 146299/225000 [288:53:56<112:11:36,  5.13s/it]
 65%|██████▌   | 146300/225000 [288:54:01<110:29:29,  5.05s/it]
                                                               
{'loss': '0.787', 'grad_norm': '1.15', 'learning_rate': '1.481e-05', 'epoch': '0.6502', 'num_input_tokens_seen': 478862808, 'train_runtime': '1.04e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146300/225000 [288:54:01<110:29:29,  5.05s/it]
 65%|██████▌   | 146301/225000 [288:54:05<106:46:04,  4.88s/it]
 65%|██████▌   | 146302/225000 [288:54:12<120:39:49,  5.52s/it]
 65%|██████▌   | 146303/225000 [288:54:24<165:16:08,  7.56s/it]
 65%|██████▌   | 146304/225000 [288:54:28<140:36:07,  6.43s/it]
 65%|██████▌   | 146305/225000 [288:54:32<123:45:21,  5.66s/it]
 65%|██████▌   | 146306/225000 [288:54:39<132:45:50,  6.07s/it]
 65%|██████▌   | 146307/225000 [288:54:47<146:52:48,  6.72s/it]
 65%|██████▌   | 146308/225000 [288:54:51<125:23:36,  5.74s/it]
 65%|██████▌   | 146309/225000 [288:54:55<118:44:53,  5.43s/it]
 65%|██████▌   | 146310/225000 [288:55:05<142:20:22,  6.51s/it]
 65%|██████▌   | 146311/225000 [288:55:08<120:42:17,  5.52s/it]
 65%|██████▌   | 146312/225000 [288:55:17<145:00:24,  6.63s/it]
 65%|██████▌   | 146313/225000 [288:55:22<137:14:50,  6.28s/it]
 65%|██████▌   | 146314/225000 [288:55:27<128:42:52,  5.89s/it]
 65%|██████▌   | 146315/225000 [288:55:33<125:27:34,  5.74s/it]
 65%|██████▌   | 146316/225000 [288:55:38<120:00:58,  5.49s/it]
 65%|██████▌   | 146317/225000 [288:55:42<115:23:26,  5.28s/it]
 65%|██████▌   | 146318/225000 [288:55:48<115:11:21,  5.27s/it]
 65%|██████▌   | 146319/225000 [288:55:53<114:56:26,  5.26s/it]
 65%|██████▌   | 146320/225000 [288:55:58<115:04:47,  5.27s/it]
 65%|██████▌   | 146321/225000 [288:56:04<116:34:49,  5.33s/it]
 65%|██████▌   | 146322/225000 [288:56:09<117:00:57,  5.35s/it]
 65%|██████▌   | 146323/225000 [288:56:14<116:00:09,  5.31s/it]
 65%|██████▌   | 146324/225000 [288:56:19<112:48:15,  5.16s/it]
 65%|██████▌   | 146325/225000 [288:56:24<110:03:58,  5.04s/it]
 65%|██████▌   | 146326/225000 [288:56:29<109:47:16,  5.02s/it]
 65%|██████▌   | 146327/225000 [288:56:34<112:14:31,  5.14s/it]
 65%|██████▌   | 146328/225000 [288:56:40<113:03:34,  5.17s/it]
 65%|██████▌   | 146329/225000 [288:56:45<115:50:58,  5.30s/it]
 65%|██████▌   | 146330/225000 [288:56:50<114:17:45,  5.23s/it]
 65%|██████▌   | 146331/225000 [288:56:57<121:18:54,  5.55s/it]
 65%|██████▌   | 146332/225000 [288:57:09<163:53:37,  7.50s/it]
 65%|██████▌   | 146333/225000 [288:57:13<145:22:00,  6.65s/it]
 65%|██████▌   | 146334/225000 [288:57:18<135:50:03,  6.22s/it]
 65%|██████▌   | 146335/225000 [288:57:26<146:14:43,  6.69s/it]
 65%|██████▌   | 146336/225000 [288:57:30<125:28:23,  5.74s/it]
 65%|██████▌   | 146337/225000 [288:57:39<146:16:26,  6.69s/it]
 65%|██████▌   | 146338/225000 [288:57:43<131:59:24,  6.04s/it]
 65%|██████▌   | 146339/225000 [288:57:50<135:38:07,  6.21s/it]
 65%|██████▌   | 146340/225000 [288:57:56<134:35:02,  6.16s/it]
 65%|██████▌   | 146341/225000 [288:57:59<117:22:31,  5.37s/it]
 65%|██████▌   | 146342/225000 [288:58:05<120:16:15,  5.50s/it]
 65%|██████▌   | 146343/225000 [288:58:12<127:35:23,  5.84s/it]
 65%|██████▌   | 146344/225000 [288:58:16<116:24:19,  5.33s/it]
 65%|██████▌   | 146345/225000 [288:58:25<137:37:56,  6.30s/it]
 65%|██████▌   | 146346/225000 [288:58:29<124:38:46,  5.71s/it]
 65%|██████▌   | 146347/225000 [288:58:34<122:21:13,  5.60s/it]
 65%|██████▌   | 146348/225000 [288:58:42<135:32:33,  6.20s/it]
 65%|██████▌   | 146349/225000 [288:58:46<121:44:27,  5.57s/it]
 65%|██████▌   | 146350/225000 [288:58:50<115:10:08,  5.27s/it]
                                                               
{'loss': '0.7821', 'grad_norm': '1.246', 'learning_rate': '1.479e-05', 'epoch': '0.6504', 'num_input_tokens_seen': 478992696, 'train_runtime': '1.04e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146350/225000 [288:58:50<115:10:08,  5.27s/it]
 65%|██████▌   | 146351/225000 [288:58:55<110:06:56,  5.04s/it]
 65%|██████▌   | 146352/225000 [288:58:59<105:06:19,  4.81s/it]
 65%|██████▌   | 146353/225000 [288:59:10<142:15:27,  6.51s/it]
 65%|██████▌   | 146354/225000 [288:59:14<127:10:20,  5.82s/it]
 65%|██████▌   | 146355/225000 [288:59:18<115:08:19,  5.27s/it]
 65%|██████▌   | 146356/225000 [288:59:22<108:21:09,  4.96s/it]
 65%|██████▌   | 146357/225000 [288:59:32<138:05:19,  6.32s/it]
 65%|██████▌   | 146358/225000 [288:59:35<119:34:26,  5.47s/it]
 65%|██████▌   | 146359/225000 [288:59:39<108:16:15,  4.96s/it]
 65%|██████▌   | 146360/225000 [288:59:44<111:55:13,  5.12s/it]
 65%|██████▌   | 146361/225000 [288:59:54<138:54:59,  6.36s/it]
 65%|██████▌   | 146362/225000 [288:59:57<119:39:48,  5.48s/it]
 65%|██████▌   | 146363/225000 [289:00:06<143:55:55,  6.59s/it]
 65%|██████▌   | 146364/225000 [289:00:11<134:14:13,  6.15s/it]
 65%|██████▌   | 146365/225000 [289:00:18<136:35:17,  6.25s/it]
 65%|██████▌   | 146366/225000 [289:00:22<121:10:44,  5.55s/it]
 65%|██████▌   | 146367/225000 [289:00:26<111:02:30,  5.08s/it]
 65%|██████▌   | 146368/225000 [289:00:32<117:15:58,  5.37s/it]
 65%|██████▌   | 146369/225000 [289:00:40<134:51:43,  6.17s/it]
 65%|██████▌   | 146370/225000 [289:00:43<117:33:00,  5.38s/it]
 65%|██████▌   | 146371/225000 [289:00:47<106:20:22,  4.87s/it]
 65%|██████▌   | 146372/225000 [289:00:57<138:44:05,  6.35s/it]
 65%|██████▌   | 146373/225000 [289:01:00<119:06:40,  5.45s/it]
 65%|██████▌   | 146374/225000 [289:01:09<137:47:34,  6.31s/it]
 65%|██████▌   | 146375/225000 [289:01:12<119:00:37,  5.45s/it]
 65%|██████▌   | 146376/225000 [289:01:16<112:46:42,  5.16s/it]
 65%|██████▌   | 146377/225000 [289:01:21<107:57:30,  4.94s/it]
 65%|██████▌   | 146378/225000 [289:01:25<104:20:59,  4.78s/it]
 65%|██████▌   | 146379/225000 [289:01:30<102:13:45,  4.68s/it]
 65%|██████▌   | 146380/225000 [289:01:35<104:11:30,  4.77s/it]
 65%|██████▌   | 146381/225000 [289:01:47<154:08:14,  7.06s/it]
 65%|██████▌   | 146382/225000 [289:01:51<134:31:52,  6.16s/it]
 65%|██████▌   | 146383/225000 [289:01:59<144:58:02,  6.64s/it]
 65%|██████▌   | 146384/225000 [289:02:02<122:19:44,  5.60s/it]
 65%|██████▌   | 146385/225000 [289:02:10<135:43:42,  6.22s/it]
 65%|██████▌   | 146386/225000 [289:02:15<127:51:39,  5.86s/it]
 65%|██████▌   | 146387/225000 [289:02:20<122:19:32,  5.60s/it]
 65%|██████▌   | 146388/225000 [289:02:25<120:10:54,  5.50s/it]
 65%|██████▌   | 146389/225000 [289:02:30<115:30:02,  5.29s/it]
 65%|██████▌   | 146390/225000 [289:02:35<115:06:26,  5.27s/it]
 65%|██████▌   | 146391/225000 [289:02:40<114:07:29,  5.23s/it]
 65%|██████▌   | 146392/225000 [289:02:46<114:56:51,  5.26s/it]
 65%|██████▌   | 146393/225000 [289:02:51<114:35:24,  5.25s/it]
 65%|██████▌   | 146394/225000 [289:02:56<113:12:06,  5.18s/it]
 65%|██████▌   | 146395/225000 [289:03:01<112:12:59,  5.14s/it]
 65%|██████▌   | 146396/225000 [289:03:05<108:31:53,  4.97s/it]
 65%|██████▌   | 146397/225000 [289:03:10<104:53:16,  4.80s/it]
 65%|██████▌   | 146398/225000 [289:03:15<106:36:34,  4.88s/it]
 65%|██████▌   | 146399/225000 [289:03:27<155:35:08,  7.13s/it]
 65%|██████▌   | 146400/225000 [289:03:34<152:09:56,  6.97s/it]
                                                               
{'loss': '0.792', 'grad_norm': '1.149', 'learning_rate': '1.478e-05', 'epoch': '0.6507', 'num_input_tokens_seen': 479121864, 'train_runtime': '1.041e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146400/225000 [289:03:34<152:09:56,  6.97s/it]
 65%|██████▌   | 146401/225000 [289:03:38<131:00:23,  6.00s/it]
 65%|██████▌   | 146402/225000 [289:03:43<126:25:39,  5.79s/it]
 65%|██████▌   | 146403/225000 [289:03:48<120:53:57,  5.54s/it]
 65%|██████▌   | 146404/225000 [289:03:53<117:49:38,  5.40s/it]
 65%|██████▌   | 146405/225000 [289:03:58<116:36:08,  5.34s/it]
 65%|██████▌   | 146406/225000 [289:04:04<116:45:42,  5.35s/it]
 65%|██████▌   | 146407/225000 [289:04:09<116:31:21,  5.34s/it]
 65%|██████▌   | 146408/225000 [289:04:14<118:28:23,  5.43s/it]
 65%|██████▌   | 146409/225000 [289:04:20<117:40:28,  5.39s/it]
 65%|██████▌   | 146410/225000 [289:04:25<115:45:32,  5.30s/it]
 65%|██████▌   | 146411/225000 [289:04:32<130:54:29,  6.00s/it]
 65%|██████▌   | 146412/225000 [289:04:43<157:36:27,  7.22s/it]
 65%|██████▌   | 146413/225000 [289:04:46<135:07:42,  6.19s/it]
 65%|██████▌   | 146414/225000 [289:04:52<129:11:32,  5.92s/it]
 65%|██████▌   | 146415/225000 [289:04:57<127:16:37,  5.83s/it]
 65%|██████▌   | 146416/225000 [289:05:03<124:32:37,  5.71s/it]
 65%|██████▌   | 146417/225000 [289:05:08<120:11:13,  5.51s/it]
 65%|██████▌   | 146418/225000 [289:05:16<136:09:25,  6.24s/it]
 65%|██████▌   | 146419/225000 [289:05:22<138:04:08,  6.33s/it]
 65%|██████▌   | 146420/225000 [289:05:28<137:49:58,  6.31s/it]
 65%|██████▌   | 146421/225000 [289:05:33<123:59:47,  5.68s/it]
 65%|██████▌   | 146422/225000 [289:05:37<113:36:08,  5.20s/it]
 65%|██████▌   | 146423/225000 [289:05:43<121:52:36,  5.58s/it]
 65%|██████▌   | 146424/225000 [289:05:50<129:00:08,  5.91s/it]
 65%|██████▌   | 146425/225000 [289:05:54<118:33:42,  5.43s/it]
 65%|██████▌   | 146426/225000 [289:06:02<133:37:22,  6.12s/it]
 65%|██████▌   | 146427/225000 [289:06:08<134:21:44,  6.16s/it]
 65%|██████▌   | 146428/225000 [289:06:12<119:25:47,  5.47s/it]
 65%|██████▌   | 146429/225000 [289:06:21<142:32:32,  6.53s/it]
 65%|██████▌   | 146430/225000 [289:06:24<121:50:58,  5.58s/it]
 65%|██████▌   | 146431/225000 [289:06:32<132:58:56,  6.09s/it]
 65%|██████▌   | 146432/225000 [289:06:36<119:00:29,  5.45s/it]
 65%|██████▌   | 146433/225000 [289:06:40<110:29:46,  5.06s/it]
 65%|██████▌   | 146434/225000 [289:06:44<105:56:49,  4.85s/it]
 65%|██████▌   | 146435/225000 [289:06:49<102:41:18,  4.71s/it]
 65%|██████▌   | 146436/225000 [289:06:59<142:41:52,  6.54s/it]
 65%|██████▌   | 146437/225000 [289:07:04<133:20:30,  6.11s/it]
 65%|██████▌   | 146438/225000 [289:07:10<128:17:34,  5.88s/it]
 65%|██████▌   | 146439/225000 [289:07:17<136:46:08,  6.27s/it]
 65%|██████▌   | 146440/225000 [289:07:21<120:17:55,  5.51s/it]
 65%|██████▌   | 146441/225000 [289:07:25<110:05:22,  5.04s/it]
 65%|██████▌   | 146442/225000 [289:07:33<128:31:11,  5.89s/it]
 65%|██████▌   | 146443/225000 [289:07:37<118:12:11,  5.42s/it]
 65%|██████▌   | 146444/225000 [289:07:41<112:31:15,  5.16s/it]
 65%|██████▌   | 146445/225000 [289:07:46<108:36:53,  4.98s/it]
 65%|██████▌   | 146446/225000 [289:07:50<103:30:17,  4.74s/it]
 65%|██████▌   | 146447/225000 [289:08:02<152:16:01,  6.98s/it]
 65%|██████▌   | 146448/225000 [289:08:07<138:49:56,  6.36s/it]
 65%|██████▌   | 146449/225000 [289:08:15<146:03:50,  6.69s/it]
 65%|██████▌   | 146450/225000 [289:08:19<127:28:00,  5.84s/it]
                                                               
{'loss': '0.7873', 'grad_norm': '1.022', 'learning_rate': '1.476e-05', 'epoch': '0.6509', 'num_input_tokens_seen': 479252832, 'train_runtime': '1.041e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146450/225000 [289:08:19<127:28:00,  5.84s/it]
 65%|██████▌   | 146451/225000 [289:08:23<116:04:56,  5.32s/it]
 65%|██████▌   | 146452/225000 [289:08:27<109:04:56,  5.00s/it]
 65%|██████▌   | 146453/225000 [289:08:31<100:13:05,  4.59s/it]
 65%|██████▌   | 146454/225000 [289:08:43<151:45:28,  6.96s/it]
 65%|██████▌   | 146455/225000 [289:08:47<134:08:06,  6.15s/it]
 65%|██████▌   | 146456/225000 [289:08:52<122:01:07,  5.59s/it]
 65%|██████▌   | 146457/225000 [289:08:56<113:58:58,  5.22s/it]
 65%|██████▌   | 146458/225000 [289:09:02<116:54:51,  5.36s/it]
 65%|██████▌   | 146459/225000 [289:09:10<136:54:34,  6.28s/it]
 65%|██████▌   | 146460/225000 [289:09:14<122:33:11,  5.62s/it]
 65%|██████▌   | 146461/225000 [289:09:18<111:14:56,  5.10s/it]
 65%|██████▌   | 146462/225000 [289:09:26<132:02:35,  6.05s/it]
 65%|██████▌   | 146463/225000 [289:09:34<139:36:02,  6.40s/it]
 65%|██████▌   | 146464/225000 [289:09:37<120:41:50,  5.53s/it]
 65%|██████▌   | 146465/225000 [289:09:41<110:25:58,  5.06s/it]
 65%|██████▌   | 146466/225000 [289:09:47<115:57:05,  5.32s/it]
 65%|██████▌   | 146467/225000 [289:09:56<139:54:13,  6.41s/it]
 65%|██████▌   | 146468/225000 [289:09:59<120:36:49,  5.53s/it]
 65%|██████▌   | 146469/225000 [289:10:05<120:21:53,  5.52s/it]
 65%|██████▌   | 146470/225000 [289:10:12<131:06:17,  6.01s/it]
 65%|██████▌   | 146471/225000 [289:10:17<123:47:57,  5.68s/it]
 65%|██████▌   | 146472/225000 [289:10:23<126:35:39,  5.80s/it]
 65%|██████▌   | 146473/225000 [289:10:27<113:11:55,  5.19s/it]
 65%|██████▌   | 146474/225000 [289:10:32<112:06:27,  5.14s/it]
 65%|██████▌   | 146475/225000 [289:10:40<132:25:50,  6.07s/it]
 65%|██████▌   | 146476/225000 [289:10:44<115:37:37,  5.30s/it]
 65%|██████▌   | 146477/225000 [289:10:47<105:51:18,  4.85s/it]
 65%|██████▌   | 146478/225000 [289:10:55<125:48:25,  5.77s/it]
 65%|██████▌   | 146479/225000 [289:11:02<129:52:59,  5.95s/it]
 65%|██████▌   | 146480/225000 [289:11:05<114:13:39,  5.24s/it]
 65%|██████▌   | 146481/225000 [289:11:11<117:54:05,  5.41s/it]
 65%|██████▌   | 146482/225000 [289:11:20<141:15:16,  6.48s/it]
 65%|██████▌   | 146483/225000 [289:11:23<120:38:12,  5.53s/it]
 65%|██████▌   | 146484/225000 [289:11:30<129:47:06,  5.95s/it]
 65%|██████▌   | 146485/225000 [289:11:35<122:35:55,  5.62s/it]
 65%|██████▌   | 146486/225000 [289:11:42<127:53:19,  5.86s/it]
 65%|██████▌   | 146487/225000 [289:11:45<112:40:28,  5.17s/it]
 65%|██████▌   | 146488/225000 [289:11:52<122:01:37,  5.60s/it]
 65%|██████▌   | 146489/225000 [289:11:57<123:06:57,  5.65s/it]
 65%|██████▌   | 146490/225000 [289:12:03<121:46:49,  5.58s/it]
 65%|██████▌   | 146491/225000 [289:12:08<121:29:57,  5.57s/it]
 65%|██████▌   | 146492/225000 [289:12:13<113:43:32,  5.21s/it]
 65%|██████▌   | 146493/225000 [289:12:17<104:40:52,  4.80s/it]
 65%|██████▌   | 146494/225000 [289:12:21<101:48:43,  4.67s/it]
 65%|██████▌   | 146495/225000 [289:12:25<99:36:33,  4.57s/it] 
 65%|██████▌   | 146496/225000 [289:12:30<97:59:13,  4.49s/it]
 65%|██████▌   | 146497/225000 [289:12:39<126:58:04,  5.82s/it]
 65%|██████▌   | 146498/225000 [289:12:47<141:22:21,  6.48s/it]
 65%|██████▌   | 146499/225000 [289:12:52<131:22:47,  6.02s/it]
 65%|██████▌   | 146500/225000 [289:12:57<128:04:28,  5.87s/it]
                                                               
{'loss': '0.7891', 'grad_norm': '1.088', 'learning_rate': '1.474e-05', 'epoch': '0.6511', 'num_input_tokens_seen': 479382528, 'train_runtime': '1.041e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146500/225000 [289:12:57<128:04:28,  5.87s/it]
 65%|██████▌   | 146501/225000 [289:13:02<122:45:20,  5.63s/it]
 65%|██████▌   | 146502/225000 [289:13:07<117:00:08,  5.37s/it]
 65%|██████▌   | 146503/225000 [289:13:12<115:44:05,  5.31s/it]
 65%|██████▌   | 146504/225000 [289:13:17<114:09:43,  5.24s/it]
 65%|██████▌   | 146505/225000 [289:13:22<114:15:18,  5.24s/it]
 65%|██████▌   | 146506/225000 [289:13:27<112:54:02,  5.18s/it]
 65%|██████▌   | 146507/225000 [289:13:32<109:59:13,  5.04s/it]
 65%|██████▌   | 146508/225000 [289:13:37<107:43:30,  4.94s/it]
 65%|██████▌   | 146509/225000 [289:13:42<107:52:51,  4.95s/it]
 65%|██████▌   | 146510/225000 [289:13:47<110:30:59,  5.07s/it]
 65%|██████▌   | 146511/225000 [289:13:52<110:40:09,  5.08s/it]
 65%|██████▌   | 146512/225000 [289:13:57<109:06:22,  5.00s/it]
 65%|██████▌   | 146513/225000 [289:14:02<109:58:58,  5.04s/it]
 65%|██████▌   | 146514/225000 [289:14:15<159:28:38,  7.31s/it]
 65%|██████▌   | 146515/225000 [289:14:20<146:14:05,  6.71s/it]
 65%|██████▌   | 146516/225000 [289:14:24<128:18:40,  5.89s/it]
 65%|██████▌   | 146517/225000 [289:14:29<120:35:33,  5.53s/it]
 65%|██████▌   | 146518/225000 [289:14:33<113:01:43,  5.18s/it]
 65%|██████▌   | 146519/225000 [289:14:42<136:35:36,  6.27s/it]
 65%|██████▌   | 146520/225000 [289:14:50<146:23:30,  6.72s/it]
 65%|██████▌   | 146521/225000 [289:14:55<135:26:13,  6.21s/it]
 65%|██████▌   | 146522/225000 [289:14:59<123:45:32,  5.68s/it]
 65%|██████▌   | 146523/225000 [289:15:04<118:59:39,  5.46s/it]
 65%|██████▌   | 146524/225000 [289:15:09<115:05:24,  5.28s/it]
 65%|██████▌   | 146525/225000 [289:15:14<111:55:07,  5.13s/it]
 65%|██████▌   | 146526/225000 [289:15:19<109:56:42,  5.04s/it]
 65%|██████▌   | 146527/225000 [289:15:23<106:38:25,  4.89s/it]
 65%|██████▌   | 146528/225000 [289:15:36<155:28:38,  7.13s/it]
 65%|██████▌   | 146529/225000 [289:15:39<132:25:48,  6.08s/it]
 65%|██████▌   | 146530/225000 [289:15:47<142:55:46,  6.56s/it]
 65%|██████▌   | 146531/225000 [289:15:53<138:12:32,  6.34s/it]
 65%|██████▌   | 146532/225000 [289:16:00<142:30:20,  6.54s/it]
 65%|██████▌   | 146533/225000 [289:16:03<120:12:07,  5.51s/it]
 65%|██████▌   | 146534/225000 [289:16:09<123:22:12,  5.66s/it]
 65%|██████▌   | 146535/225000 [289:16:16<131:23:18,  6.03s/it]
 65%|██████▌   | 146536/225000 [289:16:19<113:19:47,  5.20s/it]
 65%|██████▌   | 146537/225000 [289:16:27<132:34:41,  6.08s/it]
 65%|██████▌   | 146538/225000 [289:16:30<114:30:21,  5.25s/it]
 65%|██████▌   | 146539/225000 [289:16:34<104:21:24,  4.79s/it]
 65%|██████▌   | 146540/225000 [289:16:43<132:23:55,  6.07s/it]
 65%|██████▌   | 146541/225000 [289:16:47<115:39:06,  5.31s/it]
 65%|██████▌   | 146542/225000 [289:16:53<119:46:13,  5.50s/it]
 65%|██████▌   | 146543/225000 [289:16:57<109:50:20,  5.04s/it]
 65%|██████▌   | 146544/225000 [289:17:00<99:47:04,  4.58s/it] 
 65%|██████▌   | 146545/225000 [289:17:07<116:38:31,  5.35s/it]
 65%|██████▌   | 146546/225000 [289:17:13<121:26:33,  5.57s/it]
 65%|██████▌   | 146547/225000 [289:17:18<113:45:03,  5.22s/it]
 65%|██████▌   | 146548/225000 [289:17:23<114:59:39,  5.28s/it]
 65%|██████▌   | 146549/225000 [289:17:28<114:38:25,  5.26s/it]
 65%|██████▌   | 146550/225000 [289:17:34<114:27:31,  5.25s/it]
                                                               
{'loss': '0.7854', 'grad_norm': '1.045', 'learning_rate': '1.473e-05', 'epoch': '0.6513', 'num_input_tokens_seen': 479512352, 'train_runtime': '1.041e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146550/225000 [289:17:34<114:27:31,  5.25s/it]
 65%|██████▌   | 146551/225000 [289:17:39<113:35:30,  5.21s/it]
 65%|██████▌   | 146552/225000 [289:17:44<113:04:36,  5.19s/it]
 65%|██████▌   | 146553/225000 [289:17:50<116:05:47,  5.33s/it]
 65%|██████▌   | 146554/225000 [289:17:55<115:57:58,  5.32s/it]
 65%|██████▌   | 146555/225000 [289:18:00<114:30:27,  5.25s/it]
 65%|██████▌   | 146556/225000 [289:18:05<112:54:46,  5.18s/it]
 65%|██████▌   | 146557/225000 [289:18:10<112:12:25,  5.15s/it]
 65%|██████▌   | 146558/225000 [289:18:16<115:11:54,  5.29s/it]
 65%|██████▌   | 146559/225000 [289:18:28<159:58:50,  7.34s/it]
 65%|██████▌   | 146560/225000 [289:18:32<139:26:00,  6.40s/it]
 65%|██████▌   | 146561/225000 [289:18:40<151:22:58,  6.95s/it]
 65%|██████▌   | 146562/225000 [289:18:44<130:05:32,  5.97s/it]
 65%|██████▌   | 146563/225000 [289:18:48<120:38:33,  5.54s/it]
 65%|██████▌   | 146564/225000 [289:18:52<109:35:43,  5.03s/it]
 65%|██████▌   | 146565/225000 [289:19:03<143:58:13,  6.61s/it]
 65%|██████▌   | 146566/225000 [289:19:06<121:18:01,  5.57s/it]
 65%|██████▌   | 146567/225000 [289:19:13<133:25:54,  6.12s/it]
 65%|██████▌   | 146568/225000 [289:19:18<122:39:23,  5.63s/it]
 65%|██████▌   | 146569/225000 [289:19:22<114:21:42,  5.25s/it]
 65%|██████▌   | 146570/225000 [289:19:27<110:42:26,  5.08s/it]
 65%|██████▌   | 146571/225000 [289:19:32<111:05:17,  5.10s/it]
 65%|██████▌   | 146572/225000 [289:19:37<110:09:18,  5.06s/it]
 65%|██████▌   | 146573/225000 [289:19:42<110:03:53,  5.05s/it]
 65%|██████▌   | 146574/225000 [289:19:47<111:12:50,  5.11s/it]
 65%|██████▌   | 146575/225000 [289:19:52<110:55:30,  5.09s/it]
 65%|██████▌   | 146576/225000 [289:19:57<108:47:06,  4.99s/it]
 65%|██████▌   | 146577/225000 [289:20:02<108:14:50,  4.97s/it]
 65%|██████▌   | 146578/225000 [289:20:07<111:15:22,  5.11s/it]
 65%|██████▌   | 146579/225000 [289:20:20<162:48:32,  7.47s/it]
 65%|██████▌   | 146580/225000 [289:20:26<152:39:50,  7.01s/it]
 65%|██████▌   | 146581/225000 [289:20:30<132:17:18,  6.07s/it]
 65%|██████▌   | 146582/225000 [289:20:35<126:12:42,  5.79s/it]
 65%|██████▌   | 146583/225000 [289:20:40<122:30:46,  5.62s/it]
 65%|██████▌   | 146584/225000 [289:20:45<114:34:44,  5.26s/it]
 65%|██████▌   | 146585/225000 [289:20:50<111:58:46,  5.14s/it]
 65%|██████▌   | 146586/225000 [289:20:54<108:20:57,  4.97s/it]
 65%|██████▌   | 146587/225000 [289:20:59<107:03:58,  4.92s/it]
 65%|██████▌   | 146588/225000 [289:21:04<109:47:09,  5.04s/it]
 65%|██████▌   | 146589/225000 [289:21:16<151:40:00,  6.96s/it]
 65%|██████▌   | 146590/225000 [289:21:19<128:08:34,  5.88s/it]
 65%|██████▌   | 146591/225000 [289:21:26<134:46:34,  6.19s/it]
 65%|██████▌   | 146592/225000 [289:21:31<125:09:08,  5.75s/it]
 65%|██████▌   | 146593/225000 [289:21:35<116:46:33,  5.36s/it]
 65%|██████▌   | 146594/225000 [289:21:40<110:47:12,  5.09s/it]
 65%|██████▌   | 146595/225000 [289:21:47<126:39:50,  5.82s/it]
 65%|██████▌   | 146596/225000 [289:21:55<139:32:40,  6.41s/it]
 65%|██████▌   | 146597/225000 [289:21:58<119:39:44,  5.49s/it]
 65%|██████▌   | 146598/225000 [289:22:03<113:00:59,  5.19s/it]
 65%|██████▌   | 146599/225000 [289:22:08<114:26:49,  5.26s/it]
 65%|██████▌   | 146600/225000 [289:22:13<114:35:14,  5.26s/it]
                                                               
{'loss': '0.7786', 'grad_norm': '1.089', 'learning_rate': '1.471e-05', 'epoch': '0.6516', 'num_input_tokens_seen': 479641440, 'train_runtime': '1.042e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146600/225000 [289:22:13<114:35:14,  5.26s/it]
 65%|██████▌   | 146601/225000 [289:22:19<114:34:54,  5.26s/it]
 65%|██████▌   | 146602/225000 [289:22:24<114:45:11,  5.27s/it]
 65%|██████▌   | 146603/225000 [289:22:29<112:56:28,  5.19s/it]
 65%|██████▌   | 146604/225000 [289:22:34<110:24:53,  5.07s/it]
 65%|██████▌   | 146605/225000 [289:22:39<110:31:48,  5.08s/it]
 65%|██████▌   | 146606/225000 [289:22:44<111:03:40,  5.10s/it]
 65%|██████▌   | 146607/225000 [289:22:56<154:18:57,  7.09s/it]
 65%|██████▌   | 146608/225000 [289:23:03<154:15:52,  7.08s/it]
 65%|██████▌   | 146609/225000 [289:23:07<132:13:30,  6.07s/it]
 65%|██████▌   | 146610/225000 [289:23:13<137:47:31,  6.33s/it]
 65%|██████▌   | 146611/225000 [289:23:18<123:11:23,  5.66s/it]
 65%|██████▌   | 146612/225000 [289:23:22<113:14:43,  5.20s/it]
 65%|██████▌   | 146613/225000 [289:23:32<146:46:43,  6.74s/it]
 65%|██████▌   | 146614/225000 [289:23:36<125:13:01,  5.75s/it]
 65%|██████▌   | 146615/225000 [289:23:41<124:37:49,  5.72s/it]
 65%|██████▌   | 146616/225000 [289:23:47<127:45:08,  5.87s/it]
 65%|██████▌   | 146617/225000 [289:23:52<118:20:38,  5.44s/it]
 65%|██████▌   | 146618/225000 [289:24:01<143:51:49,  6.61s/it]
 65%|██████▌   | 146619/225000 [289:24:05<123:01:03,  5.65s/it]
 65%|██████▌   | 146620/225000 [289:24:08<108:00:00,  4.96s/it]
 65%|██████▌   | 146621/225000 [289:24:16<127:43:17,  5.87s/it]
 65%|██████▌   | 146622/225000 [289:24:20<118:59:48,  5.47s/it]
 65%|██████▌   | 146623/225000 [289:24:25<113:41:00,  5.22s/it]
 65%|██████▌   | 146624/225000 [289:24:29<108:10:51,  4.97s/it]
 65%|██████▌   | 146625/225000 [289:24:34<108:17:48,  4.97s/it]
 65%|██████▌   | 146626/225000 [289:24:39<103:21:56,  4.75s/it]
 65%|██████▌   | 146627/225000 [289:24:43<102:22:10,  4.70s/it]
 65%|██████▌   | 146628/225000 [289:24:56<152:42:39,  7.01s/it]
 65%|██████▌   | 146629/225000 [289:24:59<130:21:00,  5.99s/it]
 65%|██████▌   | 146630/225000 [289:25:04<125:16:39,  5.75s/it]
 65%|██████▌   | 146631/225000 [289:25:13<142:58:22,  6.57s/it]
 65%|██████▌   | 146632/225000 [289:25:17<125:52:21,  5.78s/it]
 65%|██████▌   | 146633/225000 [289:25:20<110:46:16,  5.09s/it]
 65%|██████▌   | 146634/225000 [289:25:25<107:26:17,  4.94s/it]
 65%|██████▌   | 146635/225000 [289:25:30<106:50:17,  4.91s/it]
 65%|██████▌   | 146636/225000 [289:25:35<109:01:01,  5.01s/it]
 65%|██████▌   | 146637/225000 [289:25:40<111:25:53,  5.12s/it]
 65%|██████▌   | 146638/225000 [289:25:45<109:44:06,  5.04s/it]
 65%|██████▌   | 146639/225000 [289:25:50<111:07:11,  5.10s/it]
 65%|██████▌   | 146640/225000 [289:25:56<112:17:01,  5.16s/it]
 65%|██████▌   | 146641/225000 [289:26:01<112:35:13,  5.17s/it]
 65%|██████▌   | 146642/225000 [289:26:12<147:43:32,  6.79s/it]
 65%|██████▌   | 146643/225000 [289:26:19<151:23:49,  6.96s/it]
 65%|██████▌   | 146644/225000 [289:26:22<129:02:58,  5.93s/it]
 65%|██████▌   | 146645/225000 [289:26:27<118:48:57,  5.46s/it]
 65%|██████▌   | 146646/225000 [289:26:31<110:32:32,  5.08s/it]
 65%|██████▌   | 146647/225000 [289:26:35<106:39:19,  4.90s/it]
 65%|██████▌   | 146648/225000 [289:26:48<154:47:07,  7.11s/it]
 65%|██████▌   | 146649/225000 [289:26:54<146:52:44,  6.75s/it]
 65%|██████▌   | 146650/225000 [289:26:59<141:04:36,  6.48s/it]
                                                               
{'loss': '0.7956', 'grad_norm': '0.9785', 'learning_rate': '1.469e-05', 'epoch': '0.6518', 'num_input_tokens_seen': 479770264, 'train_runtime': '1.042e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146650/225000 [289:27:00<141:04:36,  6.48s/it]
 65%|██████▌   | 146651/225000 [289:27:06<138:19:29,  6.36s/it]
 65%|██████▌   | 146652/225000 [289:27:11<131:43:35,  6.05s/it]
 65%|██████▌   | 146653/225000 [289:27:16<125:54:50,  5.79s/it]
 65%|██████▌   | 146654/225000 [289:27:21<121:12:53,  5.57s/it]
 65%|██████▌   | 146655/225000 [289:27:26<117:28:58,  5.40s/it]
 65%|██████▌   | 146656/225000 [289:27:32<118:30:47,  5.45s/it]
 65%|██████▌   | 146657/225000 [289:27:37<119:03:31,  5.47s/it]
 65%|██████▌   | 146658/225000 [289:27:43<118:59:39,  5.47s/it]
 65%|██████▌   | 146659/225000 [289:27:48<116:06:00,  5.34s/it]
 65%|██████▌   | 146660/225000 [289:27:53<114:47:03,  5.27s/it]
 65%|██████▌   | 146661/225000 [289:27:58<115:46:05,  5.32s/it]
 65%|██████▌   | 146662/225000 [289:28:03<114:31:24,  5.26s/it]
 65%|██████▌   | 146663/225000 [289:28:12<136:31:37,  6.27s/it]
 65%|██████▌   | 146664/225000 [289:28:17<131:09:27,  6.03s/it]
 65%|██████▌   | 146665/225000 [289:28:23<124:46:32,  5.73s/it]
 65%|██████▌   | 146666/225000 [289:28:31<139:28:51,  6.41s/it]
 65%|██████▌   | 146667/225000 [289:28:36<133:55:20,  6.15s/it]
 65%|██████▌   | 146668/225000 [289:28:41<122:58:05,  5.65s/it]
 65%|██████▌   | 146669/225000 [289:28:45<114:46:18,  5.27s/it]
 65%|██████▌   | 146670/225000 [289:28:53<131:14:21,  6.03s/it]
 65%|██████▌   | 146671/225000 [289:28:59<133:44:43,  6.15s/it]
 65%|██████▌   | 146672/225000 [289:29:02<114:31:29,  5.26s/it]
 65%|██████▌   | 146673/225000 [289:29:07<107:33:45,  4.94s/it]
 65%|██████▌   | 146674/225000 [289:29:11<101:39:46,  4.67s/it]
 65%|██████▌   | 146675/225000 [289:29:22<143:14:28,  6.58s/it]
 65%|██████▌   | 146676/225000 [289:29:26<129:49:05,  5.97s/it]
 65%|██████▌   | 146677/225000 [289:29:30<116:32:08,  5.36s/it]
 65%|██████▌   | 146678/225000 [289:29:39<140:13:33,  6.45s/it]
 65%|██████▌   | 146679/225000 [289:29:42<120:14:23,  5.53s/it]
 65%|██████▌   | 146680/225000 [289:29:51<138:06:56,  6.35s/it]
 65%|██████▌   | 146681/225000 [289:29:54<118:13:28,  5.43s/it]
 65%|██████▌   | 146682/225000 [289:30:02<137:19:45,  6.31s/it]
 65%|██████▌   | 146683/225000 [289:30:06<121:43:42,  5.60s/it]
 65%|██████▌   | 146684/225000 [289:30:13<126:48:04,  5.83s/it]
 65%|██████▌   | 146685/225000 [289:30:16<110:34:07,  5.08s/it]
 65%|██████▌   | 146686/225000 [289:30:23<123:45:54,  5.69s/it]
 65%|██████▌   | 146687/225000 [289:30:30<129:11:11,  5.94s/it]
 65%|██████▌   | 146688/225000 [289:30:33<113:35:46,  5.22s/it]
 65%|██████▌   | 146689/225000 [289:30:39<117:19:48,  5.39s/it]
 65%|██████▌   | 146690/225000 [289:30:45<121:39:20,  5.59s/it]
 65%|██████▌   | 146691/225000 [289:30:49<111:28:05,  5.12s/it]
 65%|██████▌   | 146692/225000 [289:30:53<102:04:02,  4.69s/it]
 65%|██████▌   | 146693/225000 [289:31:02<131:47:31,  6.06s/it]
 65%|██████▌   | 146694/225000 [289:31:05<113:05:29,  5.20s/it]
 65%|██████▌   | 146695/225000 [289:31:09<107:01:38,  4.92s/it]
 65%|██████▌   | 146696/225000 [289:31:16<114:17:06,  5.25s/it]
 65%|██████▌   | 146697/225000 [289:31:24<137:07:41,  6.30s/it]
 65%|██████▌   | 146698/225000 [289:31:28<120:58:26,  5.56s/it]
 65%|██████▌   | 146699/225000 [289:31:32<109:22:13,  5.03s/it]
 65%|██████▌   | 146700/225000 [289:31:41<137:22:00,  6.32s/it]
                                                               
{'loss': '0.7885', 'grad_norm': '1.04', 'learning_rate': '1.468e-05', 'epoch': '0.652', 'num_input_tokens_seen': 479899328, 'train_runtime': '1.042e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146700/225000 [289:31:41<137:22:00,  6.32s/it]
 65%|██████▌   | 146701/225000 [289:31:47<135:50:24,  6.25s/it]
 65%|██████▌   | 146702/225000 [289:31:52<122:57:42,  5.65s/it]
 65%|██████▌   | 146703/225000 [289:31:57<123:50:49,  5.69s/it]
 65%|██████▌   | 146704/225000 [289:32:01<108:59:40,  5.01s/it]
 65%|██████▌   | 146705/225000 [289:32:05<105:50:31,  4.87s/it]
 65%|██████▌   | 146706/225000 [289:32:16<141:22:01,  6.50s/it]
 65%|██████▌   | 146707/225000 [289:32:19<120:25:22,  5.54s/it]
 65%|██████▌   | 146708/225000 [289:32:24<117:57:39,  5.42s/it]
 65%|██████▌   | 146709/225000 [289:32:31<129:40:08,  5.96s/it]
 65%|██████▌   | 146710/225000 [289:32:34<111:38:54,  5.13s/it]
 65%|██████▌   | 146711/225000 [289:32:42<129:34:11,  5.96s/it]
 65%|██████▌   | 146712/225000 [289:32:46<114:15:19,  5.25s/it]
 65%|██████▌   | 146713/225000 [289:32:53<127:09:09,  5.85s/it]
 65%|██████▌   | 146714/225000 [289:32:59<127:11:18,  5.85s/it]
 65%|██████▌   | 146715/225000 [289:33:02<111:26:39,  5.12s/it]
 65%|██████▌   | 146716/225000 [289:33:11<135:12:37,  6.22s/it]
 65%|██████▌   | 146717/225000 [289:33:15<118:39:48,  5.46s/it]
 65%|██████▌   | 146718/225000 [289:33:18<105:54:05,  4.87s/it]
 65%|██████▌   | 146719/225000 [289:33:28<134:50:32,  6.20s/it]
 65%|██████▌   | 146720/225000 [289:33:31<118:19:12,  5.44s/it]
 65%|██████▌   | 146721/225000 [289:33:37<116:21:25,  5.35s/it]
 65%|██████▌   | 146722/225000 [289:33:45<133:27:43,  6.14s/it]
 65%|██████▌   | 146723/225000 [289:33:50<127:05:03,  5.84s/it]
 65%|██████▌   | 146724/225000 [289:33:58<141:05:43,  6.49s/it]
 65%|██████▌   | 146725/225000 [289:34:02<127:31:23,  5.87s/it]
 65%|██████▌   | 146726/225000 [289:34:06<112:25:43,  5.17s/it]
 65%|██████▌   | 146727/225000 [289:34:10<107:18:06,  4.94s/it]
 65%|██████▌   | 146728/225000 [289:34:15<107:42:25,  4.95s/it]
 65%|██████▌   | 146729/225000 [289:34:24<134:41:38,  6.20s/it]
 65%|██████▌   | 146730/225000 [289:34:28<116:45:48,  5.37s/it]
 65%|██████▌   | 146731/225000 [289:34:33<118:10:04,  5.44s/it]
 65%|██████▌   | 146732/225000 [289:34:41<136:19:01,  6.27s/it]
 65%|██████▌   | 146733/225000 [289:34:46<122:54:36,  5.65s/it]
 65%|██████▌   | 146734/225000 [289:34:50<115:58:15,  5.33s/it]
 65%|██████▌   | 146735/225000 [289:34:55<111:35:45,  5.13s/it]
 65%|██████▌   | 146736/225000 [289:34:59<104:09:42,  4.79s/it]
 65%|██████▌   | 146737/225000 [289:35:03<101:59:33,  4.69s/it]
 65%|██████▌   | 146738/225000 [289:35:08<101:49:50,  4.68s/it]
 65%|██████▌   | 146739/225000 [289:35:16<126:01:16,  5.80s/it]
 65%|██████▌   | 146740/225000 [289:35:25<145:35:23,  6.70s/it]
 65%|██████▌   | 146741/225000 [289:35:29<127:43:26,  5.88s/it]
 65%|██████▌   | 146742/225000 [289:35:33<117:16:38,  5.39s/it]
 65%|██████▌   | 146743/225000 [289:35:40<125:04:39,  5.75s/it]
 65%|██████▌   | 146744/225000 [289:35:48<138:22:53,  6.37s/it]
 65%|██████▌   | 146745/225000 [289:35:52<122:51:19,  5.65s/it]
 65%|██████▌   | 146746/225000 [289:35:56<114:02:07,  5.25s/it]
 65%|██████▌   | 146747/225000 [289:36:00<107:26:50,  4.94s/it]
 65%|██████▌   | 146748/225000 [289:36:11<146:17:09,  6.73s/it]
 65%|██████▌   | 146749/225000 [289:36:15<128:14:13,  5.90s/it]
 65%|██████▌   | 146750/225000 [289:36:19<115:34:06,  5.32s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '1.142', 'learning_rate': '1.466e-05', 'epoch': '0.6522', 'num_input_tokens_seen': 480029064, 'train_runtime': '1.043e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146750/225000 [289:36:19<115:34:06,  5.32s/it]
 65%|██████▌   | 146751/225000 [289:36:23<106:42:02,  4.91s/it]
 65%|██████▌   | 146752/225000 [289:36:34<145:18:25,  6.69s/it]
 65%|██████▌   | 146753/225000 [289:36:38<125:48:32,  5.79s/it]
 65%|██████▌   | 146754/225000 [289:36:46<140:48:40,  6.48s/it]
 65%|██████▌   | 146755/225000 [289:36:50<124:27:54,  5.73s/it]
 65%|██████▌   | 146756/225000 [289:36:53<110:55:03,  5.10s/it]
 65%|██████▌   | 146757/225000 [289:37:03<138:24:48,  6.37s/it]
 65%|██████▌   | 146758/225000 [289:37:07<123:28:40,  5.68s/it]
 65%|██████▌   | 146759/225000 [289:37:10<111:05:39,  5.11s/it]
 65%|██████▌   | 146760/225000 [289:37:15<107:03:32,  4.93s/it]
 65%|██████▌   | 146761/225000 [289:37:22<122:10:49,  5.62s/it]
 65%|██████▌   | 146762/225000 [289:37:31<140:34:02,  6.47s/it]
 65%|██████▌   | 146763/225000 [289:37:34<122:34:52,  5.64s/it]
 65%|██████▌   | 146764/225000 [289:37:38<111:41:26,  5.14s/it]
 65%|██████▌   | 146765/225000 [289:37:44<116:06:22,  5.34s/it]
 65%|██████▌   | 146766/225000 [289:37:52<132:41:23,  6.11s/it]
 65%|██████▌   | 146767/225000 [289:37:56<119:55:03,  5.52s/it]
 65%|██████▌   | 146768/225000 [289:38:00<110:16:39,  5.07s/it]
 65%|██████▌   | 146769/225000 [289:38:04<103:11:52,  4.75s/it]
 65%|██████▌   | 146770/225000 [289:38:14<138:42:48,  6.38s/it]
 65%|██████▌   | 146771/225000 [289:38:20<131:39:44,  6.06s/it]
 65%|██████▌   | 146772/225000 [289:38:27<141:45:57,  6.52s/it]
 65%|██████▌   | 146773/225000 [289:38:31<120:44:20,  5.56s/it]
 65%|██████▌   | 146774/225000 [289:38:34<108:19:25,  4.99s/it]
 65%|██████▌   | 146775/225000 [289:38:38<99:27:38,  4.58s/it] 
 65%|██████▌   | 146776/225000 [289:38:48<132:36:42,  6.10s/it]
 65%|██████▌   | 146777/225000 [289:38:51<117:50:34,  5.42s/it]
 65%|██████▌   | 146778/225000 [289:38:55<104:53:30,  4.83s/it]
 65%|██████▌   | 146779/225000 [289:38:59<98:16:49,  4.52s/it] 
 65%|██████▌   | 146780/225000 [289:39:10<140:50:31,  6.48s/it]
 65%|██████▌   | 146781/225000 [289:39:14<125:10:04,  5.76s/it]
 65%|██████▌   | 146782/225000 [289:39:18<112:05:50,  5.16s/it]
 65%|██████▌   | 146783/225000 [289:39:22<106:47:44,  4.92s/it]
 65%|██████▌   | 146784/225000 [289:39:33<149:10:14,  6.87s/it]
 65%|██████▌   | 146785/225000 [289:39:38<134:40:51,  6.20s/it]
 65%|██████▌   | 146786/225000 [289:39:42<122:51:02,  5.65s/it]
 65%|██████▌   | 146787/225000 [289:39:47<115:17:59,  5.31s/it]
 65%|██████▌   | 146788/225000 [289:39:51<110:22:25,  5.08s/it]
 65%|██████▌   | 146789/225000 [289:39:56<104:47:12,  4.82s/it]
 65%|██████▌   | 146790/225000 [289:40:00<104:56:22,  4.83s/it]
 65%|██████▌   | 146791/225000 [289:40:12<146:15:05,  6.73s/it]
 65%|██████▌   | 146792/225000 [289:40:15<123:56:43,  5.71s/it]
 65%|██████▌   | 146793/225000 [289:40:23<141:17:12,  6.50s/it]
 65%|██████▌   | 146794/225000 [289:40:27<122:36:38,  5.64s/it]
 65%|██████▌   | 146795/225000 [289:40:35<139:43:35,  6.43s/it]
 65%|██████▌   | 146796/225000 [289:40:40<126:21:44,  5.82s/it]
 65%|██████▌   | 146797/225000 [289:40:45<124:30:16,  5.73s/it]
 65%|██████▌   | 146798/225000 [289:40:50<119:47:41,  5.51s/it]
 65%|██████▌   | 146799/225000 [289:40:55<114:54:53,  5.29s/it]
 65%|██████▌   | 146800/225000 [289:41:00<111:04:43,  5.11s/it]
                                                               
{'loss': '0.7879', 'grad_norm': '1.169', 'learning_rate': '1.464e-05', 'epoch': '0.6524', 'num_input_tokens_seen': 480158768, 'train_runtime': '1.043e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146800/225000 [289:41:00<111:04:43,  5.11s/it]
 65%|██████▌   | 146801/225000 [289:41:04<108:29:22,  4.99s/it]
 65%|██████▌   | 146802/225000 [289:41:09<107:38:33,  4.96s/it]
 65%|██████▌   | 146803/225000 [289:41:14<109:07:17,  5.02s/it]
 65%|██████▌   | 146804/225000 [289:41:24<136:43:34,  6.29s/it]
 65%|██████▌   | 146805/225000 [289:41:30<139:51:25,  6.44s/it]
 65%|██████▌   | 146806/225000 [289:41:37<143:10:27,  6.59s/it]
 65%|██████▌   | 146807/225000 [289:41:41<125:15:59,  5.77s/it]
 65%|██████▌   | 146808/225000 [289:41:46<121:33:03,  5.60s/it]
 65%|██████▌   | 146809/225000 [289:41:53<131:25:21,  6.05s/it]
 65%|██████▌   | 146810/225000 [289:41:57<115:51:18,  5.33s/it]
 65%|██████▌   | 146811/225000 [289:42:05<131:47:17,  6.07s/it]
 65%|██████▌   | 146812/225000 [289:42:09<117:02:41,  5.39s/it]
 65%|██████▌   | 146813/225000 [289:42:13<112:11:23,  5.17s/it]
 65%|██████▌   | 146814/225000 [289:42:23<141:08:07,  6.50s/it]
 65%|██████▌   | 146815/225000 [289:42:27<125:21:56,  5.77s/it]
 65%|██████▌   | 146816/225000 [289:42:31<114:38:31,  5.28s/it]
 65%|██████▌   | 146817/225000 [289:42:35<107:23:49,  4.95s/it]
 65%|██████▌   | 146818/225000 [289:42:46<144:50:43,  6.67s/it]
 65%|██████▌   | 146819/225000 [289:42:51<131:23:13,  6.05s/it]
 65%|██████▌   | 146820/225000 [289:42:56<125:39:10,  5.79s/it]
 65%|██████▌   | 146821/225000 [289:43:00<113:15:52,  5.22s/it]
 65%|██████▌   | 146822/225000 [289:43:09<136:53:58,  6.30s/it]
 65%|██████▌   | 146823/225000 [289:43:13<123:09:41,  5.67s/it]
 65%|██████▌   | 146824/225000 [289:43:17<113:31:12,  5.23s/it]
 65%|██████▌   | 146825/225000 [289:43:21<107:19:03,  4.94s/it]
 65%|██████▌   | 146826/225000 [289:43:25<102:10:20,  4.71s/it]
 65%|██████▌   | 146827/225000 [289:43:30<102:22:25,  4.71s/it]
 65%|██████▌   | 146828/225000 [289:43:37<114:45:45,  5.29s/it]
 65%|██████▌   | 146829/225000 [289:43:47<146:41:22,  6.76s/it]
 65%|██████▌   | 146830/225000 [289:43:51<126:09:36,  5.81s/it]
 65%|██████▌   | 146831/225000 [289:43:56<121:42:44,  5.61s/it]
 65%|██████▌   | 146832/225000 [289:44:02<129:26:17,  5.96s/it]
 65%|██████▌   | 146833/225000 [289:44:07<118:53:48,  5.48s/it]
 65%|██████▌   | 146834/225000 [289:44:14<128:53:54,  5.94s/it]
 65%|██████▌   | 146835/225000 [289:44:18<114:45:55,  5.29s/it]
 65%|██████▌   | 146836/225000 [289:44:27<143:17:31,  6.60s/it]
 65%|██████▌   | 146837/225000 [289:44:31<122:06:38,  5.62s/it]
 65%|██████▌   | 146838/225000 [289:44:38<136:43:22,  6.30s/it]
 65%|██████▌   | 146839/225000 [289:44:42<119:21:50,  5.50s/it]
 65%|██████▌   | 146840/225000 [289:44:47<118:03:47,  5.44s/it]
 65%|██████▌   | 146841/225000 [289:44:53<116:47:01,  5.38s/it]
 65%|██████▌   | 146842/225000 [289:44:57<112:02:57,  5.16s/it]
 65%|██████▌   | 146843/225000 [289:45:02<108:40:14,  5.01s/it]
 65%|██████▌   | 146844/225000 [289:45:07<108:04:05,  4.98s/it]
 65%|██████▌   | 146845/225000 [289:45:12<108:33:29,  5.00s/it]
 65%|██████▌   | 146846/225000 [289:45:17<107:30:16,  4.95s/it]
 65%|██████▌   | 146847/225000 [289:45:22<106:35:16,  4.91s/it]
 65%|██████▌   | 146848/225000 [289:45:27<109:44:27,  5.06s/it]
 65%|██████▌   | 146849/225000 [289:45:32<111:36:26,  5.14s/it]
 65%|██████▌   | 146850/225000 [289:45:37<109:32:27,  5.05s/it]
                                                               
{'loss': '0.7797', 'grad_norm': '1.16', 'learning_rate': '1.463e-05', 'epoch': '0.6527', 'num_input_tokens_seen': 480287504, 'train_runtime': '1.043e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146850/225000 [289:45:37<109:32:27,  5.05s/it]
 65%|██████▌   | 146851/225000 [289:45:45<125:59:13,  5.80s/it]
 65%|██████▌   | 146852/225000 [289:45:57<166:29:03,  7.67s/it]
 65%|██████▌   | 146853/225000 [289:46:01<144:10:31,  6.64s/it]
 65%|██████▌   | 146854/225000 [289:46:05<127:40:02,  5.88s/it]
 65%|██████▌   | 146855/225000 [289:46:10<124:07:00,  5.72s/it]
 65%|██████▌   | 146856/225000 [289:46:16<124:05:54,  5.72s/it]
 65%|██████▌   | 146857/225000 [289:46:21<118:57:00,  5.48s/it]
 65%|██████▌   | 146858/225000 [289:46:27<121:00:22,  5.57s/it]
 65%|██████▌   | 146859/225000 [289:46:32<118:26:51,  5.46s/it]
 65%|██████▌   | 146860/225000 [289:46:37<118:17:30,  5.45s/it]
 65%|██████▌   | 146861/225000 [289:46:43<118:46:04,  5.47s/it]
 65%|██████▌   | 146862/225000 [289:46:48<116:36:24,  5.37s/it]
 65%|██████▌   | 146863/225000 [289:46:54<119:31:22,  5.51s/it]
 65%|██████▌   | 146864/225000 [289:46:59<119:17:49,  5.50s/it]
 65%|██████▌   | 146865/225000 [289:47:04<115:28:09,  5.32s/it]
 65%|██████▌   | 146866/225000 [289:47:10<115:29:25,  5.32s/it]
 65%|██████▌   | 146867/225000 [289:47:15<114:45:38,  5.29s/it]
 65%|██████▌   | 146868/225000 [289:47:20<114:18:18,  5.27s/it]
 65%|██████▌   | 146869/225000 [289:47:25<113:27:42,  5.23s/it]
 65%|██████▌   | 146870/225000 [289:47:31<115:53:18,  5.34s/it]
 65%|██████▌   | 146871/225000 [289:47:36<116:17:04,  5.36s/it]
 65%|██████▌   | 146872/225000 [289:47:42<116:37:46,  5.37s/it]
 65%|██████▌   | 146873/225000 [289:47:47<114:56:12,  5.30s/it]
 65%|██████▌   | 146874/225000 [289:47:59<161:59:18,  7.46s/it]
 65%|██████▌   | 146875/225000 [289:48:07<165:00:23,  7.60s/it]
 65%|██████▌   | 146876/225000 [289:48:11<138:17:13,  6.37s/it]
 65%|██████▌   | 146877/225000 [289:48:17<136:40:29,  6.30s/it]
 65%|██████▌   | 146878/225000 [289:48:24<142:04:35,  6.55s/it]
 65%|██████▌   | 146879/225000 [289:48:28<124:13:07,  5.72s/it]
 65%|██████▌   | 146880/225000 [289:48:32<117:22:37,  5.41s/it]
 65%|██████▌   | 146881/225000 [289:48:42<144:19:19,  6.65s/it]
 65%|██████▌   | 146882/225000 [289:48:46<125:59:51,  5.81s/it]
 65%|██████▌   | 146883/225000 [289:48:50<115:42:21,  5.33s/it]
 65%|██████▌   | 146884/225000 [289:48:54<110:18:15,  5.08s/it]
 65%|██████▌   | 146885/225000 [289:48:59<106:04:19,  4.89s/it]
 65%|██████▌   | 146886/225000 [289:49:04<104:09:31,  4.80s/it]
 65%|██████▌   | 146887/225000 [289:49:08<104:08:00,  4.80s/it]
 65%|██████▌   | 146888/225000 [289:49:13<104:06:14,  4.80s/it]
 65%|██████▌   | 146889/225000 [289:49:18<106:22:54,  4.90s/it]
 65%|██████▌   | 146890/225000 [289:49:30<152:24:37,  7.02s/it]
 65%|██████▌   | 146891/225000 [289:49:36<141:53:19,  6.54s/it]
 65%|██████▌   | 146892/225000 [289:49:40<130:24:44,  6.01s/it]
 65%|██████▌   | 146893/225000 [289:49:46<129:29:55,  5.97s/it]
 65%|██████▌   | 146894/225000 [289:49:50<114:15:33,  5.27s/it]
 65%|██████▌   | 146895/225000 [289:49:59<142:10:24,  6.55s/it]
 65%|██████▌   | 146896/225000 [289:50:04<126:38:10,  5.84s/it]
 65%|██████▌   | 146897/225000 [289:50:08<114:56:29,  5.30s/it]
 65%|██████▌   | 146898/225000 [289:50:17<144:06:34,  6.64s/it]
 65%|██████▌   | 146899/225000 [289:50:21<121:40:06,  5.61s/it]
 65%|██████▌   | 146900/225000 [289:50:24<107:29:46,  4.96s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.067', 'learning_rate': '1.461e-05', 'epoch': '0.6529', 'num_input_tokens_seen': 480417608, 'train_runtime': '1.043e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146900/225000 [289:50:24<107:29:46,  4.96s/it]
 65%|██████▌   | 146901/225000 [289:50:33<133:13:41,  6.14s/it]
 65%|██████▌   | 146902/225000 [289:50:36<114:37:51,  5.28s/it]
 65%|██████▌   | 146903/225000 [289:50:41<108:16:33,  4.99s/it]
 65%|██████▌   | 146904/225000 [289:50:45<103:53:20,  4.79s/it]
 65%|██████▌   | 146905/225000 [289:50:49<99:46:57,  4.60s/it] 
 65%|██████▌   | 146906/225000 [289:50:56<112:33:52,  5.19s/it]
 65%|██████▌   | 146907/225000 [289:51:07<154:23:03,  7.12s/it]
 65%|██████▌   | 146908/225000 [289:51:11<129:37:18,  5.98s/it]
 65%|██████▌   | 146909/225000 [289:51:18<138:30:51,  6.39s/it]
 65%|██████▌   | 146910/225000 [289:51:23<128:11:36,  5.91s/it]
 65%|██████▌   | 146911/225000 [289:51:27<119:38:53,  5.52s/it]
 65%|██████▌   | 146912/225000 [289:51:32<115:15:43,  5.31s/it]
 65%|██████▌   | 146913/225000 [289:51:36<108:31:11,  5.00s/it]
 65%|██████▌   | 146914/225000 [289:51:41<105:57:53,  4.89s/it]
 65%|██████▌   | 146915/225000 [289:51:49<129:03:30,  5.95s/it]
 65%|██████▌   | 146916/225000 [289:51:59<154:32:08,  7.12s/it]
 65%|██████▌   | 146917/225000 [289:52:03<131:23:49,  6.06s/it]
 65%|██████▌   | 146918/225000 [289:52:06<115:15:59,  5.31s/it]
 65%|██████▌   | 146919/225000 [289:52:15<136:58:32,  6.32s/it]
 65%|██████▌   | 146920/225000 [289:52:21<135:21:12,  6.24s/it]
 65%|██████▌   | 146921/225000 [289:52:25<122:02:59,  5.63s/it]
 65%|██████▌   | 146922/225000 [289:52:30<113:58:20,  5.26s/it]
 65%|██████▌   | 146923/225000 [289:52:34<107:16:06,  4.95s/it]
 65%|██████▌   | 146924/225000 [289:52:38<104:15:37,  4.81s/it]
 65%|██████▌   | 146925/225000 [289:52:43<102:35:15,  4.73s/it]
 65%|██████▌   | 146926/225000 [289:52:48<106:13:48,  4.90s/it]
 65%|██████▌   | 146927/225000 [289:52:54<108:49:02,  5.02s/it]
 65%|██████▌   | 146928/225000 [289:52:59<111:42:23,  5.15s/it]
 65%|██████▌   | 146929/225000 [289:53:09<139:55:06,  6.45s/it]
 65%|██████▌   | 146930/225000 [289:53:17<155:22:20,  7.16s/it]
 65%|██████▌   | 146931/225000 [289:53:21<129:18:11,  5.96s/it]
 65%|██████▌   | 146932/225000 [289:53:25<118:09:21,  5.45s/it]
 65%|██████▌   | 146933/225000 [289:53:29<111:01:37,  5.12s/it]
 65%|██████▌   | 146934/225000 [289:53:34<106:52:13,  4.93s/it]
 65%|██████▌   | 146935/225000 [289:53:41<121:58:50,  5.63s/it]
 65%|██████▌   | 146936/225000 [289:53:51<153:33:14,  7.08s/it]
 65%|██████▌   | 146937/225000 [289:53:55<129:02:12,  5.95s/it]
 65%|██████▌   | 146938/225000 [289:54:00<127:56:27,  5.90s/it]
 65%|██████▌   | 146939/225000 [289:54:08<139:05:17,  6.41s/it]
 65%|██████▌   | 146940/225000 [289:54:12<119:57:59,  5.53s/it]
 65%|██████▌   | 146941/225000 [289:54:16<110:09:55,  5.08s/it]
 65%|██████▌   | 146942/225000 [289:54:19<102:34:55,  4.73s/it]
 65%|██████▌   | 146943/225000 [289:54:28<125:15:13,  5.78s/it]
 65%|██████▌   | 146944/225000 [289:54:37<145:59:02,  6.73s/it]
 65%|██████▌   | 146945/225000 [289:54:41<128:35:30,  5.93s/it]
 65%|██████▌   | 146946/225000 [289:54:51<154:39:08,  7.13s/it]
 65%|██████▌   | 146947/225000 [289:54:55<134:57:45,  6.22s/it]
 65%|██████▌   | 146948/225000 [289:55:00<129:27:53,  5.97s/it]
 65%|██████▌   | 146949/225000 [289:55:06<126:34:12,  5.84s/it]
 65%|██████▌   | 146950/225000 [289:55:11<122:54:26,  5.67s/it]
                                                               
{'loss': '0.7873', 'grad_norm': '1.083', 'learning_rate': '1.459e-05', 'epoch': '0.6531', 'num_input_tokens_seen': 480546712, 'train_runtime': '1.044e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 146950/225000 [289:55:11<122:54:26,  5.67s/it]
 65%|██████▌   | 146951/225000 [289:55:16<119:06:04,  5.49s/it]
 65%|██████▌   | 146952/225000 [289:55:21<117:49:30,  5.43s/it]
 65%|██████▌   | 146953/225000 [289:55:27<118:49:16,  5.48s/it]
 65%|██████▌   | 146954/225000 [289:55:32<118:00:19,  5.44s/it]
 65%|██████▌   | 146955/225000 [289:55:37<114:02:47,  5.26s/it]
 65%|██████▌   | 146956/225000 [289:55:44<125:22:58,  5.78s/it]
 65%|██████▌   | 146957/225000 [289:55:51<132:20:52,  6.10s/it]
 65%|██████▌   | 146958/225000 [289:55:56<122:18:18,  5.64s/it]
 65%|██████▌   | 146959/225000 [289:56:00<112:56:36,  5.21s/it]
 65%|██████▌   | 146960/225000 [289:56:10<148:05:10,  6.83s/it]
 65%|██████▌   | 146961/225000 [289:56:14<125:28:31,  5.79s/it]
 65%|██████▌   | 146962/225000 [289:56:22<144:41:57,  6.68s/it]
 65%|██████▌   | 146963/225000 [289:56:26<123:41:49,  5.71s/it]
 65%|██████▌   | 146964/225000 [289:56:29<109:32:53,  5.05s/it]
 65%|██████▌   | 146965/225000 [289:56:36<120:19:10,  5.55s/it]
 65%|██████▌   | 146966/225000 [289:56:42<121:33:59,  5.61s/it]
 65%|██████▌   | 146967/225000 [289:56:49<128:08:53,  5.91s/it]
 65%|██████▌   | 146968/225000 [289:56:53<117:45:01,  5.43s/it]
 65%|██████▌   | 146969/225000 [289:57:00<127:29:11,  5.88s/it]
 65%|██████▌   | 146970/225000 [289:57:04<117:24:41,  5.42s/it]
 65%|██████▌   | 146971/225000 [289:57:08<107:28:57,  4.96s/it]
 65%|██████▌   | 146972/225000 [289:57:12<102:40:59,  4.74s/it]
 65%|██████▌   | 146973/225000 [289:57:22<136:23:10,  6.29s/it]
 65%|██████▌   | 146974/225000 [289:57:28<131:34:36,  6.07s/it]
 65%|██████▌   | 146975/225000 [289:57:31<115:40:44,  5.34s/it]
 65%|██████▌   | 146976/225000 [289:57:40<134:56:22,  6.23s/it]
 65%|██████▌   | 146977/225000 [289:57:45<130:11:16,  6.01s/it]
 65%|██████▌   | 146978/225000 [289:57:50<121:35:34,  5.61s/it]
 65%|██████▌   | 146979/225000 [289:57:53<105:34:13,  4.87s/it]
 65%|██████▌   | 146980/225000 [289:58:02<130:18:49,  6.01s/it]
 65%|██████▌   | 146981/225000 [289:58:05<114:39:16,  5.29s/it]
 65%|██████▌   | 146982/225000 [289:58:10<108:53:11,  5.02s/it]
 65%|██████▌   | 146983/225000 [289:58:14<104:16:25,  4.81s/it]
 65%|██████▌   | 146984/225000 [289:58:18<100:24:56,  4.63s/it]
 65%|██████▌   | 146985/225000 [289:58:22<98:21:14,  4.54s/it] 
 65%|██████▌   | 146986/225000 [289:58:36<153:47:59,  7.10s/it]
 65%|██████▌   | 146987/225000 [289:58:39<130:01:43,  6.00s/it]
 65%|██████▌   | 146988/225000 [289:58:43<117:33:36,  5.43s/it]
 65%|██████▌   | 146989/225000 [289:58:47<109:25:06,  5.05s/it]
 65%|██████▌   | 146990/225000 [289:58:52<108:01:44,  4.99s/it]
 65%|██████▌   | 146991/225000 [289:58:57<108:52:41,  5.02s/it]
 65%|██████▌   | 146992/225000 [289:59:02<110:22:40,  5.09s/it]
 65%|██████▌   | 146993/225000 [289:59:08<110:50:31,  5.12s/it]
 65%|██████▌   | 146994/225000 [289:59:13<113:05:48,  5.22s/it]
 65%|██████▌   | 146995/225000 [289:59:18<113:45:40,  5.25s/it]
 65%|██████▌   | 146996/225000 [289:59:23<112:02:22,  5.17s/it]
 65%|██████▌   | 146997/225000 [289:59:31<129:37:14,  5.98s/it]
 65%|██████▌   | 146998/225000 [289:59:43<164:02:40,  7.57s/it]
 65%|██████▌   | 146999/225000 [289:59:46<137:05:12,  6.33s/it]
 65%|██████▌   | 147000/225000 [289:59:54<145:09:18,  6.70s/it]
                                                               
{'loss': '0.7849', 'grad_norm': '1.069', 'learning_rate': '1.458e-05', 'epoch': '0.6533', 'num_input_tokens_seen': 480675520, 'train_runtime': '1.044e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147000/225000 [289:59:54<145:09:18,  6.70s/it]
 65%|██████▌   | 147001/225000 [289:59:59<137:50:09,  6.36s/it]
 65%|██████▌   | 147002/225000 [290:00:04<127:13:22,  5.87s/it]
 65%|██████▌   | 147003/225000 [290:00:08<119:06:42,  5.50s/it]
 65%|██████▌   | 147004/225000 [290:00:13<115:50:00,  5.35s/it]
 65%|██████▌   | 147005/225000 [290:00:18<113:12:52,  5.23s/it]
 65%|██████▌   | 147006/225000 [290:00:23<108:52:02,  5.03s/it]
 65%|██████▌   | 147007/225000 [290:00:27<105:50:15,  4.89s/it]
 65%|██████▌   | 147008/225000 [290:00:32<105:37:13,  4.88s/it]
 65%|██████▌   | 147009/225000 [290:00:44<149:10:56,  6.89s/it]
 65%|██████▌   | 147010/225000 [290:00:49<138:26:33,  6.39s/it]
 65%|██████▌   | 147011/225000 [290:00:57<148:51:45,  6.87s/it]
 65%|██████▌   | 147012/225000 [290:01:01<127:09:02,  5.87s/it]
 65%|██████▌   | 147013/225000 [290:01:05<114:54:38,  5.30s/it]
 65%|██████▌   | 147014/225000 [290:01:12<127:25:24,  5.88s/it]
 65%|██████▌   | 147015/225000 [290:01:16<113:19:01,  5.23s/it]
 65%|██████▌   | 147016/225000 [290:01:20<106:41:52,  4.93s/it]
 65%|██████▌   | 147017/225000 [290:01:31<145:16:36,  6.71s/it]
 65%|██████▌   | 147018/225000 [290:01:35<129:07:43,  5.96s/it]
 65%|██████▌   | 147019/225000 [290:01:42<135:05:49,  6.24s/it]
 65%|██████▌   | 147020/225000 [290:01:47<129:47:34,  5.99s/it]
 65%|██████▌   | 147021/225000 [290:01:53<127:52:25,  5.90s/it]
 65%|██████▌   | 147022/225000 [290:01:58<120:56:37,  5.58s/it]
 65%|██████▌   | 147023/225000 [290:02:03<121:24:19,  5.60s/it]
 65%|██████▌   | 147024/225000 [290:02:09<121:47:09,  5.62s/it]
 65%|██████▌   | 147025/225000 [290:02:15<120:41:08,  5.57s/it]
 65%|██████▌   | 147026/225000 [290:02:20<119:43:07,  5.53s/it]
 65%|██████▌   | 147027/225000 [290:02:29<141:47:41,  6.55s/it]
 65%|██████▌   | 147028/225000 [290:02:33<127:42:44,  5.90s/it]
 65%|██████▌   | 147029/225000 [290:02:38<122:34:49,  5.66s/it]
 65%|██████▌   | 147030/225000 [290:02:44<122:04:02,  5.64s/it]
 65%|██████▌   | 147031/225000 [290:02:49<118:59:19,  5.49s/it]
 65%|██████▌   | 147032/225000 [290:02:54<117:06:26,  5.41s/it]
 65%|██████▌   | 147033/225000 [290:03:00<116:50:37,  5.40s/it]
 65%|██████▌   | 147034/225000 [290:03:06<120:01:02,  5.54s/it]
 65%|██████▌   | 147035/225000 [290:03:14<140:56:42,  6.51s/it]
 65%|██████▌   | 147036/225000 [290:03:18<125:26:08,  5.79s/it]
 65%|██████▌   | 147037/225000 [290:03:23<116:31:13,  5.38s/it]
 65%|██████▌   | 147038/225000 [290:03:27<109:49:53,  5.07s/it]
 65%|██████▌   | 147039/225000 [290:03:31<104:21:03,  4.82s/it]
 65%|██████▌   | 147040/225000 [290:03:41<136:01:14,  6.28s/it]
 65%|██████▌   | 147041/225000 [290:03:47<136:08:15,  6.29s/it]
 65%|██████▌   | 147042/225000 [290:03:51<119:18:15,  5.51s/it]
 65%|██████▌   | 147043/225000 [290:03:59<135:59:11,  6.28s/it]
 65%|██████▌   | 147044/225000 [290:04:03<120:04:03,  5.54s/it]
 65%|██████▌   | 147045/225000 [290:04:07<109:56:46,  5.08s/it]
 65%|██████▌   | 147046/225000 [290:04:16<138:30:14,  6.40s/it]
 65%|██████▌   | 147047/225000 [290:04:20<121:31:08,  5.61s/it]
 65%|██████▌   | 147048/225000 [290:04:29<144:39:14,  6.68s/it]
 65%|██████▌   | 147049/225000 [290:04:34<128:56:03,  5.95s/it]
 65%|██████▌   | 147050/225000 [290:04:37<113:29:14,  5.24s/it]
                                                               
{'loss': '0.7802', 'grad_norm': '1.146', 'learning_rate': '1.456e-05', 'epoch': '0.6536', 'num_input_tokens_seen': 480805984, 'train_runtime': '1.044e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147050/225000 [290:04:37<113:29:14,  5.24s/it]
 65%|██████▌   | 147051/225000 [290:04:41<105:24:46,  4.87s/it]
 65%|██████▌   | 147052/225000 [290:04:50<128:33:42,  5.94s/it]
 65%|██████▌   | 147053/225000 [290:04:55<124:00:20,  5.73s/it]
 65%|██████▌   | 147054/225000 [290:05:02<131:55:09,  6.09s/it]
 65%|██████▌   | 147055/225000 [290:05:06<116:13:34,  5.37s/it]
 65%|██████▌   | 147056/225000 [290:05:13<129:54:17,  6.00s/it]
 65%|██████▌   | 147057/225000 [290:05:18<121:18:29,  5.60s/it]
 65%|██████▌   | 147058/225000 [290:05:23<118:14:57,  5.46s/it]
 65%|██████▌   | 147059/225000 [290:05:27<112:06:29,  5.18s/it]
 65%|██████▌   | 147060/225000 [290:05:32<109:42:17,  5.07s/it]
 65%|██████▌   | 147061/225000 [290:05:37<107:27:34,  4.96s/it]
 65%|██████▌   | 147062/225000 [290:05:41<102:26:21,  4.73s/it]
 65%|██████▌   | 147063/225000 [290:05:46<105:03:52,  4.85s/it]
 65%|██████▌   | 147064/225000 [290:05:51<103:43:02,  4.79s/it]
 65%|██████▌   | 147065/225000 [290:05:55<99:50:53,  4.61s/it] 
 65%|██████▌   | 147066/225000 [290:06:03<119:22:49,  5.51s/it]
 65%|██████▌   | 147067/225000 [290:06:14<159:50:10,  7.38s/it]
 65%|██████▌   | 147068/225000 [290:06:19<142:11:52,  6.57s/it]
 65%|██████▌   | 147069/225000 [290:06:24<130:24:55,  6.02s/it]
 65%|██████▌   | 147070/225000 [290:06:29<121:41:07,  5.62s/it]
 65%|██████▌   | 147071/225000 [290:06:32<110:54:34,  5.12s/it]
 65%|██████▌   | 147072/225000 [290:06:37<106:56:27,  4.94s/it]
 65%|██████▌   | 147073/225000 [290:06:42<104:32:23,  4.83s/it]
 65%|██████▌   | 147074/225000 [290:06:52<142:24:51,  6.58s/it]
 65%|██████▌   | 147075/225000 [290:06:58<140:13:00,  6.48s/it]
 65%|██████▌   | 147076/225000 [290:07:03<126:12:09,  5.83s/it]
 65%|██████▌   | 147077/225000 [290:07:07<118:25:33,  5.47s/it]
 65%|██████▌   | 147078/225000 [290:07:12<114:27:02,  5.29s/it]
 65%|██████▌   | 147079/225000 [290:07:17<110:47:14,  5.12s/it]
 65%|██████▌   | 147080/225000 [290:07:22<109:38:43,  5.07s/it]
 65%|██████▌   | 147081/225000 [290:07:27<109:06:09,  5.04s/it]
 65%|██████▌   | 147082/225000 [290:07:32<111:21:38,  5.15s/it]
 65%|██████▌   | 147083/225000 [290:07:45<160:46:33,  7.43s/it]
 65%|██████▌   | 147084/225000 [290:07:50<141:32:40,  6.54s/it]
 65%|██████▌   | 147085/225000 [290:07:53<123:33:29,  5.71s/it]
 65%|██████▌   | 147086/225000 [290:07:59<120:14:34,  5.56s/it]
 65%|██████▌   | 147087/225000 [290:08:07<140:45:34,  6.50s/it]
 65%|██████▌   | 147088/225000 [290:08:11<125:14:30,  5.79s/it]
 65%|██████▌   | 147089/225000 [290:08:15<113:55:13,  5.26s/it]
 65%|██████▌   | 147090/225000 [290:08:19<105:43:51,  4.89s/it]
 65%|██████▌   | 147091/225000 [290:08:31<149:49:54,  6.92s/it]
 65%|██████▌   | 147092/225000 [290:08:35<129:22:48,  5.98s/it]
 65%|██████▌   | 147093/225000 [290:08:39<114:16:56,  5.28s/it]
 65%|██████▌   | 147094/225000 [290:08:49<145:42:59,  6.73s/it]
 65%|██████▌   | 147095/225000 [290:08:52<124:20:26,  5.75s/it]
 65%|██████▌   | 147096/225000 [290:08:59<132:16:27,  6.11s/it]
 65%|██████▌   | 147097/225000 [290:09:03<116:33:52,  5.39s/it]
 65%|██████▌   | 147098/225000 [290:09:10<131:32:30,  6.08s/it]
 65%|██████▌   | 147099/225000 [290:09:15<123:15:11,  5.70s/it]
 65%|██████▌   | 147100/225000 [290:09:20<116:08:17,  5.37s/it]
                                                               
{'loss': '0.7761', 'grad_norm': '1.102', 'learning_rate': '1.454e-05', 'epoch': '0.6538', 'num_input_tokens_seen': 480935800, 'train_runtime': '1.045e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147100/225000 [290:09:20<116:08:17,  5.37s/it]
 65%|██████▌   | 147101/225000 [290:09:25<112:37:16,  5.20s/it]
 65%|██████▌   | 147102/225000 [290:09:29<107:50:01,  4.98s/it]
 65%|██████▌   | 147103/225000 [290:09:34<106:05:37,  4.90s/it]
 65%|██████▌   | 147104/225000 [290:09:39<107:12:27,  4.95s/it]
 65%|██████▌   | 147105/225000 [290:09:43<104:03:41,  4.81s/it]
 65%|██████▌   | 147106/225000 [290:09:51<125:20:45,  5.79s/it]
 65%|██████▌   | 147107/225000 [290:10:02<156:47:25,  7.25s/it]
 65%|██████▌   | 147108/225000 [290:10:08<144:54:28,  6.70s/it]
 65%|██████▌   | 147109/225000 [290:10:12<129:05:54,  5.97s/it]
 65%|██████▌   | 147110/225000 [290:10:16<116:22:14,  5.38s/it]
 65%|██████▌   | 147111/225000 [290:10:22<120:37:16,  5.58s/it]
 65%|██████▌   | 147112/225000 [290:10:30<139:11:02,  6.43s/it]
 65%|██████▌   | 147113/225000 [290:10:35<128:59:48,  5.96s/it]
 65%|██████▌   | 147114/225000 [290:10:40<118:45:23,  5.49s/it]
 65%|██████▌   | 147115/225000 [290:10:45<116:07:53,  5.37s/it]
 65%|██████▌   | 147116/225000 [290:10:49<112:11:39,  5.19s/it]
 65%|██████▌   | 147117/225000 [290:10:54<111:16:14,  5.14s/it]
 65%|██████▌   | 147118/225000 [290:10:59<108:00:39,  4.99s/it]
 65%|██████▌   | 147119/225000 [290:11:04<107:43:41,  4.98s/it]
 65%|██████▌   | 147120/225000 [290:11:15<147:08:06,  6.80s/it]
 65%|██████▌   | 147121/225000 [290:11:21<143:29:53,  6.63s/it]
 65%|██████▌   | 147122/225000 [290:11:25<126:47:16,  5.86s/it]
 65%|██████▌   | 147123/225000 [290:11:31<122:34:49,  5.67s/it]
 65%|██████▌   | 147124/225000 [290:11:36<120:09:27,  5.55s/it]
 65%|██████▌   | 147125/225000 [290:11:42<121:39:46,  5.62s/it]
 65%|██████▌   | 147126/225000 [290:11:47<117:33:14,  5.43s/it]
 65%|██████▌   | 147127/225000 [290:11:52<116:56:28,  5.41s/it]
 65%|██████▌   | 147128/225000 [290:11:57<117:08:19,  5.42s/it]
 65%|██████▌   | 147129/225000 [290:12:03<117:49:20,  5.45s/it]
 65%|██████▌   | 147130/225000 [290:12:08<118:01:28,  5.46s/it]
 65%|██████▌   | 147131/225000 [290:12:14<121:31:00,  5.62s/it]
 65%|██████▌   | 147132/225000 [290:12:20<120:20:53,  5.56s/it]
 65%|██████▌   | 147133/225000 [290:12:25<120:49:31,  5.59s/it]
 65%|██████▌   | 147134/225000 [290:12:32<125:54:27,  5.82s/it]
 65%|██████▌   | 147135/225000 [290:12:42<155:49:34,  7.20s/it]
 65%|██████▌   | 147136/225000 [290:12:46<132:36:23,  6.13s/it]
 65%|██████▌   | 147137/225000 [290:12:50<116:39:57,  5.39s/it]
 65%|██████▌   | 147138/225000 [290:12:59<141:21:58,  6.54s/it]
 65%|██████▌   | 147139/225000 [290:13:03<123:40:58,  5.72s/it]
 65%|██████▌   | 147140/225000 [290:13:08<120:56:25,  5.59s/it]
 65%|██████▌   | 147141/225000 [290:13:14<127:23:36,  5.89s/it]
 65%|██████▌   | 147142/225000 [290:13:20<126:10:46,  5.83s/it]
 65%|██████▌   | 147143/225000 [290:13:27<129:56:40,  6.01s/it]
 65%|██████▌   | 147144/225000 [290:13:31<121:07:49,  5.60s/it]
 65%|██████▌   | 147145/225000 [290:13:36<114:35:02,  5.30s/it]
 65%|██████▌   | 147146/225000 [290:13:40<108:09:25,  5.00s/it]
 65%|██████▌   | 147147/225000 [290:13:45<105:32:36,  4.88s/it]
 65%|██████▌   | 147148/225000 [290:13:50<105:01:35,  4.86s/it]
 65%|██████▌   | 147149/225000 [290:13:56<116:38:11,  5.39s/it]
 65%|██████▌   | 147150/225000 [290:14:07<154:24:27,  7.14s/it]
                                                               
{'loss': '0.7907', 'grad_norm': '0.9793', 'learning_rate': '1.453e-05', 'epoch': '0.654', 'num_input_tokens_seen': 481066568, 'train_runtime': '1.045e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147150/225000 [290:14:07<154:24:27,  7.14s/it]
 65%|██████▌   | 147151/225000 [290:14:11<132:12:34,  6.11s/it]
 65%|██████▌   | 147152/225000 [290:14:15<118:30:29,  5.48s/it]
 65%|██████▌   | 147153/225000 [290:14:22<125:02:30,  5.78s/it]
 65%|██████▌   | 147154/225000 [290:14:31<148:15:42,  6.86s/it]
 65%|██████▌   | 147155/225000 [290:14:34<126:34:40,  5.85s/it]
 65%|██████▌   | 147156/225000 [290:14:38<109:42:26,  5.07s/it]
 65%|██████▌   | 147157/225000 [290:14:46<127:13:48,  5.88s/it]
 65%|██████▌   | 147158/225000 [290:14:50<116:34:02,  5.39s/it]
 65%|██████▌   | 147159/225000 [290:14:54<106:00:35,  4.90s/it]
 65%|██████▌   | 147160/225000 [290:15:03<134:08:17,  6.20s/it]
 65%|██████▌   | 147161/225000 [290:15:08<126:27:58,  5.85s/it]
 65%|██████▌   | 147162/225000 [290:15:12<116:10:31,  5.37s/it]
 65%|██████▌   | 147163/225000 [290:15:16<107:23:58,  4.97s/it]
 65%|██████▌   | 147164/225000 [290:15:26<139:20:30,  6.44s/it]
 65%|██████▌   | 147165/225000 [290:15:31<129:30:30,  5.99s/it]
 65%|██████▌   | 147166/225000 [290:15:36<121:56:09,  5.64s/it]
 65%|██████▌   | 147167/225000 [290:15:43<134:17:06,  6.21s/it]
 65%|██████▌   | 147168/225000 [290:15:46<114:55:10,  5.32s/it]
 65%|██████▌   | 147169/225000 [290:15:52<114:36:37,  5.30s/it]
 65%|██████▌   | 147170/225000 [290:16:00<132:19:56,  6.12s/it]
 65%|██████▌   | 147171/225000 [290:16:04<122:32:33,  5.67s/it]
 65%|██████▌   | 147172/225000 [290:16:09<112:36:49,  5.21s/it]
 65%|██████▌   | 147173/225000 [290:16:13<110:57:42,  5.13s/it]
 65%|██████▌   | 147174/225000 [290:16:18<103:53:38,  4.81s/it]
 65%|██████▌   | 147175/225000 [290:16:22<103:23:16,  4.78s/it]
 65%|██████▌   | 147176/225000 [290:16:30<122:48:49,  5.68s/it]
 65%|██████▌   | 147177/225000 [290:16:40<148:14:06,  6.86s/it]
 65%|██████▌   | 147178/225000 [290:16:44<129:45:08,  6.00s/it]
 65%|██████▌   | 147179/225000 [290:16:48<116:17:27,  5.38s/it]
 65%|██████▌   | 147180/225000 [290:16:55<131:07:42,  6.07s/it]
 65%|██████▌   | 147181/225000 [290:17:01<129:55:44,  6.01s/it]
 65%|██████▌   | 147182/225000 [290:17:05<113:44:00,  5.26s/it]
 65%|██████▌   | 147183/225000 [290:17:15<147:14:12,  6.81s/it]
 65%|██████▌   | 147184/225000 [290:17:21<144:43:58,  6.70s/it]
 65%|██████▌   | 147185/225000 [290:17:25<126:42:15,  5.86s/it]
 65%|██████▌   | 147186/225000 [290:17:29<113:37:43,  5.26s/it]
 65%|██████▌   | 147187/225000 [290:17:33<105:02:08,  4.86s/it]
 65%|██████▌   | 147188/225000 [290:17:41<125:15:37,  5.80s/it]
 65%|██████▌   | 147189/225000 [290:17:48<134:58:29,  6.24s/it]
 65%|██████▌   | 147190/225000 [290:17:53<125:10:59,  5.79s/it]
 65%|██████▌   | 147191/225000 [290:18:00<130:46:43,  6.05s/it]
 65%|██████▌   | 147192/225000 [290:18:04<117:06:10,  5.42s/it]
 65%|██████▌   | 147193/225000 [290:18:12<138:26:02,  6.41s/it]
 65%|██████▌   | 147194/225000 [290:18:17<123:18:39,  5.71s/it]
 65%|██████▌   | 147195/225000 [290:18:22<119:33:13,  5.53s/it]
 65%|██████▌   | 147196/225000 [290:18:27<118:47:35,  5.50s/it]
 65%|██████▌   | 147197/225000 [290:18:32<115:16:26,  5.33s/it]
 65%|██████▌   | 147198/225000 [290:18:38<116:14:55,  5.38s/it]
 65%|██████▌   | 147199/225000 [290:18:42<113:03:11,  5.23s/it]
 65%|██████▌   | 147200/225000 [290:18:47<111:33:38,  5.16s/it]
                                                               
{'loss': '0.7777', 'grad_norm': '1.127', 'learning_rate': '1.451e-05', 'epoch': '0.6542', 'num_input_tokens_seen': 481197584, 'train_runtime': '1.045e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147200/225000 [290:18:47<111:33:38,  5.16s/it]
 65%|██████▌   | 147201/225000 [290:18:52<110:28:11,  5.11s/it]
 65%|██████▌   | 147202/225000 [290:18:58<111:25:31,  5.16s/it]
 65%|██████▌   | 147203/225000 [290:19:03<111:50:44,  5.18s/it]
 65%|██████▌   | 147204/225000 [290:19:08<109:24:52,  5.06s/it]
 65%|██████▌   | 147205/225000 [290:19:13<111:37:33,  5.17s/it]
 65%|██████▌   | 147206/225000 [290:19:26<160:36:23,  7.43s/it]
 65%|██████▌   | 147207/225000 [290:19:31<146:23:03,  6.77s/it]
 65%|██████▌   | 147208/225000 [290:19:35<127:48:52,  5.91s/it]
 65%|██████▌   | 147209/225000 [290:19:39<113:43:45,  5.26s/it]
 65%|██████▌   | 147210/225000 [290:19:48<140:48:38,  6.52s/it]
 65%|██████▌   | 147211/225000 [290:19:54<136:07:52,  6.30s/it]
 65%|██████▌   | 147212/225000 [290:19:58<119:18:55,  5.52s/it]
 65%|██████▌   | 147213/225000 [290:20:05<128:10:21,  5.93s/it]
 65%|██████▌   | 147214/225000 [290:20:11<132:01:37,  6.11s/it]
 65%|██████▌   | 147215/225000 [290:20:15<118:10:21,  5.47s/it]
 65%|██████▌   | 147216/225000 [290:20:19<110:33:05,  5.12s/it]
 65%|██████▌   | 147217/225000 [290:20:28<133:34:18,  6.18s/it]
 65%|██████▌   | 147218/225000 [290:20:35<140:03:39,  6.48s/it]
 65%|██████▌   | 147219/225000 [290:20:40<132:20:54,  6.13s/it]
 65%|██████▌   | 147220/225000 [290:20:44<115:44:21,  5.36s/it]
 65%|██████▌   | 147221/225000 [290:20:48<106:11:44,  4.92s/it]
 65%|██████▌   | 147222/225000 [290:20:52<101:17:47,  4.69s/it]
 65%|██████▌   | 147223/225000 [290:20:59<116:57:09,  5.41s/it]
 65%|██████▌   | 147224/225000 [290:21:09<144:45:10,  6.70s/it]
 65%|██████▌   | 147225/225000 [290:21:13<125:16:10,  5.80s/it]
 65%|██████▌   | 147226/225000 [290:21:17<113:46:54,  5.27s/it]
 65%|██████▌   | 147227/225000 [290:21:24<130:34:51,  6.04s/it]
 65%|██████▌   | 147228/225000 [290:21:30<125:45:06,  5.82s/it]
 65%|██████▌   | 147229/225000 [290:21:34<115:52:13,  5.36s/it]
 65%|██████▌   | 147230/225000 [290:21:38<108:53:15,  5.04s/it]
 65%|██████▌   | 147231/225000 [290:21:43<104:36:03,  4.84s/it]
 65%|██████▌   | 147232/225000 [290:21:50<119:07:45,  5.51s/it]
 65%|██████▌   | 147233/225000 [290:22:00<151:41:49,  7.02s/it]
 65%|██████▌   | 147234/225000 [290:22:04<129:44:55,  6.01s/it]
 65%|██████▌   | 147235/225000 [290:22:08<116:59:00,  5.42s/it]
 65%|██████▌   | 147236/225000 [290:22:18<148:30:43,  6.88s/it]
 65%|██████▌   | 147237/225000 [290:22:21<123:17:49,  5.71s/it]
 65%|██████▌   | 147238/225000 [290:22:25<113:01:36,  5.23s/it]
 65%|██████▌   | 147239/225000 [290:22:30<107:41:08,  4.99s/it]
 65%|██████▌   | 147240/225000 [290:22:34<103:50:24,  4.81s/it]
 65%|██████▌   | 147241/225000 [290:22:38<99:14:13,  4.59s/it] 
 65%|██████▌   | 147242/225000 [290:22:45<114:04:14,  5.28s/it]
 65%|██████▌   | 147243/225000 [290:22:51<115:16:46,  5.34s/it]
 65%|██████▌   | 147244/225000 [290:22:59<132:57:43,  6.16s/it]
 65%|██████▌   | 147245/225000 [290:23:07<146:08:53,  6.77s/it]
 65%|██████▌   | 147246/225000 [290:23:14<150:05:36,  6.95s/it]
 65%|██████▌   | 147247/225000 [290:23:19<134:47:13,  6.24s/it]
 65%|██████▌   | 147248/225000 [290:23:24<128:42:02,  5.96s/it]
 65%|██████▌   | 147249/225000 [290:23:32<138:05:33,  6.39s/it]
 65%|██████▌   | 147250/225000 [290:23:36<125:12:28,  5.80s/it]
                                                               
{'loss': '0.7956', 'grad_norm': '0.9736', 'learning_rate': '1.449e-05', 'epoch': '0.6544', 'num_input_tokens_seen': 481326992, 'train_runtime': '1.045e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147250/225000 [290:23:36<125:12:28,  5.80s/it]
 65%|██████▌   | 147251/225000 [290:23:41<123:03:27,  5.70s/it]
 65%|██████▌   | 147252/225000 [290:23:47<121:23:03,  5.62s/it]
 65%|██████▌   | 147253/225000 [290:23:52<119:47:28,  5.55s/it]
 65%|██████▌   | 147254/225000 [290:23:58<118:36:48,  5.49s/it]
 65%|██████▌   | 147255/225000 [290:24:03<116:09:00,  5.38s/it]
 65%|██████▌   | 147256/225000 [290:24:11<137:42:16,  6.38s/it]
 65%|██████▌   | 147257/225000 [290:24:18<136:48:51,  6.34s/it]
 65%|██████▌   | 147258/225000 [290:24:21<119:21:46,  5.53s/it]
 65%|██████▌   | 147259/225000 [290:24:29<136:16:11,  6.31s/it]
 65%|██████▌   | 147260/225000 [290:24:34<121:30:26,  5.63s/it]
 65%|██████▌   | 147261/225000 [290:24:38<116:29:45,  5.39s/it]
 65%|██████▌   | 147262/225000 [290:24:43<111:22:06,  5.16s/it]
 65%|██████▌   | 147263/225000 [290:24:48<112:14:33,  5.20s/it]
 65%|██████▌   | 147264/225000 [290:24:53<112:05:15,  5.19s/it]
 65%|██████▌   | 147265/225000 [290:24:58<110:21:45,  5.11s/it]
 65%|██████▌   | 147266/225000 [290:25:03<109:48:40,  5.09s/it]
 65%|██████▌   | 147267/225000 [290:25:08<107:21:53,  4.97s/it]
 65%|██████▌   | 147268/225000 [290:25:14<110:19:21,  5.11s/it]
 65%|██████▌   | 147269/225000 [290:25:26<158:52:40,  7.36s/it]
 65%|██████▌   | 147270/225000 [290:25:32<151:27:02,  7.01s/it]
 65%|██████▌   | 147271/225000 [290:25:41<158:58:43,  7.36s/it]
 65%|██████▌   | 147272/225000 [290:25:44<133:42:51,  6.19s/it]
 65%|██████▌   | 147273/225000 [290:25:48<116:59:17,  5.42s/it]
 65%|██████▌   | 147274/225000 [290:25:58<148:15:36,  6.87s/it]
 65%|██████▌   | 147275/225000 [290:26:02<129:56:39,  6.02s/it]
 65%|██████▌   | 147276/225000 [290:26:06<115:56:44,  5.37s/it]
 65%|██████▌   | 147277/225000 [290:26:14<135:04:03,  6.26s/it]
 65%|██████▌   | 147278/225000 [290:26:18<117:37:45,  5.45s/it]
 65%|██████▌   | 147279/225000 [290:26:23<114:36:10,  5.31s/it]
 65%|██████▌   | 147280/225000 [290:26:28<114:18:02,  5.29s/it]
 65%|██████▌   | 147281/225000 [290:26:33<114:34:42,  5.31s/it]
 65%|██████▌   | 147282/225000 [290:26:39<114:49:40,  5.32s/it]
 65%|██████▌   | 147283/225000 [290:26:43<112:09:37,  5.20s/it]
 65%|██████▌   | 147284/225000 [290:26:49<112:38:40,  5.22s/it]
 65%|██████▌   | 147285/225000 [290:26:54<114:39:12,  5.31s/it]
 65%|██████▌   | 147286/225000 [290:27:00<115:11:17,  5.34s/it]
 65%|██████▌   | 147287/225000 [290:27:05<113:40:37,  5.27s/it]
 65%|██████▌   | 147288/225000 [290:27:10<116:04:53,  5.38s/it]
 65%|██████▌   | 147289/225000 [290:27:16<115:56:22,  5.37s/it]
 65%|██████▌   | 147290/225000 [290:27:21<113:58:06,  5.28s/it]
 65%|██████▌   | 147291/225000 [290:27:26<114:00:51,  5.28s/it]
 65%|██████▌   | 147292/225000 [290:27:31<114:19:10,  5.30s/it]
 65%|██████▌   | 147293/225000 [290:27:37<113:01:09,  5.24s/it]
 65%|██████▌   | 147294/225000 [290:27:42<113:16:22,  5.25s/it]
 65%|██████▌   | 147295/225000 [290:27:53<150:53:57,  6.99s/it]
 65%|██████▌   | 147296/225000 [290:28:01<155:34:03,  7.21s/it]
 65%|██████▌   | 147297/225000 [290:28:04<131:31:49,  6.09s/it]
 65%|██████▌   | 147298/225000 [290:28:08<117:48:34,  5.46s/it]
 65%|██████▌   | 147299/225000 [290:28:18<149:21:06,  6.92s/it]
 65%|██████▌   | 147300/225000 [290:28:23<132:02:16,  6.12s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '1.045', 'learning_rate': '1.448e-05', 'epoch': '0.6547', 'num_input_tokens_seen': 481456656, 'train_runtime': '1.046e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147300/225000 [290:28:23<132:02:16,  6.12s/it]
 65%|██████▌   | 147301/225000 [290:28:28<125:57:05,  5.84s/it]
 65%|██████▌   | 147302/225000 [290:28:35<135:55:01,  6.30s/it]
 65%|██████▌   | 147303/225000 [290:28:40<124:24:07,  5.76s/it]
 65%|██████▌   | 147304/225000 [290:28:47<133:42:02,  6.19s/it]
 65%|██████▌   | 147305/225000 [290:28:51<118:28:20,  5.49s/it]
 65%|██████▌   | 147306/225000 [290:28:54<107:05:22,  4.96s/it]
 65%|██████▌   | 147307/225000 [290:29:01<116:42:00,  5.41s/it]
 65%|██████▌   | 147308/225000 [290:29:12<151:52:06,  7.04s/it]
 65%|██████▌   | 147309/225000 [290:29:15<127:07:41,  5.89s/it]
 65%|██████▌   | 147310/225000 [290:29:21<126:55:45,  5.88s/it]
 65%|██████▌   | 147311/225000 [290:29:24<111:17:28,  5.16s/it]
 65%|██████▌   | 147312/225000 [290:29:34<141:43:52,  6.57s/it]
 65%|██████▌   | 147313/225000 [290:29:39<132:33:13,  6.14s/it]
 65%|██████▌   | 147314/225000 [290:29:44<119:57:59,  5.56s/it]
 65%|██████▌   | 147315/225000 [290:29:48<115:27:48,  5.35s/it]
 65%|██████▌   | 147316/225000 [290:29:53<110:23:53,  5.12s/it]
 65%|██████▌   | 147317/225000 [290:29:58<110:03:56,  5.10s/it]
 65%|██████▌   | 147318/225000 [290:30:02<105:08:26,  4.87s/it]
 65%|██████▌   | 147319/225000 [290:30:07<105:00:12,  4.87s/it]
 65%|██████▌   | 147320/225000 [290:30:12<105:28:11,  4.89s/it]
 65%|██████▌   | 147321/225000 [290:30:17<103:18:59,  4.79s/it]
 65%|██████▌   | 147322/225000 [290:30:22<104:05:43,  4.82s/it]
 65%|██████▌   | 147323/225000 [290:30:27<104:58:09,  4.86s/it]
 65%|██████▌   | 147324/225000 [290:30:39<154:15:34,  7.15s/it]
 65%|██████▌   | 147325/225000 [290:30:46<151:43:35,  7.03s/it]
 65%|██████▌   | 147326/225000 [290:30:49<126:10:29,  5.85s/it]
 65%|██████▌   | 147327/225000 [290:30:58<146:17:14,  6.78s/it]
 65%|██████▌   | 147328/225000 [290:31:01<124:17:11,  5.76s/it]
 65%|██████▌   | 147329/225000 [290:31:08<132:22:36,  6.14s/it]
 65%|██████▌   | 147330/225000 [290:31:14<127:26:13,  5.91s/it]
 65%|██████▌   | 147331/225000 [290:31:21<134:30:45,  6.23s/it]
 65%|██████▌   | 147332/225000 [290:31:27<134:32:01,  6.24s/it]
 65%|██████▌   | 147333/225000 [290:31:30<115:49:56,  5.37s/it]
 65%|██████▌   | 147334/225000 [290:31:34<106:00:32,  4.91s/it]
 65%|██████▌   | 147335/225000 [290:31:43<135:21:07,  6.27s/it]
 65%|██████▌   | 147336/225000 [290:31:47<118:24:07,  5.49s/it]
 65%|██████▌   | 147337/225000 [290:31:52<115:25:09,  5.35s/it]
 65%|██████▌   | 147338/225000 [290:31:57<114:17:23,  5.30s/it]
 65%|██████▌   | 147339/225000 [290:32:03<114:06:21,  5.29s/it]
 65%|██████▌   | 147340/225000 [290:32:08<112:39:02,  5.22s/it]
 65%|██████▌   | 147341/225000 [290:32:13<110:57:46,  5.14s/it]
 65%|██████▌   | 147342/225000 [290:32:18<110:29:59,  5.12s/it]
 65%|██████▌   | 147343/225000 [290:32:23<110:36:20,  5.13s/it]
 65%|██████▌   | 147344/225000 [290:32:28<109:23:49,  5.07s/it]
 65%|██████▌   | 147345/225000 [290:32:34<113:53:43,  5.28s/it]
 65%|██████▌   | 147346/225000 [290:32:41<126:22:42,  5.86s/it]
 65%|██████▌   | 147347/225000 [290:32:52<161:53:18,  7.51s/it]
 65%|██████▌   | 147348/225000 [290:32:56<136:43:38,  6.34s/it]
 65%|██████▌   | 147349/225000 [290:33:00<121:25:46,  5.63s/it]
 65%|██████▌   | 147350/225000 [290:33:09<143:20:41,  6.65s/it]
                                                               
{'loss': '0.7802', 'grad_norm': '1.128', 'learning_rate': '1.446e-05', 'epoch': '0.6549', 'num_input_tokens_seen': 481586656, 'train_runtime': '1.046e+06', 'train_tokens_per_second': '460.4'}

 65%|██████▌   | 147350/225000 [290:33:09<143:20:41,  6.65s/it]
 65%|██████▌   | 147351/225000 [290:33:13<128:16:19,  5.95s/it]
 65%|██████▌   | 147352/225000 [290:33:17<117:36:51,  5.45s/it]
 65%|██████▌   | 147353/225000 [290:33:25<129:33:36,  6.01s/it]
 65%|██████▌   | 147354/225000 [290:33:32<140:32:01,  6.52s/it]
 65%|██████▌   | 147355/225000 [290:33:36<120:25:54,  5.58s/it]
 65%|██████▌   | 147356/225000 [290:33:44<134:36:28,  6.24s/it]
 65%|██████▌   | 147357/225000 [290:33:47<119:22:13,  5.53s/it]
 65%|██████▌   | 147358/225000 [290:33:53<118:08:34,  5.48s/it]
 65%|██████▌   | 147359/225000 [290:33:59<122:54:48,  5.70s/it]
 65%|██████▌   | 147360/225000 [290:34:05<124:14:31,  5.76s/it]
 65%|██████▌   | 147361/225000 [290:34:11<129:01:55,  5.98s/it]
 65%|██████▌   | 147362/225000 [290:34:15<113:55:45,  5.28s/it]
 65%|██████▌   | 147363/225000 [290:34:20<110:30:06,  5.12s/it]
 65%|██████▌   | 147364/225000 [290:34:25<107:52:39,  5.00s/it]
 65%|██████▌   | 147365/225000 [290:34:29<106:01:43,  4.92s/it]
 65%|██████▌   | 147366/225000 [290:34:34<104:40:26,  4.85s/it]
 65%|██████▌   | 147367/225000 [290:34:39<105:08:10,  4.88s/it]
 65%|██████▌   | 147368/225000 [290:34:44<104:26:59,  4.84s/it]
 65%|██████▌   | 147369/225000 [290:34:57<157:05:03,  7.28s/it]
 65%|██████▌   | 147370/225000 [290:35:01<139:14:49,  6.46s/it]
 65%|██████▌   | 147371/225000 [290:35:09<150:43:33,  6.99s/it]
 65%|██████▌   | 147372/225000 [290:35:13<127:58:34,  5.93s/it]
 65%|██████▌   | 147373/225000 [290:35:20<134:50:02,  6.25s/it]
 65%|██████▌   | 147374/225000 [290:35:26<132:14:27,  6.13s/it]
 66%|██████▌   | 147375/225000 [290:35:31<128:40:11,  5.97s/it]
 66%|██████▌   | 147376/225000 [290:35:37<124:17:15,  5.76s/it]
 66%|██████▌   | 147377/225000 [290:35:41<117:42:26,  5.46s/it]
 66%|██████▌   | 147378/225000 [290:35:46<112:59:46,  5.24s/it]
 66%|██████▌   | 147379/225000 [290:35:50<106:37:38,  4.95s/it]
 66%|██████▌   | 147380/225000 [290:35:55<103:22:49,  4.79s/it]
 66%|██████▌   | 147381/225000 [290:36:00<103:54:03,  4.82s/it]
 66%|██████▌   | 147382/225000 [290:36:05<107:56:49,  5.01s/it]
 66%|██████▌   | 147383/225000 [290:36:10<107:22:30,  4.98s/it]
 66%|██████▌   | 147384/225000 [290:36:15<107:47:37,  5.00s/it]
 66%|██████▌   | 147385/225000 [290:36:20<109:13:18,  5.07s/it]
 66%|██████▌   | 147386/225000 [290:36:26<111:09:39,  5.16s/it]
 66%|██████▌   | 147387/225000 [290:36:31<110:38:07,  5.13s/it]
 66%|██████▌   | 147388/225000 [290:36:36<110:49:43,  5.14s/it]
 66%|██████▌   | 147389/225000 [290:36:47<149:42:46,  6.94s/it]
 66%|██████▌   | 147390/225000 [290:36:53<146:01:36,  6.77s/it]
 66%|██████▌   | 147391/225000 [290:36:58<131:35:54,  6.10s/it]
 66%|██████▌   | 147392/225000 [290:37:03<122:22:13,  5.68s/it]
 66%|██████▌   | 147393/225000 [290:37:08<119:48:00,  5.56s/it]
 66%|██████▌   | 147394/225000 [290:37:12<111:51:52,  5.19s/it]
 66%|██████▌   | 147395/225000 [290:37:17<111:40:26,  5.18s/it]
 66%|██████▌   | 147396/225000 [290:37:22<109:52:49,  5.10s/it]
 66%|██████▌   | 147397/225000 [290:37:27<105:59:40,  4.92s/it]
 66%|██████▌   | 147398/225000 [290:37:34<121:18:47,  5.63s/it]
 66%|██████▌   | 147399/225000 [290:37:45<155:29:53,  7.21s/it]
 66%|██████▌   | 147400/225000 [290:37:49<136:24:46,  6.33s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.057', 'learning_rate': '1.444e-05', 'epoch': '0.6551', 'num_input_tokens_seen': 481715984, 'train_runtime': '1.046e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147400/225000 [290:37:49<136:24:46,  6.33s/it]
 66%|██████▌   | 147401/225000 [290:37:54<123:17:39,  5.72s/it]
 66%|██████▌   | 147402/225000 [290:37:58<115:01:04,  5.34s/it]
 66%|██████▌   | 147403/225000 [290:38:03<110:09:45,  5.11s/it]
 66%|██████▌   | 147404/225000 [290:38:11<132:44:37,  6.16s/it]
 66%|██████▌   | 147405/225000 [290:38:19<144:32:43,  6.71s/it]
 66%|██████▌   | 147406/225000 [290:38:23<124:57:06,  5.80s/it]
 66%|██████▌   | 147407/225000 [290:38:28<121:07:12,  5.62s/it]
 66%|██████▌   | 147408/225000 [290:38:37<141:00:40,  6.54s/it]
 66%|██████▌   | 147409/225000 [290:38:40<120:19:52,  5.58s/it]
 66%|██████▌   | 147410/225000 [290:38:45<117:07:42,  5.43s/it]
 66%|██████▌   | 147411/225000 [290:38:50<113:44:04,  5.28s/it]
 66%|██████▌   | 147412/225000 [290:38:55<111:34:51,  5.18s/it]
 66%|██████▌   | 147413/225000 [290:39:00<108:47:55,  5.05s/it]
 66%|██████▌   | 147414/225000 [290:39:04<106:00:40,  4.92s/it]
 66%|██████▌   | 147415/225000 [290:39:09<106:25:33,  4.94s/it]
 66%|██████▌   | 147416/225000 [290:39:14<106:08:37,  4.93s/it]
 66%|██████▌   | 147417/225000 [290:39:19<107:08:33,  4.97s/it]
 66%|██████▌   | 147418/225000 [290:39:32<157:02:34,  7.29s/it]
 66%|██████▌   | 147419/225000 [290:39:36<134:37:29,  6.25s/it]
 66%|██████▌   | 147420/225000 [290:39:40<122:57:36,  5.71s/it]
 66%|██████▌   | 147421/225000 [290:39:50<150:04:07,  6.96s/it]
 66%|██████▌   | 147422/225000 [290:39:53<125:57:12,  5.84s/it]
 66%|██████▌   | 147423/225000 [290:40:02<144:39:43,  6.71s/it]
 66%|██████▌   | 147424/225000 [290:40:06<125:15:36,  5.81s/it]
 66%|██████▌   | 147425/225000 [290:40:09<110:51:14,  5.14s/it]
 66%|██████▌   | 147426/225000 [290:40:19<136:55:10,  6.35s/it]
 66%|██████▌   | 147427/225000 [290:40:23<121:44:27,  5.65s/it]
 66%|██████▌   | 147428/225000 [290:40:27<110:13:07,  5.12s/it]
 66%|██████▌   | 147429/225000 [290:40:36<141:10:20,  6.55s/it]
 66%|██████▌   | 147430/225000 [290:40:40<125:10:01,  5.81s/it]
 66%|██████▌   | 147431/225000 [290:40:44<112:00:09,  5.20s/it]
 66%|██████▌   | 147432/225000 [290:40:54<142:53:27,  6.63s/it]
 66%|██████▌   | 147433/225000 [290:40:57<120:54:16,  5.61s/it]
 66%|██████▌   | 147434/225000 [290:41:05<135:42:34,  6.30s/it]
 66%|██████▌   | 147435/225000 [290:41:10<121:54:02,  5.66s/it]
 66%|██████▌   | 147436/225000 [290:41:13<110:05:32,  5.11s/it]
 66%|██████▌   | 147437/225000 [290:41:18<104:30:53,  4.85s/it]
 66%|██████▌   | 147438/225000 [290:41:28<142:39:40,  6.62s/it]
 66%|██████▌   | 147439/225000 [290:41:32<124:09:09,  5.76s/it]
 66%|██████▌   | 147440/225000 [290:41:35<108:16:25,  5.03s/it]
 66%|██████▌   | 147441/225000 [290:41:42<116:53:21,  5.43s/it]
 66%|██████▌   | 147442/225000 [290:41:50<133:57:34,  6.22s/it]
 66%|██████▌   | 147443/225000 [290:41:53<115:10:54,  5.35s/it]
 66%|██████▌   | 147444/225000 [290:41:57<105:09:55,  4.88s/it]
 66%|██████▌   | 147445/225000 [290:42:03<110:51:01,  5.15s/it]
 66%|██████▌   | 147446/225000 [290:42:13<147:05:17,  6.83s/it]
 66%|██████▌   | 147447/225000 [290:42:17<125:28:26,  5.82s/it]
 66%|██████▌   | 147448/225000 [290:42:26<143:57:42,  6.68s/it]
 66%|██████▌   | 147449/225000 [290:42:29<123:44:25,  5.74s/it]
 66%|██████▌   | 147450/225000 [290:42:33<110:09:01,  5.11s/it]
                                                               
{'loss': '0.7856', 'grad_norm': '1.097', 'learning_rate': '1.443e-05', 'epoch': '0.6553', 'num_input_tokens_seen': 481844992, 'train_runtime': '1.047e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147450/225000 [290:42:33<110:09:01,  5.11s/it]
 66%|██████▌   | 147451/225000 [290:42:42<138:29:00,  6.43s/it]
 66%|██████▌   | 147452/225000 [290:42:46<119:08:47,  5.53s/it]
 66%|██████▌   | 147453/225000 [290:42:54<138:48:41,  6.44s/it]
 66%|██████▌   | 147454/225000 [290:42:58<122:48:16,  5.70s/it]
 66%|██████▌   | 147455/225000 [290:43:02<111:28:42,  5.18s/it]
 66%|██████▌   | 147456/225000 [290:43:06<104:09:05,  4.84s/it]
 66%|██████▌   | 147457/225000 [290:43:13<117:33:25,  5.46s/it]
 66%|██████▌   | 147458/225000 [290:43:23<145:44:52,  6.77s/it]
 66%|██████▌   | 147459/225000 [290:43:27<125:52:33,  5.84s/it]
 66%|██████▌   | 147460/225000 [290:43:30<111:47:49,  5.19s/it]
 66%|██████▌   | 147461/225000 [290:43:40<141:37:35,  6.58s/it]
 66%|██████▌   | 147462/225000 [290:43:46<136:41:12,  6.35s/it]
 66%|██████▌   | 147463/225000 [290:43:52<131:30:22,  6.11s/it]
 66%|██████▌   | 147464/225000 [290:43:58<133:32:09,  6.20s/it]
 66%|██████▌   | 147465/225000 [290:44:02<121:34:49,  5.65s/it]
 66%|██████▌   | 147466/225000 [290:44:07<113:29:04,  5.27s/it]
 66%|██████▌   | 147467/225000 [290:44:11<106:19:01,  4.94s/it]
 66%|██████▌   | 147468/225000 [290:44:17<116:52:11,  5.43s/it]
 66%|██████▌   | 147469/225000 [290:44:27<141:52:34,  6.59s/it]
 66%|██████▌   | 147470/225000 [290:44:31<123:37:54,  5.74s/it]
 66%|██████▌   | 147471/225000 [290:44:35<114:44:17,  5.33s/it]
 66%|██████▌   | 147472/225000 [290:44:39<108:04:51,  5.02s/it]
 66%|██████▌   | 147473/225000 [290:44:46<116:39:12,  5.42s/it]
 66%|██████▌   | 147474/225000 [290:44:55<142:42:47,  6.63s/it]
 66%|██████▌   | 147475/225000 [290:44:58<121:28:39,  5.64s/it]
 66%|██████▌   | 147476/225000 [290:45:07<143:21:11,  6.66s/it]
 66%|██████▌   | 147477/225000 [290:45:11<122:31:40,  5.69s/it]
 66%|██████▌   | 147478/225000 [290:45:19<137:35:54,  6.39s/it]
 66%|██████▌   | 147479/225000 [290:45:23<120:46:55,  5.61s/it]
 66%|██████▌   | 147480/225000 [290:45:28<117:56:40,  5.48s/it]
 66%|██████▌   | 147481/225000 [290:45:33<116:08:21,  5.39s/it]
 66%|██████▌   | 147482/225000 [290:45:38<116:25:27,  5.41s/it]
 66%|██████▌   | 147483/225000 [290:45:46<133:02:47,  6.18s/it]
 66%|██████▌   | 147484/225000 [290:45:50<116:54:54,  5.43s/it]
 66%|██████▌   | 147485/225000 [290:45:54<107:00:38,  4.97s/it]
 66%|██████▌   | 147486/225000 [290:46:03<132:59:22,  6.18s/it]
 66%|██████▌   | 147487/225000 [290:46:06<115:15:24,  5.35s/it]
 66%|██████▌   | 147488/225000 [290:46:15<133:35:35,  6.20s/it]
 66%|██████▌   | 147489/225000 [290:46:18<113:58:05,  5.29s/it]
 66%|██████▌   | 147490/225000 [290:46:24<122:51:13,  5.71s/it]
 66%|██████▌   | 147491/225000 [290:46:28<107:48:27,  5.01s/it]
 66%|██████▌   | 147492/225000 [290:46:33<109:35:04,  5.09s/it]
 66%|██████▌   | 147493/225000 [290:46:38<110:07:37,  5.12s/it]
 66%|██████▌   | 147494/225000 [290:46:44<111:13:51,  5.17s/it]
 66%|██████▌   | 147495/225000 [290:46:49<111:25:06,  5.18s/it]
 66%|██████▌   | 147496/225000 [290:46:54<110:42:50,  5.14s/it]
 66%|██████▌   | 147497/225000 [290:47:03<135:00:29,  6.27s/it]
 66%|██████▌   | 147498/225000 [290:47:10<144:12:58,  6.70s/it]
 66%|██████▌   | 147499/225000 [290:47:15<130:39:53,  6.07s/it]
 66%|██████▌   | 147500/225000 [290:47:20<121:50:42,  5.66s/it]
                                                               
{'loss': '0.78', 'grad_norm': '1.098', 'learning_rate': '1.441e-05', 'epoch': '0.6556', 'num_input_tokens_seen': 481975496, 'train_runtime': '1.047e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147500/225000 [290:47:20<121:50:42,  5.66s/it]
 66%|██████▌   | 147501/225000 [290:47:24<114:58:19,  5.34s/it]
 66%|██████▌   | 147502/225000 [290:47:35<151:59:38,  7.06s/it]
 66%|██████▌   | 147503/225000 [290:47:39<131:22:57,  6.10s/it]
 66%|██████▌   | 147504/225000 [290:47:43<116:05:08,  5.39s/it]
 66%|██████▌   | 147505/225000 [290:47:47<108:58:36,  5.06s/it]
 66%|██████▌   | 147506/225000 [290:47:52<105:28:02,  4.90s/it]
 66%|██████▌   | 147507/225000 [290:48:04<149:46:22,  6.96s/it]
 66%|██████▌   | 147508/225000 [290:48:09<136:58:02,  6.36s/it]
 66%|██████▌   | 147509/225000 [290:48:13<127:15:04,  5.91s/it]
 66%|██████▌   | 147510/225000 [290:48:18<118:01:18,  5.48s/it]
 66%|██████▌   | 147511/225000 [290:48:23<115:01:35,  5.34s/it]
 66%|██████▌   | 147512/225000 [290:48:28<114:08:30,  5.30s/it]
 66%|██████▌   | 147513/225000 [290:48:33<111:32:43,  5.18s/it]
 66%|██████▌   | 147514/225000 [290:48:38<110:59:48,  5.16s/it]
 66%|██████▌   | 147515/225000 [290:48:43<107:59:13,  5.02s/it]
 66%|██████▌   | 147516/225000 [290:48:55<157:45:53,  7.33s/it]
 66%|██████▌   | 147517/225000 [290:49:01<144:12:02,  6.70s/it]
 66%|██████▌   | 147518/225000 [290:49:05<131:35:50,  6.11s/it]
 66%|██████▌   | 147519/225000 [290:49:10<120:04:15,  5.58s/it]
 66%|██████▌   | 147520/225000 [290:49:14<113:53:09,  5.29s/it]
 66%|██████▌   | 147521/225000 [290:49:19<109:42:09,  5.10s/it]
 66%|██████▌   | 147522/225000 [290:49:24<106:03:23,  4.93s/it]
 66%|██████▌   | 147523/225000 [290:49:36<156:56:46,  7.29s/it]
 66%|██████▌   | 147524/225000 [290:49:42<143:16:46,  6.66s/it]
 66%|██████▌   | 147525/225000 [290:49:49<151:13:00,  7.03s/it]
 66%|██████▌   | 147526/225000 [290:49:53<125:41:52,  5.84s/it]
 66%|██████▌   | 147527/225000 [290:49:59<126:30:49,  5.88s/it]
 66%|██████▌   | 147528/225000 [290:50:02<114:03:51,  5.30s/it]
 66%|██████▌   | 147529/225000 [290:50:06<105:46:39,  4.92s/it]
 66%|██████▌   | 147530/225000 [290:50:12<107:10:57,  4.98s/it]
 66%|██████▌   | 147531/225000 [290:50:23<151:44:18,  7.05s/it]
 66%|██████▌   | 147532/225000 [290:50:27<128:11:33,  5.96s/it]
 66%|██████▌   | 147533/225000 [290:50:35<142:55:31,  6.64s/it]
 66%|██████▌   | 147534/225000 [290:50:39<128:07:13,  5.95s/it]
 66%|██████▌   | 147535/225000 [290:50:43<112:07:06,  5.21s/it]
 66%|██████▌   | 147536/225000 [290:50:47<102:48:55,  4.78s/it]
 66%|██████▌   | 147537/225000 [290:50:58<144:21:27,  6.71s/it]
 66%|██████▌   | 147538/225000 [290:51:02<125:10:15,  5.82s/it]
 66%|██████▌   | 147539/225000 [290:51:05<110:08:18,  5.12s/it]
 66%|██████▌   | 147540/225000 [290:51:14<135:58:39,  6.32s/it]
 66%|██████▌   | 147541/225000 [290:51:18<117:37:18,  5.47s/it]
 66%|██████▌   | 147542/225000 [290:51:22<109:51:33,  5.11s/it]
 66%|██████▌   | 147543/225000 [290:51:26<105:21:11,  4.90s/it]
 66%|██████▌   | 147544/225000 [290:51:31<103:30:51,  4.81s/it]
 66%|██████▌   | 147545/225000 [290:51:36<103:36:28,  4.82s/it]
 66%|██████▌   | 147546/225000 [290:51:45<132:59:20,  6.18s/it]
 66%|██████▌   | 147547/225000 [290:51:50<126:35:19,  5.88s/it]
 66%|██████▌   | 147548/225000 [290:51:57<132:50:46,  6.17s/it]
 66%|██████▌   | 147549/225000 [290:52:01<119:17:03,  5.54s/it]
 66%|██████▌   | 147550/225000 [290:52:08<123:45:23,  5.75s/it]
                                                               
{'loss': '0.7726', 'grad_norm': '1.117', 'learning_rate': '1.439e-05', 'epoch': '0.6558', 'num_input_tokens_seen': 482105584, 'train_runtime': '1.047e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147550/225000 [290:52:08<123:45:23,  5.75s/it]
 66%|██████▌   | 147551/225000 [290:52:14<129:03:23,  6.00s/it]
 66%|██████▌   | 147552/225000 [290:52:19<119:32:26,  5.56s/it]
 66%|██████▌   | 147553/225000 [290:52:23<113:11:45,  5.26s/it]
 66%|██████▌   | 147554/225000 [290:52:28<109:12:25,  5.08s/it]
 66%|██████▌   | 147555/225000 [290:52:32<105:55:12,  4.92s/it]
 66%|██████▌   | 147556/225000 [290:52:38<106:45:13,  4.96s/it]
 66%|██████▌   | 147557/225000 [290:52:46<131:11:55,  6.10s/it]
 66%|██████▌   | 147558/225000 [290:52:54<140:53:24,  6.55s/it]
 66%|██████▌   | 147559/225000 [290:52:58<125:27:41,  5.83s/it]
 66%|██████▌   | 147560/225000 [290:53:02<116:29:17,  5.42s/it]
 66%|██████▌   | 147561/225000 [290:53:07<109:14:45,  5.08s/it]
 66%|██████▌   | 147562/225000 [290:53:18<147:11:39,  6.84s/it]
 66%|██████▌   | 147563/225000 [290:53:23<140:06:36,  6.51s/it]
 66%|██████▌   | 147564/225000 [290:53:29<131:50:55,  6.13s/it]
 66%|██████▌   | 147565/225000 [290:53:34<127:30:03,  5.93s/it]
 66%|██████▌   | 147566/225000 [290:53:39<121:27:34,  5.65s/it]
 66%|██████▌   | 147567/225000 [290:53:45<120:52:51,  5.62s/it]
 66%|██████▌   | 147568/225000 [290:53:50<117:10:17,  5.45s/it]
 66%|██████▌   | 147569/225000 [290:53:55<113:02:54,  5.26s/it]
 66%|██████▌   | 147570/225000 [290:54:04<139:10:24,  6.47s/it]
 66%|██████▌   | 147571/225000 [290:54:08<122:15:35,  5.68s/it]
 66%|██████▌   | 147572/225000 [290:54:11<109:03:22,  5.07s/it]
 66%|██████▌   | 147573/225000 [290:54:15<100:07:04,  4.66s/it]
 66%|██████▌   | 147574/225000 [290:54:24<130:09:14,  6.05s/it]
 66%|██████▌   | 147575/225000 [290:54:28<116:27:57,  5.42s/it]
 66%|██████▌   | 147576/225000 [290:54:32<105:16:03,  4.89s/it]
 66%|██████▌   | 147577/225000 [290:54:36<100:25:32,  4.67s/it]
 66%|██████▌   | 147578/225000 [290:54:48<145:20:12,  6.76s/it]
 66%|██████▌   | 147579/225000 [290:54:52<131:44:56,  6.13s/it]
 66%|██████▌   | 147580/225000 [290:54:58<127:31:00,  5.93s/it]
 66%|██████▌   | 147581/225000 [290:55:03<120:09:10,  5.59s/it]
 66%|██████▌   | 147582/225000 [290:55:08<121:11:55,  5.64s/it]
 66%|██████▌   | 147583/225000 [290:55:13<113:59:55,  5.30s/it]
 66%|██████▌   | 147584/225000 [290:55:18<113:21:46,  5.27s/it]
 66%|██████▌   | 147585/225000 [290:55:26<132:24:56,  6.16s/it]
 66%|██████▌   | 147586/225000 [290:55:34<138:51:36,  6.46s/it]
 66%|██████▌   | 147587/225000 [290:55:37<122:50:00,  5.71s/it]
 66%|██████▌   | 147588/225000 [290:55:46<138:26:38,  6.44s/it]
 66%|██████▌   | 147589/225000 [290:55:49<117:16:55,  5.45s/it]
 66%|██████▌   | 147590/225000 [290:55:56<129:04:37,  6.00s/it]
 66%|██████▌   | 147591/225000 [290:56:02<126:57:00,  5.90s/it]
 66%|██████▌   | 147592/225000 [290:56:09<136:51:30,  6.36s/it]
 66%|██████▌   | 147593/225000 [290:56:12<116:25:08,  5.41s/it]
 66%|██████▌   | 147594/225000 [290:56:16<106:10:12,  4.94s/it]
 66%|██████▌   | 147595/225000 [290:56:21<102:19:57,  4.76s/it]
 66%|██████▌   | 147596/225000 [290:56:25<100:11:53,  4.66s/it]
 66%|██████▌   | 147597/225000 [290:56:38<154:07:15,  7.17s/it]
 66%|██████▌   | 147598/225000 [290:56:43<138:48:10,  6.46s/it]
 66%|██████▌   | 147599/225000 [290:56:47<124:06:47,  5.77s/it]
 66%|██████▌   | 147600/225000 [290:56:51<114:54:27,  5.34s/it]
                                                               
{'loss': '0.7761', 'grad_norm': '1.202', 'learning_rate': '1.438e-05', 'epoch': '0.656', 'num_input_tokens_seen': 482235760, 'train_runtime': '1.047e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147600/225000 [290:56:51<114:54:27,  5.34s/it]
 66%|██████▌   | 147601/225000 [290:56:56<110:35:08,  5.14s/it]
 66%|██████▌   | 147602/225000 [290:57:01<107:04:29,  4.98s/it]
 66%|██████▌   | 147603/225000 [290:57:05<105:20:27,  4.90s/it]
 66%|██████▌   | 147604/225000 [290:57:17<151:52:20,  7.06s/it]
 66%|██████▌   | 147605/225000 [290:57:21<129:23:04,  6.02s/it]
 66%|██████▌   | 147606/225000 [290:57:29<139:15:30,  6.48s/it]
 66%|██████▌   | 147607/225000 [290:57:32<120:59:24,  5.63s/it]
 66%|██████▌   | 147608/225000 [290:57:37<117:32:44,  5.47s/it]
 66%|██████▌   | 147609/225000 [290:57:42<114:57:13,  5.35s/it]
 66%|██████▌   | 147610/225000 [290:57:47<112:12:41,  5.22s/it]
 66%|██████▌   | 147611/225000 [290:57:53<113:55:50,  5.30s/it]
 66%|██████▌   | 147612/225000 [290:57:58<114:39:08,  5.33s/it]
 66%|██████▌   | 147613/225000 [290:58:03<111:07:00,  5.17s/it]
 66%|██████▌   | 147614/225000 [290:58:08<109:45:19,  5.11s/it]
 66%|██████▌   | 147615/225000 [290:58:13<108:46:52,  5.06s/it]
 66%|██████▌   | 147616/225000 [290:58:18<111:04:36,  5.17s/it]
 66%|██████▌   | 147617/225000 [290:58:24<113:53:23,  5.30s/it]
 66%|██████▌   | 147618/225000 [290:58:29<110:17:21,  5.13s/it]
 66%|██████▌   | 147619/225000 [290:58:34<110:03:41,  5.12s/it]
 66%|██████▌   | 147620/225000 [290:58:39<112:15:43,  5.22s/it]
 66%|██████▌   | 147621/225000 [290:58:44<110:46:26,  5.15s/it]
 66%|██████▌   | 147622/225000 [290:58:56<152:35:32,  7.10s/it]
 66%|██████▌   | 147623/225000 [290:59:03<152:10:19,  7.08s/it]
 66%|██████▌   | 147624/225000 [290:59:08<136:40:55,  6.36s/it]
 66%|██████▌   | 147625/225000 [290:59:15<144:44:56,  6.73s/it]
 66%|██████▌   | 147626/225000 [290:59:20<130:58:39,  6.09s/it]
 66%|██████▌   | 147627/225000 [290:59:28<145:02:15,  6.75s/it]
 66%|██████▌   | 147628/225000 [290:59:33<131:12:49,  6.11s/it]
 66%|██████▌   | 147629/225000 [290:59:37<118:35:06,  5.52s/it]
 66%|██████▌   | 147630/225000 [290:59:41<110:26:28,  5.14s/it]
 66%|██████▌   | 147631/225000 [290:59:46<106:41:29,  4.96s/it]
 66%|██████▌   | 147632/225000 [290:59:56<141:16:43,  6.57s/it]
 66%|██████▌   | 147633/225000 [291:00:03<147:45:37,  6.88s/it]
 66%|██████▌   | 147634/225000 [291:00:07<124:55:35,  5.81s/it]
 66%|██████▌   | 147635/225000 [291:00:11<117:21:33,  5.46s/it]
 66%|██████▌   | 147636/225000 [291:00:16<112:45:05,  5.25s/it]
 66%|██████▌   | 147637/225000 [291:00:21<110:20:10,  5.13s/it]
 66%|██████▌   | 147638/225000 [291:00:26<109:21:18,  5.09s/it]
 66%|██████▌   | 147639/225000 [291:00:31<108:21:26,  5.04s/it]
 66%|██████▌   | 147640/225000 [291:00:36<107:37:58,  5.01s/it]
 66%|██████▌   | 147641/225000 [291:00:40<104:45:24,  4.87s/it]
 66%|██████▌   | 147642/225000 [291:00:53<154:46:40,  7.20s/it]
 66%|██████▌   | 147643/225000 [291:00:58<139:23:51,  6.49s/it]
 66%|██████▌   | 147644/225000 [291:01:05<140:43:46,  6.55s/it]
 66%|██████▌   | 147645/225000 [291:01:08<122:25:21,  5.70s/it]
 66%|██████▌   | 147646/225000 [291:01:16<132:20:35,  6.16s/it]
 66%|██████▌   | 147647/225000 [291:01:22<131:46:30,  6.13s/it]
 66%|██████▌   | 147648/225000 [291:01:26<121:24:49,  5.65s/it]
 66%|██████▌   | 147649/225000 [291:01:31<117:17:35,  5.46s/it]
 66%|██████▌   | 147650/225000 [291:01:36<110:58:37,  5.17s/it]
                                                               
{'loss': '0.7879', 'grad_norm': '1.098', 'learning_rate': '1.436e-05', 'epoch': '0.6562', 'num_input_tokens_seen': 482367320, 'train_runtime': '1.048e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147650/225000 [291:01:36<110:58:37,  5.17s/it]
 66%|██████▌   | 147651/225000 [291:01:41<109:49:38,  5.11s/it]
 66%|██████▌   | 147652/225000 [291:01:46<108:08:12,  5.03s/it]
 66%|██████▌   | 147653/225000 [291:01:50<105:45:14,  4.92s/it]
 66%|██████▌   | 147654/225000 [291:01:55<106:28:53,  4.96s/it]
 66%|██████▌   | 147655/225000 [291:02:06<145:28:26,  6.77s/it]
 66%|██████▌   | 147656/225000 [291:02:13<146:09:43,  6.80s/it]
 66%|██████▌   | 147657/225000 [291:02:17<126:09:52,  5.87s/it]
 66%|██████▌   | 147658/225000 [291:02:21<113:11:57,  5.27s/it]
 66%|██████▌   | 147659/225000 [291:02:31<143:13:35,  6.67s/it]
 66%|██████▌   | 147660/225000 [291:02:35<130:47:53,  6.09s/it]
 66%|██████▌   | 147661/225000 [291:02:40<121:21:55,  5.65s/it]
 66%|██████▌   | 147662/225000 [291:02:45<117:27:11,  5.47s/it]
 66%|██████▌   | 147663/225000 [291:02:51<118:10:45,  5.50s/it]
 66%|██████▌   | 147664/225000 [291:02:55<113:17:09,  5.27s/it]
 66%|██████▌   | 147665/225000 [291:03:01<113:50:40,  5.30s/it]
 66%|██████▌   | 147666/225000 [291:03:06<114:52:42,  5.35s/it]
 66%|██████▌   | 147667/225000 [291:03:11<113:32:18,  5.29s/it]
 66%|██████▌   | 147668/225000 [291:03:16<111:38:00,  5.20s/it]
 66%|██████▌   | 147669/225000 [291:03:22<112:32:05,  5.24s/it]
 66%|██████▌   | 147670/225000 [291:03:27<111:09:54,  5.18s/it]
 66%|██████▌   | 147671/225000 [291:03:32<112:01:02,  5.21s/it]
 66%|██████▌   | 147672/225000 [291:03:37<112:48:49,  5.25s/it]
 66%|██████▌   | 147673/225000 [291:03:43<114:07:01,  5.31s/it]
 66%|██████▌   | 147674/225000 [291:03:48<113:33:50,  5.29s/it]
 66%|██████▌   | 147675/225000 [291:03:53<113:44:01,  5.30s/it]
 66%|██████▌   | 147676/225000 [291:03:59<114:40:17,  5.34s/it]
 66%|██████▌   | 147677/225000 [291:04:04<116:01:29,  5.40s/it]
 66%|██████▌   | 147678/225000 [291:04:10<117:03:16,  5.45s/it]
 66%|██████▌   | 147679/225000 [291:04:15<118:18:52,  5.51s/it]
 66%|██████▌   | 147680/225000 [291:04:21<118:40:29,  5.53s/it]
 66%|██████▌   | 147681/225000 [291:04:26<118:11:01,  5.50s/it]
 66%|██████▌   | 147682/225000 [291:04:37<148:10:37,  6.90s/it]
 66%|██████▌   | 147683/225000 [291:04:48<177:43:36,  8.28s/it]
 66%|██████▌   | 147684/225000 [291:04:52<147:31:47,  6.87s/it]
 66%|██████▌   | 147685/225000 [291:04:58<143:30:09,  6.68s/it]
 66%|██████▌   | 147686/225000 [291:05:03<131:28:53,  6.12s/it]
 66%|██████▌   | 147687/225000 [291:05:07<119:32:08,  5.57s/it]
 66%|██████▌   | 147688/225000 [291:05:12<117:36:29,  5.48s/it]
 66%|██████▌   | 147689/225000 [291:05:17<112:42:26,  5.25s/it]
 66%|██████▌   | 147690/225000 [291:05:22<110:13:35,  5.13s/it]
 66%|██████▌   | 147691/225000 [291:05:27<110:13:27,  5.13s/it]
 66%|██████▌   | 147692/225000 [291:05:38<148:43:30,  6.93s/it]
 66%|██████▌   | 147693/225000 [291:05:43<135:33:59,  6.31s/it]
 66%|██████▌   | 147694/225000 [291:05:48<129:57:24,  6.05s/it]
 66%|██████▌   | 147695/225000 [291:05:57<144:24:30,  6.72s/it]
 66%|██████▌   | 147696/225000 [291:06:00<124:23:50,  5.79s/it]
 66%|██████▌   | 147697/225000 [291:06:09<144:01:56,  6.71s/it]
 66%|██████▌   | 147698/225000 [291:06:12<121:44:11,  5.67s/it]
 66%|██████▌   | 147699/225000 [291:06:20<135:04:03,  6.29s/it]
 66%|██████▌   | 147700/225000 [291:06:25<123:25:29,  5.75s/it]
                                                               
{'loss': '0.7991', 'grad_norm': '1.188', 'learning_rate': '1.434e-05', 'epoch': '0.6564', 'num_input_tokens_seen': 482497176, 'train_runtime': '1.048e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147700/225000 [291:06:25<123:25:29,  5.75s/it]
 66%|██████▌   | 147701/225000 [291:06:29<114:29:10,  5.33s/it]
 66%|██████▌   | 147702/225000 [291:06:34<109:14:18,  5.09s/it]
 66%|██████▌   | 147703/225000 [291:06:38<104:01:36,  4.84s/it]
 66%|██████▌   | 147704/225000 [291:06:43<105:30:08,  4.91s/it]
 66%|██████▌   | 147705/225000 [291:06:54<145:49:07,  6.79s/it]
 66%|██████▌   | 147706/225000 [291:06:58<126:21:38,  5.89s/it]
 66%|██████▌   | 147707/225000 [291:07:01<111:47:22,  5.21s/it]
 66%|██████▌   | 147708/225000 [291:07:09<129:44:11,  6.04s/it]
 66%|██████▌   | 147709/225000 [291:07:13<115:21:40,  5.37s/it]
 66%|██████▌   | 147710/225000 [291:07:20<122:45:32,  5.72s/it]
 66%|██████▌   | 147711/225000 [291:07:26<126:50:57,  5.91s/it]
 66%|██████▌   | 147712/225000 [291:07:29<109:19:37,  5.09s/it]
 66%|██████▌   | 147713/225000 [291:07:34<109:07:02,  5.08s/it]
 66%|██████▌   | 147714/225000 [291:07:39<108:33:37,  5.06s/it]
 66%|██████▌   | 147715/225000 [291:07:48<133:15:25,  6.21s/it]
 66%|██████▌   | 147716/225000 [291:07:53<124:00:45,  5.78s/it]
 66%|██████▌   | 147717/225000 [291:07:59<122:10:36,  5.69s/it]
 66%|██████▌   | 147718/225000 [291:08:04<118:26:42,  5.52s/it]
 66%|██████▌   | 147719/225000 [291:08:09<116:23:54,  5.42s/it]
 66%|██████▌   | 147720/225000 [291:08:14<115:35:39,  5.38s/it]
 66%|██████▌   | 147721/225000 [291:08:20<116:25:15,  5.42s/it]
 66%|██████▌   | 147722/225000 [291:08:25<116:49:35,  5.44s/it]
 66%|██████▌   | 147723/225000 [291:08:31<118:16:56,  5.51s/it]
 66%|██████▌   | 147724/225000 [291:08:36<115:22:28,  5.37s/it]
 66%|██████▌   | 147725/225000 [291:08:41<111:08:34,  5.18s/it]
 66%|██████▌   | 147726/225000 [291:08:46<112:33:38,  5.24s/it]
 66%|██████▌   | 147727/225000 [291:08:51<113:34:54,  5.29s/it]
 66%|██████▌   | 147728/225000 [291:08:56<111:34:55,  5.20s/it]
 66%|██████▌   | 147729/225000 [291:09:02<112:31:27,  5.24s/it]
 66%|██████▌   | 147730/225000 [291:09:07<115:18:39,  5.37s/it]
 66%|██████▌   | 147731/225000 [291:09:12<112:11:32,  5.23s/it]
 66%|██████▌   | 147732/225000 [291:09:18<113:52:05,  5.31s/it]
 66%|██████▌   | 147733/225000 [291:09:25<124:04:04,  5.78s/it]
 66%|██████▌   | 147734/225000 [291:09:32<133:42:41,  6.23s/it]
 66%|██████▌   | 147735/225000 [291:09:37<129:02:34,  6.01s/it]
 66%|██████▌   | 147736/225000 [291:09:43<124:32:34,  5.80s/it]
 66%|██████▌   | 147737/225000 [291:09:48<121:50:43,  5.68s/it]
 66%|██████▌   | 147738/225000 [291:09:54<119:50:23,  5.58s/it]
 66%|██████▌   | 147739/225000 [291:09:59<116:01:38,  5.41s/it]
 66%|██████▌   | 147740/225000 [291:10:04<113:41:49,  5.30s/it]
 66%|██████▌   | 147741/225000 [291:10:09<115:22:34,  5.38s/it]
 66%|██████▌   | 147742/225000 [291:10:15<116:09:52,  5.41s/it]
 66%|██████▌   | 147743/225000 [291:10:20<116:03:31,  5.41s/it]
 66%|██████▌   | 147744/225000 [291:10:30<142:23:20,  6.64s/it]
 66%|██████▌   | 147745/225000 [291:10:35<131:50:12,  6.14s/it]
 66%|██████▌   | 147746/225000 [291:10:42<139:19:29,  6.49s/it]
 66%|██████▌   | 147747/225000 [291:10:52<165:44:39,  7.72s/it]
 66%|██████▌   | 147748/225000 [291:10:57<143:00:21,  6.66s/it]
 66%|██████▌   | 147749/225000 [291:11:01<128:19:15,  5.98s/it]
 66%|██████▌   | 147750/225000 [291:11:06<119:09:07,  5.55s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '0.9989', 'learning_rate': '1.433e-05', 'epoch': '0.6567', 'num_input_tokens_seen': 482627136, 'train_runtime': '1.048e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147750/225000 [291:11:06<119:09:07,  5.55s/it]
 66%|██████▌   | 147751/225000 [291:11:13<129:15:34,  6.02s/it]
 66%|██████▌   | 147752/225000 [291:11:21<142:56:50,  6.66s/it]
 66%|██████▌   | 147753/225000 [291:11:25<126:59:41,  5.92s/it]
 66%|██████▌   | 147754/225000 [291:11:29<114:31:16,  5.34s/it]
 66%|██████▌   | 147755/225000 [291:11:37<130:14:18,  6.07s/it]
 66%|██████▌   | 147756/225000 [291:11:44<136:11:11,  6.35s/it]
 66%|██████▌   | 147757/225000 [291:11:47<119:20:35,  5.56s/it]
 66%|██████▌   | 147758/225000 [291:11:57<146:07:58,  6.81s/it]
 66%|██████▌   | 147759/225000 [291:12:02<131:33:59,  6.13s/it]
 66%|██████▌   | 147760/225000 [291:12:06<119:08:33,  5.55s/it]
 66%|██████▌   | 147761/225000 [291:12:10<111:49:22,  5.21s/it]
 66%|██████▌   | 147762/225000 [291:12:14<104:21:50,  4.86s/it]
 66%|██████▌   | 147763/225000 [291:12:19<103:55:21,  4.84s/it]
 66%|██████▌   | 147764/225000 [291:12:29<137:36:41,  6.41s/it]
 66%|██████▌   | 147765/225000 [291:12:35<132:55:41,  6.20s/it]
 66%|██████▌   | 147766/225000 [291:12:41<129:30:50,  6.04s/it]
 66%|██████▌   | 147767/225000 [291:12:48<135:17:18,  6.31s/it]
 66%|██████▌   | 147768/225000 [291:12:52<120:45:11,  5.63s/it]
 66%|██████▌   | 147769/225000 [291:12:57<116:51:43,  5.45s/it]
 66%|██████▌   | 147770/225000 [291:13:02<113:56:51,  5.31s/it]
 66%|██████▌   | 147771/225000 [291:13:06<109:45:06,  5.12s/it]
 66%|██████▌   | 147772/225000 [291:13:11<108:20:59,  5.05s/it]
 66%|██████▌   | 147773/225000 [291:13:16<109:13:57,  5.09s/it]
 66%|██████▌   | 147774/225000 [291:13:22<109:18:29,  5.10s/it]
 66%|██████▌   | 147775/225000 [291:13:27<110:49:48,  5.17s/it]
 66%|██████▌   | 147776/225000 [291:13:32<109:28:27,  5.10s/it]
 66%|██████▌   | 147777/225000 [291:13:37<108:02:19,  5.04s/it]
 66%|██████▌   | 147778/225000 [291:13:43<115:22:25,  5.38s/it]
 66%|██████▌   | 147779/225000 [291:13:56<165:21:48,  7.71s/it]
 66%|██████▌   | 147780/225000 [291:14:01<148:50:11,  6.94s/it]
 66%|██████▌   | 147781/225000 [291:14:06<134:58:02,  6.29s/it]
 66%|██████▌   | 147782/225000 [291:14:11<127:14:19,  5.93s/it]
 66%|██████▌   | 147783/225000 [291:14:16<121:30:42,  5.67s/it]
 66%|██████▌   | 147784/225000 [291:14:21<117:41:58,  5.49s/it]
 66%|██████▌   | 147785/225000 [291:14:26<113:55:48,  5.31s/it]
 66%|██████▌   | 147786/225000 [291:14:31<111:40:07,  5.21s/it]
 66%|██████▌   | 147787/225000 [291:14:36<110:25:22,  5.15s/it]
 66%|██████▌   | 147788/225000 [291:14:41<109:53:31,  5.12s/it]
 66%|██████▌   | 147789/225000 [291:14:46<109:39:20,  5.11s/it]
 66%|██████▌   | 147790/225000 [291:14:51<109:31:34,  5.11s/it]
 66%|██████▌   | 147791/225000 [291:14:57<110:26:07,  5.15s/it]
 66%|██████▌   | 147792/225000 [291:15:02<111:51:10,  5.22s/it]
 66%|██████▌   | 147793/225000 [291:15:07<112:05:02,  5.23s/it]
 66%|██████▌   | 147794/225000 [291:15:16<135:33:00,  6.32s/it]
 66%|██████▌   | 147795/225000 [291:15:26<158:08:03,  7.37s/it]
 66%|██████▌   | 147796/225000 [291:15:29<134:10:28,  6.26s/it]
 66%|██████▌   | 147797/225000 [291:15:38<148:03:00,  6.90s/it]
 66%|██████▌   | 147798/225000 [291:15:41<126:47:25,  5.91s/it]
 66%|██████▌   | 147799/225000 [291:15:48<133:29:35,  6.22s/it]
 66%|██████▌   | 147800/225000 [291:15:55<133:42:12,  6.23s/it]
                                                               
{'loss': '0.7882', 'grad_norm': '1.041', 'learning_rate': '1.431e-05', 'epoch': '0.6569', 'num_input_tokens_seen': 482757264, 'train_runtime': '1.049e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147800/225000 [291:15:55<133:42:12,  6.23s/it]
 66%|██████▌   | 147801/225000 [291:16:01<135:16:50,  6.31s/it]
 66%|██████▌   | 147802/225000 [291:16:05<121:27:32,  5.66s/it]
 66%|██████▌   | 147803/225000 [291:16:09<105:45:02,  4.93s/it]
 66%|██████▌   | 147804/225000 [291:16:13<103:15:21,  4.82s/it]
 66%|██████▌   | 147805/225000 [291:16:18<102:02:14,  4.76s/it]
 66%|██████▌   | 147806/225000 [291:16:22<100:01:09,  4.66s/it]
 66%|██████▌   | 147807/225000 [291:16:27<103:04:24,  4.81s/it]
 66%|██████▌   | 147808/225000 [291:16:34<114:29:07,  5.34s/it]
 66%|██████▌   | 147809/225000 [291:16:45<150:50:53,  7.04s/it]
 66%|██████▌   | 147810/225000 [291:16:48<127:57:29,  5.97s/it]
 66%|██████▌   | 147811/225000 [291:16:57<143:10:48,  6.68s/it]
 66%|██████▌   | 147812/225000 [291:17:00<122:25:52,  5.71s/it]
 66%|██████▌   | 147813/225000 [291:17:08<136:47:25,  6.38s/it]
 66%|██████▌   | 147814/225000 [291:17:12<122:58:08,  5.74s/it]
 66%|██████▌   | 147815/225000 [291:17:17<114:02:08,  5.32s/it]
 66%|██████▌   | 147816/225000 [291:17:21<107:50:34,  5.03s/it]
 66%|██████▌   | 147817/225000 [291:17:27<114:34:36,  5.34s/it]
 66%|██████▌   | 147818/225000 [291:17:36<139:35:05,  6.51s/it]
 66%|██████▌   | 147819/225000 [291:17:40<120:56:09,  5.64s/it]
 66%|██████▌   | 147820/225000 [291:17:47<132:28:55,  6.18s/it]
 66%|██████▌   | 147821/225000 [291:17:52<121:28:08,  5.67s/it]
 66%|██████▌   | 147822/225000 [291:17:57<117:02:02,  5.46s/it]
 66%|██████▌   | 147823/225000 [291:18:02<112:52:44,  5.27s/it]
 66%|██████▌   | 147824/225000 [291:18:07<114:06:33,  5.32s/it]
 66%|██████▌   | 147825/225000 [291:18:12<113:46:12,  5.31s/it]
 66%|██████▌   | 147826/225000 [291:18:17<111:35:03,  5.21s/it]
 66%|██████▌   | 147827/225000 [291:18:22<110:22:24,  5.15s/it]
 66%|██████▌   | 147828/225000 [291:18:28<110:43:05,  5.16s/it]
 66%|██████▌   | 147829/225000 [291:18:33<111:08:55,  5.19s/it]
 66%|██████▌   | 147830/225000 [291:18:38<111:00:26,  5.18s/it]
 66%|██████▌   | 147831/225000 [291:18:42<106:43:31,  4.98s/it]
 66%|██████▌   | 147832/225000 [291:18:55<152:32:22,  7.12s/it]
 66%|██████▌   | 147833/225000 [291:19:00<140:10:46,  6.54s/it]
 66%|██████▌   | 147834/225000 [291:19:04<122:27:14,  5.71s/it]
 66%|██████▌   | 147835/225000 [291:19:08<115:04:04,  5.37s/it]
 66%|██████▌   | 147836/225000 [291:19:14<121:25:30,  5.66s/it]
 66%|██████▌   | 147837/225000 [291:19:23<137:57:59,  6.44s/it]
 66%|██████▌   | 147838/225000 [291:19:26<119:13:35,  5.56s/it]
 66%|██████▌   | 147839/225000 [291:19:36<146:09:48,  6.82s/it]
 66%|██████▌   | 147840/225000 [291:19:39<123:28:58,  5.76s/it]
 66%|██████▌   | 147841/225000 [291:19:44<115:04:00,  5.37s/it]
 66%|██████▌   | 147842/225000 [291:19:48<108:59:38,  5.09s/it]
 66%|██████▌   | 147843/225000 [291:19:53<105:01:40,  4.90s/it]
 66%|██████▌   | 147844/225000 [291:20:04<145:35:01,  6.79s/it]
 66%|██████▌   | 147845/225000 [291:20:08<129:06:39,  6.02s/it]
 66%|██████▌   | 147846/225000 [291:20:14<127:00:03,  5.93s/it]
 66%|██████▌   | 147847/225000 [291:20:19<124:06:44,  5.79s/it]
 66%|██████▌   | 147848/225000 [291:20:24<117:14:34,  5.47s/it]
 66%|██████▌   | 147849/225000 [291:20:30<122:40:56,  5.72s/it]
 66%|██████▌   | 147850/225000 [291:20:34<112:43:27,  5.26s/it]
                                                               
{'loss': '0.7878', 'grad_norm': '1.083', 'learning_rate': '1.43e-05', 'epoch': '0.6571', 'num_input_tokens_seen': 482886936, 'train_runtime': '1.049e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147850/225000 [291:20:34<112:43:27,  5.26s/it]
 66%|██████▌   | 147851/225000 [291:20:43<131:44:57,  6.15s/it]
 66%|██████▌   | 147852/225000 [291:20:46<115:48:12,  5.40s/it]
 66%|██████▌   | 147853/225000 [291:20:51<112:33:29,  5.25s/it]
 66%|██████▌   | 147854/225000 [291:20:59<130:34:29,  6.09s/it]
 66%|██████▌   | 147855/225000 [291:21:03<117:47:12,  5.50s/it]
 66%|██████▌   | 147856/225000 [291:21:07<106:50:52,  4.99s/it]
 66%|██████▌   | 147857/225000 [291:21:18<146:02:12,  6.82s/it]
 66%|██████▌   | 147858/225000 [291:21:23<130:28:08,  6.09s/it]
 66%|██████▌   | 147859/225000 [291:21:26<115:10:07,  5.37s/it]
 66%|██████▌   | 147860/225000 [291:21:30<107:06:25,  5.00s/it]
 66%|██████▌   | 147861/225000 [291:21:38<124:58:56,  5.83s/it]
 66%|██████▌   | 147862/225000 [291:21:44<126:48:10,  5.92s/it]
 66%|██████▌   | 147863/225000 [291:21:48<112:34:10,  5.25s/it]
 66%|██████▌   | 147864/225000 [291:21:53<108:00:53,  5.04s/it]
 66%|██████▌   | 147865/225000 [291:22:01<126:08:59,  5.89s/it]
 66%|██████▌   | 147866/225000 [291:22:05<115:35:43,  5.40s/it]
 66%|██████▌   | 147867/225000 [291:22:12<126:33:14,  5.91s/it]
 66%|██████▌   | 147868/225000 [291:22:15<110:00:42,  5.13s/it]
 66%|██████▌   | 147869/225000 [291:22:24<131:18:06,  6.13s/it]
 66%|██████▌   | 147870/225000 [291:22:28<120:56:32,  5.64s/it]
 66%|██████▌   | 147871/225000 [291:22:32<111:42:09,  5.21s/it]
 66%|██████▌   | 147872/225000 [291:22:37<108:52:41,  5.08s/it]
 66%|██████▌   | 147873/225000 [291:22:42<104:56:54,  4.90s/it]
 66%|██████▌   | 147874/225000 [291:22:46<103:07:57,  4.81s/it]
 66%|██████▌   | 147875/225000 [291:22:51<104:56:59,  4.90s/it]
 66%|██████▌   | 147876/225000 [291:23:02<140:32:34,  6.56s/it]
 66%|██████▌   | 147877/225000 [291:23:08<138:48:24,  6.48s/it]
 66%|██████▌   | 147878/225000 [291:23:11<118:27:09,  5.53s/it]
 66%|██████▌   | 147879/225000 [291:23:21<141:52:28,  6.62s/it]
 66%|██████▌   | 147880/225000 [291:23:25<126:47:45,  5.92s/it]
 66%|██████▌   | 147881/225000 [291:23:29<116:37:01,  5.44s/it]
 66%|██████▌   | 147882/225000 [291:23:33<108:36:54,  5.07s/it]
 66%|██████▌   | 147883/225000 [291:23:44<143:20:34,  6.69s/it]
 66%|██████▌   | 147884/225000 [291:23:48<125:16:31,  5.85s/it]
 66%|██████▌   | 147885/225000 [291:23:51<111:08:09,  5.19s/it]
 66%|██████▌   | 147886/225000 [291:23:59<127:45:13,  5.96s/it]
 66%|██████▌   | 147887/225000 [291:24:04<118:00:01,  5.51s/it]
 66%|██████▌   | 147888/225000 [291:24:11<131:21:06,  6.13s/it]
 66%|██████▌   | 147889/225000 [291:24:15<114:43:15,  5.36s/it]
 66%|██████▌   | 147890/225000 [291:24:18<104:27:40,  4.88s/it]
 66%|██████▌   | 147891/225000 [291:24:28<134:14:46,  6.27s/it]
 66%|██████▌   | 147892/225000 [291:24:33<126:21:11,  5.90s/it]
 66%|██████▌   | 147893/225000 [291:24:36<110:44:17,  5.17s/it]
 66%|██████▌   | 147894/225000 [291:24:44<126:38:26,  5.91s/it]
 66%|██████▌   | 147895/225000 [291:24:50<123:38:42,  5.77s/it]
 66%|██████▌   | 147896/225000 [291:24:53<109:43:21,  5.12s/it]
 66%|██████▌   | 147897/225000 [291:25:03<141:09:32,  6.59s/it]
 66%|██████▌   | 147898/225000 [291:25:07<120:40:25,  5.63s/it]
 66%|██████▌   | 147899/225000 [291:25:10<107:55:57,  5.04s/it]
 66%|██████▌   | 147900/225000 [291:25:19<132:00:24,  6.16s/it]
                                                               
{'loss': '0.7802', 'grad_norm': '1.103', 'learning_rate': '1.428e-05', 'epoch': '0.6573', 'num_input_tokens_seen': 483016352, 'train_runtime': '1.049e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147900/225000 [291:25:19<132:00:24,  6.16s/it]
 66%|██████▌   | 147901/225000 [291:25:23<115:59:43,  5.42s/it]
 66%|██████▌   | 147902/225000 [291:25:27<108:17:14,  5.06s/it]
 66%|██████▌   | 147903/225000 [291:25:31<103:17:49,  4.82s/it]
 66%|██████▌   | 147904/225000 [291:25:41<134:30:50,  6.28s/it]
 66%|██████▌   | 147905/225000 [291:25:48<137:51:15,  6.44s/it]
 66%|██████▌   | 147906/225000 [291:25:51<119:32:19,  5.58s/it]
 66%|██████▌   | 147907/225000 [291:26:00<137:39:10,  6.43s/it]
 66%|██████▌   | 147908/225000 [291:26:06<134:09:44,  6.27s/it]
 66%|██████▌   | 147909/225000 [291:26:09<118:49:00,  5.55s/it]
 66%|██████▌   | 147910/225000 [291:26:13<108:55:59,  5.09s/it]
 66%|██████▌   | 147911/225000 [291:26:20<118:17:40,  5.52s/it]
 66%|██████▌   | 147912/225000 [291:26:28<135:47:24,  6.34s/it]
 66%|██████▌   | 147913/225000 [291:26:32<119:49:22,  5.60s/it]
 66%|██████▌   | 147914/225000 [291:26:36<111:41:42,  5.22s/it]
 66%|██████▌   | 147915/225000 [291:26:42<111:47:06,  5.22s/it]
 66%|██████▌   | 147916/225000 [291:26:47<112:19:51,  5.25s/it]
 66%|██████▌   | 147917/225000 [291:26:52<113:31:11,  5.30s/it]
 66%|██████▌   | 147918/225000 [291:26:58<114:52:20,  5.36s/it]
 66%|██████▌   | 147919/225000 [291:27:03<114:39:31,  5.36s/it]
 66%|██████▌   | 147920/225000 [291:27:08<112:04:03,  5.23s/it]
 66%|██████▌   | 147921/225000 [291:27:13<111:38:14,  5.21s/it]
 66%|██████▌   | 147922/225000 [291:27:19<112:39:36,  5.26s/it]
 66%|██████▌   | 147923/225000 [291:27:24<111:36:20,  5.21s/it]
 66%|██████▌   | 147924/225000 [291:27:29<111:41:05,  5.22s/it]
 66%|██████▌   | 147925/225000 [291:27:34<111:46:49,  5.22s/it]
 66%|██████▌   | 147926/225000 [291:27:40<112:55:23,  5.27s/it]
 66%|██████▌   | 147927/225000 [291:27:50<145:50:31,  6.81s/it]
 66%|██████▌   | 147928/225000 [291:27:59<160:23:09,  7.49s/it]
 66%|██████▌   | 147929/225000 [291:28:03<134:52:30,  6.30s/it]
 66%|██████▌   | 147930/225000 [291:28:11<147:45:55,  6.90s/it]
 66%|██████▌   | 147931/225000 [291:28:15<130:26:32,  6.09s/it]
 66%|██████▌   | 147932/225000 [291:28:21<127:47:08,  5.97s/it]
 66%|██████▌   | 147933/225000 [291:28:27<128:26:18,  6.00s/it]
 66%|██████▌   | 147934/225000 [291:28:34<132:17:02,  6.18s/it]
 66%|██████▌   | 147935/225000 [291:28:39<124:58:51,  5.84s/it]
 66%|██████▌   | 147936/225000 [291:28:42<111:11:18,  5.19s/it]
 66%|██████▌   | 147937/225000 [291:28:51<134:20:39,  6.28s/it]
 66%|██████▌   | 147938/225000 [291:28:54<115:55:55,  5.42s/it]
 66%|██████▌   | 147939/225000 [291:28:59<110:30:01,  5.16s/it]
 66%|██████▌   | 147940/225000 [291:29:04<108:27:07,  5.07s/it]
 66%|██████▌   | 147941/225000 [291:29:08<104:48:09,  4.90s/it]
 66%|██████▌   | 147942/225000 [291:29:18<136:57:29,  6.40s/it]
 66%|██████▌   | 147943/225000 [291:29:23<129:11:14,  6.04s/it]
 66%|██████▌   | 147944/225000 [291:29:28<116:30:34,  5.44s/it]
 66%|██████▌   | 147945/225000 [291:29:32<107:09:34,  5.01s/it]
 66%|██████▌   | 147946/225000 [291:29:37<108:27:19,  5.07s/it]
 66%|██████▌   | 147947/225000 [291:29:49<153:28:50,  7.17s/it]
 66%|██████▌   | 147948/225000 [291:29:53<132:43:15,  6.20s/it]
 66%|██████▌   | 147949/225000 [291:29:57<117:54:13,  5.51s/it]
 66%|██████▌   | 147950/225000 [291:30:03<121:06:04,  5.66s/it]
                                                               
{'loss': '0.7855', 'grad_norm': '1.009', 'learning_rate': '1.426e-05', 'epoch': '0.6576', 'num_input_tokens_seen': 483146896, 'train_runtime': '1.049e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 147950/225000 [291:30:03<121:06:04,  5.66s/it]
 66%|██████▌   | 147951/225000 [291:30:11<138:12:40,  6.46s/it]
 66%|██████▌   | 147952/225000 [291:30:15<119:49:01,  5.60s/it]
 66%|██████▌   | 147953/225000 [291:30:18<107:59:08,  5.05s/it]
 66%|██████▌   | 147954/225000 [291:30:29<142:34:41,  6.66s/it]
 66%|██████▌   | 147955/225000 [291:30:33<129:21:34,  6.04s/it]
 66%|██████▌   | 147956/225000 [291:30:38<120:08:43,  5.61s/it]
 66%|██████▌   | 147957/225000 [291:30:42<112:39:45,  5.26s/it]
 66%|██████▌   | 147958/225000 [291:30:47<108:42:12,  5.08s/it]
 66%|██████▌   | 147959/225000 [291:30:52<104:50:53,  4.90s/it]
 66%|██████▌   | 147960/225000 [291:30:56<102:03:42,  4.77s/it]
 66%|██████▌   | 147961/225000 [291:31:07<139:04:21,  6.50s/it]
 66%|██████▌   | 147962/225000 [291:31:13<141:39:52,  6.62s/it]
 66%|██████▌   | 147963/225000 [291:31:21<145:37:41,  6.81s/it]
 66%|██████▌   | 147964/225000 [291:31:24<123:28:28,  5.77s/it]
 66%|██████▌   | 147965/225000 [291:31:29<116:04:17,  5.42s/it]
 66%|██████▌   | 147966/225000 [291:31:36<127:13:58,  5.95s/it]
 66%|██████▌   | 147967/225000 [291:31:40<112:39:39,  5.27s/it]
 66%|██████▌   | 147968/225000 [291:31:49<140:01:24,  6.54s/it]
 66%|██████▌   | 147969/225000 [291:31:53<123:29:55,  5.77s/it]
 66%|██████▌   | 147970/225000 [291:31:59<123:09:09,  5.76s/it]
 66%|██████▌   | 147971/225000 [291:32:04<122:16:42,  5.71s/it]
 66%|██████▌   | 147972/225000 [291:32:10<120:32:54,  5.63s/it]
 66%|██████▌   | 147973/225000 [291:32:16<125:11:32,  5.85s/it]
 66%|██████▌   | 147974/225000 [291:32:25<142:23:06,  6.65s/it]
 66%|██████▌   | 147975/225000 [291:32:28<123:01:39,  5.75s/it]
 66%|██████▌   | 147976/225000 [291:32:37<140:54:57,  6.59s/it]
 66%|██████▌   | 147977/225000 [291:32:40<120:23:53,  5.63s/it]
 66%|██████▌   | 147978/225000 [291:32:47<130:49:51,  6.12s/it]
 66%|██████▌   | 147979/225000 [291:32:52<121:04:15,  5.66s/it]
 66%|██████▌   | 147980/225000 [291:32:56<107:13:15,  5.01s/it]
 66%|██████▌   | 147981/225000 [291:33:03<122:41:27,  5.73s/it]
 66%|██████▌   | 147982/225000 [291:33:06<108:08:01,  5.05s/it]
 66%|██████▌   | 147983/225000 [291:33:15<131:11:02,  6.13s/it]
 66%|██████▌   | 147984/225000 [291:33:19<116:08:21,  5.43s/it]
 66%|██████▌   | 147985/225000 [291:33:27<130:19:19,  6.09s/it]
 66%|██████▌   | 147986/225000 [291:33:32<125:15:43,  5.86s/it]
 66%|██████▌   | 147987/225000 [291:33:36<111:41:49,  5.22s/it]
 66%|██████▌   | 147988/225000 [291:33:40<104:17:43,  4.88s/it]
 66%|██████▌   | 147989/225000 [291:33:51<144:03:18,  6.73s/it]
 66%|██████▌   | 147990/225000 [291:33:54<122:30:12,  5.73s/it]
 66%|██████▌   | 147991/225000 [291:33:59<114:39:41,  5.36s/it]
 66%|██████▌   | 147992/225000 [291:34:04<113:24:35,  5.30s/it]
 66%|██████▌   | 147993/225000 [291:34:08<109:16:50,  5.11s/it]
 66%|██████▌   | 147994/225000 [291:34:13<107:45:23,  5.04s/it]
 66%|██████▌   | 147995/225000 [291:34:18<108:35:31,  5.08s/it]
 66%|██████▌   | 147996/225000 [291:34:24<110:22:43,  5.16s/it]
 66%|██████▌   | 147997/225000 [291:34:29<107:29:32,  5.03s/it]
 66%|██████▌   | 147998/225000 [291:34:34<107:18:03,  5.02s/it]
 66%|██████▌   | 147999/225000 [291:34:38<105:53:13,  4.95s/it]
 66%|██████▌   | 148000/225000 [291:34:43<106:26:05,  4.98s/it]
                                                               
{'loss': '0.7807', 'grad_norm': '1.119', 'learning_rate': '1.425e-05', 'epoch': '0.6578', 'num_input_tokens_seen': 483276608, 'train_runtime': '1.05e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148000/225000 [291:34:43<106:26:05,  4.98s/it]
 66%|██████▌   | 148001/225000 [291:34:48<106:45:31,  4.99s/it]
 66%|██████▌   | 148002/225000 [291:34:54<108:16:19,  5.06s/it]
 66%|██████▌   | 148003/225000 [291:34:59<108:52:53,  5.09s/it]
 66%|██████▌   | 148004/225000 [291:35:04<109:05:52,  5.10s/it]
 66%|██████▌   | 148005/225000 [291:35:17<158:18:14,  7.40s/it]
 66%|██████▌   | 148006/225000 [291:35:22<147:51:14,  6.91s/it]
 66%|██████▌   | 148007/225000 [291:35:26<126:49:26,  5.93s/it]
 66%|██████▌   | 148008/225000 [291:35:31<120:22:32,  5.63s/it]
 66%|██████▌   | 148009/225000 [291:35:40<142:24:50,  6.66s/it]
 66%|██████▌   | 148010/225000 [291:35:47<145:20:17,  6.80s/it]
 66%|██████▌   | 148011/225000 [291:35:51<128:10:07,  5.99s/it]
 66%|██████▌   | 148012/225000 [291:35:56<117:01:45,  5.47s/it]
 66%|██████▌   | 148013/225000 [291:36:00<109:01:35,  5.10s/it]
 66%|██████▌   | 148014/225000 [291:36:04<102:34:18,  4.80s/it]
 66%|██████▌   | 148015/225000 [291:36:11<115:22:18,  5.40s/it]
 66%|██████▌   | 148016/225000 [291:36:21<148:24:36,  6.94s/it]
 66%|██████▌   | 148017/225000 [291:36:25<128:45:49,  6.02s/it]
 66%|██████▌   | 148018/225000 [291:36:29<113:45:13,  5.32s/it]
 66%|██████▌   | 148019/225000 [291:36:33<107:03:14,  5.01s/it]
 66%|██████▌   | 148020/225000 [291:36:44<145:29:52,  6.80s/it]
 66%|██████▌   | 148021/225000 [291:36:48<127:11:48,  5.95s/it]
 66%|██████▌   | 148022/225000 [291:36:52<113:20:06,  5.30s/it]
 66%|██████▌   | 148023/225000 [291:37:00<130:23:41,  6.10s/it]
 66%|██████▌   | 148024/225000 [291:37:07<134:31:58,  6.29s/it]
 66%|██████▌   | 148025/225000 [291:37:10<118:15:58,  5.53s/it]
 66%|██████▌   | 148026/225000 [291:37:19<140:13:45,  6.56s/it]
 66%|██████▌   | 148027/225000 [291:37:23<120:16:58,  5.63s/it]
 66%|██████▌   | 148028/225000 [291:37:32<141:09:18,  6.60s/it]
 66%|██████▌   | 148029/225000 [291:37:35<119:16:41,  5.58s/it]
 66%|██████▌   | 148030/225000 [291:37:42<131:20:16,  6.14s/it]
 66%|██████▌   | 148031/225000 [291:37:46<115:55:30,  5.42s/it]
 66%|██████▌   | 148032/225000 [291:37:50<104:25:45,  4.88s/it]
 66%|██████▌   | 148033/225000 [291:37:55<107:31:37,  5.03s/it]
 66%|██████▌   | 148034/225000 [291:38:02<118:58:14,  5.56s/it]
 66%|██████▌   | 148035/225000 [291:38:06<107:59:14,  5.05s/it]
 66%|██████▌   | 148036/225000 [291:38:10<103:46:49,  4.85s/it]
 66%|██████▌   | 148037/225000 [291:38:15<104:45:06,  4.90s/it]
 66%|██████▌   | 148038/225000 [291:38:19<101:07:12,  4.73s/it]
 66%|██████▌   | 148039/225000 [291:38:26<112:57:53,  5.28s/it]
 66%|██████▌   | 148040/225000 [291:38:38<158:25:10,  7.41s/it]
 66%|██████▌   | 148041/225000 [291:38:42<131:40:22,  6.16s/it]
 66%|██████▌   | 148042/225000 [291:38:47<129:25:28,  6.05s/it]
 66%|██████▌   | 148043/225000 [291:38:52<121:51:44,  5.70s/it]
 66%|██████▌   | 148044/225000 [291:38:56<109:34:38,  5.13s/it]
 66%|██████▌   | 148045/225000 [291:39:05<135:39:36,  6.35s/it]
 66%|██████▌   | 148046/225000 [291:39:10<127:20:04,  5.96s/it]
 66%|██████▌   | 148047/225000 [291:39:15<122:11:41,  5.72s/it]
 66%|██████▌   | 148048/225000 [291:39:23<131:33:46,  6.15s/it]
 66%|██████▌   | 148049/225000 [291:39:26<112:47:45,  5.28s/it]
 66%|██████▌   | 148050/225000 [291:39:33<125:34:30,  5.87s/it]
                                                               
{'loss': '0.7787', 'grad_norm': '1.025', 'learning_rate': '1.423e-05', 'epoch': '0.658', 'num_input_tokens_seen': 483406416, 'train_runtime': '1.05e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148050/225000 [291:39:33<125:34:30,  5.87s/it]
 66%|██████▌   | 148051/225000 [291:39:37<113:25:39,  5.31s/it]
 66%|██████▌   | 148052/225000 [291:39:41<104:27:52,  4.89s/it]
 66%|██████▌   | 148053/225000 [291:39:45<98:28:02,  4.61s/it] 
 66%|██████▌   | 148054/225000 [291:39:52<113:52:20,  5.33s/it]
 66%|██████▌   | 148055/225000 [291:40:01<137:04:17,  6.41s/it]
 66%|██████▌   | 148056/225000 [291:40:04<117:00:23,  5.47s/it]
 66%|██████▌   | 148057/225000 [291:40:09<115:22:37,  5.40s/it]
 66%|██████▌   | 148058/225000 [291:40:17<130:12:11,  6.09s/it]
 66%|██████▌   | 148059/225000 [291:40:23<129:09:44,  6.04s/it]
 66%|██████▌   | 148060/225000 [291:40:27<114:54:55,  5.38s/it]
 66%|██████▌   | 148061/225000 [291:40:30<102:20:00,  4.79s/it]
 66%|██████▌   | 148062/225000 [291:40:41<137:34:13,  6.44s/it]
 66%|██████▌   | 148063/225000 [291:40:44<120:17:49,  5.63s/it]
 66%|██████▌   | 148064/225000 [291:40:47<104:24:44,  4.89s/it]
 66%|██████▌   | 148065/225000 [291:40:56<128:35:41,  6.02s/it]
 66%|██████▌   | 148066/225000 [291:41:00<115:01:01,  5.38s/it]
 66%|██████▌   | 148067/225000 [291:41:05<113:09:32,  5.30s/it]
 66%|██████▌   | 148068/225000 [291:41:14<133:19:24,  6.24s/it]
 66%|██████▌   | 148069/225000 [291:41:17<118:13:18,  5.53s/it]
 66%|██████▌   | 148070/225000 [291:41:21<108:30:37,  5.08s/it]
 66%|██████▌   | 148071/225000 [291:41:26<104:10:55,  4.88s/it]
 66%|██████▌   | 148072/225000 [291:41:31<108:04:23,  5.06s/it]
 66%|██████▌   | 148073/225000 [291:41:41<137:35:51,  6.44s/it]
 66%|██████▌   | 148074/225000 [291:41:44<118:26:21,  5.54s/it]
 66%|██████▌   | 148075/225000 [291:41:53<136:51:17,  6.40s/it]
 66%|██████▌   | 148076/225000 [291:41:57<124:41:44,  5.84s/it]
 66%|██████▌   | 148077/225000 [291:42:03<121:17:56,  5.68s/it]
 66%|██████▌   | 148078/225000 [291:42:08<116:50:58,  5.47s/it]
 66%|██████▌   | 148079/225000 [291:42:13<118:27:39,  5.54s/it]
 66%|██████▌   | 148080/225000 [291:42:18<115:44:12,  5.42s/it]
 66%|██████▌   | 148081/225000 [291:42:23<111:10:43,  5.20s/it]
 66%|██████▌   | 148082/225000 [291:42:28<109:40:00,  5.13s/it]
 66%|██████▌   | 148083/225000 [291:42:33<107:53:52,  5.05s/it]
 66%|██████▌   | 148084/225000 [291:42:38<110:19:03,  5.16s/it]
 66%|██████▌   | 148085/225000 [291:42:44<113:01:18,  5.29s/it]
 66%|██████▌   | 148086/225000 [291:42:49<110:18:28,  5.16s/it]
 66%|██████▌   | 148087/225000 [291:42:54<111:33:36,  5.22s/it]
 66%|██████▌   | 148088/225000 [291:42:59<110:07:35,  5.15s/it]
 66%|██████▌   | 148089/225000 [291:43:04<110:07:12,  5.15s/it]
 66%|██████▌   | 148090/225000 [291:43:10<111:15:08,  5.21s/it]
 66%|██████▌   | 148091/225000 [291:43:15<112:01:57,  5.24s/it]
 66%|██████▌   | 148092/225000 [291:43:21<118:21:42,  5.54s/it]
 66%|██████▌   | 148093/225000 [291:43:34<165:38:46,  7.75s/it]
 66%|██████▌   | 148094/225000 [291:43:39<144:33:16,  6.77s/it]
 66%|██████▌   | 148095/225000 [291:43:47<153:12:23,  7.17s/it]
 66%|██████▌   | 148096/225000 [291:43:51<132:29:54,  6.20s/it]
 66%|██████▌   | 148097/225000 [291:43:55<118:58:41,  5.57s/it]
 66%|██████▌   | 148098/225000 [291:43:59<109:21:52,  5.12s/it]
 66%|██████▌   | 148099/225000 [291:44:09<140:04:38,  6.56s/it]
 66%|██████▌   | 148100/225000 [291:44:13<122:20:23,  5.73s/it]
                                                               
{'loss': '0.7861', 'grad_norm': '1.045', 'learning_rate': '1.421e-05', 'epoch': '0.6582', 'num_input_tokens_seen': 483536360, 'train_runtime': '1.05e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148100/225000 [291:44:13<122:20:23,  5.73s/it]
 66%|██████▌   | 148101/225000 [291:44:16<110:37:06,  5.18s/it]
 66%|██████▌   | 148102/225000 [291:44:29<156:19:55,  7.32s/it]
 66%|██████▌   | 148103/225000 [291:44:33<134:47:57,  6.31s/it]
 66%|██████▌   | 148104/225000 [291:44:36<117:19:04,  5.49s/it]
 66%|██████▌   | 148105/225000 [291:44:41<113:49:19,  5.33s/it]
 66%|██████▌   | 148106/225000 [291:44:46<109:10:36,  5.11s/it]
 66%|██████▌   | 148107/225000 [291:44:51<106:27:23,  4.98s/it]
 66%|██████▌   | 148108/225000 [291:44:55<104:35:48,  4.90s/it]
 66%|██████▌   | 148109/225000 [291:45:07<146:32:01,  6.86s/it]
 66%|██████▌   | 148110/225000 [291:45:13<141:04:47,  6.61s/it]
 66%|██████▌   | 148111/225000 [291:45:17<124:24:58,  5.83s/it]
 66%|██████▌   | 148112/225000 [291:45:25<140:43:31,  6.59s/it]
 66%|██████▌   | 148113/225000 [291:45:31<134:36:18,  6.30s/it]
 66%|██████▌   | 148114/225000 [291:45:34<117:57:41,  5.52s/it]
 66%|██████▌   | 148115/225000 [291:45:43<139:41:22,  6.54s/it]
 66%|██████▌   | 148116/225000 [291:45:47<119:19:13,  5.59s/it]
 66%|██████▌   | 148117/225000 [291:45:50<106:07:27,  4.97s/it]
 66%|██████▌   | 148118/225000 [291:46:01<140:29:27,  6.58s/it]
 66%|██████▌   | 148119/225000 [291:46:05<127:23:26,  5.97s/it]
 66%|██████▌   | 148120/225000 [291:46:09<117:03:32,  5.48s/it]
 66%|██████▌   | 148121/225000 [291:46:14<109:57:19,  5.15s/it]
 66%|██████▌   | 148122/225000 [291:46:18<105:24:50,  4.94s/it]
 66%|██████▌   | 148123/225000 [291:46:22<100:25:30,  4.70s/it]
 66%|██████▌   | 148124/225000 [291:46:28<105:15:38,  4.93s/it]
 66%|██████▌   | 148125/225000 [291:46:38<138:39:52,  6.49s/it]
 66%|██████▌   | 148126/225000 [291:46:43<126:32:23,  5.93s/it]
 66%|██████▌   | 148127/225000 [291:46:47<119:37:13,  5.60s/it]
 66%|██████▌   | 148128/225000 [291:46:53<116:59:01,  5.48s/it]
 66%|██████▌   | 148129/225000 [291:46:58<113:28:37,  5.31s/it]
 66%|██████▌   | 148130/225000 [291:47:02<110:03:46,  5.15s/it]
 66%|██████▌   | 148131/225000 [291:47:08<111:34:14,  5.23s/it]
 66%|██████▌   | 148132/225000 [291:47:14<115:20:43,  5.40s/it]
 66%|██████▌   | 148133/225000 [291:47:25<155:55:21,  7.30s/it]
 66%|██████▌   | 148134/225000 [291:47:29<132:39:03,  6.21s/it]
 66%|██████▌   | 148135/225000 [291:47:32<115:00:23,  5.39s/it]
 66%|██████▌   | 148136/225000 [291:47:37<106:48:39,  5.00s/it]
 66%|██████▌   | 148137/225000 [291:47:41<104:31:49,  4.90s/it]
 66%|██████▌   | 148138/225000 [291:47:54<152:33:49,  7.15s/it]
 66%|██████▌   | 148139/225000 [291:47:59<141:18:13,  6.62s/it]
 66%|██████▌   | 148140/225000 [291:48:03<125:10:17,  5.86s/it]
 66%|██████▌   | 148141/225000 [291:48:07<113:13:59,  5.30s/it]
 66%|██████▌   | 148142/225000 [291:48:15<130:51:53,  6.13s/it]
 66%|██████▌   | 148143/225000 [291:48:22<132:14:28,  6.19s/it]
 66%|██████▌   | 148144/225000 [291:48:29<141:15:50,  6.62s/it]
 66%|██████▌   | 148145/225000 [291:48:33<125:26:41,  5.88s/it]
 66%|██████▌   | 148146/225000 [291:48:37<114:25:47,  5.36s/it]
 66%|██████▌   | 148147/225000 [291:48:41<105:59:12,  4.96s/it]
 66%|██████▌   | 148148/225000 [291:48:45<97:40:27,  4.58s/it] 
 66%|██████▌   | 148149/225000 [291:48:55<131:37:03,  6.17s/it]
 66%|██████▌   | 148150/225000 [291:49:00<126:10:33,  5.91s/it]
                                                               
{'loss': '0.7827', 'grad_norm': '1.216', 'learning_rate': '1.42e-05', 'epoch': '0.6584', 'num_input_tokens_seen': 483667632, 'train_runtime': '1.051e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148150/225000 [291:49:00<126:10:33,  5.91s/it]
 66%|██████▌   | 148151/225000 [291:49:05<117:10:42,  5.49s/it]
 66%|██████▌   | 148152/225000 [291:49:09<109:44:31,  5.14s/it]
 66%|██████▌   | 148153/225000 [291:49:13<101:16:08,  4.74s/it]
 66%|██████▌   | 148154/225000 [291:49:21<123:47:46,  5.80s/it]
 66%|██████▌   | 148155/225000 [291:49:28<129:33:10,  6.07s/it]
 66%|██████▌   | 148156/225000 [291:49:32<119:40:45,  5.61s/it]
 66%|██████▌   | 148157/225000 [291:49:37<113:04:32,  5.30s/it]
 66%|██████▌   | 148158/225000 [291:49:41<107:08:54,  5.02s/it]
 66%|██████▌   | 148159/225000 [291:49:46<105:12:47,  4.93s/it]
 66%|██████▌   | 148160/225000 [291:49:51<102:08:27,  4.79s/it]
 66%|██████▌   | 148161/225000 [291:49:56<105:24:36,  4.94s/it]
 66%|██████▌   | 148162/225000 [291:50:07<145:56:20,  6.84s/it]
 66%|██████▌   | 148163/225000 [291:50:12<132:16:05,  6.20s/it]
 66%|██████▌   | 148164/225000 [291:50:15<115:29:58,  5.41s/it]
 66%|██████▌   | 148165/225000 [291:50:24<136:19:41,  6.39s/it]
 66%|██████▌   | 148166/225000 [291:50:30<134:56:42,  6.32s/it]
 66%|██████▌   | 148167/225000 [291:50:34<116:42:53,  5.47s/it]
 66%|██████▌   | 148168/225000 [291:50:38<105:57:57,  4.97s/it]
 66%|██████▌   | 148169/225000 [291:50:48<144:05:33,  6.75s/it]
 66%|██████▌   | 148170/225000 [291:50:52<124:56:11,  5.85s/it]
 66%|██████▌   | 148171/225000 [291:50:56<109:05:06,  5.11s/it]
 66%|██████▌   | 148172/225000 [291:51:06<140:13:39,  6.57s/it]
 66%|██████▌   | 148173/225000 [291:51:09<118:48:57,  5.57s/it]
 66%|██████▌   | 148174/225000 [291:51:15<124:42:24,  5.84s/it]
 66%|██████▌   | 148175/225000 [291:51:19<112:41:47,  5.28s/it]
 66%|██████▌   | 148176/225000 [291:51:25<118:42:05,  5.56s/it]
 66%|██████▌   | 148177/225000 [291:51:30<110:08:11,  5.16s/it]
 66%|██████▌   | 148178/225000 [291:51:38<127:46:18,  5.99s/it]
 66%|██████▌   | 148179/225000 [291:51:42<115:19:29,  5.40s/it]
 66%|██████▌   | 148180/225000 [291:51:46<107:25:25,  5.03s/it]
 66%|██████▌   | 148181/225000 [291:51:50<105:15:01,  4.93s/it]
 66%|██████▌   | 148182/225000 [291:51:59<131:05:43,  6.14s/it]
 66%|██████▌   | 148183/225000 [291:52:07<137:45:58,  6.46s/it]
 66%|██████▌   | 148184/225000 [291:52:10<118:12:05,  5.54s/it]
 66%|██████▌   | 148185/225000 [291:52:14<108:56:28,  5.11s/it]
 66%|██████▌   | 148186/225000 [291:52:19<107:55:09,  5.06s/it]
 66%|██████▌   | 148187/225000 [291:52:25<111:00:37,  5.20s/it]
 66%|██████▌   | 148188/225000 [291:52:29<105:41:29,  4.95s/it]
 66%|██████▌   | 148189/225000 [291:52:34<105:24:00,  4.94s/it]
 66%|██████▌   | 148190/225000 [291:52:39<103:59:25,  4.87s/it]
 66%|██████▌   | 148191/225000 [291:52:44<104:22:44,  4.89s/it]
 66%|██████▌   | 148192/225000 [291:52:51<121:11:38,  5.68s/it]
 66%|██████▌   | 148193/225000 [291:53:03<160:47:43,  7.54s/it]
 66%|██████▌   | 148194/225000 [291:53:06<134:32:28,  6.31s/it]
 66%|██████▌   | 148195/225000 [291:53:11<121:28:53,  5.69s/it]
 66%|██████▌   | 148196/225000 [291:53:15<113:46:18,  5.33s/it]
 66%|██████▌   | 148197/225000 [291:53:20<108:57:38,  5.11s/it]
 66%|██████▌   | 148198/225000 [291:53:32<156:32:42,  7.34s/it]
 66%|██████▌   | 148199/225000 [291:53:36<130:58:50,  6.14s/it]
 66%|██████▌   | 148200/225000 [291:53:39<114:54:25,  5.39s/it]
                                                               
{'loss': '0.7916', 'grad_norm': '1.101', 'learning_rate': '1.418e-05', 'epoch': '0.6587', 'num_input_tokens_seen': 483797272, 'train_runtime': '1.051e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148200/225000 [291:53:39<114:54:25,  5.39s/it]
 66%|██████▌   | 148201/225000 [291:53:43<107:11:42,  5.02s/it]
 66%|██████▌   | 148202/225000 [291:53:56<154:12:13,  7.23s/it]
 66%|██████▌   | 148203/225000 [291:53:59<128:01:35,  6.00s/it]
 66%|██████▌   | 148204/225000 [291:54:06<133:39:08,  6.27s/it]
 66%|██████▌   | 148205/225000 [291:54:10<118:20:42,  5.55s/it]
 66%|██████▌   | 148206/225000 [291:54:13<105:58:27,  4.97s/it]
 66%|██████▌   | 148207/225000 [291:54:23<138:18:20,  6.48s/it]
 66%|██████▌   | 148208/225000 [291:54:27<121:31:16,  5.70s/it]
 66%|██████▌   | 148209/225000 [291:54:33<125:13:39,  5.87s/it]
 66%|██████▌   | 148210/225000 [291:54:37<108:54:56,  5.11s/it]
 66%|██████▌   | 148211/225000 [291:54:42<107:18:59,  5.03s/it]
 66%|██████▌   | 148212/225000 [291:54:47<112:20:57,  5.27s/it]
 66%|██████▌   | 148213/225000 [291:54:53<113:10:31,  5.31s/it]
 66%|██████▌   | 148214/225000 [291:54:58<113:03:50,  5.30s/it]
 66%|██████▌   | 148215/225000 [291:55:04<113:35:54,  5.33s/it]
 66%|██████▌   | 148216/225000 [291:55:09<113:50:44,  5.34s/it]
 66%|██████▌   | 148217/225000 [291:55:17<134:07:16,  6.29s/it]
 66%|██████▌   | 148218/225000 [291:55:23<132:43:10,  6.22s/it]
 66%|██████▌   | 148219/225000 [291:55:27<117:38:34,  5.52s/it]
 66%|██████▌   | 148220/225000 [291:55:37<142:52:17,  6.70s/it]
 66%|██████▌   | 148221/225000 [291:55:41<126:32:22,  5.93s/it]
 66%|██████▌   | 148222/225000 [291:55:45<113:07:03,  5.30s/it]
 66%|██████▌   | 148223/225000 [291:55:48<102:38:46,  4.81s/it]
 66%|██████▌   | 148224/225000 [291:55:58<133:30:36,  6.26s/it]
 66%|██████▌   | 148225/225000 [291:56:04<128:56:24,  6.05s/it]
 66%|██████▌   | 148226/225000 [291:56:07<113:33:32,  5.32s/it]
 66%|██████▌   | 148227/225000 [291:56:11<104:33:10,  4.90s/it]
 66%|██████▌   | 148228/225000 [291:56:16<103:49:21,  4.87s/it]
 66%|██████▌   | 148229/225000 [291:56:26<138:24:21,  6.49s/it]
 66%|██████▌   | 148230/225000 [291:56:34<148:44:49,  6.98s/it]
 66%|██████▌   | 148231/225000 [291:56:38<128:13:45,  6.01s/it]
 66%|██████▌   | 148232/225000 [291:56:42<113:53:07,  5.34s/it]
 66%|██████▌   | 148233/225000 [291:56:50<129:52:11,  6.09s/it]
 66%|██████▌   | 148234/225000 [291:56:53<114:09:20,  5.35s/it]
 66%|██████▌   | 148235/225000 [291:57:03<141:03:59,  6.62s/it]
 66%|██████▌   | 148236/225000 [291:57:06<119:52:16,  5.62s/it]
 66%|██████▌   | 148237/225000 [291:57:13<126:09:40,  5.92s/it]
 66%|██████▌   | 148238/225000 [291:57:17<113:29:49,  5.32s/it]
 66%|██████▌   | 148239/225000 [291:57:21<106:22:23,  4.99s/it]
 66%|██████▌   | 148240/225000 [291:57:28<117:54:24,  5.53s/it]
 66%|██████▌   | 148241/225000 [291:57:35<125:59:03,  5.91s/it]
 66%|██████▌   | 148242/225000 [291:57:43<141:42:18,  6.65s/it]
 66%|██████▌   | 148243/225000 [291:57:47<127:29:14,  5.98s/it]
 66%|██████▌   | 148244/225000 [291:57:52<119:38:28,  5.61s/it]
 66%|██████▌   | 148245/225000 [291:57:57<114:52:22,  5.39s/it]
 66%|██████▌   | 148246/225000 [291:58:02<110:06:11,  5.16s/it]
 66%|██████▌   | 148247/225000 [291:58:06<106:39:58,  5.00s/it]
 66%|██████▌   | 148248/225000 [291:58:11<104:59:25,  4.92s/it]
 66%|██████▌   | 148249/225000 [291:58:17<115:07:01,  5.40s/it]
 66%|██████▌   | 148250/225000 [291:58:27<143:22:49,  6.73s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '1.065', 'learning_rate': '1.416e-05', 'epoch': '0.6589', 'num_input_tokens_seen': 483926384, 'train_runtime': '1.051e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148250/225000 [291:58:27<143:22:49,  6.73s/it]
 66%|██████▌   | 148251/225000 [291:58:31<126:09:37,  5.92s/it]
 66%|██████▌   | 148252/225000 [291:58:38<133:44:06,  6.27s/it]
 66%|██████▌   | 148253/225000 [291:58:43<125:44:47,  5.90s/it]
 66%|██████▌   | 148254/225000 [291:58:48<118:00:38,  5.54s/it]
 66%|██████▌   | 148255/225000 [291:58:53<113:35:41,  5.33s/it]
 66%|██████▌   | 148256/225000 [291:58:58<109:06:54,  5.12s/it]
 66%|██████▌   | 148257/225000 [291:59:02<106:39:37,  5.00s/it]
 66%|██████▌   | 148258/225000 [291:59:07<103:36:22,  4.86s/it]
 66%|██████▌   | 148259/225000 [291:59:20<156:41:42,  7.35s/it]
 66%|██████▌   | 148260/225000 [291:59:24<135:14:25,  6.34s/it]
 66%|██████▌   | 148261/225000 [291:59:28<118:54:52,  5.58s/it]
 66%|██████▌   | 148262/225000 [291:59:32<110:42:30,  5.19s/it]
 66%|██████▌   | 148263/225000 [291:59:37<107:21:26,  5.04s/it]
 66%|██████▌   | 148264/225000 [291:59:42<106:42:45,  5.01s/it]
 66%|██████▌   | 148265/225000 [291:59:53<144:20:17,  6.77s/it]
 66%|██████▌   | 148266/225000 [291:59:57<127:56:36,  6.00s/it]
 66%|██████▌   | 148267/225000 [292:00:05<139:58:19,  6.57s/it]
 66%|██████▌   | 148268/225000 [292:00:08<119:32:30,  5.61s/it]
 66%|██████▌   | 148269/225000 [292:00:13<117:04:09,  5.49s/it]
 66%|██████▌   | 148270/225000 [292:00:19<119:38:56,  5.61s/it]
 66%|██████▌   | 148271/225000 [292:00:28<141:12:02,  6.62s/it]
 66%|██████▌   | 148272/225000 [292:00:32<119:58:37,  5.63s/it]
 66%|██████▌   | 148273/225000 [292:00:36<111:26:13,  5.23s/it]
 66%|██████▌   | 148274/225000 [292:00:45<135:05:01,  6.34s/it]
 66%|██████▌   | 148275/225000 [292:00:50<125:05:48,  5.87s/it]
 66%|██████▌   | 148276/225000 [292:00:56<128:43:46,  6.04s/it]
 66%|██████▌   | 148277/225000 [292:01:02<125:39:56,  5.90s/it]
 66%|██████▌   | 148278/225000 [292:01:07<121:03:41,  5.68s/it]
 66%|██████▌   | 148279/225000 [292:01:12<118:04:28,  5.54s/it]
 66%|██████▌   | 148280/225000 [292:01:17<114:33:31,  5.38s/it]
 66%|██████▌   | 148281/225000 [292:01:22<114:57:22,  5.39s/it]
 66%|██████▌   | 148282/225000 [292:01:28<115:11:46,  5.41s/it]
 66%|██████▌   | 148283/225000 [292:01:33<114:43:16,  5.38s/it]
 66%|██████▌   | 148284/225000 [292:01:38<114:10:00,  5.36s/it]
 66%|██████▌   | 148285/225000 [292:01:44<115:15:37,  5.41s/it]
 66%|██████▌   | 148286/225000 [292:01:50<117:53:55,  5.53s/it]
 66%|██████▌   | 148287/225000 [292:01:58<135:41:26,  6.37s/it]
 66%|██████▌   | 148288/225000 [292:02:05<137:22:33,  6.45s/it]
 66%|██████▌   | 148289/225000 [292:02:08<118:36:41,  5.57s/it]
 66%|██████▌   | 148290/225000 [292:02:13<116:38:36,  5.47s/it]
 66%|██████▌   | 148291/225000 [292:02:18<111:53:15,  5.25s/it]
 66%|██████▌   | 148292/225000 [292:02:23<110:12:56,  5.17s/it]
 66%|██████▌   | 148293/225000 [292:02:28<109:47:34,  5.15s/it]
 66%|██████▌   | 148294/225000 [292:02:34<110:27:19,  5.18s/it]
 66%|██████▌   | 148295/225000 [292:02:39<112:13:05,  5.27s/it]
 66%|██████▌   | 148296/225000 [292:02:44<113:23:04,  5.32s/it]
 66%|██████▌   | 148297/225000 [292:02:50<113:57:03,  5.35s/it]
 66%|██████▌   | 148298/225000 [292:02:55<113:43:58,  5.34s/it]
 66%|██████▌   | 148299/225000 [292:03:00<112:39:08,  5.29s/it]
 66%|██████▌   | 148300/225000 [292:03:06<113:34:43,  5.33s/it]
                                                               
{'loss': '0.7875', 'grad_norm': '1.085', 'learning_rate': '1.415e-05', 'epoch': '0.6591', 'num_input_tokens_seen': 484056848, 'train_runtime': '1.051e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148300/225000 [292:03:06<113:34:43,  5.33s/it]
 66%|██████▌   | 148301/225000 [292:03:16<146:34:39,  6.88s/it]
 66%|██████▌   | 148302/225000 [292:03:23<144:14:07,  6.77s/it]
 66%|██████▌   | 148303/225000 [292:03:26<122:58:47,  5.77s/it]
 66%|██████▌   | 148304/225000 [292:03:33<127:45:28,  6.00s/it]
 66%|██████▌   | 148305/225000 [292:03:41<143:45:01,  6.75s/it]
 66%|██████▌   | 148306/225000 [292:03:45<121:36:10,  5.71s/it]
 66%|██████▌   | 148307/225000 [292:03:48<110:15:27,  5.18s/it]
 66%|██████▌   | 148308/225000 [292:03:57<132:05:44,  6.20s/it]
 66%|██████▌   | 148309/225000 [292:04:05<141:54:45,  6.66s/it]
 66%|██████▌   | 148310/225000 [292:04:08<120:18:25,  5.65s/it]
 66%|██████▌   | 148311/225000 [292:04:16<131:53:04,  6.19s/it]
 66%|██████▌   | 148312/225000 [292:04:19<115:00:30,  5.40s/it]
 66%|██████▌   | 148313/225000 [292:04:27<129:51:57,  6.10s/it]
 66%|██████▌   | 148314/225000 [292:04:30<112:47:44,  5.30s/it]
 66%|██████▌   | 148315/225000 [292:04:34<104:36:04,  4.91s/it]
 66%|██████▌   | 148316/225000 [292:04:45<139:25:09,  6.55s/it]
 66%|██████▌   | 148317/225000 [292:04:48<121:09:55,  5.69s/it]
 66%|██████▌   | 148318/225000 [292:04:52<108:03:24,  5.07s/it]
 66%|██████▌   | 148319/225000 [292:05:01<135:10:36,  6.35s/it]
 66%|██████▌   | 148320/225000 [292:05:04<114:37:39,  5.38s/it]
 66%|██████▌   | 148321/225000 [292:05:13<133:58:35,  6.29s/it]
 66%|██████▌   | 148322/225000 [292:05:16<117:01:00,  5.49s/it]
 66%|██████▌   | 148323/225000 [292:05:20<104:53:01,  4.92s/it]
 66%|██████▌   | 148324/225000 [292:05:30<138:15:00,  6.49s/it]
 66%|██████▌   | 148325/225000 [292:05:33<116:22:46,  5.46s/it]
 66%|██████▌   | 148326/225000 [292:05:42<134:20:11,  6.31s/it]
 66%|██████▌   | 148327/225000 [292:05:46<123:46:35,  5.81s/it]
 66%|██████▌   | 148328/225000 [292:05:51<117:11:41,  5.50s/it]
 66%|██████▌   | 148329/225000 [292:05:56<112:44:15,  5.29s/it]
 66%|██████▌   | 148330/225000 [292:06:00<107:15:57,  5.04s/it]
 66%|██████▌   | 148331/225000 [292:06:05<106:12:12,  4.99s/it]
 66%|██████▌   | 148332/225000 [292:06:10<108:42:52,  5.10s/it]
 66%|██████▌   | 148333/225000 [292:06:15<107:34:14,  5.05s/it]
 66%|██████▌   | 148334/225000 [292:06:24<131:00:15,  6.15s/it]
 66%|██████▌   | 148335/225000 [292:06:34<153:33:03,  7.21s/it]
 66%|██████▌   | 148336/225000 [292:06:38<136:16:56,  6.40s/it]
 66%|██████▌   | 148337/225000 [292:06:42<121:50:50,  5.72s/it]
 66%|██████▌   | 148338/225000 [292:06:47<113:23:57,  5.33s/it]
 66%|██████▌   | 148339/225000 [292:06:52<113:17:32,  5.32s/it]
 66%|██████▌   | 148340/225000 [292:07:03<146:31:14,  6.88s/it]
 66%|██████▌   | 148341/225000 [292:07:06<125:24:43,  5.89s/it]
 66%|██████▌   | 148342/225000 [292:07:13<133:07:21,  6.25s/it]
 66%|██████▌   | 148343/225000 [292:07:19<128:25:51,  6.03s/it]
 66%|██████▌   | 148344/225000 [292:07:24<121:15:24,  5.69s/it]
 66%|██████▌   | 148345/225000 [292:07:29<116:34:11,  5.47s/it]
 66%|██████▌   | 148346/225000 [292:07:33<110:30:02,  5.19s/it]
 66%|██████▌   | 148347/225000 [292:07:38<109:36:18,  5.15s/it]
 66%|██████▌   | 148348/225000 [292:07:43<106:15:23,  4.99s/it]
 66%|██████▌   | 148349/225000 [292:07:48<104:00:52,  4.89s/it]
 66%|██████▌   | 148350/225000 [292:07:52<103:42:42,  4.87s/it]
                                                               
{'loss': '0.785', 'grad_norm': '1.087', 'learning_rate': '1.413e-05', 'epoch': '0.6593', 'num_input_tokens_seen': 484187320, 'train_runtime': '1.052e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148350/225000 [292:07:52<103:42:42,  4.87s/it]
 66%|██████▌   | 148351/225000 [292:07:57<103:58:47,  4.88s/it]
 66%|██████▌   | 148352/225000 [292:08:02<103:46:28,  4.87s/it]
 66%|██████▌   | 148353/225000 [292:08:07<106:19:22,  4.99s/it]
 66%|██████▌   | 148354/225000 [292:08:14<118:37:20,  5.57s/it]
 66%|██████▌   | 148355/225000 [292:08:27<161:39:05,  7.59s/it]
 66%|██████▌   | 148356/225000 [292:08:30<135:58:53,  6.39s/it]
 66%|██████▌   | 148357/225000 [292:08:35<123:26:02,  5.80s/it]
 66%|██████▌   | 148358/225000 [292:08:39<116:15:31,  5.46s/it]
 66%|██████▌   | 148359/225000 [292:08:44<112:18:10,  5.28s/it]
 66%|██████▌   | 148360/225000 [292:08:49<108:02:15,  5.07s/it]
 66%|██████▌   | 148361/225000 [292:08:54<109:16:47,  5.13s/it]
 66%|██████▌   | 148362/225000 [292:08:59<109:26:18,  5.14s/it]
 66%|██████▌   | 148363/225000 [292:09:12<155:28:54,  7.30s/it]
 66%|██████▌   | 148364/225000 [292:09:17<145:46:08,  6.85s/it]
 66%|██████▌   | 148365/225000 [292:09:22<133:50:33,  6.29s/it]
 66%|██████▌   | 148366/225000 [292:09:29<139:08:04,  6.54s/it]
 66%|██████▌   | 148367/225000 [292:09:33<123:00:59,  5.78s/it]
 66%|██████▌   | 148368/225000 [292:09:38<113:59:03,  5.35s/it]
 66%|██████▌   | 148369/225000 [292:09:43<110:15:57,  5.18s/it]
 66%|██████▌   | 148370/225000 [292:09:47<104:49:31,  4.92s/it]
 66%|██████▌   | 148371/225000 [292:09:59<148:59:30,  7.00s/it]
 66%|██████▌   | 148372/225000 [292:10:03<129:11:24,  6.07s/it]
 66%|██████▌   | 148373/225000 [292:10:07<116:55:51,  5.49s/it]
 66%|██████▌   | 148374/225000 [292:10:11<109:37:10,  5.15s/it]
 66%|██████▌   | 148375/225000 [292:10:18<121:41:36,  5.72s/it]
 66%|██████▌   | 148376/225000 [292:10:30<157:30:48,  7.40s/it]
 66%|██████▌   | 148377/225000 [292:10:33<131:21:46,  6.17s/it]
 66%|██████▌   | 148378/225000 [292:10:38<125:47:38,  5.91s/it]
 66%|██████▌   | 148379/225000 [292:10:45<134:02:29,  6.30s/it]
 66%|██████▌   | 148380/225000 [292:10:50<122:58:59,  5.78s/it]
 66%|██████▌   | 148381/225000 [292:10:55<115:54:50,  5.45s/it]
 66%|██████▌   | 148382/225000 [292:10:59<109:16:20,  5.13s/it]
 66%|██████▌   | 148383/225000 [292:11:04<106:38:16,  5.01s/it]
 66%|██████▌   | 148384/225000 [292:11:15<145:09:13,  6.82s/it]
 66%|██████▌   | 148385/225000 [292:11:20<134:03:54,  6.30s/it]
 66%|██████▌   | 148386/225000 [292:11:26<133:27:17,  6.27s/it]
 66%|██████▌   | 148387/225000 [292:11:30<116:54:33,  5.49s/it]
 66%|██████▌   | 148388/225000 [292:11:36<124:42:49,  5.86s/it]
 66%|██████▌   | 148389/225000 [292:11:43<128:15:10,  6.03s/it]
 66%|██████▌   | 148390/225000 [292:11:50<137:14:53,  6.45s/it]
 66%|██████▌   | 148391/225000 [292:11:54<117:02:17,  5.50s/it]
 66%|██████▌   | 148392/225000 [292:12:02<138:20:11,  6.50s/it]
 66%|██████▌   | 148393/225000 [292:12:06<119:43:43,  5.63s/it]
 66%|██████▌   | 148394/225000 [292:12:10<106:57:47,  5.03s/it]
 66%|██████▌   | 148395/225000 [292:12:18<129:59:43,  6.11s/it]
 66%|██████▌   | 148396/225000 [292:12:23<118:38:19,  5.58s/it]
 66%|██████▌   | 148397/225000 [292:12:27<109:36:49,  5.15s/it]
 66%|██████▌   | 148398/225000 [292:12:31<102:09:21,  4.80s/it]
 66%|██████▌   | 148399/225000 [292:12:36<105:17:29,  4.95s/it]
 66%|██████▌   | 148400/225000 [292:12:48<147:42:54,  6.94s/it]
                                                               
{'loss': '0.7881', 'grad_norm': '1.129', 'learning_rate': '1.411e-05', 'epoch': '0.6596', 'num_input_tokens_seen': 484317696, 'train_runtime': '1.052e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148400/225000 [292:12:48<147:42:54,  6.94s/it]
 66%|██████▌   | 148401/225000 [292:12:51<124:58:55,  5.87s/it]
 66%|██████▌   | 148402/225000 [292:12:58<132:39:16,  6.23s/it]
 66%|██████▌   | 148403/225000 [292:13:02<119:36:29,  5.62s/it]
 66%|██████▌   | 148404/225000 [292:13:07<111:24:50,  5.24s/it]
 66%|██████▌   | 148405/225000 [292:13:11<104:23:26,  4.91s/it]
 66%|██████▌   | 148406/225000 [292:13:19<128:11:17,  6.02s/it]
 66%|██████▌   | 148407/225000 [292:13:23<114:46:33,  5.39s/it]
 66%|██████▌   | 148408/225000 [292:13:29<113:49:09,  5.35s/it]
 66%|██████▌   | 148409/225000 [292:13:37<132:25:27,  6.22s/it]
 66%|██████▌   | 148410/225000 [292:13:41<121:08:36,  5.69s/it]
 66%|██████▌   | 148411/225000 [292:13:47<123:16:11,  5.79s/it]
 66%|██████▌   | 148412/225000 [292:13:51<107:52:30,  5.07s/it]
 66%|██████▌   | 148413/225000 [292:14:00<133:16:13,  6.26s/it]
 66%|██████▌   | 148414/225000 [292:14:03<115:12:30,  5.42s/it]
 66%|██████▌   | 148415/225000 [292:14:08<109:20:36,  5.14s/it]
 66%|██████▌   | 148416/225000 [292:14:13<110:46:45,  5.21s/it]
 66%|██████▌   | 148417/225000 [292:14:18<110:12:24,  5.18s/it]
 66%|██████▌   | 148418/225000 [292:14:23<110:31:49,  5.20s/it]
 66%|██████▌   | 148419/225000 [292:14:29<111:34:19,  5.24s/it]
 66%|██████▌   | 148420/225000 [292:14:34<111:47:28,  5.26s/it]
 66%|██████▌   | 148421/225000 [292:14:39<109:42:08,  5.16s/it]
 66%|██████▌   | 148422/225000 [292:14:44<109:50:54,  5.16s/it]
 66%|██████▌   | 148423/225000 [292:14:49<110:25:34,  5.19s/it]
 66%|██████▌   | 148424/225000 [292:14:58<134:24:25,  6.32s/it]
 66%|██████▌   | 148425/225000 [292:15:06<142:11:08,  6.68s/it]
 66%|██████▌   | 148426/225000 [292:15:13<144:05:19,  6.77s/it]
 66%|██████▌   | 148427/225000 [292:15:16<122:44:16,  5.77s/it]
 66%|██████▌   | 148428/225000 [292:15:25<140:18:38,  6.60s/it]
 66%|██████▌   | 148429/225000 [292:15:29<128:00:02,  6.02s/it]
 66%|██████▌   | 148430/225000 [292:15:33<112:30:42,  5.29s/it]
 66%|██████▌   | 148431/225000 [292:15:41<132:01:03,  6.21s/it]
 66%|██████▌   | 148432/225000 [292:15:49<143:22:24,  6.74s/it]
 66%|██████▌   | 148433/225000 [292:15:54<128:55:38,  6.06s/it]
 66%|██████▌   | 148434/225000 [292:15:58<115:11:22,  5.42s/it]
 66%|██████▌   | 148435/225000 [292:16:03<116:15:08,  5.47s/it]
 66%|██████▌   | 148436/225000 [292:16:13<141:04:07,  6.63s/it]
 66%|██████▌   | 148437/225000 [292:16:17<125:01:46,  5.88s/it]
 66%|██████▌   | 148438/225000 [292:16:21<115:33:22,  5.43s/it]
 66%|██████▌   | 148439/225000 [292:16:25<105:09:11,  4.94s/it]
 66%|██████▌   | 148440/225000 [292:16:35<139:49:05,  6.57s/it]
 66%|██████▌   | 148441/225000 [292:16:40<124:20:39,  5.85s/it]
 66%|██████▌   | 148442/225000 [292:16:48<139:11:19,  6.55s/it]
 66%|██████▌   | 148443/225000 [292:16:52<122:13:17,  5.75s/it]
 66%|██████▌   | 148444/225000 [292:16:56<110:49:56,  5.21s/it]
 66%|██████▌   | 148445/225000 [292:17:00<104:55:16,  4.93s/it]
 66%|██████▌   | 148446/225000 [292:17:11<142:08:48,  6.68s/it]
 66%|██████▌   | 148447/225000 [292:17:14<122:44:15,  5.77s/it]
 66%|██████▌   | 148448/225000 [292:17:18<107:41:53,  5.06s/it]
 66%|██████▌   | 148449/225000 [292:17:26<126:59:46,  5.97s/it]
 66%|██████▌   | 148450/225000 [292:17:29<108:51:15,  5.12s/it]
                                                               
{'loss': '0.7898', 'grad_norm': '1.151', 'learning_rate': '1.41e-05', 'epoch': '0.6598', 'num_input_tokens_seen': 484446696, 'train_runtime': '1.052e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148450/225000 [292:17:29<108:51:15,  5.12s/it]
 66%|██████▌   | 148451/225000 [292:17:38<133:37:44,  6.28s/it]
 66%|██████▌   | 148452/225000 [292:17:41<116:23:29,  5.47s/it]
 66%|██████▌   | 148453/225000 [292:17:51<139:03:05,  6.54s/it]
 66%|██████▌   | 148454/225000 [292:17:54<119:44:25,  5.63s/it]
 66%|██████▌   | 148455/225000 [292:18:01<130:25:00,  6.13s/it]
 66%|██████▌   | 148456/225000 [292:18:05<112:43:07,  5.30s/it]
 66%|██████▌   | 148457/225000 [292:18:10<112:50:32,  5.31s/it]
 66%|██████▌   | 148458/225000 [292:18:16<119:36:57,  5.63s/it]
 66%|██████▌   | 148459/225000 [292:18:21<110:43:15,  5.21s/it]
 66%|██████▌   | 148460/225000 [292:18:25<102:41:11,  4.83s/it]
 66%|██████▌   | 148461/225000 [292:18:29<97:24:00,  4.58s/it] 
 66%|██████▌   | 148462/225000 [292:18:39<137:13:45,  6.45s/it]
 66%|██████▌   | 148463/225000 [292:18:44<126:06:17,  5.93s/it]
 66%|██████▌   | 148464/225000 [292:18:49<118:11:29,  5.56s/it]
 66%|██████▌   | 148465/225000 [292:18:54<114:43:12,  5.40s/it]
 66%|██████▌   | 148466/225000 [292:18:58<109:44:04,  5.16s/it]
 66%|██████▌   | 148467/225000 [292:19:03<107:59:08,  5.08s/it]
 66%|██████▌   | 148468/225000 [292:19:08<108:03:33,  5.08s/it]
 66%|██████▌   | 148469/225000 [292:19:13<104:14:06,  4.90s/it]
 66%|██████▌   | 148470/225000 [292:19:18<105:35:48,  4.97s/it]
 66%|██████▌   | 148471/225000 [292:19:25<116:16:52,  5.47s/it]
 66%|██████▌   | 148472/225000 [292:19:37<158:38:51,  7.46s/it]
 66%|██████▌   | 148473/225000 [292:19:42<141:39:24,  6.66s/it]
 66%|██████▌   | 148474/225000 [292:19:46<127:44:30,  6.01s/it]
 66%|██████▌   | 148475/225000 [292:19:51<119:22:54,  5.62s/it]
 66%|██████▌   | 148476/225000 [292:19:56<115:36:33,  5.44s/it]
 66%|██████▌   | 148477/225000 [292:20:01<114:19:25,  5.38s/it]
 66%|██████▌   | 148478/225000 [292:20:06<110:32:48,  5.20s/it]
 66%|██████▌   | 148479/225000 [292:20:11<110:05:27,  5.18s/it]
 66%|██████▌   | 148480/225000 [292:20:16<109:16:08,  5.14s/it]
 66%|██████▌   | 148481/225000 [292:20:21<108:37:28,  5.11s/it]
 66%|██████▌   | 148482/225000 [292:20:28<122:30:06,  5.76s/it]
 66%|██████▌   | 148483/225000 [292:20:41<168:52:45,  7.95s/it]
 66%|██████▌   | 148484/225000 [292:20:45<141:58:32,  6.68s/it]
 66%|██████▌   | 148485/225000 [292:20:50<129:08:31,  6.08s/it]
 66%|██████▌   | 148486/225000 [292:20:54<120:16:27,  5.66s/it]
 66%|██████▌   | 148487/225000 [292:20:59<116:33:39,  5.48s/it]
 66%|██████▌   | 148488/225000 [292:21:05<114:40:37,  5.40s/it]
 66%|██████▌   | 148489/225000 [292:21:10<112:14:52,  5.28s/it]
 66%|██████▌   | 148490/225000 [292:21:15<111:28:48,  5.25s/it]
 66%|██████▌   | 148491/225000 [292:21:20<108:50:43,  5.12s/it]
 66%|██████▌   | 148492/225000 [292:21:25<109:34:22,  5.16s/it]
 66%|██████▌   | 148493/225000 [292:21:30<108:32:48,  5.11s/it]
 66%|██████▌   | 148494/225000 [292:21:35<108:21:50,  5.10s/it]
 66%|██████▌   | 148495/225000 [292:21:40<106:28:58,  5.01s/it]
 66%|██████▌   | 148496/225000 [292:21:47<122:42:32,  5.77s/it]
 66%|██████▌   | 148497/225000 [292:21:59<158:12:10,  7.44s/it]
 66%|██████▌   | 148498/225000 [292:22:03<137:33:49,  6.47s/it]
 66%|██████▌   | 148499/225000 [292:22:07<124:43:15,  5.87s/it]
 66%|██████▌   | 148500/225000 [292:22:12<116:57:29,  5.50s/it]
                                                               
{'loss': '0.7895', 'grad_norm': '1.171', 'learning_rate': '1.408e-05', 'epoch': '0.66', 'num_input_tokens_seen': 484576992, 'train_runtime': '1.053e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148500/225000 [292:22:12<116:57:29,  5.50s/it]
 66%|██████▌   | 148501/225000 [292:22:16<109:35:44,  5.16s/it]
 66%|██████▌   | 148502/225000 [292:22:21<108:59:48,  5.13s/it]
 66%|██████▌   | 148503/225000 [292:22:33<150:03:23,  7.06s/it]
 66%|██████▌   | 148504/225000 [292:22:37<128:05:31,  6.03s/it]
 66%|██████▌   | 148505/225000 [292:22:42<124:19:21,  5.85s/it]
 66%|██████▌   | 148506/225000 [292:22:50<136:31:17,  6.43s/it]
 66%|██████▌   | 148507/225000 [292:22:53<116:14:56,  5.47s/it]
 66%|██████▌   | 148508/225000 [292:23:01<134:17:12,  6.32s/it]
 66%|██████▌   | 148509/225000 [292:23:05<116:08:38,  5.47s/it]
 66%|██████▌   | 148510/225000 [292:23:09<107:07:13,  5.04s/it]
 66%|██████▌   | 148511/225000 [292:23:14<106:14:00,  5.00s/it]
 66%|██████▌   | 148512/225000 [292:23:24<139:02:53,  6.54s/it]
 66%|██████▌   | 148513/225000 [292:23:28<124:16:32,  5.85s/it]
 66%|██████▌   | 148514/225000 [292:23:32<111:38:14,  5.25s/it]
 66%|██████▌   | 148515/225000 [292:23:36<103:51:16,  4.89s/it]
 66%|██████▌   | 148516/225000 [292:23:44<122:04:08,  5.75s/it]
 66%|██████▌   | 148517/225000 [292:23:52<139:12:14,  6.55s/it]
 66%|██████▌   | 148518/225000 [292:23:56<120:44:22,  5.68s/it]
 66%|██████▌   | 148519/225000 [292:24:00<108:25:46,  5.10s/it]
 66%|██████▌   | 148520/225000 [292:24:09<138:08:36,  6.50s/it]
 66%|██████▌   | 148521/225000 [292:24:13<120:08:27,  5.66s/it]
 66%|██████▌   | 148522/225000 [292:24:17<107:12:14,  5.05s/it]
 66%|██████▌   | 148523/225000 [292:24:22<106:19:42,  5.01s/it]
 66%|██████▌   | 148524/225000 [292:24:27<107:54:58,  5.08s/it]
 66%|██████▌   | 148525/225000 [292:24:32<108:10:08,  5.09s/it]
 66%|██████▌   | 148526/225000 [292:24:37<109:49:50,  5.17s/it]
 66%|██████▌   | 148527/225000 [292:24:42<104:14:55,  4.91s/it]
 66%|██████▌   | 148528/225000 [292:24:46<103:10:02,  4.86s/it]
 66%|██████▌   | 148529/225000 [292:24:52<105:08:45,  4.95s/it]
 66%|██████▌   | 148530/225000 [292:24:57<107:40:59,  5.07s/it]
 66%|██████▌   | 148531/225000 [292:25:03<113:43:40,  5.35s/it]
 66%|██████▌   | 148532/225000 [292:25:16<162:15:08,  7.64s/it]
 66%|██████▌   | 148533/225000 [292:25:19<135:55:23,  6.40s/it]
 66%|██████▌   | 148534/225000 [292:25:23<118:52:42,  5.60s/it]
 66%|██████▌   | 148535/225000 [292:25:32<140:30:35,  6.62s/it]
 66%|██████▌   | 148536/225000 [292:25:38<135:34:20,  6.38s/it]
 66%|██████▌   | 148537/225000 [292:25:45<137:31:39,  6.48s/it]
 66%|██████▌   | 148538/225000 [292:25:48<119:35:51,  5.63s/it]
 66%|██████▌   | 148539/225000 [292:25:52<106:21:06,  5.01s/it]
 66%|██████▌   | 148540/225000 [292:25:58<112:33:13,  5.30s/it]
 66%|██████▌   | 148541/225000 [292:26:05<126:49:31,  5.97s/it]
 66%|██████▌   | 148542/225000 [292:26:09<109:25:19,  5.15s/it]
 66%|██████▌   | 148543/225000 [292:26:17<127:19:02,  5.99s/it]
 66%|██████▌   | 148544/225000 [292:26:20<112:29:08,  5.30s/it]
 66%|██████▌   | 148545/225000 [292:26:24<103:26:55,  4.87s/it]
 66%|██████▌   | 148546/225000 [292:26:30<109:55:35,  5.18s/it]
 66%|██████▌   | 148547/225000 [292:26:40<140:14:02,  6.60s/it]
 66%|██████▌   | 148548/225000 [292:26:43<119:30:55,  5.63s/it]
 66%|██████▌   | 148549/225000 [292:26:51<131:23:14,  6.19s/it]
 66%|██████▌   | 148550/225000 [292:26:56<124:22:20,  5.86s/it]
                                                               
{'loss': '0.7838', 'grad_norm': '1.056', 'learning_rate': '1.407e-05', 'epoch': '0.6602', 'num_input_tokens_seen': 484707152, 'train_runtime': '1.053e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148550/225000 [292:26:56<124:22:20,  5.86s/it]
 66%|██████▌   | 148551/225000 [292:27:02<125:01:16,  5.89s/it]
 66%|██████▌   | 148552/225000 [292:27:05<110:35:40,  5.21s/it]
 66%|██████▌   | 148553/225000 [292:27:10<108:50:54,  5.13s/it]
 66%|██████▌   | 148554/225000 [292:27:16<112:44:53,  5.31s/it]
 66%|██████▌   | 148555/225000 [292:27:21<113:01:50,  5.32s/it]
 66%|██████▌   | 148556/225000 [292:27:27<111:42:10,  5.26s/it]
 66%|██████▌   | 148557/225000 [292:27:32<113:19:37,  5.34s/it]
 66%|██████▌   | 148558/225000 [292:27:38<114:46:51,  5.41s/it]
 66%|██████▌   | 148559/225000 [292:27:43<113:57:46,  5.37s/it]
 66%|██████▌   | 148560/225000 [292:27:49<116:56:53,  5.51s/it]
 66%|██████▌   | 148561/225000 [292:27:59<144:16:52,  6.80s/it]
 66%|██████▌   | 148562/225000 [292:28:05<141:37:29,  6.67s/it]
 66%|██████▌   | 148563/225000 [292:28:09<121:55:33,  5.74s/it]
 66%|██████▌   | 148564/225000 [292:28:15<126:14:48,  5.95s/it]
 66%|██████▌   | 148565/225000 [292:28:20<121:14:25,  5.71s/it]
 66%|██████▌   | 148566/225000 [292:28:24<111:28:30,  5.25s/it]
 66%|██████▌   | 148567/225000 [292:28:31<123:47:17,  5.83s/it]
 66%|██████▌   | 148568/225000 [292:28:40<140:04:40,  6.60s/it]
 66%|██████▌   | 148569/225000 [292:28:44<122:43:11,  5.78s/it]
 66%|██████▌   | 148570/225000 [292:28:49<118:25:29,  5.58s/it]
 66%|██████▌   | 148571/225000 [292:28:54<115:25:24,  5.44s/it]
 66%|██████▌   | 148572/225000 [292:28:59<114:39:50,  5.40s/it]
 66%|██████▌   | 148573/225000 [292:29:04<111:38:57,  5.26s/it]
 66%|██████▌   | 148574/225000 [292:29:10<112:47:50,  5.31s/it]
 66%|██████▌   | 148575/225000 [292:29:15<113:22:09,  5.34s/it]
 66%|██████▌   | 148576/225000 [292:29:20<111:00:49,  5.23s/it]
 66%|██████▌   | 148577/225000 [292:29:25<110:48:18,  5.22s/it]
 66%|██████▌   | 148578/225000 [292:29:31<112:47:43,  5.31s/it]
 66%|██████▌   | 148579/225000 [292:29:36<110:01:15,  5.18s/it]
 66%|██████▌   | 148580/225000 [292:29:41<111:02:53,  5.23s/it]
 66%|██████▌   | 148581/225000 [292:29:46<111:26:24,  5.25s/it]
 66%|██████▌   | 148582/225000 [292:29:54<125:12:03,  5.90s/it]
 66%|██████▌   | 148583/225000 [292:29:59<122:34:26,  5.77s/it]
 66%|██████▌   | 148584/225000 [292:30:05<121:21:34,  5.72s/it]
 66%|██████▌   | 148585/225000 [292:30:10<119:44:43,  5.64s/it]
 66%|██████▌   | 148586/225000 [292:30:15<114:43:32,  5.40s/it]
 66%|██████▌   | 148587/225000 [292:30:21<120:53:24,  5.70s/it]
 66%|██████▌   | 148588/225000 [292:30:34<161:53:36,  7.63s/it]
 66%|██████▌   | 148589/225000 [292:30:37<134:28:54,  6.34s/it]
 66%|██████▌   | 148590/225000 [292:30:40<116:13:21,  5.48s/it]
 66%|██████▌   | 148591/225000 [292:30:51<150:22:40,  7.09s/it]
 66%|██████▌   | 148592/225000 [292:30:55<127:09:03,  5.99s/it]
 66%|██████▌   | 148593/225000 [292:31:02<136:51:46,  6.45s/it]
 66%|██████▌   | 148594/225000 [292:31:06<118:21:00,  5.58s/it]
 66%|██████▌   | 148595/225000 [292:31:11<115:46:37,  5.46s/it]
 66%|██████▌   | 148596/225000 [292:31:17<118:31:33,  5.58s/it]
 66%|██████▌   | 148597/225000 [292:31:22<113:29:52,  5.35s/it]
 66%|██████▌   | 148598/225000 [292:31:26<109:28:37,  5.16s/it]
 66%|██████▌   | 148599/225000 [292:31:32<111:13:38,  5.24s/it]
 66%|██████▌   | 148600/225000 [292:31:37<110:49:32,  5.22s/it]
                                                               
{'loss': '0.7836', 'grad_norm': '1.105', 'learning_rate': '1.405e-05', 'epoch': '0.6604', 'num_input_tokens_seen': 484836824, 'train_runtime': '1.053e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148600/225000 [292:31:37<110:49:32,  5.22s/it]
 66%|██████▌   | 148601/225000 [292:31:42<110:14:52,  5.20s/it]
 66%|██████▌   | 148602/225000 [292:31:47<111:43:52,  5.26s/it]
 66%|██████▌   | 148603/225000 [292:31:53<113:32:50,  5.35s/it]
 66%|██████▌   | 148604/225000 [292:31:58<114:07:04,  5.38s/it]
 66%|██████▌   | 148605/225000 [292:32:05<122:16:41,  5.76s/it]
 66%|██████▌   | 148606/225000 [292:32:15<147:00:02,  6.93s/it]
 66%|██████▌   | 148607/225000 [292:32:19<127:46:12,  6.02s/it]
 66%|██████▌   | 148608/225000 [292:32:23<115:26:21,  5.44s/it]
 66%|██████▌   | 148609/225000 [292:32:35<156:25:18,  7.37s/it]
 66%|██████▌   | 148610/225000 [292:32:39<134:53:34,  6.36s/it]
 66%|██████▌   | 148611/225000 [292:32:42<116:14:15,  5.48s/it]
 66%|██████▌   | 148612/225000 [292:32:46<106:17:46,  5.01s/it]
 66%|██████▌   | 148613/225000 [292:32:57<144:12:08,  6.80s/it]
 66%|██████▌   | 148614/225000 [292:33:01<124:45:08,  5.88s/it]
 66%|██████▌   | 148615/225000 [292:33:08<137:16:57,  6.47s/it]
 66%|██████▌   | 148616/225000 [292:33:12<119:19:59,  5.62s/it]
 66%|██████▌   | 148617/225000 [292:33:16<107:42:49,  5.08s/it]
 66%|██████▌   | 148618/225000 [292:33:23<121:20:59,  5.72s/it]
 66%|██████▌   | 148619/225000 [292:33:29<122:00:49,  5.75s/it]
 66%|██████▌   | 148620/225000 [292:33:32<106:34:58,  5.02s/it]
 66%|██████▌   | 148621/225000 [292:33:39<117:39:36,  5.55s/it]
 66%|██████▌   | 148622/225000 [292:33:46<126:13:23,  5.95s/it]
 66%|██████▌   | 148623/225000 [292:33:50<111:29:10,  5.25s/it]
 66%|██████▌   | 148624/225000 [292:33:53<102:51:18,  4.85s/it]
 66%|██████▌   | 148625/225000 [292:34:05<142:40:40,  6.73s/it]
 66%|██████▌   | 148626/225000 [292:34:08<119:36:43,  5.64s/it]
 66%|██████▌   | 148627/225000 [292:34:12<111:01:06,  5.23s/it]
 66%|██████▌   | 148628/225000 [292:34:16<105:41:12,  4.98s/it]
 66%|██████▌   | 148629/225000 [292:34:21<104:56:50,  4.95s/it]
 66%|██████▌   | 148630/225000 [292:34:26<105:46:25,  4.99s/it]
 66%|██████▌   | 148631/225000 [292:34:32<107:32:48,  5.07s/it]
 66%|██████▌   | 148632/225000 [292:34:37<107:39:27,  5.08s/it]
 66%|██████▌   | 148633/225000 [292:34:42<106:10:32,  5.01s/it]
 66%|██████▌   | 148634/225000 [292:34:46<105:36:56,  4.98s/it]
 66%|██████▌   | 148635/225000 [292:34:52<109:09:49,  5.15s/it]
 66%|██████▌   | 148636/225000 [292:35:05<156:43:04,  7.39s/it]
 66%|██████▌   | 148637/225000 [292:35:10<141:22:12,  6.66s/it]
 66%|██████▌   | 148638/225000 [292:35:16<142:46:41,  6.73s/it]
 66%|██████▌   | 148639/225000 [292:35:20<121:33:30,  5.73s/it]
 66%|██████▌   | 148640/225000 [292:35:30<147:21:04,  6.95s/it]
 66%|██████▌   | 148641/225000 [292:35:34<130:57:32,  6.17s/it]
 66%|██████▌   | 148642/225000 [292:35:38<117:47:37,  5.55s/it]
 66%|██████▌   | 148643/225000 [292:35:42<108:22:20,  5.11s/it]
 66%|██████▌   | 148644/225000 [292:35:49<121:49:38,  5.74s/it]
 66%|██████▌   | 148645/225000 [292:35:58<139:13:03,  6.56s/it]
 66%|██████▌   | 148646/225000 [292:36:02<124:02:57,  5.85s/it]
 66%|██████▌   | 148647/225000 [292:36:06<114:48:05,  5.41s/it]
 66%|██████▌   | 148648/225000 [292:36:11<108:35:49,  5.12s/it]
 66%|██████▌   | 148649/225000 [292:36:15<103:48:54,  4.89s/it]
 66%|██████▌   | 148650/225000 [292:36:24<127:48:05,  6.03s/it]
                                                               
{'loss': '0.7851', 'grad_norm': '1.046', 'learning_rate': '1.403e-05', 'epoch': '0.6607', 'num_input_tokens_seen': 484967016, 'train_runtime': '1.053e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148650/225000 [292:36:24<127:48:05,  6.03s/it]
 66%|██████▌   | 148651/225000 [292:36:33<146:59:07,  6.93s/it]
 66%|██████▌   | 148652/225000 [292:36:36<124:17:35,  5.86s/it]
 66%|██████▌   | 148653/225000 [292:36:46<148:10:17,  6.99s/it]
 66%|██████▌   | 148654/225000 [292:36:50<128:30:06,  6.06s/it]
 66%|██████▌   | 148655/225000 [292:36:54<113:40:18,  5.36s/it]
 66%|██████▌   | 148656/225000 [292:37:01<127:17:12,  6.00s/it]
 66%|██████▌   | 148657/225000 [292:37:07<124:47:17,  5.88s/it]
 66%|██████▌   | 148658/225000 [292:37:12<120:41:37,  5.69s/it]
 66%|██████▌   | 148659/225000 [292:37:17<116:57:25,  5.52s/it]
 66%|██████▌   | 148660/225000 [292:37:23<117:16:56,  5.53s/it]
 66%|██████▌   | 148661/225000 [292:37:27<111:25:22,  5.25s/it]
 66%|██████▌   | 148662/225000 [292:37:36<135:08:19,  6.37s/it]
 66%|██████▌   | 148663/225000 [292:37:41<123:06:47,  5.81s/it]
 66%|██████▌   | 148664/225000 [292:37:46<120:00:37,  5.66s/it]
 66%|██████▌   | 148665/225000 [292:37:51<116:42:30,  5.50s/it]
 66%|██████▌   | 148666/225000 [292:37:57<118:04:10,  5.57s/it]
 66%|██████▌   | 148667/225000 [292:38:01<111:04:28,  5.24s/it]
 66%|██████▌   | 148668/225000 [292:38:07<110:37:55,  5.22s/it]
 66%|██████▌   | 148669/225000 [292:38:12<113:17:57,  5.34s/it]
 66%|██████▌   | 148670/225000 [292:38:18<114:04:42,  5.38s/it]
 66%|██████▌   | 148671/225000 [292:38:23<111:21:02,  5.25s/it]
 66%|██████▌   | 148672/225000 [292:38:28<112:05:09,  5.29s/it]
 66%|██████▌   | 148673/225000 [292:38:33<113:14:34,  5.34s/it]
 66%|██████▌   | 148674/225000 [292:38:38<110:02:31,  5.19s/it]
 66%|██████▌   | 148675/225000 [292:38:43<108:05:45,  5.10s/it]
 66%|██████▌   | 148676/225000 [292:38:48<109:34:37,  5.17s/it]
 66%|██████▌   | 148677/225000 [292:38:54<112:42:53,  5.32s/it]
 66%|██████▌   | 148678/225000 [292:38:59<110:26:11,  5.21s/it]
 66%|██████▌   | 148679/225000 [292:39:04<110:16:39,  5.20s/it]
 66%|██████▌   | 148680/225000 [292:39:10<111:09:15,  5.24s/it]
 66%|██████▌   | 148681/225000 [292:39:15<111:45:00,  5.27s/it]
 66%|██████▌   | 148682/225000 [292:39:20<110:40:54,  5.22s/it]
 66%|██████▌   | 148683/225000 [292:39:33<158:56:50,  7.50s/it]
 66%|██████▌   | 148684/225000 [292:39:37<138:09:03,  6.52s/it]
 66%|██████▌   | 148685/225000 [292:39:41<122:47:07,  5.79s/it]
 66%|██████▌   | 148686/225000 [292:39:46<113:29:15,  5.35s/it]
 66%|██████▌   | 148687/225000 [292:39:50<106:29:22,  5.02s/it]
 66%|██████▌   | 148688/225000 [292:39:55<105:47:55,  4.99s/it]
 66%|██████▌   | 148689/225000 [292:40:07<151:10:02,  7.13s/it]
 66%|██████▌   | 148690/225000 [292:40:11<133:01:28,  6.28s/it]
 66%|██████▌   | 148691/225000 [292:40:16<124:52:04,  5.89s/it]
 66%|██████▌   | 148692/225000 [292:40:21<116:50:24,  5.51s/it]
 66%|██████▌   | 148693/225000 [292:40:26<113:01:07,  5.33s/it]
 66%|██████▌   | 148694/225000 [292:40:31<110:38:43,  5.22s/it]
 66%|██████▌   | 148695/225000 [292:40:35<107:58:36,  5.09s/it]
 66%|██████▌   | 148696/225000 [292:40:40<105:28:04,  4.98s/it]
 66%|██████▌   | 148697/225000 [292:40:45<102:01:55,  4.81s/it]
 66%|██████▌   | 148698/225000 [292:40:55<140:10:03,  6.61s/it]
 66%|██████▌   | 148699/225000 [292:41:03<146:45:57,  6.92s/it]
 66%|██████▌   | 148700/225000 [292:41:07<127:02:13,  5.99s/it]
                                                               
{'loss': '0.8022', 'grad_norm': '1.123', 'learning_rate': '1.402e-05', 'epoch': '0.6609', 'num_input_tokens_seen': 485095400, 'train_runtime': '1.054e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148700/225000 [292:41:07<127:02:13,  5.99s/it]
 66%|██████▌   | 148701/225000 [292:41:11<113:13:18,  5.34s/it]
 66%|██████▌   | 148702/225000 [292:41:20<138:59:49,  6.56s/it]
 66%|██████▌   | 148703/225000 [292:41:23<117:40:01,  5.55s/it]
 66%|██████▌   | 148704/225000 [292:41:31<132:38:17,  6.26s/it]
 66%|██████▌   | 148705/225000 [292:41:36<124:09:30,  5.86s/it]
 66%|██████▌   | 148706/225000 [292:41:40<109:55:02,  5.19s/it]
 66%|██████▌   | 148707/225000 [292:41:49<134:22:18,  6.34s/it]
 66%|██████▌   | 148708/225000 [292:41:52<115:07:09,  5.43s/it]
 66%|██████▌   | 148709/225000 [292:41:57<111:11:17,  5.25s/it]
 66%|██████▌   | 148710/225000 [292:42:01<107:07:34,  5.06s/it]
 66%|██████▌   | 148711/225000 [292:42:06<105:43:10,  4.99s/it]
 66%|██████▌   | 148712/225000 [292:42:11<106:06:30,  5.01s/it]
 66%|██████▌   | 148713/225000 [292:42:16<102:06:01,  4.82s/it]
 66%|██████▌   | 148714/225000 [292:42:20<98:15:45,  4.64s/it] 
 66%|██████▌   | 148715/225000 [292:42:25<99:24:29,  4.69s/it]
 66%|██████▌   | 148716/225000 [292:42:30<100:47:15,  4.76s/it]
 66%|██████▌   | 148717/225000 [292:42:39<130:25:23,  6.16s/it]
 66%|██████▌   | 148718/225000 [292:42:50<158:18:05,  7.47s/it]
 66%|██████▌   | 148719/225000 [292:42:53<131:26:25,  6.20s/it]
 66%|██████▌   | 148720/225000 [292:43:00<140:42:49,  6.64s/it]
 66%|██████▌   | 148721/225000 [292:43:04<120:44:25,  5.70s/it]
 66%|██████▌   | 148722/225000 [292:43:08<112:48:39,  5.32s/it]
 66%|██████▌   | 148723/225000 [292:43:18<137:08:51,  6.47s/it]
 66%|██████▌   | 148724/225000 [292:43:22<122:42:28,  5.79s/it]
 66%|██████▌   | 148725/225000 [292:43:26<111:31:25,  5.26s/it]
 66%|██████▌   | 148726/225000 [292:43:30<105:06:53,  4.96s/it]
 66%|██████▌   | 148727/225000 [292:43:35<102:00:58,  4.82s/it]
 66%|██████▌   | 148728/225000 [292:43:45<136:59:05,  6.47s/it]
 66%|██████▌   | 148729/225000 [292:43:50<128:07:45,  6.05s/it]
 66%|██████▌   | 148730/225000 [292:43:57<134:04:59,  6.33s/it]
 66%|██████▌   | 148731/225000 [292:44:01<119:19:26,  5.63s/it]
 66%|██████▌   | 148732/225000 [292:44:05<107:33:45,  5.08s/it]
 66%|██████▌   | 148733/225000 [292:44:14<134:21:51,  6.34s/it]
 66%|██████▌   | 148734/225000 [292:44:18<116:16:36,  5.49s/it]
 66%|██████▌   | 148735/225000 [292:44:22<108:07:28,  5.10s/it]
 66%|██████▌   | 148736/225000 [292:44:26<103:29:21,  4.89s/it]
 66%|██████▌   | 148737/225000 [292:44:30<99:53:11,  4.72s/it] 
 66%|██████▌   | 148738/225000 [292:44:39<125:50:41,  5.94s/it]
 66%|██████▌   | 148739/225000 [292:44:47<138:06:03,  6.52s/it]
 66%|██████▌   | 148740/225000 [292:44:51<118:27:43,  5.59s/it]
 66%|██████▌   | 148741/225000 [292:44:59<137:52:43,  6.51s/it]
 66%|██████▌   | 148742/225000 [292:45:02<117:30:56,  5.55s/it]
 66%|██████▌   | 148743/225000 [292:45:06<106:43:51,  5.04s/it]
 66%|██████▌   | 148744/225000 [292:45:10<99:39:42,  4.70s/it] 
 66%|██████▌   | 148745/225000 [292:45:22<143:49:03,  6.79s/it]
 66%|██████▌   | 148746/225000 [292:45:26<127:24:16,  6.01s/it]
 66%|██████▌   | 148747/225000 [292:45:34<138:07:01,  6.52s/it]
 66%|██████▌   | 148748/225000 [292:45:37<118:34:08,  5.60s/it]
 66%|██████▌   | 148749/225000 [292:45:46<139:10:54,  6.57s/it]
 66%|██████▌   | 148750/225000 [292:45:50<124:05:20,  5.86s/it]
                                                               
{'loss': '0.7877', 'grad_norm': '1.258', 'learning_rate': '1.4e-05', 'epoch': '0.6611', 'num_input_tokens_seen': 485225704, 'train_runtime': '1.054e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148750/225000 [292:45:50<124:05:20,  5.86s/it]
 66%|██████▌   | 148751/225000 [292:45:55<116:49:08,  5.52s/it]
 66%|██████▌   | 148752/225000 [292:45:59<103:59:32,  4.91s/it]
 66%|██████▌   | 148753/225000 [292:46:07<125:27:18,  5.92s/it]
 66%|██████▌   | 148754/225000 [292:46:11<115:04:15,  5.43s/it]
 66%|██████▌   | 148755/225000 [292:46:15<106:58:00,  5.05s/it]
 66%|██████▌   | 148756/225000 [292:46:19<101:55:12,  4.81s/it]
 66%|██████▌   | 148757/225000 [292:46:24<101:42:30,  4.80s/it]
 66%|██████▌   | 148758/225000 [292:46:30<110:35:49,  5.22s/it]
 66%|██████▌   | 148759/225000 [292:46:43<154:13:35,  7.28s/it]
 66%|██████▌   | 148760/225000 [292:46:47<136:19:22,  6.44s/it]
 66%|██████▌   | 148761/225000 [292:46:50<115:35:02,  5.46s/it]
 66%|██████▌   | 148762/225000 [292:46:58<133:08:38,  6.29s/it]
 66%|██████▌   | 148763/225000 [292:47:03<123:55:50,  5.85s/it]
 66%|██████▌   | 148764/225000 [292:47:08<116:59:25,  5.52s/it]
 66%|██████▌   | 148765/225000 [292:47:13<114:02:26,  5.39s/it]
 66%|██████▌   | 148766/225000 [292:47:18<113:21:30,  5.35s/it]
 66%|██████▌   | 148767/225000 [292:47:23<111:14:07,  5.25s/it]
 66%|██████▌   | 148768/225000 [292:47:28<106:55:40,  5.05s/it]
 66%|██████▌   | 148769/225000 [292:47:33<105:45:56,  4.99s/it]
 66%|██████▌   | 148770/225000 [292:47:45<153:39:22,  7.26s/it]
 66%|██████▌   | 148771/225000 [292:47:49<133:17:52,  6.30s/it]
 66%|██████▌   | 148772/225000 [292:47:53<115:24:06,  5.45s/it]
 66%|██████▌   | 148773/225000 [292:48:00<124:34:44,  5.88s/it]
 66%|██████▌   | 148774/225000 [292:48:08<140:47:15,  6.65s/it]
 66%|██████▌   | 148775/225000 [292:48:12<121:07:24,  5.72s/it]
 66%|██████▌   | 148776/225000 [292:48:18<122:29:31,  5.79s/it]
 66%|██████▌   | 148777/225000 [292:48:24<127:23:12,  6.02s/it]
 66%|██████▌   | 148778/225000 [292:48:28<111:43:32,  5.28s/it]
 66%|██████▌   | 148779/225000 [292:48:36<128:12:45,  6.06s/it]
 66%|██████▌   | 148780/225000 [292:48:39<110:19:36,  5.21s/it]
 66%|██████▌   | 148781/225000 [292:48:46<124:55:45,  5.90s/it]
 66%|██████▌   | 148782/225000 [292:48:50<111:16:30,  5.26s/it]
 66%|██████▌   | 148783/225000 [292:48:58<127:08:27,  6.01s/it]
 66%|██████▌   | 148784/225000 [292:49:03<122:42:28,  5.80s/it]
 66%|██████▌   | 148785/225000 [292:49:08<113:04:15,  5.34s/it]
 66%|██████▌   | 148786/225000 [292:49:14<119:20:23,  5.64s/it]
 66%|██████▌   | 148787/225000 [292:49:17<105:51:25,  5.00s/it]
 66%|██████▌   | 148788/225000 [292:49:22<104:04:46,  4.92s/it]
 66%|██████▌   | 148789/225000 [292:49:26<100:20:04,  4.74s/it]
 66%|██████▌   | 148790/225000 [292:49:31<98:49:15,  4.67s/it] 
 66%|██████▌   | 148791/225000 [292:49:36<98:35:40,  4.66s/it]
 66%|██████▌   | 148792/225000 [292:49:41<105:31:09,  4.98s/it]
 66%|██████▌   | 148793/225000 [292:49:54<152:41:10,  7.21s/it]
 66%|██████▌   | 148794/225000 [292:49:57<130:16:00,  6.15s/it]
 66%|██████▌   | 148795/225000 [292:50:01<112:50:54,  5.33s/it]
 66%|██████▌   | 148796/225000 [292:50:11<140:39:41,  6.65s/it]
 66%|██████▌   | 148797/225000 [292:50:15<126:07:35,  5.96s/it]
 66%|██████▌   | 148798/225000 [292:50:19<114:47:16,  5.42s/it]
 66%|██████▌   | 148799/225000 [292:50:23<107:42:06,  5.09s/it]
 66%|██████▌   | 148800/225000 [292:50:28<103:19:04,  4.88s/it]
                                                               
{'loss': '0.7897', 'grad_norm': '1.053', 'learning_rate': '1.398e-05', 'epoch': '0.6613', 'num_input_tokens_seen': 485356416, 'train_runtime': '1.054e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148800/225000 [292:50:28<103:19:04,  4.88s/it]
 66%|██████▌   | 148801/225000 [292:50:38<139:57:42,  6.61s/it]
 66%|██████▌   | 148802/225000 [292:50:42<121:54:59,  5.76s/it]
 66%|██████▌   | 148803/225000 [292:50:46<110:14:15,  5.21s/it]
 66%|██████▌   | 148804/225000 [292:50:51<107:35:27,  5.08s/it]
 66%|██████▌   | 148805/225000 [292:51:01<141:50:27,  6.70s/it]
 66%|██████▌   | 148806/225000 [292:51:05<122:54:26,  5.81s/it]
 66%|██████▌   | 148807/225000 [292:51:13<137:58:02,  6.52s/it]
 66%|██████▌   | 148808/225000 [292:51:17<120:01:24,  5.67s/it]
 66%|██████▌   | 148809/225000 [292:51:21<106:48:27,  5.05s/it]
 66%|██████▌   | 148810/225000 [292:51:26<109:19:51,  5.17s/it]
 66%|██████▌   | 148811/225000 [292:51:33<119:32:11,  5.65s/it]
 66%|██████▌   | 148812/225000 [292:51:40<127:53:55,  6.04s/it]
 66%|██████▌   | 148813/225000 [292:51:44<115:30:16,  5.46s/it]
 66%|██████▌   | 148814/225000 [292:51:47<104:04:35,  4.92s/it]
 66%|██████▌   | 148815/225000 [292:51:57<133:06:26,  6.29s/it]
 66%|██████▌   | 148816/225000 [292:52:00<115:11:51,  5.44s/it]
 66%|██████▌   | 148817/225000 [292:52:04<104:16:33,  4.93s/it]
 66%|██████▌   | 148818/225000 [292:52:08<98:51:33,  4.67s/it] 
 66%|██████▌   | 148819/225000 [292:52:20<146:30:25,  6.92s/it]
 66%|██████▌   | 148820/225000 [292:52:24<124:13:03,  5.87s/it]
 66%|██████▌   | 148821/225000 [292:52:27<109:05:23,  5.16s/it]
 66%|██████▌   | 148822/225000 [292:52:36<131:30:36,  6.21s/it]
 66%|██████▌   | 148823/225000 [292:52:40<115:37:50,  5.46s/it]
 66%|██████▌   | 148824/225000 [292:52:47<124:35:18,  5.89s/it]
 66%|██████▌   | 148825/225000 [292:52:54<132:56:41,  6.28s/it]
 66%|██████▌   | 148826/225000 [292:52:58<119:49:11,  5.66s/it]
 66%|██████▌   | 148827/225000 [292:53:03<113:41:45,  5.37s/it]
 66%|██████▌   | 148828/225000 [292:53:07<109:28:06,  5.17s/it]
 66%|██████▌   | 148829/225000 [292:53:12<105:47:01,  5.00s/it]
 66%|██████▌   | 148830/225000 [292:53:17<103:20:34,  4.88s/it]
 66%|██████▌   | 148831/225000 [292:53:21<100:05:33,  4.73s/it]
 66%|██████▌   | 148832/225000 [292:53:33<148:09:20,  7.00s/it]
 66%|██████▌   | 148833/225000 [292:53:37<130:19:56,  6.16s/it]
 66%|██████▌   | 148834/225000 [292:53:42<117:47:33,  5.57s/it]
 66%|██████▌   | 148835/225000 [292:53:47<115:08:03,  5.44s/it]
 66%|██████▌   | 148836/225000 [292:53:57<144:42:24,  6.84s/it]
 66%|██████▌   | 148837/225000 [292:54:01<124:09:44,  5.87s/it]
 66%|██████▌   | 148838/225000 [292:54:08<134:37:17,  6.36s/it]
 66%|██████▌   | 148839/225000 [292:54:13<126:41:30,  5.99s/it]
 66%|██████▌   | 148840/225000 [292:54:17<112:14:40,  5.31s/it]
 66%|██████▌   | 148841/225000 [292:54:21<102:04:24,  4.82s/it]
 66%|██████▌   | 148842/225000 [292:54:25<98:25:45,  4.65s/it] 
 66%|██████▌   | 148843/225000 [292:54:30<100:32:02,  4.75s/it]
 66%|██████▌   | 148844/225000 [292:54:35<103:43:06,  4.90s/it]
 66%|██████▌   | 148845/225000 [292:54:40<105:02:24,  4.97s/it]
 66%|██████▌   | 148846/225000 [292:54:45<105:25:44,  4.98s/it]
 66%|██████▌   | 148847/225000 [292:54:50<105:41:51,  5.00s/it]
 66%|██████▌   | 148848/225000 [292:55:02<150:14:33,  7.10s/it]
 66%|██████▌   | 148849/225000 [292:55:08<141:35:56,  6.69s/it]
 66%|██████▌   | 148850/225000 [292:55:11<121:15:44,  5.73s/it]
                                                               
{'loss': '0.7832', 'grad_norm': '1.137', 'learning_rate': '1.397e-05', 'epoch': '0.6616', 'num_input_tokens_seen': 485486408, 'train_runtime': '1.055e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148850/225000 [292:55:11<121:15:44,  5.73s/it]
 66%|██████▌   | 148851/225000 [292:55:21<146:30:00,  6.93s/it]
 66%|██████▌   | 148852/225000 [292:55:25<127:12:46,  6.01s/it]
 66%|██████▌   | 148853/225000 [292:55:29<113:14:12,  5.35s/it]
 66%|██████▌   | 148854/225000 [292:55:38<140:01:03,  6.62s/it]
 66%|██████▌   | 148855/225000 [292:55:42<121:32:20,  5.75s/it]
 66%|██████▌   | 148856/225000 [292:55:46<110:22:28,  5.22s/it]
 66%|██████▌   | 148857/225000 [292:55:56<141:15:11,  6.68s/it]
 66%|██████▌   | 148858/225000 [292:56:00<121:42:29,  5.75s/it]
 66%|██████▌   | 148859/225000 [292:56:04<108:52:59,  5.15s/it]
 66%|██████▌   | 148860/225000 [292:56:12<132:22:15,  6.26s/it]
 66%|██████▌   | 148861/225000 [292:56:17<121:39:24,  5.75s/it]
 66%|██████▌   | 148862/225000 [292:56:22<116:49:30,  5.52s/it]
 66%|██████▌   | 148863/225000 [292:56:26<109:18:07,  5.17s/it]
 66%|██████▌   | 148864/225000 [292:56:31<106:29:16,  5.04s/it]
 66%|██████▌   | 148865/225000 [292:56:35<102:34:56,  4.85s/it]
 66%|██████▌   | 148866/225000 [292:56:40<100:36:50,  4.76s/it]
 66%|██████▌   | 148867/225000 [292:56:53<150:05:05,  7.10s/it]
 66%|██████▌   | 148868/225000 [292:56:57<131:20:33,  6.21s/it]
 66%|██████▌   | 148869/225000 [292:57:02<122:42:19,  5.80s/it]
 66%|██████▌   | 148870/225000 [292:57:06<113:27:53,  5.37s/it]
 66%|██████▌   | 148871/225000 [292:57:10<106:25:08,  5.03s/it]
 66%|██████▌   | 148872/225000 [292:57:15<102:58:10,  4.87s/it]
 66%|██████▌   | 148873/225000 [292:57:19<101:38:36,  4.81s/it]
 66%|██████▌   | 148874/225000 [292:57:31<147:54:36,  6.99s/it]
 66%|██████▌   | 148875/225000 [292:57:37<138:33:57,  6.55s/it]
 66%|██████▌   | 148876/225000 [292:57:42<130:23:40,  6.17s/it]
 66%|██████▌   | 148877/225000 [292:57:48<128:06:38,  6.06s/it]
 66%|██████▌   | 148878/225000 [292:57:53<124:10:51,  5.87s/it]
 66%|██████▌   | 148879/225000 [292:57:59<119:13:59,  5.64s/it]
 66%|██████▌   | 148880/225000 [292:58:03<113:17:16,  5.36s/it]
 66%|██████▌   | 148881/225000 [292:58:08<109:28:00,  5.18s/it]
 66%|██████▌   | 148882/225000 [292:58:13<109:20:03,  5.17s/it]
 66%|██████▌   | 148883/225000 [292:58:18<109:04:17,  5.16s/it]
 66%|██████▌   | 148884/225000 [292:58:23<104:49:18,  4.96s/it]
 66%|██████▌   | 148885/225000 [292:58:27<101:58:04,  4.82s/it]
 66%|██████▌   | 148886/225000 [292:58:33<105:52:15,  5.01s/it]
 66%|██████▌   | 148887/225000 [292:58:38<106:04:52,  5.02s/it]
 66%|██████▌   | 148888/225000 [292:58:43<107:20:23,  5.08s/it]
 66%|██████▌   | 148889/225000 [292:58:49<110:13:03,  5.21s/it]
 66%|██████▌   | 148890/225000 [292:59:02<160:33:59,  7.59s/it]
 66%|██████▌   | 148891/225000 [292:59:07<145:19:47,  6.87s/it]
 66%|██████▌   | 148892/225000 [292:59:11<128:06:58,  6.06s/it]
 66%|██████▌   | 148893/225000 [292:59:15<116:40:06,  5.52s/it]
 66%|██████▌   | 148894/225000 [292:59:25<146:08:16,  6.91s/it]
 66%|██████▌   | 148895/225000 [292:59:29<124:35:34,  5.89s/it]
 66%|██████▌   | 148896/225000 [292:59:38<142:41:09,  6.75s/it]
 66%|██████▌   | 148897/225000 [292:59:41<123:51:55,  5.86s/it]
 66%|██████▌   | 148898/225000 [292:59:46<112:25:28,  5.32s/it]
 66%|██████▌   | 148899/225000 [292:59:56<146:15:33,  6.92s/it]
 66%|██████▌   | 148900/225000 [293:00:00<124:24:44,  5.89s/it]
                                                               
{'loss': '0.7865', 'grad_norm': '1.73', 'learning_rate': '1.395e-05', 'epoch': '0.6618', 'num_input_tokens_seen': 485616816, 'train_runtime': '1.055e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148900/225000 [293:00:00<124:24:44,  5.89s/it]
 66%|██████▌   | 148901/225000 [293:00:05<121:12:58,  5.73s/it]
 66%|██████▌   | 148902/225000 [293:00:13<134:16:51,  6.35s/it]
 66%|██████▌   | 148903/225000 [293:00:16<116:14:56,  5.50s/it]
 66%|██████▌   | 148904/225000 [293:00:25<139:21:42,  6.59s/it]
 66%|██████▌   | 148905/225000 [293:00:29<119:46:33,  5.67s/it]
 66%|██████▌   | 148906/225000 [293:00:33<107:49:11,  5.10s/it]
 66%|██████▌   | 148907/225000 [293:00:40<119:57:40,  5.68s/it]
 66%|██████▌   | 148908/225000 [293:00:46<121:39:44,  5.76s/it]
 66%|██████▌   | 148909/225000 [293:00:50<111:30:25,  5.28s/it]
 66%|██████▌   | 148910/225000 [293:00:54<104:00:03,  4.92s/it]
 66%|██████▌   | 148911/225000 [293:00:58<99:03:49,  4.69s/it] 
 66%|██████▌   | 148912/225000 [293:01:09<135:43:33,  6.42s/it]
 66%|██████▌   | 148913/225000 [293:01:15<135:21:44,  6.40s/it]
 66%|██████▌   | 148914/225000 [293:01:19<117:47:58,  5.57s/it]
 66%|██████▌   | 148915/225000 [293:01:22<106:25:28,  5.04s/it]
 66%|██████▌   | 148916/225000 [293:01:31<127:17:09,  6.02s/it]
 66%|██████▌   | 148917/225000 [293:01:35<114:00:11,  5.39s/it]
 66%|██████▌   | 148918/225000 [293:01:39<105:29:46,  4.99s/it]
 66%|██████▌   | 148919/225000 [293:01:44<109:32:06,  5.18s/it]
 66%|██████▌   | 148920/225000 [293:01:55<142:05:19,  6.72s/it]
 66%|██████▌   | 148921/225000 [293:01:58<120:22:25,  5.70s/it]
 66%|██████▌   | 148922/225000 [293:02:02<107:59:04,  5.11s/it]
 66%|██████▌   | 148923/225000 [293:02:11<136:35:55,  6.46s/it]
 66%|██████▌   | 148924/225000 [293:02:15<119:57:20,  5.68s/it]
 66%|██████▌   | 148925/225000 [293:02:19<105:38:25,  5.00s/it]
 66%|██████▌   | 148926/225000 [293:02:28<131:22:37,  6.22s/it]
 66%|██████▌   | 148927/225000 [293:02:31<112:28:29,  5.32s/it]
 66%|██████▌   | 148928/225000 [293:02:36<109:46:41,  5.20s/it]
 66%|██████▌   | 148929/225000 [293:02:44<128:48:05,  6.10s/it]
 66%|██████▌   | 148930/225000 [293:02:49<120:33:02,  5.71s/it]
 66%|██████▌   | 148931/225000 [293:02:57<135:30:54,  6.41s/it]
 66%|██████▌   | 148932/225000 [293:03:00<116:31:57,  5.52s/it]
 66%|██████▌   | 148933/225000 [293:03:09<136:12:11,  6.45s/it]
 66%|██████▌   | 148934/225000 [293:03:14<130:01:07,  6.15s/it]
 66%|██████▌   | 148935/225000 [293:03:19<118:16:27,  5.60s/it]
 66%|██████▌   | 148936/225000 [293:03:22<106:54:14,  5.06s/it]
 66%|██████▌   | 148937/225000 [293:03:32<134:11:43,  6.35s/it]
 66%|██████▌   | 148938/225000 [293:03:35<116:56:16,  5.53s/it]
 66%|██████▌   | 148939/225000 [293:03:40<109:14:15,  5.17s/it]
 66%|██████▌   | 148940/225000 [293:03:44<106:27:33,  5.04s/it]
 66%|██████▌   | 148941/225000 [293:03:49<103:24:45,  4.89s/it]
 66%|██████▌   | 148942/225000 [293:03:54<102:01:45,  4.83s/it]
 66%|██████▌   | 148943/225000 [293:03:58<101:26:16,  4.80s/it]
 66%|██████▌   | 148944/225000 [293:04:03<102:07:57,  4.83s/it]
 66%|██████▌   | 148945/225000 [293:04:12<124:08:15,  5.88s/it]
 66%|██████▌   | 148946/225000 [293:04:22<150:45:56,  7.14s/it]
 66%|██████▌   | 148947/225000 [293:04:25<128:51:41,  6.10s/it]
 66%|██████▌   | 148948/225000 [293:04:29<113:53:54,  5.39s/it]
 66%|██████▌   | 148949/225000 [293:04:39<143:11:01,  6.78s/it]
 66%|██████▌   | 148950/225000 [293:04:44<132:03:02,  6.25s/it]
                                                               
{'loss': '0.7708', 'grad_norm': '1.125', 'learning_rate': '1.393e-05', 'epoch': '0.662', 'num_input_tokens_seen': 485746920, 'train_runtime': '1.055e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 148950/225000 [293:04:44<132:03:02,  6.25s/it]
 66%|██████▌   | 148951/225000 [293:04:49<125:13:37,  5.93s/it]
 66%|██████▌   | 148952/225000 [293:04:54<119:22:18,  5.65s/it]
 66%|██████▌   | 148953/225000 [293:04:59<115:38:51,  5.47s/it]
 66%|██████▌   | 148954/225000 [293:05:05<116:11:40,  5.50s/it]
 66%|██████▌   | 148955/225000 [293:05:10<115:35:38,  5.47s/it]
 66%|██████▌   | 148956/225000 [293:05:16<114:51:44,  5.44s/it]
 66%|██████▌   | 148957/225000 [293:05:21<115:36:08,  5.47s/it]
 66%|██████▌   | 148958/225000 [293:05:27<116:41:01,  5.52s/it]
 66%|██████▌   | 148959/225000 [293:05:32<115:58:41,  5.49s/it]
 66%|██████▌   | 148960/225000 [293:05:37<112:56:51,  5.35s/it]
 66%|██████▌   | 148961/225000 [293:05:43<112:26:03,  5.32s/it]
 66%|██████▌   | 148962/225000 [293:05:48<111:20:02,  5.27s/it]
 66%|██████▌   | 148963/225000 [293:05:53<112:14:32,  5.31s/it]
 66%|██████▌   | 148964/225000 [293:05:59<113:15:15,  5.36s/it]
 66%|██████▌   | 148965/225000 [293:06:03<108:51:42,  5.15s/it]
 66%|██████▌   | 148966/225000 [293:06:09<113:41:05,  5.38s/it]
 66%|██████▌   | 148967/225000 [293:06:20<148:55:41,  7.05s/it]
 66%|██████▌   | 148968/225000 [293:06:26<140:51:01,  6.67s/it]
 66%|██████▌   | 148969/225000 [293:06:33<144:11:16,  6.83s/it]
 66%|██████▌   | 148970/225000 [293:06:37<122:18:24,  5.79s/it]
 66%|██████▌   | 148971/225000 [293:06:46<143:25:16,  6.79s/it]
 66%|██████▌   | 148972/225000 [293:06:50<125:21:56,  5.94s/it]
 66%|██████▌   | 148973/225000 [293:06:54<118:42:18,  5.62s/it]
 66%|██████▌   | 148974/225000 [293:07:03<135:18:06,  6.41s/it]
 66%|██████▌   | 148975/225000 [293:07:06<118:03:37,  5.59s/it]
 66%|██████▌   | 148976/225000 [293:07:12<114:56:50,  5.44s/it]
 66%|██████▌   | 148977/225000 [293:07:20<136:26:13,  6.46s/it]
 66%|██████▌   | 148978/225000 [293:07:24<116:27:07,  5.51s/it]
 66%|██████▌   | 148979/225000 [293:07:32<133:26:57,  6.32s/it]
 66%|██████▌   | 148980/225000 [293:07:36<120:39:54,  5.71s/it]
 66%|██████▌   | 148981/225000 [293:07:40<110:53:30,  5.25s/it]
 66%|██████▌   | 148982/225000 [293:07:44<103:45:37,  4.91s/it]
 66%|██████▌   | 148983/225000 [293:07:51<112:01:38,  5.31s/it]
 66%|██████▌   | 148984/225000 [293:08:01<145:33:22,  6.89s/it]
 66%|██████▌   | 148985/225000 [293:08:05<122:48:57,  5.82s/it]
 66%|██████▌   | 148986/225000 [293:08:11<128:23:20,  6.08s/it]
 66%|██████▌   | 148987/225000 [293:08:15<111:45:55,  5.29s/it]
 66%|██████▌   | 148988/225000 [293:08:19<103:27:29,  4.90s/it]
 66%|██████▌   | 148989/225000 [293:08:28<129:02:46,  6.11s/it]
 66%|██████▌   | 148990/225000 [293:08:34<127:19:42,  6.03s/it]
 66%|██████▌   | 148991/225000 [293:08:39<121:12:39,  5.74s/it]
 66%|██████▌   | 148992/225000 [293:08:45<127:58:58,  6.06s/it]
 66%|██████▌   | 148993/225000 [293:08:50<118:38:49,  5.62s/it]
 66%|██████▌   | 148994/225000 [293:08:57<125:48:30,  5.96s/it]
 66%|██████▌   | 148995/225000 [293:09:01<112:10:48,  5.31s/it]
 66%|██████▌   | 148996/225000 [293:09:10<139:57:54,  6.63s/it]
 66%|██████▌   | 148997/225000 [293:09:14<123:57:39,  5.87s/it]
 66%|██████▌   | 148998/225000 [293:09:18<111:38:06,  5.29s/it]
 66%|██████▌   | 148999/225000 [293:09:22<103:34:55,  4.91s/it]
 66%|██████▌   | 149000/225000 [293:09:34<145:02:17,  6.87s/it]
                                                               
{'loss': '0.7788', 'grad_norm': '1.003', 'learning_rate': '1.392e-05', 'epoch': '0.6622', 'num_input_tokens_seen': 485877584, 'train_runtime': '1.055e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 149000/225000 [293:09:34<145:02:17,  6.87s/it]
 66%|██████▌   | 149001/225000 [293:09:37<124:24:58,  5.89s/it]
 66%|██████▌   | 149002/225000 [293:09:41<110:29:08,  5.23s/it]
 66%|██████▌   | 149003/225000 [293:09:52<146:45:50,  6.95s/it]
 66%|██████▌   | 149004/225000 [293:09:55<122:42:51,  5.81s/it]
 66%|██████▌   | 149005/225000 [293:10:02<131:52:59,  6.25s/it]
 66%|██████▌   | 149006/225000 [293:10:06<116:35:20,  5.52s/it]
 66%|██████▌   | 149007/225000 [293:10:13<124:48:15,  5.91s/it]
 66%|██████▌   | 149008/225000 [293:10:19<126:36:26,  6.00s/it]
 66%|██████▌   | 149009/225000 [293:10:24<118:57:24,  5.64s/it]
 66%|██████▌   | 149010/225000 [293:10:29<112:31:52,  5.33s/it]
 66%|██████▌   | 149011/225000 [293:10:33<104:46:33,  4.96s/it]
 66%|██████▌   | 149012/225000 [293:10:37<101:04:15,  4.79s/it]
 66%|██████▌   | 149013/225000 [293:10:50<150:12:06,  7.12s/it]
 66%|██████▌   | 149014/225000 [293:10:53<127:27:44,  6.04s/it]
 66%|██████▌   | 149015/225000 [293:11:01<137:32:00,  6.52s/it]
 66%|██████▌   | 149016/225000 [293:11:05<122:31:12,  5.80s/it]
 66%|██████▌   | 149017/225000 [293:11:09<113:31:32,  5.38s/it]
 66%|██████▌   | 149018/225000 [293:11:14<107:00:31,  5.07s/it]
 66%|██████▌   | 149019/225000 [293:11:18<101:24:36,  4.80s/it]
 66%|██████▌   | 149020/225000 [293:11:22<96:38:35,  4.58s/it] 
 66%|██████▌   | 149021/225000 [293:11:27<98:04:44,  4.65s/it]
 66%|██████▌   | 149022/225000 [293:11:32<98:41:23,  4.68s/it]
 66%|██████▌   | 149023/225000 [293:11:43<141:36:16,  6.71s/it]
 66%|██████▌   | 149024/225000 [293:11:50<142:21:09,  6.75s/it]
 66%|██████▌   | 149025/225000 [293:11:53<122:10:43,  5.79s/it]
 66%|██████▌   | 149026/225000 [293:12:00<130:10:23,  6.17s/it]
 66%|██████▌   | 149027/225000 [293:12:07<132:08:28,  6.26s/it]
 66%|██████▌   | 149028/225000 [293:12:14<137:49:30,  6.53s/it]
 66%|██████▌   | 149029/225000 [293:12:18<118:58:57,  5.64s/it]
 66%|██████▌   | 149030/225000 [293:12:21<105:39:49,  5.01s/it]
 66%|██████▌   | 149031/225000 [293:12:31<135:05:54,  6.40s/it]
 66%|██████▌   | 149032/225000 [293:12:36<124:38:31,  5.91s/it]
 66%|██████▌   | 149033/225000 [293:12:39<110:07:42,  5.22s/it]
 66%|██████▌   | 149034/225000 [293:12:49<141:33:48,  6.71s/it]
 66%|██████▌   | 149035/225000 [293:12:54<126:21:57,  5.99s/it]
 66%|██████▌   | 149036/225000 [293:12:58<117:04:29,  5.55s/it]
 66%|██████▌   | 149037/225000 [293:13:03<109:48:52,  5.20s/it]
 66%|██████▌   | 149038/225000 [293:13:07<105:21:08,  4.99s/it]
 66%|██████▌   | 149039/225000 [293:13:16<128:52:30,  6.11s/it]
 66%|██████▌   | 149040/225000 [293:13:23<135:41:16,  6.43s/it]
 66%|██████▌   | 149041/225000 [293:13:29<131:00:20,  6.21s/it]
 66%|██████▌   | 149042/225000 [293:13:36<137:05:43,  6.50s/it]
 66%|██████▌   | 149043/225000 [293:13:41<125:41:24,  5.96s/it]
 66%|██████▌   | 149044/225000 [293:13:45<117:50:00,  5.58s/it]
 66%|██████▌   | 149045/225000 [293:13:50<110:29:37,  5.24s/it]
 66%|██████▌   | 149046/225000 [293:13:54<105:58:12,  5.02s/it]
 66%|██████▌   | 149047/225000 [293:13:59<105:04:40,  4.98s/it]
 66%|██████▌   | 149048/225000 [293:14:07<121:13:06,  5.75s/it]
 66%|██████▌   | 149049/225000 [293:14:17<151:34:17,  7.18s/it]
 66%|██████▌   | 149050/225000 [293:14:21<128:02:07,  6.07s/it]
                                                               
{'loss': '0.787', 'grad_norm': '1.118', 'learning_rate': '1.39e-05', 'epoch': '0.6624', 'num_input_tokens_seen': 486009344, 'train_runtime': '1.056e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▌   | 149050/225000 [293:14:21<128:02:07,  6.07s/it]
 66%|██████▌   | 149051/225000 [293:14:28<134:09:47,  6.36s/it]
 66%|██████▌   | 149052/225000 [293:14:33<126:15:50,  5.99s/it]
 66%|██████▌   | 149053/225000 [293:14:38<119:01:23,  5.64s/it]
 66%|██████▌   | 149054/225000 [293:14:43<116:43:50,  5.53s/it]
 66%|██████▌   | 149055/225000 [293:14:48<115:27:35,  5.47s/it]
 66%|██████▌   | 149056/225000 [293:14:53<113:07:54,  5.36s/it]
 66%|██████▌   | 149057/225000 [293:14:59<114:40:44,  5.44s/it]
 66%|██████▌   | 149058/225000 [293:15:05<115:39:02,  5.48s/it]
 66%|██████▌   | 149059/225000 [293:15:10<114:05:10,  5.41s/it]
 66%|██████▌   | 149060/225000 [293:15:15<114:46:18,  5.44s/it]
 66%|██████▌   | 149061/225000 [293:15:21<114:59:09,  5.45s/it]
 66%|██████▌   | 149062/225000 [293:15:26<114:26:16,  5.43s/it]
 66%|██████▋   | 149063/225000 [293:15:32<115:56:47,  5.50s/it]
 66%|██████▋   | 149064/225000 [293:15:37<116:12:55,  5.51s/it]
 66%|██████▋   | 149065/225000 [293:15:43<115:53:09,  5.49s/it]
 66%|██████▋   | 149066/225000 [293:15:48<113:57:50,  5.40s/it]
 66%|██████▋   | 149067/225000 [293:15:54<114:48:17,  5.44s/it]
 66%|██████▋   | 149068/225000 [293:15:59<113:28:38,  5.38s/it]
 66%|██████▋   | 149069/225000 [293:16:04<111:18:55,  5.28s/it]
 66%|██████▋   | 149070/225000 [293:16:14<139:33:45,  6.62s/it]
 66%|██████▋   | 149071/225000 [293:16:20<135:57:16,  6.45s/it]
 66%|██████▋   | 149072/225000 [293:16:28<148:19:23,  7.03s/it]
 66%|██████▋   | 149073/225000 [293:16:32<131:22:36,  6.23s/it]
 66%|██████▋   | 149074/225000 [293:16:37<120:57:44,  5.74s/it]
 66%|██████▋   | 149075/225000 [293:16:41<112:58:13,  5.36s/it]
 66%|██████▋   | 149076/225000 [293:16:46<108:54:27,  5.16s/it]
 66%|██████▋   | 149077/225000 [293:16:53<118:41:41,  5.63s/it]
 66%|██████▋   | 149078/225000 [293:17:05<160:24:33,  7.61s/it]
 66%|██████▋   | 149079/225000 [293:17:08<132:35:11,  6.29s/it]
 66%|██████▋   | 149080/225000 [293:17:16<140:16:36,  6.65s/it]
 66%|██████▋   | 149081/225000 [293:17:19<118:57:55,  5.64s/it]
 66%|██████▋   | 149082/225000 [293:17:26<127:06:38,  6.03s/it]
 66%|██████▋   | 149083/225000 [293:17:29<110:43:07,  5.25s/it]
 66%|██████▋   | 149084/225000 [293:17:37<123:19:55,  5.85s/it]
 66%|██████▋   | 149085/225000 [293:17:42<119:52:41,  5.68s/it]
 66%|██████▋   | 149086/225000 [293:17:46<112:28:23,  5.33s/it]
 66%|██████▋   | 149087/225000 [293:17:51<107:23:37,  5.09s/it]
 66%|██████▋   | 149088/225000 [293:17:56<104:22:45,  4.95s/it]
 66%|██████▋   | 149089/225000 [293:18:01<104:17:48,  4.95s/it]
 66%|██████▋   | 149090/225000 [293:18:13<150:17:32,  7.13s/it]
 66%|██████▋   | 149091/225000 [293:18:17<131:05:51,  6.22s/it]
 66%|██████▋   | 149092/225000 [293:18:21<118:02:15,  5.60s/it]
 66%|██████▋   | 149093/225000 [293:18:25<110:38:50,  5.25s/it]
 66%|██████▋   | 149094/225000 [293:18:30<104:56:58,  4.98s/it]
 66%|██████▋   | 149095/225000 [293:18:39<134:32:29,  6.38s/it]
 66%|██████▋   | 149096/225000 [293:18:46<137:15:12,  6.51s/it]
 66%|██████▋   | 149097/225000 [293:18:50<118:02:00,  5.60s/it]
 66%|██████▋   | 149098/225000 [293:18:53<105:17:30,  4.99s/it]
 66%|██████▋   | 149099/225000 [293:18:58<105:00:13,  4.98s/it]
 66%|██████▋   | 149100/225000 [293:19:03<106:38:13,  5.06s/it]
                                                               
{'loss': '0.7819', 'grad_norm': '1.128', 'learning_rate': '1.388e-05', 'epoch': '0.6627', 'num_input_tokens_seen': 486139960, 'train_runtime': '1.056e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149100/225000 [293:19:03<106:38:13,  5.06s/it]
 66%|██████▋   | 149101/225000 [293:19:09<107:24:52,  5.09s/it]
 66%|██████▋   | 149102/225000 [293:19:13<105:44:38,  5.02s/it]
 66%|██████▋   | 149103/225000 [293:19:18<104:32:36,  4.96s/it]
 66%|██████▋   | 149104/225000 [293:19:23<101:32:55,  4.82s/it]
 66%|██████▋   | 149105/225000 [293:19:28<105:06:39,  4.99s/it]
 66%|██████▋   | 149106/225000 [293:19:33<105:53:05,  5.02s/it]
 66%|██████▋   | 149107/225000 [293:19:39<109:41:52,  5.20s/it]
 66%|██████▋   | 149108/225000 [293:19:44<110:26:13,  5.24s/it]
 66%|██████▋   | 149109/225000 [293:19:49<110:24:03,  5.24s/it]
 66%|██████▋   | 149110/225000 [293:19:55<110:44:22,  5.25s/it]
 66%|██████▋   | 149111/225000 [293:20:04<137:03:55,  6.50s/it]
 66%|██████▋   | 149112/225000 [293:20:15<164:14:34,  7.79s/it]
 66%|██████▋   | 149113/225000 [293:20:18<135:20:24,  6.42s/it]
 66%|██████▋   | 149114/225000 [293:20:26<145:43:12,  6.91s/it]
 66%|██████▋   | 149115/225000 [293:20:30<128:05:07,  6.08s/it]
 66%|██████▋   | 149116/225000 [293:20:34<114:46:21,  5.44s/it]
 66%|██████▋   | 149117/225000 [293:20:38<106:20:04,  5.04s/it]
 66%|██████▋   | 149118/225000 [293:20:49<141:38:30,  6.72s/it]
 66%|██████▋   | 149119/225000 [293:20:52<119:40:04,  5.68s/it]
 66%|██████▋   | 149120/225000 [293:21:00<131:27:58,  6.24s/it]
 66%|██████▋   | 149121/225000 [293:21:04<115:19:13,  5.47s/it]
 66%|██████▋   | 149122/225000 [293:21:11<129:54:54,  6.16s/it]
 66%|██████▋   | 149123/225000 [293:21:16<122:25:12,  5.81s/it]
 66%|██████▋   | 149124/225000 [293:21:24<137:17:41,  6.51s/it]
 66%|██████▋   | 149125/225000 [293:21:29<124:47:56,  5.92s/it]
 66%|██████▋   | 149126/225000 [293:21:34<116:44:31,  5.54s/it]
 66%|██████▋   | 149127/225000 [293:21:38<107:34:44,  5.10s/it]
 66%|██████▋   | 149128/225000 [293:21:42<103:45:05,  4.92s/it]
 66%|██████▋   | 149129/225000 [293:21:47<102:00:44,  4.84s/it]
 66%|██████▋   | 149130/225000 [293:21:52<101:16:52,  4.81s/it]
 66%|██████▋   | 149131/225000 [293:21:59<116:12:25,  5.51s/it]
 66%|██████▋   | 149132/225000 [293:22:10<149:01:25,  7.07s/it]
 66%|██████▋   | 149133/225000 [293:22:13<126:58:53,  6.03s/it]
 66%|██████▋   | 149134/225000 [293:22:19<125:58:05,  5.98s/it]
 66%|██████▋   | 149135/225000 [293:22:26<130:08:16,  6.18s/it]
 66%|██████▋   | 149136/225000 [293:22:30<116:25:15,  5.52s/it]
 66%|██████▋   | 149137/225000 [293:22:39<137:47:35,  6.54s/it]
 66%|██████▋   | 149138/225000 [293:22:43<127:00:35,  6.03s/it]
 66%|██████▋   | 149139/225000 [293:22:48<119:34:51,  5.67s/it]
 66%|██████▋   | 149140/225000 [293:22:53<112:43:33,  5.35s/it]
 66%|██████▋   | 149141/225000 [293:22:58<109:45:13,  5.21s/it]
 66%|██████▋   | 149142/225000 [293:23:02<107:01:21,  5.08s/it]
 66%|██████▋   | 149143/225000 [293:23:07<104:30:19,  4.96s/it]
 66%|██████▋   | 149144/225000 [293:23:12<102:24:29,  4.86s/it]
 66%|██████▋   | 149145/225000 [293:23:17<102:46:31,  4.88s/it]
 66%|██████▋   | 149146/225000 [293:23:21<100:56:56,  4.79s/it]
 66%|██████▋   | 149147/225000 [293:23:29<118:31:33,  5.63s/it]
 66%|██████▋   | 149148/225000 [293:23:41<161:35:09,  7.67s/it]
 66%|██████▋   | 149149/225000 [293:23:44<133:02:47,  6.31s/it]
 66%|██████▋   | 149150/225000 [293:23:51<136:06:14,  6.46s/it]
                                                               
{'loss': '0.779', 'grad_norm': '1.005', 'learning_rate': '1.387e-05', 'epoch': '0.6629', 'num_input_tokens_seen': 486269216, 'train_runtime': '1.056e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149150/225000 [293:23:51<136:06:14,  6.46s/it]
 66%|██████▋   | 149151/225000 [293:23:57<131:07:40,  6.22s/it]
 66%|██████▋   | 149152/225000 [293:24:04<135:15:38,  6.42s/it]
 66%|██████▋   | 149153/225000 [293:24:08<121:51:46,  5.78s/it]
 66%|██████▋   | 149154/225000 [293:24:12<109:24:24,  5.19s/it]
 66%|██████▋   | 149155/225000 [293:24:16<102:25:35,  4.86s/it]
 66%|██████▋   | 149156/225000 [293:24:20<98:51:54,  4.69s/it] 
 66%|██████▋   | 149157/225000 [293:24:31<137:48:09,  6.54s/it]
 66%|██████▋   | 149158/225000 [293:24:35<122:37:13,  5.82s/it]
 66%|██████▋   | 149159/225000 [293:24:39<112:02:11,  5.32s/it]
 66%|██████▋   | 149160/225000 [293:24:44<104:59:06,  4.98s/it]
 66%|██████▋   | 149161/225000 [293:24:55<142:19:40,  6.76s/it]
 66%|██████▋   | 149162/225000 [293:24:58<120:35:06,  5.72s/it]
 66%|██████▋   | 149163/225000 [293:25:03<118:21:52,  5.62s/it]
 66%|██████▋   | 149164/225000 [293:25:08<116:15:34,  5.52s/it]
 66%|██████▋   | 149165/225000 [293:25:14<114:28:23,  5.43s/it]
 66%|██████▋   | 149166/225000 [293:25:19<114:23:45,  5.43s/it]
 66%|██████▋   | 149167/225000 [293:25:24<113:06:47,  5.37s/it]
 66%|██████▋   | 149168/225000 [293:25:30<113:49:33,  5.40s/it]
 66%|██████▋   | 149169/225000 [293:25:35<113:54:34,  5.41s/it]
 66%|██████▋   | 149170/225000 [293:25:40<111:59:22,  5.32s/it]
 66%|██████▋   | 149171/225000 [293:25:46<112:48:17,  5.36s/it]
 66%|██████▋   | 149172/225000 [293:25:51<111:40:23,  5.30s/it]
 66%|██████▋   | 149173/225000 [293:25:56<112:33:43,  5.34s/it]
 66%|██████▋   | 149174/225000 [293:26:02<112:44:35,  5.35s/it]
 66%|██████▋   | 149175/225000 [293:26:07<108:40:50,  5.16s/it]
 66%|██████▋   | 149176/225000 [293:26:12<109:07:14,  5.18s/it]
 66%|██████▋   | 149177/225000 [293:26:17<111:05:52,  5.27s/it]
 66%|██████▋   | 149178/225000 [293:26:23<113:54:54,  5.41s/it]
 66%|██████▋   | 149179/225000 [293:26:28<111:04:02,  5.27s/it]
 66%|██████▋   | 149180/225000 [293:26:33<109:43:16,  5.21s/it]
 66%|██████▋   | 149181/225000 [293:26:38<111:20:41,  5.29s/it]
 66%|██████▋   | 149182/225000 [293:26:50<151:22:57,  7.19s/it]
 66%|██████▋   | 149183/225000 [293:26:59<160:32:52,  7.62s/it]
 66%|██████▋   | 149184/225000 [293:27:03<137:42:44,  6.54s/it]
 66%|██████▋   | 149185/225000 [293:27:06<119:43:49,  5.69s/it]
 66%|██████▋   | 149186/225000 [293:27:16<141:24:46,  6.71s/it]
 66%|██████▋   | 149187/225000 [293:27:19<121:58:38,  5.79s/it]
 66%|██████▋   | 149188/225000 [293:27:27<131:40:17,  6.25s/it]
 66%|██████▋   | 149189/225000 [293:27:33<131:52:32,  6.26s/it]
 66%|██████▋   | 149190/225000 [293:27:36<114:12:10,  5.42s/it]
 66%|██████▋   | 149191/225000 [293:27:44<131:24:03,  6.24s/it]
 66%|██████▋   | 149192/225000 [293:27:48<113:04:07,  5.37s/it]
 66%|██████▋   | 149193/225000 [293:27:55<127:00:52,  6.03s/it]
 66%|██████▋   | 149194/225000 [293:27:59<110:15:44,  5.24s/it]
 66%|██████▋   | 149195/225000 [293:28:07<130:05:31,  6.18s/it]
 66%|██████▋   | 149196/225000 [293:28:12<124:51:12,  5.93s/it]
 66%|██████▋   | 149197/225000 [293:28:17<118:02:00,  5.61s/it]
 66%|██████▋   | 149198/225000 [293:28:21<105:39:44,  5.02s/it]
 66%|██████▋   | 149199/225000 [293:28:25<101:45:30,  4.83s/it]
 66%|██████▋   | 149200/225000 [293:28:29<97:22:36,  4.62s/it] 
                                                              
{'loss': '0.7757', 'grad_norm': '1.083', 'learning_rate': '1.385e-05', 'epoch': '0.6631', 'num_input_tokens_seen': 486398968, 'train_runtime': '1.057e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149200/225000 [293:28:29<97:22:36,  4.62s/it]
 66%|██████▋   | 149201/225000 [293:28:40<135:55:06,  6.46s/it]
 66%|██████▋   | 149202/225000 [293:28:46<130:27:17,  6.20s/it]
 66%|██████▋   | 149203/225000 [293:28:53<137:40:57,  6.54s/it]
 66%|██████▋   | 149204/225000 [293:28:57<118:46:06,  5.64s/it]
 66%|██████▋   | 149205/225000 [293:29:01<108:57:57,  5.18s/it]
 66%|██████▋   | 149206/225000 [293:29:06<109:26:25,  5.20s/it]
 66%|██████▋   | 149207/225000 [293:29:11<108:13:45,  5.14s/it]
 66%|██████▋   | 149208/225000 [293:29:16<107:41:53,  5.12s/it]
 66%|██████▋   | 149209/225000 [293:29:21<109:26:11,  5.20s/it]
 66%|██████▋   | 149210/225000 [293:29:27<110:30:06,  5.25s/it]
 66%|██████▋   | 149211/225000 [293:29:32<109:13:33,  5.19s/it]
 66%|██████▋   | 149212/225000 [293:29:37<111:03:32,  5.28s/it]
 66%|██████▋   | 149213/225000 [293:29:42<108:32:24,  5.16s/it]
 66%|██████▋   | 149214/225000 [293:29:47<107:46:51,  5.12s/it]
 66%|██████▋   | 149215/225000 [293:29:52<106:36:55,  5.06s/it]
 66%|██████▋   | 149216/225000 [293:29:57<104:28:01,  4.96s/it]
 66%|██████▋   | 149217/225000 [293:30:02<103:29:18,  4.92s/it]
 66%|██████▋   | 149218/225000 [293:30:09<120:22:02,  5.72s/it]
 66%|██████▋   | 149219/225000 [293:30:21<157:28:58,  7.48s/it]
 66%|██████▋   | 149220/225000 [293:30:25<133:18:00,  6.33s/it]
 66%|██████▋   | 149221/225000 [293:30:29<120:34:15,  5.73s/it]
 66%|██████▋   | 149222/225000 [293:30:33<111:39:43,  5.30s/it]
 66%|██████▋   | 149223/225000 [293:30:37<104:00:13,  4.94s/it]
 66%|██████▋   | 149224/225000 [293:30:49<149:13:04,  7.09s/it]
 66%|██████▋   | 149225/225000 [293:30:54<135:22:06,  6.43s/it]
 66%|██████▋   | 149226/225000 [293:30:58<118:57:57,  5.65s/it]
 66%|██████▋   | 149227/225000 [293:31:03<111:46:05,  5.31s/it]
 66%|██████▋   | 149228/225000 [293:31:09<120:07:42,  5.71s/it]
 66%|██████▋   | 149229/225000 [293:31:19<146:45:37,  6.97s/it]
 66%|██████▋   | 149230/225000 [293:31:23<128:10:53,  6.09s/it]
 66%|██████▋   | 149231/225000 [293:31:27<115:55:16,  5.51s/it]
 66%|██████▋   | 149232/225000 [293:31:35<130:51:19,  6.22s/it]
 66%|██████▋   | 149233/225000 [293:31:39<115:27:10,  5.49s/it]
 66%|██████▋   | 149234/225000 [293:31:44<113:14:05,  5.38s/it]
 66%|██████▋   | 149235/225000 [293:31:49<112:11:44,  5.33s/it]
 66%|██████▋   | 149236/225000 [293:31:55<111:53:41,  5.32s/it]
 66%|██████▋   | 149237/225000 [293:32:00<114:34:00,  5.44s/it]
 66%|██████▋   | 149238/225000 [293:32:06<114:29:28,  5.44s/it]
 66%|██████▋   | 149239/225000 [293:32:13<126:41:26,  6.02s/it]
 66%|██████▋   | 149240/225000 [293:32:22<144:47:07,  6.88s/it]
 66%|██████▋   | 149241/225000 [293:32:26<124:46:02,  5.93s/it]
 66%|██████▋   | 149242/225000 [293:32:33<133:34:44,  6.35s/it]
 66%|██████▋   | 149243/225000 [293:32:39<132:10:25,  6.28s/it]
 66%|██████▋   | 149244/225000 [293:32:44<119:58:55,  5.70s/it]
 66%|██████▋   | 149245/225000 [293:32:48<111:53:27,  5.32s/it]
 66%|██████▋   | 149246/225000 [293:32:52<103:53:37,  4.94s/it]
 66%|██████▋   | 149247/225000 [293:32:57<105:32:09,  5.02s/it]
 66%|██████▋   | 149248/225000 [293:33:09<145:55:08,  6.93s/it]
 66%|██████▋   | 149249/225000 [293:33:12<121:48:36,  5.79s/it]
 66%|██████▋   | 149250/225000 [293:33:16<112:44:11,  5.36s/it]
                                                               
{'loss': '0.7881', 'grad_norm': '1.136', 'learning_rate': '1.384e-05', 'epoch': '0.6633', 'num_input_tokens_seen': 486528360, 'train_runtime': '1.057e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149250/225000 [293:33:16<112:44:11,  5.36s/it]
 66%|██████▋   | 149251/225000 [293:33:25<136:02:29,  6.47s/it]
 66%|██████▋   | 149252/225000 [293:33:29<118:26:46,  5.63s/it]
 66%|██████▋   | 149253/225000 [293:33:37<131:13:27,  6.24s/it]
 66%|██████▋   | 149254/225000 [293:33:41<120:58:14,  5.75s/it]
 66%|██████▋   | 149255/225000 [293:33:46<116:04:07,  5.52s/it]
 66%|██████▋   | 149256/225000 [293:33:51<112:14:49,  5.33s/it]
 66%|██████▋   | 149257/225000 [293:33:56<109:58:07,  5.23s/it]
 66%|██████▋   | 149258/225000 [293:34:01<108:45:55,  5.17s/it]
 66%|██████▋   | 149259/225000 [293:34:06<105:08:27,  5.00s/it]
 66%|██████▋   | 149260/225000 [293:34:11<105:59:14,  5.04s/it]
 66%|██████▋   | 149261/225000 [293:34:15<103:23:41,  4.91s/it]
 66%|██████▋   | 149262/225000 [293:34:25<130:19:54,  6.19s/it]
 66%|██████▋   | 149263/225000 [293:34:35<154:01:31,  7.32s/it]
 66%|██████▋   | 149264/225000 [293:34:39<136:34:49,  6.49s/it]
 66%|██████▋   | 149265/225000 [293:34:44<126:59:46,  6.04s/it]
 66%|██████▋   | 149266/225000 [293:34:49<122:09:20,  5.81s/it]
 66%|██████▋   | 149267/225000 [293:34:54<118:01:45,  5.61s/it]
 66%|██████▋   | 149268/225000 [293:34:59<113:04:03,  5.37s/it]
 66%|██████▋   | 149269/225000 [293:35:04<111:38:49,  5.31s/it]
 66%|██████▋   | 149270/225000 [293:35:09<105:52:12,  5.03s/it]
 66%|██████▋   | 149271/225000 [293:35:21<152:42:35,  7.26s/it]
 66%|██████▋   | 149272/225000 [293:35:26<136:54:21,  6.51s/it]
 66%|██████▋   | 149273/225000 [293:35:30<121:10:42,  5.76s/it]
 66%|██████▋   | 149274/225000 [293:35:37<126:40:17,  6.02s/it]
 66%|██████▋   | 149275/225000 [293:35:44<137:05:13,  6.52s/it]
 66%|██████▋   | 149276/225000 [293:35:48<119:11:03,  5.67s/it]
 66%|██████▋   | 149277/225000 [293:35:52<106:58:37,  5.09s/it]
 66%|██████▋   | 149278/225000 [293:36:01<130:32:14,  6.21s/it]
 66%|██████▋   | 149279/225000 [293:36:04<111:45:15,  5.31s/it]
 66%|██████▋   | 149280/225000 [293:36:11<124:14:55,  5.91s/it]
 66%|██████▋   | 149281/225000 [293:36:14<107:18:17,  5.10s/it]
 66%|██████▋   | 149282/225000 [293:36:22<123:47:34,  5.89s/it]
 66%|██████▋   | 149283/225000 [293:36:26<111:10:43,  5.29s/it]
 66%|██████▋   | 149284/225000 [293:36:30<105:24:12,  5.01s/it]
 66%|██████▋   | 149285/225000 [293:36:34<99:48:26,  4.75s/it] 
 66%|██████▋   | 149286/225000 [293:36:42<119:40:15,  5.69s/it]
 66%|██████▋   | 149287/225000 [293:36:51<138:56:20,  6.61s/it]
 66%|██████▋   | 149288/225000 [293:36:56<125:49:11,  5.98s/it]
 66%|██████▋   | 149289/225000 [293:37:00<113:59:48,  5.42s/it]
 66%|██████▋   | 149290/225000 [293:37:05<109:57:33,  5.23s/it]
 66%|██████▋   | 149291/225000 [293:37:09<105:03:41,  5.00s/it]
 66%|██████▋   | 149292/225000 [293:37:15<108:58:20,  5.18s/it]
 66%|██████▋   | 149293/225000 [293:37:26<148:29:15,  7.06s/it]
 66%|██████▋   | 149294/225000 [293:37:29<124:02:36,  5.90s/it]
 66%|██████▋   | 149295/225000 [293:37:34<119:45:03,  5.69s/it]
 66%|██████▋   | 149296/225000 [293:37:39<114:35:14,  5.45s/it]
 66%|██████▋   | 149297/225000 [293:37:44<112:59:31,  5.37s/it]
 66%|██████▋   | 149298/225000 [293:37:50<111:30:50,  5.30s/it]
 66%|██████▋   | 149299/225000 [293:37:54<106:43:51,  5.08s/it]
 66%|██████▋   | 149300/225000 [293:37:59<106:21:07,  5.06s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '1.062', 'learning_rate': '1.382e-05', 'epoch': '0.6636', 'num_input_tokens_seen': 486659856, 'train_runtime': '1.057e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149300/225000 [293:37:59<106:21:07,  5.06s/it]
 66%|██████▋   | 149301/225000 [293:38:04<103:53:58,  4.94s/it]
 66%|██████▋   | 149302/225000 [293:38:09<103:26:36,  4.92s/it]
 66%|██████▋   | 149303/225000 [293:38:14<104:25:29,  4.97s/it]
 66%|██████▋   | 149304/225000 [293:38:19<103:41:54,  4.93s/it]
 66%|██████▋   | 149305/225000 [293:38:25<111:57:57,  5.33s/it]
 66%|██████▋   | 149306/225000 [293:38:37<156:20:50,  7.44s/it]
 66%|██████▋   | 149307/225000 [293:38:41<135:18:04,  6.44s/it]
 66%|██████▋   | 149308/225000 [293:38:50<146:01:20,  6.94s/it]
 66%|██████▋   | 149309/225000 [293:38:54<129:15:25,  6.15s/it]
 66%|██████▋   | 149310/225000 [293:38:59<124:41:18,  5.93s/it]
 66%|██████▋   | 149311/225000 [293:39:04<119:40:56,  5.69s/it]
 66%|██████▋   | 149312/225000 [293:39:09<114:08:14,  5.43s/it]
 66%|██████▋   | 149313/225000 [293:39:14<111:40:31,  5.31s/it]
 66%|██████▋   | 149314/225000 [293:39:19<109:15:51,  5.20s/it]
 66%|██████▋   | 149315/225000 [293:39:24<109:20:41,  5.20s/it]
 66%|██████▋   | 149316/225000 [293:39:30<110:08:51,  5.24s/it]
 66%|██████▋   | 149317/225000 [293:39:35<110:49:26,  5.27s/it]
 66%|██████▋   | 149318/225000 [293:39:40<111:25:09,  5.30s/it]
 66%|██████▋   | 149319/225000 [293:39:46<112:33:05,  5.35s/it]
 66%|██████▋   | 149320/225000 [293:39:51<108:20:01,  5.15s/it]
 66%|██████▋   | 149321/225000 [293:39:56<109:13:17,  5.20s/it]
 66%|██████▋   | 149322/225000 [293:40:07<148:59:19,  7.09s/it]
 66%|██████▋   | 149323/225000 [293:40:15<150:54:58,  7.18s/it]
 66%|██████▋   | 149324/225000 [293:40:19<132:54:45,  6.32s/it]
 66%|██████▋   | 149325/225000 [293:40:23<120:40:00,  5.74s/it]
 66%|██████▋   | 149326/225000 [293:40:28<110:47:16,  5.27s/it]
 66%|██████▋   | 149327/225000 [293:40:32<105:37:07,  5.02s/it]
 66%|██████▋   | 149328/225000 [293:40:45<157:10:19,  7.48s/it]
 66%|██████▋   | 149329/225000 [293:40:49<131:17:13,  6.25s/it]
 66%|██████▋   | 149330/225000 [293:40:53<121:00:09,  5.76s/it]
 66%|██████▋   | 149331/225000 [293:41:02<139:11:33,  6.62s/it]
 66%|██████▋   | 149332/225000 [293:41:05<119:33:15,  5.69s/it]
 66%|██████▋   | 149333/225000 [293:41:13<133:33:35,  6.35s/it]
 66%|██████▋   | 149334/225000 [293:41:18<126:00:36,  6.00s/it]
 66%|██████▋   | 149335/225000 [293:41:24<120:05:05,  5.71s/it]
 66%|██████▋   | 149336/225000 [293:41:28<112:19:22,  5.34s/it]
 66%|██████▋   | 149337/225000 [293:41:33<109:01:38,  5.19s/it]
 66%|██████▋   | 149338/225000 [293:41:38<106:14:38,  5.06s/it]
 66%|██████▋   | 149339/225000 [293:41:43<108:03:48,  5.14s/it]
 66%|██████▋   | 149340/225000 [293:41:48<106:39:55,  5.08s/it]
 66%|██████▋   | 149341/225000 [293:41:53<104:13:15,  4.96s/it]
 66%|██████▋   | 149342/225000 [293:41:57<104:10:30,  4.96s/it]
 66%|██████▋   | 149343/225000 [293:42:10<150:47:42,  7.18s/it]
 66%|██████▋   | 149344/225000 [293:42:15<139:47:48,  6.65s/it]
 66%|██████▋   | 149345/225000 [293:42:19<124:14:37,  5.91s/it]
 66%|██████▋   | 149346/225000 [293:42:23<109:39:30,  5.22s/it]
 66%|██████▋   | 149347/225000 [293:42:29<113:53:19,  5.42s/it]
 66%|██████▋   | 149348/225000 [293:42:39<140:30:02,  6.69s/it]
 66%|██████▋   | 149349/225000 [293:42:43<127:46:44,  6.08s/it]
 66%|██████▋   | 149350/225000 [293:42:48<118:02:25,  5.62s/it]
                                                               
{'loss': '0.7925', 'grad_norm': '1.093', 'learning_rate': '1.38e-05', 'epoch': '0.6638', 'num_input_tokens_seen': 486790864, 'train_runtime': '1.057e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149350/225000 [293:42:48<118:02:25,  5.62s/it]
 66%|██████▋   | 149351/225000 [293:42:52<111:00:01,  5.28s/it]
 66%|██████▋   | 149352/225000 [293:42:59<120:12:19,  5.72s/it]
 66%|██████▋   | 149353/225000 [293:43:09<148:01:30,  7.04s/it]
 66%|██████▋   | 149354/225000 [293:43:13<127:35:07,  6.07s/it]
 66%|██████▋   | 149355/225000 [293:43:17<112:10:09,  5.34s/it]
 66%|██████▋   | 149356/225000 [293:43:26<139:56:07,  6.66s/it]
 66%|██████▋   | 149357/225000 [293:43:30<123:08:01,  5.86s/it]
 66%|██████▋   | 149358/225000 [293:43:34<109:28:44,  5.21s/it]
 66%|██████▋   | 149359/225000 [293:43:39<106:12:05,  5.05s/it]
 66%|██████▋   | 149360/225000 [293:43:44<105:12:16,  5.01s/it]
 66%|██████▋   | 149361/225000 [293:43:48<104:07:55,  4.96s/it]
 66%|██████▋   | 149362/225000 [293:43:54<106:24:18,  5.06s/it]
 66%|██████▋   | 149363/225000 [293:43:59<106:33:51,  5.07s/it]
 66%|██████▋   | 149364/225000 [293:44:09<139:32:12,  6.64s/it]
 66%|██████▋   | 149365/225000 [293:44:16<140:46:13,  6.70s/it]
 66%|██████▋   | 149366/225000 [293:44:19<120:15:33,  5.72s/it]
 66%|██████▋   | 149367/225000 [293:44:25<118:58:13,  5.66s/it]
 66%|██████▋   | 149368/225000 [293:44:30<117:25:55,  5.59s/it]
 66%|██████▋   | 149369/225000 [293:44:36<114:54:01,  5.47s/it]
 66%|██████▋   | 149370/225000 [293:44:41<115:09:23,  5.48s/it]
 66%|██████▋   | 149371/225000 [293:44:52<149:04:16,  7.10s/it]
 66%|██████▋   | 149372/225000 [293:44:57<139:16:44,  6.63s/it]
 66%|██████▋   | 149373/225000 [293:45:02<126:23:28,  6.02s/it]
 66%|██████▋   | 149374/225000 [293:45:07<116:26:26,  5.54s/it]
 66%|██████▋   | 149375/225000 [293:45:12<113:00:39,  5.38s/it]
 66%|██████▋   | 149376/225000 [293:45:18<117:55:54,  5.61s/it]
 66%|██████▋   | 149377/225000 [293:45:26<134:53:07,  6.42s/it]
 66%|██████▋   | 149378/225000 [293:45:30<117:39:34,  5.60s/it]
 66%|██████▋   | 149379/225000 [293:45:36<121:13:24,  5.77s/it]
 66%|██████▋   | 149380/225000 [293:45:44<137:42:51,  6.56s/it]
 66%|██████▋   | 149381/225000 [293:45:48<118:32:52,  5.64s/it]
 66%|██████▋   | 149382/225000 [293:45:52<111:37:08,  5.31s/it]
 66%|██████▋   | 149383/225000 [293:45:58<111:27:17,  5.31s/it]
 66%|██████▋   | 149384/225000 [293:46:03<109:43:46,  5.22s/it]
 66%|██████▋   | 149385/225000 [293:46:08<110:22:10,  5.25s/it]
 66%|██████▋   | 149386/225000 [293:46:13<110:36:31,  5.27s/it]
 66%|██████▋   | 149387/225000 [293:46:18<108:29:37,  5.17s/it]
 66%|██████▋   | 149388/225000 [293:46:24<110:43:29,  5.27s/it]
 66%|██████▋   | 149389/225000 [293:46:29<109:41:59,  5.22s/it]
 66%|██████▋   | 149390/225000 [293:46:38<134:42:00,  6.41s/it]
 66%|██████▋   | 149391/225000 [293:46:42<122:07:32,  5.81s/it]
 66%|██████▋   | 149392/225000 [293:46:47<112:50:05,  5.37s/it]
 66%|██████▋   | 149393/225000 [293:46:51<105:31:01,  5.02s/it]
 66%|██████▋   | 149394/225000 [293:47:02<145:46:46,  6.94s/it]
 66%|██████▋   | 149395/225000 [293:47:07<134:02:14,  6.38s/it]
 66%|██████▋   | 149396/225000 [293:47:13<131:13:08,  6.25s/it]
 66%|██████▋   | 149397/225000 [293:47:17<115:10:37,  5.48s/it]
 66%|██████▋   | 149398/225000 [293:47:25<130:40:16,  6.22s/it]
 66%|██████▋   | 149399/225000 [293:47:29<116:09:59,  5.53s/it]
 66%|██████▋   | 149400/225000 [293:47:33<105:51:56,  5.04s/it]
                                                               
{'loss': '0.7861', 'grad_norm': '1.043', 'learning_rate': '1.379e-05', 'epoch': '0.664', 'num_input_tokens_seen': 486921184, 'train_runtime': '1.058e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149400/225000 [293:47:33<105:51:56,  5.04s/it]
 66%|██████▋   | 149401/225000 [293:47:39<115:24:11,  5.50s/it]
 66%|██████▋   | 149402/225000 [293:47:47<129:08:10,  6.15s/it]
 66%|██████▋   | 149403/225000 [293:47:50<110:41:16,  5.27s/it]
 66%|██████▋   | 149404/225000 [293:47:58<123:13:56,  5.87s/it]
 66%|██████▋   | 149405/225000 [293:48:02<115:58:15,  5.52s/it]
 66%|██████▋   | 149406/225000 [293:48:07<108:59:54,  5.19s/it]
 66%|██████▋   | 149407/225000 [293:48:12<107:16:59,  5.11s/it]
 66%|██████▋   | 149408/225000 [293:48:16<101:51:56,  4.85s/it]
 66%|██████▋   | 149409/225000 [293:48:21<102:39:43,  4.89s/it]
 66%|██████▋   | 149410/225000 [293:48:31<133:04:05,  6.34s/it]
 66%|██████▋   | 149411/225000 [293:48:37<132:35:06,  6.31s/it]
 66%|██████▋   | 149412/225000 [293:48:40<115:00:12,  5.48s/it]
 66%|██████▋   | 149413/225000 [293:48:49<135:47:07,  6.47s/it]
 66%|██████▋   | 149414/225000 [293:48:55<134:22:29,  6.40s/it]
 66%|██████▋   | 149415/225000 [293:48:59<116:28:55,  5.55s/it]
 66%|██████▋   | 149416/225000 [293:49:07<131:48:31,  6.28s/it]
 66%|██████▋   | 149417/225000 [293:49:11<117:50:18,  5.61s/it]
 66%|██████▋   | 149418/225000 [293:49:15<106:54:04,  5.09s/it]
 66%|██████▋   | 149419/225000 [293:49:19<102:07:27,  4.86s/it]
 66%|██████▋   | 149420/225000 [293:49:24<101:32:48,  4.84s/it]
 66%|██████▋   | 149421/225000 [293:49:29<100:58:00,  4.81s/it]
 66%|██████▋   | 149422/225000 [293:49:41<148:28:25,  7.07s/it]
 66%|██████▋   | 149423/225000 [293:49:44<125:49:39,  5.99s/it]
 66%|██████▋   | 149424/225000 [293:49:50<120:08:37,  5.72s/it]
 66%|██████▋   | 149425/225000 [293:49:57<132:14:09,  6.30s/it]
 66%|██████▋   | 149426/225000 [293:50:01<113:39:38,  5.41s/it]
 66%|██████▋   | 149427/225000 [293:50:10<135:56:25,  6.48s/it]
 66%|██████▋   | 149428/225000 [293:50:13<119:05:36,  5.67s/it]
 66%|██████▋   | 149429/225000 [293:50:17<105:56:19,  5.05s/it]
 66%|██████▋   | 149430/225000 [293:50:21<100:52:22,  4.81s/it]
 66%|██████▋   | 149431/225000 [293:50:30<127:55:37,  6.09s/it]
 66%|██████▋   | 149432/225000 [293:50:36<122:38:06,  5.84s/it]
 66%|██████▋   | 149433/225000 [293:50:42<123:39:10,  5.89s/it]
 66%|██████▋   | 149434/225000 [293:50:47<118:58:23,  5.67s/it]
 66%|██████▋   | 149435/225000 [293:50:54<127:03:00,  6.05s/it]
 66%|██████▋   | 149436/225000 [293:50:57<110:03:08,  5.24s/it]
 66%|██████▋   | 149437/225000 [293:51:05<126:36:02,  6.03s/it]
 66%|██████▋   | 149438/225000 [293:51:09<112:35:49,  5.36s/it]
 66%|██████▋   | 149439/225000 [293:51:13<103:31:12,  4.93s/it]
 66%|██████▋   | 149440/225000 [293:51:17<99:49:06,  4.76s/it] 
 66%|██████▋   | 149441/225000 [293:51:24<116:40:46,  5.56s/it]
 66%|██████▋   | 149442/225000 [293:51:33<133:47:43,  6.37s/it]
 66%|██████▋   | 149443/225000 [293:51:36<115:04:22,  5.48s/it]
 66%|██████▋   | 149444/225000 [293:51:44<127:38:15,  6.08s/it]
 66%|██████▋   | 149445/225000 [293:51:47<113:34:38,  5.41s/it]
 66%|██████▋   | 149446/225000 [293:51:57<139:48:23,  6.66s/it]
 66%|██████▋   | 149447/225000 [293:52:02<127:13:29,  6.06s/it]
 66%|██████▋   | 149448/225000 [293:52:05<110:12:59,  5.25s/it]
 66%|██████▋   | 149449/225000 [293:52:09<101:23:01,  4.83s/it]
 66%|██████▋   | 149450/225000 [293:52:15<106:49:06,  5.09s/it]
                                                               
{'loss': '0.7922', 'grad_norm': '1.113', 'learning_rate': '1.377e-05', 'epoch': '0.6642', 'num_input_tokens_seen': 487051416, 'train_runtime': '1.058e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149450/225000 [293:52:15<106:49:06,  5.09s/it]
 66%|██████▋   | 149451/225000 [293:52:23<130:29:53,  6.22s/it]
 66%|██████▋   | 149452/225000 [293:52:27<113:46:21,  5.42s/it]
 66%|██████▋   | 149453/225000 [293:52:36<139:50:34,  6.66s/it]
 66%|██████▋   | 149454/225000 [293:52:40<119:18:43,  5.69s/it]
 66%|██████▋   | 149455/225000 [293:52:44<111:32:05,  5.32s/it]
 66%|██████▋   | 149456/225000 [293:52:52<124:59:11,  5.96s/it]
 66%|██████▋   | 149457/225000 [293:52:55<107:07:45,  5.11s/it]
 66%|██████▋   | 149458/225000 [293:53:00<105:11:33,  5.01s/it]
 66%|██████▋   | 149459/225000 [293:53:05<107:06:23,  5.10s/it]
 66%|██████▋   | 149460/225000 [293:53:10<108:50:51,  5.19s/it]
 66%|██████▋   | 149461/225000 [293:53:16<110:06:24,  5.25s/it]
 66%|██████▋   | 149462/225000 [293:53:21<109:16:03,  5.21s/it]
 66%|██████▋   | 149463/225000 [293:53:26<108:15:51,  5.16s/it]
 66%|██████▋   | 149464/225000 [293:53:31<108:22:04,  5.16s/it]
 66%|██████▋   | 149465/225000 [293:53:44<156:03:00,  7.44s/it]
 66%|██████▋   | 149466/225000 [293:53:48<136:30:17,  6.51s/it]
 66%|██████▋   | 149467/225000 [293:53:56<144:12:47,  6.87s/it]
 66%|██████▋   | 149468/225000 [293:54:00<125:14:25,  5.97s/it]
 66%|██████▋   | 149469/225000 [293:54:06<127:49:48,  6.09s/it]
 66%|██████▋   | 149470/225000 [293:54:10<113:37:40,  5.42s/it]
 66%|██████▋   | 149471/225000 [293:54:19<133:30:28,  6.36s/it]
 66%|██████▋   | 149472/225000 [293:54:23<121:00:07,  5.77s/it]
 66%|██████▋   | 149473/225000 [293:54:27<111:43:52,  5.33s/it]
 66%|██████▋   | 149474/225000 [293:54:32<106:31:01,  5.08s/it]
 66%|██████▋   | 149475/225000 [293:54:36<102:46:05,  4.90s/it]
 66%|██████▋   | 149476/225000 [293:54:41<103:58:50,  4.96s/it]
 66%|██████▋   | 149477/225000 [293:54:54<150:11:05,  7.16s/it]
 66%|██████▋   | 149478/225000 [293:54:57<126:27:01,  6.03s/it]
 66%|██████▋   | 149479/225000 [293:55:01<112:28:33,  5.36s/it]
 66%|██████▋   | 149480/225000 [293:55:10<134:03:23,  6.39s/it]
 66%|██████▋   | 149481/225000 [293:55:13<115:53:59,  5.52s/it]
 66%|██████▋   | 149482/225000 [293:55:20<126:35:56,  6.04s/it]
 66%|██████▋   | 149483/225000 [293:55:24<109:07:10,  5.20s/it]
 66%|██████▋   | 149484/225000 [293:55:31<124:34:25,  5.94s/it]
 66%|██████▋   | 149485/225000 [293:55:36<117:43:20,  5.61s/it]
 66%|██████▋   | 149486/225000 [293:55:40<109:31:32,  5.22s/it]
 66%|██████▋   | 149487/225000 [293:55:45<104:37:15,  4.99s/it]
 66%|██████▋   | 149488/225000 [293:55:49<101:32:41,  4.84s/it]
 66%|██████▋   | 149489/225000 [293:55:54<100:39:39,  4.80s/it]
 66%|██████▋   | 149490/225000 [293:55:58<97:58:48,  4.67s/it] 
 66%|██████▋   | 149491/225000 [293:56:04<106:09:47,  5.06s/it]
 66%|██████▋   | 149492/225000 [293:56:16<149:58:39,  7.15s/it]
 66%|██████▋   | 149493/225000 [293:56:22<137:04:23,  6.54s/it]
 66%|██████▋   | 149494/225000 [293:56:26<126:18:39,  6.02s/it]
 66%|██████▋   | 149495/225000 [293:56:31<118:10:32,  5.63s/it]
 66%|██████▋   | 149496/225000 [293:56:37<117:30:03,  5.60s/it]
 66%|██████▋   | 149497/225000 [293:56:41<111:13:43,  5.30s/it]
 66%|██████▋   | 149498/225000 [293:56:46<108:15:45,  5.16s/it]
 66%|██████▋   | 149499/225000 [293:56:51<104:37:50,  4.99s/it]
 66%|██████▋   | 149500/225000 [293:56:56<107:12:39,  5.11s/it]
                                                               
{'loss': '0.7802', 'grad_norm': '1.176', 'learning_rate': '1.375e-05', 'epoch': '0.6644', 'num_input_tokens_seen': 487182696, 'train_runtime': '1.058e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149500/225000 [293:56:56<107:12:39,  5.11s/it]
 66%|██████▋   | 149501/225000 [293:57:01<105:39:40,  5.04s/it]
 66%|██████▋   | 149502/225000 [293:57:12<146:26:58,  6.98s/it]
 66%|██████▋   | 149503/225000 [293:57:18<135:37:03,  6.47s/it]
 66%|██████▋   | 149504/225000 [293:57:22<119:58:41,  5.72s/it]
 66%|██████▋   | 149505/225000 [293:57:28<123:17:32,  5.88s/it]
 66%|██████▋   | 149506/225000 [293:57:35<133:01:45,  6.34s/it]
 66%|██████▋   | 149507/225000 [293:57:39<116:22:00,  5.55s/it]
 66%|██████▋   | 149508/225000 [293:57:46<127:00:21,  6.06s/it]
 66%|██████▋   | 149509/225000 [293:57:50<111:53:20,  5.34s/it]
 66%|██████▋   | 149510/225000 [293:57:58<129:38:48,  6.18s/it]
 66%|██████▋   | 149511/225000 [293:58:02<115:20:17,  5.50s/it]
 66%|██████▋   | 149512/225000 [293:58:06<104:31:28,  4.98s/it]
 66%|██████▋   | 149513/225000 [293:58:17<140:51:08,  6.72s/it]
 66%|██████▋   | 149514/225000 [293:58:20<118:29:17,  5.65s/it]
 66%|██████▋   | 149515/225000 [293:58:28<134:07:01,  6.40s/it]
 66%|██████▋   | 149516/225000 [293:58:31<116:30:30,  5.56s/it]
 66%|██████▋   | 149517/225000 [293:58:38<124:41:55,  5.95s/it]
 66%|██████▋   | 149518/225000 [293:58:43<117:25:45,  5.60s/it]
 66%|██████▋   | 149519/225000 [293:58:52<136:05:52,  6.49s/it]
 66%|██████▋   | 149520/225000 [293:58:56<123:52:14,  5.91s/it]
 66%|██████▋   | 149521/225000 [293:59:01<115:38:03,  5.52s/it]
 66%|██████▋   | 149522/225000 [293:59:05<108:16:42,  5.16s/it]
 66%|██████▋   | 149523/225000 [293:59:10<106:59:00,  5.10s/it]
 66%|██████▋   | 149524/225000 [293:59:15<103:21:54,  4.93s/it]
 66%|██████▋   | 149525/225000 [293:59:20<103:04:25,  4.92s/it]
 66%|██████▋   | 149526/225000 [293:59:24<101:35:37,  4.85s/it]
 66%|██████▋   | 149527/225000 [293:59:29<100:01:14,  4.77s/it]
 66%|██████▋   | 149528/225000 [293:59:34<103:42:28,  4.95s/it]
 66%|██████▋   | 149529/225000 [293:59:47<154:16:13,  7.36s/it]
 66%|██████▋   | 149530/225000 [293:59:52<139:44:51,  6.67s/it]
 66%|██████▋   | 149531/225000 [294:00:00<144:53:53,  6.91s/it]
 66%|██████▋   | 149532/225000 [294:00:04<126:21:26,  6.03s/it]
 66%|██████▋   | 149533/225000 [294:00:12<140:14:52,  6.69s/it]
 66%|██████▋   | 149534/225000 [294:00:15<119:57:14,  5.72s/it]
 66%|██████▋   | 149535/225000 [294:00:20<113:18:24,  5.41s/it]
 66%|██████▋   | 149536/225000 [294:00:28<128:23:26,  6.12s/it]
 66%|██████▋   | 149537/225000 [294:00:31<112:55:51,  5.39s/it]
 66%|██████▋   | 149538/225000 [294:00:36<107:20:10,  5.12s/it]
 66%|██████▋   | 149539/225000 [294:00:40<103:18:20,  4.93s/it]
 66%|██████▋   | 149540/225000 [294:00:45<99:56:40,  4.77s/it] 
 66%|██████▋   | 149541/225000 [294:00:50<100:47:48,  4.81s/it]
 66%|██████▋   | 149542/225000 [294:00:56<107:32:26,  5.13s/it]
 66%|██████▋   | 149543/225000 [294:01:07<149:50:02,  7.15s/it]
 66%|██████▋   | 149544/225000 [294:01:12<132:28:09,  6.32s/it]
 66%|██████▋   | 149545/225000 [294:01:19<137:19:39,  6.55s/it]
 66%|██████▋   | 149546/225000 [294:01:22<118:17:03,  5.64s/it]
 66%|██████▋   | 149547/225000 [294:01:31<133:59:41,  6.39s/it]
 66%|██████▋   | 149548/225000 [294:01:35<122:26:02,  5.84s/it]
 66%|██████▋   | 149549/225000 [294:01:40<116:03:51,  5.54s/it]
 66%|██████▋   | 149550/225000 [294:01:45<109:51:18,  5.24s/it]
                                                               
{'loss': '0.7887', 'grad_norm': '1.17', 'learning_rate': '1.374e-05', 'epoch': '0.6647', 'num_input_tokens_seen': 487313176, 'train_runtime': '1.059e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149550/225000 [294:01:45<109:51:18,  5.24s/it]
 66%|██████▋   | 149551/225000 [294:01:49<104:24:49,  4.98s/it]
 66%|██████▋   | 149552/225000 [294:01:53<101:11:55,  4.83s/it]
 66%|██████▋   | 149553/225000 [294:01:58<97:27:13,  4.65s/it] 
 66%|██████▋   | 149554/225000 [294:02:08<134:12:46,  6.40s/it]
 66%|██████▋   | 149555/225000 [294:02:14<128:09:37,  6.12s/it]
 66%|██████▋   | 149556/225000 [294:02:21<138:43:05,  6.62s/it]
 66%|██████▋   | 149557/225000 [294:02:26<123:42:13,  5.90s/it]
 66%|██████▋   | 149558/225000 [294:02:32<128:50:56,  6.15s/it]
 66%|██████▋   | 149559/225000 [294:02:37<117:22:15,  5.60s/it]
 66%|██████▋   | 149560/225000 [294:02:41<111:10:23,  5.31s/it]
 66%|██████▋   | 149561/225000 [294:02:46<106:02:55,  5.06s/it]
 66%|██████▋   | 149562/225000 [294:02:56<140:09:35,  6.69s/it]
 66%|██████▋   | 149563/225000 [294:03:00<120:44:56,  5.76s/it]
 66%|██████▋   | 149564/225000 [294:03:08<137:14:50,  6.55s/it]
 66%|██████▋   | 149565/225000 [294:03:11<116:09:19,  5.54s/it]
 66%|██████▋   | 149566/225000 [294:03:15<103:43:57,  4.95s/it]
 66%|██████▋   | 149567/225000 [294:03:23<124:33:15,  5.94s/it]
 66%|██████▋   | 149568/225000 [294:03:27<109:35:52,  5.23s/it]
 66%|██████▋   | 149569/225000 [294:03:31<102:42:07,  4.90s/it]
 66%|██████▋   | 149570/225000 [294:03:37<107:03:23,  5.11s/it]
 66%|██████▋   | 149571/225000 [294:03:42<110:29:27,  5.27s/it]
 66%|██████▋   | 149572/225000 [294:03:48<111:14:46,  5.31s/it]
 66%|██████▋   | 149573/225000 [294:03:53<109:37:39,  5.23s/it]
 66%|██████▋   | 149574/225000 [294:03:58<112:13:57,  5.36s/it]
 66%|██████▋   | 149575/225000 [294:04:08<141:49:44,  6.77s/it]
 66%|██████▋   | 149576/225000 [294:04:13<128:12:20,  6.12s/it]
 66%|██████▋   | 149577/225000 [294:04:18<124:05:43,  5.92s/it]
 66%|██████▋   | 149578/225000 [294:04:26<137:17:12,  6.55s/it]
 66%|██████▋   | 149579/225000 [294:04:31<124:43:35,  5.95s/it]
 66%|██████▋   | 149580/225000 [294:04:35<115:23:17,  5.51s/it]
 66%|██████▋   | 149581/225000 [294:04:40<106:46:04,  5.10s/it]
 66%|██████▋   | 149582/225000 [294:04:44<102:12:52,  4.88s/it]
 66%|██████▋   | 149583/225000 [294:04:55<138:45:59,  6.62s/it]
 66%|██████▋   | 149584/225000 [294:04:58<119:30:29,  5.70s/it]
 66%|██████▋   | 149585/225000 [294:05:02<109:25:33,  5.22s/it]
 66%|██████▋   | 149586/225000 [294:05:12<134:20:07,  6.41s/it]
 66%|██████▋   | 149587/225000 [294:05:15<116:36:12,  5.57s/it]
 66%|██████▋   | 149588/225000 [294:05:24<139:47:02,  6.67s/it]
 66%|██████▋   | 149589/225000 [294:05:27<116:58:24,  5.58s/it]
 66%|██████▋   | 149590/225000 [294:05:35<128:26:51,  6.13s/it]
 66%|██████▋   | 149591/225000 [294:05:39<116:20:44,  5.55s/it]
 66%|██████▋   | 149592/225000 [294:05:43<103:29:24,  4.94s/it]
 66%|██████▋   | 149593/225000 [294:05:52<131:32:03,  6.28s/it]
 66%|██████▋   | 149594/225000 [294:05:55<113:07:29,  5.40s/it]
 66%|██████▋   | 149595/225000 [294:06:03<126:35:49,  6.04s/it]
 66%|██████▋   | 149596/225000 [294:06:07<112:40:12,  5.38s/it]
 66%|██████▋   | 149597/225000 [294:06:11<104:28:56,  4.99s/it]
 66%|██████▋   | 149598/225000 [294:06:15<97:07:08,  4.64s/it] 
 66%|██████▋   | 149599/225000 [294:06:26<138:07:31,  6.59s/it]
 66%|██████▋   | 149600/225000 [294:06:30<123:41:09,  5.91s/it]
                                                               
{'loss': '0.7863', 'grad_norm': '1.216', 'learning_rate': '1.372e-05', 'epoch': '0.6649', 'num_input_tokens_seen': 487443264, 'train_runtime': '1.059e+06', 'train_tokens_per_second': '460.4'}

 66%|██████▋   | 149600/225000 [294:06:30<123:41:09,  5.91s/it]
 66%|██████▋   | 149601/225000 [294:06:34<112:38:04,  5.38s/it]
 66%|██████▋   | 149602/225000 [294:06:38<105:31:54,  5.04s/it]
 66%|██████▋   | 149603/225000 [294:06:43<102:20:15,  4.89s/it]
 66%|██████▋   | 149604/225000 [294:06:47<99:53:54,  4.77s/it] 
 66%|██████▋   | 149605/225000 [294:06:58<138:01:12,  6.59s/it]
 66%|██████▋   | 149606/225000 [294:07:02<120:54:52,  5.77s/it]
 66%|██████▋   | 149607/225000 [294:07:06<108:19:06,  5.17s/it]
 66%|██████▋   | 149608/225000 [294:07:14<125:22:17,  5.99s/it]
 66%|██████▋   | 149609/225000 [294:07:20<123:27:40,  5.90s/it]
 66%|██████▋   | 149610/225000 [294:07:24<114:28:39,  5.47s/it]
 66%|██████▋   | 149611/225000 [294:07:29<111:19:46,  5.32s/it]
 66%|██████▋   | 149612/225000 [294:07:34<110:48:06,  5.29s/it]
 66%|██████▋   | 149613/225000 [294:07:39<109:28:46,  5.23s/it]
 66%|██████▋   | 149614/225000 [294:07:45<110:01:51,  5.25s/it]
 66%|██████▋   | 149615/225000 [294:07:57<156:37:01,  7.48s/it]
 66%|██████▋   | 149616/225000 [294:08:01<135:06:41,  6.45s/it]
 66%|██████▋   | 149617/225000 [294:08:08<139:49:12,  6.68s/it]
 66%|██████▋   | 149618/225000 [294:08:12<118:35:36,  5.66s/it]
 66%|██████▋   | 149619/225000 [294:08:16<109:26:26,  5.23s/it]
 66%|██████▋   | 149620/225000 [294:08:22<116:12:31,  5.55s/it]
 66%|██████▋   | 149621/225000 [294:08:32<142:01:54,  6.78s/it]
 66%|██████▋   | 149622/225000 [294:08:36<124:41:39,  5.96s/it]
 66%|██████▋   | 149623/225000 [294:08:40<112:49:48,  5.39s/it]
 66%|██████▋   | 149624/225000 [294:08:49<137:58:27,  6.59s/it]
 66%|██████▋   | 149625/225000 [294:08:53<120:14:57,  5.74s/it]
 67%|██████▋   | 149626/225000 [294:08:57<107:41:05,  5.14s/it]
 67%|██████▋   | 149627/225000 [294:09:07<135:46:47,  6.49s/it]
 67%|██████▋   | 149628/225000 [294:09:10<117:20:22,  5.60s/it]
 67%|██████▋   | 149629/225000 [294:09:17<122:29:32,  5.85s/it]
 67%|██████▋   | 149630/225000 [294:09:22<118:17:16,  5.65s/it]
 67%|██████▋   | 149631/225000 [294:09:25<105:46:55,  5.05s/it]
 67%|██████▋   | 149632/225000 [294:09:35<136:33:34,  6.52s/it]
 67%|██████▋   | 149633/225000 [294:09:40<122:35:17,  5.86s/it]
 67%|██████▋   | 149634/225000 [294:09:44<114:49:18,  5.48s/it]
 67%|██████▋   | 149635/225000 [294:09:49<109:50:54,  5.25s/it]
 67%|██████▋   | 149636/225000 [294:09:53<104:27:39,  4.99s/it]
 67%|██████▋   | 149637/225000 [294:09:58<102:09:09,  4.88s/it]
 67%|██████▋   | 149638/225000 [294:10:03<101:50:43,  4.87s/it]
 67%|██████▋   | 149639/225000 [294:10:07<99:56:40,  4.77s/it] 
 67%|██████▋   | 149640/225000 [294:10:17<130:32:18,  6.24s/it]
 67%|██████▋   | 149641/225000 [294:10:24<135:33:52,  6.48s/it]
 67%|██████▋   | 149642/225000 [294:10:28<119:10:45,  5.69s/it]
 67%|██████▋   | 149643/225000 [294:10:33<113:46:39,  5.44s/it]
 67%|██████▋   | 149644/225000 [294:10:37<108:28:38,  5.18s/it]
 67%|██████▋   | 149645/225000 [294:10:42<105:32:04,  5.04s/it]
 67%|██████▋   | 149646/225000 [294:10:47<102:55:10,  4.92s/it]
 67%|██████▋   | 149647/225000 [294:10:52<105:27:04,  5.04s/it]
 67%|██████▋   | 149648/225000 [294:10:57<107:00:15,  5.11s/it]
 67%|██████▋   | 149649/225000 [294:11:08<144:34:32,  6.91s/it]
 67%|██████▋   | 149650/225000 [294:11:14<137:07:40,  6.55s/it]
                                                               
{'loss': '0.7786', 'grad_norm': '1.109', 'learning_rate': '1.371e-05', 'epoch': '0.6651', 'num_input_tokens_seen': 487573776, 'train_runtime': '1.059e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149650/225000 [294:11:14<137:07:40,  6.55s/it]
 67%|██████▋   | 149651/225000 [294:11:19<129:35:46,  6.19s/it]
 67%|██████▋   | 149652/225000 [294:11:24<121:31:18,  5.81s/it]
 67%|██████▋   | 149653/225000 [294:11:30<117:31:30,  5.62s/it]
 67%|██████▋   | 149654/225000 [294:11:35<115:50:27,  5.53s/it]
 67%|██████▋   | 149655/225000 [294:11:40<114:31:37,  5.47s/it]
 67%|██████▋   | 149656/225000 [294:11:45<111:09:39,  5.31s/it]
 67%|██████▋   | 149657/225000 [294:11:50<107:05:45,  5.12s/it]
 67%|██████▋   | 149658/225000 [294:11:55<106:35:44,  5.09s/it]
 67%|██████▋   | 149659/225000 [294:12:06<146:58:28,  7.02s/it]
 67%|██████▋   | 149660/225000 [294:12:12<136:43:25,  6.53s/it]
 67%|██████▋   | 149661/225000 [294:12:17<128:39:39,  6.15s/it]
 67%|██████▋   | 149662/225000 [294:12:22<121:02:12,  5.78s/it]
 67%|██████▋   | 149663/225000 [294:12:27<118:49:11,  5.68s/it]
 67%|██████▋   | 149664/225000 [294:12:33<116:03:44,  5.55s/it]
 67%|██████▋   | 149665/225000 [294:12:38<114:01:21,  5.45s/it]
 67%|██████▋   | 149666/225000 [294:12:43<111:53:07,  5.35s/it]
 67%|██████▋   | 149667/225000 [294:12:48<111:14:35,  5.32s/it]
 67%|██████▋   | 149668/225000 [294:12:57<132:46:23,  6.35s/it]
 67%|██████▋   | 149669/225000 [294:13:01<119:54:51,  5.73s/it]
 67%|██████▋   | 149670/225000 [294:13:05<110:00:07,  5.26s/it]
 67%|██████▋   | 149671/225000 [294:13:14<132:57:56,  6.35s/it]
 67%|██████▋   | 149672/225000 [294:13:21<134:06:17,  6.41s/it]
 67%|██████▋   | 149673/225000 [294:13:25<121:39:39,  5.81s/it]
 67%|██████▋   | 149674/225000 [294:13:30<116:00:47,  5.54s/it]
 67%|██████▋   | 149675/225000 [294:13:35<112:04:25,  5.36s/it]
 67%|██████▋   | 149676/225000 [294:13:40<110:59:27,  5.30s/it]
 67%|██████▋   | 149677/225000 [294:13:45<107:39:53,  5.15s/it]
 67%|██████▋   | 149678/225000 [294:13:50<107:39:19,  5.15s/it]
 67%|██████▋   | 149679/225000 [294:13:55<104:54:53,  5.01s/it]
 67%|██████▋   | 149680/225000 [294:14:00<105:15:39,  5.03s/it]
 67%|██████▋   | 149681/225000 [294:14:05<106:58:54,  5.11s/it]
 67%|██████▋   | 149682/225000 [294:14:11<108:55:47,  5.21s/it]
 67%|██████▋   | 149683/225000 [294:14:16<112:30:07,  5.38s/it]
 67%|██████▋   | 149684/225000 [294:14:29<154:56:42,  7.41s/it]
 67%|██████▋   | 149685/225000 [294:14:32<131:45:33,  6.30s/it]
 67%|██████▋   | 149686/225000 [294:14:37<120:20:32,  5.75s/it]
 67%|██████▋   | 149687/225000 [294:14:45<136:26:43,  6.52s/it]
 67%|██████▋   | 149688/225000 [294:14:52<140:18:35,  6.71s/it]
 67%|██████▋   | 149689/225000 [294:14:57<125:24:56,  6.00s/it]
 67%|██████▋   | 149690/225000 [294:15:03<125:03:52,  5.98s/it]
 67%|██████▋   | 149691/225000 [294:15:06<108:59:10,  5.21s/it]
 67%|██████▋   | 149692/225000 [294:15:15<135:09:51,  6.46s/it]
 67%|██████▋   | 149693/225000 [294:15:19<116:16:16,  5.56s/it]
 67%|██████▋   | 149694/225000 [294:15:23<106:07:43,  5.07s/it]
 67%|██████▋   | 149695/225000 [294:15:31<128:40:18,  6.15s/it]
 67%|██████▋   | 149696/225000 [294:15:35<112:44:23,  5.39s/it]
 67%|██████▋   | 149697/225000 [294:15:44<134:19:10,  6.42s/it]
 67%|██████▋   | 149698/225000 [294:15:49<124:25:13,  5.95s/it]
 67%|██████▋   | 149699/225000 [294:15:53<114:50:43,  5.49s/it]
 67%|██████▋   | 149700/225000 [294:15:58<110:40:15,  5.29s/it]
                                                               
{'loss': '0.7893', 'grad_norm': '1.008', 'learning_rate': '1.369e-05', 'epoch': '0.6653', 'num_input_tokens_seen': 487703960, 'train_runtime': '1.059e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149700/225000 [294:15:58<110:40:15,  5.29s/it]
 67%|██████▋   | 149701/225000 [294:16:02<104:47:44,  5.01s/it]
 67%|██████▋   | 149702/225000 [294:16:07<101:09:53,  4.84s/it]
 67%|██████▋   | 149703/225000 [294:16:11<100:38:41,  4.81s/it]
 67%|██████▋   | 149704/225000 [294:16:21<129:41:03,  6.20s/it]
 67%|██████▋   | 149705/225000 [294:16:29<139:24:36,  6.67s/it]
 67%|██████▋   | 149706/225000 [294:16:34<128:36:18,  6.15s/it]
 67%|██████▋   | 149707/225000 [294:16:39<123:29:32,  5.90s/it]
 67%|██████▋   | 149708/225000 [294:16:45<121:43:46,  5.82s/it]
 67%|██████▋   | 149709/225000 [294:16:50<120:35:29,  5.77s/it]
 67%|██████▋   | 149710/225000 [294:16:55<114:10:19,  5.46s/it]
 67%|██████▋   | 149711/225000 [294:17:01<115:26:09,  5.52s/it]
 67%|██████▋   | 149712/225000 [294:17:06<115:23:39,  5.52s/it]
 67%|██████▋   | 149713/225000 [294:17:11<114:13:33,  5.46s/it]
 67%|██████▋   | 149714/225000 [294:17:21<139:29:54,  6.67s/it]
 67%|██████▋   | 149715/225000 [294:17:24<117:52:31,  5.64s/it]
 67%|██████▋   | 149716/225000 [294:17:28<108:22:57,  5.18s/it]
 67%|██████▋   | 149717/225000 [294:17:32<100:48:07,  4.82s/it]
 67%|██████▋   | 149718/225000 [294:17:42<133:18:05,  6.37s/it]
 67%|██████▋   | 149719/225000 [294:17:48<128:09:07,  6.13s/it]
 67%|██████▋   | 149720/225000 [294:17:52<114:26:53,  5.47s/it]
 67%|██████▋   | 149721/225000 [294:17:56<104:19:00,  4.99s/it]
 67%|██████▋   | 149722/225000 [294:18:06<138:57:29,  6.65s/it]
 67%|██████▋   | 149723/225000 [294:18:10<119:47:44,  5.73s/it]
 67%|██████▋   | 149724/225000 [294:18:14<110:55:59,  5.31s/it]
 67%|██████▋   | 149725/225000 [294:18:19<108:34:36,  5.19s/it]
 67%|██████▋   | 149726/225000 [294:18:24<106:24:05,  5.09s/it]
 67%|██████▋   | 149727/225000 [294:18:28<102:12:05,  4.89s/it]
 67%|██████▋   | 149728/225000 [294:18:34<107:28:20,  5.14s/it]
 67%|██████▋   | 149729/225000 [294:18:45<147:19:47,  7.05s/it]
 67%|██████▋   | 149730/225000 [294:18:49<123:23:14,  5.90s/it]
 67%|██████▋   | 149731/225000 [294:18:54<117:15:48,  5.61s/it]
 67%|██████▋   | 149732/225000 [294:19:01<128:53:00,  6.16s/it]
 67%|██████▋   | 149733/225000 [294:19:05<113:32:42,  5.43s/it]
 67%|██████▋   | 149734/225000 [294:19:09<103:50:36,  4.97s/it]
 67%|██████▋   | 149735/225000 [294:19:13<97:06:41,  4.64s/it] 
 67%|██████▋   | 149736/225000 [294:19:24<138:11:50,  6.61s/it]
 67%|██████▋   | 149737/225000 [294:19:30<134:59:22,  6.46s/it]
 67%|██████▋   | 149738/225000 [294:19:33<115:44:05,  5.54s/it]
 67%|██████▋   | 149739/225000 [294:19:42<136:39:50,  6.54s/it]
 67%|██████▋   | 149740/225000 [294:19:46<119:07:17,  5.70s/it]
 67%|██████▋   | 149741/225000 [294:19:50<107:33:00,  5.14s/it]
 67%|██████▋   | 149742/225000 [294:19:53<98:52:31,  4.73s/it] 
 67%|██████▋   | 149743/225000 [294:20:04<135:21:58,  6.48s/it]
 67%|██████▋   | 149744/225000 [294:20:08<119:21:18,  5.71s/it]
 67%|██████▋   | 149745/225000 [294:20:15<125:03:42,  5.98s/it]
 67%|██████▋   | 149746/225000 [294:20:20<120:02:34,  5.74s/it]
 67%|██████▋   | 149747/225000 [294:20:23<105:32:36,  5.05s/it]
 67%|██████▋   | 149748/225000 [294:20:28<106:15:38,  5.08s/it]
 67%|██████▋   | 149749/225000 [294:20:36<120:20:34,  5.76s/it]
 67%|██████▋   | 149750/225000 [294:20:41<114:37:21,  5.48s/it]
                                                               
{'loss': '0.7776', 'grad_norm': '1.13', 'learning_rate': '1.367e-05', 'epoch': '0.6656', 'num_input_tokens_seen': 487834496, 'train_runtime': '1.06e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149750/225000 [294:20:41<114:37:21,  5.48s/it]
 67%|██████▋   | 149751/225000 [294:20:46<114:35:31,  5.48s/it]
 67%|██████▋   | 149752/225000 [294:20:51<110:00:41,  5.26s/it]
 67%|██████▋   | 149753/225000 [294:20:56<109:01:03,  5.22s/it]
 67%|██████▋   | 149754/225000 [294:21:01<108:48:11,  5.21s/it]
 67%|██████▋   | 149755/225000 [294:21:06<108:47:12,  5.20s/it]
 67%|██████▋   | 149756/225000 [294:21:12<109:29:44,  5.24s/it]
 67%|██████▋   | 149757/225000 [294:21:17<108:32:50,  5.19s/it]
 67%|██████▋   | 149758/225000 [294:21:22<107:56:15,  5.16s/it]
 67%|██████▋   | 149759/225000 [294:21:27<110:08:15,  5.27s/it]
 67%|██████▋   | 149760/225000 [294:21:32<107:30:19,  5.14s/it]
 67%|██████▋   | 149761/225000 [294:21:45<154:52:10,  7.41s/it]
 67%|██████▋   | 149762/225000 [294:21:50<140:18:35,  6.71s/it]
 67%|██████▋   | 149763/225000 [294:21:54<121:40:19,  5.82s/it]
 67%|██████▋   | 149764/225000 [294:21:59<119:05:59,  5.70s/it]
 67%|██████▋   | 149765/225000 [294:22:08<136:46:17,  6.54s/it]
 67%|██████▋   | 149766/225000 [294:22:12<121:30:33,  5.81s/it]
 67%|██████▋   | 149767/225000 [294:22:16<110:50:26,  5.30s/it]
 67%|██████▋   | 149768/225000 [294:22:20<102:39:16,  4.91s/it]
 67%|██████▋   | 149769/225000 [294:22:27<119:09:11,  5.70s/it]
 67%|██████▋   | 149770/225000 [294:22:37<141:15:57,  6.76s/it]
 67%|██████▋   | 149771/225000 [294:22:40<119:47:59,  5.73s/it]
 67%|██████▋   | 149772/225000 [294:22:47<131:28:25,  6.29s/it]
 67%|██████▋   | 149773/225000 [294:22:51<115:16:29,  5.52s/it]
 67%|██████▋   | 149774/225000 [294:22:58<124:20:11,  5.95s/it]
 67%|██████▋   | 149775/225000 [294:23:04<124:59:32,  5.98s/it]
 67%|██████▋   | 149776/225000 [294:23:08<109:07:34,  5.22s/it]
 67%|██████▋   | 149777/225000 [294:23:12<100:50:27,  4.83s/it]
 67%|██████▋   | 149778/225000 [294:23:16<98:08:52,  4.70s/it] 
 67%|██████▋   | 149779/225000 [294:23:25<127:43:55,  6.11s/it]
 67%|██████▋   | 149780/225000 [294:23:33<136:47:16,  6.55s/it]
 67%|██████▋   | 149781/225000 [294:23:37<121:25:24,  5.81s/it]
 67%|██████▋   | 149782/225000 [294:23:45<132:21:57,  6.34s/it]
 67%|██████▋   | 149783/225000 [294:23:50<128:42:44,  6.16s/it]
 67%|██████▋   | 149784/225000 [294:23:55<118:52:55,  5.69s/it]
 67%|██████▋   | 149785/225000 [294:23:59<106:11:38,  5.08s/it]
 67%|██████▋   | 149786/225000 [294:24:07<127:54:05,  6.12s/it]
 67%|██████▋   | 149787/225000 [294:24:11<116:09:52,  5.56s/it]
 67%|██████▋   | 149788/225000 [294:24:16<109:08:50,  5.22s/it]
 67%|██████▋   | 149789/225000 [294:24:21<106:03:11,  5.08s/it]
 67%|██████▋   | 149790/225000 [294:24:25<103:09:03,  4.94s/it]
 67%|██████▋   | 149791/225000 [294:24:30<102:30:35,  4.91s/it]
 67%|██████▋   | 149792/225000 [294:24:41<143:29:53,  6.87s/it]
 67%|██████▋   | 149793/225000 [294:24:45<121:53:07,  5.83s/it]
 67%|██████▋   | 149794/225000 [294:24:55<147:25:16,  7.06s/it]
 67%|██████▋   | 149795/225000 [294:24:58<125:38:43,  6.01s/it]
 67%|██████▋   | 149796/225000 [294:25:05<129:00:58,  6.18s/it]
 67%|██████▋   | 149797/225000 [294:25:11<130:03:10,  6.23s/it]
 67%|██████▋   | 149798/225000 [294:25:16<123:10:00,  5.90s/it]
 67%|██████▋   | 149799/225000 [294:25:22<118:49:19,  5.69s/it]
 67%|██████▋   | 149800/225000 [294:25:26<107:51:15,  5.16s/it]
                                                               
{'loss': '0.7864', 'grad_norm': '1.154', 'learning_rate': '1.366e-05', 'epoch': '0.6658', 'num_input_tokens_seen': 487965344, 'train_runtime': '1.06e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149800/225000 [294:25:26<107:51:15,  5.16s/it]
 67%|██████▋   | 149801/225000 [294:25:30<100:23:58,  4.81s/it]
 67%|██████▋   | 149802/225000 [294:25:34<96:16:02,  4.61s/it] 
 67%|██████▋   | 149803/225000 [294:25:46<143:56:07,  6.89s/it]
 67%|██████▋   | 149804/225000 [294:25:50<123:52:33,  5.93s/it]
 67%|██████▋   | 149805/225000 [294:25:56<128:04:54,  6.13s/it]
 67%|██████▋   | 149806/225000 [294:26:01<117:09:41,  5.61s/it]
 67%|██████▋   | 149807/225000 [294:26:05<108:48:44,  5.21s/it]
 67%|██████▋   | 149808/225000 [294:26:09<101:38:00,  4.87s/it]
 67%|██████▋   | 149809/225000 [294:26:13<98:27:12,  4.71s/it] 
 67%|██████▋   | 149810/225000 [294:26:24<137:06:00,  6.56s/it]
 67%|██████▋   | 149811/225000 [294:26:27<115:02:23,  5.51s/it]
 67%|██████▋   | 149812/225000 [294:26:35<129:13:30,  6.19s/it]
 67%|██████▋   | 149813/225000 [294:26:39<117:20:45,  5.62s/it]
 67%|██████▋   | 149814/225000 [294:26:44<110:06:28,  5.27s/it]
 67%|██████▋   | 149815/225000 [294:26:48<102:27:27,  4.91s/it]
 67%|██████▋   | 149816/225000 [294:26:53<101:59:45,  4.88s/it]
 67%|██████▋   | 149817/225000 [294:26:57<98:07:38,  4.70s/it] 
 67%|██████▋   | 149818/225000 [294:27:02<97:52:34,  4.69s/it]
 67%|██████▋   | 149819/225000 [294:27:06<97:08:00,  4.65s/it]
 67%|██████▋   | 149820/225000 [294:27:12<103:14:17,  4.94s/it]
 67%|██████▋   | 149821/225000 [294:27:23<145:24:26,  6.96s/it]
 67%|██████▋   | 149822/225000 [294:27:27<124:58:22,  5.98s/it]
 67%|██████▋   | 149823/225000 [294:27:31<109:33:53,  5.25s/it]
 67%|██████▋   | 149824/225000 [294:27:35<102:19:12,  4.90s/it]
 67%|██████▋   | 149825/225000 [294:27:44<129:53:52,  6.22s/it]
 67%|██████▋   | 149826/225000 [294:27:51<133:51:11,  6.41s/it]
 67%|██████▋   | 149827/225000 [294:27:55<122:05:32,  5.85s/it]
 67%|██████▋   | 149828/225000 [294:28:00<114:37:25,  5.49s/it]
 67%|██████▋   | 149829/225000 [294:28:05<110:08:36,  5.27s/it]
 67%|██████▋   | 149830/225000 [294:28:09<104:43:12,  5.02s/it]
 67%|██████▋   | 149831/225000 [294:28:14<103:13:42,  4.94s/it]
 67%|██████▋   | 149832/225000 [294:28:21<113:27:49,  5.43s/it]
 67%|██████▋   | 149833/225000 [294:28:31<145:03:51,  6.95s/it]
 67%|██████▋   | 149834/225000 [294:28:34<122:33:47,  5.87s/it]
 67%|██████▋   | 149835/225000 [294:28:43<138:05:44,  6.61s/it]
 67%|██████▋   | 149836/225000 [294:28:46<117:18:31,  5.62s/it]
 67%|██████▋   | 149837/225000 [294:28:53<126:01:25,  6.04s/it]
 67%|██████▋   | 149838/225000 [294:28:57<109:55:20,  5.26s/it]
 67%|██████▋   | 149839/225000 [294:29:05<132:15:04,  6.33s/it]
 67%|██████▋   | 149840/225000 [294:29:09<113:39:26,  5.44s/it]
 67%|██████▋   | 149841/225000 [294:29:17<133:05:45,  6.38s/it]
 67%|██████▋   | 149842/225000 [294:29:21<116:47:46,  5.59s/it]
 67%|██████▋   | 149843/225000 [294:29:25<105:48:21,  5.07s/it]
 67%|██████▋   | 149844/225000 [294:29:34<128:24:05,  6.15s/it]
 67%|██████▋   | 149845/225000 [294:29:37<110:56:22,  5.31s/it]
 67%|██████▋   | 149846/225000 [294:29:41<101:56:31,  4.88s/it]
 67%|██████▋   | 149847/225000 [294:29:49<123:10:40,  5.90s/it]
 67%|██████▋   | 149848/225000 [294:29:53<111:36:10,  5.35s/it]
 67%|██████▋   | 149849/225000 [294:29:58<106:38:49,  5.11s/it]
 67%|██████▋   | 149850/225000 [294:30:02<101:06:15,  4.84s/it]
                                                               
{'loss': '0.7909', 'grad_norm': '1.016', 'learning_rate': '1.364e-05', 'epoch': '0.666', 'num_input_tokens_seen': 488095368, 'train_runtime': '1.06e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149850/225000 [294:30:02<101:06:15,  4.84s/it]
 67%|██████▋   | 149851/225000 [294:30:06<97:23:39,  4.67s/it] 
 67%|██████▋   | 149852/225000 [294:30:19<146:45:22,  7.03s/it]
 67%|██████▋   | 149853/225000 [294:30:22<123:35:12,  5.92s/it]
 67%|██████▋   | 149854/225000 [294:30:27<116:21:28,  5.57s/it]
 67%|██████▋   | 149855/225000 [294:30:32<112:12:32,  5.38s/it]
 67%|██████▋   | 149856/225000 [294:30:37<112:27:27,  5.39s/it]
 67%|██████▋   | 149857/225000 [294:30:42<109:57:02,  5.27s/it]
 67%|██████▋   | 149858/225000 [294:30:47<109:48:22,  5.26s/it]
 67%|██████▋   | 149859/225000 [294:30:53<110:52:51,  5.31s/it]
 67%|██████▋   | 149860/225000 [294:30:58<107:29:30,  5.15s/it]
 67%|██████▋   | 149861/225000 [294:31:08<138:00:13,  6.61s/it]
 67%|██████▋   | 149862/225000 [294:31:12<122:59:02,  5.89s/it]
 67%|██████▋   | 149863/225000 [294:31:16<111:40:11,  5.35s/it]
 67%|██████▋   | 149864/225000 [294:31:21<111:42:59,  5.35s/it]
 67%|██████▋   | 149865/225000 [294:31:31<141:55:40,  6.80s/it]
 67%|██████▋   | 149866/225000 [294:31:35<119:44:48,  5.74s/it]
 67%|██████▋   | 149867/225000 [294:31:42<128:53:16,  6.18s/it]
 67%|██████▋   | 149868/225000 [294:31:46<114:50:58,  5.50s/it]
 67%|██████▋   | 149869/225000 [294:31:49<102:30:12,  4.91s/it]
 67%|██████▋   | 149870/225000 [294:31:58<126:06:54,  6.04s/it]
 67%|██████▋   | 149871/225000 [294:32:02<110:09:39,  5.28s/it]
 67%|██████▋   | 149872/225000 [294:32:10<129:40:00,  6.21s/it]
 67%|██████▋   | 149873/225000 [294:32:13<112:56:58,  5.41s/it]
 67%|██████▋   | 149874/225000 [294:32:22<131:11:22,  6.29s/it]
 67%|██████▋   | 149875/225000 [294:32:26<117:18:42,  5.62s/it]
 67%|██████▋   | 149876/225000 [294:32:30<106:30:00,  5.10s/it]
 67%|██████▋   | 149877/225000 [294:32:34<100:55:24,  4.84s/it]
 67%|██████▋   | 149878/225000 [294:32:44<132:30:02,  6.35s/it]
 67%|██████▋   | 149879/225000 [294:32:47<112:15:20,  5.38s/it]
 67%|██████▋   | 149880/225000 [294:32:51<105:06:41,  5.04s/it]
 67%|██████▋   | 149881/225000 [294:32:56<102:26:41,  4.91s/it]
 67%|██████▋   | 149882/225000 [294:33:00<96:53:07,  4.64s/it] 
 67%|██████▋   | 149883/225000 [294:33:11<135:04:52,  6.47s/it]
 67%|██████▋   | 149884/225000 [294:33:14<115:14:41,  5.52s/it]
 67%|██████▋   | 149885/225000 [294:33:22<129:25:41,  6.20s/it]
 67%|██████▋   | 149886/225000 [294:33:26<117:45:21,  5.64s/it]
 67%|██████▋   | 149887/225000 [294:33:30<107:56:12,  5.17s/it]
 67%|██████▋   | 149888/225000 [294:33:34<99:34:27,  4.77s/it] 
 67%|██████▋   | 149889/225000 [294:33:46<144:29:27,  6.93s/it]
 67%|██████▋   | 149890/225000 [294:33:50<127:15:39,  6.10s/it]
 67%|██████▋   | 149891/225000 [294:33:53<108:55:48,  5.22s/it]
 67%|██████▋   | 149892/225000 [294:34:01<124:19:57,  5.96s/it]
 67%|██████▋   | 149893/225000 [294:34:05<111:13:58,  5.33s/it]
 67%|██████▋   | 149894/225000 [294:34:13<130:28:48,  6.25s/it]
 67%|██████▋   | 149895/225000 [294:34:17<112:56:21,  5.41s/it]
 67%|██████▋   | 149896/225000 [294:34:25<132:42:19,  6.36s/it]
 67%|██████▋   | 149897/225000 [294:34:29<114:39:56,  5.50s/it]
 67%|██████▋   | 149898/225000 [294:34:33<104:12:27,  5.00s/it]
 67%|██████▋   | 149899/225000 [294:34:43<137:31:40,  6.59s/it]
 67%|██████▋   | 149900/225000 [294:34:47<119:25:10,  5.72s/it]
                                                               
{'loss': '0.775', 'grad_norm': '1.074', 'learning_rate': '1.362e-05', 'epoch': '0.6662', 'num_input_tokens_seen': 488223296, 'train_runtime': '1.06e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149900/225000 [294:34:47<119:25:10,  5.72s/it]
 67%|██████▋   | 149901/225000 [294:34:50<106:08:19,  5.09s/it]
 67%|██████▋   | 149902/225000 [294:34:54<101:22:23,  4.86s/it]
 67%|██████▋   | 149903/225000 [294:35:03<123:57:31,  5.94s/it]
 67%|██████▋   | 149904/225000 [294:35:07<111:24:39,  5.34s/it]
 67%|██████▋   | 149905/225000 [294:35:11<104:01:37,  4.99s/it]
 67%|██████▋   | 149906/225000 [294:35:15<98:15:41,  4.71s/it] 
 67%|██████▋   | 149907/225000 [294:35:24<126:10:24,  6.05s/it]
 67%|██████▋   | 149908/225000 [294:35:30<126:11:31,  6.05s/it]
 67%|██████▋   | 149909/225000 [294:35:35<116:11:33,  5.57s/it]
 67%|██████▋   | 149910/225000 [294:35:39<105:58:58,  5.08s/it]
 67%|██████▋   | 149911/225000 [294:35:43<102:54:40,  4.93s/it]
 67%|██████▋   | 149912/225000 [294:35:51<120:05:31,  5.76s/it]
 67%|██████▋   | 149913/225000 [294:35:58<127:28:23,  6.11s/it]
 67%|██████▋   | 149914/225000 [294:36:02<116:37:34,  5.59s/it]
 67%|██████▋   | 149915/225000 [294:36:06<106:24:04,  5.10s/it]
 67%|██████▋   | 149916/225000 [294:36:16<135:41:20,  6.51s/it]
 67%|██████▋   | 149917/225000 [294:36:20<120:34:00,  5.78s/it]
 67%|██████▋   | 149918/225000 [294:36:25<115:54:20,  5.56s/it]
 67%|██████▋   | 149919/225000 [294:36:30<111:16:06,  5.34s/it]
 67%|██████▋   | 149920/225000 [294:36:35<110:10:19,  5.28s/it]
 67%|██████▋   | 149921/225000 [294:36:40<110:28:22,  5.30s/it]
 67%|██████▋   | 149922/225000 [294:36:46<111:35:18,  5.35s/it]
 67%|██████▋   | 149923/225000 [294:36:51<110:13:29,  5.29s/it]
 67%|██████▋   | 149924/225000 [294:36:57<111:38:30,  5.35s/it]
 67%|██████▋   | 149925/225000 [294:37:05<128:38:39,  6.17s/it]
 67%|██████▋   | 149926/225000 [294:37:13<145:03:19,  6.96s/it]
 67%|██████▋   | 149927/225000 [294:37:18<131:19:20,  6.30s/it]
 67%|██████▋   | 149928/225000 [294:37:23<122:13:57,  5.86s/it]
 67%|██████▋   | 149929/225000 [294:37:28<119:15:05,  5.72s/it]
 67%|██████▋   | 149930/225000 [294:37:33<114:50:57,  5.51s/it]
 67%|██████▋   | 149931/225000 [294:37:39<114:05:58,  5.47s/it]
 67%|██████▋   | 149932/225000 [294:37:44<113:54:37,  5.46s/it]
 67%|██████▋   | 149933/225000 [294:37:49<112:02:01,  5.37s/it]
 67%|██████▋   | 149934/225000 [294:37:55<112:26:58,  5.39s/it]
 67%|██████▋   | 149935/225000 [294:38:00<109:12:01,  5.24s/it]
 67%|██████▋   | 149936/225000 [294:38:05<108:55:37,  5.22s/it]
 67%|██████▋   | 149937/225000 [294:38:10<107:17:14,  5.15s/it]
 67%|██████▋   | 149938/225000 [294:38:16<111:44:36,  5.36s/it]
 67%|██████▋   | 149939/225000 [294:38:21<113:55:03,  5.46s/it]
 67%|██████▋   | 149940/225000 [294:38:27<114:25:16,  5.49s/it]
 67%|██████▋   | 149941/225000 [294:38:32<111:39:55,  5.36s/it]
 67%|██████▋   | 149942/225000 [294:38:39<121:59:32,  5.85s/it]
 67%|██████▋   | 149943/225000 [294:38:52<165:06:10,  7.92s/it]
 67%|██████▋   | 149944/225000 [294:38:55<135:20:36,  6.49s/it]
 67%|██████▋   | 149945/225000 [294:38:58<116:14:51,  5.58s/it]
 67%|██████▋   | 149946/225000 [294:39:06<128:01:59,  6.14s/it]
 67%|██████▋   | 149947/225000 [294:39:13<131:21:42,  6.30s/it]
 67%|██████▋   | 149948/225000 [294:39:16<114:14:30,  5.48s/it]
 67%|██████▋   | 149949/225000 [294:39:24<131:22:45,  6.30s/it]
 67%|██████▋   | 149950/225000 [294:39:28<113:32:52,  5.45s/it]
                                                               
{'loss': '0.7866', 'grad_norm': '1.232', 'learning_rate': '1.361e-05', 'epoch': '0.6664', 'num_input_tokens_seen': 488353216, 'train_runtime': '1.061e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 149950/225000 [294:39:28<113:32:52,  5.45s/it]
 67%|██████▋   | 149951/225000 [294:39:32<103:32:42,  4.97s/it]
 67%|██████▋   | 149952/225000 [294:39:40<121:55:17,  5.85s/it]
 67%|██████▋   | 149953/225000 [294:39:43<108:36:43,  5.21s/it]
 67%|██████▋   | 149954/225000 [294:39:52<131:35:41,  6.31s/it]
 67%|██████▋   | 149955/225000 [294:39:56<116:22:09,  5.58s/it]
 67%|██████▋   | 149956/225000 [294:40:00<105:30:05,  5.06s/it]
 67%|██████▋   | 149957/225000 [294:40:04<97:56:54,  4.70s/it] 
 67%|██████▋   | 149958/225000 [294:40:08<94:47:47,  4.55s/it]
 67%|██████▋   | 149959/225000 [294:40:18<131:50:51,  6.33s/it]
 67%|██████▋   | 149960/225000 [294:40:23<118:29:30,  5.68s/it]
 67%|██████▋   | 149961/225000 [294:40:28<118:30:25,  5.69s/it]
 67%|██████▋   | 149962/225000 [294:40:34<120:29:58,  5.78s/it]
 67%|██████▋   | 149963/225000 [294:40:38<109:49:48,  5.27s/it]
 67%|██████▋   | 149964/225000 [294:40:42<102:14:27,  4.91s/it]
 67%|██████▋   | 149965/225000 [294:40:47<100:46:06,  4.83s/it]
 67%|██████▋   | 149966/225000 [294:40:58<136:36:39,  6.55s/it]
 67%|██████▋   | 149967/225000 [294:41:01<116:50:27,  5.61s/it]
 67%|██████▋   | 149968/225000 [294:41:10<135:25:22,  6.50s/it]
 67%|██████▋   | 149969/225000 [294:41:13<115:47:59,  5.56s/it]
 67%|██████▋   | 149970/225000 [294:41:19<116:18:49,  5.58s/it]
 67%|██████▋   | 149971/225000 [294:41:25<122:56:30,  5.90s/it]
 67%|██████▋   | 149972/225000 [294:41:28<105:42:07,  5.07s/it]
 67%|██████▋   | 149973/225000 [294:41:33<99:33:49,  4.78s/it] 
 67%|██████▋   | 149974/225000 [294:41:37<100:23:14,  4.82s/it]
 67%|██████▋   | 149975/225000 [294:41:42<101:10:57,  4.86s/it]
 67%|██████▋   | 149976/225000 [294:41:47<101:14:41,  4.86s/it]
 67%|██████▋   | 149977/225000 [294:41:55<118:02:12,  5.66s/it]
 67%|██████▋   | 149978/225000 [294:42:03<135:43:08,  6.51s/it]
 67%|██████▋   | 149979/225000 [294:42:07<116:39:34,  5.60s/it]
 67%|██████▋   | 149980/225000 [294:42:11<111:22:24,  5.34s/it]
 67%|██████▋   | 149981/225000 [294:42:20<128:54:30,  6.19s/it]
 67%|██████▋   | 149982/225000 [294:42:27<133:35:44,  6.41s/it]
 67%|██████▋   | 149983/225000 [294:42:30<115:39:42,  5.55s/it]
 67%|██████▋   | 149984/225000 [294:42:36<116:34:27,  5.59s/it]
 67%|██████▋   | 149985/225000 [294:42:43<128:53:01,  6.19s/it]
 67%|██████▋   | 149986/225000 [294:42:48<116:49:39,  5.61s/it]
 67%|██████▋   | 149987/225000 [294:42:54<121:35:50,  5.84s/it]
 67%|██████▋   | 149988/225000 [294:42:59<115:34:44,  5.55s/it]
 67%|██████▋   | 149989/225000 [294:43:03<108:42:55,  5.22s/it]
 67%|██████▋   | 149990/225000 [294:43:08<103:24:42,  4.96s/it]
 67%|██████▋   | 149991/225000 [294:43:12<102:01:12,  4.90s/it]
 67%|██████▋   | 149992/225000 [294:43:17<99:16:27,  4.76s/it] 
 67%|██████▋   | 149993/225000 [294:43:21<98:09:05,  4.71s/it]
 67%|██████▋   | 149994/225000 [294:43:26<100:03:12,  4.80s/it]
 67%|██████▋   | 149995/225000 [294:43:31<99:48:57,  4.79s/it] 
 67%|██████▋   | 149996/225000 [294:43:42<135:16:33,  6.49s/it]
 67%|██████▋   | 149997/225000 [294:43:49<139:36:48,  6.70s/it]
 67%|██████▋   | 149998/225000 [294:43:52<119:54:03,  5.76s/it]
 67%|██████▋   | 149999/225000 [294:44:00<133:36:58,  6.41s/it]
 67%|██████▋   | 150000/225000 [294:44:06<130:49:33,  6.28s/it]
                                                               
{'loss': '0.7771', 'grad_norm': '1.111', 'learning_rate': '1.359e-05', 'epoch': '0.6667', 'num_input_tokens_seen': 488482464, 'train_runtime': '1.061e+06', 'train_tokens_per_second': '460.4'}

 67%|██████▋   | 150000/225000 [294:44:06<130:49:33,  6.28s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                               
{'eval_loss': '1.323', 'eval_runtime': '426', 'eval_samples_per_second': '1.174', 'eval_steps_per_second': '0.587', 'epoch': '0.6667', 'num_input_tokens_seen': 488482464}

 67%|██████▋   | 150000/225000 [294:51:12<130:49:33,  6.28s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-150000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 67%|██████▋   | 150001/225000 [294:51:17<2785:09:08, 133.69s/it]
 67%|██████▋   | 150002/225000 [294:51:21<1974:47:32, 94.79s/it] 
 67%|██████▋   | 150003/225000 [294:51:25<1405:36:15, 67.47s/it]
 67%|██████▋   | 150004/225000 [294:51:34<1040:09:50, 49.93s/it]
 67%|██████▋   | 150005/225000 [294:51:38<749:33:19, 35.98s/it] 
 67%|██████▋   | 150006/225000 [294:51:42<551:29:12, 26.47s/it]
 67%|██████▋   | 150007/225000 [294:51:48<424:46:47, 20.39s/it]
 67%|██████▋   | 150008/225000 [294:51:55<338:29:07, 16.25s/it]
 67%|██████▋   | 150009/225000 [294:52:00<269:32:07, 12.94s/it]
 67%|██████▋   | 150010/225000 [294:52:07<234:53:01, 11.28s/it]
 67%|██████▋   | 150011/225000 [294:52:11<185:43:47,  8.92s/it]
 67%|██████▋   | 150012/225000 [294:52:16<163:38:24,  7.86s/it]
 67%|██████▋   | 150013/225000 [294:52:21<145:57:53,  7.01s/it]
 67%|██████▋   | 150014/225000 [294:52:26<135:37:52,  6.51s/it]
 67%|██████▋   | 150015/225000 [294:52:31<126:20:42,  6.07s/it]
 67%|██████▋   | 150016/225000 [294:52:36<119:29:36,  5.74s/it]
 67%|██████▋   | 150017/225000 [294:52:41<114:42:37,  5.51s/it]
 67%|██████▋   | 150018/225000 [294:52:46<112:22:23,  5.40s/it]
 67%|██████▋   | 150019/225000 [294:52:52<110:08:59,  5.29s/it]
 67%|██████▋   | 150020/225000 [294:52:57<110:20:04,  5.30s/it]
 67%|██████▋   | 150021/225000 [294:53:02<112:28:08,  5.40s/it]
 67%|██████▋   | 150022/225000 [294:53:08<113:29:02,  5.45s/it]
 67%|██████▋   | 150023/225000 [294:53:13<109:19:58,  5.25s/it]
 67%|██████▋   | 150024/225000 [294:53:18<109:07:51,  5.24s/it]
 67%|██████▋   | 150025/225000 [294:53:23<106:48:57,  5.13s/it]
 67%|██████▋   | 150026/225000 [294:53:29<112:52:19,  5.42s/it]
 67%|██████▋   | 150027/225000 [294:53:41<152:06:34,  7.30s/it]
 67%|██████▋   | 150028/225000 [294:53:46<138:04:23,  6.63s/it]
 67%|██████▋   | 150029/225000 [294:53:54<146:57:46,  7.06s/it]
 67%|██████▋   | 150030/225000 [294:53:59<134:48:45,  6.47s/it]
 67%|██████▋   | 150031/225000 [294:54:04<123:35:39,  5.93s/it]
 67%|██████▋   | 150032/225000 [294:54:09<117:48:59,  5.66s/it]
 67%|██████▋   | 150033/225000 [294:54:14<113:03:50,  5.43s/it]
 67%|██████▋   | 150034/225000 [294:54:18<108:22:58,  5.20s/it]
 67%|██████▋   | 150035/225000 [294:54:24<111:25:35,  5.35s/it]
 67%|██████▋   | 150036/225000 [294:54:35<149:55:05,  7.20s/it]
 67%|██████▋   | 150037/225000 [294:54:39<128:45:53,  6.18s/it]
 67%|██████▋   | 150038/225000 [294:54:46<133:58:20,  6.43s/it]
 67%|██████▋   | 150039/225000 [294:54:52<132:37:27,  6.37s/it]
 67%|██████▋   | 150040/225000 [294:54:57<120:38:33,  5.79s/it]
 67%|██████▋   | 150041/225000 [294:55:02<113:41:59,  5.46s/it]
 67%|██████▋   | 150042/225000 [294:55:06<106:51:49,  5.13s/it]
 67%|██████▋   | 150043/225000 [294:55:10<102:58:54,  4.95s/it]
 67%|██████▋   | 150044/225000 [294:55:15<102:13:58,  4.91s/it]
 67%|██████▋   | 150045/225000 [294:55:25<129:05:26,  6.20s/it]
 67%|██████▋   | 150046/225000 [294:55:31<128:42:57,  6.18s/it]
 67%|██████▋   | 150047/225000 [294:55:36<123:24:11,  5.93s/it]
 67%|██████▋   | 150048/225000 [294:55:41<115:51:29,  5.56s/it]
 67%|██████▋   | 150049/225000 [294:55:46<115:44:45,  5.56s/it]
 67%|██████▋   | 150050/225000 [294:55:51<112:24:30,  5.40s/it]
                                                               
{'loss': '0.7795', 'grad_norm': '1.197', 'learning_rate': '1.358e-05', 'epoch': '0.6669', 'num_input_tokens_seen': 488613416, 'train_runtime': '1.062e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150050/225000 [294:55:51<112:24:30,  5.40s/it]
 67%|██████▋   | 150051/225000 [294:55:56<110:32:25,  5.31s/it]
 67%|██████▋   | 150052/225000 [294:56:05<129:56:21,  6.24s/it]
 67%|██████▋   | 150053/225000 [294:56:09<116:46:35,  5.61s/it]
 67%|██████▋   | 150054/225000 [294:56:17<132:33:21,  6.37s/it]
 67%|██████▋   | 150055/225000 [294:56:20<113:15:45,  5.44s/it]
 67%|██████▋   | 150056/225000 [294:56:29<134:03:17,  6.44s/it]
 67%|██████▋   | 150057/225000 [294:56:33<120:26:54,  5.79s/it]
 67%|██████▋   | 150058/225000 [294:56:38<110:54:52,  5.33s/it]
 67%|██████▋   | 150059/225000 [294:56:42<103:58:13,  4.99s/it]
 67%|██████▋   | 150060/225000 [294:56:47<104:11:03,  5.00s/it]
 67%|██████▋   | 150061/225000 [294:56:59<148:20:48,  7.13s/it]
 67%|██████▋   | 150062/225000 [294:57:03<126:24:02,  6.07s/it]
 67%|██████▋   | 150063/225000 [294:57:07<118:07:01,  5.67s/it]
 67%|██████▋   | 150064/225000 [294:57:12<111:43:19,  5.37s/it]
 67%|██████▋   | 150065/225000 [294:57:16<106:06:56,  5.10s/it]
 67%|██████▋   | 150066/225000 [294:57:28<144:20:39,  6.93s/it]
 67%|██████▋   | 150067/225000 [294:57:33<134:52:08,  6.48s/it]
 67%|██████▋   | 150068/225000 [294:57:40<138:56:33,  6.68s/it]
 67%|██████▋   | 150069/225000 [294:57:44<119:16:08,  5.73s/it]
 67%|██████▋   | 150070/225000 [294:57:47<104:33:42,  5.02s/it]
 67%|██████▋   | 150071/225000 [294:57:57<131:55:30,  6.34s/it]
 67%|██████▋   | 150072/225000 [294:58:00<114:17:34,  5.49s/it]
 67%|██████▋   | 150073/225000 [294:58:10<140:04:30,  6.73s/it]
 67%|██████▋   | 150074/225000 [294:58:13<119:36:00,  5.75s/it]
 67%|██████▋   | 150075/225000 [294:58:21<134:20:02,  6.45s/it]
 67%|██████▋   | 150076/225000 [294:58:25<117:09:29,  5.63s/it]
 67%|██████▋   | 150077/225000 [294:58:29<107:44:39,  5.18s/it]
 67%|██████▋   | 150078/225000 [294:58:33<99:45:57,  4.79s/it] 
 67%|██████▋   | 150079/225000 [294:58:37<93:27:22,  4.49s/it]
 67%|██████▋   | 150080/225000 [294:58:47<131:25:01,  6.31s/it]
 67%|██████▋   | 150081/225000 [294:58:51<116:12:54,  5.58s/it]
 67%|██████▋   | 150082/225000 [294:58:59<130:33:55,  6.27s/it]
 67%|██████▋   | 150083/225000 [294:59:03<113:45:24,  5.47s/it]
 67%|██████▋   | 150084/225000 [294:59:11<131:12:14,  6.30s/it]
 67%|██████▋   | 150085/225000 [294:59:14<112:23:43,  5.40s/it]
 67%|██████▋   | 150086/225000 [294:59:22<125:41:33,  6.04s/it]
 67%|██████▋   | 150087/225000 [294:59:27<120:11:02,  5.78s/it]
 67%|██████▋   | 150088/225000 [294:59:32<114:24:18,  5.50s/it]
 67%|██████▋   | 150089/225000 [294:59:37<113:19:31,  5.45s/it]
 67%|██████▋   | 150090/225000 [294:59:42<111:00:33,  5.33s/it]
 67%|██████▋   | 150091/225000 [294:59:47<108:04:22,  5.19s/it]
 67%|██████▋   | 150092/225000 [294:59:52<108:25:59,  5.21s/it]
 67%|██████▋   | 150093/225000 [294:59:57<108:11:43,  5.20s/it]
 67%|██████▋   | 150094/225000 [295:00:03<109:45:18,  5.27s/it]
 67%|██████▋   | 150095/225000 [295:00:08<111:17:37,  5.35s/it]
 67%|██████▋   | 150096/225000 [295:00:15<120:46:03,  5.80s/it]
 67%|██████▋   | 150097/225000 [295:00:24<142:16:59,  6.84s/it]
 67%|██████▋   | 150098/225000 [295:00:28<124:25:44,  5.98s/it]
 67%|██████▋   | 150099/225000 [295:00:36<135:13:05,  6.50s/it]
 67%|██████▋   | 150100/225000 [295:00:43<139:43:58,  6.72s/it]
                                                               
{'loss': '0.7813', 'grad_norm': '1.205', 'learning_rate': '1.356e-05', 'epoch': '0.6671', 'num_input_tokens_seen': 488745120, 'train_runtime': '1.062e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150100/225000 [295:00:43<139:43:58,  6.72s/it]
 67%|██████▋   | 150101/225000 [295:00:48<125:07:28,  6.01s/it]
 67%|██████▋   | 150102/225000 [295:00:52<113:55:16,  5.48s/it]
 67%|██████▋   | 150103/225000 [295:00:56<106:14:57,  5.11s/it]
 67%|██████▋   | 150104/225000 [295:01:01<101:06:08,  4.86s/it]
 67%|██████▋   | 150105/225000 [295:01:05<97:18:02,  4.68s/it] 
 67%|██████▋   | 150106/225000 [295:01:09<95:42:30,  4.60s/it]
 67%|██████▋   | 150107/225000 [295:01:21<141:17:55,  6.79s/it]
 67%|██████▋   | 150108/225000 [295:01:24<118:29:39,  5.70s/it]
 67%|██████▋   | 150109/225000 [295:01:31<125:50:12,  6.05s/it]
 67%|██████▋   | 150110/225000 [295:01:35<111:46:18,  5.37s/it]
 67%|██████▋   | 150111/225000 [295:01:44<136:41:39,  6.57s/it]
 67%|██████▋   | 150112/225000 [295:01:49<125:16:47,  6.02s/it]
 67%|██████▋   | 150113/225000 [295:01:52<108:52:27,  5.23s/it]
 67%|██████▋   | 150114/225000 [295:02:02<133:32:51,  6.42s/it]
 67%|██████▋   | 150115/225000 [295:02:05<116:35:05,  5.60s/it]
 67%|██████▋   | 150116/225000 [295:02:12<122:59:45,  5.91s/it]
 67%|██████▋   | 150117/225000 [295:02:17<120:11:16,  5.78s/it]
 67%|██████▋   | 150118/225000 [295:02:25<128:34:15,  6.18s/it]
 67%|██████▋   | 150119/225000 [295:02:30<121:55:56,  5.86s/it]
 67%|██████▋   | 150120/225000 [295:02:34<112:58:16,  5.43s/it]
 67%|██████▋   | 150121/225000 [295:02:38<105:16:33,  5.06s/it]
 67%|██████▋   | 150122/225000 [295:02:43<103:25:47,  4.97s/it]
 67%|██████▋   | 150123/225000 [295:02:48<101:32:06,  4.88s/it]
 67%|██████▋   | 150124/225000 [295:02:57<129:14:30,  6.21s/it]
 67%|██████▋   | 150125/225000 [295:03:02<122:46:53,  5.90s/it]
 67%|██████▋   | 150126/225000 [295:03:07<115:24:25,  5.55s/it]
 67%|██████▋   | 150127/225000 [295:03:12<111:05:25,  5.34s/it]
 67%|██████▋   | 150128/225000 [295:03:16<107:06:13,  5.15s/it]
 67%|██████▋   | 150129/225000 [295:03:21<105:43:44,  5.08s/it]
 67%|██████▋   | 150130/225000 [295:03:26<101:57:28,  4.90s/it]
 67%|██████▋   | 150131/225000 [295:03:31<102:44:06,  4.94s/it]
 67%|██████▋   | 150132/225000 [295:03:37<109:47:24,  5.28s/it]
 67%|██████▋   | 150133/225000 [295:03:49<151:32:24,  7.29s/it]
 67%|██████▋   | 150134/225000 [295:03:53<130:23:59,  6.27s/it]
 67%|██████▋   | 150135/225000 [295:04:01<139:21:13,  6.70s/it]
 67%|██████▋   | 150136/225000 [295:04:04<117:22:56,  5.64s/it]
 67%|██████▋   | 150137/225000 [295:04:12<131:31:41,  6.32s/it]
 67%|██████▋   | 150138/225000 [295:04:16<116:08:55,  5.59s/it]
 67%|██████▋   | 150139/225000 [295:04:19<104:52:38,  5.04s/it]
 67%|██████▋   | 150140/225000 [295:04:24<101:23:48,  4.88s/it]
 67%|██████▋   | 150141/225000 [295:04:34<135:53:38,  6.54s/it]
 67%|██████▋   | 150142/225000 [295:04:39<122:53:51,  5.91s/it]
 67%|██████▋   | 150143/225000 [295:04:43<112:00:26,  5.39s/it]
 67%|██████▋   | 150144/225000 [295:04:47<103:37:48,  4.98s/it]
 67%|██████▋   | 150145/225000 [295:04:56<131:31:15,  6.33s/it]
 67%|██████▋   | 150146/225000 [295:05:00<112:06:42,  5.39s/it]
 67%|██████▋   | 150147/225000 [295:05:07<127:50:55,  6.15s/it]
 67%|██████▋   | 150148/225000 [295:05:11<111:52:03,  5.38s/it]
 67%|██████▋   | 150149/225000 [295:05:19<125:44:51,  6.05s/it]
 67%|██████▋   | 150150/225000 [295:05:24<118:24:52,  5.70s/it]
                                                               
{'loss': '0.7945', 'grad_norm': '1.066', 'learning_rate': '1.354e-05', 'epoch': '0.6673', 'num_input_tokens_seen': 488875688, 'train_runtime': '1.062e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150150/225000 [295:05:24<118:24:52,  5.70s/it]
 67%|██████▋   | 150151/225000 [295:05:28<113:22:42,  5.45s/it]
 67%|██████▋   | 150152/225000 [295:05:33<109:27:22,  5.26s/it]
 67%|██████▋   | 150153/225000 [295:05:38<107:28:43,  5.17s/it]
 67%|██████▋   | 150154/225000 [295:05:43<104:05:42,  5.01s/it]
 67%|██████▋   | 150155/225000 [295:05:47<100:53:51,  4.85s/it]
 67%|██████▋   | 150156/225000 [295:05:53<103:25:06,  4.97s/it]
 67%|██████▋   | 150157/225000 [295:05:57<101:59:38,  4.91s/it]
 67%|██████▋   | 150158/225000 [295:06:02<101:21:11,  4.88s/it]
 67%|██████▋   | 150159/225000 [295:06:08<109:07:06,  5.25s/it]
 67%|██████▋   | 150160/225000 [295:06:21<154:25:51,  7.43s/it]
 67%|██████▋   | 150161/225000 [295:06:26<140:41:27,  6.77s/it]
 67%|██████▋   | 150162/225000 [295:06:29<120:20:45,  5.79s/it]
 67%|██████▋   | 150163/225000 [295:06:37<128:31:43,  6.18s/it]
 67%|██████▋   | 150164/225000 [295:06:42<123:33:00,  5.94s/it]
 67%|██████▋   | 150165/225000 [295:06:46<111:12:09,  5.35s/it]
 67%|██████▋   | 150166/225000 [295:06:50<103:36:39,  4.98s/it]
 67%|██████▋   | 150167/225000 [295:06:54<97:42:46,  4.70s/it] 
 67%|██████▋   | 150168/225000 [295:06:59<96:44:22,  4.65s/it]
 67%|██████▋   | 150169/225000 [295:07:11<144:53:55,  6.97s/it]
 67%|██████▋   | 150170/225000 [295:07:16<130:07:41,  6.26s/it]
 67%|██████▋   | 150171/225000 [295:07:20<115:55:15,  5.58s/it]
 67%|██████▋   | 150172/225000 [295:07:24<106:47:59,  5.14s/it]
 67%|██████▋   | 150173/225000 [295:07:28<101:01:24,  4.86s/it]
 67%|██████▋   | 150174/225000 [295:07:32<98:51:20,  4.76s/it] 
 67%|██████▋   | 150175/225000 [295:07:38<102:18:29,  4.92s/it]
 67%|██████▋   | 150176/225000 [295:07:50<146:11:32,  7.03s/it]
 67%|██████▋   | 150177/225000 [295:07:54<132:10:26,  6.36s/it]
 67%|██████▋   | 150178/225000 [295:07:58<117:03:16,  5.63s/it]
 67%|██████▋   | 150179/225000 [295:08:03<113:22:57,  5.46s/it]
 67%|██████▋   | 150180/225000 [295:08:12<133:18:24,  6.41s/it]
 67%|██████▋   | 150181/225000 [295:08:16<114:47:01,  5.52s/it]
 67%|██████▋   | 150182/225000 [295:08:20<108:17:55,  5.21s/it]
 67%|██████▋   | 150183/225000 [295:08:26<111:40:28,  5.37s/it]
 67%|██████▋   | 150184/225000 [295:08:34<130:39:35,  6.29s/it]
 67%|██████▋   | 150185/225000 [295:08:40<126:48:51,  6.10s/it]
 67%|██████▋   | 150186/225000 [295:08:46<129:15:18,  6.22s/it]
 67%|██████▋   | 150187/225000 [295:08:49<109:52:52,  5.29s/it]
 67%|██████▋   | 150188/225000 [295:08:54<105:38:53,  5.08s/it]
 67%|██████▋   | 150189/225000 [295:09:03<127:02:55,  6.11s/it]
 67%|██████▋   | 150190/225000 [295:09:06<112:28:36,  5.41s/it]
 67%|██████▋   | 150191/225000 [295:09:15<135:09:43,  6.50s/it]
 67%|██████▋   | 150192/225000 [295:09:19<113:51:14,  5.48s/it]
 67%|██████▋   | 150193/225000 [295:09:23<104:53:42,  5.05s/it]
 67%|██████▋   | 150194/225000 [295:09:27<100:05:29,  4.82s/it]
 67%|██████▋   | 150195/225000 [295:09:32<99:08:43,  4.77s/it] 
 67%|██████▋   | 150196/225000 [295:09:36<100:23:33,  4.83s/it]
 67%|██████▋   | 150197/225000 [295:09:45<122:19:39,  5.89s/it]
 67%|██████▋   | 150198/225000 [295:09:54<140:49:17,  6.78s/it]
 67%|██████▋   | 150199/225000 [295:09:57<118:56:12,  5.72s/it]
 67%|██████▋   | 150200/225000 [295:10:01<109:36:55,  5.28s/it]
                                                               
{'loss': '0.7854', 'grad_norm': '1.044', 'learning_rate': '1.353e-05', 'epoch': '0.6676', 'num_input_tokens_seen': 489005288, 'train_runtime': '1.063e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150200/225000 [295:10:01<109:36:55,  5.28s/it]
 67%|██████▋   | 150201/225000 [295:10:05<101:32:17,  4.89s/it]
 67%|██████▋   | 150202/225000 [295:10:13<120:51:08,  5.82s/it]
 67%|██████▋   | 150203/225000 [295:10:19<122:17:28,  5.89s/it]
 67%|██████▋   | 150204/225000 [295:10:23<109:50:20,  5.29s/it]
 67%|██████▋   | 150205/225000 [295:10:27<101:42:56,  4.90s/it]
 67%|██████▋   | 150206/225000 [295:10:33<109:57:54,  5.29s/it]
 67%|██████▋   | 150207/225000 [295:10:43<137:50:39,  6.63s/it]
 67%|██████▋   | 150208/225000 [295:10:49<133:07:15,  6.41s/it]
 67%|██████▋   | 150209/225000 [295:10:52<112:07:55,  5.40s/it]
 67%|██████▋   | 150210/225000 [295:11:00<131:01:06,  6.31s/it]
 67%|██████▋   | 150211/225000 [295:11:04<112:42:29,  5.43s/it]
 67%|██████▋   | 150212/225000 [295:11:08<103:44:45,  4.99s/it]
 67%|██████▋   | 150213/225000 [295:11:12<99:12:27,  4.78s/it] 
 67%|██████▋   | 150214/225000 [295:11:24<142:21:19,  6.85s/it]
 67%|██████▋   | 150215/225000 [295:11:27<118:43:43,  5.72s/it]
 67%|██████▋   | 150216/225000 [295:11:32<112:33:24,  5.42s/it]
 67%|██████▋   | 150217/225000 [295:11:37<115:15:18,  5.55s/it]
 67%|██████▋   | 150218/225000 [295:11:43<117:16:19,  5.65s/it]
 67%|██████▋   | 150219/225000 [295:11:50<123:05:41,  5.93s/it]
 67%|██████▋   | 150220/225000 [295:11:57<131:42:56,  6.34s/it]
 67%|██████▋   | 150221/225000 [295:12:01<113:43:05,  5.47s/it]
 67%|██████▋   | 150222/225000 [295:12:04<100:55:37,  4.86s/it]
 67%|██████▋   | 150223/225000 [295:12:13<126:40:31,  6.10s/it]
 67%|██████▋   | 150224/225000 [295:12:17<114:12:58,  5.50s/it]
 67%|██████▋   | 150225/225000 [295:12:21<104:44:54,  5.04s/it]
 67%|██████▋   | 150226/225000 [295:12:25<98:22:39,  4.74s/it] 
 67%|██████▋   | 150227/225000 [295:12:33<118:41:18,  5.71s/it]
 67%|██████▋   | 150228/225000 [295:12:41<133:36:21,  6.43s/it]
 67%|██████▋   | 150229/225000 [295:12:45<115:04:55,  5.54s/it]
 67%|██████▋   | 150230/225000 [295:12:53<133:07:24,  6.41s/it]
 67%|██████▋   | 150231/225000 [295:12:57<119:32:42,  5.76s/it]
 67%|██████▋   | 150232/225000 [295:13:01<109:16:00,  5.26s/it]
 67%|██████▋   | 150233/225000 [295:13:06<102:22:32,  4.93s/it]
 67%|██████▋   | 150234/225000 [295:13:10<97:04:08,  4.67s/it] 
 67%|██████▋   | 150235/225000 [295:13:19<129:02:34,  6.21s/it]
 67%|██████▋   | 150236/225000 [295:13:25<127:29:21,  6.14s/it]
 67%|██████▋   | 150237/225000 [295:13:30<114:54:51,  5.53s/it]
 67%|██████▋   | 150238/225000 [295:13:33<104:14:36,  5.02s/it]
 67%|██████▋   | 150239/225000 [295:13:41<120:20:55,  5.80s/it]
 67%|██████▋   | 150240/225000 [295:13:48<129:04:24,  6.22s/it]
 67%|██████▋   | 150241/225000 [295:13:53<122:47:15,  5.91s/it]
 67%|██████▋   | 150242/225000 [295:13:59<121:39:52,  5.86s/it]
 67%|██████▋   | 150243/225000 [295:14:04<113:57:35,  5.49s/it]
 67%|██████▋   | 150244/225000 [295:14:09<114:28:59,  5.51s/it]
 67%|██████▋   | 150245/225000 [295:14:15<112:44:35,  5.43s/it]
 67%|██████▋   | 150246/225000 [295:14:20<111:38:35,  5.38s/it]
 67%|██████▋   | 150247/225000 [295:14:25<108:54:29,  5.24s/it]
 67%|██████▋   | 150248/225000 [295:14:30<109:16:31,  5.26s/it]
 67%|██████▋   | 150249/225000 [295:14:35<109:26:02,  5.27s/it]
 67%|██████▋   | 150250/225000 [295:14:41<112:17:54,  5.41s/it]
                                                               
{'loss': '0.7821', 'grad_norm': '1.225', 'learning_rate': '1.351e-05', 'epoch': '0.6678', 'num_input_tokens_seen': 489134480, 'train_runtime': '1.063e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150250/225000 [295:14:41<112:17:54,  5.41s/it]
 67%|██████▋   | 150251/225000 [295:14:47<112:45:54,  5.43s/it]
 67%|██████▋   | 150252/225000 [295:14:52<113:24:32,  5.46s/it]
 67%|██████▋   | 150253/225000 [295:14:57<111:35:27,  5.37s/it]
 67%|██████▋   | 150254/225000 [295:15:03<113:22:39,  5.46s/it]
 67%|██████▋   | 150255/225000 [295:15:08<113:03:51,  5.45s/it]
 67%|██████▋   | 150256/225000 [295:15:19<148:01:51,  7.13s/it]
 67%|██████▋   | 150257/225000 [295:15:25<137:48:27,  6.64s/it]
 67%|██████▋   | 150258/225000 [295:15:29<120:55:24,  5.82s/it]
 67%|██████▋   | 150259/225000 [295:15:33<109:34:09,  5.28s/it]
 67%|██████▋   | 150260/225000 [295:15:37<102:13:33,  4.92s/it]
 67%|██████▋   | 150261/225000 [295:15:49<145:16:36,  7.00s/it]
 67%|██████▋   | 150262/225000 [295:15:54<134:00:54,  6.46s/it]
 67%|██████▋   | 150263/225000 [295:15:57<115:23:55,  5.56s/it]
 67%|██████▋   | 150264/225000 [295:16:06<131:48:53,  6.35s/it]
 67%|██████▋   | 150265/225000 [295:16:09<114:55:52,  5.54s/it]
 67%|██████▋   | 150266/225000 [295:16:13<104:13:02,  5.02s/it]
 67%|██████▋   | 150267/225000 [295:16:17<100:19:24,  4.83s/it]
 67%|██████▋   | 150268/225000 [295:16:23<103:01:43,  4.96s/it]
 67%|██████▋   | 150269/225000 [295:16:33<138:00:15,  6.65s/it]
 67%|██████▋   | 150270/225000 [295:16:37<119:45:00,  5.77s/it]
 67%|██████▋   | 150271/225000 [295:16:41<107:19:01,  5.17s/it]
 67%|██████▋   | 150272/225000 [295:16:51<138:11:36,  6.66s/it]
 67%|██████▋   | 150273/225000 [295:16:54<116:08:50,  5.60s/it]
 67%|██████▋   | 150274/225000 [295:16:59<112:53:36,  5.44s/it]
 67%|██████▋   | 150275/225000 [295:17:04<111:58:13,  5.39s/it]
 67%|██████▋   | 150276/225000 [295:17:10<112:43:00,  5.43s/it]
 67%|██████▋   | 150277/225000 [295:17:15<112:47:41,  5.43s/it]
 67%|██████▋   | 150278/225000 [295:17:24<129:49:45,  6.25s/it]
 67%|██████▋   | 150279/225000 [295:17:28<116:39:54,  5.62s/it]
 67%|██████▋   | 150280/225000 [295:17:32<109:22:51,  5.27s/it]
 67%|██████▋   | 150281/225000 [295:17:36<103:31:36,  4.99s/it]
 67%|██████▋   | 150282/225000 [295:17:41<101:16:43,  4.88s/it]
 67%|██████▋   | 150283/225000 [295:17:50<127:08:55,  6.13s/it]
 67%|██████▋   | 150284/225000 [295:17:57<132:30:36,  6.38s/it]
 67%|██████▋   | 150285/225000 [295:18:01<116:46:42,  5.63s/it]
 67%|██████▋   | 150286/225000 [295:18:05<108:06:13,  5.21s/it]
 67%|██████▋   | 150287/225000 [295:18:09<102:27:54,  4.94s/it]
 67%|██████▋   | 150288/225000 [295:18:14<99:43:50,  4.81s/it] 
 67%|██████▋   | 150289/225000 [295:18:24<129:46:25,  6.25s/it]
 67%|██████▋   | 150290/225000 [295:18:30<131:03:23,  6.32s/it]
 67%|██████▋   | 150291/225000 [295:18:34<114:48:50,  5.53s/it]
 67%|██████▋   | 150292/225000 [295:18:38<106:00:11,  5.11s/it]
 67%|██████▋   | 150293/225000 [295:18:44<113:09:52,  5.45s/it]
 67%|██████▋   | 150294/225000 [295:18:53<132:20:18,  6.38s/it]
 67%|██████▋   | 150295/225000 [295:18:56<115:38:01,  5.57s/it]
 67%|██████▋   | 150296/225000 [295:19:00<106:13:09,  5.12s/it]
 67%|██████▋   | 150297/225000 [295:19:04<99:33:02,  4.80s/it] 
 67%|██████▋   | 150298/225000 [295:19:12<114:50:51,  5.53s/it]
 67%|██████▋   | 150299/225000 [295:19:21<137:53:56,  6.65s/it]
 67%|██████▋   | 150300/225000 [295:19:24<118:08:55,  5.69s/it]
                                                               
{'loss': '0.7716', 'grad_norm': '1.198', 'learning_rate': '1.349e-05', 'epoch': '0.668', 'num_input_tokens_seen': 489263296, 'train_runtime': '1.063e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150300/225000 [295:19:24<118:08:55,  5.69s/it]
 67%|██████▋   | 150301/225000 [295:19:29<108:24:07,  5.22s/it]
 67%|██████▋   | 150302/225000 [295:19:33<101:38:13,  4.90s/it]
 67%|██████▋   | 150303/225000 [295:19:43<136:01:38,  6.56s/it]
 67%|██████▋   | 150304/225000 [295:19:48<124:16:56,  5.99s/it]
 67%|██████▋   | 150305/225000 [295:19:52<115:37:23,  5.57s/it]
 67%|██████▋   | 150306/225000 [295:19:57<108:06:11,  5.21s/it]
 67%|██████▋   | 150307/225000 [295:20:01<103:28:13,  4.99s/it]
 67%|██████▋   | 150308/225000 [295:20:06<100:24:44,  4.84s/it]
 67%|██████▋   | 150309/225000 [295:20:12<109:31:43,  5.28s/it]
 67%|██████▋   | 150310/225000 [295:20:24<148:04:47,  7.14s/it]
 67%|██████▋   | 150311/225000 [295:20:27<126:27:22,  6.10s/it]
 67%|██████▋   | 150312/225000 [295:20:31<113:07:11,  5.45s/it]
 67%|██████▋   | 150313/225000 [295:20:40<134:33:28,  6.49s/it]
 67%|██████▋   | 150314/225000 [295:20:46<128:58:07,  6.22s/it]
 67%|██████▋   | 150315/225000 [295:20:49<111:51:23,  5.39s/it]
 67%|██████▋   | 150316/225000 [295:20:59<137:25:19,  6.62s/it]
 67%|██████▋   | 150317/225000 [295:21:03<122:03:47,  5.88s/it]
 67%|██████▋   | 150318/225000 [295:21:06<107:39:41,  5.19s/it]
 67%|██████▋   | 150319/225000 [295:21:15<128:33:05,  6.20s/it]
 67%|██████▋   | 150320/225000 [295:21:21<125:11:12,  6.03s/it]
 67%|██████▋   | 150321/225000 [295:21:26<121:19:30,  5.85s/it]
 67%|██████▋   | 150322/225000 [295:21:31<117:06:00,  5.65s/it]
 67%|██████▋   | 150323/225000 [295:21:36<115:00:14,  5.54s/it]
 67%|██████▋   | 150324/225000 [295:21:41<112:10:48,  5.41s/it]
 67%|██████▋   | 150325/225000 [295:21:46<108:33:31,  5.23s/it]
 67%|██████▋   | 150326/225000 [295:21:51<108:06:54,  5.21s/it]
 67%|██████▋   | 150327/225000 [295:21:57<107:12:43,  5.17s/it]
 67%|██████▋   | 150328/225000 [295:22:02<106:48:00,  5.15s/it]
 67%|██████▋   | 150329/225000 [295:22:06<104:22:09,  5.03s/it]
 67%|██████▋   | 150330/225000 [295:22:11<103:57:29,  5.01s/it]
 67%|██████▋   | 150331/225000 [295:22:21<130:52:15,  6.31s/it]
 67%|██████▋   | 150332/225000 [295:22:29<146:06:27,  7.04s/it]
 67%|██████▋   | 150333/225000 [295:22:33<124:12:09,  5.99s/it]
 67%|██████▋   | 150334/225000 [295:22:42<140:55:56,  6.80s/it]
 67%|██████▋   | 150335/225000 [295:22:45<120:29:42,  5.81s/it]
 67%|██████▋   | 150336/225000 [295:22:52<128:39:59,  6.20s/it]
 67%|██████▋   | 150337/225000 [295:22:57<117:35:56,  5.67s/it]
 67%|██████▋   | 150338/225000 [295:23:01<108:52:23,  5.25s/it]
 67%|██████▋   | 150339/225000 [295:23:09<127:55:47,  6.17s/it]
 67%|██████▋   | 150340/225000 [295:23:16<132:23:03,  6.38s/it]
 67%|██████▋   | 150341/225000 [295:23:20<116:15:09,  5.61s/it]
 67%|██████▋   | 150342/225000 [295:23:25<110:17:58,  5.32s/it]
 67%|██████▋   | 150343/225000 [295:23:30<108:10:21,  5.22s/it]
 67%|██████▋   | 150344/225000 [295:23:34<103:46:25,  5.00s/it]
 67%|██████▋   | 150345/225000 [295:23:39<103:32:21,  4.99s/it]
 67%|██████▋   | 150346/225000 [295:23:44<103:50:53,  5.01s/it]
 67%|██████▋   | 150347/225000 [295:23:49<101:24:26,  4.89s/it]
 67%|██████▋   | 150348/225000 [295:23:57<123:09:36,  5.94s/it]
 67%|██████▋   | 150349/225000 [295:24:08<151:31:38,  7.31s/it]
 67%|██████▋   | 150350/225000 [295:24:11<126:45:23,  6.11s/it]
                                                               
{'loss': '0.7905', 'grad_norm': '1.065', 'learning_rate': '1.348e-05', 'epoch': '0.6682', 'num_input_tokens_seen': 489393448, 'train_runtime': '1.063e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150350/225000 [295:24:11<126:45:23,  6.11s/it]
 67%|██████▋   | 150351/225000 [295:24:14<109:50:58,  5.30s/it]
 67%|██████▋   | 150352/225000 [295:24:23<133:45:26,  6.45s/it]
 67%|██████▋   | 150353/225000 [295:24:27<113:56:47,  5.50s/it]
 67%|██████▋   | 150354/225000 [295:24:34<122:20:08,  5.90s/it]
 67%|██████▋   | 150355/225000 [295:24:40<125:25:01,  6.05s/it]
 67%|██████▋   | 150356/225000 [295:24:44<110:39:42,  5.34s/it]
 67%|██████▋   | 150357/225000 [295:24:47<100:53:01,  4.87s/it]
 67%|██████▋   | 150358/225000 [295:24:58<135:51:02,  6.55s/it]
 67%|██████▋   | 150359/225000 [295:25:03<123:41:08,  5.97s/it]
 67%|██████▋   | 150360/225000 [295:25:08<118:18:11,  5.71s/it]
 67%|██████▋   | 150361/225000 [295:25:13<113:18:14,  5.46s/it]
 67%|██████▋   | 150362/225000 [295:25:18<112:07:06,  5.41s/it]
 67%|██████▋   | 150363/225000 [295:25:23<111:18:59,  5.37s/it]
 67%|██████▋   | 150364/225000 [295:25:28<108:05:53,  5.21s/it]
 67%|██████▋   | 150365/225000 [295:25:33<107:17:54,  5.18s/it]
 67%|██████▋   | 150366/225000 [295:25:38<108:44:07,  5.24s/it]
 67%|██████▋   | 150367/225000 [295:25:48<133:52:05,  6.46s/it]
 67%|██████▋   | 150368/225000 [295:25:54<135:24:20,  6.53s/it]
 67%|██████▋   | 150369/225000 [295:26:00<128:43:06,  6.21s/it]
 67%|██████▋   | 150370/225000 [295:26:05<122:50:44,  5.93s/it]
 67%|██████▋   | 150371/225000 [295:26:10<119:05:03,  5.74s/it]
 67%|██████▋   | 150372/225000 [295:26:16<115:30:47,  5.57s/it]
 67%|██████▋   | 150373/225000 [295:26:21<111:48:13,  5.39s/it]
 67%|██████▋   | 150374/225000 [295:26:25<107:07:34,  5.17s/it]
 67%|██████▋   | 150375/225000 [295:26:30<107:26:53,  5.18s/it]
 67%|██████▋   | 150376/225000 [295:26:36<106:52:52,  5.16s/it]
 67%|██████▋   | 150377/225000 [295:26:40<105:25:36,  5.09s/it]
 67%|██████▋   | 150378/225000 [295:26:45<101:19:39,  4.89s/it]
 67%|██████▋   | 150379/225000 [295:26:50<103:28:39,  4.99s/it]
 67%|██████▋   | 150380/225000 [295:26:55<101:27:58,  4.90s/it]
 67%|██████▋   | 150381/225000 [295:27:00<102:18:22,  4.94s/it]
 67%|██████▋   | 150382/225000 [295:27:07<114:03:14,  5.50s/it]
 67%|██████▋   | 150383/225000 [295:27:18<151:07:53,  7.29s/it]
 67%|██████▋   | 150384/225000 [295:27:22<127:19:31,  6.14s/it]
 67%|██████▋   | 150385/225000 [295:27:26<114:23:12,  5.52s/it]
 67%|██████▋   | 150386/225000 [295:27:35<136:30:58,  6.59s/it]
 67%|██████▋   | 150387/225000 [295:27:39<123:38:22,  5.97s/it]
 67%|██████▋   | 150388/225000 [295:27:44<113:44:34,  5.49s/it]
 67%|██████▋   | 150389/225000 [295:27:48<107:22:22,  5.18s/it]
 67%|██████▋   | 150390/225000 [295:27:53<104:30:21,  5.04s/it]
 67%|██████▋   | 150391/225000 [295:27:58<105:41:55,  5.10s/it]
 67%|██████▋   | 150392/225000 [295:28:10<150:23:43,  7.26s/it]
 67%|██████▋   | 150393/225000 [295:28:14<128:55:56,  6.22s/it]
 67%|██████▋   | 150394/225000 [295:28:20<129:32:23,  6.25s/it]
 67%|██████▋   | 150395/225000 [295:28:26<125:32:04,  6.06s/it]
 67%|██████▋   | 150396/225000 [295:28:31<116:33:23,  5.62s/it]
 67%|██████▋   | 150397/225000 [295:28:35<110:10:57,  5.32s/it]
 67%|██████▋   | 150398/225000 [295:28:40<105:07:12,  5.07s/it]
 67%|██████▋   | 150399/225000 [295:28:44<101:43:10,  4.91s/it]
 67%|██████▋   | 150400/225000 [295:28:54<130:42:06,  6.31s/it]
                                                               
{'loss': '0.7812', 'grad_norm': '1.139', 'learning_rate': '1.346e-05', 'epoch': '0.6684', 'num_input_tokens_seen': 489522776, 'train_runtime': '1.064e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150400/225000 [295:28:54<130:42:06,  6.31s/it]
 67%|██████▋   | 150401/225000 [295:28:59<126:21:30,  6.10s/it]
 67%|██████▋   | 150402/225000 [295:29:03<108:55:29,  5.26s/it]
 67%|██████▋   | 150403/225000 [295:29:11<127:27:41,  6.15s/it]
 67%|██████▋   | 150404/225000 [295:29:14<109:02:45,  5.26s/it]
 67%|██████▋   | 150405/225000 [295:29:23<129:38:20,  6.26s/it]
 67%|██████▋   | 150406/225000 [295:29:26<112:20:13,  5.42s/it]
 67%|██████▋   | 150407/225000 [295:29:35<132:28:41,  6.39s/it]
 67%|██████▋   | 150408/225000 [295:29:39<120:34:36,  5.82s/it]
 67%|██████▋   | 150409/225000 [295:29:43<109:11:29,  5.27s/it]
 67%|██████▋   | 150410/225000 [295:29:47<98:28:53,  4.75s/it] 
 67%|██████▋   | 150411/225000 [295:29:58<138:26:02,  6.68s/it]
 67%|██████▋   | 150412/225000 [295:30:02<120:12:49,  5.80s/it]
 67%|██████▋   | 150413/225000 [295:30:08<124:52:19,  6.03s/it]
 67%|██████▋   | 150414/225000 [295:30:12<111:51:12,  5.40s/it]
 67%|██████▋   | 150415/225000 [295:30:17<105:32:22,  5.09s/it]
 67%|██████▋   | 150416/225000 [295:30:22<105:04:28,  5.07s/it]
 67%|██████▋   | 150417/225000 [295:30:27<106:19:43,  5.13s/it]
 67%|██████▋   | 150418/225000 [295:30:32<103:19:04,  4.99s/it]
 67%|██████▋   | 150419/225000 [295:30:36<102:07:32,  4.93s/it]
 67%|██████▋   | 150420/225000 [295:30:42<103:12:28,  4.98s/it]
 67%|██████▋   | 150421/225000 [295:30:47<103:57:04,  5.02s/it]
 67%|██████▋   | 150422/225000 [295:30:52<106:52:00,  5.16s/it]
 67%|██████▋   | 150423/225000 [295:30:57<106:13:37,  5.13s/it]
 67%|██████▋   | 150424/225000 [295:31:02<106:54:47,  5.16s/it]
 67%|██████▋   | 150425/225000 [295:31:08<108:23:22,  5.23s/it]
 67%|██████▋   | 150426/225000 [295:31:13<109:38:55,  5.29s/it]
 67%|██████▋   | 150427/225000 [295:31:26<153:35:12,  7.41s/it]
 67%|██████▋   | 150428/225000 [295:31:31<140:45:33,  6.80s/it]
 67%|██████▋   | 150429/225000 [295:31:35<122:58:35,  5.94s/it]
 67%|██████▋   | 150430/225000 [295:31:41<122:17:22,  5.90s/it]
 67%|██████▋   | 150431/225000 [295:31:50<142:06:22,  6.86s/it]
 67%|██████▋   | 150432/225000 [295:31:53<120:33:37,  5.82s/it]
 67%|██████▋   | 150433/225000 [295:31:57<104:58:28,  5.07s/it]
 67%|██████▋   | 150434/225000 [295:32:03<113:43:21,  5.49s/it]
 67%|██████▋   | 150435/225000 [295:32:10<119:51:33,  5.79s/it]
 67%|██████▋   | 150436/225000 [295:32:13<108:27:17,  5.24s/it]
 67%|██████▋   | 150437/225000 [295:32:17<100:16:51,  4.84s/it]
 67%|██████▋   | 150438/225000 [295:32:22<96:15:55,  4.65s/it] 
 67%|██████▋   | 150439/225000 [295:32:32<129:15:48,  6.24s/it]
 67%|██████▋   | 150440/225000 [295:32:35<111:56:38,  5.41s/it]
 67%|██████▋   | 150441/225000 [295:32:39<103:45:41,  5.01s/it]
 67%|██████▋   | 150442/225000 [295:32:49<134:10:29,  6.48s/it]
 67%|██████▋   | 150443/225000 [295:32:54<124:49:53,  6.03s/it]
 67%|██████▋   | 150444/225000 [295:33:00<123:11:07,  5.95s/it]
 67%|██████▋   | 150445/225000 [295:33:03<107:00:09,  5.17s/it]
 67%|██████▋   | 150446/225000 [295:33:10<116:17:30,  5.62s/it]
 67%|██████▋   | 150447/225000 [295:33:15<114:14:28,  5.52s/it]
 67%|██████▋   | 150448/225000 [295:33:20<111:21:48,  5.38s/it]
 67%|██████▋   | 150449/225000 [295:33:26<112:33:40,  5.44s/it]
 67%|██████▋   | 150450/225000 [295:33:31<113:41:48,  5.49s/it]
                                                               
{'loss': '0.7824', 'grad_norm': '1.231', 'learning_rate': '1.345e-05', 'epoch': '0.6687', 'num_input_tokens_seen': 489651504, 'train_runtime': '1.064e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150450/225000 [295:33:31<113:41:48,  5.49s/it]
 67%|██████▋   | 150451/225000 [295:33:39<127:05:25,  6.14s/it]
 67%|██████▋   | 150452/225000 [295:33:43<115:33:42,  5.58s/it]
 67%|██████▋   | 150453/225000 [295:33:48<109:14:04,  5.28s/it]
 67%|██████▋   | 150454/225000 [295:33:52<103:25:02,  4.99s/it]
 67%|██████▋   | 150455/225000 [295:33:57<102:39:45,  4.96s/it]
 67%|██████▋   | 150456/225000 [295:34:02<100:29:59,  4.85s/it]
 67%|██████▋   | 150457/225000 [295:34:06<99:58:35,  4.83s/it] 
 67%|██████▋   | 150458/225000 [295:34:18<143:24:46,  6.93s/it]
 67%|██████▋   | 150459/225000 [295:34:24<136:21:26,  6.59s/it]
 67%|██████▋   | 150460/225000 [295:34:29<126:51:33,  6.13s/it]
 67%|██████▋   | 150461/225000 [295:34:34<120:18:35,  5.81s/it]
 67%|██████▋   | 150462/225000 [295:34:39<117:34:05,  5.68s/it]
 67%|██████▋   | 150463/225000 [295:34:44<112:54:48,  5.45s/it]
 67%|██████▋   | 150464/225000 [295:34:49<109:21:09,  5.28s/it]
 67%|██████▋   | 150465/225000 [295:34:55<109:44:38,  5.30s/it]
 67%|██████▋   | 150466/225000 [295:35:00<110:20:40,  5.33s/it]
 67%|██████▋   | 150467/225000 [295:35:05<108:49:47,  5.26s/it]
 67%|██████▋   | 150468/225000 [295:35:10<107:41:29,  5.20s/it]
 67%|██████▋   | 150469/225000 [295:35:21<140:36:50,  6.79s/it]
 67%|██████▋   | 150470/225000 [295:35:24<120:20:30,  5.81s/it]
 67%|██████▋   | 150471/225000 [295:35:28<107:59:59,  5.22s/it]
 67%|██████▋   | 150472/225000 [295:35:32<102:18:44,  4.94s/it]
 67%|██████▋   | 150473/225000 [295:35:42<132:09:18,  6.38s/it]
 67%|██████▋   | 150474/225000 [295:35:49<135:49:02,  6.56s/it]
 67%|██████▋   | 150475/225000 [295:35:53<121:45:08,  5.88s/it]
 67%|██████▋   | 150476/225000 [295:35:59<122:47:26,  5.93s/it]
 67%|██████▋   | 150477/225000 [295:36:03<108:50:28,  5.26s/it]
 67%|██████▋   | 150478/225000 [295:36:07<100:14:31,  4.84s/it]
 67%|██████▋   | 150479/225000 [295:36:12<102:10:23,  4.94s/it]
 67%|██████▋   | 150480/225000 [295:36:18<108:38:39,  5.25s/it]
 67%|██████▋   | 150481/225000 [295:36:27<133:21:33,  6.44s/it]
 67%|██████▋   | 150482/225000 [295:36:31<116:30:44,  5.63s/it]
 67%|██████▋   | 150483/225000 [295:36:35<104:20:53,  5.04s/it]
 67%|██████▋   | 150484/225000 [295:36:43<122:34:11,  5.92s/it]
 67%|██████▋   | 150485/225000 [295:36:48<117:34:36,  5.68s/it]
 67%|██████▋   | 150486/225000 [295:36:53<115:38:15,  5.59s/it]
 67%|██████▋   | 150487/225000 [295:36:58<111:12:23,  5.37s/it]
 67%|██████▋   | 150488/225000 [295:37:02<104:05:50,  5.03s/it]
 67%|██████▋   | 150489/225000 [295:37:07<105:04:33,  5.08s/it]
 67%|██████▋   | 150490/225000 [295:37:12<102:52:22,  4.97s/it]
 67%|██████▋   | 150491/225000 [295:37:17<102:49:41,  4.97s/it]
 67%|██████▋   | 150492/225000 [295:37:22<104:13:41,  5.04s/it]
 67%|██████▋   | 150493/225000 [295:37:27<104:02:53,  5.03s/it]
 67%|██████▋   | 150494/225000 [295:37:32<103:39:07,  5.01s/it]
 67%|██████▋   | 150495/225000 [295:37:38<106:06:01,  5.13s/it]
 67%|██████▋   | 150496/225000 [295:37:43<106:19:18,  5.14s/it]
 67%|██████▋   | 150497/225000 [295:37:51<127:00:21,  6.14s/it]
 67%|██████▋   | 150498/225000 [295:38:02<154:10:47,  7.45s/it]
 67%|██████▋   | 150499/225000 [295:38:05<130:06:32,  6.29s/it]
 67%|██████▋   | 150500/225000 [295:38:13<140:06:38,  6.77s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.257', 'learning_rate': '1.343e-05', 'epoch': '0.6689', 'num_input_tokens_seen': 489781520, 'train_runtime': '1.064e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150500/225000 [295:38:13<140:06:38,  6.77s/it]
 67%|██████▋   | 150501/225000 [295:38:17<118:10:26,  5.71s/it]
 67%|██████▋   | 150502/225000 [295:38:24<126:21:34,  6.11s/it]
 67%|██████▋   | 150503/225000 [295:38:27<111:20:22,  5.38s/it]
 67%|██████▋   | 150504/225000 [295:38:35<128:39:28,  6.22s/it]
 67%|██████▋   | 150505/225000 [295:38:40<115:08:56,  5.56s/it]
 67%|██████▋   | 150506/225000 [295:38:44<107:19:28,  5.19s/it]
 67%|██████▋   | 150507/225000 [295:38:48<99:25:05,  4.80s/it] 
 67%|██████▋   | 150508/225000 [295:38:58<134:50:20,  6.52s/it]
 67%|██████▋   | 150509/225000 [295:39:02<117:57:37,  5.70s/it]
 67%|██████▋   | 150510/225000 [295:39:06<107:11:50,  5.18s/it]
 67%|██████▋   | 150511/225000 [295:39:10<100:29:21,  4.86s/it]
 67%|██████▋   | 150512/225000 [295:39:20<133:22:51,  6.45s/it]
 67%|██████▋   | 150513/225000 [295:39:24<115:11:01,  5.57s/it]
 67%|██████▋   | 150514/225000 [295:39:28<105:54:56,  5.12s/it]
 67%|██████▋   | 150515/225000 [295:39:36<123:33:43,  5.97s/it]
 67%|██████▋   | 150516/225000 [295:39:40<110:26:52,  5.34s/it]
 67%|██████▋   | 150517/225000 [295:39:44<102:50:03,  4.97s/it]
 67%|██████▋   | 150518/225000 [295:39:48<98:46:02,  4.77s/it] 
 67%|██████▋   | 150519/225000 [295:39:53<96:53:01,  4.68s/it]
 67%|██████▋   | 150520/225000 [295:39:59<110:46:18,  5.35s/it]
 67%|██████▋   | 150521/225000 [295:40:09<138:15:44,  6.68s/it]
 67%|██████▋   | 150522/225000 [295:40:13<119:15:42,  5.76s/it]
 67%|██████▋   | 150523/225000 [295:40:18<115:33:54,  5.59s/it]
 67%|██████▋   | 150524/225000 [295:40:27<135:11:47,  6.54s/it]
 67%|██████▋   | 150525/225000 [295:40:33<130:59:52,  6.33s/it]
 67%|██████▋   | 150526/225000 [295:40:36<111:31:37,  5.39s/it]
 67%|██████▋   | 150527/225000 [295:40:41<110:53:03,  5.36s/it]
 67%|██████▋   | 150528/225000 [295:40:47<111:39:54,  5.40s/it]
 67%|██████▋   | 150529/225000 [295:40:52<111:03:09,  5.37s/it]
 67%|██████▋   | 150530/225000 [295:40:57<109:56:36,  5.31s/it]
 67%|██████▋   | 150531/225000 [295:41:02<109:16:41,  5.28s/it]
 67%|██████▋   | 150532/225000 [295:41:08<110:36:32,  5.35s/it]
 67%|██████▋   | 150533/225000 [295:41:13<108:26:50,  5.24s/it]
 67%|██████▋   | 150534/225000 [295:41:18<108:30:20,  5.25s/it]
 67%|██████▋   | 150535/225000 [295:41:25<121:07:09,  5.86s/it]
 67%|██████▋   | 150536/225000 [295:41:31<117:03:20,  5.66s/it]
 67%|██████▋   | 150537/225000 [295:41:36<115:20:00,  5.58s/it]
 67%|██████▋   | 150538/225000 [295:41:41<111:58:02,  5.41s/it]
 67%|██████▋   | 150539/225000 [295:41:46<111:16:53,  5.38s/it]
 67%|██████▋   | 150540/225000 [295:41:51<106:58:06,  5.17s/it]
 67%|██████▋   | 150541/225000 [295:41:56<105:09:17,  5.08s/it]
 67%|██████▋   | 150542/225000 [295:42:01<106:42:30,  5.16s/it]
 67%|██████▋   | 150543/225000 [295:42:07<108:55:14,  5.27s/it]
 67%|██████▋   | 150544/225000 [295:42:12<108:39:59,  5.25s/it]
 67%|██████▋   | 150545/225000 [295:42:24<153:14:50,  7.41s/it]
 67%|██████▋   | 150546/225000 [295:42:30<145:03:19,  7.01s/it]
 67%|██████▋   | 150547/225000 [295:42:37<139:59:31,  6.77s/it]
 67%|██████▋   | 150548/225000 [295:42:44<145:39:10,  7.04s/it]
 67%|██████▋   | 150549/225000 [295:42:49<128:36:38,  6.22s/it]
 67%|██████▋   | 150550/225000 [295:42:53<119:43:16,  5.79s/it]
                                                               
{'loss': '0.7755', 'grad_norm': '1.187', 'learning_rate': '1.341e-05', 'epoch': '0.6691', 'num_input_tokens_seen': 489911816, 'train_runtime': '1.065e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150550/225000 [295:42:53<119:43:16,  5.79s/it]
 67%|██████▋   | 150551/225000 [295:42:58<109:37:48,  5.30s/it]
 67%|██████▋   | 150552/225000 [295:43:02<105:40:34,  5.11s/it]
 67%|██████▋   | 150553/225000 [295:43:07<101:28:50,  4.91s/it]
 67%|██████▋   | 150554/225000 [295:43:18<141:49:58,  6.86s/it]
 67%|██████▋   | 150555/225000 [295:43:23<132:10:10,  6.39s/it]
 67%|██████▋   | 150556/225000 [295:43:27<112:54:32,  5.46s/it]
 67%|██████▋   | 150557/225000 [295:43:35<129:02:34,  6.24s/it]
 67%|██████▋   | 150558/225000 [295:43:39<115:43:02,  5.60s/it]
 67%|██████▋   | 150559/225000 [295:43:43<104:06:07,  5.03s/it]
 67%|██████▋   | 150560/225000 [295:43:49<111:38:31,  5.40s/it]
 67%|██████▋   | 150561/225000 [295:43:54<113:29:14,  5.49s/it]
 67%|██████▋   | 150562/225000 [295:44:01<121:28:35,  5.87s/it]
 67%|██████▋   | 150563/225000 [295:44:05<108:10:00,  5.23s/it]
 67%|██████▋   | 150564/225000 [295:44:09<102:08:51,  4.94s/it]
 67%|██████▋   | 150565/225000 [295:44:14<102:54:54,  4.98s/it]
 67%|██████▋   | 150566/225000 [295:44:19<100:02:22,  4.84s/it]
 67%|██████▋   | 150567/225000 [295:44:23<95:49:07,  4.63s/it] 
 67%|██████▋   | 150568/225000 [295:44:33<127:30:06,  6.17s/it]
 67%|██████▋   | 150569/225000 [295:44:40<136:26:41,  6.60s/it]
 67%|██████▋   | 150570/225000 [295:44:44<118:14:13,  5.72s/it]
 67%|██████▋   | 150571/225000 [295:44:50<122:18:50,  5.92s/it]
 67%|██████▋   | 150572/225000 [295:44:57<125:03:16,  6.05s/it]
 67%|██████▋   | 150573/225000 [295:45:02<121:12:40,  5.86s/it]
 67%|██████▋   | 150574/225000 [295:45:09<124:55:33,  6.04s/it]
 67%|██████▋   | 150575/225000 [295:45:13<112:09:27,  5.43s/it]
 67%|██████▋   | 150576/225000 [295:45:17<106:21:25,  5.14s/it]
 67%|██████▋   | 150577/225000 [295:45:21<100:57:31,  4.88s/it]
 67%|██████▋   | 150578/225000 [295:45:26<100:50:22,  4.88s/it]
 67%|██████▋   | 150579/225000 [295:45:38<142:02:16,  6.87s/it]
 67%|██████▋   | 150580/225000 [295:45:41<121:10:27,  5.86s/it]
 67%|██████▋   | 150581/225000 [295:45:45<108:08:42,  5.23s/it]
 67%|██████▋   | 150582/225000 [295:45:55<138:44:30,  6.71s/it]
 67%|██████▋   | 150583/225000 [295:45:59<120:11:19,  5.81s/it]
 67%|██████▋   | 150584/225000 [295:46:06<130:21:33,  6.31s/it]
 67%|██████▋   | 150585/225000 [295:46:12<125:04:10,  6.05s/it]
 67%|██████▋   | 150586/225000 [295:46:16<110:21:28,  5.34s/it]
 67%|██████▋   | 150587/225000 [295:46:22<118:33:14,  5.74s/it]
 67%|██████▋   | 150588/225000 [295:46:27<110:34:21,  5.35s/it]
 67%|██████▋   | 150589/225000 [295:46:32<109:26:46,  5.30s/it]
 67%|██████▋   | 150590/225000 [295:46:37<108:26:19,  5.25s/it]
 67%|██████▋   | 150591/225000 [295:46:43<110:28:27,  5.34s/it]
 67%|██████▋   | 150592/225000 [295:46:47<107:19:32,  5.19s/it]
 67%|██████▋   | 150593/225000 [295:46:53<108:41:19,  5.26s/it]
 67%|██████▋   | 150594/225000 [295:46:58<109:01:34,  5.28s/it]
 67%|██████▋   | 150595/225000 [295:47:04<110:09:40,  5.33s/it]
 67%|██████▋   | 150596/225000 [295:47:09<109:13:14,  5.28s/it]
 67%|██████▋   | 150597/225000 [295:47:15<116:50:18,  5.65s/it]
 67%|██████▋   | 150598/225000 [295:47:20<110:17:42,  5.34s/it]
 67%|██████▋   | 150599/225000 [295:47:25<108:54:57,  5.27s/it]
 67%|██████▋   | 150600/225000 [295:47:30<110:19:55,  5.34s/it]
                                                               
{'loss': '0.7879', 'grad_norm': '1.05', 'learning_rate': '1.34e-05', 'epoch': '0.6693', 'num_input_tokens_seen': 490041976, 'train_runtime': '1.065e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150600/225000 [295:47:30<110:19:55,  5.34s/it]
 67%|██████▋   | 150601/225000 [295:47:36<111:12:17,  5.38s/it]
 67%|██████▋   | 150602/225000 [295:47:41<110:03:42,  5.33s/it]
 67%|██████▋   | 150603/225000 [295:47:46<108:51:08,  5.27s/it]
 67%|██████▋   | 150604/225000 [295:47:52<111:07:33,  5.38s/it]
 67%|██████▋   | 150605/225000 [295:47:58<114:52:01,  5.56s/it]
 67%|██████▋   | 150606/225000 [295:48:03<112:51:46,  5.46s/it]
 67%|██████▋   | 150607/225000 [295:48:08<109:59:59,  5.32s/it]
 67%|██████▋   | 150608/225000 [295:48:13<109:21:59,  5.29s/it]
 67%|██████▋   | 150609/225000 [295:48:18<106:47:18,  5.17s/it]
 67%|██████▋   | 150610/225000 [295:48:24<108:15:55,  5.24s/it]
 67%|██████▋   | 150611/225000 [295:48:28<105:39:03,  5.11s/it]
 67%|██████▋   | 150612/225000 [295:48:33<104:14:06,  5.04s/it]
 67%|██████▋   | 150613/225000 [295:48:39<106:35:18,  5.16s/it]
 67%|██████▋   | 150614/225000 [295:48:44<107:57:21,  5.22s/it]
 67%|██████▋   | 150615/225000 [295:48:55<145:02:48,  7.02s/it]
 67%|██████▋   | 150616/225000 [295:49:03<150:30:27,  7.28s/it]
 67%|██████▋   | 150617/225000 [295:49:07<128:35:20,  6.22s/it]
 67%|██████▋   | 150618/225000 [295:49:11<115:42:08,  5.60s/it]
 67%|██████▋   | 150619/225000 [295:49:15<105:51:24,  5.12s/it]
 67%|██████▋   | 150620/225000 [295:49:24<129:15:01,  6.26s/it]
 67%|██████▋   | 150621/225000 [295:49:31<132:27:28,  6.41s/it]
 67%|██████▋   | 150622/225000 [295:49:37<130:09:16,  6.30s/it]
 67%|██████▋   | 150623/225000 [295:49:40<112:47:59,  5.46s/it]
 67%|██████▋   | 150624/225000 [295:49:49<131:38:16,  6.37s/it]
 67%|██████▋   | 150625/225000 [295:49:53<118:00:19,  5.71s/it]
 67%|██████▋   | 150626/225000 [295:49:57<106:57:14,  5.18s/it]
 67%|██████▋   | 150627/225000 [295:50:05<124:31:21,  6.03s/it]
 67%|██████▋   | 150628/225000 [295:50:11<126:38:31,  6.13s/it]
 67%|██████▋   | 150629/225000 [295:50:16<114:58:42,  5.57s/it]
 67%|██████▋   | 150630/225000 [295:50:19<103:17:14,  5.00s/it]
 67%|██████▋   | 150631/225000 [295:50:24<98:52:24,  4.79s/it] 
 67%|██████▋   | 150632/225000 [295:50:28<97:05:38,  4.70s/it]
 67%|██████▋   | 150633/225000 [295:50:34<104:43:21,  5.07s/it]
 67%|██████▋   | 150634/225000 [295:50:44<137:15:22,  6.64s/it]
 67%|██████▋   | 150635/225000 [295:50:48<117:29:43,  5.69s/it]
 67%|██████▋   | 150636/225000 [295:50:56<131:38:08,  6.37s/it]
 67%|██████▋   | 150637/225000 [295:50:59<112:49:48,  5.46s/it]
 67%|██████▋   | 150638/225000 [295:51:06<121:16:36,  5.87s/it]
 67%|██████▋   | 150639/225000 [295:51:09<105:55:35,  5.13s/it]
 67%|██████▋   | 150640/225000 [295:51:18<128:09:39,  6.20s/it]
 67%|██████▋   | 150641/225000 [295:51:22<112:09:07,  5.43s/it]
 67%|██████▋   | 150642/225000 [295:51:29<124:43:30,  6.04s/it]
 67%|██████▋   | 150643/225000 [295:51:35<121:33:06,  5.88s/it]
 67%|██████▋   | 150644/225000 [295:51:38<107:17:48,  5.19s/it]
 67%|██████▋   | 150645/225000 [295:51:47<129:43:56,  6.28s/it]
 67%|██████▋   | 150646/225000 [295:51:52<120:55:12,  5.85s/it]
 67%|██████▋   | 150647/225000 [295:51:57<114:26:05,  5.54s/it]
 67%|██████▋   | 150648/225000 [295:52:01<108:09:54,  5.24s/it]
 67%|██████▋   | 150649/225000 [295:52:06<105:52:59,  5.13s/it]
 67%|██████▋   | 150650/225000 [295:52:10<101:32:47,  4.92s/it]
                                                               
{'loss': '0.7781', 'grad_norm': '1.139', 'learning_rate': '1.338e-05', 'epoch': '0.6696', 'num_input_tokens_seen': 490170328, 'train_runtime': '1.065e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150650/225000 [295:52:10<101:32:47,  4.92s/it]
 67%|██████▋   | 150651/225000 [295:52:15<101:36:21,  4.92s/it]
 67%|██████▋   | 150652/225000 [295:52:20<99:34:30,  4.82s/it] 
 67%|██████▋   | 150653/225000 [295:52:25<100:26:08,  4.86s/it]
 67%|██████▋   | 150654/225000 [295:52:29<98:28:30,  4.77s/it] 
 67%|██████▋   | 150655/225000 [295:52:34<98:42:03,  4.78s/it]
 67%|██████▋   | 150656/225000 [295:52:44<128:20:17,  6.21s/it]
 67%|██████▋   | 150657/225000 [295:52:52<140:24:54,  6.80s/it]
 67%|██████▋   | 150658/225000 [295:52:59<139:34:02,  6.76s/it]
 67%|██████▋   | 150659/225000 [295:53:05<138:21:33,  6.70s/it]
 67%|██████▋   | 150660/225000 [295:53:09<117:52:53,  5.71s/it]
 67%|██████▋   | 150661/225000 [295:53:18<138:20:45,  6.70s/it]
 67%|██████▋   | 150662/225000 [295:53:22<122:42:33,  5.94s/it]
 67%|██████▋   | 150663/225000 [295:53:26<112:34:19,  5.45s/it]
 67%|██████▋   | 150664/225000 [295:53:36<138:34:20,  6.71s/it]
 67%|██████▋   | 150665/225000 [295:53:41<126:53:58,  6.15s/it]
 67%|██████▋   | 150666/225000 [295:53:45<114:53:56,  5.56s/it]
 67%|██████▋   | 150667/225000 [295:53:50<112:04:55,  5.43s/it]
 67%|██████▋   | 150668/225000 [295:53:55<108:46:52,  5.27s/it]
 67%|██████▋   | 150669/225000 [295:54:00<107:13:14,  5.19s/it]
 67%|██████▋   | 150670/225000 [295:54:05<105:24:11,  5.10s/it]
 67%|██████▋   | 150671/225000 [295:54:10<103:49:06,  5.03s/it]
 67%|██████▋   | 150672/225000 [295:54:17<116:27:01,  5.64s/it]
 67%|██████▋   | 150673/225000 [295:54:26<142:21:30,  6.90s/it]
 67%|██████▋   | 150674/225000 [295:54:30<120:41:37,  5.85s/it]
 67%|██████▋   | 150675/225000 [295:54:37<126:32:37,  6.13s/it]
 67%|██████▋   | 150676/225000 [295:54:41<118:25:32,  5.74s/it]
 67%|██████▋   | 150677/225000 [295:54:45<106:35:58,  5.16s/it]
 67%|██████▋   | 150678/225000 [295:54:54<130:05:40,  6.30s/it]
 67%|██████▋   | 150679/225000 [295:54:59<119:52:10,  5.81s/it]
 67%|██████▋   | 150680/225000 [295:55:04<113:43:56,  5.51s/it]
 67%|██████▋   | 150681/225000 [295:55:08<107:13:00,  5.19s/it]
 67%|██████▋   | 150682/225000 [295:55:13<105:48:08,  5.13s/it]
 67%|██████▋   | 150683/225000 [295:55:18<105:52:56,  5.13s/it]
 67%|██████▋   | 150684/225000 [295:55:23<104:16:37,  5.05s/it]
 67%|██████▋   | 150685/225000 [295:55:28<103:56:42,  5.04s/it]
 67%|██████▋   | 150686/225000 [295:55:41<152:27:52,  7.39s/it]
 67%|██████▋   | 150687/225000 [295:55:44<127:26:07,  6.17s/it]
 67%|██████▋   | 150688/225000 [295:55:48<112:56:47,  5.47s/it]
 67%|██████▋   | 150689/225000 [295:55:57<135:27:04,  6.56s/it]
 67%|██████▋   | 150690/225000 [295:56:01<117:02:47,  5.67s/it]
 67%|██████▋   | 150691/225000 [295:56:06<111:18:08,  5.39s/it]
 67%|██████▋   | 150692/225000 [295:56:13<123:51:44,  6.00s/it]
 67%|██████▋   | 150693/225000 [295:56:19<124:16:07,  6.02s/it]
 67%|██████▋   | 150694/225000 [295:56:23<113:32:22,  5.50s/it]
 67%|██████▋   | 150695/225000 [295:56:30<122:15:10,  5.92s/it]
 67%|██████▋   | 150696/225000 [295:56:34<108:54:54,  5.28s/it]
 67%|██████▋   | 150697/225000 [295:56:40<113:56:19,  5.52s/it]
 67%|██████▋   | 150698/225000 [295:56:46<115:05:14,  5.58s/it]
 67%|██████▋   | 150699/225000 [295:56:52<120:37:45,  5.84s/it]
 67%|██████▋   | 150700/225000 [295:56:58<117:21:52,  5.69s/it]
                                                               
{'loss': '0.801', 'grad_norm': '1.119', 'learning_rate': '1.336e-05', 'epoch': '0.6698', 'num_input_tokens_seen': 490300080, 'train_runtime': '1.065e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150700/225000 [295:56:58<117:21:52,  5.69s/it]
 67%|██████▋   | 150701/225000 [295:57:02<106:31:07,  5.16s/it]
 67%|██████▋   | 150702/225000 [295:57:07<109:39:59,  5.31s/it]
 67%|██████▋   | 150703/225000 [295:57:14<117:52:10,  5.71s/it]
 67%|██████▋   | 150704/225000 [295:57:22<129:51:18,  6.29s/it]
 67%|██████▋   | 150705/225000 [295:57:25<112:43:00,  5.46s/it]
 67%|██████▋   | 150706/225000 [295:57:33<128:12:03,  6.21s/it]
 67%|██████▋   | 150707/225000 [295:57:37<113:13:33,  5.49s/it]
 67%|██████▋   | 150708/225000 [295:57:41<102:14:23,  4.95s/it]
 67%|██████▋   | 150709/225000 [295:57:51<132:59:25,  6.44s/it]
 67%|██████▋   | 150710/225000 [295:57:54<117:28:35,  5.69s/it]
 67%|██████▋   | 150711/225000 [295:58:01<124:48:59,  6.05s/it]
 67%|██████▋   | 150712/225000 [295:58:08<128:45:06,  6.24s/it]
 67%|██████▋   | 150713/225000 [295:58:12<112:06:08,  5.43s/it]
 67%|██████▋   | 150714/225000 [295:58:20<130:38:07,  6.33s/it]
 67%|██████▋   | 150715/225000 [295:58:23<112:43:51,  5.46s/it]
 67%|██████▋   | 150716/225000 [295:58:31<123:12:57,  5.97s/it]
 67%|██████▋   | 150717/225000 [295:58:36<119:25:02,  5.79s/it]
 67%|██████▋   | 150718/225000 [295:58:42<122:46:42,  5.95s/it]
 67%|██████▋   | 150719/225000 [295:58:47<117:05:47,  5.68s/it]
 67%|██████▋   | 150720/225000 [295:58:54<120:48:15,  5.85s/it]
 67%|██████▋   | 150721/225000 [295:58:57<107:13:44,  5.20s/it]
 67%|██████▋   | 150722/225000 [295:59:02<101:38:29,  4.93s/it]
 67%|██████▋   | 150723/225000 [295:59:10<124:30:46,  6.03s/it]
 67%|██████▋   | 150724/225000 [295:59:14<110:00:16,  5.33s/it]
 67%|██████▋   | 150725/225000 [295:59:18<102:44:27,  4.98s/it]
 67%|██████▋   | 150726/225000 [295:59:23<103:19:02,  5.01s/it]
 67%|██████▋   | 150727/225000 [295:59:28<103:43:35,  5.03s/it]
 67%|██████▋   | 150728/225000 [295:59:33<105:40:02,  5.12s/it]
 67%|██████▋   | 150729/225000 [295:59:39<106:14:42,  5.15s/it]
 67%|██████▋   | 150730/225000 [295:59:44<106:04:45,  5.14s/it]
 67%|██████▋   | 150731/225000 [295:59:50<110:46:48,  5.37s/it]
 67%|██████▋   | 150732/225000 [295:59:55<112:04:34,  5.43s/it]
 67%|██████▋   | 150733/225000 [296:00:00<110:28:32,  5.36s/it]
 67%|██████▋   | 150734/225000 [296:00:06<109:28:51,  5.31s/it]
 67%|██████▋   | 150735/225000 [296:00:11<109:09:36,  5.29s/it]
 67%|██████▋   | 150736/225000 [296:00:21<141:22:01,  6.85s/it]
 67%|██████▋   | 150737/225000 [296:00:27<134:17:31,  6.51s/it]
 67%|██████▋   | 150738/225000 [296:00:35<140:12:51,  6.80s/it]
 67%|██████▋   | 150739/225000 [296:00:39<123:13:40,  5.97s/it]
 67%|██████▋   | 150740/225000 [296:00:43<111:58:25,  5.43s/it]
 67%|██████▋   | 150741/225000 [296:00:53<140:34:02,  6.81s/it]
 67%|██████▋   | 150742/225000 [296:00:57<122:28:55,  5.94s/it]
 67%|██████▋   | 150743/225000 [296:01:01<109:50:13,  5.32s/it]
 67%|██████▋   | 150744/225000 [296:01:05<103:53:49,  5.04s/it]
 67%|██████▋   | 150745/225000 [296:01:16<142:32:47,  6.91s/it]
 67%|██████▋   | 150746/225000 [296:01:21<126:05:27,  6.11s/it]
 67%|██████▋   | 150747/225000 [296:01:25<116:20:05,  5.64s/it]
 67%|██████▋   | 150748/225000 [296:01:30<109:46:43,  5.32s/it]
 67%|██████▋   | 150749/225000 [296:01:34<104:49:03,  5.08s/it]
 67%|██████▋   | 150750/225000 [296:01:39<104:51:12,  5.08s/it]
                                                               
{'loss': '0.7797', 'grad_norm': '1.196', 'learning_rate': '1.335e-05', 'epoch': '0.67', 'num_input_tokens_seen': 490430976, 'train_runtime': '1.066e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150750/225000 [296:01:39<104:51:12,  5.08s/it]
 67%|██████▋   | 150751/225000 [296:01:51<143:47:39,  6.97s/it]
 67%|██████▋   | 150752/225000 [296:01:55<124:29:15,  6.04s/it]
 67%|██████▋   | 150753/225000 [296:01:58<110:51:47,  5.38s/it]
 67%|██████▋   | 150754/225000 [296:02:02<101:49:04,  4.94s/it]
 67%|██████▋   | 150755/225000 [296:02:07<100:04:38,  4.85s/it]
 67%|██████▋   | 150756/225000 [296:02:12<101:02:51,  4.90s/it]
 67%|██████▋   | 150757/225000 [296:02:17<99:12:24,  4.81s/it] 
 67%|██████▋   | 150758/225000 [296:02:22<102:39:52,  4.98s/it]
 67%|██████▋   | 150759/225000 [296:02:34<147:21:36,  7.15s/it]
 67%|██████▋   | 150760/225000 [296:02:42<149:11:48,  7.23s/it]
 67%|██████▋   | 150761/225000 [296:02:45<127:32:18,  6.18s/it]
 67%|██████▋   | 150762/225000 [296:02:54<141:34:08,  6.87s/it]
 67%|██████▋   | 150763/225000 [296:02:59<133:41:29,  6.48s/it]
 67%|██████▋   | 150764/225000 [296:03:06<132:23:04,  6.42s/it]
 67%|██████▋   | 150765/225000 [296:03:10<118:10:46,  5.73s/it]
 67%|██████▋   | 150766/225000 [296:03:14<106:34:10,  5.17s/it]
 67%|██████▋   | 150767/225000 [296:03:17<98:45:07,  4.79s/it] 
 67%|██████▋   | 150768/225000 [296:03:22<96:40:51,  4.69s/it]
 67%|██████▋   | 150769/225000 [296:03:34<139:42:27,  6.78s/it]
 67%|██████▋   | 150770/225000 [296:03:37<119:13:14,  5.78s/it]
 67%|██████▋   | 150771/225000 [296:03:42<117:04:14,  5.68s/it]
 67%|██████▋   | 150772/225000 [296:03:47<112:56:54,  5.48s/it]
 67%|██████▋   | 150773/225000 [296:03:53<110:13:25,  5.35s/it]
 67%|██████▋   | 150774/225000 [296:03:58<109:56:09,  5.33s/it]
 67%|██████▋   | 150775/225000 [296:04:03<108:49:39,  5.28s/it]
 67%|██████▋   | 150776/225000 [296:04:08<106:13:30,  5.15s/it]
 67%|██████▋   | 150777/225000 [296:04:13<105:43:17,  5.13s/it]
 67%|██████▋   | 150778/225000 [296:04:18<106:07:40,  5.15s/it]
 67%|██████▋   | 150779/225000 [296:04:23<104:29:20,  5.07s/it]
 67%|██████▋   | 150780/225000 [296:04:28<106:46:36,  5.18s/it]
 67%|██████▋   | 150781/225000 [296:04:37<126:14:10,  6.12s/it]
 67%|██████▋   | 150782/225000 [296:04:47<150:59:19,  7.32s/it]
 67%|██████▋   | 150783/225000 [296:04:52<137:24:45,  6.67s/it]
 67%|██████▋   | 150784/225000 [296:04:57<126:18:36,  6.13s/it]
 67%|██████▋   | 150785/225000 [296:05:02<119:04:00,  5.78s/it]
 67%|██████▋   | 150786/225000 [296:05:06<111:56:31,  5.43s/it]
 67%|██████▋   | 150787/225000 [296:05:12<110:03:39,  5.34s/it]
 67%|██████▋   | 150788/225000 [296:05:16<104:40:34,  5.08s/it]
 67%|██████▋   | 150789/225000 [296:05:21<103:28:27,  5.02s/it]
 67%|██████▋   | 150790/225000 [296:05:25<100:41:18,  4.88s/it]
 67%|██████▋   | 150791/225000 [296:05:37<143:22:06,  6.96s/it]
 67%|██████▋   | 150792/225000 [296:05:45<146:12:03,  7.09s/it]
 67%|██████▋   | 150793/225000 [296:05:49<128:12:11,  6.22s/it]
 67%|██████▋   | 150794/225000 [296:05:53<113:29:30,  5.51s/it]
 67%|██████▋   | 150795/225000 [296:05:57<103:48:41,  5.04s/it]
 67%|██████▋   | 150796/225000 [296:06:07<137:13:50,  6.66s/it]
 67%|██████▋   | 150797/225000 [296:06:13<132:46:29,  6.44s/it]
 67%|██████▋   | 150798/225000 [296:06:17<116:10:05,  5.64s/it]
 67%|██████▋   | 150799/225000 [296:06:24<126:08:54,  6.12s/it]
 67%|██████▋   | 150800/225000 [296:06:30<124:03:02,  6.02s/it]
                                                               
{'loss': '0.7944', 'grad_norm': '1.127', 'learning_rate': '1.333e-05', 'epoch': '0.6702', 'num_input_tokens_seen': 490561760, 'train_runtime': '1.066e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150800/225000 [296:06:30<124:03:02,  6.02s/it]
 67%|██████▋   | 150801/225000 [296:06:34<113:50:36,  5.52s/it]
 67%|██████▋   | 150802/225000 [296:06:41<122:14:30,  5.93s/it]
 67%|██████▋   | 150803/225000 [296:06:45<111:39:54,  5.42s/it]
 67%|██████▋   | 150804/225000 [296:06:53<124:00:20,  6.02s/it]
 67%|██████▋   | 150805/225000 [296:06:57<110:53:33,  5.38s/it]
 67%|██████▋   | 150806/225000 [296:07:01<101:50:06,  4.94s/it]
 67%|██████▋   | 150807/225000 [296:07:11<138:24:04,  6.72s/it]
 67%|██████▋   | 150808/225000 [296:07:15<118:13:41,  5.74s/it]
 67%|██████▋   | 150809/225000 [296:07:18<104:19:57,  5.06s/it]
 67%|██████▋   | 150810/225000 [296:07:26<123:33:34,  6.00s/it]
 67%|██████▋   | 150811/225000 [296:07:33<127:24:14,  6.18s/it]
 67%|██████▋   | 150812/225000 [296:07:38<118:30:55,  5.75s/it]
 67%|██████▋   | 150813/225000 [296:07:43<114:19:00,  5.55s/it]
 67%|██████▋   | 150814/225000 [296:07:47<107:47:24,  5.23s/it]
 67%|██████▋   | 150815/225000 [296:07:52<106:19:33,  5.16s/it]
 67%|██████▋   | 150816/225000 [296:07:57<103:03:25,  5.00s/it]
 67%|██████▋   | 150817/225000 [296:08:02<100:43:35,  4.89s/it]
 67%|██████▋   | 150818/225000 [296:08:07<100:47:13,  4.89s/it]
 67%|██████▋   | 150819/225000 [296:08:19<146:34:28,  7.11s/it]
 67%|██████▋   | 150820/225000 [296:08:23<125:54:44,  6.11s/it]
 67%|██████▋   | 150821/225000 [296:08:27<113:00:12,  5.48s/it]
 67%|██████▋   | 150822/225000 [296:08:36<138:47:47,  6.74s/it]
 67%|██████▋   | 150823/225000 [296:08:40<119:34:02,  5.80s/it]
 67%|██████▋   | 150824/225000 [296:08:44<110:01:28,  5.34s/it]
 67%|██████▋   | 150825/225000 [296:08:52<127:54:06,  6.21s/it]
 67%|██████▋   | 150826/225000 [296:08:56<112:29:24,  5.46s/it]
 67%|██████▋   | 150827/225000 [296:09:04<127:06:26,  6.17s/it]
 67%|██████▋   | 150828/225000 [296:09:07<110:23:26,  5.36s/it]
 67%|██████▋   | 150829/225000 [296:09:12<107:44:25,  5.23s/it]
 67%|██████▋   | 150830/225000 [296:09:18<109:13:57,  5.30s/it]
 67%|██████▋   | 150831/225000 [296:09:23<108:43:38,  5.28s/it]
 67%|██████▋   | 150832/225000 [296:09:28<107:50:13,  5.23s/it]
 67%|██████▋   | 150833/225000 [296:09:33<108:07:30,  5.25s/it]
 67%|██████▋   | 150834/225000 [296:09:39<108:47:39,  5.28s/it]
 67%|██████▋   | 150835/225000 [296:09:44<109:19:26,  5.31s/it]
 67%|██████▋   | 150836/225000 [296:09:50<110:13:22,  5.35s/it]
 67%|██████▋   | 150837/225000 [296:09:55<110:20:15,  5.36s/it]
 67%|██████▋   | 150838/225000 [296:10:01<112:37:36,  5.47s/it]
 67%|██████▋   | 150839/225000 [296:10:06<111:45:11,  5.42s/it]
 67%|██████▋   | 150840/225000 [296:10:19<157:17:17,  7.64s/it]
 67%|██████▋   | 150841/225000 [296:10:22<130:17:59,  6.33s/it]
 67%|██████▋   | 150842/225000 [296:10:26<114:33:44,  5.56s/it]
 67%|██████▋   | 150843/225000 [296:10:37<149:38:55,  7.26s/it]
 67%|██████▋   | 150844/225000 [296:10:41<127:19:01,  6.18s/it]
 67%|██████▋   | 150845/225000 [296:10:46<121:01:29,  5.88s/it]
 67%|██████▋   | 150846/225000 [296:10:52<119:16:55,  5.79s/it]
 67%|██████▋   | 150847/225000 [296:10:57<120:07:53,  5.83s/it]
 67%|██████▋   | 150848/225000 [296:11:04<123:40:47,  6.00s/it]
 67%|██████▋   | 150849/225000 [296:11:08<111:14:53,  5.40s/it]
 67%|██████▋   | 150850/225000 [296:11:14<115:27:52,  5.61s/it]
                                                               
{'loss': '0.7891', 'grad_norm': '1.065', 'learning_rate': '1.332e-05', 'epoch': '0.6704', 'num_input_tokens_seen': 490692072, 'train_runtime': '1.066e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150850/225000 [296:11:14<115:27:52,  5.61s/it]
 67%|██████▋   | 150851/225000 [296:11:23<137:03:18,  6.65s/it]
 67%|██████▋   | 150852/225000 [296:11:27<121:06:56,  5.88s/it]
 67%|██████▋   | 150853/225000 [296:11:31<110:37:36,  5.37s/it]
 67%|██████▋   | 150854/225000 [296:11:35<103:05:54,  5.01s/it]
 67%|██████▋   | 150855/225000 [296:11:46<137:09:34,  6.66s/it]
 67%|██████▋   | 150856/225000 [296:11:49<116:23:39,  5.65s/it]
 67%|██████▋   | 150857/225000 [296:11:57<127:07:34,  6.17s/it]
 67%|██████▋   | 150858/225000 [296:12:00<110:08:52,  5.35s/it]
 67%|██████▋   | 150859/225000 [296:12:08<128:02:39,  6.22s/it]
 67%|██████▋   | 150860/225000 [296:12:12<110:34:41,  5.37s/it]
 67%|██████▋   | 150861/225000 [296:12:17<111:14:52,  5.40s/it]
 67%|██████▋   | 150862/225000 [296:12:24<117:46:57,  5.72s/it]
 67%|██████▋   | 150863/225000 [296:12:27<104:57:48,  5.10s/it]
 67%|██████▋   | 150864/225000 [296:12:36<126:46:43,  6.16s/it]
 67%|██████▋   | 150865/225000 [296:12:40<115:30:29,  5.61s/it]
 67%|██████▋   | 150866/225000 [296:12:44<105:44:31,  5.13s/it]
 67%|██████▋   | 150867/225000 [296:12:50<110:49:08,  5.38s/it]
 67%|██████▋   | 150868/225000 [296:12:58<124:57:23,  6.07s/it]
 67%|██████▋   | 150869/225000 [296:13:02<110:24:35,  5.36s/it]
 67%|██████▋   | 150870/225000 [296:13:11<134:20:50,  6.52s/it]
 67%|██████▋   | 150871/225000 [296:13:15<118:12:18,  5.74s/it]
 67%|██████▋   | 150872/225000 [296:13:18<102:33:42,  4.98s/it]
 67%|██████▋   | 150873/225000 [296:13:27<125:53:09,  6.11s/it]
 67%|██████▋   | 150874/225000 [296:13:31<111:25:00,  5.41s/it]
 67%|██████▋   | 150875/225000 [296:13:34<101:35:13,  4.93s/it]
 67%|██████▋   | 150876/225000 [296:13:44<133:40:05,  6.49s/it]
 67%|██████▋   | 150877/225000 [296:13:48<117:06:55,  5.69s/it]
 67%|██████▋   | 150878/225000 [296:13:54<117:56:21,  5.73s/it]
 67%|██████▋   | 150879/225000 [296:14:02<131:41:08,  6.40s/it]
 67%|██████▋   | 150880/225000 [296:14:08<126:43:33,  6.16s/it]
 67%|██████▋   | 150881/225000 [296:14:11<110:22:59,  5.36s/it]
 67%|██████▋   | 150882/225000 [296:14:15<101:14:01,  4.92s/it]
 67%|██████▋   | 150883/225000 [296:14:19<95:45:04,  4.65s/it] 
 67%|██████▋   | 150884/225000 [296:14:30<134:38:31,  6.54s/it]
 67%|██████▋   | 150885/225000 [296:14:34<119:15:38,  5.79s/it]
 67%|██████▋   | 150886/225000 [296:14:40<120:03:33,  5.83s/it]
 67%|██████▋   | 150887/225000 [296:14:44<108:06:56,  5.25s/it]
 67%|██████▋   | 150888/225000 [296:14:51<119:17:07,  5.79s/it]
 67%|██████▋   | 150889/225000 [296:14:59<130:24:00,  6.33s/it]
 67%|██████▋   | 150890/225000 [296:15:02<113:53:27,  5.53s/it]
 67%|██████▋   | 150891/225000 [296:15:06<104:38:33,  5.08s/it]
 67%|██████▋   | 150892/225000 [296:15:16<133:41:00,  6.49s/it]
 67%|██████▋   | 150893/225000 [296:15:20<119:11:38,  5.79s/it]
 67%|██████▋   | 150894/225000 [296:15:24<109:17:33,  5.31s/it]
 67%|██████▋   | 150895/225000 [296:15:29<102:10:46,  4.96s/it]
 67%|██████▋   | 150896/225000 [296:15:33<99:32:09,  4.84s/it] 
 67%|██████▋   | 150897/225000 [296:15:40<113:59:37,  5.54s/it]
 67%|██████▋   | 150898/225000 [296:15:51<145:02:12,  7.05s/it]
 67%|██████▋   | 150899/225000 [296:15:55<124:20:46,  6.04s/it]
 67%|██████▋   | 150900/225000 [296:15:59<117:38:36,  5.72s/it]
                                                               
{'loss': '0.7779', 'grad_norm': '1.084', 'learning_rate': '1.33e-05', 'epoch': '0.6707', 'num_input_tokens_seen': 490822336, 'train_runtime': '1.067e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150900/225000 [296:15:59<117:38:36,  5.72s/it]
 67%|██████▋   | 150901/225000 [296:16:05<114:46:08,  5.58s/it]
 67%|██████▋   | 150902/225000 [296:16:10<110:55:37,  5.39s/it]
 67%|██████▋   | 150903/225000 [296:16:14<105:26:34,  5.12s/it]
 67%|██████▋   | 150904/225000 [296:16:19<106:31:21,  5.18s/it]
 67%|██████▋   | 150905/225000 [296:16:25<106:06:47,  5.16s/it]
 67%|██████▋   | 150906/225000 [296:16:29<103:29:21,  5.03s/it]
 67%|██████▋   | 150907/225000 [296:16:34<101:40:43,  4.94s/it]
 67%|██████▋   | 150908/225000 [296:16:44<131:35:59,  6.39s/it]
 67%|██████▋   | 150909/225000 [296:16:54<153:24:02,  7.45s/it]
 67%|██████▋   | 150910/225000 [296:16:58<130:40:01,  6.35s/it]
 67%|██████▋   | 150911/225000 [296:17:04<131:08:30,  6.37s/it]
 67%|██████▋   | 150912/225000 [296:17:10<131:15:26,  6.38s/it]
 67%|██████▋   | 150913/225000 [296:17:15<117:43:03,  5.72s/it]
 67%|██████▋   | 150914/225000 [296:17:18<106:35:40,  5.18s/it]
 67%|██████▋   | 150915/225000 [296:17:24<106:48:43,  5.19s/it]
 67%|██████▋   | 150916/225000 [296:17:29<106:58:12,  5.20s/it]
 67%|██████▋   | 150917/225000 [296:17:34<106:31:52,  5.18s/it]
 67%|██████▋   | 150918/225000 [296:17:39<106:16:21,  5.16s/it]
 67%|██████▋   | 150919/225000 [296:17:44<102:53:10,  5.00s/it]
 67%|██████▋   | 150920/225000 [296:17:49<102:53:02,  5.00s/it]
 67%|██████▋   | 150921/225000 [296:18:01<147:50:49,  7.18s/it]
 67%|██████▋   | 150922/225000 [296:18:06<132:47:01,  6.45s/it]
 67%|██████▋   | 150923/225000 [296:18:09<115:24:45,  5.61s/it]
 67%|██████▋   | 150924/225000 [296:18:17<130:04:01,  6.32s/it]
 67%|██████▋   | 150925/225000 [296:18:23<123:00:05,  5.98s/it]
 67%|██████▋   | 150926/225000 [296:18:30<130:51:29,  6.36s/it]
 67%|██████▋   | 150927/225000 [296:18:35<123:49:28,  6.02s/it]
 67%|██████▋   | 150928/225000 [296:18:40<115:53:48,  5.63s/it]
 67%|██████▋   | 150929/225000 [296:18:45<111:32:54,  5.42s/it]
 67%|██████▋   | 150930/225000 [296:18:50<108:55:20,  5.29s/it]
 67%|██████▋   | 150931/225000 [296:18:55<106:58:31,  5.20s/it]
 67%|██████▋   | 150932/225000 [296:19:00<105:56:53,  5.15s/it]
 67%|██████▋   | 150933/225000 [296:19:05<106:59:02,  5.20s/it]
 67%|██████▋   | 150934/225000 [296:19:10<105:31:40,  5.13s/it]
 67%|██████▋   | 150935/225000 [296:19:15<106:03:57,  5.16s/it]
 67%|██████▋   | 150936/225000 [296:19:20<106:24:46,  5.17s/it]
 67%|██████▋   | 150937/225000 [296:19:27<117:16:39,  5.70s/it]
 67%|██████▋   | 150938/225000 [296:19:37<143:51:46,  6.99s/it]
 67%|██████▋   | 150939/225000 [296:19:42<128:27:58,  6.24s/it]
 67%|██████▋   | 150940/225000 [296:19:46<117:26:59,  5.71s/it]
 67%|██████▋   | 150941/225000 [296:19:50<106:39:18,  5.18s/it]
 67%|██████▋   | 150942/225000 [296:19:54<99:44:33,  4.85s/it] 
 67%|██████▋   | 150943/225000 [296:20:06<139:45:23,  6.79s/it]
 67%|██████▋   | 150944/225000 [296:20:11<127:36:22,  6.20s/it]
 67%|██████▋   | 150945/225000 [296:20:15<119:41:41,  5.82s/it]
 67%|██████▋   | 150946/225000 [296:20:20<113:39:25,  5.53s/it]
 67%|██████▋   | 150947/225000 [296:20:26<111:58:49,  5.44s/it]
 67%|██████▋   | 150948/225000 [296:20:31<110:16:01,  5.36s/it]
 67%|██████▋   | 150949/225000 [296:20:36<107:56:30,  5.25s/it]
 67%|██████▋   | 150950/225000 [296:20:41<108:40:37,  5.28s/it]
                                                               
{'loss': '0.7753', 'grad_norm': '1.156', 'learning_rate': '1.328e-05', 'epoch': '0.6709', 'num_input_tokens_seen': 490952608, 'train_runtime': '1.067e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 150950/225000 [296:20:41<108:40:37,  5.28s/it]
 67%|██████▋   | 150951/225000 [296:20:46<106:25:43,  5.17s/it]
 67%|██████▋   | 150952/225000 [296:20:51<102:50:26,  5.00s/it]
 67%|██████▋   | 150953/225000 [296:20:56<102:59:58,  5.01s/it]
 67%|██████▋   | 150954/225000 [296:21:01<103:51:00,  5.05s/it]
 67%|██████▋   | 150955/225000 [296:21:06<105:22:40,  5.12s/it]
 67%|██████▋   | 150956/225000 [296:21:11<103:29:32,  5.03s/it]
 67%|██████▋   | 150957/225000 [296:21:16<103:33:52,  5.04s/it]
 67%|██████▋   | 150958/225000 [296:21:24<122:01:54,  5.93s/it]
 67%|██████▋   | 150959/225000 [296:21:37<164:12:36,  7.98s/it]
 67%|██████▋   | 150960/225000 [296:21:40<135:24:45,  6.58s/it]
 67%|██████▋   | 150961/225000 [296:21:45<126:00:49,  6.13s/it]
 67%|██████▋   | 150962/225000 [296:21:51<122:43:23,  5.97s/it]
 67%|██████▋   | 150963/225000 [296:21:59<134:11:50,  6.53s/it]
 67%|██████▋   | 150964/225000 [296:22:02<114:42:37,  5.58s/it]
 67%|██████▋   | 150965/225000 [296:22:08<119:08:07,  5.79s/it]
 67%|██████▋   | 150966/225000 [296:22:13<112:02:10,  5.45s/it]
 67%|██████▋   | 150967/225000 [296:22:18<107:53:14,  5.25s/it]
 67%|██████▋   | 150968/225000 [296:22:23<105:53:56,  5.15s/it]
 67%|██████▋   | 150969/225000 [296:22:27<102:46:34,  5.00s/it]
 67%|██████▋   | 150970/225000 [296:22:32<101:26:32,  4.93s/it]
 67%|██████▋   | 150971/225000 [296:22:37<102:18:27,  4.98s/it]
 67%|██████▋   | 150972/225000 [296:22:42<101:47:13,  4.95s/it]
 67%|██████▋   | 150973/225000 [296:22:47<103:09:34,  5.02s/it]
 67%|██████▋   | 150974/225000 [296:22:55<119:22:22,  5.81s/it]
 67%|██████▋   | 150975/225000 [296:23:06<153:54:50,  7.49s/it]
 67%|██████▋   | 150976/225000 [296:23:10<131:16:51,  6.38s/it]
 67%|██████▋   | 150977/225000 [296:23:15<123:44:35,  6.02s/it]
 67%|██████▋   | 150978/225000 [296:23:21<121:17:24,  5.90s/it]
 67%|██████▋   | 150979/225000 [296:23:25<113:59:15,  5.54s/it]
 67%|██████▋   | 150980/225000 [296:23:30<109:57:20,  5.35s/it]
 67%|██████▋   | 150981/225000 [296:23:35<106:42:47,  5.19s/it]
 67%|██████▋   | 150982/225000 [296:23:40<106:50:21,  5.20s/it]
 67%|██████▋   | 150983/225000 [296:23:46<106:55:06,  5.20s/it]
 67%|██████▋   | 150984/225000 [296:23:51<105:18:55,  5.12s/it]
 67%|██████▋   | 150985/225000 [296:23:56<106:58:57,  5.20s/it]
 67%|██████▋   | 150986/225000 [296:24:03<120:10:09,  5.84s/it]
 67%|██████▋   | 150987/225000 [296:24:13<146:58:12,  7.15s/it]
 67%|██████▋   | 150988/225000 [296:24:17<126:55:24,  6.17s/it]
 67%|██████▋   | 150989/225000 [296:24:21<114:16:53,  5.56s/it]
 67%|██████▋   | 150990/225000 [296:24:27<111:20:16,  5.42s/it]
 67%|██████▋   | 150991/225000 [296:24:32<112:48:33,  5.49s/it]
 67%|██████▋   | 150992/225000 [296:24:38<113:35:01,  5.53s/it]
 67%|██████▋   | 150993/225000 [296:24:47<137:49:09,  6.70s/it]
 67%|██████▋   | 150994/225000 [296:24:52<128:29:12,  6.25s/it]
 67%|██████▋   | 150995/225000 [296:25:00<137:39:13,  6.70s/it]
 67%|██████▋   | 150996/225000 [296:25:05<125:40:48,  6.11s/it]
 67%|██████▋   | 150997/225000 [296:25:10<116:08:05,  5.65s/it]
 67%|██████▋   | 150998/225000 [296:25:14<107:25:32,  5.23s/it]
 67%|██████▋   | 150999/225000 [296:25:18<103:40:46,  5.04s/it]
 67%|██████▋   | 151000/225000 [296:25:23<103:05:35,  5.02s/it]
                                                               
{'loss': '0.792', 'grad_norm': '1.026', 'learning_rate': '1.327e-05', 'epoch': '0.6711', 'num_input_tokens_seen': 491081904, 'train_runtime': '1.067e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151000/225000 [296:25:23<103:05:35,  5.02s/it]
 67%|██████▋   | 151001/225000 [296:25:28<103:55:36,  5.06s/it]
 67%|██████▋   | 151002/225000 [296:25:40<146:30:17,  7.13s/it]
 67%|██████▋   | 151003/225000 [296:25:44<124:48:43,  6.07s/it]
 67%|██████▋   | 151004/225000 [296:25:52<133:52:24,  6.51s/it]
 67%|██████▋   | 151005/225000 [296:25:56<119:01:56,  5.79s/it]
 67%|██████▋   | 151006/225000 [296:26:00<108:59:28,  5.30s/it]
 67%|██████▋   | 151007/225000 [296:26:04<99:44:55,  4.85s/it] 
 67%|██████▋   | 151008/225000 [296:26:14<134:24:31,  6.54s/it]
 67%|██████▋   | 151009/225000 [296:26:18<120:59:52,  5.89s/it]
 67%|██████▋   | 151010/225000 [296:26:25<123:55:11,  6.03s/it]
 67%|██████▋   | 151011/225000 [296:26:29<114:15:47,  5.56s/it]
 67%|██████▋   | 151012/225000 [296:26:34<109:49:53,  5.34s/it]
 67%|██████▋   | 151013/225000 [296:26:39<105:43:25,  5.14s/it]
 67%|██████▋   | 151014/225000 [296:26:44<102:56:58,  5.01s/it]
 67%|██████▋   | 151015/225000 [296:26:48<101:48:00,  4.95s/it]
 67%|██████▋   | 151016/225000 [296:26:53<101:25:59,  4.94s/it]
 67%|██████▋   | 151017/225000 [296:27:00<109:57:49,  5.35s/it]
 67%|██████▋   | 151018/225000 [296:27:10<143:26:43,  6.98s/it]
 67%|██████▋   | 151019/225000 [296:27:15<127:17:12,  6.19s/it]
 67%|██████▋   | 151020/225000 [296:27:21<130:51:00,  6.37s/it]
 67%|██████▋   | 151021/225000 [296:27:26<119:12:29,  5.80s/it]
 67%|██████▋   | 151022/225000 [296:27:30<109:23:47,  5.32s/it]
 67%|██████▋   | 151023/225000 [296:27:34<101:45:23,  4.95s/it]
 67%|██████▋   | 151024/225000 [296:27:44<134:09:23,  6.53s/it]
 67%|██████▋   | 151025/225000 [296:27:49<122:21:20,  5.95s/it]
 67%|██████▋   | 151026/225000 [296:27:53<109:40:24,  5.34s/it]
 67%|██████▋   | 151027/225000 [296:27:57<104:26:52,  5.08s/it]
 67%|██████▋   | 151028/225000 [296:28:09<142:39:56,  6.94s/it]
 67%|██████▋   | 151029/225000 [296:28:13<128:22:03,  6.25s/it]
 67%|██████▋   | 151030/225000 [296:28:17<113:45:41,  5.54s/it]
 67%|██████▋   | 151031/225000 [296:28:22<109:46:30,  5.34s/it]
 67%|██████▋   | 151032/225000 [296:28:26<103:30:14,  5.04s/it]
 67%|██████▋   | 151033/225000 [296:28:31<99:20:25,  4.83s/it] 
 67%|██████▋   | 151034/225000 [296:28:35<96:23:33,  4.69s/it]
 67%|██████▋   | 151035/225000 [296:28:45<125:27:57,  6.11s/it]
 67%|██████▋   | 151036/225000 [296:28:53<140:50:09,  6.85s/it]
 67%|██████▋   | 151037/225000 [296:28:58<129:18:14,  6.29s/it]
 67%|██████▋   | 151038/225000 [296:29:04<123:40:30,  6.02s/it]
 67%|██████▋   | 151039/225000 [296:29:08<114:29:53,  5.57s/it]
 67%|██████▋   | 151040/225000 [296:29:13<107:34:12,  5.24s/it]
 67%|██████▋   | 151041/225000 [296:29:17<104:06:44,  5.07s/it]
 67%|██████▋   | 151042/225000 [296:29:23<108:57:14,  5.30s/it]
 67%|██████▋   | 151043/225000 [296:29:32<133:43:34,  6.51s/it]
 67%|██████▋   | 151044/225000 [296:29:36<117:39:48,  5.73s/it]
 67%|██████▋   | 151045/225000 [296:29:41<108:50:50,  5.30s/it]
 67%|██████▋   | 151046/225000 [296:29:44<99:11:54,  4.83s/it] 
 67%|██████▋   | 151047/225000 [296:29:53<120:56:18,  5.89s/it]
 67%|██████▋   | 151048/225000 [296:29:59<123:48:02,  6.03s/it]
 67%|██████▋   | 151049/225000 [296:30:03<110:21:43,  5.37s/it]
 67%|██████▋   | 151050/225000 [296:30:12<130:23:04,  6.35s/it]
                                                               
{'loss': '0.7794', 'grad_norm': '1.179', 'learning_rate': '1.325e-05', 'epoch': '0.6713', 'num_input_tokens_seen': 491212440, 'train_runtime': '1.067e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151050/225000 [296:30:12<130:23:04,  6.35s/it]
 67%|██████▋   | 151051/225000 [296:30:15<113:41:26,  5.53s/it]
 67%|██████▋   | 151052/225000 [296:30:24<135:02:37,  6.57s/it]
 67%|██████▋   | 151053/225000 [296:30:27<114:51:10,  5.59s/it]
 67%|██████▋   | 151054/225000 [296:30:35<127:16:24,  6.20s/it]
 67%|██████▋   | 151055/225000 [296:30:39<115:56:02,  5.64s/it]
 67%|██████▋   | 151056/225000 [296:30:44<106:40:20,  5.19s/it]
 67%|██████▋   | 151057/225000 [296:30:48<100:49:54,  4.91s/it]
 67%|██████▋   | 151058/225000 [296:30:53<100:28:55,  4.89s/it]
 67%|██████▋   | 151059/225000 [296:31:03<131:24:13,  6.40s/it]
 67%|██████▋   | 151060/225000 [296:31:07<118:53:39,  5.79s/it]
 67%|██████▋   | 151061/225000 [296:31:12<112:11:03,  5.46s/it]
 67%|██████▋   | 151062/225000 [296:31:17<110:04:09,  5.36s/it]
 67%|██████▋   | 151063/225000 [296:31:22<108:18:24,  5.27s/it]
 67%|██████▋   | 151064/225000 [296:31:26<101:59:38,  4.97s/it]
 67%|██████▋   | 151065/225000 [296:31:36<129:59:12,  6.33s/it]
 67%|██████▋   | 151066/225000 [296:31:43<133:40:42,  6.51s/it]
 67%|██████▋   | 151067/225000 [296:31:47<120:15:38,  5.86s/it]
 67%|██████▋   | 151068/225000 [296:31:54<128:45:57,  6.27s/it]
 67%|██████▋   | 151069/225000 [296:31:59<120:38:14,  5.87s/it]
 67%|██████▋   | 151070/225000 [296:32:04<117:14:13,  5.71s/it]
 67%|██████▋   | 151071/225000 [296:32:09<109:47:39,  5.35s/it]
 67%|██████▋   | 151072/225000 [296:32:16<123:07:44,  6.00s/it]
 67%|██████▋   | 151073/225000 [296:32:20<111:14:44,  5.42s/it]
 67%|██████▋   | 151074/225000 [296:32:26<110:25:41,  5.38s/it]
 67%|██████▋   | 151075/225000 [296:32:31<108:41:57,  5.29s/it]
 67%|██████▋   | 151076/225000 [296:32:36<108:16:57,  5.27s/it]
 67%|██████▋   | 151077/225000 [296:32:41<107:53:52,  5.25s/it]
 67%|██████▋   | 151078/225000 [296:32:46<106:05:41,  5.17s/it]
 67%|██████▋   | 151079/225000 [296:32:51<106:20:04,  5.18s/it]
 67%|██████▋   | 151080/225000 [296:33:02<141:22:57,  6.89s/it]
 67%|██████▋   | 151081/225000 [296:33:09<142:35:59,  6.94s/it]
 67%|██████▋   | 151082/225000 [296:33:13<121:53:56,  5.94s/it]
 67%|██████▋   | 151083/225000 [296:33:19<121:19:10,  5.91s/it]
 67%|██████▋   | 151084/225000 [296:33:27<133:02:02,  6.48s/it]
 67%|██████▋   | 151085/225000 [296:33:31<120:04:27,  5.85s/it]
 67%|██████▋   | 151086/225000 [296:33:36<112:25:45,  5.48s/it]
 67%|██████▋   | 151087/225000 [296:33:40<107:55:25,  5.26s/it]
 67%|██████▋   | 151088/225000 [296:33:45<104:05:04,  5.07s/it]
 67%|██████▋   | 151089/225000 [296:33:57<146:21:12,  7.13s/it]
 67%|██████▋   | 151090/225000 [296:34:00<122:11:31,  5.95s/it]
 67%|██████▋   | 151091/225000 [296:34:07<126:18:30,  6.15s/it]
 67%|██████▋   | 151092/225000 [296:34:10<110:44:54,  5.39s/it]
 67%|██████▋   | 151093/225000 [296:34:15<104:57:59,  5.11s/it]
 67%|██████▋   | 151094/225000 [296:34:19<100:39:45,  4.90s/it]
 67%|██████▋   | 151095/225000 [296:34:24<98:28:31,  4.80s/it] 
 67%|██████▋   | 151096/225000 [296:34:29<98:27:44,  4.80s/it]
 67%|██████▋   | 151097/225000 [296:34:33<97:09:55,  4.73s/it]
 67%|██████▋   | 151098/225000 [296:34:41<113:58:29,  5.55s/it]
 67%|██████▋   | 151099/225000 [296:34:53<153:29:44,  7.48s/it]
 67%|██████▋   | 151100/225000 [296:34:56<127:22:16,  6.20s/it]
                                                               
{'loss': '0.7827', 'grad_norm': '1.165', 'learning_rate': '1.324e-05', 'epoch': '0.6716', 'num_input_tokens_seen': 491342480, 'train_runtime': '1.068e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151100/225000 [296:34:56<127:22:16,  6.20s/it]
 67%|██████▋   | 151101/225000 [296:35:02<129:15:20,  6.30s/it]
 67%|██████▋   | 151102/225000 [296:35:07<121:16:34,  5.91s/it]
 67%|██████▋   | 151103/225000 [296:35:12<115:00:40,  5.60s/it]
 67%|██████▋   | 151104/225000 [296:35:17<110:04:27,  5.36s/it]
 67%|██████▋   | 151105/225000 [296:35:22<107:05:59,  5.22s/it]
 67%|██████▋   | 151106/225000 [296:35:27<106:19:19,  5.18s/it]
 67%|██████▋   | 151107/225000 [296:35:32<103:50:36,  5.06s/it]
 67%|██████▋   | 151108/225000 [296:35:37<104:29:46,  5.09s/it]
 67%|██████▋   | 151109/225000 [296:35:42<101:53:54,  4.96s/it]
 67%|██████▋   | 151110/225000 [296:35:47<104:14:56,  5.08s/it]
 67%|██████▋   | 151111/225000 [296:35:52<104:35:21,  5.10s/it]
 67%|██████▋   | 151112/225000 [296:35:57<105:37:05,  5.15s/it]
 67%|██████▋   | 151113/225000 [296:36:08<140:05:21,  6.83s/it]
 67%|██████▋   | 151114/225000 [296:36:15<139:40:01,  6.81s/it]
 67%|██████▋   | 151115/225000 [296:36:19<126:03:12,  6.14s/it]
 67%|██████▋   | 151116/225000 [296:36:25<119:36:37,  5.83s/it]
 67%|██████▋   | 151117/225000 [296:36:33<135:49:54,  6.62s/it]
 67%|██████▋   | 151118/225000 [296:36:38<125:29:42,  6.11s/it]
 67%|██████▋   | 151119/225000 [296:36:42<111:27:35,  5.43s/it]
 67%|██████▋   | 151120/225000 [296:36:49<119:58:18,  5.85s/it]
 67%|██████▋   | 151121/225000 [296:36:52<105:07:10,  5.12s/it]
 67%|██████▋   | 151122/225000 [296:36:58<112:27:44,  5.48s/it]
 67%|██████▋   | 151123/225000 [296:37:05<117:27:55,  5.72s/it]
 67%|██████▋   | 151124/225000 [296:37:10<116:59:10,  5.70s/it]
 67%|██████▋   | 151125/225000 [296:37:16<118:20:23,  5.77s/it]
 67%|██████▋   | 151126/225000 [296:37:22<116:06:06,  5.66s/it]
 67%|██████▋   | 151127/225000 [296:37:27<114:48:54,  5.60s/it]
 67%|██████▋   | 151128/225000 [296:37:32<112:31:22,  5.48s/it]
 67%|██████▋   | 151129/225000 [296:37:38<116:52:43,  5.70s/it]
 67%|██████▋   | 151130/225000 [296:37:44<114:19:35,  5.57s/it]
 67%|██████▋   | 151131/225000 [296:37:49<112:03:39,  5.46s/it]
 67%|██████▋   | 151132/225000 [296:37:54<110:33:09,  5.39s/it]
 67%|██████▋   | 151133/225000 [296:38:00<111:53:09,  5.45s/it]
 67%|██████▋   | 151134/225000 [296:38:05<111:46:43,  5.45s/it]
 67%|██████▋   | 151135/225000 [296:38:11<110:51:23,  5.40s/it]
 67%|██████▋   | 151136/225000 [296:38:16<110:18:25,  5.38s/it]
 67%|██████▋   | 151137/225000 [296:38:21<110:18:59,  5.38s/it]
 67%|██████▋   | 151138/225000 [296:38:27<110:34:28,  5.39s/it]
 67%|██████▋   | 151139/225000 [296:38:32<109:01:14,  5.31s/it]
 67%|██████▋   | 151140/225000 [296:38:37<108:22:09,  5.28s/it]
 67%|██████▋   | 151141/225000 [296:38:43<109:58:46,  5.36s/it]
 67%|██████▋   | 151142/225000 [296:38:48<110:23:47,  5.38s/it]
 67%|██████▋   | 151143/225000 [296:38:53<110:54:10,  5.41s/it]
 67%|██████▋   | 151144/225000 [296:38:58<108:59:34,  5.31s/it]
 67%|██████▋   | 151145/225000 [296:39:04<109:33:29,  5.34s/it]
 67%|██████▋   | 151146/225000 [296:39:13<134:31:54,  6.56s/it]
 67%|██████▋   | 151147/225000 [296:39:23<154:29:23,  7.53s/it]
 67%|██████▋   | 151148/225000 [296:39:27<130:55:10,  6.38s/it]
 67%|██████▋   | 151149/225000 [296:39:31<117:51:33,  5.75s/it]
 67%|██████▋   | 151150/225000 [296:39:36<113:43:28,  5.54s/it]
                                                               
{'loss': '0.7773', 'grad_norm': '0.9739', 'learning_rate': '1.322e-05', 'epoch': '0.6718', 'num_input_tokens_seen': 491473352, 'train_runtime': '1.068e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151150/225000 [296:39:36<113:43:28,  5.54s/it]
 67%|██████▋   | 151151/225000 [296:39:42<113:29:31,  5.53s/it]
 67%|██████▋   | 151152/225000 [296:39:46<106:47:36,  5.21s/it]
 67%|██████▋   | 151153/225000 [296:39:51<107:02:17,  5.22s/it]
 67%|██████▋   | 151154/225000 [296:39:57<107:24:52,  5.24s/it]
 67%|██████▋   | 151155/225000 [296:40:01<103:28:43,  5.04s/it]
 67%|██████▋   | 151156/225000 [296:40:11<131:01:46,  6.39s/it]
 67%|██████▋   | 151157/225000 [296:40:21<155:41:54,  7.59s/it]
 67%|██████▋   | 151158/225000 [296:40:25<133:36:22,  6.51s/it]
 67%|██████▋   | 151159/225000 [296:40:31<127:20:44,  6.21s/it]
 67%|██████▋   | 151160/225000 [296:40:35<117:28:35,  5.73s/it]
 67%|██████▋   | 151161/225000 [296:40:41<115:12:13,  5.62s/it]
 67%|██████▋   | 151162/225000 [296:40:46<112:17:03,  5.47s/it]
 67%|██████▋   | 151163/225000 [296:40:51<113:00:58,  5.51s/it]
 67%|██████▋   | 151164/225000 [296:40:58<118:36:38,  5.78s/it]
 67%|██████▋   | 151165/225000 [296:41:07<137:39:14,  6.71s/it]
 67%|██████▋   | 151166/225000 [296:41:12<129:47:25,  6.33s/it]
 67%|██████▋   | 151167/225000 [296:41:17<123:21:07,  6.01s/it]
 67%|██████▋   | 151168/225000 [296:41:23<120:01:49,  5.85s/it]
 67%|██████▋   | 151169/225000 [296:41:28<116:31:29,  5.68s/it]
 67%|██████▋   | 151170/225000 [296:41:33<112:15:41,  5.47s/it]
 67%|██████▋   | 151171/225000 [296:41:38<111:25:06,  5.43s/it]
 67%|██████▋   | 151172/225000 [296:41:44<111:47:03,  5.45s/it]
 67%|██████▋   | 151173/225000 [296:41:49<108:13:21,  5.28s/it]
 67%|██████▋   | 151174/225000 [296:41:54<109:45:00,  5.35s/it]
 67%|██████▋   | 151175/225000 [296:42:00<109:49:29,  5.36s/it]
 67%|██████▋   | 151176/225000 [296:42:06<113:06:01,  5.52s/it]
 67%|██████▋   | 151177/225000 [296:42:11<110:56:45,  5.41s/it]
 67%|██████▋   | 151178/225000 [296:42:16<108:12:45,  5.28s/it]
 67%|██████▋   | 151179/225000 [296:42:21<109:57:08,  5.36s/it]
 67%|██████▋   | 151180/225000 [296:42:27<111:20:17,  5.43s/it]
 67%|██████▋   | 151181/225000 [296:42:39<151:31:25,  7.39s/it]
 67%|██████▋   | 151182/225000 [296:42:44<138:19:01,  6.75s/it]
 67%|██████▋   | 151183/225000 [296:42:49<128:14:49,  6.25s/it]
 67%|██████▋   | 151184/225000 [296:42:55<123:38:36,  6.03s/it]
 67%|██████▋   | 151185/225000 [296:43:00<117:40:16,  5.74s/it]
 67%|██████▋   | 151186/225000 [296:43:05<115:47:08,  5.65s/it]
 67%|██████▋   | 151187/225000 [296:43:11<115:50:49,  5.65s/it]
 67%|██████▋   | 151188/225000 [296:43:17<118:02:54,  5.76s/it]
 67%|██████▋   | 151189/225000 [296:43:22<116:51:49,  5.70s/it]
 67%|██████▋   | 151190/225000 [296:43:28<114:47:30,  5.60s/it]
 67%|██████▋   | 151191/225000 [296:43:33<112:58:58,  5.51s/it]
 67%|██████▋   | 151192/225000 [296:43:39<113:38:46,  5.54s/it]
 67%|██████▋   | 151193/225000 [296:43:44<113:31:42,  5.54s/it]
 67%|██████▋   | 151194/225000 [296:43:49<107:32:04,  5.25s/it]
 67%|██████▋   | 151195/225000 [296:43:54<107:48:09,  5.26s/it]
 67%|██████▋   | 151196/225000 [296:44:00<109:25:17,  5.34s/it]
 67%|██████▋   | 151197/225000 [296:44:04<105:50:22,  5.16s/it]
 67%|██████▋   | 151198/225000 [296:44:10<107:39:44,  5.25s/it]
 67%|██████▋   | 151199/225000 [296:44:21<147:17:20,  7.18s/it]
 67%|██████▋   | 151200/225000 [296:44:26<130:03:09,  6.34s/it]
                                                               
{'loss': '0.7761', 'grad_norm': '1.136', 'learning_rate': '1.32e-05', 'epoch': '0.672', 'num_input_tokens_seen': 491604496, 'train_runtime': '1.068e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151200/225000 [296:44:26<130:03:09,  6.34s/it]
 67%|██████▋   | 151201/225000 [296:44:33<134:03:16,  6.54s/it]
 67%|██████▋   | 151202/225000 [296:44:38<125:25:47,  6.12s/it]
 67%|██████▋   | 151203/225000 [296:44:45<129:15:16,  6.31s/it]
 67%|██████▋   | 151204/225000 [296:44:49<119:20:42,  5.82s/it]
 67%|██████▋   | 151205/225000 [296:44:54<110:01:56,  5.37s/it]
 67%|██████▋   | 151206/225000 [296:44:58<104:32:09,  5.10s/it]
 67%|██████▋   | 151207/225000 [296:45:02<98:35:20,  4.81s/it] 
 67%|██████▋   | 151208/225000 [296:45:07<98:33:42,  4.81s/it]
 67%|██████▋   | 151209/225000 [296:45:13<103:53:46,  5.07s/it]
 67%|██████▋   | 151210/225000 [296:45:18<106:03:46,  5.17s/it]
 67%|██████▋   | 151211/225000 [296:45:23<104:08:30,  5.08s/it]
 67%|██████▋   | 151212/225000 [296:45:28<102:43:19,  5.01s/it]
 67%|██████▋   | 151213/225000 [296:45:33<105:50:41,  5.16s/it]
 67%|██████▋   | 151214/225000 [296:45:42<127:10:54,  6.21s/it]
 67%|██████▋   | 151215/225000 [296:45:47<120:12:15,  5.86s/it]
 67%|██████▋   | 151216/225000 [296:45:53<118:39:03,  5.79s/it]
 67%|██████▋   | 151217/225000 [296:45:58<114:35:40,  5.59s/it]
 67%|██████▋   | 151218/225000 [296:46:03<114:15:31,  5.57s/it]
 67%|██████▋   | 151219/225000 [296:46:09<115:16:42,  5.62s/it]
 67%|██████▋   | 151220/225000 [296:46:15<114:07:00,  5.57s/it]
 67%|██████▋   | 151221/225000 [296:46:20<114:53:33,  5.61s/it]
 67%|██████▋   | 151222/225000 [296:46:26<114:45:58,  5.60s/it]
 67%|██████▋   | 151223/225000 [296:46:32<115:12:23,  5.62s/it]
 67%|██████▋   | 151224/225000 [296:46:37<111:03:59,  5.42s/it]
 67%|██████▋   | 151225/225000 [296:46:42<110:01:55,  5.37s/it]
 67%|██████▋   | 151226/225000 [296:46:47<108:42:57,  5.31s/it]
 67%|██████▋   | 151227/225000 [296:46:52<107:37:09,  5.25s/it]
 67%|██████▋   | 151228/225000 [296:46:58<110:14:32,  5.38s/it]
 67%|██████▋   | 151229/225000 [296:47:03<110:20:56,  5.38s/it]
 67%|██████▋   | 151230/225000 [296:47:12<130:59:54,  6.39s/it]
 67%|██████▋   | 151231/225000 [296:47:22<155:12:17,  7.57s/it]
 67%|██████▋   | 151232/225000 [296:47:26<131:38:32,  6.42s/it]
 67%|██████▋   | 151233/225000 [296:47:30<118:53:59,  5.80s/it]
 67%|██████▋   | 151234/225000 [296:47:35<110:19:26,  5.38s/it]
 67%|██████▋   | 151235/225000 [296:47:45<139:39:06,  6.82s/it]
 67%|██████▋   | 151236/225000 [296:47:48<119:42:19,  5.84s/it]
 67%|██████▋   | 151237/225000 [296:47:56<131:36:08,  6.42s/it]
 67%|██████▋   | 151238/225000 [296:48:00<115:00:19,  5.61s/it]
 67%|██████▋   | 151239/225000 [296:48:06<120:24:57,  5.88s/it]
 67%|██████▋   | 151240/225000 [296:48:11<112:25:44,  5.49s/it]
 67%|██████▋   | 151241/225000 [296:48:16<106:49:48,  5.21s/it]
 67%|██████▋   | 151242/225000 [296:48:20<103:07:08,  5.03s/it]
 67%|██████▋   | 151243/225000 [296:48:25<100:26:34,  4.90s/it]
 67%|██████▋   | 151244/225000 [296:48:35<135:23:55,  6.61s/it]
 67%|██████▋   | 151245/225000 [296:48:42<137:03:44,  6.69s/it]
 67%|██████▋   | 151246/225000 [296:48:47<125:18:55,  6.12s/it]
 67%|██████▋   | 151247/225000 [296:48:52<118:19:55,  5.78s/it]
 67%|██████▋   | 151248/225000 [296:48:57<112:45:07,  5.50s/it]
 67%|██████▋   | 151249/225000 [296:49:01<105:38:45,  5.16s/it]
 67%|██████▋   | 151250/225000 [296:49:06<105:33:38,  5.15s/it]
                                                               
{'loss': '0.7779', 'grad_norm': '1.099', 'learning_rate': '1.319e-05', 'epoch': '0.6722', 'num_input_tokens_seen': 491734032, 'train_runtime': '1.069e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151250/225000 [296:49:06<105:33:38,  5.15s/it]
 67%|██████▋   | 151251/225000 [296:49:11<103:34:54,  5.06s/it]
 67%|██████▋   | 151252/225000 [296:49:16<103:22:09,  5.05s/it]
 67%|██████▋   | 151253/225000 [296:49:21<104:10:43,  5.09s/it]
 67%|██████▋   | 151254/225000 [296:49:26<103:26:28,  5.05s/it]
 67%|██████▋   | 151255/225000 [296:49:32<106:10:10,  5.18s/it]
 67%|██████▋   | 151256/225000 [296:49:37<103:11:12,  5.04s/it]
 67%|██████▋   | 151257/225000 [296:49:46<128:06:46,  6.25s/it]
 67%|██████▋   | 151258/225000 [296:49:56<155:49:46,  7.61s/it]
 67%|██████▋   | 151259/225000 [296:50:00<128:59:18,  6.30s/it]
 67%|██████▋   | 151260/225000 [296:50:05<121:21:52,  5.93s/it]
 67%|██████▋   | 151261/225000 [296:50:12<131:28:32,  6.42s/it]
 67%|██████▋   | 151262/225000 [296:50:16<116:18:24,  5.68s/it]
 67%|██████▋   | 151263/225000 [296:50:21<109:09:41,  5.33s/it]
 67%|██████▋   | 151264/225000 [296:50:25<104:22:26,  5.10s/it]
 67%|██████▋   | 151265/225000 [296:50:30<99:43:47,  4.87s/it] 
 67%|██████▋   | 151266/225000 [296:50:35<100:17:38,  4.90s/it]
 67%|██████▋   | 151267/225000 [296:50:46<138:00:16,  6.74s/it]
 67%|██████▋   | 151268/225000 [296:50:51<131:20:20,  6.41s/it]
 67%|██████▋   | 151269/225000 [296:50:54<111:18:52,  5.44s/it]
 67%|██████▋   | 151270/225000 [296:50:59<107:50:24,  5.27s/it]
 67%|██████▋   | 151271/225000 [296:51:08<131:21:35,  6.41s/it]
 67%|██████▋   | 151272/225000 [296:51:12<113:10:49,  5.53s/it]
 67%|██████▋   | 151273/225000 [296:51:20<131:39:55,  6.43s/it]
 67%|██████▋   | 151274/225000 [296:51:24<113:08:23,  5.52s/it]
 67%|██████▋   | 151275/225000 [296:51:28<104:58:09,  5.13s/it]
 67%|██████▋   | 151276/225000 [296:51:33<102:00:27,  4.98s/it]
 67%|██████▋   | 151277/225000 [296:51:37<98:50:01,  4.83s/it] 
 67%|██████▋   | 151278/225000 [296:51:42<99:12:44,  4.84s/it]
 67%|██████▋   | 151279/225000 [296:51:47<98:04:05,  4.79s/it]
 67%|██████▋   | 151280/225000 [296:51:52<98:28:42,  4.81s/it]
 67%|██████▋   | 151281/225000 [296:51:56<98:14:13,  4.80s/it]
 67%|██████▋   | 151282/225000 [296:52:01<98:11:23,  4.80s/it]
 67%|██████▋   | 151283/225000 [296:52:12<138:19:36,  6.76s/it]
 67%|██████▋   | 151284/225000 [296:52:17<126:14:55,  6.17s/it]
 67%|██████▋   | 151285/225000 [296:52:22<118:14:12,  5.77s/it]
 67%|██████▋   | 151286/225000 [296:52:28<116:34:22,  5.69s/it]
 67%|██████▋   | 151287/225000 [296:52:33<114:12:54,  5.58s/it]
 67%|██████▋   | 151288/225000 [296:52:38<111:11:44,  5.43s/it]
 67%|██████▋   | 151289/225000 [296:52:44<112:50:16,  5.51s/it]
 67%|██████▋   | 151290/225000 [296:52:49<111:18:17,  5.44s/it]
 67%|██████▋   | 151291/225000 [296:52:54<110:52:51,  5.42s/it]
 67%|██████▋   | 151292/225000 [296:53:00<109:49:28,  5.36s/it]
 67%|██████▋   | 151293/225000 [296:53:05<110:15:26,  5.39s/it]
 67%|██████▋   | 151294/225000 [296:53:11<111:17:29,  5.44s/it]
 67%|██████▋   | 151295/225000 [296:53:16<112:28:33,  5.49s/it]
 67%|██████▋   | 151296/225000 [296:53:21<111:01:22,  5.42s/it]
 67%|██████▋   | 151297/225000 [296:53:27<109:48:12,  5.36s/it]
 67%|██████▋   | 151298/225000 [296:53:32<107:10:23,  5.23s/it]
 67%|██████▋   | 151299/225000 [296:53:37<107:25:02,  5.25s/it]
 67%|██████▋   | 151300/225000 [296:53:42<107:54:58,  5.27s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.164', 'learning_rate': '1.317e-05', 'epoch': '0.6724', 'num_input_tokens_seen': 491862952, 'train_runtime': '1.069e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151300/225000 [296:53:42<107:54:58,  5.27s/it]
 67%|██████▋   | 151301/225000 [296:53:47<107:28:32,  5.25s/it]
 67%|██████▋   | 151302/225000 [296:53:53<110:41:59,  5.41s/it]
 67%|██████▋   | 151303/225000 [296:54:05<150:32:35,  7.35s/it]
 67%|██████▋   | 151304/225000 [296:54:10<137:52:52,  6.74s/it]
 67%|██████▋   | 151305/225000 [296:54:15<126:31:29,  6.18s/it]
 67%|██████▋   | 151306/225000 [296:54:20<120:45:52,  5.90s/it]
 67%|██████▋   | 151307/225000 [296:54:26<116:54:32,  5.71s/it]
 67%|██████▋   | 151308/225000 [296:54:31<112:37:09,  5.50s/it]
 67%|██████▋   | 151309/225000 [296:54:36<112:19:06,  5.49s/it]
 67%|██████▋   | 151310/225000 [296:54:49<159:43:04,  7.80s/it]
 67%|██████▋   | 151311/225000 [296:54:55<146:28:55,  7.16s/it]
 67%|██████▋   | 151312/225000 [296:54:58<123:18:45,  6.02s/it]
 67%|██████▋   | 151313/225000 [296:55:03<111:27:38,  5.45s/it]
 67%|██████▋   | 151314/225000 [296:55:13<140:29:43,  6.86s/it]
 67%|██████▋   | 151315/225000 [296:55:16<117:59:43,  5.76s/it]
 67%|██████▋   | 151316/225000 [296:55:21<115:13:27,  5.63s/it]
 67%|██████▋   | 151317/225000 [296:55:27<116:46:50,  5.71s/it]
 67%|██████▋   | 151318/225000 [296:55:34<126:12:04,  6.17s/it]
 67%|██████▋   | 151319/225000 [296:55:38<112:12:38,  5.48s/it]
 67%|██████▋   | 151320/225000 [296:55:47<129:20:13,  6.32s/it]
 67%|██████▋   | 151321/225000 [296:55:50<111:32:10,  5.45s/it]
 67%|██████▋   | 151322/225000 [296:55:56<114:51:17,  5.61s/it]
 67%|██████▋   | 151323/225000 [296:56:02<120:24:54,  5.88s/it]
 67%|██████▋   | 151324/225000 [296:56:07<109:39:41,  5.36s/it]
 67%|██████▋   | 151325/225000 [296:56:11<102:50:57,  5.03s/it]
 67%|██████▋   | 151326/225000 [296:56:15<97:30:43,  4.76s/it] 
 67%|██████▋   | 151327/225000 [296:56:22<110:30:06,  5.40s/it]
 67%|██████▋   | 151328/225000 [296:56:31<133:12:34,  6.51s/it]
 67%|██████▋   | 151329/225000 [296:56:34<112:34:25,  5.50s/it]
 67%|██████▋   | 151330/225000 [296:56:43<132:58:45,  6.50s/it]
 67%|██████▋   | 151331/225000 [296:56:47<118:30:46,  5.79s/it]
 67%|██████▋   | 151332/225000 [296:56:50<103:42:09,  5.07s/it]
 67%|██████▋   | 151333/225000 [296:56:56<103:30:45,  5.06s/it]
 67%|██████▋   | 151334/225000 [296:57:04<126:41:30,  6.19s/it]
 67%|██████▋   | 151335/225000 [296:57:08<112:59:08,  5.52s/it]
 67%|██████▋   | 151336/225000 [296:57:16<127:24:18,  6.23s/it]
 67%|██████▋   | 151337/225000 [296:57:20<114:11:40,  5.58s/it]
 67%|██████▋   | 151338/225000 [296:57:27<121:44:13,  5.95s/it]
 67%|██████▋   | 151339/225000 [296:57:31<108:21:11,  5.30s/it]
 67%|██████▋   | 151340/225000 [296:57:34<97:52:15,  4.78s/it] 
 67%|██████▋   | 151341/225000 [296:57:39<95:45:20,  4.68s/it]
 67%|██████▋   | 151342/225000 [296:57:46<110:51:48,  5.42s/it]
 67%|██████▋   | 151343/225000 [296:57:56<136:50:21,  6.69s/it]
 67%|██████▋   | 151344/225000 [296:58:00<120:40:04,  5.90s/it]
 67%|██████▋   | 151345/225000 [296:58:04<110:26:54,  5.40s/it]
 67%|██████▋   | 151346/225000 [296:58:08<102:24:01,  5.01s/it]
 67%|██████▋   | 151347/225000 [296:58:12<96:18:23,  4.71s/it] 
 67%|██████▋   | 151348/225000 [296:58:24<140:26:38,  6.86s/it]
 67%|██████▋   | 151349/225000 [296:58:28<123:12:43,  6.02s/it]
 67%|██████▋   | 151350/225000 [296:58:32<112:03:52,  5.48s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.101', 'learning_rate': '1.316e-05', 'epoch': '0.6727', 'num_input_tokens_seen': 491991856, 'train_runtime': '1.069e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151350/225000 [296:58:32<112:03:52,  5.48s/it]
 67%|██████▋   | 151351/225000 [296:58:37<107:57:43,  5.28s/it]
 67%|██████▋   | 151352/225000 [296:58:42<105:41:27,  5.17s/it]
 67%|██████▋   | 151353/225000 [296:58:47<103:07:06,  5.04s/it]
 67%|██████▋   | 151354/225000 [296:58:51<99:41:54,  4.87s/it] 
 67%|██████▋   | 151355/225000 [296:58:56<97:32:20,  4.77s/it]
 67%|██████▋   | 151356/225000 [296:59:04<120:55:43,  5.91s/it]
 67%|██████▋   | 151357/225000 [296:59:16<153:59:27,  7.53s/it]
 67%|██████▋   | 151358/225000 [296:59:19<128:01:17,  6.26s/it]
 67%|██████▋   | 151359/225000 [296:59:27<136:54:23,  6.69s/it]
 67%|██████▋   | 151360/225000 [296:59:30<118:21:04,  5.79s/it]
 67%|██████▋   | 151361/225000 [296:59:34<108:01:57,  5.28s/it]
 67%|██████▋   | 151362/225000 [296:59:39<101:35:56,  4.97s/it]
 67%|██████▋   | 151363/225000 [296:59:44<105:30:51,  5.16s/it]
 67%|██████▋   | 151364/225000 [296:59:50<108:50:27,  5.32s/it]
 67%|██████▋   | 151365/225000 [296:59:56<114:19:43,  5.59s/it]
 67%|██████▋   | 151366/225000 [297:00:05<134:22:26,  6.57s/it]
 67%|██████▋   | 151367/225000 [297:00:09<117:35:14,  5.75s/it]
 67%|██████▋   | 151368/225000 [297:00:13<105:23:24,  5.15s/it]
 67%|██████▋   | 151369/225000 [297:00:17<103:08:13,  5.04s/it]
 67%|██████▋   | 151370/225000 [297:00:22<100:33:44,  4.92s/it]
 67%|██████▋   | 151371/225000 [297:00:34<143:57:42,  7.04s/it]
 67%|██████▋   | 151372/225000 [297:00:38<123:22:13,  6.03s/it]
 67%|██████▋   | 151373/225000 [297:00:43<116:49:49,  5.71s/it]
 67%|██████▋   | 151374/225000 [297:00:51<135:09:39,  6.61s/it]
 67%|██████▋   | 151375/225000 [297:00:56<123:41:46,  6.05s/it]
 67%|██████▋   | 151376/225000 [297:01:01<114:04:46,  5.58s/it]
 67%|██████▋   | 151377/225000 [297:01:05<106:55:14,  5.23s/it]
 67%|██████▋   | 151378/225000 [297:01:10<103:03:59,  5.04s/it]
 67%|██████▋   | 151379/225000 [297:01:18<123:02:11,  6.02s/it]
 67%|██████▋   | 151380/225000 [297:01:23<117:27:32,  5.74s/it]
 67%|██████▋   | 151381/225000 [297:01:28<111:44:27,  5.46s/it]
 67%|██████▋   | 151382/225000 [297:01:33<110:14:38,  5.39s/it]
 67%|██████▋   | 151383/225000 [297:01:38<107:03:16,  5.24s/it]
 67%|██████▋   | 151384/225000 [297:01:43<106:27:45,  5.21s/it]
 67%|██████▋   | 151385/225000 [297:01:48<105:51:03,  5.18s/it]
 67%|██████▋   | 151386/225000 [297:01:53<107:07:55,  5.24s/it]
 67%|██████▋   | 151387/225000 [297:01:59<106:36:40,  5.21s/it]
 67%|██████▋   | 151388/225000 [297:02:04<108:08:42,  5.29s/it]
 67%|██████▋   | 151389/225000 [297:02:12<123:46:43,  6.05s/it]
 67%|██████▋   | 151390/225000 [297:02:22<148:03:25,  7.24s/it]
 67%|██████▋   | 151391/225000 [297:02:26<128:06:34,  6.27s/it]
 67%|██████▋   | 151392/225000 [297:02:30<116:36:05,  5.70s/it]
 67%|██████▋   | 151393/225000 [297:02:35<108:43:56,  5.32s/it]
 67%|██████▋   | 151394/225000 [297:02:43<129:52:19,  6.35s/it]
 67%|██████▋   | 151395/225000 [297:02:50<128:56:02,  6.31s/it]
 67%|██████▋   | 151396/225000 [297:02:54<119:18:34,  5.84s/it]
 67%|██████▋   | 151397/225000 [297:03:01<122:47:06,  6.01s/it]
 67%|██████▋   | 151398/225000 [297:03:04<107:10:55,  5.24s/it]
 67%|██████▋   | 151399/225000 [297:03:12<120:22:00,  5.89s/it]
 67%|██████▋   | 151400/225000 [297:03:18<122:51:42,  6.01s/it]
                                                               
{'loss': '0.7707', 'grad_norm': '1.171', 'learning_rate': '1.314e-05', 'epoch': '0.6729', 'num_input_tokens_seen': 492121504, 'train_runtime': '1.069e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151400/225000 [297:03:18<122:51:42,  6.01s/it]
 67%|██████▋   | 151401/225000 [297:03:22<109:48:19,  5.37s/it]
 67%|██████▋   | 151402/225000 [297:03:26<101:45:49,  4.98s/it]
 67%|██████▋   | 151403/225000 [297:03:30<97:16:53,  4.76s/it] 
 67%|██████▋   | 151404/225000 [297:03:34<92:27:36,  4.52s/it]
 67%|██████▋   | 151405/225000 [297:03:41<107:02:56,  5.24s/it]
 67%|██████▋   | 151406/225000 [297:03:51<134:56:32,  6.60s/it]
 67%|██████▋   | 151407/225000 [297:03:54<114:23:19,  5.60s/it]
 67%|██████▋   | 151408/225000 [297:03:58<103:38:54,  5.07s/it]
 67%|██████▋   | 151409/225000 [297:04:02<96:31:26,  4.72s/it] 
 67%|██████▋   | 151410/225000 [297:04:06<94:10:05,  4.61s/it]
 67%|██████▋   | 151411/225000 [297:04:11<93:14:55,  4.56s/it]
 67%|██████▋   | 151412/225000 [297:04:21<131:28:57,  6.43s/it]
 67%|██████▋   | 151413/225000 [297:04:29<141:31:43,  6.92s/it]
 67%|██████▋   | 151414/225000 [297:04:33<120:50:05,  5.91s/it]
 67%|██████▋   | 151415/225000 [297:04:39<120:34:33,  5.90s/it]
 67%|██████▋   | 151416/225000 [297:04:46<128:24:13,  6.28s/it]
 67%|██████▋   | 151417/225000 [297:04:50<111:54:50,  5.48s/it]
 67%|██████▋   | 151418/225000 [297:04:56<117:00:40,  5.72s/it]
 67%|██████▋   | 151419/225000 [297:05:03<122:23:24,  5.99s/it]
 67%|██████▋   | 151420/225000 [297:05:07<113:45:06,  5.57s/it]
 67%|██████▋   | 151421/225000 [297:05:14<123:50:15,  6.06s/it]
 67%|██████▋   | 151422/225000 [297:05:19<115:17:55,  5.64s/it]
 67%|██████▋   | 151423/225000 [297:05:24<110:45:32,  5.42s/it]
 67%|██████▋   | 151424/225000 [297:05:28<104:10:49,  5.10s/it]
 67%|██████▋   | 151425/225000 [297:05:32<97:33:07,  4.77s/it] 
 67%|██████▋   | 151426/225000 [297:05:42<128:42:23,  6.30s/it]
 67%|██████▋   | 151427/225000 [297:05:47<121:18:04,  5.94s/it]
 67%|██████▋   | 151428/225000 [297:05:51<108:59:28,  5.33s/it]
 67%|██████▋   | 151429/225000 [297:06:01<134:21:32,  6.57s/it]
 67%|██████▋   | 151430/225000 [297:06:05<123:33:24,  6.05s/it]
 67%|██████▋   | 151431/225000 [297:06:10<115:03:43,  5.63s/it]
 67%|██████▋   | 151432/225000 [297:06:14<105:36:09,  5.17s/it]
 67%|██████▋   | 151433/225000 [297:06:19<103:20:06,  5.06s/it]
 67%|██████▋   | 151434/225000 [297:06:23<99:32:38,  4.87s/it] 
 67%|██████▋   | 151435/225000 [297:06:28<98:03:31,  4.80s/it]
 67%|██████▋   | 151436/225000 [297:06:33<98:24:45,  4.82s/it]
 67%|██████▋   | 151437/225000 [297:06:42<126:55:04,  6.21s/it]
 67%|██████▋   | 151438/225000 [297:06:50<135:52:06,  6.65s/it]
 67%|██████▋   | 151439/225000 [297:06:54<120:57:13,  5.92s/it]
 67%|██████▋   | 151440/225000 [297:06:59<112:39:52,  5.51s/it]
 67%|██████▋   | 151441/225000 [297:07:03<103:52:27,  5.08s/it]
 67%|██████▋   | 151442/225000 [297:07:07<99:01:31,  4.85s/it] 
 67%|██████▋   | 151443/225000 [297:07:17<131:49:50,  6.45s/it]
 67%|██████▋   | 151444/225000 [297:07:26<142:55:11,  6.99s/it]
 67%|██████▋   | 151445/225000 [297:07:30<124:55:58,  6.11s/it]
 67%|██████▋   | 151446/225000 [297:07:38<139:22:04,  6.82s/it]
 67%|██████▋   | 151447/225000 [297:07:44<133:52:32,  6.55s/it]
 67%|██████▋   | 151448/225000 [297:07:49<123:25:24,  6.04s/it]
 67%|██████▋   | 151449/225000 [297:07:53<111:07:09,  5.44s/it]
 67%|██████▋   | 151450/225000 [297:07:58<105:16:16,  5.15s/it]
                                                               
{'loss': '0.7747', 'grad_norm': '1.069', 'learning_rate': '1.312e-05', 'epoch': '0.6731', 'num_input_tokens_seen': 492251216, 'train_runtime': '1.07e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151450/225000 [297:07:58<105:16:16,  5.15s/it]
 67%|██████▋   | 151451/225000 [297:08:02<102:07:38,  5.00s/it]
 67%|██████▋   | 151452/225000 [297:08:07<102:52:15,  5.04s/it]
 67%|██████▋   | 151453/225000 [297:08:19<141:10:33,  6.91s/it]
 67%|██████▋   | 151454/225000 [297:08:22<120:50:38,  5.92s/it]
 67%|██████▋   | 151455/225000 [297:08:30<135:33:04,  6.64s/it]
 67%|██████▋   | 151456/225000 [297:08:34<118:45:10,  5.81s/it]
 67%|██████▋   | 151457/225000 [297:08:39<109:07:29,  5.34s/it]
 67%|██████▋   | 151458/225000 [297:08:43<100:22:59,  4.91s/it]
 67%|██████▋   | 151459/225000 [297:08:47<97:33:20,  4.78s/it] 
 67%|██████▋   | 151460/225000 [297:08:51<94:01:00,  4.60s/it]
 67%|██████▋   | 151461/225000 [297:08:58<105:46:01,  5.18s/it]
 67%|██████▋   | 151462/225000 [297:09:07<130:29:09,  6.39s/it]
 67%|██████▋   | 151463/225000 [297:09:12<120:43:42,  5.91s/it]
 67%|██████▋   | 151464/225000 [297:09:16<113:24:17,  5.55s/it]
 67%|██████▋   | 151465/225000 [297:09:21<106:43:39,  5.22s/it]
 67%|██████▋   | 151466/225000 [297:09:27<112:33:48,  5.51s/it]
 67%|██████▋   | 151467/225000 [297:09:38<144:04:40,  7.05s/it]
 67%|██████▋   | 151468/225000 [297:09:41<122:56:59,  6.02s/it]
 67%|██████▋   | 151469/225000 [297:09:49<134:25:49,  6.58s/it]
 67%|██████▋   | 151470/225000 [297:09:54<124:05:56,  6.08s/it]
 67%|██████▋   | 151471/225000 [297:10:00<125:36:29,  6.15s/it]
 67%|██████▋   | 151472/225000 [297:10:06<123:41:16,  6.06s/it]
 67%|██████▋   | 151473/225000 [297:10:13<126:47:14,  6.21s/it]
 67%|██████▋   | 151474/225000 [297:10:17<111:41:48,  5.47s/it]
 67%|██████▋   | 151475/225000 [297:10:24<122:12:53,  5.98s/it]
 67%|██████▋   | 151476/225000 [297:10:29<116:33:36,  5.71s/it]
 67%|██████▋   | 151477/225000 [297:10:35<117:32:35,  5.76s/it]
 67%|██████▋   | 151478/225000 [297:10:38<104:30:30,  5.12s/it]
 67%|██████▋   | 151479/225000 [297:10:42<97:53:30,  4.79s/it] 
 67%|██████▋   | 151480/225000 [297:10:52<129:09:21,  6.32s/it]
 67%|██████▋   | 151481/225000 [297:10:55<109:43:01,  5.37s/it]
 67%|██████▋   | 151482/225000 [297:11:03<121:42:16,  5.96s/it]
 67%|██████▋   | 151483/225000 [297:11:07<109:35:09,  5.37s/it]
 67%|██████▋   | 151484/225000 [297:11:11<100:44:05,  4.93s/it]
 67%|██████▋   | 151485/225000 [297:11:19<120:23:45,  5.90s/it]
 67%|██████▋   | 151486/225000 [297:11:24<116:05:52,  5.69s/it]
 67%|██████▋   | 151487/225000 [297:11:29<114:06:39,  5.59s/it]
 67%|██████▋   | 151488/225000 [297:11:38<132:02:58,  6.47s/it]
 67%|██████▋   | 151489/225000 [297:11:41<113:49:11,  5.57s/it]
 67%|██████▋   | 151490/225000 [297:11:47<111:58:26,  5.48s/it]
 67%|██████▋   | 151491/225000 [297:11:53<117:12:03,  5.74s/it]
 67%|██████▋   | 151492/225000 [297:11:57<109:03:33,  5.34s/it]
 67%|██████▋   | 151493/225000 [297:12:02<103:13:25,  5.06s/it]
 67%|██████▋   | 151494/225000 [297:12:06<98:58:20,  4.85s/it] 
 67%|██████▋   | 151495/225000 [297:12:17<133:08:00,  6.52s/it]
 67%|██████▋   | 151496/225000 [297:12:21<120:19:49,  5.89s/it]
 67%|██████▋   | 151497/225000 [297:12:25<109:02:46,  5.34s/it]
 67%|██████▋   | 151498/225000 [297:12:30<106:10:03,  5.20s/it]
 67%|██████▋   | 151499/225000 [297:12:35<104:08:36,  5.10s/it]
 67%|██████▋   | 151500/225000 [297:12:40<104:08:31,  5.10s/it]
                                                               
{'loss': '0.7872', 'grad_norm': '1.189', 'learning_rate': '1.311e-05', 'epoch': '0.6733', 'num_input_tokens_seen': 492381984, 'train_runtime': '1.07e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151500/225000 [297:12:40<104:08:31,  5.10s/it]
 67%|██████▋   | 151501/225000 [297:12:45<102:23:00,  5.01s/it]
 67%|██████▋   | 151502/225000 [297:12:53<121:03:07,  5.93s/it]
 67%|██████▋   | 151503/225000 [297:13:02<140:27:33,  6.88s/it]
 67%|██████▋   | 151504/225000 [297:13:05<118:18:28,  5.79s/it]
 67%|██████▋   | 151505/225000 [297:13:09<108:03:52,  5.29s/it]
 67%|██████▋   | 151506/225000 [297:13:14<104:12:55,  5.10s/it]
 67%|██████▋   | 151507/225000 [297:13:24<137:59:50,  6.76s/it]
 67%|██████▋   | 151508/225000 [297:13:30<133:09:22,  6.52s/it]
 67%|██████▋   | 151509/225000 [297:13:34<115:49:25,  5.67s/it]
 67%|██████▋   | 151510/225000 [297:13:38<106:48:15,  5.23s/it]
 67%|██████▋   | 151511/225000 [297:13:43<104:00:15,  5.09s/it]
 67%|██████▋   | 151512/225000 [297:13:55<143:54:47,  7.05s/it]
 67%|██████▋   | 151513/225000 [297:13:58<121:07:19,  5.93s/it]
 67%|██████▋   | 151514/225000 [297:14:04<122:30:15,  6.00s/it]
 67%|██████▋   | 151515/225000 [297:14:10<122:58:59,  6.02s/it]
 67%|██████▋   | 151516/225000 [297:14:18<131:37:35,  6.45s/it]
 67%|██████▋   | 151517/225000 [297:14:22<120:31:55,  5.90s/it]
 67%|██████▋   | 151518/225000 [297:14:29<127:50:30,  6.26s/it]
 67%|██████▋   | 151519/225000 [297:14:36<130:29:29,  6.39s/it]
 67%|██████▋   | 151520/225000 [297:14:41<118:39:35,  5.81s/it]
 67%|██████▋   | 151521/225000 [297:14:46<118:03:35,  5.78s/it]
 67%|██████▋   | 151522/225000 [297:14:52<116:33:57,  5.71s/it]
 67%|██████▋   | 151523/225000 [297:14:57<111:58:25,  5.49s/it]
 67%|██████▋   | 151524/225000 [297:15:05<129:07:50,  6.33s/it]
 67%|██████▋   | 151525/225000 [297:15:10<117:32:58,  5.76s/it]
 67%|██████▋   | 151526/225000 [297:15:14<107:25:54,  5.26s/it]
 67%|██████▋   | 151527/225000 [297:15:18<102:37:35,  5.03s/it]
 67%|██████▋   | 151528/225000 [297:15:29<137:57:12,  6.76s/it]
 67%|██████▋   | 151529/225000 [297:15:35<131:02:13,  6.42s/it]
 67%|██████▋   | 151530/225000 [297:15:39<119:22:36,  5.85s/it]
 67%|██████▋   | 151531/225000 [297:15:44<114:58:07,  5.63s/it]
 67%|██████▋   | 151532/225000 [297:15:50<114:04:02,  5.59s/it]
 67%|██████▋   | 151533/225000 [297:15:54<108:39:49,  5.32s/it]
 67%|██████▋   | 151534/225000 [297:16:00<109:06:52,  5.35s/it]
 67%|██████▋   | 151535/225000 [297:16:05<107:46:33,  5.28s/it]
 67%|██████▋   | 151536/225000 [297:16:10<103:21:45,  5.07s/it]
 67%|██████▋   | 151537/225000 [297:16:14<102:19:29,  5.01s/it]
 67%|██████▋   | 151538/225000 [297:16:20<104:38:25,  5.13s/it]
 67%|██████▋   | 151539/225000 [297:16:25<107:07:40,  5.25s/it]
 67%|██████▋   | 151540/225000 [297:16:31<107:14:06,  5.26s/it]
 67%|██████▋   | 151541/225000 [297:16:36<106:46:05,  5.23s/it]
 67%|██████▋   | 151542/225000 [297:16:41<106:31:59,  5.22s/it]
 67%|██████▋   | 151543/225000 [297:16:50<129:48:43,  6.36s/it]
 67%|██████▋   | 151544/225000 [297:17:00<150:06:15,  7.36s/it]
 67%|██████▋   | 151545/225000 [297:17:03<128:18:24,  6.29s/it]
 67%|██████▋   | 151546/225000 [297:17:08<116:28:49,  5.71s/it]
 67%|██████▋   | 151547/225000 [297:17:13<113:22:41,  5.56s/it]
 67%|██████▋   | 151548/225000 [297:17:18<111:48:20,  5.48s/it]
 67%|██████▋   | 151549/225000 [297:17:24<110:46:27,  5.43s/it]
 67%|██████▋   | 151550/225000 [297:17:29<110:59:18,  5.44s/it]
                                                               
{'loss': '0.7883', 'grad_norm': '1.108', 'learning_rate': '1.309e-05', 'epoch': '0.6736', 'num_input_tokens_seen': 492512936, 'train_runtime': '1.07e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151550/225000 [297:17:29<110:59:18,  5.44s/it]
 67%|██████▋   | 151551/225000 [297:17:34<108:03:19,  5.30s/it]
 67%|██████▋   | 151552/225000 [297:17:39<106:15:33,  5.21s/it]
 67%|██████▋   | 151553/225000 [297:17:45<108:16:03,  5.31s/it]
 67%|██████▋   | 151554/225000 [297:17:50<110:15:05,  5.40s/it]
 67%|██████▋   | 151555/225000 [297:17:55<105:56:59,  5.19s/it]
 67%|██████▋   | 151556/225000 [297:18:00<106:07:07,  5.20s/it]
 67%|██████▋   | 151557/225000 [297:18:06<106:57:04,  5.24s/it]
 67%|██████▋   | 151558/225000 [297:18:11<106:54:11,  5.24s/it]
 67%|██████▋   | 151559/225000 [297:18:16<107:32:53,  5.27s/it]
 67%|██████▋   | 151560/225000 [297:18:28<145:40:25,  7.14s/it]
 67%|██████▋   | 151561/225000 [297:18:36<151:34:07,  7.43s/it]
 67%|██████▋   | 151562/225000 [297:18:39<128:07:49,  6.28s/it]
 67%|██████▋   | 151563/225000 [297:18:44<120:29:14,  5.91s/it]
 67%|██████▋   | 151564/225000 [297:18:49<113:09:19,  5.55s/it]
 67%|██████▋   | 151565/225000 [297:18:54<111:08:50,  5.45s/it]
 67%|██████▋   | 151566/225000 [297:19:00<111:50:13,  5.48s/it]
 67%|██████▋   | 151567/225000 [297:19:05<112:00:07,  5.49s/it]
 67%|██████▋   | 151568/225000 [297:19:11<111:46:25,  5.48s/it]
 67%|██████▋   | 151569/225000 [297:19:16<111:38:27,  5.47s/it]
 67%|██████▋   | 151570/225000 [297:19:24<126:51:10,  6.22s/it]
 67%|██████▋   | 151571/225000 [297:19:30<127:05:08,  6.23s/it]
 67%|██████▋   | 151572/225000 [297:19:36<121:10:06,  5.94s/it]
 67%|██████▋   | 151573/225000 [297:19:44<136:59:17,  6.72s/it]
 67%|██████▋   | 151574/225000 [297:19:50<128:59:59,  6.32s/it]
 67%|██████▋   | 151575/225000 [297:19:54<116:10:55,  5.70s/it]
 67%|██████▋   | 151576/225000 [297:19:58<106:46:06,  5.23s/it]
 67%|██████▋   | 151577/225000 [297:20:03<102:14:49,  5.01s/it]
 67%|██████▋   | 151578/225000 [297:20:07<96:45:36,  4.74s/it] 
 67%|██████▋   | 151579/225000 [297:20:16<123:27:02,  6.05s/it]
 67%|██████▋   | 151580/225000 [297:20:24<139:19:24,  6.83s/it]
 67%|██████▋   | 151581/225000 [297:20:28<118:46:30,  5.82s/it]
 67%|██████▋   | 151582/225000 [297:20:32<107:51:13,  5.29s/it]
 67%|██████▋   | 151583/225000 [297:20:43<140:28:31,  6.89s/it]
 67%|██████▋   | 151584/225000 [297:20:47<124:27:36,  6.10s/it]
 67%|██████▋   | 151585/225000 [297:20:51<115:11:31,  5.65s/it]
 67%|██████▋   | 151586/225000 [297:20:56<107:24:45,  5.27s/it]
 67%|██████▋   | 151587/225000 [297:21:00<103:01:38,  5.05s/it]
 67%|██████▋   | 151588/225000 [297:21:08<118:13:56,  5.80s/it]
 67%|██████▋   | 151589/225000 [297:21:16<129:25:59,  6.35s/it]
 67%|██████▋   | 151590/225000 [297:21:19<109:56:21,  5.39s/it]
 67%|██████▋   | 151591/225000 [297:21:27<129:45:23,  6.36s/it]
 67%|██████▋   | 151592/225000 [297:21:31<113:12:50,  5.55s/it]
 67%|██████▋   | 151593/225000 [297:21:35<102:34:11,  5.03s/it]
 67%|██████▋   | 151594/225000 [297:21:41<109:17:28,  5.36s/it]
 67%|██████▋   | 151595/225000 [297:21:50<132:15:54,  6.49s/it]
 67%|██████▋   | 151596/225000 [297:21:53<112:27:36,  5.52s/it]
 67%|██████▋   | 151597/225000 [297:22:00<122:45:50,  6.02s/it]
 67%|██████▋   | 151598/225000 [297:22:04<109:48:58,  5.39s/it]
 67%|██████▋   | 151599/225000 [297:22:08<101:29:12,  4.98s/it]
 67%|██████▋   | 151600/225000 [297:22:20<141:09:36,  6.92s/it]
                                                               
{'loss': '0.7807', 'grad_norm': '1.006', 'learning_rate': '1.307e-05', 'epoch': '0.6738', 'num_input_tokens_seen': 492644072, 'train_runtime': '1.071e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151600/225000 [297:22:20<141:09:36,  6.92s/it]
 67%|██████▋   | 151601/225000 [297:22:23<119:40:45,  5.87s/it]
 67%|██████▋   | 151602/225000 [297:22:31<131:33:42,  6.45s/it]
 67%|██████▋   | 151603/225000 [297:22:35<114:47:28,  5.63s/it]
 67%|██████▋   | 151604/225000 [297:22:39<103:25:50,  5.07s/it]
 67%|██████▋   | 151605/225000 [297:22:44<103:32:19,  5.08s/it]
 67%|██████▋   | 151606/225000 [297:22:49<103:01:34,  5.05s/it]
 67%|██████▋   | 151607/225000 [297:22:54<101:48:50,  4.99s/it]
 67%|██████▋   | 151608/225000 [297:23:04<137:31:25,  6.75s/it]
 67%|██████▋   | 151609/225000 [297:23:10<128:40:57,  6.31s/it]
 67%|██████▋   | 151610/225000 [297:23:13<108:24:27,  5.32s/it]
 67%|██████▋   | 151611/225000 [297:23:21<129:28:34,  6.35s/it]
 67%|██████▋   | 151612/225000 [297:23:25<111:33:47,  5.47s/it]
 67%|██████▋   | 151613/225000 [297:23:34<132:40:53,  6.51s/it]
 67%|██████▋   | 151614/225000 [297:23:37<112:57:27,  5.54s/it]
 67%|██████▋   | 151615/225000 [297:23:45<130:04:50,  6.38s/it]
 67%|██████▋   | 151616/225000 [297:23:49<115:12:58,  5.65s/it]
 67%|██████▋   | 151617/225000 [297:23:53<105:23:06,  5.17s/it]
 67%|██████▋   | 151618/225000 [297:23:57<98:56:52,  4.85s/it] 
 67%|██████▋   | 151619/225000 [297:24:08<131:05:56,  6.43s/it]
 67%|██████▋   | 151620/225000 [297:24:12<117:17:38,  5.75s/it]
 67%|██████▋   | 151621/225000 [297:24:15<104:51:00,  5.14s/it]
 67%|██████▋   | 151622/225000 [297:24:25<128:29:43,  6.30s/it]
 67%|██████▋   | 151623/225000 [297:24:30<123:18:50,  6.05s/it]
 67%|██████▋   | 151624/225000 [297:24:34<112:43:01,  5.53s/it]
 67%|██████▋   | 151625/225000 [297:24:39<106:28:48,  5.22s/it]
 67%|██████▋   | 151626/225000 [297:24:44<104:55:27,  5.15s/it]
 67%|██████▋   | 151627/225000 [297:24:49<105:31:45,  5.18s/it]
 67%|██████▋   | 151628/225000 [297:24:54<104:57:12,  5.15s/it]
 67%|██████▋   | 151629/225000 [297:24:59<105:27:20,  5.17s/it]
 67%|██████▋   | 151630/225000 [297:25:04<105:24:26,  5.17s/it]
 67%|██████▋   | 151631/225000 [297:25:10<106:39:00,  5.23s/it]
 67%|██████▋   | 151632/225000 [297:25:15<107:08:12,  5.26s/it]
 67%|██████▋   | 151633/225000 [297:25:21<108:52:38,  5.34s/it]
 67%|██████▋   | 151634/225000 [297:25:26<109:08:42,  5.36s/it]
 67%|██████▋   | 151635/225000 [297:25:31<108:14:00,  5.31s/it]
 67%|██████▋   | 151636/225000 [297:25:36<107:15:06,  5.26s/it]
 67%|██████▋   | 151637/225000 [297:25:43<114:05:43,  5.60s/it]
 67%|██████▋   | 151638/225000 [297:25:55<152:25:12,  7.48s/it]
 67%|██████▋   | 151639/225000 [297:25:59<135:30:21,  6.65s/it]
 67%|██████▋   | 151640/225000 [297:26:04<120:46:45,  5.93s/it]
 67%|██████▋   | 151641/225000 [297:26:08<111:10:09,  5.46s/it]
 67%|██████▋   | 151642/225000 [297:26:13<105:47:15,  5.19s/it]
 67%|██████▋   | 151643/225000 [297:26:17<99:40:57,  4.89s/it] 
 67%|██████▋   | 151644/225000 [297:26:26<128:32:26,  6.31s/it]
 67%|██████▋   | 151645/225000 [297:26:32<124:44:20,  6.12s/it]
 67%|██████▋   | 151646/225000 [297:26:40<136:21:21,  6.69s/it]
 67%|██████▋   | 151647/225000 [297:26:44<122:11:26,  6.00s/it]
 67%|██████▋   | 151648/225000 [297:26:48<109:48:26,  5.39s/it]
 67%|██████▋   | 151649/225000 [297:26:53<104:13:14,  5.12s/it]
 67%|██████▋   | 151650/225000 [297:27:04<142:11:32,  6.98s/it]
                                                               
{'loss': '0.7855', 'grad_norm': '1.099', 'learning_rate': '1.306e-05', 'epoch': '0.674', 'num_input_tokens_seen': 492773632, 'train_runtime': '1.071e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151650/225000 [297:27:04<142:11:32,  6.98s/it]
 67%|██████▋   | 151651/225000 [297:27:08<121:37:27,  5.97s/it]
 67%|██████▋   | 151652/225000 [297:27:12<112:02:51,  5.50s/it]
 67%|██████▋   | 151653/225000 [297:27:17<106:41:30,  5.24s/it]
 67%|██████▋   | 151654/225000 [297:27:21<99:12:49,  4.87s/it] 
 67%|██████▋   | 151655/225000 [297:27:26<99:38:45,  4.89s/it]
 67%|██████▋   | 151656/225000 [297:27:38<143:53:37,  7.06s/it]
 67%|██████▋   | 151657/225000 [297:27:42<126:00:12,  6.18s/it]
 67%|██████▋   | 151658/225000 [297:27:46<113:55:50,  5.59s/it]
 67%|██████▋   | 151659/225000 [297:27:51<105:32:57,  5.18s/it]
 67%|██████▋   | 151660/225000 [297:27:57<112:23:22,  5.52s/it]
 67%|██████▋   | 151661/225000 [297:28:07<143:01:25,  7.02s/it]
 67%|██████▋   | 151662/225000 [297:28:11<122:45:27,  6.03s/it]
 67%|██████▋   | 151663/225000 [297:28:18<127:18:51,  6.25s/it]
 67%|██████▋   | 151664/225000 [297:28:23<123:04:32,  6.04s/it]
 67%|██████▋   | 151665/225000 [297:28:29<119:59:54,  5.89s/it]
 67%|██████▋   | 151666/225000 [297:28:38<137:23:31,  6.74s/it]
 67%|██████▋   | 151667/225000 [297:28:42<122:23:37,  6.01s/it]
 67%|██████▋   | 151668/225000 [297:28:46<110:53:49,  5.44s/it]
 67%|██████▋   | 151669/225000 [297:28:50<102:23:36,  5.03s/it]
 67%|██████▋   | 151670/225000 [297:28:54<96:50:45,  4.75s/it] 
 67%|██████▋   | 151671/225000 [297:29:01<111:24:53,  5.47s/it]
 67%|██████▋   | 151672/225000 [297:29:12<141:42:18,  6.96s/it]
 67%|██████▋   | 151673/225000 [297:29:17<129:18:13,  6.35s/it]
 67%|██████▋   | 151674/225000 [297:29:22<123:14:48,  6.05s/it]
 67%|██████▋   | 151675/225000 [297:29:26<112:27:30,  5.52s/it]
 67%|██████▋   | 151676/225000 [297:29:32<110:55:12,  5.45s/it]
 67%|██████▋   | 151677/225000 [297:29:36<106:41:34,  5.24s/it]
 67%|██████▋   | 151678/225000 [297:29:41<105:29:20,  5.18s/it]
 67%|██████▋   | 151679/225000 [297:29:46<102:11:36,  5.02s/it]
 67%|██████▋   | 151680/225000 [297:29:51<101:14:25,  4.97s/it]
 67%|██████▋   | 151681/225000 [297:29:56<99:40:09,  4.89s/it] 
 67%|██████▋   | 151682/225000 [297:30:01<99:54:19,  4.91s/it]
 67%|██████▋   | 151683/225000 [297:30:08<112:06:03,  5.50s/it]
 67%|██████▋   | 151684/225000 [297:30:19<148:01:44,  7.27s/it]
 67%|██████▋   | 151685/225000 [297:30:22<123:05:17,  6.04s/it]
 67%|██████▋   | 151686/225000 [297:30:30<136:23:59,  6.70s/it]
 67%|██████▋   | 151687/225000 [297:30:35<122:33:04,  6.02s/it]
 67%|██████▋   | 151688/225000 [297:30:40<119:30:09,  5.87s/it]
 67%|██████▋   | 151689/225000 [297:30:46<116:01:53,  5.70s/it]
 67%|██████▋   | 151690/225000 [297:30:51<115:48:10,  5.69s/it]
 67%|██████▋   | 151691/225000 [297:30:56<112:56:09,  5.55s/it]
 67%|██████▋   | 151692/225000 [297:31:02<113:18:09,  5.56s/it]
 67%|██████▋   | 151693/225000 [297:31:11<133:23:32,  6.55s/it]
 67%|██████▋   | 151694/225000 [297:31:16<122:38:20,  6.02s/it]
 67%|██████▋   | 151695/225000 [297:31:21<115:58:31,  5.70s/it]
 67%|██████▋   | 151696/225000 [297:31:26<116:01:02,  5.70s/it]
 67%|██████▋   | 151697/225000 [297:31:32<114:58:40,  5.65s/it]
 67%|██████▋   | 151698/225000 [297:31:38<115:22:31,  5.67s/it]
 67%|██████▋   | 151699/225000 [297:31:43<111:48:57,  5.49s/it]
 67%|██████▋   | 151700/225000 [297:31:48<112:15:46,  5.51s/it]
                                                               
{'loss': '0.7858', 'grad_norm': '1.072', 'learning_rate': '1.304e-05', 'epoch': '0.6742', 'num_input_tokens_seen': 492903360, 'train_runtime': '1.071e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151700/225000 [297:31:48<112:15:46,  5.51s/it]
 67%|██████▋   | 151701/225000 [297:31:55<117:50:26,  5.79s/it]
 67%|██████▋   | 151702/225000 [297:32:03<134:55:30,  6.63s/it]
 67%|██████▋   | 151703/225000 [297:32:08<121:57:14,  5.99s/it]
 67%|██████▋   | 151704/225000 [297:32:12<113:34:14,  5.58s/it]
 67%|██████▋   | 151705/225000 [297:32:17<105:57:00,  5.20s/it]
 67%|██████▋   | 151706/225000 [297:32:21<102:33:02,  5.04s/it]
 67%|██████▋   | 151707/225000 [297:32:26<99:04:53,  4.87s/it] 
 67%|██████▋   | 151708/225000 [297:32:34<117:21:41,  5.76s/it]
 67%|██████▋   | 151709/225000 [297:32:45<151:57:30,  7.46s/it]
 67%|██████▋   | 151710/225000 [297:32:48<126:15:54,  6.20s/it]
 67%|██████▋   | 151711/225000 [297:32:52<110:06:50,  5.41s/it]
 67%|██████▋   | 151712/225000 [297:32:59<120:53:17,  5.94s/it]
 67%|██████▋   | 151713/225000 [297:33:05<120:36:53,  5.92s/it]
 67%|██████▋   | 151714/225000 [297:33:09<106:57:47,  5.25s/it]
 67%|██████▋   | 151715/225000 [297:33:20<141:04:39,  6.93s/it]
 67%|██████▋   | 151716/225000 [297:33:24<126:23:22,  6.21s/it]
 67%|██████▋   | 151717/225000 [297:33:28<109:48:53,  5.39s/it]
 67%|██████▋   | 151718/225000 [297:33:33<109:19:46,  5.37s/it]
 67%|██████▋   | 151719/225000 [297:33:41<126:51:36,  6.23s/it]
 67%|██████▋   | 151720/225000 [297:33:46<116:50:38,  5.74s/it]
 67%|██████▋   | 151721/225000 [297:33:50<108:54:23,  5.35s/it]
 67%|██████▋   | 151722/225000 [297:33:55<103:58:27,  5.11s/it]
 67%|██████▋   | 151723/225000 [297:33:59<100:06:21,  4.92s/it]
 67%|██████▋   | 151724/225000 [297:34:04<98:15:29,  4.83s/it] 
 67%|██████▋   | 151725/225000 [297:34:16<142:48:24,  7.02s/it]
 67%|██████▋   | 151726/225000 [297:34:21<131:56:10,  6.48s/it]
 67%|██████▋   | 151727/225000 [297:34:25<116:07:57,  5.71s/it]
 67%|██████▋   | 151728/225000 [297:34:29<105:58:33,  5.21s/it]
 67%|██████▋   | 151729/225000 [297:34:36<117:22:28,  5.77s/it]
 67%|██████▋   | 151730/225000 [297:34:45<133:30:13,  6.56s/it]
 67%|██████▋   | 151731/225000 [297:34:48<114:18:34,  5.62s/it]
 67%|██████▋   | 151732/225000 [297:34:57<133:56:53,  6.58s/it]
 67%|██████▋   | 151733/225000 [297:35:01<117:09:18,  5.76s/it]
 67%|██████▋   | 151734/225000 [297:35:04<104:41:04,  5.14s/it]
 67%|██████▋   | 151735/225000 [297:35:14<133:09:52,  6.54s/it]
 67%|██████▋   | 151736/225000 [297:35:18<114:20:02,  5.62s/it]
 67%|██████▋   | 151737/225000 [297:35:22<106:38:26,  5.24s/it]
 67%|██████▋   | 151738/225000 [297:35:26<101:39:35,  5.00s/it]
 67%|██████▋   | 151739/225000 [297:35:31<97:42:21,  4.80s/it] 
 67%|██████▋   | 151740/225000 [297:35:41<133:43:06,  6.57s/it]
 67%|██████▋   | 151741/225000 [297:35:47<125:48:16,  6.18s/it]
 67%|██████▋   | 151742/225000 [297:35:53<123:38:43,  6.08s/it]
 67%|██████▋   | 151743/225000 [297:35:56<108:13:36,  5.32s/it]
 67%|██████▋   | 151744/225000 [297:36:01<108:10:14,  5.32s/it]
 67%|██████▋   | 151745/225000 [297:36:09<123:29:25,  6.07s/it]
 67%|██████▋   | 151746/225000 [297:36:13<108:05:01,  5.31s/it]
 67%|██████▋   | 151747/225000 [297:36:19<113:03:11,  5.56s/it]
 67%|██████▋   | 151748/225000 [297:36:26<124:50:19,  6.14s/it]
 67%|██████▋   | 151749/225000 [297:36:31<114:48:01,  5.64s/it]
 67%|██████▋   | 151750/225000 [297:36:36<108:39:16,  5.34s/it]
                                                               
{'loss': '0.7838', 'grad_norm': '1.045', 'learning_rate': '1.303e-05', 'epoch': '0.6744', 'num_input_tokens_seen': 493034584, 'train_runtime': '1.071e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151750/225000 [297:36:36<108:39:16,  5.34s/it]
 67%|██████▋   | 151751/225000 [297:36:40<102:23:02,  5.03s/it]
 67%|██████▋   | 151752/225000 [297:36:51<139:32:04,  6.86s/it]
 67%|██████▋   | 151753/225000 [297:36:54<117:17:48,  5.76s/it]
 67%|██████▋   | 151754/225000 [297:37:02<127:37:24,  6.27s/it]
 67%|██████▋   | 151755/225000 [297:37:07<119:26:21,  5.87s/it]
 67%|██████▋   | 151756/225000 [297:37:11<113:10:58,  5.56s/it]
 67%|██████▋   | 151757/225000 [297:37:16<107:51:51,  5.30s/it]
 67%|██████▋   | 151758/225000 [297:37:21<103:45:27,  5.10s/it]
 67%|██████▋   | 151759/225000 [297:37:26<104:13:00,  5.12s/it]
 67%|██████▋   | 151760/225000 [297:37:31<103:15:48,  5.08s/it]
 67%|██████▋   | 151761/225000 [297:37:35<100:33:03,  4.94s/it]
 67%|██████▋   | 151762/225000 [297:37:41<102:00:14,  5.01s/it]
 67%|██████▋   | 151763/225000 [297:37:52<142:59:20,  7.03s/it]
 67%|██████▋   | 151764/225000 [297:37:57<128:07:14,  6.30s/it]
 67%|██████▋   | 151765/225000 [297:38:02<120:43:30,  5.93s/it]
 67%|██████▋   | 151766/225000 [297:38:08<122:39:11,  6.03s/it]
 67%|██████▋   | 151767/225000 [297:38:16<132:35:11,  6.52s/it]
 67%|██████▋   | 151768/225000 [297:38:19<113:48:11,  5.59s/it]
 67%|██████▋   | 151769/225000 [297:38:27<125:39:03,  6.18s/it]
 67%|██████▋   | 151770/225000 [297:38:30<108:32:07,  5.34s/it]
 67%|██████▋   | 151771/225000 [297:38:38<123:57:57,  6.09s/it]
 67%|██████▋   | 151772/225000 [297:38:41<106:36:31,  5.24s/it]
 67%|██████▋   | 151773/225000 [297:38:50<128:53:15,  6.34s/it]
 67%|██████▋   | 151774/225000 [297:38:56<121:59:29,  6.00s/it]
 67%|██████▋   | 151775/225000 [297:39:01<119:24:21,  5.87s/it]
 67%|██████▋   | 151776/225000 [297:39:05<105:04:02,  5.17s/it]
 67%|██████▋   | 151777/225000 [297:39:13<124:12:55,  6.11s/it]
 67%|██████▋   | 151778/225000 [297:39:17<110:07:51,  5.41s/it]
 67%|██████▋   | 151779/225000 [297:39:22<109:34:12,  5.39s/it]
 67%|██████▋   | 151780/225000 [297:39:27<108:47:31,  5.35s/it]
 67%|██████▋   | 151781/225000 [297:39:32<107:34:13,  5.29s/it]
 67%|██████▋   | 151782/225000 [297:39:39<112:36:37,  5.54s/it]
 67%|██████▋   | 151783/225000 [297:39:46<123:55:11,  6.09s/it]
 67%|██████▋   | 151784/225000 [297:39:50<111:06:13,  5.46s/it]
 67%|██████▋   | 151785/225000 [297:39:54<99:56:47,  4.91s/it] 
 67%|██████▋   | 151786/225000 [297:40:04<130:20:51,  6.41s/it]
 67%|██████▋   | 151787/225000 [297:40:08<117:35:42,  5.78s/it]
 67%|██████▋   | 151788/225000 [297:40:12<105:46:27,  5.20s/it]
 67%|██████▋   | 151789/225000 [297:40:16<100:14:54,  4.93s/it]
 67%|██████▋   | 151790/225000 [297:40:20<96:45:35,  4.76s/it] 
 67%|██████▋   | 151791/225000 [297:40:30<126:56:12,  6.24s/it]
 67%|██████▋   | 151792/225000 [297:40:35<119:30:14,  5.88s/it]
 67%|██████▋   | 151793/225000 [297:40:39<105:15:31,  5.18s/it]
 67%|██████▋   | 151794/225000 [297:40:48<129:01:56,  6.35s/it]
 67%|██████▋   | 151795/225000 [297:40:52<114:17:29,  5.62s/it]
 67%|██████▋   | 151796/225000 [297:40:56<104:10:03,  5.12s/it]
 67%|██████▋   | 151797/225000 [297:41:02<113:16:02,  5.57s/it]
 67%|██████▋   | 151798/225000 [297:41:12<139:53:04,  6.88s/it]
 67%|██████▋   | 151799/225000 [297:41:16<119:46:11,  5.89s/it]
 67%|██████▋   | 151800/225000 [297:41:20<113:07:32,  5.56s/it]
                                                               
{'loss': '0.7856', 'grad_norm': '1.17', 'learning_rate': '1.301e-05', 'epoch': '0.6747', 'num_input_tokens_seen': 493164920, 'train_runtime': '1.072e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151800/225000 [297:41:20<113:07:32,  5.56s/it]
 67%|██████▋   | 151801/225000 [297:41:27<120:55:24,  5.95s/it]
 67%|██████▋   | 151802/225000 [297:41:32<111:39:05,  5.49s/it]
 67%|██████▋   | 151803/225000 [297:41:36<101:53:57,  5.01s/it]
 67%|██████▋   | 151804/225000 [297:41:40<96:33:23,  4.75s/it] 
 67%|██████▋   | 151805/225000 [297:41:45<96:47:44,  4.76s/it]
 67%|██████▋   | 151806/225000 [297:41:57<144:07:18,  7.09s/it]
 67%|██████▋   | 151807/225000 [297:42:02<133:34:59,  6.57s/it]
 67%|██████▋   | 151808/225000 [297:42:06<116:40:59,  5.74s/it]
 67%|██████▋   | 151809/225000 [297:42:15<132:25:23,  6.51s/it]
 67%|██████▋   | 151810/225000 [297:42:21<130:27:55,  6.42s/it]
 67%|██████▋   | 151811/225000 [297:42:26<124:49:38,  6.14s/it]
 67%|██████▋   | 151812/225000 [297:42:30<112:49:20,  5.55s/it]
 67%|██████▋   | 151813/225000 [297:42:35<106:59:11,  5.26s/it]
 67%|██████▋   | 151814/225000 [297:42:39<100:59:39,  4.97s/it]
 67%|██████▋   | 151815/225000 [297:42:44<97:35:44,  4.80s/it] 
 67%|██████▋   | 151816/225000 [297:42:55<135:32:29,  6.67s/it]
 67%|██████▋   | 151817/225000 [297:42:58<117:44:26,  5.79s/it]
 67%|██████▋   | 151818/225000 [297:43:02<106:32:03,  5.24s/it]
 67%|██████▋   | 151819/225000 [297:43:11<124:02:12,  6.10s/it]
 67%|██████▋   | 151820/225000 [297:43:17<125:42:36,  6.18s/it]
 67%|██████▋   | 151821/225000 [297:43:22<119:59:00,  5.90s/it]
 67%|██████▋   | 151822/225000 [297:43:27<116:14:02,  5.72s/it]
 67%|██████▋   | 151823/225000 [297:43:33<112:06:10,  5.51s/it]
 67%|██████▋   | 151824/225000 [297:43:37<106:51:00,  5.26s/it]
 67%|██████▋   | 151825/225000 [297:43:43<107:52:12,  5.31s/it]
 67%|██████▋   | 151826/225000 [297:43:48<106:52:00,  5.26s/it]
 67%|██████▋   | 151827/225000 [297:43:52<103:17:05,  5.08s/it]
 67%|██████▋   | 151828/225000 [297:43:58<104:57:11,  5.16s/it]
 67%|██████▋   | 151829/225000 [297:44:03<104:18:54,  5.13s/it]
 67%|██████▋   | 151830/225000 [297:44:08<103:47:54,  5.11s/it]
 67%|██████▋   | 151831/225000 [297:44:13<103:47:25,  5.11s/it]
 67%|██████▋   | 151832/225000 [297:44:18<103:53:51,  5.11s/it]
 67%|██████▋   | 151833/225000 [297:44:24<105:58:42,  5.21s/it]
 67%|██████▋   | 151834/225000 [297:44:29<106:45:27,  5.25s/it]
 67%|██████▋   | 151835/225000 [297:44:34<105:57:11,  5.21s/it]
 67%|██████▋   | 151836/225000 [297:44:39<106:45:43,  5.25s/it]
 67%|██████▋   | 151837/225000 [297:44:44<105:59:20,  5.22s/it]
 67%|██████▋   | 151838/225000 [297:44:51<113:38:29,  5.59s/it]
 67%|██████▋   | 151839/225000 [297:44:56<112:17:39,  5.53s/it]
 67%|██████▋   | 151840/225000 [297:45:02<112:23:52,  5.53s/it]
 67%|██████▋   | 151841/225000 [297:45:07<110:56:33,  5.46s/it]
 67%|██████▋   | 151842/225000 [297:45:12<108:43:07,  5.35s/it]
 67%|██████▋   | 151843/225000 [297:45:17<108:01:09,  5.32s/it]
 67%|██████▋   | 151844/225000 [297:45:23<108:37:27,  5.35s/it]
 67%|██████▋   | 151845/225000 [297:45:28<107:27:16,  5.29s/it]
 67%|██████▋   | 151846/225000 [297:45:33<104:13:05,  5.13s/it]
 67%|██████▋   | 151847/225000 [297:45:42<129:25:12,  6.37s/it]
 67%|██████▋   | 151848/225000 [297:45:53<155:11:33,  7.64s/it]
 67%|██████▋   | 151849/225000 [297:45:57<132:00:33,  6.50s/it]
 67%|██████▋   | 151850/225000 [297:46:03<129:17:37,  6.36s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '1.029', 'learning_rate': '1.299e-05', 'epoch': '0.6749', 'num_input_tokens_seen': 493294616, 'train_runtime': '1.072e+06', 'train_tokens_per_second': '460.2'}

 67%|██████▋   | 151850/225000 [297:46:03<129:17:37,  6.36s/it]
 67%|██████▋   | 151851/225000 [297:46:09<131:00:23,  6.45s/it]
 67%|██████▋   | 151852/225000 [297:46:14<118:28:58,  5.83s/it]
 67%|██████▋   | 151853/225000 [297:46:18<111:54:19,  5.51s/it]
 67%|██████▋   | 151854/225000 [297:46:23<107:24:03,  5.29s/it]
 67%|██████▋   | 151855/225000 [297:46:28<104:44:59,  5.16s/it]
 67%|██████▋   | 151856/225000 [297:46:33<102:11:18,  5.03s/it]
 67%|██████▋   | 151857/225000 [297:46:45<148:07:48,  7.29s/it]
 67%|██████▋   | 151858/225000 [297:46:49<125:35:00,  6.18s/it]
 67%|██████▋   | 151859/225000 [297:46:53<113:20:00,  5.58s/it]
 67%|██████▋   | 151860/225000 [297:47:03<139:18:27,  6.86s/it]
 67%|██████▋   | 151861/225000 [297:47:07<124:02:00,  6.11s/it]
 67%|██████▋   | 151862/225000 [297:47:12<113:36:29,  5.59s/it]
 67%|██████▋   | 151863/225000 [297:47:19<123:47:41,  6.09s/it]
 67%|██████▋   | 151864/225000 [297:47:23<113:29:46,  5.59s/it]
 67%|██████▋   | 151865/225000 [297:47:28<108:04:02,  5.32s/it]
 67%|██████▋   | 151866/225000 [297:47:33<103:32:02,  5.10s/it]
 67%|██████▋   | 151867/225000 [297:47:37<99:13:29,  4.88s/it] 
 67%|██████▋   | 151868/225000 [297:47:41<96:23:18,  4.74s/it]
 67%|██████▋   | 151869/225000 [297:47:53<138:39:50,  6.83s/it]
 67%|██████▋   | 151870/225000 [297:47:57<118:29:36,  5.83s/it]
 67%|██████▋   | 151871/225000 [297:48:01<110:23:22,  5.43s/it]
 67%|██████▋   | 151872/225000 [297:48:07<112:26:06,  5.54s/it]
 67%|██████▋   | 151873/225000 [297:48:14<121:58:37,  6.00s/it]
 67%|██████▋   | 151874/225000 [297:48:21<130:47:51,  6.44s/it]
 68%|██████▊   | 151875/225000 [297:48:25<115:26:25,  5.68s/it]
 68%|██████▊   | 151876/225000 [297:48:29<104:49:56,  5.16s/it]
 68%|██████▊   | 151877/225000 [297:48:40<139:46:31,  6.88s/it]
 68%|██████▊   | 151878/225000 [297:48:44<119:22:03,  5.88s/it]
 68%|██████▊   | 151879/225000 [297:48:47<105:16:58,  5.18s/it]
 68%|██████▊   | 151880/225000 [297:48:58<137:26:57,  6.77s/it]
 68%|██████▊   | 151881/225000 [297:49:01<118:54:50,  5.85s/it]
 68%|██████▊   | 151882/225000 [297:49:05<105:21:51,  5.19s/it]
 68%|██████▊   | 151883/225000 [297:49:09<97:15:17,  4.79s/it] 
 68%|██████▊   | 151884/225000 [297:49:20<136:38:46,  6.73s/it]
 68%|██████▊   | 151885/225000 [297:49:24<117:39:39,  5.79s/it]
 68%|██████▊   | 151886/225000 [297:49:27<104:22:14,  5.14s/it]
 68%|██████▊   | 151887/225000 [297:49:36<123:28:33,  6.08s/it]
 68%|██████▊   | 151888/225000 [297:49:40<111:53:37,  5.51s/it]
 68%|██████▊   | 151889/225000 [297:49:44<104:02:04,  5.12s/it]
 68%|██████▊   | 151890/225000 [297:49:49<99:49:30,  4.92s/it] 
 68%|██████▊   | 151891/225000 [297:49:53<98:19:46,  4.84s/it]
 68%|██████▊   | 151892/225000 [297:49:58<96:45:17,  4.76s/it]
 68%|██████▊   | 151893/225000 [297:50:09<135:58:45,  6.70s/it]
 68%|██████▊   | 151894/225000 [297:50:16<137:11:50,  6.76s/it]
 68%|██████▊   | 151895/225000 [297:50:21<126:59:43,  6.25s/it]
 68%|██████▊   | 151896/225000 [297:50:25<115:44:02,  5.70s/it]
 68%|██████▊   | 151897/225000 [297:50:30<112:06:21,  5.52s/it]
 68%|██████▊   | 151898/225000 [297:50:35<107:11:43,  5.28s/it]
 68%|██████▊   | 151899/225000 [297:50:40<103:54:23,  5.12s/it]
 68%|██████▊   | 151900/225000 [297:50:45<102:30:59,  5.05s/it]
                                                               
{'loss': '0.78', 'grad_norm': '1.042', 'learning_rate': '1.298e-05', 'epoch': '0.6751', 'num_input_tokens_seen': 493425384, 'train_runtime': '1.072e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 151900/225000 [297:50:45<102:30:59,  5.05s/it]
 68%|██████▊   | 151901/225000 [297:50:50<104:02:54,  5.12s/it]
 68%|██████▊   | 151902/225000 [297:50:55<101:49:09,  5.01s/it]
 68%|██████▊   | 151903/225000 [297:51:04<126:31:52,  6.23s/it]
 68%|██████▊   | 151904/225000 [297:51:12<137:46:00,  6.79s/it]
 68%|██████▊   | 151905/225000 [297:51:16<122:10:24,  6.02s/it]
 68%|██████▊   | 151906/225000 [297:51:21<113:39:12,  5.60s/it]
 68%|██████▊   | 151907/225000 [297:51:26<111:12:50,  5.48s/it]
 68%|██████▊   | 151908/225000 [297:51:31<106:00:28,  5.22s/it]
 68%|██████▊   | 151909/225000 [297:51:35<102:45:50,  5.06s/it]
 68%|██████▊   | 151910/225000 [297:51:41<103:47:38,  5.11s/it]
 68%|██████▊   | 151911/225000 [297:51:53<150:16:39,  7.40s/it]
 68%|██████▊   | 151912/225000 [297:51:58<130:43:39,  6.44s/it]
 68%|██████▊   | 151913/225000 [297:52:05<134:28:45,  6.62s/it]
 68%|██████▊   | 151914/225000 [297:52:08<117:31:43,  5.79s/it]
 68%|██████▊   | 151915/225000 [297:52:12<105:28:36,  5.20s/it]
 68%|██████▊   | 151916/225000 [297:52:23<138:53:08,  6.84s/it]
 68%|██████▊   | 151917/225000 [297:52:26<117:34:27,  5.79s/it]
 68%|██████▊   | 151918/225000 [297:52:30<102:36:32,  5.05s/it]
 68%|██████▊   | 151919/225000 [297:52:39<129:59:39,  6.40s/it]
 68%|██████▊   | 151920/225000 [297:52:42<109:55:44,  5.42s/it]
 68%|██████▊   | 151921/225000 [297:52:46<97:39:43,  4.81s/it] 
 68%|██████▊   | 151922/225000 [297:52:55<127:33:50,  6.28s/it]
 68%|██████▊   | 151923/225000 [297:53:00<118:10:28,  5.82s/it]
 68%|██████▊   | 151924/225000 [297:53:05<111:16:47,  5.48s/it]
 68%|██████▊   | 151925/225000 [297:53:10<109:49:04,  5.41s/it]
 68%|██████▊   | 151926/225000 [297:53:15<105:25:24,  5.19s/it]
 68%|██████▊   | 151927/225000 [297:53:20<105:02:41,  5.18s/it]
 68%|██████▊   | 151928/225000 [297:53:25<104:10:04,  5.13s/it]
 68%|██████▊   | 151929/225000 [297:53:30<101:07:49,  4.98s/it]
 68%|██████▊   | 151930/225000 [297:53:34<100:53:44,  4.97s/it]
 68%|██████▊   | 151931/225000 [297:53:39<99:34:26,  4.91s/it] 
 68%|██████▊   | 151932/225000 [297:53:46<108:21:48,  5.34s/it]
 68%|██████▊   | 151933/225000 [297:53:58<149:33:24,  7.37s/it]
 68%|██████▊   | 151934/225000 [297:54:01<125:30:45,  6.18s/it]
 68%|██████▊   | 151935/225000 [297:54:06<117:27:27,  5.79s/it]
 68%|██████▊   | 151936/225000 [297:54:13<125:55:25,  6.20s/it]
 68%|██████▊   | 151937/225000 [297:54:17<111:35:47,  5.50s/it]
 68%|██████▊   | 151938/225000 [297:54:22<111:33:05,  5.50s/it]
 68%|██████▊   | 151939/225000 [297:54:31<132:31:12,  6.53s/it]
 68%|██████▊   | 151940/225000 [297:54:35<115:21:12,  5.68s/it]
 68%|██████▊   | 151941/225000 [297:54:43<128:26:52,  6.33s/it]
 68%|██████▊   | 151942/225000 [297:54:47<113:17:14,  5.58s/it]
 68%|██████▊   | 151943/225000 [297:54:51<106:49:47,  5.26s/it]
 68%|██████▊   | 151944/225000 [297:54:56<101:28:35,  5.00s/it]
 68%|██████▊   | 151945/225000 [297:55:06<135:19:34,  6.67s/it]
 68%|██████▊   | 151946/225000 [297:55:10<115:05:39,  5.67s/it]
 68%|██████▊   | 151947/225000 [297:55:16<121:44:31,  6.00s/it]
 68%|██████▊   | 151948/225000 [297:55:20<109:23:22,  5.39s/it]
 68%|██████▊   | 151949/225000 [297:55:24<99:57:43,  4.93s/it] 
 68%|██████▊   | 151950/225000 [297:55:34<131:04:07,  6.46s/it]
                                                               
{'loss': '0.791', 'grad_norm': '1.117', 'learning_rate': '1.296e-05', 'epoch': '0.6753', 'num_input_tokens_seen': 493554856, 'train_runtime': '1.073e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 151950/225000 [297:55:34<131:04:07,  6.46s/it]
 68%|██████▊   | 151951/225000 [297:55:39<118:37:14,  5.85s/it]
 68%|██████▊   | 151952/225000 [297:55:43<111:51:10,  5.51s/it]
 68%|██████▊   | 151953/225000 [297:55:48<105:17:02,  5.19s/it]
 68%|██████▊   | 151954/225000 [297:55:53<102:23:58,  5.05s/it]
 68%|██████▊   | 151955/225000 [297:55:57<101:17:56,  4.99s/it]
 68%|██████▊   | 151956/225000 [297:56:02<100:53:10,  4.97s/it]
 68%|██████▊   | 151957/225000 [297:56:09<111:40:45,  5.50s/it]
 68%|██████▊   | 151958/225000 [297:56:21<149:53:59,  7.39s/it]
 68%|██████▊   | 151959/225000 [297:56:24<126:15:51,  6.22s/it]
 68%|██████▊   | 151960/225000 [297:56:29<114:40:18,  5.65s/it]
 68%|██████▊   | 151961/225000 [297:56:36<121:50:08,  6.01s/it]
 68%|██████▊   | 151962/225000 [297:56:43<129:30:12,  6.38s/it]
 68%|██████▊   | 151963/225000 [297:56:47<116:10:31,  5.73s/it]
 68%|██████▊   | 151964/225000 [297:56:51<106:31:00,  5.25s/it]
 68%|██████▊   | 151965/225000 [297:56:57<112:11:14,  5.53s/it]
 68%|██████▊   | 151966/225000 [297:57:05<122:38:00,  6.04s/it]
 68%|██████▊   | 151967/225000 [297:57:09<111:51:05,  5.51s/it]
 68%|██████▊   | 151968/225000 [297:57:13<105:47:00,  5.21s/it]
 68%|██████▊   | 151969/225000 [297:57:18<100:09:42,  4.94s/it]
 68%|██████▊   | 151970/225000 [297:57:29<140:13:42,  6.91s/it]
 68%|██████▊   | 151971/225000 [297:57:33<122:03:17,  6.02s/it]
 68%|██████▊   | 151972/225000 [297:57:36<104:56:17,  5.17s/it]
 68%|██████▊   | 151973/225000 [297:57:45<125:54:18,  6.21s/it]
 68%|██████▊   | 151974/225000 [297:57:49<110:05:11,  5.43s/it]
 68%|██████▊   | 151975/225000 [297:57:56<122:15:06,  6.03s/it]
 68%|██████▊   | 151976/225000 [297:57:59<106:04:55,  5.23s/it]
 68%|██████▊   | 151977/225000 [297:58:07<123:46:27,  6.10s/it]
 68%|██████▊   | 151978/225000 [297:58:13<117:57:44,  5.82s/it]
 68%|██████▊   | 151979/225000 [297:58:19<119:52:44,  5.91s/it]
 68%|██████▊   | 151980/225000 [297:58:23<110:03:10,  5.43s/it]
 68%|██████▊   | 151981/225000 [297:58:27<102:48:34,  5.07s/it]
 68%|██████▊   | 151982/225000 [297:58:32<100:33:43,  4.96s/it]
 68%|██████▊   | 151983/225000 [297:58:36<97:57:22,  4.83s/it] 
 68%|██████▊   | 151984/225000 [297:58:43<106:47:50,  5.27s/it]
 68%|██████▊   | 151985/225000 [297:58:53<139:13:24,  6.86s/it]
 68%|██████▊   | 151986/225000 [297:58:57<117:51:36,  5.81s/it]
 68%|██████▊   | 151987/225000 [297:59:04<125:59:00,  6.21s/it]
 68%|██████▊   | 151988/225000 [297:59:08<112:56:36,  5.57s/it]
 68%|██████▊   | 151989/225000 [297:59:11<100:24:29,  4.95s/it]
 68%|██████▊   | 151990/225000 [297:59:21<128:21:46,  6.33s/it]
 68%|██████▊   | 151991/225000 [297:59:25<111:18:35,  5.49s/it]
 68%|██████▊   | 151992/225000 [297:59:28<101:36:55,  5.01s/it]
 68%|██████▊   | 151993/225000 [297:59:38<131:54:40,  6.50s/it]
 68%|██████▊   | 151994/225000 [297:59:42<112:40:36,  5.56s/it]
 68%|██████▊   | 151995/225000 [297:59:49<126:01:30,  6.21s/it]
 68%|██████▊   | 151996/225000 [297:59:55<119:53:23,  5.91s/it]
 68%|██████▊   | 151997/225000 [298:00:03<132:10:34,  6.52s/it]
 68%|██████▊   | 151998/225000 [298:00:07<121:07:56,  5.97s/it]
 68%|██████▊   | 151999/225000 [298:00:12<113:45:20,  5.61s/it]
 68%|██████▊   | 152000/225000 [298:00:17<111:56:38,  5.52s/it]
                                                               
{'loss': '0.7902', 'grad_norm': '1.064', 'learning_rate': '1.295e-05', 'epoch': '0.6756', 'num_input_tokens_seen': 493684944, 'train_runtime': '1.073e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152000/225000 [298:00:17<111:56:38,  5.52s/it]
 68%|██████▊   | 152001/225000 [298:00:22<105:05:27,  5.18s/it]
 68%|██████▊   | 152002/225000 [298:00:26<100:53:40,  4.98s/it]
 68%|██████▊   | 152003/225000 [298:00:31<100:42:54,  4.97s/it]
 68%|██████▊   | 152004/225000 [298:00:36<97:59:21,  4.83s/it] 
 68%|██████▊   | 152005/225000 [298:00:41<98:26:20,  4.85s/it]
 68%|██████▊   | 152006/225000 [298:00:46<99:40:10,  4.92s/it]
 68%|██████▊   | 152007/225000 [298:00:51<102:16:56,  5.04s/it]
 68%|██████▊   | 152008/225000 [298:00:56<103:07:01,  5.09s/it]
 68%|██████▊   | 152009/225000 [298:01:05<124:50:30,  6.16s/it]
 68%|██████▊   | 152010/225000 [298:01:16<155:06:23,  7.65s/it]
 68%|██████▊   | 152011/225000 [298:01:19<129:22:41,  6.38s/it]
 68%|██████▊   | 152012/225000 [298:01:27<135:58:29,  6.71s/it]
 68%|██████▊   | 152013/225000 [298:01:30<115:29:27,  5.70s/it]
 68%|██████▊   | 152014/225000 [298:01:34<106:04:53,  5.23s/it]
 68%|██████▊   | 152015/225000 [298:01:45<137:02:13,  6.76s/it]
 68%|██████▊   | 152016/225000 [298:01:49<119:37:30,  5.90s/it]
 68%|██████▊   | 152017/225000 [298:01:53<110:15:17,  5.44s/it]
 68%|██████▊   | 152018/225000 [298:02:01<128:02:42,  6.32s/it]
 68%|██████▊   | 152019/225000 [298:02:08<127:34:30,  6.29s/it]
 68%|██████▊   | 152020/225000 [298:02:11<112:25:17,  5.55s/it]
 68%|██████▊   | 152021/225000 [298:02:16<105:12:20,  5.19s/it]
 68%|██████▊   | 152022/225000 [298:02:20<98:28:32,  4.86s/it] 
 68%|██████▊   | 152023/225000 [298:02:24<94:46:02,  4.67s/it]
 68%|██████▊   | 152024/225000 [298:02:33<121:59:49,  6.02s/it]
 68%|██████▊   | 152025/225000 [298:02:41<131:55:34,  6.51s/it]
 68%|██████▊   | 152026/225000 [298:02:44<110:34:52,  5.46s/it]
 68%|██████▊   | 152027/225000 [298:02:49<109:22:12,  5.40s/it]
 68%|██████▊   | 152028/225000 [298:02:54<106:24:38,  5.25s/it]
 68%|██████▊   | 152029/225000 [298:02:59<106:57:14,  5.28s/it]
 68%|██████▊   | 152030/225000 [298:03:04<105:30:20,  5.21s/it]
 68%|██████▊   | 152031/225000 [298:03:10<105:56:36,  5.23s/it]
 68%|██████▊   | 152032/225000 [298:03:15<106:16:22,  5.24s/it]
 68%|██████▊   | 152033/225000 [298:03:20<104:38:25,  5.16s/it]
 68%|██████▊   | 152034/225000 [298:03:25<106:18:04,  5.24s/it]
 68%|██████▊   | 152035/225000 [298:03:30<105:04:14,  5.18s/it]
 68%|██████▊   | 152036/225000 [298:03:35<102:55:04,  5.08s/it]
 68%|██████▊   | 152037/225000 [298:03:41<104:10:47,  5.14s/it]
 68%|██████▊   | 152038/225000 [298:03:46<103:34:38,  5.11s/it]
 68%|██████▊   | 152039/225000 [298:03:51<105:56:20,  5.23s/it]
 68%|██████▊   | 152040/225000 [298:03:56<106:23:06,  5.25s/it]
 68%|██████▊   | 152041/225000 [298:04:02<106:11:10,  5.24s/it]
 68%|██████▊   | 152042/225000 [298:04:07<104:36:02,  5.16s/it]
 68%|██████▊   | 152043/225000 [298:04:12<106:41:17,  5.26s/it]
 68%|██████▊   | 152044/225000 [298:04:20<123:59:45,  6.12s/it]
 68%|██████▊   | 152045/225000 [298:04:33<163:47:27,  8.08s/it]
 68%|██████▊   | 152046/225000 [298:04:36<136:07:28,  6.72s/it]
 68%|██████▊   | 152047/225000 [298:04:44<143:09:00,  7.06s/it]
 68%|██████▊   | 152048/225000 [298:04:48<123:02:44,  6.07s/it]
 68%|██████▊   | 152049/225000 [298:04:52<111:21:35,  5.50s/it]
 68%|██████▊   | 152050/225000 [298:05:00<123:03:35,  6.07s/it]
                                                               
{'loss': '0.7833', 'grad_norm': '1.029', 'learning_rate': '1.293e-05', 'epoch': '0.6758', 'num_input_tokens_seen': 493813632, 'train_runtime': '1.073e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152050/225000 [298:05:00<123:03:35,  6.07s/it]
 68%|██████▊   | 152051/225000 [298:05:07<133:31:08,  6.59s/it]
 68%|██████▊   | 152052/225000 [298:05:11<116:36:56,  5.76s/it]
 68%|██████▊   | 152053/225000 [298:05:16<108:06:50,  5.34s/it]
 68%|██████▊   | 152054/225000 [298:05:26<138:15:29,  6.82s/it]
 68%|██████▊   | 152055/225000 [298:05:30<118:56:28,  5.87s/it]
 68%|██████▊   | 152056/225000 [298:05:36<123:59:28,  6.12s/it]
 68%|██████▊   | 152057/225000 [298:05:40<109:51:03,  5.42s/it]
 68%|██████▊   | 152058/225000 [298:05:44<100:15:40,  4.95s/it]
 68%|██████▊   | 152059/225000 [298:05:54<129:59:00,  6.42s/it]
 68%|██████▊   | 152060/225000 [298:06:00<130:00:46,  6.42s/it]
 68%|██████▊   | 152061/225000 [298:06:04<117:00:18,  5.77s/it]
 68%|██████▊   | 152062/225000 [298:06:08<106:03:46,  5.23s/it]
 68%|██████▊   | 152063/225000 [298:06:13<100:59:15,  4.98s/it]
 68%|██████▊   | 152064/225000 [298:06:23<133:29:31,  6.59s/it]
 68%|██████▊   | 152065/225000 [298:06:27<116:48:20,  5.77s/it]
 68%|██████▊   | 152066/225000 [298:06:31<104:51:24,  5.18s/it]
 68%|██████▊   | 152067/225000 [298:06:39<124:49:00,  6.16s/it]
 68%|██████▊   | 152068/225000 [298:06:44<116:47:47,  5.77s/it]
 68%|██████▊   | 152069/225000 [298:06:51<125:04:19,  6.17s/it]
 68%|██████▊   | 152070/225000 [298:06:55<108:18:02,  5.35s/it]
 68%|██████▊   | 152071/225000 [298:06:59<105:42:51,  5.22s/it]
 68%|██████▊   | 152072/225000 [298:07:05<106:23:08,  5.25s/it]
 68%|██████▊   | 152073/225000 [298:07:10<104:05:00,  5.14s/it]
 68%|██████▊   | 152074/225000 [298:07:15<105:06:34,  5.19s/it]
 68%|██████▊   | 152075/225000 [298:07:20<106:02:27,  5.23s/it]
 68%|██████▊   | 152076/225000 [298:07:25<105:14:53,  5.20s/it]
 68%|██████▊   | 152077/225000 [298:07:31<105:28:13,  5.21s/it]
 68%|██████▊   | 152078/225000 [298:07:36<103:36:29,  5.11s/it]
 68%|██████▊   | 152079/225000 [298:07:41<104:12:02,  5.14s/it]
 68%|██████▊   | 152080/225000 [298:07:50<130:37:16,  6.45s/it]
 68%|██████▊   | 152081/225000 [298:08:00<150:27:36,  7.43s/it]
 68%|██████▊   | 152082/225000 [298:08:04<131:19:05,  6.48s/it]
 68%|██████▊   | 152083/225000 [298:08:09<117:46:12,  5.81s/it]
 68%|██████▊   | 152084/225000 [298:08:13<109:44:25,  5.42s/it]
 68%|██████▊   | 152085/225000 [298:08:17<103:58:03,  5.13s/it]
 68%|██████▊   | 152086/225000 [298:08:22<101:14:41,  5.00s/it]
 68%|██████▊   | 152087/225000 [298:08:27<100:14:38,  4.95s/it]
 68%|██████▊   | 152088/225000 [298:08:32<98:14:04,  4.85s/it] 
 68%|██████▊   | 152089/225000 [298:08:36<94:35:42,  4.67s/it]
 68%|██████▊   | 152090/225000 [298:08:47<136:38:02,  6.75s/it]
 68%|██████▊   | 152091/225000 [298:08:53<131:31:24,  6.49s/it]
 68%|██████▊   | 152092/225000 [298:08:58<117:48:07,  5.82s/it]
 68%|██████▊   | 152093/225000 [298:09:02<108:03:50,  5.34s/it]
 68%|██████▊   | 152094/225000 [298:09:06<101:13:05,  5.00s/it]
 68%|██████▊   | 152095/225000 [298:09:15<127:56:52,  6.32s/it]
 68%|██████▊   | 152096/225000 [298:09:24<142:03:26,  7.01s/it]
 68%|██████▊   | 152097/225000 [298:09:28<123:03:40,  6.08s/it]
 68%|██████▊   | 152098/225000 [298:09:32<112:01:58,  5.53s/it]
 68%|██████▊   | 152099/225000 [298:09:36<103:08:37,  5.09s/it]
 68%|██████▊   | 152100/225000 [298:09:46<128:48:32,  6.36s/it]
                                                               
{'loss': '0.7806', 'grad_norm': '1.098', 'learning_rate': '1.291e-05', 'epoch': '0.676', 'num_input_tokens_seen': 493944208, 'train_runtime': '1.073e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152100/225000 [298:09:46<128:48:32,  6.36s/it]
 68%|██████▊   | 152101/225000 [298:09:52<127:43:49,  6.31s/it]
 68%|██████▊   | 152102/225000 [298:09:57<120:36:17,  5.96s/it]
 68%|██████▊   | 152103/225000 [298:10:02<114:31:58,  5.66s/it]
 68%|██████▊   | 152104/225000 [298:10:07<112:31:02,  5.56s/it]
 68%|██████▊   | 152105/225000 [298:10:12<109:02:34,  5.39s/it]
 68%|██████▊   | 152106/225000 [298:10:18<108:48:36,  5.37s/it]
 68%|██████▊   | 152107/225000 [298:10:22<104:10:05,  5.14s/it]
 68%|██████▊   | 152108/225000 [298:10:31<124:16:43,  6.14s/it]
 68%|██████▊   | 152109/225000 [298:10:39<135:30:32,  6.69s/it]
 68%|██████▊   | 152110/225000 [298:10:43<120:20:08,  5.94s/it]
 68%|██████▊   | 152111/225000 [298:10:47<108:49:01,  5.37s/it]
 68%|██████▊   | 152112/225000 [298:10:54<117:27:25,  5.80s/it]
 68%|██████▊   | 152113/225000 [298:11:03<137:44:30,  6.80s/it]
 68%|██████▊   | 152114/225000 [298:11:08<125:20:55,  6.19s/it]
 68%|██████▊   | 152115/225000 [298:11:14<127:34:01,  6.30s/it]
 68%|██████▊   | 152116/225000 [298:11:18<111:15:49,  5.50s/it]
 68%|██████▊   | 152117/225000 [298:11:27<134:15:05,  6.63s/it]
 68%|██████▊   | 152118/225000 [298:11:31<118:04:48,  5.83s/it]
 68%|██████▊   | 152119/225000 [298:11:35<105:52:22,  5.23s/it]
 68%|██████▊   | 152120/225000 [298:11:43<123:17:41,  6.09s/it]
 68%|██████▊   | 152121/225000 [298:11:50<127:30:48,  6.30s/it]
 68%|██████▊   | 152122/225000 [298:11:55<118:48:59,  5.87s/it]
 68%|██████▊   | 152123/225000 [298:11:59<112:52:58,  5.58s/it]
 68%|██████▊   | 152124/225000 [298:12:04<108:47:16,  5.37s/it]
 68%|██████▊   | 152125/225000 [298:12:10<108:32:51,  5.36s/it]
 68%|██████▊   | 152126/225000 [298:12:14<104:45:14,  5.17s/it]
 68%|██████▊   | 152127/225000 [298:12:19<98:58:21,  4.89s/it] 
 68%|██████▊   | 152128/225000 [298:12:24<100:51:03,  4.98s/it]
 68%|██████▊   | 152129/225000 [298:12:29<100:32:02,  4.97s/it]
 68%|██████▊   | 152130/225000 [298:12:35<106:28:47,  5.26s/it]
 68%|██████▊   | 152131/225000 [298:12:47<149:51:45,  7.40s/it]
 68%|██████▊   | 152132/225000 [298:12:51<130:06:38,  6.43s/it]
 68%|██████▊   | 152133/225000 [298:12:58<130:40:22,  6.46s/it]
 68%|██████▊   | 152134/225000 [298:13:01<113:21:08,  5.60s/it]
 68%|██████▊   | 152135/225000 [298:13:10<133:18:35,  6.59s/it]
 68%|██████▊   | 152136/225000 [298:13:14<114:04:41,  5.64s/it]
 68%|██████▊   | 152137/225000 [298:13:17<100:53:51,  4.99s/it]
 68%|██████▊   | 152138/225000 [298:13:22<99:08:31,  4.90s/it] 
 68%|██████▊   | 152139/225000 [298:13:27<98:56:55,  4.89s/it]
 68%|██████▊   | 152140/225000 [298:13:37<130:53:41,  6.47s/it]
 68%|██████▊   | 152141/225000 [298:13:44<133:50:01,  6.61s/it]
 68%|██████▊   | 152142/225000 [298:13:49<127:36:11,  6.31s/it]
 68%|██████▊   | 152143/225000 [298:13:54<114:42:14,  5.67s/it]
 68%|██████▊   | 152144/225000 [298:13:58<104:38:04,  5.17s/it]
 68%|██████▊   | 152145/225000 [298:14:02<100:25:30,  4.96s/it]
 68%|██████▊   | 152146/225000 [298:14:06<95:06:15,  4.70s/it] 
 68%|██████▊   | 152147/225000 [298:14:16<124:50:29,  6.17s/it]
 68%|██████▊   | 152148/225000 [298:14:24<135:48:40,  6.71s/it]
 68%|██████▊   | 152149/225000 [298:14:29<126:00:48,  6.23s/it]
 68%|██████▊   | 152150/225000 [298:14:34<120:22:55,  5.95s/it]
                                                               
{'loss': '0.7762', 'grad_norm': '0.9976', 'learning_rate': '1.29e-05', 'epoch': '0.6762', 'num_input_tokens_seen': 494075864, 'train_runtime': '1.074e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152150/225000 [298:14:34<120:22:55,  5.95s/it]
 68%|██████▊   | 152151/225000 [298:14:39<115:02:27,  5.69s/it]
 68%|██████▊   | 152152/225000 [298:14:44<111:49:54,  5.53s/it]
 68%|██████▊   | 152153/225000 [298:14:49<108:59:51,  5.39s/it]
 68%|██████▊   | 152154/225000 [298:14:55<107:40:22,  5.32s/it]
 68%|██████▊   | 152155/225000 [298:15:00<107:41:37,  5.32s/it]
 68%|██████▊   | 152156/225000 [298:15:05<105:48:43,  5.23s/it]
 68%|██████▊   | 152157/225000 [298:15:10<104:22:25,  5.16s/it]
 68%|██████▊   | 152158/225000 [298:15:15<103:17:20,  5.10s/it]
 68%|██████▊   | 152159/225000 [298:15:20<102:30:24,  5.07s/it]
 68%|██████▊   | 152160/225000 [298:15:25<103:24:57,  5.11s/it]
 68%|██████▊   | 152161/225000 [298:15:30<104:11:24,  5.15s/it]
 68%|██████▊   | 152162/225000 [298:15:40<130:50:40,  6.47s/it]
 68%|██████▊   | 152163/225000 [298:15:50<151:23:19,  7.48s/it]
 68%|██████▊   | 152164/225000 [298:15:53<128:06:02,  6.33s/it]
 68%|██████▊   | 152165/225000 [298:15:57<112:24:06,  5.56s/it]
 68%|██████▊   | 152166/225000 [298:16:04<121:14:41,  5.99s/it]
 68%|██████▊   | 152167/225000 [298:16:13<138:00:44,  6.82s/it]
 68%|██████▊   | 152168/225000 [298:16:19<134:33:57,  6.65s/it]
 68%|██████▊   | 152169/225000 [298:16:23<116:19:37,  5.75s/it]
 68%|██████▊   | 152170/225000 [298:16:30<127:25:56,  6.30s/it]
 68%|██████▊   | 152171/225000 [298:16:34<109:06:51,  5.39s/it]
 68%|██████▊   | 152172/225000 [298:16:39<110:15:15,  5.45s/it]
 68%|██████▊   | 152173/225000 [298:16:47<121:22:22,  6.00s/it]
 68%|██████▊   | 152174/225000 [298:16:50<108:04:01,  5.34s/it]
 68%|██████▊   | 152175/225000 [298:16:54<99:13:36,  4.91s/it] 
 68%|██████▊   | 152176/225000 [298:17:03<122:20:37,  6.05s/it]
 68%|██████▊   | 152177/225000 [298:17:07<108:22:22,  5.36s/it]
 68%|██████▊   | 152178/225000 [298:17:12<105:11:38,  5.20s/it]
 68%|██████▊   | 152179/225000 [298:17:20<123:06:49,  6.09s/it]
 68%|██████▊   | 152180/225000 [298:17:25<120:17:25,  5.95s/it]
 68%|██████▊   | 152181/225000 [298:17:33<128:39:21,  6.36s/it]
 68%|██████▊   | 152182/225000 [298:17:38<121:56:35,  6.03s/it]
 68%|██████▊   | 152183/225000 [298:17:43<117:32:25,  5.81s/it]
 68%|██████▊   | 152184/225000 [298:17:51<129:44:05,  6.41s/it]
 68%|██████▊   | 152185/225000 [298:17:54<112:08:58,  5.54s/it]
 68%|██████▊   | 152186/225000 [298:17:58<100:55:37,  4.99s/it]
 68%|██████▊   | 152187/225000 [298:18:03<101:26:05,  5.02s/it]
 68%|██████▊   | 152188/225000 [298:18:13<129:09:47,  6.39s/it]
 68%|██████▊   | 152189/225000 [298:18:16<111:06:34,  5.49s/it]
 68%|██████▊   | 152190/225000 [298:18:26<135:17:40,  6.69s/it]
 68%|██████▊   | 152191/225000 [298:18:29<117:24:17,  5.81s/it]
 68%|██████▊   | 152192/225000 [298:18:34<107:07:36,  5.30s/it]
 68%|██████▊   | 152193/225000 [298:18:38<101:07:23,  5.00s/it]
 68%|██████▊   | 152194/225000 [298:18:46<119:51:08,  5.93s/it]
 68%|██████▊   | 152195/225000 [298:18:54<129:37:34,  6.41s/it]
 68%|██████▊   | 152196/225000 [298:18:58<119:26:05,  5.91s/it]
 68%|██████▊   | 152197/225000 [298:19:03<112:08:39,  5.55s/it]
 68%|██████▊   | 152198/225000 [298:19:07<104:56:47,  5.19s/it]
 68%|██████▊   | 152199/225000 [298:19:12<101:41:29,  5.03s/it]
 68%|██████▊   | 152200/225000 [298:19:16<98:15:03,  4.86s/it] 
                                                              
{'loss': '0.7643', 'grad_norm': '1.242', 'learning_rate': '1.288e-05', 'epoch': '0.6764', 'num_input_tokens_seen': 494205688, 'train_runtime': '1.074e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152200/225000 [298:19:16<98:15:03,  4.86s/it]
 68%|██████▊   | 152201/225000 [298:19:22<100:36:02,  4.97s/it]
 68%|██████▊   | 152202/225000 [298:19:31<128:47:47,  6.37s/it]
 68%|██████▊   | 152203/225000 [298:19:40<140:43:26,  6.96s/it]
 68%|██████▊   | 152204/225000 [298:19:44<122:12:25,  6.04s/it]
 68%|██████▊   | 152205/225000 [298:19:47<107:33:05,  5.32s/it]
 68%|██████▊   | 152206/225000 [298:19:54<113:40:30,  5.62s/it]
 68%|██████▊   | 152207/225000 [298:20:02<130:33:24,  6.46s/it]
 68%|██████▊   | 152208/225000 [298:20:06<113:27:17,  5.61s/it]
 68%|██████▊   | 152209/225000 [298:20:16<140:41:04,  6.96s/it]
 68%|██████▊   | 152210/225000 [298:20:20<124:45:34,  6.17s/it]
 68%|██████▊   | 152211/225000 [298:20:24<110:55:02,  5.49s/it]
 68%|██████▊   | 152212/225000 [298:20:28<101:38:04,  5.03s/it]
 68%|██████▊   | 152213/225000 [298:20:39<136:30:39,  6.75s/it]
 68%|██████▊   | 152214/225000 [298:20:44<126:19:05,  6.25s/it]
 68%|██████▊   | 152215/225000 [298:20:49<121:56:56,  6.03s/it]
 68%|██████▊   | 152216/225000 [298:20:54<112:24:21,  5.56s/it]
 68%|██████▊   | 152217/225000 [298:20:59<108:45:20,  5.38s/it]
 68%|██████▊   | 152218/225000 [298:21:03<105:36:02,  5.22s/it]
 68%|██████▊   | 152219/225000 [298:21:08<103:30:58,  5.12s/it]
 68%|██████▊   | 152220/225000 [298:21:13<99:32:53,  4.92s/it] 
 68%|██████▊   | 152221/225000 [298:21:18<99:50:52,  4.94s/it]
 68%|██████▊   | 152222/225000 [298:21:23<100:12:43,  4.96s/it]
 68%|██████▊   | 152223/225000 [298:21:28<100:41:10,  4.98s/it]
 68%|██████▊   | 152224/225000 [298:21:33<100:44:11,  4.98s/it]
 68%|██████▊   | 152225/225000 [298:21:38<100:55:35,  4.99s/it]
 68%|██████▊   | 152226/225000 [298:21:43<101:18:27,  5.01s/it]
 68%|██████▊   | 152227/225000 [298:21:49<110:42:46,  5.48s/it]
 68%|██████▊   | 152228/225000 [298:22:02<156:00:07,  7.72s/it]
 68%|██████▊   | 152229/225000 [298:22:06<131:25:55,  6.50s/it]
 68%|██████▊   | 152230/225000 [298:22:11<119:12:24,  5.90s/it]
 68%|██████▊   | 152231/225000 [298:22:15<112:51:53,  5.58s/it]
 68%|██████▊   | 152232/225000 [298:22:20<107:47:12,  5.33s/it]
 68%|██████▊   | 152233/225000 [298:22:25<104:05:34,  5.15s/it]
 68%|██████▊   | 152234/225000 [298:22:29<100:12:09,  4.96s/it]
 68%|██████▊   | 152235/225000 [298:22:35<106:06:24,  5.25s/it]
 68%|██████▊   | 152236/225000 [298:22:46<139:02:14,  6.88s/it]
 68%|██████▊   | 152237/225000 [298:22:50<120:18:52,  5.95s/it]
 68%|██████▊   | 152238/225000 [298:22:54<107:21:35,  5.31s/it]
 68%|██████▊   | 152239/225000 [298:22:59<107:12:47,  5.30s/it]
 68%|██████▊   | 152240/225000 [298:23:04<104:26:24,  5.17s/it]
 68%|██████▊   | 152241/225000 [298:23:09<105:37:48,  5.23s/it]
 68%|██████▊   | 152242/225000 [298:23:14<102:41:49,  5.08s/it]
 68%|██████▊   | 152243/225000 [298:23:25<140:07:50,  6.93s/it]
 68%|██████▊   | 152244/225000 [298:23:31<133:21:32,  6.60s/it]
 68%|██████▊   | 152245/225000 [298:23:35<117:00:04,  5.79s/it]
 68%|██████▊   | 152246/225000 [298:23:39<107:41:00,  5.33s/it]
 68%|██████▊   | 152247/225000 [298:23:49<138:07:29,  6.83s/it]
 68%|██████▊   | 152248/225000 [298:23:53<116:59:30,  5.79s/it]
 68%|██████▊   | 152249/225000 [298:24:02<135:18:28,  6.70s/it]
 68%|██████▊   | 152250/225000 [298:24:06<120:23:33,  5.96s/it]
                                                               
{'loss': '0.7754', 'grad_norm': '1.081', 'learning_rate': '1.287e-05', 'epoch': '0.6767', 'num_input_tokens_seen': 494335832, 'train_runtime': '1.074e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152250/225000 [298:24:06<120:23:33,  5.96s/it]
 68%|██████▊   | 152251/225000 [298:24:10<109:10:25,  5.40s/it]
 68%|██████▊   | 152252/225000 [298:24:15<108:27:39,  5.37s/it]
 68%|██████▊   | 152253/225000 [298:24:24<129:45:32,  6.42s/it]
 68%|██████▊   | 152254/225000 [298:24:28<112:22:18,  5.56s/it]
 68%|██████▊   | 152255/225000 [298:24:35<124:44:31,  6.17s/it]
 68%|██████▊   | 152256/225000 [298:24:39<111:49:45,  5.53s/it]
 68%|██████▊   | 152257/225000 [298:24:44<104:31:36,  5.17s/it]
 68%|██████▊   | 152258/225000 [298:24:53<130:43:32,  6.47s/it]
 68%|██████▊   | 152259/225000 [298:24:58<119:04:14,  5.89s/it]
 68%|██████▊   | 152260/225000 [298:25:02<111:04:13,  5.50s/it]
 68%|██████▊   | 152261/225000 [298:25:07<106:28:18,  5.27s/it]
 68%|██████▊   | 152262/225000 [298:25:12<103:07:46,  5.10s/it]
 68%|██████▊   | 152263/225000 [298:25:22<134:17:52,  6.65s/it]
 68%|██████▊   | 152264/225000 [298:25:28<130:24:53,  6.45s/it]
 68%|██████▊   | 152265/225000 [298:25:34<130:53:10,  6.48s/it]
 68%|██████▊   | 152266/225000 [298:25:38<115:59:39,  5.74s/it]
 68%|██████▊   | 152267/225000 [298:25:44<117:30:38,  5.82s/it]
 68%|██████▊   | 152268/225000 [298:25:49<106:36:13,  5.28s/it]
 68%|██████▊   | 152269/225000 [298:25:53<99:38:56,  4.93s/it] 
 68%|██████▊   | 152270/225000 [298:25:57<95:46:37,  4.74s/it]
 68%|██████▊   | 152271/225000 [298:26:09<137:38:57,  6.81s/it]
 68%|██████▊   | 152272/225000 [298:26:12<118:17:19,  5.86s/it]
 68%|██████▊   | 152273/225000 [298:26:21<134:44:37,  6.67s/it]
 68%|██████▊   | 152274/225000 [298:26:26<124:06:48,  6.14s/it]
 68%|██████▊   | 152275/225000 [298:26:31<118:05:17,  5.85s/it]
 68%|██████▊   | 152276/225000 [298:26:34<103:13:49,  5.11s/it]
 68%|██████▊   | 152277/225000 [298:26:43<127:50:06,  6.33s/it]
 68%|██████▊   | 152278/225000 [298:26:47<110:15:55,  5.46s/it]
 68%|██████▊   | 152279/225000 [298:26:54<123:37:48,  6.12s/it]
 68%|██████▊   | 152280/225000 [298:26:59<113:40:14,  5.63s/it]
 68%|██████▊   | 152281/225000 [298:27:04<108:11:10,  5.36s/it]
 68%|██████▊   | 152282/225000 [298:27:08<104:26:57,  5.17s/it]
 68%|██████▊   | 152283/225000 [298:27:13<102:28:42,  5.07s/it]
 68%|██████▊   | 152284/225000 [298:27:18<100:29:46,  4.98s/it]
 68%|██████▊   | 152285/225000 [298:27:23<99:14:18,  4.91s/it] 
 68%|██████▊   | 152286/225000 [298:27:27<97:18:33,  4.82s/it]
 68%|██████▊   | 152287/225000 [298:27:32<96:11:33,  4.76s/it]
 68%|██████▊   | 152288/225000 [298:27:37<98:14:53,  4.86s/it]
 68%|██████▊   | 152289/225000 [298:27:46<119:48:06,  5.93s/it]
 68%|██████▊   | 152290/225000 [298:27:55<138:38:43,  6.86s/it]
 68%|██████▊   | 152291/225000 [298:27:59<121:05:07,  6.00s/it]
 68%|██████▊   | 152292/225000 [298:28:03<108:59:58,  5.40s/it]
 68%|██████▊   | 152293/225000 [298:28:13<140:42:12,  6.97s/it]
 68%|██████▊   | 152294/225000 [298:28:17<121:40:47,  6.02s/it]
 68%|██████▊   | 152295/225000 [298:28:21<108:06:28,  5.35s/it]
 68%|██████▊   | 152296/225000 [298:28:28<120:50:17,  5.98s/it]
 68%|██████▊   | 152297/225000 [298:28:36<132:49:03,  6.58s/it]
 68%|██████▊   | 152298/225000 [298:28:40<113:48:24,  5.64s/it]
 68%|██████▊   | 152299/225000 [298:28:47<122:17:38,  6.06s/it]
 68%|██████▊   | 152300/225000 [298:28:51<110:55:37,  5.49s/it]
                                                               
{'loss': '0.7852', 'grad_norm': '1.138', 'learning_rate': '1.285e-05', 'epoch': '0.6769', 'num_input_tokens_seen': 494466080, 'train_runtime': '1.075e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152300/225000 [298:28:51<110:55:37,  5.49s/it]
 68%|██████▊   | 152301/225000 [298:28:55<100:00:11,  4.95s/it]
 68%|██████▊   | 152302/225000 [298:29:04<125:45:09,  6.23s/it]
 68%|██████▊   | 152303/225000 [298:29:10<123:32:11,  6.12s/it]
 68%|██████▊   | 152304/225000 [298:29:13<107:53:29,  5.34s/it]
 68%|██████▊   | 152305/225000 [298:29:22<126:23:22,  6.26s/it]
 68%|██████▊   | 152306/225000 [298:29:25<109:22:22,  5.42s/it]
 68%|██████▊   | 152307/225000 [298:29:34<130:43:48,  6.47s/it]
 68%|██████▊   | 152308/225000 [298:29:38<116:04:46,  5.75s/it]
 68%|██████▊   | 152309/225000 [298:29:46<132:32:46,  6.56s/it]
 68%|██████▊   | 152310/225000 [298:29:51<119:18:35,  5.91s/it]
 68%|██████▊   | 152311/225000 [298:29:55<110:47:12,  5.49s/it]
 68%|██████▊   | 152312/225000 [298:30:00<105:48:34,  5.24s/it]
 68%|██████▊   | 152313/225000 [298:30:05<102:24:35,  5.07s/it]
 68%|██████▊   | 152314/225000 [298:30:09<99:47:10,  4.94s/it] 
 68%|██████▊   | 152315/225000 [298:30:14<98:09:14,  4.86s/it]
 68%|██████▊   | 152316/225000 [298:30:25<136:36:42,  6.77s/it]
 68%|██████▊   | 152317/225000 [298:30:34<151:02:38,  7.48s/it]
 68%|██████▊   | 152318/225000 [298:30:38<125:31:36,  6.22s/it]
 68%|██████▊   | 152319/225000 [298:30:45<134:43:21,  6.67s/it]
 68%|██████▊   | 152320/225000 [298:30:49<115:21:31,  5.71s/it]
 68%|██████▊   | 152321/225000 [298:30:57<128:33:56,  6.37s/it]
 68%|██████▊   | 152322/225000 [298:31:00<111:04:23,  5.50s/it]
 68%|██████▊   | 152323/225000 [298:31:05<107:29:29,  5.32s/it]
 68%|██████▊   | 152324/225000 [298:31:10<106:45:19,  5.29s/it]
 68%|██████▊   | 152325/225000 [298:31:15<105:25:06,  5.22s/it]
 68%|██████▊   | 152326/225000 [298:31:20<104:29:39,  5.18s/it]
 68%|██████▊   | 152327/225000 [298:31:25<103:32:13,  5.13s/it]
 68%|██████▊   | 152328/225000 [298:31:31<105:45:23,  5.24s/it]
 68%|██████▊   | 152329/225000 [298:31:36<104:06:42,  5.16s/it]
 68%|██████▊   | 152330/225000 [298:31:41<103:26:09,  5.12s/it]
 68%|██████▊   | 152331/225000 [298:31:46<103:55:56,  5.15s/it]
 68%|██████▊   | 152332/225000 [298:31:51<104:23:07,  5.17s/it]
 68%|██████▊   | 152333/225000 [298:31:57<104:43:45,  5.19s/it]
 68%|██████▊   | 152334/225000 [298:32:02<104:32:08,  5.18s/it]
 68%|██████▊   | 152335/225000 [298:32:14<145:34:27,  7.21s/it]
 68%|██████▊   | 152336/225000 [298:32:18<130:26:57,  6.46s/it]
 68%|██████▊   | 152337/225000 [298:32:23<116:03:34,  5.75s/it]
 68%|██████▊   | 152338/225000 [298:32:27<107:33:07,  5.33s/it]
 68%|██████▊   | 152339/225000 [298:32:32<106:11:06,  5.26s/it]
 68%|██████▊   | 152340/225000 [298:32:42<134:42:18,  6.67s/it]
 68%|██████▊   | 152341/225000 [298:32:47<123:36:35,  6.12s/it]
 68%|██████▊   | 152342/225000 [298:32:52<115:34:42,  5.73s/it]
 68%|██████▊   | 152343/225000 [298:32:57<110:41:07,  5.48s/it]
 68%|██████▊   | 152344/225000 [298:33:01<106:49:02,  5.29s/it]
 68%|██████▊   | 152345/225000 [298:33:06<104:42:39,  5.19s/it]
 68%|██████▊   | 152346/225000 [298:33:11<103:28:22,  5.13s/it]
 68%|██████▊   | 152347/225000 [298:33:16<103:28:50,  5.13s/it]
 68%|██████▊   | 152348/225000 [298:33:21<100:14:25,  4.97s/it]
 68%|██████▊   | 152349/225000 [298:33:27<104:35:23,  5.18s/it]
 68%|██████▊   | 152350/225000 [298:33:39<148:39:27,  7.37s/it]
                                                               
{'loss': '0.7813', 'grad_norm': '1.084', 'learning_rate': '1.283e-05', 'epoch': '0.6771', 'num_input_tokens_seen': 494597048, 'train_runtime': '1.075e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152350/225000 [298:33:39<148:39:27,  7.37s/it]
 68%|██████▊   | 152351/225000 [298:33:45<138:59:19,  6.89s/it]
 68%|██████▊   | 152352/225000 [298:33:53<145:51:19,  7.23s/it]
 68%|██████▊   | 152353/225000 [298:33:57<123:46:09,  6.13s/it]
 68%|██████▊   | 152354/225000 [298:34:00<106:35:29,  5.28s/it]
 68%|██████▊   | 152355/225000 [298:34:04<97:02:43,  4.81s/it] 
 68%|██████▊   | 152356/225000 [298:34:14<130:34:57,  6.47s/it]
 68%|██████▊   | 152357/225000 [298:34:18<116:02:49,  5.75s/it]
 68%|██████▊   | 152358/225000 [298:34:22<106:53:28,  5.30s/it]
 68%|██████▊   | 152359/225000 [298:34:29<112:55:55,  5.60s/it]
 68%|██████▊   | 152360/225000 [298:34:37<128:35:33,  6.37s/it]
 68%|██████▊   | 152361/225000 [298:34:42<124:39:51,  6.18s/it]
 68%|██████▊   | 152362/225000 [298:34:49<125:37:17,  6.23s/it]
 68%|██████▊   | 152363/225000 [298:34:52<107:58:14,  5.35s/it]
 68%|██████▊   | 152364/225000 [298:35:01<129:51:39,  6.44s/it]
 68%|██████▊   | 152365/225000 [298:35:06<121:15:29,  6.01s/it]
 68%|██████▊   | 152366/225000 [298:35:11<116:49:38,  5.79s/it]
 68%|██████▊   | 152367/225000 [298:35:16<110:20:47,  5.47s/it]
 68%|██████▊   | 152368/225000 [298:35:21<108:29:59,  5.38s/it]
 68%|██████▊   | 152369/225000 [298:35:27<108:54:11,  5.40s/it]
 68%|██████▊   | 152370/225000 [298:35:32<107:06:12,  5.31s/it]
 68%|██████▊   | 152371/225000 [298:35:37<104:50:52,  5.20s/it]
 68%|██████▊   | 152372/225000 [298:35:42<104:07:41,  5.16s/it]
 68%|██████▊   | 152373/225000 [298:35:47<106:24:33,  5.27s/it]
 68%|██████▊   | 152374/225000 [298:35:52<105:43:12,  5.24s/it]
 68%|██████▊   | 152375/225000 [298:36:02<133:03:27,  6.60s/it]
 68%|██████▊   | 152376/225000 [298:36:10<137:29:07,  6.82s/it]
 68%|██████▊   | 152377/225000 [298:36:13<117:00:57,  5.80s/it]
 68%|██████▊   | 152378/225000 [298:36:20<125:58:26,  6.24s/it]
 68%|██████▊   | 152379/225000 [298:36:24<110:20:08,  5.47s/it]
 68%|██████▊   | 152380/225000 [298:36:29<108:44:11,  5.39s/it]
 68%|██████▊   | 152381/225000 [298:36:34<106:43:01,  5.29s/it]
 68%|██████▊   | 152382/225000 [298:36:40<106:50:00,  5.30s/it]
 68%|██████▊   | 152383/225000 [298:36:45<107:23:53,  5.32s/it]
 68%|██████▊   | 152384/225000 [298:36:51<109:21:54,  5.42s/it]
 68%|██████▊   | 152385/225000 [298:36:56<107:59:51,  5.35s/it]
 68%|██████▊   | 152386/225000 [298:37:02<110:28:42,  5.48s/it]
 68%|██████▊   | 152387/225000 [298:37:13<145:10:54,  7.20s/it]
 68%|██████▊   | 152388/225000 [298:37:17<124:47:45,  6.19s/it]
 68%|██████▊   | 152389/225000 [298:37:21<116:04:18,  5.75s/it]
 68%|██████▊   | 152390/225000 [298:37:27<112:46:39,  5.59s/it]
 68%|██████▊   | 152391/225000 [298:37:32<110:34:19,  5.48s/it]
 68%|██████▊   | 152392/225000 [298:37:37<110:23:22,  5.47s/it]
 68%|██████▊   | 152393/225000 [298:37:42<106:57:07,  5.30s/it]
 68%|██████▊   | 152394/225000 [298:37:47<104:48:14,  5.20s/it]
 68%|██████▊   | 152395/225000 [298:37:52<102:21:40,  5.08s/it]
 68%|██████▊   | 152396/225000 [298:37:57<103:22:19,  5.13s/it]
 68%|██████▊   | 152397/225000 [298:38:02<104:43:51,  5.19s/it]
 68%|██████▊   | 152398/225000 [298:38:08<103:58:16,  5.16s/it]
 68%|██████▊   | 152399/225000 [298:38:13<106:32:23,  5.28s/it]
 68%|██████▊   | 152400/225000 [298:38:19<107:33:13,  5.33s/it]
                                                               
{'loss': '0.7812', 'grad_norm': '1.091', 'learning_rate': '1.282e-05', 'epoch': '0.6773', 'num_input_tokens_seen': 494726216, 'train_runtime': '1.075e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152400/225000 [298:38:19<107:33:13,  5.33s/it]
 68%|██████▊   | 152401/225000 [298:38:24<107:48:37,  5.35s/it]
 68%|██████▊   | 152402/225000 [298:38:29<106:59:58,  5.31s/it]
 68%|██████▊   | 152403/225000 [298:38:34<107:20:17,  5.32s/it]
 68%|██████▊   | 152404/225000 [298:38:39<105:03:23,  5.21s/it]
 68%|██████▊   | 152405/225000 [298:38:45<110:08:04,  5.46s/it]
 68%|██████▊   | 152406/225000 [298:38:56<142:12:22,  7.05s/it]
 68%|██████▊   | 152407/225000 [298:39:01<125:40:13,  6.23s/it]
 68%|██████▊   | 152408/225000 [298:39:05<113:31:57,  5.63s/it]
 68%|██████▊   | 152409/225000 [298:39:09<106:51:50,  5.30s/it]
 68%|██████▊   | 152410/225000 [298:39:21<142:29:40,  7.07s/it]
 68%|██████▊   | 152411/225000 [298:39:25<127:20:21,  6.32s/it]
 68%|██████▊   | 152412/225000 [298:39:30<116:10:42,  5.76s/it]
 68%|██████▊   | 152413/225000 [298:39:34<106:40:14,  5.29s/it]
 68%|██████▊   | 152414/225000 [298:39:39<103:49:26,  5.15s/it]
 68%|██████▊   | 152415/225000 [298:39:49<133:53:38,  6.64s/it]
 68%|██████▊   | 152416/225000 [298:39:54<124:44:58,  6.19s/it]
 68%|██████▊   | 152417/225000 [298:39:58<115:33:56,  5.73s/it]
 68%|██████▊   | 152418/225000 [298:40:03<108:56:38,  5.40s/it]
 68%|██████▊   | 152419/225000 [298:40:08<107:13:20,  5.32s/it]
 68%|██████▊   | 152420/225000 [298:40:14<107:43:40,  5.34s/it]
 68%|██████▊   | 152421/225000 [298:40:19<106:23:12,  5.28s/it]
 68%|██████▊   | 152422/225000 [298:40:24<106:58:55,  5.31s/it]
 68%|██████▊   | 152423/225000 [298:40:29<106:06:17,  5.26s/it]
 68%|██████▊   | 152424/225000 [298:40:34<102:15:49,  5.07s/it]
 68%|██████▊   | 152425/225000 [298:40:39<101:48:30,  5.05s/it]
 68%|██████▊   | 152426/225000 [298:40:44<102:37:22,  5.09s/it]
 68%|██████▊   | 152427/225000 [298:40:49<104:02:12,  5.16s/it]
 68%|██████▊   | 152428/225000 [298:41:01<140:50:28,  6.99s/it]
 68%|██████▊   | 152429/225000 [298:41:10<154:55:42,  7.69s/it]
 68%|██████▊   | 152430/225000 [298:41:13<128:17:38,  6.36s/it]
 68%|██████▊   | 152431/225000 [298:41:18<119:14:19,  5.92s/it]
 68%|██████▊   | 152432/225000 [298:41:27<139:50:34,  6.94s/it]
 68%|██████▊   | 152433/225000 [298:41:32<122:50:07,  6.09s/it]
 68%|██████▊   | 152434/225000 [298:41:36<110:58:17,  5.51s/it]
 68%|██████▊   | 152435/225000 [298:41:43<121:11:57,  6.01s/it]
 68%|██████▊   | 152436/225000 [298:41:50<128:56:37,  6.40s/it]
 68%|██████▊   | 152437/225000 [298:41:54<115:01:44,  5.71s/it]
 68%|██████▊   | 152438/225000 [298:41:58<104:59:52,  5.21s/it]
 68%|██████▊   | 152439/225000 [298:42:04<107:06:08,  5.31s/it]
 68%|██████▊   | 152440/225000 [298:42:10<108:49:12,  5.40s/it]
 68%|██████▊   | 152441/225000 [298:42:15<110:13:45,  5.47s/it]
 68%|██████▊   | 152442/225000 [298:42:20<108:16:57,  5.37s/it]
 68%|██████▊   | 152443/225000 [298:42:30<131:55:08,  6.55s/it]
 68%|██████▊   | 152444/225000 [298:42:36<133:55:43,  6.65s/it]
 68%|██████▊   | 152445/225000 [298:42:41<120:31:51,  5.98s/it]
 68%|██████▊   | 152446/225000 [298:42:45<111:27:26,  5.53s/it]
 68%|██████▊   | 152447/225000 [298:42:50<104:20:51,  5.18s/it]
 68%|██████▊   | 152448/225000 [298:42:59<130:38:12,  6.48s/it]
 68%|██████▊   | 152449/225000 [298:43:05<125:48:26,  6.24s/it]
 68%|██████▊   | 152450/225000 [298:43:12<128:40:56,  6.39s/it]
                                                               
{'loss': '0.7741', 'grad_norm': '1.08', 'learning_rate': '1.28e-05', 'epoch': '0.6776', 'num_input_tokens_seen': 494855912, 'train_runtime': '1.075e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152450/225000 [298:43:12<128:40:56,  6.39s/it]
 68%|██████▊   | 152451/225000 [298:43:15<111:24:39,  5.53s/it]
 68%|██████▊   | 152452/225000 [298:43:25<135:47:31,  6.74s/it]
 68%|██████▊   | 152453/225000 [298:43:29<118:12:51,  5.87s/it]
 68%|██████▊   | 152454/225000 [298:43:33<107:12:27,  5.32s/it]
 68%|██████▊   | 152455/225000 [298:43:41<124:09:44,  6.16s/it]
 68%|██████▊   | 152456/225000 [298:43:46<117:23:27,  5.83s/it]
 68%|██████▊   | 152457/225000 [298:43:49<104:31:52,  5.19s/it]
 68%|██████▊   | 152458/225000 [298:43:54<101:29:28,  5.04s/it]
 68%|██████▊   | 152459/225000 [298:44:04<130:08:15,  6.46s/it]
 68%|██████▊   | 152460/225000 [298:44:07<110:05:16,  5.46s/it]
 68%|██████▊   | 152461/225000 [298:44:12<105:28:34,  5.23s/it]
 68%|██████▊   | 152462/225000 [298:44:16<100:56:38,  5.01s/it]
 68%|██████▊   | 152463/225000 [298:44:21<97:47:08,  4.85s/it] 
 68%|██████▊   | 152464/225000 [298:44:26<98:00:12,  4.86s/it]
 68%|██████▊   | 152465/225000 [298:44:30<96:43:22,  4.80s/it]
 68%|██████▊   | 152466/225000 [298:44:40<126:42:23,  6.29s/it]
 68%|██████▊   | 152467/225000 [298:44:48<136:50:14,  6.79s/it]
 68%|██████▊   | 152468/225000 [298:44:51<115:56:38,  5.75s/it]
 68%|██████▊   | 152469/225000 [298:45:02<144:30:01,  7.17s/it]
 68%|██████▊   | 152470/225000 [298:45:05<120:34:18,  5.98s/it]
 68%|██████▊   | 152471/225000 [298:45:09<107:00:14,  5.31s/it]
 68%|██████▊   | 152472/225000 [298:45:13<101:32:47,  5.04s/it]
 68%|██████▊   | 152473/225000 [298:45:21<116:06:38,  5.76s/it]
 68%|██████▊   | 152474/225000 [298:45:28<126:10:26,  6.26s/it]
 68%|██████▊   | 152475/225000 [298:45:32<112:36:29,  5.59s/it]
 68%|██████▊   | 152476/225000 [298:45:40<126:18:57,  6.27s/it]
 68%|██████▊   | 152477/225000 [298:45:47<129:29:01,  6.43s/it]
 68%|██████▊   | 152478/225000 [298:45:50<112:43:55,  5.60s/it]
 68%|██████▊   | 152479/225000 [298:45:54<100:41:18,  5.00s/it]
 68%|██████▊   | 152480/225000 [298:45:58<93:08:30,  4.62s/it] 
 68%|██████▊   | 152481/225000 [298:46:10<138:41:02,  6.88s/it]
 68%|██████▊   | 152482/225000 [298:46:14<119:49:26,  5.95s/it]
 68%|██████▊   | 152483/225000 [298:46:17<104:45:58,  5.20s/it]
 68%|██████▊   | 152484/225000 [298:46:22<101:35:20,  5.04s/it]
 68%|██████▊   | 152485/225000 [298:46:32<133:24:39,  6.62s/it]
 68%|██████▊   | 152486/225000 [298:46:36<113:54:44,  5.66s/it]
 68%|██████▊   | 152487/225000 [298:46:41<112:03:12,  5.56s/it]
 68%|██████▊   | 152488/225000 [298:46:46<109:30:45,  5.44s/it]
 68%|██████▊   | 152489/225000 [298:46:51<108:48:17,  5.40s/it]
 68%|██████▊   | 152490/225000 [298:46:56<105:55:37,  5.26s/it]
 68%|██████▊   | 152491/225000 [298:47:02<107:22:32,  5.33s/it]
 68%|██████▊   | 152492/225000 [298:47:07<106:01:34,  5.26s/it]
 68%|██████▊   | 152493/225000 [298:47:12<107:21:17,  5.33s/it]
 68%|██████▊   | 152494/225000 [298:47:18<107:13:40,  5.32s/it]
 68%|██████▊   | 152495/225000 [298:47:23<108:33:15,  5.39s/it]
 68%|██████▊   | 152496/225000 [298:47:29<109:26:26,  5.43s/it]
 68%|██████▊   | 152497/225000 [298:47:34<108:19:24,  5.38s/it]
 68%|██████▊   | 152498/225000 [298:47:39<103:48:24,  5.15s/it]
 68%|██████▊   | 152499/225000 [298:47:44<105:55:00,  5.26s/it]
 68%|██████▊   | 152500/225000 [298:47:50<107:28:47,  5.34s/it]
                                                               
{'loss': '0.7812', 'grad_norm': '1.209', 'learning_rate': '1.279e-05', 'epoch': '0.6778', 'num_input_tokens_seen': 494987296, 'train_runtime': '1.076e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152500/225000 [298:47:50<107:28:47,  5.34s/it]
 68%|██████▊   | 152501/225000 [298:47:55<106:35:32,  5.29s/it]
 68%|██████▊   | 152502/225000 [298:48:00<106:32:54,  5.29s/it]
 68%|██████▊   | 152503/225000 [298:48:05<106:30:25,  5.29s/it]
 68%|██████▊   | 152504/225000 [298:48:11<105:29:26,  5.24s/it]
 68%|██████▊   | 152505/225000 [298:48:16<105:38:51,  5.25s/it]
 68%|██████▊   | 152506/225000 [298:48:22<108:46:04,  5.40s/it]
 68%|██████▊   | 152507/225000 [298:48:34<153:29:31,  7.62s/it]
 68%|██████▊   | 152508/225000 [298:48:40<143:30:08,  7.13s/it]
 68%|██████▊   | 152509/225000 [298:48:44<121:59:58,  6.06s/it]
 68%|██████▊   | 152510/225000 [298:48:51<126:13:19,  6.27s/it]
 68%|██████▊   | 152511/225000 [298:48:57<129:00:20,  6.41s/it]
 68%|██████▊   | 152512/225000 [298:49:02<121:04:38,  6.01s/it]
 68%|██████▊   | 152513/225000 [298:49:09<123:22:25,  6.13s/it]
 68%|██████▊   | 152514/225000 [298:49:12<108:16:33,  5.38s/it]
 68%|██████▊   | 152515/225000 [298:49:17<101:08:08,  5.02s/it]
 68%|██████▊   | 152516/225000 [298:49:27<133:59:41,  6.66s/it]
 68%|██████▊   | 152517/225000 [298:49:31<117:24:58,  5.83s/it]
 68%|██████▊   | 152518/225000 [298:49:35<106:10:16,  5.27s/it]
 68%|██████▊   | 152519/225000 [298:49:40<103:14:50,  5.13s/it]
 68%|██████▊   | 152520/225000 [298:49:50<131:37:05,  6.54s/it]
 68%|██████▊   | 152521/225000 [298:49:54<117:19:54,  5.83s/it]
 68%|██████▊   | 152522/225000 [298:49:58<104:59:27,  5.21s/it]
 68%|██████▊   | 152523/225000 [298:50:02<102:28:05,  5.09s/it]
 68%|██████▊   | 152524/225000 [298:50:07<101:03:13,  5.02s/it]
 68%|██████▊   | 152525/225000 [298:50:12<100:50:29,  5.01s/it]
 68%|██████▊   | 152526/225000 [298:50:17<99:10:24,  4.93s/it] 
 68%|██████▊   | 152527/225000 [298:50:21<95:55:52,  4.77s/it]
 68%|██████▊   | 152528/225000 [298:50:26<96:33:56,  4.80s/it]
 68%|██████▊   | 152529/225000 [298:50:37<131:01:51,  6.51s/it]
 68%|██████▊   | 152530/225000 [298:50:46<148:49:14,  7.39s/it]
 68%|██████▊   | 152531/225000 [298:50:50<127:10:45,  6.32s/it]
 68%|██████▊   | 152532/225000 [298:50:55<119:15:54,  5.92s/it]
 68%|██████▊   | 152533/225000 [298:51:00<116:23:43,  5.78s/it]
 68%|██████▊   | 152534/225000 [298:51:06<113:46:35,  5.65s/it]
 68%|██████▊   | 152535/225000 [298:51:11<112:09:33,  5.57s/it]
 68%|██████▊   | 152536/225000 [298:51:16<107:25:48,  5.34s/it]
 68%|██████▊   | 152537/225000 [298:51:21<104:53:05,  5.21s/it]
 68%|██████▊   | 152538/225000 [298:51:26<104:16:03,  5.18s/it]
 68%|██████▊   | 152539/225000 [298:51:31<105:36:49,  5.25s/it]
 68%|██████▊   | 152540/225000 [298:51:37<105:50:41,  5.26s/it]
 68%|██████▊   | 152541/225000 [298:51:42<105:45:14,  5.25s/it]
 68%|██████▊   | 152542/225000 [298:51:49<113:43:38,  5.65s/it]
 68%|██████▊   | 152543/225000 [298:52:01<154:25:52,  7.67s/it]
 68%|██████▊   | 152544/225000 [298:52:04<129:30:24,  6.43s/it]
 68%|██████▊   | 152545/225000 [298:52:08<112:50:42,  5.61s/it]
 68%|██████▊   | 152546/225000 [298:52:17<134:17:48,  6.67s/it]
 68%|██████▊   | 152547/225000 [298:52:21<115:50:50,  5.76s/it]
 68%|██████▊   | 152548/225000 [298:52:27<119:40:41,  5.95s/it]
 68%|██████▊   | 152549/225000 [298:52:34<123:20:08,  6.13s/it]
 68%|██████▊   | 152550/225000 [298:52:40<123:19:31,  6.13s/it]
                                                               
{'loss': '0.7814', 'grad_norm': '1.07', 'learning_rate': '1.277e-05', 'epoch': '0.678', 'num_input_tokens_seen': 495116704, 'train_runtime': '1.076e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152550/225000 [298:52:40<123:19:31,  6.13s/it]
 68%|██████▊   | 152551/225000 [298:52:45<116:24:48,  5.78s/it]
 68%|██████▊   | 152552/225000 [298:52:49<105:47:55,  5.26s/it]
 68%|██████▊   | 152553/225000 [298:52:52<94:49:30,  4.71s/it] 
 68%|██████▊   | 152554/225000 [298:52:57<96:07:27,  4.78s/it]
 68%|██████▊   | 152555/225000 [298:53:02<94:42:04,  4.71s/it]
 68%|██████▊   | 152556/225000 [298:53:07<95:39:22,  4.75s/it]
 68%|██████▊   | 152557/225000 [298:53:17<127:09:53,  6.32s/it]
 68%|██████▊   | 152558/225000 [298:53:26<142:48:32,  7.10s/it]
 68%|██████▊   | 152559/225000 [298:53:29<121:01:59,  6.01s/it]
 68%|██████▊   | 152560/225000 [298:53:33<105:52:13,  5.26s/it]
 68%|██████▊   | 152561/225000 [298:53:43<137:17:13,  6.82s/it]
 68%|██████▊   | 152562/225000 [298:53:47<116:44:26,  5.80s/it]
 68%|██████▊   | 152563/225000 [298:53:50<104:54:57,  5.21s/it]
 68%|██████▊   | 152564/225000 [298:53:56<104:31:55,  5.20s/it]
 68%|██████▊   | 152565/225000 [298:54:01<104:28:10,  5.19s/it]
 68%|██████▊   | 152566/225000 [298:54:10<130:11:15,  6.47s/it]
 68%|██████▊   | 152567/225000 [298:54:15<122:24:44,  6.08s/it]
 68%|██████▊   | 152568/225000 [298:54:20<116:29:35,  5.79s/it]
 68%|██████▊   | 152569/225000 [298:54:25<111:22:36,  5.54s/it]
 68%|██████▊   | 152570/225000 [298:54:31<109:43:07,  5.45s/it]
 68%|██████▊   | 152571/225000 [298:54:36<110:07:34,  5.47s/it]
 68%|██████▊   | 152572/225000 [298:54:41<106:10:00,  5.28s/it]
 68%|██████▊   | 152573/225000 [298:54:46<105:05:29,  5.22s/it]
 68%|██████▊   | 152574/225000 [298:54:52<108:14:46,  5.38s/it]
 68%|██████▊   | 152575/225000 [298:54:57<108:00:53,  5.37s/it]
 68%|██████▊   | 152576/225000 [298:55:03<108:41:33,  5.40s/it]
 68%|██████▊   | 152577/225000 [298:55:10<118:00:51,  5.87s/it]
 68%|██████▊   | 152578/225000 [298:55:17<127:53:02,  6.36s/it]
 68%|██████▊   | 152579/225000 [298:55:22<116:58:44,  5.81s/it]
 68%|██████▊   | 152580/225000 [298:55:26<108:57:45,  5.42s/it]
 68%|██████▊   | 152581/225000 [298:55:31<103:42:03,  5.16s/it]
 68%|██████▊   | 152582/225000 [298:55:39<124:02:33,  6.17s/it]
 68%|██████▊   | 152583/225000 [298:55:47<132:04:40,  6.57s/it]
 68%|██████▊   | 152584/225000 [298:55:52<122:51:25,  6.11s/it]
 68%|██████▊   | 152585/225000 [298:55:58<121:33:57,  6.04s/it]
 68%|██████▊   | 152586/225000 [298:56:02<108:44:42,  5.41s/it]
 68%|██████▊   | 152587/225000 [298:56:06<100:40:07,  5.00s/it]
 68%|██████▊   | 152588/225000 [298:56:10<96:32:15,  4.80s/it] 
 68%|██████▊   | 152589/225000 [298:56:14<92:14:31,  4.59s/it]
 68%|██████▊   | 152590/225000 [298:56:25<132:37:27,  6.59s/it]
 68%|██████▊   | 152591/225000 [298:56:31<124:30:01,  6.19s/it]
 68%|██████▊   | 152592/225000 [298:56:37<128:10:12,  6.37s/it]
 68%|██████▊   | 152593/225000 [298:56:42<117:07:48,  5.82s/it]
 68%|██████▊   | 152594/225000 [298:56:47<112:11:21,  5.58s/it]
 68%|██████▊   | 152595/225000 [298:56:51<104:32:27,  5.20s/it]
 68%|██████▊   | 152596/225000 [298:56:56<102:32:10,  5.10s/it]
 68%|██████▊   | 152597/225000 [298:57:01<98:50:40,  4.91s/it] 
 68%|██████▊   | 152598/225000 [298:57:06<99:34:49,  4.95s/it]
 68%|██████▊   | 152599/225000 [298:57:17<140:41:49,  7.00s/it]
 68%|██████▊   | 152600/225000 [298:57:21<117:54:55,  5.86s/it]
                                                               
{'loss': '0.7712', 'grad_norm': '1.026', 'learning_rate': '1.276e-05', 'epoch': '0.6782', 'num_input_tokens_seen': 495246520, 'train_runtime': '1.076e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152600/225000 [298:57:21<117:54:55,  5.86s/it]
 68%|██████▊   | 152601/225000 [298:57:27<123:51:38,  6.16s/it]
 68%|██████▊   | 152602/225000 [298:57:32<114:20:37,  5.69s/it]
 68%|██████▊   | 152603/225000 [298:57:36<106:27:41,  5.29s/it]
 68%|██████▊   | 152604/225000 [298:57:41<100:51:48,  5.02s/it]
 68%|██████▊   | 152605/225000 [298:57:52<136:37:28,  6.79s/it]
 68%|██████▊   | 152606/225000 [298:57:57<125:28:50,  6.24s/it]
 68%|██████▊   | 152607/225000 [298:58:01<116:21:32,  5.79s/it]
 68%|██████▊   | 152608/225000 [298:58:06<111:43:56,  5.56s/it]
 68%|██████▊   | 152609/225000 [298:58:11<107:28:11,  5.34s/it]
 68%|██████▊   | 152610/225000 [298:58:16<104:14:25,  5.18s/it]
 68%|██████▊   | 152611/225000 [298:58:21<102:55:37,  5.12s/it]
 68%|██████▊   | 152612/225000 [298:58:26<102:39:56,  5.11s/it]
 68%|██████▊   | 152613/225000 [298:58:31<102:21:08,  5.09s/it]
 68%|██████▊   | 152614/225000 [298:58:36<101:39:29,  5.06s/it]
 68%|██████▊   | 152615/225000 [298:58:46<132:21:31,  6.58s/it]
 68%|██████▊   | 152616/225000 [298:58:55<142:57:12,  7.11s/it]
 68%|██████▊   | 152617/225000 [298:58:58<119:34:22,  5.95s/it]
 68%|██████▊   | 152618/225000 [298:59:05<127:58:16,  6.36s/it]
 68%|██████▊   | 152619/225000 [298:59:10<121:35:00,  6.05s/it]
 68%|██████▊   | 152620/225000 [298:59:16<116:24:12,  5.79s/it]
 68%|██████▊   | 152621/225000 [298:59:20<107:53:23,  5.37s/it]
 68%|██████▊   | 152622/225000 [298:59:24<102:08:07,  5.08s/it]
 68%|██████▊   | 152623/225000 [298:59:30<102:34:57,  5.10s/it]
 68%|██████▊   | 152624/225000 [298:59:35<103:10:58,  5.13s/it]
 68%|██████▊   | 152625/225000 [298:59:40<102:52:08,  5.12s/it]
 68%|██████▊   | 152626/225000 [298:59:45<102:58:14,  5.12s/it]
 68%|██████▊   | 152627/225000 [298:59:50<102:51:01,  5.12s/it]
 68%|██████▊   | 152628/225000 [298:59:55<103:27:25,  5.15s/it]
 68%|██████▊   | 152629/225000 [299:00:00<101:12:09,  5.03s/it]
 68%|██████▊   | 152630/225000 [299:00:05<102:33:24,  5.10s/it]
 68%|██████▊   | 152631/225000 [299:00:11<102:54:28,  5.12s/it]
 68%|██████▊   | 152632/225000 [299:00:16<102:51:52,  5.12s/it]
 68%|██████▊   | 152633/225000 [299:00:24<124:03:59,  6.17s/it]
 68%|██████▊   | 152634/225000 [299:00:32<136:14:59,  6.78s/it]
 68%|██████▊   | 152635/225000 [299:00:38<128:30:11,  6.39s/it]
 68%|██████▊   | 152636/225000 [299:00:43<118:22:59,  5.89s/it]
 68%|██████▊   | 152637/225000 [299:00:48<114:34:37,  5.70s/it]
 68%|██████▊   | 152638/225000 [299:00:54<114:00:32,  5.67s/it]
 68%|██████▊   | 152639/225000 [299:00:59<112:17:13,  5.59s/it]
 68%|██████▊   | 152640/225000 [299:01:04<111:37:03,  5.55s/it]
 68%|██████▊   | 152641/225000 [299:01:10<110:57:20,  5.52s/it]
 68%|██████▊   | 152642/225000 [299:01:16<112:50:58,  5.61s/it]
 68%|██████▊   | 152643/225000 [299:01:21<110:14:46,  5.49s/it]
 68%|██████▊   | 152644/225000 [299:01:27<115:01:34,  5.72s/it]
 68%|██████▊   | 152645/225000 [299:01:40<156:44:03,  7.80s/it]
 68%|██████▊   | 152646/225000 [299:01:45<139:38:15,  6.95s/it]
 68%|██████▊   | 152647/225000 [299:01:49<121:34:05,  6.05s/it]
 68%|██████▊   | 152648/225000 [299:01:53<110:10:25,  5.48s/it]
 68%|██████▊   | 152649/225000 [299:02:01<123:40:03,  6.15s/it]
 68%|██████▊   | 152650/225000 [299:02:07<125:35:40,  6.25s/it]
                                                               
{'loss': '0.791', 'grad_norm': '1.024', 'learning_rate': '1.274e-05', 'epoch': '0.6784', 'num_input_tokens_seen': 495377112, 'train_runtime': '1.077e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152650/225000 [299:02:07<125:35:40,  6.25s/it]
 68%|██████▊   | 152651/225000 [299:02:11<112:37:55,  5.60s/it]
 68%|██████▊   | 152652/225000 [299:02:15<103:36:57,  5.16s/it]
 68%|██████▊   | 152653/225000 [299:02:23<121:44:58,  6.06s/it]
 68%|██████▊   | 152654/225000 [299:02:31<130:11:28,  6.48s/it]
 68%|██████▊   | 152655/225000 [299:02:35<113:27:40,  5.65s/it]
 68%|██████▊   | 152656/225000 [299:02:39<103:14:21,  5.14s/it]
 68%|██████▊   | 152657/225000 [299:02:49<136:31:15,  6.79s/it]
 68%|██████▊   | 152658/225000 [299:02:53<119:29:57,  5.95s/it]
 68%|██████▊   | 152659/225000 [299:02:57<107:23:55,  5.34s/it]
 68%|██████▊   | 152660/225000 [299:03:01<100:20:18,  4.99s/it]
 68%|██████▊   | 152661/225000 [299:03:06<96:26:53,  4.80s/it] 
 68%|██████▊   | 152662/225000 [299:03:10<94:36:51,  4.71s/it]
 68%|██████▊   | 152663/225000 [299:03:21<134:18:31,  6.68s/it]
 68%|██████▊   | 152664/225000 [299:03:26<120:51:37,  6.01s/it]
 68%|██████▊   | 152665/225000 [299:03:29<106:08:30,  5.28s/it]
 68%|██████▊   | 152666/225000 [299:03:35<104:47:15,  5.22s/it]
 68%|██████▊   | 152667/225000 [299:03:40<109:12:58,  5.44s/it]
 68%|██████▊   | 152668/225000 [299:03:48<121:15:14,  6.03s/it]
 68%|██████▊   | 152669/225000 [299:03:54<123:57:16,  6.17s/it]
 68%|██████▊   | 152670/225000 [299:04:00<119:58:39,  5.97s/it]
 68%|██████▊   | 152671/225000 [299:04:09<136:03:18,  6.77s/it]
 68%|██████▊   | 152672/225000 [299:04:13<122:11:24,  6.08s/it]
 68%|██████▊   | 152673/225000 [299:04:18<112:42:49,  5.61s/it]
 68%|██████▊   | 152674/225000 [299:04:22<106:35:12,  5.31s/it]
 68%|██████▊   | 152675/225000 [299:04:27<102:43:15,  5.11s/it]
 68%|██████▊   | 152676/225000 [299:04:31<98:18:22,  4.89s/it] 
 68%|██████▊   | 152677/225000 [299:04:36<95:49:54,  4.77s/it]
 68%|██████▊   | 152678/225000 [299:04:48<140:28:21,  6.99s/it]
 68%|██████▊   | 152679/225000 [299:04:51<120:03:30,  5.98s/it]
 68%|██████▊   | 152680/225000 [299:04:56<108:38:17,  5.41s/it]
 68%|██████▊   | 152681/225000 [299:05:00<100:47:51,  5.02s/it]
 68%|██████▊   | 152682/225000 [299:05:10<133:55:15,  6.67s/it]
 68%|██████▊   | 152683/225000 [299:05:14<114:17:54,  5.69s/it]
 68%|██████▊   | 152684/225000 [299:05:19<112:36:59,  5.61s/it]
 68%|██████▊   | 152685/225000 [299:05:25<112:29:02,  5.60s/it]
 68%|██████▊   | 152686/225000 [299:05:30<111:39:29,  5.56s/it]
 68%|██████▊   | 152687/225000 [299:05:35<109:43:21,  5.46s/it]
 68%|██████▊   | 152688/225000 [299:05:41<110:29:27,  5.50s/it]
 68%|██████▊   | 152689/225000 [299:05:50<131:47:06,  6.56s/it]
 68%|██████▊   | 152690/225000 [299:05:54<115:15:06,  5.74s/it]
 68%|██████▊   | 152691/225000 [299:06:03<137:05:18,  6.83s/it]
 68%|██████▊   | 152692/225000 [299:06:07<119:40:34,  5.96s/it]
 68%|██████▊   | 152693/225000 [299:06:14<126:35:43,  6.30s/it]
 68%|██████▊   | 152694/225000 [299:06:18<112:48:16,  5.62s/it]
 68%|██████▊   | 152695/225000 [299:06:22<100:07:52,  4.99s/it]
 68%|██████▊   | 152696/225000 [299:06:26<97:21:12,  4.85s/it] 
 68%|██████▊   | 152697/225000 [299:06:31<96:10:35,  4.79s/it]
 68%|██████▊   | 152698/225000 [299:06:36<99:16:06,  4.94s/it]
 68%|██████▊   | 152699/225000 [299:06:41<101:52:56,  5.07s/it]
 68%|██████▊   | 152700/225000 [299:06:47<102:39:49,  5.11s/it]
                                                               
{'loss': '0.7909', 'grad_norm': '1.109', 'learning_rate': '1.272e-05', 'epoch': '0.6787', 'num_input_tokens_seen': 495506920, 'train_runtime': '1.077e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152700/225000 [299:06:47<102:39:49,  5.11s/it]
 68%|██████▊   | 152701/225000 [299:06:52<104:54:33,  5.22s/it]
 68%|██████▊   | 152702/225000 [299:06:57<104:38:30,  5.21s/it]
 68%|██████▊   | 152703/225000 [299:07:02<103:15:36,  5.14s/it]
 68%|██████▊   | 152704/225000 [299:07:08<104:15:05,  5.19s/it]
 68%|██████▊   | 152705/225000 [299:07:13<103:35:32,  5.16s/it]
 68%|██████▊   | 152706/225000 [299:07:25<145:32:23,  7.25s/it]
 68%|██████▊   | 152707/225000 [299:07:34<156:36:56,  7.80s/it]
 68%|██████▊   | 152708/225000 [299:07:38<133:07:05,  6.63s/it]
 68%|██████▊   | 152709/225000 [299:07:41<113:47:50,  5.67s/it]
 68%|██████▊   | 152710/225000 [299:07:50<135:23:22,  6.74s/it]
 68%|██████▊   | 152711/225000 [299:07:54<116:46:37,  5.82s/it]
 68%|██████▊   | 152712/225000 [299:07:58<103:41:17,  5.16s/it]
 68%|██████▊   | 152713/225000 [299:08:06<122:13:53,  6.09s/it]
 68%|██████▊   | 152714/225000 [299:08:11<117:00:27,  5.83s/it]
 68%|██████▊   | 152715/225000 [299:08:16<112:00:37,  5.58s/it]
 68%|██████▊   | 152716/225000 [299:08:20<101:41:57,  5.06s/it]
 68%|██████▊   | 152717/225000 [299:08:25<98:36:58,  4.91s/it] 
 68%|██████▊   | 152718/225000 [299:08:30<99:32:44,  4.96s/it]
 68%|██████▊   | 152719/225000 [299:08:35<101:27:55,  5.05s/it]
 68%|██████▊   | 152720/225000 [299:08:40<100:11:23,  4.99s/it]
 68%|██████▊   | 152721/225000 [299:08:44<97:44:37,  4.87s/it] 
 68%|██████▊   | 152722/225000 [299:08:52<115:02:50,  5.73s/it]
 68%|██████▊   | 152723/225000 [299:09:03<148:37:20,  7.40s/it]
 68%|██████▊   | 152724/225000 [299:09:07<124:10:43,  6.19s/it]
 68%|██████▊   | 152725/225000 [299:09:11<112:53:57,  5.62s/it]
 68%|██████▊   | 152726/225000 [299:09:16<106:24:00,  5.30s/it]
 68%|██████▊   | 152727/225000 [299:09:20<101:18:19,  5.05s/it]
 68%|██████▊   | 152728/225000 [299:09:25<99:08:20,  4.94s/it] 
 68%|██████▊   | 152729/225000 [299:09:29<95:05:00,  4.74s/it]
 68%|██████▊   | 152730/225000 [299:09:34<96:32:37,  4.81s/it]
 68%|██████▊   | 152731/225000 [299:09:39<99:42:29,  4.97s/it]
 68%|██████▊   | 152732/225000 [299:09:44<99:14:28,  4.94s/it]
 68%|██████▊   | 152733/225000 [299:09:57<145:34:05,  7.25s/it]
 68%|██████▊   | 152734/225000 [299:10:04<143:10:16,  7.13s/it]
 68%|██████▊   | 152735/225000 [299:10:11<142:24:12,  7.09s/it]
 68%|██████▊   | 152736/225000 [299:10:14<119:28:06,  5.95s/it]
 68%|██████▊   | 152737/225000 [299:10:21<125:02:31,  6.23s/it]
 68%|██████▊   | 152738/225000 [299:10:24<107:45:26,  5.37s/it]
 68%|██████▊   | 152739/225000 [299:10:32<120:14:34,  5.99s/it]
 68%|██████▊   | 152740/225000 [299:10:35<106:39:50,  5.31s/it]
 68%|██████▊   | 152741/225000 [299:10:43<122:22:15,  6.10s/it]
 68%|██████▊   | 152742/225000 [299:10:48<114:24:03,  5.70s/it]
 68%|██████▊   | 152743/225000 [299:10:53<108:44:36,  5.42s/it]
 68%|██████▊   | 152744/225000 [299:10:58<103:44:44,  5.17s/it]
 68%|██████▊   | 152745/225000 [299:11:02<99:22:47,  4.95s/it] 
 68%|██████▊   | 152746/225000 [299:11:07<102:42:15,  5.12s/it]
 68%|██████▊   | 152747/225000 [299:11:18<137:13:29,  6.84s/it]
 68%|██████▊   | 152748/225000 [299:11:22<119:48:09,  5.97s/it]
 68%|██████▊   | 152749/225000 [299:11:26<107:40:00,  5.36s/it]
 68%|██████▊   | 152750/225000 [299:11:32<107:08:25,  5.34s/it]
                                                               
{'loss': '0.7869', 'grad_norm': '1.08', 'learning_rate': '1.271e-05', 'epoch': '0.6789', 'num_input_tokens_seen': 495637216, 'train_runtime': '1.077e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152750/225000 [299:11:32<107:08:25,  5.34s/it]
 68%|██████▊   | 152751/225000 [299:11:42<136:06:13,  6.78s/it]
 68%|██████▊   | 152752/225000 [299:11:45<113:43:19,  5.67s/it]
 68%|██████▊   | 152753/225000 [299:11:52<122:21:18,  6.10s/it]
 68%|██████▊   | 152754/225000 [299:11:55<107:10:12,  5.34s/it]
 68%|██████▊   | 152755/225000 [299:12:04<128:42:07,  6.41s/it]
 68%|██████▊   | 152756/225000 [299:12:07<109:03:24,  5.43s/it]
 68%|██████▊   | 152757/225000 [299:12:12<101:56:19,  5.08s/it]
 68%|██████▊   | 152758/225000 [299:12:16<99:42:45,  4.97s/it] 
 68%|██████▊   | 152759/225000 [299:12:21<99:06:36,  4.94s/it]
 68%|██████▊   | 152760/225000 [299:12:26<96:33:17,  4.81s/it]
 68%|██████▊   | 152761/225000 [299:12:35<123:41:00,  6.16s/it]
 68%|██████▊   | 152762/225000 [299:12:43<135:24:41,  6.75s/it]
 68%|██████▊   | 152763/225000 [299:12:48<124:45:53,  6.22s/it]
 68%|██████▊   | 152764/225000 [299:12:55<127:10:36,  6.34s/it]
 68%|██████▊   | 152765/225000 [299:12:59<115:26:31,  5.75s/it]
 68%|██████▊   | 152766/225000 [299:13:05<113:06:33,  5.64s/it]
 68%|██████▊   | 152767/225000 [299:13:10<112:28:51,  5.61s/it]
 68%|██████▊   | 152768/225000 [299:13:17<121:57:21,  6.08s/it]
 68%|██████▊   | 152769/225000 [299:13:25<130:00:25,  6.48s/it]
 68%|██████▊   | 152770/225000 [299:13:31<125:46:41,  6.27s/it]
 68%|██████▊   | 152771/225000 [299:13:35<115:19:38,  5.75s/it]
 68%|██████▊   | 152772/225000 [299:13:40<110:00:51,  5.48s/it]
 68%|██████▊   | 152773/225000 [299:13:44<104:22:52,  5.20s/it]
 68%|██████▊   | 152774/225000 [299:13:49<99:21:43,  4.95s/it] 
 68%|██████▊   | 152775/225000 [299:13:53<97:30:18,  4.86s/it]
 68%|██████▊   | 152776/225000 [299:14:05<135:20:28,  6.75s/it]
 68%|██████▊   | 152777/225000 [299:14:11<133:17:22,  6.64s/it]
 68%|██████▊   | 152778/225000 [299:14:17<131:00:39,  6.53s/it]
 68%|██████▊   | 152779/225000 [299:14:23<125:51:59,  6.27s/it]
 68%|██████▊   | 152780/225000 [299:14:28<118:23:37,  5.90s/it]
 68%|██████▊   | 152781/225000 [299:14:33<112:40:21,  5.62s/it]
 68%|██████▊   | 152782/225000 [299:14:38<107:26:45,  5.36s/it]
 68%|██████▊   | 152783/225000 [299:14:42<100:59:39,  5.03s/it]
 68%|██████▊   | 152784/225000 [299:14:47<100:46:13,  5.02s/it]
 68%|██████▊   | 152785/225000 [299:14:52<98:58:29,  4.93s/it] 
 68%|██████▊   | 152786/225000 [299:14:56<95:36:56,  4.77s/it]
 68%|██████▊   | 152787/225000 [299:15:01<96:35:41,  4.82s/it]
 68%|██████▊   | 152788/225000 [299:15:06<94:53:24,  4.73s/it]
 68%|██████▊   | 152789/225000 [299:15:12<102:24:12,  5.11s/it]
 68%|██████▊   | 152790/225000 [299:15:24<149:36:51,  7.46s/it]
 68%|██████▊   | 152791/225000 [299:15:29<134:38:59,  6.71s/it]
 68%|██████▊   | 152792/225000 [299:15:38<146:24:52,  7.30s/it]
 68%|██████▊   | 152793/225000 [299:15:42<126:29:51,  6.31s/it]
 68%|██████▊   | 152794/225000 [299:15:46<110:34:51,  5.51s/it]
 68%|██████▊   | 152795/225000 [299:15:54<125:34:39,  6.26s/it]
 68%|██████▊   | 152796/225000 [299:15:57<109:11:50,  5.44s/it]
 68%|██████▊   | 152797/225000 [299:16:06<126:44:08,  6.32s/it]
 68%|██████▊   | 152798/225000 [299:16:10<117:16:18,  5.85s/it]
 68%|██████▊   | 152799/225000 [299:16:15<109:55:32,  5.48s/it]
 68%|██████▊   | 152800/225000 [299:16:19<102:06:21,  5.09s/it]
                                                               
{'loss': '0.7775', 'grad_norm': '1.315', 'learning_rate': '1.269e-05', 'epoch': '0.6791', 'num_input_tokens_seen': 495767160, 'train_runtime': '1.077e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152800/225000 [299:16:19<102:06:21,  5.09s/it]
 68%|██████▊   | 152801/225000 [299:16:24<98:32:20,  4.91s/it] 
 68%|██████▊   | 152802/225000 [299:16:28<93:28:28,  4.66s/it]
 68%|██████▊   | 152803/225000 [299:16:32<92:05:23,  4.59s/it]
 68%|██████▊   | 152804/225000 [299:16:44<136:38:42,  6.81s/it]
 68%|██████▊   | 152805/225000 [299:16:49<124:26:44,  6.21s/it]
 68%|██████▊   | 152806/225000 [299:16:53<108:42:48,  5.42s/it]
 68%|██████▊   | 152807/225000 [299:16:59<112:49:23,  5.63s/it]
 68%|██████▊   | 152808/225000 [299:17:08<132:51:37,  6.63s/it]
 68%|██████▊   | 152809/225000 [299:17:11<111:27:05,  5.56s/it]
 68%|██████▊   | 152810/225000 [299:17:16<111:33:01,  5.56s/it]
 68%|██████▊   | 152811/225000 [299:17:22<110:21:46,  5.50s/it]
 68%|██████▊   | 152812/225000 [299:17:26<105:48:00,  5.28s/it]
 68%|██████▊   | 152813/225000 [299:17:32<106:25:46,  5.31s/it]
 68%|██████▊   | 152814/225000 [299:17:37<106:14:02,  5.30s/it]
 68%|██████▊   | 152815/225000 [299:17:42<104:13:43,  5.20s/it]
 68%|██████▊   | 152816/225000 [299:17:47<104:45:20,  5.22s/it]
 68%|██████▊   | 152817/225000 [299:17:56<126:10:59,  6.29s/it]
 68%|██████▊   | 152818/225000 [299:18:04<136:32:32,  6.81s/it]
 68%|██████▊   | 152819/225000 [299:18:07<114:45:03,  5.72s/it]
 68%|██████▊   | 152820/225000 [299:18:12<109:07:52,  5.44s/it]
 68%|██████▊   | 152821/225000 [299:18:17<103:07:44,  5.14s/it]
 68%|██████▊   | 152822/225000 [299:18:27<133:48:47,  6.67s/it]
 68%|██████▊   | 152823/225000 [299:18:33<131:34:47,  6.56s/it]
 68%|██████▊   | 152824/225000 [299:18:37<114:03:59,  5.69s/it]
 68%|██████▊   | 152825/225000 [299:18:47<139:24:35,  6.95s/it]
 68%|██████▊   | 152826/225000 [299:18:51<120:51:41,  6.03s/it]
 68%|██████▊   | 152827/225000 [299:18:54<106:57:13,  5.33s/it]
 68%|██████▊   | 152828/225000 [299:18:58<98:57:49,  4.94s/it] 
 68%|██████▊   | 152829/225000 [299:19:03<95:54:46,  4.78s/it]
 68%|██████▊   | 152830/225000 [299:19:08<97:48:11,  4.88s/it]
 68%|██████▊   | 152831/225000 [299:19:13<100:05:55,  4.99s/it]
 68%|██████▊   | 152832/225000 [299:19:24<135:08:06,  6.74s/it]
 68%|██████▊   | 152833/225000 [299:19:30<130:11:39,  6.49s/it]
 68%|██████▊   | 152834/225000 [299:19:33<110:27:36,  5.51s/it]
 68%|██████▊   | 152835/225000 [299:19:39<112:53:09,  5.63s/it]
 68%|██████▊   | 152836/225000 [299:19:45<117:44:15,  5.87s/it]
 68%|██████▊   | 152837/225000 [299:19:50<111:10:34,  5.55s/it]
 68%|██████▊   | 152838/225000 [299:19:55<108:48:27,  5.43s/it]
 68%|██████▊   | 152839/225000 [299:20:00<104:14:26,  5.20s/it]
 68%|██████▊   | 152840/225000 [299:20:05<100:41:07,  5.02s/it]
 68%|██████▊   | 152841/225000 [299:20:09<99:51:45,  4.98s/it] 
 68%|██████▊   | 152842/225000 [299:20:14<100:04:38,  4.99s/it]
 68%|██████▊   | 152843/225000 [299:20:19<97:44:52,  4.88s/it] 
 68%|██████▊   | 152844/225000 [299:20:27<118:01:32,  5.89s/it]
 68%|██████▊   | 152845/225000 [299:20:39<154:00:55,  7.68s/it]
 68%|██████▊   | 152846/225000 [299:20:43<128:03:45,  6.39s/it]
 68%|██████▊   | 152847/225000 [299:20:48<123:17:53,  6.15s/it]
 68%|██████▊   | 152848/225000 [299:20:53<118:23:40,  5.91s/it]
 68%|██████▊   | 152849/225000 [299:20:59<116:18:43,  5.80s/it]
 68%|██████▊   | 152850/225000 [299:21:06<124:55:44,  6.23s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '1.058', 'learning_rate': '1.268e-05', 'epoch': '0.6793', 'num_input_tokens_seen': 495896208, 'train_runtime': '1.078e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152850/225000 [299:21:06<124:55:44,  6.23s/it]
 68%|██████▊   | 152851/225000 [299:21:14<133:16:45,  6.65s/it]
 68%|██████▊   | 152852/225000 [299:21:18<119:47:45,  5.98s/it]
 68%|██████▊   | 152853/225000 [299:21:22<108:49:08,  5.43s/it]
 68%|██████▊   | 152854/225000 [299:21:27<102:21:01,  5.11s/it]
 68%|██████▊   | 152855/225000 [299:21:38<137:52:47,  6.88s/it]
 68%|██████▊   | 152856/225000 [299:21:41<117:14:07,  5.85s/it]
 68%|██████▊   | 152857/225000 [299:21:47<115:12:25,  5.75s/it]
 68%|██████▊   | 152858/225000 [299:21:53<119:48:31,  5.98s/it]
 68%|██████▊   | 152859/225000 [299:21:57<104:49:10,  5.23s/it]
 68%|██████▊   | 152860/225000 [299:22:02<102:16:40,  5.10s/it]
 68%|██████▊   | 152861/225000 [299:22:09<115:58:02,  5.79s/it]
 68%|██████▊   | 152862/225000 [299:22:12<102:07:54,  5.10s/it]
 68%|██████▊   | 152863/225000 [299:22:16<94:33:18,  4.72s/it] 
 68%|██████▊   | 152864/225000 [299:22:27<133:13:50,  6.65s/it]
 68%|██████▊   | 152865/225000 [299:22:31<116:16:16,  5.80s/it]
 68%|██████▊   | 152866/225000 [299:22:36<111:45:06,  5.58s/it]
 68%|██████▊   | 152867/225000 [299:22:41<107:12:53,  5.35s/it]
 68%|██████▊   | 152868/225000 [299:22:46<106:47:12,  5.33s/it]
 68%|██████▊   | 152869/225000 [299:22:52<108:12:44,  5.40s/it]
 68%|██████▊   | 152870/225000 [299:22:57<105:39:48,  5.27s/it]
 68%|██████▊   | 152871/225000 [299:23:02<105:26:23,  5.26s/it]
 68%|██████▊   | 152872/225000 [299:23:07<103:59:59,  5.19s/it]
 68%|██████▊   | 152873/225000 [299:23:13<106:25:06,  5.31s/it]
 68%|██████▊   | 152874/225000 [299:23:18<107:57:00,  5.39s/it]
 68%|██████▊   | 152875/225000 [299:23:24<110:18:02,  5.51s/it]
 68%|██████▊   | 152876/225000 [299:23:31<116:20:49,  5.81s/it]
 68%|██████▊   | 152877/225000 [299:23:36<112:29:50,  5.62s/it]
 68%|██████▊   | 152878/225000 [299:23:41<111:13:27,  5.55s/it]
 68%|██████▊   | 152879/225000 [299:23:47<110:53:38,  5.54s/it]
 68%|██████▊   | 152880/225000 [299:23:56<134:42:24,  6.72s/it]
 68%|██████▊   | 152881/225000 [299:24:03<137:47:04,  6.88s/it]
 68%|██████▊   | 152882/225000 [299:24:10<134:39:53,  6.72s/it]
 68%|██████▊   | 152883/225000 [299:24:15<126:47:34,  6.33s/it]
 68%|██████▊   | 152884/225000 [299:24:20<119:59:30,  5.99s/it]
 68%|██████▊   | 152885/225000 [299:24:26<115:28:01,  5.76s/it]
 68%|██████▊   | 152886/225000 [299:24:31<113:37:56,  5.67s/it]
 68%|██████▊   | 152887/225000 [299:24:37<111:40:09,  5.57s/it]
 68%|██████▊   | 152888/225000 [299:24:42<110:05:56,  5.50s/it]
 68%|██████▊   | 152889/225000 [299:24:47<108:47:24,  5.43s/it]
 68%|██████▊   | 152890/225000 [299:24:52<106:57:59,  5.34s/it]
 68%|██████▊   | 152891/225000 [299:24:58<106:36:35,  5.32s/it]
 68%|██████▊   | 152892/225000 [299:25:03<104:46:08,  5.23s/it]
 68%|██████▊   | 152893/225000 [299:25:08<104:24:21,  5.21s/it]
 68%|██████▊   | 152894/225000 [299:25:21<151:16:41,  7.55s/it]
 68%|██████▊   | 152895/225000 [299:25:25<132:01:44,  6.59s/it]
 68%|██████▊   | 152896/225000 [299:25:29<118:59:17,  5.94s/it]
 68%|██████▊   | 152897/225000 [299:25:34<107:27:16,  5.37s/it]
 68%|██████▊   | 152898/225000 [299:25:38<101:13:31,  5.05s/it]
 68%|██████▊   | 152899/225000 [299:25:47<128:41:00,  6.43s/it]
 68%|██████▊   | 152900/225000 [299:25:53<124:25:45,  6.21s/it]
                                                               
{'loss': '0.787', 'grad_norm': '1.256', 'learning_rate': '1.266e-05', 'epoch': '0.6796', 'num_input_tokens_seen': 496025136, 'train_runtime': '1.078e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152900/225000 [299:25:53<124:25:45,  6.21s/it]
 68%|██████▊   | 152901/225000 [299:25:59<123:59:55,  6.19s/it]
 68%|██████▊   | 152902/225000 [299:26:03<111:23:24,  5.56s/it]
 68%|██████▊   | 152903/225000 [299:26:08<104:50:43,  5.24s/it]
 68%|██████▊   | 152904/225000 [299:26:15<117:18:33,  5.86s/it]
 68%|██████▊   | 152905/225000 [299:26:23<131:43:47,  6.58s/it]
 68%|██████▊   | 152906/225000 [299:26:28<119:08:24,  5.95s/it]
 68%|██████▊   | 152907/225000 [299:26:32<108:45:28,  5.43s/it]
 68%|██████▊   | 152908/225000 [299:26:37<105:24:38,  5.26s/it]
 68%|██████▊   | 152909/225000 [299:26:42<101:25:03,  5.06s/it]
 68%|██████▊   | 152910/225000 [299:26:46<99:12:28,  4.95s/it] 
 68%|██████▊   | 152911/225000 [299:26:51<97:39:29,  4.88s/it]
 68%|██████▊   | 152912/225000 [299:26:56<96:06:38,  4.80s/it]
 68%|██████▊   | 152913/225000 [299:27:06<128:55:08,  6.44s/it]
 68%|██████▊   | 152914/225000 [299:27:14<138:44:57,  6.93s/it]
 68%|██████▊   | 152915/225000 [299:27:21<138:24:44,  6.91s/it]
 68%|██████▊   | 152916/225000 [299:27:26<126:18:01,  6.31s/it]
 68%|██████▊   | 152917/225000 [299:27:29<109:16:23,  5.46s/it]
 68%|██████▊   | 152918/225000 [299:27:38<130:23:57,  6.51s/it]
 68%|██████▊   | 152919/225000 [299:27:42<111:47:38,  5.58s/it]
 68%|██████▊   | 152920/225000 [299:27:47<108:45:14,  5.43s/it]
 68%|██████▊   | 152921/225000 [299:27:55<125:22:49,  6.26s/it]
 68%|██████▊   | 152922/225000 [299:27:59<114:54:14,  5.74s/it]
 68%|██████▊   | 152923/225000 [299:28:04<108:22:07,  5.41s/it]
 68%|██████▊   | 152924/225000 [299:28:08<101:03:56,  5.05s/it]
 68%|██████▊   | 152925/225000 [299:28:12<95:58:51,  4.79s/it] 
 68%|██████▊   | 152926/225000 [299:28:17<95:00:34,  4.75s/it]
 68%|██████▊   | 152927/225000 [299:28:22<94:23:29,  4.71s/it]
 68%|██████▊   | 152928/225000 [299:28:33<134:07:11,  6.70s/it]
 68%|██████▊   | 152929/225000 [299:28:39<128:02:25,  6.40s/it]
 68%|██████▊   | 152930/225000 [299:28:44<121:14:52,  6.06s/it]
 68%|██████▊   | 152931/225000 [299:28:51<125:34:08,  6.27s/it]
 68%|██████▊   | 152932/225000 [299:28:55<115:00:46,  5.75s/it]
 68%|██████▊   | 152933/225000 [299:29:00<107:44:26,  5.38s/it]
 68%|██████▊   | 152934/225000 [299:29:04<102:28:39,  5.12s/it]
 68%|██████▊   | 152935/225000 [299:29:09<99:04:31,  4.95s/it] 
 68%|██████▊   | 152936/225000 [299:29:16<114:50:59,  5.74s/it]
 68%|██████▊   | 152937/225000 [299:29:26<140:28:22,  7.02s/it]
 68%|██████▊   | 152938/225000 [299:29:31<124:30:20,  6.22s/it]
 68%|██████▊   | 152939/225000 [299:29:35<110:51:00,  5.54s/it]
 68%|██████▊   | 152940/225000 [299:29:39<103:21:07,  5.16s/it]
 68%|██████▊   | 152941/225000 [299:29:50<139:03:06,  6.95s/it]
 68%|██████▊   | 152942/225000 [299:29:54<123:05:37,  6.15s/it]
 68%|██████▊   | 152943/225000 [299:30:00<116:40:29,  5.83s/it]
 68%|██████▊   | 152944/225000 [299:30:07<125:52:07,  6.29s/it]
 68%|██████▊   | 152945/225000 [299:30:10<109:00:12,  5.45s/it]
 68%|██████▊   | 152946/225000 [299:30:16<107:20:51,  5.36s/it]
 68%|██████▊   | 152947/225000 [299:30:21<106:37:43,  5.33s/it]
 68%|██████▊   | 152948/225000 [299:30:26<105:38:46,  5.28s/it]
 68%|██████▊   | 152949/225000 [299:30:31<102:57:44,  5.14s/it]
 68%|██████▊   | 152950/225000 [299:30:36<102:53:50,  5.14s/it]
                                                               
{'loss': '0.8044', 'grad_norm': '1.143', 'learning_rate': '1.264e-05', 'epoch': '0.6798', 'num_input_tokens_seen': 496154816, 'train_runtime': '1.078e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 152950/225000 [299:30:36<102:53:50,  5.14s/it]
 68%|██████▊   | 152951/225000 [299:30:41<104:11:46,  5.21s/it]
 68%|██████▊   | 152952/225000 [299:30:46<103:34:33,  5.18s/it]
 68%|██████▊   | 152953/225000 [299:30:52<104:44:37,  5.23s/it]
 68%|██████▊   | 152954/225000 [299:30:57<103:20:28,  5.16s/it]
 68%|██████▊   | 152955/225000 [299:31:02<104:15:25,  5.21s/it]
 68%|██████▊   | 152956/225000 [299:31:07<103:17:25,  5.16s/it]
 68%|██████▊   | 152957/225000 [299:31:12<103:35:52,  5.18s/it]
 68%|██████▊   | 152958/225000 [299:31:18<105:54:19,  5.29s/it]
 68%|██████▊   | 152959/225000 [299:31:23<105:15:27,  5.26s/it]
 68%|██████▊   | 152960/225000 [299:31:31<120:06:45,  6.00s/it]
 68%|██████▊   | 152961/225000 [299:31:43<158:02:01,  7.90s/it]
 68%|██████▊   | 152962/225000 [299:31:47<133:53:55,  6.69s/it]
 68%|██████▊   | 152963/225000 [299:31:54<135:45:59,  6.78s/it]
 68%|██████▊   | 152964/225000 [299:31:59<125:23:40,  6.27s/it]
 68%|██████▊   | 152965/225000 [299:32:04<115:52:33,  5.79s/it]
 68%|██████▊   | 152966/225000 [299:32:09<110:48:35,  5.54s/it]
 68%|██████▊   | 152967/225000 [299:32:13<105:24:09,  5.27s/it]
 68%|██████▊   | 152968/225000 [299:32:18<99:30:40,  4.97s/it] 
 68%|██████▊   | 152969/225000 [299:32:22<96:06:11,  4.80s/it]
 68%|██████▊   | 152970/225000 [299:32:31<118:51:26,  5.94s/it]
 68%|██████▊   | 152971/225000 [299:32:40<139:56:00,  6.99s/it]
 68%|██████▊   | 152972/225000 [299:32:44<120:30:03,  6.02s/it]
 68%|██████▊   | 152973/225000 [299:32:48<110:58:41,  5.55s/it]
 68%|██████▊   | 152974/225000 [299:32:57<132:40:33,  6.63s/it]
 68%|██████▊   | 152975/225000 [299:33:01<113:57:00,  5.70s/it]
 68%|██████▊   | 152976/225000 [299:33:09<126:34:38,  6.33s/it]
 68%|██████▊   | 152977/225000 [299:33:12<109:36:56,  5.48s/it]
 68%|██████▊   | 152978/225000 [299:33:21<129:55:13,  6.49s/it]
 68%|██████▊   | 152979/225000 [299:33:25<112:10:38,  5.61s/it]
 68%|██████▊   | 152980/225000 [299:33:29<106:09:08,  5.31s/it]
 68%|██████▊   | 152981/225000 [299:33:38<124:17:12,  6.21s/it]
 68%|██████▊   | 152982/225000 [299:33:42<111:28:55,  5.57s/it]
 68%|██████▊   | 152983/225000 [299:33:47<110:37:13,  5.53s/it]
 68%|██████▊   | 152984/225000 [299:33:52<106:06:22,  5.30s/it]
 68%|██████▊   | 152985/225000 [299:33:57<104:29:04,  5.22s/it]
 68%|██████▊   | 152986/225000 [299:34:02<104:26:25,  5.22s/it]
 68%|██████▊   | 152987/225000 [299:34:07<104:49:54,  5.24s/it]
 68%|██████▊   | 152988/225000 [299:34:13<104:31:32,  5.23s/it]
 68%|██████▊   | 152989/225000 [299:34:18<104:00:19,  5.20s/it]
 68%|██████▊   | 152990/225000 [299:34:23<105:25:50,  5.27s/it]
 68%|██████▊   | 152991/225000 [299:34:28<105:12:10,  5.26s/it]
 68%|██████▊   | 152992/225000 [299:34:39<138:04:05,  6.90s/it]
 68%|██████▊   | 152993/225000 [299:34:46<139:34:23,  6.98s/it]
 68%|██████▊   | 152994/225000 [299:34:50<121:20:36,  6.07s/it]
 68%|██████▊   | 152995/225000 [299:34:54<109:13:35,  5.46s/it]
 68%|██████▊   | 152996/225000 [299:34:58<101:24:05,  5.07s/it]
 68%|██████▊   | 152997/225000 [299:35:03<98:52:06,  4.94s/it] 
 68%|██████▊   | 152998/225000 [299:35:08<99:28:25,  4.97s/it]
 68%|██████▊   | 152999/225000 [299:35:13<101:06:09,  5.06s/it]
 68%|██████▊   | 153000/225000 [299:35:19<104:30:42,  5.23s/it]
                                                               
{'loss': '0.7892', 'grad_norm': '1.114', 'learning_rate': '1.263e-05', 'epoch': '0.68', 'num_input_tokens_seen': 496285656, 'train_runtime': '1.079e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 153000/225000 [299:35:19<104:30:42,  5.23s/it]
 68%|██████▊   | 153001/225000 [299:35:24<102:23:58,  5.12s/it]
 68%|██████▊   | 153002/225000 [299:35:35<138:06:53,  6.91s/it]
 68%|██████▊   | 153003/225000 [299:35:43<145:29:13,  7.27s/it]
 68%|██████▊   | 153004/225000 [299:35:47<122:35:29,  6.13s/it]
 68%|██████▊   | 153005/225000 [299:35:56<145:08:33,  7.26s/it]
 68%|██████▊   | 153006/225000 [299:36:00<123:28:32,  6.17s/it]
 68%|██████▊   | 153007/225000 [299:36:04<110:55:30,  5.55s/it]
 68%|██████▊   | 153008/225000 [299:36:13<128:00:25,  6.40s/it]
 68%|██████▊   | 153009/225000 [299:36:17<116:44:09,  5.84s/it]
 68%|██████▊   | 153010/225000 [299:36:24<123:25:34,  6.17s/it]
 68%|██████▊   | 153011/225000 [299:36:28<109:14:08,  5.46s/it]
 68%|██████▊   | 153012/225000 [299:36:32<99:54:46,  5.00s/it] 
 68%|██████▊   | 153013/225000 [299:36:36<96:23:00,  4.82s/it]
 68%|██████▊   | 153014/225000 [299:36:41<95:24:06,  4.77s/it]
 68%|██████▊   | 153015/225000 [299:36:46<96:46:35,  4.84s/it]
 68%|██████▊   | 153016/225000 [299:36:51<98:37:56,  4.93s/it]
 68%|██████▊   | 153017/225000 [299:37:02<135:06:38,  6.76s/it]
 68%|██████▊   | 153018/225000 [299:37:09<133:50:21,  6.69s/it]
 68%|██████▊   | 153019/225000 [299:37:15<132:24:05,  6.62s/it]
 68%|██████▊   | 153020/225000 [299:37:20<120:26:11,  6.02s/it]
 68%|██████▊   | 153021/225000 [299:37:24<111:33:12,  5.58s/it]
 68%|██████▊   | 153022/225000 [299:37:29<106:53:19,  5.35s/it]
 68%|██████▊   | 153023/225000 [299:37:34<105:26:51,  5.27s/it]
 68%|██████▊   | 153024/225000 [299:37:39<101:14:42,  5.06s/it]
 68%|██████▊   | 153025/225000 [299:37:44<101:01:02,  5.05s/it]
 68%|██████▊   | 153026/225000 [299:37:50<110:32:58,  5.53s/it]
 68%|██████▊   | 153027/225000 [299:38:02<150:01:16,  7.50s/it]
 68%|██████▊   | 153028/225000 [299:38:06<127:59:03,  6.40s/it]
 68%|██████▊   | 153029/225000 [299:38:10<111:35:34,  5.58s/it]
 68%|██████▊   | 153030/225000 [299:38:18<126:41:56,  6.34s/it]
 68%|██████▊   | 153031/225000 [299:38:22<113:36:49,  5.68s/it]
 68%|██████▊   | 153032/225000 [299:38:27<105:36:35,  5.28s/it]
 68%|██████▊   | 153033/225000 [299:38:31<98:55:24,  4.95s/it] 
 68%|██████▊   | 153034/225000 [299:38:41<132:11:06,  6.61s/it]
 68%|██████▊   | 153035/225000 [299:38:48<131:56:03,  6.60s/it]
 68%|██████▊   | 153036/225000 [299:38:51<113:27:22,  5.68s/it]
 68%|██████▊   | 153037/225000 [299:39:00<133:05:42,  6.66s/it]
 68%|██████▊   | 153038/225000 [299:39:04<117:34:44,  5.88s/it]
 68%|██████▊   | 153039/225000 [299:39:12<131:21:28,  6.57s/it]
 68%|██████▊   | 153040/225000 [299:39:17<121:07:07,  6.06s/it]
 68%|██████▊   | 153041/225000 [299:39:23<120:31:38,  6.03s/it]
 68%|██████▊   | 153042/225000 [299:39:29<117:06:43,  5.86s/it]
 68%|██████▊   | 153043/225000 [299:39:33<110:06:24,  5.51s/it]
 68%|██████▊   | 153044/225000 [299:39:41<121:08:49,  6.06s/it]
 68%|██████▊   | 153045/225000 [299:39:46<113:55:19,  5.70s/it]
 68%|██████▊   | 153046/225000 [299:39:51<110:39:00,  5.54s/it]
 68%|██████▊   | 153047/225000 [299:39:56<109:22:52,  5.47s/it]
 68%|██████▊   | 153048/225000 [299:40:01<106:17:44,  5.32s/it]
 68%|██████▊   | 153049/225000 [299:40:06<105:06:48,  5.26s/it]
 68%|██████▊   | 153050/225000 [299:40:11<104:06:09,  5.21s/it]
                                                               
{'loss': '0.7745', 'grad_norm': '1.565', 'learning_rate': '1.261e-05', 'epoch': '0.6802', 'num_input_tokens_seen': 496415856, 'train_runtime': '1.079e+06', 'train_tokens_per_second': '460.2'}

 68%|██████▊   | 153050/225000 [299:40:11<104:06:09,  5.21s/it]
 68%|██████▊   | 153051/225000 [299:40:16<101:46:33,  5.09s/it]
 68%|██████▊   | 153052/225000 [299:40:21<100:02:34,  5.01s/it]
 68%|██████▊   | 153053/225000 [299:40:26<100:17:55,  5.02s/it]
 68%|██████▊   | 153054/225000 [299:40:38<143:59:51,  7.21s/it]
 68%|██████▊   | 153055/225000 [299:40:43<130:43:13,  6.54s/it]
 68%|██████▊   | 153056/225000 [299:40:47<113:08:00,  5.66s/it]
 68%|██████▊   | 153057/225000 [299:40:51<101:40:14,  5.09s/it]
 68%|██████▊   | 153058/225000 [299:41:02<136:53:05,  6.85s/it]
 68%|██████▊   | 153059/225000 [299:41:05<114:24:00,  5.72s/it]
 68%|██████▊   | 153060/225000 [299:41:08<101:38:33,  5.09s/it]
 68%|██████▊   | 153061/225000 [299:41:18<127:54:44,  6.40s/it]
 68%|██████▊   | 153062/225000 [299:41:23<121:50:05,  6.10s/it]
 68%|██████▊   | 153063/225000 [299:41:29<119:34:05,  5.98s/it]
 68%|██████▊   | 153064/225000 [299:41:37<131:48:38,  6.60s/it]
 68%|██████▊   | 153065/225000 [299:41:44<133:12:31,  6.67s/it]
 68%|██████▊   | 153066/225000 [299:41:47<114:20:57,  5.72s/it]
 68%|██████▊   | 153067/225000 [299:41:56<130:15:53,  6.52s/it]
 68%|██████▊   | 153068/225000 [299:41:59<111:29:17,  5.58s/it]
 68%|██████▊   | 153069/225000 [299:42:06<119:29:56,  5.98s/it]
 68%|██████▊   | 153070/225000 [299:42:12<121:21:47,  6.07s/it]
 68%|██████▊   | 153071/225000 [299:42:17<114:26:57,  5.73s/it]
 68%|██████▊   | 153072/225000 [299:42:21<102:30:10,  5.13s/it]
 68%|██████▊   | 153073/225000 [299:42:29<122:45:48,  6.14s/it]
 68%|██████▊   | 153074/225000 [299:42:34<112:23:01,  5.62s/it]
 68%|██████▊   | 153075/225000 [299:42:38<106:19:37,  5.32s/it]
 68%|██████▊   | 153076/225000 [299:42:43<103:56:53,  5.20s/it]
 68%|██████▊   | 153077/225000 [299:42:48<102:14:12,  5.12s/it]
 68%|██████▊   | 153078/225000 [299:42:54<103:36:45,  5.19s/it]
 68%|██████▊   | 153079/225000 [299:43:06<144:18:20,  7.22s/it]
 68%|██████▊   | 153080/225000 [299:43:09<124:12:06,  6.22s/it]
 68%|██████▊   | 153081/225000 [299:43:13<110:59:41,  5.56s/it]
 68%|██████▊   | 153082/225000 [299:43:18<102:44:22,  5.14s/it]
 68%|██████▊   | 153083/225000 [299:43:27<128:12:38,  6.42s/it]
 68%|██████▊   | 153084/225000 [299:43:32<117:52:41,  5.90s/it]
 68%|██████▊   | 153085/225000 [299:43:37<111:09:30,  5.56s/it]
 68%|██████▊   | 153086/225000 [299:43:41<107:16:57,  5.37s/it]
 68%|██████▊   | 153087/225000 [299:43:46<103:18:58,  5.17s/it]
 68%|██████▊   | 153088/225000 [299:43:51<98:43:00,  4.94s/it] 
 68%|██████▊   | 153089/225000 [299:43:57<107:56:52,  5.40s/it]
 68%|██████▊   | 153090/225000 [299:44:09<145:51:11,  7.30s/it]
 68%|██████▊   | 153091/225000 [299:44:12<124:11:27,  6.22s/it]
 68%|██████▊   | 153092/225000 [299:44:21<135:42:15,  6.79s/it]
 68%|██████▊   | 153093/225000 [299:44:25<121:55:20,  6.10s/it]
 68%|██████▊   | 153094/225000 [299:44:30<114:08:56,  5.71s/it]
 68%|██████▊   | 153095/225000 [299:44:35<109:36:25,  5.49s/it]
 68%|██████▊   | 153096/225000 [299:44:40<107:43:55,  5.39s/it]
 68%|██████▊   | 153097/225000 [299:44:45<104:20:23,  5.22s/it]
 68%|██████▊   | 153098/225000 [299:44:50<102:24:38,  5.13s/it]
 68%|██████▊   | 153099/225000 [299:44:55<101:53:33,  5.10s/it]
 68%|██████▊   | 153100/225000 [299:45:00<101:45:44,  5.10s/it]
                                                               
{'loss': '0.7878', 'grad_norm': '1.159', 'learning_rate': '1.26e-05', 'epoch': '0.6804', 'num_input_tokens_seen': 496545928, 'train_runtime': '1.079e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153100/225000 [299:45:00<101:45:44,  5.10s/it]
 68%|██████▊   | 153101/225000 [299:45:11<137:50:21,  6.90s/it]
 68%|██████▊   | 153102/225000 [299:45:17<131:13:04,  6.57s/it]
 68%|██████▊   | 153103/225000 [299:45:23<128:47:31,  6.45s/it]
 68%|██████▊   | 153104/225000 [299:45:28<118:27:31,  5.93s/it]
 68%|██████▊   | 153105/225000 [299:45:32<110:33:58,  5.54s/it]
 68%|██████▊   | 153106/225000 [299:45:37<104:33:39,  5.24s/it]
 68%|██████▊   | 153107/225000 [299:45:42<104:14:48,  5.22s/it]
 68%|██████▊   | 153108/225000 [299:45:47<104:15:21,  5.22s/it]
 68%|██████▊   | 153109/225000 [299:45:53<105:50:14,  5.30s/it]
 68%|██████▊   | 153110/225000 [299:45:57<102:12:16,  5.12s/it]
 68%|██████▊   | 153111/225000 [299:46:10<146:23:41,  7.33s/it]
 68%|██████▊   | 153112/225000 [299:46:14<125:41:24,  6.29s/it]
 68%|██████▊   | 153113/225000 [299:46:22<139:15:14,  6.97s/it]
 68%|██████▊   | 153114/225000 [299:46:27<123:04:49,  6.16s/it]
 68%|██████▊   | 153115/225000 [299:46:31<109:48:50,  5.50s/it]
 68%|██████▊   | 153116/225000 [299:46:35<102:12:41,  5.12s/it]
 68%|██████▊   | 153117/225000 [299:46:42<116:54:44,  5.86s/it]
 68%|██████▊   | 153118/225000 [299:46:48<115:48:43,  5.80s/it]
 68%|██████▊   | 153119/225000 [299:46:52<106:19:48,  5.33s/it]
 68%|██████▊   | 153120/225000 [299:46:57<101:29:42,  5.08s/it]
 68%|██████▊   | 153121/225000 [299:47:05<122:57:06,  6.16s/it]
 68%|██████▊   | 153122/225000 [299:47:12<128:05:08,  6.42s/it]
 68%|██████▊   | 153123/225000 [299:47:16<111:53:38,  5.60s/it]
 68%|██████▊   | 153124/225000 [299:47:26<138:31:10,  6.94s/it]
 68%|██████▊   | 153125/225000 [299:47:31<127:15:40,  6.37s/it]
 68%|██████▊   | 153126/225000 [299:47:35<114:18:56,  5.73s/it]
 68%|██████▊   | 153127/225000 [299:47:40<105:38:18,  5.29s/it]
 68%|██████▊   | 153128/225000 [299:47:45<108:22:53,  5.43s/it]
 68%|██████▊   | 153129/225000 [299:47:56<136:07:10,  6.82s/it]
 68%|██████▊   | 153130/225000 [299:47:59<115:44:58,  5.80s/it]
 68%|██████▊   | 153131/225000 [299:48:07<130:18:24,  6.53s/it]
 68%|██████▊   | 153132/225000 [299:48:11<113:10:08,  5.67s/it]
 68%|██████▊   | 153133/225000 [299:48:18<121:09:14,  6.07s/it]
 68%|██████▊   | 153134/225000 [299:48:25<127:46:36,  6.40s/it]
 68%|██████▊   | 153135/225000 [299:48:29<111:57:49,  5.61s/it]
 68%|██████▊   | 153136/225000 [299:48:33<102:37:54,  5.14s/it]
 68%|██████▊   | 153137/225000 [299:48:37<94:32:41,  4.74s/it] 
 68%|██████▊   | 153138/225000 [299:48:43<105:54:37,  5.31s/it]
 68%|██████▊   | 153139/225000 [299:48:52<124:59:30,  6.26s/it]
 68%|██████▊   | 153140/225000 [299:48:56<110:59:47,  5.56s/it]
 68%|██████▊   | 153141/225000 [299:49:00<102:09:26,  5.12s/it]
 68%|██████▊   | 153142/225000 [299:49:10<135:08:52,  6.77s/it]
 68%|██████▊   | 153143/225000 [299:49:14<116:51:44,  5.85s/it]
 68%|██████▊   | 153144/225000 [299:49:22<126:42:17,  6.35s/it]
 68%|██████▊   | 153145/225000 [299:49:27<118:10:45,  5.92s/it]
 68%|██████▊   | 153146/225000 [299:49:33<119:04:40,  5.97s/it]
 68%|██████▊   | 153147/225000 [299:49:37<108:21:50,  5.43s/it]
 68%|██████▊   | 153148/225000 [299:49:44<118:44:08,  5.95s/it]
 68%|██████▊   | 153149/225000 [299:49:50<119:41:16,  6.00s/it]
 68%|██████▊   | 153150/225000 [299:49:54<105:55:35,  5.31s/it]
                                                               
{'loss': '0.7821', 'grad_norm': '1.142', 'learning_rate': '1.258e-05', 'epoch': '0.6807', 'num_input_tokens_seen': 496675480, 'train_runtime': '1.079e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153150/225000 [299:49:54<105:55:35,  5.31s/it]
 68%|██████▊   | 153151/225000 [299:50:02<121:17:26,  6.08s/it]
 68%|██████▊   | 153152/225000 [299:50:07<114:28:55,  5.74s/it]
 68%|██████▊   | 153153/225000 [299:50:10<101:16:18,  5.07s/it]
 68%|██████▊   | 153154/225000 [299:50:19<124:24:27,  6.23s/it]
 68%|██████▊   | 153155/225000 [299:50:24<117:23:29,  5.88s/it]
 68%|██████▊   | 153156/225000 [299:50:28<108:12:40,  5.42s/it]
 68%|██████▊   | 153157/225000 [299:50:34<107:13:13,  5.37s/it]
 68%|██████▊   | 153158/225000 [299:50:39<109:13:11,  5.47s/it]
 68%|██████▊   | 153159/225000 [299:50:44<106:39:21,  5.34s/it]
 68%|██████▊   | 153160/225000 [299:50:50<108:13:07,  5.42s/it]
 68%|██████▊   | 153161/225000 [299:50:56<108:30:37,  5.44s/it]
 68%|██████▊   | 153162/225000 [299:51:01<107:31:11,  5.39s/it]
 68%|██████▊   | 153163/225000 [299:51:06<105:33:15,  5.29s/it]
 68%|██████▊   | 153164/225000 [299:51:11<105:43:56,  5.30s/it]
 68%|██████▊   | 153165/225000 [299:51:17<106:19:37,  5.33s/it]
 68%|██████▊   | 153166/225000 [299:51:22<107:03:49,  5.37s/it]
 68%|██████▊   | 153167/225000 [299:51:29<117:54:10,  5.91s/it]
 68%|██████▊   | 153168/225000 [299:51:42<156:33:54,  7.85s/it]
 68%|██████▊   | 153169/225000 [299:51:45<131:56:39,  6.61s/it]
 68%|██████▊   | 153170/225000 [299:51:51<128:33:57,  6.44s/it]
 68%|██████▊   | 153171/225000 [299:51:57<123:56:06,  6.21s/it]
 68%|██████▊   | 153172/225000 [299:52:03<120:14:05,  6.03s/it]
 68%|██████▊   | 153173/225000 [299:52:10<128:04:43,  6.42s/it]
 68%|██████▊   | 153174/225000 [299:52:15<117:29:05,  5.89s/it]
 68%|██████▊   | 153175/225000 [299:52:21<118:44:08,  5.95s/it]
 68%|██████▊   | 153176/225000 [299:52:25<108:35:34,  5.44s/it]
 68%|██████▊   | 153177/225000 [299:52:29<100:59:35,  5.06s/it]
 68%|██████▊   | 153178/225000 [299:52:33<96:08:19,  4.82s/it] 
 68%|██████▊   | 153179/225000 [299:52:42<118:18:02,  5.93s/it]
 68%|██████▊   | 153180/225000 [299:52:51<134:26:20,  6.74s/it]
 68%|██████▊   | 153181/225000 [299:52:54<117:05:55,  5.87s/it]
 68%|██████▊   | 153182/225000 [299:52:58<102:54:26,  5.16s/it]
 68%|██████▊   | 153183/225000 [299:53:03<100:10:25,  5.02s/it]
 68%|██████▊   | 153184/225000 [299:53:14<140:06:08,  7.02s/it]
 68%|██████▊   | 153185/225000 [299:53:18<119:14:30,  5.98s/it]
 68%|██████▊   | 153186/225000 [299:53:22<110:04:38,  5.52s/it]
 68%|██████▊   | 153187/225000 [299:53:27<106:55:07,  5.36s/it]
 68%|██████▊   | 153188/225000 [299:53:33<107:28:23,  5.39s/it]
 68%|██████▊   | 153189/225000 [299:53:38<106:59:41,  5.36s/it]
 68%|██████▊   | 153190/225000 [299:53:43<106:41:31,  5.35s/it]
 68%|██████▊   | 153191/225000 [299:53:49<106:07:36,  5.32s/it]
 68%|██████▊   | 153192/225000 [299:53:54<104:31:40,  5.24s/it]
 68%|██████▊   | 153193/225000 [299:53:59<105:04:23,  5.27s/it]
 68%|██████▊   | 153194/225000 [299:54:04<103:57:24,  5.21s/it]
 68%|██████▊   | 153195/225000 [299:54:09<104:32:27,  5.24s/it]
 68%|██████▊   | 153196/225000 [299:54:17<115:58:45,  5.81s/it]
 68%|██████▊   | 153197/225000 [299:54:29<153:53:11,  7.72s/it]
 68%|██████▊   | 153198/225000 [299:54:35<144:46:32,  7.26s/it]
 68%|██████▊   | 153199/225000 [299:54:39<128:49:45,  6.46s/it]
 68%|██████▊   | 153200/225000 [299:54:45<120:59:08,  6.07s/it]
                                                               
{'loss': '0.789', 'grad_norm': '1.083', 'learning_rate': '1.256e-05', 'epoch': '0.6809', 'num_input_tokens_seen': 496805728, 'train_runtime': '1.08e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153200/225000 [299:54:45<120:59:08,  6.07s/it]
 68%|██████▊   | 153201/225000 [299:54:49<112:17:53,  5.63s/it]
 68%|██████▊   | 153202/225000 [299:54:54<108:07:30,  5.42s/it]
 68%|██████▊   | 153203/225000 [299:55:00<108:25:45,  5.44s/it]
 68%|██████▊   | 153204/225000 [299:55:04<104:34:19,  5.24s/it]
 68%|██████▊   | 153205/225000 [299:55:09<103:24:42,  5.19s/it]
 68%|██████▊   | 153206/225000 [299:55:15<103:26:34,  5.19s/it]
 68%|██████▊   | 153207/225000 [299:55:20<104:53:35,  5.26s/it]
 68%|██████▊   | 153208/225000 [299:55:25<102:55:35,  5.16s/it]
 68%|██████▊   | 153209/225000 [299:55:37<144:12:27,  7.23s/it]
 68%|██████▊   | 153210/225000 [299:55:42<129:11:56,  6.48s/it]
 68%|██████▊   | 153211/225000 [299:55:47<122:58:46,  6.17s/it]
 68%|██████▊   | 153212/225000 [299:55:53<119:43:40,  6.00s/it]
 68%|██████▊   | 153213/225000 [299:55:57<107:34:16,  5.39s/it]
 68%|██████▊   | 153214/225000 [299:56:01<99:50:31,  5.01s/it] 
 68%|██████▊   | 153215/225000 [299:56:05<95:03:54,  4.77s/it]
 68%|██████▊   | 153216/225000 [299:56:17<138:38:59,  6.95s/it]
 68%|██████▊   | 153217/225000 [299:56:22<128:16:22,  6.43s/it]
 68%|██████▊   | 153218/225000 [299:56:26<111:29:57,  5.59s/it]
 68%|██████▊   | 153219/225000 [299:56:30<100:54:12,  5.06s/it]
 68%|██████▊   | 153220/225000 [299:56:42<144:50:01,  7.26s/it]
 68%|██████▊   | 153221/225000 [299:56:46<121:27:34,  6.09s/it]
 68%|██████▊   | 153222/225000 [299:56:50<108:55:32,  5.46s/it]
 68%|██████▊   | 153223/225000 [299:56:55<108:02:09,  5.42s/it]
 68%|██████▊   | 153224/225000 [299:57:01<109:35:50,  5.50s/it]
 68%|██████▊   | 153225/225000 [299:57:06<107:06:38,  5.37s/it]
 68%|██████▊   | 153226/225000 [299:57:10<101:51:22,  5.11s/it]
 68%|██████▊   | 153227/225000 [299:57:15<101:10:03,  5.07s/it]
 68%|██████▊   | 153228/225000 [299:57:21<102:42:30,  5.15s/it]
 68%|██████▊   | 153229/225000 [299:57:26<104:16:30,  5.23s/it]
 68%|██████▊   | 153230/225000 [299:57:31<103:18:52,  5.18s/it]
 68%|██████▊   | 153231/225000 [299:57:37<105:11:01,  5.28s/it]
 68%|██████▊   | 153232/225000 [299:57:42<104:59:54,  5.27s/it]
 68%|██████▊   | 153233/225000 [299:57:47<105:32:57,  5.29s/it]
 68%|██████▊   | 153234/225000 [299:57:53<106:44:16,  5.35s/it]
 68%|██████▊   | 153235/225000 [299:57:58<105:28:45,  5.29s/it]
 68%|██████▊   | 153236/225000 [299:58:03<104:00:38,  5.22s/it]
 68%|██████▊   | 153237/225000 [299:58:15<147:47:57,  7.41s/it]
 68%|██████▊   | 153238/225000 [299:58:23<148:42:33,  7.46s/it]
 68%|██████▊   | 153239/225000 [299:58:30<146:08:28,  7.33s/it]
 68%|██████▊   | 153240/225000 [299:58:34<124:31:41,  6.25s/it]
 68%|██████▊   | 153241/225000 [299:58:38<113:19:57,  5.69s/it]
 68%|██████▊   | 153242/225000 [299:58:46<125:01:50,  6.27s/it]
 68%|██████▊   | 153243/225000 [299:58:49<107:01:45,  5.37s/it]
 68%|██████▊   | 153244/225000 [299:58:58<131:24:11,  6.59s/it]
 68%|██████▊   | 153245/225000 [299:59:03<118:25:53,  5.94s/it]
 68%|██████▊   | 153246/225000 [299:59:09<120:57:19,  6.07s/it]
 68%|██████▊   | 153247/225000 [299:59:14<111:55:18,  5.62s/it]
 68%|██████▊   | 153248/225000 [299:59:18<105:39:24,  5.30s/it]
 68%|██████▊   | 153249/225000 [299:59:23<101:53:37,  5.11s/it]
 68%|██████▊   | 153250/225000 [299:59:29<108:31:20,  5.45s/it]
                                                               
{'loss': '0.7818', 'grad_norm': '1.113', 'learning_rate': '1.255e-05', 'epoch': '0.6811', 'num_input_tokens_seen': 496936600, 'train_runtime': '1.08e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153250/225000 [299:59:29<108:31:20,  5.45s/it]
 68%|██████▊   | 153251/225000 [299:59:39<137:33:05,  6.90s/it]
 68%|██████▊   | 153252/225000 [299:59:43<117:35:06,  5.90s/it]
 68%|██████▊   | 153253/225000 [299:59:49<116:57:05,  5.87s/it]
 68%|██████▊   | 153254/225000 [299:59:58<137:23:57,  6.89s/it]
 68%|██████▊   | 153255/225000 [300:00:01<114:53:41,  5.77s/it]
 68%|██████▊   | 153256/225000 [300:00:05<100:57:39,  5.07s/it]
 68%|██████▊   | 153257/225000 [300:00:14<124:45:13,  6.26s/it]
 68%|██████▊   | 153258/225000 [300:00:19<116:35:52,  5.85s/it]
 68%|██████▊   | 153259/225000 [300:00:23<106:01:40,  5.32s/it]
 68%|██████▊   | 153260/225000 [300:00:27<101:53:45,  5.11s/it]
 68%|██████▊   | 153261/225000 [300:00:32<99:18:54,  4.98s/it] 
 68%|██████▊   | 153262/225000 [300:00:39<112:56:40,  5.67s/it]
 68%|██████▊   | 153263/225000 [300:00:48<128:14:13,  6.44s/it]
 68%|██████▊   | 153264/225000 [300:00:51<111:11:08,  5.58s/it]
 68%|██████▊   | 153265/225000 [300:00:56<105:52:00,  5.31s/it]
 68%|██████▊   | 153266/225000 [300:01:00<101:26:26,  5.09s/it]
 68%|██████▊   | 153267/225000 [300:01:05<99:12:19,  4.98s/it] 
 68%|██████▊   | 153268/225000 [300:01:12<110:40:32,  5.55s/it]
 68%|██████▊   | 153269/225000 [300:01:23<144:59:30,  7.28s/it]
 68%|██████▊   | 153270/225000 [300:01:28<127:59:29,  6.42s/it]
 68%|██████▊   | 153271/225000 [300:01:32<115:53:54,  5.82s/it]
 68%|██████▊   | 153272/225000 [300:01:37<109:11:52,  5.48s/it]
 68%|██████▊   | 153273/225000 [300:01:41<102:34:42,  5.15s/it]
 68%|██████▊   | 153274/225000 [300:01:46<100:15:22,  5.03s/it]
 68%|██████▊   | 153275/225000 [300:01:50<96:53:15,  4.86s/it] 
 68%|██████▊   | 153276/225000 [300:02:02<137:49:22,  6.92s/it]
 68%|██████▊   | 153277/225000 [300:02:09<139:49:19,  7.02s/it]
 68%|██████▊   | 153278/225000 [300:02:16<135:41:54,  6.81s/it]
 68%|██████▊   | 153279/225000 [300:02:20<121:39:13,  6.11s/it]
 68%|██████▊   | 153280/225000 [300:02:24<109:01:02,  5.47s/it]
 68%|██████▊   | 153281/225000 [300:02:28<101:53:37,  5.11s/it]
 68%|██████▊   | 153282/225000 [300:02:41<143:33:46,  7.21s/it]
 68%|██████▊   | 153283/225000 [300:02:44<123:47:49,  6.21s/it]
 68%|██████▊   | 153284/225000 [300:02:49<112:07:39,  5.63s/it]
 68%|██████▊   | 153285/225000 [300:02:54<107:15:30,  5.38s/it]
 68%|██████▊   | 153286/225000 [300:02:59<105:53:28,  5.32s/it]
 68%|██████▊   | 153287/225000 [300:03:04<105:06:03,  5.28s/it]
 68%|██████▊   | 153288/225000 [300:03:09<103:09:00,  5.18s/it]
 68%|██████▊   | 153289/225000 [300:03:14<104:16:58,  5.24s/it]
 68%|██████▊   | 153290/225000 [300:03:19<103:58:51,  5.22s/it]
 68%|██████▊   | 153291/225000 [300:03:24<102:52:28,  5.16s/it]
 68%|██████▊   | 153292/225000 [300:03:30<102:51:58,  5.16s/it]
 68%|██████▊   | 153293/225000 [300:03:40<132:09:40,  6.64s/it]
 68%|██████▊   | 153294/225000 [300:03:47<137:06:05,  6.88s/it]
 68%|██████▊   | 153295/225000 [300:03:52<125:12:46,  6.29s/it]
 68%|██████▊   | 153296/225000 [300:03:57<119:28:43,  6.00s/it]
 68%|██████▊   | 153297/225000 [300:04:03<115:46:54,  5.81s/it]
 68%|██████▊   | 153298/225000 [300:04:08<113:12:10,  5.68s/it]
 68%|██████▊   | 153299/225000 [300:04:13<110:23:19,  5.54s/it]
 68%|██████▊   | 153300/225000 [300:04:19<109:50:34,  5.52s/it]
                                                               
{'loss': '0.7811', 'grad_norm': '1.077', 'learning_rate': '1.253e-05', 'epoch': '0.6813', 'num_input_tokens_seen': 497067616, 'train_runtime': '1.08e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153300/225000 [300:04:19<109:50:34,  5.52s/it]
 68%|██████▊   | 153301/225000 [300:04:24<107:56:02,  5.42s/it]
 68%|██████▊   | 153302/225000 [300:04:29<106:59:51,  5.37s/it]
 68%|██████▊   | 153303/225000 [300:04:35<112:28:53,  5.65s/it]
 68%|██████▊   | 153304/225000 [300:04:41<112:53:59,  5.67s/it]
 68%|██████▊   | 153305/225000 [300:04:53<148:27:02,  7.45s/it]
 68%|██████▊   | 153306/225000 [300:04:56<121:23:34,  6.10s/it]
 68%|██████▊   | 153307/225000 [300:05:04<133:08:08,  6.69s/it]
 68%|██████▊   | 153308/225000 [300:05:08<115:39:27,  5.81s/it]
 68%|██████▊   | 153309/225000 [300:05:12<104:36:28,  5.25s/it]
 68%|██████▊   | 153310/225000 [300:05:18<114:31:02,  5.75s/it]
 68%|██████▊   | 153311/225000 [300:05:26<126:01:15,  6.33s/it]
 68%|██████▊   | 153312/225000 [300:05:30<112:40:36,  5.66s/it]
 68%|██████▊   | 153313/225000 [300:05:34<103:51:23,  5.22s/it]
 68%|██████▊   | 153314/225000 [300:05:39<100:24:31,  5.04s/it]
 68%|██████▊   | 153315/225000 [300:05:49<127:44:50,  6.42s/it]
 68%|██████▊   | 153316/225000 [300:05:52<111:15:27,  5.59s/it]
 68%|██████▊   | 153317/225000 [300:06:00<122:48:24,  6.17s/it]
 68%|██████▊   | 153318/225000 [300:06:05<114:22:33,  5.74s/it]
 68%|██████▊   | 153319/225000 [300:06:08<101:43:53,  5.11s/it]
 68%|██████▊   | 153320/225000 [300:06:19<133:38:23,  6.71s/it]
 68%|██████▊   | 153321/225000 [300:06:22<112:44:27,  5.66s/it]
 68%|██████▊   | 153322/225000 [300:06:28<115:27:01,  5.80s/it]
 68%|██████▊   | 153323/225000 [300:06:32<103:32:42,  5.20s/it]
 68%|██████▊   | 153324/225000 [300:06:37<100:48:32,  5.06s/it]
 68%|██████▊   | 153325/225000 [300:06:46<125:41:53,  6.31s/it]
 68%|██████▊   | 153326/225000 [300:06:50<112:42:53,  5.66s/it]
 68%|██████▊   | 153327/225000 [300:06:53<100:13:18,  5.03s/it]
 68%|██████▊   | 153328/225000 [300:07:03<125:34:22,  6.31s/it]
 68%|██████▊   | 153329/225000 [300:07:07<113:13:24,  5.69s/it]
 68%|██████▊   | 153330/225000 [300:07:11<104:43:19,  5.26s/it]
 68%|██████▊   | 153331/225000 [300:07:15<96:24:03,  4.84s/it] 
 68%|██████▊   | 153332/225000 [300:07:22<109:31:55,  5.50s/it]
 68%|██████▊   | 153333/225000 [300:07:31<130:30:49,  6.56s/it]
 68%|██████▊   | 153334/225000 [300:07:35<113:20:30,  5.69s/it]
 68%|██████▊   | 153335/225000 [300:07:38<100:09:50,  5.03s/it]
 68%|██████▊   | 153336/225000 [300:07:43<95:58:08,  4.82s/it] 
 68%|██████▊   | 153337/225000 [300:07:53<131:21:02,  6.60s/it]
 68%|██████▊   | 153338/225000 [300:07:58<118:20:19,  5.94s/it]
 68%|██████▊   | 153339/225000 [300:08:02<107:06:05,  5.38s/it]
 68%|██████▊   | 153340/225000 [300:08:07<102:54:47,  5.17s/it]
 68%|██████▊   | 153341/225000 [300:08:16<130:35:55,  6.56s/it]
 68%|██████▊   | 153342/225000 [300:08:20<115:54:18,  5.82s/it]
 68%|██████▊   | 153343/225000 [300:08:30<137:01:56,  6.88s/it]
 68%|██████▊   | 153344/225000 [300:08:34<122:30:18,  6.15s/it]
 68%|██████▊   | 153345/225000 [300:08:38<108:28:22,  5.45s/it]
 68%|██████▊   | 153346/225000 [300:08:42<99:52:11,  5.02s/it] 
 68%|██████▊   | 153347/225000 [300:08:49<113:16:27,  5.69s/it]
 68%|██████▊   | 153348/225000 [300:08:58<131:00:21,  6.58s/it]
 68%|██████▊   | 153349/225000 [300:09:02<113:34:02,  5.71s/it]
 68%|██████▊   | 153350/225000 [300:09:06<105:07:49,  5.28s/it]
                                                               
{'loss': '0.7766', 'grad_norm': '1.129', 'learning_rate': '1.252e-05', 'epoch': '0.6816', 'num_input_tokens_seen': 497199056, 'train_runtime': '1.081e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153350/225000 [300:09:06<105:07:49,  5.28s/it]
 68%|██████▊   | 153351/225000 [300:09:11<104:45:08,  5.26s/it]
 68%|██████▊   | 153352/225000 [300:09:17<105:54:07,  5.32s/it]
 68%|██████▊   | 153353/225000 [300:09:22<105:13:27,  5.29s/it]
 68%|██████▊   | 153354/225000 [300:09:27<101:58:13,  5.12s/it]
 68%|██████▊   | 153355/225000 [300:09:31<98:53:36,  4.97s/it] 
 68%|██████▊   | 153356/225000 [300:09:37<101:59:43,  5.13s/it]
 68%|██████▊   | 153357/225000 [300:09:42<101:30:59,  5.10s/it]
 68%|██████▊   | 153358/225000 [300:09:47<104:33:20,  5.25s/it]
 68%|██████▊   | 153359/225000 [300:09:53<105:02:51,  5.28s/it]
 68%|██████▊   | 153360/225000 [300:10:02<128:02:06,  6.43s/it]
 68%|██████▊   | 153361/225000 [300:10:12<149:10:47,  7.50s/it]
 68%|██████▊   | 153362/225000 [300:10:16<132:17:14,  6.65s/it]
 68%|██████▊   | 153363/225000 [300:10:21<121:26:04,  6.10s/it]
 68%|██████▊   | 153364/225000 [300:10:26<110:20:56,  5.55s/it]
 68%|██████▊   | 153365/225000 [300:10:30<105:44:20,  5.31s/it]
 68%|██████▊   | 153366/225000 [300:10:35<100:23:05,  5.04s/it]
 68%|██████▊   | 153367/225000 [300:10:43<119:18:11,  6.00s/it]
 68%|██████▊   | 153368/225000 [300:10:52<138:37:09,  6.97s/it]
 68%|██████▊   | 153369/225000 [300:10:56<122:27:08,  6.15s/it]
 68%|██████▊   | 153370/225000 [300:11:01<111:01:25,  5.58s/it]
 68%|██████▊   | 153371/225000 [300:11:05<104:18:37,  5.24s/it]
 68%|██████▊   | 153372/225000 [300:11:09<98:23:53,  4.95s/it] 
 68%|██████▊   | 153373/225000 [300:11:22<143:27:00,  7.21s/it]
 68%|██████▊   | 153374/225000 [300:11:26<122:47:15,  6.17s/it]
 68%|██████▊   | 153375/225000 [300:11:30<112:05:13,  5.63s/it]
 68%|██████▊   | 153376/225000 [300:11:34<104:11:28,  5.24s/it]
 68%|██████▊   | 153377/225000 [300:11:40<104:04:43,  5.23s/it]
 68%|██████▊   | 153378/225000 [300:11:45<106:33:22,  5.36s/it]
 68%|██████▊   | 153379/225000 [300:11:50<104:32:53,  5.26s/it]
 68%|██████▊   | 153380/225000 [300:12:02<141:55:03,  7.13s/it]
 68%|██████▊   | 153381/225000 [300:12:08<137:54:42,  6.93s/it]
 68%|██████▊   | 153382/225000 [300:12:12<117:32:05,  5.91s/it]
 68%|██████▊   | 153383/225000 [300:12:19<126:04:28,  6.34s/it]
 68%|██████▊   | 153384/225000 [300:12:25<121:08:51,  6.09s/it]
 68%|██████▊   | 153385/225000 [300:12:29<111:45:23,  5.62s/it]
 68%|██████▊   | 153386/225000 [300:12:34<106:25:51,  5.35s/it]
 68%|██████▊   | 153387/225000 [300:12:38<102:01:47,  5.13s/it]
 68%|██████▊   | 153388/225000 [300:12:43<100:20:36,  5.04s/it]
 68%|██████▊   | 153389/225000 [300:12:48<97:14:30,  4.89s/it] 
 68%|██████▊   | 153390/225000 [300:12:52<95:48:04,  4.82s/it]
 68%|██████▊   | 153391/225000 [300:13:04<133:52:02,  6.73s/it]
 68%|██████▊   | 153392/225000 [300:13:09<127:44:03,  6.42s/it]
 68%|██████▊   | 153393/225000 [300:13:13<113:11:28,  5.69s/it]
 68%|██████▊   | 153394/225000 [300:13:18<105:22:28,  5.30s/it]
 68%|██████▊   | 153395/225000 [300:13:22<101:21:37,  5.10s/it]
 68%|██████▊   | 153396/225000 [300:13:32<126:34:46,  6.36s/it]
 68%|██████▊   | 153397/225000 [300:13:40<138:15:04,  6.95s/it]
 68%|██████▊   | 153398/225000 [300:13:44<119:25:48,  6.00s/it]
 68%|██████▊   | 153399/225000 [300:13:51<128:41:49,  6.47s/it]
 68%|██████▊   | 153400/225000 [300:13:55<114:43:34,  5.77s/it]
                                                               
{'loss': '0.7803', 'grad_norm': '1.015', 'learning_rate': '1.25e-05', 'epoch': '0.6818', 'num_input_tokens_seen': 497329736, 'train_runtime': '1.081e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153400/225000 [300:13:55<114:43:34,  5.77s/it]
 68%|██████▊   | 153401/225000 [300:14:02<121:58:22,  6.13s/it]
 68%|██████▊   | 153402/225000 [300:14:06<107:19:24,  5.40s/it]
 68%|██████▊   | 153403/225000 [300:14:15<129:17:49,  6.50s/it]
 68%|██████▊   | 153404/225000 [300:14:20<117:10:04,  5.89s/it]
 68%|██████▊   | 153405/225000 [300:14:25<112:13:41,  5.64s/it]
 68%|██████▊   | 153406/225000 [300:14:29<106:32:12,  5.36s/it]
 68%|██████▊   | 153407/225000 [300:14:34<103:23:20,  5.20s/it]
 68%|██████▊   | 153408/225000 [300:14:39<98:53:28,  4.97s/it] 
 68%|██████▊   | 153409/225000 [300:14:44<98:53:56,  4.97s/it]
 68%|██████▊   | 153410/225000 [300:14:48<96:48:58,  4.87s/it]
 68%|██████▊   | 153411/225000 [300:15:00<138:36:35,  6.97s/it]
 68%|██████▊   | 153412/225000 [300:15:05<125:36:31,  6.32s/it]
 68%|██████▊   | 153413/225000 [300:15:09<109:07:01,  5.49s/it]
 68%|██████▊   | 153414/225000 [300:15:19<138:53:40,  6.98s/it]
 68%|██████▊   | 153415/225000 [300:15:22<116:34:52,  5.86s/it]
 68%|██████▊   | 153416/225000 [300:15:26<103:09:20,  5.19s/it]
 68%|██████▊   | 153417/225000 [300:15:30<97:55:09,  4.92s/it] 
 68%|██████▊   | 153418/225000 [300:15:41<131:53:41,  6.63s/it]
 68%|██████▊   | 153419/225000 [300:15:45<117:06:15,  5.89s/it]
 68%|██████▊   | 153420/225000 [300:15:49<106:01:38,  5.33s/it]
 68%|██████▊   | 153421/225000 [300:15:57<121:05:37,  6.09s/it]
 68%|██████▊   | 153422/225000 [300:16:03<121:10:47,  6.09s/it]
 68%|██████▊   | 153423/225000 [300:16:07<106:31:48,  5.36s/it]
 68%|██████▊   | 153424/225000 [300:16:15<122:59:16,  6.19s/it]
 68%|██████▊   | 153425/225000 [300:16:18<106:23:35,  5.35s/it]
 68%|██████▊   | 153426/225000 [300:16:22<96:47:16,  4.87s/it] 
 68%|██████▊   | 153427/225000 [300:16:31<124:15:43,  6.25s/it]
 68%|██████▊   | 153428/225000 [300:16:36<112:04:24,  5.64s/it]
 68%|██████▊   | 153429/225000 [300:16:41<111:08:46,  5.59s/it]
 68%|██████▊   | 153430/225000 [300:16:46<106:37:59,  5.36s/it]
 68%|██████▊   | 153431/225000 [300:16:51<105:25:54,  5.30s/it]
 68%|██████▊   | 153432/225000 [300:16:56<105:32:09,  5.31s/it]
 68%|██████▊   | 153433/225000 [300:17:02<107:14:30,  5.39s/it]
 68%|██████▊   | 153434/225000 [300:17:08<108:52:42,  5.48s/it]
 68%|██████▊   | 153435/225000 [300:17:13<107:28:07,  5.41s/it]
 68%|██████▊   | 153436/225000 [300:17:18<106:02:08,  5.33s/it]
 68%|██████▊   | 153437/225000 [300:17:23<106:49:31,  5.37s/it]
 68%|██████▊   | 153438/225000 [300:17:29<109:36:14,  5.51s/it]
 68%|██████▊   | 153439/225000 [300:17:35<109:09:17,  5.49s/it]
 68%|██████▊   | 153440/225000 [300:17:40<105:27:18,  5.31s/it]
 68%|██████▊   | 153441/225000 [300:17:48<123:12:32,  6.20s/it]
 68%|██████▊   | 153442/225000 [300:17:53<117:16:12,  5.90s/it]
 68%|██████▊   | 153443/225000 [300:17:58<114:07:44,  5.74s/it]
 68%|██████▊   | 153444/225000 [300:18:03<108:55:46,  5.48s/it]
 68%|██████▊   | 153445/225000 [300:18:08<106:40:11,  5.37s/it]
 68%|██████▊   | 153446/225000 [300:18:15<114:23:35,  5.76s/it]
 68%|██████▊   | 153447/225000 [300:18:22<121:47:02,  6.13s/it]
 68%|██████▊   | 153448/225000 [300:18:29<124:18:20,  6.25s/it]
 68%|██████▊   | 153449/225000 [300:18:34<118:24:46,  5.96s/it]
 68%|██████▊   | 153450/225000 [300:18:39<115:24:51,  5.81s/it]
                                                               
{'loss': '0.7972', 'grad_norm': '1.633', 'learning_rate': '1.249e-05', 'epoch': '0.682', 'num_input_tokens_seen': 497460024, 'train_runtime': '1.081e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153450/225000 [300:18:39<115:24:51,  5.81s/it]
 68%|██████▊   | 153451/225000 [300:18:45<114:03:02,  5.74s/it]
 68%|██████▊   | 153452/225000 [300:18:50<110:57:23,  5.58s/it]
 68%|██████▊   | 153453/225000 [300:18:55<109:11:00,  5.49s/it]
 68%|██████▊   | 153454/225000 [300:19:01<109:31:55,  5.51s/it]
 68%|██████▊   | 153455/225000 [300:19:14<152:52:07,  7.69s/it]
 68%|██████▊   | 153456/225000 [300:19:17<126:24:17,  6.36s/it]
 68%|██████▊   | 153457/225000 [300:19:22<119:32:53,  6.02s/it]
 68%|██████▊   | 153458/225000 [300:19:28<115:49:01,  5.83s/it]
 68%|██████▊   | 153459/225000 [300:19:33<114:16:08,  5.75s/it]
 68%|██████▊   | 153460/225000 [300:19:38<109:01:46,  5.49s/it]
 68%|██████▊   | 153461/225000 [300:19:46<125:23:42,  6.31s/it]
 68%|██████▊   | 153462/225000 [300:19:53<125:41:23,  6.33s/it]
 68%|██████▊   | 153463/225000 [300:19:58<121:46:48,  6.13s/it]
 68%|██████▊   | 153464/225000 [300:20:03<114:17:25,  5.75s/it]
 68%|██████▊   | 153465/225000 [300:20:09<114:16:52,  5.75s/it]
 68%|██████▊   | 153466/225000 [300:20:14<112:32:26,  5.66s/it]
 68%|██████▊   | 153467/225000 [300:20:20<112:11:25,  5.65s/it]
 68%|██████▊   | 153468/225000 [300:20:25<110:00:00,  5.54s/it]
 68%|██████▊   | 153469/225000 [300:20:32<114:22:46,  5.76s/it]
 68%|██████▊   | 153470/225000 [300:20:37<112:05:05,  5.64s/it]
 68%|██████▊   | 153471/225000 [300:20:42<110:13:31,  5.55s/it]
 68%|██████▊   | 153472/225000 [300:20:47<107:07:37,  5.39s/it]
 68%|██████▊   | 153473/225000 [300:20:53<106:43:09,  5.37s/it]
 68%|██████▊   | 153474/225000 [300:20:58<103:46:09,  5.22s/it]
 68%|██████▊   | 153475/225000 [300:21:03<105:00:36,  5.29s/it]
 68%|██████▊   | 153476/225000 [300:21:08<103:29:33,  5.21s/it]
 68%|██████▊   | 153477/225000 [300:21:13<105:19:16,  5.30s/it]
 68%|██████▊   | 153478/225000 [300:21:19<106:28:10,  5.36s/it]
 68%|██████▊   | 153479/225000 [300:21:25<109:07:13,  5.49s/it]
 68%|██████▊   | 153480/225000 [300:21:30<108:44:04,  5.47s/it]
 68%|██████▊   | 153481/225000 [300:21:38<123:50:13,  6.23s/it]
 68%|██████▊   | 153482/225000 [300:21:47<140:30:04,  7.07s/it]
 68%|██████▊   | 153483/225000 [300:21:51<118:47:09,  5.98s/it]
 68%|██████▊   | 153484/225000 [300:21:57<122:53:38,  6.19s/it]
 68%|██████▊   | 153485/225000 [300:22:01<107:09:27,  5.39s/it]
 68%|██████▊   | 153486/225000 [300:22:05<98:42:05,  4.97s/it] 
 68%|██████▊   | 153487/225000 [300:22:16<134:59:14,  6.80s/it]
 68%|██████▊   | 153488/225000 [300:22:22<133:18:36,  6.71s/it]
 68%|██████▊   | 153489/225000 [300:22:27<118:41:00,  5.97s/it]
 68%|██████▊   | 153490/225000 [300:22:30<105:05:35,  5.29s/it]
 68%|██████▊   | 153491/225000 [300:22:34<96:17:42,  4.85s/it] 
 68%|██████▊   | 153492/225000 [300:22:46<135:03:14,  6.80s/it]
 68%|██████▊   | 153493/225000 [300:22:49<113:59:10,  5.74s/it]
 68%|██████▊   | 153494/225000 [300:22:53<107:02:34,  5.39s/it]
 68%|██████▊   | 153495/225000 [300:22:58<104:13:07,  5.25s/it]
 68%|██████▊   | 153496/225000 [300:23:03<102:00:18,  5.14s/it]
 68%|██████▊   | 153497/225000 [300:23:08<99:19:45,  5.00s/it] 
 68%|██████▊   | 153498/225000 [300:23:12<96:41:56,  4.87s/it]
 68%|██████▊   | 153499/225000 [300:23:23<128:44:41,  6.48s/it]
 68%|██████▊   | 153500/225000 [300:23:30<132:12:21,  6.66s/it]
                                                               
{'loss': '0.7878', 'grad_norm': '0.9577', 'learning_rate': '1.247e-05', 'epoch': '0.6822', 'num_input_tokens_seen': 497590320, 'train_runtime': '1.081e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153500/225000 [300:23:30<132:12:21,  6.66s/it]
 68%|██████▊   | 153501/225000 [300:23:34<115:58:32,  5.84s/it]
 68%|██████▊   | 153502/225000 [300:23:39<110:25:42,  5.56s/it]
 68%|██████▊   | 153503/225000 [300:23:43<106:16:09,  5.35s/it]
 68%|██████▊   | 153504/225000 [300:23:48<103:35:01,  5.22s/it]
 68%|██████▊   | 153505/225000 [300:23:54<103:40:40,  5.22s/it]
 68%|██████▊   | 153506/225000 [300:23:58<101:43:32,  5.12s/it]
 68%|██████▊   | 153507/225000 [300:24:11<144:58:44,  7.30s/it]
 68%|██████▊   | 153508/225000 [300:24:16<133:15:59,  6.71s/it]
 68%|██████▊   | 153509/225000 [300:24:22<127:47:16,  6.43s/it]
 68%|██████▊   | 153510/225000 [300:24:27<119:59:22,  6.04s/it]
 68%|██████▊   | 153511/225000 [300:24:35<128:42:36,  6.48s/it]
 68%|██████▊   | 153512/225000 [300:24:39<117:22:24,  5.91s/it]
 68%|██████▊   | 153513/225000 [300:24:44<110:36:06,  5.57s/it]
 68%|██████▊   | 153514/225000 [300:24:49<109:56:28,  5.54s/it]
 68%|██████▊   | 153515/225000 [300:24:56<118:28:00,  5.97s/it]
 68%|██████▊   | 153516/225000 [300:25:01<109:30:31,  5.51s/it]
 68%|██████▊   | 153517/225000 [300:25:05<104:02:30,  5.24s/it]
 68%|██████▊   | 153518/225000 [300:25:09<96:30:17,  4.86s/it] 
 68%|██████▊   | 153519/225000 [300:25:16<104:48:34,  5.28s/it]
 68%|██████▊   | 153520/225000 [300:25:27<138:17:28,  6.96s/it]
 68%|██████▊   | 153521/225000 [300:25:30<117:00:47,  5.89s/it]
 68%|██████▊   | 153522/225000 [300:25:37<125:22:36,  6.31s/it]
 68%|██████▊   | 153523/225000 [300:25:41<110:54:17,  5.59s/it]
 68%|██████▊   | 153524/225000 [300:25:45<99:31:13,  5.01s/it] 
 68%|██████▊   | 153525/225000 [300:25:56<133:52:48,  6.74s/it]
 68%|██████▊   | 153526/225000 [300:25:59<114:05:05,  5.75s/it]
 68%|██████▊   | 153527/225000 [300:26:04<110:54:15,  5.59s/it]
 68%|██████▊   | 153528/225000 [300:26:09<108:19:26,  5.46s/it]
 68%|██████▊   | 153529/225000 [300:26:15<107:01:52,  5.39s/it]
 68%|██████▊   | 153530/225000 [300:26:23<124:41:31,  6.28s/it]
 68%|██████▊   | 153531/225000 [300:26:31<135:10:52,  6.81s/it]
 68%|██████▊   | 153532/225000 [300:26:36<125:46:26,  6.34s/it]
 68%|██████▊   | 153533/225000 [300:26:41<114:32:44,  5.77s/it]
 68%|██████▊   | 153534/225000 [300:26:45<106:15:13,  5.35s/it]
 68%|██████▊   | 153535/225000 [300:26:50<104:25:33,  5.26s/it]
 68%|██████▊   | 153536/225000 [300:26:55<101:11:30,  5.10s/it]
 68%|██████▊   | 153537/225000 [300:27:00<100:34:59,  5.07s/it]
 68%|██████▊   | 153538/225000 [300:27:05<100:50:34,  5.08s/it]
 68%|██████▊   | 153539/225000 [300:27:10<103:02:19,  5.19s/it]
 68%|██████▊   | 153540/225000 [300:27:15<101:05:03,  5.09s/it]
 68%|██████▊   | 153541/225000 [300:27:20<99:03:40,  4.99s/it] 
 68%|██████▊   | 153542/225000 [300:27:25<100:30:53,  5.06s/it]
 68%|██████▊   | 153543/225000 [300:27:31<101:42:44,  5.12s/it]
 68%|██████▊   | 153544/225000 [300:27:36<102:52:10,  5.18s/it]
 68%|██████▊   | 153545/225000 [300:27:49<149:36:30,  7.54s/it]
 68%|██████▊   | 153546/225000 [300:27:53<131:36:21,  6.63s/it]
 68%|██████▊   | 153547/225000 [300:27:58<120:48:56,  6.09s/it]
 68%|██████▊   | 153548/225000 [300:28:03<110:36:50,  5.57s/it]
 68%|██████▊   | 153549/225000 [300:28:10<124:19:58,  6.26s/it]
 68%|██████▊   | 153550/225000 [300:28:18<129:42:06,  6.54s/it]
                                                               
{'loss': '0.7817', 'grad_norm': '1.162', 'learning_rate': '1.245e-05', 'epoch': '0.6824', 'num_input_tokens_seen': 497721592, 'train_runtime': '1.082e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153550/225000 [300:28:18<129:42:06,  6.54s/it]
 68%|██████▊   | 153551/225000 [300:28:24<127:10:05,  6.41s/it]
 68%|██████▊   | 153552/225000 [300:28:30<124:33:52,  6.28s/it]
 68%|██████▊   | 153553/225000 [300:28:33<109:09:36,  5.50s/it]
 68%|██████▊   | 153554/225000 [300:28:41<124:09:35,  6.26s/it]
 68%|██████▊   | 153555/225000 [300:28:47<120:24:59,  6.07s/it]
 68%|██████▊   | 153556/225000 [300:28:51<107:02:44,  5.39s/it]
 68%|██████▊   | 153557/225000 [300:28:56<105:29:55,  5.32s/it]
 68%|██████▊   | 153558/225000 [300:29:01<102:42:39,  5.18s/it]
 68%|██████▊   | 153559/225000 [300:29:06<102:30:31,  5.17s/it]
 68%|██████▊   | 153560/225000 [300:29:11<101:59:14,  5.14s/it]
 68%|██████▊   | 153561/225000 [300:29:16<100:55:00,  5.09s/it]
 68%|██████▊   | 153562/225000 [300:29:21<98:46:28,  4.98s/it] 
 68%|██████▊   | 153563/225000 [300:29:26<102:06:17,  5.15s/it]
 68%|██████▊   | 153564/225000 [300:29:39<144:21:32,  7.27s/it]
 68%|██████▊   | 153565/225000 [300:29:46<146:39:05,  7.39s/it]
 68%|██████▊   | 153566/225000 [300:29:49<121:06:51,  6.10s/it]
 68%|██████▊   | 153567/225000 [300:29:53<108:14:06,  5.45s/it]
 68%|██████▊   | 153568/225000 [300:30:05<142:47:31,  7.20s/it]
 68%|██████▊   | 153569/225000 [300:30:09<126:21:22,  6.37s/it]
 68%|██████▊   | 153570/225000 [300:30:13<112:06:45,  5.65s/it]
 68%|██████▊   | 153571/225000 [300:30:17<102:38:22,  5.17s/it]
 68%|██████▊   | 153572/225000 [300:30:21<98:24:57,  4.96s/it] 
 68%|██████▊   | 153573/225000 [300:30:28<106:09:30,  5.35s/it]
 68%|██████▊   | 153574/225000 [300:30:38<137:49:30,  6.95s/it]
 68%|██████▊   | 153575/225000 [300:30:42<116:49:16,  5.89s/it]
 68%|██████▊   | 153576/225000 [300:30:48<116:44:36,  5.88s/it]
 68%|██████▊   | 153577/225000 [300:30:54<118:22:23,  5.97s/it]
 68%|██████▊   | 153578/225000 [300:30:58<105:49:40,  5.33s/it]
 68%|██████▊   | 153579/225000 [300:31:01<96:29:45,  4.86s/it] 
 68%|██████▊   | 153580/225000 [300:31:05<91:28:39,  4.61s/it]
 68%|██████▊   | 153581/225000 [300:31:10<92:38:54,  4.67s/it]
 68%|██████▊   | 153582/225000 [300:31:15<93:44:41,  4.73s/it]
 68%|██████▊   | 153583/225000 [300:31:20<94:55:31,  4.79s/it]
 68%|██████▊   | 153584/225000 [300:31:30<124:01:29,  6.25s/it]
 68%|██████▊   | 153585/225000 [300:31:38<138:40:11,  6.99s/it]
 68%|██████▊   | 153586/225000 [300:31:42<117:12:06,  5.91s/it]
 68%|██████▊   | 153587/225000 [300:31:50<129:52:23,  6.55s/it]
 68%|██████▊   | 153588/225000 [300:31:54<112:41:46,  5.68s/it]
 68%|██████▊   | 153589/225000 [300:32:02<129:37:47,  6.53s/it]
 68%|██████▊   | 153590/225000 [300:32:09<130:59:57,  6.60s/it]
 68%|██████▊   | 153591/225000 [300:32:12<111:01:36,  5.60s/it]
 68%|██████▊   | 153592/225000 [300:32:19<118:21:32,  5.97s/it]
 68%|██████▊   | 153593/225000 [300:32:24<112:19:34,  5.66s/it]
 68%|██████▊   | 153594/225000 [300:32:29<109:21:40,  5.51s/it]
 68%|██████▊   | 153595/225000 [300:32:34<105:46:40,  5.33s/it]
 68%|██████▊   | 153596/225000 [300:32:39<104:49:02,  5.28s/it]
 68%|██████▊   | 153597/225000 [300:32:44<103:12:03,  5.20s/it]
 68%|██████▊   | 153598/225000 [300:32:49<102:20:12,  5.16s/it]
 68%|██████▊   | 153599/225000 [300:32:55<103:52:34,  5.24s/it]
 68%|██████▊   | 153600/225000 [300:33:00<102:26:07,  5.16s/it]
                                                               
{'loss': '0.7723', 'grad_norm': '1.147', 'learning_rate': '1.244e-05', 'epoch': '0.6827', 'num_input_tokens_seen': 497850816, 'train_runtime': '1.082e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153600/225000 [300:33:00<102:26:07,  5.16s/it]
 68%|██████▊   | 153601/225000 [300:33:05<103:33:39,  5.22s/it]
 68%|██████▊   | 153602/225000 [300:33:10<101:26:01,  5.11s/it]
 68%|██████▊   | 153603/225000 [300:33:15<102:19:04,  5.16s/it]
 68%|██████▊   | 153604/225000 [300:33:27<143:55:15,  7.26s/it]
 68%|██████▊   | 153605/225000 [300:33:31<125:34:49,  6.33s/it]
 68%|██████▊   | 153606/225000 [300:33:36<112:27:48,  5.67s/it]
 68%|██████▊   | 153607/225000 [300:33:39<100:11:52,  5.05s/it]
 68%|██████▊   | 153608/225000 [300:33:44<97:50:31,  4.93s/it] 
 68%|██████▊   | 153609/225000 [300:33:49<98:07:40,  4.95s/it]
 68%|██████▊   | 153610/225000 [300:33:54<99:51:30,  5.04s/it]
 68%|██████▊   | 153611/225000 [300:34:06<142:12:22,  7.17s/it]
 68%|██████▊   | 153612/225000 [300:34:11<130:51:21,  6.60s/it]
 68%|██████▊   | 153613/225000 [300:34:15<113:47:51,  5.74s/it]
 68%|██████▊   | 153614/225000 [300:34:19<104:37:05,  5.28s/it]
 68%|██████▊   | 153615/225000 [300:34:25<104:35:04,  5.27s/it]
 68%|██████▊   | 153616/225000 [300:34:36<138:01:20,  6.96s/it]
 68%|██████▊   | 153617/225000 [300:34:39<118:08:19,  5.96s/it]
 68%|██████▊   | 153618/225000 [300:34:46<121:02:15,  6.10s/it]
 68%|██████▊   | 153619/225000 [300:34:50<113:42:27,  5.73s/it]
 68%|██████▊   | 153620/225000 [300:34:54<102:30:39,  5.17s/it]
 68%|██████▊   | 153621/225000 [300:35:02<114:38:32,  5.78s/it]
 68%|██████▊   | 153622/225000 [300:35:09<126:42:16,  6.39s/it]
 68%|██████▊   | 153623/225000 [300:35:16<126:31:40,  6.38s/it]
 68%|██████▊   | 153624/225000 [300:35:19<110:57:45,  5.60s/it]
 68%|██████▊   | 153625/225000 [300:35:24<104:04:40,  5.25s/it]
 68%|██████▊   | 153626/225000 [300:35:28<95:01:01,  4.79s/it] 
 68%|██████▊   | 153627/225000 [300:35:39<131:27:37,  6.63s/it]
 68%|██████▊   | 153628/225000 [300:35:42<112:24:41,  5.67s/it]
 68%|██████▊   | 153629/225000 [300:35:49<118:12:23,  5.96s/it]
 68%|██████▊   | 153630/225000 [300:35:52<102:26:44,  5.17s/it]
 68%|██████▊   | 153631/225000 [300:36:00<118:58:44,  6.00s/it]
 68%|██████▊   | 153632/225000 [300:36:04<108:41:01,  5.48s/it]
 68%|██████▊   | 153633/225000 [300:36:08<100:45:48,  5.08s/it]
 68%|██████▊   | 153634/225000 [300:36:14<102:47:03,  5.18s/it]
 68%|██████▊   | 153635/225000 [300:36:24<135:04:30,  6.81s/it]
 68%|██████▊   | 153636/225000 [300:36:29<121:15:33,  6.12s/it]
 68%|██████▊   | 153637/225000 [300:36:34<113:38:03,  5.73s/it]
 68%|██████▊   | 153638/225000 [300:36:38<104:14:08,  5.26s/it]
 68%|██████▊   | 153639/225000 [300:36:42<99:28:56,  5.02s/it] 
 68%|██████▊   | 153640/225000 [300:36:47<96:39:08,  4.88s/it]
 68%|██████▊   | 153641/225000 [300:36:59<137:21:47,  6.93s/it]
 68%|██████▊   | 153642/225000 [300:37:03<124:21:51,  6.27s/it]
 68%|██████▊   | 153643/225000 [300:37:08<116:23:06,  5.87s/it]
 68%|██████▊   | 153644/225000 [300:37:13<111:12:51,  5.61s/it]
 68%|██████▊   | 153645/225000 [300:37:18<106:13:08,  5.36s/it]
 68%|██████▊   | 153646/225000 [300:37:23<104:36:02,  5.28s/it]
 68%|██████▊   | 153647/225000 [300:37:28<101:54:11,  5.14s/it]
 68%|██████▊   | 153648/225000 [300:37:33<102:44:21,  5.18s/it]
 68%|██████▊   | 153649/225000 [300:37:46<148:31:04,  7.49s/it]
 68%|██████▊   | 153650/225000 [300:37:51<134:48:37,  6.80s/it]
                                                               
{'loss': '0.7853', 'grad_norm': '1.058', 'learning_rate': '1.242e-05', 'epoch': '0.6829', 'num_input_tokens_seen': 497981624, 'train_runtime': '1.082e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153650/225000 [300:37:51<134:48:37,  6.80s/it]
 68%|██████▊   | 153651/225000 [300:37:55<117:15:56,  5.92s/it]
 68%|██████▊   | 153652/225000 [300:38:03<127:10:11,  6.42s/it]
 68%|██████▊   | 153653/225000 [300:38:06<110:59:50,  5.60s/it]
 68%|██████▊   | 153654/225000 [300:38:12<110:43:40,  5.59s/it]
 68%|██████▊   | 153655/225000 [300:38:19<121:03:53,  6.11s/it]
 68%|██████▊   | 153656/225000 [300:38:24<114:06:07,  5.76s/it]
 68%|██████▊   | 153657/225000 [300:38:29<109:36:27,  5.53s/it]
 68%|██████▊   | 153658/225000 [300:38:34<108:11:30,  5.46s/it]
 68%|██████▊   | 153659/225000 [300:38:40<107:36:47,  5.43s/it]
 68%|██████▊   | 153660/225000 [300:38:45<106:26:16,  5.37s/it]
 68%|██████▊   | 153661/225000 [300:38:50<105:36:00,  5.33s/it]
 68%|██████▊   | 153662/225000 [300:38:55<103:33:07,  5.23s/it]
 68%|██████▊   | 153663/225000 [300:39:00<101:38:44,  5.13s/it]
 68%|██████▊   | 153664/225000 [300:39:05<102:02:51,  5.15s/it]
 68%|██████▊   | 153665/225000 [300:39:11<102:01:48,  5.15s/it]
 68%|██████▊   | 153666/225000 [300:39:16<102:42:02,  5.18s/it]
 68%|██████▊   | 153667/225000 [300:39:21<104:50:02,  5.29s/it]
 68%|██████▊   | 153668/225000 [300:39:26<102:52:29,  5.19s/it]
 68%|██████▊   | 153669/225000 [300:39:31<101:02:41,  5.10s/it]
 68%|██████▊   | 153670/225000 [300:39:39<119:09:42,  6.01s/it]
 68%|██████▊   | 153671/225000 [300:39:52<157:48:53,  7.96s/it]
 68%|██████▊   | 153672/225000 [300:39:55<132:02:28,  6.66s/it]
 68%|██████▊   | 153673/225000 [300:40:00<118:56:10,  6.00s/it]
 68%|██████▊   | 153674/225000 [300:40:05<112:47:05,  5.69s/it]
 68%|██████▊   | 153675/225000 [300:40:09<104:18:41,  5.26s/it]
 68%|██████▊   | 153676/225000 [300:40:14<100:24:16,  5.07s/it]
 68%|██████▊   | 153677/225000 [300:40:19<100:14:01,  5.06s/it]
 68%|██████▊   | 153678/225000 [300:40:24<103:24:32,  5.22s/it]
 68%|██████▊   | 153679/225000 [300:40:30<102:57:20,  5.20s/it]
 68%|██████▊   | 153680/225000 [300:40:35<101:35:30,  5.13s/it]
 68%|██████▊   | 153681/225000 [300:40:39<98:45:24,  4.98s/it] 
 68%|██████▊   | 153682/225000 [300:40:45<101:38:13,  5.13s/it]
 68%|██████▊   | 153683/225000 [300:40:56<136:56:30,  6.91s/it]
 68%|██████▊   | 153684/225000 [300:41:05<152:42:27,  7.71s/it]
 68%|██████▊   | 153685/225000 [300:41:09<129:53:57,  6.56s/it]
 68%|██████▊   | 153686/225000 [300:41:13<114:21:11,  5.77s/it]
 68%|██████▊   | 153687/225000 [300:41:18<108:44:03,  5.49s/it]
 68%|██████▊   | 153688/225000 [300:41:24<109:36:42,  5.53s/it]
 68%|██████▊   | 153689/225000 [300:41:34<136:01:21,  6.87s/it]
 68%|██████▊   | 153690/225000 [300:41:38<119:03:20,  6.01s/it]
 68%|██████▊   | 153691/225000 [300:41:43<114:43:00,  5.79s/it]
 68%|██████▊   | 153692/225000 [300:41:50<125:23:52,  6.33s/it]
 68%|██████▊   | 153693/225000 [300:41:55<113:24:14,  5.73s/it]
 68%|██████▊   | 153694/225000 [300:42:02<120:14:48,  6.07s/it]
 68%|██████▊   | 153695/225000 [300:42:06<107:29:02,  5.43s/it]
 68%|██████▊   | 153696/225000 [300:42:14<124:48:41,  6.30s/it]
 68%|██████▊   | 153697/225000 [300:42:21<129:40:12,  6.55s/it]
 68%|██████▊   | 153698/225000 [300:42:25<112:15:09,  5.67s/it]
 68%|██████▊   | 153699/225000 [300:42:32<124:14:06,  6.27s/it]
 68%|██████▊   | 153700/225000 [300:42:37<113:28:41,  5.73s/it]
                                                               
{'loss': '0.7828', 'grad_norm': '1.203', 'learning_rate': '1.241e-05', 'epoch': '0.6831', 'num_input_tokens_seen': 498111840, 'train_runtime': '1.083e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153700/225000 [300:42:37<113:28:41,  5.73s/it]
 68%|██████▊   | 153701/225000 [300:42:40<100:50:59,  5.09s/it]
 68%|██████▊   | 153702/225000 [300:42:50<128:57:10,  6.51s/it]
 68%|██████▊   | 153703/225000 [300:42:54<110:31:11,  5.58s/it]
 68%|██████▊   | 153704/225000 [300:42:58<104:19:39,  5.27s/it]
 68%|██████▊   | 153705/225000 [300:43:03<101:31:46,  5.13s/it]
 68%|██████▊   | 153706/225000 [300:43:08<99:19:22,  5.02s/it] 
 68%|██████▊   | 153707/225000 [300:43:13<98:36:57,  4.98s/it]
 68%|██████▊   | 153708/225000 [300:43:17<97:36:14,  4.93s/it]
 68%|██████▊   | 153709/225000 [300:43:22<95:46:47,  4.84s/it]
 68%|██████▊   | 153710/225000 [300:43:28<102:01:32,  5.15s/it]
 68%|██████▊   | 153711/225000 [300:43:39<136:40:22,  6.90s/it]
 68%|██████▊   | 153712/225000 [300:43:42<116:16:59,  5.87s/it]
 68%|██████▊   | 153713/225000 [300:43:48<117:28:06,  5.93s/it]
 68%|██████▊   | 153714/225000 [300:43:57<132:44:58,  6.70s/it]
 68%|██████▊   | 153715/225000 [300:44:01<115:19:22,  5.82s/it]
 68%|██████▊   | 153716/225000 [300:44:08<123:58:57,  6.26s/it]
 68%|██████▊   | 153717/225000 [300:44:12<107:44:31,  5.44s/it]
 68%|██████▊   | 153718/225000 [300:44:19<119:12:00,  6.02s/it]
 68%|██████▊   | 153719/225000 [300:44:23<110:02:49,  5.56s/it]
 68%|██████▊   | 153720/225000 [300:44:27<98:21:50,  4.97s/it] 
 68%|██████▊   | 153721/225000 [300:44:31<93:15:16,  4.71s/it]
 68%|██████▊   | 153722/225000 [300:44:38<109:07:15,  5.51s/it]
 68%|██████▊   | 153723/225000 [300:44:47<126:55:55,  6.41s/it]
 68%|██████▊   | 153724/225000 [300:44:51<110:17:29,  5.57s/it]
 68%|██████▊   | 153725/225000 [300:44:55<102:04:56,  5.16s/it]
 68%|██████▊   | 153726/225000 [300:45:03<118:38:50,  5.99s/it]
 68%|██████▊   | 153727/225000 [300:45:06<105:02:25,  5.31s/it]
 68%|██████▊   | 153728/225000 [300:45:15<122:56:11,  6.21s/it]
 68%|██████▊   | 153729/225000 [300:45:18<106:51:01,  5.40s/it]
 68%|██████▊   | 153730/225000 [300:45:23<105:50:19,  5.35s/it]
 68%|██████▊   | 153731/225000 [300:45:32<126:48:21,  6.41s/it]
 68%|██████▊   | 153732/225000 [300:45:37<118:20:01,  5.98s/it]
 68%|██████▊   | 153733/225000 [300:45:41<104:38:55,  5.29s/it]
 68%|██████▊   | 153734/225000 [300:45:50<127:49:18,  6.46s/it]
 68%|██████▊   | 153735/225000 [300:45:54<110:55:05,  5.60s/it]
 68%|██████▊   | 153736/225000 [300:46:00<112:18:56,  5.67s/it]
 68%|██████▊   | 153737/225000 [300:46:07<122:27:55,  6.19s/it]
 68%|██████▊   | 153738/225000 [300:46:11<109:38:27,  5.54s/it]
 68%|██████▊   | 153739/225000 [300:46:18<120:51:44,  6.11s/it]
 68%|██████▊   | 153740/225000 [300:46:25<124:53:38,  6.31s/it]
 68%|██████▊   | 153741/225000 [300:46:29<108:13:19,  5.47s/it]
 68%|██████▊   | 153742/225000 [300:46:33<98:20:43,  4.97s/it] 
 68%|██████▊   | 153743/225000 [300:46:42<123:10:49,  6.22s/it]
 68%|██████▊   | 153744/225000 [300:46:45<108:06:20,  5.46s/it]
 68%|██████▊   | 153745/225000 [300:46:51<109:30:37,  5.53s/it]
 68%|██████▊   | 153746/225000 [300:46:58<119:42:52,  6.05s/it]
 68%|██████▊   | 153747/225000 [300:47:02<104:03:41,  5.26s/it]
 68%|██████▊   | 153748/225000 [300:47:10<123:49:33,  6.26s/it]
 68%|██████▊   | 153749/225000 [300:47:14<109:49:35,  5.55s/it]
 68%|██████▊   | 153750/225000 [300:47:22<124:54:04,  6.31s/it]
                                                               
{'loss': '0.784', 'grad_norm': '1.005', 'learning_rate': '1.239e-05', 'epoch': '0.6833', 'num_input_tokens_seen': 498241512, 'train_runtime': '1.083e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153750/225000 [300:47:22<124:54:04,  6.31s/it]
 68%|██████▊   | 153751/225000 [300:47:26<111:59:58,  5.66s/it]
 68%|██████▊   | 153752/225000 [300:47:30<97:45:50,  4.94s/it] 
 68%|██████▊   | 153753/225000 [300:47:39<121:36:23,  6.14s/it]
 68%|██████▊   | 153754/225000 [300:47:42<105:47:29,  5.35s/it]
 68%|██████▊   | 153755/225000 [300:47:48<111:34:07,  5.64s/it]
 68%|██████▊   | 153756/225000 [300:47:53<106:25:33,  5.38s/it]
 68%|██████▊   | 153757/225000 [300:47:57<97:53:06,  4.95s/it] 
 68%|██████▊   | 153758/225000 [300:48:02<96:50:36,  4.89s/it]
 68%|██████▊   | 153759/225000 [300:48:07<94:52:47,  4.79s/it]
 68%|██████▊   | 153760/225000 [300:48:12<97:31:52,  4.93s/it]
 68%|██████▊   | 153761/225000 [300:48:23<134:41:30,  6.81s/it]
 68%|██████▊   | 153762/225000 [300:48:26<115:15:09,  5.82s/it]
 68%|██████▊   | 153763/225000 [300:48:35<128:53:07,  6.51s/it]
 68%|██████▊   | 153764/225000 [300:48:39<115:23:13,  5.83s/it]
 68%|██████▊   | 153765/225000 [300:48:44<108:26:52,  5.48s/it]
 68%|██████▊   | 153766/225000 [300:48:48<105:01:36,  5.31s/it]
 68%|██████▊   | 153767/225000 [300:48:53<99:55:24,  5.05s/it] 
 68%|██████▊   | 153768/225000 [300:49:04<134:39:36,  6.81s/it]
 68%|██████▊   | 153769/225000 [300:49:10<133:26:26,  6.74s/it]
 68%|██████▊   | 153770/225000 [300:49:15<122:47:24,  6.21s/it]
 68%|██████▊   | 153771/225000 [300:49:21<119:41:32,  6.05s/it]
 68%|██████▊   | 153772/225000 [300:49:26<114:19:43,  5.78s/it]
 68%|██████▊   | 153773/225000 [300:49:31<111:02:01,  5.61s/it]
 68%|██████▊   | 153774/225000 [300:49:36<104:10:30,  5.27s/it]
 68%|██████▊   | 153775/225000 [300:49:40<99:06:41,  5.01s/it] 
 68%|██████▊   | 153776/225000 [300:49:45<98:47:03,  4.99s/it]
 68%|██████▊   | 153777/225000 [300:49:50<97:08:01,  4.91s/it]
 68%|██████▊   | 153778/225000 [300:49:55<98:36:48,  4.98s/it]
 68%|██████▊   | 153779/225000 [300:50:00<96:41:19,  4.89s/it]
 68%|██████▊   | 153780/225000 [300:50:05<96:47:16,  4.89s/it]
 68%|██████▊   | 153781/225000 [300:50:15<127:15:20,  6.43s/it]
 68%|██████▊   | 153782/225000 [300:50:23<140:41:41,  7.11s/it]
 68%|██████▊   | 153783/225000 [300:50:27<119:10:20,  6.02s/it]
 68%|██████▊   | 153784/225000 [300:50:36<134:45:37,  6.81s/it]
 68%|██████▊   | 153785/225000 [300:50:39<114:05:36,  5.77s/it]
 68%|██████▊   | 153786/225000 [300:50:47<130:44:16,  6.61s/it]
 68%|██████▊   | 153787/225000 [300:50:51<111:54:18,  5.66s/it]
 68%|██████▊   | 153788/225000 [300:50:55<102:37:32,  5.19s/it]
 68%|██████▊   | 153789/225000 [300:51:00<103:29:11,  5.23s/it]
 68%|██████▊   | 153790/225000 [300:51:08<116:38:05,  5.90s/it]
 68%|██████▊   | 153791/225000 [300:51:12<105:02:16,  5.31s/it]
 68%|██████▊   | 153792/225000 [300:51:16<97:20:09,  4.92s/it] 
 68%|██████▊   | 153793/225000 [300:51:20<91:21:59,  4.62s/it]
 68%|██████▊   | 153794/225000 [300:51:32<134:38:17,  6.81s/it]
 68%|██████▊   | 153795/225000 [300:51:36<118:18:25,  5.98s/it]
 68%|██████▊   | 153796/225000 [300:51:39<106:09:57,  5.37s/it]
 68%|██████▊   | 153797/225000 [300:51:44<98:59:13,  5.00s/it] 
 68%|██████▊   | 153798/225000 [300:51:54<130:16:34,  6.59s/it]
 68%|██████▊   | 153799/225000 [300:51:58<113:20:52,  5.73s/it]
 68%|██████▊   | 153800/225000 [300:52:04<116:57:40,  5.91s/it]
                                                               
{'loss': '0.7838', 'grad_norm': '1.153', 'learning_rate': '1.238e-05', 'epoch': '0.6836', 'num_input_tokens_seen': 498370232, 'train_runtime': '1.083e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153800/225000 [300:52:04<116:57:40,  5.91s/it]
 68%|██████▊   | 153801/225000 [300:52:09<111:15:18,  5.63s/it]
 68%|██████▊   | 153802/225000 [300:52:16<119:28:10,  6.04s/it]
 68%|██████▊   | 153803/225000 [300:52:21<113:54:39,  5.76s/it]
 68%|██████▊   | 153804/225000 [300:52:25<103:11:28,  5.22s/it]
 68%|██████▊   | 153805/225000 [300:52:29<95:41:38,  4.84s/it] 
 68%|██████▊   | 153806/225000 [300:52:38<117:38:34,  5.95s/it]
 68%|██████▊   | 153807/225000 [300:52:45<124:55:47,  6.32s/it]
 68%|██████▊   | 153808/225000 [300:52:48<106:37:25,  5.39s/it]
 68%|██████▊   | 153809/225000 [300:52:52<99:42:57,  5.04s/it] 
 68%|██████▊   | 153810/225000 [300:53:02<126:59:39,  6.42s/it]
 68%|██████▊   | 153811/225000 [300:53:06<114:47:17,  5.80s/it]
 68%|██████▊   | 153812/225000 [300:53:10<104:08:34,  5.27s/it]
 68%|██████▊   | 153813/225000 [300:53:15<100:02:17,  5.06s/it]
 68%|██████▊   | 153814/225000 [300:53:19<94:56:08,  4.80s/it] 
 68%|██████▊   | 153815/225000 [300:53:28<122:16:19,  6.18s/it]
 68%|██████▊   | 153816/225000 [300:53:35<124:42:40,  6.31s/it]
 68%|██████▊   | 153817/225000 [300:53:38<107:34:03,  5.44s/it]
 68%|██████▊   | 153818/225000 [300:53:42<98:21:34,  4.97s/it] 
 68%|██████▊   | 153819/225000 [300:53:47<99:01:16,  5.01s/it]
 68%|██████▊   | 153820/225000 [300:53:52<99:43:24,  5.04s/it]
 68%|██████▊   | 153821/225000 [300:53:57<98:11:17,  4.97s/it]
 68%|██████▊   | 153822/225000 [300:54:02<99:21:19,  5.03s/it]
 68%|██████▊   | 153823/225000 [300:54:07<98:32:23,  4.98s/it]
 68%|██████▊   | 153824/225000 [300:54:12<96:55:28,  4.90s/it]
 68%|██████▊   | 153825/225000 [300:54:16<93:37:07,  4.74s/it]
 68%|██████▊   | 153826/225000 [300:54:28<132:41:01,  6.71s/it]
 68%|██████▊   | 153827/225000 [300:54:35<138:30:04,  7.01s/it]
 68%|██████▊   | 153828/225000 [300:54:39<117:25:44,  5.94s/it]
 68%|██████▊   | 153829/225000 [300:54:48<137:20:45,  6.95s/it]
 68%|██████▊   | 153830/225000 [300:54:52<119:56:01,  6.07s/it]
 68%|██████▊   | 153831/225000 [300:54:58<120:29:51,  6.10s/it]
 68%|██████▊   | 153832/225000 [300:55:02<106:08:57,  5.37s/it]
 68%|██████▊   | 153833/225000 [300:55:09<115:30:08,  5.84s/it]
 68%|██████▊   | 153834/225000 [300:55:16<120:57:31,  6.12s/it]
 68%|██████▊   | 153835/225000 [300:55:20<107:42:49,  5.45s/it]
 68%|██████▊   | 153836/225000 [300:55:28<124:49:29,  6.31s/it]
 68%|██████▊   | 153837/225000 [300:55:33<115:13:36,  5.83s/it]
 68%|██████▊   | 153838/225000 [300:55:38<114:45:08,  5.81s/it]
 68%|██████▊   | 153839/225000 [300:55:44<116:34:02,  5.90s/it]
 68%|██████▊   | 153840/225000 [300:55:48<104:56:49,  5.31s/it]
 68%|██████▊   | 153841/225000 [300:55:53<100:54:46,  5.11s/it]
 68%|██████▊   | 153842/225000 [300:55:58<97:50:07,  4.95s/it] 
 68%|██████▊   | 153843/225000 [300:56:02<96:28:27,  4.88s/it]
 68%|██████▊   | 153844/225000 [300:56:07<94:36:27,  4.79s/it]
 68%|██████▊   | 153845/225000 [300:56:12<96:50:41,  4.90s/it]
 68%|██████▊   | 153846/225000 [300:56:18<104:02:03,  5.26s/it]
 68%|██████▊   | 153847/225000 [300:56:29<139:32:45,  7.06s/it]
 68%|██████▊   | 153848/225000 [300:56:35<132:02:33,  6.68s/it]
 68%|██████▊   | 153849/225000 [300:56:39<113:54:36,  5.76s/it]
 68%|██████▊   | 153850/225000 [300:56:48<136:02:37,  6.88s/it]
                                                               
{'loss': '0.7813', 'grad_norm': '1.27', 'learning_rate': '1.236e-05', 'epoch': '0.6838', 'num_input_tokens_seen': 498500072, 'train_runtime': '1.083e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153850/225000 [300:56:48<136:02:37,  6.88s/it]
 68%|██████▊   | 153851/225000 [300:56:54<130:02:22,  6.58s/it]
 68%|██████▊   | 153852/225000 [300:56:59<117:49:51,  5.96s/it]
 68%|██████▊   | 153853/225000 [300:57:03<107:04:09,  5.42s/it]
 68%|██████▊   | 153854/225000 [300:57:08<103:22:20,  5.23s/it]
 68%|██████▊   | 153855/225000 [300:57:12<99:12:48,  5.02s/it] 
 68%|██████▊   | 153856/225000 [300:57:17<95:51:24,  4.85s/it]
 68%|██████▊   | 153857/225000 [300:57:22<96:18:31,  4.87s/it]
 68%|██████▊   | 153858/225000 [300:57:26<95:00:18,  4.81s/it]
 68%|██████▊   | 153859/225000 [300:57:37<131:12:33,  6.64s/it]
 68%|██████▊   | 153860/225000 [300:57:43<123:58:28,  6.27s/it]
 68%|██████▊   | 153861/225000 [300:57:50<131:24:52,  6.65s/it]
 68%|██████▊   | 153862/225000 [300:57:54<115:44:36,  5.86s/it]
 68%|██████▊   | 153863/225000 [300:57:58<102:15:58,  5.18s/it]
 68%|██████▊   | 153864/225000 [300:58:06<120:35:51,  6.10s/it]
 68%|██████▊   | 153865/225000 [300:58:12<121:07:14,  6.13s/it]
 68%|██████▊   | 153866/225000 [300:58:19<123:32:53,  6.25s/it]
 68%|██████▊   | 153867/225000 [300:58:23<109:31:39,  5.54s/it]
 68%|██████▊   | 153868/225000 [300:58:27<102:52:41,  5.21s/it]
 68%|██████▊   | 153869/225000 [300:58:31<97:13:21,  4.92s/it] 
 68%|██████▊   | 153870/225000 [300:58:35<91:35:28,  4.64s/it]
 68%|██████▊   | 153871/225000 [300:58:40<93:07:32,  4.71s/it]
 68%|██████▊   | 153872/225000 [300:58:44<90:59:58,  4.61s/it]
 68%|██████▊   | 153873/225000 [300:58:52<108:59:08,  5.52s/it]
 68%|██████▊   | 153874/225000 [300:59:03<141:29:18,  7.16s/it]
 68%|██████▊   | 153875/225000 [300:59:07<124:21:32,  6.29s/it]
 68%|██████▊   | 153876/225000 [300:59:11<109:49:56,  5.56s/it]
 68%|██████▊   | 153877/225000 [300:59:20<126:13:00,  6.39s/it]
 68%|██████▊   | 153878/225000 [300:59:23<108:58:01,  5.52s/it]
 68%|██████▊   | 153879/225000 [300:59:27<102:42:43,  5.20s/it]
 68%|██████▊   | 153880/225000 [300:59:32<101:02:02,  5.11s/it]
 68%|██████▊   | 153881/225000 [300:59:38<103:09:20,  5.22s/it]
 68%|██████▊   | 153882/225000 [300:59:43<101:33:45,  5.14s/it]
 68%|██████▊   | 153883/225000 [300:59:48<102:21:04,  5.18s/it]
 68%|██████▊   | 153884/225000 [300:59:58<130:11:40,  6.59s/it]
 68%|██████▊   | 153885/225000 [301:00:06<136:25:17,  6.91s/it]
 68%|██████▊   | 153886/225000 [301:00:09<118:03:10,  5.98s/it]
 68%|██████▊   | 153887/225000 [301:00:18<136:11:50,  6.89s/it]
 68%|██████▊   | 153888/225000 [301:00:22<115:01:02,  5.82s/it]
 68%|██████▊   | 153889/225000 [301:00:32<139:56:51,  7.08s/it]
 68%|██████▊   | 153890/225000 [301:00:37<129:18:17,  6.55s/it]
 68%|██████▊   | 153891/225000 [301:00:43<122:28:51,  6.20s/it]
 68%|██████▊   | 153892/225000 [301:00:49<125:59:34,  6.38s/it]
 68%|██████▊   | 153893/225000 [301:00:53<109:00:11,  5.52s/it]
 68%|██████▊   | 153894/225000 [301:01:00<119:08:00,  6.03s/it]
 68%|██████▊   | 153895/225000 [301:01:06<116:16:47,  5.89s/it]
 68%|██████▊   | 153896/225000 [301:01:13<125:25:24,  6.35s/it]
 68%|██████▊   | 153897/225000 [301:01:16<107:53:24,  5.46s/it]
 68%|██████▊   | 153898/225000 [301:01:25<126:18:53,  6.40s/it]
 68%|██████▊   | 153899/225000 [301:01:29<109:28:15,  5.54s/it]
 68%|██████▊   | 153900/225000 [301:01:34<108:28:37,  5.49s/it]
                                                               
{'loss': '0.7906', 'grad_norm': '1.002', 'learning_rate': '1.234e-05', 'epoch': '0.684', 'num_input_tokens_seen': 498630392, 'train_runtime': '1.084e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153900/225000 [301:01:34<108:28:37,  5.49s/it]
 68%|██████▊   | 153901/225000 [301:01:39<108:11:32,  5.48s/it]
 68%|██████▊   | 153902/225000 [301:01:47<118:20:42,  5.99s/it]
 68%|██████▊   | 153903/225000 [301:01:53<118:41:49,  6.01s/it]
 68%|██████▊   | 153904/225000 [301:01:57<106:53:49,  5.41s/it]
 68%|██████▊   | 153905/225000 [301:02:01<97:49:33,  4.95s/it] 
 68%|██████▊   | 153906/225000 [301:02:10<125:00:22,  6.33s/it]
 68%|██████▊   | 153907/225000 [301:02:13<106:13:38,  5.38s/it]
 68%|██████▊   | 153908/225000 [301:02:22<128:13:47,  6.49s/it]
 68%|██████▊   | 153909/225000 [301:02:27<116:59:03,  5.92s/it]
 68%|██████▊   | 153910/225000 [301:02:31<107:34:51,  5.45s/it]
 68%|██████▊   | 153911/225000 [301:02:36<101:19:16,  5.13s/it]
 68%|██████▊   | 153912/225000 [301:02:40<96:07:00,  4.87s/it] 
 68%|██████▊   | 153913/225000 [301:02:44<93:00:18,  4.71s/it]
 68%|██████▊   | 153914/225000 [301:02:49<91:20:23,  4.63s/it]
 68%|██████▊   | 153915/225000 [301:02:54<97:35:42,  4.94s/it]
 68%|██████▊   | 153916/225000 [301:03:00<100:52:39,  5.11s/it]
 68%|██████▊   | 153917/225000 [301:03:05<104:02:08,  5.27s/it]
 68%|██████▊   | 153918/225000 [301:03:16<136:49:39,  6.93s/it]
 68%|██████▊   | 153919/225000 [301:03:23<132:47:07,  6.73s/it]
 68%|██████▊   | 153920/225000 [301:03:27<116:39:18,  5.91s/it]
 68%|██████▊   | 153921/225000 [301:03:31<105:19:43,  5.33s/it]
 68%|██████▊   | 153922/225000 [301:03:36<103:15:35,  5.23s/it]
 68%|██████▊   | 153923/225000 [301:03:47<140:08:19,  7.10s/it]
 68%|██████▊   | 153924/225000 [301:03:50<118:30:39,  6.00s/it]
 68%|██████▊   | 153925/225000 [301:03:57<119:15:14,  6.04s/it]
 68%|██████▊   | 153926/225000 [301:04:03<121:14:13,  6.14s/it]
 68%|██████▊   | 153927/225000 [301:04:08<112:28:22,  5.70s/it]
 68%|██████▊   | 153928/225000 [301:04:16<125:54:43,  6.38s/it]
 68%|██████▊   | 153929/225000 [301:04:22<123:48:26,  6.27s/it]
 68%|██████▊   | 153930/225000 [301:04:26<114:27:56,  5.80s/it]
 68%|██████▊   | 153931/225000 [301:04:31<108:21:08,  5.49s/it]
 68%|██████▊   | 153932/225000 [301:04:36<103:50:29,  5.26s/it]
 68%|██████▊   | 153933/225000 [301:04:40<98:25:16,  4.99s/it] 
 68%|██████▊   | 153934/225000 [301:04:44<93:07:10,  4.72s/it]
 68%|██████▊   | 153935/225000 [301:04:49<94:51:38,  4.81s/it]
 68%|██████▊   | 153936/225000 [301:05:01<135:20:07,  6.86s/it]
 68%|██████▊   | 153937/225000 [301:05:06<123:21:17,  6.25s/it]
 68%|██████▊   | 153938/225000 [301:05:10<110:13:10,  5.58s/it]
 68%|██████▊   | 153939/225000 [301:05:18<128:41:27,  6.52s/it]
 68%|██████▊   | 153940/225000 [301:05:24<124:44:11,  6.32s/it]
 68%|██████▊   | 153941/225000 [301:05:31<127:58:43,  6.48s/it]
 68%|██████▊   | 153942/225000 [301:05:36<115:37:40,  5.86s/it]
 68%|██████▊   | 153943/225000 [301:05:40<108:59:15,  5.52s/it]
 68%|██████▊   | 153944/225000 [301:05:45<106:03:14,  5.37s/it]
 68%|██████▊   | 153945/225000 [301:05:50<103:48:45,  5.26s/it]
 68%|██████▊   | 153946/225000 [301:05:55<102:04:41,  5.17s/it]
 68%|██████▊   | 153947/225000 [301:06:00<101:39:37,  5.15s/it]
 68%|██████▊   | 153948/225000 [301:06:05<97:09:38,  4.92s/it] 
 68%|██████▊   | 153949/225000 [301:06:09<95:54:35,  4.86s/it]
 68%|██████▊   | 153950/225000 [301:06:15<97:26:08,  4.94s/it]
                                                              
{'loss': '0.7807', 'grad_norm': '1.17', 'learning_rate': '1.233e-05', 'epoch': '0.6842', 'num_input_tokens_seen': 498761528, 'train_runtime': '1.084e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 153950/225000 [301:06:15<97:26:08,  4.94s/it]
 68%|██████▊   | 153951/225000 [301:06:19<97:10:06,  4.92s/it]
 68%|██████▊   | 153952/225000 [301:06:30<132:11:04,  6.70s/it]
 68%|██████▊   | 153953/225000 [301:06:37<134:59:53,  6.84s/it]
 68%|██████▊   | 153954/225000 [301:06:45<140:57:15,  7.14s/it]
 68%|██████▊   | 153955/225000 [301:06:49<118:42:04,  6.01s/it]
 68%|██████▊   | 153956/225000 [301:06:53<110:05:21,  5.58s/it]
 68%|██████▊   | 153957/225000 [301:06:57<101:18:47,  5.13s/it]
 68%|██████▊   | 153958/225000 [301:07:02<97:34:09,  4.94s/it] 
 68%|██████▊   | 153959/225000 [301:07:07<96:44:05,  4.90s/it]
 68%|██████▊   | 153960/225000 [301:07:12<96:20:10,  4.88s/it]
 68%|██████▊   | 153961/225000 [301:07:22<130:47:02,  6.63s/it]
 68%|██████▊   | 153962/225000 [301:07:30<135:44:21,  6.88s/it]
 68%|██████▊   | 153963/225000 [301:07:36<130:36:04,  6.62s/it]
 68%|██████▊   | 153964/225000 [301:07:41<123:09:56,  6.24s/it]
 68%|██████▊   | 153965/225000 [301:07:45<107:58:43,  5.47s/it]
 68%|██████▊   | 153966/225000 [301:07:53<122:58:46,  6.23s/it]
 68%|██████▊   | 153967/225000 [301:07:58<118:40:00,  6.01s/it]
 68%|██████▊   | 153968/225000 [301:08:02<108:05:32,  5.48s/it]
 68%|██████▊   | 153969/225000 [301:08:08<106:41:08,  5.41s/it]
 68%|██████▊   | 153970/225000 [301:08:12<102:20:48,  5.19s/it]
 68%|██████▊   | 153971/225000 [301:08:17<99:32:13,  5.04s/it] 
 68%|██████▊   | 153972/225000 [301:08:22<99:27:13,  5.04s/it]
 68%|██████▊   | 153973/225000 [301:08:27<97:04:39,  4.92s/it]
 68%|██████▊   | 153974/225000 [301:08:39<142:43:51,  7.23s/it]
 68%|██████▊   | 153975/225000 [301:08:44<126:52:43,  6.43s/it]
 68%|██████▊   | 153976/225000 [301:08:48<114:50:07,  5.82s/it]
 68%|██████▊   | 153977/225000 [301:08:52<104:45:34,  5.31s/it]
 68%|██████▊   | 153978/225000 [301:08:57<99:33:18,  5.05s/it] 
 68%|██████▊   | 153979/225000 [301:09:04<111:45:12,  5.66s/it]
 68%|██████▊   | 153980/225000 [301:09:13<134:15:55,  6.81s/it]
 68%|██████▊   | 153981/225000 [301:09:17<117:29:12,  5.96s/it]
 68%|██████▊   | 153982/225000 [301:09:22<109:46:25,  5.56s/it]
 68%|██████▊   | 153983/225000 [301:09:27<103:11:42,  5.23s/it]
 68%|██████▊   | 153984/225000 [301:09:32<103:23:11,  5.24s/it]
 68%|██████▊   | 153985/225000 [301:09:37<101:51:36,  5.16s/it]
 68%|██████▊   | 153986/225000 [301:09:46<124:22:12,  6.30s/it]
 68%|██████▊   | 153987/225000 [301:09:53<127:52:27,  6.48s/it]
 68%|██████▊   | 153988/225000 [301:09:56<108:53:11,  5.52s/it]
 68%|██████▊   | 153989/225000 [301:10:01<104:43:53,  5.31s/it]
 68%|██████▊   | 153990/225000 [301:10:10<126:37:58,  6.42s/it]
 68%|██████▊   | 153991/225000 [301:10:13<108:43:45,  5.51s/it]
 68%|██████▊   | 153992/225000 [301:10:18<102:30:55,  5.20s/it]
 68%|██████▊   | 153993/225000 [301:10:22<97:16:24,  4.93s/it] 
 68%|██████▊   | 153994/225000 [301:10:29<111:59:13,  5.68s/it]
 68%|██████▊   | 153995/225000 [301:10:38<132:16:55,  6.71s/it]
 68%|██████▊   | 153996/225000 [301:10:42<112:37:53,  5.71s/it]
 68%|██████▊   | 153997/225000 [301:10:51<130:46:21,  6.63s/it]
 68%|██████▊   | 153998/225000 [301:10:54<113:09:18,  5.74s/it]
 68%|██████▊   | 153999/225000 [301:11:02<124:49:08,  6.33s/it]
 68%|██████▊   | 154000/225000 [301:11:06<112:58:23,  5.73s/it]
                                                               
{'loss': '0.7799', 'grad_norm': '1.156', 'learning_rate': '1.231e-05', 'epoch': '0.6844', 'num_input_tokens_seen': 498891768, 'train_runtime': '1.084e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 154000/225000 [301:11:06<112:58:23,  5.73s/it]
 68%|██████▊   | 154001/225000 [301:11:14<126:25:34,  6.41s/it]
 68%|██████▊   | 154002/225000 [301:11:18<109:31:51,  5.55s/it]
 68%|██████▊   | 154003/225000 [301:11:27<130:33:06,  6.62s/it]
 68%|██████▊   | 154004/225000 [301:11:30<111:17:01,  5.64s/it]
 68%|██████▊   | 154005/225000 [301:11:38<122:24:38,  6.21s/it]
 68%|██████▊   | 154006/225000 [301:11:43<118:45:58,  6.02s/it]
 68%|██████▊   | 154007/225000 [301:11:47<106:22:04,  5.39s/it]
 68%|██████▊   | 154008/225000 [301:11:56<123:50:41,  6.28s/it]
 68%|██████▊   | 154009/225000 [301:12:01<114:54:21,  5.83s/it]
 68%|██████▊   | 154010/225000 [301:12:09<127:48:19,  6.48s/it]
 68%|██████▊   | 154011/225000 [301:12:12<109:51:24,  5.57s/it]
 68%|██████▊   | 154012/225000 [301:12:18<110:46:55,  5.62s/it]
 68%|██████▊   | 154013/225000 [301:12:23<110:00:54,  5.58s/it]
 68%|██████▊   | 154014/225000 [301:12:28<108:29:45,  5.50s/it]
 68%|██████▊   | 154015/225000 [301:12:34<107:35:06,  5.46s/it]
 68%|██████▊   | 154016/225000 [301:12:39<104:45:19,  5.31s/it]
 68%|██████▊   | 154017/225000 [301:12:43<100:52:32,  5.12s/it]
 68%|██████▊   | 154018/225000 [301:12:48<98:21:13,  4.99s/it] 
 68%|██████▊   | 154019/225000 [301:12:54<100:40:38,  5.11s/it]
 68%|██████▊   | 154020/225000 [301:12:59<100:54:14,  5.12s/it]
 68%|██████▊   | 154021/225000 [301:13:04<99:23:13,  5.04s/it] 
 68%|██████▊   | 154022/225000 [301:13:09<100:04:15,  5.08s/it]
 68%|██████▊   | 154023/225000 [301:13:14<100:52:09,  5.12s/it]
 68%|██████▊   | 154024/225000 [301:13:19<102:11:19,  5.18s/it]
 68%|██████▊   | 154025/225000 [301:13:24<102:07:03,  5.18s/it]
 68%|██████▊   | 154026/225000 [301:13:34<126:54:17,  6.44s/it]
 68%|██████▊   | 154027/225000 [301:13:45<154:00:53,  7.81s/it]
 68%|██████▊   | 154028/225000 [301:13:48<126:25:44,  6.41s/it]
 68%|██████▊   | 154029/225000 [301:13:56<137:48:35,  6.99s/it]
 68%|██████▊   | 154030/225000 [301:14:02<127:07:50,  6.45s/it]
 68%|██████▊   | 154031/225000 [301:14:07<123:30:58,  6.27s/it]
 68%|██████▊   | 154032/225000 [301:14:12<112:35:45,  5.71s/it]
 68%|██████▊   | 154033/225000 [301:14:16<106:24:25,  5.40s/it]
 68%|██████▊   | 154034/225000 [301:14:21<98:50:06,  5.01s/it] 
 68%|██████▊   | 154035/225000 [301:14:25<94:47:12,  4.81s/it]
 68%|██████▊   | 154036/225000 [301:14:29<93:25:46,  4.74s/it]
 68%|██████▊   | 154037/225000 [301:14:41<135:36:21,  6.88s/it]
 68%|██████▊   | 154038/225000 [301:14:46<122:04:05,  6.19s/it]
 68%|██████▊   | 154039/225000 [301:14:51<113:22:40,  5.75s/it]
 68%|██████▊   | 154040/225000 [301:14:55<106:31:53,  5.40s/it]
 68%|██████▊   | 154041/225000 [301:15:00<101:52:45,  5.17s/it]
 68%|██████▊   | 154042/225000 [301:15:07<114:33:41,  5.81s/it]
 68%|██████▊   | 154043/225000 [301:15:16<131:45:45,  6.68s/it]
 68%|██████▊   | 154044/225000 [301:15:20<116:20:34,  5.90s/it]
 68%|██████▊   | 154045/225000 [301:15:24<105:24:29,  5.35s/it]
 68%|██████▊   | 154046/225000 [301:15:28<98:19:08,  4.99s/it] 
 68%|██████▊   | 154047/225000 [301:15:35<106:40:18,  5.41s/it]
 68%|██████▊   | 154048/225000 [301:15:44<132:01:29,  6.70s/it]
 68%|██████▊   | 154049/225000 [301:15:49<119:45:37,  6.08s/it]
 68%|██████▊   | 154050/225000 [301:15:54<111:09:39,  5.64s/it]
                                                               
{'loss': '0.7752', 'grad_norm': '1.121', 'learning_rate': '1.23e-05', 'epoch': '0.6847', 'num_input_tokens_seen': 499023016, 'train_runtime': '1.085e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 154050/225000 [301:15:54<111:09:39,  5.64s/it]
 68%|██████▊   | 154051/225000 [301:15:58<103:54:54,  5.27s/it]
 68%|██████▊   | 154052/225000 [301:16:02<98:39:47,  5.01s/it] 
 68%|██████▊   | 154053/225000 [301:16:10<114:48:06,  5.83s/it]
 68%|██████▊   | 154054/225000 [301:16:20<140:00:51,  7.10s/it]
 68%|██████▊   | 154055/225000 [301:16:24<118:27:28,  6.01s/it]
 68%|██████▊   | 154056/225000 [301:16:27<105:53:33,  5.37s/it]
 68%|██████▊   | 154057/225000 [301:16:39<141:33:01,  7.18s/it]
 68%|██████▊   | 154058/225000 [301:16:42<117:25:32,  5.96s/it]
 68%|██████▊   | 154059/225000 [301:16:48<120:38:27,  6.12s/it]
 68%|██████▊   | 154060/225000 [301:16:55<123:03:24,  6.24s/it]
 68%|██████▊   | 154061/225000 [301:17:00<112:43:02,  5.72s/it]
 68%|██████▊   | 154062/225000 [301:17:04<106:37:37,  5.41s/it]
 68%|██████▊   | 154063/225000 [301:17:09<105:47:23,  5.37s/it]
 68%|██████▊   | 154064/225000 [301:17:20<133:17:09,  6.76s/it]
 68%|██████▊   | 154065/225000 [301:17:23<111:13:37,  5.64s/it]
 68%|██████▊   | 154066/225000 [301:17:27<103:59:51,  5.28s/it]
 68%|██████▊   | 154067/225000 [301:17:31<97:29:54,  4.95s/it] 
 68%|██████▊   | 154068/225000 [301:17:36<94:52:26,  4.82s/it]
 68%|██████▊   | 154069/225000 [301:17:40<92:18:58,  4.69s/it]
 68%|██████▊   | 154070/225000 [301:17:49<119:08:17,  6.05s/it]
 68%|██████▊   | 154071/225000 [301:17:57<127:11:34,  6.46s/it]
 68%|██████▊   | 154072/225000 [301:18:02<123:02:30,  6.25s/it]
 68%|██████▊   | 154073/225000 [301:18:09<122:47:34,  6.23s/it]
 68%|██████▊   | 154074/225000 [301:18:13<111:03:31,  5.64s/it]
 68%|██████▊   | 154075/225000 [301:18:17<103:12:31,  5.24s/it]
 68%|██████▊   | 154076/225000 [301:18:22<98:21:16,  4.99s/it] 
 68%|██████▊   | 154077/225000 [301:18:31<124:05:30,  6.30s/it]
 68%|██████▊   | 154078/225000 [301:18:40<141:23:27,  7.18s/it]
 68%|██████▊   | 154079/225000 [301:18:43<118:09:18,  6.00s/it]
 68%|██████▊   | 154080/225000 [301:18:48<107:16:07,  5.45s/it]
 68%|██████▊   | 154081/225000 [301:18:53<105:14:51,  5.34s/it]
 68%|██████▊   | 154082/225000 [301:18:58<104:41:19,  5.31s/it]
 68%|██████▊   | 154083/225000 [301:19:03<103:59:33,  5.28s/it]
 68%|██████▊   | 154084/225000 [301:19:08<102:14:13,  5.19s/it]
 68%|██████▊   | 154085/225000 [301:19:13<100:20:31,  5.09s/it]
 68%|██████▊   | 154086/225000 [301:19:18<98:44:38,  5.01s/it] 
 68%|██████▊   | 154087/225000 [301:19:23<98:35:05,  5.00s/it]
 68%|██████▊   | 154088/225000 [301:19:28<99:27:20,  5.05s/it]
 68%|██████▊   | 154089/225000 [301:19:35<110:25:26,  5.61s/it]
 68%|██████▊   | 154090/225000 [301:19:49<158:16:27,  8.04s/it]
 68%|██████▊   | 154091/225000 [301:19:53<137:29:58,  6.98s/it]
 68%|██████▊   | 154092/225000 [301:20:01<141:48:56,  7.20s/it]
 68%|██████▊   | 154093/225000 [301:20:06<127:29:02,  6.47s/it]
 68%|██████▊   | 154094/225000 [301:20:09<111:07:42,  5.64s/it]
 68%|██████▊   | 154095/225000 [301:20:13<100:41:13,  5.11s/it]
 68%|██████▊   | 154096/225000 [301:20:23<131:24:18,  6.67s/it]
 68%|██████▊   | 154097/225000 [301:20:29<126:58:41,  6.45s/it]
 68%|██████▊   | 154098/225000 [301:20:33<110:04:05,  5.59s/it]
 68%|██████▊   | 154099/225000 [301:20:41<125:54:43,  6.39s/it]
 68%|██████▊   | 154100/225000 [301:20:45<113:04:41,  5.74s/it]
                                                               
{'loss': '0.7788', 'grad_norm': '1.047', 'learning_rate': '1.228e-05', 'epoch': '0.6849', 'num_input_tokens_seen': 499152072, 'train_runtime': '1.085e+06', 'train_tokens_per_second': '460.1'}

 68%|██████▊   | 154100/225000 [301:20:45<113:04:41,  5.74s/it]
 68%|██████▊   | 154101/225000 [301:20:50<103:23:58,  5.25s/it]
 68%|██████▊   | 154102/225000 [301:20:54<99:11:21,  5.04s/it] 
 68%|██████▊   | 154103/225000 [301:21:04<130:23:04,  6.62s/it]
 68%|██████▊   | 154104/225000 [301:21:10<123:20:44,  6.26s/it]
 68%|██████▊   | 154105/225000 [301:21:17<130:02:10,  6.60s/it]
 68%|██████▊   | 154106/225000 [301:21:21<114:14:16,  5.80s/it]
 68%|██████▊   | 154107/225000 [301:21:25<103:48:17,  5.27s/it]
 68%|██████▊   | 154108/225000 [301:21:29<95:55:18,  4.87s/it] 
 68%|██████▊   | 154109/225000 [301:21:36<105:12:59,  5.34s/it]
 68%|██████▊   | 154110/225000 [301:21:46<133:11:20,  6.76s/it]
 68%|██████▊   | 154111/225000 [301:21:51<123:46:37,  6.29s/it]
 68%|██████▊   | 154112/225000 [301:21:56<116:36:11,  5.92s/it]
 68%|██████▊   | 154113/225000 [301:22:00<105:44:04,  5.37s/it]
 68%|██████▊   | 154114/225000 [301:22:04<99:06:20,  5.03s/it] 
 68%|██████▊   | 154115/225000 [301:22:15<131:06:26,  6.66s/it]
 68%|██████▊   | 154116/225000 [301:22:20<123:54:57,  6.29s/it]
 68%|██████▊   | 154117/225000 [301:22:27<125:30:33,  6.37s/it]
 68%|██████▊   | 154118/225000 [301:22:31<110:38:43,  5.62s/it]
 68%|██████▊   | 154119/225000 [301:22:34<99:52:43,  5.07s/it] 
 68%|██████▊   | 154120/225000 [301:22:44<127:45:48,  6.49s/it]
 68%|██████▊   | 154121/225000 [301:22:50<123:03:00,  6.25s/it]
 68%|██████▊   | 154122/225000 [301:22:56<122:59:40,  6.25s/it]
 68%|██████▊   | 154123/225000 [301:23:01<114:49:45,  5.83s/it]
 68%|██████▊   | 154124/225000 [301:23:07<113:29:56,  5.76s/it]
 68%|██████▊   | 154125/225000 [301:23:12<108:52:54,  5.53s/it]
 69%|██████▊   | 154126/225000 [301:23:17<106:34:42,  5.41s/it]
 69%|██████▊   | 154127/225000 [301:23:21<101:36:26,  5.16s/it]
 69%|██████▊   | 154128/225000 [301:23:26<101:16:05,  5.14s/it]
 69%|██████▊   | 154129/225000 [301:23:31<98:06:05,  4.98s/it] 
 69%|██████▊   | 154130/225000 [301:23:36<99:10:50,  5.04s/it]
 69%|██████▊   | 154131/225000 [301:23:41<99:22:34,  5.05s/it]
 69%|██████▊   | 154132/225000 [301:23:46<96:13:59,  4.89s/it]
 69%|██████▊   | 154133/225000 [301:23:51<98:28:00,  5.00s/it]
 69%|██████▊   | 154134/225000 [301:23:56<99:25:33,  5.05s/it]
 69%|██████▊   | 154135/225000 [301:24:05<120:34:36,  6.13s/it]
 69%|██████▊   | 154136/225000 [301:24:16<152:16:59,  7.74s/it]
 69%|██████▊   | 154137/225000 [301:24:20<131:30:53,  6.68s/it]
 69%|██████▊   | 154138/225000 [301:24:25<117:04:23,  5.95s/it]
 69%|██████▊   | 154139/225000 [301:24:34<136:22:15,  6.93s/it]
 69%|██████▊   | 154140/225000 [301:24:39<126:59:17,  6.45s/it]
 69%|██████▊   | 154141/225000 [301:24:45<121:38:57,  6.18s/it]
 69%|██████▊   | 154142/225000 [301:24:53<132:19:04,  6.72s/it]
 69%|██████▊   | 154143/225000 [301:24:56<112:16:27,  5.70s/it]
 69%|██████▊   | 154144/225000 [301:25:03<121:47:19,  6.19s/it]
 69%|██████▊   | 154145/225000 [301:25:07<107:43:09,  5.47s/it]
 69%|██████▊   | 154146/225000 [301:25:14<113:44:51,  5.78s/it]
 69%|██████▊   | 154147/225000 [301:25:21<123:49:40,  6.29s/it]
 69%|██████▊   | 154148/225000 [301:25:25<106:08:15,  5.39s/it]
 69%|██████▊   | 154149/225000 [301:25:33<126:38:29,  6.43s/it]
 69%|██████▊   | 154150/225000 [301:25:38<117:07:37,  5.95s/it]
                                                               
{'loss': '0.7752', 'grad_norm': '1.061', 'learning_rate': '1.226e-05', 'epoch': '0.6851', 'num_input_tokens_seen': 499283336, 'train_runtime': '1.085e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154150/225000 [301:25:38<117:07:37,  5.95s/it]
 69%|██████▊   | 154151/225000 [301:25:45<121:11:16,  6.16s/it]
 69%|██████▊   | 154152/225000 [301:25:49<108:34:22,  5.52s/it]
 69%|██████▊   | 154153/225000 [301:25:53<98:53:15,  5.02s/it] 
 69%|██████▊   | 154154/225000 [301:25:57<92:55:26,  4.72s/it]
 69%|██████▊   | 154155/225000 [301:26:06<121:00:12,  6.15s/it]
 69%|██████▊   | 154156/225000 [301:26:12<117:44:02,  5.98s/it]
 69%|██████▊   | 154157/225000 [301:26:15<101:19:32,  5.15s/it]
 69%|██████▊   | 154158/225000 [301:26:19<96:32:29,  4.91s/it] 
 69%|██████▊   | 154159/225000 [301:26:25<99:11:10,  5.04s/it]
 69%|██████▊   | 154160/225000 [301:26:30<100:54:51,  5.13s/it]
 69%|██████▊   | 154161/225000 [301:26:35<97:21:35,  4.95s/it] 
 69%|██████▊   | 154162/225000 [301:26:40<101:12:45,  5.14s/it]
 69%|██████▊   | 154163/225000 [301:26:46<103:15:11,  5.25s/it]
 69%|██████▊   | 154164/225000 [301:26:51<103:30:54,  5.26s/it]
 69%|██████▊   | 154165/225000 [301:26:58<113:55:22,  5.79s/it]
 69%|██████▊   | 154166/225000 [301:27:10<152:14:37,  7.74s/it]
 69%|██████▊   | 154167/225000 [301:27:14<129:47:36,  6.60s/it]
 69%|██████▊   | 154168/225000 [301:27:18<110:19:42,  5.61s/it]
 69%|██████▊   | 154169/225000 [301:27:28<139:43:13,  7.10s/it]
 69%|██████▊   | 154170/225000 [301:27:32<120:51:49,  6.14s/it]
 69%|██████▊   | 154171/225000 [301:27:36<108:27:56,  5.51s/it]
 69%|██████▊   | 154172/225000 [301:27:45<129:57:23,  6.61s/it]
 69%|██████▊   | 154173/225000 [301:27:49<112:54:11,  5.74s/it]
 69%|██████▊   | 154174/225000 [301:27:53<101:16:23,  5.15s/it]
 69%|██████▊   | 154175/225000 [301:28:02<127:02:27,  6.46s/it]
 69%|██████▊   | 154176/225000 [301:28:06<113:37:03,  5.78s/it]
 69%|██████▊   | 154177/225000 [301:28:10<103:17:42,  5.25s/it]
 69%|██████▊   | 154178/225000 [301:28:21<132:36:36,  6.74s/it]
 69%|██████▊   | 154179/225000 [301:28:26<125:21:11,  6.37s/it]
 69%|██████▊   | 154180/225000 [301:28:31<114:07:13,  5.80s/it]
 69%|██████▊   | 154181/225000 [301:28:39<128:24:57,  6.53s/it]
 69%|██████▊   | 154182/225000 [301:28:44<117:27:15,  5.97s/it]
 69%|██████▊   | 154183/225000 [301:28:48<108:48:28,  5.53s/it]
 69%|██████▊   | 154184/225000 [301:28:53<103:52:45,  5.28s/it]
 69%|██████▊   | 154185/225000 [301:28:57<98:05:58,  4.99s/it] 
 69%|██████▊   | 154186/225000 [301:29:02<96:04:11,  4.88s/it]
 69%|██████▊   | 154187/225000 [301:29:06<94:07:18,  4.78s/it]
 69%|██████▊   | 154188/225000 [301:29:14<114:26:31,  5.82s/it]
 69%|██████▊   | 154189/225000 [301:29:24<138:57:20,  7.06s/it]
 69%|██████▊   | 154190/225000 [301:29:28<117:15:14,  5.96s/it]
 69%|██████▊   | 154191/225000 [301:29:37<135:25:34,  6.89s/it]
 69%|██████▊   | 154192/225000 [301:29:41<117:14:00,  5.96s/it]
 69%|██████▊   | 154193/225000 [301:29:44<103:03:51,  5.24s/it]
 69%|██████▊   | 154194/225000 [301:29:55<136:09:18,  6.92s/it]
 69%|██████▊   | 154195/225000 [301:29:59<115:43:47,  5.88s/it]
 69%|██████▊   | 154196/225000 [301:30:07<131:53:02,  6.71s/it]
 69%|██████▊   | 154197/225000 [301:30:11<114:10:59,  5.81s/it]
 69%|██████▊   | 154198/225000 [301:30:16<109:20:29,  5.56s/it]
 69%|██████▊   | 154199/225000 [301:30:20<103:15:34,  5.25s/it]
 69%|██████▊   | 154200/225000 [301:30:25<101:19:38,  5.15s/it]
                                                               
{'loss': '0.7783', 'grad_norm': '1.149', 'learning_rate': '1.225e-05', 'epoch': '0.6853', 'num_input_tokens_seen': 499413528, 'train_runtime': '1.085e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154200/225000 [301:30:25<101:19:38,  5.15s/it]
 69%|██████▊   | 154201/225000 [301:30:30<97:39:15,  4.97s/it] 
 69%|██████▊   | 154202/225000 [301:30:38<117:55:48,  6.00s/it]
 69%|██████▊   | 154203/225000 [301:30:48<139:45:24,  7.11s/it]
 69%|██████▊   | 154204/225000 [301:30:52<119:11:54,  6.06s/it]
 69%|██████▊   | 154205/225000 [301:30:55<105:17:05,  5.35s/it]
 69%|██████▊   | 154206/225000 [301:31:05<133:40:01,  6.80s/it]
 69%|██████▊   | 154207/225000 [301:31:09<117:10:26,  5.96s/it]
 69%|██████▊   | 154208/225000 [301:31:13<105:32:32,  5.37s/it]
 69%|██████▊   | 154209/225000 [301:31:18<98:23:41,  5.00s/it] 
 69%|██████▊   | 154210/225000 [301:31:23<101:31:04,  5.16s/it]
 69%|██████▊   | 154211/225000 [301:31:35<142:54:55,  7.27s/it]
 69%|██████▊   | 154212/225000 [301:31:41<131:29:10,  6.69s/it]
 69%|██████▊   | 154213/225000 [301:31:45<121:03:34,  6.16s/it]
 69%|██████▊   | 154214/225000 [301:31:50<112:00:51,  5.70s/it]
 69%|██████▊   | 154215/225000 [301:31:55<109:10:34,  5.55s/it]
 69%|██████▊   | 154216/225000 [301:32:00<105:33:32,  5.37s/it]
 69%|██████▊   | 154217/225000 [301:32:05<101:30:38,  5.16s/it]
 69%|██████▊   | 154218/225000 [301:32:10<100:21:08,  5.10s/it]
 69%|██████▊   | 154219/225000 [301:32:16<106:48:30,  5.43s/it]
 69%|██████▊   | 154220/225000 [301:32:29<150:10:27,  7.64s/it]
 69%|██████▊   | 154221/225000 [301:32:32<125:37:24,  6.39s/it]
 69%|██████▊   | 154222/225000 [301:32:36<110:03:26,  5.60s/it]
 69%|██████▊   | 154223/225000 [301:32:45<126:38:14,  6.44s/it]
 69%|██████▊   | 154224/225000 [301:32:49<115:58:54,  5.90s/it]
 69%|██████▊   | 154225/225000 [301:32:57<124:45:25,  6.35s/it]
 69%|██████▊   | 154226/225000 [301:33:01<114:09:19,  5.81s/it]
 69%|██████▊   | 154227/225000 [301:33:05<103:41:36,  5.27s/it]
 69%|██████▊   | 154228/225000 [301:33:10<99:52:04,  5.08s/it] 
 69%|██████▊   | 154229/225000 [301:33:14<95:38:52,  4.87s/it]
 69%|██████▊   | 154230/225000 [301:33:19<93:48:31,  4.77s/it]
 69%|██████▊   | 154231/225000 [301:33:31<139:31:52,  7.10s/it]
 69%|██████▊   | 154232/225000 [301:33:35<120:00:17,  6.10s/it]
 69%|██████▊   | 154233/225000 [301:33:39<106:14:27,  5.40s/it]
 69%|██████▊   | 154234/225000 [301:33:42<95:52:09,  4.88s/it] 
 69%|██████▊   | 154235/225000 [301:33:54<136:13:00,  6.93s/it]
 69%|██████▊   | 154236/225000 [301:33:58<115:22:19,  5.87s/it]
 69%|██████▊   | 154237/225000 [301:34:05<126:37:57,  6.44s/it]
 69%|██████▊   | 154238/225000 [301:34:10<114:08:44,  5.81s/it]
 69%|██████▊   | 154239/225000 [301:34:15<112:03:47,  5.70s/it]
 69%|██████▊   | 154240/225000 [301:34:22<119:46:58,  6.09s/it]
 69%|██████▊   | 154241/225000 [301:34:28<121:19:04,  6.17s/it]
 69%|██████▊   | 154242/225000 [301:34:33<108:55:11,  5.54s/it]
 69%|██████▊   | 154243/225000 [301:34:37<100:04:03,  5.09s/it]
 69%|██████▊   | 154244/225000 [301:34:44<111:55:45,  5.69s/it]
 69%|██████▊   | 154245/225000 [301:34:52<125:30:40,  6.39s/it]
 69%|██████▊   | 154246/225000 [301:34:56<110:55:28,  5.64s/it]
 69%|██████▊   | 154247/225000 [301:35:01<108:10:18,  5.50s/it]
 69%|██████▊   | 154248/225000 [301:35:06<104:23:28,  5.31s/it]
 69%|██████▊   | 154249/225000 [301:35:11<105:17:31,  5.36s/it]
 69%|██████▊   | 154250/225000 [301:35:22<136:06:28,  6.93s/it]
                                                               
{'loss': '0.7842', 'grad_norm': '1.024', 'learning_rate': '1.223e-05', 'epoch': '0.6856', 'num_input_tokens_seen': 499543552, 'train_runtime': '1.086e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154250/225000 [301:35:22<136:06:28,  6.93s/it]
 69%|██████▊   | 154251/225000 [301:35:26<122:51:03,  6.25s/it]
 69%|██████▊   | 154252/225000 [301:35:31<113:04:04,  5.75s/it]
 69%|██████▊   | 154253/225000 [301:35:36<107:29:14,  5.47s/it]
 69%|██████▊   | 154254/225000 [301:35:40<101:00:59,  5.14s/it]
 69%|██████▊   | 154255/225000 [301:35:50<128:55:44,  6.56s/it]
 69%|██████▊   | 154256/225000 [301:35:58<136:42:22,  6.96s/it]
 69%|██████▊   | 154257/225000 [301:36:02<118:24:08,  6.03s/it]
 69%|██████▊   | 154258/225000 [301:36:10<133:16:47,  6.78s/it]
 69%|██████▊   | 154259/225000 [301:36:14<114:55:40,  5.85s/it]
 69%|██████▊   | 154260/225000 [301:36:22<129:03:51,  6.57s/it]
 69%|██████▊   | 154261/225000 [301:36:25<108:52:23,  5.54s/it]
 69%|██████▊   | 154262/225000 [301:36:34<126:04:25,  6.42s/it]
 69%|██████▊   | 154263/225000 [301:36:37<109:46:28,  5.59s/it]
 69%|██████▊   | 154264/225000 [301:36:41<99:30:56,  5.06s/it] 
 69%|██████▊   | 154265/225000 [301:36:52<130:04:32,  6.62s/it]
 69%|██████▊   | 154266/225000 [301:36:56<120:05:19,  6.11s/it]
 69%|██████▊   | 154267/225000 [301:37:02<114:45:57,  5.84s/it]
 69%|██████▊   | 154268/225000 [301:37:07<112:11:01,  5.71s/it]
 69%|██████▊   | 154269/225000 [301:37:11<101:22:15,  5.16s/it]
 69%|██████▊   | 154270/225000 [301:37:15<95:20:40,  4.85s/it] 
 69%|██████▊   | 154271/225000 [301:37:20<92:49:02,  4.72s/it]
 69%|██████▊   | 154272/225000 [301:37:25<94:24:27,  4.81s/it]
 69%|██████▊   | 154273/225000 [301:37:37<136:42:55,  6.96s/it]
 69%|██████▊   | 154274/225000 [301:37:40<115:44:30,  5.89s/it]
 69%|██████▊   | 154275/225000 [301:37:46<116:03:58,  5.91s/it]
 69%|██████▊   | 154276/225000 [301:37:53<123:17:20,  6.28s/it]
 69%|██████▊   | 154277/225000 [301:37:57<108:14:51,  5.51s/it]
 69%|██████▊   | 154278/225000 [301:38:06<131:31:41,  6.70s/it]
 69%|██████▊   | 154279/225000 [301:38:10<114:26:49,  5.83s/it]
 69%|██████▊   | 154280/225000 [301:38:16<113:45:02,  5.79s/it]
 69%|██████▊   | 154281/225000 [301:38:23<121:22:16,  6.18s/it]
 69%|██████▊   | 154282/225000 [301:38:30<125:38:45,  6.40s/it]
 69%|██████▊   | 154283/225000 [301:38:34<115:55:17,  5.90s/it]
 69%|██████▊   | 154284/225000 [301:38:41<122:21:59,  6.23s/it]
 69%|██████▊   | 154285/225000 [301:38:46<110:21:03,  5.62s/it]
 69%|██████▊   | 154286/225000 [301:38:50<101:36:07,  5.17s/it]
 69%|██████▊   | 154287/225000 [301:38:54<96:35:27,  4.92s/it] 
 69%|██████▊   | 154288/225000 [301:38:58<90:54:36,  4.63s/it]
 69%|██████▊   | 154289/225000 [301:39:06<111:04:50,  5.66s/it]
 69%|██████▊   | 154290/225000 [301:39:15<132:04:22,  6.72s/it]
 69%|██████▊   | 154291/225000 [301:39:19<115:20:15,  5.87s/it]
 69%|██████▊   | 154292/225000 [301:39:22<100:22:08,  5.11s/it]
 69%|██████▊   | 154293/225000 [301:39:28<101:05:33,  5.15s/it]
 69%|██████▊   | 154294/225000 [301:39:38<132:12:44,  6.73s/it]
 69%|██████▊   | 154295/225000 [301:39:43<119:26:47,  6.08s/it]
 69%|██████▊   | 154296/225000 [301:39:47<110:57:15,  5.65s/it]
 69%|██████▊   | 154297/225000 [301:39:52<107:56:00,  5.50s/it]
 69%|██████▊   | 154298/225000 [301:39:57<102:37:00,  5.23s/it]
 69%|██████▊   | 154299/225000 [301:40:01<97:19:43,  4.96s/it] 
 69%|██████▊   | 154300/225000 [301:40:06<93:09:36,  4.74s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.112', 'learning_rate': '1.222e-05', 'epoch': '0.6858', 'num_input_tokens_seen': 499673616, 'train_runtime': '1.086e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154300/225000 [301:40:06<93:09:36,  4.74s/it]
 69%|██████▊   | 154301/225000 [301:40:17<133:05:35,  6.78s/it]
 69%|██████▊   | 154302/225000 [301:40:24<132:04:00,  6.72s/it]
 69%|██████▊   | 154303/225000 [301:40:28<116:43:23,  5.94s/it]
 69%|██████▊   | 154304/225000 [301:40:32<103:26:27,  5.27s/it]
 69%|██████▊   | 154305/225000 [301:40:36<97:38:15,  4.97s/it] 
 69%|██████▊   | 154306/225000 [301:40:46<129:18:30,  6.58s/it]
 69%|██████▊   | 154307/225000 [301:40:53<129:02:49,  6.57s/it]
 69%|██████▊   | 154308/225000 [301:40:56<111:12:37,  5.66s/it]
 69%|██████▊   | 154309/225000 [301:41:01<103:29:02,  5.27s/it]
 69%|██████▊   | 154310/225000 [301:41:09<119:00:51,  6.06s/it]
 69%|██████▊   | 154311/225000 [301:41:16<124:26:41,  6.34s/it]
 69%|██████▊   | 154312/225000 [301:41:24<136:34:21,  6.96s/it]
 69%|██████▊   | 154313/225000 [301:41:28<119:09:24,  6.07s/it]
 69%|██████▊   | 154314/225000 [301:41:36<130:57:18,  6.67s/it]
 69%|██████▊   | 154315/225000 [301:41:40<114:07:48,  5.81s/it]
 69%|██████▊   | 154316/225000 [301:41:44<103:37:18,  5.28s/it]
 69%|██████▊   | 154317/225000 [301:41:53<128:58:16,  6.57s/it]
 69%|██████▊   | 154318/225000 [301:41:59<122:56:08,  6.26s/it]
 69%|██████▊   | 154319/225000 [301:42:02<105:21:35,  5.37s/it]
 69%|██████▊   | 154320/225000 [301:42:12<128:34:20,  6.55s/it]
 69%|██████▊   | 154321/225000 [301:42:16<118:56:56,  6.06s/it]
 69%|██████▊   | 154322/225000 [301:42:21<112:37:25,  5.74s/it]
 69%|██████▊   | 154323/225000 [301:42:26<107:49:26,  5.49s/it]
 69%|██████▊   | 154324/225000 [301:42:31<104:16:27,  5.31s/it]
 69%|██████▊   | 154325/225000 [301:42:36<101:36:55,  5.18s/it]
 69%|██████▊   | 154326/225000 [301:42:41<99:07:24,  5.05s/it] 
 69%|██████▊   | 154327/225000 [301:42:46<99:07:56,  5.05s/it]
 69%|██████▊   | 154328/225000 [301:42:51<99:55:51,  5.09s/it]
 69%|██████▊   | 154329/225000 [301:43:04<145:03:37,  7.39s/it]
 69%|██████▊   | 154330/225000 [301:43:10<137:46:59,  7.02s/it]
 69%|██████▊   | 154331/225000 [301:43:15<124:08:51,  6.32s/it]
 69%|██████▊   | 154332/225000 [301:43:19<113:54:55,  5.80s/it]
 69%|██████▊   | 154333/225000 [301:43:24<108:54:34,  5.55s/it]
 69%|██████▊   | 154334/225000 [301:43:29<102:15:36,  5.21s/it]
 69%|██████▊   | 154335/225000 [301:43:39<130:30:14,  6.65s/it]
 69%|██████▊   | 154336/225000 [301:43:45<129:10:52,  6.58s/it]
 69%|██████▊   | 154337/225000 [301:43:53<137:13:51,  6.99s/it]
 69%|██████▊   | 154338/225000 [301:43:57<121:43:37,  6.20s/it]
 69%|██████▊   | 154339/225000 [301:44:02<111:00:09,  5.66s/it]
 69%|██████▊   | 154340/225000 [301:44:05<98:11:03,  5.00s/it] 
 69%|██████▊   | 154341/225000 [301:44:10<94:18:47,  4.81s/it]
 69%|██████▊   | 154342/225000 [301:44:15<95:37:59,  4.87s/it]
 69%|██████▊   | 154343/225000 [301:44:27<137:17:37,  7.00s/it]
 69%|██████▊   | 154344/225000 [301:44:30<117:51:32,  6.01s/it]
 69%|██████▊   | 154345/225000 [301:44:34<104:54:25,  5.35s/it]
 69%|██████▊   | 154346/225000 [301:44:43<125:36:57,  6.40s/it]
 69%|██████▊   | 154347/225000 [301:44:47<110:01:57,  5.61s/it]
 69%|██████▊   | 154348/225000 [301:44:52<109:19:32,  5.57s/it]
 69%|██████▊   | 154349/225000 [301:45:02<134:09:36,  6.84s/it]
 69%|██████▊   | 154350/225000 [301:45:07<121:35:31,  6.20s/it]
                                                               
{'loss': '0.7798', 'grad_norm': '1.222', 'learning_rate': '1.22e-05', 'epoch': '0.686', 'num_input_tokens_seen': 499802688, 'train_runtime': '1.086e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154350/225000 [301:45:07<121:35:31,  6.20s/it]
 69%|██████▊   | 154351/225000 [301:45:12<117:28:42,  5.99s/it]
 69%|██████▊   | 154352/225000 [301:45:17<111:32:35,  5.68s/it]
 69%|██████▊   | 154353/225000 [301:45:26<129:15:42,  6.59s/it]
 69%|██████▊   | 154354/225000 [301:45:30<115:44:13,  5.90s/it]
 69%|██████▊   | 154355/225000 [301:45:35<106:52:41,  5.45s/it]
 69%|██████▊   | 154356/225000 [301:45:39<102:39:51,  5.23s/it]
 69%|██████▊   | 154357/225000 [301:45:43<95:29:46,  4.87s/it] 
 69%|██████▊   | 154358/225000 [301:45:53<124:30:25,  6.35s/it]
 69%|██████▊   | 154359/225000 [301:46:02<136:48:16,  6.97s/it]
 69%|██████▊   | 154360/225000 [301:46:06<121:33:12,  6.19s/it]
 69%|██████▊   | 154361/225000 [301:46:10<110:38:27,  5.64s/it]
 69%|██████▊   | 154362/225000 [301:46:16<108:15:03,  5.52s/it]
 69%|██████▊   | 154363/225000 [301:46:24<125:29:56,  6.40s/it]
 69%|██████▊   | 154364/225000 [301:46:28<111:19:04,  5.67s/it]
 69%|██████▊   | 154365/225000 [301:46:32<101:45:12,  5.19s/it]
 69%|██████▊   | 154366/225000 [301:46:36<96:30:09,  4.92s/it] 
 69%|██████▊   | 154367/225000 [301:46:43<109:38:41,  5.59s/it]
 69%|██████▊   | 154368/225000 [301:46:53<134:51:00,  6.87s/it]
 69%|██████▊   | 154369/225000 [301:46:57<114:01:43,  5.81s/it]
 69%|██████▊   | 154370/225000 [301:47:06<132:52:07,  6.77s/it]
 69%|██████▊   | 154371/225000 [301:47:09<114:12:54,  5.82s/it]
 69%|██████▊   | 154372/225000 [301:47:13<103:24:34,  5.27s/it]
 69%|██████▊   | 154373/225000 [301:47:24<133:02:49,  6.78s/it]
 69%|██████▊   | 154374/225000 [301:47:27<114:00:28,  5.81s/it]
 69%|██████▊   | 154375/225000 [301:47:31<100:30:56,  5.12s/it]
 69%|██████▊   | 154376/225000 [301:47:40<123:31:05,  6.30s/it]
 69%|██████▊   | 154377/225000 [301:47:44<109:15:16,  5.57s/it]
 69%|██████▊   | 154378/225000 [301:47:47<99:13:44,  5.06s/it] 
 69%|██████▊   | 154379/225000 [301:47:56<118:06:01,  6.02s/it]
 69%|██████▊   | 154380/225000 [301:48:02<117:24:12,  5.98s/it]
 69%|██████▊   | 154381/225000 [301:48:06<109:12:20,  5.57s/it]
 69%|██████▊   | 154382/225000 [301:48:14<123:29:49,  6.30s/it]
 69%|██████▊   | 154383/225000 [301:48:20<122:34:16,  6.25s/it]
 69%|██████▊   | 154384/225000 [301:48:26<117:24:45,  5.99s/it]
 69%|██████▊   | 154385/225000 [301:48:31<112:49:53,  5.75s/it]
 69%|██████▊   | 154386/225000 [301:48:36<111:44:59,  5.70s/it]
 69%|██████▊   | 154387/225000 [301:48:41<107:10:23,  5.46s/it]
 69%|██████▊   | 154388/225000 [301:48:46<103:02:42,  5.25s/it]
 69%|██████▊   | 154389/225000 [301:48:51<103:03:44,  5.25s/it]
 69%|██████▊   | 154390/225000 [301:48:56<102:01:01,  5.20s/it]
 69%|██████▊   | 154391/225000 [301:49:02<103:34:18,  5.28s/it]
 69%|██████▊   | 154392/225000 [301:49:07<102:50:44,  5.24s/it]
 69%|██████▊   | 154393/225000 [301:49:12<102:15:11,  5.21s/it]
 69%|██████▊   | 154394/225000 [301:49:17<101:53:58,  5.20s/it]
 69%|██████▊   | 154395/225000 [301:49:23<102:18:54,  5.22s/it]
 69%|██████▊   | 154396/225000 [301:49:28<101:50:44,  5.19s/it]
 69%|██████▊   | 154397/225000 [301:49:33<103:13:14,  5.26s/it]
 69%|██████▊   | 154398/225000 [301:49:38<103:10:09,  5.26s/it]
 69%|██████▊   | 154399/225000 [301:49:51<146:14:09,  7.46s/it]
 69%|██████▊   | 154400/225000 [301:49:58<142:09:29,  7.25s/it]
                                                               
{'loss': '0.7683', 'grad_norm': '1.161', 'learning_rate': '1.219e-05', 'epoch': '0.6862', 'num_input_tokens_seen': 499931520, 'train_runtime': '1.087e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154400/225000 [301:49:58<142:09:29,  7.25s/it]
 69%|██████▊   | 154401/225000 [301:50:02<121:54:32,  6.22s/it]
 69%|██████▊   | 154402/225000 [301:50:06<109:57:22,  5.61s/it]
 69%|██████▊   | 154403/225000 [301:50:11<105:08:58,  5.36s/it]
 69%|██████▊   | 154404/225000 [301:50:22<143:18:39,  7.31s/it]
 69%|██████▊   | 154405/225000 [301:50:26<119:35:21,  6.10s/it]
 69%|██████▊   | 154406/225000 [301:50:35<139:28:03,  7.11s/it]
 69%|██████▊   | 154407/225000 [301:50:39<122:03:53,  6.22s/it]
 69%|██████▊   | 154408/225000 [301:50:43<109:03:58,  5.56s/it]
 69%|██████▊   | 154409/225000 [301:50:48<103:52:31,  5.30s/it]
 69%|██████▊   | 154410/225000 [301:50:59<135:31:33,  6.91s/it]
 69%|██████▊   | 154411/225000 [301:51:02<111:43:05,  5.70s/it]
 69%|██████▊   | 154412/225000 [301:51:09<122:26:29,  6.24s/it]
 69%|██████▊   | 154413/225000 [301:51:13<109:44:00,  5.60s/it]
 69%|██████▊   | 154414/225000 [301:51:17<101:11:00,  5.16s/it]
 69%|██████▊   | 154415/225000 [301:51:29<139:37:37,  7.12s/it]
 69%|██████▊   | 154416/225000 [301:51:34<127:01:58,  6.48s/it]
 69%|██████▊   | 154417/225000 [301:51:38<110:16:08,  5.62s/it]
 69%|██████▊   | 154418/225000 [301:51:44<115:48:50,  5.91s/it]
 69%|██████▊   | 154419/225000 [301:51:51<122:03:48,  6.23s/it]
 69%|██████▊   | 154420/225000 [301:51:57<117:38:21,  6.00s/it]
 69%|██████▊   | 154421/225000 [301:52:01<110:27:22,  5.63s/it]
 69%|██████▊   | 154422/225000 [301:52:07<108:10:03,  5.52s/it]
 69%|██████▊   | 154423/225000 [301:52:12<105:28:58,  5.38s/it]
 69%|██████▊   | 154424/225000 [301:52:17<105:09:30,  5.36s/it]
 69%|██████▊   | 154425/225000 [301:52:22<105:13:15,  5.37s/it]
 69%|██████▊   | 154426/225000 [301:52:28<105:11:01,  5.37s/it]
 69%|██████▊   | 154427/225000 [301:52:33<106:32:07,  5.43s/it]
 69%|██████▊   | 154428/225000 [301:52:39<106:27:24,  5.43s/it]
 69%|██████▊   | 154429/225000 [301:52:44<106:05:26,  5.41s/it]
 69%|██████▊   | 154430/225000 [301:52:49<105:31:21,  5.38s/it]
 69%|██████▊   | 154431/225000 [301:52:55<105:37:51,  5.39s/it]
 69%|██████▊   | 154432/225000 [301:53:00<106:18:31,  5.42s/it]
 69%|██████▊   | 154433/225000 [301:53:12<140:38:11,  7.17s/it]
 69%|██████▊   | 154434/225000 [301:53:21<150:28:44,  7.68s/it]
 69%|██████▊   | 154435/225000 [301:53:25<130:12:17,  6.64s/it]
 69%|██████▊   | 154436/225000 [301:53:29<117:38:09,  6.00s/it]
 69%|██████▊   | 154437/225000 [301:53:34<109:23:23,  5.58s/it]
 69%|██████▊   | 154438/225000 [301:53:39<108:27:45,  5.53s/it]
 69%|██████▊   | 154439/225000 [301:53:46<117:34:57,  6.00s/it]
 69%|██████▊   | 154440/225000 [301:53:55<132:10:22,  6.74s/it]
 69%|██████▊   | 154441/225000 [301:53:59<117:30:20,  6.00s/it]
 69%|██████▊   | 154442/225000 [301:54:07<126:59:47,  6.48s/it]
 69%|██████▊   | 154443/225000 [301:54:12<122:26:09,  6.25s/it]
 69%|██████▊   | 154444/225000 [301:54:20<127:37:19,  6.51s/it]
 69%|██████▊   | 154445/225000 [301:54:24<113:32:27,  5.79s/it]
 69%|██████▊   | 154446/225000 [301:54:28<103:15:44,  5.27s/it]
 69%|██████▊   | 154447/225000 [301:54:32<96:54:05,  4.94s/it] 
 69%|██████▊   | 154448/225000 [301:54:40<113:15:52,  5.78s/it]
 69%|██████▊   | 154449/225000 [301:54:49<132:44:33,  6.77s/it]
 69%|██████▊   | 154450/225000 [301:54:53<115:47:51,  5.91s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '1.114', 'learning_rate': '1.217e-05', 'epoch': '0.6864', 'num_input_tokens_seen': 500061096, 'train_runtime': '1.087e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154450/225000 [301:54:53<115:47:51,  5.91s/it]
 69%|██████▊   | 154451/225000 [301:55:02<134:47:08,  6.88s/it]
 69%|██████▊   | 154452/225000 [301:55:05<113:21:09,  5.78s/it]
 69%|██████▊   | 154453/225000 [301:55:10<108:21:47,  5.53s/it]
 69%|██████▊   | 154454/225000 [301:55:14<101:19:36,  5.17s/it]
 69%|██████▊   | 154455/225000 [301:55:19<97:45:25,  4.99s/it] 
 69%|██████▊   | 154456/225000 [301:55:23<95:47:57,  4.89s/it]
 69%|██████▊   | 154457/225000 [301:55:28<96:11:12,  4.91s/it]
 69%|██████▊   | 154458/225000 [301:55:33<96:35:07,  4.93s/it]
 69%|██████▊   | 154459/225000 [301:55:45<138:27:28,  7.07s/it]
 69%|██████▊   | 154460/225000 [301:55:52<136:17:04,  6.96s/it]
 69%|██████▊   | 154461/225000 [301:55:58<132:43:24,  6.77s/it]
 69%|██████▊   | 154462/225000 [301:56:03<121:14:03,  6.19s/it]
 69%|██████▊   | 154463/225000 [301:56:07<106:14:11,  5.42s/it]
 69%|██████▊   | 154464/225000 [301:56:16<125:55:36,  6.43s/it]
 69%|██████▊   | 154465/225000 [301:56:22<125:15:44,  6.39s/it]
 69%|██████▊   | 154466/225000 [301:56:25<107:06:05,  5.47s/it]
 69%|██████▊   | 154467/225000 [301:56:36<137:31:27,  7.02s/it]
 69%|██████▊   | 154468/225000 [301:56:40<120:48:30,  6.17s/it]
 69%|██████▊   | 154469/225000 [301:56:44<107:03:58,  5.46s/it]
 69%|██████▊   | 154470/225000 [301:56:52<124:26:51,  6.35s/it]
 69%|██████▊   | 154471/225000 [301:56:57<112:00:02,  5.72s/it]
 69%|██████▊   | 154472/225000 [301:57:01<101:29:09,  5.18s/it]
 69%|██████▊   | 154473/225000 [301:57:05<95:42:27,  4.89s/it] 
 69%|██████▊   | 154474/225000 [301:57:09<92:16:12,  4.71s/it]
 69%|██████▊   | 154475/225000 [301:57:19<121:01:50,  6.18s/it]
 69%|██████▊   | 154476/225000 [301:57:26<126:23:16,  6.45s/it]
 69%|██████▊   | 154477/225000 [301:57:29<109:47:01,  5.60s/it]
 69%|██████▊   | 154478/225000 [301:57:34<103:58:02,  5.31s/it]
 69%|██████▊   | 154479/225000 [301:57:39<102:12:25,  5.22s/it]
 69%|██████▊   | 154480/225000 [301:57:44<101:08:20,  5.16s/it]
 69%|██████▊   | 154481/225000 [301:57:49<101:00:01,  5.16s/it]
 69%|██████▊   | 154482/225000 [301:57:55<104:36:00,  5.34s/it]
 69%|██████▊   | 154483/225000 [301:58:08<148:10:36,  7.56s/it]
 69%|██████▊   | 154484/225000 [301:58:15<146:29:21,  7.48s/it]
 69%|██████▊   | 154485/225000 [301:58:21<136:14:31,  6.96s/it]
 69%|██████▊   | 154486/225000 [301:58:28<138:59:22,  7.10s/it]
 69%|██████▊   | 154487/225000 [301:58:32<120:00:37,  6.13s/it]
 69%|██████▊   | 154488/225000 [301:58:37<114:45:09,  5.86s/it]
 69%|██████▊   | 154489/225000 [301:58:44<119:53:52,  6.12s/it]
 69%|██████▊   | 154490/225000 [301:58:47<104:25:26,  5.33s/it]
 69%|██████▊   | 154491/225000 [301:58:52<99:46:39,  5.09s/it] 
 69%|██████▊   | 154492/225000 [301:58:56<94:14:01,  4.81s/it]
 69%|██████▊   | 154493/225000 [301:59:01<93:18:04,  4.76s/it]
 69%|██████▊   | 154494/225000 [301:59:10<116:54:32,  5.97s/it]
 69%|██████▊   | 154495/225000 [301:59:18<131:57:27,  6.74s/it]
 69%|██████▊   | 154496/225000 [301:59:21<111:15:43,  5.68s/it]
 69%|██████▊   | 154497/225000 [301:59:31<135:45:36,  6.93s/it]
 69%|██████▊   | 154498/225000 [301:59:35<114:49:33,  5.86s/it]
 69%|██████▊   | 154499/225000 [301:59:38<101:16:34,  5.17s/it]
 69%|██████▊   | 154500/225000 [301:59:48<127:27:08,  6.51s/it]
                                                               
{'loss': '0.7834', 'grad_norm': '1.075', 'learning_rate': '1.215e-05', 'epoch': '0.6867', 'num_input_tokens_seen': 500190096, 'train_runtime': '1.087e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154500/225000 [301:59:48<127:27:08,  6.51s/it]
 69%|██████▊   | 154501/225000 [301:59:54<124:03:12,  6.33s/it]
 69%|██████▊   | 154502/225000 [301:59:58<110:25:05,  5.64s/it]
 69%|██████▊   | 154503/225000 [302:00:06<126:22:22,  6.45s/it]
 69%|██████▊   | 154504/225000 [302:00:10<113:13:46,  5.78s/it]
 69%|██████▊   | 154505/225000 [302:00:14<102:04:32,  5.21s/it]
 69%|██████▊   | 154506/225000 [302:00:26<141:04:32,  7.20s/it]
 69%|██████▊   | 154507/225000 [302:00:29<119:10:09,  6.09s/it]
 69%|██████▊   | 154508/225000 [302:00:33<106:09:25,  5.42s/it]
 69%|██████▊   | 154509/225000 [302:00:43<130:55:07,  6.69s/it]
 69%|██████▊   | 154510/225000 [302:00:46<112:06:56,  5.73s/it]
 69%|██████▊   | 154511/225000 [302:00:55<127:21:05,  6.50s/it]
 69%|██████▊   | 154512/225000 [302:00:59<112:36:14,  5.75s/it]
 69%|██████▊   | 154513/225000 [302:01:03<100:50:33,  5.15s/it]
 69%|██████▊   | 154514/225000 [302:01:11<121:05:37,  6.18s/it]
 69%|██████▊   | 154515/225000 [302:01:17<119:06:54,  6.08s/it]
 69%|██████▊   | 154516/225000 [302:01:23<117:26:54,  6.00s/it]
 69%|██████▊   | 154517/225000 [302:01:29<118:07:09,  6.03s/it]
 69%|██████▊   | 154518/225000 [302:01:33<104:08:33,  5.32s/it]
 69%|██████▊   | 154519/225000 [302:01:39<111:46:36,  5.71s/it]
 69%|██████▊   | 154520/225000 [302:01:50<139:34:23,  7.13s/it]
 69%|██████▊   | 154521/225000 [302:01:53<116:52:44,  5.97s/it]
 69%|██████▊   | 154522/225000 [302:01:58<111:59:51,  5.72s/it]
 69%|██████▊   | 154523/225000 [302:02:03<107:50:32,  5.51s/it]
 69%|██████▊   | 154524/225000 [302:02:08<106:15:18,  5.43s/it]
 69%|██████▊   | 154525/225000 [302:02:13<103:19:12,  5.28s/it]
 69%|██████▊   | 154526/225000 [302:02:18<103:10:38,  5.27s/it]
 69%|██████▊   | 154527/225000 [302:02:23<97:01:08,  4.96s/it] 
 69%|██████▊   | 154528/225000 [302:02:28<99:00:18,  5.06s/it]
 69%|██████▊   | 154529/225000 [302:02:35<111:01:26,  5.67s/it]
 69%|██████▊   | 154530/225000 [302:02:47<149:38:26,  7.64s/it]
 69%|██████▊   | 154531/225000 [302:02:51<127:47:57,  6.53s/it]
 69%|██████▊   | 154532/225000 [302:02:55<112:39:10,  5.76s/it]
 69%|██████▊   | 154533/225000 [302:03:02<120:51:57,  6.17s/it]
 69%|██████▊   | 154534/225000 [302:03:08<116:40:11,  5.96s/it]
 69%|██████▊   | 154535/225000 [302:03:14<119:27:49,  6.10s/it]
 69%|██████▊   | 154536/225000 [302:03:18<105:40:50,  5.40s/it]
 69%|██████▊   | 154537/225000 [302:03:29<136:09:11,  6.96s/it]
 69%|██████▊   | 154538/225000 [302:03:33<123:27:09,  6.31s/it]
 69%|██████▊   | 154539/225000 [302:03:38<115:31:15,  5.90s/it]
 69%|██████▊   | 154540/225000 [302:03:42<105:17:44,  5.38s/it]
 69%|██████▊   | 154541/225000 [302:03:47<101:20:05,  5.18s/it]
 69%|██████▊   | 154542/225000 [302:03:52<97:53:05,  5.00s/it] 
 69%|██████▊   | 154543/225000 [302:03:57<96:51:09,  4.95s/it]
 69%|██████▊   | 154544/225000 [302:04:08<133:32:39,  6.82s/it]
 69%|██████▊   | 154545/225000 [302:04:15<133:32:39,  6.82s/it]
 69%|██████▊   | 154546/225000 [302:04:18<115:36:40,  5.91s/it]
 69%|██████▊   | 154547/225000 [302:04:23<108:09:45,  5.53s/it]
 69%|██████▊   | 154548/225000 [302:04:31<122:11:53,  6.24s/it]
 69%|██████▊   | 154549/225000 [302:04:38<127:22:42,  6.51s/it]
 69%|██████▊   | 154550/225000 [302:04:42<111:40:14,  5.71s/it]
                                                               
{'loss': '0.7716', 'grad_norm': '1.138', 'learning_rate': '1.214e-05', 'epoch': '0.6869', 'num_input_tokens_seen': 500320648, 'train_runtime': '1.087e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154550/225000 [302:04:42<111:40:14,  5.71s/it]
 69%|██████▊   | 154551/225000 [302:04:48<113:34:44,  5.80s/it]
 69%|██████▊   | 154552/225000 [302:04:56<128:31:20,  6.57s/it]
 69%|██████▊   | 154553/225000 [302:05:00<112:33:31,  5.75s/it]
 69%|██████▊   | 154554/225000 [302:05:10<136:35:59,  6.98s/it]
 69%|██████▊   | 154555/225000 [302:05:15<124:22:16,  6.36s/it]
 69%|██████▊   | 154556/225000 [302:05:20<117:03:57,  5.98s/it]
 69%|██████▊   | 154557/225000 [302:05:25<112:02:49,  5.73s/it]
 69%|██████▊   | 154558/225000 [302:05:30<107:36:22,  5.50s/it]
 69%|██████▊   | 154559/225000 [302:05:35<105:09:28,  5.37s/it]
 69%|██████▊   | 154560/225000 [302:05:43<117:32:18,  6.01s/it]
 69%|██████▊   | 154561/225000 [302:05:54<146:11:58,  7.47s/it]
 69%|██████▊   | 154562/225000 [302:05:57<124:37:46,  6.37s/it]
 69%|██████▊   | 154563/225000 [302:06:02<112:35:47,  5.75s/it]
 69%|██████▊   | 154564/225000 [302:06:08<114:46:18,  5.87s/it]
 69%|██████▊   | 154565/225000 [302:06:17<136:09:59,  6.96s/it]
 69%|██████▊   | 154566/225000 [302:06:21<119:06:38,  6.09s/it]
 69%|██████▊   | 154567/225000 [302:06:27<116:24:11,  5.95s/it]
 69%|██████▊   | 154568/225000 [302:06:37<137:27:42,  7.03s/it]
 69%|██████▊   | 154569/225000 [302:06:40<115:03:09,  5.88s/it]
 69%|██████▊   | 154570/225000 [302:06:48<129:43:30,  6.63s/it]
 69%|██████▊   | 154571/225000 [302:06:53<119:10:17,  6.09s/it]
 69%|██████▊   | 154572/225000 [302:06:57<108:51:55,  5.56s/it]
 69%|██████▊   | 154573/225000 [302:07:01<98:47:37,  5.05s/it] 
 69%|██████▊   | 154574/225000 [302:07:12<130:03:52,  6.65s/it]
 69%|██████▊   | 154575/225000 [302:07:15<114:20:37,  5.85s/it]
 69%|██████▊   | 154576/225000 [302:07:22<117:03:55,  5.98s/it]
 69%|██████▊   | 154577/225000 [302:07:28<120:58:56,  6.18s/it]
 69%|██████▊   | 154578/225000 [302:07:32<107:28:39,  5.49s/it]
 69%|██████▊   | 154579/225000 [302:07:42<129:04:41,  6.60s/it]
 69%|██████▊   | 154580/225000 [302:07:45<111:38:16,  5.71s/it]
 69%|██████▊   | 154581/225000 [302:07:49<100:06:06,  5.12s/it]
 69%|██████▊   | 154582/225000 [302:07:54<97:46:58,  5.00s/it] 
 69%|██████▊   | 154583/225000 [302:08:03<122:21:52,  6.26s/it]
 69%|██████▊   | 154584/225000 [302:08:09<120:12:24,  6.15s/it]
 69%|██████▊   | 154585/225000 [302:08:13<108:57:45,  5.57s/it]
 69%|██████▊   | 154586/225000 [302:08:22<130:55:49,  6.69s/it]
 69%|██████▊   | 154587/225000 [302:08:26<116:17:19,  5.95s/it]
 69%|██████▊   | 154588/225000 [302:08:31<105:24:35,  5.39s/it]
 69%|██████▊   | 154589/225000 [302:08:35<98:43:18,  5.05s/it] 
 69%|██████▊   | 154590/225000 [302:08:39<92:46:53,  4.74s/it]
 69%|██████▊   | 154591/225000 [302:08:48<119:33:01,  6.11s/it]
 69%|██████▊   | 154592/225000 [302:08:54<120:36:47,  6.17s/it]
 69%|██████▊   | 154593/225000 [302:08:58<107:36:07,  5.50s/it]
 69%|██████▊   | 154594/225000 [302:09:08<130:39:10,  6.68s/it]
 69%|██████▊   | 154595/225000 [302:09:12<117:52:31,  6.03s/it]
 69%|██████▊   | 154596/225000 [302:09:17<107:52:03,  5.52s/it]
 69%|██████▊   | 154597/225000 [302:09:21<102:01:31,  5.22s/it]
 69%|██████▊   | 154598/225000 [302:09:26<99:01:05,  5.06s/it] 
 69%|██████▊   | 154599/225000 [302:09:37<137:45:46,  7.04s/it]
 69%|██████▊   | 154600/225000 [302:09:44<134:59:44,  6.90s/it]
                                                               
{'loss': '0.7799', 'grad_norm': '1.178', 'learning_rate': '1.212e-05', 'epoch': '0.6871', 'num_input_tokens_seen': 500451248, 'train_runtime': '1.088e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154600/225000 [302:09:44<134:59:44,  6.90s/it]
 69%|██████▊   | 154601/225000 [302:09:48<117:15:29,  6.00s/it]
 69%|██████▊   | 154602/225000 [302:09:57<134:51:32,  6.90s/it]
 69%|██████▊   | 154603/225000 [302:10:00<115:00:10,  5.88s/it]
 69%|██████▊   | 154604/225000 [302:10:08<127:35:36,  6.53s/it]
 69%|██████▊   | 154605/225000 [302:10:13<116:58:32,  5.98s/it]
 69%|██████▊   | 154606/225000 [302:10:18<108:24:02,  5.54s/it]
 69%|██████▊   | 154607/225000 [302:10:22<103:41:51,  5.30s/it]
 69%|██████▊   | 154608/225000 [302:10:27<99:56:49,  5.11s/it] 
 69%|██████▊   | 154609/225000 [302:10:31<92:53:50,  4.75s/it]
 69%|██████▊   | 154610/225000 [302:10:36<92:51:40,  4.75s/it]
 69%|██████▊   | 154611/225000 [302:10:46<123:32:19,  6.32s/it]
 69%|██████▊   | 154612/225000 [302:10:53<129:29:46,  6.62s/it]
 69%|██████▊   | 154613/225000 [302:10:58<119:51:59,  6.13s/it]
 69%|██████▊   | 154614/225000 [302:11:05<126:39:53,  6.48s/it]
 69%|██████▊   | 154615/225000 [302:11:09<111:37:15,  5.71s/it]
 69%|██████▊   | 154616/225000 [302:11:17<124:53:43,  6.39s/it]
 69%|██████▊   | 154617/225000 [302:11:24<128:25:24,  6.57s/it]
 69%|██████▊   | 154618/225000 [302:11:28<111:32:23,  5.71s/it]
 69%|██████▊   | 154619/225000 [302:11:36<126:55:07,  6.49s/it]
 69%|██████▊   | 154620/225000 [302:11:41<116:05:04,  5.94s/it]
 69%|██████▊   | 154621/225000 [302:11:45<107:47:08,  5.51s/it]
 69%|██████▊   | 154622/225000 [302:11:51<108:20:26,  5.54s/it]
 69%|██████▊   | 154623/225000 [302:11:57<108:07:51,  5.53s/it]
 69%|██████▊   | 154624/225000 [302:12:01<104:15:07,  5.33s/it]
 69%|██████▊   | 154625/225000 [302:12:07<108:04:57,  5.53s/it]
 69%|██████▊   | 154626/225000 [302:12:16<126:30:17,  6.47s/it]
 69%|██████▊   | 154627/225000 [302:12:20<113:15:19,  5.79s/it]
 69%|██████▊   | 154628/225000 [302:12:25<107:26:57,  5.50s/it]
 69%|██████▊   | 154629/225000 [302:12:30<104:01:44,  5.32s/it]
 69%|██████▊   | 154630/225000 [302:12:42<140:53:05,  7.21s/it]
 69%|██████▊   | 154631/225000 [302:12:47<129:48:46,  6.64s/it]
 69%|██████▊   | 154632/225000 [302:12:51<113:34:55,  5.81s/it]
 69%|██████▊   | 154633/225000 [302:12:54<100:16:26,  5.13s/it]
 69%|██████▊   | 154634/225000 [302:13:04<125:04:12,  6.40s/it]
 69%|██████▊   | 154635/225000 [302:13:11<127:50:35,  6.54s/it]
 69%|██████▊   | 154636/225000 [302:13:15<113:05:11,  5.79s/it]
 69%|██████▊   | 154637/225000 [302:13:19<106:06:36,  5.43s/it]
 69%|██████▊   | 154638/225000 [302:13:23<98:29:31,  5.04s/it] 
 69%|██████▊   | 154639/225000 [302:13:28<96:34:33,  4.94s/it]
 69%|██████▊   | 154640/225000 [302:13:41<143:37:46,  7.35s/it]
 69%|██████▊   | 154641/225000 [302:13:47<134:15:01,  6.87s/it]
 69%|██████▊   | 154642/225000 [302:13:50<112:20:48,  5.75s/it]
 69%|██████▊   | 154643/225000 [302:13:57<123:07:55,  6.30s/it]
 69%|██████▊   | 154644/225000 [302:14:02<112:39:03,  5.76s/it]
 69%|██████▊   | 154645/225000 [302:14:07<108:05:19,  5.53s/it]
 69%|██████▊   | 154646/225000 [302:14:12<104:07:32,  5.33s/it]
 69%|██████▊   | 154647/225000 [302:14:16<100:15:22,  5.13s/it]
 69%|██████▊   | 154648/225000 [302:14:21<96:50:35,  4.96s/it] 
 69%|██████▊   | 154649/225000 [302:14:33<135:07:55,  6.91s/it]
 69%|██████▊   | 154650/225000 [302:14:40<137:00:49,  7.01s/it]
                                                               
{'loss': '0.7682', 'grad_norm': '1.114', 'learning_rate': '1.211e-05', 'epoch': '0.6873', 'num_input_tokens_seen': 500581240, 'train_runtime': '1.088e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▊   | 154650/225000 [302:14:40<137:00:49,  7.01s/it]
 69%|██████▊   | 154651/225000 [302:14:43<115:33:00,  5.91s/it]
 69%|██████▊   | 154652/225000 [302:14:48<110:54:20,  5.68s/it]
 69%|██████▊   | 154653/225000 [302:14:58<135:46:58,  6.95s/it]
 69%|██████▊   | 154654/225000 [302:15:05<136:10:00,  6.97s/it]
 69%|██████▊   | 154655/225000 [302:15:08<114:26:22,  5.86s/it]
 69%|██████▊   | 154656/225000 [302:15:12<100:51:14,  5.16s/it]
 69%|██████▊   | 154657/225000 [302:15:23<136:12:26,  6.97s/it]
 69%|██████▊   | 154658/225000 [302:15:26<113:55:31,  5.83s/it]
 69%|██████▊   | 154659/225000 [302:15:33<119:39:47,  6.12s/it]
 69%|██████▊   | 154660/225000 [302:15:37<107:10:34,  5.49s/it]
 69%|██████▊   | 154661/225000 [302:15:47<132:40:07,  6.79s/it]
 69%|██████▊   | 154662/225000 [302:15:51<116:40:47,  5.97s/it]
 69%|██████▊   | 154663/225000 [302:15:55<104:13:21,  5.33s/it]
 69%|██████▊   | 154664/225000 [302:16:04<128:41:33,  6.59s/it]
 69%|██████▊   | 154665/225000 [302:16:10<125:02:57,  6.40s/it]
 69%|██████▊   | 154666/225000 [302:16:14<109:20:18,  5.60s/it]
 69%|██████▊   | 154667/225000 [302:16:18<101:32:14,  5.20s/it]
 69%|██████▊   | 154668/225000 [302:16:23<96:11:16,  4.92s/it] 
 69%|██████▊   | 154669/225000 [302:16:27<93:15:11,  4.77s/it]
 69%|██████▊   | 154670/225000 [302:16:38<126:49:52,  6.49s/it]
 69%|██████▊   | 154671/225000 [302:16:44<124:13:49,  6.36s/it]
 69%|██████▊   | 154672/225000 [302:16:50<122:22:45,  6.26s/it]
 69%|██████▊   | 154673/225000 [302:16:56<125:23:49,  6.42s/it]
 69%|██████▊   | 154674/225000 [302:17:00<109:41:32,  5.62s/it]
 69%|██████▊   | 154675/225000 [302:17:05<103:51:51,  5.32s/it]
 69%|██████▊   | 154676/225000 [302:17:10<100:45:27,  5.16s/it]
 69%|██████▊   | 154677/225000 [302:17:14<98:46:58,  5.06s/it] 
 69%|██████▊   | 154678/225000 [302:17:20<100:03:15,  5.12s/it]
 69%|██████▊   | 154679/225000 [302:17:25<101:04:29,  5.17s/it]
 69%|██████▊   | 154680/225000 [302:17:37<140:21:45,  7.19s/it]
 69%|██████▊   | 154681/225000 [302:17:44<141:22:29,  7.24s/it]
 69%|██████▊   | 154682/225000 [302:17:48<120:31:31,  6.17s/it]
 69%|██████▊   | 154683/225000 [302:17:52<108:04:14,  5.53s/it]
 69%|██████▊   | 154684/225000 [302:18:04<143:47:51,  7.36s/it]
 69%|██████▊   | 154685/225000 [302:18:07<122:16:42,  6.26s/it]
 69%|██████▊   | 154686/225000 [302:18:16<137:38:12,  7.05s/it]
 69%|██████▊   | 154687/225000 [302:18:21<123:57:49,  6.35s/it]
 69%|██████▉   | 154688/225000 [302:18:26<117:17:50,  6.01s/it]
 69%|██████▉   | 154689/225000 [302:18:31<110:05:25,  5.64s/it]
 69%|██████▉   | 154690/225000 [302:18:35<104:14:08,  5.34s/it]
 69%|██████▉   | 154691/225000 [302:18:40<101:01:16,  5.17s/it]
 69%|██████▉   | 154692/225000 [302:18:51<132:23:05,  6.78s/it]
 69%|██████▉   | 154693/225000 [302:18:57<128:20:42,  6.57s/it]
 69%|██████▉   | 154694/225000 [302:19:01<113:58:48,  5.84s/it]
 69%|██████▉   | 154695/225000 [302:19:05<104:40:51,  5.36s/it]
 69%|██████▉   | 154696/225000 [302:19:10<100:02:53,  5.12s/it]
 69%|██████▉   | 154697/225000 [302:19:20<130:42:16,  6.69s/it]
 69%|██████▉   | 154698/225000 [302:19:25<118:20:07,  6.06s/it]
 69%|██████▉   | 154699/225000 [302:19:29<106:30:48,  5.45s/it]
 69%|██████▉   | 154700/225000 [302:19:34<102:39:57,  5.26s/it]
                                                               
{'loss': '0.7831', 'grad_norm': '1.099', 'learning_rate': '1.209e-05', 'epoch': '0.6876', 'num_input_tokens_seen': 500710792, 'train_runtime': '1.088e+06', 'train_tokens_per_second': '460.1'}

 69%|██████▉   | 154700/225000 [302:19:34<102:39:57,  5.26s/it]
 69%|██████▉   | 154701/225000 [302:19:38<100:27:24,  5.14s/it]
 69%|██████▉   | 154702/225000 [302:19:44<100:58:20,  5.17s/it]
 69%|██████▉   | 154703/225000 [302:19:49<101:02:15,  5.17s/it]
 69%|██████▉   | 154704/225000 [302:20:01<144:22:40,  7.39s/it]
 69%|██████▉   | 154705/225000 [302:20:06<130:14:29,  6.67s/it]
 69%|██████▉   | 154706/225000 [302:20:14<135:30:46,  6.94s/it]
 69%|██████▉   | 154707/225000 [302:20:18<115:37:27,  5.92s/it]
 69%|██████▉   | 154708/225000 [302:20:25<123:05:51,  6.30s/it]
 69%|██████▉   | 154709/225000 [302:20:31<123:08:45,  6.31s/it]
 69%|██████▉   | 154710/225000 [302:20:37<121:52:20,  6.24s/it]
 69%|██████▉   | 154711/225000 [302:20:44<126:48:36,  6.49s/it]
 69%|██████▉   | 154712/225000 [302:20:48<110:11:44,  5.64s/it]
 69%|██████▉   | 154713/225000 [302:20:52<100:02:53,  5.12s/it]
 69%|██████▉   | 154714/225000 [302:20:56<95:42:19,  4.90s/it] 
 69%|██████▉   | 154715/225000 [302:21:01<97:28:45,  4.99s/it]
 69%|██████▉   | 154716/225000 [302:21:07<100:16:53,  5.14s/it]
 69%|██████▉   | 154717/225000 [302:21:12<100:28:20,  5.15s/it]
 69%|██████▉   | 154718/225000 [302:21:25<143:55:02,  7.37s/it]
 69%|██████▉   | 154719/225000 [302:21:30<132:56:33,  6.81s/it]
 69%|██████▉   | 154720/225000 [302:21:35<123:22:34,  6.32s/it]
 69%|██████▉   | 154721/225000 [302:21:42<124:52:21,  6.40s/it]
 69%|██████▉   | 154722/225000 [302:21:47<117:58:56,  6.04s/it]
 69%|██████▉   | 154723/225000 [302:21:52<113:07:14,  5.79s/it]
 69%|██████▉   | 154724/225000 [302:21:58<110:21:04,  5.65s/it]
 69%|██████▉   | 154725/225000 [302:22:03<109:14:48,  5.60s/it]
 69%|██████▉   | 154726/225000 [302:22:08<107:23:24,  5.50s/it]
 69%|██████▉   | 154727/225000 [302:22:14<105:53:38,  5.42s/it]
 69%|██████▉   | 154728/225000 [302:22:19<106:01:33,  5.43s/it]
 69%|██████▉   | 154729/225000 [302:22:24<105:14:48,  5.39s/it]
 69%|██████▉   | 154730/225000 [302:22:30<104:50:28,  5.37s/it]
 69%|██████▉   | 154731/225000 [302:22:38<120:13:51,  6.16s/it]
 69%|██████▉   | 154732/225000 [302:22:45<126:02:17,  6.46s/it]
 69%|██████▉   | 154733/225000 [302:22:56<156:26:39,  8.02s/it]
 69%|██████▉   | 154734/225000 [302:23:00<129:18:52,  6.63s/it]
 69%|██████▉   | 154735/225000 [302:23:07<131:11:36,  6.72s/it]
 69%|██████▉   | 154736/225000 [302:23:11<114:15:57,  5.85s/it]
 69%|██████▉   | 154737/225000 [302:23:16<110:49:36,  5.68s/it]
 69%|██████▉   | 154738/225000 [302:23:21<108:58:06,  5.58s/it]
 69%|██████▉   | 154739/225000 [302:23:27<108:20:43,  5.55s/it]
 69%|██████▉   | 154740/225000 [302:23:31<102:29:04,  5.25s/it]
 69%|██████▉   | 154741/225000 [302:23:36<99:54:23,  5.12s/it] 
 69%|██████▉   | 154742/225000 [302:23:42<101:51:54,  5.22s/it]
 69%|██████▉   | 154743/225000 [302:23:47<103:44:49,  5.32s/it]
 69%|██████▉   | 154744/225000 [302:23:52<103:59:05,  5.33s/it]
 69%|██████▉   | 154745/225000 [302:23:58<104:47:11,  5.37s/it]
 69%|██████▉   | 154746/225000 [302:24:03<105:14:22,  5.39s/it]
 69%|██████▉   | 154747/225000 [302:24:09<104:32:45,  5.36s/it]
 69%|██████▉   | 154748/225000 [302:24:14<103:34:22,  5.31s/it]
 69%|██████▉   | 154749/225000 [302:24:20<108:44:41,  5.57s/it]
 69%|██████▉   | 154750/225000 [302:24:33<151:54:58,  7.79s/it]
                                                               
{'loss': '0.7797', 'grad_norm': '1.104', 'learning_rate': '1.208e-05', 'epoch': '0.6878', 'num_input_tokens_seen': 500841472, 'train_runtime': '1.089e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 154750/225000 [302:24:33<151:54:58,  7.79s/it]
 69%|██████▉   | 154751/225000 [302:24:39<142:25:39,  7.30s/it]
 69%|██████▉   | 154752/225000 [302:24:46<141:01:10,  7.23s/it]
 69%|██████▉   | 154753/225000 [302:24:50<123:14:11,  6.32s/it]
 69%|██████▉   | 154754/225000 [302:24:55<113:08:04,  5.80s/it]
 69%|██████▉   | 154755/225000 [302:24:59<105:31:52,  5.41s/it]
 69%|██████▉   | 154756/225000 [302:25:04<97:56:42,  5.02s/it] 
 69%|██████▉   | 154757/225000 [302:25:12<115:35:53,  5.92s/it]
 69%|██████▉   | 154758/225000 [302:25:20<131:31:21,  6.74s/it]
 69%|██████▉   | 154759/225000 [302:25:25<116:57:06,  5.99s/it]
 69%|██████▉   | 154760/225000 [302:25:29<108:18:52,  5.55s/it]
 69%|██████▉   | 154761/225000 [302:25:33<101:12:05,  5.19s/it]
 69%|██████▉   | 154762/225000 [302:25:43<124:48:52,  6.40s/it]
 69%|██████▉   | 154763/225000 [302:25:49<123:28:46,  6.33s/it]
 69%|██████▉   | 154764/225000 [302:25:54<115:39:36,  5.93s/it]
 69%|██████▉   | 154765/225000 [302:26:02<129:44:34,  6.65s/it]
 69%|██████▉   | 154766/225000 [302:26:06<112:15:13,  5.75s/it]
 69%|██████▉   | 154767/225000 [302:26:11<111:35:29,  5.72s/it]
 69%|██████▉   | 154768/225000 [302:26:18<118:35:13,  6.08s/it]
 69%|██████▉   | 154769/225000 [302:26:24<113:27:58,  5.82s/it]
 69%|██████▉   | 154770/225000 [302:26:28<108:04:00,  5.54s/it]
 69%|██████▉   | 154771/225000 [302:26:34<105:36:48,  5.41s/it]
 69%|██████▉   | 154772/225000 [302:26:39<105:16:46,  5.40s/it]
 69%|██████▉   | 154773/225000 [302:26:44<101:34:13,  5.21s/it]
 69%|██████▉   | 154774/225000 [302:26:49<100:46:44,  5.17s/it]
 69%|██████▉   | 154775/225000 [302:26:55<106:59:54,  5.49s/it]
 69%|██████▉   | 154776/225000 [302:27:07<144:39:00,  7.42s/it]
 69%|██████▉   | 154777/225000 [302:27:12<128:24:10,  6.58s/it]
 69%|██████▉   | 154778/225000 [302:27:16<118:19:48,  6.07s/it]
 69%|██████▉   | 154779/225000 [302:27:21<111:17:44,  5.71s/it]
 69%|██████▉   | 154780/225000 [302:27:26<105:53:32,  5.43s/it]
 69%|██████▉   | 154781/225000 [302:27:31<101:55:41,  5.23s/it]
 69%|██████▉   | 154782/225000 [302:27:36<100:53:57,  5.17s/it]
 69%|██████▉   | 154783/225000 [302:27:48<139:23:48,  7.15s/it]
 69%|██████▉   | 154784/225000 [302:27:53<126:28:26,  6.48s/it]
 69%|██████▉   | 154785/225000 [302:27:56<109:23:02,  5.61s/it]
 69%|██████▉   | 154786/225000 [302:28:05<130:27:31,  6.69s/it]
 69%|██████▉   | 154787/225000 [302:28:10<118:53:23,  6.10s/it]
 69%|██████▉   | 154788/225000 [302:28:15<110:39:13,  5.67s/it]
 69%|██████▉   | 154789/225000 [302:28:19<103:43:38,  5.32s/it]
 69%|██████▉   | 154790/225000 [302:28:24<101:21:26,  5.20s/it]
 69%|██████▉   | 154791/225000 [302:28:29<98:45:40,  5.06s/it] 
 69%|██████▉   | 154792/225000 [302:28:34<96:55:46,  4.97s/it]
 69%|██████▉   | 154793/225000 [302:28:45<135:37:02,  6.95s/it]
 69%|██████▉   | 154794/225000 [302:28:50<123:49:31,  6.35s/it]
 69%|██████▉   | 154795/225000 [302:28:58<133:51:11,  6.86s/it]
 69%|██████▉   | 154796/225000 [302:29:02<117:31:44,  6.03s/it]
 69%|██████▉   | 154797/225000 [302:29:07<108:55:27,  5.59s/it]
 69%|██████▉   | 154798/225000 [302:29:11<101:53:19,  5.22s/it]
 69%|██████▉   | 154799/225000 [302:29:16<97:04:01,  4.98s/it] 
 69%|██████▉   | 154800/225000 [302:29:27<136:15:00,  6.99s/it]
                                                               
{'loss': '0.7744', 'grad_norm': '1.024', 'learning_rate': '1.206e-05', 'epoch': '0.688', 'num_input_tokens_seen': 500971960, 'train_runtime': '1.089e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 154800/225000 [302:29:27<136:15:00,  6.99s/it]
 69%|██████▉   | 154801/225000 [302:29:32<125:05:22,  6.41s/it]
 69%|██████▉   | 154802/225000 [302:29:36<111:09:52,  5.70s/it]
 69%|██████▉   | 154803/225000 [302:29:41<106:58:28,  5.49s/it]
 69%|██████▉   | 154804/225000 [302:29:46<104:09:26,  5.34s/it]
 69%|██████▉   | 154805/225000 [302:29:52<104:41:13,  5.37s/it]
 69%|██████▉   | 154806/225000 [302:29:57<103:14:37,  5.30s/it]
 69%|██████▉   | 154807/225000 [302:30:02<102:25:25,  5.25s/it]
 69%|██████▉   | 154808/225000 [302:30:07<102:32:50,  5.26s/it]
 69%|██████▉   | 154809/225000 [302:30:13<102:24:51,  5.25s/it]
 69%|██████▉   | 154810/225000 [302:30:18<103:22:47,  5.30s/it]
 69%|██████▉   | 154811/225000 [302:30:29<134:15:03,  6.89s/it]
 69%|██████▉   | 154812/225000 [302:30:36<138:13:48,  7.09s/it]
 69%|██████▉   | 154813/225000 [302:30:41<123:55:34,  6.36s/it]
 69%|██████▉   | 154814/225000 [302:30:45<110:16:09,  5.66s/it]
 69%|██████▉   | 154815/225000 [302:30:49<103:11:18,  5.29s/it]
 69%|██████▉   | 154816/225000 [302:30:59<130:39:05,  6.70s/it]
 69%|██████▉   | 154817/225000 [302:31:07<135:05:25,  6.93s/it]
 69%|██████▉   | 154818/225000 [302:31:10<115:03:44,  5.90s/it]
 69%|██████▉   | 154819/225000 [302:31:18<127:38:10,  6.55s/it]
 69%|██████▉   | 154820/225000 [302:31:24<121:53:22,  6.25s/it]
 69%|██████▉   | 154821/225000 [302:31:29<117:25:42,  6.02s/it]
 69%|██████▉   | 154822/225000 [302:31:35<113:23:44,  5.82s/it]
 69%|██████▉   | 154823/225000 [302:31:40<110:55:54,  5.69s/it]
 69%|██████▉   | 154824/225000 [302:31:45<108:43:13,  5.58s/it]
 69%|██████▉   | 154825/225000 [302:31:51<106:22:51,  5.46s/it]
 69%|██████▉   | 154826/225000 [302:31:56<104:47:22,  5.38s/it]
 69%|██████▉   | 154827/225000 [302:32:01<104:57:50,  5.38s/it]
 69%|██████▉   | 154828/225000 [302:32:07<105:14:07,  5.40s/it]
 69%|██████▉   | 154829/225000 [302:32:12<103:47:51,  5.33s/it]
 69%|██████▉   | 154830/225000 [302:32:17<105:18:38,  5.40s/it]
 69%|██████▉   | 154831/225000 [302:32:23<104:50:58,  5.38s/it]
 69%|██████▉   | 154832/225000 [302:32:28<107:35:08,  5.52s/it]
 69%|██████▉   | 154833/225000 [302:32:34<107:18:54,  5.51s/it]
 69%|██████▉   | 154834/225000 [302:32:41<117:47:28,  6.04s/it]
 69%|██████▉   | 154835/225000 [302:32:53<151:40:46,  7.78s/it]
 69%|██████▉   | 154836/225000 [302:32:57<128:15:38,  6.58s/it]
 69%|██████▉   | 154837/225000 [302:33:05<136:45:01,  7.02s/it]
 69%|██████▉   | 154838/225000 [302:33:11<131:41:38,  6.76s/it]
 69%|██████▉   | 154839/225000 [302:33:15<115:45:09,  5.94s/it]
 69%|██████▉   | 154840/225000 [302:33:23<127:17:47,  6.53s/it]
 69%|██████▉   | 154841/225000 [302:33:28<117:21:56,  6.02s/it]
 69%|██████▉   | 154842/225000 [302:33:33<110:53:13,  5.69s/it]
 69%|██████▉   | 154843/225000 [302:33:37<104:11:21,  5.35s/it]
 69%|██████▉   | 154844/225000 [302:33:42<100:31:56,  5.16s/it]
 69%|██████▉   | 154845/225000 [302:33:52<129:48:37,  6.66s/it]
 69%|██████▉   | 154846/225000 [302:33:56<110:45:14,  5.68s/it]
 69%|██████▉   | 154847/225000 [302:34:01<111:03:00,  5.70s/it]
 69%|██████▉   | 154848/225000 [302:34:09<125:30:21,  6.44s/it]
 69%|██████▉   | 154849/225000 [302:34:13<111:10:53,  5.71s/it]
 69%|██████▉   | 154850/225000 [302:34:18<101:56:32,  5.23s/it]
                                                               
{'loss': '0.7867', 'grad_norm': '1.113', 'learning_rate': '1.205e-05', 'epoch': '0.6882', 'num_input_tokens_seen': 501102088, 'train_runtime': '1.089e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 154850/225000 [302:34:18<101:56:32,  5.23s/it]
 69%|██████▉   | 154851/225000 [302:34:22<94:15:14,  4.84s/it] 
 69%|██████▉   | 154852/225000 [302:34:32<129:11:32,  6.63s/it]
 69%|██████▉   | 154853/225000 [302:34:36<110:30:33,  5.67s/it]
 69%|██████▉   | 154854/225000 [302:34:44<124:06:14,  6.37s/it]
 69%|██████▉   | 154855/225000 [302:34:48<109:42:55,  5.63s/it]
 69%|██████▉   | 154856/225000 [302:34:52<104:20:41,  5.36s/it]
 69%|██████▉   | 154857/225000 [302:35:02<127:14:46,  6.53s/it]
 69%|██████▉   | 154858/225000 [302:35:06<113:08:39,  5.81s/it]
 69%|██████▉   | 154859/225000 [302:35:13<119:59:55,  6.16s/it]
 69%|██████▉   | 154860/225000 [302:35:18<114:53:54,  5.90s/it]
 69%|██████▉   | 154861/225000 [302:35:25<122:41:19,  6.30s/it]
 69%|██████▉   | 154862/225000 [302:35:29<108:50:55,  5.59s/it]
 69%|██████▉   | 154863/225000 [302:35:33<97:48:57,  5.02s/it] 
 69%|██████▉   | 154864/225000 [302:35:37<91:35:54,  4.70s/it]
 69%|██████▉   | 154865/225000 [302:35:48<128:50:50,  6.61s/it]
 69%|██████▉   | 154866/225000 [302:35:51<108:14:53,  5.56s/it]
 69%|██████▉   | 154867/225000 [302:35:56<105:55:09,  5.44s/it]
 69%|██████▉   | 154868/225000 [302:36:01<103:57:36,  5.34s/it]
 69%|██████▉   | 154869/225000 [302:36:07<103:35:59,  5.32s/it]
 69%|██████▉   | 154870/225000 [302:36:12<101:22:48,  5.20s/it]
 69%|██████▉   | 154871/225000 [302:36:17<104:12:14,  5.35s/it]
 69%|██████▉   | 154872/225000 [302:36:22<102:28:51,  5.26s/it]
 69%|██████▉   | 154873/225000 [302:36:27<100:55:02,  5.18s/it]
 69%|██████▉   | 154874/225000 [302:36:32<99:38:18,  5.12s/it] 
 69%|██████▉   | 154875/225000 [302:36:38<102:41:22,  5.27s/it]
 69%|██████▉   | 154876/225000 [302:36:43<101:59:22,  5.24s/it]
 69%|██████▉   | 154877/225000 [302:36:49<103:48:43,  5.33s/it]
 69%|██████▉   | 154878/225000 [302:36:54<106:25:57,  5.46s/it]
 69%|██████▉   | 154879/225000 [302:37:06<140:12:53,  7.20s/it]
 69%|██████▉   | 154880/225000 [302:37:13<143:40:58,  7.38s/it]
 69%|██████▉   | 154881/225000 [302:37:17<123:23:00,  6.33s/it]
 69%|██████▉   | 154882/225000 [302:37:24<126:11:27,  6.48s/it]
 69%|██████▉   | 154883/225000 [302:37:28<114:02:44,  5.86s/it]
 69%|██████▉   | 154884/225000 [302:37:33<107:08:41,  5.50s/it]
 69%|██████▉   | 154885/225000 [302:37:37<99:45:07,  5.12s/it] 
 69%|██████▉   | 154886/225000 [302:37:43<105:16:33,  5.41s/it]
 69%|██████▉   | 154887/225000 [302:37:54<132:45:15,  6.82s/it]
 69%|██████▉   | 154888/225000 [302:37:57<115:17:25,  5.92s/it]
 69%|██████▉   | 154889/225000 [302:38:04<116:50:32,  6.00s/it]
 69%|██████▉   | 154890/225000 [302:38:11<126:14:49,  6.48s/it]
 69%|██████▉   | 154891/225000 [302:38:15<111:20:32,  5.72s/it]
 69%|██████▉   | 154892/225000 [302:38:20<104:45:12,  5.38s/it]
 69%|██████▉   | 154893/225000 [302:38:24<101:07:32,  5.19s/it]
 69%|██████▉   | 154894/225000 [302:38:29<95:13:37,  4.89s/it] 
 69%|██████▉   | 154895/225000 [302:38:37<116:33:06,  5.99s/it]
 69%|██████▉   | 154896/225000 [302:38:46<130:44:52,  6.71s/it]
 69%|██████▉   | 154897/225000 [302:38:49<112:26:34,  5.77s/it]
 69%|██████▉   | 154898/225000 [302:38:58<127:35:30,  6.55s/it]
 69%|██████▉   | 154899/225000 [302:39:01<109:37:40,  5.63s/it]
 69%|██████▉   | 154900/225000 [302:39:07<109:46:59,  5.64s/it]
                                                               
{'loss': '0.7833', 'grad_norm': '1.202', 'learning_rate': '1.203e-05', 'epoch': '0.6884', 'num_input_tokens_seen': 501232648, 'train_runtime': '1.09e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 154900/225000 [302:39:07<109:46:59,  5.64s/it]
 69%|██████▉   | 154901/225000 [302:39:13<116:17:47,  5.97s/it]
 69%|██████▉   | 154902/225000 [302:39:18<105:19:31,  5.41s/it]
 69%|██████▉   | 154903/225000 [302:39:23<103:42:45,  5.33s/it]
 69%|██████▉   | 154904/225000 [302:39:32<126:56:00,  6.52s/it]
 69%|██████▉   | 154905/225000 [302:39:36<110:59:23,  5.70s/it]
 69%|██████▉   | 154906/225000 [302:39:43<122:03:04,  6.27s/it]
 69%|██████▉   | 154907/225000 [302:39:49<116:01:27,  5.96s/it]
 69%|██████▉   | 154908/225000 [302:39:53<105:46:21,  5.43s/it]
 69%|██████▉   | 154909/225000 [302:39:57<99:44:03,  5.12s/it] 
 69%|██████▉   | 154910/225000 [302:40:02<95:49:55,  4.92s/it]
 69%|██████▉   | 154911/225000 [302:40:06<94:08:55,  4.84s/it]
 69%|██████▉   | 154912/225000 [302:40:11<91:32:00,  4.70s/it]
 69%|██████▉   | 154913/225000 [302:40:20<119:42:57,  6.15s/it]
 69%|██████▉   | 154914/225000 [302:40:27<125:37:56,  6.45s/it]
 69%|██████▉   | 154915/225000 [302:40:35<130:16:04,  6.69s/it]
 69%|██████▉   | 154916/225000 [302:40:41<129:10:45,  6.64s/it]
 69%|██████▉   | 154917/225000 [302:40:45<110:59:57,  5.70s/it]
 69%|██████▉   | 154918/225000 [302:40:50<109:02:31,  5.60s/it]
 69%|██████▉   | 154919/225000 [302:40:56<113:40:25,  5.84s/it]
 69%|██████▉   | 154920/225000 [302:41:03<116:56:59,  6.01s/it]
 69%|██████▉   | 154921/225000 [302:41:07<103:56:44,  5.34s/it]
 69%|██████▉   | 154922/225000 [302:41:11<97:08:43,  4.99s/it] 
 69%|██████▉   | 154923/225000 [302:41:14<89:47:28,  4.61s/it]
 69%|██████▉   | 154924/225000 [302:41:25<122:27:13,  6.29s/it]
 69%|██████▉   | 154925/225000 [302:41:30<119:02:40,  6.12s/it]
 69%|██████▉   | 154926/225000 [302:41:35<109:09:20,  5.61s/it]
 69%|██████▉   | 154927/225000 [302:41:39<100:53:33,  5.18s/it]
 69%|██████▉   | 154928/225000 [302:41:44<97:15:38,  5.00s/it] 
 69%|██████▉   | 154929/225000 [302:41:48<95:38:37,  4.91s/it]
 69%|██████▉   | 154930/225000 [302:41:53<94:56:13,  4.88s/it]
 69%|██████▉   | 154931/225000 [302:41:59<100:59:19,  5.19s/it]
 69%|██████▉   | 154932/225000 [302:42:11<142:43:12,  7.33s/it]
 69%|██████▉   | 154933/225000 [302:42:16<129:00:50,  6.63s/it]
 69%|██████▉   | 154934/225000 [302:42:22<124:28:25,  6.40s/it]
 69%|██████▉   | 154935/225000 [302:42:26<110:55:44,  5.70s/it]
 69%|██████▉   | 154936/225000 [302:42:32<108:33:19,  5.58s/it]
 69%|██████▉   | 154937/225000 [302:42:39<118:24:03,  6.08s/it]
 69%|██████▉   | 154938/225000 [302:42:43<108:13:44,  5.56s/it]
 69%|██████▉   | 154939/225000 [302:42:47<97:06:45,  4.99s/it] 
 69%|██████▉   | 154940/225000 [302:42:56<121:04:47,  6.22s/it]
 69%|██████▉   | 154941/225000 [302:43:01<114:21:03,  5.88s/it]
 69%|██████▉   | 154942/225000 [302:43:08<120:18:47,  6.18s/it]
 69%|██████▉   | 154943/225000 [302:43:12<106:56:00,  5.49s/it]
 69%|██████▉   | 154944/225000 [302:43:16<98:54:57,  5.08s/it] 
 69%|██████▉   | 154945/225000 [302:43:21<99:27:18,  5.11s/it]
 69%|██████▉   | 154946/225000 [302:43:26<100:42:11,  5.18s/it]
 69%|██████▉   | 154947/225000 [302:43:32<102:15:32,  5.26s/it]
 69%|██████▉   | 154948/225000 [302:43:38<104:50:41,  5.39s/it]
 69%|██████▉   | 154949/225000 [302:43:43<105:39:01,  5.43s/it]
 69%|██████▉   | 154950/225000 [302:43:55<141:01:45,  7.25s/it]
                                                               
{'loss': '0.7813', 'grad_norm': '1.142', 'learning_rate': '1.201e-05', 'epoch': '0.6887', 'num_input_tokens_seen': 501361960, 'train_runtime': '1.09e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 154950/225000 [302:43:55<141:01:45,  7.25s/it]
 69%|██████▉   | 154951/225000 [302:43:59<126:00:32,  6.48s/it]
 69%|██████▉   | 154952/225000 [302:44:04<115:48:14,  5.95s/it]
 69%|██████▉   | 154953/225000 [302:44:09<107:51:51,  5.54s/it]
 69%|██████▉   | 154954/225000 [302:44:13<102:45:48,  5.28s/it]
 69%|██████▉   | 154955/225000 [302:44:24<136:12:38,  7.00s/it]
 69%|██████▉   | 154956/225000 [302:44:30<128:17:09,  6.59s/it]
 69%|██████▉   | 154957/225000 [302:44:34<114:52:15,  5.90s/it]
 69%|██████▉   | 154958/225000 [302:44:40<113:46:12,  5.85s/it]
 69%|██████▉   | 154959/225000 [302:44:45<111:51:16,  5.75s/it]
 69%|██████▉   | 154960/225000 [302:44:51<109:31:31,  5.63s/it]
 69%|██████▉   | 154961/225000 [302:44:57<114:32:25,  5.89s/it]
 69%|██████▉   | 154962/225000 [302:45:06<132:38:59,  6.82s/it]
 69%|██████▉   | 154963/225000 [302:45:10<112:11:00,  5.77s/it]
 69%|██████▉   | 154964/225000 [302:45:15<111:33:57,  5.73s/it]
 69%|██████▉   | 154965/225000 [302:45:23<124:39:12,  6.41s/it]
 69%|██████▉   | 154966/225000 [302:45:27<107:31:56,  5.53s/it]
 69%|██████▉   | 154967/225000 [302:45:32<104:27:12,  5.37s/it]
 69%|██████▉   | 154968/225000 [302:45:40<122:04:06,  6.27s/it]
 69%|██████▉   | 154969/225000 [302:45:45<115:32:12,  5.94s/it]
 69%|██████▉   | 154970/225000 [302:45:50<110:48:31,  5.70s/it]
 69%|██████▉   | 154971/225000 [302:45:55<105:20:56,  5.42s/it]
 69%|██████▉   | 154972/225000 [302:45:59<98:08:33,  5.05s/it] 
 69%|██████▉   | 154973/225000 [302:46:03<92:08:29,  4.74s/it]
 69%|██████▉   | 154974/225000 [302:46:09<95:50:14,  4.93s/it]
 69%|██████▉   | 154975/225000 [302:46:14<97:26:57,  5.01s/it]
 69%|██████▉   | 154976/225000 [302:46:19<96:37:40,  4.97s/it]
 69%|██████▉   | 154977/225000 [302:46:30<133:19:58,  6.85s/it]
 69%|██████▉   | 154978/225000 [302:46:37<134:19:11,  6.91s/it]
 69%|██████▉   | 154979/225000 [302:46:40<113:57:04,  5.86s/it]
 69%|██████▉   | 154980/225000 [302:46:48<125:30:04,  6.45s/it]
 69%|██████▉   | 154981/225000 [302:46:52<108:58:41,  5.60s/it]
 69%|██████▉   | 154982/225000 [302:46:57<104:44:39,  5.39s/it]
 69%|██████▉   | 154983/225000 [302:47:02<103:18:10,  5.31s/it]
 69%|██████▉   | 154984/225000 [302:47:07<100:32:32,  5.17s/it]
 69%|██████▉   | 154985/225000 [302:47:12<99:42:22,  5.13s/it] 
 69%|██████▉   | 154986/225000 [302:47:17<98:41:01,  5.07s/it]
 69%|██████▉   | 154987/225000 [302:47:21<96:57:39,  4.99s/it]
 69%|██████▉   | 154988/225000 [302:47:27<97:47:37,  5.03s/it]
 69%|██████▉   | 154989/225000 [302:47:31<95:46:04,  4.92s/it]
 69%|██████▉   | 154990/225000 [302:47:36<97:12:51,  5.00s/it]
 69%|██████▉   | 154991/225000 [302:47:44<109:52:14,  5.65s/it]
 69%|██████▉   | 154992/225000 [302:47:56<148:35:07,  7.64s/it]
 69%|██████▉   | 154993/225000 [302:48:01<133:21:37,  6.86s/it]
 69%|██████▉   | 154994/225000 [302:48:06<124:39:53,  6.41s/it]
 69%|██████▉   | 154995/225000 [302:48:11<117:03:31,  6.02s/it]
 69%|██████▉   | 154996/225000 [302:48:18<120:17:50,  6.19s/it]
 69%|██████▉   | 154997/225000 [302:48:21<103:52:39,  5.34s/it]
 69%|██████▉   | 154998/225000 [302:48:26<99:16:43,  5.11s/it] 
 69%|██████▉   | 154999/225000 [302:48:32<102:18:59,  5.26s/it]
 69%|██████▉   | 155000/225000 [302:48:39<117:31:06,  6.04s/it]
                                                               
{'loss': '0.7795', 'grad_norm': '1.074', 'learning_rate': '1.2e-05', 'epoch': '0.6889', 'num_input_tokens_seen': 501491824, 'train_runtime': '1.09e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155000/225000 [302:48:39<117:31:06,  6.04s/it]
 69%|██████▉   | 155001/225000 [302:48:46<118:58:16,  6.12s/it]
 69%|██████▉   | 155002/225000 [302:48:52<120:16:43,  6.19s/it]
 69%|██████▉   | 155003/225000 [302:48:57<110:26:34,  5.68s/it]
 69%|██████▉   | 155004/225000 [302:49:04<118:04:08,  6.07s/it]
 69%|██████▉   | 155005/225000 [302:49:08<111:15:38,  5.72s/it]
 69%|██████▉   | 155006/225000 [302:49:17<127:26:19,  6.55s/it]
 69%|██████▉   | 155007/225000 [302:49:21<112:18:32,  5.78s/it]
 69%|██████▉   | 155008/225000 [302:49:25<101:04:39,  5.20s/it]
 69%|██████▉   | 155009/225000 [302:49:29<93:40:19,  4.82s/it] 
 69%|██████▉   | 155010/225000 [302:49:41<134:53:24,  6.94s/it]
 69%|██████▉   | 155011/225000 [302:49:44<115:24:36,  5.94s/it]
 69%|██████▉   | 155012/225000 [302:49:48<102:53:30,  5.29s/it]
 69%|██████▉   | 155013/225000 [302:49:53<99:28:58,  5.12s/it] 
 69%|██████▉   | 155014/225000 [302:50:01<115:15:33,  5.93s/it]
 69%|██████▉   | 155015/225000 [302:50:05<108:01:24,  5.56s/it]
 69%|██████▉   | 155016/225000 [302:50:12<115:05:37,  5.92s/it]
 69%|██████▉   | 155017/225000 [302:50:16<103:47:40,  5.34s/it]
 69%|██████▉   | 155018/225000 [302:50:20<94:17:22,  4.85s/it] 
 69%|██████▉   | 155019/225000 [302:50:30<127:22:59,  6.55s/it]
 69%|██████▉   | 155020/225000 [302:50:35<116:01:11,  5.97s/it]
 69%|██████▉   | 155021/225000 [302:50:40<109:13:53,  5.62s/it]
 69%|██████▉   | 155022/225000 [302:50:44<104:07:02,  5.36s/it]
 69%|██████▉   | 155023/225000 [302:50:49<98:07:03,  5.05s/it] 
 69%|██████▉   | 155024/225000 [302:50:53<92:36:56,  4.76s/it]
 69%|██████▉   | 155025/225000 [302:51:02<120:32:05,  6.20s/it]
 69%|██████▉   | 155026/225000 [302:51:10<127:55:30,  6.58s/it]
 69%|██████▉   | 155027/225000 [302:51:13<109:57:30,  5.66s/it]
 69%|██████▉   | 155028/225000 [302:51:18<102:49:06,  5.29s/it]
 69%|██████▉   | 155029/225000 [302:51:22<97:19:28,  5.01s/it] 
 69%|██████▉   | 155030/225000 [302:51:27<95:59:19,  4.94s/it]
 69%|██████▉   | 155031/225000 [302:51:32<95:00:06,  4.89s/it]
 69%|██████▉   | 155032/225000 [302:51:44<139:28:40,  7.18s/it]
 69%|██████▉   | 155033/225000 [302:51:49<124:38:31,  6.41s/it]
 69%|██████▉   | 155034/225000 [302:51:53<111:11:28,  5.72s/it]
 69%|██████▉   | 155035/225000 [302:51:57<101:52:58,  5.24s/it]
 69%|██████▉   | 155036/225000 [302:52:02<101:16:50,  5.21s/it]
 69%|██████▉   | 155037/225000 [302:52:14<140:00:21,  7.20s/it]
 69%|██████▉   | 155038/225000 [302:52:18<122:36:12,  6.31s/it]
 69%|██████▉   | 155039/225000 [302:52:22<109:55:51,  5.66s/it]
 69%|██████▉   | 155040/225000 [302:52:26<100:35:50,  5.18s/it]
 69%|██████▉   | 155041/225000 [302:52:31<95:04:13,  4.89s/it] 
 69%|██████▉   | 155042/225000 [302:52:40<123:43:43,  6.37s/it]
 69%|██████▉   | 155043/225000 [302:52:46<120:49:39,  6.22s/it]
 69%|██████▉   | 155044/225000 [302:52:51<111:17:26,  5.73s/it]
 69%|██████▉   | 155045/225000 [302:52:56<107:09:55,  5.51s/it]
 69%|██████▉   | 155046/225000 [302:53:00<100:33:28,  5.17s/it]
 69%|██████▉   | 155047/225000 [302:53:05<97:20:45,  5.01s/it] 
 69%|██████▉   | 155048/225000 [302:53:10<95:59:58,  4.94s/it]
 69%|██████▉   | 155049/225000 [302:53:14<94:26:56,  4.86s/it]
 69%|██████▉   | 155050/225000 [302:53:19<93:48:26,  4.83s/it]
                                                              
{'loss': '0.7764', 'grad_norm': '1.1', 'learning_rate': '1.198e-05', 'epoch': '0.6891', 'num_input_tokens_seen': 501621848, 'train_runtime': '1.09e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155050/225000 [302:53:19<93:48:26,  4.83s/it]
 69%|██████▉   | 155051/225000 [302:53:29<125:43:02,  6.47s/it]
 69%|██████▉   | 155052/225000 [302:53:36<128:27:39,  6.61s/it]
 69%|██████▉   | 155053/225000 [302:53:40<108:38:03,  5.59s/it]
 69%|██████▉   | 155054/225000 [302:53:49<129:46:10,  6.68s/it]
 69%|██████▉   | 155055/225000 [302:53:52<112:20:48,  5.78s/it]
 69%|██████▉   | 155056/225000 [302:53:56<99:59:46,  5.15s/it] 
 69%|██████▉   | 155057/225000 [302:54:02<101:35:51,  5.23s/it]
 69%|██████▉   | 155058/225000 [302:54:10<120:04:01,  6.18s/it]
 69%|██████▉   | 155059/225000 [302:54:15<111:22:10,  5.73s/it]
 69%|██████▉   | 155060/225000 [302:54:23<125:22:16,  6.45s/it]
 69%|██████▉   | 155061/225000 [302:54:27<110:25:22,  5.68s/it]
 69%|██████▉   | 155062/225000 [302:54:30<98:48:08,  5.09s/it] 
 69%|██████▉   | 155063/225000 [302:54:34<92:36:51,  4.77s/it]
 69%|██████▉   | 155064/225000 [302:54:46<129:37:51,  6.67s/it]
 69%|██████▉   | 155065/225000 [302:54:50<116:14:29,  5.98s/it]
 69%|██████▉   | 155066/225000 [302:54:54<106:41:50,  5.49s/it]
 69%|██████▉   | 155067/225000 [302:54:58<96:38:12,  4.97s/it] 
 69%|██████▉   | 155068/225000 [302:55:03<97:32:26,  5.02s/it]
 69%|██████▉   | 155069/225000 [302:55:14<133:41:13,  6.88s/it]
 69%|██████▉   | 155070/225000 [302:55:18<115:04:49,  5.92s/it]
 69%|██████▉   | 155071/225000 [302:55:21<100:25:18,  5.17s/it]
 69%|██████▉   | 155072/225000 [302:55:31<127:36:37,  6.57s/it]
 69%|██████▉   | 155073/225000 [302:55:35<108:44:06,  5.60s/it]
 69%|██████▉   | 155074/225000 [302:55:41<112:06:04,  5.77s/it]
 69%|██████▉   | 155075/225000 [302:55:47<114:29:22,  5.89s/it]
 69%|██████▉   | 155076/225000 [302:55:55<126:23:13,  6.51s/it]
 69%|██████▉   | 155077/225000 [302:55:58<107:47:21,  5.55s/it]
 69%|██████▉   | 155078/225000 [302:56:05<116:23:36,  5.99s/it]
 69%|██████▉   | 155079/225000 [302:56:09<101:19:13,  5.22s/it]
 69%|██████▉   | 155080/225000 [302:56:13<98:23:48,  5.07s/it] 
 69%|██████▉   | 155081/225000 [302:56:21<111:47:07,  5.76s/it]
 69%|██████▉   | 155082/225000 [302:56:24<98:03:11,  5.05s/it] 
 69%|██████▉   | 155083/225000 [302:56:28<93:55:51,  4.84s/it]
 69%|██████▉   | 155084/225000 [302:56:33<92:58:16,  4.79s/it]
 69%|██████▉   | 155085/225000 [302:56:38<91:12:24,  4.70s/it]
 69%|██████▉   | 155086/225000 [302:56:50<134:21:09,  6.92s/it]
 69%|██████▉   | 155087/225000 [302:56:55<125:12:54,  6.45s/it]
 69%|██████▉   | 155088/225000 [302:56:59<108:42:12,  5.60s/it]
 69%|██████▉   | 155089/225000 [302:57:07<125:44:51,  6.48s/it]
 69%|██████▉   | 155090/225000 [302:57:12<116:05:19,  5.98s/it]
 69%|██████▉   | 155091/225000 [302:57:16<104:56:09,  5.40s/it]
 69%|██████▉   | 155092/225000 [302:57:20<99:00:57,  5.10s/it] 
 69%|██████▉   | 155093/225000 [302:57:25<92:48:50,  4.78s/it]
 69%|██████▉   | 155094/225000 [302:57:34<121:50:30,  6.27s/it]
 69%|██████▉   | 155095/225000 [302:57:38<109:18:15,  5.63s/it]
 69%|██████▉   | 155096/225000 [302:57:42<97:53:02,  5.04s/it] 
 69%|██████▉   | 155097/225000 [302:57:50<117:13:59,  6.04s/it]
 69%|██████▉   | 155098/225000 [302:57:59<134:13:48,  6.91s/it]
 69%|██████▉   | 155099/225000 [302:58:03<112:33:07,  5.80s/it]
 69%|██████▉   | 155100/225000 [302:58:10<119:28:34,  6.15s/it]
                                                               
{'loss': '0.777', 'grad_norm': '1.292', 'learning_rate': '1.197e-05', 'epoch': '0.6893', 'num_input_tokens_seen': 501750896, 'train_runtime': '1.091e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155100/225000 [302:58:10<119:28:34,  6.15s/it]
 69%|██████▉   | 155101/225000 [302:58:13<103:07:15,  5.31s/it]
 69%|██████▉   | 155102/225000 [302:58:22<124:30:11,  6.41s/it]
 69%|██████▉   | 155103/225000 [302:58:26<111:23:13,  5.74s/it]
 69%|██████▉   | 155104/225000 [302:58:31<107:00:56,  5.51s/it]
 69%|██████▉   | 155105/225000 [302:58:36<104:53:08,  5.40s/it]
 69%|██████▉   | 155106/225000 [302:58:41<103:25:11,  5.33s/it]
 69%|██████▉   | 155107/225000 [302:58:46<100:45:33,  5.19s/it]
 69%|██████▉   | 155108/225000 [302:58:51<100:12:54,  5.16s/it]
 69%|██████▉   | 155109/225000 [302:58:57<100:45:48,  5.19s/it]
 69%|██████▉   | 155110/225000 [302:59:02<101:28:06,  5.23s/it]
 69%|██████▉   | 155111/225000 [302:59:07<102:26:36,  5.28s/it]
 69%|██████▉   | 155112/225000 [302:59:13<102:45:02,  5.29s/it]
 69%|██████▉   | 155113/225000 [302:59:17<100:01:43,  5.15s/it]
 69%|██████▉   | 155114/225000 [302:59:23<101:47:09,  5.24s/it]
 69%|██████▉   | 155115/225000 [302:59:28<102:37:44,  5.29s/it]
 69%|██████▉   | 155116/225000 [302:59:33<101:34:16,  5.23s/it]
 69%|██████▉   | 155117/225000 [302:59:39<100:57:52,  5.20s/it]
 69%|██████▉   | 155118/225000 [302:59:44<101:33:25,  5.23s/it]
 69%|██████▉   | 155119/225000 [302:59:49<100:45:56,  5.19s/it]
 69%|██████▉   | 155120/225000 [303:00:02<144:12:01,  7.43s/it]
 69%|██████▉   | 155121/225000 [303:00:08<136:35:16,  7.04s/it]
 69%|██████▉   | 155122/225000 [303:00:11<113:58:13,  5.87s/it]
 69%|██████▉   | 155123/225000 [303:00:21<136:44:43,  7.04s/it]
 69%|██████▉   | 155124/225000 [303:00:24<115:14:01,  5.94s/it]
 69%|██████▉   | 155125/225000 [303:00:32<125:04:31,  6.44s/it]
 69%|██████▉   | 155126/225000 [303:00:36<114:19:48,  5.89s/it]
 69%|██████▉   | 155127/225000 [303:00:41<108:35:02,  5.59s/it]
 69%|██████▉   | 155128/225000 [303:00:46<103:43:05,  5.34s/it]
 69%|██████▉   | 155129/225000 [303:00:51<101:12:34,  5.21s/it]
 69%|██████▉   | 155130/225000 [303:00:55<97:55:35,  5.05s/it] 
 69%|██████▉   | 155131/225000 [303:01:00<93:56:28,  4.84s/it]
 69%|██████▉   | 155132/225000 [303:01:05<95:08:22,  4.90s/it]
 69%|██████▉   | 155133/225000 [303:01:14<117:48:25,  6.07s/it]
 69%|██████▉   | 155134/225000 [303:01:19<115:42:50,  5.96s/it]
 69%|██████▉   | 155135/225000 [303:01:26<120:16:17,  6.20s/it]
 69%|██████▉   | 155136/225000 [303:01:36<142:28:28,  7.34s/it]
 69%|██████▉   | 155137/225000 [303:01:39<119:20:33,  6.15s/it]
 69%|██████▉   | 155138/225000 [303:01:47<130:00:18,  6.70s/it]
 69%|██████▉   | 155139/225000 [303:01:51<110:16:09,  5.68s/it]
 69%|██████▉   | 155140/225000 [303:01:56<107:44:41,  5.55s/it]
 69%|██████▉   | 155141/225000 [303:02:01<105:16:38,  5.43s/it]
 69%|██████▉   | 155142/225000 [303:02:07<105:28:17,  5.44s/it]
 69%|██████▉   | 155143/225000 [303:02:12<105:14:17,  5.42s/it]
 69%|██████▉   | 155144/225000 [303:02:17<103:13:30,  5.32s/it]
 69%|██████▉   | 155145/225000 [303:02:22<102:12:21,  5.27s/it]
 69%|██████▉   | 155146/225000 [303:02:27<102:01:32,  5.26s/it]
 69%|██████▉   | 155147/225000 [303:02:32<100:10:56,  5.16s/it]
 69%|██████▉   | 155148/225000 [303:02:38<101:05:36,  5.21s/it]
 69%|██████▉   | 155149/225000 [303:02:43<99:20:00,  5.12s/it] 
 69%|██████▉   | 155150/225000 [303:02:48<99:27:53,  5.13s/it]
                                                              
{'loss': '0.7898', 'grad_norm': '1.222', 'learning_rate': '1.195e-05', 'epoch': '0.6896', 'num_input_tokens_seen': 501880944, 'train_runtime': '1.091e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155150/225000 [303:02:48<99:27:53,  5.13s/it]
 69%|██████▉   | 155151/225000 [303:02:57<125:32:46,  6.47s/it]
 69%|██████▉   | 155152/225000 [303:03:06<139:16:52,  7.18s/it]
 69%|██████▉   | 155153/225000 [303:03:12<129:43:36,  6.69s/it]
 69%|██████▉   | 155154/225000 [303:03:16<115:10:01,  5.94s/it]
 69%|██████▉   | 155155/225000 [303:03:20<104:16:49,  5.37s/it]
 69%|██████▉   | 155156/225000 [303:03:24<95:16:50,  4.91s/it] 
 69%|██████▉   | 155157/225000 [303:03:30<105:26:00,  5.43s/it]
 69%|██████▉   | 155158/225000 [303:03:41<133:17:46,  6.87s/it]
 69%|██████▉   | 155159/225000 [303:03:44<114:06:38,  5.88s/it]
 69%|██████▉   | 155160/225000 [303:03:48<100:40:51,  5.19s/it]
 69%|██████▉   | 155161/225000 [303:03:54<106:20:08,  5.48s/it]
 69%|██████▉   | 155162/225000 [303:04:00<110:26:16,  5.69s/it]
 69%|██████▉   | 155163/225000 [303:04:04<98:49:41,  5.09s/it] 
 69%|██████▉   | 155164/225000 [303:04:08<94:18:53,  4.86s/it]
 69%|██████▉   | 155165/225000 [303:04:12<90:36:05,  4.67s/it]
 69%|██████▉   | 155166/225000 [303:04:25<135:40:25,  6.99s/it]
 69%|██████▉   | 155167/225000 [303:04:29<120:01:49,  6.19s/it]
 69%|██████▉   | 155168/225000 [303:04:34<109:27:10,  5.64s/it]
 69%|██████▉   | 155169/225000 [303:04:38<100:32:07,  5.18s/it]
 69%|██████▉   | 155170/225000 [303:04:42<94:11:47,  4.86s/it] 
 69%|██████▉   | 155171/225000 [303:04:49<108:02:48,  5.57s/it]
 69%|██████▉   | 155172/225000 [303:04:58<128:33:07,  6.63s/it]
 69%|██████▉   | 155173/225000 [303:05:02<112:01:35,  5.78s/it]
 69%|██████▉   | 155174/225000 [303:05:05<99:07:29,  5.11s/it] 
 69%|██████▉   | 155175/225000 [303:05:10<96:59:17,  5.00s/it]
 69%|██████▉   | 155176/225000 [303:05:16<99:40:11,  5.14s/it]
 69%|██████▉   | 155177/225000 [303:05:21<102:29:11,  5.28s/it]
 69%|██████▉   | 155178/225000 [303:05:27<104:01:27,  5.36s/it]
 69%|██████▉   | 155179/225000 [303:05:32<104:48:54,  5.40s/it]
 69%|██████▉   | 155180/225000 [303:05:38<105:11:51,  5.42s/it]
 69%|██████▉   | 155181/225000 [303:05:43<105:11:08,  5.42s/it]
 69%|██████▉   | 155182/225000 [303:05:48<103:38:15,  5.34s/it]
 69%|██████▉   | 155183/225000 [303:05:53<101:12:29,  5.22s/it]
 69%|██████▉   | 155184/225000 [303:06:05<141:03:58,  7.27s/it]
 69%|██████▉   | 155185/225000 [303:06:11<130:48:56,  6.75s/it]
 69%|██████▉   | 155186/225000 [303:06:17<130:09:07,  6.71s/it]
 69%|██████▉   | 155187/225000 [303:06:22<114:51:48,  5.92s/it]
 69%|██████▉   | 155188/225000 [303:06:26<106:17:31,  5.48s/it]
 69%|██████▉   | 155189/225000 [303:06:31<102:12:14,  5.27s/it]
 69%|██████▉   | 155190/225000 [303:06:35<98:57:44,  5.10s/it] 
 69%|██████▉   | 155191/225000 [303:06:40<95:13:39,  4.91s/it]
 69%|██████▉   | 155192/225000 [303:06:44<92:16:49,  4.76s/it]
 69%|██████▉   | 155193/225000 [303:06:51<100:43:39,  5.19s/it]
 69%|██████▉   | 155194/225000 [303:07:02<137:31:03,  7.09s/it]
 69%|██████▉   | 155195/225000 [303:07:06<121:13:47,  6.25s/it]
 69%|██████▉   | 155196/225000 [303:07:11<109:48:33,  5.66s/it]
 69%|██████▉   | 155197/225000 [303:07:15<101:59:36,  5.26s/it]
 69%|██████▉   | 155198/225000 [303:07:19<97:06:28,  5.01s/it] 
 69%|██████▉   | 155199/225000 [303:07:29<123:29:45,  6.37s/it]
 69%|██████▉   | 155200/225000 [303:07:37<131:26:19,  6.78s/it]
                                                               
{'loss': '0.779', 'grad_norm': '1.182', 'learning_rate': '1.194e-05', 'epoch': '0.6898', 'num_input_tokens_seen': 502011088, 'train_runtime': '1.091e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155200/225000 [303:07:37<131:26:19,  6.78s/it]
 69%|██████▉   | 155201/225000 [303:07:40<112:44:04,  5.81s/it]
 69%|██████▉   | 155202/225000 [303:07:44<101:59:46,  5.26s/it]
 69%|██████▉   | 155203/225000 [303:07:54<128:29:34,  6.63s/it]
 69%|██████▉   | 155204/225000 [303:07:57<110:03:00,  5.68s/it]
 69%|██████▉   | 155205/225000 [303:08:02<101:05:40,  5.21s/it]
 69%|██████▉   | 155206/225000 [303:08:10<119:04:16,  6.14s/it]
 69%|██████▉   | 155207/225000 [303:08:14<105:38:03,  5.45s/it]
 69%|██████▉   | 155208/225000 [303:08:18<98:35:38,  5.09s/it] 
 69%|██████▉   | 155209/225000 [303:08:23<95:22:31,  4.92s/it]
 69%|██████▉   | 155210/225000 [303:08:27<92:14:38,  4.76s/it]
 69%|██████▉   | 155211/225000 [303:08:36<120:13:03,  6.20s/it]
 69%|██████▉   | 155212/225000 [303:08:42<114:30:17,  5.91s/it]
 69%|██████▉   | 155213/225000 [303:08:47<108:13:58,  5.58s/it]
 69%|██████▉   | 155214/225000 [303:08:51<102:25:47,  5.28s/it]
 69%|██████▉   | 155215/225000 [303:08:56<98:00:14,  5.06s/it] 
 69%|██████▉   | 155216/225000 [303:09:01<97:24:26,  5.03s/it]
 69%|██████▉   | 155217/225000 [303:09:05<96:37:36,  4.98s/it]
 69%|██████▉   | 155218/225000 [303:09:10<95:21:07,  4.92s/it]
 69%|██████▉   | 155219/225000 [303:09:22<136:29:14,  7.04s/it]
 69%|██████▉   | 155220/225000 [303:09:27<124:55:42,  6.45s/it]
 69%|██████▉   | 155221/225000 [303:09:31<111:02:44,  5.73s/it]
 69%|██████▉   | 155222/225000 [303:09:39<123:59:20,  6.40s/it]
 69%|██████▉   | 155223/225000 [303:09:46<122:45:51,  6.33s/it]
 69%|██████▉   | 155224/225000 [303:09:49<108:54:34,  5.62s/it]
 69%|██████▉   | 155225/225000 [303:09:54<103:33:21,  5.34s/it]
 69%|██████▉   | 155226/225000 [303:09:58<96:45:45,  4.99s/it] 
 69%|██████▉   | 155227/225000 [303:10:03<94:58:57,  4.90s/it]
 69%|██████▉   | 155228/225000 [303:10:14<130:24:09,  6.73s/it]
 69%|██████▉   | 155229/225000 [303:10:21<130:47:08,  6.75s/it]
 69%|██████▉   | 155230/225000 [303:10:24<112:43:02,  5.82s/it]
 69%|██████▉   | 155231/225000 [303:10:33<130:55:10,  6.76s/it]
 69%|██████▉   | 155232/225000 [303:10:37<110:31:38,  5.70s/it]
 69%|██████▉   | 155233/225000 [303:10:43<112:01:36,  5.78s/it]
 69%|██████▉   | 155234/225000 [303:10:51<130:07:00,  6.71s/it]
 69%|██████▉   | 155235/225000 [303:10:54<108:22:14,  5.59s/it]
 69%|██████▉   | 155236/225000 [303:11:02<118:28:00,  6.11s/it]
 69%|██████▉   | 155237/225000 [303:11:07<111:45:21,  5.77s/it]
 69%|██████▉   | 155238/225000 [303:11:12<106:13:58,  5.48s/it]
 69%|██████▉   | 155239/225000 [303:11:17<103:26:28,  5.34s/it]
 69%|██████▉   | 155240/225000 [303:11:21<99:29:00,  5.13s/it] 
 69%|██████▉   | 155241/225000 [303:11:26<98:05:56,  5.06s/it]
 69%|██████▉   | 155242/225000 [303:11:31<96:07:35,  4.96s/it]
 69%|██████▉   | 155243/225000 [303:11:36<96:07:02,  4.96s/it]
 69%|██████▉   | 155244/225000 [303:11:41<96:05:14,  4.96s/it]
 69%|██████▉   | 155245/225000 [303:11:46<95:24:54,  4.92s/it]
 69%|██████▉   | 155246/225000 [303:11:58<139:34:45,  7.20s/it]
 69%|██████▉   | 155247/225000 [303:12:06<144:19:02,  7.45s/it]
 69%|██████▉   | 155248/225000 [303:12:10<122:52:00,  6.34s/it]
 69%|██████▉   | 155249/225000 [303:12:14<108:24:53,  5.60s/it]
 69%|██████▉   | 155250/225000 [303:12:19<108:25:06,  5.60s/it]
                                                               
{'loss': '0.7824', 'grad_norm': '1.136', 'learning_rate': '1.192e-05', 'epoch': '0.69', 'num_input_tokens_seen': 502141328, 'train_runtime': '1.092e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155250/225000 [303:12:19<108:25:06,  5.60s/it]
 69%|██████▉   | 155251/225000 [303:12:28<127:03:12,  6.56s/it]
 69%|██████▉   | 155252/225000 [303:12:32<109:03:30,  5.63s/it]
 69%|██████▉   | 155253/225000 [303:12:35<98:18:48,  5.07s/it] 
 69%|██████▉   | 155254/225000 [303:12:43<112:22:58,  5.80s/it]
 69%|██████▉   | 155255/225000 [303:12:49<116:49:48,  6.03s/it]
 69%|██████▉   | 155256/225000 [303:12:53<103:14:43,  5.33s/it]
 69%|██████▉   | 155257/225000 [303:13:02<122:36:27,  6.33s/it]
 69%|██████▉   | 155258/225000 [303:13:05<105:33:03,  5.45s/it]
 69%|██████▉   | 155259/225000 [303:13:14<125:22:41,  6.47s/it]
 69%|██████▉   | 155260/225000 [303:13:18<110:47:19,  5.72s/it]
 69%|██████▉   | 155261/225000 [303:13:22<99:33:47,  5.14s/it] 
 69%|██████▉   | 155262/225000 [303:13:26<94:55:10,  4.90s/it]
 69%|██████▉   | 155263/225000 [303:13:31<95:00:37,  4.90s/it]
 69%|██████▉   | 155264/225000 [303:13:36<93:17:23,  4.82s/it]
 69%|██████▉   | 155265/225000 [303:13:41<94:48:08,  4.89s/it]
 69%|██████▉   | 155266/225000 [303:13:45<93:20:00,  4.82s/it]
 69%|██████▉   | 155267/225000 [303:13:55<119:57:18,  6.19s/it]
 69%|██████▉   | 155268/225000 [303:14:04<137:01:00,  7.07s/it]
 69%|██████▉   | 155269/225000 [303:14:09<123:50:15,  6.39s/it]
 69%|██████▉   | 155270/225000 [303:14:13<113:25:34,  5.86s/it]
 69%|██████▉   | 155271/225000 [303:14:18<104:07:34,  5.38s/it]
 69%|██████▉   | 155272/225000 [303:14:23<102:58:23,  5.32s/it]
 69%|██████▉   | 155273/225000 [303:14:27<99:02:25,  5.11s/it] 
 69%|██████▉   | 155274/225000 [303:14:32<94:55:50,  4.90s/it]
 69%|██████▉   | 155275/225000 [303:14:44<137:19:48,  7.09s/it]
 69%|██████▉   | 155276/225000 [303:14:49<126:53:50,  6.55s/it]
 69%|██████▉   | 155277/225000 [303:14:56<128:28:40,  6.63s/it]
 69%|██████▉   | 155278/225000 [303:15:01<116:15:51,  6.00s/it]
 69%|██████▉   | 155279/225000 [303:15:05<108:38:18,  5.61s/it]
 69%|██████▉   | 155280/225000 [303:15:10<101:56:11,  5.26s/it]
 69%|██████▉   | 155281/225000 [303:15:15<98:37:12,  5.09s/it] 
 69%|██████▉   | 155282/225000 [303:15:22<114:21:56,  5.91s/it]
 69%|██████▉   | 155283/225000 [303:15:31<133:09:33,  6.88s/it]
 69%|██████▉   | 155284/225000 [303:15:35<114:51:41,  5.93s/it]
 69%|██████▉   | 155285/225000 [303:15:42<118:18:48,  6.11s/it]
 69%|██████▉   | 155286/225000 [303:15:48<121:19:37,  6.27s/it]
 69%|██████▉   | 155287/225000 [303:15:52<105:59:47,  5.47s/it]
 69%|██████▉   | 155288/225000 [303:16:02<131:08:04,  6.77s/it]
 69%|██████▉   | 155289/225000 [303:16:06<116:15:05,  6.00s/it]
 69%|██████▉   | 155290/225000 [303:16:11<109:48:31,  5.67s/it]
 69%|██████▉   | 155291/225000 [303:16:15<102:56:53,  5.32s/it]
 69%|██████▉   | 155292/225000 [303:16:19<95:29:56,  4.93s/it] 
 69%|██████▉   | 155293/225000 [303:16:24<91:07:12,  4.71s/it]
 69%|██████▉   | 155294/225000 [303:16:36<134:47:08,  6.96s/it]
 69%|██████▉   | 155295/225000 [303:16:40<120:01:20,  6.20s/it]
 69%|██████▉   | 155296/225000 [303:16:44<106:24:18,  5.50s/it]
 69%|██████▉   | 155297/225000 [303:16:48<100:06:29,  5.17s/it]
 69%|██████▉   | 155298/225000 [303:16:53<98:00:27,  5.06s/it] 
 69%|██████▉   | 155299/225000 [303:17:05<139:01:53,  7.18s/it]
 69%|██████▉   | 155300/225000 [303:17:09<117:52:46,  6.09s/it]
                                                               
{'loss': '0.7852', 'grad_norm': '1.19', 'learning_rate': '1.191e-05', 'epoch': '0.6902', 'num_input_tokens_seen': 502270824, 'train_runtime': '1.092e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155300/225000 [303:17:09<117:52:46,  6.09s/it]
 69%|██████▉   | 155301/225000 [303:17:12<102:49:10,  5.31s/it]
 69%|██████▉   | 155302/225000 [303:17:21<123:21:38,  6.37s/it]
 69%|██████▉   | 155303/225000 [303:17:27<119:20:46,  6.16s/it]
 69%|██████▉   | 155304/225000 [303:17:31<109:42:19,  5.67s/it]
 69%|██████▉   | 155305/225000 [303:17:36<100:53:58,  5.21s/it]
 69%|██████▉   | 155306/225000 [303:17:40<96:04:22,  4.96s/it] 
 69%|██████▉   | 155307/225000 [303:17:44<91:19:02,  4.72s/it]
 69%|██████▉   | 155308/225000 [303:17:55<127:40:31,  6.60s/it]
 69%|██████▉   | 155309/225000 [303:17:59<113:43:59,  5.88s/it]
 69%|██████▉   | 155310/225000 [303:18:07<125:26:23,  6.48s/it]
 69%|██████▉   | 155311/225000 [303:18:12<114:53:55,  5.94s/it]
 69%|██████▉   | 155312/225000 [303:18:16<104:51:41,  5.42s/it]
 69%|██████▉   | 155313/225000 [303:18:22<106:37:48,  5.51s/it]
 69%|██████▉   | 155314/225000 [303:18:27<105:53:59,  5.47s/it]
 69%|██████▉   | 155315/225000 [303:18:37<131:57:46,  6.82s/it]
 69%|██████▉   | 155316/225000 [303:18:42<118:24:51,  6.12s/it]
 69%|██████▉   | 155317/225000 [303:18:46<106:01:19,  5.48s/it]
 69%|██████▉   | 155318/225000 [303:18:50<99:22:11,  5.13s/it] 
 69%|██████▉   | 155319/225000 [303:18:55<96:33:32,  4.99s/it]
 69%|██████▉   | 155320/225000 [303:19:07<137:05:51,  7.08s/it]
 69%|██████▉   | 155321/225000 [303:19:12<129:38:49,  6.70s/it]
 69%|██████▉   | 155322/225000 [303:19:17<117:10:42,  6.05s/it]
 69%|██████▉   | 155323/225000 [303:19:22<111:47:00,  5.78s/it]
 69%|██████▉   | 155324/225000 [303:19:27<107:12:59,  5.54s/it]
 69%|██████▉   | 155325/225000 [303:19:32<103:57:08,  5.37s/it]
 69%|██████▉   | 155326/225000 [303:19:37<100:03:44,  5.17s/it]
 69%|██████▉   | 155327/225000 [303:19:42<100:39:48,  5.20s/it]
 69%|██████▉   | 155328/225000 [303:19:53<134:50:31,  6.97s/it]
 69%|██████▉   | 155329/225000 [303:19:59<131:01:13,  6.77s/it]
 69%|██████▉   | 155330/225000 [303:20:07<134:41:35,  6.96s/it]
 69%|██████▉   | 155331/225000 [303:20:12<125:08:57,  6.47s/it]
 69%|██████▉   | 155332/225000 [303:20:16<110:52:43,  5.73s/it]
 69%|██████▉   | 155333/225000 [303:20:20<101:14:48,  5.23s/it]
 69%|██████▉   | 155334/225000 [303:20:27<113:00:56,  5.84s/it]
 69%|██████▉   | 155335/225000 [303:20:36<126:56:07,  6.56s/it]
 69%|██████▉   | 155336/225000 [303:20:40<111:52:32,  5.78s/it]
 69%|██████▉   | 155337/225000 [303:20:48<125:40:17,  6.49s/it]
 69%|██████▉   | 155338/225000 [303:20:52<114:35:19,  5.92s/it]
 69%|██████▉   | 155339/225000 [303:20:57<106:23:13,  5.50s/it]
 69%|██████▉   | 155340/225000 [303:21:02<101:09:15,  5.23s/it]
 69%|██████▉   | 155341/225000 [303:21:06<98:28:43,  5.09s/it] 
 69%|██████▉   | 155342/225000 [303:21:12<99:45:10,  5.16s/it]
 69%|██████▉   | 155343/225000 [303:21:16<95:59:38,  4.96s/it]
 69%|██████▉   | 155344/225000 [303:21:21<94:22:19,  4.88s/it]
 69%|██████▉   | 155345/225000 [303:21:34<141:28:38,  7.31s/it]
 69%|██████▉   | 155346/225000 [303:21:38<124:04:49,  6.41s/it]
 69%|██████▉   | 155347/225000 [303:21:45<127:19:15,  6.58s/it]
 69%|██████▉   | 155348/225000 [303:21:49<114:11:31,  5.90s/it]
 69%|██████▉   | 155349/225000 [303:21:54<104:12:02,  5.39s/it]
 69%|██████▉   | 155350/225000 [303:21:58<99:16:48,  5.13s/it] 
                                                              
{'loss': '0.787', 'grad_norm': '1.023', 'learning_rate': '1.189e-05', 'epoch': '0.6904', 'num_input_tokens_seen': 502402456, 'train_runtime': '1.092e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155350/225000 [303:21:58<99:16:48,  5.13s/it]
 69%|██████▉   | 155351/225000 [303:22:10<140:47:47,  7.28s/it]
 69%|██████▉   | 155352/225000 [303:22:15<127:22:57,  6.58s/it]
 69%|██████▉   | 155353/225000 [303:22:20<118:24:31,  6.12s/it]
 69%|██████▉   | 155354/225000 [303:22:25<110:14:27,  5.70s/it]
 69%|██████▉   | 155355/225000 [303:22:30<104:53:39,  5.42s/it]
 69%|██████▉   | 155356/225000 [303:22:35<104:00:49,  5.38s/it]
 69%|██████▉   | 155357/225000 [303:22:40<100:47:38,  5.21s/it]
 69%|██████▉   | 155358/225000 [303:22:45<98:59:19,  5.12s/it] 
 69%|██████▉   | 155359/225000 [303:22:52<111:50:42,  5.78s/it]
 69%|██████▉   | 155360/225000 [303:23:02<135:00:47,  6.98s/it]
 69%|██████▉   | 155361/225000 [303:23:07<124:03:39,  6.41s/it]
 69%|██████▉   | 155362/225000 [303:23:12<115:30:45,  5.97s/it]
 69%|██████▉   | 155363/225000 [303:23:17<109:37:44,  5.67s/it]
 69%|██████▉   | 155364/225000 [303:23:22<104:25:30,  5.40s/it]
 69%|██████▉   | 155365/225000 [303:23:27<101:34:51,  5.25s/it]
 69%|██████▉   | 155366/225000 [303:23:32<99:20:41,  5.14s/it] 
 69%|██████▉   | 155367/225000 [303:23:36<94:55:21,  4.91s/it]
 69%|██████▉   | 155368/225000 [303:23:41<96:44:58,  5.00s/it]
 69%|██████▉   | 155369/225000 [303:23:47<99:28:07,  5.14s/it]
 69%|██████▉   | 155370/225000 [303:23:58<135:22:29,  7.00s/it]
 69%|██████▉   | 155371/225000 [303:24:05<133:55:57,  6.92s/it]
 69%|██████▉   | 155372/225000 [303:24:08<115:18:58,  5.96s/it]
 69%|██████▉   | 155373/225000 [303:24:12<103:17:46,  5.34s/it]
 69%|██████▉   | 155374/225000 [303:24:16<92:40:04,  4.79s/it] 
 69%|██████▉   | 155375/225000 [303:24:26<123:31:58,  6.39s/it]
 69%|██████▉   | 155376/225000 [303:24:32<123:50:59,  6.40s/it]
 69%|██████▉   | 155377/225000 [303:24:40<130:59:16,  6.77s/it]
 69%|██████▉   | 155378/225000 [303:24:45<121:17:03,  6.27s/it]
 69%|██████▉   | 155379/225000 [303:24:48<104:42:51,  5.41s/it]
 69%|██████▉   | 155380/225000 [303:24:57<122:31:20,  6.34s/it]
 69%|██████▉   | 155381/225000 [303:25:00<106:09:32,  5.49s/it]
 69%|██████▉   | 155382/225000 [303:25:04<94:34:09,  4.89s/it] 
 69%|██████▉   | 155383/225000 [303:25:14<123:43:25,  6.40s/it]
 69%|██████▉   | 155384/225000 [303:25:17<106:45:36,  5.52s/it]
 69%|██████▉   | 155385/225000 [303:25:21<97:54:15,  5.06s/it] 
 69%|██████▉   | 155386/225000 [303:25:26<94:19:42,  4.88s/it]
 69%|██████▉   | 155387/225000 [303:25:36<123:25:59,  6.38s/it]
 69%|██████▉   | 155388/225000 [303:25:41<117:08:16,  6.06s/it]
 69%|██████▉   | 155389/225000 [303:25:48<123:41:23,  6.40s/it]
 69%|██████▉   | 155390/225000 [303:25:52<106:01:47,  5.48s/it]
 69%|██████▉   | 155391/225000 [303:25:57<107:53:21,  5.58s/it]
 69%|██████▉   | 155392/225000 [303:26:07<133:54:56,  6.93s/it]
 69%|██████▉   | 155393/225000 [303:26:11<114:15:24,  5.91s/it]
 69%|██████▉   | 155394/225000 [303:26:15<103:38:37,  5.36s/it]
 69%|██████▉   | 155395/225000 [303:26:23<120:52:48,  6.25s/it]
 69%|██████▉   | 155396/225000 [303:26:29<114:46:41,  5.94s/it]
 69%|██████▉   | 155397/225000 [303:26:33<104:48:17,  5.42s/it]
 69%|██████▉   | 155398/225000 [303:26:37<98:12:51,  5.08s/it] 
 69%|██████▉   | 155399/225000 [303:26:42<98:15:20,  5.08s/it]
 69%|██████▉   | 155400/225000 [303:26:52<123:25:28,  6.38s/it]
                                                               
{'loss': '0.7757', 'grad_norm': '1.122', 'learning_rate': '1.187e-05', 'epoch': '0.6907', 'num_input_tokens_seen': 502531880, 'train_runtime': '1.092e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155400/225000 [303:26:52<123:25:28,  6.38s/it]
 69%|██████▉   | 155401/225000 [303:26:55<108:38:01,  5.62s/it]
 69%|██████▉   | 155402/225000 [303:26:59<97:43:10,  5.05s/it] 
 69%|██████▉   | 155403/225000 [303:27:08<117:15:43,  6.07s/it]
 69%|██████▉   | 155404/225000 [303:27:13<113:44:17,  5.88s/it]
 69%|██████▉   | 155405/225000 [303:27:20<119:30:52,  6.18s/it]
 69%|██████▉   | 155406/225000 [303:27:24<109:07:40,  5.65s/it]
 69%|██████▉   | 155407/225000 [303:27:31<115:16:46,  5.96s/it]
 69%|██████▉   | 155408/225000 [303:27:35<102:51:31,  5.32s/it]
 69%|██████▉   | 155409/225000 [303:27:42<111:25:57,  5.76s/it]
 69%|██████▉   | 155410/225000 [303:27:47<109:19:14,  5.66s/it]
 69%|██████▉   | 155411/225000 [303:27:51<97:07:03,  5.02s/it] 
 69%|██████▉   | 155412/225000 [303:28:00<122:25:21,  6.33s/it]
 69%|██████▉   | 155413/225000 [303:28:04<109:35:22,  5.67s/it]
 69%|██████▉   | 155414/225000 [303:28:08<98:41:42,  5.11s/it] 
 69%|██████▉   | 155415/225000 [303:28:15<109:52:38,  5.68s/it]
 69%|██████▉   | 155416/225000 [303:28:22<120:43:30,  6.25s/it]
 69%|██████▉   | 155417/225000 [303:28:26<104:16:02,  5.39s/it]
 69%|██████▉   | 155418/225000 [303:28:30<99:24:19,  5.14s/it] 
 69%|██████▉   | 155419/225000 [303:28:41<128:33:34,  6.65s/it]
 69%|██████▉   | 155420/225000 [303:28:44<108:44:04,  5.63s/it]
 69%|██████▉   | 155421/225000 [303:28:49<103:40:00,  5.36s/it]
 69%|██████▉   | 155422/225000 [303:28:53<101:01:00,  5.23s/it]
 69%|██████▉   | 155423/225000 [303:28:58<97:03:35,  5.02s/it] 
 69%|██████▉   | 155424/225000 [303:29:03<94:43:42,  4.90s/it]
 69%|██████▉   | 155425/225000 [303:29:07<91:29:32,  4.73s/it]
 69%|██████▉   | 155426/225000 [303:29:12<95:33:45,  4.94s/it]
 69%|██████▉   | 155427/225000 [303:29:25<141:55:18,  7.34s/it]
 69%|██████▉   | 155428/225000 [303:29:29<119:14:08,  6.17s/it]
 69%|██████▉   | 155429/225000 [303:29:35<120:37:51,  6.24s/it]
 69%|██████▉   | 155430/225000 [303:29:39<105:29:44,  5.46s/it]
 69%|██████▉   | 155431/225000 [303:29:43<100:31:22,  5.20s/it]
 69%|██████▉   | 155432/225000 [303:29:47<93:16:24,  4.83s/it] 
 69%|██████▉   | 155433/225000 [303:29:52<94:10:01,  4.87s/it]
 69%|██████▉   | 155434/225000 [303:30:01<114:03:14,  5.90s/it]
 69%|██████▉   | 155435/225000 [303:30:11<137:54:37,  7.14s/it]
 69%|██████▉   | 155436/225000 [303:30:14<115:43:47,  5.99s/it]
 69%|██████▉   | 155437/225000 [303:30:21<120:39:38,  6.24s/it]
 69%|██████▉   | 155438/225000 [303:30:26<112:29:28,  5.82s/it]
 69%|██████▉   | 155439/225000 [303:30:30<103:23:55,  5.35s/it]
 69%|██████▉   | 155440/225000 [303:30:34<97:33:03,  5.05s/it] 
 69%|██████▉   | 155441/225000 [303:30:39<93:08:10,  4.82s/it]
 69%|██████▉   | 155442/225000 [303:30:49<127:16:50,  6.59s/it]
 69%|██████▉   | 155443/225000 [303:30:55<119:41:04,  6.19s/it]
 69%|██████▉   | 155444/225000 [303:30:58<104:54:38,  5.43s/it]
 69%|██████▉   | 155445/225000 [303:31:02<94:15:54,  4.88s/it] 
 69%|██████▉   | 155446/225000 [303:31:12<126:03:56,  6.52s/it]
 69%|██████▉   | 155447/225000 [303:31:16<110:43:10,  5.73s/it]
 69%|██████▉   | 155448/225000 [303:31:20<100:14:22,  5.19s/it]
 69%|██████▉   | 155449/225000 [303:31:24<91:26:56,  4.73s/it] 
 69%|██████▉   | 155450/225000 [303:31:35<129:40:04,  6.71s/it]
                                                               
{'loss': '0.782', 'grad_norm': '1.069', 'learning_rate': '1.186e-05', 'epoch': '0.6909', 'num_input_tokens_seen': 502660720, 'train_runtime': '1.093e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155450/225000 [303:31:35<129:40:04,  6.71s/it]
 69%|██████▉   | 155451/225000 [303:31:39<112:44:49,  5.84s/it]
 69%|██████▉   | 155452/225000 [303:31:43<102:56:34,  5.33s/it]
 69%|██████▉   | 155453/225000 [303:31:47<96:11:45,  4.98s/it] 
 69%|██████▉   | 155454/225000 [303:31:51<91:44:42,  4.75s/it]
 69%|██████▉   | 155455/225000 [303:31:56<92:56:20,  4.81s/it]
 69%|██████▉   | 155456/225000 [303:32:02<99:17:13,  5.14s/it]
 69%|██████▉   | 155457/225000 [303:32:14<138:40:52,  7.18s/it]
 69%|██████▉   | 155458/225000 [303:32:17<115:40:17,  5.99s/it]
 69%|██████▉   | 155459/225000 [303:32:22<105:32:36,  5.46s/it]
 69%|██████▉   | 155460/225000 [303:32:27<107:24:58,  5.56s/it]
 69%|██████▉   | 155461/225000 [303:32:34<112:22:19,  5.82s/it]
 69%|██████▉   | 155462/225000 [303:32:41<118:54:34,  6.16s/it]
 69%|██████▉   | 155463/225000 [303:32:49<133:23:14,  6.91s/it]
 69%|██████▉   | 155464/225000 [303:32:53<114:35:48,  5.93s/it]
 69%|██████▉   | 155465/225000 [303:32:57<101:43:22,  5.27s/it]
 69%|██████▉   | 155466/225000 [303:33:07<128:52:53,  6.67s/it]
 69%|██████▉   | 155467/225000 [303:33:11<117:11:41,  6.07s/it]
 69%|██████▉   | 155468/225000 [303:33:15<103:33:57,  5.36s/it]
 69%|██████▉   | 155469/225000 [303:33:19<95:04:46,  4.92s/it] 
 69%|██████▉   | 155470/225000 [303:33:23<91:25:23,  4.73s/it]
 69%|██████▉   | 155471/225000 [303:33:35<129:40:05,  6.71s/it]
 69%|██████▉   | 155472/225000 [303:33:39<116:19:44,  6.02s/it]
 69%|██████▉   | 155473/225000 [303:33:43<103:18:15,  5.35s/it]
 69%|██████▉   | 155474/225000 [303:33:53<130:23:10,  6.75s/it]
 69%|██████▉   | 155475/225000 [303:33:59<127:35:03,  6.61s/it]
 69%|██████▉   | 155476/225000 [303:34:03<112:33:13,  5.83s/it]
 69%|██████▉   | 155477/225000 [303:34:07<101:57:51,  5.28s/it]
 69%|██████▉   | 155478/225000 [303:34:14<108:50:24,  5.64s/it]
 69%|██████▉   | 155479/225000 [303:34:21<118:57:23,  6.16s/it]
 69%|██████▉   | 155480/225000 [303:34:25<108:37:50,  5.63s/it]
 69%|██████▉   | 155481/225000 [303:34:30<101:15:29,  5.24s/it]
 69%|██████▉   | 155482/225000 [303:34:34<94:25:26,  4.89s/it] 
 69%|██████▉   | 155483/225000 [303:34:38<90:19:10,  4.68s/it]
 69%|██████▉   | 155484/225000 [303:34:47<115:37:36,  5.99s/it]
 69%|██████▉   | 155485/225000 [303:34:55<124:50:39,  6.47s/it]
 69%|██████▉   | 155486/225000 [303:34:58<105:47:29,  5.48s/it]
 69%|██████▉   | 155487/225000 [303:35:06<119:22:49,  6.18s/it]
 69%|██████▉   | 155488/225000 [303:35:10<110:25:42,  5.72s/it]
 69%|██████▉   | 155489/225000 [303:35:15<107:21:20,  5.56s/it]
 69%|██████▉   | 155490/225000 [303:35:20<103:53:22,  5.38s/it]
 69%|██████▉   | 155491/225000 [303:35:25<101:03:07,  5.23s/it]
 69%|██████▉   | 155492/225000 [303:35:30<99:05:54,  5.13s/it] 
 69%|██████▉   | 155493/225000 [303:35:35<98:54:56,  5.12s/it]
 69%|██████▉   | 155494/225000 [303:35:40<99:34:47,  5.16s/it]
 69%|██████▉   | 155495/225000 [303:35:45<99:01:17,  5.13s/it]
 69%|██████▉   | 155496/225000 [303:35:57<137:05:35,  7.10s/it]
 69%|██████▉   | 155497/225000 [303:36:03<130:01:35,  6.73s/it]
 69%|██████▉   | 155498/225000 [303:36:07<113:23:49,  5.87s/it]
 69%|██████▉   | 155499/225000 [303:36:11<102:33:11,  5.31s/it]
 69%|██████▉   | 155500/225000 [303:36:20<126:45:41,  6.57s/it]
                                                               
{'loss': '0.7877', 'grad_norm': '1.089', 'learning_rate': '1.184e-05', 'epoch': '0.6911', 'num_input_tokens_seen': 502790768, 'train_runtime': '1.093e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155500/225000 [303:36:20<126:45:41,  6.57s/it]
 69%|██████▉   | 155501/225000 [303:36:25<113:35:58,  5.88s/it]
 69%|██████▉   | 155502/225000 [303:36:32<121:28:28,  6.29s/it]
 69%|██████▉   | 155503/225000 [303:36:35<104:53:59,  5.43s/it]
 69%|██████▉   | 155504/225000 [303:36:40<99:39:35,  5.16s/it] 
 69%|██████▉   | 155505/225000 [303:36:45<97:39:18,  5.06s/it]
 69%|██████▉   | 155506/225000 [303:36:50<99:23:37,  5.15s/it]
 69%|██████▉   | 155507/225000 [303:36:55<99:35:39,  5.16s/it]
 69%|██████▉   | 155508/225000 [303:37:01<101:20:53,  5.25s/it]
 69%|██████▉   | 155509/225000 [303:37:06<102:31:47,  5.31s/it]
 69%|██████▉   | 155510/225000 [303:37:12<103:43:35,  5.37s/it]
 69%|██████▉   | 155511/225000 [303:37:17<100:49:15,  5.22s/it]
 69%|██████▉   | 155512/225000 [303:37:22<101:18:32,  5.25s/it]
 69%|██████▉   | 155513/225000 [303:37:27<102:41:28,  5.32s/it]
 69%|██████▉   | 155514/225000 [303:37:33<101:45:33,  5.27s/it]
 69%|██████▉   | 155515/225000 [303:37:38<100:54:55,  5.23s/it]
 69%|██████▉   | 155516/225000 [303:37:43<99:44:43,  5.17s/it] 
 69%|██████▉   | 155517/225000 [303:37:48<99:13:48,  5.14s/it]
 69%|██████▉   | 155518/225000 [303:37:53<98:20:11,  5.10s/it]
 69%|██████▉   | 155519/225000 [303:38:02<122:49:30,  6.36s/it]
 69%|██████▉   | 155520/225000 [303:38:07<116:09:39,  6.02s/it]
 69%|██████▉   | 155521/225000 [303:38:12<110:21:01,  5.72s/it]
 69%|██████▉   | 155522/225000 [303:38:18<108:27:41,  5.62s/it]
 69%|██████▉   | 155523/225000 [303:38:23<106:19:46,  5.51s/it]
 69%|██████▉   | 155524/225000 [303:38:28<101:51:36,  5.28s/it]
 69%|██████▉   | 155525/225000 [303:38:33<99:05:19,  5.13s/it] 
 69%|██████▉   | 155526/225000 [303:38:38<100:38:08,  5.21s/it]
 69%|██████▉   | 155527/225000 [303:38:45<109:23:20,  5.67s/it]
 69%|██████▉   | 155528/225000 [303:38:58<151:34:23,  7.85s/it]
 69%|██████▉   | 155529/225000 [303:39:04<143:58:06,  7.46s/it]
 69%|██████▉   | 155530/225000 [303:39:08<121:06:30,  6.28s/it]
 69%|██████▉   | 155531/225000 [303:39:13<114:07:29,  5.91s/it]
 69%|██████▉   | 155532/225000 [303:39:21<130:45:22,  6.78s/it]
 69%|██████▉   | 155533/225000 [303:39:25<111:28:18,  5.78s/it]
 69%|██████▉   | 155534/225000 [303:39:34<129:43:06,  6.72s/it]
 69%|██████▉   | 155535/225000 [303:39:38<112:29:04,  5.83s/it]
 69%|██████▉   | 155536/225000 [303:39:45<122:59:43,  6.37s/it]
 69%|██████▉   | 155537/225000 [303:39:50<115:07:11,  5.97s/it]
 69%|██████▉   | 155538/225000 [303:39:54<104:49:57,  5.43s/it]
 69%|██████▉   | 155539/225000 [303:40:00<103:35:23,  5.37s/it]
 69%|██████▉   | 155540/225000 [303:40:04<98:33:34,  5.11s/it] 
 69%|██████▉   | 155541/225000 [303:40:08<93:40:42,  4.86s/it]
 69%|██████▉   | 155542/225000 [303:40:14<94:54:35,  4.92s/it]
 69%|██████▉   | 155543/225000 [303:40:19<96:46:26,  5.02s/it]
 69%|██████▉   | 155544/225000 [303:40:24<96:41:59,  5.01s/it]
 69%|██████▉   | 155545/225000 [303:40:28<94:35:57,  4.90s/it]
 69%|██████▉   | 155546/225000 [303:40:38<123:08:43,  6.38s/it]
 69%|██████▉   | 155547/225000 [303:40:48<141:10:57,  7.32s/it]
 69%|██████▉   | 155548/225000 [303:40:51<118:37:27,  6.15s/it]
 69%|██████▉   | 155549/225000 [303:41:00<133:42:27,  6.93s/it]
 69%|██████▉   | 155550/225000 [303:41:03<113:31:03,  5.88s/it]
                                                               
{'loss': '0.7772', 'grad_norm': '1.106', 'learning_rate': '1.183e-05', 'epoch': '0.6913', 'num_input_tokens_seen': 502920176, 'train_runtime': '1.093e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155550/225000 [303:41:03<113:31:03,  5.88s/it]
 69%|██████▉   | 155551/225000 [303:41:12<131:45:40,  6.83s/it]
 69%|██████▉   | 155552/225000 [303:41:16<112:15:08,  5.82s/it]
 69%|██████▉   | 155553/225000 [303:41:20<99:39:25,  5.17s/it] 
 69%|██████▉   | 155554/225000 [303:41:28<117:21:57,  6.08s/it]
 69%|██████▉   | 155555/225000 [303:41:33<109:59:40,  5.70s/it]
 69%|██████▉   | 155556/225000 [303:41:36<96:42:14,  5.01s/it] 
 69%|██████▉   | 155557/225000 [303:41:46<125:38:16,  6.51s/it]
 69%|██████▉   | 155558/225000 [303:41:49<107:29:40,  5.57s/it]
 69%|██████▉   | 155559/225000 [303:41:54<103:00:57,  5.34s/it]
 69%|██████▉   | 155560/225000 [303:42:02<120:24:15,  6.24s/it]
 69%|██████▉   | 155561/225000 [303:42:07<108:36:42,  5.63s/it]
 69%|██████▉   | 155562/225000 [303:42:10<97:32:10,  5.06s/it] 
 69%|██████▉   | 155563/225000 [303:42:18<114:16:09,  5.92s/it]
 69%|██████▉   | 155564/225000 [303:42:25<117:05:57,  6.07s/it]
 69%|██████▉   | 155565/225000 [303:42:29<106:35:00,  5.53s/it]
 69%|██████▉   | 155566/225000 [303:42:33<99:28:04,  5.16s/it] 
 69%|██████▉   | 155567/225000 [303:42:41<115:10:49,  5.97s/it]
 69%|██████▉   | 155568/225000 [303:42:48<122:50:48,  6.37s/it]
 69%|██████▉   | 155569/225000 [303:42:55<122:57:07,  6.38s/it]
 69%|██████▉   | 155570/225000 [303:43:01<118:44:53,  6.16s/it]
 69%|██████▉   | 155571/225000 [303:43:07<122:09:52,  6.33s/it]
 69%|██████▉   | 155572/225000 [303:43:12<113:24:11,  5.88s/it]
 69%|██████▉   | 155573/225000 [303:43:16<102:36:29,  5.32s/it]
 69%|██████▉   | 155574/225000 [303:43:25<124:36:54,  6.46s/it]
 69%|██████▉   | 155575/225000 [303:43:30<112:55:52,  5.86s/it]
 69%|██████▉   | 155576/225000 [303:43:35<112:19:21,  5.82s/it]
 69%|██████▉   | 155577/225000 [303:43:42<117:01:17,  6.07s/it]
 69%|██████▉   | 155578/225000 [303:43:47<113:20:05,  5.88s/it]
 69%|██████▉   | 155579/225000 [303:43:51<102:21:52,  5.31s/it]
 69%|██████▉   | 155580/225000 [303:43:56<94:59:30,  4.93s/it] 
 69%|██████▉   | 155581/225000 [303:44:06<129:51:26,  6.73s/it]
 69%|██████▉   | 155582/225000 [303:44:10<111:57:16,  5.81s/it]
 69%|██████▉   | 155583/225000 [303:44:15<105:54:33,  5.49s/it]
 69%|██████▉   | 155584/225000 [303:44:22<112:34:23,  5.84s/it]
 69%|██████▉   | 155585/225000 [303:44:26<105:35:17,  5.48s/it]
 69%|██████▉   | 155586/225000 [303:44:31<100:26:36,  5.21s/it]
 69%|██████▉   | 155587/225000 [303:44:36<98:33:01,  5.11s/it] 
 69%|██████▉   | 155588/225000 [303:44:40<96:24:35,  5.00s/it]
 69%|██████▉   | 155589/225000 [303:44:45<96:17:54,  4.99s/it]
 69%|██████▉   | 155590/225000 [303:44:56<129:31:08,  6.72s/it]
 69%|██████▉   | 155591/225000 [303:45:04<134:03:45,  6.95s/it]
 69%|██████▉   | 155592/225000 [303:45:08<118:50:04,  6.16s/it]
 69%|██████▉   | 155593/225000 [303:45:12<107:46:31,  5.59s/it]
 69%|██████▉   | 155594/225000 [303:45:16<99:18:47,  5.15s/it] 
 69%|██████▉   | 155595/225000 [303:45:28<136:10:29,  7.06s/it]
 69%|██████▉   | 155596/225000 [303:45:33<124:40:17,  6.47s/it]
 69%|██████▉   | 155597/225000 [303:45:38<118:58:27,  6.17s/it]
 69%|██████▉   | 155598/225000 [303:45:45<119:52:45,  6.22s/it]
 69%|██████▉   | 155599/225000 [303:45:49<109:14:16,  5.67s/it]
 69%|██████▉   | 155600/225000 [303:45:53<101:18:27,  5.26s/it]
                                                               
{'loss': '0.7794', 'grad_norm': '1.15', 'learning_rate': '1.181e-05', 'epoch': '0.6916', 'num_input_tokens_seen': 503052904, 'train_runtime': '1.094e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155600/225000 [303:45:53<101:18:27,  5.26s/it]
 69%|██████▉   | 155601/225000 [303:45:58<97:01:24,  5.03s/it] 
 69%|██████▉   | 155602/225000 [303:46:06<117:35:20,  6.10s/it]
 69%|██████▉   | 155603/225000 [303:46:11<110:08:36,  5.71s/it]
 69%|██████▉   | 155604/225000 [303:46:19<122:45:39,  6.37s/it]
 69%|██████▉   | 155605/225000 [303:46:24<111:42:34,  5.80s/it]
 69%|██████▉   | 155606/225000 [303:46:28<102:09:27,  5.30s/it]
 69%|██████▉   | 155607/225000 [303:46:32<96:30:11,  5.01s/it] 
 69%|██████▉   | 155608/225000 [303:46:37<93:07:38,  4.83s/it]
 69%|██████▉   | 155609/225000 [303:46:41<93:51:30,  4.87s/it]
 69%|██████▉   | 155610/225000 [303:46:46<94:08:37,  4.88s/it]
 69%|██████▉   | 155611/225000 [303:46:52<96:13:40,  4.99s/it]
 69%|██████▉   | 155612/225000 [303:46:57<98:03:10,  5.09s/it]
 69%|██████▉   | 155613/225000 [303:47:09<140:13:07,  7.27s/it]
 69%|██████▉   | 155614/225000 [303:47:16<136:03:38,  7.06s/it]
 69%|██████▉   | 155615/225000 [303:47:20<116:34:31,  6.05s/it]
 69%|██████▉   | 155616/225000 [303:47:29<135:33:42,  7.03s/it]
 69%|██████▉   | 155617/225000 [303:47:32<113:18:23,  5.88s/it]
 69%|██████▉   | 155618/225000 [303:47:42<134:57:22,  7.00s/it]
 69%|██████▉   | 155619/225000 [303:47:47<122:59:33,  6.38s/it]
 69%|██████▉   | 155620/225000 [303:47:52<116:27:53,  6.04s/it]
 69%|██████▉   | 155621/225000 [303:47:57<111:43:21,  5.80s/it]
 69%|██████▉   | 155622/225000 [303:48:02<105:47:17,  5.49s/it]
 69%|██████▉   | 155623/225000 [303:48:07<104:01:38,  5.40s/it]
 69%|██████▉   | 155624/225000 [303:48:12<102:12:48,  5.30s/it]
 69%|██████▉   | 155625/225000 [303:48:17<102:02:32,  5.30s/it]
 69%|██████▉   | 155626/225000 [303:48:22<100:17:00,  5.20s/it]
 69%|██████▉   | 155627/225000 [303:48:28<102:32:02,  5.32s/it]
 69%|██████▉   | 155628/225000 [303:48:37<122:02:03,  6.33s/it]
 69%|██████▉   | 155629/225000 [303:48:47<145:25:57,  7.55s/it]
 69%|██████▉   | 155630/225000 [303:48:51<124:28:52,  6.46s/it]
 69%|██████▉   | 155631/225000 [303:48:55<110:47:36,  5.75s/it]
 69%|██████▉   | 155632/225000 [303:49:05<135:37:15,  7.04s/it]
 69%|██████▉   | 155633/225000 [303:49:09<116:51:46,  6.06s/it]
 69%|██████▉   | 155634/225000 [303:49:13<104:02:59,  5.40s/it]
 69%|██████▉   | 155635/225000 [303:49:17<97:19:51,  5.05s/it] 
 69%|██████▉   | 155636/225000 [303:49:29<134:26:21,  6.98s/it]
 69%|██████▉   | 155637/225000 [303:49:32<113:43:35,  5.90s/it]
 69%|██████▉   | 155638/225000 [303:49:39<120:04:49,  6.23s/it]
 69%|██████▉   | 155639/225000 [303:49:43<107:46:18,  5.59s/it]
 69%|██████▉   | 155640/225000 [303:49:47<100:12:32,  5.20s/it]
 69%|██████▉   | 155641/225000 [303:49:51<92:27:04,  4.80s/it] 
 69%|██████▉   | 155642/225000 [303:49:59<110:03:31,  5.71s/it]
 69%|██████▉   | 155643/225000 [303:50:05<113:41:16,  5.90s/it]
 69%|██████▉   | 155644/225000 [303:50:09<103:29:58,  5.37s/it]
 69%|██████▉   | 155645/225000 [303:50:14<96:57:47,  5.03s/it] 
 69%|██████▉   | 155646/225000 [303:50:18<93:31:12,  4.85s/it]
 69%|██████▉   | 155647/225000 [303:50:26<112:57:30,  5.86s/it]
 69%|██████▉   | 155648/225000 [303:50:36<134:58:02,  7.01s/it]
 69%|██████▉   | 155649/225000 [303:50:41<124:23:56,  6.46s/it]
 69%|██████▉   | 155650/225000 [303:50:46<113:24:16,  5.89s/it]
                                                               
{'loss': '0.7717', 'grad_norm': '1.085', 'learning_rate': '1.18e-05', 'epoch': '0.6918', 'num_input_tokens_seen': 503184136, 'train_runtime': '1.094e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155650/225000 [303:50:46<113:24:16,  5.89s/it]
 69%|██████▉   | 155651/225000 [303:50:50<105:57:15,  5.50s/it]
 69%|██████▉   | 155652/225000 [303:50:55<100:49:33,  5.23s/it]
 69%|██████▉   | 155653/225000 [303:50:59<96:05:54,  4.99s/it] 
 69%|██████▉   | 155654/225000 [303:51:04<95:15:35,  4.95s/it]
 69%|██████▉   | 155655/225000 [303:51:17<137:30:36,  7.14s/it]
 69%|██████▉   | 155656/225000 [303:51:21<124:14:28,  6.45s/it]
 69%|██████▉   | 155657/225000 [303:51:25<108:21:13,  5.63s/it]
 69%|██████▉   | 155658/225000 [303:51:34<126:39:57,  6.58s/it]
 69%|██████▉   | 155659/225000 [303:51:38<110:09:42,  5.72s/it]
 69%|██████▉   | 155660/225000 [303:51:47<130:01:05,  6.75s/it]
 69%|██████▉   | 155661/225000 [303:51:51<117:03:15,  6.08s/it]
 69%|██████▉   | 155662/225000 [303:51:56<107:48:46,  5.60s/it]
 69%|██████▉   | 155663/225000 [303:52:00<100:55:44,  5.24s/it]
 69%|██████▉   | 155664/225000 [303:52:04<94:31:14,  4.91s/it] 
 69%|██████▉   | 155665/225000 [303:52:09<91:38:13,  4.76s/it]
 69%|██████▉   | 155666/225000 [303:52:13<89:11:54,  4.63s/it]
 69%|██████▉   | 155667/225000 [303:52:25<132:09:07,  6.86s/it]
 69%|██████▉   | 155668/225000 [303:52:30<120:40:32,  6.27s/it]
 69%|██████▉   | 155669/225000 [303:52:33<103:40:34,  5.38s/it]
 69%|██████▉   | 155670/225000 [303:52:42<124:50:36,  6.48s/it]
 69%|██████▉   | 155671/225000 [303:52:46<108:25:02,  5.63s/it]
 69%|██████▉   | 155672/225000 [303:52:51<102:51:19,  5.34s/it]
 69%|██████▉   | 155673/225000 [303:52:59<120:29:00,  6.26s/it]
 69%|██████▉   | 155674/225000 [303:53:02<104:04:13,  5.40s/it]
 69%|██████▉   | 155675/225000 [303:53:11<119:50:50,  6.22s/it]
 69%|██████▉   | 155676/225000 [303:53:14<105:37:33,  5.49s/it]
 69%|██████▉   | 155677/225000 [303:53:19<99:07:25,  5.15s/it] 
 69%|██████▉   | 155678/225000 [303:53:24<101:13:02,  5.26s/it]
 69%|██████▉   | 155679/225000 [303:53:32<117:39:20,  6.11s/it]
 69%|██████▉   | 155680/225000 [303:53:37<108:51:17,  5.65s/it]
 69%|██████▉   | 155681/225000 [303:53:44<117:00:22,  6.08s/it]
 69%|██████▉   | 155682/225000 [303:53:48<102:54:43,  5.34s/it]
 69%|██████▉   | 155683/225000 [303:53:55<117:41:47,  6.11s/it]
 69%|██████▉   | 155684/225000 [303:53:59<104:22:28,  5.42s/it]
 69%|██████▉   | 155685/225000 [303:54:04<102:24:52,  5.32s/it]
 69%|██████▉   | 155686/225000 [303:54:12<116:29:28,  6.05s/it]
 69%|██████▉   | 155687/225000 [303:54:16<101:05:01,  5.25s/it]
 69%|██████▉   | 155688/225000 [303:54:25<122:53:51,  6.38s/it]
 69%|██████▉   | 155689/225000 [303:54:28<106:58:21,  5.56s/it]
 69%|██████▉   | 155690/225000 [303:54:34<107:16:40,  5.57s/it]
 69%|██████▉   | 155691/225000 [303:54:42<123:44:41,  6.43s/it]
 69%|██████▉   | 155692/225000 [303:54:47<112:23:21,  5.84s/it]
 69%|██████▉   | 155693/225000 [303:54:51<104:51:27,  5.45s/it]
 69%|██████▉   | 155694/225000 [303:54:56<102:13:26,  5.31s/it]
 69%|██████▉   | 155695/225000 [303:55:00<95:53:03,  4.98s/it] 
 69%|██████▉   | 155696/225000 [303:55:05<90:56:45,  4.72s/it]
 69%|██████▉   | 155697/225000 [303:55:15<122:04:35,  6.34s/it]
 69%|██████▉   | 155698/225000 [303:55:23<131:14:28,  6.82s/it]
 69%|██████▉   | 155699/225000 [303:55:26<113:08:32,  5.88s/it]
 69%|██████▉   | 155700/225000 [303:55:34<124:40:37,  6.48s/it]
                                                               
{'loss': '0.7896', 'grad_norm': '1.109', 'learning_rate': '1.178e-05', 'epoch': '0.692', 'num_input_tokens_seen': 503313648, 'train_runtime': '1.094e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155700/225000 [303:55:34<124:40:37,  6.48s/it]
 69%|██████▉   | 155701/225000 [303:55:40<118:38:21,  6.16s/it]
 69%|██████▉   | 155702/225000 [303:55:47<124:21:14,  6.46s/it]
 69%|██████▉   | 155703/225000 [303:55:51<111:08:16,  5.77s/it]
 69%|██████▉   | 155704/225000 [303:55:55<100:26:48,  5.22s/it]
 69%|██████▉   | 155705/225000 [303:55:59<93:50:29,  4.88s/it] 
 69%|██████▉   | 155706/225000 [303:56:11<135:45:53,  7.05s/it]
 69%|██████▉   | 155707/225000 [303:56:14<114:04:53,  5.93s/it]
 69%|██████▉   | 155708/225000 [303:56:21<120:07:41,  6.24s/it]
 69%|██████▉   | 155709/225000 [303:56:26<110:57:36,  5.76s/it]
 69%|██████▉   | 155710/225000 [303:56:30<102:31:05,  5.33s/it]
 69%|██████▉   | 155711/225000 [303:56:35<97:51:59,  5.08s/it] 
 69%|██████▉   | 155712/225000 [303:56:39<95:33:31,  4.96s/it]
 69%|██████▉   | 155713/225000 [303:56:44<94:15:55,  4.90s/it]
 69%|██████▉   | 155714/225000 [303:56:49<94:31:14,  4.91s/it]
 69%|██████▉   | 155715/225000 [303:56:54<92:17:41,  4.80s/it]
 69%|██████▉   | 155716/225000 [303:57:05<132:33:07,  6.89s/it]
 69%|██████▉   | 155717/225000 [303:57:12<132:21:47,  6.88s/it]
 69%|██████▉   | 155718/225000 [303:57:16<112:21:18,  5.84s/it]
 69%|██████▉   | 155719/225000 [303:57:20<100:52:19,  5.24s/it]
 69%|██████▉   | 155720/225000 [303:57:30<130:02:11,  6.76s/it]
 69%|██████▉   | 155721/225000 [303:57:34<113:54:07,  5.92s/it]
 69%|██████▉   | 155722/225000 [303:57:38<102:34:59,  5.33s/it]
 69%|██████▉   | 155723/225000 [303:57:42<96:39:24,  5.02s/it] 
 69%|██████▉   | 155724/225000 [303:57:52<127:48:24,  6.64s/it]
 69%|██████▉   | 155725/225000 [303:57:56<110:14:25,  5.73s/it]
 69%|██████▉   | 155726/225000 [303:57:59<96:21:05,  5.01s/it] 
 69%|██████▉   | 155727/225000 [303:58:08<115:04:27,  5.98s/it]
 69%|██████▉   | 155728/225000 [303:58:11<102:35:53,  5.33s/it]
 69%|██████▉   | 155729/225000 [303:58:15<94:21:34,  4.90s/it] 
 69%|██████▉   | 155730/225000 [303:58:27<131:33:09,  6.84s/it]
 69%|██████▉   | 155731/225000 [303:58:30<113:47:37,  5.91s/it]
 69%|██████▉   | 155732/225000 [303:58:34<102:06:29,  5.31s/it]
 69%|██████▉   | 155733/225000 [303:58:43<122:58:30,  6.39s/it]
 69%|██████▉   | 155734/225000 [303:58:47<107:20:41,  5.58s/it]
 69%|██████▉   | 155735/225000 [303:58:51<98:10:08,  5.10s/it] 
 69%|██████▉   | 155736/225000 [303:58:56<97:49:31,  5.08s/it]
 69%|██████▉   | 155737/225000 [303:59:05<122:55:05,  6.39s/it]
 69%|██████▉   | 155738/225000 [303:59:09<108:27:29,  5.64s/it]
 69%|██████▉   | 155739/225000 [303:59:15<108:21:14,  5.63s/it]
 69%|██████▉   | 155740/225000 [303:59:23<123:13:18,  6.40s/it]
 69%|██████▉   | 155741/225000 [303:59:27<106:58:24,  5.56s/it]
 69%|██████▉   | 155742/225000 [303:59:31<99:15:04,  5.16s/it] 
 69%|██████▉   | 155743/225000 [303:59:35<93:39:18,  4.87s/it]
 69%|██████▉   | 155744/225000 [303:59:41<97:34:37,  5.07s/it]
 69%|██████▉   | 155745/225000 [303:59:46<99:26:27,  5.17s/it]
 69%|██████▉   | 155746/225000 [303:59:51<97:41:12,  5.08s/it]
 69%|██████▉   | 155747/225000 [303:59:56<96:26:51,  5.01s/it]
 69%|██████▉   | 155748/225000 [304:00:01<96:14:11,  5.00s/it]
 69%|██████▉   | 155749/225000 [304:00:06<97:33:13,  5.07s/it]
 69%|██████▉   | 155750/225000 [304:00:12<101:35:20,  5.28s/it]
                                                               
{'loss': '0.7744', 'grad_norm': '1.122', 'learning_rate': '1.177e-05', 'epoch': '0.6922', 'num_input_tokens_seen': 503443368, 'train_runtime': '1.094e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155750/225000 [304:00:12<101:35:20,  5.28s/it]
 69%|██████▉   | 155751/225000 [304:00:17<102:19:12,  5.32s/it]
 69%|██████▉   | 155752/225000 [304:00:23<102:52:25,  5.35s/it]
 69%|██████▉   | 155753/225000 [304:00:35<144:27:12,  7.51s/it]
 69%|██████▉   | 155754/225000 [304:00:41<135:16:41,  7.03s/it]
 69%|██████▉   | 155755/225000 [304:00:46<120:32:40,  6.27s/it]
 69%|██████▉   | 155756/225000 [304:00:50<109:58:41,  5.72s/it]
 69%|██████▉   | 155757/225000 [304:00:55<103:46:52,  5.40s/it]
 69%|██████▉   | 155758/225000 [304:00:59<98:42:56,  5.13s/it] 
 69%|██████▉   | 155759/225000 [304:01:04<97:01:00,  5.04s/it]
 69%|██████▉   | 155760/225000 [304:01:08<93:41:51,  4.87s/it]
 69%|██████▉   | 155761/225000 [304:01:19<127:11:21,  6.61s/it]
 69%|██████▉   | 155762/225000 [304:01:26<130:50:24,  6.80s/it]
 69%|██████▉   | 155763/225000 [304:01:30<110:24:17,  5.74s/it]
 69%|██████▉   | 155764/225000 [304:01:39<132:55:03,  6.91s/it]
 69%|██████▉   | 155765/225000 [304:01:43<112:18:57,  5.84s/it]
 69%|██████▉   | 155766/225000 [304:01:50<122:32:22,  6.37s/it]
 69%|██████▉   | 155767/225000 [304:01:54<106:18:58,  5.53s/it]
 69%|██████▉   | 155768/225000 [304:02:03<125:43:26,  6.54s/it]
 69%|██████▉   | 155769/225000 [304:02:06<106:54:48,  5.56s/it]
 69%|██████▉   | 155770/225000 [304:02:10<99:03:18,  5.15s/it] 
 69%|██████▉   | 155771/225000 [304:02:19<117:20:50,  6.10s/it]
 69%|██████▉   | 155772/225000 [304:02:21<99:13:39,  5.16s/it] 
 69%|██████▉   | 155773/225000 [304:02:29<111:31:55,  5.80s/it]
 69%|██████▉   | 155774/225000 [304:02:33<102:33:57,  5.33s/it]
 69%|██████▉   | 155775/225000 [304:02:37<96:57:04,  5.04s/it] 
 69%|██████▉   | 155776/225000 [304:02:42<94:14:51,  4.90s/it]
 69%|██████▉   | 155777/225000 [304:02:46<90:39:09,  4.71s/it]
 69%|██████▉   | 155778/225000 [304:02:55<116:07:54,  6.04s/it]
 69%|██████▉   | 155779/225000 [304:03:03<126:58:07,  6.60s/it]
 69%|██████▉   | 155780/225000 [304:03:09<118:58:58,  6.19s/it]
 69%|██████▉   | 155781/225000 [304:03:13<111:24:16,  5.79s/it]
 69%|██████▉   | 155782/225000 [304:03:18<106:02:09,  5.51s/it]
 69%|██████▉   | 155783/225000 [304:03:23<104:04:27,  5.41s/it]
 69%|██████▉   | 155784/225000 [304:03:28<99:58:20,  5.20s/it] 
 69%|██████▉   | 155785/225000 [304:03:34<101:16:06,  5.27s/it]
 69%|██████▉   | 155786/225000 [304:03:39<99:56:52,  5.20s/it] 
 69%|██████▉   | 155787/225000 [304:03:44<99:05:20,  5.15s/it]
 69%|██████▉   | 155788/225000 [304:03:49<98:43:03,  5.13s/it]
 69%|██████▉   | 155789/225000 [304:03:59<130:26:38,  6.79s/it]
 69%|██████▉   | 155790/225000 [304:04:06<130:08:13,  6.77s/it]
 69%|██████▉   | 155791/225000 [304:04:09<109:30:19,  5.70s/it]
 69%|██████▉   | 155792/225000 [304:04:19<132:09:22,  6.87s/it]
 69%|██████▉   | 155793/225000 [304:04:22<111:59:04,  5.83s/it]
 69%|██████▉   | 155794/225000 [304:04:29<115:50:40,  6.03s/it]
 69%|██████▉   | 155795/225000 [304:04:34<110:04:43,  5.73s/it]
 69%|██████▉   | 155796/225000 [304:04:39<105:54:24,  5.51s/it]
 69%|██████▉   | 155797/225000 [304:04:44<103:01:59,  5.36s/it]
 69%|██████▉   | 155798/225000 [304:04:49<104:30:47,  5.44s/it]
 69%|██████▉   | 155799/225000 [304:04:54<101:05:48,  5.26s/it]
 69%|██████▉   | 155800/225000 [304:04:59<100:18:27,  5.22s/it]
                                                               
{'loss': '0.7664', 'grad_norm': '1.099', 'learning_rate': '1.175e-05', 'epoch': '0.6924', 'num_input_tokens_seen': 503572776, 'train_runtime': '1.095e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155800/225000 [304:04:59<100:18:27,  5.22s/it]
 69%|██████▉   | 155801/225000 [304:05:04<99:14:16,  5.16s/it] 
 69%|██████▉   | 155802/225000 [304:05:10<99:52:01,  5.20s/it]
 69%|██████▉   | 155803/225000 [304:05:15<99:18:40,  5.17s/it]
 69%|██████▉   | 155804/225000 [304:05:20<98:31:57,  5.13s/it]
 69%|██████▉   | 155805/225000 [304:05:32<141:27:35,  7.36s/it]
 69%|██████▉   | 155806/225000 [304:05:37<127:52:46,  6.65s/it]
 69%|██████▉   | 155807/225000 [304:05:41<112:25:33,  5.85s/it]
 69%|██████▉   | 155808/225000 [304:05:47<110:05:40,  5.73s/it]
 69%|██████▉   | 155809/225000 [304:05:56<127:09:21,  6.62s/it]
 69%|██████▉   | 155810/225000 [304:06:01<120:38:39,  6.28s/it]
 69%|██████▉   | 155811/225000 [304:06:07<121:32:00,  6.32s/it]
 69%|██████▉   | 155812/225000 [304:06:12<111:06:33,  5.78s/it]
 69%|██████▉   | 155813/225000 [304:06:17<108:48:34,  5.66s/it]
 69%|██████▉   | 155814/225000 [304:06:22<104:29:19,  5.44s/it]
 69%|██████▉   | 155815/225000 [304:06:27<102:22:49,  5.33s/it]
 69%|██████▉   | 155816/225000 [304:06:32<101:14:00,  5.27s/it]
 69%|██████▉   | 155817/225000 [304:06:38<101:28:52,  5.28s/it]
 69%|██████▉   | 155818/225000 [304:06:43<101:16:40,  5.27s/it]
 69%|██████▉   | 155819/225000 [304:06:48<100:00:08,  5.20s/it]
 69%|██████▉   | 155820/225000 [304:06:53<100:30:30,  5.23s/it]
 69%|██████▉   | 155821/225000 [304:06:58<98:46:10,  5.14s/it] 
 69%|██████▉   | 155822/225000 [304:07:10<135:37:29,  7.06s/it]
 69%|██████▉   | 155823/225000 [304:07:16<128:26:26,  6.68s/it]
 69%|██████▉   | 155824/225000 [304:07:22<129:24:01,  6.73s/it]
 69%|██████▉   | 155825/225000 [304:07:27<114:50:14,  5.98s/it]
 69%|██████▉   | 155826/225000 [304:07:33<115:50:19,  6.03s/it]
 69%|██████▉   | 155827/225000 [304:07:40<121:23:52,  6.32s/it]
 69%|██████▉   | 155828/225000 [304:07:43<104:34:55,  5.44s/it]
 69%|██████▉   | 155829/225000 [304:07:49<105:50:35,  5.51s/it]
 69%|██████▉   | 155830/225000 [304:07:58<129:10:07,  6.72s/it]
 69%|██████▉   | 155831/225000 [304:08:02<109:56:09,  5.72s/it]
 69%|██████▉   | 155832/225000 [304:08:07<104:49:58,  5.46s/it]
 69%|██████▉   | 155833/225000 [304:08:12<106:01:39,  5.52s/it]
 69%|██████▉   | 155834/225000 [304:08:17<103:00:00,  5.36s/it]
 69%|██████▉   | 155835/225000 [304:08:23<103:29:04,  5.39s/it]
 69%|██████▉   | 155836/225000 [304:08:28<104:25:59,  5.44s/it]
 69%|██████▉   | 155837/225000 [304:08:34<104:12:16,  5.42s/it]
 69%|██████▉   | 155838/225000 [304:08:39<101:22:13,  5.28s/it]
 69%|██████▉   | 155839/225000 [304:08:44<102:26:28,  5.33s/it]
 69%|██████▉   | 155840/225000 [304:08:49<100:17:44,  5.22s/it]
 69%|██████▉   | 155841/225000 [304:08:54<98:58:11,  5.15s/it] 
 69%|██████▉   | 155842/225000 [304:08:59<100:33:58,  5.23s/it]
 69%|██████▉   | 155843/225000 [304:09:05<99:36:46,  5.19s/it] 
 69%|██████▉   | 155844/225000 [304:09:10<103:01:55,  5.36s/it]
 69%|██████▉   | 155845/225000 [304:09:16<103:27:48,  5.39s/it]
 69%|██████▉   | 155846/225000 [304:09:21<104:13:55,  5.43s/it]
 69%|██████▉   | 155847/225000 [304:09:27<105:11:36,  5.48s/it]
 69%|██████▉   | 155848/225000 [304:09:32<105:25:23,  5.49s/it]
 69%|██████▉   | 155849/225000 [304:09:43<132:52:55,  6.92s/it]
 69%|██████▉   | 155850/225000 [304:09:52<148:57:28,  7.75s/it]
                                                               
{'loss': '0.7697', 'grad_norm': '1.409', 'learning_rate': '1.173e-05', 'epoch': '0.6927', 'num_input_tokens_seen': 503702920, 'train_runtime': '1.095e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155850/225000 [304:09:52<148:57:28,  7.75s/it]
 69%|██████▉   | 155851/225000 [304:09:56<122:58:42,  6.40s/it]
 69%|██████▉   | 155852/225000 [304:10:05<142:10:22,  7.40s/it]
 69%|██████▉   | 155853/225000 [304:10:09<119:51:02,  6.24s/it]
 69%|██████▉   | 155854/225000 [304:10:17<129:15:54,  6.73s/it]
 69%|██████▉   | 155855/225000 [304:10:20<109:03:20,  5.68s/it]
 69%|██████▉   | 155856/225000 [304:10:27<117:57:12,  6.14s/it]
 69%|██████▉   | 155857/225000 [304:10:32<109:03:36,  5.68s/it]
 69%|██████▉   | 155858/225000 [304:10:39<117:29:25,  6.12s/it]
 69%|██████▉   | 155859/225000 [304:10:43<106:13:03,  5.53s/it]
 69%|██████▉   | 155860/225000 [304:10:49<107:21:58,  5.59s/it]
 69%|██████▉   | 155861/225000 [304:10:56<116:20:15,  6.06s/it]
 69%|██████▉   | 155862/225000 [304:11:00<102:45:16,  5.35s/it]
 69%|██████▉   | 155863/225000 [304:11:04<94:01:31,  4.90s/it] 
 69%|██████▉   | 155864/225000 [304:11:12<112:58:02,  5.88s/it]
 69%|██████▉   | 155865/225000 [304:11:19<119:27:19,  6.22s/it]
 69%|██████▉   | 155866/225000 [304:11:23<108:09:03,  5.63s/it]
 69%|██████▉   | 155867/225000 [304:11:29<108:24:58,  5.65s/it]
 69%|██████▉   | 155868/225000 [304:11:34<104:47:16,  5.46s/it]
 69%|██████▉   | 155869/225000 [304:11:39<105:30:01,  5.49s/it]
 69%|██████▉   | 155870/225000 [304:11:44<102:07:34,  5.32s/it]
 69%|██████▉   | 155871/225000 [304:11:49<102:10:02,  5.32s/it]
 69%|██████▉   | 155872/225000 [304:11:56<106:43:09,  5.56s/it]
 69%|██████▉   | 155873/225000 [304:12:04<124:09:12,  6.47s/it]
 69%|██████▉   | 155874/225000 [304:12:09<113:13:00,  5.90s/it]
 69%|██████▉   | 155875/225000 [304:12:13<105:44:06,  5.51s/it]
 69%|██████▉   | 155876/225000 [304:12:18<100:37:46,  5.24s/it]
 69%|██████▉   | 155877/225000 [304:12:22<96:21:25,  5.02s/it] 
 69%|██████▉   | 155878/225000 [304:12:27<92:19:50,  4.81s/it]
 69%|██████▉   | 155879/225000 [304:12:36<120:15:09,  6.26s/it]
 69%|██████▉   | 155880/225000 [304:12:44<126:02:01,  6.56s/it]
 69%|██████▉   | 155881/225000 [304:12:48<111:05:13,  5.79s/it]
 69%|██████▉   | 155882/225000 [304:12:56<128:22:23,  6.69s/it]
 69%|██████▉   | 155883/225000 [304:13:00<112:48:08,  5.88s/it]
 69%|██████▉   | 155884/225000 [304:13:07<115:03:58,  5.99s/it]
 69%|██████▉   | 155885/225000 [304:13:11<103:02:27,  5.37s/it]
 69%|██████▉   | 155886/225000 [304:13:15<95:55:14,  5.00s/it] 
 69%|██████▉   | 155887/225000 [304:13:27<135:55:14,  7.08s/it]
 69%|██████▉   | 155888/225000 [304:13:30<116:42:50,  6.08s/it]
 69%|██████▉   | 155889/225000 [304:13:36<116:36:06,  6.07s/it]
 69%|██████▉   | 155890/225000 [304:13:44<125:51:58,  6.56s/it]
 69%|██████▉   | 155891/225000 [304:13:47<106:26:58,  5.55s/it]
 69%|██████▉   | 155892/225000 [304:13:57<127:40:27,  6.65s/it]
 69%|██████▉   | 155893/225000 [304:14:00<108:55:40,  5.67s/it]
 69%|██████▉   | 155894/225000 [304:14:04<98:35:04,  5.14s/it] 
 69%|██████▉   | 155895/225000 [304:14:09<100:58:42,  5.26s/it]
 69%|██████▉   | 155896/225000 [304:14:18<122:43:44,  6.39s/it]
 69%|██████▉   | 155897/225000 [304:14:22<108:18:46,  5.64s/it]
 69%|██████▉   | 155898/225000 [304:14:26<99:13:43,  5.17s/it] 
 69%|██████▉   | 155899/225000 [304:14:36<123:17:15,  6.42s/it]
 69%|██████▉   | 155900/225000 [304:14:39<106:25:05,  5.54s/it]
                                                               
{'loss': '0.7746', 'grad_norm': '1.342', 'learning_rate': '1.172e-05', 'epoch': '0.6929', 'num_input_tokens_seen': 503831784, 'train_runtime': '1.095e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155900/225000 [304:14:39<106:25:05,  5.54s/it]
 69%|██████▉   | 155901/225000 [304:14:45<108:12:59,  5.64s/it]
 69%|██████▉   | 155902/225000 [304:14:54<127:05:30,  6.62s/it]
 69%|██████▉   | 155903/225000 [304:14:57<108:39:55,  5.66s/it]
 69%|██████▉   | 155904/225000 [304:15:06<123:19:15,  6.43s/it]
 69%|██████▉   | 155905/225000 [304:15:10<108:41:22,  5.66s/it]
 69%|██████▉   | 155906/225000 [304:15:14<100:55:23,  5.26s/it]
 69%|██████▉   | 155907/225000 [304:15:24<131:36:07,  6.86s/it]
 69%|██████▉   | 155908/225000 [304:15:28<111:06:59,  5.79s/it]
 69%|██████▉   | 155909/225000 [304:15:31<96:32:10,  5.03s/it] 
 69%|██████▉   | 155910/225000 [304:15:41<123:05:34,  6.41s/it]
 69%|██████▉   | 155911/225000 [304:15:44<107:30:08,  5.60s/it]
 69%|██████▉   | 155912/225000 [304:15:48<96:02:16,  5.00s/it] 
 69%|██████▉   | 155913/225000 [304:15:57<117:47:53,  6.14s/it]
 69%|██████▉   | 155914/225000 [304:16:00<102:49:20,  5.36s/it]
 69%|██████▉   | 155915/225000 [304:16:05<97:21:32,  5.07s/it] 
 69%|██████▉   | 155916/225000 [304:16:09<93:29:47,  4.87s/it]
 69%|██████▉   | 155917/225000 [304:16:21<131:42:50,  6.86s/it]
 69%|██████▉   | 155918/225000 [304:16:27<130:02:46,  6.78s/it]
 69%|██████▉   | 155919/225000 [304:16:31<110:20:50,  5.75s/it]
 69%|██████▉   | 155920/225000 [304:16:36<106:24:40,  5.55s/it]
 69%|██████▉   | 155921/225000 [304:16:41<103:37:22,  5.40s/it]
 69%|██████▉   | 155922/225000 [304:16:46<103:24:18,  5.39s/it]
 69%|██████▉   | 155923/225000 [304:16:52<103:54:25,  5.42s/it]
 69%|██████▉   | 155924/225000 [304:16:57<101:48:48,  5.31s/it]
 69%|██████▉   | 155925/225000 [304:17:02<99:48:44,  5.20s/it] 
 69%|██████▉   | 155926/225000 [304:17:07<98:40:12,  5.14s/it]
 69%|██████▉   | 155927/225000 [304:17:11<96:20:55,  5.02s/it]
 69%|██████▉   | 155928/225000 [304:17:16<96:00:05,  5.00s/it]
 69%|██████▉   | 155929/225000 [304:17:23<106:39:25,  5.56s/it]
 69%|██████▉   | 155930/225000 [304:17:36<150:11:06,  7.83s/it]
 69%|██████▉   | 155931/225000 [304:17:41<130:29:59,  6.80s/it]
 69%|██████▉   | 155932/225000 [304:17:44<111:30:27,  5.81s/it]
 69%|██████▉   | 155933/225000 [304:17:48<102:55:08,  5.36s/it]
 69%|██████▉   | 155934/225000 [304:17:53<98:33:17,  5.14s/it] 
 69%|██████▉   | 155935/225000 [304:18:01<116:35:43,  6.08s/it]
 69%|██████▉   | 155936/225000 [304:18:11<139:52:08,  7.29s/it]
 69%|██████▉   | 155937/225000 [304:18:15<117:47:29,  6.14s/it]
 69%|██████▉   | 155938/225000 [304:18:19<104:41:44,  5.46s/it]
 69%|██████▉   | 155939/225000 [304:18:28<127:57:14,  6.67s/it]
 69%|██████▉   | 155940/225000 [304:18:32<112:02:08,  5.84s/it]
 69%|██████▉   | 155941/225000 [304:18:37<107:33:43,  5.61s/it]
 69%|██████▉   | 155942/225000 [304:18:45<121:17:26,  6.32s/it]
 69%|██████▉   | 155943/225000 [304:18:49<105:39:39,  5.51s/it]
 69%|██████▉   | 155944/225000 [304:18:58<127:02:42,  6.62s/it]
 69%|██████▉   | 155945/225000 [304:19:01<108:41:33,  5.67s/it]
 69%|██████▉   | 155946/225000 [304:19:07<107:50:25,  5.62s/it]
 69%|██████▉   | 155947/225000 [304:19:12<104:59:38,  5.47s/it]
 69%|██████▉   | 155948/225000 [304:19:22<129:33:00,  6.75s/it]
 69%|██████▉   | 155949/225000 [304:19:26<114:01:57,  5.95s/it]
 69%|██████▉   | 155950/225000 [304:19:30<102:56:22,  5.37s/it]
                                                               
{'loss': '0.7729', 'grad_norm': '1.104', 'learning_rate': '1.17e-05', 'epoch': '0.6931', 'num_input_tokens_seen': 503962160, 'train_runtime': '1.096e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 155950/225000 [304:19:30<102:56:22,  5.37s/it]
 69%|██████▉   | 155951/225000 [304:19:40<128:21:02,  6.69s/it]
 69%|██████▉   | 155952/225000 [304:19:46<126:46:42,  6.61s/it]
 69%|██████▉   | 155953/225000 [304:19:50<112:32:31,  5.87s/it]
 69%|██████▉   | 155954/225000 [304:19:54<100:57:07,  5.26s/it]
 69%|██████▉   | 155955/225000 [304:19:59<100:05:10,  5.22s/it]
 69%|██████▉   | 155956/225000 [304:20:04<99:42:53,  5.20s/it] 
 69%|██████▉   | 155957/225000 [304:20:10<101:01:17,  5.27s/it]
 69%|██████▉   | 155958/225000 [304:20:15<102:47:47,  5.36s/it]
 69%|██████▉   | 155959/225000 [304:20:22<109:11:27,  5.69s/it]
 69%|██████▉   | 155960/225000 [304:20:31<128:57:52,  6.72s/it]
 69%|██████▉   | 155961/225000 [304:20:34<109:51:54,  5.73s/it]
 69%|██████▉   | 155962/225000 [304:20:43<127:57:08,  6.67s/it]
 69%|██████▉   | 155963/225000 [304:20:47<108:06:16,  5.64s/it]
 69%|██████▉   | 155964/225000 [304:20:53<112:13:13,  5.85s/it]
 69%|██████▉   | 155965/225000 [304:20:59<115:37:51,  6.03s/it]
 69%|██████▉   | 155966/225000 [304:21:03<100:55:43,  5.26s/it]
 69%|██████▉   | 155967/225000 [304:21:11<120:15:01,  6.27s/it]
 69%|██████▉   | 155968/225000 [304:21:16<110:51:11,  5.78s/it]
 69%|██████▉   | 155969/225000 [304:21:21<105:29:43,  5.50s/it]
 69%|██████▉   | 155970/225000 [304:21:25<100:14:06,  5.23s/it]
 69%|██████▉   | 155971/225000 [304:21:29<92:52:33,  4.84s/it] 
 69%|██████▉   | 155972/225000 [304:21:34<92:55:17,  4.85s/it]
 69%|██████▉   | 155973/225000 [304:21:39<94:57:15,  4.95s/it]
 69%|██████▉   | 155974/225000 [304:21:44<91:19:10,  4.76s/it]
 69%|██████▉   | 155975/225000 [304:21:49<92:12:41,  4.81s/it]
 69%|██████▉   | 155976/225000 [304:22:00<127:20:52,  6.64s/it]
 69%|██████▉   | 155977/225000 [304:22:07<132:11:39,  6.89s/it]
 69%|██████▉   | 155978/225000 [304:22:10<111:15:13,  5.80s/it]
 69%|██████▉   | 155979/225000 [304:22:15<105:23:46,  5.50s/it]
 69%|██████▉   | 155980/225000 [304:22:20<103:33:15,  5.40s/it]
 69%|██████▉   | 155981/225000 [304:22:26<104:19:07,  5.44s/it]
 69%|██████▉   | 155982/225000 [304:22:31<104:39:26,  5.46s/it]
 69%|██████▉   | 155983/225000 [304:22:36<102:05:39,  5.33s/it]
 69%|██████▉   | 155984/225000 [304:22:42<103:30:40,  5.40s/it]
 69%|██████▉   | 155985/225000 [304:22:47<104:18:36,  5.44s/it]
 69%|██████▉   | 155986/225000 [304:22:52<101:17:06,  5.28s/it]
 69%|██████▉   | 155987/225000 [304:23:01<122:26:04,  6.39s/it]
 69%|██████▉   | 155988/225000 [304:23:11<138:19:25,  7.22s/it]
 69%|██████▉   | 155989/225000 [304:23:14<117:16:23,  6.12s/it]
 69%|██████▉   | 155990/225000 [304:23:21<124:01:52,  6.47s/it]
 69%|██████▉   | 155991/225000 [304:23:25<107:22:20,  5.60s/it]
 69%|██████▉   | 155992/225000 [304:23:35<134:29:09,  7.02s/it]
 69%|██████▉   | 155993/225000 [304:23:39<113:13:35,  5.91s/it]
 69%|██████▉   | 155994/225000 [304:23:45<119:00:09,  6.21s/it]
 69%|██████▉   | 155995/225000 [304:23:49<102:34:10,  5.35s/it]
 69%|██████▉   | 155996/225000 [304:23:53<93:10:14,  4.86s/it] 
 69%|██████▉   | 155997/225000 [304:23:59<101:59:38,  5.32s/it]
 69%|██████▉   | 155998/225000 [304:24:07<119:03:46,  6.21s/it]
 69%|██████▉   | 155999/225000 [304:24:11<105:48:32,  5.52s/it]
 69%|██████▉   | 156000/225000 [304:24:15<98:05:27,  5.12s/it] 
                                                              
{'loss': '0.774', 'grad_norm': '1.259', 'learning_rate': '1.169e-05', 'epoch': '0.6933', 'num_input_tokens_seen': 504091584, 'train_runtime': '1.096e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156000/225000 [304:24:15<98:05:27,  5.12s/it]
 69%|██████▉   | 156001/225000 [304:24:25<125:32:05,  6.55s/it]
 69%|██████▉   | 156002/225000 [304:24:30<116:20:28,  6.07s/it]
 69%|██████▉   | 156003/225000 [304:24:34<103:13:04,  5.39s/it]
 69%|██████▉   | 156004/225000 [304:24:40<108:11:56,  5.65s/it]
 69%|██████▉   | 156005/225000 [304:24:47<113:17:11,  5.91s/it]
 69%|██████▉   | 156006/225000 [304:24:51<103:23:02,  5.39s/it]
 69%|██████▉   | 156007/225000 [304:24:55<98:38:03,  5.15s/it] 
 69%|██████▉   | 156008/225000 [304:25:00<95:09:43,  4.97s/it]
 69%|██████▉   | 156009/225000 [304:25:05<93:57:32,  4.90s/it]
 69%|██████▉   | 156010/225000 [304:25:11<102:06:40,  5.33s/it]
 69%|██████▉   | 156011/225000 [304:25:23<142:27:51,  7.43s/it]
 69%|██████▉   | 156012/225000 [304:25:27<118:49:45,  6.20s/it]
 69%|██████▉   | 156013/225000 [304:25:34<125:05:43,  6.53s/it]
 69%|██████▉   | 156014/225000 [304:25:38<107:36:05,  5.62s/it]
 69%|██████▉   | 156015/225000 [304:25:42<100:14:14,  5.23s/it]
 69%|██████▉   | 156016/225000 [304:25:52<125:25:17,  6.55s/it]
 69%|██████▉   | 156017/225000 [304:25:55<110:18:52,  5.76s/it]
 69%|██████▉   | 156018/225000 [304:25:59<100:11:22,  5.23s/it]
 69%|██████▉   | 156019/225000 [304:26:10<129:35:59,  6.76s/it]
 69%|██████▉   | 156020/225000 [304:26:13<109:12:31,  5.70s/it]
 69%|██████▉   | 156021/225000 [304:26:22<127:28:58,  6.65s/it]
 69%|██████▉   | 156022/225000 [304:26:25<109:23:05,  5.71s/it]
 69%|██████▉   | 156023/225000 [304:26:33<120:26:14,  6.29s/it]
 69%|██████▉   | 156024/225000 [304:26:38<113:57:59,  5.95s/it]
 69%|██████▉   | 156025/225000 [304:26:44<111:14:07,  5.81s/it]
 69%|██████▉   | 156026/225000 [304:26:49<107:54:53,  5.63s/it]
 69%|██████▉   | 156027/225000 [304:26:52<94:33:54,  4.94s/it] 
 69%|██████▉   | 156028/225000 [304:27:00<111:15:49,  5.81s/it]
 69%|██████▉   | 156029/225000 [304:27:05<107:45:16,  5.62s/it]
 69%|██████▉   | 156030/225000 [304:27:09<98:23:16,  5.14s/it] 
 69%|██████▉   | 156031/225000 [304:27:15<102:32:46,  5.35s/it]
 69%|██████▉   | 156032/225000 [304:27:23<117:09:45,  6.12s/it]
 69%|██████▉   | 156033/225000 [304:27:28<113:37:13,  5.93s/it]
 69%|██████▉   | 156034/225000 [304:27:33<107:58:08,  5.64s/it]
 69%|██████▉   | 156035/225000 [304:27:37<96:28:31,  5.04s/it] 
 69%|██████▉   | 156036/225000 [304:27:47<126:38:04,  6.61s/it]
 69%|██████▉   | 156037/225000 [304:27:51<108:09:49,  5.65s/it]
 69%|██████▉   | 156038/225000 [304:27:55<98:52:03,  5.16s/it] 
 69%|██████▉   | 156039/225000 [304:28:03<114:18:02,  5.97s/it]
 69%|██████▉   | 156040/225000 [304:28:07<103:50:25,  5.42s/it]
 69%|██████▉   | 156041/225000 [304:28:11<96:16:02,  5.03s/it] 
 69%|██████▉   | 156042/225000 [304:28:17<104:26:47,  5.45s/it]
 69%|██████▉   | 156043/225000 [304:28:26<124:03:19,  6.48s/it]
 69%|██████▉   | 156044/225000 [304:28:31<114:21:14,  5.97s/it]
 69%|██████▉   | 156045/225000 [304:28:36<107:49:06,  5.63s/it]
 69%|██████▉   | 156046/225000 [304:28:40<102:06:59,  5.33s/it]
 69%|██████▉   | 156047/225000 [304:28:45<97:15:57,  5.08s/it] 
 69%|██████▉   | 156048/225000 [304:28:50<96:11:21,  5.02s/it]
 69%|██████▉   | 156049/225000 [304:28:58<114:36:59,  5.98s/it]
 69%|██████▉   | 156050/225000 [304:29:07<130:07:24,  6.79s/it]
                                                               
{'loss': '0.7831', 'grad_norm': '1.087', 'learning_rate': '1.167e-05', 'epoch': '0.6936', 'num_input_tokens_seen': 504222288, 'train_runtime': '1.096e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156050/225000 [304:29:07<130:07:24,  6.79s/it]
 69%|██████▉   | 156051/225000 [304:29:10<110:38:57,  5.78s/it]
 69%|██████▉   | 156052/225000 [304:29:18<121:54:18,  6.37s/it]
 69%|██████▉   | 156053/225000 [304:29:21<104:52:25,  5.48s/it]
 69%|██████▉   | 156054/225000 [304:29:31<127:43:10,  6.67s/it]
 69%|██████▉   | 156055/225000 [304:29:34<108:21:36,  5.66s/it]
 69%|██████▉   | 156056/225000 [304:29:41<115:49:25,  6.05s/it]
 69%|██████▉   | 156057/225000 [304:29:46<107:34:43,  5.62s/it]
 69%|██████▉   | 156058/225000 [304:29:51<108:57:51,  5.69s/it]
 69%|██████▉   | 156059/225000 [304:29:58<116:37:41,  6.09s/it]
 69%|██████▉   | 156060/225000 [304:30:02<100:36:15,  5.25s/it]
 69%|██████▉   | 156061/225000 [304:30:06<92:08:46,  4.81s/it] 
 69%|██████▉   | 156062/225000 [304:30:15<117:39:36,  6.14s/it]
 69%|██████▉   | 156063/225000 [304:30:19<105:46:38,  5.52s/it]
 69%|██████▉   | 156064/225000 [304:30:27<119:25:29,  6.24s/it]
 69%|██████▉   | 156065/225000 [304:30:33<118:23:49,  6.18s/it]
 69%|██████▉   | 156066/225000 [304:30:37<109:23:52,  5.71s/it]
 69%|██████▉   | 156067/225000 [304:30:45<119:13:17,  6.23s/it]
 69%|██████▉   | 156068/225000 [304:30:48<101:07:42,  5.28s/it]
 69%|██████▉   | 156069/225000 [304:30:52<96:34:52,  5.04s/it] 
 69%|██████▉   | 156070/225000 [304:30:58<98:03:42,  5.12s/it]
 69%|██████▉   | 156071/225000 [304:31:03<99:59:49,  5.22s/it]
 69%|██████▉   | 156072/225000 [304:31:12<121:00:10,  6.32s/it]
 69%|██████▉   | 156073/225000 [304:31:17<110:57:27,  5.80s/it]
 69%|██████▉   | 156074/225000 [304:31:21<104:29:05,  5.46s/it]
 69%|██████▉   | 156075/225000 [304:31:26<99:06:57,  5.18s/it] 
 69%|██████▉   | 156076/225000 [304:31:30<95:29:55,  4.99s/it]
 69%|██████▉   | 156077/225000 [304:31:35<94:51:31,  4.95s/it]
 69%|██████▉   | 156078/225000 [304:31:40<92:53:19,  4.85s/it]
 69%|██████▉   | 156079/225000 [304:31:51<129:07:19,  6.74s/it]
 69%|██████▉   | 156080/225000 [304:31:55<112:16:27,  5.86s/it]
 69%|██████▉   | 156081/225000 [304:31:59<103:51:37,  5.43s/it]
 69%|██████▉   | 156082/225000 [304:32:05<105:09:27,  5.49s/it]
 69%|██████▉   | 156083/225000 [304:32:10<102:06:30,  5.33s/it]
 69%|██████▉   | 156084/225000 [304:32:15<101:30:01,  5.30s/it]
 69%|██████▉   | 156085/225000 [304:32:20<99:17:49,  5.19s/it] 
 69%|██████▉   | 156086/225000 [304:32:32<138:48:29,  7.25s/it]
 69%|██████▉   | 156087/225000 [304:32:39<135:12:30,  7.06s/it]
 69%|██████▉   | 156088/225000 [304:32:43<120:26:56,  6.29s/it]
 69%|██████▉   | 156089/225000 [304:32:47<108:24:51,  5.66s/it]
 69%|██████▉   | 156090/225000 [304:32:51<99:15:49,  5.19s/it] 
 69%|██████▉   | 156091/225000 [304:33:01<126:32:16,  6.61s/it]
 69%|██████▉   | 156092/225000 [304:33:08<125:16:06,  6.54s/it]
 69%|██████▉   | 156093/225000 [304:33:12<110:40:55,  5.78s/it]
 69%|██████▉   | 156094/225000 [304:33:16<100:27:20,  5.25s/it]
 69%|██████▉   | 156095/225000 [304:33:20<93:17:20,  4.87s/it] 
 69%|██████▉   | 156096/225000 [304:33:29<119:05:33,  6.22s/it]
 69%|██████▉   | 156097/225000 [304:33:36<123:13:57,  6.44s/it]
 69%|██████▉   | 156098/225000 [304:33:43<124:24:20,  6.50s/it]
 69%|██████▉   | 156099/225000 [304:33:48<117:57:17,  6.16s/it]
 69%|██████▉   | 156100/225000 [304:33:53<107:52:25,  5.64s/it]
                                                               
{'loss': '0.77', 'grad_norm': '1.212', 'learning_rate': '1.166e-05', 'epoch': '0.6938', 'num_input_tokens_seen': 504352136, 'train_runtime': '1.096e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156100/225000 [304:33:53<107:52:25,  5.64s/it]
 69%|██████▉   | 156101/225000 [304:33:57<100:14:03,  5.24s/it]
 69%|██████▉   | 156102/225000 [304:34:01<94:51:07,  4.96s/it] 
 69%|██████▉   | 156103/225000 [304:34:12<128:58:30,  6.74s/it]
 69%|██████▉   | 156104/225000 [304:34:19<131:42:57,  6.88s/it]
 69%|██████▉   | 156105/225000 [304:34:23<114:07:29,  5.96s/it]
 69%|██████▉   | 156106/225000 [304:34:27<103:17:16,  5.40s/it]
 69%|██████▉   | 156107/225000 [304:34:31<94:30:33,  4.94s/it] 
 69%|██████▉   | 156108/225000 [304:34:36<93:12:49,  4.87s/it]
 69%|██████▉   | 156109/225000 [304:34:47<129:48:11,  6.78s/it]
 69%|██████▉   | 156110/225000 [304:34:51<113:23:13,  5.93s/it]
 69%|██████▉   | 156111/225000 [304:34:55<102:55:03,  5.38s/it]
 69%|██████▉   | 156112/225000 [304:35:03<116:09:08,  6.07s/it]
 69%|██████▉   | 156113/225000 [304:35:08<111:03:29,  5.80s/it]
 69%|██████▉   | 156114/225000 [304:35:14<111:20:56,  5.82s/it]
 69%|██████▉   | 156115/225000 [304:35:21<120:01:46,  6.27s/it]
 69%|██████▉   | 156116/225000 [304:35:25<105:01:55,  5.49s/it]
 69%|██████▉   | 156117/225000 [304:35:33<121:37:27,  6.36s/it]
 69%|██████▉   | 156118/225000 [304:35:36<103:46:15,  5.42s/it]
 69%|██████▉   | 156119/225000 [304:35:44<118:39:21,  6.20s/it]
 69%|██████▉   | 156120/225000 [304:35:49<106:59:18,  5.59s/it]
 69%|██████▉   | 156121/225000 [304:35:53<102:52:33,  5.38s/it]
 69%|██████▉   | 156122/225000 [304:35:58<97:50:02,  5.11s/it] 
 69%|██████▉   | 156123/225000 [304:36:02<94:37:53,  4.95s/it]
 69%|██████▉   | 156124/225000 [304:36:07<94:34:49,  4.94s/it]
 69%|██████▉   | 156125/225000 [304:36:13<97:20:22,  5.09s/it]
 69%|██████▉   | 156126/225000 [304:36:18<96:50:46,  5.06s/it]
 69%|██████▉   | 156127/225000 [304:36:23<95:47:27,  5.01s/it]
 69%|██████▉   | 156128/225000 [304:36:30<111:26:30,  5.83s/it]
 69%|██████▉   | 156129/225000 [304:36:42<145:21:20,  7.60s/it]
 69%|██████▉   | 156130/225000 [304:36:46<125:00:24,  6.53s/it]
 69%|██████▉   | 156131/225000 [304:36:50<108:49:51,  5.69s/it]
 69%|██████▉   | 156132/225000 [304:36:56<109:26:57,  5.72s/it]
 69%|██████▉   | 156133/225000 [304:37:04<125:08:00,  6.54s/it]
 69%|██████▉   | 156134/225000 [304:37:11<128:12:42,  6.70s/it]
 69%|██████▉   | 156135/225000 [304:37:15<108:45:08,  5.69s/it]
 69%|██████▉   | 156136/225000 [304:37:22<119:53:21,  6.27s/it]
 69%|██████▉   | 156137/225000 [304:37:27<113:23:20,  5.93s/it]
 69%|██████▉   | 156138/225000 [304:37:32<107:40:33,  5.63s/it]
 69%|██████▉   | 156139/225000 [304:37:37<102:28:04,  5.36s/it]
 69%|██████▉   | 156140/225000 [304:37:42<98:16:48,  5.14s/it] 
 69%|██████▉   | 156141/225000 [304:37:46<94:11:37,  4.92s/it]
 69%|██████▉   | 156142/225000 [304:37:51<94:15:28,  4.93s/it]
 69%|██████▉   | 156143/225000 [304:37:56<94:15:38,  4.93s/it]
 69%|██████▉   | 156144/225000 [304:38:01<94:21:42,  4.93s/it]
 69%|██████▉   | 156145/225000 [304:38:10<118:53:12,  6.22s/it]
 69%|██████▉   | 156146/225000 [304:38:19<132:02:11,  6.90s/it]
 69%|██████▉   | 156147/225000 [304:38:23<119:16:27,  6.24s/it]
 69%|██████▉   | 156148/225000 [304:38:28<109:57:49,  5.75s/it]
 69%|██████▉   | 156149/225000 [304:38:32<101:58:44,  5.33s/it]
 69%|██████▉   | 156150/225000 [304:38:37<95:57:11,  5.02s/it] 
                                                              
{'loss': '0.7757', 'grad_norm': '1.31', 'learning_rate': '1.164e-05', 'epoch': '0.694', 'num_input_tokens_seen': 504482208, 'train_runtime': '1.097e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156150/225000 [304:38:37<95:57:11,  5.02s/it]
 69%|██████▉   | 156151/225000 [304:38:41<92:42:21,  4.85s/it]
 69%|██████▉   | 156152/225000 [304:38:46<94:30:04,  4.94s/it]
 69%|██████▉   | 156153/225000 [304:38:51<93:06:20,  4.87s/it]
 69%|██████▉   | 156154/225000 [304:39:00<119:54:41,  6.27s/it]
 69%|██████▉   | 156155/225000 [304:39:10<139:31:02,  7.30s/it]
 69%|██████▉   | 156156/225000 [304:39:14<117:46:14,  6.16s/it]
 69%|██████▉   | 156157/225000 [304:39:21<124:27:48,  6.51s/it]
 69%|██████▉   | 156158/225000 [304:39:25<113:19:31,  5.93s/it]
 69%|██████▉   | 156159/225000 [304:39:33<121:14:58,  6.34s/it]
 69%|██████▉   | 156160/225000 [304:39:36<104:37:04,  5.47s/it]
 69%|██████▉   | 156161/225000 [304:39:41<100:40:24,  5.26s/it]
 69%|██████▉   | 156162/225000 [304:39:46<97:51:54,  5.12s/it] 
 69%|██████▉   | 156163/225000 [304:39:50<95:15:45,  4.98s/it]
 69%|██████▉   | 156164/225000 [304:39:55<94:31:32,  4.94s/it]
 69%|██████▉   | 156165/225000 [304:40:00<93:44:22,  4.90s/it]
 69%|██████▉   | 156166/225000 [304:40:06<100:20:49,  5.25s/it]
 69%|██████▉   | 156167/225000 [304:40:19<145:30:42,  7.61s/it]
 69%|██████▉   | 156168/225000 [304:40:23<123:40:53,  6.47s/it]
 69%|██████▉   | 156169/225000 [304:40:27<109:55:37,  5.75s/it]
 69%|██████▉   | 156170/225000 [304:40:31<101:41:05,  5.32s/it]
 69%|██████▉   | 156171/225000 [304:40:39<116:41:47,  6.10s/it]
 69%|██████▉   | 156172/225000 [304:40:48<133:01:56,  6.96s/it]
 69%|██████▉   | 156173/225000 [304:40:52<113:29:21,  5.94s/it]
 69%|██████▉   | 156174/225000 [304:41:00<125:29:40,  6.56s/it]
 69%|██████▉   | 156175/225000 [304:41:05<114:59:57,  6.02s/it]
 69%|██████▉   | 156176/225000 [304:41:09<105:42:54,  5.53s/it]
 69%|██████▉   | 156177/225000 [304:41:13<99:18:19,  5.19s/it] 
 69%|██████▉   | 156178/225000 [304:41:18<95:17:51,  4.98s/it]
 69%|██████▉   | 156179/225000 [304:41:29<128:31:36,  6.72s/it]
 69%|██████▉   | 156180/225000 [304:41:33<115:15:02,  6.03s/it]
 69%|██████▉   | 156181/225000 [304:41:40<118:22:27,  6.19s/it]
 69%|██████▉   | 156182/225000 [304:41:43<103:24:23,  5.41s/it]
 69%|██████▉   | 156183/225000 [304:41:48<96:28:56,  5.05s/it] 
 69%|██████▉   | 156184/225000 [304:41:58<127:09:06,  6.65s/it]
 69%|██████▉   | 156185/225000 [304:42:01<109:09:14,  5.71s/it]
 69%|██████▉   | 156186/225000 [304:42:06<104:44:13,  5.48s/it]
 69%|██████▉   | 156187/225000 [304:42:13<112:17:28,  5.87s/it]
 69%|██████▉   | 156188/225000 [304:42:17<101:38:42,  5.32s/it]
 69%|██████▉   | 156189/225000 [304:42:26<120:21:57,  6.30s/it]
 69%|██████▉   | 156190/225000 [304:42:31<111:29:48,  5.83s/it]
 69%|██████▉   | 156191/225000 [304:42:35<102:17:54,  5.35s/it]
 69%|██████▉   | 156192/225000 [304:42:39<98:09:46,  5.14s/it] 
 69%|██████▉   | 156193/225000 [304:42:44<95:29:20,  5.00s/it]
 69%|██████▉   | 156194/225000 [304:42:48<92:15:46,  4.83s/it]
 69%|██████▉   | 156195/225000 [304:42:53<93:26:30,  4.89s/it]
 69%|██████▉   | 156196/225000 [304:43:05<132:15:34,  6.92s/it]
 69%|██████▉   | 156197/225000 [304:43:11<128:36:56,  6.73s/it]
 69%|██████▉   | 156198/225000 [304:43:17<124:26:30,  6.51s/it]
 69%|██████▉   | 156199/225000 [304:43:22<110:53:45,  5.80s/it]
 69%|██████▉   | 156200/225000 [304:43:26<105:06:50,  5.50s/it]
                                                               
{'loss': '0.7906', 'grad_norm': '1.081', 'learning_rate': '1.163e-05', 'epoch': '0.6942', 'num_input_tokens_seen': 504612448, 'train_runtime': '1.097e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156200/225000 [304:43:26<105:06:50,  5.50s/it]
 69%|██████▉   | 156201/225000 [304:43:31<99:43:20,  5.22s/it] 
 69%|██████▉   | 156202/225000 [304:43:36<97:13:51,  5.09s/it]
 69%|██████▉   | 156203/225000 [304:43:41<98:36:11,  5.16s/it]
 69%|██████▉   | 156204/225000 [304:43:46<98:11:50,  5.14s/it]
 69%|██████▉   | 156205/225000 [304:43:51<97:51:50,  5.12s/it]
 69%|██████▉   | 156206/225000 [304:43:56<98:31:23,  5.16s/it]
 69%|██████▉   | 156207/225000 [304:44:02<98:17:50,  5.14s/it]
 69%|██████▉   | 156208/225000 [304:44:07<97:06:21,  5.08s/it]
 69%|██████▉   | 156209/225000 [304:44:12<99:00:00,  5.18s/it]
 69%|██████▉   | 156210/225000 [304:44:18<106:04:41,  5.55s/it]
 69%|██████▉   | 156211/225000 [304:44:31<144:38:01,  7.57s/it]
 69%|██████▉   | 156212/225000 [304:44:35<125:42:30,  6.58s/it]
 69%|██████▉   | 156213/225000 [304:44:39<112:59:52,  5.91s/it]
 69%|██████▉   | 156214/225000 [304:44:43<101:12:06,  5.30s/it]
 69%|██████▉   | 156215/225000 [304:44:53<127:22:21,  6.67s/it]
 69%|██████▉   | 156216/225000 [304:44:58<118:34:55,  6.21s/it]
 69%|██████▉   | 156217/225000 [304:45:02<103:43:48,  5.43s/it]
 69%|██████▉   | 156218/225000 [304:45:06<98:46:43,  5.17s/it] 
 69%|██████▉   | 156219/225000 [304:45:11<95:41:22,  5.01s/it]
 69%|██████▉   | 156220/225000 [304:45:19<111:35:29,  5.84s/it]
 69%|██████▉   | 156221/225000 [304:45:28<129:54:08,  6.80s/it]
 69%|██████▉   | 156222/225000 [304:45:31<109:15:00,  5.72s/it]
 69%|██████▉   | 156223/225000 [304:45:35<101:24:26,  5.31s/it]
 69%|██████▉   | 156224/225000 [304:45:39<93:57:44,  4.92s/it] 
 69%|██████▉   | 156225/225000 [304:45:45<98:26:14,  5.15s/it]
 69%|██████▉   | 156226/225000 [304:45:56<133:29:53,  6.99s/it]
 69%|██████▉   | 156227/225000 [304:45:59<110:36:22,  5.79s/it]
 69%|██████▉   | 156228/225000 [304:46:03<97:34:42,  5.11s/it] 
 69%|██████▉   | 156229/225000 [304:46:13<124:08:46,  6.50s/it]
 69%|██████▉   | 156230/225000 [304:46:16<105:44:30,  5.54s/it]
 69%|██████▉   | 156231/225000 [304:46:23<114:46:20,  6.01s/it]
 69%|██████▉   | 156232/225000 [304:46:27<101:33:51,  5.32s/it]
 69%|██████▉   | 156233/225000 [304:46:32<102:11:24,  5.35s/it]
 69%|██████▉   | 156234/225000 [304:46:37<101:12:20,  5.30s/it]
 69%|██████▉   | 156235/225000 [304:46:42<99:31:38,  5.21s/it] 
 69%|██████▉   | 156236/225000 [304:46:47<98:31:19,  5.16s/it]
 69%|██████▉   | 156237/225000 [304:46:52<98:41:19,  5.17s/it]
 69%|██████▉   | 156238/225000 [304:46:58<98:57:09,  5.18s/it]
 69%|██████▉   | 156239/225000 [304:47:06<115:26:53,  6.04s/it]
 69%|██████▉   | 156240/225000 [304:47:15<133:17:55,  6.98s/it]
 69%|██████▉   | 156241/225000 [304:47:19<118:29:17,  6.20s/it]
 69%|██████▉   | 156242/225000 [304:47:24<107:53:46,  5.65s/it]
 69%|██████▉   | 156243/225000 [304:47:28<98:25:08,  5.15s/it] 
 69%|██████▉   | 156244/225000 [304:47:38<125:27:32,  6.57s/it]
 69%|██████▉   | 156245/225000 [304:47:41<110:37:19,  5.79s/it]
 69%|██████▉   | 156246/225000 [304:47:46<103:02:30,  5.40s/it]
 69%|██████▉   | 156247/225000 [304:47:50<97:05:26,  5.08s/it] 
 69%|██████▉   | 156248/225000 [304:47:58<110:16:59,  5.77s/it]
 69%|██████▉   | 156249/225000 [304:48:08<135:19:17,  7.09s/it]
 69%|██████▉   | 156250/225000 [304:48:11<113:08:17,  5.92s/it]
                                                               
{'loss': '0.7755', 'grad_norm': '1.123', 'learning_rate': '1.161e-05', 'epoch': '0.6944', 'num_input_tokens_seen': 504741968, 'train_runtime': '1.097e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156250/225000 [304:48:11<113:08:17,  5.92s/it]
 69%|██████▉   | 156251/225000 [304:48:16<110:08:40,  5.77s/it]
 69%|██████▉   | 156252/225000 [304:48:22<111:23:52,  5.83s/it]
 69%|██████▉   | 156253/225000 [304:48:26<98:42:19,  5.17s/it] 
 69%|██████▉   | 156254/225000 [304:48:35<120:15:26,  6.30s/it]
 69%|██████▉   | 156255/225000 [304:48:38<102:48:37,  5.38s/it]
 69%|██████▉   | 156256/225000 [304:48:47<120:20:32,  6.30s/it]
 69%|██████▉   | 156257/225000 [304:48:51<109:11:27,  5.72s/it]
 69%|██████▉   | 156258/225000 [304:48:55<100:17:58,  5.25s/it]
 69%|██████▉   | 156259/225000 [304:49:00<97:30:37,  5.11s/it] 
 69%|██████▉   | 156260/225000 [304:49:04<93:53:09,  4.92s/it]
 69%|██████▉   | 156261/225000 [304:49:09<91:51:57,  4.81s/it]
 69%|██████▉   | 156262/225000 [304:49:15<100:51:59,  5.28s/it]
 69%|██████▉   | 156263/225000 [304:49:26<133:52:41,  7.01s/it]
 69%|██████▉   | 156264/225000 [304:49:30<113:51:16,  5.96s/it]
 69%|██████▉   | 156265/225000 [304:49:34<100:11:34,  5.25s/it]
 69%|██████▉   | 156266/225000 [304:49:42<120:46:38,  6.33s/it]
 69%|██████▉   | 156267/225000 [304:49:46<106:17:42,  5.57s/it]
 69%|██████▉   | 156268/225000 [304:49:52<109:30:57,  5.74s/it]
 69%|██████▉   | 156269/225000 [304:49:57<106:19:55,  5.57s/it]
 69%|██████▉   | 156270/225000 [304:50:01<96:59:59,  5.08s/it] 
 69%|██████▉   | 156271/225000 [304:50:06<91:27:13,  4.79s/it]
 69%|██████▉   | 156272/225000 [304:50:17<131:29:29,  6.89s/it]
 69%|██████▉   | 156273/225000 [304:50:21<114:47:14,  6.01s/it]
 69%|██████▉   | 156274/225000 [304:50:25<103:58:34,  5.45s/it]
 69%|██████▉   | 156275/225000 [304:50:30<96:10:47,  5.04s/it] 
 69%|██████▉   | 156276/225000 [304:50:38<114:22:36,  5.99s/it]
 69%|██████▉   | 156277/225000 [304:50:47<131:44:23,  6.90s/it]
 69%|██████▉   | 156278/225000 [304:50:50<110:37:08,  5.79s/it]
 69%|██████▉   | 156279/225000 [304:50:56<113:17:58,  5.94s/it]
 69%|██████▉   | 156280/225000 [304:51:03<115:29:36,  6.05s/it]
 69%|██████▉   | 156281/225000 [304:51:06<100:55:54,  5.29s/it]
 69%|██████▉   | 156282/225000 [304:51:15<122:05:50,  6.40s/it]
 69%|██████▉   | 156283/225000 [304:51:18<104:15:00,  5.46s/it]
 69%|██████▉   | 156284/225000 [304:51:27<120:30:54,  6.31s/it]
 69%|██████▉   | 156285/225000 [304:51:31<111:34:18,  5.85s/it]
 69%|██████▉   | 156286/225000 [304:51:36<105:44:50,  5.54s/it]
 69%|██████▉   | 156287/225000 [304:51:41<99:25:45,  5.21s/it] 
 69%|██████▉   | 156288/225000 [304:51:45<97:27:16,  5.11s/it]
 69%|██████▉   | 156289/225000 [304:51:50<95:45:42,  5.02s/it]
 69%|██████▉   | 156290/225000 [304:51:55<94:09:29,  4.93s/it]
 69%|██████▉   | 156291/225000 [304:52:08<140:04:10,  7.34s/it]
 69%|██████▉   | 156292/225000 [304:52:14<129:47:53,  6.80s/it]
 69%|██████▉   | 156293/225000 [304:52:19<121:41:41,  6.38s/it]
 69%|██████▉   | 156294/225000 [304:52:23<106:12:25,  5.56s/it]
 69%|██████▉   | 156295/225000 [304:52:27<97:24:07,  5.10s/it] 
 69%|██████▉   | 156296/225000 [304:52:31<93:01:29,  4.87s/it]
 69%|██████▉   | 156297/225000 [304:52:43<133:39:33,  7.00s/it]
 69%|██████▉   | 156298/225000 [304:52:47<114:44:18,  6.01s/it]
 69%|██████▉   | 156299/225000 [304:52:50<100:24:48,  5.26s/it]
 69%|██████▉   | 156300/225000 [304:52:59<119:09:27,  6.24s/it]
                                                               
{'loss': '0.7752', 'grad_norm': '1.056', 'learning_rate': '1.16e-05', 'epoch': '0.6947', 'num_input_tokens_seen': 504871912, 'train_runtime': '1.098e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156300/225000 [304:52:59<119:09:27,  6.24s/it]
 69%|██████▉   | 156301/225000 [304:53:02<103:06:17,  5.40s/it]
 69%|██████▉   | 156302/225000 [304:53:11<123:53:27,  6.49s/it]
 69%|██████▉   | 156303/225000 [304:53:14<103:33:25,  5.43s/it]
 69%|██████▉   | 156304/225000 [304:53:21<113:03:00,  5.92s/it]
 69%|██████▉   | 156305/225000 [304:53:25<98:15:34,  5.15s/it] 
 69%|██████▉   | 156306/225000 [304:53:31<105:50:30,  5.55s/it]
 69%|██████▉   | 156307/225000 [304:53:36<104:16:24,  5.46s/it]
 69%|██████▉   | 156308/225000 [304:53:40<96:32:44,  5.06s/it] 
 69%|██████▉   | 156309/225000 [304:53:45<91:43:30,  4.81s/it]
 69%|██████▉   | 156310/225000 [304:53:49<88:00:07,  4.61s/it]
 69%|██████▉   | 156311/225000 [304:53:53<88:21:13,  4.63s/it]
 69%|██████▉   | 156312/225000 [304:54:03<114:06:20,  5.98s/it]
 69%|██████▉   | 156313/225000 [304:54:10<122:04:49,  6.40s/it]
 69%|██████▉   | 156314/225000 [304:54:17<124:37:07,  6.53s/it]
 69%|██████▉   | 156315/225000 [304:54:21<108:34:30,  5.69s/it]
 69%|██████▉   | 156316/225000 [304:54:24<96:47:11,  5.07s/it] 
 69%|██████▉   | 156317/225000 [304:54:28<92:37:45,  4.86s/it]
 69%|██████▉   | 156318/225000 [304:54:38<119:34:16,  6.27s/it]
 69%|██████▉   | 156319/225000 [304:54:42<105:30:58,  5.53s/it]
 69%|██████▉   | 156320/225000 [304:54:46<95:54:04,  5.03s/it] 
 69%|██████▉   | 156321/225000 [304:54:56<123:54:29,  6.49s/it]
 69%|██████▉   | 156322/225000 [304:54:59<105:34:14,  5.53s/it]
 69%|██████▉   | 156323/225000 [304:55:07<122:24:31,  6.42s/it]
 69%|██████▉   | 156324/225000 [304:55:12<110:11:47,  5.78s/it]
 69%|██████▉   | 156325/225000 [304:55:16<100:47:08,  5.28s/it]
 69%|██████▉   | 156326/225000 [304:55:20<94:26:26,  4.95s/it] 
 69%|██████▉   | 156327/225000 [304:55:24<89:59:30,  4.72s/it]
 69%|██████▉   | 156328/225000 [304:55:37<134:43:55,  7.06s/it]
 69%|██████▉   | 156329/225000 [304:55:40<115:06:06,  6.03s/it]
 69%|██████▉   | 156330/225000 [304:55:44<103:52:32,  5.45s/it]
 69%|██████▉   | 156331/225000 [304:55:50<105:52:41,  5.55s/it]
 69%|██████▉   | 156332/225000 [304:56:00<127:45:21,  6.70s/it]
 69%|██████▉   | 156333/225000 [304:56:03<108:02:42,  5.66s/it]
 69%|██████▉   | 156334/225000 [304:56:10<118:13:03,  6.20s/it]
 69%|██████▉   | 156335/225000 [304:56:14<103:58:52,  5.45s/it]
 69%|██████▉   | 156336/225000 [304:56:18<97:25:09,  5.11s/it] 
 69%|██████▉   | 156337/225000 [304:56:28<122:17:51,  6.41s/it]
 69%|██████▉   | 156338/225000 [304:56:31<103:37:12,  5.43s/it]
 69%|██████▉   | 156339/225000 [304:56:35<93:14:07,  4.89s/it] 
 69%|██████▉   | 156340/225000 [304:56:41<100:12:32,  5.25s/it]
 69%|██████▉   | 156341/225000 [304:56:50<121:12:00,  6.35s/it]
 69%|██████▉   | 156342/225000 [304:56:53<103:56:42,  5.45s/it]
 69%|██████▉   | 156343/225000 [304:57:02<123:54:55,  6.50s/it]
 69%|██████▉   | 156344/225000 [304:57:05<106:18:06,  5.57s/it]
 69%|██████▉   | 156345/225000 [304:57:09<95:00:38,  4.98s/it] 
 69%|██████▉   | 156346/225000 [304:57:19<122:22:25,  6.42s/it]
 69%|██████▉   | 156347/225000 [304:57:22<106:09:37,  5.57s/it]
 69%|██████▉   | 156348/225000 [304:57:26<97:09:09,  5.09s/it] 
 69%|██████▉   | 156349/225000 [304:57:36<123:49:35,  6.49s/it]
 69%|██████▉   | 156350/225000 [304:57:40<110:11:10,  5.78s/it]
                                                               
{'loss': '0.7808', 'grad_norm': '1.036', 'learning_rate': '1.158e-05', 'epoch': '0.6949', 'num_input_tokens_seen': 505002112, 'train_runtime': '1.098e+06', 'train_tokens_per_second': '460'}

 69%|██████▉   | 156350/225000 [304:57:40<110:11:10,  5.78s/it]
 69%|██████▉   | 156351/225000 [304:57:44<100:46:56,  5.29s/it]
 69%|██████▉   | 156352/225000 [304:57:53<118:34:01,  6.22s/it]
 69%|██████▉   | 156353/225000 [304:57:58<113:17:32,  5.94s/it]
 69%|██████▉   | 156354/225000 [304:58:04<114:09:06,  5.99s/it]
 69%|██████▉   | 156355/225000 [304:58:08<105:24:40,  5.53s/it]
 69%|██████▉   | 156356/225000 [304:58:15<112:41:05,  5.91s/it]
 69%|██████▉   | 156357/225000 [304:58:22<120:10:42,  6.30s/it]
 69%|██████▉   | 156358/225000 [304:58:27<109:57:46,  5.77s/it]
 69%|██████▉   | 156359/225000 [304:58:31<101:22:27,  5.32s/it]
 69%|██████▉   | 156360/225000 [304:58:36<95:40:19,  5.02s/it] 
 69%|██████▉   | 156361/225000 [304:58:40<92:48:09,  4.87s/it]
 69%|██████▉   | 156362/225000 [304:58:51<127:58:31,  6.71s/it]
 69%|██████▉   | 156363/225000 [304:58:56<118:41:34,  6.23s/it]
 69%|██████▉   | 156364/225000 [304:59:00<102:56:57,  5.40s/it]
 69%|██████▉   | 156365/225000 [304:59:06<111:09:24,  5.83s/it]
 69%|██████▉   | 156366/225000 [304:59:14<121:34:21,  6.38s/it]
 69%|██████▉   | 156367/225000 [304:59:18<105:56:45,  5.56s/it]
 69%|██████▉   | 156368/225000 [304:59:21<95:13:25,  4.99s/it] 
 69%|██████▉   | 156369/225000 [304:59:32<128:53:53,  6.76s/it]
 69%|██████▉   | 156370/225000 [304:59:37<114:57:34,  6.03s/it]
 69%|██████▉   | 156371/225000 [304:59:42<108:35:49,  5.70s/it]
 69%|██████▉   | 156372/225000 [304:59:50<126:23:47,  6.63s/it]
 69%|██████▉   | 156373/225000 [304:59:54<108:13:04,  5.68s/it]
 69%|██████▉   | 156374/225000 [305:00:02<120:31:54,  6.32s/it]
 70%|██████▉   | 156375/225000 [305:00:06<108:12:01,  5.68s/it]
 70%|██████▉   | 156376/225000 [305:00:11<102:32:04,  5.38s/it]
 70%|██████▉   | 156377/225000 [305:00:15<99:39:29,  5.23s/it] 
 70%|██████▉   | 156378/225000 [305:00:20<97:08:03,  5.10s/it]
 70%|██████▉   | 156379/225000 [305:00:25<96:21:59,  5.06s/it]
 70%|██████▉   | 156380/225000 [305:00:30<95:57:43,  5.03s/it]
 70%|██████▉   | 156381/225000 [305:00:35<95:18:42,  5.00s/it]
 70%|██████▉   | 156382/225000 [305:00:41<99:33:52,  5.22s/it]
 70%|██████▉   | 156383/225000 [305:00:53<140:33:16,  7.37s/it]
 70%|██████▉   | 156384/225000 [305:00:57<121:58:15,  6.40s/it]
 70%|██████▉   | 156385/225000 [305:01:02<111:30:57,  5.85s/it]
 70%|██████▉   | 156386/225000 [305:01:06<102:31:36,  5.38s/it]
 70%|██████▉   | 156387/225000 [305:01:11<98:24:16,  5.16s/it] 
 70%|██████▉   | 156388/225000 [305:01:15<93:33:49,  4.91s/it]
 70%|██████▉   | 156389/225000 [305:01:20<90:42:27,  4.76s/it]
 70%|██████▉   | 156390/225000 [305:01:31<130:28:38,  6.85s/it]
 70%|██████▉   | 156391/225000 [305:01:38<129:26:02,  6.79s/it]
 70%|██████▉   | 156392/225000 [305:01:42<116:11:59,  6.10s/it]
 70%|██████▉   | 156393/225000 [305:01:47<107:25:16,  5.64s/it]
 70%|██████▉   | 156394/225000 [305:01:52<102:48:20,  5.39s/it]
 70%|██████▉   | 156395/225000 [305:01:56<98:18:18,  5.16s/it] 
 70%|██████▉   | 156396/225000 [305:02:01<97:44:41,  5.13s/it]
 70%|██████▉   | 156397/225000 [305:02:14<138:53:53,  7.29s/it]
 70%|██████▉   | 156398/225000 [305:02:17<116:19:15,  6.10s/it]
 70%|██████▉   | 156399/225000 [305:02:24<123:14:24,  6.47s/it]
 70%|██████▉   | 156400/225000 [305:02:29<114:04:31,  5.99s/it]
                                                               
{'loss': '0.7813', 'grad_norm': '1.134', 'learning_rate': '1.156e-05', 'epoch': '0.6951', 'num_input_tokens_seen': 505131616, 'train_runtime': '1.098e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156400/225000 [305:02:29<114:04:31,  5.99s/it]
 70%|██████▉   | 156401/225000 [305:02:34<107:55:03,  5.66s/it]
 70%|██████▉   | 156402/225000 [305:02:39<100:25:04,  5.27s/it]
 70%|██████▉   | 156403/225000 [305:02:44<98:41:50,  5.18s/it] 
 70%|██████▉   | 156404/225000 [305:02:48<96:39:02,  5.07s/it]
 70%|██████▉   | 156405/225000 [305:02:53<95:50:43,  5.03s/it]
 70%|██████▉   | 156406/225000 [305:03:01<111:02:06,  5.83s/it]
 70%|██████▉   | 156407/225000 [305:03:11<135:53:53,  7.13s/it]
 70%|██████▉   | 156408/225000 [305:03:15<114:36:13,  6.01s/it]
 70%|██████▉   | 156409/225000 [305:03:18<101:46:58,  5.34s/it]
 70%|██████▉   | 156410/225000 [305:03:24<106:15:15,  5.58s/it]
 70%|██████▉   | 156411/225000 [305:03:32<115:29:11,  6.06s/it]
 70%|██████▉   | 156412/225000 [305:03:36<107:42:31,  5.65s/it]
 70%|██████▉   | 156413/225000 [305:03:45<126:55:45,  6.66s/it]
 70%|██████▉   | 156414/225000 [305:03:49<108:48:38,  5.71s/it]
 70%|██████▉   | 156415/225000 [305:03:52<95:45:52,  5.03s/it] 
 70%|██████▉   | 156416/225000 [305:03:58<98:32:24,  5.17s/it]
 70%|██████▉   | 156417/225000 [305:04:03<98:53:27,  5.19s/it]
 70%|██████▉   | 156418/225000 [305:04:09<101:45:28,  5.34s/it]
 70%|██████▉   | 156419/225000 [305:04:14<100:58:13,  5.30s/it]
 70%|██████▉   | 156420/225000 [305:04:19<100:52:06,  5.29s/it]
 70%|██████▉   | 156421/225000 [305:04:24<99:40:59,  5.23s/it] 
 70%|██████▉   | 156422/225000 [305:04:30<99:24:52,  5.22s/it]
 70%|██████▉   | 156423/225000 [305:04:35<100:30:53,  5.28s/it]
 70%|██████▉   | 156424/225000 [305:04:40<100:41:38,  5.29s/it]
 70%|██████▉   | 156425/225000 [305:04:46<102:05:55,  5.36s/it]
 70%|██████▉   | 156426/225000 [305:04:51<102:51:43,  5.40s/it]
 70%|██████▉   | 156427/225000 [305:04:57<103:31:48,  5.44s/it]
 70%|██████▉   | 156428/225000 [305:05:01<98:29:48,  5.17s/it] 
 70%|██████▉   | 156429/225000 [305:05:06<98:25:57,  5.17s/it]
 70%|██████▉   | 156430/225000 [305:05:19<137:55:16,  7.24s/it]
 70%|██████▉   | 156431/225000 [305:05:26<136:30:14,  7.17s/it]
 70%|██████▉   | 156432/225000 [305:05:29<115:00:00,  6.04s/it]
 70%|██████▉   | 156433/225000 [305:05:38<132:41:34,  6.97s/it]
 70%|██████▉   | 156434/225000 [305:05:42<112:22:22,  5.90s/it]
 70%|██████▉   | 156435/225000 [305:05:50<126:19:11,  6.63s/it]
 70%|██████▉   | 156436/225000 [305:05:54<110:24:30,  5.80s/it]
 70%|██████▉   | 156437/225000 [305:05:58<99:03:52,  5.20s/it] 
 70%|██████▉   | 156438/225000 [305:06:04<104:14:51,  5.47s/it]
 70%|██████▉   | 156439/225000 [305:06:13<123:57:01,  6.51s/it]
 70%|██████▉   | 156440/225000 [305:06:16<108:38:11,  5.70s/it]
 70%|██████▉   | 156441/225000 [305:06:20<97:06:32,  5.10s/it] 
 70%|██████▉   | 156442/225000 [305:06:29<119:22:16,  6.27s/it]
 70%|██████▉   | 156443/225000 [305:06:35<114:50:55,  6.03s/it]
 70%|██████▉   | 156444/225000 [305:06:38<102:01:45,  5.36s/it]
 70%|██████▉   | 156445/225000 [305:06:42<93:16:50,  4.90s/it] 
 70%|██████▉   | 156446/225000 [305:06:50<110:42:04,  5.81s/it]
 70%|██████▉   | 156447/225000 [305:06:58<123:54:29,  6.51s/it]
 70%|██████▉   | 156448/225000 [305:07:02<106:51:59,  5.61s/it]
 70%|██████▉   | 156449/225000 [305:07:10<119:37:58,  6.28s/it]
 70%|██████▉   | 156450/225000 [305:07:14<106:57:41,  5.62s/it]
                                                               
{'loss': '0.7851', 'grad_norm': '1.088', 'learning_rate': '1.155e-05', 'epoch': '0.6953', 'num_input_tokens_seen': 505260984, 'train_runtime': '1.098e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156450/225000 [305:07:14<106:57:41,  5.62s/it]
 70%|██████▉   | 156451/225000 [305:07:17<94:46:30,  4.98s/it] 
 70%|██████▉   | 156452/225000 [305:07:29<133:11:52,  7.00s/it]
 70%|██████▉   | 156453/225000 [305:07:32<112:25:00,  5.90s/it]
 70%|██████▉   | 156454/225000 [305:07:40<126:04:20,  6.62s/it]
 70%|██████▉   | 156455/225000 [305:07:44<107:40:19,  5.65s/it]
 70%|██████▉   | 156456/225000 [305:07:52<120:33:56,  6.33s/it]
 70%|██████▉   | 156457/225000 [305:07:55<102:58:39,  5.41s/it]
 70%|██████▉   | 156458/225000 [305:08:02<110:56:16,  5.83s/it]
 70%|██████▉   | 156459/225000 [305:08:06<98:50:30,  5.19s/it] 
 70%|██████▉   | 156460/225000 [305:08:15<121:44:23,  6.39s/it]
 70%|██████▉   | 156461/225000 [305:08:18<103:29:29,  5.44s/it]
 70%|██████▉   | 156462/225000 [305:08:22<93:45:52,  4.93s/it] 
 70%|██████▉   | 156463/225000 [305:08:26<89:12:28,  4.69s/it]
 70%|██████▉   | 156464/225000 [305:08:35<114:25:46,  6.01s/it]
 70%|██████▉   | 156465/225000 [305:08:41<112:01:06,  5.88s/it]
 70%|██████▉   | 156466/225000 [305:08:44<101:08:36,  5.31s/it]
 70%|██████▉   | 156467/225000 [305:08:49<93:38:56,  4.92s/it] 
 70%|██████▉   | 156468/225000 [305:08:52<88:17:31,  4.64s/it]
 70%|██████▉   | 156469/225000 [305:09:05<135:21:49,  7.11s/it]
 70%|██████▉   | 156470/225000 [305:09:10<119:03:37,  6.25s/it]
 70%|██████▉   | 156471/225000 [305:09:14<109:52:23,  5.77s/it]
 70%|██████▉   | 156472/225000 [305:09:19<101:09:05,  5.31s/it]
 70%|██████▉   | 156473/225000 [305:09:23<98:35:10,  5.18s/it] 
 70%|██████▉   | 156474/225000 [305:09:30<106:05:14,  5.57s/it]
 70%|██████▉   | 156475/225000 [305:09:41<135:34:58,  7.12s/it]
 70%|██████▉   | 156476/225000 [305:09:44<114:19:11,  6.01s/it]
 70%|██████▉   | 156477/225000 [305:09:48<104:18:49,  5.48s/it]
 70%|██████▉   | 156478/225000 [305:09:55<110:37:32,  5.81s/it]
 70%|██████▉   | 156479/225000 [305:10:01<111:42:22,  5.87s/it]
 70%|██████▉   | 156480/225000 [305:10:09<122:10:08,  6.42s/it]
 70%|██████▉   | 156481/225000 [305:10:13<109:40:01,  5.76s/it]
 70%|██████▉   | 156482/225000 [305:10:17<99:29:23,  5.23s/it] 
 70%|██████▉   | 156483/225000 [305:10:26<120:12:15,  6.32s/it]
 70%|██████▉   | 156484/225000 [305:10:29<102:41:14,  5.40s/it]
 70%|██████▉   | 156485/225000 [305:10:37<115:30:40,  6.07s/it]
 70%|██████▉   | 156486/225000 [305:10:40<100:15:27,  5.27s/it]
 70%|██████▉   | 156487/225000 [305:10:44<90:51:24,  4.77s/it] 
 70%|██████▉   | 156488/225000 [305:10:53<117:46:53,  6.19s/it]
 70%|██████▉   | 156489/225000 [305:10:59<114:34:46,  6.02s/it]
 70%|██████▉   | 156490/225000 [305:11:04<111:10:44,  5.84s/it]
 70%|██████▉   | 156491/225000 [305:11:10<111:48:41,  5.88s/it]
 70%|██████▉   | 156492/225000 [305:11:15<104:12:49,  5.48s/it]
 70%|██████▉   | 156493/225000 [305:11:19<100:33:49,  5.28s/it]
 70%|██████▉   | 156494/225000 [305:11:24<96:46:10,  5.09s/it] 
 70%|██████▉   | 156495/225000 [305:11:29<94:49:16,  4.98s/it]
 70%|██████▉   | 156496/225000 [305:11:35<104:48:00,  5.51s/it]
 70%|██████▉   | 156497/225000 [305:11:47<136:55:57,  7.20s/it]
 70%|██████▉   | 156498/225000 [305:11:50<114:10:36,  6.00s/it]
 70%|██████▉   | 156499/225000 [305:11:58<127:57:56,  6.73s/it]
 70%|██████▉   | 156500/225000 [305:12:02<112:21:40,  5.91s/it]
                                                               
{'loss': '0.7673', 'grad_norm': '1.147', 'learning_rate': '1.153e-05', 'epoch': '0.6956', 'num_input_tokens_seen': 505392168, 'train_runtime': '1.099e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156500/225000 [305:12:02<112:21:40,  5.91s/it]
 70%|██████▉   | 156501/225000 [305:12:09<115:42:18,  6.08s/it]
 70%|██████▉   | 156502/225000 [305:12:15<115:04:00,  6.05s/it]
 70%|██████▉   | 156503/225000 [305:12:18<100:34:55,  5.29s/it]
 70%|██████▉   | 156504/225000 [305:12:22<94:41:36,  4.98s/it] 
 70%|██████▉   | 156505/225000 [305:12:27<91:06:54,  4.79s/it]
 70%|██████▉   | 156506/225000 [305:12:33<96:22:13,  5.07s/it]
 70%|██████▉   | 156507/225000 [305:12:42<121:46:40,  6.40s/it]
 70%|██████▉   | 156508/225000 [305:12:46<107:35:29,  5.66s/it]
 70%|██████▉   | 156509/225000 [305:12:51<104:26:08,  5.49s/it]
 70%|██████▉   | 156510/225000 [305:12:56<100:29:22,  5.28s/it]
 70%|██████▉   | 156511/225000 [305:13:01<101:57:58,  5.36s/it]
 70%|██████▉   | 156512/225000 [305:13:08<110:28:11,  5.81s/it]
 70%|██████▉   | 156513/225000 [305:13:17<124:27:27,  6.54s/it]
 70%|██████▉   | 156514/225000 [305:13:21<112:28:13,  5.91s/it]
 70%|██████▉   | 156515/225000 [305:13:25<103:53:51,  5.46s/it]
 70%|██████▉   | 156516/225000 [305:13:30<98:19:59,  5.17s/it] 
 70%|██████▉   | 156517/225000 [305:13:42<137:45:23,  7.24s/it]
 70%|██████▉   | 156518/225000 [305:13:47<122:41:57,  6.45s/it]
 70%|██████▉   | 156519/225000 [305:13:51<109:58:39,  5.78s/it]
 70%|██████▉   | 156520/225000 [305:13:55<102:40:15,  5.40s/it]
 70%|██████▉   | 156521/225000 [305:14:00<98:04:02,  5.16s/it] 
 70%|██████▉   | 156522/225000 [305:14:05<99:22:56,  5.22s/it]
 70%|██████▉   | 156523/225000 [305:14:18<141:13:01,  7.42s/it]
 70%|██████▉   | 156524/225000 [305:14:22<123:35:55,  6.50s/it]
 70%|██████▉   | 156525/225000 [305:14:26<106:04:05,  5.58s/it]
 70%|██████▉   | 156526/225000 [305:14:30<99:13:15,  5.22s/it] 
 70%|██████▉   | 156527/225000 [305:14:35<96:24:35,  5.07s/it]
 70%|██████▉   | 156528/225000 [305:14:40<98:52:30,  5.20s/it]
 70%|██████▉   | 156529/225000 [305:14:52<136:03:44,  7.15s/it]
 70%|██████▉   | 156530/225000 [305:14:55<113:49:38,  5.98s/it]
 70%|██████▉   | 156531/225000 [305:14:59<99:05:20,  5.21s/it] 
 70%|██████▉   | 156532/225000 [305:15:04<99:08:23,  5.21s/it]
 70%|██████▉   | 156533/225000 [305:15:09<99:40:00,  5.24s/it]
 70%|██████▉   | 156534/225000 [305:15:15<103:03:56,  5.42s/it]
 70%|██████▉   | 156535/225000 [305:15:20<102:22:53,  5.38s/it]
 70%|██████▉   | 156536/225000 [305:15:26<102:38:25,  5.40s/it]
 70%|██████▉   | 156537/225000 [305:15:31<103:49:34,  5.46s/it]
 70%|██████▉   | 156538/225000 [305:15:40<121:29:25,  6.39s/it]
 70%|██████▉   | 156539/225000 [305:15:44<109:47:51,  5.77s/it]
 70%|██████▉   | 156540/225000 [305:15:48<100:19:20,  5.28s/it]
 70%|██████▉   | 156541/225000 [305:15:53<95:35:59,  5.03s/it] 
 70%|██████▉   | 156542/225000 [305:16:00<106:24:15,  5.60s/it]
 70%|██████▉   | 156543/225000 [305:16:11<141:24:35,  7.44s/it]
 70%|██████▉   | 156544/225000 [305:16:15<117:18:12,  6.17s/it]
 70%|██████▉   | 156545/225000 [305:16:20<112:40:50,  5.93s/it]
 70%|██████▉   | 156546/225000 [305:16:26<111:25:49,  5.86s/it]
 70%|██████▉   | 156547/225000 [305:16:32<115:20:42,  6.07s/it]
 70%|██████▉   | 156548/225000 [305:16:39<118:57:24,  6.26s/it]
 70%|██████▉   | 156549/225000 [305:16:43<104:55:41,  5.52s/it]
 70%|██████▉   | 156550/225000 [305:16:50<113:56:17,  5.99s/it]
                                                               
{'loss': '0.7753', 'grad_norm': '1.141', 'learning_rate': '1.152e-05', 'epoch': '0.6958', 'num_input_tokens_seen': 505520808, 'train_runtime': '1.099e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156550/225000 [305:16:50<113:56:17,  5.99s/it]
 70%|██████▉   | 156551/225000 [305:16:57<121:28:04,  6.39s/it]
 70%|██████▉   | 156552/225000 [305:17:01<109:03:01,  5.74s/it]
 70%|██████▉   | 156553/225000 [305:17:09<118:25:25,  6.23s/it]
 70%|██████▉   | 156554/225000 [305:17:15<116:51:45,  6.15s/it]
 70%|██████▉   | 156555/225000 [305:17:18<99:00:51,  5.21s/it] 
 70%|██████▉   | 156556/225000 [305:17:22<95:54:26,  5.04s/it]
 70%|██████▉   | 156557/225000 [305:17:28<97:24:16,  5.12s/it]
 70%|██████▉   | 156558/225000 [305:17:33<99:17:20,  5.22s/it]
 70%|██████▉   | 156559/225000 [305:17:38<97:04:38,  5.11s/it]
 70%|██████▉   | 156560/225000 [305:17:43<95:53:53,  5.04s/it]
 70%|██████▉   | 156561/225000 [305:17:48<96:11:07,  5.06s/it]
 70%|██████▉   | 156562/225000 [305:17:53<97:56:44,  5.15s/it]
 70%|██████▉   | 156563/225000 [305:18:06<139:26:35,  7.34s/it]
 70%|██████▉   | 156564/225000 [305:18:11<129:47:42,  6.83s/it]
 70%|██████▉   | 156565/225000 [305:18:17<124:28:49,  6.55s/it]
 70%|██████▉   | 156566/225000 [305:18:24<125:36:47,  6.61s/it]
 70%|██████▉   | 156567/225000 [305:18:28<108:06:28,  5.69s/it]
 70%|██████▉   | 156568/225000 [305:18:37<130:18:35,  6.86s/it]
 70%|██████▉   | 156569/225000 [305:18:42<120:21:21,  6.33s/it]
 70%|██████▉   | 156570/225000 [305:18:47<111:17:09,  5.85s/it]
 70%|██████▉   | 156571/225000 [305:18:52<105:13:25,  5.54s/it]
 70%|██████▉   | 156572/225000 [305:18:56<97:54:44,  5.15s/it] 
 70%|██████▉   | 156573/225000 [305:19:01<95:42:01,  5.03s/it]
 70%|██████▉   | 156574/225000 [305:19:05<93:25:45,  4.92s/it]
 70%|██████▉   | 156575/225000 [305:19:10<92:08:24,  4.85s/it]
 70%|██████▉   | 156576/225000 [305:19:15<91:20:37,  4.81s/it]
 70%|██████▉   | 156577/225000 [305:19:25<121:28:26,  6.39s/it]
 70%|██████▉   | 156578/225000 [305:19:34<137:31:11,  7.24s/it]
 70%|██████▉   | 156579/225000 [305:19:38<120:19:17,  6.33s/it]
 70%|██████▉   | 156580/225000 [305:19:46<127:17:11,  6.70s/it]
 70%|██████▉   | 156581/225000 [305:19:52<126:51:25,  6.67s/it]
 70%|██████▉   | 156582/225000 [305:19:56<111:21:23,  5.86s/it]
 70%|██████▉   | 156583/225000 [305:20:05<124:09:34,  6.53s/it]
 70%|██████▉   | 156584/225000 [305:20:08<105:51:27,  5.57s/it]
 70%|██████▉   | 156585/225000 [305:20:12<98:52:42,  5.20s/it] 
 70%|██████▉   | 156586/225000 [305:20:20<112:10:07,  5.90s/it]
 70%|██████▉   | 156587/225000 [305:20:23<97:44:06,  5.14s/it] 
 70%|██████▉   | 156588/225000 [305:20:32<117:52:22,  6.20s/it]
 70%|██████▉   | 156589/225000 [305:20:35<103:05:34,  5.43s/it]
 70%|██████▉   | 156590/225000 [305:20:43<114:10:27,  6.01s/it]
 70%|██████▉   | 156591/225000 [305:20:46<100:30:50,  5.29s/it]
 70%|██████▉   | 156592/225000 [305:20:54<113:03:09,  5.95s/it]
 70%|██████▉   | 156593/225000 [305:20:58<102:10:18,  5.38s/it]
 70%|██████▉   | 156594/225000 [305:21:02<97:17:30,  5.12s/it] 
 70%|██████▉   | 156595/225000 [305:21:11<119:41:16,  6.30s/it]
 70%|██████▉   | 156596/225000 [305:21:17<114:07:04,  6.01s/it]
 70%|██████▉   | 156597/225000 [305:21:21<105:57:57,  5.58s/it]
 70%|██████▉   | 156598/225000 [305:21:29<119:14:33,  6.28s/it]
 70%|██████▉   | 156599/225000 [305:21:33<104:49:11,  5.52s/it]
 70%|██████▉   | 156600/225000 [305:21:38<102:41:02,  5.40s/it]
                                                               
{'loss': '0.7729', 'grad_norm': '1.069', 'learning_rate': '1.15e-05', 'epoch': '0.696', 'num_input_tokens_seen': 505650576, 'train_runtime': '1.099e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156600/225000 [305:21:38<102:41:02,  5.40s/it]
 70%|██████▉   | 156601/225000 [305:21:47<119:41:20,  6.30s/it]
 70%|██████▉   | 156602/225000 [305:21:51<108:59:14,  5.74s/it]
 70%|██████▉   | 156603/225000 [305:21:56<102:27:42,  5.39s/it]
 70%|██████▉   | 156604/225000 [305:22:00<95:38:17,  5.03s/it] 
 70%|██████▉   | 156605/225000 [305:22:11<132:09:36,  6.96s/it]
 70%|██████▉   | 156606/225000 [305:22:14<110:32:07,  5.82s/it]
 70%|██████▉   | 156607/225000 [305:22:18<98:21:30,  5.18s/it] 
 70%|██████▉   | 156608/225000 [305:22:24<101:13:02,  5.33s/it]
 70%|██████▉   | 156609/225000 [305:22:33<126:07:45,  6.64s/it]
 70%|██████▉   | 156610/225000 [305:22:38<112:33:48,  5.93s/it]
 70%|██████▉   | 156611/225000 [305:22:42<102:56:41,  5.42s/it]
 70%|██████▉   | 156612/225000 [305:22:46<96:03:02,  5.06s/it] 
 70%|██████▉   | 156613/225000 [305:22:56<125:41:12,  6.62s/it]
 70%|██████▉   | 156614/225000 [305:23:01<112:57:49,  5.95s/it]
 70%|██████▉   | 156615/225000 [305:23:08<117:59:35,  6.21s/it]
 70%|██████▉   | 156616/225000 [305:23:11<103:44:33,  5.46s/it]
 70%|██████▉   | 156617/225000 [305:23:15<94:03:45,  4.95s/it] 
 70%|██████▉   | 156618/225000 [305:23:25<121:04:49,  6.37s/it]
 70%|██████▉   | 156619/225000 [305:23:28<104:14:06,  5.49s/it]
 70%|██████▉   | 156620/225000 [305:23:37<120:52:01,  6.36s/it]
 70%|██████▉   | 156621/225000 [305:23:40<103:45:06,  5.46s/it]
 70%|██████▉   | 156622/225000 [305:23:47<113:34:35,  5.98s/it]
 70%|██████▉   | 156623/225000 [305:23:51<100:52:18,  5.31s/it]
 70%|██████▉   | 156624/225000 [305:23:58<109:48:50,  5.78s/it]
 70%|██████▉   | 156625/225000 [305:24:02<100:22:16,  5.28s/it]
 70%|██████▉   | 156626/225000 [305:24:08<104:22:41,  5.50s/it]
 70%|██████▉   | 156627/225000 [305:24:17<125:10:24,  6.59s/it]
 70%|██████▉   | 156628/225000 [305:24:21<110:13:02,  5.80s/it]
 70%|██████▉   | 156629/225000 [305:24:26<103:25:25,  5.45s/it]
 70%|██████▉   | 156630/225000 [305:24:30<97:29:33,  5.13s/it] 
 70%|██████▉   | 156631/225000 [305:24:35<94:20:59,  4.97s/it]
 70%|██████▉   | 156632/225000 [305:24:39<92:30:53,  4.87s/it]
 70%|██████▉   | 156633/225000 [305:24:44<91:23:25,  4.81s/it]
 70%|██████▉   | 156634/225000 [305:24:56<131:29:36,  6.92s/it]
 70%|██████▉   | 156635/225000 [305:25:00<115:46:59,  6.10s/it]
 70%|██████▉   | 156636/225000 [305:25:04<102:24:53,  5.39s/it]
 70%|██████▉   | 156637/225000 [305:25:13<123:15:15,  6.49s/it]
 70%|██████▉   | 156638/225000 [305:25:17<112:24:59,  5.92s/it]
 70%|██████▉   | 156639/225000 [305:25:22<105:45:40,  5.57s/it]
 70%|██████▉   | 156640/225000 [305:25:30<118:26:19,  6.24s/it]
 70%|██████▉   | 156641/225000 [305:25:34<107:49:51,  5.68s/it]
 70%|██████▉   | 156642/225000 [305:25:39<100:18:11,  5.28s/it]
 70%|██████▉   | 156643/225000 [305:25:43<95:57:04,  5.05s/it] 
 70%|██████▉   | 156644/225000 [305:25:48<93:49:49,  4.94s/it]
 70%|██████▉   | 156645/225000 [305:25:55<105:13:57,  5.54s/it]
 70%|██████▉   | 156646/225000 [305:26:05<129:33:07,  6.82s/it]
 70%|██████▉   | 156647/225000 [305:26:10<120:42:42,  6.36s/it]
 70%|██████▉   | 156648/225000 [305:26:16<118:46:25,  6.26s/it]
 70%|██████▉   | 156649/225000 [305:26:19<102:34:27,  5.40s/it]
 70%|██████▉   | 156650/225000 [305:26:29<127:04:09,  6.69s/it]
                                                               
{'loss': '0.7808', 'grad_norm': '1.022', 'learning_rate': '1.149e-05', 'epoch': '0.6962', 'num_input_tokens_seen': 505779096, 'train_runtime': '1.1e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156650/225000 [305:26:29<127:04:09,  6.69s/it]
 70%|██████▉   | 156651/225000 [305:26:34<116:21:22,  6.13s/it]
 70%|██████▉   | 156652/225000 [305:26:42<127:44:41,  6.73s/it]
 70%|██████▉   | 156653/225000 [305:26:45<108:27:00,  5.71s/it]
 70%|██████▉   | 156654/225000 [305:26:51<110:05:27,  5.80s/it]
 70%|██████▉   | 156655/225000 [305:26:59<120:11:04,  6.33s/it]
 70%|██████▉   | 156656/225000 [305:27:03<110:02:20,  5.80s/it]
 70%|██████▉   | 156657/225000 [305:27:08<102:57:14,  5.42s/it]
 70%|██████▉   | 156658/225000 [305:27:12<97:25:45,  5.13s/it] 
 70%|██████▉   | 156659/225000 [305:27:17<94:09:53,  4.96s/it]
 70%|██████▉   | 156660/225000 [305:27:22<92:15:51,  4.86s/it]
 70%|██████▉   | 156661/225000 [305:27:26<89:41:08,  4.72s/it]
 70%|██████▉   | 156662/225000 [305:27:38<130:27:51,  6.87s/it]
 70%|██████▉   | 156663/225000 [305:27:45<133:00:27,  7.01s/it]
 70%|██████▉   | 156664/225000 [305:27:49<115:13:52,  6.07s/it]
 70%|██████▉   | 156665/225000 [305:27:56<119:57:54,  6.32s/it]
 70%|██████▉   | 156666/225000 [305:28:02<117:01:17,  6.16s/it]
 70%|██████▉   | 156667/225000 [305:28:06<103:25:50,  5.45s/it]
 70%|██████▉   | 156668/225000 [305:28:11<105:55:55,  5.58s/it]
 70%|██████▉   | 156669/225000 [305:28:18<109:21:58,  5.76s/it]
 70%|██████▉   | 156670/225000 [305:28:25<119:31:09,  6.30s/it]
 70%|██████▉   | 156671/225000 [305:28:29<103:34:18,  5.46s/it]
 70%|██████▉   | 156672/225000 [305:28:36<112:15:47,  5.91s/it]
 70%|██████▉   | 156673/225000 [305:28:43<121:17:12,  6.39s/it]
 70%|██████▉   | 156674/225000 [305:28:47<104:07:26,  5.49s/it]
 70%|██████▉   | 156675/225000 [305:28:51<99:18:25,  5.23s/it] 
 70%|██████▉   | 156676/225000 [305:28:56<97:19:07,  5.13s/it]
 70%|██████▉   | 156677/225000 [305:29:02<99:48:59,  5.26s/it]
 70%|██████▉   | 156678/225000 [305:29:07<99:21:06,  5.24s/it]
 70%|██████▉   | 156679/225000 [305:29:12<97:52:23,  5.16s/it]
 70%|██████▉   | 156680/225000 [305:29:17<98:21:51,  5.18s/it]
 70%|██████▉   | 156681/225000 [305:29:22<97:35:09,  5.14s/it]
 70%|██████▉   | 156682/225000 [305:29:27<98:01:51,  5.17s/it]
 70%|██████▉   | 156683/225000 [305:29:40<141:21:22,  7.45s/it]
 70%|██████▉   | 156684/225000 [305:29:47<136:49:01,  7.21s/it]
 70%|██████▉   | 156685/225000 [305:29:52<126:06:21,  6.65s/it]
 70%|██████▉   | 156686/225000 [305:29:57<117:04:51,  6.17s/it]
 70%|██████▉   | 156687/225000 [305:30:02<112:26:37,  5.93s/it]
 70%|██████▉   | 156688/225000 [305:30:08<109:13:14,  5.76s/it]
 70%|██████▉   | 156689/225000 [305:30:13<106:35:11,  5.62s/it]
 70%|██████▉   | 156690/225000 [305:30:19<106:30:05,  5.61s/it]
 70%|██████▉   | 156691/225000 [305:30:25<110:04:59,  5.80s/it]
 70%|██████▉   | 156692/225000 [305:30:36<138:08:39,  7.28s/it]
 70%|██████▉   | 156693/225000 [305:30:39<116:18:47,  6.13s/it]
 70%|██████▉   | 156694/225000 [305:30:48<133:33:00,  7.04s/it]
 70%|██████▉   | 156695/225000 [305:30:53<118:38:17,  6.25s/it]
 70%|██████▉   | 156696/225000 [305:30:57<109:48:36,  5.79s/it]
 70%|██████▉   | 156697/225000 [305:31:02<102:08:22,  5.38s/it]
 70%|██████▉   | 156698/225000 [305:31:06<96:37:39,  5.09s/it] 
 70%|██████▉   | 156699/225000 [305:31:11<92:56:23,  4.90s/it]
 70%|██████▉   | 156700/225000 [305:31:21<125:22:09,  6.61s/it]
                                                               
{'loss': '0.7796', 'grad_norm': '1.172', 'learning_rate': '1.147e-05', 'epoch': '0.6964', 'num_input_tokens_seen': 505909400, 'train_runtime': '1.1e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156700/225000 [305:31:21<125:22:09,  6.61s/it]
 70%|██████▉   | 156701/225000 [305:31:29<129:25:40,  6.82s/it]
 70%|██████▉   | 156702/225000 [305:31:32<109:39:48,  5.78s/it]
 70%|██████▉   | 156703/225000 [305:31:39<117:34:29,  6.20s/it]
 70%|██████▉   | 156704/225000 [305:31:43<104:02:12,  5.48s/it]
 70%|██████▉   | 156705/225000 [305:31:52<121:43:53,  6.42s/it]
 70%|██████▉   | 156706/225000 [305:31:56<108:17:33,  5.71s/it]
 70%|██████▉   | 156707/225000 [305:32:00<98:36:16,  5.20s/it] 
 70%|██████▉   | 156708/225000 [305:32:04<92:38:48,  4.88s/it]
 70%|██████▉   | 156709/225000 [305:32:15<127:42:23,  6.73s/it]
 70%|██████▉   | 156710/225000 [305:32:18<108:44:33,  5.73s/it]
 70%|██████▉   | 156711/225000 [305:32:22<97:31:53,  5.14s/it] 
 70%|██████▉   | 156712/225000 [305:32:34<137:27:30,  7.25s/it]
 70%|██████▉   | 156713/225000 [305:32:37<115:07:30,  6.07s/it]
 70%|██████▉   | 156714/225000 [305:32:41<101:09:54,  5.33s/it]
 70%|██████▉   | 156715/225000 [305:32:45<93:21:10,  4.92s/it] 
 70%|██████▉   | 156716/225000 [305:32:57<132:58:37,  7.01s/it]
 70%|██████▉   | 156717/225000 [305:33:01<116:26:02,  6.14s/it]
 70%|██████▉   | 156718/225000 [305:33:05<105:50:55,  5.58s/it]
 70%|██████▉   | 156719/225000 [305:33:09<96:08:06,  5.07s/it] 
 70%|██████▉   | 156720/225000 [305:33:13<91:36:17,  4.83s/it]
 70%|██████▉   | 156721/225000 [305:33:25<129:43:00,  6.84s/it]
 70%|██████▉   | 156722/225000 [305:33:30<118:53:27,  6.27s/it]
 70%|██████▉   | 156723/225000 [305:33:34<105:01:30,  5.54s/it]
 70%|██████▉   | 156724/225000 [305:33:43<123:56:57,  6.54s/it]
 70%|██████▉   | 156725/225000 [305:33:48<117:19:55,  6.19s/it]
 70%|██████▉   | 156726/225000 [305:33:52<102:21:01,  5.40s/it]
 70%|██████▉   | 156727/225000 [305:34:02<130:50:19,  6.90s/it]
 70%|██████▉   | 156728/225000 [305:34:05<110:51:36,  5.85s/it]
 70%|██████▉   | 156729/225000 [305:34:09<98:20:29,  5.19s/it] 
 70%|██████▉   | 156730/225000 [305:34:14<99:41:08,  5.26s/it]
 70%|██████▉   | 156731/225000 [305:34:24<122:04:58,  6.44s/it]
 70%|██████▉   | 156732/225000 [305:34:28<107:41:44,  5.68s/it]
 70%|██████▉   | 156733/225000 [305:34:34<111:29:53,  5.88s/it]
 70%|██████▉   | 156734/225000 [305:34:37<95:45:22,  5.05s/it] 
 70%|██████▉   | 156735/225000 [305:34:41<91:20:45,  4.82s/it]
 70%|██████▉   | 156736/225000 [305:34:46<91:57:07,  4.85s/it]
 70%|██████▉   | 156737/225000 [305:34:51<92:17:49,  4.87s/it]
 70%|██████▉   | 156738/225000 [305:34:56<92:05:03,  4.86s/it]
 70%|██████▉   | 156739/225000 [305:35:00<90:30:17,  4.77s/it]
 70%|██████▉   | 156740/225000 [305:35:13<131:55:13,  6.96s/it]
 70%|██████▉   | 156741/225000 [305:35:19<126:23:03,  6.67s/it]
 70%|██████▉   | 156742/225000 [305:35:26<129:41:32,  6.84s/it]
 70%|██████▉   | 156743/225000 [305:35:31<118:42:50,  6.26s/it]
 70%|██████▉   | 156744/225000 [305:35:34<104:37:34,  5.52s/it]
 70%|██████▉   | 156745/225000 [305:35:45<131:08:25,  6.92s/it]
 70%|██████▉   | 156746/225000 [305:35:49<116:02:51,  6.12s/it]
 70%|██████▉   | 156747/225000 [305:35:53<105:19:19,  5.56s/it]
 70%|██████▉   | 156748/225000 [305:35:57<98:26:02,  5.19s/it] 
 70%|██████▉   | 156749/225000 [305:36:02<94:09:56,  4.97s/it]
 70%|██████▉   | 156750/225000 [305:36:07<93:47:26,  4.95s/it]
                                                              
{'loss': '0.7894', 'grad_norm': '1.24', 'learning_rate': '1.146e-05', 'epoch': '0.6967', 'num_input_tokens_seen': 506038672, 'train_runtime': '1.1e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156750/225000 [305:36:07<93:47:26,  4.95s/it]
 70%|██████▉   | 156751/225000 [305:36:18<130:41:25,  6.89s/it]
 70%|██████▉   | 156752/225000 [305:36:24<124:02:15,  6.54s/it]
 70%|██████▉   | 156753/225000 [305:36:29<117:24:42,  6.19s/it]
 70%|██████▉   | 156754/225000 [305:36:34<110:15:15,  5.82s/it]
 70%|██████▉   | 156755/225000 [305:36:39<104:26:56,  5.51s/it]
 70%|██████▉   | 156756/225000 [305:36:45<103:58:23,  5.48s/it]
 70%|██████▉   | 156757/225000 [305:36:50<101:59:14,  5.38s/it]
 70%|██████▉   | 156758/225000 [305:36:55<102:39:08,  5.42s/it]
 70%|██████▉   | 156759/225000 [305:37:00<102:05:07,  5.39s/it]
 70%|██████▉   | 156760/225000 [305:37:06<102:40:40,  5.42s/it]
 70%|██████▉   | 156761/225000 [305:37:11<102:24:45,  5.40s/it]
 70%|██████▉   | 156762/225000 [305:37:16<99:52:03,  5.27s/it] 
 70%|██████▉   | 156763/225000 [305:37:22<100:25:54,  5.30s/it]
 70%|██████▉   | 156764/225000 [305:37:27<101:18:25,  5.34s/it]
 70%|██████▉   | 156765/225000 [305:37:39<136:45:39,  7.22s/it]
 70%|██████▉   | 156766/225000 [305:37:46<137:28:57,  7.25s/it]
 70%|██████▉   | 156767/225000 [305:37:50<118:10:15,  6.23s/it]
 70%|██████▉   | 156768/225000 [305:37:58<128:09:36,  6.76s/it]
 70%|██████▉   | 156769/225000 [305:38:02<112:06:30,  5.92s/it]
 70%|██████▉   | 156770/225000 [305:38:06<100:43:02,  5.31s/it]
 70%|██████▉   | 156771/225000 [305:38:16<128:06:54,  6.76s/it]
 70%|██████▉   | 156772/225000 [305:38:19<109:38:02,  5.78s/it]
 70%|██████▉   | 156773/225000 [305:38:23<97:28:04,  5.14s/it] 
 70%|██████▉   | 156774/225000 [305:38:33<123:57:33,  6.54s/it]
 70%|██████▉   | 156775/225000 [305:38:40<124:55:11,  6.59s/it]
 70%|██████▉   | 156776/225000 [305:38:43<107:14:36,  5.66s/it]
 70%|██████▉   | 156777/225000 [305:38:51<119:37:35,  6.31s/it]
 70%|██████▉   | 156778/225000 [305:38:54<103:46:45,  5.48s/it]
 70%|██████▉   | 156779/225000 [305:39:03<123:56:22,  6.54s/it]
 70%|██████▉   | 156780/225000 [305:39:07<106:43:16,  5.63s/it]
 70%|██████▉   | 156781/225000 [305:39:10<94:51:27,  5.01s/it] 
 70%|██████▉   | 156782/225000 [305:39:16<96:33:29,  5.10s/it]
 70%|██████▉   | 156783/225000 [305:39:25<118:53:35,  6.27s/it]
 70%|██████▉   | 156784/225000 [305:39:28<101:56:51,  5.38s/it]
 70%|██████▉   | 156785/225000 [305:39:36<119:04:40,  6.28s/it]
 70%|██████▉   | 156786/225000 [305:39:41<109:02:05,  5.75s/it]
 70%|██████▉   | 156787/225000 [305:39:51<130:49:15,  6.90s/it]
 70%|██████▉   | 156788/225000 [305:39:54<112:04:53,  5.92s/it]
 70%|██████▉   | 156789/225000 [305:39:59<106:58:04,  5.65s/it]
 70%|██████▉   | 156790/225000 [305:40:05<106:00:16,  5.59s/it]
 70%|██████▉   | 156791/225000 [305:40:13<123:06:17,  6.50s/it]
 70%|██████▉   | 156792/225000 [305:40:17<105:14:51,  5.55s/it]
 70%|██████▉   | 156793/225000 [305:40:25<122:52:28,  6.49s/it]
 70%|██████▉   | 156794/225000 [305:40:30<109:56:06,  5.80s/it]
 70%|██████▉   | 156795/225000 [305:40:34<101:12:11,  5.34s/it]
 70%|██████▉   | 156796/225000 [305:40:39<101:20:11,  5.35s/it]
 70%|██████▉   | 156797/225000 [305:40:50<131:18:31,  6.93s/it]
 70%|██████▉   | 156798/225000 [305:40:54<117:19:09,  6.19s/it]
 70%|██████▉   | 156799/225000 [305:40:59<110:15:33,  5.82s/it]
 70%|██████▉   | 156800/225000 [305:41:04<104:07:41,  5.50s/it]
                                                               
{'loss': '0.7696', 'grad_norm': '1.14', 'learning_rate': '1.144e-05', 'epoch': '0.6969', 'num_input_tokens_seen': 506169432, 'train_runtime': '1.1e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156800/225000 [305:41:04<104:07:41,  5.50s/it]
 70%|██████▉   | 156801/225000 [305:41:08<98:32:31,  5.20s/it] 
 70%|██████▉   | 156802/225000 [305:41:13<96:31:35,  5.10s/it]
 70%|██████▉   | 156803/225000 [305:41:18<93:05:58,  4.91s/it]
 70%|██████▉   | 156804/225000 [305:41:24<100:58:44,  5.33s/it]
 70%|██████▉   | 156805/225000 [305:41:37<144:48:45,  7.64s/it]
 70%|██████▉   | 156806/225000 [305:41:41<124:50:47,  6.59s/it]
 70%|██████▉   | 156807/225000 [305:41:49<133:25:05,  7.04s/it]
 70%|██████▉   | 156808/225000 [305:41:55<122:57:10,  6.49s/it]
 70%|██████▉   | 156809/225000 [305:42:02<126:49:42,  6.70s/it]
 70%|██████▉   | 156810/225000 [305:42:06<113:01:29,  5.97s/it]
 70%|██████▉   | 156811/225000 [305:42:10<103:53:58,  5.49s/it]
 70%|██████▉   | 156812/225000 [305:42:15<98:20:23,  5.19s/it] 
 70%|██████▉   | 156813/225000 [305:42:23<112:28:06,  5.94s/it]
 70%|██████▉   | 156814/225000 [305:42:32<134:25:03,  7.10s/it]
 70%|██████▉   | 156815/225000 [305:42:39<129:30:48,  6.84s/it]
 70%|██████▉   | 156816/225000 [305:42:42<112:09:00,  5.92s/it]
 70%|██████▉   | 156817/225000 [305:42:47<105:55:37,  5.59s/it]
 70%|██████▉   | 156818/225000 [305:42:56<124:13:48,  6.56s/it]
 70%|██████▉   | 156819/225000 [305:42:59<106:06:33,  5.60s/it]
 70%|██████▉   | 156820/225000 [305:43:08<124:53:36,  6.59s/it]
 70%|██████▉   | 156821/225000 [305:43:13<112:37:35,  5.95s/it]
 70%|██████▉   | 156822/225000 [305:43:20<120:21:34,  6.36s/it]
 70%|██████▉   | 156823/225000 [305:43:25<112:04:38,  5.92s/it]
 70%|██████▉   | 156824/225000 [305:43:31<114:18:57,  6.04s/it]
 70%|██████▉   | 156825/225000 [305:43:37<113:05:00,  5.97s/it]
 70%|██████▉   | 156826/225000 [305:43:41<100:01:45,  5.28s/it]
 70%|██████▉   | 156827/225000 [305:43:51<129:09:53,  6.82s/it]
 70%|██████▉   | 156828/225000 [305:43:56<116:33:57,  6.16s/it]
 70%|██████▉   | 156829/225000 [305:44:00<107:47:13,  5.69s/it]
 70%|██████▉   | 156830/225000 [305:44:05<103:14:09,  5.45s/it]
 70%|██████▉   | 156831/225000 [305:44:10<99:07:39,  5.23s/it] 
 70%|██████▉   | 156832/225000 [305:44:14<94:10:31,  4.97s/it]
 70%|██████▉   | 156833/225000 [305:44:21<101:09:52,  5.34s/it]
 70%|██████▉   | 156834/225000 [305:44:33<140:48:37,  7.44s/it]
 70%|██████▉   | 156835/225000 [305:44:36<118:40:32,  6.27s/it]
 70%|██████▉   | 156836/225000 [305:44:44<124:56:13,  6.60s/it]
 70%|██████▉   | 156837/225000 [305:44:47<107:44:47,  5.69s/it]
 70%|██████▉   | 156838/225000 [305:44:53<106:13:09,  5.61s/it]
 70%|██████▉   | 156839/225000 [305:45:03<132:12:07,  6.98s/it]
 70%|██████▉   | 156840/225000 [305:45:08<119:58:05,  6.34s/it]
 70%|██████▉   | 156841/225000 [305:45:13<111:32:35,  5.89s/it]
 70%|██████▉   | 156842/225000 [305:45:18<107:01:51,  5.65s/it]
 70%|██████▉   | 156843/225000 [305:45:23<104:52:13,  5.54s/it]
 70%|██████▉   | 156844/225000 [305:45:28<99:55:14,  5.28s/it] 
 70%|██████▉   | 156845/225000 [305:45:33<99:45:57,  5.27s/it]
 70%|██████▉   | 156846/225000 [305:45:38<98:19:52,  5.19s/it]
 70%|██████▉   | 156847/225000 [305:45:43<95:41:13,  5.05s/it]
 70%|██████▉   | 156848/225000 [305:45:48<96:14:11,  5.08s/it]
 70%|██████▉   | 156849/225000 [305:45:53<95:52:06,  5.06s/it]
 70%|██████▉   | 156850/225000 [305:45:58<96:44:00,  5.11s/it]
                                                              
{'loss': '0.7776', 'grad_norm': '1.277', 'learning_rate': '1.143e-05', 'epoch': '0.6971', 'num_input_tokens_seen': 506300784, 'train_runtime': '1.101e+06', 'train_tokens_per_second': '460'}

 70%|██████▉   | 156850/225000 [305:45:58<96:44:00,  5.11s/it]
 70%|██████▉   | 156851/225000 [305:46:10<137:51:13,  7.28s/it]
 70%|██████▉   | 156852/225000 [305:46:15<124:36:34,  6.58s/it]
 70%|██████▉   | 156853/225000 [305:46:19<108:48:01,  5.75s/it]
 70%|██████▉   | 156854/225000 [305:46:24<101:01:57,  5.34s/it]
 70%|██████▉   | 156855/225000 [305:46:34<131:55:50,  6.97s/it]
 70%|██████▉   | 156856/225000 [305:46:38<111:45:24,  5.90s/it]
 70%|██████▉   | 156857/225000 [305:46:46<127:05:02,  6.71s/it]
 70%|██████▉   | 156858/225000 [305:46:50<111:05:04,  5.87s/it]
 70%|██████▉   | 156859/225000 [305:46:58<120:35:15,  6.37s/it]
 70%|██████▉   | 156860/225000 [305:47:02<107:23:02,  5.67s/it]
 70%|██████▉   | 156861/225000 [305:47:10<121:33:07,  6.42s/it]
 70%|██████▉   | 156862/225000 [305:47:14<110:25:01,  5.83s/it]
 70%|██████▉   | 156863/225000 [305:47:22<119:45:45,  6.33s/it]
 70%|██████▉   | 156864/225000 [305:47:25<103:25:34,  5.46s/it]
 70%|██████▉   | 156865/225000 [305:47:32<111:47:39,  5.91s/it]
 70%|██████▉   | 156866/225000 [305:47:38<109:57:42,  5.81s/it]
 70%|██████▉   | 156867/225000 [305:47:45<116:31:11,  6.16s/it]
 70%|██████▉   | 156868/225000 [305:47:52<119:34:35,  6.32s/it]
 70%|██████▉   | 156869/225000 [305:47:56<111:24:48,  5.89s/it]
 70%|██████▉   | 156870/225000 [305:48:00<99:01:49,  5.23s/it] 
 70%|██████▉   | 156871/225000 [305:48:07<110:47:14,  5.85s/it]
 70%|██████▉   | 156872/225000 [305:48:13<110:16:03,  5.83s/it]
 70%|██████▉   | 156873/225000 [305:48:21<120:19:26,  6.36s/it]
 70%|██████▉   | 156874/225000 [305:48:26<114:37:13,  6.06s/it]
 70%|██████▉   | 156875/225000 [305:48:31<106:11:05,  5.61s/it]
 70%|██████▉   | 156876/225000 [305:48:35<98:28:56,  5.20s/it] 
 70%|██████▉   | 156877/225000 [305:48:45<125:14:17,  6.62s/it]
 70%|██████▉   | 156878/225000 [305:48:50<117:31:25,  6.21s/it]
 70%|██████▉   | 156879/225000 [305:48:53<100:57:34,  5.34s/it]
 70%|██████▉   | 156880/225000 [305:49:03<124:38:31,  6.59s/it]
 70%|██████▉   | 156881/225000 [305:49:06<107:10:57,  5.66s/it]
 70%|██████▉   | 156882/225000 [305:49:13<111:48:01,  5.91s/it]
 70%|██████▉   | 156883/225000 [305:49:18<107:14:32,  5.67s/it]
 70%|██████▉   | 156884/225000 [305:49:26<122:28:58,  6.47s/it]
 70%|██████▉   | 156885/225000 [305:49:30<104:30:24,  5.52s/it]
 70%|██████▉   | 156886/225000 [305:49:38<118:05:42,  6.24s/it]
 70%|██████▉   | 156887/225000 [305:49:41<103:30:55,  5.47s/it]
 70%|██████▉   | 156888/225000 [305:49:50<118:56:24,  6.29s/it]
 70%|██████▉   | 156889/225000 [305:49:56<119:21:33,  6.31s/it]
 70%|██████▉   | 156890/225000 [305:50:01<110:28:08,  5.84s/it]
 70%|██████▉   | 156891/225000 [305:50:05<103:43:38,  5.48s/it]
 70%|██████▉   | 156892/225000 [305:50:10<99:01:34,  5.23s/it] 
 70%|██████▉   | 156893/225000 [305:50:15<98:56:05,  5.23s/it]
 70%|██████▉   | 156894/225000 [305:50:20<97:19:56,  5.14s/it]
 70%|██████▉   | 156895/225000 [305:50:25<97:02:17,  5.13s/it]
 70%|██████▉   | 156896/225000 [305:50:30<95:06:49,  5.03s/it]
 70%|██████▉   | 156897/225000 [305:50:35<94:58:33,  5.02s/it]
 70%|██████▉   | 156898/225000 [305:50:48<138:03:10,  7.30s/it]
 70%|██████▉   | 156899/225000 [305:50:52<124:20:02,  6.57s/it]
 70%|██████▉   | 156900/225000 [305:50:58<118:43:51,  6.28s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '0.9675', 'learning_rate': '1.141e-05', 'epoch': '0.6973', 'num_input_tokens_seen': 506430936, 'train_runtime': '1.101e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 156900/225000 [305:50:58<118:43:51,  6.28s/it]
 70%|██████▉   | 156901/225000 [305:51:05<120:31:36,  6.37s/it]
 70%|██████▉   | 156902/225000 [305:51:09<109:31:58,  5.79s/it]
 70%|██████▉   | 156903/225000 [305:51:13<101:36:23,  5.37s/it]
 70%|██████▉   | 156904/225000 [305:51:24<130:58:14,  6.92s/it]
 70%|██████▉   | 156905/225000 [305:51:28<116:11:45,  6.14s/it]
 70%|██████▉   | 156906/225000 [305:51:33<109:59:39,  5.82s/it]
 70%|██████▉   | 156907/225000 [305:51:39<108:00:14,  5.71s/it]
 70%|██████▉   | 156908/225000 [305:51:44<106:32:23,  5.63s/it]
 70%|██████▉   | 156909/225000 [305:51:50<104:00:57,  5.50s/it]
 70%|██████▉   | 156910/225000 [305:51:55<101:21:34,  5.36s/it]
 70%|██████▉   | 156911/225000 [305:51:59<98:44:24,  5.22s/it] 
 70%|██████▉   | 156912/225000 [305:52:04<97:20:53,  5.15s/it]
 70%|██████▉   | 156913/225000 [305:52:10<97:51:18,  5.17s/it]
 70%|██████▉   | 156914/225000 [305:52:15<100:02:26,  5.29s/it]
 70%|██████▉   | 156915/225000 [305:52:20<98:52:27,  5.23s/it] 
 70%|██████▉   | 156916/225000 [305:52:26<99:18:52,  5.25s/it]
 70%|██████▉   | 156917/225000 [305:52:39<143:13:57,  7.57s/it]
 70%|██████▉   | 156918/225000 [305:52:44<130:17:46,  6.89s/it]
 70%|██████▉   | 156919/225000 [305:52:52<135:26:20,  7.16s/it]
 70%|██████▉   | 156920/225000 [305:52:56<117:26:04,  6.21s/it]
 70%|██████▉   | 156921/225000 [305:53:00<104:24:11,  5.52s/it]
 70%|██████▉   | 156922/225000 [305:53:09<126:45:28,  6.70s/it]
 70%|██████▉   | 156923/225000 [305:53:13<112:36:41,  5.96s/it]
 70%|██████▉   | 156924/225000 [305:53:18<105:34:31,  5.58s/it]
 70%|██████▉   | 156925/225000 [305:53:22<98:33:51,  5.21s/it] 
 70%|██████▉   | 156926/225000 [305:53:27<95:48:37,  5.07s/it]
 70%|██████▉   | 156927/225000 [305:53:31<91:44:15,  4.85s/it]
 70%|██████▉   | 156928/225000 [305:53:40<111:54:23,  5.92s/it]
 70%|██████▉   | 156929/225000 [305:53:49<132:29:00,  7.01s/it]
 70%|██████▉   | 156930/225000 [305:53:53<113:34:08,  6.01s/it]
 70%|██████▉   | 156931/225000 [305:53:58<105:10:16,  5.56s/it]
 70%|██████▉   | 156932/225000 [305:54:04<112:20:06,  5.94s/it]
 70%|██████▉   | 156933/225000 [305:54:08<99:25:42,  5.26s/it] 
 70%|██████▉   | 156934/225000 [305:54:17<123:07:10,  6.51s/it]
 70%|██████▉   | 156935/225000 [305:54:22<109:51:44,  5.81s/it]
 70%|██████▉   | 156936/225000 [305:54:26<102:30:29,  5.42s/it]
 70%|██████▉   | 156937/225000 [305:54:31<99:16:28,  5.25s/it] 
 70%|██████▉   | 156938/225000 [305:54:36<98:10:18,  5.19s/it]
 70%|██████▉   | 156939/225000 [305:54:41<97:32:41,  5.16s/it]
 70%|██████▉   | 156940/225000 [305:54:46<96:39:11,  5.11s/it]
 70%|██████▉   | 156941/225000 [305:54:51<97:34:37,  5.16s/it]
 70%|██████▉   | 156942/225000 [305:54:56<96:29:25,  5.10s/it]
 70%|██████▉   | 156943/225000 [305:55:02<97:31:10,  5.16s/it]
 70%|██████▉   | 156944/225000 [305:55:07<97:11:30,  5.14s/it]
 70%|██████▉   | 156945/225000 [305:55:14<107:03:22,  5.66s/it]
 70%|██████▉   | 156946/225000 [305:55:26<145:44:27,  7.71s/it]
 70%|██████▉   | 156947/225000 [305:55:32<136:12:33,  7.21s/it]
 70%|██████▉   | 156948/225000 [305:55:40<140:59:47,  7.46s/it]
 70%|██████▉   | 156949/225000 [305:55:44<120:23:26,  6.37s/it]
 70%|██████▉   | 156950/225000 [305:55:49<113:13:08,  5.99s/it]
                                                               
{'loss': '0.7685', 'grad_norm': '1.076', 'learning_rate': '1.14e-05', 'epoch': '0.6976', 'num_input_tokens_seen': 506560968, 'train_runtime': '1.101e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 156950/225000 [305:55:49<113:13:08,  5.99s/it]
 70%|██████▉   | 156951/225000 [305:55:58<127:42:03,  6.76s/it]
 70%|██████▉   | 156952/225000 [305:56:03<120:18:02,  6.36s/it]
 70%|██████▉   | 156953/225000 [305:56:08<109:27:09,  5.79s/it]
 70%|██████▉   | 156954/225000 [305:56:12<101:50:25,  5.39s/it]
 70%|██████▉   | 156955/225000 [305:56:16<96:16:14,  5.09s/it] 
 70%|██████▉   | 156956/225000 [305:56:21<94:23:54,  4.99s/it]
 70%|██████▉   | 156957/225000 [305:56:34<135:48:22,  7.19s/it]
 70%|██████▉   | 156958/225000 [305:56:37<116:07:18,  6.14s/it]
 70%|██████▉   | 156959/225000 [305:56:42<111:02:35,  5.88s/it]
 70%|██████▉   | 156960/225000 [305:56:50<120:06:48,  6.36s/it]
 70%|██████▉   | 156961/225000 [305:56:54<108:00:23,  5.71s/it]
 70%|██████▉   | 156962/225000 [305:56:58<98:34:01,  5.22s/it] 
 70%|██████▉   | 156963/225000 [305:57:08<122:53:25,  6.50s/it]
 70%|██████▉   | 156964/225000 [305:57:13<114:04:02,  6.04s/it]
 70%|██████▉   | 156965/225000 [305:57:16<100:33:50,  5.32s/it]
 70%|██████▉   | 156966/225000 [305:57:20<91:04:28,  4.82s/it] 
 70%|██████▉   | 156967/225000 [305:57:28<110:01:29,  5.82s/it]
 70%|██████▉   | 156968/225000 [305:57:36<120:52:00,  6.40s/it]
 70%|██████▉   | 156969/225000 [305:57:41<114:48:17,  6.08s/it]
 70%|██████▉   | 156970/225000 [305:57:47<110:25:27,  5.84s/it]
 70%|██████▉   | 156971/225000 [305:57:52<106:28:34,  5.63s/it]
 70%|██████▉   | 156972/225000 [305:57:57<103:31:44,  5.48s/it]
 70%|██████▉   | 156973/225000 [305:58:02<102:18:05,  5.41s/it]
 70%|██████▉   | 156974/225000 [305:58:07<99:13:02,  5.25s/it] 
 70%|██████▉   | 156975/225000 [305:58:12<97:55:18,  5.18s/it]
 70%|██████▉   | 156976/225000 [305:58:20<112:10:48,  5.94s/it]
 70%|██████▉   | 156977/225000 [305:58:29<132:41:06,  7.02s/it]
 70%|██████▉   | 156978/225000 [305:58:33<113:01:19,  5.98s/it]
 70%|██████▉   | 156979/225000 [305:58:37<105:10:55,  5.57s/it]
 70%|██████▉   | 156980/225000 [305:58:47<126:21:10,  6.69s/it]
 70%|██████▉   | 156981/225000 [305:58:50<105:45:10,  5.60s/it]
 70%|██████▉   | 156982/225000 [305:58:58<119:24:04,  6.32s/it]
 70%|██████▉   | 156983/225000 [305:59:04<117:45:29,  6.23s/it]
 70%|██████▉   | 156984/225000 [305:59:07<101:12:42,  5.36s/it]
 70%|██████▉   | 156985/225000 [305:59:17<126:33:45,  6.70s/it]
 70%|██████▉   | 156986/225000 [305:59:20<106:08:54,  5.62s/it]
 70%|██████▉   | 156987/225000 [305:59:24<99:10:02,  5.25s/it] 
 70%|██████▉   | 156988/225000 [305:59:33<118:56:54,  6.30s/it]
 70%|██████▉   | 156989/225000 [305:59:37<102:42:31,  5.44s/it]
 70%|██████▉   | 156990/225000 [305:59:45<119:29:07,  6.32s/it]
 70%|██████▉   | 156991/225000 [305:59:51<118:59:58,  6.30s/it]
 70%|██████▉   | 156992/225000 [305:59:55<107:00:04,  5.66s/it]
 70%|██████▉   | 156993/225000 [306:00:03<119:47:51,  6.34s/it]
 70%|██████▉   | 156994/225000 [306:00:07<105:51:24,  5.60s/it]
 70%|██████▉   | 156995/225000 [306:00:11<98:05:22,  5.19s/it] 
 70%|██████▉   | 156996/225000 [306:00:15<91:31:21,  4.85s/it]
 70%|██████▉   | 156997/225000 [306:00:26<125:16:20,  6.63s/it]
 70%|██████▉   | 156998/225000 [306:00:30<111:44:14,  5.92s/it]
 70%|██████▉   | 156999/225000 [306:00:38<123:43:19,  6.55s/it]
 70%|██████▉   | 157000/225000 [306:00:43<110:51:00,  5.87s/it]
                                                               
{'loss': '0.7811', 'grad_norm': '1.056', 'learning_rate': '1.138e-05', 'epoch': '0.6978', 'num_input_tokens_seen': 506689800, 'train_runtime': '1.102e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157000/225000 [306:00:43<110:51:00,  5.87s/it]
 70%|██████▉   | 157001/225000 [306:00:48<106:14:47,  5.62s/it]
 70%|██████▉   | 157002/225000 [306:00:53<103:07:19,  5.46s/it]
 70%|██████▉   | 157003/225000 [306:00:58<99:50:11,  5.29s/it] 
 70%|██████▉   | 157004/225000 [306:01:03<98:07:11,  5.19s/it]
 70%|██████▉   | 157005/225000 [306:01:08<97:33:34,  5.17s/it]
 70%|██████▉   | 157006/225000 [306:01:13<98:55:54,  5.24s/it]
 70%|██████▉   | 157007/225000 [306:01:18<95:42:52,  5.07s/it]
 70%|██████▉   | 157008/225000 [306:01:24<102:27:43,  5.43s/it]
 70%|██████▉   | 157009/225000 [306:01:35<135:38:23,  7.18s/it]
 70%|██████▉   | 157010/225000 [306:01:41<124:45:48,  6.61s/it]
 70%|██████▉   | 157011/225000 [306:01:44<108:31:59,  5.75s/it]
 70%|██████▉   | 157012/225000 [306:01:48<98:26:03,  5.21s/it] 
 70%|██████▉   | 157013/225000 [306:02:00<132:17:44,  7.01s/it]
 70%|██████▉   | 157014/225000 [306:02:03<112:24:29,  5.95s/it]
 70%|██████▉   | 157015/225000 [306:02:10<116:56:09,  6.19s/it]
 70%|██████▉   | 157016/225000 [306:02:16<114:58:35,  6.09s/it]
 70%|██████▉   | 157017/225000 [306:02:19<100:38:58,  5.33s/it]
 70%|██████▉   | 157018/225000 [306:02:28<121:24:28,  6.43s/it]
 70%|██████▉   | 157019/225000 [306:02:32<103:23:07,  5.47s/it]
 70%|██████▉   | 157020/225000 [306:02:41<123:29:41,  6.54s/it]
 70%|██████▉   | 157021/225000 [306:02:44<104:50:33,  5.55s/it]
 70%|██████▉   | 157022/225000 [306:02:50<107:47:30,  5.71s/it]
 70%|██████▉   | 157023/225000 [306:02:57<115:43:35,  6.13s/it]
 70%|██████▉   | 157024/225000 [306:03:01<103:04:15,  5.46s/it]
 70%|██████▉   | 157025/225000 [306:03:06<100:13:11,  5.31s/it]
 70%|██████▉   | 157026/225000 [306:03:11<98:37:14,  5.22s/it] 
 70%|██████▉   | 157027/225000 [306:03:15<93:56:11,  4.98s/it]
 70%|██████▉   | 157028/225000 [306:03:20<94:42:10,  5.02s/it]
 70%|██████▉   | 157029/225000 [306:03:28<110:10:24,  5.84s/it]
 70%|██████▉   | 157030/225000 [306:03:38<130:24:00,  6.91s/it]
 70%|██████▉   | 157031/225000 [306:03:42<116:06:01,  6.15s/it]
 70%|██████▉   | 157032/225000 [306:03:47<109:10:59,  5.78s/it]
 70%|██████▉   | 157033/225000 [306:03:52<103:06:07,  5.46s/it]
 70%|██████▉   | 157034/225000 [306:03:56<97:47:02,  5.18s/it] 
 70%|██████▉   | 157035/225000 [306:04:01<95:55:17,  5.08s/it]
 70%|██████▉   | 157036/225000 [306:04:05<92:59:31,  4.93s/it]
 70%|██████▉   | 157037/225000 [306:04:12<103:32:00,  5.48s/it]
 70%|██████▉   | 157038/225000 [306:04:25<145:30:11,  7.71s/it]
 70%|██████▉   | 157039/225000 [306:04:30<127:10:16,  6.74s/it]
 70%|██████▉   | 157040/225000 [306:04:34<115:32:14,  6.12s/it]
 70%|██████▉   | 157041/225000 [306:04:39<106:26:48,  5.64s/it]
 70%|██████▉   | 157042/225000 [306:04:43<100:00:57,  5.30s/it]
 70%|██████▉   | 157043/225000 [306:04:54<130:35:14,  6.92s/it]
 70%|██████▉   | 157044/225000 [306:05:00<122:40:30,  6.50s/it]
 70%|██████▉   | 157045/225000 [306:05:03<105:48:02,  5.60s/it]
 70%|██████▉   | 157046/225000 [306:05:07<96:57:36,  5.14s/it] 
 70%|██████▉   | 157047/225000 [306:05:14<108:14:33,  5.73s/it]
 70%|██████▉   | 157048/225000 [306:05:24<130:28:51,  6.91s/it]
 70%|██████▉   | 157049/225000 [306:05:28<114:08:50,  6.05s/it]
 70%|██████▉   | 157050/225000 [306:05:33<107:05:34,  5.67s/it]
                                                               
{'loss': '0.7833', 'grad_norm': '1.096', 'learning_rate': '1.137e-05', 'epoch': '0.698', 'num_input_tokens_seen': 506820320, 'train_runtime': '1.102e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157050/225000 [306:05:33<107:05:34,  5.67s/it]
 70%|██████▉   | 157051/225000 [306:05:38<104:20:57,  5.53s/it]
 70%|██████▉   | 157052/225000 [306:05:44<105:19:24,  5.58s/it]
 70%|██████▉   | 157053/225000 [306:05:49<105:17:35,  5.58s/it]
 70%|██████▉   | 157054/225000 [306:05:55<105:52:18,  5.61s/it]
 70%|██████▉   | 157055/225000 [306:06:05<131:36:01,  6.97s/it]
 70%|██████▉   | 157056/225000 [306:06:10<119:52:48,  6.35s/it]
 70%|██████▉   | 157057/225000 [306:06:17<123:12:12,  6.53s/it]
 70%|██████▉   | 157058/225000 [306:06:23<121:45:03,  6.45s/it]
 70%|██████▉   | 157059/225000 [306:06:30<124:37:05,  6.60s/it]
 70%|██████▉   | 157060/225000 [306:06:34<106:50:08,  5.66s/it]
 70%|██████▉   | 157061/225000 [306:06:38<97:39:27,  5.17s/it] 
 70%|██████▉   | 157062/225000 [306:06:44<103:51:32,  5.50s/it]
 70%|██████▉   | 157063/225000 [306:06:49<100:37:40,  5.33s/it]
 70%|██████▉   | 157064/225000 [306:06:53<95:14:03,  5.05s/it] 
 70%|██████▉   | 157065/225000 [306:06:58<92:52:14,  4.92s/it]
 70%|██████▉   | 157066/225000 [306:07:06<113:20:36,  6.01s/it]
 70%|██████▉   | 157067/225000 [306:07:16<132:35:55,  7.03s/it]
 70%|██████▉   | 157068/225000 [306:07:20<116:14:13,  6.16s/it]
 70%|██████▉   | 157069/225000 [306:07:24<106:02:40,  5.62s/it]
 70%|██████▉   | 157070/225000 [306:07:29<99:45:53,  5.29s/it] 
 70%|██████▉   | 157071/225000 [306:07:35<106:25:01,  5.64s/it]
 70%|██████▉   | 157072/225000 [306:07:46<137:48:46,  7.30s/it]
 70%|██████▉   | 157073/225000 [306:07:50<114:41:37,  6.08s/it]
 70%|██████▉   | 157074/225000 [306:07:55<110:54:45,  5.88s/it]
 70%|██████▉   | 157075/225000 [306:08:01<109:00:58,  5.78s/it]
 70%|██████▉   | 157076/225000 [306:08:08<117:31:13,  6.23s/it]
 70%|██████▉   | 157077/225000 [306:08:12<107:06:19,  5.68s/it]
 70%|██████▉   | 157078/225000 [306:08:16<97:21:03,  5.16s/it] 
 70%|██████▉   | 157079/225000 [306:08:20<90:25:51,  4.79s/it]
 70%|██████▉   | 157080/225000 [306:08:25<89:08:06,  4.72s/it]
 70%|██████▉   | 157081/225000 [306:08:35<118:07:33,  6.26s/it]
 70%|██████▉   | 157082/225000 [306:08:42<122:31:08,  6.49s/it]
 70%|██████▉   | 157083/225000 [306:08:46<107:53:27,  5.72s/it]
 70%|██████▉   | 157084/225000 [306:08:55<128:00:47,  6.79s/it]
 70%|██████▉   | 157085/225000 [306:08:59<113:41:13,  6.03s/it]
 70%|██████▉   | 157086/225000 [306:09:03<103:39:49,  5.50s/it]
 70%|██████▉   | 157087/225000 [306:09:07<95:36:12,  5.07s/it] 
 70%|██████▉   | 157088/225000 [306:09:16<115:41:02,  6.13s/it]
 70%|██████▉   | 157089/225000 [306:09:24<124:21:38,  6.59s/it]
 70%|██████▉   | 157090/225000 [306:09:28<112:26:07,  5.96s/it]
 70%|██████▉   | 157091/225000 [306:09:33<105:24:05,  5.59s/it]
 70%|██████▉   | 157092/225000 [306:09:37<99:38:21,  5.28s/it] 
 70%|██████▉   | 157093/225000 [306:09:42<95:44:20,  5.08s/it]
 70%|██████▉   | 157094/225000 [306:09:46<91:59:04,  4.88s/it]
 70%|██████▉   | 157095/225000 [306:09:54<104:40:22,  5.55s/it]
 70%|██████▉   | 157096/225000 [306:10:04<131:33:32,  6.97s/it]
 70%|██████▉   | 157097/225000 [306:10:07<112:35:19,  5.97s/it]
 70%|██████▉   | 157098/225000 [306:10:15<121:12:44,  6.43s/it]
 70%|██████▉   | 157099/225000 [306:10:20<110:49:31,  5.88s/it]
 70%|██████▉   | 157100/225000 [306:10:24<103:39:01,  5.50s/it]
                                                               
{'loss': '0.7722', 'grad_norm': '1.163', 'learning_rate': '1.135e-05', 'epoch': '0.6982', 'num_input_tokens_seen': 506951216, 'train_runtime': '1.102e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157100/225000 [306:10:24<103:39:01,  5.50s/it]
 70%|██████▉   | 157101/225000 [306:10:29<98:40:16,  5.23s/it] 
 70%|██████▉   | 157102/225000 [306:10:33<95:20:31,  5.06s/it]
 70%|██████▉   | 157103/225000 [306:10:41<107:00:40,  5.67s/it]
 70%|██████▉   | 157104/225000 [306:10:51<134:28:53,  7.13s/it]
 70%|██████▉   | 157105/225000 [306:10:54<112:41:55,  5.98s/it]
 70%|██████▉   | 157106/225000 [306:10:59<104:37:34,  5.55s/it]
 70%|██████▉   | 157107/225000 [306:11:03<98:12:07,  5.21s/it] 
 70%|██████▉   | 157108/225000 [306:11:08<93:53:25,  4.98s/it]
 70%|██████▉   | 157109/225000 [306:11:13<93:39:28,  4.97s/it]
 70%|██████▉   | 157110/225000 [306:11:18<92:57:15,  4.93s/it]
 70%|██████▉   | 157111/225000 [306:11:29<128:35:35,  6.82s/it]
 70%|██████▉   | 157112/225000 [306:11:36<128:54:18,  6.84s/it]
 70%|██████▉   | 157113/225000 [306:11:41<123:19:31,  6.54s/it]
 70%|██████▉   | 157114/225000 [306:11:48<120:26:15,  6.39s/it]
 70%|██████▉   | 157115/225000 [306:11:54<122:06:43,  6.48s/it]
 70%|██████▉   | 157116/225000 [306:12:01<125:22:41,  6.65s/it]
 70%|██████▉   | 157117/225000 [306:12:06<113:47:23,  6.03s/it]
 70%|██████▉   | 157118/225000 [306:12:10<104:11:25,  5.53s/it]
 70%|██████▉   | 157119/225000 [306:12:15<101:23:26,  5.38s/it]
 70%|██████▉   | 157120/225000 [306:12:20<100:02:08,  5.31s/it]
 70%|██████▉   | 157121/225000 [306:12:25<97:59:30,  5.20s/it] 
 70%|██████▉   | 157122/225000 [306:12:30<95:10:46,  5.05s/it]
 70%|██████▉   | 157123/225000 [306:12:35<95:22:55,  5.06s/it]
 70%|██████▉   | 157124/225000 [306:12:40<93:18:10,  4.95s/it]
 70%|██████▉   | 157125/225000 [306:12:45<95:47:26,  5.08s/it]
 70%|██████▉   | 157126/225000 [306:12:50<96:43:58,  5.13s/it]
 70%|██████▉   | 157127/225000 [306:12:55<95:12:25,  5.05s/it]
 70%|██████▉   | 157128/225000 [306:13:01<96:31:11,  5.12s/it]
 70%|██████▉   | 157129/225000 [306:13:05<94:28:07,  5.01s/it]
 70%|██████▉   | 157130/225000 [306:13:13<112:13:46,  5.95s/it]
 70%|██████▉   | 157131/225000 [306:13:27<152:18:01,  8.08s/it]
 70%|██████▉   | 157132/225000 [306:13:31<129:59:04,  6.89s/it]
 70%|██████▉   | 157133/225000 [306:13:35<115:31:43,  6.13s/it]
 70%|██████▉   | 157134/225000 [306:13:43<128:29:35,  6.82s/it]
 70%|██████▉   | 157135/225000 [306:13:47<110:20:14,  5.85s/it]
 70%|██████▉   | 157136/225000 [306:13:55<119:42:52,  6.35s/it]
 70%|██████▉   | 157137/225000 [306:13:59<110:49:22,  5.88s/it]
 70%|██████▉   | 157138/225000 [306:14:04<103:23:37,  5.48s/it]
 70%|██████▉   | 157139/225000 [306:14:08<97:15:56,  5.16s/it] 
 70%|██████▉   | 157140/225000 [306:14:13<94:47:52,  5.03s/it]
 70%|██████▉   | 157141/225000 [306:14:25<136:06:04,  7.22s/it]
 70%|██████▉   | 157142/225000 [306:14:29<116:12:00,  6.16s/it]
 70%|██████▉   | 157143/225000 [306:14:33<102:42:47,  5.45s/it]
 70%|██████▉   | 157144/225000 [306:14:37<96:03:39,  5.10s/it] 
 70%|██████▉   | 157145/225000 [306:14:41<88:17:40,  4.68s/it]
 70%|██████▉   | 157146/225000 [306:14:53<132:24:01,  7.02s/it]
 70%|██████▉   | 157147/225000 [306:14:56<110:01:28,  5.84s/it]
 70%|██████▉   | 157148/225000 [306:15:02<110:49:34,  5.88s/it]
 70%|██████▉   | 157149/225000 [306:15:07<105:22:30,  5.59s/it]
 70%|██████▉   | 157150/225000 [306:15:15<115:29:36,  6.13s/it]
                                                               
{'loss': '0.7767', 'grad_norm': '1.319', 'learning_rate': '1.133e-05', 'epoch': '0.6984', 'num_input_tokens_seen': 507081376, 'train_runtime': '1.103e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157150/225000 [306:15:15<115:29:36,  6.13s/it]
 70%|██████▉   | 157151/225000 [306:15:18<98:41:17,  5.24s/it] 
 70%|██████▉   | 157152/225000 [306:15:27<121:54:29,  6.47s/it]
 70%|██████▉   | 157153/225000 [306:15:31<109:00:08,  5.78s/it]
 70%|██████▉   | 157154/225000 [306:15:36<100:38:59,  5.34s/it]
 70%|██████▉   | 157155/225000 [306:15:40<96:27:48,  5.12s/it] 
 70%|██████▉   | 157156/225000 [306:15:45<93:49:28,  4.98s/it]
 70%|██████▉   | 157157/225000 [306:15:55<124:58:33,  6.63s/it]
 70%|██████▉   | 157158/225000 [306:16:01<117:52:23,  6.25s/it]
 70%|██████▉   | 157159/225000 [306:16:05<105:34:56,  5.60s/it]
 70%|██████▉   | 157160/225000 [306:16:09<97:08:03,  5.15s/it] 
 70%|██████▉   | 157161/225000 [306:16:17<113:56:22,  6.05s/it]
 70%|██████▉   | 157162/225000 [306:16:24<118:31:25,  6.29s/it]
 70%|██████▉   | 157163/225000 [306:16:28<104:53:44,  5.57s/it]
 70%|██████▉   | 157164/225000 [306:16:37<126:50:29,  6.73s/it]
 70%|██████▉   | 157165/225000 [306:16:41<109:25:49,  5.81s/it]
 70%|██████▉   | 157166/225000 [306:16:45<98:41:44,  5.24s/it] 
 70%|██████▉   | 157167/225000 [306:16:53<116:46:45,  6.20s/it]
 70%|██████▉   | 157168/225000 [306:16:58<111:26:41,  5.91s/it]
 70%|██████▉   | 157169/225000 [306:17:06<119:19:49,  6.33s/it]
 70%|██████▉   | 157170/225000 [306:17:11<113:43:40,  6.04s/it]
 70%|██████▉   | 157171/225000 [306:17:15<99:33:22,  5.28s/it] 
 70%|██████▉   | 157172/225000 [306:17:25<130:01:20,  6.90s/it]
 70%|██████▉   | 157173/225000 [306:17:29<109:42:49,  5.82s/it]
 70%|██████▉   | 157174/225000 [306:17:36<115:34:09,  6.13s/it]
 70%|██████▉   | 157175/225000 [306:17:40<104:23:09,  5.54s/it]
 70%|██████▉   | 157176/225000 [306:17:44<98:46:28,  5.24s/it] 
 70%|██████▉   | 157177/225000 [306:17:48<91:58:42,  4.88s/it]
 70%|██████▉   | 157178/225000 [306:17:54<94:42:53,  5.03s/it]
 70%|██████▉   | 157179/225000 [306:17:59<96:34:37,  5.13s/it]
 70%|██████▉   | 157180/225000 [306:18:04<98:17:44,  5.22s/it]
 70%|██████▉   | 157181/225000 [306:18:09<96:16:05,  5.11s/it]
 70%|██████▉   | 157182/225000 [306:18:19<123:51:05,  6.57s/it]
 70%|██████▉   | 157183/225000 [306:18:24<111:12:41,  5.90s/it]
 70%|██████▉   | 157184/225000 [306:18:31<119:19:56,  6.33s/it]
 70%|██████▉   | 157185/225000 [306:18:37<115:46:00,  6.15s/it]
 70%|██████▉   | 157186/225000 [306:18:41<107:05:13,  5.68s/it]
 70%|██████▉   | 157187/225000 [306:18:46<99:58:25,  5.31s/it] 
 70%|██████▉   | 157188/225000 [306:18:51<97:23:38,  5.17s/it]
 70%|██████▉   | 157189/225000 [306:18:55<92:35:48,  4.92s/it]
 70%|██████▉   | 157190/225000 [306:19:00<91:16:40,  4.85s/it]
 70%|██████▉   | 157191/225000 [306:19:12<135:10:01,  7.18s/it]
 70%|██████▉   | 157192/225000 [306:19:16<114:55:03,  6.10s/it]
 70%|██████▉   | 157193/225000 [306:19:25<130:09:43,  6.91s/it]
 70%|██████▉   | 157194/225000 [306:19:28<109:59:34,  5.84s/it]
 70%|██████▉   | 157195/225000 [306:19:32<97:39:33,  5.19s/it] 
 70%|██████▉   | 157196/225000 [306:19:36<95:31:44,  5.07s/it]
 70%|██████▉   | 157197/225000 [306:19:41<91:57:28,  4.88s/it]
 70%|██████▉   | 157198/225000 [306:19:48<102:37:21,  5.45s/it]
 70%|██████▉   | 157199/225000 [306:19:58<129:42:51,  6.89s/it]
 70%|██████▉   | 157200/225000 [306:20:02<113:29:24,  6.03s/it]
                                                               
{'loss': '0.7828', 'grad_norm': '1.102', 'learning_rate': '1.132e-05', 'epoch': '0.6987', 'num_input_tokens_seen': 507212432, 'train_runtime': '1.103e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157200/225000 [306:20:02<113:29:24,  6.03s/it]
 70%|██████▉   | 157201/225000 [306:20:06<103:05:22,  5.47s/it]
 70%|██████▉   | 157202/225000 [306:20:11<98:27:59,  5.23s/it] 
 70%|██████▉   | 157203/225000 [306:20:22<132:33:42,  7.04s/it]
 70%|██████▉   | 157204/225000 [306:20:25<112:48:35,  5.99s/it]
 70%|██████▉   | 157205/225000 [306:20:34<126:11:52,  6.70s/it]
 70%|██████▉   | 157206/225000 [306:20:39<115:59:49,  6.16s/it]
 70%|██████▉   | 157207/225000 [306:20:44<109:46:09,  5.83s/it]
 70%|██████▉   | 157208/225000 [306:20:49<106:42:21,  5.67s/it]
 70%|██████▉   | 157209/225000 [306:20:54<103:27:22,  5.49s/it]
 70%|██████▉   | 157210/225000 [306:20:59<100:30:02,  5.34s/it]
 70%|██████▉   | 157211/225000 [306:21:04<96:38:03,  5.13s/it] 
 70%|██████▉   | 157212/225000 [306:21:09<96:55:20,  5.15s/it]
 70%|██████▉   | 157213/225000 [306:21:14<97:56:18,  5.20s/it]
 70%|██████▉   | 157214/225000 [306:21:19<97:22:55,  5.17s/it]
 70%|██████▉   | 157215/225000 [306:21:31<134:02:13,  7.12s/it]
 70%|██████▉   | 157216/225000 [306:21:38<133:41:12,  7.10s/it]
 70%|██████▉   | 157217/225000 [306:21:42<115:51:48,  6.15s/it]
 70%|██████▉   | 157218/225000 [306:21:46<103:34:34,  5.50s/it]
 70%|██████▉   | 157219/225000 [306:21:50<95:38:46,  5.08s/it] 
 70%|██████▉   | 157220/225000 [306:22:01<129:36:05,  6.88s/it]
 70%|██████▉   | 157221/225000 [306:22:07<122:09:29,  6.49s/it]
 70%|██████▉   | 157222/225000 [306:22:11<110:23:49,  5.86s/it]
 70%|██████▉   | 157223/225000 [306:22:16<102:15:06,  5.43s/it]
 70%|██████▉   | 157224/225000 [306:22:20<96:30:49,  5.13s/it] 
 70%|██████▉   | 157225/225000 [306:22:24<92:18:34,  4.90s/it]
 70%|██████▉   | 157226/225000 [306:22:34<116:57:26,  6.21s/it]
 70%|██████▉   | 157227/225000 [306:22:41<121:37:29,  6.46s/it]
 70%|██████▉   | 157228/225000 [306:22:45<109:02:10,  5.79s/it]
 70%|██████▉   | 157229/225000 [306:22:49<98:53:58,  5.25s/it] 
 70%|██████▉   | 157230/225000 [306:22:53<92:15:37,  4.90s/it]
 70%|██████▉   | 157231/225000 [306:23:02<116:36:30,  6.19s/it]
 70%|██████▉   | 157232/225000 [306:23:11<128:36:49,  6.83s/it]
 70%|██████▉   | 157233/225000 [306:23:14<110:08:31,  5.85s/it]
 70%|██████▉   | 157234/225000 [306:23:23<125:00:40,  6.64s/it]
 70%|██████▉   | 157235/225000 [306:23:26<108:09:01,  5.75s/it]
 70%|██████▉   | 157236/225000 [306:23:30<97:06:17,  5.16s/it] 
 70%|██████▉   | 157237/225000 [306:23:34<92:16:41,  4.90s/it]
 70%|██████▉   | 157238/225000 [306:23:44<119:18:20,  6.34s/it]
 70%|██████▉   | 157239/225000 [306:23:48<105:34:15,  5.61s/it]
 70%|██████▉   | 157240/225000 [306:23:52<95:34:48,  5.08s/it] 
 70%|██████▉   | 157241/225000 [306:23:57<95:12:21,  5.06s/it]
 70%|██████▉   | 157242/225000 [306:24:03<101:12:27,  5.38s/it]
 70%|██████▉   | 157243/225000 [306:24:08<97:04:33,  5.16s/it] 
 70%|██████▉   | 157244/225000 [306:24:12<94:27:44,  5.02s/it]
 70%|██████▉   | 157245/225000 [306:24:18<96:04:38,  5.10s/it]
 70%|██████▉   | 157246/225000 [306:24:23<95:42:33,  5.09s/it]
 70%|██████▉   | 157247/225000 [306:24:28<97:30:25,  5.18s/it]
 70%|██████▉   | 157248/225000 [306:24:33<97:28:02,  5.18s/it]
 70%|██████▉   | 157249/225000 [306:24:39<100:33:42,  5.34s/it]
 70%|██████▉   | 157250/225000 [306:24:45<103:37:23,  5.51s/it]
                                                               
{'loss': '0.7721', 'grad_norm': '1.123', 'learning_rate': '1.13e-05', 'epoch': '0.6989', 'num_input_tokens_seen': 507342840, 'train_runtime': '1.103e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157250/225000 [306:24:45<103:37:23,  5.51s/it]
 70%|██████▉   | 157251/225000 [306:24:50<104:16:44,  5.54s/it]
 70%|██████▉   | 157252/225000 [306:24:56<101:55:54,  5.42s/it]
 70%|██████▉   | 157253/225000 [306:25:01<101:28:31,  5.39s/it]
 70%|██████▉   | 157254/225000 [306:25:11<129:14:29,  6.87s/it]
 70%|██████▉   | 157255/225000 [306:25:20<140:22:51,  7.46s/it]
 70%|██████▉   | 157256/225000 [306:25:25<127:17:00,  6.76s/it]
 70%|██████▉   | 157257/225000 [306:25:31<119:34:35,  6.35s/it]
 70%|██████▉   | 157258/225000 [306:25:36<111:41:03,  5.94s/it]
 70%|██████▉   | 157259/225000 [306:25:41<109:03:35,  5.80s/it]
 70%|██████▉   | 157260/225000 [306:25:46<106:24:07,  5.65s/it]
 70%|██████▉   | 157261/225000 [306:25:52<104:04:35,  5.53s/it]
 70%|██████▉   | 157262/225000 [306:25:57<101:09:51,  5.38s/it]
 70%|██████▉   | 157263/225000 [306:26:02<100:10:24,  5.32s/it]
 70%|██████▉   | 157264/225000 [306:26:07<99:23:07,  5.28s/it] 
 70%|██████▉   | 157265/225000 [306:26:13<100:43:43,  5.35s/it]
 70%|██████▉   | 157266/225000 [306:26:18<100:54:06,  5.36s/it]
 70%|██████▉   | 157267/225000 [306:26:29<132:18:47,  7.03s/it]
 70%|██████▉   | 157268/225000 [306:26:38<143:15:37,  7.61s/it]
 70%|██████▉   | 157269/225000 [306:26:41<119:31:42,  6.35s/it]
 70%|██████▉   | 157270/225000 [306:26:49<126:21:30,  6.72s/it]
 70%|██████▉   | 157271/225000 [306:26:52<109:07:27,  5.80s/it]
 70%|██████▉   | 157272/225000 [306:26:56<99:05:52,  5.27s/it] 
 70%|██████▉   | 157273/225000 [306:27:01<94:43:00,  5.03s/it]
 70%|██████▉   | 157274/225000 [306:27:06<94:25:55,  5.02s/it]
 70%|██████▉   | 157275/225000 [306:27:11<92:21:31,  4.91s/it]
 70%|██████▉   | 157276/225000 [306:27:15<91:09:53,  4.85s/it]
 70%|██████▉   | 157277/225000 [306:27:26<123:00:38,  6.54s/it]
 70%|██████▉   | 157278/225000 [306:27:34<131:15:50,  6.98s/it]
 70%|██████▉   | 157279/225000 [306:27:37<110:56:37,  5.90s/it]
 70%|██████▉   | 157280/225000 [306:27:45<124:29:15,  6.62s/it]
 70%|██████▉   | 157281/225000 [306:27:49<108:18:42,  5.76s/it]
 70%|██████▉   | 157282/225000 [306:27:58<122:54:33,  6.53s/it]
 70%|██████▉   | 157283/225000 [306:28:01<103:16:20,  5.49s/it]
 70%|██████▉   | 157284/225000 [306:28:06<102:31:39,  5.45s/it]
 70%|██████▉   | 157285/225000 [306:28:11<101:40:03,  5.41s/it]
 70%|██████▉   | 157286/225000 [306:28:17<102:17:05,  5.44s/it]
 70%|██████▉   | 157287/225000 [306:28:22<100:30:19,  5.34s/it]
 70%|██████▉   | 157288/225000 [306:28:27<101:09:42,  5.38s/it]
 70%|██████▉   | 157289/225000 [306:28:33<100:02:40,  5.32s/it]
 70%|██████▉   | 157290/225000 [306:28:38<99:17:17,  5.28s/it] 
 70%|██████▉   | 157291/225000 [306:28:43<97:29:01,  5.18s/it]
 70%|██████▉   | 157292/225000 [306:28:52<118:28:47,  6.30s/it]
 70%|██████▉   | 157293/225000 [306:28:57<113:26:10,  6.03s/it]
 70%|██████▉   | 157294/225000 [306:29:02<107:29:44,  5.72s/it]
 70%|██████▉   | 157295/225000 [306:29:07<104:34:28,  5.56s/it]
 70%|██████▉   | 157296/225000 [306:29:12<100:32:16,  5.35s/it]
 70%|██████▉   | 157297/225000 [306:29:18<101:26:52,  5.39s/it]
 70%|██████▉   | 157298/225000 [306:29:22<97:53:46,  5.21s/it] 
 70%|██████▉   | 157299/225000 [306:29:28<99:57:55,  5.32s/it]
 70%|██████▉   | 157300/225000 [306:29:33<101:28:12,  5.40s/it]
                                                               
{'loss': '0.7786', 'grad_norm': '1.089', 'learning_rate': '1.129e-05', 'epoch': '0.6991', 'num_input_tokens_seen': 507472552, 'train_runtime': '1.103e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157300/225000 [306:29:33<101:28:12,  5.40s/it]
 70%|██████▉   | 157301/225000 [306:29:38<98:49:45,  5.26s/it] 
 70%|██████▉   | 157302/225000 [306:29:44<99:28:02,  5.29s/it]
 70%|██████▉   | 157303/225000 [306:29:54<126:09:57,  6.71s/it]
 70%|██████▉   | 157304/225000 [306:30:04<144:15:11,  7.67s/it]
 70%|██████▉   | 157305/225000 [306:30:07<122:18:58,  6.50s/it]
 70%|██████▉   | 157306/225000 [306:30:11<105:03:26,  5.59s/it]
 70%|██████▉   | 157307/225000 [306:30:21<129:02:19,  6.86s/it]
 70%|██████▉   | 157308/225000 [306:30:26<120:12:24,  6.39s/it]
 70%|██████▉   | 157309/225000 [306:30:30<105:50:45,  5.63s/it]
 70%|██████▉   | 157310/225000 [306:30:34<97:50:05,  5.20s/it] 
 70%|██████▉   | 157311/225000 [306:30:38<92:13:41,  4.91s/it]
 70%|██████▉   | 157312/225000 [306:30:50<128:24:16,  6.83s/it]
 70%|██████▉   | 157313/225000 [306:30:54<116:37:21,  6.20s/it]
 70%|██████▉   | 157314/225000 [306:30:59<109:33:38,  5.83s/it]
 70%|██████▉   | 157315/225000 [306:31:04<103:51:08,  5.52s/it]
 70%|██████▉   | 157316/225000 [306:31:09<99:12:42,  5.28s/it] 
 70%|██████▉   | 157317/225000 [306:31:13<94:57:21,  5.05s/it]
 70%|██████▉   | 157318/225000 [306:31:18<93:54:27,  4.99s/it]
 70%|██████▉   | 157319/225000 [306:31:23<93:59:58,  5.00s/it]
 70%|██████▉   | 157320/225000 [306:31:28<94:24:04,  5.02s/it]
 70%|██████▉   | 157321/225000 [306:31:41<139:32:26,  7.42s/it]
 70%|██████▉   | 157322/225000 [306:31:50<144:11:16,  7.67s/it]
 70%|██████▉   | 157323/225000 [306:31:55<131:11:01,  6.98s/it]
 70%|██████▉   | 157324/225000 [306:31:59<111:56:21,  5.95s/it]
 70%|██████▉   | 157325/225000 [306:32:06<119:50:12,  6.37s/it]
 70%|██████▉   | 157326/225000 [306:32:10<104:31:13,  5.56s/it]
 70%|██████▉   | 157327/225000 [306:32:15<105:01:00,  5.59s/it]
 70%|██████▉   | 157328/225000 [306:32:25<126:28:50,  6.73s/it]
 70%|██████▉   | 157329/225000 [306:32:29<111:57:35,  5.96s/it]
 70%|██████▉   | 157330/225000 [306:32:33<102:17:22,  5.44s/it]
 70%|██████▉   | 157331/225000 [306:32:37<95:56:46,  5.10s/it] 
 70%|██████▉   | 157332/225000 [306:32:48<126:31:40,  6.73s/it]
 70%|██████▉   | 157333/225000 [306:32:52<109:46:19,  5.84s/it]
 70%|██████▉   | 157334/225000 [306:32:56<101:35:07,  5.40s/it]
 70%|██████▉   | 157335/225000 [306:33:01<98:41:30,  5.25s/it] 
 70%|██████▉   | 157336/225000 [306:33:06<96:06:18,  5.11s/it]
 70%|██████▉   | 157337/225000 [306:33:11<95:57:56,  5.11s/it]
 70%|██████▉   | 157338/225000 [306:33:17<101:19:11,  5.39s/it]
 70%|██████▉   | 157339/225000 [306:33:29<137:18:42,  7.31s/it]
 70%|██████▉   | 157340/225000 [306:33:34<124:26:30,  6.62s/it]
 70%|██████▉   | 157341/225000 [306:33:38<112:40:15,  6.00s/it]
 70%|██████▉   | 157342/225000 [306:33:46<122:34:51,  6.52s/it]
 70%|██████▉   | 157343/225000 [306:33:49<103:41:44,  5.52s/it]
 70%|██████▉   | 157344/225000 [306:33:57<119:17:12,  6.35s/it]
 70%|██████▉   | 157345/225000 [306:34:02<110:02:12,  5.86s/it]
 70%|██████▉   | 157346/225000 [306:34:06<101:00:48,  5.38s/it]
 70%|██████▉   | 157347/225000 [306:34:14<111:40:43,  5.94s/it]
 70%|██████▉   | 157348/225000 [306:34:18<101:19:36,  5.39s/it]
 70%|██████▉   | 157349/225000 [306:34:26<120:35:09,  6.42s/it]
 70%|██████▉   | 157350/225000 [306:34:31<111:33:45,  5.94s/it]
                                                               
{'loss': '0.7784', 'grad_norm': '1.09', 'learning_rate': '1.127e-05', 'epoch': '0.6993', 'num_input_tokens_seen': 507603424, 'train_runtime': '1.104e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157350/225000 [306:34:31<111:33:45,  5.94s/it]
 70%|██████▉   | 157351/225000 [306:34:36<105:22:50,  5.61s/it]
 70%|██████▉   | 157352/225000 [306:34:41<100:47:42,  5.36s/it]
 70%|██████▉   | 157353/225000 [306:34:45<95:30:26,  5.08s/it] 
 70%|██████▉   | 157354/225000 [306:34:50<95:20:05,  5.07s/it]
 70%|██████▉   | 157355/225000 [306:34:55<93:45:14,  4.99s/it]
 70%|██████▉   | 157356/225000 [306:35:00<94:42:18,  5.04s/it]
 70%|██████▉   | 157357/225000 [306:35:05<93:11:13,  4.96s/it]
 70%|██████▉   | 157358/225000 [306:35:18<137:20:22,  7.31s/it]
 70%|██████▉   | 157359/225000 [306:35:22<120:08:54,  6.39s/it]
 70%|██████▉   | 157360/225000 [306:35:26<105:08:07,  5.60s/it]
 70%|██████▉   | 157361/225000 [306:35:35<126:27:10,  6.73s/it]
 70%|██████▉   | 157362/225000 [306:35:41<117:59:05,  6.28s/it]
 70%|██████▉   | 157363/225000 [306:35:45<110:07:44,  5.86s/it]
 70%|██████▉   | 157364/225000 [306:35:53<117:45:01,  6.27s/it]
 70%|██████▉   | 157365/225000 [306:35:56<100:08:13,  5.33s/it]
 70%|██████▉   | 157366/225000 [306:36:00<96:07:27,  5.12s/it] 
 70%|██████▉   | 157367/225000 [306:36:05<93:37:03,  4.98s/it]
 70%|██████▉   | 157368/225000 [306:36:10<91:27:35,  4.87s/it]
 70%|██████▉   | 157369/225000 [306:36:15<93:17:26,  4.97s/it]
 70%|██████▉   | 157370/225000 [306:36:20<94:23:38,  5.02s/it]
 70%|██████▉   | 157371/225000 [306:36:30<123:50:40,  6.59s/it]
 70%|██████▉   | 157372/225000 [306:36:39<134:54:54,  7.18s/it]
 70%|██████▉   | 157373/225000 [306:36:43<119:34:36,  6.37s/it]
 70%|██████▉   | 157374/225000 [306:36:48<107:54:36,  5.74s/it]
 70%|██████▉   | 157375/225000 [306:36:52<100:30:27,  5.35s/it]
 70%|██████▉   | 157376/225000 [306:36:57<100:12:56,  5.34s/it]
 70%|██████▉   | 157377/225000 [306:37:09<137:08:20,  7.30s/it]
 70%|██████▉   | 157378/225000 [306:37:13<114:59:30,  6.12s/it]
 70%|██████▉   | 157379/225000 [306:37:17<105:29:12,  5.62s/it]
 70%|██████▉   | 157380/225000 [306:37:22<100:10:08,  5.33s/it]
 70%|██████▉   | 157381/225000 [306:37:26<95:18:55,  5.07s/it] 
 70%|██████▉   | 157382/225000 [306:37:38<130:56:44,  6.97s/it]
 70%|██████▉   | 157383/225000 [306:37:44<128:26:34,  6.84s/it]
 70%|██████▉   | 157384/225000 [306:37:50<124:27:05,  6.63s/it]
 70%|██████▉   | 157385/225000 [306:37:54<106:20:56,  5.66s/it]
 70%|██████▉   | 157386/225000 [306:38:03<128:24:00,  6.84s/it]
 70%|██████▉   | 157387/225000 [306:38:07<113:03:14,  6.02s/it]
 70%|██████▉   | 157388/225000 [306:38:12<107:55:16,  5.75s/it]
 70%|██████▉   | 157389/225000 [306:38:21<126:49:48,  6.75s/it]
 70%|██████▉   | 157390/225000 [306:38:25<108:39:48,  5.79s/it]
 70%|██████▉   | 157391/225000 [306:38:30<106:35:18,  5.68s/it]
 70%|██████▉   | 157392/225000 [306:38:36<104:22:34,  5.56s/it]
 70%|██████▉   | 157393/225000 [306:38:41<101:37:47,  5.41s/it]
 70%|██████▉   | 157394/225000 [306:38:46<98:01:42,  5.22s/it] 
 70%|██████▉   | 157395/225000 [306:38:50<95:16:22,  5.07s/it]
 70%|██████▉   | 157396/225000 [306:38:55<95:56:49,  5.11s/it]
 70%|██████▉   | 157397/225000 [306:39:01<97:55:15,  5.21s/it]
 70%|██████▉   | 157398/225000 [306:39:06<98:02:49,  5.22s/it]
 70%|██████▉   | 157399/225000 [306:39:11<96:51:41,  5.16s/it]
 70%|██████▉   | 157400/225000 [306:39:16<94:27:40,  5.03s/it]
                                                              
{'loss': '0.7835', 'grad_norm': '1.194', 'learning_rate': '1.126e-05', 'epoch': '0.6996', 'num_input_tokens_seen': 507732936, 'train_runtime': '1.104e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157400/225000 [306:39:16<94:27:40,  5.03s/it]
 70%|██████▉   | 157401/225000 [306:39:21<94:59:36,  5.06s/it]
 70%|██████▉   | 157402/225000 [306:39:33<133:32:15,  7.11s/it]
 70%|██████▉   | 157403/225000 [306:39:41<136:41:13,  7.28s/it]
 70%|██████▉   | 157404/225000 [306:39:44<114:54:09,  6.12s/it]
 70%|██████▉   | 157405/225000 [306:39:54<133:48:55,  7.13s/it]
 70%|██████▉   | 157406/225000 [306:39:58<118:06:20,  6.29s/it]
 70%|██████▉   | 157407/225000 [306:40:04<119:54:45,  6.39s/it]
 70%|██████▉   | 157408/225000 [306:40:08<103:12:58,  5.50s/it]
 70%|██████▉   | 157409/225000 [306:40:18<130:50:03,  6.97s/it]
 70%|██████▉   | 157410/225000 [306:40:24<123:26:52,  6.58s/it]
 70%|██████▉   | 157411/225000 [306:40:28<111:33:04,  5.94s/it]
 70%|██████▉   | 157412/225000 [306:40:34<110:18:07,  5.88s/it]
 70%|██████▉   | 157413/225000 [306:40:40<110:24:33,  5.88s/it]
 70%|██████▉   | 157414/225000 [306:40:44<97:54:47,  5.22s/it] 
 70%|██████▉   | 157415/225000 [306:40:54<127:59:41,  6.82s/it]
 70%|██████▉   | 157416/225000 [306:40:58<111:05:20,  5.92s/it]
 70%|██████▉   | 157417/225000 [306:41:02<98:56:43,  5.27s/it] 
 70%|██████▉   | 157418/225000 [306:41:11<123:17:01,  6.57s/it]
 70%|██████▉   | 157419/225000 [306:41:15<105:28:29,  5.62s/it]
 70%|██████▉   | 157420/225000 [306:41:23<121:38:18,  6.48s/it]
 70%|██████▉   | 157421/225000 [306:41:28<109:49:16,  5.85s/it]
 70%|██████▉   | 157422/225000 [306:41:36<122:32:35,  6.53s/it]
 70%|██████▉   | 157423/225000 [306:41:39<106:05:59,  5.65s/it]
 70%|██████▉   | 157424/225000 [306:41:43<96:14:51,  5.13s/it] 
 70%|██████▉   | 157425/225000 [306:41:48<94:17:19,  5.02s/it]
 70%|██████▉   | 157426/225000 [306:41:53<92:14:03,  4.91s/it]
 70%|██████▉   | 157427/225000 [306:41:58<92:55:55,  4.95s/it]
 70%|██████▉   | 157428/225000 [306:42:10<133:29:28,  7.11s/it]
 70%|██████▉   | 157429/225000 [306:42:16<127:33:37,  6.80s/it]
 70%|██████▉   | 157430/225000 [306:42:19<108:37:00,  5.79s/it]
 70%|██████▉   | 157431/225000 [306:42:28<121:58:19,  6.50s/it]
 70%|██████▉   | 157432/225000 [306:42:32<111:23:08,  5.93s/it]
 70%|██████▉   | 157433/225000 [306:42:36<99:44:49,  5.31s/it] 
 70%|██████▉   | 157434/225000 [306:42:40<93:03:37,  4.96s/it]
 70%|██████▉   | 157435/225000 [306:42:45<91:05:19,  4.85s/it]
 70%|██████▉   | 157436/225000 [306:42:49<89:30:14,  4.77s/it]
 70%|██████▉   | 157437/225000 [306:42:54<86:00:10,  4.58s/it]
 70%|██████▉   | 157438/225000 [306:43:00<97:19:03,  5.19s/it]
 70%|██████▉   | 157439/225000 [306:43:12<134:34:01,  7.17s/it]
 70%|██████▉   | 157440/225000 [306:43:16<117:15:45,  6.25s/it]
 70%|██████▉   | 157441/225000 [306:43:21<108:06:39,  5.76s/it]
 70%|██████▉   | 157442/225000 [306:43:25<100:36:15,  5.36s/it]
 70%|██████▉   | 157443/225000 [306:43:37<135:23:06,  7.21s/it]
 70%|██████▉   | 157444/225000 [306:43:41<120:50:00,  6.44s/it]
 70%|██████▉   | 157445/225000 [306:43:47<117:14:27,  6.25s/it]
 70%|██████▉   | 157446/225000 [306:43:53<113:31:07,  6.05s/it]
 70%|██████▉   | 157447/225000 [306:43:58<109:24:37,  5.83s/it]
 70%|██████▉   | 157448/225000 [306:44:07<127:12:40,  6.78s/it]
 70%|██████▉   | 157449/225000 [306:44:11<110:43:42,  5.90s/it]
 70%|██████▉   | 157450/225000 [306:44:19<122:16:50,  6.52s/it]
                                                               
{'loss': '0.7852', 'grad_norm': '1.019', 'learning_rate': '1.124e-05', 'epoch': '0.6998', 'num_input_tokens_seen': 507862128, 'train_runtime': '1.104e+06', 'train_tokens_per_second': '459.9'}

 70%|██████▉   | 157450/225000 [306:44:19<122:16:50,  6.52s/it]
 70%|██████▉   | 157451/225000 [306:44:24<117:06:46,  6.24s/it]
 70%|██████▉   | 157452/225000 [306:44:31<120:17:02,  6.41s/it]
 70%|██████▉   | 157453/225000 [306:44:36<109:28:52,  5.83s/it]
 70%|██████▉   | 157454/225000 [306:44:39<97:28:26,  5.20s/it] 
 70%|██████▉   | 157455/225000 [306:44:44<96:53:05,  5.16s/it]
 70%|██████▉   | 157456/225000 [306:44:53<118:34:47,  6.32s/it]
 70%|██████▉   | 157457/225000 [306:44:57<104:16:17,  5.56s/it]
 70%|██████▉   | 157458/225000 [306:45:07<127:38:25,  6.80s/it]
 70%|██████▉   | 157459/225000 [306:45:10<107:42:28,  5.74s/it]
 70%|██████▉   | 157460/225000 [306:45:16<108:36:39,  5.79s/it]
 70%|██████▉   | 157461/225000 [306:45:24<118:18:59,  6.31s/it]
 70%|██████▉   | 157462/225000 [306:45:27<104:09:26,  5.55s/it]
 70%|██████▉   | 157463/225000 [306:45:37<127:32:27,  6.80s/it]
 70%|██████▉   | 157464/225000 [306:45:40<107:25:17,  5.73s/it]
 70%|██████▉   | 157465/225000 [306:45:49<123:03:26,  6.56s/it]
 70%|██████▉   | 157466/225000 [306:45:56<125:05:38,  6.67s/it]
 70%|██████▉   | 157467/225000 [306:46:00<111:21:53,  5.94s/it]
 70%|██████▉   | 157468/225000 [306:46:08<120:26:08,  6.42s/it]
 70%|██████▉   | 157469/225000 [306:46:11<102:08:23,  5.44s/it]
 70%|██████▉   | 157470/225000 [306:46:17<107:24:06,  5.73s/it]
 70%|██████▉   | 157471/225000 [306:46:22<101:05:02,  5.39s/it]
 70%|██████▉   | 157472/225000 [306:46:26<96:33:33,  5.15s/it] 
 70%|██████▉   | 157473/225000 [306:46:31<94:28:29,  5.04s/it]
 70%|██████▉   | 157474/225000 [306:46:36<92:15:11,  4.92s/it]
 70%|██████▉   | 157475/225000 [306:46:41<91:49:19,  4.90s/it]
 70%|██████▉   | 157476/225000 [306:46:45<89:05:56,  4.75s/it]
 70%|██████▉   | 157477/225000 [306:46:57<128:25:06,  6.85s/it]
 70%|██████▉   | 157478/225000 [306:47:01<115:57:53,  6.18s/it]
 70%|██████▉   | 157479/225000 [306:47:05<102:22:55,  5.46s/it]
 70%|██████▉   | 157480/225000 [306:47:14<123:14:54,  6.57s/it]
 70%|██████▉   | 157481/225000 [306:47:20<117:55:27,  6.29s/it]
 70%|██████▉   | 157482/225000 [306:47:24<103:40:57,  5.53s/it]
 70%|██████▉   | 157483/225000 [306:47:32<117:56:15,  6.29s/it]
 70%|██████▉   | 157484/225000 [306:47:39<123:17:14,  6.57s/it]
 70%|██████▉   | 157485/225000 [306:47:43<107:41:23,  5.74s/it]
 70%|██████▉   | 157486/225000 [306:47:47<98:25:01,  5.25s/it] 
 70%|██████▉   | 157487/225000 [306:47:54<106:33:16,  5.68s/it]
 70%|██████▉   | 157488/225000 [306:48:02<123:23:39,  6.58s/it]
 70%|██████▉   | 157489/225000 [306:48:05<104:46:44,  5.59s/it]
 70%|██████▉   | 157490/225000 [306:48:14<122:28:26,  6.53s/it]
 70%|██████▉   | 157491/225000 [306:48:19<111:47:44,  5.96s/it]
 70%|██████▉   | 157492/225000 [306:48:24<106:28:27,  5.68s/it]
 70%|██████▉   | 157493/225000 [306:48:28<100:25:28,  5.36s/it]
 70%|██████▉   | 157494/225000 [306:48:33<97:32:31,  5.20s/it] 
 70%|██████▉   | 157495/225000 [306:48:38<94:02:27,  5.02s/it]
 70%|██████▉   | 157496/225000 [306:48:42<91:39:09,  4.89s/it]
 70%|██████▉   | 157497/225000 [306:48:47<91:37:27,  4.89s/it]
 70%|██████▉   | 157498/225000 [306:48:52<91:24:11,  4.87s/it]
 70%|██████▉   | 157499/225000 [306:48:57<91:21:40,  4.87s/it]
 70%|███████   | 157500/225000 [306:49:08<123:05:40,  6.57s/it]
                                                               
{'loss': '0.7874', 'grad_norm': '1.223', 'learning_rate': '1.123e-05', 'epoch': '0.7', 'num_input_tokens_seen': 507991600, 'train_runtime': '1.105e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157500/225000 [306:49:08<123:05:40,  6.57s/it]
 70%|███████   | 157501/225000 [306:49:17<136:15:09,  7.27s/it]
 70%|███████   | 157502/225000 [306:49:20<114:19:17,  6.10s/it]
 70%|███████   | 157503/225000 [306:49:29<130:18:29,  6.95s/it]
 70%|███████   | 157504/225000 [306:49:32<110:49:10,  5.91s/it]
 70%|███████   | 157505/225000 [306:49:37<105:41:32,  5.64s/it]
 70%|███████   | 157506/225000 [306:49:47<127:54:22,  6.82s/it]
 70%|███████   | 157507/225000 [306:49:51<113:36:31,  6.06s/it]
 70%|███████   | 157508/225000 [306:49:56<104:09:57,  5.56s/it]
 70%|███████   | 157509/225000 [306:50:00<98:40:54,  5.26s/it] 
 70%|███████   | 157510/225000 [306:50:04<93:27:40,  4.99s/it]
 70%|███████   | 157511/225000 [306:50:13<113:02:47,  6.03s/it]
 70%|███████   | 157512/225000 [306:50:20<120:55:52,  6.45s/it]
 70%|███████   | 157513/225000 [306:50:26<114:16:17,  6.10s/it]
 70%|███████   | 157514/225000 [306:50:30<102:57:05,  5.49s/it]
 70%|███████   | 157515/225000 [306:50:39<123:33:34,  6.59s/it]
 70%|███████   | 157516/225000 [306:50:42<104:48:10,  5.59s/it]
 70%|███████   | 157517/225000 [306:50:46<96:19:05,  5.14s/it] 
 70%|███████   | 157518/225000 [306:50:51<93:39:44,  5.00s/it]
 70%|███████   | 157519/225000 [306:50:56<91:46:44,  4.90s/it]
 70%|███████   | 157520/225000 [306:51:00<91:27:35,  4.88s/it]
 70%|███████   | 157521/225000 [306:51:05<90:24:45,  4.82s/it]
 70%|███████   | 157522/225000 [306:51:15<116:30:25,  6.22s/it]
 70%|███████   | 157523/225000 [306:51:24<132:39:42,  7.08s/it]
 70%|███████   | 157524/225000 [306:51:30<130:02:56,  6.94s/it]
 70%|███████   | 157525/225000 [306:51:34<114:38:47,  6.12s/it]
 70%|███████   | 157526/225000 [306:51:43<129:08:42,  6.89s/it]
 70%|███████   | 157527/225000 [306:51:47<109:59:49,  5.87s/it]
 70%|███████   | 157528/225000 [306:51:51<100:19:46,  5.35s/it]
 70%|███████   | 157529/225000 [306:51:55<95:32:37,  5.10s/it] 
 70%|███████   | 157530/225000 [306:52:00<92:09:27,  4.92s/it]
 70%|███████   | 157531/225000 [306:52:09<113:58:51,  6.08s/it]
 70%|███████   | 157532/225000 [306:52:16<120:11:26,  6.41s/it]
 70%|███████   | 157533/225000 [306:52:19<103:14:30,  5.51s/it]
 70%|███████   | 157534/225000 [306:52:26<109:49:59,  5.86s/it]
 70%|███████   | 157535/225000 [306:52:32<111:06:21,  5.93s/it]
 70%|███████   | 157536/225000 [306:52:36<99:58:55,  5.34s/it] 
 70%|███████   | 157537/225000 [306:52:41<98:08:50,  5.24s/it]
 70%|███████   | 157538/225000 [306:52:52<129:29:18,  6.91s/it]
 70%|███████   | 157539/225000 [306:52:56<116:39:14,  6.23s/it]
 70%|███████   | 157540/225000 [306:53:00<104:25:13,  5.57s/it]
 70%|███████   | 157541/225000 [306:53:05<97:45:32,  5.22s/it] 
 70%|███████   | 157542/225000 [306:53:09<93:32:34,  4.99s/it]
 70%|███████   | 157543/225000 [306:53:19<122:08:57,  6.52s/it]
 70%|███████   | 157544/225000 [306:53:25<114:40:18,  6.12s/it]
 70%|███████   | 157545/225000 [306:53:31<114:17:15,  6.10s/it]
 70%|███████   | 157546/225000 [306:53:35<102:44:12,  5.48s/it]
 70%|███████   | 157547/225000 [306:53:38<91:57:39,  4.91s/it] 
 70%|███████   | 157548/225000 [306:53:46<110:57:30,  5.92s/it]
 70%|███████   | 157549/225000 [306:53:53<116:37:15,  6.22s/it]
 70%|███████   | 157550/225000 [306:53:59<114:41:24,  6.12s/it]
                                                               
{'loss': '0.7871', 'grad_norm': '1.136', 'learning_rate': '1.121e-05', 'epoch': '0.7002', 'num_input_tokens_seen': 508121312, 'train_runtime': '1.105e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157550/225000 [306:53:59<114:41:24,  6.12s/it]
 70%|███████   | 157551/225000 [306:54:03<100:44:09,  5.38s/it]
 70%|███████   | 157552/225000 [306:54:13<125:40:56,  6.71s/it]
 70%|███████   | 157553/225000 [306:54:16<106:15:48,  5.67s/it]
 70%|███████   | 157554/225000 [306:54:21<101:31:38,  5.42s/it]
 70%|███████   | 157555/225000 [306:54:28<112:31:10,  6.01s/it]
 70%|███████   | 157556/225000 [306:54:32<100:07:18,  5.34s/it]
 70%|███████   | 157557/225000 [306:54:36<93:54:31,  5.01s/it] 
 70%|███████   | 157558/225000 [306:54:46<119:08:00,  6.36s/it]
 70%|███████   | 157559/225000 [306:54:50<109:14:32,  5.83s/it]
 70%|███████   | 157560/225000 [306:54:55<102:07:03,  5.45s/it]
 70%|███████   | 157561/225000 [306:54:59<95:09:31,  5.08s/it] 
 70%|███████   | 157562/225000 [306:55:04<92:16:49,  4.93s/it]
 70%|███████   | 157563/225000 [306:55:09<92:37:55,  4.94s/it]
 70%|███████   | 157564/225000 [306:55:16<106:22:41,  5.68s/it]
 70%|███████   | 157565/225000 [306:55:27<137:38:51,  7.35s/it]
 70%|███████   | 157566/225000 [306:55:31<115:06:27,  6.15s/it]
 70%|███████   | 157567/225000 [306:55:36<110:06:08,  5.88s/it]
 70%|███████   | 157568/225000 [306:55:40<102:37:47,  5.48s/it]
 70%|███████   | 157569/225000 [306:55:45<98:26:52,  5.26s/it] 
 70%|███████   | 157570/225000 [306:55:50<96:24:43,  5.15s/it]
 70%|███████   | 157571/225000 [306:55:55<95:45:54,  5.11s/it]
 70%|███████   | 157572/225000 [306:56:05<121:30:10,  6.49s/it]
 70%|███████   | 157573/225000 [306:56:15<141:19:56,  7.55s/it]
 70%|███████   | 157574/225000 [306:56:18<119:32:27,  6.38s/it]
 70%|███████   | 157575/225000 [306:56:27<130:33:38,  6.97s/it]
 70%|███████   | 157576/225000 [306:56:30<111:46:42,  5.97s/it]
 70%|███████   | 157577/225000 [306:56:39<123:52:03,  6.61s/it]
 70%|███████   | 157578/225000 [306:56:42<107:52:27,  5.76s/it]
 70%|███████   | 157579/225000 [306:56:46<98:36:26,  5.27s/it] 
 70%|███████   | 157580/225000 [306:56:57<125:54:13,  6.72s/it]
 70%|███████   | 157581/225000 [306:57:00<107:15:18,  5.73s/it]
 70%|███████   | 157582/225000 [306:57:06<110:31:54,  5.90s/it]
 70%|███████   | 157583/225000 [306:57:15<124:49:45,  6.67s/it]
 70%|███████   | 157584/225000 [306:57:19<109:10:25,  5.83s/it]
 70%|███████   | 157585/225000 [306:57:22<96:22:57,  5.15s/it] 
 70%|███████   | 157586/225000 [306:57:28<101:13:46,  5.41s/it]
 70%|███████   | 157587/225000 [306:57:39<129:37:09,  6.92s/it]
 70%|███████   | 157588/225000 [306:57:42<108:16:31,  5.78s/it]
 70%|███████   | 157589/225000 [306:57:47<105:22:49,  5.63s/it]
 70%|███████   | 157590/225000 [306:57:53<105:22:48,  5.63s/it]
 70%|███████   | 157591/225000 [306:57:58<104:44:27,  5.59s/it]
 70%|███████   | 157592/225000 [306:58:04<104:22:49,  5.57s/it]
 70%|███████   | 157593/225000 [306:58:09<103:12:29,  5.51s/it]
 70%|███████   | 157594/225000 [306:58:14<101:39:46,  5.43s/it]
 70%|███████   | 157595/225000 [306:58:20<101:59:53,  5.45s/it]
 70%|███████   | 157596/225000 [306:58:25<102:10:16,  5.46s/it]
 70%|███████   | 157597/225000 [306:58:30<99:58:09,  5.34s/it] 
 70%|███████   | 157598/225000 [306:58:36<101:22:53,  5.41s/it]
 70%|███████   | 157599/225000 [306:58:44<116:28:31,  6.22s/it]
 70%|███████   | 157600/225000 [306:58:53<133:56:24,  7.15s/it]
                                                               
{'loss': '0.7807', 'grad_norm': '1.027', 'learning_rate': '1.12e-05', 'epoch': '0.7004', 'num_input_tokens_seen': 508253016, 'train_runtime': '1.105e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157600/225000 [306:58:53<133:56:24,  7.15s/it]
 70%|███████   | 157601/225000 [306:58:59<127:16:00,  6.80s/it]
 70%|███████   | 157602/225000 [306:59:03<112:21:30,  6.00s/it]
 70%|███████   | 157603/225000 [306:59:07<101:04:55,  5.40s/it]
 70%|███████   | 157604/225000 [306:59:12<99:02:35,  5.29s/it] 
 70%|███████   | 157605/225000 [306:59:21<115:44:14,  6.18s/it]
 70%|███████   | 157606/225000 [306:59:28<123:28:04,  6.60s/it]
 70%|███████   | 157607/225000 [306:59:32<105:04:34,  5.61s/it]
 70%|███████   | 157608/225000 [306:59:40<118:58:04,  6.36s/it]
 70%|███████   | 157609/225000 [306:59:45<111:16:12,  5.94s/it]
 70%|███████   | 157610/225000 [306:59:50<107:12:23,  5.73s/it]
 70%|███████   | 157611/225000 [306:59:55<103:34:52,  5.53s/it]
 70%|███████   | 157612/225000 [307:00:00<102:17:30,  5.46s/it]
 70%|███████   | 157613/225000 [307:00:06<101:14:16,  5.41s/it]
 70%|███████   | 157614/225000 [307:00:11<100:28:06,  5.37s/it]
 70%|███████   | 157615/225000 [307:00:16<97:43:32,  5.22s/it] 
 70%|███████   | 157616/225000 [307:00:21<95:22:47,  5.10s/it]
 70%|███████   | 157617/225000 [307:00:28<107:21:56,  5.74s/it]
 70%|███████   | 157618/225000 [307:00:37<128:27:12,  6.86s/it]
 70%|███████   | 157619/225000 [307:00:42<118:48:38,  6.35s/it]
 70%|███████   | 157620/225000 [307:00:48<112:53:35,  6.03s/it]
 70%|███████   | 157621/225000 [307:00:53<107:53:33,  5.76s/it]
 70%|███████   | 157622/225000 [307:00:58<104:35:09,  5.59s/it]
 70%|███████   | 157623/225000 [307:01:03<102:47:04,  5.49s/it]
 70%|███████   | 157624/225000 [307:01:08<98:23:12,  5.26s/it] 
 70%|███████   | 157625/225000 [307:01:13<97:59:23,  5.24s/it]
 70%|███████   | 157626/225000 [307:01:19<98:34:28,  5.27s/it]
 70%|███████   | 157627/225000 [307:01:23<96:31:12,  5.16s/it]
 70%|███████   | 157628/225000 [307:01:29<99:30:22,  5.32s/it]
 70%|███████   | 157629/225000 [307:01:41<138:38:09,  7.41s/it]
 70%|███████   | 157630/225000 [307:01:46<125:34:24,  6.71s/it]
 70%|███████   | 157631/225000 [307:01:50<108:08:52,  5.78s/it]
 70%|███████   | 157632/225000 [307:01:55<101:05:01,  5.40s/it]
 70%|███████   | 157633/225000 [307:01:59<94:28:16,  5.05s/it] 
 70%|███████   | 157634/225000 [307:02:06<105:02:43,  5.61s/it]
 70%|███████   | 157635/225000 [307:02:16<130:57:00,  7.00s/it]
 70%|███████   | 157636/225000 [307:02:19<110:36:59,  5.91s/it]
 70%|███████   | 157637/225000 [307:02:28<128:02:21,  6.84s/it]
 70%|███████   | 157638/225000 [307:02:33<113:02:52,  6.04s/it]
 70%|███████   | 157639/225000 [307:02:37<102:50:58,  5.50s/it]
 70%|███████   | 157640/225000 [307:02:41<93:58:14,  5.02s/it] 
 70%|███████   | 157641/225000 [307:02:45<90:16:01,  4.82s/it]
 70%|███████   | 157642/225000 [307:02:50<91:19:40,  4.88s/it]
 70%|███████   | 157643/225000 [307:02:59<113:14:27,  6.05s/it]
 70%|███████   | 157644/225000 [307:03:08<133:02:29,  7.11s/it]
 70%|███████   | 157645/225000 [307:03:13<117:02:08,  6.26s/it]
 70%|███████   | 157646/225000 [307:03:17<104:06:02,  5.56s/it]
 70%|███████   | 157647/225000 [307:03:23<106:53:57,  5.71s/it]
 70%|███████   | 157648/225000 [307:03:30<118:24:31,  6.33s/it]
 70%|███████   | 157649/225000 [307:03:34<104:05:34,  5.56s/it]
 70%|███████   | 157650/225000 [307:03:41<113:29:35,  6.07s/it]
                                                               
{'loss': '0.7831', 'grad_norm': '1.008', 'learning_rate': '1.118e-05', 'epoch': '0.7007', 'num_input_tokens_seen': 508382512, 'train_runtime': '1.105e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157650/225000 [307:03:42<113:29:35,  6.07s/it]
 70%|███████   | 157651/225000 [307:03:50<124:35:22,  6.66s/it]
 70%|███████   | 157652/225000 [307:03:54<111:49:05,  5.98s/it]
 70%|███████   | 157653/225000 [307:03:58<103:24:20,  5.53s/it]
 70%|███████   | 157654/225000 [307:04:03<95:36:03,  5.11s/it] 
 70%|███████   | 157655/225000 [307:04:14<130:05:50,  6.95s/it]
 70%|███████   | 157656/225000 [307:04:20<127:05:41,  6.79s/it]
 70%|███████   | 157657/225000 [307:04:25<114:14:50,  6.11s/it]
 70%|███████   | 157658/225000 [307:04:31<117:44:44,  6.29s/it]
 70%|███████   | 157659/225000 [307:04:38<121:37:20,  6.50s/it]
 70%|███████   | 157660/225000 [307:04:42<103:07:55,  5.51s/it]
 70%|███████   | 157661/225000 [307:04:47<103:54:00,  5.55s/it]
 70%|███████   | 157662/225000 [307:04:54<111:20:33,  5.95s/it]
 70%|███████   | 157663/225000 [307:04:59<104:19:29,  5.58s/it]
 70%|███████   | 157664/225000 [307:05:03<98:29:05,  5.27s/it] 
 70%|███████   | 157665/225000 [307:05:08<94:46:16,  5.07s/it]
 70%|███████   | 157666/225000 [307:05:13<92:39:37,  4.95s/it]
 70%|███████   | 157667/225000 [307:05:18<92:03:19,  4.92s/it]
 70%|███████   | 157668/225000 [307:05:22<91:38:24,  4.90s/it]
 70%|███████   | 157669/225000 [307:05:31<113:30:06,  6.07s/it]
 70%|███████   | 157670/225000 [307:05:41<133:00:13,  7.11s/it]
 70%|███████   | 157671/225000 [307:05:44<113:28:26,  6.07s/it]
 70%|███████   | 157672/225000 [307:05:48<100:43:39,  5.39s/it]
 70%|███████   | 157673/225000 [307:05:57<120:47:52,  6.46s/it]
 70%|███████   | 157674/225000 [307:06:02<110:55:38,  5.93s/it]
 70%|███████   | 157675/225000 [307:06:06<98:26:49,  5.26s/it] 
 70%|███████   | 157676/225000 [307:06:15<122:26:13,  6.55s/it]
 70%|███████   | 157677/225000 [307:06:20<112:10:03,  6.00s/it]
 70%|███████   | 157678/225000 [307:06:24<104:14:01,  5.57s/it]
 70%|███████   | 157679/225000 [307:06:29<99:40:04,  5.33s/it] 
 70%|███████   | 157680/225000 [307:06:34<95:21:31,  5.10s/it]
 70%|███████   | 157681/225000 [307:06:38<92:47:08,  4.96s/it]
 70%|███████   | 157682/225000 [307:06:48<116:16:39,  6.22s/it]
 70%|███████   | 157683/225000 [307:06:58<137:52:32,  7.37s/it]
 70%|███████   | 157684/225000 [307:07:01<115:38:38,  6.18s/it]
 70%|███████   | 157685/225000 [307:07:08<122:53:13,  6.57s/it]
 70%|███████   | 157686/225000 [307:07:13<111:18:06,  5.95s/it]
 70%|███████   | 157687/225000 [307:07:18<105:39:30,  5.65s/it]
 70%|███████   | 157688/225000 [307:07:23<103:44:24,  5.55s/it]
 70%|███████   | 157689/225000 [307:07:28<100:13:15,  5.36s/it]
 70%|███████   | 157690/225000 [307:07:33<98:58:28,  5.29s/it] 
 70%|███████   | 157691/225000 [307:07:38<95:04:22,  5.08s/it]
 70%|███████   | 157692/225000 [307:07:43<96:11:21,  5.14s/it]
 70%|███████   | 157693/225000 [307:07:51<111:20:47,  5.96s/it]
 70%|███████   | 157694/225000 [307:08:02<138:57:26,  7.43s/it]
 70%|███████   | 157695/225000 [307:08:05<116:47:04,  6.25s/it]
 70%|███████   | 157696/225000 [307:08:09<104:35:51,  5.59s/it]
 70%|███████   | 157697/225000 [307:08:19<128:39:32,  6.88s/it]
 70%|███████   | 157698/225000 [307:08:23<112:10:52,  6.00s/it]
 70%|███████   | 157699/225000 [307:08:31<123:25:21,  6.60s/it]
 70%|███████   | 157700/225000 [307:08:35<108:46:42,  5.82s/it]
                                                               
{'loss': '0.7741', 'grad_norm': '1.201', 'learning_rate': '1.117e-05', 'epoch': '0.7009', 'num_input_tokens_seen': 508513408, 'train_runtime': '1.106e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157700/225000 [307:08:35<108:46:42,  5.82s/it]
 70%|███████   | 157701/225000 [307:08:43<117:23:39,  6.28s/it]
 70%|███████   | 157702/225000 [307:08:47<105:14:59,  5.63s/it]
 70%|███████   | 157703/225000 [307:08:51<99:15:39,  5.31s/it] 
 70%|███████   | 157704/225000 [307:08:56<94:55:20,  5.08s/it]
 70%|███████   | 157705/225000 [307:09:00<91:25:11,  4.89s/it]
 70%|███████   | 157706/225000 [307:09:12<128:33:49,  6.88s/it]
 70%|███████   | 157707/225000 [307:09:19<128:32:28,  6.88s/it]
 70%|███████   | 157708/225000 [307:09:22<109:55:45,  5.88s/it]
 70%|███████   | 157709/225000 [307:09:28<108:39:08,  5.81s/it]
 70%|███████   | 157710/225000 [307:09:36<122:49:27,  6.57s/it]
 70%|███████   | 157711/225000 [307:09:41<112:43:59,  6.03s/it]
 70%|███████   | 157712/225000 [307:09:46<108:42:23,  5.82s/it]
 70%|███████   | 157713/225000 [307:09:51<102:16:49,  5.47s/it]
 70%|███████   | 157714/225000 [307:09:56<101:06:38,  5.41s/it]
 70%|███████   | 157715/225000 [307:10:01<98:09:14,  5.25s/it] 
 70%|███████   | 157716/225000 [307:10:06<95:53:23,  5.13s/it]
 70%|███████   | 157717/225000 [307:10:11<96:18:36,  5.15s/it]
 70%|███████   | 157718/225000 [307:10:19<112:31:18,  6.02s/it]
 70%|███████   | 157719/225000 [307:10:29<134:35:00,  7.20s/it]
 70%|███████   | 157720/225000 [307:10:33<114:21:04,  6.12s/it]
 70%|███████   | 157721/225000 [307:10:38<110:00:44,  5.89s/it]
 70%|███████   | 157722/225000 [307:10:46<119:28:49,  6.39s/it]
 70%|███████   | 157723/225000 [307:10:50<108:05:26,  5.78s/it]
 70%|███████   | 157724/225000 [307:10:58<119:11:54,  6.38s/it]
 70%|███████   | 157725/225000 [307:11:04<115:51:54,  6.20s/it]
 70%|███████   | 157726/225000 [307:11:08<104:58:47,  5.62s/it]
 70%|███████   | 157727/225000 [307:11:12<96:49:39,  5.18s/it] 
 70%|███████   | 157728/225000 [307:11:16<92:13:10,  4.94s/it]
 70%|███████   | 157729/225000 [307:11:27<122:25:17,  6.55s/it]
 70%|███████   | 157730/225000 [307:11:32<112:33:19,  6.02s/it]
 70%|███████   | 157731/225000 [307:11:37<109:55:26,  5.88s/it]
 70%|███████   | 157732/225000 [307:11:40<96:05:19,  5.14s/it] 
 70%|███████   | 157733/225000 [307:11:50<118:03:42,  6.32s/it]
 70%|███████   | 157734/225000 [307:11:53<100:46:13,  5.39s/it]
 70%|███████   | 157735/225000 [307:12:03<125:03:52,  6.69s/it]
 70%|███████   | 157736/225000 [307:12:06<106:37:01,  5.71s/it]
 70%|███████   | 157737/225000 [307:12:10<99:07:43,  5.31s/it] 
 70%|███████   | 157738/225000 [307:12:17<108:00:41,  5.78s/it]
 70%|███████   | 157739/225000 [307:12:21<98:39:09,  5.28s/it] 
 70%|███████   | 157740/225000 [307:12:27<99:11:29,  5.31s/it]
 70%|███████   | 157741/225000 [307:12:36<122:09:02,  6.54s/it]
 70%|███████   | 157742/225000 [307:12:40<108:15:06,  5.79s/it]
 70%|███████   | 157743/225000 [307:12:46<107:05:29,  5.73s/it]
 70%|███████   | 157744/225000 [307:12:52<110:52:46,  5.94s/it]
 70%|███████   | 157745/225000 [307:12:56<100:19:05,  5.37s/it]
 70%|███████   | 157746/225000 [307:13:01<98:11:23,  5.26s/it] 
 70%|███████   | 157747/225000 [307:13:11<123:34:40,  6.62s/it]
 70%|███████   | 157748/225000 [307:13:14<104:40:44,  5.60s/it]
 70%|███████   | 157749/225000 [307:13:21<113:24:17,  6.07s/it]
 70%|███████   | 157750/225000 [307:13:27<108:28:05,  5.81s/it]
                                                               
{'loss': '0.7675', 'grad_norm': '1.115', 'learning_rate': '1.115e-05', 'epoch': '0.7011', 'num_input_tokens_seen': 508644096, 'train_runtime': '1.106e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157750/225000 [307:13:27<108:28:05,  5.81s/it]
 70%|███████   | 157751/225000 [307:13:31<101:03:26,  5.41s/it]
 70%|███████   | 157752/225000 [307:13:36<97:15:49,  5.21s/it] 
 70%|███████   | 157753/225000 [307:13:40<93:47:30,  5.02s/it]
 70%|███████   | 157754/225000 [307:13:45<92:28:43,  4.95s/it]
 70%|███████   | 157755/225000 [307:13:50<93:39:40,  5.01s/it]
 70%|███████   | 157756/225000 [307:13:57<101:49:53,  5.45s/it]
 70%|███████   | 157757/225000 [307:14:08<136:33:12,  7.31s/it]
 70%|███████   | 157758/225000 [307:14:12<116:37:48,  6.24s/it]
 70%|███████   | 157759/225000 [307:14:16<101:45:14,  5.45s/it]
 70%|███████   | 157760/225000 [307:14:25<123:10:28,  6.59s/it]
 70%|███████   | 157761/225000 [307:14:30<115:11:27,  6.17s/it]
 70%|███████   | 157762/225000 [307:14:34<102:34:27,  5.49s/it]
 70%|███████   | 157763/225000 [307:14:38<95:03:33,  5.09s/it] 
 70%|███████   | 157764/225000 [307:14:43<91:26:28,  4.90s/it]
 70%|███████   | 157765/225000 [307:14:52<116:30:02,  6.24s/it]
 70%|███████   | 157766/225000 [307:14:58<116:19:38,  6.23s/it]
 70%|███████   | 157767/225000 [307:15:06<122:05:25,  6.54s/it]
 70%|███████   | 157768/225000 [307:15:09<104:37:44,  5.60s/it]
 70%|███████   | 157769/225000 [307:15:16<110:33:24,  5.92s/it]
 70%|███████   | 157770/225000 [307:15:23<119:41:53,  6.41s/it]
 70%|███████   | 157771/225000 [307:15:27<106:39:40,  5.71s/it]
 70%|███████   | 157772/225000 [307:15:32<98:37:25,  5.28s/it] 
 70%|███████   | 157773/225000 [307:15:36<94:42:28,  5.07s/it]
 70%|███████   | 157774/225000 [307:15:41<91:19:29,  4.89s/it]
 70%|███████   | 157775/225000 [307:15:50<118:56:44,  6.37s/it]
 70%|███████   | 157776/225000 [307:15:58<126:07:09,  6.75s/it]
 70%|███████   | 157777/225000 [307:16:02<111:14:48,  5.96s/it]
 70%|███████   | 157778/225000 [307:16:07<103:40:12,  5.55s/it]
 70%|███████   | 157779/225000 [307:16:12<100:47:41,  5.40s/it]
 70%|███████   | 157780/225000 [307:16:16<95:53:32,  5.14s/it] 
 70%|███████   | 157781/225000 [307:16:21<93:26:54,  5.00s/it]
 70%|███████   | 157782/225000 [307:16:26<95:11:09,  5.10s/it]
 70%|███████   | 157783/225000 [307:16:31<95:20:04,  5.11s/it]
 70%|███████   | 157784/225000 [307:16:40<116:25:12,  6.24s/it]
 70%|███████   | 157785/225000 [307:16:52<144:37:39,  7.75s/it]
 70%|███████   | 157786/225000 [307:16:56<125:21:36,  6.71s/it]
 70%|███████   | 157787/225000 [307:17:01<114:04:45,  6.11s/it]
 70%|███████   | 157788/225000 [307:17:05<103:31:48,  5.55s/it]
 70%|███████   | 157789/225000 [307:17:13<119:07:56,  6.38s/it]
 70%|███████   | 157790/225000 [307:17:20<120:12:52,  6.44s/it]
 70%|███████   | 157791/225000 [307:17:25<111:05:45,  5.95s/it]
 70%|███████   | 157792/225000 [307:17:30<107:04:56,  5.74s/it]
 70%|███████   | 157793/225000 [307:17:35<105:02:54,  5.63s/it]
 70%|███████   | 157794/225000 [307:17:40<101:17:45,  5.43s/it]
 70%|███████   | 157795/225000 [307:17:45<98:59:24,  5.30s/it] 
 70%|███████   | 157796/225000 [307:17:51<100:03:23,  5.36s/it]
 70%|███████   | 157797/225000 [307:17:56<99:41:49,  5.34s/it] 
 70%|███████   | 157798/225000 [307:18:01<96:36:23,  5.18s/it]
 70%|███████   | 157799/225000 [307:18:06<98:12:08,  5.26s/it]
 70%|███████   | 157800/225000 [307:18:11<97:52:29,  5.24s/it]
                                                              
{'loss': '0.7819', 'grad_norm': '1.105', 'learning_rate': '1.114e-05', 'epoch': '0.7013', 'num_input_tokens_seen': 508774864, 'train_runtime': '1.106e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157800/225000 [307:18:11<97:52:29,  5.24s/it]
 70%|███████   | 157801/225000 [307:18:17<99:25:59,  5.33s/it]
 70%|███████   | 157802/225000 [307:18:27<123:56:19,  6.64s/it]
 70%|███████   | 157803/225000 [307:18:34<127:47:23,  6.85s/it]
 70%|███████   | 157804/225000 [307:18:38<114:39:44,  6.14s/it]
 70%|███████   | 157805/225000 [307:18:43<108:00:02,  5.79s/it]
 70%|███████   | 157806/225000 [307:18:48<100:54:32,  5.41s/it]
 70%|███████   | 157807/225000 [307:18:55<111:40:12,  5.98s/it]
 70%|███████   | 157808/225000 [307:19:06<138:55:12,  7.44s/it]
 70%|███████   | 157809/225000 [307:19:09<115:40:48,  6.20s/it]
 70%|███████   | 157810/225000 [307:19:16<116:33:07,  6.24s/it]
 70%|███████   | 157811/225000 [307:19:22<115:29:50,  6.19s/it]
 70%|███████   | 157812/225000 [307:19:26<102:43:00,  5.50s/it]
 70%|███████   | 157813/225000 [307:19:35<122:43:44,  6.58s/it]
 70%|███████   | 157814/225000 [307:19:41<119:14:21,  6.39s/it]
 70%|███████   | 157815/225000 [307:19:46<110:47:00,  5.94s/it]
 70%|███████   | 157816/225000 [307:19:53<116:47:54,  6.26s/it]
 70%|███████   | 157817/225000 [307:19:57<104:09:51,  5.58s/it]
 70%|███████   | 157818/225000 [307:20:01<94:46:20,  5.08s/it] 
 70%|███████   | 157819/225000 [307:20:05<88:59:57,  4.77s/it]
 70%|███████   | 157820/225000 [307:20:12<104:11:23,  5.58s/it]
 70%|███████   | 157821/225000 [307:20:19<114:16:42,  6.12s/it]
 70%|███████   | 157822/225000 [307:20:26<118:15:38,  6.34s/it]
 70%|███████   | 157823/225000 [307:20:30<103:03:01,  5.52s/it]
 70%|███████   | 157824/225000 [307:20:37<111:24:07,  5.97s/it]
 70%|███████   | 157825/225000 [307:20:42<107:37:02,  5.77s/it]
 70%|███████   | 157826/225000 [307:20:46<95:25:20,  5.11s/it] 
 70%|███████   | 157827/225000 [307:20:50<90:54:07,  4.87s/it]
 70%|███████   | 157828/225000 [307:20:54<86:03:27,  4.61s/it]
 70%|███████   | 157829/225000 [307:21:02<102:35:55,  5.50s/it]
 70%|███████   | 157830/225000 [307:21:09<114:21:52,  6.13s/it]
 70%|███████   | 157831/225000 [307:21:13<99:51:40,  5.35s/it] 
 70%|███████   | 157832/225000 [307:21:17<90:41:27,  4.86s/it]
 70%|███████   | 157833/225000 [307:21:23<102:16:11,  5.48s/it]
 70%|███████   | 157834/225000 [307:21:33<126:26:18,  6.78s/it]
 70%|███████   | 157835/225000 [307:21:39<120:18:01,  6.45s/it]
 70%|███████   | 157836/225000 [307:21:45<117:03:07,  6.27s/it]
 70%|███████   | 157837/225000 [307:21:49<105:30:14,  5.66s/it]
 70%|███████   | 157838/225000 [307:21:56<113:49:20,  6.10s/it]
 70%|███████   | 157839/225000 [307:22:00<99:57:46,  5.36s/it] 
 70%|███████   | 157840/225000 [307:22:08<114:26:16,  6.13s/it]
 70%|███████   | 157841/225000 [307:22:14<113:02:37,  6.06s/it]
 70%|███████   | 157842/225000 [307:22:20<115:20:56,  6.18s/it]
 70%|███████   | 157843/225000 [307:22:26<111:40:15,  5.99s/it]
 70%|███████   | 157844/225000 [307:22:32<113:22:52,  6.08s/it]
 70%|███████   | 157845/225000 [307:22:36<103:57:41,  5.57s/it]
 70%|███████   | 157846/225000 [307:22:43<110:26:54,  5.92s/it]
 70%|███████   | 157847/225000 [307:22:47<97:02:36,  5.20s/it] 
 70%|███████   | 157848/225000 [307:22:52<96:51:21,  5.19s/it]
 70%|███████   | 157849/225000 [307:22:57<96:54:43,  5.20s/it]
 70%|███████   | 157850/225000 [307:23:02<96:03:40,  5.15s/it]
                                                              
{'loss': '0.7876', 'grad_norm': '1.316', 'learning_rate': '1.112e-05', 'epoch': '0.7016', 'num_input_tokens_seen': 508904760, 'train_runtime': '1.107e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157850/225000 [307:23:02<96:03:40,  5.15s/it]
 70%|███████   | 157851/225000 [307:23:07<93:55:42,  5.04s/it]
 70%|███████   | 157852/225000 [307:23:12<95:13:23,  5.11s/it]
 70%|███████   | 157853/225000 [307:23:17<95:10:24,  5.10s/it]
 70%|███████   | 157854/225000 [307:23:29<132:39:50,  7.11s/it]
 70%|███████   | 157855/225000 [307:23:36<130:28:46,  7.00s/it]
 70%|███████   | 157856/225000 [307:23:42<126:13:35,  6.77s/it]
 70%|███████   | 157857/225000 [307:23:47<118:00:18,  6.33s/it]
 70%|███████   | 157858/225000 [307:23:51<102:39:51,  5.50s/it]
 70%|███████   | 157859/225000 [307:23:55<93:48:54,  5.03s/it] 
 70%|███████   | 157860/225000 [307:23:59<88:41:51,  4.76s/it]
 70%|███████   | 157861/225000 [307:24:10<122:45:18,  6.58s/it]
 70%|███████   | 157862/225000 [307:24:15<113:20:37,  6.08s/it]
 70%|███████   | 157863/225000 [307:24:18<98:43:55,  5.29s/it] 
 70%|███████   | 157864/225000 [307:24:28<124:10:09,  6.66s/it]
 70%|███████   | 157865/225000 [307:24:32<108:59:55,  5.84s/it]
 70%|███████   | 157866/225000 [307:24:36<98:48:34,  5.30s/it] 
 70%|███████   | 157867/225000 [307:24:40<93:21:34,  5.01s/it]
 70%|███████   | 157868/225000 [307:24:46<98:16:29,  5.27s/it]
 70%|███████   | 157869/225000 [307:24:57<132:47:38,  7.12s/it]
 70%|███████   | 157870/225000 [307:25:01<114:06:46,  6.12s/it]
 70%|███████   | 157871/225000 [307:25:05<100:04:28,  5.37s/it]
 70%|███████   | 157872/225000 [307:25:14<121:15:42,  6.50s/it]
 70%|███████   | 157873/225000 [307:25:19<111:16:02,  5.97s/it]
 70%|███████   | 157874/225000 [307:25:25<114:48:19,  6.16s/it]
 70%|███████   | 157875/225000 [307:25:29<102:38:53,  5.51s/it]
 70%|███████   | 157876/225000 [307:25:33<93:00:07,  4.99s/it] 
 70%|███████   | 157877/225000 [307:25:37<87:50:48,  4.71s/it]
 70%|███████   | 157878/225000 [307:25:48<123:02:53,  6.60s/it]
 70%|███████   | 157879/225000 [307:25:55<125:24:00,  6.73s/it]
 70%|███████   | 157880/225000 [307:25:58<105:33:09,  5.66s/it]
 70%|███████   | 157881/225000 [307:26:07<121:26:58,  6.51s/it]
 70%|███████   | 157882/225000 [307:26:11<108:58:27,  5.85s/it]
 70%|███████   | 157883/225000 [307:26:16<100:42:44,  5.40s/it]
 70%|███████   | 157884/225000 [307:26:20<95:33:05,  5.13s/it] 
 70%|███████   | 157885/225000 [307:26:24<90:55:24,  4.88s/it]
 70%|███████   | 157886/225000 [307:26:29<92:03:00,  4.94s/it]
 70%|███████   | 157887/225000 [307:26:41<131:56:14,  7.08s/it]
 70%|███████   | 157888/225000 [307:26:45<111:34:29,  5.99s/it]
 70%|███████   | 157889/225000 [307:26:49<98:50:13,  5.30s/it] 
 70%|███████   | 157890/225000 [307:26:57<115:48:55,  6.21s/it]
 70%|███████   | 157891/225000 [307:27:02<111:00:28,  5.95s/it]
 70%|███████   | 157892/225000 [307:27:07<102:50:39,  5.52s/it]
 70%|███████   | 157893/225000 [307:27:15<116:55:57,  6.27s/it]
 70%|███████   | 157894/225000 [307:27:19<107:10:00,  5.75s/it]
 70%|███████   | 157895/225000 [307:27:26<109:54:36,  5.90s/it]
 70%|███████   | 157896/225000 [307:27:32<110:45:26,  5.94s/it]
 70%|███████   | 157897/225000 [307:27:38<110:58:04,  5.95s/it]
 70%|███████   | 157898/225000 [307:27:41<98:56:41,  5.31s/it] 
 70%|███████   | 157899/225000 [307:27:46<94:24:02,  5.06s/it]
 70%|███████   | 157900/225000 [307:27:50<88:52:36,  4.77s/it]
                                                              
{'loss': '0.7684', 'grad_norm': '1.193', 'learning_rate': '1.111e-05', 'epoch': '0.7018', 'num_input_tokens_seen': 509034944, 'train_runtime': '1.107e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157900/225000 [307:27:50<88:52:36,  4.77s/it]
 70%|███████   | 157901/225000 [307:28:00<120:38:49,  6.47s/it]
 70%|███████   | 157902/225000 [307:28:06<116:50:58,  6.27s/it]
 70%|███████   | 157903/225000 [307:28:12<116:16:24,  6.24s/it]
 70%|███████   | 157904/225000 [307:28:16<101:07:09,  5.43s/it]
 70%|███████   | 157905/225000 [307:28:20<91:34:34,  4.91s/it] 
 70%|███████   | 157906/225000 [307:28:27<105:24:00,  5.66s/it]
 70%|███████   | 157907/225000 [307:28:35<118:58:52,  6.38s/it]
 70%|███████   | 157908/225000 [307:28:39<104:03:39,  5.58s/it]
 70%|███████   | 157909/225000 [307:28:48<124:16:13,  6.67s/it]
 70%|███████   | 157910/225000 [307:28:51<105:36:47,  5.67s/it]
 70%|███████   | 157911/225000 [307:28:59<118:15:15,  6.35s/it]
 70%|███████   | 157912/225000 [307:29:03<101:41:06,  5.46s/it]
 70%|███████   | 157913/225000 [307:29:11<116:33:28,  6.25s/it]
 70%|███████   | 157914/225000 [307:29:15<106:01:49,  5.69s/it]
 70%|███████   | 157915/225000 [307:29:20<98:51:35,  5.31s/it] 
 70%|███████   | 157916/225000 [307:29:24<94:25:25,  5.07s/it]
 70%|███████   | 157917/225000 [307:29:35<124:30:53,  6.68s/it]
 70%|███████   | 157918/225000 [307:29:39<111:03:08,  5.96s/it]
 70%|███████   | 157919/225000 [307:29:43<102:29:22,  5.50s/it]
 70%|███████   | 157920/225000 [307:29:48<95:58:20,  5.15s/it] 
 70%|███████   | 157921/225000 [307:29:54<100:36:14,  5.40s/it]
 70%|███████   | 157922/225000 [307:30:05<132:41:26,  7.12s/it]
 70%|███████   | 157923/225000 [307:30:09<117:00:17,  6.28s/it]
 70%|███████   | 157924/225000 [307:30:14<107:50:18,  5.79s/it]
 70%|███████   | 157925/225000 [307:30:18<102:01:16,  5.48s/it]
 70%|███████   | 157926/225000 [307:30:24<100:12:40,  5.38s/it]
 70%|███████   | 157927/225000 [307:30:28<95:50:55,  5.14s/it] 
 70%|███████   | 157928/225000 [307:30:33<91:42:20,  4.92s/it]
 70%|███████   | 157929/225000 [307:30:41<113:07:34,  6.07s/it]
 70%|███████   | 157930/225000 [307:30:50<127:32:41,  6.85s/it]
 70%|███████   | 157931/225000 [307:30:53<107:08:49,  5.75s/it]
 70%|███████   | 157932/225000 [307:31:02<123:34:40,  6.63s/it]
 70%|███████   | 157933/225000 [307:31:06<109:37:17,  5.88s/it]
 70%|███████   | 157934/225000 [307:31:10<99:55:20,  5.36s/it] 
 70%|███████   | 157935/225000 [307:31:21<132:50:05,  7.13s/it]
 70%|███████   | 157936/225000 [307:31:25<114:13:43,  6.13s/it]
 70%|███████   | 157937/225000 [307:31:29<100:57:14,  5.42s/it]
 70%|███████   | 157938/225000 [307:31:37<118:00:39,  6.34s/it]
 70%|███████   | 157939/225000 [307:31:44<120:27:18,  6.47s/it]
 70%|███████   | 157940/225000 [307:31:48<103:42:50,  5.57s/it]
 70%|███████   | 157941/225000 [307:31:53<105:01:57,  5.64s/it]
 70%|███████   | 157942/225000 [307:32:00<109:20:24,  5.87s/it]
 70%|███████   | 157943/225000 [307:32:05<104:12:53,  5.59s/it]
 70%|███████   | 157944/225000 [307:32:14<123:51:53,  6.65s/it]
 70%|███████   | 157945/225000 [307:32:17<105:52:36,  5.68s/it]
 70%|███████   | 157946/225000 [307:32:23<104:31:15,  5.61s/it]
 70%|███████   | 157947/225000 [307:32:29<105:56:18,  5.69s/it]
 70%|███████   | 157948/225000 [307:32:35<108:30:53,  5.83s/it]
 70%|███████   | 157949/225000 [307:32:43<120:34:06,  6.47s/it]
 70%|███████   | 157950/225000 [307:32:46<102:55:06,  5.53s/it]
                                                               
{'loss': '0.7691', 'grad_norm': '1.153', 'learning_rate': '1.109e-05', 'epoch': '0.702', 'num_input_tokens_seen': 509164888, 'train_runtime': '1.107e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 157950/225000 [307:32:46<102:55:06,  5.53s/it]
 70%|███████   | 157951/225000 [307:32:53<110:17:29,  5.92s/it]
 70%|███████   | 157952/225000 [307:33:00<115:11:13,  6.18s/it]
 70%|███████   | 157953/225000 [307:33:04<101:51:23,  5.47s/it]
 70%|███████   | 157954/225000 [307:33:08<96:44:26,  5.19s/it] 
 70%|███████   | 157955/225000 [307:33:13<92:46:14,  4.98s/it]
 70%|███████   | 157956/225000 [307:33:17<89:37:53,  4.81s/it]
 70%|███████   | 157957/225000 [307:33:22<92:08:57,  4.95s/it]
 70%|███████   | 157958/225000 [307:33:35<133:43:49,  7.18s/it]
 70%|███████   | 157959/225000 [307:33:38<113:14:30,  6.08s/it]
 70%|███████   | 157960/225000 [307:33:47<126:00:14,  6.77s/it]
 70%|███████   | 157961/225000 [307:33:50<106:19:13,  5.71s/it]
 70%|███████   | 157962/225000 [307:33:56<110:22:36,  5.93s/it]
 70%|███████   | 157963/225000 [307:34:02<111:04:23,  5.96s/it]
 70%|███████   | 157964/225000 [307:34:08<110:26:41,  5.93s/it]
 70%|███████   | 157965/225000 [307:34:14<112:25:11,  6.04s/it]
 70%|███████   | 157966/225000 [307:34:18<100:04:06,  5.37s/it]
 70%|███████   | 157967/225000 [307:34:22<91:54:38,  4.94s/it] 
 70%|███████   | 157968/225000 [307:34:30<110:12:11,  5.92s/it]
 70%|███████   | 157969/225000 [307:34:38<121:25:58,  6.52s/it]
 70%|███████   | 157970/225000 [307:34:44<114:53:38,  6.17s/it]
 70%|███████   | 157971/225000 [307:34:50<115:21:17,  6.20s/it]
 70%|███████   | 157972/225000 [307:34:54<102:46:24,  5.52s/it]
 70%|███████   | 157973/225000 [307:34:58<96:13:03,  5.17s/it] 
 70%|███████   | 157974/225000 [307:35:02<91:12:03,  4.90s/it]
 70%|███████   | 157975/225000 [307:35:13<122:30:01,  6.58s/it]
 70%|███████   | 157976/225000 [307:35:16<105:29:39,  5.67s/it]
 70%|███████   | 157977/225000 [307:35:25<120:34:56,  6.48s/it]
 70%|███████   | 157978/225000 [307:35:28<103:55:03,  5.58s/it]
 70%|███████   | 157979/225000 [307:35:33<97:52:33,  5.26s/it] 
 70%|███████   | 157980/225000 [307:35:41<111:49:33,  6.01s/it]
 70%|███████   | 157981/225000 [307:35:45<103:06:38,  5.54s/it]
 70%|███████   | 157982/225000 [307:35:50<99:18:38,  5.33s/it] 
 70%|███████   | 157983/225000 [307:35:54<93:20:21,  5.01s/it]
 70%|███████   | 157984/225000 [307:35:59<90:15:11,  4.85s/it]
 70%|███████   | 157985/225000 [307:36:04<93:14:02,  5.01s/it]
 70%|███████   | 157986/225000 [307:36:17<135:50:26,  7.30s/it]
 70%|███████   | 157987/225000 [307:36:21<119:09:18,  6.40s/it]
 70%|███████   | 157988/225000 [307:36:26<112:17:07,  6.03s/it]
 70%|███████   | 157989/225000 [307:36:33<118:57:09,  6.39s/it]
 70%|███████   | 157990/225000 [307:36:37<103:16:31,  5.55s/it]
 70%|███████   | 157991/225000 [307:36:45<115:47:09,  6.22s/it]
 70%|███████   | 157992/225000 [307:36:48<100:52:05,  5.42s/it]
 70%|███████   | 157993/225000 [307:36:59<128:06:00,  6.88s/it]
 70%|███████   | 157994/225000 [307:37:03<112:35:01,  6.05s/it]
 70%|███████   | 157995/225000 [307:37:07<100:15:30,  5.39s/it]
 70%|███████   | 157996/225000 [307:37:13<105:11:43,  5.65s/it]
 70%|███████   | 157997/225000 [307:37:20<115:17:17,  6.19s/it]
 70%|███████   | 157998/225000 [307:37:24<101:25:27,  5.45s/it]
 70%|███████   | 157999/225000 [307:37:28<93:30:10,  5.02s/it] 
 70%|███████   | 158000/225000 [307:37:33<92:52:27,  4.99s/it]
                                                              
{'loss': '0.7826', 'grad_norm': '1.109', 'learning_rate': '1.108e-05', 'epoch': '0.7022', 'num_input_tokens_seen': 509294704, 'train_runtime': '1.107e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158000/225000 [307:37:33<92:52:27,  4.99s/it]
 70%|███████   | 158001/225000 [307:37:44<125:31:59,  6.75s/it]
 70%|███████   | 158002/225000 [307:37:47<104:19:45,  5.61s/it]
 70%|███████   | 158003/225000 [307:37:51<94:31:57,  5.08s/it] 
 70%|███████   | 158004/225000 [307:37:55<92:12:13,  4.95s/it]
 70%|███████   | 158005/225000 [307:38:00<89:59:14,  4.84s/it]
 70%|███████   | 158006/225000 [307:38:12<130:24:47,  7.01s/it]
 70%|███████   | 158007/225000 [307:38:15<111:05:56,  5.97s/it]
 70%|███████   | 158008/225000 [307:38:20<100:43:43,  5.41s/it]
 70%|███████   | 158009/225000 [307:38:24<95:32:38,  5.13s/it] 
 70%|███████   | 158010/225000 [307:38:36<131:56:07,  7.09s/it]
 70%|███████   | 158011/225000 [307:38:39<112:14:28,  6.03s/it]
 70%|███████   | 158012/225000 [307:38:47<122:51:44,  6.60s/it]
 70%|███████   | 158013/225000 [307:38:51<109:19:54,  5.88s/it]
 70%|███████   | 158014/225000 [307:38:55<97:09:07,  5.22s/it] 
 70%|███████   | 158015/225000 [307:39:02<107:39:33,  5.79s/it]
 70%|███████   | 158016/225000 [307:39:10<118:03:50,  6.35s/it]
 70%|███████   | 158017/225000 [307:39:14<105:20:22,  5.66s/it]
 70%|███████   | 158018/225000 [307:39:18<99:29:57,  5.35s/it] 
 70%|███████   | 158019/225000 [307:39:23<95:21:08,  5.12s/it]
 70%|███████   | 158020/225000 [307:39:28<92:08:55,  4.95s/it]
 70%|███████   | 158021/225000 [307:39:32<90:29:31,  4.86s/it]
 70%|███████   | 158022/225000 [307:39:42<116:12:56,  6.25s/it]
 70%|███████   | 158023/225000 [307:39:50<129:35:29,  6.97s/it]
 70%|███████   | 158024/225000 [307:39:54<112:50:24,  6.07s/it]
 70%|███████   | 158025/225000 [307:40:00<112:44:25,  6.06s/it]
 70%|███████   | 158026/225000 [307:40:07<114:19:11,  6.14s/it]
 70%|███████   | 158027/225000 [307:40:11<102:57:45,  5.53s/it]
 70%|███████   | 158028/225000 [307:40:16<101:10:17,  5.44s/it]
 70%|███████   | 158029/225000 [307:40:25<120:40:28,  6.49s/it]
 70%|███████   | 158030/225000 [307:40:30<112:01:09,  6.02s/it]
 70%|███████   | 158031/225000 [307:40:35<104:35:51,  5.62s/it]
 70%|███████   | 158032/225000 [307:40:39<100:04:17,  5.38s/it]
 70%|███████   | 158033/225000 [307:40:44<97:52:09,  5.26s/it] 
 70%|███████   | 158034/225000 [307:40:50<97:56:24,  5.27s/it]
 70%|███████   | 158035/225000 [307:40:55<96:44:48,  5.20s/it]
 70%|███████   | 158036/225000 [307:41:03<114:45:36,  6.17s/it]
 70%|███████   | 158037/225000 [307:41:12<129:23:15,  6.96s/it]
 70%|███████   | 158038/225000 [307:41:16<112:33:13,  6.05s/it]
 70%|███████   | 158039/225000 [307:41:25<126:53:37,  6.82s/it]
 70%|███████   | 158040/225000 [307:41:29<116:11:34,  6.25s/it]
 70%|███████   | 158041/225000 [307:41:37<122:13:40,  6.57s/it]
 70%|███████   | 158042/225000 [307:41:40<105:28:22,  5.67s/it]
 70%|███████   | 158043/225000 [307:41:45<99:19:56,  5.34s/it] 
 70%|███████   | 158044/225000 [307:41:50<96:25:55,  5.18s/it]
 70%|███████   | 158045/225000 [307:41:55<94:07:52,  5.06s/it]
 70%|███████   | 158046/225000 [307:41:59<93:14:36,  5.01s/it]
 70%|███████   | 158047/225000 [307:42:04<91:04:43,  4.90s/it]
 70%|███████   | 158048/225000 [307:42:09<92:37:04,  4.98s/it]
 70%|███████   | 158049/225000 [307:42:16<101:05:56,  5.44s/it]
 70%|███████   | 158050/225000 [307:42:28<136:40:24,  7.35s/it]
                                                               
{'loss': '0.7677', 'grad_norm': '1.232', 'learning_rate': '1.106e-05', 'epoch': '0.7024', 'num_input_tokens_seen': 509426064, 'train_runtime': '1.108e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158050/225000 [307:42:28<136:40:24,  7.35s/it]
 70%|███████   | 158051/225000 [307:42:33<128:57:31,  6.93s/it]
 70%|███████   | 158052/225000 [307:42:38<114:20:41,  6.15s/it]
 70%|███████   | 158053/225000 [307:42:42<104:05:49,  5.60s/it]
 70%|███████   | 158054/225000 [307:42:46<96:51:40,  5.21s/it] 
 70%|███████   | 158055/225000 [307:42:51<93:57:27,  5.05s/it]
 70%|███████   | 158056/225000 [307:42:56<92:42:44,  4.99s/it]
 70%|███████   | 158057/225000 [307:43:01<90:41:30,  4.88s/it]
 70%|███████   | 158058/225000 [307:43:08<106:05:51,  5.71s/it]
 70%|███████   | 158059/225000 [307:43:20<137:39:40,  7.40s/it]
 70%|███████   | 158060/225000 [307:43:23<115:17:50,  6.20s/it]
 70%|███████   | 158061/225000 [307:43:31<126:57:15,  6.83s/it]
 70%|███████   | 158062/225000 [307:43:34<106:55:35,  5.75s/it]
 70%|███████   | 158063/225000 [307:43:42<118:26:33,  6.37s/it]
 70%|███████   | 158064/225000 [307:43:46<102:04:46,  5.49s/it]
 70%|███████   | 158065/225000 [307:43:55<122:36:13,  6.59s/it]
 70%|███████   | 158066/225000 [307:43:59<110:37:54,  5.95s/it]
 70%|███████   | 158067/225000 [307:44:04<103:59:03,  5.59s/it]
 70%|███████   | 158068/225000 [307:44:09<100:46:00,  5.42s/it]
 70%|███████   | 158069/225000 [307:44:14<96:46:36,  5.21s/it] 
 70%|███████   | 158070/225000 [307:44:19<94:56:07,  5.11s/it]
 70%|███████   | 158071/225000 [307:44:24<94:51:37,  5.10s/it]
 70%|███████   | 158072/225000 [307:44:35<128:48:59,  6.93s/it]
 70%|███████   | 158073/225000 [307:44:40<119:29:53,  6.43s/it]
 70%|███████   | 158074/225000 [307:44:43<101:44:38,  5.47s/it]
 70%|███████   | 158075/225000 [307:44:52<117:36:19,  6.33s/it]
 70%|███████   | 158076/225000 [307:44:55<99:27:44,  5.35s/it] 
 70%|███████   | 158077/225000 [307:45:03<116:01:52,  6.24s/it]
 70%|███████   | 158078/225000 [307:45:08<108:55:26,  5.86s/it]
 70%|███████   | 158079/225000 [307:45:13<101:06:05,  5.44s/it]
 70%|███████   | 158080/225000 [307:45:17<95:55:49,  5.16s/it] 
 70%|███████   | 158081/225000 [307:45:22<92:35:14,  4.98s/it]
 70%|███████   | 158082/225000 [307:45:27<95:09:57,  5.12s/it]
 70%|███████   | 158083/225000 [307:45:39<132:41:54,  7.14s/it]
 70%|███████   | 158084/225000 [307:45:42<111:15:41,  5.99s/it]
 70%|███████   | 158085/225000 [307:45:50<122:37:33,  6.60s/it]
 70%|███████   | 158086/225000 [307:45:56<115:05:57,  6.19s/it]
 70%|███████   | 158087/225000 [307:46:01<113:24:00,  6.10s/it]
 70%|███████   | 158088/225000 [307:46:05<100:47:50,  5.42s/it]
 70%|███████   | 158089/225000 [307:46:10<95:44:13,  5.15s/it] 
 70%|███████   | 158090/225000 [307:46:19<115:26:59,  6.21s/it]
 70%|███████   | 158091/225000 [307:46:22<99:50:14,  5.37s/it] 
 70%|███████   | 158092/225000 [307:46:29<110:35:37,  5.95s/it]
 70%|███████   | 158093/225000 [307:46:33<96:30:18,  5.19s/it] 
 70%|███████   | 158094/225000 [307:46:41<112:55:25,  6.08s/it]
 70%|███████   | 158095/225000 [307:46:44<96:35:49,  5.20s/it] 
 70%|███████   | 158096/225000 [307:46:53<118:40:19,  6.39s/it]
 70%|███████   | 158097/225000 [307:46:56<101:13:37,  5.45s/it]
 70%|███████   | 158098/225000 [307:47:05<120:19:30,  6.47s/it]
 70%|███████   | 158099/225000 [307:47:09<105:13:50,  5.66s/it]
 70%|███████   | 158100/225000 [307:47:17<119:08:25,  6.41s/it]
                                                               
{'loss': '0.7853', 'grad_norm': '1.099', 'learning_rate': '1.105e-05', 'epoch': '0.7027', 'num_input_tokens_seen': 509555776, 'train_runtime': '1.108e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158100/225000 [307:47:17<119:08:25,  6.41s/it]
 70%|███████   | 158101/225000 [307:47:20<102:03:01,  5.49s/it]
 70%|███████   | 158102/225000 [307:47:29<121:32:42,  6.54s/it]
 70%|███████   | 158103/225000 [307:47:33<102:56:12,  5.54s/it]
 70%|███████   | 158104/225000 [307:47:36<90:39:26,  4.88s/it] 
 70%|███████   | 158105/225000 [307:47:46<116:43:39,  6.28s/it]
 70%|███████   | 158106/225000 [307:47:49<101:50:59,  5.48s/it]
 70%|███████   | 158107/225000 [307:47:54<100:51:15,  5.43s/it]
 70%|███████   | 158108/225000 [307:48:00<101:25:10,  5.46s/it]
 70%|███████   | 158109/225000 [307:48:05<100:53:15,  5.43s/it]
 70%|███████   | 158110/225000 [307:48:14<116:33:32,  6.27s/it]
 70%|███████   | 158111/225000 [307:48:17<99:38:58,  5.36s/it] 
 70%|███████   | 158112/225000 [307:48:22<97:05:51,  5.23s/it]
 70%|███████   | 158113/225000 [307:48:27<95:07:15,  5.12s/it]
 70%|███████   | 158114/225000 [307:48:31<92:30:39,  4.98s/it]
 70%|███████   | 158115/225000 [307:48:37<94:51:04,  5.11s/it]
 70%|███████   | 158116/225000 [307:48:42<94:24:40,  5.08s/it]
 70%|███████   | 158117/225000 [307:48:47<95:34:04,  5.14s/it]
 70%|███████   | 158118/225000 [307:48:52<94:56:34,  5.11s/it]
 70%|███████   | 158119/225000 [307:49:05<136:30:03,  7.35s/it]
 70%|███████   | 158120/225000 [307:49:12<137:40:28,  7.41s/it]
 70%|███████   | 158121/225000 [307:49:16<120:03:22,  6.46s/it]
 70%|███████   | 158122/225000 [307:49:21<112:22:29,  6.05s/it]
 70%|███████   | 158123/225000 [307:49:26<106:01:21,  5.71s/it]
 70%|███████   | 158124/225000 [307:49:32<104:11:29,  5.61s/it]
 70%|███████   | 158125/225000 [307:49:37<104:28:29,  5.62s/it]
 70%|███████   | 158126/225000 [307:49:43<102:19:31,  5.51s/it]
 70%|███████   | 158127/225000 [307:49:48<101:21:44,  5.46s/it]
 70%|███████   | 158128/225000 [307:49:53<99:16:41,  5.34s/it] 
 70%|███████   | 158129/225000 [307:49:59<100:42:48,  5.42s/it]
 70%|███████   | 158130/225000 [307:50:04<101:18:23,  5.45s/it]
 70%|███████   | 158131/225000 [307:50:09<100:02:50,  5.39s/it]
 70%|███████   | 158132/225000 [307:50:15<99:32:10,  5.36s/it] 
 70%|███████   | 158133/225000 [307:50:20<100:15:32,  5.40s/it]
 70%|███████   | 158134/225000 [307:50:26<101:41:10,  5.47s/it]
 70%|███████   | 158135/225000 [307:50:31<100:59:55,  5.44s/it]
 70%|███████   | 158136/225000 [307:50:37<100:54:14,  5.43s/it]
 70%|███████   | 158137/225000 [307:50:43<107:09:45,  5.77s/it]
 70%|███████   | 158138/225000 [307:50:56<146:40:48,  7.90s/it]
 70%|███████   | 158139/225000 [307:50:59<121:11:40,  6.53s/it]
 70%|███████   | 158140/225000 [307:51:07<128:07:44,  6.90s/it]
 70%|███████   | 158141/225000 [307:51:12<115:34:27,  6.22s/it]
 70%|███████   | 158142/225000 [307:51:16<104:24:29,  5.62s/it]
 70%|███████   | 158143/225000 [307:51:21<100:56:57,  5.44s/it]
 70%|███████   | 158144/225000 [307:51:26<97:48:21,  5.27s/it] 
 70%|███████   | 158145/225000 [307:51:30<93:36:30,  5.04s/it]
 70%|███████   | 158146/225000 [307:51:35<91:06:30,  4.91s/it]
 70%|███████   | 158147/225000 [307:51:48<133:57:50,  7.21s/it]
 70%|███████   | 158148/225000 [307:51:53<123:49:19,  6.67s/it]
 70%|███████   | 158149/225000 [307:51:57<106:53:53,  5.76s/it]
 70%|███████   | 158150/225000 [307:52:04<114:01:57,  6.14s/it]
                                                               
{'loss': '0.7734', 'grad_norm': '1.065', 'learning_rate': '1.103e-05', 'epoch': '0.7029', 'num_input_tokens_seen': 509686136, 'train_runtime': '1.108e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158150/225000 [307:52:04<114:01:57,  6.14s/it]
 70%|███████   | 158151/225000 [307:52:11<120:50:36,  6.51s/it]
 70%|███████   | 158152/225000 [307:52:15<104:18:40,  5.62s/it]
 70%|███████   | 158153/225000 [307:52:24<128:08:22,  6.90s/it]
 70%|███████   | 158154/225000 [307:52:28<110:05:25,  5.93s/it]
 70%|███████   | 158155/225000 [307:52:32<98:20:33,  5.30s/it] 
 70%|███████   | 158156/225000 [307:52:37<97:13:43,  5.24s/it]
 70%|███████   | 158157/225000 [307:52:42<94:51:46,  5.11s/it]
 70%|███████   | 158158/225000 [307:52:47<94:42:24,  5.10s/it]
 70%|███████   | 158159/225000 [307:52:52<95:12:31,  5.13s/it]
 70%|███████   | 158160/225000 [307:52:57<94:44:01,  5.10s/it]
 70%|███████   | 158161/225000 [307:53:02<94:05:30,  5.07s/it]
 70%|███████   | 158162/225000 [307:53:12<119:24:49,  6.43s/it]
 70%|███████   | 158163/225000 [307:53:22<139:11:22,  7.50s/it]
 70%|███████   | 158164/225000 [307:53:25<116:03:03,  6.25s/it]
 70%|███████   | 158165/225000 [307:53:34<129:12:20,  6.96s/it]
 70%|███████   | 158166/225000 [307:53:38<114:04:09,  6.14s/it]
 70%|███████   | 158167/225000 [307:53:44<111:51:55,  6.03s/it]
 70%|███████   | 158168/225000 [307:53:48<104:01:35,  5.60s/it]
 70%|███████   | 158169/225000 [307:53:56<116:47:07,  6.29s/it]
 70%|███████   | 158170/225000 [307:54:03<119:27:41,  6.44s/it]
 70%|███████   | 158171/225000 [307:54:07<106:05:10,  5.71s/it]
 70%|███████   | 158172/225000 [307:54:11<94:55:37,  5.11s/it] 
 70%|███████   | 158173/225000 [307:54:18<106:06:44,  5.72s/it]
 70%|███████   | 158174/225000 [307:54:23<101:16:04,  5.46s/it]
 70%|███████   | 158175/225000 [307:54:27<94:35:45,  5.10s/it] 
 70%|███████   | 158176/225000 [307:54:31<90:56:39,  4.90s/it]
 70%|███████   | 158177/225000 [307:54:42<120:32:37,  6.49s/it]
 70%|███████   | 158178/225000 [307:54:46<106:34:35,  5.74s/it]
 70%|███████   | 158179/225000 [307:54:49<95:25:05,  5.14s/it] 
 70%|███████   | 158180/225000 [307:54:57<109:33:27,  5.90s/it]
 70%|███████   | 158181/225000 [307:55:06<124:18:39,  6.70s/it]
 70%|███████   | 158182/225000 [307:55:10<109:39:52,  5.91s/it]
 70%|███████   | 158183/225000 [307:55:14<98:49:19,  5.32s/it] 
 70%|███████   | 158184/225000 [307:55:18<93:17:59,  5.03s/it]
 70%|███████   | 158185/225000 [307:55:23<90:40:27,  4.89s/it]
 70%|███████   | 158186/225000 [307:55:28<92:50:21,  5.00s/it]
 70%|███████   | 158187/225000 [307:55:40<134:51:40,  7.27s/it]
 70%|███████   | 158188/225000 [307:55:45<122:13:25,  6.59s/it]
 70%|███████   | 158189/225000 [307:55:49<103:53:30,  5.60s/it]
 70%|███████   | 158190/225000 [307:55:57<119:28:05,  6.44s/it]
 70%|███████   | 158191/225000 [307:56:01<105:37:34,  5.69s/it]
 70%|███████   | 158192/225000 [307:56:05<96:35:00,  5.20s/it] 
 70%|███████   | 158193/225000 [307:56:14<116:04:06,  6.25s/it]
 70%|███████   | 158194/225000 [307:56:18<107:04:47,  5.77s/it]
 70%|███████   | 158195/225000 [307:56:23<99:14:13,  5.35s/it] 
 70%|███████   | 158196/225000 [307:56:27<95:34:55,  5.15s/it]
 70%|███████   | 158197/225000 [307:56:35<109:14:13,  5.89s/it]
 70%|███████   | 158198/225000 [307:56:44<124:11:10,  6.69s/it]
 70%|███████   | 158199/225000 [307:56:47<106:14:14,  5.73s/it]
 70%|███████   | 158200/225000 [307:56:52<100:18:23,  5.41s/it]
                                                               
{'loss': '0.7793', 'grad_norm': '1.142', 'learning_rate': '1.102e-05', 'epoch': '0.7031', 'num_input_tokens_seen': 509817000, 'train_runtime': '1.109e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158200/225000 [307:56:52<100:18:23,  5.41s/it]
 70%|███████   | 158201/225000 [307:57:00<113:31:09,  6.12s/it]
 70%|███████   | 158202/225000 [307:57:03<98:54:30,  5.33s/it] 
 70%|███████   | 158203/225000 [307:57:11<112:39:29,  6.07s/it]
 70%|███████   | 158204/225000 [307:57:16<105:30:06,  5.69s/it]
 70%|███████   | 158205/225000 [307:57:20<100:32:08,  5.42s/it]
 70%|███████   | 158206/225000 [307:57:25<97:35:52,  5.26s/it] 
 70%|███████   | 158207/225000 [307:57:30<95:12:12,  5.13s/it]
 70%|███████   | 158208/225000 [307:57:35<91:26:41,  4.93s/it]
 70%|███████   | 158209/225000 [307:57:40<91:57:11,  4.96s/it]
 70%|███████   | 158210/225000 [307:57:52<135:23:49,  7.30s/it]
 70%|███████   | 158211/225000 [307:57:58<128:42:09,  6.94s/it]
 70%|███████   | 158212/225000 [307:58:04<121:12:56,  6.53s/it]
 70%|███████   | 158213/225000 [307:58:09<111:38:16,  6.02s/it]
 70%|███████   | 158214/225000 [307:58:14<108:42:12,  5.86s/it]
 70%|███████   | 158215/225000 [307:58:20<106:29:00,  5.74s/it]
 70%|███████   | 158216/225000 [307:58:25<103:14:16,  5.57s/it]
 70%|███████   | 158217/225000 [307:58:29<96:59:48,  5.23s/it] 
 70%|███████   | 158218/225000 [307:58:34<96:04:55,  5.18s/it]
 70%|███████   | 158219/225000 [307:58:40<97:53:04,  5.28s/it]
 70%|███████   | 158220/225000 [307:58:45<96:07:31,  5.18s/it]
 70%|███████   | 158221/225000 [307:58:50<97:26:39,  5.25s/it]
 70%|███████   | 158222/225000 [307:58:56<98:59:33,  5.34s/it]
 70%|███████   | 158223/225000 [307:59:01<98:09:11,  5.29s/it]
 70%|███████   | 158224/225000 [307:59:07<99:31:58,  5.37s/it]
 70%|███████   | 158225/225000 [307:59:12<100:53:37,  5.44s/it]
 70%|███████   | 158226/225000 [307:59:25<142:51:25,  7.70s/it]
 70%|███████   | 158227/225000 [307:59:31<134:52:37,  7.27s/it]
 70%|███████   | 158228/225000 [307:59:36<118:48:26,  6.41s/it]
 70%|███████   | 158229/225000 [307:59:40<107:45:21,  5.81s/it]
 70%|███████   | 158230/225000 [307:59:45<101:36:07,  5.48s/it]
 70%|███████   | 158231/225000 [307:59:49<96:10:55,  5.19s/it] 
 70%|███████   | 158232/225000 [307:59:54<94:20:03,  5.09s/it]
 70%|███████   | 158233/225000 [307:59:59<94:10:24,  5.08s/it]
 70%|███████   | 158234/225000 [308:00:06<101:10:46,  5.46s/it]
 70%|███████   | 158235/225000 [308:00:19<142:29:41,  7.68s/it]
 70%|███████   | 158236/225000 [308:00:23<125:39:34,  6.78s/it]
 70%|███████   | 158237/225000 [308:00:30<126:55:25,  6.84s/it]
 70%|███████   | 158238/225000 [308:00:34<110:06:18,  5.94s/it]
 70%|███████   | 158239/225000 [308:00:42<121:12:29,  6.54s/it]
 70%|███████   | 158240/225000 [308:00:45<102:51:18,  5.55s/it]
 70%|███████   | 158241/225000 [308:00:53<113:38:56,  6.13s/it]
 70%|███████   | 158242/225000 [308:00:58<106:35:41,  5.75s/it]
 70%|███████   | 158243/225000 [308:01:03<104:05:40,  5.61s/it]
 70%|███████   | 158244/225000 [308:01:08<99:48:24,  5.38s/it] 
 70%|███████   | 158245/225000 [308:01:13<98:02:37,  5.29s/it]
 70%|███████   | 158246/225000 [308:01:18<96:59:26,  5.23s/it]
 70%|███████   | 158247/225000 [308:01:23<94:16:20,  5.08s/it]
 70%|███████   | 158248/225000 [308:01:28<93:02:09,  5.02s/it]
 70%|███████   | 158249/225000 [308:01:33<94:32:03,  5.10s/it]
 70%|███████   | 158250/225000 [308:01:45<135:44:43,  7.32s/it]
                                                               
{'loss': '0.7787', 'grad_norm': '1.125', 'learning_rate': '1.1e-05', 'epoch': '0.7033', 'num_input_tokens_seen': 509946888, 'train_runtime': '1.109e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158250/225000 [308:01:45<135:44:43,  7.32s/it]
 70%|███████   | 158251/225000 [308:01:50<122:08:49,  6.59s/it]
 70%|███████   | 158252/225000 [308:01:54<105:08:43,  5.67s/it]
 70%|███████   | 158253/225000 [308:02:03<124:58:53,  6.74s/it]
 70%|███████   | 158254/225000 [308:02:07<107:43:44,  5.81s/it]
 70%|███████   | 158255/225000 [308:02:11<101:59:55,  5.50s/it]
 70%|███████   | 158256/225000 [308:02:19<113:30:14,  6.12s/it]
 70%|███████   | 158257/225000 [308:02:23<104:27:55,  5.63s/it]
 70%|███████   | 158258/225000 [308:02:29<104:02:07,  5.61s/it]
 70%|███████   | 158259/225000 [308:02:40<133:16:12,  7.19s/it]
 70%|███████   | 158260/225000 [308:02:44<113:47:41,  6.14s/it]
 70%|███████   | 158261/225000 [308:02:48<104:39:40,  5.65s/it]
 70%|███████   | 158262/225000 [308:02:53<101:20:46,  5.47s/it]
 70%|███████   | 158263/225000 [308:02:58<96:52:06,  5.23s/it] 
 70%|███████   | 158264/225000 [308:03:03<94:20:59,  5.09s/it]
 70%|███████   | 158265/225000 [308:03:15<135:23:55,  7.30s/it]
 70%|███████   | 158266/225000 [308:03:19<117:32:25,  6.34s/it]
 70%|███████   | 158267/225000 [308:03:26<121:23:49,  6.55s/it]
 70%|███████   | 158268/225000 [308:03:33<125:28:00,  6.77s/it]
 70%|███████   | 158269/225000 [308:03:37<107:02:00,  5.77s/it]
 70%|███████   | 158270/225000 [308:03:45<118:49:42,  6.41s/it]
 70%|███████   | 158271/225000 [308:03:50<110:47:16,  5.98s/it]
 70%|███████   | 158272/225000 [308:03:55<104:28:13,  5.64s/it]
 70%|███████   | 158273/225000 [308:04:00<100:28:19,  5.42s/it]
 70%|███████   | 158274/225000 [308:04:05<99:01:59,  5.34s/it] 
 70%|███████   | 158275/225000 [308:04:10<97:58:21,  5.29s/it]
 70%|███████   | 158276/225000 [308:04:15<95:33:20,  5.16s/it]
 70%|███████   | 158277/225000 [308:04:20<95:35:53,  5.16s/it]
 70%|███████   | 158278/225000 [308:04:30<125:29:36,  6.77s/it]
 70%|███████   | 158279/225000 [308:04:35<114:04:10,  6.15s/it]
 70%|███████   | 158280/225000 [308:04:40<107:15:40,  5.79s/it]
 70%|███████   | 158281/225000 [308:04:48<122:10:46,  6.59s/it]
 70%|███████   | 158282/225000 [308:04:53<111:43:19,  6.03s/it]
 70%|███████   | 158283/225000 [308:04:58<105:48:35,  5.71s/it]
 70%|███████   | 158284/225000 [308:05:03<101:57:48,  5.50s/it]
 70%|███████   | 158285/225000 [308:05:08<100:45:41,  5.44s/it]
 70%|███████   | 158286/225000 [308:05:13<98:03:17,  5.29s/it] 
 70%|███████   | 158287/225000 [308:05:18<94:29:10,  5.10s/it]
 70%|███████   | 158288/225000 [308:05:23<92:54:32,  5.01s/it]
 70%|███████   | 158289/225000 [308:05:33<120:15:13,  6.49s/it]
 70%|███████   | 158290/225000 [308:05:43<140:59:44,  7.61s/it]
 70%|███████   | 158291/225000 [308:05:47<118:47:39,  6.41s/it]
 70%|███████   | 158292/225000 [308:05:50<104:03:48,  5.62s/it]
 70%|███████   | 158293/225000 [308:05:57<110:47:36,  5.98s/it]
 70%|███████   | 158294/225000 [308:06:05<120:35:24,  6.51s/it]
 70%|███████   | 158295/225000 [308:06:09<104:09:53,  5.62s/it]
 70%|███████   | 158296/225000 [308:06:18<123:16:15,  6.65s/it]
 70%|███████   | 158297/225000 [308:06:22<112:23:28,  6.07s/it]
 70%|███████   | 158298/225000 [308:06:27<104:02:14,  5.62s/it]
 70%|███████   | 158299/225000 [308:06:31<97:48:56,  5.28s/it] 
 70%|███████   | 158300/225000 [308:06:36<93:31:25,  5.05s/it]
                                                              
{'loss': '0.7752', 'grad_norm': '1.184', 'learning_rate': '1.099e-05', 'epoch': '0.7036', 'num_input_tokens_seen': 510078120, 'train_runtime': '1.109e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158300/225000 [308:06:36<93:31:25,  5.05s/it]
 70%|███████   | 158301/225000 [308:06:41<92:29:16,  4.99s/it]
 70%|███████   | 158302/225000 [308:06:46<93:38:29,  5.05s/it]
 70%|███████   | 158303/225000 [308:06:51<92:55:08,  5.02s/it]
 70%|███████   | 158304/225000 [308:06:56<93:46:36,  5.06s/it]
 70%|███████   | 158305/225000 [308:07:01<94:39:28,  5.11s/it]
 70%|███████   | 158306/225000 [308:07:08<102:53:37,  5.55s/it]
 70%|███████   | 158307/225000 [308:07:20<138:34:28,  7.48s/it]
 70%|███████   | 158308/225000 [308:07:25<127:02:58,  6.86s/it]
 70%|███████   | 158309/225000 [308:07:29<111:32:44,  6.02s/it]
 70%|███████   | 158310/225000 [308:07:34<103:01:07,  5.56s/it]
 70%|███████   | 158311/225000 [308:07:38<95:27:23,  5.15s/it] 
 70%|███████   | 158312/225000 [308:07:44<99:47:21,  5.39s/it]
 70%|███████   | 158313/225000 [308:07:54<123:56:04,  6.69s/it]
 70%|███████   | 158314/225000 [308:07:58<109:48:00,  5.93s/it]
 70%|███████   | 158315/225000 [308:08:02<99:00:23,  5.34s/it] 
 70%|███████   | 158316/225000 [308:08:10<115:35:32,  6.24s/it]
 70%|███████   | 158317/225000 [308:08:16<112:54:12,  6.10s/it]
 70%|███████   | 158318/225000 [308:08:21<108:05:07,  5.84s/it]
 70%|███████   | 158319/225000 [308:08:28<115:48:26,  6.25s/it]
 70%|███████   | 158320/225000 [308:08:32<100:26:17,  5.42s/it]
 70%|███████   | 158321/225000 [308:08:42<125:16:07,  6.76s/it]
 70%|███████   | 158322/225000 [308:08:45<107:35:02,  5.81s/it]
 70%|███████   | 158323/225000 [308:08:50<102:31:04,  5.54s/it]
 70%|███████   | 158324/225000 [308:08:58<115:23:11,  6.23s/it]
 70%|███████   | 158325/225000 [308:09:02<102:53:15,  5.56s/it]
 70%|███████   | 158326/225000 [308:09:06<94:43:21,  5.11s/it] 
 70%|███████   | 158327/225000 [308:09:15<116:53:56,  6.31s/it]
 70%|███████   | 158328/225000 [308:09:20<110:21:31,  5.96s/it]
 70%|███████   | 158329/225000 [308:09:26<106:14:39,  5.74s/it]
 70%|███████   | 158330/225000 [308:09:31<102:47:56,  5.55s/it]
 70%|███████   | 158331/225000 [308:09:36<99:11:37,  5.36s/it] 
 70%|███████   | 158332/225000 [308:09:40<94:15:29,  5.09s/it]
 70%|███████   | 158333/225000 [308:09:45<94:41:00,  5.11s/it]
 70%|███████   | 158334/225000 [308:09:50<92:35:15,  5.00s/it]
 70%|███████   | 158335/225000 [308:09:55<94:00:41,  5.08s/it]
 70%|███████   | 158336/225000 [308:10:02<102:19:36,  5.53s/it]
 70%|███████   | 158337/225000 [308:10:14<140:48:26,  7.60s/it]
 70%|███████   | 158338/225000 [308:10:18<120:33:15,  6.51s/it]
 70%|███████   | 158339/225000 [308:10:23<111:04:27,  6.00s/it]
 70%|███████   | 158340/225000 [308:10:28<103:32:39,  5.59s/it]
 70%|███████   | 158341/225000 [308:10:32<96:53:44,  5.23s/it] 
 70%|███████   | 158342/225000 [308:10:37<94:05:07,  5.08s/it]
 70%|███████   | 158343/225000 [308:10:42<92:19:11,  4.99s/it]
 70%|███████   | 158344/225000 [308:10:50<111:20:34,  6.01s/it]
 70%|███████   | 158345/225000 [308:10:59<130:11:19,  7.03s/it]
 70%|███████   | 158346/225000 [308:11:03<112:59:12,  6.10s/it]
 70%|███████   | 158347/225000 [308:11:09<110:03:58,  5.94s/it]
 70%|███████   | 158348/225000 [308:11:18<127:21:18,  6.88s/it]
 70%|███████   | 158349/225000 [308:11:22<110:31:46,  5.97s/it]
 70%|███████   | 158350/225000 [308:11:25<98:03:55,  5.30s/it] 
                                                              
{'loss': '0.7765', 'grad_norm': '1.138', 'learning_rate': '1.097e-05', 'epoch': '0.7038', 'num_input_tokens_seen': 510209528, 'train_runtime': '1.109e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158350/225000 [308:11:25<98:03:55,  5.30s/it]
 70%|███████   | 158351/225000 [308:11:33<112:03:27,  6.05s/it]
 70%|███████   | 158352/225000 [308:11:39<110:21:44,  5.96s/it]
 70%|███████   | 158353/225000 [308:11:46<116:05:01,  6.27s/it]
 70%|███████   | 158354/225000 [308:11:51<109:25:35,  5.91s/it]
 70%|███████   | 158355/225000 [308:11:56<103:41:28,  5.60s/it]
 70%|███████   | 158356/225000 [308:12:01<99:12:09,  5.36s/it] 
 70%|███████   | 158357/225000 [308:12:06<97:26:02,  5.26s/it]
 70%|███████   | 158358/225000 [308:12:11<96:18:00,  5.20s/it]
 70%|███████   | 158359/225000 [308:12:16<93:46:26,  5.07s/it]
 70%|███████   | 158360/225000 [308:12:21<93:26:24,  5.05s/it]
 70%|███████   | 158361/225000 [308:12:26<97:03:14,  5.24s/it]
 70%|███████   | 158362/225000 [308:12:39<136:05:37,  7.35s/it]
 70%|███████   | 158363/225000 [308:12:43<118:05:01,  6.38s/it]
 70%|███████   | 158364/225000 [308:12:49<116:16:26,  6.28s/it]
 70%|███████   | 158365/225000 [308:12:53<105:12:58,  5.68s/it]
 70%|███████   | 158366/225000 [308:12:58<98:53:58,  5.34s/it] 
 70%|███████   | 158367/225000 [308:13:02<93:40:06,  5.06s/it]
 70%|███████   | 158368/225000 [308:13:07<93:47:09,  5.07s/it]
 70%|███████   | 158369/225000 [308:13:18<128:40:40,  6.95s/it]
 70%|███████   | 158370/225000 [308:13:22<111:45:44,  6.04s/it]
 70%|███████   | 158371/225000 [308:13:29<116:28:11,  6.29s/it]
 70%|███████   | 158372/225000 [308:13:35<112:59:00,  6.10s/it]
 70%|███████   | 158373/225000 [308:13:42<118:55:28,  6.43s/it]
 70%|███████   | 158374/225000 [308:13:46<103:27:32,  5.59s/it]
 70%|███████   | 158375/225000 [308:13:49<93:24:55,  5.05s/it] 
 70%|███████   | 158376/225000 [308:13:55<97:30:59,  5.27s/it]
 70%|███████   | 158377/225000 [308:14:02<106:51:45,  5.77s/it]
 70%|███████   | 158378/225000 [308:14:07<102:26:59,  5.54s/it]
 70%|███████   | 158379/225000 [308:14:12<97:58:13,  5.29s/it] 
 70%|███████   | 158380/225000 [308:14:17<94:02:26,  5.08s/it]
 70%|███████   | 158381/225000 [308:14:22<94:24:34,  5.10s/it]
 70%|███████   | 158382/225000 [308:14:27<94:08:24,  5.09s/it]
 70%|███████   | 158383/225000 [308:14:32<95:45:20,  5.17s/it]
 70%|███████   | 158384/225000 [308:14:45<138:56:55,  7.51s/it]
 70%|███████   | 158385/225000 [308:14:50<124:43:55,  6.74s/it]
 70%|███████   | 158386/225000 [308:14:55<114:31:44,  6.19s/it]
 70%|███████   | 158387/225000 [308:15:00<108:26:39,  5.86s/it]
 70%|███████   | 158388/225000 [308:15:05<103:19:45,  5.58s/it]
 70%|███████   | 158389/225000 [308:15:10<101:23:48,  5.48s/it]
 70%|███████   | 158390/225000 [308:15:15<99:30:37,  5.38s/it] 
 70%|███████   | 158391/225000 [308:15:20<97:14:23,  5.26s/it]
 70%|███████   | 158392/225000 [308:15:30<121:50:39,  6.59s/it]
 70%|███████   | 158393/225000 [308:15:35<114:45:01,  6.20s/it]
 70%|███████   | 158394/225000 [308:15:39<101:49:17,  5.50s/it]
 70%|███████   | 158395/225000 [308:15:44<99:14:54,  5.36s/it] 
 70%|███████   | 158396/225000 [308:15:50<99:23:54,  5.37s/it]
 70%|███████   | 158397/225000 [308:15:54<96:19:59,  5.21s/it]
 70%|███████   | 158398/225000 [308:16:00<96:15:45,  5.20s/it]
 70%|███████   | 158399/225000 [308:16:05<96:09:06,  5.20s/it]
 70%|███████   | 158400/225000 [308:16:10<97:31:53,  5.27s/it]
                                                              
{'loss': '0.7821', 'grad_norm': '1.03', 'learning_rate': '1.095e-05', 'epoch': '0.704', 'num_input_tokens_seen': 510339768, 'train_runtime': '1.11e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158400/225000 [308:16:10<97:31:53,  5.27s/it]
 70%|███████   | 158401/225000 [308:16:16<98:51:07,  5.34s/it]
 70%|███████   | 158402/225000 [308:16:21<97:17:30,  5.26s/it]
 70%|███████   | 158403/225000 [308:16:26<99:09:08,  5.36s/it]
 70%|███████   | 158404/225000 [308:16:32<100:11:49,  5.42s/it]
 70%|███████   | 158405/225000 [308:16:37<97:38:02,  5.28s/it] 
 70%|███████   | 158406/225000 [308:16:42<97:46:09,  5.29s/it]
 70%|███████   | 158407/225000 [308:16:54<131:26:36,  7.11s/it]
 70%|███████   | 158408/225000 [308:17:01<133:48:17,  7.23s/it]
 70%|███████   | 158409/225000 [308:17:05<112:50:15,  6.10s/it]
 70%|███████   | 158410/225000 [308:17:11<117:22:26,  6.35s/it]
 70%|███████   | 158411/225000 [308:17:15<100:19:06,  5.42s/it]
 70%|███████   | 158412/225000 [308:17:26<130:04:28,  7.03s/it]
 70%|███████   | 158413/225000 [308:17:29<112:38:23,  6.09s/it]
 70%|███████   | 158414/225000 [308:17:33<100:20:54,  5.43s/it]
 70%|███████   | 158415/225000 [308:17:39<100:10:31,  5.42s/it]
 70%|███████   | 158416/225000 [308:17:48<119:47:26,  6.48s/it]
 70%|███████   | 158417/225000 [308:17:52<109:19:04,  5.91s/it]
 70%|███████   | 158418/225000 [308:17:57<101:48:06,  5.50s/it]
 70%|███████   | 158419/225000 [308:18:01<96:58:30,  5.24s/it] 
 70%|███████   | 158420/225000 [308:18:06<91:04:31,  4.92s/it]
 70%|███████   | 158421/225000 [308:18:10<87:34:39,  4.74s/it]
 70%|███████   | 158422/225000 [308:18:15<89:44:35,  4.85s/it]
 70%|███████   | 158423/225000 [308:18:20<90:28:26,  4.89s/it]
 70%|███████   | 158424/225000 [308:18:28<105:49:06,  5.72s/it]
 70%|███████   | 158425/225000 [308:18:40<144:58:09,  7.84s/it]
 70%|███████   | 158426/225000 [308:18:44<121:15:08,  6.56s/it]
 70%|███████   | 158427/225000 [308:18:49<115:24:05,  6.24s/it]
 70%|███████   | 158428/225000 [308:18:58<130:06:30,  7.04s/it]
 70%|███████   | 158429/225000 [308:19:02<110:02:40,  5.95s/it]
 70%|███████   | 158430/225000 [308:19:06<100:21:58,  5.43s/it]
 70%|███████   | 158431/225000 [308:19:10<93:08:20,  5.04s/it] 
 70%|███████   | 158432/225000 [308:19:15<90:55:29,  4.92s/it]
 70%|███████   | 158433/225000 [308:19:26<126:59:45,  6.87s/it]
 70%|███████   | 158434/225000 [308:19:31<113:48:09,  6.15s/it]
 70%|███████   | 158435/225000 [308:19:35<105:12:07,  5.69s/it]
 70%|███████   | 158436/225000 [308:19:43<116:32:39,  6.30s/it]
 70%|███████   | 158437/225000 [308:19:48<110:55:48,  6.00s/it]
 70%|███████   | 158438/225000 [308:19:53<102:50:56,  5.56s/it]
 70%|███████   | 158439/225000 [308:19:57<96:15:22,  5.21s/it] 
 70%|███████   | 158440/225000 [308:20:03<98:24:31,  5.32s/it]
 70%|███████   | 158441/225000 [308:20:12<119:06:15,  6.44s/it]
 70%|███████   | 158442/225000 [308:20:16<107:31:58,  5.82s/it]
 70%|███████   | 158443/225000 [308:20:20<96:50:31,  5.24s/it] 
 70%|███████   | 158444/225000 [308:20:30<124:26:18,  6.73s/it]
 70%|███████   | 158445/225000 [308:20:34<108:14:48,  5.86s/it]
 70%|███████   | 158446/225000 [308:20:41<113:00:15,  6.11s/it]
 70%|███████   | 158447/225000 [308:20:45<99:18:54,  5.37s/it] 
 70%|███████   | 158448/225000 [308:20:54<123:46:15,  6.70s/it]
 70%|███████   | 158449/225000 [308:20:59<114:42:23,  6.20s/it]
 70%|███████   | 158450/225000 [308:21:04<105:42:14,  5.72s/it]
                                                               
{'loss': '0.7728', 'grad_norm': '1.156', 'learning_rate': '1.094e-05', 'epoch': '0.7042', 'num_input_tokens_seen': 510469384, 'train_runtime': '1.11e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158450/225000 [308:21:04<105:42:14,  5.72s/it]
 70%|███████   | 158451/225000 [308:21:09<100:31:38,  5.44s/it]
 70%|███████   | 158452/225000 [308:21:14<98:50:25,  5.35s/it] 
 70%|███████   | 158453/225000 [308:21:19<96:38:50,  5.23s/it]
 70%|███████   | 158454/225000 [308:21:24<96:16:35,  5.21s/it]
 70%|███████   | 158455/225000 [308:21:29<94:36:21,  5.12s/it]
 70%|███████   | 158456/225000 [308:21:34<93:29:25,  5.06s/it]
 70%|███████   | 158457/225000 [308:21:39<92:18:18,  4.99s/it]
 70%|███████   | 158458/225000 [308:21:44<91:49:54,  4.97s/it]
 70%|███████   | 158459/225000 [308:21:51<102:51:04,  5.56s/it]
 70%|███████   | 158460/225000 [308:22:03<139:34:19,  7.55s/it]
 70%|███████   | 158461/225000 [308:22:06<116:36:43,  6.31s/it]
 70%|███████   | 158462/225000 [308:22:11<108:03:03,  5.85s/it]
 70%|███████   | 158463/225000 [308:22:16<105:01:32,  5.68s/it]
 70%|███████   | 158464/225000 [308:22:26<126:40:41,  6.85s/it]
 70%|███████   | 158465/225000 [308:22:30<110:25:12,  5.97s/it]
 70%|███████   | 158466/225000 [308:22:36<113:52:29,  6.16s/it]
 70%|███████   | 158467/225000 [308:22:41<107:45:11,  5.83s/it]
 70%|███████   | 158468/225000 [308:22:49<115:53:03,  6.27s/it]
 70%|███████   | 158469/225000 [308:22:54<108:25:05,  5.87s/it]
 70%|███████   | 158470/225000 [308:22:58<102:59:52,  5.57s/it]
 70%|███████   | 158471/225000 [308:23:03<98:50:56,  5.35s/it] 
 70%|███████   | 158472/225000 [308:23:08<96:31:57,  5.22s/it]
 70%|███████   | 158473/225000 [308:23:14<97:38:13,  5.28s/it]
 70%|███████   | 158474/225000 [308:23:18<95:02:48,  5.14s/it]
 70%|███████   | 158475/225000 [308:23:24<94:53:35,  5.14s/it]
 70%|███████   | 158476/225000 [308:23:28<90:58:25,  4.92s/it]
 70%|███████   | 158477/225000 [308:23:33<93:42:21,  5.07s/it]
 70%|███████   | 158478/225000 [308:23:46<132:51:28,  7.19s/it]
 70%|███████   | 158479/225000 [308:23:51<120:47:59,  6.54s/it]
 70%|███████   | 158480/225000 [308:23:55<106:38:11,  5.77s/it]
 70%|███████   | 158481/225000 [308:23:59<97:52:22,  5.30s/it] 
 70%|███████   | 158482/225000 [308:24:10<129:58:02,  7.03s/it]
 70%|███████   | 158483/225000 [308:24:14<111:39:58,  6.04s/it]
 70%|███████   | 158484/225000 [308:24:22<123:37:00,  6.69s/it]
 70%|███████   | 158485/225000 [308:24:25<106:02:42,  5.74s/it]
 70%|███████   | 158486/225000 [308:24:29<96:31:27,  5.22s/it] 
 70%|███████   | 158487/225000 [308:24:39<121:22:46,  6.57s/it]
 70%|███████   | 158488/225000 [308:24:42<103:44:55,  5.62s/it]
 70%|███████   | 158489/225000 [308:24:46<94:26:35,  5.11s/it] 
 70%|███████   | 158490/225000 [308:24:57<124:15:29,  6.73s/it]
 70%|███████   | 158491/225000 [308:25:00<105:23:37,  5.70s/it]
 70%|███████   | 158492/225000 [308:25:04<96:13:53,  5.21s/it] 
 70%|███████   | 158493/225000 [308:25:08<90:45:52,  4.91s/it]
 70%|███████   | 158494/225000 [308:25:18<117:38:56,  6.37s/it]
 70%|███████   | 158495/225000 [308:25:23<110:45:51,  6.00s/it]
 70%|███████   | 158496/225000 [308:25:27<99:50:31,  5.40s/it] 
 70%|███████   | 158497/225000 [308:25:32<94:24:26,  5.11s/it]
 70%|███████   | 158498/225000 [308:25:37<94:44:36,  5.13s/it]
 70%|███████   | 158499/225000 [308:25:42<94:51:59,  5.14s/it]
 70%|███████   | 158500/225000 [308:25:48<97:03:31,  5.25s/it]
                                                              
{'loss': '0.7746', 'grad_norm': '1.037', 'learning_rate': '1.092e-05', 'epoch': '0.7044', 'num_input_tokens_seen': 510600640, 'train_runtime': '1.11e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158500/225000 [308:25:48<97:03:31,  5.25s/it]
 70%|███████   | 158501/225000 [308:25:53<96:36:38,  5.23s/it]
 70%|███████   | 158502/225000 [308:25:57<93:26:33,  5.06s/it]
 70%|███████   | 158503/225000 [308:26:09<132:11:55,  7.16s/it]
 70%|███████   | 158504/225000 [308:26:15<125:36:32,  6.80s/it]
 70%|███████   | 158505/225000 [308:26:20<112:11:34,  6.07s/it]
 70%|███████   | 158506/225000 [308:26:25<106:27:46,  5.76s/it]
 70%|███████   | 158507/225000 [308:26:30<102:20:56,  5.54s/it]
 70%|███████   | 158508/225000 [308:26:35<100:40:58,  5.45s/it]
 70%|███████   | 158509/225000 [308:26:40<97:29:52,  5.28s/it] 
 70%|███████   | 158510/225000 [308:26:45<93:59:55,  5.09s/it]
 70%|███████   | 158511/225000 [308:26:50<93:59:24,  5.09s/it]
 70%|███████   | 158512/225000 [308:26:55<94:21:21,  5.11s/it]
 70%|███████   | 158513/225000 [308:27:00<94:20:08,  5.11s/it]
 70%|███████   | 158514/225000 [308:27:12<130:28:19,  7.06s/it]
 70%|███████   | 158515/225000 [308:27:18<126:49:48,  6.87s/it]
 70%|███████   | 158516/225000 [308:27:22<110:21:28,  5.98s/it]
 70%|███████   | 158517/225000 [308:27:26<99:44:41,  5.40s/it] 
 70%|███████   | 158518/225000 [308:27:30<91:07:00,  4.93s/it]
 70%|███████   | 158519/225000 [308:27:40<122:09:43,  6.62s/it]
 70%|███████   | 158520/225000 [308:27:47<124:04:21,  6.72s/it]
 70%|███████   | 158521/225000 [308:27:51<105:50:20,  5.73s/it]
 70%|███████   | 158522/225000 [308:28:00<125:07:11,  6.78s/it]
 70%|███████   | 158523/225000 [308:28:04<110:51:04,  6.00s/it]
 70%|███████   | 158524/225000 [308:28:08<97:46:48,  5.30s/it] 
 70%|███████   | 158525/225000 [308:28:15<108:07:36,  5.86s/it]
 70%|███████   | 158526/225000 [308:28:23<118:16:02,  6.40s/it]
 70%|███████   | 158527/225000 [308:28:27<105:46:46,  5.73s/it]
 70%|███████   | 158528/225000 [308:28:31<95:51:00,  5.19s/it] 
 70%|███████   | 158529/225000 [308:28:41<124:03:36,  6.72s/it]
 70%|███████   | 158530/225000 [308:28:46<115:36:27,  6.26s/it]
 70%|███████   | 158531/225000 [308:28:54<122:59:21,  6.66s/it]
 70%|███████   | 158532/225000 [308:28:58<108:50:05,  5.89s/it]
 70%|███████   | 158533/225000 [308:29:02<100:48:54,  5.46s/it]
 70%|███████   | 158534/225000 [308:29:07<93:40:55,  5.07s/it] 
 70%|███████   | 158535/225000 [308:29:11<89:54:41,  4.87s/it]
 70%|███████   | 158536/225000 [308:29:22<124:37:36,  6.75s/it]
 70%|███████   | 158537/225000 [308:29:26<111:25:16,  6.04s/it]
 70%|███████   | 158538/225000 [308:29:30<95:16:58,  5.16s/it] 
 70%|███████   | 158539/225000 [308:29:35<96:34:20,  5.23s/it]
 70%|███████   | 158540/225000 [308:29:40<96:43:31,  5.24s/it]
 70%|███████   | 158541/225000 [308:29:45<96:38:11,  5.23s/it]
 70%|███████   | 158542/225000 [308:29:51<99:11:20,  5.37s/it]
 70%|███████   | 158543/225000 [308:29:57<101:10:38,  5.48s/it]
 70%|███████   | 158544/225000 [308:30:03<105:31:16,  5.72s/it]
 70%|███████   | 158545/225000 [308:30:13<127:43:37,  6.92s/it]
 70%|███████   | 158546/225000 [308:30:17<113:21:36,  6.14s/it]
 70%|███████   | 158547/225000 [308:30:23<113:19:07,  6.14s/it]
 70%|███████   | 158548/225000 [308:30:27<98:47:58,  5.35s/it] 
 70%|███████   | 158549/225000 [308:30:36<120:34:10,  6.53s/it]
 70%|███████   | 158550/225000 [308:30:41<109:02:07,  5.91s/it]
                                                               
{'loss': '0.784', 'grad_norm': '1.024', 'learning_rate': '1.091e-05', 'epoch': '0.7047', 'num_input_tokens_seen': 510731168, 'train_runtime': '1.111e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158550/225000 [308:30:41<109:02:07,  5.91s/it]
 70%|███████   | 158551/225000 [308:30:45<101:57:46,  5.52s/it]
 70%|███████   | 158552/225000 [308:30:50<98:30:24,  5.34s/it] 
 70%|███████   | 158553/225000 [308:30:55<93:30:26,  5.07s/it]
 70%|███████   | 158554/225000 [308:30:59<91:37:22,  4.96s/it]
 70%|███████   | 158555/225000 [308:31:05<93:09:45,  5.05s/it]
 70%|███████   | 158556/225000 [308:31:10<95:16:13,  5.16s/it]
 70%|███████   | 158557/225000 [308:31:15<95:07:59,  5.15s/it]
 70%|███████   | 158558/225000 [308:31:20<95:23:13,  5.17s/it]
 70%|███████   | 158559/225000 [308:31:25<94:02:26,  5.10s/it]
 70%|███████   | 158560/225000 [308:31:31<95:08:11,  5.15s/it]
 70%|███████   | 158561/225000 [308:31:37<103:19:57,  5.60s/it]
 70%|███████   | 158562/225000 [308:31:50<141:28:25,  7.67s/it]
 70%|███████   | 158563/225000 [308:31:54<120:34:55,  6.53s/it]
 70%|███████   | 158564/225000 [308:32:02<131:36:56,  7.13s/it]
 70%|███████   | 158565/225000 [308:32:06<114:06:30,  6.18s/it]
 70%|███████   | 158566/225000 [308:32:11<106:10:17,  5.75s/it]
 70%|███████   | 158567/225000 [308:32:16<101:11:37,  5.48s/it]
 70%|███████   | 158568/225000 [308:32:20<96:45:10,  5.24s/it] 
 70%|███████   | 158569/225000 [308:32:25<96:14:13,  5.22s/it]
 70%|███████   | 158570/225000 [308:32:31<97:21:47,  5.28s/it]
 70%|███████   | 158571/225000 [308:32:36<93:51:12,  5.09s/it]
 70%|███████   | 158572/225000 [308:32:41<93:49:14,  5.08s/it]
 70%|███████   | 158573/225000 [308:32:46<96:06:54,  5.21s/it]
 70%|███████   | 158574/225000 [308:32:52<97:09:35,  5.27s/it]
 70%|███████   | 158575/225000 [308:32:56<95:01:18,  5.15s/it]
 70%|███████   | 158576/225000 [308:33:01<94:49:55,  5.14s/it]
 70%|███████   | 158577/225000 [308:33:13<130:53:08,  7.09s/it]
 70%|███████   | 158578/225000 [308:33:20<127:52:36,  6.93s/it]
 70%|███████   | 158579/225000 [308:33:23<109:41:36,  5.95s/it]
 70%|███████   | 158580/225000 [308:33:33<127:31:38,  6.91s/it]
 70%|███████   | 158581/225000 [308:33:36<108:23:26,  5.87s/it]
 70%|███████   | 158582/225000 [308:33:43<114:24:03,  6.20s/it]
 70%|███████   | 158583/225000 [308:33:49<113:42:17,  6.16s/it]
 70%|███████   | 158584/225000 [308:33:54<109:14:43,  5.92s/it]
 70%|███████   | 158585/225000 [308:33:59<101:02:04,  5.48s/it]
 70%|███████   | 158586/225000 [308:34:03<94:36:33,  5.13s/it] 
 70%|███████   | 158587/225000 [308:34:07<89:59:47,  4.88s/it]
 70%|███████   | 158588/225000 [308:34:18<119:57:00,  6.50s/it]
 70%|███████   | 158589/225000 [308:34:26<128:02:00,  6.94s/it]
 70%|███████   | 158590/225000 [308:34:30<114:17:47,  6.20s/it]
 70%|███████   | 158591/225000 [308:34:34<102:46:59,  5.57s/it]
 70%|███████   | 158592/225000 [308:34:39<95:58:42,  5.20s/it] 
 70%|███████   | 158593/225000 [308:34:43<91:09:28,  4.94s/it]
 70%|███████   | 158594/225000 [308:34:55<130:25:56,  7.07s/it]
 70%|███████   | 158595/225000 [308:34:59<113:45:42,  6.17s/it]
 70%|███████   | 158596/225000 [308:35:07<123:19:39,  6.69s/it]
 70%|███████   | 158597/225000 [308:35:11<107:17:19,  5.82s/it]
 70%|███████   | 158598/225000 [308:35:14<96:01:40,  5.21s/it] 
 70%|███████   | 158599/225000 [308:35:19<93:31:35,  5.07s/it]
 70%|███████   | 158600/225000 [308:35:24<94:29:00,  5.12s/it]
                                                              
{'loss': '0.7815', 'grad_norm': '1.175', 'learning_rate': '1.089e-05', 'epoch': '0.7049', 'num_input_tokens_seen': 510862200, 'train_runtime': '1.111e+06', 'train_tokens_per_second': '459.9'}

 70%|███████   | 158600/225000 [308:35:24<94:29:00,  5.12s/it]
 70%|███████   | 158601/225000 [308:35:30<95:30:07,  5.18s/it]
 70%|███████   | 158602/225000 [308:35:35<97:20:51,  5.28s/it]
 70%|███████   | 158603/225000 [308:35:43<112:01:54,  6.07s/it]
 70%|███████   | 158604/225000 [308:35:49<108:18:16,  5.87s/it]
 70%|███████   | 158605/225000 [308:35:54<105:30:16,  5.72s/it]
 70%|███████   | 158606/225000 [308:36:00<108:58:58,  5.91s/it]
 70%|███████   | 158607/225000 [308:36:10<128:48:30,  6.98s/it]
 70%|███████   | 158608/225000 [308:36:14<110:43:36,  6.00s/it]
 70%|███████   | 158609/225000 [308:36:17<98:34:11,  5.34s/it] 
 70%|███████   | 158610/225000 [308:36:26<117:09:04,  6.35s/it]
 70%|███████   | 158611/225000 [308:36:31<111:57:47,  6.07s/it]
 70%|███████   | 158612/225000 [308:36:37<109:01:16,  5.91s/it]
 70%|███████   | 158613/225000 [308:36:42<104:35:54,  5.67s/it]
 70%|███████   | 158614/225000 [308:36:46<94:17:42,  5.11s/it] 
 70%|███████   | 158615/225000 [308:36:50<89:35:04,  4.86s/it]
 70%|███████   | 158616/225000 [308:36:55<87:38:44,  4.75s/it]
 70%|███████   | 158617/225000 [308:37:02<100:19:13,  5.44s/it]
 70%|███████   | 158618/225000 [308:37:12<127:50:55,  6.93s/it]
 70%|███████   | 158619/225000 [308:37:16<109:35:02,  5.94s/it]
 70%|███████   | 158620/225000 [308:37:22<110:38:59,  6.00s/it]
 70%|███████   | 158621/225000 [308:37:28<110:12:46,  5.98s/it]
 70%|███████   | 158622/225000 [308:37:36<121:27:24,  6.59s/it]
 70%|███████   | 158623/225000 [308:37:40<109:19:04,  5.93s/it]
 70%|███████   | 158624/225000 [308:37:45<103:28:03,  5.61s/it]
 70%|███████   | 158625/225000 [308:37:51<102:15:55,  5.55s/it]
 71%|███████   | 158626/225000 [308:37:55<98:42:23,  5.35s/it] 
 71%|███████   | 158627/225000 [308:38:01<97:37:46,  5.30s/it]
 71%|███████   | 158628/225000 [308:38:06<97:03:16,  5.26s/it]
 71%|███████   | 158629/225000 [308:38:16<124:12:14,  6.74s/it]
 71%|███████   | 158630/225000 [308:38:23<124:19:03,  6.74s/it]
 71%|███████   | 158631/225000 [308:38:27<111:16:21,  6.04s/it]
 71%|███████   | 158632/225000 [308:38:31<100:26:47,  5.45s/it]
 71%|███████   | 158633/225000 [308:38:36<94:52:08,  5.15s/it] 
 71%|███████   | 158634/225000 [308:38:40<90:46:19,  4.92s/it]
 71%|███████   | 158635/225000 [308:38:52<130:57:03,  7.10s/it]
 71%|███████   | 158636/225000 [308:38:56<111:59:57,  6.08s/it]
 71%|███████   | 158637/225000 [308:39:00<101:13:58,  5.49s/it]
 71%|███████   | 158638/225000 [308:39:05<98:57:53,  5.37s/it] 
 71%|███████   | 158639/225000 [308:39:10<98:03:16,  5.32s/it]
 71%|███████   | 158640/225000 [308:39:15<95:44:57,  5.19s/it]
 71%|███████   | 158641/225000 [308:39:20<94:22:40,  5.12s/it]
 71%|███████   | 158642/225000 [308:39:25<93:17:23,  5.06s/it]
 71%|███████   | 158643/225000 [308:39:30<93:35:02,  5.08s/it]
 71%|███████   | 158644/225000 [308:39:35<94:18:23,  5.12s/it]
 71%|███████   | 158645/225000 [308:39:40<92:46:19,  5.03s/it]
 71%|███████   | 158646/225000 [308:39:45<92:49:55,  5.04s/it]
 71%|███████   | 158647/225000 [308:39:51<94:12:34,  5.11s/it]
 71%|███████   | 158648/225000 [308:39:56<94:23:20,  5.12s/it]
 71%|███████   | 158649/225000 [308:40:09<137:24:59,  7.46s/it]
 71%|███████   | 158650/225000 [308:40:14<126:24:37,  6.86s/it]
                                                               
{'loss': '0.7661', 'grad_norm': '1.408', 'learning_rate': '1.088e-05', 'epoch': '0.7051', 'num_input_tokens_seen': 510992344, 'train_runtime': '1.111e+06', 'train_tokens_per_second': '459.9'}

 71%|███████   | 158650/225000 [308:40:14<126:24:37,  6.86s/it]
 71%|███████   | 158651/225000 [308:40:19<113:30:13,  6.16s/it]
 71%|███████   | 158652/225000 [308:40:23<103:34:52,  5.62s/it]
 71%|███████   | 158653/225000 [308:40:27<96:36:19,  5.24s/it] 
 71%|███████   | 158654/225000 [308:40:34<103:12:13,  5.60s/it]
 71%|███████   | 158655/225000 [308:40:44<126:12:49,  6.85s/it]
 71%|███████   | 158656/225000 [308:40:47<106:25:14,  5.77s/it]
 71%|███████   | 158657/225000 [308:40:50<93:43:04,  5.09s/it] 
 71%|███████   | 158658/225000 [308:41:01<125:20:12,  6.80s/it]
 71%|███████   | 158659/225000 [308:41:05<108:37:13,  5.89s/it]
 71%|███████   | 158660/225000 [308:41:11<112:42:26,  6.12s/it]
 71%|███████   | 158661/225000 [308:41:17<107:42:03,  5.84s/it]
 71%|███████   | 158662/225000 [308:41:24<117:52:53,  6.40s/it]
 71%|███████   | 158663/225000 [308:41:28<101:22:46,  5.50s/it]
 71%|███████   | 158664/225000 [308:41:36<118:29:45,  6.43s/it]
 71%|███████   | 158665/225000 [308:41:42<111:43:18,  6.06s/it]
 71%|███████   | 158666/225000 [308:41:48<111:55:24,  6.07s/it]
 71%|███████   | 158667/225000 [308:41:52<103:35:47,  5.62s/it]
 71%|███████   | 158668/225000 [308:41:57<97:00:04,  5.26s/it] 
 71%|███████   | 158669/225000 [308:42:01<92:51:49,  5.04s/it]
 71%|███████   | 158670/225000 [308:42:05<88:33:02,  4.81s/it]
 71%|███████   | 158671/225000 [308:42:10<87:46:29,  4.76s/it]
 71%|███████   | 158672/225000 [308:42:20<114:11:32,  6.20s/it]
 71%|███████   | 158673/225000 [308:42:28<124:04:57,  6.73s/it]
 71%|███████   | 158674/225000 [308:42:31<106:03:11,  5.76s/it]
 71%|███████   | 158675/225000 [308:42:38<113:23:02,  6.15s/it]
 71%|███████   | 158676/225000 [308:42:42<100:25:29,  5.45s/it]
 71%|███████   | 158677/225000 [308:42:46<92:15:18,  5.01s/it] 
 71%|███████   | 158678/225000 [308:42:56<121:31:57,  6.60s/it]
 71%|███████   | 158679/225000 [308:43:00<105:59:02,  5.75s/it]
 71%|███████   | 158680/225000 [308:43:07<111:27:29,  6.05s/it]
 71%|███████   | 158681/225000 [308:43:12<105:31:32,  5.73s/it]
 71%|███████   | 158682/225000 [308:43:16<99:36:43,  5.41s/it] 
 71%|███████   | 158683/225000 [308:43:21<97:12:14,  5.28s/it]
 71%|███████   | 158684/225000 [308:43:26<93:29:41,  5.08s/it]
 71%|███████   | 158685/225000 [308:43:31<92:33:18,  5.02s/it]
 71%|███████   | 158686/225000 [308:43:36<91:26:06,  4.96s/it]
 71%|███████   | 158687/225000 [308:43:41<92:16:39,  5.01s/it]
 71%|███████   | 158688/225000 [308:43:47<100:09:46,  5.44s/it]
 71%|███████   | 158689/225000 [308:43:58<131:28:41,  7.14s/it]
 71%|███████   | 158690/225000 [308:44:02<113:41:16,  6.17s/it]
 71%|███████   | 158691/225000 [308:44:06<101:31:51,  5.51s/it]
 71%|███████   | 158692/225000 [308:44:10<92:41:07,  5.03s/it] 
 71%|███████   | 158693/225000 [308:44:22<129:38:53,  7.04s/it]
 71%|███████   | 158694/225000 [308:44:25<107:15:27,  5.82s/it]
 71%|███████   | 158695/225000 [308:44:32<115:47:48,  6.29s/it]
 71%|███████   | 158696/225000 [308:44:36<103:55:40,  5.64s/it]
 71%|███████   | 158697/225000 [308:44:41<95:16:12,  5.17s/it] 
 71%|███████   | 158698/225000 [308:44:45<93:22:59,  5.07s/it]
 71%|███████   | 158699/225000 [308:44:56<126:07:44,  6.85s/it]
 71%|███████   | 158700/225000 [308:45:00<106:11:54,  5.77s/it]
                                                               
{'loss': '0.7748', 'grad_norm': '1.032', 'learning_rate': '1.086e-05', 'epoch': '0.7053', 'num_input_tokens_seen': 511121832, 'train_runtime': '1.112e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 158700/225000 [308:45:00<106:11:54,  5.77s/it]
 71%|███████   | 158701/225000 [308:45:03<95:45:59,  5.20s/it] 
 71%|███████   | 158702/225000 [308:45:10<105:38:51,  5.74s/it]
 71%|███████   | 158703/225000 [308:45:17<108:26:20,  5.89s/it]
 71%|███████   | 158704/225000 [308:45:23<109:11:07,  5.93s/it]
 71%|███████   | 158705/225000 [308:45:28<106:46:32,  5.80s/it]
 71%|███████   | 158706/225000 [308:45:35<111:12:40,  6.04s/it]
 71%|███████   | 158707/225000 [308:45:39<100:01:18,  5.43s/it]
 71%|███████   | 158708/225000 [308:45:43<95:04:17,  5.16s/it] 
 71%|███████   | 158709/225000 [308:45:48<93:18:50,  5.07s/it]
 71%|███████   | 158710/225000 [308:45:53<92:58:18,  5.05s/it]
 71%|███████   | 158711/225000 [308:45:58<91:26:46,  4.97s/it]
 71%|███████   | 158712/225000 [308:46:03<92:35:09,  5.03s/it]
 71%|███████   | 158713/225000 [308:46:08<91:37:55,  4.98s/it]
 71%|███████   | 158714/225000 [308:46:13<90:43:36,  4.93s/it]
 71%|███████   | 158715/225000 [308:46:18<91:22:53,  4.96s/it]
 71%|███████   | 158716/225000 [308:46:27<114:26:13,  6.22s/it]
 71%|███████   | 158717/225000 [308:46:37<135:41:57,  7.37s/it]
 71%|███████   | 158718/225000 [308:46:41<116:19:35,  6.32s/it]
 71%|███████   | 158719/225000 [308:46:45<101:15:28,  5.50s/it]
 71%|███████   | 158720/225000 [308:46:52<114:15:54,  6.21s/it]
 71%|███████   | 158721/225000 [308:46:59<116:35:27,  6.33s/it]
 71%|███████   | 158722/225000 [308:47:04<106:20:10,  5.78s/it]
 71%|███████   | 158723/225000 [308:47:11<116:55:30,  6.35s/it]
 71%|███████   | 158724/225000 [308:47:18<119:33:56,  6.49s/it]
 71%|███████   | 158725/225000 [308:47:22<104:28:40,  5.68s/it]
 71%|███████   | 158726/225000 [308:47:25<93:27:30,  5.08s/it] 
 71%|███████   | 158727/225000 [308:47:31<93:37:51,  5.09s/it]
 71%|███████   | 158728/225000 [308:47:39<112:15:23,  6.10s/it]
 71%|███████   | 158729/225000 [308:47:44<103:47:33,  5.64s/it]
 71%|███████   | 158730/225000 [308:47:48<97:50:24,  5.31s/it] 
 71%|███████   | 158731/225000 [308:47:52<91:47:13,  4.99s/it]
 71%|███████   | 158732/225000 [308:47:57<89:43:50,  4.87s/it]
 71%|███████   | 158733/225000 [308:48:07<118:31:00,  6.44s/it]
 71%|███████   | 158734/225000 [308:48:13<116:19:19,  6.32s/it]
 71%|███████   | 158735/225000 [308:48:17<100:31:18,  5.46s/it]
 71%|███████   | 158736/225000 [308:48:25<118:41:28,  6.45s/it]
 71%|███████   | 158737/225000 [308:48:28<100:02:10,  5.43s/it]
 71%|███████   | 158738/225000 [308:48:33<94:20:33,  5.13s/it] 
 71%|███████   | 158739/225000 [308:48:38<92:31:29,  5.03s/it]
 71%|███████   | 158740/225000 [308:48:42<91:17:29,  4.96s/it]
 71%|███████   | 158741/225000 [308:48:47<89:44:35,  4.88s/it]
 71%|███████   | 158742/225000 [308:48:52<89:55:07,  4.89s/it]
 71%|███████   | 158743/225000 [308:48:57<92:00:03,  5.00s/it]
 71%|███████   | 158744/225000 [308:49:02<92:07:32,  5.01s/it]
 71%|███████   | 158745/225000 [308:49:07<90:50:22,  4.94s/it]
 71%|███████   | 158746/225000 [308:49:20<133:21:25,  7.25s/it]
 71%|███████   | 158747/225000 [308:49:26<128:46:13,  7.00s/it]
 71%|███████   | 158748/225000 [308:49:33<129:03:41,  7.01s/it]
 71%|███████   | 158749/225000 [308:49:37<111:47:54,  6.07s/it]
 71%|███████   | 158750/225000 [308:49:41<102:42:03,  5.58s/it]
                                                               
{'loss': '0.7876', 'grad_norm': '1.12', 'learning_rate': '1.085e-05', 'epoch': '0.7056', 'num_input_tokens_seen': 511252728, 'train_runtime': '1.112e+06', 'train_tokens_per_second': '459.9'}

 71%|███████   | 158750/225000 [308:49:41<102:42:03,  5.58s/it]
 71%|███████   | 158751/225000 [308:49:46<96:49:50,  5.26s/it] 
 71%|███████   | 158752/225000 [308:49:59<137:08:11,  7.45s/it]
 71%|███████   | 158753/225000 [308:50:02<114:32:00,  6.22s/it]
 71%|███████   | 158754/225000 [308:50:08<113:10:43,  6.15s/it]
 71%|███████   | 158755/225000 [308:50:12<101:22:35,  5.51s/it]
 71%|███████   | 158756/225000 [308:50:16<91:35:54,  4.98s/it] 
 71%|███████   | 158757/225000 [308:50:26<121:55:30,  6.63s/it]
 71%|███████   | 158758/225000 [308:50:30<105:53:12,  5.75s/it]
 71%|███████   | 158759/225000 [308:50:35<103:47:41,  5.64s/it]
 71%|███████   | 158760/225000 [308:50:43<113:47:53,  6.18s/it]
 71%|███████   | 158761/225000 [308:50:46<100:37:11,  5.47s/it]
 71%|███████   | 158762/225000 [308:50:50<91:59:13,  5.00s/it] 
 71%|███████   | 158763/225000 [308:50:55<88:09:12,  4.79s/it]
 71%|███████   | 158764/225000 [308:51:00<92:25:54,  5.02s/it]
 71%|███████   | 158765/225000 [308:51:11<123:40:15,  6.72s/it]
 71%|███████   | 158766/225000 [308:51:14<105:07:29,  5.71s/it]
 71%|███████   | 158767/225000 [308:51:18<93:12:23,  5.07s/it] 
 71%|███████   | 158768/225000 [308:51:22<88:16:23,  4.80s/it]
 71%|███████   | 158769/225000 [308:51:33<119:42:22,  6.51s/it]
 71%|███████   | 158770/225000 [308:51:36<104:49:14,  5.70s/it]
 71%|███████   | 158771/225000 [308:51:41<96:46:43,  5.26s/it] 
 71%|███████   | 158772/225000 [308:51:50<118:42:32,  6.45s/it]
 71%|███████   | 158773/225000 [308:51:54<105:10:47,  5.72s/it]
 71%|███████   | 158774/225000 [308:52:00<106:01:59,  5.76s/it]
 71%|███████   | 158775/225000 [308:52:06<111:06:55,  6.04s/it]
 71%|███████   | 158776/225000 [308:52:10<96:04:12,  5.22s/it] 
 71%|███████   | 158777/225000 [308:52:17<106:15:29,  5.78s/it]
 71%|███████   | 158778/225000 [308:52:21<96:23:44,  5.24s/it] 
 71%|███████   | 158779/225000 [308:52:24<87:41:16,  4.77s/it]
 71%|███████   | 158780/225000 [308:52:29<87:04:31,  4.73s/it]
 71%|███████   | 158781/225000 [308:52:39<116:56:25,  6.36s/it]
 71%|███████   | 158782/225000 [308:52:43<103:36:57,  5.63s/it]
 71%|███████   | 158783/225000 [308:52:47<93:27:22,  5.08s/it] 
 71%|███████   | 158784/225000 [308:52:51<88:34:30,  4.82s/it]
 71%|███████   | 158785/225000 [308:52:56<89:18:03,  4.86s/it]
 71%|███████   | 158786/225000 [308:53:01<89:03:23,  4.84s/it]
 71%|███████   | 158787/225000 [308:53:06<88:04:13,  4.79s/it]
 71%|███████   | 158788/225000 [308:53:17<127:18:43,  6.92s/it]
 71%|███████   | 158789/225000 [308:53:22<113:06:08,  6.15s/it]
 71%|███████   | 158790/225000 [308:53:26<105:00:27,  5.71s/it]
 71%|███████   | 158791/225000 [308:53:33<111:25:08,  6.06s/it]
 71%|███████   | 158792/225000 [308:53:37<98:10:19,  5.34s/it] 
 71%|███████   | 158793/225000 [308:53:47<122:19:24,  6.65s/it]
 71%|███████   | 158794/225000 [308:53:50<104:30:00,  5.68s/it]
 71%|███████   | 158795/225000 [308:53:57<111:40:53,  6.07s/it]
 71%|███████   | 158796/225000 [308:54:02<102:55:00,  5.60s/it]
 71%|███████   | 158797/225000 [308:54:06<93:34:06,  5.09s/it] 
 71%|███████   | 158798/225000 [308:54:10<92:33:59,  5.03s/it]
 71%|███████   | 158799/225000 [308:54:20<118:36:02,  6.45s/it]
 71%|███████   | 158800/225000 [308:54:24<102:35:56,  5.58s/it]
                                                               
{'loss': '0.7767', 'grad_norm': '1.056', 'learning_rate': '1.083e-05', 'epoch': '0.7058', 'num_input_tokens_seen': 511382000, 'train_runtime': '1.112e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 158800/225000 [308:54:24<102:35:56,  5.58s/it]
 71%|███████   | 158801/225000 [308:54:32<118:02:36,  6.42s/it]
 71%|███████   | 158802/225000 [308:54:36<104:14:27,  5.67s/it]
 71%|███████   | 158803/225000 [308:54:40<94:28:43,  5.14s/it] 
 71%|███████   | 158804/225000 [308:54:46<99:12:10,  5.40s/it]
 71%|███████   | 158805/225000 [308:54:54<112:18:37,  6.11s/it]
 71%|███████   | 158806/225000 [308:54:58<102:58:02,  5.60s/it]
 71%|███████   | 158807/225000 [308:55:05<112:38:49,  6.13s/it]
 71%|███████   | 158808/225000 [308:55:09<98:00:58,  5.33s/it] 
 71%|███████   | 158809/225000 [308:55:18<119:30:19,  6.50s/it]
 71%|███████   | 158810/225000 [308:55:22<104:16:12,  5.67s/it]
 71%|███████   | 158811/225000 [308:55:26<93:55:45,  5.11s/it] 
 71%|███████   | 158812/225000 [308:55:30<88:28:45,  4.81s/it]
 71%|███████   | 158813/225000 [308:55:39<111:07:11,  6.04s/it]
 71%|███████   | 158814/225000 [308:55:46<117:18:50,  6.38s/it]
 71%|███████   | 158815/225000 [308:55:49<100:53:05,  5.49s/it]
 71%|███████   | 158816/225000 [308:55:54<94:38:46,  5.15s/it] 
 71%|███████   | 158817/225000 [308:56:01<109:21:36,  5.95s/it]
 71%|███████   | 158818/225000 [308:56:05<94:47:21,  5.16s/it] 
 71%|███████   | 158819/225000 [308:56:14<115:44:58,  6.30s/it]
 71%|███████   | 158820/225000 [308:56:17<101:29:55,  5.52s/it]
 71%|███████   | 158821/225000 [308:56:21<90:12:29,  4.91s/it] 
 71%|███████   | 158822/225000 [308:56:30<112:33:29,  6.12s/it]
 71%|███████   | 158823/225000 [308:56:35<109:27:42,  5.95s/it]
 71%|███████   | 158824/225000 [308:56:41<107:02:13,  5.82s/it]
 71%|███████   | 158825/225000 [308:56:45<99:48:59,  5.43s/it] 
 71%|███████   | 158826/225000 [308:56:51<100:03:48,  5.44s/it]
 71%|███████   | 158827/225000 [308:56:57<105:09:15,  5.72s/it]
 71%|███████   | 158828/225000 [308:57:02<101:47:42,  5.54s/it]
 71%|███████   | 158829/225000 [308:57:08<101:20:00,  5.51s/it]
 71%|███████   | 158830/225000 [308:57:13<100:32:44,  5.47s/it]
 71%|███████   | 158831/225000 [308:57:18<98:57:48,  5.38s/it] 
 71%|███████   | 158832/225000 [308:57:24<97:16:21,  5.29s/it]
 71%|███████   | 158833/225000 [308:57:29<97:58:13,  5.33s/it]
 71%|███████   | 158834/225000 [308:57:38<118:11:37,  6.43s/it]
 71%|███████   | 158835/225000 [308:57:43<112:53:01,  6.14s/it]
 71%|███████   | 158836/225000 [308:57:50<117:24:11,  6.39s/it]
 71%|███████   | 158837/225000 [308:57:54<104:00:38,  5.66s/it]
 71%|███████   | 158838/225000 [308:58:00<105:57:39,  5.77s/it]
 71%|███████   | 158839/225000 [308:58:07<111:25:37,  6.06s/it]
 71%|███████   | 158840/225000 [308:58:14<115:57:21,  6.31s/it]
 71%|███████   | 158841/225000 [308:58:17<98:31:15,  5.36s/it] 
 71%|███████   | 158842/225000 [308:58:25<112:54:27,  6.14s/it]
 71%|███████   | 158843/225000 [308:58:29<98:44:40,  5.37s/it] 
 71%|███████   | 158844/225000 [308:58:34<99:01:40,  5.39s/it]
 71%|███████   | 158845/225000 [308:58:42<111:49:44,  6.09s/it]
 71%|███████   | 158846/225000 [308:58:45<98:19:39,  5.35s/it] 
 71%|███████   | 158847/225000 [308:58:49<89:22:11,  4.86s/it]
 71%|███████   | 158848/225000 [308:58:59<116:41:26,  6.35s/it]
 71%|███████   | 158849/225000 [308:59:03<101:49:10,  5.54s/it]
 71%|███████   | 158850/225000 [308:59:11<115:52:56,  6.31s/it]
                                                               
{'loss': '0.7726', 'grad_norm': '1.236', 'learning_rate': '1.082e-05', 'epoch': '0.706', 'num_input_tokens_seen': 511511752, 'train_runtime': '1.112e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 158850/225000 [308:59:11<115:52:56,  6.31s/it]
 71%|███████   | 158851/225000 [308:59:15<104:09:31,  5.67s/it]
 71%|███████   | 158852/225000 [308:59:19<94:38:21,  5.15s/it] 
 71%|███████   | 158853/225000 [308:59:23<87:28:40,  4.76s/it]
 71%|███████   | 158854/225000 [308:59:30<102:09:07,  5.56s/it]
 71%|███████   | 158855/225000 [308:59:38<115:43:58,  6.30s/it]
 71%|███████   | 158856/225000 [308:59:43<106:08:07,  5.78s/it]
 71%|███████   | 158857/225000 [308:59:48<103:32:09,  5.64s/it]
 71%|███████   | 158858/225000 [308:59:53<101:50:54,  5.54s/it]
 71%|███████   | 158859/225000 [308:59:59<99:43:44,  5.43s/it] 
 71%|███████   | 158860/225000 [309:00:04<98:54:18,  5.38s/it]
 71%|███████   | 158861/225000 [309:00:09<98:51:11,  5.38s/it]
 71%|███████   | 158862/225000 [309:00:15<99:08:59,  5.40s/it]
 71%|███████   | 158863/225000 [309:00:20<99:21:30,  5.41s/it]
 71%|███████   | 158864/225000 [309:00:25<99:05:34,  5.39s/it]
 71%|███████   | 158865/225000 [309:00:31<99:02:24,  5.39s/it]
 71%|███████   | 158866/225000 [309:00:36<99:43:07,  5.43s/it]
 71%|███████   | 158867/225000 [309:00:42<100:20:41,  5.46s/it]
 71%|███████   | 158868/225000 [309:00:47<99:02:12,  5.39s/it] 
 71%|███████   | 158869/225000 [309:00:53<99:17:05,  5.40s/it]
 71%|███████   | 158870/225000 [309:00:58<97:43:56,  5.32s/it]
 71%|███████   | 158871/225000 [309:01:10<137:48:16,  7.50s/it]
 71%|███████   | 158872/225000 [309:01:14<119:26:48,  6.50s/it]
 71%|███████   | 158873/225000 [309:01:21<119:35:12,  6.51s/it]
 71%|███████   | 158874/225000 [309:01:25<103:36:52,  5.64s/it]
 71%|███████   | 158875/225000 [309:01:36<135:35:10,  7.38s/it]
 71%|███████   | 158876/225000 [309:01:40<115:47:14,  6.30s/it]
 71%|███████   | 158877/225000 [309:01:47<119:08:27,  6.49s/it]
 71%|███████   | 158878/225000 [309:01:50<104:24:57,  5.68s/it]
 71%|███████   | 158879/225000 [309:01:54<94:11:55,  5.13s/it] 
 71%|███████   | 158880/225000 [309:02:04<121:54:26,  6.64s/it]
 71%|███████   | 158881/225000 [309:02:08<104:10:07,  5.67s/it]
 71%|███████   | 158882/225000 [309:02:12<93:19:05,  5.08s/it] 
 71%|███████   | 158883/225000 [309:02:21<116:55:47,  6.37s/it]
 71%|███████   | 158884/225000 [309:02:24<99:25:52,  5.41s/it] 
 71%|███████   | 158885/225000 [309:02:31<108:45:41,  5.92s/it]
 71%|███████   | 158886/225000 [309:02:36<99:47:11,  5.43s/it] 
 71%|███████   | 158887/225000 [309:02:39<90:34:52,  4.93s/it]
 71%|███████   | 158888/225000 [309:02:45<94:37:28,  5.15s/it]
 71%|███████   | 158889/225000 [309:02:53<109:28:16,  5.96s/it]
 71%|███████   | 158890/225000 [309:02:57<99:18:57,  5.41s/it] 
 71%|███████   | 158891/225000 [309:03:05<113:50:55,  6.20s/it]
 71%|███████   | 158892/225000 [309:03:09<99:09:00,  5.40s/it] 
 71%|███████   | 158893/225000 [309:03:17<118:43:10,  6.47s/it]
 71%|███████   | 158894/225000 [309:03:22<106:45:19,  5.81s/it]
 71%|███████   | 158895/225000 [309:03:25<94:56:20,  5.17s/it] 
 71%|███████   | 158896/225000 [309:03:33<110:04:25,  5.99s/it]
 71%|███████   | 158897/225000 [309:03:39<107:21:13,  5.85s/it]
 71%|███████   | 158898/225000 [309:03:43<98:24:43,  5.36s/it] 
 71%|███████   | 158899/225000 [309:03:47<89:36:16,  4.88s/it]
 71%|███████   | 158900/225000 [309:03:57<118:14:20,  6.44s/it]
                                                               
{'loss': '0.7773', 'grad_norm': '1.064', 'learning_rate': '1.08e-05', 'epoch': '0.7062', 'num_input_tokens_seen': 511642760, 'train_runtime': '1.113e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 158900/225000 [309:03:57<118:14:20,  6.44s/it]
 71%|███████   | 158901/225000 [309:04:01<107:32:51,  5.86s/it]
 71%|███████   | 158902/225000 [309:04:09<115:55:09,  6.31s/it]
 71%|███████   | 158903/225000 [309:04:14<109:17:54,  5.95s/it]
 71%|███████   | 158904/225000 [309:04:21<117:32:30,  6.40s/it]
 71%|███████   | 158905/225000 [309:04:25<101:19:56,  5.52s/it]
 71%|███████   | 158906/225000 [309:04:33<115:52:34,  6.31s/it]
 71%|███████   | 158907/225000 [309:04:36<99:51:37,  5.44s/it] 
 71%|███████   | 158908/225000 [309:04:40<91:17:53,  4.97s/it]
 71%|███████   | 158909/225000 [309:04:47<103:03:10,  5.61s/it]
 71%|███████   | 158910/225000 [309:04:55<114:28:02,  6.24s/it]
 71%|███████   | 158911/225000 [309:04:59<103:22:22,  5.63s/it]
 71%|███████   | 158912/225000 [309:05:04<96:28:46,  5.26s/it] 
 71%|███████   | 158913/225000 [309:05:08<88:46:02,  4.84s/it]
 71%|███████   | 158914/225000 [309:05:15<103:59:08,  5.66s/it]
 71%|███████   | 158915/225000 [309:05:22<110:19:31,  6.01s/it]
 71%|███████   | 158916/225000 [309:05:27<104:12:04,  5.68s/it]
 71%|███████   | 158917/225000 [309:05:31<95:17:03,  5.19s/it] 
 71%|███████   | 158918/225000 [309:05:36<92:44:26,  5.05s/it]
 71%|███████   | 158919/225000 [309:05:40<88:38:27,  4.83s/it]
 71%|███████   | 158920/225000 [309:05:45<89:25:45,  4.87s/it]
 71%|███████   | 158921/225000 [309:05:50<88:33:54,  4.83s/it]
 71%|███████   | 158922/225000 [309:06:02<131:40:54,  7.17s/it]
 71%|███████   | 158923/225000 [309:06:07<115:53:44,  6.31s/it]
 71%|███████   | 158924/225000 [309:06:15<125:12:16,  6.82s/it]
 71%|███████   | 158925/225000 [309:06:18<106:24:05,  5.80s/it]
 71%|███████   | 158926/225000 [309:06:27<123:01:02,  6.70s/it]
 71%|███████   | 158927/225000 [309:06:30<104:07:08,  5.67s/it]
 71%|███████   | 158928/225000 [309:06:39<119:48:59,  6.53s/it]
 71%|███████   | 158929/225000 [309:06:42<103:20:11,  5.63s/it]
 71%|███████   | 158930/225000 [309:06:50<117:50:03,  6.42s/it]
 71%|███████   | 158931/225000 [309:06:54<100:12:34,  5.46s/it]
 71%|███████   | 158932/225000 [309:06:57<89:38:53,  4.88s/it] 
 71%|███████   | 158933/225000 [309:07:01<84:25:22,  4.60s/it]
 71%|███████   | 158934/225000 [309:07:06<83:20:58,  4.54s/it]
 71%|███████   | 158935/225000 [309:07:12<93:49:41,  5.11s/it]
 71%|███████   | 158936/225000 [309:07:24<132:32:48,  7.22s/it]
 71%|███████   | 158937/225000 [309:07:27<111:00:40,  6.05s/it]
 71%|███████   | 158938/225000 [309:07:34<116:39:50,  6.36s/it]
 71%|███████   | 158939/225000 [309:07:39<107:03:11,  5.83s/it]
 71%|███████   | 158940/225000 [309:07:46<114:30:53,  6.24s/it]
 71%|███████   | 158941/225000 [309:07:50<100:38:13,  5.48s/it]
 71%|███████   | 158942/225000 [309:07:55<99:15:16,  5.41s/it] 
 71%|███████   | 158943/225000 [309:08:00<96:47:18,  5.27s/it]
 71%|███████   | 158944/225000 [309:08:05<95:51:48,  5.22s/it]
 71%|███████   | 158945/225000 [309:08:10<93:55:05,  5.12s/it]
 71%|███████   | 158946/225000 [309:08:15<91:28:50,  4.99s/it]
 71%|███████   | 158947/225000 [309:08:20<94:09:10,  5.13s/it]
 71%|███████   | 158948/225000 [309:08:25<92:36:23,  5.05s/it]
 71%|███████   | 158949/225000 [309:08:30<93:40:58,  5.11s/it]
 71%|███████   | 158950/225000 [309:08:35<93:27:49,  5.09s/it]
                                                              
{'loss': '0.7856', 'grad_norm': '1.067', 'learning_rate': '1.079e-05', 'epoch': '0.7064', 'num_input_tokens_seen': 511773520, 'train_runtime': '1.113e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 158950/225000 [309:08:35<93:27:49,  5.09s/it]
 71%|███████   | 158951/225000 [309:08:41<94:09:51,  5.13s/it]
 71%|███████   | 158952/225000 [309:08:46<95:17:22,  5.19s/it]
 71%|███████   | 158953/225000 [309:08:55<114:05:37,  6.22s/it]
 71%|███████   | 158954/225000 [309:09:04<130:24:55,  7.11s/it]
 71%|███████   | 158955/225000 [309:09:07<110:52:09,  6.04s/it]
 71%|███████   | 158956/225000 [309:09:16<123:35:15,  6.74s/it]
 71%|███████   | 158957/225000 [309:09:20<111:36:29,  6.08s/it]
 71%|███████   | 158958/225000 [309:09:25<102:57:46,  5.61s/it]
 71%|███████   | 158959/225000 [309:09:30<98:50:15,  5.39s/it] 
 71%|███████   | 158960/225000 [309:09:35<96:54:47,  5.28s/it]
 71%|███████   | 158961/225000 [309:09:42<109:05:47,  5.95s/it]
 71%|███████   | 158962/225000 [309:09:51<123:23:19,  6.73s/it]
 71%|███████   | 158963/225000 [309:09:57<119:46:45,  6.53s/it]
 71%|███████   | 158964/225000 [309:10:01<108:48:13,  5.93s/it]
 71%|███████   | 158965/225000 [309:10:07<107:19:15,  5.85s/it]
 71%|███████   | 158966/225000 [309:10:13<109:22:19,  5.96s/it]
 71%|███████   | 158967/225000 [309:10:17<97:43:31,  5.33s/it] 
 71%|███████   | 158968/225000 [309:10:28<127:39:03,  6.96s/it]
 71%|███████   | 158969/225000 [309:10:31<106:58:29,  5.83s/it]
 71%|███████   | 158970/225000 [309:10:35<94:12:13,  5.14s/it] 
 71%|███████   | 158971/225000 [309:10:44<118:45:07,  6.47s/it]
 71%|███████   | 158972/225000 [309:10:48<106:38:53,  5.81s/it]
 71%|███████   | 158973/225000 [309:10:52<95:40:07,  5.22s/it] 
 71%|███████   | 158974/225000 [309:10:56<88:51:05,  4.84s/it]
 71%|███████   | 158975/225000 [309:11:01<89:19:47,  4.87s/it]
 71%|███████   | 158976/225000 [309:11:06<89:55:57,  4.90s/it]
 71%|███████   | 158977/225000 [309:11:11<86:51:37,  4.74s/it]
 71%|███████   | 158978/225000 [309:11:16<88:12:13,  4.81s/it]
 71%|███████   | 158979/225000 [309:11:24<106:22:21,  5.80s/it]
 71%|███████   | 158980/225000 [309:11:33<126:28:38,  6.90s/it]
 71%|███████   | 158981/225000 [309:11:36<106:34:43,  5.81s/it]
 71%|███████   | 158982/225000 [309:11:45<119:49:52,  6.53s/it]
 71%|███████   | 158983/225000 [309:11:48<102:19:50,  5.58s/it]
 71%|███████   | 158984/225000 [309:11:57<122:24:44,  6.68s/it]
 71%|███████   | 158985/225000 [309:12:02<110:53:02,  6.05s/it]
 71%|███████   | 158986/225000 [309:12:09<118:30:47,  6.46s/it]
 71%|███████   | 158987/225000 [309:12:13<101:22:27,  5.53s/it]
 71%|███████   | 158988/225000 [309:12:20<111:43:44,  6.09s/it]
 71%|███████   | 158989/225000 [309:12:24<97:50:41,  5.34s/it] 
 71%|███████   | 158990/225000 [309:12:31<110:04:20,  6.00s/it]
 71%|███████   | 158991/225000 [309:12:34<93:35:19,  5.10s/it] 
 71%|███████   | 158992/225000 [309:12:39<94:17:22,  5.14s/it]
 71%|███████   | 158993/225000 [309:12:47<105:41:47,  5.76s/it]
 71%|███████   | 158994/225000 [309:12:51<97:55:56,  5.34s/it] 
 71%|███████   | 158995/225000 [309:12:59<113:09:35,  6.17s/it]
 71%|███████   | 158996/225000 [309:13:04<105:10:15,  5.74s/it]
 71%|███████   | 158997/225000 [309:13:09<100:37:56,  5.49s/it]
 71%|███████   | 158998/225000 [309:13:14<97:28:51,  5.32s/it] 
 71%|███████   | 158999/225000 [309:13:18<94:11:31,  5.14s/it]
 71%|███████   | 159000/225000 [309:13:24<97:04:34,  5.30s/it]
                                                              
{'loss': '0.7687', 'grad_norm': '1.19', 'learning_rate': '1.077e-05', 'epoch': '0.7067', 'num_input_tokens_seen': 511904064, 'train_runtime': '1.113e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159000/225000 [309:13:24<97:04:34,  5.30s/it]
 71%|███████   | 159001/225000 [309:13:29<96:53:22,  5.28s/it]
 71%|███████   | 159002/225000 [309:13:34<96:21:18,  5.26s/it]
 71%|███████   | 159003/225000 [309:13:39<95:35:48,  5.21s/it]
 71%|███████   | 159004/225000 [309:13:44<92:45:25,  5.06s/it]
 71%|███████   | 159005/225000 [309:13:49<93:07:59,  5.08s/it]
 71%|███████   | 159006/225000 [309:13:54<93:00:43,  5.07s/it]
 71%|███████   | 159007/225000 [309:14:07<131:52:33,  7.19s/it]
 71%|███████   | 159008/225000 [309:14:12<121:08:22,  6.61s/it]
 71%|███████   | 159009/225000 [309:14:15<104:42:36,  5.71s/it]
 71%|███████   | 159010/225000 [309:14:20<97:36:55,  5.33s/it] 
 71%|███████   | 159011/225000 [309:14:24<93:02:01,  5.08s/it]
 71%|███████   | 159012/225000 [309:14:36<128:58:31,  7.04s/it]
 71%|███████   | 159013/225000 [309:14:40<110:35:31,  6.03s/it]
 71%|███████   | 159014/225000 [309:14:43<96:42:58,  5.28s/it] 
 71%|███████   | 159015/225000 [309:14:52<115:40:05,  6.31s/it]
 71%|███████   | 159016/225000 [309:14:55<100:45:33,  5.50s/it]
 71%|███████   | 159017/225000 [309:15:02<108:06:16,  5.90s/it]
 71%|███████   | 159018/225000 [309:15:09<110:09:11,  6.01s/it]
 71%|███████   | 159019/225000 [309:15:13<101:57:28,  5.56s/it]
 71%|███████   | 159020/225000 [309:15:20<107:56:37,  5.89s/it]
 71%|███████   | 159021/225000 [309:15:24<97:23:59,  5.31s/it] 
 71%|███████   | 159022/225000 [309:15:32<111:25:36,  6.08s/it]
 71%|███████   | 159023/225000 [309:15:35<97:09:47,  5.30s/it] 
 71%|███████   | 159024/225000 [309:15:40<92:56:37,  5.07s/it]
 71%|███████   | 159025/225000 [309:15:44<88:02:52,  4.80s/it]
 71%|███████   | 159026/225000 [309:15:49<89:01:19,  4.86s/it]
 71%|███████   | 159027/225000 [309:15:54<92:30:28,  5.05s/it]
 71%|███████   | 159028/225000 [309:16:06<130:19:30,  7.11s/it]
 71%|███████   | 159029/225000 [309:16:11<115:45:26,  6.32s/it]
 71%|███████   | 159030/225000 [309:16:15<104:04:34,  5.68s/it]
 71%|███████   | 159031/225000 [309:16:19<98:18:14,  5.36s/it] 
 71%|███████   | 159032/225000 [309:16:24<92:28:35,  5.05s/it]
 71%|███████   | 159033/225000 [309:16:30<101:08:22,  5.52s/it]
 71%|███████   | 159034/225000 [309:16:39<120:33:33,  6.58s/it]
 71%|███████   | 159035/225000 [309:16:43<105:32:47,  5.76s/it]
 71%|███████   | 159036/225000 [309:16:47<93:17:02,  5.09s/it] 
 71%|███████   | 159037/225000 [309:16:55<111:41:23,  6.10s/it]
 71%|███████   | 159038/225000 [309:17:02<114:55:04,  6.27s/it]
 71%|███████   | 159039/225000 [309:17:07<106:02:28,  5.79s/it]
 71%|███████   | 159040/225000 [309:17:12<102:39:44,  5.60s/it]
 71%|███████   | 159041/225000 [309:17:17<98:52:35,  5.40s/it] 
 71%|███████   | 159042/225000 [309:17:22<97:09:45,  5.30s/it]
 71%|███████   | 159043/225000 [309:17:27<95:22:42,  5.21s/it]
 71%|███████   | 159044/225000 [309:17:32<93:48:17,  5.12s/it]
 71%|███████   | 159045/225000 [309:17:37<92:49:15,  5.07s/it]
 71%|███████   | 159046/225000 [309:17:42<92:16:00,  5.04s/it]
 71%|███████   | 159047/225000 [309:17:50<112:39:18,  6.15s/it]
 71%|███████   | 159048/225000 [309:18:00<129:40:11,  7.08s/it]
 71%|███████   | 159049/225000 [309:18:06<126:49:21,  6.92s/it]
 71%|███████   | 159050/225000 [309:18:11<115:34:16,  6.31s/it]
                                                               
{'loss': '0.7795', 'grad_norm': '1.033', 'learning_rate': '1.076e-05', 'epoch': '0.7069', 'num_input_tokens_seen': 512034560, 'train_runtime': '1.113e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159050/225000 [309:18:11<115:34:16,  6.31s/it]
 71%|███████   | 159051/225000 [309:18:16<106:06:51,  5.79s/it]
 71%|███████   | 159052/225000 [309:18:21<102:08:10,  5.58s/it]
 71%|███████   | 159053/225000 [309:18:26<100:32:25,  5.49s/it]
 71%|███████   | 159054/225000 [309:18:31<97:37:54,  5.33s/it] 
 71%|███████   | 159055/225000 [309:18:36<97:13:25,  5.31s/it]
 71%|███████   | 159056/225000 [309:18:41<96:12:49,  5.25s/it]
 71%|███████   | 159057/225000 [309:18:47<97:21:26,  5.31s/it]
 71%|███████   | 159058/225000 [309:18:52<99:06:06,  5.41s/it]
 71%|███████   | 159059/225000 [309:19:00<114:04:03,  6.23s/it]
 71%|███████   | 159060/225000 [309:19:09<128:15:53,  7.00s/it]
 71%|███████   | 159061/225000 [309:19:13<108:34:23,  5.93s/it]
 71%|███████   | 159062/225000 [309:19:18<106:00:00,  5.79s/it]
 71%|███████   | 159063/225000 [309:19:26<119:46:47,  6.54s/it]
 71%|███████   | 159064/225000 [309:19:30<104:23:43,  5.70s/it]
 71%|███████   | 159065/225000 [309:19:34<96:17:14,  5.26s/it] 
 71%|███████   | 159066/225000 [309:19:40<96:04:49,  5.25s/it]
 71%|███████   | 159067/225000 [309:19:50<125:29:06,  6.85s/it]
 71%|███████   | 159068/225000 [309:19:54<108:07:44,  5.90s/it]
 71%|███████   | 159069/225000 [309:20:01<115:07:51,  6.29s/it]
 71%|███████   | 159070/225000 [309:20:10<128:07:22,  7.00s/it]
 71%|███████   | 159071/225000 [309:20:14<111:57:40,  6.11s/it]
 71%|███████   | 159072/225000 [309:20:18<100:06:06,  5.47s/it]
 71%|███████   | 159073/225000 [309:20:22<92:44:25,  5.06s/it] 
 71%|███████   | 159074/225000 [309:20:29<103:16:04,  5.64s/it]
 71%|███████   | 159075/225000 [309:20:37<117:39:17,  6.42s/it]
 71%|███████   | 159076/225000 [309:20:41<101:45:39,  5.56s/it]
 71%|███████   | 159077/225000 [309:20:52<134:46:57,  7.36s/it]
 71%|███████   | 159078/225000 [309:20:56<115:24:16,  6.30s/it]
 71%|███████   | 159079/225000 [309:21:00<102:11:45,  5.58s/it]
 71%|███████   | 159080/225000 [309:21:04<91:06:40,  4.98s/it] 
 71%|███████   | 159081/225000 [309:21:15<126:56:13,  6.93s/it]
 71%|███████   | 159082/225000 [309:21:19<109:52:42,  6.00s/it]
 71%|███████   | 159083/225000 [309:21:23<101:52:46,  5.56s/it]
 71%|███████   | 159084/225000 [309:21:27<93:17:40,  5.10s/it] 
 71%|███████   | 159085/225000 [309:21:31<87:45:14,  4.79s/it]
 71%|███████   | 159086/225000 [309:21:37<89:07:54,  4.87s/it]
 71%|███████   | 159087/225000 [309:21:48<127:47:55,  6.98s/it]
 71%|███████   | 159088/225000 [309:21:53<113:31:19,  6.20s/it]
 71%|███████   | 159089/225000 [309:21:57<102:44:25,  5.61s/it]
 71%|███████   | 159090/225000 [309:22:01<96:10:32,  5.25s/it] 
 71%|███████   | 159091/225000 [309:22:06<92:05:10,  5.03s/it]
 71%|███████   | 159092/225000 [309:22:17<126:53:01,  6.93s/it]
 71%|███████   | 159093/225000 [309:22:21<107:22:23,  5.86s/it]
 71%|███████   | 159094/225000 [309:22:25<100:49:51,  5.51s/it]
 71%|███████   | 159095/225000 [309:22:34<118:16:04,  6.46s/it]
 71%|███████   | 159096/225000 [309:22:38<101:50:12,  5.56s/it]
 71%|███████   | 159097/225000 [309:22:45<113:41:23,  6.21s/it]
 71%|███████   | 159098/225000 [309:22:49<102:33:22,  5.60s/it]
 71%|███████   | 159099/225000 [309:22:54<94:37:46,  5.17s/it] 
 71%|███████   | 159100/225000 [309:23:02<114:23:30,  6.25s/it]
                                                               
{'loss': '0.7739', 'grad_norm': '1.045', 'learning_rate': '1.074e-05', 'epoch': '0.7071', 'num_input_tokens_seen': 512164224, 'train_runtime': '1.114e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159100/225000 [309:23:02<114:23:30,  6.25s/it]
 71%|███████   | 159101/225000 [309:23:06<102:35:40,  5.60s/it]
 71%|███████   | 159102/225000 [309:23:10<91:20:29,  4.99s/it] 
 71%|███████   | 159103/225000 [309:23:15<88:24:23,  4.83s/it]
 71%|███████   | 159104/225000 [309:23:19<85:38:22,  4.68s/it]
 71%|███████   | 159105/225000 [309:23:30<120:40:22,  6.59s/it]
 71%|███████   | 159106/225000 [309:23:35<115:14:10,  6.30s/it]
 71%|███████   | 159107/225000 [309:23:44<127:20:25,  6.96s/it]
 71%|███████   | 159108/225000 [309:23:47<107:30:51,  5.87s/it]
 71%|███████   | 159109/225000 [309:23:56<121:26:47,  6.64s/it]
 71%|███████   | 159110/225000 [309:24:00<109:47:33,  6.00s/it]
 71%|███████   | 159111/225000 [309:24:09<123:53:59,  6.77s/it]
 71%|███████   | 159112/225000 [309:24:12<104:35:35,  5.71s/it]
 71%|███████   | 159113/225000 [309:24:18<103:21:33,  5.65s/it]
 71%|███████   | 159114/225000 [309:24:23<101:51:32,  5.57s/it]
 71%|███████   | 159115/225000 [309:24:28<100:08:10,  5.47s/it]
 71%|███████   | 159116/225000 [309:24:34<100:32:25,  5.49s/it]
 71%|███████   | 159117/225000 [309:24:39<97:28:46,  5.33s/it] 
 71%|███████   | 159118/225000 [309:24:44<94:59:49,  5.19s/it]
 71%|███████   | 159119/225000 [309:24:49<95:51:48,  5.24s/it]
 71%|███████   | 159120/225000 [309:24:54<94:28:13,  5.16s/it]
 71%|███████   | 159121/225000 [309:24:59<95:55:00,  5.24s/it]
 71%|███████   | 159122/225000 [309:25:05<96:28:14,  5.27s/it]
 71%|███████   | 159123/225000 [309:25:10<98:52:23,  5.40s/it]
 71%|███████   | 159124/225000 [309:25:16<99:03:51,  5.41s/it]
 71%|███████   | 159125/225000 [309:25:22<101:31:57,  5.55s/it]
 71%|███████   | 159126/225000 [309:25:30<116:29:23,  6.37s/it]
 71%|███████   | 159127/225000 [309:25:40<134:52:42,  7.37s/it]
 71%|███████   | 159128/225000 [309:25:43<114:58:00,  6.28s/it]
 71%|███████   | 159129/225000 [309:25:52<127:02:00,  6.94s/it]
 71%|███████   | 159130/225000 [309:25:56<109:45:18,  6.00s/it]
 71%|███████   | 159131/225000 [309:26:00<100:50:43,  5.51s/it]
 71%|███████   | 159132/225000 [309:26:12<136:14:47,  7.45s/it]
 71%|███████   | 159133/225000 [309:26:15<113:43:36,  6.22s/it]
 71%|███████   | 159134/225000 [309:26:22<115:05:33,  6.29s/it]
 71%|███████   | 159135/225000 [309:26:25<100:02:13,  5.47s/it]
 71%|███████   | 159136/225000 [309:26:33<112:49:40,  6.17s/it]
 71%|███████   | 159137/225000 [309:26:37<102:23:12,  5.60s/it]
 71%|███████   | 159138/225000 [309:26:41<92:24:17,  5.05s/it] 
 71%|███████   | 159139/225000 [309:26:51<116:29:52,  6.37s/it]
 71%|███████   | 159140/225000 [309:26:55<105:36:48,  5.77s/it]
 71%|███████   | 159141/225000 [309:27:04<121:37:19,  6.65s/it]
 71%|███████   | 159142/225000 [309:27:07<101:40:00,  5.56s/it]
 71%|███████   | 159143/225000 [309:27:11<97:06:34,  5.31s/it] 
 71%|███████   | 159144/225000 [309:27:21<118:30:17,  6.48s/it]
 71%|███████   | 159145/225000 [309:27:24<101:43:10,  5.56s/it]
 71%|███████   | 159146/225000 [309:27:30<102:29:40,  5.60s/it]
 71%|███████   | 159147/225000 [309:27:36<107:03:36,  5.85s/it]
 71%|███████   | 159148/225000 [309:27:40<93:02:10,  5.09s/it] 
 71%|███████   | 159149/225000 [309:27:48<114:08:45,  6.24s/it]
 71%|███████   | 159150/225000 [309:27:53<104:07:10,  5.69s/it]
                                                               
{'loss': '0.792', 'grad_norm': '1.17', 'learning_rate': '1.073e-05', 'epoch': '0.7073', 'num_input_tokens_seen': 512293368, 'train_runtime': '1.114e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159150/225000 [309:27:53<104:07:10,  5.69s/it]
 71%|███████   | 159151/225000 [309:27:59<105:28:53,  5.77s/it]
 71%|███████   | 159152/225000 [309:28:03<95:43:18,  5.23s/it] 
 71%|███████   | 159153/225000 [309:28:08<95:07:28,  5.20s/it]
 71%|███████   | 159154/225000 [309:28:19<128:06:46,  7.00s/it]
 71%|███████   | 159155/225000 [309:28:23<109:33:53,  5.99s/it]
 71%|███████   | 159156/225000 [309:28:28<103:29:05,  5.66s/it]
 71%|███████   | 159157/225000 [309:28:37<121:19:42,  6.63s/it]
 71%|███████   | 159158/225000 [309:28:40<104:17:02,  5.70s/it]
 71%|███████   | 159159/225000 [309:28:47<108:27:50,  5.93s/it]
 71%|███████   | 159160/225000 [309:28:52<107:24:55,  5.87s/it]
 71%|███████   | 159161/225000 [309:28:57<101:25:31,  5.55s/it]
 71%|███████   | 159162/225000 [309:29:02<97:57:30,  5.36s/it] 
 71%|███████   | 159163/225000 [309:29:07<95:16:43,  5.21s/it]
 71%|███████   | 159164/225000 [309:29:12<93:35:28,  5.12s/it]
 71%|███████   | 159165/225000 [309:29:18<100:10:08,  5.48s/it]
 71%|███████   | 159166/225000 [309:29:29<130:25:49,  7.13s/it]
 71%|███████   | 159167/225000 [309:29:33<111:26:40,  6.09s/it]
 71%|███████   | 159168/225000 [309:29:37<102:19:48,  5.60s/it]
 71%|███████   | 159169/225000 [309:29:41<94:11:52,  5.15s/it] 
 71%|███████   | 159170/225000 [309:29:46<91:50:10,  5.02s/it]
 71%|███████   | 159171/225000 [309:29:51<89:47:32,  4.91s/it]
 71%|███████   | 159172/225000 [309:30:01<120:13:03,  6.57s/it]
 71%|███████   | 159173/225000 [309:30:07<114:33:50,  6.27s/it]
 71%|███████   | 159174/225000 [309:30:11<101:50:24,  5.57s/it]
 71%|███████   | 159175/225000 [309:30:19<114:40:04,  6.27s/it]
 71%|███████   | 159176/225000 [309:30:23<102:23:05,  5.60s/it]
 71%|███████   | 159177/225000 [309:30:27<94:51:28,  5.19s/it] 
 71%|███████   | 159178/225000 [309:30:34<106:49:22,  5.84s/it]
 71%|███████   | 159179/225000 [309:30:43<124:44:40,  6.82s/it]
 71%|███████   | 159180/225000 [309:30:47<106:35:28,  5.83s/it]
 71%|███████   | 159181/225000 [309:30:50<94:59:28,  5.20s/it] 
 71%|███████   | 159182/225000 [309:30:55<88:53:15,  4.86s/it]
 71%|███████   | 159183/225000 [309:30:59<85:53:07,  4.70s/it]
 71%|███████   | 159184/225000 [309:31:09<117:10:14,  6.41s/it]
 71%|███████   | 159185/225000 [309:31:16<116:28:59,  6.37s/it]
 71%|███████   | 159186/225000 [309:31:20<103:23:02,  5.66s/it]
 71%|███████   | 159187/225000 [309:31:26<109:54:40,  6.01s/it]
 71%|███████   | 159188/225000 [309:31:33<114:15:04,  6.25s/it]
 71%|███████   | 159189/225000 [309:31:37<100:07:01,  5.48s/it]
 71%|███████   | 159190/225000 [309:31:45<117:02:49,  6.40s/it]
 71%|███████   | 159191/225000 [309:31:50<108:48:31,  5.95s/it]
 71%|███████   | 159192/225000 [309:31:57<114:32:52,  6.27s/it]
 71%|███████   | 159193/225000 [309:32:01<100:31:34,  5.50s/it]
 71%|███████   | 159194/225000 [309:32:11<122:18:22,  6.69s/it]
 71%|███████   | 159195/225000 [309:32:15<111:09:18,  6.08s/it]
 71%|███████   | 159196/225000 [309:32:20<103:45:51,  5.68s/it]
 71%|███████   | 159197/225000 [309:32:24<97:07:55,  5.31s/it] 
 71%|███████   | 159198/225000 [309:32:29<92:54:52,  5.08s/it]
 71%|███████   | 159199/225000 [309:32:34<92:44:21,  5.07s/it]
 71%|███████   | 159200/225000 [309:32:38<88:05:53,  4.82s/it]
                                                              
{'loss': '0.7772', 'grad_norm': '1.19', 'learning_rate': '1.071e-05', 'epoch': '0.7076', 'num_input_tokens_seen': 512423392, 'train_runtime': '1.114e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159200/225000 [309:32:38<88:05:53,  4.82s/it]
 71%|███████   | 159201/225000 [309:32:44<93:43:58,  5.13s/it]
 71%|███████   | 159202/225000 [309:32:56<132:32:01,  7.25s/it]
 71%|███████   | 159203/225000 [309:33:01<120:06:32,  6.57s/it]
 71%|███████   | 159204/225000 [309:33:05<104:13:19,  5.70s/it]
 71%|███████   | 159205/225000 [309:33:14<125:00:12,  6.84s/it]
 71%|███████   | 159206/225000 [309:33:18<108:52:05,  5.96s/it]
 71%|███████   | 159207/225000 [309:33:22<96:21:58,  5.27s/it] 
 71%|███████   | 159208/225000 [309:33:27<95:08:05,  5.21s/it]
 71%|███████   | 159209/225000 [309:33:32<96:24:53,  5.28s/it]
 71%|███████   | 159210/225000 [309:33:38<98:25:33,  5.39s/it]
 71%|███████   | 159211/225000 [309:33:43<97:03:13,  5.31s/it]
 71%|███████   | 159212/225000 [309:33:49<97:49:44,  5.35s/it]
 71%|███████   | 159213/225000 [309:33:54<98:03:06,  5.37s/it]
 71%|███████   | 159214/225000 [309:33:59<96:01:50,  5.26s/it]
 71%|███████   | 159215/225000 [309:34:10<128:49:00,  7.05s/it]
 71%|███████   | 159216/225000 [309:34:14<112:53:37,  6.18s/it]
 71%|███████   | 159217/225000 [309:34:19<102:23:59,  5.60s/it]
 71%|███████   | 159218/225000 [309:34:23<96:08:10,  5.26s/it] 
 71%|███████   | 159219/225000 [309:34:36<135:45:19,  7.43s/it]
 71%|███████   | 159220/225000 [309:34:40<118:25:30,  6.48s/it]
 71%|███████   | 159221/225000 [309:34:48<125:13:03,  6.85s/it]
 71%|███████   | 159222/225000 [309:34:51<107:15:32,  5.87s/it]
 71%|███████   | 159223/225000 [309:34:57<104:31:08,  5.72s/it]
 71%|███████   | 159224/225000 [309:35:05<119:00:48,  6.51s/it]
 71%|███████   | 159225/225000 [309:35:09<104:03:43,  5.70s/it]
 71%|███████   | 159226/225000 [309:35:13<93:51:01,  5.14s/it] 
 71%|███████   | 159227/225000 [309:35:17<87:40:12,  4.80s/it]
 71%|███████   | 159228/225000 [309:35:27<116:33:01,  6.38s/it]
 71%|███████   | 159229/225000 [309:35:30<102:06:13,  5.59s/it]
 71%|███████   | 159230/225000 [309:35:34<92:03:15,  5.04s/it] 
 71%|███████   | 159231/225000 [309:35:44<118:07:22,  6.47s/it]
 71%|███████   | 159232/225000 [309:35:48<105:45:26,  5.79s/it]
 71%|███████   | 159233/225000 [309:35:53<99:12:38,  5.43s/it] 
 71%|███████   | 159234/225000 [309:35:57<95:07:37,  5.21s/it]
 71%|███████   | 159235/225000 [309:36:02<89:05:33,  4.88s/it]
 71%|███████   | 159236/225000 [309:36:06<86:52:11,  4.76s/it]
 71%|███████   | 159237/225000 [309:36:14<102:21:34,  5.60s/it]
 71%|███████   | 159238/225000 [309:36:24<127:32:20,  6.98s/it]
 71%|███████   | 159239/225000 [309:36:27<106:30:49,  5.83s/it]
 71%|███████   | 159240/225000 [309:36:34<113:58:02,  6.24s/it]
 71%|███████   | 159241/225000 [309:36:41<116:43:01,  6.39s/it]
 71%|███████   | 159242/225000 [309:36:44<99:11:24,  5.43s/it] 
 71%|███████   | 159243/225000 [309:36:49<98:28:05,  5.39s/it]
 71%|███████   | 159244/225000 [309:36:55<98:54:42,  5.42s/it]
 71%|███████   | 159245/225000 [309:37:00<98:19:28,  5.38s/it]
 71%|███████   | 159246/225000 [309:37:12<131:12:12,  7.18s/it]
 71%|███████   | 159247/225000 [309:37:15<109:52:07,  6.02s/it]
 71%|███████   | 159248/225000 [309:37:18<96:10:28,  5.27s/it] 
 71%|███████   | 159249/225000 [309:37:29<125:29:03,  6.87s/it]
 71%|███████   | 159250/225000 [309:37:33<110:42:04,  6.06s/it]
                                                               
{'loss': '0.7929', 'grad_norm': '1.14', 'learning_rate': '1.07e-05', 'epoch': '0.7078', 'num_input_tokens_seen': 512555376, 'train_runtime': '1.115e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159250/225000 [309:37:33<110:42:04,  6.06s/it]
 71%|███████   | 159251/225000 [309:37:38<103:15:59,  5.65s/it]
 71%|███████   | 159252/225000 [309:37:43<100:27:21,  5.50s/it]
 71%|███████   | 159253/225000 [309:37:48<95:46:41,  5.24s/it] 
 71%|███████   | 159254/225000 [309:37:53<94:25:41,  5.17s/it]
 71%|███████   | 159255/225000 [309:37:58<94:41:46,  5.19s/it]
 71%|███████   | 159256/225000 [309:38:03<93:52:38,  5.14s/it]
 71%|███████   | 159257/225000 [309:38:08<92:08:18,  5.05s/it]
 71%|███████   | 159258/225000 [309:38:13<92:17:26,  5.05s/it]
 71%|███████   | 159259/225000 [309:38:18<93:50:57,  5.14s/it]
 71%|███████   | 159260/225000 [309:38:23<93:28:52,  5.12s/it]
 71%|███████   | 159261/225000 [309:38:29<95:16:08,  5.22s/it]
 71%|███████   | 159262/225000 [309:38:33<92:38:32,  5.07s/it]
 71%|███████   | 159263/225000 [309:38:42<114:09:32,  6.25s/it]
 71%|███████   | 159264/225000 [309:38:54<145:06:45,  7.95s/it]
 71%|███████   | 159265/225000 [309:38:58<120:21:40,  6.59s/it]
 71%|███████   | 159266/225000 [309:39:06<129:18:34,  7.08s/it]
 71%|███████   | 159267/225000 [309:39:10<111:08:29,  6.09s/it]
 71%|███████   | 159268/225000 [309:39:15<107:30:26,  5.89s/it]
 71%|███████   | 159269/225000 [309:39:23<119:33:11,  6.55s/it]
 71%|███████   | 159270/225000 [309:39:26<101:21:44,  5.55s/it]
 71%|███████   | 159271/225000 [309:39:33<106:55:35,  5.86s/it]
 71%|███████   | 159272/225000 [309:39:39<109:16:43,  5.99s/it]
 71%|███████   | 159273/225000 [309:39:45<110:07:00,  6.03s/it]
 71%|███████   | 159274/225000 [309:39:50<101:30:12,  5.56s/it]
 71%|███████   | 159275/225000 [309:39:56<105:51:18,  5.80s/it]
 71%|███████   | 159276/225000 [309:40:00<95:59:03,  5.26s/it] 
 71%|███████   | 159277/225000 [309:40:10<119:02:14,  6.52s/it]
 71%|███████   | 159278/225000 [309:40:13<103:56:29,  5.69s/it]
 71%|███████   | 159279/225000 [309:40:17<93:55:40,  5.15s/it] 
 71%|███████   | 159280/225000 [309:40:21<86:18:45,  4.73s/it]
 71%|███████   | 159281/225000 [309:40:32<118:05:00,  6.47s/it]
 71%|███████   | 159282/225000 [309:40:36<104:23:01,  5.72s/it]
 71%|███████   | 159283/225000 [309:40:40<95:53:32,  5.25s/it] 
 71%|███████   | 159284/225000 [309:40:44<89:38:17,  4.91s/it]
 71%|███████   | 159285/225000 [309:40:49<93:10:55,  5.10s/it]
 71%|███████   | 159286/225000 [309:41:02<135:41:09,  7.43s/it]
 71%|███████   | 159287/225000 [309:41:06<115:27:27,  6.33s/it]
 71%|███████   | 159288/225000 [309:41:10<101:39:56,  5.57s/it]
 71%|███████   | 159289/225000 [309:41:16<103:21:36,  5.66s/it]
 71%|███████   | 159290/225000 [309:41:24<116:34:25,  6.39s/it]
 71%|███████   | 159291/225000 [309:41:28<101:56:38,  5.59s/it]
 71%|███████   | 159292/225000 [309:41:37<121:31:42,  6.66s/it]
 71%|███████   | 159293/225000 [309:41:40<104:46:23,  5.74s/it]
 71%|███████   | 159294/225000 [309:41:44<95:19:00,  5.22s/it] 
 71%|███████   | 159295/225000 [309:41:51<103:04:32,  5.65s/it]
 71%|███████   | 159296/225000 [309:42:01<127:42:43,  7.00s/it]
 71%|███████   | 159297/225000 [309:42:05<110:48:06,  6.07s/it]
 71%|███████   | 159298/225000 [309:42:11<112:23:11,  6.16s/it]
 71%|███████   | 159299/225000 [309:42:15<97:27:25,  5.34s/it] 
 71%|███████   | 159300/225000 [309:42:23<111:57:07,  6.13s/it]
                                                               
{'loss': '0.7851', 'grad_norm': '1.03', 'learning_rate': '1.068e-05', 'epoch': '0.708', 'num_input_tokens_seen': 512685904, 'train_runtime': '1.115e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159300/225000 [309:42:23<111:57:07,  6.13s/it]
 71%|███████   | 159301/225000 [309:42:26<96:30:22,  5.29s/it] 
 71%|███████   | 159302/225000 [309:42:34<112:53:33,  6.19s/it]
 71%|███████   | 159303/225000 [309:42:39<103:12:27,  5.66s/it]
 71%|███████   | 159304/225000 [309:42:43<94:41:58,  5.19s/it] 
 71%|███████   | 159305/225000 [309:42:47<89:42:23,  4.92s/it]
 71%|███████   | 159306/225000 [309:42:55<107:15:35,  5.88s/it]
 71%|███████   | 159307/225000 [309:43:04<120:55:29,  6.63s/it]
 71%|███████   | 159308/225000 [309:43:10<117:21:29,  6.43s/it]
 71%|███████   | 159309/225000 [309:43:15<108:43:33,  5.96s/it]
 71%|███████   | 159310/225000 [309:43:19<98:40:26,  5.41s/it] 
 71%|███████   | 159311/225000 [309:43:23<92:28:34,  5.07s/it]
 71%|███████   | 159312/225000 [309:43:31<110:22:07,  6.05s/it]
 71%|███████   | 159313/225000 [309:43:37<107:12:20,  5.88s/it]
 71%|███████   | 159314/225000 [309:43:42<104:14:03,  5.71s/it]
 71%|███████   | 159315/225000 [309:43:47<100:40:20,  5.52s/it]
 71%|███████   | 159316/225000 [309:43:53<100:00:11,  5.48s/it]
 71%|███████   | 159317/225000 [309:43:58<99:10:12,  5.44s/it] 
 71%|███████   | 159318/225000 [309:44:03<98:12:25,  5.38s/it]
 71%|███████   | 159319/225000 [309:44:08<96:39:54,  5.30s/it]
 71%|███████   | 159320/225000 [309:44:13<96:01:37,  5.26s/it]
 71%|███████   | 159321/225000 [309:44:18<94:23:31,  5.17s/it]
 71%|███████   | 159322/225000 [309:44:24<95:34:14,  5.24s/it]
 71%|███████   | 159323/225000 [309:44:29<94:41:46,  5.19s/it]
 71%|███████   | 159324/225000 [309:44:40<128:29:06,  7.04s/it]
 71%|███████   | 159325/225000 [309:44:47<128:20:25,  7.04s/it]
 71%|███████   | 159326/225000 [309:44:52<116:05:11,  6.36s/it]
 71%|███████   | 159327/225000 [309:44:57<107:55:30,  5.92s/it]
 71%|███████   | 159328/225000 [309:45:02<103:06:40,  5.65s/it]
 71%|███████   | 159329/225000 [309:45:07<101:27:59,  5.56s/it]
 71%|███████   | 159330/225000 [309:45:12<97:27:18,  5.34s/it] 
 71%|███████   | 159331/225000 [309:45:17<95:48:50,  5.25s/it]
 71%|███████   | 159332/225000 [309:45:22<93:08:40,  5.11s/it]
 71%|███████   | 159333/225000 [309:45:27<93:52:11,  5.15s/it]
 71%|███████   | 159334/225000 [309:45:36<114:05:34,  6.25s/it]
 71%|███████   | 159335/225000 [309:45:45<130:46:50,  7.17s/it]
 71%|███████   | 159336/225000 [309:45:50<116:29:35,  6.39s/it]
 71%|███████   | 159337/225000 [309:45:54<104:29:46,  5.73s/it]
 71%|███████   | 159338/225000 [309:45:59<99:27:47,  5.45s/it] 
 71%|███████   | 159339/225000 [309:46:03<94:28:51,  5.18s/it]
 71%|███████   | 159340/225000 [309:46:09<96:03:04,  5.27s/it]
 71%|███████   | 159341/225000 [309:46:20<128:08:31,  7.03s/it]
 71%|███████   | 159342/225000 [309:46:24<111:37:05,  6.12s/it]
 71%|███████   | 159343/225000 [309:46:28<98:05:46,  5.38s/it] 
 71%|███████   | 159344/225000 [309:46:34<106:07:17,  5.82s/it]
 71%|███████   | 159345/225000 [309:46:41<109:47:00,  6.02s/it]
 71%|███████   | 159346/225000 [309:46:45<96:14:22,  5.28s/it] 
 71%|███████   | 159347/225000 [309:46:51<100:19:11,  5.50s/it]
 71%|███████   | 159348/225000 [309:46:59<117:01:04,  6.42s/it]
 71%|███████   | 159349/225000 [309:47:03<104:20:41,  5.72s/it]
 71%|███████   | 159350/225000 [309:47:07<92:12:00,  5.06s/it] 
                                                              
{'loss': '0.7633', 'grad_norm': '1.057', 'learning_rate': '1.067e-05', 'epoch': '0.7082', 'num_input_tokens_seen': 512817872, 'train_runtime': '1.115e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159350/225000 [309:47:07<92:12:00,  5.06s/it]
 71%|███████   | 159351/225000 [309:47:11<87:22:18,  4.79s/it]
 71%|███████   | 159352/225000 [309:47:16<88:57:54,  4.88s/it]
 71%|███████   | 159353/225000 [309:47:21<89:27:10,  4.91s/it]
 71%|███████   | 159354/225000 [309:47:26<90:12:56,  4.95s/it]
 71%|███████   | 159355/225000 [309:47:33<102:11:55,  5.60s/it]
 71%|███████   | 159356/225000 [309:47:44<132:47:41,  7.28s/it]
 71%|███████   | 159357/225000 [309:47:50<126:16:17,  6.92s/it]
 71%|███████   | 159358/225000 [309:47:54<105:44:30,  5.80s/it]
 71%|███████   | 159359/225000 [309:48:00<111:21:36,  6.11s/it]
 71%|███████   | 159360/225000 [309:48:04<97:08:20,  5.33s/it] 
 71%|███████   | 159361/225000 [309:48:10<103:01:32,  5.65s/it]
 71%|███████   | 159362/225000 [309:48:18<112:04:50,  6.15s/it]
 71%|███████   | 159363/225000 [309:48:21<98:42:04,  5.41s/it] 
 71%|███████   | 159364/225000 [309:48:29<108:37:52,  5.96s/it]
 71%|███████   | 159365/225000 [309:48:34<103:41:46,  5.69s/it]
 71%|███████   | 159366/225000 [309:48:37<93:33:59,  5.13s/it] 
 71%|███████   | 159367/225000 [309:48:47<117:45:54,  6.46s/it]
 71%|███████   | 159368/225000 [309:48:50<101:10:30,  5.55s/it]
 71%|███████   | 159369/225000 [309:48:57<108:43:56,  5.96s/it]
 71%|███████   | 159370/225000 [309:49:02<102:07:23,  5.60s/it]
 71%|███████   | 159371/225000 [309:49:06<90:15:01,  4.95s/it] 
 71%|███████   | 159372/225000 [309:49:11<91:49:56,  5.04s/it]
 71%|███████   | 159373/225000 [309:49:16<94:07:39,  5.16s/it]
 71%|███████   | 159374/225000 [309:49:22<95:32:50,  5.24s/it]
 71%|███████   | 159375/225000 [309:49:31<120:00:02,  6.58s/it]
 71%|███████   | 159376/225000 [309:49:35<104:36:15,  5.74s/it]
 71%|███████   | 159377/225000 [309:49:43<117:04:18,  6.42s/it]
 71%|███████   | 159378/225000 [309:49:49<112:00:43,  6.14s/it]
 71%|███████   | 159379/225000 [309:49:54<107:50:45,  5.92s/it]
 71%|███████   | 159380/225000 [309:49:58<96:55:10,  5.32s/it] 
 71%|███████   | 159381/225000 [309:50:04<102:55:55,  5.65s/it]
 71%|███████   | 159382/225000 [309:50:08<93:18:37,  5.12s/it] 
 71%|███████   | 159383/225000 [309:50:19<121:59:35,  6.69s/it]
 71%|███████   | 159384/225000 [309:50:23<107:35:54,  5.90s/it]
 71%|███████   | 159385/225000 [309:50:26<94:38:31,  5.19s/it] 
 71%|███████   | 159386/225000 [309:50:30<88:06:13,  4.83s/it]
 71%|███████   | 159387/225000 [309:50:41<119:26:17,  6.55s/it]
 71%|███████   | 159388/225000 [309:50:44<99:09:30,  5.44s/it] 
 71%|███████   | 159389/225000 [309:50:51<107:42:58,  5.91s/it]
 71%|███████   | 159390/225000 [309:50:58<113:01:49,  6.20s/it]
 71%|███████   | 159391/225000 [309:51:02<104:34:59,  5.74s/it]
 71%|███████   | 159392/225000 [309:51:07<98:24:22,  5.40s/it] 
 71%|███████   | 159393/225000 [309:51:11<94:10:09,  5.17s/it]
 71%|███████   | 159394/225000 [309:51:16<91:48:25,  5.04s/it]
 71%|███████   | 159395/225000 [309:51:21<90:17:05,  4.95s/it]
 71%|███████   | 159396/225000 [309:51:30<115:33:49,  6.34s/it]
 71%|███████   | 159397/225000 [309:51:37<116:10:06,  6.37s/it]
 71%|███████   | 159398/225000 [309:51:42<106:27:31,  5.84s/it]
 71%|███████   | 159399/225000 [309:51:47<104:13:12,  5.72s/it]
 71%|███████   | 159400/225000 [309:51:52<100:43:35,  5.53s/it]
                                                               
{'loss': '0.7804', 'grad_norm': '1.079', 'learning_rate': '1.065e-05', 'epoch': '0.7084', 'num_input_tokens_seen': 512948440, 'train_runtime': '1.116e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159400/225000 [309:51:52<100:43:35,  5.53s/it]
 71%|███████   | 159401/225000 [309:51:57<98:46:36,  5.42s/it] 
 71%|███████   | 159402/225000 [309:52:02<96:30:21,  5.30s/it]
 71%|███████   | 159403/225000 [309:52:08<96:36:34,  5.30s/it]
 71%|███████   | 159404/225000 [309:52:13<96:45:56,  5.31s/it]
 71%|███████   | 159405/225000 [309:52:18<95:17:09,  5.23s/it]
 71%|███████   | 159406/225000 [309:52:23<94:51:31,  5.21s/it]
 71%|███████   | 159407/225000 [309:52:30<105:12:33,  5.77s/it]
 71%|███████   | 159408/225000 [309:52:43<142:20:20,  7.81s/it]
 71%|███████   | 159409/225000 [309:52:46<118:39:24,  6.51s/it]
 71%|███████   | 159410/225000 [309:52:53<119:35:25,  6.56s/it]
 71%|███████   | 159411/225000 [309:52:57<104:26:07,  5.73s/it]
 71%|███████   | 159412/225000 [309:53:07<127:05:25,  6.98s/it]
 71%|███████   | 159413/225000 [309:53:10<106:17:30,  5.83s/it]
 71%|███████   | 159414/225000 [309:53:17<111:30:57,  6.12s/it]
 71%|███████   | 159415/225000 [309:53:23<114:34:51,  6.29s/it]
 71%|███████   | 159416/225000 [309:53:27<102:40:25,  5.64s/it]
 71%|███████   | 159417/225000 [309:53:31<94:42:36,  5.20s/it] 
 71%|███████   | 159418/225000 [309:53:35<87:58:54,  4.83s/it]
 71%|███████   | 159419/225000 [309:53:48<131:58:26,  7.24s/it]
 71%|███████   | 159420/225000 [309:53:52<110:50:20,  6.08s/it]
 71%|███████   | 159421/225000 [309:53:55<97:11:13,  5.34s/it] 
 71%|███████   | 159422/225000 [309:54:04<117:30:47,  6.45s/it]
 71%|███████   | 159423/225000 [309:54:08<104:13:16,  5.72s/it]
 71%|███████   | 159424/225000 [309:54:12<95:31:00,  5.24s/it] 
 71%|███████   | 159425/225000 [309:54:17<92:59:58,  5.11s/it]
 71%|███████   | 159426/225000 [309:54:22<89:39:06,  4.92s/it]
 71%|███████   | 159427/225000 [309:54:26<86:45:50,  4.76s/it]
 71%|███████   | 159428/225000 [309:54:31<85:50:55,  4.71s/it]
 71%|███████   | 159429/225000 [309:54:42<122:59:54,  6.75s/it]
 71%|███████   | 159430/225000 [309:54:48<116:54:31,  6.42s/it]
 71%|███████   | 159431/225000 [309:54:53<107:06:47,  5.88s/it]
 71%|███████   | 159432/225000 [309:55:00<115:23:09,  6.34s/it]
 71%|███████   | 159433/225000 [309:55:04<104:28:12,  5.74s/it]
 71%|███████   | 159434/225000 [309:55:09<100:33:00,  5.52s/it]
 71%|███████   | 159435/225000 [309:55:14<98:41:47,  5.42s/it] 
 71%|███████   | 159436/225000 [309:55:20<99:49:37,  5.48s/it]
 71%|███████   | 159437/225000 [309:55:25<98:11:53,  5.39s/it]
 71%|███████   | 159438/225000 [309:55:34<117:04:38,  6.43s/it]
 71%|███████   | 159439/225000 [309:55:40<112:22:50,  6.17s/it]
 71%|███████   | 159440/225000 [309:55:47<117:02:10,  6.43s/it]
 71%|███████   | 159441/225000 [309:55:50<101:47:33,  5.59s/it]
 71%|███████   | 159442/225000 [309:55:58<110:17:23,  6.06s/it]
 71%|███████   | 159443/225000 [309:56:03<109:33:34,  6.02s/it]
 71%|███████   | 159444/225000 [309:56:07<95:56:49,  5.27s/it] 
 71%|███████   | 159445/225000 [309:56:13<99:46:35,  5.48s/it]
 71%|███████   | 159446/225000 [309:56:20<109:54:52,  6.04s/it]
 71%|███████   | 159447/225000 [309:56:25<101:13:18,  5.56s/it]
 71%|███████   | 159448/225000 [309:56:30<98:06:01,  5.39s/it] 
 71%|███████   | 159449/225000 [309:56:35<97:29:48,  5.35s/it]
 71%|███████   | 159450/225000 [309:56:40<94:16:59,  5.18s/it]
                                                              
{'loss': '0.7801', 'grad_norm': '1.1', 'learning_rate': '1.064e-05', 'epoch': '0.7087', 'num_input_tokens_seen': 513078032, 'train_runtime': '1.116e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159450/225000 [309:56:40<94:16:59,  5.18s/it]
 71%|███████   | 159451/225000 [309:56:45<92:41:51,  5.09s/it]
 71%|███████   | 159452/225000 [309:56:49<91:23:48,  5.02s/it]
 71%|███████   | 159453/225000 [309:56:55<94:04:32,  5.17s/it]
 71%|███████   | 159454/225000 [309:57:00<94:40:39,  5.20s/it]
 71%|███████   | 159455/225000 [309:57:05<93:06:21,  5.11s/it]
 71%|███████   | 159456/225000 [309:57:10<89:49:20,  4.93s/it]
 71%|███████   | 159457/225000 [309:57:15<90:02:45,  4.95s/it]
 71%|███████   | 159458/225000 [309:57:20<93:18:09,  5.12s/it]
 71%|███████   | 159459/225000 [309:57:25<94:10:21,  5.17s/it]
 71%|███████   | 159460/225000 [309:57:35<116:27:59,  6.40s/it]
 71%|███████   | 159461/225000 [309:57:44<129:46:56,  7.13s/it]
 71%|███████   | 159462/225000 [309:57:47<110:20:10,  6.06s/it]
 71%|███████   | 159463/225000 [309:57:56<125:38:25,  6.90s/it]
 71%|███████   | 159464/225000 [309:58:00<109:01:27,  5.99s/it]
 71%|███████   | 159465/225000 [309:58:04<98:34:39,  5.42s/it] 
 71%|███████   | 159466/225000 [309:58:09<93:56:47,  5.16s/it]
 71%|███████   | 159467/225000 [309:58:19<121:05:42,  6.65s/it]
 71%|███████   | 159468/225000 [309:58:22<104:56:44,  5.77s/it]
 71%|███████   | 159469/225000 [309:58:30<117:18:05,  6.44s/it]
 71%|███████   | 159470/225000 [309:58:35<107:38:58,  5.91s/it]
 71%|███████   | 159471/225000 [309:58:42<110:55:11,  6.09s/it]
 71%|███████   | 159472/225000 [309:58:46<102:53:54,  5.65s/it]
 71%|███████   | 159473/225000 [309:58:50<94:16:33,  5.18s/it] 
 71%|███████   | 159474/225000 [309:58:55<90:26:32,  4.97s/it]
 71%|███████   | 159475/225000 [309:59:00<90:52:00,  4.99s/it]
 71%|███████   | 159476/225000 [309:59:11<127:07:10,  6.98s/it]
 71%|███████   | 159477/225000 [309:59:15<107:40:30,  5.92s/it]
 71%|███████   | 159478/225000 [309:59:18<94:51:27,  5.21s/it] 
 71%|███████   | 159479/225000 [309:59:27<113:11:29,  6.22s/it]
 71%|███████   | 159480/225000 [309:59:30<98:04:04,  5.39s/it] 
 71%|███████   | 159481/225000 [309:59:39<115:43:51,  6.36s/it]
 71%|███████   | 159482/225000 [309:59:43<103:40:44,  5.70s/it]
 71%|███████   | 159483/225000 [309:59:48<96:39:45,  5.31s/it] 
 71%|███████   | 159484/225000 [309:59:52<91:53:05,  5.05s/it]
 71%|███████   | 159485/225000 [309:59:57<88:59:43,  4.89s/it]
 71%|███████   | 159486/225000 [310:00:01<86:13:13,  4.74s/it]
 71%|███████   | 159487/225000 [310:00:11<113:42:19,  6.25s/it]
 71%|███████   | 159488/225000 [310:00:17<116:12:24,  6.39s/it]
 71%|███████   | 159489/225000 [310:00:21<99:36:50,  5.47s/it] 
 71%|███████   | 159490/225000 [310:00:24<89:15:16,  4.90s/it]
 71%|███████   | 159491/225000 [310:00:34<115:47:17,  6.36s/it]
 71%|███████   | 159492/225000 [310:00:39<109:23:54,  6.01s/it]
 71%|███████   | 159493/225000 [310:00:46<111:59:58,  6.16s/it]
 71%|███████   | 159494/225000 [310:00:50<103:13:00,  5.67s/it]
 71%|███████   | 159495/225000 [310:00:58<112:24:42,  6.18s/it]
 71%|███████   | 159496/225000 [310:01:02<99:43:09,  5.48s/it] 
 71%|███████   | 159497/225000 [310:01:07<99:39:09,  5.48s/it]
 71%|███████   | 159498/225000 [310:01:13<100:31:29,  5.52s/it]
 71%|███████   | 159499/225000 [310:01:20<112:29:11,  6.18s/it]
 71%|███████   | 159500/225000 [310:01:26<110:01:22,  6.05s/it]
                                                               
{'loss': '0.778', 'grad_norm': '1.195', 'learning_rate': '1.062e-05', 'epoch': '0.7089', 'num_input_tokens_seen': 513206984, 'train_runtime': '1.116e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159500/225000 [310:01:26<110:01:22,  6.05s/it]
 71%|███████   | 159501/225000 [310:01:30<98:48:31,  5.43s/it] 
 71%|███████   | 159502/225000 [310:01:35<94:00:39,  5.17s/it]
 71%|███████   | 159503/225000 [310:01:40<93:50:53,  5.16s/it]
 71%|███████   | 159504/225000 [310:01:50<122:38:00,  6.74s/it]
 71%|███████   | 159505/225000 [310:01:54<103:55:05,  5.71s/it]
 71%|███████   | 159506/225000 [310:02:02<119:35:33,  6.57s/it]
 71%|███████   | 159507/225000 [310:02:06<104:32:00,  5.75s/it]
 71%|███████   | 159508/225000 [310:02:13<109:39:07,  6.03s/it]
 71%|███████   | 159509/225000 [310:02:18<105:26:41,  5.80s/it]
 71%|███████   | 159510/225000 [310:02:22<97:14:13,  5.35s/it] 
 71%|███████   | 159511/225000 [310:02:27<94:31:41,  5.20s/it]
 71%|███████   | 159512/225000 [310:02:32<94:45:55,  5.21s/it]
 71%|███████   | 159513/225000 [310:02:37<93:56:34,  5.16s/it]
 71%|███████   | 159514/225000 [310:02:42<92:21:12,  5.08s/it]
 71%|███████   | 159515/225000 [310:02:47<93:05:56,  5.12s/it]
 71%|███████   | 159516/225000 [310:02:52<91:26:46,  5.03s/it]
 71%|███████   | 159517/225000 [310:02:59<99:27:34,  5.47s/it]
 71%|███████   | 159518/225000 [310:03:11<133:58:20,  7.37s/it]
 71%|███████   | 159519/225000 [310:03:14<111:29:15,  6.13s/it]
 71%|███████   | 159520/225000 [310:03:17<98:12:55,  5.40s/it] 
 71%|███████   | 159521/225000 [310:03:22<92:42:27,  5.10s/it]
 71%|███████   | 159522/225000 [310:03:32<118:57:24,  6.54s/it]
 71%|███████   | 159523/225000 [310:03:39<123:07:34,  6.77s/it]
 71%|███████   | 159524/225000 [310:03:43<106:21:41,  5.85s/it]
 71%|███████   | 159525/225000 [310:03:51<121:32:43,  6.68s/it]
 71%|███████   | 159526/225000 [310:03:55<104:53:51,  5.77s/it]
 71%|███████   | 159527/225000 [310:04:03<115:37:20,  6.36s/it]
 71%|███████   | 159528/225000 [310:04:08<107:46:03,  5.93s/it]
 71%|███████   | 159529/225000 [310:04:12<101:31:15,  5.58s/it]
 71%|███████   | 159530/225000 [310:04:17<98:17:25,  5.40s/it] 
 71%|███████   | 159531/225000 [310:04:22<94:19:45,  5.19s/it]
 71%|███████   | 159532/225000 [310:04:27<93:22:23,  5.13s/it]
 71%|███████   | 159533/225000 [310:04:32<91:32:31,  5.03s/it]
 71%|███████   | 159534/225000 [310:04:37<90:56:51,  5.00s/it]
 71%|███████   | 159535/225000 [310:04:42<89:34:17,  4.93s/it]
 71%|███████   | 159536/225000 [310:04:46<87:26:54,  4.81s/it]
 71%|███████   | 159537/225000 [310:04:54<104:59:32,  5.77s/it]
 71%|███████   | 159538/225000 [310:05:06<138:13:31,  7.60s/it]
 71%|███████   | 159539/225000 [310:05:09<115:17:32,  6.34s/it]
 71%|███████   | 159540/225000 [310:05:13<101:44:48,  5.60s/it]
 71%|███████   | 159541/225000 [310:05:22<120:12:19,  6.61s/it]
 71%|███████   | 159542/225000 [310:05:25<100:45:38,  5.54s/it]
 71%|███████   | 159543/225000 [310:05:33<113:47:56,  6.26s/it]
 71%|███████   | 159544/225000 [310:05:37<102:38:31,  5.65s/it]
 71%|███████   | 159545/225000 [310:05:46<116:31:31,  6.41s/it]
 71%|███████   | 159546/225000 [310:05:49<100:57:00,  5.55s/it]
 71%|███████   | 159547/225000 [310:05:53<92:52:22,  5.11s/it] 
 71%|███████   | 159548/225000 [310:06:02<110:17:22,  6.07s/it]
 71%|███████   | 159549/225000 [310:06:06<101:12:12,  5.57s/it]
 71%|███████   | 159550/225000 [310:06:10<94:10:51,  5.18s/it] 
                                                              
{'loss': '0.7722', 'grad_norm': '1.031', 'learning_rate': '1.061e-05', 'epoch': '0.7091', 'num_input_tokens_seen': 513338536, 'train_runtime': '1.116e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159550/225000 [310:06:10<94:10:51,  5.18s/it]
 71%|███████   | 159551/225000 [310:06:15<89:35:10,  4.93s/it]
 71%|███████   | 159552/225000 [310:06:19<86:25:15,  4.75s/it]
 71%|███████   | 159553/225000 [310:06:30<122:51:18,  6.76s/it]
 71%|███████   | 159554/225000 [310:06:36<115:15:19,  6.34s/it]
 71%|███████   | 159555/225000 [310:06:39<100:00:15,  5.50s/it]
 71%|███████   | 159556/225000 [310:06:44<95:38:44,  5.26s/it] 
 71%|███████   | 159557/225000 [310:06:48<90:17:28,  4.97s/it]
 71%|███████   | 159558/225000 [310:06:53<90:09:01,  4.96s/it]
 71%|███████   | 159559/225000 [310:06:59<92:06:21,  5.07s/it]
 71%|███████   | 159560/225000 [310:07:04<94:55:28,  5.22s/it]
 71%|███████   | 159561/225000 [310:07:09<91:49:41,  5.05s/it]
 71%|███████   | 159562/225000 [310:07:14<91:52:43,  5.05s/it]
 71%|███████   | 159563/225000 [310:07:20<96:04:19,  5.29s/it]
 71%|███████   | 159564/225000 [310:07:25<95:46:17,  5.27s/it]
 71%|███████   | 159565/225000 [310:07:30<94:54:55,  5.22s/it]
 71%|███████   | 159566/225000 [310:07:37<104:03:45,  5.73s/it]
 71%|███████   | 159567/225000 [310:07:50<142:48:40,  7.86s/it]
 71%|███████   | 159568/225000 [310:07:54<121:45:24,  6.70s/it]
 71%|███████   | 159569/225000 [310:07:58<105:59:21,  5.83s/it]
 71%|███████   | 159570/225000 [310:08:01<93:52:33,  5.17s/it] 
 71%|███████   | 159571/225000 [310:08:13<127:51:58,  7.04s/it]
 71%|███████   | 159572/225000 [310:08:17<112:48:31,  6.21s/it]
 71%|███████   | 159573/225000 [310:08:21<100:42:34,  5.54s/it]
 71%|███████   | 159574/225000 [310:08:25<94:07:28,  5.18s/it] 
 71%|███████   | 159575/225000 [310:08:29<89:19:36,  4.92s/it]
 71%|███████   | 159576/225000 [310:08:38<110:35:11,  6.09s/it]
 71%|███████   | 159577/225000 [310:08:44<110:26:55,  6.08s/it]
 71%|███████   | 159578/225000 [310:08:49<102:02:28,  5.62s/it]
 71%|███████   | 159579/225000 [310:08:54<97:44:09,  5.38s/it] 
 71%|███████   | 159580/225000 [310:08:58<94:32:37,  5.20s/it]
 71%|███████   | 159581/225000 [310:09:06<105:39:28,  5.81s/it]
 71%|███████   | 159582/225000 [310:09:14<117:32:41,  6.47s/it]
 71%|███████   | 159583/225000 [310:09:21<120:48:59,  6.65s/it]
 71%|███████   | 159584/225000 [310:09:24<104:50:27,  5.77s/it]
 71%|███████   | 159585/225000 [310:09:29<96:07:52,  5.29s/it] 
 71%|███████   | 159586/225000 [310:09:34<95:28:34,  5.25s/it]
 71%|███████   | 159587/225000 [310:09:40<101:19:03,  5.58s/it]
 71%|███████   | 159588/225000 [310:09:48<112:25:25,  6.19s/it]
 71%|███████   | 159589/225000 [310:09:52<99:48:25,  5.49s/it] 
 71%|███████   | 159590/225000 [310:09:59<110:28:01,  6.08s/it]
 71%|███████   | 159591/225000 [310:10:05<108:21:56,  5.96s/it]
 71%|███████   | 159592/225000 [310:10:09<97:19:19,  5.36s/it] 
 71%|███████   | 159593/225000 [310:10:15<101:26:01,  5.58s/it]
 71%|███████   | 159594/225000 [310:10:22<112:11:06,  6.17s/it]
 71%|███████   | 159595/225000 [310:10:27<101:32:23,  5.59s/it]
 71%|███████   | 159596/225000 [310:10:31<94:21:34,  5.19s/it] 
 71%|███████   | 159597/225000 [310:10:35<88:10:54,  4.85s/it]
 71%|███████   | 159598/225000 [310:10:46<121:37:53,  6.70s/it]
 71%|███████   | 159599/225000 [310:10:50<108:43:37,  5.98s/it]
 71%|███████   | 159600/225000 [310:10:58<119:58:43,  6.60s/it]
                                                               
{'loss': '0.7733', 'grad_norm': '1.064', 'learning_rate': '1.059e-05', 'epoch': '0.7093', 'num_input_tokens_seen': 513468768, 'train_runtime': '1.117e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159600/225000 [310:10:58<119:58:43,  6.60s/it]
 71%|███████   | 159601/225000 [310:11:02<102:50:37,  5.66s/it]
 71%|███████   | 159602/225000 [310:11:10<114:10:58,  6.29s/it]
 71%|███████   | 159603/225000 [310:11:13<98:12:48,  5.41s/it] 
 71%|███████   | 159604/225000 [310:11:17<91:30:10,  5.04s/it]
 71%|███████   | 159605/225000 [310:11:26<114:47:38,  6.32s/it]
 71%|███████   | 159606/225000 [310:11:31<102:53:23,  5.66s/it]
 71%|███████   | 159607/225000 [310:11:37<109:10:37,  6.01s/it]
 71%|███████   | 159608/225000 [310:11:42<103:55:44,  5.72s/it]
 71%|███████   | 159609/225000 [310:11:49<111:26:51,  6.14s/it]
 71%|███████   | 159610/225000 [310:11:55<110:48:56,  6.10s/it]
 71%|███████   | 159611/225000 [310:11:59<96:43:31,  5.33s/it] 
 71%|███████   | 159612/225000 [310:12:07<112:22:24,  6.19s/it]
 71%|███████   | 159613/225000 [310:12:11<97:20:33,  5.36s/it] 
 71%|███████   | 159614/225000 [310:12:18<107:14:52,  5.90s/it]
 71%|███████   | 159615/225000 [310:12:21<95:11:05,  5.24s/it] 
 71%|███████   | 159616/225000 [310:12:28<104:02:01,  5.73s/it]
 71%|███████   | 159617/225000 [310:12:34<103:04:38,  5.68s/it]
 71%|███████   | 159618/225000 [310:12:41<108:08:42,  5.95s/it]
 71%|███████   | 159619/225000 [310:12:44<94:19:28,  5.19s/it] 
 71%|███████   | 159620/225000 [310:12:48<88:40:37,  4.88s/it]
 71%|███████   | 159621/225000 [310:12:58<115:06:29,  6.34s/it]
 71%|███████   | 159622/225000 [310:13:03<110:02:27,  6.06s/it]
 71%|███████   | 159623/225000 [310:13:08<105:15:52,  5.80s/it]
 71%|███████   | 159624/225000 [310:13:14<102:14:28,  5.63s/it]
 71%|███████   | 159625/225000 [310:13:19<100:39:43,  5.54s/it]
 71%|███████   | 159626/225000 [310:13:24<100:16:22,  5.52s/it]
 71%|███████   | 159627/225000 [310:13:29<97:14:04,  5.35s/it] 
 71%|███████   | 159628/225000 [310:13:35<95:37:27,  5.27s/it]
 71%|███████   | 159629/225000 [310:13:40<95:36:03,  5.26s/it]
 71%|███████   | 159630/225000 [310:13:45<95:44:04,  5.27s/it]
 71%|███████   | 159631/225000 [310:13:50<95:55:01,  5.28s/it]
 71%|███████   | 159632/225000 [310:13:55<92:22:04,  5.09s/it]
 71%|███████   | 159633/225000 [310:14:00<94:34:17,  5.21s/it]
 71%|███████   | 159634/225000 [310:14:05<92:27:25,  5.09s/it]
 71%|███████   | 159635/225000 [310:14:11<93:30:12,  5.15s/it]
 71%|███████   | 159636/225000 [310:14:16<93:43:22,  5.16s/it]
 71%|███████   | 159637/225000 [310:14:21<94:08:07,  5.18s/it]
 71%|███████   | 159638/225000 [310:14:27<95:59:32,  5.29s/it]
 71%|███████   | 159639/225000 [310:14:32<95:02:14,  5.23s/it]
 71%|███████   | 159640/225000 [310:14:37<94:15:12,  5.19s/it]
 71%|███████   | 159641/225000 [310:14:44<106:05:52,  5.84s/it]
 71%|███████   | 159642/225000 [310:14:54<126:45:06,  6.98s/it]
 71%|███████   | 159643/225000 [310:14:58<112:13:56,  6.18s/it]
 71%|███████   | 159644/225000 [310:15:02<101:34:33,  5.60s/it]
 71%|███████   | 159645/225000 [310:15:06<92:51:36,  5.12s/it] 
 71%|███████   | 159646/225000 [310:15:11<89:40:05,  4.94s/it]
 71%|███████   | 159647/225000 [310:15:17<96:55:17,  5.34s/it]
 71%|███████   | 159648/225000 [310:15:28<127:50:33,  7.04s/it]
 71%|███████   | 159649/225000 [310:15:32<108:38:14,  5.98s/it]
 71%|███████   | 159650/225000 [310:15:36<97:23:58,  5.37s/it] 
                                                              
{'loss': '0.7749', 'grad_norm': '1.213', 'learning_rate': '1.058e-05', 'epoch': '0.7096', 'num_input_tokens_seen': 513598944, 'train_runtime': '1.117e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159650/225000 [310:15:36<97:23:58,  5.37s/it]
 71%|███████   | 159651/225000 [310:15:44<112:22:44,  6.19s/it]
 71%|███████   | 159652/225000 [310:15:47<96:57:19,  5.34s/it] 
 71%|███████   | 159653/225000 [310:15:52<93:59:21,  5.18s/it]
 71%|███████   | 159654/225000 [310:15:57<93:40:39,  5.16s/it]
 71%|███████   | 159655/225000 [310:16:01<88:32:03,  4.88s/it]
 71%|███████   | 159656/225000 [310:16:06<86:10:57,  4.75s/it]
 71%|███████   | 159657/225000 [310:16:14<104:37:07,  5.76s/it]
 71%|███████   | 159658/225000 [310:16:24<129:30:40,  7.14s/it]
 71%|███████   | 159659/225000 [310:16:28<113:43:43,  6.27s/it]
 71%|███████   | 159660/225000 [310:16:32<101:10:50,  5.57s/it]
 71%|███████   | 159661/225000 [310:16:36<92:31:35,  5.10s/it] 
 71%|███████   | 159662/225000 [310:16:40<87:42:07,  4.83s/it]
 71%|███████   | 159663/225000 [310:16:46<89:52:49,  4.95s/it]
 71%|███████   | 159664/225000 [310:16:54<106:30:13,  5.87s/it]
 71%|███████   | 159665/225000 [310:17:02<121:14:15,  6.68s/it]
 71%|███████   | 159666/225000 [310:17:09<121:31:04,  6.70s/it]
 71%|███████   | 159667/225000 [310:17:15<115:43:35,  6.38s/it]
 71%|███████   | 159668/225000 [310:17:20<109:21:53,  6.03s/it]
 71%|███████   | 159669/225000 [310:17:24<101:17:38,  5.58s/it]
 71%|███████   | 159670/225000 [310:17:29<94:43:31,  5.22s/it] 
 71%|███████   | 159671/225000 [310:17:36<104:16:54,  5.75s/it]
 71%|███████   | 159672/225000 [310:17:45<120:56:37,  6.66s/it]
 71%|███████   | 159673/225000 [310:17:49<109:26:19,  6.03s/it]
 71%|███████   | 159674/225000 [310:17:54<101:14:18,  5.58s/it]
 71%|███████   | 159675/225000 [310:17:58<94:33:23,  5.21s/it] 
 71%|███████   | 159676/225000 [310:18:02<89:48:20,  4.95s/it]
 71%|███████   | 159677/225000 [310:18:07<88:56:54,  4.90s/it]
 71%|███████   | 159678/225000 [310:18:17<116:58:22,  6.45s/it]
 71%|███████   | 159679/225000 [310:18:22<110:30:38,  6.09s/it]
 71%|███████   | 159680/225000 [310:18:31<121:37:13,  6.70s/it]
 71%|███████   | 159681/225000 [310:18:36<112:26:00,  6.20s/it]
 71%|███████   | 159682/225000 [310:18:40<104:12:54,  5.74s/it]
 71%|███████   | 159683/225000 [310:18:45<99:11:29,  5.47s/it] 
 71%|███████   | 159684/225000 [310:18:50<94:32:17,  5.21s/it]
 71%|███████   | 159685/225000 [310:18:55<93:11:24,  5.14s/it]
 71%|███████   | 159686/225000 [310:19:00<92:04:29,  5.08s/it]
 71%|███████   | 159687/225000 [310:19:04<88:50:13,  4.90s/it]
 71%|███████   | 159688/225000 [310:19:09<90:31:27,  4.99s/it]
 71%|███████   | 159689/225000 [310:19:14<90:34:08,  4.99s/it]
 71%|███████   | 159690/225000 [310:19:19<90:41:21,  5.00s/it]
 71%|███████   | 159691/225000 [310:19:28<110:18:07,  6.08s/it]
 71%|███████   | 159692/225000 [310:19:39<136:22:02,  7.52s/it]
 71%|███████   | 159693/225000 [310:19:43<119:10:30,  6.57s/it]
 71%|███████   | 159694/225000 [310:19:47<106:53:30,  5.89s/it]
 71%|███████   | 159695/225000 [310:19:52<99:35:16,  5.49s/it] 
 71%|███████   | 159696/225000 [310:19:56<93:01:32,  5.13s/it]
 71%|███████   | 159697/225000 [310:20:05<114:36:34,  6.32s/it]
 71%|███████   | 159698/225000 [310:20:12<113:53:22,  6.28s/it]
 71%|███████   | 159699/225000 [310:20:15<96:27:13,  5.32s/it] 
 71%|███████   | 159700/225000 [310:20:23<112:18:11,  6.19s/it]
                                                               
{'loss': '0.7746', 'grad_norm': '1.179', 'learning_rate': '1.056e-05', 'epoch': '0.7098', 'num_input_tokens_seen': 513730024, 'train_runtime': '1.117e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159700/225000 [310:20:23<112:18:11,  6.19s/it]
 71%|███████   | 159701/225000 [310:20:26<95:37:01,  5.27s/it] 
 71%|███████   | 159702/225000 [310:20:35<118:31:07,  6.53s/it]
 71%|███████   | 159703/225000 [310:20:39<101:06:48,  5.57s/it]
 71%|███████   | 159704/225000 [310:20:45<104:21:07,  5.75s/it]
 71%|███████   | 159705/225000 [310:20:49<95:20:56,  5.26s/it] 
 71%|███████   | 159706/225000 [310:20:56<102:47:11,  5.67s/it]
 71%|███████   | 159707/225000 [310:21:01<98:57:00,  5.46s/it] 
 71%|███████   | 159708/225000 [310:21:05<90:39:45,  5.00s/it]
 71%|███████   | 159709/225000 [310:21:09<86:21:06,  4.76s/it]
 71%|███████   | 159710/225000 [310:21:14<86:27:42,  4.77s/it]
 71%|███████   | 159711/225000 [310:21:19<87:43:12,  4.84s/it]
 71%|███████   | 159712/225000 [310:21:29<117:57:24,  6.50s/it]
 71%|███████   | 159713/225000 [310:21:33<106:05:37,  5.85s/it]
 71%|███████   | 159714/225000 [310:21:39<107:50:14,  5.95s/it]
 71%|███████   | 159715/225000 [310:21:44<99:22:11,  5.48s/it] 
 71%|███████   | 159716/225000 [310:21:48<93:32:59,  5.16s/it]
 71%|███████   | 159717/225000 [310:21:59<123:45:05,  6.82s/it]
 71%|███████   | 159718/225000 [310:22:03<109:26:48,  6.04s/it]
 71%|███████   | 159719/225000 [310:22:07<98:01:29,  5.41s/it] 
 71%|███████   | 159720/225000 [310:22:15<111:54:56,  6.17s/it]
 71%|███████   | 159721/225000 [310:22:19<97:14:34,  5.36s/it] 
 71%|███████   | 159722/225000 [310:22:27<111:34:46,  6.15s/it]
 71%|███████   | 159723/225000 [310:22:32<107:22:31,  5.92s/it]
 71%|███████   | 159724/225000 [310:22:39<115:16:55,  6.36s/it]
 71%|███████   | 159725/225000 [310:22:44<108:17:31,  5.97s/it]
 71%|███████   | 159726/225000 [310:22:49<100:32:14,  5.54s/it]
 71%|███████   | 159727/225000 [310:22:53<94:30:42,  5.21s/it] 
 71%|███████   | 159728/225000 [310:22:59<94:23:19,  5.21s/it]
 71%|███████   | 159729/225000 [310:23:03<92:55:54,  5.13s/it]
 71%|███████   | 159730/225000 [310:23:09<92:46:45,  5.12s/it]
 71%|███████   | 159731/225000 [310:23:14<93:27:01,  5.15s/it]
 71%|███████   | 159732/225000 [310:23:19<92:54:06,  5.12s/it]
 71%|███████   | 159733/225000 [310:23:24<92:58:13,  5.13s/it]
 71%|███████   | 159734/225000 [310:23:29<93:09:30,  5.14s/it]
 71%|███████   | 159735/225000 [310:23:34<93:11:31,  5.14s/it]
 71%|███████   | 159736/225000 [310:23:40<93:51:18,  5.18s/it]
 71%|███████   | 159737/225000 [310:23:45<92:37:09,  5.11s/it]
 71%|███████   | 159738/225000 [310:23:56<125:43:44,  6.94s/it]
 71%|███████   | 159739/225000 [310:24:02<123:32:50,  6.82s/it]
 71%|███████   | 159740/225000 [310:24:06<107:45:44,  5.94s/it]
 71%|███████   | 159741/225000 [310:24:15<121:01:48,  6.68s/it]
 71%|███████   | 159742/225000 [310:24:18<104:30:25,  5.77s/it]
 71%|███████   | 159743/225000 [310:24:22<93:44:48,  5.17s/it] 
 71%|███████   | 159744/225000 [310:24:32<120:51:26,  6.67s/it]
 71%|███████   | 159745/225000 [310:24:36<104:12:24,  5.75s/it]
 71%|███████   | 159746/225000 [310:24:39<91:31:02,  5.05s/it] 
 71%|███████   | 159747/225000 [310:24:49<115:03:34,  6.35s/it]
 71%|███████   | 159748/225000 [310:24:53<102:49:44,  5.67s/it]
 71%|███████   | 159749/225000 [310:24:57<94:39:38,  5.22s/it] 
 71%|███████   | 159750/225000 [310:25:01<90:09:00,  4.97s/it]
                                                              
{'loss': '0.7843', 'grad_norm': '1.086', 'learning_rate': '1.055e-05', 'epoch': '0.71', 'num_input_tokens_seen': 513859120, 'train_runtime': '1.118e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159750/225000 [310:25:01<90:09:00,  4.97s/it]
 71%|███████   | 159751/225000 [310:25:05<84:51:01,  4.68s/it]
 71%|███████   | 159752/225000 [310:25:13<101:43:57,  5.61s/it]
 71%|███████   | 159753/225000 [310:25:22<118:04:15,  6.51s/it]
 71%|███████   | 159754/225000 [310:25:25<100:37:20,  5.55s/it]
 71%|███████   | 159755/225000 [310:25:35<123:22:35,  6.81s/it]
 71%|███████   | 159756/225000 [310:25:38<106:54:52,  5.90s/it]
 71%|███████   | 159757/225000 [310:25:42<95:15:41,  5.26s/it] 
 71%|███████   | 159758/225000 [310:25:47<93:27:49,  5.16s/it]
 71%|███████   | 159759/225000 [310:25:52<91:49:51,  5.07s/it]
 71%|███████   | 159760/225000 [310:25:57<89:42:47,  4.95s/it]
 71%|███████   | 159761/225000 [310:26:03<95:14:28,  5.26s/it]
 71%|███████   | 159762/225000 [310:26:15<134:17:16,  7.41s/it]
 71%|███████   | 159763/225000 [310:26:18<112:39:33,  6.22s/it]
 71%|███████   | 159764/225000 [310:26:22<97:35:51,  5.39s/it] 
 71%|███████   | 159765/225000 [310:26:26<90:39:43,  5.00s/it]
 71%|███████   | 159766/225000 [310:26:37<121:52:43,  6.73s/it]
 71%|███████   | 159767/225000 [310:26:40<104:14:24,  5.75s/it]
 71%|███████   | 159768/225000 [310:26:47<109:38:23,  6.05s/it]
 71%|███████   | 159769/225000 [310:26:54<113:48:17,  6.28s/it]
 71%|███████   | 159770/225000 [310:26:59<105:14:43,  5.81s/it]
 71%|███████   | 159771/225000 [310:27:04<103:55:44,  5.74s/it]
 71%|███████   | 159772/225000 [310:27:10<103:00:48,  5.69s/it]
 71%|███████   | 159773/225000 [310:27:15<100:49:48,  5.57s/it]
 71%|███████   | 159774/225000 [310:27:21<101:35:18,  5.61s/it]
 71%|███████   | 159775/225000 [310:27:26<98:21:17,  5.43s/it] 
 71%|███████   | 159776/225000 [310:27:31<97:40:45,  5.39s/it]
 71%|███████   | 159777/225000 [310:27:39<110:30:00,  6.10s/it]
 71%|███████   | 159778/225000 [310:27:44<106:58:13,  5.90s/it]
 71%|███████   | 159779/225000 [310:27:49<103:19:30,  5.70s/it]
 71%|███████   | 159780/225000 [310:27:54<99:50:02,  5.51s/it] 
 71%|███████   | 159781/225000 [310:28:00<97:24:05,  5.38s/it]
 71%|███████   | 159782/225000 [310:28:05<96:25:10,  5.32s/it]
 71%|███████   | 159783/225000 [310:28:10<98:44:15,  5.45s/it]
 71%|███████   | 159784/225000 [310:28:16<98:09:38,  5.42s/it]
 71%|███████   | 159785/225000 [310:28:24<113:32:31,  6.27s/it]
 71%|███████   | 159786/225000 [310:28:30<113:35:52,  6.27s/it]
 71%|███████   | 159787/225000 [310:28:35<103:53:15,  5.73s/it]
 71%|███████   | 159788/225000 [310:28:44<120:28:03,  6.65s/it]
 71%|███████   | 159789/225000 [310:28:48<107:14:25,  5.92s/it]
 71%|███████   | 159790/225000 [310:28:52<96:33:11,  5.33s/it] 
 71%|███████   | 159791/225000 [310:28:56<90:28:22,  4.99s/it]
 71%|███████   | 159792/225000 [310:29:04<107:17:45,  5.92s/it]
 71%|███████   | 159793/225000 [310:29:12<118:08:58,  6.52s/it]
 71%|███████   | 159794/225000 [310:29:16<103:15:17,  5.70s/it]
 71%|███████   | 159795/225000 [310:29:20<95:18:38,  5.26s/it] 
 71%|███████   | 159796/225000 [310:29:27<106:41:01,  5.89s/it]
 71%|███████   | 159797/225000 [310:29:36<121:47:13,  6.72s/it]
 71%|███████   | 159798/225000 [310:29:40<104:23:40,  5.76s/it]
 71%|███████   | 159799/225000 [310:29:43<93:51:12,  5.18s/it] 
 71%|███████   | 159800/225000 [310:29:52<110:23:39,  6.10s/it]
                                                               
{'loss': '0.7757', 'grad_norm': '1.113', 'learning_rate': '1.054e-05', 'epoch': '0.7102', 'num_input_tokens_seen': 513990296, 'train_runtime': '1.118e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159800/225000 [310:29:52<110:23:39,  6.10s/it]
 71%|███████   | 159801/225000 [310:29:56<99:25:41,  5.49s/it] 
 71%|███████   | 159802/225000 [310:30:00<90:16:05,  4.98s/it]
 71%|███████   | 159803/225000 [310:30:08<111:26:47,  6.15s/it]
 71%|███████   | 159804/225000 [310:30:15<111:43:48,  6.17s/it]
 71%|███████   | 159805/225000 [310:30:19<99:48:05,  5.51s/it] 
 71%|███████   | 159806/225000 [310:30:23<96:13:26,  5.31s/it]
 71%|███████   | 159807/225000 [310:30:28<89:30:43,  4.94s/it]
 71%|███████   | 159808/225000 [310:30:32<86:52:13,  4.80s/it]
 71%|███████   | 159809/225000 [310:30:37<86:19:05,  4.77s/it]
 71%|███████   | 159810/225000 [310:30:42<86:56:34,  4.80s/it]
 71%|███████   | 159811/225000 [310:30:47<88:53:49,  4.91s/it]
 71%|███████   | 159812/225000 [310:30:58<124:38:28,  6.88s/it]
 71%|███████   | 159813/225000 [310:31:03<114:42:35,  6.33s/it]
 71%|███████   | 159814/225000 [310:31:08<105:53:01,  5.85s/it]
 71%|███████   | 159815/225000 [310:31:13<99:16:02,  5.48s/it] 
 71%|███████   | 159816/225000 [310:31:19<104:17:47,  5.76s/it]
 71%|███████   | 159817/225000 [310:31:28<123:15:05,  6.81s/it]
 71%|███████   | 159818/225000 [310:31:32<108:10:12,  5.97s/it]
 71%|███████   | 159819/225000 [310:31:40<119:33:44,  6.60s/it]
 71%|███████   | 159820/225000 [310:31:44<102:14:55,  5.65s/it]
 71%|███████   | 159821/225000 [310:31:52<116:25:02,  6.43s/it]
 71%|███████   | 159822/225000 [310:31:56<105:39:12,  5.84s/it]
 71%|███████   | 159823/225000 [310:32:00<92:37:28,  5.12s/it] 
 71%|███████   | 159824/225000 [310:32:09<112:01:04,  6.19s/it]
 71%|███████   | 159825/225000 [310:32:13<99:54:30,  5.52s/it] 
 71%|███████   | 159826/225000 [310:32:17<94:57:36,  5.25s/it]
 71%|███████   | 159827/225000 [310:32:22<92:34:58,  5.11s/it]
 71%|███████   | 159828/225000 [310:32:27<91:20:22,  5.05s/it]
 71%|███████   | 159829/225000 [310:32:32<91:45:18,  5.07s/it]
 71%|███████   | 159830/225000 [310:32:37<92:27:07,  5.11s/it]
 71%|███████   | 159831/225000 [310:32:42<91:15:20,  5.04s/it]
 71%|███████   | 159832/225000 [310:32:53<120:58:51,  6.68s/it]
 71%|███████   | 159833/225000 [310:32:59<120:39:30,  6.67s/it]
 71%|███████   | 159834/225000 [310:33:03<103:42:37,  5.73s/it]
 71%|███████   | 159835/225000 [310:33:12<123:12:01,  6.81s/it]
 71%|███████   | 159836/225000 [310:33:16<105:06:05,  5.81s/it]
 71%|███████   | 159837/225000 [310:33:19<92:41:07,  5.12s/it] 
 71%|███████   | 159838/225000 [310:33:27<105:59:44,  5.86s/it]
 71%|███████   | 159839/225000 [310:33:32<105:27:15,  5.83s/it]
 71%|███████   | 159840/225000 [310:33:37<97:18:14,  5.38s/it] 
 71%|███████   | 159841/225000 [310:33:41<93:35:31,  5.17s/it]
 71%|███████   | 159842/225000 [310:33:46<89:20:05,  4.94s/it]
 71%|███████   | 159843/225000 [310:33:50<86:48:47,  4.80s/it]
 71%|███████   | 159844/225000 [310:33:55<87:02:09,  4.81s/it]
 71%|███████   | 159845/225000 [310:34:06<120:21:59,  6.65s/it]
 71%|███████   | 159846/225000 [310:34:11<110:04:58,  6.08s/it]
 71%|███████   | 159847/225000 [310:34:18<114:43:55,  6.34s/it]
 71%|███████   | 159848/225000 [310:34:24<112:31:50,  6.22s/it]
 71%|███████   | 159849/225000 [310:34:27<98:19:21,  5.43s/it] 
 71%|███████   | 159850/225000 [310:34:36<116:42:55,  6.45s/it]
                                                               
{'loss': '0.7676', 'grad_norm': '1.245', 'learning_rate': '1.052e-05', 'epoch': '0.7104', 'num_input_tokens_seen': 514120072, 'train_runtime': '1.118e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159850/225000 [310:34:36<116:42:55,  6.45s/it]
 71%|███████   | 159851/225000 [310:34:43<120:30:04,  6.66s/it]
 71%|███████   | 159852/225000 [310:34:47<107:17:50,  5.93s/it]
 71%|███████   | 159853/225000 [310:34:52<98:17:20,  5.43s/it] 
 71%|███████   | 159854/225000 [310:34:56<92:46:05,  5.13s/it]
 71%|███████   | 159855/225000 [310:35:01<88:43:40,  4.90s/it]
 71%|███████   | 159856/225000 [310:35:05<83:45:03,  4.63s/it]
 71%|███████   | 159857/225000 [310:35:10<85:59:06,  4.75s/it]
 71%|███████   | 159858/225000 [310:35:21<120:39:15,  6.67s/it]
 71%|███████   | 159859/225000 [310:35:25<109:39:17,  6.06s/it]
 71%|███████   | 159860/225000 [310:35:30<104:40:15,  5.78s/it]
 71%|███████   | 159861/225000 [310:35:36<101:24:29,  5.60s/it]
 71%|███████   | 159862/225000 [310:35:41<97:12:23,  5.37s/it] 
 71%|███████   | 159863/225000 [310:35:46<96:25:51,  5.33s/it]
 71%|███████   | 159864/225000 [310:35:51<95:31:07,  5.28s/it]
 71%|███████   | 159865/225000 [310:35:56<94:17:19,  5.21s/it]
 71%|███████   | 159866/225000 [310:36:01<94:22:53,  5.22s/it]
 71%|███████   | 159867/225000 [310:36:06<93:16:17,  5.16s/it]
 71%|███████   | 159868/225000 [310:36:11<93:29:32,  5.17s/it]
 71%|███████   | 159869/225000 [310:36:17<93:11:15,  5.15s/it]
 71%|███████   | 159870/225000 [310:36:22<94:49:21,  5.24s/it]
 71%|███████   | 159871/225000 [310:36:27<92:19:54,  5.10s/it]
 71%|███████   | 159872/225000 [310:36:31<90:28:14,  5.00s/it]
 71%|███████   | 159873/225000 [310:36:38<98:12:59,  5.43s/it]
 71%|███████   | 159874/225000 [310:36:50<133:33:11,  7.38s/it]
 71%|███████   | 159875/225000 [310:36:55<121:12:02,  6.70s/it]
 71%|███████   | 159876/225000 [310:37:00<111:18:05,  6.15s/it]
 71%|███████   | 159877/225000 [310:37:05<104:29:27,  5.78s/it]
 71%|███████   | 159878/225000 [310:37:10<99:59:54,  5.53s/it] 
 71%|███████   | 159879/225000 [310:37:15<96:27:49,  5.33s/it]
 71%|███████   | 159880/225000 [310:37:20<94:23:44,  5.22s/it]
 71%|███████   | 159881/225000 [310:37:25<93:18:15,  5.16s/it]
 71%|███████   | 159882/225000 [310:37:30<94:25:35,  5.22s/it]
 71%|███████   | 159883/225000 [310:37:35<94:57:29,  5.25s/it]
 71%|███████   | 159884/225000 [310:37:40<93:15:39,  5.16s/it]
 71%|███████   | 159885/225000 [310:37:52<127:57:16,  7.07s/it]
 71%|███████   | 159886/225000 [310:37:56<113:58:08,  6.30s/it]
 71%|███████   | 159887/225000 [310:38:04<120:33:55,  6.67s/it]
 71%|███████   | 159888/225000 [310:38:07<103:29:20,  5.72s/it]
 71%|███████   | 159889/225000 [310:38:15<111:48:32,  6.18s/it]
 71%|███████   | 159890/225000 [310:38:18<96:23:32,  5.33s/it] 
 71%|███████   | 159891/225000 [310:38:23<94:22:30,  5.22s/it]
 71%|███████   | 159892/225000 [310:38:32<115:23:06,  6.38s/it]
 71%|███████   | 159893/225000 [310:38:35<96:49:59,  5.35s/it] 
 71%|███████   | 159894/225000 [310:38:39<87:59:50,  4.87s/it]
 71%|███████   | 159895/225000 [310:38:43<83:23:44,  4.61s/it]
 71%|███████   | 159896/225000 [310:38:54<118:53:25,  6.57s/it]
 71%|███████   | 159897/225000 [310:38:58<104:10:29,  5.76s/it]
 71%|███████   | 159898/225000 [310:39:05<113:13:58,  6.26s/it]
 71%|███████   | 159899/225000 [310:39:09<99:39:52,  5.51s/it] 
 71%|███████   | 159900/225000 [310:39:14<97:26:46,  5.39s/it]
                                                              
{'loss': '0.7753', 'grad_norm': '1.048', 'learning_rate': '1.051e-05', 'epoch': '0.7107', 'num_input_tokens_seen': 514248272, 'train_runtime': '1.118e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159900/225000 [310:39:14<97:26:46,  5.39s/it]
 71%|███████   | 159901/225000 [310:39:20<103:21:11,  5.72s/it]
 71%|███████   | 159902/225000 [310:39:28<115:48:10,  6.40s/it]
 71%|███████   | 159903/225000 [310:39:33<104:38:28,  5.79s/it]
 71%|███████   | 159904/225000 [310:39:36<90:58:28,  5.03s/it] 
 71%|███████   | 159905/225000 [310:39:44<108:18:41,  5.99s/it]
 71%|███████   | 159906/225000 [310:39:51<113:37:56,  6.28s/it]
 71%|███████   | 159907/225000 [310:39:56<103:57:35,  5.75s/it]
 71%|███████   | 159908/225000 [310:40:00<95:33:46,  5.29s/it] 
 71%|███████   | 159909/225000 [310:40:04<90:24:34,  5.00s/it]
 71%|███████   | 159910/225000 [310:40:09<86:52:30,  4.80s/it]
 71%|███████   | 159911/225000 [310:40:18<113:40:25,  6.29s/it]
 71%|███████   | 159912/225000 [310:40:23<106:58:45,  5.92s/it]
 71%|███████   | 159913/225000 [310:40:28<101:42:56,  5.63s/it]
 71%|███████   | 159914/225000 [310:40:33<98:29:57,  5.45s/it] 
 71%|███████   | 159915/225000 [310:40:38<95:56:17,  5.31s/it]
 71%|███████   | 159916/225000 [310:40:43<92:56:06,  5.14s/it]
 71%|███████   | 159917/225000 [310:40:48<91:34:25,  5.07s/it]
 71%|███████   | 159918/225000 [310:40:53<91:07:34,  5.04s/it]
 71%|███████   | 159919/225000 [310:40:58<91:52:26,  5.08s/it]
 71%|███████   | 159920/225000 [310:41:03<92:00:14,  5.09s/it]
 71%|███████   | 159921/225000 [310:41:08<88:27:40,  4.89s/it]
 71%|███████   | 159922/225000 [310:41:13<88:32:10,  4.90s/it]
 71%|███████   | 159923/225000 [310:41:18<89:24:07,  4.95s/it]
 71%|███████   | 159924/225000 [310:41:25<103:07:33,  5.70s/it]
 71%|███████   | 159925/225000 [310:41:38<141:47:43,  7.84s/it]
 71%|███████   | 159926/225000 [310:41:42<121:09:42,  6.70s/it]
 71%|███████   | 159927/225000 [310:41:48<117:38:17,  6.51s/it]
 71%|███████   | 159928/225000 [310:41:54<116:32:39,  6.45s/it]
 71%|███████   | 159929/225000 [310:42:00<110:38:49,  6.12s/it]
 71%|███████   | 159930/225000 [310:42:03<95:38:25,  5.29s/it] 
 71%|███████   | 159931/225000 [310:42:11<109:37:27,  6.07s/it]
 71%|███████   | 159932/225000 [310:42:15<97:15:39,  5.38s/it] 
 71%|███████   | 159933/225000 [310:42:19<89:08:51,  4.93s/it]
 71%|███████   | 159934/225000 [310:42:27<106:31:56,  5.89s/it]
 71%|███████   | 159935/225000 [310:42:33<110:03:42,  6.09s/it]
 71%|███████   | 159936/225000 [310:42:38<100:23:30,  5.55s/it]
 71%|███████   | 159937/225000 [310:42:42<95:09:55,  5.27s/it] 
 71%|███████   | 159938/225000 [310:42:47<92:03:22,  5.09s/it]
 71%|███████   | 159939/225000 [310:42:51<88:38:32,  4.90s/it]
 71%|███████   | 159940/225000 [310:42:56<86:47:28,  4.80s/it]
 71%|███████   | 159941/225000 [310:43:04<101:52:49,  5.64s/it]
 71%|███████   | 159942/225000 [310:43:13<124:59:41,  6.92s/it]
 71%|███████   | 159943/225000 [310:43:17<107:01:03,  5.92s/it]
 71%|███████   | 159944/225000 [310:43:26<123:12:16,  6.82s/it]
 71%|███████   | 159945/225000 [310:43:29<103:37:00,  5.73s/it]
 71%|███████   | 159946/225000 [310:43:38<122:05:55,  6.76s/it]
 71%|███████   | 159947/225000 [310:43:42<105:48:43,  5.86s/it]
 71%|███████   | 159948/225000 [310:43:45<91:35:30,  5.07s/it] 
 71%|███████   | 159949/225000 [310:43:53<106:46:39,  5.91s/it]
 71%|███████   | 159950/225000 [310:43:57<95:28:26,  5.28s/it] 
                                                              
{'loss': '0.7699', 'grad_norm': '1.1', 'learning_rate': '1.049e-05', 'epoch': '0.7109', 'num_input_tokens_seen': 514378344, 'train_runtime': '1.119e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 159950/225000 [310:43:57<95:28:26,  5.28s/it]
 71%|███████   | 159951/225000 [310:44:06<114:18:03,  6.33s/it]
 71%|███████   | 159952/225000 [310:44:10<103:35:13,  5.73s/it]
 71%|███████   | 159953/225000 [310:44:15<101:05:34,  5.59s/it]
 71%|███████   | 159954/225000 [310:44:21<101:38:51,  5.63s/it]
 71%|███████   | 159955/225000 [310:44:26<100:16:03,  5.55s/it]
 71%|███████   | 159956/225000 [310:44:32<100:30:01,  5.56s/it]
 71%|███████   | 159957/225000 [310:44:37<99:48:23,  5.52s/it] 
 71%|███████   | 159958/225000 [310:44:43<101:23:25,  5.61s/it]
 71%|███████   | 159959/225000 [310:44:49<101:43:52,  5.63s/it]
 71%|███████   | 159960/225000 [310:44:55<102:27:26,  5.67s/it]
 71%|███████   | 159961/225000 [310:45:00<101:17:31,  5.61s/it]
 71%|███████   | 159962/225000 [310:45:09<120:33:38,  6.67s/it]
 71%|███████   | 159963/225000 [310:45:16<123:12:19,  6.82s/it]
 71%|███████   | 159964/225000 [310:45:20<106:36:39,  5.90s/it]
 71%|███████   | 159965/225000 [310:45:24<97:11:29,  5.38s/it] 
 71%|███████   | 159966/225000 [310:45:31<104:48:50,  5.80s/it]
 71%|███████   | 159967/225000 [310:45:40<120:34:38,  6.67s/it]
 71%|███████   | 159968/225000 [310:45:44<106:29:24,  5.90s/it]
 71%|███████   | 159969/225000 [310:45:49<102:23:05,  5.67s/it]
 71%|███████   | 159970/225000 [310:45:54<96:12:59,  5.33s/it] 
 71%|███████   | 159971/225000 [310:45:58<93:39:33,  5.18s/it]
 71%|███████   | 159972/225000 [310:46:03<90:59:54,  5.04s/it]
 71%|███████   | 159973/225000 [310:46:08<91:20:48,  5.06s/it]
 71%|███████   | 159974/225000 [310:46:13<90:26:49,  5.01s/it]
 71%|███████   | 159975/225000 [310:46:18<89:37:42,  4.96s/it]
 71%|███████   | 159976/225000 [310:46:23<90:10:49,  4.99s/it]
 71%|███████   | 159977/225000 [310:46:29<94:39:42,  5.24s/it]
 71%|███████   | 159978/225000 [310:46:41<132:39:18,  7.34s/it]
 71%|███████   | 159979/225000 [310:46:46<119:53:27,  6.64s/it]
 71%|███████   | 159980/225000 [310:46:55<129:28:48,  7.17s/it]
 71%|███████   | 159981/225000 [310:46:58<109:07:51,  6.04s/it]
 71%|███████   | 159982/225000 [310:47:05<113:17:01,  6.27s/it]
 71%|███████   | 159983/225000 [310:47:09<99:23:21,  5.50s/it] 
 71%|███████   | 159984/225000 [310:47:19<123:54:27,  6.86s/it]
 71%|███████   | 159985/225000 [310:47:23<109:49:47,  6.08s/it]
 71%|███████   | 159986/225000 [310:47:28<103:42:24,  5.74s/it]
 71%|███████   | 159987/225000 [310:47:33<98:27:12,  5.45s/it] 
 71%|███████   | 159988/225000 [310:47:38<96:04:14,  5.32s/it]
 71%|███████   | 159989/225000 [310:47:43<94:40:11,  5.24s/it]
 71%|███████   | 159990/225000 [310:47:47<92:39:32,  5.13s/it]
 71%|███████   | 159991/225000 [310:47:53<93:14:26,  5.16s/it]
 71%|███████   | 159992/225000 [310:47:57<90:52:36,  5.03s/it]
 71%|███████   | 159993/225000 [310:48:02<89:27:19,  4.95s/it]
 71%|███████   | 159994/225000 [310:48:11<112:26:18,  6.23s/it]
 71%|███████   | 159995/225000 [310:48:19<121:28:48,  6.73s/it]
 71%|███████   | 159996/225000 [310:48:23<104:45:54,  5.80s/it]
 71%|███████   | 159997/225000 [310:48:27<92:51:47,  5.14s/it] 
 71%|███████   | 159998/225000 [310:48:38<125:31:53,  6.95s/it]
 71%|███████   | 159999/225000 [310:48:41<106:54:40,  5.92s/it]
 71%|███████   | 160000/225000 [310:48:48<113:57:19,  6.31s/it]
                                                               
{'loss': '0.7831', 'grad_norm': '1.039', 'learning_rate': '1.048e-05', 'epoch': '0.7111', 'num_input_tokens_seen': 514509840, 'train_runtime': '1.119e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160000/225000 [310:48:48<113:57:19,  6.31s/it]
 71%|███████   | 160001/225000 [310:48:52<98:52:40,  5.48s/it] 
 71%|███████   | 160002/225000 [310:48:56<89:30:28,  4.96s/it]
 71%|███████   | 160003/225000 [310:49:05<111:38:25,  6.18s/it]
 71%|███████   | 160004/225000 [310:49:08<96:32:04,  5.35s/it] 
 71%|███████   | 160005/225000 [310:49:16<108:02:14,  5.98s/it]
 71%|███████   | 160006/225000 [310:49:21<102:51:42,  5.70s/it]
 71%|███████   | 160007/225000 [310:49:26<99:03:53,  5.49s/it] 
 71%|███████   | 160008/225000 [310:49:31<98:04:42,  5.43s/it]
 71%|███████   | 160009/225000 [310:49:36<95:02:38,  5.26s/it]
 71%|███████   | 160010/225000 [310:49:41<94:15:50,  5.22s/it]
 71%|███████   | 160011/225000 [310:49:46<94:10:31,  5.22s/it]
 71%|███████   | 160012/225000 [310:49:51<92:00:05,  5.10s/it]
 71%|███████   | 160013/225000 [310:49:56<92:41:49,  5.14s/it]
 71%|███████   | 160014/225000 [310:50:01<90:37:30,  5.02s/it]
 71%|███████   | 160015/225000 [310:50:09<105:55:40,  5.87s/it]
 71%|███████   | 160016/225000 [310:50:19<129:55:42,  7.20s/it]
 71%|███████   | 160017/225000 [310:50:23<110:47:41,  6.14s/it]
 71%|███████   | 160018/225000 [310:50:27<101:48:30,  5.64s/it]
 71%|███████   | 160019/225000 [310:50:35<112:46:22,  6.25s/it]
 71%|███████   | 160020/225000 [310:50:39<98:45:17,  5.47s/it] 
 71%|███████   | 160021/225000 [310:50:49<124:35:27,  6.90s/it]
 71%|███████   | 160022/225000 [310:50:53<109:02:53,  6.04s/it]
 71%|███████   | 160023/225000 [310:50:57<100:41:21,  5.58s/it]
 71%|███████   | 160024/225000 [310:51:01<92:07:52,  5.10s/it] 
 71%|███████   | 160025/225000 [310:51:08<98:19:15,  5.45s/it]
 71%|███████   | 160026/225000 [310:51:14<102:20:17,  5.67s/it]
 71%|███████   | 160027/225000 [310:51:21<110:03:36,  6.10s/it]
 71%|███████   | 160028/225000 [310:51:28<117:26:53,  6.51s/it]
 71%|███████   | 160029/225000 [310:51:33<105:27:15,  5.84s/it]
 71%|███████   | 160030/225000 [310:51:37<97:25:14,  5.40s/it] 
 71%|███████   | 160031/225000 [310:51:41<91:34:26,  5.07s/it]
 71%|███████   | 160032/225000 [310:51:45<86:10:26,  4.78s/it]
 71%|███████   | 160033/225000 [310:51:51<91:11:52,  5.05s/it]
 71%|███████   | 160034/225000 [310:52:02<125:21:28,  6.95s/it]
 71%|███████   | 160035/225000 [310:52:06<106:53:31,  5.92s/it]
 71%|███████   | 160036/225000 [310:52:10<96:42:37,  5.36s/it] 
 71%|███████   | 160037/225000 [310:52:14<90:21:24,  5.01s/it]
 71%|███████   | 160038/225000 [310:52:18<85:11:06,  4.72s/it]
 71%|███████   | 160039/225000 [310:52:24<90:16:50,  5.00s/it]
 71%|███████   | 160040/225000 [310:52:35<121:13:17,  6.72s/it]
 71%|███████   | 160041/225000 [310:52:39<105:52:26,  5.87s/it]
 71%|███████   | 160042/225000 [310:52:42<93:35:29,  5.19s/it] 
 71%|███████   | 160043/225000 [310:52:53<126:00:51,  6.98s/it]
 71%|███████   | 160044/225000 [310:52:58<111:33:04,  6.18s/it]
 71%|███████   | 160045/225000 [310:53:02<102:10:33,  5.66s/it]
 71%|███████   | 160046/225000 [310:53:06<95:14:05,  5.28s/it] 
 71%|███████   | 160047/225000 [310:53:11<89:30:35,  4.96s/it]
 71%|███████   | 160048/225000 [310:53:15<85:09:55,  4.72s/it]
 71%|███████   | 160049/225000 [310:53:27<126:17:56,  7.00s/it]
 71%|███████   | 160050/225000 [310:53:31<111:03:31,  6.16s/it]
                                                               
{'loss': '0.7763', 'grad_norm': '1.088', 'learning_rate': '1.046e-05', 'epoch': '0.7113', 'num_input_tokens_seen': 514640520, 'train_runtime': '1.119e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160050/225000 [310:53:31<111:03:31,  6.16s/it]
 71%|███████   | 160051/225000 [310:53:35<100:12:38,  5.55s/it]
 71%|███████   | 160052/225000 [310:53:39<91:21:31,  5.06s/it] 
 71%|███████   | 160053/225000 [310:53:50<119:48:55,  6.64s/it]
 71%|███████   | 160054/225000 [310:53:55<110:27:32,  6.12s/it]
 71%|███████   | 160055/225000 [310:54:02<119:25:39,  6.62s/it]
 71%|███████   | 160056/225000 [310:54:07<106:53:41,  5.93s/it]
 71%|███████   | 160057/225000 [310:54:11<97:38:54,  5.41s/it] 
 71%|███████   | 160058/225000 [310:54:15<91:33:51,  5.08s/it]
 71%|███████   | 160059/225000 [310:54:25<118:09:59,  6.55s/it]
 71%|███████   | 160060/225000 [310:54:28<100:15:50,  5.56s/it]
 71%|███████   | 160061/225000 [310:54:35<104:59:46,  5.82s/it]
 71%|███████   | 160062/225000 [310:54:42<111:59:27,  6.21s/it]
 71%|███████   | 160063/225000 [310:54:46<101:09:30,  5.61s/it]
 71%|███████   | 160064/225000 [310:54:50<91:43:18,  5.08s/it] 
 71%|███████   | 160065/225000 [310:54:54<84:42:02,  4.70s/it]
 71%|███████   | 160066/225000 [310:55:05<118:25:10,  6.57s/it]
 71%|███████   | 160067/225000 [310:55:09<106:28:23,  5.90s/it]
 71%|███████   | 160068/225000 [310:55:14<98:16:35,  5.45s/it] 
 71%|███████   | 160069/225000 [310:55:18<91:52:18,  5.09s/it]
 71%|███████   | 160070/225000 [310:55:29<124:21:08,  6.89s/it]
 71%|███████   | 160071/225000 [310:55:33<108:16:12,  6.00s/it]
 71%|███████   | 160072/225000 [310:55:38<101:01:47,  5.60s/it]
 71%|███████   | 160073/225000 [310:55:43<99:42:36,  5.53s/it] 
 71%|███████   | 160074/225000 [310:55:48<97:59:39,  5.43s/it]
 71%|███████   | 160075/225000 [310:55:53<95:32:39,  5.30s/it]
 71%|███████   | 160076/225000 [310:55:59<96:52:19,  5.37s/it]
 71%|███████   | 160077/225000 [310:56:04<98:06:49,  5.44s/it]
 71%|███████   | 160078/225000 [310:56:09<96:47:31,  5.37s/it]
 71%|███████   | 160079/225000 [310:56:14<95:04:41,  5.27s/it]
 71%|███████   | 160080/225000 [310:56:24<117:39:44,  6.52s/it]
 71%|███████   | 160081/225000 [310:56:30<115:08:51,  6.39s/it]
 71%|███████   | 160082/225000 [310:56:36<110:57:46,  6.15s/it]
 71%|███████   | 160083/225000 [310:56:41<107:43:06,  5.97s/it]
 71%|███████   | 160084/225000 [310:56:47<105:23:44,  5.84s/it]
 71%|███████   | 160085/225000 [310:56:52<102:36:15,  5.69s/it]
 71%|███████   | 160086/225000 [310:57:01<121:50:58,  6.76s/it]
 71%|███████   | 160087/225000 [310:57:07<114:10:13,  6.33s/it]
 71%|███████   | 160088/225000 [310:57:12<110:08:11,  6.11s/it]
 71%|███████   | 160089/225000 [310:57:17<102:45:45,  5.70s/it]
 71%|███████   | 160090/225000 [310:57:22<97:28:28,  5.41s/it] 
 71%|███████   | 160091/225000 [310:57:27<94:34:54,  5.25s/it]
 71%|███████   | 160092/225000 [310:57:32<93:30:10,  5.19s/it]
 71%|███████   | 160093/225000 [310:57:37<92:21:17,  5.12s/it]
 71%|███████   | 160094/225000 [310:57:41<90:11:11,  5.00s/it]
 71%|███████   | 160095/225000 [310:57:46<89:27:19,  4.96s/it]
 71%|███████   | 160096/225000 [310:57:51<87:05:00,  4.83s/it]
 71%|███████   | 160097/225000 [310:57:55<84:47:19,  4.70s/it]
 71%|███████   | 160098/225000 [310:58:00<86:06:23,  4.78s/it]
 71%|███████   | 160099/225000 [310:58:06<90:18:39,  5.01s/it]
 71%|███████   | 160100/225000 [310:58:12<97:38:52,  5.42s/it]
                                                              
{'loss': '0.7738', 'grad_norm': '1.064', 'learning_rate': '1.045e-05', 'epoch': '0.7116', 'num_input_tokens_seen': 514770960, 'train_runtime': '1.119e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160100/225000 [310:58:12<97:38:52,  5.42s/it]
 71%|███████   | 160101/225000 [310:58:17<96:42:13,  5.36s/it]
 71%|███████   | 160102/225000 [310:58:23<97:06:34,  5.39s/it]
 71%|███████   | 160103/225000 [310:58:28<98:00:50,  5.44s/it]
 71%|███████   | 160104/225000 [310:58:40<131:19:48,  7.29s/it]
 71%|███████   | 160105/225000 [310:58:44<113:56:18,  6.32s/it]
 71%|███████   | 160106/225000 [310:58:48<104:47:36,  5.81s/it]
 71%|███████   | 160107/225000 [310:58:53<98:22:20,  5.46s/it] 
 71%|███████   | 160108/225000 [310:59:03<125:06:14,  6.94s/it]
 71%|███████   | 160109/225000 [310:59:09<118:22:17,  6.57s/it]
 71%|███████   | 160110/225000 [310:59:18<129:48:49,  7.20s/it]
 71%|███████   | 160111/225000 [310:59:21<108:31:15,  6.02s/it]
 71%|███████   | 160112/225000 [310:59:28<111:43:10,  6.20s/it]
 71%|███████   | 160113/225000 [310:59:34<109:30:15,  6.08s/it]
 71%|███████   | 160114/225000 [310:59:37<96:44:39,  5.37s/it] 
 71%|███████   | 160115/225000 [310:59:45<112:17:28,  6.23s/it]
 71%|███████   | 160116/225000 [310:59:49<97:36:18,  5.42s/it] 
 71%|███████   | 160117/225000 [310:59:53<91:19:28,  5.07s/it]
 71%|███████   | 160118/225000 [311:00:02<113:04:09,  6.27s/it]
 71%|███████   | 160119/225000 [311:00:07<103:48:38,  5.76s/it]
 71%|███████   | 160120/225000 [311:00:11<97:14:36,  5.40s/it] 
 71%|███████   | 160121/225000 [311:00:16<91:37:34,  5.08s/it]
 71%|███████   | 160122/225000 [311:00:20<86:46:36,  4.82s/it]
 71%|███████   | 160123/225000 [311:00:25<86:07:55,  4.78s/it]
 71%|███████   | 160124/225000 [311:00:37<129:10:18,  7.17s/it]
 71%|███████   | 160125/225000 [311:00:42<117:45:10,  6.53s/it]
 71%|███████   | 160126/225000 [311:00:47<109:26:55,  6.07s/it]
 71%|███████   | 160127/225000 [311:00:53<105:59:00,  5.88s/it]
 71%|███████   | 160128/225000 [311:00:58<100:50:20,  5.60s/it]
 71%|███████   | 160129/225000 [311:01:03<100:26:45,  5.57s/it]
 71%|███████   | 160130/225000 [311:01:08<96:06:55,  5.33s/it] 
 71%|███████   | 160131/225000 [311:01:13<94:59:56,  5.27s/it]
 71%|███████   | 160132/225000 [311:01:19<95:27:36,  5.30s/it]
 71%|███████   | 160133/225000 [311:01:24<96:16:54,  5.34s/it]
 71%|███████   | 160134/225000 [311:01:29<96:32:03,  5.36s/it]
 71%|███████   | 160135/225000 [311:01:35<95:36:40,  5.31s/it]
 71%|███████   | 160136/225000 [311:01:40<93:59:00,  5.22s/it]
 71%|███████   | 160137/225000 [311:01:45<95:09:52,  5.28s/it]
 71%|███████   | 160138/225000 [311:01:51<97:05:51,  5.39s/it]
 71%|███████   | 160139/225000 [311:01:56<96:08:51,  5.34s/it]
 71%|███████   | 160140/225000 [311:02:01<96:33:21,  5.36s/it]
 71%|███████   | 160141/225000 [311:02:07<96:14:40,  5.34s/it]
 71%|███████   | 160142/225000 [311:02:12<95:15:01,  5.29s/it]
 71%|███████   | 160143/225000 [311:02:17<95:28:49,  5.30s/it]
 71%|███████   | 160144/225000 [311:02:30<134:25:22,  7.46s/it]
 71%|███████   | 160145/225000 [311:02:36<130:53:40,  7.27s/it]
 71%|███████   | 160146/225000 [311:02:42<120:26:26,  6.69s/it]
 71%|███████   | 160147/225000 [311:02:48<118:50:59,  6.60s/it]
 71%|███████   | 160148/225000 [311:02:56<123:37:32,  6.86s/it]
 71%|███████   | 160149/225000 [311:03:00<111:34:56,  6.19s/it]
 71%|███████   | 160150/225000 [311:03:05<104:49:12,  5.82s/it]
                                                               
{'loss': '0.7728', 'grad_norm': '1.092', 'learning_rate': '1.043e-05', 'epoch': '0.7118', 'num_input_tokens_seen': 514900968, 'train_runtime': '1.12e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160150/225000 [311:03:05<104:49:12,  5.82s/it]
 71%|███████   | 160151/225000 [311:03:10<100:45:56,  5.59s/it]
 71%|███████   | 160152/225000 [311:03:15<96:36:45,  5.36s/it] 
 71%|███████   | 160153/225000 [311:03:20<95:45:42,  5.32s/it]
 71%|███████   | 160154/225000 [311:03:25<93:27:48,  5.19s/it]
 71%|███████   | 160155/225000 [311:03:30<92:02:40,  5.11s/it]
 71%|███████   | 160156/225000 [311:03:35<91:57:42,  5.11s/it]
 71%|███████   | 160157/225000 [311:03:41<93:28:47,  5.19s/it]
 71%|███████   | 160158/225000 [311:03:46<93:02:27,  5.17s/it]
 71%|███████   | 160159/225000 [311:03:51<92:48:38,  5.15s/it]
 71%|███████   | 160160/225000 [311:03:56<94:02:26,  5.22s/it]
 71%|███████   | 160161/225000 [311:04:01<93:25:33,  5.19s/it]
 71%|███████   | 160162/225000 [311:04:07<93:42:31,  5.20s/it]
 71%|███████   | 160163/225000 [311:04:12<94:14:48,  5.23s/it]
 71%|███████   | 160164/225000 [311:04:17<95:56:00,  5.33s/it]
 71%|███████   | 160165/225000 [311:04:23<95:35:18,  5.31s/it]
 71%|███████   | 160166/225000 [311:04:35<134:58:59,  7.50s/it]
 71%|███████   | 160167/225000 [311:04:40<121:30:54,  6.75s/it]
 71%|███████   | 160168/225000 [311:04:47<121:22:01,  6.74s/it]
 71%|███████   | 160169/225000 [311:04:54<120:55:11,  6.71s/it]
 71%|███████   | 160170/225000 [311:04:58<108:49:35,  6.04s/it]
 71%|███████   | 160171/225000 [311:05:02<97:01:42,  5.39s/it] 
 71%|███████   | 160172/225000 [311:05:06<89:25:20,  4.97s/it]
 71%|███████   | 160173/225000 [311:05:12<96:50:11,  5.38s/it]
 71%|███████   | 160174/225000 [311:05:22<117:18:29,  6.51s/it]
 71%|███████   | 160175/225000 [311:05:25<101:20:05,  5.63s/it]
 71%|███████   | 160176/225000 [311:05:30<95:28:43,  5.30s/it] 
 71%|███████   | 160177/225000 [311:05:38<112:38:13,  6.26s/it]
 71%|███████   | 160178/225000 [311:05:42<100:42:03,  5.59s/it]
 71%|███████   | 160179/225000 [311:05:46<91:32:20,  5.08s/it] 
 71%|███████   | 160180/225000 [311:05:55<109:56:53,  6.11s/it]
 71%|███████   | 160181/225000 [311:06:01<111:20:52,  6.18s/it]
 71%|███████   | 160182/225000 [311:06:04<95:27:07,  5.30s/it] 
 71%|███████   | 160183/225000 [311:06:08<88:49:55,  4.93s/it]
 71%|███████   | 160184/225000 [311:06:16<101:42:54,  5.65s/it]
 71%|███████   | 160185/225000 [311:06:23<113:06:18,  6.28s/it]
 71%|███████   | 160186/225000 [311:06:28<103:41:42,  5.76s/it]
 71%|███████   | 160187/225000 [311:06:32<96:59:40,  5.39s/it] 
 71%|███████   | 160188/225000 [311:06:37<93:01:32,  5.17s/it]
 71%|███████   | 160189/225000 [311:06:42<90:40:55,  5.04s/it]
 71%|███████   | 160190/225000 [311:06:46<87:47:08,  4.88s/it]
 71%|███████   | 160191/225000 [311:06:57<121:02:24,  6.72s/it]
 71%|███████   | 160192/225000 [311:07:02<111:25:00,  6.19s/it]
 71%|███████   | 160193/225000 [311:07:07<106:23:04,  5.91s/it]
 71%|███████   | 160194/225000 [311:07:13<104:57:57,  5.83s/it]
 71%|███████   | 160195/225000 [311:07:18<100:52:38,  5.60s/it]
 71%|███████   | 160196/225000 [311:07:24<99:49:27,  5.55s/it] 
 71%|███████   | 160197/225000 [311:07:32<114:51:27,  6.38s/it]
 71%|███████   | 160198/225000 [311:07:36<104:13:59,  5.79s/it]
 71%|███████   | 160199/225000 [311:07:40<93:21:14,  5.19s/it] 
 71%|███████   | 160200/225000 [311:07:49<111:06:14,  6.17s/it]
                                                               
{'loss': '0.7884', 'grad_norm': '1.194', 'learning_rate': '1.042e-05', 'epoch': '0.712', 'num_input_tokens_seen': 515030216, 'train_runtime': '1.12e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160200/225000 [311:07:49<111:06:14,  6.17s/it]
 71%|███████   | 160201/225000 [311:07:52<96:42:53,  5.37s/it] 
 71%|███████   | 160202/225000 [311:07:56<87:02:18,  4.84s/it]
 71%|███████   | 160203/225000 [311:08:05<113:24:21,  6.30s/it]
 71%|███████   | 160204/225000 [311:08:10<104:34:51,  5.81s/it]
 71%|███████   | 160205/225000 [311:08:16<107:40:49,  5.98s/it]
 71%|███████   | 160206/225000 [311:08:20<96:43:08,  5.37s/it] 
 71%|███████   | 160207/225000 [311:08:24<88:06:54,  4.90s/it]
 71%|███████   | 160208/225000 [311:08:35<120:16:12,  6.68s/it]
 71%|███████   | 160209/225000 [311:08:39<107:51:35,  5.99s/it]
 71%|███████   | 160210/225000 [311:08:44<98:32:56,  5.48s/it] 
 71%|███████   | 160211/225000 [311:08:48<91:28:20,  5.08s/it]
 71%|███████   | 160212/225000 [311:08:53<93:39:12,  5.20s/it]
 71%|███████   | 160213/225000 [311:09:04<121:19:35,  6.74s/it]
 71%|███████   | 160214/225000 [311:09:08<106:24:05,  5.91s/it]
 71%|███████   | 160215/225000 [311:09:12<97:06:29,  5.40s/it] 
 71%|███████   | 160216/225000 [311:09:16<89:36:16,  4.98s/it]
 71%|███████   | 160217/225000 [311:09:26<117:18:39,  6.52s/it]
 71%|███████   | 160218/225000 [311:09:31<108:32:26,  6.03s/it]
 71%|███████   | 160219/225000 [311:09:35<96:44:35,  5.38s/it] 
 71%|███████   | 160220/225000 [311:09:42<105:24:53,  5.86s/it]
 71%|███████   | 160221/225000 [311:09:47<104:30:18,  5.81s/it]
 71%|███████   | 160222/225000 [311:09:51<92:53:50,  5.16s/it] 
 71%|███████   | 160223/225000 [311:10:01<116:49:12,  6.49s/it]
 71%|███████   | 160224/225000 [311:10:05<103:44:09,  5.77s/it]
 71%|███████   | 160225/225000 [311:10:09<95:44:56,  5.32s/it] 
 71%|███████   | 160226/225000 [311:10:13<90:09:46,  5.01s/it]
 71%|███████   | 160227/225000 [311:10:17<84:28:20,  4.69s/it]
 71%|███████   | 160228/225000 [311:10:25<102:36:30,  5.70s/it]
 71%|███████   | 160229/225000 [311:10:34<118:37:08,  6.59s/it]
 71%|███████   | 160230/225000 [311:10:38<102:15:29,  5.68s/it]
 71%|███████   | 160231/225000 [311:10:42<97:53:23,  5.44s/it] 
 71%|███████   | 160232/225000 [311:10:48<99:58:48,  5.56s/it]
 71%|███████   | 160233/225000 [311:10:54<98:52:52,  5.50s/it]
 71%|███████   | 160234/225000 [311:10:59<97:53:41,  5.44s/it]
 71%|███████   | 160235/225000 [311:11:04<97:07:25,  5.40s/it]
 71%|███████   | 160236/225000 [311:11:09<95:05:34,  5.29s/it]
 71%|███████   | 160237/225000 [311:11:15<95:11:19,  5.29s/it]
 71%|███████   | 160238/225000 [311:11:20<95:52:56,  5.33s/it]
 71%|███████   | 160239/225000 [311:11:25<95:28:54,  5.31s/it]
 71%|███████   | 160240/225000 [311:11:30<93:52:00,  5.22s/it]
 71%|███████   | 160241/225000 [311:11:35<93:54:56,  5.22s/it]
 71%|███████   | 160242/225000 [311:11:40<91:47:25,  5.10s/it]
 71%|███████   | 160243/225000 [311:11:45<90:24:07,  5.03s/it]
 71%|███████   | 160244/225000 [311:11:50<91:43:44,  5.10s/it]
 71%|███████   | 160245/225000 [311:11:56<94:10:15,  5.24s/it]
 71%|███████   | 160246/225000 [311:12:01<93:54:43,  5.22s/it]
 71%|███████   | 160247/225000 [311:12:10<113:13:02,  6.29s/it]
 71%|███████   | 160248/225000 [311:12:19<129:56:03,  7.22s/it]
 71%|███████   | 160249/225000 [311:12:23<113:17:56,  6.30s/it]
 71%|███████   | 160250/225000 [311:12:29<107:19:11,  5.97s/it]
                                                               
{'loss': '0.7714', 'grad_norm': '1.077', 'learning_rate': '1.04e-05', 'epoch': '0.7122', 'num_input_tokens_seen': 515160192, 'train_runtime': '1.12e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160250/225000 [311:12:29<107:19:11,  5.97s/it]
 71%|███████   | 160251/225000 [311:12:33<98:06:36,  5.45s/it] 
 71%|███████   | 160252/225000 [311:12:38<94:25:59,  5.25s/it]
 71%|███████   | 160253/225000 [311:12:42<91:14:58,  5.07s/it]
 71%|███████   | 160254/225000 [311:12:53<121:39:20,  6.76s/it]
 71%|███████   | 160255/225000 [311:13:00<123:03:42,  6.84s/it]
 71%|███████   | 160256/225000 [311:13:04<104:51:02,  5.83s/it]
 71%|███████   | 160257/225000 [311:13:09<100:16:16,  5.58s/it]
 71%|███████   | 160258/225000 [311:13:13<96:08:36,  5.35s/it] 
 71%|███████   | 160259/225000 [311:13:18<93:57:32,  5.22s/it]
 71%|███████   | 160260/225000 [311:13:28<119:39:36,  6.65s/it]
 71%|███████   | 160261/225000 [311:13:32<105:18:25,  5.86s/it]
 71%|███████   | 160262/225000 [311:13:37<97:09:46,  5.40s/it] 
 71%|███████   | 160263/225000 [311:13:45<110:42:09,  6.16s/it]
 71%|███████   | 160264/225000 [311:13:53<121:09:51,  6.74s/it]
 71%|███████   | 160265/225000 [311:13:56<103:53:49,  5.78s/it]
 71%|███████   | 160266/225000 [311:14:01<101:02:02,  5.62s/it]
 71%|███████   | 160267/225000 [311:14:08<107:20:31,  5.97s/it]
 71%|███████   | 160268/225000 [311:14:13<99:36:20,  5.54s/it] 
 71%|███████   | 160269/225000 [311:14:17<94:31:41,  5.26s/it]
 71%|███████   | 160270/225000 [311:14:22<91:29:52,  5.09s/it]
 71%|███████   | 160271/225000 [311:14:26<87:55:54,  4.89s/it]
 71%|███████   | 160272/225000 [311:14:31<86:33:29,  4.81s/it]
 71%|███████   | 160273/225000 [311:14:41<114:21:11,  6.36s/it]
 71%|███████   | 160274/225000 [311:14:49<122:57:45,  6.84s/it]
 71%|███████   | 160275/225000 [311:14:52<104:45:54,  5.83s/it]
 71%|███████   | 160276/225000 [311:14:57<98:18:56,  5.47s/it] 
 71%|███████   | 160277/225000 [311:15:02<94:15:45,  5.24s/it]
 71%|███████   | 160278/225000 [311:15:06<90:04:20,  5.01s/it]
 71%|███████   | 160279/225000 [311:15:11<88:34:51,  4.93s/it]
 71%|███████   | 160280/225000 [311:15:16<87:28:47,  4.87s/it]
 71%|███████   | 160281/225000 [311:15:27<119:38:53,  6.66s/it]
 71%|███████   | 160282/225000 [311:15:36<136:22:12,  7.59s/it]
 71%|███████   | 160283/225000 [311:15:40<116:10:19,  6.46s/it]
 71%|███████   | 160284/225000 [311:15:44<102:59:06,  5.73s/it]
 71%|███████   | 160285/225000 [311:15:49<97:55:41,  5.45s/it] 
 71%|███████   | 160286/225000 [311:15:54<94:32:43,  5.26s/it]
 71%|███████   | 160287/225000 [311:15:59<91:47:41,  5.11s/it]
 71%|███████   | 160288/225000 [311:16:04<91:26:39,  5.09s/it]
 71%|███████   | 160289/225000 [311:16:08<89:28:29,  4.98s/it]
 71%|███████   | 160290/225000 [311:16:18<114:49:05,  6.39s/it]
 71%|███████   | 160291/225000 [311:16:27<127:03:28,  7.07s/it]
 71%|███████   | 160292/225000 [311:16:30<108:00:58,  6.01s/it]
 71%|███████   | 160293/225000 [311:16:35<100:53:19,  5.61s/it]
 71%|███████   | 160294/225000 [311:16:40<100:56:38,  5.62s/it]
 71%|███████   | 160295/225000 [311:16:51<127:50:53,  7.11s/it]
 71%|███████   | 160296/225000 [311:16:56<113:44:07,  6.33s/it]
 71%|███████   | 160297/225000 [311:17:00<103:07:29,  5.74s/it]
 71%|███████   | 160298/225000 [311:17:04<94:57:09,  5.28s/it] 
 71%|███████   | 160299/225000 [311:17:08<89:45:58,  4.99s/it]
 71%|███████   | 160300/225000 [311:17:13<88:36:48,  4.93s/it]
                                                              
{'loss': '0.7746', 'grad_norm': '1.059', 'learning_rate': '1.039e-05', 'epoch': '0.7124', 'num_input_tokens_seen': 515290528, 'train_runtime': '1.121e+06', 'train_tokens_per_second': '459.8'}

 71%|███████   | 160300/225000 [311:17:13<88:36:48,  4.93s/it]
 71%|███████   | 160301/225000 [311:17:21<106:00:33,  5.90s/it]
 71%|███████   | 160302/225000 [311:17:31<127:22:54,  7.09s/it]
 71%|███████   | 160303/225000 [311:17:35<107:16:44,  5.97s/it]
 71%|███████   | 160304/225000 [311:17:43<118:02:00,  6.57s/it]
 71%|███████   | 160305/225000 [311:17:47<107:20:01,  5.97s/it]
 71%|███████   | 160306/225000 [311:17:52<99:16:08,  5.52s/it] 
 71%|███████   | 160307/225000 [311:17:57<95:32:51,  5.32s/it]
 71%|███████   | 160308/225000 [311:18:01<91:42:37,  5.10s/it]
 71%|███████   | 160309/225000 [311:18:06<90:06:19,  5.01s/it]
 71%|███████   | 160310/225000 [311:18:11<88:51:41,  4.95s/it]
 71%|███████   | 160311/225000 [311:18:16<88:51:49,  4.95s/it]
 71%|███████   | 160312/225000 [311:18:21<90:07:24,  5.02s/it]
 71%|███████▏  | 160313/225000 [311:18:32<125:44:01,  7.00s/it]
 71%|███████▏  | 160314/225000 [311:18:37<113:23:03,  6.31s/it]
 71%|███████▏  | 160315/225000 [311:18:41<99:14:00,  5.52s/it] 
 71%|███████▏  | 160316/225000 [311:18:45<92:51:56,  5.17s/it]
 71%|███████▏  | 160317/225000 [311:18:50<91:53:35,  5.11s/it]
 71%|███████▏  | 160318/225000 [311:19:00<117:54:57,  6.56s/it]
 71%|███████▏  | 160319/225000 [311:19:04<101:10:12,  5.63s/it]
 71%|███████▏  | 160320/225000 [311:19:07<90:57:21,  5.06s/it] 
 71%|███████▏  | 160321/225000 [311:19:16<110:55:14,  6.17s/it]
 71%|███████▏  | 160322/225000 [311:19:20<97:51:06,  5.45s/it] 
 71%|███████▏  | 160323/225000 [311:19:29<115:29:19,  6.43s/it]
 71%|███████▏  | 160324/225000 [311:19:33<104:59:19,  5.84s/it]
 71%|███████▏  | 160325/225000 [311:19:40<113:21:34,  6.31s/it]
 71%|███████▏  | 160326/225000 [311:19:46<108:24:11,  6.03s/it]
 71%|███████▏  | 160327/225000 [311:19:50<101:00:59,  5.62s/it]
 71%|███████▏  | 160328/225000 [311:19:55<96:52:08,  5.39s/it] 
 71%|███████▏  | 160329/225000 [311:20:00<94:57:11,  5.29s/it]
 71%|███████▏  | 160330/225000 [311:20:06<96:26:44,  5.37s/it]
 71%|███████▏  | 160331/225000 [311:20:11<92:46:45,  5.16s/it]
 71%|███████▏  | 160332/225000 [311:20:15<90:10:05,  5.02s/it]
 71%|███████▏  | 160333/225000 [311:20:20<90:22:20,  5.03s/it]
 71%|███████▏  | 160334/225000 [311:20:26<91:24:47,  5.09s/it]
 71%|███████▏  | 160335/225000 [311:20:31<90:42:54,  5.05s/it]
 71%|███████▏  | 160336/225000 [311:20:36<91:12:41,  5.08s/it]
 71%|███████▏  | 160337/225000 [311:20:47<127:00:17,  7.07s/it]
 71%|███████▏  | 160338/225000 [311:20:52<114:57:30,  6.40s/it]
 71%|███████▏  | 160339/225000 [311:20:56<102:05:10,  5.68s/it]
 71%|███████▏  | 160340/225000 [311:21:01<94:28:21,  5.26s/it] 
 71%|███████▏  | 160341/225000 [311:21:05<88:27:04,  4.92s/it]
 71%|███████▏  | 160342/225000 [311:21:10<90:01:18,  5.01s/it]
 71%|███████▏  | 160343/225000 [311:21:23<132:24:16,  7.37s/it]
 71%|███████▏  | 160344/225000 [311:21:26<111:00:01,  6.18s/it]
 71%|███████▏  | 160345/225000 [311:21:30<95:57:01,  5.34s/it] 
 71%|███████▏  | 160346/225000 [311:21:38<111:51:39,  6.23s/it]
 71%|███████▏  | 160347/225000 [311:21:41<96:40:59,  5.38s/it] 
 71%|███████▏  | 160348/225000 [311:21:51<119:52:52,  6.68s/it]
 71%|███████▏  | 160349/225000 [311:21:54<102:36:47,  5.71s/it]
 71%|███████▏  | 160350/225000 [311:22:04<120:44:02,  6.72s/it]
                                                               
{'loss': '0.7729', 'grad_norm': '1.077', 'learning_rate': '1.037e-05', 'epoch': '0.7127', 'num_input_tokens_seen': 515420544, 'train_runtime': '1.121e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160350/225000 [311:22:04<120:44:02,  6.72s/it]
 71%|███████▏  | 160351/225000 [311:22:07<105:02:07,  5.85s/it]
 71%|███████▏  | 160352/225000 [311:22:11<90:57:17,  5.06s/it] 
 71%|███████▏  | 160353/225000 [311:22:20<112:46:00,  6.28s/it]
 71%|███████▏  | 160354/225000 [311:22:23<96:35:48,  5.38s/it] 
 71%|███████▏  | 160355/225000 [311:22:30<107:04:27,  5.96s/it]
 71%|███████▏  | 160356/225000 [311:22:34<96:47:36,  5.39s/it] 
 71%|███████▏  | 160357/225000 [311:22:43<117:04:23,  6.52s/it]
 71%|███████▏  | 160358/225000 [311:22:47<100:02:36,  5.57s/it]
 71%|███████▏  | 160359/225000 [311:22:53<105:08:17,  5.86s/it]
 71%|███████▏  | 160360/225000 [311:22:57<95:23:39,  5.31s/it] 
 71%|███████▏  | 160361/225000 [311:23:02<89:02:06,  4.96s/it]
 71%|███████▏  | 160362/225000 [311:23:06<86:30:41,  4.82s/it]
 71%|███████▏  | 160363/225000 [311:23:11<88:01:09,  4.90s/it]
 71%|███████▏  | 160364/225000 [311:23:17<92:11:07,  5.13s/it]
 71%|███████▏  | 160365/225000 [311:23:22<94:43:14,  5.28s/it]
 71%|███████▏  | 160366/225000 [311:23:30<109:48:30,  6.12s/it]
 71%|███████▏  | 160367/225000 [311:23:39<123:00:50,  6.85s/it]
 71%|███████▏  | 160368/225000 [311:23:43<107:56:40,  6.01s/it]
 71%|███████▏  | 160369/225000 [311:23:48<101:55:34,  5.68s/it]
 71%|███████▏  | 160370/225000 [311:23:53<95:38:31,  5.33s/it] 
 71%|███████▏  | 160371/225000 [311:23:57<92:19:17,  5.14s/it]
 71%|███████▏  | 160372/225000 [311:24:09<127:36:00,  7.11s/it]
 71%|███████▏  | 160373/225000 [311:24:14<115:54:18,  6.46s/it]
 71%|███████▏  | 160374/225000 [311:24:19<108:50:35,  6.06s/it]
 71%|███████▏  | 160375/225000 [311:24:24<103:46:19,  5.78s/it]
 71%|███████▏  | 160376/225000 [311:24:28<95:28:04,  5.32s/it] 
 71%|███████▏  | 160377/225000 [311:24:33<94:18:59,  5.25s/it]
 71%|███████▏  | 160378/225000 [311:24:38<93:01:29,  5.18s/it]
 71%|███████▏  | 160379/225000 [311:24:43<91:30:52,  5.10s/it]
 71%|███████▏  | 160380/225000 [311:24:49<92:16:56,  5.14s/it]
 71%|███████▏  | 160381/225000 [311:24:54<93:39:35,  5.22s/it]
 71%|███████▏  | 160382/225000 [311:25:06<128:40:13,  7.17s/it]
 71%|███████▏  | 160383/225000 [311:25:10<110:42:22,  6.17s/it]
 71%|███████▏  | 160384/225000 [311:25:18<121:17:19,  6.76s/it]
 71%|███████▏  | 160385/225000 [311:25:23<114:44:40,  6.39s/it]
 71%|███████▏  | 160386/225000 [311:25:29<109:25:51,  6.10s/it]
 71%|███████▏  | 160387/225000 [311:25:32<97:05:00,  5.41s/it] 
 71%|███████▏  | 160388/225000 [311:25:36<88:26:25,  4.93s/it]
 71%|███████▏  | 160389/225000 [311:25:42<94:01:15,  5.24s/it]
 71%|███████▏  | 160390/225000 [311:25:51<112:25:59,  6.26s/it]
 71%|███████▏  | 160391/225000 [311:25:54<96:27:22,  5.37s/it] 
 71%|███████▏  | 160392/225000 [311:26:02<110:41:50,  6.17s/it]
 71%|███████▏  | 160393/225000 [311:26:06<95:30:28,  5.32s/it] 
 71%|███████▏  | 160394/225000 [311:26:14<113:30:55,  6.33s/it]
 71%|███████▏  | 160395/225000 [311:26:19<104:11:25,  5.81s/it]
 71%|███████▏  | 160396/225000 [311:26:23<97:26:59,  5.43s/it] 
 71%|███████▏  | 160397/225000 [311:26:28<93:34:34,  5.21s/it]
 71%|███████▏  | 160398/225000 [311:26:33<90:29:21,  5.04s/it]
 71%|███████▏  | 160399/225000 [311:26:44<126:02:06,  7.02s/it]
 71%|███████▏  | 160400/225000 [311:26:48<109:44:28,  6.12s/it]
                                                               
{'loss': '0.7869', 'grad_norm': '1.064', 'learning_rate': '1.036e-05', 'epoch': '0.7129', 'num_input_tokens_seen': 515551744, 'train_runtime': '1.121e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160400/225000 [311:26:48<109:44:28,  6.12s/it]
 71%|███████▏  | 160401/225000 [311:26:53<100:12:24,  5.58s/it]
 71%|███████▏  | 160402/225000 [311:26:58<97:39:02,  5.44s/it] 
 71%|███████▏  | 160403/225000 [311:27:02<93:00:46,  5.18s/it]
 71%|███████▏  | 160404/225000 [311:27:08<92:45:00,  5.17s/it]
 71%|███████▏  | 160405/225000 [311:27:13<92:04:14,  5.13s/it]
 71%|███████▏  | 160406/225000 [311:27:18<93:51:38,  5.23s/it]
 71%|███████▏  | 160407/225000 [311:27:23<91:43:12,  5.11s/it]
 71%|███████▏  | 160408/225000 [311:27:30<104:37:13,  5.83s/it]
 71%|███████▏  | 160409/225000 [311:27:41<132:08:10,  7.36s/it]
 71%|███████▏  | 160410/225000 [311:27:45<113:36:26,  6.33s/it]
 71%|███████▏  | 160411/225000 [311:27:49<100:37:39,  5.61s/it]
 71%|███████▏  | 160412/225000 [311:27:53<93:34:29,  5.22s/it] 
 71%|███████▏  | 160413/225000 [311:28:04<120:10:27,  6.70s/it]
 71%|███████▏  | 160414/225000 [311:28:08<107:24:12,  5.99s/it]
 71%|███████▏  | 160415/225000 [311:28:12<99:03:11,  5.52s/it] 
 71%|███████▏  | 160416/225000 [311:28:16<90:54:45,  5.07s/it]
 71%|███████▏  | 160417/225000 [311:28:21<87:06:40,  4.86s/it]
 71%|███████▏  | 160418/225000 [311:28:25<84:47:58,  4.73s/it]
 71%|███████▏  | 160419/225000 [311:28:30<84:11:40,  4.69s/it]
 71%|███████▏  | 160420/225000 [311:28:43<128:01:13,  7.14s/it]
 71%|███████▏  | 160421/225000 [311:28:47<114:51:47,  6.40s/it]
 71%|███████▏  | 160422/225000 [311:28:54<118:30:32,  6.61s/it]
 71%|███████▏  | 160423/225000 [311:28:58<102:08:06,  5.69s/it]
 71%|███████▏  | 160424/225000 [311:29:05<108:56:21,  6.07s/it]
 71%|███████▏  | 160425/225000 [311:29:11<107:20:37,  5.98s/it]
 71%|███████▏  | 160426/225000 [311:29:18<112:29:33,  6.27s/it]
 71%|███████▏  | 160427/225000 [311:29:21<98:00:41,  5.46s/it] 
 71%|███████▏  | 160428/225000 [311:29:31<119:05:10,  6.64s/it]
 71%|███████▏  | 160429/225000 [311:29:34<103:34:54,  5.77s/it]
 71%|███████▏  | 160430/225000 [311:29:38<91:30:18,  5.10s/it] 
 71%|███████▏  | 160431/225000 [311:29:46<107:04:13,  5.97s/it]
 71%|███████▏  | 160432/225000 [311:29:53<114:09:28,  6.36s/it]
 71%|███████▏  | 160433/225000 [311:29:57<99:39:40,  5.56s/it] 
 71%|███████▏  | 160434/225000 [311:30:06<116:47:44,  6.51s/it]
 71%|███████▏  | 160435/225000 [311:30:10<105:15:53,  5.87s/it]
 71%|███████▏  | 160436/225000 [311:30:14<97:47:13,  5.45s/it] 
 71%|███████▏  | 160437/225000 [311:30:19<92:37:06,  5.16s/it]
 71%|███████▏  | 160438/225000 [311:30:24<92:04:35,  5.13s/it]
 71%|███████▏  | 160439/225000 [311:30:29<90:50:27,  5.07s/it]
 71%|███████▏  | 160440/225000 [311:30:33<88:07:15,  4.91s/it]
 71%|███████▏  | 160441/225000 [311:30:40<97:16:05,  5.42s/it]
 71%|███████▏  | 160442/225000 [311:30:51<126:58:44,  7.08s/it]
 71%|███████▏  | 160443/225000 [311:30:55<108:24:00,  6.04s/it]
 71%|███████▏  | 160444/225000 [311:31:03<119:13:29,  6.65s/it]
 71%|███████▏  | 160445/225000 [311:31:08<112:52:22,  6.29s/it]
 71%|███████▏  | 160446/225000 [311:31:12<101:50:43,  5.68s/it]
 71%|███████▏  | 160447/225000 [311:31:17<96:41:31,  5.39s/it] 
 71%|███████▏  | 160448/225000 [311:31:22<92:05:34,  5.14s/it]
 71%|███████▏  | 160449/225000 [311:31:26<89:42:24,  5.00s/it]
 71%|███████▏  | 160450/225000 [311:31:33<97:40:26,  5.45s/it]
                                                              
{'loss': '0.7746', 'grad_norm': '1.045', 'learning_rate': '1.034e-05', 'epoch': '0.7131', 'num_input_tokens_seen': 515683264, 'train_runtime': '1.121e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160450/225000 [311:31:33<97:40:26,  5.45s/it]
 71%|███████▏  | 160451/225000 [311:31:42<119:45:14,  6.68s/it]
 71%|███████▏  | 160452/225000 [311:31:46<101:19:59,  5.65s/it]
 71%|███████▏  | 160453/225000 [311:31:53<108:44:12,  6.06s/it]
 71%|███████▏  | 160454/225000 [311:31:58<106:38:58,  5.95s/it]
 71%|███████▏  | 160455/225000 [311:32:02<93:17:00,  5.20s/it] 
 71%|███████▏  | 160456/225000 [311:32:10<111:54:53,  6.24s/it]
 71%|███████▏  | 160457/225000 [311:32:14<99:10:51,  5.53s/it] 
 71%|███████▏  | 160458/225000 [311:32:18<89:41:56,  5.00s/it]
 71%|███████▏  | 160459/225000 [311:32:23<90:07:27,  5.03s/it]
 71%|███████▏  | 160460/225000 [311:32:32<111:51:10,  6.24s/it]
 71%|███████▏  | 160461/225000 [311:32:38<111:31:06,  6.22s/it]
 71%|███████▏  | 160462/225000 [311:32:44<107:20:08,  5.99s/it]
 71%|███████▏  | 160463/225000 [311:32:47<92:42:07,  5.17s/it] 
 71%|███████▏  | 160464/225000 [311:32:55<105:28:24,  5.88s/it]
 71%|███████▏  | 160465/225000 [311:33:00<101:13:07,  5.65s/it]
 71%|███████▏  | 160466/225000 [311:33:04<93:11:53,  5.20s/it] 
 71%|███████▏  | 160467/225000 [311:33:08<86:39:06,  4.83s/it]
 71%|███████▏  | 160468/225000 [311:33:20<125:50:07,  7.02s/it]
 71%|███████▏  | 160469/225000 [311:33:24<108:06:08,  6.03s/it]
 71%|███████▏  | 160470/225000 [311:33:27<92:27:48,  5.16s/it] 
 71%|███████▏  | 160471/225000 [311:33:33<96:57:02,  5.41s/it]
 71%|███████▏  | 160472/225000 [311:33:42<115:59:31,  6.47s/it]
 71%|███████▏  | 160473/225000 [311:33:46<102:35:52,  5.72s/it]
 71%|███████▏  | 160474/225000 [311:33:51<100:52:27,  5.63s/it]
 71%|███████▏  | 160475/225000 [311:33:57<101:26:43,  5.66s/it]
 71%|███████▏  | 160476/225000 [311:34:02<99:24:12,  5.55s/it] 
 71%|███████▏  | 160477/225000 [311:34:08<98:57:57,  5.52s/it]
 71%|███████▏  | 160478/225000 [311:34:13<100:09:14,  5.59s/it]
 71%|███████▏  | 160479/225000 [311:34:19<99:52:34,  5.57s/it] 
 71%|███████▏  | 160480/225000 [311:34:24<98:26:50,  5.49s/it]
 71%|███████▏  | 160481/225000 [311:34:32<107:42:47,  6.01s/it]
 71%|███████▏  | 160482/225000 [311:34:38<109:50:27,  6.13s/it]
 71%|███████▏  | 160483/225000 [311:34:42<100:23:05,  5.60s/it]
 71%|███████▏  | 160484/225000 [311:34:46<92:25:48,  5.16s/it] 
 71%|███████▏  | 160485/225000 [311:34:51<86:44:57,  4.84s/it]
 71%|███████▏  | 160486/225000 [311:34:59<105:40:05,  5.90s/it]
 71%|███████▏  | 160487/225000 [311:35:07<119:48:58,  6.69s/it]
 71%|███████▏  | 160488/225000 [311:35:11<105:38:22,  5.90s/it]
 71%|███████▏  | 160489/225000 [311:35:16<95:55:17,  5.35s/it] 
 71%|███████▏  | 160490/225000 [311:35:20<90:04:41,  5.03s/it]
 71%|███████▏  | 160491/225000 [311:35:32<127:02:46,  7.09s/it]
 71%|███████▏  | 160492/225000 [311:35:36<113:59:37,  6.36s/it]
 71%|███████▏  | 160493/225000 [311:35:41<103:59:25,  5.80s/it]
 71%|███████▏  | 160494/225000 [311:35:46<98:20:29,  5.49s/it] 
 71%|███████▏  | 160495/225000 [311:35:50<94:34:18,  5.28s/it]
 71%|███████▏  | 160496/225000 [311:36:00<115:51:52,  6.47s/it]
 71%|███████▏  | 160497/225000 [311:36:07<118:05:42,  6.59s/it]
 71%|███████▏  | 160498/225000 [311:36:11<105:50:45,  5.91s/it]
 71%|███████▏  | 160499/225000 [311:36:19<118:55:43,  6.64s/it]
 71%|███████▏  | 160500/225000 [311:36:23<103:56:12,  5.80s/it]
                                                               
{'loss': '0.7678', 'grad_norm': '1.116', 'learning_rate': '1.033e-05', 'epoch': '0.7133', 'num_input_tokens_seen': 515813832, 'train_runtime': '1.122e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160500/225000 [311:36:23<103:56:12,  5.80s/it]
 71%|███████▏  | 160501/225000 [311:36:30<112:42:07,  6.29s/it]
 71%|███████▏  | 160502/225000 [311:36:34<99:25:34,  5.55s/it] 
 71%|███████▏  | 160503/225000 [311:36:43<116:56:45,  6.53s/it]
 71%|███████▏  | 160504/225000 [311:36:46<98:08:15,  5.48s/it] 
 71%|███████▏  | 160505/225000 [311:36:54<110:46:05,  6.18s/it]
 71%|███████▏  | 160506/225000 [311:36:59<102:03:23,  5.70s/it]
 71%|███████▏  | 160507/225000 [311:37:03<94:20:13,  5.27s/it] 
 71%|███████▏  | 160508/225000 [311:37:07<88:18:25,  4.93s/it]
 71%|███████▏  | 160509/225000 [311:37:11<85:07:21,  4.75s/it]
 71%|███████▏  | 160510/225000 [311:37:16<82:56:54,  4.63s/it]
 71%|███████▏  | 160511/225000 [311:37:20<83:21:04,  4.65s/it]
 71%|███████▏  | 160512/225000 [311:37:25<84:01:54,  4.69s/it]
 71%|███████▏  | 160513/225000 [311:37:32<96:53:16,  5.41s/it]
 71%|███████▏  | 160514/225000 [311:37:44<128:58:03,  7.20s/it]
 71%|███████▏  | 160515/225000 [311:37:49<118:07:43,  6.59s/it]
 71%|███████▏  | 160516/225000 [311:37:54<109:27:04,  6.11s/it]
 71%|███████▏  | 160517/225000 [311:38:02<118:41:39,  6.63s/it]
 71%|███████▏  | 160518/225000 [311:38:05<101:06:01,  5.64s/it]
 71%|███████▏  | 160519/225000 [311:38:08<88:19:08,  4.93s/it] 
 71%|███████▏  | 160520/225000 [311:38:12<83:57:45,  4.69s/it]
 71%|███████▏  | 160521/225000 [311:38:23<117:39:59,  6.57s/it]
 71%|███████▏  | 160522/225000 [311:38:28<107:33:05,  6.00s/it]
 71%|███████▏  | 160523/225000 [311:38:32<98:06:24,  5.48s/it] 
 71%|███████▏  | 160524/225000 [311:38:36<90:41:45,  5.06s/it]
 71%|███████▏  | 160525/225000 [311:38:40<85:48:15,  4.79s/it]
 71%|███████▏  | 160526/225000 [311:38:46<89:07:24,  4.98s/it]
 71%|███████▏  | 160527/225000 [311:38:57<124:32:23,  6.95s/it]
 71%|███████▏  | 160528/225000 [311:39:01<105:06:48,  5.87s/it]
 71%|███████▏  | 160529/225000 [311:39:05<97:32:59,  5.45s/it] 
 71%|███████▏  | 160530/225000 [311:39:10<95:04:35,  5.31s/it]
 71%|███████▏  | 160531/225000 [311:39:15<94:08:42,  5.26s/it]
 71%|███████▏  | 160532/225000 [311:39:20<91:14:06,  5.09s/it]
 71%|███████▏  | 160533/225000 [311:39:25<89:01:29,  4.97s/it]
 71%|███████▏  | 160534/225000 [311:39:30<89:30:43,  5.00s/it]
 71%|███████▏  | 160535/225000 [311:39:35<88:20:07,  4.93s/it]
 71%|███████▏  | 160536/225000 [311:39:40<89:31:12,  5.00s/it]
 71%|███████▏  | 160537/225000 [311:39:49<111:49:24,  6.24s/it]
 71%|███████▏  | 160538/225000 [311:39:57<123:31:58,  6.90s/it]
 71%|███████▏  | 160539/225000 [311:40:01<104:34:43,  5.84s/it]
 71%|███████▏  | 160540/225000 [311:40:09<115:15:54,  6.44s/it]
 71%|███████▏  | 160541/225000 [311:40:12<97:19:48,  5.44s/it] 
 71%|███████▏  | 160542/225000 [311:40:20<112:02:18,  6.26s/it]
 71%|███████▏  | 160543/225000 [311:40:25<106:38:26,  5.96s/it]
 71%|███████▏  | 160544/225000 [311:40:30<101:41:01,  5.68s/it]
 71%|███████▏  | 160545/225000 [311:40:35<94:54:24,  5.30s/it] 
 71%|███████▏  | 160546/225000 [311:40:40<94:06:06,  5.26s/it]
 71%|███████▏  | 160547/225000 [311:40:45<94:21:44,  5.27s/it]
 71%|███████▏  | 160548/225000 [311:40:50<95:24:33,  5.33s/it]
 71%|███████▏  | 160549/225000 [311:40:56<95:38:29,  5.34s/it]
 71%|███████▏  | 160550/225000 [311:41:00<91:53:50,  5.13s/it]
                                                              
{'loss': '0.7824', 'grad_norm': '1.119', 'learning_rate': '1.031e-05', 'epoch': '0.7136', 'num_input_tokens_seen': 515943912, 'train_runtime': '1.122e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160550/225000 [311:41:00<91:53:50,  5.13s/it]
 71%|███████▏  | 160551/225000 [311:41:05<89:29:41,  5.00s/it]
 71%|███████▏  | 160552/225000 [311:41:11<91:38:16,  5.12s/it]
 71%|███████▏  | 160553/225000 [311:41:16<93:07:09,  5.20s/it]
 71%|███████▏  | 160554/225000 [311:41:28<129:53:25,  7.26s/it]
 71%|███████▏  | 160555/225000 [311:41:31<109:02:31,  6.09s/it]
 71%|███████▏  | 160556/225000 [311:41:36<102:00:37,  5.70s/it]
 71%|███████▏  | 160557/225000 [311:41:43<106:40:47,  5.96s/it]
 71%|███████▏  | 160558/225000 [311:41:47<95:56:58,  5.36s/it] 
 71%|███████▏  | 160559/225000 [311:41:51<93:08:34,  5.20s/it]
 71%|███████▏  | 160560/225000 [311:41:56<89:31:18,  5.00s/it]
 71%|███████▏  | 160561/225000 [311:42:00<86:30:13,  4.83s/it]
 71%|███████▏  | 160562/225000 [311:42:11<116:24:05,  6.50s/it]
 71%|███████▏  | 160563/225000 [311:42:16<106:41:15,  5.96s/it]
 71%|███████▏  | 160564/225000 [311:42:23<112:54:00,  6.31s/it]
 71%|███████▏  | 160565/225000 [311:42:29<114:08:37,  6.38s/it]
 71%|███████▏  | 160566/225000 [311:42:33<99:37:29,  5.57s/it] 
 71%|███████▏  | 160567/225000 [311:42:38<98:03:23,  5.48s/it]
 71%|███████▏  | 160568/225000 [311:42:45<105:46:39,  5.91s/it]
 71%|███████▏  | 160569/225000 [311:42:49<94:32:03,  5.28s/it] 
 71%|███████▏  | 160570/225000 [311:42:55<97:34:27,  5.45s/it]
 71%|███████▏  | 160571/225000 [311:43:03<113:31:56,  6.34s/it]
 71%|███████▏  | 160572/225000 [311:43:07<100:35:40,  5.62s/it]
 71%|███████▏  | 160573/225000 [311:43:12<94:01:25,  5.25s/it] 
 71%|███████▏  | 160574/225000 [311:43:20<109:22:35,  6.11s/it]
 71%|███████▏  | 160575/225000 [311:43:24<99:20:07,  5.55s/it] 
 71%|███████▏  | 160576/225000 [311:43:28<94:16:11,  5.27s/it]
 71%|███████▏  | 160577/225000 [311:43:37<114:03:54,  6.37s/it]
 71%|███████▏  | 160578/225000 [311:43:41<98:16:35,  5.49s/it] 
 71%|███████▏  | 160579/225000 [311:43:48<106:43:13,  5.96s/it]
 71%|███████▏  | 160580/225000 [311:43:53<102:52:35,  5.75s/it]
 71%|███████▏  | 160581/225000 [311:43:57<92:55:37,  5.19s/it] 
 71%|███████▏  | 160582/225000 [311:44:01<87:11:13,  4.87s/it]
 71%|███████▏  | 160583/225000 [311:44:05<82:12:38,  4.59s/it]
 71%|███████▏  | 160584/225000 [311:44:16<113:22:43,  6.34s/it]
 71%|███████▏  | 160585/225000 [311:44:19<98:24:16,  5.50s/it] 
 71%|███████▏  | 160586/225000 [311:44:25<101:44:44,  5.69s/it]
 71%|███████▏  | 160587/225000 [311:44:35<121:32:51,  6.79s/it]
 71%|███████▏  | 160588/225000 [311:44:38<105:01:37,  5.87s/it]
 71%|███████▏  | 160589/225000 [311:44:46<116:35:34,  6.52s/it]
 71%|███████▏  | 160590/225000 [311:44:50<103:40:48,  5.79s/it]
 71%|███████▏  | 160591/225000 [311:44:54<93:24:07,  5.22s/it] 
 71%|███████▏  | 160592/225000 [311:45:03<112:25:44,  6.28s/it]
 71%|███████▏  | 160593/225000 [311:45:07<99:16:19,  5.55s/it] 
 71%|███████▏  | 160594/225000 [311:45:16<116:12:08,  6.50s/it]
 71%|███████▏  | 160595/225000 [311:45:19<100:25:10,  5.61s/it]
 71%|███████▏  | 160596/225000 [311:45:27<111:31:24,  6.23s/it]
 71%|███████▏  | 160597/225000 [311:45:30<96:29:34,  5.39s/it] 
 71%|███████▏  | 160598/225000 [311:45:35<90:45:01,  5.07s/it]
 71%|███████▏  | 160599/225000 [311:45:39<87:59:26,  4.92s/it]
 71%|███████▏  | 160600/225000 [311:45:44<85:45:42,  4.79s/it]
                                                              
{'loss': '0.7651', 'grad_norm': '1.199', 'learning_rate': '1.03e-05', 'epoch': '0.7138', 'num_input_tokens_seen': 516073408, 'train_runtime': '1.122e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160600/225000 [311:45:44<85:45:42,  4.79s/it]
 71%|███████▏  | 160601/225000 [311:45:48<85:29:01,  4.78s/it]
 71%|███████▏  | 160602/225000 [311:45:53<86:45:34,  4.85s/it]
 71%|███████▏  | 160603/225000 [311:45:58<87:19:26,  4.88s/it]
 71%|███████▏  | 160604/225000 [311:46:03<87:08:56,  4.87s/it]
 71%|███████▏  | 160605/225000 [311:46:11<104:47:16,  5.86s/it]
 71%|███████▏  | 160606/225000 [311:46:23<133:04:57,  7.44s/it]
 71%|███████▏  | 160607/225000 [311:46:26<111:00:27,  6.21s/it]
 71%|███████▏  | 160608/225000 [311:46:30<102:12:31,  5.71s/it]
 71%|███████▏  | 160609/225000 [311:46:35<97:43:07,  5.46s/it] 
 71%|███████▏  | 160610/225000 [311:46:41<97:22:44,  5.44s/it]
 71%|███████▏  | 160611/225000 [311:46:46<94:18:33,  5.27s/it]
 71%|███████▏  | 160612/225000 [311:46:51<93:10:27,  5.21s/it]
 71%|███████▏  | 160613/225000 [311:46:55<91:11:08,  5.10s/it]
 71%|███████▏  | 160614/225000 [311:47:07<123:39:36,  6.91s/it]
 71%|███████▏  | 160615/225000 [311:47:14<124:00:55,  6.93s/it]
 71%|███████▏  | 160616/225000 [311:47:19<115:58:43,  6.48s/it]
 71%|███████▏  | 160617/225000 [311:47:25<110:37:13,  6.19s/it]
 71%|███████▏  | 160618/225000 [311:47:30<109:13:07,  6.11s/it]
 71%|███████▏  | 160619/225000 [311:47:36<107:09:40,  5.99s/it]
 71%|███████▏  | 160620/225000 [311:47:40<96:33:34,  5.40s/it] 
 71%|███████▏  | 160621/225000 [311:47:44<89:35:59,  5.01s/it]
 71%|███████▏  | 160622/225000 [311:47:48<84:16:23,  4.71s/it]
 71%|███████▏  | 160623/225000 [311:47:52<80:42:26,  4.51s/it]
 71%|███████▏  | 160624/225000 [311:47:58<85:56:18,  4.81s/it]
 71%|███████▏  | 160625/225000 [311:48:10<123:27:25,  6.90s/it]
 71%|███████▏  | 160626/225000 [311:48:15<117:26:14,  6.57s/it]
 71%|███████▏  | 160627/225000 [311:48:20<106:44:08,  5.97s/it]
 71%|███████▏  | 160628/225000 [311:48:25<100:41:13,  5.63s/it]
 71%|███████▏  | 160629/225000 [311:48:29<95:13:28,  5.33s/it] 
 71%|███████▏  | 160630/225000 [311:48:34<90:15:15,  5.05s/it]
 71%|███████▏  | 160631/225000 [311:48:38<87:39:26,  4.90s/it]
 71%|███████▏  | 160632/225000 [311:48:43<86:22:26,  4.83s/it]
 71%|███████▏  | 160633/225000 [311:48:52<107:54:51,  6.04s/it]
 71%|███████▏  | 160634/225000 [311:49:03<133:37:34,  7.47s/it]
 71%|███████▏  | 160635/225000 [311:49:06<110:16:57,  6.17s/it]
 71%|███████▏  | 160636/225000 [311:49:14<120:27:39,  6.74s/it]
 71%|███████▏  | 160637/225000 [311:49:18<105:32:17,  5.90s/it]
 71%|███████▏  | 160638/225000 [311:49:22<94:26:42,  5.28s/it] 
 71%|███████▏  | 160639/225000 [311:49:32<118:51:27,  6.65s/it]
 71%|███████▏  | 160640/225000 [311:49:36<104:37:28,  5.85s/it]
 71%|███████▏  | 160641/225000 [311:49:44<119:06:50,  6.66s/it]
 71%|███████▏  | 160642/225000 [311:49:48<104:59:58,  5.87s/it]
 71%|███████▏  | 160643/225000 [311:49:52<95:51:11,  5.36s/it] 
 71%|███████▏  | 160644/225000 [311:49:56<89:10:42,  4.99s/it]
 71%|███████▏  | 160645/225000 [311:50:03<96:38:28,  5.41s/it]
 71%|███████▏  | 160646/225000 [311:50:12<117:01:36,  6.55s/it]
 71%|███████▏  | 160647/225000 [311:50:15<98:25:46,  5.51s/it] 
 71%|███████▏  | 160648/225000 [311:50:23<108:46:51,  6.09s/it]
 71%|███████▏  | 160649/225000 [311:50:26<97:12:13,  5.44s/it] 
 71%|███████▏  | 160650/225000 [311:50:31<91:24:27,  5.11s/it]
                                                              
{'loss': '0.7625', 'grad_norm': '1.03', 'learning_rate': '1.028e-05', 'epoch': '0.714', 'num_input_tokens_seen': 516203296, 'train_runtime': '1.123e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160650/225000 [311:50:31<91:24:27,  5.11s/it]
 71%|███████▏  | 160651/225000 [311:50:35<85:04:04,  4.76s/it]
 71%|███████▏  | 160652/225000 [311:50:41<90:46:55,  5.08s/it]
 71%|███████▏  | 160653/225000 [311:50:52<124:30:34,  6.97s/it]
 71%|███████▏  | 160654/225000 [311:50:55<105:49:02,  5.92s/it]
 71%|███████▏  | 160655/225000 [311:51:02<107:34:25,  6.02s/it]
 71%|███████▏  | 160656/225000 [311:51:07<101:15:45,  5.67s/it]
 71%|███████▏  | 160657/225000 [311:51:12<98:18:02,  5.50s/it] 
 71%|███████▏  | 160658/225000 [311:51:16<92:27:58,  5.17s/it]
 71%|███████▏  | 160659/225000 [311:51:21<92:00:04,  5.15s/it]
 71%|███████▏  | 160660/225000 [311:51:26<91:32:31,  5.12s/it]
 71%|███████▏  | 160661/225000 [311:51:31<90:51:40,  5.08s/it]
 71%|███████▏  | 160662/225000 [311:51:37<92:32:18,  5.18s/it]
 71%|███████▏  | 160663/225000 [311:51:42<94:03:32,  5.26s/it]
 71%|███████▏  | 160664/225000 [311:51:47<94:07:02,  5.27s/it]
 71%|███████▏  | 160665/225000 [311:51:52<92:45:29,  5.19s/it]
 71%|███████▏  | 160666/225000 [311:51:57<91:59:19,  5.15s/it]
 71%|███████▏  | 160667/225000 [311:52:03<94:27:24,  5.29s/it]
 71%|███████▏  | 160668/225000 [311:52:08<94:47:58,  5.30s/it]
 71%|███████▏  | 160669/225000 [311:52:13<93:03:34,  5.21s/it]
 71%|███████▏  | 160670/225000 [311:52:18<91:51:06,  5.14s/it]
 71%|███████▏  | 160671/225000 [311:52:25<101:19:35,  5.67s/it]
 71%|███████▏  | 160672/225000 [311:52:38<138:29:58,  7.75s/it]
 71%|███████▏  | 160673/225000 [311:52:42<119:38:44,  6.70s/it]
 71%|███████▏  | 160674/225000 [311:52:49<121:09:48,  6.78s/it]
 71%|███████▏  | 160675/225000 [311:52:55<115:17:44,  6.45s/it]
 71%|███████▏  | 160676/225000 [311:52:58<98:43:30,  5.53s/it] 
 71%|███████▏  | 160677/225000 [311:53:06<112:07:08,  6.28s/it]
 71%|███████▏  | 160678/225000 [311:53:10<99:25:52,  5.57s/it] 
 71%|███████▏  | 160679/225000 [311:53:15<94:19:05,  5.28s/it]
 71%|███████▏  | 160680/225000 [311:53:19<88:42:44,  4.97s/it]
 71%|███████▏  | 160681/225000 [311:53:24<88:09:13,  4.93s/it]
 71%|███████▏  | 160682/225000 [311:53:28<85:48:35,  4.80s/it]
 71%|███████▏  | 160683/225000 [311:53:39<118:56:29,  6.66s/it]
 71%|███████▏  | 160684/225000 [311:53:45<116:31:00,  6.52s/it]
 71%|███████▏  | 160685/225000 [311:53:49<100:39:40,  5.63s/it]
 71%|███████▏  | 160686/225000 [311:53:53<92:41:48,  5.19s/it] 
 71%|███████▏  | 160687/225000 [311:53:58<89:03:39,  4.99s/it]
 71%|███████▏  | 160688/225000 [311:54:03<88:56:23,  4.98s/it]
 71%|███████▏  | 160689/225000 [311:54:07<85:22:11,  4.78s/it]
 71%|███████▏  | 160690/225000 [311:54:12<86:24:22,  4.84s/it]
 71%|███████▏  | 160691/225000 [311:54:17<87:23:31,  4.89s/it]
 71%|███████▏  | 160692/225000 [311:54:21<84:28:46,  4.73s/it]
 71%|███████▏  | 160693/225000 [311:54:26<86:16:45,  4.83s/it]
 71%|███████▏  | 160694/225000 [311:54:38<120:49:35,  6.76s/it]
 71%|███████▏  | 160695/225000 [311:54:44<120:32:42,  6.75s/it]
 71%|███████▏  | 160696/225000 [311:54:48<101:37:59,  5.69s/it]
 71%|███████▏  | 160697/225000 [311:54:54<107:59:45,  6.05s/it]
 71%|███████▏  | 160698/225000 [311:54:58<94:16:26,  5.28s/it] 
 71%|███████▏  | 160699/225000 [311:55:07<114:20:04,  6.40s/it]
 71%|███████▏  | 160700/225000 [311:55:14<115:33:30,  6.47s/it]
                                                               
{'loss': '0.7718', 'grad_norm': '1.254', 'learning_rate': '1.027e-05', 'epoch': '0.7142', 'num_input_tokens_seen': 516332128, 'train_runtime': '1.123e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160700/225000 [311:55:14<115:33:30,  6.47s/it]
 71%|███████▏  | 160701/225000 [311:55:18<106:15:23,  5.95s/it]
 71%|███████▏  | 160702/225000 [311:55:23<97:58:15,  5.49s/it] 
 71%|███████▏  | 160703/225000 [311:55:27<92:08:47,  5.16s/it]
 71%|███████▏  | 160704/225000 [311:55:32<89:51:36,  5.03s/it]
 71%|███████▏  | 160705/225000 [311:55:37<88:03:43,  4.93s/it]
 71%|███████▏  | 160706/225000 [311:55:43<94:30:25,  5.29s/it]
 71%|███████▏  | 160707/225000 [311:55:53<122:47:25,  6.88s/it]
 71%|███████▏  | 160708/225000 [311:55:57<104:08:28,  5.83s/it]
 71%|███████▏  | 160709/225000 [311:56:02<99:25:30,  5.57s/it] 
 71%|███████▏  | 160710/225000 [311:56:09<106:52:25,  5.98s/it]
 71%|███████▏  | 160711/225000 [311:56:12<93:22:01,  5.23s/it] 
 71%|███████▏  | 160712/225000 [311:56:20<110:31:07,  6.19s/it]
 71%|███████▏  | 160713/225000 [311:56:24<99:12:46,  5.56s/it] 
 71%|███████▏  | 160714/225000 [311:56:32<110:07:09,  6.17s/it]
 71%|███████▏  | 160715/225000 [311:56:37<102:45:20,  5.75s/it]
 71%|███████▏  | 160716/225000 [311:56:41<93:54:02,  5.26s/it] 
 71%|███████▏  | 160717/225000 [311:56:49<107:46:12,  6.04s/it]
 71%|███████▏  | 160718/225000 [311:56:52<94:40:54,  5.30s/it] 
 71%|███████▏  | 160719/225000 [311:56:57<88:18:57,  4.95s/it]
 71%|███████▏  | 160720/225000 [311:57:07<120:18:06,  6.74s/it]
 71%|███████▏  | 160721/225000 [311:57:11<101:31:25,  5.69s/it]
 71%|███████▏  | 160722/225000 [311:57:18<111:58:16,  6.27s/it]
 71%|███████▏  | 160723/225000 [311:57:22<96:26:14,  5.40s/it] 
 71%|███████▏  | 160724/225000 [311:57:28<103:18:01,  5.79s/it]
 71%|███████▏  | 160725/225000 [311:57:32<89:38:03,  5.02s/it] 
 71%|███████▏  | 160726/225000 [311:57:39<103:40:07,  5.81s/it]
 71%|███████▏  | 160727/225000 [311:57:45<102:14:59,  5.73s/it]
 71%|███████▏  | 160728/225000 [311:57:49<96:13:22,  5.39s/it] 
 71%|███████▏  | 160729/225000 [311:57:54<94:17:36,  5.28s/it]
 71%|███████▏  | 160730/225000 [311:57:59<92:34:49,  5.19s/it]
 71%|███████▏  | 160731/225000 [311:58:04<90:06:37,  5.05s/it]
 71%|███████▏  | 160732/225000 [311:58:11<101:57:36,  5.71s/it]
 71%|███████▏  | 160733/225000 [311:58:23<131:33:06,  7.37s/it]
 71%|███████▏  | 160734/225000 [311:58:27<114:00:39,  6.39s/it]
 71%|███████▏  | 160735/225000 [311:58:32<108:45:04,  6.09s/it]
 71%|███████▏  | 160736/225000 [311:58:40<115:51:54,  6.49s/it]
 71%|███████▏  | 160737/225000 [311:58:43<100:09:00,  5.61s/it]
 71%|███████▏  | 160738/225000 [311:58:47<88:34:11,  4.96s/it] 
 71%|███████▏  | 160739/225000 [311:58:52<89:15:06,  5.00s/it]
 71%|███████▏  | 160740/225000 [311:59:00<109:20:16,  6.13s/it]
 71%|███████▏  | 160741/225000 [311:59:04<97:45:19,  5.48s/it] 
 71%|███████▏  | 160742/225000 [311:59:09<92:57:10,  5.21s/it]
 71%|███████▏  | 160743/225000 [311:59:13<87:59:58,  4.93s/it]
 71%|███████▏  | 160744/225000 [311:59:17<82:54:14,  4.64s/it]
 71%|███████▏  | 160745/225000 [311:59:22<82:01:14,  4.60s/it]
 71%|███████▏  | 160746/225000 [311:59:31<108:32:08,  6.08s/it]
 71%|███████▏  | 160747/225000 [311:59:39<116:33:35,  6.53s/it]
 71%|███████▏  | 160748/225000 [311:59:42<99:41:53,  5.59s/it] 
 71%|███████▏  | 160749/225000 [311:59:50<110:56:40,  6.22s/it]
 71%|███████▏  | 160750/225000 [311:59:56<109:50:08,  6.15s/it]
                                                               
{'loss': '0.7664', 'grad_norm': '1.229', 'learning_rate': '1.025e-05', 'epoch': '0.7144', 'num_input_tokens_seen': 516461712, 'train_runtime': '1.123e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160750/225000 [311:59:56<109:50:08,  6.15s/it]
 71%|███████▏  | 160751/225000 [312:00:02<108:25:33,  6.08s/it]
 71%|███████▏  | 160752/225000 [312:00:06<97:08:59,  5.44s/it] 
 71%|███████▏  | 160753/225000 [312:00:10<91:16:28,  5.11s/it]
 71%|███████▏  | 160754/225000 [312:00:20<116:03:46,  6.50s/it]
 71%|███████▏  | 160755/225000 [312:00:25<109:04:52,  6.11s/it]
 71%|███████▏  | 160756/225000 [312:00:29<95:34:25,  5.36s/it] 
 71%|███████▏  | 160757/225000 [312:00:34<95:59:58,  5.38s/it]
 71%|███████▏  | 160758/225000 [312:00:39<93:04:28,  5.22s/it]
 71%|███████▏  | 160759/225000 [312:00:47<109:21:06,  6.13s/it]
 71%|███████▏  | 160760/225000 [312:00:54<115:46:43,  6.49s/it]
 71%|███████▏  | 160761/225000 [312:00:58<101:08:08,  5.67s/it]
 71%|███████▏  | 160762/225000 [312:01:06<114:51:45,  6.44s/it]
 71%|███████▏  | 160763/225000 [312:01:10<100:19:13,  5.62s/it]
 71%|███████▏  | 160764/225000 [312:01:18<110:42:15,  6.20s/it]
 71%|███████▏  | 160765/225000 [312:01:25<118:36:50,  6.65s/it]
 71%|███████▏  | 160766/225000 [312:01:30<105:29:15,  5.91s/it]
 71%|███████▏  | 160767/225000 [312:01:34<96:27:11,  5.41s/it] 
 71%|███████▏  | 160768/225000 [312:01:38<88:30:12,  4.96s/it]
 71%|███████▏  | 160769/225000 [312:01:45<100:00:33,  5.61s/it]
 71%|███████▏  | 160770/225000 [312:01:52<110:08:14,  6.17s/it]
 71%|███████▏  | 160771/225000 [312:01:59<111:29:00,  6.25s/it]
 71%|███████▏  | 160772/225000 [312:02:03<99:02:54,  5.55s/it] 
 71%|███████▏  | 160773/225000 [312:02:07<91:20:10,  5.12s/it]
 71%|███████▏  | 160774/225000 [312:02:12<92:38:57,  5.19s/it]
 71%|███████▏  | 160775/225000 [312:02:22<118:28:05,  6.64s/it]
 71%|███████▏  | 160776/225000 [312:02:26<101:08:48,  5.67s/it]
 71%|███████▏  | 160777/225000 [312:02:29<89:41:17,  5.03s/it] 
 71%|███████▏  | 160778/225000 [312:02:34<88:29:32,  4.96s/it]
 71%|███████▏  | 160779/225000 [312:02:38<84:30:28,  4.74s/it]
 71%|███████▏  | 160780/225000 [312:02:43<83:57:59,  4.71s/it]
 71%|███████▏  | 160781/225000 [312:02:52<106:48:10,  5.99s/it]
 71%|███████▏  | 160782/225000 [312:03:01<122:08:04,  6.85s/it]
 71%|███████▏  | 160783/225000 [312:03:04<103:19:28,  5.79s/it]
 71%|███████▏  | 160784/225000 [312:03:08<95:38:44,  5.36s/it] 
 71%|███████▏  | 160785/225000 [312:03:13<90:55:43,  5.10s/it]
 71%|███████▏  | 160786/225000 [312:03:17<88:48:23,  4.98s/it]
 71%|███████▏  | 160787/225000 [312:03:27<114:16:18,  6.41s/it]
 71%|███████▏  | 160788/225000 [312:03:34<113:50:57,  6.38s/it]
 71%|███████▏  | 160789/225000 [312:03:38<101:52:32,  5.71s/it]
 71%|███████▏  | 160790/225000 [312:03:41<91:13:49,  5.11s/it] 
 71%|███████▏  | 160791/225000 [312:03:47<96:07:11,  5.39s/it]
 71%|███████▏  | 160792/225000 [312:03:59<127:45:37,  7.16s/it]
 71%|███████▏  | 160793/225000 [312:04:02<106:31:53,  5.97s/it]
 71%|███████▏  | 160794/225000 [312:04:09<111:47:58,  6.27s/it]
 71%|███████▏  | 160795/225000 [312:04:13<101:16:45,  5.68s/it]
 71%|███████▏  | 160796/225000 [312:04:17<91:12:50,  5.11s/it] 
 71%|███████▏  | 160797/225000 [312:04:21<86:35:02,  4.85s/it]
 71%|███████▏  | 160798/225000 [312:04:31<110:58:01,  6.22s/it]
 71%|███████▏  | 160799/225000 [312:04:39<120:59:31,  6.78s/it]
 71%|███████▏  | 160800/225000 [312:04:42<104:12:52,  5.84s/it]
                                                               
{'loss': '0.7856', 'grad_norm': '1.109', 'learning_rate': '1.024e-05', 'epoch': '0.7147', 'num_input_tokens_seen': 516592392, 'train_runtime': '1.123e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160800/225000 [312:04:42<104:12:52,  5.84s/it]
 71%|███████▏  | 160801/225000 [312:04:48<102:17:31,  5.74s/it]
 71%|███████▏  | 160802/225000 [312:04:56<117:28:53,  6.59s/it]
 71%|███████▏  | 160803/225000 [312:05:00<100:14:40,  5.62s/it]
 71%|███████▏  | 160804/225000 [312:05:07<111:06:22,  6.23s/it]
 71%|███████▏  | 160805/225000 [312:05:11<97:17:50,  5.46s/it] 
 71%|███████▏  | 160806/225000 [312:05:17<98:34:55,  5.53s/it]
 71%|███████▏  | 160807/225000 [312:05:23<104:19:17,  5.85s/it]
 71%|███████▏  | 160808/225000 [312:05:29<102:26:25,  5.75s/it]
 71%|███████▏  | 160809/225000 [312:05:35<103:41:10,  5.81s/it]
 71%|███████▏  | 160810/225000 [312:05:39<93:52:48,  5.27s/it] 
 71%|███████▏  | 160811/225000 [312:05:48<113:15:32,  6.35s/it]
 71%|███████▏  | 160812/225000 [312:05:51<96:30:30,  5.41s/it] 
 71%|███████▏  | 160813/225000 [312:05:57<101:48:27,  5.71s/it]
 71%|███████▏  | 160814/225000 [312:06:02<98:19:39,  5.51s/it] 
 71%|███████▏  | 160815/225000 [312:06:07<93:55:23,  5.27s/it]
 71%|███████▏  | 160816/225000 [312:06:12<93:57:39,  5.27s/it]
 71%|███████▏  | 160817/225000 [312:06:18<93:36:00,  5.25s/it]
 71%|███████▏  | 160818/225000 [312:06:23<93:36:24,  5.25s/it]
 71%|███████▏  | 160819/225000 [312:06:28<93:03:23,  5.22s/it]
 71%|███████▏  | 160820/225000 [312:06:33<93:30:09,  5.24s/it]
 71%|███████▏  | 160821/225000 [312:06:38<92:09:41,  5.17s/it]
 71%|███████▏  | 160822/225000 [312:06:45<102:36:27,  5.76s/it]
 71%|███████▏  | 160823/225000 [312:06:55<122:56:46,  6.90s/it]
 71%|███████▏  | 160824/225000 [312:07:01<116:18:29,  6.52s/it]
 71%|███████▏  | 160825/225000 [312:07:05<105:48:31,  5.94s/it]
 71%|███████▏  | 160826/225000 [312:07:10<99:33:48,  5.59s/it] 
 71%|███████▏  | 160827/225000 [312:07:15<95:22:32,  5.35s/it]
 71%|███████▏  | 160828/225000 [312:07:20<92:30:06,  5.19s/it]
 71%|███████▏  | 160829/225000 [312:07:24<90:47:20,  5.09s/it]
 71%|███████▏  | 160830/225000 [312:07:37<128:41:27,  7.22s/it]
 71%|███████▏  | 160831/225000 [312:07:41<115:22:15,  6.47s/it]
 71%|███████▏  | 160832/225000 [312:07:46<107:00:56,  6.00s/it]
 71%|███████▏  | 160833/225000 [312:07:51<101:46:13,  5.71s/it]
 71%|███████▏  | 160834/225000 [312:07:56<96:47:03,  5.43s/it] 
 71%|███████▏  | 160835/225000 [312:08:01<93:21:20,  5.24s/it]
 71%|███████▏  | 160836/225000 [312:08:06<90:31:25,  5.08s/it]
 71%|███████▏  | 160837/225000 [312:08:11<89:42:24,  5.03s/it]
 71%|███████▏  | 160838/225000 [312:08:15<87:37:24,  4.92s/it]
 71%|███████▏  | 160839/225000 [312:08:20<87:11:37,  4.89s/it]
 71%|███████▏  | 160840/225000 [312:08:25<89:05:52,  5.00s/it]
 71%|███████▏  | 160841/225000 [312:08:36<118:48:27,  6.67s/it]
 71%|███████▏  | 160842/225000 [312:08:44<129:20:17,  7.26s/it]
 71%|███████▏  | 160843/225000 [312:08:49<112:47:20,  6.33s/it]
 71%|███████▏  | 160844/225000 [312:08:53<101:28:33,  5.69s/it]
 71%|███████▏  | 160845/225000 [312:08:57<93:24:45,  5.24s/it] 
 71%|███████▏  | 160846/225000 [312:09:01<88:37:30,  4.97s/it]
 71%|███████▏  | 160847/225000 [312:09:06<85:20:48,  4.79s/it]
 71%|███████▏  | 160848/225000 [312:09:14<102:21:28,  5.74s/it]
 71%|███████▏  | 160849/225000 [312:09:24<124:36:26,  6.99s/it]
 71%|███████▏  | 160850/225000 [312:09:27<104:08:48,  5.84s/it]
                                                               
{'loss': '0.7788', 'grad_norm': '1.112', 'learning_rate': '1.022e-05', 'epoch': '0.7149', 'num_input_tokens_seen': 516722712, 'train_runtime': '1.124e+06', 'train_tokens_per_second': '459.8'}

 71%|███████▏  | 160850/225000 [312:09:27<104:08:48,  5.84s/it]
 71%|███████▏  | 160851/225000 [312:09:31<93:19:13,  5.24s/it] 
 71%|███████▏  | 160852/225000 [312:09:37<99:20:54,  5.58s/it]
 71%|███████▏  | 160853/225000 [312:09:45<112:53:29,  6.34s/it]
 71%|███████▏  | 160854/225000 [312:09:49<98:34:15,  5.53s/it] 
 71%|███████▏  | 160855/225000 [312:09:57<114:57:38,  6.45s/it]
 71%|███████▏  | 160856/225000 [312:10:01<102:08:36,  5.73s/it]
 71%|███████▏  | 160857/225000 [312:10:07<99:56:44,  5.61s/it] 
 71%|███████▏  | 160858/225000 [312:10:12<97:32:16,  5.47s/it]
 71%|███████▏  | 160859/225000 [312:10:17<96:16:17,  5.40s/it]
 71%|███████▏  | 160860/225000 [312:10:22<94:49:39,  5.32s/it]
 71%|███████▏  | 160861/225000 [312:10:27<93:43:44,  5.26s/it]
 71%|███████▏  | 160862/225000 [312:10:33<93:40:25,  5.26s/it]
 71%|███████▏  | 160863/225000 [312:10:38<94:29:39,  5.30s/it]
 71%|███████▏  | 160864/225000 [312:10:43<94:17:35,  5.29s/it]
 71%|███████▏  | 160865/225000 [312:10:48<91:47:40,  5.15s/it]
 71%|███████▏  | 160866/225000 [312:10:54<93:27:56,  5.25s/it]
 71%|███████▏  | 160867/225000 [312:10:59<93:28:02,  5.25s/it]
 71%|███████▏  | 160868/225000 [312:11:05<96:34:49,  5.42s/it]
 71%|███████▏  | 160869/225000 [312:11:10<96:31:46,  5.42s/it]
 71%|███████▏  | 160870/225000 [312:11:22<132:39:11,  7.45s/it]
 71%|███████▏  | 160871/225000 [312:11:31<141:51:29,  7.96s/it]
 71%|███████▏  | 160872/225000 [312:11:35<118:06:18,  6.63s/it]
 71%|███████▏  | 160873/225000 [312:11:39<101:55:27,  5.72s/it]
 71%|███████▏  | 160874/225000 [312:11:49<125:34:55,  7.05s/it]
 72%|███████▏  | 160875/225000 [312:11:53<110:15:28,  6.19s/it]
 72%|███████▏  | 160876/225000 [312:11:57<97:49:54,  5.49s/it] 
 72%|███████▏  | 160877/225000 [312:12:05<111:57:24,  6.29s/it]
 72%|███████▏  | 160878/225000 [312:12:10<106:42:44,  5.99s/it]
 72%|███████▏  | 160879/225000 [312:12:14<93:59:20,  5.28s/it] 
 72%|███████▏  | 160880/225000 [312:12:18<86:26:25,  4.85s/it]
 72%|███████▏  | 160881/225000 [312:12:26<103:57:41,  5.84s/it]
 72%|███████▏  | 160882/225000 [312:12:32<105:14:46,  5.91s/it]
 72%|███████▏  | 160883/225000 [312:12:37<99:21:20,  5.58s/it] 
 72%|███████▏  | 160884/225000 [312:12:41<93:09:39,  5.23s/it]
 72%|███████▏  | 160885/225000 [312:12:46<89:05:09,  5.00s/it]
 72%|███████▏  | 160886/225000 [312:12:50<88:43:31,  4.98s/it]
 72%|███████▏  | 160887/225000 [312:12:55<86:22:10,  4.85s/it]
 72%|███████▏  | 160888/225000 [312:13:02<99:24:15,  5.58s/it]
 72%|███████▏  | 160889/225000 [312:13:12<123:16:38,  6.92s/it]
 72%|███████▏  | 160890/225000 [312:13:17<111:26:29,  6.26s/it]
 72%|███████▏  | 160891/225000 [312:13:21<100:19:35,  5.63s/it]
 72%|███████▏  | 160892/225000 [312:13:26<94:14:39,  5.29s/it] 
 72%|███████▏  | 160893/225000 [312:13:31<92:01:31,  5.17s/it]
 72%|███████▏  | 160894/225000 [312:13:35<89:20:10,  5.02s/it]
 72%|███████▏  | 160895/225000 [312:13:40<88:03:02,  4.94s/it]
 72%|███████▏  | 160896/225000 [312:13:50<114:07:30,  6.41s/it]
 72%|███████▏  | 160897/225000 [312:13:57<116:20:29,  6.53s/it]
 72%|███████▏  | 160898/225000 [312:14:02<108:01:38,  6.07s/it]
 72%|███████▏  | 160899/225000 [312:14:07<102:47:39,  5.77s/it]
 72%|███████▏  | 160900/225000 [312:14:12<99:21:25,  5.58s/it] 
                                                              
{'loss': '0.7762', 'grad_norm': '1.081', 'learning_rate': '1.021e-05', 'epoch': '0.7151', 'num_input_tokens_seen': 516853088, 'train_runtime': '1.124e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 160900/225000 [312:14:12<99:21:25,  5.58s/it]
 72%|███████▏  | 160901/225000 [312:14:17<95:11:49,  5.35s/it]
 72%|███████▏  | 160902/225000 [312:14:22<95:49:11,  5.38s/it]
 72%|███████▏  | 160903/225000 [312:14:27<92:53:18,  5.22s/it]
 72%|███████▏  | 160904/225000 [312:14:32<90:29:59,  5.08s/it]
 72%|███████▏  | 160905/225000 [312:14:37<89:52:24,  5.05s/it]
 72%|███████▏  | 160906/225000 [312:14:42<89:34:35,  5.03s/it]
 72%|███████▏  | 160907/225000 [312:14:54<128:15:05,  7.20s/it]
 72%|███████▏  | 160908/225000 [312:15:01<125:24:59,  7.04s/it]
 72%|███████▏  | 160909/225000 [312:15:04<106:20:30,  5.97s/it]
 72%|███████▏  | 160910/225000 [312:15:11<109:19:49,  6.14s/it]
 72%|███████▏  | 160911/225000 [312:15:18<117:47:27,  6.62s/it]
 72%|███████▏  | 160912/225000 [312:15:22<103:44:38,  5.83s/it]
 72%|███████▏  | 160913/225000 [312:15:26<93:56:17,  5.28s/it] 
 72%|███████▏  | 160914/225000 [312:15:38<125:56:08,  7.07s/it]
 72%|███████▏  | 160915/225000 [312:15:41<106:30:06,  5.98s/it]
 72%|███████▏  | 160916/225000 [312:15:46<98:46:55,  5.55s/it] 
 72%|███████▏  | 160917/225000 [312:15:51<97:54:35,  5.50s/it]
 72%|███████▏  | 160918/225000 [312:15:56<96:59:49,  5.45s/it]
 72%|███████▏  | 160919/225000 [312:16:01<95:04:54,  5.34s/it]
 72%|███████▏  | 160920/225000 [312:16:06<93:07:03,  5.23s/it]
 72%|███████▏  | 160921/225000 [312:16:12<93:12:42,  5.24s/it]
 72%|███████▏  | 160922/225000 [312:16:17<92:46:07,  5.21s/it]
 72%|███████▏  | 160923/225000 [312:16:22<91:23:45,  5.13s/it]
 72%|███████▏  | 160924/225000 [312:16:27<90:38:09,  5.09s/it]
 72%|███████▏  | 160925/225000 [312:16:32<90:34:27,  5.09s/it]
 72%|███████▏  | 160926/225000 [312:16:40<108:11:37,  6.08s/it]
 72%|███████▏  | 160927/225000 [312:16:51<132:34:56,  7.45s/it]
 72%|███████▏  | 160928/225000 [312:16:54<110:06:23,  6.19s/it]
 72%|███████▏  | 160929/225000 [312:16:58<100:25:44,  5.64s/it]
 72%|███████▏  | 160930/225000 [312:17:03<93:32:44,  5.26s/it] 
 72%|███████▏  | 160931/225000 [312:17:07<88:57:41,  5.00s/it]
 72%|███████▏  | 160932/225000 [312:17:17<116:04:29,  6.52s/it]
 72%|███████▏  | 160933/225000 [312:17:23<113:50:05,  6.40s/it]
 72%|███████▏  | 160934/225000 [312:17:27<98:57:36,  5.56s/it] 
 72%|███████▏  | 160935/225000 [312:17:36<115:00:31,  6.46s/it]
 72%|███████▏  | 160936/225000 [312:17:41<107:40:24,  6.05s/it]
 72%|███████▏  | 160937/225000 [312:17:45<98:37:10,  5.54s/it] 
 72%|███████▏  | 160938/225000 [312:17:51<99:14:54,  5.58s/it]
 72%|███████▏  | 160939/225000 [312:17:56<99:09:15,  5.57s/it]
 72%|███████▏  | 160940/225000 [312:18:01<97:20:50,  5.47s/it]
 72%|███████▏  | 160941/225000 [312:18:09<110:34:52,  6.21s/it]
 72%|███████▏  | 160942/225000 [312:18:17<116:46:18,  6.56s/it]
 72%|███████▏  | 160943/225000 [312:18:21<106:37:37,  5.99s/it]
 72%|███████▏  | 160944/225000 [312:18:26<99:17:54,  5.58s/it] 
 72%|███████▏  | 160945/225000 [312:18:31<93:41:29,  5.27s/it]
 72%|███████▏  | 160946/225000 [312:18:35<86:59:00,  4.89s/it]
 72%|███████▏  | 160947/225000 [312:18:39<83:54:05,  4.72s/it]
 72%|███████▏  | 160948/225000 [312:18:44<87:17:29,  4.91s/it]
 72%|███████▏  | 160949/225000 [312:18:52<100:57:09,  5.67s/it]
 72%|███████▏  | 160950/225000 [312:19:03<129:38:54,  7.29s/it]
                                                               
{'loss': '0.7753', 'grad_norm': '1.028', 'learning_rate': '1.019e-05', 'epoch': '0.7153', 'num_input_tokens_seen': 516983544, 'train_runtime': '1.124e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 160950/225000 [312:19:03<129:38:54,  7.29s/it]
 72%|███████▏  | 160951/225000 [312:19:07<110:45:06,  6.23s/it]
 72%|███████▏  | 160952/225000 [312:19:14<115:03:49,  6.47s/it]
 72%|███████▏  | 160953/225000 [312:19:19<109:02:09,  6.13s/it]
 72%|███████▏  | 160954/225000 [312:19:25<107:50:34,  6.06s/it]
 72%|███████▏  | 160955/225000 [312:19:32<111:37:07,  6.27s/it]
 72%|███████▏  | 160956/225000 [312:19:37<109:16:45,  6.14s/it]
 72%|███████▏  | 160957/225000 [312:19:44<111:27:21,  6.27s/it]
 72%|███████▏  | 160958/225000 [312:19:48<101:09:24,  5.69s/it]
 72%|███████▏  | 160959/225000 [312:19:53<95:32:41,  5.37s/it] 
 72%|███████▏  | 160960/225000 [312:19:57<91:00:19,  5.12s/it]
 72%|███████▏  | 160961/225000 [312:20:02<85:49:48,  4.83s/it]
 72%|███████▏  | 160962/225000 [312:20:06<83:55:15,  4.72s/it]
 72%|███████▏  | 160963/225000 [312:20:10<81:05:32,  4.56s/it]
 72%|███████▏  | 160964/225000 [312:20:21<111:21:41,  6.26s/it]
 72%|███████▏  | 160965/225000 [312:20:29<120:33:26,  6.78s/it]
 72%|███████▏  | 160966/225000 [312:20:33<107:28:45,  6.04s/it]
 72%|███████▏  | 160967/225000 [312:20:37<97:01:37,  5.45s/it] 
 72%|███████▏  | 160968/225000 [312:20:41<88:57:55,  5.00s/it]
 72%|███████▏  | 160969/225000 [312:20:47<93:42:18,  5.27s/it]
 72%|███████▏  | 160970/225000 [312:20:57<117:59:52,  6.63s/it]
 72%|███████▏  | 160971/225000 [312:21:00<101:52:53,  5.73s/it]
 72%|███████▏  | 160972/225000 [312:21:09<116:36:56,  6.56s/it]
 72%|███████▏  | 160973/225000 [312:21:12<99:41:36,  5.61s/it] 
 72%|███████▏  | 160974/225000 [312:21:19<106:22:03,  5.98s/it]
 72%|███████▏  | 160975/225000 [312:21:24<102:31:43,  5.76s/it]
 72%|███████▏  | 160976/225000 [312:21:30<101:04:09,  5.68s/it]
 72%|███████▏  | 160977/225000 [312:21:36<106:28:21,  5.99s/it]
 72%|███████▏  | 160978/225000 [312:21:40<96:26:04,  5.42s/it] 
 72%|███████▏  | 160979/225000 [312:21:45<91:34:13,  5.15s/it]
 72%|███████▏  | 160980/225000 [312:21:49<87:34:55,  4.92s/it]
 72%|███████▏  | 160981/225000 [312:21:54<85:11:12,  4.79s/it]
 72%|███████▏  | 160982/225000 [312:22:04<111:38:25,  6.28s/it]
 72%|███████▏  | 160983/225000 [312:22:10<112:56:20,  6.35s/it]
 72%|███████▏  | 160984/225000 [312:22:14<98:35:05,  5.54s/it] 
 72%|███████▏  | 160985/225000 [312:22:22<114:46:13,  6.45s/it]
 72%|███████▏  | 160986/225000 [312:22:26<101:29:51,  5.71s/it]
 72%|███████▏  | 160987/225000 [312:22:30<90:17:36,  5.08s/it] 
 72%|███████▏  | 160988/225000 [312:22:39<112:44:21,  6.34s/it]
 72%|███████▏  | 160989/225000 [312:22:43<101:39:40,  5.72s/it]
 72%|███████▏  | 160990/225000 [312:22:48<93:40:50,  5.27s/it] 
 72%|███████▏  | 160991/225000 [312:22:52<89:26:27,  5.03s/it]
 72%|███████▏  | 160992/225000 [312:23:04<124:23:55,  7.00s/it]
 72%|███████▏  | 160993/225000 [312:23:07<105:05:17,  5.91s/it]
 72%|███████▏  | 160994/225000 [312:23:11<94:16:00,  5.30s/it] 
 72%|███████▏  | 160995/225000 [312:23:15<84:45:24,  4.77s/it]
 72%|███████▏  | 160996/225000 [312:23:24<109:55:13,  6.18s/it]
 72%|███████▏  | 160997/225000 [312:23:29<104:16:23,  5.87s/it]
 72%|███████▏  | 160998/225000 [312:23:36<108:02:30,  6.08s/it]
 72%|███████▏  | 160999/225000 [312:23:39<92:33:33,  5.21s/it] 
 72%|███████▏  | 161000/225000 [312:23:46<102:21:43,  5.76s/it]
                                                               
{'loss': '0.7751', 'grad_norm': '1.183', 'learning_rate': '1.018e-05', 'epoch': '0.7156', 'num_input_tokens_seen': 517112920, 'train_runtime': '1.125e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161000/225000 [312:23:46<102:21:43,  5.76s/it]
 72%|███████▏  | 161001/225000 [312:23:52<105:38:32,  5.94s/it]
 72%|███████▏  | 161002/225000 [312:23:56<94:32:41,  5.32s/it] 
 72%|███████▏  | 161003/225000 [312:24:00<86:15:34,  4.85s/it]
 72%|███████▏  | 161004/225000 [312:24:04<83:24:58,  4.69s/it]
 72%|███████▏  | 161005/225000 [312:24:15<117:37:38,  6.62s/it]
 72%|███████▏  | 161006/225000 [312:24:20<107:32:39,  6.05s/it]
 72%|███████▏  | 161007/225000 [312:24:26<106:18:15,  5.98s/it]
 72%|███████▏  | 161008/225000 [312:24:32<108:39:01,  6.11s/it]
 72%|███████▏  | 161009/225000 [312:24:36<94:53:44,  5.34s/it] 
 72%|███████▏  | 161010/225000 [312:24:45<113:47:27,  6.40s/it]
 72%|███████▏  | 161011/225000 [312:24:49<104:03:35,  5.85s/it]
 72%|███████▏  | 161012/225000 [312:24:58<116:58:17,  6.58s/it]
 72%|███████▏  | 161013/225000 [312:25:02<103:59:21,  5.85s/it]
 72%|███████▏  | 161014/225000 [312:25:08<104:06:10,  5.86s/it]
 72%|███████▏  | 161015/225000 [312:25:13<100:11:01,  5.64s/it]
 72%|███████▏  | 161016/225000 [312:25:17<91:30:05,  5.15s/it] 
 72%|███████▏  | 161017/225000 [312:25:22<90:04:51,  5.07s/it]
 72%|███████▏  | 161018/225000 [312:25:27<88:55:05,  5.00s/it]
 72%|███████▏  | 161019/225000 [312:25:31<87:21:45,  4.92s/it]
 72%|███████▏  | 161020/225000 [312:25:36<86:27:40,  4.86s/it]
 72%|███████▏  | 161021/225000 [312:25:41<88:14:09,  4.96s/it]
 72%|███████▏  | 161022/225000 [312:25:46<89:45:30,  5.05s/it]
 72%|███████▏  | 161023/225000 [312:25:51<89:32:23,  5.04s/it]
 72%|███████▏  | 161024/225000 [312:26:02<119:47:24,  6.74s/it]
 72%|███████▏  | 161025/225000 [312:26:10<123:48:38,  6.97s/it]
 72%|███████▏  | 161026/225000 [312:26:16<123:06:16,  6.93s/it]
 72%|███████▏  | 161027/225000 [312:26:22<113:03:40,  6.36s/it]
 72%|███████▏  | 161028/225000 [312:26:27<107:54:05,  6.07s/it]
 72%|███████▏  | 161029/225000 [312:26:31<95:01:16,  5.35s/it] 
 72%|███████▏  | 161030/225000 [312:26:40<117:36:22,  6.62s/it]
 72%|███████▏  | 161031/225000 [312:26:44<101:54:23,  5.74s/it]
 72%|███████▏  | 161032/225000 [312:26:51<107:26:01,  6.05s/it]
 72%|███████▏  | 161033/225000 [312:26:56<102:06:05,  5.75s/it]
 72%|███████▏  | 161034/225000 [312:27:03<108:43:14,  6.12s/it]
 72%|███████▏  | 161035/225000 [312:27:08<104:43:27,  5.89s/it]
 72%|███████▏  | 161036/225000 [312:27:12<93:24:11,  5.26s/it] 
 72%|███████▏  | 161037/225000 [312:27:16<86:57:26,  4.89s/it]
 72%|███████▏  | 161038/225000 [312:27:24<102:41:22,  5.78s/it]
 72%|███████▏  | 161039/225000 [312:27:30<105:42:50,  5.95s/it]
 72%|███████▏  | 161040/225000 [312:27:34<94:10:21,  5.30s/it] 
 72%|███████▏  | 161041/225000 [312:27:39<94:50:50,  5.34s/it]
 72%|███████▏  | 161042/225000 [312:27:45<94:54:16,  5.34s/it]
 72%|███████▏  | 161043/225000 [312:27:49<90:33:24,  5.10s/it]
 72%|███████▏  | 161044/225000 [312:27:54<91:22:53,  5.14s/it]
 72%|███████▏  | 161045/225000 [312:28:00<95:06:39,  5.35s/it]
 72%|███████▏  | 161046/225000 [312:28:05<94:01:28,  5.29s/it]
 72%|███████▏  | 161047/225000 [312:28:11<95:26:47,  5.37s/it]
 72%|███████▏  | 161048/225000 [312:28:16<95:06:48,  5.35s/it]
 72%|███████▏  | 161049/225000 [312:28:21<94:43:17,  5.33s/it]
 72%|███████▏  | 161050/225000 [312:28:27<95:19:49,  5.37s/it]
                                                              
{'loss': '0.7815', 'grad_norm': '1.072', 'learning_rate': '1.017e-05', 'epoch': '0.7158', 'num_input_tokens_seen': 517243736, 'train_runtime': '1.125e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161050/225000 [312:28:27<95:19:49,  5.37s/it]
 72%|███████▏  | 161051/225000 [312:28:33<97:24:05,  5.48s/it]
 72%|███████▏  | 161052/225000 [312:28:38<97:05:32,  5.47s/it]
 72%|███████▏  | 161053/225000 [312:28:48<119:15:45,  6.71s/it]
 72%|███████▏  | 161054/225000 [312:28:56<127:59:57,  7.21s/it]
 72%|███████▏  | 161055/225000 [312:29:00<107:46:23,  6.07s/it]
 72%|███████▏  | 161056/225000 [312:29:04<98:11:49,  5.53s/it] 
 72%|███████▏  | 161057/225000 [312:29:14<124:47:18,  7.03s/it]
 72%|███████▏  | 161058/225000 [312:29:21<122:23:13,  6.89s/it]
 72%|███████▏  | 161059/225000 [312:29:24<104:11:55,  5.87s/it]
 72%|███████▏  | 161060/225000 [312:29:30<101:58:17,  5.74s/it]
 72%|███████▏  | 161061/225000 [312:29:37<108:54:23,  6.13s/it]
 72%|███████▏  | 161062/225000 [312:29:44<113:33:07,  6.39s/it]
 72%|███████▏  | 161063/225000 [312:29:49<106:06:54,  5.97s/it]
 72%|███████▏  | 161064/225000 [312:29:54<103:27:01,  5.82s/it]
 72%|███████▏  | 161065/225000 [312:30:00<101:13:03,  5.70s/it]
 72%|███████▏  | 161066/225000 [312:30:05<99:08:38,  5.58s/it] 
 72%|███████▏  | 161067/225000 [312:30:10<98:04:38,  5.52s/it]
 72%|███████▏  | 161068/225000 [312:30:16<98:47:20,  5.56s/it]
 72%|███████▏  | 161069/225000 [312:30:21<96:30:46,  5.43s/it]
 72%|███████▏  | 161070/225000 [312:30:31<119:31:55,  6.73s/it]
 72%|███████▏  | 161071/225000 [312:30:34<102:23:10,  5.77s/it]
 72%|███████▏  | 161072/225000 [312:30:38<92:00:59,  5.18s/it] 
 72%|███████▏  | 161073/225000 [312:30:45<98:29:59,  5.55s/it]
 72%|███████▏  | 161074/225000 [312:30:53<111:28:53,  6.28s/it]
 72%|███████▏  | 161075/225000 [312:30:57<99:11:45,  5.59s/it] 
 72%|███████▏  | 161076/225000 [312:31:01<93:57:23,  5.29s/it]
 72%|███████▏  | 161077/225000 [312:31:06<90:35:19,  5.10s/it]
 72%|███████▏  | 161078/225000 [312:31:11<88:29:45,  4.98s/it]
 72%|███████▏  | 161079/225000 [312:31:18<101:29:28,  5.72s/it]
 72%|███████▏  | 161080/225000 [312:31:28<123:59:36,  6.98s/it]
 72%|███████▏  | 161081/225000 [312:31:31<105:28:52,  5.94s/it]
 72%|███████▏  | 161082/225000 [312:31:36<97:20:59,  5.48s/it] 
 72%|███████▏  | 161083/225000 [312:31:41<93:17:44,  5.25s/it]
 72%|███████▏  | 161084/225000 [312:31:45<89:25:34,  5.04s/it]
 72%|███████▏  | 161085/225000 [312:31:50<86:35:44,  4.88s/it]
 72%|███████▏  | 161086/225000 [312:31:54<83:09:54,  4.68s/it]
 72%|███████▏  | 161087/225000 [312:32:03<108:43:52,  6.12s/it]
 72%|███████▏  | 161088/225000 [312:32:11<114:49:18,  6.47s/it]
 72%|███████▏  | 161089/225000 [312:32:15<101:33:48,  5.72s/it]
 72%|███████▏  | 161090/225000 [312:32:18<91:23:00,  5.15s/it] 
 72%|███████▏  | 161091/225000 [312:32:28<117:01:33,  6.59s/it]
 72%|███████▏  | 161092/225000 [312:32:32<99:46:29,  5.62s/it] 
 72%|███████▏  | 161093/225000 [312:32:38<103:47:21,  5.85s/it]
 72%|███████▏  | 161094/225000 [312:32:46<113:48:41,  6.41s/it]
 72%|███████▏  | 161095/225000 [312:32:49<98:22:47,  5.54s/it] 
 72%|███████▏  | 161096/225000 [312:32:57<111:32:41,  6.28s/it]
 72%|███████▏  | 161097/225000 [312:33:01<95:22:42,  5.37s/it] 
 72%|███████▏  | 161098/225000 [312:33:09<110:43:59,  6.24s/it]
 72%|███████▏  | 161099/225000 [312:33:12<94:26:57,  5.32s/it] 
 72%|███████▏  | 161100/225000 [312:33:19<104:41:06,  5.90s/it]
                                                               
{'loss': '0.7697', 'grad_norm': '1.164', 'learning_rate': '1.015e-05', 'epoch': '0.716', 'num_input_tokens_seen': 517373528, 'train_runtime': '1.125e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161100/225000 [312:33:19<104:41:06,  5.90s/it]
 72%|███████▏  | 161101/225000 [312:33:23<90:36:08,  5.10s/it] 
 72%|███████▏  | 161102/225000 [312:33:31<107:16:18,  6.04s/it]
 72%|███████▏  | 161103/225000 [312:33:35<98:07:18,  5.53s/it] 
 72%|███████▏  | 161104/225000 [312:33:39<91:18:54,  5.14s/it]
 72%|███████▏  | 161105/225000 [312:33:44<88:29:06,  4.99s/it]
 72%|███████▏  | 161106/225000 [312:33:49<86:29:01,  4.87s/it]
 72%|███████▏  | 161107/225000 [312:33:58<108:43:54,  6.13s/it]
 72%|███████▏  | 161108/225000 [312:34:05<115:14:39,  6.49s/it]
 72%|███████▏  | 161109/225000 [312:34:08<98:46:09,  5.57s/it] 
 72%|███████▏  | 161110/225000 [312:34:17<114:36:19,  6.46s/it]
 72%|███████▏  | 161111/225000 [312:34:21<101:08:32,  5.70s/it]
 72%|███████▏  | 161112/225000 [312:34:24<89:53:17,  5.07s/it] 
 72%|███████▏  | 161113/225000 [312:34:33<109:04:34,  6.15s/it]
 72%|███████▏  | 161114/225000 [312:34:38<101:37:01,  5.73s/it]
 72%|███████▏  | 161115/225000 [312:34:43<99:22:01,  5.60s/it] 
 72%|███████▏  | 161116/225000 [312:34:49<98:48:06,  5.57s/it]
 72%|███████▏  | 161117/225000 [312:34:53<93:04:03,  5.24s/it]
 72%|███████▏  | 161118/225000 [312:34:59<93:35:08,  5.27s/it]
 72%|███████▏  | 161119/225000 [312:35:04<96:01:31,  5.41s/it]
 72%|███████▏  | 161120/225000 [312:35:13<114:04:49,  6.43s/it]
 72%|███████▏  | 161121/225000 [312:35:17<99:01:49,  5.58s/it] 
 72%|███████▏  | 161122/225000 [312:35:21<89:54:16,  5.07s/it]
 72%|███████▏  | 161123/225000 [312:35:24<82:46:03,  4.66s/it]
 72%|███████▏  | 161124/225000 [312:35:36<122:54:22,  6.93s/it]
 72%|███████▏  | 161125/225000 [312:35:41<107:59:43,  6.09s/it]
 72%|███████▏  | 161126/225000 [312:35:46<104:06:38,  5.87s/it]
 72%|███████▏  | 161127/225000 [312:35:51<97:45:11,  5.51s/it] 
 72%|███████▏  | 161128/225000 [312:35:56<96:00:27,  5.41s/it]
 72%|███████▏  | 161129/225000 [312:36:01<96:08:56,  5.42s/it]
 72%|███████▏  | 161130/225000 [312:36:06<95:06:06,  5.36s/it]
 72%|███████▏  | 161131/225000 [312:36:11<92:45:50,  5.23s/it]
 72%|███████▏  | 161132/225000 [312:36:17<93:24:48,  5.27s/it]
 72%|███████▏  | 161133/225000 [312:36:22<91:35:27,  5.16s/it]
 72%|███████▏  | 161134/225000 [312:36:27<93:16:27,  5.26s/it]
 72%|███████▏  | 161135/225000 [312:36:32<92:25:40,  5.21s/it]
 72%|███████▏  | 161136/225000 [312:36:38<93:26:05,  5.27s/it]
 72%|███████▏  | 161137/225000 [312:36:43<94:15:51,  5.31s/it]
 72%|███████▏  | 161138/225000 [312:36:48<91:56:22,  5.18s/it]
 72%|███████▏  | 161139/225000 [312:36:54<94:17:09,  5.32s/it]
 72%|███████▏  | 161140/225000 [312:37:03<118:52:11,  6.70s/it]
 72%|███████▏  | 161141/225000 [312:37:14<138:54:53,  7.83s/it]
 72%|███████▏  | 161142/225000 [312:37:18<116:43:02,  6.58s/it]
 72%|███████▏  | 161143/225000 [312:37:21<98:20:34,  5.54s/it] 
 72%|███████▏  | 161144/225000 [312:37:30<117:55:10,  6.65s/it]
 72%|███████▏  | 161145/225000 [312:37:33<100:29:26,  5.67s/it]
 72%|███████▏  | 161146/225000 [312:37:42<116:53:35,  6.59s/it]
 72%|███████▏  | 161147/225000 [312:37:45<99:46:34,  5.63s/it] 
 72%|███████▏  | 161148/225000 [312:37:52<102:52:24,  5.80s/it]
 72%|███████▏  | 161149/225000 [312:37:55<90:57:03,  5.13s/it] 
 72%|███████▏  | 161150/225000 [312:38:00<86:28:43,  4.88s/it]
                                                              
{'loss': '0.7833', 'grad_norm': '1.104', 'learning_rate': '1.014e-05', 'epoch': '0.7162', 'num_input_tokens_seen': 517503504, 'train_runtime': '1.125e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161150/225000 [312:38:00<86:28:43,  4.88s/it]
 72%|███████▏  | 161151/225000 [312:38:10<114:18:41,  6.45s/it]
 72%|███████▏  | 161152/225000 [312:38:13<97:43:53,  5.51s/it] 
 72%|███████▏  | 161153/225000 [312:38:21<112:42:38,  6.36s/it]
 72%|███████▏  | 161154/225000 [312:38:25<97:18:09,  5.49s/it] 
 72%|███████▏  | 161155/225000 [312:38:31<103:25:09,  5.83s/it]
 72%|███████▏  | 161156/225000 [312:38:38<106:16:06,  5.99s/it]
 72%|███████▏  | 161157/225000 [312:38:42<97:13:31,  5.48s/it] 
 72%|███████▏  | 161158/225000 [312:38:46<91:05:32,  5.14s/it]
 72%|███████▏  | 161159/225000 [312:38:51<87:41:51,  4.95s/it]
 72%|███████▏  | 161160/225000 [312:39:02<121:38:16,  6.86s/it]
 72%|███████▏  | 161161/225000 [312:39:06<107:20:11,  6.05s/it]
 72%|███████▏  | 161162/225000 [312:39:11<99:19:54,  5.60s/it] 
 72%|███████▏  | 161163/225000 [312:39:15<91:01:22,  5.13s/it]
 72%|███████▏  | 161164/225000 [312:39:19<87:51:00,  4.95s/it]
 72%|███████▏  | 161165/225000 [312:39:24<85:25:52,  4.82s/it]
 72%|███████▏  | 161166/225000 [312:39:34<113:21:19,  6.39s/it]
 72%|███████▏  | 161167/225000 [312:39:41<118:31:39,  6.68s/it]
 72%|███████▏  | 161168/225000 [312:39:45<102:39:39,  5.79s/it]
 72%|███████▏  | 161169/225000 [312:39:53<116:24:36,  6.57s/it]
 72%|███████▏  | 161170/225000 [312:39:57<97:41:22,  5.51s/it] 
 72%|███████▏  | 161171/225000 [312:40:05<113:13:22,  6.39s/it]
 72%|███████▏  | 161172/225000 [312:40:09<99:37:37,  5.62s/it] 
 72%|███████▏  | 161173/225000 [312:40:13<92:57:09,  5.24s/it]
 72%|███████▏  | 161174/225000 [312:40:17<85:42:04,  4.83s/it]
 72%|███████▏  | 161175/225000 [312:40:21<81:41:33,  4.61s/it]
 72%|███████▏  | 161176/225000 [312:40:32<112:54:45,  6.37s/it]
 72%|███████▏  | 161177/225000 [312:40:36<105:05:57,  5.93s/it]
 72%|███████▏  | 161178/225000 [312:40:44<112:39:48,  6.36s/it]
 72%|███████▏  | 161179/225000 [312:40:48<99:04:58,  5.59s/it] 
 72%|███████▏  | 161180/225000 [312:40:53<96:50:20,  5.46s/it]
 72%|███████▏  | 161181/225000 [312:41:02<115:03:08,  6.49s/it]
 72%|███████▏  | 161182/225000 [312:41:09<117:19:38,  6.62s/it]
 72%|███████▏  | 161183/225000 [312:41:12<99:45:00,  5.63s/it] 
 72%|███████▏  | 161184/225000 [312:41:19<106:51:19,  6.03s/it]
 72%|███████▏  | 161185/225000 [312:41:22<91:54:11,  5.18s/it] 
 72%|███████▏  | 161186/225000 [312:41:29<98:19:59,  5.55s/it]
 72%|███████▏  | 161187/225000 [312:41:35<102:43:24,  5.80s/it]
 72%|███████▏  | 161188/225000 [312:41:39<95:20:52,  5.38s/it] 
 72%|███████▏  | 161189/225000 [312:41:43<88:10:19,  4.97s/it]
 72%|███████▏  | 161190/225000 [312:41:47<83:25:03,  4.71s/it]
 72%|███████▏  | 161191/225000 [312:41:59<120:19:38,  6.79s/it]
 72%|███████▏  | 161192/225000 [312:42:04<108:58:27,  6.15s/it]
 72%|███████▏  | 161193/225000 [312:42:08<100:43:42,  5.68s/it]
 72%|███████▏  | 161194/225000 [312:42:13<94:38:27,  5.34s/it] 
 72%|███████▏  | 161195/225000 [312:42:17<90:52:35,  5.13s/it]
 72%|███████▏  | 161196/225000 [312:42:22<86:45:07,  4.89s/it]
 72%|███████▏  | 161197/225000 [312:42:26<84:16:44,  4.76s/it]
 72%|███████▏  | 161198/225000 [312:42:38<122:17:40,  6.90s/it]
 72%|███████▏  | 161199/225000 [312:42:43<112:23:31,  6.34s/it]
 72%|███████▏  | 161200/225000 [312:42:47<99:25:28,  5.61s/it] 
                                                              
{'loss': '0.7844', 'grad_norm': '1.118', 'learning_rate': '1.012e-05', 'epoch': '0.7164', 'num_input_tokens_seen': 517633712, 'train_runtime': '1.126e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161200/225000 [312:42:47<99:25:28,  5.61s/it]
 72%|███████▏  | 161201/225000 [312:42:51<90:22:07,  5.10s/it]
 72%|███████▏  | 161202/225000 [312:42:56<87:48:11,  4.95s/it]
 72%|███████▏  | 161203/225000 [312:43:00<85:05:47,  4.80s/it]
 72%|███████▏  | 161204/225000 [312:43:09<104:46:30,  5.91s/it]
 72%|███████▏  | 161205/225000 [312:43:16<113:55:39,  6.43s/it]
 72%|███████▏  | 161206/225000 [312:43:20<98:22:57,  5.55s/it] 
 72%|███████▏  | 161207/225000 [312:43:30<121:06:56,  6.83s/it]
 72%|███████▏  | 161208/225000 [312:43:34<107:46:11,  6.08s/it]
 72%|███████▏  | 161209/225000 [312:43:37<92:39:42,  5.23s/it] 
 72%|███████▏  | 161210/225000 [312:43:42<92:51:01,  5.24s/it]
 72%|███████▏  | 161211/225000 [312:43:49<97:34:21,  5.51s/it]
 72%|███████▏  | 161212/225000 [312:43:57<114:21:03,  6.45s/it]
 72%|███████▏  | 161213/225000 [312:44:03<112:07:46,  6.33s/it]
 72%|███████▏  | 161214/225000 [312:44:07<100:59:40,  5.70s/it]
 72%|███████▏  | 161215/225000 [312:44:12<93:29:12,  5.28s/it] 
 72%|███████▏  | 161216/225000 [312:44:16<86:28:29,  4.88s/it]
 72%|███████▏  | 161217/225000 [312:44:20<83:19:35,  4.70s/it]
 72%|███████▏  | 161218/225000 [312:44:32<119:48:15,  6.76s/it]
 72%|███████▏  | 161219/225000 [312:44:35<102:45:58,  5.80s/it]
 72%|███████▏  | 161220/225000 [312:44:39<91:57:06,  5.19s/it] 
 72%|███████▏  | 161221/225000 [312:44:43<86:21:59,  4.87s/it]
 72%|███████▏  | 161222/225000 [312:44:48<85:59:42,  4.85s/it]
 72%|███████▏  | 161223/225000 [312:44:52<82:39:07,  4.67s/it]
 72%|███████▏  | 161224/225000 [312:44:57<85:23:52,  4.82s/it]
 72%|███████▏  | 161225/225000 [312:45:09<123:38:01,  6.98s/it]
 72%|███████▏  | 161226/225000 [312:45:14<112:24:39,  6.35s/it]
 72%|███████▏  | 161227/225000 [312:45:19<102:06:49,  5.76s/it]
 72%|███████▏  | 161228/225000 [312:45:26<109:48:06,  6.20s/it]
 72%|███████▏  | 161229/225000 [312:45:29<96:32:22,  5.45s/it] 
 72%|███████▏  | 161230/225000 [312:45:38<115:36:57,  6.53s/it]
 72%|███████▏  | 161231/225000 [312:45:43<107:16:02,  6.06s/it]
 72%|███████▏  | 161232/225000 [312:45:49<104:56:12,  5.92s/it]
 72%|███████▏  | 161233/225000 [312:45:57<114:10:14,  6.45s/it]
 72%|███████▏  | 161234/225000 [312:46:00<96:51:10,  5.47s/it] 
 72%|███████▏  | 161235/225000 [312:46:07<107:12:36,  6.05s/it]
 72%|███████▏  | 161236/225000 [312:46:11<94:11:14,  5.32s/it] 
 72%|███████▏  | 161237/225000 [312:46:19<107:37:04,  6.08s/it]
 72%|███████▏  | 161238/225000 [312:46:22<93:58:12,  5.31s/it] 
 72%|███████▏  | 161239/225000 [312:46:30<104:27:14,  5.90s/it]
 72%|███████▏  | 161240/225000 [312:46:34<95:02:37,  5.37s/it] 
 72%|███████▏  | 161241/225000 [312:46:38<89:03:59,  5.03s/it]
 72%|███████▏  | 161242/225000 [312:46:42<85:32:22,  4.83s/it]
 72%|███████▏  | 161243/225000 [312:46:49<98:17:32,  5.55s/it]
 72%|███████▏  | 161244/225000 [312:47:00<126:37:36,  7.15s/it]
 72%|███████▏  | 161245/225000 [312:47:05<110:56:16,  6.26s/it]
 72%|███████▏  | 161246/225000 [312:47:09<100:38:36,  5.68s/it]
 72%|███████▏  | 161247/225000 [312:47:13<90:56:30,  5.14s/it] 
 72%|███████▏  | 161248/225000 [312:47:17<86:13:27,  4.87s/it]
 72%|███████▏  | 161249/225000 [312:47:29<122:35:37,  6.92s/it]
 72%|███████▏  | 161250/225000 [312:47:35<116:40:04,  6.59s/it]
                                                               
{'loss': '0.7707', 'grad_norm': '1.134', 'learning_rate': '1.011e-05', 'epoch': '0.7167', 'num_input_tokens_seen': 517764240, 'train_runtime': '1.126e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161250/225000 [312:47:35<116:40:04,  6.59s/it]
 72%|███████▏  | 161251/225000 [312:47:38<101:04:04,  5.71s/it]
 72%|███████▏  | 161252/225000 [312:47:42<90:42:24,  5.12s/it] 
 72%|███████▏  | 161253/225000 [312:47:46<83:25:52,  4.71s/it]
 72%|███████▏  | 161254/225000 [312:47:51<84:01:47,  4.75s/it]
 72%|███████▏  | 161255/225000 [312:47:55<83:31:07,  4.72s/it]
 72%|███████▏  | 161256/225000 [312:48:00<85:55:43,  4.85s/it]
 72%|███████▏  | 161257/225000 [312:48:06<90:30:38,  5.11s/it]
 72%|███████▏  | 161258/225000 [312:48:11<91:24:54,  5.16s/it]
 72%|███████▏  | 161259/225000 [312:48:17<91:36:33,  5.17s/it]
 72%|███████▏  | 161260/225000 [312:48:22<91:28:17,  5.17s/it]
 72%|███████▏  | 161261/225000 [312:48:33<125:24:32,  7.08s/it]
 72%|███████▏  | 161262/225000 [312:48:41<127:28:39,  7.20s/it]
 72%|███████▏  | 161263/225000 [312:48:44<108:34:21,  6.13s/it]
 72%|███████▏  | 161264/225000 [312:48:50<106:26:56,  6.01s/it]
 72%|███████▏  | 161265/225000 [312:48:56<107:30:14,  6.07s/it]
 72%|███████▏  | 161266/225000 [312:49:04<115:55:13,  6.55s/it]
 72%|███████▏  | 161267/225000 [312:49:08<102:06:02,  5.77s/it]
 72%|███████▏  | 161268/225000 [312:49:11<90:12:11,  5.10s/it] 
 72%|███████▏  | 161269/225000 [312:49:21<112:23:58,  6.35s/it]
 72%|███████▏  | 161270/225000 [312:49:27<114:02:57,  6.44s/it]
 72%|███████▏  | 161271/225000 [312:49:31<99:51:05,  5.64s/it] 
 72%|███████▏  | 161272/225000 [312:49:35<87:56:14,  4.97s/it]
 72%|███████▏  | 161273/225000 [312:49:43<108:50:16,  6.15s/it]
 72%|███████▏  | 161274/225000 [312:49:48<100:21:35,  5.67s/it]
 72%|███████▏  | 161275/225000 [312:49:52<92:46:37,  5.24s/it] 
 72%|███████▏  | 161276/225000 [312:49:57<91:25:26,  5.16s/it]
 72%|███████▏  | 161277/225000 [312:50:02<89:34:54,  5.06s/it]
 72%|███████▏  | 161278/225000 [312:50:07<89:04:34,  5.03s/it]
 72%|███████▏  | 161279/225000 [312:50:12<87:30:51,  4.94s/it]
 72%|███████▏  | 161280/225000 [312:50:16<84:10:38,  4.76s/it]
 72%|███████▏  | 161281/225000 [312:50:21<84:06:28,  4.75s/it]
 72%|███████▏  | 161282/225000 [312:50:31<111:09:43,  6.28s/it]
 72%|███████▏  | 161283/225000 [312:50:41<130:28:57,  7.37s/it]
 72%|███████▏  | 161284/225000 [312:50:44<109:19:16,  6.18s/it]
 72%|███████▏  | 161285/225000 [312:50:50<107:22:37,  6.07s/it]
 72%|███████▏  | 161286/225000 [312:50:56<108:25:19,  6.13s/it]
 72%|███████▏  | 161287/225000 [312:51:02<107:29:03,  6.07s/it]
 72%|███████▏  | 161288/225000 [312:51:08<108:29:24,  6.13s/it]
 72%|███████▏  | 161289/225000 [312:51:12<96:26:32,  5.45s/it] 
 72%|███████▏  | 161290/225000 [312:51:16<87:57:15,  4.97s/it]
 72%|███████▏  | 161291/225000 [312:51:20<82:15:44,  4.65s/it]
 72%|███████▏  | 161292/225000 [312:51:27<95:54:29,  5.42s/it]
 72%|███████▏  | 161293/225000 [312:51:36<115:15:06,  6.51s/it]
 72%|███████▏  | 161294/225000 [312:51:39<97:15:21,  5.50s/it] 
 72%|███████▏  | 161295/225000 [312:51:43<88:12:30,  4.98s/it]
 72%|███████▏  | 161296/225000 [312:51:48<89:13:13,  5.04s/it]
 72%|███████▏  | 161297/225000 [312:51:59<118:45:01,  6.71s/it]
 72%|███████▏  | 161298/225000 [312:52:02<102:29:31,  5.79s/it]
 72%|███████▏  | 161299/225000 [312:52:06<90:48:47,  5.13s/it] 
 72%|███████▏  | 161300/225000 [312:52:11<89:05:44,  5.04s/it]
                                                              
{'loss': '0.7723', 'grad_norm': '1.191', 'learning_rate': '1.009e-05', 'epoch': '0.7169', 'num_input_tokens_seen': 517893352, 'train_runtime': '1.126e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161300/225000 [312:52:11<89:05:44,  5.04s/it]
 72%|███████▏  | 161301/225000 [312:52:18<98:44:53,  5.58s/it]
 72%|███████▏  | 161302/225000 [312:52:24<104:17:56,  5.89s/it]
 72%|███████▏  | 161303/225000 [312:52:32<112:21:48,  6.35s/it]
 72%|███████▏  | 161304/225000 [312:52:35<96:15:43,  5.44s/it] 
 72%|███████▏  | 161305/225000 [312:52:44<114:39:49,  6.48s/it]
 72%|███████▏  | 161306/225000 [312:52:49<105:05:34,  5.94s/it]
 72%|███████▏  | 161307/225000 [312:52:53<97:27:33,  5.51s/it] 
 72%|███████▏  | 161308/225000 [312:52:58<94:11:39,  5.32s/it]
 72%|███████▏  | 161309/225000 [312:53:03<91:15:00,  5.16s/it]
 72%|███████▏  | 161310/225000 [312:53:08<89:56:47,  5.08s/it]
 72%|███████▏  | 161311/225000 [312:53:13<89:26:37,  5.06s/it]
 72%|███████▏  | 161312/225000 [312:53:17<87:21:57,  4.94s/it]
 72%|███████▏  | 161313/225000 [312:53:26<106:32:58,  6.02s/it]
 72%|███████▏  | 161314/225000 [312:53:37<131:01:39,  7.41s/it]
 72%|███████▏  | 161315/225000 [312:53:40<112:15:54,  6.35s/it]
 72%|███████▏  | 161316/225000 [312:53:44<98:17:45,  5.56s/it] 
 72%|███████▏  | 161317/225000 [312:53:54<123:02:56,  6.96s/it]
 72%|███████▏  | 161318/225000 [312:53:58<104:53:06,  5.93s/it]
 72%|███████▏  | 161319/225000 [312:54:04<107:51:56,  6.10s/it]
 72%|███████▏  | 161320/225000 [312:54:11<109:48:18,  6.21s/it]
 72%|███████▏  | 161321/225000 [312:54:15<98:57:27,  5.59s/it] 
 72%|███████▏  | 161322/225000 [312:54:19<91:05:17,  5.15s/it]
 72%|███████▏  | 161323/225000 [312:54:23<81:55:29,  4.63s/it]
 72%|███████▏  | 161324/225000 [312:54:34<117:45:58,  6.66s/it]
 72%|███████▏  | 161325/225000 [312:54:38<104:18:56,  5.90s/it]
 72%|███████▏  | 161326/225000 [312:54:43<96:24:28,  5.45s/it] 
 72%|███████▏  | 161327/225000 [312:54:47<89:55:53,  5.08s/it]
 72%|███████▏  | 161328/225000 [312:54:51<84:44:38,  4.79s/it]
 72%|███████▏  | 161329/225000 [312:54:55<82:42:08,  4.68s/it]
 72%|███████▏  | 161330/225000 [312:55:00<81:33:46,  4.61s/it]
 72%|███████▏  | 161331/225000 [312:55:09<105:46:36,  5.98s/it]
 72%|███████▏  | 161332/225000 [312:55:17<118:05:50,  6.68s/it]
 72%|███████▏  | 161333/225000 [312:55:21<101:10:53,  5.72s/it]
 72%|███████▏  | 161334/225000 [312:55:25<93:38:11,  5.29s/it] 
 72%|███████▏  | 161335/225000 [312:55:34<111:09:39,  6.29s/it]
 72%|███████▏  | 161336/225000 [312:55:37<93:47:50,  5.30s/it] 
 72%|███████▏  | 161337/225000 [312:55:43<98:30:29,  5.57s/it]
 72%|███████▏  | 161338/225000 [312:55:48<94:34:13,  5.35s/it]
 72%|███████▏  | 161339/225000 [312:55:52<90:46:00,  5.13s/it]
 72%|███████▏  | 161340/225000 [312:55:57<87:28:03,  4.95s/it]
 72%|███████▏  | 161341/225000 [312:56:01<83:39:59,  4.73s/it]
 72%|███████▏  | 161342/225000 [312:56:05<82:28:35,  4.66s/it]
 72%|███████▏  | 161343/225000 [312:56:14<104:37:45,  5.92s/it]
 72%|███████▏  | 161344/225000 [312:56:23<117:33:24,  6.65s/it]
 72%|███████▏  | 161345/225000 [312:56:26<100:33:01,  5.69s/it]
 72%|███████▏  | 161346/225000 [312:56:31<95:42:38,  5.41s/it] 
 72%|███████▏  | 161347/225000 [312:56:42<124:12:18,  7.02s/it]
 72%|███████▏  | 161348/225000 [312:56:46<111:04:17,  6.28s/it]
 72%|███████▏  | 161349/225000 [312:56:51<100:46:55,  5.70s/it]
 72%|███████▏  | 161350/225000 [312:56:55<91:46:39,  5.19s/it] 
                                                              
{'loss': '0.7667', 'grad_norm': '1.296', 'learning_rate': '1.008e-05', 'epoch': '0.7171', 'num_input_tokens_seen': 518023832, 'train_runtime': '1.127e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161350/225000 [312:56:55<91:46:39,  5.19s/it]
 72%|███████▏  | 161351/225000 [312:56:59<87:51:23,  4.97s/it]
 72%|███████▏  | 161352/225000 [312:57:05<90:48:25,  5.14s/it]
 72%|███████▏  | 161353/225000 [312:57:10<91:53:05,  5.20s/it]
 72%|███████▏  | 161354/225000 [312:57:15<92:03:14,  5.21s/it]
 72%|███████▏  | 161355/225000 [312:57:26<121:58:14,  6.90s/it]
 72%|███████▏  | 161356/225000 [312:57:33<120:26:30,  6.81s/it]
 72%|███████▏  | 161357/225000 [312:57:36<103:04:00,  5.83s/it]
 72%|███████▏  | 161358/225000 [312:57:41<99:12:59,  5.61s/it] 
 72%|███████▏  | 161359/225000 [312:57:47<98:30:55,  5.57s/it]
 72%|███████▏  | 161360/225000 [312:57:56<120:19:40,  6.81s/it]
 72%|███████▏  | 161361/225000 [312:58:01<107:21:56,  6.07s/it]
 72%|███████▏  | 161362/225000 [312:58:05<98:11:49,  5.56s/it] 
 72%|███████▏  | 161363/225000 [312:58:09<90:32:04,  5.12s/it]
 72%|███████▏  | 161364/225000 [312:58:14<88:39:51,  5.02s/it]
 72%|███████▏  | 161365/225000 [312:58:26<125:58:39,  7.13s/it]
 72%|███████▏  | 161366/225000 [312:58:29<104:27:42,  5.91s/it]
 72%|███████▏  | 161367/225000 [312:58:34<100:13:22,  5.67s/it]
 72%|███████▏  | 161368/225000 [312:58:39<97:40:03,  5.53s/it] 
 72%|███████▏  | 161369/225000 [312:58:44<95:04:49,  5.38s/it]
 72%|███████▏  | 161370/225000 [312:58:49<91:38:42,  5.19s/it]
 72%|███████▏  | 161371/225000 [312:58:55<92:51:27,  5.25s/it]
 72%|███████▏  | 161372/225000 [312:59:00<92:47:08,  5.25s/it]
 72%|███████▏  | 161373/225000 [312:59:04<88:19:52,  5.00s/it]
 72%|███████▏  | 161374/225000 [312:59:09<88:07:42,  4.99s/it]
 72%|███████▏  | 161375/225000 [312:59:15<91:02:54,  5.15s/it]
 72%|███████▏  | 161376/225000 [312:59:20<91:42:42,  5.19s/it]
 72%|███████▏  | 161377/225000 [312:59:25<90:48:31,  5.14s/it]
 72%|███████▏  | 161378/225000 [312:59:37<128:24:33,  7.27s/it]
 72%|███████▏  | 161379/225000 [312:59:45<128:47:07,  7.29s/it]
 72%|███████▏  | 161380/225000 [312:59:49<112:17:49,  6.35s/it]
 72%|███████▏  | 161381/225000 [312:59:53<101:18:00,  5.73s/it]
 72%|███████▏  | 161382/225000 [312:59:58<95:19:15,  5.39s/it] 
 72%|███████▏  | 161383/225000 [313:00:03<92:40:16,  5.24s/it]
 72%|███████▏  | 161384/225000 [313:00:07<89:20:56,  5.06s/it]
 72%|███████▏  | 161385/225000 [313:00:12<86:03:16,  4.87s/it]
 72%|███████▏  | 161386/225000 [313:00:20<105:47:43,  5.99s/it]
 72%|███████▏  | 161387/225000 [313:00:28<117:21:43,  6.64s/it]
 72%|███████▏  | 161388/225000 [313:00:34<113:18:22,  6.41s/it]
 72%|███████▏  | 161389/225000 [313:00:40<108:29:21,  6.14s/it]
 72%|███████▏  | 161390/225000 [313:00:45<105:01:15,  5.94s/it]
 72%|███████▏  | 161391/225000 [313:00:50<101:12:27,  5.73s/it]
 72%|███████▏  | 161392/225000 [313:00:56<99:15:20,  5.62s/it] 
 72%|███████▏  | 161393/225000 [313:01:01<99:10:47,  5.61s/it]
 72%|███████▏  | 161394/225000 [313:01:07<97:23:51,  5.51s/it]
 72%|███████▏  | 161395/225000 [313:01:12<98:44:06,  5.59s/it]
 72%|███████▏  | 161396/225000 [313:01:18<96:48:45,  5.48s/it]
 72%|███████▏  | 161397/225000 [313:01:23<95:25:25,  5.40s/it]
 72%|███████▏  | 161398/225000 [313:01:28<95:45:14,  5.42s/it]
 72%|███████▏  | 161399/225000 [313:01:41<131:58:13,  7.47s/it]
 72%|███████▏  | 161400/225000 [313:01:45<114:37:40,  6.49s/it]
                                                               
{'loss': '0.7666', 'grad_norm': '1.263', 'learning_rate': '1.006e-05', 'epoch': '0.7173', 'num_input_tokens_seen': 518154688, 'train_runtime': '1.127e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161400/225000 [313:01:45<114:37:40,  6.49s/it]
 72%|███████▏  | 161401/225000 [313:01:49<101:44:37,  5.76s/it]
 72%|███████▏  | 161402/225000 [313:01:53<94:00:28,  5.32s/it] 
 72%|███████▏  | 161403/225000 [313:01:58<89:04:44,  5.04s/it]
 72%|███████▏  | 161404/225000 [313:02:02<86:16:42,  4.88s/it]
 72%|███████▏  | 161405/225000 [313:02:08<92:52:26,  5.26s/it]
 72%|███████▏  | 161406/225000 [313:02:19<119:53:49,  6.79s/it]
 72%|███████▏  | 161407/225000 [313:02:22<102:37:29,  5.81s/it]
 72%|███████▏  | 161408/225000 [313:02:30<112:39:47,  6.38s/it]
 72%|███████▏  | 161409/225000 [313:02:34<98:54:52,  5.60s/it] 
 72%|███████▏  | 161410/225000 [313:02:38<90:27:42,  5.12s/it]
 72%|███████▏  | 161411/225000 [313:02:42<84:22:20,  4.78s/it]
 72%|███████▏  | 161412/225000 [313:02:46<81:56:40,  4.64s/it]
 72%|███████▏  | 161413/225000 [313:02:58<119:23:09,  6.76s/it]
 72%|███████▏  | 161414/225000 [313:03:03<110:26:30,  6.25s/it]
 72%|███████▏  | 161415/225000 [313:03:07<100:39:20,  5.70s/it]
 72%|███████▏  | 161416/225000 [313:03:11<91:40:50,  5.19s/it] 
 72%|███████▏  | 161417/225000 [313:03:15<85:20:10,  4.83s/it]
 72%|███████▏  | 161418/225000 [313:03:25<113:55:08,  6.45s/it]
 72%|███████▏  | 161419/225000 [313:03:31<110:31:43,  6.26s/it]
 72%|███████▏  | 161420/225000 [313:03:36<104:17:49,  5.91s/it]
 72%|███████▏  | 161421/225000 [313:03:43<106:38:46,  6.04s/it]
 72%|███████▏  | 161422/225000 [313:03:49<110:46:04,  6.27s/it]
 72%|███████▏  | 161423/225000 [313:03:54<104:14:10,  5.90s/it]
 72%|███████▏  | 161424/225000 [313:04:00<100:56:00,  5.72s/it]
 72%|███████▏  | 161425/225000 [313:04:04<94:49:13,  5.37s/it] 
 72%|███████▏  | 161426/225000 [313:04:09<93:01:12,  5.27s/it]
 72%|███████▏  | 161427/225000 [313:04:14<91:08:41,  5.16s/it]
 72%|███████▏  | 161428/225000 [313:04:19<88:01:54,  4.99s/it]
 72%|███████▏  | 161429/225000 [313:04:24<88:40:00,  5.02s/it]
 72%|███████▏  | 161430/225000 [313:04:29<89:17:46,  5.06s/it]
 72%|███████▏  | 161431/225000 [313:04:33<84:46:35,  4.80s/it]
 72%|███████▏  | 161432/225000 [313:04:38<84:45:48,  4.80s/it]
 72%|███████▏  | 161433/225000 [313:04:47<107:39:46,  6.10s/it]
 72%|███████▏  | 161434/225000 [313:04:56<122:53:06,  6.96s/it]
 72%|███████▏  | 161435/225000 [313:05:00<108:10:26,  6.13s/it]
 72%|███████▏  | 161436/225000 [313:05:04<97:31:13,  5.52s/it] 
 72%|███████▏  | 161437/225000 [313:05:10<97:57:51,  5.55s/it]
 72%|███████▏  | 161438/225000 [313:05:17<105:01:38,  5.95s/it]
 72%|███████▏  | 161439/225000 [313:05:22<98:37:02,  5.59s/it] 
 72%|███████▏  | 161440/225000 [313:05:27<94:52:38,  5.37s/it]
 72%|███████▏  | 161441/225000 [313:05:32<93:37:46,  5.30s/it]
 72%|███████▏  | 161442/225000 [313:05:38<99:51:06,  5.66s/it]
 72%|███████▏  | 161443/225000 [313:05:47<116:15:31,  6.59s/it]
 72%|███████▏  | 161444/225000 [313:05:52<107:21:36,  6.08s/it]
 72%|███████▏  | 161445/225000 [313:05:56<99:29:08,  5.64s/it] 
 72%|███████▏  | 161446/225000 [313:06:01<96:45:36,  5.48s/it]
 72%|███████▏  | 161447/225000 [313:06:06<91:32:27,  5.19s/it]
 72%|███████▏  | 161448/225000 [313:06:11<90:32:34,  5.13s/it]
 72%|███████▏  | 161449/225000 [313:06:16<89:38:35,  5.08s/it]
 72%|███████▏  | 161450/225000 [313:06:21<89:29:16,  5.07s/it]
                                                              
{'loss': '0.7699', 'grad_norm': '1.029', 'learning_rate': '1.005e-05', 'epoch': '0.7176', 'num_input_tokens_seen': 518284560, 'train_runtime': '1.127e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161450/225000 [313:06:21<89:29:16,  5.07s/it]
 72%|███████▏  | 161451/225000 [313:06:26<89:37:58,  5.08s/it]
 72%|███████▏  | 161452/225000 [313:06:38<126:24:04,  7.16s/it]
 72%|███████▏  | 161453/225000 [313:06:43<112:15:40,  6.36s/it]
 72%|███████▏  | 161454/225000 [313:06:50<118:04:50,  6.69s/it]
 72%|███████▏  | 161455/225000 [313:06:54<101:55:26,  5.77s/it]
 72%|███████▏  | 161456/225000 [313:07:02<113:45:37,  6.44s/it]
 72%|███████▏  | 161457/225000 [313:07:05<97:54:21,  5.55s/it] 
 72%|███████▏  | 161458/225000 [313:07:12<104:46:12,  5.94s/it]
 72%|███████▏  | 161459/225000 [313:07:16<95:08:50,  5.39s/it] 
 72%|███████▏  | 161460/225000 [313:07:20<89:06:36,  5.05s/it]
 72%|███████▏  | 161461/225000 [313:07:30<114:23:42,  6.48s/it]
 72%|███████▏  | 161462/225000 [313:07:34<99:38:56,  5.65s/it] 
 72%|███████▏  | 161463/225000 [313:07:38<89:54:13,  5.09s/it]
 72%|███████▏  | 161464/225000 [313:07:42<83:28:25,  4.73s/it]
 72%|███████▏  | 161465/225000 [313:07:53<119:15:20,  6.76s/it]
 72%|███████▏  | 161466/225000 [313:07:57<101:59:32,  5.78s/it]
 72%|███████▏  | 161467/225000 [313:08:03<106:38:31,  6.04s/it]
 72%|███████▏  | 161468/225000 [313:08:07<92:13:29,  5.23s/it] 
 72%|███████▏  | 161469/225000 [313:08:15<111:11:36,  6.30s/it]
 72%|███████▏  | 161470/225000 [313:08:19<97:13:02,  5.51s/it] 
 72%|███████▏  | 161471/225000 [313:08:28<116:57:26,  6.63s/it]
 72%|███████▏  | 161472/225000 [313:08:34<111:43:58,  6.33s/it]
 72%|███████▏  | 161473/225000 [313:08:37<96:57:15,  5.49s/it] 
 72%|███████▏  | 161474/225000 [313:08:41<85:08:44,  4.83s/it]
 72%|███████▏  | 161475/225000 [313:08:50<109:14:07,  6.19s/it]
 72%|███████▏  | 161476/225000 [313:08:55<100:20:36,  5.69s/it]
 72%|███████▏  | 161477/225000 [313:08:59<92:16:22,  5.23s/it] 
 72%|███████▏  | 161478/225000 [313:09:03<88:41:54,  5.03s/it]
 72%|███████▏  | 161479/225000 [313:09:08<88:07:56,  4.99s/it]
 72%|███████▏  | 161480/225000 [313:09:13<87:38:14,  4.97s/it]
 72%|███████▏  | 161481/225000 [313:09:18<88:21:47,  5.01s/it]
 72%|███████▏  | 161482/225000 [313:09:23<87:22:55,  4.95s/it]
 72%|███████▏  | 161483/225000 [313:09:28<86:28:23,  4.90s/it]
 72%|███████▏  | 161484/225000 [313:09:36<104:12:23,  5.91s/it]
 72%|███████▏  | 161485/225000 [313:09:44<117:06:10,  6.64s/it]
 72%|███████▏  | 161486/225000 [313:09:48<102:13:50,  5.79s/it]
 72%|███████▏  | 161487/225000 [313:09:57<119:47:22,  6.79s/it]
 72%|███████▏  | 161488/225000 [313:10:01<101:38:37,  5.76s/it]
 72%|███████▏  | 161489/225000 [313:10:04<89:43:15,  5.09s/it] 
 72%|███████▏  | 161490/225000 [313:10:09<87:27:08,  4.96s/it]
 72%|███████▏  | 161491/225000 [313:10:14<88:11:40,  5.00s/it]
 72%|███████▏  | 161492/225000 [313:10:25<122:29:29,  6.94s/it]
 72%|███████▏  | 161493/225000 [313:10:29<103:40:24,  5.88s/it]
 72%|███████▏  | 161494/225000 [313:10:33<95:22:35,  5.41s/it] 
 72%|███████▏  | 161495/225000 [313:10:38<93:27:15,  5.30s/it]
 72%|███████▏  | 161496/225000 [313:10:43<93:13:39,  5.29s/it]
 72%|███████▏  | 161497/225000 [313:10:49<92:09:41,  5.22s/it]
 72%|███████▏  | 161498/225000 [313:10:53<89:31:00,  5.07s/it]
 72%|███████▏  | 161499/225000 [313:11:00<96:10:52,  5.45s/it]
 72%|███████▏  | 161500/225000 [313:11:10<120:58:54,  6.86s/it]
                                                               
{'loss': '0.7768', 'grad_norm': '1.169', 'learning_rate': '1.003e-05', 'epoch': '0.7178', 'num_input_tokens_seen': 518414392, 'train_runtime': '1.127e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161500/225000 [313:11:10<120:58:54,  6.86s/it]
 72%|███████▏  | 161501/225000 [313:11:13<102:59:06,  5.84s/it]
 72%|███████▏  | 161502/225000 [313:11:19<104:28:58,  5.92s/it]
 72%|███████▏  | 161503/225000 [313:11:27<114:48:57,  6.51s/it]
 72%|███████▏  | 161504/225000 [313:11:31<100:48:38,  5.72s/it]
 72%|███████▏  | 161505/225000 [313:11:35<90:45:03,  5.15s/it] 
 72%|███████▏  | 161506/225000 [313:11:44<113:12:05,  6.42s/it]
 72%|███████▏  | 161507/225000 [313:11:49<101:46:52,  5.77s/it]
 72%|███████▏  | 161508/225000 [313:11:53<94:25:27,  5.35s/it] 
 72%|███████▏  | 161509/225000 [313:11:57<86:28:31,  4.90s/it]
 72%|███████▏  | 161510/225000 [313:12:07<115:26:08,  6.55s/it]
 72%|███████▏  | 161511/225000 [313:12:12<105:22:04,  5.97s/it]
 72%|███████▏  | 161512/225000 [313:12:19<113:21:51,  6.43s/it]
 72%|███████▏  | 161513/225000 [313:12:24<101:55:56,  5.78s/it]
 72%|███████▏  | 161514/225000 [313:12:27<90:24:39,  5.13s/it] 
 72%|███████▏  | 161515/225000 [313:12:31<84:18:31,  4.78s/it]
 72%|███████▏  | 161516/225000 [313:12:42<117:16:45,  6.65s/it]
 72%|███████▏  | 161517/225000 [313:12:45<97:35:38,  5.53s/it] 
 72%|███████▏  | 161518/225000 [313:12:50<95:17:28,  5.40s/it]
 72%|███████▏  | 161519/225000 [313:12:57<104:53:44,  5.95s/it]
 72%|███████▏  | 161520/225000 [313:13:03<102:15:38,  5.80s/it]
 72%|███████▏  | 161521/225000 [313:13:07<94:38:27,  5.37s/it] 
 72%|███████▏  | 161522/225000 [313:13:11<89:00:12,  5.05s/it]
 72%|███████▏  | 161523/225000 [313:13:16<85:51:13,  4.87s/it]
 72%|███████▏  | 161524/225000 [313:13:20<80:45:24,  4.58s/it]
 72%|███████▏  | 161525/225000 [313:13:31<115:05:15,  6.53s/it]
 72%|███████▏  | 161526/225000 [313:13:36<109:32:07,  6.21s/it]
 72%|███████▏  | 161527/225000 [313:13:40<95:05:49,  5.39s/it] 
 72%|███████▏  | 161528/225000 [313:13:45<92:53:08,  5.27s/it]
 72%|███████▏  | 161529/225000 [313:13:50<89:38:57,  5.08s/it]
 72%|███████▏  | 161530/225000 [313:13:54<88:46:49,  5.04s/it]
 72%|███████▏  | 161531/225000 [313:13:59<87:16:38,  4.95s/it]
 72%|███████▏  | 161532/225000 [313:14:04<87:11:32,  4.95s/it]
 72%|███████▏  | 161533/225000 [313:14:09<86:37:55,  4.91s/it]
 72%|███████▏  | 161534/225000 [313:14:14<88:18:16,  5.01s/it]
 72%|███████▏  | 161535/225000 [313:14:19<89:29:07,  5.08s/it]
 72%|███████▏  | 161536/225000 [313:14:25<90:20:19,  5.12s/it]
 72%|███████▏  | 161537/225000 [313:14:30<89:58:52,  5.10s/it]
 72%|███████▏  | 161538/225000 [313:14:37<99:14:34,  5.63s/it]
 72%|███████▏  | 161539/225000 [313:14:50<139:08:15,  7.89s/it]
 72%|███████▏  | 161540/225000 [313:14:54<121:10:13,  6.87s/it]
 72%|███████▏  | 161541/225000 [313:14:58<103:13:46,  5.86s/it]
 72%|███████▏  | 161542/225000 [313:15:06<115:06:15,  6.53s/it]
 72%|███████▏  | 161543/225000 [313:15:10<100:28:56,  5.70s/it]
 72%|███████▏  | 161544/225000 [313:15:14<92:19:49,  5.24s/it] 
 72%|███████▏  | 161545/225000 [313:15:18<85:25:27,  4.85s/it]
 72%|███████▏  | 161546/225000 [313:15:23<86:48:19,  4.92s/it]
 72%|███████▏  | 161547/225000 [313:15:35<123:05:54,  6.98s/it]
 72%|███████▏  | 161548/225000 [313:15:38<103:04:41,  5.85s/it]
 72%|███████▏  | 161549/225000 [313:15:44<107:23:52,  6.09s/it]
 72%|███████▏  | 161550/225000 [313:15:49<98:38:38,  5.60s/it] 
                                                              
{'loss': '0.7666', 'grad_norm': '1.09', 'learning_rate': '1.002e-05', 'epoch': '0.718', 'num_input_tokens_seen': 518544152, 'train_runtime': '1.128e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161550/225000 [313:15:49<98:38:38,  5.60s/it]
 72%|███████▏  | 161551/225000 [313:15:54<93:46:44,  5.32s/it]
 72%|███████▏  | 161552/225000 [313:15:58<91:13:07,  5.18s/it]
 72%|███████▏  | 161553/225000 [313:16:03<88:57:03,  5.05s/it]
 72%|███████▏  | 161554/225000 [313:16:08<88:20:32,  5.01s/it]
 72%|███████▏  | 161555/225000 [313:16:13<88:18:44,  5.01s/it]
 72%|███████▏  | 161556/225000 [313:16:18<86:50:31,  4.93s/it]
 72%|███████▏  | 161557/225000 [313:16:24<95:15:06,  5.40s/it]
 72%|███████▏  | 161558/225000 [313:16:36<128:26:27,  7.29s/it]
 72%|███████▏  | 161559/225000 [313:16:39<108:13:50,  6.14s/it]
 72%|███████▏  | 161560/225000 [313:16:43<96:22:22,  5.47s/it] 
 72%|███████▏  | 161561/225000 [313:16:48<91:01:28,  5.17s/it]
 72%|███████▏  | 161562/225000 [313:16:52<85:51:12,  4.87s/it]
 72%|███████▏  | 161563/225000 [313:16:57<86:13:40,  4.89s/it]
 72%|███████▏  | 161564/225000 [313:17:07<112:58:22,  6.41s/it]
 72%|███████▏  | 161565/225000 [313:17:14<117:27:16,  6.67s/it]
 72%|███████▏  | 161566/225000 [313:17:21<116:21:06,  6.60s/it]
 72%|███████▏  | 161567/225000 [313:17:25<102:09:39,  5.80s/it]
 72%|███████▏  | 161568/225000 [313:17:30<97:58:48,  5.56s/it] 
 72%|███████▏  | 161569/225000 [313:17:38<115:20:33,  6.55s/it]
 72%|███████▏  | 161570/225000 [313:17:42<100:17:49,  5.69s/it]
 72%|███████▏  | 161571/225000 [313:17:51<116:42:48,  6.62s/it]
 72%|███████▏  | 161572/225000 [313:17:54<98:45:02,  5.60s/it] 
 72%|███████▏  | 161573/225000 [313:18:02<109:22:39,  6.21s/it]
 72%|███████▏  | 161574/225000 [313:18:06<101:08:52,  5.74s/it]
 72%|███████▏  | 161575/225000 [313:18:10<87:17:41,  4.95s/it] 
 72%|███████▏  | 161576/225000 [313:18:14<85:21:06,  4.84s/it]
 72%|███████▏  | 161577/225000 [313:18:19<83:37:15,  4.75s/it]
 72%|███████▏  | 161578/225000 [313:18:24<86:43:47,  4.92s/it]
 72%|███████▏  | 161579/225000 [313:18:36<122:40:59,  6.96s/it]
 72%|███████▏  | 161580/225000 [313:18:39<103:59:00,  5.90s/it]
 72%|███████▏  | 161581/225000 [313:18:47<114:57:38,  6.53s/it]
 72%|███████▏  | 161582/225000 [313:18:50<97:53:55,  5.56s/it] 
 72%|███████▏  | 161583/225000 [313:18:58<109:53:48,  6.24s/it]
 72%|███████▏  | 161584/225000 [313:19:02<98:47:03,  5.61s/it] 
 72%|███████▏  | 161585/225000 [313:19:07<93:56:37,  5.33s/it]
 72%|███████▏  | 161586/225000 [313:19:11<87:41:12,  4.98s/it]
 72%|███████▏  | 161587/225000 [313:19:15<82:01:24,  4.66s/it]
 72%|███████▏  | 161588/225000 [313:19:20<81:45:38,  4.64s/it]
 72%|███████▏  | 161589/225000 [313:19:24<81:33:25,  4.63s/it]
 72%|███████▏  | 161590/225000 [313:19:37<123:18:56,  7.00s/it]
 72%|███████▏  | 161591/225000 [313:19:42<114:51:13,  6.52s/it]
 72%|███████▏  | 161592/225000 [313:19:47<107:03:52,  6.08s/it]
 72%|███████▏  | 161593/225000 [313:19:51<96:23:30,  5.47s/it] 
 72%|███████▏  | 161594/225000 [313:19:58<105:06:41,  5.97s/it]
 72%|███████▏  | 161595/225000 [313:20:02<92:37:32,  5.26s/it] 
 72%|███████▏  | 161596/225000 [313:20:08<95:22:52,  5.42s/it]
 72%|███████▏  | 161597/225000 [313:20:15<102:58:40,  5.85s/it]
 72%|███████▏  | 161598/225000 [313:20:18<91:23:53,  5.19s/it] 
 72%|███████▏  | 161599/225000 [313:20:23<89:40:55,  5.09s/it]
 72%|███████▏  | 161600/225000 [313:20:28<88:12:07,  5.01s/it]
                                                              
{'loss': '0.7793', 'grad_norm': '1.233', 'learning_rate': '1e-05', 'epoch': '0.7182', 'num_input_tokens_seen': 518674080, 'train_runtime': '1.128e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161600/225000 [313:20:28<88:12:07,  5.01s/it]
 72%|███████▏  | 161601/225000 [313:20:33<85:30:03,  4.86s/it]
 72%|███████▏  | 161602/225000 [313:20:37<85:30:50,  4.86s/it]
 72%|███████▏  | 161603/225000 [313:20:48<114:55:30,  6.53s/it]
 72%|███████▏  | 161604/225000 [313:20:55<120:43:45,  6.86s/it]
 72%|███████▏  | 161605/225000 [313:20:59<101:58:53,  5.79s/it]
 72%|███████▏  | 161606/225000 [313:21:03<93:27:38,  5.31s/it] 
 72%|███████▏  | 161607/225000 [313:21:11<108:20:22,  6.15s/it]
 72%|███████▏  | 161608/225000 [313:21:17<109:02:21,  6.19s/it]
 72%|███████▏  | 161609/225000 [313:21:22<99:16:27,  5.64s/it] 
 72%|███████▏  | 161610/225000 [313:21:26<90:57:42,  5.17s/it]
 72%|███████▏  | 161611/225000 [313:21:30<88:19:26,  5.02s/it]
 72%|███████▏  | 161612/225000 [313:21:35<84:46:00,  4.81s/it]
 72%|███████▏  | 161613/225000 [313:21:39<81:37:51,  4.64s/it]
 72%|███████▏  | 161614/225000 [313:21:45<88:57:33,  5.05s/it]
 72%|███████▏  | 161615/225000 [313:21:58<132:05:10,  7.50s/it]
 72%|███████▏  | 161616/225000 [313:22:02<110:20:43,  6.27s/it]
 72%|███████▏  | 161617/225000 [313:22:07<103:56:19,  5.90s/it]
 72%|███████▏  | 161618/225000 [313:22:12<100:35:34,  5.71s/it]
 72%|███████▏  | 161619/225000 [313:22:16<94:08:08,  5.35s/it] 
 72%|███████▏  | 161620/225000 [313:22:21<89:55:57,  5.11s/it]
 72%|███████▏  | 161621/225000 [313:22:26<89:12:54,  5.07s/it]
 72%|███████▏  | 161622/225000 [313:22:31<88:33:09,  5.03s/it]
 72%|███████▏  | 161623/225000 [313:22:35<86:02:49,  4.89s/it]
 72%|███████▏  | 161624/225000 [313:22:40<85:49:05,  4.87s/it]
 72%|███████▏  | 161625/225000 [313:22:46<87:51:55,  4.99s/it]
 72%|███████▏  | 161626/225000 [313:22:58<124:49:17,  7.09s/it]
 72%|███████▏  | 161627/225000 [313:23:03<117:48:26,  6.69s/it]
 72%|███████▏  | 161628/225000 [313:23:08<106:38:00,  6.06s/it]
 72%|███████▏  | 161629/225000 [313:23:15<112:35:30,  6.40s/it]
 72%|███████▏  | 161630/225000 [313:23:19<98:30:22,  5.60s/it] 
 72%|███████▏  | 161631/225000 [313:23:27<113:12:49,  6.43s/it]
 72%|███████▏  | 161632/225000 [313:23:31<101:18:39,  5.76s/it]
 72%|███████▏  | 161633/225000 [313:23:38<105:57:59,  6.02s/it]
 72%|███████▏  | 161634/225000 [313:23:43<101:12:30,  5.75s/it]
 72%|███████▏  | 161635/225000 [313:23:48<95:50:53,  5.45s/it] 
 72%|███████▏  | 161636/225000 [313:23:53<94:40:12,  5.38s/it]
 72%|███████▏  | 161637/225000 [313:23:58<93:18:23,  5.30s/it]
 72%|███████▏  | 161638/225000 [313:24:03<93:00:14,  5.28s/it]
 72%|███████▏  | 161639/225000 [313:24:09<92:47:19,  5.27s/it]
 72%|███████▏  | 161640/225000 [313:24:13<86:17:09,  4.90s/it]
 72%|███████▏  | 161641/225000 [313:24:18<86:23:24,  4.91s/it]
 72%|███████▏  | 161642/225000 [313:24:23<88:55:10,  5.05s/it]
 72%|███████▏  | 161643/225000 [313:24:28<89:17:57,  5.07s/it]
 72%|███████▏  | 161644/225000 [313:24:33<87:03:55,  4.95s/it]
 72%|███████▏  | 161645/225000 [313:24:38<87:30:02,  4.97s/it]
 72%|███████▏  | 161646/225000 [313:24:48<116:08:09,  6.60s/it]
 72%|███████▏  | 161647/225000 [313:24:57<126:50:28,  7.21s/it]
 72%|███████▏  | 161648/225000 [313:25:02<116:36:55,  6.63s/it]
 72%|███████▏  | 161649/225000 [313:25:07<109:34:08,  6.23s/it]
 72%|███████▏  | 161650/225000 [313:25:14<111:16:55,  6.32s/it]
                                                               
{'loss': '0.7746', 'grad_norm': '1.043', 'learning_rate': '9.989e-06', 'epoch': '0.7184', 'num_input_tokens_seen': 518803952, 'train_runtime': '1.128e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161650/225000 [313:25:14<111:16:55,  6.32s/it]
 72%|███████▏  | 161651/225000 [313:25:19<102:56:13,  5.85s/it]
 72%|███████▏  | 161652/225000 [313:25:22<89:08:41,  5.07s/it] 
 72%|███████▏  | 161653/225000 [313:25:31<109:08:12,  6.20s/it]
 72%|███████▏  | 161654/225000 [313:25:37<106:27:46,  6.05s/it]
 72%|███████▏  | 161655/225000 [313:25:41<99:22:48,  5.65s/it] 
 72%|███████▏  | 161656/225000 [313:25:46<93:32:54,  5.32s/it]
 72%|███████▏  | 161657/225000 [313:25:51<90:27:49,  5.14s/it]
 72%|███████▏  | 161658/225000 [313:25:56<90:02:58,  5.12s/it]
 72%|███████▏  | 161659/225000 [313:26:00<86:42:26,  4.93s/it]
 72%|███████▏  | 161660/225000 [313:26:06<89:28:00,  5.08s/it]
 72%|███████▏  | 161661/225000 [313:26:15<111:33:08,  6.34s/it]
 72%|███████▏  | 161662/225000 [313:26:19<100:59:50,  5.74s/it]
 72%|███████▏  | 161663/225000 [313:26:23<93:48:44,  5.33s/it] 
 72%|███████▏  | 161664/225000 [313:26:30<98:14:58,  5.58s/it]
 72%|███████▏  | 161665/225000 [313:26:39<115:54:35,  6.59s/it]
 72%|███████▏  | 161666/225000 [313:26:42<99:45:12,  5.67s/it] 
 72%|███████▏  | 161667/225000 [313:26:50<111:11:06,  6.32s/it]
 72%|███████▏  | 161668/225000 [313:26:54<99:32:58,  5.66s/it] 
 72%|███████▏  | 161669/225000 [313:26:59<93:18:38,  5.30s/it]
 72%|███████▏  | 161670/225000 [313:27:05<97:21:05,  5.53s/it]
 72%|███████▏  | 161671/225000 [313:27:14<119:52:28,  6.81s/it]
 72%|███████▏  | 161672/225000 [313:27:18<104:03:50,  5.92s/it]
 72%|███████▏  | 161673/225000 [313:27:22<94:28:29,  5.37s/it] 
 72%|███████▏  | 161674/225000 [313:27:31<111:49:32,  6.36s/it]
 72%|███████▏  | 161675/225000 [313:27:36<103:22:14,  5.88s/it]
 72%|███████▏  | 161676/225000 [313:27:41<97:43:01,  5.56s/it] 
 72%|███████▏  | 161677/225000 [313:27:46<96:53:27,  5.51s/it]
 72%|███████▏  | 161678/225000 [313:27:50<91:09:59,  5.18s/it]
 72%|███████▏  | 161679/225000 [313:27:55<89:14:16,  5.07s/it]
 72%|███████▏  | 161680/225000 [313:28:00<89:06:16,  5.07s/it]
 72%|███████▏  | 161681/225000 [313:28:05<89:57:03,  5.11s/it]
 72%|███████▏  | 161682/225000 [313:28:10<88:10:16,  5.01s/it]
 72%|███████▏  | 161683/225000 [313:28:22<125:21:09,  7.13s/it]
 72%|███████▏  | 161684/225000 [313:28:26<106:23:56,  6.05s/it]
 72%|███████▏  | 161685/225000 [313:28:32<107:50:19,  6.13s/it]
 72%|███████▏  | 161686/225000 [313:28:39<111:33:52,  6.34s/it]
 72%|███████▏  | 161687/225000 [313:28:45<110:09:18,  6.26s/it]
 72%|███████▏  | 161688/225000 [313:28:50<104:40:48,  5.95s/it]
 72%|███████▏  | 161689/225000 [313:28:55<99:14:35,  5.64s/it] 
 72%|███████▏  | 161690/225000 [313:29:00<93:41:39,  5.33s/it]
 72%|███████▏  | 161691/225000 [313:29:05<90:14:20,  5.13s/it]
 72%|███████▏  | 161692/225000 [313:29:09<87:50:52,  5.00s/it]
 72%|███████▏  | 161693/225000 [313:29:14<87:20:49,  4.97s/it]
 72%|███████▏  | 161694/225000 [313:29:19<85:57:07,  4.89s/it]
 72%|███████▏  | 161695/225000 [313:29:24<87:09:49,  4.96s/it]
 72%|███████▏  | 161696/225000 [313:29:29<86:43:57,  4.93s/it]
 72%|███████▏  | 161697/225000 [313:29:33<84:31:43,  4.81s/it]
 72%|███████▏  | 161698/225000 [313:29:45<122:59:55,  6.99s/it]
 72%|███████▏  | 161699/225000 [313:29:51<114:37:36,  6.52s/it]
 72%|███████▏  | 161700/225000 [313:29:54<97:10:02,  5.53s/it] 
                                                              
{'loss': '0.7692', 'grad_norm': '1.138', 'learning_rate': '9.975e-06', 'epoch': '0.7187', 'num_input_tokens_seen': 518934272, 'train_runtime': '1.129e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161700/225000 [313:29:54<97:10:02,  5.53s/it]
 72%|███████▏  | 161701/225000 [313:30:04<119:00:43,  6.77s/it]
 72%|███████▏  | 161702/225000 [313:30:07<101:21:23,  5.76s/it]
 72%|███████▏  | 161703/225000 [313:30:14<106:44:18,  6.07s/it]
 72%|███████▏  | 161704/225000 [313:30:19<103:35:25,  5.89s/it]
 72%|███████▏  | 161705/225000 [313:30:24<95:27:04,  5.43s/it] 
 72%|███████▏  | 161706/225000 [313:30:29<92:45:42,  5.28s/it]
 72%|███████▏  | 161707/225000 [313:30:33<89:13:45,  5.08s/it]
 72%|███████▏  | 161708/225000 [313:30:38<84:57:29,  4.83s/it]
 72%|███████▏  | 161709/225000 [313:30:49<117:36:45,  6.69s/it]
 72%|███████▏  | 161710/225000 [313:30:52<100:36:20,  5.72s/it]
 72%|███████▏  | 161711/225000 [313:30:57<99:04:31,  5.64s/it] 
 72%|███████▏  | 161712/225000 [313:31:05<110:18:17,  6.27s/it]
 72%|███████▏  | 161713/225000 [313:31:10<100:44:29,  5.73s/it]
 72%|███████▏  | 161714/225000 [313:31:14<95:29:19,  5.43s/it] 
 72%|███████▏  | 161715/225000 [313:31:19<91:59:02,  5.23s/it]
 72%|███████▏  | 161716/225000 [313:31:24<91:35:14,  5.21s/it]
 72%|███████▏  | 161717/225000 [313:31:29<89:11:09,  5.07s/it]
 72%|███████▏  | 161718/225000 [313:31:34<87:38:27,  4.99s/it]
 72%|███████▏  | 161719/225000 [313:31:39<89:00:12,  5.06s/it]
 72%|███████▏  | 161720/225000 [313:31:50<120:23:34,  6.85s/it]
 72%|███████▏  | 161721/225000 [313:31:54<103:34:40,  5.89s/it]
 72%|███████▏  | 161722/225000 [313:31:59<102:14:31,  5.82s/it]
 72%|███████▏  | 161723/225000 [313:32:08<115:13:00,  6.56s/it]
 72%|███████▏  | 161724/225000 [313:32:11<99:08:11,  5.64s/it] 
 72%|███████▏  | 161725/225000 [313:32:19<110:25:14,  6.28s/it]
 72%|███████▏  | 161726/225000 [313:32:23<97:57:56,  5.57s/it] 
 72%|███████▏  | 161727/225000 [313:32:26<87:01:56,  4.95s/it]
 72%|███████▏  | 161728/225000 [313:32:36<111:08:45,  6.32s/it]
 72%|███████▏  | 161729/225000 [313:32:40<99:17:44,  5.65s/it] 
 72%|███████▏  | 161730/225000 [313:32:44<91:28:55,  5.21s/it]
 72%|███████▏  | 161731/225000 [313:32:49<88:04:45,  5.01s/it]
 72%|███████▏  | 161732/225000 [313:32:53<82:19:07,  4.68s/it]
 72%|███████▏  | 161733/225000 [313:32:59<90:42:21,  5.16s/it]
 72%|███████▏  | 161734/225000 [313:33:09<116:08:31,  6.61s/it]
 72%|███████▏  | 161735/225000 [313:33:13<104:39:19,  5.96s/it]
 72%|███████▏  | 161736/225000 [313:33:20<106:48:42,  6.08s/it]
 72%|███████▏  | 161737/225000 [313:33:24<96:47:27,  5.51s/it] 
 72%|███████▏  | 161738/225000 [313:33:28<90:30:01,  5.15s/it]
 72%|███████▏  | 161739/225000 [313:33:37<107:24:16,  6.11s/it]
 72%|███████▏  | 161740/225000 [313:33:45<117:43:29,  6.70s/it]
 72%|███████▏  | 161741/225000 [313:33:49<105:28:41,  6.00s/it]
 72%|███████▏  | 161742/225000 [313:33:53<97:18:43,  5.54s/it] 
 72%|███████▏  | 161743/225000 [313:33:58<92:54:26,  5.29s/it]
 72%|███████▏  | 161744/225000 [313:34:03<89:59:55,  5.12s/it]
 72%|███████▏  | 161745/225000 [313:34:07<86:38:11,  4.93s/it]
 72%|███████▏  | 161746/225000 [313:34:12<84:56:37,  4.83s/it]
 72%|███████▏  | 161747/225000 [313:34:17<84:40:02,  4.82s/it]
 72%|███████▏  | 161748/225000 [313:34:21<81:55:14,  4.66s/it]
 72%|███████▏  | 161749/225000 [313:34:33<118:01:24,  6.72s/it]
 72%|███████▏  | 161750/225000 [313:34:39<114:01:52,  6.49s/it]
                                                               
{'loss': '0.7684', 'grad_norm': '1.225', 'learning_rate': '9.96e-06', 'epoch': '0.7189', 'num_input_tokens_seen': 519065144, 'train_runtime': '1.129e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161750/225000 [313:34:39<114:01:52,  6.49s/it]
 72%|███████▏  | 161751/225000 [313:34:45<114:40:44,  6.53s/it]
 72%|███████▏  | 161752/225000 [313:34:50<104:34:08,  5.95s/it]
 72%|███████▏  | 161753/225000 [313:34:54<94:44:21,  5.39s/it] 
 72%|███████▏  | 161754/225000 [313:34:58<89:37:43,  5.10s/it]
 72%|███████▏  | 161755/225000 [313:35:03<86:54:02,  4.95s/it]
 72%|███████▏  | 161756/225000 [313:35:07<84:09:15,  4.79s/it]
 72%|███████▏  | 161757/225000 [313:35:17<108:50:56,  6.20s/it]
 72%|███████▏  | 161758/225000 [313:35:24<112:10:58,  6.39s/it]
 72%|███████▏  | 161759/225000 [313:35:27<99:02:53,  5.64s/it] 
 72%|███████▏  | 161760/225000 [313:35:32<91:14:56,  5.19s/it]
 72%|███████▏  | 161761/225000 [313:35:42<117:17:21,  6.68s/it]
 72%|███████▏  | 161762/225000 [313:35:45<100:40:42,  5.73s/it]
 72%|███████▏  | 161763/225000 [313:35:51<100:38:31,  5.73s/it]
 72%|███████▏  | 161764/225000 [313:35:58<108:04:41,  6.15s/it]
 72%|███████▏  | 161765/225000 [313:36:03<100:32:36,  5.72s/it]
 72%|███████▏  | 161766/225000 [313:36:08<96:04:15,  5.47s/it] 
 72%|███████▏  | 161767/225000 [313:36:12<90:58:37,  5.18s/it]
 72%|███████▏  | 161768/225000 [313:36:18<92:29:30,  5.27s/it]
 72%|███████▏  | 161769/225000 [313:36:23<91:53:06,  5.23s/it]
 72%|███████▏  | 161770/225000 [313:36:28<90:22:25,  5.15s/it]
 72%|███████▏  | 161771/225000 [313:36:33<88:56:29,  5.06s/it]
 72%|███████▏  | 161772/225000 [313:36:38<89:02:49,  5.07s/it]
 72%|███████▏  | 161773/225000 [313:36:43<90:50:20,  5.17s/it]
 72%|███████▏  | 161774/225000 [313:36:48<91:02:45,  5.18s/it]
 72%|███████▏  | 161775/225000 [313:36:54<92:08:09,  5.25s/it]
 72%|███████▏  | 161776/225000 [313:37:02<109:29:20,  6.23s/it]
 72%|███████▏  | 161777/225000 [313:37:12<128:42:06,  7.33s/it]
 72%|███████▏  | 161778/225000 [313:37:15<106:13:24,  6.05s/it]
 72%|███████▏  | 161779/225000 [313:37:23<112:35:22,  6.41s/it]
 72%|███████▏  | 161780/225000 [313:37:26<98:55:38,  5.63s/it] 
 72%|███████▏  | 161781/225000 [313:37:33<105:48:11,  6.02s/it]
 72%|███████▏  | 161782/225000 [313:37:39<106:36:03,  6.07s/it]
 72%|███████▏  | 161783/225000 [313:37:43<93:56:01,  5.35s/it] 
 72%|███████▏  | 161784/225000 [313:37:50<104:26:30,  5.95s/it]
 72%|███████▏  | 161785/225000 [313:37:58<110:40:03,  6.30s/it]
 72%|███████▏  | 161786/225000 [313:38:01<97:02:03,  5.53s/it] 
 72%|███████▏  | 161787/225000 [313:38:05<87:58:37,  5.01s/it]
 72%|███████▏  | 161788/225000 [313:38:09<84:15:53,  4.80s/it]
 72%|███████▏  | 161789/225000 [313:38:15<87:59:36,  5.01s/it]
 72%|███████▏  | 161790/225000 [313:38:27<123:57:52,  7.06s/it]
 72%|███████▏  | 161791/225000 [313:38:31<109:29:41,  6.24s/it]
 72%|███████▏  | 161792/225000 [313:38:39<117:50:39,  6.71s/it]
 72%|███████▏  | 161793/225000 [313:38:43<102:02:22,  5.81s/it]
 72%|███████▏  | 161794/225000 [313:38:46<90:33:00,  5.16s/it] 
 72%|███████▏  | 161795/225000 [313:38:55<108:21:29,  6.17s/it]
 72%|███████▏  | 161796/225000 [313:38:59<98:32:39,  5.61s/it] 
 72%|███████▏  | 161797/225000 [313:39:04<93:39:48,  5.34s/it]
 72%|███████▏  | 161798/225000 [313:39:10<99:58:00,  5.69s/it]
 72%|███████▏  | 161799/225000 [313:39:14<91:49:53,  5.23s/it]
 72%|███████▏  | 161800/225000 [313:39:19<86:27:42,  4.93s/it]
                                                              
{'loss': '0.7801', 'grad_norm': '1.207', 'learning_rate': '9.946e-06', 'epoch': '0.7191', 'num_input_tokens_seen': 519195208, 'train_runtime': '1.129e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161800/225000 [313:39:19<86:27:42,  4.93s/it]
 72%|███████▏  | 161801/225000 [313:39:23<83:36:30,  4.76s/it]
 72%|███████▏  | 161802/225000 [313:39:35<121:04:48,  6.90s/it]
 72%|███████▏  | 161803/225000 [313:39:39<106:23:38,  6.06s/it]
 72%|███████▏  | 161804/225000 [313:39:45<105:27:24,  6.01s/it]
 72%|███████▏  | 161805/225000 [313:39:52<113:31:45,  6.47s/it]
 72%|███████▏  | 161806/225000 [313:39:56<100:05:34,  5.70s/it]
 72%|███████▏  | 161807/225000 [313:40:04<111:20:18,  6.34s/it]
 72%|███████▏  | 161808/225000 [313:40:08<96:49:53,  5.52s/it] 
 72%|███████▏  | 161809/225000 [313:40:15<103:18:49,  5.89s/it]
 72%|███████▏  | 161810/225000 [313:40:21<107:48:56,  6.14s/it]
 72%|███████▏  | 161811/225000 [313:40:25<94:59:52,  5.41s/it] 
 72%|███████▏  | 161812/225000 [313:40:29<87:33:24,  4.99s/it]
 72%|███████▏  | 161813/225000 [313:40:34<88:09:20,  5.02s/it]
 72%|███████▏  | 161814/225000 [313:40:39<88:47:45,  5.06s/it]
 72%|███████▏  | 161815/225000 [313:40:44<89:11:23,  5.08s/it]
 72%|███████▏  | 161816/225000 [313:40:50<91:10:45,  5.20s/it]
 72%|███████▏  | 161817/225000 [313:40:55<90:17:16,  5.14s/it]
 72%|███████▏  | 161818/225000 [313:41:00<90:58:38,  5.18s/it]
 72%|███████▏  | 161819/225000 [313:41:05<89:49:08,  5.12s/it]
 72%|███████▏  | 161820/225000 [313:41:11<91:47:20,  5.23s/it]
 72%|███████▏  | 161821/225000 [313:41:19<105:59:49,  6.04s/it]
 72%|███████▏  | 161822/225000 [313:41:30<132:32:02,  7.55s/it]
 72%|███████▏  | 161823/225000 [313:41:34<116:42:15,  6.65s/it]
 72%|███████▏  | 161824/225000 [313:41:42<121:43:07,  6.94s/it]
 72%|███████▏  | 161825/225000 [313:41:45<103:34:10,  5.90s/it]
 72%|███████▏  | 161826/225000 [313:41:50<95:36:02,  5.45s/it] 
 72%|███████▏  | 161827/225000 [313:41:55<93:05:21,  5.30s/it]
 72%|███████▏  | 161828/225000 [313:41:59<90:01:29,  5.13s/it]
 72%|███████▏  | 161829/225000 [313:42:08<110:03:21,  6.27s/it]
 72%|███████▏  | 161830/225000 [313:42:19<134:40:56,  7.68s/it]
 72%|███████▏  | 161831/225000 [313:42:22<111:05:25,  6.33s/it]
 72%|███████▏  | 161832/225000 [313:42:26<98:17:42,  5.60s/it] 
 72%|███████▏  | 161833/225000 [313:42:30<89:14:24,  5.09s/it]
 72%|███████▏  | 161834/225000 [313:42:35<85:18:56,  4.86s/it]
 72%|███████▏  | 161835/225000 [313:42:45<116:38:31,  6.65s/it]
 72%|███████▏  | 161836/225000 [313:42:49<99:55:37,  5.70s/it] 
 72%|███████▏  | 161837/225000 [313:42:54<99:02:14,  5.64s/it]
 72%|███████▏  | 161838/225000 [313:43:01<103:22:04,  5.89s/it]
 72%|███████▏  | 161839/225000 [313:43:05<96:25:55,  5.50s/it] 
 72%|███████▏  | 161840/225000 [313:43:10<89:26:00,  5.10s/it]
 72%|███████▏  | 161841/225000 [313:43:14<87:54:59,  5.01s/it]
 72%|███████▏  | 161842/225000 [313:43:24<113:16:41,  6.46s/it]
 72%|███████▏  | 161843/225000 [313:43:31<115:11:47,  6.57s/it]
 72%|███████▏  | 161844/225000 [313:43:35<100:04:11,  5.70s/it]
 72%|███████▏  | 161845/225000 [313:43:40<99:07:16,  5.65s/it] 
 72%|███████▏  | 161846/225000 [313:43:48<107:47:04,  6.14s/it]
 72%|███████▏  | 161847/225000 [313:43:51<94:47:43,  5.40s/it] 
 72%|███████▏  | 161848/225000 [313:43:55<88:30:12,  5.05s/it]
 72%|███████▏  | 161849/225000 [313:44:00<87:49:42,  5.01s/it]
 72%|███████▏  | 161850/225000 [313:44:05<86:32:24,  4.93s/it]
                                                              
{'loss': '0.7821', 'grad_norm': '1.139', 'learning_rate': '9.931e-06', 'epoch': '0.7193', 'num_input_tokens_seen': 519326312, 'train_runtime': '1.129e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161850/225000 [313:44:05<86:32:24,  4.93s/it]
 72%|███████▏  | 161851/225000 [313:44:10<86:00:32,  4.90s/it]
 72%|███████▏  | 161852/225000 [313:44:15<84:16:52,  4.80s/it]
 72%|███████▏  | 161853/225000 [313:44:19<84:11:50,  4.80s/it]
 72%|███████▏  | 161854/225000 [313:44:25<87:50:36,  5.01s/it]
 72%|███████▏  | 161855/225000 [313:44:36<122:13:22,  6.97s/it]
 72%|███████▏  | 161856/225000 [313:44:40<103:10:58,  5.88s/it]
 72%|███████▏  | 161857/225000 [313:44:47<113:09:09,  6.45s/it]
 72%|███████▏  | 161858/225000 [313:44:51<95:56:08,  5.47s/it] 
 72%|███████▏  | 161859/225000 [313:45:00<114:40:11,  6.54s/it]
 72%|███████▏  | 161860/225000 [313:45:03<99:47:22,  5.69s/it] 
 72%|███████▏  | 161861/225000 [313:45:07<90:07:58,  5.14s/it]
 72%|███████▏  | 161862/225000 [313:45:17<113:25:08,  6.47s/it]
 72%|███████▏  | 161863/225000 [313:45:21<100:30:33,  5.73s/it]
 72%|███████▏  | 161864/225000 [313:45:29<112:37:27,  6.42s/it]
 72%|███████▏  | 161865/225000 [313:45:33<101:32:53,  5.79s/it]
 72%|███████▏  | 161866/225000 [313:45:41<111:15:59,  6.34s/it]
 72%|███████▏  | 161867/225000 [313:45:44<95:10:56,  5.43s/it] 
 72%|███████▏  | 161868/225000 [313:45:49<94:25:06,  5.38s/it]
 72%|███████▏  | 161869/225000 [313:45:55<97:02:01,  5.53s/it]
 72%|███████▏  | 161870/225000 [313:45:59<88:09:19,  5.03s/it]
 72%|███████▏  | 161871/225000 [313:46:03<81:23:08,  4.64s/it]
 72%|███████▏  | 161872/225000 [313:46:07<80:09:03,  4.57s/it]
 72%|███████▏  | 161873/225000 [313:46:12<79:38:57,  4.54s/it]
 72%|███████▏  | 161874/225000 [313:46:17<84:21:53,  4.81s/it]
 72%|███████▏  | 161875/225000 [313:46:22<86:38:38,  4.94s/it]
 72%|███████▏  | 161876/225000 [313:46:27<86:55:40,  4.96s/it]
 72%|███████▏  | 161877/225000 [313:46:33<89:33:28,  5.11s/it]
 72%|███████▏  | 161878/225000 [313:46:38<91:06:31,  5.20s/it]
 72%|███████▏  | 161879/225000 [313:46:46<106:49:53,  6.09s/it]
 72%|███████▏  | 161880/225000 [313:46:55<121:23:13,  6.92s/it]
 72%|███████▏  | 161881/225000 [313:46:59<103:45:59,  5.92s/it]
 72%|███████▏  | 161882/225000 [313:47:08<121:01:51,  6.90s/it]
 72%|███████▏  | 161883/225000 [313:47:12<104:34:29,  5.96s/it]
 72%|███████▏  | 161884/225000 [313:47:15<90:39:28,  5.17s/it] 
 72%|███████▏  | 161885/225000 [313:47:25<114:42:39,  6.54s/it]
 72%|███████▏  | 161886/225000 [313:47:28<96:36:11,  5.51s/it] 
 72%|███████▏  | 161887/225000 [313:47:31<85:14:11,  4.86s/it]
 72%|███████▏  | 161888/225000 [313:47:35<78:48:33,  4.50s/it]
 72%|███████▏  | 161889/225000 [313:47:39<77:57:36,  4.45s/it]
 72%|███████▏  | 161890/225000 [313:47:51<118:03:35,  6.73s/it]
 72%|███████▏  | 161891/225000 [313:47:55<103:48:47,  5.92s/it]
 72%|███████▏  | 161892/225000 [313:47:59<92:49:02,  5.29s/it] 
 72%|███████▏  | 161893/225000 [313:48:03<84:37:05,  4.83s/it]
 72%|███████▏  | 161894/225000 [313:48:14<115:16:33,  6.58s/it]
 72%|███████▏  | 161895/225000 [313:48:17<100:08:02,  5.71s/it]
 72%|███████▏  | 161896/225000 [313:48:25<108:58:21,  6.22s/it]
 72%|███████▏  | 161897/225000 [313:48:29<100:26:14,  5.73s/it]
 72%|███████▏  | 161898/225000 [313:48:34<94:20:15,  5.38s/it] 
 72%|███████▏  | 161899/225000 [313:48:38<88:51:45,  5.07s/it]
 72%|███████▏  | 161900/225000 [313:48:43<88:28:04,  5.05s/it]
                                                              
{'loss': '0.7798', 'grad_norm': '1.197', 'learning_rate': '9.917e-06', 'epoch': '0.7196', 'num_input_tokens_seen': 519455840, 'train_runtime': '1.13e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161900/225000 [313:48:43<88:28:04,  5.05s/it]
 72%|███████▏  | 161901/225000 [313:48:48<85:52:50,  4.90s/it]
 72%|███████▏  | 161902/225000 [313:49:00<123:56:59,  7.07s/it]
 72%|███████▏  | 161903/225000 [313:49:04<107:03:57,  6.11s/it]
 72%|███████▏  | 161904/225000 [313:49:07<93:56:35,  5.36s/it] 
 72%|███████▏  | 161905/225000 [313:49:17<116:10:38,  6.63s/it]
 72%|███████▏  | 161906/225000 [313:49:21<100:20:33,  5.73s/it]
 72%|███████▏  | 161907/225000 [313:49:27<104:54:57,  5.99s/it]
 72%|███████▏  | 161908/225000 [313:49:33<102:05:22,  5.83s/it]
 72%|███████▏  | 161909/225000 [313:49:36<89:48:09,  5.12s/it] 
 72%|███████▏  | 161910/225000 [313:49:41<89:37:49,  5.11s/it]
 72%|███████▏  | 161911/225000 [313:49:47<90:50:29,  5.18s/it]
 72%|███████▏  | 161912/225000 [313:49:52<91:18:23,  5.21s/it]
 72%|███████▏  | 161913/225000 [313:49:57<90:22:18,  5.16s/it]
 72%|███████▏  | 161914/225000 [313:50:03<92:37:27,  5.29s/it]
 72%|███████▏  | 161915/225000 [313:50:08<92:26:37,  5.28s/it]
 72%|███████▏  | 161916/225000 [313:50:13<91:46:50,  5.24s/it]
 72%|███████▏  | 161917/225000 [313:50:18<92:51:09,  5.30s/it]
 72%|███████▏  | 161918/225000 [313:50:24<92:32:39,  5.28s/it]
 72%|███████▏  | 161919/225000 [313:50:29<91:50:20,  5.24s/it]
 72%|███████▏  | 161920/225000 [313:50:34<91:27:45,  5.22s/it]
 72%|███████▏  | 161921/225000 [313:50:42<106:16:13,  6.06s/it]
 72%|███████▏  | 161922/225000 [313:50:52<124:47:57,  7.12s/it]
 72%|███████▏  | 161923/225000 [313:50:55<106:09:25,  6.06s/it]
 72%|███████▏  | 161924/225000 [313:51:00<100:52:56,  5.76s/it]
 72%|███████▏  | 161925/225000 [313:51:05<95:27:55,  5.45s/it] 
 72%|███████▏  | 161926/225000 [313:51:11<96:35:13,  5.51s/it]
 72%|███████▏  | 161927/225000 [313:51:16<96:54:53,  5.53s/it]
 72%|███████▏  | 161928/225000 [313:51:21<95:47:51,  5.47s/it]
 72%|███████▏  | 161929/225000 [313:51:27<93:40:42,  5.35s/it]
 72%|███████▏  | 161930/225000 [313:51:32<94:06:40,  5.37s/it]
 72%|███████▏  | 161931/225000 [313:51:38<95:02:16,  5.42s/it]
 72%|███████▏  | 161932/225000 [313:51:45<107:42:59,  6.15s/it]
 72%|███████▏  | 161933/225000 [313:51:52<110:07:06,  6.29s/it]
 72%|███████▏  | 161934/225000 [313:51:56<97:02:44,  5.54s/it] 
 72%|███████▏  | 161935/225000 [313:52:05<116:08:54,  6.63s/it]
 72%|███████▏  | 161936/225000 [313:52:08<98:29:45,  5.62s/it] 
 72%|███████▏  | 161937/225000 [313:52:13<91:43:12,  5.24s/it]
 72%|███████▏  | 161938/225000 [313:52:17<89:01:01,  5.08s/it]
 72%|███████▏  | 161939/225000 [313:52:22<87:20:30,  4.99s/it]
 72%|███████▏  | 161940/225000 [313:52:27<85:01:23,  4.85s/it]
 72%|███████▏  | 161941/225000 [313:52:39<122:23:38,  6.99s/it]
 72%|███████▏  | 161942/225000 [313:52:42<105:48:27,  6.04s/it]
 72%|███████▏  | 161943/225000 [313:52:46<94:36:23,  5.40s/it] 
 72%|███████▏  | 161944/225000 [313:52:54<109:15:48,  6.24s/it]
 72%|███████▏  | 161945/225000 [313:53:01<111:21:51,  6.36s/it]
 72%|███████▏  | 161946/225000 [313:53:06<102:49:43,  5.87s/it]
 72%|███████▏  | 161947/225000 [313:53:11<98:12:18,  5.61s/it] 
 72%|███████▏  | 161948/225000 [313:53:15<93:02:27,  5.31s/it]
 72%|███████▏  | 161949/225000 [313:53:20<90:12:37,  5.15s/it]
 72%|███████▏  | 161950/225000 [313:53:25<86:02:09,  4.91s/it]
                                                              
{'loss': '0.7677', 'grad_norm': '1.11', 'learning_rate': '9.902e-06', 'epoch': '0.7198', 'num_input_tokens_seen': 519585672, 'train_runtime': '1.13e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 161950/225000 [313:53:25<86:02:09,  4.91s/it]
 72%|███████▏  | 161951/225000 [313:53:29<83:16:41,  4.76s/it]
 72%|███████▏  | 161952/225000 [313:53:36<96:33:56,  5.51s/it]
 72%|███████▏  | 161953/225000 [313:53:46<120:13:31,  6.86s/it]
 72%|███████▏  | 161954/225000 [313:53:50<103:57:38,  5.94s/it]
 72%|███████▏  | 161955/225000 [313:53:53<90:19:27,  5.16s/it] 
 72%|███████▏  | 161956/225000 [313:54:04<118:21:03,  6.76s/it]
 72%|███████▏  | 161957/225000 [313:54:10<112:48:40,  6.44s/it]
 72%|███████▏  | 161958/225000 [313:54:14<104:26:44,  5.96s/it]
 72%|███████▏  | 161959/225000 [313:54:19<97:35:19,  5.57s/it] 
 72%|███████▏  | 161960/225000 [313:54:24<93:43:54,  5.35s/it]
 72%|███████▏  | 161961/225000 [313:54:29<91:19:28,  5.22s/it]
 72%|███████▏  | 161962/225000 [313:54:34<90:25:45,  5.16s/it]
 72%|███████▏  | 161963/225000 [313:54:39<88:35:05,  5.06s/it]
 72%|███████▏  | 161964/225000 [313:54:43<85:18:06,  4.87s/it]
 72%|███████▏  | 161965/225000 [313:54:48<86:17:01,  4.93s/it]
 72%|███████▏  | 161966/225000 [313:54:53<87:11:22,  4.98s/it]
 72%|███████▏  | 161967/225000 [313:54:58<86:07:20,  4.92s/it]
 72%|███████▏  | 161968/225000 [313:55:08<112:44:50,  6.44s/it]
 72%|███████▏  | 161969/225000 [313:55:16<120:27:53,  6.88s/it]
 72%|███████▏  | 161970/225000 [313:55:20<105:40:01,  6.04s/it]
 72%|███████▏  | 161971/225000 [313:55:24<92:44:50,  5.30s/it] 
 72%|███████▏  | 161972/225000 [313:55:28<89:52:42,  5.13s/it]
 72%|███████▏  | 161973/225000 [313:55:40<122:12:46,  6.98s/it]
 72%|███████▏  | 161974/225000 [313:55:43<102:56:33,  5.88s/it]
 72%|███████▏  | 161975/225000 [313:55:50<110:22:40,  6.30s/it]
 72%|███████▏  | 161976/225000 [313:55:57<111:04:43,  6.34s/it]
 72%|███████▏  | 161977/225000 [313:56:01<100:41:15,  5.75s/it]
 72%|███████▏  | 161978/225000 [313:56:09<113:09:56,  6.46s/it]
 72%|███████▏  | 161979/225000 [313:56:13<101:16:09,  5.78s/it]
 72%|███████▏  | 161980/225000 [313:56:17<91:45:29,  5.24s/it] 
 72%|███████▏  | 161981/225000 [313:56:21<85:40:02,  4.89s/it]
 72%|███████▏  | 161982/225000 [313:56:26<84:30:55,  4.83s/it]
 72%|███████▏  | 161983/225000 [313:56:38<119:56:08,  6.85s/it]
 72%|███████▏  | 161984/225000 [313:56:41<103:36:54,  5.92s/it]
 72%|███████▏  | 161985/225000 [313:56:45<93:35:57,  5.35s/it] 
 72%|███████▏  | 161986/225000 [313:56:55<113:57:19,  6.51s/it]
 72%|███████▏  | 161987/225000 [313:56:58<96:18:11,  5.50s/it] 
 72%|███████▏  | 161988/225000 [313:57:06<111:40:40,  6.38s/it]
 72%|███████▏  | 161989/225000 [313:57:10<97:15:14,  5.56s/it] 
 72%|███████▏  | 161990/225000 [313:57:13<86:50:45,  4.96s/it]
 72%|███████▏  | 161991/225000 [313:57:17<81:26:08,  4.65s/it]
 72%|███████▏  | 161992/225000 [313:57:26<102:12:21,  5.84s/it]
 72%|███████▏  | 161993/225000 [313:57:32<105:45:40,  6.04s/it]
 72%|███████▏  | 161994/225000 [313:57:39<109:10:05,  6.24s/it]
 72%|███████▏  | 161995/225000 [313:57:43<94:38:23,  5.41s/it] 
 72%|███████▏  | 161996/225000 [313:57:51<110:30:47,  6.31s/it]
 72%|███████▏  | 161997/225000 [313:57:55<100:30:12,  5.74s/it]
 72%|███████▏  | 161998/225000 [313:58:00<94:57:01,  5.43s/it] 
 72%|███████▏  | 161999/225000 [313:58:05<91:24:29,  5.22s/it]
 72%|███████▏  | 162000/225000 [313:58:10<88:08:12,  5.04s/it]
                                                              
{'loss': '0.7705', 'grad_norm': '1.245', 'learning_rate': '9.887e-06', 'epoch': '0.72', 'num_input_tokens_seen': 519716000, 'train_runtime': '1.13e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162000/225000 [313:58:10<88:08:12,  5.04s/it]
 72%|███████▏  | 162001/225000 [313:58:14<86:04:40,  4.92s/it]
 72%|███████▏  | 162002/225000 [313:58:19<84:23:26,  4.82s/it]
 72%|███████▏  | 162003/225000 [313:58:31<125:04:13,  7.15s/it]
 72%|███████▏  | 162004/225000 [313:58:36<109:23:37,  6.25s/it]
 72%|███████▏  | 162005/225000 [313:58:39<95:18:03,  5.45s/it] 
 72%|███████▏  | 162006/225000 [313:58:44<93:59:23,  5.37s/it]
 72%|███████▏  | 162007/225000 [313:58:50<93:28:26,  5.34s/it]
 72%|███████▏  | 162008/225000 [313:58:55<93:08:14,  5.32s/it]
 72%|███████▏  | 162009/225000 [313:59:00<91:42:46,  5.24s/it]
 72%|███████▏  | 162010/225000 [313:59:05<93:07:05,  5.32s/it]
 72%|███████▏  | 162011/225000 [313:59:10<90:49:06,  5.19s/it]
 72%|███████▏  | 162012/225000 [313:59:21<119:50:20,  6.85s/it]
 72%|███████▏  | 162013/225000 [313:59:25<106:19:21,  6.08s/it]
 72%|███████▏  | 162014/225000 [313:59:31<105:42:23,  6.04s/it]
 72%|███████▏  | 162015/225000 [313:59:35<93:00:44,  5.32s/it] 
 72%|███████▏  | 162016/225000 [313:59:39<86:24:05,  4.94s/it]
 72%|███████▏  | 162017/225000 [313:59:51<124:49:54,  7.14s/it]
 72%|███████▏  | 162018/225000 [313:59:55<105:09:13,  6.01s/it]
 72%|███████▏  | 162019/225000 [314:00:01<106:51:35,  6.11s/it]
 72%|███████▏  | 162020/225000 [314:00:06<101:07:23,  5.78s/it]
 72%|███████▏  | 162021/225000 [314:00:10<94:01:49,  5.37s/it] 
 72%|███████▏  | 162022/225000 [314:00:15<88:16:19,  5.05s/it]
 72%|███████▏  | 162023/225000 [314:00:19<85:47:16,  4.90s/it]
 72%|███████▏  | 162024/225000 [314:00:24<83:03:32,  4.75s/it]
 72%|███████▏  | 162025/225000 [314:00:28<82:40:17,  4.73s/it]
 72%|███████▏  | 162026/225000 [314:00:41<123:24:29,  7.05s/it]
 72%|███████▏  | 162027/225000 [314:00:44<105:59:51,  6.06s/it]
 72%|███████▏  | 162028/225000 [314:00:48<92:46:28,  5.30s/it] 
 72%|███████▏  | 162029/225000 [314:00:53<89:10:49,  5.10s/it]
 72%|███████▏  | 162030/225000 [314:00:58<90:07:24,  5.15s/it]
 72%|███████▏  | 162031/225000 [314:01:04<93:05:07,  5.32s/it]
 72%|███████▏  | 162032/225000 [314:01:09<94:27:15,  5.40s/it]
 72%|███████▏  | 162033/225000 [314:01:15<95:05:16,  5.44s/it]
 72%|███████▏  | 162034/225000 [314:01:20<95:31:03,  5.46s/it]
 72%|███████▏  | 162035/225000 [314:01:26<99:25:19,  5.68s/it]
 72%|███████▏  | 162036/225000 [314:01:35<113:23:10,  6.48s/it]
 72%|███████▏  | 162037/225000 [314:01:39<98:50:10,  5.65s/it] 
 72%|███████▏  | 162038/225000 [314:01:42<88:10:23,  5.04s/it]
 72%|███████▏  | 162039/225000 [314:01:52<112:11:23,  6.41s/it]
 72%|███████▏  | 162040/225000 [314:01:56<98:53:07,  5.65s/it] 
 72%|███████▏  | 162041/225000 [314:01:59<88:52:25,  5.08s/it]
 72%|███████▏  | 162042/225000 [314:02:04<84:56:07,  4.86s/it]
 72%|███████▏  | 162043/225000 [314:02:16<123:33:54,  7.07s/it]
 72%|███████▏  | 162044/225000 [314:02:19<103:34:41,  5.92s/it]
 72%|███████▏  | 162045/225000 [314:02:27<113:06:36,  6.47s/it]
 72%|███████▏  | 162046/225000 [314:02:30<97:11:25,  5.56s/it] 
 72%|███████▏  | 162047/225000 [314:02:38<110:47:50,  6.34s/it]
 72%|███████▏  | 162048/225000 [314:02:42<97:40:17,  5.59s/it] 
 72%|███████▏  | 162049/225000 [314:02:46<86:18:21,  4.94s/it]
 72%|███████▏  | 162050/225000 [314:02:57<118:35:40,  6.78s/it]
                                                               
{'loss': '0.7671', 'grad_norm': '0.9691', 'learning_rate': '9.873e-06', 'epoch': '0.7202', 'num_input_tokens_seen': 519846112, 'train_runtime': '1.131e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162050/225000 [314:02:57<118:35:40,  6.78s/it]
 72%|███████▏  | 162051/225000 [314:03:00<101:12:42,  5.79s/it]
 72%|███████▏  | 162052/225000 [314:03:03<86:43:26,  4.96s/it] 
 72%|███████▏  | 162053/225000 [314:03:11<99:14:29,  5.68s/it]
 72%|███████▏  | 162054/225000 [314:03:16<94:52:45,  5.43s/it]
 72%|███████▏  | 162055/225000 [314:03:22<98:09:41,  5.61s/it]
 72%|███████▏  | 162056/225000 [314:03:25<88:09:13,  5.04s/it]
 72%|███████▏  | 162057/225000 [314:03:30<85:19:33,  4.88s/it]
 72%|███████▏  | 162058/225000 [314:03:35<84:50:32,  4.85s/it]
 72%|███████▏  | 162059/225000 [314:03:40<88:53:48,  5.08s/it]
 72%|███████▏  | 162060/225000 [314:03:45<89:04:37,  5.09s/it]
 72%|███████▏  | 162061/225000 [314:03:50<87:16:55,  4.99s/it]
 72%|███████▏  | 162062/225000 [314:03:57<98:03:11,  5.61s/it]
 72%|███████▏  | 162063/225000 [314:04:09<130:36:35,  7.47s/it]
 72%|███████▏  | 162064/225000 [314:04:13<111:16:07,  6.36s/it]
 72%|███████▏  | 162065/225000 [314:04:17<98:04:33,  5.61s/it] 
 72%|███████▏  | 162066/225000 [314:04:24<107:16:19,  6.14s/it]
 72%|███████▏  | 162067/225000 [314:04:30<109:06:44,  6.24s/it]
 72%|███████▏  | 162068/225000 [314:04:35<99:47:19,  5.71s/it] 
 72%|███████▏  | 162069/225000 [314:04:39<90:19:54,  5.17s/it]
 72%|███████▏  | 162070/225000 [314:04:43<85:30:59,  4.89s/it]
 72%|███████▏  | 162071/225000 [314:04:50<97:59:52,  5.61s/it]
 72%|███████▏  | 162072/225000 [314:04:59<112:25:27,  6.43s/it]
 72%|███████▏  | 162073/225000 [314:05:03<101:10:04,  5.79s/it]
 72%|███████▏  | 162074/225000 [314:05:07<91:04:52,  5.21s/it] 
 72%|███████▏  | 162075/225000 [314:05:11<86:06:39,  4.93s/it]
 72%|███████▏  | 162076/225000 [314:05:21<110:10:03,  6.30s/it]
 72%|███████▏  | 162077/225000 [314:05:27<110:30:19,  6.32s/it]
 72%|███████▏  | 162078/225000 [314:05:31<100:06:54,  5.73s/it]
 72%|███████▏  | 162079/225000 [314:05:35<91:51:23,  5.26s/it] 
 72%|███████▏  | 162080/225000 [314:05:47<126:24:15,  7.23s/it]
 72%|███████▏  | 162081/225000 [314:05:51<105:25:27,  6.03s/it]
 72%|███████▏  | 162082/225000 [314:05:54<92:14:12,  5.28s/it] 
 72%|███████▏  | 162083/225000 [314:06:04<116:36:06,  6.67s/it]
 72%|███████▏  | 162084/225000 [314:06:07<99:25:18,  5.69s/it] 
 72%|███████▏  | 162085/225000 [314:06:15<107:17:56,  6.14s/it]
 72%|███████▏  | 162086/225000 [314:06:21<109:48:31,  6.28s/it]
 72%|███████▏  | 162087/225000 [314:06:25<98:15:07,  5.62s/it] 
 72%|███████▏  | 162088/225000 [314:06:29<89:08:37,  5.10s/it]
 72%|███████▏  | 162089/225000 [314:06:33<84:58:49,  4.86s/it]
 72%|███████▏  | 162090/225000 [314:06:38<81:19:48,  4.65s/it]
 72%|███████▏  | 162091/225000 [314:06:45<94:31:21,  5.41s/it]
 72%|███████▏  | 162092/225000 [314:06:55<120:41:05,  6.91s/it]
 72%|███████▏  | 162093/225000 [314:06:58<101:30:05,  5.81s/it]
 72%|███████▏  | 162094/225000 [314:07:03<93:59:23,  5.38s/it] 
 72%|███████▏  | 162095/225000 [314:07:07<88:34:40,  5.07s/it]
 72%|███████▏  | 162096/225000 [314:07:16<110:37:32,  6.33s/it]
 72%|███████▏  | 162097/225000 [314:07:20<96:36:47,  5.53s/it] 
 72%|███████▏  | 162098/225000 [314:07:24<87:29:43,  5.01s/it]
 72%|███████▏  | 162099/225000 [314:07:29<89:50:03,  5.14s/it]
 72%|███████▏  | 162100/225000 [314:07:38<106:54:36,  6.12s/it]
                                                               
{'loss': '0.7788', 'grad_norm': '1.119', 'learning_rate': '9.858e-06', 'epoch': '0.7204', 'num_input_tokens_seen': 519976312, 'train_runtime': '1.131e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162100/225000 [314:07:38<106:54:36,  6.12s/it]
 72%|███████▏  | 162101/225000 [314:07:43<101:15:16,  5.80s/it]
 72%|███████▏  | 162102/225000 [314:07:48<96:50:06,  5.54s/it] 
 72%|███████▏  | 162103/225000 [314:07:53<94:25:09,  5.40s/it]
 72%|███████▏  | 162104/225000 [314:07:58<92:58:42,  5.32s/it]
 72%|███████▏  | 162105/225000 [314:08:03<90:02:50,  5.15s/it]
 72%|███████▏  | 162106/225000 [314:08:08<90:51:41,  5.20s/it]
 72%|███████▏  | 162107/225000 [314:08:13<91:28:02,  5.24s/it]
 72%|███████▏  | 162108/225000 [314:08:18<90:42:03,  5.19s/it]
 72%|███████▏  | 162109/225000 [314:08:24<90:40:03,  5.19s/it]
 72%|███████▏  | 162110/225000 [314:08:29<90:15:49,  5.17s/it]
 72%|███████▏  | 162111/225000 [314:08:36<102:36:47,  5.87s/it]
 72%|███████▏  | 162112/225000 [314:08:41<98:02:06,  5.61s/it] 
 72%|███████▏  | 162113/225000 [314:08:46<94:38:21,  5.42s/it]
 72%|███████▏  | 162114/225000 [314:08:51<91:25:20,  5.23s/it]
 72%|███████▏  | 162115/225000 [314:08:56<91:42:13,  5.25s/it]
 72%|███████▏  | 162116/225000 [314:09:01<90:51:20,  5.20s/it]
 72%|███████▏  | 162117/225000 [314:09:07<91:13:52,  5.22s/it]
 72%|███████▏  | 162118/225000 [314:09:12<92:11:00,  5.28s/it]
 72%|███████▏  | 162119/225000 [314:09:17<92:02:05,  5.27s/it]
 72%|███████▏  | 162120/225000 [314:09:22<91:30:35,  5.24s/it]
 72%|███████▏  | 162121/225000 [314:09:28<90:33:23,  5.18s/it]
 72%|███████▏  | 162122/225000 [314:09:33<91:31:17,  5.24s/it]
 72%|███████▏  | 162123/225000 [314:09:43<116:35:01,  6.67s/it]
 72%|███████▏  | 162124/225000 [314:09:51<123:07:51,  7.05s/it]
 72%|███████▏  | 162125/225000 [314:09:55<105:28:39,  6.04s/it]
 72%|███████▏  | 162126/225000 [314:10:02<111:05:40,  6.36s/it]
 72%|███████▏  | 162127/225000 [314:10:09<115:41:43,  6.62s/it]
 72%|███████▏  | 162128/225000 [314:10:13<103:32:38,  5.93s/it]
 72%|███████▏  | 162129/225000 [314:10:17<93:32:48,  5.36s/it] 
 72%|███████▏  | 162130/225000 [314:10:22<88:49:21,  5.09s/it]
 72%|███████▏  | 162131/225000 [314:10:26<84:59:35,  4.87s/it]
 72%|███████▏  | 162132/225000 [314:10:39<127:01:58,  7.27s/it]
 72%|███████▏  | 162133/225000 [314:10:42<107:17:50,  6.14s/it]
 72%|███████▏  | 162134/225000 [314:10:47<99:32:31,  5.70s/it] 
 72%|███████▏  | 162135/225000 [314:10:52<92:47:01,  5.31s/it]
 72%|███████▏  | 162136/225000 [314:10:56<88:09:15,  5.05s/it]
 72%|███████▏  | 162137/225000 [314:11:01<87:01:26,  4.98s/it]
 72%|███████▏  | 162138/225000 [314:11:08<98:47:24,  5.66s/it]
 72%|███████▏  | 162139/225000 [314:11:17<117:19:51,  6.72s/it]
 72%|███████▏  | 162140/225000 [314:11:21<104:18:34,  5.97s/it]
 72%|███████▏  | 162141/225000 [314:11:26<95:20:58,  5.46s/it] 
 72%|███████▏  | 162142/225000 [314:11:30<89:08:30,  5.11s/it]
 72%|███████▏  | 162143/225000 [314:11:34<83:48:22,  4.80s/it]
 72%|███████▏  | 162144/225000 [314:11:38<79:51:26,  4.57s/it]
 72%|███████▏  | 162145/225000 [314:11:51<120:55:08,  6.93s/it]
 72%|███████▏  | 162146/225000 [314:11:54<102:11:26,  5.85s/it]
 72%|███████▏  | 162147/225000 [314:12:01<111:09:17,  6.37s/it]
 72%|███████▏  | 162148/225000 [314:12:05<97:35:54,  5.59s/it] 
 72%|███████▏  | 162149/225000 [314:12:09<87:39:53,  5.02s/it]
 72%|███████▏  | 162150/225000 [314:12:18<107:14:49,  6.14s/it]
                                                               
{'loss': '0.7747', 'grad_norm': '1.073', 'learning_rate': '9.844e-06', 'epoch': '0.7207', 'num_input_tokens_seen': 520105800, 'train_runtime': '1.131e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162150/225000 [314:12:18<107:14:49,  6.14s/it]
 72%|███████▏  | 162151/225000 [314:12:21<93:07:34,  5.33s/it] 
 72%|███████▏  | 162152/225000 [314:12:26<88:18:36,  5.06s/it]
 72%|███████▏  | 162153/225000 [314:12:30<85:37:42,  4.90s/it]
 72%|███████▏  | 162154/225000 [314:12:35<86:23:28,  4.95s/it]
 72%|███████▏  | 162155/225000 [314:12:40<86:39:22,  4.96s/it]
 72%|███████▏  | 162156/225000 [314:12:45<86:17:10,  4.94s/it]
 72%|███████▏  | 162157/225000 [314:12:50<87:46:11,  5.03s/it]
 72%|███████▏  | 162158/225000 [314:12:56<89:27:41,  5.12s/it]
 72%|███████▏  | 162159/225000 [314:13:01<91:47:37,  5.26s/it]
 72%|███████▏  | 162160/225000 [314:13:14<132:42:17,  7.60s/it]
 72%|███████▏  | 162161/225000 [314:13:18<112:42:38,  6.46s/it]
 72%|███████▏  | 162162/225000 [314:13:26<121:35:50,  6.97s/it]
 72%|███████▏  | 162163/225000 [314:13:30<103:28:07,  5.93s/it]
 72%|███████▏  | 162164/225000 [314:13:33<92:03:53,  5.27s/it] 
 72%|███████▏  | 162165/225000 [314:13:41<106:40:03,  6.11s/it]
 72%|███████▏  | 162166/225000 [314:13:47<105:16:23,  6.03s/it]
 72%|███████▏  | 162167/225000 [314:13:51<93:15:38,  5.34s/it] 
 72%|███████▏  | 162168/225000 [314:13:55<86:31:39,  4.96s/it]
 72%|███████▏  | 162169/225000 [314:14:00<83:38:23,  4.79s/it]
 72%|███████▏  | 162170/225000 [314:14:04<83:34:24,  4.79s/it]
 72%|███████▏  | 162171/225000 [314:14:09<82:07:30,  4.71s/it]
 72%|███████▏  | 162172/225000 [314:14:14<83:15:18,  4.77s/it]
 72%|███████▏  | 162173/225000 [314:14:22<103:44:19,  5.94s/it]
 72%|███████▏  | 162174/225000 [314:14:31<117:21:57,  6.73s/it]
 72%|███████▏  | 162175/225000 [314:14:36<108:27:10,  6.21s/it]
 72%|███████▏  | 162176/225000 [314:14:41<100:28:08,  5.76s/it]
 72%|███████▏  | 162177/225000 [314:14:46<95:40:20,  5.48s/it] 
 72%|███████▏  | 162178/225000 [314:14:50<90:42:36,  5.20s/it]
 72%|███████▏  | 162179/225000 [314:14:55<89:23:50,  5.12s/it]
 72%|███████▏  | 162180/225000 [314:15:00<88:54:04,  5.09s/it]
 72%|███████▏  | 162181/225000 [314:15:05<87:05:20,  4.99s/it]
 72%|███████▏  | 162182/225000 [314:15:10<88:56:22,  5.10s/it]
 72%|███████▏  | 162183/225000 [314:15:15<89:42:55,  5.14s/it]
 72%|███████▏  | 162184/225000 [314:15:20<87:51:04,  5.03s/it]
 72%|███████▏  | 162185/225000 [314:15:30<115:20:33,  6.61s/it]
 72%|███████▏  | 162186/225000 [314:15:38<117:46:11,  6.75s/it]
 72%|███████▏  | 162187/225000 [314:15:45<120:02:38,  6.88s/it]
 72%|███████▏  | 162188/225000 [314:15:49<108:55:50,  6.24s/it]
 72%|███████▏  | 162189/225000 [314:15:54<100:51:46,  5.78s/it]
 72%|███████▏  | 162190/225000 [314:15:59<95:19:08,  5.46s/it] 
 72%|███████▏  | 162191/225000 [314:16:03<89:56:42,  5.16s/it]
 72%|███████▏  | 162192/225000 [314:16:08<87:48:38,  5.03s/it]
 72%|███████▏  | 162193/225000 [314:16:14<93:19:16,  5.35s/it]
 72%|███████▏  | 162194/225000 [314:16:26<128:12:59,  7.35s/it]
 72%|███████▏  | 162195/225000 [314:16:29<106:46:23,  6.12s/it]
 72%|███████▏  | 162196/225000 [314:16:37<113:30:14,  6.51s/it]
 72%|███████▏  | 162197/225000 [314:16:42<105:36:15,  6.05s/it]
 72%|███████▏  | 162198/225000 [314:16:47<98:42:39,  5.66s/it] 
 72%|███████▏  | 162199/225000 [314:16:52<94:54:59,  5.44s/it]
 72%|███████▏  | 162200/225000 [314:16:56<88:13:06,  5.06s/it]
                                                              
{'loss': '0.7732', 'grad_norm': '1.25', 'learning_rate': '9.829e-06', 'epoch': '0.7209', 'num_input_tokens_seen': 520236592, 'train_runtime': '1.131e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162200/225000 [314:16:56<88:13:06,  5.06s/it]
 72%|███████▏  | 162201/225000 [314:17:00<84:58:13,  4.87s/it]
 72%|███████▏  | 162202/225000 [314:17:05<85:46:16,  4.92s/it]
 72%|███████▏  | 162203/225000 [314:17:09<82:28:30,  4.73s/it]
 72%|███████▏  | 162204/225000 [314:17:14<80:58:31,  4.64s/it]
 72%|███████▏  | 162205/225000 [314:17:19<84:19:02,  4.83s/it]
 72%|███████▏  | 162206/225000 [314:17:26<92:39:35,  5.31s/it]
 72%|███████▏  | 162207/225000 [314:17:37<123:27:34,  7.08s/it]
 72%|███████▏  | 162208/225000 [314:17:40<101:59:26,  5.85s/it]
 72%|███████▏  | 162209/225000 [314:17:47<110:01:54,  6.31s/it]
 72%|███████▏  | 162210/225000 [314:17:51<99:35:37,  5.71s/it] 
 72%|███████▏  | 162211/225000 [314:17:55<89:22:14,  5.12s/it]
 72%|███████▏  | 162212/225000 [314:18:00<86:40:05,  4.97s/it]
 72%|███████▏  | 162213/225000 [314:18:08<103:48:49,  5.95s/it]
 72%|███████▏  | 162214/225000 [314:18:15<106:36:05,  6.11s/it]
 72%|███████▏  | 162215/225000 [314:18:18<93:11:18,  5.34s/it] 
 72%|███████▏  | 162216/225000 [314:18:28<116:24:22,  6.67s/it]
 72%|███████▏  | 162217/225000 [314:18:32<101:48:29,  5.84s/it]
 72%|███████▏  | 162218/225000 [314:18:35<90:01:30,  5.16s/it] 
 72%|███████▏  | 162219/225000 [314:18:46<121:11:23,  6.95s/it]
 72%|███████▏  | 162220/225000 [314:18:50<102:41:50,  5.89s/it]
 72%|███████▏  | 162221/225000 [314:18:55<100:37:53,  5.77s/it]
 72%|███████▏  | 162222/225000 [314:19:00<97:04:38,  5.57s/it] 
 72%|███████▏  | 162223/225000 [314:19:06<95:05:57,  5.45s/it]
 72%|███████▏  | 162224/225000 [314:19:11<96:06:00,  5.51s/it]
 72%|███████▏  | 162225/225000 [314:19:17<96:13:03,  5.52s/it]
 72%|███████▏  | 162226/225000 [314:19:25<110:58:26,  6.36s/it]
 72%|███████▏  | 162227/225000 [314:19:28<94:57:50,  5.45s/it] 
 72%|███████▏  | 162228/225000 [314:19:35<101:31:27,  5.82s/it]
 72%|███████▏  | 162229/225000 [314:19:39<92:23:59,  5.30s/it] 
 72%|███████▏  | 162230/225000 [314:19:47<104:52:02,  6.01s/it]
 72%|███████▏  | 162231/225000 [314:19:50<91:09:39,  5.23s/it] 
 72%|███████▏  | 162232/225000 [314:19:54<83:19:50,  4.78s/it]
 72%|███████▏  | 162233/225000 [314:20:04<112:14:54,  6.44s/it]
 72%|███████▏  | 162234/225000 [314:20:08<97:28:34,  5.59s/it] 
 72%|███████▏  | 162235/225000 [314:20:12<87:07:28,  5.00s/it]
 72%|███████▏  | 162236/225000 [314:20:17<91:42:19,  5.26s/it]
 72%|███████▏  | 162237/225000 [314:20:26<110:22:41,  6.33s/it]
 72%|███████▏  | 162238/225000 [314:20:31<102:00:29,  5.85s/it]
 72%|███████▏  | 162239/225000 [314:20:35<93:48:03,  5.38s/it] 
 72%|███████▏  | 162240/225000 [314:20:40<88:44:43,  5.09s/it]
 72%|███████▏  | 162241/225000 [314:20:44<85:11:58,  4.89s/it]
 72%|███████▏  | 162242/225000 [314:20:49<83:31:25,  4.79s/it]
 72%|███████▏  | 162243/225000 [314:20:53<82:21:28,  4.72s/it]
 72%|███████▏  | 162244/225000 [314:21:06<122:30:31,  7.03s/it]
 72%|███████▏  | 162245/225000 [314:21:10<110:53:29,  6.36s/it]
 72%|███████▏  | 162246/225000 [314:21:14<96:53:53,  5.56s/it] 
 72%|███████▏  | 162247/225000 [314:21:19<94:25:25,  5.42s/it]
 72%|███████▏  | 162248/225000 [314:21:24<93:02:42,  5.34s/it]
 72%|███████▏  | 162249/225000 [314:21:29<89:03:30,  5.11s/it]
 72%|███████▏  | 162250/225000 [314:21:35<92:07:58,  5.29s/it]
                                                              
{'loss': '0.7867', 'grad_norm': '1.421', 'learning_rate': '9.815e-06', 'epoch': '0.7211', 'num_input_tokens_seen': 520366864, 'train_runtime': '1.132e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162250/225000 [314:21:35<92:07:58,  5.29s/it]
 72%|███████▏  | 162251/225000 [314:21:40<91:05:04,  5.23s/it]
 72%|███████▏  | 162252/225000 [314:21:45<90:10:28,  5.17s/it]
 72%|███████▏  | 162253/225000 [314:21:50<91:20:08,  5.24s/it]
 72%|███████▏  | 162254/225000 [314:21:56<93:29:26,  5.36s/it]
 72%|███████▏  | 162255/225000 [314:22:01<94:10:17,  5.40s/it]
 72%|███████▏  | 162256/225000 [314:22:07<94:38:37,  5.43s/it]
 72%|███████▏  | 162257/225000 [314:22:12<95:07:04,  5.46s/it]
 72%|███████▏  | 162258/225000 [314:22:18<93:32:09,  5.37s/it]
 72%|███████▏  | 162259/225000 [314:22:23<93:34:13,  5.37s/it]
 72%|███████▏  | 162260/225000 [314:22:31<108:34:43,  6.23s/it]
 72%|███████▏  | 162261/225000 [314:22:42<134:55:27,  7.74s/it]
 72%|███████▏  | 162262/225000 [314:22:46<112:16:00,  6.44s/it]
 72%|███████▏  | 162263/225000 [314:22:50<101:22:13,  5.82s/it]
 72%|███████▏  | 162264/225000 [314:22:54<93:24:47,  5.36s/it] 
 72%|███████▏  | 162265/225000 [314:22:59<88:36:22,  5.08s/it]
 72%|███████▏  | 162266/225000 [314:23:05<94:34:56,  5.43s/it]
 72%|███████▏  | 162267/225000 [314:23:15<116:07:58,  6.66s/it]
 72%|███████▏  | 162268/225000 [314:23:20<106:58:04,  6.14s/it]
 72%|███████▏  | 162269/225000 [314:23:25<102:50:05,  5.90s/it]
 72%|███████▏  | 162270/225000 [314:23:30<95:49:45,  5.50s/it] 
 72%|███████▏  | 162271/225000 [314:23:35<94:10:25,  5.40s/it]
 72%|███████▏  | 162272/225000 [314:23:40<92:35:00,  5.31s/it]
 72%|███████▏  | 162273/225000 [314:23:44<86:30:01,  4.96s/it]
 72%|███████▏  | 162274/225000 [314:23:49<84:53:26,  4.87s/it]
 72%|███████▏  | 162275/225000 [314:23:54<86:28:46,  4.96s/it]
 72%|███████▏  | 162276/225000 [314:23:59<86:04:17,  4.94s/it]
 72%|███████▏  | 162277/225000 [314:24:03<85:21:13,  4.90s/it]
 72%|███████▏  | 162278/225000 [314:24:09<87:07:39,  5.00s/it]
 72%|███████▏  | 162279/225000 [314:24:14<88:15:34,  5.07s/it]
 72%|███████▏  | 162280/225000 [314:24:19<87:30:50,  5.02s/it]
 72%|███████▏  | 162281/225000 [314:24:29<115:06:27,  6.61s/it]
 72%|███████▏  | 162282/225000 [314:24:39<132:15:12,  7.59s/it]
 72%|███████▏  | 162283/225000 [314:24:43<111:04:16,  6.38s/it]
 72%|███████▏  | 162284/225000 [314:24:49<108:45:55,  6.24s/it]
 72%|███████▏  | 162285/225000 [314:24:53<99:12:06,  5.69s/it] 
 72%|███████▏  | 162286/225000 [314:25:00<106:35:03,  6.12s/it]
 72%|███████▏  | 162287/225000 [314:25:04<93:20:30,  5.36s/it] 
 72%|███████▏  | 162288/225000 [314:25:12<107:51:41,  6.19s/it]
 72%|███████▏  | 162289/225000 [314:25:15<94:29:46,  5.42s/it] 
 72%|███████▏  | 162290/225000 [314:25:19<87:01:42,  5.00s/it]
 72%|███████▏  | 162291/225000 [314:25:28<105:28:22,  6.05s/it]
 72%|███████▏  | 162292/225000 [314:25:31<91:00:05,  5.22s/it] 
 72%|███████▏  | 162293/225000 [314:25:40<107:19:20,  6.16s/it]
 72%|███████▏  | 162294/225000 [314:25:45<101:53:42,  5.85s/it]
 72%|███████▏  | 162295/225000 [314:25:50<97:36:34,  5.60s/it] 
 72%|███████▏  | 162296/225000 [314:25:54<92:42:47,  5.32s/it]
 72%|███████▏  | 162297/225000 [314:26:00<92:14:11,  5.30s/it]
 72%|███████▏  | 162298/225000 [314:26:05<90:15:40,  5.18s/it]
 72%|███████▏  | 162299/225000 [314:26:09<88:44:06,  5.09s/it]
 72%|███████▏  | 162300/225000 [314:26:15<88:55:52,  5.11s/it]
                                                              
{'loss': '0.7822', 'grad_norm': '1.219', 'learning_rate': '9.8e-06', 'epoch': '0.7213', 'num_input_tokens_seen': 520497792, 'train_runtime': '1.132e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162300/225000 [314:26:15<88:55:52,  5.11s/it]
 72%|███████▏  | 162301/225000 [314:26:20<91:04:43,  5.23s/it]
 72%|███████▏  | 162302/225000 [314:26:25<89:39:58,  5.15s/it]
 72%|███████▏  | 162303/225000 [314:26:30<88:56:53,  5.11s/it]
 72%|███████▏  | 162304/225000 [314:26:37<100:40:25,  5.78s/it]
 72%|███████▏  | 162305/225000 [314:26:49<131:07:44,  7.53s/it]
 72%|███████▏  | 162306/225000 [314:26:53<111:25:10,  6.40s/it]
 72%|███████▏  | 162307/225000 [314:26:57<97:51:47,  5.62s/it] 
 72%|███████▏  | 162308/225000 [314:27:05<113:22:20,  6.51s/it]
 72%|███████▏  | 162309/225000 [314:27:08<96:49:35,  5.56s/it] 
 72%|███████▏  | 162310/225000 [314:27:16<106:18:36,  6.10s/it]
 72%|███████▏  | 162311/225000 [314:27:19<90:34:08,  5.20s/it] 
 72%|███████▏  | 162312/225000 [314:27:28<110:22:15,  6.34s/it]
 72%|███████▏  | 162313/225000 [314:27:31<95:19:13,  5.47s/it] 
 72%|███████▏  | 162314/225000 [314:27:37<94:30:43,  5.43s/it]
 72%|███████▏  | 162315/225000 [314:27:43<98:14:02,  5.64s/it]
 72%|███████▏  | 162316/225000 [314:27:47<89:08:00,  5.12s/it]
 72%|███████▏  | 162317/225000 [314:27:51<82:04:08,  4.71s/it]
 72%|███████▏  | 162318/225000 [314:28:01<112:30:19,  6.46s/it]
 72%|███████▏  | 162319/225000 [314:28:05<100:11:46,  5.75s/it]
 72%|███████▏  | 162320/225000 [314:28:09<92:13:48,  5.30s/it] 
 72%|███████▏  | 162321/225000 [314:28:14<87:12:20,  5.01s/it]
 72%|███████▏  | 162322/225000 [314:28:18<83:31:37,  4.80s/it]
 72%|███████▏  | 162323/225000 [314:28:30<119:46:56,  6.88s/it]
 72%|███████▏  | 162324/225000 [314:28:34<104:56:03,  6.03s/it]
 72%|███████▏  | 162325/225000 [314:28:37<91:43:36,  5.27s/it] 
 72%|███████▏  | 162326/225000 [314:28:46<108:15:05,  6.22s/it]
 72%|███████▏  | 162327/225000 [314:28:50<99:11:34,  5.70s/it] 
 72%|███████▏  | 162328/225000 [314:28:55<93:30:43,  5.37s/it]
 72%|███████▏  | 162329/225000 [314:29:00<90:29:35,  5.20s/it]
 72%|███████▏  | 162330/225000 [314:29:04<85:36:42,  4.92s/it]
 72%|███████▏  | 162331/225000 [314:29:08<82:58:16,  4.77s/it]
 72%|███████▏  | 162332/225000 [314:29:13<81:28:05,  4.68s/it]
 72%|███████▏  | 162333/225000 [314:29:17<81:02:59,  4.66s/it]
 72%|███████▏  | 162334/225000 [314:29:22<81:08:48,  4.66s/it]
 72%|███████▏  | 162335/225000 [314:29:30<99:29:05,  5.72s/it]
 72%|███████▏  | 162336/225000 [314:29:40<122:57:24,  7.06s/it]
 72%|███████▏  | 162337/225000 [314:29:44<104:25:14,  6.00s/it]
 72%|███████▏  | 162338/225000 [314:29:48<95:55:39,  5.51s/it] 
 72%|███████▏  | 162339/225000 [314:29:58<116:51:18,  6.71s/it]
 72%|███████▏  | 162340/225000 [314:30:02<101:52:00,  5.85s/it]
 72%|███████▏  | 162341/225000 [314:30:06<92:44:11,  5.33s/it] 
 72%|███████▏  | 162342/225000 [314:30:15<111:11:15,  6.39s/it]
 72%|███████▏  | 162343/225000 [314:30:20<104:35:58,  6.01s/it]
 72%|███████▏  | 162344/225000 [314:30:25<101:11:01,  5.81s/it]
 72%|███████▏  | 162345/225000 [314:30:29<90:22:03,  5.19s/it] 
 72%|███████▏  | 162346/225000 [314:30:38<110:52:20,  6.37s/it]
 72%|███████▏  | 162347/225000 [314:30:41<95:26:45,  5.48s/it] 
 72%|███████▏  | 162348/225000 [314:30:45<86:28:05,  4.97s/it]
 72%|███████▏  | 162349/225000 [314:30:49<81:19:00,  4.67s/it]
 72%|███████▏  | 162350/225000 [314:30:56<91:21:28,  5.25s/it]
                                                              
{'loss': '0.7743', 'grad_norm': '1.08', 'learning_rate': '9.786e-06', 'epoch': '0.7216', 'num_input_tokens_seen': 520628312, 'train_runtime': '1.132e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162350/225000 [314:30:56<91:21:28,  5.25s/it]
 72%|███████▏  | 162351/225000 [314:31:06<116:30:31,  6.69s/it]
 72%|███████▏  | 162352/225000 [314:31:10<101:18:15,  5.82s/it]
 72%|███████▏  | 162353/225000 [314:31:13<89:41:08,  5.15s/it] 
 72%|███████▏  | 162354/225000 [314:31:22<110:12:03,  6.33s/it]
 72%|███████▏  | 162355/225000 [314:31:26<95:41:47,  5.50s/it] 
 72%|███████▏  | 162356/225000 [314:31:33<105:10:39,  6.04s/it]
 72%|███████▏  | 162357/225000 [314:31:38<96:26:46,  5.54s/it] 
 72%|███████▏  | 162358/225000 [314:31:42<89:27:51,  5.14s/it]
 72%|███████▏  | 162359/225000 [314:31:46<86:45:20,  4.99s/it]
 72%|███████▏  | 162360/225000 [314:31:51<83:37:04,  4.81s/it]
 72%|███████▏  | 162361/225000 [314:32:02<119:07:38,  6.85s/it]
 72%|███████▏  | 162362/225000 [314:32:06<104:05:08,  5.98s/it]
 72%|███████▏  | 162363/225000 [314:32:10<91:24:46,  5.25s/it] 
 72%|███████▏  | 162364/225000 [314:32:14<85:21:12,  4.91s/it]
 72%|███████▏  | 162365/225000 [314:32:24<111:42:02,  6.42s/it]
 72%|███████▏  | 162366/225000 [314:32:29<105:55:15,  6.09s/it]
 72%|███████▏  | 162367/225000 [314:32:32<90:40:26,  5.21s/it] 
 72%|███████▏  | 162368/225000 [314:32:40<103:53:15,  5.97s/it]
 72%|███████▏  | 162369/225000 [314:32:45<96:15:30,  5.53s/it] 
 72%|███████▏  | 162370/225000 [314:32:50<93:27:45,  5.37s/it]
 72%|███████▏  | 162371/225000 [314:32:55<91:43:07,  5.27s/it]
 72%|███████▏  | 162372/225000 [314:33:00<91:26:13,  5.26s/it]
 72%|███████▏  | 162373/225000 [314:33:05<89:52:46,  5.17s/it]
 72%|███████▏  | 162374/225000 [314:33:10<88:48:59,  5.11s/it]
 72%|███████▏  | 162375/225000 [314:33:15<89:41:44,  5.16s/it]
 72%|███████▏  | 162376/225000 [314:33:20<89:35:37,  5.15s/it]
 72%|███████▏  | 162377/225000 [314:33:25<88:25:50,  5.08s/it]
 72%|███████▏  | 162378/225000 [314:33:30<89:08:03,  5.12s/it]
 72%|███████▏  | 162379/225000 [314:33:36<90:12:23,  5.19s/it]
 72%|███████▏  | 162380/225000 [314:33:40<87:14:09,  5.02s/it]
 72%|███████▏  | 162381/225000 [314:33:46<87:52:45,  5.05s/it]
 72%|███████▏  | 162382/225000 [314:33:53<98:21:54,  5.66s/it]
 72%|███████▏  | 162383/225000 [314:34:05<133:19:56,  7.67s/it]
 72%|███████▏  | 162384/225000 [314:34:08<110:59:41,  6.38s/it]
 72%|███████▏  | 162385/225000 [314:34:16<116:19:26,  6.69s/it]
 72%|███████▏  | 162386/225000 [314:34:21<109:20:28,  6.29s/it]
 72%|███████▏  | 162387/225000 [314:34:25<94:36:21,  5.44s/it] 
 72%|███████▏  | 162388/225000 [314:34:31<101:22:26,  5.83s/it]
 72%|███████▏  | 162389/225000 [314:34:35<89:09:10,  5.13s/it] 
 72%|███████▏  | 162390/225000 [314:34:39<82:35:46,  4.75s/it]
 72%|███████▏  | 162391/225000 [314:34:47<101:48:22,  5.85s/it]
 72%|███████▏  | 162392/225000 [314:34:52<99:41:55,  5.73s/it] 
 72%|███████▏  | 162393/225000 [314:34:58<98:25:34,  5.66s/it]
 72%|███████▏  | 162394/225000 [314:35:02<89:34:21,  5.15s/it]
 72%|███████▏  | 162395/225000 [314:35:11<108:45:32,  6.25s/it]
 72%|███████▏  | 162396/225000 [314:35:18<112:17:28,  6.46s/it]
 72%|███████▏  | 162397/225000 [314:35:21<96:33:33,  5.55s/it] 
 72%|███████▏  | 162398/225000 [314:35:26<93:20:40,  5.37s/it]
 72%|███████▏  | 162399/225000 [314:35:33<103:42:43,  5.96s/it]
 72%|███████▏  | 162400/225000 [314:35:38<95:07:39,  5.47s/it] 
                                                              
{'loss': '0.7682', 'grad_norm': '1.145', 'learning_rate': '9.771e-06', 'epoch': '0.7218', 'num_input_tokens_seen': 520757872, 'train_runtime': '1.133e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162400/225000 [314:35:38<95:07:39,  5.47s/it]
 72%|███████▏  | 162401/225000 [314:35:44<98:09:13,  5.64s/it]
 72%|███████▏  | 162402/225000 [314:35:48<88:11:24,  5.07s/it]
 72%|███████▏  | 162403/225000 [314:35:52<82:41:11,  4.76s/it]
 72%|███████▏  | 162404/225000 [314:35:56<81:31:47,  4.69s/it]
 72%|███████▏  | 162405/225000 [314:36:04<98:34:20,  5.67s/it]
 72%|███████▏  | 162406/225000 [314:36:12<112:21:39,  6.46s/it]
 72%|███████▏  | 162407/225000 [314:36:16<97:46:17,  5.62s/it] 
 72%|███████▏  | 162408/225000 [314:36:23<106:40:19,  6.14s/it]
 72%|███████▏  | 162409/225000 [314:36:28<98:41:13,  5.68s/it] 
 72%|███████▏  | 162410/225000 [314:36:33<95:35:42,  5.50s/it]
 72%|███████▏  | 162411/225000 [314:36:38<90:11:35,  5.19s/it]
 72%|███████▏  | 162412/225000 [314:36:42<88:00:32,  5.06s/it]
 72%|███████▏  | 162413/225000 [314:36:48<89:29:25,  5.15s/it]
 72%|███████▏  | 162414/225000 [314:36:53<88:22:44,  5.08s/it]
 72%|███████▏  | 162415/225000 [314:36:57<86:22:57,  4.97s/it]
 72%|███████▏  | 162416/225000 [314:37:02<85:05:18,  4.89s/it]
 72%|███████▏  | 162417/225000 [314:37:11<106:20:32,  6.12s/it]
 72%|███████▏  | 162418/225000 [314:37:19<116:45:48,  6.72s/it]
 72%|███████▏  | 162419/225000 [314:37:24<105:59:53,  6.10s/it]
 72%|███████▏  | 162420/225000 [314:37:29<103:23:19,  5.95s/it]
 72%|███████▏  | 162421/225000 [314:37:35<100:12:37,  5.76s/it]
 72%|███████▏  | 162422/225000 [314:37:40<97:36:26,  5.62s/it] 
 72%|███████▏  | 162423/225000 [314:37:45<95:02:14,  5.47s/it]
 72%|███████▏  | 162424/225000 [314:37:50<92:51:20,  5.34s/it]
 72%|███████▏  | 162425/225000 [314:37:56<93:11:23,  5.36s/it]
 72%|███████▏  | 162426/225000 [314:38:01<93:21:06,  5.37s/it]
 72%|███████▏  | 162427/225000 [314:38:06<92:27:49,  5.32s/it]
 72%|███████▏  | 162428/225000 [314:38:12<93:35:36,  5.38s/it]
 72%|███████▏  | 162429/225000 [314:38:17<93:15:02,  5.37s/it]
 72%|███████▏  | 162430/225000 [314:38:23<94:49:17,  5.46s/it]
 72%|███████▏  | 162431/225000 [314:38:28<91:51:59,  5.29s/it]
 72%|███████▏  | 162432/225000 [314:38:33<91:50:46,  5.28s/it]
 72%|███████▏  | 162433/225000 [314:38:38<91:49:27,  5.28s/it]
 72%|███████▏  | 162434/225000 [314:38:43<90:19:59,  5.20s/it]
 72%|███████▏  | 162435/225000 [314:38:49<92:16:19,  5.31s/it]
 72%|███████▏  | 162436/225000 [314:38:54<93:20:21,  5.37s/it]
 72%|███████▏  | 162437/225000 [314:39:02<107:58:52,  6.21s/it]
 72%|███████▏  | 162438/225000 [314:39:13<129:04:44,  7.43s/it]
 72%|███████▏  | 162439/225000 [314:39:18<117:50:10,  6.78s/it]
 72%|███████▏  | 162440/225000 [314:39:22<106:20:20,  6.12s/it]
 72%|███████▏  | 162441/225000 [314:39:28<101:58:22,  5.87s/it]
 72%|███████▏  | 162442/225000 [314:39:33<99:38:27,  5.73s/it] 
 72%|███████▏  | 162443/225000 [314:39:38<95:30:28,  5.50s/it]
 72%|███████▏  | 162444/225000 [314:39:43<93:02:51,  5.35s/it]
 72%|███████▏  | 162445/225000 [314:39:49<93:21:46,  5.37s/it]
 72%|███████▏  | 162446/225000 [314:39:53<90:32:30,  5.21s/it]
 72%|███████▏  | 162447/225000 [314:39:59<91:17:24,  5.25s/it]
 72%|███████▏  | 162448/225000 [314:40:04<92:09:24,  5.30s/it]
 72%|███████▏  | 162449/225000 [314:40:10<92:43:53,  5.34s/it]
 72%|███████▏  | 162450/225000 [314:40:15<91:13:24,  5.25s/it]
                                                              
{'loss': '0.7792', 'grad_norm': '1.064', 'learning_rate': '9.757e-06', 'epoch': '0.722', 'num_input_tokens_seen': 520888896, 'train_runtime': '1.133e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162450/225000 [314:40:15<91:13:24,  5.25s/it]
 72%|███████▏  | 162451/225000 [314:40:27<128:04:27,  7.37s/it]
 72%|███████▏  | 162452/225000 [314:40:32<113:33:36,  6.54s/it]
 72%|███████▏  | 162453/225000 [314:40:37<109:17:59,  6.29s/it]
 72%|███████▏  | 162454/225000 [314:40:45<115:32:33,  6.65s/it]
 72%|███████▏  | 162455/225000 [314:40:49<101:09:21,  5.82s/it]
 72%|███████▏  | 162456/225000 [314:40:53<91:42:45,  5.28s/it] 
 72%|███████▏  | 162457/225000 [314:41:00<101:56:29,  5.87s/it]
 72%|███████▏  | 162458/225000 [314:41:06<105:57:29,  6.10s/it]
 72%|███████▏  | 162459/225000 [314:41:10<93:12:14,  5.37s/it] 
 72%|███████▏  | 162460/225000 [314:41:19<112:06:39,  6.45s/it]
 72%|███████▏  | 162461/225000 [314:41:23<96:24:20,  5.55s/it] 
 72%|███████▏  | 162462/225000 [314:41:27<89:46:36,  5.17s/it]
 72%|███████▏  | 162463/225000 [314:41:31<83:57:22,  4.83s/it]
 72%|███████▏  | 162464/225000 [314:41:35<82:08:53,  4.73s/it]
 72%|███████▏  | 162465/225000 [314:41:40<79:29:39,  4.58s/it]
 72%|███████▏  | 162466/225000 [314:41:44<78:13:23,  4.50s/it]
 72%|███████▏  | 162467/225000 [314:41:50<86:38:00,  4.99s/it]
 72%|███████▏  | 162468/225000 [314:42:01<118:25:14,  6.82s/it]
 72%|███████▏  | 162469/225000 [314:42:04<99:03:17,  5.70s/it] 
 72%|███████▏  | 162470/225000 [314:42:12<109:32:03,  6.31s/it]
 72%|███████▏  | 162471/225000 [314:42:16<95:23:32,  5.49s/it] 
 72%|███████▏  | 162472/225000 [314:42:19<87:07:58,  5.02s/it]
 72%|███████▏  | 162473/225000 [314:42:28<103:45:11,  5.97s/it]
 72%|███████▏  | 162474/225000 [314:42:31<90:57:37,  5.24s/it] 
 72%|███████▏  | 162475/225000 [314:42:36<86:19:39,  4.97s/it]
 72%|███████▏  | 162476/225000 [314:42:40<82:02:36,  4.72s/it]
 72%|███████▏  | 162477/225000 [314:42:44<80:50:34,  4.65s/it]
 72%|███████▏  | 162478/225000 [314:42:50<85:10:08,  4.90s/it]
 72%|███████▏  | 162479/225000 [314:43:00<115:22:40,  6.64s/it]
 72%|███████▏  | 162480/225000 [314:43:04<101:07:46,  5.82s/it]
 72%|███████▏  | 162481/225000 [314:43:08<90:28:39,  5.21s/it] 
 72%|███████▏  | 162482/225000 [314:43:13<86:51:07,  5.00s/it]
 72%|███████▏  | 162483/225000 [314:43:17<83:27:59,  4.81s/it]
 72%|███████▏  | 162484/225000 [314:43:24<96:51:15,  5.58s/it]
 72%|███████▏  | 162485/225000 [314:43:34<119:27:09,  6.88s/it]
 72%|███████▏  | 162486/225000 [314:43:38<103:38:55,  5.97s/it]
 72%|███████▏  | 162487/225000 [314:43:42<90:38:23,  5.22s/it] 
 72%|███████▏  | 162488/225000 [314:43:46<88:35:02,  5.10s/it]
 72%|███████▏  | 162489/225000 [314:43:57<116:14:40,  6.69s/it]
 72%|███████▏  | 162490/225000 [314:44:00<100:37:40,  5.80s/it]
 72%|███████▏  | 162491/225000 [314:44:10<119:11:05,  6.86s/it]
 72%|███████▏  | 162492/225000 [314:44:13<101:40:16,  5.86s/it]
 72%|███████▏  | 162493/225000 [314:44:18<92:55:10,  5.35s/it] 
 72%|███████▏  | 162494/225000 [314:44:24<97:17:30,  5.60s/it]
 72%|███████▏  | 162495/225000 [314:44:27<85:33:15,  4.93s/it]
 72%|███████▏  | 162496/225000 [314:44:36<105:03:34,  6.05s/it]
 72%|███████▏  | 162497/225000 [314:44:39<92:30:58,  5.33s/it] 
 72%|███████▏  | 162498/225000 [314:44:48<110:20:22,  6.36s/it]
 72%|███████▏  | 162499/225000 [314:44:54<106:51:57,  6.16s/it]
 72%|███████▏  | 162500/225000 [314:44:58<95:17:31,  5.49s/it] 
                                                              
{'loss': '0.7772', 'grad_norm': '1.085', 'learning_rate': '9.742e-06', 'epoch': '0.7222', 'num_input_tokens_seen': 521019184, 'train_runtime': '1.133e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162500/225000 [314:44:58<95:17:31,  5.49s/it]
 72%|███████▏  | 162501/225000 [314:45:05<106:56:21,  6.16s/it]
 72%|███████▏  | 162502/225000 [314:45:12<107:36:54,  6.20s/it]
 72%|███████▏  | 162503/225000 [314:45:15<92:14:12,  5.31s/it] 
 72%|███████▏  | 162504/225000 [314:45:21<96:06:15,  5.54s/it]
 72%|███████▏  | 162505/225000 [314:45:24<83:50:21,  4.83s/it]
 72%|███████▏  | 162506/225000 [314:45:29<84:21:06,  4.86s/it]
 72%|███████▏  | 162507/225000 [314:45:34<83:06:39,  4.79s/it]
 72%|███████▏  | 162508/225000 [314:45:39<83:53:06,  4.83s/it]
 72%|███████▏  | 162509/225000 [314:45:43<82:30:49,  4.75s/it]
 72%|███████▏  | 162510/225000 [314:45:51<96:11:12,  5.54s/it]
 72%|███████▏  | 162511/225000 [314:46:00<117:32:05,  6.77s/it]
 72%|███████▏  | 162512/225000 [314:46:04<100:14:12,  5.77s/it]
 72%|███████▏  | 162513/225000 [314:46:07<89:30:39,  5.16s/it] 
 72%|███████▏  | 162514/225000 [314:46:12<85:26:58,  4.92s/it]
 72%|███████▏  | 162515/225000 [314:46:16<82:37:40,  4.76s/it]
 72%|███████▏  | 162516/225000 [314:46:21<83:11:54,  4.79s/it]
 72%|███████▏  | 162517/225000 [314:46:29<101:53:20,  5.87s/it]
 72%|███████▏  | 162518/225000 [314:46:38<118:02:53,  6.80s/it]
 72%|███████▏  | 162519/225000 [314:46:42<98:52:50,  5.70s/it] 
 72%|███████▏  | 162520/225000 [314:46:48<100:07:54,  5.77s/it]
 72%|███████▏  | 162521/225000 [314:46:55<111:30:44,  6.43s/it]
 72%|███████▏  | 162522/225000 [314:46:59<96:14:55,  5.55s/it] 
 72%|███████▏  | 162523/225000 [314:47:07<108:23:37,  6.25s/it]
 72%|███████▏  | 162524/225000 [314:47:10<93:18:32,  5.38s/it] 
 72%|███████▏  | 162525/225000 [314:47:19<110:45:19,  6.38s/it]
 72%|███████▏  | 162526/225000 [314:47:23<97:14:51,  5.60s/it] 
 72%|███████▏  | 162527/225000 [314:47:26<87:11:52,  5.02s/it]
 72%|███████▏  | 162528/225000 [314:47:36<111:48:23,  6.44s/it]
 72%|███████▏  | 162529/225000 [314:47:40<96:39:41,  5.57s/it] 
 72%|███████▏  | 162530/225000 [314:47:48<109:56:54,  6.34s/it]
 72%|███████▏  | 162531/225000 [314:47:52<98:26:19,  5.67s/it] 
 72%|███████▏  | 162532/225000 [314:47:56<91:24:11,  5.27s/it]
 72%|███████▏  | 162533/225000 [314:48:00<84:49:32,  4.89s/it]
 72%|███████▏  | 162534/225000 [314:48:05<82:11:32,  4.74s/it]
 72%|███████▏  | 162535/225000 [314:48:09<81:08:36,  4.68s/it]
 72%|███████▏  | 162536/225000 [314:48:21<116:22:06,  6.71s/it]
 72%|███████▏  | 162537/225000 [314:48:26<107:54:35,  6.22s/it]
 72%|███████▏  | 162538/225000 [314:48:33<113:40:53,  6.55s/it]
 72%|███████▏  | 162539/225000 [314:48:38<103:21:09,  5.96s/it]
 72%|███████▏  | 162540/225000 [314:48:42<97:08:53,  5.60s/it] 
 72%|███████▏  | 162541/225000 [314:48:47<92:01:04,  5.30s/it]
 72%|███████▏  | 162542/225000 [314:48:52<89:48:16,  5.18s/it]
 72%|███████▏  | 162543/225000 [314:48:57<87:34:28,  5.05s/it]
 72%|███████▏  | 162544/225000 [314:49:01<85:42:40,  4.94s/it]
 72%|███████▏  | 162545/225000 [314:49:06<85:59:46,  4.96s/it]
 72%|███████▏  | 162546/225000 [314:49:11<84:08:41,  4.85s/it]
 72%|███████▏  | 162547/225000 [314:49:16<83:19:23,  4.80s/it]
 72%|███████▏  | 162548/225000 [314:49:25<105:21:13,  6.07s/it]
 72%|███████▏  | 162549/225000 [314:49:35<128:29:55,  7.41s/it]
 72%|███████▏  | 162550/225000 [314:49:39<109:06:19,  6.29s/it]
                                                               
{'loss': '0.7797', 'grad_norm': '1.102', 'learning_rate': '9.728e-06', 'epoch': '0.7224', 'num_input_tokens_seen': 521149720, 'train_runtime': '1.133e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162550/225000 [314:49:39<109:06:19,  6.29s/it]
 72%|███████▏  | 162551/225000 [314:49:45<110:47:21,  6.39s/it]
 72%|███████▏  | 162552/225000 [314:49:49<95:36:20,  5.51s/it] 
 72%|███████▏  | 162553/225000 [314:49:58<113:10:03,  6.52s/it]
 72%|███████▏  | 162554/225000 [314:50:01<95:48:11,  5.52s/it] 
 72%|███████▏  | 162555/225000 [314:50:09<107:39:01,  6.21s/it]
 72%|███████▏  | 162556/225000 [314:50:12<93:52:00,  5.41s/it] 
 72%|███████▏  | 162557/225000 [314:50:18<93:25:59,  5.39s/it]
 72%|███████▏  | 162558/225000 [314:50:25<103:51:21,  5.99s/it]
 72%|███████▏  | 162559/225000 [314:50:29<91:46:00,  5.29s/it] 
 72%|███████▏  | 162560/225000 [314:50:38<110:52:49,  6.39s/it]
 72%|███████▏  | 162561/225000 [314:50:42<101:04:56,  5.83s/it]
 72%|███████▏  | 162562/225000 [314:50:48<102:05:01,  5.89s/it]
 72%|███████▏  | 162563/225000 [314:50:52<91:11:52,  5.26s/it] 
 72%|███████▏  | 162564/225000 [314:51:01<112:15:36,  6.47s/it]
 72%|███████▏  | 162565/225000 [314:51:05<95:58:34,  5.53s/it] 
 72%|███████▏  | 162566/225000 [314:51:09<89:30:53,  5.16s/it]
 72%|███████▏  | 162567/225000 [314:51:17<102:36:44,  5.92s/it]
 72%|███████▏  | 162568/225000 [314:51:20<89:41:14,  5.17s/it] 
 72%|███████▏  | 162569/225000 [314:51:28<103:14:30,  5.95s/it]
 72%|███████▏  | 162570/225000 [314:51:32<92:49:33,  5.35s/it] 
 72%|███████▏  | 162571/225000 [314:51:35<83:36:38,  4.82s/it]
 72%|███████▏  | 162572/225000 [314:51:44<102:36:35,  5.92s/it]
 72%|███████▏  | 162573/225000 [314:51:47<89:53:54,  5.18s/it] 
 72%|███████▏  | 162574/225000 [314:51:55<105:28:32,  6.08s/it]
 72%|███████▏  | 162575/225000 [314:52:00<95:09:26,  5.49s/it] 
 72%|███████▏  | 162576/225000 [314:52:04<89:16:29,  5.15s/it]
 72%|███████▏  | 162577/225000 [314:52:08<83:00:22,  4.79s/it]
 72%|███████▏  | 162578/225000 [314:52:12<80:45:36,  4.66s/it]
 72%|███████▏  | 162579/225000 [314:52:20<97:32:03,  5.63s/it]
 72%|███████▏  | 162580/225000 [314:52:29<114:38:56,  6.61s/it]
 72%|███████▏  | 162581/225000 [314:52:32<97:53:01,  5.65s/it] 
 72%|███████▏  | 162582/225000 [314:52:41<112:42:01,  6.50s/it]
 72%|███████▏  | 162583/225000 [314:52:44<97:09:47,  5.60s/it] 
 72%|███████▏  | 162584/225000 [314:52:53<113:01:33,  6.52s/it]
 72%|███████▏  | 162585/225000 [314:52:57<99:49:07,  5.76s/it] 
 72%|███████▏  | 162586/225000 [314:53:01<90:54:22,  5.24s/it]
 72%|███████▏  | 162587/225000 [314:53:05<84:23:38,  4.87s/it]
 72%|███████▏  | 162588/225000 [314:53:09<79:20:39,  4.58s/it]
 72%|███████▏  | 162589/225000 [314:53:20<114:23:32,  6.60s/it]
 72%|███████▏  | 162590/225000 [314:53:24<99:29:28,  5.74s/it] 
 72%|███████▏  | 162591/225000 [314:53:30<101:13:21,  5.84s/it]
 72%|███████▏  | 162592/225000 [314:53:34<90:18:14,  5.21s/it] 
 72%|███████▏  | 162593/225000 [314:53:39<90:41:48,  5.23s/it]
 72%|███████▏  | 162594/225000 [314:53:47<105:09:23,  6.07s/it]
 72%|███████▏  | 162595/225000 [314:53:51<95:22:36,  5.50s/it] 
 72%|███████▏  | 162596/225000 [314:53:56<89:54:01,  5.19s/it]
 72%|███████▏  | 162597/225000 [314:54:00<85:18:00,  4.92s/it]
 72%|███████▏  | 162598/225000 [314:54:09<105:32:00,  6.09s/it]
 72%|███████▏  | 162599/225000 [314:54:15<105:51:48,  6.11s/it]
 72%|███████▏  | 162600/225000 [314:54:19<92:29:46,  5.34s/it] 
                                                              
{'loss': '0.7777', 'grad_norm': '1.04', 'learning_rate': '9.714e-06', 'epoch': '0.7227', 'num_input_tokens_seen': 521279640, 'train_runtime': '1.134e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162600/225000 [314:54:19<92:29:46,  5.34s/it]
 72%|███████▏  | 162601/225000 [314:54:28<111:44:53,  6.45s/it]
 72%|███████▏  | 162602/225000 [314:54:31<95:40:55,  5.52s/it] 
 72%|███████▏  | 162603/225000 [314:54:38<101:31:18,  5.86s/it]
 72%|███████▏  | 162604/225000 [314:54:43<97:13:03,  5.61s/it] 
 72%|███████▏  | 162605/225000 [314:54:46<87:36:49,  5.06s/it]
 72%|███████▏  | 162606/225000 [314:54:54<99:30:10,  5.74s/it]
 72%|███████▏  | 162607/225000 [314:55:01<107:04:41,  6.18s/it]
 72%|███████▏  | 162608/225000 [314:55:04<91:59:20,  5.31s/it] 
 72%|███████▏  | 162609/225000 [314:55:08<85:27:55,  4.93s/it]
 72%|███████▏  | 162610/225000 [314:55:12<81:19:21,  4.69s/it]
 72%|███████▏  | 162611/225000 [314:55:22<104:45:32,  6.04s/it]
 72%|███████▏  | 162612/225000 [314:55:27<100:25:44,  5.80s/it]
 72%|███████▏  | 162613/225000 [314:55:31<90:12:36,  5.21s/it] 
 72%|███████▏  | 162614/225000 [314:55:34<82:56:03,  4.79s/it]
 72%|███████▏  | 162615/225000 [314:55:44<108:16:35,  6.25s/it]
 72%|███████▏  | 162616/225000 [314:55:50<105:00:55,  6.06s/it]
 72%|███████▏  | 162617/225000 [314:55:54<94:37:27,  5.46s/it] 
 72%|███████▏  | 162618/225000 [314:55:58<88:04:54,  5.08s/it]
 72%|███████▏  | 162619/225000 [314:56:02<83:52:01,  4.84s/it]
 72%|███████▏  | 162620/225000 [314:56:08<86:49:22,  5.01s/it]
 72%|███████▏  | 162621/225000 [314:56:19<119:13:21,  6.88s/it]
 72%|███████▏  | 162622/225000 [314:56:22<101:46:17,  5.87s/it]
 72%|███████▏  | 162623/225000 [314:56:26<91:29:14,  5.28s/it] 
 72%|███████▏  | 162624/225000 [314:56:35<108:18:43,  6.25s/it]
 72%|███████▏  | 162625/225000 [314:56:40<100:19:47,  5.79s/it]
 72%|███████▏  | 162626/225000 [314:56:45<98:26:23,  5.68s/it] 
 72%|███████▏  | 162627/225000 [314:56:50<93:37:44,  5.40s/it]
 72%|███████▏  | 162628/225000 [314:56:55<91:25:44,  5.28s/it]
 72%|███████▏  | 162629/225000 [314:57:00<88:53:02,  5.13s/it]
 72%|███████▏  | 162630/225000 [314:57:04<85:02:43,  4.91s/it]
 72%|███████▏  | 162631/225000 [314:57:08<81:25:10,  4.70s/it]
 72%|███████▏  | 162632/225000 [314:57:13<81:43:44,  4.72s/it]
 72%|███████▏  | 162633/225000 [314:57:18<82:09:48,  4.74s/it]
 72%|███████▏  | 162634/225000 [314:57:30<121:41:00,  7.02s/it]
 72%|███████▏  | 162635/225000 [314:57:35<111:43:20,  6.45s/it]
 72%|███████▏  | 162636/225000 [314:57:40<102:04:49,  5.89s/it]
 72%|███████▏  | 162637/225000 [314:57:45<97:28:28,  5.63s/it] 
 72%|███████▏  | 162638/225000 [314:57:49<91:28:58,  5.28s/it]
 72%|███████▏  | 162639/225000 [314:57:54<88:56:11,  5.13s/it]
 72%|███████▏  | 162640/225000 [314:58:05<117:38:23,  6.79s/it]
 72%|███████▏  | 162641/225000 [314:58:11<113:02:54,  6.53s/it]
 72%|███████▏  | 162642/225000 [314:58:15<100:08:25,  5.78s/it]
 72%|███████▏  | 162643/225000 [314:58:19<92:04:28,  5.32s/it] 
 72%|███████▏  | 162644/225000 [314:58:23<87:53:10,  5.07s/it]
 72%|███████▏  | 162645/225000 [314:58:28<86:20:46,  4.99s/it]
 72%|███████▏  | 162646/225000 [314:58:37<104:22:19,  6.03s/it]
 72%|███████▏  | 162647/225000 [314:58:46<120:39:13,  6.97s/it]
 72%|███████▏  | 162648/225000 [314:58:50<104:25:02,  6.03s/it]
 72%|███████▏  | 162649/225000 [314:58:53<92:58:45,  5.37s/it] 
 72%|███████▏  | 162650/225000 [314:58:58<86:54:00,  5.02s/it]
                                                              
{'loss': '0.7727', 'grad_norm': '1.075', 'learning_rate': '9.699e-06', 'epoch': '0.7229', 'num_input_tokens_seen': 521410680, 'train_runtime': '1.134e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162650/225000 [314:58:58<86:54:00,  5.02s/it]
 72%|███████▏  | 162651/225000 [314:59:02<84:32:49,  4.88s/it]
 72%|███████▏  | 162652/225000 [314:59:07<82:40:25,  4.77s/it]
 72%|███████▏  | 162653/225000 [314:59:11<80:42:33,  4.66s/it]
 72%|███████▏  | 162654/225000 [314:59:24<121:08:29,  6.99s/it]
 72%|███████▏  | 162655/225000 [314:59:28<109:00:41,  6.29s/it]
 72%|███████▏  | 162656/225000 [314:59:32<94:25:28,  5.45s/it] 
 72%|███████▏  | 162657/225000 [314:59:40<110:26:52,  6.38s/it]
 72%|███████▏  | 162658/225000 [314:59:44<98:35:47,  5.69s/it] 
 72%|███████▏  | 162659/225000 [314:59:50<97:08:52,  5.61s/it]
 72%|███████▏  | 162660/225000 [314:59:56<101:25:33,  5.86s/it]
 72%|███████▏  | 162661/225000 [315:00:01<95:12:28,  5.50s/it] 
 72%|███████▏  | 162662/225000 [315:00:08<101:42:01,  5.87s/it]
 72%|███████▏  | 162663/225000 [315:00:14<106:48:59,  6.17s/it]
 72%|███████▏  | 162664/225000 [315:00:19<96:15:18,  5.56s/it] 
 72%|███████▏  | 162665/225000 [315:00:24<93:46:07,  5.42s/it]
 72%|███████▏  | 162666/225000 [315:00:32<109:29:18,  6.32s/it]
 72%|███████▏  | 162667/225000 [315:00:35<92:50:27,  5.36s/it] 
 72%|███████▏  | 162668/225000 [315:00:42<100:48:40,  5.82s/it]
 72%|███████▏  | 162669/225000 [315:00:46<89:24:54,  5.16s/it] 
 72%|███████▏  | 162670/225000 [315:00:50<84:09:27,  4.86s/it]
 72%|███████▏  | 162671/225000 [315:01:00<109:52:37,  6.35s/it]
 72%|███████▏  | 162672/225000 [315:01:03<95:49:56,  5.54s/it] 
 72%|███████▏  | 162673/225000 [315:01:08<92:30:21,  5.34s/it]
 72%|███████▏  | 162674/225000 [315:01:16<104:22:14,  6.03s/it]
 72%|███████▏  | 162675/225000 [315:01:20<94:15:06,  5.44s/it] 
 72%|███████▏  | 162676/225000 [315:01:24<88:38:38,  5.12s/it]
 72%|███████▏  | 162677/225000 [315:01:31<96:02:16,  5.55s/it]
 72%|███████▏  | 162678/225000 [315:01:39<109:36:08,  6.33s/it]
 72%|███████▏  | 162679/225000 [315:01:43<95:47:09,  5.53s/it] 
 72%|███████▏  | 162680/225000 [315:01:50<103:48:35,  6.00s/it]
 72%|███████▏  | 162681/225000 [315:01:53<89:57:41,  5.20s/it] 
 72%|███████▏  | 162682/225000 [315:02:02<110:55:15,  6.41s/it]
 72%|███████▏  | 162683/225000 [315:02:06<96:00:12,  5.55s/it] 
 72%|███████▏  | 162684/225000 [315:02:13<104:41:43,  6.05s/it]
 72%|███████▏  | 162685/225000 [315:02:19<101:43:14,  5.88s/it]
 72%|███████▏  | 162686/225000 [315:02:25<105:27:42,  6.09s/it]
 72%|███████▏  | 162687/225000 [315:02:30<100:42:24,  5.82s/it]
 72%|███████▏  | 162688/225000 [315:02:34<90:31:10,  5.23s/it] 
 72%|███████▏  | 162689/225000 [315:02:39<87:20:02,  5.05s/it]
 72%|███████▏  | 162690/225000 [315:02:48<108:26:13,  6.27s/it]
 72%|███████▏  | 162691/225000 [315:02:52<96:00:59,  5.55s/it] 
 72%|███████▏  | 162692/225000 [315:03:01<116:45:59,  6.75s/it]
 72%|███████▏  | 162693/225000 [315:03:05<100:04:44,  5.78s/it]
 72%|███████▏  | 162694/225000 [315:03:09<90:53:35,  5.25s/it] 
 72%|███████▏  | 162695/225000 [315:03:13<83:46:27,  4.84s/it]
 72%|███████▏  | 162696/225000 [315:03:23<110:09:22,  6.36s/it]
 72%|███████▏  | 162697/225000 [315:03:26<94:37:52,  5.47s/it] 
 72%|███████▏  | 162698/225000 [315:03:30<86:21:26,  4.99s/it]
 72%|███████▏  | 162699/225000 [315:03:39<106:00:48,  6.13s/it]
 72%|███████▏  | 162700/225000 [315:03:43<94:18:17,  5.45s/it] 
                                                              
{'loss': '0.7647', 'grad_norm': '1.129', 'learning_rate': '9.685e-06', 'epoch': '0.7231', 'num_input_tokens_seen': 521541400, 'train_runtime': '1.134e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162700/225000 [315:03:43<94:18:17,  5.45s/it]
 72%|███████▏  | 162701/225000 [315:03:46<83:51:32,  4.85s/it]
 72%|███████▏  | 162702/225000 [315:03:51<83:07:58,  4.80s/it]
 72%|███████▏  | 162703/225000 [315:03:55<80:38:37,  4.66s/it]
 72%|███████▏  | 162704/225000 [315:04:05<110:21:34,  6.38s/it]
 72%|███████▏  | 162705/225000 [315:04:12<109:03:47,  6.30s/it]
 72%|███████▏  | 162706/225000 [315:04:16<99:56:39,  5.78s/it] 
 72%|███████▏  | 162707/225000 [315:04:23<104:06:19,  6.02s/it]
 72%|███████▏  | 162708/225000 [315:04:28<102:36:04,  5.93s/it]
 72%|███████▏  | 162709/225000 [315:04:33<96:01:35,  5.55s/it] 
 72%|███████▏  | 162710/225000 [315:04:37<85:12:01,  4.92s/it]
 72%|███████▏  | 162711/225000 [315:04:45<103:05:31,  5.96s/it]
 72%|███████▏  | 162712/225000 [315:04:49<91:46:22,  5.30s/it] 
 72%|███████▏  | 162713/225000 [315:04:53<84:32:33,  4.89s/it]
 72%|███████▏  | 162714/225000 [315:05:03<112:16:34,  6.49s/it]
 72%|███████▏  | 162715/225000 [315:05:06<95:38:40,  5.53s/it] 
 72%|███████▏  | 162716/225000 [315:05:13<104:10:10,  6.02s/it]
 72%|███████▏  | 162717/225000 [315:05:17<91:58:59,  5.32s/it] 
 72%|███████▏  | 162718/225000 [315:05:25<105:38:25,  6.11s/it]
 72%|███████▏  | 162719/225000 [315:05:29<94:07:02,  5.44s/it] 
 72%|███████▏  | 162720/225000 [315:05:36<102:02:16,  5.90s/it]
 72%|███████▏  | 162721/225000 [315:05:39<89:54:25,  5.20s/it] 
 72%|███████▏  | 162722/225000 [315:05:43<82:10:20,  4.75s/it]
 72%|███████▏  | 162723/225000 [315:05:51<100:55:00,  5.83s/it]
 72%|███████▏  | 162724/225000 [315:05:58<103:28:51,  5.98s/it]
 72%|███████▏  | 162725/225000 [315:06:02<95:44:56,  5.54s/it] 
 72%|███████▏  | 162726/225000 [315:06:07<92:15:56,  5.33s/it]
 72%|███████▏  | 162727/225000 [315:06:12<89:25:30,  5.17s/it]
 72%|███████▏  | 162728/225000 [315:06:17<87:05:34,  5.03s/it]
 72%|███████▏  | 162729/225000 [315:06:22<86:29:03,  5.00s/it]
 72%|███████▏  | 162730/225000 [315:06:27<86:21:54,  4.99s/it]
 72%|███████▏  | 162731/225000 [315:06:32<90:53:11,  5.25s/it]
 72%|███████▏  | 162732/225000 [315:06:44<124:46:19,  7.21s/it]
 72%|███████▏  | 162733/225000 [315:06:48<106:27:45,  6.16s/it]
 72%|███████▏  | 162734/225000 [315:06:52<93:49:19,  5.42s/it] 
 72%|███████▏  | 162735/225000 [315:06:56<90:34:48,  5.24s/it]
 72%|███████▏  | 162736/225000 [315:07:07<117:12:58,  6.78s/it]
 72%|███████▏  | 162737/225000 [315:07:10<99:57:44,  5.78s/it] 
 72%|███████▏  | 162738/225000 [315:07:18<107:53:20,  6.24s/it]
 72%|███████▏  | 162739/225000 [315:07:21<96:01:43,  5.55s/it] 
 72%|███████▏  | 162740/225000 [315:07:26<88:41:26,  5.13s/it]
 72%|███████▏  | 162741/225000 [315:07:29<82:03:01,  4.74s/it]
 72%|███████▏  | 162742/225000 [315:07:34<78:28:42,  4.54s/it]
 72%|███████▏  | 162743/225000 [315:07:46<119:41:37,  6.92s/it]
 72%|███████▏  | 162744/225000 [315:07:51<108:03:56,  6.25s/it]
 72%|███████▏  | 162745/225000 [315:07:54<93:45:10,  5.42s/it] 
 72%|███████▏  | 162746/225000 [315:07:59<88:38:18,  5.13s/it]
 72%|███████▏  | 162747/225000 [315:08:03<86:24:55,  5.00s/it]
 72%|███████▏  | 162748/225000 [315:08:09<87:34:13,  5.06s/it]
 72%|███████▏  | 162749/225000 [315:08:14<87:34:10,  5.06s/it]
 72%|███████▏  | 162750/225000 [315:08:19<88:35:14,  5.12s/it]
                                                              
{'loss': '0.7779', 'grad_norm': '1.264', 'learning_rate': '9.67e-06', 'epoch': '0.7233', 'num_input_tokens_seen': 521670720, 'train_runtime': '1.134e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162750/225000 [315:08:19<88:35:14,  5.12s/it]
 72%|███████▏  | 162751/225000 [315:08:24<87:30:49,  5.06s/it]
 72%|███████▏  | 162752/225000 [315:08:29<87:45:57,  5.08s/it]
 72%|███████▏  | 162753/225000 [315:08:34<87:22:34,  5.05s/it]
 72%|███████▏  | 162754/225000 [315:08:46<124:37:02,  7.21s/it]
 72%|███████▏  | 162755/225000 [315:08:52<119:13:38,  6.90s/it]
 72%|███████▏  | 162756/225000 [315:08:56<103:39:21,  6.00s/it]
 72%|███████▏  | 162757/225000 [315:09:00<91:08:11,  5.27s/it] 
 72%|███████▏  | 162758/225000 [315:09:10<119:09:47,  6.89s/it]
 72%|███████▏  | 162759/225000 [315:09:14<100:06:10,  5.79s/it]
 72%|███████▏  | 162760/225000 [315:09:17<89:38:14,  5.18s/it] 
 72%|███████▏  | 162761/225000 [315:09:22<88:16:57,  5.11s/it]
 72%|███████▏  | 162762/225000 [315:09:27<87:31:39,  5.06s/it]
 72%|███████▏  | 162763/225000 [315:09:33<88:23:12,  5.11s/it]
 72%|███████▏  | 162764/225000 [315:09:38<87:54:34,  5.09s/it]
 72%|███████▏  | 162765/225000 [315:09:43<87:58:41,  5.09s/it]
 72%|███████▏  | 162766/225000 [315:09:48<87:52:36,  5.08s/it]
 72%|███████▏  | 162767/225000 [315:09:53<86:41:13,  5.01s/it]
 72%|███████▏  | 162768/225000 [315:09:58<88:06:32,  5.10s/it]
 72%|███████▏  | 162769/225000 [315:10:03<88:13:40,  5.10s/it]
 72%|███████▏  | 162770/225000 [315:10:15<123:34:04,  7.15s/it]
 72%|███████▏  | 162771/225000 [315:10:22<120:53:05,  6.99s/it]
 72%|███████▏  | 162772/225000 [315:10:25<102:07:10,  5.91s/it]
 72%|███████▏  | 162773/225000 [315:10:29<90:22:22,  5.23s/it] 
 72%|███████▏  | 162774/225000 [315:10:38<114:16:27,  6.61s/it]
 72%|███████▏  | 162775/225000 [315:10:42<97:58:47,  5.67s/it] 
 72%|███████▏  | 162776/225000 [315:10:51<114:03:29,  6.60s/it]
 72%|███████▏  | 162777/225000 [315:10:54<97:42:54,  5.65s/it] 
 72%|███████▏  | 162778/225000 [315:11:02<111:26:55,  6.45s/it]
 72%|███████▏  | 162779/225000 [315:11:07<99:49:14,  5.78s/it] 
 72%|███████▏  | 162780/225000 [315:11:11<91:02:58,  5.27s/it]
 72%|███████▏  | 162781/225000 [315:11:15<85:09:35,  4.93s/it]
 72%|███████▏  | 162782/225000 [315:11:19<82:25:55,  4.77s/it]
 72%|███████▏  | 162783/225000 [315:11:23<79:24:40,  4.59s/it]
 72%|███████▏  | 162784/225000 [315:11:34<111:18:26,  6.44s/it]
 72%|███████▏  | 162785/225000 [315:11:39<100:50:03,  5.83s/it]
 72%|███████▏  | 162786/225000 [315:11:42<86:55:40,  5.03s/it] 
 72%|███████▏  | 162787/225000 [315:11:46<80:55:12,  4.68s/it]
 72%|███████▏  | 162788/225000 [315:11:56<112:36:08,  6.52s/it]
 72%|███████▏  | 162789/225000 [315:12:01<105:15:45,  6.09s/it]
 72%|███████▏  | 162790/225000 [315:12:06<97:48:21,  5.66s/it] 
 72%|███████▏  | 162791/225000 [315:12:11<93:30:27,  5.41s/it]
 72%|███████▏  | 162792/225000 [315:12:16<92:53:36,  5.38s/it]
 72%|███████▏  | 162793/225000 [315:12:21<88:35:15,  5.13s/it]
 72%|███████▏  | 162794/225000 [315:12:25<85:20:41,  4.94s/it]
 72%|███████▏  | 162795/225000 [315:12:30<84:57:46,  4.92s/it]
 72%|███████▏  | 162796/225000 [315:12:35<83:51:54,  4.85s/it]
 72%|███████▏  | 162797/225000 [315:12:40<84:32:13,  4.89s/it]
 72%|███████▏  | 162798/225000 [315:12:45<84:34:14,  4.89s/it]
 72%|███████▏  | 162799/225000 [315:12:49<83:29:24,  4.83s/it]
 72%|███████▏  | 162800/225000 [315:12:55<85:32:32,  4.95s/it]
                                                              
{'loss': '0.7755', 'grad_norm': '1.188', 'learning_rate': '9.656e-06', 'epoch': '0.7236', 'num_input_tokens_seen': 521801176, 'train_runtime': '1.135e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162800/225000 [315:12:55<85:32:32,  4.95s/it]
 72%|███████▏  | 162801/225000 [315:13:07<125:25:43,  7.26s/it]
 72%|███████▏  | 162802/225000 [315:13:13<119:03:01,  6.89s/it]
 72%|███████▏  | 162803/225000 [315:13:17<103:19:18,  5.98s/it]
 72%|███████▏  | 162804/225000 [315:13:27<123:00:39,  7.12s/it]
 72%|███████▏  | 162805/225000 [315:13:30<103:20:45,  5.98s/it]
 72%|███████▏  | 162806/225000 [315:13:38<109:56:55,  6.36s/it]
 72%|███████▏  | 162807/225000 [315:13:41<93:04:14,  5.39s/it] 
 72%|███████▏  | 162808/225000 [315:13:48<102:12:26,  5.92s/it]
 72%|███████▏  | 162809/225000 [315:13:51<89:52:49,  5.20s/it] 
 72%|███████▏  | 162810/225000 [315:14:00<107:15:16,  6.21s/it]
 72%|███████▏  | 162811/225000 [315:14:04<94:21:04,  5.46s/it] 
 72%|███████▏  | 162812/225000 [315:14:07<84:29:51,  4.89s/it]
 72%|███████▏  | 162813/225000 [315:14:13<87:53:37,  5.09s/it]
 72%|███████▏  | 162814/225000 [315:14:22<108:18:16,  6.27s/it]
 72%|███████▏  | 162815/225000 [315:14:25<94:50:11,  5.49s/it] 
 72%|███████▏  | 162816/225000 [315:14:29<86:42:18,  5.02s/it]
 72%|███████▏  | 162817/225000 [315:14:34<82:22:56,  4.77s/it]
 72%|███████▏  | 162818/225000 [315:14:38<81:04:13,  4.69s/it]
 72%|███████▏  | 162819/225000 [315:14:49<111:04:17,  6.43s/it]
 72%|███████▏  | 162820/225000 [315:14:52<97:19:50,  5.64s/it] 
 72%|███████▏  | 162821/225000 [315:14:56<87:47:32,  5.08s/it]
 72%|███████▏  | 162822/225000 [315:15:01<87:49:04,  5.08s/it]
 72%|███████▏  | 162823/225000 [315:15:11<110:45:37,  6.41s/it]
 72%|███████▏  | 162824/225000 [315:15:15<97:22:33,  5.64s/it] 
 72%|███████▏  | 162825/225000 [315:15:19<89:23:33,  5.18s/it]
 72%|███████▏  | 162826/225000 [315:15:31<127:58:41,  7.41s/it]
 72%|███████▏  | 162827/225000 [315:15:34<105:16:01,  6.10s/it]
 72%|███████▏  | 162828/225000 [315:15:39<96:36:25,  5.59s/it] 
 72%|███████▏  | 162829/225000 [315:15:45<101:18:31,  5.87s/it]
 72%|███████▏  | 162830/225000 [315:15:48<85:50:15,  4.97s/it] 
 72%|███████▏  | 162831/225000 [315:15:56<98:34:35,  5.71s/it]
 72%|███████▏  | 162832/225000 [315:16:00<89:45:54,  5.20s/it]
 72%|███████▏  | 162833/225000 [315:16:04<87:18:05,  5.06s/it]
 72%|███████▏  | 162834/225000 [315:16:08<82:23:03,  4.77s/it]
 72%|███████▏  | 162835/225000 [315:16:12<78:43:37,  4.56s/it]
 72%|███████▏  | 162836/225000 [315:16:23<109:26:35,  6.34s/it]
 72%|███████▏  | 162837/225000 [315:16:27<96:48:42,  5.61s/it] 
 72%|███████▏  | 162838/225000 [315:16:35<108:23:33,  6.28s/it]
 72%|███████▏  | 162839/225000 [315:16:38<91:39:55,  5.31s/it] 
 72%|███████▏  | 162840/225000 [315:16:42<87:56:11,  5.09s/it]
 72%|███████▏  | 162841/225000 [315:16:47<85:20:07,  4.94s/it]
 72%|███████▏  | 162842/225000 [315:16:52<84:11:14,  4.88s/it]
 72%|███████▏  | 162843/225000 [315:16:56<84:12:01,  4.88s/it]
 72%|███████▏  | 162844/225000 [315:17:01<84:15:22,  4.88s/it]
 72%|███████▏  | 162845/225000 [315:17:06<82:45:58,  4.79s/it]
 72%|███████▏  | 162846/225000 [315:17:18<119:48:20,  6.94s/it]
 72%|███████▏  | 162847/225000 [315:17:22<102:59:01,  5.96s/it]
 72%|███████▏  | 162848/225000 [315:17:30<116:56:50,  6.77s/it]
 72%|███████▏  | 162849/225000 [315:17:34<103:18:09,  5.98s/it]
 72%|███████▏  | 162850/225000 [315:17:39<94:00:49,  5.45s/it] 
                                                              
{'loss': '0.7875', 'grad_norm': '1.35', 'learning_rate': '9.642e-06', 'epoch': '0.7238', 'num_input_tokens_seen': 521930536, 'train_runtime': '1.135e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162850/225000 [315:17:39<94:00:49,  5.45s/it]
 72%|███████▏  | 162851/225000 [315:17:43<87:03:26,  5.04s/it]
 72%|███████▏  | 162852/225000 [315:17:46<80:10:48,  4.64s/it]
 72%|███████▏  | 162853/225000 [315:17:51<81:16:54,  4.71s/it]
 72%|███████▏  | 162854/225000 [315:18:02<110:35:03,  6.41s/it]
 72%|███████▏  | 162855/225000 [315:18:05<96:21:21,  5.58s/it] 
 72%|███████▏  | 162856/225000 [315:18:09<87:38:47,  5.08s/it]
 72%|███████▏  | 162857/225000 [315:18:19<113:04:10,  6.55s/it]
 72%|███████▏  | 162858/225000 [315:18:23<96:26:35,  5.59s/it] 
 72%|███████▏  | 162859/225000 [315:18:30<106:52:32,  6.19s/it]
 72%|███████▏  | 162860/225000 [315:18:33<91:34:12,  5.30s/it] 
 72%|███████▏  | 162861/225000 [315:18:42<107:12:49,  6.21s/it]
 72%|███████▏  | 162862/225000 [315:18:45<94:20:42,  5.47s/it] 
 72%|███████▏  | 162863/225000 [315:18:52<99:33:46,  5.77s/it]
 72%|███████▏  | 162864/225000 [315:18:58<99:31:43,  5.77s/it]
 72%|███████▏  | 162865/225000 [315:19:02<90:51:14,  5.26s/it]
 72%|███████▏  | 162866/225000 [315:19:06<86:12:23,  4.99s/it]
 72%|███████▏  | 162867/225000 [315:19:10<83:00:03,  4.81s/it]
 72%|███████▏  | 162868/225000 [315:19:15<81:56:54,  4.75s/it]
 72%|███████▏  | 162869/225000 [315:19:20<82:39:19,  4.79s/it]
 72%|███████▏  | 162870/225000 [315:19:31<115:42:05,  6.70s/it]
 72%|███████▏  | 162871/225000 [315:19:35<103:21:35,  5.99s/it]
 72%|███████▏  | 162872/225000 [315:19:42<106:23:53,  6.17s/it]
 72%|███████▏  | 162873/225000 [315:19:46<92:40:26,  5.37s/it] 
 72%|███████▏  | 162874/225000 [315:19:54<107:43:28,  6.24s/it]
 72%|███████▏  | 162875/225000 [315:19:58<98:03:22,  5.68s/it] 
 72%|███████▏  | 162876/225000 [315:20:02<89:33:37,  5.19s/it]
 72%|███████▏  | 162877/225000 [315:20:06<83:59:26,  4.87s/it]
 72%|███████▏  | 162878/225000 [315:20:17<111:49:58,  6.48s/it]
 72%|███████▏  | 162879/225000 [315:20:21<102:04:38,  5.92s/it]
 72%|███████▏  | 162880/225000 [315:20:25<89:16:37,  5.17s/it] 
 72%|███████▏  | 162881/225000 [315:20:34<113:21:05,  6.57s/it]
 72%|███████▏  | 162882/225000 [315:20:38<96:30:16,  5.59s/it] 
 72%|███████▏  | 162883/225000 [315:20:41<85:32:22,  4.96s/it]
 72%|███████▏  | 162884/225000 [315:20:49<101:08:23,  5.86s/it]
 72%|███████▏  | 162885/225000 [315:20:53<88:43:02,  5.14s/it] 
 72%|███████▏  | 162886/225000 [315:20:57<86:43:46,  5.03s/it]
 72%|███████▏  | 162887/225000 [315:21:06<103:28:19,  6.00s/it]
 72%|███████▏  | 162888/225000 [315:21:09<89:08:21,  5.17s/it] 
 72%|███████▏  | 162889/225000 [315:21:18<106:45:34,  6.19s/it]
 72%|███████▏  | 162890/225000 [315:21:21<91:25:13,  5.30s/it] 
 72%|███████▏  | 162891/225000 [315:21:27<98:45:08,  5.72s/it]
 72%|███████▏  | 162892/225000 [315:21:32<92:27:01,  5.36s/it]
 72%|███████▏  | 162893/225000 [315:21:41<109:51:49,  6.37s/it]
 72%|███████▏  | 162894/225000 [315:21:44<93:01:13,  5.39s/it] 
 72%|███████▏  | 162895/225000 [315:21:49<91:07:32,  5.28s/it]
 72%|███████▏  | 162896/225000 [315:21:54<92:18:56,  5.35s/it]
 72%|███████▏  | 162897/225000 [315:22:02<105:07:18,  6.09s/it]
 72%|███████▏  | 162898/225000 [315:22:05<89:31:24,  5.19s/it] 
 72%|███████▏  | 162899/225000 [315:22:13<102:09:13,  5.92s/it]
 72%|███████▏  | 162900/225000 [315:22:17<93:08:50,  5.40s/it] 
                                                              
{'loss': '0.7636', 'grad_norm': '1.17', 'learning_rate': '9.627e-06', 'epoch': '0.724', 'num_input_tokens_seen': 522060928, 'train_runtime': '1.135e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162900/225000 [315:22:17<93:08:50,  5.40s/it]
 72%|███████▏  | 162901/225000 [315:22:21<85:13:34,  4.94s/it]
 72%|███████▏  | 162902/225000 [315:22:25<79:12:19,  4.59s/it]
 72%|███████▏  | 162903/225000 [315:22:35<109:55:13,  6.37s/it]
 72%|███████▏  | 162904/225000 [315:22:39<94:45:16,  5.49s/it] 
 72%|███████▏  | 162905/225000 [315:22:44<92:52:26,  5.38s/it]
 72%|███████▏  | 162906/225000 [315:22:49<90:24:16,  5.24s/it]
 72%|███████▏  | 162907/225000 [315:22:54<91:16:03,  5.29s/it]
 72%|███████▏  | 162908/225000 [315:22:59<90:45:25,  5.26s/it]
 72%|███████▏  | 162909/225000 [315:23:04<87:57:15,  5.10s/it]
 72%|███████▏  | 162910/225000 [315:23:09<85:45:25,  4.97s/it]
 72%|███████▏  | 162911/225000 [315:23:14<86:42:38,  5.03s/it]
 72%|███████▏  | 162912/225000 [315:23:19<87:20:30,  5.06s/it]
 72%|███████▏  | 162913/225000 [315:23:24<84:16:53,  4.89s/it]
 72%|███████▏  | 162914/225000 [315:23:28<83:11:41,  4.82s/it]
 72%|███████▏  | 162915/225000 [315:23:34<85:50:14,  4.98s/it]
 72%|███████▏  | 162916/225000 [315:23:39<88:05:47,  5.11s/it]
 72%|███████▏  | 162917/225000 [315:23:44<87:59:39,  5.10s/it]
 72%|███████▏  | 162918/225000 [315:23:49<88:58:56,  5.16s/it]
 72%|███████▏  | 162919/225000 [315:23:57<103:14:38,  5.99s/it]
 72%|███████▏  | 162920/225000 [315:24:10<137:56:07,  8.00s/it]
 72%|███████▏  | 162921/225000 [315:24:13<113:56:27,  6.61s/it]
 72%|███████▏  | 162922/225000 [315:24:22<122:51:02,  7.12s/it]
 72%|███████▏  | 162923/225000 [315:24:26<110:58:54,  6.44s/it]
 72%|███████▏  | 162924/225000 [315:24:30<97:34:03,  5.66s/it] 
 72%|███████▏  | 162925/225000 [315:24:35<91:31:05,  5.31s/it]
 72%|███████▏  | 162926/225000 [315:24:39<88:08:14,  5.11s/it]
 72%|███████▏  | 162927/225000 [315:24:44<84:29:37,  4.90s/it]
 72%|███████▏  | 162928/225000 [315:24:48<81:41:36,  4.74s/it]
 72%|███████▏  | 162929/225000 [315:24:53<81:55:44,  4.75s/it]
 72%|███████▏  | 162930/225000 [315:24:57<80:40:18,  4.68s/it]
 72%|███████▏  | 162931/225000 [315:25:09<116:45:47,  6.77s/it]
 72%|███████▏  | 162932/225000 [315:25:15<111:15:44,  6.45s/it]
 72%|███████▏  | 162933/225000 [315:25:20<102:44:18,  5.96s/it]
 72%|███████▏  | 162934/225000 [315:25:24<96:11:56,  5.58s/it] 
 72%|███████▏  | 162935/225000 [315:25:29<91:53:00,  5.33s/it]
 72%|███████▏  | 162936/225000 [315:25:34<89:40:39,  5.20s/it]
 72%|███████▏  | 162937/225000 [315:25:39<88:16:34,  5.12s/it]
 72%|███████▏  | 162938/225000 [315:25:44<89:03:42,  5.17s/it]
 72%|███████▏  | 162939/225000 [315:25:49<87:20:12,  5.07s/it]
 72%|███████▏  | 162940/225000 [315:26:01<123:07:39,  7.14s/it]
 72%|███████▏  | 162941/225000 [315:26:06<112:24:59,  6.52s/it]
 72%|███████▏  | 162942/225000 [315:26:11<101:33:26,  5.89s/it]
 72%|███████▏  | 162943/225000 [315:26:19<113:09:55,  6.56s/it]
 72%|███████▏  | 162944/225000 [315:26:22<97:59:17,  5.68s/it] 
 72%|███████▏  | 162945/225000 [315:26:26<87:56:50,  5.10s/it]
 72%|███████▏  | 162946/225000 [315:26:36<114:58:58,  6.67s/it]
 72%|███████▏  | 162947/225000 [315:26:40<98:51:42,  5.74s/it] 
 72%|███████▏  | 162948/225000 [315:26:45<93:25:43,  5.42s/it]
 72%|███████▏  | 162949/225000 [315:26:52<103:09:30,  5.98s/it]
 72%|███████▏  | 162950/225000 [315:26:56<91:31:56,  5.31s/it] 
                                                              
{'loss': '0.7688', 'grad_norm': '1.139', 'learning_rate': '9.613e-06', 'epoch': '0.7242', 'num_input_tokens_seen': 522190128, 'train_runtime': '1.136e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 162950/225000 [315:26:56<91:31:56,  5.31s/it]
 72%|███████▏  | 162951/225000 [315:26:59<82:29:29,  4.79s/it]
 72%|███████▏  | 162952/225000 [315:27:08<105:01:38,  6.09s/it]
 72%|███████▏  | 162953/225000 [315:27:14<101:31:42,  5.89s/it]
 72%|███████▏  | 162954/225000 [315:27:17<89:58:48,  5.22s/it] 
 72%|███████▏  | 162955/225000 [315:27:22<88:14:02,  5.12s/it]
 72%|███████▏  | 162956/225000 [315:27:27<85:36:54,  4.97s/it]
 72%|███████▏  | 162957/225000 [315:27:32<85:24:04,  4.96s/it]
 72%|███████▏  | 162958/225000 [315:27:36<82:39:16,  4.80s/it]
 72%|███████▏  | 162959/225000 [315:27:41<82:57:08,  4.81s/it]
 72%|███████▏  | 162960/225000 [315:27:46<83:03:52,  4.82s/it]
 72%|███████▏  | 162961/225000 [315:27:55<104:23:35,  6.06s/it]
 72%|███████▏  | 162962/225000 [315:28:05<123:28:15,  7.16s/it]
 72%|███████▏  | 162963/225000 [315:28:09<106:30:02,  6.18s/it]
 72%|███████▏  | 162964/225000 [315:28:13<97:01:32,  5.63s/it] 
 72%|███████▏  | 162965/225000 [315:28:18<92:24:04,  5.36s/it]
 72%|███████▏  | 162966/225000 [315:28:22<87:38:07,  5.09s/it]
 72%|███████▏  | 162967/225000 [315:28:27<84:19:40,  4.89s/it]
 72%|███████▏  | 162968/225000 [315:28:31<81:38:18,  4.74s/it]
 72%|███████▏  | 162969/225000 [315:28:43<119:16:47,  6.92s/it]
 72%|███████▏  | 162970/225000 [315:28:47<107:07:34,  6.22s/it]
 72%|███████▏  | 162971/225000 [315:28:52<97:58:59,  5.69s/it] 
 72%|███████▏  | 162972/225000 [315:28:57<92:19:21,  5.36s/it]
 72%|███████▏  | 162973/225000 [315:29:05<108:49:15,  6.32s/it]
 72%|███████▏  | 162974/225000 [315:29:12<112:03:26,  6.50s/it]
 72%|███████▏  | 162975/225000 [315:29:16<98:23:55,  5.71s/it] 
 72%|███████▏  | 162976/225000 [315:29:20<90:14:12,  5.24s/it]
 72%|███████▏  | 162977/225000 [315:29:24<84:50:09,  4.92s/it]
 72%|███████▏  | 162978/225000 [315:29:33<105:33:10,  6.13s/it]
 72%|███████▏  | 162979/225000 [315:29:40<109:35:05,  6.36s/it]
 72%|███████▏  | 162980/225000 [315:29:44<95:34:29,  5.55s/it] 
 72%|███████▏  | 162981/225000 [315:29:53<116:20:19,  6.75s/it]
 72%|███████▏  | 162982/225000 [315:29:56<98:10:08,  5.70s/it] 
 72%|███████▏  | 162983/225000 [315:30:03<100:28:53,  5.83s/it]
 72%|███████▏  | 162984/225000 [315:30:06<88:54:12,  5.16s/it] 
 72%|███████▏  | 162985/225000 [315:30:10<81:55:31,  4.76s/it]
 72%|███████▏  | 162986/225000 [315:30:20<108:51:39,  6.32s/it]
 72%|███████▏  | 162987/225000 [315:30:24<94:59:09,  5.51s/it] 
 72%|███████▏  | 162988/225000 [315:30:28<89:22:32,  5.19s/it]
 72%|███████▏  | 162989/225000 [315:30:32<82:33:45,  4.79s/it]
 72%|███████▏  | 162990/225000 [315:30:36<80:57:24,  4.70s/it]
 72%|███████▏  | 162991/225000 [315:30:43<88:51:47,  5.16s/it]
 72%|███████▏  | 162992/225000 [315:30:53<114:53:53,  6.67s/it]
 72%|███████▏  | 162993/225000 [315:30:57<100:42:38,  5.85s/it]
 72%|███████▏  | 162994/225000 [315:31:01<94:20:19,  5.48s/it] 
 72%|███████▏  | 162995/225000 [315:31:06<90:28:45,  5.25s/it]
 72%|███████▏  | 162996/225000 [315:31:11<88:57:41,  5.17s/it]
 72%|███████▏  | 162997/225000 [315:31:16<86:17:46,  5.01s/it]
 72%|███████▏  | 162998/225000 [315:31:21<85:30:57,  4.97s/it]
 72%|███████▏  | 162999/225000 [315:31:27<91:18:01,  5.30s/it]
 72%|███████▏  | 163000/225000 [315:31:37<119:12:26,  6.92s/it]
                                                               
{'loss': '0.7652', 'grad_norm': '1.068', 'learning_rate': '9.598e-06', 'epoch': '0.7244', 'num_input_tokens_seen': 522320408, 'train_runtime': '1.136e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 163000/225000 [315:31:37<119:12:26,  6.92s/it]
 72%|███████▏  | 163001/225000 [315:31:41<100:06:26,  5.81s/it]
 72%|███████▏  | 163002/225000 [315:31:51<122:09:53,  7.09s/it]
 72%|███████▏  | 163003/225000 [315:31:55<105:41:40,  6.14s/it]
 72%|███████▏  | 163004/225000 [315:31:59<94:13:50,  5.47s/it] 
 72%|███████▏  | 163005/225000 [315:32:05<96:55:38,  5.63s/it]
 72%|███████▏  | 163006/225000 [315:32:12<108:35:19,  6.31s/it]
 72%|███████▏  | 163007/225000 [315:32:17<102:17:22,  5.94s/it]
 72%|███████▏  | 163008/225000 [315:32:22<97:14:43,  5.65s/it] 
 72%|███████▏  | 163009/225000 [315:32:28<97:49:17,  5.68s/it]
 72%|███████▏  | 163010/225000 [315:32:35<104:39:35,  6.08s/it]
 72%|███████▏  | 163011/225000 [315:32:40<99:09:20,  5.76s/it] 
 72%|███████▏  | 163012/225000 [315:32:45<95:02:31,  5.52s/it]
 72%|███████▏  | 163013/225000 [315:32:50<91:25:17,  5.31s/it]
 72%|███████▏  | 163014/225000 [315:32:55<90:44:44,  5.27s/it]
 72%|███████▏  | 163015/225000 [315:33:00<90:32:30,  5.26s/it]
 72%|███████▏  | 163016/225000 [315:33:05<87:35:42,  5.09s/it]
 72%|███████▏  | 163017/225000 [315:33:10<87:15:57,  5.07s/it]
 72%|███████▏  | 163018/225000 [315:33:15<87:49:49,  5.10s/it]
 72%|███████▏  | 163019/225000 [315:33:20<87:02:25,  5.06s/it]
 72%|███████▏  | 163020/225000 [315:33:25<86:38:52,  5.03s/it]
 72%|███████▏  | 163021/225000 [315:33:30<85:21:49,  4.96s/it]
 72%|███████▏  | 163022/225000 [315:33:42<121:22:00,  7.05s/it]
 72%|███████▏  | 163023/225000 [315:33:48<117:11:38,  6.81s/it]
 72%|███████▏  | 163024/225000 [315:33:56<122:23:53,  7.11s/it]
 72%|███████▏  | 163025/225000 [315:34:00<108:29:30,  6.30s/it]
 72%|███████▏  | 163026/225000 [315:34:05<99:11:05,  5.76s/it] 
 72%|███████▏  | 163027/225000 [315:34:09<92:37:01,  5.38s/it]
 72%|███████▏  | 163028/225000 [315:34:14<89:24:13,  5.19s/it]
 72%|███████▏  | 163029/225000 [315:34:19<85:08:15,  4.95s/it]
 72%|███████▏  | 163030/225000 [315:34:23<84:17:27,  4.90s/it]
 72%|███████▏  | 163031/225000 [315:34:28<82:20:23,  4.78s/it]
 72%|███████▏  | 163032/225000 [315:34:40<121:55:23,  7.08s/it]
 72%|███████▏  | 163033/225000 [315:34:44<102:51:43,  5.98s/it]
 72%|███████▏  | 163034/225000 [315:34:48<94:55:39,  5.51s/it] 
 72%|███████▏  | 163035/225000 [315:34:57<112:53:54,  6.56s/it]
 72%|███████▏  | 163036/225000 [315:35:01<98:05:50,  5.70s/it] 
 72%|███████▏  | 163037/225000 [315:35:07<102:11:27,  5.94s/it]
 72%|███████▏  | 163038/225000 [315:35:10<86:26:21,  5.02s/it] 
 72%|███████▏  | 163039/225000 [315:35:15<85:23:58,  4.96s/it]
 72%|███████▏  | 163040/225000 [315:35:20<83:55:16,  4.88s/it]
 72%|███████▏  | 163041/225000 [315:35:25<84:25:41,  4.91s/it]
 72%|███████▏  | 163042/225000 [315:35:29<83:38:51,  4.86s/it]
 72%|███████▏  | 163043/225000 [315:35:34<83:10:08,  4.83s/it]
 72%|███████▏  | 163044/225000 [315:35:39<83:01:55,  4.82s/it]
 72%|███████▏  | 163045/225000 [315:35:47<101:33:31,  5.90s/it]
 72%|███████▏  | 163046/225000 [315:35:57<121:57:03,  7.09s/it]
 72%|███████▏  | 163047/225000 [315:36:00<100:54:53,  5.86s/it]
 72%|███████▏  | 163048/225000 [315:36:06<102:46:52,  5.97s/it]
 72%|███████▏  | 163049/225000 [315:36:11<95:01:10,  5.52s/it] 
 72%|███████▏  | 163050/225000 [315:36:16<91:20:39,  5.31s/it]
                                                              
{'loss': '0.7711', 'grad_norm': '1.216', 'learning_rate': '9.584e-06', 'epoch': '0.7247', 'num_input_tokens_seen': 522450296, 'train_runtime': '1.136e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 163050/225000 [315:36:16<91:20:39,  5.31s/it]
 72%|███████▏  | 163051/225000 [315:36:21<89:24:19,  5.20s/it]
 72%|███████▏  | 163052/225000 [315:36:26<88:21:04,  5.13s/it]
 72%|███████▏  | 163053/225000 [315:36:31<86:53:44,  5.05s/it]
 72%|███████▏  | 163054/225000 [315:36:35<85:06:36,  4.95s/it]
 72%|███████▏  | 163055/225000 [315:36:40<84:04:58,  4.89s/it]
 72%|███████▏  | 163056/225000 [315:36:46<89:26:35,  5.20s/it]
 72%|███████▏  | 163057/225000 [315:36:58<126:34:21,  7.36s/it]
 72%|███████▏  | 163058/225000 [315:37:01<105:03:30,  6.11s/it]
 72%|███████▏  | 163059/225000 [315:37:07<100:11:32,  5.82s/it]
 72%|███████▏  | 163060/225000 [315:37:14<107:49:35,  6.27s/it]
 72%|███████▏  | 163061/225000 [315:37:18<94:47:13,  5.51s/it] 
 72%|███████▏  | 163062/225000 [315:37:25<106:00:37,  6.16s/it]
 72%|███████▏  | 163063/225000 [315:37:30<98:43:58,  5.74s/it] 
 72%|███████▏  | 163064/225000 [315:37:38<107:47:23,  6.27s/it]
 72%|███████▏  | 163065/225000 [315:37:42<95:54:46,  5.57s/it] 
 72%|███████▏  | 163066/225000 [315:37:46<88:57:27,  5.17s/it]
 72%|███████▏  | 163067/225000 [315:37:50<83:02:57,  4.83s/it]
 72%|███████▏  | 163068/225000 [315:37:56<87:38:50,  5.09s/it]
 72%|███████▏  | 163069/225000 [315:38:05<110:33:11,  6.43s/it]
 72%|███████▏  | 163070/225000 [315:38:09<98:38:34,  5.73s/it] 
 72%|███████▏  | 163071/225000 [315:38:13<90:39:04,  5.27s/it]
 72%|███████▏  | 163072/225000 [315:38:18<85:33:52,  4.97s/it]
 72%|███████▏  | 163073/225000 [315:38:29<119:00:19,  6.92s/it]
 72%|███████▏  | 163074/225000 [315:38:33<102:12:24,  5.94s/it]
 72%|███████▏  | 163075/225000 [315:38:36<90:08:48,  5.24s/it] 
 72%|███████▏  | 163076/225000 [315:38:45<105:42:06,  6.15s/it]
 72%|███████▏  | 163077/225000 [315:38:50<101:49:03,  5.92s/it]
 72%|███████▏  | 163078/225000 [315:38:55<97:22:44,  5.66s/it] 
 72%|███████▏  | 163079/225000 [315:39:00<93:53:24,  5.46s/it]
 72%|███████▏  | 163080/225000 [315:39:06<94:23:42,  5.49s/it]
 72%|███████▏  | 163081/225000 [315:39:14<107:53:01,  6.27s/it]
 72%|███████▏  | 163082/225000 [315:39:17<91:55:06,  5.34s/it] 
 72%|███████▏  | 163083/225000 [315:39:24<100:13:45,  5.83s/it]
 72%|███████▏  | 163084/225000 [315:39:27<88:30:19,  5.15s/it] 
 72%|███████▏  | 163085/225000 [315:39:32<84:32:46,  4.92s/it]
 72%|███████▏  | 163086/225000 [315:39:36<81:50:45,  4.76s/it]
 72%|███████▏  | 163087/225000 [315:39:40<78:08:07,  4.54s/it]
 72%|███████▏  | 163088/225000 [315:39:48<96:53:00,  5.63s/it]
 72%|███████▏  | 163089/225000 [315:39:57<111:09:03,  6.46s/it]
 72%|███████▏  | 163090/225000 [315:40:04<115:52:51,  6.74s/it]
 72%|███████▏  | 163091/225000 [315:40:09<104:57:13,  6.10s/it]
 72%|███████▏  | 163092/225000 [315:40:13<96:19:13,  5.60s/it] 
 72%|███████▏  | 163093/225000 [315:40:18<92:54:10,  5.40s/it]
 72%|███████▏  | 163094/225000 [315:40:22<86:51:13,  5.05s/it]
 72%|███████▏  | 163095/225000 [315:40:27<83:59:29,  4.88s/it]
 72%|███████▏  | 163096/225000 [315:40:33<91:20:16,  5.31s/it]
 72%|███████▏  | 163097/225000 [315:40:44<117:24:56,  6.83s/it]
 72%|███████▏  | 163098/225000 [315:40:47<98:47:11,  5.75s/it] 
 72%|███████▏  | 163099/225000 [315:40:50<86:47:22,  5.05s/it]
 72%|███████▏  | 163100/225000 [315:40:55<86:36:45,  5.04s/it]
                                                              
{'loss': '0.7736', 'grad_norm': '1.156', 'learning_rate': '9.57e-06', 'epoch': '0.7249', 'num_input_tokens_seen': 522580368, 'train_runtime': '1.136e+06', 'train_tokens_per_second': '459.8'}

 72%|███████▏  | 163100/225000 [315:40:55<86:36:45,  5.04s/it]
 72%|███████▏  | 163101/225000 [315:41:03<102:09:47,  5.94s/it]
 72%|███████▏  | 163102/225000 [315:41:07<89:44:31,  5.22s/it] 
 72%|███████▏  | 163103/225000 [315:41:10<80:55:26,  4.71s/it]
 72%|███████▏  | 163104/225000 [315:41:21<112:54:32,  6.57s/it]
 72%|███████▏  | 163105/225000 [315:41:25<95:49:38,  5.57s/it] 
 72%|███████▏  | 163106/225000 [315:41:32<106:20:46,  6.19s/it]
 72%|███████▏  | 163107/225000 [315:41:35<91:39:21,  5.33s/it] 
 72%|███████▏  | 163108/225000 [315:41:40<85:43:38,  4.99s/it]
 72%|███████▏  | 163109/225000 [315:41:46<90:44:21,  5.28s/it]
 72%|███████▏  | 163110/225000 [315:41:53<99:20:26,  5.78s/it]
 72%|███████▏  | 163111/225000 [315:42:00<108:20:32,  6.30s/it]
 72%|███████▏  | 163112/225000 [315:42:04<96:50:24,  5.63s/it] 
 72%|███████▏  | 163113/225000 [315:42:08<90:00:18,  5.24s/it]
 72%|███████▏  | 163114/225000 [315:42:17<107:46:25,  6.27s/it]
 72%|███████▏  | 163115/225000 [315:42:24<109:03:32,  6.34s/it]
 72%|███████▏  | 163116/225000 [315:42:27<93:32:59,  5.44s/it] 
 72%|███████▏  | 163117/225000 [315:42:35<104:41:22,  6.09s/it]
 72%|███████▏  | 163118/225000 [315:42:39<94:22:31,  5.49s/it] 
 72%|███████▏  | 163119/225000 [315:42:43<88:20:05,  5.14s/it]
 72%|███████▏  | 163120/225000 [315:42:47<83:35:32,  4.86s/it]
 72%|███████▏  | 163121/225000 [315:42:52<85:08:55,  4.95s/it]
 72%|███████▏  | 163122/225000 [315:42:57<84:41:58,  4.93s/it]
 72%|███████▏  | 163123/225000 [315:43:02<85:48:50,  4.99s/it]
 72%|███████▏  | 163124/225000 [315:43:08<88:48:53,  5.17s/it]
 72%|███████▎  | 163125/225000 [315:43:13<90:07:32,  5.24s/it]
 73%|███████▎  | 163126/225000 [315:43:18<87:23:59,  5.09s/it]
 73%|███████▎  | 163127/225000 [315:43:23<86:14:31,  5.02s/it]
 73%|███████▎  | 163128/225000 [315:43:28<86:33:17,  5.04s/it]
 73%|███████▎  | 163129/225000 [315:43:34<90:55:01,  5.29s/it]
 73%|███████▎  | 163130/225000 [315:43:45<121:53:20,  7.09s/it]
 73%|███████▎  | 163131/225000 [315:43:52<121:32:08,  7.07s/it]
 73%|███████▎  | 163132/225000 [315:43:56<104:43:15,  6.09s/it]
 73%|███████▎  | 163133/225000 [315:44:04<114:30:02,  6.66s/it]
 73%|███████▎  | 163134/225000 [315:44:08<100:39:48,  5.86s/it]
 73%|███████▎  | 163135/225000 [315:44:12<91:34:07,  5.33s/it] 
 73%|███████▎  | 163136/225000 [315:44:22<115:09:39,  6.70s/it]
 73%|███████▎  | 163137/225000 [315:44:26<98:59:28,  5.76s/it] 
 73%|███████▎  | 163138/225000 [315:44:33<106:26:40,  6.19s/it]
 73%|███████▎  | 163139/225000 [315:44:37<98:19:47,  5.72s/it] 
 73%|███████▎  | 163140/225000 [315:44:42<91:54:49,  5.35s/it]
 73%|███████▎  | 163141/225000 [315:44:47<88:48:19,  5.17s/it]
 73%|███████▎  | 163142/225000 [315:44:54<99:09:19,  5.77s/it]
 73%|███████▎  | 163143/225000 [315:44:59<97:49:11,  5.69s/it]
 73%|███████▎  | 163144/225000 [315:45:06<102:08:09,  5.94s/it]
 73%|███████▎  | 163145/225000 [315:45:10<91:12:15,  5.31s/it] 
 73%|███████▎  | 163146/225000 [315:45:19<109:10:58,  6.35s/it]
 73%|███████▎  | 163147/225000 [315:45:23<97:07:06,  5.65s/it] 
 73%|███████▎  | 163148/225000 [315:45:26<86:17:25,  5.02s/it]
 73%|███████▎  | 163149/225000 [315:45:35<106:58:19,  6.23s/it]
 73%|███████▎  | 163150/225000 [315:45:40<101:27:07,  5.91s/it]
                                                               
{'loss': '0.774', 'grad_norm': '1.277', 'learning_rate': '9.555e-06', 'epoch': '0.7251', 'num_input_tokens_seen': 522710304, 'train_runtime': '1.137e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163150/225000 [315:45:40<101:27:07,  5.91s/it]
 73%|███████▎  | 163151/225000 [315:45:44<89:27:10,  5.21s/it] 
 73%|███████▎  | 163152/225000 [315:45:49<91:44:14,  5.34s/it]
 73%|███████▎  | 163153/225000 [315:45:56<98:30:00,  5.73s/it]
 73%|███████▎  | 163154/225000 [315:46:00<88:07:04,  5.13s/it]
 73%|███████▎  | 163155/225000 [315:46:09<108:09:04,  6.30s/it]
 73%|███████▎  | 163156/225000 [315:46:13<97:58:44,  5.70s/it] 
 73%|███████▎  | 163157/225000 [315:46:20<103:09:48,  6.01s/it]
 73%|███████▎  | 163158/225000 [315:46:24<93:37:28,  5.45s/it] 
 73%|███████▎  | 163159/225000 [315:46:28<88:04:45,  5.13s/it]
 73%|███████▎  | 163160/225000 [315:46:33<83:53:22,  4.88s/it]
 73%|███████▎  | 163161/225000 [315:46:39<90:28:05,  5.27s/it]
 73%|███████▎  | 163162/225000 [315:46:48<107:34:58,  6.26s/it]
 73%|███████▎  | 163163/225000 [315:46:51<94:55:47,  5.53s/it] 
 73%|███████▎  | 163164/225000 [315:46:55<87:25:31,  5.09s/it]
 73%|███████▎  | 163165/225000 [315:47:06<116:49:25,  6.80s/it]
 73%|███████▎  | 163166/225000 [315:47:10<101:00:15,  5.88s/it]
 73%|███████▎  | 163167/225000 [315:47:14<89:23:05,  5.20s/it] 
 73%|███████▎  | 163168/225000 [315:47:18<86:34:36,  5.04s/it]
 73%|███████▎  | 163169/225000 [315:47:28<110:00:44,  6.41s/it]
 73%|███████▎  | 163170/225000 [315:47:31<95:32:13,  5.56s/it] 
 73%|███████▎  | 163171/225000 [315:47:37<94:55:51,  5.53s/it]
 73%|███████▎  | 163172/225000 [315:47:43<95:47:08,  5.58s/it]
 73%|███████▎  | 163173/225000 [315:47:48<94:36:05,  5.51s/it]
 73%|███████▎  | 163174/225000 [315:47:55<103:20:18,  6.02s/it]
 73%|███████▎  | 163175/225000 [315:47:59<91:29:14,  5.33s/it] 
 73%|███████▎  | 163176/225000 [315:48:06<102:19:44,  5.96s/it]
 73%|███████▎  | 163177/225000 [315:48:12<99:10:59,  5.78s/it] 
 73%|███████▎  | 163178/225000 [315:48:18<101:09:58,  5.89s/it]
 73%|███████▎  | 163179/225000 [315:48:21<88:57:39,  5.18s/it] 
 73%|███████▎  | 163180/225000 [315:48:25<82:38:16,  4.81s/it]
 73%|███████▎  | 163181/225000 [315:48:29<79:48:46,  4.65s/it]
 73%|███████▎  | 163182/225000 [315:48:39<103:17:49,  6.02s/it]
 73%|███████▎  | 163183/225000 [315:48:43<96:46:14,  5.64s/it] 
 73%|███████▎  | 163184/225000 [315:48:50<100:59:35,  5.88s/it]
 73%|███████▎  | 163185/225000 [315:48:53<87:47:30,  5.11s/it] 
 73%|███████▎  | 163186/225000 [315:49:00<98:57:26,  5.76s/it]
 73%|███████▎  | 163187/225000 [315:49:03<84:36:22,  4.93s/it]
 73%|███████▎  | 163188/225000 [315:49:11<97:20:28,  5.67s/it]
 73%|███████▎  | 163189/225000 [315:49:15<87:20:20,  5.09s/it]
 73%|███████▎  | 163190/225000 [315:49:19<83:14:43,  4.85s/it]
 73%|███████▎  | 163191/225000 [315:49:24<83:02:54,  4.84s/it]
 73%|███████▎  | 163192/225000 [315:49:28<81:38:32,  4.76s/it]
 73%|███████▎  | 163193/225000 [315:49:40<118:26:41,  6.90s/it]
 73%|███████▎  | 163194/225000 [315:49:44<100:17:38,  5.84s/it]
 73%|███████▎  | 163195/225000 [315:49:51<109:01:25,  6.35s/it]
 73%|███████▎  | 163196/225000 [315:49:55<97:17:48,  5.67s/it] 
 73%|███████▎  | 163197/225000 [315:49:59<89:59:29,  5.24s/it]
 73%|███████▎  | 163198/225000 [315:50:04<84:55:50,  4.95s/it]
 73%|███████▎  | 163199/225000 [315:50:10<94:19:43,  5.49s/it]
 73%|███████▎  | 163200/225000 [315:50:20<114:16:00,  6.66s/it]
                                                               
{'loss': '0.7677', 'grad_norm': '1.171', 'learning_rate': '9.541e-06', 'epoch': '0.7253', 'num_input_tokens_seen': 522839152, 'train_runtime': '1.137e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163200/225000 [315:50:20<114:16:00,  6.66s/it]
 73%|███████▎  | 163201/225000 [315:50:23<96:53:14,  5.64s/it] 
 73%|███████▎  | 163202/225000 [315:50:32<114:27:15,  6.67s/it]
 73%|███████▎  | 163203/225000 [315:50:35<97:18:18,  5.67s/it] 
 73%|███████▎  | 163204/225000 [315:50:42<102:38:45,  5.98s/it]
 73%|███████▎  | 163205/225000 [315:50:45<88:52:03,  5.18s/it] 
 73%|███████▎  | 163206/225000 [315:50:54<104:40:09,  6.10s/it]
 73%|███████▎  | 163207/225000 [315:50:57<90:55:05,  5.30s/it] 
 73%|███████▎  | 163208/225000 [315:51:05<104:52:21,  6.11s/it]
 73%|███████▎  | 163209/225000 [315:51:09<90:47:36,  5.29s/it] 
 73%|███████▎  | 163210/225000 [315:51:12<82:54:30,  4.83s/it]
 73%|███████▎  | 163211/225000 [315:51:19<91:42:39,  5.34s/it]
 73%|███████▎  | 163212/225000 [315:51:26<101:53:48,  5.94s/it]
 73%|███████▎  | 163213/225000 [315:51:32<101:20:57,  5.91s/it]
 73%|███████▎  | 163214/225000 [315:51:36<89:29:27,  5.21s/it] 
 73%|███████▎  | 163215/225000 [315:51:43<98:22:59,  5.73s/it]
 73%|███████▎  | 163216/225000 [315:51:47<91:26:34,  5.33s/it]
 73%|███████▎  | 163217/225000 [315:51:53<93:26:19,  5.44s/it]
 73%|███████▎  | 163218/225000 [315:51:58<94:05:32,  5.48s/it]
 73%|███████▎  | 163219/225000 [315:52:03<90:07:52,  5.25s/it]
 73%|███████▎  | 163220/225000 [315:52:08<88:00:00,  5.13s/it]
 73%|███████▎  | 163221/225000 [315:52:13<86:14:59,  5.03s/it]
 73%|███████▎  | 163222/225000 [315:52:17<84:17:57,  4.91s/it]
 73%|███████▎  | 163223/225000 [315:52:22<83:29:04,  4.86s/it]
 73%|███████▎  | 163224/225000 [315:52:27<82:46:16,  4.82s/it]
 73%|███████▎  | 163225/225000 [315:52:31<81:31:34,  4.75s/it]
 73%|███████▎  | 163226/225000 [315:52:42<111:14:20,  6.48s/it]
 73%|███████▎  | 163227/225000 [315:52:48<109:04:11,  6.36s/it]
 73%|███████▎  | 163228/225000 [315:52:51<94:04:37,  5.48s/it] 
 73%|███████▎  | 163229/225000 [315:52:55<84:01:30,  4.90s/it]
 73%|███████▎  | 163230/225000 [315:53:05<111:22:45,  6.49s/it]
 73%|███████▎  | 163231/225000 [315:53:09<99:13:01,  5.78s/it] 
 73%|███████▎  | 163232/225000 [315:53:14<95:01:50,  5.54s/it]
 73%|███████▎  | 163233/225000 [315:53:22<105:06:41,  6.13s/it]
 73%|███████▎  | 163234/225000 [315:53:26<94:30:10,  5.51s/it] 
 73%|███████▎  | 163235/225000 [315:53:34<107:49:45,  6.28s/it]
 73%|███████▎  | 163236/225000 [315:53:38<98:55:57,  5.77s/it] 
 73%|███████▎  | 163237/225000 [315:53:45<104:50:04,  6.11s/it]
 73%|███████▎  | 163238/225000 [315:53:53<113:16:00,  6.60s/it]
 73%|███████▎  | 163239/225000 [315:53:57<98:42:02,  5.75s/it] 
 73%|███████▎  | 163240/225000 [315:54:01<88:49:12,  5.18s/it]
 73%|███████▎  | 163241/225000 [315:54:05<82:58:57,  4.84s/it]
 73%|███████▎  | 163242/225000 [315:54:12<96:22:47,  5.62s/it]
 73%|███████▎  | 163243/225000 [315:54:20<106:57:25,  6.23s/it]
 73%|███████▎  | 163244/225000 [315:54:23<92:39:21,  5.40s/it] 
 73%|███████▎  | 163245/225000 [315:54:30<100:02:55,  5.83s/it]
 73%|███████▎  | 163246/225000 [315:54:34<91:22:00,  5.33s/it] 
 73%|███████▎  | 163247/225000 [315:54:38<84:50:44,  4.95s/it]
 73%|███████▎  | 163248/225000 [315:54:43<81:18:56,  4.74s/it]
 73%|███████▎  | 163249/225000 [315:54:54<114:38:32,  6.68s/it]
 73%|███████▎  | 163250/225000 [315:54:58<102:33:48,  5.98s/it]
                                                               
{'loss': '0.7796', 'grad_norm': '1.142', 'learning_rate': '9.527e-06', 'epoch': '0.7256', 'num_input_tokens_seen': 522970136, 'train_runtime': '1.137e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163250/225000 [315:54:58<102:33:48,  5.98s/it]
 73%|███████▎  | 163251/225000 [315:55:02<91:01:03,  5.31s/it] 
 73%|███████▎  | 163252/225000 [315:55:11<110:52:47,  6.46s/it]
 73%|███████▎  | 163253/225000 [315:55:16<101:14:39,  5.90s/it]
 73%|███████▎  | 163254/225000 [315:55:21<96:34:55,  5.63s/it] 
 73%|███████▎  | 163255/225000 [315:55:25<90:14:52,  5.26s/it]
 73%|███████▎  | 163256/225000 [315:55:30<87:43:52,  5.12s/it]
 73%|███████▎  | 163257/225000 [315:55:37<100:05:59,  5.84s/it]
 73%|███████▎  | 163258/225000 [315:55:45<111:39:03,  6.51s/it]
 73%|███████▎  | 163259/225000 [315:55:50<103:42:51,  6.05s/it]
 73%|███████▎  | 163260/225000 [315:55:56<99:57:27,  5.83s/it] 
 73%|███████▎  | 163261/225000 [315:56:01<95:06:31,  5.55s/it]
 73%|███████▎  | 163262/225000 [315:56:05<91:28:44,  5.33s/it]
 73%|███████▎  | 163263/225000 [315:56:11<92:52:46,  5.42s/it]
 73%|███████▎  | 163264/225000 [315:56:16<93:29:23,  5.45s/it]
 73%|███████▎  | 163265/225000 [315:56:21<90:30:27,  5.28s/it]
 73%|███████▎  | 163266/225000 [315:56:26<88:55:40,  5.19s/it]
 73%|███████▎  | 163267/225000 [315:56:31<85:44:17,  5.00s/it]
 73%|███████▎  | 163268/225000 [315:56:37<92:40:46,  5.40s/it]
 73%|███████▎  | 163269/225000 [315:56:50<128:39:05,  7.50s/it]
 73%|███████▎  | 163270/225000 [315:56:53<107:02:00,  6.24s/it]
 73%|███████▎  | 163271/225000 [315:57:00<113:23:41,  6.61s/it]
 73%|███████▎  | 163272/225000 [315:57:04<99:04:49,  5.78s/it] 
 73%|███████▎  | 163273/225000 [315:57:12<107:47:23,  6.29s/it]
 73%|███████▎  | 163274/225000 [315:57:15<94:04:12,  5.49s/it] 
 73%|███████▎  | 163275/225000 [315:57:19<85:21:57,  4.98s/it]
 73%|███████▎  | 163276/225000 [315:57:28<107:28:56,  6.27s/it]
 73%|███████▎  | 163277/225000 [315:57:32<93:19:11,  5.44s/it] 
 73%|███████▎  | 163278/225000 [315:57:40<107:03:27,  6.24s/it]
 73%|███████▎  | 163279/225000 [315:57:44<95:54:24,  5.59s/it] 
 73%|███████▎  | 163280/225000 [315:57:50<96:24:12,  5.62s/it]
 73%|███████▎  | 163281/225000 [315:57:53<84:11:53,  4.91s/it]
 73%|███████▎  | 163282/225000 [315:58:02<106:05:11,  6.19s/it]
 73%|███████▎  | 163283/225000 [315:58:06<94:05:27,  5.49s/it] 
 73%|███████▎  | 163284/225000 [315:58:13<101:41:17,  5.93s/it]
 73%|███████▎  | 163285/225000 [315:58:20<105:58:51,  6.18s/it]
 73%|███████▎  | 163286/225000 [315:58:24<96:01:14,  5.60s/it] 
 73%|███████▎  | 163287/225000 [315:58:28<89:30:45,  5.22s/it]
 73%|███████▎  | 163288/225000 [315:58:33<84:19:33,  4.92s/it]
 73%|███████▎  | 163289/225000 [315:58:37<80:21:56,  4.69s/it]
 73%|███████▎  | 163290/225000 [315:58:46<103:25:46,  6.03s/it]
 73%|███████▎  | 163291/225000 [315:58:51<96:16:51,  5.62s/it] 
 73%|███████▎  | 163292/225000 [315:58:56<93:49:04,  5.47s/it]
 73%|███████▎  | 163293/225000 [315:59:02<99:05:44,  5.78s/it]
 73%|███████▎  | 163294/225000 [315:59:06<91:08:41,  5.32s/it]
 73%|███████▎  | 163295/225000 [315:59:11<86:07:38,  5.02s/it]
 73%|███████▎  | 163296/225000 [315:59:17<91:03:10,  5.31s/it]
 73%|███████▎  | 163297/225000 [315:59:26<111:58:17,  6.53s/it]
 73%|███████▎  | 163298/225000 [315:59:29<95:22:09,  5.56s/it] 
 73%|███████▎  | 163299/225000 [315:59:38<111:39:23,  6.51s/it]
 73%|███████▎  | 163300/225000 [315:59:41<94:53:58,  5.54s/it] 
                                                              
{'loss': '0.7736', 'grad_norm': '1.069', 'learning_rate': '9.512e-06', 'epoch': '0.7258', 'num_input_tokens_seen': 523100208, 'train_runtime': '1.138e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163300/225000 [315:59:41<94:53:58,  5.54s/it]
 73%|███████▎  | 163301/225000 [315:59:48<98:24:33,  5.74s/it]
 73%|███████▎  | 163302/225000 [315:59:53<98:45:57,  5.76s/it]
 73%|███████▎  | 163303/225000 [316:00:01<106:49:58,  6.23s/it]
 73%|███████▎  | 163304/225000 [316:00:05<96:57:06,  5.66s/it] 
 73%|███████▎  | 163305/225000 [316:00:10<90:43:20,  5.29s/it]
 73%|███████▎  | 163306/225000 [316:00:14<85:28:07,  4.99s/it]
 73%|███████▎  | 163307/225000 [316:00:18<82:12:07,  4.80s/it]
 73%|███████▎  | 163308/225000 [316:00:28<106:54:01,  6.24s/it]
 73%|███████▎  | 163309/225000 [316:00:31<93:22:36,  5.45s/it] 
 73%|███████▎  | 163310/225000 [316:00:35<86:05:34,  5.02s/it]
 73%|███████▎  | 163311/225000 [316:00:46<114:09:14,  6.66s/it]
 73%|███████▎  | 163312/225000 [316:00:50<98:39:08,  5.76s/it] 
 73%|███████▎  | 163313/225000 [316:00:54<89:44:20,  5.24s/it]
 73%|███████▎  | 163314/225000 [316:01:02<106:20:48,  6.21s/it]
 73%|███████▎  | 163315/225000 [316:01:06<93:57:19,  5.48s/it] 
 73%|███████▎  | 163316/225000 [316:01:10<87:41:35,  5.12s/it]
 73%|███████▎  | 163317/225000 [316:01:19<107:14:12,  6.26s/it]
 73%|███████▎  | 163318/225000 [316:01:23<93:29:03,  5.46s/it] 
 73%|███████▎  | 163319/225000 [316:01:28<92:45:48,  5.41s/it]
 73%|███████▎  | 163320/225000 [316:01:33<92:51:08,  5.42s/it]
 73%|███████▎  | 163321/225000 [316:01:39<92:53:16,  5.42s/it]
 73%|███████▎  | 163322/225000 [316:01:44<94:15:00,  5.50s/it]
 73%|███████▎  | 163323/225000 [316:01:49<91:20:44,  5.33s/it]
 73%|███████▎  | 163324/225000 [316:01:54<89:20:59,  5.22s/it]
 73%|███████▎  | 163325/225000 [316:01:59<86:22:14,  5.04s/it]
 73%|███████▎  | 163326/225000 [316:02:04<85:40:56,  5.00s/it]
 73%|███████▎  | 163327/225000 [316:02:09<86:17:38,  5.04s/it]
 73%|███████▎  | 163328/225000 [316:02:14<86:10:37,  5.03s/it]
 73%|███████▎  | 163329/225000 [316:02:19<85:05:49,  4.97s/it]
 73%|███████▎  | 163330/225000 [316:02:24<86:27:05,  5.05s/it]
 73%|███████▎  | 163331/225000 [316:02:29<87:25:36,  5.10s/it]
 73%|███████▎  | 163332/225000 [316:02:35<89:17:04,  5.21s/it]
 73%|███████▎  | 163333/225000 [316:02:42<98:18:31,  5.74s/it]
 73%|███████▎  | 163334/225000 [316:02:53<126:46:02,  7.40s/it]
 73%|███████▎  | 163335/225000 [316:02:58<113:06:38,  6.60s/it]
 73%|███████▎  | 163336/225000 [316:03:05<115:01:13,  6.71s/it]
 73%|███████▎  | 163337/225000 [316:03:08<98:17:06,  5.74s/it] 
 73%|███████▎  | 163338/225000 [316:03:15<104:29:42,  6.10s/it]
 73%|███████▎  | 163339/225000 [316:03:22<108:35:47,  6.34s/it]
 73%|███████▎  | 163340/225000 [316:03:27<99:07:54,  5.79s/it] 
 73%|███████▎  | 163341/225000 [316:03:32<94:50:17,  5.54s/it]
 73%|███████▎  | 163342/225000 [316:03:36<90:42:48,  5.30s/it]
 73%|███████▎  | 163343/225000 [316:03:41<89:06:20,  5.20s/it]
 73%|███████▎  | 163344/225000 [316:03:46<85:36:31,  5.00s/it]
 73%|███████▎  | 163345/225000 [316:03:50<83:58:49,  4.90s/it]
 73%|███████▎  | 163346/225000 [316:03:55<83:10:04,  4.86s/it]
 73%|███████▎  | 163347/225000 [316:04:00<81:42:17,  4.77s/it]
 73%|███████▎  | 163348/225000 [316:04:05<81:39:23,  4.77s/it]
 73%|███████▎  | 163349/225000 [316:04:10<82:45:49,  4.83s/it]
 73%|███████▎  | 163350/225000 [316:04:21<119:26:06,  6.97s/it]
                                                               
{'loss': '0.7641', 'grad_norm': '1.178', 'learning_rate': '9.498e-06', 'epoch': '0.726', 'num_input_tokens_seen': 523231120, 'train_runtime': '1.138e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163350/225000 [316:04:21<119:26:06,  6.97s/it]
 73%|███████▎  | 163351/225000 [316:04:28<118:43:56,  6.93s/it]
 73%|███████▎  | 163352/225000 [316:04:34<113:00:23,  6.60s/it]
 73%|███████▎  | 163353/225000 [316:04:40<107:11:13,  6.26s/it]
 73%|███████▎  | 163354/225000 [316:04:45<101:04:54,  5.90s/it]
 73%|███████▎  | 163355/225000 [316:04:50<97:14:28,  5.68s/it] 
 73%|███████▎  | 163356/225000 [316:04:55<92:51:33,  5.42s/it]
 73%|███████▎  | 163357/225000 [316:05:00<93:04:49,  5.44s/it]
 73%|███████▎  | 163358/225000 [316:05:05<92:21:48,  5.39s/it]
 73%|███████▎  | 163359/225000 [316:05:10<88:38:01,  5.18s/it]
 73%|███████▎  | 163360/225000 [316:05:16<91:32:08,  5.35s/it]
 73%|███████▎  | 163361/225000 [316:05:21<91:57:28,  5.37s/it]
 73%|███████▎  | 163362/225000 [316:05:27<91:30:16,  5.34s/it]
 73%|███████▎  | 163363/225000 [316:05:32<91:51:19,  5.36s/it]
 73%|███████▎  | 163364/225000 [316:05:37<92:28:40,  5.40s/it]
 73%|███████▎  | 163365/225000 [316:05:43<93:06:49,  5.44s/it]
 73%|███████▎  | 163366/225000 [316:05:48<90:30:09,  5.29s/it]
 73%|███████▎  | 163367/225000 [316:05:53<87:25:55,  5.11s/it]
 73%|███████▎  | 163368/225000 [316:05:58<88:46:50,  5.19s/it]
 73%|███████▎  | 163369/225000 [316:06:04<91:06:14,  5.32s/it]
 73%|███████▎  | 163370/225000 [316:06:09<90:40:02,  5.30s/it]
 73%|███████▎  | 163371/225000 [316:06:14<91:48:07,  5.36s/it]
 73%|███████▎  | 163372/225000 [316:06:20<92:01:57,  5.38s/it]
 73%|███████▎  | 163373/225000 [316:06:25<92:53:12,  5.43s/it]
 73%|███████▎  | 163374/225000 [316:06:35<112:34:37,  6.58s/it]
 73%|███████▎  | 163375/225000 [316:06:39<102:20:07,  5.98s/it]
 73%|███████▎  | 163376/225000 [316:06:43<93:31:15,  5.46s/it] 
 73%|███████▎  | 163377/225000 [316:06:48<88:30:11,  5.17s/it]
 73%|███████▎  | 163378/225000 [316:06:59<120:45:10,  7.05s/it]
 73%|███████▎  | 163379/225000 [316:07:06<118:16:52,  6.91s/it]
 73%|███████▎  | 163380/225000 [316:07:12<115:59:03,  6.78s/it]
 73%|███████▎  | 163381/225000 [316:07:16<98:42:33,  5.77s/it] 
 73%|███████▎  | 163382/225000 [316:07:23<104:29:12,  6.10s/it]
 73%|███████▎  | 163383/225000 [316:07:27<96:43:15,  5.65s/it] 
 73%|███████▎  | 163384/225000 [316:07:31<86:34:06,  5.06s/it]
 73%|███████▎  | 163385/225000 [316:07:40<107:00:01,  6.25s/it]
 73%|███████▎  | 163386/225000 [316:07:44<94:05:59,  5.50s/it] 
 73%|███████▎  | 163387/225000 [316:07:48<86:51:26,  5.08s/it]
 73%|███████▎  | 163388/225000 [316:07:55<98:03:03,  5.73s/it]
 73%|███████▎  | 163389/225000 [316:08:02<102:18:01,  5.98s/it]
 73%|███████▎  | 163390/225000 [316:08:06<93:08:47,  5.44s/it] 
 73%|███████▎  | 163391/225000 [316:08:10<84:54:57,  4.96s/it]
 73%|███████▎  | 163392/225000 [316:08:16<94:18:39,  5.51s/it]
 73%|███████▎  | 163393/225000 [316:08:24<105:17:17,  6.15s/it]
 73%|███████▎  | 163394/225000 [316:08:28<92:53:57,  5.43s/it] 
 73%|███████▎  | 163395/225000 [316:08:36<106:18:45,  6.21s/it]
 73%|███████▎  | 163396/225000 [316:08:40<96:54:10,  5.66s/it] 
 73%|███████▎  | 163397/225000 [316:08:47<101:20:11,  5.92s/it]
 73%|███████▎  | 163398/225000 [316:08:50<89:15:28,  5.22s/it] 
 73%|███████▎  | 163399/225000 [316:08:56<92:59:20,  5.43s/it]
 73%|███████▎  | 163400/225000 [316:09:05<109:20:03,  6.39s/it]
                                                               
{'loss': '0.7747', 'grad_norm': '1.091', 'learning_rate': '9.484e-06', 'epoch': '0.7262', 'num_input_tokens_seen': 523361288, 'train_runtime': '1.138e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163400/225000 [316:09:05<109:20:03,  6.39s/it]
 73%|███████▎  | 163401/225000 [316:09:08<93:06:21,  5.44s/it] 
 73%|███████▎  | 163402/225000 [316:09:12<86:17:52,  5.04s/it]
 73%|███████▎  | 163403/225000 [316:09:16<80:45:41,  4.72s/it]
 73%|███████▎  | 163404/225000 [316:09:21<79:00:01,  4.62s/it]
 73%|███████▎  | 163405/225000 [316:09:32<116:08:45,  6.79s/it]
 73%|███████▎  | 163406/225000 [316:09:37<102:00:32,  5.96s/it]
 73%|███████▎  | 163407/225000 [316:09:42<101:48:27,  5.95s/it]
 73%|███████▎  | 163408/225000 [316:09:46<90:35:38,  5.30s/it] 
 73%|███████▎  | 163409/225000 [316:09:51<90:22:52,  5.28s/it]
 73%|███████▎  | 163410/225000 [316:10:01<111:14:57,  6.50s/it]
 73%|███████▎  | 163411/225000 [316:10:05<97:53:10,  5.72s/it] 
 73%|███████▎  | 163412/225000 [316:10:08<87:53:29,  5.14s/it]
 73%|███████▎  | 163413/225000 [316:10:13<82:25:34,  4.82s/it]
 73%|███████▎  | 163414/225000 [316:10:22<107:45:17,  6.30s/it]
 73%|███████▎  | 163415/225000 [316:10:28<106:38:57,  6.23s/it]
 73%|███████▎  | 163416/225000 [316:10:34<101:41:41,  5.94s/it]
 73%|███████▎  | 163417/225000 [316:10:39<96:07:44,  5.62s/it] 
 73%|███████▎  | 163418/225000 [316:10:44<96:34:47,  5.65s/it]
 73%|███████▎  | 163419/225000 [316:10:49<93:41:09,  5.48s/it]
 73%|███████▎  | 163420/225000 [316:10:55<92:21:02,  5.40s/it]
 73%|███████▎  | 163421/225000 [316:11:00<90:14:15,  5.28s/it]
 73%|███████▎  | 163422/225000 [316:11:05<89:31:58,  5.23s/it]
 73%|███████▎  | 163423/225000 [316:11:10<88:05:56,  5.15s/it]
 73%|███████▎  | 163424/225000 [316:11:15<87:51:27,  5.14s/it]
 73%|███████▎  | 163425/225000 [316:11:20<88:48:21,  5.19s/it]
 73%|███████▎  | 163426/225000 [316:11:25<89:23:44,  5.23s/it]
 73%|███████▎  | 163427/225000 [316:11:31<89:32:34,  5.24s/it]
 73%|███████▎  | 163428/225000 [316:11:42<123:04:33,  7.20s/it]
 73%|███████▎  | 163429/225000 [316:11:47<108:04:52,  6.32s/it]
 73%|███████▎  | 163430/225000 [316:11:50<95:11:54,  5.57s/it] 
 73%|███████▎  | 163431/225000 [316:11:54<87:02:20,  5.09s/it]
 73%|███████▎  | 163432/225000 [316:11:59<86:24:10,  5.05s/it]
 73%|███████▎  | 163433/225000 [316:12:10<115:25:47,  6.75s/it]
 73%|███████▎  | 163434/225000 [316:12:14<101:36:33,  5.94s/it]
 73%|███████▎  | 163435/225000 [316:12:19<93:53:03,  5.49s/it] 
 73%|███████▎  | 163436/225000 [316:12:23<88:06:25,  5.15s/it]
 73%|███████▎  | 163437/225000 [316:12:28<85:18:42,  4.99s/it]
 73%|███████▎  | 163438/225000 [316:12:35<98:49:56,  5.78s/it]
 73%|███████▎  | 163439/225000 [316:12:44<115:42:18,  6.77s/it]
 73%|███████▎  | 163440/225000 [316:12:48<99:00:21,  5.79s/it] 
 73%|███████▎  | 163441/225000 [316:12:55<107:30:05,  6.29s/it]
 73%|███████▎  | 163442/225000 [316:12:58<91:37:22,  5.36s/it] 
 73%|███████▎  | 163443/225000 [316:13:05<97:07:22,  5.68s/it]
 73%|███████▎  | 163444/225000 [316:13:09<87:23:18,  5.11s/it]
 73%|███████▎  | 163445/225000 [316:13:17<106:12:08,  6.21s/it]
 73%|███████▎  | 163446/225000 [316:13:21<94:27:18,  5.52s/it] 
 73%|███████▎  | 163447/225000 [316:13:25<85:59:52,  5.03s/it]
 73%|███████▎  | 163448/225000 [316:13:29<80:06:03,  4.68s/it]
 73%|███████▎  | 163449/225000 [316:13:36<91:21:00,  5.34s/it]
 73%|███████▎  | 163450/225000 [316:13:44<105:09:04,  6.15s/it]
                                                               
{'loss': '0.7746', 'grad_norm': '1.054', 'learning_rate': '9.469e-06', 'epoch': '0.7264', 'num_input_tokens_seen': 523491312, 'train_runtime': '1.138e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163450/225000 [316:13:44<105:09:04,  6.15s/it]
 73%|███████▎  | 163451/225000 [316:13:49<97:48:35,  5.72s/it] 
 73%|███████▎  | 163452/225000 [316:13:56<106:05:21,  6.21s/it]
 73%|███████▎  | 163453/225000 [316:14:00<94:39:54,  5.54s/it] 
 73%|███████▎  | 163454/225000 [316:14:04<88:16:38,  5.16s/it]
 73%|███████▎  | 163455/225000 [316:14:12<101:24:16,  5.93s/it]
 73%|███████▎  | 163456/225000 [316:14:18<103:37:05,  6.06s/it]
 73%|███████▎  | 163457/225000 [316:14:22<90:22:41,  5.29s/it] 
 73%|███████▎  | 163458/225000 [316:14:27<91:43:45,  5.37s/it]
 73%|███████▎  | 163459/225000 [316:14:32<89:22:32,  5.23s/it]
 73%|███████▎  | 163460/225000 [316:14:38<90:07:13,  5.27s/it]
 73%|███████▎  | 163461/225000 [316:14:43<89:33:14,  5.24s/it]
 73%|███████▎  | 163462/225000 [316:14:48<87:40:17,  5.13s/it]
 73%|███████▎  | 163463/225000 [316:14:53<85:53:41,  5.02s/it]
 73%|███████▎  | 163464/225000 [316:14:58<88:06:13,  5.15s/it]
 73%|███████▎  | 163465/225000 [316:15:03<89:07:53,  5.21s/it]
 73%|███████▎  | 163466/225000 [316:15:09<90:09:36,  5.27s/it]
 73%|███████▎  | 163467/225000 [316:15:14<89:16:21,  5.22s/it]
 73%|███████▎  | 163468/225000 [316:15:19<90:48:14,  5.31s/it]
 73%|███████▎  | 163469/225000 [316:15:25<92:42:14,  5.42s/it]
 73%|███████▎  | 163470/225000 [316:15:30<88:28:22,  5.18s/it]
 73%|███████▎  | 163471/225000 [316:15:35<89:17:46,  5.22s/it]
 73%|███████▎  | 163472/225000 [316:15:47<125:03:21,  7.32s/it]
 73%|███████▎  | 163473/225000 [316:15:54<121:35:07,  7.11s/it]
 73%|███████▎  | 163474/225000 [316:15:58<105:22:06,  6.17s/it]
 73%|███████▎  | 163475/225000 [316:16:03<99:11:24,  5.80s/it] 
 73%|███████▎  | 163476/225000 [316:16:13<121:54:59,  7.13s/it]
 73%|███████▎  | 163477/225000 [316:16:17<105:47:37,  6.19s/it]
 73%|███████▎  | 163478/225000 [316:16:21<95:05:40,  5.56s/it] 
 73%|███████▎  | 163479/225000 [316:16:28<104:16:08,  6.10s/it]
 73%|███████▎  | 163480/225000 [316:16:35<105:40:53,  6.18s/it]
 73%|███████▎  | 163481/225000 [316:16:39<94:08:55,  5.51s/it] 
 73%|███████▎  | 163482/225000 [316:16:43<87:21:32,  5.11s/it]
 73%|███████▎  | 163483/225000 [316:16:51<100:16:49,  5.87s/it]
 73%|███████▎  | 163484/225000 [316:16:57<105:12:06,  6.16s/it]
 73%|███████▎  | 163485/225000 [316:17:03<102:49:08,  6.02s/it]
 73%|███████▎  | 163486/225000 [316:17:08<98:18:41,  5.75s/it] 
 73%|███████▎  | 163487/225000 [316:17:12<87:12:51,  5.10s/it]
 73%|███████▎  | 163488/225000 [316:17:17<89:50:42,  5.26s/it]
 73%|███████▎  | 163489/225000 [316:17:25<101:36:53,  5.95s/it]
 73%|███████▎  | 163490/225000 [316:17:30<95:33:35,  5.59s/it] 
 73%|███████▎  | 163491/225000 [316:17:35<92:29:22,  5.41s/it]
 73%|███████▎  | 163492/225000 [316:17:40<89:45:20,  5.25s/it]
 73%|███████▎  | 163493/225000 [316:17:44<84:58:51,  4.97s/it]
 73%|███████▎  | 163494/225000 [316:17:49<83:48:14,  4.91s/it]
 73%|███████▎  | 163495/225000 [316:17:54<84:00:35,  4.92s/it]
 73%|███████▎  | 163496/225000 [316:17:59<85:49:07,  5.02s/it]
 73%|███████▎  | 163497/225000 [316:18:04<88:12:46,  5.16s/it]
 73%|███████▎  | 163498/225000 [316:18:09<87:00:56,  5.09s/it]
 73%|███████▎  | 163499/225000 [316:18:14<86:25:02,  5.06s/it]
 73%|███████▎  | 163500/225000 [316:18:19<85:20:48,  5.00s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.412', 'learning_rate': '9.455e-06', 'epoch': '0.7267', 'num_input_tokens_seen': 523621416, 'train_runtime': '1.139e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163500/225000 [316:18:19<85:20:48,  5.00s/it]
 73%|███████▎  | 163501/225000 [316:18:24<84:11:20,  4.93s/it]
 73%|███████▎  | 163502/225000 [316:18:28<81:55:44,  4.80s/it]
 73%|███████▎  | 163503/225000 [316:18:36<95:43:24,  5.60s/it]
 73%|███████▎  | 163504/225000 [316:18:47<122:55:47,  7.20s/it]
 73%|███████▎  | 163505/225000 [316:18:50<103:04:34,  6.03s/it]
 73%|███████▎  | 163506/225000 [316:18:55<96:08:35,  5.63s/it] 
 73%|███████▎  | 163507/225000 [316:19:00<92:52:43,  5.44s/it]
 73%|███████▎  | 163508/225000 [316:19:05<89:37:35,  5.25s/it]
 73%|███████▎  | 163509/225000 [316:19:09<86:48:58,  5.08s/it]
 73%|███████▎  | 163510/225000 [316:19:14<82:51:19,  4.85s/it]
 73%|███████▎  | 163511/225000 [316:19:18<81:44:03,  4.79s/it]
 73%|███████▎  | 163512/225000 [316:19:27<101:38:50,  5.95s/it]
 73%|███████▎  | 163513/225000 [316:19:35<113:23:29,  6.64s/it]
 73%|███████▎  | 163514/225000 [316:19:41<110:20:57,  6.46s/it]
 73%|███████▎  | 163515/225000 [316:19:45<98:59:33,  5.80s/it] 
 73%|███████▎  | 163516/225000 [316:19:49<87:23:01,  5.12s/it]
 73%|███████▎  | 163517/225000 [316:19:59<112:54:15,  6.61s/it]
 73%|███████▎  | 163518/225000 [316:20:03<97:06:59,  5.69s/it] 
 73%|███████▎  | 163519/225000 [316:20:10<107:45:10,  6.31s/it]
 73%|███████▎  | 163520/225000 [316:20:14<93:55:57,  5.50s/it] 
 73%|███████▎  | 163521/225000 [316:20:20<96:37:59,  5.66s/it]
 73%|███████▎  | 163522/225000 [316:20:26<96:03:34,  5.63s/it]
 73%|███████▎  | 163523/225000 [316:20:30<89:09:48,  5.22s/it]
 73%|███████▎  | 163524/225000 [316:20:34<84:33:13,  4.95s/it]
 73%|███████▎  | 163525/225000 [316:20:45<112:38:14,  6.60s/it]
 73%|███████▎  | 163526/225000 [316:20:49<100:43:22,  5.90s/it]
 73%|███████▎  | 163527/225000 [316:20:53<91:48:23,  5.38s/it] 
 73%|███████▎  | 163528/225000 [316:20:57<85:37:32,  5.01s/it]
 73%|███████▎  | 163529/225000 [316:21:04<94:58:30,  5.56s/it]
 73%|███████▎  | 163530/225000 [316:21:13<113:36:46,  6.65s/it]
 73%|███████▎  | 163531/225000 [316:21:17<96:26:13,  5.65s/it] 
 73%|███████▎  | 163532/225000 [316:21:24<106:47:35,  6.25s/it]
 73%|███████▎  | 163533/225000 [316:21:27<91:34:45,  5.36s/it] 
 73%|███████▎  | 163534/225000 [316:21:34<97:59:25,  5.74s/it]
 73%|███████▎  | 163535/225000 [316:21:38<88:10:51,  5.16s/it]
 73%|███████▎  | 163536/225000 [316:21:47<107:50:12,  6.32s/it]
 73%|███████▎  | 163537/225000 [316:21:51<97:50:12,  5.73s/it] 
 73%|███████▎  | 163538/225000 [316:21:56<90:48:35,  5.32s/it]
 73%|███████▎  | 163539/225000 [316:22:00<86:34:04,  5.07s/it]
 73%|███████▎  | 163540/225000 [316:22:05<83:52:42,  4.91s/it]
 73%|███████▎  | 163541/225000 [316:22:09<82:09:44,  4.81s/it]
 73%|███████▎  | 163542/225000 [316:22:21<118:34:03,  6.95s/it]
 73%|███████▎  | 163543/225000 [316:22:25<102:28:40,  6.00s/it]
 73%|███████▎  | 163544/225000 [316:22:31<103:01:27,  6.04s/it]
 73%|███████▎  | 163545/225000 [316:22:36<95:16:21,  5.58s/it] 
 73%|███████▎  | 163546/225000 [316:22:43<103:26:03,  6.06s/it]
 73%|███████▎  | 163547/225000 [316:22:46<88:58:33,  5.21s/it] 
 73%|███████▎  | 163548/225000 [316:22:54<104:58:20,  6.15s/it]
 73%|███████▎  | 163549/225000 [316:22:58<92:48:59,  5.44s/it] 
 73%|███████▎  | 163550/225000 [316:23:02<85:24:37,  5.00s/it]
                                                              
{'loss': '0.7653', 'grad_norm': '1.16', 'learning_rate': '9.441e-06', 'epoch': '0.7269', 'num_input_tokens_seen': 523751320, 'train_runtime': '1.139e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163550/225000 [316:23:02<85:24:37,  5.00s/it]
 73%|███████▎  | 163551/225000 [316:23:07<85:10:10,  4.99s/it]
 73%|███████▎  | 163552/225000 [316:23:18<113:11:32,  6.63s/it]
 73%|███████▎  | 163553/225000 [316:23:21<96:38:15,  5.66s/it] 
 73%|███████▎  | 163554/225000 [316:23:26<92:25:30,  5.42s/it]
 73%|███████▎  | 163555/225000 [316:23:31<91:09:17,  5.34s/it]
 73%|███████▎  | 163556/225000 [316:23:36<88:30:35,  5.19s/it]
 73%|███████▎  | 163557/225000 [316:23:40<85:32:02,  5.01s/it]
 73%|███████▎  | 163558/225000 [316:23:45<85:31:32,  5.01s/it]
 73%|███████▎  | 163559/225000 [316:23:50<83:43:00,  4.91s/it]
 73%|███████▎  | 163560/225000 [316:23:55<84:56:50,  4.98s/it]
 73%|███████▎  | 163561/225000 [316:24:00<83:33:26,  4.90s/it]
 73%|███████▎  | 163562/225000 [316:24:11<115:29:43,  6.77s/it]
 73%|███████▎  | 163563/225000 [316:24:17<110:03:23,  6.45s/it]
 73%|███████▎  | 163564/225000 [316:24:20<95:02:21,  5.57s/it] 
 73%|███████▎  | 163565/225000 [316:24:25<90:17:13,  5.29s/it]
 73%|███████▎  | 163566/225000 [316:24:29<86:36:01,  5.07s/it]
 73%|███████▎  | 163567/225000 [316:24:35<89:54:47,  5.27s/it]
 73%|███████▎  | 163568/225000 [316:24:46<117:12:31,  6.87s/it]
 73%|███████▎  | 163569/225000 [316:24:49<100:13:53,  5.87s/it]
 73%|███████▎  | 163570/225000 [316:24:53<87:55:54,  5.15s/it] 
 73%|███████▎  | 163571/225000 [316:25:03<114:36:05,  6.72s/it]
 73%|███████▎  | 163572/225000 [316:25:07<98:10:05,  5.75s/it] 
 73%|███████▎  | 163573/225000 [316:25:15<110:41:13,  6.49s/it]
 73%|███████▎  | 163574/225000 [316:25:19<99:22:48,  5.82s/it] 
 73%|███████▎  | 163575/225000 [316:25:27<108:16:47,  6.35s/it]
 73%|███████▎  | 163576/225000 [316:25:31<96:11:56,  5.64s/it] 
 73%|███████▎  | 163577/225000 [316:25:35<86:41:49,  5.08s/it]
 73%|███████▎  | 163578/225000 [316:25:38<80:10:13,  4.70s/it]
 73%|███████▎  | 163579/225000 [316:25:42<75:02:56,  4.40s/it]
 73%|███████▎  | 163580/225000 [316:25:46<74:19:34,  4.36s/it]
 73%|███████▎  | 163581/225000 [316:25:59<116:14:28,  6.81s/it]
 73%|███████▎  | 163582/225000 [316:26:02<98:06:51,  5.75s/it] 
 73%|███████▎  | 163583/225000 [316:26:10<108:31:44,  6.36s/it]
 73%|███████▎  | 163584/225000 [316:26:13<93:25:35,  5.48s/it] 
 73%|███████▎  | 163585/225000 [316:26:17<86:18:08,  5.06s/it]
 73%|███████▎  | 163586/225000 [316:26:25<97:56:35,  5.74s/it]
 73%|███████▎  | 163587/225000 [316:26:29<89:54:41,  5.27s/it]
 73%|███████▎  | 163588/225000 [316:26:36<98:08:55,  5.75s/it]
 73%|███████▎  | 163589/225000 [316:26:41<97:56:38,  5.74s/it]
 73%|███████▎  | 163590/225000 [316:26:45<88:29:38,  5.19s/it]
 73%|███████▎  | 163591/225000 [316:26:55<111:54:06,  6.56s/it]
 73%|███████▎  | 163592/225000 [316:27:01<106:38:32,  6.25s/it]
 73%|███████▎  | 163593/225000 [316:27:04<93:11:15,  5.46s/it] 
 73%|███████▎  | 163594/225000 [316:27:10<94:42:21,  5.55s/it]
 73%|███████▎  | 163595/225000 [316:27:16<94:48:12,  5.56s/it]
 73%|███████▎  | 163596/225000 [316:27:23<103:19:06,  6.06s/it]
 73%|███████▎  | 163597/225000 [316:27:26<88:51:05,  5.21s/it] 
 73%|███████▎  | 163598/225000 [316:27:30<81:01:23,  4.75s/it]
 73%|███████▎  | 163599/225000 [316:27:34<78:45:38,  4.62s/it]
 73%|███████▎  | 163600/225000 [316:27:38<77:00:16,  4.51s/it]
                                                              
{'loss': '0.777', 'grad_norm': '1.155', 'learning_rate': '9.426e-06', 'epoch': '0.7271', 'num_input_tokens_seen': 523880920, 'train_runtime': '1.139e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163600/225000 [316:27:38<77:00:16,  4.51s/it]
 73%|███████▎  | 163601/225000 [316:27:44<82:02:30,  4.81s/it]
 73%|███████▎  | 163602/225000 [316:27:54<110:09:55,  6.46s/it]
 73%|███████▎  | 163603/225000 [316:27:58<98:17:38,  5.76s/it] 
 73%|███████▎  | 163604/225000 [316:28:06<106:45:14,  6.26s/it]
 73%|███████▎  | 163605/225000 [316:28:11<100:24:46,  5.89s/it]
 73%|███████▎  | 163606/225000 [316:28:15<93:25:14,  5.48s/it] 
 73%|███████▎  | 163607/225000 [316:28:20<90:05:22,  5.28s/it]
 73%|███████▎  | 163608/225000 [316:28:25<88:57:37,  5.22s/it]
 73%|███████▎  | 163609/225000 [316:28:30<86:44:12,  5.09s/it]
 73%|███████▎  | 163610/225000 [316:28:35<87:37:28,  5.14s/it]
 73%|███████▎  | 163611/225000 [316:28:40<85:52:46,  5.04s/it]
 73%|███████▎  | 163612/225000 [316:28:45<83:46:47,  4.91s/it]
 73%|███████▎  | 163613/225000 [316:28:49<82:15:33,  4.82s/it]
 73%|███████▎  | 163614/225000 [316:28:54<83:04:27,  4.87s/it]
 73%|███████▎  | 163615/225000 [316:28:59<82:15:38,  4.82s/it]
 73%|███████▎  | 163616/225000 [316:29:04<82:43:03,  4.85s/it]
 73%|███████▎  | 163617/225000 [316:29:10<90:53:50,  5.33s/it]
 73%|███████▎  | 163618/225000 [316:29:23<126:53:11,  7.44s/it]
 73%|███████▎  | 163619/225000 [316:29:26<106:30:10,  6.25s/it]
 73%|███████▎  | 163620/225000 [316:29:31<101:01:54,  5.93s/it]
 73%|███████▎  | 163621/225000 [316:29:37<97:56:07,  5.74s/it] 
 73%|███████▎  | 163622/225000 [316:29:42<94:49:43,  5.56s/it]
 73%|███████▎  | 163623/225000 [316:29:50<110:06:07,  6.46s/it]
 73%|███████▎  | 163624/225000 [316:29:55<99:54:17,  5.86s/it] 
 73%|███████▎  | 163625/225000 [316:29:59<90:22:43,  5.30s/it]
 73%|███████▎  | 163626/225000 [316:30:02<81:59:58,  4.81s/it]
 73%|███████▎  | 163627/225000 [316:30:12<104:07:43,  6.11s/it]
 73%|███████▎  | 163628/225000 [316:30:17<102:49:34,  6.03s/it]
 73%|███████▎  | 163629/225000 [316:30:21<89:52:54,  5.27s/it] 
 73%|███████▎  | 163630/225000 [316:30:25<82:26:28,  4.84s/it]
 73%|███████▎  | 163631/225000 [316:30:35<112:18:45,  6.59s/it]
 73%|███████▎  | 163632/225000 [316:30:39<98:18:32,  5.77s/it] 
 73%|███████▎  | 163633/225000 [316:30:43<88:10:53,  5.17s/it]
 73%|███████▎  | 163634/225000 [316:30:47<82:52:50,  4.86s/it]
 73%|███████▎  | 163635/225000 [316:30:56<101:28:31,  5.95s/it]
 73%|███████▎  | 163636/225000 [316:31:02<103:46:20,  6.09s/it]
 73%|███████▎  | 163637/225000 [316:31:06<91:35:38,  5.37s/it] 
 73%|███████▎  | 163638/225000 [316:31:10<85:51:45,  5.04s/it]
 73%|███████▎  | 163639/225000 [316:31:20<112:01:01,  6.57s/it]
 73%|███████▎  | 163640/225000 [316:31:23<95:18:27,  5.59s/it] 
 73%|███████▎  | 163641/225000 [316:31:31<107:18:27,  6.30s/it]
 73%|███████▎  | 163642/225000 [316:31:35<92:43:55,  5.44s/it] 
 73%|███████▎  | 163643/225000 [316:31:38<81:40:08,  4.79s/it]
 73%|███████▎  | 163644/225000 [316:31:47<101:35:47,  5.96s/it]
 73%|███████▎  | 163645/225000 [316:31:52<96:11:01,  5.64s/it] 
 73%|███████▎  | 163646/225000 [316:31:56<90:36:10,  5.32s/it]
 73%|███████▎  | 163647/225000 [316:32:01<89:49:21,  5.27s/it]
 73%|███████▎  | 163648/225000 [316:32:07<90:39:54,  5.32s/it]
 73%|███████▎  | 163649/225000 [316:32:12<91:34:45,  5.37s/it]
 73%|███████▎  | 163650/225000 [316:32:18<90:43:47,  5.32s/it]
                                                              
{'loss': '0.7759', 'grad_norm': '1.142', 'learning_rate': '9.412e-06', 'epoch': '0.7273', 'num_input_tokens_seen': 524011584, 'train_runtime': '1.14e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163650/225000 [316:32:18<90:43:47,  5.32s/it]
 73%|███████▎  | 163651/225000 [316:32:23<90:35:51,  5.32s/it]
 73%|███████▎  | 163652/225000 [316:32:27<86:38:39,  5.08s/it]
 73%|███████▎  | 163653/225000 [316:32:32<85:39:03,  5.03s/it]
 73%|███████▎  | 163654/225000 [316:32:37<85:30:18,  5.02s/it]
 73%|███████▎  | 163655/225000 [316:32:43<86:29:58,  5.08s/it]
 73%|███████▎  | 163656/225000 [316:32:48<87:10:22,  5.12s/it]
 73%|███████▎  | 163657/225000 [316:32:53<88:59:27,  5.22s/it]
 73%|███████▎  | 163658/225000 [316:32:59<89:18:53,  5.24s/it]
 73%|███████▎  | 163659/225000 [316:33:04<88:31:21,  5.20s/it]
 73%|███████▎  | 163660/225000 [316:33:09<88:22:00,  5.19s/it]
 73%|███████▎  | 163661/225000 [316:33:14<90:35:13,  5.32s/it]
 73%|███████▎  | 163662/225000 [316:33:27<126:15:16,  7.41s/it]
 73%|███████▎  | 163663/225000 [316:33:32<116:57:22,  6.86s/it]
 73%|███████▎  | 163664/225000 [316:33:36<98:55:18,  5.81s/it] 
 73%|███████▎  | 163665/225000 [316:33:39<86:55:22,  5.10s/it]
 73%|███████▎  | 163666/225000 [316:33:49<111:08:09,  6.52s/it]
 73%|███████▎  | 163667/225000 [316:33:53<97:16:15,  5.71s/it] 
 73%|███████▎  | 163668/225000 [316:33:57<87:45:23,  5.15s/it]
 73%|███████▎  | 163669/225000 [316:34:01<82:01:40,  4.81s/it]
 73%|███████▎  | 163670/225000 [316:34:07<88:32:31,  5.20s/it]
 73%|███████▎  | 163671/225000 [316:34:18<118:42:46,  6.97s/it]
 73%|███████▎  | 163672/225000 [316:34:21<101:35:00,  5.96s/it]
 73%|███████▎  | 163673/225000 [316:34:27<98:34:49,  5.79s/it] 
 73%|███████▎  | 163674/225000 [316:34:32<94:03:17,  5.52s/it]
 73%|███████▎  | 163675/225000 [316:34:36<88:31:00,  5.20s/it]
 73%|███████▎  | 163676/225000 [316:34:41<87:27:26,  5.13s/it]
 73%|███████▎  | 163677/225000 [316:34:46<88:19:57,  5.19s/it]
 73%|███████▎  | 163678/225000 [316:34:51<86:41:40,  5.09s/it]
 73%|███████▎  | 163679/225000 [316:34:57<87:52:44,  5.16s/it]
 73%|███████▎  | 163680/225000 [316:35:02<87:16:50,  5.12s/it]
 73%|███████▎  | 163681/225000 [316:35:07<86:43:52,  5.09s/it]
 73%|███████▎  | 163682/225000 [316:35:15<100:52:41,  5.92s/it]
 73%|███████▎  | 163683/225000 [316:35:25<123:49:14,  7.27s/it]
 73%|███████▎  | 163684/225000 [316:35:29<105:46:52,  6.21s/it]
 73%|███████▎  | 163685/225000 [316:35:33<98:38:33,  5.79s/it] 
 73%|███████▎  | 163686/225000 [316:35:38<94:07:56,  5.53s/it]
 73%|███████▎  | 163687/225000 [316:35:44<92:04:14,  5.41s/it]
 73%|███████▎  | 163688/225000 [316:35:49<91:36:34,  5.38s/it]
 73%|███████▎  | 163689/225000 [316:35:54<90:22:52,  5.31s/it]
 73%|███████▎  | 163690/225000 [316:35:59<87:48:17,  5.16s/it]
 73%|███████▎  | 163691/225000 [316:36:04<86:33:00,  5.08s/it]
 73%|███████▎  | 163692/225000 [316:36:09<86:44:38,  5.09s/it]
 73%|███████▎  | 163693/225000 [316:36:14<88:11:10,  5.18s/it]
 73%|███████▎  | 163694/225000 [316:36:26<121:56:05,  7.16s/it]
 73%|███████▎  | 163695/225000 [316:36:30<108:26:17,  6.37s/it]
 73%|███████▎  | 163696/225000 [316:36:34<93:34:43,  5.50s/it] 
 73%|███████▎  | 163697/225000 [316:36:43<112:31:31,  6.61s/it]
 73%|███████▎  | 163698/225000 [316:36:46<95:05:05,  5.58s/it] 
 73%|███████▎  | 163699/225000 [316:36:50<83:57:11,  4.93s/it]
 73%|███████▎  | 163700/225000 [316:36:58<99:13:57,  5.83s/it]
                                                              
{'loss': '0.7643', 'grad_norm': '1.086', 'learning_rate': '9.398e-06', 'epoch': '0.7276', 'num_input_tokens_seen': 524141944, 'train_runtime': '1.14e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163700/225000 [316:36:58<99:13:57,  5.83s/it]
 73%|███████▎  | 163701/225000 [316:37:03<97:39:16,  5.74s/it]
 73%|███████▎  | 163702/225000 [316:37:09<96:21:58,  5.66s/it]
 73%|███████▎  | 163703/225000 [316:37:13<87:19:48,  5.13s/it]
 73%|███████▎  | 163704/225000 [316:37:22<106:51:42,  6.28s/it]
 73%|███████▎  | 163705/225000 [316:37:26<98:28:29,  5.78s/it] 
 73%|███████▎  | 163706/225000 [316:37:30<90:15:35,  5.30s/it]
 73%|███████▎  | 163707/225000 [316:37:35<87:34:21,  5.14s/it]
 73%|███████▎  | 163708/225000 [316:37:40<85:11:43,  5.00s/it]
 73%|███████▎  | 163709/225000 [316:37:44<83:07:04,  4.88s/it]
 73%|███████▎  | 163710/225000 [316:37:49<81:44:59,  4.80s/it]
 73%|███████▎  | 163711/225000 [316:37:54<81:35:15,  4.79s/it]
 73%|███████▎  | 163712/225000 [316:38:06<118:51:41,  6.98s/it]
 73%|███████▎  | 163713/225000 [316:38:11<109:19:28,  6.42s/it]
 73%|███████▎  | 163714/225000 [316:38:16<101:10:25,  5.94s/it]
 73%|███████▎  | 163715/225000 [316:38:21<97:45:05,  5.74s/it] 
 73%|███████▎  | 163716/225000 [316:38:26<95:54:02,  5.63s/it]
 73%|███████▎  | 163717/225000 [316:38:32<94:12:18,  5.53s/it]
 73%|███████▎  | 163718/225000 [316:38:37<92:21:53,  5.43s/it]
 73%|███████▎  | 163719/225000 [316:38:42<90:34:37,  5.32s/it]
 73%|███████▎  | 163720/225000 [316:38:48<91:32:09,  5.38s/it]
 73%|███████▎  | 163721/225000 [316:38:52<88:37:23,  5.21s/it]
 73%|███████▎  | 163722/225000 [316:38:58<88:37:54,  5.21s/it]
 73%|███████▎  | 163723/225000 [316:39:03<89:05:01,  5.23s/it]
 73%|███████▎  | 163724/225000 [316:39:08<89:28:50,  5.26s/it]
 73%|███████▎  | 163725/225000 [316:39:15<96:37:59,  5.68s/it]
 73%|███████▎  | 163726/225000 [316:39:25<121:43:01,  7.15s/it]
 73%|███████▎  | 163727/225000 [316:39:29<103:52:49,  6.10s/it]
 73%|███████▎  | 163728/225000 [316:39:37<113:57:20,  6.70s/it]
 73%|███████▎  | 163729/225000 [316:39:43<110:42:37,  6.50s/it]
 73%|███████▎  | 163730/225000 [316:39:47<96:03:04,  5.64s/it] 
 73%|███████▎  | 163731/225000 [316:39:55<111:17:34,  6.54s/it]
 73%|███████▎  | 163732/225000 [316:39:59<96:24:18,  5.66s/it] 
 73%|███████▎  | 163733/225000 [316:40:03<86:24:24,  5.08s/it]
 73%|███████▎  | 163734/225000 [316:40:07<80:58:17,  4.76s/it]
 73%|███████▎  | 163735/225000 [316:40:12<85:12:39,  5.01s/it]
 73%|███████▎  | 163736/225000 [316:40:21<104:20:48,  6.13s/it]
 73%|███████▎  | 163737/225000 [316:40:25<94:52:28,  5.58s/it] 
 73%|███████▎  | 163738/225000 [316:40:30<89:06:42,  5.24s/it]
 73%|███████▎  | 163739/225000 [316:40:34<85:50:23,  5.04s/it]
 73%|███████▎  | 163740/225000 [316:40:39<83:39:33,  4.92s/it]
 73%|███████▎  | 163741/225000 [316:40:50<116:51:45,  6.87s/it]
 73%|███████▎  | 163742/225000 [316:40:54<99:39:39,  5.86s/it] 
 73%|███████▎  | 163743/225000 [316:41:00<100:57:58,  5.93s/it]
 73%|███████▎  | 163744/225000 [316:41:06<100:51:03,  5.93s/it]
 73%|███████▎  | 163745/225000 [316:41:12<99:41:26,  5.86s/it] 
 73%|███████▎  | 163746/225000 [316:41:17<97:27:46,  5.73s/it]
 73%|███████▎  | 163747/225000 [316:41:22<95:14:25,  5.60s/it]
 73%|███████▎  | 163748/225000 [316:41:28<93:18:33,  5.48s/it]
 73%|███████▎  | 163749/225000 [316:41:32<89:32:50,  5.26s/it]
 73%|███████▎  | 163750/225000 [316:41:38<92:04:58,  5.41s/it]
                                                              
{'loss': '0.7698', 'grad_norm': '1.189', 'learning_rate': '9.383e-06', 'epoch': '0.7278', 'num_input_tokens_seen': 524273144, 'train_runtime': '1.14e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163750/225000 [316:41:38<92:04:58,  5.41s/it]
 73%|███████▎  | 163751/225000 [316:41:43<90:17:58,  5.31s/it]
 73%|███████▎  | 163752/225000 [316:41:48<87:56:30,  5.17s/it]
 73%|███████▎  | 163753/225000 [316:41:53<88:12:33,  5.18s/it]
 73%|███████▎  | 163754/225000 [316:41:58<86:34:16,  5.09s/it]
 73%|███████▎  | 163755/225000 [316:42:04<89:36:52,  5.27s/it]
 73%|███████▎  | 163756/225000 [316:42:09<90:18:43,  5.31s/it]
 73%|███████▎  | 163757/225000 [316:42:15<90:29:14,  5.32s/it]
 73%|███████▎  | 163758/225000 [316:42:20<90:31:32,  5.32s/it]
 73%|███████▎  | 163759/225000 [316:42:26<92:43:54,  5.45s/it]
 73%|███████▎  | 163760/225000 [316:42:31<92:28:07,  5.44s/it]
 73%|███████▎  | 163761/225000 [316:42:43<126:40:40,  7.45s/it]
 73%|███████▎  | 163762/225000 [316:42:49<120:45:37,  7.10s/it]
 73%|███████▎  | 163763/225000 [316:42:53<102:54:11,  6.05s/it]
 73%|███████▎  | 163764/225000 [316:43:02<116:40:32,  6.86s/it]
 73%|███████▎  | 163765/225000 [316:43:07<106:19:44,  6.25s/it]
 73%|███████▎  | 163766/225000 [316:43:11<98:06:15,  5.77s/it] 
 73%|███████▎  | 163767/225000 [316:43:16<93:14:10,  5.48s/it]
 73%|███████▎  | 163768/225000 [316:43:21<89:27:19,  5.26s/it]
 73%|███████▎  | 163769/225000 [316:43:25<86:08:59,  5.07s/it]
 73%|███████▎  | 163770/225000 [316:43:30<83:11:41,  4.89s/it]
 73%|███████▎  | 163771/225000 [316:43:40<112:00:40,  6.59s/it]
 73%|███████▎  | 163772/225000 [316:43:45<101:16:19,  5.95s/it]
 73%|███████▎  | 163773/225000 [316:43:52<107:27:15,  6.32s/it]
 73%|███████▎  | 163774/225000 [316:43:56<96:04:05,  5.65s/it] 
 73%|███████▎  | 163775/225000 [316:44:00<88:17:43,  5.19s/it]
 73%|███████▎  | 163776/225000 [316:44:09<107:26:11,  6.32s/it]
 73%|███████▎  | 163777/225000 [316:44:12<91:25:56,  5.38s/it] 
 73%|███████▎  | 163778/225000 [316:44:18<93:58:30,  5.53s/it]
 73%|███████▎  | 163779/225000 [316:44:26<102:30:08,  6.03s/it]
 73%|███████▎  | 163780/225000 [316:44:29<91:18:57,  5.37s/it] 
 73%|███████▎  | 163781/225000 [316:44:33<81:25:12,  4.79s/it]
 73%|███████▎  | 163782/225000 [316:44:42<104:11:16,  6.13s/it]
 73%|███████▎  | 163783/225000 [316:44:46<93:48:42,  5.52s/it] 
 73%|███████▎  | 163784/225000 [316:44:53<102:46:23,  6.04s/it]
 73%|███████▎  | 163785/225000 [316:44:57<88:58:52,  5.23s/it] 
 73%|███████▎  | 163786/225000 [316:45:00<79:34:43,  4.68s/it]
 73%|███████▎  | 163787/225000 [316:45:09<100:43:32,  5.92s/it]
 73%|███████▎  | 163788/225000 [316:45:13<91:47:22,  5.40s/it] 
 73%|███████▎  | 163789/225000 [316:45:18<87:05:50,  5.12s/it]
 73%|███████▎  | 163790/225000 [316:45:22<82:49:00,  4.87s/it]
 73%|███████▎  | 163791/225000 [316:45:26<78:42:44,  4.63s/it]
 73%|███████▎  | 163792/225000 [316:45:31<82:47:47,  4.87s/it]
 73%|███████▎  | 163793/225000 [316:45:37<86:05:02,  5.06s/it]
 73%|███████▎  | 163794/225000 [316:45:47<109:58:10,  6.47s/it]
 73%|███████▎  | 163795/225000 [316:45:55<119:02:12,  7.00s/it]
 73%|███████▎  | 163796/225000 [316:45:59<105:26:43,  6.20s/it]
 73%|███████▎  | 163797/225000 [316:46:03<93:46:19,  5.52s/it] 
 73%|███████▎  | 163798/225000 [316:46:08<90:28:27,  5.32s/it]
 73%|███████▎  | 163799/225000 [316:46:12<83:06:33,  4.89s/it]
 73%|███████▎  | 163800/225000 [316:46:22<109:31:15,  6.44s/it]
                                                               
{'loss': '0.7802', 'grad_norm': '1.105', 'learning_rate': '9.369e-06', 'epoch': '0.728', 'num_input_tokens_seen': 524402952, 'train_runtime': '1.14e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163800/225000 [316:46:22<109:31:15,  6.44s/it]
 73%|███████▎  | 163801/225000 [316:46:27<102:52:47,  6.05s/it]
 73%|███████▎  | 163802/225000 [316:46:31<91:31:53,  5.38s/it] 
 73%|███████▎  | 163803/225000 [316:46:38<100:47:19,  5.93s/it]
 73%|███████▎  | 163804/225000 [316:46:46<108:50:41,  6.40s/it]
 73%|███████▎  | 163805/225000 [316:46:49<95:05:37,  5.59s/it] 
 73%|███████▎  | 163806/225000 [316:46:53<84:46:12,  4.99s/it]
 73%|███████▎  | 163807/225000 [316:47:02<104:47:51,  6.17s/it]
 73%|███████▎  | 163808/225000 [316:47:05<90:50:57,  5.34s/it] 
 73%|███████▎  | 163809/225000 [316:47:11<90:50:00,  5.34s/it]
 73%|███████▎  | 163810/225000 [316:47:16<89:34:33,  5.27s/it]
 73%|███████▎  | 163811/225000 [316:47:21<91:23:50,  5.38s/it]
 73%|███████▎  | 163812/225000 [316:47:27<90:23:28,  5.32s/it]
 73%|███████▎  | 163813/225000 [316:47:32<92:47:48,  5.46s/it]
 73%|███████▎  | 163814/225000 [316:47:38<92:55:32,  5.47s/it]
 73%|███████▎  | 163815/225000 [316:47:43<93:55:07,  5.53s/it]
 73%|███████▎  | 163816/225000 [316:47:49<94:07:35,  5.54s/it]
 73%|███████▎  | 163817/225000 [316:47:54<92:06:24,  5.42s/it]
 73%|███████▎  | 163818/225000 [316:48:02<106:47:45,  6.28s/it]
 73%|███████▎  | 163819/225000 [316:48:09<107:58:58,  6.35s/it]
 73%|███████▎  | 163820/225000 [316:48:13<96:16:15,  5.66s/it] 
 73%|███████▎  | 163821/225000 [316:48:17<88:02:35,  5.18s/it]
 73%|███████▎  | 163822/225000 [316:48:21<83:50:46,  4.93s/it]
 73%|███████▎  | 163823/225000 [316:48:34<121:12:41,  7.13s/it]
 73%|███████▎  | 163824/225000 [316:48:38<104:29:29,  6.15s/it]
 73%|███████▎  | 163825/225000 [316:48:41<92:22:41,  5.44s/it] 
 73%|███████▎  | 163826/225000 [316:48:45<83:33:26,  4.92s/it]
 73%|███████▎  | 163827/225000 [316:48:54<103:00:47,  6.06s/it]
 73%|███████▎  | 163828/225000 [316:48:58<91:14:49,  5.37s/it] 
 73%|███████▎  | 163829/225000 [316:49:06<104:36:45,  6.16s/it]
 73%|███████▎  | 163830/225000 [316:49:11<99:36:24,  5.86s/it] 
 73%|███████▎  | 163831/225000 [316:49:19<109:29:11,  6.44s/it]
 73%|███████▎  | 163832/225000 [316:49:22<92:36:29,  5.45s/it] 
 73%|███████▎  | 163833/225000 [316:49:27<92:06:52,  5.42s/it]
 73%|███████▎  | 163834/225000 [316:49:34<97:40:55,  5.75s/it]
 73%|███████▎  | 163835/225000 [316:49:39<95:12:38,  5.60s/it]
 73%|███████▎  | 163836/225000 [316:49:45<96:33:06,  5.68s/it]
 73%|███████▎  | 163837/225000 [316:49:50<95:26:11,  5.62s/it]
 73%|███████▎  | 163838/225000 [316:49:55<92:40:12,  5.45s/it]
 73%|███████▎  | 163839/225000 [316:50:00<89:42:13,  5.28s/it]
 73%|███████▎  | 163840/225000 [316:50:09<106:57:40,  6.30s/it]
 73%|███████▎  | 163841/225000 [316:50:12<92:06:53,  5.42s/it] 
 73%|███████▎  | 163842/225000 [316:50:19<100:37:58,  5.92s/it]
 73%|███████▎  | 163843/225000 [316:50:24<95:22:10,  5.61s/it] 
 73%|███████▎  | 163844/225000 [316:50:31<101:53:12,  6.00s/it]
 73%|███████▎  | 163845/225000 [316:50:35<91:56:19,  5.41s/it] 
 73%|███████▎  | 163846/225000 [316:50:39<85:47:10,  5.05s/it]
 73%|███████▎  | 163847/225000 [316:50:48<103:32:26,  6.10s/it]
 73%|███████▎  | 163848/225000 [316:50:52<93:18:48,  5.49s/it] 
 73%|███████▎  | 163849/225000 [316:50:55<82:09:31,  4.84s/it]
 73%|███████▎  | 163850/225000 [316:51:04<103:26:42,  6.09s/it]
                                                               
{'loss': '0.7588', 'grad_norm': '1.061', 'learning_rate': '9.355e-06', 'epoch': '0.7282', 'num_input_tokens_seen': 524533008, 'train_runtime': '1.141e+06', 'train_tokens_per_second': '459.8'}

 73%|███████▎  | 163850/225000 [316:51:04<103:26:42,  6.09s/it]
 73%|███████▎  | 163851/225000 [316:51:08<92:37:22,  5.45s/it] 
 73%|███████▎  | 163852/225000 [316:51:16<107:05:28,  6.30s/it]
 73%|███████▎  | 163853/225000 [316:51:20<91:43:59,  5.40s/it] 
 73%|███████▎  | 163854/225000 [316:51:27<99:14:58,  5.84s/it]
 73%|███████▎  | 163855/225000 [316:51:30<87:54:46,  5.18s/it]
 73%|███████▎  | 163856/225000 [316:51:40<112:10:07,  6.60s/it]
 73%|███████▎  | 163857/225000 [316:51:44<96:04:16,  5.66s/it] 
 73%|███████▎  | 163858/225000 [316:51:49<95:21:01,  5.61s/it]
 73%|███████▎  | 163859/225000 [316:51:55<96:11:29,  5.66s/it]
 73%|███████▎  | 163860/225000 [316:51:58<85:17:07,  5.02s/it]
 73%|███████▎  | 163861/225000 [316:52:02<78:41:35,  4.63s/it]
 73%|███████▎  | 163862/225000 [316:52:12<103:34:21,  6.10s/it]
 73%|███████▎  | 163863/225000 [316:52:17<100:16:23,  5.90s/it]
 73%|███████▎  | 163864/225000 [316:52:21<90:06:22,  5.31s/it] 
 73%|███████▎  | 163865/225000 [316:52:25<84:30:35,  4.98s/it]
 73%|███████▎  | 163866/225000 [316:52:30<84:47:21,  4.99s/it]
 73%|███████▎  | 163867/225000 [316:52:35<84:42:03,  4.99s/it]
 73%|███████▎  | 163868/225000 [316:52:40<84:18:10,  4.96s/it]
 73%|███████▎  | 163869/225000 [316:52:45<85:07:18,  5.01s/it]
 73%|███████▎  | 163870/225000 [316:52:50<85:05:58,  5.01s/it]
 73%|███████▎  | 163871/225000 [316:52:57<93:06:46,  5.48s/it]
 73%|███████▎  | 163872/225000 [316:53:02<92:23:23,  5.44s/it]
 73%|███████▎  | 163873/225000 [316:53:07<88:07:16,  5.19s/it]
 73%|███████▎  | 163874/225000 [316:53:11<83:39:05,  4.93s/it]
 73%|███████▎  | 163875/225000 [316:53:24<121:49:40,  7.18s/it]
 73%|███████▎  | 163876/225000 [316:53:28<108:47:36,  6.41s/it]
 73%|███████▎  | 163877/225000 [316:53:33<99:47:19,  5.88s/it] 
 73%|███████▎  | 163878/225000 [316:53:37<91:14:46,  5.37s/it]
 73%|███████▎  | 163879/225000 [316:53:42<89:01:14,  5.24s/it]
 73%|███████▎  | 163880/225000 [316:53:46<84:46:08,  4.99s/it]
 73%|███████▎  | 163881/225000 [316:53:51<84:07:55,  4.96s/it]
 73%|███████▎  | 163882/225000 [316:53:56<81:17:45,  4.79s/it]
 73%|███████▎  | 163883/225000 [316:54:00<79:11:38,  4.66s/it]
 73%|███████▎  | 163884/225000 [316:54:12<115:51:48,  6.82s/it]
 73%|███████▎  | 163885/225000 [316:54:18<111:46:07,  6.58s/it]
 73%|███████▎  | 163886/225000 [316:54:22<99:15:03,  5.85s/it] 
 73%|███████▎  | 163887/225000 [316:54:26<88:19:09,  5.20s/it]
 73%|███████▎  | 163888/225000 [316:54:35<110:13:08,  6.49s/it]
 73%|███████▎  | 163889/225000 [316:54:39<98:22:08,  5.79s/it] 
 73%|███████▎  | 163890/225000 [316:54:43<86:00:49,  5.07s/it]
 73%|███████▎  | 163891/225000 [316:54:51<104:35:42,  6.16s/it]
 73%|███████▎  | 163892/225000 [316:54:55<90:00:04,  5.30s/it] 
 73%|███████▎  | 163893/225000 [316:55:03<104:16:14,  6.14s/it]
 73%|███████▎  | 163894/225000 [316:55:07<94:56:41,  5.59s/it] 
 73%|███████▎  | 163895/225000 [316:55:12<89:23:46,  5.27s/it]
 73%|███████▎  | 163896/225000 [316:55:16<84:27:22,  4.98s/it]
 73%|███████▎  | 163897/225000 [316:55:20<81:16:21,  4.79s/it]
 73%|███████▎  | 163898/225000 [316:55:30<103:43:48,  6.11s/it]
 73%|███████▎  | 163899/225000 [316:55:36<104:22:50,  6.15s/it]
 73%|███████▎  | 163900/225000 [316:55:39<91:54:27,  5.42s/it] 
                                                              
{'loss': '0.7694', 'grad_norm': '1.202', 'learning_rate': '9.341e-06', 'epoch': '0.7284', 'num_input_tokens_seen': 524662280, 'train_runtime': '1.141e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 163900/225000 [316:55:39<91:54:27,  5.42s/it]
 73%|███████▎  | 163901/225000 [316:55:46<97:15:35,  5.73s/it]
 73%|███████▎  | 163902/225000 [316:55:53<104:21:01,  6.15s/it]
 73%|███████▎  | 163903/225000 [316:55:57<91:20:07,  5.38s/it] 
 73%|███████▎  | 163904/225000 [316:56:04<98:47:01,  5.82s/it]
 73%|███████▎  | 163905/225000 [316:56:09<98:00:43,  5.78s/it]
 73%|███████▎  | 163906/225000 [316:56:15<97:44:27,  5.76s/it]
 73%|███████▎  | 163907/225000 [316:56:19<90:18:23,  5.32s/it]
 73%|███████▎  | 163908/225000 [316:56:24<85:24:40,  5.03s/it]
 73%|███████▎  | 163909/225000 [316:56:29<87:13:19,  5.14s/it]
 73%|███████▎  | 163910/225000 [316:56:40<116:38:34,  6.87s/it]
 73%|███████▎  | 163911/225000 [316:56:43<99:03:19,  5.84s/it] 
 73%|███████▎  | 163912/225000 [316:56:47<86:31:36,  5.10s/it]
 73%|███████▎  | 163913/225000 [316:56:56<106:22:47,  6.27s/it]
 73%|███████▎  | 163914/225000 [316:57:00<94:13:41,  5.55s/it] 
 73%|███████▎  | 163915/225000 [316:57:08<110:01:41,  6.48s/it]
 73%|███████▎  | 163916/225000 [316:57:12<95:01:01,  5.60s/it] 
 73%|███████▎  | 163917/225000 [316:57:16<86:41:02,  5.11s/it]
 73%|███████▎  | 163918/225000 [316:57:20<81:38:35,  4.81s/it]
 73%|███████▎  | 163919/225000 [316:57:29<105:25:32,  6.21s/it]
 73%|███████▎  | 163920/225000 [316:57:35<101:41:56,  5.99s/it]
 73%|███████▎  | 163921/225000 [316:57:43<112:18:55,  6.62s/it]
 73%|███████▎  | 163922/225000 [316:57:47<98:50:55,  5.83s/it] 
 73%|███████▎  | 163923/225000 [316:57:51<90:30:51,  5.34s/it]
 73%|███████▎  | 163924/225000 [316:57:56<88:54:15,  5.24s/it]
 73%|███████▎  | 163925/225000 [316:58:03<99:53:14,  5.89s/it]
 73%|███████▎  | 163926/225000 [316:58:10<104:34:10,  6.16s/it]
 73%|███████▎  | 163927/225000 [316:58:14<91:17:13,  5.38s/it] 
 73%|███████▎  | 163928/225000 [316:58:17<81:43:50,  4.82s/it]
 73%|███████▎  | 163929/225000 [316:58:22<83:02:57,  4.90s/it]
 73%|███████▎  | 163930/225000 [316:58:31<101:24:49,  5.98s/it]
 73%|███████▎  | 163931/225000 [316:58:36<96:55:26,  5.71s/it] 
 73%|███████▎  | 163932/225000 [316:58:42<96:03:47,  5.66s/it]
 73%|███████▎  | 163933/225000 [316:58:47<92:39:02,  5.46s/it]
 73%|███████▎  | 163934/225000 [316:58:52<91:34:00,  5.40s/it]
 73%|███████▎  | 163935/225000 [316:58:57<91:19:45,  5.38s/it]
 73%|███████▎  | 163936/225000 [316:59:02<90:40:05,  5.35s/it]
 73%|███████▎  | 163937/225000 [316:59:08<90:21:22,  5.33s/it]
 73%|███████▎  | 163938/225000 [316:59:13<92:12:52,  5.44s/it]
 73%|███████▎  | 163939/225000 [316:59:19<92:49:16,  5.47s/it]
 73%|███████▎  | 163940/225000 [316:59:24<92:27:25,  5.45s/it]
 73%|███████▎  | 163941/225000 [316:59:30<92:24:26,  5.45s/it]
 73%|███████▎  | 163942/225000 [316:59:35<92:51:14,  5.47s/it]
 73%|███████▎  | 163943/225000 [316:59:41<93:06:40,  5.49s/it]
 73%|███████▎  | 163944/225000 [316:59:46<92:53:49,  5.48s/it]
 73%|███████▎  | 163945/225000 [316:59:51<91:23:32,  5.39s/it]
 73%|███████▎  | 163946/225000 [316:59:57<90:46:38,  5.35s/it]
 73%|███████▎  | 163947/225000 [317:00:02<90:08:53,  5.32s/it]
 73%|███████▎  | 163948/225000 [317:00:07<90:45:41,  5.35s/it]
 73%|███████▎  | 163949/225000 [317:00:13<90:13:38,  5.32s/it]
 73%|███████▎  | 163950/225000 [317:00:21<106:29:10,  6.28s/it]
                                                               
{'loss': '0.7759', 'grad_norm': '1.299', 'learning_rate': '9.326e-06', 'epoch': '0.7287', 'num_input_tokens_seen': 524793848, 'train_runtime': '1.141e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 163950/225000 [317:00:21<106:29:10,  6.28s/it]
 73%|███████▎  | 163951/225000 [317:00:27<102:00:10,  6.02s/it]
 73%|███████▎  | 163952/225000 [317:00:32<97:50:36,  5.77s/it] 
 73%|███████▎  | 163953/225000 [317:00:37<92:54:25,  5.48s/it]
 73%|███████▎  | 163954/225000 [317:00:42<93:08:58,  5.49s/it]
 73%|███████▎  | 163955/225000 [317:00:47<92:21:38,  5.45s/it]
 73%|███████▎  | 163956/225000 [317:00:53<93:07:13,  5.49s/it]
 73%|███████▎  | 163957/225000 [317:00:59<93:26:26,  5.51s/it]
 73%|███████▎  | 163958/225000 [317:01:03<89:02:45,  5.25s/it]
 73%|███████▎  | 163959/225000 [317:01:08<88:51:00,  5.24s/it]
 73%|███████▎  | 163960/225000 [317:01:13<87:41:02,  5.17s/it]
 73%|███████▎  | 163961/225000 [317:01:19<88:58:49,  5.25s/it]
 73%|███████▎  | 163962/225000 [317:01:24<90:45:30,  5.35s/it]
 73%|███████▎  | 163963/225000 [317:01:30<91:38:31,  5.41s/it]
 73%|███████▎  | 163964/225000 [317:01:36<96:11:18,  5.67s/it]
 73%|███████▎  | 163965/225000 [317:01:48<124:36:01,  7.35s/it]
 73%|███████▎  | 163966/225000 [317:01:55<124:17:00,  7.33s/it]
 73%|███████▎  | 163967/225000 [317:01:59<107:50:08,  6.36s/it]
 73%|███████▎  | 163968/225000 [317:02:03<94:58:59,  5.60s/it] 
 73%|███████▎  | 163969/225000 [317:02:07<86:18:29,  5.09s/it]
 73%|███████▎  | 163970/225000 [317:02:14<96:55:56,  5.72s/it]
 73%|███████▎  | 163971/225000 [317:02:21<106:08:18,  6.26s/it]
 73%|███████▎  | 163972/225000 [317:02:25<91:55:35,  5.42s/it] 
 73%|███████▎  | 163973/225000 [317:02:29<83:27:30,  4.92s/it]
 73%|███████▎  | 163974/225000 [317:02:39<113:05:01,  6.67s/it]
 73%|███████▎  | 163975/225000 [317:02:45<108:45:58,  6.42s/it]
 73%|███████▎  | 163976/225000 [317:02:49<97:12:12,  5.73s/it] 
 73%|███████▎  | 163977/225000 [317:02:54<90:41:23,  5.35s/it]
 73%|███████▎  | 163978/225000 [317:02:58<86:06:17,  5.08s/it]
 73%|███████▎  | 163979/225000 [317:03:02<81:16:10,  4.79s/it]
 73%|███████▎  | 163980/225000 [317:03:13<108:37:40,  6.41s/it]
 73%|███████▎  | 163981/225000 [317:03:17<97:43:29,  5.77s/it] 
 73%|███████▎  | 163982/225000 [317:03:21<89:38:25,  5.29s/it]
 73%|███████▎  | 163983/225000 [317:03:25<83:43:27,  4.94s/it]
 73%|███████▎  | 163984/225000 [317:03:35<107:18:17,  6.33s/it]
 73%|███████▎  | 163985/225000 [317:03:39<98:53:28,  5.83s/it] 
 73%|███████▎  | 163986/225000 [317:03:44<93:33:19,  5.52s/it]
 73%|███████▎  | 163987/225000 [317:03:49<89:47:05,  5.30s/it]
 73%|███████▎  | 163988/225000 [317:03:54<87:06:14,  5.14s/it]
 73%|███████▎  | 163989/225000 [317:03:58<85:00:47,  5.02s/it]
 73%|███████▎  | 163990/225000 [317:04:03<83:02:46,  4.90s/it]
 73%|███████▎  | 163991/225000 [317:04:08<82:42:56,  4.88s/it]
 73%|███████▎  | 163992/225000 [317:04:13<84:46:13,  5.00s/it]
 73%|███████▎  | 163993/225000 [317:04:18<85:46:41,  5.06s/it]
 73%|███████▎  | 163994/225000 [317:04:24<87:13:36,  5.15s/it]
 73%|███████▎  | 163995/225000 [317:04:29<88:38:46,  5.23s/it]
 73%|███████▎  | 163996/225000 [317:04:41<121:05:59,  7.15s/it]
 73%|███████▎  | 163997/225000 [317:04:48<121:27:04,  7.17s/it]
 73%|███████▎  | 163998/225000 [317:04:51<102:54:25,  6.07s/it]
 73%|███████▎  | 163999/225000 [317:04:57<100:48:16,  5.95s/it]
 73%|███████▎  | 164000/225000 [317:05:05<112:31:59,  6.64s/it]
                                                               
{'loss': '0.7888', 'grad_norm': '1.112', 'learning_rate': '9.312e-06', 'epoch': '0.7289', 'num_input_tokens_seen': 524923472, 'train_runtime': '1.142e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164000/225000 [317:05:05<112:31:59,  6.64s/it]
 73%|███████▎  | 164001/225000 [317:05:09<95:57:06,  5.66s/it] 
 73%|███████▎  | 164002/225000 [317:05:12<85:38:49,  5.05s/it]
 73%|███████▎  | 164003/225000 [317:05:22<107:58:11,  6.37s/it]
 73%|███████▎  | 164004/225000 [317:05:28<106:37:00,  6.29s/it]
 73%|███████▎  | 164005/225000 [317:05:33<98:55:25,  5.84s/it] 
 73%|███████▎  | 164006/225000 [317:05:36<86:59:13,  5.13s/it]
 73%|███████▎  | 164007/225000 [317:05:45<105:40:58,  6.24s/it]
 73%|███████▎  | 164008/225000 [317:05:49<92:08:02,  5.44s/it] 
 73%|███████▎  | 164009/225000 [317:05:53<88:29:13,  5.22s/it]
 73%|███████▎  | 164010/225000 [317:05:58<86:07:40,  5.08s/it]
 73%|███████▎  | 164011/225000 [317:06:02<82:27:01,  4.87s/it]
 73%|███████▎  | 164012/225000 [317:06:07<81:21:24,  4.80s/it]
 73%|███████▎  | 164013/225000 [317:06:12<79:32:16,  4.70s/it]
 73%|███████▎  | 164014/225000 [317:06:24<116:42:47,  6.89s/it]
 73%|███████▎  | 164015/225000 [317:06:28<103:49:56,  6.13s/it]
 73%|███████▎  | 164016/225000 [317:06:33<100:08:58,  5.91s/it]
 73%|███████▎  | 164017/225000 [317:06:39<97:04:49,  5.73s/it] 
 73%|███████▎  | 164018/225000 [317:06:44<95:51:59,  5.66s/it]
 73%|███████▎  | 164019/225000 [317:06:49<91:20:26,  5.39s/it]
 73%|███████▎  | 164020/225000 [317:06:54<92:01:32,  5.43s/it]
 73%|███████▎  | 164021/225000 [317:07:00<91:00:21,  5.37s/it]
 73%|███████▎  | 164022/225000 [317:07:05<91:38:45,  5.41s/it]
 73%|███████▎  | 164023/225000 [317:07:10<86:51:06,  5.13s/it]
 73%|███████▎  | 164024/225000 [317:07:15<87:35:27,  5.17s/it]
 73%|███████▎  | 164025/225000 [317:07:20<89:20:54,  5.28s/it]
 73%|███████▎  | 164026/225000 [317:07:25<87:24:40,  5.16s/it]
 73%|███████▎  | 164027/225000 [317:07:31<88:03:20,  5.20s/it]
 73%|███████▎  | 164028/225000 [317:07:36<88:27:07,  5.22s/it]
 73%|███████▎  | 164029/225000 [317:07:41<87:39:33,  5.18s/it]
 73%|███████▎  | 164030/225000 [317:07:46<88:43:18,  5.24s/it]
 73%|███████▎  | 164031/225000 [317:07:51<86:43:11,  5.12s/it]
 73%|███████▎  | 164032/225000 [317:07:56<86:44:54,  5.12s/it]
 73%|███████▎  | 164033/225000 [317:08:02<87:57:39,  5.19s/it]
 73%|███████▎  | 164034/225000 [317:08:07<89:22:04,  5.28s/it]
 73%|███████▎  | 164035/225000 [317:08:12<89:38:03,  5.29s/it]
 73%|███████▎  | 164036/225000 [317:08:18<88:29:24,  5.23s/it]
 73%|███████▎  | 164037/225000 [317:08:23<89:05:06,  5.26s/it]
 73%|███████▎  | 164038/225000 [317:08:28<86:47:49,  5.13s/it]
 73%|███████▎  | 164039/225000 [317:08:33<85:15:22,  5.03s/it]
 73%|███████▎  | 164040/225000 [317:08:42<108:30:54,  6.41s/it]
 73%|███████▎  | 164041/225000 [317:08:50<114:48:38,  6.78s/it]
 73%|███████▎  | 164042/225000 [317:08:55<107:41:04,  6.36s/it]
 73%|███████▎  | 164043/225000 [317:09:00<101:25:48,  5.99s/it]
 73%|███████▎  | 164044/225000 [317:09:05<96:21:36,  5.69s/it] 
 73%|███████▎  | 164045/225000 [317:09:10<93:27:26,  5.52s/it]
 73%|███████▎  | 164046/225000 [317:09:16<92:49:03,  5.48s/it]
 73%|███████▎  | 164047/225000 [317:09:21<92:57:46,  5.49s/it]
 73%|███████▎  | 164048/225000 [317:09:27<92:41:41,  5.47s/it]
 73%|███████▎  | 164049/225000 [317:09:36<110:50:56,  6.55s/it]
 73%|███████▎  | 164050/225000 [317:09:39<95:48:48,  5.66s/it] 
                                                              
{'loss': '0.7707', 'grad_norm': '1.1', 'learning_rate': '9.298e-06', 'epoch': '0.7291', 'num_input_tokens_seen': 525053184, 'train_runtime': '1.142e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164050/225000 [317:09:39<95:48:48,  5.66s/it]
 73%|███████▎  | 164051/225000 [317:09:48<112:51:23,  6.67s/it]
 73%|███████▎  | 164052/225000 [317:09:51<94:42:01,  5.59s/it] 
 73%|███████▎  | 164053/225000 [317:09:59<106:19:21,  6.28s/it]
 73%|███████▎  | 164054/225000 [317:10:03<93:23:24,  5.52s/it] 
 73%|███████▎  | 164055/225000 [317:10:07<86:30:28,  5.11s/it]
 73%|███████▎  | 164056/225000 [317:10:15<99:58:36,  5.91s/it]
 73%|███████▎  | 164057/225000 [317:10:21<100:52:07,  5.96s/it]
 73%|███████▎  | 164058/225000 [317:10:25<89:15:45,  5.27s/it] 
 73%|███████▎  | 164059/225000 [317:10:28<80:25:19,  4.75s/it]
 73%|███████▎  | 164060/225000 [317:10:37<99:14:06,  5.86s/it]
 73%|███████▎  | 164061/225000 [317:10:40<86:41:30,  5.12s/it]
 73%|███████▎  | 164062/225000 [317:10:44<78:44:59,  4.65s/it]
 73%|███████▎  | 164063/225000 [317:10:54<106:32:37,  6.29s/it]
 73%|███████▎  | 164064/225000 [317:10:58<93:24:31,  5.52s/it] 
 73%|███████▎  | 164065/225000 [317:11:01<81:59:12,  4.84s/it]
 73%|███████▎  | 164066/225000 [317:11:12<112:28:09,  6.64s/it]
 73%|███████▎  | 164067/225000 [317:11:15<97:23:56,  5.75s/it] 
 73%|███████▎  | 164068/225000 [317:11:19<87:11:12,  5.15s/it]
 73%|███████▎  | 164069/225000 [317:11:23<81:11:03,  4.80s/it]
 73%|███████▎  | 164070/225000 [317:11:34<114:20:41,  6.76s/it]
 73%|███████▎  | 164071/225000 [317:11:38<98:30:23,  5.82s/it] 
 73%|███████▎  | 164072/225000 [317:11:42<89:49:34,  5.31s/it]
 73%|███████▎  | 164073/225000 [317:11:50<103:39:33,  6.12s/it]
 73%|███████▎  | 164074/225000 [317:11:53<88:55:58,  5.25s/it] 
 73%|███████▎  | 164075/225000 [317:11:59<91:58:16,  5.43s/it]
 73%|███████▎  | 164076/225000 [317:12:06<98:02:48,  5.79s/it]
 73%|███████▎  | 164077/225000 [317:12:10<90:42:35,  5.36s/it]
 73%|███████▎  | 164078/225000 [317:12:15<86:20:48,  5.10s/it]
 73%|███████▎  | 164079/225000 [317:12:19<80:36:07,  4.76s/it]
 73%|███████▎  | 164080/225000 [317:12:24<80:52:52,  4.78s/it]
 73%|███████▎  | 164081/225000 [317:12:27<76:27:53,  4.52s/it]
 73%|███████▎  | 164082/225000 [317:12:37<101:32:45,  6.00s/it]
 73%|███████▎  | 164083/225000 [317:12:46<116:02:45,  6.86s/it]
 73%|███████▎  | 164084/225000 [317:12:50<102:04:28,  6.03s/it]
 73%|███████▎  | 164085/225000 [317:12:54<94:32:21,  5.59s/it] 
 73%|███████▎  | 164086/225000 [317:12:59<88:29:17,  5.23s/it]
 73%|███████▎  | 164087/225000 [317:13:03<82:58:04,  4.90s/it]
 73%|███████▎  | 164088/225000 [317:13:07<80:24:45,  4.75s/it]
 73%|███████▎  | 164089/225000 [317:13:15<96:25:46,  5.70s/it]
 73%|███████▎  | 164090/225000 [317:13:23<108:21:49,  6.40s/it]
 73%|███████▎  | 164091/225000 [317:13:27<94:27:02,  5.58s/it] 
 73%|███████▎  | 164092/225000 [317:13:35<105:48:08,  6.25s/it]
 73%|███████▎  | 164093/225000 [317:13:40<98:08:25,  5.80s/it] 
 73%|███████▎  | 164094/225000 [317:13:44<92:55:07,  5.49s/it]
 73%|███████▎  | 164095/225000 [317:13:49<88:47:37,  5.25s/it]
 73%|███████▎  | 164096/225000 [317:13:54<85:18:00,  5.04s/it]
 73%|███████▎  | 164097/225000 [317:13:58<81:49:11,  4.84s/it]
 73%|███████▎  | 164098/225000 [317:14:02<79:54:26,  4.72s/it]
 73%|███████▎  | 164099/225000 [317:14:09<90:41:19,  5.36s/it]
 73%|███████▎  | 164100/225000 [317:14:21<123:52:15,  7.32s/it]
                                                               
{'loss': '0.7629', 'grad_norm': '1.153', 'learning_rate': '9.284e-06', 'epoch': '0.7293', 'num_input_tokens_seen': 525183144, 'train_runtime': '1.142e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164100/225000 [317:14:21<123:52:15,  7.32s/it]
 73%|███████▎  | 164101/225000 [317:14:25<108:31:52,  6.42s/it]
 73%|███████▎  | 164102/225000 [317:14:30<101:03:54,  5.97s/it]
 73%|███████▎  | 164103/225000 [317:14:35<96:27:14,  5.70s/it] 
 73%|███████▎  | 164104/225000 [317:14:40<91:03:28,  5.38s/it]
 73%|███████▎  | 164105/225000 [317:14:45<88:44:34,  5.25s/it]
 73%|███████▎  | 164106/225000 [317:14:50<85:32:28,  5.06s/it]
 73%|███████▎  | 164107/225000 [317:14:55<84:56:06,  5.02s/it]
 73%|███████▎  | 164108/225000 [317:15:00<84:54:53,  5.02s/it]
 73%|███████▎  | 164109/225000 [317:15:04<83:16:42,  4.92s/it]
 73%|███████▎  | 164110/225000 [317:15:10<86:17:55,  5.10s/it]
 73%|███████▎  | 164111/225000 [317:15:15<86:54:55,  5.14s/it]
 73%|███████▎  | 164112/225000 [317:15:20<86:30:01,  5.11s/it]
 73%|███████▎  | 164113/225000 [317:15:25<85:26:55,  5.05s/it]
 73%|███████▎  | 164114/225000 [317:15:30<86:18:21,  5.10s/it]
 73%|███████▎  | 164115/225000 [317:15:38<97:43:57,  5.78s/it]
 73%|███████▎  | 164116/225000 [317:15:42<93:35:54,  5.53s/it]
 73%|███████▎  | 164117/225000 [317:15:47<90:30:35,  5.35s/it]
 73%|███████▎  | 164118/225000 [317:15:53<89:38:18,  5.30s/it]
 73%|███████▎  | 164119/225000 [317:15:58<88:02:21,  5.21s/it]
 73%|███████▎  | 164120/225000 [317:16:03<87:34:36,  5.18s/it]
 73%|███████▎  | 164121/225000 [317:16:08<87:39:02,  5.18s/it]
 73%|███████▎  | 164122/225000 [317:16:16<102:52:02,  6.08s/it]
 73%|███████▎  | 164123/225000 [317:16:30<140:12:15,  8.29s/it]
 73%|███████▎  | 164124/225000 [317:16:33<115:43:21,  6.84s/it]
 73%|███████▎  | 164125/225000 [317:16:37<100:31:06,  5.94s/it]
 73%|███████▎  | 164126/225000 [317:16:45<111:52:40,  6.62s/it]
 73%|███████▎  | 164127/225000 [317:16:50<101:40:07,  6.01s/it]
 73%|███████▎  | 164128/225000 [317:16:57<106:57:21,  6.33s/it]
 73%|███████▎  | 164129/225000 [317:17:00<91:22:56,  5.40s/it] 
 73%|███████▎  | 164130/225000 [317:17:06<94:30:26,  5.59s/it]
 73%|███████▎  | 164131/225000 [317:17:12<96:14:58,  5.69s/it]
 73%|███████▎  | 164132/225000 [317:17:19<102:14:14,  6.05s/it]
 73%|███████▎  | 164133/225000 [317:17:22<87:46:08,  5.19s/it] 
 73%|███████▎  | 164134/225000 [317:17:31<105:35:22,  6.25s/it]
 73%|███████▎  | 164135/225000 [317:17:34<91:39:50,  5.42s/it] 
 73%|███████▎  | 164136/225000 [317:17:38<84:03:28,  4.97s/it]
 73%|███████▎  | 164137/225000 [317:17:43<84:16:04,  4.98s/it]
 73%|███████▎  | 164138/225000 [317:17:48<85:25:10,  5.05s/it]
 73%|███████▎  | 164139/225000 [317:17:53<85:35:48,  5.06s/it]
 73%|███████▎  | 164140/225000 [317:17:58<84:00:56,  4.97s/it]
 73%|███████▎  | 164141/225000 [317:18:03<85:39:10,  5.07s/it]
 73%|███████▎  | 164142/225000 [317:18:08<84:31:26,  5.00s/it]
 73%|███████▎  | 164143/225000 [317:18:13<83:54:52,  4.96s/it]
 73%|███████▎  | 164144/225000 [317:18:18<83:20:29,  4.93s/it]
 73%|███████▎  | 164145/225000 [317:18:23<83:58:00,  4.97s/it]
 73%|███████▎  | 164146/225000 [317:18:29<90:47:21,  5.37s/it]
 73%|███████▎  | 164147/225000 [317:18:40<116:30:22,  6.89s/it]
 73%|███████▎  | 164148/225000 [317:18:43<99:57:16,  5.91s/it] 
 73%|███████▎  | 164149/225000 [317:18:53<117:25:56,  6.95s/it]
 73%|███████▎  | 164150/225000 [317:18:56<100:33:57,  5.95s/it]
                                                               
{'loss': '0.7703', 'grad_norm': '1.115', 'learning_rate': '9.27e-06', 'epoch': '0.7296', 'num_input_tokens_seen': 525312608, 'train_runtime': '1.142e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164150/225000 [317:18:56<100:33:57,  5.95s/it]
 73%|███████▎  | 164151/225000 [317:19:03<104:41:41,  6.19s/it]
 73%|███████▎  | 164152/225000 [317:19:08<97:23:25,  5.76s/it] 
 73%|███████▎  | 164153/225000 [317:19:13<92:40:16,  5.48s/it]
 73%|███████▎  | 164154/225000 [317:19:17<87:42:17,  5.19s/it]
 73%|███████▎  | 164155/225000 [317:19:22<85:44:31,  5.07s/it]
 73%|███████▎  | 164156/225000 [317:19:27<83:57:02,  4.97s/it]
 73%|███████▎  | 164157/225000 [317:19:32<83:55:43,  4.97s/it]
 73%|███████▎  | 164158/225000 [317:19:36<82:43:43,  4.90s/it]
 73%|███████▎  | 164159/225000 [317:19:41<82:27:33,  4.88s/it]
 73%|███████▎  | 164160/225000 [317:19:46<82:38:39,  4.89s/it]
 73%|███████▎  | 164161/225000 [317:19:57<113:45:00,  6.73s/it]
 73%|███████▎  | 164162/225000 [317:20:04<113:02:26,  6.69s/it]
 73%|███████▎  | 164163/225000 [317:20:07<97:05:32,  5.75s/it] 
 73%|███████▎  | 164164/225000 [317:20:17<115:56:49,  6.86s/it]
 73%|███████▎  | 164165/225000 [317:20:21<100:10:26,  5.93s/it]
 73%|███████▎  | 164166/225000 [317:20:25<90:29:56,  5.36s/it] 
 73%|███████▎  | 164167/225000 [317:20:29<86:03:27,  5.09s/it]
 73%|███████▎  | 164168/225000 [317:20:39<111:23:43,  6.59s/it]
 73%|███████▎  | 164169/225000 [317:20:43<95:52:23,  5.67s/it] 
 73%|███████▎  | 164170/225000 [317:20:47<86:16:08,  5.11s/it]
 73%|███████▎  | 164171/225000 [317:20:57<113:55:22,  6.74s/it]
 73%|███████▎  | 164172/225000 [317:21:01<97:09:34,  5.75s/it] 
 73%|███████▎  | 164173/225000 [317:21:04<86:46:12,  5.14s/it]
 73%|███████▎  | 164174/225000 [317:21:12<97:43:21,  5.78s/it]
 73%|███████▎  | 164175/225000 [317:21:15<85:36:59,  5.07s/it]
 73%|███████▎  | 164176/225000 [317:21:20<85:33:46,  5.06s/it]
 73%|███████▎  | 164177/225000 [317:21:25<84:24:52,  5.00s/it]
 73%|███████▎  | 164178/225000 [317:21:30<86:41:54,  5.13s/it]
 73%|███████▎  | 164179/225000 [317:21:35<85:05:48,  5.04s/it]
 73%|███████▎  | 164180/225000 [317:21:40<84:02:58,  4.97s/it]
 73%|███████▎  | 164181/225000 [317:21:45<84:47:41,  5.02s/it]
 73%|███████▎  | 164182/225000 [317:21:50<83:41:40,  4.95s/it]
 73%|███████▎  | 164183/225000 [317:21:55<84:32:53,  5.00s/it]
 73%|███████▎  | 164184/225000 [317:22:00<85:16:25,  5.05s/it]
 73%|███████▎  | 164185/225000 [317:22:05<84:20:02,  4.99s/it]
 73%|███████▎  | 164186/225000 [317:22:17<121:56:52,  7.22s/it]
 73%|███████▎  | 164187/225000 [317:22:23<116:09:45,  6.88s/it]
 73%|███████▎  | 164188/225000 [317:22:28<104:04:44,  6.16s/it]
 73%|███████▎  | 164189/225000 [317:22:32<93:37:55,  5.54s/it] 
 73%|███████▎  | 164190/225000 [317:22:43<118:49:30,  7.03s/it]
 73%|███████▎  | 164191/225000 [317:22:46<99:55:47,  5.92s/it] 
 73%|███████▎  | 164192/225000 [317:22:52<102:06:07,  6.04s/it]
 73%|███████▎  | 164193/225000 [317:22:58<100:55:02,  5.97s/it]
 73%|███████▎  | 164194/225000 [317:23:02<90:03:53,  5.33s/it] 
 73%|███████▎  | 164195/225000 [317:23:06<84:12:38,  4.99s/it]
 73%|███████▎  | 164196/225000 [317:23:17<112:26:27,  6.66s/it]
 73%|███████▎  | 164197/225000 [317:23:20<97:06:29,  5.75s/it] 
 73%|███████▎  | 164198/225000 [317:23:24<87:00:16,  5.15s/it]
 73%|███████▎  | 164199/225000 [317:23:33<106:09:05,  6.29s/it]
 73%|███████▎  | 164200/225000 [317:23:37<93:55:09,  5.56s/it] 
                                                              
{'loss': '0.7722', 'grad_norm': '1.131', 'learning_rate': '9.255e-06', 'epoch': '0.7298', 'num_input_tokens_seen': 525444104, 'train_runtime': '1.143e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164200/225000 [317:23:37<93:55:09,  5.56s/it]
 73%|███████▎  | 164201/225000 [317:23:41<86:17:01,  5.11s/it]
 73%|███████▎  | 164202/225000 [317:23:45<82:22:51,  4.88s/it]
 73%|███████▎  | 164203/225000 [317:23:56<112:09:51,  6.64s/it]
 73%|███████▎  | 164204/225000 [317:23:59<96:02:24,  5.69s/it] 
 73%|███████▎  | 164205/225000 [317:24:07<104:19:55,  6.18s/it]
 73%|███████▎  | 164206/225000 [317:24:10<92:01:23,  5.45s/it] 
 73%|███████▎  | 164207/225000 [317:24:15<88:51:49,  5.26s/it]
 73%|███████▎  | 164208/225000 [317:24:24<106:42:52,  6.32s/it]
 73%|███████▎  | 164209/225000 [317:24:28<92:10:16,  5.46s/it] 
 73%|███████▎  | 164210/225000 [317:24:35<104:01:16,  6.16s/it]
 73%|███████▎  | 164211/225000 [317:24:39<89:53:03,  5.32s/it] 
 73%|███████▎  | 164212/225000 [317:24:43<86:27:24,  5.12s/it]
 73%|███████▎  | 164213/225000 [317:24:48<84:11:20,  4.99s/it]
 73%|███████▎  | 164214/225000 [317:24:53<84:53:04,  5.03s/it]
 73%|███████▎  | 164215/225000 [317:24:58<85:13:54,  5.05s/it]
 73%|███████▎  | 164216/225000 [317:25:03<84:25:22,  5.00s/it]
 73%|███████▎  | 164217/225000 [317:25:08<82:53:48,  4.91s/it]
 73%|███████▎  | 164218/225000 [317:25:13<83:39:26,  4.95s/it]
 73%|███████▎  | 164219/225000 [317:25:18<82:45:17,  4.90s/it]
 73%|███████▎  | 164220/225000 [317:25:24<87:50:15,  5.20s/it]
 73%|███████▎  | 164221/225000 [317:25:35<118:51:27,  7.04s/it]
 73%|███████▎  | 164222/225000 [317:25:39<102:21:51,  6.06s/it]
 73%|███████▎  | 164223/225000 [317:25:43<91:20:21,  5.41s/it] 
 73%|███████▎  | 164224/225000 [317:25:51<106:53:26,  6.33s/it]
 73%|███████▎  | 164225/225000 [317:25:55<96:12:49,  5.70s/it] 
 73%|███████▎  | 164226/225000 [317:26:03<106:33:45,  6.31s/it]
 73%|███████▎  | 164227/225000 [317:26:06<90:16:48,  5.35s/it] 
 73%|███████▎  | 164228/225000 [317:26:14<100:54:36,  5.98s/it]
 73%|███████▎  | 164229/225000 [317:26:18<91:58:42,  5.45s/it] 
 73%|███████▎  | 164230/225000 [317:26:23<89:48:30,  5.32s/it]
 73%|███████▎  | 164231/225000 [317:26:29<94:02:15,  5.57s/it]
 73%|███████▎  | 164232/225000 [317:26:33<85:26:26,  5.06s/it]
 73%|███████▎  | 164233/225000 [317:26:41<100:14:14,  5.94s/it]
 73%|███████▎  | 164234/225000 [317:26:45<90:39:04,  5.37s/it] 
 73%|███████▎  | 164235/225000 [317:26:53<103:13:50,  6.12s/it]
 73%|███████▎  | 164236/225000 [317:26:56<89:31:33,  5.30s/it] 
 73%|███████▎  | 164237/225000 [317:27:00<83:15:09,  4.93s/it]
 73%|███████▎  | 164238/225000 [317:27:04<77:23:23,  4.59s/it]
 73%|███████▎  | 164239/225000 [317:27:08<74:52:56,  4.44s/it]
 73%|███████▎  | 164240/225000 [317:27:19<108:35:40,  6.43s/it]
 73%|███████▎  | 164241/225000 [317:27:24<101:50:07,  6.03s/it]
 73%|███████▎  | 164242/225000 [317:27:31<103:04:24,  6.11s/it]
 73%|███████▎  | 164243/225000 [317:27:36<98:21:19,  5.83s/it] 
 73%|███████▎  | 164244/225000 [317:27:40<92:47:16,  5.50s/it]
 73%|███████▎  | 164245/225000 [317:27:45<86:58:17,  5.15s/it]
 73%|███████▎  | 164246/225000 [317:27:49<84:09:31,  4.99s/it]
 73%|███████▎  | 164247/225000 [317:27:58<102:58:15,  6.10s/it]
 73%|███████▎  | 164248/225000 [317:28:06<110:40:41,  6.56s/it]
 73%|███████▎  | 164249/225000 [317:28:10<98:00:19,  5.81s/it] 
 73%|███████▎  | 164250/225000 [317:28:15<97:27:28,  5.78s/it]
                                                              
{'loss': '0.7743', 'grad_norm': '1.222', 'learning_rate': '9.241e-06', 'epoch': '0.73', 'num_input_tokens_seen': 525574080, 'train_runtime': '1.143e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164250/225000 [317:28:15<97:27:28,  5.78s/it]
 73%|███████▎  | 164251/225000 [317:28:22<101:03:53,  5.99s/it]
 73%|███████▎  | 164252/225000 [317:28:25<88:09:11,  5.22s/it] 
 73%|███████▎  | 164253/225000 [317:28:33<100:33:50,  5.96s/it]
 73%|███████▎  | 164254/225000 [317:28:37<89:45:05,  5.32s/it] 
 73%|███████▎  | 164255/225000 [317:28:45<104:31:47,  6.19s/it]
 73%|███████▎  | 164256/225000 [317:28:49<90:32:47,  5.37s/it] 
 73%|███████▎  | 164257/225000 [317:28:54<89:28:44,  5.30s/it]
 73%|███████▎  | 164258/225000 [317:28:59<90:18:22,  5.35s/it]
 73%|███████▎  | 164259/225000 [317:29:04<88:08:52,  5.22s/it]
 73%|███████▎  | 164260/225000 [317:29:09<88:43:47,  5.26s/it]
 73%|███████▎  | 164261/225000 [317:29:14<87:17:00,  5.17s/it]
 73%|███████▎  | 164262/225000 [317:29:20<86:52:52,  5.15s/it]
 73%|███████▎  | 164263/225000 [317:29:25<86:53:36,  5.15s/it]
 73%|███████▎  | 164264/225000 [317:29:29<84:27:27,  5.01s/it]
 73%|███████▎  | 164265/225000 [317:29:35<88:31:14,  5.25s/it]
 73%|███████▎  | 164266/225000 [317:29:40<88:49:49,  5.27s/it]
 73%|███████▎  | 164267/225000 [317:29:46<88:01:35,  5.22s/it]
 73%|███████▎  | 164268/225000 [317:29:50<85:58:07,  5.10s/it]
 73%|███████▎  | 164269/225000 [317:30:01<115:33:19,  6.85s/it]
 73%|███████▎  | 164270/225000 [317:30:08<116:49:08,  6.92s/it]
 73%|███████▎  | 164271/225000 [317:30:12<99:37:58,  5.91s/it] 
 73%|███████▎  | 164272/225000 [317:30:17<96:55:53,  5.75s/it]
 73%|███████▎  | 164273/225000 [317:30:23<96:24:21,  5.72s/it]
 73%|███████▎  | 164274/225000 [317:30:32<114:19:15,  6.78s/it]
 73%|███████▎  | 164275/225000 [317:30:36<100:38:46,  5.97s/it]
 73%|███████▎  | 164276/225000 [317:30:43<103:26:09,  6.13s/it]
 73%|███████▎  | 164277/225000 [317:30:46<89:48:08,  5.32s/it] 
 73%|███████▎  | 164278/225000 [317:30:52<94:18:07,  5.59s/it]
 73%|███████▎  | 164279/225000 [317:30:58<92:29:47,  5.48s/it]
 73%|███████▎  | 164280/225000 [317:31:01<82:41:23,  4.90s/it]
 73%|███████▎  | 164281/225000 [317:31:05<77:07:01,  4.57s/it]
 73%|███████▎  | 164282/225000 [317:31:12<90:32:34,  5.37s/it]
 73%|███████▎  | 164283/225000 [317:31:20<102:46:33,  6.09s/it]
 73%|███████▎  | 164284/225000 [317:31:24<93:15:06,  5.53s/it] 
 73%|███████▎  | 164285/225000 [317:31:29<88:26:49,  5.24s/it]
 73%|███████▎  | 164286/225000 [317:31:33<84:09:45,  4.99s/it]
 73%|███████▎  | 164287/225000 [317:31:38<84:01:50,  4.98s/it]
 73%|███████▎  | 164288/225000 [317:31:49<115:19:41,  6.84s/it]
 73%|███████▎  | 164289/225000 [317:31:53<98:28:11,  5.84s/it] 
 73%|███████▎  | 164290/225000 [317:31:58<96:18:15,  5.71s/it]
 73%|███████▎  | 164291/225000 [317:32:05<99:41:14,  5.91s/it]
 73%|███████▎  | 164292/225000 [317:32:08<88:22:53,  5.24s/it]
 73%|███████▎  | 164293/225000 [317:32:12<80:15:26,  4.76s/it]
 73%|███████▎  | 164294/225000 [317:32:23<110:47:36,  6.57s/it]
 73%|███████▎  | 164295/225000 [317:32:26<95:37:17,  5.67s/it] 
 73%|███████▎  | 164296/225000 [317:32:30<85:18:57,  5.06s/it]
 73%|███████▎  | 164297/225000 [317:32:34<80:45:46,  4.79s/it]
 73%|███████▎  | 164298/225000 [317:32:45<109:24:05,  6.49s/it]
 73%|███████▎  | 164299/225000 [317:32:50<102:53:48,  6.10s/it]
 73%|███████▎  | 164300/225000 [317:32:56<101:10:41,  6.00s/it]
                                                               
{'loss': '0.7773', 'grad_norm': '1.105', 'learning_rate': '9.227e-06', 'epoch': '0.7302', 'num_input_tokens_seen': 525704968, 'train_runtime': '1.143e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164300/225000 [317:32:56<101:10:41,  6.00s/it]
 73%|███████▎  | 164301/225000 [317:33:02<105:30:36,  6.26s/it]
 73%|███████▎  | 164302/225000 [317:33:06<91:22:29,  5.42s/it] 
 73%|███████▎  | 164303/225000 [317:33:10<85:08:33,  5.05s/it]
 73%|███████▎  | 164304/225000 [317:33:14<81:25:00,  4.83s/it]
 73%|███████▎  | 164305/225000 [317:33:19<80:10:48,  4.76s/it]
 73%|███████▎  | 164306/225000 [317:33:29<106:58:50,  6.35s/it]
 73%|███████▎  | 164307/225000 [317:33:34<100:56:40,  5.99s/it]
 73%|███████▎  | 164308/225000 [317:33:38<88:15:22,  5.24s/it] 
 73%|███████▎  | 164309/225000 [317:33:42<82:17:17,  4.88s/it]
 73%|███████▎  | 164310/225000 [317:33:50<101:33:11,  6.02s/it]
 73%|███████▎  | 164311/225000 [317:33:57<105:26:25,  6.25s/it]
 73%|███████▎  | 164312/225000 [317:34:03<101:43:13,  6.03s/it]
 73%|███████▎  | 164313/225000 [317:34:08<96:57:26,  5.75s/it] 
 73%|███████▎  | 164314/225000 [317:34:13<96:40:05,  5.73s/it]
 73%|███████▎  | 164315/225000 [317:34:19<93:31:21,  5.55s/it]
 73%|███████▎  | 164316/225000 [317:34:24<92:48:30,  5.51s/it]
 73%|███████▎  | 164317/225000 [317:34:29<91:56:03,  5.45s/it]
 73%|███████▎  | 164318/225000 [317:34:35<90:40:31,  5.38s/it]
 73%|███████▎  | 164319/225000 [317:34:40<90:59:29,  5.40s/it]
 73%|███████▎  | 164320/225000 [317:34:45<89:13:19,  5.29s/it]
 73%|███████▎  | 164321/225000 [317:34:50<88:09:27,  5.23s/it]
 73%|███████▎  | 164322/225000 [317:34:56<89:17:33,  5.30s/it]
 73%|███████▎  | 164323/225000 [317:35:01<89:53:37,  5.33s/it]
 73%|███████▎  | 164324/225000 [317:35:06<89:23:16,  5.30s/it]
 73%|███████▎  | 164325/225000 [317:35:11<89:12:21,  5.29s/it]
 73%|███████▎  | 164326/225000 [317:35:16<87:33:43,  5.20s/it]
 73%|███████▎  | 164327/225000 [317:35:22<89:35:25,  5.32s/it]
 73%|███████▎  | 164328/225000 [317:35:27<90:05:21,  5.35s/it]
 73%|███████▎  | 164329/225000 [317:35:33<90:16:00,  5.36s/it]
 73%|███████▎  | 164330/225000 [317:35:38<91:18:56,  5.42s/it]
 73%|███████▎  | 164331/225000 [317:35:44<90:02:52,  5.34s/it]
 73%|███████▎  | 164332/225000 [317:35:49<89:49:50,  5.33s/it]
 73%|███████▎  | 164333/225000 [317:35:54<90:12:57,  5.35s/it]
 73%|███████▎  | 164334/225000 [317:36:00<89:50:41,  5.33s/it]
 73%|███████▎  | 164335/225000 [317:36:04<87:38:50,  5.20s/it]
 73%|███████▎  | 164336/225000 [317:36:10<88:15:19,  5.24s/it]
 73%|███████▎  | 164337/225000 [317:36:22<122:55:27,  7.29s/it]
 73%|███████▎  | 164338/225000 [317:36:26<108:46:41,  6.46s/it]
 73%|███████▎  | 164339/225000 [317:36:31<97:31:45,  5.79s/it] 
 73%|███████▎  | 164340/225000 [317:36:35<91:31:08,  5.43s/it]
 73%|███████▎  | 164341/225000 [317:36:39<83:58:43,  4.98s/it]
 73%|███████▎  | 164342/225000 [317:36:44<83:33:24,  4.96s/it]
 73%|███████▎  | 164343/225000 [317:36:57<124:47:24,  7.41s/it]
 73%|███████▎  | 164344/225000 [317:37:01<105:00:05,  6.23s/it]
 73%|███████▎  | 164345/225000 [317:37:04<90:06:32,  5.35s/it] 
 73%|███████▎  | 164346/225000 [317:37:11<98:14:17,  5.83s/it]
 73%|███████▎  | 164347/225000 [317:37:17<99:07:15,  5.88s/it]
 73%|███████▎  | 164348/225000 [317:37:21<88:38:04,  5.26s/it]
 73%|███████▎  | 164349/225000 [317:37:25<82:03:38,  4.87s/it]
 73%|███████▎  | 164350/225000 [317:37:35<109:19:27,  6.49s/it]
                                                               
{'loss': '0.7744', 'grad_norm': '1.153', 'learning_rate': '9.213e-06', 'epoch': '0.7304', 'num_input_tokens_seen': 525834280, 'train_runtime': '1.143e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164350/225000 [317:37:35<109:19:27,  6.49s/it]
 73%|███████▎  | 164351/225000 [317:37:38<92:52:04,  5.51s/it] 
 73%|███████▎  | 164352/225000 [317:37:42<84:24:50,  5.01s/it]
 73%|███████▎  | 164353/225000 [317:37:50<101:33:16,  6.03s/it]
 73%|███████▎  | 164354/225000 [317:37:56<100:05:47,  5.94s/it]
 73%|███████▎  | 164355/225000 [317:38:02<101:14:30,  6.01s/it]
 73%|███████▎  | 164356/225000 [317:38:08<97:19:39,  5.78s/it] 
 73%|███████▎  | 164357/225000 [317:38:14<98:55:45,  5.87s/it]
 73%|███████▎  | 164358/225000 [317:38:17<87:15:07,  5.18s/it]
 73%|███████▎  | 164359/225000 [317:38:24<94:14:59,  5.60s/it]
 73%|███████▎  | 164360/225000 [317:38:28<89:05:46,  5.29s/it]
 73%|███████▎  | 164361/225000 [317:38:33<87:12:52,  5.18s/it]
 73%|███████▎  | 164362/225000 [317:38:38<84:03:39,  4.99s/it]
 73%|███████▎  | 164363/225000 [317:38:43<82:32:26,  4.90s/it]
 73%|███████▎  | 164364/225000 [317:38:47<79:27:25,  4.72s/it]
 73%|███████▎  | 164365/225000 [317:38:51<78:18:33,  4.65s/it]
 73%|███████▎  | 164366/225000 [317:38:56<78:44:22,  4.67s/it]
 73%|███████▎  | 164367/225000 [317:39:06<103:56:47,  6.17s/it]
 73%|███████▎  | 164368/225000 [317:39:13<107:55:55,  6.41s/it]
 73%|███████▎  | 164369/225000 [317:39:16<93:47:28,  5.57s/it] 
 73%|███████▎  | 164370/225000 [317:39:24<106:59:32,  6.35s/it]
 73%|███████▎  | 164371/225000 [317:39:28<90:34:39,  5.38s/it] 
 73%|███████▎  | 164372/225000 [317:39:32<86:04:51,  5.11s/it]
 73%|███████▎  | 164373/225000 [317:39:41<106:21:28,  6.32s/it]
 73%|███████▎  | 164374/225000 [317:39:44<90:54:20,  5.40s/it] 
 73%|███████▎  | 164375/225000 [317:39:50<93:53:56,  5.58s/it]
 73%|███████▎  | 164376/225000 [317:39:56<96:07:02,  5.71s/it]
 73%|███████▎  | 164377/225000 [317:40:01<89:15:11,  5.30s/it]
 73%|███████▎  | 164378/225000 [317:40:05<86:04:50,  5.11s/it]
 73%|███████▎  | 164379/225000 [317:40:10<82:22:01,  4.89s/it]
 73%|███████▎  | 164380/225000 [317:40:14<79:03:43,  4.70s/it]
 73%|███████▎  | 164381/225000 [317:40:19<80:12:11,  4.76s/it]
 73%|███████▎  | 164382/225000 [317:40:29<104:57:23,  6.23s/it]
 73%|███████▎  | 164383/225000 [317:40:36<111:17:08,  6.61s/it]
 73%|███████▎  | 164384/225000 [317:40:42<108:01:06,  6.42s/it]
 73%|███████▎  | 164385/225000 [317:40:47<98:58:23,  5.88s/it] 
 73%|███████▎  | 164386/225000 [317:40:51<92:26:17,  5.49s/it]
 73%|███████▎  | 164387/225000 [317:40:56<88:00:45,  5.23s/it]
 73%|███████▎  | 164388/225000 [317:41:01<85:26:01,  5.07s/it]
 73%|███████▎  | 164389/225000 [317:41:05<83:19:04,  4.95s/it]
 73%|███████▎  | 164390/225000 [317:41:10<81:57:02,  4.87s/it]
 73%|███████▎  | 164391/225000 [317:41:15<80:36:01,  4.79s/it]
 73%|███████▎  | 164392/225000 [317:41:27<117:38:27,  6.99s/it]
 73%|███████▎  | 164393/225000 [317:41:30<98:12:11,  5.83s/it] 
 73%|███████▎  | 164394/225000 [317:41:36<98:21:52,  5.84s/it]
 73%|███████▎  | 164395/225000 [317:41:43<103:09:53,  6.13s/it]
 73%|███████▎  | 164396/225000 [317:41:46<91:30:26,  5.44s/it] 
 73%|███████▎  | 164397/225000 [317:41:50<83:33:20,  4.96s/it]
 73%|███████▎  | 164398/225000 [317:41:59<105:06:13,  6.24s/it]
 73%|███████▎  | 164399/225000 [317:42:03<93:25:03,  5.55s/it] 
 73%|███████▎  | 164400/225000 [317:42:07<85:44:03,  5.09s/it]
                                                              
{'loss': '0.7761', 'grad_norm': '1.181', 'learning_rate': '9.199e-06', 'epoch': '0.7307', 'num_input_tokens_seen': 525964552, 'train_runtime': '1.144e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164400/225000 [317:42:07<85:44:03,  5.09s/it]
 73%|███████▎  | 164401/225000 [317:42:17<106:36:22,  6.33s/it]
 73%|███████▎  | 164402/225000 [317:42:20<93:04:08,  5.53s/it] 
 73%|███████▎  | 164403/225000 [317:42:29<107:45:43,  6.40s/it]
 73%|███████▎  | 164404/225000 [317:42:32<92:52:15,  5.52s/it] 
 73%|███████▎  | 164405/225000 [317:42:36<84:44:11,  5.03s/it]
 73%|███████▎  | 164406/225000 [317:42:41<85:14:13,  5.06s/it]
 73%|███████▎  | 164407/225000 [317:42:46<85:06:13,  5.06s/it]
 73%|███████▎  | 164408/225000 [317:42:51<84:00:12,  4.99s/it]
 73%|███████▎  | 164409/225000 [317:42:56<83:06:17,  4.94s/it]
 73%|███████▎  | 164410/225000 [317:43:01<81:47:18,  4.86s/it]
 73%|███████▎  | 164411/225000 [317:43:05<81:21:43,  4.83s/it]
 73%|███████▎  | 164412/225000 [317:43:10<79:21:09,  4.71s/it]
 73%|███████▎  | 164413/225000 [317:43:18<94:42:09,  5.63s/it]
 73%|███████▎  | 164414/225000 [317:43:30<129:42:48,  7.71s/it]
 73%|███████▎  | 164415/225000 [317:43:33<106:29:59,  6.33s/it]
 73%|███████▎  | 164416/225000 [317:43:39<103:01:20,  6.12s/it]
 73%|███████▎  | 164417/225000 [317:43:45<102:26:16,  6.09s/it]
 73%|███████▎  | 164418/225000 [317:43:50<95:30:36,  5.68s/it] 
 73%|███████▎  | 164419/225000 [317:43:54<91:47:25,  5.45s/it]
 73%|███████▎  | 164420/225000 [317:43:59<88:02:40,  5.23s/it]
 73%|███████▎  | 164421/225000 [317:44:04<87:17:52,  5.19s/it]
 73%|███████▎  | 164422/225000 [317:44:09<85:46:55,  5.10s/it]
 73%|███████▎  | 164423/225000 [317:44:14<82:47:49,  4.92s/it]
 73%|███████▎  | 164424/225000 [317:44:19<85:08:27,  5.06s/it]
 73%|███████▎  | 164425/225000 [317:44:24<83:05:06,  4.94s/it]
 73%|███████▎  | 164426/225000 [317:44:29<82:28:37,  4.90s/it]
 73%|███████▎  | 164427/225000 [317:44:34<84:09:21,  5.00s/it]
 73%|███████▎  | 164428/225000 [317:44:39<85:03:18,  5.06s/it]
 73%|███████▎  | 164429/225000 [317:44:44<85:01:58,  5.05s/it]
 73%|███████▎  | 164430/225000 [317:44:54<110:17:51,  6.56s/it]
 73%|███████▎  | 164431/225000 [317:45:01<111:47:43,  6.64s/it]
 73%|███████▎  | 164432/225000 [317:45:05<101:13:53,  6.02s/it]
 73%|███████▎  | 164433/225000 [317:45:10<95:45:12,  5.69s/it] 
 73%|███████▎  | 164434/225000 [317:45:15<90:29:01,  5.38s/it]
 73%|███████▎  | 164435/225000 [317:45:20<88:50:03,  5.28s/it]
 73%|███████▎  | 164436/225000 [317:45:25<85:46:31,  5.10s/it]
 73%|███████▎  | 164437/225000 [317:45:30<84:24:17,  5.02s/it]
 73%|███████▎  | 164438/225000 [317:45:35<85:05:29,  5.06s/it]
 73%|███████▎  | 164439/225000 [317:45:46<118:04:19,  7.02s/it]
 73%|███████▎  | 164440/225000 [317:45:51<105:40:00,  6.28s/it]
 73%|███████▎  | 164441/225000 [317:45:55<96:32:25,  5.74s/it] 
 73%|███████▎  | 164442/225000 [317:46:00<92:13:45,  5.48s/it]
 73%|███████▎  | 164443/225000 [317:46:05<88:07:37,  5.24s/it]
 73%|███████▎  | 164444/225000 [317:46:10<85:01:49,  5.05s/it]
 73%|███████▎  | 164445/225000 [317:46:15<84:57:56,  5.05s/it]
 73%|███████▎  | 164446/225000 [317:46:20<84:16:34,  5.01s/it]
 73%|███████▎  | 164447/225000 [317:46:27<99:11:36,  5.90s/it]
 73%|███████▎  | 164448/225000 [317:46:39<127:30:44,  7.58s/it]
 73%|███████▎  | 164449/225000 [317:46:42<105:19:53,  6.26s/it]
 73%|███████▎  | 164450/225000 [317:46:51<116:53:33,  6.95s/it]
                                                               
{'loss': '0.775', 'grad_norm': '1.126', 'learning_rate': '9.185e-06', 'epoch': '0.7309', 'num_input_tokens_seen': 526095080, 'train_runtime': '1.144e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164450/225000 [317:46:51<116:53:33,  6.95s/it]
 73%|███████▎  | 164451/225000 [317:46:54<97:10:11,  5.78s/it] 
 73%|███████▎  | 164452/225000 [317:46:58<88:34:52,  5.27s/it]
 73%|███████▎  | 164453/225000 [317:47:03<87:11:50,  5.18s/it]
 73%|███████▎  | 164454/225000 [317:47:08<86:15:22,  5.13s/it]
 73%|███████▎  | 164455/225000 [317:47:13<83:57:48,  4.99s/it]
 73%|███████▎  | 164456/225000 [317:47:18<84:25:15,  5.02s/it]
 73%|███████▎  | 164457/225000 [317:47:23<85:23:10,  5.08s/it]
 73%|███████▎  | 164458/225000 [317:47:31<99:24:48,  5.91s/it]
 73%|███████▎  | 164459/225000 [317:47:40<114:13:14,  6.79s/it]
 73%|███████▎  | 164460/225000 [317:47:43<97:37:10,  5.80s/it] 
 73%|███████▎  | 164461/225000 [317:47:47<86:05:09,  5.12s/it]
 73%|███████▎  | 164462/225000 [317:47:55<104:16:42,  6.20s/it]
 73%|███████▎  | 164463/225000 [317:47:59<94:18:07,  5.61s/it] 
 73%|███████▎  | 164464/225000 [317:48:04<86:49:52,  5.16s/it]
 73%|███████▎  | 164465/225000 [317:48:08<80:25:31,  4.78s/it]
 73%|███████▎  | 164466/225000 [317:48:12<77:28:54,  4.61s/it]
 73%|███████▎  | 164467/225000 [317:48:16<77:10:42,  4.59s/it]
 73%|███████▎  | 164468/225000 [317:48:26<102:13:15,  6.08s/it]
 73%|███████▎  | 164469/225000 [317:48:34<111:14:21,  6.62s/it]
 73%|███████▎  | 164470/225000 [317:48:38<97:40:40,  5.81s/it] 
 73%|███████▎  | 164471/225000 [317:48:42<88:41:25,  5.27s/it]
 73%|███████▎  | 164472/225000 [317:48:49<97:36:57,  5.81s/it]
 73%|███████▎  | 164473/225000 [317:48:56<106:03:06,  6.31s/it]
 73%|███████▎  | 164474/225000 [317:49:00<95:12:02,  5.66s/it] 
 73%|███████▎  | 164475/225000 [317:49:05<88:23:57,  5.26s/it]
 73%|███████▎  | 164476/225000 [317:49:09<83:23:34,  4.96s/it]
 73%|███████▎  | 164477/225000 [317:49:13<79:08:08,  4.71s/it]
 73%|███████▎  | 164478/225000 [317:49:24<111:07:03,  6.61s/it]
 73%|███████▎  | 164479/225000 [317:49:28<97:22:18,  5.79s/it] 
 73%|███████▎  | 164480/225000 [317:49:32<86:55:34,  5.17s/it]
 73%|███████▎  | 164481/225000 [317:49:37<85:33:29,  5.09s/it]
 73%|███████▎  | 164482/225000 [317:49:41<83:18:48,  4.96s/it]
 73%|███████▎  | 164483/225000 [317:49:46<84:42:00,  5.04s/it]
 73%|███████▎  | 164484/225000 [317:49:51<84:24:37,  5.02s/it]
 73%|███████▎  | 164485/225000 [317:49:56<83:07:07,  4.94s/it]
 73%|███████▎  | 164486/225000 [317:50:02<85:20:45,  5.08s/it]
 73%|███████▎  | 164487/225000 [317:50:07<86:19:18,  5.14s/it]
 73%|███████▎  | 164488/225000 [317:50:19<121:44:23,  7.24s/it]
 73%|███████▎  | 164489/225000 [317:50:24<111:16:28,  6.62s/it]
 73%|███████▎  | 164490/225000 [317:50:28<97:02:59,  5.77s/it] 
 73%|███████▎  | 164491/225000 [317:50:37<114:29:07,  6.81s/it]
 73%|███████▎  | 164492/225000 [317:50:41<97:03:43,  5.77s/it] 
 73%|███████▎  | 164493/225000 [317:50:47<102:15:07,  6.08s/it]
 73%|███████▎  | 164494/225000 [317:50:53<99:02:16,  5.89s/it] 
 73%|███████▎  | 164495/225000 [317:51:00<107:16:59,  6.38s/it]
 73%|███████▎  | 164496/225000 [317:51:05<96:19:19,  5.73s/it] 
 73%|███████▎  | 164497/225000 [317:51:09<87:20:25,  5.20s/it]
 73%|███████▎  | 164498/225000 [317:51:13<82:00:57,  4.88s/it]
 73%|███████▎  | 164499/225000 [317:51:17<78:52:15,  4.69s/it]
 73%|███████▎  | 164500/225000 [317:51:22<78:46:38,  4.69s/it]
                                                              
{'loss': '0.7738', 'grad_norm': '1.092', 'learning_rate': '9.17e-06', 'epoch': '0.7311', 'num_input_tokens_seen': 526226384, 'train_runtime': '1.144e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164500/225000 [317:51:22<78:46:38,  4.69s/it]
 73%|███████▎  | 164501/225000 [317:51:32<107:23:38,  6.39s/it]
 73%|███████▎  | 164502/225000 [317:51:37<99:00:34,  5.89s/it] 
 73%|███████▎  | 164503/225000 [317:51:43<100:54:52,  6.01s/it]
 73%|███████▎  | 164504/225000 [317:51:50<105:41:26,  6.29s/it]
 73%|███████▎  | 164505/225000 [317:51:55<98:54:39,  5.89s/it] 
 73%|███████▎  | 164506/225000 [317:51:59<91:07:48,  5.42s/it]
 73%|███████▎  | 164507/225000 [317:52:04<88:00:07,  5.24s/it]
 73%|███████▎  | 164508/225000 [317:52:09<88:04:38,  5.24s/it]
 73%|███████▎  | 164509/225000 [317:52:14<86:25:30,  5.14s/it]
 73%|███████▎  | 164510/225000 [317:52:19<85:49:50,  5.11s/it]
 73%|███████▎  | 164511/225000 [317:52:24<83:40:34,  4.98s/it]
 73%|███████▎  | 164512/225000 [317:52:29<84:49:54,  5.05s/it]
 73%|███████▎  | 164513/225000 [317:52:34<83:56:49,  5.00s/it]
 73%|███████▎  | 164514/225000 [317:52:39<85:27:12,  5.09s/it]
 73%|███████▎  | 164515/225000 [317:52:44<84:09:15,  5.01s/it]
 73%|███████▎  | 164516/225000 [317:52:49<85:22:38,  5.08s/it]
 73%|███████▎  | 164517/225000 [317:52:54<85:11:10,  5.07s/it]
 73%|███████▎  | 164518/225000 [317:53:00<85:40:51,  5.10s/it]
 73%|███████▎  | 164519/225000 [317:53:12<123:52:08,  7.37s/it]
 73%|███████▎  | 164520/225000 [317:53:16<107:46:51,  6.42s/it]
 73%|███████▎  | 164521/225000 [317:53:23<108:51:05,  6.48s/it]
 73%|███████▎  | 164522/225000 [317:53:27<96:56:51,  5.77s/it] 
 73%|███████▎  | 164523/225000 [317:53:34<103:32:19,  6.16s/it]
 73%|███████▎  | 164524/225000 [317:53:39<97:05:49,  5.78s/it] 
 73%|███████▎  | 164525/225000 [317:53:44<93:28:37,  5.56s/it]
 73%|███████▎  | 164526/225000 [317:53:49<91:28:47,  5.45s/it]
 73%|███████▎  | 164527/225000 [317:53:54<88:41:57,  5.28s/it]
 73%|███████▎  | 164528/225000 [317:53:59<87:54:23,  5.23s/it]
 73%|███████▎  | 164529/225000 [317:54:04<87:25:05,  5.20s/it]
 73%|███████▎  | 164530/225000 [317:54:10<87:29:33,  5.21s/it]
 73%|███████▎  | 164531/225000 [317:54:15<86:24:35,  5.14s/it]
 73%|███████▎  | 164532/225000 [317:54:19<83:29:03,  4.97s/it]
 73%|███████▎  | 164533/225000 [317:54:24<84:37:20,  5.04s/it]
 73%|███████▎  | 164534/225000 [317:54:30<87:04:56,  5.18s/it]
 73%|███████▎  | 164535/225000 [317:54:35<85:39:10,  5.10s/it]
 73%|███████▎  | 164536/225000 [317:54:40<85:30:39,  5.09s/it]
 73%|███████▎  | 164537/225000 [317:54:45<86:14:26,  5.13s/it]
 73%|███████▎  | 164538/225000 [317:54:50<85:42:44,  5.10s/it]
 73%|███████▎  | 164539/225000 [317:55:03<124:45:20,  7.43s/it]
 73%|███████▎  | 164540/225000 [317:55:08<111:26:43,  6.64s/it]
 73%|███████▎  | 164541/225000 [317:55:11<95:56:42,  5.71s/it] 
 73%|███████▎  | 164542/225000 [317:55:21<113:48:43,  6.78s/it]
 73%|███████▎  | 164543/225000 [317:55:24<97:37:27,  5.81s/it] 
 73%|███████▎  | 164544/225000 [317:55:28<85:31:33,  5.09s/it]
 73%|███████▎  | 164545/225000 [317:55:36<100:11:04,  5.97s/it]
 73%|███████▎  | 164546/225000 [317:55:41<94:57:50,  5.66s/it] 
 73%|███████▎  | 164547/225000 [317:55:46<93:53:19,  5.59s/it]
 73%|███████▎  | 164548/225000 [317:55:52<94:09:13,  5.61s/it]
 73%|███████▎  | 164549/225000 [317:55:56<89:56:49,  5.36s/it]
 73%|███████▎  | 164550/225000 [317:56:02<89:22:54,  5.32s/it]
                                                              
{'loss': '0.7755', 'grad_norm': '1.089', 'learning_rate': '9.157e-06', 'epoch': '0.7313', 'num_input_tokens_seen': 526356344, 'train_runtime': '1.145e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164550/225000 [317:56:02<89:22:54,  5.32s/it]
 73%|███████▎  | 164551/225000 [317:56:07<88:38:39,  5.28s/it]
 73%|███████▎  | 164552/225000 [317:56:13<90:48:12,  5.41s/it]
 73%|███████▎  | 164553/225000 [317:56:18<90:08:41,  5.37s/it]
 73%|███████▎  | 164554/225000 [317:56:23<88:48:20,  5.29s/it]
 73%|███████▎  | 164555/225000 [317:56:29<90:03:52,  5.36s/it]
 73%|███████▎  | 164556/225000 [317:56:34<90:32:16,  5.39s/it]
 73%|███████▎  | 164557/225000 [317:56:39<89:06:27,  5.31s/it]
 73%|███████▎  | 164558/225000 [317:56:50<117:21:28,  6.99s/it]
 73%|███████▎  | 164559/225000 [317:56:54<102:38:43,  6.11s/it]
 73%|███████▎  | 164560/225000 [317:57:01<106:33:11,  6.35s/it]
 73%|███████▎  | 164561/225000 [317:57:06<98:59:14,  5.90s/it] 
 73%|███████▎  | 164562/225000 [317:57:10<91:40:12,  5.46s/it]
 73%|███████▎  | 164563/225000 [317:57:15<90:36:29,  5.40s/it]
 73%|███████▎  | 164564/225000 [317:57:20<87:54:51,  5.24s/it]
 73%|███████▎  | 164565/225000 [317:57:25<86:41:50,  5.16s/it]
 73%|███████▎  | 164566/225000 [317:57:30<86:33:31,  5.16s/it]
 73%|███████▎  | 164567/225000 [317:57:35<85:01:41,  5.07s/it]
 73%|███████▎  | 164568/225000 [317:57:40<83:12:25,  4.96s/it]
 73%|███████▎  | 164569/225000 [317:57:45<81:39:04,  4.86s/it]
 73%|███████▎  | 164570/225000 [317:57:54<102:30:21,  6.11s/it]
 73%|███████▎  | 164571/225000 [317:58:02<113:29:13,  6.76s/it]
 73%|███████▎  | 164572/225000 [317:58:05<96:59:01,  5.78s/it] 
 73%|███████▎  | 164573/225000 [317:58:15<114:47:37,  6.84s/it]
 73%|███████▎  | 164574/225000 [317:58:18<98:56:56,  5.90s/it] 
 73%|███████▎  | 164575/225000 [317:58:22<87:05:27,  5.19s/it]
 73%|███████▎  | 164576/225000 [317:58:31<107:55:37,  6.43s/it]
 73%|███████▎  | 164577/225000 [317:58:35<94:28:20,  5.63s/it] 
 73%|███████▎  | 164578/225000 [317:58:39<87:04:26,  5.19s/it]
 73%|███████▎  | 164579/225000 [317:58:44<83:12:55,  4.96s/it]
 73%|███████▎  | 164580/225000 [317:58:48<80:50:12,  4.82s/it]
 73%|███████▎  | 164581/225000 [317:58:56<93:46:53,  5.59s/it]
 73%|███████▎  | 164582/225000 [317:59:05<115:03:13,  6.86s/it]
 73%|███████▎  | 164583/225000 [317:59:11<109:02:02,  6.50s/it]
 73%|███████▎  | 164584/225000 [317:59:16<103:28:07,  6.17s/it]
 73%|███████▎  | 164585/225000 [317:59:20<89:52:45,  5.36s/it] 
 73%|███████▎  | 164586/225000 [317:59:24<83:46:32,  4.99s/it]
 73%|███████▎  | 164587/225000 [317:59:28<80:08:29,  4.78s/it]
 73%|███████▎  | 164588/225000 [317:59:33<80:40:36,  4.81s/it]
 73%|███████▎  | 164589/225000 [317:59:42<101:19:00,  6.04s/it]
 73%|███████▎  | 164590/225000 [317:59:49<107:45:21,  6.42s/it]
 73%|███████▎  | 164591/225000 [317:59:54<97:47:33,  5.83s/it] 
 73%|███████▎  | 164592/225000 [317:59:58<91:46:24,  5.47s/it]
 73%|███████▎  | 164593/225000 [318:00:03<85:44:12,  5.11s/it]
 73%|███████▎  | 164594/225000 [318:00:07<81:53:10,  4.88s/it]
 73%|███████▎  | 164595/225000 [318:00:12<82:09:01,  4.90s/it]
 73%|███████▎  | 164596/225000 [318:00:23<114:15:28,  6.81s/it]
 73%|███████▎  | 164597/225000 [318:00:27<98:10:30,  5.85s/it] 
 73%|███████▎  | 164598/225000 [318:00:30<86:31:31,  5.16s/it]
 73%|███████▎  | 164599/225000 [318:00:39<102:34:33,  6.11s/it]
 73%|███████▎  | 164600/225000 [318:00:45<103:43:57,  6.18s/it]
                                                               
{'loss': '0.7682', 'grad_norm': '1.092', 'learning_rate': '9.142e-06', 'epoch': '0.7316', 'num_input_tokens_seen': 526486952, 'train_runtime': '1.145e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164600/225000 [318:00:45<103:43:57,  6.18s/it]
 73%|███████▎  | 164601/225000 [318:00:50<95:47:39,  5.71s/it] 
 73%|███████▎  | 164602/225000 [318:00:54<90:39:06,  5.40s/it]
 73%|███████▎  | 164603/225000 [318:00:59<88:37:37,  5.28s/it]
 73%|███████▎  | 164604/225000 [318:01:04<86:36:19,  5.16s/it]
 73%|███████▎  | 164605/225000 [318:01:09<85:20:09,  5.09s/it]
 73%|███████▎  | 164606/225000 [318:01:14<83:35:03,  4.98s/it]
 73%|███████▎  | 164607/225000 [318:01:19<84:33:00,  5.04s/it]
 73%|███████▎  | 164608/225000 [318:01:24<83:35:24,  4.98s/it]
 73%|███████▎  | 164609/225000 [318:01:29<83:09:09,  4.96s/it]
 73%|███████▎  | 164610/225000 [318:01:40<113:19:27,  6.76s/it]
 73%|███████▎  | 164611/225000 [318:01:46<111:22:55,  6.64s/it]
 73%|███████▎  | 164612/225000 [318:01:49<94:25:21,  5.63s/it] 
 73%|███████▎  | 164613/225000 [318:02:00<117:58:31,  7.03s/it]
 73%|███████▎  | 164614/225000 [318:02:05<107:40:14,  6.42s/it]
 73%|███████▎  | 164615/225000 [318:02:09<97:16:10,  5.80s/it] 
 73%|███████▎  | 164616/225000 [318:02:14<93:07:58,  5.55s/it]
 73%|███████▎  | 164617/225000 [318:02:19<91:06:34,  5.43s/it]
 73%|███████▎  | 164618/225000 [318:02:24<90:06:04,  5.37s/it]
 73%|███████▎  | 164619/225000 [318:02:29<87:46:14,  5.23s/it]
 73%|███████▎  | 164620/225000 [318:02:35<87:44:47,  5.23s/it]
 73%|███████▎  | 164621/225000 [318:02:40<86:08:38,  5.14s/it]
 73%|███████▎  | 164622/225000 [318:02:45<87:28:30,  5.22s/it]
 73%|███████▎  | 164623/225000 [318:02:50<88:06:00,  5.25s/it]
 73%|███████▎  | 164624/225000 [318:02:55<84:41:18,  5.05s/it]
 73%|███████▎  | 164625/225000 [318:03:00<85:21:45,  5.09s/it]
 73%|███████▎  | 164626/225000 [318:03:05<84:08:28,  5.02s/it]
 73%|███████▎  | 164627/225000 [318:03:10<83:10:44,  4.96s/it]
 73%|███████▎  | 164628/225000 [318:03:15<83:11:11,  4.96s/it]
 73%|███████▎  | 164629/225000 [318:03:23<97:46:23,  5.83s/it]
 73%|███████▎  | 164630/225000 [318:03:34<126:46:16,  7.56s/it]
 73%|███████▎  | 164631/225000 [318:03:38<106:01:09,  6.32s/it]
 73%|███████▎  | 164632/225000 [318:03:41<93:19:46,  5.57s/it] 
 73%|███████▎  | 164633/225000 [318:03:50<106:23:20,  6.34s/it]
 73%|███████▎  | 164634/225000 [318:03:54<96:24:49,  5.75s/it] 
 73%|███████▎  | 164635/225000 [318:03:58<88:14:09,  5.26s/it]
 73%|███████▎  | 164636/225000 [318:04:02<83:29:43,  4.98s/it]
 73%|███████▎  | 164637/225000 [318:04:07<80:59:54,  4.83s/it]
 73%|███████▎  | 164638/225000 [318:04:17<109:25:01,  6.53s/it]
 73%|███████▎  | 164639/225000 [318:04:21<93:26:26,  5.57s/it] 
 73%|███████▎  | 164640/225000 [318:04:28<100:10:32,  5.97s/it]
 73%|███████▎  | 164641/225000 [318:04:32<91:08:11,  5.44s/it] 
 73%|███████▎  | 164642/225000 [318:04:37<89:48:23,  5.36s/it]
 73%|███████▎  | 164643/225000 [318:04:46<107:08:13,  6.39s/it]
 73%|███████▎  | 164644/225000 [318:04:49<92:19:57,  5.51s/it] 
 73%|███████▎  | 164645/225000 [318:04:53<82:29:47,  4.92s/it]
 73%|███████▎  | 164646/225000 [318:05:02<106:40:03,  6.36s/it]
 73%|███████▎  | 164647/225000 [318:05:07<97:24:04,  5.81s/it] 
 73%|███████▎  | 164648/225000 [318:05:13<97:22:31,  5.81s/it]
 73%|███████▎  | 164649/225000 [318:05:18<95:39:36,  5.71s/it]
 73%|███████▎  | 164650/225000 [318:05:24<94:20:38,  5.63s/it]
                                                              
{'loss': '0.7792', 'grad_norm': '1.079', 'learning_rate': '9.128e-06', 'epoch': '0.7318', 'num_input_tokens_seen': 526617512, 'train_runtime': '1.145e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164650/225000 [318:05:24<94:20:38,  5.63s/it]
 73%|███████▎  | 164651/225000 [318:05:29<92:35:45,  5.52s/it]
 73%|███████▎  | 164652/225000 [318:05:37<105:15:06,  6.28s/it]
 73%|███████▎  | 164653/225000 [318:05:42<100:56:54,  6.02s/it]
 73%|███████▎  | 164654/225000 [318:05:49<105:51:56,  6.32s/it]
 73%|███████▎  | 164655/225000 [318:05:54<96:37:58,  5.76s/it] 
 73%|███████▎  | 164656/225000 [318:05:58<90:35:26,  5.40s/it]
 73%|███████▎  | 164657/225000 [318:06:03<86:25:07,  5.16s/it]
 73%|███████▎  | 164658/225000 [318:06:08<83:28:37,  4.98s/it]
 73%|███████▎  | 164659/225000 [318:06:12<79:07:53,  4.72s/it]
 73%|███████▎  | 164660/225000 [318:06:18<87:56:38,  5.25s/it]
 73%|███████▎  | 164661/225000 [318:06:29<115:26:45,  6.89s/it]
 73%|███████▎  | 164662/225000 [318:06:32<98:07:38,  5.85s/it] 
 73%|███████▎  | 164663/225000 [318:06:36<87:45:04,  5.24s/it]
 73%|███████▎  | 164664/225000 [318:06:41<86:39:29,  5.17s/it]
 73%|███████▎  | 164665/225000 [318:06:50<103:38:30,  6.18s/it]
 73%|███████▎  | 164666/225000 [318:06:55<99:01:20,  5.91s/it] 
 73%|███████▎  | 164667/225000 [318:06:59<90:34:33,  5.40s/it]
 73%|███████▎  | 164668/225000 [318:07:04<86:05:57,  5.14s/it]
 73%|███████▎  | 164669/225000 [318:07:09<84:26:37,  5.04s/it]
 73%|███████▎  | 164670/225000 [318:07:13<82:30:17,  4.92s/it]
 73%|███████▎  | 164671/225000 [318:07:18<82:16:20,  4.91s/it]
 73%|███████▎  | 164672/225000 [318:07:23<80:49:09,  4.82s/it]
 73%|███████▎  | 164673/225000 [318:07:33<109:13:55,  6.52s/it]
 73%|███████▎  | 164674/225000 [318:07:39<106:33:18,  6.36s/it]
 73%|███████▎  | 164675/225000 [318:07:42<91:25:23,  5.46s/it] 
 73%|███████▎  | 164676/225000 [318:07:51<106:43:49,  6.37s/it]
 73%|███████▎  | 164677/225000 [318:07:55<93:15:56,  5.57s/it] 
 73%|███████▎  | 164678/225000 [318:07:58<83:48:32,  5.00s/it]
 73%|███████▎  | 164679/225000 [318:08:03<79:28:25,  4.74s/it]
 73%|███████▎  | 164680/225000 [318:08:10<91:50:49,  5.48s/it]
 73%|███████▎  | 164681/225000 [318:08:18<105:35:59,  6.30s/it]
 73%|███████▎  | 164682/225000 [318:08:21<90:51:24,  5.42s/it] 
 73%|███████▎  | 164683/225000 [318:08:25<82:23:52,  4.92s/it]
 73%|███████▎  | 164684/225000 [318:08:29<76:48:48,  4.58s/it]
 73%|███████▎  | 164685/225000 [318:08:39<105:39:25,  6.31s/it]
 73%|███████▎  | 164686/225000 [318:08:43<91:55:24,  5.49s/it] 
 73%|███████▎  | 164687/225000 [318:08:47<84:17:15,  5.03s/it]
 73%|███████▎  | 164688/225000 [318:08:55<100:08:46,  5.98s/it]
 73%|███████▎  | 164689/225000 [318:09:02<104:21:35,  6.23s/it]
 73%|███████▎  | 164690/225000 [318:09:07<101:03:00,  6.03s/it]
 73%|███████▎  | 164691/225000 [318:09:13<97:08:13,  5.80s/it] 
 73%|███████▎  | 164692/225000 [318:09:19<97:56:20,  5.85s/it]
 73%|███████▎  | 164693/225000 [318:09:24<96:52:06,  5.78s/it]
 73%|███████▎  | 164694/225000 [318:09:30<95:07:12,  5.68s/it]
 73%|███████▎  | 164695/225000 [318:09:34<90:10:11,  5.38s/it]
 73%|███████▎  | 164696/225000 [318:09:40<91:22:38,  5.46s/it]
 73%|███████▎  | 164697/225000 [318:09:45<90:58:37,  5.43s/it]
 73%|███████▎  | 164698/225000 [318:09:53<101:29:45,  6.06s/it]
 73%|███████▎  | 164699/225000 [318:09:57<93:29:03,  5.58s/it] 
 73%|███████▎  | 164700/225000 [318:10:01<86:33:18,  5.17s/it]
                                                              
{'loss': '0.7728', 'grad_norm': '1.089', 'learning_rate': '9.114e-06', 'epoch': '0.732', 'num_input_tokens_seen': 526747680, 'train_runtime': '1.145e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164700/225000 [318:10:01<86:33:18,  5.17s/it]
 73%|███████▎  | 164701/225000 [318:10:06<82:33:48,  4.93s/it]
 73%|███████▎  | 164702/225000 [318:10:11<81:31:09,  4.87s/it]
 73%|███████▎  | 164703/225000 [318:10:17<91:32:41,  5.47s/it]
 73%|███████▎  | 164704/225000 [318:10:27<112:11:13,  6.70s/it]
 73%|███████▎  | 164705/225000 [318:10:31<99:35:16,  5.95s/it] 
 73%|███████▎  | 164706/225000 [318:10:36<92:31:02,  5.52s/it]
 73%|███████▎  | 164707/225000 [318:10:40<87:53:02,  5.25s/it]
 73%|███████▎  | 164708/225000 [318:10:45<86:31:04,  5.17s/it]
 73%|███████▎  | 164709/225000 [318:10:50<83:11:34,  4.97s/it]
 73%|███████▎  | 164710/225000 [318:10:55<83:08:18,  4.96s/it]
 73%|███████▎  | 164711/225000 [318:11:06<114:41:49,  6.85s/it]
 73%|███████▎  | 164712/225000 [318:11:10<101:05:28,  6.04s/it]
 73%|███████▎  | 164713/225000 [318:11:15<93:49:05,  5.60s/it] 
 73%|███████▎  | 164714/225000 [318:11:19<87:42:43,  5.24s/it]
 73%|███████▎  | 164715/225000 [318:11:23<82:27:33,  4.92s/it]
 73%|███████▎  | 164716/225000 [318:11:27<78:19:08,  4.68s/it]
 73%|███████▎  | 164717/225000 [318:11:37<104:49:53,  6.26s/it]
 73%|███████▎  | 164718/225000 [318:11:44<106:49:19,  6.38s/it]
 73%|███████▎  | 164719/225000 [318:11:47<91:52:17,  5.49s/it] 
 73%|███████▎  | 164720/225000 [318:11:51<83:22:11,  4.98s/it]
 73%|███████▎  | 164721/225000 [318:11:56<82:19:22,  4.92s/it]
 73%|███████▎  | 164722/225000 [318:12:01<83:35:01,  4.99s/it]
 73%|███████▎  | 164723/225000 [318:12:05<79:54:21,  4.77s/it]
 73%|███████▎  | 164724/225000 [318:12:11<81:38:51,  4.88s/it]
 73%|███████▎  | 164725/225000 [318:12:16<82:17:14,  4.91s/it]
 73%|███████▎  | 164726/225000 [318:12:20<82:26:14,  4.92s/it]
 73%|███████▎  | 164727/225000 [318:12:25<82:23:16,  4.92s/it]
 73%|███████▎  | 164728/225000 [318:12:30<82:32:49,  4.93s/it]
 73%|███████▎  | 164729/225000 [318:12:42<116:51:43,  6.98s/it]
 73%|███████▎  | 164730/225000 [318:12:50<121:33:51,  7.26s/it]
 73%|███████▎  | 164731/225000 [318:12:54<103:42:23,  6.19s/it]
 73%|███████▎  | 164732/225000 [318:12:58<94:07:02,  5.62s/it] 
 73%|███████▎  | 164733/225000 [318:13:06<106:41:10,  6.37s/it]
 73%|███████▎  | 164734/225000 [318:13:10<94:44:40,  5.66s/it] 
 73%|███████▎  | 164735/225000 [318:13:14<87:24:41,  5.22s/it]
 73%|███████▎  | 164736/225000 [318:13:19<82:53:15,  4.95s/it]
 73%|███████▎  | 164737/225000 [318:13:23<81:34:14,  4.87s/it]
 73%|███████▎  | 164738/225000 [318:13:29<83:49:02,  5.01s/it]
 73%|███████▎  | 164739/225000 [318:13:41<122:34:34,  7.32s/it]
 73%|███████▎  | 164740/225000 [318:13:47<115:07:42,  6.88s/it]
 73%|███████▎  | 164741/225000 [318:13:51<99:48:22,  5.96s/it] 
 73%|███████▎  | 164742/225000 [318:13:56<94:13:03,  5.63s/it]
 73%|███████▎  | 164743/225000 [318:14:01<93:22:24,  5.58s/it]
 73%|███████▎  | 164744/225000 [318:14:07<91:33:01,  5.47s/it]
 73%|███████▎  | 164745/225000 [318:14:12<88:45:24,  5.30s/it]
 73%|███████▎  | 164746/225000 [318:14:17<88:09:57,  5.27s/it]
 73%|███████▎  | 164747/225000 [318:14:22<89:08:30,  5.33s/it]
 73%|███████▎  | 164748/225000 [318:14:27<88:28:56,  5.29s/it]
 73%|███████▎  | 164749/225000 [318:14:33<88:00:18,  5.26s/it]
 73%|███████▎  | 164750/225000 [318:14:38<87:44:42,  5.24s/it]
                                                              
{'loss': '0.7718', 'grad_norm': '1.132', 'learning_rate': '9.1e-06', 'epoch': '0.7322', 'num_input_tokens_seen': 526878880, 'train_runtime': '1.146e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164750/225000 [318:14:38<87:44:42,  5.24s/it]
 73%|███████▎  | 164751/225000 [318:14:43<88:26:37,  5.28s/it]
 73%|███████▎  | 164752/225000 [318:14:48<85:17:56,  5.10s/it]
 73%|███████▎  | 164753/225000 [318:14:53<86:32:50,  5.17s/it]
 73%|███████▎  | 164754/225000 [318:14:58<87:01:06,  5.20s/it]
 73%|███████▎  | 164755/225000 [318:15:04<86:31:52,  5.17s/it]
 73%|███████▎  | 164756/225000 [318:15:09<86:21:56,  5.16s/it]
 73%|███████▎  | 164757/225000 [318:15:21<120:57:41,  7.23s/it]
 73%|███████▎  | 164758/225000 [318:15:26<111:46:13,  6.68s/it]
 73%|███████▎  | 164759/225000 [318:15:31<100:43:00,  6.02s/it]
 73%|███████▎  | 164760/225000 [318:15:38<109:02:18,  6.52s/it]
 73%|███████▎  | 164761/225000 [318:15:42<95:48:05,  5.73s/it] 
 73%|███████▎  | 164762/225000 [318:15:47<92:11:26,  5.51s/it]
 73%|███████▎  | 164763/225000 [318:15:56<107:33:56,  6.43s/it]
 73%|███████▎  | 164764/225000 [318:15:59<93:46:28,  5.60s/it] 
 73%|███████▎  | 164765/225000 [318:16:07<103:19:41,  6.18s/it]
 73%|███████▎  | 164766/225000 [318:16:10<89:53:07,  5.37s/it] 
 73%|███████▎  | 164767/225000 [318:16:16<93:18:17,  5.58s/it]
 73%|███████▎  | 164768/225000 [318:16:24<105:41:13,  6.32s/it]
 73%|███████▎  | 164769/225000 [318:16:28<90:27:40,  5.41s/it] 
 73%|███████▎  | 164770/225000 [318:16:32<84:22:48,  5.04s/it]
 73%|███████▎  | 164771/225000 [318:16:36<81:18:43,  4.86s/it]
 73%|███████▎  | 164772/225000 [318:16:40<76:39:36,  4.58s/it]
 73%|███████▎  | 164773/225000 [318:16:51<108:29:20,  6.48s/it]
 73%|███████▎  | 164774/225000 [318:16:56<98:32:50,  5.89s/it] 
 73%|███████▎  | 164775/225000 [318:17:00<92:24:08,  5.52s/it]
 73%|███████▎  | 164776/225000 [318:17:05<88:59:30,  5.32s/it]
 73%|███████▎  | 164777/225000 [318:17:10<87:30:43,  5.23s/it]
 73%|███████▎  | 164778/225000 [318:17:15<84:16:00,  5.04s/it]
 73%|███████▎  | 164779/225000 [318:17:19<81:57:39,  4.90s/it]
 73%|███████▎  | 164780/225000 [318:17:29<102:51:58,  6.15s/it]
 73%|███████▎  | 164781/225000 [318:17:36<107:19:51,  6.42s/it]
 73%|███████▎  | 164782/225000 [318:17:39<92:09:49,  5.51s/it] 
 73%|███████▎  | 164783/225000 [318:17:46<100:26:41,  6.00s/it]
 73%|███████▎  | 164784/225000 [318:17:50<92:02:26,  5.50s/it] 
 73%|███████▎  | 164785/225000 [318:17:55<86:59:15,  5.20s/it]
 73%|███████▎  | 164786/225000 [318:18:01<92:59:29,  5.56s/it]
 73%|███████▎  | 164787/225000 [318:18:10<109:43:12,  6.56s/it]
 73%|███████▎  | 164788/225000 [318:18:14<94:53:29,  5.67s/it] 
 73%|███████▎  | 164789/225000 [318:18:23<110:40:03,  6.62s/it]
 73%|███████▎  | 164790/225000 [318:18:27<101:26:24,  6.07s/it]
 73%|███████▎  | 164791/225000 [318:18:32<93:00:38,  5.56s/it] 
 73%|███████▎  | 164792/225000 [318:18:36<87:53:46,  5.26s/it]
 73%|███████▎  | 164793/225000 [318:18:41<84:22:23,  5.04s/it]
 73%|███████▎  | 164794/225000 [318:18:45<81:33:29,  4.88s/it]
 73%|███████▎  | 164795/225000 [318:18:50<81:04:06,  4.85s/it]
 73%|███████▎  | 164796/225000 [318:18:55<79:24:22,  4.75s/it]
 73%|███████▎  | 164797/225000 [318:19:00<79:48:28,  4.77s/it]
 73%|███████▎  | 164798/225000 [318:19:04<76:38:35,  4.58s/it]
 73%|███████▎  | 164799/225000 [318:19:09<78:40:26,  4.70s/it]
 73%|███████▎  | 164800/225000 [318:19:13<78:58:06,  4.72s/it]
                                                              
{'loss': '0.772', 'grad_norm': '1.089', 'learning_rate': '9.086e-06', 'epoch': '0.7324', 'num_input_tokens_seen': 527008712, 'train_runtime': '1.146e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164800/225000 [318:19:13<78:58:06,  4.72s/it]
 73%|███████▎  | 164801/225000 [318:19:19<81:21:19,  4.87s/it]
 73%|███████▎  | 164802/225000 [318:19:28<101:52:18,  6.09s/it]
 73%|███████▎  | 164803/225000 [318:19:36<114:39:29,  6.86s/it]
 73%|███████▎  | 164804/225000 [318:19:40<100:47:08,  6.03s/it]
 73%|███████▎  | 164805/225000 [318:19:44<91:29:31,  5.47s/it] 
 73%|███████▎  | 164806/225000 [318:19:51<98:43:15,  5.90s/it]
 73%|███████▎  | 164807/225000 [318:20:00<111:34:14,  6.67s/it]
 73%|███████▎  | 164808/225000 [318:20:05<105:39:42,  6.32s/it]
 73%|███████▎  | 164809/225000 [318:20:12<107:15:08,  6.41s/it]
 73%|███████▎  | 164810/225000 [318:20:16<94:30:50,  5.65s/it] 
 73%|███████▎  | 164811/225000 [318:20:23<100:44:20,  6.03s/it]
 73%|███████▎  | 164812/225000 [318:20:26<87:38:02,  5.24s/it] 
 73%|███████▎  | 164813/225000 [318:20:33<96:11:48,  5.75s/it]
 73%|███████▎  | 164814/225000 [318:20:37<85:07:50,  5.09s/it]
 73%|███████▎  | 164815/225000 [318:20:44<94:29:49,  5.65s/it]
 73%|███████▎  | 164816/225000 [318:20:51<102:45:49,  6.15s/it]
 73%|███████▎  | 164817/225000 [318:20:56<98:06:36,  5.87s/it] 
 73%|███████▎  | 164818/225000 [318:21:01<92:17:52,  5.52s/it]
 73%|███████▎  | 164819/225000 [318:21:06<90:29:17,  5.41s/it]
 73%|███████▎  | 164820/225000 [318:21:11<90:19:46,  5.40s/it]
 73%|███████▎  | 164821/225000 [318:21:17<89:01:32,  5.33s/it]
 73%|███████▎  | 164822/225000 [318:21:21<86:49:46,  5.19s/it]
 73%|███████▎  | 164823/225000 [318:21:27<87:15:08,  5.22s/it]
 73%|███████▎  | 164824/225000 [318:21:32<87:56:39,  5.26s/it]
 73%|███████▎  | 164825/225000 [318:21:37<88:15:53,  5.28s/it]
 73%|███████▎  | 164826/225000 [318:21:43<88:22:41,  5.29s/it]
 73%|███████▎  | 164827/225000 [318:21:48<87:45:56,  5.25s/it]
 73%|███████▎  | 164828/225000 [318:21:53<86:46:56,  5.19s/it]
 73%|███████▎  | 164829/225000 [318:21:58<86:46:24,  5.19s/it]
 73%|███████▎  | 164830/225000 [318:22:03<86:52:51,  5.20s/it]
 73%|███████▎  | 164831/225000 [318:22:09<87:00:19,  5.21s/it]
 73%|███████▎  | 164832/225000 [318:22:14<88:13:51,  5.28s/it]
 73%|███████▎  | 164833/225000 [318:22:25<118:03:36,  7.06s/it]
 73%|███████▎  | 164834/225000 [318:22:32<118:56:48,  7.12s/it]
 73%|███████▎  | 164835/225000 [318:22:36<103:01:37,  6.16s/it]
 73%|███████▎  | 164836/225000 [318:22:45<116:02:30,  6.94s/it]
 73%|███████▎  | 164837/225000 [318:22:52<113:12:31,  6.77s/it]
 73%|███████▎  | 164838/225000 [318:22:56<100:01:01,  5.98s/it]
 73%|███████▎  | 164839/225000 [318:23:00<89:15:25,  5.34s/it] 
 73%|███████▎  | 164840/225000 [318:23:09<112:01:42,  6.70s/it]
 73%|███████▎  | 164841/225000 [318:23:14<103:31:00,  6.19s/it]
 73%|███████▎  | 164842/225000 [318:23:18<92:24:08,  5.53s/it] 
 73%|███████▎  | 164843/225000 [318:23:22<83:06:48,  4.97s/it]
 73%|███████▎  | 164844/225000 [318:23:30<95:41:04,  5.73s/it]
 73%|███████▎  | 164845/225000 [318:23:36<98:47:28,  5.91s/it]
 73%|███████▎  | 164846/225000 [318:23:43<104:16:38,  6.24s/it]
 73%|███████▎  | 164847/225000 [318:23:47<95:45:27,  5.73s/it] 
 73%|███████▎  | 164848/225000 [318:23:52<90:37:47,  5.42s/it]
 73%|███████▎  | 164849/225000 [318:23:56<83:43:05,  5.01s/it]
 73%|███████▎  | 164850/225000 [318:24:01<81:53:31,  4.90s/it]
                                                              
{'loss': '0.7602', 'grad_norm': '1.133', 'learning_rate': '9.072e-06', 'epoch': '0.7327', 'num_input_tokens_seen': 527141056, 'train_runtime': '1.146e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164850/225000 [318:24:01<81:53:31,  4.90s/it]
 73%|███████▎  | 164851/225000 [318:24:08<94:26:32,  5.65s/it]
 73%|███████▎  | 164852/225000 [318:24:15<99:41:39,  5.97s/it]
 73%|███████▎  | 164853/225000 [318:24:23<112:06:47,  6.71s/it]
 73%|███████▎  | 164854/225000 [318:24:27<95:24:21,  5.71s/it] 
 73%|███████▎  | 164855/225000 [318:24:31<86:02:16,  5.15s/it]
 73%|███████▎  | 164856/225000 [318:24:35<79:52:30,  4.78s/it]
 73%|███████▎  | 164857/225000 [318:24:39<76:53:28,  4.60s/it]
 73%|███████▎  | 164858/225000 [318:24:49<105:44:18,  6.33s/it]
 73%|███████▎  | 164859/225000 [318:24:52<91:00:49,  5.45s/it] 
 73%|███████▎  | 164860/225000 [318:24:56<80:42:18,  4.83s/it]
 73%|███████▎  | 164861/225000 [318:25:00<77:51:23,  4.66s/it]
 73%|███████▎  | 164862/225000 [318:25:10<106:01:46,  6.35s/it]
 73%|███████▎  | 164863/225000 [318:25:15<95:51:59,  5.74s/it] 
 73%|███████▎  | 164864/225000 [318:25:22<105:20:35,  6.31s/it]
 73%|███████▎  | 164865/225000 [318:25:27<95:31:19,  5.72s/it] 
 73%|███████▎  | 164866/225000 [318:25:31<89:29:57,  5.36s/it]
 73%|███████▎  | 164867/225000 [318:25:35<83:31:14,  5.00s/it]
 73%|███████▎  | 164868/225000 [318:25:40<80:17:06,  4.81s/it]
 73%|███████▎  | 164869/225000 [318:25:49<100:52:12,  6.04s/it]
 73%|███████▎  | 164870/225000 [318:25:55<102:26:44,  6.13s/it]
 73%|███████▎  | 164871/225000 [318:25:59<91:17:49,  5.47s/it] 
 73%|███████▎  | 164872/225000 [318:26:07<106:02:16,  6.35s/it]
 73%|███████▎  | 164873/225000 [318:26:11<90:59:41,  5.45s/it] 
 73%|███████▎  | 164874/225000 [318:26:15<86:14:57,  5.16s/it]
 73%|███████▎  | 164875/225000 [318:26:21<89:08:43,  5.34s/it]
 73%|███████▎  | 164876/225000 [318:26:29<103:57:30,  6.22s/it]
 73%|███████▎  | 164877/225000 [318:26:34<94:46:05,  5.67s/it] 
 73%|███████▎  | 164878/225000 [318:26:38<86:42:01,  5.19s/it]
 73%|███████▎  | 164879/225000 [318:26:43<87:01:50,  5.21s/it]
 73%|███████▎  | 164880/225000 [318:26:52<106:30:53,  6.38s/it]
 73%|███████▎  | 164881/225000 [318:26:56<92:04:01,  5.51s/it] 
 73%|███████▎  | 164882/225000 [318:27:03<99:25:50,  5.95s/it]
 73%|███████▎  | 164883/225000 [318:27:06<89:21:13,  5.35s/it]
 73%|███████▎  | 164884/225000 [318:27:11<83:09:48,  4.98s/it]
 73%|███████▎  | 164885/225000 [318:27:16<86:36:56,  5.19s/it]
 73%|███████▎  | 164886/225000 [318:27:27<112:11:33,  6.72s/it]
 73%|███████▎  | 164887/225000 [318:27:30<98:05:31,  5.87s/it] 
 73%|███████▎  | 164888/225000 [318:27:34<86:02:44,  5.15s/it]
 73%|███████▎  | 164889/225000 [318:27:39<83:24:20,  5.00s/it]
 73%|███████▎  | 164890/225000 [318:27:44<86:33:44,  5.18s/it]
 73%|███████▎  | 164891/225000 [318:27:52<99:39:00,  5.97s/it]
 73%|███████▎  | 164892/225000 [318:27:59<105:02:27,  6.29s/it]
 73%|███████▎  | 164893/225000 [318:28:02<90:13:55,  5.40s/it] 
 73%|███████▎  | 164894/225000 [318:28:09<96:06:24,  5.76s/it]
 73%|███████▎  | 164895/225000 [318:28:13<85:51:17,  5.14s/it]
 73%|███████▎  | 164896/225000 [318:28:22<105:54:01,  6.34s/it]
 73%|███████▎  | 164897/225000 [318:28:25<91:26:59,  5.48s/it] 
 73%|███████▎  | 164898/225000 [318:28:32<96:04:03,  5.75s/it]
 73%|███████▎  | 164899/225000 [318:28:37<91:55:57,  5.51s/it]
 73%|███████▎  | 164900/225000 [318:28:41<88:43:25,  5.31s/it]
                                                              
{'loss': '0.7728', 'grad_norm': '1.198', 'learning_rate': '9.058e-06', 'epoch': '0.7329', 'num_input_tokens_seen': 527271352, 'train_runtime': '1.147e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164900/225000 [318:28:41<88:43:25,  5.31s/it]
 73%|███████▎  | 164901/225000 [318:28:45<79:12:54,  4.75s/it]
 73%|███████▎  | 164902/225000 [318:28:54<102:35:16,  6.15s/it]
 73%|███████▎  | 164903/225000 [318:28:58<89:12:15,  5.34s/it] 
 73%|███████▎  | 164904/225000 [318:29:05<98:36:07,  5.91s/it]
 73%|███████▎  | 164905/225000 [318:29:09<87:32:33,  5.24s/it]
 73%|███████▎  | 164906/225000 [318:29:14<88:46:50,  5.32s/it]
 73%|███████▎  | 164907/225000 [318:29:20<92:44:38,  5.56s/it]
 73%|███████▎  | 164908/225000 [318:29:25<89:44:53,  5.38s/it]
 73%|███████▎  | 164909/225000 [318:29:32<98:41:13,  5.91s/it]
 73%|███████▎  | 164910/225000 [318:29:36<84:52:51,  5.09s/it]
 73%|███████▎  | 164911/225000 [318:29:43<95:38:40,  5.73s/it]
 73%|███████▎  | 164912/225000 [318:29:47<89:27:14,  5.36s/it]
 73%|███████▎  | 164913/225000 [318:29:52<85:41:05,  5.13s/it]
 73%|███████▎  | 164914/225000 [318:29:57<83:41:21,  5.01s/it]
 73%|███████▎  | 164915/225000 [318:30:02<84:02:11,  5.04s/it]
 73%|███████▎  | 164916/225000 [318:30:07<83:31:33,  5.00s/it]
 73%|███████▎  | 164917/225000 [318:30:12<83:09:17,  4.98s/it]
 73%|███████▎  | 164918/225000 [318:30:16<82:00:13,  4.91s/it]
 73%|███████▎  | 164919/225000 [318:30:21<80:51:42,  4.85s/it]
 73%|███████▎  | 164920/225000 [318:30:32<110:46:41,  6.64s/it]
 73%|███████▎  | 164921/225000 [318:30:37<105:11:54,  6.30s/it]
 73%|███████▎  | 164922/225000 [318:30:42<96:36:46,  5.79s/it] 
 73%|███████▎  | 164923/225000 [318:30:47<91:21:56,  5.47s/it]
 73%|███████▎  | 164924/225000 [318:30:51<86:19:55,  5.17s/it]
 73%|███████▎  | 164925/225000 [318:30:56<83:42:57,  5.02s/it]
 73%|███████▎  | 164926/225000 [318:31:00<81:31:27,  4.89s/it]
 73%|███████▎  | 164927/225000 [318:31:05<80:09:13,  4.80s/it]
 73%|███████▎  | 164928/225000 [318:31:12<91:32:54,  5.49s/it]
 73%|███████▎  | 164929/225000 [318:31:23<116:29:59,  6.98s/it]
 73%|███████▎  | 164930/225000 [318:31:26<100:39:43,  6.03s/it]
 73%|███████▎  | 164931/225000 [318:31:35<113:06:19,  6.78s/it]
 73%|███████▎  | 164932/225000 [318:31:38<96:34:57,  5.79s/it] 
 73%|███████▎  | 164933/225000 [318:31:44<96:50:13,  5.80s/it]
 73%|███████▎  | 164934/225000 [318:31:49<93:36:32,  5.61s/it]
 73%|███████▎  | 164935/225000 [318:31:56<99:14:02,  5.95s/it]
 73%|███████▎  | 164936/225000 [318:32:01<91:38:27,  5.49s/it]
 73%|███████▎  | 164937/225000 [318:32:05<84:21:16,  5.06s/it]
 73%|███████▎  | 164938/225000 [318:32:14<104:51:16,  6.28s/it]
 73%|███████▎  | 164939/225000 [318:32:17<91:18:28,  5.47s/it] 
 73%|███████▎  | 164940/225000 [318:32:24<97:11:49,  5.83s/it]
 73%|███████▎  | 164941/225000 [318:32:29<93:18:48,  5.59s/it]
 73%|███████▎  | 164942/225000 [318:32:32<81:36:43,  4.89s/it]
 73%|███████▎  | 164943/225000 [318:32:41<98:37:52,  5.91s/it]
 73%|███████▎  | 164944/225000 [318:32:44<85:21:57,  5.12s/it]
 73%|███████▎  | 164945/225000 [318:32:52<99:54:25,  5.99s/it]
 73%|███████▎  | 164946/225000 [318:32:56<90:17:47,  5.41s/it]
 73%|███████▎  | 164947/225000 [318:32:59<79:52:58,  4.79s/it]
 73%|███████▎  | 164948/225000 [318:33:08<101:32:49,  6.09s/it]
 73%|███████▎  | 164949/225000 [318:33:12<88:48:18,  5.32s/it] 
 73%|███████▎  | 164950/225000 [318:33:16<80:26:56,  4.82s/it]
                                                              
{'loss': '0.7558', 'grad_norm': '1.09', 'learning_rate': '9.044e-06', 'epoch': '0.7331', 'num_input_tokens_seen': 527403016, 'train_runtime': '1.147e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 164950/225000 [318:33:16<80:26:56,  4.82s/it]
 73%|███████▎  | 164951/225000 [318:33:23<92:43:05,  5.56s/it]
 73%|███████▎  | 164952/225000 [318:33:28<90:36:19,  5.43s/it]
 73%|███████▎  | 164953/225000 [318:33:33<89:41:58,  5.38s/it]
 73%|███████▎  | 164954/225000 [318:33:38<89:01:31,  5.34s/it]
 73%|███████▎  | 164955/225000 [318:33:44<89:35:20,  5.37s/it]
 73%|███████▎  | 164956/225000 [318:33:49<90:21:11,  5.42s/it]
 73%|███████▎  | 164957/225000 [318:33:54<87:40:01,  5.26s/it]
 73%|███████▎  | 164958/225000 [318:34:00<89:05:56,  5.34s/it]
 73%|███████▎  | 164959/225000 [318:34:05<88:58:06,  5.33s/it]
 73%|███████▎  | 164960/225000 [318:34:10<88:23:32,  5.30s/it]
 73%|███████▎  | 164961/225000 [318:34:16<89:48:14,  5.38s/it]
 73%|███████▎  | 164962/225000 [318:34:21<89:36:46,  5.37s/it]
 73%|███████▎  | 164963/225000 [318:34:27<90:46:03,  5.44s/it]
 73%|███████▎  | 164964/225000 [318:34:32<90:47:08,  5.44s/it]
 73%|███████▎  | 164965/225000 [318:34:38<91:47:17,  5.50s/it]
 73%|███████▎  | 164966/225000 [318:34:43<89:01:33,  5.34s/it]
 73%|███████▎  | 164967/225000 [318:34:48<89:21:52,  5.36s/it]
 73%|███████▎  | 164968/225000 [318:34:54<91:04:01,  5.46s/it]
 73%|███████▎  | 164969/225000 [318:34:59<88:46:28,  5.32s/it]
 73%|███████▎  | 164970/225000 [318:35:04<88:27:10,  5.30s/it]
 73%|███████▎  | 164971/225000 [318:35:13<107:25:41,  6.44s/it]
 73%|███████▎  | 164972/225000 [318:35:19<101:38:40,  6.10s/it]
 73%|███████▎  | 164973/225000 [318:35:24<96:26:31,  5.78s/it] 
 73%|███████▎  | 164974/225000 [318:35:29<94:10:49,  5.65s/it]
 73%|███████▎  | 164975/225000 [318:35:34<92:49:22,  5.57s/it]
 73%|███████▎  | 164976/225000 [318:35:40<91:14:04,  5.47s/it]
 73%|███████▎  | 164977/225000 [318:35:45<90:37:24,  5.44s/it]
 73%|███████▎  | 164978/225000 [318:35:50<87:31:28,  5.25s/it]
 73%|███████▎  | 164979/225000 [318:35:55<87:26:50,  5.25s/it]
 73%|███████▎  | 164980/225000 [318:36:01<89:45:11,  5.38s/it]
 73%|███████▎  | 164981/225000 [318:36:07<91:35:12,  5.49s/it]
 73%|███████▎  | 164982/225000 [318:36:12<90:42:38,  5.44s/it]
 73%|███████▎  | 164983/225000 [318:36:20<104:34:43,  6.27s/it]
 73%|███████▎  | 164984/225000 [318:36:26<100:41:28,  6.04s/it]
 73%|███████▎  | 164985/225000 [318:36:31<95:03:30,  5.70s/it] 
 73%|███████▎  | 164986/225000 [318:36:36<92:03:47,  5.52s/it]
 73%|███████▎  | 164987/225000 [318:36:41<90:26:56,  5.43s/it]
 73%|███████▎  | 164988/225000 [318:36:46<87:31:49,  5.25s/it]
 73%|███████▎  | 164989/225000 [318:36:52<94:15:30,  5.65s/it]
 73%|███████▎  | 164990/225000 [318:36:59<100:59:18,  6.06s/it]
 73%|███████▎  | 164991/225000 [318:37:04<92:05:23,  5.52s/it] 
 73%|███████▎  | 164992/225000 [318:37:08<86:27:32,  5.19s/it]
 73%|███████▎  | 164993/225000 [318:37:15<96:37:19,  5.80s/it]
 73%|███████▎  | 164994/225000 [318:37:24<113:17:44,  6.80s/it]
 73%|███████▎  | 164995/225000 [318:37:28<99:11:07,  5.95s/it] 
 73%|███████▎  | 164996/225000 [318:37:32<89:03:50,  5.34s/it]
 73%|███████▎  | 164997/225000 [318:37:41<108:45:16,  6.52s/it]
 73%|███████▎  | 164998/225000 [318:37:45<94:48:22,  5.69s/it] 
 73%|███████▎  | 164999/225000 [318:37:49<85:25:19,  5.13s/it]
 73%|███████▎  | 165000/225000 [318:38:00<115:34:59,  6.93s/it]
                                                               
{'loss': '0.772', 'grad_norm': '1.122', 'learning_rate': '9.03e-06', 'epoch': '0.7333', 'num_input_tokens_seen': 527532384, 'train_runtime': '1.147e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165000/225000 [318:38:00<115:34:59,  6.93s/it]
 73%|███████▎  | 165001/225000 [318:38:03<96:10:30,  5.77s/it] 
 73%|███████▎  | 165002/225000 [318:38:10<99:19:25,  5.96s/it]
 73%|███████▎  | 165003/225000 [318:38:13<88:09:55,  5.29s/it]
 73%|███████▎  | 165004/225000 [318:38:18<83:48:56,  5.03s/it]
 73%|███████▎  | 165005/225000 [318:38:22<81:08:15,  4.87s/it]
 73%|███████▎  | 165006/225000 [318:38:34<114:25:35,  6.87s/it]
 73%|███████▎  | 165007/225000 [318:38:37<96:40:28,  5.80s/it] 
 73%|███████▎  | 165008/225000 [318:38:41<86:32:22,  5.19s/it]
 73%|███████▎  | 165009/225000 [318:38:45<78:40:36,  4.72s/it]
 73%|███████▎  | 165010/225000 [318:38:54<103:41:31,  6.22s/it]
 73%|███████▎  | 165011/225000 [318:38:58<91:02:22,  5.46s/it] 
 73%|███████▎  | 165012/225000 [318:39:06<102:51:27,  6.17s/it]
 73%|███████▎  | 165013/225000 [318:39:10<91:46:42,  5.51s/it] 
 73%|███████▎  | 165014/225000 [318:39:14<85:17:41,  5.12s/it]
 73%|███████▎  | 165015/225000 [318:39:18<78:58:56,  4.74s/it]
 73%|███████▎  | 165016/225000 [318:39:22<76:52:46,  4.61s/it]
 73%|███████▎  | 165017/225000 [318:39:32<103:11:46,  6.19s/it]
 73%|███████▎  | 165018/225000 [318:39:37<95:00:45,  5.70s/it] 
 73%|███████▎  | 165019/225000 [318:39:44<106:01:21,  6.36s/it]
 73%|███████▎  | 165020/225000 [318:39:48<89:51:58,  5.39s/it] 
 73%|███████▎  | 165021/225000 [318:39:54<96:29:48,  5.79s/it]
 73%|███████▎  | 165022/225000 [318:39:59<90:19:35,  5.42s/it]
 73%|███████▎  | 165023/225000 [318:40:06<97:22:18,  5.84s/it]
 73%|███████▎  | 165024/225000 [318:40:09<84:52:46,  5.09s/it]
 73%|███████▎  | 165025/225000 [318:40:16<95:36:00,  5.74s/it]
 73%|███████▎  | 165026/225000 [318:40:20<86:16:26,  5.18s/it]
 73%|███████▎  | 165027/225000 [318:40:25<85:28:56,  5.13s/it]
 73%|███████▎  | 165028/225000 [318:40:30<85:58:56,  5.16s/it]
 73%|███████▎  | 165029/225000 [318:40:35<84:13:05,  5.06s/it]
 73%|███████▎  | 165030/225000 [318:40:40<84:55:53,  5.10s/it]
 73%|███████▎  | 165031/225000 [318:40:45<84:45:15,  5.09s/it]
 73%|███████▎  | 165032/225000 [318:40:51<87:04:44,  5.23s/it]
 73%|███████▎  | 165033/225000 [318:40:56<86:04:46,  5.17s/it]
 73%|███████▎  | 165034/225000 [318:41:01<87:03:14,  5.23s/it]
 73%|███████▎  | 165035/225000 [318:41:07<86:44:49,  5.21s/it]
 73%|███████▎  | 165036/225000 [318:41:12<86:02:42,  5.17s/it]
 73%|███████▎  | 165037/225000 [318:41:17<85:40:30,  5.14s/it]
 73%|███████▎  | 165038/225000 [318:41:22<85:42:12,  5.15s/it]
 73%|███████▎  | 165039/225000 [318:41:30<98:53:05,  5.94s/it]
 73%|███████▎  | 165040/225000 [318:41:39<117:13:07,  7.04s/it]
 73%|███████▎  | 165041/225000 [318:41:44<103:40:18,  6.22s/it]
 73%|███████▎  | 165042/225000 [318:41:48<93:44:45,  5.63s/it] 
 73%|███████▎  | 165043/225000 [318:41:52<87:38:20,  5.26s/it]
 73%|███████▎  | 165044/225000 [318:42:01<107:09:02,  6.43s/it]
 73%|███████▎  | 165045/225000 [318:42:05<94:38:07,  5.68s/it] 
 73%|███████▎  | 165046/225000 [318:42:09<84:48:28,  5.09s/it]
 73%|███████▎  | 165047/225000 [318:42:18<104:55:48,  6.30s/it]
 73%|███████▎  | 165048/225000 [318:42:22<90:13:03,  5.42s/it] 
 73%|███████▎  | 165049/225000 [318:42:27<90:12:45,  5.42s/it]
 73%|███████▎  | 165050/225000 [318:42:33<91:35:20,  5.50s/it]
                                                              
{'loss': '0.7604', 'grad_norm': '1.108', 'learning_rate': '9.016e-06', 'epoch': '0.7336', 'num_input_tokens_seen': 527662200, 'train_runtime': '1.147e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165050/225000 [318:42:33<91:35:20,  5.50s/it]
 73%|███████▎  | 165051/225000 [318:42:38<92:00:40,  5.53s/it]
 73%|███████▎  | 165052/225000 [318:42:44<91:51:07,  5.52s/it]
 73%|███████▎  | 165053/225000 [318:42:49<89:33:25,  5.38s/it]
 73%|███████▎  | 165054/225000 [318:42:54<88:16:20,  5.30s/it]
 73%|███████▎  | 165055/225000 [318:43:00<90:07:47,  5.41s/it]
 73%|███████▎  | 165056/225000 [318:43:04<86:45:22,  5.21s/it]
 73%|███████▎  | 165057/225000 [318:43:10<88:32:33,  5.32s/it]
 73%|███████▎  | 165058/225000 [318:43:15<88:26:09,  5.31s/it]
 73%|███████▎  | 165059/225000 [318:43:21<88:51:08,  5.34s/it]
 73%|███████▎  | 165060/225000 [318:43:26<88:05:27,  5.29s/it]
 73%|███████▎  | 165061/225000 [318:43:31<85:49:00,  5.15s/it]
 73%|███████▎  | 165062/225000 [318:43:36<87:49:19,  5.27s/it]
 73%|███████▎  | 165063/225000 [318:43:42<89:36:00,  5.38s/it]
 73%|███████▎  | 165064/225000 [318:43:51<107:52:01,  6.48s/it]
 73%|███████▎  | 165065/225000 [318:43:58<112:32:44,  6.76s/it]
 73%|███████▎  | 165066/225000 [318:44:05<112:59:43,  6.79s/it]
 73%|███████▎  | 165067/225000 [318:44:10<101:54:39,  6.12s/it]
 73%|███████▎  | 165068/225000 [318:44:15<97:57:39,  5.88s/it] 
 73%|███████▎  | 165069/225000 [318:44:22<102:31:40,  6.16s/it]
 73%|███████▎  | 165070/225000 [318:44:26<94:12:57,  5.66s/it] 
 73%|███████▎  | 165071/225000 [318:44:30<86:16:38,  5.18s/it]
 73%|███████▎  | 165072/225000 [318:44:35<83:57:45,  5.04s/it]
 73%|███████▎  | 165073/225000 [318:44:40<82:29:15,  4.96s/it]
 73%|███████▎  | 165074/225000 [318:44:44<80:58:02,  4.86s/it]
 73%|███████▎  | 165075/225000 [318:44:56<115:54:24,  6.96s/it]
 73%|███████▎  | 165076/225000 [318:45:01<102:18:19,  6.15s/it]
 73%|███████▎  | 165077/225000 [318:45:06<98:17:24,  5.90s/it] 
 73%|███████▎  | 165078/225000 [318:45:09<85:34:03,  5.14s/it]
 73%|███████▎  | 165079/225000 [318:45:15<86:12:54,  5.18s/it]
 73%|███████▎  | 165080/225000 [318:45:20<86:55:29,  5.22s/it]
 73%|███████▎  | 165081/225000 [318:45:25<84:30:42,  5.08s/it]
 73%|███████▎  | 165082/225000 [318:45:29<83:07:58,  4.99s/it]
 73%|███████▎  | 165083/225000 [318:45:35<84:02:03,  5.05s/it]
 73%|███████▎  | 165084/225000 [318:45:40<83:43:47,  5.03s/it]
 73%|███████▎  | 165085/225000 [318:45:45<84:12:00,  5.06s/it]
 73%|███████▎  | 165086/225000 [318:45:50<84:55:47,  5.10s/it]
 73%|███████▎  | 165087/225000 [318:45:55<84:09:45,  5.06s/it]
 73%|███████▎  | 165088/225000 [318:46:00<85:29:42,  5.14s/it]
 73%|███████▎  | 165089/225000 [318:46:05<86:11:15,  5.18s/it]
 73%|███████▎  | 165090/225000 [318:46:11<87:55:59,  5.28s/it]
 73%|███████▎  | 165091/225000 [318:46:16<86:59:57,  5.23s/it]
 73%|███████▎  | 165092/225000 [318:46:21<87:19:55,  5.25s/it]
 73%|███████▎  | 165093/225000 [318:46:27<87:35:55,  5.26s/it]
 73%|███████▎  | 165094/225000 [318:46:39<120:52:26,  7.26s/it]
 73%|███████▎  | 165095/225000 [318:46:44<113:16:13,  6.81s/it]
 73%|███████▎  | 165096/225000 [318:46:51<111:01:21,  6.67s/it]
 73%|███████▎  | 165097/225000 [318:46:55<98:01:36,  5.89s/it] 
 73%|███████▎  | 165098/225000 [318:46:59<90:13:04,  5.42s/it]
 73%|███████▎  | 165099/225000 [318:47:03<84:40:34,  5.09s/it]
 73%|███████▎  | 165100/225000 [318:47:12<100:58:12,  6.07s/it]
                                                               
{'loss': '0.7702', 'grad_norm': '1.241', 'learning_rate': '9.002e-06', 'epoch': '0.7338', 'num_input_tokens_seen': 527792200, 'train_runtime': '1.148e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165100/225000 [318:47:12<100:58:12,  6.07s/it]
 73%|███████▎  | 165101/225000 [318:47:18<101:28:25,  6.10s/it]
 73%|███████▎  | 165102/225000 [318:47:21<87:51:51,  5.28s/it] 
 73%|███████▎  | 165103/225000 [318:47:28<93:16:40,  5.61s/it]
 73%|███████▎  | 165104/225000 [318:47:36<105:48:49,  6.36s/it]
 73%|███████▎  | 165105/225000 [318:47:41<100:07:07,  6.02s/it]
 73%|███████▎  | 165106/225000 [318:47:46<93:04:54,  5.59s/it] 
 73%|███████▎  | 165107/225000 [318:47:51<92:24:02,  5.55s/it]
 73%|███████▎  | 165108/225000 [318:47:56<89:37:03,  5.39s/it]
 73%|███████▎  | 165109/225000 [318:48:01<89:19:04,  5.37s/it]
 73%|███████▎  | 165110/225000 [318:48:07<91:37:20,  5.51s/it]
 73%|███████▎  | 165111/225000 [318:48:12<89:17:38,  5.37s/it]
 73%|███████▎  | 165112/225000 [318:48:18<88:41:38,  5.33s/it]
 73%|███████▎  | 165113/225000 [318:48:24<95:48:29,  5.76s/it]
 73%|███████▎  | 165114/225000 [318:48:32<106:45:37,  6.42s/it]
 73%|███████▎  | 165115/225000 [318:48:36<91:46:57,  5.52s/it] 
 73%|███████▎  | 165116/225000 [318:48:40<85:46:21,  5.16s/it]
 73%|███████▎  | 165117/225000 [318:48:45<85:38:49,  5.15s/it]
 73%|███████▎  | 165118/225000 [318:48:50<84:38:05,  5.09s/it]
 73%|███████▎  | 165119/225000 [318:48:55<83:02:48,  4.99s/it]
 73%|███████▎  | 165120/225000 [318:49:00<82:32:31,  4.96s/it]
 73%|███████▎  | 165121/225000 [318:49:10<107:42:37,  6.48s/it]
 73%|███████▎  | 165122/225000 [318:49:17<111:17:32,  6.69s/it]
 73%|███████▎  | 165123/225000 [318:49:22<102:42:14,  6.17s/it]
 73%|███████▎  | 165124/225000 [318:49:26<94:40:02,  5.69s/it] 
 73%|███████▎  | 165125/225000 [318:49:31<88:40:42,  5.33s/it]
 73%|███████▎  | 165126/225000 [318:49:36<85:50:50,  5.16s/it]
 73%|███████▎  | 165127/225000 [318:49:40<82:22:46,  4.95s/it]
 73%|███████▎  | 165128/225000 [318:49:45<81:05:30,  4.88s/it]
 73%|███████▎  | 165129/225000 [318:49:50<81:22:04,  4.89s/it]
 73%|███████▎  | 165130/225000 [318:49:55<80:55:47,  4.87s/it]
 73%|███████▎  | 165131/225000 [318:50:04<105:32:35,  6.35s/it]
 73%|███████▎  | 165132/225000 [318:50:12<109:30:12,  6.58s/it]
 73%|███████▎  | 165133/225000 [318:50:15<94:25:34,  5.68s/it] 
 73%|███████▎  | 165134/225000 [318:50:23<105:08:36,  6.32s/it]
 73%|███████▎  | 165135/225000 [318:50:27<95:18:45,  5.73s/it] 
 73%|███████▎  | 165136/225000 [318:50:34<100:03:56,  6.02s/it]
 73%|███████▎  | 165137/225000 [318:50:39<96:12:22,  5.79s/it] 
 73%|███████▎  | 165138/225000 [318:50:47<106:38:42,  6.41s/it]
 73%|███████▎  | 165139/225000 [318:50:52<98:04:00,  5.90s/it] 
 73%|███████▎  | 165140/225000 [318:50:57<92:39:43,  5.57s/it]
 73%|███████▎  | 165141/225000 [318:51:02<89:47:33,  5.40s/it]
 73%|███████▎  | 165142/225000 [318:51:06<86:12:56,  5.19s/it]
 73%|███████▎  | 165143/225000 [318:51:11<83:42:03,  5.03s/it]
 73%|███████▎  | 165144/225000 [318:51:16<84:24:56,  5.08s/it]
 73%|███████▎  | 165145/225000 [318:51:21<83:20:41,  5.01s/it]
 73%|███████▎  | 165146/225000 [318:51:26<83:14:59,  5.01s/it]
 73%|███████▎  | 165147/225000 [318:51:31<82:06:59,  4.94s/it]
 73%|███████▎  | 165148/225000 [318:51:36<81:32:09,  4.90s/it]
 73%|███████▎  | 165149/225000 [318:51:41<82:01:39,  4.93s/it]
 73%|███████▎  | 165150/225000 [318:51:45<79:48:13,  4.80s/it]
                                                              
{'loss': '0.7681', 'grad_norm': '1.138', 'learning_rate': '8.988e-06', 'epoch': '0.734', 'num_input_tokens_seen': 527921456, 'train_runtime': '1.148e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165150/225000 [318:51:45<79:48:13,  4.80s/it]
 73%|███████▎  | 165151/225000 [318:51:54<99:04:46,  5.96s/it]
 73%|███████▎  | 165152/225000 [318:52:05<127:37:23,  7.68s/it]
 73%|███████▎  | 165153/225000 [318:52:09<105:28:19,  6.34s/it]
 73%|███████▎  | 165154/225000 [318:52:16<112:15:58,  6.75s/it]
 73%|███████▎  | 165155/225000 [318:52:20<98:07:58,  5.90s/it] 
 73%|███████▎  | 165156/225000 [318:52:25<89:58:21,  5.41s/it]
 73%|███████▎  | 165157/225000 [318:52:29<83:43:23,  5.04s/it]
 73%|███████▎  | 165158/225000 [318:52:33<79:59:14,  4.81s/it]
 73%|███████▎  | 165159/225000 [318:52:43<107:31:50,  6.47s/it]
 73%|███████▎  | 165160/225000 [318:52:47<91:23:38,  5.50s/it] 
 73%|███████▎  | 165161/225000 [318:52:50<81:28:12,  4.90s/it]
 73%|███████▎  | 165162/225000 [318:52:58<98:45:47,  5.94s/it]
 73%|███████▎  | 165163/225000 [318:53:03<90:55:11,  5.47s/it]
 73%|███████▎  | 165164/225000 [318:53:07<84:48:45,  5.10s/it]
 73%|███████▎  | 165165/225000 [318:53:12<82:26:56,  4.96s/it]
 73%|███████▎  | 165166/225000 [318:53:16<79:37:16,  4.79s/it]
 73%|███████▎  | 165167/225000 [318:53:21<78:17:06,  4.71s/it]
 73%|███████▎  | 165168/225000 [318:53:25<76:51:36,  4.62s/it]
 73%|███████▎  | 165169/225000 [318:53:38<117:07:27,  7.05s/it]
 73%|███████▎  | 165170/225000 [318:53:43<106:12:01,  6.39s/it]
 73%|███████▎  | 165171/225000 [318:53:47<97:43:22,  5.88s/it] 
 73%|███████▎  | 165172/225000 [318:53:52<93:21:36,  5.62s/it]
 73%|███████▎  | 165173/225000 [318:53:57<89:46:31,  5.40s/it]
 73%|███████▎  | 165174/225000 [318:54:02<86:29:39,  5.20s/it]
 73%|███████▎  | 165175/225000 [318:54:07<84:40:49,  5.10s/it]
 73%|███████▎  | 165176/225000 [318:54:12<83:36:17,  5.03s/it]
 73%|███████▎  | 165177/225000 [318:54:17<84:14:20,  5.07s/it]
 73%|███████▎  | 165178/225000 [318:54:22<83:23:42,  5.02s/it]
 73%|███████▎  | 165179/225000 [318:54:26<80:47:45,  4.86s/it]
 73%|███████▎  | 165180/225000 [318:54:34<94:08:08,  5.67s/it]
 73%|███████▎  | 165181/225000 [318:54:44<118:25:36,  7.13s/it]
 73%|███████▎  | 165182/225000 [318:54:48<101:52:24,  6.13s/it]
 73%|███████▎  | 165183/225000 [318:54:53<98:08:37,  5.91s/it] 
 73%|███████▎  | 165184/225000 [318:55:01<107:40:06,  6.48s/it]
 73%|███████▎  | 165185/225000 [318:55:07<103:32:05,  6.23s/it]
 73%|███████▎  | 165186/225000 [318:55:11<93:24:18,  5.62s/it] 
 73%|███████▎  | 165187/225000 [318:55:15<84:28:13,  5.08s/it]
 73%|███████▎  | 165188/225000 [318:55:24<103:26:18,  6.23s/it]
 73%|███████▎  | 165189/225000 [318:55:27<90:07:30,  5.42s/it] 
 73%|███████▎  | 165190/225000 [318:55:35<100:11:47,  6.03s/it]
 73%|███████▎  | 165191/225000 [318:55:41<98:53:09,  5.95s/it] 
 73%|███████▎  | 165192/225000 [318:55:45<90:42:06,  5.46s/it]
 73%|███████▎  | 165193/225000 [318:55:50<88:56:54,  5.35s/it]
 73%|███████▎  | 165194/225000 [318:55:55<89:22:24,  5.38s/it]
 73%|███████▎  | 165195/225000 [318:56:01<89:24:37,  5.38s/it]
 73%|███████▎  | 165196/225000 [318:56:06<86:07:33,  5.18s/it]
 73%|███████▎  | 165197/225000 [318:56:11<87:02:53,  5.24s/it]
 73%|███████▎  | 165198/225000 [318:56:16<87:59:04,  5.30s/it]
 73%|███████▎  | 165199/225000 [318:56:22<90:35:27,  5.45s/it]
 73%|███████▎  | 165200/225000 [318:56:27<87:54:47,  5.29s/it]
                                                              
{'loss': '0.768', 'grad_norm': '1.099', 'learning_rate': '8.974e-06', 'epoch': '0.7342', 'num_input_tokens_seen': 528051368, 'train_runtime': '1.148e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165200/225000 [318:56:27<87:54:47,  5.29s/it]
 73%|███████▎  | 165201/225000 [318:56:33<88:45:12,  5.34s/it]
 73%|███████▎  | 165202/225000 [318:56:38<89:49:57,  5.41s/it]
 73%|███████▎  | 165203/225000 [318:56:43<89:17:13,  5.38s/it]
 73%|███████▎  | 165204/225000 [318:56:49<90:32:15,  5.45s/it]
 73%|███████▎  | 165205/225000 [318:56:55<90:46:50,  5.47s/it]
 73%|███████▎  | 165206/225000 [318:57:00<91:27:45,  5.51s/it]
 73%|███████▎  | 165207/225000 [318:57:05<90:07:37,  5.43s/it]
 73%|███████▎  | 165208/225000 [318:57:11<89:31:51,  5.39s/it]
 73%|███████▎  | 165209/225000 [318:57:16<91:22:50,  5.50s/it]
 73%|███████▎  | 165210/225000 [318:57:22<90:04:06,  5.42s/it]
 73%|███████▎  | 165211/225000 [318:57:27<90:50:07,  5.47s/it]
 73%|███████▎  | 165212/225000 [318:57:33<92:16:38,  5.56s/it]
 73%|███████▎  | 165213/225000 [318:57:38<90:03:23,  5.42s/it]
 73%|███████▎  | 165214/225000 [318:57:43<89:15:42,  5.37s/it]
 73%|███████▎  | 165215/225000 [318:57:49<89:26:28,  5.39s/it]
 73%|███████▎  | 165216/225000 [318:57:54<89:26:38,  5.39s/it]
 73%|███████▎  | 165217/225000 [318:57:59<88:42:19,  5.34s/it]
 73%|███████▎  | 165218/225000 [318:58:11<117:38:36,  7.08s/it]
 73%|███████▎  | 165219/225000 [318:58:18<118:51:35,  7.16s/it]
 73%|███████▎  | 165220/225000 [318:58:23<108:41:36,  6.55s/it]
 73%|███████▎  | 165221/225000 [318:58:28<99:36:31,  6.00s/it] 
 73%|███████▎  | 165222/225000 [318:58:33<95:30:45,  5.75s/it]
 73%|███████▎  | 165223/225000 [318:58:38<92:45:13,  5.59s/it]
 73%|███████▎  | 165224/225000 [318:58:43<88:51:50,  5.35s/it]
 73%|███████▎  | 165225/225000 [318:58:48<88:51:12,  5.35s/it]
 73%|███████▎  | 165226/225000 [318:58:53<87:24:20,  5.26s/it]
 73%|███████▎  | 165227/225000 [318:58:59<88:40:51,  5.34s/it]
 73%|███████▎  | 165228/225000 [318:59:04<87:26:45,  5.27s/it]
 73%|███████▎  | 165229/225000 [318:59:09<87:32:36,  5.27s/it]
 73%|███████▎  | 165230/225000 [318:59:15<88:51:41,  5.35s/it]
 73%|███████▎  | 165231/225000 [318:59:20<87:08:23,  5.25s/it]
 73%|███████▎  | 165232/225000 [318:59:25<86:04:15,  5.18s/it]
 73%|███████▎  | 165233/225000 [318:59:30<86:11:48,  5.19s/it]
 73%|███████▎  | 165234/225000 [318:59:35<86:40:04,  5.22s/it]
 73%|███████▎  | 165235/225000 [318:59:41<89:03:25,  5.36s/it]
 73%|███████▎  | 165236/225000 [318:59:47<89:55:13,  5.42s/it]
 73%|███████▎  | 165237/225000 [318:59:52<89:40:03,  5.40s/it]
 73%|███████▎  | 165238/225000 [318:59:57<87:04:46,  5.25s/it]
 73%|███████▎  | 165239/225000 [319:00:02<88:55:57,  5.36s/it]
 73%|███████▎  | 165240/225000 [319:00:07<86:19:16,  5.20s/it]
 73%|███████▎  | 165241/225000 [319:00:15<98:46:22,  5.95s/it]
 73%|███████▎  | 165242/225000 [319:00:27<129:53:26,  7.83s/it]
 73%|███████▎  | 165243/225000 [319:00:31<108:29:04,  6.54s/it]
 73%|███████▎  | 165244/225000 [319:00:38<112:10:07,  6.76s/it]
 73%|███████▎  | 165245/225000 [319:00:44<106:45:49,  6.43s/it]
 73%|███████▎  | 165246/225000 [319:00:47<91:50:28,  5.53s/it] 
 73%|███████▎  | 165247/225000 [319:00:54<99:33:21,  6.00s/it]
 73%|███████▎  | 165248/225000 [319:01:00<98:08:32,  5.91s/it]
 73%|███████▎  | 165249/225000 [319:01:05<91:46:17,  5.53s/it]
 73%|███████▎  | 165250/225000 [319:01:09<88:32:57,  5.34s/it]
                                                              
{'loss': '0.7646', 'grad_norm': '1.078', 'learning_rate': '8.96e-06', 'epoch': '0.7344', 'num_input_tokens_seen': 528182240, 'train_runtime': '1.148e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165250/225000 [319:01:09<88:32:57,  5.34s/it]
 73%|███████▎  | 165251/225000 [319:01:14<84:17:37,  5.08s/it]
 73%|███████▎  | 165252/225000 [319:01:18<79:39:29,  4.80s/it]
 73%|███████▎  | 165253/225000 [319:01:29<109:05:40,  6.57s/it]
 73%|███████▎  | 165254/225000 [319:01:34<101:04:50,  6.09s/it]
 73%|███████▎  | 165255/225000 [319:01:37<88:12:14,  5.31s/it] 
 73%|███████▎  | 165256/225000 [319:01:44<94:29:48,  5.69s/it]
 73%|███████▎  | 165257/225000 [319:01:49<92:59:08,  5.60s/it]
 73%|███████▎  | 165258/225000 [319:01:57<103:23:53,  6.23s/it]
 73%|███████▎  | 165259/225000 [319:02:00<89:08:06,  5.37s/it] 
 73%|███████▎  | 165260/225000 [319:02:04<80:30:45,  4.85s/it]
 73%|███████▎  | 165261/225000 [319:02:13<100:37:18,  6.06s/it]
 73%|███████▎  | 165262/225000 [319:02:16<88:42:49,  5.35s/it] 
 73%|███████▎  | 165263/225000 [319:02:20<80:06:11,  4.83s/it]
 73%|███████▎  | 165264/225000 [319:02:26<87:37:42,  5.28s/it]
 73%|███████▎  | 165265/225000 [319:02:33<96:00:22,  5.79s/it]
 73%|███████▎  | 165266/225000 [319:02:40<100:15:42,  6.04s/it]
 73%|███████▎  | 165267/225000 [319:02:45<93:45:46,  5.65s/it] 
 73%|███████▎  | 165268/225000 [319:02:48<82:00:46,  4.94s/it]
 73%|███████▎  | 165269/225000 [319:02:56<98:12:14,  5.92s/it]
 73%|███████▎  | 165270/225000 [319:03:00<86:11:46,  5.20s/it]
 73%|███████▎  | 165271/225000 [319:03:04<81:25:44,  4.91s/it]
 73%|███████▎  | 165272/225000 [319:03:08<78:30:21,  4.73s/it]
 73%|███████▎  | 165273/225000 [319:03:12<74:28:57,  4.49s/it]
 73%|███████▎  | 165274/225000 [319:03:17<74:05:48,  4.47s/it]
 73%|███████▎  | 165275/225000 [319:03:21<74:39:16,  4.50s/it]
 73%|███████▎  | 165276/225000 [319:03:32<107:26:31,  6.48s/it]
 73%|███████▎  | 165277/225000 [319:03:38<105:29:57,  6.36s/it]
 73%|███████▎  | 165278/225000 [319:03:42<92:37:24,  5.58s/it] 
 73%|███████▎  | 165279/225000 [319:03:51<107:48:48,  6.50s/it]
 73%|███████▎  | 165280/225000 [319:03:54<92:02:13,  5.55s/it] 
 73%|███████▎  | 165281/225000 [319:03:58<84:08:20,  5.07s/it]
 73%|███████▎  | 165282/225000 [319:04:08<106:34:42,  6.42s/it]
 73%|███████▎  | 165283/225000 [319:04:11<91:44:47,  5.53s/it] 
 73%|███████▎  | 165284/225000 [319:04:15<83:31:29,  5.04s/it]
 73%|███████▎  | 165285/225000 [319:04:24<103:35:35,  6.25s/it]
 73%|███████▎  | 165286/225000 [319:04:29<97:39:31,  5.89s/it] 
 73%|███████▎  | 165287/225000 [319:04:33<86:40:48,  5.23s/it]
 73%|███████▎  | 165288/225000 [319:04:37<82:01:04,  4.94s/it]
 73%|███████▎  | 165289/225000 [319:04:43<86:46:29,  5.23s/it]
 73%|███████▎  | 165290/225000 [319:04:50<96:51:14,  5.84s/it]
 73%|███████▎  | 165291/225000 [319:04:56<94:11:36,  5.68s/it]
 73%|███████▎  | 165292/225000 [319:05:02<97:07:28,  5.86s/it]
 73%|███████▎  | 165293/225000 [319:05:06<88:52:03,  5.36s/it]
 73%|███████▎  | 165294/225000 [319:05:10<83:08:56,  5.01s/it]
 73%|███████▎  | 165295/225000 [319:05:20<105:01:40,  6.33s/it]
 73%|███████▎  | 165296/225000 [319:05:23<90:11:34,  5.44s/it] 
 73%|███████▎  | 165297/225000 [319:05:31<100:33:10,  6.06s/it]
 73%|███████▎  | 165298/225000 [319:05:34<88:49:51,  5.36s/it] 
 73%|███████▎  | 165299/225000 [319:05:39<86:56:45,  5.24s/it]
 73%|███████▎  | 165300/225000 [319:05:46<94:45:42,  5.71s/it]
                                                              
{'loss': '0.7744', 'grad_norm': '1.089', 'learning_rate': '8.946e-06', 'epoch': '0.7347', 'num_input_tokens_seen': 528312088, 'train_runtime': '1.149e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165300/225000 [319:05:46<94:45:42,  5.71s/it]
 73%|███████▎  | 165301/225000 [319:05:50<84:29:56,  5.10s/it]
 73%|███████▎  | 165302/225000 [319:05:56<91:40:32,  5.53s/it]
 73%|███████▎  | 165303/225000 [319:06:03<95:36:04,  5.77s/it]
 73%|███████▎  | 165304/225000 [319:06:06<86:29:55,  5.22s/it]
 73%|███████▎  | 165305/225000 [319:06:11<81:24:26,  4.91s/it]
 73%|███████▎  | 165306/225000 [319:06:15<76:09:19,  4.59s/it]
 73%|███████▎  | 165307/225000 [319:06:25<107:19:37,  6.47s/it]
 73%|███████▎  | 165308/225000 [319:06:29<91:26:01,  5.51s/it] 
 73%|███████▎  | 165309/225000 [319:06:33<84:50:39,  5.12s/it]
 73%|███████▎  | 165310/225000 [319:06:40<96:35:03,  5.83s/it]
 73%|███████▎  | 165311/225000 [319:06:44<87:33:29,  5.28s/it]
 73%|███████▎  | 165312/225000 [319:06:48<81:32:48,  4.92s/it]
 73%|███████▎  | 165313/225000 [319:06:53<77:53:37,  4.70s/it]
 73%|███████▎  | 165314/225000 [319:06:57<77:41:16,  4.69s/it]
 73%|███████▎  | 165315/225000 [319:07:02<77:59:23,  4.70s/it]
 73%|███████▎  | 165316/225000 [319:07:09<87:40:27,  5.29s/it]
 73%|███████▎  | 165317/225000 [319:07:20<117:24:54,  7.08s/it]
 73%|███████▎  | 165318/225000 [319:07:24<103:10:20,  6.22s/it]
 73%|███████▎  | 165319/225000 [319:07:28<93:49:20,  5.66s/it] 
 73%|███████▎  | 165320/225000 [319:07:33<86:56:51,  5.24s/it]
 73%|███████▎  | 165321/225000 [319:07:42<107:05:29,  6.46s/it]
 73%|███████▎  | 165322/225000 [319:07:50<112:33:52,  6.79s/it]
 73%|███████▎  | 165323/225000 [319:07:53<97:07:54,  5.86s/it] 
 73%|███████▎  | 165324/225000 [319:07:57<85:44:23,  5.17s/it]
 73%|███████▎  | 165325/225000 [319:08:08<113:06:26,  6.82s/it]
 73%|███████▎  | 165326/225000 [319:08:11<95:36:55,  5.77s/it] 
 73%|███████▎  | 165327/225000 [319:08:18<104:03:25,  6.28s/it]
 73%|███████▎  | 165328/225000 [319:08:22<89:34:03,  5.40s/it] 
 73%|███████▎  | 165329/225000 [319:08:29<99:40:09,  6.01s/it]
 73%|███████▎  | 165330/225000 [319:08:32<85:28:52,  5.16s/it]
 73%|███████▎  | 165331/225000 [319:08:38<87:39:07,  5.29s/it]
 73%|███████▎  | 165332/225000 [319:08:43<88:02:50,  5.31s/it]
 73%|███████▎  | 165333/225000 [319:08:48<86:40:02,  5.23s/it]
 73%|███████▎  | 165334/225000 [319:08:56<97:41:22,  5.89s/it]
 73%|███████▎  | 165335/225000 [319:08:59<84:06:20,  5.07s/it]
 73%|███████▎  | 165336/225000 [319:09:07<100:43:09,  6.08s/it]
 73%|███████▎  | 165337/225000 [319:09:12<92:27:15,  5.58s/it] 
 73%|███████▎  | 165338/225000 [319:09:16<86:40:19,  5.23s/it]
 73%|███████▎  | 165339/225000 [319:09:20<81:07:47,  4.90s/it]
 73%|███████▎  | 165340/225000 [319:09:26<87:47:43,  5.30s/it]
 73%|███████▎  | 165341/225000 [319:09:36<106:52:06,  6.45s/it]
 73%|███████▎  | 165342/225000 [319:09:42<107:44:24,  6.50s/it]
 73%|███████▎  | 165343/225000 [319:09:46<92:25:29,  5.58s/it] 
 73%|███████▎  | 165344/225000 [319:09:53<100:48:41,  6.08s/it]
 73%|███████▎  | 165345/225000 [319:09:56<87:54:04,  5.30s/it] 
 73%|███████▎  | 165346/225000 [319:10:02<90:21:56,  5.45s/it]
 73%|███████▎  | 165347/225000 [319:10:09<96:30:55,  5.82s/it]
 73%|███████▎  | 165348/225000 [319:10:12<83:25:08,  5.03s/it]
 73%|███████▎  | 165349/225000 [319:10:16<78:08:25,  4.72s/it]
 73%|███████▎  | 165350/225000 [319:10:20<74:42:39,  4.51s/it]
                                                              
{'loss': '0.7742', 'grad_norm': '1.054', 'learning_rate': '8.932e-06', 'epoch': '0.7349', 'num_input_tokens_seen': 528441448, 'train_runtime': '1.149e+06', 'train_tokens_per_second': '459.9'}

 73%|███████▎  | 165350/225000 [319:10:20<74:42:39,  4.51s/it]
 73%|███████▎  | 165351/225000 [319:10:31<104:35:05,  6.31s/it]
 73%|███████▎  | 165352/225000 [319:10:34<89:17:41,  5.39s/it] 
 73%|███████▎  | 165353/225000 [319:10:40<91:37:19,  5.53s/it]
 73%|███████▎  | 165354/225000 [319:10:46<96:32:12,  5.83s/it]
 73%|███████▎  | 165355/225000 [319:10:52<94:05:45,  5.68s/it]
 73%|███████▎  | 165356/225000 [319:10:58<97:02:42,  5.86s/it]
 73%|███████▎  | 165357/225000 [319:11:02<88:31:53,  5.34s/it]
 73%|███████▎  | 165358/225000 [319:11:07<86:30:10,  5.22s/it]
 73%|███████▎  | 165359/225000 [319:11:11<82:43:39,  4.99s/it]
 73%|███████▎  | 165360/225000 [319:11:16<82:50:57,  5.00s/it]
 73%|███████▎  | 165361/225000 [319:11:21<80:35:36,  4.86s/it]
 73%|███████▎  | 165362/225000 [319:11:26<80:05:47,  4.83s/it]
 73%|███████▎  | 165363/225000 [319:11:31<80:47:39,  4.88s/it]
 73%|███████▎  | 165364/225000 [319:11:36<82:00:42,  4.95s/it]
 73%|███████▎  | 165365/225000 [319:11:41<82:46:01,  5.00s/it]
 73%|███████▎  | 165366/225000 [319:11:47<86:31:37,  5.22s/it]
 73%|███████▎  | 165367/225000 [319:11:59<124:19:31,  7.51s/it]
 73%|███████▎  | 165368/225000 [319:12:03<103:53:44,  6.27s/it]
 73%|███████▎  | 165369/225000 [319:12:08<99:16:05,  5.99s/it] 
 73%|███████▎  | 165370/225000 [319:12:15<104:42:18,  6.32s/it]
 73%|███████▎  | 165371/225000 [319:12:22<107:53:50,  6.51s/it]
 73%|███████▎  | 165372/225000 [319:12:26<93:00:28,  5.62s/it] 
 73%|███████▎  | 165373/225000 [319:12:33<101:28:24,  6.13s/it]
 73%|███████▎  | 165374/225000 [319:12:38<93:05:57,  5.62s/it] 
 74%|███████▎  | 165375/225000 [319:12:43<90:37:51,  5.47s/it]
 74%|███████▎  | 165376/225000 [319:12:49<92:36:42,  5.59s/it]
 74%|███████▎  | 165377/225000 [319:12:52<83:18:59,  5.03s/it]
 74%|███████▎  | 165378/225000 [319:12:56<77:45:56,  4.70s/it]
 74%|███████▎  | 165379/225000 [319:13:06<102:27:33,  6.19s/it]
 74%|███████▎  | 165380/225000 [319:13:11<98:59:08,  5.98s/it] 
 74%|███████▎  | 165381/225000 [319:13:16<92:26:54,  5.58s/it]
 74%|███████▎  | 165382/225000 [319:13:23<101:23:47,  6.12s/it]
 74%|███████▎  | 165383/225000 [319:13:27<88:02:40,  5.32s/it] 
 74%|███████▎  | 165384/225000 [319:13:35<100:45:49,  6.08s/it]
 74%|███████▎  | 165385/225000 [319:13:39<89:29:50,  5.40s/it] 
 74%|███████▎  | 165386/225000 [319:13:45<95:43:46,  5.78s/it]
 74%|███████▎  | 165387/225000 [319:13:50<92:29:25,  5.59s/it]
 74%|███████▎  | 165388/225000 [319:13:56<90:52:07,  5.49s/it]
 74%|███████▎  | 165389/225000 [319:14:00<85:46:43,  5.18s/it]
 74%|███████▎  | 165390/225000 [319:14:05<84:01:34,  5.07s/it]
 74%|███████▎  | 165391/225000 [319:14:09<81:44:48,  4.94s/it]
 74%|███████▎  | 165392/225000 [319:14:14<82:05:55,  4.96s/it]
 74%|███████▎  | 165393/225000 [319:14:21<87:45:00,  5.30s/it]
 74%|███████▎  | 165394/225000 [319:14:31<111:13:18,  6.72s/it]
 74%|███████▎  | 165395/225000 [319:14:35<97:59:21,  5.92s/it] 
 74%|███████▎  | 165396/225000 [319:14:40<96:57:18,  5.86s/it]
 74%|███████▎  | 165397/225000 [319:14:47<101:07:57,  6.11s/it]
 74%|███████▎  | 165398/225000 [319:14:53<98:02:25,  5.92s/it] 
 74%|███████▎  | 165399/225000 [319:14:57<92:22:03,  5.58s/it]
 74%|███████▎  | 165400/225000 [319:15:02<88:17:29,  5.33s/it]
                                                              
{'loss': '0.7672', 'grad_norm': '1.078', 'learning_rate': '8.918e-06', 'epoch': '0.7351', 'num_input_tokens_seen': 528571128, 'train_runtime': '1.149e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165400/225000 [319:15:02<88:17:29,  5.33s/it]
 74%|███████▎  | 165401/225000 [319:15:07<85:43:51,  5.18s/it]
 74%|███████▎  | 165402/225000 [319:15:11<81:21:35,  4.91s/it]
 74%|███████▎  | 165403/225000 [319:15:16<78:47:18,  4.76s/it]
 74%|███████▎  | 165404/225000 [319:15:28<117:54:31,  7.12s/it]
 74%|███████▎  | 165405/225000 [319:15:32<99:43:33,  6.02s/it] 
 74%|███████▎  | 165406/225000 [319:15:36<91:26:41,  5.52s/it]
 74%|███████▎  | 165407/225000 [319:15:41<89:38:27,  5.42s/it]
 74%|███████▎  | 165408/225000 [319:15:47<90:10:41,  5.45s/it]
 74%|███████▎  | 165409/225000 [319:15:52<91:08:20,  5.51s/it]
 74%|███████▎  | 165410/225000 [319:16:00<101:05:21,  6.11s/it]
 74%|███████▎  | 165411/225000 [319:16:06<99:37:59,  6.02s/it] 
 74%|███████▎  | 165412/225000 [319:16:10<93:07:27,  5.63s/it]
 74%|███████▎  | 165413/225000 [319:16:15<86:16:00,  5.21s/it]
 74%|███████▎  | 165414/225000 [319:16:19<80:07:04,  4.84s/it]
 74%|███████▎  | 165415/225000 [319:16:23<79:11:24,  4.78s/it]
 74%|███████▎  | 165416/225000 [319:16:35<114:39:57,  6.93s/it]
 74%|███████▎  | 165417/225000 [319:16:40<104:04:39,  6.29s/it]
 74%|███████▎  | 165418/225000 [319:16:45<98:09:38,  5.93s/it] 
 74%|███████▎  | 165419/225000 [319:16:49<89:40:28,  5.42s/it]
 74%|███████▎  | 165420/225000 [319:16:54<87:06:20,  5.26s/it]
 74%|███████▎  | 165421/225000 [319:16:59<86:38:11,  5.23s/it]
 74%|███████▎  | 165422/225000 [319:17:05<87:14:16,  5.27s/it]
 74%|███████▎  | 165423/225000 [319:17:10<87:09:28,  5.27s/it]
 74%|███████▎  | 165424/225000 [319:17:15<86:43:53,  5.24s/it]
 74%|███████▎  | 165425/225000 [319:17:20<84:38:07,  5.11s/it]
 74%|███████▎  | 165426/225000 [319:17:26<86:35:09,  5.23s/it]
 74%|███████▎  | 165427/225000 [319:17:30<83:27:56,  5.04s/it]
 74%|███████▎  | 165428/225000 [319:17:35<84:08:25,  5.08s/it]
 74%|███████▎  | 165429/225000 [319:17:41<84:44:37,  5.12s/it]
 74%|███████▎  | 165430/225000 [319:17:45<83:00:08,  5.02s/it]
 74%|███████▎  | 165431/225000 [319:17:50<83:01:43,  5.02s/it]
 74%|███████▎  | 165432/225000 [319:17:56<85:08:04,  5.15s/it]
 74%|███████▎  | 165433/225000 [319:18:01<86:14:43,  5.21s/it]
 74%|███████▎  | 165434/225000 [319:18:06<85:42:13,  5.18s/it]
 74%|███████▎  | 165435/225000 [319:18:11<85:52:08,  5.19s/it]
 74%|███████▎  | 165436/225000 [319:18:17<87:54:36,  5.31s/it]
 74%|███████▎  | 165437/225000 [319:18:23<89:00:54,  5.38s/it]
 74%|███████▎  | 165438/225000 [319:18:28<90:14:03,  5.45s/it]
 74%|███████▎  | 165439/225000 [319:18:36<99:50:36,  6.03s/it]
 74%|███████▎  | 165440/225000 [319:18:45<116:52:19,  7.06s/it]
 74%|███████▎  | 165441/225000 [319:18:49<100:24:54,  6.07s/it]
 74%|███████▎  | 165442/225000 [319:18:54<95:55:30,  5.80s/it] 
 74%|███████▎  | 165443/225000 [319:18:59<94:31:56,  5.71s/it]
 74%|███████▎  | 165444/225000 [319:19:09<111:58:45,  6.77s/it]
 74%|███████▎  | 165445/225000 [319:19:12<94:34:00,  5.72s/it] 
 74%|███████▎  | 165446/225000 [319:19:17<92:08:15,  5.57s/it]
 74%|███████▎  | 165447/225000 [319:19:25<100:44:31,  6.09s/it]
 74%|███████▎  | 165448/225000 [319:19:31<104:01:02,  6.29s/it]
 74%|███████▎  | 165449/225000 [319:19:37<99:59:05,  6.04s/it] 
 74%|███████▎  | 165450/225000 [319:19:40<88:32:00,  5.35s/it]
                                                              
{'loss': '0.7686', 'grad_norm': '1.069', 'learning_rate': '8.904e-06', 'epoch': '0.7353', 'num_input_tokens_seen': 528701408, 'train_runtime': '1.15e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165450/225000 [319:19:40<88:32:00,  5.35s/it]
 74%|███████▎  | 165451/225000 [319:19:49<105:51:22,  6.40s/it]
 74%|███████▎  | 165452/225000 [319:19:54<95:48:49,  5.79s/it] 
 74%|███████▎  | 165453/225000 [319:19:59<91:02:12,  5.50s/it]
 74%|███████▎  | 165454/225000 [319:20:03<86:54:38,  5.25s/it]
 74%|███████▎  | 165455/225000 [319:20:08<83:57:28,  5.08s/it]
 74%|███████▎  | 165456/225000 [319:20:13<82:01:38,  4.96s/it]
 74%|███████▎  | 165457/225000 [319:20:17<79:19:57,  4.80s/it]
 74%|███████▎  | 165458/225000 [319:20:22<78:42:41,  4.76s/it]
 74%|███████▎  | 165459/225000 [319:20:30<99:00:25,  5.99s/it]
 74%|███████▎  | 165460/225000 [319:20:40<115:57:23,  7.01s/it]
 74%|███████▎  | 165461/225000 [319:20:44<100:28:23,  6.08s/it]
 74%|███████▎  | 165462/225000 [319:20:48<89:06:37,  5.39s/it] 
 74%|███████▎  | 165463/225000 [319:20:56<104:39:01,  6.33s/it]
 74%|███████▎  | 165464/225000 [319:21:01<95:14:35,  5.76s/it] 
 74%|███████▎  | 165465/225000 [319:21:08<105:39:00,  6.39s/it]
 74%|███████▎  | 165466/225000 [319:21:13<95:31:40,  5.78s/it] 
 74%|███████▎  | 165467/225000 [319:21:18<92:26:33,  5.59s/it]
 74%|███████▎  | 165468/225000 [319:21:23<91:18:36,  5.52s/it]
 74%|███████▎  | 165469/225000 [319:21:28<89:50:10,  5.43s/it]
 74%|███████▎  | 165470/225000 [319:21:34<90:32:19,  5.48s/it]
 74%|███████▎  | 165471/225000 [319:21:40<90:57:20,  5.50s/it]
 74%|███████▎  | 165472/225000 [319:21:45<91:16:16,  5.52s/it]
 74%|███████▎  | 165473/225000 [319:21:50<89:21:13,  5.40s/it]
 74%|███████▎  | 165474/225000 [319:21:55<86:22:36,  5.22s/it]
 74%|███████▎  | 165475/225000 [319:22:00<86:00:32,  5.20s/it]
 74%|███████▎  | 165476/225000 [319:22:05<86:02:08,  5.20s/it]
 74%|███████▎  | 165477/225000 [319:22:11<86:33:35,  5.24s/it]
 74%|███████▎  | 165478/225000 [319:22:19<99:28:35,  6.02s/it]
 74%|███████▎  | 165479/225000 [319:22:28<116:17:16,  7.03s/it]
 74%|███████▎  | 165480/225000 [319:22:33<107:09:48,  6.48s/it]
 74%|███████▎  | 165481/225000 [319:22:38<96:32:10,  5.84s/it] 
 74%|███████▎  | 165482/225000 [319:22:42<87:55:52,  5.32s/it]
 74%|███████▎  | 165483/225000 [319:22:46<80:50:45,  4.89s/it]
 74%|███████▎  | 165484/225000 [319:22:57<115:09:19,  6.97s/it]
 74%|███████▎  | 165485/225000 [319:23:01<97:10:24,  5.88s/it] 
 74%|███████▎  | 165486/225000 [319:23:08<104:12:58,  6.30s/it]
 74%|███████▎  | 165487/225000 [319:23:11<89:09:41,  5.39s/it] 
 74%|███████▎  | 165488/225000 [319:23:15<83:06:59,  5.03s/it]
 74%|███████▎  | 165489/225000 [319:23:20<83:05:17,  5.03s/it]
 74%|███████▎  | 165490/225000 [319:23:26<83:43:08,  5.06s/it]
 74%|███████▎  | 165491/225000 [319:23:30<81:11:46,  4.91s/it]
 74%|███████▎  | 165492/225000 [319:23:35<79:31:38,  4.81s/it]
 74%|███████▎  | 165493/225000 [319:23:44<102:54:41,  6.23s/it]
 74%|███████▎  | 165494/225000 [319:23:51<103:03:46,  6.24s/it]
 74%|███████▎  | 165495/225000 [319:23:55<94:45:03,  5.73s/it] 
 74%|███████▎  | 165496/225000 [319:24:00<92:46:09,  5.61s/it]
 74%|███████▎  | 165497/225000 [319:24:05<89:10:54,  5.40s/it]
 74%|███████▎  | 165498/225000 [319:24:11<88:58:01,  5.38s/it]
 74%|███████▎  | 165499/225000 [319:24:16<87:17:15,  5.28s/it]
 74%|███████▎  | 165500/225000 [319:24:21<88:14:58,  5.34s/it]
                                                              
{'loss': '0.7744', 'grad_norm': '1.091', 'learning_rate': '8.89e-06', 'epoch': '0.7356', 'num_input_tokens_seen': 528831864, 'train_runtime': '1.15e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165500/225000 [319:24:21<88:14:58,  5.34s/it]
 74%|███████▎  | 165501/225000 [319:24:26<87:37:10,  5.30s/it]
 74%|███████▎  | 165502/225000 [319:24:31<85:46:03,  5.19s/it]
 74%|███████▎  | 165503/225000 [319:24:36<85:37:35,  5.18s/it]
 74%|███████▎  | 165504/225000 [319:24:41<83:40:20,  5.06s/it]
 74%|███████▎  | 165505/225000 [319:24:46<82:27:40,  4.99s/it]
 74%|███████▎  | 165506/225000 [319:24:55<99:56:25,  6.05s/it]
 74%|███████▎  | 165507/225000 [319:25:06<125:56:07,  7.62s/it]
 74%|███████▎  | 165508/225000 [319:25:10<107:29:55,  6.50s/it]
 74%|███████▎  | 165509/225000 [319:25:13<92:37:10,  5.60s/it] 
 74%|███████▎  | 165510/225000 [319:25:22<107:55:00,  6.53s/it]
 74%|███████▎  | 165511/225000 [319:25:28<105:13:10,  6.37s/it]
 74%|███████▎  | 165512/225000 [319:25:32<91:38:21,  5.55s/it] 
 74%|███████▎  | 165513/225000 [319:25:40<103:21:16,  6.25s/it]
 74%|███████▎  | 165514/225000 [319:25:44<95:01:12,  5.75s/it] 
 74%|███████▎  | 165515/225000 [319:25:49<90:20:05,  5.47s/it]
 74%|███████▎  | 165516/225000 [319:25:55<93:17:00,  5.65s/it]
 74%|███████▎  | 165517/225000 [319:26:02<100:20:38,  6.07s/it]
 74%|███████▎  | 165518/225000 [319:26:06<88:05:00,  5.33s/it] 
 74%|███████▎  | 165519/225000 [319:26:09<78:55:13,  4.78s/it]
 74%|███████▎  | 165520/225000 [319:26:18<98:45:30,  5.98s/it]
 74%|███████▎  | 165521/225000 [319:26:21<85:30:14,  5.18s/it]
 74%|███████▎  | 165522/225000 [319:26:28<91:44:39,  5.55s/it]
 74%|███████▎  | 165523/225000 [319:26:36<105:24:44,  6.38s/it]
 74%|███████▎  | 165524/225000 [319:26:39<89:37:40,  5.43s/it] 
 74%|███████▎  | 165525/225000 [319:26:46<97:41:45,  5.91s/it]
 74%|███████▎  | 165526/225000 [319:26:50<84:56:34,  5.14s/it]
 74%|███████▎  | 165527/225000 [319:26:56<92:35:35,  5.60s/it]
 74%|███████▎  | 165528/225000 [319:27:01<86:47:10,  5.25s/it]
 74%|███████▎  | 165529/225000 [319:27:05<81:09:32,  4.91s/it]
 74%|███████▎  | 165530/225000 [319:27:13<96:31:19,  5.84s/it]
 74%|███████▎  | 165531/225000 [319:27:17<88:05:37,  5.33s/it]
 74%|███████▎  | 165532/225000 [319:27:21<81:03:26,  4.91s/it]
 74%|███████▎  | 165533/225000 [319:27:29<97:34:46,  5.91s/it]
 74%|███████▎  | 165534/225000 [319:27:36<103:29:58,  6.27s/it]
 74%|███████▎  | 165535/225000 [319:27:40<90:59:19,  5.51s/it] 
 74%|███████▎  | 165536/225000 [319:27:44<81:52:56,  4.96s/it]
 74%|███████▎  | 165537/225000 [319:27:49<83:13:50,  5.04s/it]
 74%|███████▎  | 165538/225000 [319:27:58<105:34:55,  6.39s/it]
 74%|███████▎  | 165539/225000 [319:28:02<91:20:48,  5.53s/it] 
 74%|███████▎  | 165540/225000 [319:28:06<83:05:49,  5.03s/it]
 74%|███████▎  | 165541/225000 [319:28:10<77:53:23,  4.72s/it]
 74%|███████▎  | 165542/225000 [319:28:21<108:07:48,  6.55s/it]
 74%|███████▎  | 165543/225000 [319:28:25<98:08:34,  5.94s/it] 
 74%|███████▎  | 165544/225000 [319:28:29<89:33:06,  5.42s/it]
 74%|███████▎  | 165545/225000 [319:28:33<83:12:32,  5.04s/it]
 74%|███████▎  | 165546/225000 [319:28:38<81:05:24,  4.91s/it]
 74%|███████▎  | 165547/225000 [319:28:49<109:49:39,  6.65s/it]
 74%|███████▎  | 165548/225000 [319:28:53<100:23:20,  6.08s/it]
 74%|███████▎  | 165549/225000 [319:28:57<86:54:26,  5.26s/it] 
 74%|███████▎  | 165550/225000 [319:29:06<105:51:28,  6.41s/it]
                                                               
{'loss': '0.7733', 'grad_norm': '1.183', 'learning_rate': '8.876e-06', 'epoch': '0.7358', 'num_input_tokens_seen': 528961144, 'train_runtime': '1.15e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165550/225000 [319:29:06<105:51:28,  6.41s/it]
 74%|███████▎  | 165551/225000 [319:29:09<90:52:53,  5.50s/it] 
 74%|███████▎  | 165552/225000 [319:29:13<80:38:40,  4.88s/it]
 74%|███████▎  | 165553/225000 [319:29:22<103:57:09,  6.30s/it]
 74%|███████▎  | 165554/225000 [319:29:26<89:01:10,  5.39s/it] 
 74%|███████▎  | 165555/225000 [319:29:34<103:56:29,  6.29s/it]
 74%|███████▎  | 165556/225000 [319:29:38<91:03:03,  5.51s/it] 
 74%|███████▎  | 165557/225000 [319:29:42<84:48:53,  5.14s/it]
 74%|███████▎  | 165558/225000 [319:29:47<82:03:27,  4.97s/it]
 74%|███████▎  | 165559/225000 [319:29:56<103:57:15,  6.30s/it]
 74%|███████▎  | 165560/225000 [319:30:00<90:42:18,  5.49s/it] 
 74%|███████▎  | 165561/225000 [319:30:08<102:50:34,  6.23s/it]
 74%|███████▎  | 165562/225000 [319:30:11<89:38:47,  5.43s/it] 
 74%|███████▎  | 165563/225000 [319:30:18<95:41:19,  5.80s/it]
 74%|███████▎  | 165564/225000 [319:30:23<93:35:14,  5.67s/it]
 74%|███████▎  | 165565/225000 [319:30:27<82:40:27,  5.01s/it]
 74%|███████▎  | 165566/225000 [319:30:35<101:55:47,  6.17s/it]
 74%|███████▎  | 165567/225000 [319:30:39<89:43:30,  5.43s/it] 
 74%|███████▎  | 165568/225000 [319:30:46<95:04:09,  5.76s/it]
 74%|███████▎  | 165569/225000 [319:30:51<91:13:53,  5.53s/it]
 74%|███████▎  | 165570/225000 [319:30:54<81:56:30,  4.96s/it]
 74%|███████▎  | 165571/225000 [319:31:03<99:14:57,  6.01s/it]
 74%|███████▎  | 165572/225000 [319:31:07<89:20:58,  5.41s/it]
 74%|███████▎  | 165573/225000 [319:31:11<83:19:53,  5.05s/it]
 74%|███████▎  | 165574/225000 [319:31:16<83:01:40,  5.03s/it]
 74%|███████▎  | 165575/225000 [319:31:26<107:24:06,  6.51s/it]
 74%|███████▎  | 165576/225000 [319:31:30<97:28:35,  5.91s/it] 
 74%|███████▎  | 165577/225000 [319:31:35<92:01:11,  5.57s/it]
 74%|███████▎  | 165578/225000 [319:31:40<86:03:57,  5.21s/it]
 74%|███████▎  | 165579/225000 [319:31:44<82:30:09,  5.00s/it]
 74%|███████▎  | 165580/225000 [319:31:51<94:10:19,  5.71s/it]
 74%|███████▎  | 165581/225000 [319:32:00<109:49:02,  6.65s/it]
 74%|███████▎  | 165582/225000 [319:32:04<93:25:16,  5.66s/it] 
 74%|███████▎  | 165583/225000 [319:32:10<96:47:53,  5.86s/it]
 74%|███████▎  | 165584/225000 [319:32:15<92:27:07,  5.60s/it]
 74%|███████▎  | 165585/225000 [319:32:20<90:00:45,  5.45s/it]
 74%|███████▎  | 165586/225000 [319:32:25<88:52:36,  5.39s/it]
 74%|███████▎  | 165587/225000 [319:32:33<101:42:47,  6.16s/it]
 74%|███████▎  | 165588/225000 [319:32:37<89:07:42,  5.40s/it] 
 74%|███████▎  | 165589/225000 [319:32:41<80:04:10,  4.85s/it]
 74%|███████▎  | 165590/225000 [319:32:50<103:51:10,  6.29s/it]
 74%|███████▎  | 165591/225000 [319:32:54<92:00:02,  5.57s/it] 
 74%|███████▎  | 165592/225000 [319:32:59<89:56:14,  5.45s/it]
 74%|███████▎  | 165593/225000 [319:33:06<98:52:09,  5.99s/it]
 74%|███████▎  | 165594/225000 [319:33:10<85:02:06,  5.15s/it]
 74%|███████▎  | 165595/225000 [319:33:13<77:17:56,  4.68s/it]
 74%|███████▎  | 165596/225000 [319:33:22<97:04:02,  5.88s/it]
 74%|███████▎  | 165597/225000 [319:33:26<86:04:41,  5.22s/it]
 74%|███████▎  | 165598/225000 [319:33:30<79:29:58,  4.82s/it]
 74%|███████▎  | 165599/225000 [319:33:34<79:13:50,  4.80s/it]
 74%|███████▎  | 165600/225000 [319:33:44<104:04:49,  6.31s/it]
                                                               
{'loss': '0.7707', 'grad_norm': '1.111', 'learning_rate': '8.862e-06', 'epoch': '0.736', 'num_input_tokens_seen': 529090232, 'train_runtime': '1.15e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165600/225000 [319:33:44<104:04:49,  6.31s/it]
 74%|███████▎  | 165601/225000 [319:33:48<93:22:00,  5.66s/it] 
 74%|███████▎  | 165602/225000 [319:33:52<85:16:37,  5.17s/it]
 74%|███████▎  | 165603/225000 [319:33:57<80:55:13,  4.90s/it]
 74%|███████▎  | 165604/225000 [319:34:01<77:48:50,  4.72s/it]
 74%|███████▎  | 165605/225000 [319:34:05<76:23:54,  4.63s/it]
 74%|███████▎  | 165606/225000 [319:34:18<116:02:44,  7.03s/it]
 74%|███████▎  | 165607/225000 [319:34:22<100:16:08,  6.08s/it]
 74%|███████▎  | 165608/225000 [319:34:26<91:17:01,  5.53s/it] 
 74%|███████▎  | 165609/225000 [319:34:30<84:45:42,  5.14s/it]
 74%|███████▎  | 165610/225000 [319:34:38<96:22:06,  5.84s/it]
 74%|███████▎  | 165611/225000 [319:34:45<105:59:24,  6.42s/it]
 74%|███████▎  | 165612/225000 [319:34:49<92:00:48,  5.58s/it] 
 74%|███████▎  | 165613/225000 [319:34:54<89:02:53,  5.40s/it]
 74%|███████▎  | 165614/225000 [319:35:00<89:18:02,  5.41s/it]
 74%|███████▎  | 165615/225000 [319:35:05<87:52:55,  5.33s/it]
 74%|███████▎  | 165616/225000 [319:35:10<86:35:36,  5.25s/it]
 74%|███████▎  | 165617/225000 [319:35:15<87:18:14,  5.29s/it]
 74%|███████▎  | 165618/225000 [319:35:20<85:25:20,  5.18s/it]
 74%|███████▎  | 165619/225000 [319:35:25<86:07:20,  5.22s/it]
 74%|███████▎  | 165620/225000 [319:35:31<87:48:01,  5.32s/it]
 74%|███████▎  | 165621/225000 [319:35:36<88:57:46,  5.39s/it]
 74%|███████▎  | 165622/225000 [319:35:42<88:59:54,  5.40s/it]
 74%|███████▎  | 165623/225000 [319:35:54<121:15:47,  7.35s/it]
 74%|███████▎  | 165624/225000 [319:35:58<105:23:36,  6.39s/it]
 74%|███████▎  | 165625/225000 [319:36:02<94:03:44,  5.70s/it] 
 74%|███████▎  | 165626/225000 [319:36:06<87:56:27,  5.33s/it]
 74%|███████▎  | 165627/225000 [319:36:17<113:56:35,  6.91s/it]
 74%|███████▎  | 165628/225000 [319:36:21<97:14:59,  5.90s/it] 
 74%|███████▎  | 165629/225000 [319:36:28<103:19:47,  6.27s/it]
 74%|███████▎  | 165630/225000 [319:36:32<92:55:02,  5.63s/it] 
 74%|███████▎  | 165631/225000 [319:36:37<90:44:58,  5.50s/it]
 74%|███████▎  | 165632/225000 [319:36:45<104:08:21,  6.31s/it]
 74%|███████▎  | 165633/225000 [319:36:50<98:20:18,  5.96s/it] 
 74%|███████▎  | 165634/225000 [319:36:55<91:58:27,  5.58s/it]
 74%|███████▎  | 165635/225000 [319:37:00<90:14:14,  5.47s/it]
 74%|███████▎  | 165636/225000 [319:37:06<88:43:08,  5.38s/it]
 74%|███████▎  | 165637/225000 [319:37:10<85:34:04,  5.19s/it]
 74%|███████▎  | 165638/225000 [319:37:16<85:53:50,  5.21s/it]
 74%|███████▎  | 165639/225000 [319:37:21<85:36:19,  5.19s/it]
 74%|███████▎  | 165640/225000 [319:37:26<85:30:42,  5.19s/it]
 74%|███████▎  | 165641/225000 [319:37:31<83:48:11,  5.08s/it]
 74%|███████▎  | 165642/225000 [319:37:36<83:31:53,  5.07s/it]
 74%|███████▎  | 165643/225000 [319:37:41<83:09:43,  5.04s/it]
 74%|███████▎  | 165644/225000 [319:37:46<84:02:31,  5.10s/it]
 74%|███████▎  | 165645/225000 [319:37:51<83:33:46,  5.07s/it]
 74%|███████▎  | 165646/225000 [319:37:56<85:18:22,  5.17s/it]
 74%|███████▎  | 165647/225000 [319:38:01<82:56:10,  5.03s/it]
 74%|███████▎  | 165648/225000 [319:38:09<98:53:25,  6.00s/it]
 74%|███████▎  | 165649/225000 [319:38:20<121:10:25,  7.35s/it]
 74%|███████▎  | 165650/225000 [319:38:24<103:20:29,  6.27s/it]
                                                               
{'loss': '0.7801', 'grad_norm': '1.079', 'learning_rate': '8.848e-06', 'epoch': '0.7362', 'num_input_tokens_seen': 529220368, 'train_runtime': '1.151e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165650/225000 [319:38:24<103:20:29,  6.27s/it]
 74%|███████▎  | 165651/225000 [319:38:27<90:01:31,  5.46s/it] 
 74%|███████▎  | 165652/225000 [319:38:36<106:53:11,  6.48s/it]
 74%|███████▎  | 165653/225000 [319:38:39<91:31:00,  5.55s/it] 
 74%|███████▎  | 165654/225000 [319:38:44<85:29:23,  5.19s/it]
 74%|███████▎  | 165655/225000 [319:38:52<103:01:06,  6.25s/it]
 74%|███████▎  | 165656/225000 [319:38:56<90:59:21,  5.52s/it] 
 74%|███████▎  | 165657/225000 [319:39:00<83:12:50,  5.05s/it]
 74%|███████▎  | 165658/225000 [319:39:10<108:16:15,  6.57s/it]
 74%|███████▎  | 165659/225000 [319:39:14<94:16:25,  5.72s/it] 
 74%|███████▎  | 165660/225000 [319:39:21<99:17:14,  6.02s/it]
 74%|███████▎  | 165661/225000 [319:39:24<87:19:19,  5.30s/it]
 74%|███████▎  | 165662/225000 [319:39:33<105:59:10,  6.43s/it]
 74%|███████▎  | 165663/225000 [319:39:37<90:27:44,  5.49s/it] 
 74%|███████▎  | 165664/225000 [319:39:43<96:30:42,  5.86s/it]
 74%|███████▎  | 165665/225000 [319:39:49<93:54:12,  5.70s/it]
 74%|███████▎  | 165666/225000 [319:39:53<87:02:05,  5.28s/it]
 74%|███████▎  | 165667/225000 [319:39:57<81:11:59,  4.93s/it]
 74%|███████▎  | 165668/225000 [319:40:02<78:23:05,  4.76s/it]
 74%|███████▎  | 165669/225000 [319:40:11<102:23:33,  6.21s/it]
 74%|███████▎  | 165670/225000 [319:40:16<97:11:44,  5.90s/it] 
 74%|███████▎  | 165671/225000 [319:40:22<94:48:39,  5.75s/it]
 74%|███████▎  | 165672/225000 [319:40:28<97:46:04,  5.93s/it]
 74%|███████▎  | 165673/225000 [319:40:32<87:20:42,  5.30s/it]
 74%|███████▎  | 165674/225000 [319:40:36<79:43:47,  4.84s/it]
 74%|███████▎  | 165675/225000 [319:40:45<101:41:12,  6.17s/it]
 74%|███████▎  | 165676/225000 [319:40:49<91:35:54,  5.56s/it] 
 74%|███████▎  | 165677/225000 [319:40:53<85:27:39,  5.19s/it]
 74%|███████▎  | 165678/225000 [319:40:58<84:08:00,  5.11s/it]
 74%|███████▎  | 165679/225000 [319:41:09<111:50:24,  6.79s/it]
 74%|███████▎  | 165680/225000 [319:41:12<94:21:27,  5.73s/it] 
 74%|███████▎  | 165681/225000 [319:41:16<84:14:05,  5.11s/it]
 74%|███████▎  | 165682/225000 [319:41:21<82:22:36,  5.00s/it]
 74%|███████▎  | 165683/225000 [319:41:25<78:46:02,  4.78s/it]
 74%|███████▎  | 165684/225000 [319:41:32<88:19:17,  5.36s/it]
 74%|███████▎  | 165685/225000 [319:41:42<111:53:41,  6.79s/it]
 74%|███████▎  | 165686/225000 [319:41:46<98:46:51,  6.00s/it] 
 74%|███████▎  | 165687/225000 [319:41:50<90:19:11,  5.48s/it]
 74%|███████▎  | 165688/225000 [319:41:55<86:08:58,  5.23s/it]
 74%|███████▎  | 165689/225000 [319:41:59<81:22:04,  4.94s/it]
 74%|███████▎  | 165690/225000 [319:42:03<78:30:37,  4.77s/it]
 74%|███████▎  | 165691/225000 [319:42:14<106:16:46,  6.45s/it]
 74%|███████▎  | 165692/225000 [319:42:19<97:20:36,  5.91s/it] 
 74%|███████▎  | 165693/225000 [319:42:26<104:19:15,  6.33s/it]
 74%|███████▎  | 165694/225000 [319:42:30<92:52:06,  5.64s/it] 
 74%|███████▎  | 165695/225000 [319:42:34<85:30:02,  5.19s/it]
 74%|███████▎  | 165696/225000 [319:42:39<83:03:24,  5.04s/it]
 74%|███████▎  | 165697/225000 [319:42:49<109:26:49,  6.64s/it]
 74%|███████▎  | 165698/225000 [319:42:52<92:34:39,  5.62s/it] 
 74%|███████▎  | 165699/225000 [319:43:00<103:04:35,  6.26s/it]
 74%|███████▎  | 165700/225000 [319:43:05<95:48:57,  5.82s/it] 
                                                              
{'loss': '0.7592', 'grad_norm': '1.144', 'learning_rate': '8.834e-06', 'epoch': '0.7364', 'num_input_tokens_seen': 529351000, 'train_runtime': '1.151e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165700/225000 [319:43:05<95:48:57,  5.82s/it]
 74%|███████▎  | 165701/225000 [319:43:10<91:10:26,  5.54s/it]
 74%|███████▎  | 165702/225000 [319:43:15<87:35:57,  5.32s/it]
 74%|███████▎  | 165703/225000 [319:43:19<84:39:18,  5.14s/it]
 74%|███████▎  | 165704/225000 [319:43:25<85:14:08,  5.17s/it]
 74%|███████▎  | 165705/225000 [319:43:29<83:47:59,  5.09s/it]
 74%|███████▎  | 165706/225000 [319:43:34<81:55:38,  4.97s/it]
 74%|███████▎  | 165707/225000 [319:43:39<80:17:07,  4.87s/it]
 74%|███████▎  | 165708/225000 [319:43:44<82:09:42,  4.99s/it]
 74%|███████▎  | 165709/225000 [319:43:52<97:06:40,  5.90s/it]
 74%|███████▎  | 165710/225000 [319:44:02<116:36:27,  7.08s/it]
 74%|███████▎  | 165711/225000 [319:44:06<99:51:09,  6.06s/it] 
 74%|███████▎  | 165712/225000 [319:44:14<110:05:11,  6.68s/it]
 74%|███████▎  | 165713/225000 [319:44:17<95:10:53,  5.78s/it] 
 74%|███████▎  | 165714/225000 [319:44:22<88:33:09,  5.38s/it]
 74%|███████▎  | 165715/225000 [319:44:27<87:47:40,  5.33s/it]
 74%|███████▎  | 165716/225000 [319:44:34<93:30:51,  5.68s/it]
 74%|███████▎  | 165717/225000 [319:44:41<104:35:06,  6.35s/it]
 74%|███████▎  | 165718/225000 [319:44:45<92:11:57,  5.60s/it] 
 74%|███████▎  | 165719/225000 [319:44:49<83:33:20,  5.07s/it]
 74%|███████▎  | 165720/225000 [319:44:59<106:22:30,  6.46s/it]
 74%|███████▎  | 165721/225000 [319:45:02<89:39:30,  5.44s/it] 
 74%|███████▎  | 165722/225000 [319:45:08<93:25:19,  5.67s/it]
 74%|███████▎  | 165723/225000 [319:45:13<90:18:40,  5.48s/it]
 74%|███████▎  | 165724/225000 [319:45:18<86:04:34,  5.23s/it]
 74%|███████▎  | 165725/225000 [319:45:23<84:43:54,  5.15s/it]
 74%|███████▎  | 165726/225000 [319:45:27<82:45:49,  5.03s/it]
 74%|███████▎  | 165727/225000 [319:45:33<82:45:32,  5.03s/it]
 74%|███████▎  | 165728/225000 [319:45:37<80:26:04,  4.89s/it]
 74%|███████▎  | 165729/225000 [319:45:42<80:10:44,  4.87s/it]
 74%|███████▎  | 165730/225000 [319:45:47<81:12:54,  4.93s/it]
 74%|███████▎  | 165731/225000 [319:45:52<82:22:55,  5.00s/it]
 74%|███████▎  | 165732/225000 [319:45:57<81:24:55,  4.95s/it]
 74%|███████▎  | 165733/225000 [319:46:02<80:37:36,  4.90s/it]
 74%|███████▎  | 165734/225000 [319:46:07<81:33:23,  4.95s/it]
 74%|███████▎  | 165735/225000 [319:46:12<81:15:16,  4.94s/it]
 74%|███████▎  | 165736/225000 [319:46:17<81:30:35,  4.95s/it]
 74%|███████▎  | 165737/225000 [319:46:22<81:41:12,  4.96s/it]
 74%|███████▎  | 165738/225000 [319:46:32<109:59:30,  6.68s/it]
 74%|███████▎  | 165739/225000 [319:46:38<104:17:53,  6.34s/it]
 74%|███████▎  | 165740/225000 [319:46:42<93:09:25,  5.66s/it] 
 74%|███████▎  | 165741/225000 [319:46:47<88:15:53,  5.36s/it]
 74%|███████▎  | 165742/225000 [319:46:51<84:00:24,  5.10s/it]
 74%|███████▎  | 165743/225000 [319:46:56<82:23:12,  5.01s/it]
 74%|███████▎  | 165744/225000 [319:47:01<80:52:33,  4.91s/it]
 74%|███████▎  | 165745/225000 [319:47:05<80:07:00,  4.87s/it]
 74%|███████▎  | 165746/225000 [319:47:10<79:54:14,  4.85s/it]
 74%|███████▎  | 165747/225000 [319:47:23<120:09:24,  7.30s/it]
 74%|███████▎  | 165748/225000 [319:47:28<109:54:00,  6.68s/it]
 74%|███████▎  | 165749/225000 [319:47:32<95:02:16,  5.77s/it] 
 74%|███████▎  | 165750/225000 [319:47:41<112:33:52,  6.84s/it]
                                                               
{'loss': '0.78', 'grad_norm': '1.142', 'learning_rate': '8.82e-06', 'epoch': '0.7367', 'num_input_tokens_seen': 529479968, 'train_runtime': '1.151e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165750/225000 [319:47:41<112:33:52,  6.84s/it]
 74%|███████▎  | 165751/225000 [319:47:46<100:21:45,  6.10s/it]
 74%|███████▎  | 165752/225000 [319:47:51<93:27:05,  5.68s/it] 
 74%|███████▎  | 165753/225000 [319:47:57<95:27:36,  5.80s/it]
 74%|███████▎  | 165754/225000 [319:48:02<95:13:59,  5.79s/it]
 74%|███████▎  | 165755/225000 [319:48:06<85:09:34,  5.17s/it]
 74%|███████▎  | 165756/225000 [319:48:10<81:01:59,  4.92s/it]
 74%|███████▎  | 165757/225000 [319:48:15<81:37:46,  4.96s/it]
 74%|███████▎  | 165758/225000 [319:48:28<117:32:12,  7.14s/it]
 74%|███████▎  | 165759/225000 [319:48:31<99:54:52,  6.07s/it] 
 74%|███████▎  | 165760/225000 [319:48:35<88:16:21,  5.36s/it]
 74%|███████▎  | 165761/225000 [319:48:44<107:41:56,  6.54s/it]
 74%|███████▎  | 165762/225000 [319:48:48<92:03:30,  5.59s/it] 
 74%|███████▎  | 165763/225000 [319:48:51<81:17:09,  4.94s/it]
 74%|███████▎  | 165764/225000 [319:49:01<104:01:30,  6.32s/it]
 74%|███████▎  | 165765/225000 [319:49:06<97:58:51,  5.95s/it] 
 74%|███████▎  | 165766/225000 [319:49:11<96:25:36,  5.86s/it]
 74%|███████▎  | 165767/225000 [319:49:18<98:49:19,  6.01s/it]
 74%|███████▎  | 165768/225000 [319:49:22<92:29:40,  5.62s/it]
 74%|███████▎  | 165769/225000 [319:49:27<87:06:23,  5.29s/it]
 74%|███████▎  | 165770/225000 [319:49:32<83:43:15,  5.09s/it]
 74%|███████▎  | 165771/225000 [319:49:37<82:52:16,  5.04s/it]
 74%|███████▎  | 165772/225000 [319:49:41<80:08:29,  4.87s/it]
 74%|███████▎  | 165773/225000 [319:49:51<106:29:55,  6.47s/it]
 74%|███████▎  | 165774/225000 [319:49:55<93:11:36,  5.66s/it] 
 74%|███████▎  | 165775/225000 [319:49:59<86:30:15,  5.26s/it]
 74%|███████▎  | 165776/225000 [319:50:08<102:13:09,  6.21s/it]
 74%|███████▎  | 165777/225000 [319:50:12<90:26:26,  5.50s/it] 
 74%|███████▎  | 165778/225000 [319:50:19<100:07:57,  6.09s/it]
 74%|███████▎  | 165779/225000 [319:50:24<92:18:04,  5.61s/it] 
 74%|███████▎  | 165780/225000 [319:50:31<99:13:09,  6.03s/it]
 74%|███████▎  | 165781/225000 [319:50:34<88:01:51,  5.35s/it]
 74%|███████▎  | 165782/225000 [319:50:38<81:21:40,  4.95s/it]
 74%|███████▎  | 165783/225000 [319:50:45<87:33:42,  5.32s/it]
 74%|███████▎  | 165784/225000 [319:50:51<91:48:13,  5.58s/it]
 74%|███████▎  | 165785/225000 [319:50:58<99:31:42,  6.05s/it]
 74%|███████▎  | 165786/225000 [319:51:03<95:44:06,  5.82s/it]
 74%|███████▎  | 165787/225000 [319:51:08<91:41:46,  5.57s/it]
 74%|███████▎  | 165788/225000 [319:51:13<89:42:52,  5.45s/it]
 74%|███████▎  | 165789/225000 [319:51:18<85:24:28,  5.19s/it]
 74%|███████▎  | 165790/225000 [319:51:23<84:04:47,  5.11s/it]
 74%|███████▎  | 165791/225000 [319:51:28<84:43:21,  5.15s/it]
 74%|███████▎  | 165792/225000 [319:51:33<85:23:26,  5.19s/it]
 74%|███████▎  | 165793/225000 [319:51:38<84:47:52,  5.16s/it]
 74%|███████▎  | 165794/225000 [319:51:43<82:59:52,  5.05s/it]
 74%|███████▎  | 165795/225000 [319:51:48<84:10:19,  5.12s/it]
 74%|███████▎  | 165796/225000 [319:51:55<92:22:58,  5.62s/it]
 74%|███████▎  | 165797/225000 [319:52:04<107:42:54,  6.55s/it]
 74%|███████▎  | 165798/225000 [319:52:09<98:06:04,  5.97s/it] 
 74%|███████▎  | 165799/225000 [319:52:13<91:07:40,  5.54s/it]
 74%|███████▎  | 165800/225000 [319:52:18<87:10:49,  5.30s/it]
                                                              
{'loss': '0.7704', 'grad_norm': '1.124', 'learning_rate': '8.806e-06', 'epoch': '0.7369', 'num_input_tokens_seen': 529610576, 'train_runtime': '1.152e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165800/225000 [319:52:18<87:10:49,  5.30s/it]
 74%|███████▎  | 165801/225000 [319:52:23<84:49:05,  5.16s/it]
 74%|███████▎  | 165802/225000 [319:52:28<83:15:16,  5.06s/it]
 74%|███████▎  | 165803/225000 [319:52:32<81:28:46,  4.96s/it]
 74%|███████▎  | 165804/225000 [319:52:37<81:49:40,  4.98s/it]
 74%|███████▎  | 165805/225000 [319:52:42<82:38:04,  5.03s/it]
 74%|███████▎  | 165806/225000 [319:52:48<83:29:12,  5.08s/it]
 74%|███████▎  | 165807/225000 [319:52:53<84:05:13,  5.11s/it]
 74%|███████▎  | 165808/225000 [319:52:58<82:01:56,  4.99s/it]
 74%|███████▎  | 165809/225000 [319:53:10<120:26:00,  7.32s/it]
 74%|███████▎  | 165810/225000 [319:53:15<105:47:16,  6.43s/it]
 74%|███████▎  | 165811/225000 [319:53:19<95:14:43,  5.79s/it] 
 74%|███████▎  | 165812/225000 [319:53:24<92:04:52,  5.60s/it]
 74%|███████▎  | 165813/225000 [319:53:35<118:03:05,  7.18s/it]
 74%|███████▎  | 165814/225000 [319:53:38<98:19:25,  5.98s/it] 
 74%|███████▎  | 165815/225000 [319:53:42<89:55:06,  5.47s/it]
 74%|███████▎  | 165816/225000 [319:53:47<86:38:48,  5.27s/it]
 74%|███████▎  | 165817/225000 [319:53:52<83:20:56,  5.07s/it]
 74%|███████▎  | 165818/225000 [319:53:57<81:50:13,  4.98s/it]
 74%|███████▎  | 165819/225000 [319:54:01<79:44:43,  4.85s/it]
 74%|███████▎  | 165820/225000 [319:54:06<78:41:09,  4.79s/it]
 74%|███████▎  | 165821/225000 [319:54:11<79:01:11,  4.81s/it]
 74%|███████▎  | 165822/225000 [319:54:20<99:16:31,  6.04s/it]
 74%|███████▎  | 165823/225000 [319:54:29<115:20:25,  7.02s/it]
 74%|███████▎  | 165824/225000 [319:54:36<114:19:26,  6.95s/it]
 74%|███████▎  | 165825/225000 [319:54:41<104:27:29,  6.35s/it]
 74%|███████▎  | 165826/225000 [319:54:46<97:59:16,  5.96s/it] 
 74%|███████▎  | 165827/225000 [319:54:51<95:07:08,  5.79s/it]
 74%|███████▎  | 165828/225000 [319:54:56<91:25:11,  5.56s/it]
 74%|███████▎  | 165829/225000 [319:55:01<87:40:32,  5.33s/it]
 74%|███████▎  | 165830/225000 [319:55:06<86:10:15,  5.24s/it]
 74%|███████▎  | 165831/225000 [319:55:11<86:22:29,  5.26s/it]
 74%|███████▎  | 165832/225000 [319:55:17<86:58:14,  5.29s/it]
 74%|███████▎  | 165833/225000 [319:55:22<87:11:12,  5.30s/it]
 74%|███████▎  | 165834/225000 [319:55:27<86:49:22,  5.28s/it]
 74%|███████▎  | 165835/225000 [319:55:32<85:15:51,  5.19s/it]
 74%|███████▎  | 165836/225000 [319:55:38<90:08:52,  5.49s/it]
 74%|███████▎  | 165837/225000 [319:55:51<125:11:58,  7.62s/it]
 74%|███████▎  | 165838/225000 [319:55:54<103:22:51,  6.29s/it]
 74%|███████▎  | 165839/225000 [319:55:59<97:27:42,  5.93s/it] 
 74%|███████▎  | 165840/225000 [319:56:06<103:57:17,  6.33s/it]
 74%|███████▎  | 165841/225000 [319:56:10<92:41:23,  5.64s/it] 
 74%|███████▎  | 165842/225000 [319:56:14<84:23:59,  5.14s/it]
 74%|███████▎  | 165843/225000 [319:56:25<109:33:48,  6.67s/it]
 74%|███████▎  | 165844/225000 [319:56:28<94:36:00,  5.76s/it] 
 74%|███████▎  | 165845/225000 [319:56:32<85:12:53,  5.19s/it]
 74%|███████▎  | 165846/225000 [319:56:38<90:36:13,  5.51s/it]
 74%|███████▎  | 165847/225000 [319:56:46<99:41:22,  6.07s/it]
 74%|███████▎  | 165848/225000 [319:56:49<86:55:14,  5.29s/it]
 74%|███████▎  | 165849/225000 [319:56:53<79:20:41,  4.83s/it]
 74%|███████▎  | 165850/225000 [319:57:04<110:27:12,  6.72s/it]
                                                               
{'loss': '0.7647', 'grad_norm': '1.216', 'learning_rate': '8.792e-06', 'epoch': '0.7371', 'num_input_tokens_seen': 529741856, 'train_runtime': '1.152e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165850/225000 [319:57:04<110:27:12,  6.72s/it]
 74%|███████▎  | 165851/225000 [319:57:08<94:24:07,  5.75s/it] 
 74%|███████▎  | 165852/225000 [319:57:16<105:20:39,  6.41s/it]
 74%|███████▎  | 165853/225000 [319:57:19<90:49:44,  5.53s/it] 
 74%|███████▎  | 165854/225000 [319:57:23<81:28:15,  4.96s/it]
 74%|███████▎  | 165855/225000 [319:57:26<75:45:29,  4.61s/it]
 74%|███████▎  | 165856/225000 [319:57:37<104:03:13,  6.33s/it]
 74%|███████▎  | 165857/225000 [319:57:40<90:01:19,  5.48s/it] 
 74%|███████▎  | 165858/225000 [319:57:44<80:59:07,  4.93s/it]
 74%|███████▎  | 165859/225000 [319:57:53<101:54:39,  6.20s/it]
 74%|███████▎  | 165860/225000 [319:57:57<89:25:29,  5.44s/it] 
 74%|███████▎  | 165861/225000 [319:58:00<79:35:29,  4.85s/it]
 74%|███████▎  | 165862/225000 [319:58:05<76:56:03,  4.68s/it]
 74%|███████▎  | 165863/225000 [319:58:09<74:55:13,  4.56s/it]
 74%|███████▎  | 165864/225000 [319:58:14<79:35:45,  4.85s/it]
 74%|███████▎  | 165865/225000 [319:58:26<114:05:59,  6.95s/it]
 74%|███████▎  | 165866/225000 [319:58:31<102:38:01,  6.25s/it]
 74%|███████▎  | 165867/225000 [319:58:35<93:23:47,  5.69s/it] 
 74%|███████▎  | 165868/225000 [319:58:39<86:34:31,  5.27s/it]
 74%|███████▎  | 165869/225000 [319:58:44<82:44:57,  5.04s/it]
 74%|███████▎  | 165870/225000 [319:58:49<80:23:42,  4.89s/it]
 74%|███████▎  | 165871/225000 [319:58:57<96:49:25,  5.89s/it]
 74%|███████▎  | 165872/225000 [319:59:04<105:09:46,  6.40s/it]
 74%|███████▎  | 165873/225000 [319:59:09<95:57:09,  5.84s/it] 
 74%|███████▎  | 165874/225000 [319:59:13<89:36:46,  5.46s/it]
 74%|███████▎  | 165875/225000 [319:59:18<83:53:20,  5.11s/it]
 74%|███████▎  | 165876/225000 [319:59:22<81:31:29,  4.96s/it]
 74%|███████▎  | 165877/225000 [319:59:27<78:59:56,  4.81s/it]
 74%|███████▎  | 165878/225000 [319:59:34<88:43:28,  5.40s/it]
 74%|███████▎  | 165879/225000 [319:59:45<117:53:21,  7.18s/it]
 74%|███████▎  | 165880/225000 [319:59:48<96:39:24,  5.89s/it] 
 74%|███████▎  | 165881/225000 [319:59:53<91:18:58,  5.56s/it]
 74%|███████▎  | 165882/225000 [319:59:57<86:54:35,  5.29s/it]
 74%|███████▎  | 165883/225000 [320:00:02<83:47:13,  5.10s/it]
 74%|███████▎  | 165884/225000 [320:00:07<83:16:28,  5.07s/it]
 74%|███████▎  | 165885/225000 [320:00:12<81:30:39,  4.96s/it]
 74%|███████▎  | 165886/225000 [320:00:16<80:54:43,  4.93s/it]
 74%|███████▎  | 165887/225000 [320:00:21<80:38:57,  4.91s/it]
 74%|███████▎  | 165888/225000 [320:00:26<78:45:40,  4.80s/it]
 74%|███████▎  | 165889/225000 [320:00:38<116:50:31,  7.12s/it]
 74%|███████▎  | 165890/225000 [320:00:44<108:13:29,  6.59s/it]
 74%|███████▎  | 165891/225000 [320:00:48<97:20:13,  5.93s/it] 
 74%|███████▎  | 165892/225000 [320:00:56<106:48:11,  6.50s/it]
 74%|███████▎  | 165893/225000 [320:01:01<98:40:35,  6.01s/it] 
 74%|███████▎  | 165894/225000 [320:01:06<92:23:58,  5.63s/it]
 74%|███████▎  | 165895/225000 [320:01:11<89:50:00,  5.47s/it]
 74%|███████▎  | 165896/225000 [320:01:16<86:42:37,  5.28s/it]
 74%|███████▎  | 165897/225000 [320:01:20<83:10:56,  5.07s/it]
 74%|███████▎  | 165898/225000 [320:01:25<80:54:26,  4.93s/it]
 74%|███████▎  | 165899/225000 [320:01:29<77:44:26,  4.74s/it]
 74%|███████▎  | 165900/225000 [320:01:36<89:20:36,  5.44s/it]
                                                              
{'loss': '0.783', 'grad_norm': '1.211', 'learning_rate': '8.778e-06', 'epoch': '0.7373', 'num_input_tokens_seen': 529871256, 'train_runtime': '1.152e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▎  | 165900/225000 [320:01:36<89:20:36,  5.44s/it]
 74%|███████▎  | 165901/225000 [320:01:46<112:08:23,  6.83s/it]
 74%|███████▎  | 165902/225000 [320:01:50<99:17:03,  6.05s/it] 
 74%|███████▎  | 165903/225000 [320:01:54<89:22:09,  5.44s/it]
 74%|███████▎  | 165904/225000 [320:01:58<81:46:43,  4.98s/it]
 74%|███████▎  | 165905/225000 [320:02:08<105:51:45,  6.45s/it]
 74%|███████▎  | 165906/225000 [320:02:13<99:43:19,  6.08s/it] 
 74%|███████▎  | 165907/225000 [320:02:18<92:19:10,  5.62s/it]
 74%|███████▎  | 165908/225000 [320:02:22<83:40:33,  5.10s/it]
 74%|███████▎  | 165909/225000 [320:02:26<80:03:05,  4.88s/it]
 74%|███████▎  | 165910/225000 [320:02:31<79:30:37,  4.84s/it]
 74%|███████▎  | 165911/225000 [320:02:36<79:24:31,  4.84s/it]
 74%|███████▎  | 165912/225000 [320:02:48<113:46:05,  6.93s/it]
 74%|███████▎  | 165913/225000 [320:02:52<100:54:34,  6.15s/it]
 74%|███████▎  | 165914/225000 [320:02:57<93:42:22,  5.71s/it] 
 74%|███████▎  | 165915/225000 [320:03:02<92:57:35,  5.66s/it]
 74%|███████▎  | 165916/225000 [320:03:07<89:09:08,  5.43s/it]
 74%|███████▎  | 165917/225000 [320:03:12<86:05:01,  5.25s/it]
 74%|███████▎  | 165918/225000 [320:03:17<83:31:10,  5.09s/it]
 74%|███████▎  | 165919/225000 [320:03:22<83:19:09,  5.08s/it]
 74%|███████▎  | 165920/225000 [320:03:27<84:31:08,  5.15s/it]
 74%|███████▎  | 165921/225000 [320:03:31<81:12:40,  4.95s/it]
 74%|███████▎  | 165922/225000 [320:03:36<81:42:35,  4.98s/it]
 74%|███████▎  | 165923/225000 [320:03:41<81:27:15,  4.96s/it]
 74%|███████▎  | 165924/225000 [320:03:46<81:28:51,  4.97s/it]
 74%|███████▎  | 165925/225000 [320:03:52<82:18:50,  5.02s/it]
 74%|███████▎  | 165926/225000 [320:03:57<84:14:42,  5.13s/it]
 74%|███████▎  | 165927/225000 [320:04:02<82:20:54,  5.02s/it]
 74%|███████▎  | 165928/225000 [320:04:14<117:50:51,  7.18s/it]
 74%|███████▎  | 165929/225000 [320:04:21<117:23:05,  7.15s/it]
 74%|███████▎  | 165930/225000 [320:04:25<99:26:16,  6.06s/it] 
 74%|███████▎  | 165931/225000 [320:04:33<111:33:18,  6.80s/it]
 74%|███████▎  | 165932/225000 [320:04:37<95:46:51,  5.84s/it] 
 74%|███████▎  | 165933/225000 [320:04:45<107:21:19,  6.54s/it]
 74%|███████▎  | 165934/225000 [320:04:49<97:42:07,  5.95s/it] 
 74%|███████▎  | 165935/225000 [320:04:54<90:52:49,  5.54s/it]
 74%|███████▎  | 165936/225000 [320:04:58<85:39:47,  5.22s/it]
 74%|███████▎  | 165937/225000 [320:05:03<81:56:21,  4.99s/it]
 74%|███████▍  | 165938/225000 [320:05:12<102:56:57,  6.28s/it]
 74%|███████▍  | 165939/225000 [320:05:16<88:45:21,  5.41s/it] 
 74%|███████▍  | 165940/225000 [320:05:25<109:10:38,  6.65s/it]
 74%|███████▍  | 165941/225000 [320:05:29<93:38:29,  5.71s/it] 
 74%|███████▍  | 165942/225000 [320:05:32<83:33:05,  5.09s/it]
 74%|███████▍  | 165943/225000 [320:05:36<77:09:58,  4.70s/it]
 74%|███████▍  | 165944/225000 [320:05:40<75:32:08,  4.60s/it]
 74%|███████▍  | 165945/225000 [320:05:48<92:10:02,  5.62s/it]
 74%|███████▍  | 165946/225000 [320:05:56<103:49:22,  6.33s/it]
 74%|███████▍  | 165947/225000 [320:06:00<90:20:05,  5.51s/it] 
 74%|███████▍  | 165948/225000 [320:06:06<92:33:22,  5.64s/it]
 74%|███████▍  | 165949/225000 [320:06:12<96:02:01,  5.85s/it]
 74%|███████▍  | 165950/225000 [320:06:16<84:45:01,  5.17s/it]
                                                              
{'loss': '0.7694', 'grad_norm': '1.266', 'learning_rate': '8.764e-06', 'epoch': '0.7376', 'num_input_tokens_seen': 530002280, 'train_runtime': '1.152e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 165950/225000 [320:06:16<84:45:01,  5.17s/it]
 74%|███████▍  | 165951/225000 [320:06:21<82:22:04,  5.02s/it]
 74%|███████▍  | 165952/225000 [320:06:25<79:07:30,  4.82s/it]
 74%|███████▍  | 165953/225000 [320:06:35<106:34:00,  6.50s/it]
 74%|███████▍  | 165954/225000 [320:06:39<91:56:52,  5.61s/it] 
 74%|███████▍  | 165955/225000 [320:06:42<80:17:39,  4.90s/it]
 74%|███████▍  | 165956/225000 [320:06:47<78:05:36,  4.76s/it]
 74%|███████▍  | 165957/225000 [320:06:55<93:52:40,  5.72s/it]
 74%|███████▍  | 165958/225000 [320:07:01<97:37:24,  5.95s/it]
 74%|███████▍  | 165959/225000 [320:07:05<87:35:28,  5.34s/it]
 74%|███████▍  | 165960/225000 [320:07:10<86:35:53,  5.28s/it]
 74%|███████▍  | 165961/225000 [320:07:15<86:00:24,  5.24s/it]
 74%|███████▍  | 165962/225000 [320:07:20<85:16:48,  5.20s/it]
 74%|███████▍  | 165963/225000 [320:07:25<82:38:17,  5.04s/it]
 74%|███████▍  | 165964/225000 [320:07:31<85:12:57,  5.20s/it]
 74%|███████▍  | 165965/225000 [320:07:36<85:38:50,  5.22s/it]
 74%|███████▍  | 165966/225000 [320:07:41<85:18:52,  5.20s/it]
 74%|███████▍  | 165967/225000 [320:07:46<84:41:07,  5.16s/it]
 74%|███████▍  | 165968/225000 [320:07:51<83:15:32,  5.08s/it]
 74%|███████▍  | 165969/225000 [320:07:56<84:19:32,  5.14s/it]
 74%|███████▍  | 165970/225000 [320:08:01<83:46:52,  5.11s/it]
 74%|███████▍  | 165971/225000 [320:08:07<84:42:55,  5.17s/it]
 74%|███████▍  | 165972/225000 [320:08:15<100:47:52,  6.15s/it]
 74%|███████▍  | 165973/225000 [320:08:25<119:50:56,  7.31s/it]
 74%|███████▍  | 165974/225000 [320:08:28<100:48:11,  6.15s/it]
 74%|███████▍  | 165975/225000 [320:08:37<110:21:21,  6.73s/it]
 74%|███████▍  | 165976/225000 [320:08:41<101:02:23,  6.16s/it]
 74%|███████▍  | 165977/225000 [320:08:45<87:51:43,  5.36s/it] 
 74%|███████▍  | 165978/225000 [320:08:53<103:25:27,  6.31s/it]
 74%|███████▍  | 165979/225000 [320:08:59<99:07:10,  6.05s/it] 
 74%|███████▍  | 165980/225000 [320:09:02<85:48:27,  5.23s/it]
 74%|███████▍  | 165981/225000 [320:09:11<105:51:25,  6.46s/it]
 74%|███████▍  | 165982/225000 [320:09:15<90:28:23,  5.52s/it] 
 74%|███████▍  | 165983/225000 [320:09:19<82:47:44,  5.05s/it]
 74%|███████▍  | 165984/225000 [320:09:23<79:43:53,  4.86s/it]
 74%|███████▍  | 165985/225000 [320:09:28<79:01:42,  4.82s/it]
 74%|███████▍  | 165986/225000 [320:09:37<98:33:55,  6.01s/it]
 74%|███████▍  | 165987/225000 [320:09:45<109:03:25,  6.65s/it]
 74%|███████▍  | 165988/225000 [320:09:48<93:35:14,  5.71s/it] 
 74%|███████▍  | 165989/225000 [320:09:52<83:41:28,  5.11s/it]
 74%|███████▍  | 165990/225000 [320:09:58<87:43:09,  5.35s/it]
 74%|███████▍  | 165991/225000 [320:10:06<102:55:38,  6.28s/it]
 74%|███████▍  | 165992/225000 [320:10:11<93:42:05,  5.72s/it] 
 74%|███████▍  | 165993/225000 [320:10:15<86:20:47,  5.27s/it]
 74%|███████▍  | 165994/225000 [320:10:20<83:14:49,  5.08s/it]
 74%|███████▍  | 165995/225000 [320:10:24<77:06:17,  4.70s/it]
 74%|███████▍  | 165996/225000 [320:10:30<85:34:58,  5.22s/it]
 74%|███████▍  | 165997/225000 [320:10:38<101:30:15,  6.19s/it]
 74%|███████▍  | 165998/225000 [320:10:43<94:41:36,  5.78s/it] 
 74%|███████▍  | 165999/225000 [320:10:48<89:45:21,  5.48s/it]
 74%|███████▍  | 166000/225000 [320:10:53<89:11:37,  5.44s/it]
                                                              
{'loss': '0.7811', 'grad_norm': '1.139', 'learning_rate': '8.75e-06', 'epoch': '0.7378', 'num_input_tokens_seen': 530132640, 'train_runtime': '1.153e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166000/225000 [320:10:53<89:11:37,  5.44s/it]
 74%|███████▍  | 166001/225000 [320:10:58<85:52:39,  5.24s/it]
 74%|███████▍  | 166002/225000 [320:11:03<83:42:06,  5.11s/it]
 74%|███████▍  | 166003/225000 [320:11:08<82:09:36,  5.01s/it]
 74%|███████▍  | 166004/225000 [320:11:12<80:42:49,  4.93s/it]
 74%|███████▍  | 166005/225000 [320:11:17<77:54:39,  4.75s/it]
 74%|███████▍  | 166006/225000 [320:11:22<78:39:48,  4.80s/it]
 74%|███████▍  | 166007/225000 [320:11:34<114:08:27,  6.97s/it]
 74%|███████▍  | 166008/225000 [320:11:38<103:26:44,  6.31s/it]
 74%|███████▍  | 166009/225000 [320:11:43<92:18:06,  5.63s/it] 
 74%|███████▍  | 166010/225000 [320:11:47<86:16:10,  5.26s/it]
 74%|███████▍  | 166011/225000 [320:11:51<80:11:46,  4.89s/it]
 74%|███████▍  | 166012/225000 [320:11:55<77:59:44,  4.76s/it]
 74%|███████▍  | 166013/225000 [320:12:04<99:12:15,  6.05s/it]
 74%|███████▍  | 166014/225000 [320:12:11<102:38:11,  6.26s/it]
 74%|███████▍  | 166015/225000 [320:12:15<89:56:19,  5.49s/it] 
 74%|███████▍  | 166016/225000 [320:12:18<80:22:32,  4.91s/it]
 74%|███████▍  | 166017/225000 [320:12:26<94:54:34,  5.79s/it]
 74%|███████▍  | 166018/225000 [320:12:33<97:06:09,  5.93s/it]
 74%|███████▍  | 166019/225000 [320:12:36<86:16:31,  5.27s/it]
 74%|███████▍  | 166020/225000 [320:12:41<81:34:38,  4.98s/it]
 74%|███████▍  | 166021/225000 [320:12:47<90:24:24,  5.52s/it]
 74%|███████▍  | 166022/225000 [320:12:57<109:02:42,  6.66s/it]
 74%|███████▍  | 166023/225000 [320:13:00<93:20:39,  5.70s/it] 
 74%|███████▍  | 166024/225000 [320:13:05<89:56:52,  5.49s/it]
 74%|███████▍  | 166025/225000 [320:13:11<89:45:14,  5.48s/it]
 74%|███████▍  | 166026/225000 [320:13:16<88:28:13,  5.40s/it]
 74%|███████▍  | 166027/225000 [320:13:21<88:13:12,  5.39s/it]
 74%|███████▍  | 166028/225000 [320:13:27<89:21:47,  5.46s/it]
 74%|███████▍  | 166029/225000 [320:13:32<88:50:40,  5.42s/it]
 74%|███████▍  | 166030/225000 [320:13:37<85:39:09,  5.23s/it]
 74%|███████▍  | 166031/225000 [320:13:42<84:25:47,  5.15s/it]
 74%|███████▍  | 166032/225000 [320:13:47<83:12:08,  5.08s/it]
 74%|███████▍  | 166033/225000 [320:13:52<85:46:29,  5.24s/it]
 74%|███████▍  | 166034/225000 [320:13:57<84:00:50,  5.13s/it]
 74%|███████▍  | 166035/225000 [320:14:02<81:35:56,  4.98s/it]
 74%|███████▍  | 166036/225000 [320:14:07<82:40:37,  5.05s/it]
 74%|███████▍  | 166037/225000 [320:14:12<83:11:31,  5.08s/it]
 74%|███████▍  | 166038/225000 [320:14:17<83:38:55,  5.11s/it]
 74%|███████▍  | 166039/225000 [320:14:23<85:48:51,  5.24s/it]
 74%|███████▍  | 166040/225000 [320:14:28<86:24:22,  5.28s/it]
 74%|███████▍  | 166041/225000 [320:14:33<85:23:40,  5.21s/it]
 74%|███████▍  | 166042/225000 [320:14:39<85:49:16,  5.24s/it]
 74%|███████▍  | 166043/225000 [320:14:44<85:39:54,  5.23s/it]
 74%|███████▍  | 166044/225000 [320:14:52<100:40:35,  6.15s/it]
 74%|███████▍  | 166045/225000 [320:14:57<95:15:19,  5.82s/it] 
 74%|███████▍  | 166046/225000 [320:15:03<92:38:43,  5.66s/it]
 74%|███████▍  | 166047/225000 [320:15:08<91:59:40,  5.62s/it]
 74%|███████▍  | 166048/225000 [320:15:13<90:12:56,  5.51s/it]
 74%|███████▍  | 166049/225000 [320:15:19<91:04:17,  5.56s/it]
 74%|███████▍  | 166050/225000 [320:15:24<89:35:53,  5.47s/it]
                                                              
{'loss': '0.7829', 'grad_norm': '1.103', 'learning_rate': '8.737e-06', 'epoch': '0.738', 'num_input_tokens_seen': 530261672, 'train_runtime': '1.153e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166050/225000 [320:15:24<89:35:53,  5.47s/it]
 74%|███████▍  | 166051/225000 [320:15:30<90:08:56,  5.51s/it]
 74%|███████▍  | 166052/225000 [320:15:35<89:21:53,  5.46s/it]
 74%|███████▍  | 166053/225000 [320:15:43<101:06:26,  6.17s/it]
 74%|███████▍  | 166054/225000 [320:15:48<93:48:09,  5.73s/it] 
 74%|███████▍  | 166055/225000 [320:15:53<93:43:29,  5.72s/it]
 74%|███████▍  | 166056/225000 [320:15:59<93:05:35,  5.69s/it]
 74%|███████▍  | 166057/225000 [320:16:05<92:50:18,  5.67s/it]
 74%|███████▍  | 166058/225000 [320:16:10<91:16:42,  5.58s/it]
 74%|███████▍  | 166059/225000 [320:16:22<122:18:31,  7.47s/it]
 74%|███████▍  | 166060/225000 [320:16:26<107:41:44,  6.58s/it]
 74%|███████▍  | 166061/225000 [320:16:30<95:15:47,  5.82s/it] 
 74%|███████▍  | 166062/225000 [320:16:35<87:07:52,  5.32s/it]
 74%|███████▍  | 166063/225000 [320:16:39<83:46:17,  5.12s/it]
 74%|███████▍  | 166064/225000 [320:16:49<106:08:20,  6.48s/it]
 74%|███████▍  | 166065/225000 [320:16:53<94:07:43,  5.75s/it] 
 74%|███████▍  | 166066/225000 [320:16:57<85:43:35,  5.24s/it]
 74%|███████▍  | 166067/225000 [320:17:01<80:20:15,  4.91s/it]
 74%|███████▍  | 166068/225000 [320:17:08<89:59:11,  5.50s/it]
 74%|███████▍  | 166069/225000 [320:17:18<111:04:40,  6.79s/it]
 74%|███████▍  | 166070/225000 [320:17:22<99:47:02,  6.10s/it] 
 74%|███████▍  | 166071/225000 [320:17:27<93:52:57,  5.74s/it]
 74%|███████▍  | 166072/225000 [320:17:32<88:29:15,  5.41s/it]
 74%|███████▍  | 166073/225000 [320:17:36<84:40:49,  5.17s/it]
 74%|███████▍  | 166074/225000 [320:17:41<79:39:37,  4.87s/it]
 74%|███████▍  | 166075/225000 [320:17:45<77:41:41,  4.75s/it]
 74%|███████▍  | 166076/225000 [320:17:50<78:01:22,  4.77s/it]
 74%|███████▍  | 166077/225000 [320:17:55<78:24:11,  4.79s/it]
 74%|███████▍  | 166078/225000 [320:18:00<78:45:35,  4.81s/it]
 74%|███████▍  | 166079/225000 [320:18:07<92:36:31,  5.66s/it]
 74%|███████▍  | 166080/225000 [320:18:19<122:28:09,  7.48s/it]
 74%|███████▍  | 166081/225000 [320:18:23<105:22:11,  6.44s/it]
 74%|███████▍  | 166082/225000 [320:18:26<89:36:21,  5.48s/it] 
 74%|███████▍  | 166083/225000 [320:18:31<86:18:31,  5.27s/it]
 74%|███████▍  | 166084/225000 [320:18:36<83:47:39,  5.12s/it]
 74%|███████▍  | 166085/225000 [320:18:41<83:48:55,  5.12s/it]
 74%|███████▍  | 166086/225000 [320:18:46<84:38:36,  5.17s/it]
 74%|███████▍  | 166087/225000 [320:18:51<83:05:42,  5.08s/it]
 74%|███████▍  | 166088/225000 [320:18:56<83:05:01,  5.08s/it]
 74%|███████▍  | 166089/225000 [320:19:01<80:30:26,  4.92s/it]
 74%|███████▍  | 166090/225000 [320:19:11<107:49:55,  6.59s/it]
 74%|███████▍  | 166091/225000 [320:19:19<111:37:20,  6.82s/it]
 74%|███████▍  | 166092/225000 [320:19:27<117:24:49,  7.18s/it]
 74%|███████▍  | 166093/225000 [320:19:30<99:18:16,  6.07s/it] 
 74%|███████▍  | 166094/225000 [320:19:35<94:19:36,  5.76s/it]
 74%|███████▍  | 166095/225000 [320:19:42<99:09:50,  6.06s/it]
 74%|███████▍  | 166096/225000 [320:19:46<91:55:56,  5.62s/it]
 74%|███████▍  | 166097/225000 [320:19:51<85:07:26,  5.20s/it]
 74%|███████▍  | 166098/225000 [320:19:55<78:34:23,  4.80s/it]
 74%|███████▍  | 166099/225000 [320:20:04<101:45:40,  6.22s/it]
 74%|███████▍  | 166100/225000 [320:20:09<93:29:16,  5.71s/it] 
                                                              
{'loss': '0.7778', 'grad_norm': '1.138', 'learning_rate': '8.723e-06', 'epoch': '0.7382', 'num_input_tokens_seen': 530391152, 'train_runtime': '1.153e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166100/225000 [320:20:09<93:29:16,  5.71s/it]
 74%|███████▍  | 166101/225000 [320:20:12<82:53:52,  5.07s/it]
 74%|███████▍  | 166102/225000 [320:20:21<100:42:00,  6.16s/it]
 74%|███████▍  | 166103/225000 [320:20:25<88:33:46,  5.41s/it] 
 74%|███████▍  | 166104/225000 [320:20:30<90:29:13,  5.53s/it]
 74%|███████▍  | 166105/225000 [320:20:38<98:55:07,  6.05s/it]
 74%|███████▍  | 166106/225000 [320:20:41<86:01:35,  5.26s/it]
 74%|███████▍  | 166107/225000 [320:20:45<80:25:53,  4.92s/it]
 74%|███████▍  | 166108/225000 [320:20:49<76:21:31,  4.67s/it]
 74%|███████▍  | 166109/225000 [320:20:58<96:56:34,  5.93s/it]
 74%|███████▍  | 166110/225000 [320:21:03<91:00:44,  5.56s/it]
 74%|███████▍  | 166111/225000 [320:21:07<84:37:04,  5.17s/it]
 74%|███████▍  | 166112/225000 [320:21:11<80:02:31,  4.89s/it]
 74%|███████▍  | 166113/225000 [320:21:16<77:48:57,  4.76s/it]
 74%|███████▍  | 166114/225000 [320:21:20<76:44:02,  4.69s/it]
 74%|███████▍  | 166115/225000 [320:21:32<112:41:51,  6.89s/it]
 74%|███████▍  | 166116/225000 [320:21:36<95:35:13,  5.84s/it] 
 74%|███████▍  | 166117/225000 [320:21:39<84:11:23,  5.15s/it]
 74%|███████▍  | 166118/225000 [320:21:46<93:48:34,  5.74s/it]
 74%|███████▍  | 166119/225000 [320:21:50<84:17:21,  5.15s/it]
 74%|███████▍  | 166120/225000 [320:21:54<78:30:53,  4.80s/it]
 74%|███████▍  | 166121/225000 [320:21:58<74:14:30,  4.54s/it]
 74%|███████▍  | 166122/225000 [320:22:04<81:12:43,  4.97s/it]
 74%|███████▍  | 166123/225000 [320:22:13<103:15:22,  6.31s/it]
 74%|███████▍  | 166124/225000 [320:22:17<90:48:38,  5.55s/it] 
 74%|███████▍  | 166125/225000 [320:22:22<85:24:50,  5.22s/it]
 74%|███████▍  | 166126/225000 [320:22:32<110:33:21,  6.76s/it]
 74%|███████▍  | 166127/225000 [320:22:37<99:32:29,  6.09s/it] 
 74%|███████▍  | 166128/225000 [320:22:41<91:36:25,  5.60s/it]
 74%|███████▍  | 166129/225000 [320:22:46<86:49:40,  5.31s/it]
 74%|███████▍  | 166130/225000 [320:22:50<84:16:50,  5.15s/it]
 74%|███████▍  | 166131/225000 [320:22:55<80:50:23,  4.94s/it]
 74%|███████▍  | 166132/225000 [320:23:00<79:35:24,  4.87s/it]
 74%|███████▍  | 166133/225000 [320:23:06<87:43:29,  5.36s/it]
 74%|███████▍  | 166134/225000 [320:23:17<113:30:45,  6.94s/it]
 74%|███████▍  | 166135/225000 [320:23:21<98:09:10,  6.00s/it] 
 74%|███████▍  | 166136/225000 [320:23:29<112:44:02,  6.89s/it]
 74%|███████▍  | 166137/225000 [320:23:33<94:30:34,  5.78s/it] 
 74%|███████▍  | 166138/225000 [320:23:37<87:51:38,  5.37s/it]
 74%|███████▍  | 166139/225000 [320:23:42<84:48:35,  5.19s/it]
 74%|███████▍  | 166140/225000 [320:23:47<82:28:20,  5.04s/it]
 74%|███████▍  | 166141/225000 [320:23:51<80:08:11,  4.90s/it]
 74%|███████▍  | 166142/225000 [320:23:55<77:17:12,  4.73s/it]
 74%|███████▍  | 166143/225000 [320:24:00<77:38:48,  4.75s/it]
 74%|███████▍  | 166144/225000 [320:24:12<111:33:35,  6.82s/it]
 74%|███████▍  | 166145/225000 [320:24:16<96:07:17,  5.88s/it] 
 74%|███████▍  | 166146/225000 [320:24:21<93:03:38,  5.69s/it]
 74%|███████▍  | 166147/225000 [320:24:30<108:17:42,  6.62s/it]
 74%|███████▍  | 166148/225000 [320:24:33<92:18:33,  5.65s/it] 
 74%|███████▍  | 166149/225000 [320:24:38<87:32:15,  5.35s/it]
 74%|███████▍  | 166150/225000 [320:24:42<83:14:15,  5.09s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.286', 'learning_rate': '8.709e-06', 'epoch': '0.7384', 'num_input_tokens_seen': 530519600, 'train_runtime': '1.153e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166150/225000 [320:24:42<83:14:15,  5.09s/it]
 74%|███████▍  | 166151/225000 [320:24:47<79:42:39,  4.88s/it]
 74%|███████▍  | 166152/225000 [320:24:51<77:07:02,  4.72s/it]
 74%|███████▍  | 166153/225000 [320:24:56<77:59:55,  4.77s/it]
 74%|███████▍  | 166154/225000 [320:25:06<103:15:21,  6.32s/it]
 74%|███████▍  | 166155/225000 [320:25:11<100:34:19,  6.15s/it]
 74%|███████▍  | 166156/225000 [320:25:18<102:57:26,  6.30s/it]
 74%|███████▍  | 166157/225000 [320:25:22<93:09:40,  5.70s/it] 
 74%|███████▍  | 166158/225000 [320:25:28<94:01:42,  5.75s/it]
 74%|███████▍  | 166159/225000 [320:25:34<95:40:46,  5.85s/it]
 74%|███████▍  | 166160/225000 [320:25:42<102:40:01,  6.28s/it]
 74%|███████▍  | 166161/225000 [320:25:46<92:25:17,  5.65s/it] 
 74%|███████▍  | 166162/225000 [320:25:53<99:38:59,  6.10s/it]
 74%|███████▍  | 166163/225000 [320:25:56<86:13:24,  5.28s/it]
 74%|███████▍  | 166164/225000 [320:26:03<93:54:46,  5.75s/it]
 74%|███████▍  | 166165/225000 [320:26:07<84:46:12,  5.19s/it]
 74%|███████▍  | 166166/225000 [320:26:11<77:32:03,  4.74s/it]
 74%|███████▍  | 166167/225000 [320:26:19<94:47:22,  5.80s/it]
 74%|███████▍  | 166168/225000 [320:26:25<97:04:45,  5.94s/it]
 74%|███████▍  | 166169/225000 [320:26:31<93:27:14,  5.72s/it]
 74%|███████▍  | 166170/225000 [320:26:35<88:32:57,  5.42s/it]
 74%|███████▍  | 166171/225000 [320:26:40<85:22:02,  5.22s/it]
 74%|███████▍  | 166172/225000 [320:26:45<82:22:49,  5.04s/it]
 74%|███████▍  | 166173/225000 [320:26:49<80:55:08,  4.95s/it]
 74%|███████▍  | 166174/225000 [320:26:54<81:08:53,  4.97s/it]
 74%|███████▍  | 166175/225000 [320:26:59<80:28:37,  4.93s/it]
 74%|███████▍  | 166176/225000 [320:27:04<81:41:40,  5.00s/it]
 74%|███████▍  | 166177/225000 [320:27:15<110:40:35,  6.77s/it]
 74%|███████▍  | 166178/225000 [320:27:20<99:14:20,  6.07s/it] 
 74%|███████▍  | 166179/225000 [320:27:24<90:16:01,  5.52s/it]
 74%|███████▍  | 166180/225000 [320:27:29<86:19:52,  5.28s/it]
 74%|███████▍  | 166181/225000 [320:27:33<82:53:25,  5.07s/it]
 74%|███████▍  | 166182/225000 [320:27:38<81:59:22,  5.02s/it]
 74%|███████▍  | 166183/225000 [320:27:43<83:20:58,  5.10s/it]
 74%|███████▍  | 166184/225000 [320:27:48<82:35:43,  5.06s/it]
 74%|███████▍  | 166185/225000 [320:28:00<114:12:43,  6.99s/it]
 74%|███████▍  | 166186/225000 [320:28:05<104:05:02,  6.37s/it]
 74%|███████▍  | 166187/225000 [320:28:12<105:39:40,  6.47s/it]
 74%|███████▍  | 166188/225000 [320:28:16<96:18:33,  5.90s/it] 
 74%|███████▍  | 166189/225000 [320:28:20<86:11:52,  5.28s/it]
 74%|███████▍  | 166190/225000 [320:28:25<84:34:14,  5.18s/it]
 74%|███████▍  | 166191/225000 [320:28:30<82:53:36,  5.07s/it]
 74%|███████▍  | 166192/225000 [320:28:34<79:06:56,  4.84s/it]
 74%|███████▍  | 166193/225000 [320:28:46<115:11:27,  7.05s/it]
 74%|███████▍  | 166194/225000 [320:28:50<99:56:27,  6.12s/it] 
 74%|███████▍  | 166195/225000 [320:28:54<87:43:44,  5.37s/it]
 74%|███████▍  | 166196/225000 [320:29:00<94:06:17,  5.76s/it]
 74%|███████▍  | 166197/225000 [320:29:08<103:23:43,  6.33s/it]
 74%|███████▍  | 166198/225000 [320:29:12<90:11:04,  5.52s/it] 
 74%|███████▍  | 166199/225000 [320:29:19<98:50:18,  6.05s/it]
 74%|███████▍  | 166200/225000 [320:29:23<90:02:59,  5.51s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.126', 'learning_rate': '8.695e-06', 'epoch': '0.7387', 'num_input_tokens_seen': 530649840, 'train_runtime': '1.154e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166200/225000 [320:29:23<90:02:59,  5.51s/it]
 74%|███████▍  | 166201/225000 [320:29:28<86:01:46,  5.27s/it]
 74%|███████▍  | 166202/225000 [320:29:32<81:41:22,  5.00s/it]
 74%|███████▍  | 166203/225000 [320:29:37<80:27:17,  4.93s/it]
 74%|███████▍  | 166204/225000 [320:29:42<79:58:15,  4.90s/it]
 74%|███████▍  | 166205/225000 [320:29:47<81:24:30,  4.98s/it]
 74%|███████▍  | 166206/225000 [320:29:58<111:04:42,  6.80s/it]
 74%|███████▍  | 166207/225000 [320:30:02<95:40:33,  5.86s/it] 
 74%|███████▍  | 166208/225000 [320:30:06<85:46:31,  5.25s/it]
 74%|███████▍  | 166209/225000 [320:30:16<108:37:14,  6.65s/it]
 74%|███████▍  | 166210/225000 [320:30:21<100:21:40,  6.15s/it]
 74%|███████▍  | 166211/225000 [320:30:24<88:52:07,  5.44s/it] 
 74%|███████▍  | 166212/225000 [320:30:28<80:57:37,  4.96s/it]
 74%|███████▍  | 166213/225000 [320:30:38<103:04:22,  6.31s/it]
 74%|███████▍  | 166214/225000 [320:30:42<91:08:44,  5.58s/it] 
 74%|███████▍  | 166215/225000 [320:30:45<82:14:59,  5.04s/it]
 74%|███████▍  | 166216/225000 [320:30:55<102:47:58,  6.30s/it]
 74%|███████▍  | 166217/225000 [320:30:58<90:15:21,  5.53s/it] 
 74%|███████▍  | 166218/225000 [320:31:06<99:53:57,  6.12s/it]
 74%|███████▍  | 166219/225000 [320:31:09<87:15:14,  5.34s/it]
 74%|███████▍  | 166220/225000 [320:31:13<79:01:01,  4.84s/it]
 74%|███████▍  | 166221/225000 [320:31:22<100:12:10,  6.14s/it]
 74%|███████▍  | 166222/225000 [320:31:25<86:37:09,  5.31s/it] 
 74%|███████▍  | 166223/225000 [320:31:34<100:20:36,  6.15s/it]
 74%|███████▍  | 166224/225000 [320:31:37<89:22:10,  5.47s/it] 
 74%|███████▍  | 166225/225000 [320:31:41<80:29:22,  4.93s/it]
 74%|███████▍  | 166226/225000 [320:31:50<100:13:09,  6.14s/it]
 74%|███████▍  | 166227/225000 [320:31:54<87:16:51,  5.35s/it] 
 74%|███████▍  | 166228/225000 [320:32:00<94:39:32,  5.80s/it]
 74%|███████▍  | 166229/225000 [320:32:04<82:07:07,  5.03s/it]
 74%|███████▍  | 166230/225000 [320:32:11<94:58:48,  5.82s/it]
 74%|███████▍  | 166231/225000 [320:32:18<99:08:44,  6.07s/it]
 74%|███████▍  | 166232/225000 [320:32:22<87:58:24,  5.39s/it]
 74%|███████▍  | 166233/225000 [320:32:29<98:02:41,  6.01s/it]
 74%|███████▍  | 166234/225000 [320:32:34<90:00:12,  5.51s/it]
 74%|███████▍  | 166235/225000 [320:32:38<85:07:01,  5.21s/it]
 74%|███████▍  | 166236/225000 [320:32:42<80:22:14,  4.92s/it]
 74%|███████▍  | 166237/225000 [320:32:47<76:52:29,  4.71s/it]
 74%|███████▍  | 166238/225000 [320:32:52<79:34:06,  4.87s/it]
 74%|███████▍  | 166239/225000 [320:33:03<112:35:52,  6.90s/it]
 74%|███████▍  | 166240/225000 [320:33:07<93:58:14,  5.76s/it] 
 74%|███████▍  | 166241/225000 [320:33:14<101:45:12,  6.23s/it]
 74%|███████▍  | 166242/225000 [320:33:19<96:29:28,  5.91s/it] 
 74%|███████▍  | 166243/225000 [320:33:24<91:21:12,  5.60s/it]
 74%|███████▍  | 166244/225000 [320:33:29<89:42:09,  5.50s/it]
 74%|███████▍  | 166245/225000 [320:33:34<85:43:52,  5.25s/it]
 74%|███████▍  | 166246/225000 [320:33:39<83:49:21,  5.14s/it]
 74%|███████▍  | 166247/225000 [320:33:44<83:06:49,  5.09s/it]
 74%|███████▍  | 166248/225000 [320:33:49<85:43:22,  5.25s/it]
 74%|███████▍  | 166249/225000 [320:33:54<83:59:41,  5.15s/it]
 74%|███████▍  | 166250/225000 [320:33:59<84:12:55,  5.16s/it]
                                                              
{'loss': '0.7697', 'grad_norm': '1.172', 'learning_rate': '8.681e-06', 'epoch': '0.7389', 'num_input_tokens_seen': 530779472, 'train_runtime': '1.154e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166250/225000 [320:33:59<84:12:55,  5.16s/it]
 74%|███████▍  | 166251/225000 [320:34:04<82:28:07,  5.05s/it]
 74%|███████▍  | 166252/225000 [320:34:12<96:27:19,  5.91s/it]
 74%|███████▍  | 166253/225000 [320:34:20<104:53:43,  6.43s/it]
 74%|███████▍  | 166254/225000 [320:34:23<91:28:20,  5.61s/it] 
 74%|███████▍  | 166255/225000 [320:34:32<106:35:59,  6.53s/it]
 74%|███████▍  | 166256/225000 [320:34:35<90:27:58,  5.54s/it] 
 74%|███████▍  | 166257/225000 [320:34:43<102:44:18,  6.30s/it]
 74%|███████▍  | 166258/225000 [320:34:48<93:44:14,  5.74s/it] 
 74%|███████▍  | 166259/225000 [320:34:52<84:28:05,  5.18s/it]
 74%|███████▍  | 166260/225000 [320:34:56<80:58:55,  4.96s/it]
 74%|███████▍  | 166261/225000 [320:35:00<77:16:53,  4.74s/it]
 74%|███████▍  | 166262/225000 [320:35:05<74:43:12,  4.58s/it]
 74%|███████▍  | 166263/225000 [320:35:09<74:15:10,  4.55s/it]
 74%|███████▍  | 166264/225000 [320:35:21<110:34:40,  6.78s/it]
 74%|███████▍  | 166265/225000 [320:35:26<101:00:12,  6.19s/it]
 74%|███████▍  | 166266/225000 [320:35:30<89:01:44,  5.46s/it] 
 74%|███████▍  | 166267/225000 [320:35:35<90:06:56,  5.52s/it]
 74%|███████▍  | 166268/225000 [320:35:41<88:57:04,  5.45s/it]
 74%|███████▍  | 166269/225000 [320:35:46<89:01:14,  5.46s/it]
 74%|███████▍  | 166270/225000 [320:35:54<98:28:38,  6.04s/it]
 74%|███████▍  | 166271/225000 [320:36:00<100:55:41,  6.19s/it]
 74%|███████▍  | 166272/225000 [320:36:07<103:39:29,  6.35s/it]
 74%|███████▍  | 166273/225000 [320:36:10<88:23:55,  5.42s/it] 
 74%|███████▍  | 166274/225000 [320:36:14<79:33:20,  4.88s/it]
 74%|███████▍  | 166275/225000 [320:36:20<89:08:07,  5.46s/it]
 74%|███████▍  | 166276/225000 [320:36:27<95:57:51,  5.88s/it]
 74%|███████▍  | 166277/225000 [320:36:31<85:13:25,  5.22s/it]
 74%|███████▍  | 166278/225000 [320:36:39<97:49:34,  6.00s/it]
 74%|███████▍  | 166279/225000 [320:36:45<96:58:57,  5.95s/it]
 74%|███████▍  | 166280/225000 [320:36:48<85:34:39,  5.25s/it]
 74%|███████▍  | 166281/225000 [320:36:57<103:44:48,  6.36s/it]
 74%|███████▍  | 166282/225000 [320:37:01<91:35:31,  5.62s/it] 
 74%|███████▍  | 166283/225000 [320:37:05<82:42:32,  5.07s/it]
 74%|███████▍  | 166284/225000 [320:37:10<82:42:59,  5.07s/it]
 74%|███████▍  | 166285/225000 [320:37:19<102:22:21,  6.28s/it]
 74%|███████▍  | 166286/225000 [320:37:22<87:30:56,  5.37s/it] 
 74%|███████▍  | 166287/225000 [320:37:31<103:17:26,  6.33s/it]
 74%|███████▍  | 166288/225000 [320:37:34<89:45:07,  5.50s/it] 
 74%|███████▍  | 166289/225000 [320:37:39<85:18:01,  5.23s/it]
 74%|███████▍  | 166290/225000 [320:37:43<80:07:20,  4.91s/it]
 74%|███████▍  | 166291/225000 [320:37:53<103:47:12,  6.36s/it]
 74%|███████▍  | 166292/225000 [320:37:57<89:57:05,  5.52s/it] 
 74%|███████▍  | 166293/225000 [320:38:00<82:12:26,  5.04s/it]
 74%|███████▍  | 166294/225000 [320:38:04<74:45:34,  4.58s/it]
 74%|███████▍  | 166295/225000 [320:38:15<106:40:30,  6.54s/it]
 74%|███████▍  | 166296/225000 [320:38:20<98:19:19,  6.03s/it] 
 74%|███████▍  | 166297/225000 [320:38:24<91:13:42,  5.59s/it]
 74%|███████▍  | 166298/225000 [320:38:29<84:55:26,  5.21s/it]
 74%|███████▍  | 166299/225000 [320:38:33<81:31:05,  5.00s/it]
 74%|███████▍  | 166300/225000 [320:38:38<80:09:12,  4.92s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.19', 'learning_rate': '8.667e-06', 'epoch': '0.7391', 'num_input_tokens_seen': 530908752, 'train_runtime': '1.154e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166300/225000 [320:38:38<80:09:12,  4.92s/it]
 74%|███████▍  | 166301/225000 [320:38:43<80:26:31,  4.93s/it]
 74%|███████▍  | 166302/225000 [320:38:47<77:57:55,  4.78s/it]
 74%|███████▍  | 166303/225000 [320:38:52<76:04:58,  4.67s/it]
 74%|███████▍  | 166304/225000 [320:38:57<76:07:44,  4.67s/it]
 74%|███████▍  | 166305/225000 [320:39:03<85:28:15,  5.24s/it]
 74%|███████▍  | 166306/225000 [320:39:14<115:27:40,  7.08s/it]
 74%|███████▍  | 166307/225000 [320:39:19<102:12:35,  6.27s/it]
 74%|███████▍  | 166308/225000 [320:39:23<93:48:34,  5.75s/it] 
 74%|███████▍  | 166309/225000 [320:39:28<86:48:12,  5.32s/it]
 74%|███████▍  | 166310/225000 [320:39:32<82:59:43,  5.09s/it]
 74%|███████▍  | 166311/225000 [320:39:40<96:41:28,  5.93s/it]
 74%|███████▍  | 166312/225000 [320:39:48<104:52:25,  6.43s/it]
 74%|███████▍  | 166313/225000 [320:39:52<95:13:55,  5.84s/it] 
 74%|███████▍  | 166314/225000 [320:39:57<88:08:17,  5.41s/it]
 74%|███████▍  | 166315/225000 [320:40:01<83:12:12,  5.10s/it]
 74%|███████▍  | 166316/225000 [320:40:10<100:10:21,  6.15s/it]
 74%|███████▍  | 166317/225000 [320:40:17<107:20:12,  6.58s/it]
 74%|███████▍  | 166318/225000 [320:40:21<91:27:31,  5.61s/it] 
 74%|███████▍  | 166319/225000 [320:40:29<105:18:09,  6.46s/it]
 74%|███████▍  | 166320/225000 [320:40:33<92:25:28,  5.67s/it] 
 74%|███████▍  | 166321/225000 [320:40:37<84:58:48,  5.21s/it]
 74%|███████▍  | 166322/225000 [320:40:41<79:38:40,  4.89s/it]
 74%|███████▍  | 166323/225000 [320:40:45<75:21:02,  4.62s/it]
 74%|███████▍  | 166324/225000 [320:40:56<105:31:50,  6.47s/it]
 74%|███████▍  | 166325/225000 [320:40:59<90:33:09,  5.56s/it] 
 74%|███████▍  | 166326/225000 [320:41:04<86:05:50,  5.28s/it]
 74%|███████▍  | 166327/225000 [320:41:11<95:48:02,  5.88s/it]
 74%|███████▍  | 166328/225000 [320:41:15<85:31:24,  5.25s/it]
 74%|███████▍  | 166329/225000 [320:41:18<76:01:32,  4.66s/it]
 74%|███████▍  | 166330/225000 [320:41:27<96:30:30,  5.92s/it]
 74%|███████▍  | 166331/225000 [320:41:30<83:56:13,  5.15s/it]
 74%|███████▍  | 166332/225000 [320:41:35<78:38:14,  4.83s/it]
 74%|███████▍  | 166333/225000 [320:41:43<96:14:46,  5.91s/it]
 74%|███████▍  | 166334/225000 [320:41:46<84:14:09,  5.17s/it]
 74%|███████▍  | 166335/225000 [320:41:52<84:38:57,  5.19s/it]
 74%|███████▍  | 166336/225000 [320:41:57<84:39:34,  5.20s/it]
 74%|███████▍  | 166337/225000 [320:42:03<88:09:22,  5.41s/it]
 74%|███████▍  | 166338/225000 [320:42:08<88:13:08,  5.41s/it]
 74%|███████▍  | 166339/225000 [320:42:14<88:16:37,  5.42s/it]
 74%|███████▍  | 166340/225000 [320:42:19<87:21:16,  5.36s/it]
 74%|███████▍  | 166341/225000 [320:42:24<87:02:51,  5.34s/it]
 74%|███████▍  | 166342/225000 [320:42:30<87:23:41,  5.36s/it]
 74%|███████▍  | 166343/225000 [320:42:35<88:06:03,  5.41s/it]
 74%|███████▍  | 166344/225000 [320:42:40<86:53:45,  5.33s/it]
 74%|███████▍  | 166345/225000 [320:42:46<86:42:37,  5.32s/it]
 74%|███████▍  | 166346/225000 [320:42:51<86:45:02,  5.32s/it]
 74%|███████▍  | 166347/225000 [320:42:56<86:18:59,  5.30s/it]
 74%|███████▍  | 166348/225000 [320:43:02<87:15:16,  5.36s/it]
 74%|███████▍  | 166349/225000 [320:43:07<87:51:28,  5.39s/it]
 74%|███████▍  | 166350/225000 [320:43:12<87:51:02,  5.39s/it]
                                                              
{'loss': '0.7699', 'grad_norm': '1.093', 'learning_rate': '8.654e-06', 'epoch': '0.7393', 'num_input_tokens_seen': 531038256, 'train_runtime': '1.155e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166350/225000 [320:43:12<87:51:02,  5.39s/it]
 74%|███████▍  | 166351/225000 [320:43:24<116:57:18,  7.18s/it]
 74%|███████▍  | 166352/225000 [320:43:27<99:15:22,  6.09s/it] 
 74%|███████▍  | 166353/225000 [320:43:32<91:17:22,  5.60s/it]
 74%|███████▍  | 166354/225000 [320:43:41<110:15:40,  6.77s/it]
 74%|███████▍  | 166355/225000 [320:43:48<108:35:55,  6.67s/it]
 74%|███████▍  | 166356/225000 [320:43:52<97:54:59,  6.01s/it] 
 74%|███████▍  | 166357/225000 [320:43:57<89:34:22,  5.50s/it]
 74%|███████▍  | 166358/225000 [320:44:01<82:28:42,  5.06s/it]
 74%|███████▍  | 166359/225000 [320:44:05<78:22:56,  4.81s/it]
 74%|███████▍  | 166360/225000 [320:44:11<85:41:25,  5.26s/it]
 74%|███████▍  | 166361/225000 [320:44:20<105:34:04,  6.48s/it]
 74%|███████▍  | 166362/225000 [320:44:24<90:30:12,  5.56s/it] 
 74%|███████▍  | 166363/225000 [320:44:27<80:50:17,  4.96s/it]
 74%|███████▍  | 166364/225000 [320:44:36<100:03:49,  6.14s/it]
 74%|███████▍  | 166365/225000 [320:44:44<109:11:19,  6.70s/it]
 74%|███████▍  | 166366/225000 [320:44:47<91:46:07,  5.63s/it] 
 74%|███████▍  | 166367/225000 [320:44:51<81:18:14,  4.99s/it]
 74%|███████▍  | 166368/225000 [320:44:54<72:52:30,  4.47s/it]
 74%|███████▍  | 166369/225000 [320:45:02<91:21:41,  5.61s/it]
 74%|███████▍  | 166370/225000 [320:45:06<82:09:06,  5.04s/it]
 74%|███████▍  | 166371/225000 [320:45:10<73:38:14,  4.52s/it]
 74%|███████▍  | 166372/225000 [320:45:18<92:43:04,  5.69s/it]
 74%|███████▍  | 166373/225000 [320:45:22<84:07:44,  5.17s/it]
 74%|███████▍  | 166374/225000 [320:45:26<79:24:39,  4.88s/it]
 74%|███████▍  | 166375/225000 [320:45:30<76:19:33,  4.69s/it]
 74%|███████▍  | 166376/225000 [320:45:35<75:35:59,  4.64s/it]
 74%|███████▍  | 166377/225000 [320:45:47<111:06:01,  6.82s/it]
 74%|███████▍  | 166378/225000 [320:45:50<93:17:32,  5.73s/it] 
 74%|███████▍  | 166379/225000 [320:45:53<81:12:31,  4.99s/it]
 74%|███████▍  | 166380/225000 [320:45:57<77:10:10,  4.74s/it]
 74%|███████▍  | 166381/225000 [320:46:02<75:35:51,  4.64s/it]
 74%|███████▍  | 166382/225000 [320:46:06<75:33:45,  4.64s/it]
 74%|███████▍  | 166383/225000 [320:46:12<77:46:42,  4.78s/it]
 74%|███████▍  | 166384/225000 [320:46:16<77:07:20,  4.74s/it]
 74%|███████▍  | 166385/225000 [320:46:21<76:02:30,  4.67s/it]
 74%|███████▍  | 166386/225000 [320:46:33<113:46:15,  6.99s/it]
 74%|███████▍  | 166387/225000 [320:46:39<107:07:07,  6.58s/it]
 74%|███████▍  | 166388/225000 [320:46:42<92:43:24,  5.70s/it] 
 74%|███████▍  | 166389/225000 [320:46:50<103:14:19,  6.34s/it]
 74%|███████▍  | 166390/225000 [320:46:55<96:41:57,  5.94s/it] 
 74%|███████▍  | 166391/225000 [320:46:59<85:52:21,  5.27s/it]
 74%|███████▍  | 166392/225000 [320:47:05<91:09:17,  5.60s/it]
 74%|███████▍  | 166393/225000 [320:47:12<97:20:15,  5.98s/it]
 74%|███████▍  | 166394/225000 [320:47:16<85:54:33,  5.28s/it]
 74%|███████▍  | 166395/225000 [320:47:21<86:49:25,  5.33s/it]
 74%|███████▍  | 166396/225000 [320:47:29<98:39:58,  6.06s/it]
 74%|███████▍  | 166397/225000 [320:47:32<85:27:43,  5.25s/it]
 74%|███████▍  | 166398/225000 [320:47:36<78:57:33,  4.85s/it]
 74%|███████▍  | 166399/225000 [320:47:41<76:01:35,  4.67s/it]
 74%|███████▍  | 166400/225000 [320:47:45<74:43:55,  4.59s/it]
                                                              
{'loss': '0.7651', 'grad_norm': '1.129', 'learning_rate': '8.64e-06', 'epoch': '0.7396', 'num_input_tokens_seen': 531166848, 'train_runtime': '1.155e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166400/225000 [320:47:45<74:43:55,  4.59s/it]
 74%|███████▍  | 166401/225000 [320:47:50<75:13:27,  4.62s/it]
 74%|███████▍  | 166402/225000 [320:47:58<93:36:38,  5.75s/it]
 74%|███████▍  | 166403/225000 [320:48:07<108:50:45,  6.69s/it]
 74%|███████▍  | 166404/225000 [320:48:10<93:39:39,  5.75s/it] 
 74%|███████▍  | 166405/225000 [320:48:14<83:11:53,  5.11s/it]
 74%|███████▍  | 166406/225000 [320:48:19<84:41:05,  5.20s/it]
 74%|███████▍  | 166407/225000 [320:48:24<83:39:56,  5.14s/it]
 74%|███████▍  | 166408/225000 [320:48:34<104:10:01,  6.40s/it]
 74%|███████▍  | 166409/225000 [320:48:37<90:12:38,  5.54s/it] 
 74%|███████▍  | 166410/225000 [320:48:44<97:16:18,  5.98s/it]
 74%|███████▍  | 166411/225000 [320:48:50<94:47:25,  5.82s/it]
 74%|███████▍  | 166412/225000 [320:48:56<96:08:51,  5.91s/it]
 74%|███████▍  | 166413/225000 [320:49:00<85:44:53,  5.27s/it]
 74%|███████▍  | 166414/225000 [320:49:07<95:06:54,  5.84s/it]
 74%|███████▍  | 166415/225000 [320:49:12<89:26:28,  5.50s/it]
 74%|███████▍  | 166416/225000 [320:49:15<78:12:28,  4.81s/it]
 74%|███████▍  | 166417/225000 [320:49:24<101:49:46,  6.26s/it]
 74%|███████▍  | 166418/225000 [320:49:27<85:59:33,  5.28s/it] 
 74%|███████▍  | 166419/225000 [320:49:32<83:50:13,  5.15s/it]
 74%|███████▍  | 166420/225000 [320:49:38<85:30:10,  5.25s/it]
 74%|███████▍  | 166421/225000 [320:49:43<83:29:40,  5.13s/it]
 74%|███████▍  | 166422/225000 [320:49:48<84:08:15,  5.17s/it]
 74%|███████▍  | 166423/225000 [320:49:53<82:47:12,  5.09s/it]
 74%|███████▍  | 166424/225000 [320:49:58<84:57:57,  5.22s/it]
 74%|███████▍  | 166425/225000 [320:50:03<83:14:29,  5.12s/it]
 74%|███████▍  | 166426/225000 [320:50:09<85:07:30,  5.23s/it]
 74%|███████▍  | 166427/225000 [320:50:14<84:40:20,  5.20s/it]
 74%|███████▍  | 166428/225000 [320:50:19<86:05:21,  5.29s/it]
 74%|███████▍  | 166429/225000 [320:50:25<85:53:31,  5.28s/it]
 74%|███████▍  | 166430/225000 [320:50:29<82:36:13,  5.08s/it]
 74%|███████▍  | 166431/225000 [320:50:35<84:51:03,  5.22s/it]
 74%|███████▍  | 166432/225000 [320:50:40<83:47:12,  5.15s/it]
 74%|███████▍  | 166433/225000 [320:50:45<85:35:34,  5.26s/it]
 74%|███████▍  | 166434/225000 [320:50:50<84:04:53,  5.17s/it]
 74%|███████▍  | 166435/225000 [320:51:02<118:37:04,  7.29s/it]
 74%|███████▍  | 166436/225000 [320:51:08<109:39:55,  6.74s/it]
 74%|███████▍  | 166437/225000 [320:51:12<98:19:33,  6.04s/it] 
 74%|███████▍  | 166438/225000 [320:51:16<88:53:05,  5.46s/it]
 74%|███████▍  | 166439/225000 [320:51:21<83:44:16,  5.15s/it]
 74%|███████▍  | 166440/225000 [320:51:30<105:13:28,  6.47s/it]
 74%|███████▍  | 166441/225000 [320:51:36<99:38:48,  6.13s/it] 
 74%|███████▍  | 166442/225000 [320:51:40<89:20:02,  5.49s/it]
 74%|███████▍  | 166443/225000 [320:51:44<82:07:24,  5.05s/it]
 74%|███████▍  | 166444/225000 [320:51:54<108:03:27,  6.64s/it]
 74%|███████▍  | 166445/225000 [320:51:59<99:47:37,  6.14s/it] 
 74%|███████▍  | 166446/225000 [320:52:03<87:40:01,  5.39s/it]
 74%|███████▍  | 166447/225000 [320:52:07<80:14:14,  4.93s/it]
 74%|███████▍  | 166448/225000 [320:52:18<109:43:46,  6.75s/it]
 74%|███████▍  | 166449/225000 [320:52:21<93:58:34,  5.78s/it] 
 74%|███████▍  | 166450/225000 [320:52:28<97:45:45,  6.01s/it]
                                                              
{'loss': '0.7701', 'grad_norm': '1.123', 'learning_rate': '8.626e-06', 'epoch': '0.7398', 'num_input_tokens_seen': 531296840, 'train_runtime': '1.155e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166450/225000 [320:52:28<97:45:45,  6.01s/it]
 74%|███████▍  | 166451/225000 [320:52:32<91:36:53,  5.63s/it]
 74%|███████▍  | 166452/225000 [320:52:40<102:45:45,  6.32s/it]
 74%|███████▍  | 166453/225000 [320:52:45<96:52:19,  5.96s/it] 
 74%|███████▍  | 166454/225000 [320:52:49<87:44:33,  5.40s/it]
 74%|███████▍  | 166455/225000 [320:52:57<99:11:01,  6.10s/it]
 74%|███████▍  | 166456/225000 [320:53:02<91:16:20,  5.61s/it]
 74%|███████▍  | 166457/225000 [320:53:05<81:07:35,  4.99s/it]
 74%|███████▍  | 166458/225000 [320:53:14<97:55:13,  6.02s/it]
 74%|███████▍  | 166459/225000 [320:53:17<84:43:00,  5.21s/it]
 74%|███████▍  | 166460/225000 [320:53:24<94:20:29,  5.80s/it]
 74%|███████▍  | 166461/225000 [320:53:28<84:10:22,  5.18s/it]
 74%|███████▍  | 166462/225000 [320:53:32<78:29:57,  4.83s/it]
 74%|███████▍  | 166463/225000 [320:53:43<107:01:45,  6.58s/it]
 74%|███████▍  | 166464/225000 [320:53:46<90:41:33,  5.58s/it] 
 74%|███████▍  | 166465/225000 [320:53:51<88:17:33,  5.43s/it]
 74%|███████▍  | 166466/225000 [320:53:59<102:44:10,  6.32s/it]
 74%|███████▍  | 166467/225000 [320:54:03<87:50:29,  5.40s/it] 
 74%|███████▍  | 166468/225000 [320:54:07<84:31:47,  5.20s/it]
 74%|███████▍  | 166469/225000 [320:54:12<82:50:36,  5.10s/it]
 74%|███████▍  | 166470/225000 [320:54:17<80:12:25,  4.93s/it]
 74%|███████▍  | 166471/225000 [320:54:22<80:01:10,  4.92s/it]
 74%|███████▍  | 166472/225000 [320:54:26<78:49:23,  4.85s/it]
 74%|███████▍  | 166473/225000 [320:54:30<75:47:01,  4.66s/it]
 74%|███████▍  | 166474/225000 [320:54:36<81:06:31,  4.99s/it]
 74%|███████▍  | 166475/225000 [320:54:48<112:38:04,  6.93s/it]
 74%|███████▍  | 166476/225000 [320:54:51<97:03:08,  5.97s/it] 
 74%|███████▍  | 166477/225000 [320:54:59<104:49:36,  6.45s/it]
 74%|███████▍  | 166478/225000 [320:55:05<100:25:56,  6.18s/it]
 74%|███████▍  | 166479/225000 [320:55:08<86:05:38,  5.30s/it] 
 74%|███████▍  | 166480/225000 [320:55:15<97:07:06,  5.97s/it]
 74%|███████▍  | 166481/225000 [320:55:19<83:46:36,  5.15s/it]
 74%|███████▍  | 166482/225000 [320:55:24<83:29:18,  5.14s/it]
 74%|███████▍  | 166483/225000 [320:55:31<91:58:03,  5.66s/it]
 74%|███████▍  | 166484/225000 [320:55:34<83:07:28,  5.11s/it]
 74%|███████▍  | 166485/225000 [320:55:38<78:08:18,  4.81s/it]
 74%|███████▍  | 166486/225000 [320:55:48<101:34:49,  6.25s/it]
 74%|███████▍  | 166487/225000 [320:55:53<95:27:44,  5.87s/it] 
 74%|███████▍  | 166488/225000 [320:55:58<91:27:54,  5.63s/it]
 74%|███████▍  | 166489/225000 [320:56:03<87:47:47,  5.40s/it]
 74%|███████▍  | 166490/225000 [320:56:08<84:48:06,  5.22s/it]
 74%|███████▍  | 166491/225000 [320:56:13<82:41:23,  5.09s/it]
 74%|███████▍  | 166492/225000 [320:56:17<81:06:04,  4.99s/it]
 74%|███████▍  | 166493/225000 [320:56:22<80:37:24,  4.96s/it]
 74%|███████▍  | 166494/225000 [320:56:27<81:34:00,  5.02s/it]
 74%|███████▍  | 166495/225000 [320:56:32<81:17:16,  5.00s/it]
 74%|███████▍  | 166496/225000 [320:56:37<81:50:17,  5.04s/it]
 74%|███████▍  | 166497/225000 [320:56:42<78:50:06,  4.85s/it]
 74%|███████▍  | 166498/225000 [320:56:50<96:28:16,  5.94s/it]
 74%|███████▍  | 166499/225000 [320:57:01<118:12:29,  7.27s/it]
 74%|███████▍  | 166500/225000 [320:57:04<100:00:42,  6.15s/it]
                                                               
{'loss': '0.7758', 'grad_norm': '1.209', 'learning_rate': '8.612e-06', 'epoch': '0.74', 'num_input_tokens_seen': 531426432, 'train_runtime': '1.155e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166500/225000 [320:57:04<100:00:42,  6.15s/it]
 74%|███████▍  | 166501/225000 [320:57:12<107:32:03,  6.62s/it]
 74%|███████▍  | 166502/225000 [320:57:15<91:47:53,  5.65s/it] 
 74%|███████▍  | 166503/225000 [320:57:20<89:03:30,  5.48s/it]
 74%|███████▍  | 166504/225000 [320:57:26<89:46:48,  5.53s/it]
 74%|███████▍  | 166505/225000 [320:57:34<100:32:56,  6.19s/it]
 74%|███████▍  | 166506/225000 [320:57:37<86:57:33,  5.35s/it] 
 74%|███████▍  | 166507/225000 [320:57:45<99:59:48,  6.15s/it]
 74%|███████▍  | 166508/225000 [320:57:49<85:53:44,  5.29s/it]
 74%|███████▍  | 166509/225000 [320:57:56<96:27:06,  5.94s/it]
 74%|███████▍  | 166510/225000 [320:57:59<84:27:35,  5.20s/it]
 74%|███████▍  | 166511/225000 [320:58:09<105:17:33,  6.48s/it]
 74%|███████▍  | 166512/225000 [320:58:12<91:02:16,  5.60s/it] 
 74%|███████▍  | 166513/225000 [320:58:16<81:24:39,  5.01s/it]
 74%|███████▍  | 166514/225000 [320:58:24<96:57:02,  5.97s/it]
 74%|███████▍  | 166515/225000 [320:58:28<85:37:26,  5.27s/it]
 74%|███████▍  | 166516/225000 [320:58:36<101:05:50,  6.22s/it]
 74%|███████▍  | 166517/225000 [320:58:40<89:03:37,  5.48s/it] 
 74%|███████▍  | 166518/225000 [320:58:44<79:27:14,  4.89s/it]
 74%|███████▍  | 166519/225000 [320:58:48<75:05:49,  4.62s/it]
 74%|███████▍  | 166520/225000 [320:58:56<91:32:37,  5.64s/it]
 74%|███████▍  | 166521/225000 [320:59:03<101:43:50,  6.26s/it]
 74%|███████▍  | 166522/225000 [320:59:07<88:31:18,  5.45s/it] 
 74%|███████▍  | 166523/225000 [320:59:11<80:52:13,  4.98s/it]
 74%|███████▍  | 166524/225000 [320:59:20<102:01:59,  6.28s/it]
 74%|███████▍  | 166525/225000 [320:59:25<95:13:55,  5.86s/it] 
 74%|███████▍  | 166526/225000 [320:59:29<84:01:17,  5.17s/it]
 74%|███████▍  | 166527/225000 [320:59:34<83:07:35,  5.12s/it]
 74%|███████▍  | 166528/225000 [320:59:44<107:23:22,  6.61s/it]
 74%|███████▍  | 166529/225000 [320:59:48<98:17:02,  6.05s/it] 
 74%|███████▍  | 166530/225000 [320:59:54<94:46:40,  5.84s/it]
 74%|███████▍  | 166531/225000 [320:59:59<90:08:49,  5.55s/it]
 74%|███████▍  | 166532/225000 [321:00:04<89:45:26,  5.53s/it]
 74%|███████▍  | 166533/225000 [321:00:09<87:38:36,  5.40s/it]
 74%|███████▍  | 166534/225000 [321:00:15<87:24:58,  5.38s/it]
 74%|███████▍  | 166535/225000 [321:00:20<86:15:27,  5.31s/it]
 74%|███████▍  | 166536/225000 [321:00:25<84:59:04,  5.23s/it]
 74%|███████▍  | 166537/225000 [321:00:30<85:36:35,  5.27s/it]
 74%|███████▍  | 166538/225000 [321:00:35<85:01:59,  5.24s/it]
 74%|███████▍  | 166539/225000 [321:00:40<84:59:09,  5.23s/it]
 74%|███████▍  | 166540/225000 [321:00:46<85:32:55,  5.27s/it]
 74%|███████▍  | 166541/225000 [321:00:51<85:54:54,  5.29s/it]
 74%|███████▍  | 166542/225000 [321:00:56<83:48:31,  5.16s/it]
 74%|███████▍  | 166543/225000 [321:01:01<82:40:33,  5.09s/it]
 74%|███████▍  | 166544/225000 [321:01:08<92:04:28,  5.67s/it]
 74%|███████▍  | 166545/225000 [321:01:17<106:18:15,  6.55s/it]
 74%|███████▍  | 166546/225000 [321:01:20<91:49:42,  5.66s/it] 
 74%|███████▍  | 166547/225000 [321:01:28<101:11:33,  6.23s/it]
 74%|███████▍  | 166548/225000 [321:01:35<104:59:21,  6.47s/it]
 74%|███████▍  | 166549/225000 [321:01:39<96:25:10,  5.94s/it] 
 74%|███████▍  | 166550/225000 [321:01:45<92:11:59,  5.68s/it]
                                                              
{'loss': '0.7688', 'grad_norm': '1.16', 'learning_rate': '8.599e-06', 'epoch': '0.7402', 'num_input_tokens_seen': 531554608, 'train_runtime': '1.156e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166550/225000 [321:01:45<92:11:59,  5.68s/it]
 74%|███████▍  | 166551/225000 [321:01:50<89:11:47,  5.49s/it]
 74%|███████▍  | 166552/225000 [321:01:55<87:23:21,  5.38s/it]
 74%|███████▍  | 166553/225000 [321:02:00<85:32:41,  5.27s/it]
 74%|███████▍  | 166554/225000 [321:02:04<83:08:46,  5.12s/it]
 74%|███████▍  | 166555/225000 [321:02:10<83:20:43,  5.13s/it]
 74%|███████▍  | 166556/225000 [321:02:15<83:22:32,  5.14s/it]
 74%|███████▍  | 166557/225000 [321:02:20<82:37:11,  5.09s/it]
 74%|███████▍  | 166558/225000 [321:02:25<83:36:04,  5.15s/it]
 74%|███████▍  | 166559/225000 [321:02:30<80:58:18,  4.99s/it]
 74%|███████▍  | 166560/225000 [321:02:35<80:25:51,  4.95s/it]
 74%|███████▍  | 166561/225000 [321:02:40<81:20:56,  5.01s/it]
 74%|███████▍  | 166562/225000 [321:02:44<76:51:01,  4.73s/it]
 74%|███████▍  | 166563/225000 [321:02:56<115:45:10,  7.13s/it]
 74%|███████▍  | 166564/225000 [321:03:01<101:07:07,  6.23s/it]
 74%|███████▍  | 166565/225000 [321:03:04<89:33:23,  5.52s/it] 
 74%|███████▍  | 166566/225000 [321:03:09<86:28:21,  5.33s/it]
 74%|███████▍  | 166567/225000 [321:03:14<84:36:14,  5.21s/it]
 74%|███████▍  | 166568/225000 [321:03:19<83:16:07,  5.13s/it]
 74%|███████▍  | 166569/225000 [321:03:24<81:24:27,  5.02s/it]
 74%|███████▍  | 166570/225000 [321:03:29<79:25:47,  4.89s/it]
 74%|███████▍  | 166571/225000 [321:03:33<78:16:16,  4.82s/it]
 74%|███████▍  | 166572/225000 [321:03:45<114:10:50,  7.04s/it]
 74%|███████▍  | 166573/225000 [321:03:50<103:34:25,  6.38s/it]
 74%|███████▍  | 166574/225000 [321:03:57<106:33:16,  6.57s/it]
 74%|███████▍  | 166575/225000 [321:04:02<99:09:34,  6.11s/it] 
 74%|███████▍  | 166576/225000 [321:04:07<91:07:33,  5.62s/it]
 74%|███████▍  | 166577/225000 [321:04:12<87:29:59,  5.39s/it]
 74%|███████▍  | 166578/225000 [321:04:16<81:37:07,  5.03s/it]
 74%|███████▍  | 166579/225000 [321:04:21<82:41:39,  5.10s/it]
 74%|███████▍  | 166580/225000 [321:04:33<114:05:29,  7.03s/it]
 74%|███████▍  | 166581/225000 [321:04:36<95:04:49,  5.86s/it] 
 74%|███████▍  | 166582/225000 [321:04:43<102:22:56,  6.31s/it]
 74%|███████▍  | 166583/225000 [321:04:47<88:03:22,  5.43s/it] 
 74%|███████▍  | 166584/225000 [321:04:51<82:28:41,  5.08s/it]
 74%|███████▍  | 166585/225000 [321:04:56<84:24:27,  5.20s/it]
 74%|███████▍  | 166586/225000 [321:05:01<83:21:07,  5.14s/it]
 74%|███████▍  | 166587/225000 [321:05:06<83:24:51,  5.14s/it]
 74%|███████▍  | 166588/225000 [321:05:11<82:11:27,  5.07s/it]
 74%|███████▍  | 166589/225000 [321:05:16<82:37:16,  5.09s/it]
 74%|███████▍  | 166590/225000 [321:05:26<106:23:51,  6.56s/it]
 74%|███████▍  | 166591/225000 [321:05:33<104:52:32,  6.46s/it]
 74%|███████▍  | 166592/225000 [321:05:37<93:07:54,  5.74s/it] 
 74%|███████▍  | 166593/225000 [321:05:41<85:44:22,  5.28s/it]
 74%|███████▍  | 166594/225000 [321:05:50<101:39:00,  6.27s/it]
 74%|███████▍  | 166595/225000 [321:05:56<103:36:30,  6.39s/it]
 74%|███████▍  | 166596/225000 [321:06:00<89:07:11,  5.49s/it] 
 74%|███████▍  | 166597/225000 [321:06:06<91:26:12,  5.64s/it]
 74%|███████▍  | 166598/225000 [321:06:12<96:53:56,  5.97s/it]
 74%|███████▍  | 166599/225000 [321:06:16<88:10:51,  5.44s/it]
 74%|███████▍  | 166600/225000 [321:06:20<80:48:42,  4.98s/it]
                                                              
{'loss': '0.7744', 'grad_norm': '1.101', 'learning_rate': '8.585e-06', 'epoch': '0.7404', 'num_input_tokens_seen': 531684696, 'train_runtime': '1.156e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166600/225000 [321:06:20<80:48:42,  4.98s/it]
 74%|███████▍  | 166601/225000 [321:06:29<98:15:07,  6.06s/it]
 74%|███████▍  | 166602/225000 [321:06:34<95:30:30,  5.89s/it]
 74%|███████▍  | 166603/225000 [321:06:39<87:06:25,  5.37s/it]
 74%|███████▍  | 166604/225000 [321:06:47<100:02:54,  6.17s/it]
 74%|███████▍  | 166605/225000 [321:06:50<86:17:58,  5.32s/it] 
 74%|███████▍  | 166606/225000 [321:06:58<100:06:09,  6.17s/it]
 74%|███████▍  | 166607/225000 [321:07:02<89:46:43,  5.53s/it] 
 74%|███████▍  | 166608/225000 [321:07:06<83:07:04,  5.12s/it]
 74%|███████▍  | 166609/225000 [321:07:10<76:22:47,  4.71s/it]
 74%|███████▍  | 166610/225000 [321:07:19<98:36:54,  6.08s/it]
 74%|███████▍  | 166611/225000 [321:07:25<95:42:11,  5.90s/it]
 74%|███████▍  | 166612/225000 [321:07:29<85:21:41,  5.26s/it]
 74%|███████▍  | 166613/225000 [321:07:38<102:49:41,  6.34s/it]
 74%|███████▍  | 166614/225000 [321:07:41<88:28:40,  5.46s/it] 
 74%|███████▍  | 166615/225000 [321:07:45<81:30:47,  5.03s/it]
 74%|███████▍  | 166616/225000 [321:07:49<77:30:57,  4.78s/it]
 74%|███████▍  | 166617/225000 [321:08:00<107:41:22,  6.64s/it]
 74%|███████▍  | 166618/225000 [321:08:04<95:58:23,  5.92s/it] 
 74%|███████▍  | 166619/225000 [321:08:08<86:58:14,  5.36s/it]
 74%|███████▍  | 166620/225000 [321:08:13<81:40:45,  5.04s/it]
 74%|███████▍  | 166621/225000 [321:08:17<77:25:54,  4.77s/it]
 74%|███████▍  | 166622/225000 [321:08:26<98:03:51,  6.05s/it]
 74%|███████▍  | 166623/225000 [321:08:31<94:40:37,  5.84s/it]
 74%|███████▍  | 166624/225000 [321:08:35<86:24:56,  5.33s/it]
 74%|███████▍  | 166625/225000 [321:08:40<82:07:20,  5.06s/it]
 74%|███████▍  | 166626/225000 [321:08:44<79:32:20,  4.91s/it]
 74%|███████▍  | 166627/225000 [321:08:49<77:04:37,  4.75s/it]
 74%|███████▍  | 166628/225000 [321:08:53<75:36:57,  4.66s/it]
 74%|███████▍  | 166629/225000 [321:09:04<105:00:00,  6.48s/it]
 74%|███████▍  | 166630/225000 [321:09:09<100:30:22,  6.20s/it]
 74%|███████▍  | 166631/225000 [321:09:13<88:47:40,  5.48s/it] 
 74%|███████▍  | 166632/225000 [321:09:23<110:05:34,  6.79s/it]
 74%|███████▍  | 166633/225000 [321:09:26<93:33:44,  5.77s/it] 
 74%|███████▍  | 166634/225000 [321:09:31<85:26:38,  5.27s/it]
 74%|███████▍  | 166635/225000 [321:09:39<98:44:03,  6.09s/it]
 74%|███████▍  | 166636/225000 [321:09:42<86:26:42,  5.33s/it]
 74%|███████▍  | 166637/225000 [321:09:49<95:25:37,  5.89s/it]
 74%|███████▍  | 166638/225000 [321:09:55<94:36:44,  5.84s/it]
 74%|███████▍  | 166639/225000 [321:10:00<88:28:28,  5.46s/it]
 74%|███████▍  | 166640/225000 [321:10:04<83:44:44,  5.17s/it]
 74%|███████▍  | 166641/225000 [321:10:09<81:42:05,  5.04s/it]
 74%|███████▍  | 166642/225000 [321:10:14<80:47:38,  4.98s/it]
 74%|███████▍  | 166643/225000 [321:10:18<78:18:16,  4.83s/it]
 74%|███████▍  | 166644/225000 [321:10:23<76:38:51,  4.73s/it]
 74%|███████▍  | 166645/225000 [321:10:27<76:59:27,  4.75s/it]
 74%|███████▍  | 166646/225000 [321:10:34<85:44:37,  5.29s/it]
 74%|███████▍  | 166647/225000 [321:10:45<115:04:29,  7.10s/it]
 74%|███████▍  | 166648/225000 [321:10:49<96:25:38,  5.95s/it] 
 74%|███████▍  | 166649/225000 [321:10:53<89:28:56,  5.52s/it]
 74%|███████▍  | 166650/225000 [321:11:00<96:03:11,  5.93s/it]
                                                              
{'loss': '0.7758', 'grad_norm': '1.14', 'learning_rate': '8.571e-06', 'epoch': '0.7407', 'num_input_tokens_seen': 531813728, 'train_runtime': '1.156e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166650/225000 [321:11:00<96:03:11,  5.93s/it]
 74%|███████▍  | 166651/225000 [321:11:04<84:48:37,  5.23s/it]
 74%|███████▍  | 166652/225000 [321:11:12<100:18:57,  6.19s/it]
 74%|███████▍  | 166653/225000 [321:11:16<87:45:35,  5.41s/it] 
 74%|███████▍  | 166654/225000 [321:11:22<90:09:19,  5.56s/it]
 74%|███████▍  | 166655/225000 [321:11:28<96:42:17,  5.97s/it]
 74%|███████▍  | 166656/225000 [321:11:34<93:14:58,  5.75s/it]
 74%|███████▍  | 166657/225000 [321:11:39<89:28:39,  5.52s/it]
 74%|███████▍  | 166658/225000 [321:11:44<86:05:10,  5.31s/it]
 74%|███████▍  | 166659/225000 [321:11:49<84:59:04,  5.24s/it]
 74%|███████▍  | 166660/225000 [321:11:54<84:26:00,  5.21s/it]
 74%|███████▍  | 166661/225000 [321:11:59<83:06:41,  5.13s/it]
 74%|███████▍  | 166662/225000 [321:12:03<80:50:52,  4.99s/it]
 74%|███████▍  | 166663/225000 [321:12:08<78:50:53,  4.87s/it]
 74%|███████▍  | 166664/225000 [321:12:13<79:26:53,  4.90s/it]
 74%|███████▍  | 166665/225000 [321:12:18<78:09:17,  4.82s/it]
 74%|███████▍  | 166666/225000 [321:12:23<79:47:42,  4.92s/it]
 74%|███████▍  | 166667/225000 [321:12:34<112:00:41,  6.91s/it]
 74%|███████▍  | 166668/225000 [321:12:38<95:48:30,  5.91s/it] 
 74%|███████▍  | 166669/225000 [321:12:44<97:19:24,  6.01s/it]
 74%|███████▍  | 166670/225000 [321:12:51<103:44:52,  6.40s/it]
 74%|███████▍  | 166671/225000 [321:12:58<106:50:45,  6.59s/it]
 74%|███████▍  | 166672/225000 [321:13:02<91:51:21,  5.67s/it] 
 74%|███████▍  | 166673/225000 [321:13:09<96:39:34,  5.97s/it]
 74%|███████▍  | 166674/225000 [321:13:12<85:14:33,  5.26s/it]
 74%|███████▍  | 166675/225000 [321:13:16<76:59:06,  4.75s/it]
 74%|███████▍  | 166676/225000 [321:13:20<75:13:38,  4.64s/it]
 74%|███████▍  | 166677/225000 [321:13:33<112:40:30,  6.95s/it]
 74%|███████▍  | 166678/225000 [321:13:36<95:50:29,  5.92s/it] 
 74%|███████▍  | 166679/225000 [321:13:43<100:09:46,  6.18s/it]
 74%|███████▍  | 166680/225000 [321:13:47<90:19:25,  5.58s/it] 
 74%|███████▍  | 166681/225000 [321:13:51<80:54:13,  4.99s/it]
 74%|███████▍  | 166682/225000 [321:13:55<77:35:26,  4.79s/it]
 74%|███████▍  | 166683/225000 [321:14:05<105:01:10,  6.48s/it]
 74%|███████▍  | 166684/225000 [321:14:09<91:37:32,  5.66s/it] 
 74%|███████▍  | 166685/225000 [321:14:13<81:34:21,  5.04s/it]
 74%|███████▍  | 166686/225000 [321:14:20<94:17:39,  5.82s/it]
 74%|███████▍  | 166687/225000 [321:14:27<96:37:04,  5.96s/it]
 74%|███████▍  | 166688/225000 [321:14:31<87:16:52,  5.39s/it]
 74%|███████▍  | 166689/225000 [321:14:35<80:08:11,  4.95s/it]
 74%|███████▍  | 166690/225000 [321:14:39<77:08:25,  4.76s/it]
 74%|███████▍  | 166691/225000 [321:14:49<104:13:33,  6.43s/it]
 74%|███████▍  | 166692/225000 [321:14:54<93:58:48,  5.80s/it] 
 74%|███████▍  | 166693/225000 [321:14:57<84:18:49,  5.21s/it]
 74%|███████▍  | 166694/225000 [321:15:02<78:59:23,  4.88s/it]
 74%|███████▍  | 166695/225000 [321:15:07<80:18:56,  4.96s/it]
 74%|███████▍  | 166696/225000 [321:15:17<108:00:51,  6.67s/it]
 74%|███████▍  | 166697/225000 [321:15:21<92:27:20,  5.71s/it] 
 74%|███████▍  | 166698/225000 [321:15:29<105:50:55,  6.54s/it]
 74%|███████▍  | 166699/225000 [321:15:33<94:15:12,  5.82s/it] 
 74%|███████▍  | 166700/225000 [321:15:37<85:08:54,  5.26s/it]
                                                              
{'loss': '0.7727', 'grad_norm': '1.231', 'learning_rate': '8.557e-06', 'epoch': '0.7409', 'num_input_tokens_seen': 531944536, 'train_runtime': '1.157e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166700/225000 [321:15:37<85:08:54,  5.26s/it]
 74%|███████▍  | 166701/225000 [321:15:43<87:44:36,  5.42s/it]
 74%|███████▍  | 166702/225000 [321:15:53<107:21:06,  6.63s/it]
 74%|███████▍  | 166703/225000 [321:15:56<91:29:33,  5.65s/it] 
 74%|███████▍  | 166704/225000 [321:16:03<99:50:59,  6.17s/it]
 74%|███████▍  | 166705/225000 [321:16:09<97:36:58,  6.03s/it]
 74%|███████▍  | 166706/225000 [321:16:14<92:28:45,  5.71s/it]
 74%|███████▍  | 166707/225000 [321:16:19<89:51:17,  5.55s/it]
 74%|███████▍  | 166708/225000 [321:16:24<88:42:01,  5.48s/it]
 74%|███████▍  | 166709/225000 [321:16:30<88:27:33,  5.46s/it]
 74%|███████▍  | 166710/225000 [321:16:36<92:24:37,  5.71s/it]
 74%|███████▍  | 166711/225000 [321:16:42<92:10:16,  5.69s/it]
 74%|███████▍  | 166712/225000 [321:16:46<82:53:34,  5.12s/it]
 74%|███████▍  | 166713/225000 [321:16:50<80:12:48,  4.95s/it]
 74%|███████▍  | 166714/225000 [321:16:55<79:55:39,  4.94s/it]
 74%|███████▍  | 166715/225000 [321:17:00<78:43:38,  4.86s/it]
 74%|███████▍  | 166716/225000 [321:17:05<78:27:07,  4.85s/it]
 74%|███████▍  | 166717/225000 [321:17:17<115:13:42,  7.12s/it]
 74%|███████▍  | 166718/225000 [321:17:20<96:36:54,  5.97s/it] 
 74%|███████▍  | 166719/225000 [321:17:28<107:07:41,  6.62s/it]
 74%|███████▍  | 166720/225000 [321:17:35<107:10:38,  6.62s/it]
 74%|███████▍  | 166721/225000 [321:17:39<94:14:25,  5.82s/it] 
 74%|███████▍  | 166722/225000 [321:17:43<83:25:18,  5.15s/it]
 74%|███████▍  | 166723/225000 [321:17:47<78:07:53,  4.83s/it]
 74%|███████▍  | 166724/225000 [321:17:58<110:15:37,  6.81s/it]
 74%|███████▍  | 166725/225000 [321:18:03<99:14:17,  6.13s/it] 
 74%|███████▍  | 166726/225000 [321:18:07<90:33:39,  5.59s/it]
 74%|███████▍  | 166727/225000 [321:18:11<82:12:48,  5.08s/it]
 74%|███████▍  | 166728/225000 [321:18:15<77:13:40,  4.77s/it]
 74%|███████▍  | 166729/225000 [321:18:22<88:22:56,  5.46s/it]
 74%|███████▍  | 166730/225000 [321:18:31<105:42:43,  6.53s/it]
 74%|███████▍  | 166731/225000 [321:18:34<90:01:19,  5.56s/it] 
 74%|███████▍  | 166732/225000 [321:18:39<85:16:09,  5.27s/it]
 74%|███████▍  | 166733/225000 [321:18:44<83:26:53,  5.16s/it]
 74%|███████▍  | 166734/225000 [321:18:49<82:38:53,  5.11s/it]
 74%|███████▍  | 166735/225000 [321:18:53<79:02:24,  4.88s/it]
 74%|███████▍  | 166736/225000 [321:19:05<114:21:23,  7.07s/it]
 74%|███████▍  | 166737/225000 [321:19:09<98:38:09,  6.09s/it] 
 74%|███████▍  | 166738/225000 [321:19:13<88:20:36,  5.46s/it]
 74%|███████▍  | 166739/225000 [321:19:17<81:21:22,  5.03s/it]
 74%|███████▍  | 166740/225000 [321:19:21<75:58:15,  4.69s/it]
 74%|███████▍  | 166741/225000 [321:19:25<74:10:48,  4.58s/it]
 74%|███████▍  | 166742/225000 [321:19:36<101:03:08,  6.24s/it]
 74%|███████▍  | 166743/225000 [321:19:42<102:54:42,  6.36s/it]
 74%|███████▍  | 166744/225000 [321:19:45<87:23:08,  5.40s/it] 
 74%|███████▍  | 166745/225000 [321:19:53<99:07:04,  6.13s/it]
 74%|███████▍  | 166746/225000 [321:19:57<86:27:16,  5.34s/it]
 74%|███████▍  | 166747/225000 [321:20:04<96:59:10,  5.99s/it]
 74%|███████▍  | 166748/225000 [321:20:08<87:11:38,  5.39s/it]
 74%|███████▍  | 166749/225000 [321:20:13<83:23:28,  5.15s/it]
 74%|███████▍  | 166750/225000 [321:20:17<79:30:19,  4.91s/it]
                                                              
{'loss': '0.7673', 'grad_norm': '1.141', 'learning_rate': '8.544e-06', 'epoch': '0.7411', 'num_input_tokens_seen': 532074368, 'train_runtime': '1.157e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166750/225000 [321:20:17<79:30:19,  4.91s/it]
 74%|███████▍  | 166751/225000 [321:20:26<99:00:02,  6.12s/it]
 74%|███████▍  | 166752/225000 [321:20:30<89:25:19,  5.53s/it]
 74%|███████▍  | 166753/225000 [321:20:35<84:33:07,  5.23s/it]
 74%|███████▍  | 166754/225000 [321:20:39<78:23:27,  4.85s/it]
 74%|███████▍  | 166755/225000 [321:20:44<80:15:12,  4.96s/it]
 74%|███████▍  | 166756/225000 [321:20:55<111:03:55,  6.86s/it]
 74%|███████▍  | 166757/225000 [321:20:59<95:33:26,  5.91s/it] 
 74%|███████▍  | 166758/225000 [321:21:06<102:33:36,  6.34s/it]
 74%|███████▍  | 166759/225000 [321:21:10<89:08:44,  5.51s/it] 
 74%|███████▍  | 166760/225000 [321:21:17<96:02:37,  5.94s/it]
 74%|███████▍  | 166761/225000 [321:21:20<84:29:32,  5.22s/it]
 74%|███████▍  | 166762/225000 [321:21:24<79:31:21,  4.92s/it]
 74%|███████▍  | 166763/225000 [321:21:32<92:00:47,  5.69s/it]
 74%|███████▍  | 166764/225000 [321:21:37<88:33:59,  5.47s/it]
 74%|███████▍  | 166765/225000 [321:21:41<80:09:36,  4.96s/it]
 74%|███████▍  | 166766/225000 [321:21:50<101:35:39,  6.28s/it]
 74%|███████▍  | 166767/225000 [321:21:54<89:09:12,  5.51s/it] 
 74%|███████▍  | 166768/225000 [321:21:58<81:36:32,  5.05s/it]
 74%|███████▍  | 166769/225000 [321:22:02<75:28:50,  4.67s/it]
 74%|███████▍  | 166770/225000 [321:22:06<74:27:10,  4.60s/it]
 74%|███████▍  | 166771/225000 [321:22:17<104:27:43,  6.46s/it]
 74%|███████▍  | 166772/225000 [321:22:20<89:38:26,  5.54s/it] 
 74%|███████▍  | 166773/225000 [321:22:28<101:25:04,  6.27s/it]
 74%|███████▍  | 166774/225000 [321:22:32<87:48:32,  5.43s/it] 
 74%|███████▍  | 166775/225000 [321:22:40<101:02:29,  6.25s/it]
 74%|███████▍  | 166776/225000 [321:22:43<88:13:16,  5.45s/it] 
 74%|███████▍  | 166777/225000 [321:22:52<101:52:09,  6.30s/it]
 74%|███████▍  | 166778/225000 [321:22:55<87:45:49,  5.43s/it] 
 74%|███████▍  | 166779/225000 [321:23:01<88:09:10,  5.45s/it]
 74%|███████▍  | 166780/225000 [321:23:07<94:22:53,  5.84s/it]
 74%|███████▍  | 166781/225000 [321:23:13<93:54:04,  5.81s/it]
 74%|███████▍  | 166782/225000 [321:23:16<81:57:35,  5.07s/it]
 74%|███████▍  | 166783/225000 [321:23:24<92:53:00,  5.74s/it]
 74%|███████▍  | 166784/225000 [321:23:27<83:14:16,  5.15s/it]
 74%|███████▍  | 166785/225000 [321:23:32<79:52:35,  4.94s/it]
 74%|███████▍  | 166786/225000 [321:23:37<78:57:03,  4.88s/it]
 74%|███████▍  | 166787/225000 [321:23:41<77:28:15,  4.79s/it]
 74%|███████▍  | 166788/225000 [321:23:46<78:35:17,  4.86s/it]
 74%|███████▍  | 166789/225000 [321:23:51<79:01:09,  4.89s/it]
 74%|███████▍  | 166790/225000 [321:23:56<79:27:34,  4.91s/it]
 74%|███████▍  | 166791/225000 [321:24:05<98:14:54,  6.08s/it]
 74%|███████▍  | 166792/225000 [321:24:15<119:31:36,  7.39s/it]
 74%|███████▍  | 166793/225000 [321:24:19<99:14:55,  6.14s/it] 
 74%|███████▍  | 166794/225000 [321:24:24<96:47:07,  5.99s/it]
 74%|███████▍  | 166795/225000 [321:24:29<92:45:15,  5.74s/it]
 74%|███████▍  | 166796/225000 [321:24:34<88:29:39,  5.47s/it]
 74%|███████▍  | 166797/225000 [321:24:40<88:31:32,  5.48s/it]
 74%|███████▍  | 166798/225000 [321:24:45<85:15:53,  5.27s/it]
 74%|███████▍  | 166799/225000 [321:24:50<87:28:03,  5.41s/it]
 74%|███████▍  | 166800/225000 [321:24:56<87:16:42,  5.40s/it]
                                                              
{'loss': '0.7784', 'grad_norm': '1.134', 'learning_rate': '8.53e-06', 'epoch': '0.7413', 'num_input_tokens_seen': 532205472, 'train_runtime': '1.157e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166800/225000 [321:24:56<87:16:42,  5.40s/it]
 74%|███████▍  | 166801/225000 [321:25:04<99:41:33,  6.17s/it]
 74%|███████▍  | 166802/225000 [321:25:13<113:22:19,  7.01s/it]
 74%|███████▍  | 166803/225000 [321:25:17<99:12:12,  6.14s/it] 
 74%|███████▍  | 166804/225000 [321:25:21<91:39:36,  5.67s/it]
 74%|███████▍  | 166805/225000 [321:25:26<86:34:33,  5.36s/it]
 74%|███████▍  | 166806/225000 [321:25:30<82:53:38,  5.13s/it]
 74%|███████▍  | 166807/225000 [321:25:35<78:45:47,  4.87s/it]
 74%|███████▍  | 166808/225000 [321:25:39<77:07:13,  4.77s/it]
 74%|███████▍  | 166809/225000 [321:25:51<110:47:47,  6.85s/it]
 74%|███████▍  | 166810/225000 [321:25:56<102:15:10,  6.33s/it]
 74%|███████▍  | 166811/225000 [321:26:00<90:47:12,  5.62s/it] 
 74%|███████▍  | 166812/225000 [321:26:04<83:46:39,  5.18s/it]
 74%|███████▍  | 166813/225000 [321:26:09<82:00:06,  5.07s/it]
 74%|███████▍  | 166814/225000 [321:26:20<112:42:57,  6.97s/it]
 74%|███████▍  | 166815/225000 [321:26:25<102:18:00,  6.33s/it]
 74%|███████▍  | 166816/225000 [321:26:30<95:08:31,  5.89s/it] 
 74%|███████▍  | 166817/225000 [321:26:35<88:51:42,  5.50s/it]
 74%|███████▍  | 166818/225000 [321:26:40<85:45:01,  5.31s/it]
 74%|███████▍  | 166819/225000 [321:26:44<82:06:36,  5.08s/it]
 74%|███████▍  | 166820/225000 [321:26:49<78:52:21,  4.88s/it]
 74%|███████▍  | 166821/225000 [321:26:54<79:52:49,  4.94s/it]
 74%|███████▍  | 166822/225000 [321:26:58<78:58:59,  4.89s/it]
 74%|███████▍  | 166823/225000 [321:27:03<77:55:45,  4.82s/it]
 74%|███████▍  | 166824/225000 [321:27:08<78:56:39,  4.89s/it]
 74%|███████▍  | 166825/225000 [321:27:16<94:28:09,  5.85s/it]
 74%|███████▍  | 166826/225000 [321:27:25<109:32:48,  6.78s/it]
 74%|███████▍  | 166827/225000 [321:27:29<93:59:31,  5.82s/it] 
 74%|███████▍  | 166828/225000 [321:27:33<84:50:07,  5.25s/it]
 74%|███████▍  | 166829/225000 [321:27:44<113:55:16,  7.05s/it]
 74%|███████▍  | 166830/225000 [321:27:47<95:39:31,  5.92s/it] 
 74%|███████▍  | 166831/225000 [321:27:52<88:04:19,  5.45s/it]
 74%|███████▍  | 166832/225000 [321:28:00<102:50:00,  6.36s/it]
 74%|███████▍  | 166833/225000 [321:28:04<93:11:30,  5.77s/it] 
 74%|███████▍  | 166834/225000 [321:28:09<87:40:35,  5.43s/it]
 74%|███████▍  | 166835/225000 [321:28:14<83:59:22,  5.20s/it]
 74%|███████▍  | 166836/225000 [321:28:18<81:18:02,  5.03s/it]
 74%|███████▍  | 166837/225000 [321:28:23<78:48:38,  4.88s/it]
 74%|███████▍  | 166838/225000 [321:28:35<112:36:30,  6.97s/it]
 74%|███████▍  | 166839/225000 [321:28:39<99:12:31,  6.14s/it] 
 74%|███████▍  | 166840/225000 [321:28:43<87:35:23,  5.42s/it]
 74%|███████▍  | 166841/225000 [321:28:47<81:18:22,  5.03s/it]
 74%|███████▍  | 166842/225000 [321:28:57<108:08:35,  6.69s/it]
 74%|███████▍  | 166843/225000 [321:29:02<98:10:48,  6.08s/it] 
 74%|███████▍  | 166844/225000 [321:29:06<88:09:15,  5.46s/it]
 74%|███████▍  | 166845/225000 [321:29:10<80:04:05,  4.96s/it]
 74%|███████▍  | 166846/225000 [321:29:21<111:03:34,  6.88s/it]
 74%|███████▍  | 166847/225000 [321:29:25<94:41:43,  5.86s/it] 
 74%|███████▍  | 166848/225000 [321:29:29<87:33:02,  5.42s/it]
 74%|███████▍  | 166849/225000 [321:29:36<94:37:45,  5.86s/it]
 74%|███████▍  | 166850/225000 [321:29:40<83:52:11,  5.19s/it]
                                                              
{'loss': '0.7762', 'grad_norm': '1.124', 'learning_rate': '8.516e-06', 'epoch': '0.7416', 'num_input_tokens_seen': 532335648, 'train_runtime': '1.157e+06', 'train_tokens_per_second': '459.9'}

 74%|███████▍  | 166850/225000 [321:29:40<83:52:11,  5.19s/it]
 74%|███████▍  | 166851/225000 [321:29:47<95:00:50,  5.88s/it]
 74%|███████▍  | 166852/225000 [321:29:50<82:47:36,  5.13s/it]
 74%|███████▍  | 166853/225000 [321:29:54<76:41:16,  4.75s/it]
 74%|███████▍  | 166854/225000 [321:30:03<98:15:48,  6.08s/it]
 74%|███████▍  | 166855/225000 [321:30:07<86:26:28,  5.35s/it]
 74%|███████▍  | 166856/225000 [321:30:14<93:41:31,  5.80s/it]
 74%|███████▍  | 166857/225000 [321:30:18<83:33:04,  5.17s/it]
 74%|███████▍  | 166858/225000 [321:30:26<97:33:45,  6.04s/it]
 74%|███████▍  | 166859/225000 [321:30:29<84:01:31,  5.20s/it]
 74%|███████▍  | 166860/225000 [321:30:33<77:02:31,  4.77s/it]
 74%|███████▍  | 166861/225000 [321:30:37<73:07:35,  4.53s/it]
 74%|███████▍  | 166862/225000 [321:30:46<94:36:18,  5.86s/it]
 74%|███████▍  | 166863/225000 [321:30:51<90:43:30,  5.62s/it]
 74%|███████▍  | 166864/225000 [321:30:57<92:49:27,  5.75s/it]
 74%|███████▍  | 166865/225000 [321:31:02<89:41:38,  5.55s/it]
 74%|███████▍  | 166866/225000 [321:31:07<86:51:56,  5.38s/it]
 74%|███████▍  | 166867/225000 [321:31:11<83:12:29,  5.15s/it]
 74%|███████▍  | 166868/225000 [321:31:16<81:43:03,  5.06s/it]
 74%|███████▍  | 166869/225000 [321:31:21<81:02:34,  5.02s/it]
 74%|███████▍  | 166870/225000 [321:31:26<79:57:58,  4.95s/it]
 74%|███████▍  | 166871/225000 [321:31:31<81:10:20,  5.03s/it]
 74%|███████▍  | 166872/225000 [321:31:36<81:30:11,  5.05s/it]
 74%|███████▍  | 166873/225000 [321:31:42<82:24:44,  5.10s/it]
 74%|███████▍  | 166874/225000 [321:31:47<82:30:52,  5.11s/it]
 74%|███████▍  | 166875/225000 [321:31:52<83:05:42,  5.15s/it]
 74%|███████▍  | 166876/225000 [321:31:57<81:21:49,  5.04s/it]
 74%|███████▍  | 166877/225000 [321:32:02<81:25:06,  5.04s/it]
 74%|███████▍  | 166878/225000 [321:32:07<82:38:06,  5.12s/it]
 74%|███████▍  | 166879/225000 [321:32:12<81:49:11,  5.07s/it]
 74%|███████▍  | 166880/225000 [321:32:17<82:54:16,  5.14s/it]
 74%|███████▍  | 166881/225000 [321:32:28<109:14:31,  6.77s/it]
 74%|███████▍  | 166882/225000 [321:32:37<121:33:23,  7.53s/it]
 74%|███████▍  | 166883/225000 [321:32:40<101:04:16,  6.26s/it]
 74%|███████▍  | 166884/225000 [321:32:47<103:11:08,  6.39s/it]
 74%|███████▍  | 166885/225000 [321:32:51<92:39:29,  5.74s/it] 
 74%|███████▍  | 166886/225000 [321:32:55<84:17:10,  5.22s/it]
 74%|███████▍  | 166887/225000 [321:32:59<77:51:02,  4.82s/it]
 74%|███████▍  | 166888/225000 [321:33:03<74:37:28,  4.62s/it]
 74%|███████▍  | 166889/225000 [321:33:16<111:53:43,  6.93s/it]
 74%|███████▍  | 166890/225000 [321:33:21<103:06:33,  6.39s/it]
 74%|███████▍  | 166891/225000 [321:33:24<88:01:15,  5.45s/it] 
 74%|███████▍  | 166892/225000 [321:33:32<99:22:36,  6.16s/it]
 74%|███████▍  | 166893/225000 [321:33:37<92:02:46,  5.70s/it]
 74%|███████▍  | 166894/225000 [321:33:41<86:46:29,  5.38s/it]
 74%|███████▍  | 166895/225000 [321:33:46<81:43:25,  5.06s/it]
 74%|███████▍  | 166896/225000 [321:33:50<81:01:33,  5.02s/it]
 74%|███████▍  | 166897/225000 [321:33:56<82:30:18,  5.11s/it]
 74%|███████▍  | 166898/225000 [321:34:01<81:15:14,  5.03s/it]
 74%|███████▍  | 166899/225000 [321:34:06<81:54:01,  5.07s/it]
 74%|███████▍  | 166900/225000 [321:34:11<80:05:23,  4.96s/it]
                                                              
{'loss': '0.7731', 'grad_norm': '1.157', 'learning_rate': '8.502e-06', 'epoch': '0.7418', 'num_input_tokens_seen': 532465168, 'train_runtime': '1.158e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 166900/225000 [321:34:11<80:05:23,  4.96s/it]
 74%|███████▍  | 166901/225000 [321:34:16<80:19:05,  4.98s/it]
 74%|███████▍  | 166902/225000 [321:34:28<118:08:38,  7.32s/it]
 74%|███████▍  | 166903/225000 [321:34:32<102:10:29,  6.33s/it]
 74%|███████▍  | 166904/225000 [321:34:36<91:21:57,  5.66s/it] 
 74%|███████▍  | 166905/225000 [321:34:41<83:55:47,  5.20s/it]
 74%|███████▍  | 166906/225000 [321:34:45<78:11:19,  4.85s/it]
 74%|███████▍  | 166907/225000 [321:34:57<112:43:44,  6.99s/it]
 74%|███████▍  | 166908/225000 [321:35:01<98:30:01,  6.10s/it] 
 74%|███████▍  | 166909/225000 [321:35:07<99:10:56,  6.15s/it]
 74%|███████▍  | 166910/225000 [321:35:11<87:57:09,  5.45s/it]
 74%|███████▍  | 166911/225000 [321:35:16<86:49:32,  5.38s/it]
 74%|███████▍  | 166912/225000 [321:35:21<84:02:29,  5.21s/it]
 74%|███████▍  | 166913/225000 [321:35:26<83:50:31,  5.20s/it]
 74%|███████▍  | 166914/225000 [321:35:31<81:54:33,  5.08s/it]
 74%|███████▍  | 166915/225000 [321:35:36<80:55:44,  5.02s/it]
 74%|███████▍  | 166916/225000 [321:35:41<80:48:58,  5.01s/it]
 74%|███████▍  | 166917/225000 [321:35:46<81:51:53,  5.07s/it]
 74%|███████▍  | 166918/225000 [321:35:51<81:39:36,  5.06s/it]
 74%|███████▍  | 166919/225000 [321:35:59<95:34:22,  5.92s/it]
 74%|███████▍  | 166920/225000 [321:36:09<117:09:43,  7.26s/it]
 74%|███████▍  | 166921/225000 [321:36:13<99:26:30,  6.16s/it] 
 74%|███████▍  | 166922/225000 [321:36:18<96:46:45,  6.00s/it]
 74%|███████▍  | 166923/225000 [321:36:24<92:59:55,  5.76s/it]
 74%|███████▍  | 166924/225000 [321:36:29<92:13:15,  5.72s/it]
 74%|███████▍  | 166925/225000 [321:36:38<108:00:48,  6.70s/it]
 74%|███████▍  | 166926/225000 [321:36:42<94:31:01,  5.86s/it] 
 74%|███████▍  | 166927/225000 [321:36:49<101:42:14,  6.30s/it]
 74%|███████▍  | 166928/225000 [321:36:53<89:19:39,  5.54s/it] 
 74%|███████▍  | 166929/225000 [321:36:57<80:08:10,  4.97s/it]
 74%|███████▍  | 166930/225000 [321:37:02<81:51:44,  5.07s/it]
 74%|███████▍  | 166931/225000 [321:37:11<98:07:13,  6.08s/it]
 74%|███████▍  | 166932/225000 [321:37:14<85:18:01,  5.29s/it]
 74%|███████▍  | 166933/225000 [321:37:23<102:19:47,  6.34s/it]
 74%|███████▍  | 166934/225000 [321:37:27<91:31:59,  5.67s/it] 
 74%|███████▍  | 166935/225000 [321:37:31<84:27:42,  5.24s/it]
 74%|███████▍  | 166936/225000 [321:37:35<77:19:52,  4.79s/it]
 74%|███████▍  | 166937/225000 [321:37:40<77:21:57,  4.80s/it]
 74%|███████▍  | 166938/225000 [321:37:47<91:19:40,  5.66s/it]
 74%|███████▍  | 166939/225000 [321:37:52<86:05:23,  5.34s/it]
 74%|███████▍  | 166940/225000 [321:37:56<81:59:07,  5.08s/it]
 74%|███████▍  | 166941/225000 [321:38:01<81:52:54,  5.08s/it]
 74%|███████▍  | 166942/225000 [321:38:06<81:19:42,  5.04s/it]
 74%|███████▍  | 166943/225000 [321:38:11<80:23:31,  4.98s/it]
 74%|███████▍  | 166944/225000 [321:38:16<80:24:23,  4.99s/it]
 74%|███████▍  | 166945/225000 [321:38:21<79:37:23,  4.94s/it]
 74%|███████▍  | 166946/225000 [321:38:26<79:59:29,  4.96s/it]
 74%|███████▍  | 166947/225000 [321:38:31<81:55:47,  5.08s/it]
 74%|███████▍  | 166948/225000 [321:38:37<82:56:30,  5.14s/it]
 74%|███████▍  | 166949/225000 [321:38:42<82:31:13,  5.12s/it]
 74%|███████▍  | 166950/225000 [321:38:48<88:03:14,  5.46s/it]
                                                              
{'loss': '0.7735', 'grad_norm': '1.05', 'learning_rate': '8.489e-06', 'epoch': '0.742', 'num_input_tokens_seen': 532595216, 'train_runtime': '1.158e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 166950/225000 [321:38:48<88:03:14,  5.46s/it]
 74%|███████▍  | 166951/225000 [321:38:59<112:25:08,  6.97s/it]
 74%|███████▍  | 166952/225000 [321:39:04<106:29:30,  6.60s/it]
 74%|███████▍  | 166953/225000 [321:39:09<99:27:08,  6.17s/it] 
 74%|███████▍  | 166954/225000 [321:39:13<88:29:11,  5.49s/it]
 74%|███████▍  | 166955/225000 [321:39:20<94:43:27,  5.87s/it]
 74%|███████▍  | 166956/225000 [321:39:27<100:22:37,  6.23s/it]
 74%|███████▍  | 166957/225000 [321:39:31<87:03:55,  5.40s/it] 
 74%|███████▍  | 166958/225000 [321:39:38<98:23:39,  6.10s/it]
 74%|███████▍  | 166959/225000 [321:39:42<83:43:35,  5.19s/it]
 74%|███████▍  | 166960/225000 [321:39:50<99:41:28,  6.18s/it]
 74%|███████▍  | 166961/225000 [321:39:54<88:43:34,  5.50s/it]
 74%|███████▍  | 166962/225000 [321:40:01<97:24:11,  6.04s/it]
 74%|███████▍  | 166963/225000 [321:40:05<84:13:38,  5.22s/it]
 74%|███████▍  | 166964/225000 [321:40:12<94:40:20,  5.87s/it]
 74%|███████▍  | 166965/225000 [321:40:17<90:02:00,  5.58s/it]
 74%|███████▍  | 166966/225000 [321:40:21<85:19:20,  5.29s/it]
 74%|███████▍  | 166967/225000 [321:40:26<82:21:13,  5.11s/it]
 74%|███████▍  | 166968/225000 [321:40:31<79:04:19,  4.91s/it]
 74%|███████▍  | 166969/225000 [321:40:35<76:16:16,  4.73s/it]
 74%|███████▍  | 166970/225000 [321:40:39<74:42:27,  4.63s/it]
 74%|███████▍  | 166971/225000 [321:40:43<72:11:12,  4.48s/it]
 74%|███████▍  | 166972/225000 [321:40:53<95:02:15,  5.90s/it]
 74%|███████▍  | 166973/225000 [321:41:01<105:30:38,  6.55s/it]
 74%|███████▍  | 166974/225000 [321:41:04<88:48:21,  5.51s/it] 
 74%|███████▍  | 166975/225000 [321:41:11<99:25:47,  6.17s/it]
 74%|███████▍  | 166976/225000 [321:41:17<94:08:36,  5.84s/it]
 74%|███████▍  | 166977/225000 [321:41:20<84:47:37,  5.26s/it]
 74%|███████▍  | 166978/225000 [321:41:24<78:00:51,  4.84s/it]
 74%|███████▍  | 166979/225000 [321:41:28<73:40:03,  4.57s/it]
 74%|███████▍  | 166980/225000 [321:41:38<98:19:04,  6.10s/it]
 74%|███████▍  | 166981/225000 [321:41:44<97:15:33,  6.03s/it]
 74%|███████▍  | 166982/225000 [321:41:47<85:09:48,  5.28s/it]
 74%|███████▍  | 166983/225000 [321:41:57<108:35:56,  6.74s/it]
 74%|███████▍  | 166984/225000 [321:42:01<92:10:24,  5.72s/it] 
 74%|███████▍  | 166985/225000 [321:42:07<93:51:09,  5.82s/it]
 74%|███████▍  | 166986/225000 [321:42:12<90:34:19,  5.62s/it]
 74%|███████▍  | 166987/225000 [321:42:17<88:05:00,  5.47s/it]
 74%|███████▍  | 166988/225000 [321:42:22<85:32:19,  5.31s/it]
 74%|███████▍  | 166989/225000 [321:42:27<81:36:33,  5.06s/it]
 74%|███████▍  | 166990/225000 [321:42:32<81:42:46,  5.07s/it]
 74%|███████▍  | 166991/225000 [321:42:37<82:13:51,  5.10s/it]
 74%|███████▍  | 166992/225000 [321:42:42<80:58:02,  5.02s/it]
 74%|███████▍  | 166993/225000 [321:42:47<80:37:27,  5.00s/it]
 74%|███████▍  | 166994/225000 [321:42:52<80:07:32,  4.97s/it]
 74%|███████▍  | 166995/225000 [321:42:57<80:26:54,  4.99s/it]
 74%|███████▍  | 166996/225000 [321:43:02<80:13:02,  4.98s/it]
 74%|███████▍  | 166997/225000 [321:43:06<79:22:26,  4.93s/it]
 74%|███████▍  | 166998/225000 [321:43:11<78:40:59,  4.88s/it]
 74%|███████▍  | 166999/225000 [321:43:19<93:24:00,  5.80s/it]
 74%|███████▍  | 167000/225000 [321:43:29<115:36:03,  7.18s/it]
                                                               
{'loss': '0.7881', 'grad_norm': '1.279', 'learning_rate': '8.475e-06', 'epoch': '0.7422', 'num_input_tokens_seen': 532724312, 'train_runtime': '1.158e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167000/225000 [321:43:29<115:36:03,  7.18s/it]
 74%|███████▍  | 167001/225000 [321:43:35<106:11:18,  6.59s/it]
 74%|███████▍  | 167002/225000 [321:43:40<100:36:15,  6.24s/it]
 74%|███████▍  | 167003/225000 [321:43:45<95:57:01,  5.96s/it] 
 74%|███████▍  | 167004/225000 [321:43:50<90:46:51,  5.64s/it]
 74%|███████▍  | 167005/225000 [321:43:55<88:53:59,  5.52s/it]
 74%|███████▍  | 167006/225000 [321:44:01<91:09:45,  5.66s/it]
 74%|███████▍  | 167007/225000 [321:44:06<87:44:59,  5.45s/it]
 74%|███████▍  | 167008/225000 [321:44:11<84:43:24,  5.26s/it]
 74%|███████▍  | 167009/225000 [321:44:16<83:57:45,  5.21s/it]
 74%|███████▍  | 167010/225000 [321:44:21<83:33:13,  5.19s/it]
 74%|███████▍  | 167011/225000 [321:44:26<81:34:38,  5.06s/it]
 74%|███████▍  | 167012/225000 [321:44:32<82:32:02,  5.12s/it]
 74%|███████▍  | 167013/225000 [321:44:37<82:47:33,  5.14s/it]
 74%|███████▍  | 167014/225000 [321:44:42<84:22:31,  5.24s/it]
 74%|███████▍  | 167015/225000 [321:44:54<117:36:34,  7.30s/it]
 74%|███████▍  | 167016/225000 [321:44:59<103:59:53,  6.46s/it]
 74%|███████▍  | 167017/225000 [321:45:03<93:50:07,  5.83s/it] 
 74%|███████▍  | 167018/225000 [321:45:08<89:34:37,  5.56s/it]
 74%|███████▍  | 167019/225000 [321:45:13<84:10:10,  5.23s/it]
 74%|███████▍  | 167020/225000 [321:45:17<80:45:03,  5.01s/it]
 74%|███████▍  | 167021/225000 [321:45:22<79:57:21,  4.96s/it]
 74%|███████▍  | 167022/225000 [321:45:27<80:40:26,  5.01s/it]
 74%|███████▍  | 167023/225000 [321:45:32<80:48:14,  5.02s/it]
 74%|███████▍  | 167024/225000 [321:45:41<100:54:35,  6.27s/it]
 74%|███████▍  | 167025/225000 [321:45:50<111:53:00,  6.95s/it]
 74%|███████▍  | 167026/225000 [321:45:53<94:05:24,  5.84s/it] 
 74%|███████▍  | 167027/225000 [321:46:00<98:44:36,  6.13s/it]
 74%|███████▍  | 167028/225000 [321:46:07<103:09:24,  6.41s/it]
 74%|███████▍  | 167029/225000 [321:46:11<92:31:13,  5.75s/it] 
 74%|███████▍  | 167030/225000 [321:46:15<86:04:27,  5.35s/it]
 74%|███████▍  | 167031/225000 [321:46:20<81:41:09,  5.07s/it]
 74%|███████▍  | 167032/225000 [321:46:24<78:08:43,  4.85s/it]
 74%|███████▍  | 167033/225000 [321:46:31<85:50:35,  5.33s/it]
 74%|███████▍  | 167034/225000 [321:46:40<105:55:50,  6.58s/it]
 74%|███████▍  | 167035/225000 [321:46:44<91:15:54,  5.67s/it] 
 74%|███████▍  | 167036/225000 [321:46:52<105:05:19,  6.53s/it]
 74%|███████▍  | 167037/225000 [321:46:55<89:00:57,  5.53s/it] 
 74%|███████▍  | 167038/225000 [321:47:01<88:26:02,  5.49s/it]
 74%|███████▍  | 167039/225000 [321:47:08<96:32:28,  6.00s/it]
 74%|███████▍  | 167040/225000 [321:47:13<90:16:30,  5.61s/it]
 74%|███████▍  | 167041/225000 [321:47:20<100:23:04,  6.24s/it]
 74%|███████▍  | 167042/225000 [321:47:25<92:32:42,  5.75s/it] 
 74%|███████▍  | 167043/225000 [321:47:30<86:31:19,  5.37s/it]
 74%|███████▍  | 167044/225000 [321:47:34<81:46:43,  5.08s/it]
 74%|███████▍  | 167045/225000 [321:47:38<77:06:43,  4.79s/it]
 74%|███████▍  | 167046/225000 [321:47:42<75:18:03,  4.68s/it]
 74%|███████▍  | 167047/225000 [321:47:47<75:00:17,  4.66s/it]
 74%|███████▍  | 167048/225000 [321:47:58<105:33:26,  6.56s/it]
 74%|███████▍  | 167049/225000 [321:48:03<96:27:06,  5.99s/it] 
 74%|███████▍  | 167050/225000 [321:48:06<84:18:57,  5.24s/it]
                                                              
{'loss': '0.7751', 'grad_norm': '1.243', 'learning_rate': '8.461e-06', 'epoch': '0.7424', 'num_input_tokens_seen': 532854856, 'train_runtime': '1.158e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167050/225000 [321:48:06<84:18:57,  5.24s/it]
 74%|███████▍  | 167051/225000 [321:48:16<105:08:10,  6.53s/it]
 74%|███████▍  | 167052/225000 [321:48:19<89:18:54,  5.55s/it] 
 74%|███████▍  | 167053/225000 [321:48:24<85:19:14,  5.30s/it]
 74%|███████▍  | 167054/225000 [321:48:31<96:18:53,  5.98s/it]
 74%|███████▍  | 167055/225000 [321:48:35<86:46:13,  5.39s/it]
 74%|███████▍  | 167056/225000 [321:48:39<78:52:29,  4.90s/it]
 74%|███████▍  | 167057/225000 [321:48:49<104:03:08,  6.46s/it]
 74%|███████▍  | 167058/225000 [321:48:52<88:10:18,  5.48s/it] 
 74%|███████▍  | 167059/225000 [321:49:00<100:24:03,  6.24s/it]
 74%|███████▍  | 167060/225000 [321:49:04<86:39:18,  5.38s/it] 
 74%|███████▍  | 167061/225000 [321:49:11<93:52:54,  5.83s/it]
 74%|███████▍  | 167062/225000 [321:49:14<82:28:51,  5.12s/it]
 74%|███████▍  | 167063/225000 [321:49:22<93:33:59,  5.81s/it]
 74%|███████▍  | 167064/225000 [321:49:26<84:29:27,  5.25s/it]
 74%|███████▍  | 167065/225000 [321:49:30<80:20:51,  4.99s/it]
 74%|███████▍  | 167066/225000 [321:49:35<82:15:36,  5.11s/it]
 74%|███████▍  | 167067/225000 [321:49:41<83:56:49,  5.22s/it]
 74%|███████▍  | 167068/225000 [321:49:46<83:37:03,  5.20s/it]
 74%|███████▍  | 167069/225000 [321:49:51<82:58:08,  5.16s/it]
 74%|███████▍  | 167070/225000 [321:49:56<82:50:36,  5.15s/it]
 74%|███████▍  | 167071/225000 [321:50:02<84:38:51,  5.26s/it]
 74%|███████▍  | 167072/225000 [321:50:09<95:46:39,  5.95s/it]
 74%|███████▍  | 167073/225000 [321:50:14<88:19:17,  5.49s/it]
 74%|███████▍  | 167074/225000 [321:50:18<80:53:12,  5.03s/it]
 74%|███████▍  | 167075/225000 [321:50:22<77:01:03,  4.79s/it]
 74%|███████▍  | 167076/225000 [321:50:26<76:17:45,  4.74s/it]
 74%|███████▍  | 167077/225000 [321:50:35<94:51:58,  5.90s/it]
 74%|███████▍  | 167078/225000 [321:50:44<108:16:55,  6.73s/it]
 74%|███████▍  | 167079/225000 [321:50:47<93:22:33,  5.80s/it] 
 74%|███████▍  | 167080/225000 [321:50:51<82:41:00,  5.14s/it]
 74%|███████▍  | 167081/225000 [321:50:57<86:55:06,  5.40s/it]
 74%|███████▍  | 167082/225000 [321:51:03<88:33:03,  5.50s/it]
 74%|███████▍  | 167083/225000 [321:51:08<89:32:09,  5.57s/it]
 74%|███████▍  | 167084/225000 [321:51:15<93:32:58,  5.81s/it]
 74%|███████▍  | 167085/225000 [321:51:20<92:01:59,  5.72s/it]
 74%|███████▍  | 167086/225000 [321:51:25<89:29:42,  5.56s/it]
 74%|███████▍  | 167087/225000 [321:51:31<88:32:20,  5.50s/it]
 74%|███████▍  | 167088/225000 [321:51:36<85:23:43,  5.31s/it]
 74%|███████▍  | 167089/225000 [321:51:41<84:40:24,  5.26s/it]
 74%|███████▍  | 167090/225000 [321:51:47<86:43:32,  5.39s/it]
 74%|███████▍  | 167091/225000 [321:51:52<87:09:11,  5.42s/it]
 74%|███████▍  | 167092/225000 [321:51:58<88:14:57,  5.49s/it]
 74%|███████▍  | 167093/225000 [321:52:03<87:38:21,  5.45s/it]
 74%|███████▍  | 167094/225000 [321:52:08<85:36:01,  5.32s/it]
 74%|███████▍  | 167095/225000 [321:52:15<92:58:13,  5.78s/it]
 74%|███████▍  | 167096/225000 [321:52:24<109:25:31,  6.80s/it]
 74%|███████▍  | 167097/225000 [321:52:29<98:06:44,  6.10s/it] 
 74%|███████▍  | 167098/225000 [321:52:33<89:59:07,  5.59s/it]
 74%|███████▍  | 167099/225000 [321:52:37<83:13:02,  5.17s/it]
 74%|███████▍  | 167100/225000 [321:52:47<104:52:24,  6.52s/it]
                                                               
{'loss': '0.7694', 'grad_norm': '1.101', 'learning_rate': '8.447e-06', 'epoch': '0.7427', 'num_input_tokens_seen': 532984752, 'train_runtime': '1.159e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167100/225000 [321:52:47<104:52:24,  6.52s/it]
 74%|███████▍  | 167101/225000 [321:52:52<97:24:17,  6.06s/it] 
 74%|███████▍  | 167102/225000 [321:52:57<93:05:58,  5.79s/it]
 74%|███████▍  | 167103/225000 [321:53:02<89:54:50,  5.59s/it]
 74%|███████▍  | 167104/225000 [321:53:07<88:27:59,  5.50s/it]
 74%|███████▍  | 167105/225000 [321:53:13<86:43:52,  5.39s/it]
 74%|███████▍  | 167106/225000 [321:53:18<85:01:19,  5.29s/it]
 74%|███████▍  | 167107/225000 [321:53:23<84:48:56,  5.27s/it]
 74%|███████▍  | 167108/225000 [321:53:28<84:49:30,  5.27s/it]
 74%|███████▍  | 167109/225000 [321:53:33<81:56:37,  5.10s/it]
 74%|███████▍  | 167110/225000 [321:53:38<83:09:44,  5.17s/it]
 74%|███████▍  | 167111/225000 [321:53:43<83:51:15,  5.21s/it]
 74%|███████▍  | 167112/225000 [321:53:49<87:34:24,  5.45s/it]
 74%|███████▍  | 167113/225000 [321:53:55<87:04:43,  5.42s/it]
 74%|███████▍  | 167114/225000 [321:54:00<85:49:41,  5.34s/it]
 74%|███████▍  | 167115/225000 [321:54:05<86:16:44,  5.37s/it]
 74%|███████▍  | 167116/225000 [321:54:11<87:42:16,  5.45s/it]
 74%|███████▍  | 167117/225000 [321:54:16<85:41:24,  5.33s/it]
 74%|███████▍  | 167118/225000 [321:54:21<85:29:53,  5.32s/it]
 74%|███████▍  | 167119/225000 [321:54:27<86:40:23,  5.39s/it]
 74%|███████▍  | 167120/225000 [321:54:32<86:18:37,  5.37s/it]
 74%|███████▍  | 167121/225000 [321:54:38<86:47:07,  5.40s/it]
 74%|███████▍  | 167122/225000 [321:54:43<87:43:26,  5.46s/it]
 74%|███████▍  | 167123/225000 [321:54:49<86:51:07,  5.40s/it]
 74%|███████▍  | 167124/225000 [321:54:54<88:47:14,  5.52s/it]
 74%|███████▍  | 167125/225000 [321:55:05<115:44:44,  7.20s/it]
 74%|███████▍  | 167126/225000 [321:55:11<108:08:18,  6.73s/it]
 74%|███████▍  | 167127/225000 [321:55:15<93:01:46,  5.79s/it] 
 74%|███████▍  | 167128/225000 [321:55:22<99:16:18,  6.18s/it]
 74%|███████▍  | 167129/225000 [321:55:25<85:53:49,  5.34s/it]
 74%|███████▍  | 167130/225000 [321:55:30<81:46:17,  5.09s/it]
 74%|███████▍  | 167131/225000 [321:55:34<79:34:51,  4.95s/it]
 74%|███████▍  | 167132/225000 [321:55:39<79:09:19,  4.92s/it]
 74%|███████▍  | 167133/225000 [321:55:44<78:48:10,  4.90s/it]
 74%|███████▍  | 167134/225000 [321:55:52<96:10:31,  5.98s/it]
 74%|███████▍  | 167135/225000 [321:56:02<113:51:58,  7.08s/it]
 74%|███████▍  | 167136/225000 [321:56:06<97:46:07,  6.08s/it] 
 74%|███████▍  | 167137/225000 [321:56:13<104:34:48,  6.51s/it]
 74%|███████▍  | 167138/225000 [321:56:18<97:49:54,  6.09s/it] 
 74%|███████▍  | 167139/225000 [321:56:23<88:22:36,  5.50s/it]
 74%|███████▍  | 167140/225000 [321:56:26<79:45:38,  4.96s/it]
 74%|███████▍  | 167141/225000 [321:56:31<80:42:27,  5.02s/it]
 74%|███████▍  | 167142/225000 [321:56:39<94:02:29,  5.85s/it]
 74%|███████▍  | 167143/225000 [321:56:44<87:42:12,  5.46s/it]
 74%|███████▍  | 167144/225000 [321:56:49<85:46:28,  5.34s/it]
 74%|███████▍  | 167145/225000 [321:56:53<80:53:00,  5.03s/it]
 74%|███████▍  | 167146/225000 [321:56:58<78:50:13,  4.91s/it]
 74%|███████▍  | 167147/225000 [321:57:02<76:41:29,  4.77s/it]
 74%|███████▍  | 167148/225000 [321:57:07<76:53:30,  4.78s/it]
 74%|███████▍  | 167149/225000 [321:57:12<77:42:47,  4.84s/it]
 74%|███████▍  | 167150/225000 [321:57:20<91:26:36,  5.69s/it]
                                                              
{'loss': '0.7653', 'grad_norm': '1.173', 'learning_rate': '8.434e-06', 'epoch': '0.7429', 'num_input_tokens_seen': 533115088, 'train_runtime': '1.159e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167150/225000 [321:57:20<91:26:36,  5.69s/it]
 74%|███████▍  | 167151/225000 [321:57:29<107:41:13,  6.70s/it]
 74%|███████▍  | 167152/225000 [321:57:32<93:11:22,  5.80s/it] 
 74%|███████▍  | 167153/225000 [321:57:37<86:00:22,  5.35s/it]
 74%|███████▍  | 167154/225000 [321:57:47<111:44:03,  6.95s/it]
 74%|███████▍  | 167155/225000 [321:57:51<94:55:00,  5.91s/it] 
 74%|███████▍  | 167156/225000 [321:58:00<111:11:19,  6.92s/it]
 74%|███████▍  | 167157/225000 [321:58:04<94:33:59,  5.89s/it] 
 74%|███████▍  | 167158/225000 [321:58:11<101:40:07,  6.33s/it]
 74%|███████▍  | 167159/225000 [321:58:14<85:46:19,  5.34s/it] 
 74%|███████▍  | 167160/225000 [321:58:20<88:14:25,  5.49s/it]
 74%|███████▍  | 167161/225000 [321:58:23<78:38:06,  4.89s/it]
 74%|███████▍  | 167162/225000 [321:58:32<97:20:03,  6.06s/it]
 74%|███████▍  | 167163/225000 [321:58:36<84:50:47,  5.28s/it]
 74%|███████▍  | 167164/225000 [321:58:44<97:47:08,  6.09s/it]
 74%|███████▍  | 167165/225000 [321:58:47<85:17:06,  5.31s/it]
 74%|███████▍  | 167166/225000 [321:58:54<92:04:15,  5.73s/it]
 74%|███████▍  | 167167/225000 [321:58:57<81:18:16,  5.06s/it]
 74%|███████▍  | 167168/225000 [321:59:03<84:34:14,  5.26s/it]
 74%|███████▍  | 167169/225000 [321:59:11<96:34:52,  6.01s/it]
 74%|███████▍  | 167170/225000 [321:59:16<92:41:37,  5.77s/it]
 74%|███████▍  | 167171/225000 [321:59:21<88:27:24,  5.51s/it]
 74%|███████▍  | 167172/225000 [321:59:26<86:04:56,  5.36s/it]
 74%|███████▍  | 167173/225000 [321:59:30<81:58:01,  5.10s/it]
 74%|███████▍  | 167174/225000 [321:59:35<81:03:49,  5.05s/it]
 74%|███████▍  | 167175/225000 [321:59:40<80:31:44,  5.01s/it]
 74%|███████▍  | 167176/225000 [321:59:45<80:30:06,  5.01s/it]
 74%|███████▍  | 167177/225000 [321:59:50<80:49:38,  5.03s/it]
 74%|███████▍  | 167178/225000 [321:59:55<79:22:07,  4.94s/it]
 74%|███████▍  | 167179/225000 [322:00:00<79:39:31,  4.96s/it]
 74%|███████▍  | 167180/225000 [322:00:05<81:28:33,  5.07s/it]
 74%|███████▍  | 167181/225000 [322:00:11<81:58:45,  5.10s/it]
 74%|███████▍  | 167182/225000 [322:00:15<80:35:10,  5.02s/it]
 74%|███████▍  | 167183/225000 [322:00:26<109:07:15,  6.79s/it]
 74%|███████▍  | 167184/225000 [322:00:32<103:31:30,  6.45s/it]
 74%|███████▍  | 167185/225000 [322:00:40<109:26:15,  6.81s/it]
 74%|███████▍  | 167186/225000 [322:00:43<93:13:42,  5.81s/it] 
 74%|███████▍  | 167187/225000 [322:00:52<107:44:26,  6.71s/it]
 74%|███████▍  | 167188/225000 [322:00:56<96:16:59,  6.00s/it] 
 74%|███████▍  | 167189/225000 [322:01:00<86:50:54,  5.41s/it]
 74%|███████▍  | 167190/225000 [322:01:04<80:07:08,  4.99s/it]
 74%|███████▍  | 167191/225000 [322:01:09<78:38:03,  4.90s/it]
 74%|███████▍  | 167192/225000 [322:01:20<109:16:50,  6.81s/it]
 74%|███████▍  | 167193/225000 [322:01:24<93:08:24,  5.80s/it] 
 74%|███████▍  | 167194/225000 [322:01:29<88:13:31,  5.49s/it]
 74%|███████▍  | 167195/225000 [322:01:33<83:49:57,  5.22s/it]
 74%|███████▍  | 167196/225000 [322:01:38<84:31:00,  5.26s/it]
 74%|███████▍  | 167197/225000 [322:01:44<83:51:02,  5.22s/it]
 74%|███████▍  | 167198/225000 [322:01:48<81:15:28,  5.06s/it]
 74%|███████▍  | 167199/225000 [322:01:53<80:05:21,  4.99s/it]
 74%|███████▍  | 167200/225000 [322:01:59<82:42:36,  5.15s/it]
                                                              
{'loss': '0.7643', 'grad_norm': '1.107', 'learning_rate': '8.42e-06', 'epoch': '0.7431', 'num_input_tokens_seen': 533245560, 'train_runtime': '1.159e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167200/225000 [322:01:59<82:42:36,  5.15s/it]
 74%|███████▍  | 167201/225000 [322:02:03<80:46:04,  5.03s/it]
 74%|███████▍  | 167202/225000 [322:02:09<81:42:42,  5.09s/it]
 74%|███████▍  | 167203/225000 [322:02:14<84:09:37,  5.24s/it]
 74%|███████▍  | 167204/225000 [322:02:19<84:15:25,  5.25s/it]
 74%|███████▍  | 167205/225000 [322:02:24<82:24:25,  5.13s/it]
 74%|███████▍  | 167206/225000 [322:02:30<84:02:39,  5.24s/it]
 74%|███████▍  | 167207/225000 [322:02:35<84:13:13,  5.25s/it]
 74%|███████▍  | 167208/225000 [322:02:41<85:07:14,  5.30s/it]
 74%|███████▍  | 167209/225000 [322:02:51<110:51:54,  6.91s/it]
 74%|███████▍  | 167210/225000 [322:02:57<107:43:53,  6.71s/it]
 74%|███████▍  | 167211/225000 [322:03:05<112:07:51,  6.99s/it]
 74%|███████▍  | 167212/225000 [322:03:09<98:04:32,  6.11s/it] 
 74%|███████▍  | 167213/225000 [322:03:17<104:28:48,  6.51s/it]
 74%|███████▍  | 167214/225000 [322:03:20<91:13:48,  5.68s/it] 
 74%|███████▍  | 167215/225000 [322:03:29<104:18:34,  6.50s/it]
 74%|███████▍  | 167216/225000 [322:03:32<88:40:56,  5.53s/it] 
 74%|███████▍  | 167217/225000 [322:03:38<93:27:48,  5.82s/it]
 74%|███████▍  | 167218/225000 [322:03:45<94:38:17,  5.90s/it]
 74%|███████▍  | 167219/225000 [322:03:50<90:52:13,  5.66s/it]
 74%|███████▍  | 167220/225000 [322:03:55<87:51:21,  5.47s/it]
 74%|███████▍  | 167221/225000 [322:04:00<84:50:25,  5.29s/it]
 74%|███████▍  | 167222/225000 [322:04:04<82:39:26,  5.15s/it]
 74%|███████▍  | 167223/225000 [322:04:10<85:04:30,  5.30s/it]
 74%|███████▍  | 167224/225000 [322:04:15<84:54:11,  5.29s/it]
 74%|███████▍  | 167225/225000 [322:04:20<82:08:15,  5.12s/it]
 74%|███████▍  | 167226/225000 [322:04:25<82:49:34,  5.16s/it]
 74%|███████▍  | 167227/225000 [322:04:30<83:04:29,  5.18s/it]
 74%|███████▍  | 167228/225000 [322:04:36<82:59:46,  5.17s/it]
 74%|███████▍  | 167229/225000 [322:04:41<82:34:59,  5.15s/it]
 74%|███████▍  | 167230/225000 [322:04:46<82:57:02,  5.17s/it]
 74%|███████▍  | 167231/225000 [322:04:51<80:18:15,  5.00s/it]
 74%|███████▍  | 167232/225000 [322:04:56<81:49:53,  5.10s/it]
 74%|███████▍  | 167233/225000 [322:05:08<115:19:29,  7.19s/it]
 74%|███████▍  | 167234/225000 [322:05:14<108:39:21,  6.77s/it]
 74%|███████▍  | 167235/225000 [322:05:17<92:00:50,  5.73s/it] 
 74%|███████▍  | 167236/225000 [322:05:27<111:26:54,  6.95s/it]
 74%|███████▍  | 167237/225000 [322:05:30<93:18:19,  5.82s/it] 
 74%|███████▍  | 167238/225000 [322:05:35<88:28:14,  5.51s/it]
 74%|███████▍  | 167239/225000 [322:05:40<88:05:29,  5.49s/it]
 74%|███████▍  | 167240/225000 [322:05:46<87:56:23,  5.48s/it]
 74%|███████▍  | 167241/225000 [322:05:51<85:26:11,  5.33s/it]
 74%|███████▍  | 167242/225000 [322:05:56<85:21:24,  5.32s/it]
 74%|███████▍  | 167243/225000 [322:06:01<82:34:19,  5.15s/it]
 74%|███████▍  | 167244/225000 [322:06:06<83:37:14,  5.21s/it]
 74%|███████▍  | 167245/225000 [322:06:11<83:20:41,  5.20s/it]
 74%|███████▍  | 167246/225000 [322:06:16<83:00:20,  5.17s/it]
 74%|███████▍  | 167247/225000 [322:06:21<82:36:24,  5.15s/it]
 74%|███████▍  | 167248/225000 [322:06:27<82:00:09,  5.11s/it]
 74%|███████▍  | 167249/225000 [322:06:32<84:02:56,  5.24s/it]
 74%|███████▍  | 167250/225000 [322:06:37<82:28:10,  5.14s/it]
                                                              
{'loss': '0.7782', 'grad_norm': '1.404', 'learning_rate': '8.406e-06', 'epoch': '0.7433', 'num_input_tokens_seen': 533375744, 'train_runtime': '1.16e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167250/225000 [322:06:37<82:28:10,  5.14s/it]
 74%|███████▍  | 167251/225000 [322:06:42<81:54:34,  5.11s/it]
 74%|███████▍  | 167252/225000 [322:06:47<82:38:10,  5.15s/it]
 74%|███████▍  | 167253/225000 [322:06:52<83:01:24,  5.18s/it]
 74%|███████▍  | 167254/225000 [322:06:58<84:32:36,  5.27s/it]
 74%|███████▍  | 167255/225000 [322:07:03<81:08:33,  5.06s/it]
 74%|███████▍  | 167256/225000 [322:07:07<79:59:34,  4.99s/it]
 74%|███████▍  | 167257/225000 [322:07:13<81:05:56,  5.06s/it]
 74%|███████▍  | 167258/225000 [322:07:18<82:40:03,  5.15s/it]
 74%|███████▍  | 167259/225000 [322:07:30<113:47:24,  7.09s/it]
 74%|███████▍  | 167260/225000 [322:07:37<115:14:54,  7.19s/it]
 74%|███████▍  | 167261/225000 [322:07:41<97:51:31,  6.10s/it] 
 74%|███████▍  | 167262/225000 [322:07:48<104:35:57,  6.52s/it]
 74%|███████▍  | 167263/225000 [322:07:53<99:04:15,  6.18s/it] 
 74%|███████▍  | 167264/225000 [322:08:00<102:02:35,  6.36s/it]
 74%|███████▍  | 167265/225000 [322:08:04<91:26:15,  5.70s/it] 
 74%|███████▍  | 167266/225000 [322:08:11<97:27:10,  6.08s/it]
 74%|███████▍  | 167267/225000 [322:08:15<84:21:46,  5.26s/it]
 74%|███████▍  | 167268/225000 [322:08:20<86:19:51,  5.38s/it]
 74%|███████▍  | 167269/225000 [322:08:27<94:46:08,  5.91s/it]
 74%|███████▍  | 167270/225000 [322:08:31<83:41:48,  5.22s/it]
 74%|███████▍  | 167271/225000 [322:08:41<104:51:04,  6.54s/it]
 74%|███████▍  | 167272/225000 [322:08:44<89:36:19,  5.59s/it] 
 74%|███████▍  | 167273/225000 [322:08:52<99:00:39,  6.17s/it]
 74%|███████▍  | 167274/225000 [322:08:55<85:31:24,  5.33s/it]
 74%|███████▍  | 167275/225000 [322:08:59<81:00:08,  5.05s/it]
 74%|███████▍  | 167276/225000 [322:09:06<89:09:27,  5.56s/it]
 74%|███████▍  | 167277/225000 [322:09:13<94:16:34,  5.88s/it]
 74%|███████▍  | 167278/225000 [322:09:18<92:16:49,  5.76s/it]
 74%|███████▍  | 167279/225000 [322:09:22<81:08:29,  5.06s/it]
 74%|███████▍  | 167280/225000 [322:09:30<94:56:17,  5.92s/it]
 74%|███████▍  | 167281/225000 [322:09:33<83:24:57,  5.20s/it]
 74%|███████▍  | 167282/225000 [322:09:37<77:05:12,  4.81s/it]
 74%|███████▍  | 167283/225000 [322:09:42<80:04:49,  4.99s/it]
 74%|███████▍  | 167284/225000 [322:09:49<89:28:18,  5.58s/it]
 74%|███████▍  | 167285/225000 [322:09:53<79:47:10,  4.98s/it]
 74%|███████▍  | 167286/225000 [322:09:57<76:10:17,  4.75s/it]
 74%|███████▍  | 167287/225000 [322:10:06<95:31:00,  5.96s/it]
 74%|███████▍  | 167288/225000 [322:10:10<84:38:36,  5.28s/it]
 74%|███████▍  | 167289/225000 [322:10:18<98:59:39,  6.18s/it]
 74%|███████▍  | 167290/225000 [322:10:21<86:04:25,  5.37s/it]
 74%|███████▍  | 167291/225000 [322:10:30<101:40:35,  6.34s/it]
 74%|███████▍  | 167292/225000 [322:10:34<88:45:47,  5.54s/it] 
 74%|███████▍  | 167293/225000 [322:10:41<99:14:42,  6.19s/it]
 74%|███████▍  | 167294/225000 [322:10:45<85:15:49,  5.32s/it]
 74%|███████▍  | 167295/225000 [322:10:49<81:24:11,  5.08s/it]
 74%|███████▍  | 167296/225000 [322:10:54<81:35:11,  5.09s/it]
 74%|███████▍  | 167297/225000 [322:10:59<80:34:08,  5.03s/it]
 74%|███████▍  | 167298/225000 [322:11:04<77:49:04,  4.86s/it]
 74%|███████▍  | 167299/225000 [322:11:08<76:52:18,  4.80s/it]
 74%|███████▍  | 167300/225000 [322:11:13<77:01:35,  4.81s/it]
                                                              
{'loss': '0.769', 'grad_norm': '1.201', 'learning_rate': '8.393e-06', 'epoch': '0.7436', 'num_input_tokens_seen': 533504640, 'train_runtime': '1.16e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167300/225000 [322:11:13<77:01:35,  4.81s/it]
 74%|███████▍  | 167301/225000 [322:11:18<75:43:32,  4.72s/it]
 74%|███████▍  | 167302/225000 [322:11:24<83:08:27,  5.19s/it]
 74%|███████▍  | 167303/225000 [322:11:35<113:06:24,  7.06s/it]
 74%|███████▍  | 167304/225000 [322:11:39<95:30:36,  5.96s/it] 
 74%|███████▍  | 167305/225000 [322:11:42<84:42:40,  5.29s/it]
 74%|███████▍  | 167306/225000 [322:11:49<91:50:17,  5.73s/it]
 74%|███████▍  | 167307/225000 [322:11:55<93:52:23,  5.86s/it]
 74%|███████▍  | 167308/225000 [322:12:00<85:35:33,  5.34s/it]
 74%|███████▍  | 167309/225000 [322:12:06<89:48:45,  5.60s/it]
 74%|███████▍  | 167310/225000 [322:12:12<92:05:36,  5.75s/it]
 74%|███████▍  | 167311/225000 [322:12:15<81:36:33,  5.09s/it]
 74%|███████▍  | 167312/225000 [322:12:19<76:21:12,  4.76s/it]
 74%|███████▍  | 167313/225000 [322:12:25<78:54:23,  4.92s/it]
 74%|███████▍  | 167314/225000 [322:12:30<83:08:03,  5.19s/it]
 74%|███████▍  | 167315/225000 [322:12:36<83:39:12,  5.22s/it]
 74%|███████▍  | 167316/225000 [322:12:41<83:22:58,  5.20s/it]
 74%|███████▍  | 167317/225000 [322:12:46<82:54:28,  5.17s/it]
 74%|███████▍  | 167318/225000 [322:12:51<79:44:35,  4.98s/it]
 74%|███████▍  | 167319/225000 [322:12:56<80:00:05,  4.99s/it]
 74%|███████▍  | 167320/225000 [322:13:01<80:28:22,  5.02s/it]
 74%|███████▍  | 167321/225000 [322:13:13<115:37:00,  7.22s/it]
 74%|███████▍  | 167322/225000 [322:13:17<98:40:15,  6.16s/it] 
 74%|███████▍  | 167323/225000 [322:13:24<101:45:09,  6.35s/it]
 74%|███████▍  | 167324/225000 [322:13:28<91:49:02,  5.73s/it] 
 74%|███████▍  | 167325/225000 [322:13:32<85:10:34,  5.32s/it]
 74%|███████▍  | 167326/225000 [322:13:37<80:47:08,  5.04s/it]
 74%|███████▍  | 167327/225000 [322:13:44<93:42:57,  5.85s/it]
 74%|███████▍  | 167328/225000 [322:13:52<104:00:13,  6.49s/it]
 74%|███████▍  | 167329/225000 [322:13:56<90:14:39,  5.63s/it] 
 74%|███████▍  | 167330/225000 [322:14:00<83:44:20,  5.23s/it]
 74%|███████▍  | 167331/225000 [322:14:04<78:07:56,  4.88s/it]
 74%|███████▍  | 167332/225000 [322:14:09<79:36:52,  4.97s/it]
 74%|███████▍  | 167333/225000 [322:14:20<108:12:41,  6.76s/it]
 74%|███████▍  | 167334/225000 [322:14:25<96:47:38,  6.04s/it] 
 74%|███████▍  | 167335/225000 [322:14:29<87:13:39,  5.45s/it]
 74%|███████▍  | 167336/225000 [322:14:33<80:12:19,  5.01s/it]
 74%|███████▍  | 167337/225000 [322:14:43<104:09:33,  6.50s/it]
 74%|███████▍  | 167338/225000 [322:14:48<97:18:47,  6.08s/it] 
 74%|███████▍  | 167339/225000 [322:14:52<86:54:58,  5.43s/it]
 74%|███████▍  | 167340/225000 [322:14:58<93:08:13,  5.82s/it]
 74%|███████▍  | 167341/225000 [322:15:05<97:58:51,  6.12s/it]
 74%|███████▍  | 167342/225000 [322:15:09<84:52:31,  5.30s/it]
 74%|███████▍  | 167343/225000 [322:15:15<91:00:49,  5.68s/it]
 74%|███████▍  | 167344/225000 [322:15:20<86:10:16,  5.38s/it]
 74%|███████▍  | 167345/225000 [322:15:26<91:03:45,  5.69s/it]
 74%|███████▍  | 167346/225000 [322:15:30<81:54:34,  5.11s/it]
 74%|███████▍  | 167347/225000 [322:15:34<78:07:57,  4.88s/it]
 74%|███████▍  | 167348/225000 [322:15:39<75:13:23,  4.70s/it]
 74%|███████▍  | 167349/225000 [322:15:43<74:05:28,  4.63s/it]
 74%|███████▍  | 167350/225000 [322:15:47<71:50:12,  4.49s/it]
                                                              
{'loss': '0.7763', 'grad_norm': '1.228', 'learning_rate': '8.379e-06', 'epoch': '0.7438', 'num_input_tokens_seen': 533635128, 'train_runtime': '1.16e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167350/225000 [322:15:47<71:50:12,  4.49s/it]
 74%|███████▍  | 167351/225000 [322:15:59<106:09:54,  6.63s/it]
 74%|███████▍  | 167352/225000 [322:16:02<91:06:57,  5.69s/it] 
 74%|███████▍  | 167353/225000 [322:16:09<97:24:16,  6.08s/it]
 74%|███████▍  | 167354/225000 [322:16:13<84:31:44,  5.28s/it]
 74%|███████▍  | 167355/225000 [322:16:21<96:52:31,  6.05s/it]
 74%|███████▍  | 167356/225000 [322:16:24<83:43:58,  5.23s/it]
 74%|███████▍  | 167357/225000 [322:16:29<80:31:07,  5.03s/it]
 74%|███████▍  | 167358/225000 [322:16:34<81:49:39,  5.11s/it]
 74%|███████▍  | 167359/225000 [322:16:39<82:17:07,  5.14s/it]
 74%|███████▍  | 167360/225000 [322:16:44<80:37:30,  5.04s/it]
 74%|███████▍  | 167361/225000 [322:16:49<82:28:50,  5.15s/it]
 74%|███████▍  | 167362/225000 [322:16:54<81:23:28,  5.08s/it]
 74%|███████▍  | 167363/225000 [322:16:59<81:28:06,  5.09s/it]
 74%|███████▍  | 167364/225000 [322:17:05<82:50:29,  5.17s/it]
 74%|███████▍  | 167365/225000 [322:17:10<82:43:37,  5.17s/it]
 74%|███████▍  | 167366/225000 [322:17:15<84:32:04,  5.28s/it]
 74%|███████▍  | 167367/225000 [322:17:21<85:37:05,  5.35s/it]
 74%|███████▍  | 167368/225000 [322:17:26<84:30:20,  5.28s/it]
 74%|███████▍  | 167369/225000 [322:17:32<85:42:59,  5.35s/it]
 74%|███████▍  | 167370/225000 [322:17:42<108:58:41,  6.81s/it]
 74%|███████▍  | 167371/225000 [322:17:49<111:42:00,  6.98s/it]
 74%|███████▍  | 167372/225000 [322:17:52<93:49:34,  5.86s/it] 
 74%|███████▍  | 167373/225000 [322:18:01<106:49:47,  6.67s/it]
 74%|███████▍  | 167374/225000 [322:18:04<89:56:17,  5.62s/it] 
 74%|███████▍  | 167375/225000 [322:18:13<104:49:14,  6.55s/it]
 74%|███████▍  | 167376/225000 [322:18:16<88:20:03,  5.52s/it] 
 74%|███████▍  | 167377/225000 [322:18:22<91:25:43,  5.71s/it]
 74%|███████▍  | 167378/225000 [322:18:29<96:42:35,  6.04s/it]
 74%|███████▍  | 167379/225000 [322:18:32<82:22:52,  5.15s/it]
 74%|███████▍  | 167380/225000 [322:18:39<89:02:25,  5.56s/it]
 74%|███████▍  | 167381/225000 [322:18:44<89:31:29,  5.59s/it]
 74%|███████▍  | 167382/225000 [322:18:49<84:46:37,  5.30s/it]
 74%|███████▍  | 167383/225000 [322:18:53<77:31:01,  4.84s/it]
 74%|███████▍  | 167384/225000 [322:19:02<98:26:47,  6.15s/it]
 74%|███████▍  | 167385/225000 [322:19:05<84:33:50,  5.28s/it]
 74%|███████▍  | 167386/225000 [322:19:08<75:36:17,  4.72s/it]
 74%|███████▍  | 167387/225000 [322:19:18<99:36:11,  6.22s/it]
 74%|███████▍  | 167388/225000 [322:19:23<91:33:40,  5.72s/it]
 74%|███████▍  | 167389/225000 [322:19:29<94:32:05,  5.91s/it]
 74%|███████▍  | 167390/225000 [322:19:33<83:59:14,  5.25s/it]
 74%|███████▍  | 167391/225000 [322:19:41<98:16:09,  6.14s/it]
 74%|███████▍  | 167392/225000 [322:19:44<84:37:24,  5.29s/it]
 74%|███████▍  | 167393/225000 [322:19:50<86:55:29,  5.43s/it]
 74%|███████▍  | 167394/225000 [322:19:55<84:35:29,  5.29s/it]
 74%|███████▍  | 167395/225000 [322:20:01<89:58:18,  5.62s/it]
 74%|███████▍  | 167396/225000 [322:20:08<94:09:47,  5.88s/it]
 74%|███████▍  | 167397/225000 [322:20:12<85:04:41,  5.32s/it]
 74%|███████▍  | 167398/225000 [322:20:20<97:26:04,  6.09s/it]
 74%|███████▍  | 167399/225000 [322:20:24<86:04:43,  5.38s/it]
 74%|███████▍  | 167400/225000 [322:20:27<78:23:55,  4.90s/it]
                                                              
{'loss': '0.7713', 'grad_norm': '1.079', 'learning_rate': '8.366e-06', 'epoch': '0.744', 'num_input_tokens_seen': 533764840, 'train_runtime': '1.16e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167400/225000 [322:20:27<78:23:55,  4.90s/it]
 74%|███████▍  | 167401/225000 [322:20:32<79:10:45,  4.95s/it]
 74%|███████▍  | 167402/225000 [322:20:37<77:54:20,  4.87s/it]
 74%|███████▍  | 167403/225000 [322:20:42<78:48:24,  4.93s/it]
 74%|███████▍  | 167404/225000 [322:20:47<79:27:45,  4.97s/it]
 74%|███████▍  | 167405/225000 [322:20:52<79:08:59,  4.95s/it]
 74%|███████▍  | 167406/225000 [322:20:58<81:24:44,  5.09s/it]
 74%|███████▍  | 167407/225000 [322:21:03<82:18:09,  5.14s/it]
 74%|███████▍  | 167408/225000 [322:21:08<81:14:21,  5.08s/it]
 74%|███████▍  | 167409/225000 [322:21:13<80:53:15,  5.06s/it]
 74%|███████▍  | 167410/225000 [322:21:18<82:10:38,  5.14s/it]
 74%|███████▍  | 167411/225000 [322:21:23<80:28:06,  5.03s/it]
 74%|███████▍  | 167412/225000 [322:21:28<81:34:59,  5.10s/it]
 74%|███████▍  | 167413/225000 [322:21:34<83:23:35,  5.21s/it]
 74%|███████▍  | 167414/225000 [322:21:38<81:54:50,  5.12s/it]
 74%|███████▍  | 167415/225000 [322:21:46<91:14:04,  5.70s/it]
 74%|███████▍  | 167416/225000 [322:21:57<119:49:10,  7.49s/it]
 74%|███████▍  | 167417/225000 [322:22:01<102:12:32,  6.39s/it]
 74%|███████▍  | 167418/225000 [322:22:05<88:44:12,  5.55s/it] 
 74%|███████▍  | 167419/225000 [322:22:14<105:29:02,  6.59s/it]
 74%|███████▍  | 167420/225000 [322:22:17<90:37:30,  5.67s/it] 
 74%|███████▍  | 167421/225000 [322:22:21<83:58:18,  5.25s/it]
 74%|███████▍  | 167422/225000 [322:22:26<80:05:07,  5.01s/it]
 74%|███████▍  | 167423/225000 [322:22:31<79:54:13,  5.00s/it]
 74%|███████▍  | 167424/225000 [322:22:44<118:05:52,  7.38s/it]
 74%|███████▍  | 167425/225000 [322:22:48<102:30:35,  6.41s/it]
 74%|███████▍  | 167426/225000 [322:22:52<90:21:27,  5.65s/it] 
 74%|███████▍  | 167427/225000 [322:22:56<85:27:50,  5.34s/it]
 74%|███████▍  | 167428/225000 [322:23:01<83:23:29,  5.21s/it]
 74%|███████▍  | 167429/225000 [322:23:06<80:51:57,  5.06s/it]
 74%|███████▍  | 167430/225000 [322:23:11<79:00:41,  4.94s/it]
 74%|███████▍  | 167431/225000 [322:23:21<106:49:11,  6.68s/it]
 74%|███████▍  | 167432/225000 [322:23:27<100:02:58,  6.26s/it]
 74%|███████▍  | 167433/225000 [322:23:33<100:15:32,  6.27s/it]
 74%|███████▍  | 167434/225000 [322:23:38<92:54:07,  5.81s/it] 
 74%|███████▍  | 167435/225000 [322:23:43<88:32:25,  5.54s/it]
 74%|███████▍  | 167436/225000 [322:23:48<85:20:17,  5.34s/it]
 74%|███████▍  | 167437/225000 [322:23:52<82:15:19,  5.14s/it]
 74%|███████▍  | 167438/225000 [322:23:58<83:01:50,  5.19s/it]
 74%|███████▍  | 167439/225000 [322:24:03<84:50:34,  5.31s/it]
 74%|███████▍  | 167440/225000 [322:24:15<114:59:48,  7.19s/it]
 74%|███████▍  | 167441/225000 [322:24:18<97:19:50,  6.09s/it] 
 74%|███████▍  | 167442/225000 [322:24:26<104:33:06,  6.54s/it]
 74%|███████▍  | 167443/225000 [322:24:29<90:10:45,  5.64s/it] 
 74%|███████▍  | 167444/225000 [322:24:33<82:04:54,  5.13s/it]
 74%|███████▍  | 167445/225000 [322:24:38<80:30:04,  5.04s/it]
 74%|███████▍  | 167446/225000 [322:24:43<78:52:07,  4.93s/it]
 74%|███████▍  | 167447/225000 [322:24:48<80:32:01,  5.04s/it]
 74%|███████▍  | 167448/225000 [322:24:53<81:46:30,  5.12s/it]
 74%|███████▍  | 167449/225000 [322:24:58<80:20:27,  5.03s/it]
 74%|███████▍  | 167450/225000 [322:25:03<80:05:14,  5.01s/it]
                                                              
{'loss': '0.7703', 'grad_norm': '1.12', 'learning_rate': '8.352e-06', 'epoch': '0.7442', 'num_input_tokens_seen': 533896008, 'train_runtime': '1.161e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167450/225000 [322:25:03<80:05:14,  5.01s/it]
 74%|███████▍  | 167451/225000 [322:25:09<82:15:27,  5.15s/it]
 74%|███████▍  | 167452/225000 [322:25:14<81:13:57,  5.08s/it]
 74%|███████▍  | 167453/225000 [322:25:18<79:58:47,  5.00s/it]
 74%|███████▍  | 167454/225000 [322:25:23<79:19:34,  4.96s/it]
 74%|███████▍  | 167455/225000 [322:25:28<79:31:07,  4.97s/it]
 74%|███████▍  | 167456/225000 [322:25:34<81:02:38,  5.07s/it]
 74%|███████▍  | 167457/225000 [322:25:39<83:29:11,  5.22s/it]
 74%|███████▍  | 167458/225000 [322:25:44<82:18:11,  5.15s/it]
 74%|███████▍  | 167459/225000 [322:25:53<99:51:43,  6.25s/it]
 74%|███████▍  | 167460/225000 [322:26:04<124:41:13,  7.80s/it]
 74%|███████▍  | 167461/225000 [322:26:08<106:49:27,  6.68s/it]
 74%|███████▍  | 167462/225000 [322:26:13<94:57:58,  5.94s/it] 
 74%|███████▍  | 167463/225000 [322:26:17<86:53:32,  5.44s/it]
 74%|███████▍  | 167464/225000 [322:26:23<90:06:12,  5.64s/it]
 74%|███████▍  | 167465/225000 [322:26:31<102:02:39,  6.38s/it]
 74%|███████▍  | 167466/225000 [322:26:35<89:10:38,  5.58s/it] 
 74%|███████▍  | 167467/225000 [322:26:38<78:37:18,  4.92s/it]
 74%|███████▍  | 167468/225000 [322:26:47<99:46:10,  6.24s/it]
 74%|███████▍  | 167469/225000 [322:26:51<85:33:50,  5.35s/it]
 74%|███████▍  | 167470/225000 [322:26:59<101:33:58,  6.36s/it]
 74%|███████▍  | 167471/225000 [322:27:04<94:11:59,  5.89s/it] 
 74%|███████▍  | 167472/225000 [322:27:09<90:38:54,  5.67s/it]
 74%|███████▍  | 167473/225000 [322:27:14<87:22:23,  5.47s/it]
 74%|███████▍  | 167474/225000 [322:27:20<87:18:55,  5.46s/it]
 74%|███████▍  | 167475/225000 [322:27:25<84:29:54,  5.29s/it]
 74%|███████▍  | 167476/225000 [322:27:30<83:35:01,  5.23s/it]
 74%|███████▍  | 167477/225000 [322:27:35<84:07:52,  5.27s/it]
 74%|███████▍  | 167478/225000 [322:27:40<82:55:30,  5.19s/it]
 74%|███████▍  | 167479/225000 [322:27:45<80:32:43,  5.04s/it]
 74%|███████▍  | 167480/225000 [322:27:50<81:39:25,  5.11s/it]
 74%|███████▍  | 167481/225000 [322:27:56<83:03:57,  5.20s/it]
 74%|███████▍  | 167482/225000 [322:28:01<83:46:35,  5.24s/it]
 74%|███████▍  | 167483/225000 [322:28:06<82:06:11,  5.14s/it]
 74%|███████▍  | 167484/225000 [322:28:11<80:28:33,  5.04s/it]
 74%|███████▍  | 167485/225000 [322:28:16<81:31:42,  5.10s/it]
 74%|███████▍  | 167486/225000 [322:28:21<81:44:16,  5.12s/it]
 74%|███████▍  | 167487/225000 [322:28:31<106:30:28,  6.67s/it]
 74%|███████▍  | 167488/225000 [322:28:37<103:23:05,  6.47s/it]
 74%|███████▍  | 167489/225000 [322:28:41<89:15:03,  5.59s/it] 
 74%|███████▍  | 167490/225000 [322:28:46<87:56:55,  5.51s/it]
 74%|███████▍  | 167491/225000 [322:28:54<98:33:52,  6.17s/it]
 74%|███████▍  | 167492/225000 [322:28:57<84:55:40,  5.32s/it]
 74%|███████▍  | 167493/225000 [322:29:04<93:06:22,  5.83s/it]
 74%|███████▍  | 167494/225000 [322:29:10<94:22:11,  5.91s/it]
 74%|███████▍  | 167495/225000 [322:29:15<87:55:46,  5.50s/it]
 74%|███████▍  | 167496/225000 [322:29:20<83:52:42,  5.25s/it]
 74%|███████▍  | 167497/225000 [322:29:24<81:33:52,  5.11s/it]
 74%|███████▍  | 167498/225000 [322:29:29<79:38:01,  4.99s/it]
 74%|███████▍  | 167499/225000 [322:29:34<79:56:38,  5.01s/it]
 74%|███████▍  | 167500/225000 [322:29:39<81:50:49,  5.12s/it]
                                                              
{'loss': '0.77', 'grad_norm': '1.058', 'learning_rate': '8.338e-06', 'epoch': '0.7444', 'num_input_tokens_seen': 534025616, 'train_runtime': '1.161e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167500/225000 [322:29:39<81:50:49,  5.12s/it]
 74%|███████▍  | 167501/225000 [322:29:44<79:08:06,  4.95s/it]
 74%|███████▍  | 167502/225000 [322:29:49<79:51:40,  5.00s/it]
 74%|███████▍  | 167503/225000 [322:29:54<80:02:34,  5.01s/it]
 74%|███████▍  | 167504/225000 [322:30:02<93:40:07,  5.86s/it]
 74%|███████▍  | 167505/225000 [322:30:13<117:14:11,  7.34s/it]
 74%|███████▍  | 167506/225000 [322:30:18<106:44:39,  6.68s/it]
 74%|███████▍  | 167507/225000 [322:30:23<97:57:30,  6.13s/it] 
 74%|███████▍  | 167508/225000 [322:30:28<93:31:51,  5.86s/it]
 74%|███████▍  | 167509/225000 [322:30:33<90:52:27,  5.69s/it]
 74%|███████▍  | 167510/225000 [322:30:40<96:25:04,  6.04s/it]
 74%|███████▍  | 167511/225000 [322:30:44<86:02:09,  5.39s/it]
 74%|███████▍  | 167512/225000 [322:30:49<85:06:59,  5.33s/it]
 74%|███████▍  | 167513/225000 [322:30:57<96:00:04,  6.01s/it]
 74%|███████▍  | 167514/225000 [322:31:01<86:31:22,  5.42s/it]
 74%|███████▍  | 167515/225000 [322:31:05<80:41:58,  5.05s/it]
 74%|███████▍  | 167516/225000 [322:31:09<74:32:10,  4.67s/it]
 74%|███████▍  | 167517/225000 [322:31:15<80:36:02,  5.05s/it]
 74%|███████▍  | 167518/225000 [322:31:24<101:19:12,  6.35s/it]
 74%|███████▍  | 167519/225000 [322:31:28<87:09:00,  5.46s/it] 
 74%|███████▍  | 167520/225000 [322:31:34<93:13:38,  5.84s/it]
 74%|███████▍  | 167521/225000 [322:31:38<85:02:17,  5.33s/it]
 74%|███████▍  | 167522/225000 [322:31:46<96:19:42,  6.03s/it]
 74%|███████▍  | 167523/225000 [322:31:50<84:42:42,  5.31s/it]
 74%|███████▍  | 167524/225000 [322:31:55<86:03:27,  5.39s/it]
 74%|███████▍  | 167525/225000 [322:32:03<97:27:25,  6.10s/it]
 74%|███████▍  | 167526/225000 [322:32:07<88:23:14,  5.54s/it]
 74%|███████▍  | 167527/225000 [322:32:12<83:03:25,  5.20s/it]
 74%|███████▍  | 167528/225000 [322:32:16<80:23:40,  5.04s/it]
 74%|███████▍  | 167529/225000 [322:32:21<77:54:50,  4.88s/it]
 74%|███████▍  | 167530/225000 [322:32:25<76:17:52,  4.78s/it]
 74%|███████▍  | 167531/225000 [322:32:36<106:19:39,  6.66s/it]
 74%|███████▍  | 167532/225000 [322:32:40<91:40:53,  5.74s/it] 
 74%|███████▍  | 167533/225000 [322:32:43<79:53:22,  5.00s/it]
 74%|███████▍  | 167534/225000 [322:32:53<104:05:55,  6.52s/it]
 74%|███████▍  | 167535/225000 [322:32:58<93:10:48,  5.84s/it] 
 74%|███████▍  | 167536/225000 [322:33:02<84:40:12,  5.30s/it]
 74%|███████▍  | 167537/225000 [322:33:08<89:03:15,  5.58s/it]
 74%|███████▍  | 167538/225000 [322:33:16<99:12:55,  6.22s/it]
 74%|███████▍  | 167539/225000 [322:33:19<87:04:19,  5.46s/it]
 74%|███████▍  | 167540/225000 [322:33:23<79:03:59,  4.95s/it]
 74%|███████▍  | 167541/225000 [322:33:28<81:03:43,  5.08s/it]
 74%|███████▍  | 167542/225000 [322:33:38<102:25:52,  6.42s/it]
 74%|███████▍  | 167543/225000 [322:33:41<87:15:41,  5.47s/it] 
 74%|███████▍  | 167544/225000 [322:33:45<79:53:55,  5.01s/it]
 74%|███████▍  | 167545/225000 [322:33:50<77:10:19,  4.84s/it]
 74%|███████▍  | 167546/225000 [322:33:54<75:17:25,  4.72s/it]
 74%|███████▍  | 167547/225000 [322:34:04<102:11:03,  6.40s/it]
 74%|███████▍  | 167548/225000 [322:34:12<105:43:40,  6.63s/it]
 74%|███████▍  | 167549/225000 [322:34:15<90:32:46,  5.67s/it] 
 74%|███████▍  | 167550/225000 [322:34:19<82:52:03,  5.19s/it]
                                                              
{'loss': '0.7772', 'grad_norm': '1.163', 'learning_rate': '8.325e-06', 'epoch': '0.7447', 'num_input_tokens_seen': 534156904, 'train_runtime': '1.161e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167550/225000 [322:34:19<82:52:03,  5.19s/it]
 74%|███████▍  | 167551/225000 [322:34:28<102:42:24,  6.44s/it]
 74%|███████▍  | 167552/225000 [322:34:32<89:35:17,  5.61s/it] 
 74%|███████▍  | 167553/225000 [322:34:36<80:39:30,  5.05s/it]
 74%|███████▍  | 167554/225000 [322:34:42<86:51:43,  5.44s/it]
 74%|███████▍  | 167555/225000 [322:34:50<97:03:14,  6.08s/it]
 74%|███████▍  | 167556/225000 [322:34:54<87:13:29,  5.47s/it]
 74%|███████▍  | 167557/225000 [322:34:59<83:33:06,  5.24s/it]
 74%|███████▍  | 167558/225000 [322:35:03<81:19:23,  5.10s/it]
 74%|███████▍  | 167559/225000 [322:35:08<80:56:21,  5.07s/it]
 74%|███████▍  | 167560/225000 [322:35:13<80:06:23,  5.02s/it]
 74%|███████▍  | 167561/225000 [322:35:18<80:09:59,  5.02s/it]
 74%|███████▍  | 167562/225000 [322:35:23<79:35:14,  4.99s/it]
 74%|███████▍  | 167563/225000 [322:35:29<81:37:41,  5.12s/it]
 74%|███████▍  | 167564/225000 [322:35:34<81:40:57,  5.12s/it]
 74%|███████▍  | 167565/225000 [322:35:39<81:43:45,  5.12s/it]
 74%|███████▍  | 167566/225000 [322:35:44<82:16:01,  5.16s/it]
 74%|███████▍  | 167567/225000 [322:35:49<81:37:34,  5.12s/it]
 74%|███████▍  | 167568/225000 [322:35:54<81:55:18,  5.14s/it]
 74%|███████▍  | 167569/225000 [322:35:59<81:41:53,  5.12s/it]
 74%|███████▍  | 167570/225000 [322:36:04<79:45:15,  5.00s/it]
 74%|███████▍  | 167571/225000 [322:36:09<81:20:37,  5.10s/it]
 74%|███████▍  | 167572/225000 [322:36:18<99:20:53,  6.23s/it]
 74%|███████▍  | 167573/225000 [322:36:28<114:07:37,  7.15s/it]
 74%|███████▍  | 167574/225000 [322:36:31<97:29:36,  6.11s/it] 
 74%|███████▍  | 167575/225000 [322:36:35<86:52:05,  5.45s/it]
 74%|███████▍  | 167576/225000 [322:36:39<79:33:28,  4.99s/it]
 74%|███████▍  | 167577/225000 [322:36:51<112:14:15,  7.04s/it]
 74%|███████▍  | 167578/225000 [322:36:54<95:44:23,  6.00s/it] 
 74%|███████▍  | 167579/225000 [322:36:58<84:32:15,  5.30s/it]
 74%|███████▍  | 167580/225000 [322:37:06<98:39:51,  6.19s/it]
 74%|███████▍  | 167581/225000 [322:37:13<102:13:03,  6.41s/it]
 74%|███████▍  | 167582/225000 [322:37:18<92:22:26,  5.79s/it] 
 74%|███████▍  | 167583/225000 [322:37:22<84:18:05,  5.29s/it]
 74%|███████▍  | 167584/225000 [322:37:26<78:55:33,  4.95s/it]
 74%|███████▍  | 167585/225000 [322:37:30<75:27:23,  4.73s/it]
 74%|███████▍  | 167586/225000 [322:37:35<76:19:54,  4.79s/it]
 74%|███████▍  | 167587/225000 [322:37:43<89:07:10,  5.59s/it]
 74%|███████▍  | 167588/225000 [322:37:52<106:06:11,  6.65s/it]
 74%|███████▍  | 167589/225000 [322:37:55<92:14:14,  5.78s/it] 
 74%|███████▍  | 167590/225000 [322:38:00<84:45:19,  5.31s/it]
 74%|███████▍  | 167591/225000 [322:38:04<79:02:45,  4.96s/it]
 74%|███████▍  | 167592/225000 [322:38:10<86:29:45,  5.42s/it]
 74%|███████▍  | 167593/225000 [322:38:18<99:40:53,  6.25s/it]
 74%|███████▍  | 167594/225000 [322:38:22<85:48:14,  5.38s/it]
 74%|███████▍  | 167595/225000 [322:38:31<105:11:46,  6.60s/it]
 74%|███████▍  | 167596/225000 [322:38:34<88:59:03,  5.58s/it] 
 74%|███████▍  | 167597/225000 [322:38:39<82:51:39,  5.20s/it]
 74%|███████▍  | 167598/225000 [322:38:43<79:47:16,  5.00s/it]
 74%|███████▍  | 167599/225000 [322:38:48<79:23:58,  4.98s/it]
 74%|███████▍  | 167600/225000 [322:38:53<79:31:07,  4.99s/it]
                                                              
{'loss': '0.7621', 'grad_norm': '1.082', 'learning_rate': '8.311e-06', 'epoch': '0.7449', 'num_input_tokens_seen': 534286528, 'train_runtime': '1.162e+06', 'train_tokens_per_second': '460'}

 74%|███████▍  | 167600/225000 [322:38:53<79:31:07,  4.99s/it]
 74%|███████▍  | 167601/225000 [322:38:58<78:55:09,  4.95s/it]
 74%|███████▍  | 167602/225000 [322:39:11<116:09:45,  7.29s/it]
 74%|███████▍  | 167603/225000 [322:39:16<103:44:42,  6.51s/it]
 74%|███████▍  | 167604/225000 [322:39:22<105:02:06,  6.59s/it]
 74%|███████▍  | 167605/225000 [322:39:26<90:49:54,  5.70s/it] 
 74%|███████▍  | 167606/225000 [322:39:32<93:51:27,  5.89s/it]
 74%|███████▍  | 167607/225000 [322:39:40<101:15:03,  6.35s/it]
 74%|███████▍  | 167608/225000 [322:39:43<87:43:44,  5.50s/it] 
 74%|███████▍  | 167609/225000 [322:39:50<93:29:47,  5.86s/it]
 74%|███████▍  | 167610/225000 [322:39:54<84:50:42,  5.32s/it]
 74%|███████▍  | 167611/225000 [322:40:01<94:59:55,  5.96s/it]
 74%|███████▍  | 167612/225000 [322:40:06<88:41:27,  5.56s/it]
 74%|███████▍  | 167613/225000 [322:40:13<93:16:36,  5.85s/it]
 74%|███████▍  | 167614/225000 [322:40:17<85:14:01,  5.35s/it]
 74%|███████▍  | 167615/225000 [322:40:21<78:43:04,  4.94s/it]
 74%|███████▍  | 167616/225000 [322:40:25<75:15:19,  4.72s/it]
 74%|███████▍  | 167617/225000 [322:40:34<96:40:59,  6.07s/it]
 74%|███████▍  | 167618/225000 [322:40:40<93:59:10,  5.90s/it]
 74%|███████▍  | 167619/225000 [322:40:43<81:53:31,  5.14s/it]
 74%|███████▍  | 167620/225000 [322:40:48<82:35:44,  5.18s/it]
 74%|███████▍  | 167621/225000 [322:40:53<82:24:51,  5.17s/it]
 74%|███████▍  | 167622/225000 [322:40:59<84:39:08,  5.31s/it]
 74%|███████▍  | 167623/225000 [322:41:04<84:07:22,  5.28s/it]
 74%|███████▍  | 167624/225000 [322:41:09<81:19:07,  5.10s/it]
 74%|███████▍  | 167625/225000 [322:41:18<101:24:05,  6.36s/it]
 75%|███████▍  | 167626/225000 [322:41:22<87:07:56,  5.47s/it] 
 75%|███████▍  | 167627/225000 [322:41:29<98:20:49,  6.17s/it]
 75%|███████▍  | 167628/225000 [322:41:34<90:12:02,  5.66s/it]
 75%|███████▍  | 167629/225000 [322:41:40<90:04:37,  5.65s/it]
 75%|███████▍  | 167630/225000 [322:41:43<79:50:48,  5.01s/it]
 75%|███████▍  | 167631/225000 [322:41:52<96:30:38,  6.06s/it]
 75%|███████▍  | 167632/225000 [322:41:55<82:47:01,  5.19s/it]
 75%|███████▍  | 167633/225000 [322:42:02<93:34:23,  5.87s/it]
 75%|███████▍  | 167634/225000 [322:42:06<83:22:17,  5.23s/it]
 75%|███████▍  | 167635/225000 [322:42:10<76:37:19,  4.81s/it]
 75%|███████▍  | 167636/225000 [322:42:16<82:54:57,  5.20s/it]
 75%|███████▍  | 167637/225000 [322:42:26<108:06:54,  6.79s/it]
 75%|███████▍  | 167638/225000 [322:42:30<91:28:35,  5.74s/it] 
 75%|███████▍  | 167639/225000 [322:42:33<80:43:29,  5.07s/it]
 75%|███████▍  | 167640/225000 [322:42:41<91:49:49,  5.76s/it]
 75%|███████▍  | 167641/225000 [322:42:46<89:02:04,  5.59s/it]
 75%|███████▍  | 167642/225000 [322:42:51<86:51:38,  5.45s/it]
 75%|███████▍  | 167643/225000 [322:42:57<88:44:16,  5.57s/it]
 75%|███████▍  | 167644/225000 [322:43:02<87:52:44,  5.52s/it]
 75%|███████▍  | 167645/225000 [322:43:08<87:26:07,  5.49s/it]
 75%|███████▍  | 167646/225000 [322:43:13<86:36:59,  5.44s/it]
 75%|███████▍  | 167647/225000 [322:43:18<87:28:41,  5.49s/it]
 75%|███████▍  | 167648/225000 [322:43:24<86:35:37,  5.44s/it]
 75%|███████▍  | 167649/225000 [322:43:29<84:28:47,  5.30s/it]
 75%|███████▍  | 167650/225000 [322:43:34<85:17:18,  5.35s/it]
                                                              
{'loss': '0.7813', 'grad_norm': '1.091', 'learning_rate': '8.297e-06', 'epoch': '0.7451', 'num_input_tokens_seen': 534416776, 'train_runtime': '1.162e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167650/225000 [322:43:34<85:17:18,  5.35s/it]
 75%|███████▍  | 167651/225000 [322:43:40<85:16:51,  5.35s/it]
 75%|███████▍  | 167652/225000 [322:43:45<86:14:57,  5.41s/it]
 75%|███████▍  | 167653/225000 [322:43:54<101:33:42,  6.38s/it]
 75%|███████▍  | 167654/225000 [322:43:58<93:12:48,  5.85s/it] 
 75%|███████▍  | 167655/225000 [322:44:03<86:44:00,  5.44s/it]
 75%|███████▍  | 167656/225000 [322:44:08<82:56:18,  5.21s/it]
 75%|███████▍  | 167657/225000 [322:44:12<78:19:47,  4.92s/it]
 75%|███████▍  | 167658/225000 [322:44:21<99:46:58,  6.26s/it]
 75%|███████▍  | 167659/225000 [322:44:27<98:40:03,  6.19s/it]
 75%|███████▍  | 167660/225000 [322:44:32<89:38:13,  5.63s/it]
 75%|███████▍  | 167661/225000 [322:44:36<82:38:24,  5.19s/it]
 75%|███████▍  | 167662/225000 [322:44:44<95:33:19,  6.00s/it]
 75%|███████▍  | 167663/225000 [322:44:51<101:44:15,  6.39s/it]
 75%|███████▍  | 167664/225000 [322:44:54<87:33:48,  5.50s/it] 
 75%|███████▍  | 167665/225000 [322:44:59<83:11:28,  5.22s/it]
 75%|███████▍  | 167666/225000 [322:45:03<80:06:34,  5.03s/it]
 75%|███████▍  | 167667/225000 [322:45:09<81:15:33,  5.10s/it]
 75%|███████▍  | 167668/225000 [322:45:14<82:25:04,  5.18s/it]
 75%|███████▍  | 167669/225000 [322:45:19<81:03:53,  5.09s/it]
 75%|███████▍  | 167670/225000 [322:45:24<81:10:13,  5.10s/it]
 75%|███████▍  | 167671/225000 [322:45:29<80:47:59,  5.07s/it]
 75%|███████▍  | 167672/225000 [322:45:34<79:41:56,  5.00s/it]
 75%|███████▍  | 167673/225000 [322:45:39<81:35:10,  5.12s/it]
 75%|███████▍  | 167674/225000 [322:45:48<96:22:09,  6.05s/it]
 75%|███████▍  | 167675/225000 [322:45:58<116:08:28,  7.29s/it]
 75%|███████▍  | 167676/225000 [322:46:01<98:42:55,  6.20s/it] 
 75%|███████▍  | 167677/225000 [322:46:05<87:42:26,  5.51s/it]
 75%|███████▍  | 167678/225000 [322:46:14<104:34:05,  6.57s/it]
 75%|███████▍  | 167679/225000 [322:46:20<98:37:07,  6.19s/it] 
 75%|███████▍  | 167680/225000 [322:46:23<87:02:14,  5.47s/it]
 75%|███████▍  | 167681/225000 [322:46:27<80:18:46,  5.04s/it]
 75%|███████▍  | 167682/225000 [322:46:37<99:31:07,  6.25s/it]
 75%|███████▍  | 167683/225000 [322:46:40<84:14:48,  5.29s/it]
 75%|███████▍  | 167684/225000 [322:46:45<83:05:02,  5.22s/it]
 75%|███████▍  | 167685/225000 [322:46:50<84:08:49,  5.29s/it]
 75%|███████▍  | 167686/225000 [322:46:55<83:48:42,  5.26s/it]
 75%|███████▍  | 167687/225000 [322:47:01<83:47:58,  5.26s/it]
 75%|███████▍  | 167688/225000 [322:47:06<83:59:15,  5.28s/it]
 75%|███████▍  | 167689/225000 [322:47:11<81:59:07,  5.15s/it]
 75%|███████▍  | 167690/225000 [322:47:16<80:46:34,  5.07s/it]
 75%|███████▍  | 167691/225000 [322:47:21<82:27:30,  5.18s/it]
 75%|███████▍  | 167692/225000 [322:47:26<80:55:15,  5.08s/it]
 75%|███████▍  | 167693/225000 [322:47:31<83:08:24,  5.22s/it]
 75%|███████▍  | 167694/225000 [322:47:37<84:48:31,  5.33s/it]
 75%|███████▍  | 167695/225000 [322:47:45<96:44:26,  6.08s/it]
 75%|███████▍  | 167696/225000 [322:47:50<92:44:37,  5.83s/it]
 75%|███████▍  | 167697/225000 [322:47:55<89:46:21,  5.64s/it]
 75%|███████▍  | 167698/225000 [322:48:00<87:18:43,  5.49s/it]
 75%|███████▍  | 167699/225000 [322:48:06<85:57:36,  5.40s/it]
 75%|███████▍  | 167700/225000 [322:48:11<84:37:20,  5.32s/it]
                                                              
{'loss': '0.7776', 'grad_norm': '1.183', 'learning_rate': '8.284e-06', 'epoch': '0.7453', 'num_input_tokens_seen': 534547312, 'train_runtime': '1.162e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167700/225000 [322:48:11<84:37:20,  5.32s/it]
 75%|███████▍  | 167701/225000 [322:48:15<81:47:28,  5.14s/it]
 75%|███████▍  | 167702/225000 [322:48:21<84:31:07,  5.31s/it]
 75%|███████▍  | 167703/225000 [322:48:27<85:59:56,  5.40s/it]
 75%|███████▍  | 167704/225000 [322:48:32<85:02:34,  5.34s/it]
 75%|███████▍  | 167705/225000 [322:48:43<110:07:26,  6.92s/it]
 75%|███████▍  | 167706/225000 [322:48:48<101:47:20,  6.40s/it]
 75%|███████▍  | 167707/225000 [322:48:53<97:27:33,  6.12s/it] 
 75%|███████▍  | 167708/225000 [322:48:58<92:32:20,  5.81s/it]
 75%|███████▍  | 167709/225000 [322:49:03<88:39:16,  5.57s/it]
 75%|███████▍  | 167710/225000 [322:49:08<85:02:37,  5.34s/it]
 75%|███████▍  | 167711/225000 [322:49:14<86:04:57,  5.41s/it]
 75%|███████▍  | 167712/225000 [322:49:19<84:29:41,  5.31s/it]
 75%|███████▍  | 167713/225000 [322:49:30<113:22:18,  7.12s/it]
 75%|███████▍  | 167714/225000 [322:49:34<95:16:31,  5.99s/it] 
 75%|███████▍  | 167715/225000 [322:49:42<106:42:46,  6.71s/it]
 75%|███████▍  | 167716/225000 [322:49:46<92:54:26,  5.84s/it] 
 75%|███████▍  | 167717/225000 [322:49:49<82:39:46,  5.20s/it]
 75%|███████▍  | 167718/225000 [322:49:58<98:27:53,  6.19s/it]
 75%|███████▍  | 167719/225000 [322:50:01<84:41:17,  5.32s/it]
 75%|███████▍  | 167720/225000 [322:50:08<90:00:38,  5.66s/it]
 75%|███████▍  | 167721/225000 [322:50:15<99:26:32,  6.25s/it]
 75%|███████▍  | 167722/225000 [322:50:19<86:30:23,  5.44s/it]
 75%|███████▍  | 167723/225000 [322:50:23<81:06:32,  5.10s/it]
 75%|███████▍  | 167724/225000 [322:50:30<87:30:08,  5.50s/it]
 75%|███████▍  | 167725/225000 [322:50:35<85:49:18,  5.39s/it]
 75%|███████▍  | 167726/225000 [322:50:40<86:49:09,  5.46s/it]
 75%|███████▍  | 167727/225000 [322:50:45<82:46:55,  5.20s/it]
 75%|███████▍  | 167728/225000 [322:50:49<79:02:01,  4.97s/it]
 75%|███████▍  | 167729/225000 [322:50:54<76:35:11,  4.81s/it]
 75%|███████▍  | 167730/225000 [322:50:59<76:43:12,  4.82s/it]
 75%|███████▍  | 167731/225000 [322:51:03<74:07:30,  4.66s/it]
 75%|███████▍  | 167732/225000 [322:51:12<96:22:23,  6.06s/it]
 75%|███████▍  | 167733/225000 [322:51:20<103:22:55,  6.50s/it]
 75%|███████▍  | 167734/225000 [322:51:24<91:56:46,  5.78s/it] 
 75%|███████▍  | 167735/225000 [322:51:28<82:54:02,  5.21s/it]
 75%|███████▍  | 167736/225000 [322:51:32<78:41:07,  4.95s/it]
 75%|███████▍  | 167737/225000 [322:51:38<84:41:15,  5.32s/it]
 75%|███████▍  | 167738/225000 [322:51:47<102:32:22,  6.45s/it]
 75%|███████▍  | 167739/225000 [322:51:52<92:39:07,  5.83s/it] 
 75%|███████▍  | 167740/225000 [322:51:59<99:05:09,  6.23s/it]
 75%|███████▍  | 167741/225000 [322:52:03<88:21:26,  5.56s/it]
 75%|███████▍  | 167742/225000 [322:52:08<85:45:00,  5.39s/it]
 75%|███████▍  | 167743/225000 [322:52:16<98:39:04,  6.20s/it]
 75%|███████▍  | 167744/225000 [322:52:20<89:55:42,  5.65s/it]
 75%|███████▍  | 167745/225000 [322:52:25<84:21:32,  5.30s/it]
 75%|███████▍  | 167746/225000 [322:52:29<80:11:59,  5.04s/it]
 75%|███████▍  | 167747/225000 [322:52:34<76:40:40,  4.82s/it]
 75%|███████▍  | 167748/225000 [322:52:44<101:27:42,  6.38s/it]
 75%|███████▍  | 167749/225000 [322:52:50<103:04:09,  6.48s/it]
 75%|███████▍  | 167750/225000 [322:52:54<90:52:56,  5.71s/it] 
                                                              
{'loss': '0.7652', 'grad_norm': '1.206', 'learning_rate': '8.27e-06', 'epoch': '0.7456', 'num_input_tokens_seen': 534677744, 'train_runtime': '1.162e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167750/225000 [322:52:54<90:52:56,  5.71s/it]
 75%|███████▍  | 167751/225000 [322:53:01<96:22:15,  6.06s/it]
 75%|███████▍  | 167752/225000 [322:53:05<87:47:17,  5.52s/it]
 75%|███████▍  | 167753/225000 [322:53:09<79:51:16,  5.02s/it]
 75%|███████▍  | 167754/225000 [322:53:14<77:20:19,  4.86s/it]
 75%|███████▍  | 167755/225000 [322:53:18<74:59:48,  4.72s/it]
 75%|███████▍  | 167756/225000 [322:53:22<72:06:35,  4.53s/it]
 75%|███████▍  | 167757/225000 [322:53:27<72:14:49,  4.54s/it]
 75%|███████▍  | 167758/225000 [322:53:38<102:56:19,  6.47s/it]
 75%|███████▍  | 167759/225000 [322:53:44<102:33:10,  6.45s/it]
 75%|███████▍  | 167760/225000 [322:53:48<90:00:17,  5.66s/it] 
 75%|███████▍  | 167761/225000 [322:53:52<81:39:33,  5.14s/it]
 75%|███████▍  | 167762/225000 [322:54:02<103:40:13,  6.52s/it]
 75%|███████▍  | 167763/225000 [322:54:05<88:49:10,  5.59s/it] 
 75%|███████▍  | 167764/225000 [322:54:09<82:02:43,  5.16s/it]
 75%|███████▍  | 167765/225000 [322:54:13<77:26:50,  4.87s/it]
 75%|███████▍  | 167766/225000 [322:54:20<87:58:06,  5.53s/it]
 75%|███████▍  | 167767/225000 [322:54:28<97:12:37,  6.11s/it]
 75%|███████▍  | 167768/225000 [322:54:32<89:24:32,  5.62s/it]
 75%|███████▍  | 167769/225000 [322:54:37<83:45:57,  5.27s/it]
 75%|███████▍  | 167770/225000 [322:54:42<81:46:54,  5.14s/it]
 75%|███████▍  | 167771/225000 [322:54:47<80:05:37,  5.04s/it]
 75%|███████▍  | 167772/225000 [322:54:51<77:28:04,  4.87s/it]
 75%|███████▍  | 167773/225000 [322:54:56<76:49:56,  4.83s/it]
 75%|███████▍  | 167774/225000 [322:55:08<111:44:23,  7.03s/it]
 75%|███████▍  | 167775/225000 [322:55:12<100:06:03,  6.30s/it]
 75%|███████▍  | 167776/225000 [322:55:16<87:34:50,  5.51s/it] 
 75%|███████▍  | 167777/225000 [322:55:22<89:11:45,  5.61s/it]
 75%|███████▍  | 167778/225000 [322:55:29<94:31:18,  5.95s/it]
 75%|███████▍  | 167779/225000 [322:55:36<98:43:43,  6.21s/it]
 75%|███████▍  | 167780/225000 [322:55:39<86:48:52,  5.46s/it]
 75%|███████▍  | 167781/225000 [322:55:48<100:52:02,  6.35s/it]
 75%|███████▍  | 167782/225000 [322:55:52<89:53:17,  5.66s/it] 
 75%|███████▍  | 167783/225000 [322:55:56<82:05:53,  5.17s/it]
 75%|███████▍  | 167784/225000 [322:56:00<76:24:11,  4.81s/it]
 75%|███████▍  | 167785/225000 [322:56:07<89:04:47,  5.60s/it]
 75%|███████▍  | 167786/225000 [322:56:14<95:40:27,  6.02s/it]
 75%|███████▍  | 167787/225000 [322:56:18<85:29:29,  5.38s/it]
 75%|███████▍  | 167788/225000 [322:56:22<78:53:09,  4.96s/it]
 75%|███████▍  | 167789/225000 [322:56:31<98:46:17,  6.22s/it]
 75%|███████▍  | 167790/225000 [322:56:36<90:36:39,  5.70s/it]
 75%|███████▍  | 167791/225000 [322:56:40<86:05:32,  5.42s/it]
 75%|███████▍  | 167792/225000 [322:56:45<80:48:10,  5.08s/it]
 75%|███████▍  | 167793/225000 [322:56:49<78:35:27,  4.95s/it]
 75%|███████▍  | 167794/225000 [322:56:54<77:10:39,  4.86s/it]
 75%|███████▍  | 167795/225000 [322:56:59<78:04:06,  4.91s/it]
 75%|███████▍  | 167796/225000 [322:57:04<76:09:32,  4.79s/it]
 75%|███████▍  | 167797/225000 [322:57:16<111:29:01,  7.02s/it]
 75%|███████▍  | 167798/225000 [322:57:20<98:28:32,  6.20s/it] 
 75%|███████▍  | 167799/225000 [322:57:25<91:10:38,  5.74s/it]
 75%|███████▍  | 167800/225000 [322:57:30<88:31:28,  5.57s/it]
                                                              
{'loss': '0.7766', 'grad_norm': '1.162', 'learning_rate': '8.257e-06', 'epoch': '0.7458', 'num_input_tokens_seen': 534807680, 'train_runtime': '1.163e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167800/225000 [322:57:30<88:31:28,  5.57s/it]
 75%|███████▍  | 167801/225000 [322:57:35<84:47:45,  5.34s/it]
 75%|███████▍  | 167802/225000 [322:57:40<83:42:22,  5.27s/it]
 75%|███████▍  | 167803/225000 [322:57:45<80:55:36,  5.09s/it]
 75%|███████▍  | 167804/225000 [322:57:49<76:58:07,  4.84s/it]
 75%|███████▍  | 167805/225000 [322:57:53<76:22:10,  4.81s/it]
 75%|███████▍  | 167806/225000 [322:57:59<77:33:20,  4.88s/it]
 75%|███████▍  | 167807/225000 [322:58:03<74:58:41,  4.72s/it]
 75%|███████▍  | 167808/225000 [322:58:08<74:42:37,  4.70s/it]
 75%|███████▍  | 167809/225000 [322:58:18<100:50:32,  6.35s/it]
 75%|███████▍  | 167810/225000 [322:58:25<106:53:19,  6.73s/it]
 75%|███████▍  | 167811/225000 [322:58:29<91:49:19,  5.78s/it] 
 75%|███████▍  | 167812/225000 [322:58:37<102:13:56,  6.44s/it]
 75%|███████▍  | 167813/225000 [322:58:43<99:11:39,  6.24s/it] 
 75%|███████▍  | 167814/225000 [322:58:48<94:36:43,  5.96s/it]
 75%|███████▍  | 167815/225000 [322:58:55<100:09:42,  6.31s/it]
 75%|███████▍  | 167816/225000 [322:58:58<86:01:00,  5.42s/it] 
 75%|███████▍  | 167817/225000 [322:59:02<78:28:55,  4.94s/it]
 75%|███████▍  | 167818/225000 [322:59:10<92:06:03,  5.80s/it]
 75%|███████▍  | 167819/225000 [322:59:17<96:30:13,  6.08s/it]
 75%|███████▍  | 167820/225000 [322:59:22<91:58:57,  5.79s/it]
 75%|███████▍  | 167821/225000 [322:59:30<100:36:25,  6.33s/it]
 75%|███████▍  | 167822/225000 [322:59:33<85:55:20,  5.41s/it] 
 75%|███████▍  | 167823/225000 [322:59:41<99:58:56,  6.30s/it]
 75%|███████▍  | 167824/225000 [322:59:45<89:09:29,  5.61s/it]
 75%|███████▍  | 167825/225000 [322:59:51<91:53:45,  5.79s/it]
 75%|███████▍  | 167826/225000 [322:59:55<82:35:37,  5.20s/it]
 75%|███████▍  | 167827/225000 [322:59:59<77:04:13,  4.85s/it]
 75%|███████▍  | 167828/225000 [323:00:03<72:00:25,  4.53s/it]
 75%|███████▍  | 167829/225000 [323:00:13<97:08:17,  6.12s/it]
 75%|███████▍  | 167830/225000 [323:00:16<84:31:05,  5.32s/it]
 75%|███████▍  | 167831/225000 [323:00:24<97:55:18,  6.17s/it]
 75%|███████▍  | 167832/225000 [323:00:28<85:58:59,  5.41s/it]
 75%|███████▍  | 167833/225000 [323:00:32<78:39:34,  4.95s/it]
 75%|███████▍  | 167834/225000 [323:00:36<72:53:06,  4.59s/it]
 75%|███████▍  | 167835/225000 [323:00:46<100:11:35,  6.31s/it]
 75%|███████▍  | 167836/225000 [323:00:53<104:35:54,  6.59s/it]
 75%|███████▍  | 167837/225000 [323:00:59<98:39:34,  6.21s/it] 
 75%|███████▍  | 167838/225000 [323:01:03<91:02:48,  5.73s/it]
 75%|███████▍  | 167839/225000 [323:01:08<88:04:12,  5.55s/it]
 75%|███████▍  | 167840/225000 [323:01:13<85:50:30,  5.41s/it]
 75%|███████▍  | 167841/225000 [323:01:19<85:05:09,  5.36s/it]
 75%|███████▍  | 167842/225000 [323:01:24<82:54:46,  5.22s/it]
 75%|███████▍  | 167843/225000 [323:01:28<81:34:27,  5.14s/it]
 75%|███████▍  | 167844/225000 [323:01:34<82:11:40,  5.18s/it]
 75%|███████▍  | 167845/225000 [323:01:39<83:10:50,  5.24s/it]
 75%|███████▍  | 167846/225000 [323:01:44<81:55:42,  5.16s/it]
 75%|███████▍  | 167847/225000 [323:01:49<80:49:28,  5.09s/it]
 75%|███████▍  | 167848/225000 [323:01:54<81:18:25,  5.12s/it]
 75%|███████▍  | 167849/225000 [323:01:59<80:18:28,  5.06s/it]
 75%|███████▍  | 167850/225000 [323:02:04<80:31:00,  5.07s/it]
                                                              
{'loss': '0.7737', 'grad_norm': '1.098', 'learning_rate': '8.243e-06', 'epoch': '0.746', 'num_input_tokens_seen': 534938128, 'train_runtime': '1.163e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167850/225000 [323:02:04<80:31:00,  5.07s/it]
 75%|███████▍  | 167851/225000 [323:02:09<80:32:20,  5.07s/it]
 75%|███████▍  | 167852/225000 [323:02:15<81:49:14,  5.15s/it]
 75%|███████▍  | 167853/225000 [323:02:20<83:43:29,  5.27s/it]
 75%|███████▍  | 167854/225000 [323:02:26<83:43:58,  5.27s/it]
 75%|███████▍  | 167855/225000 [323:02:31<85:11:37,  5.37s/it]
 75%|███████▍  | 167856/225000 [323:02:39<96:59:42,  6.11s/it]
 75%|███████▍  | 167857/225000 [323:02:50<119:20:21,  7.52s/it]
 75%|███████▍  | 167858/225000 [323:02:53<100:40:05,  6.34s/it]
 75%|███████▍  | 167859/225000 [323:03:00<103:19:25,  6.51s/it]
 75%|███████▍  | 167860/225000 [323:03:05<93:40:34,  5.90s/it] 
 75%|███████▍  | 167861/225000 [323:03:11<94:23:32,  5.95s/it]
 75%|███████▍  | 167862/225000 [323:03:17<98:01:43,  6.18s/it]
 75%|███████▍  | 167863/225000 [323:03:24<97:53:05,  6.17s/it]
 75%|███████▍  | 167864/225000 [323:03:27<85:48:24,  5.41s/it]
 75%|███████▍  | 167865/225000 [323:03:35<96:06:55,  6.06s/it]
 75%|███████▍  | 167866/225000 [323:03:40<92:20:59,  5.82s/it]
 75%|███████▍  | 167867/225000 [323:03:44<83:10:46,  5.24s/it]
 75%|███████▍  | 167868/225000 [323:03:48<79:36:49,  5.02s/it]
 75%|███████▍  | 167869/225000 [323:03:53<74:59:46,  4.73s/it]
 75%|███████▍  | 167870/225000 [323:03:57<72:51:04,  4.59s/it]
 75%|███████▍  | 167871/225000 [323:04:08<102:59:58,  6.49s/it]
 75%|███████▍  | 167872/225000 [323:04:13<97:51:29,  6.17s/it] 
 75%|███████▍  | 167873/225000 [323:04:17<86:05:34,  5.43s/it]
 75%|███████▍  | 167874/225000 [323:04:20<77:01:29,  4.85s/it]
 75%|███████▍  | 167875/225000 [323:04:29<93:03:05,  5.86s/it]
 75%|███████▍  | 167876/225000 [323:04:35<96:05:03,  6.06s/it]
 75%|███████▍  | 167877/225000 [323:04:40<88:20:47,  5.57s/it]
 75%|███████▍  | 167878/225000 [323:04:44<82:54:03,  5.22s/it]
 75%|███████▍  | 167879/225000 [323:04:49<79:56:45,  5.04s/it]
 75%|███████▍  | 167880/225000 [323:04:54<79:53:19,  5.04s/it]
 75%|███████▍  | 167881/225000 [323:05:02<97:22:39,  6.14s/it]
 75%|███████▍  | 167882/225000 [323:05:09<100:55:48,  6.36s/it]
 75%|███████▍  | 167883/225000 [323:05:14<95:26:09,  6.02s/it] 
 75%|███████▍  | 167884/225000 [323:05:20<92:27:27,  5.83s/it]
 75%|███████▍  | 167885/225000 [323:05:25<89:45:38,  5.66s/it]
 75%|███████▍  | 167886/225000 [323:05:30<88:48:18,  5.60s/it]
 75%|███████▍  | 167887/225000 [323:05:36<86:52:11,  5.48s/it]
 75%|███████▍  | 167888/225000 [323:05:41<85:47:29,  5.41s/it]
 75%|███████▍  | 167889/225000 [323:05:46<83:52:14,  5.29s/it]
 75%|███████▍  | 167890/225000 [323:05:51<83:18:25,  5.25s/it]
 75%|███████▍  | 167891/225000 [323:05:58<92:52:31,  5.85s/it]
 75%|███████▍  | 167892/225000 [323:06:04<90:36:01,  5.71s/it]
 75%|███████▍  | 167893/225000 [323:06:08<85:59:47,  5.42s/it]
 75%|███████▍  | 167894/225000 [323:06:14<85:23:04,  5.38s/it]
 75%|███████▍  | 167895/225000 [323:06:19<82:27:09,  5.20s/it]
 75%|███████▍  | 167896/225000 [323:06:24<81:46:35,  5.16s/it]
 75%|███████▍  | 167897/225000 [323:06:29<84:26:47,  5.32s/it]
 75%|███████▍  | 167898/225000 [323:06:34<82:31:51,  5.20s/it]
 75%|███████▍  | 167899/225000 [323:06:40<83:54:55,  5.29s/it]
 75%|███████▍  | 167900/225000 [323:06:45<83:27:51,  5.26s/it]
                                                              
{'loss': '0.7696', 'grad_norm': '1.161', 'learning_rate': '8.23e-06', 'epoch': '0.7462', 'num_input_tokens_seen': 535069752, 'train_runtime': '1.163e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167900/225000 [323:06:45<83:27:51,  5.26s/it]
 75%|███████▍  | 167901/225000 [323:06:50<84:08:05,  5.30s/it]
 75%|███████▍  | 167902/225000 [323:06:56<85:43:02,  5.40s/it]
 75%|███████▍  | 167903/225000 [323:07:01<85:14:16,  5.37s/it]
 75%|███████▍  | 167904/225000 [323:07:07<85:20:03,  5.38s/it]
 75%|███████▍  | 167905/225000 [323:07:12<84:59:49,  5.36s/it]
 75%|███████▍  | 167906/225000 [323:07:17<85:41:16,  5.40s/it]
 75%|███████▍  | 167907/225000 [323:07:23<86:55:51,  5.48s/it]
 75%|███████▍  | 167908/225000 [323:07:28<86:05:51,  5.43s/it]
 75%|███████▍  | 167909/225000 [323:07:34<86:09:48,  5.43s/it]
 75%|███████▍  | 167910/225000 [323:07:45<113:38:42,  7.17s/it]
 75%|███████▍  | 167911/225000 [323:07:49<100:10:57,  6.32s/it]
 75%|███████▍  | 167912/225000 [323:07:54<90:39:58,  5.72s/it] 
 75%|███████▍  | 167913/225000 [323:07:58<84:55:22,  5.36s/it]
 75%|███████▍  | 167914/225000 [323:08:03<81:55:06,  5.17s/it]
 75%|███████▍  | 167915/225000 [323:08:11<93:42:47,  5.91s/it]
 75%|███████▍  | 167916/225000 [323:08:17<97:54:21,  6.17s/it]
 75%|███████▍  | 167917/225000 [323:08:21<87:10:16,  5.50s/it]
 75%|███████▍  | 167918/225000 [323:08:28<91:19:22,  5.76s/it]
 75%|███████▍  | 167919/225000 [323:08:35<100:15:59,  6.32s/it]
 75%|███████▍  | 167920/225000 [323:08:39<89:23:14,  5.64s/it] 
 75%|███████▍  | 167921/225000 [323:08:44<83:18:42,  5.25s/it]
 75%|███████▍  | 167922/225000 [323:08:48<76:59:10,  4.86s/it]
 75%|███████▍  | 167923/225000 [323:08:52<73:52:52,  4.66s/it]
 75%|███████▍  | 167924/225000 [323:09:03<106:27:27,  6.71s/it]
 75%|███████▍  | 167925/225000 [323:09:07<90:47:11,  5.73s/it] 
 75%|███████▍  | 167926/225000 [323:09:13<92:18:00,  5.82s/it]
 75%|███████▍  | 167927/225000 [323:09:17<85:55:54,  5.42s/it]
 75%|███████▍  | 167928/225000 [323:09:24<93:47:52,  5.92s/it]
 75%|███████▍  | 167929/225000 [323:09:28<84:20:37,  5.32s/it]
 75%|███████▍  | 167930/225000 [323:09:33<79:45:40,  5.03s/it]
 75%|███████▍  | 167931/225000 [323:09:37<76:25:09,  4.82s/it]
 75%|███████▍  | 167932/225000 [323:09:44<86:34:00,  5.46s/it]
 75%|███████▍  | 167933/225000 [323:09:52<96:27:15,  6.08s/it]
 75%|███████▍  | 167934/225000 [323:09:58<96:58:46,  6.12s/it]
 75%|███████▍  | 167935/225000 [323:10:01<85:40:38,  5.41s/it]
 75%|███████▍  | 167936/225000 [323:10:06<79:39:16,  5.03s/it]
 75%|███████▍  | 167937/225000 [323:10:11<79:13:40,  5.00s/it]
 75%|███████▍  | 167938/225000 [323:10:15<77:31:45,  4.89s/it]
 75%|███████▍  | 167939/225000 [323:10:19<74:39:45,  4.71s/it]
 75%|███████▍  | 167940/225000 [323:10:29<98:44:14,  6.23s/it]
 75%|███████▍  | 167941/225000 [323:10:33<86:01:45,  5.43s/it]
 75%|███████▍  | 167942/225000 [323:10:36<77:00:20,  4.86s/it]
 75%|███████▍  | 167943/225000 [323:10:45<97:10:54,  6.13s/it]
 75%|███████▍  | 167944/225000 [323:10:50<88:35:02,  5.59s/it]
 75%|███████▍  | 167945/225000 [323:10:54<82:21:01,  5.20s/it]
 75%|███████▍  | 167946/225000 [323:10:58<78:49:58,  4.97s/it]
 75%|███████▍  | 167947/225000 [323:11:03<75:34:41,  4.77s/it]
 75%|███████▍  | 167948/225000 [323:11:07<74:34:43,  4.71s/it]
 75%|███████▍  | 167949/225000 [323:11:12<72:11:20,  4.56s/it]
 75%|███████▍  | 167950/225000 [323:11:20<90:36:16,  5.72s/it]
                                                              
{'loss': '0.7779', 'grad_norm': '1.139', 'learning_rate': '8.216e-06', 'epoch': '0.7464', 'num_input_tokens_seen': 535199824, 'train_runtime': '1.163e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 167950/225000 [323:11:20<90:36:16,  5.72s/it]
 75%|███████▍  | 167951/225000 [323:11:29<105:13:27,  6.64s/it]
 75%|███████▍  | 167952/225000 [323:11:33<94:06:20,  5.94s/it] 
 75%|███████▍  | 167953/225000 [323:11:38<88:40:20,  5.60s/it]
 75%|███████▍  | 167954/225000 [323:11:44<91:56:06,  5.80s/it]
 75%|███████▍  | 167955/225000 [323:11:48<84:37:11,  5.34s/it]
 75%|███████▍  | 167956/225000 [323:11:54<86:13:09,  5.44s/it]
 75%|███████▍  | 167957/225000 [323:12:01<92:19:52,  5.83s/it]
 75%|███████▍  | 167958/225000 [323:12:06<90:46:15,  5.73s/it]
 75%|███████▍  | 167959/225000 [323:12:11<86:52:56,  5.48s/it]
 75%|███████▍  | 167960/225000 [323:12:17<90:06:42,  5.69s/it]
 75%|███████▍  | 167961/225000 [323:12:23<89:21:05,  5.64s/it]
 75%|███████▍  | 167962/225000 [323:12:29<91:47:00,  5.79s/it]
 75%|███████▍  | 167963/225000 [323:12:35<90:19:26,  5.70s/it]
 75%|███████▍  | 167964/225000 [323:12:41<92:01:00,  5.81s/it]
 75%|███████▍  | 167965/225000 [323:12:46<88:52:06,  5.61s/it]
 75%|███████▍  | 167966/225000 [323:12:49<79:34:16,  5.02s/it]
 75%|███████▍  | 167967/225000 [323:12:58<98:16:00,  6.20s/it]
 75%|███████▍  | 167968/225000 [323:13:02<84:47:27,  5.35s/it]
 75%|███████▍  | 167969/225000 [323:13:09<92:56:34,  5.87s/it]
 75%|███████▍  | 167970/225000 [323:13:14<88:52:18,  5.61s/it]
 75%|███████▍  | 167971/225000 [323:13:19<86:53:22,  5.48s/it]
 75%|███████▍  | 167972/225000 [323:13:24<84:49:59,  5.36s/it]
 75%|███████▍  | 167973/225000 [323:13:29<83:34:54,  5.28s/it]
 75%|███████▍  | 167974/225000 [323:13:34<82:16:11,  5.19s/it]
 75%|███████▍  | 167975/225000 [323:13:39<82:27:59,  5.21s/it]
 75%|███████▍  | 167976/225000 [323:13:44<81:55:51,  5.17s/it]
 75%|███████▍  | 167977/225000 [323:13:49<80:58:32,  5.11s/it]
 75%|███████▍  | 167978/225000 [323:13:54<78:49:40,  4.98s/it]
 75%|███████▍  | 167979/225000 [323:13:59<77:32:52,  4.90s/it]
 75%|███████▍  | 167980/225000 [323:14:04<78:58:05,  4.99s/it]
 75%|███████▍  | 167981/225000 [323:14:09<80:04:29,  5.06s/it]
 75%|███████▍  | 167982/225000 [323:14:15<82:02:15,  5.18s/it]
 75%|███████▍  | 167983/225000 [323:14:20<83:15:28,  5.26s/it]
 75%|███████▍  | 167984/225000 [323:14:26<84:05:37,  5.31s/it]
 75%|███████▍  | 167985/225000 [323:14:31<83:10:26,  5.25s/it]
 75%|███████▍  | 167986/225000 [323:14:36<81:34:33,  5.15s/it]
 75%|███████▍  | 167987/225000 [323:14:43<90:46:54,  5.73s/it]
 75%|███████▍  | 167988/225000 [323:14:53<113:56:44,  7.20s/it]
 75%|███████▍  | 167989/225000 [323:14:57<98:45:10,  6.24s/it] 
 75%|███████▍  | 167990/225000 [323:15:01<87:35:27,  5.53s/it]
 75%|███████▍  | 167991/225000 [323:15:06<82:23:55,  5.20s/it]
 75%|███████▍  | 167992/225000 [323:15:16<107:42:37,  6.80s/it]
 75%|███████▍  | 167993/225000 [323:15:20<93:04:36,  5.88s/it] 
 75%|███████▍  | 167994/225000 [323:15:27<97:10:25,  6.14s/it]
 75%|███████▍  | 167995/225000 [323:15:31<89:37:29,  5.66s/it]
 75%|███████▍  | 167996/225000 [323:15:36<83:24:33,  5.27s/it]
 75%|███████▍  | 167997/225000 [323:15:40<81:42:36,  5.16s/it]
 75%|███████▍  | 167998/225000 [323:15:48<94:56:38,  6.00s/it]
 75%|███████▍  | 167999/225000 [323:15:52<83:59:31,  5.30s/it]
 75%|███████▍  | 168000/225000 [323:16:00<95:33:05,  6.03s/it]
                                                              
{'loss': '0.7687', 'grad_norm': '1.254', 'learning_rate': '8.203e-06', 'epoch': '0.7467', 'num_input_tokens_seen': 535328952, 'train_runtime': '1.164e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168000/225000 [323:16:00<95:33:05,  6.03s/it]
 75%|███████▍  | 168001/225000 [323:16:03<84:08:37,  5.31s/it]
 75%|███████▍  | 168002/225000 [323:16:11<95:08:14,  6.01s/it]
 75%|███████▍  | 168003/225000 [323:16:15<86:57:37,  5.49s/it]
 75%|███████▍  | 168004/225000 [323:16:22<91:23:09,  5.77s/it]
 75%|███████▍  | 168005/225000 [323:16:29<95:55:15,  6.06s/it]
 75%|███████▍  | 168006/225000 [323:16:32<84:04:16,  5.31s/it]
 75%|███████▍  | 168007/225000 [323:16:39<93:43:03,  5.92s/it]
 75%|███████▍  | 168008/225000 [323:16:43<81:24:44,  5.14s/it]
 75%|███████▍  | 168009/225000 [323:16:50<91:11:43,  5.76s/it]
 75%|███████▍  | 168010/225000 [323:16:53<80:25:16,  5.08s/it]
 75%|███████▍  | 168011/225000 [323:17:00<89:45:31,  5.67s/it]
 75%|███████▍  | 168012/225000 [323:17:06<87:43:03,  5.54s/it]
 75%|███████▍  | 168013/225000 [323:17:13<94:02:33,  5.94s/it]
 75%|███████▍  | 168014/225000 [323:17:17<87:44:58,  5.54s/it]
 75%|███████▍  | 168015/225000 [323:17:22<83:47:12,  5.29s/it]
 75%|███████▍  | 168016/225000 [323:17:27<80:50:04,  5.11s/it]
 75%|███████▍  | 168017/225000 [323:17:31<78:17:12,  4.95s/it]
 75%|███████▍  | 168018/225000 [323:17:36<76:55:57,  4.86s/it]
 75%|███████▍  | 168019/225000 [323:17:40<74:00:37,  4.68s/it]
 75%|███████▍  | 168020/225000 [323:17:45<73:45:20,  4.66s/it]
 75%|███████▍  | 168021/225000 [323:17:50<77:27:22,  4.89s/it]
 75%|███████▍  | 168022/225000 [323:18:02<109:32:45,  6.92s/it]
 75%|███████▍  | 168023/225000 [323:18:06<94:36:24,  5.98s/it] 
 75%|███████▍  | 168024/225000 [323:18:12<95:51:00,  6.06s/it]
 75%|███████▍  | 168025/225000 [323:18:16<85:42:31,  5.42s/it]
 75%|███████▍  | 168026/225000 [323:18:20<78:21:31,  4.95s/it]
 75%|███████▍  | 168027/225000 [323:18:23<73:10:37,  4.62s/it]
 75%|███████▍  | 168028/225000 [323:18:33<94:56:08,  6.00s/it]
 75%|███████▍  | 168029/225000 [323:18:38<93:02:34,  5.88s/it]
 75%|███████▍  | 168030/225000 [323:18:43<86:23:23,  5.46s/it]
 75%|███████▍  | 168031/225000 [323:18:47<80:36:04,  5.09s/it]
 75%|███████▍  | 168032/225000 [323:18:51<77:24:59,  4.89s/it]
 75%|███████▍  | 168033/225000 [323:18:56<75:43:33,  4.79s/it]
 75%|███████▍  | 168034/225000 [323:19:01<74:57:09,  4.74s/it]
 75%|███████▍  | 168035/225000 [323:19:08<87:57:16,  5.56s/it]
 75%|███████▍  | 168036/225000 [323:19:18<108:40:44,  6.87s/it]
 75%|███████▍  | 168037/225000 [323:19:22<93:25:49,  5.90s/it] 
 75%|███████▍  | 168038/225000 [323:19:30<104:17:12,  6.59s/it]
 75%|███████▍  | 168039/225000 [323:19:33<89:13:46,  5.64s/it] 
 75%|███████▍  | 168040/225000 [323:19:38<83:05:31,  5.25s/it]
 75%|███████▍  | 168041/225000 [323:19:42<79:52:28,  5.05s/it]
 75%|███████▍  | 168042/225000 [323:19:47<78:24:17,  4.96s/it]
 75%|███████▍  | 168043/225000 [323:19:51<76:33:20,  4.84s/it]
 75%|███████▍  | 168044/225000 [323:19:57<81:19:11,  5.14s/it]
 75%|███████▍  | 168045/225000 [323:20:08<106:16:18,  6.72s/it]
 75%|███████▍  | 168046/225000 [323:20:11<91:01:20,  5.75s/it] 
 75%|███████▍  | 168047/225000 [323:20:19<100:24:17,  6.35s/it]
 75%|███████▍  | 168048/225000 [323:20:24<95:02:33,  6.01s/it] 
 75%|███████▍  | 168049/225000 [323:20:29<90:55:55,  5.75s/it]
 75%|███████▍  | 168050/225000 [323:20:35<89:01:20,  5.63s/it]
                                                              
{'loss': '0.7558', 'grad_norm': '1.164', 'learning_rate': '8.189e-06', 'epoch': '0.7469', 'num_input_tokens_seen': 535459328, 'train_runtime': '1.164e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168050/225000 [323:20:35<89:01:20,  5.63s/it]
 75%|███████▍  | 168051/225000 [323:20:39<85:10:50,  5.38s/it]
 75%|███████▍  | 168052/225000 [323:20:45<83:48:21,  5.30s/it]
 75%|███████▍  | 168053/225000 [323:20:50<82:13:28,  5.20s/it]
 75%|███████▍  | 168054/225000 [323:20:55<81:51:19,  5.17s/it]
 75%|███████▍  | 168055/225000 [323:21:00<81:06:02,  5.13s/it]
 75%|███████▍  | 168056/225000 [323:21:05<81:14:09,  5.14s/it]
 75%|███████▍  | 168057/225000 [323:21:10<81:40:21,  5.16s/it]
 75%|███████▍  | 168058/225000 [323:21:15<82:46:10,  5.23s/it]
 75%|███████▍  | 168059/225000 [323:21:26<107:19:20,  6.79s/it]
 75%|███████▍  | 168060/225000 [323:21:31<99:54:45,  6.32s/it] 
 75%|███████▍  | 168061/225000 [323:21:36<95:07:02,  6.01s/it]
 75%|███████▍  | 168062/225000 [323:21:42<91:24:08,  5.78s/it]
 75%|███████▍  | 168063/225000 [323:21:47<88:24:58,  5.59s/it]
 75%|███████▍  | 168064/225000 [323:21:52<87:25:46,  5.53s/it]
 75%|███████▍  | 168065/225000 [323:21:57<86:25:48,  5.46s/it]
 75%|███████▍  | 168066/225000 [323:22:03<85:41:44,  5.42s/it]
 75%|███████▍  | 168067/225000 [323:22:09<87:38:02,  5.54s/it]
 75%|███████▍  | 168068/225000 [323:22:14<87:15:15,  5.52s/it]
 75%|███████▍  | 168069/225000 [323:22:19<84:50:57,  5.37s/it]
 75%|███████▍  | 168070/225000 [323:22:26<93:17:12,  5.90s/it]
 75%|███████▍  | 168071/225000 [323:22:32<94:17:05,  5.96s/it]
 75%|███████▍  | 168072/225000 [323:22:37<86:48:19,  5.49s/it]
 75%|███████▍  | 168073/225000 [323:22:45<98:12:01,  6.21s/it]
 75%|███████▍  | 168074/225000 [323:22:48<84:56:22,  5.37s/it]
 75%|███████▍  | 168075/225000 [323:22:53<81:47:25,  5.17s/it]
 75%|███████▍  | 168076/225000 [323:22:57<79:46:09,  5.04s/it]
 75%|███████▍  | 168077/225000 [323:23:02<77:43:44,  4.92s/it]
 75%|███████▍  | 168078/225000 [323:23:07<78:04:51,  4.94s/it]
 75%|███████▍  | 168079/225000 [323:23:12<76:37:54,  4.85s/it]
 75%|███████▍  | 168080/225000 [323:23:17<78:05:35,  4.94s/it]
 75%|███████▍  | 168081/225000 [323:23:22<77:58:22,  4.93s/it]
 75%|███████▍  | 168082/225000 [323:23:27<78:18:48,  4.95s/it]
 75%|███████▍  | 168083/225000 [323:23:32<78:46:05,  4.98s/it]
 75%|███████▍  | 168084/225000 [323:23:43<109:20:49,  6.92s/it]
 75%|███████▍  | 168085/225000 [323:23:47<94:39:59,  5.99s/it] 
 75%|███████▍  | 168086/225000 [323:23:54<101:23:43,  6.41s/it]
 75%|███████▍  | 168087/225000 [323:23:59<90:41:19,  5.74s/it] 
 75%|███████▍  | 168088/225000 [323:24:03<83:43:55,  5.30s/it]
 75%|███████▍  | 168089/225000 [323:24:07<79:04:44,  5.00s/it]
 75%|███████▍  | 168090/225000 [323:24:12<77:27:03,  4.90s/it]
 75%|███████▍  | 168091/225000 [323:24:17<77:24:36,  4.90s/it]
 75%|███████▍  | 168092/225000 [323:24:27<101:05:08,  6.39s/it]
 75%|███████▍  | 168093/225000 [323:24:34<106:40:13,  6.75s/it]
 75%|███████▍  | 168094/225000 [323:24:37<89:58:57,  5.69s/it] 
 75%|███████▍  | 168095/225000 [323:24:45<98:37:59,  6.24s/it]
 75%|███████▍  | 168096/225000 [323:24:50<92:50:59,  5.87s/it]
 75%|███████▍  | 168097/225000 [323:24:54<82:29:10,  5.22s/it]
 75%|███████▍  | 168098/225000 [323:25:01<92:24:38,  5.85s/it]
 75%|███████▍  | 168099/225000 [323:25:05<84:08:03,  5.32s/it]
 75%|███████▍  | 168100/225000 [323:25:09<79:02:49,  5.00s/it]
                                                              
{'loss': '0.7661', 'grad_norm': '1.096', 'learning_rate': '8.175e-06', 'epoch': '0.7471', 'num_input_tokens_seen': 535590504, 'train_runtime': '1.164e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168100/225000 [323:25:09<79:02:49,  5.00s/it]
 75%|███████▍  | 168101/225000 [323:25:18<98:36:45,  6.24s/it]
 75%|███████▍  | 168102/225000 [323:25:25<101:19:27,  6.41s/it]
 75%|███████▍  | 168103/225000 [323:25:30<91:04:32,  5.76s/it] 
 75%|███████▍  | 168104/225000 [323:25:34<86:43:16,  5.49s/it]
 75%|███████▍  | 168105/225000 [323:25:39<83:48:30,  5.30s/it]
 75%|███████▍  | 168106/225000 [323:25:45<83:53:05,  5.31s/it]
 75%|███████▍  | 168107/225000 [323:25:50<82:56:50,  5.25s/it]
 75%|███████▍  | 168108/225000 [323:25:55<83:04:59,  5.26s/it]
 75%|███████▍  | 168109/225000 [323:26:00<81:15:28,  5.14s/it]
 75%|███████▍  | 168110/225000 [323:26:05<81:31:55,  5.16s/it]
 75%|███████▍  | 168111/225000 [323:26:09<76:43:43,  4.86s/it]
 75%|███████▍  | 168112/225000 [323:26:14<74:30:30,  4.72s/it]
 75%|███████▍  | 168113/225000 [323:26:19<77:51:55,  4.93s/it]
 75%|███████▍  | 168114/225000 [323:26:24<76:38:58,  4.85s/it]
 75%|███████▍  | 168115/225000 [323:26:29<76:51:58,  4.86s/it]
 75%|███████▍  | 168116/225000 [323:26:34<78:13:28,  4.95s/it]
 75%|███████▍  | 168117/225000 [323:26:44<101:32:12,  6.43s/it]
 75%|███████▍  | 168118/225000 [323:26:52<112:36:37,  7.13s/it]
 75%|███████▍  | 168119/225000 [323:26:57<98:55:20,  6.26s/it] 
 75%|███████▍  | 168120/225000 [323:27:02<95:57:51,  6.07s/it]
 75%|███████▍  | 168121/225000 [323:27:08<93:17:02,  5.90s/it]
 75%|███████▍  | 168122/225000 [323:27:16<106:13:27,  6.72s/it]
 75%|███████▍  | 168123/225000 [323:27:20<89:56:55,  5.69s/it] 
 75%|███████▍  | 168124/225000 [323:27:24<85:27:12,  5.41s/it]
 75%|███████▍  | 168125/225000 [323:27:30<84:27:05,  5.35s/it]
 75%|███████▍  | 168126/225000 [323:27:34<80:11:20,  5.08s/it]
 75%|███████▍  | 168127/225000 [323:27:39<79:20:50,  5.02s/it]
 75%|███████▍  | 168128/225000 [323:27:44<79:21:59,  5.02s/it]
 75%|███████▍  | 168129/225000 [323:27:49<78:33:55,  4.97s/it]
 75%|███████▍  | 168130/225000 [323:27:54<78:00:58,  4.94s/it]
 75%|███████▍  | 168131/225000 [323:27:59<78:47:10,  4.99s/it]
 75%|███████▍  | 168132/225000 [323:28:04<77:33:27,  4.91s/it]
 75%|███████▍  | 168133/225000 [323:28:09<78:50:02,  4.99s/it]
 75%|███████▍  | 168134/225000 [323:28:21<112:43:58,  7.14s/it]
 75%|███████▍  | 168135/225000 [323:28:25<100:07:33,  6.34s/it]
 75%|███████▍  | 168136/225000 [323:28:30<90:55:00,  5.76s/it] 
 75%|███████▍  | 168137/225000 [323:28:35<87:25:49,  5.54s/it]
 75%|███████▍  | 168138/225000 [323:28:39<83:11:45,  5.27s/it]
 75%|███████▍  | 168139/225000 [323:28:44<79:48:01,  5.05s/it]
 75%|███████▍  | 168140/225000 [323:28:55<109:52:08,  6.96s/it]
 75%|███████▍  | 168141/225000 [323:28:59<96:18:52,  6.10s/it] 
 75%|███████▍  | 168142/225000 [323:29:05<95:13:45,  6.03s/it]
 75%|███████▍  | 168143/225000 [323:29:09<83:15:57,  5.27s/it]
 75%|███████▍  | 168144/225000 [323:29:13<78:11:03,  4.95s/it]
 75%|███████▍  | 168145/225000 [323:29:22<95:44:04,  6.06s/it]
 75%|███████▍  | 168146/225000 [323:29:25<84:38:58,  5.36s/it]
 75%|███████▍  | 168147/225000 [323:29:30<78:53:59,  5.00s/it]
 75%|███████▍  | 168148/225000 [323:29:34<74:54:34,  4.74s/it]
 75%|███████▍  | 168149/225000 [323:29:43<97:02:40,  6.15s/it]
 75%|███████▍  | 168150/225000 [323:29:48<90:38:50,  5.74s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '1.21', 'learning_rate': '8.162e-06', 'epoch': '0.7473', 'num_input_tokens_seen': 535720880, 'train_runtime': '1.165e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168150/225000 [323:29:48<90:38:50,  5.74s/it]
 75%|███████▍  | 168151/225000 [323:29:52<81:41:57,  5.17s/it]
 75%|███████▍  | 168152/225000 [323:29:56<75:25:18,  4.78s/it]
 75%|███████▍  | 168153/225000 [323:30:06<101:15:09,  6.41s/it]
 75%|███████▍  | 168154/225000 [323:30:09<87:50:31,  5.56s/it] 
 75%|███████▍  | 168155/225000 [323:30:17<98:23:45,  6.23s/it]
 75%|███████▍  | 168156/225000 [323:30:20<83:49:07,  5.31s/it]
 75%|███████▍  | 168157/225000 [323:30:29<101:22:12,  6.42s/it]
 75%|███████▍  | 168158/225000 [323:30:33<86:33:55,  5.48s/it] 
 75%|███████▍  | 168159/225000 [323:30:36<75:11:32,  4.76s/it]
 75%|███████▍  | 168160/225000 [323:30:45<95:41:18,  6.06s/it]
 75%|███████▍  | 168161/225000 [323:30:48<84:20:47,  5.34s/it]
 75%|███████▍  | 168162/225000 [323:30:52<76:20:20,  4.84s/it]
 75%|███████▍  | 168163/225000 [323:31:01<94:09:01,  5.96s/it]
 75%|███████▍  | 168164/225000 [323:31:03<78:59:57,  5.00s/it]
 75%|███████▍  | 168165/225000 [323:31:12<95:27:35,  6.05s/it]
 75%|███████▍  | 168166/225000 [323:31:16<85:41:27,  5.43s/it]
 75%|███████▍  | 168167/225000 [323:31:21<83:16:45,  5.28s/it]
 75%|███████▍  | 168168/225000 [323:31:27<85:28:38,  5.41s/it]
 75%|███████▍  | 168169/225000 [323:31:33<91:57:05,  5.82s/it]
 75%|███████▍  | 168170/225000 [323:31:37<81:13:56,  5.15s/it]
 75%|███████▍  | 168171/225000 [323:31:42<81:10:59,  5.14s/it]
 75%|███████▍  | 168172/225000 [323:31:47<80:46:44,  5.12s/it]
 75%|███████▍  | 168173/225000 [323:31:52<80:18:12,  5.09s/it]
 75%|███████▍  | 168174/225000 [323:31:57<80:45:56,  5.12s/it]
 75%|███████▍  | 168175/225000 [323:32:02<79:13:37,  5.02s/it]
 75%|███████▍  | 168176/225000 [323:32:07<76:23:06,  4.84s/it]
 75%|███████▍  | 168177/225000 [323:32:12<76:49:53,  4.87s/it]
 75%|███████▍  | 168178/225000 [323:32:16<76:12:48,  4.83s/it]
 75%|███████▍  | 168179/225000 [323:32:22<78:37:13,  4.98s/it]
 75%|███████▍  | 168180/225000 [323:32:27<80:23:56,  5.09s/it]
 75%|███████▍  | 168181/225000 [323:32:32<80:19:54,  5.09s/it]
 75%|███████▍  | 168182/225000 [323:32:37<79:43:14,  5.05s/it]
 75%|███████▍  | 168183/225000 [323:32:49<111:57:34,  7.09s/it]
 75%|███████▍  | 168184/225000 [323:32:54<100:45:47,  6.38s/it]
 75%|███████▍  | 168185/225000 [323:32:58<93:16:13,  5.91s/it] 
 75%|███████▍  | 168186/225000 [323:33:04<89:58:55,  5.70s/it]
 75%|███████▍  | 168187/225000 [323:33:09<86:18:03,  5.47s/it]
 75%|███████▍  | 168188/225000 [323:33:14<85:48:28,  5.44s/it]
 75%|███████▍  | 168189/225000 [323:33:19<82:00:27,  5.20s/it]
 75%|███████▍  | 168190/225000 [323:33:23<79:47:21,  5.06s/it]
 75%|███████▍  | 168191/225000 [323:33:28<79:14:36,  5.02s/it]
 75%|███████▍  | 168192/225000 [323:33:33<78:31:20,  4.98s/it]
 75%|███████▍  | 168193/225000 [323:33:38<77:52:30,  4.94s/it]
 75%|███████▍  | 168194/225000 [323:33:43<78:51:44,  5.00s/it]
 75%|███████▍  | 168195/225000 [323:33:48<79:17:44,  5.03s/it]
 75%|███████▍  | 168196/225000 [323:33:53<79:03:05,  5.01s/it]
 75%|███████▍  | 168197/225000 [323:34:05<109:43:27,  6.95s/it]
 75%|███████▍  | 168198/225000 [323:34:11<108:46:49,  6.89s/it]
 75%|███████▍  | 168199/225000 [323:34:15<92:29:59,  5.86s/it] 
 75%|███████▍  | 168200/225000 [323:34:21<92:41:44,  5.88s/it]
                                                              
{'loss': '0.7714', 'grad_norm': '1.179', 'learning_rate': '8.148e-06', 'epoch': '0.7476', 'num_input_tokens_seen': 535851224, 'train_runtime': '1.165e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168200/225000 [323:34:21<92:41:44,  5.88s/it]
 75%|███████▍  | 168201/225000 [323:34:27<96:45:16,  6.13s/it]
 75%|███████▍  | 168202/225000 [323:34:31<83:19:39,  5.28s/it]
 75%|███████▍  | 168203/225000 [323:34:38<92:43:16,  5.88s/it]
 75%|███████▍  | 168204/225000 [323:34:44<93:32:54,  5.93s/it]
 75%|███████▍  | 168205/225000 [323:34:49<87:11:17,  5.53s/it]
 75%|███████▍  | 168206/225000 [323:34:53<82:51:39,  5.25s/it]
 75%|███████▍  | 168207/225000 [323:34:58<79:40:28,  5.05s/it]
 75%|███████▍  | 168208/225000 [323:35:02<74:57:21,  4.75s/it]
 75%|███████▍  | 168209/225000 [323:35:06<73:33:56,  4.66s/it]
 75%|███████▍  | 168210/225000 [323:35:11<74:26:30,  4.72s/it]
 75%|███████▍  | 168211/225000 [323:35:17<80:31:21,  5.10s/it]
 75%|███████▍  | 168212/225000 [323:35:28<107:30:50,  6.82s/it]
 75%|███████▍  | 168213/225000 [323:35:32<94:42:58,  6.00s/it] 
 75%|███████▍  | 168214/225000 [323:35:36<84:50:17,  5.38s/it]
 75%|███████▍  | 168215/225000 [323:35:40<76:50:14,  4.87s/it]
 75%|███████▍  | 168216/225000 [323:35:47<86:28:41,  5.48s/it]
 75%|███████▍  | 168217/225000 [323:35:55<100:49:31,  6.39s/it]
 75%|███████▍  | 168218/225000 [323:35:58<86:12:20,  5.47s/it] 
 75%|███████▍  | 168219/225000 [323:36:07<99:07:37,  6.28s/it]
 75%|███████▍  | 168220/225000 [323:36:10<86:22:45,  5.48s/it]
 75%|███████▍  | 168221/225000 [323:36:14<76:25:25,  4.85s/it]
 75%|███████▍  | 168222/225000 [323:36:23<96:43:23,  6.13s/it]
 75%|███████▍  | 168223/225000 [323:36:28<92:57:51,  5.89s/it]
 75%|███████▍  | 168224/225000 [323:36:32<84:53:40,  5.38s/it]
 75%|███████▍  | 168225/225000 [323:36:36<77:14:31,  4.90s/it]
 75%|███████▍  | 168226/225000 [323:36:45<94:12:46,  5.97s/it]
 75%|███████▍  | 168227/225000 [323:36:50<92:19:57,  5.85s/it]
 75%|███████▍  | 168228/225000 [323:36:54<85:14:38,  5.41s/it]
 75%|███████▍  | 168229/225000 [323:37:00<87:33:57,  5.55s/it]
 75%|███████▍  | 168230/225000 [323:37:05<84:21:06,  5.35s/it]
 75%|███████▍  | 168231/225000 [323:37:10<81:40:11,  5.18s/it]
 75%|███████▍  | 168232/225000 [323:37:15<79:51:38,  5.06s/it]
 75%|███████▍  | 168233/225000 [323:37:20<78:52:34,  5.00s/it]
 75%|███████▍  | 168234/225000 [323:37:24<77:15:42,  4.90s/it]
 75%|███████▍  | 168235/225000 [323:37:29<75:43:49,  4.80s/it]
 75%|███████▍  | 168236/225000 [323:37:34<76:47:33,  4.87s/it]
 75%|███████▍  | 168237/225000 [323:37:39<75:58:10,  4.82s/it]
 75%|███████▍  | 168238/225000 [323:37:44<78:23:59,  4.97s/it]
 75%|███████▍  | 168239/225000 [323:37:55<107:27:53,  6.82s/it]
 75%|███████▍  | 168240/225000 [323:38:00<100:34:43,  6.38s/it]
 75%|███████▍  | 168241/225000 [323:38:04<89:27:08,  5.67s/it] 
 75%|███████▍  | 168242/225000 [323:38:08<80:16:00,  5.09s/it]
 75%|███████▍  | 168243/225000 [323:38:17<99:30:27,  6.31s/it]
 75%|███████▍  | 168244/225000 [323:38:22<92:16:35,  5.85s/it]
 75%|███████▍  | 168245/225000 [323:38:30<101:13:08,  6.42s/it]
 75%|███████▍  | 168246/225000 [323:38:33<86:04:21,  5.46s/it] 
 75%|███████▍  | 168247/225000 [323:38:39<85:56:28,  5.45s/it]
 75%|███████▍  | 168248/225000 [323:38:44<85:38:24,  5.43s/it]
 75%|███████▍  | 168249/225000 [323:38:52<99:48:44,  6.33s/it]
 75%|███████▍  | 168250/225000 [323:38:56<88:07:13,  5.59s/it]
                                                              
{'loss': '0.7607', 'grad_norm': '1.214', 'learning_rate': '8.135e-06', 'epoch': '0.7478', 'num_input_tokens_seen': 535981824, 'train_runtime': '1.165e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168250/225000 [323:38:56<88:07:13,  5.59s/it]
 75%|███████▍  | 168251/225000 [323:39:00<80:02:55,  5.08s/it]
 75%|███████▍  | 168252/225000 [323:39:04<72:22:44,  4.59s/it]
 75%|███████▍  | 168253/225000 [323:39:07<67:26:36,  4.28s/it]
 75%|███████▍  | 168254/225000 [323:39:18<96:50:27,  6.14s/it]
 75%|███████▍  | 168255/225000 [323:39:21<84:29:17,  5.36s/it]
 75%|███████▍  | 168256/225000 [323:39:27<84:46:42,  5.38s/it]
 75%|███████▍  | 168257/225000 [323:39:31<82:06:45,  5.21s/it]
 75%|███████▍  | 168258/225000 [323:39:36<81:00:38,  5.14s/it]
 75%|███████▍  | 168259/225000 [323:39:42<81:26:30,  5.17s/it]
 75%|███████▍  | 168260/225000 [323:39:47<81:33:14,  5.17s/it]
 75%|███████▍  | 168261/225000 [323:39:52<82:45:21,  5.25s/it]
 75%|███████▍  | 168262/225000 [323:40:01<97:51:45,  6.21s/it]
 75%|███████▍  | 168263/225000 [323:40:05<88:31:19,  5.62s/it]
 75%|███████▍  | 168264/225000 [323:40:09<81:35:24,  5.18s/it]
 75%|███████▍  | 168265/225000 [323:40:13<77:40:48,  4.93s/it]
 75%|███████▍  | 168266/225000 [323:40:20<84:36:21,  5.37s/it]
 75%|███████▍  | 168267/225000 [323:40:30<105:37:04,  6.70s/it]
 75%|███████▍  | 168268/225000 [323:40:33<89:47:54,  5.70s/it] 
 75%|███████▍  | 168269/225000 [323:40:38<88:35:38,  5.62s/it]
 75%|███████▍  | 168270/225000 [323:40:47<101:00:30,  6.41s/it]
 75%|███████▍  | 168271/225000 [323:40:51<89:54:36,  5.71s/it] 
 75%|███████▍  | 168272/225000 [323:40:55<81:28:32,  5.17s/it]
 75%|███████▍  | 168273/225000 [323:40:59<77:54:53,  4.94s/it]
 75%|███████▍  | 168274/225000 [323:41:04<76:15:30,  4.84s/it]
 75%|███████▍  | 168275/225000 [323:41:08<75:15:54,  4.78s/it]
 75%|███████▍  | 168276/225000 [323:41:19<104:55:39,  6.66s/it]
 75%|███████▍  | 168277/225000 [323:41:25<98:48:11,  6.27s/it] 
 75%|███████▍  | 168278/225000 [323:41:30<94:19:42,  5.99s/it]
 75%|███████▍  | 168279/225000 [323:41:35<87:47:14,  5.57s/it]
 75%|███████▍  | 168280/225000 [323:41:42<95:07:52,  6.04s/it]
 75%|███████▍  | 168281/225000 [323:41:45<83:42:45,  5.31s/it]
 75%|███████▍  | 168282/225000 [323:41:54<100:08:42,  6.36s/it]
 75%|███████▍  | 168283/225000 [323:41:58<86:53:36,  5.52s/it] 
 75%|███████▍  | 168284/225000 [323:42:04<88:21:48,  5.61s/it]
 75%|███████▍  | 168285/225000 [323:42:09<87:55:05,  5.58s/it]
 75%|███████▍  | 168286/225000 [323:42:17<98:44:06,  6.27s/it]
 75%|███████▍  | 168287/225000 [323:42:21<86:42:49,  5.50s/it]
 75%|███████▍  | 168288/225000 [323:42:25<81:53:53,  5.20s/it]
 75%|███████▍  | 168289/225000 [323:42:33<93:45:32,  5.95s/it]
 75%|███████▍  | 168290/225000 [323:42:37<83:19:49,  5.29s/it]
 75%|███████▍  | 168291/225000 [323:42:40<75:24:01,  4.79s/it]
 75%|███████▍  | 168292/225000 [323:42:50<100:22:19,  6.37s/it]
 75%|███████▍  | 168293/225000 [323:42:55<91:50:36,  5.83s/it] 
 75%|███████▍  | 168294/225000 [323:42:59<85:45:52,  5.44s/it]
 75%|███████▍  | 168295/225000 [323:43:04<81:06:04,  5.15s/it]
 75%|███████▍  | 168296/225000 [323:43:08<78:38:28,  4.99s/it]
 75%|███████▍  | 168297/225000 [323:43:13<75:58:04,  4.82s/it]
 75%|███████▍  | 168298/225000 [323:43:17<74:02:09,  4.70s/it]
 75%|███████▍  | 168299/225000 [323:43:22<74:57:19,  4.76s/it]
 75%|███████▍  | 168300/225000 [323:43:33<102:28:40,  6.51s/it]
                                                               
{'loss': '0.7696', 'grad_norm': '1.206', 'learning_rate': '8.122e-06', 'epoch': '0.748', 'num_input_tokens_seen': 536111992, 'train_runtime': '1.165e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168300/225000 [323:43:33<102:28:40,  6.51s/it]
 75%|███████▍  | 168301/225000 [323:43:39<102:04:49,  6.48s/it]
 75%|███████▍  | 168302/225000 [323:43:43<91:41:33,  5.82s/it] 
 75%|███████▍  | 168303/225000 [323:43:49<90:26:29,  5.74s/it]
 75%|███████▍  | 168304/225000 [323:43:57<100:18:09,  6.37s/it]
 75%|███████▍  | 168305/225000 [323:44:01<90:38:31,  5.76s/it] 
 75%|███████▍  | 168306/225000 [323:44:05<82:37:15,  5.25s/it]
 75%|███████▍  | 168307/225000 [323:44:10<79:04:07,  5.02s/it]
 75%|███████▍  | 168308/225000 [323:44:14<76:20:03,  4.85s/it]
 75%|███████▍  | 168309/225000 [323:44:18<72:29:20,  4.60s/it]
 75%|███████▍  | 168310/225000 [323:44:29<103:24:05,  6.57s/it]
 75%|███████▍  | 168311/225000 [323:44:33<91:49:35,  5.83s/it] 
 75%|███████▍  | 168312/225000 [323:44:38<84:06:11,  5.34s/it]
 75%|███████▍  | 168313/225000 [323:44:42<81:14:16,  5.16s/it]
 75%|███████▍  | 168314/225000 [323:44:47<77:35:31,  4.93s/it]
 75%|███████▍  | 168315/225000 [323:44:51<74:35:52,  4.74s/it]
 75%|███████▍  | 168316/225000 [323:45:01<98:35:13,  6.26s/it]
 75%|███████▍  | 168317/225000 [323:45:07<97:07:46,  6.17s/it]
 75%|███████▍  | 168318/225000 [323:45:11<85:54:45,  5.46s/it]
 75%|███████▍  | 168319/225000 [323:45:16<83:18:24,  5.29s/it]
 75%|███████▍  | 168320/225000 [323:45:23<92:03:25,  5.85s/it]
 75%|███████▍  | 168321/225000 [323:45:27<83:37:36,  5.31s/it]
 75%|███████▍  | 168322/225000 [323:45:31<78:13:42,  4.97s/it]
 75%|███████▍  | 168323/225000 [323:45:35<73:40:12,  4.68s/it]
 75%|███████▍  | 168324/225000 [323:45:45<98:22:32,  6.25s/it]
 75%|███████▍  | 168325/225000 [323:45:49<89:42:27,  5.70s/it]
 75%|███████▍  | 168326/225000 [323:45:55<91:31:53,  5.81s/it]
 75%|███████▍  | 168327/225000 [323:46:01<89:42:17,  5.70s/it]
 75%|███████▍  | 168328/225000 [323:46:05<80:32:53,  5.12s/it]
 75%|███████▍  | 168329/225000 [323:46:08<74:49:47,  4.75s/it]
 75%|███████▍  | 168330/225000 [323:46:18<95:20:13,  6.06s/it]
 75%|███████▍  | 168331/225000 [323:46:23<93:11:36,  5.92s/it]
 75%|███████▍  | 168332/225000 [323:46:28<86:06:10,  5.47s/it]
 75%|███████▍  | 168333/225000 [323:46:32<82:09:12,  5.22s/it]
 75%|███████▍  | 168334/225000 [323:46:37<78:45:00,  5.00s/it]
 75%|███████▍  | 168335/225000 [323:46:41<76:14:22,  4.84s/it]
 75%|███████▍  | 168336/225000 [323:46:46<75:01:14,  4.77s/it]
 75%|███████▍  | 168337/225000 [323:46:50<71:53:42,  4.57s/it]
 75%|███████▍  | 168338/225000 [323:46:57<84:09:26,  5.35s/it]
 75%|███████▍  | 168339/225000 [323:47:06<103:15:11,  6.56s/it]
 75%|███████▍  | 168340/225000 [323:47:10<89:15:03,  5.67s/it] 
 75%|███████▍  | 168341/225000 [323:47:18<99:59:09,  6.35s/it]
 75%|███████▍  | 168342/225000 [323:47:21<86:04:17,  5.47s/it]
 75%|███████▍  | 168343/225000 [323:47:29<97:05:02,  6.17s/it]
 75%|███████▍  | 168344/225000 [323:47:33<88:02:12,  5.59s/it]
 75%|███████▍  | 168345/225000 [323:47:40<94:39:26,  6.01s/it]
 75%|███████▍  | 168346/225000 [323:47:45<89:11:01,  5.67s/it]
 75%|███████▍  | 168347/225000 [323:47:50<84:08:49,  5.35s/it]
 75%|███████▍  | 168348/225000 [323:47:54<79:38:07,  5.06s/it]
 75%|███████▍  | 168349/225000 [323:47:59<76:40:42,  4.87s/it]
 75%|███████▍  | 168350/225000 [323:48:04<77:10:51,  4.90s/it]
                                                              
{'loss': '0.7701', 'grad_norm': '1.167', 'learning_rate': '8.108e-06', 'epoch': '0.7482', 'num_input_tokens_seen': 536241280, 'train_runtime': '1.166e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168350/225000 [323:48:04<77:10:51,  4.90s/it]
 75%|███████▍  | 168351/225000 [323:48:09<77:05:52,  4.90s/it]
 75%|███████▍  | 168352/225000 [323:48:14<78:30:28,  4.99s/it]
 75%|███████▍  | 168353/225000 [323:48:18<76:27:27,  4.86s/it]
 75%|███████▍  | 168354/225000 [323:48:23<76:27:52,  4.86s/it]
 75%|███████▍  | 168355/225000 [323:48:28<75:20:48,  4.79s/it]
 75%|███████▍  | 168356/225000 [323:48:37<96:04:19,  6.11s/it]
 75%|███████▍  | 168357/225000 [323:48:46<108:37:03,  6.90s/it]
 75%|███████▍  | 168358/225000 [323:48:49<91:08:57,  5.79s/it] 
 75%|███████▍  | 168359/225000 [323:48:53<84:09:18,  5.35s/it]
 75%|███████▍  | 168360/225000 [323:48:58<82:19:02,  5.23s/it]
 75%|███████▍  | 168361/225000 [323:49:03<79:07:03,  5.03s/it]
 75%|███████▍  | 168362/225000 [323:49:08<78:20:13,  4.98s/it]
 75%|███████▍  | 168363/225000 [323:49:12<75:36:09,  4.81s/it]
 75%|███████▍  | 168364/225000 [323:49:24<107:37:00,  6.84s/it]
 75%|███████▍  | 168365/225000 [323:49:27<91:21:22,  5.81s/it] 
 75%|███████▍  | 168366/225000 [323:49:31<83:49:39,  5.33s/it]
 75%|███████▍  | 168367/225000 [323:49:39<97:29:17,  6.20s/it]
 75%|███████▍  | 168368/225000 [323:49:43<86:29:51,  5.50s/it]
 75%|███████▍  | 168369/225000 [323:49:50<89:52:02,  5.71s/it]
 75%|███████▍  | 168370/225000 [323:49:54<81:47:26,  5.20s/it]
 75%|███████▍  | 168371/225000 [323:49:58<76:15:11,  4.85s/it]
 75%|███████▍  | 168372/225000 [323:50:08<104:12:03,  6.62s/it]
 75%|███████▍  | 168373/225000 [323:50:12<89:27:20,  5.69s/it] 
 75%|███████▍  | 168374/225000 [323:50:15<77:38:18,  4.94s/it]
 75%|███████▍  | 168375/225000 [323:50:24<95:43:45,  6.09s/it]
 75%|███████▍  | 168376/225000 [323:50:31<99:14:17,  6.31s/it]
 75%|███████▍  | 168377/225000 [323:50:35<89:08:41,  5.67s/it]
 75%|███████▍  | 168378/225000 [323:50:39<81:14:35,  5.17s/it]
 75%|███████▍  | 168379/225000 [323:50:43<78:10:58,  4.97s/it]
 75%|███████▍  | 168380/225000 [323:50:48<75:24:13,  4.79s/it]
 75%|███████▍  | 168381/225000 [323:50:52<73:20:20,  4.66s/it]
 75%|███████▍  | 168382/225000 [323:51:01<92:34:32,  5.89s/it]
 75%|███████▍  | 168383/225000 [323:51:07<94:57:22,  6.04s/it]
 75%|███████▍  | 168384/225000 [323:51:11<82:25:18,  5.24s/it]
 75%|███████▍  | 168385/225000 [323:51:20<103:24:50,  6.58s/it]
 75%|███████▍  | 168386/225000 [323:51:23<87:43:23,  5.58s/it] 
 75%|███████▍  | 168387/225000 [323:51:30<91:18:52,  5.81s/it]
 75%|███████▍  | 168388/225000 [323:51:34<81:23:57,  5.18s/it]
 75%|███████▍  | 168389/225000 [323:51:38<76:42:45,  4.88s/it]
 75%|███████▍  | 168390/225000 [323:51:42<73:18:04,  4.66s/it]
 75%|███████▍  | 168391/225000 [323:51:53<102:15:11,  6.50s/it]
 75%|███████▍  | 168392/225000 [323:51:56<87:07:23,  5.54s/it] 
 75%|███████▍  | 168393/225000 [323:52:04<98:00:28,  6.23s/it]
 75%|███████▍  | 168394/225000 [323:52:08<86:27:40,  5.50s/it]
 75%|███████▍  | 168395/225000 [323:52:12<79:31:28,  5.06s/it]
 75%|███████▍  | 168396/225000 [323:52:15<72:26:24,  4.61s/it]
 75%|███████▍  | 168397/225000 [323:52:24<94:11:48,  5.99s/it]
 75%|███████▍  | 168398/225000 [323:52:29<88:13:43,  5.61s/it]
 75%|███████▍  | 168399/225000 [323:52:33<80:23:15,  5.11s/it]
 75%|███████▍  | 168400/225000 [323:52:43<102:35:30,  6.53s/it]
                                                               
{'loss': '0.764', 'grad_norm': '1.044', 'learning_rate': '8.095e-06', 'epoch': '0.7484', 'num_input_tokens_seen': 536371448, 'train_runtime': '1.166e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168400/225000 [323:52:43<102:35:30,  6.53s/it]
 75%|███████▍  | 168401/225000 [323:52:46<87:07:05,  5.54s/it] 
 75%|███████▍  | 168402/225000 [323:52:52<89:30:52,  5.69s/it]
 75%|███████▍  | 168403/225000 [323:52:57<86:10:45,  5.48s/it]
 75%|███████▍  | 168404/225000 [323:53:02<82:34:30,  5.25s/it]
 75%|███████▍  | 168405/225000 [323:53:07<80:00:00,  5.09s/it]
 75%|███████▍  | 168406/225000 [323:53:12<79:17:34,  5.04s/it]
 75%|███████▍  | 168407/225000 [323:53:17<79:07:24,  5.03s/it]
 75%|███████▍  | 168408/225000 [323:53:21<77:13:58,  4.91s/it]
 75%|███████▍  | 168409/225000 [323:53:26<78:32:00,  5.00s/it]
 75%|███████▍  | 168410/225000 [323:53:31<78:30:16,  4.99s/it]
 75%|███████▍  | 168411/225000 [323:53:36<77:06:48,  4.91s/it]
 75%|███████▍  | 168412/225000 [323:53:41<75:52:17,  4.83s/it]
 75%|███████▍  | 168413/225000 [323:53:46<77:41:39,  4.94s/it]
 75%|███████▍  | 168414/225000 [323:53:51<77:17:30,  4.92s/it]
 75%|███████▍  | 168415/225000 [323:54:02<107:00:56,  6.81s/it]
 75%|███████▍  | 168416/225000 [323:54:08<101:18:00,  6.44s/it]
 75%|███████▍  | 168417/225000 [323:54:14<101:16:13,  6.44s/it]
 75%|███████▍  | 168418/225000 [323:54:19<94:39:29,  6.02s/it] 
 75%|███████▍  | 168419/225000 [323:54:24<90:03:57,  5.73s/it]
 75%|███████▍  | 168420/225000 [323:54:29<86:23:46,  5.50s/it]
 75%|███████▍  | 168421/225000 [323:54:34<83:38:18,  5.32s/it]
 75%|███████▍  | 168422/225000 [323:54:39<82:00:56,  5.22s/it]
 75%|███████▍  | 168423/225000 [323:54:44<81:33:57,  5.19s/it]
 75%|███████▍  | 168424/225000 [323:54:49<78:45:26,  5.01s/it]
 75%|███████▍  | 168425/225000 [323:54:53<77:11:56,  4.91s/it]
 75%|███████▍  | 168426/225000 [323:54:58<76:22:33,  4.86s/it]
 75%|███████▍  | 168427/225000 [323:55:03<75:52:29,  4.83s/it]
 75%|███████▍  | 168428/225000 [323:55:14<103:20:12,  6.58s/it]
 75%|███████▍  | 168429/225000 [323:55:19<97:57:23,  6.23s/it] 
 75%|███████▍  | 168430/225000 [323:55:25<95:48:37,  6.10s/it]
 75%|███████▍  | 168431/225000 [323:55:29<89:20:03,  5.69s/it]
 75%|███████▍  | 168432/225000 [323:55:34<84:25:04,  5.37s/it]
 75%|███████▍  | 168433/225000 [323:55:39<80:25:13,  5.12s/it]
 75%|███████▍  | 168434/225000 [323:55:43<77:56:39,  4.96s/it]
 75%|███████▍  | 168435/225000 [323:55:52<97:30:42,  6.21s/it]
 75%|███████▍  | 168436/225000 [323:55:58<96:12:41,  6.12s/it]
 75%|███████▍  | 168437/225000 [323:56:02<83:21:48,  5.31s/it]
 75%|███████▍  | 168438/225000 [323:56:09<91:58:52,  5.85s/it]
 75%|███████▍  | 168439/225000 [323:56:15<94:22:12,  6.01s/it]
 75%|███████▍  | 168440/225000 [323:56:19<83:32:50,  5.32s/it]
 75%|███████▍  | 168441/225000 [323:56:28<103:48:47,  6.61s/it]
 75%|███████▍  | 168442/225000 [323:56:32<91:31:53,  5.83s/it] 
 75%|███████▍  | 168443/225000 [323:56:36<82:27:41,  5.25s/it]
 75%|███████▍  | 168444/225000 [323:56:40<76:08:33,  4.85s/it]
 75%|███████▍  | 168445/225000 [323:56:48<89:47:04,  5.72s/it]
 75%|███████▍  | 168446/225000 [323:56:56<99:00:49,  6.30s/it]
 75%|███████▍  | 168447/225000 [323:57:00<88:49:29,  5.65s/it]
 75%|███████▍  | 168448/225000 [323:57:04<83:12:33,  5.30s/it]
 75%|███████▍  | 168449/225000 [323:57:10<85:28:11,  5.44s/it]
 75%|███████▍  | 168450/225000 [323:57:18<95:45:58,  6.10s/it]
                                                              
{'loss': '0.7662', 'grad_norm': '1.129', 'learning_rate': '8.081e-06', 'epoch': '0.7487', 'num_input_tokens_seen': 536501312, 'train_runtime': '1.166e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168450/225000 [323:57:18<95:45:58,  6.10s/it]
 75%|███████▍  | 168451/225000 [323:57:22<88:55:43,  5.66s/it]
 75%|███████▍  | 168452/225000 [323:57:28<88:09:16,  5.61s/it]
 75%|███████▍  | 168453/225000 [323:57:34<88:40:41,  5.65s/it]
 75%|███████▍  | 168454/225000 [323:57:39<87:44:48,  5.59s/it]
 75%|███████▍  | 168455/225000 [323:57:47<97:48:58,  6.23s/it]
 75%|███████▍  | 168456/225000 [323:57:51<86:27:48,  5.50s/it]
 75%|███████▍  | 168457/225000 [323:57:54<78:48:24,  5.02s/it]
 75%|███████▍  | 168458/225000 [323:58:04<99:44:35,  6.35s/it]
 75%|███████▍  | 168459/225000 [323:58:08<90:01:18,  5.73s/it]
 75%|███████▍  | 168460/225000 [323:58:13<84:10:13,  5.36s/it]
 75%|███████▍  | 168461/225000 [323:58:17<80:23:02,  5.12s/it]
 75%|███████▍  | 168462/225000 [323:58:22<78:23:17,  4.99s/it]
 75%|███████▍  | 168463/225000 [323:58:32<100:58:57,  6.43s/it]
 75%|███████▍  | 168464/225000 [323:58:38<98:25:36,  6.27s/it] 
 75%|███████▍  | 168465/225000 [323:58:41<86:15:51,  5.49s/it]
 75%|███████▍  | 168466/225000 [323:58:50<101:07:30,  6.44s/it]
 75%|███████▍  | 168467/225000 [323:58:54<88:39:04,  5.65s/it] 
 75%|███████▍  | 168468/225000 [323:58:58<82:26:40,  5.25s/it]
 75%|███████▍  | 168469/225000 [323:59:02<75:52:38,  4.83s/it]
 75%|███████▍  | 168470/225000 [323:59:07<77:46:14,  4.95s/it]
 75%|███████▍  | 168471/225000 [323:59:17<101:26:40,  6.46s/it]
 75%|███████▍  | 168472/225000 [323:59:20<86:24:48,  5.50s/it] 
 75%|███████▍  | 168473/225000 [323:59:27<90:09:44,  5.74s/it]
 75%|███████▍  | 168474/225000 [323:59:30<79:53:06,  5.09s/it]
 75%|███████▍  | 168475/225000 [323:59:39<97:40:21,  6.22s/it]
 75%|███████▍  | 168476/225000 [323:59:43<88:27:07,  5.63s/it]
 75%|███████▍  | 168477/225000 [323:59:50<91:00:27,  5.80s/it]
 75%|███████▍  | 168478/225000 [323:59:53<81:11:36,  5.17s/it]
 75%|███████▍  | 168479/225000 [323:59:57<73:43:05,  4.70s/it]
 75%|███████▍  | 168480/225000 [324:00:06<95:27:22,  6.08s/it]
 75%|███████▍  | 168481/225000 [324:00:09<81:41:27,  5.20s/it]
 75%|███████▍  | 168482/225000 [324:00:12<71:16:33,  4.54s/it]
 75%|███████▍  | 168483/225000 [324:00:21<91:15:15,  5.81s/it]
 75%|███████▍  | 168484/225000 [324:00:25<83:16:41,  5.30s/it]
 75%|███████▍  | 168485/225000 [324:00:29<76:15:24,  4.86s/it]
 75%|███████▍  | 168486/225000 [324:00:33<72:10:28,  4.60s/it]
 75%|███████▍  | 168487/225000 [324:00:38<71:35:55,  4.56s/it]
 75%|███████▍  | 168488/225000 [324:00:43<75:48:14,  4.83s/it]
 75%|███████▍  | 168489/225000 [324:00:52<96:28:26,  6.15s/it]
 75%|███████▍  | 168490/225000 [324:00:57<87:53:54,  5.60s/it]
 75%|███████▍  | 168491/225000 [324:01:03<92:57:36,  5.92s/it]
 75%|███████▍  | 168492/225000 [324:01:10<95:44:28,  6.10s/it]
 75%|███████▍  | 168493/225000 [324:01:13<82:56:39,  5.28s/it]
 75%|███████▍  | 168494/225000 [324:01:20<90:13:16,  5.75s/it]
 75%|███████▍  | 168495/225000 [324:01:24<83:57:38,  5.35s/it]
 75%|███████▍  | 168496/225000 [324:01:28<77:15:55,  4.92s/it]
 75%|███████▍  | 168497/225000 [324:01:33<74:20:06,  4.74s/it]
 75%|███████▍  | 168498/225000 [324:01:37<71:55:14,  4.58s/it]
 75%|███████▍  | 168499/225000 [324:01:47<100:12:23,  6.38s/it]
 75%|███████▍  | 168500/225000 [324:01:54<101:17:44,  6.45s/it]
                                                               
{'loss': '0.7654', 'grad_norm': '1.088', 'learning_rate': '8.068e-06', 'epoch': '0.7489', 'num_input_tokens_seen': 536632088, 'train_runtime': '1.167e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168500/225000 [324:01:54<101:17:44,  6.45s/it]
 75%|███████▍  | 168501/225000 [324:01:58<87:45:25,  5.59s/it] 
 75%|███████▍  | 168502/225000 [324:02:03<87:52:24,  5.60s/it]
 75%|███████▍  | 168503/225000 [324:02:09<86:49:04,  5.53s/it]
 75%|███████▍  | 168504/225000 [324:02:14<84:36:52,  5.39s/it]
 75%|███████▍  | 168505/225000 [324:02:19<82:19:53,  5.25s/it]
 75%|███████▍  | 168506/225000 [324:02:25<88:41:02,  5.65s/it]
 75%|███████▍  | 168507/225000 [324:02:30<82:46:43,  5.28s/it]
 75%|███████▍  | 168508/225000 [324:02:34<80:26:02,  5.13s/it]
 75%|███████▍  | 168509/225000 [324:02:39<78:09:41,  4.98s/it]
 75%|███████▍  | 168510/225000 [324:02:44<76:16:02,  4.86s/it]
 75%|███████▍  | 168511/225000 [324:02:48<73:38:41,  4.69s/it]
 75%|███████▍  | 168512/225000 [324:02:59<102:38:27,  6.54s/it]
 75%|███████▍  | 168513/225000 [324:03:03<90:24:46,  5.76s/it] 
 75%|███████▍  | 168514/225000 [324:03:09<93:44:30,  5.97s/it]
 75%|███████▍  | 168515/225000 [324:03:13<83:06:02,  5.30s/it]
 75%|███████▍  | 168516/225000 [324:03:21<99:01:14,  6.31s/it]
 75%|███████▍  | 168517/225000 [324:03:25<86:13:02,  5.50s/it]
 75%|███████▍  | 168518/225000 [324:03:32<92:33:00,  5.90s/it]
 75%|███████▍  | 168519/225000 [324:03:36<84:05:54,  5.36s/it]
 75%|███████▍  | 168520/225000 [324:03:40<77:26:44,  4.94s/it]
 75%|███████▍  | 168521/225000 [324:03:44<73:20:27,  4.67s/it]
 75%|███████▍  | 168522/225000 [324:03:54<97:31:51,  6.22s/it]
 75%|███████▍  | 168523/225000 [324:04:00<95:37:57,  6.10s/it]
 75%|███████▍  | 168524/225000 [324:04:07<101:23:27,  6.46s/it]
 75%|███████▍  | 168525/225000 [324:04:10<86:57:09,  5.54s/it] 
 75%|███████▍  | 168526/225000 [324:04:14<79:12:51,  5.05s/it]
 75%|███████▍  | 168527/225000 [324:04:18<74:19:39,  4.74s/it]
 75%|███████▍  | 168528/225000 [324:04:29<103:10:54,  6.58s/it]
 75%|███████▍  | 168529/225000 [324:04:33<88:32:43,  5.64s/it] 
 75%|███████▍  | 168530/225000 [324:04:36<78:01:44,  4.97s/it]
 75%|███████▍  | 168531/225000 [324:04:40<72:33:25,  4.63s/it]
 75%|███████▍  | 168532/225000 [324:04:45<73:27:10,  4.68s/it]
 75%|███████▍  | 168533/225000 [324:04:49<71:50:13,  4.58s/it]
 75%|███████▍  | 168534/225000 [324:04:54<72:56:41,  4.65s/it]
 75%|███████▍  | 168535/225000 [324:04:58<72:33:33,  4.63s/it]
 75%|███████▍  | 168536/225000 [324:05:08<95:48:36,  6.11s/it]
 75%|███████▍  | 168537/225000 [324:05:15<102:27:31,  6.53s/it]
 75%|███████▍  | 168538/225000 [324:05:19<88:37:36,  5.65s/it] 
 75%|███████▍  | 168539/225000 [324:05:23<78:41:43,  5.02s/it]
 75%|███████▍  | 168540/225000 [324:05:33<102:42:52,  6.55s/it]
 75%|███████▍  | 168541/225000 [324:05:36<87:56:40,  5.61s/it] 
 75%|███████▍  | 168542/225000 [324:05:40<80:22:02,  5.12s/it]
 75%|███████▍  | 168543/225000 [324:05:44<76:45:19,  4.89s/it]
 75%|███████▍  | 168544/225000 [324:05:49<75:50:46,  4.84s/it]
 75%|███████▍  | 168545/225000 [324:05:54<77:42:28,  4.96s/it]
 75%|███████▍  | 168546/225000 [324:06:00<78:22:44,  5.00s/it]
 75%|███████▍  | 168547/225000 [324:06:04<78:01:30,  4.98s/it]
 75%|███████▍  | 168548/225000 [324:06:09<78:12:49,  4.99s/it]
 75%|███████▍  | 168549/225000 [324:06:15<78:59:18,  5.04s/it]
 75%|███████▍  | 168550/225000 [324:06:27<114:28:20,  7.30s/it]
                                                               
{'loss': '0.7741', 'grad_norm': '1.249', 'learning_rate': '8.055e-06', 'epoch': '0.7491', 'num_input_tokens_seen': 536761280, 'train_runtime': '1.167e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168550/225000 [324:06:27<114:28:20,  7.30s/it]
 75%|███████▍  | 168551/225000 [324:06:31<95:51:35,  6.11s/it] 
 75%|███████▍  | 168552/225000 [324:06:37<97:06:21,  6.19s/it]
 75%|███████▍  | 168553/225000 [324:06:41<85:08:15,  5.43s/it]
 75%|███████▍  | 168554/225000 [324:06:48<92:41:45,  5.91s/it]
 75%|███████▍  | 168555/225000 [324:06:54<96:23:02,  6.15s/it]
 75%|███████▍  | 168556/225000 [324:07:00<95:28:23,  6.09s/it]
 75%|███████▍  | 168557/225000 [324:07:05<89:47:29,  5.73s/it]
 75%|███████▍  | 168558/225000 [324:07:10<85:09:03,  5.43s/it]
 75%|███████▍  | 168559/225000 [324:07:15<82:03:03,  5.23s/it]
 75%|███████▍  | 168560/225000 [324:07:20<80:27:48,  5.13s/it]
 75%|███████▍  | 168561/225000 [324:07:25<79:39:18,  5.08s/it]
 75%|███████▍  | 168562/225000 [324:07:29<78:09:27,  4.99s/it]
 75%|███████▍  | 168563/225000 [324:07:35<80:35:10,  5.14s/it]
 75%|███████▍  | 168564/225000 [324:07:39<78:33:51,  5.01s/it]
 75%|███████▍  | 168565/225000 [324:07:45<78:47:38,  5.03s/it]
 75%|███████▍  | 168566/225000 [324:07:49<76:39:25,  4.89s/it]
 75%|███████▍  | 168567/225000 [324:07:54<77:13:21,  4.93s/it]
 75%|███████▍  | 168568/225000 [324:07:59<76:55:27,  4.91s/it]
 75%|███████▍  | 168569/225000 [324:08:04<76:12:49,  4.86s/it]
 75%|███████▍  | 168570/225000 [324:08:09<76:24:30,  4.87s/it]
 75%|███████▍  | 168571/225000 [324:08:16<87:37:48,  5.59s/it]
 75%|███████▍  | 168572/225000 [324:08:27<115:23:19,  7.36s/it]
 75%|███████▍  | 168573/225000 [324:08:31<99:15:29,  6.33s/it] 
 75%|███████▍  | 168574/225000 [324:08:35<87:33:16,  5.59s/it]
 75%|███████▍  | 168575/225000 [324:08:39<79:23:30,  5.07s/it]
 75%|███████▍  | 168576/225000 [324:08:48<95:40:49,  6.10s/it]
 75%|███████▍  | 168577/225000 [324:08:54<95:38:30,  6.10s/it]
 75%|███████▍  | 168578/225000 [324:08:57<84:05:34,  5.37s/it]
 75%|███████▍  | 168579/225000 [324:09:01<77:39:33,  4.96s/it]
 75%|███████▍  | 168580/225000 [324:09:13<107:25:27,  6.85s/it]
 75%|███████▍  | 168581/225000 [324:09:16<92:23:02,  5.89s/it] 
 75%|███████▍  | 168582/225000 [324:09:20<80:43:33,  5.15s/it]
 75%|███████▍  | 168583/225000 [324:09:27<92:27:41,  5.90s/it]
 75%|███████▍  | 168584/225000 [324:09:32<88:17:04,  5.63s/it]
 75%|███████▍  | 168585/225000 [324:09:41<103:40:51,  6.62s/it]
 75%|███████▍  | 168586/225000 [324:09:45<88:42:22,  5.66s/it] 
 75%|███████▍  | 168587/225000 [324:09:48<78:50:40,  5.03s/it]
 75%|███████▍  | 168588/225000 [324:09:52<72:45:06,  4.64s/it]
 75%|███████▍  | 168589/225000 [324:09:57<74:51:38,  4.78s/it]
 75%|███████▍  | 168590/225000 [324:10:02<75:29:27,  4.82s/it]
 75%|███████▍  | 168591/225000 [324:10:07<77:23:57,  4.94s/it]
 75%|███████▍  | 168592/225000 [324:10:12<78:03:34,  4.98s/it]
 75%|███████▍  | 168593/225000 [324:10:18<80:08:26,  5.11s/it]
 75%|███████▍  | 168594/225000 [324:10:23<79:27:21,  5.07s/it]
 75%|███████▍  | 168595/225000 [324:10:28<81:26:27,  5.20s/it]
 75%|███████▍  | 168596/225000 [324:10:33<81:10:29,  5.18s/it]
 75%|███████▍  | 168597/225000 [324:10:39<82:03:27,  5.24s/it]
 75%|███████▍  | 168598/225000 [324:10:44<81:40:54,  5.21s/it]
 75%|███████▍  | 168599/225000 [324:10:49<79:45:34,  5.09s/it]
 75%|███████▍  | 168600/225000 [324:10:56<92:12:52,  5.89s/it]
                                                              
{'loss': '0.77', 'grad_norm': '1.13', 'learning_rate': '8.041e-06', 'epoch': '0.7493', 'num_input_tokens_seen': 536891736, 'train_runtime': '1.167e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168600/225000 [324:10:56<92:12:52,  5.89s/it]
 75%|███████▍  | 168601/225000 [324:11:05<105:48:20,  6.75s/it]
 75%|███████▍  | 168602/225000 [324:11:08<88:26:57,  5.65s/it] 
 75%|███████▍  | 168603/225000 [324:11:14<87:44:53,  5.60s/it]
 75%|███████▍  | 168604/225000 [324:11:20<92:01:00,  5.87s/it]
 75%|███████▍  | 168605/225000 [324:11:25<85:19:49,  5.45s/it]
 75%|███████▍  | 168606/225000 [324:11:29<79:45:40,  5.09s/it]
 75%|███████▍  | 168607/225000 [324:11:33<75:27:27,  4.82s/it]
 75%|███████▍  | 168608/225000 [324:11:46<111:13:06,  7.10s/it]
 75%|███████▍  | 168609/225000 [324:11:50<99:04:56,  6.33s/it] 
 75%|███████▍  | 168610/225000 [324:11:54<89:01:44,  5.68s/it]
 75%|███████▍  | 168611/225000 [324:11:59<84:21:19,  5.39s/it]
 75%|███████▍  | 168612/225000 [324:12:03<79:13:49,  5.06s/it]
 75%|███████▍  | 168613/225000 [324:12:07<75:25:21,  4.82s/it]
 75%|███████▍  | 168614/225000 [324:12:12<71:56:52,  4.59s/it]
 75%|███████▍  | 168615/225000 [324:12:23<103:15:10,  6.59s/it]
 75%|███████▍  | 168616/225000 [324:12:26<87:56:08,  5.61s/it] 
 75%|███████▍  | 168617/225000 [324:12:30<80:08:09,  5.12s/it]
 75%|███████▍  | 168618/225000 [324:12:34<76:40:41,  4.90s/it]
 75%|███████▍  | 168619/225000 [324:12:39<74:50:45,  4.78s/it]
 75%|███████▍  | 168620/225000 [324:12:44<73:47:32,  4.71s/it]
 75%|███████▍  | 168621/225000 [324:12:49<75:53:57,  4.85s/it]
 75%|███████▍  | 168622/225000 [324:13:01<111:58:08,  7.15s/it]
 75%|███████▍  | 168623/225000 [324:13:05<95:18:22,  6.09s/it] 
 75%|███████▍  | 168624/225000 [324:13:08<83:40:47,  5.34s/it]
 75%|███████▍  | 168625/225000 [324:13:14<84:45:39,  5.41s/it]
 75%|███████▍  | 168626/225000 [324:13:20<89:09:05,  5.69s/it]
 75%|███████▍  | 168627/225000 [324:13:28<96:14:24,  6.15s/it]
 75%|███████▍  | 168628/225000 [324:13:31<83:56:36,  5.36s/it]
 75%|███████▍  | 168629/225000 [324:13:39<94:32:23,  6.04s/it]
 75%|███████▍  | 168630/225000 [324:13:43<86:12:13,  5.51s/it]
 75%|███████▍  | 168631/225000 [324:13:47<79:47:07,  5.10s/it]
 75%|███████▍  | 168632/225000 [324:13:51<75:25:00,  4.82s/it]
 75%|███████▍  | 168633/225000 [324:13:55<72:32:00,  4.63s/it]
 75%|███████▍  | 168634/225000 [324:14:05<95:56:05,  6.13s/it]
 75%|███████▍  | 168635/225000 [324:14:10<91:42:44,  5.86s/it]
 75%|███████▍  | 168636/225000 [324:14:15<85:48:47,  5.48s/it]
 75%|███████▍  | 168637/225000 [324:14:19<79:41:21,  5.09s/it]
 75%|███████▍  | 168638/225000 [324:14:24<77:36:51,  4.96s/it]
 75%|███████▍  | 168639/225000 [324:14:28<74:03:56,  4.73s/it]
 75%|███████▍  | 168640/225000 [324:14:32<73:06:37,  4.67s/it]
 75%|███████▍  | 168641/225000 [324:14:40<85:27:16,  5.46s/it]
 75%|███████▍  | 168642/225000 [324:14:48<99:49:53,  6.38s/it]
 75%|███████▍  | 168643/225000 [324:14:53<90:39:31,  5.79s/it]
 75%|███████▍  | 168644/225000 [324:14:57<85:07:55,  5.44s/it]
 75%|███████▍  | 168645/225000 [324:15:02<81:30:26,  5.21s/it]
 75%|███████▍  | 168646/225000 [324:15:06<77:30:59,  4.95s/it]
 75%|███████▍  | 168647/225000 [324:15:14<90:02:58,  5.75s/it]
 75%|███████▍  | 168648/225000 [324:15:21<98:19:00,  6.28s/it]
 75%|███████▍  | 168649/225000 [324:15:26<91:19:35,  5.83s/it]
 75%|███████▍  | 168650/225000 [324:15:31<86:18:10,  5.51s/it]
                                                              
{'loss': '0.7701', 'grad_norm': '1.152', 'learning_rate': '8.028e-06', 'epoch': '0.7496', 'num_input_tokens_seen': 537021496, 'train_runtime': '1.167e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168650/225000 [324:15:31<86:18:10,  5.51s/it]
 75%|███████▍  | 168651/225000 [324:15:36<83:22:47,  5.33s/it]
 75%|███████▍  | 168652/225000 [324:15:41<81:31:06,  5.21s/it]
 75%|███████▍  | 168653/225000 [324:15:46<80:38:39,  5.15s/it]
 75%|███████▍  | 168654/225000 [324:15:51<80:06:53,  5.12s/it]
 75%|███████▍  | 168655/225000 [324:15:56<79:29:50,  5.08s/it]
 75%|███████▍  | 168656/225000 [324:16:01<77:45:34,  4.97s/it]
 75%|███████▍  | 168657/225000 [324:16:06<77:30:15,  4.95s/it]
 75%|███████▍  | 168658/225000 [324:16:10<77:24:34,  4.95s/it]
 75%|███████▍  | 168659/225000 [324:16:15<76:03:48,  4.86s/it]
 75%|███████▍  | 168660/225000 [324:16:20<78:14:15,  5.00s/it]
 75%|███████▍  | 168661/225000 [324:16:25<78:01:04,  4.99s/it]
 75%|███████▍  | 168662/225000 [324:16:31<78:40:50,  5.03s/it]
 75%|███████▍  | 168663/225000 [324:16:36<79:10:16,  5.06s/it]
 75%|███████▍  | 168664/225000 [324:16:41<79:05:33,  5.05s/it]
 75%|███████▍  | 168665/225000 [324:16:53<111:07:55,  7.10s/it]
 75%|███████▍  | 168666/225000 [324:16:58<104:03:25,  6.65s/it]
 75%|███████▍  | 168667/225000 [324:17:02<90:28:34,  5.78s/it] 
 75%|███████▍  | 168668/225000 [324:17:10<99:46:33,  6.38s/it]
 75%|███████▍  | 168669/225000 [324:17:15<92:33:40,  5.92s/it]
 75%|███████▍  | 168670/225000 [324:17:18<83:12:50,  5.32s/it]
 75%|███████▍  | 168671/225000 [324:17:22<75:11:15,  4.81s/it]
 75%|███████▍  | 168672/225000 [324:17:26<73:19:37,  4.69s/it]
 75%|███████▍  | 168673/225000 [324:17:31<70:16:30,  4.49s/it]
 75%|███████▍  | 168674/225000 [324:17:36<74:01:53,  4.73s/it]
 75%|███████▍  | 168675/225000 [324:17:47<106:01:36,  6.78s/it]
 75%|███████▍  | 168676/225000 [324:17:51<92:01:02,  5.88s/it] 
 75%|███████▍  | 168677/225000 [324:17:57<89:46:14,  5.74s/it]
 75%|███████▍  | 168678/225000 [324:18:06<105:44:15,  6.76s/it]
 75%|███████▍  | 168679/225000 [324:18:09<89:49:31,  5.74s/it] 
 75%|███████▍  | 168680/225000 [324:18:14<86:41:22,  5.54s/it]
 75%|███████▍  | 168681/225000 [324:18:21<93:47:35,  6.00s/it]
 75%|███████▍  | 168682/225000 [324:18:27<91:43:17,  5.86s/it]
 75%|███████▍  | 168683/225000 [324:18:31<84:25:52,  5.40s/it]
 75%|███████▍  | 168684/225000 [324:18:38<90:59:06,  5.82s/it]
 75%|███████▍  | 168685/225000 [324:18:42<81:17:18,  5.20s/it]
 75%|███████▍  | 168686/225000 [324:18:45<74:47:21,  4.78s/it]
 75%|███████▍  | 168687/225000 [324:18:54<94:21:08,  6.03s/it]
 75%|███████▍  | 168688/225000 [324:18:59<85:35:14,  5.47s/it]
 75%|███████▍  | 168689/225000 [324:19:03<80:24:52,  5.14s/it]
 75%|███████▍  | 168690/225000 [324:19:12<99:01:04,  6.33s/it]
 75%|███████▍  | 168691/225000 [324:19:16<86:03:25,  5.50s/it]
 75%|███████▍  | 168692/225000 [324:19:19<76:29:52,  4.89s/it]
 75%|███████▍  | 168693/225000 [324:19:29<98:59:44,  6.33s/it]
 75%|███████▍  | 168694/225000 [324:19:33<90:33:53,  5.79s/it]
 75%|███████▍  | 168695/225000 [324:19:38<84:55:50,  5.43s/it]
 75%|███████▍  | 168696/225000 [324:19:43<81:26:32,  5.21s/it]
 75%|███████▍  | 168697/225000 [324:19:47<77:12:00,  4.94s/it]
 75%|███████▍  | 168698/225000 [324:19:51<74:05:05,  4.74s/it]
 75%|███████▍  | 168699/225000 [324:19:56<74:11:29,  4.74s/it]
 75%|███████▍  | 168700/225000 [324:20:01<73:47:18,  4.72s/it]
                                                              
{'loss': '0.7594', 'grad_norm': '1.184', 'learning_rate': '8.014e-06', 'epoch': '0.7498', 'num_input_tokens_seen': 537152640, 'train_runtime': '1.168e+06', 'train_tokens_per_second': '460'}

 75%|███████▍  | 168700/225000 [324:20:01<73:47:18,  4.72s/it]
 75%|███████▍  | 168701/225000 [324:20:05<73:49:18,  4.72s/it]
 75%|███████▍  | 168702/225000 [324:20:10<74:13:44,  4.75s/it]
 75%|███████▍  | 168703/225000 [324:20:22<105:42:42,  6.76s/it]
 75%|███████▍  | 168704/225000 [324:20:27<100:12:54,  6.41s/it]
 75%|███████▍  | 168705/225000 [324:20:34<103:42:38,  6.63s/it]
 75%|███████▍  | 168706/225000 [324:20:38<88:59:13,  5.69s/it] 
 75%|███████▍  | 168707/225000 [324:20:42<83:30:06,  5.34s/it]
 75%|███████▍  | 168708/225000 [324:20:47<82:11:15,  5.26s/it]
 75%|███████▍  | 168709/225000 [324:20:52<78:58:50,  5.05s/it]
 75%|███████▍  | 168710/225000 [324:20:57<78:52:49,  5.04s/it]
 75%|███████▍  | 168711/225000 [324:21:02<77:52:36,  4.98s/it]
 75%|███████▍  | 168712/225000 [324:21:07<77:36:50,  4.96s/it]
 75%|███████▍  | 168713/225000 [324:21:12<78:18:13,  5.01s/it]
 75%|███████▍  | 168714/225000 [324:21:20<91:39:20,  5.86s/it]
 75%|███████▍  | 168715/225000 [324:21:29<109:57:22,  7.03s/it]
 75%|███████▍  | 168716/225000 [324:21:33<93:09:16,  5.96s/it] 
 75%|███████▍  | 168717/225000 [324:21:41<102:00:28,  6.52s/it]
 75%|███████▍  | 168718/225000 [324:21:45<90:24:49,  5.78s/it] 
 75%|███████▍  | 168719/225000 [324:21:49<80:52:19,  5.17s/it]
 75%|███████▍  | 168720/225000 [324:21:58<99:46:05,  6.38s/it]
 75%|███████▍  | 168721/225000 [324:22:01<85:20:12,  5.46s/it]
 75%|███████▍  | 168722/225000 [324:22:09<97:35:14,  6.24s/it]
 75%|███████▍  | 168723/225000 [324:22:12<83:52:28,  5.37s/it]
 75%|███████▍  | 168724/225000 [324:22:21<100:15:46,  6.41s/it]
 75%|███████▍  | 168725/225000 [324:22:25<87:20:24,  5.59s/it] 
 75%|███████▍  | 168726/225000 [324:22:29<79:08:43,  5.06s/it]
 75%|███████▍  | 168727/225000 [324:22:34<81:21:04,  5.20s/it]
 75%|███████▍  | 168728/225000 [324:22:43<97:21:59,  6.23s/it]
 75%|███████▍  | 168729/225000 [324:22:46<83:45:13,  5.36s/it]
 75%|███████▍  | 168730/225000 [324:22:52<84:37:06,  5.41s/it]
 75%|███████▍  | 168731/225000 [324:22:57<82:35:56,  5.28s/it]
 75%|███████▍  | 168732/225000 [324:23:02<80:01:33,  5.12s/it]
 75%|███████▍  | 168733/225000 [324:23:07<79:57:59,  5.12s/it]
 75%|███████▍  | 168734/225000 [324:23:15<96:55:03,  6.20s/it]
 75%|███████▍  | 168735/225000 [324:23:19<86:08:42,  5.51s/it]
 75%|███████▍  | 168736/225000 [324:23:27<94:19:57,  6.04s/it]
 75%|███████▍  | 168737/225000 [324:23:30<82:31:17,  5.28s/it]
 75%|███████▍  | 168738/225000 [324:23:34<77:05:05,  4.93s/it]
 75%|███████▍  | 168739/225000 [324:23:42<88:35:52,  5.67s/it]
 75%|███████▍  | 168740/225000 [324:23:45<79:38:28,  5.10s/it]
 75%|███████▍  | 168741/225000 [324:23:52<86:58:43,  5.57s/it]
 75%|███████▍  | 168742/225000 [324:23:59<92:11:25,  5.90s/it]
 75%|███████▍  | 168743/225000 [324:24:03<86:15:14,  5.52s/it]
 75%|███████▍  | 168744/225000 [324:24:08<82:45:04,  5.30s/it]
 75%|███████▍  | 168745/225000 [324:24:13<82:58:04,  5.31s/it]
 75%|███████▍  | 168746/225000 [324:24:18<81:13:29,  5.20s/it]
 75%|███████▍  | 168747/225000 [324:24:24<82:57:36,  5.31s/it]
 75%|███████▍  | 168748/225000 [324:24:29<81:18:19,  5.20s/it]
 75%|███████▍  | 168749/225000 [324:24:34<80:40:20,  5.16s/it]
 75%|███████▌  | 168750/225000 [324:24:39<81:59:05,  5.25s/it]
                                                              
{'loss': '0.7562', 'grad_norm': '1.244', 'learning_rate': '8.001e-06', 'epoch': '0.75', 'num_input_tokens_seen': 537282512, 'train_runtime': '1.168e+06', 'train_tokens_per_second': '460'}

 75%|███████▌  | 168750/225000 [324:24:39<81:59:05,  5.25s/it]
 75%|███████▌  | 168751/225000 [324:24:44<80:00:22,  5.12s/it]
 75%|███████▌  | 168752/225000 [324:24:49<80:38:37,  5.16s/it]
 75%|███████▌  | 168753/225000 [324:24:54<79:27:40,  5.09s/it]
 75%|███████▌  | 168754/225000 [324:24:59<79:08:10,  5.07s/it]
 75%|███████▌  | 168755/225000 [324:25:04<77:27:16,  4.96s/it]
 75%|███████▌  | 168756/225000 [324:25:09<78:09:48,  5.00s/it]
 75%|███████▌  | 168757/225000 [324:25:14<78:55:01,  5.05s/it]
 75%|███████▌  | 168758/225000 [324:25:20<80:41:16,  5.16s/it]
 75%|███████▌  | 168759/225000 [324:25:25<79:19:43,  5.08s/it]
 75%|███████▌  | 168760/225000 [324:25:30<79:43:33,  5.10s/it]
 75%|███████▌  | 168761/225000 [324:25:34<77:34:29,  4.97s/it]
 75%|███████▌  | 168762/225000 [324:25:40<78:50:16,  5.05s/it]
 75%|███████▌  | 168763/225000 [324:25:45<78:26:57,  5.02s/it]
 75%|███████▌  | 168764/225000 [324:25:57<111:49:54,  7.16s/it]
 75%|███████▌  | 168765/225000 [324:26:01<97:45:57,  6.26s/it] 
 75%|███████▌  | 168766/225000 [324:26:05<89:13:04,  5.71s/it]
 75%|███████▌  | 168767/225000 [324:26:10<81:37:43,  5.23s/it]
 75%|███████▌  | 168768/225000 [324:26:14<79:42:41,  5.10s/it]
 75%|███████▌  | 168769/225000 [324:26:23<96:23:46,  6.17s/it]
 75%|███████▌  | 168770/225000 [324:26:30<98:21:03,  6.30s/it]
 75%|███████▌  | 168771/225000 [324:26:33<83:44:03,  5.36s/it]
 75%|███████▌  | 168772/225000 [324:26:41<96:10:51,  6.16s/it]
 75%|███████▌  | 168773/225000 [324:26:45<87:16:59,  5.59s/it]
 75%|███████▌  | 168774/225000 [324:26:49<81:02:16,  5.19s/it]
 75%|███████▌  | 168775/225000 [324:26:54<76:59:42,  4.93s/it]
 75%|███████▌  | 168776/225000 [324:27:04<101:48:51,  6.52s/it]
 75%|███████▌  | 168777/225000 [324:27:07<87:47:09,  5.62s/it] 
 75%|███████▌  | 168778/225000 [324:27:11<77:48:20,  4.98s/it]
 75%|███████▌  | 168779/225000 [324:27:21<102:30:13,  6.56s/it]
 75%|███████▌  | 168780/225000 [324:27:25<88:48:54,  5.69s/it] 
 75%|███████▌  | 168781/225000 [324:27:29<82:26:48,  5.28s/it]
 75%|███████▌  | 168782/225000 [324:27:37<92:30:31,  5.92s/it]
 75%|███████▌  | 168783/225000 [324:27:42<90:55:46,  5.82s/it]
 75%|███████▌  | 168784/225000 [324:27:47<86:44:35,  5.55s/it]
 75%|███████▌  | 168785/225000 [324:27:52<83:28:03,  5.35s/it]
 75%|███████▌  | 168786/225000 [324:27:59<89:32:36,  5.73s/it]
 75%|███████▌  | 168787/225000 [324:28:03<82:05:50,  5.26s/it]
 75%|███████▌  | 168788/225000 [324:28:07<77:56:15,  4.99s/it]
 75%|███████▌  | 168789/225000 [324:28:12<75:48:33,  4.86s/it]
 75%|███████▌  | 168790/225000 [324:28:16<75:10:09,  4.81s/it]
 75%|███████▌  | 168791/225000 [324:28:20<72:13:33,  4.63s/it]
 75%|███████▌  | 168792/225000 [324:28:25<70:00:42,  4.48s/it]
 75%|███████▌  | 168793/225000 [324:28:32<81:37:23,  5.23s/it]
 75%|███████▌  | 168794/225000 [324:28:41<99:22:06,  6.36s/it]
 75%|███████▌  | 168795/225000 [324:28:44<84:40:24,  5.42s/it]
 75%|███████▌  | 168796/225000 [324:28:48<79:37:43,  5.10s/it]
 75%|███████▌  | 168797/225000 [324:28:52<75:25:07,  4.83s/it]
 75%|███████▌  | 168798/225000 [324:29:03<103:09:37,  6.61s/it]
 75%|███████▌  | 168799/225000 [324:29:08<92:58:09,  5.96s/it] 
 75%|███████▌  | 168800/225000 [324:29:13<90:43:42,  5.81s/it]
                                                              
{'loss': '0.7643', 'grad_norm': '1.16', 'learning_rate': '7.988e-06', 'epoch': '0.7502', 'num_input_tokens_seen': 537413176, 'train_runtime': '1.168e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 168800/225000 [324:29:13<90:43:42,  5.81s/it]
 75%|███████▌  | 168801/225000 [324:29:18<88:08:09,  5.65s/it]
 75%|███████▌  | 168802/225000 [324:29:24<88:32:37,  5.67s/it]
 75%|███████▌  | 168803/225000 [324:29:32<98:41:46,  6.32s/it]
 75%|███████▌  | 168804/225000 [324:29:37<90:38:18,  5.81s/it]
 75%|███████▌  | 168805/225000 [324:29:41<86:26:37,  5.54s/it]
 75%|███████▌  | 168806/225000 [324:29:46<82:53:14,  5.31s/it]
 75%|███████▌  | 168807/225000 [324:29:51<81:13:22,  5.20s/it]
 75%|███████▌  | 168808/225000 [324:29:56<80:32:01,  5.16s/it]
 75%|███████▌  | 168809/225000 [324:30:01<79:45:30,  5.11s/it]
 75%|███████▌  | 168810/225000 [324:30:06<79:52:24,  5.12s/it]
 75%|███████▌  | 168811/225000 [324:30:12<80:54:35,  5.18s/it]
 75%|███████▌  | 168812/225000 [324:30:17<80:38:01,  5.17s/it]
 75%|███████▌  | 168813/225000 [324:30:22<81:23:32,  5.21s/it]
 75%|███████▌  | 168814/225000 [324:30:27<81:22:16,  5.21s/it]
 75%|███████▌  | 168815/225000 [324:30:38<106:06:01,  6.80s/it]
 75%|███████▌  | 168816/225000 [324:30:43<98:07:39,  6.29s/it] 
 75%|███████▌  | 168817/225000 [324:30:50<101:57:31,  6.53s/it]
 75%|███████▌  | 168818/225000 [324:30:56<98:29:41,  6.31s/it] 
 75%|███████▌  | 168819/225000 [324:31:00<90:08:49,  5.78s/it]
 75%|███████▌  | 168820/225000 [324:31:05<84:02:19,  5.39s/it]
 75%|███████▌  | 168821/225000 [324:31:09<79:10:30,  5.07s/it]
 75%|███████▌  | 168822/225000 [324:31:13<73:18:59,  4.70s/it]
 75%|███████▌  | 168823/225000 [324:31:17<72:10:01,  4.62s/it]
 75%|███████▌  | 168824/225000 [324:31:29<105:40:10,  6.77s/it]
 75%|███████▌  | 168825/225000 [324:31:32<89:11:53,  5.72s/it] 
 75%|███████▌  | 168826/225000 [324:31:41<100:41:14,  6.45s/it]
 75%|███████▌  | 168827/225000 [324:31:44<86:19:21,  5.53s/it] 
 75%|███████▌  | 168828/225000 [324:31:52<96:30:55,  6.19s/it]
 75%|███████▌  | 168829/225000 [324:31:57<90:11:43,  5.78s/it]
 75%|███████▌  | 168830/225000 [324:32:01<81:31:45,  5.23s/it]
 75%|███████▌  | 168831/225000 [324:32:07<85:17:43,  5.47s/it]
 75%|███████▌  | 168832/225000 [324:32:15<97:32:40,  6.25s/it]
 75%|███████▌  | 168833/225000 [324:32:19<88:39:43,  5.68s/it]
 75%|███████▌  | 168834/225000 [324:32:24<84:05:37,  5.39s/it]
 75%|███████▌  | 168835/225000 [324:32:28<79:47:38,  5.11s/it]
 75%|███████▌  | 168836/225000 [324:32:32<75:39:28,  4.85s/it]
 75%|███████▌  | 168837/225000 [324:32:37<73:21:01,  4.70s/it]
 75%|███████▌  | 168838/225000 [324:32:42<75:33:24,  4.84s/it]
 75%|███████▌  | 168839/225000 [324:32:53<106:23:16,  6.82s/it]
 75%|███████▌  | 168840/225000 [324:32:57<89:47:58,  5.76s/it] 
 75%|███████▌  | 168841/225000 [324:33:02<85:51:51,  5.50s/it]
 75%|███████▌  | 168842/225000 [324:33:06<82:11:57,  5.27s/it]
 75%|███████▌  | 168843/225000 [324:33:11<81:33:55,  5.23s/it]
 75%|███████▌  | 168844/225000 [324:33:16<77:49:16,  4.99s/it]
 75%|███████▌  | 168845/225000 [324:33:21<76:21:59,  4.90s/it]
 75%|███████▌  | 168846/225000 [324:33:25<74:03:03,  4.75s/it]
 75%|███████▌  | 168847/225000 [324:33:30<76:21:26,  4.90s/it]
 75%|███████▌  | 168848/225000 [324:33:35<75:36:18,  4.85s/it]
 75%|███████▌  | 168849/225000 [324:33:40<74:54:48,  4.80s/it]
 75%|███████▌  | 168850/225000 [324:33:51<104:40:13,  6.71s/it]
                                                               
{'loss': '0.7598', 'grad_norm': '1.242', 'learning_rate': '7.974e-06', 'epoch': '0.7504', 'num_input_tokens_seen': 537543824, 'train_runtime': '1.168e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 168850/225000 [324:33:51<104:40:13,  6.71s/it]
 75%|███████▌  | 168851/225000 [324:33:55<94:53:11,  6.08s/it] 
 75%|███████▌  | 168852/225000 [324:33:59<83:00:34,  5.32s/it]
 75%|███████▌  | 168853/225000 [324:34:08<99:57:09,  6.41s/it]
 75%|███████▌  | 168854/225000 [324:34:11<86:32:11,  5.55s/it]
 75%|███████▌  | 168855/225000 [324:34:15<79:13:18,  5.08s/it]
 75%|███████▌  | 168856/225000 [324:34:20<78:25:44,  5.03s/it]
 75%|███████▌  | 168857/225000 [324:34:26<81:19:20,  5.21s/it]
 75%|███████▌  | 168858/225000 [324:34:31<80:06:34,  5.14s/it]
 75%|███████▌  | 168859/225000 [324:34:36<81:47:06,  5.24s/it]
 75%|███████▌  | 168860/225000 [324:34:41<81:05:22,  5.20s/it]
 75%|███████▌  | 168861/225000 [324:34:48<86:52:56,  5.57s/it]
 75%|███████▌  | 168862/225000 [324:34:57<102:51:53,  6.60s/it]
 75%|███████▌  | 168863/225000 [324:35:01<93:01:13,  5.97s/it] 
 75%|███████▌  | 168864/225000 [324:35:06<86:25:21,  5.54s/it]
 75%|███████▌  | 168865/225000 [324:35:10<81:34:06,  5.23s/it]
 75%|███████▌  | 168866/225000 [324:35:15<78:03:29,  5.01s/it]
 75%|███████▌  | 168867/225000 [324:35:19<74:02:32,  4.75s/it]
 75%|███████▌  | 168868/225000 [324:35:32<111:08:55,  7.13s/it]
 75%|███████▌  | 168869/225000 [324:35:35<94:12:27,  6.04s/it] 
 75%|███████▌  | 168870/225000 [324:35:40<88:30:21,  5.68s/it]
 75%|███████▌  | 168871/225000 [324:35:46<89:02:12,  5.71s/it]
 75%|███████▌  | 168872/225000 [324:35:51<85:47:14,  5.50s/it]
 75%|███████▌  | 168873/225000 [324:35:56<82:49:33,  5.31s/it]
 75%|███████▌  | 168874/225000 [324:36:01<83:36:34,  5.36s/it]
 75%|███████▌  | 168875/225000 [324:36:07<83:02:07,  5.33s/it]
 75%|███████▌  | 168876/225000 [324:36:11<80:42:44,  5.18s/it]
 75%|███████▌  | 168877/225000 [324:36:17<81:06:31,  5.20s/it]
 75%|███████▌  | 168878/225000 [324:36:21<79:24:33,  5.09s/it]
 75%|███████▌  | 168879/225000 [324:36:34<113:12:44,  7.26s/it]
 75%|███████▌  | 168880/225000 [324:36:37<95:36:33,  6.13s/it] 
 75%|███████▌  | 168881/225000 [324:36:41<82:55:59,  5.32s/it]
 75%|███████▌  | 168882/225000 [324:36:50<101:38:00,  6.52s/it]
 75%|███████▌  | 168883/225000 [324:36:55<93:37:06,  6.01s/it] 
 75%|███████▌  | 168884/225000 [324:37:00<89:38:50,  5.75s/it]
 75%|███████▌  | 168885/225000 [324:37:05<85:31:34,  5.49s/it]
 75%|███████▌  | 168886/225000 [324:37:10<84:47:23,  5.44s/it]
 75%|███████▌  | 168887/225000 [324:37:15<83:45:20,  5.37s/it]
 75%|███████▌  | 168888/225000 [324:37:20<80:33:25,  5.17s/it]
 75%|███████▌  | 168889/225000 [324:37:25<79:13:51,  5.08s/it]
 75%|███████▌  | 168890/225000 [324:37:30<77:47:39,  4.99s/it]
 75%|███████▌  | 168891/225000 [324:37:35<77:20:23,  4.96s/it]
 75%|███████▌  | 168892/225000 [324:37:39<76:39:57,  4.92s/it]
 75%|███████▌  | 168893/225000 [324:37:45<78:04:11,  5.01s/it]
 75%|███████▌  | 168894/225000 [324:37:49<75:50:00,  4.87s/it]
 75%|███████▌  | 168895/225000 [324:37:54<74:18:54,  4.77s/it]
 75%|███████▌  | 168896/225000 [324:38:06<110:36:09,  7.10s/it]
 75%|███████▌  | 168897/225000 [324:38:11<101:19:05,  6.50s/it]
 75%|███████▌  | 168898/225000 [324:38:19<107:12:16,  6.88s/it]
 75%|███████▌  | 168899/225000 [324:38:24<98:33:19,  6.32s/it] 
 75%|███████▌  | 168900/225000 [324:38:29<91:22:30,  5.86s/it]
                                                              
{'loss': '0.7672', 'grad_norm': '1.047', 'learning_rate': '7.961e-06', 'epoch': '0.7507', 'num_input_tokens_seen': 537674536, 'train_runtime': '1.169e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 168900/225000 [324:38:29<91:22:30,  5.86s/it]
 75%|███████▌  | 168901/225000 [324:38:34<86:30:00,  5.55s/it]
 75%|███████▌  | 168902/225000 [324:38:39<83:35:11,  5.36s/it]
 75%|███████▌  | 168903/225000 [324:38:43<80:44:10,  5.18s/it]
 75%|███████▌  | 168904/225000 [324:38:48<77:26:28,  4.97s/it]
 75%|███████▌  | 168905/225000 [324:38:53<75:49:46,  4.87s/it]
 75%|███████▌  | 168906/225000 [324:38:57<73:52:45,  4.74s/it]
 75%|███████▌  | 168907/225000 [324:39:02<74:45:40,  4.80s/it]
 75%|███████▌  | 168908/225000 [324:39:07<75:16:50,  4.83s/it]
 75%|███████▌  | 168909/225000 [324:39:12<76:30:55,  4.91s/it]
 75%|███████▌  | 168910/225000 [324:39:17<75:40:03,  4.86s/it]
 75%|███████▌  | 168911/225000 [324:39:27<102:24:50,  6.57s/it]
 75%|███████▌  | 168912/225000 [324:39:34<105:30:45,  6.77s/it]
 75%|███████▌  | 168913/225000 [324:39:38<89:59:43,  5.78s/it] 
 75%|███████▌  | 168914/225000 [324:39:48<108:46:09,  6.98s/it]
 75%|███████▌  | 168915/225000 [324:39:52<94:45:59,  6.08s/it] 
 75%|███████▌  | 168916/225000 [324:39:56<84:13:23,  5.41s/it]
 75%|███████▌  | 168917/225000 [324:39:59<77:02:29,  4.95s/it]
 75%|███████▌  | 168918/225000 [324:40:03<72:10:36,  4.63s/it]
 75%|███████▌  | 168919/225000 [324:40:07<68:36:25,  4.40s/it]
 75%|███████▌  | 168920/225000 [324:40:17<92:24:13,  5.93s/it]
 75%|███████▌  | 168921/225000 [324:40:22<91:45:23,  5.89s/it]
 75%|███████▌  | 168922/225000 [324:40:27<86:25:30,  5.55s/it]
 75%|███████▌  | 168923/225000 [324:40:32<81:34:09,  5.24s/it]
 75%|███████▌  | 168924/225000 [324:40:36<78:19:44,  5.03s/it]
 75%|███████▌  | 168925/225000 [324:40:41<75:54:18,  4.87s/it]
 75%|███████▌  | 168926/225000 [324:40:45<73:12:28,  4.70s/it]
 75%|███████▌  | 168927/225000 [324:40:53<86:22:03,  5.54s/it]
 75%|███████▌  | 168928/225000 [324:41:02<102:10:54,  6.56s/it]
 75%|███████▌  | 168929/225000 [324:41:07<98:08:44,  6.30s/it] 
 75%|███████▌  | 168930/225000 [324:41:14<98:52:42,  6.35s/it]
 75%|███████▌  | 168931/225000 [324:41:17<85:21:41,  5.48s/it]
 75%|███████▌  | 168932/225000 [324:41:22<80:18:23,  5.16s/it]
 75%|███████▌  | 168933/225000 [324:41:26<75:54:43,  4.87s/it]
 75%|███████▌  | 168934/225000 [324:41:30<74:23:58,  4.78s/it]
 75%|███████▌  | 168935/225000 [324:41:35<74:07:46,  4.76s/it]
 75%|███████▌  | 168936/225000 [324:41:47<108:19:18,  6.96s/it]
 75%|███████▌  | 168937/225000 [324:41:51<95:33:27,  6.14s/it] 
 75%|███████▌  | 168938/225000 [324:41:55<84:05:14,  5.40s/it]
 75%|███████▌  | 168939/225000 [324:42:04<100:10:20,  6.43s/it]
 75%|███████▌  | 168940/225000 [324:42:08<90:35:14,  5.82s/it] 
 75%|███████▌  | 168941/225000 [324:42:12<82:18:26,  5.29s/it]
 75%|███████▌  | 168942/225000 [324:42:16<75:03:45,  4.82s/it]
 75%|███████▌  | 168943/225000 [324:42:24<91:14:02,  5.86s/it]
 75%|███████▌  | 168944/225000 [324:42:31<93:32:51,  6.01s/it]
 75%|███████▌  | 168945/225000 [324:42:35<83:37:46,  5.37s/it]
 75%|███████▌  | 168946/225000 [324:42:39<78:43:03,  5.06s/it]
 75%|███████▌  | 168947/225000 [324:42:43<74:41:48,  4.80s/it]
 75%|███████▌  | 168948/225000 [324:42:47<72:32:43,  4.66s/it]
 75%|███████▌  | 168949/225000 [324:42:59<103:36:10,  6.65s/it]
 75%|███████▌  | 168950/225000 [324:43:02<89:16:15,  5.73s/it] 
                                                              
{'loss': '0.7621', 'grad_norm': '1.249', 'learning_rate': '7.947e-06', 'epoch': '0.7509', 'num_input_tokens_seen': 537804080, 'train_runtime': '1.169e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 168950/225000 [324:43:02<89:16:15,  5.73s/it]
 75%|███████▌  | 168951/225000 [324:43:06<79:59:18,  5.14s/it]
 75%|███████▌  | 168952/225000 [324:43:15<95:56:30,  6.16s/it]
 75%|███████▌  | 168953/225000 [324:43:18<83:34:33,  5.37s/it]
 75%|███████▌  | 168954/225000 [324:43:21<72:32:12,  4.66s/it]
 75%|███████▌  | 168955/225000 [324:43:31<95:42:29,  6.15s/it]
 75%|███████▌  | 168956/225000 [324:43:34<83:13:59,  5.35s/it]
 75%|███████▌  | 168957/225000 [324:43:42<95:11:38,  6.11s/it]
 75%|███████▌  | 168958/225000 [324:43:46<82:45:25,  5.32s/it]
 75%|███████▌  | 168959/225000 [324:43:51<81:19:54,  5.22s/it]
 75%|███████▌  | 168960/225000 [324:43:56<80:59:15,  5.20s/it]
 75%|███████▌  | 168961/225000 [324:44:01<80:34:44,  5.18s/it]
 75%|███████▌  | 168962/225000 [324:44:06<80:35:01,  5.18s/it]
 75%|███████▌  | 168963/225000 [324:44:12<82:20:11,  5.29s/it]
 75%|███████▌  | 168964/225000 [324:44:17<81:03:41,  5.21s/it]
 75%|███████▌  | 168965/225000 [324:44:22<81:06:41,  5.21s/it]
 75%|███████▌  | 168966/225000 [324:44:27<82:29:42,  5.30s/it]
 75%|███████▌  | 168967/225000 [324:44:33<82:14:03,  5.28s/it]
 75%|███████▌  | 168968/225000 [324:44:38<80:55:17,  5.20s/it]
 75%|███████▌  | 168969/225000 [324:44:43<81:15:08,  5.22s/it]
 75%|███████▌  | 168970/225000 [324:44:48<80:53:54,  5.20s/it]
 75%|███████▌  | 168971/225000 [324:44:54<82:42:03,  5.31s/it]
 75%|███████▌  | 168972/225000 [324:45:05<111:57:15,  7.19s/it]
 75%|███████▌  | 168973/225000 [324:45:11<107:03:32,  6.88s/it]
 75%|███████▌  | 168974/225000 [324:45:18<104:11:39,  6.70s/it]
 75%|███████▌  | 168975/225000 [324:45:21<88:54:37,  5.71s/it] 
 75%|███████▌  | 168976/225000 [324:45:28<94:14:39,  6.06s/it]
 75%|███████▌  | 168977/225000 [324:45:33<89:23:54,  5.74s/it]
 75%|███████▌  | 168978/225000 [324:45:39<89:50:35,  5.77s/it]
 75%|███████▌  | 168979/225000 [324:45:42<80:33:17,  5.18s/it]
 75%|███████▌  | 168980/225000 [324:45:51<97:45:51,  6.28s/it]
 75%|███████▌  | 168981/225000 [324:45:55<85:05:40,  5.47s/it]
 75%|███████▌  | 168982/225000 [324:45:59<76:49:03,  4.94s/it]
 75%|███████▌  | 168983/225000 [324:46:03<75:48:54,  4.87s/it]
 75%|███████▌  | 168984/225000 [324:46:08<76:24:17,  4.91s/it]
 75%|███████▌  | 168985/225000 [324:46:13<77:08:30,  4.96s/it]
 75%|███████▌  | 168986/225000 [324:46:18<76:40:52,  4.93s/it]
 75%|███████▌  | 168987/225000 [324:46:23<77:04:27,  4.95s/it]
 75%|███████▌  | 168988/225000 [324:46:28<76:30:24,  4.92s/it]
 75%|███████▌  | 168989/225000 [324:46:33<76:02:03,  4.89s/it]
 75%|███████▌  | 168990/225000 [324:46:38<76:59:25,  4.95s/it]
 75%|███████▌  | 168991/225000 [324:46:43<76:15:24,  4.90s/it]
 75%|███████▌  | 168992/225000 [324:46:51<92:27:56,  5.94s/it]
 75%|███████▌  | 168993/225000 [324:47:01<111:16:51,  7.15s/it]
 75%|███████▌  | 168994/225000 [324:47:06<98:58:26,  6.36s/it] 
 75%|███████▌  | 168995/225000 [324:47:10<89:18:36,  5.74s/it]
 75%|███████▌  | 168996/225000 [324:47:15<84:34:02,  5.44s/it]
 75%|███████▌  | 168997/225000 [324:47:19<79:23:25,  5.10s/it]
 75%|███████▌  | 168998/225000 [324:47:24<77:57:40,  5.01s/it]
 75%|███████▌  | 168999/225000 [324:47:32<94:17:56,  6.06s/it]
 75%|███████▌  | 169000/225000 [324:47:40<101:25:39,  6.52s/it]
                                                               
{'loss': '0.758', 'grad_norm': '1.194', 'learning_rate': '7.934e-06', 'epoch': '0.7511', 'num_input_tokens_seen': 537935496, 'train_runtime': '1.169e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169000/225000 [324:47:40<101:25:39,  6.52s/it]
 75%|███████▌  | 169001/225000 [324:47:45<93:49:50,  6.03s/it] 
 75%|███████▌  | 169002/225000 [324:47:49<86:51:50,  5.58s/it]
 75%|███████▌  | 169003/225000 [324:47:54<84:02:10,  5.40s/it]
 75%|███████▌  | 169004/225000 [324:47:59<81:52:17,  5.26s/it]
 75%|███████▌  | 169005/225000 [324:48:04<81:31:13,  5.24s/it]
 75%|███████▌  | 169006/225000 [324:48:10<80:53:02,  5.20s/it]
 75%|███████▌  | 169007/225000 [324:48:15<81:18:20,  5.23s/it]
 75%|███████▌  | 169008/225000 [324:48:20<79:54:45,  5.14s/it]
 75%|███████▌  | 169009/225000 [324:48:25<79:46:21,  5.13s/it]
 75%|███████▌  | 169010/225000 [324:48:30<80:17:08,  5.16s/it]
 75%|███████▌  | 169011/225000 [324:48:35<79:47:06,  5.13s/it]
 75%|███████▌  | 169012/225000 [324:48:40<78:46:31,  5.07s/it]
 75%|███████▌  | 169013/225000 [324:48:45<79:33:23,  5.12s/it]
 75%|███████▌  | 169014/225000 [324:48:50<77:08:32,  4.96s/it]
 75%|███████▌  | 169015/225000 [324:48:54<73:59:51,  4.76s/it]
 75%|███████▌  | 169016/225000 [324:49:05<104:22:48,  6.71s/it]
 75%|███████▌  | 169017/225000 [324:49:11<99:21:50,  6.39s/it] 
 75%|███████▌  | 169018/225000 [324:49:20<108:40:46,  6.99s/it]
 75%|███████▌  | 169019/225000 [324:49:23<93:08:46,  5.99s/it] 
 75%|███████▌  | 169020/225000 [324:49:29<91:53:03,  5.91s/it]
 75%|███████▌  | 169021/225000 [324:49:33<85:41:40,  5.51s/it]
 75%|███████▌  | 169022/225000 [324:49:41<95:27:50,  6.14s/it]
 75%|███████▌  | 169023/225000 [324:49:45<83:39:52,  5.38s/it]
 75%|███████▌  | 169024/225000 [324:49:53<95:58:05,  6.17s/it]
 75%|███████▌  | 169025/225000 [324:49:57<85:11:01,  5.48s/it]
 75%|███████▌  | 169026/225000 [324:50:01<78:07:08,  5.02s/it]
 75%|███████▌  | 169027/225000 [324:50:04<71:27:46,  4.60s/it]
 75%|███████▌  | 169028/225000 [324:50:13<92:46:06,  5.97s/it]
 75%|███████▌  | 169029/225000 [324:50:19<90:40:36,  5.83s/it]
 75%|███████▌  | 169030/225000 [324:50:23<82:43:34,  5.32s/it]
 75%|███████▌  | 169031/225000 [324:50:27<77:47:27,  5.00s/it]
 75%|███████▌  | 169032/225000 [324:50:32<75:18:49,  4.84s/it]
 75%|███████▌  | 169033/225000 [324:50:37<76:14:18,  4.90s/it]
 75%|███████▌  | 169034/225000 [324:50:48<105:24:56,  6.78s/it]
 75%|███████▌  | 169035/225000 [324:50:51<88:30:52,  5.69s/it] 
 75%|███████▌  | 169036/225000 [324:50:57<90:12:57,  5.80s/it]
 75%|███████▌  | 169037/225000 [324:51:01<80:03:52,  5.15s/it]
 75%|███████▌  | 169038/225000 [324:51:09<92:23:46,  5.94s/it]
 75%|███████▌  | 169039/225000 [324:51:13<84:37:02,  5.44s/it]
 75%|███████▌  | 169040/225000 [324:51:17<78:44:16,  5.07s/it]
 75%|███████▌  | 169041/225000 [324:51:22<77:08:14,  4.96s/it]
 75%|███████▌  | 169042/225000 [324:51:26<73:45:58,  4.75s/it]
 75%|███████▌  | 169043/225000 [324:51:30<71:26:46,  4.60s/it]
 75%|███████▌  | 169044/225000 [324:51:38<84:10:47,  5.42s/it]
 75%|███████▌  | 169045/225000 [324:51:46<98:08:29,  6.31s/it]
 75%|███████▌  | 169046/225000 [324:51:50<87:20:57,  5.62s/it]
 75%|███████▌  | 169047/225000 [324:51:54<81:06:42,  5.22s/it]
 75%|███████▌  | 169048/225000 [324:51:59<78:20:02,  5.04s/it]
 75%|███████▌  | 169049/225000 [324:52:07<94:25:29,  6.08s/it]
 75%|███████▌  | 169050/225000 [324:52:14<98:54:07,  6.36s/it]
                                                              
{'loss': '0.7729', 'grad_norm': '1.083', 'learning_rate': '7.921e-06', 'epoch': '0.7513', 'num_input_tokens_seen': 538065104, 'train_runtime': '1.17e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169050/225000 [324:52:14<98:54:07,  6.36s/it]
 75%|███████▌  | 169051/225000 [324:52:18<85:23:41,  5.49s/it]
 75%|███████▌  | 169052/225000 [324:52:25<93:40:47,  6.03s/it]
 75%|███████▌  | 169053/225000 [324:52:30<87:57:11,  5.66s/it]
 75%|███████▌  | 169054/225000 [324:52:34<81:13:38,  5.23s/it]
 75%|███████▌  | 169055/225000 [324:52:39<78:16:48,  5.04s/it]
 75%|███████▌  | 169056/225000 [324:52:43<76:27:02,  4.92s/it]
 75%|███████▌  | 169057/225000 [324:52:48<74:15:53,  4.78s/it]
 75%|███████▌  | 169058/225000 [324:52:52<73:33:30,  4.73s/it]
 75%|███████▌  | 169059/225000 [324:52:58<77:34:01,  4.99s/it]
 75%|███████▌  | 169060/225000 [324:53:10<111:11:04,  7.16s/it]
 75%|███████▌  | 169061/225000 [324:53:14<93:37:05,  6.02s/it] 
 75%|███████▌  | 169062/225000 [324:53:20<94:27:59,  6.08s/it]
 75%|███████▌  | 169063/225000 [324:53:26<92:42:22,  5.97s/it]
 75%|███████▌  | 169064/225000 [324:53:30<86:47:25,  5.59s/it]
 75%|███████▌  | 169065/225000 [324:53:36<85:24:53,  5.50s/it]
 75%|███████▌  | 169066/225000 [324:53:41<83:27:23,  5.37s/it]
 75%|███████▌  | 169067/225000 [324:53:46<82:21:27,  5.30s/it]
 75%|███████▌  | 169068/225000 [324:53:51<82:10:50,  5.29s/it]
 75%|███████▌  | 169069/225000 [324:53:56<80:32:45,  5.18s/it]
 75%|███████▌  | 169070/225000 [324:54:01<80:52:15,  5.21s/it]
 75%|███████▌  | 169071/225000 [324:54:06<79:43:48,  5.13s/it]
 75%|███████▌  | 169072/225000 [324:54:11<79:09:07,  5.09s/it]
 75%|███████▌  | 169073/225000 [324:54:16<78:34:30,  5.06s/it]
 75%|███████▌  | 169074/225000 [324:54:21<78:02:09,  5.02s/it]
 75%|███████▌  | 169075/225000 [324:54:33<111:07:05,  7.15s/it]
 75%|███████▌  | 169076/225000 [324:54:36<91:28:24,  5.89s/it] 
 75%|███████▌  | 169077/225000 [324:54:44<99:16:31,  6.39s/it]
 75%|███████▌  | 169078/225000 [324:54:49<93:56:58,  6.05s/it]
 75%|███████▌  | 169079/225000 [324:54:54<87:49:01,  5.65s/it]
 75%|███████▌  | 169080/225000 [324:54:58<83:10:42,  5.35s/it]
 75%|███████▌  | 169081/225000 [324:55:03<81:16:21,  5.23s/it]
 75%|███████▌  | 169082/225000 [324:55:08<80:43:01,  5.20s/it]
 75%|███████▌  | 169083/225000 [324:55:14<81:45:44,  5.26s/it]
 75%|███████▌  | 169084/225000 [324:55:19<81:26:46,  5.24s/it]
 75%|███████▌  | 169085/225000 [324:55:24<80:42:15,  5.20s/it]
 75%|███████▌  | 169086/225000 [324:55:29<78:12:49,  5.04s/it]
 75%|███████▌  | 169087/225000 [324:55:34<78:42:25,  5.07s/it]
 75%|███████▌  | 169088/225000 [324:55:39<79:08:30,  5.10s/it]
 75%|███████▌  | 169089/225000 [324:55:44<80:04:15,  5.16s/it]
 75%|███████▌  | 169090/225000 [324:55:49<79:01:16,  5.09s/it]
 75%|███████▌  | 169091/225000 [324:55:59<101:32:24,  6.54s/it]
 75%|███████▌  | 169092/225000 [324:56:08<113:59:25,  7.34s/it]
 75%|███████▌  | 169093/225000 [324:56:12<96:18:30,  6.20s/it] 
 75%|███████▌  | 169094/225000 [324:56:19<101:36:48,  6.54s/it]
 75%|███████▌  | 169095/225000 [324:56:24<93:19:27,  6.01s/it] 
 75%|███████▌  | 169096/225000 [324:56:29<88:47:35,  5.72s/it]
 75%|███████▌  | 169097/225000 [324:56:35<90:13:33,  5.81s/it]
 75%|███████▌  | 169098/225000 [324:56:40<85:12:39,  5.49s/it]
 75%|███████▌  | 169099/225000 [324:56:45<84:36:06,  5.45s/it]
 75%|███████▌  | 169100/225000 [324:56:52<88:41:08,  5.71s/it]
                                                              
{'loss': '0.7577', 'grad_norm': '1.183', 'learning_rate': '7.907e-06', 'epoch': '0.7516', 'num_input_tokens_seen': 538194792, 'train_runtime': '1.17e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169100/225000 [324:56:52<88:41:08,  5.71s/it]
 75%|███████▌  | 169101/225000 [324:56:55<78:53:50,  5.08s/it]
 75%|███████▌  | 169102/225000 [324:57:00<77:59:23,  5.02s/it]
 75%|███████▌  | 169103/225000 [324:57:05<77:48:51,  5.01s/it]
 75%|███████▌  | 169104/225000 [324:57:10<76:29:46,  4.93s/it]
 75%|███████▌  | 169105/225000 [324:57:15<75:53:13,  4.89s/it]
 75%|███████▌  | 169106/225000 [324:57:19<73:24:33,  4.73s/it]
 75%|███████▌  | 169107/225000 [324:57:24<73:10:55,  4.71s/it]
 75%|███████▌  | 169108/225000 [324:57:35<106:24:17,  6.85s/it]
 75%|███████▌  | 169109/225000 [324:57:39<90:29:08,  5.83s/it] 
 75%|███████▌  | 169110/225000 [324:57:46<98:11:41,  6.32s/it]
 75%|███████▌  | 169111/225000 [324:57:50<87:44:24,  5.65s/it]
 75%|███████▌  | 169112/225000 [324:57:55<83:30:03,  5.38s/it]
 75%|███████▌  | 169113/225000 [324:57:59<77:46:30,  5.01s/it]
 75%|███████▌  | 169114/225000 [324:58:04<74:14:03,  4.78s/it]
 75%|███████▌  | 169115/225000 [324:58:12<89:18:17,  5.75s/it]
 75%|███████▌  | 169116/225000 [324:58:21<105:28:51,  6.79s/it]
 75%|███████▌  | 169117/225000 [324:58:24<89:24:03,  5.76s/it] 
 75%|███████▌  | 169118/225000 [324:58:30<91:39:00,  5.90s/it]
 75%|███████▌  | 169119/225000 [324:58:36<91:02:29,  5.87s/it]
 75%|███████▌  | 169120/225000 [324:58:40<80:19:38,  5.17s/it]
 75%|███████▌  | 169121/225000 [324:58:44<76:59:30,  4.96s/it]
 75%|███████▌  | 169122/225000 [324:58:49<76:40:23,  4.94s/it]
 75%|███████▌  | 169123/225000 [324:58:54<76:56:01,  4.96s/it]
 75%|███████▌  | 169124/225000 [324:58:59<77:51:45,  5.02s/it]
 75%|███████▌  | 169125/225000 [324:59:04<78:21:24,  5.05s/it]
 75%|███████▌  | 169126/225000 [324:59:09<76:40:08,  4.94s/it]
 75%|███████▌  | 169127/225000 [324:59:14<76:38:16,  4.94s/it]
 75%|███████▌  | 169128/225000 [324:59:19<77:20:04,  4.98s/it]
 75%|███████▌  | 169129/225000 [324:59:31<109:06:06,  7.03s/it]
 75%|███████▌  | 169130/225000 [324:59:34<92:27:15,  5.96s/it] 
 75%|███████▌  | 169131/225000 [324:59:38<82:49:57,  5.34s/it]
 75%|███████▌  | 169132/225000 [324:59:46<92:56:42,  5.99s/it]
 75%|███████▌  | 169133/225000 [324:59:52<92:44:34,  5.98s/it]
 75%|███████▌  | 169134/225000 [324:59:56<84:56:28,  5.47s/it]
 75%|███████▌  | 169135/225000 [325:00:00<79:03:12,  5.09s/it]
 75%|███████▌  | 169136/225000 [325:00:05<75:46:08,  4.88s/it]
 75%|███████▌  | 169137/225000 [325:00:13<90:40:06,  5.84s/it]
 75%|███████▌  | 169138/225000 [325:00:20<99:43:49,  6.43s/it]
 75%|███████▌  | 169139/225000 [325:00:25<91:57:29,  5.93s/it]
 75%|███████▌  | 169140/225000 [325:00:32<95:42:09,  6.17s/it]
 75%|███████▌  | 169141/225000 [325:00:36<87:42:42,  5.65s/it]
 75%|███████▌  | 169142/225000 [325:00:41<83:10:13,  5.36s/it]
 75%|███████▌  | 169143/225000 [325:00:46<80:25:10,  5.18s/it]
 75%|███████▌  | 169144/225000 [325:00:51<78:45:26,  5.08s/it]
 75%|███████▌  | 169145/225000 [325:00:55<77:02:11,  4.97s/it]
 75%|███████▌  | 169146/225000 [325:01:00<74:30:13,  4.80s/it]
 75%|███████▌  | 169147/225000 [325:01:05<75:31:37,  4.87s/it]
 75%|███████▌  | 169148/225000 [325:01:10<76:01:44,  4.90s/it]
 75%|███████▌  | 169149/225000 [325:01:15<75:21:47,  4.86s/it]
 75%|███████▌  | 169150/225000 [325:01:19<74:33:38,  4.81s/it]
                                                              
{'loss': '0.7662', 'grad_norm': '1.241', 'learning_rate': '7.894e-06', 'epoch': '0.7518', 'num_input_tokens_seen': 538325048, 'train_runtime': '1.17e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169150/225000 [325:01:19<74:33:38,  4.81s/it]
 75%|███████▌  | 169151/225000 [325:01:24<75:32:19,  4.87s/it]
 75%|███████▌  | 169152/225000 [325:01:29<74:59:46,  4.83s/it]
 75%|███████▌  | 169153/225000 [325:01:40<105:09:19,  6.78s/it]
 75%|███████▌  | 169154/225000 [325:01:48<107:25:30,  6.92s/it]
 75%|███████▌  | 169155/225000 [325:01:51<91:45:25,  5.92s/it] 
 75%|███████▌  | 169156/225000 [325:02:00<106:44:58,  6.88s/it]
 75%|███████▌  | 169157/225000 [325:02:04<91:26:56,  5.90s/it] 
 75%|███████▌  | 169158/225000 [325:02:11<96:34:19,  6.23s/it]
 75%|███████▌  | 169159/225000 [325:02:15<85:06:20,  5.49s/it]
 75%|███████▌  | 169160/225000 [325:02:19<78:46:38,  5.08s/it]
 75%|███████▌  | 169161/225000 [325:02:24<81:12:39,  5.24s/it]
 75%|███████▌  | 169162/225000 [325:02:34<99:41:39,  6.43s/it]
 75%|███████▌  | 169163/225000 [325:02:38<89:54:02,  5.80s/it]
 75%|███████▌  | 169164/225000 [325:02:45<94:32:48,  6.10s/it]
 75%|███████▌  | 169165/225000 [325:02:49<85:30:11,  5.51s/it]
 75%|███████▌  | 169166/225000 [325:02:53<80:02:11,  5.16s/it]
 75%|███████▌  | 169167/225000 [325:02:57<74:50:49,  4.83s/it]
 75%|███████▌  | 169168/225000 [325:03:02<73:55:57,  4.77s/it]
 75%|███████▌  | 169169/225000 [325:03:07<73:17:44,  4.73s/it]
 75%|███████▌  | 169170/225000 [325:03:14<88:30:03,  5.71s/it]
 75%|███████▌  | 169171/225000 [325:03:24<103:55:29,  6.70s/it]
 75%|███████▌  | 169172/225000 [325:03:27<89:48:15,  5.79s/it] 
 75%|███████▌  | 169173/225000 [325:03:31<81:00:37,  5.22s/it]
 75%|███████▌  | 169174/225000 [325:03:35<75:57:19,  4.90s/it]
 75%|███████▌  | 169175/225000 [325:03:45<98:22:18,  6.34s/it]
 75%|███████▌  | 169176/225000 [325:03:49<85:49:06,  5.53s/it]
 75%|███████▌  | 169177/225000 [325:03:54<85:41:44,  5.53s/it]
 75%|███████▌  | 169178/225000 [325:04:01<93:03:47,  6.00s/it]
 75%|███████▌  | 169179/225000 [325:04:05<82:06:04,  5.29s/it]
 75%|███████▌  | 169180/225000 [325:04:09<75:33:02,  4.87s/it]
 75%|███████▌  | 169181/225000 [325:04:18<97:49:52,  6.31s/it]
 75%|███████▌  | 169182/225000 [325:04:22<83:32:20,  5.39s/it]
 75%|███████▌  | 169183/225000 [325:04:30<97:27:01,  6.29s/it]
 75%|███████▌  | 169184/225000 [325:04:34<85:23:16,  5.51s/it]
 75%|███████▌  | 169185/225000 [325:04:41<93:54:32,  6.06s/it]
 75%|███████▌  | 169186/225000 [325:04:44<81:34:51,  5.26s/it]
 75%|███████▌  | 169187/225000 [325:04:53<98:57:43,  6.38s/it]
 75%|███████▌  | 169188/225000 [325:04:57<87:32:47,  5.65s/it]
 75%|███████▌  | 169189/225000 [325:05:02<82:46:32,  5.34s/it]
 75%|███████▌  | 169190/225000 [325:05:07<80:52:50,  5.22s/it]
 75%|███████▌  | 169191/225000 [325:05:11<76:32:35,  4.94s/it]
 75%|███████▌  | 169192/225000 [325:05:16<74:47:22,  4.82s/it]
 75%|███████▌  | 169193/225000 [325:05:20<72:59:35,  4.71s/it]
 75%|███████▌  | 169194/225000 [325:05:25<71:46:12,  4.63s/it]
 75%|███████▌  | 169195/225000 [325:05:36<100:42:26,  6.50s/it]
 75%|███████▌  | 169196/225000 [325:05:41<96:03:37,  6.20s/it] 
 75%|███████▌  | 169197/225000 [325:05:45<83:49:33,  5.41s/it]
 75%|███████▌  | 169198/225000 [325:05:54<102:45:19,  6.63s/it]
 75%|███████▌  | 169199/225000 [325:05:58<90:23:05,  5.83s/it] 
 75%|███████▌  | 169200/225000 [325:06:03<84:14:05,  5.43s/it]
                                                              
{'loss': '0.7709', 'grad_norm': '1.103', 'learning_rate': '7.881e-06', 'epoch': '0.752', 'num_input_tokens_seen': 538455600, 'train_runtime': '1.17e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169200/225000 [325:06:03<84:14:05,  5.43s/it]
 75%|███████▌  | 169201/225000 [325:06:07<80:25:04,  5.19s/it]
 75%|███████▌  | 169202/225000 [325:06:12<78:25:48,  5.06s/it]
 75%|███████▌  | 169203/225000 [325:06:16<75:25:17,  4.87s/it]
 75%|███████▌  | 169204/225000 [325:06:21<75:30:22,  4.87s/it]
 75%|███████▌  | 169205/225000 [325:06:26<75:12:02,  4.85s/it]
 75%|███████▌  | 169206/225000 [325:06:31<76:06:08,  4.91s/it]
 75%|███████▌  | 169207/225000 [325:06:41<99:48:43,  6.44s/it]
 75%|███████▌  | 169208/225000 [325:06:49<107:43:35,  6.95s/it]
 75%|███████▌  | 169209/225000 [325:06:53<91:20:39,  5.89s/it] 
 75%|███████▌  | 169210/225000 [325:07:02<105:46:10,  6.83s/it]
 75%|███████▌  | 169211/225000 [325:07:06<92:55:21,  6.00s/it] 
 75%|███████▌  | 169212/225000 [325:07:09<82:40:16,  5.33s/it]
 75%|███████▌  | 169213/225000 [325:07:14<78:43:32,  5.08s/it]
 75%|███████▌  | 169214/225000 [325:07:19<78:18:16,  5.05s/it]
 75%|███████▌  | 169215/225000 [325:07:24<77:23:24,  4.99s/it]
 75%|███████▌  | 169216/225000 [325:07:29<79:16:37,  5.12s/it]
 75%|███████▌  | 169217/225000 [325:07:34<77:01:53,  4.97s/it]
 75%|███████▌  | 169218/225000 [325:07:39<76:07:52,  4.91s/it]
 75%|███████▌  | 169219/225000 [325:07:44<76:53:46,  4.96s/it]
 75%|███████▌  | 169220/225000 [325:07:49<77:38:27,  5.01s/it]
 75%|███████▌  | 169221/225000 [325:07:54<79:34:02,  5.14s/it]
 75%|███████▌  | 169222/225000 [325:07:59<78:41:03,  5.08s/it]
 75%|███████▌  | 169223/225000 [325:08:04<79:00:42,  5.10s/it]
 75%|███████▌  | 169224/225000 [325:08:11<85:18:41,  5.51s/it]
 75%|███████▌  | 169225/225000 [325:08:23<115:59:41,  7.49s/it]
 75%|███████▌  | 169226/225000 [325:08:26<96:16:06,  6.21s/it] 
 75%|███████▌  | 169227/225000 [325:08:36<112:04:46,  7.23s/it]
 75%|███████▌  | 169228/225000 [325:08:40<95:43:50,  6.18s/it] 
 75%|███████▌  | 169229/225000 [325:08:43<84:23:24,  5.45s/it]
 75%|███████▌  | 169230/225000 [325:08:48<81:38:46,  5.27s/it]
 75%|███████▌  | 169231/225000 [325:08:53<78:54:14,  5.09s/it]
 75%|███████▌  | 169232/225000 [325:08:58<78:28:08,  5.07s/it]
 75%|███████▌  | 169233/225000 [325:09:03<77:43:35,  5.02s/it]
 75%|███████▌  | 169234/225000 [325:09:07<74:13:10,  4.79s/it]
 75%|███████▌  | 169235/225000 [325:09:12<75:38:02,  4.88s/it]
 75%|███████▌  | 169236/225000 [325:09:17<75:21:16,  4.86s/it]
 75%|███████▌  | 169237/225000 [325:09:29<110:23:45,  7.13s/it]
 75%|███████▌  | 169238/225000 [325:09:35<104:53:48,  6.77s/it]
 75%|███████▌  | 169239/225000 [325:09:39<92:59:44,  6.00s/it] 
 75%|███████▌  | 169240/225000 [325:09:43<83:23:46,  5.38s/it]
 75%|███████▌  | 169241/225000 [325:09:48<79:25:42,  5.13s/it]
 75%|███████▌  | 169242/225000 [325:09:52<75:21:41,  4.87s/it]
 75%|███████▌  | 169243/225000 [325:10:03<100:50:44,  6.51s/it]
 75%|███████▌  | 169244/225000 [325:10:07<89:37:52,  5.79s/it] 
 75%|███████▌  | 169245/225000 [325:10:11<83:44:02,  5.41s/it]
 75%|███████▌  | 169246/225000 [325:10:16<80:32:17,  5.20s/it]
 75%|███████▌  | 169247/225000 [325:10:20<77:43:46,  5.02s/it]
 75%|███████▌  | 169248/225000 [325:10:28<89:08:47,  5.76s/it]
 75%|███████▌  | 169249/225000 [325:10:36<101:49:22,  6.57s/it]
 75%|███████▌  | 169250/225000 [325:10:40<90:17:09,  5.83s/it] 
                                                              
{'loss': '0.772', 'grad_norm': '1.25', 'learning_rate': '7.867e-06', 'epoch': '0.7522', 'num_input_tokens_seen': 538585456, 'train_runtime': '1.171e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169250/225000 [325:10:40<90:17:09,  5.83s/it]
 75%|███████▌  | 169251/225000 [325:10:45<82:30:25,  5.33s/it]
 75%|███████▌  | 169252/225000 [325:10:49<76:41:52,  4.95s/it]
 75%|███████▌  | 169253/225000 [325:10:54<78:57:43,  5.10s/it]
 75%|███████▌  | 169254/225000 [325:11:04<99:52:46,  6.45s/it]
 75%|███████▌  | 169255/225000 [325:11:08<91:05:03,  5.88s/it]
 75%|███████▌  | 169256/225000 [325:11:13<83:57:44,  5.42s/it]
 75%|███████▌  | 169257/225000 [325:11:17<80:55:13,  5.23s/it]
 75%|███████▌  | 169258/225000 [325:11:22<77:41:51,  5.02s/it]
 75%|███████▌  | 169259/225000 [325:11:26<74:46:12,  4.83s/it]
 75%|███████▌  | 169260/225000 [325:11:31<74:07:45,  4.79s/it]
 75%|███████▌  | 169261/225000 [325:11:42<103:32:41,  6.69s/it]
 75%|███████▌  | 169262/225000 [325:11:48<97:21:43,  6.29s/it] 
 75%|███████▌  | 169263/225000 [325:11:51<84:40:39,  5.47s/it]
 75%|███████▌  | 169264/225000 [325:11:58<89:19:01,  5.77s/it]
 75%|███████▌  | 169265/225000 [325:12:04<90:10:43,  5.82s/it]
 75%|███████▌  | 169266/225000 [325:12:10<91:32:44,  5.91s/it]
 75%|███████▌  | 169267/225000 [325:12:15<89:47:35,  5.80s/it]
 75%|███████▌  | 169268/225000 [325:12:21<90:45:59,  5.86s/it]
 75%|███████▌  | 169269/225000 [325:12:25<81:33:26,  5.27s/it]
 75%|███████▌  | 169270/225000 [325:12:29<77:03:45,  4.98s/it]
 75%|███████▌  | 169271/225000 [325:12:33<71:25:27,  4.61s/it]
 75%|███████▌  | 169272/225000 [325:12:38<73:03:33,  4.72s/it]
 75%|███████▌  | 169273/225000 [325:12:49<103:43:33,  6.70s/it]
 75%|███████▌  | 169274/225000 [325:12:53<87:01:52,  5.62s/it] 
 75%|███████▌  | 169275/225000 [325:12:56<75:42:42,  4.89s/it]
 75%|███████▌  | 169276/225000 [325:13:05<96:24:53,  6.23s/it]
 75%|███████▌  | 169277/225000 [325:13:09<83:48:40,  5.41s/it]
 75%|███████▌  | 169278/225000 [325:13:15<88:58:21,  5.75s/it]
 75%|███████▌  | 169279/225000 [325:13:19<79:37:42,  5.14s/it]
 75%|███████▌  | 169280/225000 [325:13:24<81:07:34,  5.24s/it]
 75%|███████▌  | 169281/225000 [325:13:32<90:32:33,  5.85s/it]
 75%|███████▌  | 169282/225000 [325:13:35<80:28:25,  5.20s/it]
 75%|███████▌  | 169283/225000 [325:13:39<74:43:59,  4.83s/it]
 75%|███████▌  | 169284/225000 [325:13:45<77:08:33,  4.98s/it]
 75%|███████▌  | 169285/225000 [325:13:54<98:00:36,  6.33s/it]
 75%|███████▌  | 169286/225000 [325:13:57<84:21:35,  5.45s/it]
 75%|███████▌  | 169287/225000 [325:14:02<82:25:45,  5.33s/it]
 75%|███████▌  | 169288/225000 [325:14:08<83:34:41,  5.40s/it]
 75%|███████▌  | 169289/225000 [325:14:13<82:48:24,  5.35s/it]
 75%|███████▌  | 169290/225000 [325:14:18<80:46:28,  5.22s/it]
 75%|███████▌  | 169291/225000 [325:14:23<80:19:00,  5.19s/it]
 75%|███████▌  | 169292/225000 [325:14:29<80:54:47,  5.23s/it]
 75%|███████▌  | 169293/225000 [325:14:33<78:55:43,  5.10s/it]
 75%|███████▌  | 169294/225000 [325:14:39<79:46:45,  5.16s/it]
 75%|███████▌  | 169295/225000 [325:14:44<80:15:35,  5.19s/it]
 75%|███████▌  | 169296/225000 [325:14:49<79:46:28,  5.16s/it]
 75%|███████▌  | 169297/225000 [325:14:54<80:25:19,  5.20s/it]
 75%|███████▌  | 169298/225000 [325:15:00<80:55:14,  5.23s/it]
 75%|███████▌  | 169299/225000 [325:15:05<79:59:19,  5.17s/it]
 75%|███████▌  | 169300/225000 [325:15:10<79:51:04,  5.16s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.222', 'learning_rate': '7.854e-06', 'epoch': '0.7524', 'num_input_tokens_seen': 538714928, 'train_runtime': '1.171e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169300/225000 [325:15:10<79:51:04,  5.16s/it]
 75%|███████▌  | 169301/225000 [325:15:15<79:49:27,  5.16s/it]
 75%|███████▌  | 169302/225000 [325:15:27<113:13:34,  7.32s/it]
 75%|███████▌  | 169303/225000 [325:15:31<98:08:14,  6.34s/it] 
 75%|███████▌  | 169304/225000 [325:15:35<86:23:20,  5.58s/it]
 75%|███████▌  | 169305/225000 [325:15:45<106:48:45,  6.90s/it]
 75%|███████▌  | 169306/225000 [325:15:50<96:35:31,  6.24s/it] 
 75%|███████▌  | 169307/225000 [325:15:56<96:57:46,  6.27s/it]
 75%|███████▌  | 169308/225000 [325:16:01<87:55:44,  5.68s/it]
 75%|███████▌  | 169309/225000 [325:16:07<92:54:09,  6.01s/it]
 75%|███████▌  | 169310/225000 [325:16:12<85:59:52,  5.56s/it]
 75%|███████▌  | 169311/225000 [325:16:16<81:47:08,  5.29s/it]
 75%|███████▌  | 169312/225000 [325:16:22<80:54:05,  5.23s/it]
 75%|███████▌  | 169313/225000 [325:16:27<79:30:49,  5.14s/it]
 75%|███████▌  | 169314/225000 [325:16:31<74:44:55,  4.83s/it]
 75%|███████▌  | 169315/225000 [325:16:35<73:35:45,  4.76s/it]
 75%|███████▌  | 169316/225000 [325:16:40<74:19:38,  4.81s/it]
 75%|███████▌  | 169317/225000 [325:16:45<75:01:15,  4.85s/it]
 75%|███████▌  | 169318/225000 [325:16:50<73:08:02,  4.73s/it]
 75%|███████▌  | 169319/225000 [325:16:54<74:09:18,  4.79s/it]
 75%|███████▌  | 169320/225000 [325:16:59<75:01:28,  4.85s/it]
 75%|███████▌  | 169321/225000 [325:17:12<110:53:43,  7.17s/it]
 75%|███████▌  | 169322/225000 [325:17:16<95:28:16,  6.17s/it] 
 75%|███████▌  | 169323/225000 [325:17:23<101:21:44,  6.55s/it]
 75%|███████▌  | 169324/225000 [325:17:27<85:42:56,  5.54s/it] 
 75%|███████▌  | 169325/225000 [325:17:36<102:21:56,  6.62s/it]
 75%|███████▌  | 169326/225000 [325:17:39<88:12:13,  5.70s/it] 
 75%|███████▌  | 169327/225000 [325:17:43<77:50:08,  5.03s/it]
 75%|███████▌  | 169328/225000 [325:17:52<99:39:22,  6.44s/it]
 75%|███████▌  | 169329/225000 [325:17:57<92:51:05,  6.00s/it]
 75%|███████▌  | 169330/225000 [325:18:03<92:42:00,  5.99s/it]
 75%|███████▌  | 169331/225000 [325:18:09<88:56:26,  5.75s/it]
 75%|███████▌  | 169332/225000 [325:18:13<81:30:23,  5.27s/it]
 75%|███████▌  | 169333/225000 [325:18:17<76:52:51,  4.97s/it]
 75%|███████▌  | 169334/225000 [325:18:22<75:54:17,  4.91s/it]
 75%|███████▌  | 169335/225000 [325:18:26<74:00:00,  4.79s/it]
 75%|███████▌  | 169336/225000 [325:18:31<74:43:01,  4.83s/it]
 75%|███████▌  | 169337/225000 [325:18:36<74:45:10,  4.83s/it]
 75%|███████▌  | 169338/225000 [325:18:40<73:01:05,  4.72s/it]
 75%|███████▌  | 169339/225000 [325:18:49<92:54:06,  6.01s/it]
 75%|███████▌  | 169340/225000 [325:18:57<100:28:48,  6.50s/it]
 75%|███████▌  | 169341/225000 [325:19:01<87:19:56,  5.65s/it] 
 75%|███████▌  | 169342/225000 [325:19:05<79:52:01,  5.17s/it]
 75%|███████▌  | 169343/225000 [325:19:09<74:57:24,  4.85s/it]
 75%|███████▌  | 169344/225000 [325:19:21<106:46:57,  6.91s/it]
 75%|███████▌  | 169345/225000 [325:19:24<89:58:30,  5.82s/it] 
 75%|███████▌  | 169346/225000 [325:19:29<84:36:53,  5.47s/it]
 75%|███████▌  | 169347/225000 [325:19:33<81:44:54,  5.29s/it]
 75%|███████▌  | 169348/225000 [325:19:38<79:47:21,  5.16s/it]
 75%|███████▌  | 169349/225000 [325:19:43<79:40:44,  5.15s/it]
 75%|███████▌  | 169350/225000 [325:19:49<79:37:55,  5.15s/it]
                                                              
{'loss': '0.7711', 'grad_norm': '1.375', 'learning_rate': '7.841e-06', 'epoch': '0.7527', 'num_input_tokens_seen': 538845072, 'train_runtime': '1.171e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169350/225000 [325:19:49<79:37:55,  5.15s/it]
 75%|███████▌  | 169351/225000 [325:19:54<78:37:21,  5.09s/it]
 75%|███████▌  | 169352/225000 [325:19:59<78:36:15,  5.09s/it]
 75%|███████▌  | 169353/225000 [325:20:03<77:00:58,  4.98s/it]
 75%|███████▌  | 169354/225000 [325:20:08<76:19:45,  4.94s/it]
 75%|███████▌  | 169355/225000 [325:20:13<74:36:50,  4.83s/it]
 75%|███████▌  | 169356/225000 [325:20:18<75:44:05,  4.90s/it]
 75%|███████▌  | 169357/225000 [325:20:23<74:58:11,  4.85s/it]
 75%|███████▌  | 169358/225000 [325:20:27<74:42:49,  4.83s/it]
 75%|███████▌  | 169359/225000 [325:20:39<107:59:41,  6.99s/it]
 75%|███████▌  | 169360/225000 [325:20:45<103:11:57,  6.68s/it]
 75%|███████▌  | 169361/225000 [325:20:52<103:40:04,  6.71s/it]
 75%|███████▌  | 169362/225000 [325:20:57<93:59:03,  6.08s/it] 
 75%|███████▌  | 169363/225000 [325:21:04<100:34:11,  6.51s/it]
 75%|███████▌  | 169364/225000 [325:21:09<94:01:32,  6.08s/it] 
 75%|███████▌  | 169365/225000 [325:21:14<86:38:15,  5.61s/it]
 75%|███████▌  | 169366/225000 [325:21:19<82:30:21,  5.34s/it]
 75%|███████▌  | 169367/225000 [325:21:23<79:26:15,  5.14s/it]
 75%|███████▌  | 169368/225000 [325:21:28<77:50:46,  5.04s/it]
 75%|███████▌  | 169369/225000 [325:21:33<76:24:33,  4.94s/it]
 75%|███████▌  | 169370/225000 [325:21:38<75:37:11,  4.89s/it]
 75%|███████▌  | 169371/225000 [325:21:42<75:04:46,  4.86s/it]
 75%|███████▌  | 169372/225000 [325:21:47<75:50:40,  4.91s/it]
 75%|███████▌  | 169373/225000 [325:21:52<75:57:03,  4.92s/it]
 75%|███████▌  | 169374/225000 [325:21:59<83:41:27,  5.42s/it]
 75%|███████▌  | 169375/225000 [325:22:10<110:12:06,  7.13s/it]
 75%|███████▌  | 169376/225000 [325:22:13<92:20:06,  5.98s/it] 
 75%|███████▌  | 169377/225000 [325:22:19<89:46:53,  5.81s/it]
 75%|███████▌  | 169378/225000 [325:22:24<88:52:38,  5.75s/it]
 75%|███████▌  | 169379/225000 [325:22:33<101:59:53,  6.60s/it]
 75%|███████▌  | 169380/225000 [325:22:36<87:57:41,  5.69s/it] 
 75%|███████▌  | 169381/225000 [325:22:41<80:42:39,  5.22s/it]
 75%|███████▌  | 169382/225000 [325:22:46<80:47:18,  5.23s/it]
 75%|███████▌  | 169383/225000 [325:22:55<97:08:33,  6.29s/it]
 75%|███████▌  | 169384/225000 [325:22:58<83:48:28,  5.42s/it]
 75%|███████▌  | 169385/225000 [325:23:02<79:18:25,  5.13s/it]
 75%|███████▌  | 169386/225000 [325:23:07<75:45:34,  4.90s/it]
 75%|███████▌  | 169387/225000 [325:23:11<74:37:57,  4.83s/it]
 75%|███████▌  | 169388/225000 [325:23:17<76:38:26,  4.96s/it]
 75%|███████▌  | 169389/225000 [325:23:22<77:24:39,  5.01s/it]
 75%|███████▌  | 169390/225000 [325:23:27<79:35:50,  5.15s/it]
 75%|███████▌  | 169391/225000 [325:23:32<77:35:46,  5.02s/it]
 75%|███████▌  | 169392/225000 [325:23:40<89:20:27,  5.78s/it]
 75%|███████▌  | 169393/225000 [325:23:52<118:33:02,  7.67s/it]
 75%|███████▌  | 169394/225000 [325:23:55<97:38:16,  6.32s/it] 
 75%|███████▌  | 169395/225000 [325:23:59<88:17:37,  5.72s/it]
 75%|███████▌  | 169396/225000 [325:24:03<80:50:01,  5.23s/it]
 75%|███████▌  | 169397/225000 [325:24:08<77:03:30,  4.99s/it]
 75%|███████▌  | 169398/225000 [325:24:17<97:51:58,  6.34s/it]
 75%|███████▌  | 169399/225000 [325:24:21<88:08:24,  5.71s/it]
 75%|███████▌  | 169400/225000 [325:24:29<95:02:09,  6.15s/it]
                                                              
{'loss': '0.7636', 'grad_norm': '1.135', 'learning_rate': '7.828e-06', 'epoch': '0.7529', 'num_input_tokens_seen': 538975352, 'train_runtime': '1.171e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169400/225000 [325:24:29<95:02:09,  6.15s/it]
 75%|███████▌  | 169401/225000 [325:24:32<81:20:02,  5.27s/it]
 75%|███████▌  | 169402/225000 [325:24:35<72:35:21,  4.70s/it]
 75%|███████▌  | 169403/225000 [325:24:43<86:05:54,  5.58s/it]
 75%|███████▌  | 169404/225000 [325:24:50<95:11:04,  6.16s/it]
 75%|███████▌  | 169405/225000 [325:24:56<92:47:22,  6.01s/it]
 75%|███████▌  | 169406/225000 [325:25:01<89:17:42,  5.78s/it]
 75%|███████▌  | 169407/225000 [325:25:07<89:23:58,  5.79s/it]
 75%|███████▌  | 169408/225000 [325:25:12<87:40:43,  5.68s/it]
 75%|███████▌  | 169409/225000 [325:25:18<85:20:31,  5.53s/it]
 75%|███████▌  | 169410/225000 [325:25:23<84:41:26,  5.48s/it]
 75%|███████▌  | 169411/225000 [325:25:30<93:53:59,  6.08s/it]
 75%|███████▌  | 169412/225000 [325:25:37<96:17:29,  6.24s/it]
 75%|███████▌  | 169413/225000 [325:25:41<83:18:24,  5.40s/it]
 75%|███████▌  | 169414/225000 [325:25:48<93:55:48,  6.08s/it]
 75%|███████▌  | 169415/225000 [325:25:52<82:18:14,  5.33s/it]
 75%|███████▌  | 169416/225000 [325:25:59<92:45:18,  6.01s/it]
 75%|███████▌  | 169417/225000 [325:26:04<86:12:55,  5.58s/it]
 75%|███████▌  | 169418/225000 [325:26:09<81:40:14,  5.29s/it]
 75%|███████▌  | 169419/225000 [325:26:13<79:44:37,  5.17s/it]
 75%|███████▌  | 169420/225000 [325:26:18<75:55:14,  4.92s/it]
 75%|███████▌  | 169421/225000 [325:26:22<74:05:42,  4.80s/it]
 75%|███████▌  | 169422/225000 [325:26:34<104:07:48,  6.74s/it]
 75%|███████▌  | 169423/225000 [325:26:37<89:41:13,  5.81s/it] 
 75%|███████▌  | 169424/225000 [325:26:41<79:41:15,  5.16s/it]
 75%|███████▌  | 169425/225000 [325:26:49<93:29:36,  6.06s/it]
 75%|███████▌  | 169426/225000 [325:26:53<82:57:58,  5.37s/it]
 75%|███████▌  | 169427/225000 [325:26:57<76:29:40,  4.96s/it]
 75%|███████▌  | 169428/225000 [325:27:04<86:43:33,  5.62s/it]
 75%|███████▌  | 169429/225000 [325:27:13<103:14:07,  6.69s/it]
 75%|███████▌  | 169430/225000 [325:27:17<89:58:14,  5.83s/it] 
 75%|███████▌  | 169431/225000 [325:27:20<79:14:08,  5.13s/it]
 75%|███████▌  | 169432/225000 [325:27:30<99:16:43,  6.43s/it]
 75%|███████▌  | 169433/225000 [325:27:34<86:57:56,  5.63s/it]
 75%|███████▌  | 169434/225000 [325:27:37<77:44:00,  5.04s/it]
 75%|███████▌  | 169435/225000 [325:27:45<92:01:46,  5.96s/it]
 75%|███████▌  | 169436/225000 [325:27:51<89:35:49,  5.81s/it]
 75%|███████▌  | 169437/225000 [325:27:55<79:52:35,  5.18s/it]
 75%|███████▌  | 169438/225000 [325:28:02<88:48:01,  5.75s/it]
 75%|███████▌  | 169439/225000 [325:28:07<86:29:22,  5.60s/it]
 75%|███████▌  | 169440/225000 [325:28:10<75:19:48,  4.88s/it]
 75%|███████▌  | 169441/225000 [325:28:19<95:21:58,  6.18s/it]
 75%|███████▌  | 169442/225000 [325:28:23<84:27:33,  5.47s/it]
 75%|███████▌  | 169443/225000 [325:28:27<76:39:08,  4.97s/it]
 75%|███████▌  | 169444/225000 [325:28:35<89:48:05,  5.82s/it]
 75%|███████▌  | 169445/225000 [325:28:41<90:05:28,  5.84s/it]
 75%|███████▌  | 169446/225000 [325:28:47<93:11:16,  6.04s/it]
 75%|███████▌  | 169447/225000 [325:28:51<81:23:54,  5.27s/it]
 75%|███████▌  | 169448/225000 [325:28:56<82:55:38,  5.37s/it]
 75%|███████▌  | 169449/225000 [325:29:03<89:44:30,  5.82s/it]
 75%|███████▌  | 169450/225000 [325:29:08<84:33:53,  5.48s/it]
                                                              
{'loss': '0.7631', 'grad_norm': '1.166', 'learning_rate': '7.814e-06', 'epoch': '0.7531', 'num_input_tokens_seen': 539106552, 'train_runtime': '1.172e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169450/225000 [325:29:08<84:33:53,  5.48s/it]
 75%|███████▌  | 169451/225000 [325:29:13<81:26:18,  5.28s/it]
 75%|███████▌  | 169452/225000 [325:29:17<78:39:02,  5.10s/it]
 75%|███████▌  | 169453/225000 [325:29:22<74:36:12,  4.84s/it]
 75%|███████▌  | 169454/225000 [325:29:26<74:36:40,  4.84s/it]
 75%|███████▌  | 169455/225000 [325:29:31<72:52:42,  4.72s/it]
 75%|███████▌  | 169456/225000 [325:29:36<73:19:34,  4.75s/it]
 75%|███████▌  | 169457/225000 [325:29:40<73:33:34,  4.77s/it]
 75%|███████▌  | 169458/225000 [325:29:46<77:05:34,  5.00s/it]
 75%|███████▌  | 169459/225000 [325:29:57<106:16:23,  6.89s/it]
 75%|███████▌  | 169460/225000 [325:30:02<98:06:06,  6.36s/it] 
 75%|███████▌  | 169461/225000 [325:30:06<83:35:23,  5.42s/it]
 75%|███████▌  | 169462/225000 [325:30:14<99:15:24,  6.43s/it]
 75%|███████▌  | 169463/225000 [325:30:18<85:27:57,  5.54s/it]
 75%|███████▌  | 169464/225000 [325:30:25<93:04:23,  6.03s/it]
 75%|███████▌  | 169465/225000 [325:30:29<82:17:27,  5.33s/it]
 75%|███████▌  | 169466/225000 [325:30:34<80:58:34,  5.25s/it]
 75%|███████▌  | 169467/225000 [325:30:42<96:00:19,  6.22s/it]
 75%|███████▌  | 169468/225000 [325:30:46<85:04:40,  5.52s/it]
 75%|███████▌  | 169469/225000 [325:30:51<79:44:27,  5.17s/it]
 75%|███████▌  | 169470/225000 [325:30:55<75:49:40,  4.92s/it]
 75%|███████▌  | 169471/225000 [325:30:59<73:19:49,  4.75s/it]
 75%|███████▌  | 169472/225000 [325:31:04<72:44:36,  4.72s/it]
 75%|███████▌  | 169473/225000 [325:31:08<71:41:11,  4.65s/it]
 75%|███████▌  | 169474/225000 [325:31:14<75:31:20,  4.90s/it]
 75%|███████▌  | 169475/225000 [325:31:26<107:57:33,  7.00s/it]
 75%|███████▌  | 169476/225000 [325:31:29<90:06:33,  5.84s/it] 
 75%|███████▌  | 169477/225000 [325:31:35<92:43:30,  6.01s/it]
 75%|███████▌  | 169478/225000 [325:31:39<80:16:11,  5.20s/it]
 75%|███████▌  | 169479/225000 [325:31:47<93:07:11,  6.04s/it]
 75%|███████▌  | 169480/225000 [325:31:50<83:08:56,  5.39s/it]
 75%|███████▌  | 169481/225000 [325:31:57<88:33:15,  5.74s/it]
 75%|███████▌  | 169482/225000 [325:32:03<91:08:26,  5.91s/it]
 75%|███████▌  | 169483/225000 [325:32:07<79:01:38,  5.12s/it]
 75%|███████▌  | 169484/225000 [325:32:15<93:32:14,  6.07s/it]
 75%|███████▌  | 169485/225000 [325:32:19<82:17:05,  5.34s/it]
 75%|███████▌  | 169486/225000 [325:32:22<74:44:20,  4.85s/it]
 75%|███████▌  | 169487/225000 [325:32:30<89:39:28,  5.81s/it]
 75%|███████▌  | 169488/225000 [325:32:36<88:18:35,  5.73s/it]
 75%|███████▌  | 169489/225000 [325:32:39<77:25:09,  5.02s/it]
 75%|███████▌  | 169490/225000 [325:32:47<88:51:10,  5.76s/it]
 75%|███████▌  | 169491/225000 [325:32:52<88:28:32,  5.74s/it]
 75%|███████▌  | 169492/225000 [325:32:59<93:44:39,  6.08s/it]
 75%|███████▌  | 169493/225000 [325:33:04<85:39:12,  5.56s/it]
 75%|███████▌  | 169494/225000 [325:33:08<78:10:53,  5.07s/it]
 75%|███████▌  | 169495/225000 [325:33:12<73:15:18,  4.75s/it]
 75%|███████▌  | 169496/225000 [325:33:16<69:53:10,  4.53s/it]
 75%|███████▌  | 169497/225000 [325:33:25<94:21:02,  6.12s/it]
 75%|███████▌  | 169498/225000 [325:33:30<85:46:19,  5.56s/it]
 75%|███████▌  | 169499/225000 [325:33:34<79:18:56,  5.14s/it]
 75%|███████▌  | 169500/225000 [325:33:38<76:32:39,  4.97s/it]
                                                              
{'loss': '0.7716', 'grad_norm': '1.164', 'learning_rate': '7.801e-06', 'epoch': '0.7533', 'num_input_tokens_seen': 539235688, 'train_runtime': '1.172e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169500/225000 [325:33:38<76:32:39,  4.97s/it]
 75%|███████▌  | 169501/225000 [325:33:43<73:52:51,  4.79s/it]
 75%|███████▌  | 169502/225000 [325:33:46<68:55:41,  4.47s/it]
 75%|███████▌  | 169503/225000 [325:33:51<68:10:53,  4.42s/it]
 75%|███████▌  | 169504/225000 [325:33:56<71:03:59,  4.61s/it]
 75%|███████▌  | 169505/225000 [325:34:01<75:34:04,  4.90s/it]
 75%|███████▌  | 169506/225000 [325:34:07<78:43:09,  5.11s/it]
 75%|███████▌  | 169507/225000 [325:34:12<80:18:54,  5.21s/it]
 75%|███████▌  | 169508/225000 [325:34:18<80:31:45,  5.22s/it]
 75%|███████▌  | 169509/225000 [325:34:23<80:13:58,  5.21s/it]
 75%|███████▌  | 169510/225000 [325:34:28<78:48:47,  5.11s/it]
 75%|███████▌  | 169511/225000 [325:34:33<78:03:05,  5.06s/it]
 75%|███████▌  | 169512/225000 [325:34:38<77:33:33,  5.03s/it]
 75%|███████▌  | 169513/225000 [325:34:43<80:05:38,  5.20s/it]
 75%|███████▌  | 169514/225000 [325:34:56<113:52:33,  7.39s/it]
 75%|███████▌  | 169515/225000 [325:35:00<100:13:07,  6.50s/it]
 75%|███████▌  | 169516/225000 [325:35:04<89:44:15,  5.82s/it] 
 75%|███████▌  | 169517/225000 [325:35:09<81:48:29,  5.31s/it]
 75%|███████▌  | 169518/225000 [325:35:17<95:10:56,  6.18s/it]
 75%|███████▌  | 169519/225000 [325:35:25<105:31:02,  6.85s/it]
 75%|███████▌  | 169520/225000 [325:35:29<91:43:20,  5.95s/it] 
 75%|███████▌  | 169521/225000 [325:35:33<81:22:14,  5.28s/it]
 75%|███████▌  | 169522/225000 [325:35:40<89:14:37,  5.79s/it]
 75%|███████▌  | 169523/225000 [325:35:45<88:41:36,  5.76s/it]
 75%|███████▌  | 169524/225000 [325:35:49<80:15:39,  5.21s/it]
 75%|███████▌  | 169525/225000 [325:35:53<75:19:48,  4.89s/it]
 75%|███████▌  | 169526/225000 [325:35:58<73:43:26,  4.78s/it]
 75%|███████▌  | 169527/225000 [325:36:10<105:02:22,  6.82s/it]
 75%|███████▌  | 169528/225000 [325:36:13<88:40:40,  5.75s/it] 
 75%|███████▌  | 169529/225000 [325:36:20<93:59:46,  6.10s/it]
 75%|███████▌  | 169530/225000 [325:36:23<83:12:49,  5.40s/it]
 75%|███████▌  | 169531/225000 [325:36:28<78:13:23,  5.08s/it]
 75%|███████▌  | 169532/225000 [325:36:32<73:42:10,  4.78s/it]
 75%|███████▌  | 169533/225000 [325:36:36<70:11:31,  4.56s/it]
 75%|███████▌  | 169534/225000 [325:36:47<98:56:43,  6.42s/it]
 75%|███████▌  | 169535/225000 [325:36:53<96:34:39,  6.27s/it]
 75%|███████▌  | 169536/225000 [325:36:56<83:03:24,  5.39s/it]
 75%|███████▌  | 169537/225000 [325:37:00<78:09:29,  5.07s/it]
 75%|███████▌  | 169538/225000 [325:37:08<91:46:27,  5.96s/it]
 75%|███████▌  | 169539/225000 [325:37:12<80:59:55,  5.26s/it]
 75%|███████▌  | 169540/225000 [325:37:16<75:32:15,  4.90s/it]
 75%|███████▌  | 169541/225000 [325:37:21<73:55:12,  4.80s/it]
 75%|███████▌  | 169542/225000 [325:37:26<75:33:00,  4.90s/it]
 75%|███████▌  | 169543/225000 [325:37:31<76:33:54,  4.97s/it]
 75%|███████▌  | 169544/225000 [325:37:36<75:11:54,  4.88s/it]
 75%|███████▌  | 169545/225000 [325:37:41<76:48:17,  4.99s/it]
 75%|███████▌  | 169546/225000 [325:37:45<74:46:44,  4.85s/it]
 75%|███████▌  | 169547/225000 [325:37:50<75:43:17,  4.92s/it]
 75%|███████▌  | 169548/225000 [325:37:56<76:44:35,  4.98s/it]
 75%|███████▌  | 169549/225000 [325:38:00<76:14:52,  4.95s/it]
 75%|███████▌  | 169550/225000 [325:38:06<77:31:14,  5.03s/it]
                                                              
{'loss': '0.7556', 'grad_norm': '1.049', 'learning_rate': '7.788e-06', 'epoch': '0.7536', 'num_input_tokens_seen': 539364032, 'train_runtime': '1.172e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169550/225000 [325:38:06<77:31:14,  5.03s/it]
 75%|███████▌  | 169551/225000 [325:38:11<78:12:26,  5.08s/it]
 75%|███████▌  | 169552/225000 [325:38:16<77:46:11,  5.05s/it]
 75%|███████▌  | 169553/225000 [325:38:27<104:53:40,  6.81s/it]
 75%|███████▌  | 169554/225000 [325:38:33<101:31:00,  6.59s/it]
 75%|███████▌  | 169555/225000 [325:38:41<108:09:03,  7.02s/it]
 75%|███████▌  | 169556/225000 [325:38:44<91:36:45,  5.95s/it] 
 75%|███████▌  | 169557/225000 [325:38:48<82:19:42,  5.35s/it]
 75%|███████▌  | 169558/225000 [325:38:52<76:49:19,  4.99s/it]
 75%|███████▌  | 169559/225000 [325:38:59<84:00:51,  5.46s/it]
 75%|███████▌  | 169560/225000 [325:39:08<101:34:48,  6.60s/it]
 75%|███████▌  | 169561/225000 [325:39:12<88:39:37,  5.76s/it] 
 75%|███████▌  | 169562/225000 [325:39:17<86:01:06,  5.59s/it]
 75%|███████▌  | 169563/225000 [325:39:23<88:40:56,  5.76s/it]
 75%|███████▌  | 169564/225000 [325:39:28<86:08:54,  5.59s/it]
 75%|███████▌  | 169565/225000 [325:39:33<82:37:41,  5.37s/it]
 75%|███████▌  | 169566/225000 [325:39:38<81:15:00,  5.28s/it]
 75%|███████▌  | 169567/225000 [325:39:43<80:19:18,  5.22s/it]
 75%|███████▌  | 169568/225000 [325:39:51<92:58:38,  6.04s/it]
 75%|███████▌  | 169569/225000 [325:39:55<82:25:03,  5.35s/it]
 75%|███████▌  | 169570/225000 [325:39:59<76:27:23,  4.97s/it]
 75%|███████▌  | 169571/225000 [325:40:09<96:17:53,  6.25s/it]
 75%|███████▌  | 169572/225000 [325:40:12<82:43:53,  5.37s/it]
 75%|███████▌  | 169573/225000 [325:40:20<96:11:50,  6.25s/it]
 75%|███████▌  | 169574/225000 [325:40:24<87:35:27,  5.69s/it]
 75%|███████▌  | 169575/225000 [325:40:29<79:56:41,  5.19s/it]
 75%|███████▌  | 169576/225000 [325:40:33<76:02:26,  4.94s/it]
 75%|███████▌  | 169577/225000 [325:40:37<72:52:53,  4.73s/it]
 75%|███████▌  | 169578/225000 [325:40:48<102:54:49,  6.68s/it]
 75%|███████▌  | 169579/225000 [325:40:52<88:34:35,  5.75s/it] 
 75%|███████▌  | 169580/225000 [325:40:56<78:32:00,  5.10s/it]
 75%|███████▌  | 169581/225000 [325:41:00<73:55:15,  4.80s/it]
 75%|███████▌  | 169582/225000 [325:41:04<73:01:30,  4.74s/it]
 75%|███████▌  | 169583/225000 [325:41:09<72:53:17,  4.73s/it]
 75%|███████▌  | 169584/225000 [325:41:14<72:58:41,  4.74s/it]
 75%|███████▌  | 169585/225000 [325:41:21<82:34:53,  5.36s/it]
 75%|███████▌  | 169586/225000 [325:41:31<105:12:54,  6.84s/it]
 75%|███████▌  | 169587/225000 [325:41:34<88:53:47,  5.78s/it] 
 75%|███████▌  | 169588/225000 [325:41:39<83:33:54,  5.43s/it]
 75%|███████▌  | 169589/225000 [325:41:47<97:46:40,  6.35s/it]
 75%|███████▌  | 169590/225000 [325:41:50<82:58:43,  5.39s/it]
 75%|███████▌  | 169591/225000 [325:41:58<93:15:55,  6.06s/it]
 75%|███████▌  | 169592/225000 [325:42:01<80:36:48,  5.24s/it]
 75%|███████▌  | 169593/225000 [325:42:09<92:59:46,  6.04s/it]
 75%|███████▌  | 169594/225000 [325:42:13<80:41:12,  5.24s/it]
 75%|███████▌  | 169595/225000 [325:42:20<90:05:06,  5.85s/it]
 75%|███████▌  | 169596/225000 [325:42:25<87:49:00,  5.71s/it]
 75%|███████▌  | 169597/225000 [325:42:30<83:58:08,  5.46s/it]
 75%|███████▌  | 169598/225000 [325:42:37<90:22:47,  5.87s/it]
 75%|███████▌  | 169599/225000 [325:42:43<90:00:01,  5.85s/it]
 75%|███████▌  | 169600/225000 [325:42:48<87:44:50,  5.70s/it]
                                                              
{'loss': '0.7595', 'grad_norm': '1.29', 'learning_rate': '7.775e-06', 'epoch': '0.7538', 'num_input_tokens_seen': 539494640, 'train_runtime': '1.173e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169600/225000 [325:42:48<87:44:50,  5.70s/it]
 75%|███████▌  | 169601/225000 [325:42:53<81:51:51,  5.32s/it]
 75%|███████▌  | 169602/225000 [325:42:57<76:27:23,  4.97s/it]
 75%|███████▌  | 169603/225000 [325:43:01<73:22:38,  4.77s/it]
 75%|███████▌  | 169604/225000 [325:43:05<69:36:18,  4.52s/it]
 75%|███████▌  | 169605/225000 [325:43:12<82:54:20,  5.39s/it]
 75%|███████▌  | 169606/225000 [325:43:21<99:01:29,  6.44s/it]
 75%|███████▌  | 169607/225000 [325:43:25<84:50:23,  5.51s/it]
 75%|███████▌  | 169608/225000 [325:43:33<96:36:09,  6.28s/it]
 75%|███████▌  | 169609/225000 [325:43:36<82:49:22,  5.38s/it]
 75%|███████▌  | 169610/225000 [325:43:42<85:30:56,  5.56s/it]
 75%|███████▌  | 169611/225000 [325:43:47<82:25:39,  5.36s/it]
 75%|███████▌  | 169612/225000 [325:43:50<74:28:06,  4.84s/it]
 75%|███████▌  | 169613/225000 [325:43:56<77:21:15,  5.03s/it]
 75%|███████▌  | 169614/225000 [325:44:01<77:45:21,  5.05s/it]
 75%|███████▌  | 169615/225000 [325:44:06<79:29:14,  5.17s/it]
 75%|███████▌  | 169616/225000 [325:44:12<80:57:05,  5.26s/it]
 75%|███████▌  | 169617/225000 [325:44:18<82:55:22,  5.39s/it]
 75%|███████▌  | 169618/225000 [325:44:23<82:45:40,  5.38s/it]
 75%|███████▌  | 169619/225000 [325:44:28<83:00:37,  5.40s/it]
 75%|███████▌  | 169620/225000 [325:44:34<82:28:14,  5.36s/it]
 75%|███████▌  | 169621/225000 [325:44:39<83:00:14,  5.40s/it]
 75%|███████▌  | 169622/225000 [325:44:45<83:35:42,  5.43s/it]
 75%|███████▌  | 169623/225000 [325:44:50<83:17:45,  5.41s/it]
 75%|███████▌  | 169624/225000 [325:44:55<82:32:57,  5.37s/it]
 75%|███████▌  | 169625/225000 [325:45:01<82:36:57,  5.37s/it]
 75%|███████▌  | 169626/225000 [325:45:06<82:56:40,  5.39s/it]
 75%|███████▌  | 169627/225000 [325:45:11<82:29:30,  5.36s/it]
 75%|███████▌  | 169628/225000 [325:45:17<81:07:18,  5.27s/it]
 75%|███████▌  | 169629/225000 [325:45:22<81:09:29,  5.28s/it]
 75%|███████▌  | 169630/225000 [325:45:27<81:01:27,  5.27s/it]
 75%|███████▌  | 169631/225000 [325:45:32<81:18:39,  5.29s/it]
 75%|███████▌  | 169632/225000 [325:45:38<80:49:57,  5.26s/it]
 75%|███████▌  | 169633/225000 [325:45:43<80:03:07,  5.21s/it]
 75%|███████▌  | 169634/225000 [325:45:48<79:27:50,  5.17s/it]
 75%|███████▌  | 169635/225000 [325:45:53<79:44:03,  5.18s/it]
 75%|███████▌  | 169636/225000 [325:45:58<79:34:35,  5.17s/it]
 75%|███████▌  | 169637/225000 [325:46:03<80:10:59,  5.21s/it]
 75%|███████▌  | 169638/225000 [325:46:09<80:53:23,  5.26s/it]
 75%|███████▌  | 169639/225000 [325:46:14<79:27:30,  5.17s/it]
 75%|███████▌  | 169640/225000 [325:46:19<81:44:01,  5.32s/it]
 75%|███████▌  | 169641/225000 [325:46:25<81:10:42,  5.28s/it]
 75%|███████▌  | 169642/225000 [325:46:30<83:08:12,  5.41s/it]
 75%|███████▌  | 169643/225000 [325:46:42<112:27:22,  7.31s/it]
 75%|███████▌  | 169644/225000 [325:46:46<97:54:14,  6.37s/it] 
 75%|███████▌  | 169645/225000 [325:46:50<88:22:27,  5.75s/it]
 75%|███████▌  | 169646/225000 [325:46:55<83:19:30,  5.42s/it]
 75%|███████▌  | 169647/225000 [325:47:00<78:39:40,  5.12s/it]
 75%|███████▌  | 169648/225000 [325:47:11<105:31:43,  6.86s/it]
 75%|███████▌  | 169649/225000 [325:47:16<98:59:12,  6.44s/it] 
 75%|███████▌  | 169650/225000 [325:47:20<86:26:51,  5.62s/it]
                                                              
{'loss': '0.7702', 'grad_norm': '1.101', 'learning_rate': '7.761e-06', 'epoch': '0.754', 'num_input_tokens_seen': 539624704, 'train_runtime': '1.173e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169650/225000 [325:47:20<86:26:51,  5.62s/it]
 75%|███████▌  | 169651/225000 [325:47:29<103:20:07,  6.72s/it]
 75%|███████▌  | 169652/225000 [325:47:32<87:51:31,  5.71s/it] 
 75%|███████▌  | 169653/225000 [325:47:41<100:46:19,  6.55s/it]
 75%|███████▌  | 169654/225000 [325:47:44<84:22:32,  5.49s/it] 
 75%|███████▌  | 169655/225000 [325:47:51<90:14:23,  5.87s/it]
 75%|███████▌  | 169656/225000 [325:47:54<77:53:27,  5.07s/it]
 75%|███████▌  | 169657/225000 [325:47:58<73:57:35,  4.81s/it]
 75%|███████▌  | 169658/225000 [325:48:02<69:49:39,  4.54s/it]
 75%|███████▌  | 169659/225000 [325:48:07<72:48:01,  4.74s/it]
 75%|███████▌  | 169660/225000 [325:48:17<97:15:37,  6.33s/it]
 75%|███████▌  | 169661/225000 [325:48:20<83:25:59,  5.43s/it]
 75%|███████▌  | 169662/225000 [325:48:27<89:24:44,  5.82s/it]
 75%|███████▌  | 169663/225000 [325:48:32<83:30:15,  5.43s/it]
 75%|███████▌  | 169664/225000 [325:48:39<90:01:02,  5.86s/it]
 75%|███████▌  | 169665/225000 [325:48:45<94:34:03,  6.15s/it]
 75%|███████▌  | 169666/225000 [325:48:49<82:02:14,  5.34s/it]
 75%|███████▌  | 169667/225000 [325:48:53<75:43:12,  4.93s/it]
 75%|███████▌  | 169668/225000 [325:49:00<85:55:41,  5.59s/it]
 75%|███████▌  | 169669/225000 [325:49:04<77:30:36,  5.04s/it]
 75%|███████▌  | 169670/225000 [325:49:13<97:26:29,  6.34s/it]
 75%|███████▌  | 169671/225000 [325:49:17<86:49:52,  5.65s/it]
 75%|███████▌  | 169672/225000 [325:49:21<80:19:19,  5.23s/it]
 75%|███████▌  | 169673/225000 [325:49:26<75:33:43,  4.92s/it]
 75%|███████▌  | 169674/225000 [325:49:30<71:16:28,  4.64s/it]
 75%|███████▌  | 169675/225000 [325:49:36<81:28:35,  5.30s/it]
 75%|███████▌  | 169676/225000 [325:49:43<88:40:50,  5.77s/it]
 75%|███████▌  | 169677/225000 [325:49:48<82:16:20,  5.35s/it]
 75%|███████▌  | 169678/225000 [325:49:53<81:39:45,  5.31s/it]
 75%|███████▌  | 169679/225000 [325:49:58<81:14:04,  5.29s/it]
 75%|███████▌  | 169680/225000 [325:50:03<79:38:22,  5.18s/it]
 75%|███████▌  | 169681/225000 [325:50:09<80:59:12,  5.27s/it]
 75%|███████▌  | 169682/225000 [325:50:14<80:38:03,  5.25s/it]
 75%|███████▌  | 169683/225000 [325:50:19<78:44:46,  5.12s/it]
 75%|███████▌  | 169684/225000 [325:50:24<79:24:04,  5.17s/it]
 75%|███████▌  | 169685/225000 [325:50:29<80:05:49,  5.21s/it]
 75%|███████▌  | 169686/225000 [325:50:34<79:14:59,  5.16s/it]
 75%|███████▌  | 169687/225000 [325:50:39<78:08:50,  5.09s/it]
 75%|███████▌  | 169688/225000 [325:50:45<80:39:38,  5.25s/it]
 75%|███████▌  | 169689/225000 [325:50:50<79:13:43,  5.16s/it]
 75%|███████▌  | 169690/225000 [325:50:55<79:40:53,  5.19s/it]
 75%|███████▌  | 169691/225000 [325:51:00<80:03:02,  5.21s/it]
 75%|███████▌  | 169692/225000 [325:51:05<79:58:23,  5.21s/it]
 75%|███████▌  | 169693/225000 [325:51:10<79:09:53,  5.15s/it]
 75%|███████▌  | 169694/225000 [325:51:15<76:19:36,  4.97s/it]
 75%|███████▌  | 169695/225000 [325:51:21<79:14:55,  5.16s/it]
 75%|███████▌  | 169696/225000 [325:51:27<83:39:02,  5.45s/it]
 75%|███████▌  | 169697/225000 [325:51:40<118:23:22,  7.71s/it]
 75%|███████▌  | 169698/225000 [325:51:44<103:12:05,  6.72s/it]
 75%|███████▌  | 169699/225000 [325:51:48<89:03:32,  5.80s/it] 
 75%|███████▌  | 169700/225000 [325:51:52<82:55:24,  5.40s/it]
                                                              
{'loss': '0.7687', 'grad_norm': '1.15', 'learning_rate': '7.748e-06', 'epoch': '0.7542', 'num_input_tokens_seen': 539753944, 'train_runtime': '1.173e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169700/225000 [325:51:52<82:55:24,  5.40s/it]
 75%|███████▌  | 169701/225000 [325:52:01<99:05:55,  6.45s/it]
 75%|███████▌  | 169702/225000 [325:52:06<93:38:51,  6.10s/it]
 75%|███████▌  | 169703/225000 [325:52:14<101:07:33,  6.58s/it]
 75%|███████▌  | 169704/225000 [325:52:17<86:18:47,  5.62s/it] 
 75%|███████▌  | 169705/225000 [325:52:24<91:38:27,  5.97s/it]
 75%|███████▌  | 169706/225000 [325:52:28<82:21:10,  5.36s/it]
 75%|███████▌  | 169707/225000 [325:52:32<75:38:38,  4.93s/it]
 75%|███████▌  | 169708/225000 [325:52:36<72:10:50,  4.70s/it]
 75%|███████▌  | 169709/225000 [325:52:46<93:29:09,  6.09s/it]
 75%|███████▌  | 169710/225000 [325:52:52<93:14:45,  6.07s/it]
 75%|███████▌  | 169711/225000 [325:52:55<82:16:04,  5.36s/it]
 75%|███████▌  | 169712/225000 [325:53:00<78:21:04,  5.10s/it]
 75%|███████▌  | 169713/225000 [325:53:07<89:29:51,  5.83s/it]
 75%|███████▌  | 169714/225000 [325:53:11<79:06:07,  5.15s/it]
 75%|███████▌  | 169715/225000 [325:53:18<90:11:27,  5.87s/it]
 75%|███████▌  | 169716/225000 [325:53:24<90:06:12,  5.87s/it]
 75%|███████▌  | 169717/225000 [325:53:28<80:33:29,  5.25s/it]
 75%|███████▌  | 169718/225000 [325:53:32<73:59:00,  4.82s/it]
 75%|███████▌  | 169719/225000 [325:53:42<96:35:53,  6.29s/it]
 75%|███████▌  | 169720/225000 [325:53:45<83:17:48,  5.42s/it]
 75%|███████▌  | 169721/225000 [325:53:52<91:41:08,  5.97s/it]
 75%|███████▌  | 169722/225000 [325:53:55<78:57:04,  5.14s/it]
 75%|███████▌  | 169723/225000 [325:54:03<91:57:52,  5.99s/it]
 75%|███████▌  | 169724/225000 [325:54:07<79:34:54,  5.18s/it]
 75%|███████▌  | 169725/225000 [325:54:14<90:37:35,  5.90s/it]
 75%|███████▌  | 169726/225000 [325:54:18<80:42:31,  5.26s/it]
 75%|███████▌  | 169727/225000 [325:54:22<75:42:24,  4.93s/it]
 75%|███████▌  | 169728/225000 [325:54:27<72:51:28,  4.75s/it]
 75%|███████▌  | 169729/225000 [325:54:34<84:47:00,  5.52s/it]
 75%|███████▌  | 169730/225000 [325:54:41<93:11:24,  6.07s/it]
 75%|███████▌  | 169731/225000 [325:54:45<80:39:38,  5.25s/it]
 75%|███████▌  | 169732/225000 [325:54:50<80:28:24,  5.24s/it]
 75%|███████▌  | 169733/225000 [325:54:57<90:32:39,  5.90s/it]
 75%|███████▌  | 169734/225000 [325:55:02<83:50:50,  5.46s/it]
 75%|███████▌  | 169735/225000 [325:55:06<79:44:05,  5.19s/it]
 75%|███████▌  | 169736/225000 [325:55:11<79:02:15,  5.15s/it]
 75%|███████▌  | 169737/225000 [325:55:16<75:28:01,  4.92s/it]
 75%|███████▌  | 169738/225000 [325:55:20<73:13:18,  4.77s/it]
 75%|███████▌  | 169739/225000 [325:55:25<72:57:12,  4.75s/it]
 75%|███████▌  | 169740/225000 [325:55:30<73:08:12,  4.76s/it]
 75%|███████▌  | 169741/225000 [325:55:34<73:13:25,  4.77s/it]
 75%|███████▌  | 169742/225000 [325:55:41<81:36:54,  5.32s/it]
 75%|███████▌  | 169743/225000 [325:55:52<105:42:23,  6.89s/it]
 75%|███████▌  | 169744/225000 [325:55:55<89:05:15,  5.80s/it] 
 75%|███████▌  | 169745/225000 [325:55:59<79:56:05,  5.21s/it]
 75%|███████▌  | 169746/225000 [325:56:10<107:19:15,  6.99s/it]
 75%|███████▌  | 169747/225000 [325:56:13<91:23:41,  5.95s/it] 
 75%|███████▌  | 169748/225000 [325:56:17<80:19:20,  5.23s/it]
 75%|███████▌  | 169749/225000 [325:56:24<87:02:29,  5.67s/it]
 75%|███████▌  | 169750/225000 [325:56:28<79:21:47,  5.17s/it]
                                                              
{'loss': '0.7625', 'grad_norm': '1.177', 'learning_rate': '7.735e-06', 'epoch': '0.7544', 'num_input_tokens_seen': 539883768, 'train_runtime': '1.173e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169750/225000 [325:56:28<79:21:47,  5.17s/it]
 75%|███████▌  | 169751/225000 [325:56:34<86:31:13,  5.64s/it]
 75%|███████▌  | 169752/225000 [325:56:39<83:57:43,  5.47s/it]
 75%|███████▌  | 169753/225000 [325:56:44<79:26:41,  5.18s/it]
 75%|███████▌  | 169754/225000 [325:56:50<84:51:37,  5.53s/it]
 75%|███████▌  | 169755/225000 [325:56:54<76:29:54,  4.98s/it]
 75%|███████▌  | 169756/225000 [325:56:59<75:49:53,  4.94s/it]
 75%|███████▌  | 169757/225000 [325:57:09<99:40:19,  6.50s/it]
 75%|███████▌  | 169758/225000 [325:57:12<84:15:54,  5.49s/it]
 75%|███████▌  | 169759/225000 [325:57:16<78:16:38,  5.10s/it]
 75%|███████▌  | 169760/225000 [325:57:21<75:17:11,  4.91s/it]
 75%|███████▌  | 169761/225000 [325:57:28<86:14:52,  5.62s/it]
 75%|███████▌  | 169762/225000 [325:57:34<90:22:05,  5.89s/it]
 75%|███████▌  | 169763/225000 [325:57:38<79:45:43,  5.20s/it]
 75%|███████▌  | 169764/225000 [325:57:43<77:24:17,  5.04s/it]
 75%|███████▌  | 169765/225000 [325:57:48<80:32:31,  5.25s/it]
 75%|███████▌  | 169766/225000 [325:57:54<81:23:57,  5.31s/it]
 75%|███████▌  | 169767/225000 [325:57:59<80:22:03,  5.24s/it]
 75%|███████▌  | 169768/225000 [325:58:04<80:34:37,  5.25s/it]
 75%|███████▌  | 169769/225000 [325:58:10<80:27:36,  5.24s/it]
 75%|███████▌  | 169770/225000 [325:58:15<80:31:38,  5.25s/it]
 75%|███████▌  | 169771/225000 [325:58:20<82:01:58,  5.35s/it]
 75%|███████▌  | 169772/225000 [325:58:26<81:32:49,  5.32s/it]
 75%|███████▌  | 169773/225000 [325:58:31<80:42:21,  5.26s/it]
 75%|███████▌  | 169774/225000 [325:58:36<80:53:21,  5.27s/it]
 75%|███████▌  | 169775/225000 [325:58:41<80:39:21,  5.26s/it]
 75%|███████▌  | 169776/225000 [325:58:46<78:00:32,  5.09s/it]
 75%|███████▌  | 169777/225000 [325:58:51<79:45:59,  5.20s/it]
 75%|███████▌  | 169778/225000 [325:58:57<80:25:55,  5.24s/it]
 75%|███████▌  | 169779/225000 [325:59:02<82:00:26,  5.35s/it]
 75%|███████▌  | 169780/225000 [325:59:11<97:40:40,  6.37s/it]
 75%|███████▌  | 169781/225000 [325:59:19<105:08:06,  6.85s/it]
 75%|███████▌  | 169782/225000 [325:59:26<104:47:55,  6.83s/it]
 75%|███████▌  | 169783/225000 [325:59:30<90:28:20,  5.90s/it] 
 75%|███████▌  | 169784/225000 [325:59:33<79:39:55,  5.19s/it]
 75%|███████▌  | 169785/225000 [325:59:42<98:19:17,  6.41s/it]
 75%|███████▌  | 169786/225000 [325:59:47<91:10:24,  5.94s/it]
 75%|███████▌  | 169787/225000 [325:59:52<85:55:52,  5.60s/it]
 75%|███████▌  | 169788/225000 [325:59:57<84:01:09,  5.48s/it]
 75%|███████▌  | 169789/225000 [326:00:02<80:49:35,  5.27s/it]
 75%|███████▌  | 169790/225000 [326:00:07<79:59:15,  5.22s/it]
 75%|███████▌  | 169791/225000 [326:00:12<79:27:49,  5.18s/it]
 75%|███████▌  | 169792/225000 [326:00:17<78:40:02,  5.13s/it]
 75%|███████▌  | 169793/225000 [326:00:22<77:29:58,  5.05s/it]
 75%|███████▌  | 169794/225000 [326:00:27<78:19:51,  5.11s/it]
 75%|███████▌  | 169795/225000 [326:00:32<77:52:41,  5.08s/it]
 75%|███████▌  | 169796/225000 [326:00:37<78:17:29,  5.11s/it]
 75%|███████▌  | 169797/225000 [326:00:43<78:43:23,  5.13s/it]
 75%|███████▌  | 169798/225000 [326:00:48<78:20:47,  5.11s/it]
 75%|███████▌  | 169799/225000 [326:00:53<78:31:14,  5.12s/it]
 75%|███████▌  | 169800/225000 [326:00:58<79:55:39,  5.21s/it]
                                                              
{'loss': '0.7704', 'grad_norm': '1.019', 'learning_rate': '7.722e-06', 'epoch': '0.7547', 'num_input_tokens_seen': 540013240, 'train_runtime': '1.174e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169800/225000 [326:00:58<79:55:39,  5.21s/it]
 75%|███████▌  | 169801/225000 [326:01:04<79:50:14,  5.21s/it]
 75%|███████▌  | 169802/225000 [326:01:09<79:40:50,  5.20s/it]
 75%|███████▌  | 169803/225000 [326:01:14<80:32:01,  5.25s/it]
 75%|███████▌  | 169804/225000 [326:01:25<107:44:22,  7.03s/it]
 75%|███████▌  | 169805/225000 [326:01:33<109:29:23,  7.14s/it]
 75%|███████▌  | 169806/225000 [326:01:36<91:18:38,  5.96s/it] 
 75%|███████▌  | 169807/225000 [326:01:41<86:19:09,  5.63s/it]
 75%|███████▌  | 169808/225000 [326:01:46<85:39:16,  5.59s/it]
 75%|███████▌  | 169809/225000 [326:01:51<83:25:28,  5.44s/it]
 75%|███████▌  | 169810/225000 [326:01:57<82:27:04,  5.38s/it]
 75%|███████▌  | 169811/225000 [326:02:02<83:08:05,  5.42s/it]
 75%|███████▌  | 169812/225000 [326:02:08<83:44:27,  5.46s/it]
 75%|███████▌  | 169813/225000 [326:02:13<85:23:20,  5.57s/it]
 75%|███████▌  | 169814/225000 [326:02:19<86:30:36,  5.64s/it]
 75%|███████▌  | 169815/225000 [326:02:25<85:23:40,  5.57s/it]
 75%|███████▌  | 169816/225000 [326:02:30<83:28:51,  5.45s/it]
 75%|███████▌  | 169817/225000 [326:02:35<82:18:23,  5.37s/it]
 75%|███████▌  | 169818/225000 [326:02:41<83:06:31,  5.42s/it]
 75%|███████▌  | 169819/225000 [326:02:46<82:01:16,  5.35s/it]
 75%|███████▌  | 169820/225000 [326:02:51<82:54:55,  5.41s/it]
 75%|███████▌  | 169821/225000 [326:02:57<83:33:12,  5.45s/it]
 75%|███████▌  | 169822/225000 [326:03:02<82:50:11,  5.40s/it]
 75%|███████▌  | 169823/225000 [326:03:06<77:37:11,  5.06s/it]
 75%|███████▌  | 169824/225000 [326:03:12<79:39:58,  5.20s/it]
 75%|███████▌  | 169825/225000 [326:03:17<80:18:10,  5.24s/it]
 75%|███████▌  | 169826/225000 [326:03:25<90:44:58,  5.92s/it]
 75%|███████▌  | 169827/225000 [326:03:32<96:03:49,  6.27s/it]
 75%|███████▌  | 169828/225000 [326:03:35<83:44:04,  5.46s/it]
 75%|███████▌  | 169829/225000 [326:03:43<92:21:43,  6.03s/it]
 75%|███████▌  | 169830/225000 [326:03:51<101:49:25,  6.64s/it]
 75%|███████▌  | 169831/225000 [326:03:55<91:09:29,  5.95s/it] 
 75%|███████▌  | 169832/225000 [326:04:03<97:37:47,  6.37s/it]
 75%|███████▌  | 169833/225000 [326:04:06<85:25:21,  5.57s/it]
 75%|███████▌  | 169834/225000 [326:04:10<76:06:27,  4.97s/it]
 75%|███████▌  | 169835/225000 [326:04:18<93:03:58,  6.07s/it]
 75%|███████▌  | 169836/225000 [326:04:22<80:54:28,  5.28s/it]
 75%|███████▌  | 169837/225000 [326:04:30<92:31:24,  6.04s/it]
 75%|███████▌  | 169838/225000 [326:04:34<82:36:32,  5.39s/it]
 75%|███████▌  | 169839/225000 [326:04:38<80:23:10,  5.25s/it]
 75%|███████▌  | 169840/225000 [326:04:44<81:28:26,  5.32s/it]
 75%|███████▌  | 169841/225000 [326:04:49<80:14:01,  5.24s/it]
 75%|███████▌  | 169842/225000 [326:04:54<78:07:50,  5.10s/it]
 75%|███████▌  | 169843/225000 [326:04:59<77:34:15,  5.06s/it]
 75%|███████▌  | 169844/225000 [326:05:10<106:53:55,  6.98s/it]
 75%|███████▌  | 169845/225000 [326:05:14<92:24:14,  6.03s/it] 
 75%|███████▌  | 169846/225000 [326:05:18<81:50:41,  5.34s/it]
 75%|███████▌  | 169847/225000 [326:05:22<76:35:53,  5.00s/it]
 75%|███████▌  | 169848/225000 [326:05:26<74:02:09,  4.83s/it]
 75%|███████▌  | 169849/225000 [326:05:31<71:50:18,  4.69s/it]
 75%|███████▌  | 169850/225000 [326:05:42<103:22:55,  6.75s/it]
                                                               
{'loss': '0.7645', 'grad_norm': '1.161', 'learning_rate': '7.708e-06', 'epoch': '0.7549', 'num_input_tokens_seen': 540143720, 'train_runtime': '1.174e+06', 'train_tokens_per_second': '460.1'}

 75%|███████▌  | 169850/225000 [326:05:42<103:22:55,  6.75s/it]
 75%|███████▌  | 169851/225000 [326:05:48<97:08:07,  6.34s/it] 
 75%|███████▌  | 169852/225000 [326:05:52<89:02:40,  5.81s/it]
 75%|███████▌  | 169853/225000 [326:05:57<82:41:33,  5.40s/it]
 75%|███████▌  | 169854/225000 [326:06:01<79:37:56,  5.20s/it]
 75%|███████▌  | 169855/225000 [326:06:06<77:56:51,  5.09s/it]
 75%|███████▌  | 169856/225000 [326:06:10<73:49:46,  4.82s/it]
 75%|███████▌  | 169857/225000 [326:06:15<71:26:18,  4.66s/it]
 75%|███████▌  | 169858/225000 [326:06:27<104:08:53,  6.80s/it]
 75%|███████▌  | 169859/225000 [326:06:31<94:05:42,  6.14s/it] 
 75%|███████▌  | 169860/225000 [326:06:36<86:35:43,  5.65s/it]
 75%|███████▌  | 169861/225000 [326:06:40<82:22:23,  5.38s/it]
 75%|███████▌  | 169862/225000 [326:06:45<81:00:17,  5.29s/it]
 75%|███████▌  | 169863/225000 [326:06:50<76:32:40,  5.00s/it]
 75%|███████▌  | 169864/225000 [326:06:55<76:20:29,  4.98s/it]
 75%|███████▌  | 169865/225000 [326:06:59<74:50:48,  4.89s/it]
 75%|███████▌  | 169866/225000 [326:07:08<90:09:05,  5.89s/it]
 75%|███████▌  | 169867/225000 [326:07:16<99:42:01,  6.51s/it]
 75%|███████▌  | 169868/225000 [326:07:19<85:43:30,  5.60s/it]
 75%|███████▌  | 169869/225000 [326:07:26<93:49:15,  6.13s/it]
 75%|███████▌  | 169870/225000 [326:07:31<84:49:26,  5.54s/it]
 75%|███████▌  | 169871/225000 [326:07:35<79:33:26,  5.20s/it]
 75%|███████▌  | 169872/225000 [326:07:42<87:38:21,  5.72s/it]
 75%|███████▌  | 169873/225000 [326:07:49<93:25:52,  6.10s/it]
 75%|███████▌  | 169874/225000 [326:07:53<82:48:20,  5.41s/it]
 76%|███████▌  | 169875/225000 [326:07:57<76:08:21,  4.97s/it]
 76%|███████▌  | 169876/225000 [326:08:01<71:34:52,  4.67s/it]
 76%|███████▌  | 169877/225000 [326:08:11<99:32:34,  6.50s/it]
 76%|███████▌  | 169878/225000 [326:08:15<85:33:42,  5.59s/it]
 76%|███████▌  | 169879/225000 [326:08:23<95:17:17,  6.22s/it]
 76%|███████▌  | 169880/225000 [326:08:26<83:27:01,  5.45s/it]
 76%|███████▌  | 169881/225000 [326:08:30<74:25:52,  4.86s/it]
 76%|███████▌  | 169882/225000 [326:08:39<93:44:47,  6.12s/it]
 76%|███████▌  | 169883/225000 [326:08:43<83:18:35,  5.44s/it]
 76%|███████▌  | 169884/225000 [326:08:47<78:14:10,  5.11s/it]
 76%|███████▌  | 169885/225000 [326:08:51<74:35:36,  4.87s/it]
 76%|███████▌  | 169886/225000 [326:08:56<72:14:46,  4.72s/it]
 76%|███████▌  | 169887/225000 [326:09:07<103:50:23,  6.78s/it]
 76%|███████▌  | 169888/225000 [326:09:12<94:34:18,  6.18s/it] 
 76%|███████▌  | 169889/225000 [326:09:16<82:54:58,  5.42s/it]
 76%|███████▌  | 169890/225000 [326:09:20<76:40:06,  5.01s/it]
 76%|███████▌  | 169891/225000 [326:09:24<72:47:35,  4.76s/it]
 76%|███████▌  | 169892/225000 [326:09:31<84:09:18,  5.50s/it]
 76%|███████▌  | 169893/225000 [326:09:38<89:38:25,  5.86s/it]
 76%|███████▌  | 169894/225000 [326:09:42<80:23:00,  5.25s/it]
 76%|███████▌  | 169895/225000 [326:09:46<74:57:04,  4.90s/it]
 76%|███████▌  | 169896/225000 [326:09:57<106:05:36,  6.93s/it]
 76%|███████▌  | 169897/225000 [326:10:02<93:21:55,  6.10s/it] 
 76%|███████▌  | 169898/225000 [326:10:06<84:04:01,  5.49s/it]
 76%|███████▌  | 169899/225000 [326:10:10<77:29:12,  5.06s/it]
 76%|███████▌  | 169900/225000 [326:10:14<75:47:01,  4.95s/it]
                                                              
{'loss': '0.7895', 'grad_norm': '1.143', 'learning_rate': '7.695e-06', 'epoch': '0.7551', 'num_input_tokens_seen': 540273016, 'train_runtime': '1.174e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 169900/225000 [326:10:14<75:47:01,  4.95s/it]
 76%|███████▌  | 169901/225000 [326:10:20<76:46:59,  5.02s/it]
 76%|███████▌  | 169902/225000 [326:10:24<76:06:41,  4.97s/it]
 76%|███████▌  | 169903/225000 [326:10:34<97:21:07,  6.36s/it]
 76%|███████▌  | 169904/225000 [326:10:41<98:16:12,  6.42s/it]
 76%|███████▌  | 169905/225000 [326:10:44<85:01:44,  5.56s/it]
 76%|███████▌  | 169906/225000 [326:10:51<90:35:03,  5.92s/it]
 76%|███████▌  | 169907/225000 [326:10:55<81:27:55,  5.32s/it]
 76%|███████▌  | 169908/225000 [326:10:59<74:15:22,  4.85s/it]
 76%|███████▌  | 169909/225000 [326:11:03<70:33:05,  4.61s/it]
 76%|███████▌  | 169910/225000 [326:11:12<94:22:26,  6.17s/it]
 76%|███████▌  | 169911/225000 [326:11:18<92:32:16,  6.05s/it]
 76%|███████▌  | 169912/225000 [326:11:22<83:03:13,  5.43s/it]
 76%|███████▌  | 169913/225000 [326:11:26<74:50:09,  4.89s/it]
 76%|███████▌  | 169914/225000 [326:11:35<96:25:19,  6.30s/it]
 76%|███████▌  | 169915/225000 [326:11:39<83:49:25,  5.48s/it]
 76%|███████▌  | 169916/225000 [326:11:44<81:32:04,  5.33s/it]
 76%|███████▌  | 169917/225000 [326:11:51<89:10:37,  5.83s/it]
 76%|███████▌  | 169918/225000 [326:11:54<78:19:04,  5.12s/it]
 76%|███████▌  | 169919/225000 [326:12:02<91:20:49,  5.97s/it]
 76%|███████▌  | 169920/225000 [326:12:06<80:05:16,  5.23s/it]
 76%|███████▌  | 169921/225000 [326:12:11<79:26:17,  5.19s/it]
 76%|███████▌  | 169922/225000 [326:12:19<90:27:18,  5.91s/it]
 76%|███████▌  | 169923/225000 [326:12:22<79:22:39,  5.19s/it]
 76%|███████▌  | 169924/225000 [326:12:26<72:32:04,  4.74s/it]
 76%|███████▌  | 169925/225000 [326:12:32<77:36:58,  5.07s/it]
 76%|███████▌  | 169926/225000 [326:12:40<92:43:36,  6.06s/it]
 76%|███████▌  | 169927/225000 [326:12:44<82:05:30,  5.37s/it]
 76%|███████▌  | 169928/225000 [326:12:51<89:06:43,  5.83s/it]
 76%|███████▌  | 169929/225000 [326:12:54<80:01:56,  5.23s/it]
 76%|███████▌  | 169930/225000 [326:12:58<72:46:00,  4.76s/it]
 76%|███████▌  | 169931/225000 [326:13:05<84:18:05,  5.51s/it]
 76%|███████▌  | 169932/225000 [326:13:13<92:35:36,  6.05s/it]
 76%|███████▌  | 169933/225000 [326:13:16<80:42:11,  5.28s/it]
 76%|███████▌  | 169934/225000 [326:13:20<72:40:11,  4.75s/it]
 76%|███████▌  | 169935/225000 [326:13:29<91:36:26,  5.99s/it]
 76%|███████▌  | 169936/225000 [326:13:32<78:20:04,  5.12s/it]
 76%|███████▌  | 169937/225000 [326:13:37<77:42:59,  5.08s/it]
 76%|███████▌  | 169938/225000 [326:13:45<92:37:28,  6.06s/it]
 76%|███████▌  | 169939/225000 [326:13:48<80:53:36,  5.29s/it]
 76%|███████▌  | 169940/225000 [326:13:57<95:55:16,  6.27s/it]
 76%|███████▌  | 169941/225000 [326:14:01<85:54:57,  5.62s/it]
 76%|███████▌  | 169942/225000 [326:14:06<80:45:21,  5.28s/it]
 76%|███████▌  | 169943/225000 [326:14:10<76:31:00,  5.00s/it]
 76%|███████▌  | 169944/225000 [326:14:14<74:13:08,  4.85s/it]
 76%|███████▌  | 169945/225000 [326:14:19<72:05:54,  4.71s/it]
 76%|███████▌  | 169946/225000 [326:14:29<98:42:03,  6.45s/it]
 76%|███████▌  | 169947/225000 [326:14:34<89:27:09,  5.85s/it]
 76%|███████▌  | 169948/225000 [326:14:38<82:21:42,  5.39s/it]
 76%|███████▌  | 169949/225000 [326:14:43<78:37:56,  5.14s/it]
 76%|███████▌  | 169950/225000 [326:14:47<75:16:23,  4.92s/it]
                                                              
{'loss': '0.7742', 'grad_norm': '1.187', 'learning_rate': '7.682e-06', 'epoch': '0.7553', 'num_input_tokens_seen': 540403024, 'train_runtime': '1.174e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 169950/225000 [326:14:47<75:16:23,  4.92s/it]
 76%|███████▌  | 169951/225000 [326:14:53<81:44:17,  5.35s/it]
 76%|███████▌  | 169952/225000 [326:15:04<107:28:51,  7.03s/it]
 76%|███████▌  | 169953/225000 [326:15:09<97:49:54,  6.40s/it] 
 76%|███████▌  | 169954/225000 [326:15:15<93:12:42,  6.10s/it]
 76%|███████▌  | 169955/225000 [326:15:20<88:16:18,  5.77s/it]
 76%|███████▌  | 169956/225000 [326:15:24<83:41:16,  5.47s/it]
 76%|███████▌  | 169957/225000 [326:15:29<81:15:29,  5.31s/it]
 76%|███████▌  | 169958/225000 [326:15:34<78:12:26,  5.12s/it]
 76%|███████▌  | 169959/225000 [326:15:39<75:54:00,  4.96s/it]
 76%|███████▌  | 169960/225000 [326:15:43<74:57:02,  4.90s/it]
 76%|███████▌  | 169961/225000 [326:15:49<76:32:05,  5.01s/it]
 76%|███████▌  | 169962/225000 [326:15:54<76:12:38,  4.98s/it]
 76%|███████▌  | 169963/225000 [326:15:58<75:33:48,  4.94s/it]
 76%|███████▌  | 169964/225000 [326:16:06<89:09:58,  5.83s/it]
 76%|███████▌  | 169965/225000 [326:16:16<108:44:41,  7.11s/it]
 76%|███████▌  | 169966/225000 [326:16:20<90:58:52,  5.95s/it] 
 76%|███████▌  | 169967/225000 [326:16:26<94:24:07,  6.18s/it]
 76%|███████▌  | 169968/225000 [326:16:32<90:04:43,  5.89s/it]
 76%|███████▌  | 169969/225000 [326:16:36<82:42:44,  5.41s/it]
 76%|███████▌  | 169970/225000 [326:16:40<74:53:47,  4.90s/it]
 76%|███████▌  | 169971/225000 [326:16:44<72:03:08,  4.71s/it]
 76%|███████▌  | 169972/225000 [326:16:53<93:31:49,  6.12s/it]
 76%|███████▌  | 169973/225000 [326:16:58<88:02:57,  5.76s/it]
 76%|███████▌  | 169974/225000 [326:17:03<82:02:06,  5.37s/it]
 76%|███████▌  | 169975/225000 [326:17:07<75:58:30,  4.97s/it]
 76%|███████▌  | 169976/225000 [326:17:11<74:23:07,  4.87s/it]
 76%|███████▌  | 169977/225000 [326:17:16<72:13:11,  4.73s/it]
 76%|███████▌  | 169978/225000 [326:17:20<70:57:57,  4.64s/it]
 76%|███████▌  | 169979/225000 [326:17:30<92:45:05,  6.07s/it]
 76%|███████▌  | 169980/225000 [326:17:35<89:19:29,  5.84s/it]
 76%|███████▌  | 169981/225000 [326:17:39<79:07:21,  5.18s/it]
 76%|███████▌  | 169982/225000 [326:17:48<96:21:19,  6.30s/it]
 76%|███████▌  | 169983/225000 [326:17:51<84:03:44,  5.50s/it]
 76%|███████▌  | 169984/225000 [326:17:59<96:37:24,  6.32s/it]
 76%|███████▌  | 169985/225000 [326:18:03<82:36:17,  5.41s/it]
 76%|███████▌  | 169986/225000 [326:18:06<73:04:16,  4.78s/it]
 76%|███████▌  | 169987/225000 [326:18:15<93:12:46,  6.10s/it]
 76%|███████▌  | 169988/225000 [326:18:18<79:15:15,  5.19s/it]
 76%|███████▌  | 169989/225000 [326:18:22<72:16:18,  4.73s/it]
 76%|███████▌  | 169990/225000 [326:18:30<89:13:16,  5.84s/it]
 76%|███████▌  | 169991/225000 [326:18:34<77:42:29,  5.09s/it]
 76%|███████▌  | 169992/225000 [326:18:41<88:23:29,  5.78s/it]
 76%|███████▌  | 169993/225000 [326:18:46<83:55:36,  5.49s/it]
 76%|███████▌  | 169994/225000 [326:18:52<87:53:32,  5.75s/it]
 76%|███████▌  | 169995/225000 [326:18:57<84:43:02,  5.54s/it]
 76%|███████▌  | 169996/225000 [326:19:03<83:51:43,  5.49s/it]
 76%|███████▌  | 169997/225000 [326:19:08<81:42:58,  5.35s/it]
 76%|███████▌  | 169998/225000 [326:19:13<82:12:14,  5.38s/it]
 76%|███████▌  | 169999/225000 [326:19:18<79:18:24,  5.19s/it]
 76%|███████▌  | 170000/225000 [326:19:23<79:43:19,  5.22s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.262', 'learning_rate': '7.669e-06', 'epoch': '0.7556', 'num_input_tokens_seen': 540533160, 'train_runtime': '1.175e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170000/225000 [326:19:23<79:43:19,  5.22s/it]
 76%|███████▌  | 170001/225000 [326:19:28<78:01:38,  5.11s/it]
 76%|███████▌  | 170002/225000 [326:19:33<76:07:54,  4.98s/it]
 76%|███████▌  | 170003/225000 [326:19:38<76:49:09,  5.03s/it]
 76%|███████▌  | 170004/225000 [326:19:50<110:48:29,  7.25s/it]
 76%|███████▌  | 170005/225000 [326:19:55<98:16:44,  6.43s/it] 
 76%|███████▌  | 170006/225000 [326:19:58<85:14:12,  5.58s/it]
 76%|███████▌  | 170007/225000 [326:20:08<102:39:48,  6.72s/it]
 76%|███████▌  | 170008/225000 [326:20:11<86:03:28,  5.63s/it] 
 76%|███████▌  | 170009/225000 [326:20:14<75:11:29,  4.92s/it]
 76%|███████▌  | 170010/225000 [326:20:19<73:26:26,  4.81s/it]
 76%|███████▌  | 170011/225000 [326:20:23<71:28:45,  4.68s/it]
 76%|███████▌  | 170012/225000 [326:20:28<70:45:20,  4.63s/it]
 76%|███████▌  | 170013/225000 [326:20:38<98:59:03,  6.48s/it]
 76%|███████▌  | 170014/225000 [326:20:43<90:47:31,  5.94s/it]
 76%|███████▌  | 170015/225000 [326:20:50<93:54:09,  6.15s/it]
 76%|███████▌  | 170016/225000 [326:20:55<89:47:39,  5.88s/it]
 76%|███████▌  | 170017/225000 [326:21:01<91:12:59,  5.97s/it]
 76%|███████▌  | 170018/225000 [326:21:06<88:00:39,  5.76s/it]
 76%|███████▌  | 170019/225000 [326:21:11<84:01:36,  5.50s/it]
 76%|███████▌  | 170020/225000 [326:21:16<81:26:21,  5.33s/it]
 76%|███████▌  | 170021/225000 [326:21:21<79:55:22,  5.23s/it]
 76%|███████▌  | 170022/225000 [326:21:27<80:19:04,  5.26s/it]
 76%|███████▌  | 170023/225000 [326:21:31<78:23:14,  5.13s/it]
 76%|███████▌  | 170024/225000 [326:21:36<78:17:13,  5.13s/it]
 76%|███████▌  | 170025/225000 [326:21:42<78:07:30,  5.12s/it]
 76%|███████▌  | 170026/225000 [326:21:47<79:04:54,  5.18s/it]
 76%|███████▌  | 170027/225000 [326:21:52<78:45:33,  5.16s/it]
 76%|███████▌  | 170028/225000 [326:21:57<79:52:35,  5.23s/it]
 76%|███████▌  | 170029/225000 [326:22:03<79:40:04,  5.22s/it]
 76%|███████▌  | 170030/225000 [326:22:07<78:13:23,  5.12s/it]
 76%|███████▌  | 170031/225000 [326:22:12<77:00:12,  5.04s/it]
 76%|███████▌  | 170032/225000 [326:22:17<77:21:46,  5.07s/it]
 76%|███████▌  | 170033/225000 [326:22:23<78:49:18,  5.16s/it]
 76%|███████▌  | 170034/225000 [326:22:28<80:51:37,  5.30s/it]
 76%|███████▌  | 170035/225000 [326:22:34<80:41:17,  5.28s/it]
 76%|███████▌  | 170036/225000 [326:22:39<80:32:50,  5.28s/it]
 76%|███████▌  | 170037/225000 [326:22:45<82:36:30,  5.41s/it]
 76%|███████▌  | 170038/225000 [326:22:50<81:09:15,  5.32s/it]
 76%|███████▌  | 170039/225000 [326:22:55<79:42:51,  5.22s/it]
 76%|███████▌  | 170040/225000 [326:23:00<80:02:15,  5.24s/it]
 76%|███████▌  | 170041/225000 [326:23:12<111:14:46,  7.29s/it]
 76%|███████▌  | 170042/225000 [326:23:17<102:23:22,  6.71s/it]
 76%|███████▌  | 170043/225000 [326:23:23<97:42:56,  6.40s/it] 
 76%|███████▌  | 170044/225000 [326:23:28<90:13:09,  5.91s/it]
 76%|███████▌  | 170045/225000 [326:23:34<91:38:22,  6.00s/it]
 76%|███████▌  | 170046/225000 [326:23:38<79:29:48,  5.21s/it]
 76%|███████▌  | 170047/225000 [326:23:47<99:02:57,  6.49s/it]
 76%|███████▌  | 170048/225000 [326:23:50<84:05:03,  5.51s/it]
 76%|███████▌  | 170049/225000 [326:23:56<84:03:59,  5.51s/it]
 76%|███████▌  | 170050/225000 [326:24:01<82:48:12,  5.42s/it]
                                                              
{'loss': '0.7766', 'grad_norm': '1.327', 'learning_rate': '7.656e-06', 'epoch': '0.7558', 'num_input_tokens_seen': 540663864, 'train_runtime': '1.175e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170050/225000 [326:24:01<82:48:12,  5.42s/it]
 76%|███████▌  | 170051/225000 [326:24:06<79:09:41,  5.19s/it]
 76%|███████▌  | 170052/225000 [326:24:10<74:53:55,  4.91s/it]
 76%|███████▌  | 170053/225000 [326:24:14<71:48:31,  4.70s/it]
 76%|███████▌  | 170054/225000 [326:24:23<90:45:56,  5.95s/it]
 76%|███████▌  | 170055/225000 [326:24:29<90:25:40,  5.92s/it]
 76%|███████▌  | 170056/225000 [326:24:34<85:44:40,  5.62s/it]
 76%|███████▌  | 170057/225000 [326:24:39<84:14:42,  5.52s/it]
 76%|███████▌  | 170058/225000 [326:24:44<81:30:47,  5.34s/it]
 76%|███████▌  | 170059/225000 [326:24:49<82:40:55,  5.42s/it]
 76%|███████▌  | 170060/225000 [326:24:54<79:51:08,  5.23s/it]
 76%|███████▌  | 170061/225000 [326:25:00<80:35:16,  5.28s/it]
 76%|███████▌  | 170062/225000 [326:25:05<79:31:16,  5.21s/it]
 76%|███████▌  | 170063/225000 [326:25:10<81:08:49,  5.32s/it]
 76%|███████▌  | 170064/225000 [326:25:16<80:55:26,  5.30s/it]
 76%|███████▌  | 170065/225000 [326:25:21<80:23:09,  5.27s/it]
 76%|███████▌  | 170066/225000 [326:25:27<83:58:16,  5.50s/it]
 76%|███████▌  | 170067/225000 [326:25:35<94:20:32,  6.18s/it]
 76%|███████▌  | 170068/225000 [326:25:39<86:26:15,  5.66s/it]
 76%|███████▌  | 170069/225000 [326:25:43<79:22:18,  5.20s/it]
 76%|███████▌  | 170070/225000 [326:25:48<76:33:51,  5.02s/it]
 76%|███████▌  | 170071/225000 [326:25:53<75:47:16,  4.97s/it]
 76%|███████▌  | 170072/225000 [326:26:01<90:56:55,  5.96s/it]
 76%|███████▌  | 170073/225000 [326:26:09<99:46:02,  6.54s/it]
 76%|███████▌  | 170074/225000 [326:26:13<89:28:18,  5.86s/it]
 76%|███████▌  | 170075/225000 [326:26:17<82:09:13,  5.38s/it]
 76%|███████▌  | 170076/225000 [326:26:22<78:07:24,  5.12s/it]
 76%|███████▌  | 170077/225000 [326:26:26<74:55:54,  4.91s/it]
 76%|███████▌  | 170078/225000 [326:26:31<74:28:58,  4.88s/it]
 76%|███████▌  | 170079/225000 [326:26:36<72:57:15,  4.78s/it]
 76%|███████▌  | 170080/225000 [326:26:40<73:03:32,  4.79s/it]
 76%|███████▌  | 170081/225000 [326:26:45<72:25:43,  4.75s/it]
 76%|███████▌  | 170082/225000 [326:26:56<98:29:14,  6.46s/it]
 76%|███████▌  | 170083/225000 [326:27:03<103:29:18,  6.78s/it]
 76%|███████▌  | 170084/225000 [326:27:07<88:25:31,  5.80s/it] 
 76%|███████▌  | 170085/225000 [326:27:14<97:50:49,  6.41s/it]
 76%|███████▌  | 170086/225000 [326:27:20<93:27:37,  6.13s/it]
 76%|███████▌  | 170087/225000 [326:27:24<82:59:31,  5.44s/it]
 76%|███████▌  | 170088/225000 [326:27:33<100:02:24,  6.56s/it]
 76%|███████▌  | 170089/225000 [326:27:38<93:29:52,  6.13s/it] 
 76%|███████▌  | 170090/225000 [326:27:42<84:57:42,  5.57s/it]
 76%|███████▌  | 170091/225000 [326:27:48<87:19:16,  5.73s/it]
 76%|███████▌  | 170092/225000 [326:27:55<90:10:20,  5.91s/it]
 76%|███████▌  | 170093/225000 [326:27:59<82:15:14,  5.39s/it]
 76%|███████▌  | 170094/225000 [326:28:03<76:43:57,  5.03s/it]
 76%|███████▌  | 170095/225000 [326:28:07<71:56:03,  4.72s/it]
 76%|███████▌  | 170096/225000 [326:28:16<92:50:06,  6.09s/it]
 76%|███████▌  | 170097/225000 [326:28:20<81:49:15,  5.37s/it]
 76%|███████▌  | 170098/225000 [326:28:24<76:09:21,  4.99s/it]
 76%|███████▌  | 170099/225000 [326:28:28<71:24:32,  4.68s/it]
 76%|███████▌  | 170100/225000 [326:28:37<88:40:28,  5.81s/it]
                                                              
{'loss': '0.753', 'grad_norm': '1.203', 'learning_rate': '7.643e-06', 'epoch': '0.756', 'num_input_tokens_seen': 540793920, 'train_runtime': '1.175e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170100/225000 [326:28:37<88:40:28,  5.81s/it]
 76%|███████▌  | 170101/225000 [326:28:42<88:15:08,  5.79s/it]
 76%|███████▌  | 170102/225000 [326:28:47<84:31:32,  5.54s/it]
 76%|███████▌  | 170103/225000 [326:28:53<87:41:28,  5.75s/it]
 76%|███████▌  | 170104/225000 [326:29:01<95:58:39,  6.29s/it]
 76%|███████▌  | 170105/225000 [326:29:08<98:04:02,  6.43s/it]
 76%|███████▌  | 170106/225000 [326:29:12<87:54:17,  5.76s/it]
 76%|███████▌  | 170107/225000 [326:29:16<80:32:39,  5.28s/it]
 76%|███████▌  | 170108/225000 [326:29:20<73:34:59,  4.83s/it]
 76%|███████▌  | 170109/225000 [326:29:29<93:34:14,  6.14s/it]
 76%|███████▌  | 170110/225000 [326:29:34<86:36:46,  5.68s/it]
 76%|███████▌  | 170111/225000 [326:29:39<84:25:54,  5.54s/it]
 76%|███████▌  | 170112/225000 [326:29:43<75:50:50,  4.97s/it]
 76%|███████▌  | 170113/225000 [326:29:49<82:46:43,  5.43s/it]
 76%|███████▌  | 170114/225000 [326:29:55<85:36:50,  5.62s/it]
 76%|███████▌  | 170115/225000 [326:30:00<84:25:15,  5.54s/it]
 76%|███████▌  | 170116/225000 [326:30:04<73:21:39,  4.81s/it]
 76%|███████▌  | 170117/225000 [326:30:12<87:47:01,  5.76s/it]
 76%|███████▌  | 170118/225000 [326:30:15<78:08:32,  5.13s/it]
 76%|███████▌  | 170119/225000 [326:30:19<71:08:17,  4.67s/it]
 76%|███████▌  | 170120/225000 [326:30:24<71:33:40,  4.69s/it]
 76%|███████▌  | 170121/225000 [326:30:29<73:35:08,  4.83s/it]
 76%|███████▌  | 170122/225000 [326:30:34<75:06:32,  4.93s/it]
 76%|███████▌  | 170123/225000 [326:30:39<77:06:13,  5.06s/it]
 76%|███████▌  | 170124/225000 [326:30:44<76:31:07,  5.02s/it]
 76%|███████▌  | 170125/225000 [326:30:50<78:51:24,  5.17s/it]
 76%|███████▌  | 170126/225000 [326:30:55<77:07:16,  5.06s/it]
 76%|███████▌  | 170127/225000 [326:30:59<76:39:26,  5.03s/it]
 76%|███████▌  | 170128/225000 [326:31:05<77:10:18,  5.06s/it]
 76%|███████▌  | 170129/225000 [326:31:10<77:56:08,  5.11s/it]
 76%|███████▌  | 170130/225000 [326:31:20<101:47:48,  6.68s/it]
 76%|███████▌  | 170131/225000 [326:31:28<107:31:35,  7.05s/it]
 76%|███████▌  | 170132/225000 [326:31:35<109:04:17,  7.16s/it]
 76%|███████▌  | 170133/225000 [326:31:39<94:26:42,  6.20s/it] 
 76%|███████▌  | 170134/225000 [326:31:43<83:13:45,  5.46s/it]
 76%|███████▌  | 170135/225000 [326:31:48<80:48:50,  5.30s/it]
 76%|███████▌  | 170136/225000 [326:31:54<81:15:12,  5.33s/it]
 76%|███████▌  | 170137/225000 [326:31:59<80:23:31,  5.28s/it]
 76%|███████▌  | 170138/225000 [326:32:04<78:36:12,  5.16s/it]
 76%|███████▌  | 170139/225000 [326:32:09<78:44:44,  5.17s/it]
 76%|███████▌  | 170140/225000 [326:32:14<77:12:27,  5.07s/it]
 76%|███████▌  | 170141/225000 [326:32:18<76:19:46,  5.01s/it]
 76%|███████▌  | 170142/225000 [326:32:23<75:19:59,  4.94s/it]
 76%|███████▌  | 170143/225000 [326:32:29<77:18:48,  5.07s/it]
 76%|███████▌  | 170144/225000 [326:32:41<110:03:28,  7.22s/it]
 76%|███████▌  | 170145/225000 [326:32:45<96:38:14,  6.34s/it] 
 76%|███████▌  | 170146/225000 [326:32:53<102:42:19,  6.74s/it]
 76%|███████▌  | 170147/225000 [326:32:56<87:47:01,  5.76s/it] 
 76%|███████▌  | 170148/225000 [326:33:00<78:03:08,  5.12s/it]
 76%|███████▌  | 170149/225000 [326:33:10<101:09:38,  6.64s/it]
 76%|███████▌  | 170150/225000 [326:33:14<87:39:38,  5.75s/it] 
                                                              
{'loss': '0.7583', 'grad_norm': '1.11', 'learning_rate': '7.629e-06', 'epoch': '0.7562', 'num_input_tokens_seen': 540924416, 'train_runtime': '1.176e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170150/225000 [326:33:14<87:39:38,  5.75s/it]
 76%|███████▌  | 170151/225000 [326:33:18<78:48:59,  5.17s/it]
 76%|███████▌  | 170152/225000 [326:33:23<78:58:15,  5.18s/it]
 76%|███████▌  | 170153/225000 [326:33:28<80:46:14,  5.30s/it]
 76%|███████▌  | 170154/225000 [326:33:34<81:34:25,  5.35s/it]
 76%|███████▌  | 170155/225000 [326:33:40<85:54:54,  5.64s/it]
 76%|███████▌  | 170156/225000 [326:33:50<102:56:44,  6.76s/it]
 76%|███████▌  | 170157/225000 [326:33:53<89:16:34,  5.86s/it] 
 76%|███████▌  | 170158/225000 [326:33:59<88:11:14,  5.79s/it]
 76%|███████▌  | 170159/225000 [326:34:05<87:39:40,  5.75s/it]
 76%|███████▌  | 170160/225000 [326:34:10<86:14:10,  5.66s/it]
 76%|███████▌  | 170161/225000 [326:34:15<84:50:09,  5.57s/it]
 76%|███████▌  | 170162/225000 [326:34:21<85:19:44,  5.60s/it]
 76%|███████▌  | 170163/225000 [326:34:26<83:06:12,  5.46s/it]
 76%|███████▌  | 170164/225000 [326:34:32<82:28:48,  5.41s/it]
 76%|███████▌  | 170165/225000 [326:34:37<82:32:23,  5.42s/it]
 76%|███████▌  | 170166/225000 [326:34:42<82:05:20,  5.39s/it]
 76%|███████▌  | 170167/225000 [326:34:48<83:03:05,  5.45s/it]
 76%|███████▌  | 170168/225000 [326:34:52<78:58:01,  5.18s/it]
 76%|███████▌  | 170169/225000 [326:34:57<76:32:43,  5.03s/it]
 76%|███████▌  | 170170/225000 [326:35:06<92:56:10,  6.10s/it]
 76%|███████▌  | 170171/225000 [326:35:10<85:39:35,  5.62s/it]
 76%|███████▌  | 170172/225000 [326:35:14<78:26:05,  5.15s/it]
 76%|███████▌  | 170173/225000 [326:35:19<74:43:43,  4.91s/it]
 76%|███████▌  | 170174/225000 [326:35:23<72:51:28,  4.78s/it]
 76%|███████▌  | 170175/225000 [326:35:27<70:58:03,  4.66s/it]
 76%|███████▌  | 170176/225000 [326:35:38<100:07:51,  6.58s/it]
 76%|███████▌  | 170177/225000 [326:35:43<91:35:58,  6.01s/it] 
 76%|███████▌  | 170178/225000 [326:35:49<88:21:33,  5.80s/it]
 76%|███████▌  | 170179/225000 [326:35:55<91:24:00,  6.00s/it]
 76%|███████▌  | 170180/225000 [326:35:59<82:06:55,  5.39s/it]
 76%|███████▌  | 170181/225000 [326:36:06<88:59:44,  5.84s/it]
 76%|███████▌  | 170182/225000 [326:36:12<90:36:49,  5.95s/it]
 76%|███████▌  | 170183/225000 [326:36:17<84:05:19,  5.52s/it]
 76%|███████▌  | 170184/225000 [326:36:21<80:37:25,  5.29s/it]
 76%|███████▌  | 170185/225000 [326:36:26<78:01:12,  5.12s/it]
 76%|███████▌  | 170186/225000 [326:36:31<77:55:52,  5.12s/it]
 76%|███████▌  | 170187/225000 [326:36:36<78:17:27,  5.14s/it]
 76%|███████▌  | 170188/225000 [326:36:42<78:52:10,  5.18s/it]
 76%|███████▌  | 170189/225000 [326:36:47<78:31:20,  5.16s/it]
 76%|███████▌  | 170190/225000 [326:36:51<75:36:00,  4.97s/it]
 76%|███████▌  | 170191/225000 [326:36:56<75:47:19,  4.98s/it]
 76%|███████▌  | 170192/225000 [326:37:01<75:53:44,  4.99s/it]
 76%|███████▌  | 170193/225000 [326:37:06<75:00:12,  4.93s/it]
 76%|███████▌  | 170194/225000 [326:37:12<77:30:52,  5.09s/it]
 76%|███████▌  | 170195/225000 [326:37:17<77:19:51,  5.08s/it]
 76%|███████▌  | 170196/225000 [326:37:21<76:07:51,  5.00s/it]
 76%|███████▌  | 170197/225000 [326:37:30<90:22:25,  5.94s/it]
 76%|███████▌  | 170198/225000 [326:37:41<114:31:03,  7.52s/it]
 76%|███████▌  | 170199/225000 [326:37:44<94:55:40,  6.24s/it] 
 76%|███████▌  | 170200/225000 [326:37:51<97:37:29,  6.41s/it]
                                                              
{'loss': '0.7753', 'grad_norm': '1.235', 'learning_rate': '7.616e-06', 'epoch': '0.7564', 'num_input_tokens_seen': 541054248, 'train_runtime': '1.176e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170200/225000 [326:37:51<97:37:29,  6.41s/it]
 76%|███████▌  | 170201/225000 [326:37:54<84:53:56,  5.58s/it]
 76%|███████▌  | 170202/225000 [326:38:02<95:01:07,  6.24s/it]
 76%|███████▌  | 170203/225000 [326:38:06<84:53:25,  5.58s/it]
 76%|███████▌  | 170204/225000 [326:38:10<77:33:47,  5.10s/it]
 76%|███████▌  | 170205/225000 [326:38:20<97:12:27,  6.39s/it]
 76%|███████▌  | 170206/225000 [326:38:24<88:27:19,  5.81s/it]
 76%|███████▌  | 170207/225000 [326:38:28<81:16:00,  5.34s/it]
 76%|███████▌  | 170208/225000 [326:38:32<75:52:13,  4.98s/it]
 76%|███████▌  | 170209/225000 [326:38:37<72:50:51,  4.79s/it]
 76%|███████▌  | 170210/225000 [326:38:48<100:07:44,  6.58s/it]
 76%|███████▌  | 170211/225000 [326:38:51<84:06:29,  5.53s/it] 
 76%|███████▌  | 170212/225000 [326:38:59<97:30:20,  6.41s/it]
 76%|███████▌  | 170213/225000 [326:39:02<83:39:22,  5.50s/it]
 76%|███████▌  | 170214/225000 [326:39:06<76:29:53,  5.03s/it]
 76%|███████▌  | 170215/225000 [326:39:11<72:25:25,  4.76s/it]
 76%|███████▌  | 170216/225000 [326:39:15<69:28:48,  4.57s/it]
 76%|███████▌  | 170217/225000 [326:39:24<92:37:03,  6.09s/it]
 76%|███████▌  | 170218/225000 [326:39:31<93:38:38,  6.15s/it]
 76%|███████▌  | 170219/225000 [326:39:34<82:43:52,  5.44s/it]
 76%|███████▌  | 170220/225000 [326:39:43<97:00:25,  6.38s/it]
 76%|███████▌  | 170221/225000 [326:39:47<84:37:38,  5.56s/it]
 76%|███████▌  | 170222/225000 [326:39:52<84:10:58,  5.53s/it]
 76%|███████▌  | 170223/225000 [326:39:59<92:56:33,  6.11s/it]
 76%|███████▌  | 170224/225000 [326:40:04<84:16:02,  5.54s/it]
 76%|███████▌  | 170225/225000 [326:40:07<75:36:21,  4.97s/it]
 76%|███████▌  | 170226/225000 [326:40:11<71:47:43,  4.72s/it]
 76%|███████▌  | 170227/225000 [326:40:23<100:57:53,  6.64s/it]
 76%|███████▌  | 170228/225000 [326:40:26<85:33:35,  5.62s/it] 
 76%|███████▌  | 170229/225000 [326:40:30<78:26:17,  5.16s/it]
 76%|███████▌  | 170230/225000 [326:40:37<88:39:51,  5.83s/it]
 76%|███████▌  | 170231/225000 [326:40:42<82:10:31,  5.40s/it]
 76%|███████▌  | 170232/225000 [326:40:46<75:31:37,  4.96s/it]
 76%|███████▌  | 170233/225000 [326:40:50<73:08:33,  4.81s/it]
 76%|███████▌  | 170234/225000 [326:40:59<91:49:08,  6.04s/it]
 76%|███████▌  | 170235/225000 [326:41:04<84:51:12,  5.58s/it]
 76%|███████▌  | 170236/225000 [326:41:08<81:08:06,  5.33s/it]
 76%|███████▌  | 170237/225000 [326:41:14<80:52:23,  5.32s/it]
 76%|███████▌  | 170238/225000 [326:41:22<94:00:06,  6.18s/it]
 76%|███████▌  | 170239/225000 [326:41:26<85:24:47,  5.62s/it]
 76%|███████▌  | 170240/225000 [326:41:30<78:34:28,  5.17s/it]
 76%|███████▌  | 170241/225000 [326:41:35<76:26:12,  5.03s/it]
 76%|███████▌  | 170242/225000 [326:41:40<75:01:00,  4.93s/it]
 76%|███████▌  | 170243/225000 [326:41:44<72:12:11,  4.75s/it]
 76%|███████▌  | 170244/225000 [326:41:49<72:17:19,  4.75s/it]
 76%|███████▌  | 170245/225000 [326:41:54<73:51:51,  4.86s/it]
 76%|███████▌  | 170246/225000 [326:41:58<71:15:50,  4.69s/it]
 76%|███████▌  | 170247/225000 [326:42:10<102:35:18,  6.75s/it]
 76%|███████▌  | 170248/225000 [326:42:14<93:38:38,  6.16s/it] 
 76%|███████▌  | 170249/225000 [326:42:20<88:57:20,  5.85s/it]
 76%|███████▌  | 170250/225000 [326:42:25<87:05:33,  5.73s/it]
                                                              
{'loss': '0.7644', 'grad_norm': '1.126', 'learning_rate': '7.603e-06', 'epoch': '0.7567', 'num_input_tokens_seen': 541184160, 'train_runtime': '1.176e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170250/225000 [326:42:25<87:05:33,  5.73s/it]
 76%|███████▌  | 170251/225000 [326:42:30<84:49:51,  5.58s/it]
 76%|███████▌  | 170252/225000 [326:42:35<80:30:40,  5.29s/it]
 76%|███████▌  | 170253/225000 [326:42:40<78:58:22,  5.19s/it]
 76%|███████▌  | 170254/225000 [326:42:45<79:27:52,  5.23s/it]
 76%|███████▌  | 170255/225000 [326:42:50<79:40:34,  5.24s/it]
 76%|███████▌  | 170256/225000 [326:42:55<78:06:11,  5.14s/it]
 76%|███████▌  | 170257/225000 [326:43:00<76:47:06,  5.05s/it]
 76%|███████▌  | 170258/225000 [326:43:05<77:22:54,  5.09s/it]
 76%|███████▌  | 170259/225000 [326:43:10<77:42:27,  5.11s/it]
 76%|███████▌  | 170260/225000 [326:43:15<77:20:13,  5.09s/it]
 76%|███████▌  | 170261/225000 [326:43:21<78:47:40,  5.18s/it]
 76%|███████▌  | 170262/225000 [326:43:26<78:04:30,  5.13s/it]
 76%|███████▌  | 170263/225000 [326:43:30<75:32:41,  4.97s/it]
 76%|███████▌  | 170264/225000 [326:43:37<81:29:38,  5.36s/it]
 76%|███████▌  | 170265/225000 [326:43:48<106:40:13,  7.02s/it]
 76%|███████▌  | 170266/225000 [326:43:52<95:32:53,  6.28s/it] 
 76%|███████▌  | 170267/225000 [326:43:57<89:36:05,  5.89s/it]
 76%|███████▌  | 170268/225000 [326:44:02<84:33:42,  5.56s/it]
 76%|███████▌  | 170269/225000 [326:44:07<82:32:54,  5.43s/it]
 76%|███████▌  | 170270/225000 [326:44:13<82:29:23,  5.43s/it]
 76%|███████▌  | 170271/225000 [326:44:18<81:34:54,  5.37s/it]
 76%|███████▌  | 170272/225000 [326:44:23<81:39:11,  5.37s/it]
 76%|███████▌  | 170273/225000 [326:44:28<80:18:03,  5.28s/it]
 76%|███████▌  | 170274/225000 [326:44:33<78:02:04,  5.13s/it]
 76%|███████▌  | 170275/225000 [326:44:43<98:18:09,  6.47s/it]
 76%|███████▌  | 170276/225000 [326:44:47<89:22:11,  5.88s/it]
 76%|███████▌  | 170277/225000 [326:44:51<81:52:03,  5.39s/it]
 76%|███████▌  | 170278/225000 [326:44:56<77:05:10,  5.07s/it]
 76%|███████▌  | 170279/225000 [326:45:00<73:28:05,  4.83s/it]
 76%|███████▌  | 170280/225000 [326:45:11<102:15:40,  6.73s/it]
 76%|███████▌  | 170281/225000 [326:45:17<98:33:38,  6.48s/it] 
 76%|███████▌  | 170282/225000 [326:45:22<90:30:44,  5.95s/it]
 76%|███████▌  | 170283/225000 [326:45:26<83:27:52,  5.49s/it]
 76%|███████▌  | 170284/225000 [326:45:31<78:46:57,  5.18s/it]
 76%|███████▌  | 170285/225000 [326:45:35<75:32:40,  4.97s/it]
 76%|███████▌  | 170286/225000 [326:45:40<74:50:57,  4.92s/it]
 76%|███████▌  | 170287/225000 [326:45:44<72:32:35,  4.77s/it]
 76%|███████▌  | 170288/225000 [326:45:49<73:27:51,  4.83s/it]
 76%|███████▌  | 170289/225000 [326:45:54<74:02:25,  4.87s/it]
 76%|███████▌  | 170290/225000 [326:46:03<91:31:56,  6.02s/it]
 76%|███████▌  | 170291/225000 [326:46:12<104:06:02,  6.85s/it]
 76%|███████▌  | 170292/225000 [326:46:15<89:28:05,  5.89s/it] 
 76%|███████▌  | 170293/225000 [326:46:20<83:04:51,  5.47s/it]
 76%|███████▌  | 170294/225000 [326:46:24<78:34:17,  5.17s/it]
 76%|███████▌  | 170295/225000 [326:46:29<76:44:32,  5.05s/it]
 76%|███████▌  | 170296/225000 [326:46:34<76:04:25,  5.01s/it]
 76%|███████▌  | 170297/225000 [326:46:39<74:21:45,  4.89s/it]
 76%|███████▌  | 170298/225000 [326:46:43<73:18:59,  4.83s/it]
 76%|███████▌  | 170299/225000 [326:46:48<72:07:04,  4.75s/it]
 76%|███████▌  | 170300/225000 [326:46:58<95:09:07,  6.26s/it]
                                                              
{'loss': '0.7568', 'grad_norm': '1.08', 'learning_rate': '7.59e-06', 'epoch': '0.7569', 'num_input_tokens_seen': 541313952, 'train_runtime': '1.176e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170300/225000 [326:46:58<95:09:07,  6.26s/it]
 76%|███████▌  | 170301/225000 [326:47:07<107:29:51,  7.07s/it]
 76%|███████▌  | 170302/225000 [326:47:11<95:59:26,  6.32s/it] 
 76%|███████▌  | 170303/225000 [326:47:17<91:41:42,  6.04s/it]
 76%|███████▌  | 170304/225000 [326:47:22<87:52:51,  5.78s/it]
 76%|███████▌  | 170305/225000 [326:47:27<83:42:49,  5.51s/it]
 76%|███████▌  | 170306/225000 [326:47:31<78:43:54,  5.18s/it]
 76%|███████▌  | 170307/225000 [326:47:36<77:13:09,  5.08s/it]
 76%|███████▌  | 170308/225000 [326:47:41<76:13:02,  5.02s/it]
 76%|███████▌  | 170309/225000 [326:47:46<77:01:41,  5.07s/it]
 76%|███████▌  | 170310/225000 [326:47:51<76:27:48,  5.03s/it]
 76%|███████▌  | 170311/225000 [326:47:56<76:05:05,  5.01s/it]
 76%|███████▌  | 170312/225000 [326:48:01<74:54:17,  4.93s/it]
 76%|███████▌  | 170313/225000 [326:48:05<73:02:02,  4.81s/it]
 76%|███████▌  | 170314/225000 [326:48:14<90:15:46,  5.94s/it]
 76%|███████▌  | 170315/225000 [326:48:20<93:56:18,  6.18s/it]
 76%|███████▌  | 170316/225000 [326:48:24<80:16:55,  5.29s/it]
 76%|███████▌  | 170317/225000 [326:48:33<96:50:41,  6.38s/it]
 76%|███████▌  | 170318/225000 [326:48:37<86:50:57,  5.72s/it]
 76%|███████▌  | 170319/225000 [326:48:41<79:54:48,  5.26s/it]
 76%|███████▌  | 170320/225000 [326:48:47<81:57:42,  5.40s/it]
 76%|███████▌  | 170321/225000 [326:48:55<94:42:29,  6.24s/it]
 76%|███████▌  | 170322/225000 [326:49:00<89:33:03,  5.90s/it]
 76%|███████▌  | 170323/225000 [326:49:06<92:10:49,  6.07s/it]
 76%|███████▌  | 170324/225000 [326:49:11<84:00:57,  5.53s/it]
 76%|███████▌  | 170325/225000 [326:49:15<79:38:57,  5.24s/it]
 76%|███████▌  | 170326/225000 [326:49:25<101:22:56,  6.68s/it]
 76%|███████▌  | 170327/225000 [326:49:29<85:57:29,  5.66s/it] 
 76%|███████▌  | 170328/225000 [326:49:36<96:03:46,  6.33s/it]
 76%|███████▌  | 170329/225000 [326:49:41<87:59:01,  5.79s/it]
 76%|███████▌  | 170330/225000 [326:49:46<83:23:05,  5.49s/it]
 76%|███████▌  | 170331/225000 [326:49:51<83:35:04,  5.50s/it]
 76%|███████▌  | 170332/225000 [326:49:55<74:28:57,  4.90s/it]
 76%|███████▌  | 170333/225000 [326:50:04<92:30:25,  6.09s/it]
 76%|███████▌  | 170334/225000 [326:50:08<82:25:15,  5.43s/it]
 76%|███████▌  | 170335/225000 [326:50:11<73:20:58,  4.83s/it]
 76%|███████▌  | 170336/225000 [326:50:17<77:24:34,  5.10s/it]
 76%|███████▌  | 170337/225000 [326:50:26<96:30:51,  6.36s/it]
 76%|███████▌  | 170338/225000 [326:50:29<83:14:52,  5.48s/it]
 76%|███████▌  | 170339/225000 [326:50:36<90:04:02,  5.93s/it]
 76%|███████▌  | 170340/225000 [326:50:42<86:55:10,  5.72s/it]
 76%|███████▌  | 170341/225000 [326:50:47<85:38:21,  5.64s/it]
 76%|███████▌  | 170342/225000 [326:50:52<80:08:53,  5.28s/it]
 76%|███████▌  | 170343/225000 [326:50:56<77:45:44,  5.12s/it]
 76%|███████▌  | 170344/225000 [326:51:01<75:29:32,  4.97s/it]
 76%|███████▌  | 170345/225000 [326:51:06<77:22:50,  5.10s/it]
 76%|███████▌  | 170346/225000 [326:51:11<77:35:10,  5.11s/it]
 76%|███████▌  | 170347/225000 [326:51:17<77:27:16,  5.10s/it]
 76%|███████▌  | 170348/225000 [326:51:22<76:45:27,  5.06s/it]
 76%|███████▌  | 170349/225000 [326:51:27<76:31:29,  5.04s/it]
 76%|███████▌  | 170350/225000 [326:51:31<75:42:31,  4.99s/it]
                                                              
{'loss': '0.7671', 'grad_norm': '1.065', 'learning_rate': '7.577e-06', 'epoch': '0.7571', 'num_input_tokens_seen': 541444112, 'train_runtime': '1.177e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170350/225000 [326:51:31<75:42:31,  4.99s/it]
 76%|███████▌  | 170351/225000 [326:51:43<107:02:59,  7.05s/it]
 76%|███████▌  | 170352/225000 [326:51:47<94:10:02,  6.20s/it] 
 76%|███████▌  | 170353/225000 [326:51:52<85:04:03,  5.60s/it]
 76%|███████▌  | 170354/225000 [326:51:57<82:29:04,  5.43s/it]
 76%|███████▌  | 170355/225000 [326:52:02<81:59:28,  5.40s/it]
 76%|███████▌  | 170356/225000 [326:52:08<82:37:17,  5.44s/it]
 76%|███████▌  | 170357/225000 [326:52:12<78:33:56,  5.18s/it]
 76%|███████▌  | 170358/225000 [326:52:17<78:21:23,  5.16s/it]
 76%|███████▌  | 170359/225000 [326:52:23<80:08:26,  5.28s/it]
 76%|███████▌  | 170360/225000 [326:52:28<79:56:50,  5.27s/it]
 76%|███████▌  | 170361/225000 [326:52:33<79:40:36,  5.25s/it]
 76%|███████▌  | 170362/225000 [326:52:39<81:00:53,  5.34s/it]
 76%|███████▌  | 170363/225000 [326:52:44<81:06:52,  5.34s/it]
 76%|███████▌  | 170364/225000 [326:52:50<81:44:48,  5.39s/it]
 76%|███████▌  | 170365/225000 [326:52:55<82:22:12,  5.43s/it]
 76%|███████▌  | 170366/225000 [326:53:01<84:04:16,  5.54s/it]
 76%|███████▌  | 170367/225000 [326:53:06<82:36:48,  5.44s/it]
 76%|███████▌  | 170368/225000 [326:53:12<83:15:01,  5.49s/it]
 76%|███████▌  | 170369/225000 [326:53:17<81:45:31,  5.39s/it]
 76%|███████▌  | 170370/225000 [326:53:22<81:47:38,  5.39s/it]
 76%|███████▌  | 170371/225000 [326:53:28<81:28:56,  5.37s/it]
 76%|███████▌  | 170372/225000 [326:53:33<79:19:19,  5.23s/it]
 76%|███████▌  | 170373/225000 [326:53:38<79:13:25,  5.22s/it]
 76%|███████▌  | 170374/225000 [326:53:43<80:09:23,  5.28s/it]
 76%|███████▌  | 170375/225000 [326:53:56<114:30:03,  7.55s/it]
 76%|███████▌  | 170376/225000 [326:54:01<100:59:16,  6.66s/it]
 76%|███████▌  | 170377/225000 [326:54:06<93:24:32,  6.16s/it] 
 76%|███████▌  | 170378/225000 [326:54:11<91:10:35,  6.01s/it]
 76%|███████▌  | 170379/225000 [326:54:15<82:55:06,  5.47s/it]
 76%|███████▌  | 170380/225000 [326:54:23<91:27:27,  6.03s/it]
 76%|███████▌  | 170381/225000 [326:54:28<86:06:41,  5.68s/it]
 76%|███████▌  | 170382/225000 [326:54:32<81:03:13,  5.34s/it]
 76%|███████▌  | 170383/225000 [326:54:37<77:36:31,  5.12s/it]
 76%|███████▌  | 170384/225000 [326:54:41<75:27:41,  4.97s/it]
 76%|███████▌  | 170385/225000 [326:54:45<70:56:01,  4.68s/it]
 76%|███████▌  | 170386/225000 [326:54:50<71:18:09,  4.70s/it]
 76%|███████▌  | 170387/225000 [326:54:57<79:22:47,  5.23s/it]
 76%|███████▌  | 170388/225000 [326:55:08<107:38:28,  7.10s/it]
 76%|███████▌  | 170389/225000 [326:55:12<93:19:49,  6.15s/it] 
 76%|███████▌  | 170390/225000 [326:55:18<93:42:24,  6.18s/it]
 76%|███████▌  | 170391/225000 [326:55:24<90:43:40,  5.98s/it]
 76%|███████▌  | 170392/225000 [326:55:29<85:44:15,  5.65s/it]
 76%|███████▌  | 170393/225000 [326:55:33<79:15:14,  5.22s/it]
 76%|███████▌  | 170394/225000 [326:55:37<75:14:14,  4.96s/it]
 76%|███████▌  | 170395/225000 [326:55:41<70:56:42,  4.68s/it]
 76%|███████▌  | 170396/225000 [326:55:45<68:07:36,  4.49s/it]
 76%|███████▌  | 170397/225000 [326:55:57<102:47:45,  6.78s/it]
 76%|███████▌  | 170398/225000 [326:56:01<87:26:14,  5.76s/it] 
 76%|███████▌  | 170399/225000 [326:56:06<86:36:26,  5.71s/it]
 76%|███████▌  | 170400/225000 [326:56:12<85:54:20,  5.66s/it]
                                                              
{'loss': '0.7637', 'grad_norm': '1.131', 'learning_rate': '7.564e-06', 'epoch': '0.7573', 'num_input_tokens_seen': 541574072, 'train_runtime': '1.177e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170400/225000 [326:56:12<85:54:20,  5.66s/it]
 76%|███████▌  | 170401/225000 [326:56:18<85:40:39,  5.65s/it]
 76%|███████▌  | 170402/225000 [326:56:23<83:57:13,  5.54s/it]
 76%|███████▌  | 170403/225000 [326:56:29<88:09:21,  5.81s/it]
 76%|███████▌  | 170404/225000 [326:56:34<82:31:39,  5.44s/it]
 76%|███████▌  | 170405/225000 [326:56:38<77:58:39,  5.14s/it]
 76%|███████▌  | 170406/225000 [326:56:43<75:02:12,  4.95s/it]
 76%|███████▌  | 170407/225000 [326:56:47<72:40:50,  4.79s/it]
 76%|███████▌  | 170408/225000 [326:56:51<69:42:56,  4.60s/it]
 76%|███████▌  | 170409/225000 [326:56:56<68:40:19,  4.53s/it]
 76%|███████▌  | 170410/225000 [326:57:07<99:41:48,  6.57s/it]
 76%|███████▌  | 170411/225000 [326:57:11<86:00:54,  5.67s/it]
 76%|███████▌  | 170412/225000 [326:57:15<82:00:54,  5.41s/it]
 76%|███████▌  | 170413/225000 [326:57:20<78:00:42,  5.14s/it]
 76%|███████▌  | 170414/225000 [326:57:25<76:43:39,  5.06s/it]
 76%|███████▌  | 170415/225000 [326:57:30<76:12:11,  5.03s/it]
 76%|███████▌  | 170416/225000 [326:57:35<74:57:04,  4.94s/it]
 76%|███████▌  | 170417/225000 [326:57:40<79:09:47,  5.22s/it]
 76%|███████▌  | 170418/225000 [326:57:51<103:28:45,  6.83s/it]
 76%|███████▌  | 170419/225000 [326:57:54<87:22:21,  5.76s/it] 
 76%|███████▌  | 170420/225000 [326:58:01<91:21:01,  6.03s/it]
 76%|███████▌  | 170421/225000 [326:58:07<90:27:44,  5.97s/it]
 76%|███████▌  | 170422/225000 [326:58:10<80:18:10,  5.30s/it]
 76%|███████▌  | 170423/225000 [326:58:20<99:13:32,  6.55s/it]
 76%|███████▌  | 170424/225000 [326:58:24<89:50:53,  5.93s/it]
 76%|███████▌  | 170425/225000 [326:58:29<82:51:18,  5.47s/it]
 76%|███████▌  | 170426/225000 [326:58:33<75:46:45,  5.00s/it]
 76%|███████▌  | 170427/225000 [326:58:37<72:52:17,  4.81s/it]
 76%|███████▌  | 170428/225000 [326:58:45<86:45:06,  5.72s/it]
 76%|███████▌  | 170429/225000 [326:58:51<90:17:41,  5.96s/it]
 76%|███████▌  | 170430/225000 [326:58:55<79:12:13,  5.23s/it]
 76%|███████▌  | 170431/225000 [326:59:01<81:43:13,  5.39s/it]
 76%|███████▌  | 170432/225000 [326:59:10<98:40:53,  6.51s/it]
 76%|███████▌  | 170433/225000 [326:59:14<87:37:32,  5.78s/it]
 76%|███████▌  | 170434/225000 [326:59:18<80:50:17,  5.33s/it]
 76%|███████▌  | 170435/225000 [326:59:23<78:12:36,  5.16s/it]
 76%|███████▌  | 170436/225000 [326:59:27<74:35:52,  4.92s/it]
 76%|███████▌  | 170437/225000 [326:59:32<72:51:30,  4.81s/it]
 76%|███████▌  | 170438/225000 [326:59:37<74:21:32,  4.91s/it]
 76%|███████▌  | 170439/225000 [326:59:42<74:11:46,  4.90s/it]
 76%|███████▌  | 170440/225000 [326:59:47<75:29:57,  4.98s/it]
 76%|███████▌  | 170441/225000 [326:59:52<75:57:14,  5.01s/it]
 76%|███████▌  | 170442/225000 [327:00:04<104:49:53,  6.92s/it]
 76%|███████▌  | 170443/225000 [327:00:10<103:57:40,  6.86s/it]
 76%|███████▌  | 170444/225000 [327:00:15<95:09:26,  6.28s/it] 
 76%|███████▌  | 170445/225000 [327:00:22<97:21:29,  6.42s/it]
 76%|███████▌  | 170446/225000 [327:00:26<86:24:01,  5.70s/it]
 76%|███████▌  | 170447/225000 [327:00:30<79:50:45,  5.27s/it]
 76%|███████▌  | 170448/225000 [327:00:35<75:57:39,  5.01s/it]
 76%|███████▌  | 170449/225000 [327:00:39<71:47:15,  4.74s/it]
 76%|███████▌  | 170450/225000 [327:00:48<94:26:45,  6.23s/it]
                                                              
{'loss': '0.7594', 'grad_norm': '1.185', 'learning_rate': '7.551e-06', 'epoch': '0.7576', 'num_input_tokens_seen': 541703792, 'train_runtime': '1.177e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170450/225000 [327:00:48<94:26:45,  6.23s/it]
 76%|███████▌  | 170451/225000 [327:00:53<88:14:19,  5.82s/it]
 76%|███████▌  | 170452/225000 [327:00:58<82:32:23,  5.45s/it]
 76%|███████▌  | 170453/225000 [327:01:03<81:29:29,  5.38s/it]
 76%|███████▌  | 170454/225000 [327:01:08<81:04:33,  5.35s/it]
 76%|███████▌  | 170455/225000 [327:01:13<79:22:24,  5.24s/it]
 76%|███████▌  | 170456/225000 [327:01:18<77:56:39,  5.14s/it]
 76%|███████▌  | 170457/225000 [327:01:23<74:30:24,  4.92s/it]
 76%|███████▌  | 170458/225000 [327:01:28<74:34:32,  4.92s/it]
 76%|███████▌  | 170459/225000 [327:01:33<74:46:56,  4.94s/it]
 76%|███████▌  | 170460/225000 [327:01:38<75:38:41,  4.99s/it]
 76%|███████▌  | 170461/225000 [327:01:43<76:29:23,  5.05s/it]
 76%|███████▌  | 170462/225000 [327:01:48<76:41:39,  5.06s/it]
 76%|███████▌  | 170463/225000 [327:01:53<77:27:39,  5.11s/it]
 76%|███████▌  | 170464/225000 [327:01:58<77:49:08,  5.14s/it]
 76%|███████▌  | 170465/225000 [327:02:04<78:21:11,  5.17s/it]
 76%|███████▌  | 170466/225000 [327:02:09<79:36:22,  5.26s/it]
 76%|███████▌  | 170467/225000 [327:02:17<90:40:09,  5.99s/it]
 76%|███████▌  | 170468/225000 [327:02:29<118:19:47,  7.81s/it]
 76%|███████▌  | 170469/225000 [327:02:32<98:32:16,  6.51s/it] 
 76%|███████▌  | 170470/225000 [327:02:39<98:08:23,  6.48s/it]
 76%|███████▌  | 170471/225000 [327:02:44<94:40:12,  6.25s/it]
 76%|███████▌  | 170472/225000 [327:02:48<83:38:05,  5.52s/it]
 76%|███████▌  | 170473/225000 [327:02:57<96:22:54,  6.36s/it]
 76%|███████▌  | 170474/225000 [327:03:00<82:45:52,  5.46s/it]
 76%|███████▌  | 170475/225000 [327:03:06<84:27:08,  5.58s/it]
 76%|███████▌  | 170476/225000 [327:03:11<82:47:59,  5.47s/it]
 76%|███████▌  | 170477/225000 [327:03:15<74:54:46,  4.95s/it]
 76%|███████▌  | 170478/225000 [327:03:24<95:17:06,  6.29s/it]
 76%|███████▌  | 170479/225000 [327:03:28<82:15:05,  5.43s/it]
 76%|███████▌  | 170480/225000 [327:03:33<81:20:32,  5.37s/it]
 76%|███████▌  | 170481/225000 [327:03:40<89:01:53,  5.88s/it]
 76%|███████▌  | 170482/225000 [327:03:44<80:31:15,  5.32s/it]
 76%|███████▌  | 170483/225000 [327:03:48<76:56:08,  5.08s/it]
 76%|███████▌  | 170484/225000 [327:03:57<92:47:30,  6.13s/it]
 76%|███████▌  | 170485/225000 [327:04:00<80:15:11,  5.30s/it]
 76%|███████▌  | 170486/225000 [327:04:08<88:44:15,  5.86s/it]
 76%|███████▌  | 170487/225000 [327:04:12<80:19:31,  5.30s/it]
 76%|███████▌  | 170488/225000 [327:04:16<75:54:58,  5.01s/it]
 76%|███████▌  | 170489/225000 [327:04:20<72:25:44,  4.78s/it]
 76%|███████▌  | 170490/225000 [327:04:24<68:51:34,  4.55s/it]
 76%|███████▌  | 170491/225000 [327:04:29<70:43:04,  4.67s/it]
 76%|███████▌  | 170492/225000 [327:04:34<72:33:40,  4.79s/it]
 76%|███████▌  | 170493/225000 [327:04:40<75:09:50,  4.96s/it]
 76%|███████▌  | 170494/225000 [327:04:45<77:24:24,  5.11s/it]
 76%|███████▌  | 170495/225000 [327:04:51<80:11:47,  5.30s/it]
 76%|███████▌  | 170496/225000 [327:04:56<79:35:46,  5.26s/it]
 76%|███████▌  | 170497/225000 [327:05:01<78:32:49,  5.19s/it]
 76%|███████▌  | 170498/225000 [327:05:06<78:37:05,  5.19s/it]
 76%|███████▌  | 170499/225000 [327:05:12<79:28:34,  5.25s/it]
 76%|███████▌  | 170500/225000 [327:05:22<105:01:39,  6.94s/it]
                                                               
{'loss': '0.7675', 'grad_norm': '1.245', 'learning_rate': '7.538e-06', 'epoch': '0.7578', 'num_input_tokens_seen': 541833672, 'train_runtime': '1.178e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170500/225000 [327:05:22<105:01:39,  6.94s/it]
 76%|███████▌  | 170501/225000 [327:05:28<96:59:57,  6.41s/it] 
 76%|███████▌  | 170502/225000 [327:05:34<97:48:28,  6.46s/it]
 76%|███████▌  | 170503/225000 [327:05:38<86:24:09,  5.71s/it]
 76%|███████▌  | 170504/225000 [327:05:41<75:03:59,  4.96s/it]
 76%|███████▌  | 170505/225000 [327:05:51<96:54:59,  6.40s/it]
 76%|███████▌  | 170506/225000 [327:05:55<84:17:15,  5.57s/it]
 76%|███████▌  | 170507/225000 [327:06:00<81:04:53,  5.36s/it]
 76%|███████▌  | 170508/225000 [327:06:05<79:43:47,  5.27s/it]
 76%|███████▌  | 170509/225000 [327:06:09<75:46:05,  5.01s/it]
 76%|███████▌  | 170510/225000 [327:06:14<76:19:50,  5.04s/it]
 76%|███████▌  | 170511/225000 [327:06:19<76:52:03,  5.08s/it]
 76%|███████▌  | 170512/225000 [327:06:24<75:49:49,  5.01s/it]
 76%|███████▌  | 170513/225000 [327:06:29<76:17:58,  5.04s/it]
 76%|███████▌  | 170514/225000 [327:06:34<75:04:58,  4.96s/it]
 76%|███████▌  | 170515/225000 [327:06:39<74:55:14,  4.95s/it]
 76%|███████▌  | 170516/225000 [327:06:44<76:58:06,  5.09s/it]
 76%|███████▌  | 170517/225000 [327:06:57<109:28:21,  7.23s/it]
 76%|███████▌  | 170518/225000 [327:07:01<98:33:16,  6.51s/it] 
 76%|███████▌  | 170519/225000 [327:07:05<85:07:43,  5.63s/it]
 76%|███████▌  | 170520/225000 [327:07:13<93:53:33,  6.20s/it]
 76%|███████▌  | 170521/225000 [327:07:17<88:09:26,  5.83s/it]
 76%|███████▌  | 170522/225000 [327:07:22<81:45:50,  5.40s/it]
 76%|███████▌  | 170523/225000 [327:07:30<94:04:06,  6.22s/it]
 76%|███████▌  | 170524/225000 [327:07:37<97:23:43,  6.44s/it]
 76%|███████▌  | 170525/225000 [327:07:41<84:12:57,  5.57s/it]
 76%|███████▌  | 170526/225000 [327:07:44<74:06:07,  4.90s/it]
 76%|███████▌  | 170527/225000 [327:07:52<87:32:40,  5.79s/it]
 76%|███████▌  | 170528/225000 [327:07:55<76:24:27,  5.05s/it]
 76%|███████▌  | 170529/225000 [327:08:02<86:34:29,  5.72s/it]
 76%|███████▌  | 170530/225000 [327:08:06<76:39:12,  5.07s/it]
 76%|███████▌  | 170531/225000 [327:08:10<71:29:07,  4.72s/it]
 76%|███████▌  | 170532/225000 [327:08:19<93:51:09,  6.20s/it]
 76%|███████▌  | 170533/225000 [327:08:23<82:33:52,  5.46s/it]
 76%|███████▌  | 170534/225000 [327:08:31<95:33:19,  6.32s/it]
 76%|███████▌  | 170535/225000 [327:08:36<88:20:03,  5.84s/it]
 76%|███████▌  | 170536/225000 [327:08:40<78:00:51,  5.16s/it]
 76%|███████▌  | 170537/225000 [327:08:48<92:59:42,  6.15s/it]
 76%|███████▌  | 170538/225000 [327:08:52<82:01:13,  5.42s/it]
 76%|███████▌  | 170539/225000 [327:08:56<74:27:06,  4.92s/it]
 76%|███████▌  | 170540/225000 [327:09:00<71:46:25,  4.74s/it]
 76%|███████▌  | 170541/225000 [327:09:05<70:49:27,  4.68s/it]
 76%|███████▌  | 170542/225000 [327:09:09<70:32:23,  4.66s/it]
 76%|███████▌  | 170543/225000 [327:09:14<70:23:17,  4.65s/it]
 76%|███████▌  | 170544/225000 [327:09:18<69:23:49,  4.59s/it]
 76%|███████▌  | 170545/225000 [327:09:30<102:01:27,  6.74s/it]
 76%|███████▌  | 170546/225000 [327:09:35<95:52:21,  6.34s/it] 
 76%|███████▌  | 170547/225000 [327:09:39<82:36:14,  5.46s/it]
 76%|███████▌  | 170548/225000 [327:09:46<91:38:41,  6.06s/it]
 76%|███████▌  | 170549/225000 [327:09:52<88:42:53,  5.87s/it]
 76%|███████▌  | 170550/225000 [327:09:56<80:44:35,  5.34s/it]
                                                              
{'loss': '0.7604', 'grad_norm': '1.151', 'learning_rate': '7.525e-06', 'epoch': '0.758', 'num_input_tokens_seen': 541962720, 'train_runtime': '1.178e+06', 'train_tokens_per_second': '460.1'}

 76%|███████▌  | 170550/225000 [327:09:56<80:44:35,  5.34s/it]
 76%|███████▌  | 170551/225000 [327:10:00<75:12:43,  4.97s/it]
 76%|███████▌  | 170552/225000 [327:10:08<90:44:17,  6.00s/it]
 76%|███████▌  | 170553/225000 [327:10:15<93:39:02,  6.19s/it]
 76%|███████▌  | 170554/225000 [327:10:19<85:26:37,  5.65s/it]
 76%|███████▌  | 170555/225000 [327:10:24<80:52:28,  5.35s/it]
 76%|███████▌  | 170556/225000 [327:10:28<74:44:35,  4.94s/it]
 76%|███████▌  | 170557/225000 [327:10:33<73:20:06,  4.85s/it]
 76%|███████▌  | 170558/225000 [327:10:42<95:21:10,  6.31s/it]
 76%|███████▌  | 170559/225000 [327:10:47<90:00:53,  5.95s/it]
 76%|███████▌  | 170560/225000 [327:10:52<83:03:46,  5.49s/it]
 76%|███████▌  | 170561/225000 [327:11:00<95:13:42,  6.30s/it]
 76%|███████▌  | 170562/225000 [327:11:05<89:41:30,  5.93s/it]
 76%|███████▌  | 170563/225000 [327:11:10<83:48:01,  5.54s/it]
 76%|███████▌  | 170564/225000 [327:11:14<78:00:16,  5.16s/it]
 76%|███████▌  | 170565/225000 [327:11:18<73:28:28,  4.86s/it]
 76%|███████▌  | 170566/225000 [327:11:23<71:29:43,  4.73s/it]
 76%|███████▌  | 170567/225000 [327:11:35<105:23:20,  6.97s/it]
 76%|███████▌  | 170568/225000 [327:11:40<95:31:23,  6.32s/it] 
 76%|███████▌  | 170569/225000 [327:11:44<87:31:44,  5.79s/it]
 76%|███████▌  | 170570/225000 [327:11:49<82:39:22,  5.47s/it]
 76%|███████▌  | 170571/225000 [327:11:54<79:11:18,  5.24s/it]
 76%|███████▌  | 170572/225000 [327:11:58<75:56:18,  5.02s/it]
 76%|███████▌  | 170573/225000 [327:12:02<72:17:24,  4.78s/it]
 76%|███████▌  | 170574/225000 [327:12:07<72:16:36,  4.78s/it]
 76%|███████▌  | 170575/225000 [327:12:12<72:57:41,  4.83s/it]
 76%|███████▌  | 170576/225000 [327:12:17<71:33:10,  4.73s/it]
 76%|███████▌  | 170577/225000 [327:12:22<73:45:02,  4.88s/it]
 76%|███████▌  | 170578/225000 [327:12:27<73:55:32,  4.89s/it]
 76%|███████▌  | 170579/225000 [327:12:37<100:45:11,  6.66s/it]
 76%|███████▌  | 170580/225000 [327:12:45<105:24:38,  6.97s/it]
 76%|███████▌  | 170581/225000 [327:12:49<92:19:40,  6.11s/it] 
 76%|███████▌  | 170582/225000 [327:12:53<81:31:17,  5.39s/it]
 76%|███████▌  | 170583/225000 [327:12:58<80:49:31,  5.35s/it]
 76%|███████▌  | 170584/225000 [327:13:07<96:16:15,  6.37s/it]
 76%|███████▌  | 170585/225000 [327:13:11<85:01:52,  5.63s/it]
 76%|███████▌  | 170586/225000 [327:13:14<75:29:16,  4.99s/it]
 76%|███████▌  | 170587/225000 [327:13:18<69:38:02,  4.61s/it]
 76%|███████▌  | 170588/225000 [327:13:28<91:28:15,  6.05s/it]
 76%|███████▌  | 170589/225000 [327:13:32<85:37:36,  5.67s/it]
 76%|███████▌  | 170590/225000 [327:13:40<93:50:30,  6.21s/it]
 76%|███████▌  | 170591/225000 [327:13:43<81:55:41,  5.42s/it]
 76%|███████▌  | 170592/225000 [327:13:48<77:58:39,  5.16s/it]
 76%|███████▌  | 170593/225000 [327:13:56<90:35:21,  5.99s/it]
 76%|███████▌  | 170594/225000 [327:13:59<76:46:13,  5.08s/it]
 76%|███████▌  | 170595/225000 [327:14:07<90:12:52,  5.97s/it]
 76%|███████▌  | 170596/225000 [327:14:11<79:47:57,  5.28s/it]
 76%|███████▌  | 170597/225000 [327:14:14<73:12:43,  4.84s/it]
 76%|███████▌  | 170598/225000 [327:14:23<90:47:11,  6.01s/it]
 76%|███████▌  | 170599/225000 [327:14:28<84:05:17,  5.56s/it]
 76%|███████▌  | 170600/225000 [327:14:31<75:49:38,  5.02s/it]
                                                              
{'loss': '0.7756', 'grad_norm': '1.169', 'learning_rate': '7.512e-06', 'epoch': '0.7582', 'num_input_tokens_seen': 542092368, 'train_runtime': '1.178e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170600/225000 [327:14:31<75:49:38,  5.02s/it]
 76%|███████▌  | 170601/225000 [327:14:37<77:20:36,  5.12s/it]
 76%|███████▌  | 170602/225000 [327:14:43<84:17:20,  5.58s/it]
 76%|███████▌  | 170603/225000 [327:14:48<80:20:26,  5.32s/it]
 76%|███████▌  | 170604/225000 [327:14:53<77:53:02,  5.15s/it]
 76%|███████▌  | 170605/225000 [327:14:57<71:16:17,  4.72s/it]
 76%|███████▌  | 170606/225000 [327:15:01<69:54:07,  4.63s/it]
 76%|███████▌  | 170607/225000 [327:15:05<69:37:00,  4.61s/it]
 76%|███████▌  | 170608/225000 [327:15:10<68:34:37,  4.54s/it]
 76%|███████▌  | 170609/225000 [327:15:17<80:53:47,  5.35s/it]
 76%|███████▌  | 170610/225000 [327:15:26<97:28:31,  6.45s/it]
 76%|███████▌  | 170611/225000 [327:15:30<84:36:22,  5.60s/it]
 76%|███████▌  | 170612/225000 [327:15:33<76:02:22,  5.03s/it]
 76%|███████▌  | 170613/225000 [327:15:42<93:11:12,  6.17s/it]
 76%|███████▌  | 170614/225000 [327:15:48<91:14:13,  6.04s/it]
 76%|███████▌  | 170615/225000 [327:15:52<82:59:39,  5.49s/it]
 76%|███████▌  | 170616/225000 [327:15:57<78:44:37,  5.21s/it]
 76%|███████▌  | 170617/225000 [327:16:01<73:49:40,  4.89s/it]
 76%|███████▌  | 170618/225000 [327:16:05<72:01:09,  4.77s/it]
 76%|███████▌  | 170619/225000 [327:16:16<97:55:36,  6.48s/it]
 76%|███████▌  | 170620/225000 [327:16:20<88:57:39,  5.89s/it]
 76%|███████▌  | 170621/225000 [327:16:25<80:56:03,  5.36s/it]
 76%|███████▌  | 170622/225000 [327:16:33<96:55:38,  6.42s/it]
 76%|███████▌  | 170623/225000 [327:16:37<84:00:42,  5.56s/it]
 76%|███████▌  | 170624/225000 [327:16:41<75:57:35,  5.03s/it]
 76%|███████▌  | 170625/225000 [327:16:45<72:38:38,  4.81s/it]
 76%|███████▌  | 170626/225000 [327:16:53<86:31:28,  5.73s/it]
 76%|███████▌  | 170627/225000 [327:17:00<92:30:27,  6.12s/it]
 76%|███████▌  | 170628/225000 [327:17:04<84:47:53,  5.61s/it]
 76%|███████▌  | 170629/225000 [327:17:09<81:46:42,  5.41s/it]
 76%|███████▌  | 170630/225000 [327:17:14<77:06:06,  5.11s/it]
 76%|███████▌  | 170631/225000 [327:17:19<75:54:14,  5.03s/it]
 76%|███████▌  | 170632/225000 [327:17:24<75:25:28,  4.99s/it]
 76%|███████▌  | 170633/225000 [327:17:29<76:04:05,  5.04s/it]
 76%|███████▌  | 170634/225000 [327:17:33<75:15:17,  4.98s/it]
 76%|███████▌  | 170635/225000 [327:17:38<74:11:48,  4.91s/it]
 76%|███████▌  | 170636/225000 [327:17:43<75:35:20,  5.01s/it]
 76%|███████▌  | 170637/225000 [327:17:48<75:01:19,  4.97s/it]
 76%|███████▌  | 170638/225000 [327:18:01<110:25:34,  7.31s/it]
 76%|███████▌  | 170639/225000 [327:18:06<97:17:50,  6.44s/it] 
 76%|███████▌  | 170640/225000 [327:18:09<83:39:31,  5.54s/it]
 76%|███████▌  | 170641/225000 [327:18:17<95:52:28,  6.35s/it]
 76%|███████▌  | 170642/225000 [327:18:21<84:37:09,  5.60s/it]
 76%|███████▌  | 170643/225000 [327:18:25<76:45:28,  5.08s/it]
 76%|███████▌  | 170644/225000 [327:18:29<71:46:45,  4.75s/it]
 76%|███████▌  | 170645/225000 [327:18:40<101:13:05,  6.70s/it]
 76%|███████▌  | 170646/225000 [327:18:44<87:46:59,  5.81s/it] 
 76%|███████▌  | 170647/225000 [327:18:48<77:40:00,  5.14s/it]
 76%|███████▌  | 170648/225000 [327:18:57<97:42:47,  6.47s/it]
 76%|███████▌  | 170649/225000 [327:19:01<86:24:22,  5.72s/it]
 76%|███████▌  | 170650/225000 [327:19:05<76:40:18,  5.08s/it]
                                                              
{'loss': '0.7733', 'grad_norm': '1.153', 'learning_rate': '7.499e-06', 'epoch': '0.7584', 'num_input_tokens_seen': 542221760, 'train_runtime': '1.178e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170650/225000 [327:19:05<76:40:18,  5.08s/it]
 76%|███████▌  | 170651/225000 [327:19:10<79:25:17,  5.26s/it]
 76%|███████▌  | 170652/225000 [327:19:18<89:27:28,  5.93s/it]
 76%|███████▌  | 170653/225000 [327:19:22<81:50:47,  5.42s/it]
 76%|███████▌  | 170654/225000 [327:19:26<76:39:42,  5.08s/it]
 76%|███████▌  | 170655/225000 [327:19:31<73:10:36,  4.85s/it]
 76%|███████▌  | 170656/225000 [327:19:38<84:19:16,  5.59s/it]
 76%|███████▌  | 170657/225000 [327:19:46<93:50:33,  6.22s/it]
 76%|███████▌  | 170658/225000 [327:19:49<82:04:47,  5.44s/it]
 76%|███████▌  | 170659/225000 [327:19:53<74:47:58,  4.96s/it]
 76%|███████▌  | 170660/225000 [327:19:59<77:55:08,  5.16s/it]
 76%|███████▌  | 170661/225000 [327:20:07<90:24:32,  5.99s/it]
 76%|███████▌  | 170662/225000 [327:20:13<93:09:55,  6.17s/it]
 76%|███████▌  | 170663/225000 [327:20:20<93:58:43,  6.23s/it]
 76%|███████▌  | 170664/225000 [327:20:25<91:16:24,  6.05s/it]
 76%|███████▌  | 170665/225000 [327:20:33<98:33:22,  6.53s/it]
 76%|███████▌  | 170666/225000 [327:20:36<84:18:55,  5.59s/it]
 76%|███████▌  | 170667/225000 [327:20:41<80:23:59,  5.33s/it]
 76%|███████▌  | 170668/225000 [327:20:47<82:32:31,  5.47s/it]
 76%|███████▌  | 170669/225000 [327:20:53<86:03:02,  5.70s/it]
 76%|███████▌  | 170670/225000 [327:20:59<85:49:11,  5.69s/it]
 76%|███████▌  | 170671/225000 [327:21:04<82:35:59,  5.47s/it]
 76%|███████▌  | 170672/225000 [327:21:09<80:13:14,  5.32s/it]
 76%|███████▌  | 170673/225000 [327:21:14<78:47:18,  5.22s/it]
 76%|███████▌  | 170674/225000 [327:21:18<76:33:58,  5.07s/it]
 76%|███████▌  | 170675/225000 [327:21:23<73:45:29,  4.89s/it]
 76%|███████▌  | 170676/225000 [327:21:27<69:44:32,  4.62s/it]
 76%|███████▌  | 170677/225000 [327:21:31<69:40:52,  4.62s/it]
 76%|███████▌  | 170678/225000 [327:21:37<72:36:15,  4.81s/it]
 76%|███████▌  | 170679/225000 [327:21:42<73:53:08,  4.90s/it]
 76%|███████▌  | 170680/225000 [327:21:47<75:02:19,  4.97s/it]
 76%|███████▌  | 170681/225000 [327:21:52<74:06:15,  4.91s/it]
 76%|███████▌  | 170682/225000 [327:22:03<104:46:13,  6.94s/it]
 76%|███████▌  | 170683/225000 [327:22:09<100:34:18,  6.67s/it]
 76%|███████▌  | 170684/225000 [327:22:14<92:16:20,  6.12s/it] 
 76%|███████▌  | 170685/225000 [327:22:21<94:15:57,  6.25s/it]
 76%|███████▌  | 170686/225000 [327:22:25<84:25:12,  5.60s/it]
 76%|███████▌  | 170687/225000 [327:22:29<76:58:05,  5.10s/it]
 76%|███████▌  | 170688/225000 [327:22:33<74:10:41,  4.92s/it]
 76%|███████▌  | 170689/225000 [327:22:37<70:32:04,  4.68s/it]
 76%|███████▌  | 170690/225000 [327:22:49<104:05:43,  6.90s/it]
 76%|███████▌  | 170691/225000 [327:22:53<89:27:43,  5.93s/it] 
 76%|███████▌  | 170692/225000 [327:22:57<79:19:14,  5.26s/it]
 76%|███████▌  | 170693/225000 [327:23:06<95:29:30,  6.33s/it]
 76%|███████▌  | 170694/225000 [327:23:09<82:30:29,  5.47s/it]
 76%|███████▌  | 170695/225000 [327:23:16<88:28:30,  5.87s/it]
 76%|███████▌  | 170696/225000 [327:23:19<77:42:03,  5.15s/it]
 76%|███████▌  | 170697/225000 [327:23:26<83:24:45,  5.53s/it]
 76%|███████▌  | 170698/225000 [327:23:31<83:29:17,  5.53s/it]
 76%|███████▌  | 170699/225000 [327:23:37<84:24:15,  5.60s/it]
 76%|███████▌  | 170700/225000 [327:23:41<76:02:25,  5.04s/it]
                                                              
{'loss': '0.779', 'grad_norm': '1.038', 'learning_rate': '7.486e-06', 'epoch': '0.7587', 'num_input_tokens_seen': 542352696, 'train_runtime': '1.179e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170700/225000 [327:23:41<76:02:25,  5.04s/it]
 76%|███████▌  | 170701/225000 [327:23:45<70:30:32,  4.67s/it]
 76%|███████▌  | 170702/225000 [327:23:55<94:25:32,  6.26s/it]
 76%|███████▌  | 170703/225000 [327:23:59<86:43:19,  5.75s/it]
 76%|███████▌  | 170704/225000 [327:24:04<81:57:20,  5.43s/it]
 76%|███████▌  | 170705/225000 [327:24:08<76:21:48,  5.06s/it]
 76%|███████▌  | 170706/225000 [327:24:12<71:30:04,  4.74s/it]
 76%|███████▌  | 170707/225000 [327:24:17<71:15:13,  4.72s/it]
 76%|███████▌  | 170708/225000 [327:24:21<70:54:53,  4.70s/it]
 76%|███████▌  | 170709/225000 [327:24:26<71:29:55,  4.74s/it]
 76%|███████▌  | 170710/225000 [327:24:31<71:28:02,  4.74s/it]
 76%|███████▌  | 170711/225000 [327:24:36<71:48:35,  4.76s/it]
 76%|███████▌  | 170712/225000 [327:24:45<93:39:26,  6.21s/it]
 76%|███████▌  | 170713/225000 [327:24:52<96:40:23,  6.41s/it]
 76%|███████▌  | 170714/225000 [327:24:59<98:37:13,  6.54s/it]
 76%|███████▌  | 170715/225000 [327:25:03<87:53:09,  5.83s/it]
 76%|███████▌  | 170716/225000 [327:25:10<93:30:46,  6.20s/it]
 76%|███████▌  | 170717/225000 [327:25:14<80:38:02,  5.35s/it]
 76%|███████▌  | 170718/225000 [327:25:20<85:44:56,  5.69s/it]
 76%|███████▌  | 170719/225000 [327:25:26<88:37:42,  5.88s/it]
 76%|███████▌  | 170720/225000 [327:25:31<81:05:10,  5.38s/it]
 76%|███████▌  | 170721/225000 [327:25:35<76:06:19,  5.05s/it]
 76%|███████▌  | 170722/225000 [327:25:39<71:47:10,  4.76s/it]
 76%|███████▌  | 170723/225000 [327:25:44<71:15:54,  4.73s/it]
 76%|███████▌  | 170724/225000 [327:25:48<68:29:59,  4.54s/it]
 76%|███████▌  | 170725/225000 [327:25:52<65:50:40,  4.37s/it]
 76%|███████▌  | 170726/225000 [327:26:02<92:19:30,  6.12s/it]
 76%|███████▌  | 170727/225000 [327:26:07<88:25:20,  5.87s/it]
 76%|███████▌  | 170728/225000 [327:26:13<85:52:37,  5.70s/it]
 76%|███████▌  | 170729/225000 [327:26:18<82:37:27,  5.48s/it]
 76%|███████▌  | 170730/225000 [327:26:22<79:02:30,  5.24s/it]
 76%|███████▌  | 170731/225000 [327:26:27<77:01:54,  5.11s/it]
 76%|███████▌  | 170732/225000 [327:26:32<76:26:00,  5.07s/it]
 76%|███████▌  | 170733/225000 [327:26:37<75:01:17,  4.98s/it]
 76%|███████▌  | 170734/225000 [327:26:42<74:53:33,  4.97s/it]
 76%|███████▌  | 170735/225000 [327:26:47<74:39:11,  4.95s/it]
 76%|███████▌  | 170736/225000 [327:26:51<73:10:06,  4.85s/it]
 76%|███████▌  | 170737/225000 [327:27:02<101:30:35,  6.73s/it]
 76%|███████▌  | 170738/225000 [327:27:10<103:40:53,  6.88s/it]
 76%|███████▌  | 170739/225000 [327:27:15<97:10:43,  6.45s/it] 
 76%|███████▌  | 170740/225000 [327:27:21<96:08:23,  6.38s/it]
 76%|███████▌  | 170741/225000 [327:27:26<88:21:33,  5.86s/it]
 76%|███████▌  | 170742/225000 [327:27:31<86:46:17,  5.76s/it]
 76%|███████▌  | 170743/225000 [327:27:35<77:09:04,  5.12s/it]
 76%|███████▌  | 170744/225000 [327:27:44<92:21:12,  6.13s/it]
 76%|███████▌  | 170745/225000 [327:27:48<85:21:41,  5.66s/it]
 76%|███████▌  | 170746/225000 [327:27:53<79:56:25,  5.30s/it]
 76%|███████▌  | 170747/225000 [327:27:57<76:20:00,  5.07s/it]
 76%|███████▌  | 170748/225000 [327:28:02<73:32:24,  4.88s/it]
 76%|███████▌  | 170749/225000 [327:28:06<72:28:41,  4.81s/it]
 76%|███████▌  | 170750/225000 [327:28:11<73:15:51,  4.86s/it]
                                                              
{'loss': '0.7694', 'grad_norm': '1.219', 'learning_rate': '7.473e-06', 'epoch': '0.7589', 'num_input_tokens_seen': 542482024, 'train_runtime': '1.179e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170750/225000 [327:28:11<73:15:51,  4.86s/it]
 76%|███████▌  | 170751/225000 [327:28:16<71:40:06,  4.76s/it]
 76%|███████▌  | 170752/225000 [327:28:24<86:18:09,  5.73s/it]
 76%|███████▌  | 170753/225000 [327:28:33<102:07:01,  6.78s/it]
 76%|███████▌  | 170754/225000 [327:28:37<89:15:38,  5.92s/it] 
 76%|███████▌  | 170755/225000 [327:28:43<88:07:55,  5.85s/it]
 76%|███████▌  | 170756/225000 [327:28:47<79:46:30,  5.29s/it]
 76%|███████▌  | 170757/225000 [327:28:51<74:55:46,  4.97s/it]
 76%|███████▌  | 170758/225000 [327:28:55<71:31:22,  4.75s/it]
 76%|███████▌  | 170759/225000 [327:29:00<71:11:56,  4.73s/it]
 76%|███████▌  | 170760/225000 [327:29:10<96:02:15,  6.37s/it]
 76%|███████▌  | 170761/225000 [327:29:14<83:48:39,  5.56s/it]
 76%|███████▌  | 170762/225000 [327:29:17<73:56:04,  4.91s/it]
 76%|███████▌  | 170763/225000 [327:29:26<91:44:45,  6.09s/it]
 76%|███████▌  | 170764/225000 [327:29:30<82:58:38,  5.51s/it]
 76%|███████▌  | 170765/225000 [327:29:34<75:41:19,  5.02s/it]
 76%|███████▌  | 170766/225000 [327:29:41<84:27:02,  5.61s/it]
 76%|███████▌  | 170767/225000 [327:29:48<89:54:55,  5.97s/it]
 76%|███████▌  | 170768/225000 [327:29:54<92:27:12,  6.14s/it]
 76%|███████▌  | 170769/225000 [327:29:58<80:32:24,  5.35s/it]
 76%|███████▌  | 170770/225000 [327:30:05<89:05:48,  5.91s/it]
 76%|███████▌  | 170771/225000 [327:30:10<83:56:55,  5.57s/it]
 76%|███████▌  | 170772/225000 [327:30:15<82:21:49,  5.47s/it]
 76%|███████▌  | 170773/225000 [327:30:20<80:01:03,  5.31s/it]
 76%|███████▌  | 170774/225000 [327:30:25<79:30:30,  5.28s/it]
 76%|███████▌  | 170775/225000 [327:30:30<77:07:20,  5.12s/it]
 76%|███████▌  | 170776/225000 [327:30:34<74:42:01,  4.96s/it]
 76%|███████▌  | 170777/225000 [327:30:39<74:16:23,  4.93s/it]
 76%|███████▌  | 170778/225000 [327:30:44<75:24:24,  5.01s/it]
 76%|███████▌  | 170779/225000 [327:30:49<75:09:13,  4.99s/it]
 76%|███████▌  | 170780/225000 [327:30:55<76:06:10,  5.05s/it]
 76%|███████▌  | 170781/225000 [327:31:06<104:15:32,  6.92s/it]
 76%|███████▌  | 170782/225000 [327:31:12<99:52:35,  6.63s/it] 
 76%|███████▌  | 170783/225000 [327:31:16<90:19:25,  6.00s/it]
 76%|███████▌  | 170784/225000 [327:31:23<92:55:59,  6.17s/it]
 76%|███████▌  | 170785/225000 [327:31:27<81:31:59,  5.41s/it]
 76%|███████▌  | 170786/225000 [327:31:35<94:20:46,  6.26s/it]
 76%|███████▌  | 170787/225000 [327:31:38<81:54:30,  5.44s/it]
 76%|███████▌  | 170788/225000 [327:31:42<73:23:46,  4.87s/it]
 76%|███████▌  | 170789/225000 [327:31:50<86:19:49,  5.73s/it]
 76%|███████▌  | 170790/225000 [327:31:53<77:00:57,  5.11s/it]
 76%|███████▌  | 170791/225000 [327:31:57<70:05:32,  4.65s/it]
 76%|███████▌  | 170792/225000 [327:32:05<87:21:28,  5.80s/it]
 76%|███████▌  | 170793/225000 [327:32:09<77:58:43,  5.18s/it]
 76%|███████▌  | 170794/225000 [327:32:16<84:37:32,  5.62s/it]
 76%|███████▌  | 170795/225000 [327:32:19<75:12:07,  4.99s/it]
 76%|███████▌  | 170796/225000 [327:32:23<69:05:51,  4.59s/it]
 76%|███████▌  | 170797/225000 [327:32:31<85:24:07,  5.67s/it]
 76%|███████▌  | 170798/225000 [327:32:36<83:36:00,  5.55s/it]
 76%|███████▌  | 170799/225000 [327:32:45<96:58:14,  6.44s/it]
 76%|███████▌  | 170800/225000 [327:32:48<82:05:51,  5.45s/it]
                                                              
{'loss': '0.7743', 'grad_norm': '1.23', 'learning_rate': '7.46e-06', 'epoch': '0.7591', 'num_input_tokens_seen': 542612096, 'train_runtime': '1.179e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170800/225000 [327:32:48<82:05:51,  5.45s/it]
 76%|███████▌  | 170801/225000 [327:32:55<88:20:16,  5.87s/it]
 76%|███████▌  | 170802/225000 [327:32:59<79:28:08,  5.28s/it]
 76%|███████▌  | 170803/225000 [327:33:04<79:33:29,  5.28s/it]
 76%|███████▌  | 170804/225000 [327:33:10<80:57:15,  5.38s/it]
 76%|███████▌  | 170805/225000 [327:33:15<79:00:09,  5.25s/it]
 76%|███████▌  | 170806/225000 [327:33:20<78:47:37,  5.23s/it]
 76%|███████▌  | 170807/225000 [327:33:25<79:05:58,  5.25s/it]
 76%|███████▌  | 170808/225000 [327:33:31<81:27:25,  5.41s/it]
 76%|███████▌  | 170809/225000 [327:33:36<80:05:32,  5.32s/it]
 76%|███████▌  | 170810/225000 [327:33:42<82:37:22,  5.49s/it]
 76%|███████▌  | 170811/225000 [327:33:47<81:07:08,  5.39s/it]
 76%|███████▌  | 170812/225000 [327:33:53<82:26:05,  5.48s/it]
 76%|███████▌  | 170813/225000 [327:33:59<84:24:45,  5.61s/it]
 76%|███████▌  | 170814/225000 [327:34:04<83:53:20,  5.57s/it]
 76%|███████▌  | 170815/225000 [327:34:10<86:20:40,  5.74s/it]
 76%|███████▌  | 170816/225000 [327:34:19<101:31:36,  6.75s/it]
 76%|███████▌  | 170817/225000 [327:34:23<86:04:44,  5.72s/it] 
 76%|███████▌  | 170818/225000 [327:34:27<80:23:13,  5.34s/it]
 76%|███████▌  | 170819/225000 [327:34:32<77:46:08,  5.17s/it]
 76%|███████▌  | 170820/225000 [327:34:37<76:17:43,  5.07s/it]
 76%|███████▌  | 170821/225000 [327:34:42<77:38:00,  5.16s/it]
 76%|███████▌  | 170822/225000 [327:34:47<77:15:04,  5.13s/it]
 76%|███████▌  | 170823/225000 [327:34:53<78:51:06,  5.24s/it]
 76%|███████▌  | 170824/225000 [327:34:58<77:32:55,  5.15s/it]
 76%|███████▌  | 170825/225000 [327:35:02<75:39:05,  5.03s/it]
 76%|███████▌  | 170826/225000 [327:35:07<74:12:40,  4.93s/it]
 76%|███████▌  | 170827/225000 [327:35:12<76:26:04,  5.08s/it]
 76%|███████▌  | 170828/225000 [327:35:17<74:48:33,  4.97s/it]
 76%|███████▌  | 170829/225000 [327:35:22<73:20:54,  4.87s/it]
 76%|███████▌  | 170830/225000 [327:35:27<73:53:59,  4.91s/it]
 76%|███████▌  | 170831/225000 [327:35:32<75:38:21,  5.03s/it]
 76%|███████▌  | 170832/225000 [327:35:37<76:10:20,  5.06s/it]
 76%|███████▌  | 170833/225000 [327:35:42<75:51:19,  5.04s/it]
 76%|███████▌  | 170834/225000 [327:35:47<75:19:18,  5.01s/it]
 76%|███████▌  | 170835/225000 [327:35:55<90:20:32,  6.00s/it]
 76%|███████▌  | 170836/225000 [327:36:05<104:21:32,  6.94s/it]
 76%|███████▌  | 170837/225000 [327:36:11<104:08:28,  6.92s/it]
 76%|███████▌  | 170838/225000 [327:36:16<91:22:44,  6.07s/it] 
 76%|███████▌  | 170839/225000 [327:36:20<84:55:03,  5.64s/it]
 76%|███████▌  | 170840/225000 [327:36:25<80:42:18,  5.36s/it]
 76%|███████▌  | 170841/225000 [327:36:30<77:35:39,  5.16s/it]
 76%|███████▌  | 170842/225000 [327:36:34<74:22:56,  4.94s/it]
 76%|███████▌  | 170843/225000 [327:36:39<74:27:56,  4.95s/it]
 76%|███████▌  | 170844/225000 [327:36:44<74:41:08,  4.96s/it]
 76%|███████▌  | 170845/225000 [327:36:50<79:30:37,  5.29s/it]
 76%|███████▌  | 170846/225000 [327:37:01<106:35:31,  7.09s/it]
 76%|███████▌  | 170847/225000 [327:37:05<92:47:07,  6.17s/it] 
 76%|███████▌  | 170848/225000 [327:37:10<84:07:12,  5.59s/it]
 76%|███████▌  | 170849/225000 [327:37:14<78:11:08,  5.20s/it]
 76%|███████▌  | 170850/225000 [327:37:18<72:40:27,  4.83s/it]
                                                              
{'loss': '0.76', 'grad_norm': '1.262', 'learning_rate': '7.447e-06', 'epoch': '0.7593', 'num_input_tokens_seen': 542742296, 'train_runtime': '1.179e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170850/225000 [327:37:18<72:40:27,  4.83s/it]
 76%|███████▌  | 170851/225000 [327:37:28<97:42:39,  6.50s/it]
 76%|███████▌  | 170852/225000 [327:37:35<96:54:33,  6.44s/it]
 76%|███████▌  | 170853/225000 [327:37:38<85:24:19,  5.68s/it]
 76%|███████▌  | 170854/225000 [327:37:47<99:41:54,  6.63s/it]
 76%|███████▌  | 170855/225000 [327:37:51<84:18:51,  5.61s/it]
 76%|███████▌  | 170856/225000 [327:37:55<78:31:10,  5.22s/it]
 76%|███████▌  | 170857/225000 [327:38:03<92:34:59,  6.16s/it]
 76%|███████▌  | 170858/225000 [327:38:09<88:58:27,  5.92s/it]
 76%|███████▌  | 170859/225000 [327:38:12<79:12:09,  5.27s/it]
 76%|███████▌  | 170860/225000 [327:38:17<75:13:56,  5.00s/it]
 76%|███████▌  | 170861/225000 [327:38:21<72:16:25,  4.81s/it]
 76%|███████▌  | 170862/225000 [327:38:29<85:35:21,  5.69s/it]
 76%|███████▌  | 170863/225000 [327:38:37<95:15:21,  6.33s/it]
 76%|███████▌  | 170864/225000 [327:38:40<83:00:33,  5.52s/it]
 76%|███████▌  | 170865/225000 [327:38:47<88:07:54,  5.86s/it]
 76%|███████▌  | 170866/225000 [327:38:53<88:43:12,  5.90s/it]
 76%|███████▌  | 170867/225000 [327:38:59<89:26:58,  5.95s/it]
 76%|███████▌  | 170868/225000 [327:39:03<80:36:02,  5.36s/it]
 76%|███████▌  | 170869/225000 [327:39:07<76:46:22,  5.11s/it]
 76%|███████▌  | 170870/225000 [327:39:12<75:13:30,  5.00s/it]
 76%|███████▌  | 170871/225000 [327:39:17<72:52:03,  4.85s/it]
 76%|███████▌  | 170872/225000 [327:39:21<72:17:50,  4.81s/it]
 76%|███████▌  | 170873/225000 [327:39:26<69:57:11,  4.65s/it]
 76%|███████▌  | 170874/225000 [327:39:30<69:57:11,  4.65s/it]
 76%|███████▌  | 170875/225000 [327:39:41<95:38:25,  6.36s/it]
 76%|███████▌  | 170876/225000 [327:39:45<84:30:28,  5.62s/it]
 76%|███████▌  | 170877/225000 [327:39:52<91:45:42,  6.10s/it]
 76%|███████▌  | 170878/225000 [327:39:58<90:22:08,  6.01s/it]
 76%|███████▌  | 170879/225000 [327:40:01<78:50:47,  5.24s/it]
 76%|███████▌  | 170880/225000 [327:40:09<93:09:05,  6.20s/it]
 76%|███████▌  | 170881/225000 [327:40:13<80:40:13,  5.37s/it]
 76%|███████▌  | 170882/225000 [327:40:17<73:08:49,  4.87s/it]
 76%|███████▌  | 170883/225000 [327:40:22<75:12:20,  5.00s/it]
 76%|███████▌  | 170884/225000 [327:40:32<96:23:12,  6.41s/it]
 76%|███████▌  | 170885/225000 [327:40:36<85:36:52,  5.70s/it]
 76%|███████▌  | 170886/225000 [327:40:40<78:00:44,  5.19s/it]
 76%|███████▌  | 170887/225000 [327:40:47<87:46:28,  5.84s/it]
 76%|███████▌  | 170888/225000 [327:40:54<90:56:54,  6.05s/it]
 76%|███████▌  | 170889/225000 [327:40:59<87:01:48,  5.79s/it]
 76%|███████▌  | 170890/225000 [327:41:04<84:15:28,  5.61s/it]
 76%|███████▌  | 170891/225000 [327:41:09<83:02:02,  5.52s/it]
 76%|███████▌  | 170892/225000 [327:41:14<80:20:15,  5.35s/it]
 76%|███████▌  | 170893/225000 [327:41:19<78:00:05,  5.19s/it]
 76%|███████▌  | 170894/225000 [327:41:24<77:25:24,  5.15s/it]
 76%|███████▌  | 170895/225000 [327:41:30<78:46:24,  5.24s/it]
 76%|███████▌  | 170896/225000 [327:41:34<76:25:49,  5.09s/it]
 76%|███████▌  | 170897/225000 [327:41:40<78:02:05,  5.19s/it]
 76%|███████▌  | 170898/225000 [327:41:45<79:38:27,  5.30s/it]
 76%|███████▌  | 170899/225000 [327:41:50<79:08:09,  5.27s/it]
 76%|███████▌  | 170900/225000 [327:41:56<79:39:21,  5.30s/it]
                                                              
{'loss': '0.7724', 'grad_norm': '1.185', 'learning_rate': '7.434e-06', 'epoch': '0.7596', 'num_input_tokens_seen': 542874200, 'train_runtime': '1.18e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170900/225000 [327:41:56<79:39:21,  5.30s/it]
 76%|███████▌  | 170901/225000 [327:42:01<79:42:02,  5.30s/it]
 76%|███████▌  | 170902/225000 [327:42:06<78:32:04,  5.23s/it]
 76%|███████▌  | 170903/225000 [327:42:11<78:50:37,  5.25s/it]
 76%|███████▌  | 170904/225000 [327:42:17<78:50:48,  5.25s/it]
 76%|███████▌  | 170905/225000 [327:42:21<75:18:24,  5.01s/it]
 76%|███████▌  | 170906/225000 [327:42:27<77:06:09,  5.13s/it]
 76%|███████▌  | 170907/225000 [327:42:32<76:44:28,  5.11s/it]
 76%|███████▌  | 170908/225000 [327:42:37<77:42:21,  5.17s/it]
 76%|███████▌  | 170909/225000 [327:42:42<75:39:28,  5.04s/it]
 76%|███████▌  | 170910/225000 [327:42:47<76:07:01,  5.07s/it]
 76%|███████▌  | 170911/225000 [327:42:52<78:02:03,  5.19s/it]
 76%|███████▌  | 170912/225000 [327:43:04<105:27:45,  7.02s/it]
 76%|███████▌  | 170913/225000 [327:43:11<109:30:20,  7.29s/it]
 76%|███████▌  | 170914/225000 [327:43:15<91:59:50,  6.12s/it] 
 76%|███████▌  | 170915/225000 [327:43:18<79:32:15,  5.29s/it]
 76%|███████▌  | 170916/225000 [327:43:28<97:40:40,  6.50s/it]
 76%|███████▌  | 170917/225000 [327:43:31<84:28:28,  5.62s/it]
 76%|███████▌  | 170918/225000 [327:43:37<87:19:17,  5.81s/it]
 76%|███████▌  | 170919/225000 [327:43:43<86:16:18,  5.74s/it]
 76%|███████▌  | 170920/225000 [327:43:47<77:31:21,  5.16s/it]
 76%|███████▌  | 170921/225000 [327:43:52<76:29:02,  5.09s/it]
 76%|███████▌  | 170922/225000 [327:44:02<98:03:37,  6.53s/it]
 76%|███████▌  | 170923/225000 [327:44:05<84:42:06,  5.64s/it]
 76%|███████▌  | 170924/225000 [327:44:09<78:16:16,  5.21s/it]
 76%|███████▌  | 170925/225000 [327:44:17<90:02:41,  5.99s/it]
 76%|███████▌  | 170926/225000 [327:44:22<82:41:14,  5.50s/it]
 76%|███████▌  | 170927/225000 [327:44:26<78:05:54,  5.20s/it]
 76%|███████▌  | 170928/225000 [327:44:31<75:11:29,  5.01s/it]
 76%|███████▌  | 170929/225000 [327:44:35<73:54:54,  4.92s/it]
 76%|███████▌  | 170930/225000 [327:44:40<73:09:48,  4.87s/it]
 76%|███████▌  | 170931/225000 [327:44:45<73:38:07,  4.90s/it]
 76%|███████▌  | 170932/225000 [327:44:56<98:48:28,  6.58s/it]
 76%|███████▌  | 170933/225000 [327:45:01<93:46:51,  6.24s/it]
 76%|███████▌  | 170934/225000 [327:45:05<82:15:45,  5.48s/it]
 76%|███████▌  | 170935/225000 [327:45:09<75:47:11,  5.05s/it]
 76%|███████▌  | 170936/225000 [327:45:13<71:37:02,  4.77s/it]
 76%|███████▌  | 170937/225000 [327:45:25<103:21:38,  6.88s/it]
 76%|███████▌  | 170938/225000 [327:45:28<88:13:43,  5.88s/it] 
 76%|███████▌  | 170939/225000 [327:45:32<78:23:37,  5.22s/it]
 76%|███████▌  | 170940/225000 [327:45:41<94:45:47,  6.31s/it]
 76%|███████▌  | 170941/225000 [327:45:45<86:54:35,  5.79s/it]
 76%|███████▌  | 170942/225000 [327:45:50<83:01:23,  5.53s/it]
 76%|███████▌  | 170943/225000 [327:45:55<78:08:44,  5.20s/it]
 76%|███████▌  | 170944/225000 [327:45:59<74:56:22,  4.99s/it]
 76%|███████▌  | 170945/225000 [327:46:04<74:24:31,  4.96s/it]
 76%|███████▌  | 170946/225000 [327:46:09<74:44:47,  4.98s/it]
 76%|███████▌  | 170947/225000 [327:46:20<100:25:06,  6.69s/it]
 76%|███████▌  | 170948/225000 [327:46:23<85:30:30,  5.70s/it] 
 76%|███████▌  | 170949/225000 [327:46:28<83:07:31,  5.54s/it]
 76%|███████▌  | 170950/225000 [327:46:37<98:33:21,  6.56s/it]
                                                              
{'loss': '0.7589', 'grad_norm': '1.069', 'learning_rate': '7.421e-06', 'epoch': '0.7598', 'num_input_tokens_seen': 543004920, 'train_runtime': '1.18e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 170950/225000 [327:46:37<98:33:21,  6.56s/it]
 76%|███████▌  | 170951/225000 [327:46:41<84:42:18,  5.64s/it]
 76%|███████▌  | 170952/225000 [327:46:47<88:47:05,  5.91s/it]
 76%|███████▌  | 170953/225000 [327:46:51<80:06:07,  5.34s/it]
 76%|███████▌  | 170954/225000 [327:46:55<73:44:07,  4.91s/it]
 76%|███████▌  | 170955/225000 [327:47:05<97:09:36,  6.47s/it]
 76%|███████▌  | 170956/225000 [327:47:09<84:21:50,  5.62s/it]
 76%|███████▌  | 170957/225000 [327:47:12<74:44:13,  4.98s/it]
 76%|███████▌  | 170958/225000 [327:47:20<86:32:58,  5.77s/it]
 76%|███████▌  | 170959/225000 [327:47:24<77:07:08,  5.14s/it]
 76%|███████▌  | 170960/225000 [327:47:27<70:17:20,  4.68s/it]
 76%|███████▌  | 170961/225000 [327:47:32<70:14:21,  4.68s/it]
 76%|███████▌  | 170962/225000 [327:47:40<85:25:07,  5.69s/it]
 76%|███████▌  | 170963/225000 [327:47:44<76:32:57,  5.10s/it]
 76%|███████▌  | 170964/225000 [327:47:51<87:05:53,  5.80s/it]
 76%|███████▌  | 170965/225000 [327:47:56<84:25:15,  5.62s/it]
 76%|███████▌  | 170966/225000 [327:48:00<75:03:55,  5.00s/it]
 76%|███████▌  | 170967/225000 [327:48:05<74:18:26,  4.95s/it]
 76%|███████▌  | 170968/225000 [327:48:10<74:06:47,  4.94s/it]
 76%|███████▌  | 170969/225000 [327:48:15<73:32:27,  4.90s/it]
 76%|███████▌  | 170970/225000 [327:48:20<75:39:58,  5.04s/it]
 76%|███████▌  | 170971/225000 [327:48:24<72:45:46,  4.85s/it]
 76%|███████▌  | 170972/225000 [327:48:30<74:24:56,  4.96s/it]
 76%|███████▌  | 170973/225000 [327:48:37<86:16:26,  5.75s/it]
 76%|███████▌  | 170974/225000 [327:48:47<104:06:57,  6.94s/it]
 76%|███████▌  | 170975/225000 [327:48:51<90:20:00,  6.02s/it] 
 76%|███████▌  | 170976/225000 [327:48:54<79:49:43,  5.32s/it]
 76%|███████▌  | 170977/225000 [327:49:04<97:18:42,  6.48s/it]
 76%|███████▌  | 170978/225000 [327:49:07<84:38:29,  5.64s/it]
 76%|███████▌  | 170979/225000 [327:49:12<78:32:59,  5.23s/it]
 76%|███████▌  | 170980/225000 [327:49:19<90:36:13,  6.04s/it]
 76%|███████▌  | 170981/225000 [327:49:25<88:50:11,  5.92s/it]
 76%|███████▌  | 170982/225000 [327:49:29<78:34:32,  5.24s/it]
 76%|███████▌  | 170983/225000 [327:49:36<88:08:30,  5.87s/it]
 76%|███████▌  | 170984/225000 [327:49:41<82:12:30,  5.48s/it]
 76%|███████▌  | 170985/225000 [327:49:45<77:33:24,  5.17s/it]
 76%|███████▌  | 170986/225000 [327:49:50<76:45:55,  5.12s/it]
 76%|███████▌  | 170987/225000 [327:49:55<75:24:47,  5.03s/it]
 76%|███████▌  | 170988/225000 [327:50:00<74:15:34,  4.95s/it]
 76%|███████▌  | 170989/225000 [327:50:04<73:11:43,  4.88s/it]
 76%|███████▌  | 170990/225000 [327:50:15<99:49:42,  6.65s/it]
 76%|███████▌  | 170991/225000 [327:50:21<95:40:39,  6.38s/it]
 76%|███████▌  | 170992/225000 [327:50:25<83:18:14,  5.55s/it]
 76%|███████▌  | 170993/225000 [327:50:32<93:21:35,  6.22s/it]
 76%|███████▌  | 170994/225000 [327:50:36<81:07:30,  5.41s/it]
 76%|███████▌  | 170995/225000 [327:50:44<92:03:25,  6.14s/it]
 76%|███████▌  | 170996/225000 [327:50:49<87:20:34,  5.82s/it]
 76%|███████▌  | 170997/225000 [327:50:54<82:56:01,  5.53s/it]
 76%|███████▌  | 170998/225000 [327:50:58<79:56:45,  5.33s/it]
 76%|███████▌  | 170999/225000 [327:51:04<78:46:11,  5.25s/it]
 76%|███████▌  | 171000/225000 [327:51:09<79:34:05,  5.30s/it]
                                                              
{'loss': '0.7593', 'grad_norm': '1.146', 'learning_rate': '7.408e-06', 'epoch': '0.76', 'num_input_tokens_seen': 543135856, 'train_runtime': '1.18e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171000/225000 [327:51:09<79:34:05,  5.30s/it]
 76%|███████▌  | 171001/225000 [327:51:14<77:29:01,  5.17s/it]
 76%|███████▌  | 171002/225000 [327:51:19<77:26:05,  5.16s/it]
 76%|███████▌  | 171003/225000 [327:51:29<99:11:13,  6.61s/it]
 76%|███████▌  | 171004/225000 [327:51:32<84:34:52,  5.64s/it]
 76%|███████▌  | 171005/225000 [327:51:36<75:51:10,  5.06s/it]
 76%|███████▌  | 171006/225000 [327:51:45<94:41:29,  6.31s/it]
 76%|███████▌  | 171007/225000 [327:51:49<83:12:48,  5.55s/it]
 76%|███████▌  | 171008/225000 [327:51:52<72:23:44,  4.83s/it]
 76%|███████▌  | 171009/225000 [327:51:59<80:48:23,  5.39s/it]
 76%|███████▌  | 171010/225000 [327:52:05<82:37:14,  5.51s/it]
 76%|███████▌  | 171011/225000 [327:52:09<76:03:11,  5.07s/it]
 76%|███████▌  | 171012/225000 [327:52:16<86:48:29,  5.79s/it]
 76%|███████▌  | 171013/225000 [327:52:20<76:57:10,  5.13s/it]
 76%|███████▌  | 171014/225000 [327:52:28<92:22:39,  6.16s/it]
 76%|███████▌  | 171015/225000 [327:52:32<80:30:50,  5.37s/it]
 76%|███████▌  | 171016/225000 [327:52:38<83:11:43,  5.55s/it]
 76%|███████▌  | 171017/225000 [327:52:44<85:21:20,  5.69s/it]
 76%|███████▌  | 171018/225000 [327:52:48<77:31:47,  5.17s/it]
 76%|███████▌  | 171019/225000 [327:52:51<69:06:27,  4.61s/it]
 76%|███████▌  | 171020/225000 [327:52:59<83:34:48,  5.57s/it]
 76%|███████▌  | 171021/225000 [327:53:04<82:34:11,  5.51s/it]
 76%|███████▌  | 171022/225000 [327:53:09<78:08:08,  5.21s/it]
 76%|███████▌  | 171023/225000 [327:53:13<74:33:02,  4.97s/it]
 76%|███████▌  | 171024/225000 [327:53:18<73:28:14,  4.90s/it]
 76%|███████▌  | 171025/225000 [327:53:22<70:18:07,  4.69s/it]
 76%|███████▌  | 171026/225000 [327:53:27<71:25:47,  4.76s/it]
 76%|███████▌  | 171027/225000 [327:53:32<73:48:13,  4.92s/it]
 76%|███████▌  | 171028/225000 [327:53:37<70:46:54,  4.72s/it]
 76%|███████▌  | 171029/225000 [327:53:48<102:39:41,  6.85s/it]
 76%|███████▌  | 171030/225000 [327:53:52<86:34:20,  5.77s/it] 
 76%|███████▌  | 171031/225000 [327:53:56<78:33:27,  5.24s/it]
 76%|███████▌  | 171032/225000 [327:54:01<79:25:38,  5.30s/it]
 76%|███████▌  | 171033/225000 [327:54:09<90:02:22,  6.01s/it]
 76%|███████▌  | 171034/225000 [327:54:12<77:31:52,  5.17s/it]
 76%|███████▌  | 171035/225000 [327:54:22<98:09:31,  6.55s/it]
 76%|███████▌  | 171036/225000 [327:54:26<88:56:04,  5.93s/it]
 76%|███████▌  | 171037/225000 [327:54:30<80:24:17,  5.36s/it]
 76%|███████▌  | 171038/225000 [327:54:35<75:44:59,  5.05s/it]
 76%|███████▌  | 171039/225000 [327:54:39<72:44:40,  4.85s/it]
 76%|███████▌  | 171040/225000 [327:54:43<69:00:43,  4.60s/it]
 76%|███████▌  | 171041/225000 [327:54:54<99:34:49,  6.64s/it]
 76%|███████▌  | 171042/225000 [327:54:58<87:42:23,  5.85s/it]
 76%|███████▌  | 171043/225000 [327:55:03<79:36:46,  5.31s/it]
 76%|███████▌  | 171044/225000 [327:55:07<74:25:04,  4.97s/it]
 76%|███████▌  | 171045/225000 [327:55:17<100:23:26,  6.70s/it]
 76%|███████▌  | 171046/225000 [327:55:23<93:59:29,  6.27s/it] 
 76%|███████▌  | 171047/225000 [327:55:27<86:12:47,  5.75s/it]
 76%|███████▌  | 171048/225000 [327:55:32<81:32:34,  5.44s/it]
 76%|███████▌  | 171049/225000 [327:55:36<75:01:47,  5.01s/it]
 76%|███████▌  | 171050/225000 [327:55:41<73:59:40,  4.94s/it]
                                                              
{'loss': '0.767', 'grad_norm': '1.16', 'learning_rate': '7.395e-06', 'epoch': '0.7602', 'num_input_tokens_seen': 543265888, 'train_runtime': '1.181e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171050/225000 [327:55:41<73:59:40,  4.94s/it]
 76%|███████▌  | 171051/225000 [327:55:46<73:51:53,  4.93s/it]
 76%|███████▌  | 171052/225000 [327:55:57<103:21:41,  6.90s/it]
 76%|███████▌  | 171053/225000 [327:56:00<87:29:45,  5.84s/it] 
 76%|███████▌  | 171054/225000 [327:56:07<91:06:04,  6.08s/it]
 76%|███████▌  | 171055/225000 [327:56:11<82:55:24,  5.53s/it]
 76%|███████▌  | 171056/225000 [327:56:16<76:44:50,  5.12s/it]
 76%|███████▌  | 171057/225000 [327:56:20<72:35:13,  4.84s/it]
 76%|███████▌  | 171058/225000 [327:56:26<80:42:14,  5.39s/it]
 76%|███████▌  | 171059/225000 [327:56:35<94:49:43,  6.33s/it]
 76%|███████▌  | 171060/225000 [327:56:39<82:35:57,  5.51s/it]
 76%|███████▌  | 171061/225000 [327:56:42<75:27:08,  5.04s/it]
 76%|███████▌  | 171062/225000 [327:56:46<69:53:07,  4.66s/it]
 76%|███████▌  | 171063/225000 [327:56:52<74:41:32,  4.99s/it]
 76%|███████▌  | 171064/225000 [327:57:03<102:51:40,  6.87s/it]
 76%|███████▌  | 171065/225000 [327:57:07<87:13:58,  5.82s/it] 
 76%|███████▌  | 171066/225000 [327:57:10<77:46:37,  5.19s/it]
 76%|███████▌  | 171067/225000 [327:57:15<74:05:57,  4.95s/it]
 76%|███████▌  | 171068/225000 [327:57:19<70:42:47,  4.72s/it]
 76%|███████▌  | 171069/225000 [327:57:27<84:10:38,  5.62s/it]
 76%|███████▌  | 171070/225000 [327:57:35<95:11:16,  6.35s/it]
 76%|███████▌  | 171071/225000 [327:57:40<92:32:07,  6.18s/it]
 76%|███████▌  | 171072/225000 [327:57:45<85:59:02,  5.74s/it]
 76%|███████▌  | 171073/225000 [327:57:49<76:15:18,  5.09s/it]
 76%|███████▌  | 171074/225000 [327:57:56<85:07:34,  5.68s/it]
 76%|███████▌  | 171075/225000 [327:57:59<74:36:10,  4.98s/it]
 76%|███████▌  | 171076/225000 [327:58:08<92:33:59,  6.18s/it]
 76%|███████▌  | 171077/225000 [327:58:12<80:25:16,  5.37s/it]
 76%|███████▌  | 171078/225000 [327:58:15<73:08:14,  4.88s/it]
 76%|███████▌  | 171079/225000 [327:58:25<93:26:40,  6.24s/it]
 76%|███████▌  | 171080/225000 [327:58:29<82:08:05,  5.48s/it]
 76%|███████▌  | 171081/225000 [327:58:32<75:14:10,  5.02s/it]
 76%|███████▌  | 171082/225000 [327:58:36<69:27:33,  4.64s/it]
 76%|███████▌  | 171083/225000 [327:58:44<81:55:01,  5.47s/it]
 76%|███████▌  | 171084/225000 [327:58:48<76:26:54,  5.10s/it]
 76%|███████▌  | 171085/225000 [327:58:52<71:53:22,  4.80s/it]
 76%|███████▌  | 171086/225000 [327:58:56<68:16:55,  4.56s/it]
 76%|███████▌  | 171087/225000 [327:59:07<96:10:44,  6.42s/it]
 76%|███████▌  | 171088/225000 [327:59:10<84:14:08,  5.62s/it]
 76%|███████▌  | 171089/225000 [327:59:14<76:14:32,  5.09s/it]
 76%|███████▌  | 171090/225000 [327:59:18<71:39:53,  4.79s/it]
 76%|███████▌  | 171091/225000 [327:59:22<67:13:30,  4.49s/it]
 76%|███████▌  | 171092/225000 [327:59:34<98:30:54,  6.58s/it]
 76%|███████▌  | 171093/225000 [327:59:37<83:52:36,  5.60s/it]
 76%|███████▌  | 171094/225000 [327:59:45<93:41:57,  6.26s/it]
 76%|███████▌  | 171095/225000 [327:59:48<80:04:22,  5.35s/it]
 76%|███████▌  | 171096/225000 [327:59:53<80:26:28,  5.37s/it]
 76%|███████▌  | 171097/225000 [327:59:59<79:44:19,  5.33s/it]
 76%|███████▌  | 171098/225000 [328:00:04<79:03:43,  5.28s/it]
 76%|███████▌  | 171099/225000 [328:00:10<81:19:04,  5.43s/it]
 76%|███████▌  | 171100/225000 [328:00:15<80:39:33,  5.39s/it]
                                                              
{'loss': '0.7745', 'grad_norm': '1.106', 'learning_rate': '7.382e-06', 'epoch': '0.7604', 'num_input_tokens_seen': 543395528, 'train_runtime': '1.181e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171100/225000 [328:00:15<80:39:33,  5.39s/it]
 76%|███████▌  | 171101/225000 [328:00:24<95:15:28,  6.36s/it]
 76%|███████▌  | 171102/225000 [328:00:26<79:55:11,  5.34s/it]
 76%|███████▌  | 171103/225000 [328:00:31<76:52:20,  5.13s/it]
 76%|███████▌  | 171104/225000 [328:00:36<77:52:39,  5.20s/it]
 76%|███████▌  | 171105/225000 [328:00:42<78:33:32,  5.25s/it]
 76%|███████▌  | 171106/225000 [328:00:47<77:27:19,  5.17s/it]
 76%|███████▌  | 171107/225000 [328:00:52<76:55:15,  5.14s/it]
 76%|███████▌  | 171108/225000 [328:00:57<78:06:00,  5.22s/it]
 76%|███████▌  | 171109/225000 [328:01:03<79:15:36,  5.29s/it]
 76%|███████▌  | 171110/225000 [328:01:08<77:40:35,  5.19s/it]
 76%|███████▌  | 171111/225000 [328:01:13<78:12:14,  5.22s/it]
 76%|███████▌  | 171112/225000 [328:01:18<78:10:50,  5.22s/it]
 76%|███████▌  | 171113/225000 [328:01:23<77:48:41,  5.20s/it]
 76%|███████▌  | 171114/225000 [328:01:33<99:47:18,  6.67s/it]
 76%|███████▌  | 171115/225000 [328:01:39<93:32:40,  6.25s/it]
 76%|███████▌  | 171116/225000 [328:01:44<89:48:18,  6.00s/it]
 76%|███████▌  | 171117/225000 [328:01:49<85:30:21,  5.71s/it]
 76%|███████▌  | 171118/225000 [328:01:55<84:44:12,  5.66s/it]
 76%|███████▌  | 171119/225000 [328:02:00<82:44:03,  5.53s/it]
 76%|███████▌  | 171120/225000 [328:02:05<82:34:48,  5.52s/it]
 76%|███████▌  | 171121/225000 [328:02:10<80:18:11,  5.37s/it]
 76%|███████▌  | 171122/225000 [328:02:16<81:08:28,  5.42s/it]
 76%|███████▌  | 171123/225000 [328:02:21<79:56:18,  5.34s/it]
 76%|███████▌  | 171124/225000 [328:02:26<78:40:19,  5.26s/it]
 76%|███████▌  | 171125/225000 [328:02:31<78:30:26,  5.25s/it]
 76%|███████▌  | 171126/225000 [328:02:37<77:42:28,  5.19s/it]
 76%|███████▌  | 171127/225000 [328:02:42<78:48:13,  5.27s/it]
 76%|███████▌  | 171128/225000 [328:02:48<80:51:36,  5.40s/it]
 76%|███████▌  | 171129/225000 [328:02:53<80:28:31,  5.38s/it]
 76%|███████▌  | 171130/225000 [328:02:58<77:40:27,  5.19s/it]
 76%|███████▌  | 171131/225000 [328:03:03<75:58:29,  5.08s/it]
 76%|███████▌  | 171132/225000 [328:03:09<81:20:27,  5.44s/it]
 76%|███████▌  | 171133/225000 [328:03:14<81:29:29,  5.45s/it]
 76%|███████▌  | 171134/225000 [328:03:21<85:27:19,  5.71s/it]
 76%|███████▌  | 171135/225000 [328:03:30<101:36:40,  6.79s/it]
 76%|███████▌  | 171136/225000 [328:03:34<87:07:22,  5.82s/it] 
 76%|███████▌  | 171137/225000 [328:03:43<105:11:36,  7.03s/it]
 76%|███████▌  | 171138/225000 [328:03:47<88:32:26,  5.92s/it] 
 76%|███████▌  | 171139/225000 [328:03:52<85:43:09,  5.73s/it]
 76%|███████▌  | 171140/225000 [328:03:59<92:01:57,  6.15s/it]
 76%|███████▌  | 171141/225000 [328:04:04<85:26:29,  5.71s/it]
 76%|███████▌  | 171142/225000 [328:04:09<81:44:06,  5.46s/it]
 76%|███████▌  | 171143/225000 [328:04:13<75:46:33,  5.07s/it]
 76%|███████▌  | 171144/225000 [328:04:17<74:03:36,  4.95s/it]
 76%|███████▌  | 171145/225000 [328:04:22<72:49:51,  4.87s/it]
 76%|███████▌  | 171146/225000 [328:04:27<72:16:24,  4.83s/it]
 76%|███████▌  | 171147/225000 [328:04:37<96:05:07,  6.42s/it]
 76%|███████▌  | 171148/225000 [328:04:41<83:03:47,  5.55s/it]
 76%|███████▌  | 171149/225000 [328:04:49<95:48:56,  6.41s/it]
 76%|███████▌  | 171150/225000 [328:04:54<88:21:47,  5.91s/it]
                                                              
{'loss': '0.777', 'grad_norm': '1.099', 'learning_rate': '7.369e-06', 'epoch': '0.7607', 'num_input_tokens_seen': 543526840, 'train_runtime': '1.181e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171150/225000 [328:04:54<88:21:47,  5.91s/it]
 76%|███████▌  | 171151/225000 [328:04:59<85:15:25,  5.70s/it]
 76%|███████▌  | 171152/225000 [328:05:03<78:34:38,  5.25s/it]
 76%|███████▌  | 171153/225000 [328:05:07<73:33:00,  4.92s/it]
 76%|███████▌  | 171154/225000 [328:05:11<68:37:54,  4.59s/it]
 76%|███████▌  | 171155/225000 [328:05:21<92:36:42,  6.19s/it]
 76%|███████▌  | 171156/225000 [328:05:26<87:23:23,  5.84s/it]
 76%|███████▌  | 171157/225000 [328:05:30<77:30:19,  5.18s/it]
 76%|███████▌  | 171158/225000 [328:05:38<93:39:46,  6.26s/it]
 76%|███████▌  | 171159/225000 [328:05:42<80:32:33,  5.39s/it]
 76%|███████▌  | 171160/225000 [328:05:45<70:52:24,  4.74s/it]
 76%|███████▌  | 171161/225000 [328:05:54<87:53:20,  5.88s/it]
 76%|███████▌  | 171162/225000 [328:05:57<77:59:16,  5.21s/it]
 76%|███████▌  | 171163/225000 [328:06:04<84:40:53,  5.66s/it]
 76%|███████▌  | 171164/225000 [328:06:08<77:00:01,  5.15s/it]
 76%|███████▌  | 171165/225000 [328:06:12<71:12:13,  4.76s/it]
 76%|███████▌  | 171166/225000 [328:06:19<83:30:44,  5.58s/it]
 76%|███████▌  | 171167/225000 [328:06:25<82:18:42,  5.50s/it]
 76%|███████▌  | 171168/225000 [328:06:28<73:49:55,  4.94s/it]
 76%|███████▌  | 171169/225000 [328:06:32<70:43:53,  4.73s/it]
 76%|███████▌  | 171170/225000 [328:06:37<70:27:42,  4.71s/it]
 76%|███████▌  | 171171/225000 [328:06:47<92:03:18,  6.16s/it]
 76%|███████▌  | 171172/225000 [328:06:52<89:43:03,  6.00s/it]
 76%|███████▌  | 171173/225000 [328:06:56<78:38:32,  5.26s/it]
 76%|███████▌  | 171174/225000 [328:07:01<79:05:15,  5.29s/it]
 76%|███████▌  | 171175/225000 [328:07:08<88:08:04,  5.89s/it]
 76%|███████▌  | 171176/225000 [328:07:14<88:31:51,  5.92s/it]
 76%|███████▌  | 171177/225000 [328:07:18<79:30:39,  5.32s/it]
 76%|███████▌  | 171178/225000 [328:07:23<76:25:24,  5.11s/it]
 76%|███████▌  | 171179/225000 [328:07:27<72:33:30,  4.85s/it]
 76%|███████▌  | 171180/225000 [328:07:32<72:46:11,  4.87s/it]
 76%|███████▌  | 171181/225000 [328:07:40<84:59:59,  5.69s/it]
 76%|███████▌  | 171182/225000 [328:07:47<91:10:55,  6.10s/it]
 76%|███████▌  | 171183/225000 [328:07:51<81:08:10,  5.43s/it]
 76%|███████▌  | 171184/225000 [328:07:59<93:44:57,  6.27s/it]
 76%|███████▌  | 171185/225000 [328:08:02<81:22:47,  5.44s/it]
 76%|███████▌  | 171186/225000 [328:08:07<77:23:42,  5.18s/it]
 76%|███████▌  | 171187/225000 [328:08:15<91:41:16,  6.13s/it]
 76%|███████▌  | 171188/225000 [328:08:19<82:10:00,  5.50s/it]
 76%|███████▌  | 171189/225000 [328:08:23<74:55:33,  5.01s/it]
 76%|███████▌  | 171190/225000 [328:08:33<96:29:46,  6.46s/it]
 76%|███████▌  | 171191/225000 [328:08:36<82:09:18,  5.50s/it]
 76%|███████▌  | 171192/225000 [328:08:44<89:57:08,  6.02s/it]
 76%|███████▌  | 171193/225000 [328:08:47<79:05:48,  5.29s/it]
 76%|███████▌  | 171194/225000 [328:08:51<70:56:39,  4.75s/it]
 76%|███████▌  | 171195/225000 [328:09:00<92:06:33,  6.16s/it]
 76%|███████▌  | 171196/225000 [328:09:04<79:46:22,  5.34s/it]
 76%|███████▌  | 171197/225000 [328:09:09<81:32:14,  5.46s/it]
 76%|███████▌  | 171198/225000 [328:09:15<81:33:04,  5.46s/it]
 76%|███████▌  | 171199/225000 [328:09:22<90:14:50,  6.04s/it]
 76%|███████▌  | 171200/225000 [328:09:26<79:26:44,  5.32s/it]
                                                              
{'loss': '0.7616', 'grad_norm': '1.169', 'learning_rate': '7.356e-06', 'epoch': '0.7609', 'num_input_tokens_seen': 543657072, 'train_runtime': '1.181e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171200/225000 [328:09:26<79:26:44,  5.32s/it]
 76%|███████▌  | 171201/225000 [328:09:31<81:06:02,  5.43s/it]
 76%|███████▌  | 171202/225000 [328:09:37<82:52:14,  5.55s/it]
 76%|███████▌  | 171203/225000 [328:09:43<82:08:10,  5.50s/it]
 76%|███████▌  | 171204/225000 [328:09:48<82:26:35,  5.52s/it]
 76%|███████▌  | 171205/225000 [328:09:56<94:08:54,  6.30s/it]
 76%|███████▌  | 171206/225000 [328:10:00<81:53:00,  5.48s/it]
 76%|███████▌  | 171207/225000 [328:10:08<93:42:23,  6.27s/it]
 76%|███████▌  | 171208/225000 [328:10:12<83:48:47,  5.61s/it]
 76%|███████▌  | 171209/225000 [328:10:20<93:31:23,  6.26s/it]
 76%|███████▌  | 171210/225000 [328:10:24<85:09:45,  5.70s/it]
 76%|███████▌  | 171211/225000 [328:10:29<79:19:22,  5.31s/it]
 76%|███████▌  | 171212/225000 [328:10:33<75:10:00,  5.03s/it]
 76%|███████▌  | 171213/225000 [328:10:37<71:43:15,  4.80s/it]
 76%|███████▌  | 171214/225000 [328:10:42<69:23:07,  4.64s/it]
 76%|███████▌  | 171215/225000 [328:10:46<68:08:26,  4.56s/it]
 76%|███████▌  | 171216/225000 [328:10:54<85:52:40,  5.75s/it]
 76%|███████▌  | 171217/225000 [328:11:02<92:19:48,  6.18s/it]
 76%|███████▌  | 171218/225000 [328:11:05<80:32:35,  5.39s/it]
 76%|███████▌  | 171219/225000 [328:11:09<72:54:34,  4.88s/it]
 76%|███████▌  | 171220/225000 [328:11:17<89:36:44,  6.00s/it]
 76%|███████▌  | 171221/225000 [328:11:23<87:51:30,  5.88s/it]
 76%|███████▌  | 171222/225000 [328:11:28<81:46:10,  5.47s/it]
 76%|███████▌  | 171223/225000 [328:11:32<78:35:53,  5.26s/it]
 76%|███████▌  | 171224/225000 [328:11:37<76:41:59,  5.13s/it]
 76%|███████▌  | 171225/225000 [328:11:42<73:33:51,  4.92s/it]
 76%|███████▌  | 171226/225000 [328:11:46<71:01:18,  4.75s/it]
 76%|███████▌  | 171227/225000 [328:11:54<86:35:11,  5.80s/it]
 76%|███████▌  | 171228/225000 [328:12:01<89:21:14,  5.98s/it]
 76%|███████▌  | 171229/225000 [328:12:04<77:48:18,  5.21s/it]
 76%|███████▌  | 171230/225000 [328:12:09<74:54:54,  5.02s/it]
 76%|███████▌  | 171231/225000 [328:12:13<72:57:26,  4.88s/it]
 76%|███████▌  | 171232/225000 [328:12:19<75:11:55,  5.03s/it]
 76%|███████▌  | 171233/225000 [328:12:24<75:29:37,  5.05s/it]
 76%|███████▌  | 171234/225000 [328:12:29<75:12:36,  5.04s/it]
 76%|███████▌  | 171235/225000 [328:12:34<76:13:27,  5.10s/it]
 76%|███████▌  | 171236/225000 [328:12:39<75:08:32,  5.03s/it]
 76%|███████▌  | 171237/225000 [328:12:44<75:06:27,  5.03s/it]
 76%|███████▌  | 171238/225000 [328:12:49<75:29:46,  5.06s/it]
 76%|███████▌  | 171239/225000 [328:12:54<76:11:59,  5.10s/it]
 76%|███████▌  | 171240/225000 [328:13:04<96:38:18,  6.47s/it]
 76%|███████▌  | 171241/225000 [328:13:09<90:43:34,  6.08s/it]
 76%|███████▌  | 171242/225000 [328:13:14<87:17:24,  5.85s/it]
 76%|███████▌  | 171243/225000 [328:13:19<84:05:56,  5.63s/it]
 76%|███████▌  | 171244/225000 [328:13:25<82:36:10,  5.53s/it]
 76%|███████▌  | 171245/225000 [328:13:30<80:19:33,  5.38s/it]
 76%|███████▌  | 171246/225000 [328:13:35<78:41:22,  5.27s/it]
 76%|███████▌  | 171247/225000 [328:13:40<78:20:06,  5.25s/it]
 76%|███████▌  | 171248/225000 [328:13:45<78:31:38,  5.26s/it]
 76%|███████▌  | 171249/225000 [328:13:51<79:09:23,  5.30s/it]
 76%|███████▌  | 171250/225000 [328:13:56<79:52:33,  5.35s/it]
                                                              
{'loss': '0.7662', 'grad_norm': '1.167', 'learning_rate': '7.343e-06', 'epoch': '0.7611', 'num_input_tokens_seen': 543787536, 'train_runtime': '1.182e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171250/225000 [328:13:56<79:52:33,  5.35s/it]
 76%|███████▌  | 171251/225000 [328:14:01<80:02:30,  5.36s/it]
 76%|███████▌  | 171252/225000 [328:14:07<80:06:48,  5.37s/it]
 76%|███████▌  | 171253/225000 [328:14:12<79:52:06,  5.35s/it]
 76%|███████▌  | 171254/225000 [328:14:23<103:24:15,  6.93s/it]
 76%|███████▌  | 171255/225000 [328:14:28<96:26:08,  6.46s/it] 
 76%|███████▌  | 171256/225000 [328:14:32<86:18:01,  5.78s/it]
 76%|███████▌  | 171257/225000 [328:14:37<79:40:49,  5.34s/it]
 76%|███████▌  | 171258/225000 [328:14:47<102:10:05,  6.84s/it]
 76%|███████▌  | 171259/225000 [328:14:53<98:24:18,  6.59s/it] 
 76%|███████▌  | 171260/225000 [328:14:57<87:51:00,  5.89s/it]
 76%|███████▌  | 171261/225000 [328:15:02<82:41:35,  5.54s/it]
 76%|███████▌  | 171262/225000 [328:15:07<80:43:08,  5.41s/it]
 76%|███████▌  | 171263/225000 [328:15:12<77:44:44,  5.21s/it]
 76%|███████▌  | 171264/225000 [328:15:16<74:27:42,  4.99s/it]
 76%|███████▌  | 171265/225000 [328:15:22<76:16:27,  5.11s/it]
 76%|███████▌  | 171266/225000 [328:15:27<75:34:24,  5.06s/it]
 76%|███████▌  | 171267/225000 [328:15:31<72:50:11,  4.88s/it]
 76%|███████▌  | 171268/225000 [328:15:36<73:14:07,  4.91s/it]
 76%|███████▌  | 171269/225000 [328:15:42<75:59:16,  5.09s/it]
 76%|███████▌  | 171270/225000 [328:15:46<74:29:06,  4.99s/it]
 76%|███████▌  | 171271/225000 [328:15:51<74:28:46,  4.99s/it]
 76%|███████▌  | 171272/225000 [328:16:01<94:43:38,  6.35s/it]
 76%|███████▌  | 171273/225000 [328:16:09<100:45:41,  6.75s/it]
 76%|███████▌  | 171274/225000 [328:16:13<92:05:06,  6.17s/it] 
 76%|███████▌  | 171275/225000 [328:16:18<83:29:47,  5.59s/it]
 76%|███████▌  | 171276/225000 [328:16:22<79:08:26,  5.30s/it]
 76%|███████▌  | 171277/225000 [328:16:27<74:58:14,  5.02s/it]
 76%|███████▌  | 171278/225000 [328:16:31<73:02:31,  4.89s/it]
 76%|███████▌  | 171279/225000 [328:16:36<72:18:57,  4.85s/it]
 76%|███████▌  | 171280/225000 [328:16:47<101:06:19,  6.78s/it]
 76%|███████▌  | 171281/225000 [328:16:51<85:46:20,  5.75s/it] 
 76%|███████▌  | 171282/225000 [328:16:54<75:13:41,  5.04s/it]
 76%|███████▌  | 171283/225000 [328:17:04<98:21:43,  6.59s/it]
 76%|███████▌  | 171284/225000 [328:17:07<83:37:02,  5.60s/it]
 76%|███████▌  | 171285/225000 [328:17:12<76:55:10,  5.16s/it]
 76%|███████▌  | 171286/225000 [328:17:16<73:47:09,  4.95s/it]
 76%|███████▌  | 171287/225000 [328:17:20<71:10:29,  4.77s/it]
 76%|███████▌  | 171288/225000 [328:17:25<68:56:43,  4.62s/it]
 76%|███████▌  | 171289/225000 [328:17:29<69:02:41,  4.63s/it]
 76%|███████▌  | 171290/225000 [328:17:34<71:31:42,  4.79s/it]
 76%|███████▌  | 171291/225000 [328:17:43<87:11:55,  5.84s/it]
 76%|███████▌  | 171292/225000 [328:17:51<96:12:12,  6.45s/it]
 76%|███████▌  | 171293/225000 [328:17:54<83:32:47,  5.60s/it]
 76%|███████▌  | 171294/225000 [328:18:05<106:03:55,  7.11s/it]
 76%|███████▌  | 171295/225000 [328:18:09<92:47:58,  6.22s/it] 
 76%|███████▌  | 171296/225000 [328:18:13<83:44:20,  5.61s/it]
 76%|███████▌  | 171297/225000 [328:18:17<76:57:41,  5.16s/it]
 76%|███████▌  | 171298/225000 [328:18:27<97:21:48,  6.53s/it]
 76%|███████▌  | 171299/225000 [328:18:30<83:08:04,  5.57s/it]
 76%|███████▌  | 171300/225000 [328:18:36<84:12:30,  5.65s/it]
                                                              
{'loss': '0.7697', 'grad_norm': '1.274', 'learning_rate': '7.33e-06', 'epoch': '0.7613', 'num_input_tokens_seen': 543916816, 'train_runtime': '1.182e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171300/225000 [328:18:36<84:12:30,  5.65s/it]
 76%|███████▌  | 171301/225000 [328:18:44<93:33:00,  6.27s/it]
 76%|███████▌  | 171302/225000 [328:18:47<79:00:35,  5.30s/it]
 76%|███████▌  | 171303/225000 [328:18:51<74:45:15,  5.01s/it]
 76%|███████▌  | 171304/225000 [328:18:57<76:58:29,  5.16s/it]
 76%|███████▌  | 171305/225000 [328:19:04<85:54:55,  5.76s/it]
 76%|███████▌  | 171306/225000 [328:19:09<81:13:11,  5.45s/it]
 76%|███████▌  | 171307/225000 [328:19:14<79:33:12,  5.33s/it]
 76%|███████▌  | 171308/225000 [328:19:18<75:36:11,  5.07s/it]
 76%|███████▌  | 171309/225000 [328:19:23<73:49:00,  4.95s/it]
 76%|███████▌  | 171310/225000 [328:19:28<73:02:11,  4.90s/it]
 76%|███████▌  | 171311/225000 [328:19:32<71:57:09,  4.82s/it]
 76%|███████▌  | 171312/225000 [328:19:39<81:44:51,  5.48s/it]
 76%|███████▌  | 171313/225000 [328:19:48<96:58:49,  6.50s/it]
 76%|███████▌  | 171314/225000 [328:19:52<84:26:14,  5.66s/it]
 76%|███████▌  | 171315/225000 [328:20:01<97:50:52,  6.56s/it]
 76%|███████▌  | 171316/225000 [328:20:04<84:29:49,  5.67s/it]
 76%|███████▌  | 171317/225000 [328:20:12<95:08:34,  6.38s/it]
 76%|███████▌  | 171318/225000 [328:20:16<81:44:07,  5.48s/it]
 76%|███████▌  | 171319/225000 [328:20:20<75:45:20,  5.08s/it]
 76%|███████▌  | 171320/225000 [328:20:24<69:56:44,  4.69s/it]
 76%|███████▌  | 171321/225000 [328:20:28<67:19:13,  4.51s/it]
 76%|███████▌  | 171322/225000 [328:20:34<74:15:05,  4.98s/it]
 76%|███████▌  | 171323/225000 [328:20:45<104:23:24,  7.00s/it]
 76%|███████▌  | 171324/225000 [328:20:49<88:17:31,  5.92s/it] 
 76%|███████▌  | 171325/225000 [328:20:52<77:26:33,  5.19s/it]
 76%|███████▌  | 171326/225000 [328:21:01<93:45:24,  6.29s/it]
 76%|███████▌  | 171327/225000 [328:21:05<80:50:51,  5.42s/it]
 76%|███████▌  | 171328/225000 [328:21:09<75:41:09,  5.08s/it]
 76%|███████▌  | 171329/225000 [328:21:13<73:28:27,  4.93s/it]
 76%|███████▌  | 171330/225000 [328:21:18<70:29:34,  4.73s/it]
 76%|███████▌  | 171331/225000 [328:21:23<75:29:27,  5.06s/it]
 76%|███████▌  | 171332/225000 [328:21:34<98:34:56,  6.61s/it]
 76%|███████▌  | 171333/225000 [328:21:37<83:58:05,  5.63s/it]
 76%|███████▌  | 171334/225000 [328:21:45<92:11:04,  6.18s/it]
 76%|███████▌  | 171335/225000 [328:21:48<81:40:53,  5.48s/it]
 76%|███████▌  | 171336/225000 [328:21:52<73:05:50,  4.90s/it]
 76%|███████▌  | 171337/225000 [328:21:58<80:31:01,  5.40s/it]
 76%|███████▌  | 171338/225000 [328:22:05<87:13:15,  5.85s/it]
 76%|███████▌  | 171339/225000 [328:22:09<76:53:21,  5.16s/it]
 76%|███████▌  | 171340/225000 [328:22:18<94:26:21,  6.34s/it]
 76%|███████▌  | 171341/225000 [328:22:22<84:26:38,  5.67s/it]
 76%|███████▌  | 171342/225000 [328:22:26<75:20:29,  5.05s/it]
 76%|███████▌  | 171343/225000 [328:22:30<70:17:50,  4.72s/it]
 76%|███████▌  | 171344/225000 [328:22:40<96:12:08,  6.45s/it]
 76%|███████▌  | 171345/225000 [328:22:44<82:21:22,  5.53s/it]
 76%|███████▌  | 171346/225000 [328:22:52<93:16:06,  6.26s/it]
 76%|███████▌  | 171347/225000 [328:22:55<81:21:20,  5.46s/it]
 76%|███████▌  | 171348/225000 [328:23:03<93:06:11,  6.25s/it]
 76%|███████▌  | 171349/225000 [328:23:07<82:09:33,  5.51s/it]
 76%|███████▌  | 171350/225000 [328:23:11<73:33:27,  4.94s/it]
                                                              
{'loss': '0.7747', 'grad_norm': '1.171', 'learning_rate': '7.317e-06', 'epoch': '0.7616', 'num_input_tokens_seen': 544046704, 'train_runtime': '1.182e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171350/225000 [328:23:11<73:33:27,  4.94s/it]
 76%|███████▌  | 171351/225000 [328:23:16<76:33:32,  5.14s/it]
 76%|███████▌  | 171352/225000 [328:23:23<85:02:20,  5.71s/it]
 76%|███████▌  | 171353/225000 [328:23:27<75:23:53,  5.06s/it]
 76%|███████▌  | 171354/225000 [328:23:35<89:40:11,  6.02s/it]
 76%|███████▌  | 171355/225000 [328:23:38<77:17:50,  5.19s/it]
 76%|███████▌  | 171356/225000 [328:23:46<87:22:14,  5.86s/it]
 76%|███████▌  | 171357/225000 [328:23:50<79:46:31,  5.35s/it]
 76%|███████▌  | 171358/225000 [328:23:55<77:17:31,  5.19s/it]
 76%|███████▌  | 171359/225000 [328:23:59<74:36:19,  5.01s/it]
 76%|███████▌  | 171360/225000 [328:24:04<71:29:55,  4.80s/it]
 76%|███████▌  | 171361/225000 [328:24:08<69:23:30,  4.66s/it]
 76%|███████▌  | 171362/225000 [328:24:12<68:49:53,  4.62s/it]
 76%|███████▌  | 171363/225000 [328:24:19<75:44:07,  5.08s/it]
 76%|███████▌  | 171364/225000 [328:24:30<102:07:35,  6.85s/it]
 76%|███████▌  | 171365/225000 [328:24:35<94:38:21,  6.35s/it] 
 76%|███████▌  | 171366/225000 [328:24:40<88:01:38,  5.91s/it]
 76%|███████▌  | 171367/225000 [328:24:45<84:17:21,  5.66s/it]
 76%|███████▌  | 171368/225000 [328:24:50<82:47:40,  5.56s/it]
 76%|███████▌  | 171369/225000 [328:24:55<81:12:17,  5.45s/it]
 76%|███████▌  | 171370/225000 [328:25:00<79:02:32,  5.31s/it]
 76%|███████▌  | 171371/225000 [328:25:06<80:36:46,  5.41s/it]
 76%|███████▌  | 171372/225000 [328:25:11<80:10:55,  5.38s/it]
 76%|███████▌  | 171373/225000 [328:25:16<79:07:53,  5.31s/it]
 76%|███████▌  | 171374/225000 [328:25:21<77:18:43,  5.19s/it]
 76%|███████▌  | 171375/225000 [328:25:33<105:22:25,  7.07s/it]
 76%|███████▌  | 171376/225000 [328:25:36<90:12:23,  6.06s/it] 
 76%|███████▌  | 171377/225000 [328:25:40<80:04:43,  5.38s/it]
 76%|███████▌  | 171378/225000 [328:25:48<90:51:07,  6.10s/it]
 76%|███████▌  | 171379/225000 [328:25:52<81:11:33,  5.45s/it]
 76%|███████▌  | 171380/225000 [328:25:56<77:21:48,  5.19s/it]
 76%|███████▌  | 171381/225000 [328:26:05<92:11:40,  6.19s/it]
 76%|███████▌  | 171382/225000 [328:26:09<80:43:10,  5.42s/it]
 76%|███████▌  | 171383/225000 [328:26:16<89:30:39,  6.01s/it]
 76%|███████▌  | 171384/225000 [328:26:22<88:02:00,  5.91s/it]
 76%|███████▌  | 171385/225000 [328:26:25<77:45:28,  5.22s/it]
 76%|███████▌  | 171386/225000 [328:26:31<79:47:58,  5.36s/it]
 76%|███████▌  | 171387/225000 [328:26:36<76:37:17,  5.14s/it]
 76%|███████▌  | 171388/225000 [328:26:43<85:34:37,  5.75s/it]
 76%|███████▌  | 171389/225000 [328:26:47<76:28:54,  5.14s/it]
 76%|███████▌  | 171390/225000 [328:26:50<71:16:11,  4.79s/it]
 76%|███████▌  | 171391/225000 [328:26:54<67:30:49,  4.53s/it]
 76%|███████▌  | 171392/225000 [328:26:59<65:48:24,  4.42s/it]
 76%|███████▌  | 171393/225000 [328:27:10<98:08:32,  6.59s/it]
 76%|███████▌  | 171394/225000 [328:27:14<83:54:32,  5.64s/it]
 76%|███████▌  | 171395/225000 [328:27:17<75:41:55,  5.08s/it]
 76%|███████▌  | 171396/225000 [328:27:26<91:29:08,  6.14s/it]
 76%|███████▌  | 171397/225000 [328:27:31<85:08:08,  5.72s/it]
 76%|███████▌  | 171398/225000 [328:27:34<75:28:26,  5.07s/it]
 76%|███████▌  | 171399/225000 [328:27:43<90:46:29,  6.10s/it]
 76%|███████▌  | 171400/225000 [328:27:47<82:32:54,  5.54s/it]
                                                              
{'loss': '0.7551', 'grad_norm': '1.182', 'learning_rate': '7.304e-06', 'epoch': '0.7618', 'num_input_tokens_seen': 544177040, 'train_runtime': '1.182e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171400/225000 [328:27:47<82:32:54,  5.54s/it]
 76%|███████▌  | 171401/225000 [328:27:53<85:07:13,  5.72s/it]
 76%|███████▌  | 171402/225000 [328:28:00<89:06:51,  5.99s/it]
 76%|███████▌  | 171403/225000 [328:28:05<83:22:57,  5.60s/it]
 76%|███████▌  | 171404/225000 [328:28:09<79:43:24,  5.35s/it]
 76%|███████▌  | 171405/225000 [328:28:13<74:07:00,  4.98s/it]
 76%|███████▌  | 171406/225000 [328:28:18<72:13:40,  4.85s/it]
 76%|███████▌  | 171407/225000 [328:28:22<69:23:10,  4.66s/it]
 76%|███████▌  | 171408/225000 [328:28:29<77:55:47,  5.23s/it]
 76%|███████▌  | 171409/225000 [328:28:37<92:11:50,  6.19s/it]
 76%|███████▌  | 171410/225000 [328:28:41<79:59:55,  5.37s/it]
 76%|███████▌  | 171411/225000 [328:28:48<88:29:32,  5.94s/it]
 76%|███████▌  | 171412/225000 [328:28:53<83:04:26,  5.58s/it]
 76%|███████▌  | 171413/225000 [328:28:56<73:29:38,  4.94s/it]
 76%|███████▌  | 171414/225000 [328:29:05<93:23:26,  6.27s/it]
 76%|███████▌  | 171415/225000 [328:29:09<81:15:31,  5.46s/it]
 76%|███████▌  | 171416/225000 [328:29:16<87:17:49,  5.86s/it]
 76%|███████▌  | 171417/225000 [328:29:20<78:50:51,  5.30s/it]
 76%|███████▌  | 171418/225000 [328:29:24<74:09:09,  4.98s/it]
 76%|███████▌  | 171419/225000 [328:29:30<78:23:44,  5.27s/it]
 76%|███████▌  | 171420/225000 [328:29:33<69:47:06,  4.69s/it]
 76%|███████▌  | 171421/225000 [328:29:38<67:38:38,  4.55s/it]
 76%|███████▌  | 171422/225000 [328:29:42<66:32:24,  4.47s/it]
 76%|███████▌  | 171423/225000 [328:29:47<67:57:54,  4.57s/it]
 76%|███████▌  | 171424/225000 [328:29:52<69:43:59,  4.69s/it]
 76%|███████▌  | 171425/225000 [328:30:04<104:45:12,  7.04s/it]
 76%|███████▌  | 171426/225000 [328:30:08<90:53:07,  6.11s/it] 
 76%|███████▌  | 171427/225000 [328:30:15<95:14:18,  6.40s/it]
 76%|███████▌  | 171428/225000 [328:30:19<82:31:21,  5.55s/it]
 76%|███████▌  | 171429/225000 [328:30:25<86:21:02,  5.80s/it]
 76%|███████▌  | 171430/225000 [328:30:31<86:02:58,  5.78s/it]
 76%|███████▌  | 171431/225000 [328:30:34<75:50:56,  5.10s/it]
 76%|███████▌  | 171432/225000 [328:30:38<70:28:00,  4.74s/it]
 76%|███████▌  | 171433/225000 [328:30:49<96:57:51,  6.52s/it]
 76%|███████▌  | 171434/225000 [328:30:52<83:44:09,  5.63s/it]
 76%|███████▌  | 171435/225000 [328:30:56<74:47:54,  5.03s/it]
 76%|███████▌  | 171436/225000 [328:31:04<88:52:40,  5.97s/it]
 76%|███████▌  | 171437/225000 [328:31:10<87:46:22,  5.90s/it]
 76%|███████▌  | 171438/225000 [328:31:14<78:17:02,  5.26s/it]
 76%|███████▌  | 171439/225000 [328:31:23<94:51:59,  6.38s/it]
 76%|███████▌  | 171440/225000 [328:31:26<81:36:02,  5.48s/it]
 76%|███████▌  | 171441/225000 [328:31:35<94:30:35,  6.35s/it]
 76%|███████▌  | 171442/225000 [328:31:38<81:33:27,  5.48s/it]
 76%|███████▌  | 171443/225000 [328:31:42<74:11:25,  4.99s/it]
 76%|███████▌  | 171444/225000 [328:31:46<69:49:47,  4.69s/it]
 76%|███████▌  | 171445/225000 [328:31:50<68:11:18,  4.58s/it]
 76%|███████▌  | 171446/225000 [328:31:55<68:21:34,  4.60s/it]
 76%|███████▌  | 171447/225000 [328:32:06<99:11:14,  6.67s/it]
 76%|███████▌  | 171448/225000 [328:32:11<89:16:02,  6.00s/it]
 76%|███████▌  | 171449/225000 [328:32:15<82:28:06,  5.54s/it]
 76%|███████▌  | 171450/225000 [328:32:20<78:38:54,  5.29s/it]
                                                              
{'loss': '0.763', 'grad_norm': '1.146', 'learning_rate': '7.291e-06', 'epoch': '0.762', 'num_input_tokens_seen': 544307128, 'train_runtime': '1.183e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171450/225000 [328:32:20<78:38:54,  5.29s/it]
 76%|███████▌  | 171451/225000 [328:32:24<74:14:28,  4.99s/it]
 76%|███████▌  | 171452/225000 [328:32:28<70:50:02,  4.76s/it]
 76%|███████▌  | 171453/225000 [328:32:33<70:48:35,  4.76s/it]
 76%|███████▌  | 171454/225000 [328:32:43<91:54:09,  6.18s/it]
 76%|███████▌  | 171455/225000 [328:32:49<91:27:16,  6.15s/it]
 76%|███████▌  | 171456/225000 [328:32:55<90:01:11,  6.05s/it]
 76%|███████▌  | 171457/225000 [328:32:59<81:30:33,  5.48s/it]
 76%|███████▌  | 171458/225000 [328:33:06<89:32:19,  6.02s/it]
 76%|███████▌  | 171459/225000 [328:33:11<83:01:59,  5.58s/it]
 76%|███████▌  | 171460/225000 [328:33:15<76:34:34,  5.15s/it]
 76%|███████▌  | 171461/225000 [328:33:19<72:28:19,  4.87s/it]
 76%|███████▌  | 171462/225000 [328:33:23<69:26:05,  4.67s/it]
 76%|███████▌  | 171463/225000 [328:33:28<71:38:56,  4.82s/it]
 76%|███████▌  | 171464/225000 [328:33:39<98:51:44,  6.65s/it]
 76%|███████▌  | 171465/225000 [328:33:43<84:18:32,  5.67s/it]
 76%|███████▌  | 171466/225000 [328:33:50<90:27:29,  6.08s/it]
 76%|███████▌  | 171467/225000 [328:33:54<83:57:44,  5.65s/it]
 76%|███████▌  | 171468/225000 [328:33:59<79:59:26,  5.38s/it]
 76%|███████▌  | 171469/225000 [328:34:04<76:54:20,  5.17s/it]
 76%|███████▌  | 171470/225000 [328:34:08<73:52:30,  4.97s/it]
 76%|███████▌  | 171471/225000 [328:34:13<72:41:42,  4.89s/it]
 76%|███████▌  | 171472/225000 [328:34:18<72:28:08,  4.87s/it]
 76%|███████▌  | 171473/225000 [328:34:22<71:07:58,  4.78s/it]
 76%|███████▌  | 171474/225000 [328:34:28<73:31:51,  4.95s/it]
 76%|███████▌  | 171475/225000 [328:34:40<106:28:02,  7.16s/it]
 76%|███████▌  | 171476/225000 [328:34:44<90:41:31,  6.10s/it] 
 76%|███████▌  | 171477/225000 [328:34:47<80:01:51,  5.38s/it]
 76%|███████▌  | 171478/225000 [328:34:51<72:36:52,  4.88s/it]
 76%|███████▌  | 171479/225000 [328:35:02<98:20:25,  6.61s/it]
 76%|███████▌  | 171480/225000 [328:35:05<85:10:44,  5.73s/it]
 76%|███████▌  | 171481/225000 [328:35:09<76:52:46,  5.17s/it]
 76%|███████▌  | 171482/225000 [328:35:14<74:52:06,  5.04s/it]
 76%|███████▌  | 171483/225000 [328:35:19<75:04:55,  5.05s/it]
 76%|███████▌  | 171484/225000 [328:35:24<74:56:38,  5.04s/it]
 76%|███████▌  | 171485/225000 [328:35:31<83:12:18,  5.60s/it]
 76%|███████▌  | 171486/225000 [328:35:42<106:05:14,  7.14s/it]
 76%|███████▌  | 171487/225000 [328:35:45<88:36:44,  5.96s/it] 
 76%|███████▌  | 171488/225000 [328:35:48<78:08:56,  5.26s/it]
 76%|███████▌  | 171489/225000 [328:35:57<94:10:30,  6.34s/it]
 76%|███████▌  | 171490/225000 [328:36:01<84:05:43,  5.66s/it]
 76%|███████▌  | 171491/225000 [328:36:06<79:50:58,  5.37s/it]
 76%|███████▌  | 171492/225000 [328:36:10<74:14:08,  4.99s/it]
 76%|███████▌  | 171493/225000 [328:36:15<72:28:17,  4.88s/it]
 76%|███████▌  | 171494/225000 [328:36:19<69:55:56,  4.71s/it]
 76%|███████▌  | 171495/225000 [328:36:24<69:17:09,  4.66s/it]
 76%|███████▌  | 171496/225000 [328:36:28<68:26:45,  4.61s/it]
 76%|███████▌  | 171497/225000 [328:36:37<89:20:12,  6.01s/it]
 76%|███████▌  | 171498/225000 [328:36:45<97:52:58,  6.59s/it]
 76%|███████▌  | 171499/225000 [328:36:49<86:03:31,  5.79s/it]
 76%|███████▌  | 171500/225000 [328:36:53<78:54:28,  5.31s/it]
                                                              
{'loss': '0.753', 'grad_norm': '1.114', 'learning_rate': '7.278e-06', 'epoch': '0.7622', 'num_input_tokens_seen': 544436680, 'train_runtime': '1.183e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171500/225000 [328:36:53<78:54:28,  5.31s/it]
 76%|███████▌  | 171501/225000 [328:36:57<72:27:34,  4.88s/it]
 76%|███████▌  | 171502/225000 [328:37:08<96:16:54,  6.48s/it]
 76%|███████▌  | 171503/225000 [328:37:12<86:55:13,  5.85s/it]
 76%|███████▌  | 171504/225000 [328:37:16<76:43:14,  5.16s/it]
 76%|███████▌  | 171505/225000 [328:37:22<84:40:23,  5.70s/it]
 76%|███████▌  | 171506/225000 [328:37:27<78:40:08,  5.29s/it]
 76%|███████▌  | 171507/225000 [328:37:31<73:38:29,  4.96s/it]
 76%|███████▌  | 171508/225000 [328:37:35<70:47:10,  4.76s/it]
 76%|███████▌  | 171509/225000 [328:37:41<73:29:32,  4.95s/it]
 76%|███████▌  | 171510/225000 [328:37:52<100:14:48,  6.75s/it]
 76%|███████▌  | 171511/225000 [328:37:55<86:37:44,  5.83s/it] 
 76%|███████▌  | 171512/225000 [328:38:04<97:25:17,  6.56s/it]
 76%|███████▌  | 171513/225000 [328:38:08<89:25:44,  6.02s/it]
 76%|███████▌  | 171514/225000 [328:38:13<81:25:56,  5.48s/it]
 76%|███████▌  | 171515/225000 [328:38:16<74:33:29,  5.02s/it]
 76%|███████▌  | 171516/225000 [328:38:24<86:26:19,  5.82s/it]
 76%|███████▌  | 171517/225000 [328:38:28<75:23:21,  5.07s/it]
 76%|███████▌  | 171518/225000 [328:38:32<72:34:44,  4.89s/it]
 76%|███████▌  | 171519/225000 [328:38:37<72:53:48,  4.91s/it]
 76%|███████▌  | 171520/225000 [328:38:42<73:50:58,  4.97s/it]
 76%|███████▌  | 171521/225000 [328:38:47<74:48:19,  5.04s/it]
 76%|███████▌  | 171522/225000 [328:38:53<76:27:11,  5.15s/it]
 76%|███████▌  | 171523/225000 [328:38:58<76:33:00,  5.15s/it]
 76%|███████▌  | 171524/225000 [328:39:03<76:42:08,  5.16s/it]
 76%|███████▌  | 171525/225000 [328:39:08<77:47:41,  5.24s/it]
 76%|███████▌  | 171526/225000 [328:39:14<79:20:59,  5.34s/it]
 76%|███████▌  | 171527/225000 [328:39:24<101:06:08,  6.81s/it]
 76%|███████▌  | 171528/225000 [328:39:29<90:16:20,  6.08s/it] 
 76%|███████▌  | 171529/225000 [328:39:33<82:15:10,  5.54s/it]
 76%|███████▌  | 171530/225000 [328:39:37<76:27:35,  5.15s/it]
 76%|███████▌  | 171531/225000 [328:39:41<70:47:58,  4.77s/it]
 76%|███████▌  | 171532/225000 [328:39:52<97:53:49,  6.59s/it]
 76%|███████▌  | 171533/225000 [328:39:55<83:57:10,  5.65s/it]
 76%|███████▌  | 171534/225000 [328:39:58<73:01:53,  4.92s/it]
 76%|███████▌  | 171535/225000 [328:40:04<74:06:58,  4.99s/it]
 76%|███████▌  | 171536/225000 [328:40:09<75:22:29,  5.08s/it]
 76%|███████▌  | 171537/225000 [328:40:14<75:09:06,  5.06s/it]
 76%|███████▌  | 171538/225000 [328:40:19<75:20:02,  5.07s/it]
 76%|███████▌  | 171539/225000 [328:40:24<75:54:20,  5.11s/it]
 76%|███████▌  | 171540/225000 [328:40:29<76:27:55,  5.15s/it]
 76%|███████▌  | 171541/225000 [328:40:35<77:01:01,  5.19s/it]
 76%|███████▌  | 171542/225000 [328:40:40<77:05:57,  5.19s/it]
 76%|███████▌  | 171543/225000 [328:40:45<77:34:34,  5.22s/it]
 76%|███████▌  | 171544/225000 [328:40:53<88:26:19,  5.96s/it]
 76%|███████▌  | 171545/225000 [328:41:04<109:09:10,  7.35s/it]
 76%|███████▌  | 171546/225000 [328:41:07<93:29:11,  6.30s/it] 
 76%|███████▌  | 171547/225000 [328:41:11<82:26:24,  5.55s/it]
 76%|███████▌  | 171548/225000 [328:41:16<79:53:51,  5.38s/it]
 76%|███████▌  | 171549/225000 [328:41:21<78:34:10,  5.29s/it]
 76%|███████▌  | 171550/225000 [328:41:27<78:36:56,  5.29s/it]
                                                              
{'loss': '0.7801', 'grad_norm': '1.197', 'learning_rate': '7.266e-06', 'epoch': '0.7624', 'num_input_tokens_seen': 544566664, 'train_runtime': '1.183e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▌  | 171550/225000 [328:41:27<78:36:56,  5.29s/it]
 76%|███████▌  | 171551/225000 [328:41:32<79:34:56,  5.36s/it]
 76%|███████▌  | 171552/225000 [328:41:38<80:13:13,  5.40s/it]
 76%|███████▌  | 171553/225000 [328:41:43<80:22:07,  5.41s/it]
 76%|███████▌  | 171554/225000 [328:41:48<79:13:34,  5.34s/it]
 76%|███████▌  | 171555/225000 [328:41:54<80:31:56,  5.42s/it]
 76%|███████▌  | 171556/225000 [328:41:59<79:17:21,  5.34s/it]
 76%|███████▌  | 171557/225000 [328:42:05<80:54:12,  5.45s/it]
 76%|███████▌  | 171558/225000 [328:42:10<78:51:02,  5.31s/it]
 76%|███████▌  | 171559/225000 [328:42:15<78:34:08,  5.29s/it]
 76%|███████▌  | 171560/225000 [328:42:22<88:25:20,  5.96s/it]
 76%|███████▌  | 171561/225000 [328:42:34<111:50:00,  7.53s/it]
 76%|███████▌  | 171562/225000 [328:42:37<94:23:57,  6.36s/it] 
 76%|███████▋  | 171563/225000 [328:42:41<83:24:00,  5.62s/it]
 76%|███████▋  | 171564/225000 [328:42:50<96:08:11,  6.48s/it]
 76%|███████▋  | 171565/225000 [328:42:54<88:19:38,  5.95s/it]
 76%|███████▋  | 171566/225000 [328:42:58<76:43:36,  5.17s/it]
 76%|███████▋  | 171567/225000 [328:43:02<74:49:46,  5.04s/it]
 76%|███████▋  | 171568/225000 [328:43:07<74:03:40,  4.99s/it]
 76%|███████▋  | 171569/225000 [328:43:12<74:28:19,  5.02s/it]
 76%|███████▋  | 171570/225000 [328:43:17<74:15:39,  5.00s/it]
 76%|███████▋  | 171571/225000 [328:43:23<75:53:52,  5.11s/it]
 76%|███████▋  | 171572/225000 [328:43:28<75:09:57,  5.06s/it]
 76%|███████▋  | 171573/225000 [328:43:33<77:23:48,  5.22s/it]
 76%|███████▋  | 171574/225000 [328:43:39<78:10:20,  5.27s/it]
 76%|███████▋  | 171575/225000 [328:43:44<76:46:06,  5.17s/it]
 76%|███████▋  | 171576/225000 [328:43:49<76:30:52,  5.16s/it]
 76%|███████▋  | 171577/225000 [328:43:54<76:56:21,  5.18s/it]
 76%|███████▋  | 171578/225000 [328:43:59<78:32:27,  5.29s/it]
 76%|███████▋  | 171579/225000 [328:44:05<77:53:56,  5.25s/it]
 76%|███████▋  | 171580/225000 [328:44:10<78:02:52,  5.26s/it]
 76%|███████▋  | 171581/225000 [328:44:15<78:30:42,  5.29s/it]
 76%|███████▋  | 171582/225000 [328:44:20<77:17:16,  5.21s/it]
 76%|███████▋  | 171583/225000 [328:44:25<76:56:41,  5.19s/it]
 76%|███████▋  | 171584/225000 [328:44:31<77:31:08,  5.22s/it]
 76%|███████▋  | 171585/225000 [328:44:40<94:52:03,  6.39s/it]
 76%|███████▋  | 171586/225000 [328:44:48<100:29:16,  6.77s/it]
 76%|███████▋  | 171587/225000 [328:44:54<101:18:59,  6.83s/it]
 76%|███████▋  | 171588/225000 [328:44:59<89:48:44,  6.05s/it] 
 76%|███████▋  | 171589/225000 [328:45:05<90:48:45,  6.12s/it]
 76%|███████▋  | 171590/225000 [328:45:11<91:32:48,  6.17s/it]
 76%|███████▋  | 171591/225000 [328:45:15<80:21:14,  5.42s/it]
 76%|███████▋  | 171592/225000 [328:45:23<90:30:23,  6.10s/it]
 76%|███████▋  | 171593/225000 [328:45:26<78:22:28,  5.28s/it]
 76%|███████▋  | 171594/225000 [328:45:30<74:35:44,  5.03s/it]
 76%|███████▋  | 171595/225000 [328:45:38<85:34:34,  5.77s/it]
 76%|███████▋  | 171596/225000 [328:45:42<79:00:06,  5.33s/it]
 76%|███████▋  | 171597/225000 [328:45:46<72:38:13,  4.90s/it]
 76%|███████▋  | 171598/225000 [328:45:50<69:07:18,  4.66s/it]
 76%|███████▋  | 171599/225000 [328:46:02<99:57:07,  6.74s/it]
 76%|███████▋  | 171600/225000 [328:46:05<85:33:07,  5.77s/it]
                                                              
{'loss': '0.7645', 'grad_norm': '1.217', 'learning_rate': '7.253e-06', 'epoch': '0.7627', 'num_input_tokens_seen': 544696376, 'train_runtime': '1.184e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171600/225000 [328:46:05<85:33:07,  5.77s/it]
 76%|███████▋  | 171601/225000 [328:46:09<75:45:06,  5.11s/it]
 76%|███████▋  | 171602/225000 [328:46:18<94:23:19,  6.36s/it]
 76%|███████▋  | 171603/225000 [328:46:22<81:41:25,  5.51s/it]
 76%|███████▋  | 171604/225000 [328:46:27<79:35:27,  5.37s/it]
 76%|███████▋  | 171605/225000 [328:46:32<79:50:41,  5.38s/it]
 76%|███████▋  | 171606/225000 [328:46:37<78:26:54,  5.29s/it]
 76%|███████▋  | 171607/225000 [328:46:42<75:53:19,  5.12s/it]
 76%|███████▋  | 171608/225000 [328:46:47<73:44:17,  4.97s/it]
 76%|███████▋  | 171609/225000 [328:46:51<72:21:10,  4.88s/it]
 76%|███████▋  | 171610/225000 [328:46:56<71:18:36,  4.81s/it]
 76%|███████▋  | 171611/225000 [328:47:01<71:24:42,  4.82s/it]
 76%|███████▋  | 171612/225000 [328:47:06<72:31:09,  4.89s/it]
 76%|███████▋  | 171613/225000 [328:47:11<73:46:59,  4.98s/it]
 76%|███████▋  | 171614/225000 [328:47:16<74:05:09,  5.00s/it]
 76%|███████▋  | 171615/225000 [328:47:21<74:52:51,  5.05s/it]
 76%|███████▋  | 171616/225000 [328:47:26<75:46:25,  5.11s/it]
 76%|███████▋  | 171617/225000 [328:47:33<81:16:42,  5.48s/it]
 76%|███████▋  | 171618/225000 [328:47:45<111:31:58,  7.52s/it]
 76%|███████▋  | 171619/225000 [328:47:50<100:00:55,  6.75s/it]
 76%|███████▋  | 171620/225000 [328:47:54<85:58:27,  5.80s/it] 
 76%|███████▋  | 171621/225000 [328:48:02<98:09:16,  6.62s/it]
 76%|███████▋  | 171622/225000 [328:48:06<85:49:33,  5.79s/it]
 76%|███████▋  | 171623/225000 [328:48:09<75:46:34,  5.11s/it]
 76%|███████▋  | 171624/225000 [328:48:16<82:06:31,  5.54s/it]
 76%|███████▋  | 171625/225000 [328:48:24<91:32:53,  6.17s/it]
 76%|███████▋  | 171626/225000 [328:48:27<80:05:46,  5.40s/it]
 76%|███████▋  | 171627/225000 [328:48:31<72:24:27,  4.88s/it]
 76%|███████▋  | 171628/225000 [328:48:35<68:06:32,  4.59s/it]
 76%|███████▋  | 171629/225000 [328:48:44<88:15:56,  5.95s/it]
 76%|███████▋  | 171630/225000 [328:48:50<88:15:55,  5.95s/it]
 76%|███████▋  | 171631/225000 [328:48:55<83:41:38,  5.65s/it]
 76%|███████▋  | 171632/225000 [328:49:00<80:38:21,  5.44s/it]
 76%|███████▋  | 171633/225000 [328:49:06<81:59:31,  5.53s/it]
 76%|███████▋  | 171634/225000 [328:49:11<80:46:09,  5.45s/it]
 76%|███████▋  | 171635/225000 [328:49:16<80:00:02,  5.40s/it]
 76%|███████▋  | 171636/225000 [328:49:21<79:06:15,  5.34s/it]
 76%|███████▋  | 171637/225000 [328:49:26<74:30:45,  5.03s/it]
 76%|███████▋  | 171638/225000 [328:49:30<73:41:34,  4.97s/it]
 76%|███████▋  | 171639/225000 [328:49:36<76:24:44,  5.16s/it]
 76%|███████▋  | 171640/225000 [328:49:41<76:08:10,  5.14s/it]
 76%|███████▋  | 171641/225000 [328:49:46<75:32:11,  5.10s/it]
 76%|███████▋  | 171642/225000 [328:49:51<76:10:29,  5.14s/it]
 76%|███████▋  | 171643/225000 [328:50:02<98:55:01,  6.67s/it]
 76%|███████▋  | 171644/225000 [328:50:06<89:03:22,  6.01s/it]
 76%|███████▋  | 171645/225000 [328:50:11<82:04:35,  5.54s/it]
 76%|███████▋  | 171646/225000 [328:50:15<77:04:16,  5.20s/it]
 76%|███████▋  | 171647/225000 [328:50:19<73:20:35,  4.95s/it]
 76%|███████▋  | 171648/225000 [328:50:25<76:23:25,  5.15s/it]
 76%|███████▋  | 171649/225000 [328:50:37<105:14:58,  7.10s/it]
 76%|███████▋  | 171650/225000 [328:50:41<91:18:40,  6.16s/it] 
                                                              
{'loss': '0.7665', 'grad_norm': '1.124', 'learning_rate': '7.24e-06', 'epoch': '0.7629', 'num_input_tokens_seen': 544826640, 'train_runtime': '1.184e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171650/225000 [328:50:41<91:18:40,  6.16s/it]
 76%|███████▋  | 171651/225000 [328:50:45<83:39:06,  5.64s/it]
 76%|███████▋  | 171652/225000 [328:50:50<80:40:01,  5.44s/it]
 76%|███████▋  | 171653/225000 [328:50:55<80:40:38,  5.44s/it]
 76%|███████▋  | 171654/225000 [328:51:01<79:21:07,  5.36s/it]
 76%|███████▋  | 171655/225000 [328:51:06<79:17:04,  5.35s/it]
 76%|███████▋  | 171656/225000 [328:51:11<79:39:55,  5.38s/it]
 76%|███████▋  | 171657/225000 [328:51:17<79:43:27,  5.38s/it]
 76%|███████▋  | 171658/225000 [328:51:22<78:30:48,  5.30s/it]
 76%|███████▋  | 171659/225000 [328:51:27<77:49:04,  5.25s/it]
 76%|███████▋  | 171660/225000 [328:51:32<75:57:08,  5.13s/it]
 76%|███████▋  | 171661/225000 [328:51:41<92:32:58,  6.25s/it]
 76%|███████▋  | 171662/225000 [328:51:46<88:43:05,  5.99s/it]
 76%|███████▋  | 171663/225000 [328:51:51<86:07:19,  5.81s/it]
 76%|███████▋  | 171664/225000 [328:51:57<84:57:56,  5.73s/it]
 76%|███████▋  | 171665/225000 [328:52:02<82:39:35,  5.58s/it]
 76%|███████▋  | 171666/225000 [328:52:07<78:41:39,  5.31s/it]
 76%|███████▋  | 171667/225000 [328:52:12<76:24:56,  5.16s/it]
 76%|███████▋  | 171668/225000 [328:52:17<77:27:41,  5.23s/it]
 76%|███████▋  | 171669/225000 [328:52:22<77:59:14,  5.26s/it]
 76%|███████▋  | 171670/225000 [328:52:29<81:55:10,  5.53s/it]
 76%|███████▋  | 171671/225000 [328:52:34<79:09:16,  5.34s/it]
 76%|███████▋  | 171672/225000 [328:52:39<78:13:06,  5.28s/it]
 76%|███████▋  | 171673/225000 [328:52:44<78:42:55,  5.31s/it]
 76%|███████▋  | 171674/225000 [328:52:50<79:47:53,  5.39s/it]
 76%|███████▋  | 171675/225000 [328:52:55<81:04:37,  5.47s/it]
 76%|███████▋  | 171676/225000 [328:53:01<81:17:38,  5.49s/it]
 76%|███████▋  | 171677/225000 [328:53:09<93:13:50,  6.29s/it]
 76%|███████▋  | 171678/225000 [328:53:15<94:16:57,  6.37s/it]
 76%|███████▋  | 171679/225000 [328:53:20<85:00:24,  5.74s/it]
 76%|███████▋  | 171680/225000 [328:53:24<79:43:43,  5.38s/it]
 76%|███████▋  | 171681/225000 [328:53:28<73:51:32,  4.99s/it]
 76%|███████▋  | 171682/225000 [328:53:38<94:06:33,  6.35s/it]
 76%|███████▋  | 171683/225000 [328:53:42<82:08:09,  5.55s/it]
 76%|███████▋  | 171684/225000 [328:53:49<90:18:55,  6.10s/it]
 76%|███████▋  | 171685/225000 [328:53:54<86:44:09,  5.86s/it]
 76%|███████▋  | 171686/225000 [328:54:00<85:20:46,  5.76s/it]
 76%|███████▋  | 171687/225000 [328:54:05<82:35:35,  5.58s/it]
 76%|███████▋  | 171688/225000 [328:54:10<81:57:46,  5.53s/it]
 76%|███████▋  | 171689/225000 [328:54:15<78:56:33,  5.33s/it]
 76%|███████▋  | 171690/225000 [328:54:21<79:04:17,  5.34s/it]
 76%|███████▋  | 171691/225000 [328:54:26<78:09:12,  5.28s/it]
 76%|███████▋  | 171692/225000 [328:54:30<74:37:45,  5.04s/it]
 76%|███████▋  | 171693/225000 [328:54:35<74:15:13,  5.01s/it]
 76%|███████▋  | 171694/225000 [328:54:40<74:28:02,  5.03s/it]
 76%|███████▋  | 171695/225000 [328:54:46<76:54:21,  5.19s/it]
 76%|███████▋  | 171696/225000 [328:54:51<76:53:30,  5.19s/it]
 76%|███████▋  | 171697/225000 [328:54:56<77:38:34,  5.24s/it]
 76%|███████▋  | 171698/225000 [328:55:02<77:25:43,  5.23s/it]
 76%|███████▋  | 171699/225000 [328:55:06<75:35:48,  5.11s/it]
 76%|███████▋  | 171700/225000 [328:55:12<75:39:40,  5.11s/it]
                                                              
{'loss': '0.7775', 'grad_norm': '1.202', 'learning_rate': '7.227e-06', 'epoch': '0.7631', 'num_input_tokens_seen': 544957584, 'train_runtime': '1.184e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171700/225000 [328:55:12<75:39:40,  5.11s/it]
 76%|███████▋  | 171701/225000 [328:55:17<76:01:46,  5.14s/it]
 76%|███████▋  | 171702/225000 [328:55:22<75:22:51,  5.09s/it]
 76%|███████▋  | 171703/225000 [328:55:27<75:46:19,  5.12s/it]
 76%|███████▋  | 171704/225000 [328:55:32<76:13:07,  5.15s/it]
 76%|███████▋  | 171705/225000 [328:55:43<100:39:22,  6.80s/it]
 76%|███████▋  | 171706/225000 [328:55:48<93:23:08,  6.31s/it] 
 76%|███████▋  | 171707/225000 [328:55:53<89:31:21,  6.05s/it]
 76%|███████▋  | 171708/225000 [328:55:59<87:06:43,  5.88s/it]
 76%|███████▋  | 171709/225000 [328:56:04<83:22:09,  5.63s/it]
 76%|███████▋  | 171710/225000 [328:56:09<81:57:24,  5.54s/it]
 76%|███████▋  | 171711/225000 [328:56:15<81:18:06,  5.49s/it]
 76%|███████▋  | 171712/225000 [328:56:20<80:23:27,  5.43s/it]
 76%|███████▋  | 171713/225000 [328:56:25<78:40:22,  5.32s/it]
 76%|███████▋  | 171714/225000 [328:56:30<77:32:51,  5.24s/it]
 76%|███████▋  | 171715/225000 [328:56:35<76:56:19,  5.20s/it]
 76%|███████▋  | 171716/225000 [328:56:40<74:57:47,  5.06s/it]
 76%|███████▋  | 171717/225000 [328:56:45<73:52:34,  4.99s/it]
 76%|███████▋  | 171718/225000 [328:56:50<73:37:38,  4.97s/it]
 76%|███████▋  | 171719/225000 [328:56:55<75:30:54,  5.10s/it]
 76%|███████▋  | 171720/225000 [328:57:01<78:54:57,  5.33s/it]
 76%|███████▋  | 171721/225000 [328:57:07<83:15:53,  5.63s/it]
 76%|███████▋  | 171722/225000 [328:57:20<113:19:54,  7.66s/it]
 76%|███████▋  | 171723/225000 [328:57:24<98:27:19,  6.65s/it] 
 76%|███████▋  | 171724/225000 [328:57:32<102:59:53,  6.96s/it]
 76%|███████▋  | 171725/225000 [328:57:36<93:46:08,  6.34s/it] 
 76%|███████▋  | 171726/225000 [328:57:41<87:21:58,  5.90s/it]
 76%|███████▋  | 171727/225000 [328:57:46<83:26:30,  5.64s/it]
 76%|███████▋  | 171728/225000 [328:57:51<80:36:44,  5.45s/it]
 76%|███████▋  | 171729/225000 [328:57:56<77:24:21,  5.23s/it]
 76%|███████▋  | 171730/225000 [328:58:01<76:19:18,  5.16s/it]
 76%|███████▋  | 171731/225000 [328:58:06<75:38:53,  5.11s/it]
 76%|███████▋  | 171732/225000 [328:58:11<74:41:24,  5.05s/it]
 76%|███████▋  | 171733/225000 [328:58:15<71:41:30,  4.85s/it]
 76%|███████▋  | 171734/225000 [328:58:20<72:43:42,  4.92s/it]
 76%|███████▋  | 171735/225000 [328:58:25<71:55:16,  4.86s/it]
 76%|███████▋  | 171736/225000 [328:58:30<72:31:40,  4.90s/it]
 76%|███████▋  | 171737/225000 [328:58:40<93:25:07,  6.31s/it]
 76%|███████▋  | 171738/225000 [328:58:47<96:15:11,  6.51s/it]
 76%|███████▋  | 171739/225000 [328:58:50<82:51:58,  5.60s/it]
 76%|███████▋  | 171740/225000 [328:58:54<73:37:16,  4.98s/it]
 76%|███████▋  | 171741/225000 [328:59:01<85:33:49,  5.78s/it]
 76%|███████▋  | 171742/225000 [328:59:08<87:16:35,  5.90s/it]
 76%|███████▋  | 171743/225000 [328:59:11<76:37:56,  5.18s/it]
 76%|███████▋  | 171744/225000 [328:59:19<89:10:46,  6.03s/it]
 76%|███████▋  | 171745/225000 [328:59:24<82:20:00,  5.57s/it]
 76%|███████▋  | 171746/225000 [328:59:30<86:19:11,  5.84s/it]
 76%|███████▋  | 171747/225000 [328:59:35<84:20:29,  5.70s/it]
 76%|███████▋  | 171748/225000 [328:59:39<76:41:53,  5.19s/it]
 76%|███████▋  | 171749/225000 [328:59:44<74:30:18,  5.04s/it]
 76%|███████▋  | 171750/225000 [328:59:49<73:26:42,  4.97s/it]
                                                              
{'loss': '0.7706', 'grad_norm': '1.077', 'learning_rate': '7.214e-06', 'epoch': '0.7633', 'num_input_tokens_seen': 545088080, 'train_runtime': '1.184e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171750/225000 [328:59:49<73:26:42,  4.97s/it]
 76%|███████▋  | 171751/225000 [328:59:54<71:54:19,  4.86s/it]
 76%|███████▋  | 171752/225000 [328:59:58<69:04:51,  4.67s/it]
 76%|███████▋  | 171753/225000 [329:00:02<67:00:15,  4.53s/it]
 76%|███████▋  | 171754/225000 [329:00:14<99:10:00,  6.70s/it]
 76%|███████▋  | 171755/225000 [329:00:18<89:42:44,  6.07s/it]
 76%|███████▋  | 171756/225000 [329:00:22<79:27:22,  5.37s/it]
 76%|███████▋  | 171757/225000 [329:00:30<91:53:57,  6.21s/it]
 76%|███████▋  | 171758/225000 [329:00:34<82:40:33,  5.59s/it]
 76%|███████▋  | 171759/225000 [329:00:39<76:25:31,  5.17s/it]
 76%|███████▋  | 171760/225000 [329:00:42<69:40:43,  4.71s/it]
 76%|███████▋  | 171761/225000 [329:00:51<88:21:03,  5.97s/it]
 76%|███████▋  | 171762/225000 [329:00:55<80:56:14,  5.47s/it]
 76%|███████▋  | 171763/225000 [329:01:02<87:26:16,  5.91s/it]
 76%|███████▋  | 171764/225000 [329:01:09<90:13:40,  6.10s/it]
 76%|███████▋  | 171765/225000 [329:01:13<79:16:53,  5.36s/it]
 76%|███████▋  | 171766/225000 [329:01:20<87:55:22,  5.95s/it]
 76%|███████▋  | 171767/225000 [329:01:24<82:03:31,  5.55s/it]
 76%|███████▋  | 171768/225000 [329:01:29<76:43:24,  5.19s/it]
 76%|███████▋  | 171769/225000 [329:01:34<74:53:31,  5.06s/it]
 76%|███████▋  | 171770/225000 [329:01:39<74:53:16,  5.06s/it]
 76%|███████▋  | 171771/225000 [329:01:43<73:40:41,  4.98s/it]
 76%|███████▋  | 171772/225000 [329:01:48<73:58:11,  5.00s/it]
 76%|███████▋  | 171773/225000 [329:01:53<73:07:18,  4.95s/it]
 76%|███████▋  | 171774/225000 [329:02:03<92:37:31,  6.26s/it]
 76%|███████▋  | 171775/225000 [329:02:09<91:40:19,  6.20s/it]
 76%|███████▋  | 171776/225000 [329:02:12<78:52:14,  5.33s/it]
 76%|███████▋  | 171777/225000 [329:02:20<92:51:06,  6.28s/it]
 76%|███████▋  | 171778/225000 [329:02:25<84:59:42,  5.75s/it]
 76%|███████▋  | 171779/225000 [329:02:30<80:03:46,  5.42s/it]
 76%|███████▋  | 171780/225000 [329:02:34<76:32:52,  5.18s/it]
 76%|███████▋  | 171781/225000 [329:02:39<73:29:06,  4.97s/it]
 76%|███████▋  | 171782/225000 [329:02:44<73:20:41,  4.96s/it]
 76%|███████▋  | 171783/225000 [329:02:48<72:24:19,  4.90s/it]
 76%|███████▋  | 171784/225000 [329:02:53<71:14:32,  4.82s/it]
 76%|███████▋  | 171785/225000 [329:03:05<102:03:50,  6.90s/it]
 76%|███████▋  | 171786/225000 [329:03:10<93:16:51,  6.31s/it] 
 76%|███████▋  | 171787/225000 [329:03:13<80:23:40,  5.44s/it]
 76%|███████▋  | 171788/225000 [329:03:22<94:47:43,  6.41s/it]
 76%|███████▋  | 171789/225000 [329:03:27<90:15:04,  6.11s/it]
 76%|███████▋  | 171790/225000 [329:03:31<78:38:54,  5.32s/it]
 76%|███████▋  | 171791/225000 [329:03:39<89:57:19,  6.09s/it]
 76%|███████▋  | 171792/225000 [329:03:44<87:20:43,  5.91s/it]
 76%|███████▋  | 171793/225000 [329:03:49<83:58:17,  5.68s/it]
 76%|███████▋  | 171794/225000 [329:03:54<79:52:43,  5.40s/it]
 76%|███████▋  | 171795/225000 [329:03:59<76:07:26,  5.15s/it]
 76%|███████▋  | 171796/225000 [329:04:04<76:35:30,  5.18s/it]
 76%|███████▋  | 171797/225000 [329:04:09<78:15:01,  5.29s/it]
 76%|███████▋  | 171798/225000 [329:04:14<76:12:35,  5.16s/it]
 76%|███████▋  | 171799/225000 [329:04:19<74:52:32,  5.07s/it]
 76%|███████▋  | 171800/225000 [329:04:25<77:13:33,  5.23s/it]
                                                              
{'loss': '0.7641', 'grad_norm': '1.146', 'learning_rate': '7.201e-06', 'epoch': '0.7636', 'num_input_tokens_seen': 545217200, 'train_runtime': '1.185e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171800/225000 [329:04:25<77:13:33,  5.23s/it]
 76%|███████▋  | 171801/225000 [329:04:30<76:05:38,  5.15s/it]
 76%|███████▋  | 171802/225000 [329:04:35<75:52:32,  5.13s/it]
 76%|███████▋  | 171803/225000 [329:04:40<76:48:50,  5.20s/it]
 76%|███████▋  | 171804/225000 [329:04:48<87:56:41,  5.95s/it]
 76%|███████▋  | 171805/225000 [329:04:55<91:32:12,  6.19s/it]
 76%|███████▋  | 171806/225000 [329:04:59<83:08:23,  5.63s/it]
 76%|███████▋  | 171807/225000 [329:05:03<77:27:46,  5.24s/it]
 76%|███████▋  | 171808/225000 [329:05:07<72:18:18,  4.89s/it]
 76%|███████▋  | 171809/225000 [329:05:14<80:23:13,  5.44s/it]
 76%|███████▋  | 171810/225000 [329:05:23<95:38:33,  6.47s/it]
 76%|███████▋  | 171811/225000 [329:05:29<93:52:16,  6.35s/it]
 76%|███████▋  | 171812/225000 [329:05:36<98:29:27,  6.67s/it]
 76%|███████▋  | 171813/225000 [329:05:41<91:32:57,  6.20s/it]
 76%|███████▋  | 171814/225000 [329:05:46<84:44:31,  5.74s/it]
 76%|███████▋  | 171815/225000 [329:05:51<79:46:19,  5.40s/it]
 76%|███████▋  | 171816/225000 [329:05:55<76:21:35,  5.17s/it]
 76%|███████▋  | 171817/225000 [329:06:00<72:38:11,  4.92s/it]
 76%|███████▋  | 171818/225000 [329:06:04<71:32:58,  4.84s/it]
 76%|███████▋  | 171819/225000 [329:06:09<72:10:35,  4.89s/it]
 76%|███████▋  | 171820/225000 [329:06:21<102:50:11,  6.96s/it]
 76%|███████▋  | 171821/225000 [329:06:25<87:11:59,  5.90s/it] 
 76%|███████▋  | 171822/225000 [329:06:29<81:00:21,  5.48s/it]
 76%|███████▋  | 171823/225000 [329:06:34<77:48:19,  5.27s/it]
 76%|███████▋  | 171824/225000 [329:06:38<74:27:46,  5.04s/it]
 76%|███████▋  | 171825/225000 [329:06:43<72:39:37,  4.92s/it]
 76%|███████▋  | 171826/225000 [329:06:48<72:08:46,  4.88s/it]
 76%|███████▋  | 171827/225000 [329:06:54<77:58:19,  5.28s/it]
 76%|███████▋  | 171828/225000 [329:07:03<93:11:41,  6.31s/it]
 76%|███████▋  | 171829/225000 [329:07:06<79:25:36,  5.38s/it]
 76%|███████▋  | 171830/225000 [329:07:13<88:23:48,  5.99s/it]
 76%|███████▋  | 171831/225000 [329:07:19<86:54:04,  5.88s/it]
 76%|███████▋  | 171832/225000 [329:07:22<76:07:59,  5.15s/it]
 76%|███████▋  | 171833/225000 [329:07:27<71:49:04,  4.86s/it]
 76%|███████▋  | 171834/225000 [329:07:31<70:26:05,  4.77s/it]
 76%|███████▋  | 171835/225000 [329:07:36<69:53:14,  4.73s/it]
 76%|███████▋  | 171836/225000 [329:07:43<80:13:16,  5.43s/it]
 76%|███████▋  | 171837/225000 [329:07:52<94:59:31,  6.43s/it]
 76%|███████▋  | 171838/225000 [329:07:55<81:04:11,  5.49s/it]
 76%|███████▋  | 171839/225000 [329:08:00<77:08:01,  5.22s/it]
 76%|███████▋  | 171840/225000 [329:08:08<92:55:36,  6.29s/it]
 76%|███████▋  | 171841/225000 [329:08:12<81:14:58,  5.50s/it]
 76%|███████▋  | 171842/225000 [329:08:17<80:38:03,  5.46s/it]
 76%|███████▋  | 171843/225000 [329:08:23<81:20:58,  5.51s/it]
 76%|███████▋  | 171844/225000 [329:08:27<73:19:23,  4.97s/it]
 76%|███████▋  | 171845/225000 [329:08:35<87:53:58,  5.95s/it]
 76%|███████▋  | 171846/225000 [329:08:38<76:49:27,  5.20s/it]
 76%|███████▋  | 171847/225000 [329:08:43<74:22:35,  5.04s/it]
 76%|███████▋  | 171848/225000 [329:08:48<74:50:32,  5.07s/it]
 76%|███████▋  | 171849/225000 [329:08:53<74:07:13,  5.02s/it]
 76%|███████▋  | 171850/225000 [329:08:58<74:32:14,  5.05s/it]
                                                              
{'loss': '0.764', 'grad_norm': '1.195', 'learning_rate': '7.189e-06', 'epoch': '0.7638', 'num_input_tokens_seen': 545347640, 'train_runtime': '1.185e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171850/225000 [329:08:58<74:32:14,  5.05s/it]
 76%|███████▋  | 171851/225000 [329:09:04<75:45:39,  5.13s/it]
 76%|███████▋  | 171852/225000 [329:09:09<77:07:23,  5.22s/it]
 76%|███████▋  | 171853/225000 [329:09:15<79:31:32,  5.39s/it]
 76%|███████▋  | 171854/225000 [329:09:20<78:03:15,  5.29s/it]
 76%|███████▋  | 171855/225000 [329:09:25<76:11:52,  5.16s/it]
 76%|███████▋  | 171856/225000 [329:09:30<76:10:02,  5.16s/it]
 76%|███████▋  | 171857/225000 [329:09:35<77:20:57,  5.24s/it]
 76%|███████▋  | 171858/225000 [329:09:40<76:54:57,  5.21s/it]
 76%|███████▋  | 171859/225000 [329:09:46<77:38:46,  5.26s/it]
 76%|███████▋  | 171860/225000 [329:09:55<94:46:09,  6.42s/it]
 76%|███████▋  | 171861/225000 [329:10:00<90:01:03,  6.10s/it]
 76%|███████▋  | 171862/225000 [329:10:06<87:05:22,  5.90s/it]
 76%|███████▋  | 171863/225000 [329:10:11<83:33:05,  5.66s/it]
 76%|███████▋  | 171864/225000 [329:10:17<84:01:34,  5.69s/it]
 76%|███████▋  | 171865/225000 [329:10:22<82:59:13,  5.62s/it]
 76%|███████▋  | 171866/225000 [329:10:27<82:10:44,  5.57s/it]
 76%|███████▋  | 171867/225000 [329:10:32<78:53:49,  5.35s/it]
 76%|███████▋  | 171868/225000 [329:10:40<90:24:39,  6.13s/it]
 76%|███████▋  | 171869/225000 [329:10:46<88:01:13,  5.96s/it]
 76%|███████▋  | 171870/225000 [329:10:50<81:35:28,  5.53s/it]
 76%|███████▋  | 171871/225000 [329:10:55<79:48:14,  5.41s/it]
 76%|███████▋  | 171872/225000 [329:11:01<80:04:41,  5.43s/it]
 76%|███████▋  | 171873/225000 [329:11:06<79:40:44,  5.40s/it]
 76%|███████▋  | 171874/225000 [329:11:11<77:39:07,  5.26s/it]
 76%|███████▋  | 171875/225000 [329:11:17<79:21:16,  5.38s/it]
 76%|███████▋  | 171876/225000 [329:11:22<80:33:39,  5.46s/it]
 76%|███████▋  | 171877/225000 [329:11:28<81:51:39,  5.55s/it]
 76%|███████▋  | 171878/225000 [329:11:34<82:12:41,  5.57s/it]
 76%|███████▋  | 171879/225000 [329:11:39<81:20:24,  5.51s/it]
 76%|███████▋  | 171880/225000 [329:11:45<80:40:10,  5.47s/it]
 76%|███████▋  | 171881/225000 [329:11:50<78:57:15,  5.35s/it]
 76%|███████▋  | 171882/225000 [329:11:55<77:29:22,  5.25s/it]
 76%|███████▋  | 171883/225000 [329:12:00<78:43:35,  5.34s/it]
 76%|███████▋  | 171884/225000 [329:12:06<80:12:00,  5.44s/it]
 76%|███████▋  | 171885/225000 [329:12:11<78:39:51,  5.33s/it]
 76%|███████▋  | 171886/225000 [329:12:16<79:30:11,  5.39s/it]
 76%|███████▋  | 171887/225000 [329:12:24<90:00:28,  6.10s/it]
 76%|███████▋  | 171888/225000 [329:12:30<87:09:44,  5.91s/it]
 76%|███████▋  | 171889/225000 [329:12:35<82:15:36,  5.58s/it]
 76%|███████▋  | 171890/225000 [329:12:40<81:31:32,  5.53s/it]
 76%|███████▋  | 171891/225000 [329:12:45<81:22:34,  5.52s/it]
 76%|███████▋  | 171892/225000 [329:12:51<79:48:00,  5.41s/it]
 76%|███████▋  | 171893/225000 [329:12:56<80:16:36,  5.44s/it]
 76%|███████▋  | 171894/225000 [329:13:01<78:44:15,  5.34s/it]
 76%|███████▋  | 171895/225000 [329:13:10<96:09:00,  6.52s/it]
 76%|███████▋  | 171896/225000 [329:13:15<88:24:48,  5.99s/it]
 76%|███████▋  | 171897/225000 [329:13:22<91:48:17,  6.22s/it]
 76%|███████▋  | 171898/225000 [329:13:26<81:02:34,  5.49s/it]
 76%|███████▋  | 171899/225000 [329:13:32<84:46:32,  5.75s/it]
 76%|███████▋  | 171900/225000 [329:13:40<92:17:01,  6.26s/it]
                                                              
{'loss': '0.7782', 'grad_norm': '1.109', 'learning_rate': '7.176e-06', 'epoch': '0.764', 'num_input_tokens_seen': 545477160, 'train_runtime': '1.185e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171900/225000 [329:13:40<92:17:01,  6.26s/it]
 76%|███████▋  | 171901/225000 [329:13:44<83:47:00,  5.68s/it]
 76%|███████▋  | 171902/225000 [329:13:50<83:38:31,  5.67s/it]
 76%|███████▋  | 171903/225000 [329:13:53<74:23:13,  5.04s/it]
 76%|███████▋  | 171904/225000 [329:13:59<79:18:38,  5.38s/it]
 76%|███████▋  | 171905/225000 [329:14:05<82:07:05,  5.57s/it]
 76%|███████▋  | 171906/225000 [329:14:10<80:20:19,  5.45s/it]
 76%|███████▋  | 171907/225000 [329:14:19<91:46:11,  6.22s/it]
 76%|███████▋  | 171908/225000 [329:14:22<77:40:47,  5.27s/it]
 76%|███████▋  | 171909/225000 [329:14:28<82:26:10,  5.59s/it]
 76%|███████▋  | 171910/225000 [329:14:33<81:20:16,  5.52s/it]
 76%|███████▋  | 171911/225000 [329:14:37<75:31:14,  5.12s/it]
 76%|███████▋  | 171912/225000 [329:14:44<83:17:12,  5.65s/it]
 76%|███████▋  | 171913/225000 [329:14:50<84:14:52,  5.71s/it]
 76%|███████▋  | 171914/225000 [329:14:54<75:10:33,  5.10s/it]
 76%|███████▋  | 171915/225000 [329:14:58<73:06:49,  4.96s/it]
 76%|███████▋  | 171916/225000 [329:15:08<93:57:32,  6.37s/it]
 76%|███████▋  | 171917/225000 [329:15:12<82:42:09,  5.61s/it]
 76%|███████▋  | 171918/225000 [329:15:16<75:09:41,  5.10s/it]
 76%|███████▋  | 171919/225000 [329:15:20<70:46:32,  4.80s/it]
 76%|███████▋  | 171920/225000 [329:15:24<66:58:58,  4.54s/it]
 76%|███████▋  | 171921/225000 [329:15:35<97:10:11,  6.59s/it]
 76%|███████▋  | 171922/225000 [329:15:39<84:50:01,  5.75s/it]
 76%|███████▋  | 171923/225000 [329:15:43<76:54:44,  5.22s/it]
 76%|███████▋  | 171924/225000 [329:15:47<70:53:01,  4.81s/it]
 76%|███████▋  | 171925/225000 [329:15:57<94:08:20,  6.39s/it]
 76%|███████▋  | 171926/225000 [329:16:00<81:19:43,  5.52s/it]
 76%|███████▋  | 171927/225000 [329:16:08<89:04:00,  6.04s/it]
 76%|███████▋  | 171928/225000 [329:16:12<79:02:40,  5.36s/it]
 76%|███████▋  | 171929/225000 [329:16:17<77:51:44,  5.28s/it]
 76%|███████▋  | 171930/225000 [329:16:22<77:31:08,  5.26s/it]
 76%|███████▋  | 171931/225000 [329:16:26<70:44:59,  4.80s/it]
 76%|███████▋  | 171932/225000 [329:16:31<73:19:39,  4.97s/it]
 76%|███████▋  | 171933/225000 [329:16:39<85:48:38,  5.82s/it]
 76%|███████▋  | 171934/225000 [329:16:42<73:49:48,  5.01s/it]
 76%|███████▋  | 171935/225000 [329:16:48<80:49:00,  5.48s/it]
 76%|███████▋  | 171936/225000 [329:16:54<80:01:23,  5.43s/it]
 76%|███████▋  | 171937/225000 [329:17:00<85:37:28,  5.81s/it]
 76%|███████▋  | 171938/225000 [329:17:06<85:30:01,  5.80s/it]
 76%|███████▋  | 171939/225000 [329:17:12<84:17:59,  5.72s/it]
 76%|███████▋  | 171940/225000 [329:17:17<81:05:16,  5.50s/it]
 76%|███████▋  | 171941/225000 [329:17:22<79:25:53,  5.39s/it]
 76%|███████▋  | 171942/225000 [329:17:27<79:18:34,  5.38s/it]
 76%|███████▋  | 171943/225000 [329:17:33<79:32:37,  5.40s/it]
 76%|███████▋  | 171944/225000 [329:17:38<78:19:02,  5.31s/it]
 76%|███████▋  | 171945/225000 [329:17:43<77:28:02,  5.26s/it]
 76%|███████▋  | 171946/225000 [329:17:48<77:28:46,  5.26s/it]
 76%|███████▋  | 171947/225000 [329:17:53<75:56:32,  5.15s/it]
 76%|███████▋  | 171948/225000 [329:17:58<76:31:19,  5.19s/it]
 76%|███████▋  | 171949/225000 [329:18:03<76:18:27,  5.18s/it]
 76%|███████▋  | 171950/225000 [329:18:09<77:32:17,  5.26s/it]
                                                              
{'loss': '0.7788', 'grad_norm': '1.3', 'learning_rate': '7.163e-06', 'epoch': '0.7642', 'num_input_tokens_seen': 545605480, 'train_runtime': '1.185e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 171950/225000 [329:18:09<77:32:17,  5.26s/it]
 76%|███████▋  | 171951/225000 [329:18:14<77:28:47,  5.26s/it]
 76%|███████▋  | 171952/225000 [329:18:25<99:55:05,  6.78s/it]
 76%|███████▋  | 171953/225000 [329:18:30<94:45:15,  6.43s/it]
 76%|███████▋  | 171954/225000 [329:18:34<82:30:10,  5.60s/it]
 76%|███████▋  | 171955/225000 [329:18:37<72:53:28,  4.95s/it]
 76%|███████▋  | 171956/225000 [329:18:46<91:24:08,  6.20s/it]
 76%|███████▋  | 171957/225000 [329:18:50<81:12:24,  5.51s/it]
 76%|███████▋  | 171958/225000 [329:18:54<75:02:21,  5.09s/it]
 76%|███████▋  | 171959/225000 [329:19:02<85:51:21,  5.83s/it]
 76%|███████▋  | 171960/225000 [329:19:08<86:30:13,  5.87s/it]
 76%|███████▋  | 171961/225000 [329:19:14<86:30:38,  5.87s/it]
 76%|███████▋  | 171962/225000 [329:19:18<78:36:08,  5.34s/it]
 76%|███████▋  | 171963/225000 [329:19:22<73:22:31,  4.98s/it]
 76%|███████▋  | 171964/225000 [329:19:31<89:49:19,  6.10s/it]
 76%|███████▋  | 171965/225000 [329:19:35<82:47:53,  5.62s/it]
 76%|███████▋  | 171966/225000 [329:19:38<72:12:37,  4.90s/it]
 76%|███████▋  | 171967/225000 [329:19:45<79:29:42,  5.40s/it]
 76%|███████▋  | 171968/225000 [329:19:49<73:21:44,  4.98s/it]
 76%|███████▋  | 171969/225000 [329:19:53<69:55:53,  4.75s/it]
 76%|███████▋  | 171970/225000 [329:19:58<69:42:16,  4.73s/it]
 76%|███████▋  | 171971/225000 [329:20:03<71:07:19,  4.83s/it]
 76%|███████▋  | 171972/225000 [329:20:08<72:14:09,  4.90s/it]
 76%|███████▋  | 171973/225000 [329:20:13<72:03:21,  4.89s/it]
 76%|███████▋  | 171974/225000 [329:20:18<72:40:34,  4.93s/it]
 76%|███████▋  | 171975/225000 [329:20:23<73:32:05,  4.99s/it]
 76%|███████▋  | 171976/225000 [329:20:28<72:27:30,  4.92s/it]
 76%|███████▋  | 171977/225000 [329:20:37<89:21:15,  6.07s/it]
 76%|███████▋  | 171978/225000 [329:20:45<101:27:18,  6.89s/it]
 76%|███████▋  | 171979/225000 [329:20:52<101:00:27,  6.86s/it]
 76%|███████▋  | 171980/225000 [329:20:56<86:06:47,  5.85s/it] 
 76%|███████▋  | 171981/225000 [329:21:02<88:28:53,  6.01s/it]
 76%|███████▋  | 171982/225000 [329:21:08<87:03:36,  5.91s/it]
 76%|███████▋  | 171983/225000 [329:21:11<76:49:42,  5.22s/it]
 76%|███████▋  | 171984/225000 [329:21:20<93:40:20,  6.36s/it]
 76%|███████▋  | 171985/225000 [329:21:24<79:48:27,  5.42s/it]
 76%|███████▋  | 171986/225000 [329:21:30<85:06:48,  5.78s/it]
 76%|███████▋  | 171987/225000 [329:21:37<87:44:33,  5.96s/it]
 76%|███████▋  | 171988/225000 [329:21:43<88:11:49,  5.99s/it]
 76%|███████▋  | 171989/225000 [329:21:46<75:12:41,  5.11s/it]
 76%|███████▋  | 171990/225000 [329:21:51<75:24:41,  5.12s/it]
 76%|███████▋  | 171991/225000 [329:21:56<75:54:18,  5.15s/it]
 76%|███████▋  | 171992/225000 [329:22:01<76:33:30,  5.20s/it]
 76%|███████▋  | 171993/225000 [329:22:06<74:05:28,  5.03s/it]
 76%|███████▋  | 171994/225000 [329:22:11<73:51:04,  5.02s/it]
 76%|███████▋  | 171995/225000 [329:22:16<72:16:06,  4.91s/it]
 76%|███████▋  | 171996/225000 [329:22:22<77:58:23,  5.30s/it]
 76%|███████▋  | 171997/225000 [329:22:31<93:45:43,  6.37s/it]
 76%|███████▋  | 171998/225000 [329:22:34<80:18:02,  5.45s/it]
 76%|███████▋  | 171999/225000 [329:22:41<86:24:06,  5.87s/it]
 76%|███████▋  | 172000/225000 [329:22:48<93:10:35,  6.33s/it]
                                                              
{'loss': '0.7661', 'grad_norm': '1.142', 'learning_rate': '7.15e-06', 'epoch': '0.7644', 'num_input_tokens_seen': 545735896, 'train_runtime': '1.186e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 172000/225000 [329:22:48<93:10:35,  6.33s/it]
 76%|███████▋  | 172001/225000 [329:22:52<80:39:29,  5.48s/it]
 76%|███████▋  | 172002/225000 [329:22:59<87:31:18,  5.95s/it]
 76%|███████▋  | 172003/225000 [329:23:04<82:43:35,  5.62s/it]
 76%|███████▋  | 172004/225000 [329:23:08<79:04:15,  5.37s/it]
 76%|███████▋  | 172005/225000 [329:23:14<80:30:56,  5.47s/it]
 76%|███████▋  | 172006/225000 [329:23:20<81:11:57,  5.52s/it]
 76%|███████▋  | 172007/225000 [329:23:25<80:38:57,  5.48s/it]
 76%|███████▋  | 172008/225000 [329:23:31<81:17:24,  5.52s/it]
 76%|███████▋  | 172009/225000 [329:23:36<79:07:31,  5.38s/it]
 76%|███████▋  | 172010/225000 [329:23:41<79:24:00,  5.39s/it]
 76%|███████▋  | 172011/225000 [329:23:46<78:11:50,  5.31s/it]
 76%|███████▋  | 172012/225000 [329:23:51<76:44:52,  5.21s/it]
 76%|███████▋  | 172013/225000 [329:23:57<78:34:34,  5.34s/it]
 76%|███████▋  | 172014/225000 [329:24:02<78:27:03,  5.33s/it]
 76%|███████▋  | 172015/225000 [329:24:08<79:09:21,  5.38s/it]
 76%|███████▋  | 172016/225000 [329:24:13<79:00:10,  5.37s/it]
 76%|███████▋  | 172017/225000 [329:24:19<79:57:18,  5.43s/it]
 76%|███████▋  | 172018/225000 [329:24:24<78:31:18,  5.34s/it]
 76%|███████▋  | 172019/225000 [329:24:29<78:30:55,  5.34s/it]
 76%|███████▋  | 172020/225000 [329:24:35<80:07:55,  5.44s/it]
 76%|███████▋  | 172021/225000 [329:24:40<78:07:25,  5.31s/it]
 76%|███████▋  | 172022/225000 [329:24:45<78:24:26,  5.33s/it]
 76%|███████▋  | 172023/225000 [329:24:50<78:09:24,  5.31s/it]
 76%|███████▋  | 172024/225000 [329:24:56<77:46:29,  5.29s/it]
 76%|███████▋  | 172025/225000 [329:25:01<78:28:46,  5.33s/it]
 76%|███████▋  | 172026/225000 [329:25:07<79:16:10,  5.39s/it]
 76%|███████▋  | 172027/225000 [329:25:17<100:57:11,  6.86s/it]
 76%|███████▋  | 172028/225000 [329:25:23<95:23:17,  6.48s/it] 
 76%|███████▋  | 172029/225000 [329:25:29<97:18:22,  6.61s/it]
 76%|███████▋  | 172030/225000 [329:25:33<83:25:32,  5.67s/it]
 76%|███████▋  | 172031/225000 [329:25:36<73:22:24,  4.99s/it]
 76%|███████▋  | 172032/225000 [329:25:45<88:39:11,  6.03s/it]
 76%|███████▋  | 172033/225000 [329:25:50<86:58:17,  5.91s/it]
 76%|███████▋  | 172034/225000 [329:25:55<82:23:33,  5.60s/it]
 76%|███████▋  | 172035/225000 [329:26:02<86:31:40,  5.88s/it]
 76%|███████▋  | 172036/225000 [329:26:06<80:25:27,  5.47s/it]
 76%|███████▋  | 172037/225000 [329:26:11<77:02:00,  5.24s/it]
 76%|███████▋  | 172038/225000 [329:26:16<74:21:19,  5.05s/it]
 76%|███████▋  | 172039/225000 [329:26:20<71:31:48,  4.86s/it]
 76%|███████▋  | 172040/225000 [329:26:25<70:40:51,  4.80s/it]
 76%|███████▋  | 172041/225000 [329:26:29<69:52:27,  4.75s/it]
 76%|███████▋  | 172042/225000 [329:26:34<69:27:28,  4.72s/it]
 76%|███████▋  | 172043/225000 [329:26:42<84:08:36,  5.72s/it]
 76%|███████▋  | 172044/225000 [329:26:51<97:02:52,  6.60s/it]
 76%|███████▋  | 172045/225000 [329:26:54<84:10:53,  5.72s/it]
 76%|███████▋  | 172046/225000 [329:27:01<86:35:04,  5.89s/it]
 76%|███████▋  | 172047/225000 [329:27:08<91:34:57,  6.23s/it]
 76%|███████▋  | 172048/225000 [329:27:11<80:04:33,  5.44s/it]
 76%|███████▋  | 172049/225000 [329:27:19<89:16:22,  6.07s/it]
 76%|███████▋  | 172050/225000 [329:27:24<86:45:29,  5.90s/it]
                                                              
{'loss': '0.7479', 'grad_norm': '1.073', 'learning_rate': '7.137e-06', 'epoch': '0.7647', 'num_input_tokens_seen': 545865040, 'train_runtime': '1.186e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 172050/225000 [329:27:24<86:45:29,  5.90s/it]
 76%|███████▋  | 172051/225000 [329:27:28<75:25:23,  5.13s/it]
 76%|███████▋  | 172052/225000 [329:27:36<87:37:45,  5.96s/it]
 76%|███████▋  | 172053/225000 [329:27:39<75:32:08,  5.14s/it]
 76%|███████▋  | 172054/225000 [329:27:46<83:32:23,  5.68s/it]
 76%|███████▋  | 172055/225000 [329:27:49<73:06:02,  4.97s/it]
 76%|███████▋  | 172056/225000 [329:27:57<87:57:15,  5.98s/it]
 76%|███████▋  | 172057/225000 [329:28:01<76:45:40,  5.22s/it]
 76%|███████▋  | 172058/225000 [329:28:05<71:31:36,  4.86s/it]
 76%|███████▋  | 172059/225000 [329:28:09<69:43:51,  4.74s/it]
 76%|███████▋  | 172060/225000 [329:28:14<67:54:04,  4.62s/it]
 76%|███████▋  | 172061/225000 [329:28:21<78:26:39,  5.33s/it]
 76%|███████▋  | 172062/225000 [329:28:29<89:49:28,  6.11s/it]
 76%|███████▋  | 172063/225000 [329:28:32<80:04:23,  5.45s/it]
 76%|███████▋  | 172064/225000 [329:28:37<75:55:44,  5.16s/it]
 76%|███████▋  | 172065/225000 [329:28:43<78:35:00,  5.34s/it]
 76%|███████▋  | 172066/225000 [329:28:52<97:24:40,  6.62s/it]
 76%|███████▋  | 172067/225000 [329:28:56<83:51:45,  5.70s/it]
 76%|███████▋  | 172068/225000 [329:28:59<74:28:44,  5.07s/it]
 76%|███████▋  | 172069/225000 [329:29:04<70:06:01,  4.77s/it]
 76%|███████▋  | 172070/225000 [329:29:13<89:06:11,  6.06s/it]
 76%|███████▋  | 172071/225000 [329:29:17<80:37:21,  5.48s/it]
 76%|███████▋  | 172072/225000 [329:29:25<91:39:02,  6.23s/it]
 76%|███████▋  | 172073/225000 [329:29:28<79:32:33,  5.41s/it]
 76%|███████▋  | 172074/225000 [329:29:33<77:39:30,  5.28s/it]
 76%|███████▋  | 172075/225000 [329:29:39<78:12:17,  5.32s/it]
 76%|███████▋  | 172076/225000 [329:29:44<77:03:35,  5.24s/it]
 76%|███████▋  | 172077/225000 [329:29:49<76:35:52,  5.21s/it]
 76%|███████▋  | 172078/225000 [329:29:55<78:41:32,  5.35s/it]
 76%|███████▋  | 172079/225000 [329:30:00<77:43:39,  5.29s/it]
 76%|███████▋  | 172080/225000 [329:30:05<79:09:53,  5.39s/it]
 76%|███████▋  | 172081/225000 [329:30:10<78:21:21,  5.33s/it]
 76%|███████▋  | 172082/225000 [329:30:15<76:42:08,  5.22s/it]
 76%|███████▋  | 172083/225000 [329:30:21<78:16:27,  5.33s/it]
 76%|███████▋  | 172084/225000 [329:30:30<94:11:00,  6.41s/it]
 76%|███████▋  | 172085/225000 [329:30:35<86:18:11,  5.87s/it]
 76%|███████▋  | 172086/225000 [329:30:39<80:14:17,  5.46s/it]
 76%|███████▋  | 172087/225000 [329:30:43<73:40:10,  5.01s/it]
 76%|███████▋  | 172088/225000 [329:30:47<70:31:51,  4.80s/it]
 76%|███████▋  | 172089/225000 [329:30:58<94:49:29,  6.45s/it]
 76%|███████▋  | 172090/225000 [329:31:01<82:14:12,  5.60s/it]
 76%|███████▋  | 172091/225000 [329:31:05<72:20:59,  4.92s/it]
 76%|███████▋  | 172092/225000 [329:31:13<89:21:16,  6.08s/it]
 76%|███████▋  | 172093/225000 [329:31:17<78:14:00,  5.32s/it]
 76%|███████▋  | 172094/225000 [329:31:25<91:38:03,  6.24s/it]
 76%|███████▋  | 172095/225000 [329:31:29<79:17:46,  5.40s/it]
 76%|███████▋  | 172096/225000 [329:31:34<78:09:14,  5.32s/it]
 76%|███████▋  | 172097/225000 [329:31:42<89:16:30,  6.08s/it]
 76%|███████▋  | 172098/225000 [329:31:45<77:53:59,  5.30s/it]
 76%|███████▋  | 172099/225000 [329:31:51<78:41:11,  5.35s/it]
 76%|███████▋  | 172100/225000 [329:31:56<79:09:09,  5.39s/it]
                                                              
{'loss': '0.7632', 'grad_norm': '1.304', 'learning_rate': '7.125e-06', 'epoch': '0.7649', 'num_input_tokens_seen': 545995360, 'train_runtime': '1.186e+06', 'train_tokens_per_second': '460.2'}

 76%|███████▋  | 172100/225000 [329:31:56<79:09:09,  5.39s/it]
 76%|███████▋  | 172101/225000 [329:32:02<79:51:49,  5.44s/it]
 76%|███████▋  | 172102/225000 [329:32:07<80:02:28,  5.45s/it]
 76%|███████▋  | 172103/225000 [329:32:13<80:44:39,  5.50s/it]
 76%|███████▋  | 172104/225000 [329:32:18<81:03:20,  5.52s/it]
 76%|███████▋  | 172105/225000 [329:32:24<80:33:55,  5.48s/it]
 76%|███████▋  | 172106/225000 [329:32:32<93:22:53,  6.36s/it]
 76%|███████▋  | 172107/225000 [329:32:36<84:27:27,  5.75s/it]
 76%|███████▋  | 172108/225000 [329:32:40<76:49:50,  5.23s/it]
 76%|███████▋  | 172109/225000 [329:32:45<73:09:08,  4.98s/it]
 76%|███████▋  | 172110/225000 [329:32:54<92:33:52,  6.30s/it]
 76%|███████▋  | 172111/225000 [329:32:58<81:21:03,  5.54s/it]
 76%|███████▋  | 172112/225000 [329:33:02<74:32:18,  5.07s/it]
 76%|███████▋  | 172113/225000 [329:33:12<95:58:22,  6.53s/it]
 76%|███████▋  | 172114/225000 [329:33:15<82:01:56,  5.58s/it]
 76%|███████▋  | 172115/225000 [329:33:19<75:24:58,  5.13s/it]
 76%|███████▋  | 172116/225000 [329:33:24<74:16:00,  5.06s/it]
 76%|███████▋  | 172117/225000 [329:33:29<71:49:49,  4.89s/it]
 76%|███████▋  | 172118/225000 [329:33:33<70:45:05,  4.82s/it]
 76%|███████▋  | 172119/225000 [329:33:45<100:20:19,  6.83s/it]
 76%|███████▋  | 172120/225000 [329:33:48<84:56:20,  5.78s/it] 
 76%|███████▋  | 172121/225000 [329:33:53<78:42:24,  5.36s/it]
 76%|███████▋  | 172122/225000 [329:33:58<78:09:54,  5.32s/it]
 76%|███████▋  | 172123/225000 [329:34:09<101:34:16,  6.92s/it]
 76%|███████▋  | 172124/225000 [329:34:12<87:57:31,  5.99s/it] 
 76%|███████▋  | 172125/225000 [329:34:16<78:31:40,  5.35s/it]
 77%|███████▋  | 172126/225000 [329:34:21<75:55:33,  5.17s/it]
 77%|███████▋  | 172127/225000 [329:34:26<74:20:49,  5.06s/it]
 77%|███████▋  | 172128/225000 [329:34:31<75:28:51,  5.14s/it]
 77%|███████▋  | 172129/225000 [329:34:36<74:51:04,  5.10s/it]
 77%|███████▋  | 172130/225000 [329:34:41<71:53:31,  4.90s/it]
 77%|███████▋  | 172131/225000 [329:34:45<72:02:13,  4.91s/it]
 77%|███████▋  | 172132/225000 [329:34:51<72:57:07,  4.97s/it]
 77%|███████▋  | 172133/225000 [329:34:56<73:54:10,  5.03s/it]
 77%|███████▋  | 172134/225000 [329:35:01<73:13:55,  4.99s/it]
 77%|███████▋  | 172135/225000 [329:35:06<75:44:36,  5.16s/it]
 77%|███████▋  | 172136/225000 [329:35:18<105:36:30,  7.19s/it]
 77%|███████▋  | 172137/225000 [329:35:22<89:02:20,  6.06s/it] 
 77%|███████▋  | 172138/225000 [329:35:26<83:48:48,  5.71s/it]
 77%|███████▋  | 172139/225000 [329:35:32<81:13:35,  5.53s/it]
 77%|███████▋  | 172140/225000 [329:35:36<77:53:25,  5.30s/it]
 77%|███████▋  | 172141/225000 [329:35:41<74:20:42,  5.06s/it]
 77%|███████▋  | 172142/225000 [329:35:46<74:50:48,  5.10s/it]
 77%|███████▋  | 172143/225000 [329:35:51<75:00:57,  5.11s/it]
 77%|███████▋  | 172144/225000 [329:35:56<74:41:28,  5.09s/it]
 77%|███████▋  | 172145/225000 [329:36:06<97:03:02,  6.61s/it]
 77%|███████▋  | 172146/225000 [329:36:13<98:05:52,  6.68s/it]
 77%|███████▋  | 172147/225000 [329:36:17<84:47:59,  5.78s/it]
 77%|███████▋  | 172148/225000 [329:36:22<80:08:40,  5.46s/it]
 77%|███████▋  | 172149/225000 [329:36:27<78:58:44,  5.38s/it]
 77%|███████▋  | 172150/225000 [329:36:31<75:29:53,  5.14s/it]
                                                              
{'loss': '0.7602', 'grad_norm': '1.112', 'learning_rate': '7.112e-06', 'epoch': '0.7651', 'num_input_tokens_seen': 546126080, 'train_runtime': '1.187e+06', 'train_tokens_per_second': '460.2'}

 77%|███████▋  | 172150/225000 [329:36:31<75:29:53,  5.14s/it]
 77%|███████▋  | 172151/225000 [329:36:36<74:59:39,  5.11s/it]
 77%|███████▋  | 172152/225000 [329:36:41<73:57:33,  5.04s/it]
 77%|███████▋  | 172153/225000 [329:36:47<75:29:08,  5.14s/it]
 77%|███████▋  | 172154/225000 [329:36:51<74:11:26,  5.05s/it]
 77%|███████▋  | 172155/225000 [329:36:57<76:09:30,  5.19s/it]
 77%|███████▋  | 172156/225000 [329:37:02<76:19:16,  5.20s/it]
 77%|███████▋  | 172157/225000 [329:37:08<77:05:51,  5.25s/it]
 77%|███████▋  | 172158/225000 [329:37:17<95:43:19,  6.52s/it]
 77%|███████▋  | 172159/225000 [329:37:25<101:45:54,  6.93s/it]
 77%|███████▋  | 172160/225000 [329:37:30<93:38:46,  6.38s/it] 
 77%|███████▋  | 172161/225000 [329:37:35<88:27:32,  6.03s/it]
 77%|███████▋  | 172162/225000 [329:37:40<81:38:54,  5.56s/it]
 77%|███████▋  | 172163/225000 [329:37:44<76:05:32,  5.18s/it]
 77%|███████▋  | 172164/225000 [329:37:48<71:41:30,  4.88s/it]
 77%|███████▋  | 172165/225000 [329:37:52<68:56:33,  4.70s/it]
 77%|███████▋  | 172166/225000 [329:38:01<84:15:55,  5.74s/it]
 77%|███████▋  | 172167/225000 [329:38:10<101:57:29,  6.95s/it]
 77%|███████▋  | 172168/225000 [329:38:15<89:53:37,  6.13s/it] 
 77%|███████▋  | 172169/225000 [329:38:19<80:54:59,  5.51s/it]
 77%|███████▋  | 172170/225000 [329:38:23<74:31:35,  5.08s/it]
 77%|███████▋  | 172171/225000 [329:38:27<69:46:31,  4.75s/it]
 77%|███████▋  | 172172/225000 [329:38:37<95:01:05,  6.48s/it]
 77%|███████▋  | 172173/225000 [329:38:40<80:30:25,  5.49s/it]
 77%|███████▋  | 172174/225000 [329:38:48<89:53:17,  6.13s/it]
 77%|███████▋  | 172175/225000 [329:38:52<82:22:51,  5.61s/it]
 77%|███████▋  | 172176/225000 [329:38:57<77:46:27,  5.30s/it]
 77%|███████▋  | 172177/225000 [329:39:02<74:42:11,  5.09s/it]
 77%|███████▋  | 172178/225000 [329:39:06<71:51:27,  4.90s/it]
 77%|███████▋  | 172179/225000 [329:39:11<70:03:07,  4.77s/it]
 77%|███████▋  | 172180/225000 [329:39:15<68:35:02,  4.67s/it]
 77%|███████▋  | 172181/225000 [329:39:20<67:54:10,  4.63s/it]
 77%|███████▋  | 172182/225000 [329:39:29<89:30:56,  6.10s/it]
 77%|███████▋  | 172183/225000 [329:39:37<95:27:46,  6.51s/it]
 77%|███████▋  | 172184/225000 [329:39:41<85:03:21,  5.80s/it]
 77%|███████▋  | 172185/225000 [329:39:45<78:56:00,  5.38s/it]
 77%|███████▋  | 172186/225000 [329:39:49<72:14:08,  4.92s/it]
 77%|███████▋  | 172187/225000 [329:39:59<95:35:23,  6.52s/it]
 77%|███████▋  | 172188/225000 [329:40:03<84:37:07,  5.77s/it]
 77%|███████▋  | 172189/225000 [329:40:07<76:03:23,  5.18s/it]
 77%|███████▋  | 172190/225000 [329:40:15<90:32:17,  6.17s/it]
 77%|███████▋  | 172191/225000 [329:40:20<81:13:59,  5.54s/it]
 77%|███████▋  | 172192/225000 [329:40:27<87:27:35,  5.96s/it]
 77%|███████▋  | 172193/225000 [329:40:32<85:58:55,  5.86s/it]
 77%|███████▋  | 172194/225000 [329:40:36<76:30:47,  5.22s/it]
 77%|███████▋  | 172195/225000 [329:40:46<97:04:27,  6.62s/it]
 77%|███████▋  | 172196/225000 [329:40:49<82:55:04,  5.65s/it]
 77%|███████▋  | 172197/225000 [329:40:53<74:58:39,  5.11s/it]
 77%|███████▋  | 172198/225000 [329:40:59<79:37:30,  5.43s/it]
 77%|███████▋  | 172199/225000 [329:41:06<86:53:09,  5.92s/it]
 77%|███████▋  | 172200/225000 [329:41:10<77:22:47,  5.28s/it]
                                                              
{'loss': '0.7733', 'grad_norm': '1.096', 'learning_rate': '7.099e-06', 'epoch': '0.7653', 'num_input_tokens_seen': 546256992, 'train_runtime': '1.187e+06', 'train_tokens_per_second': '460.2'}

 77%|███████▋  | 172200/225000 [329:41:10<77:22:47,  5.28s/it]
 77%|███████▋  | 172201/225000 [329:41:14<71:42:09,  4.89s/it]
 77%|███████▋  | 172202/225000 [329:41:20<75:06:13,  5.12s/it]
 77%|███████▋  | 172203/225000 [329:41:28<87:44:06,  5.98s/it]
 77%|███████▋  | 172204/225000 [329:41:31<77:28:53,  5.28s/it]
 77%|███████▋  | 172205/225000 [329:41:39<89:08:13,  6.08s/it]
 77%|███████▋  | 172206/225000 [329:41:43<81:10:36,  5.54s/it]
 77%|███████▋  | 172207/225000 [329:41:48<77:02:37,  5.25s/it]
 77%|███████▋  | 172208/225000 [329:41:52<72:26:23,  4.94s/it]
 77%|███████▋  | 172209/225000 [329:41:57<70:33:54,  4.81s/it]
 77%|███████▋  | 172210/225000 [329:42:01<69:31:13,  4.74s/it]
 77%|███████▋  | 172211/225000 [329:42:06<67:20:52,  4.59s/it]
 77%|███████▋  | 172212/225000 [329:42:17<97:51:55,  6.67s/it]
 77%|███████▋  | 172213/225000 [329:42:21<87:23:35,  5.96s/it]
 77%|███████▋  | 172214/225000 [329:42:26<80:17:15,  5.48s/it]
 77%|███████▋  | 172215/225000 [329:42:30<75:02:19,  5.12s/it]
 77%|███████▋  | 172216/225000 [329:42:34<71:48:29,  4.90s/it]
 77%|███████▋  | 172217/225000 [329:42:39<69:22:20,  4.73s/it]
 77%|███████▋  | 172218/225000 [329:42:50<99:29:20,  6.79s/it]
 77%|███████▋  | 172219/225000 [329:42:54<86:25:37,  5.89s/it]
 77%|███████▋  | 172220/225000 [329:42:58<77:19:58,  5.27s/it]
 77%|███████▋  | 172221/225000 [329:43:07<93:14:54,  6.36s/it]
 77%|███████▋  | 172222/225000 [329:43:10<80:43:35,  5.51s/it]
 77%|███████▋  | 172223/225000 [329:43:18<91:00:39,  6.21s/it]
 77%|███████▋  | 172224/225000 [329:43:22<81:52:01,  5.58s/it]
 77%|███████▋  | 172225/225000 [329:43:27<75:55:51,  5.18s/it]
 77%|███████▋  | 172226/225000 [329:43:31<70:54:18,  4.84s/it]
 77%|███████▋  | 172227/225000 [329:43:35<68:13:47,  4.65s/it]
 77%|███████▋  | 172228/225000 [329:43:47<99:13:22,  6.77s/it]
 77%|███████▋  | 172229/225000 [329:43:50<84:02:20,  5.73s/it]
 77%|███████▋  | 172230/225000 [329:43:54<77:52:19,  5.31s/it]
 77%|███████▋  | 172231/225000 [329:43:58<73:07:09,  4.99s/it]
 77%|███████▋  | 172232/225000 [329:44:03<69:53:41,  4.77s/it]
 77%|███████▋  | 172233/225000 [329:44:09<78:34:47,  5.36s/it]
 77%|███████▋  | 172234/225000 [329:44:19<97:09:04,  6.63s/it]
 77%|███████▋  | 172235/225000 [329:44:23<87:11:02,  5.95s/it]
 77%|███████▋  | 172236/225000 [329:44:27<78:06:02,  5.33s/it]
 77%|███████▋  | 172237/225000 [329:44:33<81:29:11,  5.56s/it]
 77%|███████▋  | 172238/225000 [329:44:42<95:07:35,  6.49s/it]
 77%|███████▋  | 172239/225000 [329:44:46<82:07:53,  5.60s/it]
 77%|███████▋  | 172240/225000 [329:44:50<75:50:31,  5.17s/it]
 77%|███████▋  | 172241/225000 [329:44:54<71:23:11,  4.87s/it]
 77%|███████▋  | 172242/225000 [329:44:58<68:10:30,  4.65s/it]
 77%|███████▋  | 172243/225000 [329:45:08<93:15:04,  6.36s/it]
 77%|███████▋  | 172244/225000 [329:45:13<84:51:41,  5.79s/it]
 77%|███████▋  | 172245/225000 [329:45:18<81:18:03,  5.55s/it]
 77%|███████▋  | 172246/225000 [329:45:25<87:43:42,  5.99s/it]
 77%|███████▋  | 172247/225000 [329:45:29<77:15:39,  5.27s/it]
 77%|███████▋  | 172248/225000 [329:45:33<73:12:43,  5.00s/it]
 77%|███████▋  | 172249/225000 [329:45:41<88:30:10,  6.04s/it]
 77%|███████▋  | 172250/225000 [329:45:45<80:03:36,  5.46s/it]
                                                              
{'loss': '0.7715', 'grad_norm': '1.094', 'learning_rate': '7.086e-06', 'epoch': '0.7656', 'num_input_tokens_seen': 546387400, 'train_runtime': '1.187e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172250/225000 [329:45:45<80:03:36,  5.46s/it]
 77%|███████▋  | 172251/225000 [329:45:49<71:33:19,  4.88s/it]
 77%|███████▋  | 172252/225000 [329:45:55<78:22:22,  5.35s/it]
 77%|███████▋  | 172253/225000 [329:46:03<87:57:50,  6.00s/it]
 77%|███████▋  | 172254/225000 [329:46:06<75:30:09,  5.15s/it]
 77%|███████▋  | 172255/225000 [329:46:11<74:17:42,  5.07s/it]
 77%|███████▋  | 172256/225000 [329:46:16<73:40:23,  5.03s/it]
 77%|███████▋  | 172257/225000 [329:46:21<73:54:55,  5.05s/it]
 77%|███████▋  | 172258/225000 [329:46:26<75:41:11,  5.17s/it]
 77%|███████▋  | 172259/225000 [329:46:32<76:48:03,  5.24s/it]
 77%|███████▋  | 172260/225000 [329:46:37<78:08:46,  5.33s/it]
 77%|███████▋  | 172261/225000 [329:46:44<82:04:43,  5.60s/it]
 77%|███████▋  | 172262/225000 [329:46:53<100:19:20,  6.85s/it]
 77%|███████▋  | 172263/225000 [329:46:59<93:59:04,  6.42s/it] 
 77%|███████▋  | 172264/225000 [329:47:04<88:38:35,  6.05s/it]
 77%|███████▋  | 172265/225000 [329:47:08<79:25:18,  5.42s/it]
 77%|███████▋  | 172266/225000 [329:47:12<72:44:23,  4.97s/it]
 77%|███████▋  | 172267/225000 [329:47:17<71:35:06,  4.89s/it]
 77%|███████▋  | 172268/225000 [329:47:25<88:34:57,  6.05s/it]
 77%|███████▋  | 172269/225000 [329:47:29<77:50:37,  5.31s/it]
 77%|███████▋  | 172270/225000 [329:47:35<79:24:33,  5.42s/it]
 77%|███████▋  | 172271/225000 [329:47:40<80:13:43,  5.48s/it]
 77%|███████▋  | 172272/225000 [329:47:46<81:03:18,  5.53s/it]
 77%|███████▋  | 172273/225000 [329:47:51<78:41:29,  5.37s/it]
 77%|███████▋  | 172274/225000 [329:47:56<79:24:52,  5.42s/it]
 77%|███████▋  | 172275/225000 [329:48:02<78:30:39,  5.36s/it]
 77%|███████▋  | 172276/225000 [329:48:07<77:48:17,  5.31s/it]
 77%|███████▋  | 172277/225000 [329:48:12<77:53:28,  5.32s/it]
 77%|███████▋  | 172278/225000 [329:48:18<79:28:57,  5.43s/it]
 77%|███████▋  | 172279/225000 [329:48:23<78:24:30,  5.35s/it]
 77%|███████▋  | 172280/225000 [329:48:29<79:55:01,  5.46s/it]
 77%|███████▋  | 172281/225000 [329:48:34<80:14:07,  5.48s/it]
 77%|███████▋  | 172282/225000 [329:48:44<98:21:56,  6.72s/it]
 77%|███████▋  | 172283/225000 [329:48:47<84:26:43,  5.77s/it]
 77%|███████▋  | 172284/225000 [329:48:52<79:27:21,  5.43s/it]
 77%|███████▋  | 172285/225000 [329:49:01<93:10:35,  6.36s/it]
 77%|███████▋  | 172286/225000 [329:49:04<80:48:27,  5.52s/it]
 77%|███████▋  | 172287/225000 [329:49:07<71:12:54,  4.86s/it]
 77%|███████▋  | 172288/225000 [329:49:17<89:44:39,  6.13s/it]
 77%|███████▋  | 172289/225000 [329:49:21<81:10:29,  5.54s/it]
 77%|███████▋  | 172290/225000 [329:49:25<74:28:53,  5.09s/it]
 77%|███████▋  | 172291/225000 [329:49:31<79:51:14,  5.45s/it]
 77%|███████▋  | 172292/225000 [329:49:39<88:37:22,  6.05s/it]
 77%|███████▋  | 172293/225000 [329:49:43<81:14:55,  5.55s/it]
 77%|███████▋  | 172294/225000 [329:49:47<74:49:30,  5.11s/it]
 77%|███████▋  | 172295/225000 [329:49:51<70:49:49,  4.84s/it]
 77%|███████▋  | 172296/225000 [329:50:02<95:28:30,  6.52s/it]
 77%|███████▋  | 172297/225000 [329:50:05<83:37:11,  5.71s/it]
 77%|███████▋  | 172298/225000 [329:50:09<76:04:19,  5.20s/it]
 77%|███████▋  | 172299/225000 [329:50:19<94:51:20,  6.48s/it]
 77%|███████▋  | 172300/225000 [329:50:23<85:49:14,  5.86s/it]
                                                              
{'loss': '0.774', 'grad_norm': '1.101', 'learning_rate': '7.074e-06', 'epoch': '0.7658', 'num_input_tokens_seen': 546516472, 'train_runtime': '1.187e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172300/225000 [329:50:23<85:49:14,  5.86s/it]
 77%|███████▋  | 172301/225000 [329:50:28<78:24:50,  5.36s/it]
 77%|███████▋  | 172302/225000 [329:50:32<73:28:36,  5.02s/it]
 77%|███████▋  | 172303/225000 [329:50:36<69:47:23,  4.77s/it]
 77%|███████▋  | 172304/225000 [329:50:40<67:55:43,  4.64s/it]
 77%|███████▋  | 172305/225000 [329:50:51<95:11:36,  6.50s/it]
 77%|███████▋  | 172306/225000 [329:50:55<82:12:30,  5.62s/it]
 77%|███████▋  | 172307/225000 [329:51:00<81:30:03,  5.57s/it]
 77%|███████▋  | 172308/225000 [329:51:06<84:56:18,  5.80s/it]
 77%|███████▋  | 172309/225000 [329:51:10<75:20:14,  5.15s/it]
 77%|███████▋  | 172310/225000 [329:51:14<69:23:24,  4.74s/it]
 77%|███████▋  | 172311/225000 [329:51:24<92:35:49,  6.33s/it]
 77%|███████▋  | 172312/225000 [329:51:28<81:36:00,  5.58s/it]
 77%|███████▋  | 172313/225000 [329:51:32<73:39:04,  5.03s/it]
 77%|███████▋  | 172314/225000 [329:51:41<92:24:59,  6.31s/it]
 77%|███████▋  | 172315/225000 [329:51:45<82:17:29,  5.62s/it]
 77%|███████▋  | 172316/225000 [329:51:49<75:13:15,  5.14s/it]
 77%|███████▋  | 172317/225000 [329:51:53<69:27:08,  4.75s/it]
 77%|███████▋  | 172318/225000 [329:52:01<83:42:28,  5.72s/it]
 77%|███████▋  | 172319/225000 [329:52:07<87:32:56,  5.98s/it]
 77%|███████▋  | 172320/225000 [329:52:11<76:52:05,  5.25s/it]
 77%|███████▋  | 172321/225000 [329:52:16<75:06:27,  5.13s/it]
 77%|███████▋  | 172322/225000 [329:52:22<80:27:02,  5.50s/it]
 77%|███████▋  | 172323/225000 [329:52:27<77:41:38,  5.31s/it]
 77%|███████▋  | 172324/225000 [329:52:34<84:35:14,  5.78s/it]
 77%|███████▋  | 172325/225000 [329:52:38<76:51:15,  5.25s/it]
 77%|███████▋  | 172326/225000 [329:52:42<74:20:53,  5.08s/it]
 77%|███████▋  | 172327/225000 [329:52:47<73:17:50,  5.01s/it]
 77%|███████▋  | 172328/225000 [329:52:52<73:24:08,  5.02s/it]
 77%|███████▋  | 172329/225000 [329:52:57<71:12:58,  4.87s/it]
 77%|███████▋  | 172330/225000 [329:53:02<72:50:43,  4.98s/it]
 77%|███████▋  | 172331/225000 [329:53:06<69:51:33,  4.77s/it]
 77%|███████▋  | 172332/225000 [329:53:11<70:12:38,  4.80s/it]
 77%|███████▋  | 172333/225000 [329:53:16<71:00:34,  4.85s/it]
 77%|███████▋  | 172334/225000 [329:53:29<105:00:51,  7.18s/it]
 77%|███████▋  | 172335/225000 [329:53:35<99:41:28,  6.81s/it] 
 77%|███████▋  | 172336/225000 [329:53:39<86:14:16,  5.90s/it]
 77%|███████▋  | 172337/225000 [329:53:44<83:48:15,  5.73s/it]
 77%|███████▋  | 172338/225000 [329:53:49<82:22:08,  5.63s/it]
 77%|███████▋  | 172339/225000 [329:53:55<80:36:29,  5.51s/it]
 77%|███████▋  | 172340/225000 [329:53:59<78:16:54,  5.35s/it]
 77%|███████▋  | 172341/225000 [329:54:05<78:03:20,  5.34s/it]
 77%|███████▋  | 172342/225000 [329:54:10<77:13:29,  5.28s/it]
 77%|███████▋  | 172343/225000 [329:54:15<77:31:11,  5.30s/it]
 77%|███████▋  | 172344/225000 [329:54:21<77:52:55,  5.32s/it]
 77%|███████▋  | 172345/225000 [329:54:26<77:36:07,  5.31s/it]
 77%|███████▋  | 172346/225000 [329:54:31<75:56:38,  5.19s/it]
 77%|███████▋  | 172347/225000 [329:54:36<76:14:31,  5.21s/it]
 77%|███████▋  | 172348/225000 [329:54:41<76:28:47,  5.23s/it]
 77%|███████▋  | 172349/225000 [329:54:47<78:17:41,  5.35s/it]
 77%|███████▋  | 172350/225000 [329:54:52<76:27:07,  5.23s/it]
                                                              
{'loss': '0.7626', 'grad_norm': '1.148', 'learning_rate': '7.061e-06', 'epoch': '0.766', 'num_input_tokens_seen': 546646320, 'train_runtime': '1.188e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172350/225000 [329:54:52<76:27:07,  5.23s/it]
 77%|███████▋  | 172351/225000 [329:54:57<75:56:11,  5.19s/it]
 77%|███████▋  | 172352/225000 [329:55:02<76:04:13,  5.20s/it]
 77%|███████▋  | 172353/225000 [329:55:08<76:28:34,  5.23s/it]
 77%|███████▋  | 172354/225000 [329:55:13<75:25:36,  5.16s/it]
 77%|███████▋  | 172355/225000 [329:55:17<74:21:44,  5.09s/it]
 77%|███████▋  | 172356/225000 [329:55:23<75:35:02,  5.17s/it]
 77%|███████▋  | 172357/225000 [329:55:28<75:55:48,  5.19s/it]
 77%|███████▋  | 172358/225000 [329:55:33<74:59:04,  5.13s/it]
 77%|███████▋  | 172359/225000 [329:55:39<76:21:13,  5.22s/it]
 77%|███████▋  | 172360/225000 [329:55:44<77:15:36,  5.28s/it]
 77%|███████▋  | 172361/225000 [329:55:49<77:44:59,  5.32s/it]
 77%|███████▋  | 172362/225000 [329:55:55<78:23:26,  5.36s/it]
 77%|███████▋  | 172363/225000 [329:56:00<79:28:19,  5.44s/it]
 77%|███████▋  | 172364/225000 [329:56:10<99:36:17,  6.81s/it]
 77%|███████▋  | 172365/225000 [329:56:18<103:40:24,  7.09s/it]
 77%|███████▋  | 172366/225000 [329:56:22<88:32:20,  6.06s/it] 
 77%|███████▋  | 172367/225000 [329:56:27<84:11:24,  5.76s/it]
 77%|███████▋  | 172368/225000 [329:56:36<99:13:05,  6.79s/it]
 77%|███████▋  | 172369/225000 [329:56:40<84:33:40,  5.78s/it]
 77%|███████▋  | 172370/225000 [329:56:44<77:05:20,  5.27s/it]
 77%|███████▋  | 172371/225000 [329:56:52<91:27:22,  6.26s/it]
 77%|███████▋  | 172372/225000 [329:56:56<82:04:58,  5.61s/it]
 77%|███████▋  | 172373/225000 [329:57:04<90:48:39,  6.21s/it]
 77%|███████▋  | 172374/225000 [329:57:07<78:10:10,  5.35s/it]
 77%|███████▋  | 172375/225000 [329:57:15<87:34:27,  5.99s/it]
 77%|███████▋  | 172376/225000 [329:57:18<77:07:11,  5.28s/it]
 77%|███████▋  | 172377/225000 [329:57:24<77:50:17,  5.32s/it]
 77%|███████▋  | 172378/225000 [329:57:31<87:15:33,  5.97s/it]
 77%|███████▋  | 172379/225000 [329:57:36<81:54:58,  5.60s/it]
 77%|███████▋  | 172380/225000 [329:57:42<83:36:14,  5.72s/it]
 77%|███████▋  | 172381/225000 [329:57:46<77:00:05,  5.27s/it]
 77%|███████▋  | 172382/225000 [329:57:51<73:18:42,  5.02s/it]
 77%|███████▋  | 172383/225000 [329:57:55<70:54:48,  4.85s/it]
 77%|███████▋  | 172384/225000 [329:57:59<68:50:48,  4.71s/it]
 77%|███████▋  | 172385/225000 [329:58:04<68:39:21,  4.70s/it]
 77%|███████▋  | 172386/225000 [329:58:12<84:40:48,  5.79s/it]
 77%|███████▋  | 172387/225000 [329:58:20<94:08:06,  6.44s/it]
 77%|███████▋  | 172388/225000 [329:58:26<89:57:27,  6.16s/it]
 77%|███████▋  | 172389/225000 [329:58:31<86:46:27,  5.94s/it]
 77%|███████▋  | 172390/225000 [329:58:36<81:38:59,  5.59s/it]
 77%|███████▋  | 172391/225000 [329:58:41<77:42:15,  5.32s/it]
 77%|███████▋  | 172392/225000 [329:58:45<74:35:45,  5.10s/it]
 77%|███████▋  | 172393/225000 [329:58:50<74:27:39,  5.10s/it]
 77%|███████▋  | 172394/225000 [329:58:55<73:22:38,  5.02s/it]
 77%|███████▋  | 172395/225000 [329:59:00<72:15:46,  4.95s/it]
 77%|███████▋  | 172396/225000 [329:59:05<72:28:15,  4.96s/it]
 77%|███████▋  | 172397/225000 [329:59:10<71:58:06,  4.93s/it]
 77%|███████▋  | 172398/225000 [329:59:15<73:17:40,  5.02s/it]
 77%|███████▋  | 172399/225000 [329:59:20<74:13:37,  5.08s/it]
 77%|███████▋  | 172400/225000 [329:59:26<74:38:41,  5.11s/it]
                                                              
{'loss': '0.7678', 'grad_norm': '1.363', 'learning_rate': '7.048e-06', 'epoch': '0.7662', 'num_input_tokens_seen': 546775592, 'train_runtime': '1.188e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172400/225000 [329:59:26<74:38:41,  5.11s/it]
 77%|███████▋  | 172401/225000 [329:59:34<89:47:49,  6.15s/it]
 77%|███████▋  | 172402/225000 [329:59:42<98:48:04,  6.76s/it]
 77%|███████▋  | 172403/225000 [329:59:47<89:47:53,  6.15s/it]
 77%|███████▋  | 172404/225000 [329:59:52<82:24:58,  5.64s/it]
 77%|███████▋  | 172405/225000 [329:59:56<77:28:22,  5.30s/it]
 77%|███████▋  | 172406/225000 [330:00:00<72:27:58,  4.96s/it]
 77%|███████▋  | 172407/225000 [330:00:11<99:56:37,  6.84s/it]
 77%|███████▋  | 172408/225000 [330:00:15<84:31:27,  5.79s/it]
 77%|███████▋  | 172409/225000 [330:00:22<90:11:00,  6.17s/it]
 77%|███████▋  | 172410/225000 [330:00:26<81:44:13,  5.60s/it]
 77%|███████▋  | 172411/225000 [330:00:35<97:02:45,  6.64s/it]
 77%|███████▋  | 172412/225000 [330:00:39<82:55:25,  5.68s/it]
 77%|███████▋  | 172413/225000 [330:00:45<85:53:47,  5.88s/it]
 77%|███████▋  | 172414/225000 [330:00:49<77:15:59,  5.29s/it]
 77%|███████▋  | 172415/225000 [330:00:53<72:24:40,  4.96s/it]
 77%|███████▋  | 172416/225000 [330:01:01<83:59:08,  5.75s/it]
 77%|███████▋  | 172417/225000 [330:01:06<81:19:18,  5.57s/it]
 77%|███████▋  | 172418/225000 [330:01:10<77:34:10,  5.31s/it]
 77%|███████▋  | 172419/225000 [330:01:17<80:58:11,  5.54s/it]
 77%|███████▋  | 172420/225000 [330:01:21<74:49:46,  5.12s/it]
 77%|███████▋  | 172421/225000 [330:01:25<71:01:39,  4.86s/it]
 77%|███████▋  | 172422/225000 [330:01:29<68:19:07,  4.68s/it]
 77%|███████▋  | 172423/225000 [330:01:33<65:53:30,  4.51s/it]
 77%|███████▋  | 172424/225000 [330:01:42<85:56:01,  5.88s/it]
 77%|███████▋  | 172425/225000 [330:01:46<76:58:35,  5.27s/it]
 77%|███████▋  | 172426/225000 [330:01:54<87:05:15,  5.96s/it]
 77%|███████▋  | 172427/225000 [330:01:58<77:39:45,  5.32s/it]
 77%|███████▋  | 172428/225000 [330:02:02<73:40:15,  5.04s/it]
 77%|███████▋  | 172429/225000 [330:02:06<69:53:03,  4.79s/it]
 77%|███████▋  | 172430/225000 [330:02:16<90:52:57,  6.22s/it]
 77%|███████▋  | 172431/225000 [330:02:21<85:37:52,  5.86s/it]
 77%|███████▋  | 172432/225000 [330:02:28<93:09:42,  6.38s/it]
 77%|███████▋  | 172433/225000 [330:02:32<81:37:10,  5.59s/it]
 77%|███████▋  | 172434/225000 [330:02:37<79:56:39,  5.48s/it]
 77%|███████▋  | 172435/225000 [330:02:43<80:59:03,  5.55s/it]
 77%|███████▋  | 172436/225000 [330:02:51<92:54:46,  6.36s/it]
 77%|███████▋  | 172437/225000 [330:02:55<79:03:25,  5.41s/it]
 77%|███████▋  | 172438/225000 [330:02:59<72:47:55,  4.99s/it]
 77%|███████▋  | 172439/225000 [330:03:08<90:28:22,  6.20s/it]
 77%|███████▋  | 172440/225000 [330:03:12<83:43:08,  5.73s/it]
 77%|███████▋  | 172441/225000 [330:03:16<77:15:15,  5.29s/it]
 77%|███████▋  | 172442/225000 [330:03:21<74:37:47,  5.11s/it]
 77%|███████▋  | 172443/225000 [330:03:26<73:03:25,  5.00s/it]
 77%|███████▋  | 172444/225000 [330:03:31<73:33:16,  5.04s/it]
 77%|███████▋  | 172445/225000 [330:03:36<72:29:33,  4.97s/it]
 77%|███████▋  | 172446/225000 [330:03:40<69:28:13,  4.76s/it]
 77%|███████▋  | 172447/225000 [330:03:44<66:50:29,  4.58s/it]
 77%|███████▋  | 172448/225000 [330:03:54<89:08:45,  6.11s/it]
 77%|███████▋  | 172449/225000 [330:04:02<95:50:14,  6.57s/it]
 77%|███████▋  | 172450/225000 [330:04:06<84:24:41,  5.78s/it]
                                                              
{'loss': '0.7736', 'grad_norm': '1.257', 'learning_rate': '7.035e-06', 'epoch': '0.7664', 'num_input_tokens_seen': 546906616, 'train_runtime': '1.188e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172450/225000 [330:04:06<84:24:41,  5.78s/it]
 77%|███████▋  | 172451/225000 [330:04:10<77:15:56,  5.29s/it]
 77%|███████▋  | 172452/225000 [330:04:14<71:19:24,  4.89s/it]
 77%|███████▋  | 172453/225000 [330:04:20<77:35:59,  5.32s/it]
 77%|███████▋  | 172454/225000 [330:04:28<91:28:06,  6.27s/it]
 77%|███████▋  | 172455/225000 [330:04:32<81:06:28,  5.56s/it]
 77%|███████▋  | 172456/225000 [330:04:37<77:12:23,  5.29s/it]
 77%|███████▋  | 172457/225000 [330:04:41<73:17:50,  5.02s/it]
 77%|███████▋  | 172458/225000 [330:04:46<72:24:14,  4.96s/it]
 77%|███████▋  | 172459/225000 [330:04:51<71:54:41,  4.93s/it]
 77%|███████▋  | 172460/225000 [330:04:56<71:27:24,  4.90s/it]
 77%|███████▋  | 172461/225000 [330:05:06<96:02:12,  6.58s/it]
 77%|███████▋  | 172462/225000 [330:05:10<83:56:03,  5.75s/it]
 77%|███████▋  | 172463/225000 [330:05:14<74:11:21,  5.08s/it]
 77%|███████▋  | 172464/225000 [330:05:22<87:01:50,  5.96s/it]
 77%|███████▋  | 172465/225000 [330:05:26<81:01:12,  5.55s/it]
 77%|███████▋  | 172466/225000 [330:05:31<77:35:56,  5.32s/it]
 77%|███████▋  | 172467/225000 [330:05:36<73:56:25,  5.07s/it]
 77%|███████▋  | 172468/225000 [330:05:46<97:57:08,  6.71s/it]
 77%|███████▋  | 172469/225000 [330:05:50<85:28:15,  5.86s/it]
 77%|███████▋  | 172470/225000 [330:05:54<76:39:05,  5.25s/it]
 77%|███████▋  | 172471/225000 [330:06:02<90:03:59,  6.17s/it]
 77%|███████▋  | 172472/225000 [330:06:06<78:49:56,  5.40s/it]
 77%|███████▋  | 172473/225000 [330:06:10<72:36:47,  4.98s/it]
 77%|███████▋  | 172474/225000 [330:06:15<71:56:13,  4.93s/it]
 77%|███████▋  | 172475/225000 [330:06:19<71:16:09,  4.88s/it]
 77%|███████▋  | 172476/225000 [330:06:24<70:54:01,  4.86s/it]
 77%|███████▋  | 172477/225000 [330:06:29<70:29:17,  4.83s/it]
 77%|███████▋  | 172478/225000 [330:06:41<102:07:58,  7.00s/it]
 77%|███████▋  | 172479/225000 [330:06:45<90:40:09,  6.21s/it] 
 77%|███████▋  | 172480/225000 [330:06:50<82:48:30,  5.68s/it]
 77%|███████▋  | 172481/225000 [330:06:55<79:23:34,  5.44s/it]
 77%|███████▋  | 172482/225000 [330:07:00<80:19:37,  5.51s/it]
 77%|███████▋  | 172483/225000 [330:07:08<88:26:42,  6.06s/it]
 77%|███████▋  | 172484/225000 [330:07:14<90:53:39,  6.23s/it]
 77%|███████▋  | 172485/225000 [330:07:18<80:43:33,  5.53s/it]
 77%|███████▋  | 172486/225000 [330:07:22<73:18:21,  5.03s/it]
 77%|███████▋  | 172487/225000 [330:07:33<96:58:35,  6.65s/it]
 77%|███████▋  | 172488/225000 [330:07:36<84:56:05,  5.82s/it]
 77%|███████▋  | 172489/225000 [330:07:41<77:24:45,  5.31s/it]
 77%|███████▋  | 172490/225000 [330:07:45<72:40:16,  4.98s/it]
 77%|███████▋  | 172491/225000 [330:07:49<68:15:38,  4.68s/it]
 77%|███████▋  | 172492/225000 [330:07:54<69:09:16,  4.74s/it]
 77%|███████▋  | 172493/225000 [330:08:03<88:04:45,  6.04s/it]
 77%|███████▋  | 172494/225000 [330:08:07<81:06:13,  5.56s/it]
 77%|███████▋  | 172495/225000 [330:08:12<79:53:33,  5.48s/it]
 77%|███████▋  | 172496/225000 [330:08:17<75:17:56,  5.16s/it]
 77%|███████▋  | 172497/225000 [330:08:24<85:21:31,  5.85s/it]
 77%|███████▋  | 172498/225000 [330:08:28<74:26:48,  5.10s/it]
 77%|███████▋  | 172499/225000 [330:08:35<86:05:31,  5.90s/it]
 77%|███████▋  | 172500/225000 [330:08:39<75:54:07,  5.20s/it]
                                                              
{'loss': '0.7701', 'grad_norm': '1.091', 'learning_rate': '7.023e-06', 'epoch': '0.7667', 'num_input_tokens_seen': 547037200, 'train_runtime': '1.189e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172500/225000 [330:08:39<75:54:07,  5.20s/it]
 77%|███████▋  | 172501/225000 [330:08:47<86:51:35,  5.96s/it]
 77%|███████▋  | 172502/225000 [330:08:50<75:33:29,  5.18s/it]
 77%|███████▋  | 172503/225000 [330:08:57<83:25:47,  5.72s/it]
 77%|███████▋  | 172504/225000 [330:09:01<74:30:13,  5.11s/it]
 77%|███████▋  | 172505/225000 [330:09:07<79:34:30,  5.46s/it]
 77%|███████▋  | 172506/225000 [330:09:13<84:03:52,  5.77s/it]
 77%|███████▋  | 172507/225000 [330:09:17<76:21:21,  5.24s/it]
 77%|███████▋  | 172508/225000 [330:09:21<70:37:44,  4.84s/it]
 77%|███████▋  | 172509/225000 [330:09:28<76:05:06,  5.22s/it]
 77%|███████▋  | 172510/225000 [330:09:37<93:50:57,  6.44s/it]
 77%|███████▋  | 172511/225000 [330:09:40<78:46:53,  5.40s/it]
 77%|███████▋  | 172512/225000 [330:09:44<71:31:52,  4.91s/it]
 77%|███████▋  | 172513/225000 [330:09:53<91:32:40,  6.28s/it]
 77%|███████▋  | 172514/225000 [330:09:57<79:57:24,  5.48s/it]
 77%|███████▋  | 172515/225000 [330:10:04<88:32:22,  6.07s/it]
 77%|███████▋  | 172516/225000 [330:10:08<80:57:14,  5.55s/it]
 77%|███████▋  | 172517/225000 [330:10:13<75:37:09,  5.19s/it]
 77%|███████▋  | 172518/225000 [330:10:17<73:29:46,  5.04s/it]
 77%|███████▋  | 172519/225000 [330:10:22<70:35:40,  4.84s/it]
 77%|███████▋  | 172520/225000 [330:10:26<68:57:30,  4.73s/it]
 77%|███████▋  | 172521/225000 [330:10:31<68:25:26,  4.69s/it]
 77%|███████▋  | 172522/225000 [330:10:36<68:41:11,  4.71s/it]
 77%|███████▋  | 172523/225000 [330:10:46<93:04:33,  6.39s/it]
 77%|███████▋  | 172524/225000 [330:10:50<82:37:15,  5.67s/it]
 77%|███████▋  | 172525/225000 [330:10:56<83:59:32,  5.76s/it]
 77%|███████▋  | 172526/225000 [330:11:03<88:05:07,  6.04s/it]
 77%|███████▋  | 172527/225000 [330:11:07<80:23:03,  5.51s/it]
 77%|███████▋  | 172528/225000 [330:11:11<76:14:21,  5.23s/it]
 77%|███████▋  | 172529/225000 [330:11:16<72:14:05,  4.96s/it]
 77%|███████▋  | 172530/225000 [330:11:21<72:06:15,  4.95s/it]
 77%|███████▋  | 172531/225000 [330:11:25<68:52:43,  4.73s/it]
 77%|███████▋  | 172532/225000 [330:11:30<69:32:54,  4.77s/it]
 77%|███████▋  | 172533/225000 [330:11:35<69:30:54,  4.77s/it]
 77%|███████▋  | 172534/225000 [330:11:46<96:56:00,  6.65s/it]
 77%|███████▋  | 172535/225000 [330:11:51<90:37:25,  6.22s/it]
 77%|███████▋  | 172536/225000 [330:11:54<79:15:52,  5.44s/it]
 77%|███████▋  | 172537/225000 [330:11:58<71:50:54,  4.93s/it]
 77%|███████▋  | 172538/225000 [330:12:03<72:17:35,  4.96s/it]
 77%|███████▋  | 172539/225000 [330:12:13<92:47:42,  6.37s/it]
 77%|███████▋  | 172540/225000 [330:12:16<80:14:51,  5.51s/it]
 77%|███████▋  | 172541/225000 [330:12:20<73:29:28,  5.04s/it]
 77%|███████▋  | 172542/225000 [330:12:27<81:20:30,  5.58s/it]
 77%|███████▋  | 172543/225000 [330:12:34<85:36:06,  5.87s/it]
 77%|███████▋  | 172544/225000 [330:12:37<74:49:50,  5.14s/it]
 77%|███████▋  | 172545/225000 [330:12:45<84:44:44,  5.82s/it]
 77%|███████▋  | 172546/225000 [330:12:49<79:00:15,  5.42s/it]
 77%|███████▋  | 172547/225000 [330:12:54<75:03:31,  5.15s/it]
 77%|███████▋  | 172548/225000 [330:12:58<72:20:54,  4.97s/it]
 77%|███████▋  | 172549/225000 [330:13:03<70:58:19,  4.87s/it]
 77%|███████▋  | 172550/225000 [330:13:07<70:02:16,  4.81s/it]
                                                              
{'loss': '0.7837', 'grad_norm': '1.126', 'learning_rate': '7.01e-06', 'epoch': '0.7669', 'num_input_tokens_seen': 547166328, 'train_runtime': '1.189e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172550/225000 [330:13:07<70:02:16,  4.81s/it]
 77%|███████▋  | 172551/225000 [330:13:12<70:00:12,  4.80s/it]
 77%|███████▋  | 172552/225000 [330:13:17<69:14:40,  4.75s/it]
 77%|███████▋  | 172553/225000 [330:13:22<70:12:35,  4.82s/it]
 77%|███████▋  | 172554/225000 [330:13:27<69:49:46,  4.79s/it]
 77%|███████▋  | 172555/225000 [330:13:31<68:18:39,  4.69s/it]
 77%|███████▋  | 172556/225000 [330:13:41<91:44:02,  6.30s/it]
 77%|███████▋  | 172557/225000 [330:13:49<99:12:42,  6.81s/it]
 77%|███████▋  | 172558/225000 [330:13:53<86:20:56,  5.93s/it]
 77%|███████▋  | 172559/225000 [330:13:57<78:45:31,  5.41s/it]
 77%|███████▋  | 172560/225000 [330:14:04<84:58:18,  5.83s/it]
 77%|███████▋  | 172561/225000 [330:14:10<87:24:27,  6.00s/it]
 77%|███████▋  | 172562/225000 [330:14:14<79:16:19,  5.44s/it]
 77%|███████▋  | 172563/225000 [330:14:19<76:19:46,  5.24s/it]
 77%|███████▋  | 172564/225000 [330:14:24<73:20:10,  5.03s/it]
 77%|███████▋  | 172565/225000 [330:14:28<70:05:35,  4.81s/it]
 77%|███████▋  | 172566/225000 [330:14:32<67:29:22,  4.63s/it]
 77%|███████▋  | 172567/225000 [330:14:42<91:13:56,  6.26s/it]
 77%|███████▋  | 172568/225000 [330:14:48<88:15:36,  6.06s/it]
 77%|███████▋  | 172569/225000 [330:14:53<83:24:23,  5.73s/it]
 77%|███████▋  | 172570/225000 [330:14:58<80:18:54,  5.51s/it]
 77%|███████▋  | 172571/225000 [330:15:03<78:05:32,  5.36s/it]
 77%|███████▋  | 172572/225000 [330:15:08<76:22:00,  5.24s/it]
 77%|███████▋  | 172573/225000 [330:15:13<76:08:38,  5.23s/it]
 77%|███████▋  | 172574/225000 [330:15:18<75:31:59,  5.19s/it]
 77%|███████▋  | 172575/225000 [330:15:23<75:19:58,  5.17s/it]
 77%|███████▋  | 172576/225000 [330:15:28<72:36:29,  4.99s/it]
 77%|███████▋  | 172577/225000 [330:15:33<74:16:40,  5.10s/it]
 77%|███████▋  | 172578/225000 [330:15:39<76:25:45,  5.25s/it]
 77%|███████▋  | 172579/225000 [330:15:44<74:34:42,  5.12s/it]
 77%|███████▋  | 172580/225000 [330:15:54<95:38:10,  6.57s/it]
 77%|███████▋  | 172581/225000 [330:16:01<98:52:42,  6.79s/it]
 77%|███████▋  | 172582/225000 [330:16:04<84:49:38,  5.83s/it]
 77%|███████▋  | 172583/225000 [330:16:13<95:00:33,  6.53s/it]
 77%|███████▋  | 172584/225000 [330:16:16<80:27:07,  5.53s/it]
 77%|███████▋  | 172585/225000 [330:16:20<76:22:04,  5.25s/it]
 77%|███████▋  | 172586/225000 [330:16:28<85:57:53,  5.90s/it]
 77%|███████▋  | 172587/225000 [330:16:34<86:30:48,  5.94s/it]
 77%|███████▋  | 172588/225000 [330:16:37<74:59:40,  5.15s/it]
 77%|███████▋  | 172589/225000 [330:16:43<79:27:29,  5.46s/it]
 77%|███████▋  | 172590/225000 [330:16:49<82:20:46,  5.66s/it]
 77%|███████▋  | 172591/225000 [330:16:53<73:53:02,  5.08s/it]
 77%|███████▋  | 172592/225000 [330:16:56<65:51:42,  4.52s/it]
 77%|███████▋  | 172593/225000 [330:17:04<80:40:33,  5.54s/it]
 77%|███████▋  | 172594/225000 [330:17:11<84:37:31,  5.81s/it]
 77%|███████▋  | 172595/225000 [330:17:14<73:30:05,  5.05s/it]
 77%|███████▋  | 172596/225000 [330:17:21<83:20:31,  5.73s/it]
 77%|███████▋  | 172597/225000 [330:17:25<75:41:12,  5.20s/it]
 77%|███████▋  | 172598/225000 [330:17:29<69:32:48,  4.78s/it]
 77%|███████▋  | 172599/225000 [330:17:34<69:27:24,  4.77s/it]
 77%|███████▋  | 172600/225000 [330:17:44<92:48:52,  6.38s/it]
                                                              
{'loss': '0.7747', 'grad_norm': '1.075', 'learning_rate': '6.998e-06', 'epoch': '0.7671', 'num_input_tokens_seen': 547296000, 'train_runtime': '1.189e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172600/225000 [330:17:44<92:48:52,  6.38s/it]
 77%|███████▋  | 172601/225000 [330:17:48<80:28:31,  5.53s/it]
 77%|███████▋  | 172602/225000 [330:17:57<96:25:47,  6.63s/it]
 77%|███████▋  | 172603/225000 [330:18:00<80:03:56,  5.50s/it]
 77%|███████▋  | 172604/225000 [330:18:04<74:22:24,  5.11s/it]
 77%|███████▋  | 172605/225000 [330:18:09<73:20:08,  5.04s/it]
 77%|███████▋  | 172606/225000 [330:18:14<72:39:10,  4.99s/it]
 77%|███████▋  | 172607/225000 [330:18:18<71:55:20,  4.94s/it]
 77%|███████▋  | 172608/225000 [330:18:23<72:00:03,  4.95s/it]
 77%|███████▋  | 172609/225000 [330:18:28<69:41:30,  4.79s/it]
 77%|███████▋  | 172610/225000 [330:18:33<71:01:29,  4.88s/it]
 77%|███████▋  | 172611/225000 [330:18:38<70:34:58,  4.85s/it]
 77%|███████▋  | 172612/225000 [330:18:43<71:07:53,  4.89s/it]
 77%|███████▋  | 172613/225000 [330:18:47<70:51:14,  4.87s/it]
 77%|███████▋  | 172614/225000 [330:18:52<70:47:40,  4.87s/it]
 77%|███████▋  | 172615/225000 [330:18:57<70:28:38,  4.84s/it]
 77%|███████▋  | 172616/225000 [330:19:02<71:50:15,  4.94s/it]
 77%|███████▋  | 172617/225000 [330:19:14<99:24:38,  6.83s/it]
 77%|███████▋  | 172618/225000 [330:19:19<95:28:22,  6.56s/it]
 77%|███████▋  | 172619/225000 [330:19:23<83:14:36,  5.72s/it]
 77%|███████▋  | 172620/225000 [330:19:33<99:12:32,  6.82s/it]
 77%|███████▋  | 172621/225000 [330:19:36<85:29:44,  5.88s/it]
 77%|███████▋  | 172622/225000 [330:19:43<88:30:42,  6.08s/it]
 77%|███████▋  | 172623/225000 [330:19:46<76:47:54,  5.28s/it]
 77%|███████▋  | 172624/225000 [330:19:55<92:54:54,  6.39s/it]
 77%|███████▋  | 172625/225000 [330:20:00<86:15:07,  5.93s/it]
 77%|███████▋  | 172626/225000 [330:20:05<80:23:15,  5.53s/it]
 77%|███████▋  | 172627/225000 [330:20:09<74:43:28,  5.14s/it]
 77%|███████▋  | 172628/225000 [330:20:14<72:58:40,  5.02s/it]
 77%|███████▋  | 172629/225000 [330:20:18<71:08:57,  4.89s/it]
 77%|███████▋  | 172630/225000 [330:20:23<70:05:09,  4.82s/it]
 77%|███████▋  | 172631/225000 [330:20:27<68:22:40,  4.70s/it]
 77%|███████▋  | 172632/225000 [330:20:37<89:07:39,  6.13s/it]
 77%|███████▋  | 172633/225000 [330:20:44<93:10:37,  6.41s/it]
 77%|███████▋  | 172634/225000 [330:20:51<95:08:17,  6.54s/it]
 77%|███████▋  | 172635/225000 [330:20:54<80:34:02,  5.54s/it]
 77%|███████▋  | 172636/225000 [330:21:01<86:13:26,  5.93s/it]
 77%|███████▋  | 172637/225000 [330:21:05<78:30:36,  5.40s/it]
 77%|███████▋  | 172638/225000 [330:21:09<71:21:47,  4.91s/it]
 77%|███████▋  | 172639/225000 [330:21:17<88:27:56,  6.08s/it]
 77%|███████▋  | 172640/225000 [330:21:21<76:41:48,  5.27s/it]
 77%|███████▋  | 172641/225000 [330:21:26<76:08:55,  5.24s/it]
 77%|███████▋  | 172642/225000 [330:21:32<81:04:54,  5.57s/it]
 77%|███████▋  | 172643/225000 [330:21:36<70:57:59,  4.88s/it]
 77%|███████▋  | 172644/225000 [330:21:43<82:08:03,  5.65s/it]
 77%|███████▋  | 172645/225000 [330:21:49<84:07:56,  5.79s/it]
 77%|███████▋  | 172646/225000 [330:21:53<74:31:55,  5.13s/it]
 77%|███████▋  | 172647/225000 [330:22:00<85:04:13,  5.85s/it]
 77%|███████▋  | 172648/225000 [330:22:05<79:12:02,  5.45s/it]
 77%|███████▋  | 172649/225000 [330:22:09<74:21:03,  5.11s/it]
 77%|███████▋  | 172650/225000 [330:22:14<71:30:51,  4.92s/it]
                                                              
{'loss': '0.7763', 'grad_norm': '1.037', 'learning_rate': '6.985e-06', 'epoch': '0.7673', 'num_input_tokens_seen': 547425824, 'train_runtime': '1.189e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172650/225000 [330:22:14<71:30:51,  4.92s/it]
 77%|███████▋  | 172651/225000 [330:22:18<69:22:12,  4.77s/it]
 77%|███████▋  | 172652/225000 [330:22:22<67:25:30,  4.64s/it]
 77%|███████▋  | 172653/225000 [330:22:33<94:28:57,  6.50s/it]
 77%|███████▋  | 172654/225000 [330:22:37<81:34:09,  5.61s/it]
 77%|███████▋  | 172655/225000 [330:22:41<73:46:13,  5.07s/it]
 77%|███████▋  | 172656/225000 [330:22:44<67:57:01,  4.67s/it]
 77%|███████▋  | 172657/225000 [330:22:55<94:30:43,  6.50s/it]
 77%|███████▋  | 172658/225000 [330:22:58<80:53:36,  5.56s/it]
 77%|███████▋  | 172659/225000 [330:23:05<86:03:24,  5.92s/it]
 77%|███████▋  | 172660/225000 [330:23:09<74:51:23,  5.15s/it]
 77%|███████▋  | 172661/225000 [330:23:18<93:10:15,  6.41s/it]
 77%|███████▋  | 172662/225000 [330:23:22<84:28:21,  5.81s/it]
 77%|███████▋  | 172663/225000 [330:23:26<76:58:43,  5.29s/it]
 77%|███████▋  | 172664/225000 [330:23:30<71:47:56,  4.94s/it]
 77%|███████▋  | 172665/225000 [330:23:42<98:55:05,  6.80s/it]
 77%|███████▋  | 172666/225000 [330:23:45<86:00:23,  5.92s/it]
 77%|███████▋  | 172667/225000 [330:23:49<76:12:02,  5.24s/it]
 77%|███████▋  | 172668/225000 [330:23:53<69:55:53,  4.81s/it]
 77%|███████▋  | 172669/225000 [330:24:01<84:28:49,  5.81s/it]
 77%|███████▋  | 172670/225000 [330:24:05<74:39:29,  5.14s/it]
 77%|███████▋  | 172671/225000 [330:24:09<70:46:02,  4.87s/it]
 77%|███████▋  | 172672/225000 [330:24:18<87:52:12,  6.05s/it]
 77%|███████▋  | 172673/225000 [330:24:22<79:07:05,  5.44s/it]
 77%|███████▋  | 172674/225000 [330:24:26<73:58:33,  5.09s/it]
 77%|███████▋  | 172675/225000 [330:24:30<69:45:51,  4.80s/it]
 77%|███████▋  | 172676/225000 [330:24:34<66:04:43,  4.55s/it]
 77%|███████▋  | 172677/225000 [330:24:45<92:55:42,  6.39s/it]
 77%|███████▋  | 172678/225000 [330:24:48<79:58:45,  5.50s/it]
 77%|███████▋  | 172679/225000 [330:24:55<85:16:37,  5.87s/it]
 77%|███████▋  | 172680/225000 [330:25:02<89:46:43,  6.18s/it]
 77%|███████▋  | 172681/225000 [330:25:05<77:39:40,  5.34s/it]
 77%|███████▋  | 172682/225000 [330:25:09<71:23:26,  4.91s/it]
 77%|███████▋  | 172683/225000 [330:25:17<86:24:27,  5.95s/it]
 77%|███████▋  | 172684/225000 [330:25:24<88:54:55,  6.12s/it]
 77%|███████▋  | 172685/225000 [330:25:27<77:26:29,  5.33s/it]
 77%|███████▋  | 172686/225000 [330:25:31<71:19:16,  4.91s/it]
 77%|███████▋  | 172687/225000 [330:25:37<73:42:11,  5.07s/it]
 77%|███████▋  | 172688/225000 [330:25:42<74:54:04,  5.15s/it]
 77%|███████▋  | 172689/225000 [330:25:52<95:25:18,  6.57s/it]
 77%|███████▋  | 172690/225000 [330:25:56<84:45:17,  5.83s/it]
 77%|███████▋  | 172691/225000 [330:26:00<76:45:37,  5.28s/it]
 77%|███████▋  | 172692/225000 [330:26:08<87:51:47,  6.05s/it]
 77%|███████▋  | 172693/225000 [330:26:13<84:17:29,  5.80s/it]
 77%|███████▋  | 172694/225000 [330:26:17<76:26:39,  5.26s/it]
 77%|███████▋  | 172695/225000 [330:26:23<79:49:26,  5.49s/it]
 77%|███████▋  | 172696/225000 [330:26:32<92:39:39,  6.38s/it]
 77%|███████▋  | 172697/225000 [330:26:35<79:33:40,  5.48s/it]
 77%|███████▋  | 172698/225000 [330:26:41<83:19:46,  5.74s/it]
 77%|███████▋  | 172699/225000 [330:26:45<75:40:55,  5.21s/it]
 77%|███████▋  | 172700/225000 [330:26:51<76:12:13,  5.25s/it]
                                                              
{'loss': '0.7738', 'grad_norm': '1.134', 'learning_rate': '6.972e-06', 'epoch': '0.7676', 'num_input_tokens_seen': 547556864, 'train_runtime': '1.19e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172700/225000 [330:26:51<76:12:13,  5.25s/it]
 77%|███████▋  | 172701/225000 [330:26:59<89:13:13,  6.14s/it]
 77%|███████▋  | 172702/225000 [330:27:02<77:34:08,  5.34s/it]
 77%|███████▋  | 172703/225000 [330:27:12<94:20:24,  6.49s/it]
 77%|███████▋  | 172704/225000 [330:27:15<80:44:17,  5.56s/it]
 77%|███████▋  | 172705/225000 [330:27:19<75:32:33,  5.20s/it]
 77%|███████▋  | 172706/225000 [330:27:27<84:02:27,  5.79s/it]
 77%|███████▋  | 172707/225000 [330:27:30<75:52:53,  5.22s/it]
 77%|███████▋  | 172708/225000 [330:27:34<69:19:46,  4.77s/it]
 77%|███████▋  | 172709/225000 [330:27:38<65:42:05,  4.52s/it]
 77%|███████▋  | 172710/225000 [330:27:49<91:41:57,  6.31s/it]
 77%|███████▋  | 172711/225000 [330:27:53<84:03:00,  5.79s/it]
 77%|███████▋  | 172712/225000 [330:27:58<80:51:24,  5.57s/it]
 77%|███████▋  | 172713/225000 [330:28:04<83:16:42,  5.73s/it]
 77%|███████▋  | 172714/225000 [330:28:08<74:42:08,  5.14s/it]
 77%|███████▋  | 172715/225000 [330:28:12<70:24:43,  4.85s/it]
 77%|███████▋  | 172716/225000 [330:28:16<66:46:26,  4.60s/it]
 77%|███████▋  | 172717/225000 [330:28:27<92:24:18,  6.36s/it]
 77%|███████▋  | 172718/225000 [330:28:30<78:33:39,  5.41s/it]
 77%|███████▋  | 172719/225000 [330:28:37<86:50:00,  5.98s/it]
 77%|███████▋  | 172720/225000 [330:28:41<78:23:27,  5.40s/it]
 77%|███████▋  | 172721/225000 [330:28:45<72:53:00,  5.02s/it]
 77%|███████▋  | 172722/225000 [330:28:50<69:36:16,  4.79s/it]
 77%|███████▋  | 172723/225000 [330:29:00<92:42:33,  6.38s/it]
 77%|███████▋  | 172724/225000 [330:29:05<87:16:05,  6.01s/it]
 77%|███████▋  | 172725/225000 [330:29:08<74:38:07,  5.14s/it]
 77%|███████▋  | 172726/225000 [330:29:17<90:26:04,  6.23s/it]
 77%|███████▋  | 172727/225000 [330:29:20<78:48:06,  5.43s/it]
 77%|███████▋  | 172728/225000 [330:29:28<90:26:20,  6.23s/it]
 77%|███████▋  | 172729/225000 [330:29:32<77:47:56,  5.36s/it]
 77%|███████▋  | 172730/225000 [330:29:40<91:07:22,  6.28s/it]
 77%|███████▋  | 172731/225000 [330:29:43<77:24:01,  5.33s/it]
 77%|███████▋  | 172732/225000 [330:29:47<68:46:56,  4.74s/it]
 77%|███████▋  | 172733/225000 [330:29:56<88:19:49,  6.08s/it]
 77%|███████▋  | 172734/225000 [330:29:59<75:41:06,  5.21s/it]
 77%|███████▋  | 172735/225000 [330:30:05<77:51:55,  5.36s/it]
 77%|███████▋  | 172736/225000 [330:30:12<87:51:19,  6.05s/it]
 77%|███████▋  | 172737/225000 [330:30:17<79:12:47,  5.46s/it]
 77%|███████▋  | 172738/225000 [330:30:20<71:53:04,  4.95s/it]
 77%|███████▋  | 172739/225000 [330:30:30<91:23:01,  6.29s/it]
 77%|███████▋  | 172740/225000 [330:30:35<86:12:21,  5.94s/it]
 77%|███████▋  | 172741/225000 [330:30:40<81:52:25,  5.64s/it]
 77%|███████▋  | 172742/225000 [330:30:44<77:28:07,  5.34s/it]
 77%|███████▋  | 172743/225000 [330:30:50<76:27:19,  5.27s/it]
 77%|███████▋  | 172744/225000 [330:30:54<73:47:58,  5.08s/it]
 77%|███████▋  | 172745/225000 [330:30:59<74:44:17,  5.15s/it]
 77%|███████▋  | 172746/225000 [330:31:05<74:21:23,  5.12s/it]
 77%|███████▋  | 172747/225000 [330:31:10<74:29:07,  5.13s/it]
 77%|███████▋  | 172748/225000 [330:31:15<75:45:48,  5.22s/it]
 77%|███████▋  | 172749/225000 [330:31:20<74:23:30,  5.13s/it]
 77%|███████▋  | 172750/225000 [330:31:25<74:28:38,  5.13s/it]
                                                              
{'loss': '0.764', 'grad_norm': '1.162', 'learning_rate': '6.96e-06', 'epoch': '0.7678', 'num_input_tokens_seen': 547687512, 'train_runtime': '1.19e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172750/225000 [330:31:25<74:28:38,  5.13s/it]
 77%|███████▋  | 172751/225000 [330:31:30<74:55:03,  5.16s/it]
 77%|███████▋  | 172752/225000 [330:31:35<73:53:13,  5.09s/it]
 77%|███████▋  | 172753/225000 [330:31:40<71:50:43,  4.95s/it]
 77%|███████▋  | 172754/225000 [330:31:45<70:28:36,  4.86s/it]
 77%|███████▋  | 172755/225000 [330:31:50<72:32:42,  5.00s/it]
 77%|███████▋  | 172756/225000 [330:31:55<72:10:08,  4.97s/it]
 77%|███████▋  | 172757/225000 [330:32:00<72:30:29,  5.00s/it]
 77%|███████▋  | 172758/225000 [330:32:05<72:43:09,  5.01s/it]
 77%|███████▋  | 172759/225000 [330:32:10<72:35:27,  5.00s/it]
 77%|███████▋  | 172760/225000 [330:32:23<105:46:30,  7.29s/it]
 77%|███████▋  | 172761/225000 [330:32:28<99:13:11,  6.84s/it] 
 77%|███████▋  | 172762/225000 [330:32:35<100:15:06,  6.91s/it]
 77%|███████▋  | 172763/225000 [330:32:40<88:15:15,  6.08s/it] 
 77%|███████▋  | 172764/225000 [330:32:44<80:41:12,  5.56s/it]
 77%|███████▋  | 172765/225000 [330:32:49<76:44:39,  5.29s/it]
 77%|███████▋  | 172766/225000 [330:32:53<71:42:30,  4.94s/it]
 77%|███████▋  | 172767/225000 [330:32:57<69:06:13,  4.76s/it]
 77%|███████▋  | 172768/225000 [330:33:06<85:45:00,  5.91s/it]
 77%|███████▋  | 172769/225000 [330:33:13<90:21:34,  6.23s/it]
 77%|███████▋  | 172770/225000 [330:33:18<86:50:10,  5.99s/it]
 77%|███████▋  | 172771/225000 [330:33:25<89:06:05,  6.14s/it]
 77%|███████▋  | 172772/225000 [330:33:28<79:11:50,  5.46s/it]
 77%|███████▋  | 172773/225000 [330:33:38<95:44:15,  6.60s/it]
 77%|███████▋  | 172774/225000 [330:33:41<82:57:08,  5.72s/it]
 77%|███████▋  | 172775/225000 [330:33:45<74:11:47,  5.11s/it]
 77%|███████▋  | 172776/225000 [330:33:49<69:30:10,  4.79s/it]
 77%|███████▋  | 172777/225000 [330:33:53<67:12:17,  4.63s/it]
 77%|███████▋  | 172778/225000 [330:33:58<66:44:20,  4.60s/it]
 77%|███████▋  | 172779/225000 [330:34:02<66:04:38,  4.56s/it]
 77%|███████▋  | 172780/225000 [330:34:07<66:04:54,  4.56s/it]
 77%|███████▋  | 172781/225000 [330:34:19<98:49:00,  6.81s/it]
 77%|███████▋  | 172782/225000 [330:34:23<86:54:25,  5.99s/it]
 77%|███████▋  | 172783/225000 [330:34:27<78:16:05,  5.40s/it]
 77%|███████▋  | 172784/225000 [330:34:31<73:30:36,  5.07s/it]
 77%|███████▋  | 172785/225000 [330:34:36<71:53:00,  4.96s/it]
 77%|███████▋  | 172786/225000 [330:34:40<69:41:02,  4.80s/it]
 77%|███████▋  | 172787/225000 [330:34:45<67:35:32,  4.66s/it]
 77%|███████▋  | 172788/225000 [330:34:52<79:41:01,  5.49s/it]
 77%|███████▋  | 172789/225000 [330:35:02<96:25:40,  6.65s/it]
 77%|███████▋  | 172790/225000 [330:35:05<82:01:40,  5.66s/it]
 77%|███████▋  | 172791/225000 [330:35:08<72:29:44,  5.00s/it]
 77%|███████▋  | 172792/225000 [330:35:14<73:29:41,  5.07s/it]
 77%|███████▋  | 172793/225000 [330:35:19<74:21:46,  5.13s/it]
 77%|███████▋  | 172794/225000 [330:35:24<76:01:37,  5.24s/it]
 77%|███████▋  | 172795/225000 [330:35:30<75:59:26,  5.24s/it]
 77%|███████▋  | 172796/225000 [330:35:34<73:22:09,  5.06s/it]
 77%|███████▋  | 172797/225000 [330:35:39<74:09:27,  5.11s/it]
 77%|███████▋  | 172798/225000 [330:35:52<106:05:26,  7.32s/it]
 77%|███████▋  | 172799/225000 [330:35:55<89:05:51,  6.14s/it] 
 77%|███████▋  | 172800/225000 [330:36:01<87:01:50,  6.00s/it]
                                                              
{'loss': '0.7715', 'grad_norm': '1.134', 'learning_rate': '6.947e-06', 'epoch': '0.768', 'num_input_tokens_seen': 547816960, 'train_runtime': '1.19e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172800/225000 [330:36:01<87:01:50,  6.00s/it]
 77%|███████▋  | 172801/225000 [330:36:08<89:28:31,  6.17s/it]
 77%|███████▋  | 172802/225000 [330:36:12<83:15:57,  5.74s/it]
 77%|███████▋  | 172803/225000 [330:36:17<77:26:32,  5.34s/it]
 77%|███████▋  | 172804/225000 [330:36:21<73:30:15,  5.07s/it]
 77%|███████▋  | 172805/225000 [330:36:26<70:41:56,  4.88s/it]
 77%|███████▋  | 172806/225000 [330:36:30<69:12:06,  4.77s/it]
 77%|███████▋  | 172807/225000 [330:36:34<66:54:33,  4.62s/it]
 77%|███████▋  | 172808/225000 [330:36:45<91:44:18,  6.33s/it]
 77%|███████▋  | 172809/225000 [330:36:51<92:16:13,  6.36s/it]
 77%|███████▋  | 172810/225000 [330:36:55<83:00:45,  5.73s/it]
 77%|███████▋  | 172811/225000 [330:37:00<76:05:22,  5.25s/it]
 77%|███████▋  | 172812/225000 [330:37:04<71:04:48,  4.90s/it]
 77%|███████▋  | 172813/225000 [330:37:14<96:37:27,  6.67s/it]
 77%|███████▋  | 172814/225000 [330:37:18<82:55:52,  5.72s/it]
 77%|███████▋  | 172815/225000 [330:37:22<74:11:50,  5.12s/it]
 77%|███████▋  | 172816/225000 [330:37:28<80:49:33,  5.58s/it]
 77%|███████▋  | 172817/225000 [330:37:35<86:27:37,  5.96s/it]
 77%|███████▋  | 172818/225000 [330:37:39<76:16:43,  5.26s/it]
 77%|███████▋  | 172819/225000 [330:37:47<90:17:07,  6.23s/it]
 77%|███████▋  | 172820/225000 [330:37:53<88:06:00,  6.08s/it]
 77%|███████▋  | 172821/225000 [330:37:57<81:10:04,  5.60s/it]
 77%|███████▋  | 172822/225000 [330:38:02<75:47:29,  5.23s/it]
 77%|███████▋  | 172823/225000 [330:38:06<72:13:49,  4.98s/it]
 77%|███████▋  | 172824/225000 [330:38:11<70:26:50,  4.86s/it]
 77%|███████▋  | 172825/225000 [330:38:15<66:51:27,  4.61s/it]
 77%|███████▋  | 172826/225000 [330:38:19<66:12:12,  4.57s/it]
 77%|███████▋  | 172827/225000 [330:38:31<95:16:07,  6.57s/it]
 77%|███████▋  | 172828/225000 [330:38:36<90:17:37,  6.23s/it]
 77%|███████▋  | 172829/225000 [330:38:40<79:09:17,  5.46s/it]
 77%|███████▋  | 172830/225000 [330:38:46<81:24:19,  5.62s/it]
 77%|███████▋  | 172831/225000 [330:38:52<85:35:25,  5.91s/it]
 77%|███████▋  | 172832/225000 [330:38:58<85:40:06,  5.91s/it]
 77%|███████▋  | 172833/225000 [330:39:02<76:39:11,  5.29s/it]
 77%|███████▋  | 172834/225000 [330:39:06<69:30:54,  4.80s/it]
 77%|███████▋  | 172835/225000 [330:39:14<84:57:46,  5.86s/it]
 77%|███████▋  | 172836/225000 [330:39:17<73:57:18,  5.10s/it]
 77%|███████▋  | 172837/225000 [330:39:25<86:21:28,  5.96s/it]
 77%|███████▋  | 172838/225000 [330:39:29<75:59:47,  5.24s/it]
 77%|███████▋  | 172839/225000 [330:39:33<69:45:48,  4.81s/it]
 77%|███████▋  | 172840/225000 [330:39:41<85:58:28,  5.93s/it]
 77%|███████▋  | 172841/225000 [330:39:46<82:22:45,  5.69s/it]
 77%|███████▋  | 172842/225000 [330:39:50<74:46:59,  5.16s/it]
 77%|███████▋  | 172843/225000 [330:39:56<78:05:39,  5.39s/it]
 77%|███████▋  | 172844/225000 [330:40:02<80:40:23,  5.57s/it]
 77%|███████▋  | 172845/225000 [330:40:06<73:05:06,  5.04s/it]
 77%|███████▋  | 172846/225000 [330:40:16<92:56:06,  6.41s/it]
 77%|███████▋  | 172847/225000 [330:40:20<83:13:25,  5.74s/it]
 77%|███████▋  | 172848/225000 [330:40:23<74:31:52,  5.14s/it]
 77%|███████▋  | 172849/225000 [330:40:32<89:45:28,  6.20s/it]
 77%|███████▋  | 172850/225000 [330:40:37<82:24:51,  5.69s/it]
                                                              
{'loss': '0.7615', 'grad_norm': '1.058', 'learning_rate': '6.934e-06', 'epoch': '0.7682', 'num_input_tokens_seen': 547945624, 'train_runtime': '1.19e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172850/225000 [330:40:37<82:24:51,  5.69s/it]
 77%|███████▋  | 172851/225000 [330:40:41<75:31:45,  5.21s/it]
 77%|███████▋  | 172852/225000 [330:40:45<70:12:34,  4.85s/it]
 77%|███████▋  | 172853/225000 [330:40:51<75:29:14,  5.21s/it]
 77%|███████▋  | 172854/225000 [330:40:59<88:36:43,  6.12s/it]
 77%|███████▋  | 172855/225000 [330:41:03<78:00:19,  5.39s/it]
 77%|███████▋  | 172856/225000 [330:41:07<71:11:32,  4.92s/it]
 77%|███████▋  | 172857/225000 [330:41:17<95:41:14,  6.61s/it]
 77%|███████▋  | 172858/225000 [330:41:22<87:36:54,  6.05s/it]
 77%|███████▋  | 172859/225000 [330:41:26<79:40:13,  5.50s/it]
 77%|███████▋  | 172860/225000 [330:41:30<73:35:45,  5.08s/it]
 77%|███████▋  | 172861/225000 [330:41:35<71:46:51,  4.96s/it]
 77%|███████▋  | 172862/225000 [330:41:39<70:22:53,  4.86s/it]
 77%|███████▋  | 172863/225000 [330:41:44<68:18:19,  4.72s/it]
 77%|███████▋  | 172864/225000 [330:41:48<65:57:11,  4.55s/it]
 77%|███████▋  | 172865/225000 [330:41:55<75:39:58,  5.22s/it]
 77%|███████▋  | 172866/225000 [330:42:04<91:52:09,  6.34s/it]
 77%|███████▋  | 172867/225000 [330:42:08<83:09:54,  5.74s/it]
 77%|███████▋  | 172868/225000 [330:42:14<85:33:58,  5.91s/it]
 77%|███████▋  | 172869/225000 [330:42:18<73:41:32,  5.09s/it]
 77%|███████▋  | 172870/225000 [330:42:25<83:01:09,  5.73s/it]
 77%|███████▋  | 172871/225000 [330:42:29<74:20:36,  5.13s/it]
 77%|███████▋  | 172872/225000 [330:42:37<88:25:08,  6.11s/it]
 77%|███████▋  | 172873/225000 [330:42:41<78:41:47,  5.43s/it]
 77%|███████▋  | 172874/225000 [330:42:45<73:42:53,  5.09s/it]
 77%|███████▋  | 172875/225000 [330:42:49<70:42:36,  4.88s/it]
 77%|███████▋  | 172876/225000 [330:42:54<67:59:05,  4.70s/it]
 77%|███████▋  | 172877/225000 [330:43:05<98:02:58,  6.77s/it]
 77%|███████▋  | 172878/225000 [330:43:10<86:58:33,  6.01s/it]
 77%|███████▋  | 172879/225000 [330:43:13<77:41:53,  5.37s/it]
 77%|███████▋  | 172880/225000 [330:43:17<71:37:25,  4.95s/it]
 77%|███████▋  | 172881/225000 [330:43:27<91:25:34,  6.32s/it]
 77%|███████▋  | 172882/225000 [330:43:32<85:43:12,  5.92s/it]
 77%|███████▋  | 172883/225000 [330:43:37<80:50:29,  5.58s/it]
 77%|███████▋  | 172884/225000 [330:43:41<76:10:57,  5.26s/it]
 77%|███████▋  | 172885/225000 [330:43:46<75:09:41,  5.19s/it]
 77%|███████▋  | 172886/225000 [330:43:51<71:42:29,  4.95s/it]
 77%|███████▋  | 172887/225000 [330:43:55<69:27:56,  4.80s/it]
 77%|███████▋  | 172888/225000 [330:44:00<70:31:58,  4.87s/it]
 77%|███████▋  | 172889/225000 [330:44:05<70:12:40,  4.85s/it]
 77%|███████▋  | 172890/225000 [330:44:10<69:29:22,  4.80s/it]
 77%|███████▋  | 172891/225000 [330:44:15<70:06:37,  4.84s/it]
 77%|███████▋  | 172892/225000 [330:44:25<93:21:34,  6.45s/it]
 77%|███████▋  | 172893/225000 [330:44:31<93:47:59,  6.48s/it]
 77%|███████▋  | 172894/225000 [330:44:35<82:30:31,  5.70s/it]
 77%|███████▋  | 172895/225000 [330:44:42<85:08:50,  5.88s/it]
 77%|███████▋  | 172896/225000 [330:44:49<91:05:08,  6.29s/it]
 77%|███████▋  | 172897/225000 [330:44:52<79:49:48,  5.52s/it]
 77%|███████▋  | 172898/225000 [330:45:01<93:01:05,  6.43s/it]
 77%|███████▋  | 172899/225000 [330:45:04<80:05:48,  5.53s/it]
 77%|███████▋  | 172900/225000 [330:45:11<85:32:05,  5.91s/it]
                                                              
{'loss': '0.767', 'grad_norm': '1.104', 'learning_rate': '6.922e-06', 'epoch': '0.7684', 'num_input_tokens_seen': 548074760, 'train_runtime': '1.191e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172900/225000 [330:45:11<85:32:05,  5.91s/it]
 77%|███████▋  | 172901/225000 [330:45:15<77:39:02,  5.37s/it]
 77%|███████▋  | 172902/225000 [330:45:19<70:50:54,  4.90s/it]
 77%|███████▋  | 172903/225000 [330:45:29<92:03:40,  6.36s/it]
 77%|███████▋  | 172904/225000 [330:45:32<79:18:09,  5.48s/it]
 77%|███████▋  | 172905/225000 [330:45:40<86:56:58,  6.01s/it]
 77%|███████▋  | 172906/225000 [330:45:44<81:46:53,  5.65s/it]
 77%|███████▋  | 172907/225000 [330:45:48<74:48:36,  5.17s/it]
 77%|███████▋  | 172908/225000 [330:45:53<72:36:48,  5.02s/it]
 77%|███████▋  | 172909/225000 [330:45:58<72:33:15,  5.01s/it]
 77%|███████▋  | 172910/225000 [330:46:03<72:13:35,  4.99s/it]
 77%|███████▋  | 172911/225000 [330:46:08<72:10:57,  4.99s/it]
 77%|███████▋  | 172912/225000 [330:46:13<71:23:09,  4.93s/it]
 77%|███████▋  | 172913/225000 [330:46:18<70:09:52,  4.85s/it]
 77%|███████▋  | 172914/225000 [330:46:23<71:20:30,  4.93s/it]
 77%|███████▋  | 172915/225000 [330:46:27<70:40:33,  4.88s/it]
 77%|███████▋  | 172916/225000 [330:46:32<70:21:38,  4.86s/it]
 77%|███████▋  | 172917/225000 [330:46:37<69:48:35,  4.83s/it]
 77%|███████▋  | 172918/225000 [330:46:44<80:53:07,  5.59s/it]
 77%|███████▋  | 172919/225000 [330:46:56<106:37:25,  7.37s/it]
 77%|███████▋  | 172920/225000 [330:46:59<89:09:23,  6.16s/it] 
 77%|███████▋  | 172921/225000 [330:47:05<85:26:53,  5.91s/it]
 77%|███████▋  | 172922/225000 [330:47:10<81:46:43,  5.65s/it]
 77%|███████▋  | 172923/225000 [330:47:15<79:57:26,  5.53s/it]
 77%|███████▋  | 172924/225000 [330:47:20<79:45:09,  5.51s/it]
 77%|███████▋  | 172925/225000 [330:47:26<79:11:55,  5.48s/it]
 77%|███████▋  | 172926/225000 [330:47:31<78:48:20,  5.45s/it]
 77%|███████▋  | 172927/225000 [330:47:36<77:02:59,  5.33s/it]
 77%|███████▋  | 172928/225000 [330:47:42<78:04:15,  5.40s/it]
 77%|███████▋  | 172929/225000 [330:47:47<75:52:28,  5.25s/it]
 77%|███████▋  | 172930/225000 [330:47:52<75:40:28,  5.23s/it]
 77%|███████▋  | 172931/225000 [330:47:57<76:10:28,  5.27s/it]
 77%|███████▋  | 172932/225000 [330:48:03<77:20:58,  5.35s/it]
 77%|███████▋  | 172933/225000 [330:48:08<77:31:18,  5.36s/it]
 77%|███████▋  | 172934/225000 [330:48:13<77:59:44,  5.39s/it]
 77%|███████▋  | 172935/225000 [330:48:19<78:31:52,  5.43s/it]
 77%|███████▋  | 172936/225000 [330:48:24<77:41:46,  5.37s/it]
 77%|███████▋  | 172937/225000 [330:48:30<77:13:50,  5.34s/it]
 77%|███████▋  | 172938/225000 [330:48:35<78:58:19,  5.46s/it]
 77%|███████▋  | 172939/225000 [330:48:40<77:22:31,  5.35s/it]
 77%|███████▋  | 172940/225000 [330:48:46<77:34:33,  5.36s/it]
 77%|███████▋  | 172941/225000 [330:48:51<78:10:40,  5.41s/it]
 77%|███████▋  | 172942/225000 [330:48:57<77:43:01,  5.37s/it]
 77%|███████▋  | 172943/225000 [330:49:02<77:19:53,  5.35s/it]
 77%|███████▋  | 172944/225000 [330:49:07<77:32:21,  5.36s/it]
 77%|███████▋  | 172945/225000 [330:49:17<97:26:53,  6.74s/it]
 77%|███████▋  | 172946/225000 [330:49:23<92:48:52,  6.42s/it]
 77%|███████▋  | 172947/225000 [330:49:28<86:26:39,  5.98s/it]
 77%|███████▋  | 172948/225000 [330:49:33<81:17:19,  5.62s/it]
 77%|███████▋  | 172949/225000 [330:49:38<78:56:44,  5.46s/it]
 77%|███████▋  | 172950/225000 [330:49:43<78:14:07,  5.41s/it]
                                                              
{'loss': '0.7713', 'grad_norm': '1.202', 'learning_rate': '6.909e-06', 'epoch': '0.7687', 'num_input_tokens_seen': 548204328, 'train_runtime': '1.191e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 172950/225000 [330:49:43<78:14:07,  5.41s/it]
 77%|███████▋  | 172951/225000 [330:49:48<78:17:02,  5.41s/it]
 77%|███████▋  | 172952/225000 [330:49:54<77:15:57,  5.34s/it]
 77%|███████▋  | 172953/225000 [330:49:59<75:55:54,  5.25s/it]
 77%|███████▋  | 172954/225000 [330:50:04<77:23:57,  5.35s/it]
 77%|███████▋  | 172955/225000 [330:50:09<76:57:36,  5.32s/it]
 77%|███████▋  | 172956/225000 [330:50:15<76:59:55,  5.33s/it]
 77%|███████▋  | 172957/225000 [330:50:20<74:52:19,  5.18s/it]
 77%|███████▋  | 172958/225000 [330:50:25<73:47:03,  5.10s/it]
 77%|███████▋  | 172959/225000 [330:50:30<75:11:34,  5.20s/it]
 77%|███████▋  | 172960/225000 [330:50:35<73:31:38,  5.09s/it]
 77%|███████▋  | 172961/225000 [330:50:40<73:59:27,  5.12s/it]
 77%|███████▋  | 172962/225000 [330:50:46<76:14:23,  5.27s/it]
 77%|███████▋  | 172963/225000 [330:50:51<76:00:19,  5.26s/it]
 77%|███████▋  | 172964/225000 [330:50:56<76:36:47,  5.30s/it]
 77%|███████▋  | 172965/225000 [330:51:01<76:02:37,  5.26s/it]
 77%|███████▋  | 172966/225000 [330:51:07<77:16:11,  5.35s/it]
 77%|███████▋  | 172967/225000 [330:51:12<77:53:50,  5.39s/it]
 77%|███████▋  | 172968/225000 [330:51:18<76:54:17,  5.32s/it]
 77%|███████▋  | 172969/225000 [330:51:23<77:18:17,  5.35s/it]
 77%|███████▋  | 172970/225000 [330:51:28<76:14:57,  5.28s/it]
 77%|███████▋  | 172971/225000 [330:51:39<99:31:20,  6.89s/it]
 77%|███████▋  | 172972/225000 [330:51:44<94:20:31,  6.53s/it]
 77%|███████▋  | 172973/225000 [330:51:52<100:32:48,  6.96s/it]
 77%|███████▋  | 172974/225000 [330:51:56<85:29:27,  5.92s/it] 
 77%|███████▋  | 172975/225000 [330:52:03<89:11:48,  6.17s/it]
 77%|███████▋  | 172976/225000 [330:52:08<84:42:06,  5.86s/it]
 77%|███████▋  | 172977/225000 [330:52:13<80:41:26,  5.58s/it]
 77%|███████▋  | 172978/225000 [330:52:18<78:06:37,  5.41s/it]
 77%|███████▋  | 172979/225000 [330:52:23<75:17:50,  5.21s/it]
 77%|███████▋  | 172980/225000 [330:52:28<75:55:00,  5.25s/it]
 77%|███████▋  | 172981/225000 [330:52:33<76:48:11,  5.32s/it]
 77%|███████▋  | 172982/225000 [330:52:38<75:09:22,  5.20s/it]
 77%|███████▋  | 172983/225000 [330:52:44<75:26:13,  5.22s/it]
 77%|███████▋  | 172984/225000 [330:52:49<74:56:34,  5.19s/it]
 77%|███████▋  | 172985/225000 [330:52:53<73:18:03,  5.07s/it]
 77%|███████▋  | 172986/225000 [330:52:59<74:40:19,  5.17s/it]
 77%|███████▋  | 172987/225000 [330:53:04<74:33:09,  5.16s/it]
 77%|███████▋  | 172988/225000 [330:53:09<75:40:56,  5.24s/it]
 77%|███████▋  | 172989/225000 [330:53:15<75:57:54,  5.26s/it]
 77%|███████▋  | 172990/225000 [330:53:22<85:39:06,  5.93s/it]
 77%|███████▋  | 172991/225000 [330:53:27<81:48:54,  5.66s/it]
 77%|███████▋  | 172992/225000 [330:53:32<79:38:44,  5.51s/it]
 77%|███████▋  | 172993/225000 [330:53:37<77:38:20,  5.37s/it]
 77%|███████▋  | 172994/225000 [330:53:43<77:02:50,  5.33s/it]
 77%|███████▋  | 172995/225000 [330:53:48<76:50:10,  5.32s/it]
 77%|███████▋  | 172996/225000 [330:53:54<77:57:29,  5.40s/it]
 77%|███████▋  | 172997/225000 [330:53:59<77:43:23,  5.38s/it]
 77%|███████▋  | 172998/225000 [330:54:04<75:18:30,  5.21s/it]
 77%|███████▋  | 172999/225000 [330:54:10<81:02:10,  5.61s/it]
 77%|███████▋  | 173000/225000 [330:54:16<80:55:44,  5.60s/it]
                                                              
{'loss': '0.7742', 'grad_norm': '1.078', 'learning_rate': '6.897e-06', 'epoch': '0.7689', 'num_input_tokens_seen': 548333904, 'train_runtime': '1.191e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173000/225000 [330:54:18<80:55:44,  5.60s/it]
 77%|███████▋  | 173001/225000 [330:54:23<88:47:45,  6.15s/it]
 77%|███████▋  | 173002/225000 [330:54:28<84:39:41,  5.86s/it]
 77%|███████▋  | 173003/225000 [330:54:33<81:06:15,  5.62s/it]
 77%|███████▋  | 173004/225000 [330:54:39<78:37:40,  5.44s/it]
 77%|███████▋  | 173005/225000 [330:54:44<77:48:19,  5.39s/it]
 77%|███████▋  | 173006/225000 [330:54:49<77:06:54,  5.34s/it]
 77%|███████▋  | 173007/225000 [330:54:58<92:26:36,  6.40s/it]
 77%|███████▋  | 173008/225000 [330:55:02<83:23:19,  5.77s/it]
 77%|███████▋  | 173009/225000 [330:55:07<77:40:49,  5.38s/it]
 77%|███████▋  | 173010/225000 [330:55:11<72:18:18,  5.01s/it]
 77%|███████▋  | 173011/225000 [330:55:21<94:39:10,  6.55s/it]
 77%|███████▋  | 173012/225000 [330:55:26<87:16:38,  6.04s/it]
 77%|███████▋  | 173013/225000 [330:55:30<77:13:41,  5.35s/it]
 77%|███████▋  | 173014/225000 [330:55:33<70:46:04,  4.90s/it]
 77%|███████▋  | 173015/225000 [330:55:38<68:18:14,  4.73s/it]
 77%|███████▋  | 173016/225000 [330:55:43<71:51:48,  4.98s/it]
 77%|███████▋  | 173017/225000 [330:55:53<92:33:31,  6.41s/it]
 77%|███████▋  | 173018/225000 [330:55:57<80:42:56,  5.59s/it]
 77%|███████▋  | 173019/225000 [330:56:02<79:07:11,  5.48s/it]
 77%|███████▋  | 173020/225000 [330:56:08<79:35:25,  5.51s/it]
 77%|███████▋  | 173021/225000 [330:56:14<83:16:01,  5.77s/it]
 77%|███████▋  | 173022/225000 [330:56:20<86:33:02,  5.99s/it]
 77%|███████▋  | 173023/225000 [330:56:25<79:36:09,  5.51s/it]
 77%|███████▋  | 173024/225000 [330:56:29<73:05:05,  5.06s/it]
 77%|███████▋  | 173025/225000 [330:56:33<69:59:12,  4.85s/it]
 77%|███████▋  | 173026/225000 [330:56:38<69:14:16,  4.80s/it]
 77%|███████▋  | 173027/225000 [330:56:42<68:40:13,  4.76s/it]
 77%|███████▋  | 173028/225000 [330:56:51<83:49:22,  5.81s/it]
 77%|███████▋  | 173029/225000 [330:56:59<96:20:41,  6.67s/it]
 77%|███████▋  | 173030/225000 [330:57:05<92:45:28,  6.43s/it]
 77%|███████▋  | 173031/225000 [330:57:09<82:35:16,  5.72s/it]
 77%|███████▋  | 173032/225000 [330:57:14<76:18:23,  5.29s/it]
 77%|███████▋  | 173033/225000 [330:57:18<71:30:16,  4.95s/it]
 77%|███████▋  | 173034/225000 [330:57:22<66:36:28,  4.61s/it]
 77%|███████▋  | 173035/225000 [330:57:30<83:24:33,  5.78s/it]
 77%|███████▋  | 173036/225000 [330:57:38<92:14:47,  6.39s/it]
 77%|███████▋  | 173037/225000 [330:57:45<95:09:51,  6.59s/it]
 77%|███████▋  | 173038/225000 [330:57:48<81:01:29,  5.61s/it]
 77%|███████▋  | 173039/225000 [330:57:54<82:17:36,  5.70s/it]
 77%|███████▋  | 173040/225000 [330:57:58<73:45:45,  5.11s/it]
 77%|███████▋  | 173041/225000 [330:58:05<83:07:39,  5.76s/it]
 77%|███████▋  | 173042/225000 [330:58:09<75:53:55,  5.26s/it]
 77%|███████▋  | 173043/225000 [330:58:16<80:39:17,  5.59s/it]
 77%|███████▋  | 173044/225000 [330:58:23<89:02:49,  6.17s/it]
 77%|███████▋  | 173045/225000 [330:58:28<83:05:15,  5.76s/it]
 77%|███████▋  | 173046/225000 [330:58:32<76:51:50,  5.33s/it]
 77%|███████▋  | 173047/225000 [330:58:37<74:00:53,  5.13s/it]
 77%|███████▋  | 173048/225000 [330:58:42<71:33:10,  4.96s/it]
 77%|███████▋  | 173049/225000 [330:58:46<70:56:47,  4.92s/it]
 77%|███████▋  | 173050/225000 [330:58:51<69:47:12,  4.84s/it]
                                                              
{'loss': '0.762', 'grad_norm': '1.06', 'learning_rate': '6.884e-06', 'epoch': '0.7691', 'num_input_tokens_seen': 548465144, 'train_runtime': '1.192e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173050/225000 [330:58:51<69:47:12,  4.84s/it]
 77%|███████▋  | 173051/225000 [330:59:03<98:59:18,  6.86s/it]
 77%|███████▋  | 173052/225000 [330:59:07<89:08:47,  6.18s/it]
 77%|███████▋  | 173053/225000 [330:59:11<77:32:53,  5.37s/it]
 77%|███████▋  | 173054/225000 [330:59:19<90:13:36,  6.25s/it]
 77%|███████▋  | 173055/225000 [330:59:23<78:17:31,  5.43s/it]
 77%|███████▋  | 173056/225000 [330:59:28<76:41:51,  5.32s/it]
 77%|███████▋  | 173057/225000 [330:59:34<81:24:35,  5.64s/it]
 77%|███████▋  | 173058/225000 [330:59:38<75:13:00,  5.21s/it]
 77%|███████▋  | 173059/225000 [330:59:42<70:28:56,  4.89s/it]
 77%|███████▋  | 173060/225000 [330:59:47<70:07:25,  4.86s/it]
 77%|███████▋  | 173061/225000 [330:59:52<69:42:14,  4.83s/it]
 77%|███████▋  | 173062/225000 [330:59:57<70:48:34,  4.91s/it]
 77%|███████▋  | 173063/225000 [331:00:02<72:02:35,  4.99s/it]
 77%|███████▋  | 173064/225000 [331:00:07<70:48:08,  4.91s/it]
 77%|███████▋  | 173065/225000 [331:00:12<70:45:46,  4.91s/it]
 77%|███████▋  | 173066/225000 [331:00:17<72:33:54,  5.03s/it]
 77%|███████▋  | 173067/225000 [331:00:28<99:35:59,  6.90s/it]
 77%|███████▋  | 173068/225000 [331:00:34<93:56:30,  6.51s/it]
 77%|███████▋  | 173069/225000 [331:00:40<93:59:28,  6.52s/it]
 77%|███████▋  | 173070/225000 [331:00:44<82:00:32,  5.69s/it]
 77%|███████▋  | 173071/225000 [331:00:53<95:33:03,  6.62s/it]
 77%|███████▋  | 173072/225000 [331:00:58<87:22:30,  6.06s/it]
 77%|███████▋  | 173073/225000 [331:01:04<88:18:11,  6.12s/it]
 77%|███████▋  | 173074/225000 [331:01:07<75:27:50,  5.23s/it]
 77%|███████▋  | 173075/225000 [331:01:12<74:57:56,  5.20s/it]
 77%|███████▋  | 173076/225000 [331:01:21<88:11:41,  6.11s/it]
 77%|███████▋  | 173077/225000 [331:01:24<77:50:19,  5.40s/it]
 77%|███████▋  | 173078/225000 [331:01:29<74:41:59,  5.18s/it]
 77%|███████▋  | 173079/225000 [331:01:34<73:43:28,  5.11s/it]
 77%|███████▋  | 173080/225000 [331:01:39<73:58:29,  5.13s/it]
 77%|███████▋  | 173081/225000 [331:01:44<74:01:51,  5.13s/it]
 77%|███████▋  | 173082/225000 [331:01:50<75:08:11,  5.21s/it]
 77%|███████▋  | 173083/225000 [331:01:54<72:15:06,  5.01s/it]
 77%|███████▋  | 173084/225000 [331:01:59<72:45:48,  5.05s/it]
 77%|███████▋  | 173085/225000 [331:02:04<70:25:39,  4.88s/it]
 77%|███████▋  | 173086/225000 [331:02:09<71:21:30,  4.95s/it]
 77%|███████▋  | 173087/225000 [331:02:14<71:13:11,  4.94s/it]
 77%|███████▋  | 173088/225000 [331:02:19<71:29:48,  4.96s/it]
 77%|███████▋  | 173089/225000 [331:02:25<74:51:11,  5.19s/it]
 77%|███████▋  | 173090/225000 [331:02:37<106:09:41,  7.36s/it]
 77%|███████▋  | 173091/225000 [331:02:40<88:39:13,  6.15s/it] 
 77%|███████▋  | 173092/225000 [331:02:45<82:40:11,  5.73s/it]
 77%|███████▋  | 173093/225000 [331:02:54<95:36:17,  6.63s/it]
 77%|███████▋  | 173094/225000 [331:02:58<85:53:49,  5.96s/it]
 77%|███████▋  | 173095/225000 [331:03:03<79:00:08,  5.48s/it]
 77%|███████▋  | 173096/225000 [331:03:07<73:46:42,  5.12s/it]
 77%|███████▋  | 173097/225000 [331:03:11<71:38:51,  4.97s/it]
 77%|███████▋  | 173098/225000 [331:03:16<68:42:47,  4.77s/it]
 77%|███████▋  | 173099/225000 [331:03:27<97:14:56,  6.75s/it]
 77%|███████▋  | 173100/225000 [331:03:32<88:34:31,  6.14s/it]
                                                              
{'loss': '0.7698', 'grad_norm': '1.343', 'learning_rate': '6.871e-06', 'epoch': '0.7693', 'num_input_tokens_seen': 548595784, 'train_runtime': '1.192e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173100/225000 [331:03:32<88:34:31,  6.14s/it]
 77%|███████▋  | 173101/225000 [331:03:37<83:45:03,  5.81s/it]
 77%|███████▋  | 173102/225000 [331:03:44<89:05:11,  6.18s/it]
 77%|███████▋  | 173103/225000 [331:03:49<83:56:16,  5.82s/it]
 77%|███████▋  | 173104/225000 [331:03:54<81:24:25,  5.65s/it]
 77%|███████▋  | 173105/225000 [331:03:59<78:55:54,  5.48s/it]
 77%|███████▋  | 173106/225000 [331:04:04<74:57:31,  5.20s/it]
 77%|███████▋  | 173107/225000 [331:04:09<75:29:21,  5.24s/it]
 77%|███████▋  | 173108/225000 [331:04:14<73:55:03,  5.13s/it]
 77%|███████▋  | 173109/225000 [331:04:19<73:26:09,  5.09s/it]
 77%|███████▋  | 173110/225000 [331:04:24<71:25:01,  4.95s/it]
 77%|███████▋  | 173111/225000 [331:04:29<73:48:17,  5.12s/it]
 77%|███████▋  | 173112/225000 [331:04:34<73:55:41,  5.13s/it]
 77%|███████▋  | 173113/225000 [331:04:39<74:01:05,  5.14s/it]
 77%|███████▋  | 173114/225000 [331:04:45<73:56:33,  5.13s/it]
 77%|███████▋  | 173115/225000 [331:04:50<73:25:43,  5.09s/it]
 77%|███████▋  | 173116/225000 [331:04:55<74:33:00,  5.17s/it]
 77%|███████▋  | 173117/225000 [331:05:00<74:25:59,  5.16s/it]
 77%|███████▋  | 173118/225000 [331:05:05<73:33:49,  5.10s/it]
 77%|███████▋  | 173119/225000 [331:05:10<73:27:47,  5.10s/it]
 77%|███████▋  | 173120/225000 [331:05:15<72:48:33,  5.05s/it]
 77%|███████▋  | 173121/225000 [331:05:26<99:45:15,  6.92s/it]
 77%|███████▋  | 173122/225000 [331:05:34<104:55:07,  7.28s/it]
 77%|███████▋  | 173123/225000 [331:05:39<92:28:03,  6.42s/it] 
 77%|███████▋  | 173124/225000 [331:05:43<83:26:35,  5.79s/it]
 77%|███████▋  | 173125/225000 [331:05:48<79:21:09,  5.51s/it]
 77%|███████▋  | 173126/225000 [331:05:52<74:22:36,  5.16s/it]
 77%|███████▋  | 173127/225000 [331:05:57<70:24:01,  4.89s/it]
 77%|███████▋  | 173128/225000 [331:06:01<70:07:47,  4.87s/it]
 77%|███████▋  | 173129/225000 [331:06:13<98:28:17,  6.83s/it]
 77%|███████▋  | 173130/225000 [331:06:16<82:25:02,  5.72s/it]
 77%|███████▋  | 173131/225000 [331:06:24<90:24:26,  6.27s/it]
 77%|███████▋  | 173132/225000 [331:06:28<81:32:30,  5.66s/it]
 77%|███████▋  | 173133/225000 [331:06:32<76:31:20,  5.31s/it]
 77%|███████▋  | 173134/225000 [331:06:36<71:29:38,  4.96s/it]
 77%|███████▋  | 173135/225000 [331:06:41<69:47:13,  4.84s/it]
 77%|███████▋  | 173136/225000 [331:06:45<68:11:04,  4.73s/it]
 77%|███████▋  | 173137/225000 [331:06:50<67:10:18,  4.66s/it]
 77%|███████▋  | 173138/225000 [331:06:59<84:59:02,  5.90s/it]
 77%|███████▋  | 173139/225000 [331:07:06<91:35:12,  6.36s/it]
 77%|███████▋  | 173140/225000 [331:07:10<81:56:18,  5.69s/it]
 77%|███████▋  | 173141/225000 [331:07:15<76:06:17,  5.28s/it]
 77%|███████▋  | 173142/225000 [331:07:19<71:15:39,  4.95s/it]
 77%|███████▋  | 173143/225000 [331:07:26<81:30:49,  5.66s/it]
 77%|███████▋  | 173144/225000 [331:07:35<94:15:15,  6.54s/it]
 77%|███████▋  | 173145/225000 [331:07:40<89:54:33,  6.24s/it]
 77%|███████▋  | 173146/225000 [331:07:44<78:32:24,  5.45s/it]
 77%|███████▋  | 173147/225000 [331:07:51<84:58:57,  5.90s/it]
 77%|███████▋  | 173148/225000 [331:07:55<75:55:23,  5.27s/it]
 77%|███████▋  | 173149/225000 [331:08:01<81:42:38,  5.67s/it]
 77%|███████▋  | 173150/225000 [331:08:05<72:32:06,  5.04s/it]
                                                              
{'loss': '0.7633', 'grad_norm': '1.11', 'learning_rate': '6.859e-06', 'epoch': '0.7696', 'num_input_tokens_seen': 548727680, 'train_runtime': '1.192e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173150/225000 [331:08:05<72:32:06,  5.04s/it]
 77%|███████▋  | 173151/225000 [331:08:12<80:24:14,  5.58s/it]
 77%|███████▋  | 173152/225000 [331:08:16<75:04:16,  5.21s/it]
 77%|███████▋  | 173153/225000 [331:08:20<71:34:59,  4.97s/it]
 77%|███████▋  | 173154/225000 [331:08:25<69:00:43,  4.79s/it]
 77%|███████▋  | 173155/225000 [331:08:29<68:40:42,  4.77s/it]
 77%|███████▋  | 173156/225000 [331:08:34<67:13:26,  4.67s/it]
 77%|███████▋  | 173157/225000 [331:08:46<97:53:08,  6.80s/it]
 77%|███████▋  | 173158/225000 [331:08:51<91:13:16,  6.33s/it]
 77%|███████▋  | 173159/225000 [331:08:56<87:46:46,  6.10s/it]
 77%|███████▋  | 173160/225000 [331:09:02<84:46:59,  5.89s/it]
 77%|███████▋  | 173161/225000 [331:09:07<82:14:59,  5.71s/it]
 77%|███████▋  | 173162/225000 [331:09:13<80:37:32,  5.60s/it]
 77%|███████▋  | 173163/225000 [331:09:17<76:49:17,  5.34s/it]
 77%|███████▋  | 173164/225000 [331:09:23<76:44:40,  5.33s/it]
 77%|███████▋  | 173165/225000 [331:09:28<75:47:36,  5.26s/it]
 77%|███████▋  | 173166/225000 [331:09:33<74:01:24,  5.14s/it]
 77%|███████▋  | 173167/225000 [331:09:38<75:58:24,  5.28s/it]
 77%|███████▋  | 173168/225000 [331:09:43<74:52:59,  5.20s/it]
 77%|███████▋  | 173169/225000 [331:09:48<74:12:51,  5.15s/it]
 77%|███████▋  | 173170/225000 [331:09:54<75:17:37,  5.23s/it]
 77%|███████▋  | 173171/225000 [331:09:59<74:43:41,  5.19s/it]
 77%|███████▋  | 173172/225000 [331:10:04<76:01:13,  5.28s/it]
 77%|███████▋  | 173173/225000 [331:10:09<75:57:18,  5.28s/it]
 77%|███████▋  | 173174/225000 [331:10:15<76:01:34,  5.28s/it]
 77%|███████▋  | 173175/225000 [331:10:20<76:44:38,  5.33s/it]
 77%|███████▋  | 173176/225000 [331:10:26<76:41:30,  5.33s/it]
 77%|███████▋  | 173177/225000 [331:10:31<77:08:12,  5.36s/it]
 77%|███████▋  | 173178/225000 [331:10:36<76:25:55,  5.31s/it]
 77%|███████▋  | 173179/225000 [331:10:42<77:31:00,  5.39s/it]
 77%|███████▋  | 173180/225000 [331:10:47<78:14:13,  5.44s/it]
 77%|███████▋  | 173181/225000 [331:10:52<75:02:05,  5.21s/it]
 77%|███████▋  | 173182/225000 [331:10:57<76:24:07,  5.31s/it]
 77%|███████▋  | 173183/225000 [331:11:03<75:23:44,  5.24s/it]
 77%|███████▋  | 173184/225000 [331:11:08<75:40:51,  5.26s/it]
 77%|███████▋  | 173185/225000 [331:11:13<76:38:52,  5.33s/it]
 77%|███████▋  | 173186/225000 [331:11:23<94:58:20,  6.60s/it]
 77%|███████▋  | 173187/225000 [331:11:28<88:27:38,  6.15s/it]
 77%|███████▋  | 173188/225000 [331:11:34<86:12:37,  5.99s/it]
 77%|███████▋  | 173189/225000 [331:11:39<82:51:34,  5.76s/it]
 77%|███████▋  | 173190/225000 [331:11:44<79:36:02,  5.53s/it]
 77%|███████▋  | 173191/225000 [331:11:49<76:53:59,  5.34s/it]
 77%|███████▋  | 173192/225000 [331:11:54<75:42:33,  5.26s/it]
 77%|███████▋  | 173193/225000 [331:11:59<75:19:19,  5.23s/it]
 77%|███████▋  | 173194/225000 [331:12:04<73:44:22,  5.12s/it]
 77%|███████▋  | 173195/225000 [331:12:09<74:39:22,  5.19s/it]
 77%|███████▋  | 173196/225000 [331:12:14<72:49:51,  5.06s/it]
 77%|███████▋  | 173197/225000 [331:12:19<74:05:54,  5.15s/it]
 77%|███████▋  | 173198/225000 [331:12:25<75:23:55,  5.24s/it]
 77%|███████▋  | 173199/225000 [331:12:30<75:14:18,  5.23s/it]
 77%|███████▋  | 173200/225000 [331:12:35<74:50:48,  5.20s/it]
                                                              
{'loss': '0.7577', 'grad_norm': '1.244', 'learning_rate': '6.846e-06', 'epoch': '0.7698', 'num_input_tokens_seen': 548855944, 'train_runtime': '1.192e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173200/225000 [331:12:35<74:50:48,  5.20s/it]
 77%|███████▋  | 173201/225000 [331:12:40<73:37:50,  5.12s/it]
 77%|███████▋  | 173202/225000 [331:12:46<76:13:53,  5.30s/it]
 77%|███████▋  | 173203/225000 [331:12:51<75:59:58,  5.28s/it]
 77%|███████▋  | 173204/225000 [331:12:57<77:27:23,  5.38s/it]
 77%|███████▋  | 173205/225000 [331:13:02<75:54:15,  5.28s/it]
 77%|███████▋  | 173206/225000 [331:13:07<76:44:05,  5.33s/it]
 77%|███████▋  | 173207/225000 [331:13:12<74:45:27,  5.20s/it]
 77%|███████▋  | 173208/225000 [331:13:17<76:04:45,  5.29s/it]
 77%|███████▋  | 173209/225000 [331:13:23<76:50:50,  5.34s/it]
 77%|███████▋  | 173210/225000 [331:13:35<105:59:53,  7.37s/it]
 77%|███████▋  | 173211/225000 [331:13:40<95:07:51,  6.61s/it] 
 77%|███████▋  | 173212/225000 [331:13:44<85:55:16,  5.97s/it]
 77%|███████▋  | 173213/225000 [331:13:49<79:59:03,  5.56s/it]
 77%|███████▋  | 173214/225000 [331:13:54<77:44:46,  5.40s/it]
 77%|███████▋  | 173215/225000 [331:13:59<76:44:17,  5.33s/it]
 77%|███████▋  | 173216/225000 [331:14:04<75:05:17,  5.22s/it]
 77%|███████▋  | 173217/225000 [331:14:09<73:07:51,  5.08s/it]
 77%|███████▋  | 173218/225000 [331:14:20<97:20:03,  6.77s/it]
 77%|███████▋  | 173219/225000 [331:14:26<94:44:48,  6.59s/it]
 77%|███████▋  | 173220/225000 [331:14:33<96:50:52,  6.73s/it]
 77%|███████▋  | 173221/225000 [331:14:36<81:43:09,  5.68s/it]
 77%|███████▋  | 173222/225000 [331:14:40<72:05:10,  5.01s/it]
 77%|███████▋  | 173223/225000 [331:14:44<68:49:07,  4.78s/it]
 77%|███████▋  | 173224/225000 [331:14:49<69:16:20,  4.82s/it]
 77%|███████▋  | 173225/225000 [331:14:54<69:49:33,  4.86s/it]
 77%|███████▋  | 173226/225000 [331:14:59<72:06:49,  5.01s/it]
 77%|███████▋  | 173227/225000 [331:15:04<73:31:57,  5.11s/it]
 77%|███████▋  | 173228/225000 [331:15:09<71:28:44,  4.97s/it]
 77%|███████▋  | 173229/225000 [331:15:14<70:19:10,  4.89s/it]
 77%|███████▋  | 173230/225000 [331:15:22<83:02:21,  5.77s/it]
 77%|███████▋  | 173231/225000 [331:15:32<104:53:59,  7.29s/it]
 77%|███████▋  | 173232/225000 [331:15:37<91:23:56,  6.36s/it] 
 77%|███████▋  | 173233/225000 [331:15:41<84:07:30,  5.85s/it]
 77%|███████▋  | 173234/225000 [331:15:46<79:17:37,  5.51s/it]
 77%|███████▋  | 173235/225000 [331:15:51<75:39:20,  5.26s/it]
 77%|███████▋  | 173236/225000 [331:15:55<70:49:13,  4.93s/it]
 77%|███████▋  | 173237/225000 [331:16:00<73:24:48,  5.11s/it]
 77%|███████▋  | 173238/225000 [331:16:11<95:35:00,  6.65s/it]
 77%|███████▋  | 173239/225000 [331:16:15<85:35:55,  5.95s/it]
 77%|███████▋  | 173240/225000 [331:16:19<77:40:23,  5.40s/it]
 77%|███████▋  | 173241/225000 [331:16:23<72:53:11,  5.07s/it]
 77%|███████▋  | 173242/225000 [331:16:27<69:14:25,  4.82s/it]
 77%|███████▋  | 173243/225000 [331:16:39<96:58:01,  6.74s/it]
 77%|███████▋  | 173244/225000 [331:16:43<85:23:27,  5.94s/it]
 77%|███████▋  | 173245/225000 [331:16:47<79:21:03,  5.52s/it]
 77%|███████▋  | 173246/225000 [331:16:52<76:22:20,  5.31s/it]
 77%|███████▋  | 173247/225000 [331:16:57<76:02:42,  5.29s/it]
 77%|███████▋  | 173248/225000 [331:17:03<75:46:20,  5.27s/it]
 77%|███████▋  | 173249/225000 [331:17:08<74:28:00,  5.18s/it]
 77%|███████▋  | 173250/225000 [331:17:12<72:39:12,  5.05s/it]
                                                              
{'loss': '0.7699', 'grad_norm': '1.219', 'learning_rate': '6.834e-06', 'epoch': '0.77', 'num_input_tokens_seen': 548987256, 'train_runtime': '1.193e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173250/225000 [331:17:12<72:39:12,  5.05s/it]
 77%|███████▋  | 173251/225000 [331:17:18<73:38:56,  5.12s/it]
 77%|███████▋  | 173252/225000 [331:17:23<74:42:07,  5.20s/it]
 77%|███████▋  | 173253/225000 [331:17:28<72:56:50,  5.07s/it]
 77%|███████▋  | 173254/225000 [331:17:33<73:01:52,  5.08s/it]
 77%|███████▋  | 173255/225000 [331:17:38<73:18:25,  5.10s/it]
 77%|███████▋  | 173256/225000 [331:17:44<75:41:38,  5.27s/it]
 77%|███████▋  | 173257/225000 [331:17:49<74:47:41,  5.20s/it]
 77%|███████▋  | 173258/225000 [331:17:59<96:02:10,  6.68s/it]
 77%|███████▋  | 173259/225000 [331:18:05<93:20:55,  6.49s/it]
 77%|███████▋  | 173260/225000 [331:18:12<95:50:14,  6.67s/it]
 77%|███████▋  | 173261/225000 [331:18:16<83:45:35,  5.83s/it]
 77%|███████▋  | 173262/225000 [331:18:19<73:28:25,  5.11s/it]
 77%|███████▋  | 173263/225000 [331:18:28<88:58:34,  6.19s/it]
 77%|███████▋  | 173264/225000 [331:18:33<82:59:42,  5.78s/it]
 77%|███████▋  | 173265/225000 [331:18:38<79:56:50,  5.56s/it]
 77%|███████▋  | 173266/225000 [331:18:43<76:00:02,  5.29s/it]
 77%|███████▋  | 173267/225000 [331:18:47<72:50:34,  5.07s/it]
 77%|███████▋  | 173268/225000 [331:18:52<71:41:54,  4.99s/it]
 77%|███████▋  | 173269/225000 [331:18:57<70:59:08,  4.94s/it]
 77%|███████▋  | 173270/225000 [331:19:01<70:01:03,  4.87s/it]
 77%|███████▋  | 173271/225000 [331:19:06<69:02:51,  4.81s/it]
 77%|███████▋  | 173272/225000 [331:19:11<69:42:22,  4.85s/it]
 77%|███████▋  | 173273/225000 [331:19:16<70:29:14,  4.91s/it]
 77%|███████▋  | 173274/225000 [331:19:21<71:01:43,  4.94s/it]
 77%|███████▋  | 173275/225000 [331:19:34<103:24:34,  7.20s/it]
 77%|███████▋  | 173276/225000 [331:19:37<88:34:02,  6.16s/it] 
 77%|███████▋  | 173277/225000 [331:19:42<80:27:30,  5.60s/it]
 77%|███████▋  | 173278/225000 [331:19:46<76:00:05,  5.29s/it]
 77%|███████▋  | 173279/225000 [331:19:51<73:03:31,  5.09s/it]
 77%|███████▋  | 173280/225000 [331:19:55<69:59:33,  4.87s/it]
 77%|███████▋  | 173281/225000 [331:20:07<98:32:53,  6.86s/it]
 77%|███████▋  | 173282/225000 [331:20:12<90:44:14,  6.32s/it]
 77%|███████▋  | 173283/225000 [331:20:16<80:56:49,  5.63s/it]
 77%|███████▋  | 173284/225000 [331:20:20<74:16:20,  5.17s/it]
 77%|███████▋  | 173285/225000 [331:20:30<97:04:18,  6.76s/it]
 77%|███████▋  | 173286/225000 [331:20:35<86:14:19,  6.00s/it]
 77%|███████▋  | 173287/225000 [331:20:39<78:39:25,  5.48s/it]
 77%|███████▋  | 173288/225000 [331:20:43<71:32:51,  4.98s/it]
 77%|███████▋  | 173289/225000 [331:20:47<67:28:49,  4.70s/it]
 77%|███████▋  | 173290/225000 [331:20:57<91:05:06,  6.34s/it]
 77%|███████▋  | 173291/225000 [331:21:01<83:21:15,  5.80s/it]
 77%|███████▋  | 173292/225000 [331:21:04<71:34:56,  4.98s/it]
 77%|███████▋  | 173293/225000 [331:21:13<87:34:32,  6.10s/it]
 77%|███████▋  | 173294/225000 [331:21:16<75:18:32,  5.24s/it]
 77%|███████▋  | 173295/225000 [331:21:24<87:07:19,  6.07s/it]
 77%|███████▋  | 173296/225000 [331:21:28<77:20:20,  5.38s/it]
 77%|███████▋  | 173297/225000 [331:21:34<81:09:14,  5.65s/it]
 77%|███████▋  | 173298/225000 [331:21:38<73:42:36,  5.13s/it]
 77%|███████▋  | 173299/225000 [331:21:42<67:29:45,  4.70s/it]
 77%|███████▋  | 173300/225000 [331:21:46<65:15:03,  4.54s/it]
                                                              
{'loss': '0.7555', 'grad_norm': '1.154', 'learning_rate': '6.821e-06', 'epoch': '0.7702', 'num_input_tokens_seen': 549116952, 'train_runtime': '1.193e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173300/225000 [331:21:46<65:15:03,  4.54s/it]
 77%|███████▋  | 173301/225000 [331:21:59<100:31:08,  7.00s/it]
 77%|███████▋  | 173302/225000 [331:22:02<84:57:41,  5.92s/it] 
 77%|███████▋  | 173303/225000 [331:22:06<74:43:41,  5.20s/it]
 77%|███████▋  | 173304/225000 [331:22:14<85:46:26,  5.97s/it]
 77%|███████▋  | 173305/225000 [331:22:19<83:44:17,  5.83s/it]
 77%|███████▋  | 173306/225000 [331:22:24<78:18:30,  5.45s/it]
 77%|███████▋  | 173307/225000 [331:22:28<72:01:14,  5.02s/it]
 77%|███████▋  | 173308/225000 [331:22:36<84:40:41,  5.90s/it]
 77%|███████▋  | 173309/225000 [331:22:39<74:45:58,  5.21s/it]
 77%|███████▋  | 173310/225000 [331:22:43<68:02:54,  4.74s/it]
 77%|███████▋  | 173311/225000 [331:22:50<80:09:08,  5.58s/it]
 77%|███████▋  | 173312/225000 [331:22:54<71:04:05,  4.95s/it]
 77%|███████▋  | 173313/225000 [331:23:02<86:26:38,  6.02s/it]
 77%|███████▋  | 173314/225000 [331:23:07<78:22:20,  5.46s/it]
 77%|███████▋  | 173315/225000 [331:23:12<80:12:21,  5.59s/it]
 77%|███████▋  | 173316/225000 [331:23:19<83:15:25,  5.80s/it]
 77%|███████▋  | 173317/225000 [331:23:24<79:28:03,  5.54s/it]
 77%|███████▋  | 173318/225000 [331:23:29<78:32:02,  5.47s/it]
 77%|███████▋  | 173319/225000 [331:23:35<78:50:57,  5.49s/it]
 77%|███████▋  | 173320/225000 [331:23:40<78:51:56,  5.49s/it]
 77%|███████▋  | 173321/225000 [331:23:46<79:16:27,  5.52s/it]
 77%|███████▋  | 173322/225000 [331:23:51<78:57:57,  5.50s/it]
 77%|███████▋  | 173323/225000 [331:23:56<75:56:52,  5.29s/it]
 77%|███████▋  | 173324/225000 [331:24:03<85:42:49,  5.97s/it]
 77%|███████▋  | 173325/225000 [331:24:07<76:19:02,  5.32s/it]
 77%|███████▋  | 173326/225000 [331:24:12<75:12:03,  5.24s/it]
 77%|███████▋  | 173327/225000 [331:24:20<87:45:35,  6.11s/it]
 77%|███████▋  | 173328/225000 [331:24:24<77:32:51,  5.40s/it]
 77%|███████▋  | 173329/225000 [331:24:29<74:05:13,  5.16s/it]
 77%|███████▋  | 173330/225000 [331:24:37<89:11:32,  6.21s/it]
 77%|███████▋  | 173331/225000 [331:24:41<77:21:19,  5.39s/it]
 77%|███████▋  | 173332/225000 [331:24:47<79:43:30,  5.55s/it]
 77%|███████▋  | 173333/225000 [331:24:54<84:30:31,  5.89s/it]
 77%|███████▋  | 173334/225000 [331:24:58<76:24:20,  5.32s/it]
 77%|███████▋  | 173335/225000 [331:25:04<82:51:11,  5.77s/it]
 77%|███████▋  | 173336/225000 [331:25:08<75:24:39,  5.25s/it]
 77%|███████▋  | 173337/225000 [331:25:12<70:05:57,  4.88s/it]
 77%|███████▋  | 173338/225000 [331:25:17<67:58:50,  4.74s/it]
 77%|███████▋  | 173339/225000 [331:25:23<75:16:33,  5.25s/it]
 77%|███████▋  | 173340/225000 [331:25:32<92:18:53,  6.43s/it]
 77%|███████▋  | 173341/225000 [331:25:36<80:38:25,  5.62s/it]
 77%|███████▋  | 173342/225000 [331:25:40<73:29:17,  5.12s/it]
 77%|███████▋  | 173343/225000 [331:25:44<69:58:54,  4.88s/it]
 77%|███████▋  | 173344/225000 [331:25:53<86:01:44,  6.00s/it]
 77%|███████▋  | 173345/225000 [331:26:00<88:22:50,  6.16s/it]
 77%|███████▋  | 173346/225000 [331:26:03<75:26:26,  5.26s/it]
 77%|███████▋  | 173347/225000 [331:26:12<91:00:59,  6.34s/it]
 77%|███████▋  | 173348/225000 [331:26:15<79:43:49,  5.56s/it]
 77%|███████▋  | 173349/225000 [331:26:23<86:50:00,  6.05s/it]
 77%|███████▋  | 173350/225000 [331:26:26<75:43:02,  5.28s/it]
                                                              
{'loss': '0.7583', 'grad_norm': '1.11', 'learning_rate': '6.809e-06', 'epoch': '0.7704', 'num_input_tokens_seen': 549247224, 'train_runtime': '1.193e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173350/225000 [331:26:26<75:43:02,  5.28s/it]
 77%|███████▋  | 173351/225000 [331:26:33<82:16:16,  5.73s/it]
 77%|███████▋  | 173352/225000 [331:26:38<80:01:27,  5.58s/it]
 77%|███████▋  | 173353/225000 [331:26:42<73:33:43,  5.13s/it]
 77%|███████▋  | 173354/225000 [331:26:47<71:26:08,  4.98s/it]
 77%|███████▋  | 173355/225000 [331:26:51<68:53:47,  4.80s/it]
 77%|███████▋  | 173356/225000 [331:26:57<73:14:29,  5.11s/it]
 77%|███████▋  | 173357/225000 [331:27:07<92:23:15,  6.44s/it]
 77%|███████▋  | 173358/225000 [331:27:10<80:59:57,  5.65s/it]
 77%|███████▋  | 173359/225000 [331:27:18<88:41:29,  6.18s/it]
 77%|███████▋  | 173360/225000 [331:27:23<83:20:32,  5.81s/it]
 77%|███████▋  | 173361/225000 [331:27:28<79:14:55,  5.52s/it]
 77%|███████▋  | 173362/225000 [331:27:31<71:09:24,  4.96s/it]
 77%|███████▋  | 173363/225000 [331:27:39<83:27:40,  5.82s/it]
 77%|███████▋  | 173364/225000 [331:27:44<80:13:30,  5.59s/it]
 77%|███████▋  | 173365/225000 [331:27:49<77:47:07,  5.42s/it]
 77%|███████▋  | 173366/225000 [331:27:54<75:57:56,  5.30s/it]
 77%|███████▋  | 173367/225000 [331:27:59<74:00:58,  5.16s/it]
 77%|███████▋  | 173368/225000 [331:28:04<73:39:46,  5.14s/it]
 77%|███████▋  | 173369/225000 [331:28:09<74:58:29,  5.23s/it]
 77%|███████▋  | 173370/225000 [331:28:15<74:28:23,  5.19s/it]
 77%|███████▋  | 173371/225000 [331:28:20<74:43:03,  5.21s/it]
 77%|███████▋  | 173372/225000 [331:28:25<73:31:08,  5.13s/it]
 77%|███████▋  | 173373/225000 [331:28:30<73:00:38,  5.09s/it]
 77%|███████▋  | 173374/225000 [331:28:35<73:57:56,  5.16s/it]
 77%|███████▋  | 173375/225000 [331:28:40<73:34:03,  5.13s/it]
 77%|███████▋  | 173376/225000 [331:28:45<72:33:34,  5.06s/it]
 77%|███████▋  | 173377/225000 [331:28:54<87:22:26,  6.09s/it]
 77%|███████▋  | 173378/225000 [331:29:03<101:04:29,  7.05s/it]
 77%|███████▋  | 173379/225000 [331:29:06<86:13:44,  6.01s/it] 
 77%|███████▋  | 173380/225000 [331:29:10<77:16:26,  5.39s/it]
 77%|███████▋  | 173381/225000 [331:29:18<85:33:26,  5.97s/it]
 77%|███████▋  | 173382/225000 [331:29:24<85:19:04,  5.95s/it]
 77%|███████▋  | 173383/225000 [331:29:27<74:44:35,  5.21s/it]
 77%|███████▋  | 173384/225000 [331:29:32<74:10:02,  5.17s/it]
 77%|███████▋  | 173385/225000 [331:29:40<87:29:24,  6.10s/it]
 77%|███████▋  | 173386/225000 [331:29:44<75:25:24,  5.26s/it]
 77%|███████▋  | 173387/225000 [331:29:53<93:56:48,  6.55s/it]
 77%|███████▋  | 173388/225000 [331:29:57<80:30:03,  5.62s/it]
 77%|███████▋  | 173389/225000 [331:30:05<91:11:48,  6.36s/it]
 77%|███████▋  | 173390/225000 [331:30:08<77:30:46,  5.41s/it]
 77%|███████▋  | 173391/225000 [331:30:11<68:43:40,  4.79s/it]
 77%|███████▋  | 173392/225000 [331:30:15<64:38:21,  4.51s/it]
 77%|███████▋  | 173393/225000 [331:30:26<93:37:36,  6.53s/it]
 77%|███████▋  | 173394/225000 [331:30:30<80:55:10,  5.64s/it]
 77%|███████▋  | 173395/225000 [331:30:34<72:39:36,  5.07s/it]
 77%|███████▋  | 173396/225000 [331:30:40<78:53:59,  5.50s/it]
 77%|███████▋  | 173397/225000 [331:30:48<88:10:35,  6.15s/it]
 77%|███████▋  | 173398/225000 [331:30:51<75:45:47,  5.29s/it]
 77%|███████▋  | 173399/225000 [331:30:59<84:40:37,  5.91s/it]
 77%|███████▋  | 173400/225000 [331:31:03<79:34:08,  5.55s/it]
                                                              
{'loss': '0.7746', 'grad_norm': '1.086', 'learning_rate': '6.796e-06', 'epoch': '0.7707', 'num_input_tokens_seen': 549376864, 'train_runtime': '1.193e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173400/225000 [331:31:03<79:34:08,  5.55s/it]
 77%|███████▋  | 173401/225000 [331:31:08<77:34:03,  5.41s/it]
 77%|███████▋  | 173402/225000 [331:31:13<74:15:44,  5.18s/it]
 77%|███████▋  | 173403/225000 [331:31:18<71:49:16,  5.01s/it]
 77%|███████▋  | 173404/225000 [331:31:23<71:19:39,  4.98s/it]
 77%|███████▋  | 173405/225000 [331:31:34<97:05:30,  6.77s/it]
 77%|███████▋  | 173406/225000 [331:31:37<81:59:50,  5.72s/it]
 77%|███████▋  | 173407/225000 [331:31:44<87:50:19,  6.13s/it]
 77%|███████▋  | 173408/225000 [331:31:49<82:24:40,  5.75s/it]
 77%|███████▋  | 173409/225000 [331:31:54<79:23:55,  5.54s/it]
 77%|███████▋  | 173410/225000 [331:31:59<77:20:01,  5.40s/it]
 77%|███████▋  | 173411/225000 [331:32:04<75:31:47,  5.27s/it]
 77%|███████▋  | 173412/225000 [331:32:08<72:23:24,  5.05s/it]
 77%|███████▋  | 173413/225000 [331:32:13<71:40:49,  5.00s/it]
 77%|███████▋  | 173414/225000 [331:32:18<71:25:47,  4.98s/it]
 77%|███████▋  | 173415/225000 [331:32:23<71:31:30,  4.99s/it]
 77%|███████▋  | 173416/225000 [331:32:28<69:12:36,  4.83s/it]
 77%|███████▋  | 173417/225000 [331:32:33<70:31:59,  4.92s/it]
 77%|███████▋  | 173418/225000 [331:32:38<71:20:52,  4.98s/it]
 77%|███████▋  | 173419/225000 [331:32:49<98:14:38,  6.86s/it]
 77%|███████▋  | 173420/225000 [331:32:55<94:41:40,  6.61s/it]
 77%|███████▋  | 173421/225000 [331:32:59<84:09:10,  5.87s/it]
 77%|███████▋  | 173422/225000 [331:33:04<78:22:53,  5.47s/it]
 77%|███████▋  | 173423/225000 [331:33:08<74:43:24,  5.22s/it]
 77%|███████▋  | 173424/225000 [331:33:13<71:02:05,  4.96s/it]
 77%|███████▋  | 173425/225000 [331:33:25<100:13:56,  7.00s/it]
 77%|███████▋  | 173426/225000 [331:33:29<88:56:24,  6.21s/it] 
 77%|███████▋  | 173427/225000 [331:33:33<79:02:10,  5.52s/it]
 77%|███████▋  | 173428/225000 [331:33:37<74:20:34,  5.19s/it]
 77%|███████▋  | 173429/225000 [331:33:41<67:50:35,  4.74s/it]
 77%|███████▋  | 173430/225000 [331:33:49<83:29:01,  5.83s/it]
 77%|███████▋  | 173431/225000 [331:33:58<93:36:28,  6.53s/it]
 77%|███████▋  | 173432/225000 [331:34:01<79:34:16,  5.55s/it]
 77%|███████▋  | 173433/225000 [331:34:08<88:43:54,  6.19s/it]
 77%|███████▋  | 173434/225000 [331:34:12<78:45:10,  5.50s/it]
 77%|███████▋  | 173435/225000 [331:34:16<71:53:18,  5.02s/it]
 77%|███████▋  | 173436/225000 [331:34:20<67:44:20,  4.73s/it]
 77%|███████▋  | 173437/225000 [331:34:31<95:07:26,  6.64s/it]
 77%|███████▋  | 173438/225000 [331:34:35<81:40:59,  5.70s/it]
 77%|███████▋  | 173439/225000 [331:34:39<72:58:18,  5.09s/it]
 77%|███████▋  | 173440/225000 [331:34:47<88:33:41,  6.18s/it]
 77%|███████▋  | 173441/225000 [331:34:51<77:39:47,  5.42s/it]
 77%|███████▋  | 173442/225000 [331:34:55<69:40:13,  4.86s/it]
 77%|███████▋  | 173443/225000 [331:35:04<89:58:33,  6.28s/it]
 77%|███████▋  | 173444/225000 [331:35:08<77:55:39,  5.44s/it]
 77%|███████▋  | 173445/225000 [331:35:11<69:40:53,  4.87s/it]
 77%|███████▋  | 173446/225000 [331:35:20<85:34:39,  5.98s/it]
 77%|███████▋  | 173447/225000 [331:35:23<74:38:28,  5.21s/it]
 77%|███████▋  | 173448/225000 [331:35:30<81:17:48,  5.68s/it]
 77%|███████▋  | 173449/225000 [331:35:34<74:59:12,  5.24s/it]
 77%|███████▋  | 173450/225000 [331:35:43<88:48:05,  6.20s/it]
                                                              
{'loss': '0.767', 'grad_norm': '1.223', 'learning_rate': '6.784e-06', 'epoch': '0.7709', 'num_input_tokens_seen': 549507368, 'train_runtime': '1.194e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173450/225000 [331:35:43<88:48:05,  6.20s/it]
 77%|███████▋  | 173451/225000 [331:35:46<78:41:13,  5.50s/it]
 77%|███████▋  | 173452/225000 [331:35:51<72:49:08,  5.09s/it]
 77%|███████▋  | 173453/225000 [331:35:55<68:35:14,  4.79s/it]
 77%|███████▋  | 173454/225000 [331:35:59<65:20:50,  4.56s/it]
 77%|███████▋  | 173455/225000 [331:36:04<70:08:24,  4.90s/it]
 77%|███████▋  | 173456/225000 [331:36:13<88:01:03,  6.15s/it]
 77%|███████▋  | 173457/225000 [331:36:17<76:27:27,  5.34s/it]
 77%|███████▋  | 173458/225000 [331:36:24<82:51:17,  5.79s/it]
 77%|███████▋  | 173459/225000 [331:36:29<81:35:35,  5.70s/it]
 77%|███████▋  | 173460/225000 [331:36:36<86:55:58,  6.07s/it]
 77%|███████▋  | 173461/225000 [331:36:39<74:04:27,  5.17s/it]
 77%|███████▋  | 173462/225000 [331:36:43<67:47:25,  4.74s/it]
 77%|███████▋  | 173463/225000 [331:36:53<90:47:23,  6.34s/it]
 77%|███████▋  | 173464/225000 [331:36:56<78:18:52,  5.47s/it]
 77%|███████▋  | 173465/225000 [331:37:01<73:04:43,  5.10s/it]
 77%|███████▋  | 173466/225000 [331:37:08<80:29:30,  5.62s/it]
 77%|███████▋  | 173467/225000 [331:37:11<72:25:31,  5.06s/it]
 77%|███████▋  | 173468/225000 [331:37:15<67:20:57,  4.70s/it]
 77%|███████▋  | 173469/225000 [331:37:25<89:46:36,  6.27s/it]
 77%|███████▋  | 173470/225000 [331:37:29<79:22:22,  5.55s/it]
 77%|███████▋  | 173471/225000 [331:37:33<72:35:58,  5.07s/it]
 77%|███████▋  | 173472/225000 [331:37:39<78:36:15,  5.49s/it]
 77%|███████▋  | 173473/225000 [331:37:47<87:39:09,  6.12s/it]
 77%|███████▋  | 173474/225000 [331:37:53<88:09:14,  6.16s/it]
 77%|███████▋  | 173475/225000 [331:37:59<84:42:44,  5.92s/it]
 77%|███████▋  | 173476/225000 [331:38:03<77:10:00,  5.39s/it]
 77%|███████▋  | 173477/225000 [331:38:12<94:41:26,  6.62s/it]
 77%|███████▋  | 173478/225000 [331:38:16<81:04:29,  5.66s/it]
 77%|███████▋  | 173479/225000 [331:38:20<77:13:51,  5.40s/it]
 77%|███████▋  | 173480/225000 [331:38:28<88:32:34,  6.19s/it]
 77%|███████▋  | 173481/225000 [331:38:33<79:34:35,  5.56s/it]
 77%|███████▋  | 173482/225000 [331:38:37<75:43:44,  5.29s/it]
 77%|███████▋  | 173483/225000 [331:38:42<71:53:58,  5.02s/it]
 77%|███████▋  | 173484/225000 [331:38:46<70:35:29,  4.93s/it]
 77%|███████▋  | 173485/225000 [331:38:50<67:08:30,  4.69s/it]
 77%|███████▋  | 173486/225000 [331:38:55<65:29:34,  4.58s/it]
 77%|███████▋  | 173487/225000 [331:39:03<80:55:28,  5.66s/it]
 77%|███████▋  | 173488/225000 [331:39:11<89:16:23,  6.24s/it]
 77%|███████▋  | 173489/225000 [331:39:15<82:55:57,  5.80s/it]
 77%|███████▋  | 173490/225000 [331:39:20<77:29:00,  5.42s/it]
 77%|███████▋  | 173491/225000 [331:39:24<74:17:04,  5.19s/it]
 77%|███████▋  | 173492/225000 [331:39:29<71:28:26,  5.00s/it]
 77%|███████▋  | 173493/225000 [331:39:34<69:57:45,  4.89s/it]
 77%|███████▋  | 173494/225000 [331:39:46<101:45:33,  7.11s/it]
 77%|███████▋  | 173495/225000 [331:39:50<88:39:08,  6.20s/it] 
 77%|███████▋  | 173496/225000 [331:39:58<95:04:56,  6.65s/it]
 77%|███████▋  | 173497/225000 [331:40:02<84:44:43,  5.92s/it]
 77%|███████▋  | 173498/225000 [331:40:09<89:24:44,  6.25s/it]
 77%|███████▋  | 173499/225000 [331:40:13<78:23:43,  5.48s/it]
 77%|███████▋  | 173500/225000 [331:40:18<79:50:02,  5.58s/it]
                                                              
{'loss': '0.7639', 'grad_norm': '1.004', 'learning_rate': '6.771e-06', 'epoch': '0.7711', 'num_input_tokens_seen': 549638176, 'train_runtime': '1.194e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173500/225000 [331:40:18<79:50:02,  5.58s/it]
 77%|███████▋  | 173501/225000 [331:40:26<88:21:27,  6.18s/it]
 77%|███████▋  | 173502/225000 [331:40:30<79:38:34,  5.57s/it]
 77%|███████▋  | 173503/225000 [331:40:35<75:15:20,  5.26s/it]
 77%|███████▋  | 173504/225000 [331:40:39<71:30:10,  5.00s/it]
 77%|███████▋  | 173505/225000 [331:40:44<69:29:46,  4.86s/it]
 77%|███████▋  | 173506/225000 [331:40:48<68:30:25,  4.79s/it]
 77%|███████▋  | 173507/225000 [331:40:57<85:44:14,  5.99s/it]
 77%|███████▋  | 173508/225000 [331:41:02<80:55:57,  5.66s/it]
 77%|███████▋  | 173509/225000 [331:41:07<79:07:03,  5.53s/it]
 77%|███████▋  | 173510/225000 [331:41:12<74:22:48,  5.20s/it]
 77%|███████▋  | 173511/225000 [331:41:17<75:33:15,  5.28s/it]
 77%|███████▋  | 173512/225000 [331:41:24<83:14:13,  5.82s/it]
 77%|███████▋  | 173513/225000 [331:41:32<90:07:15,  6.30s/it]
 77%|███████▋  | 173514/225000 [331:41:37<85:50:59,  6.00s/it]
 77%|███████▋  | 173515/225000 [331:41:42<80:42:47,  5.64s/it]
 77%|███████▋  | 173516/225000 [331:41:47<77:46:17,  5.44s/it]
 77%|███████▋  | 173517/225000 [331:41:52<76:09:54,  5.33s/it]
 77%|███████▋  | 173518/225000 [331:41:57<75:30:18,  5.28s/it]
 77%|███████▋  | 173519/225000 [331:42:02<73:55:41,  5.17s/it]
 77%|███████▋  | 173520/225000 [331:42:07<75:14:49,  5.26s/it]
 77%|███████▋  | 173521/225000 [331:42:13<75:39:11,  5.29s/it]
 77%|███████▋  | 173522/225000 [331:42:17<72:39:54,  5.08s/it]
 77%|███████▋  | 173523/225000 [331:42:22<72:37:37,  5.08s/it]
 77%|███████▋  | 173524/225000 [331:42:28<73:55:58,  5.17s/it]
 77%|███████▋  | 173525/225000 [331:42:33<72:50:07,  5.09s/it]
 77%|███████▋  | 173526/225000 [331:42:39<77:18:56,  5.41s/it]
 77%|███████▋  | 173527/225000 [331:42:52<111:02:45,  7.77s/it]
 77%|███████▋  | 173528/225000 [331:42:55<92:36:45,  6.48s/it] 
 77%|███████▋  | 173529/225000 [331:42:59<79:38:05,  5.57s/it]
 77%|███████▋  | 173530/225000 [331:43:03<74:00:52,  5.18s/it]
 77%|███████▋  | 173531/225000 [331:43:08<70:57:14,  4.96s/it]
 77%|███████▋  | 173532/225000 [331:43:18<95:17:35,  6.67s/it]
 77%|███████▋  | 173533/225000 [331:43:24<89:29:59,  6.26s/it]
 77%|███████▋  | 173534/225000 [331:43:27<78:30:31,  5.49s/it]
 77%|███████▋  | 173535/225000 [331:43:32<76:51:45,  5.38s/it]
 77%|███████▋  | 173536/225000 [331:43:38<75:49:55,  5.30s/it]
 77%|███████▋  | 173537/225000 [331:43:42<73:45:09,  5.16s/it]
 77%|███████▋  | 173538/225000 [331:43:47<72:12:49,  5.05s/it]
 77%|███████▋  | 173539/225000 [331:43:52<69:59:11,  4.90s/it]
 77%|███████▋  | 173540/225000 [331:43:57<70:37:26,  4.94s/it]
 77%|███████▋  | 173541/225000 [331:44:02<70:44:59,  4.95s/it]
 77%|███████▋  | 173542/225000 [331:44:06<67:57:45,  4.75s/it]
 77%|███████▋  | 173543/225000 [331:44:11<67:39:03,  4.73s/it]
 77%|███████▋  | 173544/225000 [331:44:20<89:07:42,  6.24s/it]
 77%|███████▋  | 173545/225000 [331:44:29<99:25:20,  6.96s/it]
 77%|███████▋  | 173546/225000 [331:44:33<86:11:00,  6.03s/it]
 77%|███████▋  | 173547/225000 [331:44:40<90:54:15,  6.36s/it]
 77%|███████▋  | 173548/225000 [331:44:44<78:30:49,  5.49s/it]
 77%|███████▋  | 173549/225000 [331:44:51<85:59:17,  6.02s/it]
 77%|███████▋  | 173550/225000 [331:44:54<74:00:03,  5.18s/it]
                                                              
{'loss': '0.7611', 'grad_norm': '1.19', 'learning_rate': '6.759e-06', 'epoch': '0.7713', 'num_input_tokens_seen': 549769080, 'train_runtime': '1.194e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173550/225000 [331:44:54<74:00:03,  5.18s/it]
 77%|███████▋  | 173551/225000 [331:45:02<88:08:24,  6.17s/it]
 77%|███████▋  | 173552/225000 [331:45:06<75:50:39,  5.31s/it]
 77%|███████▋  | 173553/225000 [331:45:13<85:34:32,  5.99s/it]
 77%|███████▋  | 173554/225000 [331:45:17<76:04:28,  5.32s/it]
 77%|███████▋  | 173555/225000 [331:45:22<73:51:17,  5.17s/it]
 77%|███████▋  | 173556/225000 [331:45:27<72:52:43,  5.10s/it]
 77%|███████▋  | 173557/225000 [331:45:32<73:14:45,  5.13s/it]
 77%|███████▋  | 173558/225000 [331:45:37<72:28:58,  5.07s/it]
 77%|███████▋  | 173559/225000 [331:45:42<70:27:10,  4.93s/it]
 77%|███████▋  | 173560/225000 [331:45:46<69:48:24,  4.89s/it]
 77%|███████▋  | 173561/225000 [331:45:51<69:17:22,  4.85s/it]
 77%|███████▋  | 173562/225000 [331:46:03<97:25:45,  6.82s/it]
 77%|███████▋  | 173563/225000 [331:46:07<85:44:38,  6.00s/it]
 77%|███████▋  | 173564/225000 [331:46:11<79:43:51,  5.58s/it]
 77%|███████▋  | 173565/225000 [331:46:16<75:39:26,  5.30s/it]
 77%|███████▋  | 173566/225000 [331:46:20<72:23:41,  5.07s/it]
 77%|███████▋  | 173567/225000 [331:46:25<71:00:32,  4.97s/it]
 77%|███████▋  | 173568/225000 [331:46:31<76:43:16,  5.37s/it]
 77%|███████▋  | 173569/225000 [331:46:41<94:54:54,  6.64s/it]
 77%|███████▋  | 173570/225000 [331:46:45<82:42:16,  5.79s/it]
 77%|███████▋  | 173571/225000 [331:46:49<74:48:22,  5.24s/it]
 77%|███████▋  | 173572/225000 [331:46:53<69:54:21,  4.89s/it]
 77%|███████▋  | 173573/225000 [331:47:02<89:09:17,  6.24s/it]
 77%|███████▋  | 173574/225000 [331:47:06<77:31:11,  5.43s/it]
 77%|███████▋  | 173575/225000 [331:47:15<91:34:54,  6.41s/it]
 77%|███████▋  | 173576/225000 [331:47:18<77:35:53,  5.43s/it]
 77%|███████▋  | 173577/225000 [331:47:25<87:25:25,  6.12s/it]
 77%|███████▋  | 173578/225000 [331:47:29<75:52:10,  5.31s/it]
 77%|███████▋  | 173579/225000 [331:47:37<88:39:44,  6.21s/it]
 77%|███████▋  | 173580/225000 [331:47:41<76:49:15,  5.38s/it]
 77%|███████▋  | 173581/225000 [331:47:46<76:33:42,  5.36s/it]
 77%|███████▋  | 173582/225000 [331:47:52<81:17:15,  5.69s/it]
 77%|███████▋  | 173583/225000 [331:47:57<74:46:01,  5.23s/it]
 77%|███████▋  | 173584/225000 [331:48:01<69:17:20,  4.85s/it]
 77%|███████▋  | 173585/225000 [331:48:05<67:01:18,  4.69s/it]
 77%|███████▋  | 173586/225000 [331:48:09<65:27:44,  4.58s/it]
 77%|███████▋  | 173587/225000 [331:48:20<91:16:04,  6.39s/it]
 77%|███████▋  | 173588/225000 [331:48:24<81:11:56,  5.69s/it]
 77%|███████▋  | 173589/225000 [331:48:28<73:31:31,  5.15s/it]
 77%|███████▋  | 173590/225000 [331:48:32<69:08:21,  4.84s/it]
 77%|███████▋  | 173591/225000 [331:48:41<86:13:30,  6.04s/it]
 77%|███████▋  | 173592/225000 [331:48:46<82:52:01,  5.80s/it]
 77%|███████▋  | 173593/225000 [331:48:49<71:16:59,  4.99s/it]
 77%|███████▋  | 173594/225000 [331:48:58<88:52:06,  6.22s/it]
 77%|███████▋  | 173595/225000 [331:49:02<77:14:32,  5.41s/it]
 77%|███████▋  | 173596/225000 [331:49:06<71:51:00,  5.03s/it]
 77%|███████▋  | 173597/225000 [331:49:14<86:16:55,  6.04s/it]
 77%|███████▋  | 173598/225000 [331:49:19<82:53:43,  5.81s/it]
 77%|███████▋  | 173599/225000 [331:49:24<79:42:23,  5.58s/it]
 77%|███████▋  | 173600/225000 [331:49:30<79:37:47,  5.58s/it]
                                                              
{'loss': '0.7683', 'grad_norm': '1.05', 'learning_rate': '6.746e-06', 'epoch': '0.7716', 'num_input_tokens_seen': 549899760, 'train_runtime': '1.195e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173600/225000 [331:49:30<79:37:47,  5.58s/it]
 77%|███████▋  | 173601/225000 [331:49:36<79:47:47,  5.59s/it]
 77%|███████▋  | 173602/225000 [331:49:43<86:07:00,  6.03s/it]
 77%|███████▋  | 173603/225000 [331:49:46<73:51:47,  5.17s/it]
 77%|███████▋  | 173604/225000 [331:49:50<70:17:35,  4.92s/it]
 77%|███████▋  | 173605/225000 [331:49:55<70:39:07,  4.95s/it]
 77%|███████▋  | 173606/225000 [331:50:00<68:32:58,  4.80s/it]
 77%|███████▋  | 173607/225000 [331:50:04<66:54:19,  4.69s/it]
 77%|███████▋  | 173608/225000 [331:50:15<93:52:02,  6.58s/it]
 77%|███████▋  | 173609/225000 [331:50:23<98:15:18,  6.88s/it]
 77%|███████▋  | 173610/225000 [331:50:27<85:38:52,  6.00s/it]
 77%|███████▋  | 173611/225000 [331:50:31<78:06:02,  5.47s/it]
 77%|███████▋  | 173612/225000 [331:50:35<72:05:07,  5.05s/it]
 77%|███████▋  | 173613/225000 [331:50:43<86:06:03,  6.03s/it]
 77%|███████▋  | 173614/225000 [331:50:49<84:16:18,  5.90s/it]
 77%|███████▋  | 173615/225000 [331:50:55<84:08:26,  5.89s/it]
 77%|███████▋  | 173616/225000 [331:50:58<74:30:44,  5.22s/it]
 77%|███████▋  | 173617/225000 [331:51:05<78:49:09,  5.52s/it]
 77%|███████▋  | 173618/225000 [331:51:12<84:43:10,  5.94s/it]
 77%|███████▋  | 173619/225000 [331:51:16<79:27:40,  5.57s/it]
 77%|███████▋  | 173620/225000 [331:51:22<78:32:48,  5.50s/it]
 77%|███████▋  | 173621/225000 [331:51:29<84:36:58,  5.93s/it]
 77%|███████▋  | 173622/225000 [331:51:33<76:18:27,  5.35s/it]
 77%|███████▋  | 173623/225000 [331:51:36<69:44:16,  4.89s/it]
 77%|███████▋  | 173624/225000 [331:51:46<91:56:02,  6.44s/it]
 77%|███████▋  | 173625/225000 [331:51:50<81:52:09,  5.74s/it]
 77%|███████▋  | 173626/225000 [331:51:55<78:01:42,  5.47s/it]
 77%|███████▋  | 173627/225000 [331:52:00<76:43:18,  5.38s/it]
 77%|███████▋  | 173628/225000 [331:52:05<74:04:31,  5.19s/it]
 77%|███████▋  | 173629/225000 [331:52:10<72:59:23,  5.12s/it]
 77%|███████▋  | 173630/225000 [331:52:15<70:45:11,  4.96s/it]
 77%|███████▋  | 173631/225000 [331:52:20<71:15:00,  4.99s/it]
 77%|███████▋  | 173632/225000 [331:52:26<74:10:42,  5.20s/it]
 77%|███████▋  | 173633/225000 [331:52:31<73:36:29,  5.16s/it]
 77%|███████▋  | 173634/225000 [331:52:35<72:26:39,  5.08s/it]
 77%|███████▋  | 173635/225000 [331:52:40<71:50:10,  5.03s/it]
 77%|███████▋  | 173636/225000 [331:52:45<70:50:19,  4.96s/it]
 77%|███████▋  | 173637/225000 [331:52:54<88:54:49,  6.23s/it]
 77%|███████▋  | 173638/225000 [331:53:03<101:00:36,  7.08s/it]
 77%|███████▋  | 173639/225000 [331:53:07<84:22:52,  5.91s/it] 
 77%|███████▋  | 173640/225000 [331:53:14<90:44:01,  6.36s/it]
 77%|███████▋  | 173641/225000 [331:53:18<79:33:42,  5.58s/it]
 77%|███████▋  | 173642/225000 [331:53:26<89:51:44,  6.30s/it]
 77%|███████▋  | 173643/225000 [331:53:32<89:14:09,  6.26s/it]
 77%|███████▋  | 173644/225000 [331:53:35<77:01:22,  5.40s/it]
 77%|███████▋  | 173645/225000 [331:53:43<88:02:36,  6.17s/it]
 77%|███████▋  | 173646/225000 [331:53:47<76:20:39,  5.35s/it]
 77%|███████▋  | 173647/225000 [331:53:51<72:53:04,  5.11s/it]
 77%|███████▋  | 173648/225000 [331:53:56<70:55:43,  4.97s/it]
 77%|███████▋  | 173649/225000 [331:54:01<70:28:57,  4.94s/it]
 77%|███████▋  | 173650/225000 [331:54:12<98:49:10,  6.93s/it]
                                                              
{'loss': '0.7719', 'grad_norm': '1.102', 'learning_rate': '6.734e-06', 'epoch': '0.7718', 'num_input_tokens_seen': 550031240, 'train_runtime': '1.195e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173650/225000 [331:54:12<98:49:10,  6.93s/it]
 77%|███████▋  | 173651/225000 [331:54:16<83:45:03,  5.87s/it]
 77%|███████▋  | 173652/225000 [331:54:19<73:05:25,  5.12s/it]
 77%|███████▋  | 173653/225000 [331:54:26<80:55:16,  5.67s/it]
 77%|███████▋  | 173654/225000 [331:54:31<76:09:21,  5.34s/it]
 77%|███████▋  | 173655/225000 [331:54:35<70:05:41,  4.91s/it]
 77%|███████▋  | 173656/225000 [331:54:42<82:15:42,  5.77s/it]
 77%|███████▋  | 173657/225000 [331:54:49<85:46:09,  6.01s/it]
 77%|███████▋  | 173658/225000 [331:54:53<79:28:10,  5.57s/it]
 77%|███████▋  | 173659/225000 [331:54:58<76:46:33,  5.38s/it]
 77%|███████▋  | 173660/225000 [331:55:03<73:43:04,  5.17s/it]
 77%|███████▋  | 173661/225000 [331:55:08<71:15:38,  5.00s/it]
 77%|███████▋  | 173662/225000 [331:55:13<73:18:29,  5.14s/it]
 77%|███████▋  | 173663/225000 [331:55:18<72:39:11,  5.09s/it]
 77%|███████▋  | 173664/225000 [331:55:23<72:13:29,  5.06s/it]
 77%|███████▋  | 173665/225000 [331:55:28<71:18:07,  5.00s/it]
 77%|███████▋  | 173666/225000 [331:55:41<104:40:41,  7.34s/it]
 77%|███████▋  | 173667/225000 [331:55:46<94:59:09,  6.66s/it] 
 77%|███████▋  | 173668/225000 [331:55:50<82:52:39,  5.81s/it]
 77%|███████▋  | 173669/225000 [331:55:54<75:37:31,  5.30s/it]
 77%|███████▋  | 173670/225000 [331:55:58<72:38:47,  5.10s/it]
 77%|███████▋  | 173671/225000 [331:56:03<69:56:47,  4.91s/it]
 77%|███████▋  | 173672/225000 [331:56:08<70:32:26,  4.95s/it]
 77%|███████▋  | 173673/225000 [331:56:20<102:14:45,  7.17s/it]
 77%|███████▋  | 173674/225000 [331:56:24<88:10:06,  6.18s/it] 
 77%|███████▋  | 173675/225000 [331:56:28<77:40:47,  5.45s/it]
 77%|███████▋  | 173676/225000 [331:56:38<99:15:36,  6.96s/it]
 77%|███████▋  | 173677/225000 [331:56:42<84:18:37,  5.91s/it]
 77%|███████▋  | 173678/225000 [331:56:45<74:08:03,  5.20s/it]
 77%|███████▋  | 173679/225000 [331:56:50<70:43:26,  4.96s/it]
 77%|███████▋  | 173680/225000 [331:56:59<87:24:09,  6.13s/it]
 77%|███████▋  | 173681/225000 [331:57:03<79:09:30,  5.55s/it]
 77%|███████▋  | 173682/225000 [331:57:07<73:48:15,  5.18s/it]
 77%|███████▋  | 173683/225000 [331:57:12<70:57:04,  4.98s/it]
 77%|███████▋  | 173684/225000 [331:57:22<91:39:28,  6.43s/it]
 77%|███████▋  | 173685/225000 [331:57:26<83:40:36,  5.87s/it]
 77%|███████▋  | 173686/225000 [331:57:30<77:19:06,  5.42s/it]
 77%|███████▋  | 173687/225000 [331:57:35<72:53:28,  5.11s/it]
 77%|███████▋  | 173688/225000 [331:57:40<71:01:46,  4.98s/it]
 77%|███████▋  | 173689/225000 [331:57:46<78:52:40,  5.53s/it]
 77%|███████▋  | 173690/225000 [331:57:55<92:47:13,  6.51s/it]
 77%|███████▋  | 173691/225000 [331:57:59<79:25:35,  5.57s/it]
 77%|███████▋  | 173692/225000 [331:58:07<93:22:16,  6.55s/it]
 77%|███████▋  | 173693/225000 [331:58:12<83:19:34,  5.85s/it]
 77%|███████▋  | 173694/225000 [331:58:17<80:00:31,  5.61s/it]
 77%|███████▋  | 173695/225000 [331:58:22<77:47:28,  5.46s/it]
 77%|███████▋  | 173696/225000 [331:58:27<77:46:08,  5.46s/it]
 77%|███████▋  | 173697/225000 [331:58:33<77:22:00,  5.43s/it]
 77%|███████▋  | 173698/225000 [331:58:38<76:58:25,  5.40s/it]
 77%|███████▋  | 173699/225000 [331:58:43<77:28:03,  5.44s/it]
 77%|███████▋  | 173700/225000 [331:58:48<75:43:24,  5.31s/it]
                                                              
{'loss': '0.7705', 'grad_norm': '1.104', 'learning_rate': '6.721e-06', 'epoch': '0.772', 'num_input_tokens_seen': 550162608, 'train_runtime': '1.195e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173700/225000 [331:58:48<75:43:24,  5.31s/it]
 77%|███████▋  | 173701/225000 [331:58:54<75:43:20,  5.31s/it]
 77%|███████▋  | 173702/225000 [331:58:59<75:44:51,  5.32s/it]
 77%|███████▋  | 173703/225000 [331:59:04<75:16:43,  5.28s/it]
 77%|███████▋  | 173704/225000 [331:59:09<74:22:39,  5.22s/it]
 77%|███████▋  | 173705/225000 [331:59:14<73:58:21,  5.19s/it]
 77%|███████▋  | 173706/225000 [331:59:20<73:43:59,  5.17s/it]
 77%|███████▋  | 173707/225000 [331:59:25<75:14:59,  5.28s/it]
 77%|███████▋  | 173708/225000 [331:59:31<76:37:11,  5.38s/it]
 77%|███████▋  | 173709/225000 [331:59:36<76:46:45,  5.39s/it]
 77%|███████▋  | 173710/225000 [331:59:41<75:39:32,  5.31s/it]
 77%|███████▋  | 173711/225000 [331:59:53<101:45:42,  7.14s/it]
 77%|███████▋  | 173712/225000 [331:59:57<89:16:20,  6.27s/it] 
 77%|███████▋  | 173713/225000 [332:00:01<79:26:16,  5.58s/it]
 77%|███████▋  | 173714/225000 [332:00:07<82:19:36,  5.78s/it]
 77%|███████▋  | 173715/225000 [332:00:16<95:33:45,  6.71s/it]
 77%|███████▋  | 173716/225000 [332:00:20<82:23:04,  5.78s/it]
 77%|███████▋  | 173717/225000 [332:00:27<91:11:06,  6.40s/it]
 77%|███████▋  | 173718/225000 [332:00:31<79:55:48,  5.61s/it]
 77%|███████▋  | 173719/225000 [332:00:39<87:13:11,  6.12s/it]
 77%|███████▋  | 173720/225000 [332:00:42<77:44:38,  5.46s/it]
 77%|███████▋  | 173721/225000 [332:00:47<71:55:31,  5.05s/it]
 77%|███████▋  | 173722/225000 [332:00:51<68:27:27,  4.81s/it]
 77%|███████▋  | 173723/225000 [332:01:00<89:17:28,  6.27s/it]
 77%|███████▋  | 173724/225000 [332:01:04<77:19:01,  5.43s/it]
 77%|███████▋  | 173725/225000 [332:01:12<90:12:57,  6.33s/it]
 77%|███████▋  | 173726/225000 [332:01:16<79:28:51,  5.58s/it]
 77%|███████▋  | 173727/225000 [332:01:20<73:29:21,  5.16s/it]
 77%|███████▋  | 173728/225000 [332:01:25<68:55:02,  4.84s/it]
 77%|███████▋  | 173729/225000 [332:01:34<87:55:34,  6.17s/it]
 77%|███████▋  | 173730/225000 [332:01:39<84:30:37,  5.93s/it]
 77%|███████▋  | 173731/225000 [332:01:43<73:57:56,  5.19s/it]
 77%|███████▋  | 173732/225000 [332:01:50<81:22:24,  5.71s/it]
 77%|███████▋  | 173733/225000 [332:01:56<84:24:12,  5.93s/it]
 77%|███████▋  | 173734/225000 [332:02:00<74:54:39,  5.26s/it]
 77%|███████▋  | 173735/225000 [332:02:04<69:37:22,  4.89s/it]
 77%|███████▋  | 173736/225000 [332:02:14<94:12:00,  6.62s/it]
 77%|███████▋  | 173737/225000 [332:02:18<80:52:47,  5.68s/it]
 77%|███████▋  | 173738/225000 [332:02:25<86:32:36,  6.08s/it]
 77%|███████▋  | 173739/225000 [332:02:29<77:05:31,  5.41s/it]
 77%|███████▋  | 173740/225000 [332:02:33<73:05:44,  5.13s/it]
 77%|███████▋  | 173741/225000 [332:02:42<88:07:19,  6.19s/it]
 77%|███████▋  | 173742/225000 [332:02:46<77:17:44,  5.43s/it]
 77%|███████▋  | 173743/225000 [332:02:49<69:31:50,  4.88s/it]
 77%|███████▋  | 173744/225000 [332:02:57<83:10:05,  5.84s/it]
 77%|███████▋  | 173745/225000 [332:03:06<94:54:46,  6.67s/it]
 77%|███████▋  | 173746/225000 [332:03:09<81:02:08,  5.69s/it]
 77%|███████▋  | 173747/225000 [332:03:15<81:35:14,  5.73s/it]
 77%|███████▋  | 173748/225000 [332:03:19<74:25:40,  5.23s/it]
 77%|███████▋  | 173749/225000 [332:03:23<69:13:52,  4.86s/it]
 77%|███████▋  | 173750/225000 [332:03:32<87:45:02,  6.16s/it]
                                                              
{'loss': '0.7755', 'grad_norm': '1.082', 'learning_rate': '6.709e-06', 'epoch': '0.7722', 'num_input_tokens_seen': 550294056, 'train_runtime': '1.195e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173750/225000 [332:03:32<87:45:02,  6.16s/it]
 77%|███████▋  | 173751/225000 [332:03:36<78:56:25,  5.55s/it]
 77%|███████▋  | 173752/225000 [332:03:40<72:03:11,  5.06s/it]
 77%|███████▋  | 173753/225000 [332:03:46<74:32:10,  5.24s/it]
 77%|███████▋  | 173754/225000 [332:03:56<95:23:36,  6.70s/it]
 77%|███████▋  | 173755/225000 [332:03:59<81:14:27,  5.71s/it]
 77%|███████▋  | 173756/225000 [332:04:03<71:41:02,  5.04s/it]
 77%|███████▋  | 173757/225000 [332:04:08<70:58:39,  4.99s/it]
 77%|███████▋  | 173758/225000 [332:04:13<70:50:30,  4.98s/it]
 77%|███████▋  | 173759/225000 [332:04:17<69:41:12,  4.90s/it]
 77%|███████▋  | 173760/225000 [332:04:23<70:37:55,  4.96s/it]
 77%|███████▋  | 173761/225000 [332:04:28<71:33:57,  5.03s/it]
 77%|███████▋  | 173762/225000 [332:04:33<71:07:42,  5.00s/it]
 77%|███████▋  | 173763/225000 [332:04:38<72:10:43,  5.07s/it]
 77%|███████▋  | 173764/225000 [332:04:43<71:53:13,  5.05s/it]
 77%|███████▋  | 173765/225000 [332:04:48<72:57:22,  5.13s/it]
 77%|███████▋  | 173766/225000 [332:04:53<72:32:21,  5.10s/it]
 77%|███████▋  | 173767/225000 [332:04:58<73:00:22,  5.13s/it]
 77%|███████▋  | 173768/225000 [332:05:10<98:51:13,  6.95s/it]
 77%|███████▋  | 173769/225000 [332:05:17<98:39:04,  6.93s/it]
 77%|███████▋  | 173770/225000 [332:05:21<86:53:06,  6.11s/it]
 77%|███████▋  | 173771/225000 [332:05:28<92:40:11,  6.51s/it]
 77%|███████▋  | 173772/225000 [332:05:35<95:49:10,  6.73s/it]
 77%|███████▋  | 173773/225000 [332:05:39<83:41:28,  5.88s/it]
 77%|███████▋  | 173774/225000 [332:05:46<86:42:28,  6.09s/it]
 77%|███████▋  | 173775/225000 [332:05:49<75:36:14,  5.31s/it]
 77%|███████▋  | 173776/225000 [332:05:54<73:21:29,  5.16s/it]
 77%|███████▋  | 173777/225000 [332:05:59<73:20:19,  5.15s/it]
 77%|███████▋  | 173778/225000 [332:06:04<72:23:26,  5.09s/it]
 77%|███████▋  | 173779/225000 [332:06:09<71:38:57,  5.04s/it]
 77%|███████▋  | 173780/225000 [332:06:14<69:17:27,  4.87s/it]
 77%|███████▋  | 173781/225000 [332:06:19<71:14:16,  5.01s/it]
 77%|███████▋  | 173782/225000 [332:06:24<71:26:36,  5.02s/it]
 77%|███████▋  | 173783/225000 [332:06:29<72:47:59,  5.12s/it]
 77%|███████▋  | 173784/225000 [332:06:35<73:19:14,  5.15s/it]
 77%|███████▋  | 173785/225000 [332:06:40<72:46:45,  5.12s/it]
 77%|███████▋  | 173786/225000 [332:06:45<73:36:39,  5.17s/it]
 77%|███████▋  | 173787/225000 [332:06:51<75:10:58,  5.28s/it]
 77%|███████▋  | 173788/225000 [332:06:56<73:56:02,  5.20s/it]
 77%|███████▋  | 173789/225000 [332:07:06<97:52:50,  6.88s/it]
 77%|███████▋  | 173790/225000 [332:07:11<89:05:46,  6.26s/it]
 77%|███████▋  | 173791/225000 [332:07:16<84:10:23,  5.92s/it]
 77%|███████▋  | 173792/225000 [332:07:21<81:07:36,  5.70s/it]
 77%|███████▋  | 173793/225000 [332:07:26<78:02:40,  5.49s/it]
 77%|███████▋  | 173794/225000 [332:07:32<77:36:00,  5.46s/it]
 77%|███████▋  | 173795/225000 [332:07:37<76:12:40,  5.36s/it]
 77%|███████▋  | 173796/225000 [332:07:42<76:24:16,  5.37s/it]
 77%|███████▋  | 173797/225000 [332:07:47<75:04:47,  5.28s/it]
 77%|███████▋  | 173798/225000 [332:07:53<75:24:56,  5.30s/it]
 77%|███████▋  | 173799/225000 [332:07:58<76:02:17,  5.35s/it]
 77%|███████▋  | 173800/225000 [332:08:04<75:38:34,  5.32s/it]
                                                              
{'loss': '0.7575', 'grad_norm': '1.188', 'learning_rate': '6.696e-06', 'epoch': '0.7724', 'num_input_tokens_seen': 550424352, 'train_runtime': '1.196e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173800/225000 [332:08:04<75:38:34,  5.32s/it]
 77%|███████▋  | 173801/225000 [332:08:09<75:02:50,  5.28s/it]
 77%|███████▋  | 173802/225000 [332:08:14<75:22:42,  5.30s/it]
 77%|███████▋  | 173803/225000 [332:08:25<100:42:32,  7.08s/it]
 77%|███████▋  | 173804/225000 [332:08:32<100:55:50,  7.10s/it]
 77%|███████▋  | 173805/225000 [332:08:37<89:42:35,  6.31s/it] 
 77%|███████▋  | 173806/225000 [332:08:42<83:07:15,  5.85s/it]
 77%|███████▋  | 173807/225000 [332:08:46<77:42:07,  5.46s/it]
 77%|███████▋  | 173808/225000 [332:08:51<73:44:16,  5.19s/it]
 77%|███████▋  | 173809/225000 [332:08:56<73:28:10,  5.17s/it]
 77%|███████▋  | 173810/225000 [332:09:02<77:05:07,  5.42s/it]
 77%|███████▋  | 173811/225000 [332:09:12<97:52:31,  6.88s/it]
 77%|███████▋  | 173812/225000 [332:09:16<82:59:33,  5.84s/it]
 77%|███████▋  | 173813/225000 [332:09:19<74:00:25,  5.20s/it]
 77%|███████▋  | 173814/225000 [332:09:28<89:11:55,  6.27s/it]
 77%|███████▋  | 173815/225000 [332:09:36<95:49:13,  6.74s/it]
 77%|███████▋  | 173816/225000 [332:09:40<83:34:29,  5.88s/it]
 77%|███████▋  | 173817/225000 [332:09:45<80:57:12,  5.69s/it]
 77%|███████▋  | 173818/225000 [332:09:50<77:59:50,  5.49s/it]
 77%|███████▋  | 173819/225000 [332:09:55<74:53:43,  5.27s/it]
 77%|███████▋  | 173820/225000 [332:09:59<71:46:05,  5.05s/it]
 77%|███████▋  | 173821/225000 [332:10:05<72:39:19,  5.11s/it]
 77%|███████▋  | 173822/225000 [332:10:10<72:27:13,  5.10s/it]
 77%|███████▋  | 173823/225000 [332:10:15<71:49:28,  5.05s/it]
 77%|███████▋  | 173824/225000 [332:10:20<72:50:11,  5.12s/it]
 77%|███████▋  | 173825/225000 [332:10:31<98:16:55,  6.91s/it]
 77%|███████▋  | 173826/225000 [332:10:37<93:57:35,  6.61s/it]
 77%|███████▋  | 173827/225000 [332:10:41<81:46:29,  5.75s/it]
 77%|███████▋  | 173828/225000 [332:10:46<79:05:09,  5.56s/it]
 77%|███████▋  | 173829/225000 [332:10:51<77:13:24,  5.43s/it]
 77%|███████▋  | 173830/225000 [332:10:56<76:50:48,  5.41s/it]
 77%|███████▋  | 173831/225000 [332:11:01<75:32:23,  5.31s/it]
 77%|███████▋  | 173832/225000 [332:11:07<75:45:31,  5.33s/it]
 77%|███████▋  | 173833/225000 [332:11:12<74:40:05,  5.25s/it]
 77%|███████▋  | 173834/225000 [332:11:17<73:24:04,  5.16s/it]
 77%|███████▋  | 173835/225000 [332:11:22<72:15:39,  5.08s/it]
 77%|███████▋  | 173836/225000 [332:11:27<72:02:15,  5.07s/it]
 77%|███████▋  | 173837/225000 [332:11:32<72:53:53,  5.13s/it]
 77%|███████▋  | 173838/225000 [332:11:37<73:52:53,  5.20s/it]
 77%|███████▋  | 173839/225000 [332:11:42<72:34:25,  5.11s/it]
 77%|███████▋  | 173840/225000 [332:11:48<76:52:59,  5.41s/it]
 77%|███████▋  | 173841/225000 [332:12:00<103:14:57,  7.27s/it]
 77%|███████▋  | 173842/225000 [332:12:04<88:32:11,  6.23s/it] 
 77%|███████▋  | 173843/225000 [332:12:07<77:04:09,  5.42s/it]
 77%|███████▋  | 173844/225000 [332:12:13<78:50:23,  5.55s/it]
 77%|███████▋  | 173845/225000 [332:12:20<83:50:00,  5.90s/it]
 77%|███████▋  | 173846/225000 [332:12:23<73:56:22,  5.20s/it]
 77%|███████▋  | 173847/225000 [332:12:33<93:05:06,  6.55s/it]
 77%|███████▋  | 173848/225000 [332:12:37<80:51:07,  5.69s/it]
 77%|███████▋  | 173849/225000 [332:12:43<82:19:21,  5.79s/it]
 77%|███████▋  | 173850/225000 [332:12:47<73:37:34,  5.18s/it]
                                                              
{'loss': '0.7674', 'grad_norm': '1.189', 'learning_rate': '6.684e-06', 'epoch': '0.7727', 'num_input_tokens_seen': 550555808, 'train_runtime': '1.196e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173850/225000 [332:12:47<73:37:34,  5.18s/it]
 77%|███████▋  | 173851/225000 [332:12:55<86:08:31,  6.06s/it]
 77%|███████▋  | 173852/225000 [332:12:59<80:35:19,  5.67s/it]
 77%|███████▋  | 173853/225000 [332:13:04<74:38:09,  5.25s/it]
 77%|███████▋  | 173854/225000 [332:13:08<71:32:44,  5.04s/it]
 77%|███████▋  | 173855/225000 [332:13:13<69:11:55,  4.87s/it]
 77%|███████▋  | 173856/225000 [332:13:17<68:18:36,  4.81s/it]
 77%|███████▋  | 173857/225000 [332:13:25<79:11:15,  5.57s/it]
 77%|███████▋  | 173858/225000 [332:13:34<92:44:02,  6.53s/it]
 77%|███████▋  | 173859/225000 [332:13:37<79:31:41,  5.60s/it]
 77%|███████▋  | 173860/225000 [332:13:43<80:14:59,  5.65s/it]
 77%|███████▋  | 173861/225000 [332:13:49<83:35:59,  5.89s/it]
 77%|███████▋  | 173862/225000 [332:13:54<78:37:13,  5.53s/it]
 77%|███████▋  | 173863/225000 [332:13:59<77:36:16,  5.46s/it]
 77%|███████▋  | 173864/225000 [332:14:05<79:03:14,  5.57s/it]
 77%|███████▋  | 173865/225000 [332:14:09<74:06:52,  5.22s/it]
 77%|███████▋  | 173866/225000 [332:14:14<70:36:30,  4.97s/it]
 77%|███████▋  | 173867/225000 [332:14:18<69:27:05,  4.89s/it]
 77%|███████▋  | 173868/225000 [332:14:23<66:25:15,  4.68s/it]
 77%|███████▋  | 173869/225000 [332:14:27<65:13:10,  4.59s/it]
 77%|███████▋  | 173870/225000 [332:14:39<94:37:09,  6.66s/it]
 77%|███████▋  | 173871/225000 [332:14:44<87:29:16,  6.16s/it]
 77%|███████▋  | 173872/225000 [332:14:51<91:49:07,  6.47s/it]
 77%|███████▋  | 173873/225000 [332:14:54<79:58:40,  5.63s/it]
 77%|███████▋  | 173874/225000 [332:15:02<86:33:50,  6.10s/it]
 77%|███████▋  | 173875/225000 [332:15:05<76:35:01,  5.39s/it]
 77%|███████▋  | 173876/225000 [332:15:12<83:39:15,  5.89s/it]
 77%|███████▋  | 173877/225000 [332:15:16<72:44:18,  5.12s/it]
 77%|███████▋  | 173878/225000 [332:15:22<78:31:29,  5.53s/it]
 77%|███████▋  | 173879/225000 [332:15:26<69:25:14,  4.89s/it]
 77%|███████▋  | 173880/225000 [332:15:31<72:09:14,  5.08s/it]
 77%|███████▋  | 173881/225000 [332:15:39<83:51:47,  5.91s/it]
 77%|███████▋  | 173882/225000 [332:15:43<75:43:22,  5.33s/it]
 77%|███████▋  | 173883/225000 [332:15:51<85:25:16,  6.02s/it]
 77%|███████▋  | 173884/225000 [332:15:55<77:19:42,  5.45s/it]
 77%|███████▋  | 173885/225000 [332:15:59<70:45:47,  4.98s/it]
 77%|███████▋  | 173886/225000 [332:16:07<85:58:21,  6.06s/it]
 77%|███████▋  | 173887/225000 [332:16:14<88:02:23,  6.20s/it]
 77%|███████▋  | 173888/225000 [332:16:17<77:33:38,  5.46s/it]
 77%|███████▋  | 173889/225000 [332:16:22<73:47:36,  5.20s/it]
 77%|███████▋  | 173890/225000 [332:16:27<72:51:54,  5.13s/it]
 77%|███████▋  | 173891/225000 [332:16:32<73:19:18,  5.16s/it]
 77%|███████▋  | 173892/225000 [332:16:37<73:39:45,  5.19s/it]
 77%|███████▋  | 173893/225000 [332:16:43<73:45:59,  5.20s/it]
 77%|███████▋  | 173894/225000 [332:16:48<73:10:03,  5.15s/it]
 77%|███████▋  | 173895/225000 [332:16:53<73:41:46,  5.19s/it]
 77%|███████▋  | 173896/225000 [332:16:59<75:36:45,  5.33s/it]
 77%|███████▋  | 173897/225000 [332:17:04<76:29:56,  5.39s/it]
 77%|███████▋  | 173898/225000 [332:17:10<77:20:11,  5.45s/it]
 77%|███████▋  | 173899/225000 [332:17:15<77:30:50,  5.46s/it]
 77%|███████▋  | 173900/225000 [332:17:21<77:05:19,  5.43s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.138', 'learning_rate': '6.671e-06', 'epoch': '0.7729', 'num_input_tokens_seen': 550686192, 'train_runtime': '1.196e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173900/225000 [332:17:21<77:05:19,  5.43s/it]
 77%|███████▋  | 173901/225000 [332:17:26<77:35:24,  5.47s/it]
 77%|███████▋  | 173902/225000 [332:17:32<78:04:58,  5.50s/it]
 77%|███████▋  | 173903/225000 [332:17:37<77:25:34,  5.46s/it]
 77%|███████▋  | 173904/225000 [332:17:42<77:18:19,  5.45s/it]
 77%|███████▋  | 173905/225000 [332:17:48<77:12:15,  5.44s/it]
 77%|███████▋  | 173906/225000 [332:17:53<75:20:05,  5.31s/it]
 77%|███████▋  | 173907/225000 [332:17:58<75:01:26,  5.29s/it]
 77%|███████▋  | 173908/225000 [332:18:04<76:47:30,  5.41s/it]
 77%|███████▋  | 173909/225000 [332:18:09<75:50:40,  5.34s/it]
 77%|███████▋  | 173910/225000 [332:18:14<74:55:47,  5.28s/it]
 77%|███████▋  | 173911/225000 [332:18:20<75:41:27,  5.33s/it]
 77%|███████▋  | 173912/225000 [332:18:28<90:18:44,  6.36s/it]
 77%|███████▋  | 173913/225000 [332:18:39<106:16:11,  7.49s/it]
 77%|███████▋  | 173914/225000 [332:18:42<87:54:45,  6.20s/it] 
 77%|███████▋  | 173915/225000 [332:18:49<93:43:02,  6.60s/it]
 77%|███████▋  | 173916/225000 [332:18:52<79:11:16,  5.58s/it]
 77%|███████▋  | 173917/225000 [332:18:57<76:43:19,  5.41s/it]
 77%|███████▋  | 173918/225000 [332:19:02<73:02:35,  5.15s/it]
 77%|███████▋  | 173919/225000 [332:19:07<70:50:05,  4.99s/it]
 77%|███████▋  | 173920/225000 [332:19:11<70:07:08,  4.94s/it]
 77%|███████▋  | 173921/225000 [332:19:16<68:53:35,  4.86s/it]
 77%|███████▋  | 173922/225000 [332:19:29<101:18:36,  7.14s/it]
 77%|███████▋  | 173923/225000 [332:19:33<87:41:17,  6.18s/it] 
 77%|███████▋  | 173924/225000 [332:19:36<76:33:35,  5.40s/it]
 77%|███████▋  | 173925/225000 [332:19:45<90:57:08,  6.41s/it]
 77%|███████▋  | 173926/225000 [332:19:50<85:21:17,  6.02s/it]
 77%|███████▋  | 173927/225000 [332:19:55<80:22:57,  5.67s/it]
 77%|███████▋  | 173928/225000 [332:20:00<76:53:02,  5.42s/it]
 77%|███████▋  | 173929/225000 [332:20:04<73:57:02,  5.21s/it]
 77%|███████▋  | 173930/225000 [332:20:09<69:48:00,  4.92s/it]
 77%|███████▋  | 173931/225000 [332:20:13<68:48:18,  4.85s/it]
 77%|███████▋  | 173932/225000 [332:20:18<68:32:16,  4.83s/it]
 77%|███████▋  | 173933/225000 [332:20:23<69:28:18,  4.90s/it]
 77%|███████▋  | 173934/225000 [332:20:32<87:30:09,  6.17s/it]
 77%|███████▋  | 173935/225000 [332:20:40<93:49:01,  6.61s/it]
 77%|███████▋  | 173936/225000 [332:20:45<87:43:01,  6.18s/it]
 77%|███████▋  | 173937/225000 [332:20:49<79:11:25,  5.58s/it]
 77%|███████▋  | 173938/225000 [332:20:57<86:11:59,  6.08s/it]
 77%|███████▋  | 173939/225000 [332:21:02<83:04:42,  5.86s/it]
 77%|███████▋  | 173940/225000 [332:21:06<76:32:07,  5.40s/it]
 77%|███████▋  | 173941/225000 [332:21:10<70:20:38,  4.96s/it]
 77%|███████▋  | 173942/225000 [332:21:15<68:14:27,  4.81s/it]
 77%|███████▋  | 173943/225000 [332:21:20<71:46:37,  5.06s/it]
 77%|███████▋  | 173944/225000 [332:21:30<91:20:47,  6.44s/it]
 77%|███████▋  | 173945/225000 [332:21:34<81:54:31,  5.78s/it]
 77%|███████▋  | 173946/225000 [332:21:38<72:56:06,  5.14s/it]
 77%|███████▋  | 173947/225000 [332:21:44<79:05:46,  5.58s/it]
 77%|███████▋  | 173948/225000 [332:21:50<79:46:04,  5.62s/it]
 77%|███████▋  | 173949/225000 [332:21:57<83:51:39,  5.91s/it]
 77%|███████▋  | 173950/225000 [332:22:00<73:38:10,  5.19s/it]
                                                              
{'loss': '0.7708', 'grad_norm': '1.035', 'learning_rate': '6.659e-06', 'epoch': '0.7731', 'num_input_tokens_seen': 550817312, 'train_runtime': '1.197e+06', 'train_tokens_per_second': '460.3'}

 77%|███████▋  | 173950/225000 [332:22:00<73:38:10,  5.19s/it]
 77%|███████▋  | 173951/225000 [332:22:08<84:48:05,  5.98s/it]
 77%|███████▋  | 173952/225000 [332:22:12<75:28:19,  5.32s/it]
 77%|███████▋  | 173953/225000 [332:22:15<67:05:59,  4.73s/it]
 77%|███████▋  | 173954/225000 [332:22:23<79:17:03,  5.59s/it]
 77%|███████▋  | 173955/225000 [332:22:28<78:36:40,  5.54s/it]
 77%|███████▋  | 173956/225000 [332:22:33<75:59:12,  5.36s/it]
 77%|███████▋  | 173957/225000 [332:22:38<74:14:57,  5.24s/it]
 77%|███████▋  | 173958/225000 [332:22:43<73:54:20,  5.21s/it]
 77%|███████▋  | 173959/225000 [332:22:48<72:31:09,  5.11s/it]
 77%|███████▋  | 173960/225000 [332:22:53<70:20:18,  4.96s/it]
 77%|███████▋  | 173961/225000 [332:22:57<67:58:21,  4.79s/it]
 77%|███████▋  | 173962/225000 [332:23:01<65:42:46,  4.64s/it]
 77%|███████▋  | 173963/225000 [332:23:07<68:15:46,  4.82s/it]
 77%|███████▋  | 173964/225000 [332:23:12<69:25:21,  4.90s/it]
 77%|███████▋  | 173965/225000 [332:23:17<69:50:14,  4.93s/it]
 77%|███████▋  | 173966/225000 [332:23:22<70:25:01,  4.97s/it]
 77%|███████▋  | 173967/225000 [332:23:27<71:03:13,  5.01s/it]
 77%|███████▋  | 173968/225000 [332:23:32<71:40:22,  5.06s/it]
 77%|███████▋  | 173969/225000 [332:23:38<73:48:24,  5.21s/it]
 77%|███████▋  | 173970/225000 [332:23:43<74:59:50,  5.29s/it]
 77%|███████▋  | 173971/225000 [332:23:48<73:04:34,  5.16s/it]
 77%|███████▋  | 173972/225000 [332:23:55<80:28:24,  5.68s/it]
 77%|███████▋  | 173973/225000 [332:24:08<111:31:50,  7.87s/it]
 77%|███████▋  | 173974/225000 [332:24:11<91:58:50,  6.49s/it] 
 77%|███████▋  | 173975/225000 [332:24:18<94:10:33,  6.64s/it]
 77%|███████▋  | 173976/225000 [332:24:22<80:42:34,  5.69s/it]
 77%|███████▋  | 173977/225000 [332:24:25<71:38:23,  5.05s/it]
 77%|███████▋  | 173978/225000 [332:24:35<91:34:10,  6.46s/it]
 77%|███████▋  | 173979/225000 [332:24:39<80:21:09,  5.67s/it]
 77%|███████▋  | 173980/225000 [332:24:43<72:37:53,  5.12s/it]
 77%|███████▋  | 173981/225000 [332:24:47<68:35:42,  4.84s/it]
 77%|███████▋  | 173982/225000 [332:24:51<65:47:47,  4.64s/it]
 77%|███████▋  | 173983/225000 [332:25:00<85:54:21,  6.06s/it]
 77%|███████▋  | 173984/225000 [332:25:06<85:36:53,  6.04s/it]
 77%|███████▋  | 173985/225000 [332:25:10<75:24:35,  5.32s/it]
 77%|███████▋  | 173986/225000 [332:25:18<85:42:59,  6.05s/it]
 77%|███████▋  | 173987/225000 [332:25:22<76:23:37,  5.39s/it]
 77%|███████▋  | 173988/225000 [332:25:27<78:13:10,  5.52s/it]
 77%|███████▋  | 173989/225000 [332:25:34<82:40:51,  5.84s/it]
 77%|███████▋  | 173990/225000 [332:25:40<81:47:58,  5.77s/it]
 77%|███████▋  | 173991/225000 [332:25:43<71:13:12,  5.03s/it]
 77%|███████▋  | 173992/225000 [332:25:47<66:55:22,  4.72s/it]
 77%|███████▋  | 173993/225000 [332:25:52<67:35:14,  4.77s/it]
 77%|███████▋  | 173994/225000 [332:25:57<70:39:27,  4.99s/it]
 77%|███████▋  | 173995/225000 [332:26:02<71:42:07,  5.06s/it]
 77%|███████▋  | 173996/225000 [332:26:07<67:54:46,  4.79s/it]
 77%|███████▋  | 173997/225000 [332:26:12<68:29:36,  4.83s/it]
 77%|███████▋  | 173998/225000 [332:26:17<71:04:47,  5.02s/it]
 77%|███████▋  | 173999/225000 [332:26:22<69:44:02,  4.92s/it]
 77%|███████▋  | 174000/225000 [332:26:27<73:03:16,  5.16s/it]
                                                              
{'loss': '0.7595', 'grad_norm': '1.213', 'learning_rate': '6.647e-06', 'epoch': '0.7733', 'num_input_tokens_seen': 550946440, 'train_runtime': '1.197e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174000/225000 [332:26:27<73:03:16,  5.16s/it]
 77%|███████▋  | 174001/225000 [332:26:33<73:40:31,  5.20s/it]
 77%|███████▋  | 174002/225000 [332:26:38<74:05:00,  5.23s/it]
 77%|███████▋  | 174003/225000 [332:26:43<72:34:23,  5.12s/it]
 77%|███████▋  | 174004/225000 [332:26:48<74:07:09,  5.23s/it]
 77%|███████▋  | 174005/225000 [332:26:53<73:00:02,  5.15s/it]
 77%|███████▋  | 174006/225000 [332:26:59<73:27:20,  5.19s/it]
 77%|███████▋  | 174007/225000 [332:27:04<72:24:14,  5.11s/it]
 77%|███████▋  | 174008/225000 [332:27:08<69:41:10,  4.92s/it]
 77%|███████▋  | 174009/225000 [332:27:20<98:20:50,  6.94s/it]
 77%|███████▋  | 174010/225000 [332:27:26<97:50:59,  6.91s/it]
 77%|███████▋  | 174011/225000 [332:27:31<88:11:37,  6.23s/it]
 77%|███████▋  | 174012/225000 [332:27:36<82:06:46,  5.80s/it]
 77%|███████▋  | 174013/225000 [332:27:40<76:29:46,  5.40s/it]
 77%|███████▋  | 174014/225000 [332:27:45<73:28:16,  5.19s/it]
 77%|███████▋  | 174015/225000 [332:27:50<71:16:14,  5.03s/it]
 77%|███████▋  | 174016/225000 [332:27:54<69:48:07,  4.93s/it]
 77%|███████▋  | 174017/225000 [332:28:04<91:18:48,  6.45s/it]
 77%|███████▋  | 174018/225000 [332:28:10<88:45:43,  6.27s/it]
 77%|███████▋  | 174019/225000 [332:28:15<82:25:46,  5.82s/it]
 77%|███████▋  | 174020/225000 [332:28:20<78:05:23,  5.51s/it]
 77%|███████▋  | 174021/225000 [332:28:25<75:50:00,  5.36s/it]
 77%|███████▋  | 174022/225000 [332:28:30<72:58:08,  5.15s/it]
 77%|███████▋  | 174023/225000 [332:28:35<73:01:36,  5.16s/it]
 77%|███████▋  | 174024/225000 [332:28:40<72:38:31,  5.13s/it]
 77%|███████▋  | 174025/225000 [332:28:45<71:51:54,  5.08s/it]
 77%|███████▋  | 174026/225000 [332:28:50<72:16:23,  5.10s/it]
 77%|███████▋  | 174027/225000 [332:28:55<71:27:22,  5.05s/it]
 77%|███████▋  | 174028/225000 [332:29:00<72:02:56,  5.09s/it]
 77%|███████▋  | 174029/225000 [332:29:05<71:07:08,  5.02s/it]
 77%|███████▋  | 174030/225000 [332:29:10<70:09:33,  4.96s/it]
 77%|███████▋  | 174031/225000 [332:29:23<104:09:58,  7.36s/it]
 77%|███████▋  | 174032/225000 [332:29:26<88:03:21,  6.22s/it] 
 77%|███████▋  | 174033/225000 [332:29:33<91:18:13,  6.45s/it]
 77%|███████▋  | 174034/225000 [332:29:36<77:59:29,  5.51s/it]
 77%|███████▋  | 174035/225000 [332:29:45<91:00:32,  6.43s/it]
 77%|███████▋  | 174036/225000 [332:29:49<82:30:28,  5.83s/it]
 77%|███████▋  | 174037/225000 [332:29:54<77:40:39,  5.49s/it]
 77%|███████▋  | 174038/225000 [332:29:59<74:36:15,  5.27s/it]
 77%|███████▋  | 174039/225000 [332:30:04<73:03:32,  5.16s/it]
 77%|███████▋  | 174040/225000 [332:30:08<70:10:01,  4.96s/it]
 77%|███████▋  | 174041/225000 [332:30:13<67:57:51,  4.80s/it]
 77%|███████▋  | 174042/225000 [332:30:17<65:58:55,  4.66s/it]
 77%|███████▋  | 174043/225000 [332:30:28<91:48:40,  6.49s/it]
 77%|███████▋  | 174044/225000 [332:30:32<83:52:23,  5.93s/it]
 77%|███████▋  | 174045/225000 [332:30:39<86:22:05,  6.10s/it]
 77%|███████▋  | 174046/225000 [332:30:44<83:23:05,  5.89s/it]
 77%|███████▋  | 174047/225000 [332:30:48<75:05:23,  5.31s/it]
 77%|███████▋  | 174048/225000 [332:30:53<70:38:56,  4.99s/it]
 77%|███████▋  | 174049/225000 [332:31:01<84:54:09,  6.00s/it]
 77%|███████▋  | 174050/225000 [332:31:06<81:50:43,  5.78s/it]
                                                              
{'loss': '0.7669', 'grad_norm': '1.188', 'learning_rate': '6.634e-06', 'epoch': '0.7736', 'num_input_tokens_seen': 551077352, 'train_runtime': '1.197e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174050/225000 [332:31:06<81:50:43,  5.78s/it]
 77%|███████▋  | 174051/225000 [332:31:11<75:39:54,  5.35s/it]
 77%|███████▋  | 174052/225000 [332:31:15<72:36:06,  5.13s/it]
 77%|███████▋  | 174053/225000 [332:31:20<72:06:19,  5.10s/it]
 77%|███████▋  | 174054/225000 [332:31:25<70:00:39,  4.95s/it]
 77%|███████▋  | 174055/225000 [332:31:30<69:30:07,  4.91s/it]
 77%|███████▋  | 174056/225000 [332:31:36<74:27:43,  5.26s/it]
 77%|███████▋  | 174057/225000 [332:31:46<94:14:59,  6.66s/it]
 77%|███████▋  | 174058/225000 [332:31:49<82:31:14,  5.83s/it]
 77%|███████▋  | 174059/225000 [332:31:53<73:25:35,  5.19s/it]
 77%|███████▋  | 174060/225000 [332:32:02<90:35:38,  6.40s/it]
 77%|███████▋  | 174061/225000 [332:32:06<77:33:53,  5.48s/it]
 77%|███████▋  | 174062/225000 [332:32:13<87:03:31,  6.15s/it]
 77%|███████▋  | 174063/225000 [332:32:17<77:26:08,  5.47s/it]
 77%|███████▋  | 174064/225000 [332:32:26<91:28:48,  6.47s/it]
 77%|███████▋  | 174065/225000 [332:32:29<76:30:32,  5.41s/it]
 77%|███████▋  | 174066/225000 [332:32:35<79:37:29,  5.63s/it]
 77%|███████▋  | 174067/225000 [332:32:40<75:22:25,  5.33s/it]
 77%|███████▋  | 174068/225000 [332:32:44<71:04:50,  5.02s/it]
 77%|███████▋  | 174069/225000 [332:32:49<69:59:54,  4.95s/it]
 77%|███████▋  | 174070/225000 [332:32:54<70:24:34,  4.98s/it]
 77%|███████▋  | 174071/225000 [332:32:58<68:28:46,  4.84s/it]
 77%|███████▋  | 174072/225000 [332:33:03<69:04:08,  4.88s/it]
 77%|███████▋  | 174073/225000 [332:33:08<67:42:30,  4.79s/it]
 77%|███████▋  | 174074/225000 [332:33:13<68:46:07,  4.86s/it]
 77%|███████▋  | 174075/225000 [332:33:18<68:03:43,  4.81s/it]
 77%|███████▋  | 174076/225000 [332:33:23<68:47:34,  4.86s/it]
 77%|███████▋  | 174077/225000 [332:33:30<80:38:05,  5.70s/it]
 77%|███████▋  | 174078/225000 [332:33:41<99:46:57,  7.05s/it]
 77%|███████▋  | 174079/225000 [332:33:44<83:49:17,  5.93s/it]
 77%|███████▋  | 174080/225000 [332:33:48<77:02:14,  5.45s/it]
 77%|███████▋  | 174081/225000 [332:33:58<96:40:44,  6.84s/it]
 77%|███████▋  | 174082/225000 [332:34:02<82:27:03,  5.83s/it]
 77%|███████▋  | 174083/225000 [332:34:05<72:50:16,  5.15s/it]
 77%|███████▋  | 174084/225000 [332:34:15<90:12:28,  6.38s/it]
 77%|███████▋  | 174085/225000 [332:34:18<78:14:17,  5.53s/it]
 77%|███████▋  | 174086/225000 [332:34:22<69:43:33,  4.93s/it]
 77%|███████▋  | 174087/225000 [332:34:31<88:45:47,  6.28s/it]
 77%|███████▋  | 174088/225000 [332:34:35<77:08:13,  5.45s/it]
 77%|███████▋  | 174089/225000 [332:34:39<71:06:56,  5.03s/it]
 77%|███████▋  | 174090/225000 [332:34:46<80:56:21,  5.72s/it]
 77%|███████▋  | 174091/225000 [332:34:53<84:39:49,  5.99s/it]
 77%|███████▋  | 174092/225000 [332:34:56<74:49:24,  5.29s/it]
 77%|███████▋  | 174093/225000 [332:35:05<88:56:44,  6.29s/it]
 77%|███████▋  | 174094/225000 [332:35:08<76:43:19,  5.43s/it]
 77%|███████▋  | 174095/225000 [332:35:17<89:09:42,  6.31s/it]
 77%|███████▋  | 174096/225000 [332:35:20<76:48:57,  5.43s/it]
 77%|███████▋  | 174097/225000 [332:35:27<84:56:18,  6.01s/it]
 77%|███████▋  | 174098/225000 [332:35:31<75:13:16,  5.32s/it]
 77%|███████▋  | 174099/225000 [332:35:37<77:35:15,  5.49s/it]
 77%|███████▋  | 174100/225000 [332:35:43<81:07:18,  5.74s/it]
                                                              
{'loss': '0.7587', 'grad_norm': '1.175', 'learning_rate': '6.622e-06', 'epoch': '0.7738', 'num_input_tokens_seen': 551206944, 'train_runtime': '1.197e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174100/225000 [332:35:43<81:07:18,  5.74s/it]
 77%|███████▋  | 174101/225000 [332:35:47<71:34:32,  5.06s/it]
 77%|███████▋  | 174102/225000 [332:35:51<66:44:30,  4.72s/it]
 77%|███████▋  | 174103/225000 [332:35:59<83:03:52,  5.88s/it]
 77%|███████▋  | 174104/225000 [332:36:04<76:42:53,  5.43s/it]
 77%|███████▋  | 174105/225000 [332:36:07<69:01:18,  4.88s/it]
 77%|███████▋  | 174106/225000 [332:36:15<80:44:07,  5.71s/it]
 77%|███████▋  | 174107/225000 [332:36:19<75:21:47,  5.33s/it]
 77%|███████▋  | 174108/225000 [332:36:24<70:33:19,  4.99s/it]
 77%|███████▋  | 174109/225000 [332:36:28<68:45:42,  4.86s/it]
 77%|███████▋  | 174110/225000 [332:36:33<68:39:17,  4.86s/it]
 77%|███████▋  | 174111/225000 [332:36:43<88:33:05,  6.26s/it]
 77%|███████▋  | 174112/225000 [332:36:50<92:55:57,  6.57s/it]
 77%|███████▋  | 174113/225000 [332:36:54<82:29:25,  5.84s/it]
 77%|███████▋  | 174114/225000 [332:36:58<73:52:10,  5.23s/it]
 77%|███████▋  | 174115/225000 [332:37:02<67:55:30,  4.81s/it]
 77%|███████▋  | 174116/225000 [332:37:11<89:05:38,  6.30s/it]
 77%|███████▋  | 174117/225000 [332:37:15<79:10:41,  5.60s/it]
 77%|███████▋  | 174118/225000 [332:37:22<84:38:34,  5.99s/it]
 77%|███████▋  | 174119/225000 [332:37:27<80:47:28,  5.72s/it]
 77%|███████▋  | 174120/225000 [332:37:33<78:55:47,  5.58s/it]
 77%|███████▋  | 174121/225000 [332:37:36<68:58:59,  4.88s/it]
 77%|███████▋  | 174122/225000 [332:37:43<79:37:37,  5.63s/it]
 77%|███████▋  | 174123/225000 [332:37:50<82:48:23,  5.86s/it]
 77%|███████▋  | 174124/225000 [332:37:54<75:54:27,  5.37s/it]
 77%|███████▋  | 174125/225000 [332:37:59<73:45:49,  5.22s/it]
 77%|███████▋  | 174126/225000 [332:38:04<72:38:43,  5.14s/it]
 77%|███████▋  | 174127/225000 [332:38:08<69:47:00,  4.94s/it]
 77%|███████▋  | 174128/225000 [332:38:13<70:27:52,  4.99s/it]
 77%|███████▋  | 174129/225000 [332:38:18<70:17:59,  4.97s/it]
 77%|███████▋  | 174130/225000 [332:38:23<69:16:10,  4.90s/it]
 77%|███████▋  | 174131/225000 [332:38:28<69:28:04,  4.92s/it]
 77%|███████▋  | 174132/225000 [332:38:36<82:27:22,  5.84s/it]
 77%|███████▋  | 174133/225000 [332:38:46<101:21:58,  7.17s/it]
 77%|███████▋  | 174134/225000 [332:38:51<90:05:27,  6.38s/it] 
 77%|███████▋  | 174135/225000 [332:38:55<81:29:38,  5.77s/it]
 77%|███████▋  | 174136/225000 [332:39:00<76:13:24,  5.39s/it]
 77%|███████▋  | 174137/225000 [332:39:04<73:03:59,  5.17s/it]
 77%|███████▋  | 174138/225000 [332:39:09<70:55:53,  5.02s/it]
 77%|███████▋  | 174139/225000 [332:39:14<71:03:04,  5.03s/it]
 77%|███████▋  | 174140/225000 [332:39:18<66:21:43,  4.70s/it]
 77%|███████▋  | 174141/225000 [332:39:23<70:08:40,  4.97s/it]
 77%|███████▋  | 174142/225000 [332:39:33<91:10:53,  6.45s/it]
 77%|███████▋  | 174143/225000 [332:39:37<79:48:19,  5.65s/it]
 77%|███████▋  | 174144/225000 [332:39:41<74:27:08,  5.27s/it]
 77%|███████▋  | 174145/225000 [332:39:46<73:17:15,  5.19s/it]
 77%|███████▋  | 174146/225000 [332:39:52<73:58:16,  5.24s/it]
 77%|███████▋  | 174147/225000 [332:39:57<74:56:56,  5.31s/it]
 77%|███████▋  | 174148/225000 [332:40:02<74:05:39,  5.25s/it]
 77%|███████▋  | 174149/225000 [332:40:08<74:43:07,  5.29s/it]
 77%|███████▋  | 174150/225000 [332:40:13<74:05:17,  5.25s/it]
                                                              
{'loss': '0.7663', 'grad_norm': '1.149', 'learning_rate': '6.609e-06', 'epoch': '0.774', 'num_input_tokens_seen': 551337648, 'train_runtime': '1.198e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174150/225000 [332:40:13<74:05:17,  5.25s/it]
 77%|███████▋  | 174151/225000 [332:40:18<72:48:39,  5.15s/it]
 77%|███████▋  | 174152/225000 [332:40:23<73:45:03,  5.22s/it]
 77%|███████▋  | 174153/225000 [332:40:28<72:59:34,  5.17s/it]
 77%|███████▋  | 174154/225000 [332:40:34<74:23:15,  5.27s/it]
 77%|███████▋  | 174155/225000 [332:40:39<75:08:22,  5.32s/it]
 77%|███████▋  | 174156/225000 [332:40:44<73:48:59,  5.23s/it]
 77%|███████▋  | 174157/225000 [332:40:50<74:40:25,  5.29s/it]
 77%|███████▋  | 174158/225000 [332:40:55<73:15:38,  5.19s/it]
 77%|███████▋  | 174159/225000 [332:41:00<72:43:55,  5.15s/it]
 77%|███████▋  | 174160/225000 [332:41:05<72:47:10,  5.15s/it]
 77%|███████▋  | 174161/225000 [332:41:10<73:06:21,  5.18s/it]
 77%|███████▋  | 174162/225000 [332:41:16<74:10:38,  5.25s/it]
 77%|███████▋  | 174163/225000 [332:41:20<72:15:58,  5.12s/it]
 77%|███████▋  | 174164/225000 [332:41:26<73:40:53,  5.22s/it]
 77%|███████▋  | 174165/225000 [332:41:32<78:20:28,  5.55s/it]
 77%|███████▋  | 174166/225000 [332:41:45<111:09:24,  7.87s/it]
 77%|███████▋  | 174167/225000 [332:41:50<95:23:39,  6.76s/it] 
 77%|███████▋  | 174168/225000 [332:41:54<83:57:11,  5.95s/it]
 77%|███████▋  | 174169/225000 [332:41:58<75:41:08,  5.36s/it]
 77%|███████▋  | 174170/225000 [332:42:05<83:07:42,  5.89s/it]
 77%|███████▋  | 174171/225000 [332:42:11<86:13:31,  6.11s/it]
 77%|███████▋  | 174172/225000 [332:42:15<75:24:14,  5.34s/it]
 77%|███████▋  | 174173/225000 [332:42:25<93:55:54,  6.65s/it]
 77%|███████▋  | 174174/225000 [332:42:28<81:01:40,  5.74s/it]
 77%|███████▋  | 174175/225000 [332:42:32<72:33:22,  5.14s/it]
 77%|███████▋  | 174176/225000 [332:42:40<83:51:52,  5.94s/it]
 77%|███████▋  | 174177/225000 [332:42:45<82:47:55,  5.86s/it]
 77%|███████▋  | 174178/225000 [332:42:49<72:07:44,  5.11s/it]
 77%|███████▋  | 174179/225000 [332:42:58<89:28:16,  6.34s/it]
 77%|███████▋  | 174180/225000 [332:43:02<78:35:15,  5.57s/it]
 77%|███████▋  | 174181/225000 [332:43:07<76:52:28,  5.45s/it]
 77%|███████▋  | 174182/225000 [332:43:13<81:06:21,  5.75s/it]
 77%|███████▋  | 174183/225000 [332:43:18<75:21:51,  5.34s/it]
 77%|███████▋  | 174184/225000 [332:43:23<73:31:06,  5.21s/it]
 77%|███████▋  | 174185/225000 [332:43:28<73:18:46,  5.19s/it]
 77%|███████▋  | 174186/225000 [332:43:33<72:41:33,  5.15s/it]
 77%|███████▋  | 174187/225000 [332:43:38<73:49:22,  5.23s/it]
 77%|███████▋  | 174188/225000 [332:43:44<74:46:21,  5.30s/it]
 77%|███████▋  | 174189/225000 [332:43:49<72:29:31,  5.14s/it]
 77%|███████▋  | 174190/225000 [332:43:53<71:02:15,  5.03s/it]
 77%|███████▋  | 174191/225000 [332:43:58<71:09:38,  5.04s/it]
 77%|███████▋  | 174192/225000 [332:44:04<72:26:51,  5.13s/it]
 77%|███████▋  | 174193/225000 [332:44:09<74:34:44,  5.28s/it]
 77%|███████▋  | 174194/225000 [332:44:14<72:40:17,  5.15s/it]
 77%|███████▋  | 174195/225000 [332:44:20<74:32:22,  5.28s/it]
 77%|███████▋  | 174196/225000 [332:44:25<73:05:10,  5.18s/it]
 77%|███████▋  | 174197/225000 [332:44:29<71:20:55,  5.06s/it]
 77%|███████▋  | 174198/225000 [332:44:35<72:45:39,  5.16s/it]
 77%|███████▋  | 174199/225000 [332:44:40<72:18:22,  5.12s/it]
 77%|███████▋  | 174200/225000 [332:44:46<74:23:12,  5.27s/it]
                                                              
{'loss': '0.7822', 'grad_norm': '1.096', 'learning_rate': '6.597e-06', 'epoch': '0.7742', 'num_input_tokens_seen': 551468768, 'train_runtime': '1.198e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174200/225000 [332:44:46<74:23:12,  5.27s/it]
 77%|███████▋  | 174201/225000 [332:44:51<77:17:27,  5.48s/it]
 77%|███████▋  | 174202/225000 [332:45:04<106:15:52,  7.53s/it]
 77%|███████▋  | 174203/225000 [332:45:07<88:37:22,  6.28s/it] 
 77%|███████▋  | 174204/225000 [332:45:15<94:37:50,  6.71s/it]
 77%|███████▋  | 174205/225000 [332:45:18<80:41:48,  5.72s/it]
 77%|███████▋  | 174206/225000 [332:45:26<88:02:22,  6.24s/it]
 77%|███████▋  | 174207/225000 [332:45:29<73:51:19,  5.23s/it]
 77%|███████▋  | 174208/225000 [332:45:33<69:45:29,  4.94s/it]
 77%|███████▋  | 174209/225000 [332:45:38<68:58:04,  4.89s/it]
 77%|███████▋  | 174210/225000 [332:45:42<67:29:55,  4.78s/it]
 77%|███████▋  | 174211/225000 [332:45:47<66:34:43,  4.72s/it]
 77%|███████▋  | 174212/225000 [332:45:52<67:21:14,  4.77s/it]
 77%|███████▋  | 174213/225000 [332:45:56<65:03:14,  4.61s/it]
 77%|███████▋  | 174214/225000 [332:46:01<65:22:06,  4.63s/it]
 77%|███████▋  | 174215/225000 [332:46:13<97:01:38,  6.88s/it]
 77%|███████▋  | 174216/225000 [332:46:16<82:30:04,  5.85s/it]
 77%|███████▋  | 174217/225000 [332:46:23<87:57:41,  6.24s/it]
 77%|███████▋  | 174218/225000 [332:46:28<79:34:50,  5.64s/it]
 77%|███████▋  | 174219/225000 [332:46:32<74:55:05,  5.31s/it]
 77%|███████▋  | 174220/225000 [332:46:36<69:14:33,  4.91s/it]
 77%|███████▋  | 174221/225000 [332:46:41<67:24:47,  4.78s/it]
 77%|███████▋  | 174222/225000 [332:46:45<66:14:56,  4.70s/it]
 77%|███████▋  | 174223/225000 [332:46:53<78:31:33,  5.57s/it]
 77%|███████▋  | 174224/225000 [332:47:02<92:56:16,  6.59s/it]
 77%|███████▋  | 174225/225000 [332:47:06<83:19:33,  5.91s/it]
 77%|███████▋  | 174226/225000 [332:47:11<78:00:46,  5.53s/it]
 77%|███████▋  | 174227/225000 [332:47:15<71:44:49,  5.09s/it]
 77%|███████▋  | 174228/225000 [332:47:19<69:55:59,  4.96s/it]
 77%|███████▋  | 174229/225000 [332:47:29<89:29:22,  6.35s/it]
 77%|███████▋  | 174230/225000 [332:47:33<82:03:25,  5.82s/it]
 77%|███████▋  | 174231/225000 [332:47:37<72:08:55,  5.12s/it]
 77%|███████▋  | 174232/225000 [332:47:44<79:08:23,  5.61s/it]
 77%|███████▋  | 174233/225000 [332:47:50<80:03:28,  5.68s/it]
 77%|███████▋  | 174234/225000 [332:47:53<70:54:04,  5.03s/it]
 77%|███████▋  | 174235/225000 [332:47:57<67:09:23,  4.76s/it]
 77%|███████▋  | 174236/225000 [332:48:06<83:59:30,  5.96s/it]
 77%|███████▋  | 174237/225000 [332:48:10<78:06:32,  5.54s/it]
 77%|███████▋  | 174238/225000 [332:48:19<90:11:25,  6.40s/it]
 77%|███████▋  | 174239/225000 [332:48:22<77:59:38,  5.53s/it]
 77%|███████▋  | 174240/225000 [332:48:27<75:51:05,  5.38s/it]
 77%|███████▋  | 174241/225000 [332:48:33<77:17:22,  5.48s/it]
 77%|███████▋  | 174242/225000 [332:48:39<77:11:56,  5.48s/it]
 77%|███████▋  | 174243/225000 [332:48:44<76:06:59,  5.40s/it]
 77%|███████▋  | 174244/225000 [332:48:53<90:20:51,  6.41s/it]
 77%|███████▋  | 174245/225000 [332:48:56<79:42:21,  5.65s/it]
 77%|███████▋  | 174246/225000 [332:49:01<74:51:29,  5.31s/it]
 77%|███████▋  | 174247/225000 [332:49:08<83:59:43,  5.96s/it]
 77%|███████▋  | 174248/225000 [332:49:13<77:43:24,  5.51s/it]
 77%|███████▋  | 174249/225000 [332:49:20<83:50:25,  5.95s/it]
 77%|███████▋  | 174250/225000 [332:49:24<76:03:53,  5.40s/it]
                                                              
{'loss': '0.7623', 'grad_norm': '1.136', 'learning_rate': '6.585e-06', 'epoch': '0.7744', 'num_input_tokens_seen': 551599312, 'train_runtime': '1.198e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174250/225000 [332:49:24<76:03:53,  5.40s/it]
 77%|███████▋  | 174251/225000 [332:49:28<72:09:05,  5.12s/it]
 77%|███████▋  | 174252/225000 [332:49:33<70:31:43,  5.00s/it]
 77%|███████▋  | 174253/225000 [332:49:38<69:58:53,  4.96s/it]
 77%|███████▋  | 174254/225000 [332:49:43<68:38:38,  4.87s/it]
 77%|███████▋  | 174255/225000 [332:49:47<66:08:40,  4.69s/it]
 77%|███████▋  | 174256/225000 [332:49:52<67:22:31,  4.78s/it]
 77%|███████▋  | 174257/225000 [332:49:57<67:16:03,  4.77s/it]
 77%|███████▋  | 174258/225000 [332:50:01<66:46:40,  4.74s/it]
 77%|███████▋  | 174259/225000 [332:50:06<66:58:10,  4.75s/it]
 77%|███████▋  | 174260/225000 [332:50:11<67:11:57,  4.77s/it]
 77%|███████▋  | 174261/225000 [332:50:24<101:00:18,  7.17s/it]
 77%|███████▋  | 174262/225000 [332:50:28<88:20:15,  6.27s/it] 
 77%|███████▋  | 174263/225000 [332:50:31<76:36:14,  5.44s/it]
 77%|███████▋  | 174264/225000 [332:50:38<79:25:44,  5.64s/it]
 77%|███████▋  | 174265/225000 [332:50:44<82:29:34,  5.85s/it]
 77%|███████▋  | 174266/225000 [332:50:48<73:30:08,  5.22s/it]
 77%|███████▋  | 174267/225000 [332:50:58<95:10:57,  6.75s/it]
 77%|███████▋  | 174268/225000 [332:51:01<80:39:25,  5.72s/it]
 77%|███████▋  | 174269/225000 [332:51:05<74:05:51,  5.26s/it]
 77%|███████▋  | 174270/225000 [332:51:10<70:45:00,  5.02s/it]
 77%|███████▋  | 174271/225000 [332:51:14<68:35:39,  4.87s/it]
 77%|███████▋  | 174272/225000 [332:51:19<66:16:41,  4.70s/it]
 77%|███████▋  | 174273/225000 [332:51:24<68:52:02,  4.89s/it]
 77%|███████▋  | 174274/225000 [332:51:35<95:36:52,  6.79s/it]
 77%|███████▋  | 174275/225000 [332:51:39<82:01:36,  5.82s/it]
 77%|███████▋  | 174276/225000 [332:51:44<77:16:07,  5.48s/it]
 77%|███████▋  | 174277/225000 [332:51:49<75:34:09,  5.36s/it]
 77%|███████▋  | 174278/225000 [332:51:54<73:33:18,  5.22s/it]
 77%|███████▋  | 174279/225000 [332:51:59<72:48:20,  5.17s/it]
 77%|███████▋  | 174280/225000 [332:52:04<71:52:27,  5.10s/it]
 77%|███████▋  | 174281/225000 [332:52:08<70:51:49,  5.03s/it]
 77%|███████▋  | 174282/225000 [332:52:13<69:12:45,  4.91s/it]
 77%|███████▋  | 174283/225000 [332:52:18<70:25:07,  5.00s/it]
 77%|███████▋  | 174284/225000 [332:52:23<70:10:26,  4.98s/it]
 77%|███████▋  | 174285/225000 [332:52:28<71:07:56,  5.05s/it]
 77%|███████▋  | 174286/225000 [332:52:33<70:02:57,  4.97s/it]
 77%|███████▋  | 174287/225000 [332:52:38<70:21:10,  4.99s/it]
 77%|███████▋  | 174288/225000 [332:52:44<73:48:03,  5.24s/it]
 77%|███████▋  | 174289/225000 [332:52:49<73:01:41,  5.18s/it]
 77%|███████▋  | 174290/225000 [332:52:59<94:11:28,  6.69s/it]
 77%|███████▋  | 174291/225000 [332:53:07<100:23:38,  7.13s/it]
 77%|███████▋  | 174292/225000 [332:53:12<90:17:07,  6.41s/it] 
 77%|███████▋  | 174293/225000 [332:53:16<81:00:22,  5.75s/it]
 77%|███████▋  | 174294/225000 [332:53:20<73:51:05,  5.24s/it]
 77%|███████▋  | 174295/225000 [332:53:25<70:45:46,  5.02s/it]
 77%|███████▋  | 174296/225000 [332:53:30<68:51:36,  4.89s/it]
 77%|███████▋  | 174297/225000 [332:53:34<68:58:10,  4.90s/it]
 77%|███████▋  | 174298/225000 [332:53:46<96:05:06,  6.82s/it]
 77%|███████▋  | 174299/225000 [332:53:51<89:36:30,  6.36s/it]
 77%|███████▋  | 174300/225000 [332:53:57<90:00:27,  6.39s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.234', 'learning_rate': '6.572e-06', 'epoch': '0.7747', 'num_input_tokens_seen': 551730104, 'train_runtime': '1.198e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174300/225000 [332:53:58<90:00:27,  6.39s/it]
 77%|███████▋  | 174301/225000 [332:54:02<83:51:01,  5.95s/it]
 77%|███████▋  | 174302/225000 [332:54:08<80:31:34,  5.72s/it]
 77%|███████▋  | 174303/225000 [332:54:13<78:50:30,  5.60s/it]
 77%|███████▋  | 174304/225000 [332:54:18<76:58:12,  5.47s/it]
 77%|███████▋  | 174305/225000 [332:54:23<74:46:32,  5.31s/it]
 77%|███████▋  | 174306/225000 [332:54:28<72:40:04,  5.16s/it]
 77%|███████▋  | 174307/225000 [332:54:33<73:45:06,  5.24s/it]
 77%|███████▋  | 174308/225000 [332:54:39<74:33:23,  5.29s/it]
 77%|███████▋  | 174309/225000 [332:54:44<74:05:00,  5.26s/it]
 77%|███████▋  | 174310/225000 [332:54:49<72:26:45,  5.15s/it]
 77%|███████▋  | 174311/225000 [332:54:53<68:49:22,  4.89s/it]
 77%|███████▋  | 174312/225000 [332:54:58<69:47:41,  4.96s/it]
 77%|███████▋  | 174313/225000 [332:55:04<71:45:49,  5.10s/it]
 77%|███████▋  | 174314/225000 [332:55:09<71:42:45,  5.09s/it]
 77%|███████▋  | 174315/225000 [332:55:14<73:32:53,  5.22s/it]
 77%|███████▋  | 174316/225000 [332:55:19<73:36:36,  5.23s/it]
 77%|███████▋  | 174317/225000 [332:55:25<73:20:04,  5.21s/it]
 77%|███████▋  | 174318/225000 [332:55:30<73:48:19,  5.24s/it]
 77%|███████▋  | 174319/225000 [332:55:35<74:29:07,  5.29s/it]
 77%|███████▋  | 174320/225000 [332:55:40<73:10:16,  5.20s/it]
 77%|███████▋  | 174321/225000 [332:55:46<73:31:42,  5.22s/it]
 77%|███████▋  | 174322/225000 [332:55:56<95:34:02,  6.79s/it]
 77%|███████▋  | 174323/225000 [332:56:03<96:34:41,  6.86s/it]
 77%|███████▋  | 174324/225000 [332:56:08<88:20:12,  6.28s/it]
 77%|███████▋  | 174325/225000 [332:56:14<87:22:41,  6.21s/it]
 77%|███████▋  | 174326/225000 [332:56:18<78:18:39,  5.56s/it]
 77%|███████▋  | 174327/225000 [332:56:22<71:43:49,  5.10s/it]
 77%|███████▋  | 174328/225000 [332:56:26<68:41:07,  4.88s/it]
 77%|███████▋  | 174329/225000 [332:56:31<67:55:41,  4.83s/it]
 77%|███████▋  | 174330/225000 [332:56:42<94:39:39,  6.73s/it]
 77%|███████▋  | 174331/225000 [332:56:46<81:37:04,  5.80s/it]
 77%|███████▋  | 174332/225000 [332:56:50<73:22:25,  5.21s/it]
 77%|███████▋  | 174333/225000 [332:56:56<77:09:32,  5.48s/it]
 77%|███████▋  | 174334/225000 [332:57:03<84:39:43,  6.02s/it]
 77%|███████▋  | 174335/225000 [332:57:07<73:57:50,  5.26s/it]
 77%|███████▋  | 174336/225000 [332:57:13<80:30:38,  5.72s/it]
 77%|███████▋  | 174337/225000 [332:57:21<86:21:17,  6.14s/it]
 77%|███████▋  | 174338/225000 [332:57:24<75:09:42,  5.34s/it]
 77%|███████▋  | 174339/225000 [332:57:31<81:51:25,  5.82s/it]
 77%|███████▋  | 174340/225000 [332:57:36<77:39:24,  5.52s/it]
 77%|███████▋  | 174341/225000 [332:57:42<81:21:28,  5.78s/it]
 77%|███████▋  | 174342/225000 [332:57:48<82:51:58,  5.89s/it]
 77%|███████▋  | 174343/225000 [332:57:54<81:29:05,  5.79s/it]
 77%|███████▋  | 174344/225000 [332:57:59<77:52:49,  5.53s/it]
 77%|███████▋  | 174345/225000 [332:58:04<77:38:59,  5.52s/it]
 77%|███████▋  | 174346/225000 [332:58:09<75:52:03,  5.39s/it]
 77%|███████▋  | 174347/225000 [332:58:14<74:34:40,  5.30s/it]
 77%|███████▋  | 174348/225000 [332:58:19<73:05:48,  5.20s/it]
 77%|███████▋  | 174349/225000 [332:58:25<73:00:36,  5.19s/it]
 77%|███████▋  | 174350/225000 [332:58:29<71:32:16,  5.08s/it]
                                                              
{'loss': '0.7735', 'grad_norm': '1.221', 'learning_rate': '6.56e-06', 'epoch': '0.7749', 'num_input_tokens_seen': 551859968, 'train_runtime': '1.199e+06', 'train_tokens_per_second': '460.4'}

 77%|███████▋  | 174350/225000 [332:58:29<71:32:16,  5.08s/it]
 77%|███████▋  | 174351/225000 [332:58:34<69:12:45,  4.92s/it]
 77%|███████▋  | 174352/225000 [332:58:39<71:04:23,  5.05s/it]
 77%|███████▋  | 174353/225000 [332:58:44<70:09:59,  4.99s/it]
 77%|███████▋  | 174354/225000 [332:58:49<69:53:53,  4.97s/it]
 77%|███████▋  | 174355/225000 [332:58:54<71:22:48,  5.07s/it]
 77%|███████▋  | 174356/225000 [332:58:59<71:15:37,  5.07s/it]
 77%|███████▋  | 174357/225000 [332:59:05<71:47:58,  5.10s/it]
 77%|███████▋  | 174358/225000 [332:59:10<71:13:03,  5.06s/it]
 77%|███████▋  | 174359/225000 [332:59:22<103:56:05,  7.39s/it]
 77%|███████▋  | 174360/225000 [332:59:26<85:50:41,  6.10s/it] 
 77%|███████▋  | 174361/225000 [332:59:32<87:07:49,  6.19s/it]
 77%|███████▋  | 174362/225000 [332:59:36<76:55:43,  5.47s/it]
 77%|███████▋  | 174363/225000 [332:59:44<87:22:19,  6.21s/it]
 77%|███████▋  | 174364/225000 [332:59:49<83:05:50,  5.91s/it]
 77%|███████▋  | 174365/225000 [332:59:55<83:15:10,  5.92s/it]
 77%|███████▋  | 174366/225000 [332:59:59<77:46:00,  5.53s/it]
 77%|███████▋  | 174367/225000 [333:00:04<73:40:43,  5.24s/it]
 77%|███████▋  | 174368/225000 [333:00:09<72:22:53,  5.15s/it]
 77%|███████▋  | 174369/225000 [333:00:14<71:00:39,  5.05s/it]
 77%|███████▋  | 174370/225000 [333:00:18<69:41:00,  4.95s/it]
 77%|███████▋  | 174371/225000 [333:00:23<69:04:18,  4.91s/it]
 77%|███████▋  | 174372/225000 [333:00:28<69:00:16,  4.91s/it]
 77%|███████▋  | 174373/225000 [333:00:33<68:19:10,  4.86s/it]
 77%|███████▋  | 174374/225000 [333:00:38<69:47:39,  4.96s/it]
 78%|███████▊  | 174375/225000 [333:00:43<71:07:18,  5.06s/it]
 78%|███████▊  | 174376/225000 [333:00:55<96:59:36,  6.90s/it]
 78%|███████▊  | 174377/225000 [333:01:00<91:37:27,  6.52s/it]
 78%|███████▊  | 174378/225000 [333:01:04<78:17:14,  5.57s/it]
 78%|███████▊  | 174379/225000 [333:01:11<84:53:44,  6.04s/it]
 78%|███████▊  | 174380/225000 [333:01:17<85:58:42,  6.11s/it]
 78%|███████▊  | 174381/225000 [333:01:21<77:18:33,  5.50s/it]
 78%|███████▊  | 174382/225000 [333:01:26<74:46:58,  5.32s/it]
 78%|███████▊  | 174383/225000 [333:01:31<73:27:45,  5.22s/it]
 78%|███████▊  | 174384/225000 [333:01:36<72:00:07,  5.12s/it]
 78%|███████▊  | 174385/225000 [333:01:41<71:07:12,  5.06s/it]
 78%|███████▊  | 174386/225000 [333:01:46<70:56:29,  5.05s/it]
 78%|███████▊  | 174387/225000 [333:01:51<69:42:11,  4.96s/it]
 78%|███████▊  | 174388/225000 [333:01:55<68:11:28,  4.85s/it]
 78%|███████▊  | 174389/225000 [333:02:00<67:21:50,  4.79s/it]
 78%|███████▊  | 174390/225000 [333:02:12<97:03:30,  6.90s/it]
 78%|███████▊  | 174391/225000 [333:02:16<87:28:18,  6.22s/it]
 78%|███████▊  | 174392/225000 [333:02:22<86:37:32,  6.16s/it]
 78%|███████▊  | 174393/225000 [333:02:26<75:36:21,  5.38s/it]
 78%|███████▊  | 174394/225000 [333:02:33<81:05:42,  5.77s/it]
 78%|███████▊  | 174395/225000 [333:02:40<89:20:42,  6.36s/it]
 78%|███████▊  | 174396/225000 [333:02:44<77:34:49,  5.52s/it]
 78%|███████▊  | 174397/225000 [333:02:50<78:50:28,  5.61s/it]
 78%|███████▊  | 174398/225000 [333:02:55<77:28:35,  5.51s/it]
 78%|███████▊  | 174399/225000 [333:02:59<70:01:46,  4.98s/it]
 78%|███████▊  | 174400/225000 [333:03:03<66:14:27,  4.71s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.299', 'learning_rate': '6.548e-06', 'epoch': '0.7751', 'num_input_tokens_seen': 551989608, 'train_runtime': '1.199e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174400/225000 [333:03:03<66:14:27,  4.71s/it]
 78%|███████▊  | 174401/225000 [333:03:07<65:52:38,  4.69s/it]
 78%|███████▊  | 174402/225000 [333:03:18<91:57:20,  6.54s/it]
 78%|███████▊  | 174403/225000 [333:03:22<82:11:32,  5.85s/it]
 78%|███████▊  | 174404/225000 [333:03:26<73:04:18,  5.20s/it]
 78%|███████▊  | 174405/225000 [333:03:34<84:07:17,  5.99s/it]
 78%|███████▊  | 174406/225000 [333:03:38<75:44:13,  5.39s/it]
 78%|███████▊  | 174407/225000 [333:03:42<69:23:45,  4.94s/it]
 78%|███████▊  | 174408/225000 [333:03:46<67:47:19,  4.82s/it]
 78%|███████▊  | 174409/225000 [333:03:56<87:26:03,  6.22s/it]
 78%|███████▊  | 174410/225000 [333:03:59<75:07:31,  5.35s/it]
 78%|███████▊  | 174411/225000 [333:04:03<67:13:43,  4.78s/it]
 78%|███████▊  | 174412/225000 [333:04:12<86:49:51,  6.18s/it]
 78%|███████▊  | 174413/225000 [333:04:15<74:40:37,  5.31s/it]
 78%|███████▊  | 174414/225000 [333:04:21<73:59:39,  5.27s/it]
 78%|███████▊  | 174415/225000 [333:04:25<72:31:20,  5.16s/it]
 78%|███████▊  | 174416/225000 [333:04:31<72:36:48,  5.17s/it]
 78%|███████▊  | 174417/225000 [333:04:36<71:23:18,  5.08s/it]
 78%|███████▊  | 174418/225000 [333:04:41<71:04:46,  5.06s/it]
 78%|███████▊  | 174419/225000 [333:04:45<70:31:50,  5.02s/it]
 78%|███████▊  | 174420/225000 [333:04:50<67:48:41,  4.83s/it]
 78%|███████▊  | 174421/225000 [333:04:55<67:33:49,  4.81s/it]
 78%|███████▊  | 174422/225000 [333:04:59<67:01:21,  4.77s/it]
 78%|███████▊  | 174423/225000 [333:05:04<67:08:54,  4.78s/it]
 78%|███████▊  | 174424/225000 [333:05:09<68:21:32,  4.87s/it]
 78%|███████▊  | 174425/225000 [333:05:14<69:34:39,  4.95s/it]
 78%|███████▊  | 174426/225000 [333:05:19<69:00:45,  4.91s/it]
 78%|███████▊  | 174427/225000 [333:05:25<72:59:03,  5.20s/it]
 78%|███████▊  | 174428/225000 [333:05:37<101:02:03,  7.19s/it]
 78%|███████▊  | 174429/225000 [333:05:40<85:51:02,  6.11s/it] 
 78%|███████▊  | 174430/225000 [333:05:48<91:21:23,  6.50s/it]
 78%|███████▊  | 174431/225000 [333:05:51<78:10:20,  5.57s/it]
 78%|███████▊  | 174432/225000 [333:05:55<69:47:58,  4.97s/it]
 78%|███████▊  | 174433/225000 [333:06:01<76:11:59,  5.42s/it]
 78%|███████▊  | 174434/225000 [333:06:09<84:49:26,  6.04s/it]
 78%|███████▊  | 174435/225000 [333:06:13<76:28:39,  5.44s/it]
 78%|███████▊  | 174436/225000 [333:06:17<71:40:07,  5.10s/it]
 78%|███████▊  | 174437/225000 [333:06:21<68:23:27,  4.87s/it]
 78%|███████▊  | 174438/225000 [333:06:26<68:05:56,  4.85s/it]
 78%|███████▊  | 174439/225000 [333:06:33<76:46:53,  5.47s/it]
 78%|███████▊  | 174440/225000 [333:06:42<88:59:47,  6.34s/it]
 78%|███████▊  | 174441/225000 [333:06:45<77:38:51,  5.53s/it]
 78%|███████▊  | 174442/225000 [333:06:51<79:49:00,  5.68s/it]
 78%|███████▊  | 174443/225000 [333:07:00<92:08:15,  6.56s/it]
 78%|███████▊  | 174444/225000 [333:07:03<79:11:42,  5.64s/it]
 78%|███████▊  | 174445/225000 [333:07:07<70:10:39,  5.00s/it]
 78%|███████▊  | 174446/225000 [333:07:15<84:54:12,  6.05s/it]
 78%|███████▊  | 174447/225000 [333:07:19<74:39:06,  5.32s/it]
 78%|███████▊  | 174448/225000 [333:07:23<68:51:33,  4.90s/it]
 78%|███████▊  | 174449/225000 [333:07:27<65:05:20,  4.64s/it]
 78%|███████▊  | 174450/225000 [333:07:32<66:01:12,  4.70s/it]
                                                              
{'loss': '0.7668', 'grad_norm': '1.221', 'learning_rate': '6.535e-06', 'epoch': '0.7753', 'num_input_tokens_seen': 552118264, 'train_runtime': '1.199e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174450/225000 [333:07:32<66:01:12,  4.70s/it]
 78%|███████▊  | 174451/225000 [333:07:41<87:21:14,  6.22s/it]
 78%|███████▊  | 174452/225000 [333:07:45<75:33:35,  5.38s/it]
 78%|███████▊  | 174453/225000 [333:07:49<69:21:45,  4.94s/it]
 78%|███████▊  | 174454/225000 [333:07:55<76:04:44,  5.42s/it]
 78%|███████▊  | 174455/225000 [333:08:02<80:25:30,  5.73s/it]
 78%|███████▊  | 174456/225000 [333:08:07<79:20:50,  5.65s/it]
 78%|███████▊  | 174457/225000 [333:08:15<88:54:32,  6.33s/it]
 78%|███████▊  | 174458/225000 [333:08:19<77:26:18,  5.52s/it]
 78%|███████▊  | 174459/225000 [333:08:22<67:57:01,  4.84s/it]
 78%|███████▊  | 174460/225000 [333:08:31<86:51:01,  6.19s/it]
 78%|███████▊  | 174461/225000 [333:08:35<76:38:50,  5.46s/it]
 78%|███████▊  | 174462/225000 [333:08:39<68:16:00,  4.86s/it]
 78%|███████▊  | 174463/225000 [333:08:48<87:37:23,  6.24s/it]
 78%|███████▊  | 174464/225000 [333:08:52<77:11:08,  5.50s/it]
 78%|███████▊  | 174465/225000 [333:08:55<69:02:50,  4.92s/it]
 78%|███████▊  | 174466/225000 [333:09:00<65:37:22,  4.67s/it]
 78%|███████▊  | 174467/225000 [333:09:08<83:27:41,  5.95s/it]
 78%|███████▊  | 174468/225000 [333:09:13<78:39:43,  5.60s/it]
 78%|███████▊  | 174469/225000 [333:09:18<74:39:34,  5.32s/it]
 78%|███████▊  | 174470/225000 [333:09:22<71:06:28,  5.07s/it]
 78%|███████▊  | 174471/225000 [333:09:27<70:51:37,  5.05s/it]
 78%|███████▊  | 174472/225000 [333:09:32<70:33:06,  5.03s/it]
 78%|███████▊  | 174473/225000 [333:09:37<68:10:25,  4.86s/it]
 78%|███████▊  | 174474/225000 [333:09:42<70:06:43,  5.00s/it]
 78%|███████▊  | 174475/225000 [333:09:47<69:23:09,  4.94s/it]
 78%|███████▊  | 174476/225000 [333:09:52<68:56:41,  4.91s/it]
 78%|███████▊  | 174477/225000 [333:09:57<68:21:41,  4.87s/it]
 78%|███████▊  | 174478/225000 [333:10:01<67:58:55,  4.84s/it]
 78%|███████▊  | 174479/225000 [333:10:07<71:12:16,  5.07s/it]
 78%|███████▊  | 174480/225000 [333:10:19<100:27:52,  7.16s/it]
 78%|███████▊  | 174481/225000 [333:10:23<85:12:06,  6.07s/it] 
 78%|███████▊  | 174482/225000 [333:10:29<88:15:14,  6.29s/it]
 78%|███████▊  | 174483/225000 [333:10:33<76:41:39,  5.47s/it]
 78%|███████▊  | 174484/225000 [333:10:41<86:45:43,  6.18s/it]
 78%|███████▊  | 174485/225000 [333:10:45<77:33:29,  5.53s/it]
 78%|███████▊  | 174486/225000 [333:10:49<71:16:58,  5.08s/it]
 78%|███████▊  | 174487/225000 [333:10:59<91:40:43,  6.53s/it]
 78%|███████▊  | 174488/225000 [333:11:02<80:12:06,  5.72s/it]
 78%|███████▊  | 174489/225000 [333:11:06<70:43:13,  5.04s/it]
 78%|███████▊  | 174490/225000 [333:11:15<86:21:08,  6.15s/it]
 78%|███████▊  | 174491/225000 [333:11:18<74:58:53,  5.34s/it]
 78%|███████▊  | 174492/225000 [333:11:26<83:26:25,  5.95s/it]
 78%|███████▊  | 174493/225000 [333:11:29<71:54:56,  5.13s/it]
 78%|███████▊  | 174494/225000 [333:11:33<69:54:32,  4.98s/it]
 78%|███████▊  | 174495/225000 [333:11:38<68:20:06,  4.87s/it]
 78%|███████▊  | 174496/225000 [333:11:43<68:30:56,  4.88s/it]
 78%|███████▊  | 174497/225000 [333:11:48<67:55:58,  4.84s/it]
 78%|███████▊  | 174498/225000 [333:11:53<68:49:31,  4.91s/it]
 78%|███████▊  | 174499/225000 [333:11:58<68:41:52,  4.90s/it]
 78%|███████▊  | 174500/225000 [333:12:03<70:39:44,  5.04s/it]
                                                              
{'loss': '0.7654', 'grad_norm': '1.099', 'learning_rate': '6.523e-06', 'epoch': '0.7756', 'num_input_tokens_seen': 552248336, 'train_runtime': '1.2e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174500/225000 [333:12:03<70:39:44,  5.04s/it]
 78%|███████▊  | 174501/225000 [333:12:08<70:38:18,  5.04s/it]
 78%|███████▊  | 174502/225000 [333:12:13<71:47:16,  5.12s/it]
 78%|███████▊  | 174503/225000 [333:12:26<104:23:42,  7.44s/it]
 78%|███████▊  | 174504/225000 [333:12:30<88:30:23,  6.31s/it] 
 78%|███████▊  | 174505/225000 [333:12:35<85:40:47,  6.11s/it]
 78%|███████▊  | 174506/225000 [333:12:42<86:59:07,  6.20s/it]
 78%|███████▊  | 174507/225000 [333:12:46<79:51:41,  5.69s/it]
 78%|███████▊  | 174508/225000 [333:12:53<84:54:13,  6.05s/it]
 78%|███████▊  | 174509/225000 [333:12:58<81:04:48,  5.78s/it]
 78%|███████▊  | 174510/225000 [333:13:02<72:04:51,  5.14s/it]
 78%|███████▊  | 174511/225000 [333:13:06<66:48:33,  4.76s/it]
 78%|███████▊  | 174512/225000 [333:13:14<81:36:52,  5.82s/it]
 78%|███████▊  | 174513/225000 [333:13:20<81:39:35,  5.82s/it]
 78%|███████▊  | 174514/225000 [333:13:25<77:59:15,  5.56s/it]
 78%|███████▊  | 174515/225000 [333:13:30<74:23:32,  5.30s/it]
 78%|███████▊  | 174516/225000 [333:13:34<72:01:07,  5.14s/it]
 78%|███████▊  | 174517/225000 [333:13:39<70:08:16,  5.00s/it]
 78%|███████▊  | 174518/225000 [333:13:44<69:38:50,  4.97s/it]
 78%|███████▊  | 174519/225000 [333:13:49<69:17:27,  4.94s/it]
 78%|███████▊  | 174520/225000 [333:13:54<68:17:08,  4.87s/it]
 78%|███████▊  | 174521/225000 [333:13:59<70:17:04,  5.01s/it]
 78%|███████▊  | 174522/225000 [333:14:03<67:03:08,  4.78s/it]
 78%|███████▊  | 174523/225000 [333:14:08<67:38:32,  4.82s/it]
 78%|███████▊  | 174524/225000 [333:14:16<81:38:03,  5.82s/it]
 78%|███████▊  | 174525/225000 [333:14:26<97:35:14,  6.96s/it]
 78%|███████▊  | 174526/225000 [333:14:30<83:56:32,  5.99s/it]
 78%|███████▊  | 174527/225000 [333:14:33<74:42:13,  5.33s/it]
 78%|███████▊  | 174528/225000 [333:14:43<94:24:33,  6.73s/it]
 78%|███████▊  | 174529/225000 [333:14:47<79:56:09,  5.70s/it]
 78%|███████▊  | 174530/225000 [333:14:54<84:40:41,  6.04s/it]
 78%|███████▊  | 174531/225000 [333:14:58<76:04:20,  5.43s/it]
 78%|███████▊  | 174532/225000 [333:15:01<68:57:45,  4.92s/it]
 78%|███████▊  | 174533/225000 [333:15:05<64:58:03,  4.63s/it]
 78%|███████▊  | 174534/225000 [333:15:16<91:28:44,  6.53s/it]
 78%|███████▊  | 174535/225000 [333:15:19<77:57:05,  5.56s/it]
 78%|███████▊  | 174536/225000 [333:15:26<82:42:46,  5.90s/it]
 78%|███████▊  | 174537/225000 [333:15:29<71:32:47,  5.10s/it]
 78%|███████▊  | 174538/225000 [333:15:37<83:02:36,  5.92s/it]
 78%|███████▊  | 174539/225000 [333:15:41<73:28:13,  5.24s/it]
 78%|███████▊  | 174540/225000 [333:15:45<67:32:05,  4.82s/it]
 78%|███████▊  | 174541/225000 [333:15:53<82:32:55,  5.89s/it]
 78%|███████▊  | 174542/225000 [333:15:57<72:01:33,  5.14s/it]
 78%|███████▊  | 174543/225000 [333:16:00<65:49:02,  4.70s/it]
 78%|███████▊  | 174544/225000 [333:16:09<82:09:37,  5.86s/it]
 78%|███████▊  | 174545/225000 [333:16:12<72:25:44,  5.17s/it]
 78%|███████▊  | 174546/225000 [333:16:20<81:14:01,  5.80s/it]
 78%|███████▊  | 174547/225000 [333:16:25<78:38:44,  5.61s/it]
 78%|███████▊  | 174548/225000 [333:16:32<86:42:48,  6.19s/it]
 78%|███████▊  | 174549/225000 [333:16:36<77:14:45,  5.51s/it]
 78%|███████▊  | 174550/225000 [333:16:44<85:00:53,  6.07s/it]
                                                              
{'loss': '0.7697', 'grad_norm': '1.123', 'learning_rate': '6.511e-06', 'epoch': '0.7758', 'num_input_tokens_seen': 552378600, 'train_runtime': '1.2e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174550/225000 [333:16:44<85:00:53,  6.07s/it]
 78%|███████▊  | 174551/225000 [333:16:47<75:49:35,  5.41s/it]
 78%|███████▊  | 174552/225000 [333:16:52<70:26:05,  5.03s/it]
 78%|███████▊  | 174553/225000 [333:16:56<66:45:23,  4.76s/it]
 78%|███████▊  | 174554/225000 [333:17:04<79:39:33,  5.68s/it]
 78%|███████▊  | 174555/225000 [333:17:10<80:51:05,  5.77s/it]
 78%|███████▊  | 174556/225000 [333:17:13<71:51:56,  5.13s/it]
 78%|███████▊  | 174557/225000 [333:17:17<66:03:05,  4.71s/it]
 78%|███████▊  | 174558/225000 [333:17:26<86:05:27,  6.14s/it]
 78%|███████▊  | 174559/225000 [333:17:30<74:56:30,  5.35s/it]
 78%|███████▊  | 174560/225000 [333:17:38<86:53:04,  6.20s/it]
 78%|███████▊  | 174561/225000 [333:17:42<76:02:31,  5.43s/it]
 78%|███████▊  | 174562/225000 [333:17:46<69:32:31,  4.96s/it]
 78%|███████▊  | 174563/225000 [333:17:54<83:38:57,  5.97s/it]
 78%|███████▊  | 174564/225000 [333:17:59<79:49:19,  5.70s/it]
 78%|███████▊  | 174565/225000 [333:18:02<70:00:30,  5.00s/it]
 78%|███████▊  | 174566/225000 [333:18:12<88:14:06,  6.30s/it]
 78%|███████▊  | 174567/225000 [333:18:15<76:29:06,  5.46s/it]
 78%|███████▊  | 174568/225000 [333:18:23<84:57:53,  6.07s/it]
 78%|███████▊  | 174569/225000 [333:18:28<80:36:06,  5.75s/it]
 78%|███████▊  | 174570/225000 [333:18:33<79:02:37,  5.64s/it]
 78%|███████▊  | 174571/225000 [333:18:37<70:07:32,  5.01s/it]
 78%|███████▊  | 174572/225000 [333:18:41<69:26:10,  4.96s/it]
 78%|███████▊  | 174573/225000 [333:18:46<68:54:48,  4.92s/it]
 78%|███████▊  | 174574/225000 [333:18:51<69:24:38,  4.96s/it]
 78%|███████▊  | 174575/225000 [333:18:56<68:10:41,  4.87s/it]
 78%|███████▊  | 174576/225000 [333:19:01<68:12:48,  4.87s/it]
 78%|███████▊  | 174577/225000 [333:19:06<68:59:46,  4.93s/it]
 78%|███████▊  | 174578/225000 [333:19:11<70:24:18,  5.03s/it]
 78%|███████▊  | 174579/225000 [333:19:16<70:07:40,  5.01s/it]
 78%|███████▊  | 174580/225000 [333:19:26<88:55:34,  6.35s/it]
 78%|███████▊  | 174581/225000 [333:19:33<92:56:41,  6.64s/it]
 78%|███████▊  | 174582/225000 [333:19:37<82:04:47,  5.86s/it]
 78%|███████▊  | 174583/225000 [333:19:41<73:39:34,  5.26s/it]
 78%|███████▊  | 174584/225000 [333:19:52<97:36:01,  6.97s/it]
 78%|███████▊  | 174585/225000 [333:19:55<82:54:35,  5.92s/it]
 78%|███████▊  | 174586/225000 [333:20:00<78:01:29,  5.57s/it]
 78%|███████▊  | 174587/225000 [333:20:09<92:19:36,  6.59s/it]
 78%|███████▊  | 174588/225000 [333:20:12<78:51:05,  5.63s/it]
 78%|███████▊  | 174589/225000 [333:20:20<88:11:28,  6.30s/it]
 78%|███████▊  | 174590/225000 [333:20:24<77:44:46,  5.55s/it]
 78%|███████▊  | 174591/225000 [333:20:32<89:00:07,  6.36s/it]
 78%|███████▊  | 174592/225000 [333:20:35<75:33:59,  5.40s/it]
 78%|███████▊  | 174593/225000 [333:20:42<81:43:53,  5.84s/it]
 78%|███████▊  | 174594/225000 [333:20:46<71:35:46,  5.11s/it]
 78%|███████▊  | 174595/225000 [333:20:50<66:28:35,  4.75s/it]
 78%|███████▊  | 174596/225000 [333:20:54<64:25:39,  4.60s/it]
 78%|███████▊  | 174597/225000 [333:21:00<70:03:00,  5.00s/it]
 78%|███████▊  | 174598/225000 [333:21:10<89:46:32,  6.41s/it]
 78%|███████▊  | 174599/225000 [333:21:14<81:29:47,  5.82s/it]
 78%|███████▊  | 174600/225000 [333:21:18<74:44:55,  5.34s/it]
                                                              
{'loss': '0.774', 'grad_norm': '1.203', 'learning_rate': '6.499e-06', 'epoch': '0.776', 'num_input_tokens_seen': 552509408, 'train_runtime': '1.2e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174600/225000 [333:21:18<74:44:55,  5.34s/it]
 78%|███████▊  | 174601/225000 [333:21:23<70:50:49,  5.06s/it]
 78%|███████▊  | 174602/225000 [333:21:27<68:21:20,  4.88s/it]
 78%|███████▊  | 174603/225000 [333:21:37<89:16:23,  6.38s/it]
 78%|███████▊  | 174604/225000 [333:21:41<81:44:48,  5.84s/it]
 78%|███████▊  | 174605/225000 [333:21:47<78:31:05,  5.61s/it]
 78%|███████▊  | 174606/225000 [333:21:53<83:40:32,  5.98s/it]
 78%|███████▊  | 174607/225000 [333:21:57<73:08:17,  5.22s/it]
 78%|███████▊  | 174608/225000 [333:22:05<84:19:05,  6.02s/it]
 78%|███████▊  | 174609/225000 [333:22:09<76:42:04,  5.48s/it]
 78%|███████▊  | 174610/225000 [333:22:13<70:51:27,  5.06s/it]
 78%|███████▊  | 174611/225000 [333:22:17<66:16:45,  4.74s/it]
 78%|███████▊  | 174612/225000 [333:22:21<64:05:54,  4.58s/it]
 78%|███████▊  | 174613/225000 [333:22:30<81:52:51,  5.85s/it]
 78%|███████▊  | 174614/225000 [333:22:36<83:38:39,  5.98s/it]
 78%|███████▊  | 174615/225000 [333:22:40<75:03:36,  5.36s/it]
 78%|███████▊  | 174616/225000 [333:22:44<68:43:48,  4.91s/it]
 78%|███████▊  | 174617/225000 [333:22:51<78:12:20,  5.59s/it]
 78%|███████▊  | 174618/225000 [333:22:58<83:37:00,  5.97s/it]
 78%|███████▊  | 174619/225000 [333:23:01<71:51:07,  5.13s/it]
 78%|███████▊  | 174620/225000 [333:23:09<81:02:25,  5.79s/it]
 78%|███████▊  | 174621/225000 [333:23:13<75:47:51,  5.42s/it]
 78%|███████▊  | 174622/225000 [333:23:18<74:11:36,  5.30s/it]
 78%|███████▊  | 174623/225000 [333:23:23<73:56:34,  5.28s/it]
 78%|███████▊  | 174624/225000 [333:23:32<88:12:53,  6.30s/it]
 78%|███████▊  | 174625/225000 [333:23:36<79:23:13,  5.67s/it]
 78%|███████▊  | 174626/225000 [333:23:40<71:41:08,  5.12s/it]
 78%|███████▊  | 174627/225000 [333:23:48<84:36:40,  6.05s/it]
 78%|███████▊  | 174628/225000 [333:23:52<73:05:21,  5.22s/it]
 78%|███████▊  | 174629/225000 [333:23:55<66:15:34,  4.74s/it]
 78%|███████▊  | 174630/225000 [333:24:00<67:20:00,  4.81s/it]
 78%|███████▊  | 174631/225000 [333:24:05<68:16:01,  4.88s/it]
 78%|███████▊  | 174632/225000 [333:24:10<68:09:26,  4.87s/it]
 78%|███████▊  | 174633/225000 [333:24:15<69:05:25,  4.94s/it]
 78%|███████▊  | 174634/225000 [333:24:21<70:54:14,  5.07s/it]
 78%|███████▊  | 174635/225000 [333:24:26<71:10:13,  5.09s/it]
 78%|███████▊  | 174636/225000 [333:24:36<91:37:58,  6.55s/it]
 78%|███████▊  | 174637/225000 [333:24:43<95:45:14,  6.84s/it]
 78%|███████▊  | 174638/225000 [333:24:51<97:42:19,  6.98s/it]
 78%|███████▊  | 174639/225000 [333:24:54<82:18:55,  5.88s/it]
 78%|███████▊  | 174640/225000 [333:25:00<84:09:59,  6.02s/it]
 78%|███████▊  | 174641/225000 [333:25:06<81:12:44,  5.81s/it]
 78%|███████▊  | 174642/225000 [333:25:11<78:43:23,  5.63s/it]
 78%|███████▊  | 174643/225000 [333:25:16<77:13:55,  5.52s/it]
 78%|███████▊  | 174644/225000 [333:25:21<76:59:52,  5.50s/it]
 78%|███████▊  | 174645/225000 [333:25:27<75:10:05,  5.37s/it]
 78%|███████▊  | 174646/225000 [333:25:32<74:45:01,  5.34s/it]
 78%|███████▊  | 174647/225000 [333:25:36<71:33:12,  5.12s/it]
 78%|███████▊  | 174648/225000 [333:25:41<69:03:38,  4.94s/it]
 78%|███████▊  | 174649/225000 [333:25:46<70:35:32,  5.05s/it]
 78%|███████▊  | 174650/225000 [333:25:51<69:11:16,  4.95s/it]
                                                              
{'loss': '0.7606', 'grad_norm': '1.147', 'learning_rate': '6.486e-06', 'epoch': '0.7762', 'num_input_tokens_seen': 552640152, 'train_runtime': '1.2e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174650/225000 [333:25:51<69:11:16,  4.95s/it]
 78%|███████▊  | 174651/225000 [333:25:56<70:01:14,  5.01s/it]
 78%|███████▊  | 174652/225000 [333:26:01<71:18:14,  5.10s/it]
 78%|███████▊  | 174653/225000 [333:26:07<71:41:47,  5.13s/it]
 78%|███████▊  | 174654/225000 [333:26:12<71:43:47,  5.13s/it]
 78%|███████▊  | 174655/225000 [333:26:17<72:02:13,  5.15s/it]
 78%|███████▊  | 174656/225000 [333:26:23<74:55:53,  5.36s/it]
 78%|███████▊  | 174657/225000 [333:26:28<73:45:20,  5.27s/it]
 78%|███████▊  | 174658/225000 [333:26:33<73:22:30,  5.25s/it]
 78%|███████▊  | 174659/225000 [333:26:42<89:15:03,  6.38s/it]
 78%|███████▊  | 174660/225000 [333:26:52<105:38:31,  7.55s/it]
 78%|███████▊  | 174661/225000 [333:26:56<88:36:16,  6.34s/it] 
 78%|███████▊  | 174662/225000 [333:26:59<76:09:30,  5.45s/it]
 78%|███████▊  | 174663/225000 [333:27:09<94:32:59,  6.76s/it]
 78%|███████▊  | 174664/225000 [333:27:13<81:01:15,  5.79s/it]
 78%|███████▊  | 174665/225000 [333:27:16<72:27:18,  5.18s/it]
 78%|███████▊  | 174666/225000 [333:27:23<77:55:23,  5.57s/it]
 78%|███████▊  | 174667/225000 [333:27:30<85:48:39,  6.14s/it]
 78%|███████▊  | 174668/225000 [333:27:35<78:34:55,  5.62s/it]
 78%|███████▊  | 174669/225000 [333:27:39<72:51:48,  5.21s/it]
 78%|███████▊  | 174670/225000 [333:27:43<69:17:14,  4.96s/it]
 78%|███████▊  | 174671/225000 [333:27:48<67:01:32,  4.79s/it]
 78%|███████▊  | 174672/225000 [333:27:57<85:48:47,  6.14s/it]
 78%|███████▊  | 174673/225000 [333:28:04<89:35:09,  6.41s/it]
 78%|███████▊  | 174674/225000 [333:28:09<83:29:29,  5.97s/it]
 78%|███████▊  | 174675/225000 [333:28:14<79:34:58,  5.69s/it]
 78%|███████▊  | 174676/225000 [333:28:19<77:14:29,  5.53s/it]
 78%|███████▊  | 174677/225000 [333:28:24<74:04:16,  5.30s/it]
 78%|███████▊  | 174678/225000 [333:28:29<74:15:56,  5.31s/it]
 78%|███████▊  | 174679/225000 [333:28:35<74:15:18,  5.31s/it]
 78%|███████▊  | 174680/225000 [333:28:40<76:28:04,  5.47s/it]
 78%|███████▊  | 174681/225000 [333:28:46<75:04:22,  5.37s/it]
 78%|███████▊  | 174682/225000 [333:28:51<74:03:56,  5.30s/it]
 78%|███████▊  | 174683/225000 [333:28:56<72:33:59,  5.19s/it]
 78%|███████▊  | 174684/225000 [333:29:01<71:18:16,  5.10s/it]
 78%|███████▊  | 174685/225000 [333:29:06<70:52:59,  5.07s/it]
 78%|███████▊  | 174686/225000 [333:29:10<69:52:05,  5.00s/it]
 78%|███████▊  | 174687/225000 [333:29:15<69:47:24,  4.99s/it]
 78%|███████▊  | 174688/225000 [333:29:21<72:00:09,  5.15s/it]
 78%|███████▊  | 174689/225000 [333:29:26<71:17:55,  5.10s/it]
 78%|███████▊  | 174690/225000 [333:29:30<69:13:27,  4.95s/it]
 78%|███████▊  | 174691/225000 [333:29:35<68:43:26,  4.92s/it]
 78%|███████▊  | 174692/225000 [333:29:40<68:55:10,  4.93s/it]
 78%|███████▊  | 174693/225000 [333:29:45<69:46:06,  4.99s/it]
 78%|███████▊  | 174694/225000 [333:29:51<71:22:19,  5.11s/it]
 78%|███████▊  | 174695/225000 [333:29:56<70:53:29,  5.07s/it]
 78%|███████▊  | 174696/225000 [333:30:01<72:27:30,  5.19s/it]
 78%|███████▊  | 174697/225000 [333:30:06<72:43:46,  5.20s/it]
 78%|███████▊  | 174698/225000 [333:30:13<79:41:02,  5.70s/it]
 78%|███████▊  | 174699/225000 [333:30:25<105:04:02,  7.52s/it]
 78%|███████▊  | 174700/225000 [333:30:29<90:27:13,  6.47s/it] 
                                                              
{'loss': '0.7664', 'grad_norm': '1.241', 'learning_rate': '6.474e-06', 'epoch': '0.7764', 'num_input_tokens_seen': 552770264, 'train_runtime': '1.201e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174700/225000 [333:30:29<90:27:13,  6.47s/it]
 78%|███████▊  | 174701/225000 [333:30:33<80:21:27,  5.75s/it]
 78%|███████▊  | 174702/225000 [333:30:37<72:14:57,  5.17s/it]
 78%|███████▊  | 174703/225000 [333:30:47<92:19:19,  6.61s/it]
 78%|███████▊  | 174704/225000 [333:30:51<81:19:44,  5.82s/it]
 78%|███████▊  | 174705/225000 [333:30:55<75:14:26,  5.39s/it]
 78%|███████▊  | 174706/225000 [333:31:00<71:24:10,  5.11s/it]
 78%|███████▊  | 174707/225000 [333:31:04<68:39:14,  4.91s/it]
 78%|███████▊  | 174708/225000 [333:31:15<91:21:50,  6.54s/it]
 78%|███████▊  | 174709/225000 [333:31:20<87:26:48,  6.26s/it]
 78%|███████▊  | 174710/225000 [333:31:24<76:02:50,  5.44s/it]
 78%|███████▊  | 174711/225000 [333:31:28<73:07:47,  5.24s/it]
 78%|███████▊  | 174712/225000 [333:31:34<73:17:34,  5.25s/it]
 78%|███████▊  | 174713/225000 [333:31:39<73:57:38,  5.29s/it]
 78%|███████▊  | 174714/225000 [333:31:45<75:35:29,  5.41s/it]
 78%|███████▊  | 174715/225000 [333:31:52<81:31:17,  5.84s/it]
 78%|███████▊  | 174716/225000 [333:31:57<79:30:11,  5.69s/it]
 78%|███████▊  | 174717/225000 [333:32:02<76:19:08,  5.46s/it]
 78%|███████▊  | 174718/225000 [333:32:07<74:57:03,  5.37s/it]
 78%|███████▊  | 174719/225000 [333:32:13<75:42:38,  5.42s/it]
 78%|███████▊  | 174720/225000 [333:32:18<75:01:16,  5.37s/it]
 78%|███████▊  | 174721/225000 [333:32:23<72:21:18,  5.18s/it]
 78%|███████▊  | 174722/225000 [333:32:28<73:12:15,  5.24s/it]
 78%|███████▊  | 174723/225000 [333:32:33<73:53:13,  5.29s/it]
 78%|███████▊  | 174724/225000 [333:32:38<70:46:16,  5.07s/it]
 78%|███████▊  | 174725/225000 [333:32:43<70:47:39,  5.07s/it]
 78%|███████▊  | 174726/225000 [333:32:48<71:02:50,  5.09s/it]
 78%|███████▊  | 174727/225000 [333:32:53<69:45:36,  5.00s/it]
 78%|███████▊  | 174728/225000 [333:32:59<73:37:43,  5.27s/it]
 78%|███████▊  | 174729/225000 [333:33:04<72:34:17,  5.20s/it]
 78%|███████▊  | 174730/225000 [333:33:09<72:24:53,  5.19s/it]
 78%|███████▊  | 174731/225000 [333:33:16<81:26:01,  5.83s/it]
 78%|███████▊  | 174732/225000 [333:33:22<79:55:26,  5.72s/it]
 78%|███████▊  | 174733/225000 [333:33:30<90:59:37,  6.52s/it]
 78%|███████▊  | 174734/225000 [333:33:37<91:58:08,  6.59s/it]
 78%|███████▊  | 174735/225000 [333:33:41<79:33:49,  5.70s/it]
 78%|███████▊  | 174736/225000 [333:33:45<73:13:37,  5.24s/it]
 78%|███████▊  | 174737/225000 [333:33:50<73:38:17,  5.27s/it]
 78%|███████▊  | 174738/225000 [333:33:55<73:08:18,  5.24s/it]
 78%|███████▊  | 174739/225000 [333:34:01<73:06:47,  5.24s/it]
 78%|███████▊  | 174740/225000 [333:34:07<76:12:23,  5.46s/it]
 78%|███████▊  | 174741/225000 [333:34:14<85:52:44,  6.15s/it]
 78%|███████▊  | 174742/225000 [333:34:20<82:13:11,  5.89s/it]
 78%|███████▊  | 174743/225000 [333:34:28<91:58:20,  6.59s/it]
 78%|███████▊  | 174744/225000 [333:34:32<80:32:49,  5.77s/it]
 78%|███████▊  | 174745/225000 [333:34:39<88:22:14,  6.33s/it]
 78%|███████▊  | 174746/225000 [333:34:45<85:51:02,  6.15s/it]
 78%|███████▊  | 174747/225000 [333:34:49<75:33:34,  5.41s/it]
 78%|███████▊  | 174748/225000 [333:34:57<89:38:09,  6.42s/it]
 78%|███████▊  | 174749/225000 [333:35:03<87:29:36,  6.27s/it]
 78%|███████▊  | 174750/225000 [333:35:09<83:18:21,  5.97s/it]
                                                              
{'loss': '0.7722', 'grad_norm': '1.245', 'learning_rate': '6.462e-06', 'epoch': '0.7767', 'num_input_tokens_seen': 552901352, 'train_runtime': '1.201e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174750/225000 [333:35:09<83:18:21,  5.97s/it]
 78%|███████▊  | 174751/225000 [333:35:14<80:36:28,  5.78s/it]
 78%|███████▊  | 174752/225000 [333:35:19<76:39:34,  5.49s/it]
 78%|███████▊  | 174753/225000 [333:35:24<75:21:35,  5.40s/it]
 78%|███████▊  | 174754/225000 [333:35:29<74:46:53,  5.36s/it]
 78%|███████▊  | 174755/225000 [333:35:34<72:01:28,  5.16s/it]
 78%|███████▊  | 174756/225000 [333:35:39<70:44:00,  5.07s/it]
 78%|███████▊  | 174757/225000 [333:35:43<68:58:48,  4.94s/it]
 78%|███████▊  | 174758/225000 [333:35:49<70:41:20,  5.07s/it]
 78%|███████▊  | 174759/225000 [333:35:53<67:46:24,  4.86s/it]
 78%|███████▊  | 174760/225000 [333:35:58<67:16:15,  4.82s/it]
 78%|███████▊  | 174761/225000 [333:36:03<67:59:31,  4.87s/it]
 78%|███████▊  | 174762/225000 [333:36:08<68:37:45,  4.92s/it]
 78%|███████▊  | 174763/225000 [333:36:13<71:13:08,  5.10s/it]
 78%|███████▊  | 174764/225000 [333:36:25<97:57:03,  7.02s/it]
 78%|███████▊  | 174765/225000 [333:36:30<91:21:20,  6.55s/it]
 78%|███████▊  | 174766/225000 [333:36:35<82:37:10,  5.92s/it]
 78%|███████▊  | 174767/225000 [333:36:39<75:09:43,  5.39s/it]
 78%|███████▊  | 174768/225000 [333:36:50<97:04:39,  6.96s/it]
 78%|███████▊  | 174769/225000 [333:36:54<85:36:16,  6.14s/it]
 78%|███████▊  | 174770/225000 [333:36:58<76:27:25,  5.48s/it]
 78%|███████▊  | 174771/225000 [333:37:02<69:01:37,  4.95s/it]
 78%|███████▊  | 174772/225000 [333:37:06<66:20:41,  4.76s/it]
 78%|███████▊  | 174773/225000 [333:37:13<75:24:46,  5.41s/it]
 78%|███████▊  | 174774/225000 [333:37:21<86:42:23,  6.21s/it]
 78%|███████▊  | 174775/225000 [333:37:24<74:49:46,  5.36s/it]
 78%|███████▊  | 174776/225000 [333:37:28<67:28:03,  4.84s/it]
 78%|███████▊  | 174777/225000 [333:37:39<94:23:56,  6.77s/it]
 78%|███████▊  | 174778/225000 [333:37:42<79:08:46,  5.67s/it]
 78%|███████▊  | 174779/225000 [333:37:46<69:54:51,  5.01s/it]
 78%|███████▊  | 174780/225000 [333:37:50<68:43:15,  4.93s/it]
 78%|███████▊  | 174781/225000 [333:37:55<66:37:54,  4.78s/it]
 78%|███████▊  | 174782/225000 [333:38:00<66:54:20,  4.80s/it]
 78%|███████▊  | 174783/225000 [333:38:04<64:35:33,  4.63s/it]
 78%|███████▊  | 174784/225000 [333:38:12<80:00:33,  5.74s/it]
 78%|███████▊  | 174785/225000 [333:38:20<87:59:55,  6.31s/it]
 78%|███████▊  | 174786/225000 [333:38:23<75:13:15,  5.39s/it]
 78%|███████▊  | 174787/225000 [333:38:31<84:19:11,  6.05s/it]
 78%|███████▊  | 174788/225000 [333:38:35<76:58:25,  5.52s/it]
 78%|███████▊  | 174789/225000 [333:38:39<71:54:25,  5.16s/it]
 78%|███████▊  | 174790/225000 [333:38:43<67:46:32,  4.86s/it]
 78%|███████▊  | 174791/225000 [333:38:48<67:02:59,  4.81s/it]
 78%|███████▊  | 174792/225000 [333:38:53<66:14:18,  4.75s/it]
 78%|███████▊  | 174793/225000 [333:39:02<85:27:08,  6.13s/it]
 78%|███████▊  | 174794/225000 [333:39:08<82:45:17,  5.93s/it]
 78%|███████▊  | 174795/225000 [333:39:14<84:37:04,  6.07s/it]
 78%|███████▊  | 174796/225000 [333:39:18<74:26:39,  5.34s/it]
 78%|███████▊  | 174797/225000 [333:39:25<84:14:28,  6.04s/it]
 78%|███████▊  | 174798/225000 [333:39:31<81:18:31,  5.83s/it]
 78%|███████▊  | 174799/225000 [333:39:35<76:20:07,  5.47s/it]
 78%|███████▊  | 174800/225000 [333:39:40<71:49:02,  5.15s/it]
                                                              
{'loss': '0.7659', 'grad_norm': '1.207', 'learning_rate': '6.45e-06', 'epoch': '0.7769', 'num_input_tokens_seen': 553030680, 'train_runtime': '1.201e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174800/225000 [333:39:40<71:49:02,  5.15s/it]
 78%|███████▊  | 174801/225000 [333:39:45<71:07:25,  5.10s/it]
 78%|███████▊  | 174802/225000 [333:39:49<68:13:09,  4.89s/it]
 78%|███████▊  | 174803/225000 [333:39:54<67:43:31,  4.86s/it]
 78%|███████▊  | 174804/225000 [333:39:59<68:03:09,  4.88s/it]
 78%|███████▊  | 174805/225000 [333:40:04<70:44:57,  5.07s/it]
 78%|███████▊  | 174806/225000 [333:40:15<93:27:26,  6.70s/it]
 78%|███████▊  | 174807/225000 [333:40:18<79:52:31,  5.73s/it]
 78%|███████▊  | 174808/225000 [333:40:26<87:13:23,  6.26s/it]
 78%|███████▊  | 174809/225000 [333:40:29<74:34:07,  5.35s/it]
 78%|███████▊  | 174810/225000 [333:40:37<86:29:18,  6.20s/it]
 78%|███████▊  | 174811/225000 [333:40:41<74:27:58,  5.34s/it]
 78%|███████▊  | 174812/225000 [333:40:48<83:41:24,  6.00s/it]
 78%|███████▊  | 174813/225000 [333:40:52<76:38:25,  5.50s/it]
 78%|███████▊  | 174814/225000 [333:40:59<83:27:06,  5.99s/it]
 78%|███████▊  | 174815/225000 [333:41:05<81:44:34,  5.86s/it]
 78%|███████▊  | 174816/225000 [333:41:09<71:59:33,  5.16s/it]
 78%|███████▊  | 174817/225000 [333:41:16<80:41:46,  5.79s/it]
 78%|███████▊  | 174818/225000 [333:41:23<86:29:29,  6.20s/it]
 78%|███████▊  | 174819/225000 [333:41:27<78:17:23,  5.62s/it]
 78%|███████▊  | 174820/225000 [333:41:32<72:38:23,  5.21s/it]
 78%|███████▊  | 174821/225000 [333:41:36<71:29:52,  5.13s/it]
 78%|███████▊  | 174822/225000 [333:41:41<69:42:51,  5.00s/it]
 78%|███████▊  | 174823/225000 [333:41:46<68:39:38,  4.93s/it]
 78%|███████▊  | 174824/225000 [333:41:51<69:04:24,  4.96s/it]
 78%|███████▊  | 174825/225000 [333:41:55<66:47:20,  4.79s/it]
 78%|███████▊  | 174826/225000 [333:42:00<67:45:36,  4.86s/it]
 78%|███████▊  | 174827/225000 [333:42:05<67:12:51,  4.82s/it]
 78%|███████▊  | 174828/225000 [333:42:10<65:54:25,  4.73s/it]
 78%|███████▊  | 174829/225000 [333:42:14<65:43:53,  4.72s/it]
 78%|███████▊  | 174830/225000 [333:42:19<66:04:10,  4.74s/it]
 78%|███████▊  | 174831/225000 [333:42:30<90:14:51,  6.48s/it]
 78%|███████▊  | 174832/225000 [333:42:37<93:30:56,  6.71s/it]
 78%|███████▊  | 174833/225000 [333:42:41<80:44:29,  5.79s/it]
 78%|███████▊  | 174834/225000 [333:42:49<93:25:22,  6.70s/it]
 78%|███████▊  | 174835/225000 [333:42:53<78:28:38,  5.63s/it]
 78%|███████▊  | 174836/225000 [333:42:59<80:26:50,  5.77s/it]
 78%|███████▊  | 174837/225000 [333:43:04<78:58:15,  5.67s/it]
 78%|███████▊  | 174838/225000 [333:43:08<70:01:48,  5.03s/it]
 78%|███████▊  | 174839/225000 [333:43:15<81:38:59,  5.86s/it]
 78%|███████▊  | 174840/225000 [333:43:19<70:55:12,  5.09s/it]
 78%|███████▊  | 174841/225000 [333:43:27<83:29:43,  5.99s/it]
 78%|███████▊  | 174842/225000 [333:43:30<72:50:15,  5.23s/it]
 78%|███████▊  | 174843/225000 [333:43:34<67:30:59,  4.85s/it]
 78%|███████▊  | 174844/225000 [333:43:39<66:13:31,  4.75s/it]
 78%|███████▊  | 174845/225000 [333:43:49<91:05:23,  6.54s/it]
 78%|███████▊  | 174846/225000 [333:43:55<85:31:10,  6.14s/it]
 78%|███████▊  | 174847/225000 [333:43:58<74:50:00,  5.37s/it]
 78%|███████▊  | 174848/225000 [333:44:02<69:39:36,  5.00s/it]
 78%|███████▊  | 174849/225000 [333:44:06<64:13:21,  4.61s/it]
 78%|███████▊  | 174850/225000 [333:44:16<86:59:58,  6.25s/it]
                                                              
{'loss': '0.7606', 'grad_norm': '1.249', 'learning_rate': '6.437e-06', 'epoch': '0.7771', 'num_input_tokens_seen': 553161872, 'train_runtime': '1.201e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174850/225000 [333:44:16<86:59:58,  6.25s/it]
 78%|███████▊  | 174851/225000 [333:44:20<76:02:53,  5.46s/it]
 78%|███████▊  | 174852/225000 [333:44:23<66:04:28,  4.74s/it]
 78%|███████▊  | 174853/225000 [333:44:31<79:16:03,  5.69s/it]
 78%|███████▊  | 174854/225000 [333:44:34<70:13:28,  5.04s/it]
 78%|███████▊  | 174855/225000 [333:44:44<90:11:18,  6.47s/it]
 78%|███████▊  | 174856/225000 [333:44:47<74:45:44,  5.37s/it]
 78%|███████▊  | 174857/225000 [333:44:54<81:40:10,  5.86s/it]
 78%|███████▊  | 174858/225000 [333:44:58<72:43:42,  5.22s/it]
 78%|███████▊  | 174859/225000 [333:45:01<66:42:06,  4.79s/it]
 78%|███████▊  | 174860/225000 [333:45:08<72:57:37,  5.24s/it]
 78%|███████▊  | 174861/225000 [333:45:14<78:11:23,  5.61s/it]
 78%|███████▊  | 174862/225000 [333:45:18<72:36:43,  5.21s/it]
 78%|███████▊  | 174863/225000 [333:45:22<66:24:29,  4.77s/it]
 78%|███████▊  | 174864/225000 [333:45:26<63:05:03,  4.53s/it]
 78%|███████▊  | 174865/225000 [333:45:34<76:51:29,  5.52s/it]
 78%|███████▊  | 174866/225000 [333:45:43<91:16:59,  6.55s/it]
 78%|███████▊  | 174867/225000 [333:45:46<78:26:34,  5.63s/it]
 78%|███████▊  | 174868/225000 [333:45:50<70:21:34,  5.05s/it]
 78%|███████▊  | 174869/225000 [333:45:59<86:47:25,  6.23s/it]
 78%|███████▊  | 174870/225000 [333:46:02<72:55:50,  5.24s/it]
 78%|███████▊  | 174871/225000 [333:46:09<81:29:06,  5.85s/it]
 78%|███████▊  | 174872/225000 [333:46:13<71:19:19,  5.12s/it]
 78%|███████▊  | 174873/225000 [333:46:16<63:38:36,  4.57s/it]
 78%|███████▊  | 174874/225000 [333:46:20<61:21:46,  4.41s/it]
 78%|███████▊  | 174875/225000 [333:46:30<86:17:21,  6.20s/it]
 78%|███████▊  | 174876/225000 [333:46:34<75:55:37,  5.45s/it]
 78%|███████▊  | 174877/225000 [333:46:38<69:47:01,  5.01s/it]
 78%|███████▊  | 174878/225000 [333:46:42<65:08:35,  4.68s/it]
 78%|███████▊  | 174879/225000 [333:46:48<70:26:47,  5.06s/it]
 78%|███████▊  | 174880/225000 [333:46:57<87:01:01,  6.25s/it]
 78%|███████▊  | 174881/225000 [333:47:00<74:37:49,  5.36s/it]
 78%|███████▊  | 174882/225000 [333:47:09<89:20:46,  6.42s/it]
 78%|███████▊  | 174883/225000 [333:47:14<83:02:23,  5.96s/it]
 78%|███████▊  | 174884/225000 [333:47:19<78:08:49,  5.61s/it]
 78%|███████▊  | 174885/225000 [333:47:23<73:34:08,  5.28s/it]
 78%|███████▊  | 174886/225000 [333:47:28<72:19:48,  5.20s/it]
 78%|███████▊  | 174887/225000 [333:47:33<69:17:21,  4.98s/it]
 78%|███████▊  | 174888/225000 [333:47:40<79:13:31,  5.69s/it]
 78%|███████▊  | 174889/225000 [333:47:46<79:39:12,  5.72s/it]
 78%|███████▊  | 174890/225000 [333:47:53<85:42:32,  6.16s/it]
 78%|███████▊  | 174891/225000 [333:47:58<80:37:32,  5.79s/it]
 78%|███████▊  | 174892/225000 [333:48:05<84:51:19,  6.10s/it]
 78%|███████▊  | 174893/225000 [333:48:08<73:42:35,  5.30s/it]
 78%|███████▊  | 174894/225000 [333:48:15<78:46:50,  5.66s/it]
 78%|███████▊  | 174895/225000 [333:48:19<71:52:57,  5.16s/it]
 78%|███████▊  | 174896/225000 [333:48:23<66:25:55,  4.77s/it]
 78%|███████▊  | 174897/225000 [333:48:27<63:21:51,  4.55s/it]
 78%|███████▊  | 174898/225000 [333:48:31<63:15:49,  4.55s/it]
 78%|███████▊  | 174899/225000 [333:48:37<66:20:02,  4.77s/it]
 78%|███████▊  | 174900/225000 [333:48:41<66:54:23,  4.81s/it]
                                                              
{'loss': '0.7652', 'grad_norm': '1.214', 'learning_rate': '6.425e-06', 'epoch': '0.7773', 'num_input_tokens_seen': 553291464, 'train_runtime': '1.202e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174900/225000 [333:48:41<66:54:23,  4.81s/it]
 78%|███████▊  | 174901/225000 [333:48:47<69:23:14,  4.99s/it]
 78%|███████▊  | 174902/225000 [333:48:52<69:39:09,  5.01s/it]
 78%|███████▊  | 174903/225000 [333:48:57<70:59:02,  5.10s/it]
 78%|███████▊  | 174904/225000 [333:49:02<71:22:41,  5.13s/it]
 78%|███████▊  | 174905/225000 [333:49:08<72:21:39,  5.20s/it]
 78%|███████▊  | 174906/225000 [333:49:13<73:08:24,  5.26s/it]
 78%|███████▊  | 174907/225000 [333:49:18<72:10:20,  5.19s/it]
 78%|███████▊  | 174908/225000 [333:49:23<69:37:20,  5.00s/it]
 78%|███████▊  | 174909/225000 [333:49:28<69:48:09,  5.02s/it]
 78%|███████▊  | 174910/225000 [333:49:33<71:22:03,  5.13s/it]
 78%|███████▊  | 174911/225000 [333:49:39<73:44:24,  5.30s/it]
 78%|███████▊  | 174912/225000 [333:49:44<74:23:08,  5.35s/it]
 78%|███████▊  | 174913/225000 [333:49:49<73:02:34,  5.25s/it]
 78%|███████▊  | 174914/225000 [333:49:54<72:00:22,  5.18s/it]
 78%|███████▊  | 174915/225000 [333:50:00<72:03:57,  5.18s/it]
 78%|███████▊  | 174916/225000 [333:50:05<72:17:49,  5.20s/it]
 78%|███████▊  | 174917/225000 [333:50:16<95:24:21,  6.86s/it]
 78%|███████▊  | 174918/225000 [333:50:23<97:50:56,  7.03s/it]
 78%|███████▊  | 174919/225000 [333:50:27<85:16:14,  6.13s/it]
 78%|███████▊  | 174920/225000 [333:50:31<75:48:33,  5.45s/it]
 78%|███████▊  | 174921/225000 [333:50:39<87:34:28,  6.30s/it]
 78%|███████▊  | 174922/225000 [333:50:45<83:48:00,  6.02s/it]
 78%|███████▊  | 174923/225000 [333:50:48<74:54:26,  5.39s/it]
 78%|███████▊  | 174924/225000 [333:50:52<68:53:12,  4.95s/it]
 78%|███████▊  | 174925/225000 [333:50:57<66:15:59,  4.76s/it]
 78%|███████▊  | 174926/225000 [333:51:04<76:50:00,  5.52s/it]
 78%|███████▊  | 174927/225000 [333:51:11<85:00:32,  6.11s/it]
 78%|███████▊  | 174928/225000 [333:51:16<77:11:08,  5.55s/it]
 78%|███████▊  | 174929/225000 [333:51:21<75:42:03,  5.44s/it]
 78%|███████▊  | 174930/225000 [333:51:26<74:45:45,  5.38s/it]
 78%|███████▊  | 174931/225000 [333:51:31<73:04:28,  5.25s/it]
 78%|███████▊  | 174932/225000 [333:51:37<74:22:58,  5.35s/it]
 78%|███████▊  | 174933/225000 [333:51:42<72:59:11,  5.25s/it]
 78%|███████▊  | 174934/225000 [333:51:46<70:48:46,  5.09s/it]
 78%|███████▊  | 174935/225000 [333:51:51<69:43:34,  5.01s/it]
 78%|███████▊  | 174936/225000 [333:51:56<69:42:07,  5.01s/it]
 78%|███████▊  | 174937/225000 [333:52:02<71:33:02,  5.15s/it]
 78%|███████▊  | 174938/225000 [333:52:07<73:15:33,  5.27s/it]
 78%|███████▊  | 174939/225000 [333:52:12<71:32:56,  5.15s/it]
 78%|███████▊  | 174940/225000 [333:52:17<71:31:41,  5.14s/it]
 78%|███████▊  | 174941/225000 [333:52:22<69:53:07,  5.03s/it]
 78%|███████▊  | 174942/225000 [333:52:27<70:14:43,  5.05s/it]
 78%|███████▊  | 174943/225000 [333:52:32<69:34:40,  5.00s/it]
 78%|███████▊  | 174944/225000 [333:52:37<69:23:08,  4.99s/it]
 78%|███████▊  | 174945/225000 [333:52:42<71:35:03,  5.15s/it]
 78%|███████▊  | 174946/225000 [333:52:55<101:26:42,  7.30s/it]
 78%|███████▊  | 174947/225000 [333:53:00<92:42:16,  6.67s/it] 
 78%|███████▊  | 174948/225000 [333:53:04<80:02:02,  5.76s/it]
 78%|███████▊  | 174949/225000 [333:53:08<72:34:45,  5.22s/it]
 78%|███████▊  | 174950/225000 [333:53:14<79:23:13,  5.71s/it]
                                                              
{'loss': '0.7617', 'grad_norm': '1.316', 'learning_rate': '6.413e-06', 'epoch': '0.7776', 'num_input_tokens_seen': 553421592, 'train_runtime': '1.202e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 174950/225000 [333:53:14<79:23:13,  5.71s/it]
 78%|███████▊  | 174951/225000 [333:53:22<87:39:25,  6.31s/it]
 78%|███████▊  | 174952/225000 [333:53:26<76:19:57,  5.49s/it]
 78%|███████▊  | 174953/225000 [333:53:31<76:31:50,  5.51s/it]
 78%|███████▊  | 174954/225000 [333:53:39<86:41:48,  6.24s/it]
 78%|███████▊  | 174955/225000 [333:53:43<77:59:40,  5.61s/it]
 78%|███████▊  | 174956/225000 [333:53:47<71:32:50,  5.15s/it]
 78%|███████▊  | 174957/225000 [333:53:55<82:29:33,  5.93s/it]
 78%|███████▊  | 174958/225000 [333:54:02<84:49:19,  6.10s/it]
 78%|███████▊  | 174959/225000 [333:54:06<77:53:30,  5.60s/it]
 78%|███████▊  | 174960/225000 [333:54:11<73:02:32,  5.25s/it]
 78%|███████▊  | 174961/225000 [333:54:15<69:05:46,  4.97s/it]
 78%|███████▊  | 174962/225000 [333:54:19<66:29:54,  4.78s/it]
 78%|███████▊  | 174963/225000 [333:54:29<86:26:27,  6.22s/it]
 78%|███████▊  | 174964/225000 [333:54:33<78:34:53,  5.65s/it]
 78%|███████▊  | 174965/225000 [333:54:37<72:18:01,  5.20s/it]
 78%|███████▊  | 174966/225000 [333:54:41<66:49:09,  4.81s/it]
 78%|███████▊  | 174967/225000 [333:54:46<65:39:02,  4.72s/it]
 78%|███████▊  | 174968/225000 [333:54:50<64:28:25,  4.64s/it]
 78%|███████▊  | 174969/225000 [333:54:56<71:42:20,  5.16s/it]
 78%|███████▊  | 174970/225000 [333:55:07<95:36:40,  6.88s/it]
 78%|███████▊  | 174971/225000 [333:55:12<85:17:58,  6.14s/it]
 78%|███████▊  | 174972/225000 [333:55:16<77:23:58,  5.57s/it]
 78%|███████▊  | 174973/225000 [333:55:20<71:02:26,  5.11s/it]
 78%|███████▊  | 174974/225000 [333:55:30<89:37:17,  6.45s/it]
 78%|███████▊  | 174975/225000 [333:55:35<83:52:19,  6.04s/it]
 78%|███████▊  | 174976/225000 [333:55:39<74:39:54,  5.37s/it]
 78%|███████▊  | 174977/225000 [333:55:43<68:55:13,  4.96s/it]
 78%|███████▊  | 174978/225000 [333:55:53<92:16:50,  6.64s/it]
 78%|███████▊  | 174979/225000 [333:55:57<79:47:47,  5.74s/it]
 78%|███████▊  | 174980/225000 [333:56:00<70:03:23,  5.04s/it]
 78%|███████▊  | 174981/225000 [333:56:08<80:09:52,  5.77s/it]
 78%|███████▊  | 174982/225000 [333:56:13<77:07:02,  5.55s/it]
 78%|███████▊  | 174983/225000 [333:56:17<73:21:31,  5.28s/it]
 78%|███████▊  | 174984/225000 [333:56:22<71:42:31,  5.16s/it]
 78%|███████▊  | 174985/225000 [333:56:27<70:10:08,  5.05s/it]
 78%|███████▊  | 174986/225000 [333:56:32<69:16:58,  4.99s/it]
 78%|███████▊  | 174987/225000 [333:56:37<70:19:25,  5.06s/it]
 78%|███████▊  | 174988/225000 [333:56:43<73:42:57,  5.31s/it]
 78%|███████▊  | 174989/225000 [333:56:54<95:39:06,  6.89s/it]
 78%|███████▊  | 174990/225000 [333:56:57<81:48:49,  5.89s/it]
 78%|███████▊  | 174991/225000 [333:57:01<72:01:25,  5.18s/it]
 78%|███████▊  | 174992/225000 [333:57:09<84:39:15,  6.09s/it]
 78%|███████▊  | 174993/225000 [333:57:13<75:48:28,  5.46s/it]
 78%|███████▊  | 174994/225000 [333:57:17<70:42:15,  5.09s/it]
 78%|███████▊  | 174995/225000 [333:57:21<67:34:08,  4.86s/it]
 78%|███████▊  | 174996/225000 [333:57:29<81:07:14,  5.84s/it]
 78%|███████▊  | 174997/225000 [333:57:35<81:29:45,  5.87s/it]
 78%|███████▊  | 174998/225000 [333:57:39<71:09:29,  5.12s/it]
 78%|███████▊  | 174999/225000 [333:57:47<83:49:25,  6.04s/it]
 78%|███████▊  | 175000/225000 [333:57:51<75:03:31,  5.40s/it]
                                                              
{'loss': '0.7668', 'grad_norm': '1.147', 'learning_rate': '6.401e-06', 'epoch': '0.7778', 'num_input_tokens_seen': 553552592, 'train_runtime': '1.202e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 175000/225000 [333:57:51<75:03:31,  5.40s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                              
{'eval_loss': '1.321', 'eval_runtime': '420.1', 'eval_samples_per_second': '1.19', 'eval_steps_per_second': '0.595', 'epoch': '0.7778', 'num_input_tokens_seen': 553552592}

 78%|███████▊  | 175000/225000 [334:04:51<75:03:31,  5.40s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-175000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 78%|███████▊  | 175001/225000 [334:04:55<1820:45:38, 131.10s/it]
 78%|███████▊  | 175002/225000 [334:05:00<1293:33:42, 93.14s/it] 
 78%|███████▊  | 175003/225000 [334:05:05<927:24:09, 66.78s/it] 
 78%|███████▊  | 175004/225000 [334:05:10<671:33:30, 48.36s/it]
 78%|███████▊  | 175005/225000 [334:05:16<491:34:27, 35.40s/it]
 78%|███████▊  | 175006/225000 [334:05:21<366:42:52, 26.41s/it]
 78%|███████▊  | 175007/225000 [334:05:26<276:58:49, 19.95s/it]
 78%|███████▊  | 175008/225000 [334:05:31<214:32:31, 15.45s/it]
 78%|███████▊  | 175009/225000 [334:05:36<171:45:15, 12.37s/it]
 78%|███████▊  | 175010/225000 [334:05:41<142:38:22, 10.27s/it]
 78%|███████▊  | 175011/225000 [334:05:47<121:49:15,  8.77s/it]
 78%|███████▊  | 175012/225000 [334:05:52<108:42:24,  7.83s/it]
 78%|███████▊  | 175013/225000 [334:05:58<98:10:22,  7.07s/it] 
 78%|███████▊  | 175014/225000 [334:06:03<89:44:22,  6.46s/it]
 78%|███████▊  | 175015/225000 [334:06:08<85:19:49,  6.15s/it]
 78%|███████▊  | 175016/225000 [334:06:13<80:16:30,  5.78s/it]
 78%|███████▊  | 175017/225000 [334:06:18<77:08:09,  5.56s/it]
 78%|███████▊  | 175018/225000 [334:06:23<74:52:32,  5.39s/it]
 78%|███████▊  | 175019/225000 [334:06:29<75:45:49,  5.46s/it]
 78%|███████▊  | 175020/225000 [334:06:34<73:02:33,  5.26s/it]
 78%|███████▊  | 175021/225000 [334:06:39<75:24:33,  5.43s/it]
 78%|███████▊  | 175022/225000 [334:06:45<75:04:48,  5.41s/it]
 78%|███████▊  | 175023/225000 [334:06:50<74:12:59,  5.35s/it]
 78%|███████▊  | 175024/225000 [334:06:55<73:03:36,  5.26s/it]
 78%|███████▊  | 175025/225000 [334:07:06<96:10:22,  6.93s/it]
 78%|███████▊  | 175026/225000 [334:07:13<99:12:27,  7.15s/it]
 78%|███████▊  | 175027/225000 [334:07:17<83:18:58,  6.00s/it]
 78%|███████▊  | 175028/225000 [334:07:25<91:08:42,  6.57s/it]
 78%|███████▊  | 175029/225000 [334:07:30<85:09:51,  6.14s/it]
 78%|███████▊  | 175030/225000 [334:07:35<80:29:08,  5.80s/it]
 78%|███████▊  | 175031/225000 [334:07:40<77:15:28,  5.57s/it]
 78%|███████▊  | 175032/225000 [334:07:45<74:26:19,  5.36s/it]
 78%|███████▊  | 175033/225000 [334:07:49<71:24:51,  5.15s/it]
 78%|███████▊  | 175034/225000 [334:07:54<68:20:10,  4.92s/it]
 78%|███████▊  | 175035/225000 [334:07:59<69:35:02,  5.01s/it]
 78%|███████▊  | 175036/225000 [334:08:04<70:09:06,  5.05s/it]
 78%|███████▊  | 175037/225000 [334:08:09<67:31:09,  4.86s/it]
 78%|███████▊  | 175038/225000 [334:08:17<84:12:10,  6.07s/it]
 78%|███████▊  | 175039/225000 [334:08:28<101:38:19,  7.32s/it]
 78%|███████▊  | 175040/225000 [334:08:32<87:07:48,  6.28s/it] 
 78%|███████▊  | 175041/225000 [334:08:40<95:22:56,  6.87s/it]
 78%|███████▊  | 175042/225000 [334:08:43<81:20:14,  5.86s/it]
 78%|███████▊  | 175043/225000 [334:08:47<72:30:11,  5.22s/it]
 78%|███████▊  | 175044/225000 [334:08:57<92:24:35,  6.66s/it]
 78%|███████▊  | 175045/225000 [334:09:00<78:52:42,  5.68s/it]
 78%|███████▊  | 175046/225000 [334:09:05<72:39:26,  5.24s/it]
 78%|███████▊  | 175047/225000 [334:09:12<81:42:31,  5.89s/it]
 78%|███████▊  | 175048/225000 [334:09:19<85:19:09,  6.15s/it]
 78%|███████▊  | 175049/225000 [334:09:23<79:04:57,  5.70s/it]
 78%|███████▊  | 175050/225000 [334:09:30<82:17:57,  5.93s/it]
                                                              
{'loss': '0.7719', 'grad_norm': '1.069', 'learning_rate': '6.388e-06', 'epoch': '0.778', 'num_input_tokens_seen': 553683072, 'train_runtime': '1.203e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175050/225000 [334:09:30<82:17:57,  5.93s/it]
 78%|███████▊  | 175051/225000 [334:09:34<73:45:25,  5.32s/it]
 78%|███████▊  | 175052/225000 [334:09:38<67:06:46,  4.84s/it]
 78%|███████▊  | 175053/225000 [334:09:41<62:58:54,  4.54s/it]
 78%|███████▊  | 175054/225000 [334:09:49<75:37:55,  5.45s/it]
 78%|███████▊  | 175055/225000 [334:09:57<85:48:57,  6.19s/it]
 78%|███████▊  | 175056/225000 [334:10:00<73:37:10,  5.31s/it]
 78%|███████▊  | 175057/225000 [334:10:08<84:00:17,  6.06s/it]
 78%|███████▊  | 175058/225000 [334:10:11<73:43:46,  5.31s/it]
 78%|███████▊  | 175059/225000 [334:10:17<74:04:12,  5.34s/it]
 78%|███████▊  | 175060/225000 [334:10:25<84:05:53,  6.06s/it]
 78%|███████▊  | 175061/225000 [334:10:28<72:28:48,  5.22s/it]
 78%|███████▊  | 175062/225000 [334:10:34<77:09:32,  5.56s/it]
 78%|███████▊  | 175063/225000 [334:10:39<72:06:49,  5.20s/it]
 78%|███████▊  | 175064/225000 [334:10:44<74:00:57,  5.34s/it]
 78%|███████▊  | 175065/225000 [334:10:49<71:33:31,  5.16s/it]
 78%|███████▊  | 175066/225000 [334:10:58<88:54:04,  6.41s/it]
 78%|███████▊  | 175067/225000 [334:11:03<80:24:56,  5.80s/it]
 78%|███████▊  | 175068/225000 [334:11:07<73:59:12,  5.33s/it]
 78%|███████▊  | 175069/225000 [334:11:11<70:44:08,  5.10s/it]
 78%|███████▊  | 175070/225000 [334:11:16<68:46:00,  4.96s/it]
 78%|███████▊  | 175071/225000 [334:11:20<66:15:55,  4.78s/it]
 78%|███████▊  | 175072/225000 [334:11:28<75:54:17,  5.47s/it]
 78%|███████▊  | 175073/225000 [334:11:38<96:05:03,  6.93s/it]
 78%|███████▊  | 175074/225000 [334:11:44<91:43:52,  6.61s/it]
 78%|███████▊  | 175075/225000 [334:11:48<82:42:24,  5.96s/it]
 78%|███████▊  | 175076/225000 [334:11:52<72:58:25,  5.26s/it]
 78%|███████▊  | 175077/225000 [334:12:01<88:53:11,  6.41s/it]
 78%|███████▊  | 175078/225000 [334:12:04<75:13:27,  5.42s/it]
 78%|███████▊  | 175079/225000 [334:12:10<76:21:20,  5.51s/it]
 78%|███████▊  | 175080/225000 [334:12:16<80:04:59,  5.78s/it]
 78%|███████▊  | 175081/225000 [334:12:20<72:46:06,  5.25s/it]
 78%|███████▊  | 175082/225000 [334:12:24<67:30:11,  4.87s/it]
 78%|███████▊  | 175083/225000 [334:12:28<64:25:18,  4.65s/it]
 78%|███████▊  | 175084/225000 [334:12:39<87:52:24,  6.34s/it]
 78%|███████▊  | 175085/225000 [334:12:42<76:14:42,  5.50s/it]
 78%|███████▊  | 175086/225000 [334:12:51<88:44:43,  6.40s/it]
 78%|███████▊  | 175087/225000 [334:12:55<80:30:12,  5.81s/it]
 78%|███████▊  | 175088/225000 [334:13:00<75:20:18,  5.43s/it]
 78%|███████▊  | 175089/225000 [334:13:04<72:00:23,  5.19s/it]
 78%|███████▊  | 175090/225000 [334:13:09<69:46:36,  5.03s/it]
 78%|███████▊  | 175091/225000 [334:13:13<67:49:06,  4.89s/it]
 78%|███████▊  | 175092/225000 [334:13:18<66:24:57,  4.79s/it]
 78%|███████▊  | 175093/225000 [334:13:26<77:47:56,  5.61s/it]
 78%|███████▊  | 175094/225000 [334:13:34<89:17:48,  6.44s/it]
 78%|███████▊  | 175095/225000 [334:13:39<82:28:28,  5.95s/it]
 78%|███████▊  | 175096/225000 [334:13:46<87:42:35,  6.33s/it]
 78%|███████▊  | 175097/225000 [334:13:49<75:30:19,  5.45s/it]
 78%|███████▊  | 175098/225000 [334:13:59<91:20:31,  6.59s/it]
 78%|███████▊  | 175099/225000 [334:14:01<75:54:03,  5.48s/it]
 78%|███████▊  | 175100/225000 [334:14:06<70:43:25,  5.10s/it]
                                                              
{'loss': '0.7727', 'grad_norm': '1.061', 'learning_rate': '6.376e-06', 'epoch': '0.7782', 'num_input_tokens_seen': 553813776, 'train_runtime': '1.203e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175100/225000 [334:14:06<70:43:25,  5.10s/it]
 78%|███████▊  | 175101/225000 [334:14:10<67:49:05,  4.89s/it]
 78%|███████▊  | 175102/225000 [334:14:21<92:22:44,  6.66s/it]
 78%|███████▊  | 175103/225000 [334:14:25<82:33:18,  5.96s/it]
 78%|███████▊  | 175104/225000 [334:14:30<76:59:59,  5.56s/it]
 78%|███████▊  | 175105/225000 [334:14:34<73:14:57,  5.29s/it]
 78%|███████▊  | 175106/225000 [334:14:39<70:11:20,  5.06s/it]
 78%|███████▊  | 175107/225000 [334:14:43<67:38:29,  4.88s/it]
 78%|███████▊  | 175108/225000 [334:14:48<67:37:01,  4.88s/it]
 78%|███████▊  | 175109/225000 [334:14:53<67:49:40,  4.89s/it]
 78%|███████▊  | 175110/225000 [334:14:58<67:55:08,  4.90s/it]
 78%|███████▊  | 175111/225000 [334:15:03<66:26:20,  4.79s/it]
 78%|███████▊  | 175112/225000 [334:15:12<83:04:14,  5.99s/it]
 78%|███████▊  | 175113/225000 [334:15:20<94:39:46,  6.83s/it]
 78%|███████▊  | 175114/225000 [334:15:24<81:06:11,  5.85s/it]
 78%|███████▊  | 175115/225000 [334:15:32<88:45:38,  6.41s/it]
 78%|███████▊  | 175116/225000 [334:15:37<83:26:19,  6.02s/it]
 78%|███████▊  | 175117/225000 [334:15:41<78:05:07,  5.64s/it]
 78%|███████▊  | 175118/225000 [334:15:48<83:02:28,  5.99s/it]
 78%|███████▊  | 175119/225000 [334:15:52<72:44:14,  5.25s/it]
 78%|███████▊  | 175120/225000 [334:16:02<93:03:52,  6.72s/it]
 78%|███████▊  | 175121/225000 [334:16:06<82:06:52,  5.93s/it]
 78%|███████▊  | 175122/225000 [334:16:10<73:14:13,  5.29s/it]
 78%|███████▊  | 175123/225000 [334:16:14<68:34:02,  4.95s/it]
 78%|███████▊  | 175124/225000 [334:16:22<83:03:52,  6.00s/it]
 78%|███████▊  | 175125/225000 [334:16:27<78:44:50,  5.68s/it]
 78%|███████▊  | 175126/225000 [334:16:32<74:00:20,  5.34s/it]
 78%|███████▊  | 175127/225000 [334:16:37<71:08:53,  5.14s/it]
 78%|███████▊  | 175128/225000 [334:16:42<70:53:21,  5.12s/it]
 78%|███████▊  | 175129/225000 [334:16:46<68:35:25,  4.95s/it]
 78%|███████▊  | 175130/225000 [334:16:51<67:40:27,  4.89s/it]
 78%|███████▊  | 175131/225000 [334:16:56<67:06:25,  4.84s/it]
 78%|███████▊  | 175132/225000 [334:17:00<66:45:14,  4.82s/it]
 78%|███████▊  | 175133/225000 [334:17:12<95:29:13,  6.89s/it]
 78%|███████▊  | 175134/225000 [334:17:19<94:05:16,  6.79s/it]
 78%|███████▊  | 175135/225000 [334:17:22<79:50:30,  5.76s/it]
 78%|███████▊  | 175136/225000 [334:17:27<74:31:59,  5.38s/it]
 78%|███████▊  | 175137/225000 [334:17:35<87:14:55,  6.30s/it]
 78%|███████▊  | 175138/225000 [334:17:39<77:11:22,  5.57s/it]
 78%|███████▊  | 175139/225000 [334:17:43<69:42:41,  5.03s/it]
 78%|███████▊  | 175140/225000 [334:17:47<66:07:07,  4.77s/it]
 78%|███████▊  | 175141/225000 [334:17:57<89:42:13,  6.48s/it]
 78%|███████▊  | 175142/225000 [334:18:02<80:43:16,  5.83s/it]
 78%|███████▊  | 175143/225000 [334:18:08<84:16:06,  6.08s/it]
 78%|███████▊  | 175144/225000 [334:18:13<78:53:07,  5.70s/it]
 78%|███████▊  | 175145/225000 [334:18:20<85:09:51,  6.15s/it]
 78%|███████▊  | 175146/225000 [334:18:25<78:26:20,  5.66s/it]
 78%|███████▊  | 175147/225000 [334:18:29<73:42:29,  5.32s/it]
 78%|███████▊  | 175148/225000 [334:18:34<69:00:11,  4.98s/it]
 78%|███████▊  | 175149/225000 [334:18:38<68:17:22,  4.93s/it]
 78%|███████▊  | 175150/225000 [334:18:43<67:48:30,  4.90s/it]
                                                              
{'loss': '0.7642', 'grad_norm': '1.207', 'learning_rate': '6.364e-06', 'epoch': '0.7784', 'num_input_tokens_seen': 553944384, 'train_runtime': '1.204e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175150/225000 [334:18:43<67:48:30,  4.90s/it]
 78%|███████▊  | 175151/225000 [334:18:48<68:25:59,  4.94s/it]
 78%|███████▊  | 175152/225000 [334:18:53<69:08:54,  4.99s/it]
 78%|███████▊  | 175153/225000 [334:18:58<68:19:24,  4.93s/it]
 78%|███████▊  | 175154/225000 [334:19:03<68:35:29,  4.95s/it]
 78%|███████▊  | 175155/225000 [334:19:14<94:23:42,  6.82s/it]
 78%|███████▊  | 175156/225000 [334:19:20<91:49:49,  6.63s/it]
 78%|███████▊  | 175157/225000 [334:19:27<92:20:52,  6.67s/it]
 78%|███████▊  | 175158/225000 [334:19:31<79:24:36,  5.74s/it]
 78%|███████▊  | 175159/225000 [334:19:35<73:07:17,  5.28s/it]
 78%|███████▊  | 175160/225000 [334:19:39<67:28:56,  4.87s/it]
 78%|███████▊  | 175161/225000 [334:19:49<88:40:03,  6.40s/it]
 78%|███████▊  | 175162/225000 [334:19:52<76:33:27,  5.53s/it]
 78%|███████▊  | 175163/225000 [334:20:00<84:40:22,  6.12s/it]
 78%|███████▊  | 175164/225000 [334:20:05<81:24:36,  5.88s/it]
 78%|███████▊  | 175165/225000 [334:20:09<73:55:24,  5.34s/it]
 78%|███████▊  | 175166/225000 [334:20:14<70:22:40,  5.08s/it]
 78%|███████▊  | 175167/225000 [334:20:18<68:09:29,  4.92s/it]
 78%|███████▊  | 175168/225000 [334:20:27<83:10:40,  6.01s/it]
 78%|███████▊  | 175169/225000 [334:20:33<82:19:10,  5.95s/it]
 78%|███████▊  | 175170/225000 [334:20:36<72:49:08,  5.26s/it]
 78%|███████▊  | 175171/225000 [334:20:45<88:09:49,  6.37s/it]
 78%|███████▊  | 175172/225000 [334:20:49<77:51:48,  5.63s/it]
 78%|███████▊  | 175173/225000 [334:20:53<70:29:43,  5.09s/it]
 78%|███████▊  | 175174/225000 [334:21:02<87:37:18,  6.33s/it]
 78%|███████▊  | 175175/225000 [334:21:07<80:25:21,  5.81s/it]
 78%|███████▊  | 175176/225000 [334:21:11<74:22:42,  5.37s/it]
 78%|███████▊  | 175177/225000 [334:21:16<70:50:52,  5.12s/it]
 78%|███████▊  | 175178/225000 [334:21:21<69:33:37,  5.03s/it]
 78%|███████▊  | 175179/225000 [334:21:25<68:44:59,  4.97s/it]
 78%|███████▊  | 175180/225000 [334:21:30<66:45:43,  4.82s/it]
 78%|███████▊  | 175181/225000 [334:21:34<65:53:06,  4.76s/it]
 78%|███████▊  | 175182/225000 [334:21:46<92:35:03,  6.69s/it]
 78%|███████▊  | 175183/225000 [334:21:50<82:37:06,  5.97s/it]
 78%|███████▊  | 175184/225000 [334:21:58<91:58:42,  6.65s/it]
 78%|███████▊  | 175185/225000 [334:22:02<79:19:33,  5.73s/it]
 78%|███████▊  | 175186/225000 [334:22:05<70:18:12,  5.08s/it]
 78%|███████▊  | 175187/225000 [334:22:14<83:19:26,  6.02s/it]
 78%|███████▊  | 175188/225000 [334:22:18<76:04:43,  5.50s/it]
 78%|███████▊  | 175189/225000 [334:22:25<81:15:51,  5.87s/it]
 78%|███████▊  | 175190/225000 [334:22:30<80:46:31,  5.84s/it]
 78%|███████▊  | 175191/225000 [334:22:36<80:20:40,  5.81s/it]
 78%|███████▊  | 175192/225000 [334:22:41<78:38:43,  5.68s/it]
 78%|███████▊  | 175193/225000 [334:22:45<68:45:51,  4.97s/it]
 78%|███████▊  | 175194/225000 [334:22:53<81:50:47,  5.92s/it]
 78%|███████▊  | 175195/225000 [334:22:57<73:21:34,  5.30s/it]
 78%|███████▊  | 175196/225000 [334:23:01<67:43:40,  4.90s/it]
 78%|███████▊  | 175197/225000 [334:23:06<68:15:08,  4.93s/it]
 78%|███████▊  | 175198/225000 [334:23:11<69:10:19,  5.00s/it]
 78%|███████▊  | 175199/225000 [334:23:15<67:09:15,  4.85s/it]
 78%|███████▊  | 175200/225000 [334:23:21<71:39:47,  5.18s/it]
                                                              
{'loss': '0.7582', 'grad_norm': '1.184', 'learning_rate': '6.352e-06', 'epoch': '0.7787', 'num_input_tokens_seen': 554075080, 'train_runtime': '1.204e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175200/225000 [334:23:21<71:39:47,  5.18s/it]
 78%|███████▊  | 175201/225000 [334:23:30<87:36:18,  6.33s/it]
 78%|███████▊  | 175202/225000 [334:23:35<79:02:08,  5.71s/it]
 78%|███████▊  | 175203/225000 [334:23:39<73:27:44,  5.31s/it]
 78%|███████▊  | 175204/225000 [334:23:43<69:38:49,  5.04s/it]
 78%|███████▊  | 175205/225000 [334:23:48<67:18:36,  4.87s/it]
 78%|███████▊  | 175206/225000 [334:23:56<78:57:29,  5.71s/it]
 78%|███████▊  | 175207/225000 [334:24:05<95:22:54,  6.90s/it]
 78%|███████▊  | 175208/225000 [334:24:09<81:53:20,  5.92s/it]
 78%|███████▊  | 175209/225000 [334:24:14<80:39:45,  5.83s/it]
 78%|███████▊  | 175210/225000 [334:24:19<76:56:16,  5.56s/it]
 78%|███████▊  | 175211/225000 [334:24:27<86:51:34,  6.28s/it]
 78%|███████▊  | 175212/225000 [334:24:32<79:43:55,  5.77s/it]
 78%|███████▊  | 175213/225000 [334:24:36<70:45:11,  5.12s/it]
 78%|███████▊  | 175214/225000 [334:24:41<72:07:39,  5.22s/it]
 78%|███████▊  | 175215/225000 [334:24:46<72:08:06,  5.22s/it]
 78%|███████▊  | 175216/225000 [334:24:51<71:30:31,  5.17s/it]
 78%|███████▊  | 175217/225000 [334:24:56<71:02:12,  5.14s/it]
 78%|███████▊  | 175218/225000 [334:25:02<73:07:38,  5.29s/it]
 78%|███████▊  | 175219/225000 [334:25:07<73:35:51,  5.32s/it]
 78%|███████▊  | 175220/225000 [334:25:13<74:52:10,  5.41s/it]
 78%|███████▊  | 175221/225000 [334:25:18<74:37:29,  5.40s/it]
 78%|███████▊  | 175222/225000 [334:25:24<73:52:15,  5.34s/it]
 78%|███████▊  | 175223/225000 [334:25:29<74:51:49,  5.41s/it]
 78%|███████▊  | 175224/225000 [334:25:34<73:21:32,  5.31s/it]
 78%|███████▊  | 175225/225000 [334:25:40<73:40:34,  5.33s/it]
 78%|███████▊  | 175226/225000 [334:25:45<74:32:31,  5.39s/it]
 78%|███████▊  | 175227/225000 [334:25:51<75:25:02,  5.45s/it]
 78%|███████▊  | 175228/225000 [334:25:56<74:50:13,  5.41s/it]
 78%|███████▊  | 175229/225000 [334:26:01<74:57:07,  5.42s/it]
 78%|███████▊  | 175230/225000 [334:26:07<74:57:52,  5.42s/it]
 78%|███████▊  | 175231/225000 [334:26:12<74:30:53,  5.39s/it]
 78%|███████▊  | 175232/225000 [334:26:18<75:07:11,  5.43s/it]
 78%|███████▊  | 175233/225000 [334:26:23<74:38:42,  5.40s/it]
 78%|███████▊  | 175234/225000 [334:26:28<73:46:08,  5.34s/it]
 78%|███████▊  | 175235/225000 [334:26:34<73:27:56,  5.31s/it]
 78%|███████▊  | 175236/225000 [334:26:39<74:00:12,  5.35s/it]
 78%|███████▊  | 175237/225000 [334:26:45<76:06:47,  5.51s/it]
 78%|███████▊  | 175238/225000 [334:26:57<104:56:00,  7.59s/it]
 78%|███████▊  | 175239/225000 [334:27:04<99:27:41,  7.20s/it] 
 78%|███████▊  | 175240/225000 [334:27:07<84:40:51,  6.13s/it]
 78%|███████▊  | 175241/225000 [334:27:11<75:09:56,  5.44s/it]
 78%|███████▊  | 175242/225000 [334:27:15<69:27:05,  5.02s/it]
 78%|███████▊  | 175243/225000 [334:27:27<96:26:39,  6.98s/it]
 78%|███████▊  | 175244/225000 [334:27:30<81:05:54,  5.87s/it]
 78%|███████▊  | 175245/225000 [334:27:37<85:35:03,  6.19s/it]
 78%|███████▊  | 175246/225000 [334:27:41<76:00:46,  5.50s/it]
 78%|███████▊  | 175247/225000 [334:27:45<69:10:39,  5.01s/it]
 78%|███████▊  | 175248/225000 [334:27:49<66:44:20,  4.83s/it]
 78%|███████▊  | 175249/225000 [334:27:53<62:57:25,  4.56s/it]
 78%|███████▊  | 175250/225000 [334:28:02<82:56:52,  6.00s/it]
                                                              
{'loss': '0.7656', 'grad_norm': '1.116', 'learning_rate': '6.34e-06', 'epoch': '0.7789', 'num_input_tokens_seen': 554205632, 'train_runtime': '1.204e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175250/225000 [334:28:02<82:56:52,  6.00s/it]
 78%|███████▊  | 175251/225000 [334:28:08<80:24:17,  5.82s/it]
 78%|███████▊  | 175252/225000 [334:28:11<69:02:10,  5.00s/it]
 78%|███████▊  | 175253/225000 [334:28:15<65:25:32,  4.73s/it]
 78%|███████▊  | 175254/225000 [334:28:25<89:34:39,  6.48s/it]
 78%|███████▊  | 175255/225000 [334:28:30<80:40:45,  5.84s/it]
 78%|███████▊  | 175256/225000 [334:28:34<75:07:47,  5.44s/it]
 78%|███████▊  | 175257/225000 [334:28:39<71:33:43,  5.18s/it]
 78%|███████▊  | 175258/225000 [334:28:43<68:23:00,  4.95s/it]
 78%|███████▊  | 175259/225000 [334:28:48<66:33:58,  4.82s/it]
 78%|███████▊  | 175260/225000 [334:28:52<64:59:27,  4.70s/it]
 78%|███████▊  | 175261/225000 [334:28:57<64:38:24,  4.68s/it]
 78%|███████▊  | 175262/225000 [334:29:04<76:07:03,  5.51s/it]
 78%|███████▊  | 175263/225000 [334:29:12<86:02:56,  6.23s/it]
 78%|███████▊  | 175264/225000 [334:29:16<76:08:57,  5.51s/it]
 78%|███████▊  | 175265/225000 [334:29:20<68:18:13,  4.94s/it]
 78%|███████▊  | 175266/225000 [334:29:29<85:10:13,  6.17s/it]
 78%|███████▊  | 175267/225000 [334:29:33<76:31:44,  5.54s/it]
 78%|███████▊  | 175268/225000 [334:29:38<75:15:25,  5.45s/it]
 78%|███████▊  | 175269/225000 [334:29:44<77:08:15,  5.58s/it]
 78%|███████▊  | 175270/225000 [334:29:52<89:03:00,  6.45s/it]
 78%|███████▊  | 175271/225000 [334:29:57<82:34:34,  5.98s/it]
 78%|███████▊  | 175272/225000 [334:30:02<76:11:42,  5.52s/it]
 78%|███████▊  | 175273/225000 [334:30:06<71:10:37,  5.15s/it]
 78%|███████▊  | 175274/225000 [334:30:10<66:38:51,  4.83s/it]
 78%|███████▊  | 175275/225000 [334:30:15<65:45:09,  4.76s/it]
 78%|███████▊  | 175276/225000 [334:30:20<66:09:07,  4.79s/it]
 78%|███████▊  | 175277/225000 [334:30:24<64:50:42,  4.69s/it]
 78%|███████▊  | 175278/225000 [334:30:28<63:51:43,  4.62s/it]
 78%|███████▊  | 175279/225000 [334:30:33<64:43:40,  4.69s/it]
 78%|███████▊  | 175280/225000 [334:30:41<76:13:35,  5.52s/it]
 78%|███████▊  | 175281/225000 [334:30:50<93:03:00,  6.74s/it]
 78%|███████▊  | 175282/225000 [334:30:54<80:13:57,  5.81s/it]
 78%|███████▊  | 175283/225000 [334:30:58<71:56:50,  5.21s/it]
 78%|███████▊  | 175284/225000 [334:31:08<94:21:52,  6.83s/it]
 78%|███████▊  | 175285/225000 [334:31:12<79:43:04,  5.77s/it]
 78%|███████▊  | 175286/225000 [334:31:16<73:58:54,  5.36s/it]
 78%|███████▊  | 175287/225000 [334:31:22<74:49:42,  5.42s/it]
 78%|███████▊  | 175288/225000 [334:31:27<75:05:12,  5.44s/it]
 78%|███████▊  | 175289/225000 [334:31:32<74:19:52,  5.38s/it]
 78%|███████▊  | 175290/225000 [334:31:37<72:01:10,  5.22s/it]
 78%|███████▊  | 175291/225000 [334:31:45<84:23:35,  6.11s/it]
 78%|███████▊  | 175292/225000 [334:31:49<75:58:35,  5.50s/it]
 78%|███████▊  | 175293/225000 [334:31:54<70:53:55,  5.13s/it]
 78%|███████▊  | 175294/225000 [334:32:00<74:01:32,  5.36s/it]
 78%|███████▊  | 175295/225000 [334:32:08<85:51:19,  6.22s/it]
 78%|███████▊  | 175296/225000 [334:32:11<74:58:46,  5.43s/it]
 78%|███████▊  | 175297/225000 [334:32:17<75:40:28,  5.48s/it]
 78%|███████▊  | 175298/225000 [334:32:23<78:08:51,  5.66s/it]
 78%|███████▊  | 175299/225000 [334:32:27<70:22:40,  5.10s/it]
 78%|███████▊  | 175300/225000 [334:32:31<66:32:12,  4.82s/it]
                                                              
{'loss': '0.7743', 'grad_norm': '1.023', 'learning_rate': '6.328e-06', 'epoch': '0.7791', 'num_input_tokens_seen': 554335032, 'train_runtime': '1.204e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175300/225000 [334:32:31<66:32:12,  4.82s/it]
 78%|███████▊  | 175301/225000 [334:32:35<62:28:09,  4.53s/it]
 78%|███████▊  | 175302/225000 [334:32:45<84:02:38,  6.09s/it]
 78%|███████▊  | 175303/225000 [334:32:49<77:01:56,  5.58s/it]
 78%|███████▊  | 175304/225000 [334:32:53<69:58:25,  5.07s/it]
 78%|███████▊  | 175305/225000 [334:32:57<67:16:19,  4.87s/it]
 78%|███████▊  | 175306/225000 [334:33:01<64:04:06,  4.64s/it]
 78%|███████▊  | 175307/225000 [334:33:09<74:37:30,  5.41s/it]
 78%|███████▊  | 175308/225000 [334:33:18<91:51:29,  6.65s/it]
 78%|███████▊  | 175309/225000 [334:33:22<78:31:22,  5.69s/it]
 78%|███████▊  | 175310/225000 [334:33:27<79:01:48,  5.73s/it]
 78%|███████▊  | 175311/225000 [334:33:35<87:30:50,  6.34s/it]
 78%|███████▊  | 175312/225000 [334:33:39<74:50:19,  5.42s/it]
 78%|███████▊  | 175313/225000 [334:33:43<69:04:42,  5.00s/it]
 78%|███████▊  | 175314/225000 [334:33:51<81:48:53,  5.93s/it]
 78%|███████▊  | 175315/225000 [334:33:54<70:31:46,  5.11s/it]
 78%|███████▊  | 175316/225000 [334:34:02<83:19:25,  6.04s/it]
 78%|███████▊  | 175317/225000 [334:34:06<73:48:28,  5.35s/it]
 78%|███████▊  | 175318/225000 [334:34:11<74:35:14,  5.40s/it]
 78%|███████▊  | 175319/225000 [334:34:16<72:19:49,  5.24s/it]
 78%|███████▊  | 175320/225000 [334:34:21<71:23:48,  5.17s/it]
 78%|███████▊  | 175321/225000 [334:34:26<71:18:52,  5.17s/it]
 78%|███████▊  | 175322/225000 [334:34:32<71:41:33,  5.20s/it]
 78%|███████▊  | 175323/225000 [334:34:36<70:17:08,  5.09s/it]
 78%|███████▊  | 175324/225000 [334:34:41<66:18:28,  4.81s/it]
 78%|███████▊  | 175325/225000 [334:34:46<67:37:41,  4.90s/it]
 78%|███████▊  | 175326/225000 [334:34:51<69:55:08,  5.07s/it]
 78%|███████▊  | 175327/225000 [334:34:56<70:47:17,  5.13s/it]
 78%|███████▊  | 175328/225000 [334:35:01<70:24:31,  5.10s/it]
 78%|███████▊  | 175329/225000 [334:35:07<70:04:36,  5.08s/it]
 78%|███████▊  | 175330/225000 [334:35:12<70:22:40,  5.10s/it]
 78%|███████▊  | 175331/225000 [334:35:17<71:20:49,  5.17s/it]
 78%|███████▊  | 175332/225000 [334:35:22<71:21:13,  5.17s/it]
 78%|███████▊  | 175333/225000 [334:35:27<68:43:17,  4.98s/it]
 78%|███████▊  | 175334/225000 [334:35:37<89:16:06,  6.47s/it]
 78%|███████▊  | 175335/225000 [334:35:46<99:42:33,  7.23s/it]
 78%|███████▊  | 175336/225000 [334:35:49<84:40:50,  6.14s/it]
 78%|███████▊  | 175337/225000 [334:35:55<84:58:14,  6.16s/it]
 78%|███████▊  | 175338/225000 [334:36:02<85:41:10,  6.21s/it]
 78%|███████▊  | 175339/225000 [334:36:08<86:24:54,  6.26s/it]
 78%|███████▊  | 175340/225000 [334:36:13<80:34:16,  5.84s/it]
 78%|███████▊  | 175341/225000 [334:36:17<71:28:39,  5.18s/it]
 78%|███████▊  | 175342/225000 [334:36:20<65:07:55,  4.72s/it]
 78%|███████▊  | 175343/225000 [334:36:26<68:35:10,  4.97s/it]
 78%|███████▊  | 175344/225000 [334:36:35<84:40:23,  6.14s/it]
 78%|███████▊  | 175345/225000 [334:36:38<73:23:33,  5.32s/it]
 78%|███████▊  | 175346/225000 [334:36:46<85:55:26,  6.23s/it]
 78%|███████▊  | 175347/225000 [334:36:50<72:37:12,  5.27s/it]
 78%|███████▊  | 175348/225000 [334:36:54<67:23:28,  4.89s/it]
 78%|███████▊  | 175349/225000 [334:36:58<65:19:32,  4.74s/it]
 78%|███████▊  | 175350/225000 [334:37:08<86:26:18,  6.27s/it]
                                                              
{'loss': '0.7752', 'grad_norm': '1.223', 'learning_rate': '6.316e-06', 'epoch': '0.7793', 'num_input_tokens_seen': 554464912, 'train_runtime': '1.205e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175350/225000 [334:37:08<86:26:18,  6.27s/it]
 78%|███████▊  | 175351/225000 [334:37:11<75:05:33,  5.44s/it]
 78%|███████▊  | 175352/225000 [334:37:19<85:42:52,  6.22s/it]
 78%|███████▊  | 175353/225000 [334:37:23<75:27:43,  5.47s/it]
 78%|███████▊  | 175354/225000 [334:37:29<77:51:08,  5.65s/it]
 78%|███████▊  | 175355/225000 [334:37:33<70:07:20,  5.08s/it]
 78%|███████▊  | 175356/225000 [334:37:37<65:26:39,  4.75s/it]
 78%|███████▊  | 175357/225000 [334:37:46<83:38:54,  6.07s/it]
 78%|███████▊  | 175358/225000 [334:37:50<73:24:23,  5.32s/it]
 78%|███████▊  | 175359/225000 [334:37:58<84:31:17,  6.13s/it]
 78%|███████▊  | 175360/225000 [334:38:02<77:00:29,  5.58s/it]
 78%|███████▊  | 175361/225000 [334:38:07<74:21:04,  5.39s/it]
 78%|███████▊  | 175362/225000 [334:38:13<76:41:03,  5.56s/it]
 78%|███████▊  | 175363/225000 [334:38:16<67:58:29,  4.93s/it]
 78%|███████▊  | 175364/225000 [334:38:20<61:45:35,  4.48s/it]
 78%|███████▊  | 175365/225000 [334:38:30<85:57:31,  6.23s/it]
 78%|███████▊  | 175366/225000 [334:38:34<78:15:17,  5.68s/it]
 78%|███████▊  | 175367/225000 [334:38:39<73:17:34,  5.32s/it]
 78%|███████▊  | 175368/225000 [334:38:43<69:28:16,  5.04s/it]
 78%|███████▊  | 175369/225000 [334:38:47<65:27:48,  4.75s/it]
 78%|███████▊  | 175370/225000 [334:38:53<69:06:22,  5.01s/it]
 78%|███████▊  | 175371/225000 [334:39:02<86:15:27,  6.26s/it]
 78%|███████▊  | 175372/225000 [334:39:06<74:36:18,  5.41s/it]
 78%|███████▊  | 175373/225000 [334:39:13<81:42:00,  5.93s/it]
 78%|███████▊  | 175374/225000 [334:39:16<71:13:32,  5.17s/it]
 78%|███████▊  | 175375/225000 [334:39:21<70:03:20,  5.08s/it]
 78%|███████▊  | 175376/225000 [334:39:26<69:45:10,  5.06s/it]
 78%|███████▊  | 175377/225000 [334:39:31<69:36:01,  5.05s/it]
 78%|███████▊  | 175378/225000 [334:39:36<69:10:50,  5.02s/it]
 78%|███████▊  | 175379/225000 [334:39:41<70:12:05,  5.09s/it]
 78%|███████▊  | 175380/225000 [334:39:46<71:11:12,  5.16s/it]
 78%|███████▊  | 175381/225000 [334:39:51<70:03:22,  5.08s/it]
 78%|███████▊  | 175382/225000 [334:39:56<70:01:57,  5.08s/it]
 78%|███████▊  | 175383/225000 [334:40:02<70:22:36,  5.11s/it]
 78%|███████▊  | 175384/225000 [334:40:07<71:17:08,  5.17s/it]
 78%|███████▊  | 175385/225000 [334:40:12<71:27:56,  5.19s/it]
 78%|███████▊  | 175386/225000 [334:40:17<70:05:39,  5.09s/it]
 78%|███████▊  | 175387/225000 [334:40:22<67:54:17,  4.93s/it]
 78%|███████▊  | 175388/225000 [334:40:27<69:04:55,  5.01s/it]
 78%|███████▊  | 175389/225000 [334:40:39<100:31:35,  7.29s/it]
 78%|███████▊  | 175390/225000 [334:40:44<89:39:01,  6.51s/it] 
 78%|███████▊  | 175391/225000 [334:40:49<83:29:17,  6.06s/it]
 78%|███████▊  | 175392/225000 [334:40:54<77:06:21,  5.60s/it]
 78%|███████▊  | 175393/225000 [334:40:58<74:07:20,  5.38s/it]
 78%|███████▊  | 175394/225000 [334:41:03<72:07:17,  5.23s/it]
 78%|███████▊  | 175395/225000 [334:41:08<70:35:14,  5.12s/it]
 78%|███████▊  | 175396/225000 [334:41:19<95:52:26,  6.96s/it]
 78%|███████▊  | 175397/225000 [334:41:23<81:57:55,  5.95s/it]
 78%|███████▊  | 175398/225000 [334:41:30<84:34:36,  6.14s/it]
 78%|███████▊  | 175399/225000 [334:41:34<78:34:53,  5.70s/it]
 78%|███████▊  | 175400/225000 [334:41:38<70:32:04,  5.12s/it]
                                                              
{'loss': '0.7658', 'grad_norm': '1.106', 'learning_rate': '6.303e-06', 'epoch': '0.7796', 'num_input_tokens_seen': 554594912, 'train_runtime': '1.205e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175400/225000 [334:41:38<70:32:04,  5.12s/it]
 78%|███████▊  | 175401/225000 [334:41:47<86:02:08,  6.24s/it]
 78%|███████▊  | 175402/225000 [334:41:51<75:30:26,  5.48s/it]
 78%|███████▊  | 175403/225000 [334:41:59<85:44:33,  6.22s/it]
 78%|███████▊  | 175404/225000 [334:42:02<75:00:58,  5.45s/it]
 78%|███████▊  | 175405/225000 [334:42:07<70:05:06,  5.09s/it]
 78%|███████▊  | 175406/225000 [334:42:12<71:20:14,  5.18s/it]
 78%|███████▊  | 175407/225000 [334:42:17<72:11:34,  5.24s/it]
 78%|███████▊  | 175408/225000 [334:42:23<72:13:21,  5.24s/it]
 78%|███████▊  | 175409/225000 [334:42:29<78:18:15,  5.68s/it]
 78%|███████▊  | 175410/225000 [334:42:36<82:56:42,  6.02s/it]
 78%|███████▊  | 175411/225000 [334:42:41<79:16:29,  5.76s/it]
 78%|███████▊  | 175412/225000 [334:42:46<74:03:04,  5.38s/it]
 78%|███████▊  | 175413/225000 [334:42:51<73:44:54,  5.35s/it]
 78%|███████▊  | 175414/225000 [334:42:56<71:25:37,  5.19s/it]
 78%|███████▊  | 175415/225000 [334:43:01<69:50:58,  5.07s/it]
 78%|███████▊  | 175416/225000 [334:43:06<69:44:43,  5.06s/it]
 78%|███████▊  | 175417/225000 [334:43:10<68:41:37,  4.99s/it]
 78%|███████▊  | 175418/225000 [334:43:16<69:36:58,  5.05s/it]
 78%|███████▊  | 175419/225000 [334:43:21<68:47:07,  4.99s/it]
 78%|███████▊  | 175420/225000 [334:43:25<67:30:04,  4.90s/it]
 78%|███████▊  | 175421/225000 [334:43:30<68:00:44,  4.94s/it]
 78%|███████▊  | 175422/225000 [334:43:35<68:10:12,  4.95s/it]
 78%|███████▊  | 175423/225000 [334:43:44<83:12:42,  6.04s/it]
 78%|███████▊  | 175424/225000 [334:43:53<95:48:58,  6.96s/it]
 78%|███████▊  | 175425/225000 [334:43:56<81:39:19,  5.93s/it]
 78%|███████▊  | 175426/225000 [334:44:02<80:50:47,  5.87s/it]
 78%|███████▊  | 175427/225000 [334:44:09<85:18:43,  6.20s/it]
 78%|███████▊  | 175428/225000 [334:44:12<73:42:49,  5.35s/it]
 78%|███████▊  | 175429/225000 [334:44:20<84:05:04,  6.11s/it]
 78%|███████▊  | 175430/225000 [334:44:24<73:08:36,  5.31s/it]
 78%|███████▊  | 175431/225000 [334:44:32<83:37:51,  6.07s/it]
 78%|███████▊  | 175432/225000 [334:44:35<74:05:12,  5.38s/it]
 78%|███████▊  | 175433/225000 [334:44:39<66:54:37,  4.86s/it]
 78%|███████▊  | 175434/225000 [334:44:43<62:41:09,  4.55s/it]
 78%|███████▊  | 175435/225000 [334:44:53<86:24:29,  6.28s/it]
 78%|███████▊  | 175436/225000 [334:44:58<80:14:14,  5.83s/it]
 78%|███████▊  | 175437/225000 [334:45:03<75:19:38,  5.47s/it]
 78%|███████▊  | 175438/225000 [334:45:08<72:54:57,  5.30s/it]
 78%|███████▊  | 175439/225000 [334:45:13<72:10:33,  5.24s/it]
 78%|███████▊  | 175440/225000 [334:45:17<70:30:28,  5.12s/it]
 78%|███████▊  | 175441/225000 [334:45:22<70:06:16,  5.09s/it]
 78%|███████▊  | 175442/225000 [334:45:27<68:42:51,  4.99s/it]
 78%|███████▊  | 175443/225000 [334:45:39<96:47:49,  7.03s/it]
 78%|███████▊  | 175444/225000 [334:45:42<81:15:18,  5.90s/it]
 78%|███████▊  | 175445/225000 [334:45:49<83:46:44,  6.09s/it]
 78%|███████▊  | 175446/225000 [334:45:53<76:23:50,  5.55s/it]
 78%|███████▊  | 175447/225000 [334:45:57<70:30:07,  5.12s/it]
 78%|███████▊  | 175448/225000 [334:46:03<74:34:37,  5.42s/it]
 78%|███████▊  | 175449/225000 [334:46:13<93:08:32,  6.77s/it]
 78%|███████▊  | 175450/225000 [334:46:16<78:16:01,  5.69s/it]
                                                              
{'loss': '0.7614', 'grad_norm': '1.124', 'learning_rate': '6.291e-06', 'epoch': '0.7798', 'num_input_tokens_seen': 554725216, 'train_runtime': '1.205e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175450/225000 [334:46:16<78:16:01,  5.69s/it]
 78%|███████▊  | 175451/225000 [334:46:20<69:59:42,  5.09s/it]
 78%|███████▊  | 175452/225000 [334:46:24<66:58:27,  4.87s/it]
 78%|███████▊  | 175453/225000 [334:46:29<66:53:54,  4.86s/it]
 78%|███████▊  | 175454/225000 [334:46:34<66:25:45,  4.83s/it]
 78%|███████▊  | 175455/225000 [334:46:39<66:46:56,  4.85s/it]
 78%|███████▊  | 175456/225000 [334:46:45<70:02:53,  5.09s/it]
 78%|███████▊  | 175457/225000 [334:46:55<93:32:38,  6.80s/it]
 78%|███████▊  | 175458/225000 [334:47:00<82:32:03,  6.00s/it]
 78%|███████▊  | 175459/225000 [334:47:03<73:37:55,  5.35s/it]
 78%|███████▊  | 175460/225000 [334:47:11<82:43:27,  6.01s/it]
 78%|███████▊  | 175461/225000 [334:47:18<85:42:10,  6.23s/it]
 78%|███████▊  | 175462/225000 [334:47:22<77:32:05,  5.63s/it]
 78%|███████▊  | 175463/225000 [334:47:26<72:30:55,  5.27s/it]
 78%|███████▊  | 175464/225000 [334:47:31<70:20:56,  5.11s/it]
 78%|███████▊  | 175465/225000 [334:47:36<67:34:22,  4.91s/it]
 78%|███████▊  | 175466/225000 [334:47:40<65:29:24,  4.76s/it]
 78%|███████▊  | 175467/225000 [334:47:44<64:08:17,  4.66s/it]
 78%|███████▊  | 175468/225000 [334:47:50<69:16:41,  5.04s/it]
 78%|███████▊  | 175469/225000 [334:48:01<93:43:58,  6.81s/it]
 78%|███████▊  | 175470/225000 [334:48:04<79:02:38,  5.75s/it]
 78%|███████▊  | 175471/225000 [334:48:08<70:24:01,  5.12s/it]
 78%|███████▊  | 175472/225000 [334:48:14<75:06:25,  5.46s/it]
 78%|███████▊  | 175473/225000 [334:48:22<84:19:26,  6.13s/it]
 78%|███████▊  | 175474/225000 [334:48:25<72:56:21,  5.30s/it]
 78%|███████▊  | 175475/225000 [334:48:33<82:54:06,  6.03s/it]
 78%|███████▊  | 175476/225000 [334:48:37<72:47:02,  5.29s/it]
 78%|███████▊  | 175477/225000 [334:48:44<78:56:00,  5.74s/it]
 78%|███████▊  | 175478/225000 [334:48:48<74:51:12,  5.44s/it]
 78%|███████▊  | 175479/225000 [334:48:53<71:50:09,  5.22s/it]
 78%|███████▊  | 175480/225000 [334:48:58<71:07:07,  5.17s/it]
 78%|███████▊  | 175481/225000 [334:49:03<70:07:37,  5.10s/it]
 78%|███████▊  | 175482/225000 [334:49:08<69:32:35,  5.06s/it]
 78%|███████▊  | 175483/225000 [334:49:13<68:35:27,  4.99s/it]
 78%|███████▊  | 175484/225000 [334:49:17<66:20:22,  4.82s/it]
 78%|███████▊  | 175485/225000 [334:49:22<66:42:56,  4.85s/it]
 78%|███████▊  | 175486/225000 [334:49:27<67:48:26,  4.93s/it]
 78%|███████▊  | 175487/225000 [334:49:32<67:36:42,  4.92s/it]
 78%|███████▊  | 175488/225000 [334:49:41<83:30:00,  6.07s/it]
 78%|███████▊  | 175489/225000 [334:49:50<97:25:44,  7.08s/it]
 78%|███████▊  | 175490/225000 [334:49:55<88:53:58,  6.46s/it]
 78%|███████▊  | 175491/225000 [334:50:00<82:04:23,  5.97s/it]
 78%|███████▊  | 175492/225000 [334:50:05<77:30:58,  5.64s/it]
 78%|███████▊  | 175493/225000 [334:50:10<74:12:51,  5.40s/it]
 78%|███████▊  | 175494/225000 [334:50:15<73:27:52,  5.34s/it]
 78%|███████▊  | 175495/225000 [334:50:20<72:26:09,  5.27s/it]
 78%|███████▊  | 175496/225000 [334:50:25<72:46:50,  5.29s/it]
 78%|███████▊  | 175497/225000 [334:50:31<71:57:58,  5.23s/it]
 78%|███████▊  | 175498/225000 [334:50:36<71:37:27,  5.21s/it]
 78%|███████▊  | 175499/225000 [334:50:41<71:40:50,  5.21s/it]
 78%|███████▊  | 175500/225000 [334:50:46<71:07:25,  5.17s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '1.192', 'learning_rate': '6.279e-06', 'epoch': '0.78', 'num_input_tokens_seen': 554854088, 'train_runtime': '1.205e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175500/225000 [334:50:46<71:07:25,  5.17s/it]
 78%|███████▊  | 175501/225000 [334:50:51<71:04:47,  5.17s/it]
 78%|███████▊  | 175502/225000 [334:50:57<72:06:06,  5.24s/it]
 78%|███████▊  | 175503/225000 [334:51:02<70:56:55,  5.16s/it]
 78%|███████▊  | 175504/225000 [334:51:07<71:11:44,  5.18s/it]
 78%|███████▊  | 175505/225000 [334:51:11<69:09:28,  5.03s/it]
 78%|███████▊  | 175506/225000 [334:51:21<87:40:58,  6.38s/it]
 78%|███████▊  | 175507/225000 [334:51:29<94:02:23,  6.84s/it]
 78%|███████▊  | 175508/225000 [334:51:33<84:37:33,  6.16s/it]
 78%|███████▊  | 175509/225000 [334:51:38<77:42:35,  5.65s/it]
 78%|███████▊  | 175510/225000 [334:51:43<74:00:55,  5.38s/it]
 78%|███████▊  | 175511/225000 [334:51:47<69:12:03,  5.03s/it]
 78%|███████▊  | 175512/225000 [334:51:52<67:41:25,  4.92s/it]
 78%|███████▊  | 175513/225000 [334:51:56<67:15:02,  4.89s/it]
 78%|███████▊  | 175514/225000 [334:52:07<90:38:24,  6.59s/it]
 78%|███████▊  | 175515/225000 [334:52:15<95:40:03,  6.96s/it]
 78%|███████▊  | 175516/225000 [334:52:18<81:13:55,  5.91s/it]
 78%|███████▊  | 175517/225000 [334:52:24<78:51:28,  5.74s/it]
 78%|███████▊  | 175518/225000 [334:52:30<79:33:06,  5.79s/it]
 78%|███████▊  | 175519/225000 [334:52:33<71:04:44,  5.17s/it]
 78%|███████▊  | 175520/225000 [334:52:43<91:25:14,  6.65s/it]
 78%|███████▊  | 175521/225000 [334:52:47<79:21:29,  5.77s/it]
 78%|███████▊  | 175522/225000 [334:52:54<84:48:24,  6.17s/it]
 78%|███████▊  | 175523/225000 [334:52:58<74:59:04,  5.46s/it]
 78%|███████▊  | 175524/225000 [334:53:03<72:37:05,  5.28s/it]
 78%|███████▊  | 175525/225000 [334:53:08<72:07:05,  5.25s/it]
 78%|███████▊  | 175526/225000 [334:53:13<71:51:55,  5.23s/it]
 78%|███████▊  | 175527/225000 [334:53:18<71:34:21,  5.21s/it]
 78%|███████▊  | 175528/225000 [334:53:23<69:42:48,  5.07s/it]
 78%|███████▊  | 175529/225000 [334:53:28<68:47:33,  5.01s/it]
 78%|███████▊  | 175530/225000 [334:53:33<69:43:02,  5.07s/it]
 78%|███████▊  | 175531/225000 [334:53:38<68:57:49,  5.02s/it]
 78%|███████▊  | 175532/225000 [334:53:43<69:50:51,  5.08s/it]
 78%|███████▊  | 175533/225000 [334:53:53<86:58:33,  6.33s/it]
 78%|███████▊  | 175534/225000 [334:54:00<92:26:00,  6.73s/it]
 78%|███████▊  | 175535/225000 [334:54:04<80:40:28,  5.87s/it]
 78%|███████▊  | 175536/225000 [334:54:08<73:58:50,  5.38s/it]
 78%|███████▊  | 175537/225000 [334:54:13<69:00:35,  5.02s/it]
 78%|███████▊  | 175538/225000 [334:54:23<91:05:58,  6.63s/it]
 78%|███████▊  | 175539/225000 [334:54:29<88:52:35,  6.47s/it]
 78%|███████▊  | 175540/225000 [334:54:32<76:26:10,  5.56s/it]
 78%|███████▊  | 175541/225000 [334:54:39<82:25:04,  6.00s/it]
 78%|███████▊  | 175542/225000 [334:54:43<71:23:15,  5.20s/it]
 78%|███████▊  | 175543/225000 [334:54:50<78:22:51,  5.71s/it]
 78%|███████▊  | 175544/225000 [334:54:54<72:24:21,  5.27s/it]
 78%|███████▊  | 175545/225000 [334:54:58<66:39:03,  4.85s/it]
 78%|███████▊  | 175546/225000 [334:55:02<62:02:58,  4.52s/it]
 78%|███████▊  | 175547/225000 [334:55:12<87:26:09,  6.37s/it]
 78%|███████▊  | 175548/225000 [334:55:16<77:09:21,  5.62s/it]
 78%|███████▊  | 175549/225000 [334:55:20<70:58:23,  5.17s/it]
 78%|███████▊  | 175550/225000 [334:55:24<65:26:14,  4.76s/it]
                                                              
{'loss': '0.76', 'grad_norm': '1.216', 'learning_rate': '6.267e-06', 'epoch': '0.7802', 'num_input_tokens_seen': 554985032, 'train_runtime': '1.206e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175550/225000 [334:55:24<65:26:14,  4.76s/it]
 78%|███████▊  | 175551/225000 [334:55:30<72:19:09,  5.27s/it]
 78%|███████▊  | 175552/225000 [334:55:38<83:34:42,  6.08s/it]
 78%|███████▊  | 175553/225000 [334:55:42<71:40:27,  5.22s/it]
 78%|███████▊  | 175554/225000 [334:55:49<81:45:33,  5.95s/it]
 78%|███████▊  | 175555/225000 [334:55:53<72:03:15,  5.25s/it]
 78%|███████▊  | 175556/225000 [334:55:58<70:35:21,  5.14s/it]
 78%|███████▊  | 175557/225000 [334:56:03<70:22:11,  5.12s/it]
 78%|███████▊  | 175558/225000 [334:56:08<68:55:18,  5.02s/it]
 78%|███████▊  | 175559/225000 [334:56:12<66:13:40,  4.82s/it]
 78%|███████▊  | 175560/225000 [334:56:17<66:36:31,  4.85s/it]
 78%|███████▊  | 175561/225000 [334:56:22<65:55:19,  4.80s/it]
 78%|███████▊  | 175562/225000 [334:56:27<67:12:16,  4.89s/it]
 78%|███████▊  | 175563/225000 [334:56:33<71:33:15,  5.21s/it]
 78%|███████▊  | 175564/225000 [334:56:38<70:46:31,  5.15s/it]
 78%|███████▊  | 175565/225000 [334:56:43<70:14:23,  5.12s/it]
 78%|███████▊  | 175566/225000 [334:56:47<68:13:53,  4.97s/it]
 78%|███████▊  | 175567/225000 [334:56:54<75:05:47,  5.47s/it]
 78%|███████▊  | 175568/225000 [334:57:07<104:30:53,  7.61s/it]
 78%|███████▊  | 175569/225000 [334:57:11<89:43:08,  6.53s/it] 
 78%|███████▊  | 175570/225000 [334:57:18<93:34:21,  6.81s/it]
 78%|███████▊  | 175571/225000 [334:57:22<80:52:04,  5.89s/it]
 78%|███████▊  | 175572/225000 [334:57:26<72:13:27,  5.26s/it]
 78%|███████▊  | 175573/225000 [334:57:35<89:19:08,  6.51s/it]
 78%|███████▊  | 175574/225000 [334:57:39<77:19:46,  5.63s/it]
 78%|███████▊  | 175575/225000 [334:57:46<85:11:58,  6.21s/it]
 78%|███████▊  | 175576/225000 [334:57:50<77:19:33,  5.63s/it]
 78%|███████▊  | 175577/225000 [334:57:55<70:49:19,  5.16s/it]
 78%|███████▊  | 175578/225000 [334:58:03<85:04:05,  6.20s/it]
 78%|███████▊  | 175579/225000 [334:58:07<76:56:45,  5.61s/it]
 78%|███████▊  | 175580/225000 [334:58:16<87:25:10,  6.37s/it]
 78%|███████▊  | 175581/225000 [334:58:19<75:42:46,  5.52s/it]
 78%|███████▊  | 175582/225000 [334:58:27<87:29:52,  6.37s/it]
 78%|███████▊  | 175583/225000 [334:58:31<75:26:11,  5.50s/it]
 78%|███████▊  | 175584/225000 [334:58:35<71:47:47,  5.23s/it]
 78%|███████▊  | 175585/225000 [334:58:41<72:12:12,  5.26s/it]
 78%|███████▊  | 175586/225000 [334:58:47<77:32:07,  5.65s/it]
 78%|███████▊  | 175587/225000 [334:58:51<68:47:13,  5.01s/it]
 78%|███████▊  | 175588/225000 [334:58:55<65:27:30,  4.77s/it]
 78%|███████▊  | 175589/225000 [334:59:00<66:09:54,  4.82s/it]
 78%|███████▊  | 175590/225000 [334:59:09<81:57:41,  5.97s/it]
 78%|███████▊  | 175591/225000 [334:59:14<78:24:56,  5.71s/it]
 78%|███████▊  | 175592/225000 [334:59:19<76:21:16,  5.56s/it]
 78%|███████▊  | 175593/225000 [334:59:24<74:25:01,  5.42s/it]
 78%|███████▊  | 175594/225000 [334:59:29<72:35:12,  5.29s/it]
 78%|███████▊  | 175595/225000 [334:59:34<70:51:42,  5.16s/it]
 78%|███████▊  | 175596/225000 [334:59:39<71:19:01,  5.20s/it]
 78%|███████▊  | 175597/225000 [334:59:44<69:08:05,  5.04s/it]
 78%|███████▊  | 175598/225000 [334:59:49<67:58:37,  4.95s/it]
 78%|███████▊  | 175599/225000 [334:59:54<68:42:13,  5.01s/it]
 78%|███████▊  | 175600/225000 [334:59:59<68:34:41,  5.00s/it]
                                                              
{'loss': '0.7594', 'grad_norm': '1.282', 'learning_rate': '6.255e-06', 'epoch': '0.7804', 'num_input_tokens_seen': 555115552, 'train_runtime': '1.206e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175600/225000 [334:59:59<68:34:41,  5.00s/it]
 78%|███████▊  | 175601/225000 [335:00:04<68:42:57,  5.01s/it]
 78%|███████▊  | 175602/225000 [335:00:15<96:11:02,  7.01s/it]
 78%|███████▊  | 175603/225000 [335:00:19<81:16:10,  5.92s/it]
 78%|███████▊  | 175604/225000 [335:00:28<92:47:54,  6.76s/it]
 78%|███████▊  | 175605/225000 [335:00:31<79:12:18,  5.77s/it]
 78%|███████▊  | 175606/225000 [335:00:34<69:03:34,  5.03s/it]
 78%|███████▊  | 175607/225000 [335:00:41<77:24:20,  5.64s/it]
 78%|███████▊  | 175608/225000 [335:00:45<67:49:05,  4.94s/it]
 78%|███████▊  | 175609/225000 [335:00:49<66:18:36,  4.83s/it]
 78%|███████▊  | 175610/225000 [335:00:53<62:39:41,  4.57s/it]
 78%|███████▊  | 175611/225000 [335:00:58<62:42:09,  4.57s/it]
 78%|███████▊  | 175612/225000 [335:01:02<62:38:33,  4.57s/it]
 78%|███████▊  | 175613/225000 [335:01:15<93:53:46,  6.84s/it]
 78%|███████▊  | 175614/225000 [335:01:19<83:52:53,  6.11s/it]
 78%|███████▊  | 175615/225000 [335:01:22<72:26:14,  5.28s/it]
 78%|███████▊  | 175616/225000 [335:01:31<88:04:00,  6.42s/it]
 78%|███████▊  | 175617/225000 [335:01:37<82:49:29,  6.04s/it]
 78%|███████▊  | 175618/225000 [335:01:40<74:00:08,  5.39s/it]
 78%|███████▊  | 175619/225000 [335:01:45<69:35:14,  5.07s/it]
 78%|███████▊  | 175620/225000 [335:01:49<65:25:07,  4.77s/it]
 78%|███████▊  | 175621/225000 [335:01:56<74:45:32,  5.45s/it]
 78%|███████▊  | 175622/225000 [335:02:03<83:35:00,  6.09s/it]
 78%|███████▊  | 175623/225000 [335:02:08<79:21:41,  5.79s/it]
 78%|███████▊  | 175624/225000 [335:02:16<84:55:07,  6.19s/it]
 78%|███████▊  | 175625/225000 [335:02:19<75:05:55,  5.48s/it]
 78%|███████▊  | 175626/225000 [335:02:28<87:02:18,  6.35s/it]
 78%|███████▊  | 175627/225000 [335:02:31<75:59:17,  5.54s/it]
 78%|███████▊  | 175628/225000 [335:02:35<68:20:42,  4.98s/it]
 78%|███████▊  | 175629/225000 [335:02:40<66:31:47,  4.85s/it]
 78%|███████▊  | 175630/225000 [335:02:44<63:47:54,  4.65s/it]
 78%|███████▊  | 175631/225000 [335:02:54<85:51:43,  6.26s/it]
 78%|███████▊  | 175632/225000 [335:02:59<79:30:16,  5.80s/it]
 78%|███████▊  | 175633/225000 [335:03:02<69:54:58,  5.10s/it]
 78%|███████▊  | 175634/225000 [335:03:10<81:28:30,  5.94s/it]
 78%|███████▊  | 175635/225000 [335:03:16<79:53:21,  5.83s/it]
 78%|███████▊  | 175636/225000 [335:03:19<71:38:59,  5.23s/it]
 78%|███████▊  | 175637/225000 [335:03:24<67:54:42,  4.95s/it]
 78%|███████▊  | 175638/225000 [335:03:32<81:05:24,  5.91s/it]
 78%|███████▊  | 175639/225000 [335:03:35<69:35:53,  5.08s/it]
 78%|███████▊  | 175640/225000 [335:03:39<63:33:14,  4.64s/it]
 78%|███████▊  | 175641/225000 [335:03:45<72:00:17,  5.25s/it]
 78%|███████▊  | 175642/225000 [335:03:54<86:30:26,  6.31s/it]
 78%|███████▊  | 175643/225000 [335:03:57<74:14:37,  5.42s/it]
 78%|███████▊  | 175644/225000 [335:04:06<86:42:46,  6.32s/it]
 78%|███████▊  | 175645/225000 [335:04:09<74:46:31,  5.45s/it]
 78%|███████▊  | 175646/225000 [335:04:15<76:49:19,  5.60s/it]
 78%|███████▊  | 175647/225000 [335:04:21<75:46:14,  5.53s/it]
 78%|███████▊  | 175648/225000 [335:04:24<69:08:58,  5.04s/it]
 78%|███████▊  | 175649/225000 [335:04:32<79:16:32,  5.78s/it]
 78%|███████▊  | 175650/225000 [335:04:36<73:02:32,  5.33s/it]
                                                              
{'loss': '0.77', 'grad_norm': '1.182', 'learning_rate': '6.243e-06', 'epoch': '0.7807', 'num_input_tokens_seen': 555245056, 'train_runtime': '1.206e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175650/225000 [335:04:36<73:02:32,  5.33s/it]
 78%|███████▊  | 175651/225000 [335:04:40<66:13:01,  4.83s/it]
 78%|███████▊  | 175652/225000 [335:04:46<72:41:14,  5.30s/it]
 78%|███████▊  | 175653/225000 [335:04:53<77:54:32,  5.68s/it]
 78%|███████▊  | 175654/225000 [335:04:56<68:08:47,  4.97s/it]
 78%|███████▊  | 175655/225000 [335:05:05<84:38:53,  6.18s/it]
 78%|███████▊  | 175656/225000 [335:05:08<72:10:21,  5.27s/it]
 78%|███████▊  | 175657/225000 [335:05:13<71:33:32,  5.22s/it]
 78%|███████▊  | 175658/225000 [335:05:18<70:30:30,  5.14s/it]
 78%|███████▊  | 175659/225000 [335:05:26<80:07:42,  5.85s/it]
 78%|███████▊  | 175660/225000 [335:05:32<80:19:27,  5.86s/it]
 78%|███████▊  | 175661/225000 [335:05:36<75:26:58,  5.51s/it]
 78%|███████▊  | 175662/225000 [335:05:40<68:56:42,  5.03s/it]
 78%|███████▊  | 175663/225000 [335:05:45<67:04:51,  4.89s/it]
 78%|███████▊  | 175664/225000 [335:05:49<64:50:43,  4.73s/it]
 78%|███████▊  | 175665/225000 [335:06:00<91:03:18,  6.64s/it]
 78%|███████▊  | 175666/225000 [335:06:05<83:27:14,  6.09s/it]
 78%|███████▊  | 175667/225000 [335:06:11<81:23:42,  5.94s/it]
 78%|███████▊  | 175668/225000 [335:06:16<77:48:07,  5.68s/it]
 78%|███████▊  | 175669/225000 [335:06:22<80:15:18,  5.86s/it]
 78%|███████▊  | 175670/225000 [335:06:28<81:30:30,  5.95s/it]
 78%|███████▊  | 175671/225000 [335:06:32<72:04:25,  5.26s/it]
 78%|███████▊  | 175672/225000 [335:06:35<63:44:58,  4.65s/it]
 78%|███████▊  | 175673/225000 [335:06:40<65:50:24,  4.81s/it]
 78%|███████▊  | 175674/225000 [335:06:45<67:06:12,  4.90s/it]
 78%|███████▊  | 175675/225000 [335:06:51<71:27:16,  5.22s/it]
 78%|███████▊  | 175676/225000 [335:06:57<71:21:22,  5.21s/it]
 78%|███████▊  | 175677/225000 [335:07:02<72:26:37,  5.29s/it]
 78%|███████▊  | 175678/225000 [335:07:08<73:46:33,  5.38s/it]
 78%|███████▊  | 175679/225000 [335:07:13<73:20:06,  5.35s/it]
 78%|███████▊  | 175680/225000 [335:07:18<73:16:46,  5.35s/it]
 78%|███████▊  | 175681/225000 [335:07:24<74:06:57,  5.41s/it]
 78%|███████▊  | 175682/225000 [335:07:29<73:29:46,  5.36s/it]
 78%|███████▊  | 175683/225000 [335:07:35<75:20:57,  5.50s/it]
 78%|███████▊  | 175684/225000 [335:07:41<76:02:41,  5.55s/it]
 78%|███████▊  | 175685/225000 [335:07:46<75:47:06,  5.53s/it]
 78%|███████▊  | 175686/225000 [335:07:55<87:44:52,  6.41s/it]
 78%|███████▊  | 175687/225000 [335:08:01<89:43:23,  6.55s/it]
 78%|███████▊  | 175688/225000 [335:08:07<87:18:32,  6.37s/it]
 78%|███████▊  | 175689/225000 [335:08:12<81:37:01,  5.96s/it]
 78%|███████▊  | 175690/225000 [335:08:16<71:32:25,  5.22s/it]
 78%|███████▊  | 175691/225000 [335:08:24<85:14:06,  6.22s/it]
 78%|███████▊  | 175692/225000 [335:08:28<75:47:52,  5.53s/it]
 78%|███████▊  | 175693/225000 [335:08:36<85:17:37,  6.23s/it]
 78%|███████▊  | 175694/225000 [335:08:40<74:51:48,  5.47s/it]
 78%|███████▊  | 175695/225000 [335:08:44<67:41:17,  4.94s/it]
 78%|███████▊  | 175696/225000 [335:08:52<80:34:17,  5.88s/it]
 78%|███████▊  | 175697/225000 [335:08:58<81:35:49,  5.96s/it]
 78%|███████▊  | 175698/225000 [335:09:02<72:35:24,  5.30s/it]
 78%|███████▊  | 175699/225000 [335:09:06<67:46:05,  4.95s/it]
 78%|███████▊  | 175700/225000 [335:09:10<63:35:18,  4.64s/it]
                                                              
{'loss': '0.7703', 'grad_norm': '1.263', 'learning_rate': '6.231e-06', 'epoch': '0.7809', 'num_input_tokens_seen': 555375664, 'train_runtime': '1.207e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175700/225000 [335:09:10<63:35:18,  4.64s/it]
 78%|███████▊  | 175701/225000 [335:09:18<80:45:45,  5.90s/it]
 78%|███████▊  | 175702/225000 [335:09:22<72:50:59,  5.32s/it]
 78%|███████▊  | 175703/225000 [335:09:26<65:57:58,  4.82s/it]
 78%|███████▊  | 175704/225000 [335:09:30<61:48:02,  4.51s/it]
 78%|███████▊  | 175705/225000 [335:09:39<81:53:10,  5.98s/it]
 78%|███████▊  | 175706/225000 [335:09:43<72:16:47,  5.28s/it]
 78%|███████▊  | 175707/225000 [335:09:47<69:06:33,  5.05s/it]
 78%|███████▊  | 175708/225000 [335:09:56<85:09:48,  6.22s/it]
 78%|███████▊  | 175709/225000 [335:10:00<74:23:05,  5.43s/it]
 78%|███████▊  | 175710/225000 [335:10:08<86:20:29,  6.31s/it]
 78%|███████▊  | 175711/225000 [335:10:12<76:39:06,  5.60s/it]
 78%|███████▊  | 175712/225000 [335:10:17<71:30:09,  5.22s/it]
 78%|███████▊  | 175713/225000 [335:10:21<68:32:59,  5.01s/it]
 78%|███████▊  | 175714/225000 [335:10:26<66:11:59,  4.84s/it]
 78%|███████▊  | 175715/225000 [335:10:30<65:23:12,  4.78s/it]
 78%|███████▊  | 175716/225000 [335:10:41<89:06:09,  6.51s/it]
 78%|███████▊  | 175717/225000 [335:10:45<78:10:23,  5.71s/it]
 78%|███████▊  | 175718/225000 [335:10:49<72:11:06,  5.27s/it]
 78%|███████▊  | 175719/225000 [335:10:53<66:57:58,  4.89s/it]
 78%|███████▊  | 175720/225000 [335:11:03<90:03:36,  6.58s/it]
 78%|███████▊  | 175721/225000 [335:11:07<76:27:45,  5.59s/it]
 78%|███████▊  | 175722/225000 [335:11:11<69:41:07,  5.09s/it]
 78%|███████▊  | 175723/225000 [335:11:19<83:49:20,  6.12s/it]
 78%|███████▊  | 175724/225000 [335:11:23<75:25:21,  5.51s/it]
 78%|███████▊  | 175725/225000 [335:11:27<67:54:19,  4.96s/it]
 78%|███████▊  | 175726/225000 [335:11:30<61:44:41,  4.51s/it]
 78%|███████▊  | 175727/225000 [335:11:39<80:18:55,  5.87s/it]
 78%|███████▊  | 175728/225000 [335:11:43<72:58:04,  5.33s/it]
 78%|███████▊  | 175729/225000 [335:11:49<75:01:33,  5.48s/it]
 78%|███████▊  | 175730/225000 [335:11:55<77:25:33,  5.66s/it]
 78%|███████▊  | 175731/225000 [335:12:01<76:16:28,  5.57s/it]
 78%|███████▊  | 175732/225000 [335:12:07<80:41:39,  5.90s/it]
 78%|███████▊  | 175733/225000 [335:12:11<69:44:01,  5.10s/it]
 78%|███████▊  | 175734/225000 [335:12:15<67:34:18,  4.94s/it]
 78%|███████▊  | 175735/225000 [335:12:19<63:56:23,  4.67s/it]
 78%|███████▊  | 175736/225000 [335:12:24<65:02:32,  4.75s/it]
 78%|███████▊  | 175737/225000 [335:12:29<64:45:24,  4.73s/it]
 78%|███████▊  | 175738/225000 [335:12:34<67:27:33,  4.93s/it]
 78%|███████▊  | 175739/225000 [335:12:39<67:58:05,  4.97s/it]
 78%|███████▊  | 175740/225000 [335:12:44<65:56:59,  4.82s/it]
 78%|███████▊  | 175741/225000 [335:12:49<65:50:17,  4.81s/it]
 78%|███████▊  | 175742/225000 [335:13:00<93:42:37,  6.85s/it]
 78%|███████▊  | 175743/225000 [335:13:06<87:36:42,  6.40s/it]
 78%|███████▊  | 175744/225000 [335:13:10<81:13:29,  5.94s/it]
 78%|███████▊  | 175745/225000 [335:13:15<75:26:19,  5.51s/it]
 78%|███████▊  | 175746/225000 [335:13:20<72:14:27,  5.28s/it]
 78%|███████▊  | 175747/225000 [335:13:24<70:15:47,  5.14s/it]
 78%|███████▊  | 175748/225000 [335:13:29<68:09:00,  4.98s/it]
 78%|███████▊  | 175749/225000 [335:13:34<67:37:28,  4.94s/it]
 78%|███████▊  | 175750/225000 [335:13:39<69:06:01,  5.05s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.152', 'learning_rate': '6.219e-06', 'epoch': '0.7811', 'num_input_tokens_seen': 555505632, 'train_runtime': '1.207e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175750/225000 [335:13:39<69:06:01,  5.05s/it]
 78%|███████▊  | 175751/225000 [335:13:44<69:01:24,  5.05s/it]
 78%|███████▊  | 175752/225000 [335:13:49<69:40:34,  5.09s/it]
 78%|███████▊  | 175753/225000 [335:13:54<68:44:38,  5.03s/it]
 78%|███████▊  | 175754/225000 [335:14:00<69:19:51,  5.07s/it]
 78%|███████▊  | 175755/225000 [335:14:04<68:53:21,  5.04s/it]
 78%|███████▊  | 175756/225000 [335:14:10<71:17:07,  5.21s/it]
 78%|███████▊  | 175757/225000 [335:14:22<99:42:30,  7.29s/it]
 78%|███████▊  | 175758/225000 [335:14:28<94:30:42,  6.91s/it]
 78%|███████▊  | 175759/225000 [335:14:32<80:49:38,  5.91s/it]
 78%|███████▊  | 175760/225000 [335:14:36<73:03:27,  5.34s/it]
 78%|███████▊  | 175761/225000 [335:14:43<80:01:25,  5.85s/it]
 78%|███████▊  | 175762/225000 [335:14:50<83:55:44,  6.14s/it]
 78%|███████▊  | 175763/225000 [335:14:53<73:12:42,  5.35s/it]
 78%|███████▊  | 175764/225000 [335:15:00<80:34:01,  5.89s/it]
 78%|███████▊  | 175765/225000 [335:15:07<83:20:04,  6.09s/it]
 78%|███████▊  | 175766/225000 [335:15:10<72:02:18,  5.27s/it]
 78%|███████▊  | 175767/225000 [335:15:14<65:20:48,  4.78s/it]
 78%|███████▊  | 175768/225000 [335:15:23<83:47:54,  6.13s/it]
 78%|███████▊  | 175769/225000 [335:15:28<77:28:03,  5.66s/it]
 78%|███████▊  | 175770/225000 [335:15:32<70:43:58,  5.17s/it]
 78%|███████▊  | 175771/225000 [335:15:36<67:49:22,  4.96s/it]
 78%|███████▊  | 175772/225000 [335:15:41<67:15:58,  4.92s/it]
 78%|███████▊  | 175773/225000 [335:15:46<65:17:37,  4.77s/it]
 78%|███████▊  | 175774/225000 [335:15:50<64:40:50,  4.73s/it]
 78%|███████▊  | 175775/225000 [335:16:02<94:53:20,  6.94s/it]
 78%|███████▊  | 175776/225000 [335:16:06<83:49:04,  6.13s/it]
 78%|███████▊  | 175777/225000 [335:16:10<73:30:12,  5.38s/it]
 78%|███████▊  | 175778/225000 [335:16:18<85:20:48,  6.24s/it]
 78%|███████▊  | 175779/225000 [335:16:23<80:14:44,  5.87s/it]
 78%|███████▊  | 175780/225000 [335:16:27<73:08:47,  5.35s/it]
 78%|███████▊  | 175781/225000 [335:16:32<67:46:33,  4.96s/it]
 78%|███████▊  | 175782/225000 [335:16:36<66:18:54,  4.85s/it]
 78%|███████▊  | 175783/225000 [335:16:41<67:51:52,  4.96s/it]
 78%|███████▊  | 175784/225000 [335:16:53<93:53:47,  6.87s/it]
 78%|███████▊  | 175785/225000 [335:16:57<83:10:46,  6.08s/it]
 78%|███████▊  | 175786/225000 [335:17:01<75:34:27,  5.53s/it]
 78%|███████▊  | 175787/225000 [335:17:06<70:45:15,  5.18s/it]
 78%|███████▊  | 175788/225000 [335:17:10<69:07:22,  5.06s/it]
 78%|███████▊  | 175789/225000 [335:17:14<65:30:43,  4.79s/it]
 78%|███████▊  | 175790/225000 [335:17:19<63:42:24,  4.66s/it]
 78%|███████▊  | 175791/225000 [335:17:28<82:25:54,  6.03s/it]
 78%|███████▊  | 175792/225000 [335:17:36<89:20:54,  6.54s/it]
 78%|███████▊  | 175793/225000 [335:17:39<76:13:17,  5.58s/it]
 78%|███████▊  | 175794/225000 [335:17:46<81:43:49,  5.98s/it]
 78%|███████▊  | 175795/225000 [335:17:50<74:57:11,  5.48s/it]
 78%|███████▊  | 175796/225000 [335:17:55<70:57:21,  5.19s/it]
 78%|███████▊  | 175797/225000 [335:17:59<67:13:26,  4.92s/it]
 78%|███████▊  | 175798/225000 [335:18:04<65:43:37,  4.81s/it]
 78%|███████▊  | 175799/225000 [335:18:08<64:19:21,  4.71s/it]
 78%|███████▊  | 175800/225000 [335:18:20<93:20:45,  6.83s/it]
                                                              
{'loss': '0.7603', 'grad_norm': '1.186', 'learning_rate': '6.207e-06', 'epoch': '0.7813', 'num_input_tokens_seen': 555635384, 'train_runtime': '1.207e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175800/225000 [335:18:20<93:20:45,  6.83s/it]
 78%|███████▊  | 175801/225000 [335:18:23<79:32:22,  5.82s/it]
 78%|███████▊  | 175802/225000 [335:18:29<78:59:09,  5.78s/it]
 78%|███████▊  | 175803/225000 [335:18:36<83:09:08,  6.08s/it]
 78%|███████▊  | 175804/225000 [335:18:40<74:21:29,  5.44s/it]
 78%|███████▊  | 175805/225000 [335:18:44<67:19:21,  4.93s/it]
 78%|███████▊  | 175806/225000 [335:18:47<62:33:05,  4.58s/it]
 78%|███████▊  | 175807/225000 [335:18:59<92:34:06,  6.77s/it]
 78%|███████▊  | 175808/225000 [335:19:03<78:58:30,  5.78s/it]
 78%|███████▊  | 175809/225000 [335:19:07<73:20:38,  5.37s/it]
 78%|███████▊  | 175810/225000 [335:19:16<86:51:34,  6.36s/it]
 78%|███████▊  | 175811/225000 [335:19:19<73:49:45,  5.40s/it]
 78%|███████▊  | 175812/225000 [335:19:27<82:51:23,  6.06s/it]
 78%|███████▊  | 175813/225000 [335:19:30<73:04:06,  5.35s/it]
 78%|███████▊  | 175814/225000 [335:19:34<66:04:42,  4.84s/it]
 78%|███████▊  | 175815/225000 [335:19:42<80:53:15,  5.92s/it]
 78%|███████▊  | 175816/225000 [335:19:47<77:42:21,  5.69s/it]
 78%|███████▊  | 175817/225000 [335:19:52<72:03:54,  5.27s/it]
 78%|███████▊  | 175818/225000 [335:19:57<70:00:05,  5.12s/it]
 78%|███████▊  | 175819/225000 [335:20:02<69:23:17,  5.08s/it]
 78%|███████▊  | 175820/225000 [335:20:07<69:38:37,  5.10s/it]
 78%|███████▊  | 175821/225000 [335:20:11<68:36:59,  5.02s/it]
 78%|███████▊  | 175822/225000 [335:20:16<66:56:25,  4.90s/it]
 78%|███████▊  | 175823/225000 [335:20:21<66:41:29,  4.88s/it]
 78%|███████▊  | 175824/225000 [335:20:26<66:04:39,  4.84s/it]
 78%|███████▊  | 175825/225000 [335:20:36<88:03:53,  6.45s/it]
 78%|███████▊  | 175826/225000 [335:20:42<88:33:15,  6.48s/it]
 78%|███████▊  | 175827/225000 [335:20:46<75:34:31,  5.53s/it]
 78%|███████▊  | 175828/225000 [335:20:54<85:45:17,  6.28s/it]
 78%|███████▊  | 175829/225000 [335:20:57<74:14:21,  5.44s/it]
 78%|███████▊  | 175830/225000 [335:21:01<65:25:29,  4.79s/it]
 78%|███████▊  | 175831/225000 [335:21:11<86:48:39,  6.36s/it]
 78%|███████▊  | 175832/225000 [335:21:14<76:09:24,  5.58s/it]
 78%|███████▊  | 175833/225000 [335:21:18<68:32:57,  5.02s/it]
 78%|███████▊  | 175834/225000 [335:21:22<64:44:43,  4.74s/it]
 78%|███████▊  | 175835/225000 [335:21:32<84:58:13,  6.22s/it]
 78%|███████▊  | 175836/225000 [335:21:35<74:22:49,  5.45s/it]
 78%|███████▊  | 175837/225000 [335:21:43<83:18:46,  6.10s/it]
 78%|███████▊  | 175838/225000 [335:21:46<71:59:17,  5.27s/it]
 78%|███████▊  | 175839/225000 [335:21:50<66:23:10,  4.86s/it]
 78%|███████▊  | 175840/225000 [335:21:55<65:48:23,  4.82s/it]
 78%|███████▊  | 175841/225000 [335:22:00<65:31:28,  4.80s/it]
 78%|███████▊  | 175842/225000 [335:22:04<64:49:34,  4.75s/it]
 78%|███████▊  | 175843/225000 [335:22:09<66:00:53,  4.83s/it]
 78%|███████▊  | 175844/225000 [335:22:21<92:42:42,  6.79s/it]
 78%|███████▊  | 175845/225000 [335:22:24<79:46:21,  5.84s/it]
 78%|███████▊  | 175846/225000 [335:22:28<70:21:03,  5.15s/it]
 78%|███████▊  | 175847/225000 [335:22:38<91:10:36,  6.68s/it]
 78%|███████▊  | 175848/225000 [335:22:43<81:54:28,  6.00s/it]
 78%|███████▊  | 175849/225000 [335:22:46<73:02:50,  5.35s/it]
 78%|███████▊  | 175850/225000 [335:22:51<67:58:14,  4.98s/it]
                                                              
{'loss': '0.7706', 'grad_norm': '1.23', 'learning_rate': '6.195e-06', 'epoch': '0.7816', 'num_input_tokens_seen': 555763560, 'train_runtime': '1.207e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175850/225000 [335:22:51<67:58:14,  4.98s/it]
 78%|███████▊  | 175851/225000 [335:22:55<65:12:35,  4.78s/it]
 78%|███████▊  | 175852/225000 [335:22:59<62:43:42,  4.59s/it]
 78%|███████▊  | 175853/225000 [335:23:12<97:09:28,  7.12s/it]
 78%|███████▊  | 175854/225000 [335:23:17<86:36:05,  6.34s/it]
 78%|███████▊  | 175855/225000 [335:23:21<79:21:51,  5.81s/it]
 78%|███████▊  | 175856/225000 [335:23:26<74:15:06,  5.44s/it]
 78%|███████▊  | 175857/225000 [335:23:30<70:16:08,  5.15s/it]
 78%|███████▊  | 175858/225000 [335:23:35<67:14:12,  4.93s/it]
 78%|███████▊  | 175859/225000 [335:23:39<66:36:43,  4.88s/it]
 78%|███████▊  | 175860/225000 [335:23:44<66:01:45,  4.84s/it]
 78%|███████▊  | 175861/225000 [335:23:52<78:24:03,  5.74s/it]
 78%|███████▊  | 175862/225000 [335:24:00<86:41:35,  6.35s/it]
 78%|███████▊  | 175863/225000 [335:24:03<75:59:59,  5.57s/it]
 78%|███████▊  | 175864/225000 [335:24:12<86:27:55,  6.33s/it]
 78%|███████▊  | 175865/225000 [335:24:15<74:52:40,  5.49s/it]
 78%|███████▊  | 175866/225000 [335:24:22<82:10:21,  6.02s/it]
 78%|███████▊  | 175867/225000 [335:24:26<73:11:07,  5.36s/it]
 78%|███████▊  | 175868/225000 [335:24:30<65:27:47,  4.80s/it]
 78%|███████▊  | 175869/225000 [335:24:39<84:38:36,  6.20s/it]
 78%|███████▊  | 175870/225000 [335:24:44<77:09:14,  5.65s/it]
 78%|███████▊  | 175871/225000 [335:24:47<69:05:08,  5.06s/it]
 78%|███████▊  | 175872/225000 [335:24:55<78:54:24,  5.78s/it]
 78%|███████▊  | 175873/225000 [335:24:59<73:53:51,  5.42s/it]
 78%|███████▊  | 175874/225000 [335:25:06<78:56:48,  5.79s/it]
 78%|███████▊  | 175875/225000 [335:25:10<71:17:33,  5.22s/it]
 78%|███████▊  | 175876/225000 [335:25:14<66:27:11,  4.87s/it]
 78%|███████▊  | 175877/225000 [335:25:18<62:54:47,  4.61s/it]
 78%|███████▊  | 175878/225000 [335:25:21<58:44:13,  4.30s/it]
 78%|███████▊  | 175879/225000 [335:25:32<84:21:38,  6.18s/it]
 78%|███████▊  | 175880/225000 [335:25:36<76:27:11,  5.60s/it]
 78%|███████▊  | 175881/225000 [335:25:41<72:24:07,  5.31s/it]
 78%|███████▊  | 175882/225000 [335:25:45<69:26:17,  5.09s/it]
 78%|███████▊  | 175883/225000 [335:25:50<66:49:59,  4.90s/it]
 78%|███████▊  | 175884/225000 [335:25:54<65:07:50,  4.77s/it]
 78%|███████▊  | 175885/225000 [335:26:08<99:38:55,  7.30s/it]
 78%|███████▊  | 175886/225000 [335:26:12<85:58:52,  6.30s/it]
 78%|███████▊  | 175887/225000 [335:26:17<83:24:40,  6.11s/it]
 78%|███████▊  | 175888/225000 [335:26:23<82:46:50,  6.07s/it]
 78%|███████▊  | 175889/225000 [335:26:27<74:20:49,  5.45s/it]
 78%|███████▊  | 175890/225000 [335:26:31<66:49:35,  4.90s/it]
 78%|███████▊  | 175891/225000 [335:26:40<83:18:33,  6.11s/it]
 78%|███████▊  | 175892/225000 [335:26:44<73:49:15,  5.41s/it]
 78%|███████▊  | 175893/225000 [335:26:49<75:07:55,  5.51s/it]
 78%|███████▊  | 175894/225000 [335:26:55<75:24:03,  5.53s/it]
 78%|███████▊  | 175895/225000 [335:27:03<87:25:27,  6.41s/it]
 78%|███████▊  | 175896/225000 [335:27:07<74:50:22,  5.49s/it]
 78%|███████▊  | 175897/225000 [335:27:11<68:53:07,  5.05s/it]
 78%|███████▊  | 175898/225000 [335:27:15<65:27:24,  4.80s/it]
 78%|███████▊  | 175899/225000 [335:27:19<64:27:49,  4.73s/it]
 78%|███████▊  | 175900/225000 [335:27:24<65:43:57,  4.82s/it]
                                                              
{'loss': '0.7642', 'grad_norm': '1.134', 'learning_rate': '6.183e-06', 'epoch': '0.7818', 'num_input_tokens_seen': 555893256, 'train_runtime': '1.208e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175900/225000 [335:27:24<65:43:57,  4.82s/it]
 78%|███████▊  | 175901/225000 [335:27:37<95:46:23,  7.02s/it]
 78%|███████▊  | 175902/225000 [335:27:40<80:24:53,  5.90s/it]
 78%|███████▊  | 175903/225000 [335:27:43<69:35:18,  5.10s/it]
 78%|███████▊  | 175904/225000 [335:27:48<70:19:28,  5.16s/it]
 78%|███████▊  | 175905/225000 [335:27:54<71:24:03,  5.24s/it]
 78%|███████▊  | 175906/225000 [335:27:59<71:02:36,  5.21s/it]
 78%|███████▊  | 175907/225000 [335:28:05<72:57:49,  5.35s/it]
 78%|███████▊  | 175908/225000 [335:28:09<70:37:09,  5.18s/it]
 78%|███████▊  | 175909/225000 [335:28:15<70:50:38,  5.20s/it]
 78%|███████▊  | 175910/225000 [335:28:20<71:41:55,  5.26s/it]
 78%|███████▊  | 175911/225000 [335:28:25<72:00:20,  5.28s/it]
 78%|███████▊  | 175912/225000 [335:28:31<72:56:31,  5.35s/it]
 78%|███████▊  | 175913/225000 [335:28:36<71:39:14,  5.26s/it]
 78%|███████▊  | 175914/225000 [335:28:41<71:25:18,  5.24s/it]
 78%|███████▊  | 175915/225000 [335:28:47<71:56:42,  5.28s/it]
 78%|███████▊  | 175916/225000 [335:28:52<72:10:42,  5.29s/it]
 78%|███████▊  | 175917/225000 [335:28:57<70:57:45,  5.20s/it]
 78%|███████▊  | 175918/225000 [335:29:02<71:23:48,  5.24s/it]
 78%|███████▊  | 175919/225000 [335:29:08<72:41:50,  5.33s/it]
 78%|███████▊  | 175920/225000 [335:29:13<72:33:55,  5.32s/it]
 78%|███████▊  | 175921/225000 [335:29:19<74:04:28,  5.43s/it]
 78%|███████▊  | 175922/225000 [335:29:24<72:21:28,  5.31s/it]
 78%|███████▊  | 175923/225000 [335:29:29<71:56:14,  5.28s/it]
 78%|███████▊  | 175924/225000 [335:29:41<100:38:37,  7.38s/it]
 78%|███████▊  | 175925/225000 [335:29:47<92:18:51,  6.77s/it] 
 78%|███████▊  | 175926/225000 [335:29:50<79:20:48,  5.82s/it]
 78%|███████▊  | 175927/225000 [335:29:58<87:06:38,  6.39s/it]
 78%|███████▊  | 175928/225000 [335:30:01<73:43:35,  5.41s/it]
 78%|███████▊  | 175929/225000 [335:30:09<83:11:47,  6.10s/it]
 78%|███████▊  | 175930/225000 [335:30:13<73:42:07,  5.41s/it]
 78%|███████▊  | 175931/225000 [335:30:18<73:16:10,  5.38s/it]
 78%|███████▊  | 175932/225000 [335:30:26<85:44:11,  6.29s/it]
 78%|███████▊  | 175933/225000 [335:30:30<74:36:20,  5.47s/it]
 78%|███████▊  | 175934/225000 [335:30:33<66:22:10,  4.87s/it]
 78%|███████▊  | 175935/225000 [335:30:37<62:42:36,  4.60s/it]
 78%|███████▊  | 175936/225000 [335:30:48<87:02:22,  6.39s/it]
 78%|███████▊  | 175937/225000 [335:30:51<73:58:24,  5.43s/it]
 78%|███████▊  | 175938/225000 [335:30:58<81:28:55,  5.98s/it]
 78%|███████▊  | 175939/225000 [335:31:02<71:15:39,  5.23s/it]
 78%|███████▊  | 175940/225000 [335:31:05<64:53:30,  4.76s/it]
 78%|███████▊  | 175941/225000 [335:31:14<80:04:20,  5.88s/it]
 78%|███████▊  | 175942/225000 [335:31:18<71:20:03,  5.23s/it]
 78%|███████▊  | 175943/225000 [335:31:25<79:35:54,  5.84s/it]
 78%|███████▊  | 175944/225000 [335:31:29<73:18:08,  5.38s/it]
 78%|███████▊  | 175945/225000 [335:31:35<74:14:43,  5.45s/it]
 78%|███████▊  | 175946/225000 [335:31:42<80:07:38,  5.88s/it]
 78%|███████▊  | 175947/225000 [335:31:45<68:31:54,  5.03s/it]
 78%|███████▊  | 175948/225000 [335:31:52<79:37:40,  5.84s/it]
 78%|███████▊  | 175949/225000 [335:31:56<69:21:26,  5.09s/it]
 78%|███████▊  | 175950/225000 [335:32:00<66:15:43,  4.86s/it]
                                                              
{'loss': '0.7652', 'grad_norm': '1.111', 'learning_rate': '6.171e-06', 'epoch': '0.782', 'num_input_tokens_seen': 556023000, 'train_runtime': '1.208e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 175950/225000 [335:32:00<66:15:43,  4.86s/it]
 78%|███████▊  | 175951/225000 [335:32:08<77:33:32,  5.69s/it]
 78%|███████▊  | 175952/225000 [335:32:11<67:59:27,  4.99s/it]
 78%|███████▊  | 175953/225000 [335:32:16<67:04:30,  4.92s/it]
 78%|███████▊  | 175954/225000 [335:32:22<70:21:45,  5.16s/it]
 78%|███████▊  | 175955/225000 [335:32:27<71:28:54,  5.25s/it]
 78%|███████▊  | 175956/225000 [335:32:33<72:43:29,  5.34s/it]
 78%|███████▊  | 175957/225000 [335:32:38<72:45:27,  5.34s/it]
 78%|███████▊  | 175958/225000 [335:32:43<73:31:56,  5.40s/it]
 78%|███████▊  | 175959/225000 [335:32:49<75:16:56,  5.53s/it]
 78%|███████▊  | 175960/225000 [335:32:57<82:38:41,  6.07s/it]
 78%|███████▊  | 175961/225000 [335:33:04<87:33:03,  6.43s/it]
 78%|███████▊  | 175962/225000 [335:33:11<90:38:24,  6.65s/it]
 78%|███████▊  | 175963/225000 [335:33:14<76:44:45,  5.63s/it]
 78%|███████▊  | 175964/225000 [335:33:22<85:16:30,  6.26s/it]
 78%|███████▊  | 175965/225000 [335:33:25<72:47:10,  5.34s/it]
 78%|███████▊  | 175966/225000 [335:33:30<71:34:03,  5.25s/it]
 78%|███████▊  | 175967/225000 [335:33:35<70:18:45,  5.16s/it]
 78%|███████▊  | 175968/225000 [335:33:40<69:35:48,  5.11s/it]
 78%|███████▊  | 175969/225000 [335:33:45<69:33:49,  5.11s/it]
 78%|███████▊  | 175970/225000 [335:33:50<67:59:18,  4.99s/it]
 78%|███████▊  | 175971/225000 [335:33:55<67:57:47,  4.99s/it]
 78%|███████▊  | 175972/225000 [335:34:00<66:34:56,  4.89s/it]
 78%|███████▊  | 175973/225000 [335:34:04<65:49:38,  4.83s/it]
 78%|███████▊  | 175974/225000 [335:34:09<65:56:10,  4.84s/it]
 78%|███████▊  | 175975/225000 [335:34:14<67:00:39,  4.92s/it]
 78%|███████▊  | 175976/225000 [335:34:19<66:31:04,  4.88s/it]
 78%|███████▊  | 175977/225000 [335:34:24<64:40:55,  4.75s/it]
 78%|███████▊  | 175978/225000 [335:34:29<66:23:04,  4.88s/it]
 78%|███████▊  | 175979/225000 [335:34:34<66:57:23,  4.92s/it]
 78%|███████▊  | 175980/225000 [335:34:44<89:45:24,  6.59s/it]
 78%|███████▊  | 175981/225000 [335:34:53<98:41:38,  7.25s/it]
 78%|███████▊  | 175982/225000 [335:34:57<83:24:43,  6.13s/it]
 78%|███████▊  | 175983/225000 [335:35:04<89:27:56,  6.57s/it]
 78%|███████▊  | 175984/225000 [335:35:09<82:27:13,  6.06s/it]
 78%|███████▊  | 175985/225000 [335:35:15<81:51:07,  6.01s/it]
 78%|███████▊  | 175986/225000 [335:35:18<71:13:21,  5.23s/it]
 78%|███████▊  | 175987/225000 [335:35:24<74:50:14,  5.50s/it]
 78%|███████▊  | 175988/225000 [335:35:31<77:44:29,  5.71s/it]
 78%|███████▊  | 175989/225000 [335:35:34<69:39:54,  5.12s/it]
 78%|███████▊  | 175990/225000 [335:35:38<64:32:53,  4.74s/it]
 78%|███████▊  | 175991/225000 [335:35:47<81:21:31,  5.98s/it]
 78%|███████▊  | 175992/225000 [335:35:51<73:33:49,  5.40s/it]
 78%|███████▊  | 175993/225000 [335:35:55<67:43:45,  4.98s/it]
 78%|███████▊  | 175994/225000 [335:36:00<64:59:18,  4.77s/it]
 78%|███████▊  | 175995/225000 [335:36:04<64:28:25,  4.74s/it]
 78%|███████▊  | 175996/225000 [335:36:12<78:33:05,  5.77s/it]
 78%|███████▊  | 175997/225000 [335:36:20<88:00:37,  6.47s/it]
 78%|███████▊  | 175998/225000 [335:36:26<83:13:06,  6.11s/it]
 78%|███████▊  | 175999/225000 [335:36:30<73:42:14,  5.41s/it]
 78%|███████▊  | 176000/225000 [335:36:35<74:35:06,  5.48s/it]
                                                              
{'loss': '0.7743', 'grad_norm': '1.198', 'learning_rate': '6.159e-06', 'epoch': '0.7822', 'num_input_tokens_seen': 556152488, 'train_runtime': '1.208e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176000/225000 [335:36:35<74:35:06,  5.48s/it]
 78%|███████▊  | 176001/225000 [335:36:39<69:54:41,  5.14s/it]
 78%|███████▊  | 176002/225000 [335:36:46<77:11:39,  5.67s/it]
 78%|███████▊  | 176003/225000 [335:36:50<70:38:34,  5.19s/it]
 78%|███████▊  | 176004/225000 [335:36:55<67:05:04,  4.93s/it]
 78%|███████▊  | 176005/225000 [335:36:59<62:17:47,  4.58s/it]
 78%|███████▊  | 176006/225000 [335:37:03<60:21:21,  4.43s/it]
 78%|███████▊  | 176007/225000 [335:37:07<60:20:10,  4.43s/it]
 78%|███████▊  | 176008/225000 [335:37:19<91:17:01,  6.71s/it]
 78%|███████▊  | 176009/225000 [335:37:23<77:56:59,  5.73s/it]
 78%|███████▊  | 176010/225000 [335:37:30<85:39:41,  6.29s/it]
 78%|███████▊  | 176011/225000 [335:37:34<74:39:17,  5.49s/it]
 78%|███████▊  | 176012/225000 [335:37:41<81:45:30,  6.01s/it]
 78%|███████▊  | 176013/225000 [335:37:44<70:31:25,  5.18s/it]
 78%|███████▊  | 176014/225000 [335:37:52<80:28:32,  5.91s/it]
 78%|███████▊  | 176015/225000 [335:37:55<70:27:33,  5.18s/it]
 78%|███████▊  | 176016/225000 [335:37:59<63:46:30,  4.69s/it]
 78%|███████▊  | 176017/225000 [335:38:02<59:03:46,  4.34s/it]
 78%|███████▊  | 176018/225000 [335:38:13<83:32:23,  6.14s/it]
 78%|███████▊  | 176019/225000 [335:38:16<73:12:58,  5.38s/it]
 78%|███████▊  | 176020/225000 [335:38:20<67:33:58,  4.97s/it]
 78%|███████▊  | 176021/225000 [335:38:24<63:56:29,  4.70s/it]
 78%|███████▊  | 176022/225000 [335:38:29<61:35:08,  4.53s/it]
 78%|███████▊  | 176023/225000 [335:38:39<84:25:47,  6.21s/it]
 78%|███████▊  | 176024/225000 [335:38:42<71:40:34,  5.27s/it]
 78%|███████▊  | 176025/225000 [335:38:45<65:26:57,  4.81s/it]
 78%|███████▊  | 176026/225000 [335:38:50<64:04:19,  4.71s/it]
 78%|███████▊  | 176027/225000 [335:38:54<63:14:54,  4.65s/it]
 78%|███████▊  | 176028/225000 [335:38:59<63:48:53,  4.69s/it]
 78%|███████▊  | 176029/225000 [335:39:04<64:06:59,  4.71s/it]
 78%|███████▊  | 176030/225000 [335:39:16<92:50:27,  6.83s/it]
 78%|███████▊  | 176031/225000 [335:39:20<80:55:26,  5.95s/it]
 78%|███████▊  | 176032/225000 [335:39:23<71:10:22,  5.23s/it]
 78%|███████▊  | 176033/225000 [335:39:33<88:23:59,  6.50s/it]
 78%|███████▊  | 176034/225000 [335:39:38<82:22:30,  6.06s/it]
 78%|███████▊  | 176035/225000 [335:39:43<78:51:09,  5.80s/it]
 78%|███████▊  | 176036/225000 [335:39:48<75:35:41,  5.56s/it]
 78%|███████▊  | 176037/225000 [335:39:53<72:33:30,  5.33s/it]
 78%|███████▊  | 176038/225000 [335:39:58<70:59:17,  5.22s/it]
 78%|███████▊  | 176039/225000 [335:40:03<71:49:52,  5.28s/it]
 78%|███████▊  | 176040/225000 [335:40:08<71:39:17,  5.27s/it]
 78%|███████▊  | 176041/225000 [335:40:14<72:08:02,  5.30s/it]
 78%|███████▊  | 176042/225000 [335:40:19<70:44:32,  5.20s/it]
 78%|███████▊  | 176043/225000 [335:40:24<70:48:35,  5.21s/it]
 78%|███████▊  | 176044/225000 [335:40:29<70:56:58,  5.22s/it]
 78%|███████▊  | 176045/225000 [335:40:34<71:05:18,  5.23s/it]
 78%|███████▊  | 176046/225000 [335:40:40<71:26:13,  5.25s/it]
 78%|███████▊  | 176047/225000 [335:40:45<72:24:48,  5.33s/it]
 78%|███████▊  | 176048/225000 [335:40:50<71:55:04,  5.29s/it]
 78%|███████▊  | 176049/225000 [335:40:56<71:46:14,  5.28s/it]
 78%|███████▊  | 176050/225000 [335:41:01<70:11:42,  5.16s/it]
                                                              
{'loss': '0.7705', 'grad_norm': '1.203', 'learning_rate': '6.147e-06', 'epoch': '0.7824', 'num_input_tokens_seen': 556280688, 'train_runtime': '1.208e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176050/225000 [335:41:01<70:11:42,  5.16s/it]
 78%|███████▊  | 176051/225000 [335:41:06<70:05:17,  5.15s/it]
 78%|███████▊  | 176052/225000 [335:41:11<70:53:42,  5.21s/it]
 78%|███████▊  | 176053/225000 [335:41:16<70:59:08,  5.22s/it]
 78%|███████▊  | 176054/225000 [335:41:22<72:35:53,  5.34s/it]
 78%|███████▊  | 176055/225000 [335:41:27<72:34:25,  5.34s/it]
 78%|███████▊  | 176056/225000 [335:41:39<97:13:01,  7.15s/it]
 78%|███████▊  | 176057/225000 [335:41:43<87:31:05,  6.44s/it]
 78%|███████▊  | 176058/225000 [335:41:47<76:11:09,  5.60s/it]
 78%|███████▊  | 176059/225000 [335:41:51<70:41:29,  5.20s/it]
 78%|███████▊  | 176060/225000 [335:42:01<89:10:57,  6.56s/it]
 78%|███████▊  | 176061/225000 [335:42:07<85:04:36,  6.26s/it]
 78%|███████▊  | 176062/225000 [335:42:13<86:13:07,  6.34s/it]
 78%|███████▊  | 176063/225000 [335:42:17<76:04:35,  5.60s/it]
 78%|███████▊  | 176064/225000 [335:42:21<69:09:55,  5.09s/it]
 78%|███████▊  | 176065/225000 [335:42:30<86:59:33,  6.40s/it]
 78%|███████▊  | 176066/225000 [335:42:34<76:33:00,  5.63s/it]
 78%|███████▊  | 176067/225000 [335:42:40<77:51:49,  5.73s/it]
 78%|███████▊  | 176068/225000 [335:42:46<79:33:04,  5.85s/it]
 78%|███████▊  | 176069/225000 [335:42:50<71:31:37,  5.26s/it]
 78%|███████▊  | 176070/225000 [335:42:54<67:00:22,  4.93s/it]
 78%|███████▊  | 176071/225000 [335:43:02<76:48:41,  5.65s/it]
 78%|███████▊  | 176072/225000 [335:43:07<74:17:54,  5.47s/it]
 78%|███████▊  | 176073/225000 [335:43:12<72:17:10,  5.32s/it]
 78%|███████▊  | 176074/225000 [335:43:18<76:58:36,  5.66s/it]
 78%|███████▊  | 176075/225000 [335:43:23<72:17:46,  5.32s/it]
 78%|███████▊  | 176076/225000 [335:43:27<67:35:18,  4.97s/it]
 78%|███████▊  | 176077/225000 [335:43:31<65:12:49,  4.80s/it]
 78%|███████▊  | 176078/225000 [335:43:36<63:31:59,  4.68s/it]
 78%|███████▊  | 176079/225000 [335:43:40<62:53:30,  4.63s/it]
 78%|███████▊  | 176080/225000 [335:43:49<78:39:41,  5.79s/it]
 78%|███████▊  | 176081/225000 [335:43:56<86:10:56,  6.34s/it]
 78%|███████▊  | 176082/225000 [335:44:00<76:47:23,  5.65s/it]
 78%|███████▊  | 176083/225000 [335:44:11<96:52:56,  7.13s/it]
 78%|███████▊  | 176084/225000 [335:44:14<81:13:27,  5.98s/it]
 78%|███████▊  | 176085/225000 [335:44:21<86:01:16,  6.33s/it]
 78%|███████▊  | 176086/225000 [335:44:25<75:31:11,  5.56s/it]
 78%|███████▊  | 176087/225000 [335:44:28<66:32:08,  4.90s/it]
 78%|███████▊  | 176088/225000 [335:44:36<78:58:23,  5.81s/it]
 78%|███████▊  | 176089/225000 [335:44:42<77:32:30,  5.71s/it]
 78%|███████▊  | 176090/225000 [335:44:45<68:44:15,  5.06s/it]
 78%|███████▊  | 176091/225000 [335:44:54<83:44:40,  6.16s/it]
 78%|███████▊  | 176092/225000 [335:44:57<72:09:45,  5.31s/it]
 78%|███████▊  | 176093/225000 [335:45:01<66:17:50,  4.88s/it]
 78%|███████▊  | 176094/225000 [335:45:06<65:57:49,  4.86s/it]
 78%|███████▊  | 176095/225000 [335:45:10<62:24:55,  4.59s/it]
 78%|███████▊  | 176096/225000 [335:45:15<62:34:51,  4.61s/it]
 78%|███████▊  | 176097/225000 [335:45:19<62:17:47,  4.59s/it]
 78%|███████▊  | 176098/225000 [335:45:24<61:31:36,  4.53s/it]
 78%|███████▊  | 176099/225000 [335:45:28<62:06:33,  4.57s/it]
 78%|███████▊  | 176100/225000 [335:45:39<85:26:57,  6.29s/it]
                                                              
{'loss': '0.762', 'grad_norm': '1.214', 'learning_rate': '6.135e-06', 'epoch': '0.7827', 'num_input_tokens_seen': 556411720, 'train_runtime': '1.209e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176100/225000 [335:45:39<85:26:57,  6.29s/it]
 78%|███████▊  | 176101/225000 [335:45:46<88:18:33,  6.50s/it]
 78%|███████▊  | 176102/225000 [335:45:49<76:28:41,  5.63s/it]
 78%|███████▊  | 176103/225000 [335:45:57<85:27:56,  6.29s/it]
 78%|███████▊  | 176104/225000 [335:46:03<81:59:52,  6.04s/it]
 78%|███████▊  | 176105/225000 [335:46:06<72:00:10,  5.30s/it]
 78%|███████▊  | 176106/225000 [335:46:10<67:44:38,  4.99s/it]
 78%|███████▊  | 176107/225000 [335:46:15<65:34:53,  4.83s/it]
 78%|███████▊  | 176108/225000 [335:46:23<79:36:01,  5.86s/it]
 78%|███████▊  | 176109/225000 [335:46:28<75:46:13,  5.58s/it]
 78%|███████▊  | 176110/225000 [335:46:35<82:53:14,  6.10s/it]
 78%|███████▊  | 176111/225000 [335:46:42<85:15:21,  6.28s/it]
 78%|███████▊  | 176112/225000 [335:46:45<73:00:43,  5.38s/it]
 78%|███████▊  | 176113/225000 [335:46:53<84:12:23,  6.20s/it]
 78%|███████▊  | 176114/225000 [335:46:57<74:26:50,  5.48s/it]
 78%|███████▊  | 176115/225000 [335:47:01<68:25:56,  5.04s/it]
 78%|███████▊  | 176116/225000 [335:47:11<86:35:47,  6.38s/it]
 78%|███████▊  | 176117/225000 [335:47:14<74:20:55,  5.48s/it]
 78%|███████▊  | 176118/225000 [335:47:19<73:07:38,  5.39s/it]
 78%|███████▊  | 176119/225000 [335:47:25<73:26:59,  5.41s/it]
 78%|███████▊  | 176120/225000 [335:47:30<72:45:19,  5.36s/it]
 78%|███████▊  | 176121/225000 [335:47:35<73:10:20,  5.39s/it]
 78%|███████▊  | 176122/225000 [335:47:41<73:40:32,  5.43s/it]
 78%|███████▊  | 176123/225000 [335:47:46<72:29:00,  5.34s/it]
 78%|███████▊  | 176124/225000 [335:47:51<72:36:25,  5.35s/it]
 78%|███████▊  | 176125/225000 [335:47:57<73:49:07,  5.44s/it]
 78%|███████▊  | 176126/225000 [335:48:02<73:05:31,  5.38s/it]
 78%|███████▊  | 176127/225000 [335:48:08<73:14:22,  5.39s/it]
 78%|███████▊  | 176128/225000 [335:48:13<71:59:30,  5.30s/it]
 78%|███████▊  | 176129/225000 [335:48:18<71:43:08,  5.28s/it]
 78%|███████▊  | 176130/225000 [335:48:23<71:03:21,  5.23s/it]
 78%|███████▊  | 176131/225000 [335:48:29<71:41:06,  5.28s/it]
 78%|███████▊  | 176132/225000 [335:48:34<73:19:04,  5.40s/it]
 78%|███████▊  | 176133/225000 [335:48:40<73:52:44,  5.44s/it]
 78%|███████▊  | 176134/225000 [335:48:45<73:40:41,  5.43s/it]
 78%|███████▊  | 176135/225000 [335:48:50<72:07:49,  5.31s/it]
 78%|███████▊  | 176136/225000 [335:48:56<73:11:50,  5.39s/it]
 78%|███████▊  | 176137/225000 [335:49:01<71:52:07,  5.29s/it]
 78%|███████▊  | 176138/225000 [335:49:06<72:50:15,  5.37s/it]
 78%|███████▊  | 176139/225000 [335:49:12<72:45:14,  5.36s/it]
 78%|███████▊  | 176140/225000 [335:49:17<72:00:57,  5.31s/it]
 78%|███████▊  | 176141/225000 [335:49:22<70:59:26,  5.23s/it]
 78%|███████▊  | 176142/225000 [335:49:27<70:07:18,  5.17s/it]
 78%|███████▊  | 176143/225000 [335:49:32<71:01:40,  5.23s/it]
 78%|███████▊  | 176144/225000 [335:49:38<70:23:54,  5.19s/it]
 78%|███████▊  | 176145/225000 [335:49:43<69:56:38,  5.15s/it]
 78%|███████▊  | 176146/225000 [335:49:48<71:15:22,  5.25s/it]
 78%|███████▊  | 176147/225000 [335:49:53<71:50:18,  5.29s/it]
 78%|███████▊  | 176148/225000 [335:49:59<71:26:43,  5.26s/it]
 78%|███████▊  | 176149/225000 [335:50:04<69:53:05,  5.15s/it]
 78%|███████▊  | 176150/225000 [335:50:09<69:39:35,  5.13s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '1.19', 'learning_rate': '6.123e-06', 'epoch': '0.7829', 'num_input_tokens_seen': 556543056, 'train_runtime': '1.209e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176150/225000 [335:50:09<69:39:35,  5.13s/it]
 78%|███████▊  | 176151/225000 [335:50:14<70:06:47,  5.17s/it]
 78%|███████▊  | 176152/225000 [335:50:26<100:09:57,  7.38s/it]
 78%|███████▊  | 176153/225000 [335:50:32<93:58:48,  6.93s/it] 
 78%|███████▊  | 176154/225000 [335:50:36<81:27:55,  6.00s/it]
 78%|███████▊  | 176155/225000 [335:50:40<73:03:28,  5.38s/it]
 78%|███████▊  | 176156/225000 [335:50:50<90:27:53,  6.67s/it]
 78%|███████▊  | 176157/225000 [335:50:53<78:21:00,  5.77s/it]
 78%|███████▊  | 176158/225000 [335:50:57<70:07:02,  5.17s/it]
 78%|███████▊  | 176159/225000 [335:51:03<73:25:19,  5.41s/it]
 78%|███████▊  | 176160/225000 [335:51:12<86:12:50,  6.35s/it]
 78%|███████▊  | 176161/225000 [335:51:15<74:08:34,  5.47s/it]
 78%|███████▊  | 176162/225000 [335:51:19<68:54:32,  5.08s/it]
 78%|███████▊  | 176163/225000 [335:51:24<66:07:51,  4.87s/it]
 78%|███████▊  | 176164/225000 [335:51:28<65:16:32,  4.81s/it]
 78%|███████▊  | 176165/225000 [335:51:33<63:56:34,  4.71s/it]
 78%|███████▊  | 176166/225000 [335:51:38<64:31:40,  4.76s/it]
 78%|███████▊  | 176167/225000 [335:51:42<64:30:11,  4.76s/it]
 78%|███████▊  | 176168/225000 [335:51:54<91:49:52,  6.77s/it]
 78%|███████▊  | 176169/225000 [335:52:00<88:58:28,  6.56s/it]
 78%|███████▊  | 176170/225000 [335:52:04<76:47:33,  5.66s/it]
 78%|███████▊  | 176171/225000 [335:52:11<85:49:23,  6.33s/it]
 78%|███████▊  | 176172/225000 [335:52:17<82:14:48,  6.06s/it]
 78%|███████▊  | 176173/225000 [335:52:21<72:50:37,  5.37s/it]
 78%|███████▊  | 176174/225000 [335:52:29<85:31:03,  6.31s/it]
 78%|███████▊  | 176175/225000 [335:52:33<74:19:22,  5.48s/it]
 78%|███████▊  | 176176/225000 [335:52:36<66:50:44,  4.93s/it]
 78%|███████▊  | 176177/225000 [335:52:46<84:51:53,  6.26s/it]
 78%|███████▊  | 176178/225000 [335:52:52<86:58:41,  6.41s/it]
 78%|███████▊  | 176179/225000 [335:52:57<79:32:37,  5.87s/it]
 78%|███████▊  | 176180/225000 [335:53:01<73:12:12,  5.40s/it]
 78%|███████▊  | 176181/225000 [335:53:06<70:06:11,  5.17s/it]
 78%|███████▊  | 176182/225000 [335:53:11<67:40:50,  4.99s/it]
 78%|███████▊  | 176183/225000 [335:53:15<67:25:47,  4.97s/it]
 78%|███████▊  | 176184/225000 [335:53:20<67:17:54,  4.96s/it]
 78%|███████▊  | 176185/225000 [335:53:26<68:33:59,  5.06s/it]
 78%|███████▊  | 176186/225000 [335:53:34<82:52:59,  6.11s/it]
 78%|███████▊  | 176187/225000 [335:53:43<93:03:38,  6.86s/it]
 78%|███████▊  | 176188/225000 [335:53:47<80:28:13,  5.93s/it]
 78%|███████▊  | 176189/225000 [335:53:50<71:43:20,  5.29s/it]
 78%|███████▊  | 176190/225000 [335:53:54<66:20:30,  4.89s/it]
 78%|███████▊  | 176191/225000 [335:54:05<88:24:40,  6.52s/it]
 78%|███████▊  | 176192/225000 [335:54:09<78:50:50,  5.82s/it]
 78%|███████▊  | 176193/225000 [335:54:12<69:05:36,  5.10s/it]
 78%|███████▊  | 176194/225000 [335:54:20<81:09:20,  5.99s/it]
 78%|███████▊  | 176195/225000 [335:54:24<72:11:36,  5.33s/it]
 78%|███████▊  | 176196/225000 [335:54:29<69:52:12,  5.15s/it]
 78%|███████▊  | 176197/225000 [335:54:33<67:01:15,  4.94s/it]
 78%|███████▊  | 176198/225000 [335:54:44<88:13:04,  6.51s/it]
 78%|███████▊  | 176199/225000 [335:54:49<84:54:57,  6.26s/it]
 78%|███████▊  | 176200/225000 [335:54:53<76:09:33,  5.62s/it]
                                                              
{'loss': '0.7587', 'grad_norm': '1.182', 'learning_rate': '6.111e-06', 'epoch': '0.7831', 'num_input_tokens_seen': 556673888, 'train_runtime': '1.209e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176200/225000 [335:54:53<76:09:33,  5.62s/it]
 78%|███████▊  | 176201/225000 [335:54:59<75:38:15,  5.58s/it]
 78%|███████▊  | 176202/225000 [335:55:07<85:56:42,  6.34s/it]
 78%|███████▊  | 176203/225000 [335:55:11<77:37:53,  5.73s/it]
 78%|███████▊  | 176204/225000 [335:55:16<72:14:56,  5.33s/it]
 78%|███████▊  | 176205/225000 [335:55:20<67:12:32,  4.96s/it]
 78%|███████▊  | 176206/225000 [335:55:24<65:20:16,  4.82s/it]
 78%|███████▊  | 176207/225000 [335:55:29<64:31:30,  4.76s/it]
 78%|███████▊  | 176208/225000 [335:55:36<75:11:11,  5.55s/it]
 78%|███████▊  | 176209/225000 [335:55:45<86:53:12,  6.41s/it]
 78%|███████▊  | 176210/225000 [335:55:48<75:10:00,  5.55s/it]
 78%|███████▊  | 176211/225000 [335:55:54<75:57:32,  5.60s/it]
 78%|███████▊  | 176212/225000 [335:56:01<81:08:29,  5.99s/it]
 78%|███████▊  | 176213/225000 [335:56:04<70:28:56,  5.20s/it]
 78%|███████▊  | 176214/225000 [335:56:12<81:39:00,  6.03s/it]
 78%|███████▊  | 176215/225000 [335:56:16<71:32:06,  5.28s/it]
 78%|███████▊  | 176216/225000 [335:56:19<63:53:41,  4.72s/it]
 78%|███████▊  | 176217/225000 [335:56:24<65:19:58,  4.82s/it]
 78%|███████▊  | 176218/225000 [335:56:30<67:32:23,  4.98s/it]
 78%|███████▊  | 176219/225000 [335:56:35<68:52:14,  5.08s/it]
 78%|███████▊  | 176220/225000 [335:56:40<68:35:32,  5.06s/it]
 78%|███████▊  | 176221/225000 [335:56:45<68:15:24,  5.04s/it]
 78%|███████▊  | 176222/225000 [335:56:50<67:25:50,  4.98s/it]
 78%|███████▊  | 176223/225000 [335:56:55<67:55:12,  5.01s/it]
 78%|███████▊  | 176224/225000 [335:57:00<70:01:24,  5.17s/it]
 78%|███████▊  | 176225/225000 [335:57:06<70:24:02,  5.20s/it]
 78%|███████▊  | 176226/225000 [335:57:11<70:49:51,  5.23s/it]
 78%|███████▊  | 176227/225000 [335:57:16<72:19:09,  5.34s/it]
 78%|███████▊  | 176228/225000 [335:57:21<70:37:18,  5.21s/it]
 78%|███████▊  | 176229/225000 [335:57:26<69:25:22,  5.12s/it]
 78%|███████▊  | 176230/225000 [335:57:33<74:46:22,  5.52s/it]
 78%|███████▊  | 176231/225000 [335:57:43<92:55:17,  6.86s/it]
 78%|███████▊  | 176232/225000 [335:57:46<79:59:09,  5.90s/it]
 78%|███████▊  | 176233/225000 [335:57:51<74:08:01,  5.47s/it]
 78%|███████▊  | 176234/225000 [335:58:01<93:07:35,  6.87s/it]
 78%|███████▊  | 176235/225000 [335:58:05<79:34:56,  5.88s/it]
 78%|███████▊  | 176236/225000 [335:58:08<69:25:58,  5.13s/it]
 78%|███████▊  | 176237/225000 [335:58:17<83:42:00,  6.18s/it]
 78%|███████▊  | 176238/225000 [335:58:23<83:53:43,  6.19s/it]
 78%|███████▊  | 176239/225000 [335:58:26<73:27:24,  5.42s/it]
 78%|███████▊  | 176240/225000 [335:58:30<66:42:36,  4.93s/it]
 78%|███████▊  | 176241/225000 [335:58:39<82:05:27,  6.06s/it]
 78%|███████▊  | 176242/225000 [335:58:42<70:07:25,  5.18s/it]
 78%|███████▊  | 176243/225000 [335:58:46<64:03:46,  4.73s/it]
 78%|███████▊  | 176244/225000 [335:58:54<77:57:48,  5.76s/it]
 78%|███████▊  | 176245/225000 [335:58:57<67:53:38,  5.01s/it]
 78%|███████▊  | 176246/225000 [335:59:01<62:21:39,  4.60s/it]
 78%|███████▊  | 176247/225000 [335:59:09<75:34:43,  5.58s/it]
 78%|███████▊  | 176248/225000 [335:59:15<78:16:07,  5.78s/it]
 78%|███████▊  | 176249/225000 [335:59:18<68:48:45,  5.08s/it]
 78%|███████▊  | 176250/225000 [335:59:27<81:47:57,  6.04s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.313', 'learning_rate': '6.099e-06', 'epoch': '0.7833', 'num_input_tokens_seen': 556804184, 'train_runtime': '1.21e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176250/225000 [335:59:27<81:47:57,  6.04s/it]
 78%|███████▊  | 176251/225000 [335:59:30<71:53:12,  5.31s/it]
 78%|███████▊  | 176252/225000 [335:59:34<65:52:40,  4.87s/it]
 78%|███████▊  | 176253/225000 [335:59:42<76:40:45,  5.66s/it]
 78%|███████▊  | 176254/225000 [335:59:49<84:45:13,  6.26s/it]
 78%|███████▊  | 176255/225000 [335:59:53<75:02:57,  5.54s/it]
 78%|███████▊  | 176256/225000 [335:59:57<68:16:46,  5.04s/it]
 78%|███████▊  | 176257/225000 [336:00:01<64:31:58,  4.77s/it]
 78%|███████▊  | 176258/225000 [336:00:09<77:50:05,  5.75s/it]
 78%|███████▊  | 176259/225000 [336:00:13<71:29:35,  5.28s/it]
 78%|███████▊  | 176260/225000 [336:00:18<67:35:44,  4.99s/it]
 78%|███████▊  | 176261/225000 [336:00:27<84:50:09,  6.27s/it]
 78%|███████▊  | 176262/225000 [336:00:33<82:45:29,  6.11s/it]
 78%|███████▊  | 176263/225000 [336:00:37<74:41:50,  5.52s/it]
 78%|███████▊  | 176264/225000 [336:00:41<67:44:42,  5.00s/it]
 78%|███████▊  | 176265/225000 [336:00:45<64:36:47,  4.77s/it]
 78%|███████▊  | 176266/225000 [336:00:49<61:36:35,  4.55s/it]
 78%|███████▊  | 176267/225000 [336:00:56<74:16:45,  5.49s/it]
 78%|███████▊  | 176268/225000 [336:01:04<84:25:59,  6.24s/it]
 78%|███████▊  | 176269/225000 [336:01:08<73:05:31,  5.40s/it]
 78%|███████▊  | 176270/225000 [336:01:17<87:27:01,  6.46s/it]
 78%|███████▊  | 176271/225000 [336:01:21<76:32:36,  5.65s/it]
 78%|███████▊  | 176272/225000 [336:01:24<67:49:49,  5.01s/it]
 78%|███████▊  | 176273/225000 [336:01:32<79:33:10,  5.88s/it]
 78%|███████▊  | 176274/225000 [336:01:37<76:01:32,  5.62s/it]
 78%|███████▊  | 176275/225000 [336:01:41<67:22:42,  4.98s/it]
 78%|███████▊  | 176276/225000 [336:01:51<88:03:51,  6.51s/it]
 78%|███████▊  | 176277/225000 [336:01:54<76:43:04,  5.67s/it]
 78%|███████▊  | 176278/225000 [336:02:00<76:39:20,  5.66s/it]
 78%|███████▊  | 176279/225000 [336:02:05<75:55:42,  5.61s/it]
 78%|███████▊  | 176280/225000 [336:02:11<75:06:47,  5.55s/it]
 78%|███████▊  | 176281/225000 [336:02:18<83:23:08,  6.16s/it]
 78%|███████▊  | 176282/225000 [336:02:22<73:44:22,  5.45s/it]
 78%|███████▊  | 176283/225000 [336:02:28<76:41:02,  5.67s/it]
 78%|███████▊  | 176284/225000 [336:02:34<77:24:18,  5.72s/it]
 78%|███████▊  | 176285/225000 [336:02:40<76:39:57,  5.67s/it]
 78%|███████▊  | 176286/225000 [336:02:43<66:37:53,  4.92s/it]
 78%|███████▊  | 176287/225000 [336:02:48<66:44:19,  4.93s/it]
 78%|███████▊  | 176288/225000 [336:02:54<69:25:24,  5.13s/it]
 78%|███████▊  | 176289/225000 [336:03:03<86:41:22,  6.41s/it]
 78%|███████▊  | 176290/225000 [336:03:07<78:49:31,  5.83s/it]
 78%|███████▊  | 176291/225000 [336:03:13<79:30:52,  5.88s/it]
 78%|███████▊  | 176292/225000 [336:03:17<68:51:46,  5.09s/it]
 78%|███████▊  | 176293/225000 [336:03:22<68:32:32,  5.07s/it]
 78%|███████▊  | 176294/225000 [336:03:27<71:11:01,  5.26s/it]
 78%|███████▊  | 176295/225000 [336:03:30<62:23:26,  4.61s/it]
 78%|███████▊  | 176296/225000 [336:03:39<79:12:18,  5.85s/it]
 78%|███████▊  | 176297/225000 [336:03:42<68:36:06,  5.07s/it]
 78%|███████▊  | 176298/225000 [336:03:46<63:33:08,  4.70s/it]
 78%|███████▊  | 176299/225000 [336:03:50<60:42:29,  4.49s/it]
 78%|███████▊  | 176300/225000 [336:03:58<73:11:03,  5.41s/it]
                                                              
{'loss': '0.764', 'grad_norm': '1.117', 'learning_rate': '6.087e-06', 'epoch': '0.7836', 'num_input_tokens_seen': 556933896, 'train_runtime': '1.21e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176300/225000 [336:03:58<73:11:03,  5.41s/it]
 78%|███████▊  | 176301/225000 [336:04:05<78:42:51,  5.82s/it]
 78%|███████▊  | 176302/225000 [336:04:08<68:57:34,  5.10s/it]
 78%|███████▊  | 176303/225000 [336:04:17<83:04:16,  6.14s/it]
 78%|███████▊  | 176304/225000 [336:04:20<73:06:38,  5.40s/it]
 78%|███████▊  | 176305/225000 [336:04:25<69:56:08,  5.17s/it]
 78%|███████▊  | 176306/225000 [336:04:34<85:28:30,  6.32s/it]
 78%|███████▊  | 176307/225000 [336:04:38<75:34:36,  5.59s/it]
 78%|███████▊  | 176308/225000 [336:04:46<86:35:21,  6.40s/it]
 78%|███████▊  | 176309/225000 [336:04:51<79:05:58,  5.85s/it]
 78%|███████▊  | 176310/225000 [336:04:55<73:54:22,  5.46s/it]
 78%|███████▊  | 176311/225000 [336:05:00<69:05:38,  5.11s/it]
 78%|███████▊  | 176312/225000 [336:05:04<65:40:04,  4.86s/it]
 78%|███████▊  | 176313/225000 [336:05:09<66:01:12,  4.88s/it]
 78%|███████▊  | 176314/225000 [336:05:14<65:40:16,  4.86s/it]
 78%|███████▊  | 176315/225000 [336:05:18<65:27:01,  4.84s/it]
 78%|███████▊  | 176316/225000 [336:05:23<64:28:19,  4.77s/it]
 78%|███████▊  | 176317/225000 [336:05:34<91:29:51,  6.77s/it]
 78%|███████▊  | 176318/225000 [336:05:41<90:47:21,  6.71s/it]
 78%|███████▊  | 176319/225000 [336:05:45<81:25:39,  6.02s/it]
 78%|███████▊  | 176320/225000 [336:05:49<73:18:46,  5.42s/it]
 78%|███████▊  | 176321/225000 [336:05:54<68:11:54,  5.04s/it]
 78%|███████▊  | 176322/225000 [336:05:58<64:50:59,  4.80s/it]
 78%|███████▊  | 176323/225000 [336:06:07<81:58:54,  6.06s/it]
 78%|███████▊  | 176324/225000 [336:06:13<84:29:22,  6.25s/it]
 78%|███████▊  | 176325/225000 [336:06:19<79:45:40,  5.90s/it]
 78%|███████▊  | 176326/225000 [336:06:25<80:46:02,  5.97s/it]
 78%|███████▊  | 176327/225000 [336:06:28<70:01:40,  5.18s/it]
 78%|███████▊  | 176328/225000 [336:06:34<73:05:29,  5.41s/it]
 78%|███████▊  | 176329/225000 [336:06:43<86:04:01,  6.37s/it]
 78%|███████▊  | 176330/225000 [336:06:46<74:05:27,  5.48s/it]
 78%|███████▊  | 176331/225000 [336:06:51<71:09:20,  5.26s/it]
 78%|███████▊  | 176332/225000 [336:06:55<68:29:22,  5.07s/it]
 78%|███████▊  | 176333/225000 [336:06:59<64:15:10,  4.75s/it]
 78%|███████▊  | 176334/225000 [336:07:05<66:26:07,  4.91s/it]
 78%|███████▊  | 176335/225000 [336:07:09<65:20:24,  4.83s/it]
 78%|███████▊  | 176336/225000 [336:07:14<63:11:07,  4.67s/it]
 78%|███████▊  | 176337/225000 [336:07:19<64:40:27,  4.78s/it]
 78%|███████▊  | 176338/225000 [336:07:23<62:11:09,  4.60s/it]
 78%|███████▊  | 176339/225000 [336:07:36<96:36:52,  7.15s/it]
 78%|███████▊  | 176340/225000 [336:07:40<86:03:19,  6.37s/it]
 78%|███████▊  | 176341/225000 [336:07:47<87:12:57,  6.45s/it]
 78%|███████▊  | 176342/225000 [336:07:50<74:34:53,  5.52s/it]
 78%|███████▊  | 176343/225000 [336:07:58<84:04:46,  6.22s/it]
 78%|███████▊  | 176344/225000 [336:08:02<74:15:44,  5.49s/it]
 78%|███████▊  | 176345/225000 [336:08:07<70:11:10,  5.19s/it]
 78%|███████▊  | 176346/225000 [336:08:11<68:12:40,  5.05s/it]
 78%|███████▊  | 176347/225000 [336:08:21<86:23:50,  6.39s/it]
 78%|███████▊  | 176348/225000 [336:08:24<74:22:03,  5.50s/it]
 78%|███████▊  | 176349/225000 [336:08:28<68:21:30,  5.06s/it]
 78%|███████▊  | 176350/225000 [336:08:33<67:43:22,  5.01s/it]
                                                              
{'loss': '0.7749', 'grad_norm': '1.072', 'learning_rate': '6.075e-06', 'epoch': '0.7838', 'num_input_tokens_seen': 557064536, 'train_runtime': '1.21e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176350/225000 [336:08:33<67:43:22,  5.01s/it]
 78%|███████▊  | 176351/225000 [336:08:38<67:08:55,  4.97s/it]
 78%|███████▊  | 176352/225000 [336:08:42<64:51:23,  4.80s/it]
 78%|███████▊  | 176353/225000 [336:08:47<62:49:20,  4.65s/it]
 78%|███████▊  | 176354/225000 [336:08:51<63:03:54,  4.67s/it]
 78%|███████▊  | 176355/225000 [336:08:56<63:06:07,  4.67s/it]
 78%|███████▊  | 176356/225000 [336:09:01<64:42:25,  4.79s/it]
 78%|███████▊  | 176357/225000 [336:09:10<80:12:39,  5.94s/it]
 78%|███████▊  | 176358/225000 [336:09:18<89:46:21,  6.64s/it]
 78%|███████▊  | 176359/225000 [336:09:21<76:12:41,  5.64s/it]
 78%|███████▊  | 176360/225000 [336:09:29<82:50:08,  6.13s/it]
 78%|███████▊  | 176361/225000 [336:09:36<86:17:14,  6.39s/it]
 78%|███████▊  | 176362/225000 [336:09:39<75:47:43,  5.61s/it]
 78%|███████▊  | 176363/225000 [336:09:43<67:44:38,  5.01s/it]
 78%|███████▊  | 176364/225000 [336:09:52<83:06:33,  6.15s/it]
 78%|███████▊  | 176365/225000 [336:09:56<74:58:14,  5.55s/it]
 78%|███████▊  | 176366/225000 [336:10:00<69:32:48,  5.15s/it]
 78%|███████▊  | 176367/225000 [336:10:05<66:12:36,  4.90s/it]
 78%|███████▊  | 176368/225000 [336:10:14<82:40:28,  6.12s/it]
 78%|███████▊  | 176369/225000 [336:10:18<77:27:11,  5.73s/it]
 78%|███████▊  | 176370/225000 [336:10:25<82:30:53,  6.11s/it]
 78%|███████▊  | 176371/225000 [336:10:29<72:53:57,  5.40s/it]
 78%|███████▊  | 176372/225000 [336:10:33<65:32:19,  4.85s/it]
 78%|███████▊  | 176373/225000 [336:10:38<66:29:52,  4.92s/it]
 78%|███████▊  | 176374/225000 [336:10:43<67:28:28,  5.00s/it]
 78%|███████▊  | 176375/225000 [336:10:51<81:48:57,  6.06s/it]
 78%|███████▊  | 176376/225000 [336:10:56<77:46:19,  5.76s/it]
 78%|███████▊  | 176377/225000 [336:11:01<71:09:06,  5.27s/it]
 78%|███████▊  | 176378/225000 [336:11:06<71:10:57,  5.27s/it]
 78%|███████▊  | 176379/225000 [336:11:11<70:18:07,  5.21s/it]
 78%|███████▊  | 176380/225000 [336:11:16<71:29:35,  5.29s/it]
 78%|███████▊  | 176381/225000 [336:11:22<71:48:28,  5.32s/it]
 78%|███████▊  | 176382/225000 [336:11:27<72:22:47,  5.36s/it]
 78%|███████▊  | 176383/225000 [336:11:32<71:02:14,  5.26s/it]
 78%|███████▊  | 176384/225000 [336:11:37<69:01:19,  5.11s/it]
 78%|███████▊  | 176385/225000 [336:11:42<67:31:58,  5.00s/it]
 78%|███████▊  | 176386/225000 [336:11:46<63:41:50,  4.72s/it]
 78%|███████▊  | 176387/225000 [336:11:51<65:14:50,  4.83s/it]
 78%|███████▊  | 176388/225000 [336:11:56<66:17:48,  4.91s/it]
 78%|███████▊  | 176389/225000 [336:12:01<67:07:46,  4.97s/it]
 78%|███████▊  | 176390/225000 [336:12:06<67:39:56,  5.01s/it]
 78%|███████▊  | 176391/225000 [336:12:11<67:55:13,  5.03s/it]
 78%|███████▊  | 176392/225000 [336:12:17<69:40:32,  5.16s/it]
 78%|███████▊  | 176393/225000 [336:12:22<67:56:25,  5.03s/it]
 78%|███████▊  | 176394/225000 [336:12:27<68:51:59,  5.10s/it]
 78%|███████▊  | 176395/225000 [336:12:32<69:19:13,  5.13s/it]
 78%|███████▊  | 176396/225000 [336:12:38<70:47:38,  5.24s/it]
 78%|███████▊  | 176397/225000 [336:12:44<75:50:21,  5.62s/it]
 78%|███████▊  | 176398/225000 [336:12:57<104:11:03,  7.72s/it]
 78%|███████▊  | 176399/225000 [336:13:00<87:17:28,  6.47s/it] 
 78%|███████▊  | 176400/225000 [336:13:04<75:16:44,  5.58s/it]
                                                              
{'loss': '0.751', 'grad_norm': '1.165', 'learning_rate': '6.063e-06', 'epoch': '0.784', 'num_input_tokens_seen': 557193496, 'train_runtime': '1.21e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176400/225000 [336:13:04<75:16:44,  5.58s/it]
 78%|███████▊  | 176401/225000 [336:13:07<67:01:35,  4.97s/it]
 78%|███████▊  | 176402/225000 [336:13:17<87:16:54,  6.47s/it]
 78%|███████▊  | 176403/225000 [336:13:22<80:52:18,  5.99s/it]
 78%|███████▊  | 176404/225000 [336:13:27<76:20:26,  5.66s/it]
 78%|███████▊  | 176405/225000 [336:13:32<72:45:03,  5.39s/it]
 78%|███████▊  | 176406/225000 [336:13:37<70:33:37,  5.23s/it]
 78%|███████▊  | 176407/225000 [336:13:42<69:36:06,  5.16s/it]
 78%|███████▊  | 176408/225000 [336:13:47<69:36:16,  5.16s/it]
 78%|███████▊  | 176409/225000 [336:13:52<68:13:51,  5.06s/it]
 78%|███████▊  | 176410/225000 [336:14:03<95:03:00,  7.04s/it]
 78%|███████▊  | 176411/225000 [336:14:07<80:18:06,  5.95s/it]
 78%|███████▊  | 176412/225000 [336:14:11<73:31:36,  5.45s/it]
 78%|███████▊  | 176413/225000 [336:14:20<87:27:35,  6.48s/it]
 78%|███████▊  | 176414/225000 [336:14:23<74:32:10,  5.52s/it]
 78%|███████▊  | 176415/225000 [336:14:28<72:41:46,  5.39s/it]
 78%|███████▊  | 176416/225000 [336:14:33<71:47:32,  5.32s/it]
 78%|███████▊  | 176417/225000 [336:14:38<71:14:52,  5.28s/it]
 78%|███████▊  | 176418/225000 [336:14:43<69:47:51,  5.17s/it]
 78%|███████▊  | 176419/225000 [336:14:49<70:42:30,  5.24s/it]
 78%|███████▊  | 176420/225000 [336:14:54<71:02:12,  5.26s/it]
 78%|███████▊  | 176421/225000 [336:14:59<69:55:41,  5.18s/it]
 78%|███████▊  | 176422/225000 [336:15:06<77:11:09,  5.72s/it]
 78%|███████▊  | 176423/225000 [336:15:10<70:48:44,  5.25s/it]
 78%|███████▊  | 176424/225000 [336:15:15<67:46:44,  5.02s/it]
 78%|███████▊  | 176425/225000 [336:15:19<64:36:21,  4.79s/it]
 78%|███████▊  | 176426/225000 [336:15:32<97:19:43,  7.21s/it]
 78%|███████▊  | 176427/225000 [336:15:36<85:32:47,  6.34s/it]
 78%|███████▊  | 176428/225000 [336:15:40<75:13:46,  5.58s/it]
 78%|███████▊  | 176429/225000 [336:15:47<80:53:23,  6.00s/it]
 78%|███████▊  | 176430/225000 [336:15:52<78:17:05,  5.80s/it]
 78%|███████▊  | 176431/225000 [336:15:56<69:48:16,  5.17s/it]
 78%|███████▊  | 176432/225000 [336:16:01<68:52:15,  5.10s/it]
 78%|███████▊  | 176433/225000 [336:16:06<68:03:54,  5.05s/it]
 78%|███████▊  | 176434/225000 [336:16:11<69:27:03,  5.15s/it]
 78%|███████▊  | 176435/225000 [336:16:16<69:36:41,  5.16s/it]
 78%|███████▊  | 176436/225000 [336:16:21<68:49:35,  5.10s/it]
 78%|███████▊  | 176437/225000 [336:16:26<68:31:03,  5.08s/it]
 78%|███████▊  | 176438/225000 [336:16:32<69:54:03,  5.18s/it]
 78%|███████▊  | 176439/225000 [336:16:37<70:02:49,  5.19s/it]
 78%|███████▊  | 176440/225000 [336:16:42<71:09:30,  5.28s/it]
 78%|███████▊  | 176441/225000 [336:16:48<71:00:22,  5.26s/it]
 78%|███████▊  | 176442/225000 [336:16:53<69:09:10,  5.13s/it]
 78%|███████▊  | 176443/225000 [336:16:58<69:26:33,  5.15s/it]
 78%|███████▊  | 176444/225000 [336:17:03<68:34:37,  5.08s/it]
 78%|███████▊  | 176445/225000 [336:17:07<67:17:57,  4.99s/it]
 78%|███████▊  | 176446/225000 [336:17:12<67:05:48,  4.97s/it]
 78%|███████▊  | 176447/225000 [336:17:17<67:28:31,  5.00s/it]
 78%|███████▊  | 176448/225000 [336:17:30<97:30:15,  7.23s/it]
 78%|███████▊  | 176449/225000 [336:17:34<85:13:47,  6.32s/it]
 78%|███████▊  | 176450/225000 [336:17:42<90:51:48,  6.74s/it]
                                                              
{'loss': '0.7638', 'grad_norm': '1.088', 'learning_rate': '6.051e-06', 'epoch': '0.7842', 'num_input_tokens_seen': 557323712, 'train_runtime': '1.211e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176450/225000 [336:17:42<90:51:48,  6.74s/it]
 78%|███████▊  | 176451/225000 [336:17:45<77:17:45,  5.73s/it]
 78%|███████▊  | 176452/225000 [336:17:49<69:14:16,  5.13s/it]
 78%|███████▊  | 176453/225000 [336:17:57<80:20:21,  5.96s/it]
 78%|███████▊  | 176454/225000 [336:18:02<78:47:28,  5.84s/it]
 78%|███████▊  | 176455/225000 [336:18:06<71:21:49,  5.29s/it]
 78%|███████▊  | 176456/225000 [336:18:10<66:37:12,  4.94s/it]
 78%|███████▊  | 176457/225000 [336:18:17<73:12:57,  5.43s/it]
 78%|███████▊  | 176458/225000 [336:18:25<85:08:24,  6.31s/it]
 78%|███████▊  | 176459/225000 [336:18:29<73:57:05,  5.48s/it]
 78%|███████▊  | 176460/225000 [336:18:32<65:53:41,  4.89s/it]
 78%|███████▊  | 176461/225000 [336:18:41<78:59:52,  5.86s/it]
 78%|███████▊  | 176462/225000 [336:18:46<77:29:02,  5.75s/it]
 78%|███████▊  | 176463/225000 [336:18:50<68:21:07,  5.07s/it]
 78%|███████▊  | 176464/225000 [336:18:58<83:24:24,  6.19s/it]
 78%|███████▊  | 176465/225000 [336:19:01<70:44:51,  5.25s/it]
 78%|███████▊  | 176466/225000 [336:19:06<67:10:39,  4.98s/it]
 78%|███████▊  | 176467/225000 [336:19:10<65:09:05,  4.83s/it]
 78%|███████▊  | 176468/225000 [336:19:15<63:12:01,  4.69s/it]
 78%|███████▊  | 176469/225000 [336:19:25<84:55:18,  6.30s/it]
 78%|███████▊  | 176470/225000 [336:19:30<80:32:01,  5.97s/it]
 78%|███████▊  | 176471/225000 [336:19:34<73:01:15,  5.42s/it]
 78%|███████▊  | 176472/225000 [336:19:38<68:53:38,  5.11s/it]
 78%|███████▊  | 176473/225000 [336:19:43<66:56:57,  4.97s/it]
 78%|███████▊  | 176474/225000 [336:19:48<67:10:40,  4.98s/it]
 78%|███████▊  | 176475/225000 [336:19:57<82:53:41,  6.15s/it]
 78%|███████▊  | 176476/225000 [336:20:04<85:55:42,  6.38s/it]
 78%|███████▊  | 176477/225000 [336:20:07<74:28:23,  5.53s/it]
 78%|███████▊  | 176478/225000 [336:20:11<66:51:37,  4.96s/it]
 78%|███████▊  | 176479/225000 [336:20:20<82:53:16,  6.15s/it]
 78%|███████▊  | 176480/225000 [336:20:24<75:55:47,  5.63s/it]
 78%|███████▊  | 176481/225000 [336:20:29<71:53:22,  5.33s/it]
 78%|███████▊  | 176482/225000 [336:20:34<69:55:05,  5.19s/it]
 78%|███████▊  | 176483/225000 [336:20:39<69:39:16,  5.17s/it]
 78%|███████▊  | 176484/225000 [336:20:44<68:25:22,  5.08s/it]
 78%|███████▊  | 176485/225000 [336:20:49<66:59:13,  4.97s/it]
 78%|███████▊  | 176486/225000 [336:20:53<64:50:24,  4.81s/it]
 78%|███████▊  | 176487/225000 [336:21:00<71:39:47,  5.32s/it]
 78%|███████▊  | 176488/225000 [336:21:09<89:52:37,  6.67s/it]
 78%|███████▊  | 176489/225000 [336:21:13<79:45:13,  5.92s/it]
 78%|███████▊  | 176490/225000 [336:21:18<74:02:07,  5.49s/it]
 78%|███████▊  | 176491/225000 [336:21:22<69:05:36,  5.13s/it]
 78%|███████▊  | 176492/225000 [336:21:27<66:33:06,  4.94s/it]
 78%|███████▊  | 176493/225000 [336:21:38<92:14:20,  6.85s/it]
 78%|███████▊  | 176494/225000 [336:21:41<77:05:53,  5.72s/it]
 78%|███████▊  | 176495/225000 [336:21:45<69:50:00,  5.18s/it]
 78%|███████▊  | 176496/225000 [336:21:51<70:53:00,  5.26s/it]
 78%|███████▊  | 176497/225000 [336:22:00<87:11:38,  6.47s/it]
 78%|███████▊  | 176498/225000 [336:22:03<75:28:10,  5.60s/it]
 78%|███████▊  | 176499/225000 [336:22:07<67:53:19,  5.04s/it]
 78%|███████▊  | 176500/225000 [336:22:17<88:19:53,  6.56s/it]
                                                              
{'loss': '0.7616', 'grad_norm': '1.065', 'learning_rate': '6.039e-06', 'epoch': '0.7844', 'num_input_tokens_seen': 557454248, 'train_runtime': '1.211e+06', 'train_tokens_per_second': '460.3'}

 78%|███████▊  | 176500/225000 [336:22:17<88:19:53,  6.56s/it]
 78%|███████▊  | 176501/225000 [336:22:21<75:15:46,  5.59s/it]
 78%|███████▊  | 176502/225000 [336:22:29<85:51:32,  6.37s/it]
 78%|███████▊  | 176503/225000 [336:22:32<74:38:06,  5.54s/it]
 78%|███████▊  | 176504/225000 [336:22:36<66:24:51,  4.93s/it]
 78%|███████▊  | 176505/225000 [336:22:40<63:38:30,  4.72s/it]
 78%|███████▊  | 176506/225000 [336:22:50<83:32:42,  6.20s/it]
 78%|███████▊  | 176507/225000 [336:22:53<72:59:44,  5.42s/it]
 78%|███████▊  | 176508/225000 [336:23:01<79:59:29,  5.94s/it]
 78%|███████▊  | 176509/225000 [336:23:04<71:53:39,  5.34s/it]
 78%|███████▊  | 176510/225000 [336:23:08<66:14:02,  4.92s/it]
 78%|███████▊  | 176511/225000 [336:23:15<72:29:27,  5.38s/it]
 78%|███████▊  | 176512/225000 [336:23:22<80:03:47,  5.94s/it]
 78%|███████▊  | 176513/225000 [336:23:26<72:34:08,  5.39s/it]
 78%|███████▊  | 176514/225000 [336:23:30<67:51:09,  5.04s/it]
 78%|███████▊  | 176515/225000 [336:23:35<65:26:40,  4.86s/it]
 78%|███████▊  | 176516/225000 [336:23:39<60:50:22,  4.52s/it]
 78%|███████▊  | 176517/225000 [336:23:44<63:04:23,  4.68s/it]
 78%|███████▊  | 176518/225000 [336:23:49<64:32:39,  4.79s/it]
 78%|███████▊  | 176519/225000 [336:23:54<65:52:01,  4.89s/it]
 78%|███████▊  | 176520/225000 [336:23:59<66:56:40,  4.97s/it]
 78%|███████▊  | 176521/225000 [336:24:04<67:28:31,  5.01s/it]
 78%|███████▊  | 176522/225000 [336:24:12<79:59:14,  5.94s/it]
 78%|███████▊  | 176523/225000 [336:24:21<90:22:44,  6.71s/it]
 78%|███████▊  | 176524/225000 [336:24:24<78:27:59,  5.83s/it]
 78%|███████▊  | 176525/225000 [336:24:30<77:07:49,  5.73s/it]
 78%|███████▊  | 176526/225000 [336:24:39<91:02:52,  6.76s/it]
 78%|███████▊  | 176527/225000 [336:24:42<77:20:43,  5.74s/it]
 78%|███████▊  | 176528/225000 [336:24:48<77:51:05,  5.78s/it]
 78%|███████▊  | 176529/225000 [336:24:52<68:39:38,  5.10s/it]
 78%|███████▊  | 176530/225000 [336:24:59<74:56:02,  5.57s/it]
 78%|███████▊  | 176531/225000 [336:25:03<69:15:31,  5.14s/it]
 78%|███████▊  | 176532/225000 [336:25:07<66:05:15,  4.91s/it]
 78%|███████▊  | 176533/225000 [336:25:11<63:31:17,  4.72s/it]
 78%|███████▊  | 176534/225000 [336:25:21<82:37:26,  6.14s/it]
 78%|███████▊  | 176535/225000 [336:25:25<73:29:30,  5.46s/it]
 78%|███████▊  | 176536/225000 [336:25:28<66:32:26,  4.94s/it]
 78%|███████▊  | 176537/225000 [336:25:38<83:43:51,  6.22s/it]
 78%|███████▊  | 176538/225000 [336:25:41<72:41:16,  5.40s/it]
 78%|███████▊  | 176539/225000 [336:25:49<82:05:24,  6.10s/it]
 78%|███████▊  | 176540/225000 [336:25:53<73:30:57,  5.46s/it]
 78%|███████▊  | 176541/225000 [336:25:56<66:10:29,  4.92s/it]
 78%|███████▊  | 176542/225000 [336:26:03<72:11:53,  5.36s/it]
 78%|███████▊  | 176543/225000 [336:26:10<81:01:11,  6.02s/it]
 78%|███████▊  | 176544/225000 [336:26:15<73:29:59,  5.46s/it]
 78%|███████▊  | 176545/225000 [336:26:20<72:28:39,  5.38s/it]
 78%|███████▊  | 176546/225000 [336:26:25<72:25:57,  5.38s/it]
 78%|███████▊  | 176547/225000 [336:26:30<70:51:05,  5.26s/it]
 78%|███████▊  | 176548/225000 [336:26:35<71:18:59,  5.30s/it]
 78%|███████▊  | 176549/225000 [336:26:41<72:16:42,  5.37s/it]
 78%|███████▊  | 176550/225000 [336:26:47<74:33:27,  5.54s/it]
                                                              
{'loss': '0.767', 'grad_norm': '1.104', 'learning_rate': '6.027e-06', 'epoch': '0.7847', 'num_input_tokens_seen': 557585048, 'train_runtime': '1.211e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 176550/225000 [336:26:47<74:33:27,  5.54s/it]
 78%|███████▊  | 176551/225000 [336:26:54<81:24:30,  6.05s/it]
 78%|███████▊  | 176552/225000 [336:26:57<70:13:28,  5.22s/it]
 78%|███████▊  | 176553/225000 [336:27:05<81:30:33,  6.06s/it]
 78%|███████▊  | 176554/225000 [336:27:09<71:25:33,  5.31s/it]
 78%|███████▊  | 176555/225000 [336:27:13<67:53:26,  5.05s/it]
 78%|███████▊  | 176556/225000 [336:27:18<65:14:11,  4.85s/it]
 78%|███████▊  | 176557/225000 [336:27:22<63:25:19,  4.71s/it]
 78%|███████▊  | 176558/225000 [336:27:27<64:19:18,  4.78s/it]
 78%|███████▊  | 176559/225000 [336:27:32<63:58:06,  4.75s/it]
 78%|███████▊  | 176560/225000 [336:27:42<85:43:59,  6.37s/it]
 78%|███████▊  | 176561/225000 [336:27:48<82:15:22,  6.11s/it]
 78%|███████▊  | 176562/225000 [336:27:52<74:14:01,  5.52s/it]
 78%|███████▊  | 176563/225000 [336:27:56<69:20:00,  5.15s/it]
 78%|███████▊  | 176564/225000 [336:28:05<85:52:37,  6.38s/it]
 78%|███████▊  | 176565/225000 [336:28:10<80:40:33,  6.00s/it]
 78%|███████▊  | 176566/225000 [336:28:18<85:54:17,  6.39s/it]
 78%|███████▊  | 176567/225000 [336:28:21<73:43:26,  5.48s/it]
 78%|███████▊  | 176568/225000 [336:28:25<68:37:59,  5.10s/it]
 78%|███████▊  | 176569/225000 [336:28:34<85:12:54,  6.33s/it]
 78%|███████▊  | 176570/225000 [336:28:39<77:39:15,  5.77s/it]
 78%|███████▊  | 176571/225000 [336:28:42<67:56:55,  5.05s/it]
 78%|███████▊  | 176572/225000 [336:28:46<64:43:18,  4.81s/it]
 78%|███████▊  | 176573/225000 [336:28:51<64:40:33,  4.81s/it]
 78%|███████▊  | 176574/225000 [336:28:56<64:20:22,  4.78s/it]
 78%|███████▊  | 176575/225000 [336:29:01<65:15:00,  4.85s/it]
 78%|███████▊  | 176576/225000 [336:29:06<65:46:50,  4.89s/it]
 78%|███████▊  | 176577/225000 [336:29:11<66:03:43,  4.91s/it]
 78%|███████▊  | 176578/225000 [336:29:16<66:52:27,  4.97s/it]
 78%|███████▊  | 176579/225000 [336:29:23<76:14:56,  5.67s/it]
 78%|███████▊  | 176580/225000 [336:29:34<96:44:53,  7.19s/it]
 78%|███████▊  | 176581/225000 [336:29:38<84:14:30,  6.26s/it]
 78%|███████▊  | 176582/225000 [336:29:42<72:56:40,  5.42s/it]
 78%|███████▊  | 176583/225000 [336:29:47<73:05:53,  5.44s/it]
 78%|███████▊  | 176584/225000 [336:29:56<85:35:34,  6.36s/it]
 78%|███████▊  | 176585/225000 [336:30:02<87:11:04,  6.48s/it]
 78%|███████▊  | 176586/225000 [336:30:06<75:03:18,  5.58s/it]
 78%|███████▊  | 176587/225000 [336:30:10<67:54:28,  5.05s/it]
 78%|███████▊  | 176588/225000 [336:30:17<77:01:03,  5.73s/it]
 78%|███████▊  | 176589/225000 [336:30:21<69:04:29,  5.14s/it]
 78%|███████▊  | 176590/225000 [336:30:24<63:03:20,  4.69s/it]
 78%|███████▊  | 176591/225000 [336:30:34<81:29:07,  6.06s/it]
 78%|███████▊  | 176592/225000 [336:30:38<74:45:42,  5.56s/it]
 78%|███████▊  | 176593/225000 [336:30:43<70:45:13,  5.26s/it]
 78%|███████▊  | 176594/225000 [336:30:47<67:47:30,  5.04s/it]
 78%|███████▊  | 176595/225000 [336:30:52<66:22:50,  4.94s/it]
 78%|███████▊  | 176596/225000 [336:30:56<64:17:34,  4.78s/it]
 78%|███████▊  | 176597/225000 [336:31:01<63:12:11,  4.70s/it]
 78%|███████▊  | 176598/225000 [336:31:08<71:34:04,  5.32s/it]
 78%|███████▊  | 176599/225000 [336:31:12<68:27:16,  5.09s/it]
 78%|███████▊  | 176600/225000 [336:31:17<67:47:15,  5.04s/it]
                                                              
{'loss': '0.7696', 'grad_norm': '1.2', 'learning_rate': '6.015e-06', 'epoch': '0.7849', 'num_input_tokens_seen': 557715136, 'train_runtime': '1.211e+06', 'train_tokens_per_second': '460.4'}

 78%|███████▊  | 176600/225000 [336:31:17<67:47:15,  5.04s/it]
 78%|███████▊  | 176601/225000 [336:31:22<66:35:33,  4.95s/it]
 78%|███████▊  | 176602/225000 [336:31:31<82:39:33,  6.15s/it]
 78%|███████▊  | 176603/225000 [336:31:38<88:41:17,  6.60s/it]
 78%|███████▊  | 176604/225000 [336:31:43<80:56:53,  6.02s/it]
 78%|███████▊  | 176605/225000 [336:31:48<77:29:38,  5.76s/it]
 78%|███████▊  | 176606/225000 [336:31:52<70:34:55,  5.25s/it]
 78%|███████▊  | 176607/225000 [336:31:56<64:31:05,  4.80s/it]
 78%|███████▊  | 176608/225000 [336:32:03<72:52:04,  5.42s/it]
 78%|███████▊  | 176609/225000 [336:32:11<82:53:49,  6.17s/it]
 78%|███████▊  | 176610/225000 [336:32:17<83:29:00,  6.21s/it]
 78%|███████▊  | 176611/225000 [336:32:21<72:43:54,  5.41s/it]
 78%|███████▊  | 176612/225000 [336:32:25<66:36:33,  4.96s/it]
 78%|███████▊  | 176613/225000 [336:32:34<84:50:26,  6.31s/it]
 78%|███████▊  | 176614/225000 [336:32:40<81:41:42,  6.08s/it]
 78%|███████▊  | 176615/225000 [336:32:44<74:14:53,  5.52s/it]
 78%|███████▊  | 176616/225000 [336:32:47<66:09:21,  4.92s/it]
 78%|███████▊  | 176617/225000 [336:32:56<81:50:30,  6.09s/it]
 78%|███████▊  | 176618/225000 [336:33:01<75:32:59,  5.62s/it]
 78%|███████▊  | 176619/225000 [336:33:05<71:57:24,  5.35s/it]
 78%|███████▊  | 176620/225000 [336:33:10<68:51:19,  5.12s/it]
 78%|███████▊  | 176621/225000 [336:33:15<66:41:13,  4.96s/it]
 78%|███████▊  | 176622/225000 [336:33:19<65:45:49,  4.89s/it]
 78%|███████▊  | 176623/225000 [336:33:24<65:36:56,  4.88s/it]
 78%|███████▊  | 176624/225000 [336:33:29<64:38:18,  4.81s/it]
 78%|███████▊  | 176625/225000 [336:33:34<66:17:00,  4.93s/it]
 79%|███████▊  | 176626/225000 [336:33:38<63:46:02,  4.75s/it]
 79%|███████▊  | 176627/225000 [336:33:44<66:00:49,  4.91s/it]
 79%|███████▊  | 176628/225000 [336:33:55<90:45:09,  6.75s/it]
 79%|███████▊  | 176629/225000 [336:34:00<86:44:20,  6.46s/it]
 79%|███████▊  | 176630/225000 [336:34:05<79:02:57,  5.88s/it]
 79%|███████▊  | 176631/225000 [336:34:10<75:01:00,  5.58s/it]
 79%|███████▊  | 176632/225000 [336:34:14<71:03:12,  5.29s/it]
 79%|███████▊  | 176633/225000 [336:34:19<68:01:34,  5.06s/it]
 79%|███████▊  | 176634/225000 [336:34:24<65:52:38,  4.90s/it]
 79%|███████▊  | 176635/225000 [336:34:29<66:47:20,  4.97s/it]
 79%|███████▊  | 176636/225000 [336:34:33<65:48:51,  4.90s/it]
 79%|███████▊  | 176637/225000 [336:34:38<65:33:08,  4.88s/it]
 79%|███████▊  | 176638/225000 [336:34:43<64:15:00,  4.78s/it]
 79%|███████▊  | 176639/225000 [336:34:48<66:16:56,  4.93s/it]
 79%|███████▊  | 176640/225000 [336:34:52<63:23:36,  4.72s/it]
 79%|███████▊  | 176641/225000 [336:35:03<85:55:06,  6.40s/it]
 79%|███████▊  | 176642/225000 [336:35:06<73:03:35,  5.44s/it]
 79%|███████▊  | 176643/225000 [336:35:11<73:37:32,  5.48s/it]
 79%|███████▊  | 176644/225000 [336:35:19<83:02:32,  6.18s/it]
 79%|███████▊  | 176645/225000 [336:35:23<71:31:40,  5.33s/it]
 79%|███████▊  | 176646/225000 [336:35:27<66:45:55,  4.97s/it]
 79%|███████▊  | 176647/225000 [336:35:32<67:42:45,  5.04s/it]
 79%|███████▊  | 176648/225000 [336:35:37<67:44:23,  5.04s/it]
 79%|███████▊  | 176649/225000 [336:35:42<67:19:19,  5.01s/it]
 79%|███████▊  | 176650/225000 [336:35:47<66:27:19,  4.95s/it]
                                                              
{'loss': '0.7697', 'grad_norm': '1.335', 'learning_rate': '6.004e-06', 'epoch': '0.7851', 'num_input_tokens_seen': 557843872, 'train_runtime': '1.212e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176650/225000 [336:35:47<66:27:19,  4.95s/it]
 79%|███████▊  | 176651/225000 [336:35:51<65:08:48,  4.85s/it]
 79%|███████▊  | 176652/225000 [336:35:56<65:44:33,  4.90s/it]
 79%|███████▊  | 176653/225000 [336:36:05<81:11:09,  6.05s/it]
 79%|███████▊  | 176654/225000 [336:36:13<89:06:28,  6.64s/it]
 79%|███████▊  | 176655/225000 [336:36:16<74:46:35,  5.57s/it]
 79%|███████▊  | 176656/225000 [336:36:25<89:53:51,  6.69s/it]
 79%|███████▊  | 176657/225000 [336:36:29<75:39:13,  5.63s/it]
 79%|███████▊  | 176658/225000 [336:36:38<89:01:35,  6.63s/it]
 79%|███████▊  | 176659/225000 [336:36:41<75:35:42,  5.63s/it]
 79%|███████▊  | 176660/225000 [336:36:45<69:59:33,  5.21s/it]
 79%|███████▊  | 176661/225000 [336:36:50<66:49:35,  4.98s/it]
 79%|███████▊  | 176662/225000 [336:36:54<65:18:44,  4.86s/it]
 79%|███████▊  | 176663/225000 [336:36:59<64:17:18,  4.79s/it]
 79%|███████▊  | 176664/225000 [336:37:04<64:36:39,  4.81s/it]
 79%|███████▊  | 176665/225000 [336:37:13<84:21:15,  6.28s/it]
 79%|███████▊  | 176666/225000 [336:37:19<80:17:56,  5.98s/it]
 79%|███████▊  | 176667/225000 [336:37:25<83:39:10,  6.23s/it]
 79%|███████▊  | 176668/225000 [336:37:29<73:18:53,  5.46s/it]
 79%|███████▊  | 176669/225000 [336:37:33<65:46:53,  4.90s/it]
 79%|███████▊  | 176670/225000 [336:37:41<81:22:53,  6.06s/it]
 79%|███████▊  | 176671/225000 [336:37:47<77:48:39,  5.80s/it]
 79%|███████▊  | 176672/225000 [336:37:52<74:19:23,  5.54s/it]
 79%|███████▊  | 176673/225000 [336:37:56<70:00:50,  5.22s/it]
 79%|███████▊  | 176674/225000 [336:38:01<68:04:25,  5.07s/it]
 79%|███████▊  | 176675/225000 [336:38:06<68:10:20,  5.08s/it]
 79%|███████▊  | 176676/225000 [336:38:11<68:27:51,  5.10s/it]
 79%|███████▊  | 176677/225000 [336:38:16<68:34:28,  5.11s/it]
 79%|███████▊  | 176678/225000 [336:38:21<67:46:01,  5.05s/it]
 79%|███████▊  | 176679/225000 [336:38:26<66:31:27,  4.96s/it]
 79%|███████▊  | 176680/225000 [336:38:31<66:01:31,  4.92s/it]
 79%|███████▊  | 176681/225000 [336:38:35<64:35:15,  4.81s/it]
 79%|███████▊  | 176682/225000 [336:38:40<63:20:46,  4.72s/it]
 79%|███████▊  | 176683/225000 [336:38:45<64:12:21,  4.78s/it]
 79%|███████▊  | 176684/225000 [336:38:56<91:47:01,  6.84s/it]
 79%|███████▊  | 176685/225000 [336:39:03<89:36:33,  6.68s/it]
 79%|███████▊  | 176686/225000 [336:39:06<76:36:31,  5.71s/it]
 79%|███████▊  | 176687/225000 [336:39:12<77:36:10,  5.78s/it]
 79%|███████▊  | 176688/225000 [336:39:19<81:11:51,  6.05s/it]
 79%|███████▊  | 176689/225000 [336:39:25<81:03:42,  6.04s/it]
 79%|███████▊  | 176690/225000 [336:39:31<84:23:15,  6.29s/it]
 79%|███████▊  | 176691/225000 [336:39:36<78:15:07,  5.83s/it]
 79%|███████▊  | 176692/225000 [336:39:40<70:10:26,  5.23s/it]
 79%|███████▊  | 176693/225000 [336:39:46<72:07:38,  5.38s/it]
 79%|███████▊  | 176694/225000 [336:39:51<70:38:07,  5.26s/it]
 79%|███████▊  | 176695/225000 [336:39:56<70:13:28,  5.23s/it]
 79%|███████▊  | 176696/225000 [336:40:01<69:49:16,  5.20s/it]
 79%|███████▊  | 176697/225000 [336:40:06<67:22:19,  5.02s/it]
 79%|███████▊  | 176698/225000 [336:40:10<66:01:19,  4.92s/it]
 79%|███████▊  | 176699/225000 [336:40:15<66:07:50,  4.93s/it]
 79%|███████▊  | 176700/225000 [336:40:20<67:07:38,  5.00s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.172', 'learning_rate': '5.992e-06', 'epoch': '0.7853', 'num_input_tokens_seen': 557973064, 'train_runtime': '1.212e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176700/225000 [336:40:20<67:07:38,  5.00s/it]
 79%|███████▊  | 176701/225000 [336:40:26<67:33:09,  5.04s/it]
 79%|███████▊  | 176702/225000 [336:40:31<68:08:39,  5.08s/it]
 79%|███████▊  | 176703/225000 [336:40:36<66:47:22,  4.98s/it]
 79%|███████▊  | 176704/225000 [336:40:41<67:26:53,  5.03s/it]
 79%|███████▊  | 176705/225000 [336:40:46<69:22:07,  5.17s/it]
 79%|███████▊  | 176706/225000 [336:40:51<68:52:03,  5.13s/it]
 79%|███████▊  | 176707/225000 [336:41:03<93:55:09,  7.00s/it]
 79%|███████▊  | 176708/225000 [336:41:08<86:46:36,  6.47s/it]
 79%|███████▊  | 176709/225000 [336:41:13<82:44:01,  6.17s/it]
 79%|███████▊  | 176710/225000 [336:41:18<77:50:06,  5.80s/it]
 79%|███████▊  | 176711/225000 [336:41:22<70:31:09,  5.26s/it]
 79%|███████▊  | 176712/225000 [336:41:28<73:28:20,  5.48s/it]
 79%|███████▊  | 176713/225000 [336:41:36<82:51:50,  6.18s/it]
 79%|███████▊  | 176714/225000 [336:41:41<78:16:26,  5.84s/it]
 79%|███████▊  | 176715/225000 [336:41:47<78:06:30,  5.82s/it]
 79%|███████▊  | 176716/225000 [336:41:51<72:12:18,  5.38s/it]
 79%|███████▊  | 176717/225000 [336:41:56<70:09:08,  5.23s/it]
 79%|███████▊  | 176718/225000 [336:42:01<68:19:03,  5.09s/it]
 79%|███████▊  | 176719/225000 [336:42:05<65:13:55,  4.86s/it]
 79%|███████▊  | 176720/225000 [336:42:09<62:43:36,  4.68s/it]
 79%|███████▊  | 176721/225000 [336:42:14<61:52:51,  4.61s/it]
 79%|███████▊  | 176722/225000 [336:42:18<61:08:16,  4.56s/it]
 79%|███████▊  | 176723/225000 [336:42:23<60:55:39,  4.54s/it]
 79%|███████▊  | 176724/225000 [336:42:30<71:33:10,  5.34s/it]
 79%|███████▊  | 176725/225000 [336:42:42<98:52:29,  7.37s/it]
 79%|███████▊  | 176726/225000 [336:42:45<82:03:42,  6.12s/it]
 79%|███████▊  | 176727/225000 [336:42:51<80:57:06,  6.04s/it]
 79%|███████▊  | 176728/225000 [336:42:56<77:23:53,  5.77s/it]
 79%|███████▊  | 176729/225000 [336:43:01<71:44:50,  5.35s/it]
 79%|███████▊  | 176730/225000 [336:43:06<69:49:17,  5.21s/it]
 79%|███████▊  | 176731/225000 [336:43:10<68:09:27,  5.08s/it]
 79%|███████▊  | 176732/225000 [336:43:15<67:43:49,  5.05s/it]
 79%|███████▊  | 176733/225000 [336:43:20<66:39:04,  4.97s/it]
 79%|███████▊  | 176734/225000 [336:43:25<65:36:37,  4.89s/it]
 79%|███████▊  | 176735/225000 [336:43:38<97:02:18,  7.24s/it]
 79%|███████▊  | 176736/225000 [336:43:42<85:01:28,  6.34s/it]
 79%|███████▊  | 176737/225000 [336:43:47<78:47:50,  5.88s/it]
 79%|███████▊  | 176738/225000 [336:43:51<74:31:16,  5.56s/it]
 79%|███████▊  | 176739/225000 [336:43:56<71:56:44,  5.37s/it]
 79%|███████▊  | 176740/225000 [336:44:01<68:51:47,  5.14s/it]
 79%|███████▊  | 176741/225000 [336:44:06<68:41:27,  5.12s/it]
 79%|███████▊  | 176742/225000 [336:44:11<68:25:05,  5.10s/it]
 79%|███████▊  | 176743/225000 [336:44:16<67:26:01,  5.03s/it]
 79%|███████▊  | 176744/225000 [336:44:21<66:21:11,  4.95s/it]
 79%|███████▊  | 176745/225000 [336:44:26<66:15:11,  4.94s/it]
 79%|███████▊  | 176746/225000 [336:44:31<66:17:34,  4.95s/it]
 79%|███████▊  | 176747/225000 [336:44:35<66:05:03,  4.93s/it]
 79%|███████▊  | 176748/225000 [336:44:41<67:18:04,  5.02s/it]
 79%|███████▊  | 176749/225000 [336:44:46<68:50:55,  5.14s/it]
 79%|███████▊  | 176750/225000 [336:44:58<96:20:04,  7.19s/it]
                                                              
{'loss': '0.7596', 'grad_norm': '1.168', 'learning_rate': '5.98e-06', 'epoch': '0.7856', 'num_input_tokens_seen': 558101960, 'train_runtime': '1.212e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176750/225000 [336:44:58<96:20:04,  7.19s/it]
 79%|███████▊  | 176751/225000 [336:45:02<81:52:22,  6.11s/it]
 79%|███████▊  | 176752/225000 [336:45:06<75:52:36,  5.66s/it]
 79%|███████▊  | 176753/225000 [336:45:11<70:17:10,  5.24s/it]
 79%|███████▊  | 176754/225000 [336:45:15<66:34:47,  4.97s/it]
 79%|███████▊  | 176755/225000 [336:45:21<69:23:30,  5.18s/it]
 79%|███████▊  | 176756/225000 [336:45:33<98:11:34,  7.33s/it]
 79%|███████▊  | 176757/225000 [336:45:36<81:53:32,  6.11s/it]
 79%|███████▊  | 176758/225000 [336:45:40<71:42:29,  5.35s/it]
 79%|███████▊  | 176759/225000 [336:45:47<80:35:45,  6.01s/it]
 79%|███████▊  | 176760/225000 [336:45:52<74:24:54,  5.55s/it]
 79%|███████▊  | 176761/225000 [336:45:56<70:33:28,  5.27s/it]
 79%|███████▊  | 176762/225000 [336:46:01<67:19:57,  5.03s/it]
 79%|███████▊  | 176763/225000 [336:46:05<63:07:39,  4.71s/it]
 79%|███████▊  | 176764/225000 [336:46:16<88:11:26,  6.58s/it]
 79%|███████▊  | 176765/225000 [336:46:20<78:33:39,  5.86s/it]
 79%|███████▊  | 176766/225000 [336:46:24<69:33:26,  5.19s/it]
 79%|███████▊  | 176767/225000 [336:46:33<85:10:58,  6.36s/it]
 79%|███████▊  | 176768/225000 [336:46:36<74:19:15,  5.55s/it]
 79%|███████▊  | 176769/225000 [336:46:43<77:29:27,  5.78s/it]
 79%|███████▊  | 176770/225000 [336:46:46<68:29:53,  5.11s/it]
 79%|███████▊  | 176771/225000 [336:46:51<65:32:05,  4.89s/it]
 79%|███████▊  | 176772/225000 [336:46:55<64:10:36,  4.79s/it]
 79%|███████▊  | 176773/225000 [336:47:04<79:02:38,  5.90s/it]
 79%|███████▊  | 176774/225000 [336:47:10<82:10:19,  6.13s/it]
 79%|███████▊  | 176775/225000 [336:47:16<79:25:28,  5.93s/it]
 79%|███████▊  | 176776/225000 [336:47:22<79:36:52,  5.94s/it]
 79%|███████▊  | 176777/225000 [336:47:27<78:29:57,  5.86s/it]
 79%|███████▊  | 176778/225000 [336:47:33<76:33:03,  5.71s/it]
 79%|███████▊  | 176779/225000 [336:47:36<67:34:20,  5.04s/it]
 79%|███████▊  | 176780/225000 [336:47:45<80:33:14,  6.01s/it]
 79%|███████▊  | 176781/225000 [336:47:48<68:40:51,  5.13s/it]
 79%|███████▊  | 176782/225000 [336:47:54<72:06:05,  5.38s/it]
 79%|███████▊  | 176783/225000 [336:47:58<67:56:06,  5.07s/it]
 79%|███████▊  | 176784/225000 [336:48:05<74:57:18,  5.60s/it]
 79%|███████▊  | 176785/225000 [336:48:09<69:54:46,  5.22s/it]
 79%|███████▊  | 176786/225000 [336:48:17<79:36:52,  5.94s/it]
 79%|███████▊  | 176787/225000 [336:48:20<69:11:54,  5.17s/it]
 79%|███████▊  | 176788/225000 [336:48:25<70:06:03,  5.23s/it]
 79%|███████▊  | 176789/225000 [336:48:31<70:36:33,  5.27s/it]
 79%|███████▊  | 176790/225000 [336:48:36<69:55:59,  5.22s/it]
 79%|███████▊  | 176791/225000 [336:48:41<68:25:39,  5.11s/it]
 79%|███████▊  | 176792/225000 [336:48:46<67:48:05,  5.06s/it]
 79%|███████▊  | 176793/225000 [336:48:51<70:14:52,  5.25s/it]
 79%|███████▊  | 176794/225000 [336:48:57<71:21:12,  5.33s/it]
 79%|███████▊  | 176795/225000 [336:49:04<78:55:29,  5.89s/it]
 79%|███████▊  | 176796/225000 [336:49:08<71:20:10,  5.33s/it]
 79%|███████▊  | 176797/225000 [336:49:12<67:19:03,  5.03s/it]
 79%|███████▊  | 176798/225000 [336:49:17<64:36:14,  4.82s/it]
 79%|███████▊  | 176799/225000 [336:49:21<61:27:35,  4.59s/it]
 79%|███████▊  | 176800/225000 [336:49:25<61:09:26,  4.57s/it]
                                                              
{'loss': '0.7714', 'grad_norm': '1.155', 'learning_rate': '5.968e-06', 'epoch': '0.7858', 'num_input_tokens_seen': 558231208, 'train_runtime': '1.213e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176800/225000 [336:49:25<61:09:26,  4.57s/it]
 79%|███████▊  | 176801/225000 [336:49:33<72:32:02,  5.42s/it]
 79%|███████▊  | 176802/225000 [336:49:43<92:33:10,  6.91s/it]
 79%|███████▊  | 176803/225000 [336:49:46<77:39:39,  5.80s/it]
 79%|███████▊  | 176804/225000 [336:49:53<82:31:17,  6.16s/it]
 79%|███████▊  | 176805/225000 [336:49:58<77:04:36,  5.76s/it]
 79%|███████▊  | 176806/225000 [336:50:03<72:35:28,  5.42s/it]
 79%|███████▊  | 176807/225000 [336:50:08<70:53:46,  5.30s/it]
 79%|███████▊  | 176808/225000 [336:50:13<68:59:01,  5.15s/it]
 79%|███████▊  | 176809/225000 [336:50:17<66:56:46,  5.00s/it]
 79%|███████▊  | 176810/225000 [336:50:22<66:39:42,  4.98s/it]
 79%|███████▊  | 176811/225000 [336:50:27<64:46:53,  4.84s/it]
 79%|███████▊  | 176812/225000 [336:50:32<64:44:12,  4.84s/it]
 79%|███████▊  | 176813/225000 [336:50:43<89:23:48,  6.68s/it]
 79%|███████▊  | 176814/225000 [336:50:46<76:57:15,  5.75s/it]
 79%|███████▊  | 176815/225000 [336:50:54<84:23:26,  6.31s/it]
 79%|███████▊  | 176816/225000 [336:50:58<77:11:02,  5.77s/it]
 79%|███████▊  | 176817/225000 [336:51:03<72:11:30,  5.39s/it]
 79%|███████▊  | 176818/225000 [336:51:07<69:25:28,  5.19s/it]
 79%|███████▊  | 176819/225000 [336:51:12<68:21:16,  5.11s/it]
 79%|███████▊  | 176820/225000 [336:51:17<66:35:58,  4.98s/it]
 79%|███████▊  | 176821/225000 [336:51:22<65:23:09,  4.89s/it]
 79%|███████▊  | 176822/225000 [336:51:26<63:55:31,  4.78s/it]
 79%|███████▊  | 176823/225000 [336:51:31<61:47:55,  4.62s/it]
 79%|███████▊  | 176824/225000 [336:51:40<82:38:12,  6.18s/it]
 79%|███████▊  | 176825/225000 [336:51:47<83:41:56,  6.25s/it]
 79%|███████▊  | 176826/225000 [336:51:50<72:31:10,  5.42s/it]
 79%|███████▊  | 176827/225000 [336:52:00<88:32:25,  6.62s/it]
 79%|███████▊  | 176828/225000 [336:52:03<77:01:15,  5.76s/it]
 79%|███████▊  | 176829/225000 [336:52:07<69:51:53,  5.22s/it]
 79%|███████▊  | 176830/225000 [336:52:14<74:20:19,  5.56s/it]
 79%|███████▊  | 176831/225000 [336:52:21<82:58:16,  6.20s/it]
 79%|███████▊  | 176832/225000 [336:52:25<71:00:42,  5.31s/it]
 79%|███████▊  | 176833/225000 [336:52:33<82:54:57,  6.20s/it]
 79%|███████▊  | 176834/225000 [336:52:37<73:12:02,  5.47s/it]
 79%|███████▊  | 176835/225000 [336:52:40<64:13:13,  4.80s/it]
 79%|███████▊  | 176836/225000 [336:52:45<63:41:32,  4.76s/it]
 79%|███████▊  | 176837/225000 [336:52:50<65:14:59,  4.88s/it]
 79%|███████▊  | 176838/225000 [336:52:55<66:51:41,  5.00s/it]
 79%|███████▊  | 176839/225000 [336:53:00<68:44:04,  5.14s/it]
 79%|███████▊  | 176840/225000 [336:53:06<69:59:15,  5.23s/it]
 79%|███████▊  | 176841/225000 [336:53:11<69:49:20,  5.22s/it]
 79%|███████▊  | 176842/225000 [336:53:17<70:59:26,  5.31s/it]
 79%|███████▊  | 176843/225000 [336:53:22<71:50:11,  5.37s/it]
 79%|███████▊  | 176844/225000 [336:53:27<70:09:54,  5.25s/it]
 79%|███████▊  | 176845/225000 [336:53:32<70:27:30,  5.27s/it]
 79%|███████▊  | 176846/225000 [336:53:38<71:07:01,  5.32s/it]
 79%|███████▊  | 176847/225000 [336:53:43<70:29:42,  5.27s/it]
 79%|███████▊  | 176848/225000 [336:53:48<69:32:17,  5.20s/it]
 79%|███████▊  | 176849/225000 [336:53:53<67:42:08,  5.06s/it]
 79%|███████▊  | 176850/225000 [336:53:57<66:17:00,  4.96s/it]
                                                              
{'loss': '0.76', 'grad_norm': '1.179', 'learning_rate': '5.956e-06', 'epoch': '0.786', 'num_input_tokens_seen': 558361536, 'train_runtime': '1.213e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176850/225000 [336:53:57<66:17:00,  4.96s/it]
 79%|███████▊  | 176851/225000 [336:54:03<66:49:32,  5.00s/it]
 79%|███████▊  | 176852/225000 [336:54:08<67:51:03,  5.07s/it]
 79%|███████▊  | 176853/225000 [336:54:13<68:15:44,  5.10s/it]
 79%|███████▊  | 176854/225000 [336:54:20<76:33:15,  5.72s/it]
 79%|███████▊  | 176855/225000 [336:54:28<86:02:56,  6.43s/it]
 79%|███████▊  | 176856/225000 [336:54:33<80:58:31,  6.05s/it]
 79%|███████▊  | 176857/225000 [336:54:39<79:26:39,  5.94s/it]
 79%|███████▊  | 176858/225000 [336:54:45<78:03:41,  5.84s/it]
 79%|███████▊  | 176859/225000 [336:54:50<74:36:14,  5.58s/it]
 79%|███████▊  | 176860/225000 [336:54:55<71:45:03,  5.37s/it]
 79%|███████▊  | 176861/225000 [336:55:00<72:42:37,  5.44s/it]
 79%|███████▊  | 176862/225000 [336:55:05<70:44:58,  5.29s/it]
 79%|███████▊  | 176863/225000 [336:55:10<69:21:45,  5.19s/it]
 79%|███████▊  | 176864/225000 [336:55:15<69:41:45,  5.21s/it]
 79%|███████▊  | 176865/225000 [336:55:21<69:38:16,  5.21s/it]
 79%|███████▊  | 176866/225000 [336:55:26<70:33:32,  5.28s/it]
 79%|███████▊  | 176867/225000 [336:55:32<72:02:49,  5.39s/it]
 79%|███████▊  | 176868/225000 [336:55:37<71:14:30,  5.33s/it]
 79%|███████▊  | 176869/225000 [336:55:42<70:22:25,  5.26s/it]
 79%|███████▊  | 176870/225000 [336:55:47<70:06:00,  5.24s/it]
 79%|███████▊  | 176871/225000 [336:55:52<70:17:36,  5.26s/it]
 79%|███████▊  | 176872/225000 [336:56:00<79:39:49,  5.96s/it]
 79%|███████▊  | 176873/225000 [336:56:09<91:56:17,  6.88s/it]
 79%|███████▊  | 176874/225000 [336:56:12<77:09:14,  5.77s/it]
 79%|███████▊  | 176875/225000 [336:56:21<89:19:00,  6.68s/it]
 79%|███████▊  | 176876/225000 [336:56:25<77:01:29,  5.76s/it]
 79%|███████▊  | 176877/225000 [336:56:29<69:35:08,  5.21s/it]
 79%|███████▊  | 176878/225000 [336:56:34<69:30:07,  5.20s/it]
 79%|███████▊  | 176879/225000 [336:56:43<86:23:10,  6.46s/it]
 79%|███████▊  | 176880/225000 [336:56:46<73:44:41,  5.52s/it]
 79%|███████▊  | 176881/225000 [336:56:53<76:49:50,  5.75s/it]
 79%|███████▊  | 176882/225000 [336:56:59<79:21:21,  5.94s/it]
 79%|███████▊  | 176883/225000 [336:57:04<73:29:50,  5.50s/it]
 79%|███████▊  | 176884/225000 [336:57:08<68:23:26,  5.12s/it]
 79%|███████▊  | 176885/225000 [336:57:12<64:57:32,  4.86s/it]
 79%|███████▊  | 176886/225000 [336:57:16<63:03:45,  4.72s/it]
 79%|███████▊  | 176887/225000 [336:57:26<81:28:49,  6.10s/it]
 79%|███████▊  | 176888/225000 [336:57:33<84:42:14,  6.34s/it]
 79%|███████▊  | 176889/225000 [336:57:37<76:46:10,  5.74s/it]
 79%|███████▊  | 176890/225000 [336:57:45<86:59:33,  6.51s/it]
 79%|███████▊  | 176891/225000 [336:57:49<76:44:51,  5.74s/it]
 79%|███████▊  | 176892/225000 [336:57:53<69:50:02,  5.23s/it]
 79%|███████▊  | 176893/225000 [336:57:58<66:54:37,  5.01s/it]
 79%|███████▊  | 176894/225000 [336:58:07<84:59:41,  6.36s/it]
 79%|███████▊  | 176895/225000 [336:58:11<72:51:26,  5.45s/it]
 79%|███████▊  | 176896/225000 [336:58:16<71:00:36,  5.31s/it]
 79%|███████▊  | 176897/225000 [336:58:22<75:23:30,  5.64s/it]
 79%|███████▊  | 176898/225000 [336:58:26<66:47:17,  5.00s/it]
 79%|███████▊  | 176899/225000 [336:58:29<62:04:00,  4.65s/it]
 79%|███████▊  | 176900/225000 [336:58:39<82:53:59,  6.20s/it]
                                                              
{'loss': '0.7626', 'grad_norm': '1.217', 'learning_rate': '5.944e-06', 'epoch': '0.7862', 'num_input_tokens_seen': 558492520, 'train_runtime': '1.213e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176900/225000 [336:58:39<82:53:59,  6.20s/it]
 79%|███████▊  | 176901/225000 [336:58:43<71:48:34,  5.37s/it]
 79%|███████▊  | 176902/225000 [336:58:50<79:21:04,  5.94s/it]
 79%|███████▊  | 176903/225000 [336:58:55<74:45:18,  5.60s/it]
 79%|███████▊  | 176904/225000 [336:59:00<74:27:50,  5.57s/it]
 79%|███████▊  | 176905/225000 [336:59:05<73:12:15,  5.48s/it]
 79%|███████▊  | 176906/225000 [336:59:11<72:02:20,  5.39s/it]
 79%|███████▊  | 176907/225000 [336:59:16<71:48:11,  5.37s/it]
 79%|███████▊  | 176908/225000 [336:59:21<71:29:30,  5.35s/it]
 79%|███████▊  | 176909/225000 [336:59:26<69:17:46,  5.19s/it]
 79%|███████▊  | 176910/225000 [336:59:31<69:35:30,  5.21s/it]
 79%|███████▊  | 176911/225000 [336:59:37<69:51:25,  5.23s/it]
 79%|███████▊  | 176912/225000 [336:59:42<68:53:27,  5.16s/it]
 79%|███████▊  | 176913/225000 [336:59:47<68:01:01,  5.09s/it]
 79%|███████▊  | 176914/225000 [336:59:52<67:54:06,  5.08s/it]
 79%|███████▊  | 176915/225000 [336:59:56<66:58:33,  5.01s/it]
 79%|███████▊  | 176916/225000 [337:00:02<67:54:57,  5.08s/it]
 79%|███████▊  | 176917/225000 [337:00:07<68:03:20,  5.10s/it]
 79%|███████▊  | 176918/225000 [337:00:12<68:03:40,  5.10s/it]
 79%|███████▊  | 176919/225000 [337:00:17<67:59:43,  5.09s/it]
 79%|███████▊  | 176920/225000 [337:00:22<67:29:30,  5.05s/it]
 79%|███████▊  | 176921/225000 [337:00:27<68:18:49,  5.12s/it]
 79%|███████▊  | 176922/225000 [337:00:33<72:32:26,  5.43s/it]
 79%|███████▊  | 176923/225000 [337:00:46<100:07:38,  7.50s/it]
 79%|███████▊  | 176924/225000 [337:00:49<83:27:19,  6.25s/it] 
 79%|███████▊  | 176925/225000 [337:00:53<72:37:55,  5.44s/it]
 79%|███████▊  | 176926/225000 [337:01:01<84:47:07,  6.35s/it]
 79%|███████▊  | 176927/225000 [337:01:05<73:53:44,  5.53s/it]
 79%|███████▊  | 176928/225000 [337:01:09<68:33:05,  5.13s/it]
 79%|███████▊  | 176929/225000 [337:01:13<63:36:04,  4.76s/it]
 79%|███████▊  | 176930/225000 [337:01:22<79:47:56,  5.98s/it]
 79%|███████▊  | 176931/225000 [337:01:25<70:07:26,  5.25s/it]
 79%|███████▊  | 176932/225000 [337:01:33<80:23:28,  6.02s/it]
 79%|███████▊  | 176933/225000 [337:01:38<77:04:44,  5.77s/it]
 79%|███████▊  | 176934/225000 [337:01:43<74:42:59,  5.60s/it]
 79%|███████▊  | 176935/225000 [337:01:48<71:14:34,  5.34s/it]
 79%|███████▊  | 176936/225000 [337:01:53<68:16:31,  5.11s/it]
 79%|███████▊  | 176937/225000 [337:01:58<69:06:55,  5.18s/it]
 79%|███████▊  | 176938/225000 [337:02:03<68:38:36,  5.14s/it]
 79%|███████▊  | 176939/225000 [337:02:08<68:08:17,  5.10s/it]
 79%|███████▊  | 176940/225000 [337:02:13<67:53:37,  5.09s/it]
 79%|███████▊  | 176941/225000 [337:02:18<66:07:55,  4.95s/it]
 79%|███████▊  | 176942/225000 [337:02:23<66:18:33,  4.97s/it]
 79%|███████▊  | 176943/225000 [337:02:28<67:23:18,  5.05s/it]
 79%|███████▊  | 176944/225000 [337:02:39<92:51:46,  6.96s/it]
 79%|███████▊  | 176945/225000 [337:02:45<88:57:03,  6.66s/it]
 79%|███████▊  | 176946/225000 [337:02:49<76:31:16,  5.73s/it]
 79%|███████▊  | 176947/225000 [337:02:56<79:54:14,  5.99s/it]
 79%|███████▊  | 176948/225000 [337:03:00<75:29:47,  5.66s/it]
 79%|███████▊  | 176949/225000 [337:03:04<68:12:00,  5.11s/it]
 79%|███████▊  | 176950/225000 [337:03:09<66:28:14,  4.98s/it]
                                                              
{'loss': '0.7507', 'grad_norm': '1.134', 'learning_rate': '5.933e-06', 'epoch': '0.7864', 'num_input_tokens_seen': 558622592, 'train_runtime': '1.213e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 176950/225000 [337:03:09<66:28:14,  4.98s/it]
 79%|███████▊  | 176951/225000 [337:03:14<66:27:48,  4.98s/it]
 79%|███████▊  | 176952/225000 [337:03:19<65:24:31,  4.90s/it]
 79%|███████▊  | 176953/225000 [337:03:23<64:58:52,  4.87s/it]
 79%|███████▊  | 176954/225000 [337:03:28<63:39:45,  4.77s/it]
 79%|███████▊  | 176955/225000 [337:03:33<62:36:52,  4.69s/it]
 79%|███████▊  | 176956/225000 [337:03:41<77:03:45,  5.77s/it]
 79%|███████▊  | 176957/225000 [337:03:50<89:12:01,  6.68s/it]
 79%|███████▊  | 176958/225000 [337:03:54<80:05:17,  6.00s/it]
 79%|███████▊  | 176959/225000 [337:04:01<84:32:56,  6.34s/it]
 79%|███████▊  | 176960/225000 [337:04:05<76:31:06,  5.73s/it]
 79%|███████▊  | 176961/225000 [337:04:10<70:40:39,  5.30s/it]
 79%|███████▊  | 176962/225000 [337:04:14<66:33:26,  4.99s/it]
 79%|███████▊  | 176963/225000 [337:04:21<72:38:39,  5.44s/it]
 79%|███████▊  | 176964/225000 [337:04:29<83:08:43,  6.23s/it]
 79%|███████▊  | 176965/225000 [337:04:32<70:36:08,  5.29s/it]
 79%|███████▊  | 176966/225000 [337:04:38<75:03:53,  5.63s/it]
 79%|███████▊  | 176967/225000 [337:04:42<67:09:10,  5.03s/it]
 79%|███████▊  | 176968/225000 [337:04:50<81:56:10,  6.14s/it]
 79%|███████▊  | 176969/225000 [337:04:54<71:49:10,  5.38s/it]
 79%|███████▊  | 176970/225000 [337:04:59<69:09:57,  5.18s/it]
 79%|███████▊  | 176971/225000 [337:05:03<65:42:19,  4.92s/it]
 79%|███████▊  | 176972/225000 [337:05:08<64:13:01,  4.81s/it]
 79%|███████▊  | 176973/225000 [337:05:12<62:59:03,  4.72s/it]
 79%|███████▊  | 176974/225000 [337:05:16<61:05:06,  4.58s/it]
 79%|███████▊  | 176975/225000 [337:05:29<92:12:57,  6.91s/it]
 79%|███████▊  | 176976/225000 [337:05:34<86:13:33,  6.46s/it]
 79%|███████▊  | 176977/225000 [337:05:39<81:09:31,  6.08s/it]
 79%|███████▊  | 176978/225000 [337:05:44<76:01:27,  5.70s/it]
 79%|███████▊  | 176979/225000 [337:05:50<74:27:31,  5.58s/it]
 79%|███████▊  | 176980/225000 [337:05:55<74:10:26,  5.56s/it]
 79%|███████▊  | 176981/225000 [337:06:00<72:22:43,  5.43s/it]
 79%|███████▊  | 176982/225000 [337:06:05<70:33:01,  5.29s/it]
 79%|███████▊  | 176983/225000 [337:06:10<70:29:27,  5.28s/it]
 79%|███████▊  | 176984/225000 [337:06:16<70:33:42,  5.29s/it]
 79%|███████▊  | 176985/225000 [337:06:21<69:50:57,  5.24s/it]
 79%|███████▊  | 176986/225000 [337:06:26<70:10:35,  5.26s/it]
 79%|███████▊  | 176987/225000 [337:06:31<68:46:24,  5.16s/it]
 79%|███████▊  | 176988/225000 [337:06:36<69:05:03,  5.18s/it]
 79%|███████▊  | 176989/225000 [337:06:41<68:59:31,  5.17s/it]
 79%|███████▊  | 176990/225000 [337:06:47<68:40:07,  5.15s/it]
 79%|███████▊  | 176991/225000 [337:06:52<68:34:06,  5.14s/it]
 79%|███████▊  | 176992/225000 [337:07:01<83:46:21,  6.28s/it]
 79%|███████▊  | 176993/225000 [337:07:10<97:22:47,  7.30s/it]
 79%|███████▊  | 176994/225000 [337:07:15<85:06:24,  6.38s/it]
 79%|███████▊  | 176995/225000 [337:07:18<75:02:09,  5.63s/it]
 79%|███████▊  | 176996/225000 [337:07:22<68:00:35,  5.10s/it]
 79%|███████▊  | 176997/225000 [337:07:26<63:17:57,  4.75s/it]
 79%|███████▊  | 176998/225000 [337:07:37<87:23:03,  6.55s/it]
 79%|███████▊  | 176999/225000 [337:07:42<81:14:22,  6.09s/it]
 79%|███████▊  | 177000/225000 [337:07:46<73:27:38,  5.51s/it]
                                                              
{'loss': '0.7642', 'grad_norm': '1.167', 'learning_rate': '5.921e-06', 'epoch': '0.7867', 'num_input_tokens_seen': 558753496, 'train_runtime': '1.214e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 177000/225000 [337:07:46<73:27:38,  5.51s/it]
 79%|███████▊  | 177001/225000 [337:07:50<66:54:33,  5.02s/it]
 79%|███████▊  | 177002/225000 [337:07:57<75:51:54,  5.69s/it]
 79%|███████▊  | 177003/225000 [337:08:03<76:36:15,  5.75s/it]
 79%|███████▊  | 177004/225000 [337:08:09<76:15:56,  5.72s/it]
 79%|███████▊  | 177005/225000 [337:08:15<79:18:53,  5.95s/it]
 79%|███████▊  | 177006/225000 [337:08:20<75:05:08,  5.63s/it]
 79%|███████▊  | 177007/225000 [337:08:25<71:36:49,  5.37s/it]
 79%|███████▊  | 177008/225000 [337:08:30<72:20:18,  5.43s/it]
 79%|███████▊  | 177009/225000 [337:08:37<78:06:42,  5.86s/it]
 79%|███████▊  | 177010/225000 [337:08:42<73:49:17,  5.54s/it]
 79%|███████▊  | 177011/225000 [337:08:47<69:35:52,  5.22s/it]
 79%|███████▊  | 177012/225000 [337:08:51<66:37:37,  5.00s/it]
 79%|███████▊  | 177013/225000 [337:08:55<64:03:39,  4.81s/it]
 79%|███████▊  | 177014/225000 [337:09:06<87:38:15,  6.57s/it]
 79%|███████▊  | 177015/225000 [337:09:13<87:59:37,  6.60s/it]
 79%|███████▊  | 177016/225000 [337:09:16<75:21:17,  5.65s/it]
 79%|███████▊  | 177017/225000 [337:09:20<67:04:54,  5.03s/it]
 79%|███████▊  | 177018/225000 [337:09:25<67:56:40,  5.10s/it]
 79%|███████▊  | 177019/225000 [337:09:32<75:16:03,  5.65s/it]
 79%|███████▊  | 177020/225000 [337:09:39<81:26:40,  6.11s/it]
 79%|███████▊  | 177021/225000 [337:09:43<72:42:38,  5.46s/it]
 79%|███████▊  | 177022/225000 [337:09:51<81:13:02,  6.09s/it]
 79%|███████▊  | 177023/225000 [337:09:54<70:16:58,  5.27s/it]
 79%|███████▊  | 177024/225000 [337:10:00<72:56:43,  5.47s/it]
 79%|███████▊  | 177025/225000 [337:10:06<73:25:01,  5.51s/it]
 79%|███████▊  | 177026/225000 [337:10:09<64:59:55,  4.88s/it]
 79%|███████▊  | 177027/225000 [337:10:18<81:15:46,  6.10s/it]
 79%|███████▊  | 177028/225000 [337:10:22<72:02:59,  5.41s/it]
 79%|███████▊  | 177029/225000 [337:10:26<69:23:30,  5.21s/it]
 79%|███████▊  | 177030/225000 [337:10:35<83:18:05,  6.25s/it]
 79%|███████▊  | 177031/225000 [337:10:39<72:41:39,  5.46s/it]
 79%|███████▊  | 177032/225000 [337:10:42<65:01:08,  4.88s/it]
 79%|███████▊  | 177033/225000 [337:10:51<82:04:17,  6.16s/it]
 79%|███████▊  | 177034/225000 [337:10:55<71:08:59,  5.34s/it]
 79%|███████▊  | 177035/225000 [337:11:02<78:01:07,  5.86s/it]
 79%|███████▊  | 177036/225000 [337:11:06<68:59:42,  5.18s/it]
 79%|███████▊  | 177037/225000 [337:11:10<68:00:18,  5.10s/it]
 79%|███████▊  | 177038/225000 [337:11:20<83:47:14,  6.29s/it]
 79%|███████▊  | 177039/225000 [337:11:23<72:27:32,  5.44s/it]
 79%|███████▊  | 177040/225000 [337:11:27<67:46:31,  5.09s/it]
 79%|███████▊  | 177041/225000 [337:11:35<78:40:05,  5.91s/it]
 79%|███████▊  | 177042/225000 [337:11:39<70:11:43,  5.27s/it]
 79%|███████▊  | 177043/225000 [337:11:48<85:48:41,  6.44s/it]
 79%|███████▊  | 177044/225000 [337:11:52<74:06:06,  5.56s/it]
 79%|███████▊  | 177045/225000 [337:11:55<63:55:19,  4.80s/it]
 79%|███████▊  | 177046/225000 [337:12:03<77:14:36,  5.80s/it]
 79%|███████▊  | 177047/225000 [337:12:06<68:25:58,  5.14s/it]
 79%|███████▊  | 177048/225000 [337:12:15<84:31:42,  6.35s/it]
 79%|███████▊  | 177049/225000 [337:12:19<72:27:40,  5.44s/it]
 79%|███████▊  | 177050/225000 [337:12:23<68:15:15,  5.12s/it]
                                                              
{'loss': '0.7656', 'grad_norm': '1.077', 'learning_rate': '5.909e-06', 'epoch': '0.7869', 'num_input_tokens_seen': 558884096, 'train_runtime': '1.214e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 177050/225000 [337:12:23<68:15:15,  5.12s/it]
 79%|███████▊  | 177051/225000 [337:12:31<79:31:59,  5.97s/it]
 79%|███████▊  | 177052/225000 [337:12:35<70:30:41,  5.29s/it]
 79%|███████▊  | 177053/225000 [337:12:38<62:56:25,  4.73s/it]
 79%|███████▊  | 177054/225000 [337:12:48<82:09:51,  6.17s/it]
 79%|███████▊  | 177055/225000 [337:12:51<72:25:38,  5.44s/it]
 79%|███████▊  | 177056/225000 [337:12:55<66:40:53,  5.01s/it]
 79%|███████▊  | 177057/225000 [337:13:04<79:17:34,  5.95s/it]
 79%|███████▊  | 177058/225000 [337:13:09<76:11:13,  5.72s/it]
 79%|███████▊  | 177059/225000 [337:13:16<82:47:14,  6.22s/it]
 79%|███████▊  | 177060/225000 [337:13:20<72:54:18,  5.47s/it]
 79%|███████▊  | 177061/225000 [337:13:23<65:03:55,  4.89s/it]
 79%|███████▊  | 177062/225000 [337:13:28<62:24:31,  4.69s/it]
 79%|███████▊  | 177063/225000 [337:13:33<66:08:48,  4.97s/it]
 79%|███████▊  | 177064/225000 [337:13:39<68:35:11,  5.15s/it]
 79%|███████▊  | 177065/225000 [337:13:44<70:03:16,  5.26s/it]
 79%|███████▊  | 177066/225000 [337:13:50<70:47:20,  5.32s/it]
 79%|███████▊  | 177067/225000 [337:13:55<71:54:12,  5.40s/it]
 79%|███████▊  | 177068/225000 [337:14:00<69:59:49,  5.26s/it]
 79%|███████▊  | 177069/225000 [337:14:05<68:09:03,  5.12s/it]
 79%|███████▊  | 177070/225000 [337:14:10<69:01:56,  5.18s/it]
 79%|███████▊  | 177071/225000 [337:14:16<69:59:49,  5.26s/it]
 79%|███████▊  | 177072/225000 [337:14:21<70:03:24,  5.26s/it]
 79%|███████▊  | 177073/225000 [337:14:26<69:14:54,  5.20s/it]
 79%|███████▊  | 177074/225000 [337:14:35<84:09:52,  6.32s/it]
 79%|███████▊  | 177075/225000 [337:14:40<77:03:42,  5.79s/it]
 79%|███████▊  | 177076/225000 [337:14:44<71:55:57,  5.40s/it]
 79%|███████▊  | 177077/225000 [337:14:49<69:17:34,  5.21s/it]
 79%|███████▊  | 177078/225000 [337:14:53<66:17:22,  4.98s/it]
 79%|███████▊  | 177079/225000 [337:14:58<64:25:34,  4.84s/it]
 79%|███████▊  | 177080/225000 [337:15:03<65:54:52,  4.95s/it]
 79%|███████▊  | 177081/225000 [337:15:15<92:24:51,  6.94s/it]
 79%|███████▊  | 177082/225000 [337:15:20<86:26:54,  6.49s/it]
 79%|███████▊  | 177083/225000 [337:15:26<82:19:17,  6.18s/it]
 79%|███████▊  | 177084/225000 [337:15:31<80:07:35,  6.02s/it]
 79%|███████▊  | 177085/225000 [337:15:37<78:05:10,  5.87s/it]
 79%|███████▊  | 177086/225000 [337:15:43<78:17:17,  5.88s/it]
 79%|███████▊  | 177087/225000 [337:15:48<77:43:19,  5.84s/it]
 79%|███████▊  | 177088/225000 [337:15:52<68:14:50,  5.13s/it]
 79%|███████▊  | 177089/225000 [337:15:59<77:53:34,  5.85s/it]
 79%|███████▊  | 177090/225000 [337:16:03<68:30:40,  5.15s/it]
 79%|███████▊  | 177091/225000 [337:16:10<76:44:52,  5.77s/it]
 79%|███████▊  | 177092/225000 [337:16:14<69:19:45,  5.21s/it]
 79%|███████▊  | 177093/225000 [337:16:18<64:31:23,  4.85s/it]
 79%|███████▊  | 177094/225000 [337:16:25<74:34:40,  5.60s/it]
 79%|███████▊  | 177095/225000 [337:16:32<79:14:45,  5.96s/it]
 79%|███████▊  | 177096/225000 [337:16:39<83:19:56,  6.26s/it]
 79%|███████▊  | 177097/225000 [337:16:42<70:41:59,  5.31s/it]
 79%|███████▊  | 177098/225000 [337:16:46<63:29:47,  4.77s/it]
 79%|███████▊  | 177099/225000 [337:16:53<73:15:36,  5.51s/it]
 79%|███████▊  | 177100/225000 [337:16:59<73:59:07,  5.56s/it]
                                                              
{'loss': '0.7547', 'grad_norm': '1.064', 'learning_rate': '5.897e-06', 'epoch': '0.7871', 'num_input_tokens_seen': 559014152, 'train_runtime': '1.214e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 177100/225000 [337:16:59<73:59:07,  5.56s/it]
 79%|███████▊  | 177101/225000 [337:17:02<65:57:57,  4.96s/it]
 79%|███████▊  | 177102/225000 [337:17:09<74:27:30,  5.60s/it]
 79%|███████▊  | 177103/225000 [337:17:16<77:29:33,  5.82s/it]
 79%|███████▊  | 177104/225000 [337:17:21<76:35:19,  5.76s/it]
 79%|███████▊  | 177105/225000 [337:17:27<74:27:11,  5.60s/it]
 79%|███████▊  | 177106/225000 [337:17:30<66:11:17,  4.98s/it]
 79%|███████▊  | 177107/225000 [337:17:33<59:36:54,  4.48s/it]
 79%|███████▊  | 177108/225000 [337:17:37<56:18:46,  4.23s/it]
 79%|███████▊  | 177109/225000 [337:17:45<70:47:14,  5.32s/it]
 79%|███████▊  | 177110/225000 [337:17:51<75:27:15,  5.67s/it]
 79%|███████▊  | 177111/225000 [337:17:55<67:59:45,  5.11s/it]
 79%|███████▊  | 177112/225000 [337:17:59<63:39:35,  4.79s/it]
 79%|███████▊  | 177113/225000 [337:18:05<66:33:26,  5.00s/it]
 79%|███████▊  | 177114/225000 [337:18:10<66:44:53,  5.02s/it]
 79%|███████▊  | 177115/225000 [337:18:15<66:06:00,  4.97s/it]
 79%|███████▊  | 177116/225000 [337:18:20<66:35:39,  5.01s/it]
 79%|███████▊  | 177117/225000 [337:18:25<66:41:07,  5.01s/it]
 79%|███████▊  | 177118/225000 [337:18:30<67:50:25,  5.10s/it]
 79%|███████▊  | 177119/225000 [337:18:36<70:30:28,  5.30s/it]
 79%|███████▊  | 177120/225000 [337:18:41<70:37:55,  5.31s/it]
 79%|███████▊  | 177121/225000 [337:18:46<66:53:58,  5.03s/it]
 79%|███████▊  | 177122/225000 [337:18:51<67:37:57,  5.09s/it]
 79%|███████▊  | 177123/225000 [337:18:56<68:10:29,  5.13s/it]
 79%|███████▊  | 177124/225000 [337:19:01<66:40:39,  5.01s/it]
 79%|███████▊  | 177125/225000 [337:19:07<69:36:27,  5.23s/it]
 79%|███████▊  | 177126/225000 [337:19:12<70:54:41,  5.33s/it]
 79%|███████▊  | 177127/225000 [337:19:17<68:39:06,  5.16s/it]
 79%|███████▊  | 177128/225000 [337:19:22<69:24:39,  5.22s/it]
 79%|███████▊  | 177129/225000 [337:19:34<96:47:04,  7.28s/it]
 79%|███████▊  | 177130/225000 [337:19:40<89:41:18,  6.74s/it]
 79%|███████▊  | 177131/225000 [337:19:44<80:10:49,  6.03s/it]
 79%|███████▊  | 177132/225000 [337:19:49<75:22:57,  5.67s/it]
 79%|███████▊  | 177133/225000 [337:19:53<70:36:00,  5.31s/it]
 79%|███████▊  | 177134/225000 [337:19:58<69:08:44,  5.20s/it]
 79%|███████▊  | 177135/225000 [337:20:03<67:52:20,  5.10s/it]
 79%|███████▊  | 177136/225000 [337:20:08<66:53:02,  5.03s/it]
 79%|███████▊  | 177137/225000 [337:20:13<66:06:41,  4.97s/it]
 79%|███████▊  | 177138/225000 [337:20:18<65:42:53,  4.94s/it]
 79%|███████▊  | 177139/225000 [337:20:22<63:52:06,  4.80s/it]
 79%|███████▊  | 177140/225000 [337:20:29<70:10:57,  5.28s/it]
 79%|███████▊  | 177141/225000 [337:20:41<98:10:14,  7.38s/it]
 79%|███████▊  | 177142/225000 [337:20:46<86:53:12,  6.54s/it]
 79%|███████▊  | 177143/225000 [337:20:50<78:30:45,  5.91s/it]
 79%|███████▊  | 177144/225000 [337:20:54<72:28:29,  5.45s/it]
 79%|███████▊  | 177145/225000 [337:20:59<67:44:16,  5.10s/it]
 79%|███████▊  | 177146/225000 [337:21:03<64:16:26,  4.84s/it]
 79%|███████▊  | 177147/225000 [337:21:07<63:24:36,  4.77s/it]
 79%|███████▊  | 177148/225000 [337:21:16<78:08:57,  5.88s/it]
 79%|███████▊  | 177149/225000 [337:21:25<89:20:10,  6.72s/it]
 79%|███████▊  | 177150/225000 [337:21:28<75:37:20,  5.69s/it]
                                                              
{'loss': '0.7641', 'grad_norm': '1.06', 'learning_rate': '5.885e-06', 'epoch': '0.7873', 'num_input_tokens_seen': 559143352, 'train_runtime': '1.214e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▊  | 177150/225000 [337:21:28<75:37:20,  5.69s/it]
 79%|███████▊  | 177151/225000 [337:21:36<83:20:01,  6.27s/it]
 79%|███████▊  | 177152/225000 [337:21:39<73:46:59,  5.55s/it]
 79%|███████▊  | 177153/225000 [337:21:44<69:21:45,  5.22s/it]
 79%|███████▊  | 177154/225000 [337:21:48<64:21:08,  4.84s/it]
 79%|███████▊  | 177155/225000 [337:21:52<60:28:11,  4.55s/it]
 79%|███████▊  | 177156/225000 [337:22:02<85:07:53,  6.41s/it]
 79%|███████▊  | 177157/225000 [337:22:06<73:25:43,  5.53s/it]
 79%|███████▊  | 177158/225000 [337:22:10<67:59:00,  5.12s/it]
 79%|███████▊  | 177159/225000 [337:22:15<66:18:50,  4.99s/it]
 79%|███████▊  | 177160/225000 [337:22:20<66:12:30,  4.98s/it]
 79%|███████▊  | 177161/225000 [337:22:25<68:52:21,  5.18s/it]
 79%|███████▊  | 177162/225000 [337:22:30<68:31:01,  5.16s/it]
 79%|███████▊  | 177163/225000 [337:22:36<69:12:02,  5.21s/it]
 79%|███████▊  | 177164/225000 [337:22:41<67:23:34,  5.07s/it]
 79%|███████▊  | 177165/225000 [337:22:45<66:01:37,  4.97s/it]
 79%|███████▊  | 177166/225000 [337:22:50<66:22:30,  5.00s/it]
 79%|███████▊  | 177167/225000 [337:22:55<67:01:43,  5.04s/it]
 79%|███████▊  | 177168/225000 [337:23:01<67:45:36,  5.10s/it]
 79%|███████▊  | 177169/225000 [337:23:05<66:23:39,  5.00s/it]
 79%|███████▊  | 177170/225000 [337:23:10<66:22:04,  5.00s/it]
 79%|███████▊  | 177171/225000 [337:23:15<66:15:56,  4.99s/it]
 79%|███████▊  | 177172/225000 [337:23:20<66:05:33,  4.97s/it]
 79%|███████▊  | 177173/225000 [337:23:26<67:50:57,  5.11s/it]
 79%|███████▊  | 177174/225000 [337:23:31<67:19:52,  5.07s/it]
 79%|███████▊  | 177175/225000 [337:23:36<66:37:05,  5.01s/it]
 79%|███████▊  | 177176/225000 [337:23:40<65:21:51,  4.92s/it]
 79%|███████▊  | 177177/225000 [337:23:45<62:31:55,  4.71s/it]
 79%|███████▊  | 177178/225000 [337:23:50<64:01:46,  4.82s/it]
 79%|███████▊  | 177179/225000 [337:23:56<71:55:45,  5.41s/it]
 79%|███████▊  | 177180/225000 [337:24:09<99:20:21,  7.48s/it]
 79%|███████▊  | 177181/225000 [337:24:13<87:29:10,  6.59s/it]
 79%|███████▊  | 177182/225000 [337:24:19<84:26:38,  6.36s/it]
 79%|███████▊  | 177183/225000 [337:24:25<81:52:45,  6.16s/it]
 79%|███████▊  | 177184/225000 [337:24:30<77:14:42,  5.82s/it]
 79%|███████▊  | 177185/225000 [337:24:34<72:27:25,  5.46s/it]
 79%|███████▊  | 177186/225000 [337:24:39<69:34:23,  5.24s/it]
 79%|███████▊  | 177187/225000 [337:24:44<66:46:55,  5.03s/it]
 79%|███████▉  | 177188/225000 [337:24:49<68:14:56,  5.14s/it]
 79%|███████▉  | 177189/225000 [337:24:54<68:26:47,  5.15s/it]
 79%|███████▉  | 177190/225000 [337:24:59<66:32:51,  5.01s/it]
 79%|███████▉  | 177191/225000 [337:25:04<67:01:37,  5.05s/it]
 79%|███████▉  | 177192/225000 [337:25:09<66:05:40,  4.98s/it]
 79%|███████▉  | 177193/225000 [337:25:14<64:41:25,  4.87s/it]
 79%|███████▉  | 177194/225000 [337:25:18<62:39:34,  4.72s/it]
 79%|███████▉  | 177195/225000 [337:25:23<63:01:48,  4.75s/it]
 79%|███████▉  | 177196/225000 [337:25:33<84:22:27,  6.35s/it]
 79%|███████▉  | 177197/225000 [337:25:40<86:39:59,  6.53s/it]
 79%|███████▉  | 177198/225000 [337:25:44<76:42:53,  5.78s/it]
 79%|███████▉  | 177199/225000 [337:25:48<69:29:33,  5.23s/it]
 79%|███████▉  | 177200/225000 [337:25:52<66:12:55,  4.99s/it]
                                                              
{'loss': '0.7781', 'grad_norm': '1.147', 'learning_rate': '5.874e-06', 'epoch': '0.7876', 'num_input_tokens_seen': 559271184, 'train_runtime': '1.215e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177200/225000 [337:25:52<66:12:55,  4.99s/it]
 79%|███████▉  | 177201/225000 [337:25:56<63:34:02,  4.79s/it]
 79%|███████▉  | 177202/225000 [337:26:06<81:53:07,  6.17s/it]
 79%|███████▉  | 177203/225000 [337:26:14<87:58:30,  6.63s/it]
 79%|███████▉  | 177204/225000 [337:26:17<76:27:15,  5.76s/it]
 79%|███████▉  | 177205/225000 [337:26:26<87:05:33,  6.56s/it]
 79%|███████▉  | 177206/225000 [337:26:30<77:42:13,  5.85s/it]
 79%|███████▉  | 177207/225000 [337:26:34<70:10:45,  5.29s/it]
 79%|███████▉  | 177208/225000 [337:26:41<79:07:42,  5.96s/it]
 79%|███████▉  | 177209/225000 [337:26:47<78:03:20,  5.88s/it]
 79%|███████▉  | 177210/225000 [337:26:54<83:58:25,  6.33s/it]
 79%|███████▉  | 177211/225000 [337:26:58<71:49:30,  5.41s/it]
 79%|███████▉  | 177212/225000 [337:27:01<63:49:18,  4.81s/it]
 79%|███████▉  | 177213/225000 [337:27:10<78:12:38,  5.89s/it]
 79%|███████▉  | 177214/225000 [337:27:13<70:17:01,  5.29s/it]
 79%|███████▉  | 177215/225000 [337:27:17<64:16:00,  4.84s/it]
 79%|███████▉  | 177216/225000 [337:27:25<76:12:36,  5.74s/it]
 79%|███████▉  | 177217/225000 [337:27:31<75:58:45,  5.72s/it]
 79%|███████▉  | 177218/225000 [337:27:35<70:18:35,  5.30s/it]
 79%|███████▉  | 177219/225000 [337:27:40<69:00:48,  5.20s/it]
 79%|███████▉  | 177220/225000 [337:27:45<66:22:44,  5.00s/it]
 79%|███████▉  | 177221/225000 [337:27:49<64:54:53,  4.89s/it]
 79%|███████▉  | 177222/225000 [337:27:54<66:01:45,  4.98s/it]
 79%|███████▉  | 177223/225000 [337:27:59<64:15:16,  4.84s/it]
 79%|███████▉  | 177224/225000 [337:28:10<88:00:04,  6.63s/it]
 79%|███████▉  | 177225/225000 [337:28:14<76:38:00,  5.77s/it]
 79%|███████▉  | 177226/225000 [337:28:18<69:36:00,  5.24s/it]
 79%|███████▉  | 177227/225000 [337:28:26<84:02:09,  6.33s/it]
 79%|███████▉  | 177228/225000 [337:28:30<72:59:47,  5.50s/it]
 79%|███████▉  | 177229/225000 [337:28:34<65:35:12,  4.94s/it]
 79%|███████▉  | 177230/225000 [337:28:43<84:58:33,  6.40s/it]
 79%|███████▉  | 177231/225000 [337:28:47<73:40:56,  5.55s/it]
 79%|███████▉  | 177232/225000 [337:28:51<65:40:39,  4.95s/it]
 79%|███████▉  | 177233/225000 [337:29:00<84:20:37,  6.36s/it]
 79%|███████▉  | 177234/225000 [337:29:03<72:10:23,  5.44s/it]
 79%|███████▉  | 177235/225000 [337:29:10<77:53:12,  5.87s/it]
 79%|███████▉  | 177236/225000 [337:29:14<71:02:38,  5.35s/it]
 79%|███████▉  | 177237/225000 [337:29:18<65:22:48,  4.93s/it]
 79%|███████▉  | 177238/225000 [337:29:23<63:08:54,  4.76s/it]
 79%|███████▉  | 177239/225000 [337:29:27<61:15:28,  4.62s/it]
 79%|███████▉  | 177240/225000 [337:29:31<59:29:44,  4.48s/it]
 79%|███████▉  | 177241/225000 [337:29:36<59:10:55,  4.46s/it]
 79%|███████▉  | 177242/225000 [337:29:41<62:11:27,  4.69s/it]
 79%|███████▉  | 177243/225000 [337:29:46<64:05:09,  4.83s/it]
 79%|███████▉  | 177244/225000 [337:29:51<65:27:19,  4.93s/it]
 79%|███████▉  | 177245/225000 [337:30:04<98:25:01,  7.42s/it]
 79%|███████▉  | 177246/225000 [337:30:09<86:56:30,  6.55s/it]
 79%|███████▉  | 177247/225000 [337:30:16<90:44:08,  6.84s/it]
 79%|███████▉  | 177248/225000 [337:30:20<78:33:39,  5.92s/it]
 79%|███████▉  | 177249/225000 [337:30:24<69:51:10,  5.27s/it]
 79%|███████▉  | 177250/225000 [337:30:34<87:33:14,  6.60s/it]
                                                              
{'loss': '0.767', 'grad_norm': '1.195', 'learning_rate': '5.862e-06', 'epoch': '0.7878', 'num_input_tokens_seen': 559401592, 'train_runtime': '1.215e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177250/225000 [337:30:34<87:33:14,  6.60s/it]
 79%|███████▉  | 177251/225000 [337:30:38<78:35:22,  5.93s/it]
 79%|███████▉  | 177252/225000 [337:30:42<69:08:20,  5.21s/it]
 79%|███████▉  | 177253/225000 [337:30:48<74:52:16,  5.65s/it]
 79%|███████▉  | 177254/225000 [337:30:54<76:30:21,  5.77s/it]
 79%|███████▉  | 177255/225000 [337:30:58<68:18:31,  5.15s/it]
 79%|███████▉  | 177256/225000 [337:31:08<85:58:24,  6.48s/it]
 79%|███████▉  | 177257/225000 [337:31:12<76:37:46,  5.78s/it]
 79%|███████▉  | 177258/225000 [337:31:16<69:42:36,  5.26s/it]
 79%|███████▉  | 177259/225000 [337:31:20<65:57:08,  4.97s/it]
 79%|███████▉  | 177260/225000 [337:31:25<66:32:35,  5.02s/it]
 79%|███████▉  | 177261/225000 [337:31:30<66:30:51,  5.02s/it]
 79%|███████▉  | 177262/225000 [337:31:35<66:25:14,  5.01s/it]
 79%|███████▉  | 177263/225000 [337:31:39<63:27:48,  4.79s/it]
 79%|███████▉  | 177264/225000 [337:31:44<64:10:27,  4.84s/it]
 79%|███████▉  | 177265/225000 [337:31:52<73:30:57,  5.54s/it]
 79%|███████▉  | 177266/225000 [337:32:03<95:46:35,  7.22s/it]
 79%|███████▉  | 177267/225000 [337:32:06<79:06:56,  5.97s/it]
 79%|███████▉  | 177268/225000 [337:32:14<86:54:22,  6.55s/it]
 79%|███████▉  | 177269/225000 [337:32:18<77:39:02,  5.86s/it]
 79%|███████▉  | 177270/225000 [337:32:22<71:42:27,  5.41s/it]
 79%|███████▉  | 177271/225000 [337:32:27<66:53:52,  5.05s/it]
 79%|███████▉  | 177272/225000 [337:32:32<68:34:26,  5.17s/it]
 79%|███████▉  | 177273/225000 [337:32:40<80:31:41,  6.07s/it]
 79%|███████▉  | 177274/225000 [337:32:44<73:19:03,  5.53s/it]
 79%|███████▉  | 177275/225000 [337:32:48<66:13:40,  5.00s/it]
 79%|███████▉  | 177276/225000 [337:32:55<75:25:00,  5.69s/it]
 79%|███████▉  | 177277/225000 [337:33:02<76:54:35,  5.80s/it]
 79%|███████▉  | 177278/225000 [337:33:05<68:09:08,  5.14s/it]
 79%|███████▉  | 177279/225000 [337:33:13<80:01:56,  6.04s/it]
 79%|███████▉  | 177280/225000 [337:33:19<80:17:30,  6.06s/it]
 79%|███████▉  | 177281/225000 [337:33:23<71:40:31,  5.41s/it]
 79%|███████▉  | 177282/225000 [337:33:32<83:02:19,  6.26s/it]
 79%|███████▉  | 177283/225000 [337:33:35<72:37:18,  5.48s/it]
 79%|███████▉  | 177284/225000 [337:33:39<65:38:14,  4.95s/it]
 79%|███████▉  | 177285/225000 [337:33:42<59:17:53,  4.47s/it]
 79%|███████▉  | 177286/225000 [337:33:51<77:52:16,  5.88s/it]
 79%|███████▉  | 177287/225000 [337:33:55<69:30:26,  5.24s/it]
 79%|███████▉  | 177288/225000 [337:33:59<65:17:46,  4.93s/it]
 79%|███████▉  | 177289/225000 [337:34:03<60:20:53,  4.55s/it]
 79%|███████▉  | 177290/225000 [337:34:13<82:51:09,  6.25s/it]
 79%|███████▉  | 177291/225000 [337:34:17<71:47:43,  5.42s/it]
 79%|███████▉  | 177292/225000 [337:34:24<80:23:37,  6.07s/it]
 79%|███████▉  | 177293/225000 [337:34:28<71:53:03,  5.42s/it]
 79%|███████▉  | 177294/225000 [337:34:32<66:02:05,  4.98s/it]
 79%|███████▉  | 177295/225000 [337:34:36<61:46:24,  4.66s/it]
 79%|███████▉  | 177296/225000 [337:34:46<83:58:28,  6.34s/it]
 79%|███████▉  | 177297/225000 [337:34:51<75:23:36,  5.69s/it]
 79%|███████▉  | 177298/225000 [337:34:54<68:21:51,  5.16s/it]
 79%|███████▉  | 177299/225000 [337:34:59<65:22:18,  4.93s/it]
 79%|███████▉  | 177300/225000 [337:35:08<81:40:32,  6.16s/it]
                                                              
{'loss': '0.764', 'grad_norm': '1.115', 'learning_rate': '5.85e-06', 'epoch': '0.788', 'num_input_tokens_seen': 559532352, 'train_runtime': '1.215e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177300/225000 [337:35:08<81:40:32,  6.16s/it]
 79%|███████▉  | 177301/225000 [337:35:11<70:59:55,  5.36s/it]
 79%|███████▉  | 177302/225000 [337:35:18<77:32:18,  5.85s/it]
 79%|███████▉  | 177303/225000 [337:35:25<79:09:56,  5.98s/it]
 79%|███████▉  | 177304/225000 [337:35:29<71:40:14,  5.41s/it]
 79%|███████▉  | 177305/225000 [337:35:33<67:55:34,  5.13s/it]
 79%|███████▉  | 177306/225000 [337:35:37<63:58:54,  4.83s/it]
 79%|███████▉  | 177307/225000 [337:35:42<63:26:36,  4.79s/it]
 79%|███████▉  | 177308/225000 [337:35:53<86:34:03,  6.53s/it]
 79%|███████▉  | 177309/225000 [337:35:56<74:47:03,  5.65s/it]
 79%|███████▉  | 177310/225000 [337:36:00<66:22:30,  5.01s/it]
 79%|███████▉  | 177311/225000 [337:36:11<89:30:36,  6.76s/it]
 79%|███████▉  | 177312/225000 [337:36:14<74:57:57,  5.66s/it]
 79%|███████▉  | 177313/225000 [337:36:18<71:40:37,  5.41s/it]
 79%|███████▉  | 177314/225000 [337:36:23<68:29:55,  5.17s/it]
 79%|███████▉  | 177315/225000 [337:36:28<69:22:31,  5.24s/it]
 79%|███████▉  | 177316/225000 [337:36:33<67:37:48,  5.11s/it]
 79%|███████▉  | 177317/225000 [337:36:39<69:07:52,  5.22s/it]
 79%|███████▉  | 177318/225000 [337:36:44<70:08:21,  5.30s/it]
 79%|███████▉  | 177319/225000 [337:36:49<69:21:31,  5.24s/it]
 79%|███████▉  | 177320/225000 [337:36:54<68:24:55,  5.17s/it]
 79%|███████▉  | 177321/225000 [337:37:00<69:11:40,  5.22s/it]
 79%|███████▉  | 177322/225000 [337:37:05<67:56:24,  5.13s/it]
 79%|███████▉  | 177323/225000 [337:37:10<70:26:17,  5.32s/it]
 79%|███████▉  | 177324/225000 [337:37:16<71:27:04,  5.40s/it]
 79%|███████▉  | 177325/225000 [337:37:21<70:15:39,  5.31s/it]
 79%|███████▉  | 177326/225000 [337:37:26<70:06:19,  5.29s/it]
 79%|███████▉  | 177327/225000 [337:37:31<68:52:49,  5.20s/it]
 79%|███████▉  | 177328/225000 [337:37:36<67:03:44,  5.06s/it]
 79%|███████▉  | 177329/225000 [337:37:41<67:31:38,  5.10s/it]
 79%|███████▉  | 177330/225000 [337:37:46<67:53:13,  5.13s/it]
 79%|███████▉  | 177331/225000 [337:37:52<69:10:33,  5.22s/it]
 79%|███████▉  | 177332/225000 [337:37:57<68:53:18,  5.20s/it]
 79%|███████▉  | 177333/225000 [337:38:02<69:28:33,  5.25s/it]
 79%|███████▉  | 177334/225000 [337:38:07<68:22:58,  5.16s/it]
 79%|███████▉  | 177335/225000 [337:38:13<69:02:16,  5.21s/it]
 79%|███████▉  | 177336/225000 [337:38:24<94:33:33,  7.14s/it]
 79%|███████▉  | 177337/225000 [337:38:29<85:17:04,  6.44s/it]
 79%|███████▉  | 177338/225000 [337:38:35<81:29:58,  6.16s/it]
 79%|███████▉  | 177339/225000 [337:38:40<78:41:58,  5.94s/it]
 79%|███████▉  | 177340/225000 [337:38:45<76:16:19,  5.76s/it]
 79%|███████▉  | 177341/225000 [337:38:51<77:29:23,  5.85s/it]
 79%|███████▉  | 177342/225000 [337:38:59<84:33:59,  6.39s/it]
 79%|███████▉  | 177343/225000 [337:39:03<74:22:15,  5.62s/it]
 79%|███████▉  | 177344/225000 [337:39:08<71:18:53,  5.39s/it]
 79%|███████▉  | 177345/225000 [337:39:13<70:37:58,  5.34s/it]
 79%|███████▉  | 177346/225000 [337:39:18<70:45:55,  5.35s/it]
 79%|███████▉  | 177347/225000 [337:39:23<68:22:28,  5.17s/it]
 79%|███████▉  | 177348/225000 [337:39:28<66:56:16,  5.06s/it]
 79%|███████▉  | 177349/225000 [337:39:33<67:55:54,  5.13s/it]
 79%|███████▉  | 177350/225000 [337:39:38<68:27:20,  5.17s/it]
                                                              
{'loss': '0.7592', 'grad_norm': '1.065', 'learning_rate': '5.838e-06', 'epoch': '0.7882', 'num_input_tokens_seen': 559662240, 'train_runtime': '1.216e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177350/225000 [337:39:38<68:27:20,  5.17s/it]
 79%|███████▉  | 177351/225000 [337:39:44<68:33:53,  5.18s/it]
 79%|███████▉  | 177352/225000 [337:39:49<68:26:46,  5.17s/it]
 79%|███████▉  | 177353/225000 [337:39:54<67:53:10,  5.13s/it]
 79%|███████▉  | 177354/225000 [337:39:59<68:25:05,  5.17s/it]
 79%|███████▉  | 177355/225000 [337:40:05<69:45:49,  5.27s/it]
 79%|███████▉  | 177356/225000 [337:40:10<69:41:44,  5.27s/it]
 79%|███████▉  | 177357/225000 [337:40:15<68:23:30,  5.17s/it]
 79%|███████▉  | 177358/225000 [337:40:20<68:09:19,  5.15s/it]
 79%|███████▉  | 177359/225000 [337:40:25<69:12:48,  5.23s/it]
 79%|███████▉  | 177360/225000 [337:40:31<69:52:33,  5.28s/it]
 79%|███████▉  | 177361/225000 [337:40:36<68:19:58,  5.16s/it]
 79%|███████▉  | 177362/225000 [337:40:41<69:17:07,  5.24s/it]
 79%|███████▉  | 177363/225000 [337:40:46<69:14:23,  5.23s/it]
 79%|███████▉  | 177364/225000 [337:40:51<68:10:43,  5.15s/it]
 79%|███████▉  | 177365/225000 [337:40:56<68:07:27,  5.15s/it]
 79%|███████▉  | 177366/225000 [337:41:02<68:12:38,  5.16s/it]
 79%|███████▉  | 177367/225000 [337:41:07<69:48:23,  5.28s/it]
 79%|███████▉  | 177368/225000 [337:41:12<69:05:45,  5.22s/it]
 79%|███████▉  | 177369/225000 [337:41:18<73:14:05,  5.54s/it]
 79%|███████▉  | 177370/225000 [337:41:31<100:09:34,  7.57s/it]
 79%|███████▉  | 177371/225000 [337:41:34<83:55:46,  6.34s/it] 
 79%|███████▉  | 177372/225000 [337:41:38<74:41:35,  5.65s/it]
 79%|███████▉  | 177373/225000 [337:41:43<69:43:17,  5.27s/it]
 79%|███████▉  | 177374/225000 [337:41:50<76:13:09,  5.76s/it]
 79%|███████▉  | 177375/225000 [337:41:58<85:06:52,  6.43s/it]
 79%|███████▉  | 177376/225000 [337:42:01<74:53:36,  5.66s/it]
 79%|███████▉  | 177377/225000 [337:42:08<76:55:02,  5.81s/it]
 79%|███████▉  | 177378/225000 [337:42:12<71:49:12,  5.43s/it]
 79%|███████▉  | 177379/225000 [337:42:16<66:59:39,  5.06s/it]
 79%|███████▉  | 177380/225000 [337:42:21<65:26:26,  4.95s/it]
 79%|███████▉  | 177381/225000 [337:42:26<64:06:26,  4.85s/it]
 79%|███████▉  | 177382/225000 [337:42:30<62:51:30,  4.75s/it]
 79%|███████▉  | 177383/225000 [337:42:41<87:47:58,  6.64s/it]
 79%|███████▉  | 177384/225000 [337:42:45<77:42:11,  5.87s/it]
 79%|███████▉  | 177385/225000 [337:42:49<70:52:16,  5.36s/it]
 79%|███████▉  | 177386/225000 [337:42:54<66:21:51,  5.02s/it]
 79%|███████▉  | 177387/225000 [337:43:00<71:14:24,  5.39s/it]
 79%|███████▉  | 177388/225000 [337:43:07<79:14:21,  5.99s/it]
 79%|███████▉  | 177389/225000 [337:43:11<69:53:50,  5.29s/it]
 79%|███████▉  | 177390/225000 [337:43:15<63:34:40,  4.81s/it]
 79%|███████▉  | 177391/225000 [337:43:19<62:06:12,  4.70s/it]
 79%|███████▉  | 177392/225000 [337:43:24<62:16:43,  4.71s/it]
 79%|███████▉  | 177393/225000 [337:43:29<63:24:25,  4.79s/it]
 79%|███████▉  | 177394/225000 [337:43:33<62:35:17,  4.73s/it]
 79%|███████▉  | 177395/225000 [337:43:39<64:03:01,  4.84s/it]
 79%|███████▉  | 177396/225000 [337:43:44<64:48:15,  4.90s/it]
 79%|███████▉  | 177397/225000 [337:43:49<65:04:43,  4.92s/it]
 79%|███████▉  | 177398/225000 [337:43:53<64:32:25,  4.88s/it]
 79%|███████▉  | 177399/225000 [337:43:58<65:22:52,  4.94s/it]
 79%|███████▉  | 177400/225000 [337:44:09<89:44:44,  6.79s/it]
                                                              
{'loss': '0.7544', 'grad_norm': '1.157', 'learning_rate': '5.827e-06', 'epoch': '0.7884', 'num_input_tokens_seen': 559792048, 'train_runtime': '1.216e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177400/225000 [337:44:10<89:44:44,  6.79s/it]
 79%|███████▉  | 177401/225000 [337:44:17<92:05:56,  6.97s/it]
 79%|███████▉  | 177402/225000 [337:44:21<79:44:10,  6.03s/it]
 79%|███████▉  | 177403/225000 [337:44:27<82:17:23,  6.22s/it]
 79%|███████▉  | 177404/225000 [337:44:32<74:56:01,  5.67s/it]
 79%|███████▉  | 177405/225000 [337:44:37<75:08:45,  5.68s/it]
 79%|███████▉  | 177406/225000 [337:44:44<78:54:42,  5.97s/it]
 79%|███████▉  | 177407/225000 [337:44:50<77:49:35,  5.89s/it]
 79%|███████▉  | 177408/225000 [337:44:56<77:19:23,  5.85s/it]
 79%|███████▉  | 177409/225000 [337:45:00<70:19:42,  5.32s/it]
 79%|███████▉  | 177410/225000 [337:45:04<65:09:06,  4.93s/it]
 79%|███████▉  | 177411/225000 [337:45:09<65:36:19,  4.96s/it]
 79%|███████▉  | 177412/225000 [337:45:14<65:28:35,  4.95s/it]
 79%|███████▉  | 177413/225000 [337:45:19<65:33:47,  4.96s/it]
 79%|███████▉  | 177414/225000 [337:45:24<65:57:56,  4.99s/it]
 79%|███████▉  | 177415/225000 [337:45:29<67:12:29,  5.08s/it]
 79%|███████▉  | 177416/225000 [337:45:34<68:08:12,  5.15s/it]
 79%|███████▉  | 177417/225000 [337:45:40<68:39:20,  5.19s/it]
 79%|███████▉  | 177418/225000 [337:45:44<66:52:41,  5.06s/it]
 79%|███████▉  | 177419/225000 [337:45:49<67:03:30,  5.07s/it]
 79%|███████▉  | 177420/225000 [337:45:55<68:00:30,  5.15s/it]
 79%|███████▉  | 177421/225000 [337:46:00<67:31:56,  5.11s/it]
 79%|███████▉  | 177422/225000 [337:46:05<67:53:20,  5.14s/it]
 79%|███████▉  | 177423/225000 [337:46:10<68:45:10,  5.20s/it]
 79%|███████▉  | 177424/225000 [337:46:15<68:22:40,  5.17s/it]
 79%|███████▉  | 177425/225000 [337:46:21<69:23:58,  5.25s/it]
 79%|███████▉  | 177426/225000 [337:46:26<68:53:19,  5.21s/it]
 79%|███████▉  | 177427/225000 [337:46:31<69:13:08,  5.24s/it]
 79%|███████▉  | 177428/225000 [337:46:36<68:30:08,  5.18s/it]
 79%|███████▉  | 177429/225000 [337:46:42<69:25:48,  5.25s/it]
 79%|███████▉  | 177430/225000 [337:46:47<68:51:59,  5.21s/it]
 79%|███████▉  | 177431/225000 [337:46:52<67:54:19,  5.14s/it]
 79%|███████▉  | 177432/225000 [337:46:57<68:33:09,  5.19s/it]
 79%|███████▉  | 177433/225000 [337:47:02<67:35:52,  5.12s/it]
 79%|███████▉  | 177434/225000 [337:47:14<94:54:58,  7.18s/it]
 79%|███████▉  | 177435/225000 [337:47:20<88:53:52,  6.73s/it]
 79%|███████▉  | 177436/225000 [337:47:26<88:43:42,  6.72s/it]
 79%|███████▉  | 177437/225000 [337:47:32<85:05:57,  6.44s/it]
 79%|███████▉  | 177438/225000 [337:47:36<74:12:01,  5.62s/it]
 79%|███████▉  | 177439/225000 [337:47:41<71:22:03,  5.40s/it]
 79%|███████▉  | 177440/225000 [337:47:46<70:58:57,  5.37s/it]
 79%|███████▉  | 177441/225000 [337:47:51<70:09:40,  5.31s/it]
 79%|███████▉  | 177442/225000 [337:47:57<69:58:55,  5.30s/it]
 79%|███████▉  | 177443/225000 [337:48:02<70:08:15,  5.31s/it]
 79%|███████▉  | 177444/225000 [337:48:07<69:17:49,  5.25s/it]
 79%|███████▉  | 177445/225000 [337:48:12<68:44:49,  5.20s/it]
 79%|███████▉  | 177446/225000 [337:48:18<69:34:25,  5.27s/it]
 79%|███████▉  | 177447/225000 [337:48:23<70:31:12,  5.34s/it]
 79%|███████▉  | 177448/225000 [337:48:34<93:04:20,  7.05s/it]
 79%|███████▉  | 177449/225000 [337:48:38<81:17:55,  6.15s/it]
 79%|███████▉  | 177450/225000 [337:48:42<72:54:07,  5.52s/it]
                                                              
{'loss': '0.7633', 'grad_norm': '1.106', 'learning_rate': '5.815e-06', 'epoch': '0.7887', 'num_input_tokens_seen': 559921464, 'train_runtime': '1.216e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177450/225000 [337:48:42<72:54:07,  5.52s/it]
 79%|███████▉  | 177451/225000 [337:48:47<68:38:18,  5.20s/it]
 79%|███████▉  | 177452/225000 [337:48:51<65:34:27,  4.96s/it]
 79%|███████▉  | 177453/225000 [337:48:55<62:21:00,  4.72s/it]
 79%|███████▉  | 177454/225000 [337:49:06<86:07:32,  6.52s/it]
 79%|███████▉  | 177455/225000 [337:49:10<76:47:51,  5.81s/it]
 79%|███████▉  | 177456/225000 [337:49:14<68:26:41,  5.18s/it]
 79%|███████▉  | 177457/225000 [337:49:20<70:36:52,  5.35s/it]
 79%|███████▉  | 177458/225000 [337:49:27<77:09:01,  5.84s/it]
 79%|███████▉  | 177459/225000 [337:49:31<69:44:07,  5.28s/it]
 79%|███████▉  | 177460/225000 [337:49:35<65:27:40,  4.96s/it]
 79%|███████▉  | 177461/225000 [337:49:39<63:09:13,  4.78s/it]
 79%|███████▉  | 177462/225000 [337:49:46<73:23:41,  5.56s/it]
 79%|███████▉  | 177463/225000 [337:49:55<85:40:00,  6.49s/it]
 79%|███████▉  | 177464/225000 [337:49:59<73:50:41,  5.59s/it]
 79%|███████▉  | 177465/225000 [337:50:07<84:16:09,  6.38s/it]
 79%|███████▉  | 177466/225000 [337:50:10<73:14:38,  5.55s/it]
 79%|███████▉  | 177467/225000 [337:50:16<74:06:55,  5.61s/it]
 79%|███████▉  | 177468/225000 [337:50:23<77:55:35,  5.90s/it]
 79%|███████▉  | 177469/225000 [337:50:27<71:38:58,  5.43s/it]
 79%|███████▉  | 177470/225000 [337:50:31<66:23:57,  5.03s/it]
 79%|███████▉  | 177471/225000 [337:50:35<62:32:41,  4.74s/it]
 79%|███████▉  | 177472/225000 [337:50:41<65:24:02,  4.95s/it]
 79%|███████▉  | 177473/225000 [337:50:50<80:35:31,  6.10s/it]
 79%|███████▉  | 177474/225000 [337:50:55<76:33:36,  5.80s/it]
 79%|███████▉  | 177475/225000 [337:51:00<74:26:59,  5.64s/it]
 79%|███████▉  | 177476/225000 [337:51:04<68:08:55,  5.16s/it]
 79%|███████▉  | 177477/225000 [337:51:08<65:06:38,  4.93s/it]
 79%|███████▉  | 177478/225000 [337:51:13<62:24:20,  4.73s/it]
 79%|███████▉  | 177479/225000 [337:51:18<63:54:35,  4.84s/it]
 79%|███████▉  | 177480/225000 [337:51:29<89:01:13,  6.74s/it]
 79%|███████▉  | 177481/225000 [337:51:33<76:47:13,  5.82s/it]
 79%|███████▉  | 177482/225000 [337:51:37<69:55:15,  5.30s/it]
 79%|███████▉  | 177483/225000 [337:51:45<81:33:16,  6.18s/it]
 79%|███████▉  | 177484/225000 [337:51:48<70:55:54,  5.37s/it]
 79%|███████▉  | 177485/225000 [337:51:52<63:49:18,  4.84s/it]
 79%|███████▉  | 177486/225000 [337:52:01<80:18:11,  6.08s/it]
 79%|███████▉  | 177487/225000 [337:52:05<70:56:47,  5.38s/it]
 79%|███████▉  | 177488/225000 [337:52:11<75:42:34,  5.74s/it]
 79%|███████▉  | 177489/225000 [337:52:16<72:57:12,  5.53s/it]
 79%|███████▉  | 177490/225000 [337:52:21<68:28:44,  5.19s/it]
 79%|███████▉  | 177491/225000 [337:52:25<65:44:52,  4.98s/it]
 79%|███████▉  | 177492/225000 [337:52:29<62:56:46,  4.77s/it]
 79%|███████▉  | 177493/225000 [337:52:37<72:14:02,  5.47s/it]
 79%|███████▉  | 177494/225000 [337:52:44<81:48:43,  6.20s/it]
 79%|███████▉  | 177495/225000 [337:52:48<71:17:03,  5.40s/it]
 79%|███████▉  | 177496/225000 [337:52:52<67:44:05,  5.13s/it]
 79%|███████▉  | 177497/225000 [337:52:57<64:01:42,  4.85s/it]
 79%|███████▉  | 177498/225000 [337:53:01<62:09:32,  4.71s/it]
 79%|███████▉  | 177499/225000 [337:53:05<60:54:51,  4.62s/it]
 79%|███████▉  | 177500/225000 [337:53:10<60:48:38,  4.61s/it]
                                                              
{'loss': '0.7561', 'grad_norm': '1.025', 'learning_rate': '5.803e-06', 'epoch': '0.7889', 'num_input_tokens_seen': 560052544, 'train_runtime': '1.216e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177500/225000 [337:53:10<60:48:38,  4.61s/it]
 79%|███████▉  | 177501/225000 [337:53:14<59:30:55,  4.51s/it]
 79%|███████▉  | 177502/225000 [337:53:26<89:43:28,  6.80s/it]
 79%|███████▉  | 177503/225000 [337:53:32<84:40:28,  6.42s/it]
 79%|███████▉  | 177504/225000 [337:53:36<73:45:40,  5.59s/it]
 79%|███████▉  | 177505/225000 [337:53:44<86:15:03,  6.54s/it]
 79%|███████▉  | 177506/225000 [337:53:48<73:15:54,  5.55s/it]
 79%|███████▉  | 177507/225000 [337:53:52<66:30:23,  5.04s/it]
 79%|███████▉  | 177508/225000 [337:53:56<64:22:47,  4.88s/it]
 79%|███████▉  | 177509/225000 [337:54:00<62:47:42,  4.76s/it]
 79%|███████▉  | 177510/225000 [337:54:05<62:37:52,  4.75s/it]
 79%|███████▉  | 177511/225000 [337:54:17<89:20:49,  6.77s/it]
 79%|███████▉  | 177512/225000 [337:54:20<75:08:48,  5.70s/it]
 79%|███████▉  | 177513/225000 [337:54:27<81:25:33,  6.17s/it]
 79%|███████▉  | 177514/225000 [337:54:31<73:33:39,  5.58s/it]
 79%|███████▉  | 177515/225000 [337:54:36<69:27:47,  5.27s/it]
 79%|███████▉  | 177516/225000 [337:54:40<65:04:01,  4.93s/it]
 79%|███████▉  | 177517/225000 [337:54:44<63:04:44,  4.78s/it]
 79%|███████▉  | 177518/225000 [337:54:49<60:37:08,  4.60s/it]
 79%|███████▉  | 177519/225000 [337:54:53<59:06:08,  4.48s/it]
 79%|███████▉  | 177520/225000 [337:55:04<84:19:23,  6.39s/it]
 79%|███████▉  | 177521/225000 [337:55:08<74:42:43,  5.66s/it]
 79%|███████▉  | 177522/225000 [337:55:11<67:03:38,  5.08s/it]
 79%|███████▉  | 177523/225000 [337:55:16<65:34:20,  4.97s/it]
 79%|███████▉  | 177524/225000 [337:55:25<81:49:15,  6.20s/it]
 79%|███████▉  | 177525/225000 [337:55:30<74:23:13,  5.64s/it]
 79%|███████▉  | 177526/225000 [337:55:34<71:22:08,  5.41s/it]
 79%|███████▉  | 177527/225000 [337:55:39<68:23:55,  5.19s/it]
 79%|███████▉  | 177528/225000 [337:55:43<64:21:58,  4.88s/it]
 79%|███████▉  | 177529/225000 [337:55:48<62:56:20,  4.77s/it]
 79%|███████▉  | 177530/225000 [337:55:52<60:45:06,  4.61s/it]
 79%|███████▉  | 177531/225000 [337:56:02<81:56:55,  6.21s/it]
 79%|███████▉  | 177532/225000 [337:56:08<80:16:40,  6.09s/it]
 79%|███████▉  | 177533/225000 [337:56:11<69:31:14,  5.27s/it]
 79%|███████▉  | 177534/225000 [337:56:19<79:29:17,  6.03s/it]
 79%|███████▉  | 177535/225000 [337:56:23<70:09:49,  5.32s/it]
 79%|███████▉  | 177536/225000 [337:56:29<74:39:05,  5.66s/it]
 79%|███████▉  | 177537/225000 [337:56:35<75:30:35,  5.73s/it]
 79%|███████▉  | 177538/225000 [337:56:39<68:36:24,  5.20s/it]
 79%|███████▉  | 177539/225000 [337:56:43<63:31:18,  4.82s/it]
 79%|███████▉  | 177540/225000 [337:56:52<80:09:24,  6.08s/it]
 79%|███████▉  | 177541/225000 [337:56:56<73:47:32,  5.60s/it]
 79%|███████▉  | 177542/225000 [337:57:04<80:21:00,  6.10s/it]
 79%|███████▉  | 177543/225000 [337:57:07<68:10:11,  5.17s/it]
 79%|███████▉  | 177544/225000 [337:57:11<65:20:20,  4.96s/it]
 79%|███████▉  | 177545/225000 [337:57:18<73:34:08,  5.58s/it]
 79%|███████▉  | 177546/225000 [337:57:22<66:09:24,  5.02s/it]
 79%|███████▉  | 177547/225000 [337:57:26<61:48:04,  4.69s/it]
 79%|███████▉  | 177548/225000 [337:57:34<77:00:50,  5.84s/it]
 79%|███████▉  | 177549/225000 [337:57:37<65:30:30,  4.97s/it]
 79%|███████▉  | 177550/225000 [337:57:41<61:46:27,  4.69s/it]
                                                              
{'loss': '0.7669', 'grad_norm': '1.117', 'learning_rate': '5.792e-06', 'epoch': '0.7891', 'num_input_tokens_seen': 560181744, 'train_runtime': '1.217e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177550/225000 [337:57:41<61:46:27,  4.69s/it]
 79%|███████▉  | 177551/225000 [337:57:45<59:00:48,  4.48s/it]
 79%|███████▉  | 177552/225000 [337:57:56<82:36:04,  6.27s/it]
 79%|███████▉  | 177553/225000 [337:57:59<71:48:21,  5.45s/it]
 79%|███████▉  | 177554/225000 [337:58:06<76:43:49,  5.82s/it]
 79%|███████▉  | 177555/225000 [337:58:09<67:37:08,  5.13s/it]
 79%|███████▉  | 177556/225000 [337:58:13<61:39:46,  4.68s/it]
 79%|███████▉  | 177557/225000 [337:58:22<77:58:21,  5.92s/it]
 79%|███████▉  | 177558/225000 [337:58:26<71:09:46,  5.40s/it]
 79%|███████▉  | 177559/225000 [337:58:30<65:51:14,  5.00s/it]
 79%|███████▉  | 177560/225000 [337:58:38<78:56:32,  5.99s/it]
 79%|███████▉  | 177561/225000 [337:58:43<74:33:43,  5.66s/it]
 79%|███████▉  | 177562/225000 [337:58:50<78:35:51,  5.96s/it]
 79%|███████▉  | 177563/225000 [337:58:54<72:09:36,  5.48s/it]
 79%|███████▉  | 177564/225000 [337:58:58<65:52:22,  5.00s/it]
 79%|███████▉  | 177565/225000 [337:59:03<64:28:41,  4.89s/it]
 79%|███████▉  | 177566/225000 [337:59:07<61:19:49,  4.65s/it]
 79%|███████▉  | 177567/225000 [337:59:12<63:18:20,  4.80s/it]
 79%|███████▉  | 177568/225000 [337:59:22<82:21:33,  6.25s/it]
 79%|███████▉  | 177569/225000 [337:59:26<73:30:51,  5.58s/it]
 79%|███████▉  | 177570/225000 [337:59:29<65:43:37,  4.99s/it]
 79%|███████▉  | 177571/225000 [337:59:33<60:59:33,  4.63s/it]
 79%|███████▉  | 177572/225000 [337:59:43<82:19:54,  6.25s/it]
 79%|███████▉  | 177573/225000 [337:59:47<71:41:44,  5.44s/it]
 79%|███████▉  | 177574/225000 [337:59:54<78:31:06,  5.96s/it]
 79%|███████▉  | 177575/225000 [337:59:59<75:29:04,  5.73s/it]
 79%|███████▉  | 177576/225000 [338:00:03<66:46:04,  5.07s/it]
 79%|███████▉  | 177577/225000 [338:00:11<81:23:23,  6.18s/it]
 79%|███████▉  | 177578/225000 [338:00:15<72:01:25,  5.47s/it]
 79%|███████▉  | 177579/225000 [338:00:19<65:04:38,  4.94s/it]
 79%|███████▉  | 177580/225000 [338:00:27<76:35:22,  5.81s/it]
 79%|███████▉  | 177581/225000 [338:00:35<85:59:54,  6.53s/it]
 79%|███████▉  | 177582/225000 [338:00:38<72:41:09,  5.52s/it]
 79%|███████▉  | 177583/225000 [338:00:43<68:39:17,  5.21s/it]
 79%|███████▉  | 177584/225000 [338:00:49<72:32:29,  5.51s/it]
 79%|███████▉  | 177585/225000 [338:00:53<68:06:34,  5.17s/it]
 79%|███████▉  | 177586/225000 [338:00:57<63:18:21,  4.81s/it]
 79%|███████▉  | 177587/225000 [338:01:01<59:42:51,  4.53s/it]
 79%|███████▉  | 177588/225000 [338:01:06<60:26:32,  4.59s/it]
 79%|███████▉  | 177589/225000 [338:01:11<64:08:40,  4.87s/it]
 79%|███████▉  | 177590/225000 [338:01:17<66:04:13,  5.02s/it]
 79%|███████▉  | 177591/225000 [338:01:22<66:09:44,  5.02s/it]
 79%|███████▉  | 177592/225000 [338:01:27<67:55:29,  5.16s/it]
 79%|███████▉  | 177593/225000 [338:01:33<69:09:31,  5.25s/it]
 79%|███████▉  | 177594/225000 [338:01:38<69:25:40,  5.27s/it]
 79%|███████▉  | 177595/225000 [338:01:43<68:08:28,  5.17s/it]
 79%|███████▉  | 177596/225000 [338:01:48<66:55:24,  5.08s/it]
 79%|███████▉  | 177597/225000 [338:01:53<68:25:12,  5.20s/it]
 79%|███████▉  | 177598/225000 [338:01:59<70:13:31,  5.33s/it]
 79%|███████▉  | 177599/225000 [338:02:04<69:12:07,  5.26s/it]
 79%|███████▉  | 177600/225000 [338:02:09<70:28:47,  5.35s/it]
                                                              
{'loss': '0.7656', 'grad_norm': '1.081', 'learning_rate': '5.78e-06', 'epoch': '0.7893', 'num_input_tokens_seen': 560312744, 'train_runtime': '1.217e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177600/225000 [338:02:09<70:28:47,  5.35s/it]
 79%|███████▉  | 177601/225000 [338:02:15<71:14:31,  5.41s/it]
 79%|███████▉  | 177602/225000 [338:02:20<69:51:37,  5.31s/it]
 79%|███████▉  | 177603/225000 [338:02:25<69:12:27,  5.26s/it]
 79%|███████▉  | 177604/225000 [338:02:30<68:56:04,  5.24s/it]
 79%|███████▉  | 177605/225000 [338:02:36<68:37:51,  5.21s/it]
 79%|███████▉  | 177606/225000 [338:02:43<76:05:36,  5.78s/it]
 79%|███████▉  | 177607/225000 [338:02:48<73:07:55,  5.56s/it]
 79%|███████▉  | 177608/225000 [338:02:53<71:50:24,  5.46s/it]
 79%|███████▉  | 177609/225000 [338:02:58<69:42:02,  5.29s/it]
 79%|███████▉  | 177610/225000 [338:03:03<69:50:54,  5.31s/it]
 79%|███████▉  | 177611/225000 [338:03:08<69:09:59,  5.25s/it]
 79%|███████▉  | 177612/225000 [338:03:19<90:51:25,  6.90s/it]
 79%|███████▉  | 177613/225000 [338:03:23<78:13:43,  5.94s/it]
 79%|███████▉  | 177614/225000 [338:03:29<77:56:28,  5.92s/it]
 79%|███████▉  | 177615/225000 [338:03:35<79:41:43,  6.05s/it]
 79%|███████▉  | 177616/225000 [338:03:40<75:06:26,  5.71s/it]
 79%|███████▉  | 177617/225000 [338:03:45<71:01:38,  5.40s/it]
 79%|███████▉  | 177618/225000 [338:03:49<67:24:31,  5.12s/it]
 79%|███████▉  | 177619/225000 [338:03:54<65:48:31,  5.00s/it]
 79%|███████▉  | 177620/225000 [338:03:58<63:17:32,  4.81s/it]
 79%|███████▉  | 177621/225000 [338:04:02<61:19:22,  4.66s/it]
 79%|███████▉  | 177622/225000 [338:04:10<73:01:49,  5.55s/it]
 79%|███████▉  | 177623/225000 [338:04:19<86:17:18,  6.56s/it]
 79%|███████▉  | 177624/225000 [338:04:22<73:39:34,  5.60s/it]
 79%|███████▉  | 177625/225000 [338:04:30<83:20:42,  6.33s/it]
 79%|███████▉  | 177626/225000 [338:04:35<77:19:11,  5.88s/it]
 79%|███████▉  | 177627/225000 [338:04:39<69:55:05,  5.31s/it]
 79%|███████▉  | 177628/225000 [338:04:43<64:44:59,  4.92s/it]
 79%|███████▉  | 177629/225000 [338:04:54<86:31:25,  6.58s/it]
 79%|███████▉  | 177630/225000 [338:04:57<74:42:39,  5.68s/it]
 79%|███████▉  | 177631/225000 [338:05:01<67:52:45,  5.16s/it]
 79%|███████▉  | 177632/225000 [338:05:05<61:59:53,  4.71s/it]
 79%|███████▉  | 177633/225000 [338:05:14<80:01:07,  6.08s/it]
 79%|███████▉  | 177634/225000 [338:05:18<72:32:59,  5.51s/it]
 79%|███████▉  | 177635/225000 [338:05:23<67:39:02,  5.14s/it]
 79%|███████▉  | 177636/225000 [338:05:27<63:38:18,  4.84s/it]
 79%|███████▉  | 177637/225000 [338:05:31<61:06:52,  4.65s/it]
 79%|███████▉  | 177638/225000 [338:05:42<88:28:02,  6.72s/it]
 79%|███████▉  | 177639/225000 [338:05:46<75:46:14,  5.76s/it]
 79%|███████▉  | 177640/225000 [338:05:49<66:29:47,  5.05s/it]
 79%|███████▉  | 177641/225000 [338:05:58<81:57:39,  6.23s/it]
 79%|███████▉  | 177642/225000 [338:06:02<72:07:01,  5.48s/it]
 79%|███████▉  | 177643/225000 [338:06:06<65:49:37,  5.00s/it]
 79%|███████▉  | 177644/225000 [338:06:10<63:06:10,  4.80s/it]
 79%|███████▉  | 177645/225000 [338:06:15<61:23:42,  4.67s/it]
 79%|███████▉  | 177646/225000 [338:06:19<60:50:01,  4.62s/it]
 79%|███████▉  | 177647/225000 [338:06:23<59:02:14,  4.49s/it]
 79%|███████▉  | 177648/225000 [338:06:30<69:08:59,  5.26s/it]
 79%|███████▉  | 177649/225000 [338:06:40<87:18:11,  6.64s/it]
 79%|███████▉  | 177650/225000 [338:06:44<75:17:10,  5.72s/it]
                                                              
{'loss': '0.7641', 'grad_norm': '1.102', 'learning_rate': '5.768e-06', 'epoch': '0.7896', 'num_input_tokens_seen': 560441672, 'train_runtime': '1.217e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177650/225000 [338:06:44<75:17:10,  5.72s/it]
 79%|███████▉  | 177651/225000 [338:06:51<80:07:59,  6.09s/it]
 79%|███████▉  | 177652/225000 [338:06:55<70:41:13,  5.37s/it]
 79%|███████▉  | 177653/225000 [338:06:59<65:11:08,  4.96s/it]
 79%|███████▉  | 177654/225000 [338:07:08<84:14:53,  6.41s/it]
 79%|███████▉  | 177655/225000 [338:07:12<72:41:24,  5.53s/it]
 79%|███████▉  | 177656/225000 [338:07:20<83:18:24,  6.33s/it]
 79%|███████▉  | 177657/225000 [338:07:24<73:54:36,  5.62s/it]
 79%|███████▉  | 177658/225000 [338:07:28<66:48:04,  5.08s/it]
 79%|███████▉  | 177659/225000 [338:07:32<61:46:33,  4.70s/it]
 79%|███████▉  | 177660/225000 [338:07:36<60:09:18,  4.57s/it]
 79%|███████▉  | 177661/225000 [338:07:40<57:49:01,  4.40s/it]
 79%|███████▉  | 177662/225000 [338:07:44<58:10:29,  4.42s/it]
 79%|███████▉  | 177663/225000 [338:07:57<90:47:41,  6.90s/it]
 79%|███████▉  | 177664/225000 [338:08:00<76:42:16,  5.83s/it]
 79%|███████▉  | 177665/225000 [338:08:05<71:28:00,  5.44s/it]
 79%|███████▉  | 177666/225000 [338:08:12<76:28:43,  5.82s/it]
 79%|███████▉  | 177667/225000 [338:08:16<71:00:05,  5.40s/it]
 79%|███████▉  | 177668/225000 [338:08:20<67:03:03,  5.10s/it]
 79%|███████▉  | 177669/225000 [338:08:25<65:28:01,  4.98s/it]
 79%|███████▉  | 177670/225000 [338:08:30<63:44:32,  4.85s/it]
 79%|███████▉  | 177671/225000 [338:08:40<83:52:29,  6.38s/it]
 79%|███████▉  | 177672/225000 [338:08:43<71:43:39,  5.46s/it]
 79%|███████▉  | 177673/225000 [338:08:49<74:53:02,  5.70s/it]
 79%|███████▉  | 177674/225000 [338:08:53<67:13:54,  5.11s/it]
 79%|███████▉  | 177675/225000 [338:09:02<83:25:08,  6.35s/it]
 79%|███████▉  | 177676/225000 [338:09:05<71:35:01,  5.45s/it]
 79%|███████▉  | 177677/225000 [338:09:09<62:34:48,  4.76s/it]
 79%|███████▉  | 177678/225000 [338:09:17<76:01:39,  5.78s/it]
 79%|███████▉  | 177679/225000 [338:09:21<68:55:30,  5.24s/it]
 79%|███████▉  | 177680/225000 [338:09:28<77:50:55,  5.92s/it]
 79%|███████▉  | 177681/225000 [338:09:32<70:12:54,  5.34s/it]
 79%|███████▉  | 177682/225000 [338:09:36<64:33:48,  4.91s/it]
 79%|███████▉  | 177683/225000 [338:09:40<60:25:49,  4.60s/it]
 79%|███████▉  | 177684/225000 [338:09:44<58:47:23,  4.47s/it]
 79%|███████▉  | 177685/225000 [338:09:49<61:35:13,  4.69s/it]
 79%|███████▉  | 177686/225000 [338:09:54<61:24:52,  4.67s/it]
 79%|███████▉  | 177687/225000 [338:09:59<63:02:26,  4.80s/it]
 79%|███████▉  | 177688/225000 [338:10:05<65:38:15,  4.99s/it]
 79%|███████▉  | 177689/225000 [338:10:09<63:59:53,  4.87s/it]
 79%|███████▉  | 177690/225000 [338:10:14<63:54:10,  4.86s/it]
 79%|███████▉  | 177691/225000 [338:10:18<60:11:39,  4.58s/it]
 79%|███████▉  | 177692/225000 [338:10:27<77:48:22,  5.92s/it]
 79%|███████▉  | 177693/225000 [338:10:36<91:10:41,  6.94s/it]
 79%|███████▉  | 177694/225000 [338:10:40<78:33:47,  5.98s/it]
 79%|███████▉  | 177695/225000 [338:10:44<70:24:45,  5.36s/it]
 79%|███████▉  | 177696/225000 [338:10:48<66:48:02,  5.08s/it]
 79%|███████▉  | 177697/225000 [338:10:53<64:16:59,  4.89s/it]
 79%|███████▉  | 177698/225000 [338:10:57<62:14:27,  4.74s/it]
 79%|███████▉  | 177699/225000 [338:11:09<88:39:35,  6.75s/it]
 79%|███████▉  | 177700/225000 [338:11:13<79:41:16,  6.07s/it]
                                                              
{'loss': '0.7613', 'grad_norm': '1.135', 'learning_rate': '5.757e-06', 'epoch': '0.7898', 'num_input_tokens_seen': 560571104, 'train_runtime': '1.217e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177700/225000 [338:11:13<79:41:16,  6.07s/it]
 79%|███████▉  | 177701/225000 [338:11:17<69:19:52,  5.28s/it]
 79%|███████▉  | 177702/225000 [338:11:20<63:16:15,  4.82s/it]
 79%|███████▉  | 177703/225000 [338:11:30<83:14:27,  6.34s/it]
 79%|███████▉  | 177704/225000 [338:11:34<71:19:44,  5.43s/it]
 79%|███████▉  | 177705/225000 [338:11:37<63:54:29,  4.86s/it]
 79%|███████▉  | 177706/225000 [338:11:43<66:21:16,  5.05s/it]
 79%|███████▉  | 177707/225000 [338:11:48<68:10:09,  5.19s/it]
 79%|███████▉  | 177708/225000 [338:11:54<70:35:06,  5.37s/it]
 79%|███████▉  | 177709/225000 [338:11:59<68:59:57,  5.25s/it]
 79%|███████▉  | 177710/225000 [338:12:05<70:45:39,  5.39s/it]
 79%|███████▉  | 177711/225000 [338:12:09<68:46:06,  5.24s/it]
 79%|███████▉  | 177712/225000 [338:12:15<69:39:01,  5.30s/it]
 79%|███████▉  | 177713/225000 [338:12:20<69:02:08,  5.26s/it]
 79%|███████▉  | 177714/225000 [338:12:25<69:54:40,  5.32s/it]
 79%|███████▉  | 177715/225000 [338:12:31<69:30:14,  5.29s/it]
 79%|███████▉  | 177716/225000 [338:12:36<69:51:01,  5.32s/it]
 79%|███████▉  | 177717/225000 [338:12:41<67:26:47,  5.14s/it]
 79%|███████▉  | 177718/225000 [338:12:46<68:48:17,  5.24s/it]
 79%|███████▉  | 177719/225000 [338:12:52<69:56:26,  5.33s/it]
 79%|███████▉  | 177720/225000 [338:12:57<69:45:13,  5.31s/it]
 79%|███████▉  | 177721/225000 [338:13:02<66:48:57,  5.09s/it]
 79%|███████▉  | 177722/225000 [338:13:07<67:54:28,  5.17s/it]
 79%|███████▉  | 177723/225000 [338:13:12<68:36:21,  5.22s/it]
 79%|███████▉  | 177724/225000 [338:13:18<70:16:04,  5.35s/it]
 79%|███████▉  | 177725/225000 [338:13:23<69:23:24,  5.28s/it]
 79%|███████▉  | 177726/225000 [338:13:29<69:50:51,  5.32s/it]
 79%|███████▉  | 177727/225000 [338:13:34<69:34:07,  5.30s/it]
 79%|███████▉  | 177728/225000 [338:13:39<70:41:56,  5.38s/it]
 79%|███████▉  | 177729/225000 [338:13:45<71:20:43,  5.43s/it]
 79%|███████▉  | 177730/225000 [338:13:50<68:53:32,  5.25s/it]
 79%|███████▉  | 177731/225000 [338:13:55<68:27:25,  5.21s/it]
 79%|███████▉  | 177732/225000 [338:14:01<71:40:33,  5.46s/it]
 79%|███████▉  | 177733/225000 [338:14:11<88:18:58,  6.73s/it]
 79%|███████▉  | 177734/225000 [338:14:14<77:03:26,  5.87s/it]
 79%|███████▉  | 177735/225000 [338:14:18<68:27:20,  5.21s/it]
 79%|███████▉  | 177736/225000 [338:14:29<90:03:08,  6.86s/it]
 79%|███████▉  | 177737/225000 [338:14:33<81:17:31,  6.19s/it]
 79%|███████▉  | 177738/225000 [338:14:38<74:55:38,  5.71s/it]
 79%|███████▉  | 177739/225000 [338:14:43<70:45:02,  5.39s/it]
 79%|███████▉  | 177740/225000 [338:14:47<66:24:39,  5.06s/it]
 79%|███████▉  | 177741/225000 [338:14:52<64:18:54,  4.90s/it]
 79%|███████▉  | 177742/225000 [338:14:56<64:21:18,  4.90s/it]
 79%|███████▉  | 177743/225000 [338:15:01<64:43:17,  4.93s/it]
 79%|███████▉  | 177744/225000 [338:15:06<62:55:26,  4.79s/it]
 79%|███████▉  | 177745/225000 [338:15:11<63:50:56,  4.86s/it]
 79%|███████▉  | 177746/225000 [338:15:23<93:43:03,  7.14s/it]
 79%|███████▉  | 177747/225000 [338:15:26<77:33:34,  5.91s/it]
 79%|███████▉  | 177748/225000 [338:15:30<67:50:29,  5.17s/it]
 79%|███████▉  | 177749/225000 [338:15:38<80:37:39,  6.14s/it]
 79%|███████▉  | 177750/225000 [338:15:42<71:28:50,  5.45s/it]
                                                              
{'loss': '0.7636', 'grad_norm': '1.093', 'learning_rate': '5.745e-06', 'epoch': '0.79', 'num_input_tokens_seen': 560700312, 'train_runtime': '1.218e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177750/225000 [338:15:42<71:28:50,  5.45s/it]
 79%|███████▉  | 177751/225000 [338:15:47<69:22:20,  5.29s/it]
 79%|███████▉  | 177752/225000 [338:15:54<75:08:14,  5.73s/it]
 79%|███████▉  | 177753/225000 [338:16:00<75:52:28,  5.78s/it]
 79%|███████▉  | 177754/225000 [338:16:06<78:33:07,  5.99s/it]
 79%|███████▉  | 177755/225000 [338:16:10<68:35:45,  5.23s/it]
 79%|███████▉  | 177756/225000 [338:16:17<77:39:34,  5.92s/it]
 79%|███████▉  | 177757/225000 [338:16:20<66:22:21,  5.06s/it]
 79%|███████▉  | 177758/225000 [338:16:25<67:25:20,  5.14s/it]
 79%|███████▉  | 177759/225000 [338:16:34<80:11:49,  6.11s/it]
 79%|███████▉  | 177760/225000 [338:16:37<69:55:16,  5.33s/it]
 79%|███████▉  | 177761/225000 [338:16:42<66:18:42,  5.05s/it]
 79%|███████▉  | 177762/225000 [338:16:50<79:05:01,  6.03s/it]
 79%|███████▉  | 177763/225000 [338:16:54<72:08:35,  5.50s/it]
 79%|███████▉  | 177764/225000 [338:16:59<69:53:34,  5.33s/it]
 79%|███████▉  | 177765/225000 [338:17:04<67:48:06,  5.17s/it]
 79%|███████▉  | 177766/225000 [338:17:09<66:15:32,  5.05s/it]
 79%|███████▉  | 177767/225000 [338:17:14<66:00:13,  5.03s/it]
 79%|███████▉  | 177768/225000 [338:17:19<65:59:04,  5.03s/it]
 79%|███████▉  | 177769/225000 [338:17:24<66:49:49,  5.09s/it]
 79%|███████▉  | 177770/225000 [338:17:29<66:34:13,  5.07s/it]
 79%|███████▉  | 177771/225000 [338:17:34<64:36:43,  4.93s/it]
 79%|███████▉  | 177772/225000 [338:17:38<63:56:05,  4.87s/it]
 79%|███████▉  | 177773/225000 [338:17:43<64:15:15,  4.90s/it]
 79%|███████▉  | 177774/225000 [338:17:53<82:30:36,  6.29s/it]
 79%|███████▉  | 177775/225000 [338:18:01<89:31:09,  6.82s/it]
 79%|███████▉  | 177776/225000 [338:18:07<84:19:19,  6.43s/it]
 79%|███████▉  | 177777/225000 [338:18:11<76:58:43,  5.87s/it]
 79%|███████▉  | 177778/225000 [338:18:14<67:15:10,  5.13s/it]
 79%|███████▉  | 177779/225000 [338:18:19<65:11:31,  4.97s/it]
 79%|███████▉  | 177780/225000 [338:18:24<66:05:34,  5.04s/it]
 79%|███████▉  | 177781/225000 [338:18:29<66:24:25,  5.06s/it]
 79%|███████▉  | 177782/225000 [338:18:35<67:26:29,  5.14s/it]
 79%|███████▉  | 177783/225000 [338:18:40<66:47:15,  5.09s/it]
 79%|███████▉  | 177784/225000 [338:18:44<63:56:34,  4.88s/it]
 79%|███████▉  | 177785/225000 [338:18:49<65:19:46,  4.98s/it]
 79%|███████▉  | 177786/225000 [338:18:54<64:49:21,  4.94s/it]
 79%|███████▉  | 177787/225000 [338:18:59<65:43:40,  5.01s/it]
 79%|███████▉  | 177788/225000 [338:19:12<94:05:50,  7.18s/it]
 79%|███████▉  | 177789/225000 [338:19:15<80:24:34,  6.13s/it]
 79%|███████▉  | 177790/225000 [338:19:19<69:52:06,  5.33s/it]
 79%|███████▉  | 177791/225000 [338:19:24<69:21:05,  5.29s/it]
 79%|███████▉  | 177792/225000 [338:19:33<82:47:52,  6.31s/it]
 79%|███████▉  | 177793/225000 [338:19:38<78:39:13,  6.00s/it]
 79%|███████▉  | 177794/225000 [338:19:43<73:47:01,  5.63s/it]
 79%|███████▉  | 177795/225000 [338:19:48<71:33:12,  5.46s/it]
 79%|███████▉  | 177796/225000 [338:19:53<70:19:21,  5.36s/it]
 79%|███████▉  | 177797/225000 [338:19:58<68:20:29,  5.21s/it]
 79%|███████▉  | 177798/225000 [338:20:03<68:46:43,  5.25s/it]
 79%|███████▉  | 177799/225000 [338:20:08<68:30:25,  5.23s/it]
 79%|███████▉  | 177800/225000 [338:20:13<67:36:52,  5.16s/it]
                                                              
{'loss': '0.7589', 'grad_norm': '1.064', 'learning_rate': '5.733e-06', 'epoch': '0.7902', 'num_input_tokens_seen': 560829952, 'train_runtime': '1.218e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177800/225000 [338:20:13<67:36:52,  5.16s/it]
 79%|███████▉  | 177801/225000 [338:20:18<65:29:20,  5.00s/it]
 79%|███████▉  | 177802/225000 [338:20:23<66:22:24,  5.06s/it]
 79%|███████▉  | 177803/225000 [338:20:28<67:01:46,  5.11s/it]
 79%|███████▉  | 177804/225000 [338:20:33<66:35:23,  5.08s/it]
 79%|███████▉  | 177805/225000 [338:20:38<66:02:13,  5.04s/it]
 79%|███████▉  | 177806/225000 [338:20:43<64:44:53,  4.94s/it]
 79%|███████▉  | 177807/225000 [338:20:48<66:27:33,  5.07s/it]
 79%|███████▉  | 177808/225000 [338:20:53<64:46:35,  4.94s/it]
 79%|███████▉  | 177809/225000 [338:20:58<65:57:36,  5.03s/it]
 79%|███████▉  | 177810/225000 [338:21:03<66:39:07,  5.08s/it]
 79%|███████▉  | 177811/225000 [338:21:08<64:21:57,  4.91s/it]
 79%|███████▉  | 177812/225000 [338:21:13<66:27:31,  5.07s/it]
 79%|███████▉  | 177813/225000 [338:21:19<68:49:17,  5.25s/it]
 79%|███████▉  | 177814/225000 [338:21:24<68:54:50,  5.26s/it]
 79%|███████▉  | 177815/225000 [338:21:29<68:22:52,  5.22s/it]
 79%|███████▉  | 177816/225000 [338:21:35<68:19:31,  5.21s/it]
 79%|███████▉  | 177817/225000 [338:21:40<68:12:40,  5.20s/it]
 79%|███████▉  | 177818/225000 [338:21:51<91:25:05,  6.98s/it]
 79%|███████▉  | 177819/225000 [338:21:56<85:38:39,  6.53s/it]
 79%|███████▉  | 177820/225000 [338:22:00<74:45:50,  5.70s/it]
 79%|███████▉  | 177821/225000 [338:22:04<67:36:12,  5.16s/it]
 79%|███████▉  | 177822/225000 [338:22:14<86:55:10,  6.63s/it]
 79%|███████▉  | 177823/225000 [338:22:17<73:39:27,  5.62s/it]
 79%|███████▉  | 177824/225000 [338:22:24<78:19:01,  5.98s/it]
 79%|███████▉  | 177825/225000 [338:22:28<69:13:21,  5.28s/it]
 79%|███████▉  | 177826/225000 [338:22:35<76:43:08,  5.85s/it]
 79%|███████▉  | 177827/225000 [338:22:41<76:25:08,  5.83s/it]
 79%|███████▉  | 177828/225000 [338:22:45<68:10:23,  5.20s/it]
 79%|███████▉  | 177829/225000 [338:22:54<84:58:01,  6.48s/it]
 79%|███████▉  | 177830/225000 [338:22:57<72:40:33,  5.55s/it]
 79%|███████▉  | 177831/225000 [338:23:01<66:54:49,  5.11s/it]
 79%|███████▉  | 177832/225000 [338:23:06<64:59:31,  4.96s/it]
 79%|███████▉  | 177833/225000 [338:23:11<63:21:15,  4.84s/it]
 79%|███████▉  | 177834/225000 [338:23:15<60:19:45,  4.60s/it]
 79%|███████▉  | 177835/225000 [338:23:19<60:08:36,  4.59s/it]
 79%|███████▉  | 177836/225000 [338:23:31<87:05:09,  6.65s/it]
 79%|███████▉  | 177837/225000 [338:23:34<75:02:43,  5.73s/it]
 79%|███████▉  | 177838/225000 [338:23:38<67:16:37,  5.14s/it]
 79%|███████▉  | 177839/225000 [338:23:46<79:20:14,  6.06s/it]
 79%|███████▉  | 177840/225000 [338:23:50<71:00:36,  5.42s/it]
 79%|███████▉  | 177841/225000 [338:23:57<77:06:51,  5.89s/it]
 79%|███████▉  | 177842/225000 [338:24:02<74:41:46,  5.70s/it]
 79%|███████▉  | 177843/225000 [338:24:07<69:12:03,  5.28s/it]
 79%|███████▉  | 177844/225000 [338:24:11<65:05:09,  4.97s/it]
 79%|███████▉  | 177845/225000 [338:24:15<60:53:27,  4.65s/it]
 79%|███████▉  | 177846/225000 [338:24:20<62:11:48,  4.75s/it]
 79%|███████▉  | 177847/225000 [338:24:30<83:01:14,  6.34s/it]
 79%|███████▉  | 177848/225000 [338:24:33<71:34:27,  5.46s/it]
 79%|███████▉  | 177849/225000 [338:24:37<64:39:12,  4.94s/it]
 79%|███████▉  | 177850/225000 [338:24:45<78:19:18,  5.98s/it]
                                                              
{'loss': '0.7664', 'grad_norm': '1.138', 'learning_rate': '5.722e-06', 'epoch': '0.7904', 'num_input_tokens_seen': 560960248, 'train_runtime': '1.218e+06', 'train_tokens_per_second': '460.4'}

 79%|███████▉  | 177850/225000 [338:24:45<78:19:18,  5.98s/it]
 79%|███████▉  | 177851/225000 [338:24:49<70:11:14,  5.36s/it]
 79%|███████▉  | 177852/225000 [338:24:53<64:50:44,  4.95s/it]
 79%|███████▉  | 177853/225000 [338:24:57<61:23:30,  4.69s/it]
 79%|███████▉  | 177854/225000 [338:25:02<61:20:14,  4.68s/it]
 79%|███████▉  | 177855/225000 [338:25:07<61:39:28,  4.71s/it]
 79%|███████▉  | 177856/225000 [338:25:11<60:04:36,  4.59s/it]
 79%|███████▉  | 177857/225000 [338:25:15<58:42:12,  4.48s/it]
 79%|███████▉  | 177858/225000 [338:25:28<88:53:55,  6.79s/it]
 79%|███████▉  | 177859/225000 [338:25:32<79:10:00,  6.05s/it]
 79%|███████▉  | 177860/225000 [338:25:39<81:29:01,  6.22s/it]
 79%|███████▉  | 177861/225000 [338:25:42<71:49:56,  5.49s/it]
 79%|███████▉  | 177862/225000 [338:25:50<82:19:49,  6.29s/it]
 79%|███████▉  | 177863/225000 [338:25:54<72:10:20,  5.51s/it]
 79%|███████▉  | 177864/225000 [338:25:59<69:59:01,  5.34s/it]
 79%|███████▉  | 177865/225000 [338:26:06<74:08:00,  5.66s/it]
 79%|███████▉  | 177866/225000 [338:26:10<68:29:21,  5.23s/it]
 79%|███████▉  | 177867/225000 [338:26:15<67:20:01,  5.14s/it]
 79%|███████▉  | 177868/225000 [338:26:20<67:48:44,  5.18s/it]
 79%|███████▉  | 177869/225000 [338:26:25<68:40:46,  5.25s/it]
 79%|███████▉  | 177870/225000 [338:26:31<68:53:23,  5.26s/it]
 79%|███████▉  | 177871/225000 [338:26:36<69:29:07,  5.31s/it]
 79%|███████▉  | 177872/225000 [338:26:41<66:59:14,  5.12s/it]
 79%|███████▉  | 177873/225000 [338:26:46<66:40:21,  5.09s/it]
 79%|███████▉  | 177874/225000 [338:26:54<78:20:11,  5.98s/it]
 79%|███████▉  | 177875/225000 [338:26:58<72:58:03,  5.57s/it]
 79%|███████▉  | 177876/225000 [338:27:03<67:20:50,  5.14s/it]
 79%|███████▉  | 177877/225000 [338:27:07<63:51:45,  4.88s/it]
 79%|███████▉  | 177878/225000 [338:27:13<69:09:56,  5.28s/it]
 79%|███████▉  | 177879/225000 [338:27:22<83:33:22,  6.38s/it]
 79%|███████▉  | 177880/225000 [338:27:26<74:18:07,  5.68s/it]
 79%|███████▉  | 177881/225000 [338:27:30<67:40:16,  5.17s/it]
 79%|███████▉  | 177882/225000 [338:27:34<62:22:01,  4.77s/it]
 79%|███████▉  | 177883/225000 [338:27:46<89:53:56,  6.87s/it]
 79%|███████▉  | 177884/225000 [338:27:49<75:15:55,  5.75s/it]
 79%|███████▉  | 177885/225000 [338:27:54<72:43:47,  5.56s/it]
 79%|███████▉  | 177886/225000 [338:28:00<76:37:55,  5.86s/it]
 79%|███████▉  | 177887/225000 [338:28:04<67:19:02,  5.14s/it]
 79%|███████▉  | 177888/225000 [338:28:11<73:38:49,  5.63s/it]
 79%|███████▉  | 177889/225000 [338:28:16<74:00:16,  5.66s/it]
 79%|███████▉  | 177890/225000 [338:28:20<65:17:36,  4.99s/it]
 79%|███████▉  | 177891/225000 [338:28:28<79:31:07,  6.08s/it]
 79%|███████▉  | 177892/225000 [338:28:32<68:44:39,  5.25s/it]
 79%|███████▉  | 177893/225000 [338:28:39<76:48:38,  5.87s/it]
 79%|███████▉  | 177894/225000 [338:28:44<71:48:45,  5.49s/it]
 79%|███████▉  | 177895/225000 [338:28:48<67:17:22,  5.14s/it]
 79%|███████▉  | 177896/225000 [338:28:53<65:00:53,  4.97s/it]
 79%|███████▉  | 177897/225000 [338:28:57<62:49:00,  4.80s/it]
 79%|███████▉  | 177898/225000 [338:29:01<60:44:58,  4.64s/it]
 79%|███████▉  | 177899/225000 [338:29:06<60:37:58,  4.63s/it]
 79%|███████▉  | 177900/225000 [338:29:15<78:45:05,  6.02s/it]
                                                              
{'loss': '0.762', 'grad_norm': '1.169', 'learning_rate': '5.71e-06', 'epoch': '0.7907', 'num_input_tokens_seen': 561089256, 'train_runtime': '1.219e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 177900/225000 [338:29:15<78:45:05,  6.02s/it]
 79%|███████▉  | 177901/225000 [338:29:24<89:30:26,  6.84s/it]
 79%|███████▉  | 177902/225000 [338:29:28<77:26:03,  5.92s/it]
 79%|███████▉  | 177903/225000 [338:29:31<68:22:15,  5.23s/it]
 79%|███████▉  | 177904/225000 [338:29:39<76:35:14,  5.85s/it]
 79%|███████▉  | 177905/225000 [338:29:44<75:11:32,  5.75s/it]
 79%|███████▉  | 177906/225000 [338:29:49<70:48:45,  5.41s/it]
 79%|███████▉  | 177907/225000 [338:29:56<79:39:45,  6.09s/it]
 79%|███████▉  | 177908/225000 [338:30:00<70:39:14,  5.40s/it]
 79%|███████▉  | 177909/225000 [338:30:04<63:51:27,  4.88s/it]
 79%|███████▉  | 177910/225000 [338:30:11<71:10:04,  5.44s/it]
 79%|███████▉  | 177911/225000 [338:30:17<75:19:41,  5.76s/it]
 79%|███████▉  | 177912/225000 [338:30:22<73:44:26,  5.64s/it]
 79%|███████▉  | 177913/225000 [338:30:28<73:39:18,  5.63s/it]
 79%|███████▉  | 177914/225000 [338:30:36<83:32:44,  6.39s/it]
 79%|███████▉  | 177915/225000 [338:30:40<73:27:25,  5.62s/it]
 79%|███████▉  | 177916/225000 [338:30:44<65:51:50,  5.04s/it]
 79%|███████▉  | 177917/225000 [338:30:51<76:19:31,  5.84s/it]
 79%|███████▉  | 177918/225000 [338:30:56<70:08:55,  5.36s/it]
 79%|███████▉  | 177919/225000 [338:31:00<67:30:25,  5.16s/it]
 79%|███████▉  | 177920/225000 [338:31:05<64:28:07,  4.93s/it]
 79%|███████▉  | 177921/225000 [338:31:13<78:46:32,  6.02s/it]
 79%|███████▉  | 177922/225000 [338:31:17<68:22:16,  5.23s/it]
 79%|███████▉  | 177923/225000 [338:31:25<80:11:47,  6.13s/it]
 79%|███████▉  | 177924/225000 [338:31:29<73:06:10,  5.59s/it]
 79%|███████▉  | 177925/225000 [338:31:34<69:02:06,  5.28s/it]
 79%|███████▉  | 177926/225000 [338:31:39<67:39:57,  5.17s/it]
 79%|███████▉  | 177927/225000 [338:31:44<66:28:54,  5.08s/it]
 79%|███████▉  | 177928/225000 [338:31:48<64:45:41,  4.95s/it]
 79%|███████▉  | 177929/225000 [338:31:58<82:04:03,  6.28s/it]
 79%|███████▉  | 177930/225000 [338:32:02<72:38:33,  5.56s/it]
 79%|███████▉  | 177931/225000 [338:32:05<66:04:00,  5.05s/it]
 79%|███████▉  | 177932/225000 [338:32:12<70:19:31,  5.38s/it]
 79%|███████▉  | 177933/225000 [338:32:20<83:16:29,  6.37s/it]
 79%|███████▉  | 177934/225000 [338:32:24<71:50:10,  5.49s/it]
 79%|███████▉  | 177935/225000 [338:32:28<65:26:45,  5.01s/it]
 79%|███████▉  | 177936/225000 [338:32:37<83:24:46,  6.38s/it]
 79%|███████▉  | 177937/225000 [338:32:40<70:21:46,  5.38s/it]
 79%|███████▉  | 177938/225000 [338:32:47<75:56:08,  5.81s/it]
 79%|███████▉  | 177939/225000 [338:32:51<67:28:35,  5.16s/it]
 79%|███████▉  | 177940/225000 [338:32:54<61:35:09,  4.71s/it]
 79%|███████▉  | 177941/225000 [338:33:04<79:30:58,  6.08s/it]
 79%|███████▉  | 177942/225000 [338:33:07<69:44:13,  5.33s/it]
 79%|███████▉  | 177943/225000 [338:33:11<62:33:21,  4.79s/it]
 79%|███████▉  | 177944/225000 [338:33:19<76:23:34,  5.84s/it]
 79%|███████▉  | 177945/225000 [338:33:23<67:23:21,  5.16s/it]
 79%|███████▉  | 177946/225000 [338:33:30<77:11:37,  5.91s/it]
 79%|███████▉  | 177947/225000 [338:33:34<70:23:46,  5.39s/it]
 79%|███████▉  | 177948/225000 [338:33:38<64:48:53,  4.96s/it]
 79%|███████▉  | 177949/225000 [338:33:43<62:24:39,  4.78s/it]
 79%|███████▉  | 177950/225000 [338:33:48<62:56:48,  4.82s/it]
                                                              
{'loss': '0.7612', 'grad_norm': '1.139', 'learning_rate': '5.698e-06', 'epoch': '0.7909', 'num_input_tokens_seen': 561219040, 'train_runtime': '1.219e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 177950/225000 [338:33:48<62:56:48,  4.82s/it]
 79%|███████▉  | 177951/225000 [338:33:53<64:15:37,  4.92s/it]
 79%|███████▉  | 177952/225000 [338:33:58<64:24:02,  4.93s/it]
 79%|███████▉  | 177953/225000 [338:34:06<76:05:58,  5.82s/it]
 79%|███████▉  | 177954/225000 [338:34:15<91:11:13,  6.98s/it]
 79%|███████▉  | 177955/225000 [338:34:20<83:57:59,  6.43s/it]
 79%|███████▉  | 177956/225000 [338:34:25<75:18:31,  5.76s/it]
 79%|███████▉  | 177957/225000 [338:34:28<66:29:42,  5.09s/it]
 79%|███████▉  | 177958/225000 [338:34:36<78:32:05,  6.01s/it]
 79%|███████▉  | 177959/225000 [338:34:41<72:23:30,  5.54s/it]
 79%|███████▉  | 177960/225000 [338:34:47<75:19:02,  5.76s/it]
 79%|███████▉  | 177961/225000 [338:34:51<68:51:55,  5.27s/it]
 79%|███████▉  | 177962/225000 [338:34:55<63:57:24,  4.89s/it]
 79%|███████▉  | 177963/225000 [338:34:59<59:34:56,  4.56s/it]
 79%|███████▉  | 177964/225000 [338:35:06<70:21:16,  5.38s/it]
 79%|███████▉  | 177965/225000 [338:35:14<78:21:19,  6.00s/it]
 79%|███████▉  | 177966/225000 [338:35:17<68:09:59,  5.22s/it]
 79%|███████▉  | 177967/225000 [338:35:21<62:26:43,  4.78s/it]
 79%|███████▉  | 177968/225000 [338:35:25<61:08:35,  4.68s/it]
 79%|███████▉  | 177969/225000 [338:35:30<60:55:00,  4.66s/it]
 79%|███████▉  | 177970/225000 [338:35:40<83:34:42,  6.40s/it]
 79%|███████▉  | 177971/225000 [338:35:44<73:32:29,  5.63s/it]
 79%|███████▉  | 177972/225000 [338:35:48<66:05:55,  5.06s/it]
 79%|███████▉  | 177973/225000 [338:35:52<62:11:51,  4.76s/it]
 79%|███████▉  | 177974/225000 [338:36:02<83:44:40,  6.41s/it]
 79%|███████▉  | 177975/225000 [338:36:07<77:09:32,  5.91s/it]
 79%|███████▉  | 177976/225000 [338:36:10<67:35:21,  5.17s/it]
 79%|███████▉  | 177977/225000 [338:36:17<74:16:00,  5.69s/it]
 79%|███████▉  | 177978/225000 [338:36:22<69:35:34,  5.33s/it]
 79%|███████▉  | 177979/225000 [338:36:26<65:56:15,  5.05s/it]
 79%|███████▉  | 177980/225000 [338:36:31<64:26:43,  4.93s/it]
 79%|███████▉  | 177981/225000 [338:36:35<60:42:39,  4.65s/it]
 79%|███████▉  | 177982/225000 [338:36:39<59:20:40,  4.54s/it]
 79%|███████▉  | 177983/225000 [338:36:47<71:53:59,  5.51s/it]
 79%|███████▉  | 177984/225000 [338:36:54<76:30:05,  5.86s/it]
 79%|███████▉  | 177985/225000 [338:36:58<69:51:19,  5.35s/it]
 79%|███████▉  | 177986/225000 [338:37:02<65:21:06,  5.00s/it]
 79%|███████▉  | 177987/225000 [338:37:06<61:41:23,  4.72s/it]
 79%|███████▉  | 177988/225000 [338:37:12<67:52:53,  5.20s/it]
 79%|███████▉  | 177989/225000 [338:37:21<82:06:54,  6.29s/it]
 79%|███████▉  | 177990/225000 [338:37:25<72:12:30,  5.53s/it]
 79%|███████▉  | 177991/225000 [338:37:29<65:24:41,  5.01s/it]
 79%|███████▉  | 177992/225000 [338:37:38<83:35:14,  6.40s/it]
 79%|███████▉  | 177993/225000 [338:37:42<74:08:49,  5.68s/it]
 79%|███████▉  | 177994/225000 [338:37:46<66:45:37,  5.11s/it]
 79%|███████▉  | 177995/225000 [338:37:51<64:14:47,  4.92s/it]
 79%|███████▉  | 177996/225000 [338:37:55<62:03:15,  4.75s/it]
 79%|███████▉  | 177997/225000 [338:38:08<93:37:43,  7.17s/it]
 79%|███████▉  | 177998/225000 [338:38:11<77:17:35,  5.92s/it]
 79%|███████▉  | 177999/225000 [338:38:17<80:09:36,  6.14s/it]
 79%|███████▉  | 178000/225000 [338:38:22<73:01:16,  5.59s/it]
                                                              
{'loss': '0.7495', 'grad_norm': '1.124', 'learning_rate': '5.687e-06', 'epoch': '0.7911', 'num_input_tokens_seen': 561348040, 'train_runtime': '1.219e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178000/225000 [338:38:22<73:01:16,  5.59s/it]
 79%|███████▉  | 178001/225000 [338:38:26<67:50:52,  5.20s/it]
 79%|███████▉  | 178002/225000 [338:38:30<63:45:13,  4.88s/it]
 79%|███████▉  | 178003/225000 [338:38:40<83:44:01,  6.41s/it]
 79%|███████▉  | 178004/225000 [338:38:44<72:51:52,  5.58s/it]
 79%|███████▉  | 178005/225000 [338:38:49<71:33:05,  5.48s/it]
 79%|███████▉  | 178006/225000 [338:38:56<77:22:01,  5.93s/it]
 79%|███████▉  | 178007/225000 [338:38:59<66:03:21,  5.06s/it]
 79%|███████▉  | 178008/225000 [338:39:07<75:50:58,  5.81s/it]
 79%|███████▉  | 178009/225000 [338:39:10<67:46:25,  5.19s/it]
 79%|███████▉  | 178010/225000 [338:39:14<62:29:10,  4.79s/it]
 79%|███████▉  | 178011/225000 [338:39:18<60:18:38,  4.62s/it]
 79%|███████▉  | 178012/225000 [338:39:23<58:13:19,  4.46s/it]
 79%|███████▉  | 178013/225000 [338:39:32<78:24:38,  6.01s/it]
 79%|███████▉  | 178014/225000 [338:39:37<74:40:45,  5.72s/it]
 79%|███████▉  | 178015/225000 [338:39:41<67:17:21,  5.16s/it]
 79%|███████▉  | 178016/225000 [338:39:45<62:13:39,  4.77s/it]
 79%|███████▉  | 178017/225000 [338:39:55<84:24:07,  6.47s/it]
 79%|███████▉  | 178018/225000 [338:39:59<72:03:24,  5.52s/it]
 79%|███████▉  | 178019/225000 [338:40:07<83:22:57,  6.39s/it]
 79%|███████▉  | 178020/225000 [338:40:11<72:59:53,  5.59s/it]
 79%|███████▉  | 178021/225000 [338:40:15<67:10:39,  5.15s/it]
 79%|███████▉  | 178022/225000 [338:40:24<83:20:12,  6.39s/it]
 79%|███████▉  | 178023/225000 [338:40:27<70:37:55,  5.41s/it]
 79%|███████▉  | 178024/225000 [338:40:32<65:47:23,  5.04s/it]
 79%|███████▉  | 178025/225000 [338:40:35<61:33:32,  4.72s/it]
 79%|███████▉  | 178026/225000 [338:40:40<60:31:18,  4.64s/it]
 79%|███████▉  | 178027/225000 [338:40:51<84:48:39,  6.50s/it]
 79%|███████▉  | 178028/225000 [338:40:54<73:01:49,  5.60s/it]
 79%|███████▉  | 178029/225000 [338:40:58<64:50:01,  4.97s/it]
 79%|███████▉  | 178030/225000 [338:41:08<83:59:46,  6.44s/it]
 79%|███████▉  | 178031/225000 [338:41:11<73:18:29,  5.62s/it]
 79%|███████▉  | 178032/225000 [338:41:15<66:51:30,  5.12s/it]
 79%|███████▉  | 178033/225000 [338:41:19<62:15:47,  4.77s/it]
 79%|███████▉  | 178034/225000 [338:41:29<82:21:56,  6.31s/it]
 79%|███████▉  | 178035/225000 [338:41:33<73:43:04,  5.65s/it]
 79%|███████▉  | 178036/225000 [338:41:37<67:44:20,  5.19s/it]
 79%|███████▉  | 178037/225000 [338:41:41<63:11:43,  4.84s/it]
 79%|███████▉  | 178038/225000 [338:41:46<60:32:14,  4.64s/it]
 79%|███████▉  | 178039/225000 [338:41:55<80:50:57,  6.20s/it]
 79%|███████▉  | 178040/225000 [338:42:00<74:04:43,  5.68s/it]
 79%|███████▉  | 178041/225000 [338:42:04<69:08:45,  5.30s/it]
 79%|███████▉  | 178042/225000 [338:42:09<66:46:18,  5.12s/it]
 79%|███████▉  | 178043/225000 [338:42:13<62:50:35,  4.82s/it]
 79%|███████▉  | 178044/225000 [338:42:18<61:47:21,  4.74s/it]
 79%|███████▉  | 178045/225000 [338:42:25<71:20:21,  5.47s/it]
 79%|███████▉  | 178046/225000 [338:42:33<82:37:41,  6.34s/it]
 79%|███████▉  | 178047/225000 [338:42:37<72:48:52,  5.58s/it]
 79%|███████▉  | 178048/225000 [338:42:41<66:27:28,  5.10s/it]
 79%|███████▉  | 178049/225000 [338:42:50<80:05:53,  6.14s/it]
 79%|███████▉  | 178050/225000 [338:42:54<73:05:18,  5.60s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.065', 'learning_rate': '5.675e-06', 'epoch': '0.7913', 'num_input_tokens_seen': 561477840, 'train_runtime': '1.219e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178050/225000 [338:42:54<73:05:18,  5.60s/it]
 79%|███████▉  | 178051/225000 [338:42:58<68:38:51,  5.26s/it]
 79%|███████▉  | 178052/225000 [338:43:03<65:24:08,  5.02s/it]
 79%|███████▉  | 178053/225000 [338:43:07<63:39:06,  4.88s/it]
 79%|███████▉  | 178054/225000 [338:43:12<62:37:41,  4.80s/it]
 79%|███████▉  | 178055/225000 [338:43:17<61:42:34,  4.73s/it]
 79%|███████▉  | 178056/225000 [338:43:27<83:05:51,  6.37s/it]
 79%|███████▉  | 178057/225000 [338:43:31<76:05:35,  5.84s/it]
 79%|███████▉  | 178058/225000 [338:43:38<78:41:23,  6.03s/it]
 79%|███████▉  | 178059/225000 [338:43:43<73:50:04,  5.66s/it]
 79%|███████▉  | 178060/225000 [338:43:48<74:24:33,  5.71s/it]
 79%|███████▉  | 178061/225000 [338:43:53<71:39:55,  5.50s/it]
 79%|███████▉  | 178062/225000 [338:43:59<70:22:34,  5.40s/it]
 79%|███████▉  | 178063/225000 [338:44:04<69:50:20,  5.36s/it]
 79%|███████▉  | 178064/225000 [338:44:09<67:37:34,  5.19s/it]
 79%|███████▉  | 178065/225000 [338:44:14<67:22:58,  5.17s/it]
 79%|███████▉  | 178066/225000 [338:44:19<66:15:04,  5.08s/it]
 79%|███████▉  | 178067/225000 [338:44:24<66:25:19,  5.09s/it]
 79%|███████▉  | 178068/225000 [338:44:29<67:14:24,  5.16s/it]
 79%|███████▉  | 178069/225000 [338:44:35<68:42:41,  5.27s/it]
 79%|███████▉  | 178070/225000 [338:44:40<68:45:28,  5.27s/it]
 79%|███████▉  | 178071/225000 [338:44:45<67:22:14,  5.17s/it]
 79%|███████▉  | 178072/225000 [338:44:50<67:27:01,  5.17s/it]
 79%|███████▉  | 178073/225000 [338:44:55<65:16:41,  5.01s/it]
 79%|███████▉  | 178074/225000 [338:45:00<65:48:10,  5.05s/it]
 79%|███████▉  | 178075/225000 [338:45:05<66:14:18,  5.08s/it]
 79%|███████▉  | 178076/225000 [338:45:10<67:01:12,  5.14s/it]
 79%|███████▉  | 178077/225000 [338:45:23<95:57:10,  7.36s/it]
 79%|███████▉  | 178078/225000 [338:45:29<90:53:13,  6.97s/it]
 79%|███████▉  | 178079/225000 [338:45:32<77:25:40,  5.94s/it]
 79%|███████▉  | 178080/225000 [338:45:40<82:05:13,  6.30s/it]
 79%|███████▉  | 178081/225000 [338:45:44<73:21:33,  5.63s/it]
 79%|███████▉  | 178082/225000 [338:45:51<80:03:05,  6.14s/it]
 79%|███████▉  | 178083/225000 [338:45:56<76:58:07,  5.91s/it]
 79%|███████▉  | 178084/225000 [338:46:02<74:31:47,  5.72s/it]
 79%|███████▉  | 178085/225000 [338:46:08<77:19:33,  5.93s/it]
 79%|███████▉  | 178086/225000 [338:46:11<66:53:20,  5.13s/it]
 79%|███████▉  | 178087/225000 [338:46:19<77:31:45,  5.95s/it]
 79%|███████▉  | 178088/225000 [338:46:23<68:47:53,  5.28s/it]
 79%|███████▉  | 178089/225000 [338:46:27<62:43:39,  4.81s/it]
 79%|███████▉  | 178090/225000 [338:46:35<76:03:19,  5.84s/it]
 79%|███████▉  | 178091/225000 [338:46:40<73:41:28,  5.66s/it]
 79%|███████▉  | 178092/225000 [338:46:45<72:11:37,  5.54s/it]
 79%|███████▉  | 178093/225000 [338:46:50<67:10:57,  5.16s/it]
 79%|███████▉  | 178094/225000 [338:46:54<64:29:16,  4.95s/it]
 79%|███████▉  | 178095/225000 [338:46:59<62:59:31,  4.83s/it]
 79%|███████▉  | 178096/225000 [338:47:03<60:52:20,  4.67s/it]
 79%|███████▉  | 178097/225000 [338:47:13<84:04:58,  6.45s/it]
 79%|███████▉  | 178098/225000 [338:47:17<73:40:15,  5.65s/it]
 79%|███████▉  | 178099/225000 [338:47:25<80:17:38,  6.16s/it]
 79%|███████▉  | 178100/225000 [338:47:28<68:37:11,  5.27s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.118', 'learning_rate': '5.664e-06', 'epoch': '0.7916', 'num_input_tokens_seen': 561607584, 'train_runtime': '1.22e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178100/225000 [338:47:28<68:37:11,  5.27s/it]
 79%|███████▉  | 178101/225000 [338:47:34<74:05:10,  5.69s/it]
 79%|███████▉  | 178102/225000 [338:47:39<68:53:13,  5.29s/it]
 79%|███████▉  | 178103/225000 [338:47:43<66:12:35,  5.08s/it]
 79%|███████▉  | 178104/225000 [338:47:48<63:02:41,  4.84s/it]
 79%|███████▉  | 178105/225000 [338:47:58<83:10:02,  6.38s/it]
 79%|███████▉  | 178106/225000 [338:48:02<73:38:15,  5.65s/it]
 79%|███████▉  | 178107/225000 [338:48:06<70:13:27,  5.39s/it]
 79%|███████▉  | 178108/225000 [338:48:12<68:59:42,  5.30s/it]
 79%|███████▉  | 178109/225000 [338:48:16<67:45:32,  5.20s/it]
 79%|███████▉  | 178110/225000 [338:48:22<67:58:41,  5.22s/it]
 79%|███████▉  | 178111/225000 [338:48:27<67:57:35,  5.22s/it]
 79%|███████▉  | 178112/225000 [338:48:32<67:31:54,  5.18s/it]
 79%|███████▉  | 178113/225000 [338:48:37<66:45:59,  5.13s/it]
 79%|███████▉  | 178114/225000 [338:48:42<65:35:24,  5.04s/it]
 79%|███████▉  | 178115/225000 [338:48:47<64:58:33,  4.99s/it]
 79%|███████▉  | 178116/225000 [338:48:51<63:49:51,  4.90s/it]
 79%|███████▉  | 178117/225000 [338:48:56<64:22:09,  4.94s/it]
 79%|███████▉  | 178118/225000 [338:49:01<63:35:27,  4.88s/it]
 79%|███████▉  | 178119/225000 [338:49:06<64:47:14,  4.98s/it]
 79%|███████▉  | 178120/225000 [338:49:12<66:06:16,  5.08s/it]
 79%|███████▉  | 178121/225000 [338:49:16<64:45:50,  4.97s/it]
 79%|███████▉  | 178122/225000 [338:49:21<64:21:14,  4.94s/it]
 79%|███████▉  | 178123/225000 [338:49:26<65:03:12,  5.00s/it]
 79%|███████▉  | 178124/225000 [338:49:36<83:13:19,  6.39s/it]
 79%|███████▉  | 178125/225000 [338:49:41<77:33:34,  5.96s/it]
 79%|███████▉  | 178126/225000 [338:49:45<71:11:09,  5.47s/it]
 79%|███████▉  | 178127/225000 [338:49:50<67:50:31,  5.21s/it]
 79%|███████▉  | 178128/225000 [338:49:54<63:39:05,  4.89s/it]
 79%|███████▉  | 178129/225000 [338:50:03<77:41:47,  5.97s/it]
 79%|███████▉  | 178130/225000 [338:50:11<85:47:34,  6.59s/it]
 79%|███████▉  | 178131/225000 [338:50:14<73:40:40,  5.66s/it]
 79%|███████▉  | 178132/225000 [338:50:20<73:18:42,  5.63s/it]
 79%|███████▉  | 178133/225000 [338:50:25<72:06:00,  5.54s/it]
 79%|███████▉  | 178134/225000 [338:50:29<66:15:22,  5.09s/it]
 79%|███████▉  | 178135/225000 [338:50:34<63:44:15,  4.90s/it]
 79%|███████▉  | 178136/225000 [338:50:38<63:36:24,  4.89s/it]
 79%|███████▉  | 178137/225000 [338:50:48<83:01:01,  6.38s/it]
 79%|███████▉  | 178138/225000 [338:50:52<70:56:51,  5.45s/it]
 79%|███████▉  | 178139/225000 [338:50:59<79:41:50,  6.12s/it]
 79%|███████▉  | 178140/225000 [338:51:06<81:55:46,  6.29s/it]
 79%|███████▉  | 178141/225000 [338:51:10<74:19:17,  5.71s/it]
 79%|███████▉  | 178142/225000 [338:51:16<76:12:58,  5.86s/it]
 79%|███████▉  | 178143/225000 [338:51:21<71:00:51,  5.46s/it]
 79%|███████▉  | 178144/225000 [338:51:28<75:16:29,  5.78s/it]
 79%|███████▉  | 178145/225000 [338:51:33<74:36:00,  5.73s/it]
 79%|███████▉  | 178146/225000 [338:51:40<77:46:46,  5.98s/it]
 79%|███████▉  | 178147/225000 [338:51:43<67:59:06,  5.22s/it]
 79%|███████▉  | 178148/225000 [338:51:50<73:56:25,  5.68s/it]
 79%|███████▉  | 178149/225000 [338:51:55<71:06:21,  5.46s/it]
 79%|███████▉  | 178150/225000 [338:51:59<64:27:54,  4.95s/it]
                                                              
{'loss': '0.7634', 'grad_norm': '1.341', 'learning_rate': '5.652e-06', 'epoch': '0.7918', 'num_input_tokens_seen': 561736456, 'train_runtime': '1.22e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178150/225000 [338:51:59<64:27:54,  4.95s/it]
 79%|███████▉  | 178151/225000 [338:52:03<61:00:46,  4.69s/it]
 79%|███████▉  | 178152/225000 [338:52:07<59:35:27,  4.58s/it]
 79%|███████▉  | 178153/225000 [338:52:13<63:50:49,  4.91s/it]
 79%|███████▉  | 178154/225000 [338:52:18<65:40:26,  5.05s/it]
 79%|███████▉  | 178155/225000 [338:52:24<67:24:39,  5.18s/it]
 79%|███████▉  | 178156/225000 [338:52:33<83:03:56,  6.38s/it]
 79%|███████▉  | 178157/225000 [338:52:38<80:08:11,  6.16s/it]
 79%|███████▉  | 178158/225000 [338:52:42<72:10:11,  5.55s/it]
 79%|███████▉  | 178159/225000 [338:52:47<69:30:18,  5.34s/it]
 79%|███████▉  | 178160/225000 [338:52:52<66:48:54,  5.14s/it]
 79%|███████▉  | 178161/225000 [338:52:57<64:51:11,  4.98s/it]
 79%|███████▉  | 178162/225000 [338:53:01<63:45:58,  4.90s/it]
 79%|███████▉  | 178163/225000 [338:53:06<63:00:44,  4.84s/it]
 79%|███████▉  | 178164/225000 [338:53:18<90:32:17,  6.96s/it]
 79%|███████▉  | 178165/225000 [338:53:22<77:20:23,  5.94s/it]
 79%|███████▉  | 178166/225000 [338:53:28<81:14:23,  6.24s/it]
 79%|███████▉  | 178167/225000 [338:53:34<78:24:33,  6.03s/it]
 79%|███████▉  | 178168/225000 [338:53:38<70:35:02,  5.43s/it]
 79%|███████▉  | 178169/225000 [338:53:42<64:30:23,  4.96s/it]
 79%|███████▉  | 178170/225000 [338:53:47<64:32:57,  4.96s/it]
 79%|███████▉  | 178171/225000 [338:53:56<79:09:34,  6.09s/it]
 79%|███████▉  | 178172/225000 [338:53:59<70:00:50,  5.38s/it]
 79%|███████▉  | 178173/225000 [338:54:05<72:43:28,  5.59s/it]
 79%|███████▉  | 178174/225000 [338:54:12<76:24:20,  5.87s/it]
 79%|███████▉  | 178175/225000 [338:54:15<65:47:55,  5.06s/it]
 79%|███████▉  | 178176/225000 [338:54:24<80:01:33,  6.15s/it]
 79%|███████▉  | 178177/225000 [338:54:28<74:01:37,  5.69s/it]
 79%|███████▉  | 178178/225000 [338:54:33<70:11:22,  5.40s/it]
 79%|███████▉  | 178179/225000 [338:54:38<67:01:33,  5.15s/it]
 79%|███████▉  | 178180/225000 [338:54:42<64:12:53,  4.94s/it]
 79%|███████▉  | 178181/225000 [338:54:47<62:12:38,  4.78s/it]
 79%|███████▉  | 178182/225000 [338:54:51<60:41:03,  4.67s/it]
 79%|███████▉  | 178183/225000 [338:55:00<79:26:41,  6.11s/it]
 79%|███████▉  | 178184/225000 [338:55:07<79:33:32,  6.12s/it]
 79%|███████▉  | 178185/225000 [338:55:10<69:16:28,  5.33s/it]
 79%|███████▉  | 178186/225000 [338:55:16<72:23:39,  5.57s/it]
 79%|███████▉  | 178187/225000 [338:55:19<63:41:07,  4.90s/it]
 79%|███████▉  | 178188/225000 [338:55:26<71:52:00,  5.53s/it]
 79%|███████▉  | 178189/225000 [338:55:33<75:25:43,  5.80s/it]
 79%|███████▉  | 178190/225000 [338:55:37<68:08:32,  5.24s/it]
 79%|███████▉  | 178191/225000 [338:55:41<64:52:40,  4.99s/it]
 79%|███████▉  | 178192/225000 [338:55:46<62:53:04,  4.84s/it]
 79%|███████▉  | 178193/225000 [338:55:50<62:12:44,  4.78s/it]
 79%|███████▉  | 178194/225000 [338:55:55<62:33:21,  4.81s/it]
 79%|███████▉  | 178195/225000 [338:56:06<85:41:39,  6.59s/it]
 79%|███████▉  | 178196/225000 [338:56:09<72:51:16,  5.60s/it]
 79%|███████▉  | 178197/225000 [338:56:13<64:48:39,  4.99s/it]
 79%|███████▉  | 178198/225000 [338:56:17<60:43:06,  4.67s/it]
 79%|███████▉  | 178199/225000 [338:56:22<61:23:35,  4.72s/it]
 79%|███████▉  | 178200/225000 [338:56:33<87:20:00,  6.72s/it]
                                                              
{'loss': '0.7458', 'grad_norm': '1.268', 'learning_rate': '5.64e-06', 'epoch': '0.792', 'num_input_tokens_seen': 561865808, 'train_runtime': '1.22e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178200/225000 [338:56:33<87:20:00,  6.72s/it]
 79%|███████▉  | 178201/225000 [338:56:37<77:14:57,  5.94s/it]
 79%|███████▉  | 178202/225000 [338:56:41<70:48:25,  5.45s/it]
 79%|███████▉  | 178203/225000 [338:56:45<65:10:15,  5.01s/it]
 79%|███████▉  | 178204/225000 [338:56:49<61:13:53,  4.71s/it]
 79%|███████▉  | 178205/225000 [338:56:54<59:34:16,  4.58s/it]
 79%|███████▉  | 178206/225000 [338:57:06<89:33:15,  6.89s/it]
 79%|███████▉  | 178207/225000 [338:57:09<75:05:05,  5.78s/it]
 79%|███████▉  | 178208/225000 [338:57:14<72:41:28,  5.59s/it]
 79%|███████▉  | 178209/225000 [338:57:20<73:21:33,  5.64s/it]
 79%|███████▉  | 178210/225000 [338:57:25<70:03:54,  5.39s/it]
 79%|███████▉  | 178211/225000 [338:57:30<67:56:24,  5.23s/it]
 79%|███████▉  | 178212/225000 [338:57:34<65:38:13,  5.05s/it]
 79%|███████▉  | 178213/225000 [338:57:39<64:24:28,  4.96s/it]
 79%|███████▉  | 178214/225000 [338:57:44<62:17:52,  4.79s/it]
 79%|███████▉  | 178215/225000 [338:57:49<63:07:20,  4.86s/it]
 79%|███████▉  | 178216/225000 [338:57:54<63:50:37,  4.91s/it]
 79%|███████▉  | 178217/225000 [338:57:59<64:56:30,  5.00s/it]
 79%|███████▉  | 178218/225000 [338:58:04<63:49:35,  4.91s/it]
 79%|███████▉  | 178219/225000 [338:58:08<63:01:50,  4.85s/it]
 79%|███████▉  | 178220/225000 [338:58:18<83:16:16,  6.41s/it]
 79%|███████▉  | 178221/225000 [338:58:26<86:59:27,  6.69s/it]
 79%|███████▉  | 178222/225000 [338:58:29<75:32:33,  5.81s/it]
 79%|███████▉  | 178223/225000 [338:58:38<87:01:22,  6.70s/it]
 79%|███████▉  | 178224/225000 [338:58:41<73:21:02,  5.65s/it]
 79%|███████▉  | 178225/225000 [338:58:48<78:25:56,  6.04s/it]
 79%|███████▉  | 178226/225000 [338:58:52<68:50:47,  5.30s/it]
 79%|███████▉  | 178227/225000 [338:58:56<63:01:07,  4.85s/it]
 79%|███████▉  | 178228/225000 [338:59:00<59:25:49,  4.57s/it]
 79%|███████▉  | 178229/225000 [338:59:09<78:35:05,  6.05s/it]
 79%|███████▉  | 178230/225000 [338:59:13<69:25:22,  5.34s/it]
 79%|███████▉  | 178231/225000 [338:59:20<77:46:21,  5.99s/it]
 79%|███████▉  | 178232/225000 [338:59:23<67:02:58,  5.16s/it]
 79%|███████▉  | 178233/225000 [338:59:31<75:48:03,  5.83s/it]
 79%|███████▉  | 178234/225000 [338:59:35<70:45:24,  5.45s/it]
 79%|███████▉  | 178235/225000 [338:59:40<67:59:49,  5.23s/it]
 79%|███████▉  | 178236/225000 [338:59:45<66:30:57,  5.12s/it]
 79%|███████▉  | 178237/225000 [338:59:50<65:13:19,  5.02s/it]
 79%|███████▉  | 178238/225000 [338:59:54<63:30:58,  4.89s/it]
 79%|███████▉  | 178239/225000 [338:59:59<63:58:39,  4.93s/it]
 79%|███████▉  | 178240/225000 [339:00:04<63:14:40,  4.87s/it]
 79%|███████▉  | 178241/225000 [339:00:09<64:22:20,  4.96s/it]
 79%|███████▉  | 178242/225000 [339:00:14<64:40:45,  4.98s/it]
 79%|███████▉  | 178243/225000 [339:00:19<64:42:13,  4.98s/it]
 79%|███████▉  | 178244/225000 [339:00:24<61:39:32,  4.75s/it]
 79%|███████▉  | 178245/225000 [339:00:29<63:17:45,  4.87s/it]
 79%|███████▉  | 178246/225000 [339:00:41<93:53:11,  7.23s/it]
 79%|███████▉  | 178247/225000 [339:00:46<83:45:28,  6.45s/it]
 79%|███████▉  | 178248/225000 [339:00:50<73:11:47,  5.64s/it]
 79%|███████▉  | 178249/225000 [339:00:59<86:24:58,  6.65s/it]
 79%|███████▉  | 178250/225000 [339:01:04<82:18:19,  6.34s/it]
                                                              
{'loss': '0.7501', 'grad_norm': '1.138', 'learning_rate': '5.629e-06', 'epoch': '0.7922', 'num_input_tokens_seen': 561995560, 'train_runtime': '1.22e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178250/225000 [339:01:04<82:18:19,  6.34s/it]
 79%|███████▉  | 178251/225000 [339:01:10<78:10:29,  6.02s/it]
 79%|███████▉  | 178252/225000 [339:01:15<73:28:00,  5.66s/it]
 79%|███████▉  | 178253/225000 [339:01:19<69:47:06,  5.37s/it]
 79%|███████▉  | 178254/225000 [339:01:24<68:29:49,  5.28s/it]
 79%|███████▉  | 178255/225000 [339:01:29<67:03:09,  5.16s/it]
 79%|███████▉  | 178256/225000 [339:01:34<64:08:45,  4.94s/it]
 79%|███████▉  | 178257/225000 [339:01:38<63:50:38,  4.92s/it]
 79%|███████▉  | 178258/225000 [339:01:43<63:28:50,  4.89s/it]
 79%|███████▉  | 178259/225000 [339:01:48<63:53:18,  4.92s/it]
 79%|███████▉  | 178260/225000 [339:01:53<64:21:53,  4.96s/it]
 79%|███████▉  | 178261/225000 [339:01:58<64:57:43,  5.00s/it]
 79%|███████▉  | 178262/225000 [339:02:11<94:14:48,  7.26s/it]
 79%|███████▉  | 178263/225000 [339:02:15<80:19:16,  6.19s/it]
 79%|███████▉  | 178264/225000 [339:02:18<69:41:04,  5.37s/it]
 79%|███████▉  | 178265/225000 [339:02:27<84:25:01,  6.50s/it]
 79%|███████▉  | 178266/225000 [339:02:30<71:33:15,  5.51s/it]
 79%|███████▉  | 178267/225000 [339:02:34<65:04:03,  5.01s/it]
 79%|███████▉  | 178268/225000 [339:02:39<62:30:59,  4.82s/it]
 79%|███████▉  | 178269/225000 [339:02:43<59:50:36,  4.61s/it]
 79%|███████▉  | 178270/225000 [339:02:55<90:45:35,  6.99s/it]
 79%|███████▉  | 178271/225000 [339:03:00<82:25:24,  6.35s/it]
 79%|███████▉  | 178272/225000 [339:03:05<76:11:32,  5.87s/it]
 79%|███████▉  | 178273/225000 [339:03:10<71:06:14,  5.48s/it]
 79%|███████▉  | 178274/225000 [339:03:14<66:56:51,  5.16s/it]
 79%|███████▉  | 178275/225000 [339:03:18<63:36:12,  4.90s/it]
 79%|███████▉  | 178276/225000 [339:03:23<62:01:11,  4.78s/it]
 79%|███████▉  | 178277/225000 [339:03:28<63:21:50,  4.88s/it]
 79%|███████▉  | 178278/225000 [339:03:32<61:59:15,  4.78s/it]
 79%|███████▉  | 178279/225000 [339:03:37<62:42:01,  4.83s/it]
 79%|███████▉  | 178280/225000 [339:03:42<63:20:20,  4.88s/it]
 79%|███████▉  | 178281/225000 [339:03:47<63:19:26,  4.88s/it]
 79%|███████▉  | 178282/225000 [339:03:52<64:07:19,  4.94s/it]
 79%|███████▉  | 178283/225000 [339:04:05<92:34:44,  7.13s/it]
 79%|███████▉  | 178284/225000 [339:04:08<79:21:03,  6.11s/it]
 79%|███████▉  | 178285/225000 [339:04:14<78:48:45,  6.07s/it]
 79%|███████▉  | 178286/225000 [339:04:19<75:12:34,  5.80s/it]
 79%|███████▉  | 178287/225000 [339:04:28<86:13:25,  6.64s/it]
 79%|███████▉  | 178288/225000 [339:04:31<73:06:26,  5.63s/it]
 79%|███████▉  | 178289/225000 [339:04:35<65:28:00,  5.05s/it]
 79%|███████▉  | 178290/225000 [339:04:41<70:40:09,  5.45s/it]
 79%|███████▉  | 178291/225000 [339:04:48<74:18:16,  5.73s/it]
 79%|███████▉  | 178292/225000 [339:04:56<85:09:26,  6.56s/it]
 79%|███████▉  | 178293/225000 [339:05:00<73:03:54,  5.63s/it]
 79%|███████▉  | 178294/225000 [339:05:03<65:36:02,  5.06s/it]
 79%|███████▉  | 178295/225000 [339:05:12<78:36:49,  6.06s/it]
 79%|███████▉  | 178296/225000 [339:05:15<68:14:12,  5.26s/it]
 79%|███████▉  | 178297/225000 [339:05:22<72:39:21,  5.60s/it]
 79%|███████▉  | 178298/225000 [339:05:25<63:55:57,  4.93s/it]
 79%|███████▉  | 178299/225000 [339:05:32<73:33:31,  5.67s/it]
 79%|███████▉  | 178300/225000 [339:05:36<65:01:34,  5.01s/it]
                                                              
{'loss': '0.7649', 'grad_norm': '1.22', 'learning_rate': '5.617e-06', 'epoch': '0.7924', 'num_input_tokens_seen': 562125072, 'train_runtime': '1.221e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178300/225000 [339:05:36<65:01:34,  5.01s/it]
 79%|███████▉  | 178301/225000 [339:05:40<60:43:32,  4.68s/it]
 79%|███████▉  | 178302/225000 [339:05:48<76:26:19,  5.89s/it]
 79%|███████▉  | 178303/225000 [339:05:52<67:16:11,  5.19s/it]
 79%|███████▉  | 178304/225000 [339:05:57<68:04:29,  5.25s/it]
 79%|███████▉  | 178305/225000 [339:06:03<68:20:14,  5.27s/it]
 79%|███████▉  | 178306/225000 [339:06:08<67:51:05,  5.23s/it]
 79%|███████▉  | 178307/225000 [339:06:13<67:46:41,  5.23s/it]
 79%|███████▉  | 178308/225000 [339:06:18<65:52:41,  5.08s/it]
 79%|███████▉  | 178309/225000 [339:06:22<63:13:25,  4.87s/it]
 79%|███████▉  | 178310/225000 [339:06:27<63:41:08,  4.91s/it]
 79%|███████▉  | 178311/225000 [339:06:32<64:40:28,  4.99s/it]
 79%|███████▉  | 178312/225000 [339:06:38<65:58:49,  5.09s/it]
 79%|███████▉  | 178313/225000 [339:06:43<68:18:23,  5.27s/it]
 79%|███████▉  | 178314/225000 [339:06:48<67:30:52,  5.21s/it]
 79%|███████▉  | 178315/225000 [339:07:01<96:36:54,  7.45s/it]
 79%|███████▉  | 178316/225000 [339:07:05<84:27:39,  6.51s/it]
 79%|███████▉  | 178317/225000 [339:07:11<82:12:01,  6.34s/it]
 79%|███████▉  | 178318/225000 [339:07:16<73:51:23,  5.70s/it]
 79%|███████▉  | 178319/225000 [339:07:19<65:22:04,  5.04s/it]
 79%|███████▉  | 178320/225000 [339:07:29<82:45:03,  6.38s/it]
 79%|███████▉  | 178321/225000 [339:07:32<71:35:48,  5.52s/it]
 79%|███████▉  | 178322/225000 [339:07:36<64:49:43,  5.00s/it]
 79%|███████▉  | 178323/225000 [339:07:40<60:09:34,  4.64s/it]
 79%|███████▉  | 178324/225000 [339:07:45<62:37:01,  4.83s/it]
 79%|███████▉  | 178325/225000 [339:07:52<72:29:52,  5.59s/it]
 79%|███████▉  | 178326/225000 [339:07:59<78:07:02,  6.03s/it]
 79%|███████▉  | 178327/225000 [339:08:03<69:56:36,  5.39s/it]
 79%|███████▉  | 178328/225000 [339:08:07<64:57:38,  5.01s/it]
 79%|███████▉  | 178329/225000 [339:08:12<61:20:51,  4.73s/it]
 79%|███████▉  | 178330/225000 [339:08:17<65:55:33,  5.09s/it]
 79%|███████▉  | 178331/225000 [339:08:27<83:13:03,  6.42s/it]
 79%|███████▉  | 178332/225000 [339:08:30<72:04:27,  5.56s/it]
 79%|███████▉  | 178333/225000 [339:08:34<63:23:47,  4.89s/it]
 79%|███████▉  | 178334/225000 [339:08:43<78:35:20,  6.06s/it]
 79%|███████▉  | 178335/225000 [339:08:46<68:58:26,  5.32s/it]
 79%|███████▉  | 178336/225000 [339:08:50<64:46:10,  5.00s/it]
 79%|███████▉  | 178337/225000 [339:08:55<62:21:02,  4.81s/it]
 79%|███████▉  | 178338/225000 [339:09:04<80:10:54,  6.19s/it]
 79%|███████▉  | 178339/225000 [339:09:10<77:51:07,  6.01s/it]
 79%|███████▉  | 178340/225000 [339:09:13<68:08:36,  5.26s/it]
 79%|███████▉  | 178341/225000 [339:09:22<81:36:08,  6.30s/it]
 79%|███████▉  | 178342/225000 [339:09:26<71:16:55,  5.50s/it]
 79%|███████▉  | 178343/225000 [339:09:30<66:25:05,  5.12s/it]
 79%|███████▉  | 178344/225000 [339:09:37<74:15:21,  5.73s/it]
 79%|███████▉  | 178345/225000 [339:09:41<67:07:49,  5.18s/it]
 79%|███████▉  | 178346/225000 [339:09:45<64:03:59,  4.94s/it]
 79%|███████▉  | 178347/225000 [339:09:50<62:53:36,  4.85s/it]
 79%|███████▉  | 178348/225000 [339:09:55<62:36:14,  4.83s/it]
 79%|███████▉  | 178349/225000 [339:09:59<60:58:08,  4.70s/it]
 79%|███████▉  | 178350/225000 [339:10:04<60:03:15,  4.63s/it]
                                                              
{'loss': '0.7582', 'grad_norm': '1.305', 'learning_rate': '5.606e-06', 'epoch': '0.7927', 'num_input_tokens_seen': 562254736, 'train_runtime': '1.221e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178350/225000 [339:10:04<60:03:15,  4.63s/it]
 79%|███████▉  | 178351/225000 [339:10:13<77:13:33,  5.96s/it]
 79%|███████▉  | 178352/225000 [339:10:21<86:01:33,  6.64s/it]
 79%|███████▉  | 178353/225000 [339:10:24<73:33:02,  5.68s/it]
 79%|███████▉  | 178354/225000 [339:10:32<80:06:30,  6.18s/it]
 79%|███████▉  | 178355/225000 [339:10:38<79:28:30,  6.13s/it]
 79%|███████▉  | 178356/225000 [339:10:43<74:12:46,  5.73s/it]
 79%|███████▉  | 178357/225000 [339:10:46<65:51:31,  5.08s/it]
 79%|███████▉  | 178358/225000 [339:10:54<76:34:08,  5.91s/it]
 79%|███████▉  | 178359/225000 [339:10:58<68:14:12,  5.27s/it]
 79%|███████▉  | 178360/225000 [339:11:01<61:56:25,  4.78s/it]
 79%|███████▉  | 178361/225000 [339:11:10<76:36:15,  5.91s/it]
 79%|███████▉  | 178362/225000 [339:11:14<67:42:15,  5.23s/it]
 79%|███████▉  | 178363/225000 [339:11:21<77:31:43,  5.98s/it]
 79%|███████▉  | 178364/225000 [339:11:27<75:19:10,  5.81s/it]
 79%|███████▉  | 178365/225000 [339:11:34<81:09:09,  6.26s/it]
 79%|███████▉  | 178366/225000 [339:11:38<70:26:08,  5.44s/it]
 79%|███████▉  | 178367/225000 [339:11:41<63:14:53,  4.88s/it]
 79%|███████▉  | 178368/225000 [339:11:51<81:48:31,  6.32s/it]
 79%|███████▉  | 178369/225000 [339:11:55<71:55:23,  5.55s/it]
 79%|███████▉  | 178370/225000 [339:12:02<80:35:39,  6.22s/it]
 79%|███████▉  | 178371/225000 [339:12:06<71:24:35,  5.51s/it]
 79%|███████▉  | 178372/225000 [339:12:10<63:11:39,  4.88s/it]
 79%|███████▉  | 178373/225000 [339:12:18<76:04:21,  5.87s/it]
 79%|███████▉  | 178374/225000 [339:12:21<66:23:00,  5.13s/it]
 79%|███████▉  | 178375/225000 [339:12:28<74:16:20,  5.73s/it]
 79%|███████▉  | 178376/225000 [339:12:34<72:51:14,  5.63s/it]
 79%|███████▉  | 178377/225000 [339:12:39<70:59:09,  5.48s/it]
 79%|███████▉  | 178378/225000 [339:12:44<71:04:59,  5.49s/it]
 79%|███████▉  | 178379/225000 [339:12:51<76:14:05,  5.89s/it]
 79%|███████▉  | 178380/225000 [339:12:56<70:30:19,  5.44s/it]
 79%|███████▉  | 178381/225000 [339:13:00<65:11:03,  5.03s/it]
 79%|███████▉  | 178382/225000 [339:13:05<67:21:09,  5.20s/it]
 79%|███████▉  | 178383/225000 [339:13:13<78:26:08,  6.06s/it]
 79%|███████▉  | 178384/225000 [339:13:17<67:57:34,  5.25s/it]
 79%|███████▉  | 178385/225000 [339:13:25<80:10:39,  6.19s/it]
 79%|███████▉  | 178386/225000 [339:13:29<73:18:38,  5.66s/it]
 79%|███████▉  | 178387/225000 [339:13:34<67:55:51,  5.25s/it]
 79%|███████▉  | 178388/225000 [339:13:38<64:10:09,  4.96s/it]
 79%|███████▉  | 178389/225000 [339:13:42<61:24:48,  4.74s/it]
 79%|███████▉  | 178390/225000 [339:13:46<59:08:12,  4.57s/it]
 79%|███████▉  | 178391/225000 [339:13:53<65:38:01,  5.07s/it]
 79%|███████▉  | 178392/225000 [339:14:01<78:49:34,  6.09s/it]
 79%|███████▉  | 178393/225000 [339:14:06<74:48:19,  5.78s/it]
 79%|███████▉  | 178394/225000 [339:14:12<73:47:31,  5.70s/it]
 79%|███████▉  | 178395/225000 [339:14:17<70:40:44,  5.46s/it]
 79%|███████▉  | 178396/225000 [339:14:22<70:28:58,  5.44s/it]
 79%|███████▉  | 178397/225000 [339:14:27<68:18:42,  5.28s/it]
 79%|███████▉  | 178398/225000 [339:14:32<66:47:35,  5.16s/it]
 79%|███████▉  | 178399/225000 [339:14:37<65:55:50,  5.09s/it]
 79%|███████▉  | 178400/225000 [339:14:42<67:16:24,  5.20s/it]
                                                              
{'loss': '0.7602', 'grad_norm': '1.035', 'learning_rate': '5.594e-06', 'epoch': '0.7929', 'num_input_tokens_seen': 562385416, 'train_runtime': '1.221e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178400/225000 [339:14:42<67:16:24,  5.20s/it]
 79%|███████▉  | 178401/225000 [339:14:48<67:55:44,  5.25s/it]
 79%|███████▉  | 178402/225000 [339:14:53<67:02:25,  5.18s/it]
 79%|███████▉  | 178403/225000 [339:14:58<66:59:08,  5.18s/it]
 79%|███████▉  | 178404/225000 [339:15:03<67:11:25,  5.19s/it]
 79%|███████▉  | 178405/225000 [339:15:08<67:55:08,  5.25s/it]
 79%|███████▉  | 178406/225000 [339:15:14<68:12:59,  5.27s/it]
 79%|███████▉  | 178407/225000 [339:15:19<68:11:11,  5.27s/it]
 79%|███████▉  | 178408/225000 [339:15:24<68:57:50,  5.33s/it]
 79%|███████▉  | 178409/225000 [339:15:29<67:44:13,  5.23s/it]
 79%|███████▉  | 178410/225000 [339:15:35<68:42:02,  5.31s/it]
 79%|███████▉  | 178411/225000 [339:15:40<67:27:08,  5.21s/it]
 79%|███████▉  | 178412/225000 [339:15:45<66:44:15,  5.16s/it]
 79%|███████▉  | 178413/225000 [339:15:50<66:10:22,  5.11s/it]
 79%|███████▉  | 178414/225000 [339:15:55<66:30:39,  5.14s/it]
 79%|███████▉  | 178415/225000 [339:16:01<67:44:57,  5.24s/it]
 79%|███████▉  | 178416/225000 [339:16:06<68:01:41,  5.26s/it]
 79%|███████▉  | 178417/225000 [339:16:11<67:34:50,  5.22s/it]
 79%|███████▉  | 178418/225000 [339:16:16<66:22:23,  5.13s/it]
 79%|███████▉  | 178419/225000 [339:16:21<66:14:19,  5.12s/it]
 79%|███████▉  | 178420/225000 [339:16:26<66:37:22,  5.15s/it]
 79%|███████▉  | 178421/225000 [339:16:35<79:24:01,  6.14s/it]
 79%|███████▉  | 178422/225000 [339:16:45<96:08:03,  7.43s/it]
 79%|███████▉  | 178423/225000 [339:16:49<82:05:30,  6.34s/it]
 79%|███████▉  | 178424/225000 [339:16:53<72:46:17,  5.62s/it]
 79%|███████▉  | 178425/225000 [339:16:57<66:48:54,  5.16s/it]
 79%|███████▉  | 178426/225000 [339:17:08<88:57:42,  6.88s/it]
 79%|███████▉  | 178427/225000 [339:17:12<77:37:58,  6.00s/it]
 79%|███████▉  | 178428/225000 [339:17:15<67:35:55,  5.23s/it]
 79%|███████▉  | 178429/225000 [339:17:20<66:28:09,  5.14s/it]
 79%|███████▉  | 178430/225000 [339:17:27<74:24:38,  5.75s/it]
 79%|███████▉  | 178431/225000 [339:17:31<64:28:37,  4.98s/it]
 79%|███████▉  | 178432/225000 [339:17:39<76:31:23,  5.92s/it]
 79%|███████▉  | 178433/225000 [339:17:42<66:07:36,  5.11s/it]
 79%|███████▉  | 178434/225000 [339:17:50<77:34:26,  6.00s/it]
 79%|███████▉  | 178435/225000 [339:17:53<67:47:46,  5.24s/it]
 79%|███████▉  | 178436/225000 [339:17:59<69:17:20,  5.36s/it]
 79%|███████▉  | 178437/225000 [339:18:05<72:19:39,  5.59s/it]
 79%|███████▉  | 178438/225000 [339:18:11<72:56:16,  5.64s/it]
 79%|███████▉  | 178439/225000 [339:18:16<71:30:58,  5.53s/it]
 79%|███████▉  | 178440/225000 [339:18:23<77:29:02,  5.99s/it]
 79%|███████▉  | 178441/225000 [339:18:28<72:44:38,  5.62s/it]
 79%|███████▉  | 178442/225000 [339:18:32<67:34:15,  5.22s/it]
 79%|███████▉  | 178443/225000 [339:18:40<78:42:19,  6.09s/it]
 79%|███████▉  | 178444/225000 [339:18:44<69:14:20,  5.35s/it]
 79%|███████▉  | 178445/225000 [339:18:52<78:32:20,  6.07s/it]
 79%|███████▉  | 178446/225000 [339:18:55<67:58:16,  5.26s/it]
 79%|███████▉  | 178447/225000 [339:19:03<77:52:01,  6.02s/it]
 79%|███████▉  | 178448/225000 [339:19:06<67:48:50,  5.24s/it]
 79%|███████▉  | 178449/225000 [339:19:14<75:52:33,  5.87s/it]
 79%|███████▉  | 178450/225000 [339:19:18<69:56:29,  5.41s/it]
                                                              
{'loss': '0.7644', 'grad_norm': '1.063', 'learning_rate': '5.583e-06', 'epoch': '0.7931', 'num_input_tokens_seen': 562515592, 'train_runtime': '1.222e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178450/225000 [339:19:18<69:56:29,  5.41s/it]
 79%|███████▉  | 178451/225000 [339:19:22<62:20:54,  4.82s/it]
 79%|███████▉  | 178452/225000 [339:19:26<60:15:49,  4.66s/it]
 79%|███████▉  | 178453/225000 [339:19:30<57:55:25,  4.48s/it]
 79%|███████▉  | 178454/225000 [339:19:34<57:15:21,  4.43s/it]
 79%|███████▉  | 178455/225000 [339:19:39<57:55:01,  4.48s/it]
 79%|███████▉  | 178456/225000 [339:19:50<82:52:03,  6.41s/it]
 79%|███████▉  | 178457/225000 [339:19:53<70:30:59,  5.45s/it]
 79%|███████▉  | 178458/225000 [339:20:00<78:38:48,  6.08s/it]
 79%|███████▉  | 178459/225000 [339:20:04<68:29:29,  5.30s/it]
 79%|███████▉  | 178460/225000 [339:20:10<71:31:18,  5.53s/it]
 79%|███████▉  | 178461/225000 [339:20:17<75:49:58,  5.87s/it]
 79%|███████▉  | 178462/225000 [339:20:20<66:03:22,  5.11s/it]
 79%|███████▉  | 178463/225000 [339:20:28<77:58:46,  6.03s/it]
 79%|███████▉  | 178464/225000 [339:20:33<71:19:40,  5.52s/it]
 79%|███████▉  | 178465/225000 [339:20:37<65:45:21,  5.09s/it]
 79%|███████▉  | 178466/225000 [339:20:40<60:29:53,  4.68s/it]
 79%|███████▉  | 178467/225000 [339:20:49<76:05:58,  5.89s/it]
 79%|███████▉  | 178468/225000 [339:20:53<67:44:03,  5.24s/it]
 79%|███████▉  | 178469/225000 [339:20:59<71:51:24,  5.56s/it]
 79%|███████▉  | 178470/225000 [339:21:06<76:03:25,  5.88s/it]
 79%|███████▉  | 178471/225000 [339:21:09<67:24:34,  5.22s/it]
 79%|███████▉  | 178472/225000 [339:21:14<66:31:08,  5.15s/it]
 79%|███████▉  | 178473/225000 [339:21:19<66:28:24,  5.14s/it]
 79%|███████▉  | 178474/225000 [339:21:24<64:06:03,  4.96s/it]
 79%|███████▉  | 178475/225000 [339:21:29<66:09:57,  5.12s/it]
 79%|███████▉  | 178476/225000 [339:21:34<64:27:13,  4.99s/it]
 79%|███████▉  | 178477/225000 [339:21:39<64:30:08,  4.99s/it]
 79%|███████▉  | 178478/225000 [339:21:44<64:12:05,  4.97s/it]
 79%|███████▉  | 178479/225000 [339:21:49<63:22:13,  4.90s/it]
 79%|███████▉  | 178480/225000 [339:21:56<70:45:13,  5.48s/it]
 79%|███████▉  | 178481/225000 [339:22:06<87:55:09,  6.80s/it]
 79%|███████▉  | 178482/225000 [339:22:09<74:45:57,  5.79s/it]
 79%|███████▉  | 178483/225000 [339:22:16<81:21:06,  6.30s/it]
 79%|███████▉  | 178484/225000 [339:22:20<71:12:59,  5.51s/it]
 79%|███████▉  | 178485/225000 [339:22:26<71:51:15,  5.56s/it]
 79%|███████▉  | 178486/225000 [339:22:33<76:20:47,  5.91s/it]
 79%|███████▉  | 178487/225000 [339:22:36<66:42:16,  5.16s/it]
 79%|███████▉  | 178488/225000 [339:22:40<62:57:51,  4.87s/it]
 79%|███████▉  | 178489/225000 [339:22:44<59:49:10,  4.63s/it]
 79%|███████▉  | 178490/225000 [339:22:55<82:13:14,  6.36s/it]
 79%|███████▉  | 178491/225000 [339:22:58<70:40:30,  5.47s/it]
 79%|███████▉  | 178492/225000 [339:23:04<73:04:23,  5.66s/it]
 79%|███████▉  | 178493/225000 [339:23:11<77:04:07,  5.97s/it]
 79%|███████▉  | 178494/225000 [339:23:18<83:08:49,  6.44s/it]
 79%|███████▉  | 178495/225000 [339:23:22<70:57:55,  5.49s/it]
 79%|███████▉  | 178496/225000 [339:23:30<82:04:30,  6.35s/it]
 79%|███████▉  | 178497/225000 [339:23:34<71:56:09,  5.57s/it]
 79%|███████▉  | 178498/225000 [339:23:37<64:29:12,  4.99s/it]
 79%|███████▉  | 178499/225000 [339:23:45<75:18:45,  5.83s/it]
 79%|███████▉  | 178500/225000 [339:23:49<67:31:19,  5.23s/it]
                                                              
{'loss': '0.7675', 'grad_norm': '1.337', 'learning_rate': '5.571e-06', 'epoch': '0.7933', 'num_input_tokens_seen': 562645016, 'train_runtime': '1.222e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178500/225000 [339:23:49<67:31:19,  5.23s/it]
 79%|███████▉  | 178501/225000 [339:23:53<61:28:04,  4.76s/it]
 79%|███████▉  | 178502/225000 [339:24:01<77:11:57,  5.98s/it]
 79%|███████▉  | 178503/225000 [339:24:05<66:09:19,  5.12s/it]
 79%|███████▉  | 178504/225000 [339:24:11<71:40:11,  5.55s/it]
 79%|███████▉  | 178505/225000 [339:24:15<64:07:51,  4.97s/it]
 79%|███████▉  | 178506/225000 [339:24:20<64:06:25,  4.96s/it]
 79%|███████▉  | 178507/225000 [339:24:27<74:55:54,  5.80s/it]
 79%|███████▉  | 178508/225000 [339:24:31<66:03:57,  5.12s/it]
 79%|███████▉  | 178509/225000 [339:24:36<64:20:30,  4.98s/it]
 79%|███████▉  | 178510/225000 [339:24:40<63:16:16,  4.90s/it]
 79%|███████▉  | 178511/225000 [339:24:46<64:19:49,  4.98s/it]
 79%|███████▉  | 178512/225000 [339:24:50<64:12:58,  4.97s/it]
 79%|███████▉  | 178513/225000 [339:24:55<63:32:52,  4.92s/it]
 79%|███████▉  | 178514/225000 [339:25:00<63:45:58,  4.94s/it]
 79%|███████▉  | 178515/225000 [339:25:05<64:04:20,  4.96s/it]
 79%|███████▉  | 178516/225000 [339:25:10<63:00:07,  4.88s/it]
 79%|███████▉  | 178517/225000 [339:25:15<63:51:42,  4.95s/it]
 79%|███████▉  | 178518/225000 [339:25:20<64:11:53,  4.97s/it]
 79%|███████▉  | 178519/225000 [339:25:32<90:56:25,  7.04s/it]
 79%|███████▉  | 178520/225000 [339:25:37<84:17:54,  6.53s/it]
 79%|███████▉  | 178521/225000 [339:25:42<79:06:49,  6.13s/it]
 79%|███████▉  | 178522/225000 [339:25:49<80:15:34,  6.22s/it]
 79%|███████▉  | 178523/225000 [339:25:53<70:40:46,  5.47s/it]
 79%|███████▉  | 178524/225000 [339:26:00<77:37:26,  6.01s/it]
 79%|███████▉  | 178525/225000 [339:26:03<67:03:23,  5.19s/it]
 79%|███████▉  | 178526/225000 [339:26:08<65:27:57,  5.07s/it]
 79%|███████▉  | 178527/225000 [339:26:13<66:36:57,  5.16s/it]
 79%|███████▉  | 178528/225000 [339:26:19<67:25:14,  5.22s/it]
 79%|███████▉  | 178529/225000 [339:26:24<69:06:55,  5.35s/it]
 79%|███████▉  | 178530/225000 [339:26:30<69:44:46,  5.40s/it]
 79%|███████▉  | 178531/225000 [339:26:35<70:15:16,  5.44s/it]
 79%|███████▉  | 178532/225000 [339:26:40<68:34:08,  5.31s/it]
 79%|███████▉  | 178533/225000 [339:26:50<85:18:17,  6.61s/it]
 79%|███████▉  | 178534/225000 [339:26:54<76:21:51,  5.92s/it]
 79%|███████▉  | 178535/225000 [339:26:58<69:19:10,  5.37s/it]
 79%|███████▉  | 178536/225000 [339:27:03<66:51:06,  5.18s/it]
 79%|███████▉  | 178537/225000 [339:27:08<64:40:31,  5.01s/it]
 79%|███████▉  | 178538/225000 [339:27:12<62:00:19,  4.80s/it]
 79%|███████▉  | 178539/225000 [339:27:20<72:20:07,  5.60s/it]
 79%|███████▉  | 178540/225000 [339:27:29<86:39:43,  6.72s/it]
 79%|███████▉  | 178541/225000 [339:27:33<74:33:28,  5.78s/it]
 79%|███████▉  | 178542/225000 [339:27:36<67:33:05,  5.23s/it]
 79%|███████▉  | 178543/225000 [339:27:44<78:15:42,  6.06s/it]
 79%|███████▉  | 178544/225000 [339:27:50<76:05:42,  5.90s/it]
 79%|███████▉  | 178545/225000 [339:27:56<75:07:17,  5.82s/it]
 79%|███████▉  | 178546/225000 [339:28:01<72:38:17,  5.63s/it]
 79%|███████▉  | 178547/225000 [339:28:06<69:25:44,  5.38s/it]
 79%|███████▉  | 178548/225000 [339:28:11<69:32:02,  5.39s/it]
 79%|███████▉  | 178549/225000 [339:28:19<80:11:02,  6.21s/it]
 79%|███████▉  | 178550/225000 [339:28:23<70:58:23,  5.50s/it]
                                                              
{'loss': '0.7575', 'grad_norm': '1.183', 'learning_rate': '5.56e-06', 'epoch': '0.7936', 'num_input_tokens_seen': 562775360, 'train_runtime': '1.222e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178550/225000 [339:28:23<70:58:23,  5.50s/it]
 79%|███████▉  | 178551/225000 [339:28:27<63:25:52,  4.92s/it]
 79%|███████▉  | 178552/225000 [339:28:35<77:53:31,  6.04s/it]
 79%|███████▉  | 178553/225000 [339:28:40<73:13:53,  5.68s/it]
 79%|███████▉  | 178554/225000 [339:28:44<65:30:17,  5.08s/it]
 79%|███████▉  | 178555/225000 [339:28:52<78:29:18,  6.08s/it]
 79%|███████▉  | 178556/225000 [339:28:55<66:37:56,  5.16s/it]
 79%|███████▉  | 178557/225000 [339:28:59<61:04:25,  4.73s/it]
 79%|███████▉  | 178558/225000 [339:29:03<57:59:37,  4.50s/it]
 79%|███████▉  | 178559/225000 [339:29:11<72:35:04,  5.63s/it]
 79%|███████▉  | 178560/225000 [339:29:18<77:52:05,  6.04s/it]
 79%|███████▉  | 178561/225000 [339:29:22<68:43:17,  5.33s/it]
 79%|███████▉  | 178562/225000 [339:29:29<75:48:05,  5.88s/it]
 79%|███████▉  | 178563/225000 [339:29:33<68:35:58,  5.32s/it]
 79%|███████▉  | 178564/225000 [339:29:40<73:45:57,  5.72s/it]
 79%|███████▉  | 178565/225000 [339:29:45<71:51:56,  5.57s/it]
 79%|███████▉  | 178566/225000 [339:29:52<78:47:34,  6.11s/it]
 79%|███████▉  | 178567/225000 [339:29:55<67:48:10,  5.26s/it]
 79%|███████▉  | 178568/225000 [339:29:59<61:59:58,  4.81s/it]
 79%|███████▉  | 178569/225000 [339:30:08<77:16:35,  5.99s/it]
 79%|███████▉  | 178570/225000 [339:30:11<66:42:40,  5.17s/it]
 79%|███████▉  | 178571/225000 [339:30:18<73:10:02,  5.67s/it]
 79%|███████▉  | 178572/225000 [339:30:22<65:52:28,  5.11s/it]
 79%|███████▉  | 178573/225000 [339:30:25<59:49:31,  4.64s/it]
 79%|███████▉  | 178574/225000 [339:30:31<64:04:10,  4.97s/it]
 79%|███████▉  | 178575/225000 [339:30:39<75:50:16,  5.88s/it]
 79%|███████▉  | 178576/225000 [339:30:42<65:57:12,  5.11s/it]
 79%|███████▉  | 178577/225000 [339:30:47<64:50:01,  5.03s/it]
 79%|███████▉  | 178578/225000 [339:30:55<73:21:31,  5.69s/it]
 79%|███████▉  | 178579/225000 [339:30:58<66:26:38,  5.15s/it]
 79%|███████▉  | 178580/225000 [339:31:03<62:46:21,  4.87s/it]
 79%|███████▉  | 178581/225000 [339:31:12<80:33:42,  6.25s/it]
 79%|███████▉  | 178582/225000 [339:31:16<69:52:33,  5.42s/it]
 79%|███████▉  | 178583/225000 [339:31:19<63:24:36,  4.92s/it]
 79%|███████▉  | 178584/225000 [339:31:28<78:41:52,  6.10s/it]
 79%|███████▉  | 178585/225000 [339:31:32<68:22:47,  5.30s/it]
 79%|███████▉  | 178586/225000 [339:31:36<62:48:01,  4.87s/it]
 79%|███████▉  | 178587/225000 [339:31:44<78:31:57,  6.09s/it]
 79%|███████▉  | 178588/225000 [339:31:50<74:51:47,  5.81s/it]
 79%|███████▉  | 178589/225000 [339:31:55<71:47:45,  5.57s/it]
 79%|███████▉  | 178590/225000 [339:32:00<70:39:33,  5.48s/it]
 79%|███████▉  | 178591/225000 [339:32:05<68:46:19,  5.33s/it]
 79%|███████▉  | 178592/225000 [339:32:10<66:58:08,  5.19s/it]
 79%|███████▉  | 178593/225000 [339:32:15<65:50:18,  5.11s/it]
 79%|███████▉  | 178594/225000 [339:32:19<64:43:31,  5.02s/it]
 79%|███████▉  | 178595/225000 [339:32:24<64:27:52,  5.00s/it]
 79%|███████▉  | 178596/225000 [339:32:30<64:55:17,  5.04s/it]
 79%|███████▉  | 178597/225000 [339:32:35<65:27:10,  5.08s/it]
 79%|███████▉  | 178598/225000 [339:32:40<66:12:55,  5.14s/it]
 79%|███████▉  | 178599/225000 [339:32:45<65:54:28,  5.11s/it]
 79%|███████▉  | 178600/225000 [339:32:50<64:50:20,  5.03s/it]
                                                              
{'loss': '0.7666', 'grad_norm': '1.304', 'learning_rate': '5.549e-06', 'epoch': '0.7938', 'num_input_tokens_seen': 562905312, 'train_runtime': '1.222e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178600/225000 [339:32:50<64:50:20,  5.03s/it]
 79%|███████▉  | 178601/225000 [339:32:55<65:35:05,  5.09s/it]
 79%|███████▉  | 178602/225000 [339:33:00<66:15:31,  5.14s/it]
 79%|███████▉  | 178603/225000 [339:33:06<67:22:53,  5.23s/it]
 79%|███████▉  | 178604/225000 [339:33:16<87:59:59,  6.83s/it]
 79%|███████▉  | 178605/225000 [339:33:26<98:44:42,  7.66s/it]
 79%|███████▉  | 178606/225000 [339:33:30<84:09:01,  6.53s/it]
 79%|███████▉  | 178607/225000 [339:33:35<79:58:32,  6.21s/it]
 79%|███████▉  | 178608/225000 [339:33:41<77:43:32,  6.03s/it]
 79%|███████▉  | 178609/225000 [339:33:46<75:54:08,  5.89s/it]
 79%|███████▉  | 178610/225000 [339:33:52<73:25:12,  5.70s/it]
 79%|███████▉  | 178611/225000 [339:33:57<72:55:38,  5.66s/it]
 79%|███████▉  | 178612/225000 [339:34:07<86:34:04,  6.72s/it]
 79%|███████▉  | 178613/225000 [339:34:10<75:36:41,  5.87s/it]
 79%|███████▉  | 178614/225000 [339:34:17<77:17:25,  6.00s/it]
 79%|███████▉  | 178615/225000 [339:34:22<76:26:20,  5.93s/it]
 79%|███████▉  | 178616/225000 [339:34:26<67:37:16,  5.25s/it]
 79%|███████▉  | 178617/225000 [339:34:30<64:07:36,  4.98s/it]
 79%|███████▉  | 178618/225000 [339:34:35<62:17:57,  4.84s/it]
 79%|███████▉  | 178619/225000 [339:34:39<60:41:38,  4.71s/it]
 79%|███████▉  | 178620/225000 [339:34:51<87:29:14,  6.79s/it]
 79%|███████▉  | 178621/225000 [339:34:55<78:17:19,  6.08s/it]
 79%|███████▉  | 178622/225000 [339:34:59<69:23:54,  5.39s/it]
 79%|███████▉  | 178623/225000 [339:35:04<65:20:11,  5.07s/it]
 79%|███████▉  | 178624/225000 [339:35:08<62:29:26,  4.85s/it]
 79%|███████▉  | 178625/225000 [339:35:18<83:17:43,  6.47s/it]
 79%|███████▉  | 178626/225000 [339:35:22<73:29:32,  5.71s/it]
 79%|███████▉  | 178627/225000 [339:35:26<68:17:09,  5.30s/it]
 79%|███████▉  | 178628/225000 [339:35:31<63:36:15,  4.94s/it]
 79%|███████▉  | 178629/225000 [339:35:41<86:26:31,  6.71s/it]
 79%|███████▉  | 178630/225000 [339:35:45<74:25:18,  5.78s/it]
 79%|███████▉  | 178631/225000 [339:35:52<79:22:45,  6.16s/it]
 79%|███████▉  | 178632/225000 [339:35:57<73:02:31,  5.67s/it]
 79%|███████▉  | 178633/225000 [339:36:01<67:38:59,  5.25s/it]
 79%|███████▉  | 178634/225000 [339:36:06<65:59:12,  5.12s/it]
 79%|███████▉  | 178635/225000 [339:36:10<64:14:46,  4.99s/it]
 79%|███████▉  | 178636/225000 [339:36:15<62:50:12,  4.88s/it]
 79%|███████▉  | 178637/225000 [339:36:20<61:55:21,  4.81s/it]
 79%|███████▉  | 178638/225000 [339:36:24<61:38:56,  4.79s/it]
 79%|███████▉  | 178639/225000 [339:36:29<61:50:15,  4.80s/it]
 79%|███████▉  | 178640/225000 [339:36:38<78:07:46,  6.07s/it]
 79%|███████▉  | 178641/225000 [339:36:46<83:10:31,  6.46s/it]
 79%|███████▉  | 178642/225000 [339:36:49<72:26:04,  5.63s/it]
 79%|███████▉  | 178643/225000 [339:36:57<80:03:03,  6.22s/it]
 79%|███████▉  | 178644/225000 [339:37:02<74:41:49,  5.80s/it]
 79%|███████▉  | 178645/225000 [339:37:08<76:58:33,  5.98s/it]
 79%|███████▉  | 178646/225000 [339:37:12<67:58:05,  5.28s/it]
 79%|███████▉  | 178647/225000 [339:37:18<72:53:04,  5.66s/it]
 79%|███████▉  | 178648/225000 [339:37:24<71:48:48,  5.58s/it]
 79%|███████▉  | 178649/225000 [339:37:27<63:44:02,  4.95s/it]
 79%|███████▉  | 178650/225000 [339:37:35<76:44:38,  5.96s/it]
                                                              
{'loss': '0.7563', 'grad_norm': '1.137', 'learning_rate': '5.537e-06', 'epoch': '0.794', 'num_input_tokens_seen': 563036200, 'train_runtime': '1.223e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178650/225000 [339:37:35<76:44:38,  5.96s/it]
 79%|███████▉  | 178651/225000 [339:37:39<69:20:07,  5.39s/it]
 79%|███████▉  | 178652/225000 [339:37:44<64:37:46,  5.02s/it]
 79%|███████▉  | 178653/225000 [339:37:48<60:32:31,  4.70s/it]
 79%|███████▉  | 178654/225000 [339:37:51<56:59:58,  4.43s/it]
 79%|███████▉  | 178655/225000 [339:38:00<72:43:29,  5.65s/it]
 79%|███████▉  | 178656/225000 [339:38:07<77:46:39,  6.04s/it]
 79%|███████▉  | 178657/225000 [339:38:11<69:26:30,  5.39s/it]
 79%|███████▉  | 178658/225000 [339:38:17<72:29:49,  5.63s/it]
 79%|███████▉  | 178659/225000 [339:38:21<65:08:35,  5.06s/it]
 79%|███████▉  | 178660/225000 [339:38:26<64:56:26,  5.05s/it]
 79%|███████▉  | 178661/225000 [339:38:35<80:38:43,  6.27s/it]
 79%|███████▉  | 178662/225000 [339:38:39<70:52:45,  5.51s/it]
 79%|███████▉  | 178663/225000 [339:38:46<77:27:30,  6.02s/it]
 79%|███████▉  | 178664/225000 [339:38:49<68:46:40,  5.34s/it]
 79%|███████▉  | 178665/225000 [339:38:53<62:13:58,  4.84s/it]
 79%|███████▉  | 178666/225000 [339:38:57<59:09:55,  4.60s/it]
 79%|███████▉  | 178667/225000 [339:39:08<83:15:23,  6.47s/it]
 79%|███████▉  | 178668/225000 [339:39:12<72:31:15,  5.63s/it]
 79%|███████▉  | 178669/225000 [339:39:17<71:34:55,  5.56s/it]
 79%|███████▉  | 178670/225000 [339:39:23<73:10:09,  5.69s/it]
 79%|███████▉  | 178671/225000 [339:39:28<68:45:27,  5.34s/it]
 79%|███████▉  | 178672/225000 [339:39:33<70:50:53,  5.51s/it]
 79%|███████▉  | 178673/225000 [339:39:38<67:46:00,  5.27s/it]
 79%|███████▉  | 178674/225000 [339:39:46<75:48:24,  5.89s/it]
 79%|███████▉  | 178675/225000 [339:39:50<69:50:50,  5.43s/it]
 79%|███████▉  | 178676/225000 [339:39:54<66:23:14,  5.16s/it]
 79%|███████▉  | 178677/225000 [339:39:59<62:41:26,  4.87s/it]
 79%|███████▉  | 178678/225000 [339:40:03<60:38:58,  4.71s/it]
 79%|███████▉  | 178679/225000 [339:40:07<59:36:28,  4.63s/it]
 79%|███████▉  | 178680/225000 [339:40:16<72:57:55,  5.67s/it]
 79%|███████▉  | 178681/225000 [339:40:23<80:52:22,  6.29s/it]
 79%|███████▉  | 178682/225000 [339:40:27<69:40:29,  5.42s/it]
 79%|███████▉  | 178683/225000 [339:40:34<78:04:26,  6.07s/it]
 79%|███████▉  | 178684/225000 [339:40:40<78:00:07,  6.06s/it]
 79%|███████▉  | 178685/225000 [339:40:43<66:56:04,  5.20s/it]
 79%|███████▉  | 178686/225000 [339:40:52<77:56:26,  6.06s/it]
 79%|███████▉  | 178687/225000 [339:40:54<65:11:22,  5.07s/it]
 79%|███████▉  | 178688/225000 [339:40:59<64:54:16,  5.05s/it]
 79%|███████▉  | 178689/225000 [339:41:04<62:38:18,  4.87s/it]
 79%|███████▉  | 178690/225000 [339:41:08<59:20:22,  4.61s/it]
 79%|███████▉  | 178691/225000 [339:41:12<58:00:34,  4.51s/it]
 79%|███████▉  | 178692/225000 [339:41:23<84:56:21,  6.60s/it]
 79%|███████▉  | 178693/225000 [339:41:27<74:46:56,  5.81s/it]
 79%|███████▉  | 178694/225000 [339:41:31<66:54:51,  5.20s/it]
 79%|███████▉  | 178695/225000 [339:41:35<62:45:47,  4.88s/it]
 79%|███████▉  | 178696/225000 [339:41:41<66:06:31,  5.14s/it]
 79%|███████▉  | 178697/225000 [339:41:50<81:47:09,  6.36s/it]
 79%|███████▉  | 178698/225000 [339:41:55<74:43:27,  5.81s/it]
 79%|███████▉  | 178699/225000 [339:42:00<73:10:14,  5.69s/it]
 79%|███████▉  | 178700/225000 [339:42:06<71:30:04,  5.56s/it]
                                                              
{'loss': '0.7596', 'grad_norm': '1.055', 'learning_rate': '5.526e-06', 'epoch': '0.7942', 'num_input_tokens_seen': 563166624, 'train_runtime': '1.223e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178700/225000 [339:42:06<71:30:04,  5.56s/it]
 79%|███████▉  | 178701/225000 [339:42:10<68:58:25,  5.36s/it]
 79%|███████▉  | 178702/225000 [339:42:16<69:08:12,  5.38s/it]
 79%|███████▉  | 178703/225000 [339:42:20<65:48:39,  5.12s/it]
 79%|███████▉  | 178704/225000 [339:42:25<65:06:47,  5.06s/it]
 79%|███████▉  | 178705/225000 [339:42:30<65:21:38,  5.08s/it]
 79%|███████▉  | 178706/225000 [339:42:35<63:58:54,  4.98s/it]
 79%|███████▉  | 178707/225000 [339:42:40<64:31:07,  5.02s/it]
 79%|███████▉  | 178708/225000 [339:42:45<63:36:51,  4.95s/it]
 79%|███████▉  | 178709/225000 [339:42:50<64:04:02,  4.98s/it]
 79%|███████▉  | 178710/225000 [339:42:55<63:49:46,  4.96s/it]
 79%|███████▉  | 178711/225000 [339:43:00<62:37:05,  4.87s/it]
 79%|███████▉  | 178712/225000 [339:43:12<92:30:00,  7.19s/it]
 79%|███████▉  | 178713/225000 [339:43:19<89:12:09,  6.94s/it]
 79%|███████▉  | 178714/225000 [339:43:23<77:54:43,  6.06s/it]
 79%|███████▉  | 178715/225000 [339:43:29<79:09:49,  6.16s/it]
 79%|███████▉  | 178716/225000 [339:43:33<71:07:45,  5.53s/it]
 79%|███████▉  | 178717/225000 [339:43:37<64:40:28,  5.03s/it]
 79%|███████▉  | 178718/225000 [339:43:41<61:30:34,  4.78s/it]
 79%|███████▉  | 178719/225000 [339:43:52<85:06:59,  6.62s/it]
 79%|███████▉  | 178720/225000 [339:43:56<73:13:01,  5.70s/it]
 79%|███████▉  | 178721/225000 [339:43:59<65:16:47,  5.08s/it]
 79%|███████▉  | 178722/225000 [339:44:10<88:40:58,  6.90s/it]
 79%|███████▉  | 178723/225000 [339:44:14<74:53:21,  5.83s/it]
 79%|███████▉  | 178724/225000 [339:44:18<69:41:55,  5.42s/it]
 79%|███████▉  | 178725/225000 [339:44:26<77:55:36,  6.06s/it]
 79%|███████▉  | 178726/225000 [339:44:30<69:51:56,  5.44s/it]
 79%|███████▉  | 178727/225000 [339:44:34<66:44:26,  5.19s/it]
 79%|███████▉  | 178728/225000 [339:44:39<63:51:57,  4.97s/it]
 79%|███████▉  | 178729/225000 [339:44:44<63:35:55,  4.95s/it]
 79%|███████▉  | 178730/225000 [339:44:48<63:00:41,  4.90s/it]
 79%|███████▉  | 178731/225000 [339:44:53<62:11:36,  4.84s/it]
 79%|███████▉  | 178732/225000 [339:44:58<61:34:00,  4.79s/it]
 79%|███████▉  | 178733/225000 [339:45:08<82:20:21,  6.41s/it]
 79%|███████▉  | 178734/225000 [339:45:14<82:03:24,  6.38s/it]
 79%|███████▉  | 178735/225000 [339:45:21<83:54:31,  6.53s/it]
 79%|███████▉  | 178736/225000 [339:45:26<75:51:00,  5.90s/it]
 79%|███████▉  | 178737/225000 [339:45:30<69:26:41,  5.40s/it]
 79%|███████▉  | 178738/225000 [339:45:34<65:33:27,  5.10s/it]
 79%|███████▉  | 178739/225000 [339:45:38<61:55:29,  4.82s/it]
 79%|███████▉  | 178740/225000 [339:45:42<58:05:07,  4.52s/it]
 79%|███████▉  | 178741/225000 [339:45:53<81:14:23,  6.32s/it]
 79%|███████▉  | 178742/225000 [339:45:56<70:10:41,  5.46s/it]
 79%|███████▉  | 178743/225000 [339:46:01<65:47:50,  5.12s/it]
 79%|███████▉  | 178744/225000 [339:46:05<63:06:10,  4.91s/it]
 79%|███████▉  | 178745/225000 [339:46:10<62:07:11,  4.83s/it]
 79%|███████▉  | 178746/225000 [339:46:14<61:17:09,  4.77s/it]
 79%|███████▉  | 178747/225000 [339:46:20<66:00:41,  5.14s/it]
 79%|███████▉  | 178748/225000 [339:46:32<89:53:31,  7.00s/it]
 79%|███████▉  | 178749/225000 [339:46:35<77:33:04,  6.04s/it]
 79%|███████▉  | 178750/225000 [339:46:40<70:31:09,  5.49s/it]
                                                              
{'loss': '0.7621', 'grad_norm': '1.42', 'learning_rate': '5.514e-06', 'epoch': '0.7944', 'num_input_tokens_seen': 563297120, 'train_runtime': '1.223e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178750/225000 [339:46:40<70:31:09,  5.49s/it]
 79%|███████▉  | 178751/225000 [339:46:50<88:32:23,  6.89s/it]
 79%|███████▉  | 178752/225000 [339:46:53<74:32:30,  5.80s/it]
 79%|███████▉  | 178753/225000 [339:47:01<81:36:47,  6.35s/it]
 79%|███████▉  | 178754/225000 [339:47:05<72:22:16,  5.63s/it]
 79%|███████▉  | 178755/225000 [339:47:09<66:59:31,  5.22s/it]
 79%|███████▉  | 178756/225000 [339:47:12<60:08:42,  4.68s/it]
 79%|███████▉  | 178757/225000 [339:47:17<58:19:46,  4.54s/it]
 79%|███████▉  | 178758/225000 [339:47:26<75:43:26,  5.90s/it]
 79%|███████▉  | 178759/225000 [339:47:29<66:50:45,  5.20s/it]
 79%|███████▉  | 178760/225000 [339:47:37<77:38:57,  6.05s/it]
 79%|███████▉  | 178761/225000 [339:47:42<72:36:50,  5.65s/it]
 79%|███████▉  | 178762/225000 [339:47:47<70:44:02,  5.51s/it]
 79%|███████▉  | 178763/225000 [339:47:52<68:19:28,  5.32s/it]
 79%|███████▉  | 178764/225000 [339:47:57<67:00:25,  5.22s/it]
 79%|███████▉  | 178765/225000 [339:48:02<64:56:43,  5.06s/it]
 79%|███████▉  | 178766/225000 [339:48:07<65:00:11,  5.06s/it]
 79%|███████▉  | 178767/225000 [339:48:11<63:52:47,  4.97s/it]
 79%|███████▉  | 178768/225000 [339:48:16<63:12:53,  4.92s/it]
 79%|███████▉  | 178769/225000 [339:48:21<62:52:50,  4.90s/it]
 79%|███████▉  | 178770/225000 [339:48:26<64:24:45,  5.02s/it]
 79%|███████▉  | 178771/225000 [339:48:32<64:48:24,  5.05s/it]
 79%|███████▉  | 178772/225000 [339:48:36<63:27:12,  4.94s/it]
 79%|███████▉  | 178773/225000 [339:48:47<87:40:05,  6.83s/it]
 79%|███████▉  | 178774/225000 [339:48:51<76:52:52,  5.99s/it]
 79%|███████▉  | 178775/225000 [339:48:56<69:56:57,  5.45s/it]
 79%|███████▉  | 178776/225000 [339:49:01<70:02:47,  5.46s/it]
 79%|███████▉  | 178777/225000 [339:49:12<89:18:49,  6.96s/it]
 79%|███████▉  | 178778/225000 [339:49:18<85:23:40,  6.65s/it]
 79%|███████▉  | 178779/225000 [339:49:22<75:23:39,  5.87s/it]
 79%|███████▉  | 178780/225000 [339:49:26<69:13:16,  5.39s/it]
 79%|███████▉  | 178781/225000 [339:49:32<73:46:03,  5.75s/it]
 79%|███████▉  | 178782/225000 [339:49:41<83:36:48,  6.51s/it]
 79%|███████▉  | 178783/225000 [339:49:45<73:49:26,  5.75s/it]
 79%|███████▉  | 178784/225000 [339:49:49<69:23:33,  5.41s/it]
 79%|███████▉  | 178785/225000 [339:49:57<78:05:51,  6.08s/it]
 79%|███████▉  | 178786/225000 [339:50:00<67:54:28,  5.29s/it]
 79%|███████▉  | 178787/225000 [339:50:06<67:25:08,  5.25s/it]
 79%|███████▉  | 178788/225000 [339:50:11<67:07:51,  5.23s/it]
 79%|███████▉  | 178789/225000 [339:50:16<68:27:12,  5.33s/it]
 79%|███████▉  | 178790/225000 [339:50:22<69:09:30,  5.39s/it]
 79%|███████▉  | 178791/225000 [339:50:27<69:13:40,  5.39s/it]
 79%|███████▉  | 178792/225000 [339:50:33<69:20:36,  5.40s/it]
 79%|███████▉  | 178793/225000 [339:50:38<68:28:12,  5.33s/it]
 79%|███████▉  | 178794/225000 [339:50:43<68:21:21,  5.33s/it]
 79%|███████▉  | 178795/225000 [339:50:49<69:28:13,  5.41s/it]
 79%|███████▉  | 178796/225000 [339:50:54<69:13:58,  5.39s/it]
 79%|███████▉  | 178797/225000 [339:50:59<66:30:47,  5.18s/it]
 79%|███████▉  | 178798/225000 [339:51:04<66:26:38,  5.18s/it]
 79%|███████▉  | 178799/225000 [339:51:10<70:53:35,  5.52s/it]
 79%|███████▉  | 178800/225000 [339:51:16<71:52:31,  5.60s/it]
                                                              
{'loss': '0.7618', 'grad_norm': '1.335', 'learning_rate': '5.503e-06', 'epoch': '0.7947', 'num_input_tokens_seen': 563427696, 'train_runtime': '1.223e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178800/225000 [339:51:16<71:52:31,  5.60s/it]
 79%|███████▉  | 178801/225000 [339:51:22<73:05:00,  5.69s/it]
 79%|███████▉  | 178802/225000 [339:51:30<81:31:50,  6.35s/it]
 79%|███████▉  | 178803/225000 [339:51:34<74:06:00,  5.77s/it]
 79%|███████▉  | 178804/225000 [339:51:39<68:41:23,  5.35s/it]
 79%|███████▉  | 178805/225000 [339:51:43<63:50:28,  4.98s/it]
 79%|███████▉  | 178806/225000 [339:51:47<60:55:10,  4.75s/it]
 79%|███████▉  | 178807/225000 [339:51:56<79:11:52,  6.17s/it]
 79%|███████▉  | 178808/225000 [339:52:04<82:30:26,  6.43s/it]
 79%|███████▉  | 178809/225000 [339:52:07<71:47:15,  5.59s/it]
 79%|███████▉  | 178810/225000 [339:52:12<70:16:13,  5.48s/it]
 79%|███████▉  | 178811/225000 [339:52:20<78:09:30,  6.09s/it]
 79%|███████▉  | 178812/225000 [339:52:23<67:26:36,  5.26s/it]
 79%|███████▉  | 178813/225000 [339:52:27<63:13:28,  4.93s/it]
 79%|███████▉  | 178814/225000 [339:52:36<78:23:42,  6.11s/it]
 79%|███████▉  | 178815/225000 [339:52:40<69:10:56,  5.39s/it]
 79%|███████▉  | 178816/225000 [339:52:46<70:04:39,  5.46s/it]
 79%|███████▉  | 178817/225000 [339:52:52<72:44:56,  5.67s/it]
 79%|███████▉  | 178818/225000 [339:52:56<66:22:37,  5.17s/it]
 79%|███████▉  | 178819/225000 [339:53:01<66:30:32,  5.18s/it]
 79%|███████▉  | 178820/225000 [339:53:06<66:00:30,  5.15s/it]
 79%|███████▉  | 178821/225000 [339:53:11<66:26:56,  5.18s/it]
 79%|███████▉  | 178822/225000 [339:53:17<66:37:02,  5.19s/it]
 79%|███████▉  | 178823/225000 [339:53:21<65:35:03,  5.11s/it]
 79%|███████▉  | 178824/225000 [339:53:26<64:18:34,  5.01s/it]
 79%|███████▉  | 178825/225000 [339:53:33<70:35:33,  5.50s/it]
 79%|███████▉  | 178826/225000 [339:53:41<81:11:36,  6.33s/it]
 79%|███████▉  | 178827/225000 [339:53:44<68:58:29,  5.38s/it]
 79%|███████▉  | 178828/225000 [339:53:52<77:55:49,  6.08s/it]
 79%|███████▉  | 178829/225000 [339:53:56<70:21:49,  5.49s/it]
 79%|███████▉  | 178830/225000 [339:54:03<75:33:03,  5.89s/it]
 79%|███████▉  | 178831/225000 [339:54:07<67:26:58,  5.26s/it]
 79%|███████▉  | 178832/225000 [339:54:14<76:57:23,  6.00s/it]
 79%|███████▉  | 178833/225000 [339:54:19<72:26:21,  5.65s/it]
 79%|███████▉  | 178834/225000 [339:54:23<65:35:50,  5.12s/it]
 79%|███████▉  | 178835/225000 [339:54:27<59:59:03,  4.68s/it]
 79%|███████▉  | 178836/225000 [339:54:37<82:36:35,  6.44s/it]
 79%|███████▉  | 178837/225000 [339:54:41<70:53:06,  5.53s/it]
 79%|███████▉  | 178838/225000 [339:54:46<69:49:16,  5.45s/it]
 79%|███████▉  | 178839/225000 [339:54:51<68:47:19,  5.36s/it]
 79%|███████▉  | 178840/225000 [339:54:56<68:36:18,  5.35s/it]
 79%|███████▉  | 178841/225000 [339:55:01<67:01:32,  5.23s/it]
 79%|███████▉  | 178842/225000 [339:55:07<68:12:45,  5.32s/it]
 79%|███████▉  | 178843/225000 [339:55:12<67:33:45,  5.27s/it]
 79%|███████▉  | 178844/225000 [339:55:18<68:11:05,  5.32s/it]
 79%|███████▉  | 178845/225000 [339:55:23<67:09:03,  5.24s/it]
 79%|███████▉  | 178846/225000 [339:55:28<68:16:51,  5.33s/it]
 79%|███████▉  | 178847/225000 [339:55:33<67:41:13,  5.28s/it]
 79%|███████▉  | 178848/225000 [339:55:39<68:42:05,  5.36s/it]
 79%|███████▉  | 178849/225000 [339:55:44<68:24:20,  5.34s/it]
 79%|███████▉  | 178850/225000 [339:55:50<69:18:48,  5.41s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.081', 'learning_rate': '5.491e-06', 'epoch': '0.7949', 'num_input_tokens_seen': 563557456, 'train_runtime': '1.224e+06', 'train_tokens_per_second': '460.5'}

 79%|███████▉  | 178850/225000 [339:55:50<69:18:48,  5.41s/it]
 79%|███████▉  | 178851/225000 [339:55:55<69:53:00,  5.45s/it]
 79%|███████▉  | 178852/225000 [339:56:01<70:05:47,  5.47s/it]
 79%|███████▉  | 178853/225000 [339:56:06<69:31:27,  5.42s/it]
 79%|███████▉  | 178854/225000 [339:56:12<69:53:51,  5.45s/it]
 79%|███████▉  | 178855/225000 [339:56:17<70:49:48,  5.53s/it]
 79%|███████▉  | 178856/225000 [339:56:23<71:27:05,  5.57s/it]
 79%|███████▉  | 178857/225000 [339:56:28<70:30:55,  5.50s/it]
 79%|███████▉  | 178858/225000 [339:56:33<67:18:31,  5.25s/it]
 79%|███████▉  | 178859/225000 [339:56:38<66:55:33,  5.22s/it]
 79%|███████▉  | 178860/225000 [339:56:43<65:34:02,  5.12s/it]
 79%|███████▉  | 178861/225000 [339:56:49<67:12:17,  5.24s/it]
 79%|███████▉  | 178862/225000 [339:56:56<75:13:51,  5.87s/it]
 79%|███████▉  | 178863/225000 [339:57:01<73:39:08,  5.75s/it]
 79%|███████▉  | 178864/225000 [339:57:06<71:12:05,  5.56s/it]
 79%|███████▉  | 178865/225000 [339:57:12<71:27:16,  5.58s/it]
 79%|███████▉  | 178866/225000 [339:57:17<69:55:45,  5.46s/it]
 79%|███████▉  | 178867/225000 [339:57:22<67:10:31,  5.24s/it]
 79%|███████▉  | 178868/225000 [339:57:27<67:14:22,  5.25s/it]
 79%|███████▉  | 178869/225000 [339:57:32<66:58:53,  5.23s/it]
 79%|███████▉  | 178870/225000 [339:57:39<72:15:45,  5.64s/it]
 79%|███████▉  | 178871/225000 [339:57:47<79:30:03,  6.20s/it]
 79%|███████▉  | 178872/225000 [339:57:50<68:26:47,  5.34s/it]
 79%|███████▉  | 178873/225000 [339:57:59<81:50:33,  6.39s/it]
 79%|███████▉  | 178874/225000 [339:58:03<73:59:33,  5.77s/it]
 80%|███████▉  | 178875/225000 [339:58:09<73:38:53,  5.75s/it]
 80%|███████▉  | 178876/225000 [339:58:13<69:07:31,  5.40s/it]
 80%|███████▉  | 178877/225000 [339:58:18<68:02:05,  5.31s/it]
 80%|███████▉  | 178878/225000 [339:58:24<68:33:47,  5.35s/it]
 80%|███████▉  | 178879/225000 [339:58:29<66:14:27,  5.17s/it]
 80%|███████▉  | 178880/225000 [339:58:34<66:27:13,  5.19s/it]
 80%|███████▉  | 178881/225000 [339:58:39<65:15:21,  5.09s/it]
 80%|███████▉  | 178882/225000 [339:58:44<66:17:54,  5.18s/it]
 80%|███████▉  | 178883/225000 [339:58:49<66:19:24,  5.18s/it]
 80%|███████▉  | 178884/225000 [339:58:55<67:00:44,  5.23s/it]
 80%|███████▉  | 178885/225000 [339:59:00<65:45:40,  5.13s/it]
 80%|███████▉  | 178886/225000 [339:59:05<67:24:06,  5.26s/it]
 80%|███████▉  | 178887/225000 [339:59:10<66:49:41,  5.22s/it]
 80%|███████▉  | 178888/225000 [339:59:15<66:40:37,  5.21s/it]
 80%|███████▉  | 178889/225000 [339:59:21<66:50:28,  5.22s/it]
 80%|███████▉  | 178890/225000 [339:59:25<64:18:30,  5.02s/it]
 80%|███████▉  | 178891/225000 [339:59:30<65:15:51,  5.10s/it]
 80%|███████▉  | 178892/225000 [339:59:36<68:24:44,  5.34s/it]
 80%|███████▉  | 178893/225000 [339:59:49<94:33:08,  7.38s/it]
 80%|███████▉  | 178894/225000 [339:59:53<82:38:42,  6.45s/it]
 80%|███████▉  | 178895/225000 [339:59:57<75:43:05,  5.91s/it]
 80%|███████▉  | 178896/225000 [340:00:03<73:49:12,  5.76s/it]
 80%|███████▉  | 178897/225000 [340:00:06<65:00:36,  5.08s/it]
 80%|███████▉  | 178898/225000 [340:00:16<81:30:39,  6.37s/it]
 80%|███████▉  | 178899/225000 [340:00:20<75:13:23,  5.87s/it]
 80%|███████▉  | 178900/225000 [340:00:25<68:48:42,  5.37s/it]
                                                              
{'loss': '0.7677', 'grad_norm': '1.283', 'learning_rate': '5.48e-06', 'epoch': '0.7951', 'num_input_tokens_seen': 563687048, 'train_runtime': '1.224e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 178900/225000 [340:00:25<68:48:42,  5.37s/it]
 80%|███████▉  | 178901/225000 [340:00:29<64:58:38,  5.07s/it]
 80%|███████▉  | 178902/225000 [340:00:34<63:23:36,  4.95s/it]
 80%|███████▉  | 178903/225000 [340:00:38<62:00:22,  4.84s/it]
 80%|███████▉  | 178904/225000 [340:00:43<60:42:55,  4.74s/it]
 80%|███████▉  | 178905/225000 [340:00:54<86:37:06,  6.76s/it]
 80%|███████▉  | 178906/225000 [340:01:00<81:52:50,  6.39s/it]
 80%|███████▉  | 178907/225000 [340:01:05<76:17:51,  5.96s/it]
 80%|███████▉  | 178908/225000 [340:01:10<74:40:22,  5.83s/it]
 80%|███████▉  | 178909/225000 [340:01:16<75:53:52,  5.93s/it]
 80%|███████▉  | 178910/225000 [340:01:21<68:45:53,  5.37s/it]
 80%|███████▉  | 178911/225000 [340:01:28<75:12:36,  5.87s/it]
 80%|███████▉  | 178912/225000 [340:01:31<65:08:10,  5.09s/it]
 80%|███████▉  | 178913/225000 [340:01:39<75:06:34,  5.87s/it]
 80%|███████▉  | 178914/225000 [340:01:42<65:26:48,  5.11s/it]
 80%|███████▉  | 178915/225000 [340:01:49<72:04:33,  5.63s/it]
 80%|███████▉  | 178916/225000 [340:01:54<70:10:22,  5.48s/it]
 80%|███████▉  | 178917/225000 [340:01:58<63:37:47,  4.97s/it]
 80%|███████▉  | 178918/225000 [340:02:01<59:18:35,  4.63s/it]
 80%|███████▉  | 178919/225000 [340:02:06<59:16:10,  4.63s/it]
 80%|███████▉  | 178920/225000 [340:02:10<57:59:50,  4.53s/it]
 80%|███████▉  | 178921/225000 [340:02:14<56:05:26,  4.38s/it]
 80%|███████▉  | 178922/225000 [340:02:23<72:25:18,  5.66s/it]
 80%|███████▉  | 178923/225000 [340:02:32<84:51:05,  6.63s/it]
 80%|███████▉  | 178924/225000 [340:02:36<73:51:45,  5.77s/it]
 80%|███████▉  | 178925/225000 [340:02:44<82:09:03,  6.42s/it]
 80%|███████▉  | 178926/225000 [340:02:48<73:56:08,  5.78s/it]
 80%|███████▉  | 178927/225000 [340:02:56<82:03:49,  6.41s/it]
 80%|███████▉  | 178928/225000 [340:02:59<69:44:32,  5.45s/it]
 80%|███████▉  | 178929/225000 [340:03:05<73:35:47,  5.75s/it]
 80%|███████▉  | 178930/225000 [340:03:11<71:34:18,  5.59s/it]
 80%|███████▉  | 178931/225000 [340:03:16<71:52:56,  5.62s/it]
 80%|███████▉  | 178932/225000 [340:03:22<71:21:46,  5.58s/it]
 80%|███████▉  | 178933/225000 [340:03:27<71:09:41,  5.56s/it]
 80%|███████▉  | 178934/225000 [340:03:33<69:57:21,  5.47s/it]
 80%|███████▉  | 178935/225000 [340:03:38<69:35:10,  5.44s/it]
 80%|███████▉  | 178936/225000 [340:03:43<68:42:49,  5.37s/it]
 80%|███████▉  | 178937/225000 [340:03:48<68:10:11,  5.33s/it]
 80%|███████▉  | 178938/225000 [340:03:54<68:49:37,  5.38s/it]
 80%|███████▉  | 178939/225000 [340:03:59<67:03:32,  5.24s/it]
 80%|███████▉  | 178940/225000 [340:04:07<77:06:34,  6.03s/it]
 80%|███████▉  | 178941/225000 [340:04:12<75:35:03,  5.91s/it]
 80%|███████▉  | 178942/225000 [340:04:17<72:36:39,  5.68s/it]
 80%|███████▉  | 178943/225000 [340:04:22<69:54:33,  5.46s/it]
 80%|███████▉  | 178944/225000 [340:04:27<67:26:53,  5.27s/it]
 80%|███████▉  | 178945/225000 [340:04:33<67:55:13,  5.31s/it]
 80%|███████▉  | 178946/225000 [340:04:41<77:39:09,  6.07s/it]
 80%|███████▉  | 178947/225000 [340:04:46<75:10:44,  5.88s/it]
 80%|███████▉  | 178948/225000 [340:04:50<68:07:54,  5.33s/it]
 80%|███████▉  | 178949/225000 [340:04:54<64:48:33,  5.07s/it]
 80%|███████▉  | 178950/225000 [340:04:59<62:41:08,  4.90s/it]
                                                              
{'loss': '0.7586', 'grad_norm': '1.154', 'learning_rate': '5.468e-06', 'epoch': '0.7953', 'num_input_tokens_seen': 563817376, 'train_runtime': '1.224e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 178950/225000 [340:04:59<62:41:08,  4.90s/it]
 80%|███████▉  | 178951/225000 [340:05:03<59:43:54,  4.67s/it]
 80%|███████▉  | 178952/225000 [340:05:14<84:34:42,  6.61s/it]
 80%|███████▉  | 178953/225000 [340:05:18<74:59:23,  5.86s/it]
 80%|███████▉  | 178954/225000 [340:05:23<69:13:20,  5.41s/it]
 80%|███████▉  | 178955/225000 [340:05:27<64:13:12,  5.02s/it]
 80%|███████▉  | 178956/225000 [340:05:37<84:20:05,  6.59s/it]
 80%|███████▉  | 178957/225000 [340:05:41<72:10:12,  5.64s/it]
 80%|███████▉  | 178958/225000 [340:05:49<82:00:31,  6.41s/it]
 80%|███████▉  | 178959/225000 [340:05:52<71:49:16,  5.62s/it]
 80%|███████▉  | 178960/225000 [340:05:56<64:12:19,  5.02s/it]
 80%|███████▉  | 178961/225000 [340:06:05<79:44:19,  6.24s/it]
 80%|███████▉  | 178962/225000 [340:06:09<69:51:05,  5.46s/it]
 80%|███████▉  | 178963/225000 [340:06:15<70:51:50,  5.54s/it]
 80%|███████▉  | 178964/225000 [340:06:20<70:16:14,  5.50s/it]
 80%|███████▉  | 178965/225000 [340:06:25<66:47:01,  5.22s/it]
 80%|███████▉  | 178966/225000 [340:06:32<74:51:26,  5.85s/it]
 80%|███████▉  | 178967/225000 [340:06:36<68:39:18,  5.37s/it]
 80%|███████▉  | 178968/225000 [340:06:40<64:43:34,  5.06s/it]
 80%|███████▉  | 178969/225000 [340:06:44<60:43:18,  4.75s/it]
 80%|███████▉  | 178970/225000 [340:06:54<80:50:20,  6.32s/it]
 80%|███████▉  | 178971/225000 [340:06:59<74:42:04,  5.84s/it]
 80%|███████▉  | 178972/225000 [340:07:04<72:25:25,  5.66s/it]
 80%|███████▉  | 178973/225000 [340:07:09<69:15:49,  5.42s/it]
 80%|███████▉  | 178974/225000 [340:07:14<68:20:58,  5.35s/it]
 80%|███████▉  | 178975/225000 [340:07:19<66:48:39,  5.23s/it]
 80%|███████▉  | 178976/225000 [340:07:24<65:05:25,  5.09s/it]
 80%|███████▉  | 178977/225000 [340:07:29<65:16:46,  5.11s/it]
 80%|███████▉  | 178978/225000 [340:07:35<66:19:16,  5.19s/it]
 80%|███████▉  | 178979/225000 [340:07:40<65:56:59,  5.16s/it]
 80%|███████▉  | 178980/225000 [340:07:45<65:20:10,  5.11s/it]
 80%|███████▉  | 178981/225000 [340:07:50<64:05:53,  5.01s/it]
 80%|███████▉  | 178982/225000 [340:07:55<65:27:44,  5.12s/it]
 80%|███████▉  | 178983/225000 [340:08:00<64:27:33,  5.04s/it]
 80%|███████▉  | 178984/225000 [340:08:05<64:40:46,  5.06s/it]
 80%|███████▉  | 178985/225000 [340:08:10<64:06:14,  5.02s/it]
 80%|███████▉  | 178986/225000 [340:08:15<63:05:54,  4.94s/it]
 80%|███████▉  | 178987/225000 [340:08:20<66:15:36,  5.18s/it]
 80%|███████▉  | 178988/225000 [340:08:25<65:34:10,  5.13s/it]
 80%|███████▉  | 178989/225000 [340:08:37<91:50:18,  7.19s/it]
 80%|███████▉  | 178990/225000 [340:08:42<80:26:04,  6.29s/it]
 80%|███████▉  | 178991/225000 [340:08:48<80:01:08,  6.26s/it]
 80%|███████▉  | 178992/225000 [340:08:53<75:43:51,  5.93s/it]
 80%|███████▉  | 178993/225000 [340:08:57<68:34:07,  5.37s/it]
 80%|███████▉  | 178994/225000 [340:09:01<64:04:05,  5.01s/it]
 80%|███████▉  | 178995/225000 [340:09:07<66:42:00,  5.22s/it]
 80%|███████▉  | 178996/225000 [340:09:16<81:40:27,  6.39s/it]
 80%|███████▉  | 178997/225000 [340:09:20<74:03:59,  5.80s/it]
 80%|███████▉  | 178998/225000 [340:09:25<70:07:43,  5.49s/it]
 80%|███████▉  | 178999/225000 [340:09:30<66:07:09,  5.17s/it]
 80%|███████▉  | 179000/225000 [340:09:34<64:31:51,  5.05s/it]
                                                              
{'loss': '0.7598', 'grad_norm': '1.192', 'learning_rate': '5.457e-06', 'epoch': '0.7956', 'num_input_tokens_seen': 563948400, 'train_runtime': '1.225e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179000/225000 [340:09:34<64:31:51,  5.05s/it]
 80%|███████▉  | 179001/225000 [340:09:39<63:15:23,  4.95s/it]
 80%|███████▉  | 179002/225000 [340:09:44<62:12:59,  4.87s/it]
 80%|███████▉  | 179003/225000 [340:09:56<90:13:55,  7.06s/it]
 80%|███████▉  | 179004/225000 [340:10:00<77:47:06,  6.09s/it]
 80%|███████▉  | 179005/225000 [340:10:03<68:06:59,  5.33s/it]
 80%|███████▉  | 179006/225000 [340:10:12<82:47:18,  6.48s/it]
 80%|███████▉  | 179007/225000 [340:10:16<70:51:07,  5.55s/it]
 80%|███████▉  | 179008/225000 [340:10:22<71:56:42,  5.63s/it]
 80%|███████▉  | 179009/225000 [340:10:26<66:16:26,  5.19s/it]
 80%|███████▉  | 179010/225000 [340:10:30<61:40:06,  4.83s/it]
 80%|███████▉  | 179011/225000 [340:10:37<70:05:03,  5.49s/it]
 80%|███████▉  | 179012/225000 [340:10:45<79:00:55,  6.19s/it]
 80%|███████▉  | 179013/225000 [340:10:49<73:08:39,  5.73s/it]
 80%|███████▉  | 179014/225000 [340:10:54<70:30:04,  5.52s/it]
 80%|███████▉  | 179015/225000 [340:10:59<68:35:50,  5.37s/it]
 80%|███████▉  | 179016/225000 [340:11:04<66:06:31,  5.18s/it]
 80%|███████▉  | 179017/225000 [340:11:09<63:50:25,  5.00s/it]
 80%|███████▉  | 179018/225000 [340:11:13<63:16:39,  4.95s/it]
 80%|███████▉  | 179019/225000 [340:11:19<63:55:28,  5.00s/it]
 80%|███████▉  | 179020/225000 [340:11:23<62:49:23,  4.92s/it]
 80%|███████▉  | 179021/225000 [340:11:30<71:11:15,  5.57s/it]
 80%|███████▉  | 179022/225000 [340:11:41<91:12:52,  7.14s/it]
 80%|███████▉  | 179023/225000 [340:11:44<76:15:30,  5.97s/it]
 80%|███████▉  | 179024/225000 [340:11:51<80:06:19,  6.27s/it]
 80%|███████▉  | 179025/225000 [340:11:55<68:43:52,  5.38s/it]
 80%|███████▉  | 179026/225000 [340:12:02<76:32:57,  5.99s/it]
 80%|███████▉  | 179027/225000 [340:12:06<67:01:40,  5.25s/it]
 80%|███████▉  | 179028/225000 [340:12:09<60:27:41,  4.73s/it]
 80%|███████▉  | 179029/225000 [340:12:18<75:30:59,  5.91s/it]
 80%|███████▉  | 179030/225000 [340:12:24<77:53:15,  6.10s/it]
 80%|███████▉  | 179031/225000 [340:12:27<66:07:31,  5.18s/it]
 80%|███████▉  | 179032/225000 [340:12:31<58:54:27,  4.61s/it]
 80%|███████▉  | 179033/225000 [340:12:39<71:32:21,  5.60s/it]
 80%|███████▉  | 179034/225000 [340:12:42<64:06:20,  5.02s/it]
 80%|███████▉  | 179035/225000 [340:12:47<61:37:34,  4.83s/it]
 80%|███████▉  | 179036/225000 [340:12:56<77:20:49,  6.06s/it]
 80%|███████▉  | 179037/225000 [340:13:00<72:03:46,  5.64s/it]
 80%|███████▉  | 179038/225000 [340:13:05<68:59:46,  5.40s/it]
 80%|███████▉  | 179039/225000 [340:13:10<67:29:30,  5.29s/it]
 80%|███████▉  | 179040/225000 [340:13:15<65:33:49,  5.14s/it]
 80%|███████▉  | 179041/225000 [340:13:20<64:18:07,  5.04s/it]
 80%|███████▉  | 179042/225000 [340:13:25<63:20:06,  4.96s/it]
 80%|███████▉  | 179043/225000 [340:13:29<61:44:03,  4.84s/it]
 80%|███████▉  | 179044/225000 [340:13:33<58:19:05,  4.57s/it]
 80%|███████▉  | 179045/225000 [340:13:38<59:37:06,  4.67s/it]
 80%|███████▉  | 179046/225000 [340:13:43<61:36:13,  4.83s/it]
 80%|███████▉  | 179047/225000 [340:13:48<61:02:04,  4.78s/it]
 80%|███████▉  | 179048/225000 [340:13:52<60:13:05,  4.72s/it]
 80%|███████▉  | 179049/225000 [340:14:06<92:49:19,  7.27s/it]
 80%|███████▉  | 179050/225000 [340:14:11<84:19:57,  6.61s/it]
                                                              
{'loss': '0.7691', 'grad_norm': '1.133', 'learning_rate': '5.446e-06', 'epoch': '0.7958', 'num_input_tokens_seen': 564078152, 'train_runtime': '1.225e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179050/225000 [340:14:11<84:19:57,  6.61s/it]
 80%|███████▉  | 179051/225000 [340:14:15<74:45:35,  5.86s/it]
 80%|███████▉  | 179052/225000 [340:14:19<67:51:09,  5.32s/it]
 80%|███████▉  | 179053/225000 [340:14:29<87:18:50,  6.84s/it]
 80%|███████▉  | 179054/225000 [340:14:32<73:29:13,  5.76s/it]
 80%|███████▉  | 179055/225000 [340:14:39<76:52:31,  6.02s/it]
 80%|███████▉  | 179056/225000 [340:14:43<68:20:53,  5.36s/it]
 80%|███████▉  | 179057/225000 [340:14:47<61:59:02,  4.86s/it]
 80%|███████▉  | 179058/225000 [340:14:56<80:45:46,  6.33s/it]
 80%|███████▉  | 179059/225000 [340:15:00<68:47:09,  5.39s/it]
 80%|███████▉  | 179060/225000 [340:15:06<74:23:13,  5.83s/it]
 80%|███████▉  | 179061/225000 [340:15:12<73:14:33,  5.74s/it]
 80%|███████▉  | 179062/225000 [340:15:15<63:42:22,  4.99s/it]
 80%|███████▉  | 179063/225000 [340:15:23<75:44:49,  5.94s/it]
 80%|███████▉  | 179064/225000 [340:15:28<71:32:05,  5.61s/it]
 80%|███████▉  | 179065/225000 [340:15:32<64:44:00,  5.07s/it]
 80%|███████▉  | 179066/225000 [340:15:40<74:27:42,  5.84s/it]
 80%|███████▉  | 179067/225000 [340:15:44<69:03:14,  5.41s/it]
 80%|███████▉  | 179068/225000 [340:15:48<62:22:27,  4.89s/it]
 80%|███████▉  | 179069/225000 [340:15:56<74:12:58,  5.82s/it]
 80%|███████▉  | 179070/225000 [340:15:59<65:10:01,  5.11s/it]
 80%|███████▉  | 179071/225000 [340:16:09<81:59:10,  6.43s/it]
 80%|███████▉  | 179072/225000 [340:16:12<69:47:14,  5.47s/it]
 80%|███████▉  | 179073/225000 [340:16:15<62:27:57,  4.90s/it]
 80%|███████▉  | 179074/225000 [340:16:24<75:53:01,  5.95s/it]
 80%|███████▉  | 179075/225000 [340:16:27<65:14:15,  5.11s/it]
 80%|███████▉  | 179076/225000 [340:16:31<59:37:56,  4.67s/it]
 80%|███████▉  | 179077/225000 [340:16:40<76:51:46,  6.03s/it]
 80%|███████▉  | 179078/225000 [340:16:44<71:38:25,  5.62s/it]
 80%|███████▉  | 179079/225000 [340:16:48<64:05:34,  5.02s/it]
 80%|███████▉  | 179080/225000 [340:16:51<57:46:08,  4.53s/it]
 80%|███████▉  | 179081/225000 [340:16:59<70:16:18,  5.51s/it]
 80%|███████▉  | 179082/225000 [340:17:06<74:07:23,  5.81s/it]
 80%|███████▉  | 179083/225000 [340:17:10<69:54:05,  5.48s/it]
 80%|███████▉  | 179084/225000 [340:17:17<71:58:33,  5.64s/it]
 80%|███████▉  | 179085/225000 [340:17:20<64:23:22,  5.05s/it]
 80%|███████▉  | 179086/225000 [340:17:24<58:58:56,  4.62s/it]
 80%|███████▉  | 179087/225000 [340:17:30<65:15:52,  5.12s/it]
 80%|███████▉  | 179088/225000 [340:17:38<77:38:35,  6.09s/it]
 80%|███████▉  | 179089/225000 [340:17:42<68:03:55,  5.34s/it]
 80%|███████▉  | 179090/225000 [340:17:51<82:07:47,  6.44s/it]
 80%|███████▉  | 179091/225000 [340:17:55<72:27:20,  5.68s/it]
 80%|███████▉  | 179092/225000 [340:17:59<65:07:06,  5.11s/it]
 80%|███████▉  | 179093/225000 [340:18:08<80:09:07,  6.29s/it]
 80%|███████▉  | 179094/225000 [340:18:11<69:33:51,  5.46s/it]
 80%|███████▉  | 179095/225000 [340:18:16<68:19:16,  5.36s/it]
 80%|███████▉  | 179096/225000 [340:18:24<76:42:41,  6.02s/it]
 80%|███████▉  | 179097/225000 [340:18:28<67:36:11,  5.30s/it]
 80%|███████▉  | 179098/225000 [340:18:31<61:24:23,  4.82s/it]
 80%|███████▉  | 179099/225000 [340:18:40<76:52:02,  6.03s/it]
 80%|███████▉  | 179100/225000 [340:18:43<66:08:34,  5.19s/it]
                                                              
{'loss': '0.7608', 'grad_norm': '1.129', 'learning_rate': '5.434e-06', 'epoch': '0.796', 'num_input_tokens_seen': 564208112, 'train_runtime': '1.225e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179100/225000 [340:18:43<66:08:34,  5.19s/it]
 80%|███████▉  | 179101/225000 [340:18:47<62:05:52,  4.87s/it]
 80%|███████▉  | 179102/225000 [340:18:56<77:15:32,  6.06s/it]
 80%|███████▉  | 179103/225000 [340:19:00<68:50:41,  5.40s/it]
 80%|███████▉  | 179104/225000 [340:19:04<63:15:19,  4.96s/it]
 80%|███████▉  | 179105/225000 [340:19:08<58:02:34,  4.55s/it]
 80%|███████▉  | 179106/225000 [340:19:18<79:45:54,  6.26s/it]
 80%|███████▉  | 179107/225000 [340:19:21<69:05:09,  5.42s/it]
 80%|███████▉  | 179108/225000 [340:19:25<61:11:31,  4.80s/it]
 80%|███████▉  | 179109/225000 [340:19:33<75:45:06,  5.94s/it]
 80%|███████▉  | 179110/225000 [340:19:37<67:51:29,  5.32s/it]
 80%|███████▉  | 179111/225000 [340:19:41<62:17:40,  4.89s/it]
 80%|███████▉  | 179112/225000 [340:19:51<80:11:18,  6.29s/it]
 80%|███████▉  | 179113/225000 [340:19:56<75:22:02,  5.91s/it]
 80%|███████▉  | 179114/225000 [340:19:59<67:07:10,  5.27s/it]
 80%|███████▉  | 179115/225000 [340:20:08<79:12:40,  6.21s/it]
 80%|███████▉  | 179116/225000 [340:20:12<72:07:04,  5.66s/it]
 80%|███████▉  | 179117/225000 [340:20:18<71:11:29,  5.59s/it]
 80%|███████▉  | 179118/225000 [340:20:22<67:36:52,  5.31s/it]
 80%|███████▉  | 179119/225000 [340:20:27<65:04:22,  5.11s/it]
 80%|███████▉  | 179120/225000 [340:20:31<61:13:29,  4.80s/it]
 80%|███████▉  | 179121/225000 [340:20:36<60:35:13,  4.75s/it]
 80%|███████▉  | 179122/225000 [340:20:40<60:09:36,  4.72s/it]
 80%|███████▉  | 179123/225000 [340:20:45<59:49:53,  4.70s/it]
 80%|███████▉  | 179124/225000 [340:20:49<58:59:50,  4.63s/it]
 80%|███████▉  | 179125/225000 [340:20:58<73:38:26,  5.78s/it]
 80%|███████▉  | 179126/225000 [340:21:07<85:37:08,  6.72s/it]
 80%|███████▉  | 179127/225000 [340:21:10<72:08:29,  5.66s/it]
 80%|███████▉  | 179128/225000 [340:21:17<78:29:02,  6.16s/it]
 80%|███████▉  | 179129/225000 [340:21:21<69:45:32,  5.47s/it]
 80%|███████▉  | 179130/225000 [340:21:30<80:28:33,  6.32s/it]
 80%|███████▉  | 179131/225000 [340:21:34<72:06:29,  5.66s/it]
 80%|███████▉  | 179132/225000 [340:21:38<66:10:27,  5.19s/it]
 80%|███████▉  | 179133/225000 [340:21:42<62:41:41,  4.92s/it]
 80%|███████▉  | 179134/225000 [340:21:47<61:20:25,  4.81s/it]
 80%|███████▉  | 179135/225000 [340:21:50<57:26:19,  4.51s/it]
 80%|███████▉  | 179136/225000 [340:22:01<80:20:57,  6.31s/it]
 80%|███████▉  | 179137/225000 [340:22:06<75:39:00,  5.94s/it]
 80%|███████▉  | 179138/225000 [340:22:09<65:52:09,  5.17s/it]
 80%|███████▉  | 179139/225000 [340:22:14<62:06:38,  4.88s/it]
 80%|███████▉  | 179140/225000 [340:22:23<79:16:28,  6.22s/it]
 80%|███████▉  | 179141/225000 [340:22:27<72:56:48,  5.73s/it]
 80%|███████▉  | 179142/225000 [340:22:33<73:55:23,  5.80s/it]
 80%|███████▉  | 179143/225000 [340:22:37<67:10:27,  5.27s/it]
 80%|███████▉  | 179144/225000 [340:22:41<61:48:21,  4.85s/it]
 80%|███████▉  | 179145/225000 [340:22:46<59:51:22,  4.70s/it]
 80%|███████▉  | 179146/225000 [340:22:54<74:18:44,  5.83s/it]
 80%|███████▉  | 179147/225000 [340:23:01<76:47:20,  6.03s/it]
 80%|███████▉  | 179148/225000 [340:23:06<72:37:07,  5.70s/it]
 80%|███████▉  | 179149/225000 [340:23:11<71:11:12,  5.59s/it]
 80%|███████▉  | 179150/225000 [340:23:15<66:28:24,  5.22s/it]
                                                              
{'loss': '0.7518', 'grad_norm': '1.103', 'learning_rate': '5.423e-06', 'epoch': '0.7962', 'num_input_tokens_seen': 564338240, 'train_runtime': '1.225e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179150/225000 [340:23:15<66:28:24,  5.22s/it]
 80%|███████▉  | 179151/225000 [340:23:20<63:37:26,  5.00s/it]
 80%|███████▉  | 179152/225000 [340:23:24<60:02:03,  4.71s/it]
 80%|███████▉  | 179153/225000 [340:23:29<60:25:03,  4.74s/it]
 80%|███████▉  | 179154/225000 [340:23:39<81:04:33,  6.37s/it]
 80%|███████▉  | 179155/225000 [340:23:42<70:18:19,  5.52s/it]
 80%|███████▉  | 179156/225000 [340:23:51<80:45:12,  6.34s/it]
 80%|███████▉  | 179157/225000 [340:23:54<69:39:59,  5.47s/it]
 80%|███████▉  | 179158/225000 [340:23:59<68:30:50,  5.38s/it]
 80%|███████▉  | 179159/225000 [340:24:05<69:14:28,  5.44s/it]
 80%|███████▉  | 179160/225000 [340:24:13<81:40:57,  6.41s/it]
 80%|███████▉  | 179161/225000 [340:24:16<68:44:26,  5.40s/it]
 80%|███████▉  | 179162/225000 [340:24:24<76:34:31,  6.01s/it]
 80%|███████▉  | 179163/225000 [340:24:29<71:37:44,  5.63s/it]
 80%|███████▉  | 179164/225000 [340:24:33<68:31:00,  5.38s/it]
 80%|███████▉  | 179165/225000 [340:24:38<66:39:58,  5.24s/it]
 80%|███████▉  | 179166/225000 [340:24:43<63:34:23,  4.99s/it]
 80%|███████▉  | 179167/225000 [340:24:47<61:31:44,  4.83s/it]
 80%|███████▉  | 179168/225000 [340:24:52<62:20:06,  4.90s/it]
 80%|███████▉  | 179169/225000 [340:24:57<61:42:02,  4.85s/it]
 80%|███████▉  | 179170/225000 [340:25:02<62:21:49,  4.90s/it]
 80%|███████▉  | 179171/225000 [340:25:14<90:31:38,  7.11s/it]
 80%|███████▉  | 179172/225000 [340:25:18<75:46:47,  5.95s/it]
 80%|███████▉  | 179173/225000 [340:25:21<64:53:28,  5.10s/it]
 80%|███████▉  | 179174/225000 [340:25:30<79:10:17,  6.22s/it]
 80%|███████▉  | 179175/225000 [340:25:33<68:38:48,  5.39s/it]
 80%|███████▉  | 179176/225000 [340:25:37<64:26:17,  5.06s/it]
 80%|███████▉  | 179177/225000 [340:25:42<61:43:42,  4.85s/it]
 80%|███████▉  | 179178/225000 [340:25:45<57:52:55,  4.55s/it]
 80%|███████▉  | 179179/225000 [340:25:56<82:00:29,  6.44s/it]
 80%|███████▉  | 179180/225000 [340:26:00<72:08:43,  5.67s/it]
 80%|███████▉  | 179181/225000 [340:26:04<65:28:42,  5.14s/it]
 80%|███████▉  | 179182/225000 [340:26:08<61:09:18,  4.81s/it]
 80%|███████▉  | 179183/225000 [340:26:12<58:02:52,  4.56s/it]
 80%|███████▉  | 179184/225000 [340:26:23<81:09:34,  6.38s/it]
 80%|███████▉  | 179185/225000 [340:26:29<79:05:26,  6.21s/it]
 80%|███████▉  | 179186/225000 [340:26:34<75:29:02,  5.93s/it]
 80%|███████▉  | 179187/225000 [340:26:39<71:40:25,  5.63s/it]
 80%|███████▉  | 179188/225000 [340:26:46<77:18:04,  6.07s/it]
 80%|███████▉  | 179189/225000 [340:26:50<68:57:17,  5.42s/it]
 80%|███████▉  | 179190/225000 [340:26:55<67:56:34,  5.34s/it]
 80%|███████▉  | 179191/225000 [340:27:00<65:53:45,  5.18s/it]
 80%|███████▉  | 179192/225000 [340:27:05<64:38:19,  5.08s/it]
 80%|███████▉  | 179193/225000 [340:27:10<66:13:25,  5.20s/it]
 80%|███████▉  | 179194/225000 [340:27:15<66:21:10,  5.21s/it]
 80%|███████▉  | 179195/225000 [340:27:20<65:29:23,  5.15s/it]
 80%|███████▉  | 179196/225000 [340:27:26<66:15:18,  5.21s/it]
 80%|███████▉  | 179197/225000 [340:27:31<65:23:30,  5.14s/it]
 80%|███████▉  | 179198/225000 [340:27:36<65:22:11,  5.14s/it]
 80%|███████▉  | 179199/225000 [340:27:41<63:55:48,  5.02s/it]
 80%|███████▉  | 179200/225000 [340:27:45<63:23:37,  4.98s/it]
                                                              
{'loss': '0.771', 'grad_norm': '1.122', 'learning_rate': '5.412e-06', 'epoch': '0.7964', 'num_input_tokens_seen': 564469424, 'train_runtime': '1.226e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179200/225000 [340:27:45<63:23:37,  4.98s/it]
 80%|███████▉  | 179201/225000 [340:27:51<64:00:36,  5.03s/it]
 80%|███████▉  | 179202/225000 [340:27:56<64:58:19,  5.11s/it]
 80%|███████▉  | 179203/225000 [340:28:01<63:28:54,  4.99s/it]
 80%|███████▉  | 179204/225000 [340:28:06<64:10:18,  5.04s/it]
 80%|███████▉  | 179205/225000 [340:28:18<90:08:36,  7.09s/it]
 80%|███████▉  | 179206/225000 [340:28:25<92:45:46,  7.29s/it]
 80%|███████▉  | 179207/225000 [340:28:29<77:12:54,  6.07s/it]
 80%|███████▉  | 179208/225000 [340:28:37<86:15:51,  6.78s/it]
 80%|███████▉  | 179209/225000 [340:28:42<79:17:15,  6.23s/it]
 80%|███████▉  | 179210/225000 [340:28:45<67:49:35,  5.33s/it]
 80%|███████▉  | 179211/225000 [340:28:54<80:18:03,  6.31s/it]
 80%|███████▉  | 179212/225000 [340:28:57<69:19:57,  5.45s/it]
 80%|███████▉  | 179213/225000 [340:29:05<78:07:56,  6.14s/it]
 80%|███████▉  | 179214/225000 [340:29:09<69:20:33,  5.45s/it]
 80%|███████▉  | 179215/225000 [340:29:13<63:10:23,  4.97s/it]
 80%|███████▉  | 179216/225000 [340:29:22<78:30:35,  6.17s/it]
 80%|███████▉  | 179217/225000 [340:29:26<69:59:45,  5.50s/it]
 80%|███████▉  | 179218/225000 [340:29:30<64:27:23,  5.07s/it]
 80%|███████▉  | 179219/225000 [340:29:34<60:24:41,  4.75s/it]
 80%|███████▉  | 179220/225000 [340:29:44<83:39:34,  6.58s/it]
 80%|███████▉  | 179221/225000 [340:29:48<70:09:11,  5.52s/it]
 80%|███████▉  | 179222/225000 [340:29:53<69:15:59,  5.45s/it]
 80%|███████▉  | 179223/225000 [340:29:58<69:04:22,  5.43s/it]
 80%|███████▉  | 179224/225000 [340:30:04<69:39:49,  5.48s/it]
 80%|███████▉  | 179225/225000 [340:30:09<70:08:00,  5.52s/it]
 80%|███████▉  | 179226/225000 [340:30:15<69:18:27,  5.45s/it]
 80%|███████▉  | 179227/225000 [340:30:20<67:47:43,  5.33s/it]
 80%|███████▉  | 179228/225000 [340:30:25<67:25:29,  5.30s/it]
 80%|███████▉  | 179229/225000 [340:30:31<71:29:25,  5.62s/it]
 80%|███████▉  | 179230/225000 [340:30:38<74:09:19,  5.83s/it]
 80%|███████▉  | 179231/225000 [340:30:43<71:52:13,  5.65s/it]
 80%|███████▉  | 179232/225000 [340:30:47<65:18:40,  5.14s/it]
 80%|███████▉  | 179233/225000 [340:30:51<62:59:55,  4.96s/it]
 80%|███████▉  | 179234/225000 [340:31:00<75:16:27,  5.92s/it]
 80%|███████▉  | 179235/225000 [340:31:04<70:56:44,  5.58s/it]
 80%|███████▉  | 179236/225000 [340:31:10<71:14:34,  5.60s/it]
 80%|███████▉  | 179237/225000 [340:31:15<70:28:50,  5.54s/it]
 80%|███████▉  | 179238/225000 [340:31:21<69:13:56,  5.45s/it]
 80%|███████▉  | 179239/225000 [340:31:26<68:26:53,  5.38s/it]
 80%|███████▉  | 179240/225000 [340:31:31<67:22:24,  5.30s/it]
 80%|███████▉  | 179241/225000 [340:31:37<70:55:47,  5.58s/it]
 80%|███████▉  | 179242/225000 [340:31:42<68:01:55,  5.35s/it]
 80%|███████▉  | 179243/225000 [340:31:47<67:30:34,  5.31s/it]
 80%|███████▉  | 179244/225000 [340:31:53<67:24:34,  5.30s/it]
 80%|███████▉  | 179245/225000 [340:31:58<67:18:26,  5.30s/it]
 80%|███████▉  | 179246/225000 [340:32:03<65:32:58,  5.16s/it]
 80%|███████▉  | 179247/225000 [340:32:08<64:49:39,  5.10s/it]
 80%|███████▉  | 179248/225000 [340:32:13<65:53:37,  5.18s/it]
 80%|███████▉  | 179249/225000 [340:32:20<73:43:49,  5.80s/it]
 80%|███████▉  | 179250/225000 [340:32:26<73:47:41,  5.81s/it]
                                                              
{'loss': '0.7642', 'grad_norm': '1.129', 'learning_rate': '5.4e-06', 'epoch': '0.7967', 'num_input_tokens_seen': 564599968, 'train_runtime': '1.226e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179250/225000 [340:32:26<73:47:41,  5.81s/it]
 80%|███████▉  | 179251/225000 [340:32:31<71:59:59,  5.67s/it]
 80%|███████▉  | 179252/225000 [340:32:37<71:00:58,  5.59s/it]
 80%|███████▉  | 179253/225000 [340:32:44<78:28:57,  6.18s/it]
 80%|███████▉  | 179254/225000 [340:32:49<72:11:00,  5.68s/it]
 80%|███████▉  | 179255/225000 [340:32:54<70:36:46,  5.56s/it]
 80%|███████▉  | 179256/225000 [340:33:00<69:57:26,  5.51s/it]
 80%|███████▉  | 179257/225000 [340:33:05<71:01:50,  5.59s/it]
 80%|███████▉  | 179258/225000 [340:33:11<70:56:59,  5.58s/it]
 80%|███████▉  | 179259/225000 [340:33:16<70:20:28,  5.54s/it]
 80%|███████▉  | 179260/225000 [340:33:23<73:22:27,  5.77s/it]
 80%|███████▉  | 179261/225000 [340:33:29<76:36:22,  6.03s/it]
 80%|███████▉  | 179262/225000 [340:33:34<70:58:27,  5.59s/it]
 80%|███████▉  | 179263/225000 [340:33:38<66:53:38,  5.27s/it]
 80%|███████▉  | 179264/225000 [340:33:43<63:28:17,  5.00s/it]
 80%|███████▉  | 179265/225000 [340:33:47<60:54:24,  4.79s/it]
 80%|███████▉  | 179266/225000 [340:33:51<59:36:22,  4.69s/it]
 80%|███████▉  | 179267/225000 [340:34:04<89:02:36,  7.01s/it]
 80%|███████▉  | 179268/225000 [340:34:07<75:19:37,  5.93s/it]
 80%|███████▉  | 179269/225000 [340:34:15<81:53:27,  6.45s/it]
 80%|███████▉  | 179270/225000 [340:34:18<70:44:51,  5.57s/it]
 80%|███████▉  | 179271/225000 [340:34:22<64:41:38,  5.09s/it]
 80%|███████▉  | 179272/225000 [340:34:26<60:12:12,  4.74s/it]
 80%|███████▉  | 179273/225000 [340:34:30<56:30:08,  4.45s/it]
 80%|███████▉  | 179274/225000 [340:34:41<79:36:09,  6.27s/it]
 80%|███████▉  | 179275/225000 [340:34:44<69:53:49,  5.50s/it]
 80%|███████▉  | 179276/225000 [340:34:48<63:46:41,  5.02s/it]
 80%|███████▉  | 179277/225000 [340:34:56<75:33:05,  5.95s/it]
 80%|███████▉  | 179278/225000 [340:35:00<66:59:02,  5.27s/it]
 80%|███████▉  | 179279/225000 [340:35:03<59:45:50,  4.71s/it]
 80%|███████▉  | 179280/225000 [340:35:13<79:17:32,  6.24s/it]
 80%|███████▉  | 179281/225000 [340:35:17<71:16:14,  5.61s/it]
 80%|███████▉  | 179282/225000 [340:35:21<64:47:12,  5.10s/it]
 80%|███████▉  | 179283/225000 [340:35:25<61:01:20,  4.81s/it]
 80%|███████▉  | 179284/225000 [340:35:30<58:41:14,  4.62s/it]
 80%|███████▉  | 179285/225000 [340:35:34<57:45:49,  4.55s/it]
 80%|███████▉  | 179286/225000 [340:35:40<64:40:55,  5.09s/it]
 80%|███████▉  | 179287/225000 [340:35:50<81:39:19,  6.43s/it]
 80%|███████▉  | 179288/225000 [340:35:53<68:53:43,  5.43s/it]
 80%|███████▉  | 179289/225000 [340:35:56<60:39:20,  4.78s/it]
 80%|███████▉  | 179290/225000 [340:36:05<74:33:54,  5.87s/it]
 80%|███████▉  | 179291/225000 [340:36:10<73:32:50,  5.79s/it]
 80%|███████▉  | 179292/225000 [340:36:16<73:31:42,  5.79s/it]
 80%|███████▉  | 179293/225000 [340:36:20<68:06:41,  5.36s/it]
 80%|███████▉  | 179294/225000 [340:36:25<64:41:39,  5.10s/it]
 80%|███████▉  | 179295/225000 [340:36:29<62:30:03,  4.92s/it]
 80%|███████▉  | 179296/225000 [340:36:34<60:29:17,  4.76s/it]
 80%|███████▉  | 179297/225000 [340:36:45<86:21:48,  6.80s/it]
 80%|███████▉  | 179298/225000 [340:36:50<78:32:20,  6.19s/it]
 80%|███████▉  | 179299/225000 [340:36:55<75:09:09,  5.92s/it]
 80%|███████▉  | 179300/225000 [340:36:59<66:41:02,  5.25s/it]
                                                              
{'loss': '0.763', 'grad_norm': '1.08', 'learning_rate': '5.389e-06', 'epoch': '0.7969', 'num_input_tokens_seen': 564729688, 'train_runtime': '1.226e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179300/225000 [340:36:59<66:41:02,  5.25s/it]
 80%|███████▉  | 179301/225000 [340:37:08<81:21:03,  6.41s/it]
 80%|███████▉  | 179302/225000 [340:37:13<73:05:56,  5.76s/it]
 80%|███████▉  | 179303/225000 [340:37:17<67:03:55,  5.28s/it]
 80%|███████▉  | 179304/225000 [340:37:21<63:19:37,  4.99s/it]
 80%|███████▉  | 179305/225000 [340:37:25<60:09:49,  4.74s/it]
 80%|███████▉  | 179306/225000 [340:37:29<57:58:17,  4.57s/it]
 80%|███████▉  | 179307/225000 [340:37:34<57:56:44,  4.57s/it]
 80%|███████▉  | 179308/225000 [340:37:43<73:56:40,  5.83s/it]
 80%|███████▉  | 179309/225000 [340:37:50<80:17:26,  6.33s/it]
 80%|███████▉  | 179310/225000 [340:37:56<77:19:43,  6.09s/it]
 80%|███████▉  | 179311/225000 [340:38:01<74:00:53,  5.83s/it]
 80%|███████▉  | 179312/225000 [340:38:06<72:14:26,  5.69s/it]
 80%|███████▉  | 179313/225000 [340:38:12<73:09:56,  5.77s/it]
 80%|███████▉  | 179314/225000 [340:38:18<71:58:31,  5.67s/it]
 80%|███████▉  | 179315/225000 [340:38:23<71:04:26,  5.60s/it]
 80%|███████▉  | 179316/225000 [340:38:29<73:38:54,  5.80s/it]
 80%|███████▉  | 179317/225000 [340:38:35<72:29:37,  5.71s/it]
 80%|███████▉  | 179318/225000 [340:38:39<67:13:57,  5.30s/it]
 80%|███████▉  | 179319/225000 [340:38:44<63:55:36,  5.04s/it]
 80%|███████▉  | 179320/225000 [340:38:48<62:50:49,  4.95s/it]
 80%|███████▉  | 179321/225000 [340:38:59<84:58:18,  6.70s/it]
 80%|███████▉  | 179322/225000 [340:39:02<72:12:31,  5.69s/it]
 80%|███████▉  | 179323/225000 [340:39:10<80:19:38,  6.33s/it]
 80%|███████▉  | 179324/225000 [340:39:14<69:19:25,  5.46s/it]
 80%|███████▉  | 179325/225000 [340:39:19<67:12:58,  5.30s/it]
 80%|███████▉  | 179326/225000 [340:39:23<65:01:18,  5.12s/it]
 80%|███████▉  | 179327/225000 [340:39:29<66:41:52,  5.26s/it]
 80%|███████▉  | 179328/225000 [340:39:34<65:50:46,  5.19s/it]
 80%|███████▉  | 179329/225000 [340:39:39<64:21:30,  5.07s/it]
 80%|███████▉  | 179330/225000 [340:39:44<65:15:24,  5.14s/it]
 80%|███████▉  | 179331/225000 [340:39:49<64:45:01,  5.10s/it]
 80%|███████▉  | 179332/225000 [340:39:55<66:05:40,  5.21s/it]
 80%|███████▉  | 179333/225000 [340:39:59<64:46:35,  5.11s/it]
 80%|███████▉  | 179334/225000 [340:40:05<65:44:17,  5.18s/it]
 80%|███████▉  | 179335/225000 [340:40:10<64:33:11,  5.09s/it]
 80%|███████▉  | 179336/225000 [340:40:15<64:48:12,  5.11s/it]
 80%|███████▉  | 179337/225000 [340:40:20<65:24:53,  5.16s/it]
 80%|███████▉  | 179338/225000 [340:40:25<66:08:33,  5.21s/it]
 80%|███████▉  | 179339/225000 [340:40:31<66:56:39,  5.28s/it]
 80%|███████▉  | 179340/225000 [340:40:36<66:34:36,  5.25s/it]
 80%|███████▉  | 179341/225000 [340:40:41<66:59:34,  5.28s/it]
 80%|███████▉  | 179342/225000 [340:40:53<92:03:22,  7.26s/it]
 80%|███████▉  | 179343/225000 [340:40:59<85:54:03,  6.77s/it]
 80%|███████▉  | 179344/225000 [340:41:03<74:52:29,  5.90s/it]
 80%|███████▉  | 179345/225000 [340:41:07<67:13:37,  5.30s/it]
 80%|███████▉  | 179346/225000 [340:41:17<84:26:51,  6.66s/it]
 80%|███████▉  | 179347/225000 [340:41:20<73:23:36,  5.79s/it]
 80%|███████▉  | 179348/225000 [340:41:27<76:58:30,  6.07s/it]
 80%|███████▉  | 179349/225000 [340:41:32<71:15:12,  5.62s/it]
 80%|███████▉  | 179350/225000 [340:41:36<66:20:17,  5.23s/it]
                                                              
{'loss': '0.7576', 'grad_norm': '1.258', 'learning_rate': '5.378e-06', 'epoch': '0.7971', 'num_input_tokens_seen': 564858760, 'train_runtime': '1.226e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179350/225000 [340:41:36<66:20:17,  5.23s/it]
 80%|███████▉  | 179351/225000 [340:41:40<63:37:45,  5.02s/it]
 80%|███████▉  | 179352/225000 [340:41:45<61:05:33,  4.82s/it]
 80%|███████▉  | 179353/225000 [340:41:49<60:01:57,  4.73s/it]
 80%|███████▉  | 179354/225000 [340:41:54<58:06:48,  4.58s/it]
 80%|███████▉  | 179355/225000 [340:41:59<60:26:03,  4.77s/it]
 80%|███████▉  | 179356/225000 [340:42:04<63:20:14,  5.00s/it]
 80%|███████▉  | 179357/225000 [340:42:13<75:47:26,  5.98s/it]
 80%|███████▉  | 179358/225000 [340:42:21<87:01:48,  6.86s/it]
 80%|███████▉  | 179359/225000 [340:42:27<81:49:21,  6.45s/it]
 80%|███████▉  | 179360/225000 [340:42:32<78:01:07,  6.15s/it]
 80%|███████▉  | 179361/225000 [340:42:40<84:14:53,  6.65s/it]
 80%|███████▉  | 179362/225000 [340:42:45<75:38:52,  5.97s/it]
 80%|███████▉  | 179363/225000 [340:42:49<68:35:35,  5.41s/it]
 80%|███████▉  | 179364/225000 [340:42:53<63:47:17,  5.03s/it]
 80%|███████▉  | 179365/225000 [340:42:57<60:29:25,  4.77s/it]
 80%|███████▉  | 179366/225000 [340:43:01<58:25:57,  4.61s/it]
 80%|███████▉  | 179367/225000 [340:43:08<68:15:22,  5.38s/it]
 80%|███████▉  | 179368/225000 [340:43:17<78:41:49,  6.21s/it]
 80%|███████▉  | 179369/225000 [340:43:20<69:52:11,  5.51s/it]
 80%|███████▉  | 179370/225000 [340:43:28<78:57:28,  6.23s/it]
 80%|███████▉  | 179371/225000 [340:43:32<68:30:11,  5.40s/it]
 80%|███████▉  | 179372/225000 [340:43:37<65:44:01,  5.19s/it]
 80%|███████▉  | 179373/225000 [340:43:41<63:38:58,  5.02s/it]
 80%|███████▉  | 179374/225000 [340:43:46<63:55:20,  5.04s/it]
 80%|███████▉  | 179375/225000 [340:43:57<84:47:32,  6.69s/it]
 80%|███████▉  | 179376/225000 [340:44:00<73:04:48,  5.77s/it]
 80%|███████▉  | 179377/225000 [340:44:06<72:50:26,  5.75s/it]
 80%|███████▉  | 179378/225000 [340:44:12<74:23:47,  5.87s/it]
 80%|███████▉  | 179379/225000 [340:44:18<74:42:24,  5.90s/it]
 80%|███████▉  | 179380/225000 [340:44:22<67:48:40,  5.35s/it]
 80%|███████▉  | 179381/225000 [340:44:26<62:52:16,  4.96s/it]
 80%|███████▉  | 179382/225000 [340:44:32<66:26:27,  5.24s/it]
 80%|███████▉  | 179383/225000 [340:44:42<82:58:09,  6.55s/it]
 80%|███████▉  | 179384/225000 [340:44:45<71:16:37,  5.63s/it]
 80%|███████▉  | 179385/225000 [340:44:52<73:26:49,  5.80s/it]
 80%|███████▉  | 179386/225000 [340:44:57<71:57:18,  5.68s/it]
 80%|███████▉  | 179387/225000 [340:45:00<63:49:24,  5.04s/it]
 80%|███████▉  | 179388/225000 [340:45:08<72:00:03,  5.68s/it]
 80%|███████▉  | 179389/225000 [340:45:11<63:44:16,  5.03s/it]
 80%|███████▉  | 179390/225000 [340:45:16<63:27:19,  5.01s/it]
 80%|███████▉  | 179391/225000 [340:45:25<76:37:06,  6.05s/it]
 80%|███████▉  | 179392/225000 [340:45:28<68:03:46,  5.37s/it]
 80%|███████▉  | 179393/225000 [340:45:37<79:22:17,  6.27s/it]
 80%|███████▉  | 179394/225000 [340:45:41<71:09:27,  5.62s/it]
 80%|███████▉  | 179395/225000 [340:45:45<64:56:32,  5.13s/it]
 80%|███████▉  | 179396/225000 [340:45:49<61:52:52,  4.88s/it]
 80%|███████▉  | 179397/225000 [340:45:53<59:25:55,  4.69s/it]
 80%|███████▉  | 179398/225000 [340:46:04<83:48:35,  6.62s/it]
 80%|███████▉  | 179399/225000 [340:46:08<71:31:36,  5.65s/it]
 80%|███████▉  | 179400/225000 [340:46:11<63:22:45,  5.00s/it]
                                                              
{'loss': '0.7634', 'grad_norm': '1.086', 'learning_rate': '5.366e-06', 'epoch': '0.7973', 'num_input_tokens_seen': 564989168, 'train_runtime': '1.227e+06', 'train_tokens_per_second': '460.5'}

 80%|███████▉  | 179400/225000 [340:46:11<63:22:45,  5.00s/it]
 80%|███████▉  | 179401/225000 [340:46:16<63:35:15,  5.02s/it]
 80%|███████▉  | 179402/225000 [340:46:21<62:41:48,  4.95s/it]
 80%|███████▉  | 179403/225000 [340:46:26<61:44:45,  4.88s/it]
 80%|███████▉  | 179404/225000 [340:46:31<62:25:30,  4.93s/it]
 80%|███████▉  | 179405/225000 [340:46:36<61:21:27,  4.84s/it]
 80%|███████▉  | 179406/225000 [340:46:41<61:49:18,  4.88s/it]
 80%|███████▉  | 179407/225000 [340:46:45<61:49:56,  4.88s/it]
 80%|███████▉  | 179408/225000 [340:46:56<84:42:09,  6.69s/it]
 80%|███████▉  | 179409/225000 [340:47:05<90:22:43,  7.14s/it]
 80%|███████▉  | 179410/225000 [340:47:08<76:44:15,  6.06s/it]
 80%|███████▉  | 179411/225000 [340:47:13<74:05:19,  5.85s/it]
 80%|███████▉  | 179412/225000 [340:47:21<79:25:47,  6.27s/it]
 80%|███████▉  | 179413/225000 [340:47:24<69:26:42,  5.48s/it]
 80%|███████▉  | 179414/225000 [340:47:33<80:17:26,  6.34s/it]
 80%|███████▉  | 179415/225000 [340:47:36<69:00:59,  5.45s/it]
 80%|███████▉  | 179416/225000 [340:47:43<74:06:04,  5.85s/it]
 80%|███████▉  | 179417/225000 [340:47:46<65:40:26,  5.19s/it]
 80%|███████▉  | 179418/225000 [340:47:50<60:44:08,  4.80s/it]
 80%|███████▉  | 179419/225000 [340:47:55<61:18:05,  4.84s/it]
 80%|███████▉  | 179420/225000 [340:48:01<62:37:24,  4.95s/it]
 80%|███████▉  | 179421/225000 [340:48:05<61:31:06,  4.86s/it]
 80%|███████▉  | 179422/225000 [340:48:11<64:06:07,  5.06s/it]
 80%|███████▉  | 179423/225000 [340:48:16<65:33:02,  5.18s/it]
 80%|███████▉  | 179424/225000 [340:48:21<65:01:25,  5.14s/it]
 80%|███████▉  | 179425/225000 [340:48:28<71:56:07,  5.68s/it]
 80%|███████▉  | 179426/225000 [340:48:38<88:51:20,  7.02s/it]
 80%|███████▉  | 179427/225000 [340:48:42<76:28:45,  6.04s/it]
 80%|███████▉  | 179428/225000 [340:48:46<70:08:42,  5.54s/it]
 80%|███████▉  | 179429/225000 [340:48:53<75:35:10,  5.97s/it]
 80%|███████▉  | 179430/225000 [340:48:57<67:00:03,  5.29s/it]
 80%|███████▉  | 179431/225000 [340:49:01<62:06:07,  4.91s/it]
 80%|███████▉  | 179432/225000 [340:49:13<87:07:54,  6.88s/it]
 80%|███████▉  | 179433/225000 [340:49:16<72:17:28,  5.71s/it]
 80%|███████▉  | 179434/225000 [340:49:21<71:59:18,  5.69s/it]
 80%|███████▉  | 179435/225000 [340:49:27<74:04:08,  5.85s/it]
 80%|███████▉  | 179436/225000 [340:49:31<64:54:07,  5.13s/it]
 80%|███████▉  | 179437/225000 [340:49:35<61:39:18,  4.87s/it]
 80%|███████▉  | 179438/225000 [340:49:40<61:54:14,  4.89s/it]
 80%|███████▉  | 179439/225000 [340:49:44<59:38:37,  4.71s/it]
 80%|███████▉  | 179440/225000 [340:49:49<58:16:29,  4.60s/it]
 80%|███████▉  | 179441/225000 [340:49:53<58:24:11,  4.61s/it]
 80%|███████▉  | 179442/225000 [340:49:58<58:43:25,  4.64s/it]
 80%|███████▉  | 179443/225000 [340:50:05<68:56:44,  5.45s/it]
 80%|███████▉  | 179444/225000 [340:50:14<80:39:41,  6.37s/it]
 80%|███████▉  | 179445/225000 [340:50:17<69:54:33,  5.52s/it]
 80%|███████▉  | 179446/225000 [340:50:25<76:30:31,  6.05s/it]
 80%|███████▉  | 179447/225000 [340:50:30<71:56:48,  5.69s/it]
 80%|███████▉  | 179448/225000 [340:50:34<68:00:30,  5.37s/it]
 80%|███████▉  | 179449/225000 [340:50:39<65:48:17,  5.20s/it]
 80%|███████▉  | 179450/225000 [340:50:44<64:54:00,  5.13s/it]
                                                              
{'loss': '0.7515', 'grad_norm': '1.077', 'learning_rate': '5.355e-06', 'epoch': '0.7976', 'num_input_tokens_seen': 565118128, 'train_runtime': '1.227e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179450/225000 [340:50:44<64:54:00,  5.13s/it]
 80%|███████▉  | 179451/225000 [340:50:49<64:09:10,  5.07s/it]
 80%|███████▉  | 179452/225000 [340:50:59<84:45:48,  6.70s/it]
 80%|███████▉  | 179453/225000 [340:51:07<88:58:56,  7.03s/it]
 80%|███████▉  | 179454/225000 [340:51:12<78:44:06,  6.22s/it]
 80%|███████▉  | 179455/225000 [340:51:16<72:59:01,  5.77s/it]
 80%|███████▉  | 179456/225000 [340:51:21<70:47:33,  5.60s/it]
 80%|███████▉  | 179457/225000 [340:51:26<67:36:46,  5.34s/it]
 80%|███████▉  | 179458/225000 [340:51:31<66:51:07,  5.28s/it]
 80%|███████▉  | 179459/225000 [340:51:36<65:20:00,  5.16s/it]
 80%|███████▉  | 179460/225000 [340:51:41<64:21:51,  5.09s/it]
 80%|███████▉  | 179461/225000 [340:51:47<66:19:47,  5.24s/it]
 80%|███████▉  | 179462/225000 [340:51:52<66:09:17,  5.23s/it]
 80%|███████▉  | 179463/225000 [340:51:57<65:47:36,  5.20s/it]
 80%|███████▉  | 179464/225000 [340:52:02<64:41:14,  5.11s/it]
 80%|███████▉  | 179465/225000 [340:52:07<65:10:37,  5.15s/it]
 80%|███████▉  | 179466/225000 [340:52:19<91:56:15,  7.27s/it]
 80%|███████▉  | 179467/225000 [340:52:25<85:21:25,  6.75s/it]
 80%|███████▉  | 179468/225000 [340:52:29<74:56:51,  5.93s/it]
 80%|███████▉  | 179469/225000 [340:52:39<89:47:55,  7.10s/it]
 80%|███████▉  | 179470/225000 [340:52:43<76:41:30,  6.06s/it]
 80%|███████▉  | 179471/225000 [340:52:47<70:58:21,  5.61s/it]
 80%|███████▉  | 179472/225000 [340:52:56<83:31:56,  6.61s/it]
 80%|███████▉  | 179473/225000 [340:53:00<72:07:11,  5.70s/it]
 80%|███████▉  | 179474/225000 [340:53:03<64:57:18,  5.14s/it]
 80%|███████▉  | 179475/225000 [340:53:10<72:08:22,  5.70s/it]
 80%|███████▉  | 179476/225000 [340:53:16<72:58:29,  5.77s/it]
 80%|███████▉  | 179477/225000 [340:53:23<76:46:53,  6.07s/it]
 80%|███████▉  | 179478/225000 [340:53:28<72:03:32,  5.70s/it]
 80%|███████▉  | 179479/225000 [340:53:33<70:01:19,  5.54s/it]
 80%|███████▉  | 179480/225000 [340:53:38<69:02:17,  5.46s/it]
 80%|███████▉  | 179481/225000 [340:53:44<68:47:53,  5.44s/it]
 80%|███████▉  | 179482/225000 [340:53:48<64:46:03,  5.12s/it]
 80%|███████▉  | 179483/225000 [340:53:54<66:23:15,  5.25s/it]
 80%|███████▉  | 179484/225000 [340:53:59<65:34:58,  5.19s/it]
 80%|███████▉  | 179485/225000 [340:54:04<63:51:45,  5.05s/it]
 80%|███████▉  | 179486/225000 [340:54:08<63:21:19,  5.01s/it]
 80%|███████▉  | 179487/225000 [340:54:14<65:24:44,  5.17s/it]
 80%|███████▉  | 179488/225000 [340:54:19<63:33:28,  5.03s/it]
 80%|███████▉  | 179489/225000 [340:54:23<62:39:22,  4.96s/it]
 80%|███████▉  | 179490/225000 [340:54:29<63:08:47,  5.00s/it]
 80%|███████▉  | 179491/225000 [340:54:34<63:57:38,  5.06s/it]
 80%|███████▉  | 179492/225000 [340:54:39<62:56:47,  4.98s/it]
 80%|███████▉  | 179493/225000 [340:54:44<63:26:16,  5.02s/it]
 80%|███████▉  | 179494/225000 [340:54:49<64:15:59,  5.08s/it]
 80%|███████▉  | 179495/225000 [340:55:01<89:47:57,  7.10s/it]
 80%|███████▉  | 179496/225000 [340:55:06<82:22:43,  6.52s/it]
 80%|███████▉  | 179497/225000 [340:55:10<71:43:32,  5.67s/it]
 80%|███████▉  | 179498/225000 [340:55:16<75:43:36,  5.99s/it]
 80%|███████▉  | 179499/225000 [340:55:22<74:40:02,  5.91s/it]
 80%|███████▉  | 179500/225000 [340:55:30<81:10:59,  6.42s/it]
                                                              
{'loss': '0.7589', 'grad_norm': '1.105', 'learning_rate': '5.344e-06', 'epoch': '0.7978', 'num_input_tokens_seen': 565250080, 'train_runtime': '1.227e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179500/225000 [340:55:30<81:10:59,  6.42s/it]
 80%|███████▉  | 179501/225000 [340:55:33<70:14:48,  5.56s/it]
 80%|███████▉  | 179502/225000 [340:55:37<63:45:45,  5.05s/it]
 80%|███████▉  | 179503/225000 [340:55:44<71:22:30,  5.65s/it]
 80%|███████▉  | 179504/225000 [340:55:51<74:24:54,  5.89s/it]
 80%|███████▉  | 179505/225000 [340:55:54<65:16:26,  5.17s/it]
 80%|███████▉  | 179506/225000 [340:55:59<63:19:08,  5.01s/it]
 80%|███████▉  | 179507/225000 [340:56:06<72:56:15,  5.77s/it]
 80%|███████▉  | 179508/225000 [340:56:10<63:50:02,  5.05s/it]
 80%|███████▉  | 179509/225000 [340:56:17<72:45:03,  5.76s/it]
 80%|███████▉  | 179510/225000 [340:56:21<65:46:28,  5.21s/it]
 80%|███████▉  | 179511/225000 [340:56:24<59:25:20,  4.70s/it]
 80%|███████▉  | 179512/225000 [340:56:34<78:38:42,  6.22s/it]
 80%|███████▉  | 179513/225000 [340:56:38<68:25:39,  5.42s/it]
 80%|███████▉  | 179514/225000 [340:56:45<74:46:11,  5.92s/it]
 80%|███████▉  | 179515/225000 [340:56:48<65:59:58,  5.22s/it]
 80%|███████▉  | 179516/225000 [340:56:56<75:32:50,  5.98s/it]
 80%|███████▉  | 179517/225000 [340:57:00<68:11:50,  5.40s/it]
 80%|███████▉  | 179518/225000 [340:57:04<62:47:03,  4.97s/it]
 80%|███████▉  | 179519/225000 [340:57:08<59:56:02,  4.74s/it]
 80%|███████▉  | 179520/225000 [340:57:16<71:41:12,  5.67s/it]
 80%|███████▉  | 179521/225000 [340:57:24<80:58:13,  6.41s/it]
 80%|███████▉  | 179522/225000 [340:57:29<72:59:13,  5.78s/it]
 80%|███████▉  | 179523/225000 [340:57:33<67:55:56,  5.38s/it]
 80%|███████▉  | 179524/225000 [340:57:38<66:27:43,  5.26s/it]
 80%|███████▉  | 179525/225000 [340:57:43<64:24:29,  5.10s/it]
 80%|███████▉  | 179526/225000 [340:57:48<64:01:41,  5.07s/it]
 80%|███████▉  | 179527/225000 [340:57:52<61:39:10,  4.88s/it]
 80%|███████▉  | 179528/225000 [340:57:57<61:44:35,  4.89s/it]
 80%|███████▉  | 179529/225000 [340:58:02<61:23:12,  4.86s/it]
 80%|███████▉  | 179530/225000 [340:58:07<60:45:58,  4.81s/it]
 80%|███████▉  | 179531/225000 [340:58:12<61:10:01,  4.84s/it]
 80%|███████▉  | 179532/225000 [340:58:22<81:07:28,  6.42s/it]
 80%|███████▉  | 179533/225000 [340:58:29<86:23:18,  6.84s/it]
 80%|███████▉  | 179534/225000 [340:58:36<85:11:50,  6.75s/it]
 80%|███████▉  | 179535/225000 [340:58:40<73:28:22,  5.82s/it]
 80%|███████▉  | 179536/225000 [340:58:46<75:49:53,  6.00s/it]
 80%|███████▉  | 179537/225000 [340:58:50<66:31:56,  5.27s/it]
 80%|███████▉  | 179538/225000 [340:58:59<83:38:44,  6.62s/it]
 80%|███████▉  | 179539/225000 [340:59:03<71:48:03,  5.69s/it]
 80%|███████▉  | 179540/225000 [340:59:11<81:04:19,  6.42s/it]
 80%|███████▉  | 179541/225000 [340:59:16<73:35:06,  5.83s/it]
 80%|███████▉  | 179542/225000 [340:59:20<68:40:25,  5.44s/it]
 80%|███████▉  | 179543/225000 [340:59:24<64:45:14,  5.13s/it]
 80%|███████▉  | 179544/225000 [340:59:29<62:26:03,  4.94s/it]
 80%|███████▉  | 179545/225000 [340:59:34<60:59:34,  4.83s/it]
 80%|███████▉  | 179546/225000 [340:59:38<59:24:35,  4.71s/it]
 80%|███████▉  | 179547/225000 [340:59:42<58:01:36,  4.60s/it]
 80%|███████▉  | 179548/225000 [340:59:47<56:46:50,  4.50s/it]
 80%|███████▉  | 179549/225000 [340:59:54<66:31:17,  5.27s/it]
 80%|███████▉  | 179550/225000 [341:00:03<82:12:07,  6.51s/it]
                                                              
{'loss': '0.7622', 'grad_norm': '1.189', 'learning_rate': '5.332e-06', 'epoch': '0.798', 'num_input_tokens_seen': 565380392, 'train_runtime': '1.228e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179550/225000 [341:00:03<82:12:07,  6.51s/it]
 80%|███████▉  | 179551/225000 [341:00:06<69:39:16,  5.52s/it]
 80%|███████▉  | 179552/225000 [341:00:14<79:20:04,  6.28s/it]
 80%|███████▉  | 179553/225000 [341:00:18<67:56:04,  5.38s/it]
 80%|███████▉  | 179554/225000 [341:00:23<67:54:06,  5.38s/it]
 80%|███████▉  | 179555/225000 [341:00:31<77:20:55,  6.13s/it]
 80%|███████▉  | 179556/225000 [341:00:35<69:59:24,  5.54s/it]
 80%|███████▉  | 179557/225000 [341:00:39<62:31:01,  4.95s/it]
 80%|███████▉  | 179558/225000 [341:00:44<65:24:31,  5.18s/it]
 80%|███████▉  | 179559/225000 [341:00:52<74:32:43,  5.91s/it]
 80%|███████▉  | 179560/225000 [341:00:56<66:26:51,  5.26s/it]
 80%|███████▉  | 179561/225000 [341:01:00<63:09:10,  5.00s/it]
 80%|███████▉  | 179562/225000 [341:01:04<59:24:30,  4.71s/it]
 80%|███████▉  | 179563/225000 [341:01:14<80:42:55,  6.40s/it]
 80%|███████▉  | 179564/225000 [341:01:19<73:50:49,  5.85s/it]
 80%|███████▉  | 179565/225000 [341:01:22<64:21:41,  5.10s/it]
 80%|███████▉  | 179566/225000 [341:01:26<60:13:12,  4.77s/it]
 80%|███████▉  | 179567/225000 [341:01:30<56:46:13,  4.50s/it]
 80%|███████▉  | 179568/225000 [341:01:40<78:41:06,  6.23s/it]
 80%|███████▉  | 179569/225000 [341:01:44<68:58:48,  5.47s/it]
 80%|███████▉  | 179570/225000 [341:01:52<78:34:18,  6.23s/it]
 80%|███████▉  | 179571/225000 [341:01:56<68:04:08,  5.39s/it]
 80%|███████▉  | 179572/225000 [341:01:59<61:40:30,  4.89s/it]
 80%|███████▉  | 179573/225000 [341:02:09<78:20:03,  6.21s/it]
 80%|███████▉  | 179574/225000 [341:02:13<69:48:28,  5.53s/it]
 80%|███████▉  | 179575/225000 [341:02:17<64:43:59,  5.13s/it]
 80%|███████▉  | 179576/225000 [341:02:22<63:30:47,  5.03s/it]
 80%|███████▉  | 179577/225000 [341:02:31<80:06:21,  6.35s/it]
 80%|███████▉  | 179578/225000 [341:02:34<68:17:23,  5.41s/it]
 80%|███████▉  | 179579/225000 [341:02:39<65:15:18,  5.17s/it]
 80%|███████▉  | 179580/225000 [341:02:43<62:59:02,  4.99s/it]
 80%|███████▉  | 179581/225000 [341:02:48<61:38:23,  4.89s/it]
 80%|███████▉  | 179582/225000 [341:02:53<60:17:35,  4.78s/it]
 80%|███████▉  | 179583/225000 [341:02:57<59:39:57,  4.73s/it]
 80%|███████▉  | 179584/225000 [341:03:02<59:46:46,  4.74s/it]
 80%|███████▉  | 179585/225000 [341:03:15<90:54:16,  7.21s/it]
 80%|███████▉  | 179586/225000 [341:03:19<79:56:09,  6.34s/it]
 80%|███████▉  | 179587/225000 [341:03:24<73:00:34,  5.79s/it]
 80%|███████▉  | 179588/225000 [341:03:32<82:07:22,  6.51s/it]
 80%|███████▉  | 179589/225000 [341:03:35<70:53:33,  5.62s/it]
 80%|███████▉  | 179590/225000 [341:03:44<81:34:25,  6.47s/it]
 80%|███████▉  | 179591/225000 [341:03:47<70:09:26,  5.56s/it]
 80%|███████▉  | 179592/225000 [341:03:56<82:22:51,  6.53s/it]
 80%|███████▉  | 179593/225000 [341:03:59<69:54:25,  5.54s/it]
 80%|███████▉  | 179594/225000 [341:04:03<61:24:31,  4.87s/it]
 80%|███████▉  | 179595/225000 [341:04:08<62:26:41,  4.95s/it]
 80%|███████▉  | 179596/225000 [341:04:13<63:32:36,  5.04s/it]
 80%|███████▉  | 179597/225000 [341:04:18<62:08:11,  4.93s/it]
 80%|███████▉  | 179598/225000 [341:04:23<63:14:41,  5.01s/it]
 80%|███████▉  | 179599/225000 [341:04:28<62:16:32,  4.94s/it]
 80%|███████▉  | 179600/225000 [341:04:33<63:49:48,  5.06s/it]
                                                              
{'loss': '0.7603', 'grad_norm': '1.068', 'learning_rate': '5.321e-06', 'epoch': '0.7982', 'num_input_tokens_seen': 565508536, 'train_runtime': '1.228e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179600/225000 [341:04:33<63:49:48,  5.06s/it]
 80%|███████▉  | 179601/225000 [341:04:38<62:52:27,  4.99s/it]
 80%|███████▉  | 179602/225000 [341:04:43<63:45:39,  5.06s/it]
 80%|███████▉  | 179603/225000 [341:04:48<63:09:52,  5.01s/it]
 80%|███████▉  | 179604/225000 [341:04:53<63:10:12,  5.01s/it]
 80%|███████▉  | 179605/225000 [341:04:58<64:53:17,  5.15s/it]
 80%|███████▉  | 179606/225000 [341:05:04<64:43:42,  5.13s/it]
 80%|███████▉  | 179607/225000 [341:05:09<65:23:10,  5.19s/it]
 80%|███████▉  | 179608/225000 [341:05:14<65:56:09,  5.23s/it]
 80%|███████▉  | 179609/225000 [341:05:19<65:43:49,  5.21s/it]
 80%|███████▉  | 179610/225000 [341:05:25<65:36:36,  5.20s/it]
 80%|███████▉  | 179611/225000 [341:05:33<77:56:25,  6.18s/it]
 80%|███████▉  | 179612/225000 [341:05:44<97:56:40,  7.77s/it]
 80%|███████▉  | 179613/225000 [341:05:48<82:20:21,  6.53s/it]
 80%|███████▉  | 179614/225000 [341:05:52<73:20:15,  5.82s/it]
 80%|███████▉  | 179615/225000 [341:05:56<66:30:53,  5.28s/it]
 80%|███████▉  | 179616/225000 [341:06:00<62:17:21,  4.94s/it]
 80%|███████▉  | 179617/225000 [341:06:06<65:03:16,  5.16s/it]
 80%|███████▉  | 179618/225000 [341:06:11<64:55:56,  5.15s/it]
 80%|███████▉  | 179619/225000 [341:06:17<65:30:33,  5.20s/it]
 80%|███████▉  | 179620/225000 [341:06:22<66:01:03,  5.24s/it]
 80%|███████▉  | 179621/225000 [341:06:27<67:12:22,  5.33s/it]
 80%|███████▉  | 179622/225000 [341:06:33<67:08:28,  5.33s/it]
 80%|███████▉  | 179623/225000 [341:06:38<66:53:25,  5.31s/it]
 80%|███████▉  | 179624/225000 [341:06:43<66:41:00,  5.29s/it]
 80%|███████▉  | 179625/225000 [341:06:49<68:16:33,  5.42s/it]
 80%|███████▉  | 179626/225000 [341:06:54<67:41:51,  5.37s/it]
 80%|███████▉  | 179627/225000 [341:07:00<68:21:10,  5.42s/it]
 80%|███████▉  | 179628/225000 [341:07:05<67:52:41,  5.39s/it]
 80%|███████▉  | 179629/225000 [341:07:11<67:58:54,  5.39s/it]
 80%|███████▉  | 179630/225000 [341:07:16<66:33:11,  5.28s/it]
 80%|███████▉  | 179631/225000 [341:07:22<71:55:27,  5.71s/it]
 80%|███████▉  | 179632/225000 [341:07:33<92:42:52,  7.36s/it]
 80%|███████▉  | 179633/225000 [341:07:38<82:33:46,  6.55s/it]
 80%|███████▉  | 179634/225000 [341:07:43<74:32:04,  5.91s/it]
 80%|███████▉  | 179635/225000 [341:07:47<68:34:29,  5.44s/it]
 80%|███████▉  | 179636/225000 [341:07:51<64:40:00,  5.13s/it]
 80%|███████▉  | 179637/225000 [341:07:56<63:40:17,  5.05s/it]
 80%|███████▉  | 179638/225000 [341:08:01<62:11:02,  4.94s/it]
 80%|███████▉  | 179639/225000 [341:08:12<87:16:52,  6.93s/it]
 80%|███████▉  | 179640/225000 [341:08:17<79:26:55,  6.31s/it]
 80%|███████▉  | 179641/225000 [341:08:21<68:33:11,  5.44s/it]
 80%|███████▉  | 179642/225000 [341:08:30<83:43:18,  6.64s/it]
 80%|███████▉  | 179643/225000 [341:08:34<72:48:33,  5.78s/it]
 80%|███████▉  | 179644/225000 [341:08:38<64:46:56,  5.14s/it]
 80%|███████▉  | 179645/225000 [341:08:42<63:10:40,  5.01s/it]
 80%|███████▉  | 179646/225000 [341:08:47<61:17:08,  4.86s/it]
 80%|███████▉  | 179647/225000 [341:08:52<63:29:41,  5.04s/it]
 80%|███████▉  | 179648/225000 [341:09:04<90:14:16,  7.16s/it]
 80%|███████▉  | 179649/225000 [341:09:08<76:02:14,  6.04s/it]
 80%|███████▉  | 179650/225000 [341:09:13<72:30:47,  5.76s/it]
                                                              
{'loss': '0.7636', 'grad_norm': '1.243', 'learning_rate': '5.31e-06', 'epoch': '0.7984', 'num_input_tokens_seen': 565638568, 'train_runtime': '1.228e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179650/225000 [341:09:13<72:30:47,  5.76s/it]
 80%|███████▉  | 179651/225000 [341:09:18<70:22:28,  5.59s/it]
 80%|███████▉  | 179652/225000 [341:09:22<66:01:32,  5.24s/it]
 80%|███████▉  | 179653/225000 [341:09:27<65:09:47,  5.17s/it]
 80%|███████▉  | 179654/225000 [341:09:33<64:46:18,  5.14s/it]
 80%|███████▉  | 179655/225000 [341:09:37<63:55:06,  5.07s/it]
 80%|███████▉  | 179656/225000 [341:09:43<63:57:51,  5.08s/it]
 80%|███████▉  | 179657/225000 [341:09:47<61:39:15,  4.90s/it]
 80%|███████▉  | 179658/225000 [341:09:52<62:42:51,  4.98s/it]
 80%|███████▉  | 179659/225000 [341:09:57<62:28:29,  4.96s/it]
 80%|███████▉  | 179660/225000 [341:10:03<64:10:31,  5.10s/it]
 80%|███████▉  | 179661/225000 [341:10:08<64:07:23,  5.09s/it]
 80%|███████▉  | 179662/225000 [341:10:13<64:01:01,  5.08s/it]
 80%|███████▉  | 179663/225000 [341:10:18<63:50:31,  5.07s/it]
 80%|███████▉  | 179664/225000 [341:10:23<64:37:34,  5.13s/it]
 80%|███████▉  | 179665/225000 [341:10:28<65:14:24,  5.18s/it]
 80%|███████▉  | 179666/225000 [341:10:34<66:38:25,  5.29s/it]
 80%|███████▉  | 179667/225000 [341:10:39<66:44:28,  5.30s/it]
 80%|███████▉  | 179668/225000 [341:10:44<65:12:20,  5.18s/it]
 80%|███████▉  | 179669/225000 [341:10:56<91:22:59,  7.26s/it]
 80%|███████▉  | 179670/225000 [341:11:00<80:02:46,  6.36s/it]
 80%|███████▉  | 179671/225000 [341:11:05<71:35:42,  5.69s/it]
 80%|███████▉  | 179672/225000 [341:11:09<67:20:22,  5.35s/it]
 80%|███████▉  | 179673/225000 [341:11:13<63:30:20,  5.04s/it]
 80%|███████▉  | 179674/225000 [341:11:20<71:07:42,  5.65s/it]
 80%|███████▉  | 179675/225000 [341:11:29<80:47:59,  6.42s/it]
 80%|███████▉  | 179676/225000 [341:11:32<69:38:20,  5.53s/it]
 80%|███████▉  | 179677/225000 [341:11:36<61:55:27,  4.92s/it]
 80%|███████▉  | 179678/225000 [341:11:45<77:14:53,  6.14s/it]
 80%|███████▉  | 179679/225000 [341:11:49<69:48:30,  5.55s/it]
 80%|███████▉  | 179680/225000 [341:11:57<80:39:05,  6.41s/it]
 80%|███████▉  | 179681/225000 [341:12:02<75:58:49,  6.04s/it]
 80%|███████▉  | 179682/225000 [341:12:07<72:09:55,  5.73s/it]
 80%|███████▉  | 179683/225000 [341:12:14<74:34:22,  5.92s/it]
 80%|███████▉  | 179684/225000 [341:12:17<65:07:11,  5.17s/it]
 80%|███████▉  | 179685/225000 [341:12:23<67:57:06,  5.40s/it]
 80%|███████▉  | 179686/225000 [341:12:30<72:25:18,  5.75s/it]
 80%|███████▉  | 179687/225000 [341:12:34<67:29:30,  5.36s/it]
 80%|███████▉  | 179688/225000 [341:12:39<65:45:42,  5.22s/it]
 80%|███████▉  | 179689/225000 [341:12:44<62:52:29,  5.00s/it]
 80%|███████▉  | 179690/225000 [341:12:48<61:15:41,  4.87s/it]
 80%|███████▉  | 179691/225000 [341:12:52<59:17:59,  4.71s/it]
 80%|███████▉  | 179692/225000 [341:12:57<58:26:13,  4.64s/it]
 80%|███████▉  | 179693/225000 [341:13:07<78:22:04,  6.23s/it]
 80%|███████▉  | 179694/225000 [341:13:12<73:53:53,  5.87s/it]
 80%|███████▉  | 179695/225000 [341:13:20<82:33:25,  6.56s/it]
 80%|███████▉  | 179696/225000 [341:13:25<76:06:40,  6.05s/it]
 80%|███████▉  | 179697/225000 [341:13:30<72:24:40,  5.75s/it]
 80%|███████▉  | 179698/225000 [341:13:35<68:56:43,  5.48s/it]
 80%|███████▉  | 179699/225000 [341:13:40<66:24:21,  5.28s/it]
 80%|███████▉  | 179700/225000 [341:13:44<64:47:38,  5.15s/it]
                                                              
{'loss': '0.7594', 'grad_norm': '1.119', 'learning_rate': '5.299e-06', 'epoch': '0.7987', 'num_input_tokens_seen': 565768056, 'train_runtime': '1.228e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179700/225000 [341:13:44<64:47:38,  5.15s/it]
 80%|███████▉  | 179701/225000 [341:13:49<63:23:32,  5.04s/it]
 80%|███████▉  | 179702/225000 [341:13:54<62:05:57,  4.94s/it]
 80%|███████▉  | 179703/225000 [341:13:59<62:16:04,  4.95s/it]
 80%|███████▉  | 179704/225000 [341:14:05<67:45:32,  5.39s/it]
 80%|███████▉  | 179705/225000 [341:14:16<87:56:44,  6.99s/it]
 80%|███████▉  | 179706/225000 [341:14:20<74:44:22,  5.94s/it]
 80%|███████▉  | 179707/225000 [341:14:23<66:05:41,  5.25s/it]
 80%|███████▉  | 179708/225000 [341:14:32<78:56:25,  6.27s/it]
 80%|███████▉  | 179709/225000 [341:14:36<72:42:20,  5.78s/it]
 80%|███████▉  | 179710/225000 [341:14:41<67:34:52,  5.37s/it]
 80%|███████▉  | 179711/225000 [341:14:45<62:32:16,  4.97s/it]
 80%|███████▉  | 179712/225000 [341:14:50<62:01:15,  4.93s/it]
 80%|███████▉  | 179713/225000 [341:14:54<60:32:18,  4.81s/it]
 80%|███████▉  | 179714/225000 [341:15:05<82:50:38,  6.59s/it]
 80%|███████▉  | 179715/225000 [341:15:08<69:59:24,  5.56s/it]
 80%|███████▉  | 179716/225000 [341:15:15<74:07:58,  5.89s/it]
 80%|███████▉  | 179717/225000 [341:15:19<68:55:55,  5.48s/it]
 80%|███████▉  | 179718/225000 [341:15:24<64:48:28,  5.15s/it]
 80%|███████▉  | 179719/225000 [341:15:28<61:22:50,  4.88s/it]
 80%|███████▉  | 179720/225000 [341:15:36<74:07:39,  5.89s/it]
 80%|███████▉  | 179721/225000 [341:15:44<80:06:36,  6.37s/it]
 80%|███████▉  | 179722/225000 [341:15:47<69:25:45,  5.52s/it]
 80%|███████▉  | 179723/225000 [341:15:55<77:52:09,  6.19s/it]
 80%|███████▉  | 179724/225000 [341:16:00<74:37:39,  5.93s/it]
 80%|███████▉  | 179725/225000 [341:16:06<72:53:58,  5.80s/it]
 80%|███████▉  | 179726/225000 [341:16:12<72:17:45,  5.75s/it]
 80%|███████▉  | 179727/225000 [341:16:16<69:06:01,  5.49s/it]
 80%|███████▉  | 179728/225000 [341:16:25<79:30:18,  6.32s/it]
 80%|███████▉  | 179729/225000 [341:16:28<68:58:50,  5.49s/it]
 80%|███████▉  | 179730/225000 [341:16:32<62:32:16,  4.97s/it]
 80%|███████▉  | 179731/225000 [341:16:40<72:19:43,  5.75s/it]
 80%|███████▉  | 179732/225000 [341:16:45<70:09:18,  5.58s/it]
 80%|███████▉  | 179733/225000 [341:16:50<68:37:30,  5.46s/it]
 80%|███████▉  | 179734/225000 [341:16:55<66:09:55,  5.26s/it]
 80%|███████▉  | 179735/225000 [341:16:59<64:11:03,  5.10s/it]
 80%|███████▉  | 179736/225000 [341:17:04<63:16:09,  5.03s/it]
 80%|███████▉  | 179737/225000 [341:17:09<61:06:03,  4.86s/it]
 80%|███████▉  | 179738/225000 [341:17:14<61:19:59,  4.88s/it]
 80%|███████▉  | 179739/225000 [341:17:19<62:16:09,  4.95s/it]
 80%|███████▉  | 179740/225000 [341:17:24<61:19:17,  4.88s/it]
 80%|███████▉  | 179741/225000 [341:17:29<62:09:38,  4.94s/it]
 80%|███████▉  | 179742/225000 [341:17:39<81:35:10,  6.49s/it]
 80%|███████▉  | 179743/225000 [341:17:48<90:55:32,  7.23s/it]
 80%|███████▉  | 179744/225000 [341:17:51<76:38:27,  6.10s/it]
 80%|███████▉  | 179745/225000 [341:17:56<71:36:16,  5.70s/it]
 80%|███████▉  | 179746/225000 [341:18:01<68:27:23,  5.45s/it]
 80%|███████▉  | 179747/225000 [341:18:06<66:24:25,  5.28s/it]
 80%|███████▉  | 179748/225000 [341:18:11<65:05:19,  5.18s/it]
 80%|███████▉  | 179749/225000 [341:18:15<63:48:15,  5.08s/it]
 80%|███████▉  | 179750/225000 [341:18:21<63:55:29,  5.09s/it]
                                                              
{'loss': '0.7595', 'grad_norm': '1.114', 'learning_rate': '5.287e-06', 'epoch': '0.7989', 'num_input_tokens_seen': 565899136, 'train_runtime': '1.229e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179750/225000 [341:18:21<63:55:29,  5.09s/it]
 80%|███████▉  | 179751/225000 [341:18:29<77:19:25,  6.15s/it]
 80%|███████▉  | 179752/225000 [341:18:37<84:13:38,  6.70s/it]
 80%|███████▉  | 179753/225000 [341:18:40<71:01:29,  5.65s/it]
 80%|███████▉  | 179754/225000 [341:18:48<78:47:37,  6.27s/it]
 80%|███████▉  | 179755/225000 [341:18:52<68:24:08,  5.44s/it]
 80%|███████▉  | 179756/225000 [341:18:58<72:22:17,  5.76s/it]
 80%|███████▉  | 179757/225000 [341:19:03<69:23:21,  5.52s/it]
 80%|███████▉  | 179758/225000 [341:19:08<67:57:26,  5.41s/it]
 80%|███████▉  | 179759/225000 [341:19:13<65:57:32,  5.25s/it]
 80%|███████▉  | 179760/225000 [341:19:18<65:34:22,  5.22s/it]
 80%|███████▉  | 179761/225000 [341:19:24<66:32:07,  5.29s/it]
 80%|███████▉  | 179762/225000 [341:19:29<65:38:02,  5.22s/it]
 80%|███████▉  | 179763/225000 [341:19:34<65:46:08,  5.23s/it]
 80%|███████▉  | 179764/225000 [341:19:39<66:12:54,  5.27s/it]
 80%|███████▉  | 179765/225000 [341:19:45<66:30:50,  5.29s/it]
 80%|███████▉  | 179766/225000 [341:19:50<65:59:08,  5.25s/it]
 80%|███████▉  | 179767/225000 [341:19:55<66:34:46,  5.30s/it]
 80%|███████▉  | 179768/225000 [341:20:01<66:50:02,  5.32s/it]
 80%|███████▉  | 179769/225000 [341:20:06<68:13:29,  5.43s/it]
 80%|███████▉  | 179770/225000 [341:20:11<67:08:27,  5.34s/it]
 80%|███████▉  | 179771/225000 [341:20:17<67:00:36,  5.33s/it]
 80%|███████▉  | 179772/225000 [341:20:22<66:47:42,  5.32s/it]
 80%|███████▉  | 179773/225000 [341:20:31<80:42:49,  6.42s/it]
 80%|███████▉  | 179774/225000 [341:20:38<82:17:40,  6.55s/it]
 80%|███████▉  | 179775/225000 [341:20:42<72:28:01,  5.77s/it]
 80%|███████▉  | 179776/225000 [341:20:47<69:36:58,  5.54s/it]
 80%|███████▉  | 179777/225000 [341:20:52<67:06:00,  5.34s/it]
 80%|███████▉  | 179778/225000 [341:20:57<65:50:15,  5.24s/it]
 80%|███████▉  | 179779/225000 [341:21:02<66:27:13,  5.29s/it]
 80%|███████▉  | 179780/225000 [341:21:07<65:26:58,  5.21s/it]
 80%|███████▉  | 179781/225000 [341:21:12<64:43:05,  5.15s/it]
 80%|███████▉  | 179782/225000 [341:21:17<64:03:54,  5.10s/it]
 80%|███████▉  | 179783/225000 [341:21:23<65:58:46,  5.25s/it]
 80%|███████▉  | 179784/225000 [341:21:28<64:32:26,  5.14s/it]
 80%|███████▉  | 179785/225000 [341:21:33<65:25:16,  5.21s/it]
 80%|███████▉  | 179786/225000 [341:21:38<66:21:29,  5.28s/it]
 80%|███████▉  | 179787/225000 [341:21:45<69:35:23,  5.54s/it]
 80%|███████▉  | 179788/225000 [341:21:50<68:31:47,  5.46s/it]
 80%|███████▉  | 179789/225000 [341:22:02<94:14:00,  7.50s/it]
 80%|███████▉  | 179790/225000 [341:22:09<90:40:41,  7.22s/it]
 80%|███████▉  | 179791/225000 [341:22:13<80:06:27,  6.38s/it]
 80%|███████▉  | 179792/225000 [341:22:18<72:52:08,  5.80s/it]
 80%|███████▉  | 179793/225000 [341:22:22<67:39:50,  5.39s/it]
 80%|███████▉  | 179794/225000 [341:22:26<63:15:12,  5.04s/it]
 80%|███████▉  | 179795/225000 [341:22:30<60:04:28,  4.78s/it]
 80%|███████▉  | 179796/225000 [341:22:40<78:22:31,  6.24s/it]
 80%|███████▉  | 179797/225000 [341:22:47<79:28:07,  6.33s/it]
 80%|███████▉  | 179798/225000 [341:22:52<74:58:02,  5.97s/it]
 80%|███████▉  | 179799/225000 [341:22:57<73:43:46,  5.87s/it]
 80%|███████▉  | 179800/225000 [341:23:03<71:56:27,  5.73s/it]
                                                              
{'loss': '0.7609', 'grad_norm': '1.132', 'learning_rate': '5.276e-06', 'epoch': '0.7991', 'num_input_tokens_seen': 566029272, 'train_runtime': '1.229e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179800/225000 [341:23:03<71:56:27,  5.73s/it]
 80%|███████▉  | 179801/225000 [341:23:09<75:13:57,  5.99s/it]
 80%|███████▉  | 179802/225000 [341:23:14<70:49:49,  5.64s/it]
 80%|███████▉  | 179803/225000 [341:23:21<74:46:37,  5.96s/it]
 80%|███████▉  | 179804/225000 [341:23:24<64:50:00,  5.16s/it]
 80%|███████▉  | 179805/225000 [341:23:31<69:15:45,  5.52s/it]
 80%|███████▉  | 179806/225000 [341:23:34<62:50:34,  5.01s/it]
 80%|███████▉  | 179807/225000 [341:23:38<58:48:54,  4.69s/it]
 80%|███████▉  | 179808/225000 [341:23:48<78:25:15,  6.25s/it]
 80%|███████▉  | 179809/225000 [341:23:52<70:07:16,  5.59s/it]
 80%|███████▉  | 179810/225000 [341:23:56<63:30:11,  5.06s/it]
 80%|███████▉  | 179811/225000 [341:24:00<57:58:22,  4.62s/it]
 80%|███████▉  | 179812/225000 [341:24:09<76:26:46,  6.09s/it]
 80%|███████▉  | 179813/225000 [341:24:13<68:57:11,  5.49s/it]
 80%|███████▉  | 179814/225000 [341:24:16<60:19:06,  4.81s/it]
 80%|███████▉  | 179815/225000 [341:24:20<57:05:43,  4.55s/it]
 80%|███████▉  | 179816/225000 [341:24:30<77:53:19,  6.21s/it]
 80%|███████▉  | 179817/225000 [341:24:35<71:19:17,  5.68s/it]
 80%|███████▉  | 179818/225000 [341:24:40<67:14:14,  5.36s/it]
 80%|███████▉  | 179819/225000 [341:24:44<65:16:00,  5.20s/it]
 80%|███████▉  | 179820/225000 [341:24:49<64:31:47,  5.14s/it]
 80%|███████▉  | 179821/225000 [341:24:54<62:27:06,  4.98s/it]
 80%|███████▉  | 179822/225000 [341:24:59<61:34:05,  4.91s/it]
 80%|███████▉  | 179823/225000 [341:25:04<61:10:10,  4.87s/it]
 80%|███████▉  | 179824/225000 [341:25:08<59:48:12,  4.77s/it]
 80%|███████▉  | 179825/225000 [341:25:16<71:04:16,  5.66s/it]
 80%|███████▉  | 179826/225000 [341:25:27<90:11:56,  7.19s/it]
 80%|███████▉  | 179827/225000 [341:25:31<78:29:28,  6.26s/it]
 80%|███████▉  | 179828/225000 [341:25:35<71:12:57,  5.68s/it]
 80%|███████▉  | 179829/225000 [341:25:39<66:54:23,  5.33s/it]
 80%|███████▉  | 179830/225000 [341:25:51<88:43:08,  7.07s/it]
 80%|███████▉  | 179831/225000 [341:25:54<73:33:30,  5.86s/it]
 80%|███████▉  | 179832/225000 [341:25:59<71:08:29,  5.67s/it]
 80%|███████▉  | 179833/225000 [341:26:04<68:57:39,  5.50s/it]
 80%|███████▉  | 179834/225000 [341:26:09<68:09:44,  5.43s/it]
 80%|███████▉  | 179835/225000 [341:26:15<67:54:35,  5.41s/it]
 80%|███████▉  | 179836/225000 [341:26:20<67:30:02,  5.38s/it]
 80%|███████▉  | 179837/225000 [341:26:25<65:23:53,  5.21s/it]
 80%|███████▉  | 179838/225000 [341:26:30<64:40:30,  5.16s/it]
 80%|███████▉  | 179839/225000 [341:26:35<65:39:53,  5.23s/it]
 80%|███████▉  | 179840/225000 [341:26:41<66:08:37,  5.27s/it]
 80%|███████▉  | 179841/225000 [341:26:46<66:38:25,  5.31s/it]
 80%|███████▉  | 179842/225000 [341:26:51<66:16:54,  5.28s/it]
 80%|███████▉  | 179843/225000 [341:26:56<65:28:44,  5.22s/it]
 80%|███████▉  | 179844/225000 [341:27:02<66:04:54,  5.27s/it]
 80%|███████▉  | 179845/225000 [341:27:07<66:17:58,  5.29s/it]
 80%|███████▉  | 179846/225000 [341:27:12<66:45:02,  5.32s/it]
 80%|███████▉  | 179847/225000 [341:27:18<67:17:22,  5.36s/it]
 80%|███████▉  | 179848/225000 [341:27:23<68:05:46,  5.43s/it]
 80%|███████▉  | 179849/225000 [341:27:29<67:30:34,  5.38s/it]
 80%|███████▉  | 179850/225000 [341:27:34<66:16:15,  5.28s/it]
                                                              
{'loss': '0.764', 'grad_norm': '1.192', 'learning_rate': '5.265e-06', 'epoch': '0.7993', 'num_input_tokens_seen': 566159592, 'train_runtime': '1.229e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179850/225000 [341:27:34<66:16:15,  5.28s/it]
 80%|███████▉  | 179851/225000 [341:27:39<65:52:43,  5.25s/it]
 80%|███████▉  | 179852/225000 [341:27:44<66:58:29,  5.34s/it]
 80%|███████▉  | 179853/225000 [341:27:50<67:25:06,  5.38s/it]
 80%|███████▉  | 179854/225000 [341:27:55<66:28:53,  5.30s/it]
 80%|███████▉  | 179855/225000 [341:28:00<66:49:03,  5.33s/it]
 80%|███████▉  | 179856/225000 [341:28:06<67:21:03,  5.37s/it]
 80%|███████▉  | 179857/225000 [341:28:11<66:52:07,  5.33s/it]
 80%|███████▉  | 179858/225000 [341:28:17<67:20:22,  5.37s/it]
 80%|███████▉  | 179859/225000 [341:28:22<67:24:53,  5.38s/it]
 80%|███████▉  | 179860/225000 [341:28:27<65:59:34,  5.26s/it]
 80%|███████▉  | 179861/225000 [341:28:39<91:59:15,  7.34s/it]
 80%|███████▉  | 179862/225000 [341:28:45<87:19:06,  6.96s/it]
 80%|███████▉  | 179863/225000 [341:28:50<77:27:44,  6.18s/it]
 80%|███████▉  | 179864/225000 [341:28:54<69:05:02,  5.51s/it]
 80%|███████▉  | 179865/225000 [341:28:58<63:50:33,  5.09s/it]
 80%|███████▉  | 179866/225000 [341:29:02<60:20:01,  4.81s/it]
 80%|███████▉  | 179867/225000 [341:29:13<82:32:46,  6.58s/it]
 80%|███████▉  | 179868/225000 [341:29:17<72:42:09,  5.80s/it]
 80%|███████▉  | 179869/225000 [341:29:20<65:12:52,  5.20s/it]
 80%|███████▉  | 179870/225000 [341:29:28<74:52:34,  5.97s/it]
 80%|███████▉  | 179871/225000 [341:29:34<74:48:36,  5.97s/it]
 80%|███████▉  | 179872/225000 [341:29:41<77:40:29,  6.20s/it]
 80%|███████▉  | 179873/225000 [341:29:45<71:27:26,  5.70s/it]
 80%|███████▉  | 179874/225000 [341:29:51<70:01:53,  5.59s/it]
 80%|███████▉  | 179875/225000 [341:29:56<69:35:04,  5.55s/it]
 80%|███████▉  | 179876/225000 [341:30:01<67:54:59,  5.42s/it]
 80%|███████▉  | 179877/225000 [341:30:06<67:19:41,  5.37s/it]
 80%|███████▉  | 179878/225000 [341:30:12<66:36:00,  5.31s/it]
 80%|███████▉  | 179879/225000 [341:30:17<65:54:05,  5.26s/it]
 80%|███████▉  | 179880/225000 [341:30:22<66:10:13,  5.28s/it]
 80%|███████▉  | 179881/225000 [341:30:27<64:38:54,  5.16s/it]
 80%|███████▉  | 179882/225000 [341:30:32<65:32:14,  5.23s/it]
 80%|███████▉  | 179883/225000 [341:30:38<65:59:19,  5.27s/it]
 80%|███████▉  | 179884/225000 [341:30:43<66:13:15,  5.28s/it]
 80%|███████▉  | 179885/225000 [341:30:48<66:16:22,  5.29s/it]
 80%|███████▉  | 179886/225000 [341:30:53<65:14:49,  5.21s/it]
 80%|███████▉  | 179887/225000 [341:30:59<66:17:44,  5.29s/it]
 80%|███████▉  | 179888/225000 [341:31:04<64:44:36,  5.17s/it]
 80%|███████▉  | 179889/225000 [341:31:09<65:21:59,  5.22s/it]
 80%|███████▉  | 179890/225000 [341:31:14<65:47:52,  5.25s/it]
 80%|███████▉  | 179891/225000 [341:31:19<64:46:03,  5.17s/it]
 80%|███████▉  | 179892/225000 [341:31:24<63:59:53,  5.11s/it]
 80%|███████▉  | 179893/225000 [341:31:30<64:41:08,  5.16s/it]
 80%|███████▉  | 179894/225000 [341:31:35<65:01:03,  5.19s/it]
 80%|███████▉  | 179895/225000 [341:31:40<64:23:20,  5.14s/it]
 80%|███████▉  | 179896/225000 [341:31:47<70:15:21,  5.61s/it]
 80%|███████▉  | 179897/225000 [341:31:58<93:32:00,  7.47s/it]
 80%|███████▉  | 179898/225000 [341:32:02<80:05:15,  6.39s/it]
 80%|███████▉  | 179899/225000 [341:32:06<70:43:16,  5.65s/it]
 80%|███████▉  | 179900/225000 [341:32:10<64:17:08,  5.13s/it]
                                                              
{'loss': '0.7589', 'grad_norm': '1.118', 'learning_rate': '5.254e-06', 'epoch': '0.7996', 'num_input_tokens_seen': 566290232, 'train_runtime': '1.23e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179900/225000 [341:32:10<64:17:08,  5.13s/it]
 80%|███████▉  | 179901/225000 [341:32:22<87:45:07,  7.00s/it]
 80%|███████▉  | 179902/225000 [341:32:25<74:17:00,  5.93s/it]
 80%|███████▉  | 179903/225000 [341:32:31<76:05:51,  6.07s/it]
 80%|███████▉  | 179904/225000 [341:32:35<66:35:59,  5.32s/it]
 80%|███████▉  | 179905/225000 [341:32:42<73:03:05,  5.83s/it]
 80%|███████▉  | 179906/225000 [341:32:49<77:46:45,  6.21s/it]
 80%|███████▉  | 179907/225000 [341:32:52<66:35:08,  5.32s/it]
 80%|███████▉  | 179908/225000 [341:33:00<74:16:43,  5.93s/it]
 80%|███████▉  | 179909/225000 [341:33:05<73:31:20,  5.87s/it]
 80%|███████▉  | 179910/225000 [341:33:11<72:44:17,  5.81s/it]
 80%|███████▉  | 179911/225000 [341:33:16<69:14:55,  5.53s/it]
 80%|███████▉  | 179912/225000 [341:33:21<66:03:13,  5.27s/it]
 80%|███████▉  | 179913/225000 [341:33:26<65:14:23,  5.21s/it]
 80%|███████▉  | 179914/225000 [341:33:31<64:17:00,  5.13s/it]
 80%|███████▉  | 179915/225000 [341:33:35<62:46:58,  5.01s/it]
 80%|███████▉  | 179916/225000 [341:33:40<62:57:17,  5.03s/it]
 80%|███████▉  | 179917/225000 [341:33:45<63:15:09,  5.05s/it]
 80%|███████▉  | 179918/225000 [341:33:50<62:01:57,  4.95s/it]
 80%|███████▉  | 179919/225000 [341:33:55<60:37:31,  4.84s/it]
 80%|███████▉  | 179920/225000 [341:34:00<61:18:37,  4.90s/it]
 80%|███████▉  | 179921/225000 [341:34:05<61:52:28,  4.94s/it]
 80%|███████▉  | 179922/225000 [341:34:10<62:52:36,  5.02s/it]
 80%|███████▉  | 179923/225000 [341:34:21<83:52:28,  6.70s/it]
 80%|███████▉  | 179924/225000 [341:34:29<88:54:46,  7.10s/it]
 80%|███████▉  | 179925/225000 [341:34:32<74:40:02,  5.96s/it]
 80%|███████▉  | 179926/225000 [341:34:39<79:45:53,  6.37s/it]
 80%|███████▉  | 179927/225000 [341:34:43<68:36:39,  5.48s/it]
 80%|███████▉  | 179928/225000 [341:34:51<80:26:37,  6.43s/it]
 80%|███████▉  | 179929/225000 [341:34:56<72:51:40,  5.82s/it]
 80%|███████▉  | 179930/225000 [341:35:02<73:17:30,  5.85s/it]
 80%|███████▉  | 179931/225000 [341:35:05<64:26:51,  5.15s/it]
 80%|███████▉  | 179932/225000 [341:35:13<73:13:41,  5.85s/it]
 80%|███████▉  | 179933/225000 [341:35:17<66:28:43,  5.31s/it]
 80%|███████▉  | 179934/225000 [341:35:21<61:19:14,  4.90s/it]
 80%|███████▉  | 179935/225000 [341:35:30<76:52:46,  6.14s/it]
 80%|███████▉  | 179936/225000 [341:35:33<67:10:52,  5.37s/it]
 80%|███████▉  | 179937/225000 [341:35:40<72:31:24,  5.79s/it]
 80%|███████▉  | 179938/225000 [341:35:45<71:02:28,  5.68s/it]
 80%|███████▉  | 179939/225000 [341:35:52<75:37:44,  6.04s/it]
 80%|███████▉  | 179940/225000 [341:35:56<66:20:48,  5.30s/it]
 80%|███████▉  | 179941/225000 [341:36:00<60:16:16,  4.82s/it]
 80%|███████▉  | 179942/225000 [341:36:09<77:59:44,  6.23s/it]
 80%|███████▉  | 179943/225000 [341:36:14<72:07:58,  5.76s/it]
 80%|███████▉  | 179944/225000 [341:36:17<63:46:12,  5.10s/it]
 80%|███████▉  | 179945/225000 [341:36:26<76:11:05,  6.09s/it]
 80%|███████▉  | 179946/225000 [341:36:30<67:56:31,  5.43s/it]
 80%|███████▉  | 179947/225000 [341:36:34<62:36:36,  5.00s/it]
 80%|███████▉  | 179948/225000 [341:36:38<59:18:29,  4.74s/it]
 80%|███████▉  | 179949/225000 [341:36:42<56:40:50,  4.53s/it]
 80%|███████▉  | 179950/225000 [341:36:53<81:19:24,  6.50s/it]
                                                              
{'loss': '0.765', 'grad_norm': '1.139', 'learning_rate': '5.243e-06', 'epoch': '0.7998', 'num_input_tokens_seen': 566418928, 'train_runtime': '1.23e+06', 'train_tokens_per_second': '460.6'}

 80%|███████▉  | 179950/225000 [341:36:53<81:19:24,  6.50s/it]
 80%|███████▉  | 179951/225000 [341:36:57<73:18:05,  5.86s/it]
 80%|███████▉  | 179952/225000 [341:37:02<68:45:55,  5.50s/it]
 80%|███████▉  | 179953/225000 [341:37:06<64:26:10,  5.15s/it]
 80%|███████▉  | 179954/225000 [341:37:11<61:48:07,  4.94s/it]
 80%|███████▉  | 179955/225000 [341:37:15<59:08:07,  4.73s/it]
 80%|███████▉  | 179956/225000 [341:37:20<59:22:26,  4.75s/it]
 80%|███████▉  | 179957/225000 [341:37:24<58:43:48,  4.69s/it]
 80%|███████▉  | 179958/225000 [341:37:34<77:46:46,  6.22s/it]
 80%|███████▉  | 179959/225000 [341:37:42<83:29:54,  6.67s/it]
 80%|███████▉  | 179960/225000 [341:37:46<72:43:12,  5.81s/it]
 80%|███████▉  | 179961/225000 [341:37:50<65:51:29,  5.26s/it]
 80%|███████▉  | 179962/225000 [341:37:59<80:33:55,  6.44s/it]
 80%|███████▉  | 179963/225000 [341:38:02<69:09:34,  5.53s/it]
 80%|███████▉  | 179964/225000 [341:38:07<67:35:30,  5.40s/it]
 80%|███████▉  | 179965/225000 [341:38:13<69:12:09,  5.53s/it]
 80%|███████▉  | 179966/225000 [341:38:17<61:46:57,  4.94s/it]
 80%|███████▉  | 179967/225000 [341:38:21<59:25:04,  4.75s/it]
 80%|███████▉  | 179968/225000 [341:38:26<61:57:58,  4.95s/it]
 80%|███████▉  | 179969/225000 [341:38:32<62:21:47,  4.99s/it]
 80%|███████▉  | 179970/225000 [341:38:41<77:41:57,  6.21s/it]
 80%|███████▉  | 179971/225000 [341:38:44<68:24:57,  5.47s/it]
 80%|███████▉  | 179972/225000 [341:38:48<62:45:55,  5.02s/it]
 80%|███████▉  | 179973/225000 [341:38:57<78:04:20,  6.24s/it]
 80%|███████▉  | 179974/225000 [341:39:02<71:16:21,  5.70s/it]
 80%|███████▉  | 179975/225000 [341:39:06<65:04:25,  5.20s/it]
 80%|███████▉  | 179976/225000 [341:39:10<62:32:02,  5.00s/it]
 80%|███████▉  | 179977/225000 [341:39:15<61:40:26,  4.93s/it]
 80%|███████▉  | 179978/225000 [341:39:20<60:01:06,  4.80s/it]
 80%|███████▉  | 179979/225000 [341:39:24<59:12:25,  4.73s/it]
 80%|███████▉  | 179980/225000 [341:39:32<69:46:28,  5.58s/it]
 80%|███████▉  | 179981/225000 [341:39:40<79:19:51,  6.34s/it]
 80%|███████▉  | 179982/225000 [341:39:44<72:24:34,  5.79s/it]
 80%|███████▉  | 179983/225000 [341:39:49<67:07:11,  5.37s/it]
 80%|███████▉  | 179984/225000 [341:39:53<61:52:34,  4.95s/it]
 80%|███████▉  | 179985/225000 [341:39:58<62:56:25,  5.03s/it]
 80%|███████▉  | 179986/225000 [341:40:09<85:39:44,  6.85s/it]
 80%|███████▉  | 179987/225000 [341:40:13<73:52:00,  5.91s/it]
 80%|███████▉  | 179988/225000 [341:40:16<64:17:27,  5.14s/it]
 80%|███████▉  | 179989/225000 [341:40:25<76:57:05,  6.15s/it]
 80%|███████▉  | 179990/225000 [341:40:29<69:24:44,  5.55s/it]
 80%|███████▉  | 179991/225000 [341:40:33<65:35:42,  5.25s/it]
 80%|███████▉  | 179992/225000 [341:40:37<60:58:00,  4.88s/it]
 80%|███████▉  | 179993/225000 [341:40:47<79:41:45,  6.37s/it]
 80%|███████▉  | 179994/225000 [341:40:51<71:37:59,  5.73s/it]
 80%|███████▉  | 179995/225000 [341:40:55<64:55:33,  5.19s/it]
 80%|███████▉  | 179996/225000 [341:41:00<62:00:50,  4.96s/it]
 80%|███████▉  | 179997/225000 [341:41:04<60:32:32,  4.84s/it]
 80%|███████▉  | 179998/225000 [341:41:10<61:53:19,  4.95s/it]
 80%|███████▉  | 179999/225000 [341:41:17<71:48:34,  5.74s/it]
 80%|████████  | 180000/225000 [341:41:27<85:29:44,  6.84s/it]
                                                              
{'loss': '0.7673', 'grad_norm': '1.033', 'learning_rate': '5.231e-06', 'epoch': '0.8', 'num_input_tokens_seen': 566549208, 'train_runtime': '1.23e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180000/225000 [341:41:27<85:29:44,  6.84s/it]
 80%|████████  | 180001/225000 [341:41:30<72:48:53,  5.83s/it]
 80%|████████  | 180002/225000 [341:41:34<65:29:16,  5.24s/it]
 80%|████████  | 180003/225000 [341:41:44<83:44:38,  6.70s/it]
 80%|████████  | 180004/225000 [341:41:48<74:19:33,  5.95s/it]
 80%|████████  | 180005/225000 [341:41:53<68:20:16,  5.47s/it]
 80%|████████  | 180006/225000 [341:41:57<63:36:46,  5.09s/it]
 80%|████████  | 180007/225000 [341:42:02<65:04:00,  5.21s/it]
 80%|████████  | 180008/225000 [341:42:07<63:33:21,  5.09s/it]
 80%|████████  | 180009/225000 [341:42:12<63:50:10,  5.11s/it]
 80%|████████  | 180010/225000 [341:42:17<64:30:23,  5.16s/it]
 80%|████████  | 180011/225000 [341:42:23<65:11:45,  5.22s/it]
 80%|████████  | 180012/225000 [341:42:29<67:10:03,  5.37s/it]
 80%|████████  | 180013/225000 [341:42:34<67:57:43,  5.44s/it]
 80%|████████  | 180014/225000 [341:42:39<66:14:42,  5.30s/it]
 80%|████████  | 180015/225000 [341:42:44<64:30:51,  5.16s/it]
 80%|████████  | 180016/225000 [341:42:49<65:32:31,  5.25s/it]
 80%|████████  | 180017/225000 [341:42:55<65:09:52,  5.22s/it]
 80%|████████  | 180018/225000 [341:43:00<65:28:09,  5.24s/it]
 80%|████████  | 180019/225000 [341:43:05<65:08:20,  5.21s/it]
 80%|████████  | 180020/225000 [341:43:10<65:46:06,  5.26s/it]
 80%|████████  | 180021/225000 [341:43:16<66:31:22,  5.32s/it]
 80%|████████  | 180022/225000 [341:43:21<65:57:45,  5.28s/it]
 80%|████████  | 180023/225000 [341:43:26<66:15:56,  5.30s/it]
 80%|████████  | 180024/225000 [341:43:35<79:14:19,  6.34s/it]
 80%|████████  | 180025/225000 [341:43:45<93:40:35,  7.50s/it]
 80%|████████  | 180026/225000 [341:43:49<78:51:26,  6.31s/it]
 80%|████████  | 180027/225000 [341:43:53<68:43:19,  5.50s/it]
 80%|████████  | 180028/225000 [341:44:02<83:55:48,  6.72s/it]
 80%|████████  | 180029/225000 [341:44:06<71:41:15,  5.74s/it]
 80%|████████  | 180030/225000 [341:44:12<75:05:17,  6.01s/it]
 80%|████████  | 180031/225000 [341:44:16<67:03:00,  5.37s/it]
 80%|████████  | 180032/225000 [341:44:22<70:49:56,  5.67s/it]
 80%|████████  | 180033/225000 [341:44:27<68:37:04,  5.49s/it]
 80%|████████  | 180034/225000 [341:44:32<63:25:21,  5.08s/it]
 80%|████████  | 180035/225000 [341:44:37<63:24:16,  5.08s/it]
 80%|████████  | 180036/225000 [341:44:42<64:51:48,  5.19s/it]
 80%|████████  | 180037/225000 [341:44:47<63:28:34,  5.08s/it]
 80%|████████  | 180038/225000 [341:44:52<64:03:08,  5.13s/it]
 80%|████████  | 180039/225000 [341:44:57<64:38:34,  5.18s/it]
 80%|████████  | 180040/225000 [341:45:03<64:32:03,  5.17s/it]
 80%|████████  | 180041/225000 [341:45:07<63:05:11,  5.05s/it]
 80%|████████  | 180042/225000 [341:45:13<63:49:51,  5.11s/it]
 80%|████████  | 180043/225000 [341:45:18<63:44:26,  5.10s/it]
 80%|████████  | 180044/225000 [341:45:23<64:59:45,  5.20s/it]
 80%|████████  | 180045/225000 [341:45:28<64:52:35,  5.20s/it]
 80%|████████  | 180046/225000 [341:45:34<65:13:24,  5.22s/it]
 80%|████████  | 180047/225000 [341:45:39<65:07:35,  5.22s/it]
 80%|████████  | 180048/225000 [341:45:44<65:20:21,  5.23s/it]
 80%|████████  | 180049/225000 [341:45:50<66:16:59,  5.31s/it]
 80%|████████  | 180050/225000 [341:45:55<65:45:07,  5.27s/it]
                                                              
{'loss': '0.7539', 'grad_norm': '1.201', 'learning_rate': '5.22e-06', 'epoch': '0.8002', 'num_input_tokens_seen': 566679272, 'train_runtime': '1.23e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180050/225000 [341:45:55<65:45:07,  5.27s/it]
 80%|████████  | 180051/225000 [341:46:00<66:50:52,  5.35s/it]
 80%|████████  | 180052/225000 [341:46:05<65:33:09,  5.25s/it]
 80%|████████  | 180053/225000 [341:46:11<65:53:15,  5.28s/it]
 80%|████████  | 180054/225000 [341:46:16<66:46:23,  5.35s/it]
 80%|████████  | 180055/225000 [341:46:21<65:24:42,  5.24s/it]
 80%|████████  | 180056/225000 [341:46:27<65:48:15,  5.27s/it]
 80%|████████  | 180057/225000 [341:46:32<66:42:16,  5.34s/it]
 80%|████████  | 180058/225000 [341:46:37<66:09:46,  5.30s/it]
 80%|████████  | 180059/225000 [341:46:42<65:41:39,  5.26s/it]
 80%|████████  | 180060/225000 [341:46:48<65:48:21,  5.27s/it]
 80%|████████  | 180061/225000 [341:46:57<81:19:36,  6.51s/it]
 80%|████████  | 180062/225000 [341:47:06<88:32:14,  7.09s/it]
 80%|████████  | 180063/225000 [341:47:09<75:05:33,  6.02s/it]
 80%|████████  | 180064/225000 [341:47:18<84:45:16,  6.79s/it]
 80%|████████  | 180065/225000 [341:47:21<71:41:02,  5.74s/it]
 80%|████████  | 180066/225000 [341:47:25<64:08:18,  5.14s/it]
 80%|████████  | 180067/225000 [341:47:34<81:33:31,  6.53s/it]
 80%|████████  | 180068/225000 [341:47:38<70:14:06,  5.63s/it]
 80%|████████  | 180069/225000 [341:47:42<64:00:10,  5.13s/it]
 80%|████████  | 180070/225000 [341:47:46<60:14:52,  4.83s/it]
 80%|████████  | 180071/225000 [341:47:52<64:10:31,  5.14s/it]
 80%|████████  | 180072/225000 [341:48:02<82:43:55,  6.63s/it]
 80%|████████  | 180073/225000 [341:48:06<71:07:45,  5.70s/it]
 80%|████████  | 180074/225000 [341:48:11<71:37:42,  5.74s/it]
 80%|████████  | 180075/225000 [341:48:19<77:20:58,  6.20s/it]
 80%|████████  | 180076/225000 [341:48:23<68:37:28,  5.50s/it]
 80%|████████  | 180077/225000 [341:48:26<61:54:37,  4.96s/it]
 80%|████████  | 180078/225000 [341:48:31<59:24:40,  4.76s/it]
 80%|████████  | 180079/225000 [341:48:40<76:45:07,  6.15s/it]
 80%|████████  | 180080/225000 [341:48:44<69:21:45,  5.56s/it]
 80%|████████  | 180081/225000 [341:48:48<64:47:05,  5.19s/it]
 80%|████████  | 180082/225000 [341:48:53<60:55:59,  4.88s/it]
 80%|████████  | 180083/225000 [341:49:02<78:01:21,  6.25s/it]
 80%|████████  | 180084/225000 [341:49:06<68:46:04,  5.51s/it]
 80%|████████  | 180085/225000 [341:49:15<82:44:40,  6.63s/it]
 80%|████████  | 180086/225000 [341:49:19<71:15:17,  5.71s/it]
 80%|████████  | 180087/225000 [341:49:22<63:20:36,  5.08s/it]
 80%|████████  | 180088/225000 [341:49:31<75:52:18,  6.08s/it]
 80%|████████  | 180089/225000 [341:49:35<68:11:30,  5.47s/it]
 80%|████████  | 180090/225000 [341:49:39<62:15:55,  4.99s/it]
 80%|████████  | 180091/225000 [341:49:43<59:40:08,  4.78s/it]
 80%|████████  | 180092/225000 [341:49:47<57:49:24,  4.64s/it]
 80%|████████  | 180093/225000 [341:49:52<56:47:25,  4.55s/it]
 80%|████████  | 180094/225000 [341:50:02<78:55:30,  6.33s/it]
 80%|████████  | 180095/225000 [341:50:06<68:28:54,  5.49s/it]
 80%|████████  | 180096/225000 [341:50:12<71:29:55,  5.73s/it]
 80%|████████  | 180097/225000 [341:50:19<76:26:07,  6.13s/it]
 80%|████████  | 180098/225000 [341:50:23<70:26:08,  5.65s/it]
 80%|████████  | 180099/225000 [341:50:28<66:37:37,  5.34s/it]
 80%|████████  | 180100/225000 [341:50:33<66:00:15,  5.29s/it]
                                                              
{'loss': '0.7573', 'grad_norm': '1.08', 'learning_rate': '5.209e-06', 'epoch': '0.8004', 'num_input_tokens_seen': 566810864, 'train_runtime': '1.231e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180100/225000 [341:50:33<66:00:15,  5.29s/it]
 80%|████████  | 180101/225000 [341:50:38<63:31:36,  5.09s/it]
 80%|████████  | 180102/225000 [341:50:43<62:04:17,  4.98s/it]
 80%|████████  | 180103/225000 [341:50:47<61:50:07,  4.96s/it]
 80%|████████  | 180104/225000 [341:50:52<59:23:54,  4.76s/it]
 80%|████████  | 180105/225000 [341:50:56<59:09:28,  4.74s/it]
 80%|████████  | 180106/225000 [341:51:06<77:21:39,  6.20s/it]
 80%|████████  | 180107/225000 [341:51:15<87:58:09,  7.05s/it]
 80%|████████  | 180108/225000 [341:51:19<75:46:19,  6.08s/it]
 80%|████████  | 180109/225000 [341:51:23<67:40:20,  5.43s/it]
 80%|████████  | 180110/225000 [341:51:32<81:08:35,  6.51s/it]
 80%|████████  | 180111/225000 [341:51:35<68:45:10,  5.51s/it]
 80%|████████  | 180112/225000 [341:51:40<66:10:46,  5.31s/it]
 80%|████████  | 180113/225000 [341:51:49<79:51:47,  6.41s/it]
 80%|████████  | 180114/225000 [341:51:52<68:54:16,  5.53s/it]
 80%|████████  | 180115/225000 [341:51:57<66:07:47,  5.30s/it]
 80%|████████  | 180116/225000 [341:52:02<66:21:06,  5.32s/it]
 80%|████████  | 180117/225000 [341:52:08<66:27:46,  5.33s/it]
 80%|████████  | 180118/225000 [341:52:13<66:08:44,  5.31s/it]
 80%|████████  | 180119/225000 [341:52:19<66:42:07,  5.35s/it]
 80%|████████  | 180120/225000 [341:52:24<67:14:35,  5.39s/it]
 80%|████████  | 180121/225000 [341:52:30<68:07:29,  5.46s/it]
 80%|████████  | 180122/225000 [341:52:35<67:20:23,  5.40s/it]
 80%|████████  | 180123/225000 [341:52:40<66:11:06,  5.31s/it]
 80%|████████  | 180124/225000 [341:52:45<66:47:55,  5.36s/it]
 80%|████████  | 180125/225000 [341:52:51<68:23:35,  5.49s/it]
 80%|████████  | 180126/225000 [341:52:57<68:10:17,  5.47s/it]
 80%|████████  | 180127/225000 [341:53:02<67:57:37,  5.45s/it]
 80%|████████  | 180128/225000 [341:53:07<65:00:29,  5.22s/it]
 80%|████████  | 180129/225000 [341:53:15<77:42:43,  6.23s/it]
 80%|████████  | 180130/225000 [341:53:20<69:49:40,  5.60s/it]
 80%|████████  | 180131/225000 [341:53:26<71:36:33,  5.75s/it]
 80%|████████  | 180132/225000 [341:53:32<73:02:23,  5.86s/it]
 80%|████████  | 180133/225000 [341:53:36<66:12:03,  5.31s/it]
 80%|████████  | 180134/225000 [341:53:40<62:44:16,  5.03s/it]
 80%|████████  | 180135/225000 [341:53:45<60:35:20,  4.86s/it]
 80%|████████  | 180136/225000 [341:53:49<57:37:39,  4.62s/it]
 80%|████████  | 180137/225000 [341:53:54<60:32:42,  4.86s/it]
 80%|████████  | 180138/225000 [341:54:06<86:38:34,  6.95s/it]
 80%|████████  | 180139/225000 [341:54:09<72:32:47,  5.82s/it]
 80%|████████  | 180140/225000 [341:54:13<64:16:55,  5.16s/it]
 80%|████████  | 180141/225000 [341:54:21<76:49:20,  6.17s/it]
 80%|████████  | 180142/225000 [341:54:25<68:59:40,  5.54s/it]
 80%|████████  | 180143/225000 [341:54:29<63:07:54,  5.07s/it]
 80%|████████  | 180144/225000 [341:54:33<58:45:26,  4.72s/it]
 80%|████████  | 180145/225000 [341:54:43<78:35:58,  6.31s/it]
 80%|████████  | 180146/225000 [341:54:47<70:42:50,  5.68s/it]
 80%|████████  | 180147/225000 [341:54:51<64:39:07,  5.19s/it]
 80%|████████  | 180148/225000 [341:54:55<59:18:20,  4.76s/it]
 80%|████████  | 180149/225000 [341:54:59<56:15:21,  4.52s/it]
 80%|████████  | 180150/225000 [341:55:03<54:46:34,  4.40s/it]
                                                              
{'loss': '0.7616', 'grad_norm': '1.034', 'learning_rate': '5.198e-06', 'epoch': '0.8007', 'num_input_tokens_seen': 566939560, 'train_runtime': '1.231e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180150/225000 [341:55:03<54:46:34,  4.40s/it]
 80%|████████  | 180151/225000 [341:55:14<80:09:14,  6.43s/it]
 80%|████████  | 180152/225000 [341:55:18<70:46:08,  5.68s/it]
 80%|████████  | 180153/225000 [341:55:22<64:16:36,  5.16s/it]
 80%|████████  | 180154/225000 [341:55:28<64:22:50,  5.17s/it]
 80%|████████  | 180155/225000 [341:55:36<77:38:43,  6.23s/it]
 80%|████████  | 180156/225000 [341:55:44<82:00:29,  6.58s/it]
 80%|████████  | 180157/225000 [341:55:48<72:15:32,  5.80s/it]
 80%|████████  | 180158/225000 [341:55:52<66:51:38,  5.37s/it]
 80%|████████  | 180159/225000 [341:55:56<61:19:42,  4.92s/it]
 80%|████████  | 180160/225000 [341:56:00<60:00:07,  4.82s/it]
 80%|████████  | 180161/225000 [341:56:11<81:11:15,  6.52s/it]
 80%|████████  | 180162/225000 [341:56:15<71:13:34,  5.72s/it]
 80%|████████  | 180163/225000 [341:56:19<64:23:32,  5.17s/it]
 80%|████████  | 180164/225000 [341:56:26<72:07:56,  5.79s/it]
 80%|████████  | 180165/225000 [341:56:32<73:34:10,  5.91s/it]
 80%|████████  | 180166/225000 [341:56:36<65:55:55,  5.29s/it]
 80%|████████  | 180167/225000 [341:56:40<59:57:17,  4.81s/it]
 80%|████████  | 180168/225000 [341:56:47<68:18:28,  5.49s/it]
 80%|████████  | 180169/225000 [341:56:54<73:23:16,  5.89s/it]
 80%|████████  | 180170/225000 [341:57:00<73:56:15,  5.94s/it]
 80%|████████  | 180171/225000 [341:57:03<63:37:00,  5.11s/it]
 80%|████████  | 180172/225000 [341:57:10<71:14:13,  5.72s/it]
 80%|████████  | 180173/225000 [341:57:14<66:14:59,  5.32s/it]
 80%|████████  | 180174/225000 [341:57:18<60:59:57,  4.90s/it]
 80%|████████  | 180175/225000 [341:57:23<59:00:31,  4.74s/it]
 80%|████████  | 180176/225000 [341:57:27<59:07:23,  4.75s/it]
 80%|████████  | 180177/225000 [341:57:32<57:38:48,  4.63s/it]
 80%|████████  | 180178/225000 [341:57:43<83:02:39,  6.67s/it]
 80%|████████  | 180179/225000 [341:57:47<71:44:55,  5.76s/it]
 80%|████████  | 180180/225000 [341:57:50<62:32:15,  5.02s/it]
 80%|████████  | 180181/225000 [341:57:58<74:27:41,  5.98s/it]
 80%|████████  | 180182/225000 [341:58:04<73:51:49,  5.93s/it]
 80%|████████  | 180183/225000 [341:58:07<63:38:48,  5.11s/it]
 80%|████████  | 180184/225000 [341:58:16<77:02:35,  6.19s/it]
 80%|████████  | 180185/225000 [341:58:20<67:04:34,  5.39s/it]
 80%|████████  | 180186/225000 [341:58:28<77:31:54,  6.23s/it]
 80%|████████  | 180187/225000 [341:58:31<67:44:15,  5.44s/it]
 80%|████████  | 180188/225000 [341:58:35<61:56:50,  4.98s/it]
 80%|████████  | 180189/225000 [341:58:41<63:30:38,  5.10s/it]
 80%|████████  | 180190/225000 [341:58:48<71:56:40,  5.78s/it]
 80%|████████  | 180191/225000 [341:58:52<64:32:39,  5.19s/it]
 80%|████████  | 180192/225000 [341:58:56<59:13:58,  4.76s/it]
 80%|████████  | 180193/225000 [341:59:00<56:55:43,  4.57s/it]
 80%|████████  | 180194/225000 [341:59:04<57:15:39,  4.60s/it]
 80%|████████  | 180195/225000 [341:59:16<83:55:30,  6.74s/it]
 80%|████████  | 180196/225000 [341:59:20<74:47:18,  6.01s/it]
 80%|████████  | 180197/225000 [341:59:24<66:17:03,  5.33s/it]
 80%|████████  | 180198/225000 [341:59:28<62:42:24,  5.04s/it]
 80%|████████  | 180199/225000 [341:59:35<68:01:36,  5.47s/it]
 80%|████████  | 180200/225000 [341:59:44<80:48:27,  6.49s/it]
                                                              
{'loss': '0.766', 'grad_norm': '1.3', 'learning_rate': '5.187e-06', 'epoch': '0.8009', 'num_input_tokens_seen': 567069648, 'train_runtime': '1.231e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180200/225000 [341:59:44<80:48:27,  6.49s/it]
 80%|████████  | 180201/225000 [341:59:47<68:41:26,  5.52s/it]
 80%|████████  | 180202/225000 [341:59:51<61:53:01,  4.97s/it]
 80%|████████  | 180203/225000 [342:00:00<79:06:56,  6.36s/it]
 80%|████████  | 180204/225000 [342:00:04<68:46:45,  5.53s/it]
 80%|████████  | 180205/225000 [342:00:08<61:42:06,  4.96s/it]
 80%|████████  | 180206/225000 [342:00:15<71:38:04,  5.76s/it]
 80%|████████  | 180207/225000 [342:00:20<69:29:43,  5.59s/it]
 80%|████████  | 180208/225000 [342:00:27<73:23:38,  5.90s/it]
 80%|████████  | 180209/225000 [342:00:31<66:24:25,  5.34s/it]
 80%|████████  | 180210/225000 [342:00:35<60:10:26,  4.84s/it]
 80%|████████  | 180211/225000 [342:00:40<60:18:02,  4.85s/it]
 80%|████████  | 180212/225000 [342:00:49<78:08:18,  6.28s/it]
 80%|████████  | 180213/225000 [342:00:53<67:45:30,  5.45s/it]
 80%|████████  | 180214/225000 [342:01:00<75:44:49,  6.09s/it]
 80%|████████  | 180215/225000 [342:01:05<69:21:38,  5.58s/it]
 80%|████████  | 180216/225000 [342:01:12<74:21:52,  5.98s/it]
 80%|████████  | 180217/225000 [342:01:15<65:55:45,  5.30s/it]
 80%|████████  | 180218/225000 [342:01:19<59:48:09,  4.81s/it]
 80%|████████  | 180219/225000 [342:01:29<79:42:04,  6.41s/it]
 80%|████████  | 180220/225000 [342:01:32<68:29:58,  5.51s/it]
 80%|████████  | 180221/225000 [342:01:38<67:48:21,  5.45s/it]
 80%|████████  | 180222/225000 [342:01:43<67:20:30,  5.41s/it]
 80%|████████  | 180223/225000 [342:01:49<67:26:40,  5.42s/it]
 80%|████████  | 180224/225000 [342:01:54<66:06:30,  5.32s/it]
 80%|████████  | 180225/225000 [342:01:59<65:15:32,  5.25s/it]
 80%|████████  | 180226/225000 [342:02:04<65:39:27,  5.28s/it]
 80%|████████  | 180227/225000 [342:02:14<82:20:39,  6.62s/it]
 80%|████████  | 180228/225000 [342:02:19<78:21:16,  6.30s/it]
 80%|████████  | 180229/225000 [342:02:23<66:28:31,  5.35s/it]
 80%|████████  | 180230/225000 [342:02:28<66:46:42,  5.37s/it]
 80%|████████  | 180231/225000 [342:02:35<71:56:16,  5.78s/it]
 80%|████████  | 180232/225000 [342:02:39<64:57:29,  5.22s/it]
 80%|████████  | 180233/225000 [342:02:42<59:30:40,  4.79s/it]
 80%|████████  | 180234/225000 [342:02:48<63:14:00,  5.09s/it]
 80%|████████  | 180235/225000 [342:02:57<76:47:21,  6.18s/it]
 80%|████████  | 180236/225000 [342:03:02<71:47:37,  5.77s/it]
 80%|████████  | 180237/225000 [342:03:07<68:59:11,  5.55s/it]
 80%|████████  | 180238/225000 [342:03:11<65:00:50,  5.23s/it]
 80%|████████  | 180239/225000 [342:03:16<63:49:51,  5.13s/it]
 80%|████████  | 180240/225000 [342:03:21<64:18:03,  5.17s/it]
 80%|████████  | 180241/225000 [342:03:27<64:59:25,  5.23s/it]
 80%|████████  | 180242/225000 [342:03:31<62:06:53,  5.00s/it]
 80%|████████  | 180243/225000 [342:03:36<61:27:48,  4.94s/it]
 80%|████████  | 180244/225000 [342:03:41<62:52:31,  5.06s/it]
 80%|████████  | 180245/225000 [342:03:46<62:25:28,  5.02s/it]
 80%|████████  | 180246/225000 [342:03:52<64:01:02,  5.15s/it]
 80%|████████  | 180247/225000 [342:03:57<64:16:36,  5.17s/it]
 80%|████████  | 180248/225000 [342:04:02<62:52:44,  5.06s/it]
 80%|████████  | 180249/225000 [342:04:07<63:37:12,  5.12s/it]
 80%|████████  | 180250/225000 [342:04:12<64:58:33,  5.23s/it]
                                                              
{'loss': '0.7631', 'grad_norm': '1.095', 'learning_rate': '5.176e-06', 'epoch': '0.8011', 'num_input_tokens_seen': 567198976, 'train_runtime': '1.231e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180250/225000 [342:04:12<64:58:33,  5.23s/it]
 80%|████████  | 180251/225000 [342:04:18<65:00:47,  5.23s/it]
 80%|████████  | 180252/225000 [342:04:23<64:52:15,  5.22s/it]
 80%|████████  | 180253/225000 [342:04:28<64:43:21,  5.21s/it]
 80%|████████  | 180254/225000 [342:04:33<65:23:22,  5.26s/it]
 80%|████████  | 180255/225000 [342:04:39<64:50:35,  5.22s/it]
 80%|████████  | 180256/225000 [342:04:44<65:29:19,  5.27s/it]
 80%|████████  | 180257/225000 [342:04:49<63:56:34,  5.14s/it]
 80%|████████  | 180258/225000 [342:04:57<75:10:54,  6.05s/it]
 80%|████████  | 180259/225000 [342:05:02<71:58:37,  5.79s/it]
 80%|████████  | 180260/225000 [342:05:08<70:24:35,  5.67s/it]
 80%|████████  | 180261/225000 [342:05:13<68:15:28,  5.49s/it]
 80%|████████  | 180262/225000 [342:05:18<66:25:44,  5.35s/it]
 80%|████████  | 180263/225000 [342:05:23<65:14:16,  5.25s/it]
 80%|████████  | 180264/225000 [342:05:28<64:20:22,  5.18s/it]
 80%|████████  | 180265/225000 [342:05:34<70:14:20,  5.65s/it]
 80%|████████  | 180266/225000 [342:05:39<67:25:03,  5.43s/it]
 80%|████████  | 180267/225000 [342:05:44<65:57:48,  5.31s/it]
 80%|████████  | 180268/225000 [342:05:50<65:35:10,  5.28s/it]
 80%|████████  | 180269/225000 [342:05:55<64:26:34,  5.19s/it]
 80%|████████  | 180270/225000 [342:06:00<64:23:25,  5.18s/it]
 80%|████████  | 180271/225000 [342:06:05<65:01:51,  5.23s/it]
 80%|████████  | 180272/225000 [342:06:11<66:12:59,  5.33s/it]
 80%|████████  | 180273/225000 [342:06:16<66:13:58,  5.33s/it]
 80%|████████  | 180274/225000 [342:06:24<77:00:33,  6.20s/it]
 80%|████████  | 180275/225000 [342:06:34<88:37:29,  7.13s/it]
 80%|████████  | 180276/225000 [342:06:37<74:08:16,  5.97s/it]
 80%|████████  | 180277/225000 [342:06:46<86:36:09,  6.97s/it]
 80%|████████  | 180278/225000 [342:06:49<72:30:47,  5.84s/it]
 80%|████████  | 180279/225000 [342:06:57<79:03:36,  6.36s/it]
 80%|████████  | 180280/225000 [342:07:01<68:55:31,  5.55s/it]
 80%|████████  | 180281/225000 [342:07:08<75:53:14,  6.11s/it]
 80%|████████  | 180282/225000 [342:07:13<72:00:16,  5.80s/it]
 80%|████████  | 180283/225000 [342:07:17<65:43:19,  5.29s/it]
 80%|████████  | 180284/225000 [342:07:21<60:28:30,  4.87s/it]
 80%|████████  | 180285/225000 [342:07:27<66:27:53,  5.35s/it]
 80%|████████  | 180286/225000 [342:07:34<70:42:15,  5.69s/it]
 80%|████████  | 180287/225000 [342:07:38<63:26:17,  5.11s/it]
 80%|████████  | 180288/225000 [342:07:46<73:45:21,  5.94s/it]
 80%|████████  | 180289/225000 [342:07:50<67:46:28,  5.46s/it]
 80%|████████  | 180290/225000 [342:07:53<60:23:02,  4.86s/it]
 80%|████████  | 180291/225000 [342:07:57<57:11:06,  4.60s/it]
 80%|████████  | 180292/225000 [342:08:02<57:28:45,  4.63s/it]
 80%|████████  | 180293/225000 [342:08:06<56:40:30,  4.56s/it]
 80%|████████  | 180294/225000 [342:08:11<57:13:47,  4.61s/it]
 80%|████████  | 180295/225000 [342:08:16<56:43:21,  4.57s/it]
 80%|████████  | 180296/225000 [342:08:20<56:04:25,  4.52s/it]
 80%|████████  | 180297/225000 [342:08:25<57:47:12,  4.65s/it]
 80%|████████  | 180298/225000 [342:08:30<59:25:17,  4.79s/it]
 80%|████████  | 180299/225000 [342:08:36<61:48:04,  4.98s/it]
 80%|████████  | 180300/225000 [342:08:40<61:26:33,  4.95s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.118', 'learning_rate': '5.164e-06', 'epoch': '0.8013', 'num_input_tokens_seen': 567328760, 'train_runtime': '1.232e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180300/225000 [342:08:40<61:26:33,  4.95s/it]
 80%|████████  | 180301/225000 [342:08:49<73:39:31,  5.93s/it]
 80%|████████  | 180302/225000 [342:08:58<87:11:27,  7.02s/it]
 80%|████████  | 180303/225000 [342:09:02<74:46:33,  6.02s/it]
 80%|████████  | 180304/225000 [342:09:06<66:09:30,  5.33s/it]
 80%|████████  | 180305/225000 [342:09:13<75:28:04,  6.08s/it]
 80%|████████  | 180306/225000 [342:09:18<70:18:55,  5.66s/it]
 80%|████████  | 180307/225000 [342:09:22<65:21:44,  5.26s/it]
 80%|████████  | 180308/225000 [342:09:28<66:32:04,  5.36s/it]
 80%|████████  | 180309/225000 [342:09:35<73:17:27,  5.90s/it]
 80%|████████  | 180310/225000 [342:09:42<74:56:28,  6.04s/it]
 80%|████████  | 180311/225000 [342:09:45<66:27:03,  5.35s/it]
 80%|████████  | 180312/225000 [342:09:49<61:48:10,  4.98s/it]
 80%|████████  | 180313/225000 [342:09:55<62:57:33,  5.07s/it]
 80%|████████  | 180314/225000 [342:10:00<64:22:52,  5.19s/it]
 80%|████████  | 180315/225000 [342:10:05<64:18:26,  5.18s/it]
 80%|████████  | 180316/225000 [342:10:11<64:24:52,  5.19s/it]
 80%|████████  | 180317/225000 [342:10:16<64:05:45,  5.16s/it]
 80%|████████  | 180318/225000 [342:10:21<64:28:49,  5.20s/it]
 80%|████████  | 180319/225000 [342:10:26<64:57:04,  5.23s/it]
 80%|████████  | 180320/225000 [342:10:32<65:52:51,  5.31s/it]
 80%|████████  | 180321/225000 [342:10:39<71:48:49,  5.79s/it]
 80%|████████  | 180322/225000 [342:10:44<69:02:07,  5.56s/it]
 80%|████████  | 180323/225000 [342:10:49<66:53:24,  5.39s/it]
 80%|████████  | 180324/225000 [342:10:54<65:47:07,  5.30s/it]
 80%|████████  | 180325/225000 [342:10:59<67:11:54,  5.41s/it]
 80%|████████  | 180326/225000 [342:11:04<65:16:22,  5.26s/it]
 80%|████████  | 180327/225000 [342:11:09<64:49:40,  5.22s/it]
 80%|████████  | 180328/225000 [342:11:18<77:58:30,  6.28s/it]
 80%|████████  | 180329/225000 [342:11:26<83:32:32,  6.73s/it]
 80%|████████  | 180330/225000 [342:11:33<83:34:25,  6.74s/it]
 80%|████████  | 180331/225000 [342:11:37<73:07:36,  5.89s/it]
 80%|████████  | 180332/225000 [342:11:41<66:50:23,  5.39s/it]
 80%|████████  | 180333/225000 [342:11:45<62:43:53,  5.06s/it]
 80%|████████  | 180334/225000 [342:11:57<87:26:18,  7.05s/it]
 80%|████████  | 180335/225000 [342:12:01<76:10:57,  6.14s/it]
 80%|████████  | 180336/225000 [342:12:04<66:10:14,  5.33s/it]
 80%|████████  | 180337/225000 [342:12:11<69:46:46,  5.62s/it]
 80%|████████  | 180338/225000 [342:12:19<79:21:47,  6.40s/it]
 80%|████████  | 180339/225000 [342:12:23<69:43:05,  5.62s/it]
 80%|████████  | 180340/225000 [342:12:27<65:56:52,  5.32s/it]
 80%|████████  | 180341/225000 [342:12:32<64:14:57,  5.18s/it]
 80%|████████  | 180342/225000 [342:12:37<61:59:00,  5.00s/it]
 80%|████████  | 180343/225000 [342:12:41<59:35:50,  4.80s/it]
 80%|████████  | 180344/225000 [342:12:46<58:33:19,  4.72s/it]
 80%|████████  | 180345/225000 [342:12:50<57:57:31,  4.67s/it]
 80%|████████  | 180346/225000 [342:12:55<58:03:08,  4.68s/it]
 80%|████████  | 180347/225000 [342:13:00<58:36:11,  4.72s/it]
 80%|████████  | 180348/225000 [342:13:05<59:23:40,  4.79s/it]
 80%|████████  | 180349/225000 [342:13:16<85:08:19,  6.86s/it]
 80%|████████  | 180350/225000 [342:13:22<80:19:42,  6.48s/it]
                                                              
{'loss': '0.7626', 'grad_norm': '1.11', 'learning_rate': '5.153e-06', 'epoch': '0.8016', 'num_input_tokens_seen': 567459120, 'train_runtime': '1.232e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180350/225000 [342:13:22<80:19:42,  6.48s/it]
 80%|████████  | 180351/225000 [342:13:25<67:48:10,  5.47s/it]
 80%|████████  | 180352/225000 [342:13:29<63:22:52,  5.11s/it]
 80%|████████  | 180353/225000 [342:13:34<62:04:51,  5.01s/it]
 80%|████████  | 180354/225000 [342:13:38<59:53:06,  4.83s/it]
 80%|████████  | 180355/225000 [342:13:43<59:31:23,  4.80s/it]
 80%|████████  | 180356/225000 [342:13:48<61:04:16,  4.92s/it]
 80%|████████  | 180357/225000 [342:13:53<61:13:47,  4.94s/it]
 80%|████████  | 180358/225000 [342:13:58<59:58:41,  4.84s/it]
 80%|████████  | 180359/225000 [342:14:06<72:17:32,  5.83s/it]
 80%|████████  | 180360/225000 [342:14:14<80:42:52,  6.51s/it]
 80%|████████  | 180361/225000 [342:14:19<75:57:22,  6.13s/it]
 80%|████████  | 180362/225000 [342:14:26<76:18:36,  6.15s/it]
 80%|████████  | 180363/225000 [342:14:30<68:02:00,  5.49s/it]
 80%|████████  | 180364/225000 [342:14:34<62:26:42,  5.04s/it]
 80%|████████  | 180365/225000 [342:14:44<81:22:47,  6.56s/it]
 80%|████████  | 180366/225000 [342:14:49<76:33:37,  6.18s/it]
 80%|████████  | 180367/225000 [342:14:52<66:35:03,  5.37s/it]
 80%|████████  | 180368/225000 [342:14:57<63:54:36,  5.15s/it]
 80%|████████  | 180369/225000 [342:15:01<60:14:01,  4.86s/it]
 80%|████████  | 180370/225000 [342:15:06<59:38:08,  4.81s/it]
 80%|████████  | 180371/225000 [342:15:19<88:24:21,  7.13s/it]
 80%|████████  | 180372/225000 [342:15:22<75:15:40,  6.07s/it]
 80%|████████  | 180373/225000 [342:15:30<81:44:15,  6.59s/it]
 80%|████████  | 180374/225000 [342:15:34<72:12:09,  5.82s/it]
 80%|████████  | 180375/225000 [342:15:38<63:44:52,  5.14s/it]
 80%|████████  | 180376/225000 [342:15:46<75:06:54,  6.06s/it]
 80%|████████  | 180377/225000 [342:15:49<65:22:45,  5.27s/it]
 80%|████████  | 180378/225000 [342:15:55<66:44:56,  5.39s/it]
 80%|████████  | 180379/225000 [342:16:01<69:32:26,  5.61s/it]
 80%|████████  | 180380/225000 [342:16:05<62:32:36,  5.05s/it]
 80%|████████  | 180381/225000 [342:16:09<61:06:41,  4.93s/it]
 80%|████████  | 180382/225000 [342:16:17<72:06:34,  5.82s/it]
 80%|████████  | 180383/225000 [342:16:23<70:11:26,  5.66s/it]
 80%|████████  | 180384/225000 [342:16:28<67:42:36,  5.46s/it]
 80%|████████  | 180385/225000 [342:16:32<64:46:41,  5.23s/it]
 80%|████████  | 180386/225000 [342:16:37<63:54:45,  5.16s/it]
 80%|████████  | 180387/225000 [342:16:42<64:28:08,  5.20s/it]
 80%|████████  | 180388/225000 [342:16:48<64:17:08,  5.19s/it]
 80%|████████  | 180389/225000 [342:16:53<64:11:12,  5.18s/it]
 80%|████████  | 180390/225000 [342:16:58<63:54:07,  5.16s/it]
 80%|████████  | 180391/225000 [342:17:03<64:29:21,  5.20s/it]
 80%|████████  | 180392/225000 [342:17:08<63:55:11,  5.16s/it]
 80%|████████  | 180393/225000 [342:17:13<63:20:44,  5.11s/it]
 80%|████████  | 180394/225000 [342:17:18<63:23:30,  5.12s/it]
 80%|████████  | 180395/225000 [342:17:24<64:04:29,  5.17s/it]
 80%|████████  | 180396/225000 [342:17:32<74:21:53,  6.00s/it]
 80%|████████  | 180397/225000 [342:17:42<89:18:24,  7.21s/it]
 80%|████████  | 180398/225000 [342:17:45<74:07:41,  5.98s/it]
 80%|████████  | 180399/225000 [342:17:53<83:27:33,  6.74s/it]
 80%|████████  | 180400/225000 [342:17:57<70:44:25,  5.71s/it]
                                                              
{'loss': '0.7651', 'grad_norm': '1.029', 'learning_rate': '5.142e-06', 'epoch': '0.8018', 'num_input_tokens_seen': 567589288, 'train_runtime': '1.232e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180400/225000 [342:17:57<70:44:25,  5.71s/it]
 80%|████████  | 180401/225000 [342:18:04<76:50:45,  6.20s/it]
 80%|████████  | 180402/225000 [342:18:08<70:38:49,  5.70s/it]
 80%|████████  | 180403/225000 [342:18:13<65:48:38,  5.31s/it]
 80%|████████  | 180404/225000 [342:18:17<62:43:44,  5.06s/it]
 80%|████████  | 180405/225000 [342:18:22<60:07:58,  4.85s/it]
 80%|████████  | 180406/225000 [342:18:26<59:06:28,  4.77s/it]
 80%|████████  | 180407/225000 [342:18:31<58:08:24,  4.69s/it]
 80%|████████  | 180408/225000 [342:18:36<58:28:41,  4.72s/it]
 80%|████████  | 180409/225000 [342:18:41<61:25:40,  4.96s/it]
 80%|████████  | 180410/225000 [342:18:52<82:42:25,  6.68s/it]
 80%|████████  | 180411/225000 [342:18:55<70:31:21,  5.69s/it]
 80%|████████  | 180412/225000 [342:19:03<76:23:55,  6.17s/it]
 80%|████████  | 180413/225000 [342:19:09<76:24:59,  6.17s/it]
 80%|████████  | 180414/225000 [342:19:13<69:25:37,  5.61s/it]
 80%|████████  | 180415/225000 [342:19:18<65:35:45,  5.30s/it]
 80%|████████  | 180416/225000 [342:19:22<61:39:05,  4.98s/it]
 80%|████████  | 180417/225000 [342:19:26<60:10:05,  4.86s/it]
 80%|████████  | 180418/225000 [342:19:31<58:39:05,  4.74s/it]
 80%|████████  | 180419/225000 [342:19:36<59:05:18,  4.77s/it]
 80%|████████  | 180420/225000 [342:19:43<69:13:49,  5.59s/it]
 80%|████████  | 180421/225000 [342:19:53<85:41:28,  6.92s/it]
 80%|████████  | 180422/225000 [342:19:56<71:47:20,  5.80s/it]
 80%|████████  | 180423/225000 [342:20:00<64:05:04,  5.18s/it]
 80%|████████  | 180424/225000 [342:20:10<81:36:50,  6.59s/it]
 80%|████████  | 180425/225000 [342:20:13<69:49:31,  5.64s/it]
 80%|████████  | 180426/225000 [342:20:18<65:12:02,  5.27s/it]
 80%|████████  | 180427/225000 [342:20:25<73:07:37,  5.91s/it]
 80%|████████  | 180428/225000 [342:20:29<64:32:05,  5.21s/it]
 80%|████████  | 180429/225000 [342:20:38<77:43:00,  6.28s/it]
 80%|████████  | 180430/225000 [342:20:41<67:52:18,  5.48s/it]
 80%|████████  | 180431/225000 [342:20:48<72:32:03,  5.86s/it]
 80%|████████  | 180432/225000 [342:20:52<64:22:40,  5.20s/it]
 80%|████████  | 180433/225000 [342:20:56<61:22:28,  4.96s/it]
 80%|████████  | 180434/225000 [342:21:00<58:03:47,  4.69s/it]
 80%|████████  | 180435/225000 [342:21:10<77:33:52,  6.27s/it]
 80%|████████  | 180436/225000 [342:21:13<66:24:26,  5.36s/it]
 80%|████████  | 180437/225000 [342:21:20<70:21:56,  5.68s/it]
 80%|████████  | 180438/225000 [342:21:24<64:29:15,  5.21s/it]
 80%|████████  | 180439/225000 [342:21:28<59:13:45,  4.79s/it]
 80%|████████  | 180440/225000 [342:21:36<71:57:40,  5.81s/it]
 80%|████████  | 180441/225000 [342:21:41<69:17:05,  5.60s/it]
 80%|████████  | 180442/225000 [342:21:45<63:23:56,  5.12s/it]
 80%|████████  | 180443/225000 [342:21:52<72:20:57,  5.85s/it]
 80%|████████  | 180444/225000 [342:21:59<75:10:26,  6.07s/it]
 80%|████████  | 180445/225000 [342:22:03<68:39:54,  5.55s/it]
 80%|████████  | 180446/225000 [342:22:07<60:58:33,  4.93s/it]
 80%|████████  | 180447/225000 [342:22:16<77:16:31,  6.24s/it]
 80%|████████  | 180448/225000 [342:22:21<73:33:49,  5.94s/it]
 80%|████████  | 180449/225000 [342:22:26<70:09:18,  5.67s/it]
 80%|████████  | 180450/225000 [342:22:30<63:49:54,  5.16s/it]
                                                              
{'loss': '0.7655', 'grad_norm': '1.228', 'learning_rate': '5.131e-06', 'epoch': '0.802', 'num_input_tokens_seen': 567718936, 'train_runtime': '1.233e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180450/225000 [342:22:30<63:49:54,  5.16s/it]
 80%|████████  | 180451/225000 [342:22:34<59:09:30,  4.78s/it]
 80%|████████  | 180452/225000 [342:22:40<61:53:25,  5.00s/it]
 80%|████████  | 180453/225000 [342:22:49<78:14:25,  6.32s/it]
 80%|████████  | 180454/225000 [342:22:53<68:39:40,  5.55s/it]
 80%|████████  | 180455/225000 [342:23:01<76:56:46,  6.22s/it]
 80%|████████  | 180456/225000 [342:23:05<67:56:57,  5.49s/it]
 80%|████████  | 180457/225000 [342:23:09<62:29:44,  5.05s/it]
 80%|████████  | 180458/225000 [342:23:13<61:04:04,  4.94s/it]
 80%|████████  | 180459/225000 [342:23:22<76:10:21,  6.16s/it]
 80%|████████  | 180460/225000 [342:23:26<66:16:26,  5.36s/it]
 80%|████████  | 180461/225000 [342:23:33<73:03:22,  5.90s/it]
 80%|████████  | 180462/225000 [342:23:36<63:05:12,  5.10s/it]
 80%|████████  | 180463/225000 [342:23:41<63:57:01,  5.17s/it]
 80%|████████  | 180464/225000 [342:23:47<64:41:37,  5.23s/it]
 80%|████████  | 180465/225000 [342:23:55<75:45:54,  6.12s/it]
 80%|████████  | 180466/225000 [342:23:59<67:13:57,  5.43s/it]
 80%|████████  | 180467/225000 [342:24:02<59:35:06,  4.82s/it]
 80%|████████  | 180468/225000 [342:24:10<70:40:30,  5.71s/it]
 80%|████████  | 180469/225000 [342:24:17<73:57:14,  5.98s/it]
 80%|████████  | 180470/225000 [342:24:21<68:41:51,  5.55s/it]
 80%|████████  | 180471/225000 [342:24:25<63:28:25,  5.13s/it]
 80%|████████  | 180472/225000 [342:24:29<59:44:02,  4.83s/it]
 80%|████████  | 180473/225000 [342:24:34<58:42:44,  4.75s/it]
 80%|████████  | 180474/225000 [342:24:38<56:41:47,  4.58s/it]
 80%|████████  | 180475/225000 [342:24:47<71:54:09,  5.81s/it]
 80%|████████  | 180476/225000 [342:24:56<82:14:20,  6.65s/it]
 80%|████████  | 180477/225000 [342:25:02<80:47:20,  6.53s/it]
 80%|████████  | 180478/225000 [342:25:06<70:28:15,  5.70s/it]
 80%|████████  | 180479/225000 [342:25:09<62:43:08,  5.07s/it]
 80%|████████  | 180480/225000 [342:25:20<83:52:56,  6.78s/it]
 80%|████████  | 180481/225000 [342:25:23<70:01:10,  5.66s/it]
 80%|████████  | 180482/225000 [342:25:30<73:53:31,  5.98s/it]
 80%|████████  | 180483/225000 [342:25:35<70:04:17,  5.67s/it]
 80%|████████  | 180484/225000 [342:25:40<68:40:10,  5.55s/it]
 80%|████████  | 180485/225000 [342:25:45<67:19:38,  5.44s/it]
 80%|████████  | 180486/225000 [342:25:50<66:28:02,  5.38s/it]
 80%|████████  | 180487/225000 [342:25:55<63:28:57,  5.13s/it]
 80%|████████  | 180488/225000 [342:25:59<61:26:49,  4.97s/it]
 80%|████████  | 180489/225000 [342:26:05<64:05:53,  5.18s/it]
 80%|████████  | 180490/225000 [342:26:10<64:07:35,  5.19s/it]
 80%|████████  | 180491/225000 [342:26:15<63:53:52,  5.17s/it]
 80%|████████  | 180492/225000 [342:26:21<63:30:15,  5.14s/it]
 80%|████████  | 180493/225000 [342:26:25<61:45:10,  4.99s/it]
 80%|████████  | 180494/225000 [342:26:30<61:26:43,  4.97s/it]
 80%|████████  | 180495/225000 [342:26:35<60:39:53,  4.91s/it]
 80%|████████  | 180496/225000 [342:26:44<75:20:48,  6.09s/it]
 80%|████████  | 180497/225000 [342:26:53<87:30:34,  7.08s/it]
 80%|████████  | 180498/225000 [342:26:56<72:32:35,  5.87s/it]
 80%|████████  | 180499/225000 [342:27:01<67:30:42,  5.46s/it]
 80%|████████  | 180500/225000 [342:27:06<66:23:55,  5.37s/it]
                                                              
{'loss': '0.7586', 'grad_norm': '1.15', 'learning_rate': '5.12e-06', 'epoch': '0.8022', 'num_input_tokens_seen': 567850872, 'train_runtime': '1.233e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180500/225000 [342:27:06<66:23:55,  5.37s/it]
 80%|████████  | 180501/225000 [342:27:11<64:50:35,  5.25s/it]
 80%|████████  | 180502/225000 [342:27:16<64:07:38,  5.19s/it]
 80%|████████  | 180503/225000 [342:27:21<63:27:41,  5.13s/it]
 80%|████████  | 180504/225000 [342:27:25<60:59:11,  4.93s/it]
 80%|████████  | 180505/225000 [342:27:30<61:51:52,  5.01s/it]
 80%|████████  | 180506/225000 [342:27:35<61:44:07,  5.00s/it]
 80%|████████  | 180507/225000 [342:27:42<65:43:41,  5.32s/it]
 80%|████████  | 180508/225000 [342:27:54<92:10:40,  7.46s/it]
 80%|████████  | 180509/225000 [342:27:58<77:39:05,  6.28s/it]
 80%|████████  | 180510/225000 [342:28:04<77:54:08,  6.30s/it]
 80%|████████  | 180511/225000 [342:28:09<73:25:53,  5.94s/it]
 80%|████████  | 180512/225000 [342:28:12<64:33:35,  5.22s/it]
 80%|████████  | 180513/225000 [342:28:21<77:44:09,  6.29s/it]
 80%|████████  | 180514/225000 [342:28:26<70:43:47,  5.72s/it]
 80%|████████  | 180515/225000 [342:28:32<72:30:38,  5.87s/it]
 80%|████████  | 180516/225000 [342:28:37<68:38:56,  5.56s/it]
 80%|████████  | 180517/225000 [342:28:43<72:10:19,  5.84s/it]
 80%|████████  | 180518/225000 [342:28:48<68:16:43,  5.53s/it]
 80%|████████  | 180519/225000 [342:28:53<66:21:12,  5.37s/it]
 80%|████████  | 180520/225000 [342:29:00<72:13:11,  5.85s/it]
 80%|████████  | 180521/225000 [342:29:05<70:06:02,  5.67s/it]
 80%|████████  | 180522/225000 [342:29:10<68:30:53,  5.55s/it]
 80%|████████  | 180523/225000 [342:29:14<61:41:38,  4.99s/it]
 80%|████████  | 180524/225000 [342:29:18<58:01:27,  4.70s/it]
 80%|████████  | 180525/225000 [342:29:23<58:29:53,  4.74s/it]
 80%|████████  | 180526/225000 [342:29:33<78:32:41,  6.36s/it]
 80%|████████  | 180527/225000 [342:29:37<69:56:10,  5.66s/it]
 80%|████████  | 180528/225000 [342:29:41<63:36:51,  5.15s/it]
 80%|████████  | 180529/225000 [342:29:45<60:24:21,  4.89s/it]
 80%|████████  | 180530/225000 [342:29:53<68:50:04,  5.57s/it]
 80%|████████  | 180531/225000 [342:30:00<75:41:00,  6.13s/it]
 80%|████████  | 180532/225000 [342:30:03<65:18:46,  5.29s/it]
 80%|████████  | 180533/225000 [342:30:11<75:14:19,  6.09s/it]
 80%|████████  | 180534/225000 [342:30:15<66:26:04,  5.38s/it]
 80%|████████  | 180535/225000 [342:30:19<59:47:40,  4.84s/it]
 80%|████████  | 180536/225000 [342:30:25<65:43:52,  5.32s/it]
 80%|████████  | 180537/225000 [342:30:32<70:41:40,  5.72s/it]
 80%|████████  | 180538/225000 [342:30:35<61:44:15,  5.00s/it]
 80%|████████  | 180539/225000 [342:30:43<72:42:53,  5.89s/it]
 80%|████████  | 180540/225000 [342:30:47<66:08:59,  5.36s/it]
 80%|████████  | 180541/225000 [342:30:51<60:32:04,  4.90s/it]
 80%|████████  | 180542/225000 [342:30:55<56:14:57,  4.55s/it]
 80%|████████  | 180543/225000 [342:31:01<62:56:28,  5.10s/it]
 80%|████████  | 180544/225000 [342:31:10<77:33:50,  6.28s/it]
 80%|████████  | 180545/225000 [342:31:14<67:56:22,  5.50s/it]
 80%|████████  | 180546/225000 [342:31:17<60:58:11,  4.94s/it]
 80%|████████  | 180547/225000 [342:31:26<75:13:26,  6.09s/it]
 80%|████████  | 180548/225000 [342:31:30<65:48:11,  5.33s/it]
 80%|████████  | 180549/225000 [342:31:34<63:38:27,  5.15s/it]
 80%|████████  | 180550/225000 [342:31:43<74:38:32,  6.05s/it]
                                                              
{'loss': '0.7551', 'grad_norm': 'nan', 'learning_rate': '5.109e-06', 'epoch': '0.8024', 'num_input_tokens_seen': 567981112, 'train_runtime': '1.233e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180550/225000 [342:31:43<74:38:32,  6.05s/it]
 80%|████████  | 180551/225000 [342:31:46<65:46:12,  5.33s/it]
 80%|████████  | 180552/225000 [342:31:55<76:41:23,  6.21s/it]
 80%|████████  | 180553/225000 [342:31:58<68:13:21,  5.53s/it]
 80%|████████  | 180554/225000 [342:32:03<63:02:04,  5.11s/it]
 80%|████████  | 180555/225000 [342:32:07<59:30:56,  4.82s/it]
 80%|████████  | 180556/225000 [342:32:11<57:41:34,  4.67s/it]
 80%|████████  | 180557/225000 [342:32:19<70:55:48,  5.75s/it]
 80%|████████  | 180558/225000 [342:32:26<74:21:28,  6.02s/it]
 80%|████████  | 180559/225000 [342:32:31<69:05:41,  5.60s/it]
 80%|████████  | 180560/225000 [342:32:38<75:28:15,  6.11s/it]
 80%|████████  | 180561/225000 [342:32:42<68:45:16,  5.57s/it]
 80%|████████  | 180562/225000 [342:32:46<63:24:08,  5.14s/it]
 80%|████████  | 180563/225000 [342:32:51<61:29:33,  4.98s/it]
 80%|████████  | 180564/225000 [342:32:55<58:04:21,  4.70s/it]
 80%|████████  | 180565/225000 [342:33:07<84:36:26,  6.85s/it]
 80%|████████  | 180566/225000 [342:33:11<75:56:55,  6.15s/it]
 80%|████████  | 180567/225000 [342:33:16<71:06:26,  5.76s/it]
 80%|████████  | 180568/225000 [342:33:21<67:30:22,  5.47s/it]
 80%|████████  | 180569/225000 [342:33:26<64:00:49,  5.19s/it]
 80%|████████  | 180570/225000 [342:33:30<62:05:21,  5.03s/it]
 80%|████████  | 180571/225000 [342:33:35<61:07:05,  4.95s/it]
 80%|████████  | 180572/225000 [342:33:40<61:08:41,  4.95s/it]
 80%|████████  | 180573/225000 [342:33:45<60:08:35,  4.87s/it]
 80%|████████  | 180574/225000 [342:33:54<75:34:26,  6.12s/it]
 80%|████████  | 180575/225000 [342:34:00<77:56:17,  6.32s/it]
 80%|████████  | 180576/225000 [342:34:04<68:02:14,  5.51s/it]
 80%|████████  | 180577/225000 [342:34:12<77:30:12,  6.28s/it]
 80%|████████  | 180578/225000 [342:34:16<67:45:01,  5.49s/it]
 80%|████████  | 180579/225000 [342:34:20<61:44:21,  5.00s/it]
 80%|████████  | 180580/225000 [342:34:23<56:39:49,  4.59s/it]
 80%|████████  | 180581/225000 [342:34:28<57:54:04,  4.69s/it]
 80%|████████  | 180582/225000 [342:34:34<60:41:51,  4.92s/it]
 80%|████████  | 180583/225000 [342:34:39<63:24:42,  5.14s/it]
 80%|████████  | 180584/225000 [342:34:50<85:26:58,  6.93s/it]
 80%|████████  | 180585/225000 [342:34:54<72:17:47,  5.86s/it]
 80%|████████  | 180586/225000 [342:34:58<67:10:34,  5.44s/it]
 80%|████████  | 180587/225000 [342:35:04<66:44:26,  5.41s/it]
 80%|████████  | 180588/225000 [342:35:09<66:33:52,  5.40s/it]
 80%|████████  | 180589/225000 [342:35:14<64:24:55,  5.22s/it]
 80%|████████  | 180590/225000 [342:35:19<64:19:39,  5.21s/it]
 80%|████████  | 180591/225000 [342:35:24<63:19:49,  5.13s/it]
 80%|████████  | 180592/225000 [342:35:29<62:33:45,  5.07s/it]
 80%|████████  | 180593/225000 [342:35:34<61:21:28,  4.97s/it]
 80%|████████  | 180594/225000 [342:35:39<62:28:23,  5.06s/it]
 80%|████████  | 180595/225000 [342:35:44<63:06:28,  5.12s/it]
 80%|████████  | 180596/225000 [342:35:50<64:50:46,  5.26s/it]
 80%|████████  | 180597/225000 [342:35:55<64:56:58,  5.27s/it]
 80%|████████  | 180598/225000 [342:36:04<78:15:55,  6.35s/it]
 80%|████████  | 180599/225000 [342:36:12<83:58:34,  6.81s/it]
 80%|████████  | 180600/225000 [342:36:18<81:07:29,  6.58s/it]
                                                              
{'loss': '0.7745', 'grad_norm': '1.132', 'learning_rate': '5.098e-06', 'epoch': '0.8027', 'num_input_tokens_seen': 568111008, 'train_runtime': '1.233e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180600/225000 [342:36:18<81:07:29,  6.58s/it]
 80%|████████  | 180601/225000 [342:36:22<71:06:00,  5.76s/it]
 80%|████████  | 180602/225000 [342:36:26<65:09:31,  5.28s/it]
 80%|████████  | 180603/225000 [342:36:34<76:15:36,  6.18s/it]
 80%|████████  | 180604/225000 [342:36:40<73:37:02,  5.97s/it]
 80%|████████  | 180605/225000 [342:36:45<71:41:17,  5.81s/it]
 80%|████████  | 180606/225000 [342:36:51<74:01:11,  6.00s/it]
 80%|████████  | 180607/225000 [342:36:55<64:41:43,  5.25s/it]
 80%|████████  | 180608/225000 [342:37:03<73:32:41,  5.96s/it]
 80%|████████  | 180609/225000 [342:37:07<68:22:55,  5.55s/it]
 80%|████████  | 180610/225000 [342:37:12<66:48:32,  5.42s/it]
 80%|████████  | 180611/225000 [342:37:17<65:52:47,  5.34s/it]
 80%|████████  | 180612/225000 [342:37:22<64:45:17,  5.25s/it]
 80%|████████  | 180613/225000 [342:37:28<63:55:23,  5.18s/it]
 80%|████████  | 180614/225000 [342:37:33<63:58:43,  5.19s/it]
 80%|████████  | 180615/225000 [342:37:38<64:12:16,  5.21s/it]
 80%|████████  | 180616/225000 [342:37:43<62:11:51,  5.04s/it]
 80%|████████  | 180617/225000 [342:37:48<63:00:40,  5.11s/it]
 80%|████████  | 180618/225000 [342:37:53<63:59:57,  5.19s/it]
 80%|████████  | 180619/225000 [342:37:59<64:53:12,  5.26s/it]
 80%|████████  | 180620/225000 [342:38:04<63:30:07,  5.15s/it]
 80%|████████  | 180621/225000 [342:38:09<63:36:31,  5.16s/it]
 80%|████████  | 180622/225000 [342:38:14<62:56:18,  5.11s/it]
 80%|████████  | 180623/225000 [342:38:19<63:42:18,  5.17s/it]
 80%|████████  | 180624/225000 [342:38:24<62:06:23,  5.04s/it]
 80%|████████  | 180625/225000 [342:38:33<76:02:31,  6.17s/it]
 80%|████████  | 180626/225000 [342:38:42<87:08:45,  7.07s/it]
 80%|████████  | 180627/225000 [342:38:45<72:52:24,  5.91s/it]
 80%|████████  | 180628/225000 [342:38:50<70:39:20,  5.73s/it]
 80%|████████  | 180629/225000 [342:38:55<66:44:04,  5.41s/it]
 80%|████████  | 180630/225000 [342:39:00<67:02:51,  5.44s/it]
 80%|████████  | 180631/225000 [342:39:06<66:34:52,  5.40s/it]
 80%|████████  | 180632/225000 [342:39:11<65:38:49,  5.33s/it]
 80%|████████  | 180633/225000 [342:39:16<65:44:05,  5.33s/it]
 80%|████████  | 180634/225000 [342:39:21<65:01:16,  5.28s/it]
 80%|████████  | 180635/225000 [342:39:26<63:33:57,  5.16s/it]
 80%|████████  | 180636/225000 [342:39:32<64:20:53,  5.22s/it]
 80%|████████  | 180637/225000 [342:39:37<63:21:07,  5.14s/it]
 80%|████████  | 180638/225000 [342:39:42<64:33:44,  5.24s/it]
 80%|████████  | 180639/225000 [342:39:47<63:58:55,  5.19s/it]
 80%|████████  | 180640/225000 [342:39:52<64:13:58,  5.21s/it]
 80%|████████  | 180641/225000 [342:39:58<64:00:09,  5.19s/it]
 80%|████████  | 180642/225000 [342:40:03<65:19:12,  5.30s/it]
 80%|████████  | 180643/225000 [342:40:08<64:56:51,  5.27s/it]
 80%|████████  | 180644/225000 [342:40:14<65:15:10,  5.30s/it]
 80%|████████  | 180645/225000 [342:40:19<65:55:54,  5.35s/it]
 80%|████████  | 180646/225000 [342:40:24<64:52:23,  5.27s/it]
 80%|████████  | 180647/225000 [342:40:30<65:23:26,  5.31s/it]
 80%|████████  | 180648/225000 [342:40:35<63:49:18,  5.18s/it]
 80%|████████  | 180649/225000 [342:40:40<63:55:35,  5.19s/it]
 80%|████████  | 180650/225000 [342:40:52<91:26:28,  7.42s/it]
                                                              
{'loss': '0.7663', 'grad_norm': '1.153', 'learning_rate': '5.087e-06', 'epoch': '0.8029', 'num_input_tokens_seen': 568240992, 'train_runtime': '1.234e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180650/225000 [342:40:52<91:26:28,  7.42s/it]
 80%|████████  | 180651/225000 [342:40:57<79:34:58,  6.46s/it]
 80%|████████  | 180652/225000 [342:41:01<71:47:52,  5.83s/it]
 80%|████████  | 180653/225000 [342:41:05<65:34:04,  5.32s/it]
 80%|████████  | 180654/225000 [342:41:09<62:03:04,  5.04s/it]
 80%|████████  | 180655/225000 [342:41:13<58:09:40,  4.72s/it]
 80%|████████  | 180656/225000 [342:41:18<56:22:02,  4.58s/it]
 80%|████████  | 180657/225000 [342:41:29<80:24:14,  6.53s/it]
 80%|████████  | 180658/225000 [342:41:33<73:39:51,  5.98s/it]
 80%|████████  | 180659/225000 [342:41:38<68:26:03,  5.56s/it]
 80%|████████  | 180660/225000 [342:41:42<63:55:40,  5.19s/it]
 80%|████████  | 180661/225000 [342:41:47<62:09:56,  5.05s/it]
 80%|████████  | 180662/225000 [342:41:52<60:32:54,  4.92s/it]
 80%|████████  | 180663/225000 [342:41:56<59:08:29,  4.80s/it]
 80%|████████  | 180664/225000 [342:42:09<88:46:13,  7.21s/it]
 80%|████████  | 180665/225000 [342:42:13<77:18:14,  6.28s/it]
 80%|████████  | 180666/225000 [342:42:17<69:08:17,  5.61s/it]
 80%|████████  | 180667/225000 [342:42:22<64:50:35,  5.27s/it]
 80%|████████  | 180668/225000 [342:42:26<61:02:24,  4.96s/it]
 80%|████████  | 180669/225000 [342:42:31<60:22:05,  4.90s/it]
 80%|████████  | 180670/225000 [342:42:35<59:47:23,  4.86s/it]
 80%|████████  | 180671/225000 [342:42:40<58:23:33,  4.74s/it]
 80%|████████  | 180672/225000 [342:42:45<58:32:38,  4.75s/it]
 80%|████████  | 180673/225000 [342:42:56<81:16:56,  6.60s/it]
 80%|████████  | 180674/225000 [342:43:01<76:28:20,  6.21s/it]
 80%|████████  | 180675/225000 [342:43:05<67:07:43,  5.45s/it]
 80%|████████  | 180676/225000 [342:43:08<60:30:42,  4.91s/it]
 80%|████████  | 180677/225000 [342:43:18<77:13:12,  6.27s/it]
 80%|████████  | 180678/225000 [342:43:23<74:24:50,  6.04s/it]
 80%|████████  | 180679/225000 [342:43:26<64:04:37,  5.20s/it]
 80%|████████  | 180680/225000 [342:43:35<74:53:19,  6.08s/it]
 80%|████████  | 180681/225000 [342:43:38<66:03:53,  5.37s/it]
 80%|████████  | 180682/225000 [342:43:42<60:56:03,  4.95s/it]
 80%|████████  | 180683/225000 [342:43:50<71:07:20,  5.78s/it]
 80%|████████  | 180684/225000 [342:43:57<76:42:31,  6.23s/it]
 80%|████████  | 180685/225000 [342:44:01<66:19:49,  5.39s/it]
 80%|████████  | 180686/225000 [342:44:08<73:22:49,  5.96s/it]
 80%|████████  | 180687/225000 [342:44:12<65:10:25,  5.29s/it]
 80%|████████  | 180688/225000 [342:44:17<65:40:09,  5.34s/it]
 80%|████████  | 180689/225000 [342:44:24<71:26:57,  5.80s/it]
 80%|████████  | 180690/225000 [342:44:28<64:50:43,  5.27s/it]
 80%|████████  | 180691/225000 [342:44:32<58:53:41,  4.79s/it]
 80%|████████  | 180692/225000 [342:44:41<76:32:54,  6.22s/it]
 80%|████████  | 180693/225000 [342:44:45<66:43:10,  5.42s/it]
 80%|████████  | 180694/225000 [342:44:50<67:01:22,  5.45s/it]
 80%|████████  | 180695/225000 [342:44:58<73:26:13,  5.97s/it]
 80%|████████  | 180696/225000 [342:45:00<62:16:57,  5.06s/it]
 80%|████████  | 180697/225000 [342:45:07<68:28:50,  5.56s/it]
 80%|████████  | 180698/225000 [342:45:12<65:38:31,  5.33s/it]
 80%|████████  | 180699/225000 [342:45:17<63:26:29,  5.16s/it]
 80%|████████  | 180700/225000 [342:45:21<61:50:29,  5.03s/it]
                                                              
{'loss': '0.7679', 'grad_norm': '1.29', 'learning_rate': '5.076e-06', 'epoch': '0.8031', 'num_input_tokens_seen': 568370528, 'train_runtime': '1.234e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180700/225000 [342:45:21<61:50:29,  5.03s/it]
 80%|████████  | 180701/225000 [342:45:26<59:28:42,  4.83s/it]
 80%|████████  | 180702/225000 [342:45:31<59:49:04,  4.86s/it]
 80%|████████  | 180703/225000 [342:45:36<59:53:09,  4.87s/it]
 80%|████████  | 180704/225000 [342:45:40<59:22:41,  4.83s/it]
 80%|████████  | 180705/225000 [342:45:45<58:49:01,  4.78s/it]
 80%|████████  | 180706/225000 [342:45:50<58:00:40,  4.71s/it]
 80%|████████  | 180707/225000 [342:45:54<57:30:30,  4.67s/it]
 80%|████████  | 180708/225000 [342:45:59<59:05:35,  4.80s/it]
 80%|████████  | 180709/225000 [342:46:04<57:39:45,  4.69s/it]
 80%|████████  | 180710/225000 [342:46:16<84:23:16,  6.86s/it]
 80%|████████  | 180711/225000 [342:46:22<83:40:37,  6.80s/it]
 80%|████████  | 180712/225000 [342:46:27<75:12:05,  6.11s/it]
 80%|████████  | 180713/225000 [342:46:32<70:00:15,  5.69s/it]
 80%|████████  | 180714/225000 [342:46:36<66:50:56,  5.43s/it]
 80%|████████  | 180715/225000 [342:46:41<62:54:38,  5.11s/it]
 80%|████████  | 180716/225000 [342:46:45<61:20:28,  4.99s/it]
 80%|████████  | 180717/225000 [342:46:52<68:26:29,  5.56s/it]
 80%|████████  | 180718/225000 [342:47:05<94:25:12,  7.68s/it]
 80%|████████  | 180719/225000 [342:47:09<79:19:19,  6.45s/it]
 80%|████████  | 180720/225000 [342:47:13<70:17:19,  5.71s/it]
 80%|████████  | 180721/225000 [342:47:19<72:56:18,  5.93s/it]
 80%|████████  | 180722/225000 [342:47:26<77:21:39,  6.29s/it]
 80%|████████  | 180723/225000 [342:47:30<70:05:41,  5.70s/it]
 80%|████████  | 180724/225000 [342:47:34<63:30:36,  5.16s/it]
 80%|████████  | 180725/225000 [342:47:38<59:10:26,  4.81s/it]
 80%|████████  | 180726/225000 [342:47:43<60:01:39,  4.88s/it]
 80%|████████  | 180727/225000 [342:47:54<79:27:43,  6.46s/it]
 80%|████████  | 180728/225000 [342:47:57<69:03:05,  5.61s/it]
 80%|████████  | 180729/225000 [342:48:02<65:06:36,  5.29s/it]
 80%|████████  | 180730/225000 [342:48:06<62:13:14,  5.06s/it]
 80%|████████  | 180731/225000 [342:48:10<58:30:51,  4.76s/it]
 80%|████████  | 180732/225000 [342:48:17<64:07:01,  5.21s/it]
 80%|████████  | 180733/225000 [342:48:28<88:11:02,  7.17s/it]
 80%|████████  | 180734/225000 [342:48:32<74:18:53,  6.04s/it]
 80%|████████  | 180735/225000 [342:48:39<77:14:58,  6.28s/it]
 80%|████████  | 180736/225000 [342:48:43<70:01:20,  5.69s/it]
 80%|████████  | 180737/225000 [342:48:50<76:53:32,  6.25s/it]
 80%|████████  | 180738/225000 [342:48:54<66:57:09,  5.45s/it]
 80%|████████  | 180739/225000 [342:49:02<75:44:25,  6.16s/it]
 80%|████████  | 180740/225000 [342:49:06<70:01:03,  5.70s/it]
 80%|████████  | 180741/225000 [342:49:11<66:30:06,  5.41s/it]
 80%|████████  | 180742/225000 [342:49:18<71:36:57,  5.83s/it]
 80%|████████  | 180743/225000 [342:49:21<62:47:23,  5.11s/it]
 80%|████████  | 180744/225000 [342:49:29<73:14:01,  5.96s/it]
 80%|████████  | 180745/225000 [342:49:33<65:04:34,  5.29s/it]
 80%|████████  | 180746/225000 [342:49:39<66:40:00,  5.42s/it]
 80%|████████  | 180747/225000 [342:49:46<71:28:18,  5.81s/it]
 80%|████████  | 180748/225000 [342:49:49<63:04:05,  5.13s/it]
 80%|████████  | 180749/225000 [342:49:56<68:51:23,  5.60s/it]
 80%|████████  | 180750/225000 [342:50:00<65:10:58,  5.30s/it]
                                                              
{'loss': '0.7605', 'grad_norm': '1.382', 'learning_rate': '5.065e-06', 'epoch': '0.8033', 'num_input_tokens_seen': 568500880, 'train_runtime': '1.234e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180750/225000 [342:50:00<65:10:58,  5.30s/it]
 80%|████████  | 180751/225000 [342:50:04<60:27:05,  4.92s/it]
 80%|████████  | 180752/225000 [342:50:08<57:01:29,  4.64s/it]
 80%|████████  | 180753/225000 [342:50:13<55:11:45,  4.49s/it]
 80%|████████  | 180754/225000 [342:50:18<58:05:36,  4.73s/it]
 80%|████████  | 180755/225000 [342:50:23<61:23:58,  5.00s/it]
 80%|████████  | 180756/225000 [342:50:29<62:15:09,  5.07s/it]
 80%|████████  | 180757/225000 [342:50:34<63:35:04,  5.17s/it]
 80%|████████  | 180758/225000 [342:50:39<63:30:29,  5.17s/it]
 80%|████████  | 180759/225000 [342:50:44<63:37:39,  5.18s/it]
 80%|████████  | 180760/225000 [342:50:50<63:51:50,  5.20s/it]
 80%|████████  | 180761/225000 [342:50:55<63:00:11,  5.13s/it]
 80%|████████  | 180762/225000 [342:51:07<88:56:20,  7.24s/it]
 80%|████████  | 180763/225000 [342:51:13<83:40:40,  6.81s/it]
 80%|████████  | 180764/225000 [342:51:16<71:57:42,  5.86s/it]
 80%|████████  | 180765/225000 [342:51:21<66:17:25,  5.39s/it]
 80%|████████  | 180766/225000 [342:51:25<62:53:47,  5.12s/it]
 80%|████████  | 180767/225000 [342:51:31<65:08:30,  5.30s/it]
 80%|████████  | 180768/225000 [342:51:40<80:00:24,  6.51s/it]
 80%|████████  | 180769/225000 [342:51:47<80:30:04,  6.55s/it]
 80%|████████  | 180770/225000 [342:51:50<68:49:47,  5.60s/it]
 80%|████████  | 180771/225000 [342:51:56<68:25:34,  5.57s/it]
 80%|████████  | 180772/225000 [342:52:03<75:15:51,  6.13s/it]
 80%|████████  | 180773/225000 [342:52:07<67:17:16,  5.48s/it]
 80%|████████  | 180774/225000 [342:52:14<72:49:58,  5.93s/it]
 80%|████████  | 180775/225000 [342:52:17<62:54:18,  5.12s/it]
 80%|████████  | 180776/225000 [342:52:22<62:45:13,  5.11s/it]
 80%|████████  | 180777/225000 [342:52:27<62:32:42,  5.09s/it]
 80%|████████  | 180778/225000 [342:52:32<61:32:04,  5.01s/it]
 80%|████████  | 180779/225000 [342:52:37<60:05:34,  4.89s/it]
 80%|████████  | 180780/225000 [342:52:41<59:10:39,  4.82s/it]
 80%|████████  | 180781/225000 [342:52:47<60:26:16,  4.92s/it]
 80%|████████  | 180782/225000 [342:52:52<60:48:51,  4.95s/it]
 80%|████████  | 180783/225000 [342:52:57<60:49:42,  4.95s/it]
 80%|████████  | 180784/225000 [342:53:02<60:48:12,  4.95s/it]
 80%|████████  | 180785/225000 [342:53:07<61:43:34,  5.03s/it]
 80%|████████  | 180786/225000 [342:53:12<61:34:14,  5.01s/it]
 80%|████████  | 180787/225000 [342:53:17<61:09:44,  4.98s/it]
 80%|████████  | 180788/225000 [342:53:24<69:35:53,  5.67s/it]
 80%|████████  | 180789/225000 [342:53:35<91:19:06,  7.44s/it]
 80%|████████  | 180790/225000 [342:53:39<76:56:06,  6.26s/it]
 80%|████████  | 180791/225000 [342:53:46<79:26:41,  6.47s/it]
 80%|████████  | 180792/225000 [342:53:50<70:52:39,  5.77s/it]
 80%|████████  | 180793/225000 [342:53:54<63:57:21,  5.21s/it]
 80%|████████  | 180794/225000 [342:53:58<58:37:06,  4.77s/it]
 80%|████████  | 180795/225000 [342:54:08<80:02:00,  6.52s/it]
 80%|████████  | 180796/225000 [342:54:12<69:16:26,  5.64s/it]
 80%|████████  | 180797/225000 [342:54:16<62:26:56,  5.09s/it]
 80%|████████  | 180798/225000 [342:54:24<74:57:40,  6.11s/it]
 80%|████████  | 180799/225000 [342:54:29<68:39:36,  5.59s/it]
 80%|████████  | 180800/225000 [342:54:33<62:39:23,  5.10s/it]
                                                              
{'loss': '0.7659', 'grad_norm': '1.33', 'learning_rate': '5.054e-06', 'epoch': '0.8036', 'num_input_tokens_seen': 568631432, 'train_runtime': '1.234e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180800/225000 [342:54:33<62:39:23,  5.10s/it]
 80%|████████  | 180801/225000 [342:54:37<60:13:26,  4.91s/it]
 80%|████████  | 180802/225000 [342:54:42<59:48:59,  4.87s/it]
 80%|████████  | 180803/225000 [342:54:46<59:05:22,  4.81s/it]
 80%|████████  | 180804/225000 [342:54:53<65:43:55,  5.35s/it]
 80%|████████  | 180805/225000 [342:55:03<83:05:28,  6.77s/it]
 80%|████████  | 180806/225000 [342:55:06<70:05:39,  5.71s/it]
 80%|████████  | 180807/225000 [342:55:13<73:00:36,  5.95s/it]
 80%|████████  | 180808/225000 [342:55:17<65:57:27,  5.37s/it]
 80%|████████  | 180809/225000 [342:55:21<61:35:06,  5.02s/it]
 80%|████████  | 180810/225000 [342:55:26<61:00:48,  4.97s/it]
 80%|████████  | 180811/225000 [342:55:30<58:13:15,  4.74s/it]
 80%|████████  | 180812/225000 [342:55:35<58:06:35,  4.73s/it]
 80%|████████  | 180813/225000 [342:55:44<72:57:33,  5.94s/it]
 80%|████████  | 180814/225000 [342:55:52<80:51:22,  6.59s/it]
 80%|████████  | 180815/225000 [342:55:55<69:17:52,  5.65s/it]
 80%|████████  | 180816/225000 [342:55:59<63:46:22,  5.20s/it]
 80%|████████  | 180817/225000 [342:56:07<72:13:47,  5.89s/it]
 80%|████████  | 180818/225000 [342:56:14<76:36:50,  6.24s/it]
 80%|████████  | 180819/225000 [342:56:18<67:14:17,  5.48s/it]
 80%|████████  | 180820/225000 [342:56:26<79:12:17,  6.45s/it]
 80%|████████  | 180821/225000 [342:56:30<67:33:22,  5.50s/it]
 80%|████████  | 180822/225000 [342:56:36<72:17:08,  5.89s/it]
 80%|████████  | 180823/225000 [342:56:40<63:13:41,  5.15s/it]
 80%|████████  | 180824/225000 [342:56:48<75:10:18,  6.13s/it]
 80%|████████  | 180825/225000 [342:56:53<69:17:21,  5.65s/it]
 80%|████████  | 180826/225000 [342:56:59<70:35:02,  5.75s/it]
 80%|████████  | 180827/225000 [342:57:04<67:48:44,  5.53s/it]
 80%|████████  | 180828/225000 [342:57:10<69:04:31,  5.63s/it]
 80%|████████  | 180829/225000 [342:57:15<68:53:30,  5.61s/it]
 80%|████████  | 180830/225000 [342:57:19<63:11:06,  5.15s/it]
 80%|████████  | 180831/225000 [342:57:24<60:49:24,  4.96s/it]
 80%|████████  | 180832/225000 [342:57:28<56:51:29,  4.63s/it]
 80%|████████  | 180833/225000 [342:57:32<57:15:04,  4.67s/it]
 80%|████████  | 180834/225000 [342:57:39<64:27:48,  5.25s/it]
 80%|████████  | 180835/225000 [342:57:49<83:30:43,  6.81s/it]
 80%|████████  | 180836/225000 [342:57:55<77:51:10,  6.35s/it]
 80%|████████  | 180837/225000 [342:58:00<72:39:48,  5.92s/it]
 80%|████████  | 180838/225000 [342:58:05<69:06:38,  5.63s/it]
 80%|████████  | 180839/225000 [342:58:10<67:20:27,  5.49s/it]
 80%|████████  | 180840/225000 [342:58:15<65:25:07,  5.33s/it]
 80%|████████  | 180841/225000 [342:58:20<63:47:48,  5.20s/it]
 80%|████████  | 180842/225000 [342:58:25<63:31:43,  5.18s/it]
 80%|████████  | 180843/225000 [342:58:29<61:51:21,  5.04s/it]
 80%|████████  | 180844/225000 [342:58:34<61:40:22,  5.03s/it]
 80%|████████  | 180845/225000 [342:58:40<62:15:49,  5.08s/it]
 80%|████████  | 180846/225000 [342:58:44<61:12:08,  4.99s/it]
 80%|████████  | 180847/225000 [342:58:49<61:03:05,  4.98s/it]
 80%|████████  | 180848/225000 [342:58:55<61:42:19,  5.03s/it]
 80%|████████  | 180849/225000 [342:59:00<61:32:52,  5.02s/it]
 80%|████████  | 180850/225000 [342:59:05<62:20:05,  5.08s/it]
                                                              
{'loss': '0.7652', 'grad_norm': '1.075', 'learning_rate': '5.043e-06', 'epoch': '0.8038', 'num_input_tokens_seen': 568761920, 'train_runtime': '1.235e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180850/225000 [342:59:05<62:20:05,  5.08s/it]
 80%|████████  | 180851/225000 [342:59:10<61:25:27,  5.01s/it]
 80%|████████  | 180852/225000 [342:59:15<61:51:50,  5.04s/it]
 80%|████████  | 180853/225000 [342:59:22<70:56:54,  5.79s/it]
 80%|████████  | 180854/225000 [342:59:33<88:50:31,  7.24s/it]
 80%|████████  | 180855/225000 [342:59:36<74:34:36,  6.08s/it]
 80%|████████  | 180856/225000 [342:59:43<78:11:45,  6.38s/it]
 80%|████████  | 180857/225000 [342:59:48<70:51:17,  5.78s/it]
 80%|████████  | 180858/225000 [342:59:52<64:38:12,  5.27s/it]
 80%|████████  | 180859/225000 [342:59:58<68:11:47,  5.56s/it]
 80%|████████  | 180860/225000 [343:00:03<67:13:12,  5.48s/it]
 80%|████████  | 180861/225000 [343:00:07<61:47:46,  5.04s/it]
 80%|████████  | 180862/225000 [343:00:14<68:18:19,  5.57s/it]
 80%|████████  | 180863/225000 [343:00:22<76:58:35,  6.28s/it]
 80%|████████  | 180864/225000 [343:00:25<65:43:12,  5.36s/it]
 80%|████████  | 180865/225000 [343:00:33<74:32:16,  6.08s/it]
 80%|████████  | 180866/225000 [343:00:37<66:14:45,  5.40s/it]
 80%|████████  | 180867/225000 [343:00:40<59:17:51,  4.84s/it]
 80%|████████  | 180868/225000 [343:00:50<76:15:53,  6.22s/it]
 80%|████████  | 180869/225000 [343:00:53<66:38:33,  5.44s/it]
 80%|████████  | 180870/225000 [343:01:01<73:05:36,  5.96s/it]
 80%|████████  | 180871/225000 [343:01:06<70:25:00,  5.74s/it]
 80%|████████  | 180872/225000 [343:01:11<68:37:37,  5.60s/it]
 80%|████████  | 180873/225000 [343:01:16<66:31:02,  5.43s/it]
 80%|████████  | 180874/225000 [343:01:21<65:34:48,  5.35s/it]
 80%|████████  | 180875/225000 [343:01:27<66:21:20,  5.41s/it]
 80%|████████  | 180876/225000 [343:01:32<64:51:53,  5.29s/it]
 80%|████████  | 180877/225000 [343:01:38<65:53:52,  5.38s/it]
 80%|████████  | 180878/225000 [343:01:43<66:37:11,  5.44s/it]
 80%|████████  | 180879/225000 [343:01:48<65:41:42,  5.36s/it]
 80%|████████  | 180880/225000 [343:01:54<65:29:26,  5.34s/it]
 80%|████████  | 180881/225000 [343:01:58<63:27:39,  5.18s/it]
 80%|████████  | 180882/225000 [343:02:03<62:44:10,  5.12s/it]
 80%|████████  | 180883/225000 [343:02:08<60:31:11,  4.94s/it]
 80%|████████  | 180884/225000 [343:02:13<61:34:59,  5.03s/it]
 80%|████████  | 180885/225000 [343:02:19<63:01:37,  5.14s/it]
 80%|████████  | 180886/225000 [343:02:24<63:11:31,  5.16s/it]
 80%|████████  | 180887/225000 [343:02:29<62:45:39,  5.12s/it]
 80%|████████  | 180888/225000 [343:02:34<61:28:45,  5.02s/it]
 80%|████████  | 180889/225000 [343:02:39<63:12:12,  5.16s/it]
 80%|████████  | 180890/225000 [343:02:44<64:00:54,  5.22s/it]
 80%|████████  | 180891/225000 [343:02:50<64:55:25,  5.30s/it]
 80%|████████  | 180892/225000 [343:02:55<65:00:59,  5.31s/it]
 80%|████████  | 180893/225000 [343:03:01<68:44:33,  5.61s/it]
 80%|████████  | 180894/225000 [343:03:12<85:22:03,  6.97s/it]
 80%|████████  | 180895/225000 [343:03:15<73:28:44,  6.00s/it]
 80%|████████  | 180896/225000 [343:03:24<81:31:16,  6.65s/it]
 80%|████████  | 180897/225000 [343:03:30<80:14:28,  6.55s/it]
 80%|████████  | 180898/225000 [343:03:34<71:27:04,  5.83s/it]
 80%|████████  | 180899/225000 [343:03:38<64:29:05,  5.26s/it]
 80%|████████  | 180900/225000 [343:03:44<67:26:37,  5.51s/it]
                                                              
{'loss': '0.7563', 'grad_norm': '1.165', 'learning_rate': '5.032e-06', 'epoch': '0.804', 'num_input_tokens_seen': 568891360, 'train_runtime': '1.235e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180900/225000 [343:03:44<67:26:37,  5.51s/it]
 80%|████████  | 180901/225000 [343:03:52<75:29:30,  6.16s/it]
 80%|████████  | 180902/225000 [343:03:55<64:39:28,  5.28s/it]
 80%|████████  | 180903/225000 [343:04:04<77:09:14,  6.30s/it]
 80%|████████  | 180904/225000 [343:04:07<66:27:49,  5.43s/it]
 80%|████████  | 180905/225000 [343:04:10<59:12:00,  4.83s/it]
 80%|████████  | 180906/225000 [343:04:18<70:35:24,  5.76s/it]
 80%|████████  | 180907/225000 [343:04:22<62:45:13,  5.12s/it]
 80%|████████  | 180908/225000 [343:04:26<57:48:39,  4.72s/it]
 80%|████████  | 180909/225000 [343:04:30<54:37:05,  4.46s/it]
 80%|████████  | 180910/225000 [343:04:38<69:47:27,  5.70s/it]
 80%|████████  | 180911/225000 [343:04:45<72:22:18,  5.91s/it]
 80%|████████  | 180912/225000 [343:04:49<65:52:07,  5.38s/it]
 80%|████████  | 180913/225000 [343:04:53<60:04:20,  4.91s/it]
 80%|████████  | 180914/225000 [343:04:57<57:00:04,  4.65s/it]
 80%|████████  | 180915/225000 [343:05:07<77:10:32,  6.30s/it]
 80%|████████  | 180916/225000 [343:05:10<66:36:16,  5.44s/it]
 80%|████████  | 180917/225000 [343:05:18<73:22:59,  5.99s/it]
 80%|████████  | 180918/225000 [343:05:21<63:28:20,  5.18s/it]
 80%|████████  | 180919/225000 [343:05:25<58:48:59,  4.80s/it]
 80%|████████  | 180920/225000 [343:05:29<57:02:20,  4.66s/it]
 80%|████████  | 180921/225000 [343:05:35<63:38:20,  5.20s/it]
 80%|████████  | 180922/225000 [343:05:44<76:01:22,  6.21s/it]
 80%|████████  | 180923/225000 [343:05:48<66:39:16,  5.44s/it]
 80%|████████  | 180924/225000 [343:05:54<71:02:04,  5.80s/it]
 80%|████████  | 180925/225000 [343:06:00<69:35:28,  5.68s/it]
 80%|████████  | 180926/225000 [343:06:07<75:39:35,  6.18s/it]
 80%|████████  | 180927/225000 [343:06:11<66:36:04,  5.44s/it]
 80%|████████  | 180928/225000 [343:06:17<69:44:53,  5.70s/it]
 80%|████████  | 180929/225000 [343:06:21<62:03:36,  5.07s/it]
 80%|████████  | 180930/225000 [343:06:26<63:27:48,  5.18s/it]
 80%|████████  | 180931/225000 [343:06:32<67:07:37,  5.48s/it]
 80%|████████  | 180932/225000 [343:06:37<62:35:21,  5.11s/it]
 80%|████████  | 180933/225000 [343:06:44<71:35:06,  5.85s/it]
 80%|████████  | 180934/225000 [343:06:47<61:28:57,  5.02s/it]
 80%|████████  | 180935/225000 [343:06:54<67:59:59,  5.56s/it]
 80%|████████  | 180936/225000 [343:06:58<62:03:01,  5.07s/it]
 80%|████████  | 180937/225000 [343:07:02<57:22:32,  4.69s/it]
 80%|████████  | 180938/225000 [343:07:06<56:57:39,  4.65s/it]
 80%|████████  | 180939/225000 [343:07:11<55:07:24,  4.50s/it]
 80%|████████  | 180940/225000 [343:07:23<83:50:53,  6.85s/it]
 80%|████████  | 180941/225000 [343:07:27<72:47:34,  5.95s/it]
 80%|████████  | 180942/225000 [343:07:32<70:40:28,  5.77s/it]
 80%|████████  | 180943/225000 [343:07:37<67:05:57,  5.48s/it]
 80%|████████  | 180944/225000 [343:07:42<66:24:07,  5.43s/it]
 80%|████████  | 180945/225000 [343:07:47<64:39:23,  5.28s/it]
 80%|████████  | 180946/225000 [343:07:52<63:59:22,  5.23s/it]
 80%|████████  | 180947/225000 [343:07:57<61:00:04,  4.98s/it]
 80%|████████  | 180948/225000 [343:08:01<59:54:00,  4.90s/it]
 80%|████████  | 180949/225000 [343:08:06<60:42:25,  4.96s/it]
 80%|████████  | 180950/225000 [343:08:12<63:50:18,  5.22s/it]
                                                              
{'loss': '0.7661', 'grad_norm': '0.9874', 'learning_rate': '5.021e-06', 'epoch': '0.8042', 'num_input_tokens_seen': 569021600, 'train_runtime': '1.235e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 180950/225000 [343:08:12<63:50:18,  5.22s/it]
 80%|████████  | 180951/225000 [343:08:18<64:16:38,  5.25s/it]
 80%|████████  | 180952/225000 [343:08:22<62:11:06,  5.08s/it]
 80%|████████  | 180953/225000 [343:08:27<62:21:34,  5.10s/it]
 80%|████████  | 180954/225000 [343:08:33<63:09:46,  5.16s/it]
 80%|████████  | 180955/225000 [343:08:38<62:06:16,  5.08s/it]
 80%|████████  | 180956/225000 [343:08:42<61:29:08,  5.03s/it]
 80%|████████  | 180957/225000 [343:08:48<63:09:06,  5.16s/it]
 80%|████████  | 180958/225000 [343:08:53<63:44:29,  5.21s/it]
 80%|████████  | 180959/225000 [343:09:05<87:07:43,  7.12s/it]
 80%|████████  | 180960/225000 [343:09:10<80:56:25,  6.62s/it]
 80%|████████  | 180961/225000 [343:09:14<69:37:04,  5.69s/it]
 80%|████████  | 180962/225000 [343:09:22<79:10:35,  6.47s/it]
 80%|████████  | 180963/225000 [343:09:26<68:22:21,  5.59s/it]
 80%|████████  | 180964/225000 [343:09:33<75:33:46,  6.18s/it]
 80%|████████  | 180965/225000 [343:09:37<66:35:48,  5.44s/it]
 80%|████████  | 180966/225000 [343:09:41<60:56:27,  4.98s/it]
 80%|████████  | 180967/225000 [343:09:45<56:11:26,  4.59s/it]
 80%|████████  | 180968/225000 [343:09:55<77:12:38,  6.31s/it]
 80%|████████  | 180969/225000 [343:09:59<68:32:11,  5.60s/it]
 80%|████████  | 180970/225000 [343:10:03<62:17:48,  5.09s/it]
 80%|████████  | 180971/225000 [343:10:07<57:55:32,  4.74s/it]
 80%|████████  | 180972/225000 [343:10:11<56:28:33,  4.62s/it]
 80%|████████  | 180973/225000 [343:10:21<77:05:01,  6.30s/it]
 80%|████████  | 180974/225000 [343:10:25<67:19:05,  5.50s/it]
 80%|████████  | 180975/225000 [343:10:30<64:26:27,  5.27s/it]
 80%|████████  | 180976/225000 [343:10:34<60:59:21,  4.99s/it]
 80%|████████  | 180977/225000 [343:10:39<59:59:38,  4.91s/it]
 80%|████████  | 180978/225000 [343:10:44<60:22:15,  4.94s/it]
 80%|████████  | 180979/225000 [343:10:48<59:52:42,  4.90s/it]
 80%|████████  | 180980/225000 [343:10:58<76:09:21,  6.23s/it]
 80%|████████  | 180981/225000 [343:11:05<80:16:54,  6.57s/it]
 80%|████████  | 180982/225000 [343:11:09<71:16:04,  5.83s/it]
 80%|████████  | 180983/225000 [343:11:13<64:21:19,  5.26s/it]
 80%|████████  | 180984/225000 [343:11:22<79:11:20,  6.48s/it]
 80%|████████  | 180985/225000 [343:11:26<68:39:14,  5.62s/it]
 80%|████████  | 180986/225000 [343:11:30<61:11:56,  5.01s/it]
 80%|████████  | 180987/225000 [343:11:40<79:00:10,  6.46s/it]
 80%|████████  | 180988/225000 [343:11:43<68:57:07,  5.64s/it]
 80%|████████  | 180989/225000 [343:11:47<61:05:14,  5.00s/it]
 80%|████████  | 180990/225000 [343:11:57<78:41:43,  6.44s/it]
 80%|████████  | 180991/225000 [343:12:00<68:16:44,  5.59s/it]
 80%|████████  | 180992/225000 [343:12:04<62:32:41,  5.12s/it]
 80%|████████  | 180993/225000 [343:12:08<57:16:37,  4.69s/it]
 80%|████████  | 180994/225000 [343:12:12<53:53:20,  4.41s/it]
 80%|████████  | 180995/225000 [343:12:23<78:08:50,  6.39s/it]
 80%|████████  | 180996/225000 [343:12:26<66:24:27,  5.43s/it]
 80%|████████  | 180997/225000 [343:12:29<59:28:27,  4.87s/it]
 80%|████████  | 180998/225000 [343:12:37<68:00:00,  5.56s/it]
 80%|████████  | 180999/225000 [343:12:40<59:56:07,  4.90s/it]
 80%|████████  | 181000/225000 [343:12:48<72:15:25,  5.91s/it]
                                                              
{'loss': '0.7677', 'grad_norm': '1.048', 'learning_rate': '5.01e-06', 'epoch': '0.8044', 'num_input_tokens_seen': 569150824, 'train_runtime': '1.236e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 181000/225000 [343:12:48<72:15:25,  5.91s/it]
 80%|████████  | 181001/225000 [343:12:53<68:47:48,  5.63s/it]
 80%|████████  | 181002/225000 [343:12:57<62:20:52,  5.10s/it]
 80%|████████  | 181003/225000 [343:13:03<66:56:10,  5.48s/it]
 80%|████████  | 181004/225000 [343:13:07<58:44:34,  4.81s/it]
 80%|████████  | 181005/225000 [343:13:13<63:32:31,  5.20s/it]
 80%|████████  | 181006/225000 [343:13:20<72:56:02,  5.97s/it]
 80%|████████  | 181007/225000 [343:13:25<67:21:08,  5.51s/it]
 80%|████████  | 181008/225000 [343:13:29<63:27:50,  5.19s/it]
 80%|████████  | 181009/225000 [343:13:34<60:48:31,  4.98s/it]
 80%|████████  | 181010/225000 [343:13:38<59:29:59,  4.87s/it]
 80%|████████  | 181011/225000 [343:13:43<59:05:45,  4.84s/it]
 80%|████████  | 181012/225000 [343:13:48<58:34:38,  4.79s/it]
 80%|████████  | 181013/225000 [343:13:52<56:27:14,  4.62s/it]
 80%|████████  | 181014/225000 [343:13:56<54:57:53,  4.50s/it]
 80%|████████  | 181015/225000 [343:14:06<75:19:06,  6.16s/it]
 80%|████████  | 181016/225000 [343:14:12<73:16:24,  6.00s/it]
 80%|████████  | 181017/225000 [343:14:18<73:46:49,  6.04s/it]
 80%|████████  | 181018/225000 [343:14:24<73:41:01,  6.03s/it]
 80%|████████  | 181019/225000 [343:14:29<67:52:10,  5.56s/it]
 80%|████████  | 181020/225000 [343:14:33<63:52:46,  5.23s/it]
 80%|████████  | 181021/225000 [343:14:38<61:03:45,  5.00s/it]
 80%|████████  | 181022/225000 [343:14:42<59:23:17,  4.86s/it]
 80%|████████  | 181023/225000 [343:14:47<57:51:12,  4.74s/it]
 80%|████████  | 181024/225000 [343:14:51<56:06:07,  4.59s/it]
 80%|████████  | 181025/225000 [343:15:02<82:06:05,  6.72s/it]
 80%|████████  | 181026/225000 [343:15:06<69:16:39,  5.67s/it]
 80%|████████  | 181027/225000 [343:15:10<63:33:56,  5.20s/it]
 80%|████████  | 181028/225000 [343:15:14<59:48:28,  4.90s/it]
 80%|████████  | 181029/225000 [343:15:24<78:24:20,  6.42s/it]
 80%|████████  | 181030/225000 [343:15:30<76:17:06,  6.25s/it]
 80%|████████  | 181031/225000 [343:15:34<68:28:06,  5.61s/it]
 80%|████████  | 181032/225000 [343:15:38<61:43:33,  5.05s/it]
 80%|████████  | 181033/225000 [343:15:47<78:15:53,  6.41s/it]
 80%|████████  | 181034/225000 [343:15:50<66:22:42,  5.44s/it]
 80%|████████  | 181035/225000 [343:15:54<60:11:19,  4.93s/it]
 80%|████████  | 181036/225000 [343:16:05<81:13:24,  6.65s/it]
 80%|████████  | 181037/225000 [343:16:10<74:35:19,  6.11s/it]
 80%|████████  | 181038/225000 [343:16:14<69:29:35,  5.69s/it]
 80%|████████  | 181039/225000 [343:16:19<66:46:30,  5.47s/it]
 80%|████████  | 181040/225000 [343:16:24<64:47:21,  5.31s/it]
 80%|████████  | 181041/225000 [343:16:28<60:44:36,  4.97s/it]
 80%|████████  | 181042/225000 [343:16:34<61:05:12,  5.00s/it]
 80%|████████  | 181043/225000 [343:16:38<58:41:07,  4.81s/it]
 80%|████████  | 181044/225000 [343:16:43<58:12:33,  4.77s/it]
 80%|████████  | 181045/225000 [343:16:47<57:18:10,  4.69s/it]
 80%|████████  | 181046/225000 [343:16:52<58:05:39,  4.76s/it]
 80%|████████  | 181047/225000 [343:16:57<58:04:32,  4.76s/it]
 80%|████████  | 181048/225000 [343:17:05<69:20:22,  5.68s/it]
 80%|████████  | 181049/225000 [343:17:14<82:50:25,  6.79s/it]
 80%|████████  | 181050/225000 [343:17:17<70:04:57,  5.74s/it]
                                                              
{'loss': '0.7614', 'grad_norm': '1.106', 'learning_rate': '4.999e-06', 'epoch': '0.8047', 'num_input_tokens_seen': 569279536, 'train_runtime': '1.236e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 181050/225000 [343:17:17<70:04:57,  5.74s/it]
 80%|████████  | 181051/225000 [343:17:21<62:14:23,  5.10s/it]
 80%|████████  | 181052/225000 [343:17:30<76:18:47,  6.25s/it]
 80%|████████  | 181053/225000 [343:17:33<66:14:42,  5.43s/it]
 80%|████████  | 181054/225000 [343:17:38<62:21:11,  5.11s/it]
 80%|████████  | 181055/225000 [343:17:46<74:13:56,  6.08s/it]
 80%|████████  | 181056/225000 [343:17:49<64:26:17,  5.28s/it]
 80%|████████  | 181057/225000 [343:17:58<77:22:11,  6.34s/it]
 80%|████████  | 181058/225000 [343:18:02<67:42:44,  5.55s/it]
 80%|████████  | 181059/225000 [343:18:09<73:04:47,  5.99s/it]
 80%|████████  | 181060/225000 [343:18:14<69:50:07,  5.72s/it]
 80%|████████  | 181061/225000 [343:18:17<60:49:10,  4.98s/it]
 80%|████████  | 181062/225000 [343:18:25<70:30:48,  5.78s/it]
 80%|████████  | 181063/225000 [343:18:29<63:26:00,  5.20s/it]
 80%|████████  | 181064/225000 [343:18:37<73:18:07,  6.01s/it]
 80%|████████  | 181065/225000 [343:18:41<67:08:26,  5.50s/it]
 80%|████████  | 181066/225000 [343:18:45<62:19:12,  5.11s/it]
 80%|████████  | 181067/225000 [343:18:50<59:33:17,  4.88s/it]
 80%|████████  | 181068/225000 [343:18:54<58:37:51,  4.80s/it]
 80%|████████  | 181069/225000 [343:18:59<57:06:31,  4.68s/it]
 80%|████████  | 181070/225000 [343:19:03<56:26:06,  4.62s/it]
 80%|████████  | 181071/225000 [343:19:13<77:40:28,  6.37s/it]
 80%|████████  | 181072/225000 [343:19:18<71:05:04,  5.83s/it]
 80%|████████  | 181073/225000 [343:19:22<65:13:13,  5.35s/it]
 80%|████████  | 181074/225000 [343:19:27<62:12:12,  5.10s/it]
 80%|████████  | 181075/225000 [343:19:31<59:46:16,  4.90s/it]
 80%|████████  | 181076/225000 [343:19:36<59:03:34,  4.84s/it]
 80%|████████  | 181077/225000 [343:19:40<58:00:50,  4.75s/it]
 80%|████████  | 181078/225000 [343:19:53<85:41:07,  7.02s/it]
 80%|████████  | 181079/225000 [343:19:56<73:00:33,  5.98s/it]
 80%|████████  | 181080/225000 [343:20:04<79:12:02,  6.49s/it]
 80%|████████  | 181081/225000 [343:20:08<68:35:28,  5.62s/it]
 80%|████████  | 181082/225000 [343:20:16<78:37:03,  6.44s/it]
 80%|████████  | 181083/225000 [343:20:21<72:46:20,  5.97s/it]
 80%|████████  | 181084/225000 [343:20:25<66:09:14,  5.42s/it]
 80%|████████  | 181085/225000 [343:20:29<60:49:17,  4.99s/it]
 80%|████████  | 181086/225000 [343:20:33<56:30:45,  4.63s/it]
 80%|████████  | 181087/225000 [343:20:42<73:52:29,  6.06s/it]
 80%|████████  | 181088/225000 [343:20:45<63:17:22,  5.19s/it]
 80%|████████  | 181089/225000 [343:20:54<77:21:40,  6.34s/it]
 80%|████████  | 181090/225000 [343:20:58<68:09:09,  5.59s/it]
 80%|████████  | 181091/225000 [343:21:02<61:19:54,  5.03s/it]
 80%|████████  | 181092/225000 [343:21:10<71:23:55,  5.85s/it]
 80%|████████  | 181093/225000 [343:21:14<66:36:59,  5.46s/it]
 80%|████████  | 181094/225000 [343:21:19<62:56:40,  5.16s/it]
 80%|████████  | 181095/225000 [343:21:23<61:25:06,  5.04s/it]
 80%|████████  | 181096/225000 [343:21:28<61:35:21,  5.05s/it]
 80%|████████  | 181097/225000 [343:21:33<58:12:36,  4.77s/it]
 80%|████████  | 181098/225000 [343:21:37<57:08:27,  4.69s/it]
 80%|████████  | 181099/225000 [343:21:41<55:59:47,  4.59s/it]
 80%|████████  | 181100/225000 [343:21:52<77:45:19,  6.38s/it]
                                                              
{'loss': '0.7585', 'grad_norm': '1.192', 'learning_rate': '4.988e-06', 'epoch': '0.8049', 'num_input_tokens_seen': 569410848, 'train_runtime': '1.236e+06', 'train_tokens_per_second': '460.6'}

 80%|████████  | 181100/225000 [343:21:52<77:45:19,  6.38s/it]
 80%|████████  | 181101/225000 [343:21:58<76:38:12,  6.28s/it]
 80%|████████  | 181102/225000 [343:22:05<80:37:26,  6.61s/it]
 80%|████████  | 181103/225000 [343:22:10<72:18:38,  5.93s/it]
 80%|████████  | 181104/225000 [343:22:14<66:54:21,  5.49s/it]
 80%|████████  | 181105/225000 [343:22:19<64:34:38,  5.30s/it]
 80%|████████  | 181106/225000 [343:22:24<62:12:04,  5.10s/it]
 80%|████████  | 181107/225000 [343:22:28<60:35:14,  4.97s/it]
 80%|████████  | 181108/225000 [343:22:33<59:13:37,  4.86s/it]
 80%|████████  | 181109/225000 [343:22:38<58:51:20,  4.83s/it]
 80%|████████  | 181110/225000 [343:22:48<80:03:41,  6.57s/it]
 80%|████████  | 181111/225000 [343:22:54<76:14:44,  6.25s/it]
 80%|████████  | 181112/225000 [343:22:57<65:48:51,  5.40s/it]
 80%|████████  | 181113/225000 [343:23:02<62:28:07,  5.12s/it]
 80%|████████  | 181114/225000 [343:23:06<58:46:57,  4.82s/it]
 80%|████████  | 181115/225000 [343:23:10<57:12:03,  4.69s/it]
 80%|████████  | 181116/225000 [343:23:15<55:50:36,  4.58s/it]
 80%|████████  | 181117/225000 [343:23:19<56:27:20,  4.63s/it]
 80%|████████  | 181118/225000 [343:23:24<55:53:49,  4.59s/it]
 80%|████████  | 181119/225000 [343:23:35<78:20:59,  6.43s/it]
 80%|████████  | 181120/225000 [343:23:41<79:06:33,  6.49s/it]
 80%|████████  | 181121/225000 [343:23:46<71:07:13,  5.83s/it]
 80%|████████  | 181122/225000 [343:23:50<64:58:37,  5.33s/it]
 80%|████████  | 181123/225000 [343:23:54<60:50:19,  4.99s/it]
 80%|████████  | 181124/225000 [343:23:58<59:01:41,  4.84s/it]
 80%|████████  | 181125/225000 [343:24:03<59:29:29,  4.88s/it]
 81%|████████  | 181126/225000 [343:24:14<80:50:38,  6.63s/it]
 81%|████████  | 181127/225000 [343:24:18<69:54:49,  5.74s/it]
 81%|████████  | 181128/225000 [343:24:24<72:47:16,  5.97s/it]
 81%|████████  | 181129/225000 [343:24:27<62:30:03,  5.13s/it]
 81%|████████  | 181130/225000 [343:24:34<68:07:43,  5.59s/it]
 81%|████████  | 181131/225000 [343:24:39<66:28:44,  5.46s/it]
 81%|████████  | 181132/225000 [343:24:45<66:00:53,  5.42s/it]
 81%|████████  | 181133/225000 [343:24:50<64:52:48,  5.32s/it]
 81%|████████  | 181134/225000 [343:24:55<64:13:41,  5.27s/it]
 81%|████████  | 181135/225000 [343:25:00<62:17:22,  5.11s/it]
 81%|████████  | 181136/225000 [343:25:05<62:04:26,  5.09s/it]
 81%|████████  | 181137/225000 [343:25:10<63:01:25,  5.17s/it]
 81%|████████  | 181138/225000 [343:25:15<63:27:43,  5.21s/it]
 81%|████████  | 181139/225000 [343:25:20<62:09:24,  5.10s/it]
 81%|████████  | 181140/225000 [343:25:25<61:55:17,  5.08s/it]
 81%|████████  | 181141/225000 [343:25:30<61:54:42,  5.08s/it]
 81%|████████  | 181142/225000 [343:25:35<61:56:48,  5.08s/it]
 81%|████████  | 181143/225000 [343:25:41<63:44:49,  5.23s/it]
 81%|████████  | 181144/225000 [343:25:54<92:07:12,  7.56s/it]
 81%|████████  | 181145/225000 [343:25:58<78:32:21,  6.45s/it]
 81%|████████  | 181146/225000 [343:26:01<67:56:13,  5.58s/it]
 81%|████████  | 181147/225000 [343:26:05<62:57:58,  5.17s/it]
 81%|████████  | 181148/225000 [343:26:12<69:01:04,  5.67s/it]
 81%|████████  | 181149/225000 [343:26:20<77:26:49,  6.36s/it]
 81%|████████  | 181150/225000 [343:26:24<68:07:27,  5.59s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.387', 'learning_rate': '4.977e-06', 'epoch': '0.8051', 'num_input_tokens_seen': 569540376, 'train_runtime': '1.236e+06', 'train_tokens_per_second': '460.6'}

 81%|████████  | 181150/225000 [343:26:24<68:07:27,  5.59s/it]
 81%|████████  | 181151/225000 [343:26:28<60:41:11,  4.98s/it]
 81%|████████  | 181152/225000 [343:26:36<74:00:52,  6.08s/it]
 81%|████████  | 181153/225000 [343:26:40<66:27:41,  5.46s/it]
 81%|████████  | 181154/225000 [343:26:44<60:58:54,  5.01s/it]
 81%|████████  | 181155/225000 [343:26:49<60:56:51,  5.00s/it]
 81%|████████  | 181156/225000 [343:26:54<58:53:31,  4.84s/it]
 81%|████████  | 181157/225000 [343:26:59<60:51:57,  5.00s/it]
 81%|████████  | 181158/225000 [343:27:10<81:53:31,  6.72s/it]
 81%|████████  | 181159/225000 [343:27:13<69:40:06,  5.72s/it]
 81%|████████  | 181160/225000 [343:27:17<64:22:09,  5.29s/it]
 81%|████████  | 181161/225000 [343:27:25<71:41:44,  5.89s/it]
 81%|████████  | 181162/225000 [343:27:30<69:58:07,  5.75s/it]
 81%|████████  | 181163/225000 [343:27:35<66:32:07,  5.46s/it]
 81%|████████  | 181164/225000 [343:27:42<72:57:45,  5.99s/it]
 81%|████████  | 181165/225000 [343:27:46<65:42:03,  5.40s/it]
 81%|████████  | 181166/225000 [343:27:50<60:21:59,  4.96s/it]
 81%|████████  | 181167/225000 [343:27:54<56:32:14,  4.64s/it]
 81%|████████  | 181168/225000 [343:28:04<74:51:44,  6.15s/it]
 81%|████████  | 181169/225000 [343:28:08<69:02:06,  5.67s/it]
 81%|████████  | 181170/225000 [343:28:12<63:32:38,  5.22s/it]
 81%|████████  | 181171/225000 [343:28:17<59:46:12,  4.91s/it]
 81%|████████  | 181172/225000 [343:28:21<57:44:53,  4.74s/it]
 81%|████████  | 181173/225000 [343:28:26<57:03:31,  4.69s/it]
 81%|████████  | 181174/225000 [343:28:30<56:15:43,  4.62s/it]
 81%|████████  | 181175/225000 [343:28:42<82:47:03,  6.80s/it]
 81%|████████  | 181176/225000 [343:28:46<73:58:27,  6.08s/it]
 81%|████████  | 181177/225000 [343:28:52<72:37:51,  5.97s/it]
 81%|████████  | 181178/225000 [343:28:58<74:15:12,  6.10s/it]
 81%|████████  | 181179/225000 [343:29:02<65:29:17,  5.38s/it]
 81%|████████  | 181180/225000 [343:29:12<80:29:54,  6.61s/it]
 81%|████████  | 181181/225000 [343:29:15<68:33:54,  5.63s/it]
 81%|████████  | 181182/225000 [343:29:20<67:43:24,  5.56s/it]
 81%|████████  | 181183/225000 [343:29:26<66:57:39,  5.50s/it]
 81%|████████  | 181184/225000 [343:29:30<62:31:04,  5.14s/it]
 81%|████████  | 181185/225000 [343:29:34<60:05:55,  4.94s/it]
 81%|████████  | 181186/225000 [343:29:39<58:22:21,  4.80s/it]
 81%|████████  | 181187/225000 [343:29:44<59:14:17,  4.87s/it]
 81%|████████  | 181188/225000 [343:29:49<59:12:32,  4.87s/it]
 81%|████████  | 181189/225000 [343:29:53<58:10:12,  4.78s/it]
 81%|████████  | 181190/225000 [343:29:58<57:03:46,  4.69s/it]
 81%|████████  | 181191/225000 [343:30:03<58:48:06,  4.83s/it]
 81%|████████  | 181192/225000 [343:30:08<59:20:59,  4.88s/it]
 81%|████████  | 181193/225000 [343:30:13<58:43:17,  4.83s/it]
 81%|████████  | 181194/225000 [343:30:25<86:55:04,  7.14s/it]
 81%|████████  | 181195/225000 [343:30:29<75:21:11,  6.19s/it]
 81%|████████  | 181196/225000 [343:30:34<68:50:31,  5.66s/it]
 81%|████████  | 181197/225000 [343:30:38<62:37:36,  5.15s/it]
 81%|████████  | 181198/225000 [343:30:48<81:41:16,  6.71s/it]
 81%|████████  | 181199/225000 [343:30:53<76:53:53,  6.32s/it]
 81%|████████  | 181200/225000 [343:30:57<67:00:05,  5.51s/it]
                                                              
{'loss': '0.7696', 'grad_norm': '1.169', 'learning_rate': '4.966e-06', 'epoch': '0.8053', 'num_input_tokens_seen': 569670256, 'train_runtime': '1.237e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181200/225000 [343:30:57<67:00:05,  5.51s/it]
 81%|████████  | 181201/225000 [343:31:03<67:49:32,  5.57s/it]
 81%|████████  | 181202/225000 [343:31:11<78:48:04,  6.48s/it]
 81%|████████  | 181203/225000 [343:31:15<69:55:41,  5.75s/it]
 81%|████████  | 181204/225000 [343:31:19<63:37:52,  5.23s/it]
 81%|████████  | 181205/225000 [343:31:23<57:12:20,  4.70s/it]
 81%|████████  | 181206/225000 [343:31:32<75:08:32,  6.18s/it]
 81%|████████  | 181207/225000 [343:31:36<66:29:29,  5.47s/it]
 81%|████████  | 181208/225000 [343:31:40<61:47:56,  5.08s/it]
 81%|████████  | 181209/225000 [343:31:44<57:30:52,  4.73s/it]
 81%|████████  | 181210/225000 [343:31:52<67:36:18,  5.56s/it]
 81%|████████  | 181211/225000 [343:32:00<75:56:35,  6.24s/it]
 81%|████████  | 181212/225000 [343:32:05<73:29:29,  6.04s/it]
 81%|████████  | 181213/225000 [343:32:10<70:07:38,  5.77s/it]
 81%|████████  | 181214/225000 [343:32:15<67:15:22,  5.53s/it]
 81%|████████  | 181215/225000 [343:32:21<67:08:26,  5.52s/it]
 81%|████████  | 181216/225000 [343:32:26<66:53:56,  5.50s/it]
 81%|████████  | 181217/225000 [343:32:32<66:36:06,  5.48s/it]
 81%|████████  | 181218/225000 [343:32:37<65:41:04,  5.40s/it]
 81%|████████  | 181219/225000 [343:32:42<65:39:15,  5.40s/it]
 81%|████████  | 181220/225000 [343:32:48<65:02:35,  5.35s/it]
 81%|████████  | 181221/225000 [343:32:53<64:00:02,  5.26s/it]
 81%|████████  | 181222/225000 [343:32:58<63:37:58,  5.23s/it]
 81%|████████  | 181223/225000 [343:33:03<64:15:39,  5.28s/it]
 81%|████████  | 181224/225000 [343:33:08<63:36:50,  5.23s/it]
 81%|████████  | 181225/225000 [343:33:14<65:04:42,  5.35s/it]
 81%|████████  | 181226/225000 [343:33:19<65:33:23,  5.39s/it]
 81%|████████  | 181227/225000 [343:33:25<65:25:50,  5.38s/it]
 81%|████████  | 181228/225000 [343:33:30<64:24:42,  5.30s/it]
 81%|████████  | 181229/225000 [343:33:35<64:50:05,  5.33s/it]
 81%|████████  | 181230/225000 [343:33:41<65:02:40,  5.35s/it]
 81%|████████  | 181231/225000 [343:33:46<65:27:03,  5.38s/it]
 81%|████████  | 181232/225000 [343:33:51<65:02:02,  5.35s/it]
 81%|████████  | 181233/225000 [343:33:57<65:31:46,  5.39s/it]
 81%|████████  | 181234/225000 [343:34:02<65:20:14,  5.37s/it]
 81%|████████  | 181235/225000 [343:34:07<64:01:47,  5.27s/it]
 81%|████████  | 181236/225000 [343:34:12<62:33:29,  5.15s/it]
 81%|████████  | 181237/225000 [343:34:17<61:34:52,  5.07s/it]
 81%|████████  | 181238/225000 [343:34:22<61:26:00,  5.05s/it]
 81%|████████  | 181239/225000 [343:34:28<63:26:15,  5.22s/it]
 81%|████████  | 181240/225000 [343:34:33<63:27:27,  5.22s/it]
 81%|████████  | 181241/225000 [343:34:38<62:13:52,  5.12s/it]
 81%|████████  | 181242/225000 [343:34:43<62:26:32,  5.14s/it]
 81%|████████  | 181243/225000 [343:34:50<68:51:01,  5.66s/it]
 81%|████████  | 181244/225000 [343:34:59<80:02:37,  6.59s/it]
 81%|████████  | 181245/225000 [343:35:02<69:42:10,  5.73s/it]
 81%|████████  | 181246/225000 [343:35:11<81:33:42,  6.71s/it]
 81%|████████  | 181247/225000 [343:35:15<70:31:41,  5.80s/it]
 81%|████████  | 181248/225000 [343:35:18<61:16:27,  5.04s/it]
 81%|████████  | 181249/225000 [343:35:28<80:00:11,  6.58s/it]
 81%|████████  | 181250/225000 [343:35:32<70:09:30,  5.77s/it]
                                                              
{'loss': '0.7491', 'grad_norm': '1.211', 'learning_rate': '4.955e-06', 'epoch': '0.8056', 'num_input_tokens_seen': 569800392, 'train_runtime': '1.237e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181250/225000 [343:35:32<70:09:30,  5.77s/it]
 81%|████████  | 181251/225000 [343:35:38<69:58:57,  5.76s/it]
 81%|████████  | 181252/225000 [343:35:43<68:27:29,  5.63s/it]
 81%|████████  | 181253/225000 [343:35:49<68:24:29,  5.63s/it]
 81%|████████  | 181254/225000 [343:35:55<71:18:42,  5.87s/it]
 81%|████████  | 181255/225000 [343:36:00<66:56:11,  5.51s/it]
 81%|████████  | 181256/225000 [343:36:05<64:29:26,  5.31s/it]
 81%|████████  | 181257/225000 [343:36:10<63:18:06,  5.21s/it]
 81%|████████  | 181258/225000 [343:36:15<62:04:39,  5.11s/it]
 81%|████████  | 181259/225000 [343:36:20<61:01:34,  5.02s/it]
 81%|████████  | 181260/225000 [343:36:25<61:34:59,  5.07s/it]
 81%|████████  | 181261/225000 [343:36:30<60:57:59,  5.02s/it]
 81%|████████  | 181262/225000 [343:36:34<60:11:09,  4.95s/it]
 81%|████████  | 181263/225000 [343:36:40<60:38:45,  4.99s/it]
 81%|████████  | 181264/225000 [343:36:44<59:49:42,  4.92s/it]
 81%|████████  | 181265/225000 [343:36:49<59:14:22,  4.88s/it]
 81%|████████  | 181266/225000 [343:36:54<60:13:48,  4.96s/it]
 81%|████████  | 181267/225000 [343:36:59<59:54:26,  4.93s/it]
 81%|████████  | 181268/225000 [343:37:08<73:53:50,  6.08s/it]
 81%|████████  | 181269/225000 [343:37:18<88:26:19,  7.28s/it]
 81%|████████  | 181270/225000 [343:37:21<74:26:39,  6.13s/it]
 81%|████████  | 181271/225000 [343:37:26<69:45:29,  5.74s/it]
 81%|████████  | 181272/225000 [343:37:33<71:51:15,  5.92s/it]
 81%|████████  | 181273/225000 [343:37:36<63:57:50,  5.27s/it]
 81%|████████  | 181274/225000 [343:37:40<57:53:44,  4.77s/it]
 81%|████████  | 181275/225000 [343:37:46<61:46:13,  5.09s/it]
 81%|████████  | 181276/225000 [343:37:54<71:55:02,  5.92s/it]
 81%|████████  | 181277/225000 [343:38:00<73:18:26,  6.04s/it]
 81%|████████  | 181278/225000 [343:38:04<66:07:51,  5.45s/it]
 81%|████████  | 181279/225000 [343:38:08<59:21:55,  4.89s/it]
 81%|████████  | 181280/225000 [343:38:17<75:11:09,  6.19s/it]
 81%|████████  | 181281/225000 [343:38:21<69:34:06,  5.73s/it]
 81%|████████  | 181282/225000 [343:38:25<62:07:58,  5.12s/it]
 81%|████████  | 181283/225000 [343:38:33<72:57:27,  6.01s/it]
 81%|████████  | 181284/225000 [343:38:38<69:10:39,  5.70s/it]
 81%|████████  | 181285/225000 [343:38:43<66:31:42,  5.48s/it]
 81%|████████  | 181286/225000 [343:38:48<64:58:43,  5.35s/it]
 81%|████████  | 181287/225000 [343:38:53<63:06:25,  5.20s/it]
 81%|████████  | 181288/225000 [343:38:58<60:28:40,  4.98s/it]
 81%|████████  | 181289/225000 [343:39:02<59:47:55,  4.92s/it]
 81%|████████  | 181290/225000 [343:39:07<59:16:56,  4.88s/it]
 81%|████████  | 181291/225000 [343:39:12<59:21:10,  4.89s/it]
 81%|████████  | 181292/225000 [343:39:17<60:04:01,  4.95s/it]
 81%|████████  | 181293/225000 [343:39:23<61:42:47,  5.08s/it]
 81%|████████  | 181294/225000 [343:39:31<75:51:00,  6.25s/it]
 81%|████████  | 181295/225000 [343:39:40<84:13:40,  6.94s/it]
 81%|████████  | 181296/225000 [343:39:44<75:02:21,  6.18s/it]
 81%|████████  | 181297/225000 [343:39:49<68:47:26,  5.67s/it]
 81%|████████  | 181298/225000 [343:39:53<63:08:07,  5.20s/it]
 81%|████████  | 181299/225000 [343:39:57<59:50:54,  4.93s/it]
 81%|████████  | 181300/225000 [343:40:08<80:47:31,  6.66s/it]
                                                              
{'loss': '0.7615', 'grad_norm': '1.216', 'learning_rate': '4.944e-06', 'epoch': '0.8058', 'num_input_tokens_seen': 569930576, 'train_runtime': '1.237e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181300/225000 [343:40:08<80:47:31,  6.66s/it]
 81%|████████  | 181301/225000 [343:40:13<73:22:43,  6.05s/it]
 81%|████████  | 181302/225000 [343:40:17<68:45:00,  5.66s/it]
 81%|████████  | 181303/225000 [343:40:22<66:15:47,  5.46s/it]
 81%|████████  | 181304/225000 [343:40:27<63:29:58,  5.23s/it]
 81%|████████  | 181305/225000 [343:40:32<61:40:01,  5.08s/it]
 81%|████████  | 181306/225000 [343:40:36<59:41:46,  4.92s/it]
 81%|████████  | 181307/225000 [343:40:41<58:19:37,  4.81s/it]
 81%|████████  | 181308/225000 [343:40:46<61:08:29,  5.04s/it]
 81%|████████  | 181309/225000 [343:40:59<89:31:52,  7.38s/it]
 81%|████████  | 181310/225000 [343:41:03<75:14:20,  6.20s/it]
 81%|████████  | 181311/225000 [343:41:06<65:01:29,  5.36s/it]
 81%|████████  | 181312/225000 [343:41:13<70:55:27,  5.84s/it]
 81%|████████  | 181313/225000 [343:41:19<71:03:37,  5.86s/it]
 81%|████████  | 181314/225000 [343:41:24<68:23:34,  5.64s/it]
 81%|████████  | 181315/225000 [343:41:31<71:32:53,  5.90s/it]
 81%|████████  | 181316/225000 [343:41:35<65:14:16,  5.38s/it]
 81%|████████  | 181317/225000 [343:41:38<58:29:11,  4.82s/it]
 81%|████████  | 181318/225000 [343:41:47<73:26:49,  6.05s/it]
 81%|████████  | 181319/225000 [343:41:51<64:24:45,  5.31s/it]
 81%|████████  | 181320/225000 [343:41:57<69:21:50,  5.72s/it]
 81%|████████  | 181321/225000 [343:42:02<63:35:58,  5.24s/it]
 81%|████████  | 181322/225000 [343:42:10<73:11:37,  6.03s/it]
 81%|████████  | 181323/225000 [343:42:13<63:49:06,  5.26s/it]
 81%|████████  | 181324/225000 [343:42:16<56:52:49,  4.69s/it]
 81%|████████  | 181325/225000 [343:42:25<70:08:57,  5.78s/it]
 81%|████████  | 181326/225000 [343:42:30<68:35:07,  5.65s/it]
 81%|████████  | 181327/225000 [343:42:34<62:23:31,  5.14s/it]
 81%|████████  | 181328/225000 [343:42:38<57:09:48,  4.71s/it]
 81%|████████  | 181329/225000 [343:42:46<71:22:42,  5.88s/it]
 81%|████████  | 181330/225000 [343:42:51<65:50:15,  5.43s/it]
 81%|████████  | 181331/225000 [343:42:55<60:30:45,  4.99s/it]
 81%|████████  | 181332/225000 [343:43:04<77:58:49,  6.43s/it]
 81%|████████  | 181333/225000 [343:43:10<74:20:35,  6.13s/it]
 81%|████████  | 181334/225000 [343:43:15<70:36:56,  5.82s/it]
 81%|████████  | 181335/225000 [343:43:20<66:53:34,  5.52s/it]
 81%|████████  | 181336/225000 [343:43:25<66:34:40,  5.49s/it]
 81%|████████  | 181337/225000 [343:43:31<66:08:23,  5.45s/it]
 81%|████████  | 181338/225000 [343:43:36<66:24:12,  5.48s/it]
 81%|████████  | 181339/225000 [343:43:42<68:13:48,  5.63s/it]
 81%|████████  | 181340/225000 [343:43:48<69:35:38,  5.74s/it]
 81%|████████  | 181341/225000 [343:43:52<62:46:15,  5.18s/it]
 81%|████████  | 181342/225000 [343:44:01<75:43:05,  6.24s/it]
 81%|████████  | 181343/225000 [343:44:05<70:15:41,  5.79s/it]
 81%|████████  | 181344/225000 [343:44:10<64:51:07,  5.35s/it]
 81%|████████  | 181345/225000 [343:44:14<61:28:27,  5.07s/it]
 81%|████████  | 181346/225000 [343:44:19<59:27:33,  4.90s/it]
 81%|████████  | 181347/225000 [343:44:23<57:03:09,  4.71s/it]
 81%|████████  | 181348/225000 [343:44:27<56:17:21,  4.64s/it]
 81%|████████  | 181349/225000 [343:44:32<56:04:34,  4.62s/it]
 81%|████████  | 181350/225000 [343:44:42<74:34:54,  6.15s/it]
                                                              
{'loss': '0.7595', 'grad_norm': '1.043', 'learning_rate': '4.934e-06', 'epoch': '0.806', 'num_input_tokens_seen': 570060216, 'train_runtime': '1.237e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181350/225000 [343:44:42<74:34:54,  6.15s/it]
 81%|████████  | 181351/225000 [343:44:49<80:27:11,  6.64s/it]
 81%|████████  | 181352/225000 [343:44:53<68:23:32,  5.64s/it]
 81%|████████  | 181353/225000 [343:44:57<62:49:40,  5.18s/it]
 81%|████████  | 181354/225000 [343:45:06<78:40:26,  6.49s/it]
 81%|████████  | 181355/225000 [343:45:10<67:49:22,  5.59s/it]
 81%|████████  | 181356/225000 [343:45:18<75:35:10,  6.23s/it]
 81%|████████  | 181357/225000 [343:45:21<64:58:00,  5.36s/it]
 81%|████████  | 181358/225000 [343:45:29<75:16:52,  6.21s/it]
 81%|████████  | 181359/225000 [343:45:33<65:14:44,  5.38s/it]
 81%|████████  | 181360/225000 [343:45:37<60:03:18,  4.95s/it]
 81%|████████  | 181361/225000 [343:45:48<84:08:05,  6.94s/it]
 81%|████████  | 181362/225000 [343:45:52<73:28:32,  6.06s/it]
 81%|████████  | 181363/225000 [343:45:56<66:34:38,  5.49s/it]
 81%|████████  | 181364/225000 [343:46:00<61:31:55,  5.08s/it]
 81%|████████  | 181365/225000 [343:46:10<77:37:17,  6.40s/it]
 81%|████████  | 181366/225000 [343:46:14<69:51:46,  5.76s/it]
 81%|████████  | 181367/225000 [343:46:18<63:42:49,  5.26s/it]
 81%|████████  | 181368/225000 [343:46:23<60:50:57,  5.02s/it]
 81%|████████  | 181369/225000 [343:46:27<58:35:06,  4.83s/it]
 81%|████████  | 181370/225000 [343:46:31<55:41:59,  4.60s/it]
 81%|████████  | 181371/225000 [343:46:42<78:54:29,  6.51s/it]
 81%|████████  | 181372/225000 [343:46:46<68:19:36,  5.64s/it]
 81%|████████  | 181373/225000 [343:46:50<61:46:52,  5.10s/it]
 81%|████████  | 181374/225000 [343:47:00<79:34:28,  6.57s/it]
 81%|████████  | 181375/225000 [343:47:03<69:49:55,  5.76s/it]
 81%|████████  | 181376/225000 [343:47:07<62:26:41,  5.15s/it]
 81%|████████  | 181377/225000 [343:47:11<57:54:57,  4.78s/it]
 81%|████████  | 181378/225000 [343:47:21<76:21:42,  6.30s/it]
 81%|████████  | 181379/225000 [343:47:24<65:57:02,  5.44s/it]
 81%|████████  | 181380/225000 [343:47:28<60:14:43,  4.97s/it]
 81%|████████  | 181381/225000 [343:47:37<75:07:41,  6.20s/it]
 81%|████████  | 181382/225000 [343:47:42<69:57:55,  5.77s/it]
 81%|████████  | 181383/225000 [343:47:47<65:20:16,  5.39s/it]
 81%|████████  | 181384/225000 [343:47:51<63:06:23,  5.21s/it]
 81%|████████  | 181385/225000 [343:47:56<62:20:51,  5.15s/it]
 81%|████████  | 181386/225000 [343:48:01<60:55:15,  5.03s/it]
 81%|████████  | 181387/225000 [343:48:06<59:25:59,  4.91s/it]
 81%|████████  | 181388/225000 [343:48:11<59:53:13,  4.94s/it]
 81%|████████  | 181389/225000 [343:48:16<59:27:13,  4.91s/it]
 81%|████████  | 181390/225000 [343:48:21<60:26:31,  4.99s/it]
 81%|████████  | 181391/225000 [343:48:26<59:33:44,  4.92s/it]
 81%|████████  | 181392/225000 [343:48:32<65:27:59,  5.40s/it]
 81%|████████  | 181393/225000 [343:48:43<86:04:36,  7.11s/it]
 81%|████████  | 181394/225000 [343:48:48<79:27:21,  6.56s/it]
 81%|████████  | 181395/225000 [343:48:53<71:31:29,  5.91s/it]
 81%|████████  | 181396/225000 [343:48:56<63:22:33,  5.23s/it]
 81%|████████  | 181397/225000 [343:49:05<75:54:07,  6.27s/it]
 81%|████████  | 181398/225000 [343:49:09<65:20:14,  5.39s/it]
 81%|████████  | 181399/225000 [343:49:16<73:07:51,  6.04s/it]
 81%|████████  | 181400/225000 [343:49:21<69:03:25,  5.70s/it]
                                                              
{'loss': '0.7622', 'grad_norm': '1.139', 'learning_rate': '4.923e-06', 'epoch': '0.8062', 'num_input_tokens_seen': 570190680, 'train_runtime': '1.238e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181400/225000 [343:49:21<69:03:25,  5.70s/it]
 81%|████████  | 181401/225000 [343:49:25<64:05:12,  5.29s/it]
 81%|████████  | 181402/225000 [343:49:29<59:46:12,  4.94s/it]
 81%|████████  | 181403/225000 [343:49:33<56:33:30,  4.67s/it]
 81%|████████  | 181404/225000 [343:49:38<55:40:48,  4.60s/it]
 81%|████████  | 181405/225000 [343:49:42<55:38:03,  4.59s/it]
 81%|████████  | 181406/225000 [343:49:47<55:39:39,  4.60s/it]
 81%|████████  | 181407/225000 [343:49:59<81:35:35,  6.74s/it]
 81%|████████  | 181408/225000 [343:50:03<71:20:14,  5.89s/it]
 81%|████████  | 181409/225000 [343:50:06<62:43:39,  5.18s/it]
 81%|████████  | 181410/225000 [343:50:10<58:54:52,  4.87s/it]
 81%|████████  | 181411/225000 [343:50:21<78:36:51,  6.49s/it]
 81%|████████  | 181412/225000 [343:50:24<68:52:04,  5.69s/it]
 81%|████████  | 181413/225000 [343:50:28<60:00:41,  4.96s/it]
 81%|████████  | 181414/225000 [343:50:32<57:40:21,  4.76s/it]
 81%|████████  | 181415/225000 [343:50:38<60:42:36,  5.01s/it]
 81%|████████  | 181416/225000 [343:50:43<61:57:12,  5.12s/it]
 81%|████████  | 181417/225000 [343:50:48<62:11:31,  5.14s/it]
 81%|████████  | 181418/225000 [343:50:53<61:01:47,  5.04s/it]
 81%|████████  | 181419/225000 [343:50:58<62:27:17,  5.16s/it]
 81%|████████  | 181420/225000 [343:51:04<64:26:08,  5.32s/it]
 81%|████████  | 181421/225000 [343:51:09<63:40:08,  5.26s/it]
 81%|████████  | 181422/225000 [343:51:15<63:47:39,  5.27s/it]
 81%|████████  | 181423/225000 [343:51:20<64:13:38,  5.31s/it]
 81%|████████  | 181424/225000 [343:51:25<63:28:02,  5.24s/it]
 81%|████████  | 181425/225000 [343:51:30<64:07:47,  5.30s/it]
 81%|████████  | 181426/225000 [343:51:36<64:43:05,  5.35s/it]
 81%|████████  | 181427/225000 [343:51:41<64:38:53,  5.34s/it]
 81%|████████  | 181428/225000 [343:51:47<65:40:44,  5.43s/it]
 81%|████████  | 181429/225000 [343:51:52<65:20:03,  5.40s/it]
 81%|████████  | 181430/225000 [343:51:58<65:41:22,  5.43s/it]
 81%|████████  | 181431/225000 [343:52:03<64:01:12,  5.29s/it]
 81%|████████  | 181432/225000 [343:52:11<73:59:00,  6.11s/it]
 81%|████████  | 181433/225000 [343:52:16<71:11:17,  5.88s/it]
 81%|████████  | 181434/225000 [343:52:21<67:15:19,  5.56s/it]
 81%|████████  | 181435/225000 [343:52:26<66:04:22,  5.46s/it]
 81%|████████  | 181436/225000 [343:52:32<66:32:45,  5.50s/it]
 81%|████████  | 181437/225000 [343:52:37<66:07:38,  5.46s/it]
 81%|████████  | 181438/225000 [343:52:45<75:20:47,  6.23s/it]
 81%|████████  | 181439/225000 [343:52:50<68:57:53,  5.70s/it]
 81%|████████  | 181440/225000 [343:52:55<67:42:57,  5.60s/it]
 81%|████████  | 181441/225000 [343:53:00<65:49:26,  5.44s/it]
 81%|████████  | 181442/225000 [343:53:05<65:15:55,  5.39s/it]
 81%|████████  | 181443/225000 [343:53:11<64:54:39,  5.36s/it]
 81%|████████  | 181444/225000 [343:53:16<65:36:29,  5.42s/it]
 81%|████████  | 181445/225000 [343:53:21<63:33:12,  5.25s/it]
 81%|████████  | 181446/225000 [343:53:26<63:11:24,  5.22s/it]
 81%|████████  | 181447/225000 [343:53:31<63:01:48,  5.21s/it]
 81%|████████  | 181448/225000 [343:53:37<63:06:34,  5.22s/it]
 81%|████████  | 181449/225000 [343:53:42<63:55:57,  5.28s/it]
 81%|████████  | 181450/225000 [343:53:47<64:17:13,  5.31s/it]
                                                              
{'loss': '0.775', 'grad_norm': '1.164', 'learning_rate': '4.912e-06', 'epoch': '0.8064', 'num_input_tokens_seen': 570320048, 'train_runtime': '1.238e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181450/225000 [343:53:47<64:17:13,  5.31s/it]
 81%|████████  | 181451/225000 [343:53:52<63:37:54,  5.26s/it]
 81%|████████  | 181452/225000 [343:53:58<64:55:42,  5.37s/it]
 81%|████████  | 181453/225000 [343:54:03<64:03:04,  5.30s/it]
 81%|████████  | 181454/225000 [343:54:09<65:23:06,  5.41s/it]
 81%|████████  | 181455/225000 [343:54:13<61:38:04,  5.10s/it]
 81%|████████  | 181456/225000 [343:54:19<62:37:35,  5.18s/it]
 81%|████████  | 181457/225000 [343:54:24<63:39:47,  5.26s/it]
 81%|████████  | 181458/225000 [343:54:29<63:51:18,  5.28s/it]
 81%|████████  | 181459/225000 [343:54:35<63:49:58,  5.28s/it]
 81%|████████  | 181460/225000 [343:54:40<64:10:14,  5.31s/it]
 81%|████████  | 181461/225000 [343:54:45<64:32:24,  5.34s/it]
 81%|████████  | 181462/225000 [343:54:51<64:17:12,  5.32s/it]
 81%|████████  | 181463/225000 [343:54:56<64:16:00,  5.31s/it]
 81%|████████  | 181464/225000 [343:55:02<65:11:15,  5.39s/it]
 81%|████████  | 181465/225000 [343:55:07<66:21:34,  5.49s/it]
 81%|████████  | 181466/225000 [343:55:12<64:38:33,  5.35s/it]
 81%|████████  | 181467/225000 [343:55:18<65:37:44,  5.43s/it]
 81%|████████  | 181468/225000 [343:55:29<87:20:51,  7.22s/it]
 81%|████████  | 181469/225000 [343:55:33<75:58:14,  6.28s/it]
 81%|████████  | 181470/225000 [343:55:38<68:42:08,  5.68s/it]
 81%|████████  | 181471/225000 [343:55:42<62:25:26,  5.16s/it]
 81%|████████  | 181472/225000 [343:55:46<60:02:47,  4.97s/it]
 81%|████████  | 181473/225000 [343:55:57<79:26:46,  6.57s/it]
 81%|████████  | 181474/225000 [343:56:02<75:20:25,  6.23s/it]
 81%|████████  | 181475/225000 [343:56:08<73:06:49,  6.05s/it]
 81%|████████  | 181476/225000 [343:56:13<71:42:59,  5.93s/it]
 81%|████████  | 181477/225000 [343:56:17<65:02:24,  5.38s/it]
 81%|████████  | 181478/225000 [343:56:21<60:30:31,  5.01s/it]
 81%|████████  | 181479/225000 [343:56:32<80:38:38,  6.67s/it]
 81%|████████  | 181480/225000 [343:56:35<68:55:53,  5.70s/it]
 81%|████████  | 181481/225000 [343:56:42<73:19:41,  6.07s/it]
 81%|████████  | 181482/225000 [343:56:46<64:40:23,  5.35s/it]
 81%|████████  | 181483/225000 [343:56:50<58:25:10,  4.83s/it]
 81%|████████  | 181484/225000 [343:56:58<72:46:07,  6.02s/it]
 81%|████████  | 181485/225000 [343:57:04<69:10:23,  5.72s/it]
 81%|████████  | 181486/225000 [343:57:07<59:51:12,  4.95s/it]
 81%|████████  | 181487/225000 [343:57:12<62:52:28,  5.20s/it]
 81%|████████  | 181488/225000 [343:57:20<71:17:39,  5.90s/it]
 81%|████████  | 181489/225000 [343:57:23<62:04:52,  5.14s/it]
 81%|████████  | 181490/225000 [343:57:32<74:39:24,  6.18s/it]
 81%|████████  | 181491/225000 [343:57:35<64:45:56,  5.36s/it]
 81%|████████  | 181492/225000 [343:57:40<60:36:25,  5.01s/it]
 81%|████████  | 181493/225000 [343:57:49<75:36:10,  6.26s/it]
 81%|████████  | 181494/225000 [343:57:52<65:37:30,  5.43s/it]
 81%|████████  | 181495/225000 [343:57:57<61:22:55,  5.08s/it]
 81%|████████  | 181496/225000 [343:58:01<58:57:07,  4.88s/it]
 81%|████████  | 181497/225000 [343:58:05<56:34:32,  4.68s/it]
 81%|████████  | 181498/225000 [343:58:12<62:51:44,  5.20s/it]
 81%|████████  | 181499/225000 [343:58:20<75:39:58,  6.26s/it]
 81%|████████  | 181500/225000 [343:58:24<67:04:13,  5.55s/it]
                                                              
{'loss': '0.7547', 'grad_norm': '1.063', 'learning_rate': '4.901e-06', 'epoch': '0.8067', 'num_input_tokens_seen': 570449416, 'train_runtime': '1.238e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181500/225000 [343:58:24<67:04:13,  5.55s/it]
 81%|████████  | 181501/225000 [343:58:28<59:23:03,  4.91s/it]
 81%|████████  | 181502/225000 [343:58:36<73:28:18,  6.08s/it]
 81%|████████  | 181503/225000 [343:58:41<69:28:08,  5.75s/it]
 81%|████████  | 181504/225000 [343:58:48<73:19:20,  6.07s/it]
 81%|████████  | 181505/225000 [343:58:54<72:14:35,  5.98s/it]
 81%|████████  | 181506/225000 [343:58:58<63:46:23,  5.28s/it]
 81%|████████  | 181507/225000 [343:59:01<57:23:03,  4.75s/it]
 81%|████████  | 181508/225000 [343:59:11<74:07:26,  6.14s/it]
 81%|████████  | 181509/225000 [343:59:14<65:09:45,  5.39s/it]
 81%|████████  | 181510/225000 [343:59:18<58:20:22,  4.83s/it]
 81%|████████  | 181511/225000 [343:59:28<77:18:28,  6.40s/it]
 81%|████████  | 181512/225000 [343:59:31<65:38:30,  5.43s/it]
 81%|████████  | 181513/225000 [343:59:36<62:33:09,  5.18s/it]
 81%|████████  | 181514/225000 [343:59:43<71:49:25,  5.95s/it]
 81%|████████  | 181515/225000 [343:59:47<62:21:42,  5.16s/it]
 81%|████████  | 181516/225000 [343:59:51<60:17:42,  4.99s/it]
 81%|████████  | 181517/225000 [343:59:55<56:11:20,  4.65s/it]
 81%|████████  | 181518/225000 [344:00:00<55:52:10,  4.63s/it]
 81%|████████  | 181519/225000 [344:00:04<56:03:07,  4.64s/it]
 81%|████████  | 181520/225000 [344:00:08<53:44:46,  4.45s/it]
 81%|████████  | 181521/225000 [344:00:13<55:49:26,  4.62s/it]
 81%|████████  | 181522/225000 [344:00:20<64:23:55,  5.33s/it]
 81%|████████  | 181523/225000 [344:00:30<80:33:06,  6.67s/it]
 81%|████████  | 181524/225000 [344:00:34<68:52:21,  5.70s/it]
 81%|████████  | 181525/225000 [344:00:39<69:46:21,  5.78s/it]
 81%|████████  | 181526/225000 [344:00:46<71:15:16,  5.90s/it]
 81%|████████  | 181527/225000 [344:00:53<76:28:59,  6.33s/it]
 81%|████████  | 181528/225000 [344:00:59<75:53:25,  6.28s/it]
 81%|████████  | 181529/225000 [344:01:03<66:38:53,  5.52s/it]
 81%|████████  | 181530/225000 [344:01:11<74:15:23,  6.15s/it]
 81%|████████  | 181531/225000 [344:01:14<63:01:43,  5.22s/it]
 81%|████████  | 181532/225000 [344:01:19<63:25:46,  5.25s/it]
 81%|████████  | 181533/225000 [344:01:25<65:58:37,  5.46s/it]
 81%|████████  | 181534/225000 [344:01:29<60:57:29,  5.05s/it]
 81%|████████  | 181535/225000 [344:01:38<76:43:15,  6.35s/it]
 81%|████████  | 181536/225000 [344:01:42<66:19:20,  5.49s/it]
 81%|████████  | 181537/225000 [344:01:46<61:48:25,  5.12s/it]
 81%|████████  | 181538/225000 [344:01:50<58:10:20,  4.82s/it]
 81%|████████  | 181539/225000 [344:01:55<58:53:56,  4.88s/it]
 81%|████████  | 181540/225000 [344:02:00<59:42:34,  4.95s/it]
 81%|████████  | 181541/225000 [344:02:05<59:44:55,  4.95s/it]
 81%|████████  | 181542/225000 [344:02:12<66:15:20,  5.49s/it]
 81%|████████  | 181543/225000 [344:02:23<87:42:05,  7.27s/it]
 81%|████████  | 181544/225000 [344:02:26<71:45:13,  5.94s/it]
 81%|████████  | 181545/225000 [344:02:30<65:23:04,  5.42s/it]
 81%|████████  | 181546/225000 [344:02:38<73:49:07,  6.12s/it]
 81%|████████  | 181547/225000 [344:02:42<64:35:22,  5.35s/it]
 81%|████████  | 181548/225000 [344:02:47<63:59:49,  5.30s/it]
 81%|████████  | 181549/225000 [344:02:53<64:58:35,  5.38s/it]
 81%|████████  | 181550/225000 [344:02:58<65:41:23,  5.44s/it]
                                                              
{'loss': '0.7671', 'grad_norm': '1.092', 'learning_rate': '4.89e-06', 'epoch': '0.8069', 'num_input_tokens_seen': 570579936, 'train_runtime': '1.239e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181550/225000 [344:02:58<65:41:23,  5.44s/it]
 81%|████████  | 181551/225000 [344:03:04<66:44:13,  5.53s/it]
 81%|████████  | 181552/225000 [344:03:12<75:14:03,  6.23s/it]
 81%|████████  | 181553/225000 [344:03:15<65:12:27,  5.40s/it]
 81%|████████  | 181554/225000 [344:03:24<77:32:52,  6.43s/it]
 81%|████████  | 181555/225000 [344:03:29<72:30:47,  6.01s/it]
 81%|████████  | 181556/225000 [344:03:34<68:10:25,  5.65s/it]
 81%|████████  | 181557/225000 [344:03:39<66:46:17,  5.53s/it]
 81%|████████  | 181558/225000 [344:03:44<64:50:41,  5.37s/it]
 81%|████████  | 181559/225000 [344:03:48<60:54:51,  5.05s/it]
 81%|████████  | 181560/225000 [344:03:54<61:04:35,  5.06s/it]
 81%|████████  | 181561/225000 [344:03:59<60:55:58,  5.05s/it]
 81%|████████  | 181562/225000 [344:04:04<61:40:31,  5.11s/it]
 81%|████████  | 181563/225000 [344:04:09<60:53:09,  5.05s/it]
 81%|████████  | 181564/225000 [344:04:13<59:40:30,  4.95s/it]
 81%|████████  | 181565/225000 [344:04:19<60:59:27,  5.06s/it]
 81%|████████  | 181566/225000 [344:04:24<60:39:33,  5.03s/it]
 81%|████████  | 181567/225000 [344:04:29<60:10:51,  4.99s/it]
 81%|████████  | 181568/225000 [344:04:34<62:07:02,  5.15s/it]
 81%|████████  | 181569/225000 [344:04:39<61:28:10,  5.10s/it]
 81%|████████  | 181570/225000 [344:04:44<61:13:45,  5.08s/it]
 81%|████████  | 181571/225000 [344:04:49<59:47:25,  4.96s/it]
 81%|████████  | 181572/225000 [344:04:54<61:06:18,  5.07s/it]
 81%|████████  | 181573/225000 [344:05:06<87:06:06,  7.22s/it]
 81%|████████  | 181574/225000 [344:05:11<76:45:07,  6.36s/it]
 81%|████████  | 181575/225000 [344:05:15<70:51:15,  5.87s/it]
 81%|████████  | 181576/225000 [344:05:22<73:28:57,  6.09s/it]
 81%|████████  | 181577/225000 [344:05:28<73:50:56,  6.12s/it]
 81%|████████  | 181578/225000 [344:05:34<71:37:41,  5.94s/it]
 81%|████████  | 181579/225000 [344:05:40<73:41:05,  6.11s/it]
 81%|████████  | 181580/225000 [344:05:44<64:54:59,  5.38s/it]
 81%|████████  | 181581/225000 [344:05:47<57:41:26,  4.78s/it]
 81%|████████  | 181582/225000 [344:05:55<67:57:54,  5.64s/it]
 81%|████████  | 181583/225000 [344:06:00<65:40:03,  5.44s/it]
 81%|████████  | 181584/225000 [344:06:05<63:36:41,  5.27s/it]
 81%|████████  | 181585/225000 [344:06:09<60:17:52,  5.00s/it]
 81%|████████  | 181586/225000 [344:06:14<60:25:51,  5.01s/it]
 81%|████████  | 181587/225000 [344:06:19<60:16:31,  5.00s/it]
 81%|████████  | 181588/225000 [344:06:24<61:11:12,  5.07s/it]
 81%|████████  | 181589/225000 [344:06:29<60:26:18,  5.01s/it]
 81%|████████  | 181590/225000 [344:06:35<61:24:32,  5.09s/it]
 81%|████████  | 181591/225000 [344:06:39<60:17:56,  5.00s/it]
 81%|████████  | 181592/225000 [344:06:51<84:45:23,  7.03s/it]
 81%|████████  | 181593/225000 [344:06:55<73:44:41,  6.12s/it]
 81%|████████  | 181594/225000 [344:06:59<66:36:47,  5.52s/it]
 81%|████████  | 181595/225000 [344:07:03<60:40:59,  5.03s/it]
 81%|████████  | 181596/225000 [344:07:14<81:19:07,  6.74s/it]
 81%|████████  | 181597/225000 [344:07:17<69:07:07,  5.73s/it]
 81%|████████  | 181598/225000 [344:07:22<64:06:17,  5.32s/it]
 81%|████████  | 181599/225000 [344:07:27<62:53:11,  5.22s/it]
 81%|████████  | 181600/225000 [344:07:31<59:48:25,  4.96s/it]
                                                              
{'loss': '0.7473', 'grad_norm': '1.077', 'learning_rate': '4.879e-06', 'epoch': '0.8071', 'num_input_tokens_seen': 570710448, 'train_runtime': '1.239e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181600/225000 [344:07:31<59:48:25,  4.96s/it]
 81%|████████  | 181601/225000 [344:07:39<69:09:26,  5.74s/it]
 81%|████████  | 181602/225000 [344:07:47<79:06:28,  6.56s/it]
 81%|████████  | 181603/225000 [344:07:51<68:06:13,  5.65s/it]
 81%|████████  | 181604/225000 [344:07:54<61:23:47,  5.09s/it]
 81%|████████  | 181605/225000 [344:07:58<55:25:34,  4.60s/it]
 81%|████████  | 181606/225000 [344:08:05<66:07:12,  5.49s/it]
 81%|████████  | 181607/225000 [344:08:14<78:13:33,  6.49s/it]
 81%|████████  | 181608/225000 [344:08:18<69:08:32,  5.74s/it]
 81%|████████  | 181609/225000 [344:08:22<62:27:43,  5.18s/it]
 81%|████████  | 181610/225000 [344:08:27<60:33:08,  5.02s/it]
 81%|████████  | 181611/225000 [344:08:31<58:00:56,  4.81s/it]
 81%|████████  | 181612/225000 [344:08:42<79:42:22,  6.61s/it]
 81%|████████  | 181613/225000 [344:08:45<68:14:31,  5.66s/it]
 81%|████████  | 181614/225000 [344:08:52<73:43:57,  6.12s/it]
 81%|████████  | 181615/225000 [344:08:58<72:29:14,  6.01s/it]
 81%|████████  | 181616/225000 [344:09:03<67:09:54,  5.57s/it]
 81%|████████  | 181617/225000 [344:09:08<65:53:47,  5.47s/it]
 81%|████████  | 181618/225000 [344:09:13<65:50:43,  5.46s/it]
 81%|████████  | 181619/225000 [344:09:18<63:27:05,  5.27s/it]
 81%|████████  | 181620/225000 [344:09:24<64:39:50,  5.37s/it]
 81%|████████  | 181621/225000 [344:09:29<64:50:30,  5.38s/it]
 81%|████████  | 181622/225000 [344:09:35<65:09:42,  5.41s/it]
 81%|████████  | 181623/225000 [344:09:40<63:19:34,  5.26s/it]
 81%|████████  | 181624/225000 [344:09:45<63:54:26,  5.30s/it]
 81%|████████  | 181625/225000 [344:09:51<64:41:51,  5.37s/it]
 81%|████████  | 181626/225000 [344:09:56<64:57:02,  5.39s/it]
 81%|████████  | 181627/225000 [344:10:01<62:58:46,  5.23s/it]
 81%|████████  | 181628/225000 [344:10:06<62:38:23,  5.20s/it]
 81%|████████  | 181629/225000 [344:10:11<61:13:18,  5.08s/it]
 81%|████████  | 181630/225000 [344:10:16<61:32:27,  5.11s/it]
 81%|████████  | 181631/225000 [344:10:22<63:35:43,  5.28s/it]
 81%|████████  | 181632/225000 [344:10:27<64:27:20,  5.35s/it]
 81%|████████  | 181633/225000 [344:10:32<63:22:37,  5.26s/it]
 81%|████████  | 181634/225000 [344:10:38<63:43:59,  5.29s/it]
 81%|████████  | 181635/225000 [344:10:45<71:30:06,  5.94s/it]
 81%|████████  | 181636/225000 [344:10:55<85:52:59,  7.13s/it]
 81%|████████  | 181637/225000 [344:10:59<76:32:53,  6.36s/it]
 81%|████████  | 181638/225000 [344:11:04<69:04:46,  5.74s/it]
 81%|████████  | 181639/225000 [344:11:08<63:37:33,  5.28s/it]
 81%|████████  | 181640/225000 [344:11:12<60:26:52,  5.02s/it]
 81%|████████  | 181641/225000 [344:11:23<80:47:45,  6.71s/it]
 81%|████████  | 181642/225000 [344:11:27<69:10:03,  5.74s/it]
 81%|████████  | 181643/225000 [344:11:32<69:39:36,  5.78s/it]
 81%|████████  | 181644/225000 [344:11:38<69:37:02,  5.78s/it]
 81%|████████  | 181645/225000 [344:11:43<65:45:23,  5.46s/it]
 81%|████████  | 181646/225000 [344:11:47<61:53:32,  5.14s/it]
 81%|████████  | 181647/225000 [344:11:52<60:09:00,  4.99s/it]
 81%|████████  | 181648/225000 [344:11:56<57:32:15,  4.78s/it]
 81%|████████  | 181649/225000 [344:12:03<64:46:44,  5.38s/it]
 81%|████████  | 181650/225000 [344:12:12<79:38:03,  6.61s/it]
                                                              
{'loss': '0.753', 'grad_norm': '1.114', 'learning_rate': '4.868e-06', 'epoch': '0.8073', 'num_input_tokens_seen': 570841048, 'train_runtime': '1.239e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181650/225000 [344:12:12<79:38:03,  6.61s/it]
 81%|████████  | 181651/225000 [344:12:16<70:00:43,  5.81s/it]
 81%|████████  | 181652/225000 [344:12:21<67:04:32,  5.57s/it]
 81%|████████  | 181653/225000 [344:12:27<66:44:37,  5.54s/it]
 81%|████████  | 181654/225000 [344:12:33<67:29:40,  5.61s/it]
 81%|████████  | 181655/225000 [344:12:38<65:56:56,  5.48s/it]
 81%|████████  | 181656/225000 [344:12:43<64:26:24,  5.35s/it]
 81%|████████  | 181657/225000 [344:12:48<62:05:58,  5.16s/it]
 81%|████████  | 181658/225000 [344:12:53<61:38:23,  5.12s/it]
 81%|████████  | 181659/225000 [344:12:58<62:06:59,  5.16s/it]
 81%|████████  | 181660/225000 [344:13:03<61:59:59,  5.15s/it]
 81%|████████  | 181661/225000 [344:13:08<62:24:50,  5.18s/it]
 81%|████████  | 181662/225000 [344:13:13<62:22:53,  5.18s/it]
 81%|████████  | 181663/225000 [344:13:19<63:17:15,  5.26s/it]
 81%|████████  | 181664/225000 [344:13:24<64:23:52,  5.35s/it]
 81%|████████  | 181665/225000 [344:13:30<64:20:17,  5.34s/it]
 81%|████████  | 181666/225000 [344:13:39<78:25:31,  6.52s/it]
 81%|████████  | 181667/225000 [344:13:46<78:57:43,  6.56s/it]
 81%|████████  | 181668/225000 [344:13:50<70:49:54,  5.88s/it]
 81%|████████  | 181669/225000 [344:13:54<64:26:25,  5.35s/it]
 81%|████████  | 181670/225000 [344:13:59<60:59:59,  5.07s/it]
 81%|████████  | 181671/225000 [344:14:04<63:40:34,  5.29s/it]
 81%|████████  | 181672/225000 [344:14:15<82:24:18,  6.85s/it]
 81%|████████  | 181673/225000 [344:14:18<69:46:55,  5.80s/it]
 81%|████████  | 181674/225000 [344:14:22<62:03:00,  5.16s/it]
 81%|████████  | 181675/225000 [344:14:26<59:36:03,  4.95s/it]
 81%|████████  | 181676/225000 [344:14:34<67:56:57,  5.65s/it]
 81%|████████  | 181677/225000 [344:14:42<78:54:06,  6.56s/it]
 81%|████████  | 181678/225000 [344:14:46<67:02:05,  5.57s/it]
 81%|████████  | 181679/225000 [344:14:53<75:10:24,  6.25s/it]
 81%|████████  | 181680/225000 [344:14:57<64:43:49,  5.38s/it]
 81%|████████  | 181681/225000 [344:15:04<70:05:36,  5.83s/it]
 81%|████████  | 181682/225000 [344:15:07<62:00:18,  5.15s/it]
 81%|████████  | 181683/225000 [344:15:11<55:52:50,  4.64s/it]
 81%|████████  | 181684/225000 [344:15:21<77:33:00,  6.45s/it]
 81%|████████  | 181685/225000 [344:15:25<66:09:18,  5.50s/it]
 81%|████████  | 181686/225000 [344:15:33<75:13:27,  6.25s/it]
 81%|████████  | 181687/225000 [344:15:36<64:46:14,  5.38s/it]
 81%|████████  | 181688/225000 [344:15:40<58:52:28,  4.89s/it]
 81%|████████  | 181689/225000 [344:15:44<56:17:23,  4.68s/it]
 81%|████████  | 181690/225000 [344:15:48<55:00:57,  4.57s/it]
 81%|████████  | 181691/225000 [344:15:53<55:09:21,  4.58s/it]
 81%|████████  | 181692/225000 [344:15:58<55:49:13,  4.64s/it]
 81%|████████  | 181693/225000 [344:16:03<57:28:30,  4.78s/it]
 81%|████████  | 181694/225000 [344:16:07<56:55:04,  4.73s/it]
 81%|████████  | 181695/225000 [344:16:12<57:41:13,  4.80s/it]
 81%|████████  | 181696/225000 [344:16:17<57:06:40,  4.75s/it]
 81%|████████  | 181697/225000 [344:16:22<59:09:42,  4.92s/it]
 81%|████████  | 181698/225000 [344:16:28<63:05:45,  5.25s/it]
 81%|████████  | 181699/225000 [344:16:39<82:34:45,  6.87s/it]
 81%|████████  | 181700/225000 [344:16:42<69:42:30,  5.80s/it]
                                                              
{'loss': '0.7633', 'grad_norm': '1.152', 'learning_rate': '4.858e-06', 'epoch': '0.8076', 'num_input_tokens_seen': 570970504, 'train_runtime': '1.239e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181700/225000 [344:16:42<69:42:30,  5.80s/it]
 81%|████████  | 181701/225000 [344:16:46<62:28:04,  5.19s/it]
 81%|████████  | 181702/225000 [344:16:50<58:37:26,  4.87s/it]
 81%|████████  | 181703/225000 [344:17:01<80:02:01,  6.65s/it]
 81%|████████  | 181704/225000 [344:17:04<68:47:36,  5.72s/it]
 81%|████████  | 181705/225000 [344:17:14<82:23:36,  6.85s/it]
 81%|████████  | 181706/225000 [344:17:19<76:28:55,  6.36s/it]
 81%|████████  | 181707/225000 [344:17:24<72:41:13,  6.04s/it]
 81%|████████  | 181708/225000 [344:17:30<70:03:46,  5.83s/it]
 81%|████████  | 181709/225000 [344:17:35<67:19:34,  5.60s/it]
 81%|████████  | 181710/225000 [344:17:40<65:03:05,  5.41s/it]
 81%|████████  | 181711/225000 [344:17:44<62:41:16,  5.21s/it]
 81%|████████  | 181712/225000 [344:17:49<61:50:54,  5.14s/it]
 81%|████████  | 181713/225000 [344:17:54<60:30:38,  5.03s/it]
 81%|████████  | 181714/225000 [344:17:59<59:13:07,  4.93s/it]
 81%|████████  | 181715/225000 [344:18:04<58:32:55,  4.87s/it]
 81%|████████  | 181716/225000 [344:18:08<58:18:01,  4.85s/it]
 81%|████████  | 181717/225000 [344:18:14<59:34:34,  4.96s/it]
 81%|████████  | 181718/225000 [344:18:19<60:18:11,  5.02s/it]
 81%|████████  | 181719/225000 [344:18:29<79:50:01,  6.64s/it]
 81%|████████  | 181720/225000 [344:18:36<78:56:59,  6.57s/it]
 81%|████████  | 181721/225000 [344:18:40<69:47:22,  5.81s/it]
 81%|████████  | 181722/225000 [344:18:44<62:52:15,  5.23s/it]
 81%|████████  | 181723/225000 [344:18:54<83:11:14,  6.92s/it]
 81%|████████  | 181724/225000 [344:18:58<71:08:55,  5.92s/it]
 81%|████████  | 181725/225000 [344:19:01<62:13:44,  5.18s/it]
 81%|████████  | 181726/225000 [344:19:08<67:07:26,  5.58s/it]
 81%|████████  | 181727/225000 [344:19:14<70:16:34,  5.85s/it]
 81%|████████  | 181728/225000 [344:19:18<63:37:06,  5.29s/it]
 81%|████████  | 181729/225000 [344:19:22<58:04:42,  4.83s/it]
 81%|████████  | 181730/225000 [344:19:26<55:22:02,  4.61s/it]
 81%|████████  | 181731/225000 [344:19:38<79:57:53,  6.65s/it]
 81%|████████  | 181732/225000 [344:19:42<71:17:57,  5.93s/it]
 81%|████████  | 181733/225000 [344:19:46<65:23:17,  5.44s/it]
 81%|████████  | 181734/225000 [344:19:51<61:20:42,  5.10s/it]
 81%|████████  | 181735/225000 [344:19:55<58:32:24,  4.87s/it]
 81%|████████  | 181736/225000 [344:20:00<60:15:44,  5.01s/it]
 81%|████████  | 181737/225000 [344:20:11<81:06:34,  6.75s/it]
 81%|████████  | 181738/225000 [344:20:14<67:15:06,  5.60s/it]
 81%|████████  | 181739/225000 [344:20:18<62:41:49,  5.22s/it]
 81%|████████  | 181740/225000 [344:20:25<69:51:36,  5.81s/it]
 81%|████████  | 181741/225000 [344:20:29<62:59:10,  5.24s/it]
 81%|████████  | 181742/225000 [344:20:33<57:46:00,  4.81s/it]
 81%|████████  | 181743/225000 [344:20:37<53:57:21,  4.49s/it]
 81%|████████  | 181744/225000 [344:20:47<74:35:44,  6.21s/it]
 81%|████████  | 181745/225000 [344:20:50<63:25:14,  5.28s/it]
 81%|████████  | 181746/225000 [344:20:57<68:41:49,  5.72s/it]
 81%|████████  | 181747/225000 [344:21:01<60:49:43,  5.06s/it]
 81%|████████  | 181748/225000 [344:21:05<57:22:42,  4.78s/it]
 81%|████████  | 181749/225000 [344:21:09<56:31:19,  4.70s/it]
 81%|████████  | 181750/225000 [344:21:14<56:14:27,  4.68s/it]
                                                              
{'loss': '0.7703', 'grad_norm': '1.213', 'learning_rate': '4.847e-06', 'epoch': '0.8078', 'num_input_tokens_seen': 571100944, 'train_runtime': '1.24e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181750/225000 [344:21:14<56:14:27,  4.68s/it]
 81%|████████  | 181751/225000 [344:21:24<74:16:13,  6.18s/it]
 81%|████████  | 181752/225000 [344:21:29<72:14:36,  6.01s/it]
 81%|████████  | 181753/225000 [344:21:34<68:08:54,  5.67s/it]
 81%|████████  | 181754/225000 [344:21:39<65:05:09,  5.42s/it]
 81%|████████  | 181755/225000 [344:21:44<63:00:18,  5.24s/it]
 81%|████████  | 181756/225000 [344:21:48<60:15:43,  5.02s/it]
 81%|████████  | 181757/225000 [344:21:52<57:50:18,  4.82s/it]
 81%|████████  | 181758/225000 [344:21:57<57:44:08,  4.81s/it]
 81%|████████  | 181759/225000 [344:22:02<58:10:09,  4.84s/it]
 81%|████████  | 181760/225000 [344:22:13<79:18:34,  6.60s/it]
 81%|████████  | 181761/225000 [344:22:17<69:56:46,  5.82s/it]
 81%|████████  | 181762/225000 [344:22:21<62:28:46,  5.20s/it]
 81%|████████  | 181763/225000 [344:22:24<56:34:52,  4.71s/it]
 81%|████████  | 181764/225000 [344:22:34<73:51:01,  6.15s/it]
 81%|████████  | 181765/225000 [344:22:38<68:25:08,  5.70s/it]
 81%|████████  | 181766/225000 [344:22:43<63:06:39,  5.26s/it]
 81%|████████  | 181767/225000 [344:22:47<58:57:13,  4.91s/it]
 81%|████████  | 181768/225000 [344:22:51<57:28:39,  4.79s/it]
 81%|████████  | 181769/225000 [344:22:56<57:04:32,  4.75s/it]
 81%|████████  | 181770/225000 [344:23:01<56:46:36,  4.73s/it]
 81%|████████  | 181771/225000 [344:23:12<79:31:28,  6.62s/it]
 81%|████████  | 181772/225000 [344:23:16<69:55:51,  5.82s/it]
 81%|████████  | 181773/225000 [344:23:19<63:02:08,  5.25s/it]
 81%|████████  | 181774/225000 [344:23:24<59:23:11,  4.95s/it]
 81%|████████  | 181775/225000 [344:23:33<74:06:25,  6.17s/it]
 81%|████████  | 181776/225000 [344:23:37<66:48:30,  5.56s/it]
 81%|████████  | 181777/225000 [344:23:41<60:01:38,  5.00s/it]
 81%|████████  | 181778/225000 [344:23:48<69:33:59,  5.79s/it]
 81%|████████  | 181779/225000 [344:23:56<76:18:00,  6.36s/it]
 81%|████████  | 181780/225000 [344:24:00<69:42:16,  5.81s/it]
 81%|████████  | 181781/225000 [344:24:05<64:03:14,  5.34s/it]
 81%|████████  | 181782/225000 [344:24:09<61:37:38,  5.13s/it]
 81%|████████  | 181783/225000 [344:24:14<59:40:25,  4.97s/it]
 81%|████████  | 181784/225000 [344:24:18<56:15:27,  4.69s/it]
 81%|████████  | 181785/225000 [344:24:29<78:40:55,  6.55s/it]
 81%|████████  | 181786/225000 [344:24:34<74:50:07,  6.23s/it]
 81%|████████  | 181787/225000 [344:24:39<70:39:04,  5.89s/it]
 81%|████████  | 181788/225000 [344:24:45<69:06:42,  5.76s/it]
 81%|████████  | 181789/225000 [344:24:50<66:19:33,  5.53s/it]
 81%|████████  | 181790/225000 [344:24:55<66:01:54,  5.50s/it]
 81%|████████  | 181791/225000 [344:25:01<65:30:16,  5.46s/it]
 81%|████████  | 181792/225000 [344:25:05<63:05:14,  5.26s/it]
 81%|████████  | 181793/225000 [344:25:10<60:35:43,  5.05s/it]
 81%|████████  | 181794/225000 [344:25:15<60:15:39,  5.02s/it]
 81%|████████  | 181795/225000 [344:25:20<60:50:28,  5.07s/it]
 81%|████████  | 181796/225000 [344:25:26<62:46:48,  5.23s/it]
 81%|████████  | 181797/225000 [344:25:31<61:49:11,  5.15s/it]
 81%|████████  | 181798/225000 [344:25:36<61:23:06,  5.12s/it]
 81%|████████  | 181799/225000 [344:25:41<60:46:36,  5.06s/it]
 81%|████████  | 181800/225000 [344:25:46<61:07:38,  5.09s/it]
                                                              
{'loss': '0.7666', 'grad_norm': '1.089', 'learning_rate': '4.836e-06', 'epoch': '0.808', 'num_input_tokens_seen': 571230568, 'train_runtime': '1.24e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181800/225000 [344:25:46<61:07:38,  5.09s/it]
 81%|████████  | 181801/225000 [344:25:51<62:00:54,  5.17s/it]
 81%|████████  | 181802/225000 [344:25:56<62:27:11,  5.20s/it]
 81%|████████  | 181803/225000 [344:26:02<62:47:58,  5.23s/it]
 81%|████████  | 181804/225000 [344:26:07<63:03:54,  5.26s/it]
 81%|████████  | 181805/225000 [344:26:12<63:20:27,  5.28s/it]
 81%|████████  | 181806/225000 [344:26:17<62:07:41,  5.18s/it]
 81%|████████  | 181807/225000 [344:26:30<88:04:34,  7.34s/it]
 81%|████████  | 181808/225000 [344:26:35<79:49:34,  6.65s/it]
 81%|████████  | 181809/225000 [344:26:39<69:24:06,  5.78s/it]
 81%|████████  | 181810/225000 [344:26:43<63:22:26,  5.28s/it]
 81%|████████  | 181811/225000 [344:26:51<73:09:09,  6.10s/it]
 81%|████████  | 181812/225000 [344:26:57<73:22:52,  6.12s/it]
 81%|████████  | 181813/225000 [344:27:01<67:47:58,  5.65s/it]
 81%|████████  | 181814/225000 [344:27:06<65:20:25,  5.45s/it]
 81%|████████  | 181815/225000 [344:27:12<64:22:33,  5.37s/it]
 81%|████████  | 181816/225000 [344:27:17<63:43:58,  5.31s/it]
 81%|████████  | 181817/225000 [344:27:22<62:23:21,  5.20s/it]
 81%|████████  | 181818/225000 [344:27:27<62:06:18,  5.18s/it]
 81%|████████  | 181819/225000 [344:27:32<61:48:23,  5.15s/it]
 81%|████████  | 181820/225000 [344:27:37<62:32:02,  5.21s/it]
 81%|████████  | 181821/225000 [344:27:42<59:40:40,  4.98s/it]
 81%|████████  | 181822/225000 [344:27:50<70:47:37,  5.90s/it]
 81%|████████  | 181823/225000 [344:27:58<80:34:18,  6.72s/it]
 81%|████████  | 181824/225000 [344:28:02<69:13:54,  5.77s/it]
 81%|████████  | 181825/225000 [344:28:07<67:36:47,  5.64s/it]
 81%|████████  | 181826/225000 [344:28:16<77:31:31,  6.46s/it]
 81%|████████  | 181827/225000 [344:28:19<66:46:51,  5.57s/it]
 81%|████████  | 181828/225000 [344:28:23<60:26:53,  5.04s/it]
 81%|████████  | 181829/225000 [344:28:27<58:33:59,  4.88s/it]
 81%|████████  | 181830/225000 [344:28:32<57:45:41,  4.82s/it]
 81%|████████  | 181831/225000 [344:28:37<57:56:08,  4.83s/it]
 81%|████████  | 181832/225000 [344:28:42<58:35:36,  4.89s/it]
 81%|████████  | 181833/225000 [344:28:47<57:29:15,  4.79s/it]
 81%|████████  | 181834/225000 [344:28:52<58:36:04,  4.89s/it]
 81%|████████  | 181835/225000 [344:28:56<57:35:31,  4.80s/it]
 81%|████████  | 181836/225000 [344:29:02<60:31:39,  5.05s/it]
 81%|████████  | 181837/225000 [344:29:14<84:19:25,  7.03s/it]
 81%|████████  | 181838/225000 [344:29:17<72:16:48,  6.03s/it]
 81%|████████  | 181839/225000 [344:29:22<67:37:43,  5.64s/it]
 81%|████████  | 181840/225000 [344:29:31<79:05:08,  6.60s/it]
 81%|████████  | 181841/225000 [344:29:34<67:18:59,  5.62s/it]
 81%|████████  | 181842/225000 [344:29:41<70:31:46,  5.88s/it]
 81%|████████  | 181843/225000 [344:29:45<63:38:04,  5.31s/it]
 81%|████████  | 181844/225000 [344:29:48<57:47:03,  4.82s/it]
 81%|████████  | 181845/225000 [344:29:58<74:18:30,  6.20s/it]
 81%|████████  | 181846/225000 [344:30:01<64:20:37,  5.37s/it]
 81%|████████  | 181847/225000 [344:30:07<65:21:21,  5.45s/it]
 81%|████████  | 181848/225000 [344:30:15<73:43:20,  6.15s/it]
 81%|████████  | 181849/225000 [344:30:18<62:32:41,  5.22s/it]
 81%|████████  | 181850/225000 [344:30:22<58:49:09,  4.91s/it]
                                                              
{'loss': '0.7621', 'grad_norm': '1.251', 'learning_rate': '4.825e-06', 'epoch': '0.8082', 'num_input_tokens_seen': 571359704, 'train_runtime': '1.24e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181850/225000 [344:30:22<58:49:09,  4.91s/it]
 81%|████████  | 181851/225000 [344:30:31<75:01:47,  6.26s/it]
 81%|████████  | 181852/225000 [344:30:35<67:22:48,  5.62s/it]
 81%|████████  | 181853/225000 [344:30:40<63:00:57,  5.26s/it]
 81%|████████  | 181854/225000 [344:30:44<60:00:38,  5.01s/it]
 81%|████████  | 181855/225000 [344:30:48<56:57:29,  4.75s/it]
 81%|████████  | 181856/225000 [344:30:53<56:04:47,  4.68s/it]
 81%|████████  | 181857/225000 [344:31:03<77:34:30,  6.47s/it]
 81%|████████  | 181858/225000 [344:31:07<66:39:02,  5.56s/it]
 81%|████████  | 181859/225000 [344:31:16<78:18:54,  6.54s/it]
 81%|████████  | 181860/225000 [344:31:19<66:12:46,  5.53s/it]
 81%|████████  | 181861/225000 [344:31:25<69:25:44,  5.79s/it]
 81%|████████  | 181862/225000 [344:31:31<68:10:30,  5.69s/it]
 81%|████████  | 181863/225000 [344:31:34<61:01:05,  5.09s/it]
 81%|████████  | 181864/225000 [344:31:38<55:44:09,  4.65s/it]
 81%|████████  | 181865/225000 [344:31:43<58:20:45,  4.87s/it]
 81%|████████  | 181866/225000 [344:31:49<59:55:45,  5.00s/it]
 81%|████████  | 181867/225000 [344:31:54<61:45:27,  5.15s/it]
 81%|████████  | 181868/225000 [344:32:00<62:00:28,  5.18s/it]
 81%|████████  | 181869/225000 [344:32:05<61:46:50,  5.16s/it]
 81%|████████  | 181870/225000 [344:32:10<60:57:09,  5.09s/it]
 81%|████████  | 181871/225000 [344:32:14<60:21:33,  5.04s/it]
 81%|████████  | 181872/225000 [344:32:20<62:22:59,  5.21s/it]
 81%|████████  | 181873/225000 [344:32:25<61:14:01,  5.11s/it]
 81%|████████  | 181874/225000 [344:32:35<78:16:22,  6.53s/it]
 81%|████████  | 181875/225000 [344:32:39<71:25:47,  5.96s/it]
 81%|████████  | 181876/225000 [344:32:44<66:16:56,  5.53s/it]
 81%|████████  | 181877/225000 [344:32:52<74:18:48,  6.20s/it]
 81%|████████  | 181878/225000 [344:32:59<77:51:26,  6.50s/it]
 81%|████████  | 181879/225000 [344:33:03<70:47:15,  5.91s/it]
 81%|████████  | 181880/225000 [344:33:08<66:05:39,  5.52s/it]
 81%|████████  | 181881/225000 [344:33:13<63:03:14,  5.26s/it]
 81%|████████  | 181882/225000 [344:33:17<59:10:27,  4.94s/it]
 81%|████████  | 181883/225000 [344:33:21<57:24:59,  4.79s/it]
 81%|████████  | 181884/225000 [344:33:26<56:58:14,  4.76s/it]
 81%|████████  | 181885/225000 [344:33:39<85:42:59,  7.16s/it]
 81%|████████  | 181886/225000 [344:33:42<72:28:09,  6.05s/it]
 81%|████████  | 181887/225000 [344:33:50<77:56:58,  6.51s/it]
 81%|████████  | 181888/225000 [344:33:53<67:01:27,  5.60s/it]
 81%|████████  | 181889/225000 [344:34:01<74:49:12,  6.25s/it]
 81%|████████  | 181890/225000 [344:34:06<70:20:52,  5.87s/it]
 81%|████████  | 181891/225000 [344:34:11<66:16:13,  5.53s/it]
 81%|████████  | 181892/225000 [344:34:15<62:57:37,  5.26s/it]
 81%|████████  | 181893/225000 [344:34:20<60:50:15,  5.08s/it]
 81%|████████  | 181894/225000 [344:34:25<59:37:11,  4.98s/it]
 81%|████████  | 181895/225000 [344:34:29<57:59:55,  4.84s/it]
 81%|████████  | 181896/225000 [344:34:34<56:48:56,  4.75s/it]
 81%|████████  | 181897/225000 [344:34:43<73:01:40,  6.10s/it]
 81%|████████  | 181898/225000 [344:34:51<79:30:12,  6.64s/it]
 81%|████████  | 181899/225000 [344:34:55<69:23:18,  5.80s/it]
 81%|████████  | 181900/225000 [344:34:59<63:07:38,  5.27s/it]
                                                              
{'loss': '0.7625', 'grad_norm': '1.171', 'learning_rate': '4.814e-06', 'epoch': '0.8084', 'num_input_tokens_seen': 571490048, 'train_runtime': '1.24e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181900/225000 [344:34:59<63:07:38,  5.27s/it]
 81%|████████  | 181901/225000 [344:35:03<58:07:17,  4.85s/it]
 81%|████████  | 181902/225000 [344:35:15<85:14:24,  7.12s/it]
 81%|████████  | 181903/225000 [344:35:19<72:34:54,  6.06s/it]
 81%|████████  | 181904/225000 [344:35:22<63:53:29,  5.34s/it]
 81%|████████  | 181905/225000 [344:35:26<56:27:45,  4.72s/it]
 81%|████████  | 181906/225000 [344:35:35<74:05:19,  6.19s/it]
 81%|████████  | 181907/225000 [344:35:39<64:24:16,  5.38s/it]
 81%|████████  | 181908/225000 [344:35:43<58:25:44,  4.88s/it]
 81%|████████  | 181909/225000 [344:35:53<76:44:51,  6.41s/it]
 81%|████████  | 181910/225000 [344:35:58<72:21:19,  6.05s/it]
 81%|████████  | 181911/225000 [344:36:03<68:19:49,  5.71s/it]
 81%|████████  | 181912/225000 [344:36:06<60:30:42,  5.06s/it]
 81%|████████  | 181913/225000 [344:36:10<55:32:34,  4.64s/it]
 81%|████████  | 181914/225000 [344:36:19<73:04:30,  6.11s/it]
 81%|████████  | 181915/225000 [344:36:24<69:22:21,  5.80s/it]
 81%|████████  | 181916/225000 [344:36:31<70:17:33,  5.87s/it]
 81%|████████  | 181917/225000 [344:36:34<61:38:35,  5.15s/it]
 81%|████████  | 181918/225000 [344:36:40<65:56:47,  5.51s/it]
 81%|████████  | 181919/225000 [344:36:46<67:35:43,  5.65s/it]
 81%|████████  | 181920/225000 [344:36:50<59:17:58,  4.96s/it]
 81%|████████  | 181921/225000 [344:36:57<68:21:34,  5.71s/it]
 81%|████████  | 181922/225000 [344:37:03<67:36:48,  5.65s/it]
 81%|████████  | 181923/225000 [344:37:06<59:08:44,  4.94s/it]
 81%|████████  | 181924/225000 [344:37:13<67:24:11,  5.63s/it]
 81%|████████  | 181925/225000 [344:37:17<62:05:48,  5.19s/it]
 81%|████████  | 181926/225000 [344:37:22<60:35:52,  5.06s/it]
 81%|████████  | 181927/225000 [344:37:27<60:00:30,  5.02s/it]
 81%|████████  | 181928/225000 [344:37:32<59:32:50,  4.98s/it]
 81%|████████  | 181929/225000 [344:37:37<59:36:00,  4.98s/it]
 81%|████████  | 181930/225000 [344:37:42<59:43:13,  4.99s/it]
 81%|████████  | 181931/225000 [344:37:47<59:08:19,  4.94s/it]
 81%|████████  | 181932/225000 [344:37:52<60:00:46,  5.02s/it]
 81%|████████  | 181933/225000 [344:37:57<59:59:47,  5.02s/it]
 81%|████████  | 181934/225000 [344:38:02<60:47:32,  5.08s/it]
 81%|████████  | 181935/225000 [344:38:07<61:31:51,  5.14s/it]
 81%|████████  | 181936/225000 [344:38:12<61:10:21,  5.11s/it]
 81%|████████  | 181937/225000 [344:38:21<73:57:18,  6.18s/it]
 81%|████████  | 181938/225000 [344:38:30<82:29:12,  6.90s/it]
 81%|████████  | 181939/225000 [344:38:33<69:28:49,  5.81s/it]
 81%|████████  | 181940/225000 [344:38:38<64:50:25,  5.42s/it]
 81%|████████  | 181941/225000 [344:38:46<75:48:05,  6.34s/it]
 81%|████████  | 181942/225000 [344:38:50<66:04:48,  5.52s/it]
 81%|████████  | 181943/225000 [344:38:54<60:18:53,  5.04s/it]
 81%|████████  | 181944/225000 [344:39:01<69:18:16,  5.79s/it]
 81%|████████  | 181945/225000 [344:39:10<79:36:39,  6.66s/it]
 81%|████████  | 181946/225000 [344:39:13<68:35:37,  5.74s/it]
 81%|████████  | 181947/225000 [344:39:17<61:08:42,  5.11s/it]
 81%|████████  | 181948/225000 [344:39:21<57:17:29,  4.79s/it]
 81%|████████  | 181949/225000 [344:39:26<58:04:31,  4.86s/it]
 81%|████████  | 181950/225000 [344:39:31<59:27:11,  4.97s/it]
                                                              
{'loss': '0.7591', 'grad_norm': '1.255', 'learning_rate': '4.804e-06', 'epoch': '0.8087', 'num_input_tokens_seen': 571619424, 'train_runtime': '1.241e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 181950/225000 [344:39:31<59:27:11,  4.97s/it]
 81%|████████  | 181951/225000 [344:39:37<60:53:24,  5.09s/it]
 81%|████████  | 181952/225000 [344:39:42<61:38:01,  5.15s/it]
 81%|████████  | 181953/225000 [344:39:47<62:49:57,  5.25s/it]
 81%|████████  | 181954/225000 [344:39:52<61:23:14,  5.13s/it]
 81%|████████  | 181955/225000 [344:39:58<61:57:44,  5.18s/it]
 81%|████████  | 181956/225000 [344:40:03<62:56:14,  5.26s/it]
 81%|████████  | 181957/225000 [344:40:08<62:24:13,  5.22s/it]
 81%|████████  | 181958/225000 [344:40:14<63:19:36,  5.30s/it]
 81%|████████  | 181959/225000 [344:40:18<61:37:17,  5.15s/it]
 81%|████████  | 181960/225000 [344:40:24<62:26:03,  5.22s/it]
 81%|████████  | 181961/225000 [344:40:29<62:55:55,  5.26s/it]
 81%|████████  | 181962/225000 [344:40:35<63:02:26,  5.27s/it]
 81%|████████  | 181963/225000 [344:40:40<63:03:14,  5.27s/it]
 81%|████████  | 181964/225000 [344:40:45<63:43:42,  5.33s/it]
 81%|████████  | 181965/225000 [344:40:50<63:15:11,  5.29s/it]
 81%|████████  | 181966/225000 [344:41:01<82:19:19,  6.89s/it]
 81%|████████  | 181967/225000 [344:41:07<79:08:03,  6.62s/it]
 81%|████████  | 181968/225000 [344:41:11<68:55:29,  5.77s/it]
 81%|████████  | 181969/225000 [344:41:15<62:22:59,  5.22s/it]
 81%|████████  | 181970/225000 [344:41:20<63:25:25,  5.31s/it]
 81%|████████  | 181971/225000 [344:41:29<74:59:45,  6.27s/it]
 81%|████████  | 181972/225000 [344:41:33<67:09:40,  5.62s/it]
 81%|████████  | 181973/225000 [344:41:38<64:14:56,  5.38s/it]
 81%|████████  | 181974/225000 [344:41:42<61:51:55,  5.18s/it]
 81%|████████  | 181975/225000 [344:41:47<59:19:19,  4.96s/it]
 81%|████████  | 181976/225000 [344:41:52<58:18:31,  4.88s/it]
 81%|████████  | 181977/225000 [344:41:56<57:46:21,  4.83s/it]
 81%|████████  | 181978/225000 [344:42:01<57:46:56,  4.84s/it]
 81%|████████  | 181979/225000 [344:42:12<80:46:20,  6.76s/it]
 81%|████████  | 181980/225000 [344:42:18<77:45:27,  6.51s/it]
 81%|████████  | 181981/225000 [344:42:24<75:55:42,  6.35s/it]
 81%|████████  | 181982/225000 [344:42:30<73:38:00,  6.16s/it]
 81%|████████  | 181983/225000 [344:42:35<70:03:26,  5.86s/it]
 81%|████████  | 181984/225000 [344:42:39<64:07:03,  5.37s/it]
 81%|████████  | 181985/225000 [344:42:44<61:27:52,  5.14s/it]
 81%|████████  | 181986/225000 [344:42:49<60:47:39,  5.09s/it]
 81%|████████  | 181987/225000 [344:42:54<58:57:38,  4.93s/it]
 81%|████████  | 181988/225000 [344:42:58<58:43:49,  4.92s/it]
 81%|████████  | 181989/225000 [344:43:03<58:51:37,  4.93s/it]
 81%|████████  | 181990/225000 [344:43:13<74:09:15,  6.21s/it]
 81%|████████  | 181991/225000 [344:43:19<75:02:16,  6.28s/it]
 81%|████████  | 181992/225000 [344:43:24<70:57:05,  5.94s/it]
 81%|████████  | 181993/225000 [344:43:32<79:16:30,  6.64s/it]
 81%|████████  | 181994/225000 [344:43:36<68:57:43,  5.77s/it]
 81%|████████  | 181995/225000 [344:43:40<63:00:39,  5.27s/it]
 81%|████████  | 181996/225000 [344:43:47<67:25:20,  5.64s/it]
 81%|████████  | 181997/225000 [344:43:54<74:02:21,  6.20s/it]
 81%|████████  | 181998/225000 [344:43:58<64:49:03,  5.43s/it]
 81%|████████  | 181999/225000 [344:44:03<63:46:06,  5.34s/it]
 81%|████████  | 182000/225000 [344:44:11<73:38:17,  6.17s/it]
                                                              
{'loss': '0.7601', 'grad_norm': '1.219', 'learning_rate': '4.793e-06', 'epoch': '0.8089', 'num_input_tokens_seen': 571750672, 'train_runtime': '1.241e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182000/225000 [344:44:11<73:38:17,  6.17s/it]
 81%|████████  | 182001/225000 [344:44:15<64:36:34,  5.41s/it]
 81%|████████  | 182002/225000 [344:44:21<66:43:31,  5.59s/it]
 81%|████████  | 182003/225000 [344:44:28<73:32:11,  6.16s/it]
 81%|████████  | 182004/225000 [344:44:32<66:09:25,  5.54s/it]
 81%|████████  | 182005/225000 [344:44:36<61:00:02,  5.11s/it]
 81%|████████  | 182006/225000 [344:44:44<71:26:18,  5.98s/it]
 81%|████████  | 182007/225000 [344:44:50<69:30:54,  5.82s/it]
 81%|████████  | 182008/225000 [344:44:53<61:21:41,  5.14s/it]
 81%|████████  | 182009/225000 [344:45:03<75:46:07,  6.34s/it]
 81%|████████  | 182010/225000 [344:45:06<64:24:32,  5.39s/it]
 81%|████████  | 182011/225000 [344:45:11<63:53:15,  5.35s/it]
 81%|████████  | 182012/225000 [344:45:17<65:05:43,  5.45s/it]
 81%|████████  | 182013/225000 [344:45:24<70:10:33,  5.88s/it]
 81%|████████  | 182014/225000 [344:45:28<64:34:14,  5.41s/it]
 81%|████████  | 182015/225000 [344:45:33<62:45:02,  5.26s/it]
 81%|████████  | 182016/225000 [344:45:37<59:33:20,  4.99s/it]
 81%|████████  | 182017/225000 [344:45:42<58:18:41,  4.88s/it]
 81%|████████  | 182018/225000 [344:45:46<56:28:15,  4.73s/it]
 81%|████████  | 182019/225000 [344:45:51<56:22:59,  4.72s/it]
 81%|████████  | 182020/225000 [344:45:55<55:27:22,  4.65s/it]
 81%|████████  | 182021/225000 [344:46:06<78:13:53,  6.55s/it]
 81%|████████  | 182022/225000 [344:46:12<74:22:05,  6.23s/it]
 81%|████████  | 182023/225000 [344:46:16<66:36:02,  5.58s/it]
 81%|████████  | 182024/225000 [344:46:20<61:51:42,  5.18s/it]
 81%|████████  | 182025/225000 [344:46:27<66:46:09,  5.59s/it]
 81%|████████  | 182026/225000 [344:46:35<76:03:28,  6.37s/it]
 81%|████████  | 182027/225000 [344:46:38<65:42:48,  5.51s/it]
 81%|████████  | 182028/225000 [344:46:44<65:04:41,  5.45s/it]
 81%|████████  | 182029/225000 [344:46:51<72:10:21,  6.05s/it]
 81%|████████  | 182030/225000 [344:46:57<71:30:59,  5.99s/it]
 81%|████████  | 182031/225000 [344:47:02<66:29:42,  5.57s/it]
 81%|████████  | 182032/225000 [344:47:06<63:10:01,  5.29s/it]
 81%|████████  | 182033/225000 [344:47:10<58:52:47,  4.93s/it]
 81%|████████  | 182034/225000 [344:47:15<58:16:09,  4.88s/it]
 81%|████████  | 182035/225000 [344:47:20<57:04:44,  4.78s/it]
 81%|████████  | 182036/225000 [344:47:24<54:12:58,  4.54s/it]
 81%|████████  | 182037/225000 [344:47:36<83:49:00,  7.02s/it]
 81%|████████  | 182038/225000 [344:47:41<76:16:54,  6.39s/it]
 81%|████████  | 182039/225000 [344:47:47<75:03:42,  6.29s/it]
 81%|████████  | 182040/225000 [344:47:50<63:17:02,  5.30s/it]
 81%|████████  | 182041/225000 [344:47:57<68:51:37,  5.77s/it]
 81%|████████  | 182042/225000 [344:48:01<62:36:13,  5.25s/it]
 81%|████████  | 182043/225000 [344:48:07<62:47:27,  5.26s/it]
 81%|████████  | 182044/225000 [344:48:15<74:04:56,  6.21s/it]
 81%|████████  | 182045/225000 [344:48:18<63:02:44,  5.28s/it]
 81%|████████  | 182046/225000 [344:48:22<56:21:19,  4.72s/it]
 81%|████████  | 182047/225000 [344:48:31<73:55:52,  6.20s/it]
 81%|████████  | 182048/225000 [344:48:35<65:13:25,  5.47s/it]
 81%|████████  | 182049/225000 [344:48:39<59:11:12,  4.96s/it]
 81%|████████  | 182050/225000 [344:48:43<56:40:10,  4.75s/it]
                                                              
{'loss': '0.775', 'grad_norm': '1.227', 'learning_rate': '4.782e-06', 'epoch': '0.8091', 'num_input_tokens_seen': 571881488, 'train_runtime': '1.241e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182050/225000 [344:48:43<56:40:10,  4.75s/it]
 81%|████████  | 182051/225000 [344:48:48<57:21:26,  4.81s/it]
 81%|████████  | 182052/225000 [344:48:53<59:07:09,  4.96s/it]
 81%|████████  | 182053/225000 [344:48:58<57:33:18,  4.82s/it]
 81%|████████  | 182054/225000 [344:49:03<60:18:44,  5.06s/it]
 81%|████████  | 182055/225000 [344:49:09<61:26:21,  5.15s/it]
 81%|████████  | 182056/225000 [344:49:14<61:39:06,  5.17s/it]
 81%|████████  | 182057/225000 [344:49:19<61:08:51,  5.13s/it]
 81%|████████  | 182058/225000 [344:49:24<60:50:22,  5.10s/it]
 81%|████████  | 182059/225000 [344:49:30<62:41:36,  5.26s/it]
 81%|████████  | 182060/225000 [344:49:35<61:56:18,  5.19s/it]
 81%|████████  | 182061/225000 [344:49:40<61:17:13,  5.14s/it]
 81%|████████  | 182062/225000 [344:49:45<62:53:11,  5.27s/it]
 81%|████████  | 182063/225000 [344:49:57<86:10:52,  7.23s/it]
 81%|████████  | 182064/225000 [344:50:02<79:45:28,  6.69s/it]
 81%|████████  | 182065/225000 [344:50:08<74:26:25,  6.24s/it]
 81%|████████  | 182066/225000 [344:50:13<70:23:54,  5.90s/it]
 81%|████████  | 182067/225000 [344:50:18<66:18:08,  5.56s/it]
 81%|████████  | 182068/225000 [344:50:23<64:54:16,  5.44s/it]
 81%|████████  | 182069/225000 [344:50:28<65:07:32,  5.46s/it]
 81%|████████  | 182070/225000 [344:50:34<65:09:14,  5.46s/it]
 81%|████████  | 182071/225000 [344:50:43<79:51:45,  6.70s/it]
 81%|████████  | 182072/225000 [344:50:47<70:45:21,  5.93s/it]
 81%|████████  | 182073/225000 [344:50:52<64:47:32,  5.43s/it]
 81%|████████  | 182074/225000 [344:50:56<61:50:46,  5.19s/it]
 81%|████████  | 182075/225000 [344:51:01<58:33:49,  4.91s/it]
 81%|████████  | 182076/225000 [344:51:05<56:13:18,  4.72s/it]
 81%|████████  | 182077/225000 [344:51:12<66:42:06,  5.59s/it]
 81%|████████  | 182078/225000 [344:51:20<73:14:46,  6.14s/it]
 81%|████████  | 182079/225000 [344:51:24<64:30:22,  5.41s/it]
 81%|████████  | 182080/225000 [344:51:31<72:36:38,  6.09s/it]
 81%|████████  | 182081/225000 [344:51:36<68:10:56,  5.72s/it]
 81%|████████  | 182082/225000 [344:51:41<64:05:25,  5.38s/it]
 81%|████████  | 182083/225000 [344:51:46<62:12:55,  5.22s/it]
 81%|████████  | 182084/225000 [344:51:50<60:39:21,  5.09s/it]
 81%|████████  | 182085/225000 [344:51:55<58:40:25,  4.92s/it]
 81%|████████  | 182086/225000 [344:51:59<57:28:04,  4.82s/it]
 81%|████████  | 182087/225000 [344:52:12<83:36:34,  7.01s/it]
 81%|████████  | 182088/225000 [344:52:16<73:04:43,  6.13s/it]
 81%|████████  | 182089/225000 [344:52:21<71:29:45,  6.00s/it]
 81%|████████  | 182090/225000 [344:52:26<67:17:14,  5.65s/it]
 81%|████████  | 182091/225000 [344:52:31<65:48:00,  5.52s/it]
 81%|████████  | 182092/225000 [344:52:37<66:25:32,  5.57s/it]
 81%|████████  | 182093/225000 [344:52:42<64:56:33,  5.45s/it]
 81%|████████  | 182094/225000 [344:52:48<64:26:38,  5.41s/it]
 81%|████████  | 182095/225000 [344:52:53<64:50:24,  5.44s/it]
 81%|████████  | 182096/225000 [344:52:58<62:29:05,  5.24s/it]
 81%|████████  | 182097/225000 [344:53:04<66:19:28,  5.57s/it]
 81%|████████  | 182098/225000 [344:53:11<71:30:55,  6.00s/it]
 81%|████████  | 182099/225000 [344:53:16<65:40:22,  5.51s/it]
 81%|████████  | 182100/225000 [344:53:20<61:06:59,  5.13s/it]
                                                              
{'loss': '0.7608', 'grad_norm': '1.143', 'learning_rate': '4.771e-06', 'epoch': '0.8093', 'num_input_tokens_seen': 572010640, 'train_runtime': '1.242e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182100/225000 [344:53:20<61:06:59,  5.13s/it]
 81%|████████  | 182101/225000 [344:53:24<57:49:47,  4.85s/it]
 81%|████████  | 182102/225000 [344:53:29<57:25:51,  4.82s/it]
 81%|████████  | 182103/225000 [344:53:33<57:03:48,  4.79s/it]
 81%|████████  | 182104/225000 [344:53:45<80:46:47,  6.78s/it]
 81%|████████  | 182105/225000 [344:53:48<68:23:10,  5.74s/it]
 81%|████████  | 182106/225000 [344:53:55<71:40:38,  6.02s/it]
 81%|████████  | 182107/225000 [344:53:59<66:08:38,  5.55s/it]
 81%|████████  | 182108/225000 [344:54:07<72:46:24,  6.11s/it]
 81%|████████  | 182109/225000 [344:54:12<71:19:06,  5.99s/it]
 81%|████████  | 182110/225000 [344:54:17<66:57:06,  5.62s/it]
 81%|████████  | 182111/225000 [344:54:22<64:53:28,  5.45s/it]
 81%|████████  | 182112/225000 [344:54:27<62:18:43,  5.23s/it]
 81%|████████  | 182113/225000 [344:54:32<60:08:15,  5.05s/it]
 81%|████████  | 182114/225000 [344:54:36<58:21:58,  4.90s/it]
 81%|████████  | 182115/225000 [344:54:41<57:31:41,  4.83s/it]
 81%|████████  | 182116/225000 [344:54:46<57:01:22,  4.79s/it]
 81%|████████  | 182117/225000 [344:54:50<56:38:06,  4.75s/it]
 81%|████████  | 182118/225000 [344:55:00<75:54:06,  6.37s/it]
 81%|████████  | 182119/225000 [344:55:06<73:31:05,  6.17s/it]
 81%|████████  | 182120/225000 [344:55:10<66:25:53,  5.58s/it]
 81%|████████  | 182121/225000 [344:55:15<62:42:46,  5.27s/it]
 81%|████████  | 182122/225000 [344:55:19<59:22:51,  4.99s/it]
 81%|████████  | 182123/225000 [344:55:23<56:49:34,  4.77s/it]
 81%|████████  | 182124/225000 [344:55:28<55:40:49,  4.68s/it]
 81%|████████  | 182125/225000 [344:55:33<57:08:20,  4.80s/it]
 81%|████████  | 182126/225000 [344:55:46<85:56:39,  7.22s/it]
 81%|████████  | 182127/225000 [344:55:50<75:33:05,  6.34s/it]
 81%|████████  | 182128/225000 [344:55:54<68:07:39,  5.72s/it]
 81%|████████  | 182129/225000 [344:55:59<63:38:17,  5.34s/it]
 81%|████████  | 182130/225000 [344:56:03<60:13:53,  5.06s/it]
 81%|████████  | 182131/225000 [344:56:07<57:03:45,  4.79s/it]
 81%|████████  | 182132/225000 [344:56:16<71:50:52,  6.03s/it]
 81%|████████  | 182133/225000 [344:56:26<83:57:22,  7.05s/it]
 81%|████████  | 182134/225000 [344:56:29<70:01:38,  5.88s/it]
 81%|████████  | 182135/225000 [344:56:37<76:49:04,  6.45s/it]
 81%|████████  | 182136/225000 [344:56:40<66:22:04,  5.57s/it]
 81%|████████  | 182137/225000 [344:56:49<77:03:46,  6.47s/it]
 81%|████████  | 182138/225000 [344:56:53<67:57:26,  5.71s/it]
 81%|████████  | 182139/225000 [344:56:57<61:32:23,  5.17s/it]
 81%|████████  | 182140/225000 [344:57:05<71:19:12,  5.99s/it]
 81%|████████  | 182141/225000 [344:57:09<67:21:38,  5.66s/it]
 81%|████████  | 182142/225000 [344:57:17<73:12:14,  6.15s/it]
 81%|████████  | 182143/225000 [344:57:20<62:22:52,  5.24s/it]
 81%|████████  | 182144/225000 [344:57:25<62:21:03,  5.24s/it]
 81%|████████  | 182145/225000 [344:57:30<61:18:29,  5.15s/it]
 81%|████████  | 182146/225000 [344:57:35<62:30:54,  5.25s/it]
 81%|████████  | 182147/225000 [344:57:41<64:02:48,  5.38s/it]
 81%|████████  | 182148/225000 [344:57:47<64:51:43,  5.45s/it]
 81%|████████  | 182149/225000 [344:57:52<64:14:04,  5.40s/it]
 81%|████████  | 182150/225000 [344:57:57<63:31:50,  5.34s/it]
                                                              
{'loss': '0.7605', 'grad_norm': '1.194', 'learning_rate': '4.761e-06', 'epoch': '0.8096', 'num_input_tokens_seen': 572141888, 'train_runtime': '1.242e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182150/225000 [344:57:57<63:31:50,  5.34s/it]
 81%|████████  | 182151/225000 [344:58:03<63:18:46,  5.32s/it]
 81%|████████  | 182152/225000 [344:58:08<63:40:52,  5.35s/it]
 81%|████████  | 182153/225000 [344:58:13<62:17:42,  5.23s/it]
 81%|████████  | 182154/225000 [344:58:18<60:42:31,  5.10s/it]
 81%|████████  | 182155/225000 [344:58:23<60:26:50,  5.08s/it]
 81%|████████  | 182156/225000 [344:58:28<59:30:34,  5.00s/it]
 81%|████████  | 182157/225000 [344:58:33<60:02:26,  5.05s/it]
 81%|████████  | 182158/225000 [344:58:38<61:04:06,  5.13s/it]
 81%|████████  | 182159/225000 [344:58:43<61:07:26,  5.14s/it]
 81%|████████  | 182160/225000 [344:58:48<61:33:16,  5.17s/it]
 81%|████████  | 182161/225000 [344:58:53<59:15:57,  4.98s/it]
 81%|████████  | 182162/225000 [344:58:58<59:48:23,  5.03s/it]
 81%|████████  | 182163/225000 [344:59:04<61:21:11,  5.16s/it]
 81%|████████  | 182164/225000 [344:59:08<60:14:57,  5.06s/it]
 81%|████████  | 182165/225000 [344:59:14<61:30:19,  5.17s/it]
 81%|████████  | 182166/225000 [344:59:23<76:24:32,  6.42s/it]
 81%|████████  | 182167/225000 [344:59:35<94:15:26,  7.92s/it]
 81%|████████  | 182168/225000 [344:59:38<78:31:24,  6.60s/it]
 81%|████████  | 182169/225000 [344:59:42<69:16:45,  5.82s/it]
 81%|████████  | 182170/225000 [344:59:47<66:47:17,  5.61s/it]
 81%|████████  | 182171/225000 [344:59:52<62:48:35,  5.28s/it]
 81%|████████  | 182172/225000 [344:59:57<61:03:45,  5.13s/it]
 81%|████████  | 182173/225000 [345:00:01<58:51:29,  4.95s/it]
 81%|████████  | 182174/225000 [345:00:05<57:01:01,  4.79s/it]
 81%|████████  | 182175/225000 [345:00:10<55:44:56,  4.69s/it]
 81%|████████  | 182176/225000 [345:00:15<55:43:31,  4.68s/it]
 81%|████████  | 182177/225000 [345:00:21<61:11:08,  5.14s/it]
 81%|████████  | 182178/225000 [345:00:33<86:20:33,  7.26s/it]
 81%|████████  | 182179/225000 [345:00:36<71:57:27,  6.05s/it]
 81%|████████  | 182180/225000 [345:00:40<62:34:56,  5.26s/it]
 81%|████████  | 182181/225000 [345:00:44<60:03:58,  5.05s/it]
 81%|████████  | 182182/225000 [345:00:54<75:30:01,  6.35s/it]
 81%|████████  | 182183/225000 [345:00:57<64:37:13,  5.43s/it]
 81%|████████  | 182184/225000 [345:01:05<73:39:00,  6.19s/it]
 81%|████████  | 182185/225000 [345:01:09<64:51:27,  5.45s/it]
 81%|████████  | 182186/225000 [345:01:17<76:15:18,  6.41s/it]
 81%|████████  | 182187/225000 [345:01:22<69:04:59,  5.81s/it]
 81%|████████  | 182188/225000 [345:01:25<61:38:19,  5.18s/it]
 81%|████████  | 182189/225000 [345:01:33<72:11:14,  6.07s/it]
 81%|████████  | 182190/225000 [345:01:38<66:02:29,  5.55s/it]
 81%|████████  | 182191/225000 [345:01:42<62:11:35,  5.23s/it]
 81%|████████  | 182192/225000 [345:01:47<59:20:50,  4.99s/it]
 81%|████████  | 182193/225000 [345:01:51<55:09:19,  4.64s/it]
 81%|████████  | 182194/225000 [345:01:55<54:14:44,  4.56s/it]
 81%|████████  | 182195/225000 [345:02:00<54:57:22,  4.62s/it]
 81%|████████  | 182196/225000 [345:02:12<81:51:05,  6.88s/it]
 81%|████████  | 182197/225000 [345:02:16<72:25:59,  6.09s/it]
 81%|████████  | 182198/225000 [345:02:20<62:55:46,  5.29s/it]
 81%|████████  | 182199/225000 [345:02:28<72:52:43,  6.13s/it]
 81%|████████  | 182200/225000 [345:02:31<63:49:13,  5.37s/it]
                                                              
{'loss': '0.7565', 'grad_norm': '1.131', 'learning_rate': '4.75e-06', 'epoch': '0.8098', 'num_input_tokens_seen': 572270440, 'train_runtime': '1.242e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182200/225000 [345:02:31<63:49:13,  5.37s/it]
 81%|████████  | 182201/225000 [345:02:40<77:09:16,  6.49s/it]
 81%|████████  | 182202/225000 [345:02:43<64:34:16,  5.43s/it]
 81%|████████  | 182203/225000 [345:02:47<56:58:26,  4.79s/it]
 81%|████████  | 182204/225000 [345:02:55<69:02:22,  5.81s/it]
 81%|████████  | 182205/225000 [345:02:59<62:04:36,  5.22s/it]
 81%|████████  | 182206/225000 [345:03:06<71:14:16,  5.99s/it]
 81%|████████  | 182207/225000 [345:03:11<67:13:25,  5.66s/it]
 81%|████████  | 182208/225000 [345:03:18<69:53:17,  5.88s/it]
 81%|████████  | 182209/225000 [345:03:21<61:18:58,  5.16s/it]
 81%|████████  | 182210/225000 [345:03:29<72:39:30,  6.11s/it]
 81%|████████  | 182211/225000 [345:03:34<67:06:12,  5.65s/it]
 81%|████████  | 182212/225000 [345:03:40<69:53:18,  5.88s/it]
 81%|████████  | 182213/225000 [345:03:46<69:43:25,  5.87s/it]
 81%|████████  | 182214/225000 [345:03:52<68:23:49,  5.75s/it]
 81%|████████  | 182215/225000 [345:03:57<66:21:22,  5.58s/it]
 81%|████████  | 182216/225000 [345:04:02<65:32:00,  5.51s/it]
 81%|████████  | 182217/225000 [345:04:08<64:44:08,  5.45s/it]
 81%|████████  | 182218/225000 [345:04:13<63:03:46,  5.31s/it]
 81%|████████  | 182219/225000 [345:04:18<62:35:53,  5.27s/it]
 81%|████████  | 182220/225000 [345:04:23<62:02:26,  5.22s/it]
 81%|████████  | 182221/225000 [345:04:28<62:53:01,  5.29s/it]
 81%|████████  | 182222/225000 [345:04:34<63:16:01,  5.32s/it]
 81%|████████  | 182223/225000 [345:04:39<63:43:51,  5.36s/it]
 81%|████████  | 182224/225000 [345:04:45<63:55:22,  5.38s/it]
 81%|████████  | 182225/225000 [345:04:50<62:51:06,  5.29s/it]
 81%|████████  | 182226/225000 [345:04:55<61:56:01,  5.21s/it]
 81%|████████  | 182227/225000 [345:05:00<63:09:47,  5.32s/it]
 81%|████████  | 182228/225000 [345:05:06<63:10:43,  5.32s/it]
 81%|████████  | 182229/225000 [345:05:11<64:54:17,  5.46s/it]
 81%|████████  | 182230/225000 [345:05:17<65:41:23,  5.53s/it]
 81%|████████  | 182231/225000 [345:05:22<64:02:29,  5.39s/it]
 81%|████████  | 182232/225000 [345:05:28<65:13:20,  5.49s/it]
 81%|████████  | 182233/225000 [345:05:33<65:22:43,  5.50s/it]
 81%|████████  | 182234/225000 [345:05:39<64:39:35,  5.44s/it]
 81%|████████  | 182235/225000 [345:05:44<62:30:14,  5.26s/it]
 81%|████████  | 182236/225000 [345:05:49<62:07:19,  5.23s/it]
 81%|████████  | 182237/225000 [345:05:54<62:39:46,  5.28s/it]
 81%|████████  | 182238/225000 [345:05:59<60:52:49,  5.13s/it]
 81%|████████  | 182239/225000 [345:06:04<61:45:27,  5.20s/it]
 81%|████████  | 182240/225000 [345:06:09<61:41:23,  5.19s/it]
 81%|████████  | 182241/225000 [345:06:15<61:41:21,  5.19s/it]
 81%|████████  | 182242/225000 [345:06:20<61:18:14,  5.16s/it]
 81%|████████  | 182243/225000 [345:06:25<62:38:28,  5.27s/it]
 81%|████████  | 182244/225000 [345:06:34<75:43:06,  6.38s/it]
 81%|████████  | 182245/225000 [345:06:39<70:53:17,  5.97s/it]
 81%|████████  | 182246/225000 [345:06:45<68:55:27,  5.80s/it]
 81%|████████  | 182247/225000 [345:06:50<66:09:11,  5.57s/it]
 81%|████████  | 182248/225000 [345:06:55<64:17:29,  5.41s/it]
 81%|████████  | 182249/225000 [345:07:00<64:19:02,  5.42s/it]
 81%|████████  | 182250/225000 [345:07:06<64:50:58,  5.46s/it]
                                                              
{'loss': '0.7681', 'grad_norm': '1.117', 'learning_rate': '4.739e-06', 'epoch': '0.81', 'num_input_tokens_seen': 572401288, 'train_runtime': '1.242e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182250/225000 [345:07:06<64:50:58,  5.46s/it]
 81%|████████  | 182251/225000 [345:07:11<63:42:58,  5.37s/it]
 81%|████████  | 182252/225000 [345:07:16<61:59:53,  5.22s/it]
 81%|████████  | 182253/225000 [345:07:21<63:01:37,  5.31s/it]
 81%|████████  | 182254/225000 [345:07:31<78:33:34,  6.62s/it]
 81%|████████  | 182255/225000 [345:07:39<83:51:54,  7.06s/it]
 81%|████████  | 182256/225000 [345:07:42<70:50:21,  5.97s/it]
 81%|████████  | 182257/225000 [345:07:47<65:11:49,  5.49s/it]
 81%|████████  | 182258/225000 [345:07:51<61:27:36,  5.18s/it]
 81%|████████  | 182259/225000 [345:08:03<84:52:52,  7.15s/it]
 81%|████████  | 182260/225000 [345:08:07<73:57:23,  6.23s/it]
 81%|████████  | 182261/225000 [345:08:11<66:33:40,  5.61s/it]
 81%|████████  | 182262/225000 [345:08:16<62:25:19,  5.26s/it]
 81%|████████  | 182263/225000 [345:08:21<61:53:41,  5.21s/it]
 81%|████████  | 182264/225000 [345:08:33<85:23:52,  7.19s/it]
 81%|████████  | 182265/225000 [345:08:36<73:16:59,  6.17s/it]
 81%|████████  | 182266/225000 [345:08:40<64:24:53,  5.43s/it]
 81%|████████  | 182267/225000 [345:08:43<57:17:39,  4.83s/it]
 81%|████████  | 182268/225000 [345:08:54<76:41:21,  6.46s/it]
 81%|████████  | 182269/225000 [345:08:58<67:32:36,  5.69s/it]
 81%|████████  | 182270/225000 [345:09:03<66:25:33,  5.60s/it]
 81%|████████  | 182271/225000 [345:09:10<71:38:38,  6.04s/it]
 81%|████████  | 182272/225000 [345:09:14<62:38:42,  5.28s/it]
 81%|████████  | 182273/225000 [345:09:22<72:08:59,  6.08s/it]
 81%|████████  | 182274/225000 [345:09:26<64:53:14,  5.47s/it]
 81%|████████  | 182275/225000 [345:09:30<59:35:46,  5.02s/it]
 81%|████████  | 182276/225000 [345:09:34<55:45:58,  4.70s/it]
 81%|████████  | 182277/225000 [345:09:44<76:06:54,  6.41s/it]
 81%|████████  | 182278/225000 [345:09:47<64:53:44,  5.47s/it]
 81%|████████  | 182279/225000 [345:09:55<73:15:58,  6.17s/it]
 81%|████████  | 182280/225000 [345:09:59<65:35:19,  5.53s/it]
 81%|████████  | 182281/225000 [345:10:03<59:59:26,  5.06s/it]
 81%|████████  | 182282/225000 [345:10:07<54:54:12,  4.63s/it]
 81%|████████  | 182283/225000 [345:10:16<71:48:14,  6.05s/it]
 81%|████████  | 182284/225000 [345:10:22<70:46:45,  5.97s/it]
 81%|████████  | 182285/225000 [345:10:25<60:56:13,  5.14s/it]
 81%|████████  | 182286/225000 [345:10:33<73:03:36,  6.16s/it]
 81%|████████  | 182287/225000 [345:10:40<72:36:57,  6.12s/it]
 81%|████████  | 182288/225000 [345:10:44<65:04:27,  5.48s/it]
 81%|████████  | 182289/225000 [345:10:51<71:55:41,  6.06s/it]
 81%|████████  | 182290/225000 [345:10:54<62:38:05,  5.28s/it]
 81%|████████  | 182291/225000 [345:10:58<56:29:58,  4.76s/it]
 81%|████████  | 182292/225000 [345:11:07<72:44:28,  6.13s/it]
 81%|████████  | 182293/225000 [345:11:14<73:53:50,  6.23s/it]
 81%|████████  | 182294/225000 [345:11:18<65:22:07,  5.51s/it]
 81%|████████  | 182295/225000 [345:11:21<58:16:56,  4.91s/it]
 81%|████████  | 182296/225000 [345:11:26<57:41:06,  4.86s/it]
 81%|████████  | 182297/225000 [345:11:35<73:03:00,  6.16s/it]
 81%|████████  | 182298/225000 [345:11:38<63:09:23,  5.32s/it]
 81%|████████  | 182299/225000 [345:11:44<62:39:44,  5.28s/it]
 81%|████████  | 182300/225000 [345:11:49<62:31:57,  5.27s/it]
                                                              
{'loss': '0.7592', 'grad_norm': '1.14', 'learning_rate': '4.729e-06', 'epoch': '0.8102', 'num_input_tokens_seen': 572531456, 'train_runtime': '1.243e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182300/225000 [345:11:49<62:31:57,  5.27s/it]
 81%|████████  | 182301/225000 [345:11:52<56:17:33,  4.75s/it]
 81%|████████  | 182302/225000 [345:11:59<62:51:06,  5.30s/it]
 81%|████████  | 182303/225000 [345:12:05<66:57:36,  5.65s/it]
 81%|████████  | 182304/225000 [345:12:09<60:33:40,  5.11s/it]
 81%|████████  | 182305/225000 [345:12:13<56:43:51,  4.78s/it]
 81%|████████  | 182306/225000 [345:12:24<77:05:59,  6.50s/it]
 81%|████████  | 182307/225000 [345:12:27<65:10:16,  5.50s/it]
 81%|████████  | 182308/225000 [345:12:33<68:06:08,  5.74s/it]
 81%|████████  | 182309/225000 [345:12:40<70:20:33,  5.93s/it]
 81%|████████  | 182310/225000 [345:12:44<65:03:56,  5.49s/it]
 81%|████████  | 182311/225000 [345:12:48<61:16:03,  5.17s/it]
 81%|████████  | 182312/225000 [345:12:53<59:41:25,  5.03s/it]
 81%|████████  | 182313/225000 [345:12:58<59:34:16,  5.02s/it]
 81%|████████  | 182314/225000 [345:13:03<58:19:26,  4.92s/it]
 81%|████████  | 182315/225000 [345:13:14<81:40:07,  6.89s/it]
 81%|████████  | 182316/225000 [345:13:19<72:31:46,  6.12s/it]
 81%|████████  | 182317/225000 [345:13:23<67:47:16,  5.72s/it]
 81%|████████  | 182318/225000 [345:13:28<63:45:08,  5.38s/it]
 81%|████████  | 182319/225000 [345:13:33<60:46:17,  5.13s/it]
 81%|████████  | 182320/225000 [345:13:37<58:26:56,  4.93s/it]
 81%|████████  | 182321/225000 [345:13:42<57:04:41,  4.81s/it]
 81%|████████  | 182322/225000 [345:13:46<56:07:49,  4.73s/it]
 81%|████████  | 182323/225000 [345:13:51<55:03:56,  4.65s/it]
 81%|████████  | 182324/225000 [345:13:57<61:55:02,  5.22s/it]
 81%|████████  | 182325/225000 [345:14:09<85:36:37,  7.22s/it]
 81%|████████  | 182326/225000 [345:14:13<72:38:54,  6.13s/it]
 81%|████████  | 182327/225000 [345:14:16<63:45:23,  5.38s/it]
 81%|████████  | 182328/225000 [345:14:21<60:28:24,  5.10s/it]
 81%|████████  | 182329/225000 [345:14:25<56:40:25,  4.78s/it]
 81%|████████  | 182330/225000 [345:14:29<56:25:42,  4.76s/it]
 81%|████████  | 182331/225000 [345:14:34<55:43:29,  4.70s/it]
 81%|████████  | 182332/225000 [345:14:45<78:29:53,  6.62s/it]
 81%|████████  | 182333/225000 [345:14:50<73:04:58,  6.17s/it]
 81%|████████  | 182334/225000 [345:14:55<68:01:20,  5.74s/it]
 81%|████████  | 182335/225000 [345:15:00<65:12:38,  5.50s/it]
 81%|████████  | 182336/225000 [345:15:05<62:37:07,  5.28s/it]
 81%|████████  | 182337/225000 [345:15:10<61:11:55,  5.16s/it]
 81%|████████  | 182338/225000 [345:15:14<60:10:52,  5.08s/it]
 81%|████████  | 182339/225000 [345:15:19<58:54:56,  4.97s/it]
 81%|████████  | 182340/225000 [345:15:25<62:58:10,  5.31s/it]
 81%|████████  | 182341/225000 [345:15:38<88:27:27,  7.46s/it]
 81%|████████  | 182342/225000 [345:15:42<77:30:37,  6.54s/it]
 81%|████████  | 182343/225000 [345:15:48<76:20:17,  6.44s/it]
 81%|████████  | 182344/225000 [345:15:53<69:42:53,  5.88s/it]
 81%|████████  | 182345/225000 [345:15:58<66:14:32,  5.59s/it]
 81%|████████  | 182346/225000 [345:16:03<64:13:38,  5.42s/it]
 81%|████████  | 182347/225000 [345:16:09<65:02:54,  5.49s/it]
 81%|████████  | 182348/225000 [345:16:14<63:53:15,  5.39s/it]
 81%|████████  | 182349/225000 [345:16:19<62:22:00,  5.26s/it]
 81%|████████  | 182350/225000 [345:16:24<62:10:44,  5.25s/it]
                                                              
{'loss': '0.7708', 'grad_norm': '1.142', 'learning_rate': '4.718e-06', 'epoch': '0.8104', 'num_input_tokens_seen': 572661544, 'train_runtime': '1.243e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182350/225000 [345:16:24<62:10:44,  5.25s/it]
 81%|████████  | 182351/225000 [345:16:29<61:42:44,  5.21s/it]
 81%|████████  | 182352/225000 [345:16:33<58:55:01,  4.97s/it]
 81%|████████  | 182353/225000 [345:16:38<58:44:39,  4.96s/it]
 81%|████████  | 182354/225000 [345:16:44<59:35:01,  5.03s/it]
 81%|████████  | 182355/225000 [345:16:49<59:30:01,  5.02s/it]
 81%|████████  | 182356/225000 [345:17:01<86:09:02,  7.27s/it]
 81%|████████  | 182357/225000 [345:17:07<81:33:04,  6.88s/it]
 81%|████████  | 182358/225000 [345:17:11<70:20:13,  5.94s/it]
 81%|████████  | 182359/225000 [345:17:15<62:46:00,  5.30s/it]
 81%|████████  | 182360/225000 [345:17:23<74:02:59,  6.25s/it]
 81%|████████  | 182361/225000 [345:17:27<65:24:05,  5.52s/it]
 81%|████████  | 182362/225000 [345:17:34<72:36:01,  6.13s/it]
 81%|████████  | 182363/225000 [345:17:38<63:04:26,  5.33s/it]
 81%|████████  | 182364/225000 [345:17:45<69:12:12,  5.84s/it]
 81%|████████  | 182365/225000 [345:17:48<61:05:44,  5.16s/it]
 81%|████████  | 182366/225000 [345:17:57<72:45:51,  6.14s/it]
 81%|████████  | 182367/225000 [345:18:00<62:49:18,  5.30s/it]
 81%|████████  | 182368/225000 [345:18:08<72:17:37,  6.10s/it]
 81%|████████  | 182369/225000 [345:18:12<62:52:12,  5.31s/it]
 81%|████████  | 182370/225000 [345:18:16<58:12:48,  4.92s/it]
 81%|████████  | 182371/225000 [345:18:20<54:50:54,  4.63s/it]
 81%|████████  | 182372/225000 [345:18:29<71:40:23,  6.05s/it]
 81%|████████  | 182373/225000 [345:18:33<64:24:56,  5.44s/it]
 81%|████████  | 182374/225000 [345:18:37<58:26:33,  4.94s/it]
 81%|████████  | 182375/225000 [345:18:42<60:12:33,  5.09s/it]
 81%|████████  | 182376/225000 [345:18:48<60:58:13,  5.15s/it]
 81%|████████  | 182377/225000 [345:18:52<60:12:31,  5.09s/it]
 81%|████████  | 182378/225000 [345:18:57<59:52:09,  5.06s/it]
 81%|████████  | 182379/225000 [345:19:03<60:51:24,  5.14s/it]
 81%|████████  | 182380/225000 [345:19:08<61:06:23,  5.16s/it]
 81%|████████  | 182381/225000 [345:19:14<62:32:24,  5.28s/it]
 81%|████████  | 182382/225000 [345:19:19<61:48:45,  5.22s/it]
 81%|████████  | 182383/225000 [345:19:24<62:52:22,  5.31s/it]
 81%|████████  | 182384/225000 [345:19:30<63:03:12,  5.33s/it]
 81%|████████  | 182385/225000 [345:19:35<63:54:36,  5.40s/it]
 81%|████████  | 182386/225000 [345:19:40<62:18:04,  5.26s/it]
 81%|████████  | 182387/225000 [345:19:48<73:29:44,  6.21s/it]
 81%|████████  | 182388/225000 [345:19:53<68:16:50,  5.77s/it]
 81%|████████  | 182389/225000 [345:19:59<66:43:25,  5.64s/it]
 81%|████████  | 182390/225000 [345:20:04<66:01:10,  5.58s/it]
 81%|████████  | 182391/225000 [345:20:09<64:56:51,  5.49s/it]
 81%|████████  | 182392/225000 [345:20:15<65:10:06,  5.51s/it]
 81%|████████  | 182393/225000 [345:20:20<64:54:58,  5.48s/it]
 81%|████████  | 182394/225000 [345:20:27<68:39:54,  5.80s/it]
 81%|████████  | 182395/225000 [345:20:32<66:42:00,  5.64s/it]
 81%|████████  | 182396/225000 [345:20:37<64:33:15,  5.45s/it]
 81%|████████  | 182397/225000 [345:20:42<64:06:53,  5.42s/it]
 81%|████████  | 182398/225000 [345:20:48<63:03:40,  5.33s/it]
 81%|████████  | 182399/225000 [345:20:53<62:34:27,  5.29s/it]
 81%|████████  | 182400/225000 [345:20:58<62:50:14,  5.31s/it]
                                                              
{'loss': '0.758', 'grad_norm': '1.208', 'learning_rate': '4.707e-06', 'epoch': '0.8107', 'num_input_tokens_seen': 572790152, 'train_runtime': '1.243e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182400/225000 [345:20:58<62:50:14,  5.31s/it]
 81%|████████  | 182401/225000 [345:21:03<62:01:41,  5.24s/it]
 81%|████████  | 182402/225000 [345:21:12<74:45:00,  6.32s/it]
 81%|████████  | 182403/225000 [345:21:16<66:25:36,  5.61s/it]
 81%|████████  | 182404/225000 [345:21:20<61:50:16,  5.23s/it]
 81%|████████  | 182405/225000 [345:21:25<60:23:35,  5.10s/it]
 81%|████████  | 182406/225000 [345:21:35<76:56:05,  6.50s/it]
 81%|████████  | 182407/225000 [345:21:38<65:54:33,  5.57s/it]
 81%|████████  | 182408/225000 [345:21:43<61:42:16,  5.22s/it]
 81%|████████  | 182409/225000 [345:21:51<71:43:46,  6.06s/it]
 81%|████████  | 182410/225000 [345:21:55<64:45:39,  5.47s/it]
 81%|████████  | 182411/225000 [345:21:59<59:59:00,  5.07s/it]
 81%|████████  | 182412/225000 [345:22:03<56:22:10,  4.76s/it]
 81%|████████  | 182413/225000 [345:22:07<52:48:31,  4.46s/it]
 81%|████████  | 182414/225000 [345:22:18<75:18:58,  6.37s/it]
 81%|████████  | 182415/225000 [345:22:21<66:24:22,  5.61s/it]
 81%|████████  | 182416/225000 [345:22:26<61:21:46,  5.19s/it]
 81%|████████  | 182417/225000 [345:22:34<74:25:31,  6.29s/it]
 81%|████████  | 182418/225000 [345:22:40<71:32:52,  6.05s/it]
 81%|████████  | 182419/225000 [345:22:44<65:40:17,  5.55s/it]
 81%|████████  | 182420/225000 [345:22:49<61:14:26,  5.18s/it]
 81%|████████  | 182421/225000 [345:22:53<56:49:00,  4.80s/it]
 81%|████████  | 182422/225000 [345:22:57<55:25:57,  4.69s/it]
 81%|████████  | 182423/225000 [345:23:06<71:54:53,  6.08s/it]
 81%|████████  | 182424/225000 [345:23:11<66:32:38,  5.63s/it]
 81%|████████  | 182425/225000 [345:23:18<70:17:49,  5.94s/it]
 81%|████████  | 182426/225000 [345:23:22<66:13:46,  5.60s/it]
 81%|████████  | 182427/225000 [345:23:27<61:46:18,  5.22s/it]
 81%|████████  | 182428/225000 [345:23:32<61:00:06,  5.16s/it]
 81%|████████  | 182429/225000 [345:23:37<60:10:35,  5.09s/it]
 81%|████████  | 182430/225000 [345:23:41<58:09:42,  4.92s/it]
 81%|████████  | 182431/225000 [345:23:46<56:53:48,  4.81s/it]
 81%|████████  | 182432/225000 [345:23:51<57:19:16,  4.85s/it]
 81%|████████  | 182433/225000 [345:23:55<56:47:56,  4.80s/it]
 81%|████████  | 182434/225000 [345:24:04<70:34:39,  5.97s/it]
 81%|████████  | 182435/225000 [345:24:13<80:06:47,  6.78s/it]
 81%|████████  | 182436/225000 [345:24:16<68:06:43,  5.76s/it]
 81%|████████  | 182437/225000 [345:24:24<75:12:34,  6.36s/it]
 81%|████████  | 182438/225000 [345:24:29<70:35:34,  5.97s/it]
 81%|████████  | 182439/225000 [345:24:33<63:45:40,  5.39s/it]
 81%|████████  | 182440/225000 [345:24:37<60:31:58,  5.12s/it]
 81%|████████  | 182441/225000 [345:24:42<58:59:39,  4.99s/it]
 81%|████████  | 182442/225000 [345:24:47<58:12:01,  4.92s/it]
 81%|████████  | 182443/225000 [345:24:51<56:39:47,  4.79s/it]
 81%|████████  | 182444/225000 [345:24:56<57:00:51,  4.82s/it]
 81%|████████  | 182445/225000 [345:25:07<76:17:48,  6.45s/it]
 81%|████████  | 182446/225000 [345:25:13<77:40:49,  6.57s/it]
 81%|████████  | 182447/225000 [345:25:17<66:41:37,  5.64s/it]
 81%|████████  | 182448/225000 [345:25:24<71:24:45,  6.04s/it]
 81%|████████  | 182449/225000 [345:25:28<63:54:11,  5.41s/it]
 81%|████████  | 182450/225000 [345:25:36<75:22:26,  6.38s/it]
                                                              
{'loss': '0.7534', 'grad_norm': '1.284', 'learning_rate': '4.696e-06', 'epoch': '0.8109', 'num_input_tokens_seen': 572920272, 'train_runtime': '1.244e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182450/225000 [345:25:36<75:22:26,  6.38s/it]
 81%|████████  | 182451/225000 [345:25:40<65:11:35,  5.52s/it]
 81%|████████  | 182452/225000 [345:25:44<58:30:48,  4.95s/it]
 81%|████████  | 182453/225000 [345:25:52<71:01:28,  6.01s/it]
 81%|████████  | 182454/225000 [345:25:57<65:35:26,  5.55s/it]
 81%|████████  | 182455/225000 [345:26:01<60:50:09,  5.15s/it]
 81%|████████  | 182456/225000 [345:26:05<59:07:08,  5.00s/it]
 81%|████████  | 182457/225000 [345:26:10<58:38:57,  4.96s/it]
 81%|████████  | 182458/225000 [345:26:15<57:20:04,  4.85s/it]
 81%|████████  | 182459/225000 [345:26:19<55:54:04,  4.73s/it]
 81%|████████  | 182460/225000 [345:26:24<54:55:00,  4.65s/it]
 81%|████████  | 182461/225000 [345:26:34<76:33:10,  6.48s/it]
 81%|████████  | 182462/225000 [345:26:42<80:23:44,  6.80s/it]
 81%|████████  | 182463/225000 [345:26:48<76:40:21,  6.49s/it]
 81%|████████  | 182464/225000 [345:26:51<65:04:33,  5.51s/it]
 81%|████████  | 182465/225000 [345:26:59<73:58:33,  6.26s/it]
 81%|████████  | 182466/225000 [345:27:05<73:24:57,  6.21s/it]
 81%|████████  | 182467/225000 [345:27:09<65:34:14,  5.55s/it]
 81%|████████  | 182468/225000 [345:27:13<59:18:21,  5.02s/it]
 81%|████████  | 182469/225000 [345:27:17<56:20:24,  4.77s/it]
 81%|████████  | 182470/225000 [345:27:27<72:58:54,  6.18s/it]
 81%|████████  | 182471/225000 [345:27:30<63:36:17,  5.38s/it]
 81%|████████  | 182472/225000 [345:27:37<69:05:37,  5.85s/it]
 81%|████████  | 182473/225000 [345:27:41<61:21:04,  5.19s/it]
 81%|████████  | 182474/225000 [345:27:46<60:08:52,  5.09s/it]
 81%|████████  | 182475/225000 [345:27:51<61:23:48,  5.20s/it]
 81%|████████  | 182476/225000 [345:27:56<61:08:05,  5.18s/it]
 81%|████████  | 182477/225000 [345:28:01<61:29:03,  5.21s/it]
 81%|████████  | 182478/225000 [345:28:07<62:40:19,  5.31s/it]
 81%|████████  | 182479/225000 [345:28:12<62:47:48,  5.32s/it]
 81%|████████  | 182480/225000 [345:28:17<62:18:08,  5.27s/it]
 81%|████████  | 182481/225000 [345:28:22<60:21:57,  5.11s/it]
 81%|████████  | 182482/225000 [345:28:27<58:56:21,  4.99s/it]
 81%|████████  | 182483/225000 [345:28:37<76:42:34,  6.50s/it]
 81%|████████  | 182484/225000 [345:28:41<68:22:23,  5.79s/it]
 81%|████████  | 182485/225000 [345:28:46<63:40:38,  5.39s/it]
 81%|████████  | 182486/225000 [345:28:50<59:58:28,  5.08s/it]
 81%|████████  | 182487/225000 [345:28:55<60:23:15,  5.11s/it]
 81%|████████  | 182488/225000 [345:29:08<86:23:28,  7.32s/it]
 81%|████████  | 182489/225000 [345:29:11<72:12:58,  6.12s/it]
 81%|████████  | 182490/225000 [345:29:19<79:37:53,  6.74s/it]
 81%|████████  | 182491/225000 [345:29:23<71:21:52,  6.04s/it]
 81%|████████  | 182492/225000 [345:29:27<64:12:47,  5.44s/it]
 81%|████████  | 182493/225000 [345:29:32<60:32:23,  5.13s/it]
 81%|████████  | 182494/225000 [345:29:36<56:19:25,  4.77s/it]
 81%|████████  | 182495/225000 [345:29:40<54:30:06,  4.62s/it]
 81%|████████  | 182496/225000 [345:29:52<81:03:06,  6.86s/it]
 81%|████████  | 182497/225000 [345:29:56<70:31:34,  5.97s/it]
 81%|████████  | 182498/225000 [345:30:00<62:03:03,  5.26s/it]
 81%|████████  | 182499/225000 [345:30:08<72:54:18,  6.18s/it]
 81%|████████  | 182500/225000 [345:30:12<65:53:22,  5.58s/it]
                                                              
{'loss': '0.7532', 'grad_norm': '1.166', 'learning_rate': '4.686e-06', 'epoch': '0.8111', 'num_input_tokens_seen': 573050296, 'train_runtime': '1.244e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182500/225000 [345:30:12<65:53:22,  5.58s/it]
 81%|████████  | 182501/225000 [345:30:16<59:36:38,  5.05s/it]
 81%|████████  | 182502/225000 [345:30:20<56:20:37,  4.77s/it]
 81%|████████  | 182503/225000 [345:30:31<78:09:40,  6.62s/it]
 81%|████████  | 182504/225000 [345:30:35<68:35:43,  5.81s/it]
 81%|████████  | 182505/225000 [345:30:39<62:26:03,  5.29s/it]
 81%|████████  | 182506/225000 [345:30:43<58:13:23,  4.93s/it]
 81%|████████  | 182507/225000 [345:30:47<54:35:31,  4.63s/it]
 81%|████████  | 182508/225000 [345:30:58<76:08:05,  6.45s/it]
 81%|████████  | 182509/225000 [345:31:02<67:11:20,  5.69s/it]
 81%|████████  | 182510/225000 [345:31:07<66:07:49,  5.60s/it]
 81%|████████  | 182511/225000 [345:31:12<65:02:23,  5.51s/it]
 81%|████████  | 182512/225000 [345:31:16<59:36:34,  5.05s/it]
 81%|████████  | 182513/225000 [345:31:21<59:11:37,  5.02s/it]
 81%|████████  | 182514/225000 [345:31:26<59:11:43,  5.02s/it]
 81%|████████  | 182515/225000 [345:31:32<60:17:42,  5.11s/it]
 81%|████████  | 182516/225000 [345:31:37<60:25:16,  5.12s/it]
 81%|████████  | 182517/225000 [345:31:42<60:49:30,  5.15s/it]
 81%|████████  | 182518/225000 [345:31:47<60:21:39,  5.12s/it]
 81%|████████  | 182519/225000 [345:31:53<62:04:06,  5.26s/it]
 81%|████████  | 182520/225000 [345:31:58<62:48:53,  5.32s/it]
 81%|████████  | 182521/225000 [345:32:03<62:47:57,  5.32s/it]
 81%|████████  | 182522/225000 [345:32:09<63:40:42,  5.40s/it]
 81%|████████  | 182523/225000 [345:32:14<63:04:13,  5.35s/it]
 81%|████████  | 182524/225000 [345:32:25<82:43:18,  7.01s/it]
 81%|████████  | 182525/225000 [345:32:30<74:58:28,  6.35s/it]
 81%|████████  | 182526/225000 [345:32:36<73:00:08,  6.19s/it]
 81%|████████  | 182527/225000 [345:32:41<68:31:07,  5.81s/it]
 81%|████████  | 182528/225000 [345:32:49<77:07:21,  6.54s/it]
 81%|████████  | 182529/225000 [345:32:52<66:27:44,  5.63s/it]
 81%|████████  | 182530/225000 [345:32:57<61:09:29,  5.18s/it]
 81%|████████  | 182531/225000 [345:33:01<58:59:24,  5.00s/it]
 81%|████████  | 182532/225000 [345:33:06<57:19:27,  4.86s/it]
 81%|████████  | 182533/225000 [345:33:10<56:50:56,  4.82s/it]
 81%|████████  | 182534/225000 [345:33:15<56:05:14,  4.75s/it]
 81%|████████  | 182535/225000 [345:33:20<56:20:20,  4.78s/it]
 81%|████████  | 182536/225000 [345:33:33<85:42:24,  7.27s/it]
 81%|████████  | 182537/225000 [345:33:37<75:55:13,  6.44s/it]
 81%|████████  | 182538/225000 [345:33:43<71:42:07,  6.08s/it]
 81%|████████  | 182539/225000 [345:33:46<61:31:28,  5.22s/it]
 81%|████████  | 182540/225000 [345:33:50<59:21:35,  5.03s/it]
 81%|████████  | 182541/225000 [345:33:56<61:25:54,  5.21s/it]
 81%|████████  | 182542/225000 [345:34:01<62:07:54,  5.27s/it]
 81%|████████  | 182543/225000 [345:34:09<71:23:37,  6.05s/it]
 81%|████████  | 182544/225000 [345:34:16<72:36:06,  6.16s/it]
 81%|████████  | 182545/225000 [345:34:22<73:51:28,  6.26s/it]
 81%|████████  | 182546/225000 [345:34:26<64:40:08,  5.48s/it]
 81%|████████  | 182547/225000 [345:34:33<70:45:34,  6.00s/it]
 81%|████████  | 182548/225000 [345:34:39<70:47:18,  6.00s/it]
 81%|████████  | 182549/225000 [345:34:42<61:21:19,  5.20s/it]
 81%|████████  | 182550/225000 [345:34:47<58:41:43,  4.98s/it]
                                                              
{'loss': '0.7667', 'grad_norm': '1.169', 'learning_rate': '4.675e-06', 'epoch': '0.8113', 'num_input_tokens_seen': 573179400, 'train_runtime': '1.244e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182550/225000 [345:34:47<58:41:43,  4.98s/it]
 81%|████████  | 182551/225000 [345:34:52<58:11:40,  4.94s/it]
 81%|████████  | 182552/225000 [345:34:57<59:39:03,  5.06s/it]
 81%|████████  | 182553/225000 [345:35:02<59:53:51,  5.08s/it]
 81%|████████  | 182554/225000 [345:35:08<61:20:05,  5.20s/it]
 81%|████████  | 182555/225000 [345:35:13<61:39:55,  5.23s/it]
 81%|████████  | 182556/225000 [345:35:19<62:57:55,  5.34s/it]
 81%|████████  | 182557/225000 [345:35:23<60:53:38,  5.17s/it]
 81%|████████  | 182558/225000 [345:35:28<59:51:24,  5.08s/it]
 81%|████████  | 182559/225000 [345:35:33<60:17:10,  5.11s/it]
 81%|████████  | 182560/225000 [345:35:39<60:17:32,  5.11s/it]
 81%|████████  | 182561/225000 [345:35:44<62:59:52,  5.34s/it]
 81%|████████  | 182562/225000 [345:35:52<69:47:22,  5.92s/it]
 81%|████████  | 182563/225000 [345:35:57<68:59:43,  5.85s/it]
 81%|████████  | 182564/225000 [345:36:02<66:14:10,  5.62s/it]
 81%|████████  | 182565/225000 [345:36:08<65:19:03,  5.54s/it]
 81%|████████  | 182566/225000 [345:36:13<64:47:23,  5.50s/it]
 81%|████████  | 182567/225000 [345:36:22<77:53:06,  6.61s/it]
 81%|████████  | 182568/225000 [345:36:27<70:54:06,  6.02s/it]
 81%|████████  | 182569/225000 [345:36:31<63:18:36,  5.37s/it]
 81%|████████  | 182570/225000 [345:36:40<75:39:44,  6.42s/it]
 81%|████████  | 182571/225000 [345:36:43<65:12:12,  5.53s/it]
 81%|████████  | 182572/225000 [345:36:47<59:40:44,  5.06s/it]
 81%|████████  | 182573/225000 [345:36:52<58:04:48,  4.93s/it]
 81%|████████  | 182574/225000 [345:36:56<57:00:32,  4.84s/it]
 81%|████████  | 182575/225000 [345:37:01<57:22:19,  4.87s/it]
 81%|████████  | 182576/225000 [345:37:11<74:00:56,  6.28s/it]
 81%|████████  | 182577/225000 [345:37:18<76:29:52,  6.49s/it]
 81%|████████  | 182578/225000 [345:37:26<80:46:08,  6.85s/it]
 81%|████████  | 182579/225000 [345:37:29<67:57:32,  5.77s/it]
 81%|████████  | 182580/225000 [345:37:33<60:50:49,  5.16s/it]
 81%|████████  | 182581/225000 [345:37:40<68:54:50,  5.85s/it]
 81%|████████  | 182582/225000 [345:37:45<66:06:37,  5.61s/it]
 81%|████████  | 182583/225000 [345:37:50<64:07:05,  5.44s/it]
 81%|████████  | 182584/225000 [345:37:55<61:29:08,  5.22s/it]
 81%|████████  | 182585/225000 [345:38:00<59:24:21,  5.04s/it]
 81%|████████  | 182586/225000 [345:38:04<57:33:52,  4.89s/it]
 81%|████████  | 182587/225000 [345:38:09<57:29:38,  4.88s/it]
 81%|████████  | 182588/225000 [345:38:14<57:16:56,  4.86s/it]
 81%|████████  | 182589/225000 [345:38:19<57:54:32,  4.92s/it]
 81%|████████  | 182590/225000 [345:38:23<57:06:04,  4.85s/it]
 81%|████████  | 182591/225000 [345:38:28<57:40:51,  4.90s/it]
 81%|████████  | 182592/225000 [345:38:33<57:28:45,  4.88s/it]
 81%|████████  | 182593/225000 [345:38:38<58:03:00,  4.93s/it]
 81%|████████  | 182594/225000 [345:38:44<59:10:06,  5.02s/it]
 81%|████████  | 182595/225000 [345:38:49<59:08:59,  5.02s/it]
 81%|████████  | 182596/225000 [345:38:59<78:20:38,  6.65s/it]
 81%|████████  | 182597/225000 [345:39:07<81:29:07,  6.92s/it]
 81%|████████  | 182598/225000 [345:39:10<68:20:24,  5.80s/it]
 81%|████████  | 182599/225000 [345:39:17<74:24:57,  6.32s/it]
 81%|████████  | 182600/225000 [345:39:21<64:16:21,  5.46s/it]
                                                              
{'loss': '0.7608', 'grad_norm': '1.236', 'learning_rate': '4.665e-06', 'epoch': '0.8116', 'num_input_tokens_seen': 573308896, 'train_runtime': '1.244e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182600/225000 [345:39:21<64:16:21,  5.46s/it]
 81%|████████  | 182601/225000 [345:39:29<74:11:42,  6.30s/it]
 81%|████████  | 182602/225000 [345:39:33<66:49:40,  5.67s/it]
 81%|████████  | 182603/225000 [345:39:41<72:44:49,  6.18s/it]
 81%|████████  | 182604/225000 [345:39:44<63:00:24,  5.35s/it]
 81%|████████  | 182605/225000 [345:39:48<57:40:35,  4.90s/it]
 81%|████████  | 182606/225000 [345:39:57<70:50:20,  6.02s/it]
 81%|████████  | 182607/225000 [345:40:00<61:47:35,  5.25s/it]
 81%|████████  | 182608/225000 [345:40:04<58:02:06,  4.93s/it]
 81%|████████  | 182609/225000 [345:40:09<57:19:58,  4.87s/it]
 81%|████████  | 182610/225000 [345:40:14<58:14:20,  4.95s/it]
 81%|████████  | 182611/225000 [345:40:19<58:45:30,  4.99s/it]
 81%|████████  | 182612/225000 [345:40:24<59:09:45,  5.02s/it]
 81%|████████  | 182613/225000 [345:40:29<59:45:05,  5.07s/it]
 81%|████████  | 182614/225000 [345:40:34<59:42:57,  5.07s/it]
 81%|████████  | 182615/225000 [345:40:39<59:15:13,  5.03s/it]
 81%|████████  | 182616/225000 [345:40:44<59:28:18,  5.05s/it]
 81%|████████  | 182617/225000 [345:40:56<83:22:25,  7.08s/it]
 81%|████████  | 182618/225000 [345:41:01<76:40:36,  6.51s/it]
 81%|████████  | 182619/225000 [345:41:05<66:46:34,  5.67s/it]
 81%|████████  | 182620/225000 [345:41:13<73:13:41,  6.22s/it]
 81%|████████  | 182621/225000 [345:41:19<74:24:12,  6.32s/it]
 81%|████████  | 182622/225000 [345:41:23<64:24:58,  5.47s/it]
 81%|████████  | 182623/225000 [345:41:31<73:18:24,  6.23s/it]
 81%|████████  | 182624/225000 [345:41:36<68:56:51,  5.86s/it]
 81%|████████  | 182625/225000 [345:41:41<65:07:30,  5.53s/it]
 81%|████████  | 182626/225000 [345:41:46<64:20:19,  5.47s/it]
 81%|████████  | 182627/225000 [345:41:51<63:28:06,  5.39s/it]
 81%|████████  | 182628/225000 [345:41:56<63:12:33,  5.37s/it]
 81%|████████  | 182629/225000 [345:42:01<60:50:13,  5.17s/it]
 81%|████████  | 182630/225000 [345:42:06<61:29:26,  5.22s/it]
 81%|████████  | 182631/225000 [345:42:11<60:18:27,  5.12s/it]
 81%|████████  | 182632/225000 [345:42:17<60:52:24,  5.17s/it]
 81%|████████  | 182633/225000 [345:42:21<59:46:39,  5.08s/it]
 81%|████████  | 182634/225000 [345:42:26<58:25:43,  4.96s/it]
 81%|████████  | 182635/225000 [345:42:32<60:29:19,  5.14s/it]
 81%|████████  | 182636/225000 [345:42:37<59:39:36,  5.07s/it]
 81%|████████  | 182637/225000 [345:42:42<60:43:32,  5.16s/it]
 81%|████████  | 182638/225000 [345:42:47<61:20:04,  5.21s/it]
 81%|████████  | 182639/225000 [345:42:53<61:58:53,  5.27s/it]
 81%|████████  | 182640/225000 [345:42:58<61:36:22,  5.24s/it]
 81%|████████  | 182641/225000 [345:43:05<68:26:05,  5.82s/it]
 81%|████████  | 182642/225000 [345:43:17<89:17:51,  7.59s/it]
 81%|████████  | 182643/225000 [345:43:20<74:48:26,  6.36s/it]
 81%|████████  | 182644/225000 [345:43:24<64:54:57,  5.52s/it]
 81%|████████  | 182645/225000 [345:43:33<78:22:17,  6.66s/it]
 81%|████████  | 182646/225000 [345:43:37<67:33:52,  5.74s/it]
 81%|████████  | 182647/225000 [345:43:46<79:07:19,  6.73s/it]
 81%|████████  | 182648/225000 [345:43:50<69:44:07,  5.93s/it]
 81%|████████  | 182649/225000 [345:43:56<69:03:53,  5.87s/it]
 81%|████████  | 182650/225000 [345:44:00<64:59:34,  5.52s/it]
                                                              
{'loss': '0.7632', 'grad_norm': '1.292', 'learning_rate': '4.654e-06', 'epoch': '0.8118', 'num_input_tokens_seen': 573438904, 'train_runtime': '1.245e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182650/225000 [345:44:00<64:59:34,  5.52s/it]
 81%|████████  | 182651/225000 [345:44:05<61:50:45,  5.26s/it]
 81%|████████  | 182652/225000 [345:44:09<59:25:55,  5.05s/it]
 81%|████████  | 182653/225000 [345:44:14<57:49:33,  4.92s/it]
 81%|████████  | 182654/225000 [345:44:19<58:17:10,  4.96s/it]
 81%|████████  | 182655/225000 [345:44:23<56:10:47,  4.78s/it]
 81%|████████  | 182656/225000 [345:44:28<54:13:37,  4.61s/it]
 81%|████████  | 182657/225000 [345:44:38<75:36:42,  6.43s/it]
 81%|████████  | 182658/225000 [345:44:43<68:35:32,  5.83s/it]
 81%|████████  | 182659/225000 [345:44:47<63:25:04,  5.39s/it]
 81%|████████  | 182660/225000 [345:44:52<59:58:10,  5.10s/it]
 81%|████████  | 182661/225000 [345:44:57<60:03:41,  5.11s/it]
 81%|████████  | 182662/225000 [345:45:02<60:13:13,  5.12s/it]
 81%|████████  | 182663/225000 [345:45:07<60:11:46,  5.12s/it]
 81%|████████  | 182664/225000 [345:45:12<60:33:31,  5.15s/it]
 81%|████████  | 182665/225000 [345:45:17<60:34:31,  5.15s/it]
 81%|████████  | 182666/225000 [345:45:22<59:50:06,  5.09s/it]
 81%|████████  | 182667/225000 [345:45:34<82:29:50,  7.02s/it]
 81%|████████  | 182668/225000 [345:45:40<79:02:32,  6.72s/it]
 81%|████████  | 182669/225000 [345:45:43<67:57:01,  5.78s/it]
 81%|████████  | 182670/225000 [345:45:47<61:20:35,  5.22s/it]
 81%|████████  | 182671/225000 [345:45:52<59:02:45,  5.02s/it]
 81%|████████  | 182672/225000 [345:45:56<56:06:37,  4.77s/it]
 81%|████████  | 182673/225000 [345:46:05<71:19:01,  6.07s/it]
 81%|████████  | 182674/225000 [345:46:13<75:53:26,  6.45s/it]
 81%|████████  | 182675/225000 [345:46:19<74:52:58,  6.37s/it]
 81%|████████  | 182676/225000 [345:46:25<73:16:17,  6.23s/it]
 81%|████████  | 182677/225000 [345:46:29<66:50:09,  5.69s/it]
 81%|████████  | 182678/225000 [345:46:34<64:15:10,  5.47s/it]
 81%|████████  | 182679/225000 [345:46:39<62:54:12,  5.35s/it]
 81%|████████  | 182680/225000 [345:46:44<59:57:58,  5.10s/it]
 81%|████████  | 182681/225000 [345:46:49<59:41:32,  5.08s/it]
 81%|████████  | 182682/225000 [345:46:53<58:19:53,  4.96s/it]
 81%|████████  | 182683/225000 [345:46:58<58:20:35,  4.96s/it]
 81%|████████  | 182684/225000 [345:47:03<57:53:37,  4.93s/it]
 81%|████████  | 182685/225000 [345:47:16<85:36:35,  7.28s/it]
 81%|████████  | 182686/225000 [345:47:20<75:47:47,  6.45s/it]
 81%|████████  | 182687/225000 [345:47:25<69:25:21,  5.91s/it]
 81%|████████  | 182688/225000 [345:47:30<65:52:17,  5.60s/it]
 81%|████████  | 182689/225000 [345:47:35<63:07:07,  5.37s/it]
 81%|████████  | 182690/225000 [345:47:40<61:51:25,  5.26s/it]
 81%|████████  | 182691/225000 [345:47:44<59:22:49,  5.05s/it]
 81%|████████  | 182692/225000 [345:47:49<57:04:22,  4.86s/it]
 81%|████████  | 182693/225000 [345:47:54<57:33:14,  4.90s/it]
 81%|████████  | 182694/225000 [345:47:59<57:09:50,  4.86s/it]
 81%|████████  | 182695/225000 [345:48:03<56:07:10,  4.78s/it]
 81%|████████  | 182696/225000 [345:48:12<69:18:07,  5.90s/it]
 81%|████████  | 182697/225000 [345:48:20<78:20:10,  6.67s/it]
 81%|████████  | 182698/225000 [345:48:24<67:22:11,  5.73s/it]
 81%|████████  | 182699/225000 [345:48:28<63:17:37,  5.39s/it]
 81%|████████  | 182700/225000 [345:48:33<62:13:14,  5.30s/it]
                                                              
{'loss': '0.7632', 'grad_norm': '1.161', 'learning_rate': '4.644e-06', 'epoch': '0.812', 'num_input_tokens_seen': 573569496, 'train_runtime': '1.245e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182700/225000 [345:48:33<62:13:14,  5.30s/it]
 81%|████████  | 182701/225000 [345:48:38<60:36:40,  5.16s/it]
 81%|████████  | 182702/225000 [345:48:43<60:53:10,  5.18s/it]
 81%|████████  | 182703/225000 [345:48:48<60:01:30,  5.11s/it]
 81%|████████  | 182704/225000 [345:48:53<60:09:02,  5.12s/it]
 81%|████████  | 182705/225000 [345:48:59<60:43:07,  5.17s/it]
 81%|████████  | 182706/225000 [345:49:04<60:26:05,  5.14s/it]
 81%|████████  | 182707/225000 [345:49:09<60:52:08,  5.18s/it]
 81%|████████  | 182708/225000 [345:49:15<64:03:34,  5.45s/it]
 81%|████████  | 182709/225000 [345:49:26<83:17:43,  7.09s/it]
 81%|████████  | 182710/225000 [345:49:30<72:05:45,  6.14s/it]
 81%|████████  | 182711/225000 [345:49:34<65:02:53,  5.54s/it]
 81%|████████  | 182712/225000 [345:49:43<78:31:46,  6.69s/it]
 81%|████████  | 182713/225000 [345:49:47<68:09:37,  5.80s/it]
 81%|████████  | 182714/225000 [345:49:51<61:16:29,  5.22s/it]
 81%|████████  | 182715/225000 [345:49:56<58:48:00,  5.01s/it]
 81%|████████  | 182716/225000 [345:50:00<56:54:14,  4.84s/it]
 81%|████████  | 182717/225000 [345:50:04<54:20:41,  4.63s/it]
 81%|████████  | 182718/225000 [345:50:11<61:32:07,  5.24s/it]
 81%|████████  | 182719/225000 [345:50:23<84:19:39,  7.18s/it]
 81%|████████  | 182720/225000 [345:50:25<69:10:57,  5.89s/it]
 81%|████████  | 182721/225000 [345:50:30<65:21:16,  5.56s/it]
 81%|████████  | 182722/225000 [345:50:35<62:26:08,  5.32s/it]
 81%|████████  | 182723/225000 [345:50:40<61:34:19,  5.24s/it]
 81%|████████  | 182724/225000 [345:50:45<59:46:31,  5.09s/it]
 81%|████████  | 182725/225000 [345:50:50<59:35:35,  5.07s/it]
 81%|████████  | 182726/225000 [345:50:55<59:26:40,  5.06s/it]
 81%|████████  | 182727/225000 [345:51:00<59:38:39,  5.08s/it]
 81%|████████  | 182728/225000 [345:51:05<61:12:01,  5.21s/it]
 81%|████████  | 182729/225000 [345:51:10<59:15:51,  5.05s/it]
 81%|████████  | 182730/225000 [345:51:15<59:09:01,  5.04s/it]
 81%|████████  | 182731/225000 [345:51:20<57:48:54,  4.92s/it]
 81%|████████  | 182732/225000 [345:51:25<58:03:11,  4.94s/it]
 81%|████████  | 182733/225000 [345:51:34<73:40:18,  6.27s/it]
 81%|████████  | 182734/225000 [345:51:42<80:39:52,  6.87s/it]
 81%|████████  | 182735/225000 [345:51:47<72:16:01,  6.16s/it]
 81%|████████  | 182736/225000 [345:51:52<66:55:04,  5.70s/it]
 81%|████████  | 182737/225000 [345:51:56<61:45:24,  5.26s/it]
 81%|████████  | 182738/225000 [345:52:01<59:59:41,  5.11s/it]
 81%|████████  | 182739/225000 [345:52:05<58:20:40,  4.97s/it]
 81%|████████  | 182740/225000 [345:52:10<57:31:50,  4.90s/it]
 81%|████████  | 182741/225000 [345:52:20<77:06:46,  6.57s/it]
 81%|████████  | 182742/225000 [345:52:27<76:41:43,  6.53s/it]
 81%|████████  | 182743/225000 [345:52:31<67:22:07,  5.74s/it]
 81%|████████  | 182744/225000 [345:52:38<74:03:25,  6.31s/it]
 81%|████████  | 182745/225000 [345:52:43<68:04:18,  5.80s/it]
 81%|████████  | 182746/225000 [345:52:47<62:52:39,  5.36s/it]
 81%|████████  | 182747/225000 [345:52:52<58:58:54,  5.03s/it]
 81%|████████  | 182748/225000 [345:52:56<57:35:07,  4.91s/it]
 81%|████████  | 182749/225000 [345:53:01<55:46:15,  4.75s/it]
 81%|████████  | 182750/225000 [345:53:08<63:59:19,  5.45s/it]
                                                              
{'loss': '0.7532', 'grad_norm': '1.215', 'learning_rate': '4.633e-06', 'epoch': '0.8122', 'num_input_tokens_seen': 573699464, 'train_runtime': '1.245e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182750/225000 [345:53:08<63:59:19,  5.45s/it]
 81%|████████  | 182751/225000 [345:53:18<82:32:07,  7.03s/it]
 81%|████████  | 182752/225000 [345:53:22<70:25:16,  6.00s/it]
 81%|████████  | 182753/225000 [345:53:26<63:11:43,  5.39s/it]
 81%|████████  | 182754/225000 [345:53:33<69:43:41,  5.94s/it]
 81%|████████  | 182755/225000 [345:53:40<71:20:27,  6.08s/it]
 81%|████████  | 182756/225000 [345:53:44<64:46:18,  5.52s/it]
 81%|████████  | 182757/225000 [345:53:48<61:26:16,  5.24s/it]
 81%|████████  | 182758/225000 [345:53:53<60:06:46,  5.12s/it]
 81%|████████  | 182759/225000 [345:53:58<59:49:53,  5.10s/it]
 81%|████████  | 182760/225000 [345:54:03<58:10:33,  4.96s/it]
 81%|████████  | 182761/225000 [345:54:07<56:28:02,  4.81s/it]
 81%|████████  | 182762/225000 [345:54:12<57:07:13,  4.87s/it]
 81%|████████  | 182763/225000 [345:54:19<63:57:37,  5.45s/it]
 81%|████████  | 182764/225000 [345:54:30<81:43:58,  6.97s/it]
 81%|████████  | 182765/225000 [345:54:33<69:07:32,  5.89s/it]
 81%|████████  | 182766/225000 [345:54:37<63:33:30,  5.42s/it]
 81%|████████  | 182767/225000 [345:54:42<60:34:33,  5.16s/it]
 81%|████████  | 182768/225000 [345:54:47<59:01:23,  5.03s/it]
 81%|████████  | 182769/225000 [345:54:51<57:48:49,  4.93s/it]
 81%|████████  | 182770/225000 [345:55:02<76:35:31,  6.53s/it]
 81%|████████  | 182771/225000 [345:55:07<72:18:55,  6.16s/it]
 81%|████████  | 182772/225000 [345:55:11<64:44:01,  5.52s/it]
 81%|████████  | 182773/225000 [345:55:15<59:16:04,  5.05s/it]
 81%|████████  | 182774/225000 [345:55:25<77:21:13,  6.59s/it]
 81%|████████  | 182775/225000 [345:55:30<69:58:32,  5.97s/it]
 81%|████████  | 182776/225000 [345:55:34<65:38:30,  5.60s/it]
 81%|████████  | 182777/225000 [345:55:40<64:23:13,  5.49s/it]
 81%|████████  | 182778/225000 [345:55:45<63:05:21,  5.38s/it]
 81%|████████  | 182779/225000 [345:55:50<63:49:32,  5.44s/it]
 81%|████████  | 182780/225000 [345:55:55<62:22:33,  5.32s/it]
 81%|████████  | 182781/225000 [345:56:01<61:51:48,  5.28s/it]
 81%|████████  | 182782/225000 [345:56:06<61:36:42,  5.25s/it]
 81%|████████  | 182783/225000 [345:56:11<61:32:39,  5.25s/it]
 81%|████████  | 182784/225000 [345:56:16<60:17:17,  5.14s/it]
 81%|████████  | 182785/225000 [345:56:21<60:14:17,  5.14s/it]
 81%|████████  | 182786/225000 [345:56:26<61:16:11,  5.23s/it]
 81%|████████  | 182787/225000 [345:56:32<61:58:13,  5.28s/it]
 81%|████████  | 182788/225000 [345:56:37<60:35:10,  5.17s/it]
 81%|████████  | 182789/225000 [345:56:42<60:23:46,  5.15s/it]
 81%|████████  | 182790/225000 [345:56:47<61:15:28,  5.22s/it]
 81%|████████  | 182791/225000 [345:56:53<62:26:15,  5.33s/it]
 81%|████████  | 182792/225000 [345:56:58<62:15:11,  5.31s/it]
 81%|████████  | 182793/225000 [345:57:03<62:00:19,  5.29s/it]
 81%|████████  | 182794/225000 [345:57:09<62:38:36,  5.34s/it]
 81%|████████  | 182795/225000 [345:57:14<62:24:18,  5.32s/it]
 81%|████████  | 182796/225000 [345:57:20<63:54:02,  5.45s/it]
 81%|████████  | 182797/225000 [345:57:25<64:35:25,  5.51s/it]
 81%|████████  | 182798/225000 [345:57:31<63:56:06,  5.45s/it]
 81%|████████  | 182799/225000 [345:57:36<62:10:50,  5.30s/it]
 81%|████████  | 182800/225000 [345:57:42<64:14:08,  5.48s/it]
                                                              
{'loss': '0.7555', 'grad_norm': '0.9793', 'learning_rate': '4.622e-06', 'epoch': '0.8124', 'num_input_tokens_seen': 573830152, 'train_runtime': '1.245e+06', 'train_tokens_per_second': '460.7'}

 81%|████████  | 182800/225000 [345:57:42<64:14:08,  5.48s/it]
 81%|████████  | 182801/225000 [345:57:51<76:56:46,  6.56s/it]
 81%|████████  | 182802/225000 [345:58:01<91:20:25,  7.79s/it]
 81%|████████  | 182803/225000 [345:58:05<76:39:40,  6.54s/it]
 81%|████████  | 182804/225000 [345:58:12<78:07:38,  6.67s/it]
 81%|████████  | 182805/225000 [345:58:16<67:19:08,  5.74s/it]
 81%|████████  | 182806/225000 [345:58:19<60:57:05,  5.20s/it]
 81%|████████  | 182807/225000 [345:58:28<71:06:37,  6.07s/it]
 81%|████████  | 182808/225000 [345:58:31<63:12:42,  5.39s/it]
 81%|████████  | 182809/225000 [345:58:37<62:39:14,  5.35s/it]
 81%|████████  | 182810/225000 [345:58:42<61:20:44,  5.23s/it]
 81%|████████  | 182811/225000 [345:58:47<62:40:27,  5.35s/it]
 81%|████████  | 182812/225000 [345:58:52<60:32:26,  5.17s/it]
 81%|████████▏ | 182813/225000 [345:58:58<63:13:14,  5.39s/it]
 81%|████████▏ | 182814/225000 [345:59:03<63:34:50,  5.43s/it]
 81%|████████▏ | 182815/225000 [345:59:09<63:59:18,  5.46s/it]
 81%|████████▏ | 182816/225000 [345:59:14<64:17:32,  5.49s/it]
 81%|████████▏ | 182817/225000 [345:59:23<74:37:40,  6.37s/it]
 81%|████████▏ | 182818/225000 [345:59:26<64:50:00,  5.53s/it]
 81%|████████▏ | 182819/225000 [345:59:35<73:51:46,  6.30s/it]
 81%|████████▏ | 182820/225000 [345:59:38<63:22:31,  5.41s/it]
 81%|████████▏ | 182821/225000 [345:59:46<72:34:41,  6.19s/it]
 81%|████████▏ | 182822/225000 [345:59:49<62:29:16,  5.33s/it]
 81%|████████▏ | 182823/225000 [345:59:56<67:45:23,  5.78s/it]
 81%|████████▏ | 182824/225000 [346:00:00<61:54:18,  5.28s/it]
 81%|████████▏ | 182825/225000 [346:00:05<58:31:42,  5.00s/it]
 81%|████████▏ | 182826/225000 [346:00:09<56:06:38,  4.79s/it]
 81%|████████▏ | 182827/225000 [346:00:13<54:50:01,  4.68s/it]
 81%|████████▏ | 182828/225000 [346:00:19<59:07:05,  5.05s/it]
 81%|████████▏ | 182829/225000 [346:00:30<80:37:32,  6.88s/it]
 81%|████████▏ | 182830/225000 [346:00:35<72:43:40,  6.21s/it]
 81%|████████▏ | 182831/225000 [346:00:41<70:37:52,  6.03s/it]
 81%|████████▏ | 182832/225000 [346:00:45<65:46:27,  5.62s/it]
 81%|████████▏ | 182833/225000 [346:00:50<61:40:38,  5.27s/it]
 81%|████████▏ | 182834/225000 [346:00:55<60:14:19,  5.14s/it]
 81%|████████▏ | 182835/225000 [346:01:00<60:58:24,  5.21s/it]
 81%|████████▏ | 182836/225000 [346:01:12<85:08:51,  7.27s/it]
 81%|████████▏ | 182837/225000 [346:01:16<72:07:00,  6.16s/it]
 81%|████████▏ | 182838/225000 [346:01:19<62:34:15,  5.34s/it]
 81%|████████▏ | 182839/225000 [346:01:24<61:03:50,  5.21s/it]
 81%|████████▏ | 182840/225000 [346:01:32<72:33:57,  6.20s/it]
 81%|████████▏ | 182841/225000 [346:01:36<64:39:59,  5.52s/it]
 81%|████████▏ | 182842/225000 [346:01:40<59:26:27,  5.08s/it]
 81%|████████▏ | 182843/225000 [346:01:44<55:34:45,  4.75s/it]
 81%|████████▏ | 182844/225000 [346:01:53<69:41:42,  5.95s/it]
 81%|████████▏ | 182845/225000 [346:01:57<61:07:45,  5.22s/it]
 81%|████████▏ | 182846/225000 [346:02:05<73:50:54,  6.31s/it]
 81%|████████▏ | 182847/225000 [346:02:09<64:03:19,  5.47s/it]
 81%|████████▏ | 182848/225000 [346:02:16<68:34:14,  5.86s/it]
 81%|████████▏ | 182849/225000 [346:02:21<66:25:47,  5.67s/it]
 81%|████████▏ | 182850/225000 [346:02:25<62:01:59,  5.30s/it]
                                                              
{'loss': '0.7702', 'grad_norm': '1.09', 'learning_rate': '4.612e-06', 'epoch': '0.8127', 'num_input_tokens_seen': 573962320, 'train_runtime': '1.246e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 182850/225000 [346:02:25<62:01:59,  5.30s/it]
 81%|████████▏ | 182851/225000 [346:02:30<58:16:15,  4.98s/it]
 81%|████████▏ | 182852/225000 [346:02:40<77:45:28,  6.64s/it]
 81%|████████▏ | 182853/225000 [346:02:44<68:38:12,  5.86s/it]
 81%|████████▏ | 182854/225000 [346:02:48<62:58:57,  5.38s/it]
 81%|████████▏ | 182855/225000 [346:02:53<58:45:14,  5.02s/it]
 81%|████████▏ | 182856/225000 [346:02:59<62:46:19,  5.36s/it]
 81%|████████▏ | 182857/225000 [346:03:07<74:11:08,  6.34s/it]
 81%|████████▏ | 182858/225000 [346:03:14<75:02:38,  6.41s/it]
 81%|████████▏ | 182859/225000 [346:03:19<70:43:36,  6.04s/it]
 81%|████████▏ | 182860/225000 [346:03:23<64:23:28,  5.50s/it]
 81%|████████▏ | 182861/225000 [346:03:27<59:06:44,  5.05s/it]
 81%|████████▏ | 182862/225000 [346:03:31<54:44:58,  4.68s/it]
 81%|████████▏ | 182863/225000 [346:03:35<51:07:31,  4.37s/it]
 81%|████████▏ | 182864/225000 [346:03:47<77:11:27,  6.60s/it]
 81%|████████▏ | 182865/225000 [346:03:50<66:53:47,  5.72s/it]
 81%|████████▏ | 182866/225000 [346:03:54<60:59:16,  5.21s/it]
 81%|████████▏ | 182867/225000 [346:04:04<75:15:13,  6.43s/it]
 81%|████████▏ | 182868/225000 [346:04:07<64:49:39,  5.54s/it]
 81%|████████▏ | 182869/225000 [346:04:11<58:38:45,  5.01s/it]
 81%|████████▏ | 182870/225000 [346:04:17<61:05:58,  5.22s/it]
 81%|████████▏ | 182871/225000 [346:04:25<71:45:43,  6.13s/it]
 81%|████████▏ | 182872/225000 [346:04:29<66:23:23,  5.67s/it]
 81%|████████▏ | 182873/225000 [346:04:34<62:53:50,  5.37s/it]
 81%|████████▏ | 182874/225000 [346:04:38<57:51:36,  4.94s/it]
 81%|████████▏ | 182875/225000 [346:04:43<57:34:46,  4.92s/it]
 81%|████████▏ | 182876/225000 [346:04:53<76:21:25,  6.53s/it]
 81%|████████▏ | 182877/225000 [346:04:56<64:54:43,  5.55s/it]
 81%|████████▏ | 182878/225000 [346:05:05<74:10:12,  6.34s/it]
 81%|████████▏ | 182879/225000 [346:05:08<64:08:27,  5.48s/it]
 81%|████████▏ | 182880/225000 [346:05:13<61:09:44,  5.23s/it]
 81%|████████▏ | 182881/225000 [346:05:20<66:54:46,  5.72s/it]
 81%|████████▏ | 182882/225000 [346:05:23<59:54:35,  5.12s/it]
 81%|████████▏ | 182883/225000 [346:05:31<68:38:32,  5.87s/it]
 81%|████████▏ | 182884/225000 [346:05:35<63:20:11,  5.41s/it]
 81%|████████▏ | 182885/225000 [346:05:40<60:35:08,  5.18s/it]
 81%|████████▏ | 182886/225000 [346:05:45<61:07:58,  5.23s/it]
 81%|████████▏ | 182887/225000 [346:05:54<73:09:31,  6.25s/it]
 81%|████████▏ | 182888/225000 [346:05:58<65:34:17,  5.61s/it]
 81%|████████▏ | 182889/225000 [346:06:01<57:43:31,  4.93s/it]
 81%|████████▏ | 182890/225000 [346:06:07<58:22:25,  4.99s/it]
 81%|████████▏ | 182891/225000 [346:06:12<59:33:42,  5.09s/it]
 81%|████████▏ | 182892/225000 [346:06:17<60:18:00,  5.16s/it]
 81%|████████▏ | 182893/225000 [346:06:23<62:16:43,  5.32s/it]
 81%|████████▏ | 182894/225000 [346:06:28<61:04:06,  5.22s/it]
 81%|████████▏ | 182895/225000 [346:06:33<60:19:04,  5.16s/it]
 81%|████████▏ | 182896/225000 [346:06:38<59:04:34,  5.05s/it]
 81%|████████▏ | 182897/225000 [346:06:43<60:34:13,  5.18s/it]
 81%|████████▏ | 182898/225000 [346:06:49<61:47:33,  5.28s/it]
 81%|████████▏ | 182899/225000 [346:06:54<61:20:06,  5.24s/it]
 81%|████████▏ | 182900/225000 [346:06:59<61:06:39,  5.23s/it]
                                                              
{'loss': '0.7527', 'grad_norm': '1.196', 'learning_rate': '4.601e-06', 'epoch': '0.8129', 'num_input_tokens_seen': 574092464, 'train_runtime': '1.246e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 182900/225000 [346:06:59<61:06:39,  5.23s/it]
 81%|████████▏ | 182901/225000 [346:07:04<60:46:40,  5.20s/it]
 81%|████████▏ | 182902/225000 [346:07:11<66:24:34,  5.68s/it]
 81%|████████▏ | 182903/225000 [346:07:24<91:00:30,  7.78s/it]
 81%|████████▏ | 182904/225000 [346:07:27<76:16:07,  6.52s/it]
 81%|████████▏ | 182905/225000 [346:07:31<66:53:08,  5.72s/it]
 81%|████████▏ | 182906/225000 [346:07:35<60:45:18,  5.20s/it]
 81%|████████▏ | 182907/225000 [346:07:40<58:35:56,  5.01s/it]
 81%|████████▏ | 182908/225000 [346:07:44<57:18:26,  4.90s/it]
 81%|████████▏ | 182909/225000 [346:07:55<78:36:42,  6.72s/it]
 81%|████████▏ | 182910/225000 [346:08:02<78:43:00,  6.73s/it]
 81%|████████▏ | 182911/225000 [346:08:08<75:37:47,  6.47s/it]
 81%|████████▏ | 182912/225000 [346:08:12<67:07:07,  5.74s/it]
 81%|████████▏ | 182913/225000 [346:08:16<62:52:10,  5.38s/it]
 81%|████████▏ | 182914/225000 [346:08:20<58:05:15,  4.97s/it]
 81%|████████▏ | 182915/225000 [346:08:25<55:05:53,  4.71s/it]
 81%|████████▏ | 182916/225000 [346:08:29<54:12:12,  4.64s/it]
 81%|████████▏ | 182917/225000 [346:08:40<78:07:10,  6.68s/it]
 81%|████████▏ | 182918/225000 [346:08:45<69:02:12,  5.91s/it]
 81%|████████▏ | 182919/225000 [346:08:49<62:21:34,  5.33s/it]
 81%|████████▏ | 182920/225000 [346:08:58<78:09:56,  6.69s/it]
 81%|████████▏ | 182921/225000 [346:09:02<67:19:05,  5.76s/it]
 81%|████████▏ | 182922/225000 [346:09:09<72:34:25,  6.21s/it]
 81%|████████▏ | 182923/225000 [346:09:14<67:11:58,  5.75s/it]
 81%|████████▏ | 182924/225000 [346:09:20<66:54:58,  5.73s/it]
 81%|████████▏ | 182925/225000 [346:09:24<63:09:28,  5.40s/it]
 81%|████████▏ | 182926/225000 [346:09:31<68:55:38,  5.90s/it]
 81%|████████▏ | 182927/225000 [346:09:35<60:48:36,  5.20s/it]
 81%|████████▏ | 182928/225000 [346:09:38<54:59:37,  4.71s/it]
 81%|████████▏ | 182929/225000 [346:09:47<68:22:45,  5.85s/it]
 81%|████████▏ | 182930/225000 [346:09:51<60:38:21,  5.19s/it]
 81%|████████▏ | 182931/225000 [346:09:54<55:08:09,  4.72s/it]
 81%|████████▏ | 182932/225000 [346:10:04<73:35:24,  6.30s/it]
 81%|████████▏ | 182933/225000 [346:10:09<67:38:01,  5.79s/it]
 81%|████████▏ | 182934/225000 [346:10:12<59:51:48,  5.12s/it]
 81%|████████▏ | 182935/225000 [346:10:16<54:42:09,  4.68s/it]
 81%|████████▏ | 182936/225000 [346:10:25<68:29:44,  5.86s/it]
 81%|████████▏ | 182937/225000 [346:10:30<65:04:36,  5.57s/it]
 81%|████████▏ | 182938/225000 [346:10:37<73:29:51,  6.29s/it]
 81%|████████▏ | 182939/225000 [346:10:42<66:10:38,  5.66s/it]
 81%|████████▏ | 182940/225000 [346:10:47<64:34:00,  5.53s/it]
 81%|████████▏ | 182941/225000 [346:10:52<62:40:38,  5.36s/it]
 81%|████████▏ | 182942/225000 [346:10:57<61:57:51,  5.30s/it]
 81%|████████▏ | 182943/225000 [346:11:02<61:24:57,  5.26s/it]
 81%|████████▏ | 182944/225000 [346:11:07<61:20:32,  5.25s/it]
 81%|████████▏ | 182945/225000 [346:11:12<60:42:40,  5.20s/it]
 81%|████████▏ | 182946/225000 [346:11:17<59:59:47,  5.14s/it]
 81%|████████▏ | 182947/225000 [346:11:23<60:33:52,  5.18s/it]
 81%|████████▏ | 182948/225000 [346:11:28<60:10:24,  5.15s/it]
 81%|████████▏ | 182949/225000 [346:11:33<60:15:43,  5.16s/it]
 81%|████████▏ | 182950/225000 [346:11:38<59:39:32,  5.11s/it]
                                                              
{'loss': '0.766', 'grad_norm': '1.301', 'learning_rate': '4.591e-06', 'epoch': '0.8131', 'num_input_tokens_seen': 574221272, 'train_runtime': '1.246e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 182950/225000 [346:11:38<59:39:32,  5.11s/it]
 81%|████████▏ | 182951/225000 [346:11:43<59:00:40,  5.05s/it]
 81%|████████▏ | 182952/225000 [346:11:48<59:41:41,  5.11s/it]
 81%|████████▏ | 182953/225000 [346:11:54<60:30:35,  5.18s/it]
 81%|████████▏ | 182954/225000 [346:11:59<61:30:05,  5.27s/it]
 81%|████████▏ | 182955/225000 [346:12:04<61:18:23,  5.25s/it]
 81%|████████▏ | 182956/225000 [346:12:10<62:11:23,  5.32s/it]
 81%|████████▏ | 182957/225000 [346:12:15<63:02:40,  5.40s/it]
 81%|████████▏ | 182958/225000 [346:12:21<62:39:09,  5.36s/it]
 81%|████████▏ | 182959/225000 [346:12:26<62:44:20,  5.37s/it]
 81%|████████▏ | 182960/225000 [346:12:37<81:05:43,  6.94s/it]
 81%|████████▏ | 182961/225000 [346:12:45<85:30:27,  7.32s/it]
 81%|████████▏ | 182962/225000 [346:12:51<83:05:34,  7.12s/it]
 81%|████████▏ | 182963/225000 [346:12:56<75:50:06,  6.49s/it]
 81%|████████▏ | 182964/225000 [346:13:00<65:37:38,  5.62s/it]
 81%|████████▏ | 182965/225000 [346:13:04<59:16:01,  5.08s/it]
 81%|████████▏ | 182966/225000 [346:13:12<70:50:37,  6.07s/it]
 81%|████████▏ | 182967/225000 [346:13:16<63:45:04,  5.46s/it]
 81%|████████▏ | 182968/225000 [346:13:25<75:01:06,  6.43s/it]
 81%|████████▏ | 182969/225000 [346:13:29<66:49:00,  5.72s/it]
 81%|████████▏ | 182970/225000 [346:13:33<61:23:38,  5.26s/it]
 81%|████████▏ | 182971/225000 [346:13:37<57:26:38,  4.92s/it]
 81%|████████▏ | 182972/225000 [346:13:46<68:51:32,  5.90s/it]
 81%|████████▏ | 182973/225000 [346:13:51<68:06:11,  5.83s/it]
 81%|████████▏ | 182974/225000 [346:13:58<72:58:46,  6.25s/it]
 81%|████████▏ | 182975/225000 [346:14:02<62:24:31,  5.35s/it]
 81%|████████▏ | 182976/225000 [346:14:09<70:24:50,  6.03s/it]
 81%|████████▏ | 182977/225000 [346:14:13<62:20:23,  5.34s/it]
 81%|████████▏ | 182978/225000 [346:14:17<56:55:50,  4.88s/it]
 81%|████████▏ | 182979/225000 [346:14:26<73:27:13,  6.29s/it]
 81%|████████▏ | 182980/225000 [346:14:30<62:13:11,  5.33s/it]
 81%|████████▏ | 182981/225000 [346:14:37<69:39:00,  5.97s/it]
 81%|████████▏ | 182982/225000 [346:14:42<65:17:06,  5.59s/it]
 81%|████████▏ | 182983/225000 [346:14:48<69:18:38,  5.94s/it]
 81%|████████▏ | 182984/225000 [346:14:52<60:19:14,  5.17s/it]
 81%|████████▏ | 182985/225000 [346:14:56<56:20:53,  4.83s/it]
 81%|████████▏ | 182986/225000 [346:15:00<55:01:04,  4.71s/it]
 81%|████████▏ | 182987/225000 [346:15:05<54:24:14,  4.66s/it]
 81%|████████▏ | 182988/225000 [346:15:10<56:03:25,  4.80s/it]
 81%|████████▏ | 182989/225000 [346:15:22<82:37:22,  7.08s/it]
 81%|████████▏ | 182990/225000 [346:15:26<69:35:49,  5.96s/it]
 81%|████████▏ | 182991/225000 [346:15:33<73:42:12,  6.32s/it]
 81%|████████▏ | 182992/225000 [346:15:36<64:14:10,  5.50s/it]
 81%|████████▏ | 182993/225000 [346:15:45<73:33:44,  6.30s/it]
 81%|████████▏ | 182994/225000 [346:15:48<63:02:49,  5.40s/it]
 81%|████████▏ | 182995/225000 [346:15:51<55:40:02,  4.77s/it]
 81%|████████▏ | 182996/225000 [346:16:01<72:13:38,  6.19s/it]
 81%|████████▏ | 182997/225000 [346:16:05<65:55:19,  5.65s/it]
 81%|████████▏ | 182998/225000 [346:16:10<63:25:03,  5.44s/it]
 81%|████████▏ | 182999/225000 [346:16:15<60:55:26,  5.22s/it]
 81%|████████▏ | 183000/225000 [346:16:20<61:11:53,  5.25s/it]
                                                              
{'loss': '0.7711', 'grad_norm': '1.205', 'learning_rate': '4.58e-06', 'epoch': '0.8133', 'num_input_tokens_seen': 574352408, 'train_runtime': '1.247e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 183000/225000 [346:16:20<61:11:53,  5.25s/it]
 81%|████████▏ | 183001/225000 [346:16:25<60:26:23,  5.18s/it]
 81%|████████▏ | 183002/225000 [346:16:30<59:27:38,  5.10s/it]
 81%|████████▏ | 183003/225000 [346:16:35<58:31:37,  5.02s/it]
 81%|████████▏ | 183004/225000 [346:16:40<58:34:53,  5.02s/it]
 81%|████████▏ | 183005/225000 [346:16:51<78:17:51,  6.71s/it]
 81%|████████▏ | 183006/225000 [346:16:58<80:53:39,  6.93s/it]
 81%|████████▏ | 183007/225000 [346:17:01<68:03:14,  5.83s/it]
 81%|████████▏ | 183008/225000 [346:17:06<63:19:11,  5.43s/it]
 81%|████████▏ | 183009/225000 [346:17:10<59:33:55,  5.11s/it]
 81%|████████▏ | 183010/225000 [346:17:15<57:33:46,  4.94s/it]
 81%|████████▏ | 183011/225000 [346:17:19<56:03:28,  4.81s/it]
 81%|████████▏ | 183012/225000 [346:17:24<56:08:33,  4.81s/it]
 81%|████████▏ | 183013/225000 [346:17:29<55:40:53,  4.77s/it]
 81%|████████▏ | 183014/225000 [346:17:42<86:21:49,  7.41s/it]
 81%|████████▏ | 183015/225000 [346:17:47<76:29:42,  6.56s/it]
 81%|████████▏ | 183016/225000 [346:17:50<66:33:16,  5.71s/it]
 81%|████████▏ | 183017/225000 [346:17:56<66:33:07,  5.71s/it]
 81%|████████▏ | 183018/225000 [346:18:03<69:10:05,  5.93s/it]
 81%|████████▏ | 183019/225000 [346:18:06<60:50:12,  5.22s/it]
 81%|████████▏ | 183020/225000 [346:18:11<58:54:04,  5.05s/it]
 81%|████████▏ | 183021/225000 [346:18:16<58:18:10,  5.00s/it]
 81%|████████▏ | 183022/225000 [346:18:21<58:27:25,  5.01s/it]
 81%|████████▏ | 183023/225000 [346:18:26<58:05:02,  4.98s/it]
 81%|████████▏ | 183024/225000 [346:18:31<58:13:31,  4.99s/it]
 81%|████████▏ | 183025/225000 [346:18:36<59:17:43,  5.09s/it]
 81%|████████▏ | 183026/225000 [346:18:41<57:49:55,  4.96s/it]
 81%|████████▏ | 183027/225000 [346:18:45<56:32:27,  4.85s/it]
 81%|████████▏ | 183028/225000 [346:18:50<57:35:12,  4.94s/it]
 81%|████████▏ | 183029/225000 [346:18:55<56:09:21,  4.82s/it]
 81%|████████▏ | 183030/225000 [346:19:06<78:58:37,  6.77s/it]
 81%|████████▏ | 183031/225000 [346:19:14<80:54:23,  6.94s/it]
 81%|████████▏ | 183032/225000 [346:19:17<67:55:47,  5.83s/it]
 81%|████████▏ | 183033/225000 [346:19:26<78:04:53,  6.70s/it]
 81%|████████▏ | 183034/225000 [346:19:28<64:43:51,  5.55s/it]
 81%|████████▏ | 183035/225000 [346:19:33<62:18:52,  5.35s/it]
 81%|████████▏ | 183036/225000 [346:19:38<61:27:48,  5.27s/it]
 81%|████████▏ | 183037/225000 [346:19:43<60:30:52,  5.19s/it]
 81%|████████▏ | 183038/225000 [346:19:49<61:42:36,  5.29s/it]
 81%|████████▏ | 183039/225000 [346:19:54<60:47:55,  5.22s/it]
 81%|████████▏ | 183040/225000 [346:19:59<61:12:22,  5.25s/it]
 81%|████████▏ | 183041/225000 [346:20:04<60:27:57,  5.19s/it]
 81%|████████▏ | 183042/225000 [346:20:10<60:28:40,  5.19s/it]
 81%|████████▏ | 183043/225000 [346:20:15<60:36:53,  5.20s/it]
 81%|████████▏ | 183044/225000 [346:20:20<60:07:42,  5.16s/it]
 81%|████████▏ | 183045/225000 [346:20:25<59:27:58,  5.10s/it]
 81%|████████▏ | 183046/225000 [346:20:30<59:39:27,  5.12s/it]
 81%|████████▏ | 183047/225000 [346:20:35<58:20:08,  5.01s/it]
 81%|████████▏ | 183048/225000 [346:20:40<58:18:32,  5.00s/it]
 81%|████████▏ | 183049/225000 [346:20:45<58:51:51,  5.05s/it]
 81%|████████▏ | 183050/225000 [346:20:50<59:40:50,  5.12s/it]
                                                              
{'loss': '0.7654', 'grad_norm': '1.192', 'learning_rate': '4.57e-06', 'epoch': '0.8136', 'num_input_tokens_seen': 574481992, 'train_runtime': '1.247e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 183050/225000 [346:20:50<59:40:50,  5.12s/it]
 81%|████████▏ | 183051/225000 [346:21:01<79:38:26,  6.83s/it]
 81%|████████▏ | 183052/225000 [346:21:09<83:47:35,  7.19s/it]
 81%|████████▏ | 183053/225000 [346:21:12<70:11:21,  6.02s/it]
 81%|████████▏ | 183054/225000 [346:21:18<68:02:09,  5.84s/it]
 81%|████████▏ | 183055/225000 [346:21:23<65:10:18,  5.59s/it]
 81%|████████▏ | 183056/225000 [346:21:28<64:34:20,  5.54s/it]
 81%|████████▏ | 183057/225000 [346:21:33<62:36:39,  5.37s/it]
 81%|████████▏ | 183058/225000 [346:21:38<61:29:51,  5.28s/it]
 81%|████████▏ | 183059/225000 [346:21:43<61:11:11,  5.25s/it]
 81%|████████▏ | 183060/225000 [346:21:49<61:47:45,  5.30s/it]
 81%|████████▏ | 183061/225000 [346:21:54<62:57:11,  5.40s/it]
 81%|████████▏ | 183062/225000 [346:22:00<63:47:47,  5.48s/it]
 81%|████████▏ | 183063/225000 [346:22:05<61:30:07,  5.28s/it]
 81%|████████▏ | 183064/225000 [346:22:11<62:44:00,  5.39s/it]
 81%|████████▏ | 183065/225000 [346:22:17<66:32:17,  5.71s/it]
 81%|████████▏ | 183066/225000 [346:22:22<65:21:02,  5.61s/it]
 81%|████████▏ | 183067/225000 [346:22:28<64:02:20,  5.50s/it]
 81%|████████▏ | 183068/225000 [346:22:32<61:32:32,  5.28s/it]
 81%|████████▏ | 183069/225000 [346:22:38<61:47:42,  5.31s/it]
 81%|████████▏ | 183070/225000 [346:22:43<61:20:11,  5.27s/it]
 81%|████████▏ | 183071/225000 [346:22:49<62:20:23,  5.35s/it]
 81%|████████▏ | 183072/225000 [346:22:54<62:52:18,  5.40s/it]
 81%|████████▏ | 183073/225000 [346:22:59<61:57:32,  5.32s/it]
 81%|████████▏ | 183074/225000 [346:23:05<62:48:04,  5.39s/it]
 81%|████████▏ | 183075/225000 [346:23:10<63:18:42,  5.44s/it]
 81%|████████▏ | 183076/225000 [346:23:16<63:47:53,  5.48s/it]
 81%|████████▏ | 183077/225000 [346:23:21<63:34:32,  5.46s/it]
 81%|████████▏ | 183078/225000 [346:23:26<62:49:26,  5.39s/it]
 81%|████████▏ | 183079/225000 [346:23:32<64:05:50,  5.50s/it]
 81%|████████▏ | 183080/225000 [346:23:38<63:28:52,  5.45s/it]
 81%|████████▏ | 183081/225000 [346:23:43<62:45:59,  5.39s/it]
 81%|████████▏ | 183082/225000 [346:23:48<61:32:42,  5.29s/it]
 81%|████████▏ | 183083/225000 [346:23:53<62:43:23,  5.39s/it]
 81%|████████▏ | 183084/225000 [346:23:59<63:39:25,  5.47s/it]
 81%|████████▏ | 183085/225000 [346:24:05<64:52:15,  5.57s/it]
 81%|████████▏ | 183086/225000 [346:24:10<64:35:27,  5.55s/it]
 81%|████████▏ | 183087/225000 [346:24:16<64:00:57,  5.50s/it]
 81%|████████▏ | 183088/225000 [346:24:21<63:13:32,  5.43s/it]
 81%|████████▏ | 183089/225000 [346:24:27<63:12:36,  5.43s/it]
 81%|████████▏ | 183090/225000 [346:24:32<63:28:51,  5.45s/it]
 81%|████████▏ | 183091/225000 [346:24:38<63:47:11,  5.48s/it]
 81%|████████▏ | 183092/225000 [346:24:43<62:24:52,  5.36s/it]
 81%|████████▏ | 183093/225000 [346:24:48<62:58:45,  5.41s/it]
 81%|████████▏ | 183094/225000 [346:24:53<62:15:12,  5.35s/it]
 81%|████████▏ | 183095/225000 [346:25:03<77:32:11,  6.66s/it]
 81%|████████▏ | 183096/225000 [346:25:13<88:36:43,  7.61s/it]
 81%|████████▏ | 183097/225000 [346:25:17<75:33:31,  6.49s/it]
 81%|████████▏ | 183098/225000 [346:25:21<68:10:24,  5.86s/it]
 81%|████████▏ | 183099/225000 [346:25:25<62:32:50,  5.37s/it]
 81%|████████▏ | 183100/225000 [346:25:30<59:17:27,  5.09s/it]
                                                              
{'loss': '0.7497', 'grad_norm': '1.151', 'learning_rate': '4.559e-06', 'epoch': '0.8138', 'num_input_tokens_seen': 574612344, 'train_runtime': '1.247e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 183100/225000 [346:25:30<59:17:27,  5.09s/it]
 81%|████████▏ | 183101/225000 [346:25:35<58:26:06,  5.02s/it]
 81%|████████▏ | 183102/225000 [346:25:46<81:47:07,  7.03s/it]
 81%|████████▏ | 183103/225000 [346:25:51<74:31:01,  6.40s/it]
 81%|████████▏ | 183104/225000 [346:25:59<78:13:14,  6.72s/it]
 81%|████████▏ | 183105/225000 [346:26:03<68:26:11,  5.88s/it]
 81%|████████▏ | 183106/225000 [346:26:06<59:15:06,  5.09s/it]
 81%|████████▏ | 183107/225000 [346:26:14<69:57:14,  6.01s/it]
 81%|████████▏ | 183108/225000 [346:26:18<63:45:35,  5.48s/it]
 81%|████████▏ | 183109/225000 [346:26:22<58:23:30,  5.02s/it]
 81%|████████▏ | 183110/225000 [346:26:27<55:43:59,  4.79s/it]
 81%|████████▏ | 183111/225000 [346:26:36<70:12:16,  6.03s/it]
 81%|████████▏ | 183112/225000 [346:26:39<61:48:20,  5.31s/it]
 81%|████████▏ | 183113/225000 [346:26:43<55:05:36,  4.74s/it]
 81%|████████▏ | 183114/225000 [346:26:52<70:15:21,  6.04s/it]
 81%|████████▏ | 183115/225000 [346:26:55<61:32:48,  5.29s/it]
 81%|████████▏ | 183116/225000 [346:27:03<71:10:46,  6.12s/it]
 81%|████████▏ | 183117/225000 [346:27:07<62:38:30,  5.38s/it]
 81%|████████▏ | 183118/225000 [346:27:14<67:48:37,  5.83s/it]
 81%|████████▏ | 183119/225000 [346:27:18<61:05:56,  5.25s/it]
 81%|████████▏ | 183120/225000 [346:27:23<60:16:50,  5.18s/it]
 81%|████████▏ | 183121/225000 [346:27:31<72:05:49,  6.20s/it]
 81%|████████▏ | 183122/225000 [346:27:35<63:43:13,  5.48s/it]
 81%|████████▏ | 183123/225000 [346:27:39<56:49:11,  4.88s/it]
 81%|████████▏ | 183124/225000 [346:27:50<78:03:03,  6.71s/it]
 81%|████████▏ | 183125/225000 [346:27:53<66:36:30,  5.73s/it]
 81%|████████▏ | 183126/225000 [346:27:57<59:13:52,  5.09s/it]
 81%|████████▏ | 183127/225000 [346:28:05<70:38:48,  6.07s/it]
 81%|████████▏ | 183128/225000 [346:28:09<61:53:09,  5.32s/it]
 81%|████████▏ | 183129/225000 [346:28:13<57:54:55,  4.98s/it]
 81%|████████▏ | 183130/225000 [346:28:21<70:54:27,  6.10s/it]
 81%|████████▏ | 183131/225000 [346:28:26<64:38:53,  5.56s/it]
 81%|████████▏ | 183132/225000 [346:28:30<60:37:33,  5.21s/it]
 81%|████████▏ | 183133/225000 [346:28:35<57:47:58,  4.97s/it]
 81%|████████▏ | 183134/225000 [346:28:39<55:40:29,  4.79s/it]
 81%|████████▏ | 183135/225000 [346:28:48<71:54:14,  6.18s/it]
 81%|████████▏ | 183136/225000 [346:28:54<70:54:56,  6.10s/it]
 81%|████████▏ | 183137/225000 [346:28:58<62:37:20,  5.39s/it]
 81%|████████▏ | 183138/225000 [346:29:05<69:13:08,  5.95s/it]
 81%|████████▏ | 183139/225000 [346:29:11<68:20:04,  5.88s/it]
 81%|████████▏ | 183140/225000 [346:29:16<64:03:44,  5.51s/it]
 81%|████████▏ | 183141/225000 [346:29:20<61:07:55,  5.26s/it]
 81%|████████▏ | 183142/225000 [346:29:25<60:14:51,  5.18s/it]
 81%|████████▏ | 183143/225000 [346:29:30<59:57:15,  5.16s/it]
 81%|████████▏ | 183144/225000 [346:29:35<58:39:40,  5.05s/it]
 81%|████████▏ | 183145/225000 [346:29:40<56:48:28,  4.89s/it]
 81%|████████▏ | 183146/225000 [346:29:44<56:10:19,  4.83s/it]
 81%|████████▏ | 183147/225000 [346:29:57<82:17:12,  7.08s/it]
 81%|████████▏ | 183148/225000 [346:30:02<76:16:19,  6.56s/it]
 81%|████████▏ | 183149/225000 [346:30:06<66:45:25,  5.74s/it]
 81%|████████▏ | 183150/225000 [346:30:10<61:04:51,  5.25s/it]
                                                              
{'loss': '0.7604', 'grad_norm': '1.149', 'learning_rate': '4.549e-06', 'epoch': '0.814', 'num_input_tokens_seen': 574742192, 'train_runtime': '1.247e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 183150/225000 [346:30:10<61:04:51,  5.25s/it]
 81%|████████▏ | 183151/225000 [346:30:22<83:27:12,  7.18s/it]
 81%|████████▏ | 183152/225000 [346:30:25<70:34:14,  6.07s/it]
 81%|████████▏ | 183153/225000 [346:30:30<64:57:21,  5.59s/it]
 81%|████████▏ | 183154/225000 [346:30:34<60:47:05,  5.23s/it]
 81%|████████▏ | 183155/225000 [346:30:39<58:35:28,  5.04s/it]
 81%|████████▏ | 183156/225000 [346:30:47<69:07:22,  5.95s/it]
 81%|████████▏ | 183157/225000 [346:30:57<83:34:09,  7.19s/it]
 81%|████████▏ | 183158/225000 [346:31:00<70:57:54,  6.11s/it]
 81%|████████▏ | 183159/225000 [346:31:04<62:10:25,  5.35s/it]
 81%|████████▏ | 183160/225000 [346:31:13<74:19:07,  6.39s/it]
 81%|████████▏ | 183161/225000 [346:31:16<63:24:52,  5.46s/it]
 81%|████████▏ | 183162/225000 [346:31:23<67:39:52,  5.82s/it]
 81%|████████▏ | 183163/225000 [346:31:26<59:40:43,  5.14s/it]
 81%|████████▏ | 183164/225000 [346:31:31<57:37:39,  4.96s/it]
 81%|████████▏ | 183165/225000 [346:31:35<56:44:11,  4.88s/it]
 81%|████████▏ | 183166/225000 [346:31:41<57:23:27,  4.94s/it]
 81%|████████▏ | 183167/225000 [346:31:45<57:25:28,  4.94s/it]
 81%|████████▏ | 183168/225000 [346:31:50<57:29:53,  4.95s/it]
 81%|████████▏ | 183169/225000 [346:32:01<78:43:33,  6.78s/it]
 81%|████████▏ | 183170/225000 [346:32:09<79:45:22,  6.86s/it]
 81%|████████▏ | 183171/225000 [346:32:12<67:58:23,  5.85s/it]
 81%|████████▏ | 183172/225000 [346:32:17<66:38:09,  5.74s/it]
 81%|████████▏ | 183173/225000 [346:32:25<72:37:51,  6.25s/it]
 81%|████████▏ | 183174/225000 [346:32:29<63:21:12,  5.45s/it]
 81%|████████▏ | 183175/225000 [346:32:35<66:41:10,  5.74s/it]
 81%|████████▏ | 183176/225000 [346:32:43<73:30:24,  6.33s/it]
 81%|████████▏ | 183177/225000 [346:32:47<67:05:39,  5.78s/it]
 81%|████████▏ | 183178/225000 [346:32:51<61:30:47,  5.30s/it]
 81%|████████▏ | 183179/225000 [346:32:55<57:11:45,  4.92s/it]
 81%|████████▏ | 183180/225000 [346:33:00<55:06:32,  4.74s/it]
 81%|████████▏ | 183181/225000 [346:33:04<53:37:48,  4.62s/it]
 81%|████████▏ | 183182/225000 [346:33:13<69:58:45,  6.02s/it]
 81%|████████▏ | 183183/225000 [346:33:20<71:31:44,  6.16s/it]
 81%|████████▏ | 183184/225000 [346:33:23<62:36:01,  5.39s/it]
 81%|████████▏ | 183185/225000 [346:33:33<77:41:56,  6.69s/it]
 81%|████████▏ | 183186/225000 [346:33:37<67:15:28,  5.79s/it]
 81%|████████▏ | 183187/225000 [346:33:40<59:38:46,  5.14s/it]
 81%|████████▏ | 183188/225000 [346:33:45<59:16:35,  5.10s/it]
 81%|████████▏ | 183189/225000 [346:33:55<75:40:29,  6.52s/it]
 81%|████████▏ | 183190/225000 [346:34:00<68:55:34,  5.93s/it]
 81%|████████▏ | 183191/225000 [346:34:04<63:56:33,  5.51s/it]
 81%|████████▏ | 183192/225000 [346:34:09<60:22:25,  5.20s/it]
 81%|████████▏ | 183193/225000 [346:34:13<56:17:42,  4.85s/it]
 81%|████████▏ | 183194/225000 [346:34:17<55:23:09,  4.77s/it]
 81%|████████▏ | 183195/225000 [346:34:22<55:43:25,  4.80s/it]
 81%|████████▏ | 183196/225000 [346:34:28<57:40:50,  4.97s/it]
 81%|████████▏ | 183197/225000 [346:34:32<57:12:12,  4.93s/it]
 81%|████████▏ | 183198/225000 [346:34:38<58:24:13,  5.03s/it]
 81%|████████▏ | 183199/225000 [346:34:43<59:29:58,  5.12s/it]
 81%|████████▏ | 183200/225000 [346:34:54<80:16:12,  6.91s/it]
                                                              
{'loss': '0.7548', 'grad_norm': '1.202', 'learning_rate': '4.538e-06', 'epoch': '0.8142', 'num_input_tokens_seen': 574873344, 'train_runtime': '1.248e+06', 'train_tokens_per_second': '460.7'}

 81%|████████▏ | 183200/225000 [346:34:54<80:16:12,  6.91s/it]
 81%|████████▏ | 183201/225000 [346:35:00<76:00:56,  6.55s/it]
 81%|████████▏ | 183202/225000 [346:35:04<66:59:07,  5.77s/it]
 81%|████████▏ | 183203/225000 [346:35:09<66:28:23,  5.73s/it]
 81%|████████▏ | 183204/225000 [346:35:15<67:05:57,  5.78s/it]
 81%|████████▏ | 183205/225000 [346:35:24<75:43:43,  6.52s/it]
 81%|████████▏ | 183206/225000 [346:35:31<78:08:53,  6.73s/it]
 81%|████████▏ | 183207/225000 [346:35:34<66:25:29,  5.72s/it]
 81%|████████▏ | 183208/225000 [346:35:42<74:17:42,  6.40s/it]
 81%|████████▏ | 183209/225000 [346:35:46<65:44:11,  5.66s/it]
 81%|████████▏ | 183210/225000 [346:35:49<57:34:23,  4.96s/it]
 81%|████████▏ | 183211/225000 [346:35:55<59:51:04,  5.16s/it]
 81%|████████▏ | 183212/225000 [346:36:03<70:28:56,  6.07s/it]
 81%|████████▏ | 183213/225000 [346:36:06<59:31:44,  5.13s/it]
 81%|████████▏ | 183214/225000 [346:36:15<73:32:57,  6.34s/it]
 81%|████████▏ | 183215/225000 [346:36:19<64:34:13,  5.56s/it]
 81%|████████▏ | 183216/225000 [346:36:26<69:56:44,  6.03s/it]
 81%|████████▏ | 183217/225000 [346:36:31<65:16:02,  5.62s/it]
 81%|████████▏ | 183218/225000 [346:36:35<61:07:41,  5.27s/it]
 81%|████████▏ | 183219/225000 [346:36:40<57:31:49,  4.96s/it]
 81%|████████▏ | 183220/225000 [346:36:44<54:38:08,  4.71s/it]
 81%|████████▏ | 183221/225000 [346:36:48<53:42:14,  4.63s/it]
 81%|████████▏ | 183222/225000 [346:36:52<52:21:22,  4.51s/it]
 81%|████████▏ | 183223/225000 [346:36:58<55:49:31,  4.81s/it]
 81%|████████▏ | 183224/225000 [346:37:08<74:48:59,  6.45s/it]
 81%|████████▏ | 183225/225000 [346:37:12<64:41:49,  5.58s/it]
 81%|████████▏ | 183226/225000 [346:37:20<73:22:02,  6.32s/it]
 81%|████████▏ | 183227/225000 [346:37:23<63:23:19,  5.46s/it]
 81%|████████▏ | 183228/225000 [346:37:31<70:58:43,  6.12s/it]
 81%|████████▏ | 183229/225000 [346:37:39<76:41:17,  6.61s/it]
 81%|████████▏ | 183230/225000 [346:37:42<64:58:25,  5.60s/it]
 81%|████████▏ | 183231/225000 [346:37:46<59:06:05,  5.09s/it]
 81%|████████▏ | 183232/225000 [346:37:53<66:27:48,  5.73s/it]
 81%|████████▏ | 183233/225000 [346:37:58<62:49:09,  5.41s/it]
 81%|████████▏ | 183234/225000 [346:38:03<62:11:47,  5.36s/it]
 81%|████████▏ | 183235/225000 [346:38:08<60:32:30,  5.22s/it]
 81%|████████▏ | 183236/225000 [346:38:13<60:13:37,  5.19s/it]
 81%|████████▏ | 183237/225000 [346:38:18<58:19:03,  5.03s/it]
 81%|████████▏ | 183238/225000 [346:38:23<58:53:47,  5.08s/it]
 81%|████████▏ | 183239/225000 [346:38:27<57:37:49,  4.97s/it]
 81%|████████▏ | 183240/225000 [346:38:32<57:49:56,  4.99s/it]
 81%|████████▏ | 183241/225000 [346:38:37<57:35:48,  4.97s/it]
 81%|████████▏ | 183242/225000 [346:38:42<56:58:42,  4.91s/it]
 81%|████████▏ | 183243/225000 [346:38:47<57:39:33,  4.97s/it]
 81%|████████▏ | 183244/225000 [346:38:52<56:16:49,  4.85s/it]
 81%|████████▏ | 183245/225000 [346:39:00<68:20:14,  5.89s/it]
 81%|████████▏ | 183246/225000 [346:39:09<79:09:20,  6.82s/it]
 81%|████████▏ | 183247/225000 [346:39:13<69:40:08,  6.01s/it]
 81%|████████▏ | 183248/225000 [346:39:17<62:59:00,  5.43s/it]
 81%|████████▏ | 183249/225000 [346:39:28<82:44:08,  7.13s/it]
 81%|████████▏ | 183250/225000 [346:39:33<72:02:49,  6.21s/it]
                                                              
{'loss': '0.7633', 'grad_norm': '1.087', 'learning_rate': '4.528e-06', 'epoch': '0.8144', 'num_input_tokens_seen': 575003328, 'train_runtime': '1.248e+06', 'train_tokens_per_second': '460.8'}

 81%|████████▏ | 183250/225000 [346:39:33<72:02:49,  6.21s/it]
 81%|████████▏ | 183251/225000 [346:39:40<74:39:44,  6.44s/it]
 81%|████████▏ | 183252/225000 [346:39:43<63:34:15,  5.48s/it]
 81%|████████▏ | 183253/225000 [346:39:47<59:39:56,  5.15s/it]
 81%|████████▏ | 183254/225000 [346:39:51<56:07:28,  4.84s/it]
 81%|████████▏ | 183255/225000 [346:40:01<72:53:04,  6.29s/it]
 81%|████████▏ | 183256/225000 [346:40:06<67:27:18,  5.82s/it]
 81%|████████▏ | 183257/225000 [346:40:10<63:18:26,  5.46s/it]
 81%|████████▏ | 183258/225000 [346:40:15<61:23:18,  5.29s/it]
 81%|████████▏ | 183259/225000 [346:40:20<58:24:20,  5.04s/it]
 81%|████████▏ | 183260/225000 [346:40:24<56:48:59,  4.90s/it]
 81%|████████▏ | 183261/225000 [346:40:29<56:11:02,  4.85s/it]
 81%|████████▏ | 183262/225000 [346:40:34<56:00:46,  4.83s/it]
 81%|████████▏ | 183263/225000 [346:40:46<81:16:16,  7.01s/it]
 81%|████████▏ | 183264/225000 [346:40:49<67:46:23,  5.85s/it]
 81%|████████▏ | 183265/225000 [346:40:57<76:41:51,  6.62s/it]
 81%|████████▏ | 183266/225000 [346:41:01<67:32:55,  5.83s/it]
 81%|████████▏ | 183267/225000 [346:41:05<60:57:52,  5.26s/it]
 81%|████████▏ | 183268/225000 [346:41:10<57:34:45,  4.97s/it]
 81%|████████▏ | 183269/225000 [346:41:14<55:41:48,  4.80s/it]
 81%|████████▏ | 183270/225000 [346:41:25<76:03:27,  6.56s/it]
 81%|████████▏ | 183271/225000 [346:41:29<67:56:19,  5.86s/it]
 81%|████████▏ | 183272/225000 [346:41:33<61:56:50,  5.34s/it]
 81%|████████▏ | 183273/225000 [346:41:38<60:22:56,  5.21s/it]
 81%|████████▏ | 183274/225000 [346:41:43<59:24:29,  5.13s/it]
 81%|████████▏ | 183275/225000 [346:41:47<56:22:49,  4.86s/it]
 81%|████████▏ | 183276/225000 [346:41:52<56:44:45,  4.90s/it]
 81%|████████▏ | 183277/225000 [346:41:56<54:09:52,  4.67s/it]
 81%|████████▏ | 183278/225000 [346:42:01<55:45:48,  4.81s/it]
 81%|████████▏ | 183279/225000 [346:42:06<55:23:16,  4.78s/it]
 81%|████████▏ | 183280/225000 [346:42:11<56:58:08,  4.92s/it]
 81%|████████▏ | 183281/225000 [346:42:24<83:16:12,  7.19s/it]
 81%|████████▏ | 183282/225000 [346:42:29<75:14:40,  6.49s/it]
 81%|████████▏ | 183283/225000 [346:42:35<74:36:47,  6.44s/it]
 81%|████████▏ | 183284/225000 [346:42:41<72:57:58,  6.30s/it]
 81%|████████▏ | 183285/225000 [346:42:45<64:54:48,  5.60s/it]
 81%|████████▏ | 183286/225000 [346:42:49<58:17:52,  5.03s/it]
 81%|████████▏ | 183287/225000 [346:42:54<58:11:35,  5.02s/it]
 81%|████████▏ | 183288/225000 [346:43:01<66:01:48,  5.70s/it]
 81%|████████▏ | 183289/225000 [346:43:08<72:25:36,  6.25s/it]
 81%|████████▏ | 183290/225000 [346:43:14<70:45:09,  6.11s/it]
 81%|████████▏ | 183291/225000 [346:43:18<61:32:22,  5.31s/it]
 81%|████████▏ | 183292/225000 [346:43:26<72:15:02,  6.24s/it]
 81%|████████▏ | 183293/225000 [346:43:30<65:17:46,  5.64s/it]
 81%|████████▏ | 183294/225000 [346:43:34<58:29:21,  5.05s/it]
 81%|████████▏ | 183295/225000 [346:43:38<54:31:41,  4.71s/it]
 81%|████████▏ | 183296/225000 [346:43:49<76:02:36,  6.56s/it]
 81%|████████▏ | 183297/225000 [346:43:56<77:17:56,  6.67s/it]
 81%|████████▏ | 183298/225000 [346:44:00<67:39:35,  5.84s/it]
 81%|████████▏ | 183299/225000 [346:44:04<61:05:38,  5.27s/it]
 81%|████████▏ | 183300/225000 [346:44:08<56:38:21,  4.89s/it]
                                                              
{'loss': '0.7558', 'grad_norm': '1.229', 'learning_rate': '4.517e-06', 'epoch': '0.8147', 'num_input_tokens_seen': 575133976, 'train_runtime': '1.248e+06', 'train_tokens_per_second': '460.8'}

 81%|████████▏ | 183300/225000 [346:44:08<56:38:21,  4.89s/it]
 81%|████████▏ | 183301/225000 [346:44:17<73:13:25,  6.32s/it]
 81%|████████▏ | 183302/225000 [346:44:21<63:22:07,  5.47s/it]
 81%|████████▏ | 183303/225000 [346:44:24<56:23:34,  4.87s/it]
 81%|████████▏ | 183304/225000 [346:44:28<54:08:50,  4.68s/it]
 81%|████████▏ | 183305/225000 [346:44:33<54:29:34,  4.71s/it]
 81%|████████▏ | 183306/225000 [346:44:38<56:00:08,  4.84s/it]
 81%|████████▏ | 183307/225000 [346:44:43<56:37:58,  4.89s/it]
 81%|████████▏ | 183308/225000 [346:44:52<68:32:46,  5.92s/it]
 81%|████████▏ | 183309/225000 [346:45:01<79:57:26,  6.90s/it]
 81%|████████▏ | 183310/225000 [346:45:05<71:36:36,  6.18s/it]
 81%|████████▏ | 183311/225000 [346:45:10<66:34:46,  5.75s/it]
 81%|████████▏ | 183312/225000 [346:45:15<62:15:31,  5.38s/it]
 81%|████████▏ | 183313/225000 [346:45:20<61:04:32,  5.27s/it]
 81%|████████▏ | 183314/225000 [346:45:24<56:49:31,  4.91s/it]
 81%|████████▏ | 183315/225000 [346:45:28<56:20:46,  4.87s/it]
 81%|████████▏ | 183316/225000 [346:45:41<82:03:05,  7.09s/it]
 81%|████████▏ | 183317/225000 [346:45:47<79:05:43,  6.83s/it]
 81%|████████▏ | 183318/225000 [346:45:51<70:52:12,  6.12s/it]
 81%|████████▏ | 183319/225000 [346:45:56<63:56:41,  5.52s/it]
 81%|████████▏ | 183320/225000 [346:46:00<59:54:43,  5.17s/it]
 81%|████████▏ | 183321/225000 [346:46:04<56:52:53,  4.91s/it]
 81%|████████▏ | 183322/225000 [346:46:16<79:57:32,  6.91s/it]
 81%|████████▏ | 183323/225000 [346:46:20<69:07:54,  5.97s/it]
 81%|████████▏ | 183324/225000 [346:46:23<61:04:16,  5.28s/it]
 81%|████████▏ | 183325/225000 [346:46:27<57:36:25,  4.98s/it]
 81%|████████▏ | 183326/225000 [346:46:36<71:37:36,  6.19s/it]
 81%|████████▏ | 183327/225000 [346:46:40<62:25:04,  5.39s/it]
 81%|████████▏ | 183328/225000 [346:46:44<56:08:41,  4.85s/it]
 81%|████████▏ | 183329/225000 [346:46:55<79:58:58,  6.91s/it]
 81%|████████▏ | 183330/225000 [346:46:59<69:27:51,  6.00s/it]
 81%|████████▏ | 183331/225000 [346:47:03<61:55:19,  5.35s/it]
 81%|████████▏ | 183332/225000 [346:47:07<57:12:08,  4.94s/it]
 81%|████████▏ | 183333/225000 [346:47:16<70:26:06,  6.09s/it]
 81%|████████▏ | 183334/225000 [346:47:20<63:13:42,  5.46s/it]
 81%|████████▏ | 183335/225000 [346:47:26<64:22:17,  5.56s/it]
 81%|████████▏ | 183336/225000 [346:47:30<58:56:55,  5.09s/it]
 81%|████████▏ | 183337/225000 [346:47:34<57:09:20,  4.94s/it]
 81%|████████▏ | 183338/225000 [346:47:39<56:46:15,  4.91s/it]
 81%|████████▏ | 183339/225000 [346:47:44<57:09:01,  4.94s/it]
 81%|████████▏ | 183340/225000 [346:47:49<57:00:22,  4.93s/it]
 81%|████████▏ | 183341/225000 [346:47:54<56:40:22,  4.90s/it]
 81%|████████▏ | 183342/225000 [346:47:59<56:53:45,  4.92s/it]
 81%|████████▏ | 183343/225000 [346:48:04<57:47:21,  4.99s/it]
 81%|████████▏ | 183344/225000 [346:48:12<67:47:57,  5.86s/it]
 81%|████████▏ | 183345/225000 [346:48:21<81:03:41,  7.01s/it]
 81%|████████▏ | 183346/225000 [346:48:25<70:02:14,  6.05s/it]
 81%|████████▏ | 183347/225000 [346:48:29<61:37:36,  5.33s/it]
 81%|████████▏ | 183348/225000 [346:48:38<75:45:32,  6.55s/it]
 81%|████████▏ | 183349/225000 [346:48:44<73:39:18,  6.37s/it]
 81%|████████▏ | 183350/225000 [346:48:50<71:00:55,  6.14s/it]
                                                              
{'loss': '0.7582', 'grad_norm': '1.163', 'learning_rate': '4.507e-06', 'epoch': '0.8149', 'num_input_tokens_seen': 575264768, 'train_runtime': '1.249e+06', 'train_tokens_per_second': '460.8'}

 81%|████████▏ | 183350/225000 [346:48:50<71:00:55,  6.14s/it]
 81%|████████▏ | 183351/225000 [346:48:53<61:39:14,  5.33s/it]
 81%|████████▏ | 183352/225000 [346:48:57<56:01:45,  4.84s/it]
 81%|████████▏ | 183353/225000 [346:49:01<54:56:38,  4.75s/it]
 81%|████████▏ | 183354/225000 [346:49:06<54:00:04,  4.67s/it]
 81%|████████▏ | 183355/225000 [346:49:11<54:58:15,  4.75s/it]
 81%|████████▏ | 183356/225000 [346:49:17<60:47:54,  5.26s/it]
 81%|████████▏ | 183357/225000 [346:49:29<83:29:11,  7.22s/it]
 81%|████████▏ | 183358/225000 [346:49:33<72:40:03,  6.28s/it]
 81%|████████▏ | 183359/225000 [346:49:37<64:22:10,  5.56s/it]
 81%|████████▏ | 183360/225000 [346:49:41<59:56:49,  5.18s/it]
 81%|████████▏ | 183361/225000 [346:49:46<57:31:31,  4.97s/it]
 81%|████████▏ | 183362/225000 [346:49:52<61:10:25,  5.29s/it]
 81%|████████▏ | 183363/225000 [346:50:03<80:05:14,  6.92s/it]
 81%|████████▏ | 183364/225000 [346:50:06<67:45:37,  5.86s/it]
 81%|████████▏ | 183365/225000 [346:50:14<76:10:21,  6.59s/it]
 81%|████████▏ | 183366/225000 [346:50:18<64:39:36,  5.59s/it]
 81%|████████▏ | 183367/225000 [346:50:23<62:21:10,  5.39s/it]
 81%|████████▏ | 183368/225000 [346:50:27<60:02:20,  5.19s/it]
 81%|████████▏ | 183369/225000 [346:50:33<60:40:15,  5.25s/it]
 81%|████████▏ | 183370/225000 [346:50:38<59:25:27,  5.14s/it]
 81%|████████▏ | 183371/225000 [346:50:43<59:53:30,  5.18s/it]
 81%|████████▏ | 183372/225000 [346:50:48<59:16:12,  5.13s/it]
 81%|████████▏ | 183373/225000 [346:50:53<59:23:58,  5.14s/it]
 81%|████████▏ | 183374/225000 [346:50:58<58:40:57,  5.08s/it]
 82%|████████▏ | 183375/225000 [346:51:03<60:00:30,  5.19s/it]
 82%|████████▏ | 183376/225000 [346:51:09<60:21:34,  5.22s/it]
 82%|████████▏ | 183377/225000 [346:51:14<61:19:22,  5.30s/it]
 82%|████████▏ | 183378/225000 [346:51:19<60:31:51,  5.24s/it]
 82%|████████▏ | 183379/225000 [346:51:26<64:57:34,  5.62s/it]
 82%|████████▏ | 183380/225000 [346:51:36<81:57:41,  7.09s/it]
 82%|████████▏ | 183381/225000 [346:51:41<74:57:36,  6.48s/it]
 82%|████████▏ | 183382/225000 [346:51:47<71:59:27,  6.23s/it]
 82%|████████▏ | 183383/225000 [346:51:51<64:59:35,  5.62s/it]
 82%|████████▏ | 183384/225000 [346:51:55<60:29:37,  5.23s/it]
 82%|████████▏ | 183385/225000 [346:52:00<58:55:03,  5.10s/it]
 82%|████████▏ | 183386/225000 [346:52:05<57:11:14,  4.95s/it]
 82%|████████▏ | 183387/225000 [346:52:10<57:01:53,  4.93s/it]
 82%|████████▏ | 183388/225000 [346:52:14<55:50:39,  4.83s/it]
 82%|████████▏ | 183389/225000 [346:52:19<55:54:11,  4.84s/it]
 82%|████████▏ | 183390/225000 [346:52:24<54:51:39,  4.75s/it]
 82%|████████▏ | 183391/225000 [346:52:36<79:40:32,  6.89s/it]
 82%|████████▏ | 183392/225000 [346:52:42<78:26:40,  6.79s/it]
 82%|████████▏ | 183393/225000 [346:52:47<70:34:56,  6.11s/it]
 82%|████████▏ | 183394/225000 [346:52:51<64:07:44,  5.55s/it]
 82%|████████▏ | 183395/225000 [346:52:56<60:49:59,  5.26s/it]
 82%|████████▏ | 183396/225000 [346:53:00<57:35:09,  4.98s/it]
 82%|████████▏ | 183397/225000 [346:53:04<55:07:20,  4.77s/it]
 82%|████████▏ | 183398/225000 [346:53:17<82:06:17,  7.10s/it]
 82%|████████▏ | 183399/225000 [346:53:21<71:18:03,  6.17s/it]
 82%|████████▏ | 183400/225000 [346:53:24<61:49:01,  5.35s/it]
                                                              
{'loss': '0.7683', 'grad_norm': '1.202', 'learning_rate': '4.496e-06', 'epoch': '0.8151', 'num_input_tokens_seen': 575396152, 'train_runtime': '1.249e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183400/225000 [346:53:24<61:49:01,  5.35s/it]
 82%|████████▏ | 183401/225000 [346:53:33<74:36:11,  6.46s/it]
 82%|████████▏ | 183402/225000 [346:53:37<65:32:08,  5.67s/it]
 82%|████████▏ | 183403/225000 [346:53:42<61:54:01,  5.36s/it]
 82%|████████▏ | 183404/225000 [346:53:46<60:05:16,  5.20s/it]
 82%|████████▏ | 183405/225000 [346:53:51<59:24:24,  5.14s/it]
 82%|████████▏ | 183406/225000 [346:53:56<57:26:58,  4.97s/it]
 82%|████████▏ | 183407/225000 [346:54:01<57:38:17,  4.99s/it]
 82%|████████▏ | 183408/225000 [346:54:06<56:29:02,  4.89s/it]
 82%|████████▏ | 183409/225000 [346:54:17<79:40:29,  6.90s/it]
 82%|████████▏ | 183410/225000 [346:54:22<70:49:31,  6.13s/it]
 82%|████████▏ | 183411/225000 [346:54:28<70:49:54,  6.13s/it]
 82%|████████▏ | 183412/225000 [346:54:31<61:56:56,  5.36s/it]
 82%|████████▏ | 183413/225000 [346:54:36<60:48:49,  5.26s/it]
 82%|████████▏ | 183414/225000 [346:54:41<58:45:40,  5.09s/it]
 82%|████████▏ | 183415/225000 [346:54:46<57:09:31,  4.95s/it]
 82%|████████▏ | 183416/225000 [346:54:50<55:47:05,  4.83s/it]
 82%|████████▏ | 183417/225000 [346:54:58<66:50:46,  5.79s/it]
 82%|████████▏ | 183418/225000 [346:55:08<80:41:09,  6.99s/it]
 82%|████████▏ | 183419/225000 [346:55:12<71:00:06,  6.15s/it]
 82%|████████▏ | 183420/225000 [346:55:17<64:43:21,  5.60s/it]
 82%|████████▏ | 183421/225000 [346:55:21<61:27:51,  5.32s/it]
 82%|████████▏ | 183422/225000 [346:55:26<58:41:20,  5.08s/it]
 82%|████████▏ | 183423/225000 [346:55:30<56:42:27,  4.91s/it]
 82%|████████▏ | 183424/225000 [346:55:41<76:13:28,  6.60s/it]
 82%|████████▏ | 183425/225000 [346:55:46<70:39:22,  6.12s/it]
 82%|████████▏ | 183426/225000 [346:55:51<67:18:07,  5.83s/it]
 82%|████████▏ | 183427/225000 [346:55:56<63:53:15,  5.53s/it]
 82%|████████▏ | 183428/225000 [346:56:01<63:02:33,  5.46s/it]
 82%|████████▏ | 183429/225000 [346:56:06<61:16:14,  5.31s/it]
 82%|████████▏ | 183430/225000 [346:56:11<60:02:48,  5.20s/it]
 82%|████████▏ | 183431/225000 [346:56:16<58:46:00,  5.09s/it]
 82%|████████▏ | 183432/225000 [346:56:21<58:38:57,  5.08s/it]
 82%|████████▏ | 183433/225000 [346:56:26<59:28:16,  5.15s/it]
 82%|████████▏ | 183434/225000 [346:56:31<59:07:48,  5.12s/it]
 82%|████████▏ | 183435/225000 [346:56:36<57:54:54,  5.02s/it]
 82%|████████▏ | 183436/225000 [346:56:41<58:09:42,  5.04s/it]
 82%|████████▏ | 183437/225000 [346:56:46<57:18:16,  4.96s/it]
 82%|████████▏ | 183438/225000 [346:56:51<59:17:48,  5.14s/it]
 82%|████████▏ | 183439/225000 [346:56:56<58:51:26,  5.10s/it]
 82%|████████▏ | 183440/225000 [346:57:06<74:05:38,  6.42s/it]
 82%|████████▏ | 183441/225000 [346:57:13<77:08:04,  6.68s/it]
 82%|████████▏ | 183442/225000 [346:57:20<78:00:01,  6.76s/it]
 82%|████████▏ | 183443/225000 [346:57:24<69:08:58,  5.99s/it]
 82%|████████▏ | 183444/225000 [346:57:31<72:59:58,  6.32s/it]
 82%|████████▏ | 183445/225000 [346:57:39<78:16:51,  6.78s/it]
 82%|████████▏ | 183446/225000 [346:57:43<66:38:39,  5.77s/it]
 82%|████████▏ | 183447/225000 [346:57:46<59:19:40,  5.14s/it]
 82%|████████▏ | 183448/225000 [346:57:56<75:28:14,  6.54s/it]
 82%|████████▏ | 183449/225000 [346:58:00<64:36:37,  5.60s/it]
 82%|████████▏ | 183450/225000 [346:58:07<70:26:43,  6.10s/it]
                                                              
{'loss': '0.7584', 'grad_norm': '1.203', 'learning_rate': '4.486e-06', 'epoch': '0.8153', 'num_input_tokens_seen': 575526240, 'train_runtime': '1.249e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183450/225000 [346:58:07<70:26:43,  6.10s/it]
 82%|████████▏ | 183451/225000 [346:58:11<63:24:08,  5.49s/it]
 82%|████████▏ | 183452/225000 [346:58:15<59:04:16,  5.12s/it]
 82%|████████▏ | 183453/225000 [346:58:23<67:44:01,  5.87s/it]
 82%|████████▏ | 183454/225000 [346:58:30<72:02:18,  6.24s/it]
 82%|████████▏ | 183455/225000 [346:58:33<61:44:45,  5.35s/it]
 82%|████████▏ | 183456/225000 [346:58:38<58:09:11,  5.04s/it]
 82%|████████▏ | 183457/225000 [346:58:46<71:11:50,  6.17s/it]
 82%|████████▏ | 183458/225000 [346:58:52<69:35:00,  6.03s/it]
 82%|████████▏ | 183459/225000 [346:58:55<59:58:54,  5.20s/it]
 82%|████████▏ | 183460/225000 [346:59:00<58:51:25,  5.10s/it]
 82%|████████▏ | 183461/225000 [346:59:05<57:57:04,  5.02s/it]
 82%|████████▏ | 183462/225000 [346:59:10<57:33:19,  4.99s/it]
 82%|████████▏ | 183463/225000 [346:59:14<55:59:04,  4.85s/it]
 82%|████████▏ | 183464/225000 [346:59:19<55:11:10,  4.78s/it]
 82%|████████▏ | 183465/225000 [346:59:28<68:02:34,  5.90s/it]
 82%|████████▏ | 183466/225000 [346:59:36<77:49:45,  6.75s/it]
 82%|████████▏ | 183467/225000 [346:59:41<69:30:55,  6.03s/it]
 82%|████████▏ | 183468/225000 [346:59:45<64:06:03,  5.56s/it]
 82%|████████▏ | 183469/225000 [346:59:50<60:23:28,  5.23s/it]
 82%|████████▏ | 183470/225000 [346:59:54<58:43:17,  5.09s/it]
 82%|████████▏ | 183471/225000 [346:59:59<56:42:01,  4.92s/it]
 82%|████████▏ | 183472/225000 [347:00:04<56:14:31,  4.88s/it]
 82%|████████▏ | 183473/225000 [347:00:08<56:05:04,  4.86s/it]
 82%|████████▏ | 183474/225000 [347:00:21<82:29:13,  7.15s/it]
 82%|████████▏ | 183475/225000 [347:00:26<75:40:46,  6.56s/it]
 82%|████████▏ | 183476/225000 [347:00:30<65:31:51,  5.68s/it]
 82%|████████▏ | 183477/225000 [347:00:39<78:11:36,  6.78s/it]
 82%|████████▏ | 183478/225000 [347:00:43<68:33:27,  5.94s/it]
 82%|████████▏ | 183479/225000 [347:00:47<60:20:44,  5.23s/it]
 82%|████████▏ | 183480/225000 [347:00:55<70:25:38,  6.11s/it]
 82%|████████▏ | 183481/225000 [347:00:58<61:21:15,  5.32s/it]
 82%|████████▏ | 183482/225000 [347:01:05<67:47:22,  5.88s/it]
 82%|████████▏ | 183483/225000 [347:01:09<61:05:21,  5.30s/it]
 82%|████████▏ | 183484/225000 [347:01:13<56:50:34,  4.93s/it]
 82%|████████▏ | 183485/225000 [347:01:18<54:32:35,  4.73s/it]
 82%|████████▏ | 183486/225000 [347:01:28<73:54:41,  6.41s/it]
 82%|████████▏ | 183487/225000 [347:01:33<68:10:08,  5.91s/it]
 82%|████████▏ | 183488/225000 [347:01:36<60:09:41,  5.22s/it]
 82%|████████▏ | 183489/225000 [347:01:41<57:49:24,  5.01s/it]
 82%|████████▏ | 183490/225000 [347:01:49<69:41:59,  6.04s/it]
 82%|████████▏ | 183491/225000 [347:01:53<60:14:03,  5.22s/it]
 82%|████████▏ | 183492/225000 [347:01:57<55:24:46,  4.81s/it]
 82%|████████▏ | 183493/225000 [347:02:01<52:56:12,  4.59s/it]
 82%|████████▏ | 183494/225000 [347:02:08<61:07:15,  5.30s/it]
 82%|████████▏ | 183495/225000 [347:02:16<70:12:16,  6.09s/it]
 82%|████████▏ | 183496/225000 [347:02:20<63:39:40,  5.52s/it]
 82%|████████▏ | 183497/225000 [347:02:25<61:04:33,  5.30s/it]
 82%|████████▏ | 183498/225000 [347:02:29<58:08:29,  5.04s/it]
 82%|████████▏ | 183499/225000 [347:02:34<57:41:44,  5.00s/it]
 82%|████████▏ | 183500/225000 [347:02:39<57:43:37,  5.01s/it]
                                                              
{'loss': '0.7599', 'grad_norm': '1.116', 'learning_rate': '4.475e-06', 'epoch': '0.8156', 'num_input_tokens_seen': 575657032, 'train_runtime': '1.249e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183500/225000 [347:02:39<57:43:37,  5.01s/it]
 82%|████████▏ | 183501/225000 [347:02:44<56:47:54,  4.93s/it]
 82%|████████▏ | 183502/225000 [347:02:55<79:15:50,  6.88s/it]
 82%|████████▏ | 183503/225000 [347:03:00<71:59:49,  6.25s/it]
 82%|████████▏ | 183504/225000 [347:03:08<78:02:12,  6.77s/it]
 82%|████████▏ | 183505/225000 [347:03:11<66:18:39,  5.75s/it]
 82%|████████▏ | 183506/225000 [347:03:15<60:00:27,  5.21s/it]
 82%|████████▏ | 183507/225000 [347:03:25<76:29:55,  6.64s/it]
 82%|████████▏ | 183508/225000 [347:03:28<64:56:55,  5.64s/it]
 82%|████████▏ | 183509/225000 [347:03:33<61:17:24,  5.32s/it]
 82%|████████▏ | 183510/225000 [347:03:39<62:14:50,  5.40s/it]
 82%|████████▏ | 183511/225000 [347:03:44<62:41:37,  5.44s/it]
 82%|████████▏ | 183512/225000 [347:03:50<62:54:25,  5.46s/it]
 82%|████████▏ | 183513/225000 [347:03:55<63:30:01,  5.51s/it]
 82%|████████▏ | 183514/225000 [347:04:03<71:23:55,  6.20s/it]
 82%|████████▏ | 183515/225000 [347:04:09<70:54:36,  6.15s/it]
 82%|████████▏ | 183516/225000 [347:04:13<64:03:46,  5.56s/it]
 82%|████████▏ | 183517/225000 [347:04:20<68:50:29,  5.97s/it]
 82%|████████▏ | 183518/225000 [347:04:23<59:26:16,  5.16s/it]
 82%|████████▏ | 183519/225000 [347:04:27<53:41:33,  4.66s/it]
 82%|████████▏ | 183520/225000 [347:04:31<52:23:10,  4.55s/it]
 82%|████████▏ | 183521/225000 [347:04:40<65:43:18,  5.70s/it]
 82%|████████▏ | 183522/225000 [347:04:48<73:35:37,  6.39s/it]
 82%|████████▏ | 183523/225000 [347:04:51<63:43:46,  5.53s/it]
 82%|████████▏ | 183524/225000 [347:04:55<57:44:43,  5.01s/it]
 82%|████████▏ | 183525/225000 [347:05:02<64:50:08,  5.63s/it]
 82%|████████▏ | 183526/225000 [347:05:07<61:52:51,  5.37s/it]
 82%|████████▏ | 183527/225000 [347:05:12<62:17:53,  5.41s/it]
 82%|████████▏ | 183528/225000 [347:05:17<60:31:12,  5.25s/it]
 82%|████████▏ | 183529/225000 [347:05:22<59:30:33,  5.17s/it]
 82%|████████▏ | 183530/225000 [347:05:28<60:20:26,  5.24s/it]
 82%|████████▏ | 183531/225000 [347:05:33<59:54:16,  5.20s/it]
 82%|████████▏ | 183532/225000 [347:05:38<59:03:44,  5.13s/it]
 82%|████████▏ | 183533/225000 [347:05:43<58:26:07,  5.07s/it]
 82%|████████▏ | 183534/225000 [347:05:48<58:40:02,  5.09s/it]
 82%|████████▏ | 183535/225000 [347:05:53<58:46:15,  5.10s/it]
 82%|████████▏ | 183536/225000 [347:05:58<59:15:59,  5.15s/it]
 82%|████████▏ | 183537/225000 [347:06:07<72:37:50,  6.31s/it]
 82%|████████▏ | 183538/225000 [347:06:15<77:48:31,  6.76s/it]
 82%|████████▏ | 183539/225000 [347:06:19<68:06:49,  5.91s/it]
 82%|████████▏ | 183540/225000 [347:06:27<75:36:11,  6.56s/it]
 82%|████████▏ | 183541/225000 [347:06:31<66:22:06,  5.76s/it]
 82%|████████▏ | 183542/225000 [347:06:35<59:49:10,  5.19s/it]
 82%|████████▏ | 183543/225000 [347:06:40<59:34:53,  5.17s/it]
 82%|████████▏ | 183544/225000 [347:06:45<58:40:55,  5.10s/it]
 82%|████████▏ | 183545/225000 [347:06:49<57:09:23,  4.96s/it]
 82%|████████▏ | 183546/225000 [347:06:55<59:01:36,  5.13s/it]
 82%|████████▏ | 183547/225000 [347:06:59<55:48:46,  4.85s/it]
 82%|████████▏ | 183548/225000 [347:07:07<66:22:36,  5.76s/it]
 82%|████████▏ | 183549/225000 [347:07:19<87:45:08,  7.62s/it]
 82%|████████▏ | 183550/225000 [347:07:22<73:10:09,  6.35s/it]
                                                              
{'loss': '0.7635', 'grad_norm': '1.099', 'learning_rate': '4.465e-06', 'epoch': '0.8158', 'num_input_tokens_seen': 575786976, 'train_runtime': '1.25e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183550/225000 [347:07:22<73:10:09,  6.35s/it]
 82%|████████▏ | 183551/225000 [347:07:29<72:49:41,  6.33s/it]
 82%|████████▏ | 183552/225000 [347:07:33<66:34:30,  5.78s/it]
 82%|████████▏ | 183553/225000 [347:07:37<60:24:48,  5.25s/it]
 82%|████████▏ | 183554/225000 [347:07:41<56:54:06,  4.94s/it]
 82%|████████▏ | 183555/225000 [347:07:46<55:49:20,  4.85s/it]
 82%|████████▏ | 183556/225000 [347:07:55<71:35:25,  6.22s/it]
 82%|████████▏ | 183557/225000 [347:07:59<63:02:31,  5.48s/it]
 82%|████████▏ | 183558/225000 [347:08:06<69:21:57,  6.03s/it]
 82%|████████▏ | 183559/225000 [347:08:13<71:10:04,  6.18s/it]
 82%|████████▏ | 183560/225000 [347:08:18<65:35:20,  5.70s/it]
 82%|████████▏ | 183561/225000 [347:08:22<60:38:28,  5.27s/it]
 82%|████████▏ | 183562/225000 [347:08:26<57:21:17,  4.98s/it]
 82%|████████▏ | 183563/225000 [347:08:31<58:24:52,  5.07s/it]
 82%|████████▏ | 183564/225000 [347:08:36<56:44:59,  4.93s/it]
 82%|████████▏ | 183565/225000 [347:08:41<56:11:06,  4.88s/it]
 82%|████████▏ | 183566/225000 [347:08:47<62:13:09,  5.41s/it]
 82%|████████▏ | 183567/225000 [347:08:59<83:56:20,  7.29s/it]
 82%|████████▏ | 183568/225000 [347:09:03<73:47:58,  6.41s/it]
 82%|████████▏ | 183569/225000 [347:09:08<66:32:34,  5.78s/it]
 82%|████████▏ | 183570/225000 [347:09:12<60:51:21,  5.29s/it]
 82%|████████▏ | 183571/225000 [347:09:16<57:24:04,  4.99s/it]
 82%|████████▏ | 183572/225000 [347:09:26<72:28:45,  6.30s/it]
 82%|████████▏ | 183573/225000 [347:09:29<62:27:31,  5.43s/it]
 82%|████████▏ | 183574/225000 [347:09:36<66:33:24,  5.78s/it]
 82%|████████▏ | 183575/225000 [347:09:41<63:49:35,  5.55s/it]
 82%|████████▏ | 183576/225000 [347:09:46<65:05:17,  5.66s/it]
 82%|████████▏ | 183577/225000 [347:09:52<65:13:50,  5.67s/it]
 82%|████████▏ | 183578/225000 [347:09:58<64:57:08,  5.65s/it]
 82%|████████▏ | 183579/225000 [347:10:03<64:49:12,  5.63s/it]
 82%|████████▏ | 183580/225000 [347:10:08<62:34:47,  5.44s/it]
 82%|████████▏ | 183581/225000 [347:10:13<61:20:49,  5.33s/it]
 82%|████████▏ | 183582/225000 [347:10:19<61:59:50,  5.39s/it]
 82%|████████▏ | 183583/225000 [347:10:24<61:56:14,  5.38s/it]
 82%|████████▏ | 183584/225000 [347:10:30<61:15:01,  5.32s/it]
 82%|████████▏ | 183585/225000 [347:10:35<60:27:19,  5.26s/it]
 82%|████████▏ | 183586/225000 [347:10:40<61:33:43,  5.35s/it]
 82%|████████▏ | 183587/225000 [347:10:46<62:10:05,  5.40s/it]
 82%|████████▏ | 183588/225000 [347:10:51<61:59:38,  5.39s/it]
 82%|████████▏ | 183589/225000 [347:10:56<61:38:47,  5.36s/it]
 82%|████████▏ | 183590/225000 [347:11:02<61:44:05,  5.37s/it]
 82%|████████▏ | 183591/225000 [347:11:07<61:45:47,  5.37s/it]
 82%|████████▏ | 183592/225000 [347:11:13<61:51:18,  5.38s/it]
 82%|████████▏ | 183593/225000 [347:11:18<62:12:09,  5.41s/it]
 82%|████████▏ | 183594/225000 [347:11:23<62:08:39,  5.40s/it]
 82%|████████▏ | 183595/225000 [347:11:29<62:56:47,  5.47s/it]
 82%|████████▏ | 183596/225000 [347:11:35<62:58:23,  5.48s/it]
 82%|████████▏ | 183597/225000 [347:11:40<61:33:18,  5.35s/it]
 82%|████████▏ | 183598/225000 [347:11:45<61:26:18,  5.34s/it]
 82%|████████▏ | 183599/225000 [347:11:50<61:58:19,  5.39s/it]
 82%|████████▏ | 183600/225000 [347:11:55<60:43:22,  5.28s/it]
                                                              
{'loss': '0.7683', 'grad_norm': '1.162', 'learning_rate': '4.454e-06', 'epoch': '0.816', 'num_input_tokens_seen': 575917368, 'train_runtime': '1.25e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183600/225000 [347:11:55<60:43:22,  5.28s/it]
 82%|████████▏ | 183601/225000 [347:12:06<78:34:53,  6.83s/it]
 82%|████████▏ | 183602/225000 [347:12:11<73:41:42,  6.41s/it]
 82%|████████▏ | 183603/225000 [347:12:19<78:44:54,  6.85s/it]
 82%|████████▏ | 183604/225000 [347:12:22<66:21:08,  5.77s/it]
 82%|████████▏ | 183605/225000 [347:12:30<72:20:50,  6.29s/it]
 82%|████████▏ | 183606/225000 [347:12:34<65:30:56,  5.70s/it]
 82%|████████▏ | 183607/225000 [347:12:38<60:27:17,  5.26s/it]
 82%|████████▏ | 183608/225000 [347:12:43<56:44:40,  4.94s/it]
 82%|████████▏ | 183609/225000 [347:12:47<55:24:05,  4.82s/it]
 82%|████████▏ | 183610/225000 [347:12:54<62:35:18,  5.44s/it]
 82%|████████▏ | 183611/225000 [347:13:03<73:18:23,  6.38s/it]
 82%|████████▏ | 183612/225000 [347:13:06<63:51:01,  5.55s/it]
 82%|████████▏ | 183613/225000 [347:13:11<61:17:18,  5.33s/it]
 82%|████████▏ | 183614/225000 [347:13:16<59:48:51,  5.20s/it]
 82%|████████▏ | 183615/225000 [347:13:21<57:43:47,  5.02s/it]
 82%|████████▏ | 183616/225000 [347:13:26<58:04:01,  5.05s/it]
 82%|████████▏ | 183617/225000 [347:13:37<80:31:32,  7.01s/it]
 82%|████████▏ | 183618/225000 [347:13:42<73:14:59,  6.37s/it]
 82%|████████▏ | 183619/225000 [347:13:48<71:27:39,  6.22s/it]
 82%|████████▏ | 183620/225000 [347:13:53<68:41:58,  5.98s/it]
 82%|████████▏ | 183621/225000 [347:14:01<72:58:47,  6.35s/it]
 82%|████████▏ | 183622/225000 [347:14:04<63:06:31,  5.49s/it]
 82%|████████▏ | 183623/225000 [347:14:10<64:29:41,  5.61s/it]
 82%|████████▏ | 183624/225000 [347:14:17<69:15:47,  6.03s/it]
 82%|████████▏ | 183625/225000 [347:14:21<61:26:05,  5.35s/it]
 82%|████████▏ | 183626/225000 [347:14:29<69:34:14,  6.05s/it]
 82%|████████▏ | 183627/225000 [347:14:33<62:46:27,  5.46s/it]
 82%|████████▏ | 183628/225000 [347:14:37<59:53:16,  5.21s/it]
 82%|████████▏ | 183629/225000 [347:14:42<58:29:29,  5.09s/it]
 82%|████████▏ | 183630/225000 [347:14:47<57:02:17,  4.96s/it]
 82%|████████▏ | 183631/225000 [347:14:51<55:35:07,  4.84s/it]
 82%|████████▏ | 183632/225000 [347:14:59<65:26:39,  5.70s/it]
 82%|████████▏ | 183633/225000 [347:15:10<83:15:53,  7.25s/it]
 82%|████████▏ | 183634/225000 [347:15:13<68:27:57,  5.96s/it]
 82%|████████▏ | 183635/225000 [347:15:19<68:00:33,  5.92s/it]
 82%|████████▏ | 183636/225000 [347:15:24<66:51:49,  5.82s/it]
 82%|████████▏ | 183637/225000 [347:15:28<60:13:47,  5.24s/it]
 82%|████████▏ | 183638/225000 [347:15:33<58:01:31,  5.05s/it]
 82%|████████▏ | 183639/225000 [347:15:42<71:14:15,  6.20s/it]
 82%|████████▏ | 183640/225000 [347:15:46<64:20:59,  5.60s/it]
 82%|████████▏ | 183641/225000 [347:15:50<59:51:37,  5.21s/it]
 82%|████████▏ | 183642/225000 [347:15:55<57:26:58,  5.00s/it]
 82%|████████▏ | 183643/225000 [347:15:59<54:48:00,  4.77s/it]
 82%|████████▏ | 183644/225000 [347:16:03<53:01:44,  4.62s/it]
 82%|████████▏ | 183645/225000 [347:16:15<79:14:18,  6.90s/it]
 82%|████████▏ | 183646/225000 [347:16:19<69:55:57,  6.09s/it]
 82%|████████▏ | 183647/225000 [347:16:23<62:48:35,  5.47s/it]
 82%|████████▏ | 183648/225000 [347:16:28<58:31:14,  5.09s/it]
 82%|████████▏ | 183649/225000 [347:16:38<74:47:42,  6.51s/it]
 82%|████████▏ | 183650/225000 [347:16:41<64:22:41,  5.60s/it]
                                                              
{'loss': '0.7642', 'grad_norm': '1.244', 'learning_rate': '4.444e-06', 'epoch': '0.8162', 'num_input_tokens_seen': 576047880, 'train_runtime': '1.25e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183650/225000 [347:16:41<64:22:41,  5.60s/it]
 82%|████████▏ | 183651/225000 [347:16:48<68:08:44,  5.93s/it]
 82%|████████▏ | 183652/225000 [347:16:51<59:49:34,  5.21s/it]
 82%|████████▏ | 183653/225000 [347:17:00<70:22:48,  6.13s/it]
 82%|████████▏ | 183654/225000 [347:17:03<61:43:36,  5.37s/it]
 82%|████████▏ | 183655/225000 [347:17:07<57:22:26,  5.00s/it]
 82%|████████▏ | 183656/225000 [347:17:14<62:29:46,  5.44s/it]
 82%|████████▏ | 183657/225000 [347:17:18<59:19:19,  5.17s/it]
 82%|████████▏ | 183658/225000 [347:17:23<56:51:34,  4.95s/it]
 82%|████████▏ | 183659/225000 [347:17:27<54:13:52,  4.72s/it]
 82%|████████▏ | 183660/225000 [347:17:32<54:12:27,  4.72s/it]
 82%|████████▏ | 183661/225000 [347:17:36<54:10:45,  4.72s/it]
 82%|████████▏ | 183662/225000 [347:17:41<53:24:48,  4.65s/it]
 82%|████████▏ | 183663/225000 [347:17:48<62:00:23,  5.40s/it]
 82%|████████▏ | 183664/225000 [347:17:58<79:10:34,  6.90s/it]
 82%|████████▏ | 183665/225000 [347:18:02<67:58:43,  5.92s/it]
 82%|████████▏ | 183666/225000 [347:18:05<59:17:25,  5.16s/it]
 82%|████████▏ | 183667/225000 [347:18:15<75:48:12,  6.60s/it]
 82%|████████▏ | 183668/225000 [347:18:19<64:15:28,  5.60s/it]
 82%|████████▏ | 183669/225000 [347:18:23<60:19:39,  5.25s/it]
 82%|████████▏ | 183670/225000 [347:18:30<65:30:52,  5.71s/it]
 82%|████████▏ | 183671/225000 [347:18:33<57:29:34,  5.01s/it]
 82%|████████▏ | 183672/225000 [347:18:37<54:56:56,  4.79s/it]
 82%|████████▏ | 183673/225000 [347:18:42<52:42:09,  4.59s/it]
 82%|████████▏ | 183674/225000 [347:18:46<52:38:39,  4.59s/it]
 82%|████████▏ | 183675/225000 [347:18:57<72:41:23,  6.33s/it]
 82%|████████▏ | 183676/225000 [347:19:02<69:57:40,  6.09s/it]
 82%|████████▏ | 183677/225000 [347:19:06<61:22:21,  5.35s/it]
 82%|████████▏ | 183678/225000 [347:19:12<63:01:29,  5.49s/it]
 82%|████████▏ | 183679/225000 [347:19:19<69:52:59,  6.09s/it]
 82%|████████▏ | 183680/225000 [347:19:23<62:37:54,  5.46s/it]
 82%|████████▏ | 183681/225000 [347:19:30<66:18:41,  5.78s/it]
 82%|████████▏ | 183682/225000 [347:19:38<74:20:14,  6.48s/it]
 82%|████████▏ | 183683/225000 [347:19:41<64:06:20,  5.59s/it]
 82%|████████▏ | 183684/225000 [347:19:45<57:53:45,  5.04s/it]
 82%|████████▏ | 183685/225000 [347:19:54<73:25:06,  6.40s/it]
 82%|████████▏ | 183686/225000 [347:20:01<72:49:08,  6.35s/it]
 82%|████████▏ | 183687/225000 [347:20:05<65:48:30,  5.73s/it]
 82%|████████▏ | 183688/225000 [347:20:10<62:30:17,  5.45s/it]
 82%|████████▏ | 183689/225000 [347:20:14<58:09:30,  5.07s/it]
 82%|████████▏ | 183690/225000 [347:20:18<53:41:05,  4.68s/it]
 82%|████████▏ | 183691/225000 [347:20:29<76:52:00,  6.70s/it]
 82%|████████▏ | 183692/225000 [347:20:34<70:05:17,  6.11s/it]
 82%|████████▏ | 183693/225000 [347:20:38<64:06:54,  5.59s/it]
 82%|████████▏ | 183694/225000 [347:20:46<71:22:05,  6.22s/it]
 82%|████████▏ | 183695/225000 [347:20:50<65:27:20,  5.70s/it]
 82%|████████▏ | 183696/225000 [347:20:57<67:20:22,  5.87s/it]
 82%|████████▏ | 183697/225000 [347:21:03<67:06:57,  5.85s/it]
 82%|████████▏ | 183698/225000 [347:21:07<62:42:00,  5.47s/it]
 82%|████████▏ | 183699/225000 [347:21:11<58:18:32,  5.08s/it]
 82%|████████▏ | 183700/225000 [347:21:15<54:41:37,  4.77s/it]
                                                              
{'loss': '0.7536', 'grad_norm': '1.168', 'learning_rate': '4.434e-06', 'epoch': '0.8164', 'num_input_tokens_seen': 576176984, 'train_runtime': '1.25e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183700/225000 [347:21:15<54:41:37,  4.77s/it]
 82%|████████▏ | 183701/225000 [347:21:20<54:08:03,  4.72s/it]
 82%|████████▏ | 183702/225000 [347:21:29<70:05:50,  6.11s/it]
 82%|████████▏ | 183703/225000 [347:21:34<66:50:53,  5.83s/it]
 82%|████████▏ | 183704/225000 [347:21:38<57:58:37,  5.05s/it]
 82%|████████▏ | 183705/225000 [347:21:45<66:25:35,  5.79s/it]
 82%|████████▏ | 183706/225000 [347:21:49<60:49:03,  5.30s/it]
 82%|████████▏ | 183707/225000 [347:21:54<58:21:15,  5.09s/it]
 82%|████████▏ | 183708/225000 [347:21:59<57:25:35,  5.01s/it]
 82%|████████▏ | 183709/225000 [347:22:04<57:07:41,  4.98s/it]
 82%|████████▏ | 183710/225000 [347:22:09<57:24:36,  5.01s/it]
 82%|████████▏ | 183711/225000 [347:22:14<57:27:13,  5.01s/it]
 82%|████████▏ | 183712/225000 [347:22:19<57:59:09,  5.06s/it]
 82%|████████▏ | 183713/225000 [347:22:24<58:24:11,  5.09s/it]
 82%|████████▏ | 183714/225000 [347:22:29<58:35:01,  5.11s/it]
 82%|████████▏ | 183715/225000 [347:22:41<80:40:07,  7.03s/it]
 82%|████████▏ | 183716/225000 [347:22:47<77:15:49,  6.74s/it]
 82%|████████▏ | 183717/225000 [347:22:51<70:08:10,  6.12s/it]
 82%|████████▏ | 183718/225000 [347:22:56<63:49:23,  5.57s/it]
 82%|████████▏ | 183719/225000 [347:23:00<59:55:39,  5.23s/it]
 82%|████████▏ | 183720/225000 [347:23:04<56:37:32,  4.94s/it]
 82%|████████▏ | 183721/225000 [347:23:14<72:50:40,  6.35s/it]
 82%|████████▏ | 183722/225000 [347:23:22<78:36:23,  6.86s/it]
 82%|████████▏ | 183723/225000 [347:23:26<67:17:06,  5.87s/it]
 82%|████████▏ | 183724/225000 [347:23:30<60:31:55,  5.28s/it]
 82%|████████▏ | 183725/225000 [347:23:39<74:09:54,  6.47s/it]
 82%|████████▏ | 183726/225000 [347:23:44<69:48:27,  6.09s/it]
 82%|████████▏ | 183727/225000 [347:23:49<66:31:45,  5.80s/it]
 82%|████████▏ | 183728/225000 [347:23:54<64:37:13,  5.64s/it]
 82%|████████▏ | 183729/225000 [347:24:00<63:08:57,  5.51s/it]
 82%|████████▏ | 183730/225000 [347:24:04<60:32:31,  5.28s/it]
 82%|████████▏ | 183731/225000 [347:24:09<57:27:21,  5.01s/it]
 82%|████████▏ | 183732/225000 [347:24:14<58:16:09,  5.08s/it]
 82%|████████▏ | 183733/225000 [347:24:19<56:05:13,  4.89s/it]
 82%|████████▏ | 183734/225000 [347:24:23<55:01:58,  4.80s/it]
 82%|████████▏ | 183735/225000 [347:24:28<54:28:49,  4.75s/it]
 82%|████████▏ | 183736/225000 [347:24:32<53:05:01,  4.63s/it]
 82%|████████▏ | 183737/225000 [347:24:42<72:58:24,  6.37s/it]
 82%|████████▏ | 183738/225000 [347:24:51<80:19:50,  7.01s/it]
 82%|████████▏ | 183739/225000 [347:24:55<70:42:28,  6.17s/it]
 82%|████████▏ | 183740/225000 [347:25:00<65:12:57,  5.69s/it]
 82%|████████▏ | 183741/225000 [347:25:04<60:44:21,  5.30s/it]
 82%|████████▏ | 183742/225000 [347:25:09<58:21:44,  5.09s/it]
 82%|████████▏ | 183743/225000 [347:25:13<56:47:21,  4.96s/it]
 82%|████████▏ | 183744/225000 [347:25:19<59:22:44,  5.18s/it]
 82%|████████▏ | 183745/225000 [347:25:32<85:25:13,  7.45s/it]
 82%|████████▏ | 183746/225000 [347:25:36<73:49:45,  6.44s/it]
 82%|████████▏ | 183747/225000 [347:25:40<65:16:28,  5.70s/it]
 82%|████████▏ | 183748/225000 [347:25:44<58:54:02,  5.14s/it]
 82%|████████▏ | 183749/225000 [347:25:48<56:27:11,  4.93s/it]
 82%|████████▏ | 183750/225000 [347:25:54<58:06:41,  5.07s/it]
                                                              
{'loss': '0.7659', 'grad_norm': '1.187', 'learning_rate': '4.423e-06', 'epoch': '0.8167', 'num_input_tokens_seen': 576307008, 'train_runtime': '1.251e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183750/225000 [347:25:54<58:06:41,  5.07s/it]
 82%|████████▏ | 183751/225000 [347:25:59<57:39:44,  5.03s/it]
 82%|████████▏ | 183752/225000 [347:26:04<58:45:01,  5.13s/it]
 82%|████████▏ | 183753/225000 [347:26:09<60:21:32,  5.27s/it]
 82%|████████▏ | 183754/225000 [347:26:19<75:39:36,  6.60s/it]
 82%|████████▏ | 183755/225000 [347:26:24<71:02:32,  6.20s/it]
 82%|████████▏ | 183756/225000 [347:26:32<76:35:10,  6.68s/it]
 82%|████████▏ | 183757/225000 [347:26:36<67:03:18,  5.85s/it]
 82%|████████▏ | 183758/225000 [347:26:44<75:26:34,  6.59s/it]
 82%|████████▏ | 183759/225000 [347:26:48<63:36:26,  5.55s/it]
 82%|████████▏ | 183760/225000 [347:26:56<73:03:00,  6.38s/it]
 82%|████████▏ | 183761/225000 [347:27:00<65:25:14,  5.71s/it]
 82%|████████▏ | 183762/225000 [347:27:05<61:23:40,  5.36s/it]
 82%|████████▏ | 183763/225000 [347:27:09<57:46:42,  5.04s/it]
 82%|████████▏ | 183764/225000 [347:27:13<53:50:03,  4.70s/it]
 82%|████████▏ | 183765/225000 [347:27:20<63:07:01,  5.51s/it]
 82%|████████▏ | 183766/225000 [347:27:29<73:57:22,  6.46s/it]
 82%|████████▏ | 183767/225000 [347:27:33<66:51:56,  5.84s/it]
 82%|████████▏ | 183768/225000 [347:27:37<60:46:46,  5.31s/it]
 82%|████████▏ | 183769/225000 [347:27:42<57:48:41,  5.05s/it]
 82%|████████▏ | 183770/225000 [347:27:46<54:12:10,  4.73s/it]
 82%|████████▏ | 183771/225000 [347:27:53<61:58:08,  5.41s/it]
 82%|████████▏ | 183772/225000 [347:28:02<74:27:09,  6.50s/it]
 82%|████████▏ | 183773/225000 [347:28:06<64:46:28,  5.66s/it]
 82%|████████▏ | 183774/225000 [347:28:14<75:41:18,  6.61s/it]
 82%|████████▏ | 183775/225000 [347:28:19<68:18:45,  5.97s/it]
 82%|████████▏ | 183776/225000 [347:28:23<62:52:25,  5.49s/it]
 82%|████████▏ | 183777/225000 [347:28:28<58:53:55,  5.14s/it]
 82%|████████▏ | 183778/225000 [347:28:32<55:29:42,  4.85s/it]
 82%|████████▏ | 183779/225000 [347:28:36<52:09:48,  4.56s/it]
 82%|████████▏ | 183780/225000 [347:28:43<63:14:29,  5.52s/it]
 82%|████████▏ | 183781/225000 [347:28:52<74:41:51,  6.52s/it]
 82%|████████▏ | 183782/225000 [347:28:55<63:17:49,  5.53s/it]
 82%|████████▏ | 183783/225000 [347:29:05<75:36:24,  6.60s/it]
 82%|████████▏ | 183784/225000 [347:29:07<62:57:32,  5.50s/it]
 82%|████████▏ | 183785/225000 [347:29:13<61:29:15,  5.37s/it]
 82%|████████▏ | 183786/225000 [347:29:18<60:20:50,  5.27s/it]
 82%|████████▏ | 183787/225000 [347:29:25<68:23:58,  5.97s/it]
 82%|████████▏ | 183788/225000 [347:29:32<72:05:37,  6.30s/it]
 82%|████████▏ | 183789/225000 [347:29:36<62:01:42,  5.42s/it]
 82%|████████▏ | 183790/225000 [347:29:40<57:31:19,  5.02s/it]
 82%|████████▏ | 183791/225000 [347:29:44<53:23:55,  4.66s/it]
 82%|████████▏ | 183792/225000 [347:29:50<59:19:53,  5.18s/it]
 82%|████████▏ | 183793/225000 [347:29:59<74:18:22,  6.49s/it]
 82%|████████▏ | 183794/225000 [347:30:03<64:17:46,  5.62s/it]
 82%|████████▏ | 183795/225000 [347:30:11<73:28:05,  6.42s/it]
 82%|████████▏ | 183796/225000 [347:30:17<71:22:40,  6.24s/it]
 82%|████████▏ | 183797/225000 [347:30:22<66:11:31,  5.78s/it]
 82%|████████▏ | 183798/225000 [347:30:26<61:33:48,  5.38s/it]
 82%|████████▏ | 183799/225000 [347:30:31<57:48:37,  5.05s/it]
 82%|████████▏ | 183800/225000 [347:30:35<57:12:21,  5.00s/it]
                                                              
{'loss': '0.7603', 'grad_norm': '1.093', 'learning_rate': '4.413e-06', 'epoch': '0.8169', 'num_input_tokens_seen': 576437288, 'train_runtime': '1.251e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183800/225000 [347:30:35<57:12:21,  5.00s/it]
 82%|████████▏ | 183801/225000 [347:30:45<73:48:33,  6.45s/it]
 82%|████████▏ | 183802/225000 [347:30:49<63:07:41,  5.52s/it]
 82%|████████▏ | 183803/225000 [347:30:52<55:50:47,  4.88s/it]
 82%|████████▏ | 183804/225000 [347:31:00<67:52:27,  5.93s/it]
 82%|████████▏ | 183805/225000 [347:31:06<65:57:22,  5.76s/it]
 82%|████████▏ | 183806/225000 [347:31:10<60:00:38,  5.24s/it]
 82%|████████▏ | 183807/225000 [347:31:18<68:53:19,  6.02s/it]
 82%|████████▏ | 183808/225000 [347:31:21<58:53:39,  5.15s/it]
 82%|████████▏ | 183809/225000 [347:31:24<54:02:06,  4.72s/it]
 82%|████████▏ | 183810/225000 [347:31:35<75:33:30,  6.60s/it]
 82%|████████▏ | 183811/225000 [347:31:40<67:58:35,  5.94s/it]
 82%|████████▏ | 183812/225000 [347:31:44<62:33:34,  5.47s/it]
 82%|████████▏ | 183813/225000 [347:31:48<57:32:26,  5.03s/it]
 82%|████████▏ | 183814/225000 [347:31:54<61:23:35,  5.37s/it]
 82%|████████▏ | 183815/225000 [347:32:05<77:49:08,  6.80s/it]
 82%|████████▏ | 183816/225000 [347:32:08<65:47:11,  5.75s/it]
 82%|████████▏ | 183817/225000 [347:32:16<75:25:03,  6.59s/it]
 82%|████████▏ | 183818/225000 [347:32:20<64:26:35,  5.63s/it]
 82%|████████▏ | 183819/225000 [347:32:28<73:30:23,  6.43s/it]
 82%|████████▏ | 183820/225000 [347:32:32<63:25:45,  5.55s/it]
 82%|████████▏ | 183821/225000 [347:32:35<57:26:08,  5.02s/it]
 82%|████████▏ | 183822/225000 [347:32:39<53:15:32,  4.66s/it]
 82%|████████▏ | 183823/225000 [347:32:44<52:58:55,  4.63s/it]
 82%|████████▏ | 183824/225000 [347:32:48<52:18:55,  4.57s/it]
 82%|████████▏ | 183825/225000 [347:33:00<78:44:28,  6.88s/it]
 82%|████████▏ | 183826/225000 [347:33:04<66:38:14,  5.83s/it]
 82%|████████▏ | 183827/225000 [347:33:07<59:09:40,  5.17s/it]
 82%|████████▏ | 183828/225000 [347:33:11<54:35:19,  4.77s/it]
 82%|████████▏ | 183829/225000 [347:33:20<66:33:58,  5.82s/it]
 82%|████████▏ | 183830/225000 [347:33:25<65:31:56,  5.73s/it]
 82%|████████▏ | 183831/225000 [347:33:32<69:44:51,  6.10s/it]
 82%|████████▏ | 183832/225000 [347:33:39<73:13:28,  6.40s/it]
 82%|████████▏ | 183833/225000 [347:33:44<67:11:31,  5.88s/it]
 82%|████████▏ | 183834/225000 [347:33:48<61:36:15,  5.39s/it]
 82%|████████▏ | 183835/225000 [347:33:52<56:37:36,  4.95s/it]
 82%|████████▏ | 183836/225000 [347:33:56<53:58:49,  4.72s/it]
 82%|████████▏ | 183837/225000 [347:34:05<69:11:29,  6.05s/it]
 82%|████████▏ | 183838/225000 [347:34:11<68:44:14,  6.01s/it]
 82%|████████▏ | 183839/225000 [347:34:15<60:06:38,  5.26s/it]
 82%|████████▏ | 183840/225000 [347:34:24<72:42:42,  6.36s/it]
 82%|████████▏ | 183841/225000 [347:34:27<62:26:44,  5.46s/it]
 82%|████████▏ | 183842/225000 [347:34:35<71:28:03,  6.25s/it]
 82%|████████▏ | 183843/225000 [347:34:39<62:37:31,  5.48s/it]
 82%|████████▏ | 183844/225000 [347:34:43<56:42:56,  4.96s/it]
 82%|████████▏ | 183845/225000 [347:34:52<73:22:14,  6.42s/it]
 82%|████████▏ | 183846/225000 [347:34:56<65:14:42,  5.71s/it]
 82%|████████▏ | 183847/225000 [347:35:01<60:04:35,  5.26s/it]
 82%|████████▏ | 183848/225000 [347:35:05<56:41:41,  4.96s/it]
 82%|████████▏ | 183849/225000 [347:35:12<62:42:20,  5.49s/it]
 82%|████████▏ | 183850/225000 [347:35:19<67:40:09,  5.92s/it]
                                                              
{'loss': '0.7578', 'grad_norm': '1.206', 'learning_rate': '4.403e-06', 'epoch': '0.8171', 'num_input_tokens_seen': 576567776, 'train_runtime': '1.251e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183850/225000 [347:35:19<67:40:09,  5.92s/it]
 82%|████████▏ | 183851/225000 [347:35:23<62:52:27,  5.50s/it]
 82%|████████▏ | 183852/225000 [347:35:31<70:45:49,  6.19s/it]
 82%|████████▏ | 183853/225000 [347:35:34<61:25:43,  5.37s/it]
 82%|████████▏ | 183854/225000 [347:35:43<71:36:51,  6.27s/it]
 82%|████████▏ | 183855/225000 [347:35:47<64:18:02,  5.63s/it]
 82%|████████▏ | 183856/225000 [347:35:54<68:40:39,  6.01s/it]
 82%|████████▏ | 183857/225000 [347:35:58<64:11:13,  5.62s/it]
 82%|████████▏ | 183858/225000 [347:36:03<62:02:09,  5.43s/it]
 82%|████████▏ | 183859/225000 [347:36:08<60:14:51,  5.27s/it]
 82%|████████▏ | 183860/225000 [347:36:13<58:51:41,  5.15s/it]
 82%|████████▏ | 183861/225000 [347:36:18<58:18:27,  5.10s/it]
 82%|████████▏ | 183862/225000 [347:36:23<58:21:26,  5.11s/it]
 82%|████████▏ | 183863/225000 [347:36:28<56:14:31,  4.92s/it]
 82%|████████▏ | 183864/225000 [347:36:33<56:13:24,  4.92s/it]
 82%|████████▏ | 183865/225000 [347:36:39<61:05:59,  5.35s/it]
 82%|████████▏ | 183866/225000 [347:36:51<82:11:48,  7.19s/it]
 82%|████████▏ | 183867/225000 [347:36:54<69:17:15,  6.06s/it]
 82%|████████▏ | 183868/225000 [347:37:03<78:08:43,  6.84s/it]
 82%|████████▏ | 183869/225000 [347:37:06<66:22:25,  5.81s/it]
 82%|████████▏ | 183870/225000 [347:37:10<58:28:40,  5.12s/it]
 82%|████████▏ | 183871/225000 [347:37:19<71:40:03,  6.27s/it]
 82%|████████▏ | 183872/225000 [347:37:24<67:21:43,  5.90s/it]
 82%|████████▏ | 183873/225000 [347:37:28<62:10:26,  5.44s/it]
 82%|████████▏ | 183874/225000 [347:37:33<60:26:11,  5.29s/it]
 82%|████████▏ | 183875/225000 [347:37:38<60:15:34,  5.27s/it]
 82%|████████▏ | 183876/225000 [347:37:43<57:54:38,  5.07s/it]
 82%|████████▏ | 183877/225000 [347:37:48<57:37:43,  5.04s/it]
 82%|████████▏ | 183878/225000 [347:37:55<64:34:34,  5.65s/it]
 82%|████████▏ | 183879/225000 [347:38:04<76:41:47,  6.71s/it]
 82%|████████▏ | 183880/225000 [347:38:07<65:04:25,  5.70s/it]
 82%|████████▏ | 183881/225000 [347:38:12<62:24:11,  5.46s/it]
 82%|████████▏ | 183882/225000 [347:38:17<61:30:13,  5.38s/it]
 82%|████████▏ | 183883/225000 [347:38:23<61:09:29,  5.35s/it]
 82%|████████▏ | 183884/225000 [347:38:28<60:12:16,  5.27s/it]
 82%|████████▏ | 183885/225000 [347:38:33<59:56:55,  5.25s/it]
 82%|████████▏ | 183886/225000 [347:38:38<59:51:42,  5.24s/it]
 82%|████████▏ | 183887/225000 [347:38:45<66:57:48,  5.86s/it]
 82%|████████▏ | 183888/225000 [347:38:52<68:29:00,  6.00s/it]
 82%|████████▏ | 183889/225000 [347:38:57<66:04:44,  5.79s/it]
 82%|████████▏ | 183890/225000 [347:39:02<62:52:45,  5.51s/it]
 82%|████████▏ | 183891/225000 [347:39:08<63:29:01,  5.56s/it]
 82%|████████▏ | 183892/225000 [347:39:12<61:09:48,  5.36s/it]
 82%|████████▏ | 183893/225000 [347:39:18<61:14:29,  5.36s/it]
 82%|████████▏ | 183894/225000 [347:39:24<62:11:16,  5.45s/it]
 82%|████████▏ | 183895/225000 [347:39:31<70:49:09,  6.20s/it]
 82%|████████▏ | 183896/225000 [347:39:37<67:07:40,  5.88s/it]
 82%|████████▏ | 183897/225000 [347:39:42<66:39:56,  5.84s/it]
 82%|████████▏ | 183898/225000 [347:39:48<64:24:41,  5.64s/it]
 82%|████████▏ | 183899/225000 [347:39:53<62:28:51,  5.47s/it]
 82%|████████▏ | 183900/225000 [347:39:58<62:33:56,  5.48s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '1.115', 'learning_rate': '4.392e-06', 'epoch': '0.8173', 'num_input_tokens_seen': 576697112, 'train_runtime': '1.252e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183900/225000 [347:39:58<62:33:56,  5.48s/it]
 82%|████████▏ | 183901/225000 [347:40:03<62:02:42,  5.43s/it]
 82%|████████▏ | 183902/225000 [347:40:09<63:32:21,  5.57s/it]
 82%|████████▏ | 183903/225000 [347:40:15<63:11:17,  5.54s/it]
 82%|████████▏ | 183904/225000 [347:40:20<62:03:04,  5.44s/it]
 82%|████████▏ | 183905/225000 [347:40:25<60:46:34,  5.32s/it]
 82%|████████▏ | 183906/225000 [347:40:30<61:08:53,  5.36s/it]
 82%|████████▏ | 183907/225000 [347:40:38<69:10:04,  6.06s/it]
 82%|████████▏ | 183908/225000 [347:40:45<70:29:01,  6.17s/it]
 82%|████████▏ | 183909/225000 [347:40:50<67:10:32,  5.89s/it]
 82%|████████▏ | 183910/225000 [347:40:55<63:59:03,  5.61s/it]
 82%|████████▏ | 183911/225000 [347:41:00<62:56:49,  5.52s/it]
 82%|████████▏ | 183912/225000 [347:41:07<67:51:26,  5.95s/it]
 82%|████████▏ | 183913/225000 [347:41:12<64:46:45,  5.68s/it]
 82%|████████▏ | 183914/225000 [347:41:18<63:55:35,  5.60s/it]
 82%|████████▏ | 183915/225000 [347:41:23<62:56:25,  5.52s/it]
 82%|████████▏ | 183916/225000 [347:41:28<63:24:31,  5.56s/it]
 82%|████████▏ | 183917/225000 [347:41:34<63:01:49,  5.52s/it]
 82%|████████▏ | 183918/225000 [347:41:39<61:53:23,  5.42s/it]
 82%|████████▏ | 183919/225000 [347:41:45<62:51:54,  5.51s/it]
 82%|████████▏ | 183920/225000 [347:41:50<62:47:51,  5.50s/it]
 82%|████████▏ | 183921/225000 [347:41:56<62:21:05,  5.46s/it]
 82%|████████▏ | 183922/225000 [347:42:04<71:56:27,  6.30s/it]
 82%|████████▏ | 183923/225000 [347:42:11<75:00:42,  6.57s/it]
 82%|████████▏ | 183924/225000 [347:42:17<71:12:34,  6.24s/it]
 82%|████████▏ | 183925/225000 [347:42:23<72:39:56,  6.37s/it]
 82%|████████▏ | 183926/225000 [347:42:27<62:55:48,  5.52s/it]
 82%|████████▏ | 183927/225000 [347:42:35<70:50:13,  6.21s/it]
 82%|████████▏ | 183928/225000 [347:42:41<71:35:26,  6.27s/it]
 82%|████████▏ | 183929/225000 [347:42:46<66:39:55,  5.84s/it]
 82%|████████▏ | 183930/225000 [347:42:51<63:11:47,  5.54s/it]
 82%|████████▏ | 183931/225000 [347:42:55<59:31:13,  5.22s/it]
 82%|████████▏ | 183932/225000 [347:43:00<57:10:27,  5.01s/it]
 82%|████████▏ | 183933/225000 [347:43:05<58:14:43,  5.11s/it]
 82%|████████▏ | 183934/225000 [347:43:10<58:48:57,  5.16s/it]
 82%|████████▏ | 183935/225000 [347:43:15<57:55:06,  5.08s/it]
 82%|████████▏ | 183936/225000 [347:43:20<57:44:15,  5.06s/it]
 82%|████████▏ | 183937/225000 [347:43:25<57:24:13,  5.03s/it]
 82%|████████▏ | 183938/225000 [347:43:37<80:23:25,  7.05s/it]
 82%|████████▏ | 183939/225000 [347:43:40<67:23:40,  5.91s/it]
 82%|████████▏ | 183940/225000 [347:43:47<72:07:23,  6.32s/it]
 82%|████████▏ | 183941/225000 [347:43:52<66:47:03,  5.86s/it]
 82%|████████▏ | 183942/225000 [347:43:57<62:40:14,  5.50s/it]
 82%|████████▏ | 183943/225000 [347:44:01<59:10:43,  5.19s/it]
 82%|████████▏ | 183944/225000 [347:44:06<56:33:40,  4.96s/it]
 82%|████████▏ | 183945/225000 [347:44:10<55:14:03,  4.84s/it]
 82%|████████▏ | 183946/225000 [347:44:15<55:01:57,  4.83s/it]
 82%|████████▏ | 183947/225000 [347:44:20<54:10:54,  4.75s/it]
 82%|████████▏ | 183948/225000 [347:44:24<52:14:57,  4.58s/it]
 82%|████████▏ | 183949/225000 [347:44:29<52:35:11,  4.61s/it]
 82%|████████▏ | 183950/225000 [347:44:41<77:54:47,  6.83s/it]
                                                              
{'loss': '0.7509', 'grad_norm': '1.272', 'learning_rate': '4.382e-06', 'epoch': '0.8176', 'num_input_tokens_seen': 576827536, 'train_runtime': '1.252e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 183950/225000 [347:44:41<77:54:47,  6.83s/it]
 82%|████████▏ | 183951/225000 [347:44:46<73:18:10,  6.43s/it]
 82%|████████▏ | 183952/225000 [347:44:50<65:24:53,  5.74s/it]
 82%|████████▏ | 183953/225000 [347:44:55<60:22:27,  5.30s/it]
 82%|████████▏ | 183954/225000 [347:44:59<56:46:55,  4.98s/it]
 82%|████████▏ | 183955/225000 [347:45:11<80:05:39,  7.02s/it]
 82%|████████▏ | 183956/225000 [347:45:15<70:00:17,  6.14s/it]
 82%|████████▏ | 183957/225000 [347:45:18<60:34:59,  5.31s/it]
 82%|████████▏ | 183958/225000 [347:45:28<75:11:06,  6.59s/it]
 82%|████████▏ | 183959/225000 [347:45:31<63:44:11,  5.59s/it]
 82%|████████▏ | 183960/225000 [347:45:38<69:47:06,  6.12s/it]
 82%|████████▏ | 183961/225000 [347:45:42<61:00:40,  5.35s/it]
 82%|████████▏ | 183962/225000 [347:45:49<65:58:30,  5.79s/it]
 82%|████████▏ | 183963/225000 [347:45:54<66:26:41,  5.83s/it]
 82%|████████▏ | 183964/225000 [347:45:58<58:53:19,  5.17s/it]
 82%|████████▏ | 183965/225000 [347:46:03<56:58:09,  5.00s/it]
 82%|████████▏ | 183966/225000 [347:46:07<55:46:25,  4.89s/it]
 82%|████████▏ | 183967/225000 [347:46:12<55:41:07,  4.89s/it]
 82%|████████▏ | 183968/225000 [347:46:17<55:37:07,  4.88s/it]
 82%|████████▏ | 183969/225000 [347:46:22<55:38:15,  4.88s/it]
 82%|████████▏ | 183970/225000 [347:46:34<80:38:48,  7.08s/it]
 82%|████████▏ | 183971/225000 [347:46:38<68:42:34,  6.03s/it]
 82%|████████▏ | 183972/225000 [347:46:41<60:51:21,  5.34s/it]
 82%|████████▏ | 183973/225000 [347:46:50<70:26:48,  6.18s/it]
 82%|████████▏ | 183974/225000 [347:46:54<64:30:44,  5.66s/it]
 82%|████████▏ | 183975/225000 [347:46:59<61:52:16,  5.43s/it]
 82%|████████▏ | 183976/225000 [347:47:04<60:14:09,  5.29s/it]
 82%|████████▏ | 183977/225000 [347:47:08<57:32:48,  5.05s/it]
 82%|████████▏ | 183978/225000 [347:47:13<55:49:50,  4.90s/it]
 82%|████████▏ | 183979/225000 [347:47:18<55:13:01,  4.85s/it]
 82%|████████▏ | 183980/225000 [347:47:23<56:39:45,  4.97s/it]
 82%|████████▏ | 183981/225000 [347:47:28<55:21:21,  4.86s/it]
 82%|████████▏ | 183982/225000 [347:47:37<69:35:48,  6.11s/it]
 82%|████████▏ | 183983/225000 [347:47:45<79:10:45,  6.95s/it]
 82%|████████▏ | 183984/225000 [347:47:51<73:33:20,  6.46s/it]
 82%|████████▏ | 183985/225000 [347:47:58<74:54:32,  6.57s/it]
 82%|████████▏ | 183986/225000 [347:48:02<69:01:43,  6.06s/it]
 82%|████████▏ | 183987/225000 [347:48:08<67:21:03,  5.91s/it]
 82%|████████▏ | 183988/225000 [347:48:13<64:15:06,  5.64s/it]
 82%|████████▏ | 183989/225000 [347:48:19<63:36:06,  5.58s/it]
 82%|████████▏ | 183990/225000 [347:48:24<62:27:26,  5.48s/it]
 82%|████████▏ | 183991/225000 [347:48:29<61:58:17,  5.44s/it]
 82%|████████▏ | 183992/225000 [347:48:34<60:51:00,  5.34s/it]
 82%|████████▏ | 183993/225000 [347:48:40<61:49:09,  5.43s/it]
 82%|████████▏ | 183994/225000 [347:48:46<64:54:11,  5.70s/it]
 82%|████████▏ | 183995/225000 [347:48:51<63:22:39,  5.56s/it]
 82%|████████▏ | 183996/225000 [347:48:57<62:45:27,  5.51s/it]
 82%|████████▏ | 183997/225000 [347:49:06<73:38:25,  6.47s/it]
 82%|████████▏ | 183998/225000 [347:49:09<64:18:33,  5.65s/it]
 82%|████████▏ | 183999/225000 [347:49:13<58:21:59,  5.12s/it]
 82%|████████▏ | 184000/225000 [347:49:22<71:38:55,  6.29s/it]
                                                              
{'loss': '0.7628', 'grad_norm': '1.314', 'learning_rate': '4.372e-06', 'epoch': '0.8178', 'num_input_tokens_seen': 576958040, 'train_runtime': '1.252e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184000/225000 [347:49:22<71:38:55,  6.29s/it]
 82%|████████▏ | 184001/225000 [347:49:26<62:50:08,  5.52s/it]
 82%|████████▏ | 184002/225000 [347:49:33<69:45:15,  6.13s/it]
 82%|████████▏ | 184003/225000 [347:49:38<62:56:25,  5.53s/it]
 82%|████████▏ | 184004/225000 [347:49:42<58:22:03,  5.13s/it]
 82%|████████▏ | 184005/225000 [347:49:46<54:58:29,  4.83s/it]
 82%|████████▏ | 184006/225000 [347:49:50<52:23:48,  4.60s/it]
 82%|████████▏ | 184007/225000 [347:49:59<67:20:58,  5.91s/it]
 82%|████████▏ | 184008/225000 [347:50:04<65:12:16,  5.73s/it]
 82%|████████▏ | 184009/225000 [347:50:09<62:33:10,  5.49s/it]
 82%|████████▏ | 184010/225000 [347:50:16<67:22:31,  5.92s/it]
 82%|████████▏ | 184011/225000 [347:50:20<60:03:56,  5.28s/it]
 82%|████████▏ | 184012/225000 [347:50:25<58:49:24,  5.17s/it]
 82%|████████▏ | 184013/225000 [347:50:30<60:22:25,  5.30s/it]
 82%|████████▏ | 184014/225000 [347:50:36<60:52:01,  5.35s/it]
 82%|████████▏ | 184015/225000 [347:50:46<76:39:12,  6.73s/it]
 82%|████████▏ | 184016/225000 [347:50:50<66:38:31,  5.85s/it]
 82%|████████▏ | 184017/225000 [347:50:53<59:51:05,  5.26s/it]
 82%|████████▏ | 184018/225000 [347:51:00<63:18:48,  5.56s/it]
 82%|████████▏ | 184019/225000 [347:51:07<68:10:08,  5.99s/it]
 82%|████████▏ | 184020/225000 [347:51:10<58:41:16,  5.16s/it]
 82%|████████▏ | 184021/225000 [347:51:14<56:30:43,  4.96s/it]
 82%|████████▏ | 184022/225000 [347:51:20<58:38:29,  5.15s/it]
 82%|████████▏ | 184023/225000 [347:51:25<57:30:57,  5.05s/it]
 82%|████████▏ | 184024/225000 [347:51:30<58:02:08,  5.10s/it]
 82%|████████▏ | 184025/225000 [347:51:35<56:19:10,  4.95s/it]
 82%|████████▏ | 184026/225000 [347:51:40<57:00:02,  5.01s/it]
 82%|████████▏ | 184027/225000 [347:51:45<58:56:08,  5.18s/it]
 82%|████████▏ | 184028/225000 [347:51:51<59:11:53,  5.20s/it]
 82%|████████▏ | 184029/225000 [347:51:55<57:25:31,  5.05s/it]
 82%|████████▏ | 184030/225000 [347:52:04<68:27:07,  6.01s/it]
 82%|████████▏ | 184031/225000 [347:52:14<84:00:32,  7.38s/it]
 82%|████████▏ | 184032/225000 [347:52:19<76:22:50,  6.71s/it]
 82%|████████▏ | 184033/225000 [347:52:24<69:45:11,  6.13s/it]
 82%|████████▏ | 184034/225000 [347:52:29<64:18:32,  5.65s/it]
 82%|████████▏ | 184035/225000 [347:52:35<66:04:31,  5.81s/it]
 82%|████████▏ | 184036/225000 [347:52:38<58:14:14,  5.12s/it]
 82%|████████▏ | 184037/225000 [347:52:47<70:05:47,  6.16s/it]
 82%|████████▏ | 184038/225000 [347:52:51<62:21:14,  5.48s/it]
 82%|████████▏ | 184039/225000 [347:52:54<55:10:52,  4.85s/it]
 82%|████████▏ | 184040/225000 [347:53:05<74:44:37,  6.57s/it]
 82%|████████▏ | 184041/225000 [347:53:08<63:30:46,  5.58s/it]
 82%|████████▏ | 184042/225000 [347:53:14<66:25:27,  5.84s/it]
 82%|████████▏ | 184043/225000 [347:53:19<62:21:41,  5.48s/it]
 82%|████████▏ | 184044/225000 [347:53:24<59:43:50,  5.25s/it]
 82%|████████▏ | 184045/225000 [347:53:28<57:35:23,  5.06s/it]
 82%|████████▏ | 184046/225000 [347:53:33<56:06:01,  4.93s/it]
 82%|████████▏ | 184047/225000 [347:53:38<56:50:56,  5.00s/it]
 82%|████████▏ | 184048/225000 [347:53:43<57:04:14,  5.02s/it]
 82%|████████▏ | 184049/225000 [347:53:48<57:30:05,  5.05s/it]
 82%|████████▏ | 184050/225000 [347:53:53<55:58:50,  4.92s/it]
                                                              
{'loss': '0.7685', 'grad_norm': '1.278', 'learning_rate': '4.361e-06', 'epoch': '0.818', 'num_input_tokens_seen': 577087208, 'train_runtime': '1.252e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184050/225000 [347:53:53<55:58:50,  4.92s/it]
 82%|████████▏ | 184051/225000 [347:54:01<66:41:36,  5.86s/it]
 82%|████████▏ | 184052/225000 [347:54:11<79:23:29,  6.98s/it]
 82%|████████▏ | 184053/225000 [347:54:16<72:47:52,  6.40s/it]
 82%|████████▏ | 184054/225000 [347:54:21<67:30:16,  5.94s/it]
 82%|████████▏ | 184055/225000 [347:54:26<64:41:25,  5.69s/it]
 82%|████████▏ | 184056/225000 [347:54:31<63:33:34,  5.59s/it]
 82%|████████▏ | 184057/225000 [347:54:36<61:35:22,  5.42s/it]
 82%|████████▏ | 184058/225000 [347:54:40<58:03:50,  5.11s/it]
 82%|████████▏ | 184059/225000 [347:54:46<59:19:38,  5.22s/it]
 82%|████████▏ | 184060/225000 [347:54:51<58:33:50,  5.15s/it]
 82%|████████▏ | 184061/225000 [347:54:56<58:43:50,  5.16s/it]
 82%|████████▏ | 184062/225000 [347:55:01<58:52:08,  5.18s/it]
 82%|████████▏ | 184063/225000 [347:55:12<76:31:31,  6.73s/it]
 82%|████████▏ | 184064/225000 [347:55:17<71:46:05,  6.31s/it]
 82%|████████▏ | 184065/225000 [347:55:20<61:30:56,  5.41s/it]
 82%|████████▏ | 184066/225000 [347:55:28<70:55:57,  6.24s/it]
 82%|████████▏ | 184067/225000 [347:55:32<62:57:14,  5.54s/it]
 82%|████████▏ | 184068/225000 [347:55:37<61:11:47,  5.38s/it]
 82%|████████▏ | 184069/225000 [347:55:43<61:19:19,  5.39s/it]
 82%|████████▏ | 184070/225000 [347:55:48<61:13:58,  5.39s/it]
 82%|████████▏ | 184071/225000 [347:55:54<61:28:29,  5.41s/it]
 82%|████████▏ | 184072/225000 [347:55:59<61:22:10,  5.40s/it]
 82%|████████▏ | 184073/225000 [347:56:05<62:10:22,  5.47s/it]
 82%|████████▏ | 184074/225000 [347:56:15<79:05:57,  6.96s/it]
 82%|████████▏ | 184075/225000 [347:56:18<66:02:43,  5.81s/it]
 82%|████████▏ | 184076/225000 [347:56:23<62:19:31,  5.48s/it]
 82%|████████▏ | 184077/225000 [347:56:28<59:36:09,  5.24s/it]
 82%|████████▏ | 184078/225000 [347:56:33<59:06:42,  5.20s/it]
 82%|████████▏ | 184079/225000 [347:56:38<59:23:48,  5.23s/it]
 82%|████████▏ | 184080/225000 [347:56:43<58:40:09,  5.16s/it]
 82%|████████▏ | 184081/225000 [347:56:48<59:18:24,  5.22s/it]
 82%|████████▏ | 184082/225000 [347:56:53<58:34:11,  5.15s/it]
 82%|████████▏ | 184083/225000 [347:56:58<57:37:09,  5.07s/it]
 82%|████████▏ | 184084/225000 [347:57:03<57:55:29,  5.10s/it]
 82%|████████▏ | 184085/225000 [347:57:08<57:30:00,  5.06s/it]
 82%|████████▏ | 184086/225000 [347:57:13<57:16:16,  5.04s/it]
 82%|████████▏ | 184087/225000 [347:57:24<76:03:05,  6.69s/it]
 82%|████████▏ | 184088/225000 [347:57:29<70:54:26,  6.24s/it]
 82%|████████▏ | 184089/225000 [347:57:35<68:40:59,  6.04s/it]
 82%|████████▏ | 184090/225000 [347:57:40<65:33:18,  5.77s/it]
 82%|████████▏ | 184091/225000 [347:57:45<64:02:35,  5.64s/it]
 82%|████████▏ | 184092/225000 [347:57:50<61:51:25,  5.44s/it]
 82%|████████▏ | 184093/225000 [347:57:55<60:19:17,  5.31s/it]
 82%|████████▏ | 184094/225000 [347:58:01<61:09:25,  5.38s/it]
 82%|████████▏ | 184095/225000 [347:58:06<60:12:04,  5.30s/it]
 82%|████████▏ | 184096/225000 [347:58:11<60:47:52,  5.35s/it]
 82%|████████▏ | 184097/225000 [347:58:17<60:27:09,  5.32s/it]
 82%|████████▏ | 184098/225000 [347:58:22<61:59:25,  5.46s/it]
 82%|████████▏ | 184099/225000 [347:58:28<61:09:13,  5.38s/it]
 82%|████████▏ | 184100/225000 [347:58:33<60:57:35,  5.37s/it]
                                                              
{'loss': '0.7653', 'grad_norm': '1.131', 'learning_rate': '4.351e-06', 'epoch': '0.8182', 'num_input_tokens_seen': 577216712, 'train_runtime': '1.253e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184100/225000 [347:58:33<60:57:35,  5.37s/it]
 82%|████████▏ | 184101/225000 [347:58:42<75:17:30,  6.63s/it]
 82%|████████▏ | 184102/225000 [347:58:51<82:00:17,  7.22s/it]
 82%|████████▏ | 184103/225000 [347:58:55<72:40:58,  6.40s/it]
 82%|████████▏ | 184104/225000 [347:59:00<64:53:16,  5.71s/it]
 82%|████████▏ | 184105/225000 [347:59:04<60:05:14,  5.29s/it]
 82%|████████▏ | 184106/225000 [347:59:15<79:07:48,  6.97s/it]
 82%|████████▏ | 184107/225000 [347:59:19<68:17:10,  6.01s/it]
 82%|████████▏ | 184108/225000 [347:59:23<62:00:00,  5.46s/it]
 82%|████████▏ | 184109/225000 [347:59:27<57:35:30,  5.07s/it]
 82%|████████▏ | 184110/225000 [347:59:37<74:40:10,  6.57s/it]
 82%|████████▏ | 184111/225000 [347:59:42<68:24:32,  6.02s/it]
 82%|████████▏ | 184112/225000 [347:59:48<70:54:34,  6.24s/it]
 82%|████████▏ | 184113/225000 [347:59:53<65:10:44,  5.74s/it]
 82%|████████▏ | 184114/225000 [347:59:57<58:17:50,  5.13s/it]
 82%|████████▏ | 184115/225000 [348:00:02<59:26:14,  5.23s/it]
 82%|████████▏ | 184116/225000 [348:00:09<65:03:38,  5.73s/it]
 82%|████████▏ | 184117/225000 [348:00:15<67:04:36,  5.91s/it]
 82%|████████▏ | 184118/225000 [348:00:20<62:40:54,  5.52s/it]
 82%|████████▏ | 184119/225000 [348:00:25<59:22:08,  5.23s/it]
 82%|████████▏ | 184120/225000 [348:00:32<65:59:27,  5.81s/it]
 82%|████████▏ | 184121/225000 [348:00:38<67:26:45,  5.94s/it]
 82%|████████▏ | 184122/225000 [348:00:44<67:22:31,  5.93s/it]
 82%|████████▏ | 184123/225000 [348:00:50<69:05:20,  6.08s/it]
 82%|████████▏ | 184124/225000 [348:00:55<65:44:45,  5.79s/it]
 82%|████████▏ | 184125/225000 [348:01:00<62:36:26,  5.51s/it]
 82%|████████▏ | 184126/225000 [348:01:05<61:23:36,  5.41s/it]
 82%|████████▏ | 184127/225000 [348:01:11<60:08:18,  5.30s/it]
 82%|████████▏ | 184128/225000 [348:01:16<59:18:14,  5.22s/it]
 82%|████████▏ | 184129/225000 [348:01:20<57:39:11,  5.08s/it]
 82%|████████▏ | 184130/225000 [348:01:25<57:15:35,  5.04s/it]
 82%|████████▏ | 184131/225000 [348:01:30<57:40:14,  5.08s/it]
 82%|████████▏ | 184132/225000 [348:01:35<56:57:31,  5.02s/it]
 82%|████████▏ | 184133/225000 [348:01:40<57:06:24,  5.03s/it]
 82%|████████▏ | 184134/225000 [348:01:45<55:54:53,  4.93s/it]
 82%|████████▏ | 184135/225000 [348:01:50<56:38:17,  4.99s/it]
 82%|████████▏ | 184136/225000 [348:01:55<57:02:11,  5.02s/it]
 82%|████████▏ | 184137/225000 [348:02:00<56:11:30,  4.95s/it]
 82%|████████▏ | 184138/225000 [348:02:05<55:44:40,  4.91s/it]
 82%|████████▏ | 184139/225000 [348:02:17<81:08:31,  7.15s/it]
 82%|████████▏ | 184140/225000 [348:02:24<81:01:03,  7.14s/it]
 82%|████████▏ | 184141/225000 [348:02:28<68:41:25,  6.05s/it]
 82%|████████▏ | 184142/225000 [348:02:33<65:11:09,  5.74s/it]
 82%|████████▏ | 184143/225000 [348:02:39<64:42:49,  5.70s/it]
 82%|████████▏ | 184144/225000 [348:02:44<62:29:48,  5.51s/it]
 82%|████████▏ | 184145/225000 [348:02:49<62:39:56,  5.52s/it]
 82%|████████▏ | 184146/225000 [348:02:58<73:00:43,  6.43s/it]
 82%|████████▏ | 184147/225000 [348:03:03<70:05:22,  6.18s/it]
 82%|████████▏ | 184148/225000 [348:03:08<66:24:59,  5.85s/it]
 82%|████████▏ | 184149/225000 [348:03:14<65:11:54,  5.75s/it]
 82%|████████▏ | 184150/225000 [348:03:21<69:59:38,  6.17s/it]
                                                              
{'loss': '0.7792', 'grad_norm': '1.118', 'learning_rate': '4.341e-06', 'epoch': '0.8184', 'num_input_tokens_seen': 577347480, 'train_runtime': '1.253e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184150/225000 [348:03:21<69:59:38,  6.17s/it]
 82%|████████▏ | 184151/225000 [348:03:25<60:54:50,  5.37s/it]
 82%|████████▏ | 184152/225000 [348:03:29<59:17:10,  5.22s/it]
 82%|████████▏ | 184153/225000 [348:03:34<56:53:35,  5.01s/it]
 82%|████████▏ | 184154/225000 [348:03:39<56:02:59,  4.94s/it]
 82%|████████▏ | 184155/225000 [348:03:49<75:38:46,  6.67s/it]
 82%|████████▏ | 184156/225000 [348:03:54<70:03:20,  6.17s/it]
 82%|████████▏ | 184157/225000 [348:03:58<62:31:24,  5.51s/it]
 82%|████████▏ | 184158/225000 [348:04:07<72:52:06,  6.42s/it]
 82%|████████▏ | 184159/225000 [348:04:12<67:29:41,  5.95s/it]
 82%|████████▏ | 184160/225000 [348:04:17<65:08:58,  5.74s/it]
 82%|████████▏ | 184161/225000 [348:04:21<59:09:32,  5.21s/it]
 82%|████████▏ | 184162/225000 [348:04:25<55:30:55,  4.89s/it]
 82%|████████▏ | 184163/225000 [348:04:36<77:16:30,  6.81s/it]
 82%|████████▏ | 184164/225000 [348:04:40<65:50:12,  5.80s/it]
 82%|████████▏ | 184165/225000 [348:04:47<70:53:37,  6.25s/it]
 82%|████████▏ | 184166/225000 [348:04:51<63:58:13,  5.64s/it]
 82%|████████▏ | 184167/225000 [348:04:56<59:18:18,  5.23s/it]
 82%|████████▏ | 184168/225000 [348:05:00<55:54:43,  4.93s/it]
 82%|████████▏ | 184169/225000 [348:05:04<53:33:12,  4.72s/it]
 82%|████████▏ | 184170/225000 [348:05:08<52:13:27,  4.60s/it]
 82%|████████▏ | 184171/225000 [348:05:20<75:17:13,  6.64s/it]
 82%|████████▏ | 184172/225000 [348:05:24<68:21:20,  6.03s/it]
 82%|████████▏ | 184173/225000 [348:05:28<59:28:12,  5.24s/it]
 82%|████████▏ | 184174/225000 [348:05:33<57:55:08,  5.11s/it]
 82%|████████▏ | 184175/225000 [348:05:37<56:24:52,  4.97s/it]
 82%|████████▏ | 184176/225000 [348:05:43<57:30:48,  5.07s/it]
 82%|████████▏ | 184177/225000 [348:05:47<56:06:09,  4.95s/it]
 82%|████████▏ | 184178/225000 [348:05:52<56:14:03,  4.96s/it]
 82%|████████▏ | 184179/225000 [348:05:57<56:48:31,  5.01s/it]
 82%|████████▏ | 184180/225000 [348:06:02<56:58:53,  5.03s/it]
 82%|████████▏ | 184181/225000 [348:06:07<56:54:03,  5.02s/it]
 82%|████████▏ | 184182/225000 [348:06:13<57:38:59,  5.08s/it]
 82%|████████▏ | 184183/225000 [348:06:23<73:48:58,  6.51s/it]
 82%|████████▏ | 184184/225000 [348:06:30<76:53:01,  6.78s/it]
 82%|████████▏ | 184185/225000 [348:06:35<70:30:48,  6.22s/it]
 82%|████████▏ | 184186/225000 [348:06:41<68:49:47,  6.07s/it]
 82%|████████▏ | 184187/225000 [348:06:46<67:23:42,  5.94s/it]
 82%|████████▏ | 184188/225000 [348:06:52<66:46:37,  5.89s/it]
 82%|████████▏ | 184189/225000 [348:06:58<65:30:25,  5.78s/it]
 82%|████████▏ | 184190/225000 [348:07:03<65:04:48,  5.74s/it]
 82%|████████▏ | 184191/225000 [348:07:10<68:14:05,  6.02s/it]
 82%|████████▏ | 184192/225000 [348:07:15<66:38:40,  5.88s/it]
 82%|████████▏ | 184193/225000 [348:07:20<63:50:23,  5.63s/it]
 82%|████████▏ | 184194/225000 [348:07:24<57:06:46,  5.04s/it]
 82%|████████▏ | 184195/225000 [348:07:33<68:34:15,  6.05s/it]
 82%|████████▏ | 184196/225000 [348:07:37<64:15:39,  5.67s/it]
 82%|████████▏ | 184197/225000 [348:07:41<56:17:31,  4.97s/it]
 82%|████████▏ | 184198/225000 [348:07:49<66:32:36,  5.87s/it]
 82%|████████▏ | 184199/225000 [348:07:52<58:41:46,  5.18s/it]
 82%|████████▏ | 184200/225000 [348:07:56<54:14:07,  4.79s/it]
                                                              
{'loss': '0.7725', 'grad_norm': '1.161', 'learning_rate': '4.33e-06', 'epoch': '0.8187', 'num_input_tokens_seen': 577477504, 'train_runtime': '1.253e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184200/225000 [348:07:56<54:14:07,  4.79s/it]
 82%|████████▏ | 184201/225000 [348:08:02<58:13:16,  5.14s/it]
 82%|████████▏ | 184202/225000 [348:08:09<65:48:31,  5.81s/it]
 82%|████████▏ | 184203/225000 [348:08:15<64:03:24,  5.65s/it]
 82%|████████▏ | 184204/225000 [348:08:21<66:45:33,  5.89s/it]
 82%|████████▏ | 184205/225000 [348:08:25<60:03:38,  5.30s/it]
 82%|████████▏ | 184206/225000 [348:08:29<55:49:09,  4.93s/it]
 82%|████████▏ | 184207/225000 [348:08:33<53:06:28,  4.69s/it]
 82%|████████▏ | 184208/225000 [348:08:42<67:48:30,  5.98s/it]
 82%|████████▏ | 184209/225000 [348:08:46<59:37:10,  5.26s/it]
 82%|████████▏ | 184210/225000 [348:08:54<71:11:18,  6.28s/it]
 82%|████████▏ | 184211/225000 [348:08:58<63:17:18,  5.59s/it]
 82%|████████▏ | 184212/225000 [348:09:02<57:03:15,  5.04s/it]
 82%|████████▏ | 184213/225000 [348:09:06<53:09:05,  4.69s/it]
 82%|████████▏ | 184214/225000 [348:09:18<76:13:32,  6.73s/it]
 82%|████████▏ | 184215/225000 [348:09:21<65:51:35,  5.81s/it]
 82%|████████▏ | 184216/225000 [348:09:25<58:51:11,  5.19s/it]
 82%|████████▏ | 184217/225000 [348:09:34<72:56:36,  6.44s/it]
 82%|████████▏ | 184218/225000 [348:09:38<62:14:52,  5.49s/it]
 82%|████████▏ | 184219/225000 [348:09:45<69:08:42,  6.10s/it]
 82%|████████▏ | 184220/225000 [348:09:49<60:17:59,  5.32s/it]
 82%|████████▏ | 184221/225000 [348:09:53<57:49:14,  5.10s/it]
 82%|████████▏ | 184222/225000 [348:09:58<55:47:24,  4.93s/it]
 82%|████████▏ | 184223/225000 [348:10:02<54:53:03,  4.85s/it]
 82%|████████▏ | 184224/225000 [348:10:07<54:56:12,  4.85s/it]
 82%|████████▏ | 184225/225000 [348:10:12<53:59:59,  4.77s/it]
 82%|████████▏ | 184226/225000 [348:10:16<53:12:26,  4.70s/it]
 82%|████████▏ | 184227/225000 [348:10:26<70:23:15,  6.21s/it]
 82%|████████▏ | 184228/225000 [348:10:36<81:50:19,  7.23s/it]
 82%|████████▏ | 184229/225000 [348:10:39<68:57:42,  6.09s/it]
 82%|████████▏ | 184230/225000 [348:10:45<67:01:21,  5.92s/it]
 82%|████████▏ | 184231/225000 [348:10:49<63:13:55,  5.58s/it]
 82%|████████▏ | 184232/225000 [348:10:55<63:32:27,  5.61s/it]
 82%|████████▏ | 184233/225000 [348:11:01<63:38:33,  5.62s/it]
 82%|████████▏ | 184234/225000 [348:11:06<62:51:22,  5.55s/it]
 82%|████████▏ | 184235/225000 [348:11:12<62:31:12,  5.52s/it]
 82%|████████▏ | 184236/225000 [348:11:17<61:51:28,  5.46s/it]
 82%|████████▏ | 184237/225000 [348:11:22<62:05:30,  5.48s/it]
 82%|████████▏ | 184238/225000 [348:11:28<61:08:24,  5.40s/it]
 82%|████████▏ | 184239/225000 [348:11:33<61:26:50,  5.43s/it]
 82%|████████▏ | 184240/225000 [348:11:39<61:31:24,  5.43s/it]
 82%|████████▏ | 184241/225000 [348:11:44<61:51:01,  5.46s/it]
 82%|████████▏ | 184242/225000 [348:11:54<77:45:05,  6.87s/it]
 82%|████████▏ | 184243/225000 [348:12:00<72:28:10,  6.40s/it]
 82%|████████▏ | 184244/225000 [348:12:06<73:08:17,  6.46s/it]
 82%|████████▏ | 184245/225000 [348:12:10<64:46:25,  5.72s/it]
 82%|████████▏ | 184246/225000 [348:12:14<59:28:03,  5.25s/it]
 82%|████████▏ | 184247/225000 [348:12:19<56:22:18,  4.98s/it]
 82%|████████▏ | 184248/225000 [348:12:23<54:45:10,  4.84s/it]
 82%|████████▏ | 184249/225000 [348:12:29<57:44:08,  5.10s/it]
 82%|████████▏ | 184250/225000 [348:12:39<73:51:17,  6.52s/it]
                                                              
{'loss': '0.7617', 'grad_norm': '1.204', 'learning_rate': '4.32e-06', 'epoch': '0.8189', 'num_input_tokens_seen': 577607392, 'train_runtime': '1.254e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184250/225000 [348:12:39<73:51:17,  6.52s/it]
 82%|████████▏ | 184251/225000 [348:12:47<80:04:03,  7.07s/it]
 82%|████████▏ | 184252/225000 [348:12:52<71:14:57,  6.29s/it]
 82%|████████▏ | 184253/225000 [348:12:56<65:01:32,  5.75s/it]
 82%|████████▏ | 184254/225000 [348:13:00<59:56:14,  5.30s/it]
 82%|████████▏ | 184255/225000 [348:13:05<57:23:31,  5.07s/it]
 82%|████████▏ | 184256/225000 [348:13:15<73:48:35,  6.52s/it]
 82%|████████▏ | 184257/225000 [348:13:20<70:41:48,  6.25s/it]
 82%|████████▏ | 184258/225000 [348:13:24<62:46:07,  5.55s/it]
 82%|████████▏ | 184259/225000 [348:13:28<57:52:22,  5.11s/it]
 82%|████████▏ | 184260/225000 [348:13:33<56:03:39,  4.95s/it]
 82%|████████▏ | 184261/225000 [348:13:38<55:57:44,  4.95s/it]
 82%|████████▏ | 184262/225000 [348:13:48<73:36:34,  6.50s/it]
 82%|████████▏ | 184263/225000 [348:13:52<66:00:30,  5.83s/it]
 82%|████████▏ | 184264/225000 [348:13:57<60:50:11,  5.38s/it]
 82%|████████▏ | 184265/225000 [348:14:02<59:36:16,  5.27s/it]
 82%|████████▏ | 184266/225000 [348:14:13<81:20:31,  7.19s/it]
 82%|████████▏ | 184267/225000 [348:14:17<68:00:04,  6.01s/it]
 82%|████████▏ | 184268/225000 [348:14:21<61:31:21,  5.44s/it]
 82%|████████▏ | 184269/225000 [348:14:25<57:02:08,  5.04s/it]
 82%|████████▏ | 184270/225000 [348:14:35<73:54:19,  6.53s/it]
 82%|████████▏ | 184271/225000 [348:14:39<64:56:18,  5.74s/it]
 82%|████████▏ | 184272/225000 [348:14:43<59:28:53,  5.26s/it]
 82%|████████▏ | 184273/225000 [348:14:47<56:03:12,  4.95s/it]
 82%|████████▏ | 184274/225000 [348:14:52<54:40:40,  4.83s/it]
 82%|████████▏ | 184275/225000 [348:15:02<73:51:08,  6.53s/it]
 82%|████████▏ | 184276/225000 [348:15:05<62:27:53,  5.52s/it]
 82%|████████▏ | 184277/225000 [348:15:13<68:27:57,  6.05s/it]
 82%|████████▏ | 184278/225000 [348:15:16<60:47:41,  5.37s/it]
 82%|████████▏ | 184279/225000 [348:15:20<56:08:17,  4.96s/it]
 82%|████████▏ | 184280/225000 [348:15:24<52:30:28,  4.64s/it]
 82%|████████▏ | 184281/225000 [348:15:33<65:55:27,  5.83s/it]
 82%|████████▏ | 184282/225000 [348:15:37<58:42:01,  5.19s/it]
 82%|████████▏ | 184283/225000 [348:15:41<55:38:42,  4.92s/it]
 82%|████████▏ | 184284/225000 [348:15:46<56:54:27,  5.03s/it]
 82%|████████▏ | 184285/225000 [348:15:51<57:43:06,  5.10s/it]
 82%|████████▏ | 184286/225000 [348:15:56<56:53:36,  5.03s/it]
 82%|████████▏ | 184287/225000 [348:16:01<56:16:01,  4.98s/it]
 82%|████████▏ | 184288/225000 [348:16:13<80:33:49,  7.12s/it]
 82%|████████▏ | 184289/225000 [348:16:19<75:43:37,  6.70s/it]
 82%|████████▏ | 184290/225000 [348:16:22<64:40:09,  5.72s/it]
 82%|████████▏ | 184291/225000 [348:16:28<63:31:56,  5.62s/it]
 82%|████████▏ | 184292/225000 [348:16:36<70:57:11,  6.27s/it]
 82%|████████▏ | 184293/225000 [348:16:39<62:14:51,  5.50s/it]
 82%|████████▏ | 184294/225000 [348:16:47<70:28:34,  6.23s/it]
 82%|████████▏ | 184295/225000 [348:16:50<59:42:11,  5.28s/it]
 82%|████████▏ | 184296/225000 [348:16:54<55:53:58,  4.94s/it]
 82%|████████▏ | 184297/225000 [348:16:59<53:31:05,  4.73s/it]
 82%|████████▏ | 184298/225000 [348:17:03<51:57:43,  4.60s/it]
 82%|████████▏ | 184299/225000 [348:17:15<75:45:28,  6.70s/it]
 82%|████████▏ | 184300/225000 [348:17:18<64:05:35,  5.67s/it]
                                                              
{'loss': '0.7576', 'grad_norm': '1.081', 'learning_rate': '4.31e-06', 'epoch': '0.8191', 'num_input_tokens_seen': 577737424, 'train_runtime': '1.254e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184300/225000 [348:17:18<64:05:35,  5.67s/it]
 82%|████████▏ | 184301/225000 [348:17:22<58:55:29,  5.21s/it]
 82%|████████▏ | 184302/225000 [348:17:30<69:26:17,  6.14s/it]
 82%|████████▏ | 184303/225000 [348:17:35<64:38:30,  5.72s/it]
 82%|████████▏ | 184304/225000 [348:17:39<60:17:12,  5.33s/it]
 82%|████████▏ | 184305/225000 [348:17:44<57:50:48,  5.12s/it]
 82%|████████▏ | 184306/225000 [348:17:49<56:03:22,  4.96s/it]
 82%|████████▏ | 184307/225000 [348:17:55<59:26:27,  5.26s/it]
 82%|████████▏ | 184308/225000 [348:18:06<78:38:54,  6.96s/it]
 82%|████████▏ | 184309/225000 [348:18:09<66:41:26,  5.90s/it]
 82%|████████▏ | 184310/225000 [348:18:14<62:48:09,  5.56s/it]
 82%|████████▏ | 184311/225000 [348:18:18<60:03:18,  5.31s/it]
 82%|████████▏ | 184312/225000 [348:18:23<57:51:19,  5.12s/it]
 82%|████████▏ | 184313/225000 [348:18:28<56:37:43,  5.01s/it]
 82%|████████▏ | 184314/225000 [348:18:33<55:19:08,  4.89s/it]
 82%|████████▏ | 184315/225000 [348:18:37<54:29:37,  4.82s/it]
 82%|████████▏ | 184316/225000 [348:18:42<55:41:12,  4.93s/it]
 82%|████████▏ | 184317/225000 [348:18:47<55:11:50,  4.88s/it]
 82%|████████▏ | 184318/225000 [348:18:57<71:30:18,  6.33s/it]
 82%|████████▏ | 184319/225000 [348:19:04<73:56:56,  6.54s/it]
 82%|████████▏ | 184320/225000 [348:19:09<69:50:40,  6.18s/it]
 82%|████████▏ | 184321/225000 [348:19:16<72:37:17,  6.43s/it]
 82%|████████▏ | 184322/225000 [348:19:20<62:43:11,  5.55s/it]
 82%|████████▏ | 184323/225000 [348:19:27<68:16:53,  6.04s/it]
 82%|████████▏ | 184324/225000 [348:19:32<66:40:39,  5.90s/it]
 82%|████████▏ | 184325/225000 [348:19:36<58:38:53,  5.19s/it]
 82%|████████▏ | 184326/225000 [348:19:45<73:01:40,  6.46s/it]
 82%|████████▏ | 184327/225000 [348:19:49<63:44:38,  5.64s/it]
 82%|████████▏ | 184328/225000 [348:19:56<67:18:06,  5.96s/it]
 82%|████████▏ | 184329/225000 [348:20:02<66:47:07,  5.91s/it]
 82%|████████▏ | 184330/225000 [348:20:08<69:35:29,  6.16s/it]
 82%|████████▏ | 184331/225000 [348:20:13<63:37:09,  5.63s/it]
 82%|████████▏ | 184332/225000 [348:20:17<59:55:35,  5.30s/it]
 82%|████████▏ | 184333/225000 [348:20:22<58:08:37,  5.15s/it]
 82%|████████▏ | 184334/225000 [348:20:27<56:18:56,  4.99s/it]
 82%|████████▏ | 184335/225000 [348:20:31<55:01:37,  4.87s/it]
 82%|████████▏ | 184336/225000 [348:20:36<54:06:46,  4.79s/it]
 82%|████████▏ | 184337/225000 [348:20:45<68:53:03,  6.10s/it]
 82%|████████▏ | 184338/225000 [348:20:53<75:42:12,  6.70s/it]
 82%|████████▏ | 184339/225000 [348:20:57<65:18:09,  5.78s/it]
 82%|████████▏ | 184340/225000 [348:21:01<60:42:52,  5.38s/it]
 82%|████████▏ | 184341/225000 [348:21:06<59:01:40,  5.23s/it]
 82%|████████▏ | 184342/225000 [348:21:16<74:37:00,  6.61s/it]
 82%|████████▏ | 184343/225000 [348:21:21<70:24:40,  6.23s/it]
 82%|████████▏ | 184344/225000 [348:21:24<59:39:32,  5.28s/it]
 82%|████████▏ | 184345/225000 [348:21:33<70:56:40,  6.28s/it]
 82%|████████▏ | 184346/225000 [348:21:38<64:58:11,  5.75s/it]
 82%|████████▏ | 184347/225000 [348:21:42<60:01:52,  5.32s/it]
 82%|████████▏ | 184348/225000 [348:21:46<56:47:12,  5.03s/it]
 82%|████████▏ | 184349/225000 [348:21:51<54:57:14,  4.87s/it]
 82%|████████▏ | 184350/225000 [348:21:55<54:42:57,  4.85s/it]
                                                              
{'loss': '0.7482', 'grad_norm': '1.19', 'learning_rate': '4.3e-06', 'epoch': '0.8193', 'num_input_tokens_seen': 577867880, 'train_runtime': '1.254e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184350/225000 [348:21:55<54:42:57,  4.85s/it]
 82%|████████▏ | 184351/225000 [348:22:05<70:51:00,  6.27s/it]
 82%|████████▏ | 184352/225000 [348:22:13<74:47:59,  6.62s/it]
 82%|████████▏ | 184353/225000 [348:22:16<63:34:38,  5.63s/it]
 82%|████████▏ | 184354/225000 [348:22:22<65:45:38,  5.82s/it]
 82%|████████▏ | 184355/225000 [348:22:28<65:36:41,  5.81s/it]
 82%|████████▏ | 184356/225000 [348:22:32<58:37:20,  5.19s/it]
 82%|████████▏ | 184357/225000 [348:22:37<58:34:30,  5.19s/it]
 82%|████████▏ | 184358/225000 [348:22:42<59:00:36,  5.23s/it]
 82%|████████▏ | 184359/225000 [348:22:52<74:09:36,  6.57s/it]
 82%|████████▏ | 184360/225000 [348:22:55<63:23:53,  5.62s/it]
 82%|████████▏ | 184361/225000 [348:22:59<58:18:48,  5.17s/it]
 82%|████████▏ | 184362/225000 [348:23:03<54:37:58,  4.84s/it]
 82%|████████▏ | 184363/225000 [348:23:15<76:35:40,  6.79s/it]
 82%|████████▏ | 184364/225000 [348:23:19<67:14:10,  5.96s/it]
 82%|████████▏ | 184365/225000 [348:23:25<69:23:19,  6.15s/it]
 82%|████████▏ | 184366/225000 [348:23:33<74:40:13,  6.62s/it]
 82%|████████▏ | 184367/225000 [348:23:37<64:41:01,  5.73s/it]
 82%|████████▏ | 184368/225000 [348:23:40<57:36:51,  5.10s/it]
 82%|████████▏ | 184369/225000 [348:23:50<73:43:56,  6.53s/it]
 82%|████████▏ | 184370/225000 [348:23:55<67:40:03,  6.00s/it]
 82%|████████▏ | 184371/225000 [348:24:00<63:08:41,  5.60s/it]
 82%|████████▏ | 184372/225000 [348:24:04<59:18:06,  5.25s/it]
 82%|████████▏ | 184373/225000 [348:24:09<56:35:38,  5.01s/it]
 82%|████████▏ | 184374/225000 [348:24:19<75:31:26,  6.69s/it]
 82%|████████▏ | 184375/225000 [348:24:23<64:46:54,  5.74s/it]
 82%|████████▏ | 184376/225000 [348:24:27<58:28:54,  5.18s/it]
 82%|████████▏ | 184377/225000 [348:24:30<53:45:09,  4.76s/it]
 82%|████████▏ | 184378/225000 [348:24:36<58:11:49,  5.16s/it]
 82%|████████▏ | 184379/225000 [348:24:45<70:46:40,  6.27s/it]
 82%|████████▏ | 184380/225000 [348:24:50<66:29:05,  5.89s/it]
 82%|████████▏ | 184381/225000 [348:24:55<62:52:29,  5.57s/it]
 82%|████████▏ | 184382/225000 [348:24:59<57:26:07,  5.09s/it]
 82%|████████▏ | 184383/225000 [348:25:04<57:01:30,  5.05s/it]
 82%|████████▏ | 184384/225000 [348:25:09<55:50:16,  4.95s/it]
 82%|████████▏ | 184385/225000 [348:25:14<57:54:19,  5.13s/it]
 82%|████████▏ | 184386/225000 [348:25:19<56:30:53,  5.01s/it]
 82%|████████▏ | 184387/225000 [348:25:24<56:19:27,  4.99s/it]
 82%|████████▏ | 184388/225000 [348:25:29<57:36:34,  5.11s/it]
 82%|████████▏ | 184389/225000 [348:25:34<56:07:03,  4.97s/it]
 82%|████████▏ | 184390/225000 [348:25:39<56:18:15,  4.99s/it]
 82%|████████▏ | 184391/225000 [348:25:44<57:14:17,  5.07s/it]
 82%|████████▏ | 184392/225000 [348:25:50<57:52:41,  5.13s/it]
 82%|████████▏ | 184393/225000 [348:25:58<68:44:55,  6.09s/it]
 82%|████████▏ | 184394/225000 [348:26:09<83:50:59,  7.43s/it]
 82%|████████▏ | 184395/225000 [348:26:12<69:25:55,  6.16s/it]
 82%|████████▏ | 184396/225000 [348:26:18<71:03:36,  6.30s/it]
 82%|████████▏ | 184397/225000 [348:26:24<68:38:28,  6.09s/it]
 82%|████████▏ | 184398/225000 [348:26:28<63:08:40,  5.60s/it]
 82%|████████▏ | 184399/225000 [348:26:33<60:21:56,  5.35s/it]
 82%|████████▏ | 184400/225000 [348:26:38<58:09:31,  5.16s/it]
                                                              
{'loss': '0.7597', 'grad_norm': '1.326', 'learning_rate': '4.289e-06', 'epoch': '0.8196', 'num_input_tokens_seen': 577997688, 'train_runtime': '1.254e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184400/225000 [348:26:38<58:09:31,  5.16s/it]
 82%|████████▏ | 184401/225000 [348:26:43<56:33:58,  5.02s/it]
 82%|████████▏ | 184402/225000 [348:26:47<56:19:59,  5.00s/it]
 82%|████████▏ | 184403/225000 [348:26:53<56:27:53,  5.01s/it]
 82%|████████▏ | 184404/225000 [348:26:58<56:44:39,  5.03s/it]
 82%|████████▏ | 184405/225000 [348:27:03<57:31:06,  5.10s/it]
 82%|████████▏ | 184406/225000 [348:27:08<57:57:09,  5.14s/it]
 82%|████████▏ | 184407/225000 [348:27:20<81:11:27,  7.20s/it]
 82%|████████▏ | 184408/225000 [348:27:27<78:55:27,  7.00s/it]
 82%|████████▏ | 184409/225000 [348:27:31<68:27:40,  6.07s/it]
 82%|████████▏ | 184410/225000 [348:27:34<61:01:53,  5.41s/it]
 82%|████████▏ | 184411/225000 [348:27:46<82:31:24,  7.32s/it]
 82%|████████▏ | 184412/225000 [348:27:50<70:22:00,  6.24s/it]
 82%|████████▏ | 184413/225000 [348:27:54<64:03:09,  5.68s/it]
 82%|████████▏ | 184414/225000 [348:27:59<60:56:37,  5.41s/it]
 82%|████████▏ | 184415/225000 [348:28:04<58:12:32,  5.16s/it]
 82%|████████▏ | 184416/225000 [348:28:09<57:15:39,  5.08s/it]
 82%|████████▏ | 184417/225000 [348:28:13<55:10:17,  4.89s/it]
 82%|████████▏ | 184418/225000 [348:28:18<54:30:57,  4.84s/it]
 82%|████████▏ | 184419/225000 [348:28:25<62:50:28,  5.57s/it]
 82%|████████▏ | 184420/225000 [348:28:35<77:30:24,  6.88s/it]
 82%|████████▏ | 184421/225000 [348:28:39<68:29:13,  6.08s/it]
 82%|████████▏ | 184422/225000 [348:28:43<62:43:46,  5.57s/it]
 82%|████████▏ | 184423/225000 [348:28:48<58:36:44,  5.20s/it]
 82%|████████▏ | 184424/225000 [348:28:52<55:40:09,  4.94s/it]
 82%|████████▏ | 184425/225000 [348:29:02<72:35:22,  6.44s/it]
 82%|████████▏ | 184426/225000 [348:29:07<67:03:04,  5.95s/it]
 82%|████████▏ | 184427/225000 [348:29:11<62:18:32,  5.53s/it]
 82%|████████▏ | 184428/225000 [348:29:16<59:03:09,  5.24s/it]
 82%|████████▏ | 184429/225000 [348:29:21<58:05:20,  5.15s/it]
 82%|████████▏ | 184430/225000 [348:29:25<55:00:42,  4.88s/it]
 82%|████████▏ | 184431/225000 [348:29:30<53:59:30,  4.79s/it]
 82%|████████▏ | 184432/225000 [348:29:34<53:00:19,  4.70s/it]
 82%|████████▏ | 184433/225000 [348:29:43<67:56:38,  6.03s/it]
 82%|████████▏ | 184434/225000 [348:29:50<70:21:00,  6.24s/it]
 82%|████████▏ | 184435/225000 [348:29:54<62:40:28,  5.56s/it]
 82%|████████▏ | 184436/225000 [348:29:58<56:03:25,  4.97s/it]
 82%|████████▏ | 184437/225000 [348:30:10<79:19:52,  7.04s/it]
 82%|████████▏ | 184438/225000 [348:30:13<68:21:29,  6.07s/it]
 82%|████████▏ | 184439/225000 [348:30:16<57:58:33,  5.15s/it]
 82%|████████▏ | 184440/225000 [348:30:21<57:15:15,  5.08s/it]
 82%|████████▏ | 184441/225000 [348:30:27<57:44:47,  5.13s/it]
 82%|████████▏ | 184442/225000 [348:30:32<58:41:22,  5.21s/it]
 82%|████████▏ | 184443/225000 [348:30:38<63:00:02,  5.59s/it]
 82%|████████▏ | 184444/225000 [348:30:48<76:08:23,  6.76s/it]
 82%|████████▏ | 184445/225000 [348:30:53<70:58:09,  6.30s/it]
 82%|████████▏ | 184446/225000 [348:30:58<67:08:08,  5.96s/it]
 82%|████████▏ | 184447/225000 [348:31:04<64:58:31,  5.77s/it]
 82%|████████▏ | 184448/225000 [348:31:09<61:53:21,  5.49s/it]
 82%|████████▏ | 184449/225000 [348:31:14<60:57:54,  5.41s/it]
 82%|████████▏ | 184450/225000 [348:31:19<60:45:41,  5.39s/it]
                                                              
{'loss': '0.762', 'grad_norm': '1.062', 'learning_rate': '4.279e-06', 'epoch': '0.8198', 'num_input_tokens_seen': 578127848, 'train_runtime': '1.255e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184450/225000 [348:31:19<60:45:41,  5.39s/it]
 82%|████████▏ | 184451/225000 [348:31:24<59:46:47,  5.31s/it]
 82%|████████▏ | 184452/225000 [348:31:29<59:18:26,  5.27s/it]
 82%|████████▏ | 184453/225000 [348:31:34<58:36:54,  5.20s/it]
 82%|████████▏ | 184454/225000 [348:31:39<57:54:48,  5.14s/it]
 82%|████████▏ | 184455/225000 [348:31:45<58:52:52,  5.23s/it]
 82%|████████▏ | 184456/225000 [348:31:50<58:37:19,  5.21s/it]
 82%|████████▏ | 184457/225000 [348:31:58<67:53:32,  6.03s/it]
 82%|████████▏ | 184458/225000 [348:32:09<84:25:58,  7.50s/it]
 82%|████████▏ | 184459/225000 [348:32:12<71:17:22,  6.33s/it]
 82%|████████▏ | 184460/225000 [348:32:21<77:19:46,  6.87s/it]
 82%|████████▏ | 184461/225000 [348:32:24<65:00:04,  5.77s/it]
 82%|████████▏ | 184462/225000 [348:32:31<69:24:46,  6.16s/it]
 82%|████████▏ | 184463/225000 [348:32:35<61:26:23,  5.46s/it]
 82%|████████▏ | 184464/225000 [348:32:40<61:46:35,  5.49s/it]
 82%|████████▏ | 184465/225000 [348:32:46<63:28:13,  5.64s/it]
 82%|████████▏ | 184466/225000 [348:32:51<61:49:12,  5.49s/it]
 82%|████████▏ | 184467/225000 [348:32:59<67:18:27,  5.98s/it]
 82%|████████▏ | 184468/225000 [348:33:03<62:58:31,  5.59s/it]
 82%|████████▏ | 184469/225000 [348:33:08<60:38:08,  5.39s/it]
 82%|████████▏ | 184470/225000 [348:33:13<59:32:28,  5.29s/it]
 82%|████████▏ | 184471/225000 [348:33:18<57:13:01,  5.08s/it]
 82%|████████▏ | 184472/225000 [348:33:23<56:30:37,  5.02s/it]
 82%|████████▏ | 184473/225000 [348:33:28<57:00:48,  5.06s/it]
 82%|████████▏ | 184474/225000 [348:33:33<56:12:01,  4.99s/it]
 82%|████████▏ | 184475/225000 [348:33:39<61:43:29,  5.48s/it]
 82%|████████▏ | 184476/225000 [348:33:50<78:41:37,  6.99s/it]
 82%|████████▏ | 184477/225000 [348:33:54<69:38:31,  6.19s/it]
 82%|████████▏ | 184478/225000 [348:33:58<63:05:49,  5.61s/it]
 82%|████████▏ | 184479/225000 [348:34:03<59:19:45,  5.27s/it]
 82%|████████▏ | 184480/225000 [348:34:07<56:51:55,  5.05s/it]
 82%|████████▏ | 184481/225000 [348:34:12<56:10:06,  4.99s/it]
 82%|████████▏ | 184482/225000 [348:34:17<56:53:21,  5.05s/it]
 82%|████████▏ | 184483/225000 [348:34:28<76:35:14,  6.80s/it]
 82%|████████▏ | 184484/225000 [348:34:33<69:22:09,  6.16s/it]
 82%|████████▏ | 184485/225000 [348:34:38<65:39:45,  5.83s/it]
 82%|████████▏ | 184486/225000 [348:34:43<61:20:07,  5.45s/it]
 82%|████████▏ | 184487/225000 [348:34:48<60:28:49,  5.37s/it]
 82%|████████▏ | 184488/225000 [348:34:53<59:00:23,  5.24s/it]
 82%|████████▏ | 184489/225000 [348:34:57<57:04:40,  5.07s/it]
 82%|████████▏ | 184490/225000 [348:35:02<56:50:00,  5.05s/it]
 82%|████████▏ | 184491/225000 [348:35:07<55:31:46,  4.93s/it]
 82%|████████▏ | 184492/225000 [348:35:12<55:01:23,  4.89s/it]
 82%|████████▏ | 184493/225000 [348:35:24<78:07:07,  6.94s/it]
 82%|████████▏ | 184494/225000 [348:35:31<79:35:07,  7.07s/it]
 82%|████████▏ | 184495/225000 [348:35:34<67:28:20,  6.00s/it]
 82%|████████▏ | 184496/225000 [348:35:39<62:37:16,  5.57s/it]
 82%|████████▏ | 184497/225000 [348:35:44<62:11:34,  5.53s/it]
 82%|████████▏ | 184498/225000 [348:35:50<61:02:15,  5.43s/it]
 82%|████████▏ | 184499/225000 [348:35:55<61:13:42,  5.44s/it]
 82%|████████▏ | 184500/225000 [348:36:01<61:17:31,  5.45s/it]
                                                              
{'loss': '0.7589', 'grad_norm': '1.206', 'learning_rate': '4.269e-06', 'epoch': '0.82', 'num_input_tokens_seen': 578258032, 'train_runtime': '1.255e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184500/225000 [348:36:01<61:17:31,  5.45s/it]
 82%|████████▏ | 184501/225000 [348:36:06<61:22:24,  5.46s/it]
 82%|████████▏ | 184502/225000 [348:36:11<59:25:08,  5.28s/it]
 82%|████████▏ | 184503/225000 [348:36:16<59:39:56,  5.30s/it]
 82%|████████▏ | 184504/225000 [348:36:22<60:13:06,  5.35s/it]
 82%|████████▏ | 184505/225000 [348:36:27<59:54:14,  5.33s/it]
 82%|████████▏ | 184506/225000 [348:36:32<59:20:03,  5.27s/it]
 82%|████████▏ | 184507/225000 [348:36:38<62:47:37,  5.58s/it]
 82%|████████▏ | 184508/225000 [348:36:43<60:43:33,  5.40s/it]
 82%|████████▏ | 184509/225000 [348:36:49<59:46:39,  5.31s/it]
 82%|████████▏ | 184510/225000 [348:36:54<58:40:09,  5.22s/it]
 82%|████████▏ | 184511/225000 [348:36:59<58:27:15,  5.20s/it]
 82%|████████▏ | 184512/225000 [348:37:04<59:32:40,  5.29s/it]
 82%|████████▏ | 184513/225000 [348:37:09<57:51:56,  5.15s/it]
 82%|████████▏ | 184514/225000 [348:37:14<58:14:09,  5.18s/it]
 82%|████████▏ | 184515/225000 [348:37:20<58:44:55,  5.22s/it]
 82%|████████▏ | 184516/225000 [348:37:25<58:44:43,  5.22s/it]
 82%|████████▏ | 184517/225000 [348:37:30<58:05:43,  5.17s/it]
 82%|████████▏ | 184518/225000 [348:37:35<58:17:48,  5.18s/it]
 82%|████████▏ | 184519/225000 [348:37:40<57:06:07,  5.08s/it]
 82%|████████▏ | 184520/225000 [348:37:51<78:11:03,  6.95s/it]
 82%|████████▏ | 184521/225000 [348:37:59<80:40:38,  7.18s/it]
 82%|████████▏ | 184522/225000 [348:38:03<69:17:15,  6.16s/it]
 82%|████████▏ | 184523/225000 [348:38:06<61:01:58,  5.43s/it]
 82%|████████▏ | 184524/225000 [348:38:16<74:48:13,  6.65s/it]
 82%|████████▏ | 184525/225000 [348:38:20<66:04:17,  5.88s/it]
 82%|████████▏ | 184526/225000 [348:38:26<65:15:24,  5.80s/it]
 82%|████████▏ | 184527/225000 [348:38:31<64:01:47,  5.70s/it]
 82%|████████▏ | 184528/225000 [348:38:36<63:01:19,  5.61s/it]
 82%|████████▏ | 184529/225000 [348:38:42<62:59:50,  5.60s/it]
 82%|████████▏ | 184530/225000 [348:38:47<59:34:05,  5.30s/it]
 82%|████████▏ | 184531/225000 [348:38:52<59:48:18,  5.32s/it]
 82%|████████▏ | 184532/225000 [348:38:57<57:13:58,  5.09s/it]
 82%|████████▏ | 184533/225000 [348:39:02<56:50:00,  5.06s/it]
 82%|████████▏ | 184534/225000 [348:39:07<57:35:56,  5.12s/it]
 82%|████████▏ | 184535/225000 [348:39:12<58:41:26,  5.22s/it]
 82%|████████▏ | 184536/225000 [348:39:18<61:00:34,  5.43s/it]
 82%|████████▏ | 184537/225000 [348:39:23<60:07:20,  5.35s/it]
 82%|████████▏ | 184538/225000 [348:39:31<67:13:53,  5.98s/it]
 82%|████████▏ | 184539/225000 [348:39:41<81:28:15,  7.25s/it]
 82%|████████▏ | 184540/225000 [348:39:45<69:42:17,  6.20s/it]
 82%|████████▏ | 184541/225000 [348:39:51<68:27:09,  6.09s/it]
 82%|████████▏ | 184542/225000 [348:39:57<68:10:50,  6.07s/it]
 82%|████████▏ | 184543/225000 [348:40:01<61:45:25,  5.50s/it]
 82%|████████▏ | 184544/225000 [348:40:05<56:22:33,  5.02s/it]
 82%|████████▏ | 184545/225000 [348:40:14<70:12:34,  6.25s/it]
 82%|████████▏ | 184546/225000 [348:40:17<61:28:12,  5.47s/it]
 82%|████████▏ | 184547/225000 [348:40:21<55:06:42,  4.90s/it]
 82%|████████▏ | 184548/225000 [348:40:32<73:46:48,  6.57s/it]
 82%|████████▏ | 184549/225000 [348:40:35<62:31:31,  5.56s/it]
 82%|████████▏ | 184550/225000 [348:40:42<67:18:22,  5.99s/it]
                                                              
{'loss': '0.7531', 'grad_norm': '1.293', 'learning_rate': '4.259e-06', 'epoch': '0.8202', 'num_input_tokens_seen': 578387936, 'train_runtime': '1.255e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184550/225000 [348:40:42<67:18:22,  5.99s/it]
 82%|████████▏ | 184551/225000 [348:40:45<59:05:59,  5.26s/it]
 82%|████████▏ | 184552/225000 [348:40:49<52:31:57,  4.68s/it]
 82%|████████▏ | 184553/225000 [348:40:54<56:05:52,  4.99s/it]
 82%|████████▏ | 184554/225000 [348:41:02<66:13:59,  5.90s/it]
 82%|████████▏ | 184555/225000 [348:41:06<58:45:33,  5.23s/it]
 82%|████████▏ | 184556/225000 [348:41:10<52:59:22,  4.72s/it]
 82%|████████▏ | 184557/225000 [348:41:13<49:34:20,  4.41s/it]
 82%|████████▏ | 184558/225000 [348:41:22<64:59:21,  5.79s/it]
 82%|████████▏ | 184559/225000 [348:41:30<70:28:40,  6.27s/it]
 82%|████████▏ | 184560/225000 [348:41:34<62:28:25,  5.56s/it]
 82%|████████▏ | 184561/225000 [348:41:38<57:54:08,  5.15s/it]
 82%|████████▏ | 184562/225000 [348:41:42<53:46:57,  4.79s/it]
 82%|████████▏ | 184563/225000 [348:41:46<52:20:21,  4.66s/it]
 82%|████████▏ | 184564/225000 [348:41:51<53:57:48,  4.80s/it]
 82%|████████▏ | 184565/225000 [348:42:00<67:58:26,  6.05s/it]
 82%|████████▏ | 184566/225000 [348:42:06<65:43:50,  5.85s/it]
 82%|████████▏ | 184567/225000 [348:42:10<61:11:00,  5.45s/it]
 82%|████████▏ | 184568/225000 [348:42:14<56:28:26,  5.03s/it]
 82%|████████▏ | 184569/225000 [348:42:18<54:25:29,  4.85s/it]
 82%|████████▏ | 184570/225000 [348:42:23<53:41:44,  4.78s/it]
 82%|████████▏ | 184571/225000 [348:42:28<54:25:01,  4.85s/it]
 82%|████████▏ | 184572/225000 [348:42:34<57:09:25,  5.09s/it]
 82%|████████▏ | 184573/225000 [348:42:39<58:05:54,  5.17s/it]
 82%|████████▏ | 184574/225000 [348:42:44<58:10:50,  5.18s/it]
 82%|████████▏ | 184575/225000 [348:42:50<60:06:21,  5.35s/it]
 82%|████████▏ | 184576/225000 [348:42:55<59:47:14,  5.32s/it]
 82%|████████▏ | 184577/225000 [348:43:01<59:20:22,  5.28s/it]
 82%|████████▏ | 184578/225000 [348:43:06<60:42:07,  5.41s/it]
 82%|████████▏ | 184579/225000 [348:43:11<60:08:36,  5.36s/it]
 82%|████████▏ | 184580/225000 [348:43:17<60:41:41,  5.41s/it]
 82%|████████▏ | 184581/225000 [348:43:30<85:54:59,  7.65s/it]
 82%|████████▏ | 184582/225000 [348:43:35<76:50:25,  6.84s/it]
 82%|████████▏ | 184583/225000 [348:43:39<66:19:48,  5.91s/it]
 82%|████████▏ | 184584/225000 [348:43:43<62:42:22,  5.59s/it]
 82%|████████▏ | 184585/225000 [348:43:48<59:09:33,  5.27s/it]
 82%|████████▏ | 184586/225000 [348:43:53<56:45:11,  5.06s/it]
 82%|████████▏ | 184587/225000 [348:44:04<79:16:48,  7.06s/it]
 82%|████████▏ | 184588/225000 [348:44:08<68:35:36,  6.11s/it]
 82%|████████▏ | 184589/225000 [348:44:17<76:24:43,  6.81s/it]
 82%|████████▏ | 184590/225000 [348:44:21<69:16:52,  6.17s/it]
 82%|████████▏ | 184591/225000 [348:44:26<63:14:27,  5.63s/it]
 82%|████████▏ | 184592/225000 [348:44:30<58:22:54,  5.20s/it]
 82%|████████▏ | 184593/225000 [348:44:35<56:45:26,  5.06s/it]
 82%|████████▏ | 184594/225000 [348:44:39<55:42:05,  4.96s/it]
 82%|████████▏ | 184595/225000 [348:44:44<54:41:45,  4.87s/it]
 82%|████████▏ | 184596/225000 [348:44:49<53:52:54,  4.80s/it]
 82%|████████▏ | 184597/225000 [348:44:54<55:00:41,  4.90s/it]
 82%|████████▏ | 184598/225000 [348:45:04<74:45:27,  6.66s/it]
 82%|████████▏ | 184599/225000 [348:45:10<71:07:19,  6.34s/it]
 82%|████████▏ | 184600/225000 [348:45:14<61:20:40,  5.47s/it]
                                                              
{'loss': '0.7546', 'grad_norm': '1.082', 'learning_rate': '4.249e-06', 'epoch': '0.8204', 'num_input_tokens_seen': 578517536, 'train_runtime': '1.256e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184600/225000 [348:45:14<61:20:40,  5.47s/it]
 82%|████████▏ | 184601/225000 [348:45:21<68:45:09,  6.13s/it]
 82%|████████▏ | 184602/225000 [348:45:28<71:25:59,  6.37s/it]
 82%|████████▏ | 184603/225000 [348:45:33<68:09:59,  6.07s/it]
 82%|████████▏ | 184604/225000 [348:45:39<64:54:31,  5.78s/it]
 82%|████████▏ | 184605/225000 [348:45:43<60:57:11,  5.43s/it]
 82%|████████▏ | 184606/225000 [348:45:48<59:40:02,  5.32s/it]
 82%|████████▏ | 184607/225000 [348:45:53<58:17:10,  5.19s/it]
 82%|████████▏ | 184608/225000 [348:45:58<57:22:04,  5.11s/it]
 82%|████████▏ | 184609/225000 [348:46:03<56:48:45,  5.06s/it]
 82%|████████▏ | 184610/225000 [348:46:08<55:57:14,  4.99s/it]
 82%|████████▏ | 184611/225000 [348:46:13<56:28:30,  5.03s/it]
 82%|████████▏ | 184612/225000 [348:46:20<63:34:29,  5.67s/it]
 82%|████████▏ | 184613/225000 [348:46:31<81:12:45,  7.24s/it]
 82%|████████▏ | 184614/225000 [348:46:34<67:22:03,  6.01s/it]
 82%|████████▏ | 184615/225000 [348:46:40<68:04:35,  6.07s/it]
 82%|████████▏ | 184616/225000 [348:46:47<69:33:53,  6.20s/it]
 82%|████████▏ | 184617/225000 [348:46:53<70:14:56,  6.26s/it]
 82%|████████▏ | 184618/225000 [348:46:57<63:00:39,  5.62s/it]
 82%|████████▏ | 184619/225000 [348:47:01<57:50:13,  5.16s/it]
 82%|████████▏ | 184620/225000 [348:47:06<54:55:39,  4.90s/it]
 82%|████████▏ | 184621/225000 [348:47:16<73:38:04,  6.56s/it]
 82%|████████▏ | 184622/225000 [348:47:20<63:30:09,  5.66s/it]
 82%|████████▏ | 184623/225000 [348:47:27<67:37:44,  6.03s/it]
 82%|████████▏ | 184624/225000 [348:47:31<61:58:56,  5.53s/it]
 82%|████████▏ | 184625/225000 [348:47:36<59:17:41,  5.29s/it]
 82%|████████▏ | 184626/225000 [348:47:40<56:01:46,  5.00s/it]
 82%|████████▏ | 184627/225000 [348:47:44<53:33:16,  4.78s/it]
 82%|████████▏ | 184628/225000 [348:47:49<51:48:58,  4.62s/it]
 82%|████████▏ | 184629/225000 [348:47:59<71:27:37,  6.37s/it]
 82%|████████▏ | 184630/225000 [348:48:02<60:52:32,  5.43s/it]
 82%|████████▏ | 184631/225000 [348:48:10<68:14:43,  6.09s/it]
 82%|████████▏ | 184632/225000 [348:48:14<59:53:44,  5.34s/it]
 82%|████████▏ | 184633/225000 [348:48:21<68:23:14,  6.10s/it]
 82%|████████▏ | 184634/225000 [348:48:25<60:58:34,  5.44s/it]
 82%|████████▏ | 184635/225000 [348:48:30<57:15:39,  5.11s/it]
 82%|████████▏ | 184636/225000 [348:48:34<53:49:14,  4.80s/it]
 82%|████████▏ | 184637/225000 [348:48:38<51:54:15,  4.63s/it]
 82%|████████▏ | 184638/225000 [348:48:48<71:27:37,  6.37s/it]
 82%|████████▏ | 184639/225000 [348:48:52<62:36:41,  5.58s/it]
 82%|████████▏ | 184640/225000 [348:48:56<56:29:50,  5.04s/it]
 82%|████████▏ | 184641/225000 [348:49:05<71:16:10,  6.36s/it]
 82%|████████▏ | 184642/225000 [348:49:12<72:28:26,  6.46s/it]
 82%|████████▏ | 184643/225000 [348:49:15<61:45:32,  5.51s/it]
 82%|████████▏ | 184644/225000 [348:49:19<54:45:52,  4.89s/it]
 82%|████████▏ | 184645/225000 [348:49:23<53:51:22,  4.80s/it]
 82%|████████▏ | 184646/225000 [348:49:31<62:22:46,  5.56s/it]
 82%|████████▏ | 184647/225000 [348:49:39<72:24:28,  6.46s/it]
 82%|████████▏ | 184648/225000 [348:49:42<61:37:13,  5.50s/it]
 82%|████████▏ | 184649/225000 [348:49:47<57:17:45,  5.11s/it]
 82%|████████▏ | 184650/225000 [348:49:51<55:58:08,  4.99s/it]
                                                              
{'loss': '0.7538', 'grad_norm': '0.9585', 'learning_rate': '4.239e-06', 'epoch': '0.8207', 'num_input_tokens_seen': 578647136, 'train_runtime': '1.256e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184650/225000 [348:49:51<55:58:08,  4.99s/it]
 82%|████████▏ | 184651/225000 [348:49:56<54:09:06,  4.83s/it]
 82%|████████▏ | 184652/225000 [348:50:00<52:41:47,  4.70s/it]
 82%|████████▏ | 184653/225000 [348:50:05<52:38:47,  4.70s/it]
 82%|████████▏ | 184654/225000 [348:50:13<64:31:48,  5.76s/it]
 82%|████████▏ | 184655/225000 [348:50:23<77:23:21,  6.91s/it]
 82%|████████▏ | 184656/225000 [348:50:26<66:40:42,  5.95s/it]
 82%|████████▏ | 184657/225000 [348:50:30<58:50:00,  5.25s/it]
 82%|████████▏ | 184658/225000 [348:50:38<67:56:57,  6.06s/it]
 82%|████████▏ | 184659/225000 [348:50:43<64:41:45,  5.77s/it]
 82%|████████▏ | 184660/225000 [348:50:47<57:24:36,  5.12s/it]
 82%|████████▏ | 184661/225000 [348:50:55<66:46:37,  5.96s/it]
 82%|████████▏ | 184662/225000 [348:51:00<64:17:38,  5.74s/it]
 82%|████████▏ | 184663/225000 [348:51:04<57:04:01,  5.09s/it]
 82%|████████▏ | 184664/225000 [348:51:13<73:09:17,  6.53s/it]
 82%|████████▏ | 184665/225000 [348:51:16<61:37:39,  5.50s/it]
 82%|████████▏ | 184666/225000 [348:51:24<66:46:00,  5.96s/it]
 82%|████████▏ | 184667/225000 [348:51:29<65:34:16,  5.85s/it]
 82%|████████▏ | 184668/225000 [348:51:33<59:14:00,  5.29s/it]
 82%|████████▏ | 184669/225000 [348:51:37<55:22:15,  4.94s/it]
 82%|████████▏ | 184670/225000 [348:51:42<54:02:33,  4.82s/it]
 82%|████████▏ | 184671/225000 [348:51:46<53:00:41,  4.73s/it]
 82%|████████▏ | 184672/225000 [348:51:51<51:46:52,  4.62s/it]
 82%|████████▏ | 184673/225000 [348:52:03<76:40:05,  6.84s/it]
 82%|████████▏ | 184674/225000 [348:52:06<65:19:06,  5.83s/it]
 82%|████████▏ | 184675/225000 [348:52:11<61:00:39,  5.45s/it]
 82%|████████▏ | 184676/225000 [348:52:19<70:29:59,  6.29s/it]
 82%|████████▏ | 184677/225000 [348:52:24<66:41:24,  5.95s/it]
 82%|████████▏ | 184678/225000 [348:52:29<63:11:47,  5.64s/it]
 82%|████████▏ | 184679/225000 [348:52:34<62:11:47,  5.55s/it]
 82%|████████▏ | 184680/225000 [348:52:40<62:26:52,  5.58s/it]
 82%|████████▏ | 184681/225000 [348:52:45<60:34:38,  5.41s/it]
 82%|████████▏ | 184682/225000 [348:52:50<59:04:02,  5.27s/it]
 82%|████████▏ | 184683/225000 [348:52:55<57:17:02,  5.12s/it]
 82%|████████▏ | 184684/225000 [348:53:00<56:54:34,  5.08s/it]
 82%|████████▏ | 184685/225000 [348:53:05<56:07:26,  5.01s/it]
 82%|████████▏ | 184686/225000 [348:53:10<57:51:09,  5.17s/it]
 82%|████████▏ | 184687/225000 [348:53:15<58:25:19,  5.22s/it]
 82%|████████▏ | 184688/225000 [348:53:21<60:05:51,  5.37s/it]
 82%|████████▏ | 184689/225000 [348:53:26<58:41:17,  5.24s/it]
 82%|████████▏ | 184690/225000 [348:53:31<58:57:07,  5.26s/it]
 82%|████████▏ | 184691/225000 [348:53:37<59:16:22,  5.29s/it]
 82%|████████▏ | 184692/225000 [348:53:42<59:30:41,  5.32s/it]
 82%|████████▏ | 184693/225000 [348:53:51<71:58:58,  6.43s/it]
 82%|████████▏ | 184694/225000 [348:54:00<81:03:42,  7.24s/it]
 82%|████████▏ | 184695/225000 [348:54:04<69:50:38,  6.24s/it]
 82%|████████▏ | 184696/225000 [348:54:08<61:31:41,  5.50s/it]
 82%|████████▏ | 184697/225000 [348:54:16<69:29:37,  6.21s/it]
 82%|████████▏ | 184698/225000 [348:54:22<70:19:36,  6.28s/it]
 82%|████████▏ | 184699/225000 [348:54:26<61:06:11,  5.46s/it]
 82%|████████▏ | 184700/225000 [348:54:35<72:31:53,  6.48s/it]
                                                              
{'loss': '0.7608', 'grad_norm': '1.353', 'learning_rate': '4.228e-06', 'epoch': '0.8209', 'num_input_tokens_seen': 578776592, 'train_runtime': '1.256e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184700/225000 [348:54:35<72:31:53,  6.48s/it]
 82%|████████▏ | 184701/225000 [348:54:38<61:48:38,  5.52s/it]
 82%|████████▏ | 184702/225000 [348:54:46<71:19:00,  6.37s/it]
 82%|████████▏ | 184703/225000 [348:54:50<61:07:10,  5.46s/it]
 82%|████████▏ | 184704/225000 [348:54:54<58:55:27,  5.26s/it]
 82%|████████▏ | 184705/225000 [348:55:00<58:36:05,  5.24s/it]
 82%|████████▏ | 184706/225000 [348:55:05<58:12:45,  5.20s/it]
 82%|████████▏ | 184707/225000 [348:55:10<56:54:51,  5.09s/it]
 82%|████████▏ | 184708/225000 [348:55:14<56:05:37,  5.01s/it]
 82%|████████▏ | 184709/225000 [348:55:19<55:32:34,  4.96s/it]
 82%|████████▏ | 184710/225000 [348:55:24<54:24:26,  4.86s/it]
 82%|████████▏ | 184711/225000 [348:55:29<55:25:45,  4.95s/it]
 82%|████████▏ | 184712/225000 [348:55:34<56:37:38,  5.06s/it]
 82%|████████▏ | 184713/225000 [348:55:45<76:42:02,  6.85s/it]
 82%|████████▏ | 184714/225000 [348:55:52<76:46:24,  6.86s/it]
 82%|████████▏ | 184715/225000 [348:55:56<67:10:32,  6.00s/it]
 82%|████████▏ | 184716/225000 [348:56:02<64:40:17,  5.78s/it]
 82%|████████▏ | 184717/225000 [348:56:07<62:27:00,  5.58s/it]
 82%|████████▏ | 184718/225000 [348:56:12<61:55:03,  5.53s/it]
 82%|████████▏ | 184719/225000 [348:56:18<61:41:34,  5.51s/it]
 82%|████████▏ | 184720/225000 [348:56:23<61:29:32,  5.50s/it]
 82%|████████▏ | 184721/225000 [348:56:30<67:11:02,  6.00s/it]
 82%|████████▏ | 184722/225000 [348:56:35<64:31:32,  5.77s/it]
 82%|████████▏ | 184723/225000 [348:56:41<62:58:27,  5.63s/it]
 82%|████████▏ | 184724/225000 [348:56:46<63:05:07,  5.64s/it]
 82%|████████▏ | 184725/225000 [348:56:52<62:02:38,  5.55s/it]
 82%|████████▏ | 184726/225000 [348:56:57<60:53:09,  5.44s/it]
 82%|████████▏ | 184727/225000 [348:57:06<72:48:36,  6.51s/it]
 82%|████████▏ | 184728/225000 [348:57:13<73:03:11,  6.53s/it]
 82%|████████▏ | 184729/225000 [348:57:16<64:29:23,  5.77s/it]
 82%|████████▏ | 184730/225000 [348:57:24<71:52:10,  6.42s/it]
 82%|████████▏ | 184731/225000 [348:57:29<64:34:43,  5.77s/it]
 82%|████████▏ | 184732/225000 [348:57:35<67:27:43,  6.03s/it]
 82%|████████▏ | 184733/225000 [348:57:41<65:24:08,  5.85s/it]
 82%|████████▏ | 184734/225000 [348:57:44<58:14:36,  5.21s/it]
 82%|████████▏ | 184735/225000 [348:57:50<60:31:07,  5.41s/it]
 82%|████████▏ | 184736/225000 [348:57:59<69:52:39,  6.25s/it]
 82%|████████▏ | 184737/225000 [348:58:02<59:20:55,  5.31s/it]
 82%|████████▏ | 184738/225000 [348:58:09<66:35:31,  5.95s/it]
 82%|████████▏ | 184739/225000 [348:58:14<63:36:11,  5.69s/it]
 82%|████████▏ | 184740/225000 [348:58:19<59:08:01,  5.29s/it]
 82%|████████▏ | 184741/225000 [348:58:23<56:30:11,  5.05s/it]
 82%|████████▏ | 184742/225000 [348:58:27<53:33:27,  4.79s/it]
 82%|████████▏ | 184743/225000 [348:58:32<52:31:06,  4.70s/it]
 82%|████████▏ | 184744/225000 [348:58:38<58:05:19,  5.19s/it]
 82%|████████▏ | 184745/225000 [348:58:48<74:46:44,  6.69s/it]
 82%|████████▏ | 184746/225000 [348:58:52<64:38:16,  5.78s/it]
 82%|████████▏ | 184747/225000 [348:58:56<57:29:00,  5.14s/it]
 82%|████████▏ | 184748/225000 [348:59:00<55:32:57,  4.97s/it]
 82%|████████▏ | 184749/225000 [348:59:05<55:41:11,  4.98s/it]
 82%|████████▏ | 184750/225000 [348:59:10<55:57:16,  5.00s/it]
                                                              
{'loss': '0.7601', 'grad_norm': '1.094', 'learning_rate': '4.218e-06', 'epoch': '0.8211', 'num_input_tokens_seen': 578906728, 'train_runtime': '1.256e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184750/225000 [348:59:10<55:57:16,  5.00s/it]
 82%|████████▏ | 184751/225000 [348:59:21<76:32:14,  6.85s/it]
 82%|████████▏ | 184752/225000 [348:59:26<67:47:15,  6.06s/it]
 82%|████████▏ | 184753/225000 [348:59:30<61:51:22,  5.53s/it]
 82%|████████▏ | 184754/225000 [348:59:34<57:28:48,  5.14s/it]
 82%|████████▏ | 184755/225000 [348:59:38<54:44:38,  4.90s/it]
 82%|████████▏ | 184756/225000 [348:59:46<62:55:21,  5.63s/it]
 82%|████████▏ | 184757/225000 [348:59:55<73:26:09,  6.57s/it]
 82%|████████▏ | 184758/225000 [348:59:58<63:28:59,  5.68s/it]
 82%|████████▏ | 184759/225000 [349:00:02<58:49:23,  5.26s/it]
 82%|████████▏ | 184760/225000 [349:00:08<58:16:25,  5.21s/it]
 82%|████████▏ | 184761/225000 [349:00:12<56:09:42,  5.02s/it]
 82%|████████▏ | 184762/225000 [349:00:17<55:22:14,  4.95s/it]
 82%|████████▏ | 184763/225000 [349:00:22<54:37:29,  4.89s/it]
 82%|████████▏ | 184764/225000 [349:00:26<54:35:16,  4.88s/it]
 82%|████████▏ | 184765/225000 [349:00:38<77:41:33,  6.95s/it]
 82%|████████▏ | 184766/225000 [349:00:44<72:14:44,  6.46s/it]
 82%|████████▏ | 184767/225000 [349:00:49<68:19:27,  6.11s/it]
 82%|████████▏ | 184768/225000 [349:00:52<58:57:40,  5.28s/it]
 82%|████████▏ | 184769/225000 [349:01:01<71:24:21,  6.39s/it]
 82%|████████▏ | 184770/225000 [349:01:05<61:36:13,  5.51s/it]
 82%|████████▏ | 184771/225000 [349:01:13<70:31:27,  6.31s/it]
 82%|████████▏ | 184772/225000 [349:01:17<61:50:52,  5.53s/it]
 82%|████████▏ | 184773/225000 [349:01:20<55:53:53,  5.00s/it]
 82%|████████▏ | 184774/225000 [349:01:24<52:49:37,  4.73s/it]
 82%|████████▏ | 184775/225000 [349:01:35<73:33:03,  6.58s/it]
 82%|████████▏ | 184776/225000 [349:01:39<62:54:26,  5.63s/it]
 82%|████████▏ | 184777/225000 [349:01:46<67:25:39,  6.03s/it]
 82%|████████▏ | 184778/225000 [349:01:50<61:32:21,  5.51s/it]
 82%|████████▏ | 184779/225000 [349:01:54<56:29:39,  5.06s/it]
 82%|████████▏ | 184780/225000 [349:01:58<52:30:36,  4.70s/it]
 82%|████████▏ | 184781/225000 [349:02:09<74:15:56,  6.65s/it]
 82%|████████▏ | 184782/225000 [349:02:12<63:23:21,  5.67s/it]
 82%|████████▏ | 184783/225000 [349:02:18<62:35:55,  5.60s/it]
 82%|████████▏ | 184784/225000 [349:02:23<61:42:36,  5.52s/it]
 82%|████████▏ | 184785/225000 [349:02:28<59:03:05,  5.29s/it]
 82%|████████▏ | 184786/225000 [349:02:33<59:41:55,  5.34s/it]
 82%|████████▏ | 184787/225000 [349:02:39<60:25:39,  5.41s/it]
 82%|████████▏ | 184788/225000 [349:02:44<58:12:19,  5.21s/it]
 82%|████████▏ | 184789/225000 [349:02:49<58:39:47,  5.25s/it]
 82%|████████▏ | 184790/225000 [349:02:54<57:52:40,  5.18s/it]
 82%|████████▏ | 184791/225000 [349:03:00<58:47:07,  5.26s/it]
 82%|████████▏ | 184792/225000 [349:03:05<58:50:31,  5.27s/it]
 82%|████████▏ | 184793/225000 [349:03:10<60:02:21,  5.38s/it]
 82%|████████▏ | 184794/225000 [349:03:16<60:29:35,  5.42s/it]
 82%|████████▏ | 184795/225000 [349:03:21<59:46:22,  5.35s/it]
 82%|████████▏ | 184796/225000 [349:03:33<82:32:15,  7.39s/it]
 82%|████████▏ | 184797/225000 [349:03:36<68:21:25,  6.12s/it]
 82%|████████▏ | 184798/225000 [349:03:41<62:54:44,  5.63s/it]
 82%|████████▏ | 184799/225000 [349:03:46<60:30:18,  5.42s/it]
 82%|████████▏ | 184800/225000 [349:03:51<57:55:55,  5.19s/it]
                                                              
{'loss': '0.7656', 'grad_norm': '1.157', 'learning_rate': '4.208e-06', 'epoch': '0.8213', 'num_input_tokens_seen': 579037744, 'train_runtime': '1.257e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184800/225000 [349:03:51<57:55:55,  5.19s/it]
 82%|████████▏ | 184801/225000 [349:03:56<58:01:52,  5.20s/it]
 82%|████████▏ | 184802/225000 [349:04:08<80:16:36,  7.19s/it]
 82%|████████▏ | 184803/225000 [349:04:12<70:33:10,  6.32s/it]
 82%|████████▏ | 184804/225000 [349:04:15<60:31:13,  5.42s/it]
 82%|████████▏ | 184805/225000 [349:04:22<64:55:10,  5.81s/it]
 82%|████████▏ | 184806/225000 [349:04:27<62:12:41,  5.57s/it]
 82%|████████▏ | 184807/225000 [349:04:31<56:59:59,  5.11s/it]
 82%|████████▏ | 184808/225000 [349:04:36<56:43:29,  5.08s/it]
 82%|████████▏ | 184809/225000 [349:04:46<73:19:06,  6.57s/it]
 82%|████████▏ | 184810/225000 [349:04:50<63:39:19,  5.70s/it]
 82%|████████▏ | 184811/225000 [349:04:57<68:35:08,  6.14s/it]
 82%|████████▏ | 184812/225000 [349:05:01<61:20:50,  5.50s/it]
 82%|████████▏ | 184813/225000 [349:05:05<57:53:33,  5.19s/it]
 82%|████████▏ | 184814/225000 [349:05:10<56:13:37,  5.04s/it]
 82%|████████▏ | 184815/225000 [349:05:15<55:12:26,  4.95s/it]
 82%|████████▏ | 184816/225000 [349:05:20<55:13:18,  4.95s/it]
 82%|████████▏ | 184817/225000 [349:05:25<55:31:12,  4.97s/it]
 82%|████████▏ | 184818/225000 [349:05:33<65:14:41,  5.85s/it]
 82%|████████▏ | 184819/225000 [349:05:41<75:03:51,  6.73s/it]
 82%|████████▏ | 184820/225000 [349:05:45<64:01:28,  5.74s/it]
 82%|████████▏ | 184821/225000 [349:05:49<58:49:28,  5.27s/it]
 82%|████████▏ | 184822/225000 [349:05:57<69:30:49,  6.23s/it]
 82%|████████▏ | 184823/225000 [349:06:02<62:36:08,  5.61s/it]
 82%|████████▏ | 184824/225000 [349:06:09<69:11:31,  6.20s/it]
 82%|████████▏ | 184825/225000 [349:06:13<62:12:50,  5.57s/it]
 82%|████████▏ | 184826/225000 [349:06:17<57:23:58,  5.14s/it]
 82%|████████▏ | 184827/225000 [349:06:22<55:03:08,  4.93s/it]
 82%|████████▏ | 184828/225000 [349:06:26<53:31:29,  4.80s/it]
 82%|████████▏ | 184829/225000 [349:06:31<53:34:45,  4.80s/it]
 82%|████████▏ | 184830/225000 [349:06:36<54:18:38,  4.87s/it]
 82%|████████▏ | 184831/225000 [349:06:48<78:04:18,  7.00s/it]
 82%|████████▏ | 184832/225000 [349:06:52<68:39:35,  6.15s/it]
 82%|████████▏ | 184833/225000 [349:06:56<60:21:05,  5.41s/it]
 82%|████████▏ | 184834/225000 [349:07:05<71:29:15,  6.41s/it]
 82%|████████▏ | 184835/225000 [349:07:08<61:28:35,  5.51s/it]
 82%|████████▏ | 184836/225000 [349:07:14<61:26:55,  5.51s/it]
 82%|████████▏ | 184837/225000 [349:07:19<61:02:05,  5.47s/it]
 82%|████████▏ | 184838/225000 [349:07:25<61:34:04,  5.52s/it]
 82%|████████▏ | 184839/225000 [349:07:30<61:55:09,  5.55s/it]
 82%|████████▏ | 184840/225000 [349:07:36<62:27:44,  5.60s/it]
 82%|████████▏ | 184841/225000 [349:07:42<63:07:52,  5.66s/it]
 82%|████████▏ | 184842/225000 [349:07:47<61:26:29,  5.51s/it]
 82%|████████▏ | 184843/225000 [349:07:57<77:53:28,  6.98s/it]
 82%|████████▏ | 184844/225000 [349:08:01<66:35:57,  5.97s/it]
 82%|████████▏ | 184845/225000 [349:08:05<59:57:54,  5.38s/it]
 82%|████████▏ | 184846/225000 [349:08:14<71:05:00,  6.37s/it]
 82%|████████▏ | 184847/225000 [349:08:18<65:30:18,  5.87s/it]
 82%|████████▏ | 184848/225000 [349:08:23<61:14:02,  5.49s/it]
 82%|████████▏ | 184849/225000 [349:08:27<57:13:52,  5.13s/it]
 82%|████████▏ | 184850/225000 [349:08:32<56:09:26,  5.04s/it]
                                                              
{'loss': '0.7664', 'grad_norm': '1.146', 'learning_rate': '4.198e-06', 'epoch': '0.8216', 'num_input_tokens_seen': 579168456, 'train_runtime': '1.257e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184850/225000 [349:08:32<56:09:26,  5.04s/it]
 82%|████████▏ | 184851/225000 [349:08:37<55:14:10,  4.95s/it]
 82%|████████▏ | 184852/225000 [349:08:41<52:54:04,  4.74s/it]
 82%|████████▏ | 184853/225000 [349:08:46<51:48:12,  4.65s/it]
 82%|████████▏ | 184854/225000 [349:08:52<57:02:02,  5.11s/it]
 82%|████████▏ | 184855/225000 [349:09:03<77:26:06,  6.94s/it]
 82%|████████▏ | 184856/225000 [349:09:06<65:49:38,  5.90s/it]
 82%|████████▏ | 184857/225000 [349:09:14<72:22:26,  6.49s/it]
 82%|████████▏ | 184858/225000 [349:09:18<62:20:55,  5.59s/it]
 82%|████████▏ | 184859/225000 [349:09:26<70:23:28,  6.31s/it]
 82%|████████▏ | 184860/225000 [349:09:29<60:33:23,  5.43s/it]
 82%|████████▏ | 184861/225000 [349:09:37<67:35:38,  6.06s/it]
 82%|████████▏ | 184862/225000 [349:09:40<58:19:35,  5.23s/it]
 82%|████████▏ | 184863/225000 [349:09:49<69:31:47,  6.24s/it]
 82%|████████▏ | 184864/225000 [349:09:52<60:55:52,  5.47s/it]
 82%|████████▏ | 184865/225000 [349:09:56<55:04:54,  4.94s/it]
 82%|████████▏ | 184866/225000 [349:10:02<59:24:27,  5.33s/it]
 82%|████████▏ | 184867/225000 [349:10:10<67:44:48,  6.08s/it]
 82%|████████▏ | 184868/225000 [349:10:13<58:45:15,  5.27s/it]
 82%|████████▏ | 184869/225000 [349:10:20<62:34:20,  5.61s/it]
 82%|████████▏ | 184870/225000 [349:10:25<61:58:22,  5.56s/it]
 82%|████████▏ | 184871/225000 [349:10:33<70:17:55,  6.31s/it]
 82%|████████▏ | 184872/225000 [349:10:37<62:17:15,  5.59s/it]
 82%|████████▏ | 184873/225000 [349:10:41<56:10:40,  5.04s/it]
 82%|████████▏ | 184874/225000 [349:10:50<69:34:41,  6.24s/it]
 82%|████████▏ | 184875/225000 [349:10:54<62:20:17,  5.59s/it]
 82%|████████▏ | 184876/225000 [349:11:02<70:02:36,  6.28s/it]
 82%|████████▏ | 184877/225000 [349:11:06<61:22:04,  5.51s/it]
 82%|████████▏ | 184878/225000 [349:11:09<55:31:42,  4.98s/it]
 82%|████████▏ | 184879/225000 [349:11:15<57:19:55,  5.14s/it]
 82%|████████▏ | 184880/225000 [349:11:23<68:04:59,  6.11s/it]
 82%|████████▏ | 184881/225000 [349:11:28<61:39:36,  5.53s/it]
 82%|████████▏ | 184882/225000 [349:11:32<56:55:40,  5.11s/it]
 82%|████████▏ | 184883/225000 [349:11:39<64:16:16,  5.77s/it]
 82%|████████▏ | 184884/225000 [349:11:46<68:13:51,  6.12s/it]
 82%|████████▏ | 184885/225000 [349:11:49<59:01:12,  5.30s/it]
 82%|████████▏ | 184886/225000 [349:11:57<67:56:36,  6.10s/it]
 82%|████████▏ | 184887/225000 [349:12:01<59:29:44,  5.34s/it]
 82%|████████▏ | 184888/225000 [349:12:05<57:00:18,  5.12s/it]
 82%|████████▏ | 184889/225000 [349:12:11<57:16:02,  5.14s/it]
 82%|████████▏ | 184890/225000 [349:12:16<57:32:00,  5.16s/it]
 82%|████████▏ | 184891/225000 [349:12:21<57:04:36,  5.12s/it]
 82%|████████▏ | 184892/225000 [349:12:26<56:34:09,  5.08s/it]
 82%|████████▏ | 184893/225000 [349:12:33<64:53:28,  5.82s/it]
 82%|████████▏ | 184894/225000 [349:12:41<70:18:20,  6.31s/it]
 82%|████████▏ | 184895/225000 [349:12:45<63:23:14,  5.69s/it]
 82%|████████▏ | 184896/225000 [349:12:49<58:23:00,  5.24s/it]
 82%|████████▏ | 184897/225000 [349:12:55<59:55:12,  5.38s/it]
 82%|████████▏ | 184898/225000 [349:13:06<77:49:40,  6.99s/it]
 82%|████████▏ | 184899/225000 [349:13:09<66:20:09,  5.96s/it]
 82%|████████▏ | 184900/225000 [349:13:13<59:39:42,  5.36s/it]
                                                              
{'loss': '0.7651', 'grad_norm': '1.13', 'learning_rate': '4.188e-06', 'epoch': '0.8218', 'num_input_tokens_seen': 579298792, 'train_runtime': '1.257e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184900/225000 [349:13:13<59:39:42,  5.36s/it]
 82%|████████▏ | 184901/225000 [349:13:17<55:23:04,  4.97s/it]
 82%|████████▏ | 184902/225000 [349:13:26<68:17:55,  6.13s/it]
 82%|████████▏ | 184903/225000 [349:13:34<74:11:11,  6.66s/it]
 82%|████████▏ | 184904/225000 [349:13:38<64:27:26,  5.79s/it]
 82%|████████▏ | 184905/225000 [349:13:42<60:27:11,  5.43s/it]
 82%|████████▏ | 184906/225000 [349:13:47<56:50:57,  5.10s/it]
 82%|████████▏ | 184907/225000 [349:13:51<55:06:36,  4.95s/it]
 82%|████████▏ | 184908/225000 [349:13:56<53:24:51,  4.80s/it]
 82%|████████▏ | 184909/225000 [349:14:01<53:29:03,  4.80s/it]
 82%|████████▏ | 184910/225000 [349:14:05<50:48:18,  4.56s/it]
 82%|████████▏ | 184911/225000 [349:14:17<76:37:04,  6.88s/it]
 82%|████████▏ | 184912/225000 [349:14:21<66:29:40,  5.97s/it]
 82%|████████▏ | 184913/225000 [349:14:24<58:09:30,  5.22s/it]
 82%|████████▏ | 184914/225000 [349:14:34<73:06:28,  6.57s/it]
 82%|████████▏ | 184915/225000 [349:14:37<63:08:04,  5.67s/it]
 82%|████████▏ | 184916/225000 [349:14:41<55:29:25,  4.98s/it]
 82%|████████▏ | 184917/225000 [349:14:48<64:01:31,  5.75s/it]
 82%|████████▏ | 184918/225000 [349:14:52<57:53:45,  5.20s/it]
 82%|████████▏ | 184919/225000 [349:14:56<53:08:39,  4.77s/it]
 82%|████████▏ | 184920/225000 [349:15:06<70:15:18,  6.31s/it]
 82%|████████▏ | 184921/225000 [349:15:10<62:38:51,  5.63s/it]
 82%|████████▏ | 184922/225000 [349:15:14<58:06:48,  5.22s/it]
 82%|████████▏ | 184923/225000 [349:15:19<55:00:11,  4.94s/it]
 82%|████████▏ | 184924/225000 [349:15:23<53:26:29,  4.80s/it]
 82%|████████▏ | 184925/225000 [349:15:33<71:18:56,  6.41s/it]
 82%|████████▏ | 184926/225000 [349:15:39<68:52:14,  6.19s/it]
 82%|████████▏ | 184927/225000 [349:15:42<58:56:51,  5.30s/it]
 82%|████████▏ | 184928/225000 [349:15:49<64:37:15,  5.81s/it]
 82%|████████▏ | 184929/225000 [349:15:54<61:50:35,  5.56s/it]
 82%|████████▏ | 184930/225000 [349:15:59<59:34:39,  5.35s/it]
 82%|████████▏ | 184931/225000 [349:16:03<56:12:16,  5.05s/it]
 82%|████████▏ | 184932/225000 [349:16:07<53:15:44,  4.79s/it]
 82%|████████▏ | 184933/225000 [349:16:14<58:30:40,  5.26s/it]
 82%|████████▏ | 184934/225000 [349:16:25<77:12:28,  6.94s/it]
 82%|████████▏ | 184935/225000 [349:16:28<65:34:48,  5.89s/it]
 82%|████████▏ | 184936/225000 [349:16:36<71:07:45,  6.39s/it]
 82%|████████▏ | 184937/225000 [349:16:40<63:29:12,  5.70s/it]
 82%|████████▏ | 184938/225000 [349:16:44<59:27:12,  5.34s/it]
 82%|████████▏ | 184939/225000 [349:16:48<55:07:03,  4.95s/it]
 82%|████████▏ | 184940/225000 [349:16:53<54:10:08,  4.87s/it]
 82%|████████▏ | 184941/225000 [349:17:03<71:38:18,  6.44s/it]
 82%|████████▏ | 184942/225000 [349:17:07<62:04:55,  5.58s/it]
 82%|████████▏ | 184943/225000 [349:17:10<56:03:46,  5.04s/it]
 82%|████████▏ | 184944/225000 [349:17:19<68:43:13,  6.18s/it]
 82%|████████▏ | 184945/225000 [349:17:22<57:39:56,  5.18s/it]
 82%|████████▏ | 184946/225000 [349:17:27<55:15:29,  4.97s/it]
 82%|████████▏ | 184947/225000 [349:17:31<52:21:00,  4.71s/it]
 82%|████████▏ | 184948/225000 [349:17:35<51:29:37,  4.63s/it]
 82%|████████▏ | 184949/225000 [349:17:40<53:48:19,  4.84s/it]
 82%|████████▏ | 184950/225000 [349:17:52<75:17:05,  6.77s/it]
                                                              
{'loss': '0.7626', 'grad_norm': '1.147', 'learning_rate': '4.178e-06', 'epoch': '0.822', 'num_input_tokens_seen': 579428248, 'train_runtime': '1.257e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 184950/225000 [349:17:52<75:17:05,  6.77s/it]
 82%|████████▏ | 184951/225000 [349:17:56<65:58:19,  5.93s/it]
 82%|████████▏ | 184952/225000 [349:18:00<60:43:12,  5.46s/it]
 82%|████████▏ | 184953/225000 [349:18:04<57:13:10,  5.14s/it]
 82%|████████▏ | 184954/225000 [349:18:09<55:17:42,  4.97s/it]
 82%|████████▏ | 184955/225000 [349:18:19<71:48:12,  6.46s/it]
 82%|████████▏ | 184956/225000 [349:18:23<63:03:52,  5.67s/it]
 82%|████████▏ | 184957/225000 [349:18:28<59:53:17,  5.38s/it]
 82%|████████▏ | 184958/225000 [349:18:35<67:34:00,  6.07s/it]
 82%|████████▏ | 184959/225000 [349:18:40<63:43:28,  5.73s/it]
 82%|████████▏ | 184960/225000 [349:18:48<70:02:07,  6.30s/it]
 82%|████████▏ | 184961/225000 [349:18:52<63:15:04,  5.69s/it]
 82%|████████▏ | 184962/225000 [349:18:58<65:30:11,  5.89s/it]
 82%|████████▏ | 184963/225000 [349:19:02<59:18:45,  5.33s/it]
 82%|████████▏ | 184964/225000 [349:19:07<55:18:24,  4.97s/it]
 82%|████████▏ | 184965/225000 [349:19:16<70:15:16,  6.32s/it]
 82%|████████▏ | 184966/225000 [349:19:21<66:57:12,  6.02s/it]
 82%|████████▏ | 184967/225000 [349:19:26<63:19:27,  5.69s/it]
 82%|████████▏ | 184968/225000 [349:19:32<61:49:49,  5.56s/it]
 82%|████████▏ | 184969/225000 [349:19:36<57:52:31,  5.20s/it]
 82%|████████▏ | 184970/225000 [349:19:41<56:46:35,  5.11s/it]
 82%|████████▏ | 184971/225000 [349:19:46<56:23:29,  5.07s/it]
 82%|████████▏ | 184972/225000 [349:19:51<56:17:45,  5.06s/it]
 82%|████████▏ | 184973/225000 [349:19:55<54:30:02,  4.90s/it]
 82%|████████▏ | 184974/225000 [349:20:00<54:10:15,  4.87s/it]
 82%|████████▏ | 184975/225000 [349:20:05<55:03:09,  4.95s/it]
 82%|████████▏ | 184976/225000 [349:20:13<62:55:24,  5.66s/it]
 82%|████████▏ | 184977/225000 [349:20:23<80:24:14,  7.23s/it]
 82%|████████▏ | 184978/225000 [349:20:27<69:19:26,  6.24s/it]
 82%|████████▏ | 184979/225000 [349:20:31<61:55:44,  5.57s/it]
 82%|████████▏ | 184980/225000 [349:20:37<62:52:39,  5.66s/it]
 82%|████████▏ | 184981/225000 [349:20:46<73:18:06,  6.59s/it]
 82%|████████▏ | 184982/225000 [349:20:50<65:54:56,  5.93s/it]
 82%|████████▏ | 184983/225000 [349:20:57<68:15:11,  6.14s/it]
 82%|████████▏ | 184984/225000 [349:21:02<63:51:59,  5.75s/it]
 82%|████████▏ | 184985/225000 [349:21:07<60:20:55,  5.43s/it]
 82%|████████▏ | 184986/225000 [349:21:11<57:20:08,  5.16s/it]
 82%|████████▏ | 184987/225000 [349:21:16<57:32:12,  5.18s/it]
 82%|████████▏ | 184988/225000 [349:21:21<56:11:25,  5.06s/it]
 82%|████████▏ | 184989/225000 [349:21:26<56:37:10,  5.09s/it]
 82%|████████▏ | 184990/225000 [349:21:31<56:30:17,  5.08s/it]
 82%|████████▏ | 184991/225000 [349:21:36<55:47:44,  5.02s/it]
 82%|████████▏ | 184992/225000 [349:21:41<55:05:42,  4.96s/it]
 82%|████████▏ | 184993/225000 [349:21:46<54:09:03,  4.87s/it]
 82%|████████▏ | 184994/225000 [349:21:50<53:41:12,  4.83s/it]
 82%|████████▏ | 184995/225000 [349:22:00<67:58:39,  6.12s/it]
 82%|████████▏ | 184996/225000 [349:22:06<68:48:30,  6.19s/it]
 82%|████████▏ | 184997/225000 [349:22:13<72:03:22,  6.48s/it]
 82%|████████▏ | 184998/225000 [349:22:17<61:53:25,  5.57s/it]
 82%|████████▏ | 184999/225000 [349:22:25<70:54:39,  6.38s/it]
 82%|████████▏ | 185000/225000 [349:22:30<65:53:17,  5.93s/it]
                                                              
{'loss': '0.7594', 'grad_norm': '1.202', 'learning_rate': '4.168e-06', 'epoch': '0.8222', 'num_input_tokens_seen': 579558240, 'train_runtime': '1.258e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185000/225000 [349:22:30<65:53:17,  5.93s/it]
 82%|████████▏ | 185001/225000 [349:22:35<63:14:37,  5.69s/it]
 82%|████████▏ | 185002/225000 [349:22:40<60:12:40,  5.42s/it]
 82%|████████▏ | 185003/225000 [349:22:44<56:36:06,  5.09s/it]
 82%|████████▏ | 185004/225000 [349:22:49<55:02:25,  4.95s/it]
 82%|████████▏ | 185005/225000 [349:22:53<53:58:10,  4.86s/it]
 82%|████████▏ | 185006/225000 [349:23:02<65:51:03,  5.93s/it]
 82%|████████▏ | 185007/225000 [349:23:09<69:17:34,  6.24s/it]
 82%|████████▏ | 185008/225000 [349:23:13<61:46:12,  5.56s/it]
 82%|████████▏ | 185009/225000 [349:23:17<57:25:52,  5.17s/it]
 82%|████████▏ | 185010/225000 [349:23:21<54:38:44,  4.92s/it]
 82%|████████▏ | 185011/225000 [349:23:31<71:58:02,  6.48s/it]
 82%|████████▏ | 185012/225000 [349:23:35<63:54:07,  5.75s/it]
 82%|████████▏ | 185013/225000 [349:23:40<58:48:46,  5.29s/it]
 82%|████████▏ | 185014/225000 [349:23:44<54:58:01,  4.95s/it]
 82%|████████▏ | 185015/225000 [349:23:48<52:25:20,  4.72s/it]
 82%|████████▏ | 185016/225000 [349:23:53<54:05:33,  4.87s/it]
 82%|████████▏ | 185017/225000 [349:24:00<62:00:54,  5.58s/it]
 82%|████████▏ | 185018/225000 [349:24:06<60:52:25,  5.48s/it]
 82%|████████▏ | 185019/225000 [349:24:14<69:20:28,  6.24s/it]
 82%|████████▏ | 185020/225000 [349:24:21<72:03:47,  6.49s/it]
 82%|████████▏ | 185021/225000 [349:24:24<61:12:22,  5.51s/it]
 82%|████████▏ | 185022/225000 [349:24:32<71:18:48,  6.42s/it]
 82%|████████▏ | 185023/225000 [349:24:39<72:59:02,  6.57s/it]
 82%|████████▏ | 185024/225000 [349:24:43<63:46:06,  5.74s/it]
 82%|████████▏ | 185025/225000 [349:24:48<61:08:21,  5.51s/it]
 82%|████████▏ | 185026/225000 [349:24:56<70:29:56,  6.35s/it]
 82%|████████▏ | 185027/225000 [349:25:00<62:34:13,  5.64s/it]
 82%|████████▏ | 185028/225000 [349:25:05<58:27:54,  5.27s/it]
 82%|████████▏ | 185029/225000 [349:25:10<57:40:26,  5.19s/it]
 82%|████████▏ | 185030/225000 [349:25:18<67:03:08,  6.04s/it]
 82%|████████▏ | 185031/225000 [349:25:24<66:20:08,  5.97s/it]
 82%|████████▏ | 185032/225000 [349:25:28<61:18:16,  5.52s/it]
 82%|████████▏ | 185033/225000 [349:25:33<58:04:20,  5.23s/it]
 82%|████████▏ | 185034/225000 [349:25:37<55:25:27,  4.99s/it]
 82%|████████▏ | 185035/225000 [349:25:42<54:24:29,  4.90s/it]
 82%|████████▏ | 185036/225000 [349:25:47<56:01:46,  5.05s/it]
 82%|████████▏ | 185037/225000 [349:25:59<78:16:39,  7.05s/it]
 82%|████████▏ | 185038/225000 [349:26:03<66:54:09,  6.03s/it]
 82%|████████▏ | 185039/225000 [349:26:08<65:51:41,  5.93s/it]
 82%|████████▏ | 185040/225000 [349:26:14<64:13:16,  5.79s/it]
 82%|████████▏ | 185041/225000 [349:26:20<66:49:37,  6.02s/it]
 82%|████████▏ | 185042/225000 [349:26:25<61:25:37,  5.53s/it]
 82%|████████▏ | 185043/225000 [349:26:28<54:33:59,  4.92s/it]
 82%|████████▏ | 185044/225000 [349:26:32<52:33:59,  4.74s/it]
 82%|████████▏ | 185045/225000 [349:26:37<51:50:54,  4.67s/it]
 82%|████████▏ | 185046/225000 [349:26:41<49:18:25,  4.44s/it]
 82%|████████▏ | 185047/225000 [349:26:52<70:59:45,  6.40s/it]
 82%|████████▏ | 185048/225000 [349:27:00<75:26:56,  6.80s/it]
 82%|████████▏ | 185049/225000 [349:27:04<67:30:36,  6.08s/it]
 82%|████████▏ | 185050/225000 [349:27:08<61:38:47,  5.56s/it]
                                                              
{'loss': '0.7663', 'grad_norm': '1.193', 'learning_rate': '4.158e-06', 'epoch': '0.8224', 'num_input_tokens_seen': 579688312, 'train_runtime': '1.258e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185050/225000 [349:27:08<61:38:47,  5.56s/it]
 82%|████████▏ | 185051/225000 [349:27:13<57:14:16,  5.16s/it]
 82%|████████▏ | 185052/225000 [349:27:16<53:01:34,  4.78s/it]
 82%|████████▏ | 185053/225000 [349:27:27<72:27:49,  6.53s/it]
 82%|████████▏ | 185054/225000 [349:27:33<68:59:27,  6.22s/it]
 82%|████████▏ | 185055/225000 [349:27:36<59:25:46,  5.36s/it]
 82%|████████▏ | 185056/225000 [349:27:44<68:53:43,  6.21s/it]
 82%|████████▏ | 185057/225000 [349:27:48<62:40:58,  5.65s/it]
 82%|████████▏ | 185058/225000 [349:27:53<57:42:40,  5.20s/it]
 82%|████████▏ | 185059/225000 [349:27:57<55:03:48,  4.96s/it]
 82%|████████▏ | 185060/225000 [349:28:04<60:34:56,  5.46s/it]
 82%|████████▏ | 185061/225000 [349:28:13<74:05:33,  6.68s/it]
 82%|████████▏ | 185062/225000 [349:28:17<63:42:48,  5.74s/it]
 82%|████████▏ | 185063/225000 [349:28:21<58:00:03,  5.23s/it]
 82%|████████▏ | 185064/225000 [349:28:24<51:56:50,  4.68s/it]
 82%|████████▏ | 185065/225000 [349:28:33<64:45:46,  5.84s/it]
 82%|████████▏ | 185066/225000 [349:28:37<60:06:37,  5.42s/it]
 82%|████████▏ | 185067/225000 [349:28:44<65:18:57,  5.89s/it]
 82%|████████▏ | 185068/225000 [349:28:52<70:18:30,  6.34s/it]
 82%|████████▏ | 185069/225000 [349:28:55<60:43:37,  5.47s/it]
 82%|████████▏ | 185070/225000 [349:28:59<57:15:26,  5.16s/it]
 82%|████████▏ | 185071/225000 [349:29:04<54:56:02,  4.95s/it]
 82%|████████▏ | 185072/225000 [349:29:09<56:26:46,  5.09s/it]
 82%|████████▏ | 185073/225000 [349:29:14<56:50:35,  5.13s/it]
 82%|████████▏ | 185074/225000 [349:29:20<56:53:22,  5.13s/it]
 82%|████████▏ | 185075/225000 [349:29:25<57:06:52,  5.15s/it]
 82%|████████▏ | 185076/225000 [349:29:30<55:53:59,  5.04s/it]
 82%|████████▏ | 185077/225000 [349:29:35<56:52:36,  5.13s/it]
 82%|████████▏ | 185078/225000 [349:29:44<71:11:30,  6.42s/it]
 82%|████████▏ | 185079/225000 [349:29:52<74:49:13,  6.75s/it]
 82%|████████▏ | 185080/225000 [349:29:55<61:23:44,  5.54s/it]
 82%|████████▏ | 185081/225000 [349:29:58<55:03:57,  4.97s/it]
 82%|████████▏ | 185082/225000 [349:30:06<65:58:27,  5.95s/it]
 82%|████████▏ | 185083/225000 [349:30:10<57:53:19,  5.22s/it]
 82%|████████▏ | 185084/225000 [349:30:15<57:18:44,  5.17s/it]
 82%|████████▏ | 185085/225000 [349:30:23<67:43:40,  6.11s/it]
 82%|████████▏ | 185086/225000 [349:30:26<57:42:43,  5.21s/it]
 82%|████████▏ | 185087/225000 [349:30:35<68:20:05,  6.16s/it]
 82%|████████▏ | 185088/225000 [349:30:38<59:37:47,  5.38s/it]
 82%|████████▏ | 185089/225000 [349:30:46<67:06:45,  6.05s/it]
 82%|████████▏ | 185090/225000 [349:30:50<59:31:05,  5.37s/it]
 82%|████████▏ | 185091/225000 [349:30:54<54:02:33,  4.87s/it]
 82%|████████▏ | 185092/225000 [349:31:03<70:16:13,  6.34s/it]
 82%|████████▏ | 185093/225000 [349:31:07<62:00:13,  5.59s/it]
 82%|████████▏ | 185094/225000 [349:31:13<61:34:45,  5.56s/it]
 82%|████████▏ | 185095/225000 [349:31:21<70:19:46,  6.34s/it]
 82%|████████▏ | 185096/225000 [349:31:24<60:27:40,  5.45s/it]
 82%|████████▏ | 185097/225000 [349:31:31<65:15:32,  5.89s/it]
 82%|████████▏ | 185098/225000 [349:31:35<57:08:21,  5.16s/it]
 82%|████████▏ | 185099/225000 [349:31:39<54:16:30,  4.90s/it]
 82%|████████▏ | 185100/225000 [349:31:46<63:23:23,  5.72s/it]
                                                              
{'loss': '0.7751', 'grad_norm': '1.176', 'learning_rate': '4.147e-06', 'epoch': '0.8227', 'num_input_tokens_seen': 579817576, 'train_runtime': '1.258e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185100/225000 [349:31:46<63:23:23,  5.72s/it]
 82%|████████▏ | 185101/225000 [349:31:50<56:53:57,  5.13s/it]
 82%|████████▏ | 185102/225000 [349:31:54<52:46:08,  4.76s/it]
 82%|████████▏ | 185103/225000 [349:32:03<65:05:34,  5.87s/it]
 82%|████████▏ | 185104/225000 [349:32:06<58:14:30,  5.26s/it]
 82%|████████▏ | 185105/225000 [349:32:10<52:27:26,  4.73s/it]
 82%|████████▏ | 185106/225000 [349:32:19<65:53:24,  5.95s/it]
 82%|████████▏ | 185107/225000 [349:32:23<62:04:32,  5.60s/it]
 82%|████████▏ | 185108/225000 [349:32:30<65:57:36,  5.95s/it]
 82%|████████▏ | 185109/225000 [349:32:36<65:03:55,  5.87s/it]
 82%|████████▏ | 185110/225000 [349:32:41<63:36:59,  5.74s/it]
 82%|████████▏ | 185111/225000 [349:32:47<62:15:02,  5.62s/it]
 82%|████████▏ | 185112/225000 [349:32:51<57:38:46,  5.20s/it]
 82%|████████▏ | 185113/225000 [349:33:00<71:47:37,  6.48s/it]
 82%|████████▏ | 185114/225000 [349:33:05<64:07:45,  5.79s/it]
 82%|████████▏ | 185115/225000 [349:33:09<58:42:29,  5.30s/it]
 82%|████████▏ | 185116/225000 [349:33:13<55:36:58,  5.02s/it]
 82%|████████▏ | 185117/225000 [349:33:18<54:25:45,  4.91s/it]
 82%|████████▏ | 185118/225000 [349:33:24<57:37:58,  5.20s/it]
 82%|████████▏ | 185119/225000 [349:33:35<79:30:43,  7.18s/it]
 82%|████████▏ | 185120/225000 [349:33:40<69:52:18,  6.31s/it]
 82%|████████▏ | 185121/225000 [349:33:44<61:58:27,  5.59s/it]
 82%|████████▏ | 185122/225000 [349:33:48<57:39:13,  5.20s/it]
 82%|████████▏ | 185123/225000 [349:33:52<55:22:01,  5.00s/it]
 82%|████████▏ | 185124/225000 [349:34:00<65:25:06,  5.91s/it]
 82%|████████▏ | 185125/225000 [349:34:07<68:46:13,  6.21s/it]
 82%|████████▏ | 185126/225000 [349:34:12<63:02:32,  5.69s/it]
 82%|████████▏ | 185127/225000 [349:34:19<66:53:41,  6.04s/it]
 82%|████████▏ | 185128/225000 [349:34:23<59:52:38,  5.41s/it]
 82%|████████▏ | 185129/225000 [349:34:27<56:29:09,  5.10s/it]
 82%|████████▏ | 185130/225000 [349:34:31<53:58:57,  4.87s/it]
 82%|████████▏ | 185131/225000 [349:34:36<53:38:11,  4.84s/it]
 82%|████████▏ | 185132/225000 [349:34:41<52:46:03,  4.76s/it]
 82%|████████▏ | 185133/225000 [349:34:52<74:01:33,  6.68s/it]
 82%|████████▏ | 185134/225000 [349:34:57<69:50:03,  6.31s/it]
 82%|████████▏ | 185135/225000 [349:35:01<61:06:15,  5.52s/it]
 82%|████████▏ | 185136/225000 [349:35:10<73:02:58,  6.60s/it]
 82%|████████▏ | 185137/225000 [349:35:14<64:51:08,  5.86s/it]
 82%|████████▏ | 185138/225000 [349:35:19<59:40:25,  5.39s/it]
 82%|████████▏ | 185139/225000 [349:35:23<56:56:43,  5.14s/it]
 82%|████████▏ | 185140/225000 [349:35:28<55:08:10,  4.98s/it]
 82%|████████▏ | 185141/225000 [349:35:32<52:47:24,  4.77s/it]
 82%|████████▏ | 185142/225000 [349:35:41<65:56:54,  5.96s/it]
 82%|████████▏ | 185143/225000 [349:35:48<71:05:08,  6.42s/it]
 82%|████████▏ | 185144/225000 [349:35:51<60:24:15,  5.46s/it]
 82%|████████▏ | 185145/225000 [349:35:59<68:26:17,  6.18s/it]
 82%|████████▏ | 185146/225000 [349:36:03<59:52:42,  5.41s/it]
 82%|████████▏ | 185147/225000 [349:36:07<56:52:47,  5.14s/it]
 82%|████████▏ | 185148/225000 [349:36:12<54:46:44,  4.95s/it]
 82%|████████▏ | 185149/225000 [349:36:16<52:28:29,  4.74s/it]
 82%|████████▏ | 185150/225000 [349:36:20<51:03:10,  4.61s/it]
                                                              
{'loss': '0.7666', 'grad_norm': '1.118', 'learning_rate': '4.137e-06', 'epoch': '0.8229', 'num_input_tokens_seen': 579949296, 'train_runtime': '1.259e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185150/225000 [349:36:20<51:03:10,  4.61s/it]
 82%|████████▏ | 185151/225000 [349:36:27<58:51:35,  5.32s/it]
 82%|████████▏ | 185152/225000 [349:36:38<74:39:36,  6.75s/it]
 82%|████████▏ | 185153/225000 [349:36:41<63:09:46,  5.71s/it]
 82%|████████▏ | 185154/225000 [349:36:49<70:36:59,  6.38s/it]
 82%|████████▏ | 185155/225000 [349:36:54<65:39:44,  5.93s/it]
 82%|████████▏ | 185156/225000 [349:36:59<64:24:31,  5.82s/it]
 82%|████████▏ | 185157/225000 [349:37:03<56:38:51,  5.12s/it]
 82%|████████▏ | 185158/225000 [349:37:07<52:59:49,  4.79s/it]
 82%|████████▏ | 185159/225000 [349:37:11<51:26:19,  4.65s/it]
 82%|████████▏ | 185160/225000 [349:37:15<50:54:25,  4.60s/it]
 82%|████████▏ | 185161/225000 [349:37:29<79:31:00,  7.19s/it]
 82%|████████▏ | 185162/225000 [349:37:32<67:50:03,  6.13s/it]
 82%|████████▏ | 185163/225000 [349:37:36<59:36:57,  5.39s/it]
 82%|████████▏ | 185164/225000 [349:37:41<58:12:56,  5.26s/it]
 82%|████████▏ | 185165/225000 [349:37:46<58:42:02,  5.30s/it]
 82%|████████▏ | 185166/225000 [349:37:56<71:45:33,  6.49s/it]
 82%|████████▏ | 185167/225000 [349:37:59<61:29:48,  5.56s/it]
 82%|████████▏ | 185168/225000 [349:38:07<68:33:55,  6.20s/it]
 82%|████████▏ | 185169/225000 [349:38:11<61:20:42,  5.54s/it]
 82%|████████▏ | 185170/225000 [349:38:15<56:20:36,  5.09s/it]
 82%|████████▏ | 185171/225000 [349:38:23<67:48:04,  6.13s/it]
 82%|████████▏ | 185172/225000 [349:38:27<58:12:20,  5.26s/it]
 82%|████████▏ | 185173/225000 [349:38:34<64:39:27,  5.84s/it]
 82%|████████▏ | 185174/225000 [349:38:37<57:04:56,  5.16s/it]
 82%|████████▏ | 185175/225000 [349:38:46<67:42:49,  6.12s/it]
 82%|████████▏ | 185176/225000 [349:38:49<58:14:44,  5.27s/it]
 82%|████████▏ | 185177/225000 [349:38:57<65:44:50,  5.94s/it]
 82%|████████▏ | 185178/225000 [349:39:01<59:26:08,  5.37s/it]
 82%|████████▏ | 185179/225000 [349:39:05<54:59:18,  4.97s/it]
 82%|████████▏ | 185180/225000 [349:39:09<53:55:18,  4.87s/it]
 82%|████████▏ | 185181/225000 [349:39:14<51:59:01,  4.70s/it]
 82%|████████▏ | 185182/225000 [349:39:18<51:29:48,  4.66s/it]
 82%|████████▏ | 185183/225000 [349:39:31<77:43:02,  7.03s/it]
 82%|████████▏ | 185184/225000 [349:39:34<67:08:05,  6.07s/it]
 82%|████████▏ | 185185/225000 [349:39:39<61:37:43,  5.57s/it]
 82%|████████▏ | 185186/225000 [349:39:47<69:45:46,  6.31s/it]
 82%|████████▏ | 185187/225000 [349:39:50<59:09:56,  5.35s/it]
 82%|████████▏ | 185188/225000 [349:39:57<64:53:34,  5.87s/it]
 82%|████████▏ | 185189/225000 [349:40:02<61:03:47,  5.52s/it]
 82%|████████▏ | 185190/225000 [349:40:07<60:23:54,  5.46s/it]
 82%|████████▏ | 185191/225000 [349:40:15<67:05:50,  6.07s/it]
 82%|████████▏ | 185192/225000 [349:40:18<57:53:43,  5.24s/it]
 82%|████████▏ | 185193/225000 [349:40:25<63:43:18,  5.76s/it]
 82%|████████▏ | 185194/225000 [349:40:29<59:45:59,  5.41s/it]
 82%|████████▏ | 185195/225000 [349:40:33<53:57:19,  4.88s/it]
 82%|████████▏ | 185196/225000 [349:40:41<64:31:14,  5.84s/it]
 82%|████████▏ | 185197/225000 [349:40:47<62:46:47,  5.68s/it]
 82%|████████▏ | 185198/225000 [349:40:50<57:10:45,  5.17s/it]
 82%|████████▏ | 185199/225000 [349:40:54<53:17:43,  4.82s/it]
 82%|████████▏ | 185200/225000 [349:41:05<72:29:30,  6.56s/it]
                                                              
{'loss': '0.7575', 'grad_norm': '1.295', 'learning_rate': '4.127e-06', 'epoch': '0.8231', 'num_input_tokens_seen': 580080584, 'train_runtime': '1.259e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185200/225000 [349:41:05<72:29:30,  6.56s/it]
 82%|████████▏ | 185201/225000 [349:41:08<61:08:49,  5.53s/it]
 82%|████████▏ | 185202/225000 [349:41:16<68:10:40,  6.17s/it]
 82%|████████▏ | 185203/225000 [349:41:20<62:06:37,  5.62s/it]
 82%|████████▏ | 185204/225000 [349:41:24<55:16:20,  5.00s/it]
 82%|████████▏ | 185205/225000 [349:41:32<65:02:16,  5.88s/it]
 82%|████████▏ | 185206/225000 [349:41:36<58:19:57,  5.28s/it]
 82%|████████▏ | 185207/225000 [349:41:39<52:08:24,  4.72s/it]
 82%|████████▏ | 185208/225000 [349:41:44<53:58:37,  4.88s/it]
 82%|████████▏ | 185209/225000 [349:41:53<68:20:11,  6.18s/it]
 82%|████████▏ | 185210/225000 [349:41:58<62:05:21,  5.62s/it]
 82%|████████▏ | 185211/225000 [349:42:02<57:40:56,  5.22s/it]
 82%|████████▏ | 185212/225000 [349:42:07<55:11:56,  4.99s/it]
 82%|████████▏ | 185213/225000 [349:42:11<53:10:27,  4.81s/it]
 82%|████████▏ | 185214/225000 [349:42:16<52:24:41,  4.74s/it]
 82%|████████▏ | 185215/225000 [349:42:26<70:23:13,  6.37s/it]
 82%|████████▏ | 185216/225000 [349:42:29<60:50:57,  5.51s/it]
 82%|████████▏ | 185217/225000 [349:42:33<56:37:09,  5.12s/it]
 82%|████████▏ | 185218/225000 [349:42:41<66:01:23,  5.97s/it]
 82%|████████▏ | 185219/225000 [349:42:47<66:04:47,  5.98s/it]
 82%|████████▏ | 185220/225000 [349:42:53<64:08:14,  5.80s/it]
 82%|████████▏ | 185221/225000 [349:42:57<59:13:36,  5.36s/it]
 82%|████████▏ | 185222/225000 [349:43:01<54:16:43,  4.91s/it]
 82%|████████▏ | 185223/225000 [349:43:05<51:33:07,  4.67s/it]
 82%|████████▏ | 185224/225000 [349:43:14<64:37:29,  5.85s/it]
 82%|████████▏ | 185225/225000 [349:43:21<70:07:26,  6.35s/it]
 82%|████████▏ | 185226/225000 [349:43:26<64:42:27,  5.86s/it]
 82%|████████▏ | 185227/225000 [349:43:30<60:22:21,  5.46s/it]
 82%|████████▏ | 185228/225000 [349:43:35<57:29:14,  5.20s/it]
 82%|████████▏ | 185229/225000 [349:43:39<54:20:08,  4.92s/it]
 82%|████████▏ | 185230/225000 [349:43:44<53:38:30,  4.86s/it]
 82%|████████▏ | 185231/225000 [349:43:49<53:38:58,  4.86s/it]
 82%|████████▏ | 185232/225000 [349:43:53<52:24:57,  4.74s/it]
 82%|████████▏ | 185233/225000 [349:43:58<51:20:20,  4.65s/it]
 82%|████████▏ | 185234/225000 [349:44:04<55:05:58,  4.99s/it]
 82%|████████▏ | 185235/225000 [349:44:15<78:01:19,  7.06s/it]
 82%|████████▏ | 185236/225000 [349:44:19<67:21:49,  6.10s/it]
 82%|████████▏ | 185237/225000 [349:44:23<60:05:18,  5.44s/it]
 82%|████████▏ | 185238/225000 [349:44:32<69:46:14,  6.32s/it]
 82%|████████▏ | 185239/225000 [349:44:36<65:07:03,  5.90s/it]
 82%|████████▏ | 185240/225000 [349:44:44<69:52:55,  6.33s/it]
 82%|████████▏ | 185241/225000 [349:44:47<60:32:01,  5.48s/it]
 82%|████████▏ | 185242/225000 [349:44:56<71:09:46,  6.44s/it]
 82%|████████▏ | 185243/225000 [349:44:59<61:01:19,  5.53s/it]
 82%|████████▏ | 185244/225000 [349:45:05<60:24:26,  5.47s/it]
 82%|████████▏ | 185245/225000 [349:45:13<69:33:30,  6.30s/it]
 82%|████████▏ | 185246/225000 [349:45:17<63:39:53,  5.77s/it]
 82%|████████▏ | 185247/225000 [349:45:22<61:04:45,  5.53s/it]
 82%|████████▏ | 185248/225000 [349:45:28<60:38:38,  5.49s/it]
 82%|████████▏ | 185249/225000 [349:45:33<59:52:18,  5.42s/it]
 82%|████████▏ | 185250/225000 [349:45:38<59:46:39,  5.41s/it]
                                                              
{'loss': '0.7523', 'grad_norm': '1.088', 'learning_rate': '4.117e-06', 'epoch': '0.8233', 'num_input_tokens_seen': 580210224, 'train_runtime': '1.259e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185250/225000 [349:45:38<59:46:39,  5.41s/it]
 82%|████████▏ | 185251/225000 [349:45:44<59:15:42,  5.37s/it]
 82%|████████▏ | 185252/225000 [349:45:49<58:24:28,  5.29s/it]
 82%|████████▏ | 185253/225000 [349:45:54<58:22:54,  5.29s/it]
 82%|████████▏ | 185254/225000 [349:45:59<57:33:40,  5.21s/it]
 82%|████████▏ | 185255/225000 [349:46:04<56:33:54,  5.12s/it]
 82%|████████▏ | 185256/225000 [349:46:09<55:55:43,  5.07s/it]
 82%|████████▏ | 185257/225000 [349:46:14<56:56:09,  5.16s/it]
 82%|████████▏ | 185258/225000 [349:46:19<56:16:32,  5.10s/it]
 82%|████████▏ | 185259/225000 [349:46:24<56:20:52,  5.10s/it]
 82%|████████▏ | 185260/225000 [349:46:30<56:04:48,  5.08s/it]
 82%|████████▏ | 185261/225000 [349:46:35<55:50:52,  5.06s/it]
 82%|████████▏ | 185262/225000 [349:46:45<73:26:31,  6.65s/it]
 82%|████████▏ | 185263/225000 [349:46:53<78:15:29,  7.09s/it]
 82%|████████▏ | 185264/225000 [349:46:56<66:19:08,  6.01s/it]
 82%|████████▏ | 185265/225000 [349:47:04<70:10:07,  6.36s/it]
 82%|████████▏ | 185266/225000 [349:47:08<63:02:53,  5.71s/it]
 82%|████████▏ | 185267/225000 [349:47:12<57:37:00,  5.22s/it]
 82%|████████▏ | 185268/225000 [349:47:17<56:06:51,  5.08s/it]
 82%|████████▏ | 185269/225000 [349:47:21<53:54:01,  4.88s/it]
 82%|████████▏ | 185270/225000 [349:47:32<72:15:14,  6.55s/it]
 82%|████████▏ | 185271/225000 [349:47:35<61:18:13,  5.55s/it]
 82%|████████▏ | 185272/225000 [349:47:43<68:48:28,  6.24s/it]
 82%|████████▏ | 185273/225000 [349:47:47<64:06:08,  5.81s/it]
 82%|████████▏ | 185274/225000 [349:47:52<60:40:39,  5.50s/it]
 82%|████████▏ | 185275/225000 [349:47:57<57:31:11,  5.21s/it]
 82%|████████▏ | 185276/225000 [349:48:02<56:54:46,  5.16s/it]
 82%|████████▏ | 185277/225000 [349:48:07<56:49:39,  5.15s/it]
 82%|████████▏ | 185278/225000 [349:48:12<56:49:27,  5.15s/it]
 82%|████████▏ | 185279/225000 [349:48:17<55:48:18,  5.06s/it]
 82%|████████▏ | 185280/225000 [349:48:22<55:20:18,  5.02s/it]
 82%|████████▏ | 185281/225000 [349:48:26<53:47:30,  4.88s/it]
 82%|████████▏ | 185282/225000 [349:48:31<53:43:08,  4.87s/it]
 82%|████████▏ | 185283/225000 [349:48:36<54:11:26,  4.91s/it]
 82%|████████▏ | 185284/225000 [349:48:46<69:49:01,  6.33s/it]
 82%|████████▏ | 185285/225000 [349:48:54<74:28:41,  6.75s/it]
 82%|████████▏ | 185286/225000 [349:48:57<62:51:28,  5.70s/it]
 82%|████████▏ | 185287/225000 [349:49:01<59:05:32,  5.36s/it]
 82%|████████▏ | 185288/225000 [349:49:07<58:28:34,  5.30s/it]
 82%|████████▏ | 185289/225000 [349:49:11<57:08:03,  5.18s/it]
 82%|████████▏ | 185290/225000 [349:49:16<56:06:09,  5.09s/it]
 82%|████████▏ | 185291/225000 [349:49:22<56:55:17,  5.16s/it]
 82%|████████▏ | 185292/225000 [349:49:27<58:19:46,  5.29s/it]
 82%|████████▏ | 185293/225000 [349:49:33<59:40:46,  5.41s/it]
 82%|████████▏ | 185294/225000 [349:49:38<60:00:04,  5.44s/it]
 82%|████████▏ | 185295/225000 [349:49:44<58:51:55,  5.34s/it]
 82%|████████▏ | 185296/225000 [349:49:49<58:11:30,  5.28s/it]
 82%|████████▏ | 185297/225000 [349:49:58<70:11:46,  6.36s/it]
 82%|████████▏ | 185298/225000 [349:50:04<71:05:17,  6.45s/it]
 82%|████████▏ | 185299/225000 [349:50:12<74:57:22,  6.80s/it]
 82%|████████▏ | 185300/225000 [349:50:16<66:43:48,  6.05s/it]
                                                              
{'loss': '0.757', 'grad_norm': '1.23', 'learning_rate': '4.107e-06', 'epoch': '0.8236', 'num_input_tokens_seen': 580339192, 'train_runtime': '1.259e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185300/225000 [349:50:16<66:43:48,  6.05s/it]
 82%|████████▏ | 185301/225000 [349:50:21<63:05:28,  5.72s/it]
 82%|████████▏ | 185302/225000 [349:50:29<70:26:18,  6.39s/it]
 82%|████████▏ | 185303/225000 [349:50:34<64:02:24,  5.81s/it]
 82%|████████▏ | 185304/225000 [349:50:38<59:10:04,  5.37s/it]
 82%|████████▏ | 185305/225000 [349:50:42<56:22:12,  5.11s/it]
 82%|████████▏ | 185306/225000 [349:50:47<54:13:06,  4.92s/it]
 82%|████████▏ | 185307/225000 [349:50:55<66:15:34,  6.01s/it]
 82%|████████▏ | 185308/225000 [349:51:02<67:53:35,  6.16s/it]
 82%|████████▏ | 185309/225000 [349:51:06<62:32:54,  5.67s/it]
 82%|████████▏ | 185310/225000 [349:51:13<66:11:08,  6.00s/it]
 82%|████████▏ | 185311/225000 [349:51:17<58:47:55,  5.33s/it]
 82%|████████▏ | 185312/225000 [349:51:21<53:25:08,  4.85s/it]
 82%|████████▏ | 185313/225000 [349:51:25<51:15:49,  4.65s/it]
 82%|████████▏ | 185314/225000 [349:51:36<71:58:05,  6.53s/it]
 82%|████████▏ | 185315/225000 [349:51:39<61:22:08,  5.57s/it]
 82%|████████▏ | 185316/225000 [349:51:43<55:28:50,  5.03s/it]
 82%|████████▏ | 185317/225000 [349:51:50<61:52:32,  5.61s/it]
 82%|████████▏ | 185318/225000 [349:51:58<69:12:08,  6.28s/it]
 82%|████████▏ | 185319/225000 [349:52:01<59:45:34,  5.42s/it]
 82%|████████▏ | 185320/225000 [349:52:09<68:15:33,  6.19s/it]
 82%|████████▏ | 185321/225000 [349:52:13<60:51:43,  5.52s/it]
 82%|████████▏ | 185322/225000 [349:52:17<55:52:08,  5.07s/it]
 82%|████████▏ | 185323/225000 [349:52:21<53:04:00,  4.81s/it]
 82%|████████▏ | 185324/225000 [349:52:26<51:49:26,  4.70s/it]
 82%|████████▏ | 185325/225000 [349:52:37<73:52:44,  6.70s/it]
 82%|████████▏ | 185326/225000 [349:52:40<62:22:56,  5.66s/it]
 82%|████████▏ | 185327/225000 [349:52:48<67:23:13,  6.11s/it]
 82%|████████▏ | 185328/225000 [349:52:52<63:12:08,  5.74s/it]
 82%|████████▏ | 185329/225000 [349:52:57<60:03:31,  5.45s/it]
 82%|████████▏ | 185330/225000 [349:53:02<57:27:02,  5.21s/it]
 82%|████████▏ | 185331/225000 [349:53:07<56:23:33,  5.12s/it]
 82%|████████▏ | 185332/225000 [349:53:11<54:14:13,  4.92s/it]
 82%|████████▏ | 185333/225000 [349:53:16<53:50:43,  4.89s/it]
 82%|████████▏ | 185334/225000 [349:53:21<53:56:03,  4.89s/it]
 82%|████████▏ | 185335/225000 [349:53:26<54:15:13,  4.92s/it]
 82%|████████▏ | 185336/225000 [349:53:34<65:18:23,  5.93s/it]
 82%|████████▏ | 185337/225000 [349:53:41<69:04:27,  6.27s/it]
 82%|████████▏ | 185338/225000 [349:53:45<60:30:47,  5.49s/it]
 82%|████████▏ | 185339/225000 [349:53:54<73:10:01,  6.64s/it]
 82%|████████▏ | 185340/225000 [349:54:00<69:38:46,  6.32s/it]
 82%|████████▏ | 185341/225000 [349:54:06<68:46:30,  6.24s/it]
 82%|████████▏ | 185342/225000 [349:54:10<62:28:24,  5.67s/it]
 82%|████████▏ | 185343/225000 [349:54:14<57:22:32,  5.21s/it]
 82%|████████▏ | 185344/225000 [349:54:18<53:18:41,  4.84s/it]
 82%|████████▏ | 185345/225000 [349:54:27<65:58:08,  5.99s/it]
 82%|████████▏ | 185346/225000 [349:54:31<60:43:56,  5.51s/it]
 82%|████████▏ | 185347/225000 [349:54:35<54:07:39,  4.91s/it]
 82%|████████▏ | 185348/225000 [349:54:39<50:33:40,  4.59s/it]
 82%|████████▏ | 185349/225000 [349:54:51<74:44:08,  6.79s/it]
 82%|████████▏ | 185350/225000 [349:54:54<64:52:29,  5.89s/it]
                                                              
{'loss': '0.7627', 'grad_norm': '1.115', 'learning_rate': '4.097e-06', 'epoch': '0.8238', 'num_input_tokens_seen': 580468448, 'train_runtime': '1.26e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185350/225000 [349:54:54<64:52:29,  5.89s/it]
 82%|████████▏ | 185351/225000 [349:54:58<56:17:20,  5.11s/it]
 82%|████████▏ | 185352/225000 [349:55:07<69:20:50,  6.30s/it]
 82%|████████▏ | 185353/225000 [349:55:10<60:37:37,  5.51s/it]
 82%|████████▏ | 185354/225000 [349:55:15<57:49:11,  5.25s/it]
 82%|████████▏ | 185355/225000 [349:55:20<56:14:13,  5.11s/it]
 82%|████████▏ | 185356/225000 [349:55:25<56:58:42,  5.17s/it]
 82%|████████▏ | 185357/225000 [349:55:31<58:54:48,  5.35s/it]
 82%|████████▏ | 185358/225000 [349:55:36<56:54:27,  5.17s/it]
 82%|████████▏ | 185359/225000 [349:55:41<57:07:57,  5.19s/it]
 82%|████████▏ | 185360/225000 [349:55:46<57:07:34,  5.19s/it]
 82%|████████▏ | 185361/225000 [349:55:51<55:27:47,  5.04s/it]
 82%|████████▏ | 185362/225000 [349:55:56<55:33:33,  5.05s/it]
 82%|████████▏ | 185363/225000 [349:56:01<57:12:45,  5.20s/it]
 82%|████████▏ | 185364/225000 [349:56:07<57:15:58,  5.20s/it]
 82%|████████▏ | 185365/225000 [349:56:12<57:30:20,  5.22s/it]
 82%|████████▏ | 185366/225000 [349:56:17<56:57:14,  5.17s/it]
 82%|████████▏ | 185367/225000 [349:56:22<57:07:53,  5.19s/it]
 82%|████████▏ | 185368/225000 [349:56:28<59:40:30,  5.42s/it]
 82%|████████▏ | 185369/225000 [349:56:36<66:27:13,  6.04s/it]
 82%|████████▏ | 185370/225000 [349:56:40<62:15:20,  5.66s/it]
 82%|████████▏ | 185371/225000 [349:56:45<60:06:29,  5.46s/it]
 82%|████████▏ | 185372/225000 [349:56:51<59:49:35,  5.43s/it]
 82%|████████▏ | 185373/225000 [349:56:56<59:19:56,  5.39s/it]
 82%|████████▏ | 185374/225000 [349:57:01<59:22:31,  5.39s/it]
 82%|████████▏ | 185375/225000 [349:57:07<59:31:43,  5.41s/it]
 82%|████████▏ | 185376/225000 [349:57:12<56:54:19,  5.17s/it]
 82%|████████▏ | 185377/225000 [349:57:24<82:21:11,  7.48s/it]
 82%|████████▏ | 185378/225000 [349:57:28<71:10:28,  6.47s/it]
 82%|████████▏ | 185379/225000 [349:57:32<62:57:23,  5.72s/it]
 82%|████████▏ | 185380/225000 [349:57:40<68:26:51,  6.22s/it]
 82%|████████▏ | 185381/225000 [349:57:44<60:29:52,  5.50s/it]
 82%|████████▏ | 185382/225000 [349:57:52<69:12:37,  6.29s/it]
 82%|████████▏ | 185383/225000 [349:57:56<61:46:48,  5.61s/it]
 82%|████████▏ | 185384/225000 [349:58:00<57:50:12,  5.26s/it]
 82%|████████▏ | 185385/225000 [349:58:05<55:26:21,  5.04s/it]
 82%|████████▏ | 185386/225000 [349:58:09<53:22:37,  4.85s/it]
 82%|████████▏ | 185387/225000 [349:58:14<52:38:14,  4.78s/it]
 82%|████████▏ | 185388/225000 [349:58:21<60:41:12,  5.52s/it]
 82%|████████▏ | 185389/225000 [349:58:30<71:03:36,  6.46s/it]
 82%|████████▏ | 185390/225000 [349:58:34<63:38:46,  5.78s/it]
 82%|████████▏ | 185391/225000 [349:58:38<59:27:01,  5.40s/it]
 82%|████████▏ | 185392/225000 [349:58:43<55:50:17,  5.08s/it]
 82%|████████▏ | 185393/225000 [349:58:47<54:05:07,  4.92s/it]
 82%|████████▏ | 185394/225000 [349:58:52<52:50:30,  4.80s/it]
 82%|████████▏ | 185395/225000 [349:58:56<50:55:13,  4.63s/it]
 82%|████████▏ | 185396/225000 [349:59:05<63:34:16,  5.78s/it]
 82%|████████▏ | 185397/225000 [349:59:14<75:17:07,  6.84s/it]
 82%|████████▏ | 185398/225000 [349:59:18<66:41:28,  6.06s/it]
 82%|████████▏ | 185399/225000 [349:59:22<61:00:21,  5.55s/it]
 82%|████████▏ | 185400/225000 [349:59:27<56:35:25,  5.14s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.035', 'learning_rate': '4.087e-06', 'epoch': '0.824', 'num_input_tokens_seen': 580598488, 'train_runtime': '1.26e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185400/225000 [349:59:27<56:35:25,  5.14s/it]
 82%|████████▏ | 185401/225000 [349:59:31<53:20:40,  4.85s/it]
 82%|████████▏ | 185402/225000 [349:59:36<55:26:03,  5.04s/it]
 82%|████████▏ | 185403/225000 [349:59:48<77:18:22,  7.03s/it]
 82%|████████▏ | 185404/225000 [349:59:52<66:09:35,  6.02s/it]
 82%|████████▏ | 185405/225000 [349:59:57<63:22:33,  5.76s/it]
 82%|████████▏ | 185406/225000 [350:00:04<67:10:25,  6.11s/it]
 82%|████████▏ | 185407/225000 [350:00:08<61:34:23,  5.60s/it]
 82%|████████▏ | 185408/225000 [350:00:12<57:35:44,  5.24s/it]
 82%|████████▏ | 185409/225000 [350:00:17<54:46:28,  4.98s/it]
 82%|████████▏ | 185410/225000 [350:00:21<52:37:37,  4.79s/it]
 82%|████████▏ | 185411/225000 [350:00:26<53:37:07,  4.88s/it]
 82%|████████▏ | 185412/225000 [350:00:37<73:04:32,  6.65s/it]
 82%|████████▏ | 185413/225000 [350:00:41<64:50:05,  5.90s/it]
 82%|████████▏ | 185414/225000 [350:00:46<60:29:42,  5.50s/it]
 82%|████████▏ | 185415/225000 [350:00:50<57:23:56,  5.22s/it]
 82%|████████▏ | 185416/225000 [350:00:55<54:36:56,  4.97s/it]
 82%|████████▏ | 185417/225000 [350:00:59<52:05:28,  4.74s/it]
 82%|████████▏ | 185418/225000 [350:01:11<74:48:08,  6.80s/it]
 82%|████████▏ | 185419/225000 [350:01:16<69:27:55,  6.32s/it]
 82%|████████▏ | 185420/225000 [350:01:20<61:33:08,  5.60s/it]
 82%|████████▏ | 185421/225000 [350:01:23<55:42:31,  5.07s/it]
 82%|████████▏ | 185422/225000 [350:01:34<72:44:10,  6.62s/it]
 82%|████████▏ | 185423/225000 [350:01:39<67:07:19,  6.11s/it]
 82%|████████▏ | 185424/225000 [350:01:43<62:25:58,  5.68s/it]
 82%|████████▏ | 185425/225000 [350:01:48<57:51:16,  5.26s/it]
 82%|████████▏ | 185426/225000 [350:01:52<55:14:19,  5.02s/it]
 82%|████████▏ | 185427/225000 [350:01:57<54:47:24,  4.98s/it]
 82%|████████▏ | 185428/225000 [350:02:01<52:55:47,  4.82s/it]
 82%|████████▏ | 185429/225000 [350:02:12<70:37:10,  6.42s/it]
 82%|████████▏ | 185430/225000 [350:02:17<67:22:07,  6.13s/it]
 82%|████████▏ | 185431/225000 [350:02:24<69:31:26,  6.33s/it]
 82%|████████▏ | 185432/225000 [350:02:27<59:47:12,  5.44s/it]
 82%|████████▏ | 185433/225000 [350:02:36<70:58:59,  6.46s/it]
 82%|████████▏ | 185434/225000 [350:02:39<60:46:05,  5.53s/it]
 82%|████████▏ | 185435/225000 [350:02:43<54:24:55,  4.95s/it]
 82%|████████▏ | 185436/225000 [350:02:48<56:15:26,  5.12s/it]
 82%|████████▏ | 185437/225000 [350:02:54<56:21:40,  5.13s/it]
 82%|████████▏ | 185438/225000 [350:02:59<57:52:59,  5.27s/it]
 82%|████████▏ | 185439/225000 [350:03:04<57:16:57,  5.21s/it]
 82%|████████▏ | 185440/225000 [350:03:10<57:32:06,  5.24s/it]
 82%|████████▏ | 185441/225000 [350:03:15<57:05:21,  5.20s/it]
 82%|████████▏ | 185442/225000 [350:03:20<57:06:01,  5.20s/it]
 82%|████████▏ | 185443/225000 [350:03:25<57:38:33,  5.25s/it]
 82%|████████▏ | 185444/225000 [350:03:31<59:56:17,  5.45s/it]
 82%|████████▏ | 185445/225000 [350:03:41<75:18:08,  6.85s/it]
 82%|████████▏ | 185446/225000 [350:03:45<64:13:48,  5.85s/it]
 82%|████████▏ | 185447/225000 [350:03:49<57:06:48,  5.20s/it]
 82%|████████▏ | 185448/225000 [350:04:00<78:00:15,  7.10s/it]
 82%|████████▏ | 185449/225000 [350:04:05<70:41:08,  6.43s/it]
 82%|████████▏ | 185450/225000 [350:04:10<66:11:50,  6.03s/it]
                                                              
{'loss': '0.7683', 'grad_norm': '1.167', 'learning_rate': '4.077e-06', 'epoch': '0.8242', 'num_input_tokens_seen': 580728760, 'train_runtime': '1.26e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185450/225000 [350:04:10<66:11:50,  6.03s/it]
 82%|████████▏ | 185451/225000 [350:04:15<62:09:39,  5.66s/it]
 82%|████████▏ | 185452/225000 [350:04:20<59:16:51,  5.40s/it]
 82%|████████▏ | 185453/225000 [350:04:25<58:12:42,  5.30s/it]
 82%|████████▏ | 185454/225000 [350:04:30<57:54:59,  5.27s/it]
 82%|████████▏ | 185455/225000 [350:04:35<56:46:33,  5.17s/it]
 82%|████████▏ | 185456/225000 [350:04:40<55:33:20,  5.06s/it]
 82%|████████▏ | 185457/225000 [350:04:44<54:40:16,  4.98s/it]
 82%|████████▏ | 185458/225000 [350:04:49<54:44:51,  4.98s/it]
 82%|████████▏ | 185459/225000 [350:04:54<54:17:19,  4.94s/it]
 82%|████████▏ | 185460/225000 [350:05:03<67:40:37,  6.16s/it]
 82%|████████▏ | 185461/225000 [350:05:09<65:53:09,  6.00s/it]
 82%|████████▏ | 185462/225000 [350:05:13<59:05:36,  5.38s/it]
 82%|████████▏ | 185463/225000 [350:05:23<75:06:53,  6.84s/it]
 82%|████████▏ | 185464/225000 [350:05:27<66:29:22,  6.05s/it]
 82%|████████▏ | 185465/225000 [350:05:32<61:33:42,  5.61s/it]
 82%|████████▏ | 185466/225000 [350:05:36<58:32:47,  5.33s/it]
 82%|████████▏ | 185467/225000 [350:05:41<56:50:30,  5.18s/it]
 82%|████████▏ | 185468/225000 [350:05:46<55:10:40,  5.02s/it]
 82%|████████▏ | 185469/225000 [350:05:51<54:02:47,  4.92s/it]
 82%|████████▏ | 185470/225000 [350:05:55<53:27:45,  4.87s/it]
 82%|████████▏ | 185471/225000 [350:06:07<76:34:18,  6.97s/it]
 82%|████████▏ | 185472/225000 [350:06:13<70:45:17,  6.44s/it]
 82%|████████▏ | 185473/225000 [350:06:16<61:20:56,  5.59s/it]
 82%|████████▏ | 185474/225000 [350:06:23<65:18:21,  5.95s/it]
 82%|████████▏ | 185475/225000 [350:06:26<56:24:03,  5.14s/it]
 82%|████████▏ | 185476/225000 [350:06:30<52:51:00,  4.81s/it]
 82%|████████▏ | 185477/225000 [350:06:34<50:14:27,  4.58s/it]
 82%|████████▏ | 185478/225000 [350:06:43<64:32:49,  5.88s/it]
 82%|████████▏ | 185479/225000 [350:06:49<65:29:23,  5.97s/it]
 82%|████████▏ | 185480/225000 [350:06:53<57:15:59,  5.22s/it]
 82%|████████▏ | 185481/225000 [350:07:00<64:27:27,  5.87s/it]
 82%|████████▏ | 185482/225000 [350:07:04<56:35:14,  5.15s/it]
 82%|████████▏ | 185483/225000 [350:07:13<70:59:04,  6.47s/it]
 82%|████████▏ | 185484/225000 [350:07:17<62:19:09,  5.68s/it]
 82%|████████▏ | 185485/225000 [350:07:21<56:02:45,  5.11s/it]
 82%|████████▏ | 185486/225000 [350:07:25<52:46:19,  4.81s/it]
 82%|████████▏ | 185487/225000 [350:07:35<70:02:51,  6.38s/it]
 82%|████████▏ | 185488/225000 [350:07:38<60:34:10,  5.52s/it]
 82%|████████▏ | 185489/225000 [350:07:46<67:11:38,  6.12s/it]
 82%|████████▏ | 185490/225000 [350:07:50<59:24:55,  5.41s/it]
 82%|████████▏ | 185491/225000 [350:07:54<54:19:37,  4.95s/it]
 82%|████████▏ | 185492/225000 [350:07:58<51:57:47,  4.73s/it]
 82%|████████▏ | 185493/225000 [350:08:08<68:37:16,  6.25s/it]
 82%|████████▏ | 185494/225000 [350:08:12<61:25:41,  5.60s/it]
 82%|████████▏ | 185495/225000 [350:08:16<57:23:34,  5.23s/it]
 82%|████████▏ | 185496/225000 [350:08:20<53:39:43,  4.89s/it]
 82%|████████▏ | 185497/225000 [350:08:24<51:02:39,  4.65s/it]
 82%|████████▏ | 185498/225000 [350:08:34<69:03:38,  6.29s/it]
 82%|████████▏ | 185499/225000 [350:08:39<62:16:01,  5.67s/it]
 82%|████████▏ | 185500/225000 [350:08:43<56:20:24,  5.13s/it]
                                                              
{'loss': '0.7499', 'grad_norm': '1.198', 'learning_rate': '4.067e-06', 'epoch': '0.8244', 'num_input_tokens_seen': 580858392, 'train_runtime': '1.261e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185500/225000 [350:08:43<56:20:24,  5.13s/it]
 82%|████████▏ | 185501/225000 [350:08:50<65:36:43,  5.98s/it]
 82%|████████▏ | 185502/225000 [350:08:54<58:50:23,  5.36s/it]
 82%|████████▏ | 185503/225000 [350:09:01<64:15:05,  5.86s/it]
 82%|████████▏ | 185504/225000 [350:09:05<58:15:06,  5.31s/it]
 82%|████████▏ | 185505/225000 [350:09:10<54:24:54,  4.96s/it]
 82%|████████▏ | 185506/225000 [350:09:18<66:56:19,  6.10s/it]
 82%|████████▏ | 185507/225000 [350:09:22<59:15:02,  5.40s/it]
 82%|████████▏ | 185508/225000 [350:09:27<57:53:59,  5.28s/it]
 82%|████████▏ | 185509/225000 [350:09:32<56:32:13,  5.15s/it]
 82%|████████▏ | 185510/225000 [350:09:37<55:31:50,  5.06s/it]
 82%|████████▏ | 185511/225000 [350:09:42<55:12:51,  5.03s/it]
 82%|████████▏ | 185512/225000 [350:09:47<55:43:08,  5.08s/it]
 82%|████████▏ | 185513/225000 [350:09:52<55:17:24,  5.04s/it]
 82%|████████▏ | 185514/225000 [350:09:57<55:12:11,  5.03s/it]
 82%|████████▏ | 185515/225000 [350:10:02<55:17:34,  5.04s/it]
 82%|████████▏ | 185516/225000 [350:10:07<56:13:59,  5.13s/it]
 82%|████████▏ | 185517/225000 [350:10:12<54:35:15,  4.98s/it]
 82%|████████▏ | 185518/225000 [350:10:17<54:31:58,  4.97s/it]
 82%|████████▏ | 185519/225000 [350:10:25<64:36:59,  5.89s/it]
 82%|████████▏ | 185520/225000 [350:10:35<76:57:49,  7.02s/it]
 82%|████████▏ | 185521/225000 [350:10:39<67:50:21,  6.19s/it]
 82%|████████▏ | 185522/225000 [350:10:43<61:12:32,  5.58s/it]
 82%|████████▏ | 185523/225000 [350:10:52<71:30:42,  6.52s/it]
 82%|████████▏ | 185524/225000 [350:10:57<67:40:38,  6.17s/it]
 82%|████████▏ | 185525/225000 [350:11:03<67:34:16,  6.16s/it]
 82%|████████▏ | 185526/225000 [350:11:09<66:55:02,  6.10s/it]
 82%|████████▏ | 185527/225000 [350:11:15<65:48:36,  6.00s/it]
 82%|████████▏ | 185528/225000 [350:11:19<60:20:10,  5.50s/it]
 82%|████████▏ | 185529/225000 [350:11:23<54:02:39,  4.93s/it]
 82%|████████▏ | 185530/225000 [350:11:32<68:16:35,  6.23s/it]
 82%|████████▏ | 185531/225000 [350:11:38<65:57:30,  6.02s/it]
 82%|████████▏ | 185532/225000 [350:11:42<60:32:21,  5.52s/it]
 82%|████████▏ | 185533/225000 [350:11:45<53:40:49,  4.90s/it]
 82%|████████▏ | 185534/225000 [350:11:50<50:56:17,  4.65s/it]
 82%|████████▏ | 185535/225000 [350:11:56<56:17:00,  5.13s/it]
 82%|████████▏ | 185536/225000 [350:12:04<65:15:26,  5.95s/it]
 82%|████████▏ | 185537/225000 [350:12:08<60:11:19,  5.49s/it]
 82%|████████▏ | 185538/225000 [350:12:12<56:03:00,  5.11s/it]
 82%|████████▏ | 185539/225000 [350:12:16<52:10:02,  4.76s/it]
 82%|████████▏ | 185540/225000 [350:12:20<48:57:33,  4.47s/it]
 82%|████████▏ | 185541/225000 [350:12:33<76:21:49,  6.97s/it]
 82%|████████▏ | 185542/225000 [350:12:36<64:30:06,  5.88s/it]
 82%|████████▏ | 185543/225000 [350:12:40<57:44:22,  5.27s/it]
 82%|████████▏ | 185544/225000 [350:12:49<68:38:40,  6.26s/it]
 82%|████████▏ | 185545/225000 [350:12:52<59:59:51,  5.47s/it]
 82%|████████▏ | 185546/225000 [350:12:56<55:19:13,  5.05s/it]
 82%|████████▏ | 185547/225000 [350:13:02<58:01:07,  5.29s/it]
 82%|████████▏ | 185548/225000 [350:13:11<71:03:18,  6.48s/it]
 82%|████████▏ | 185549/225000 [350:13:15<61:54:39,  5.65s/it]
 82%|████████▏ | 185550/225000 [350:13:19<56:40:11,  5.17s/it]
                                                              
{'loss': '0.7666', 'grad_norm': '0.9316', 'learning_rate': '4.057e-06', 'epoch': '0.8247', 'num_input_tokens_seen': 580988808, 'train_runtime': '1.261e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185550/225000 [350:13:19<56:40:11,  5.17s/it]
 82%|████████▏ | 185551/225000 [350:13:29<72:00:43,  6.57s/it]
 82%|████████▏ | 185552/225000 [350:13:33<62:16:58,  5.68s/it]
 82%|████████▏ | 185553/225000 [350:13:36<54:39:31,  4.99s/it]
 82%|████████▏ | 185554/225000 [350:13:45<68:39:02,  6.27s/it]
 82%|████████▏ | 185555/225000 [350:13:49<60:09:32,  5.49s/it]
 82%|████████▏ | 185556/225000 [350:13:53<54:18:15,  4.96s/it]
 82%|████████▏ | 185557/225000 [350:13:58<54:40:50,  4.99s/it]
 82%|████████▏ | 185558/225000 [350:14:03<56:45:46,  5.18s/it]
 82%|████████▏ | 185559/225000 [350:14:08<55:23:46,  5.06s/it]
 82%|████████▏ | 185560/225000 [350:14:13<55:12:59,  5.04s/it]
 82%|████████▏ | 185561/225000 [350:14:18<55:43:20,  5.09s/it]
 82%|████████▏ | 185562/225000 [350:14:24<56:38:44,  5.17s/it]
 82%|████████▏ | 185563/225000 [350:14:34<73:16:11,  6.69s/it]
 82%|████████▏ | 185564/225000 [350:14:38<66:15:15,  6.05s/it]
 82%|████████▏ | 185565/225000 [350:14:43<62:19:09,  5.69s/it]
 82%|████████▏ | 185566/225000 [350:14:48<59:27:11,  5.43s/it]
 82%|████████▏ | 185567/225000 [350:14:53<56:15:19,  5.14s/it]
 82%|████████▏ | 185568/225000 [350:14:57<53:55:50,  4.92s/it]
 82%|████████▏ | 185569/225000 [350:15:01<52:34:48,  4.80s/it]
 82%|████████▏ | 185570/225000 [350:15:06<52:29:22,  4.79s/it]
 82%|████████▏ | 185571/225000 [350:15:15<66:27:32,  6.07s/it]
 82%|████████▏ | 185572/225000 [350:15:23<72:37:00,  6.63s/it]
 82%|████████▏ | 185573/225000 [350:15:27<62:00:00,  5.66s/it]
 82%|████████▏ | 185574/225000 [350:15:31<57:52:50,  5.29s/it]
 82%|████████▏ | 185575/225000 [350:15:40<70:34:06,  6.44s/it]
 82%|████████▏ | 185576/225000 [350:15:44<61:13:56,  5.59s/it]
 82%|████████▏ | 185577/225000 [350:15:51<67:20:15,  6.15s/it]
 82%|████████▏ | 185578/225000 [350:15:55<58:55:21,  5.38s/it]
 82%|████████▏ | 185579/225000 [350:15:58<51:51:07,  4.74s/it]
 82%|████████▏ | 185580/225000 [350:16:07<64:35:17,  5.90s/it]
 82%|████████▏ | 185581/225000 [350:16:11<57:58:11,  5.29s/it]
 82%|████████▏ | 185582/225000 [350:16:16<58:38:33,  5.36s/it]
 82%|████████▏ | 185583/225000 [350:16:23<62:24:32,  5.70s/it]
 82%|████████▏ | 185584/225000 [350:16:26<54:27:12,  4.97s/it]
 82%|████████▏ | 185585/225000 [350:16:36<71:47:55,  6.56s/it]
 82%|████████▏ | 185586/225000 [350:16:39<61:18:26,  5.60s/it]
 82%|████████▏ | 185587/225000 [350:16:46<65:15:06,  5.96s/it]
 82%|████████▏ | 185588/225000 [350:16:51<59:38:21,  5.45s/it]
 82%|████████▏ | 185589/225000 [350:16:55<55:22:48,  5.06s/it]
 82%|████████▏ | 185590/225000 [350:16:59<52:22:19,  4.78s/it]
 82%|████████▏ | 185591/225000 [350:17:03<50:17:14,  4.59s/it]
 82%|████████▏ | 185592/225000 [350:17:13<69:15:41,  6.33s/it]
 82%|████████▏ | 185593/225000 [350:17:17<59:03:02,  5.39s/it]
 82%|████████▏ | 185594/225000 [350:17:24<66:57:42,  6.12s/it]
 82%|████████▏ | 185595/225000 [350:17:28<59:12:25,  5.41s/it]
 82%|████████▏ | 185596/225000 [350:17:36<67:32:45,  6.17s/it]
 82%|████████▏ | 185597/225000 [350:17:40<60:53:44,  5.56s/it]
 82%|████████▏ | 185598/225000 [350:17:47<65:44:35,  6.01s/it]
 82%|████████▏ | 185599/225000 [350:17:51<57:59:16,  5.30s/it]
 82%|████████▏ | 185600/225000 [350:17:55<54:26:25,  4.97s/it]
                                                              
{'loss': '0.7615', 'grad_norm': '1.108', 'learning_rate': '4.047e-06', 'epoch': '0.8249', 'num_input_tokens_seen': 581119032, 'train_runtime': '1.261e+06', 'train_tokens_per_second': '460.8'}

 82%|████████▏ | 185600/225000 [350:17:55<54:26:25,  4.97s/it]
 82%|████████▏ | 185601/225000 [350:17:59<50:04:13,  4.58s/it]
 82%|████████▏ | 185602/225000 [350:18:04<52:39:21,  4.81s/it]
 82%|████████▏ | 185603/225000 [350:18:13<66:32:03,  6.08s/it]
 82%|████████▏ | 185604/225000 [350:18:16<56:54:59,  5.20s/it]
 82%|████████▏ | 185605/225000 [350:18:24<63:39:32,  5.82s/it]
 82%|████████▏ | 185606/225000 [350:18:28<60:06:56,  5.49s/it]
 82%|████████▏ | 185607/225000 [350:18:33<57:13:40,  5.23s/it]
 82%|████████▏ | 185608/225000 [350:18:38<55:22:21,  5.06s/it]
 82%|████████▏ | 185609/225000 [350:18:43<55:03:25,  5.03s/it]
 82%|████████▏ | 185610/225000 [350:18:47<54:04:59,  4.94s/it]
 82%|████████▏ | 185611/225000 [350:18:52<53:33:55,  4.90s/it]
 82%|████████▏ | 185612/225000 [350:18:57<55:15:13,  5.05s/it]
 82%|████████▏ | 185613/225000 [350:19:03<55:14:03,  5.05s/it]
 82%|████████▏ | 185614/225000 [350:19:07<53:59:44,  4.94s/it]
 82%|████████▏ | 185615/225000 [350:19:18<71:57:42,  6.58s/it]
 82%|████████▏ | 185616/225000 [350:19:23<69:13:32,  6.33s/it]
 82%|████████▏ | 185617/225000 [350:19:32<76:22:45,  6.98s/it]
 82%|████████▏ | 185618/225000 [350:19:35<64:21:52,  5.88s/it]
 82%|████████▏ | 185619/225000 [350:19:42<68:27:36,  6.26s/it]
 82%|████████▏ | 185620/225000 [350:19:46<59:01:54,  5.40s/it]
 82%|████████▏ | 185621/225000 [350:19:50<54:30:14,  4.98s/it]
 82%|████████▏ | 185622/225000 [350:19:54<51:21:51,  4.70s/it]
 82%|████████▏ | 185623/225000 [350:19:58<50:59:27,  4.66s/it]
 82%|████████▏ | 185624/225000 [350:20:07<65:39:27,  6.00s/it]
 82%|████████▎ | 185625/225000 [350:20:16<73:00:37,  6.68s/it]
 83%|████████▎ | 185626/225000 [350:20:19<62:25:38,  5.71s/it]
 83%|████████▎ | 185627/225000 [350:20:23<54:47:43,  5.01s/it]
 83%|████████▎ | 185628/225000 [350:20:31<65:51:05,  6.02s/it]
 83%|████████▎ | 185629/225000 [350:20:35<60:41:47,  5.55s/it]
 83%|████████▎ | 185630/225000 [350:20:39<55:48:07,  5.10s/it]
 83%|████████▎ | 185631/225000 [350:20:44<54:08:19,  4.95s/it]
 83%|████████▎ | 185632/225000 [350:20:49<53:22:30,  4.88s/it]
 83%|████████▎ | 185633/225000 [350:20:53<52:11:24,  4.77s/it]
 83%|████████▎ | 185634/225000 [350:20:58<52:25:56,  4.79s/it]
 83%|████████▎ | 185635/225000 [350:21:09<73:55:35,  6.76s/it]
 83%|████████▎ | 185636/225000 [350:21:14<66:09:12,  6.05s/it]
 83%|████████▎ | 185637/225000 [350:21:22<72:15:07,  6.61s/it]
 83%|████████▎ | 185638/225000 [350:21:26<64:08:09,  5.87s/it]
 83%|████████▎ | 185639/225000 [350:21:30<59:04:39,  5.40s/it]
 83%|████████▎ | 185640/225000 [350:21:35<55:43:22,  5.10s/it]
 83%|████████▎ | 185641/225000 [350:21:39<53:21:42,  4.88s/it]
 83%|████████▎ | 185642/225000 [350:21:46<60:03:27,  5.49s/it]
 83%|████████▎ | 185643/225000 [350:21:56<76:05:46,  6.96s/it]
 83%|████████▎ | 185644/225000 [350:21:59<63:48:23,  5.84s/it]
 83%|████████▎ | 185645/225000 [350:22:06<66:19:15,  6.07s/it]
 83%|████████▎ | 185646/225000 [350:22:10<60:02:59,  5.49s/it]
 83%|████████▎ | 185647/225000 [350:22:16<61:45:35,  5.65s/it]
 83%|████████▎ | 185648/225000 [350:22:20<54:30:43,  4.99s/it]
 83%|████████▎ | 185649/225000 [350:22:24<52:03:35,  4.76s/it]
 83%|████████▎ | 185650/225000 [350:22:28<50:34:57,  4.63s/it]
                                                              
{'loss': '0.7571', 'grad_norm': '1.076', 'learning_rate': '4.037e-06', 'epoch': '0.8251', 'num_input_tokens_seen': 581248024, 'train_runtime': '1.261e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185650/225000 [350:22:28<50:34:57,  4.63s/it]
 83%|████████▎ | 185651/225000 [350:22:33<49:48:27,  4.56s/it]
 83%|████████▎ | 185652/225000 [350:22:45<74:07:11,  6.78s/it]
 83%|████████▎ | 185653/225000 [350:22:49<67:12:40,  6.15s/it]
 83%|████████▎ | 185654/225000 [350:22:53<58:33:08,  5.36s/it]
 83%|████████▎ | 185655/225000 [350:22:58<56:29:13,  5.17s/it]
 83%|████████▎ | 185656/225000 [350:23:03<57:59:46,  5.31s/it]
 83%|████████▎ | 185657/225000 [350:23:08<56:32:10,  5.17s/it]
 83%|████████▎ | 185658/225000 [350:23:14<57:32:32,  5.27s/it]
 83%|████████▎ | 185659/225000 [350:23:19<58:12:19,  5.33s/it]
 83%|████████▎ | 185660/225000 [350:23:24<58:31:50,  5.36s/it]
 83%|████████▎ | 185661/225000 [350:23:30<58:36:44,  5.36s/it]
 83%|████████▎ | 185662/225000 [350:23:35<57:42:44,  5.28s/it]
 83%|████████▎ | 185663/225000 [350:23:40<57:09:09,  5.23s/it]
 83%|████████▎ | 185664/225000 [350:23:45<57:09:18,  5.23s/it]
 83%|████████▎ | 185665/225000 [350:23:50<55:39:39,  5.09s/it]
 83%|████████▎ | 185666/225000 [350:24:00<71:54:22,  6.58s/it]
 83%|████████▎ | 185667/225000 [350:24:03<60:45:52,  5.56s/it]
 83%|████████▎ | 185668/225000 [350:24:07<54:56:53,  5.03s/it]
 83%|████████▎ | 185669/225000 [350:24:16<66:59:48,  6.13s/it]
 83%|████████▎ | 185670/225000 [350:24:21<63:34:51,  5.82s/it]
 83%|████████▎ | 185671/225000 [350:24:28<66:41:29,  6.10s/it]
 83%|████████▎ | 185672/225000 [350:24:32<62:14:11,  5.70s/it]
 83%|████████▎ | 185673/225000 [350:24:37<59:17:32,  5.43s/it]
 83%|████████▎ | 185674/225000 [350:24:41<55:48:21,  5.11s/it]
 83%|████████▎ | 185675/225000 [350:24:46<53:29:08,  4.90s/it]
 83%|████████▎ | 185676/225000 [350:24:51<52:41:45,  4.82s/it]
 83%|████████▎ | 185677/225000 [350:24:55<52:06:52,  4.77s/it]
 83%|████████▎ | 185678/225000 [350:25:00<52:28:00,  4.80s/it]
 83%|████████▎ | 185679/225000 [350:25:05<52:22:10,  4.79s/it]
 83%|████████▎ | 185680/225000 [350:25:10<53:22:05,  4.89s/it]
 83%|████████▎ | 185681/225000 [350:25:21<75:02:19,  6.87s/it]
 83%|████████▎ | 185682/225000 [350:25:29<77:58:25,  7.14s/it]
 83%|████████▎ | 185683/225000 [350:25:33<65:27:09,  5.99s/it]
 83%|████████▎ | 185684/225000 [350:25:42<76:57:10,  7.05s/it]
 83%|████████▎ | 185685/225000 [350:25:46<67:46:36,  6.21s/it]
 83%|████████▎ | 185686/225000 [350:25:50<59:51:58,  5.48s/it]
 83%|████████▎ | 185687/225000 [350:25:57<63:05:01,  5.78s/it]
 83%|████████▎ | 185688/225000 [350:26:03<66:04:46,  6.05s/it]
 83%|████████▎ | 185689/225000 [350:26:08<62:08:29,  5.69s/it]
 83%|████████▎ | 185690/225000 [350:26:13<60:03:11,  5.50s/it]
 83%|████████▎ | 185691/225000 [350:26:18<58:20:56,  5.34s/it]
 83%|████████▎ | 185692/225000 [350:26:23<55:53:32,  5.12s/it]
 83%|████████▎ | 185693/225000 [350:26:27<53:03:12,  4.86s/it]
 83%|████████▎ | 185694/225000 [350:26:32<52:59:37,  4.85s/it]
 83%|████████▎ | 185695/225000 [350:26:36<51:08:36,  4.68s/it]
 83%|████████▎ | 185696/225000 [350:26:40<50:09:02,  4.59s/it]
 83%|████████▎ | 185697/225000 [350:26:45<50:39:46,  4.64s/it]
 83%|████████▎ | 185698/225000 [350:26:50<51:29:26,  4.72s/it]
 83%|████████▎ | 185699/225000 [350:26:55<51:05:24,  4.68s/it]
 83%|████████▎ | 185700/225000 [350:27:05<69:03:30,  6.33s/it]
                                                              
{'loss': '0.7576', 'grad_norm': '1.158', 'learning_rate': '4.027e-06', 'epoch': '0.8253', 'num_input_tokens_seen': 581378240, 'train_runtime': '1.262e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185700/225000 [350:27:05<69:03:30,  6.33s/it]
 83%|████████▎ | 185701/225000 [350:27:14<76:49:14,  7.04s/it]
 83%|████████▎ | 185702/225000 [350:27:18<67:01:32,  6.14s/it]
 83%|████████▎ | 185703/225000 [350:27:21<57:54:16,  5.30s/it]
 83%|████████▎ | 185704/225000 [350:27:30<68:51:29,  6.31s/it]
 83%|████████▎ | 185705/225000 [350:27:35<64:36:43,  5.92s/it]
 83%|████████▎ | 185706/225000 [350:27:42<68:11:07,  6.25s/it]
 83%|████████▎ | 185707/225000 [350:27:47<64:49:13,  5.94s/it]
 83%|████████▎ | 185708/225000 [350:27:52<62:27:18,  5.72s/it]
 83%|████████▎ | 185709/225000 [350:27:57<58:58:39,  5.40s/it]
 83%|████████▎ | 185710/225000 [350:28:02<57:39:16,  5.28s/it]
 83%|████████▎ | 185711/225000 [350:28:07<57:47:42,  5.30s/it]
 83%|████████▎ | 185712/225000 [350:28:12<57:31:38,  5.27s/it]
 83%|████████▎ | 185713/225000 [350:28:17<56:03:36,  5.14s/it]
 83%|████████▎ | 185714/225000 [350:28:22<55:48:05,  5.11s/it]
 83%|████████▎ | 185715/225000 [350:28:27<54:56:27,  5.03s/it]
 83%|████████▎ | 185716/225000 [350:28:32<54:23:21,  4.98s/it]
 83%|████████▎ | 185717/225000 [350:28:37<55:15:39,  5.06s/it]
 83%|████████▎ | 185718/225000 [350:28:48<75:04:00,  6.88s/it]
 83%|████████▎ | 185719/225000 [350:28:54<70:07:23,  6.43s/it]
 83%|████████▎ | 185720/225000 [350:28:59<65:42:23,  6.02s/it]
 83%|████████▎ | 185721/225000 [350:29:06<69:08:28,  6.34s/it]
 83%|████████▎ | 185722/225000 [350:29:09<60:10:18,  5.52s/it]
 83%|████████▎ | 185723/225000 [350:29:13<53:36:12,  4.91s/it]
 83%|████████▎ | 185724/225000 [350:29:24<73:41:46,  6.75s/it]
 83%|████████▎ | 185725/225000 [350:29:27<62:31:17,  5.73s/it]
 83%|████████▎ | 185726/225000 [350:29:31<55:14:16,  5.06s/it]
 83%|████████▎ | 185727/225000 [350:29:40<69:49:54,  6.40s/it]
 83%|████████▎ | 185728/225000 [350:29:44<60:55:01,  5.58s/it]
 83%|████████▎ | 185729/225000 [350:29:48<54:59:31,  5.04s/it]
 83%|████████▎ | 185730/225000 [350:29:58<71:23:24,  6.54s/it]
 83%|████████▎ | 185731/225000 [350:30:01<61:49:59,  5.67s/it]
 83%|████████▎ | 185732/225000 [350:30:05<55:02:57,  5.05s/it]
 83%|████████▎ | 185733/225000 [350:30:13<65:24:00,  6.00s/it]
 83%|████████▎ | 185734/225000 [350:30:17<57:40:45,  5.29s/it]
 83%|████████▎ | 185735/225000 [350:30:25<65:29:12,  6.00s/it]
 83%|████████▎ | 185736/225000 [350:30:28<58:17:31,  5.34s/it]
 83%|████████▎ | 185737/225000 [350:30:32<54:17:15,  4.98s/it]
 83%|████████▎ | 185738/225000 [350:30:38<55:30:00,  5.09s/it]
 83%|████████▎ | 185739/225000 [350:30:43<55:43:52,  5.11s/it]
 83%|████████▎ | 185740/225000 [350:30:48<55:52:09,  5.12s/it]
 83%|████████▎ | 185741/225000 [350:30:53<55:48:38,  5.12s/it]
 83%|████████▎ | 185742/225000 [350:30:58<55:34:38,  5.10s/it]
 83%|████████▎ | 185743/225000 [350:31:04<56:30:05,  5.18s/it]
 83%|████████▎ | 185744/225000 [350:31:09<57:05:42,  5.24s/it]
 83%|████████▎ | 185745/225000 [350:31:15<58:42:06,  5.38s/it]
 83%|████████▎ | 185746/225000 [350:31:22<65:10:46,  5.98s/it]
 83%|████████▎ | 185747/225000 [350:31:32<78:59:13,  7.24s/it]
 83%|████████▎ | 185748/225000 [350:31:38<73:11:16,  6.71s/it]
 83%|████████▎ | 185749/225000 [350:31:43<68:10:23,  6.25s/it]
 83%|████████▎ | 185750/225000 [350:31:48<64:06:28,  5.88s/it]
                                                              
{'loss': '0.7457', 'grad_norm': '1.192', 'learning_rate': '4.017e-06', 'epoch': '0.8256', 'num_input_tokens_seen': 581508952, 'train_runtime': '1.262e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185750/225000 [350:31:48<64:06:28,  5.88s/it]
 83%|████████▎ | 185751/225000 [350:31:53<61:19:31,  5.62s/it]
 83%|████████▎ | 185752/225000 [350:31:58<58:41:52,  5.38s/it]
 83%|████████▎ | 185753/225000 [350:32:03<58:09:17,  5.33s/it]
 83%|████████▎ | 185754/225000 [350:32:08<57:26:41,  5.27s/it]
 83%|████████▎ | 185755/225000 [350:32:13<55:54:21,  5.13s/it]
 83%|████████▎ | 185756/225000 [350:32:18<56:03:20,  5.14s/it]
 83%|████████▎ | 185757/225000 [350:32:23<56:28:40,  5.18s/it]
 83%|████████▎ | 185758/225000 [350:32:28<55:45:46,  5.12s/it]
 83%|████████▎ | 185759/225000 [350:32:34<56:36:43,  5.19s/it]
 83%|████████▎ | 185760/225000 [350:32:43<68:39:36,  6.30s/it]
 83%|████████▎ | 185761/225000 [350:32:47<61:11:55,  5.61s/it]
 83%|████████▎ | 185762/225000 [350:32:51<56:46:29,  5.21s/it]
 83%|████████▎ | 185763/225000 [350:32:55<53:15:32,  4.89s/it]
 83%|████████▎ | 185764/225000 [350:33:03<63:57:12,  5.87s/it]
 83%|████████▎ | 185765/225000 [350:33:09<64:39:58,  5.93s/it]
 83%|████████▎ | 185766/225000 [350:33:14<59:07:16,  5.42s/it]
 83%|████████▎ | 185767/225000 [350:33:21<66:04:50,  6.06s/it]
 83%|████████▎ | 185768/225000 [350:33:25<58:07:55,  5.33s/it]
 83%|████████▎ | 185769/225000 [350:33:33<66:36:24,  6.11s/it]
 83%|████████▎ | 185770/225000 [350:33:36<58:43:22,  5.39s/it]
 83%|████████▎ | 185771/225000 [350:33:40<53:54:40,  4.95s/it]
 83%|████████▎ | 185772/225000 [350:33:48<62:28:48,  5.73s/it]
 83%|████████▎ | 185773/225000 [350:33:54<64:11:19,  5.89s/it]
 83%|████████▎ | 185774/225000 [350:33:58<58:29:01,  5.37s/it]
 83%|████████▎ | 185775/225000 [350:34:02<54:39:11,  5.02s/it]
 83%|████████▎ | 185776/225000 [350:34:06<51:27:45,  4.72s/it]
 83%|████████▎ | 185777/225000 [350:34:17<71:32:05,  6.57s/it]
 83%|████████▎ | 185778/225000 [350:34:21<60:26:52,  5.55s/it]
 83%|████████▎ | 185779/225000 [350:34:27<65:01:21,  5.97s/it]
 83%|████████▎ | 185780/225000 [350:34:31<57:27:13,  5.27s/it]
 83%|████████▎ | 185781/225000 [350:34:36<55:18:42,  5.08s/it]
 83%|████████▎ | 185782/225000 [350:34:43<63:58:13,  5.87s/it]
 83%|████████▎ | 185783/225000 [350:34:47<55:18:03,  5.08s/it]
 83%|████████▎ | 185784/225000 [350:34:55<66:15:21,  6.08s/it]
 83%|████████▎ | 185785/225000 [350:34:59<59:34:05,  5.47s/it]
 83%|████████▎ | 185786/225000 [350:35:03<55:49:47,  5.13s/it]
 83%|████████▎ | 185787/225000 [350:35:08<52:25:24,  4.81s/it]
 83%|████████▎ | 185788/225000 [350:35:12<49:43:28,  4.57s/it]
 83%|████████▎ | 185789/225000 [350:35:16<48:22:43,  4.44s/it]
 83%|████████▎ | 185790/225000 [350:35:27<70:55:14,  6.51s/it]
 83%|████████▎ | 185791/225000 [350:35:32<64:42:01,  5.94s/it]
 83%|████████▎ | 185792/225000 [350:35:36<58:35:48,  5.38s/it]
 83%|████████▎ | 185793/225000 [350:35:39<52:32:38,  4.82s/it]
 83%|████████▎ | 185794/225000 [350:35:43<50:02:33,  4.60s/it]
 83%|████████▎ | 185795/225000 [350:35:55<74:26:57,  6.84s/it]
 83%|████████▎ | 185796/225000 [350:35:59<63:30:57,  5.83s/it]
 83%|████████▎ | 185797/225000 [350:36:07<70:15:42,  6.45s/it]
 83%|████████▎ | 185798/225000 [350:36:10<60:22:00,  5.54s/it]
 83%|████████▎ | 185799/225000 [350:36:19<69:46:43,  6.41s/it]
 83%|████████▎ | 185800/225000 [350:36:23<62:50:49,  5.77s/it]
                                                              
{'loss': '0.7712', 'grad_norm': '1.152', 'learning_rate': '4.007e-06', 'epoch': '0.8258', 'num_input_tokens_seen': 581638128, 'train_runtime': '1.262e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185800/225000 [350:36:23<62:50:49,  5.77s/it]
 83%|████████▎ | 185801/225000 [350:36:27<56:57:44,  5.23s/it]
 83%|████████▎ | 185802/225000 [350:36:31<52:26:06,  4.82s/it]
 83%|████████▎ | 185803/225000 [350:36:41<70:47:59,  6.50s/it]
 83%|████████▎ | 185804/225000 [350:36:45<61:14:02,  5.62s/it]
 83%|████████▎ | 185805/225000 [350:36:49<56:56:52,  5.23s/it]
 83%|████████▎ | 185806/225000 [350:36:56<62:54:35,  5.78s/it]
 83%|████████▎ | 185807/225000 [350:37:00<55:38:34,  5.11s/it]
 83%|████████▎ | 185808/225000 [350:37:08<65:39:37,  6.03s/it]
 83%|████████▎ | 185809/225000 [350:37:12<58:09:45,  5.34s/it]
 83%|████████▎ | 185810/225000 [350:37:16<54:07:06,  4.97s/it]
 83%|████████▎ | 185811/225000 [350:37:20<50:43:58,  4.66s/it]
 83%|████████▎ | 185812/225000 [350:37:31<72:26:31,  6.65s/it]
 83%|████████▎ | 185813/225000 [350:37:34<61:59:33,  5.70s/it]
 83%|████████▎ | 185814/225000 [350:37:38<56:32:22,  5.19s/it]
 83%|████████▎ | 185815/225000 [350:37:42<52:22:35,  4.81s/it]
 83%|████████▎ | 185816/225000 [350:37:47<50:39:10,  4.65s/it]
 83%|████████▎ | 185817/225000 [350:37:58<73:33:13,  6.76s/it]
 83%|████████▎ | 185818/225000 [350:38:02<63:48:07,  5.86s/it]
 83%|████████▎ | 185819/225000 [350:38:06<57:48:43,  5.31s/it]
 83%|████████▎ | 185820/225000 [350:38:15<68:03:29,  6.25s/it]
 83%|████████▎ | 185821/225000 [350:38:18<57:55:39,  5.32s/it]
 83%|████████▎ | 185822/225000 [350:38:21<52:21:49,  4.81s/it]
 83%|████████▎ | 185823/225000 [350:38:29<61:14:30,  5.63s/it]
 83%|████████▎ | 185824/225000 [350:38:34<58:46:32,  5.40s/it]
 83%|████████▎ | 185825/225000 [350:38:39<58:22:11,  5.36s/it]
 83%|████████▎ | 185826/225000 [350:38:44<57:33:38,  5.29s/it]
 83%|████████▎ | 185827/225000 [350:38:49<55:57:16,  5.14s/it]
 83%|████████▎ | 185828/225000 [350:38:54<56:26:14,  5.19s/it]
 83%|████████▎ | 185829/225000 [350:38:59<56:23:15,  5.18s/it]
 83%|████████▎ | 185830/225000 [350:39:04<56:07:56,  5.16s/it]
 83%|████████▎ | 185831/225000 [350:39:09<54:35:05,  5.02s/it]
 83%|████████▎ | 185832/225000 [350:39:14<55:03:50,  5.06s/it]
 83%|████████▎ | 185833/225000 [350:39:19<53:06:20,  4.88s/it]
 83%|████████▎ | 185834/225000 [350:39:24<53:16:56,  4.90s/it]
 83%|████████▎ | 185835/225000 [350:39:37<79:30:57,  7.31s/it]
 83%|████████▎ | 185836/225000 [350:39:42<74:21:04,  6.83s/it]
 83%|████████▎ | 185837/225000 [350:39:47<66:46:54,  6.14s/it]
 83%|████████▎ | 185838/225000 [350:39:52<63:08:26,  5.80s/it]
 83%|████████▎ | 185839/225000 [350:39:57<59:44:31,  5.49s/it]
 83%|████████▎ | 185840/225000 [350:40:01<56:35:11,  5.20s/it]
 83%|████████▎ | 185841/225000 [350:40:06<55:42:38,  5.12s/it]
 83%|████████▎ | 185842/225000 [350:40:11<54:33:41,  5.02s/it]
 83%|████████▎ | 185843/225000 [350:40:20<66:42:36,  6.13s/it]
 83%|████████▎ | 185844/225000 [350:40:28<73:35:30,  6.77s/it]
 83%|████████▎ | 185845/225000 [350:40:31<62:40:02,  5.76s/it]
 83%|████████▎ | 185846/225000 [350:40:35<55:25:18,  5.10s/it]
 83%|████████▎ | 185847/225000 [350:40:44<67:26:36,  6.20s/it]
 83%|████████▎ | 185848/225000 [350:40:47<58:13:35,  5.35s/it]
 83%|████████▎ | 185849/225000 [350:40:55<65:37:48,  6.03s/it]
 83%|████████▎ | 185850/225000 [350:41:00<64:23:44,  5.92s/it]
                                                              
{'loss': '0.7724', 'grad_norm': '1.154', 'learning_rate': '3.998e-06', 'epoch': '0.826', 'num_input_tokens_seen': 581768232, 'train_runtime': '1.262e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185850/225000 [350:41:00<64:23:44,  5.92s/it]
 83%|████████▎ | 185851/225000 [350:41:05<59:18:03,  5.45s/it]
 83%|████████▎ | 185852/225000 [350:41:12<64:46:14,  5.96s/it]
 83%|████████▎ | 185853/225000 [350:41:16<59:36:31,  5.48s/it]
 83%|████████▎ | 185854/225000 [350:41:20<54:36:48,  5.02s/it]
 83%|████████▎ | 185855/225000 [350:41:29<67:35:19,  6.22s/it]
 83%|████████▎ | 185856/225000 [350:41:34<62:18:36,  5.73s/it]
 83%|████████▎ | 185857/225000 [350:41:40<62:53:14,  5.78s/it]
 83%|████████▎ | 185858/225000 [350:41:43<55:11:53,  5.08s/it]
 83%|████████▎ | 185859/225000 [350:41:51<65:10:50,  6.00s/it]
 83%|████████▎ | 185860/225000 [350:41:54<55:38:01,  5.12s/it]
 83%|████████▎ | 185861/225000 [350:42:02<64:07:44,  5.90s/it]
 83%|████████▎ | 185862/225000 [350:42:07<60:08:47,  5.53s/it]
 83%|████████▎ | 185863/225000 [350:42:11<56:22:24,  5.19s/it]
 83%|████████▎ | 185864/225000 [350:42:18<62:26:30,  5.74s/it]
 83%|████████▎ | 185865/225000 [350:42:21<54:43:51,  5.03s/it]
 83%|████████▎ | 185866/225000 [350:42:31<68:02:21,  6.26s/it]
 83%|████████▎ | 185867/225000 [350:42:34<58:46:09,  5.41s/it]
 83%|████████▎ | 185868/225000 [350:42:42<66:05:11,  6.08s/it]
 83%|████████▎ | 185869/225000 [350:42:47<62:59:41,  5.80s/it]
 83%|████████▎ | 185870/225000 [350:42:51<57:57:13,  5.33s/it]
 83%|████████▎ | 185871/225000 [350:42:55<53:19:20,  4.91s/it]
 83%|████████▎ | 185872/225000 [350:42:59<51:03:12,  4.70s/it]
 83%|████████▎ | 185873/225000 [350:43:03<48:36:15,  4.47s/it]
 83%|████████▎ | 185874/225000 [350:43:08<50:41:35,  4.66s/it]
 83%|████████▎ | 185875/225000 [350:43:14<54:01:34,  4.97s/it]
 83%|████████▎ | 185876/225000 [350:43:19<54:44:19,  5.04s/it]
 83%|████████▎ | 185877/225000 [350:43:24<55:42:16,  5.13s/it]
 83%|████████▎ | 185878/225000 [350:43:30<56:44:04,  5.22s/it]
 83%|████████▎ | 185879/225000 [350:43:35<56:55:35,  5.24s/it]
 83%|████████▎ | 185880/225000 [350:43:41<58:09:34,  5.35s/it]
 83%|████████▎ | 185881/225000 [350:43:46<58:15:11,  5.36s/it]
 83%|████████▎ | 185882/225000 [350:43:51<56:35:09,  5.21s/it]
 83%|████████▎ | 185883/225000 [350:43:56<54:49:17,  5.05s/it]
 83%|████████▎ | 185884/225000 [350:44:01<55:58:15,  5.15s/it]
 83%|████████▎ | 185885/225000 [350:44:06<56:25:42,  5.19s/it]
 83%|████████▎ | 185886/225000 [350:44:11<55:51:48,  5.14s/it]
 83%|████████▎ | 185887/225000 [350:44:17<57:33:16,  5.30s/it]
 83%|████████▎ | 185888/225000 [350:44:25<66:55:09,  6.16s/it]
 83%|████████▎ | 185889/225000 [350:44:34<76:00:10,  7.00s/it]
 83%|████████▎ | 185890/225000 [350:44:38<65:25:58,  6.02s/it]
 83%|████████▎ | 185891/225000 [350:44:42<57:51:30,  5.33s/it]
 83%|████████▎ | 185892/225000 [350:44:53<77:47:33,  7.16s/it]
 83%|████████▎ | 185893/225000 [350:44:57<68:33:49,  6.31s/it]
 83%|████████▎ | 185894/225000 [350:45:01<61:18:15,  5.64s/it]
 83%|████████▎ | 185895/225000 [350:45:06<56:36:47,  5.21s/it]
 83%|████████▎ | 185896/225000 [350:45:14<67:07:01,  6.18s/it]
 83%|████████▎ | 185897/225000 [350:45:20<65:31:47,  6.03s/it]
 83%|████████▎ | 185898/225000 [350:45:27<69:52:59,  6.43s/it]
 83%|████████▎ | 185899/225000 [350:45:32<64:13:17,  5.91s/it]
 83%|████████▎ | 185900/225000 [350:45:36<57:22:45,  5.28s/it]
                                                              
{'loss': '0.7468', 'grad_norm': '1.249', 'learning_rate': '3.988e-06', 'epoch': '0.8262', 'num_input_tokens_seen': 581899032, 'train_runtime': '1.263e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185900/225000 [350:45:36<57:22:45,  5.28s/it]
 83%|████████▎ | 185901/225000 [350:45:39<52:36:54,  4.84s/it]
 83%|████████▎ | 185902/225000 [350:45:45<55:03:12,  5.07s/it]
 83%|████████▎ | 185903/225000 [350:45:54<66:48:10,  6.15s/it]
 83%|████████▎ | 185904/225000 [350:46:00<66:11:37,  6.10s/it]
 83%|████████▎ | 185905/225000 [350:46:03<58:00:54,  5.34s/it]
 83%|████████▎ | 185906/225000 [350:46:08<57:24:26,  5.29s/it]
 83%|████████▎ | 185907/225000 [350:46:14<57:21:44,  5.28s/it]
 83%|████████▎ | 185908/225000 [350:46:19<56:49:01,  5.23s/it]
 83%|████████▎ | 185909/225000 [350:46:24<57:09:18,  5.26s/it]
 83%|████████▎ | 185910/225000 [350:46:29<57:10:47,  5.27s/it]
 83%|████████▎ | 185911/225000 [350:46:35<57:37:15,  5.31s/it]
 83%|████████▎ | 185912/225000 [350:46:40<57:57:47,  5.34s/it]
 83%|████████▎ | 185913/225000 [350:46:45<56:04:32,  5.16s/it]
 83%|████████▎ | 185914/225000 [350:46:50<55:49:04,  5.14s/it]
 83%|████████▎ | 185915/225000 [350:46:55<56:07:15,  5.17s/it]
 83%|████████▎ | 185916/225000 [350:47:01<56:49:18,  5.23s/it]
 83%|████████▎ | 185917/225000 [350:47:10<68:34:10,  6.32s/it]
 83%|████████▎ | 185918/225000 [350:47:19<77:34:24,  7.15s/it]
 83%|████████▎ | 185919/225000 [350:47:22<65:34:53,  6.04s/it]
 83%|████████▎ | 185920/225000 [350:47:30<70:44:36,  6.52s/it]
 83%|████████▎ | 185921/225000 [350:47:35<68:02:34,  6.27s/it]
 83%|████████▎ | 185922/225000 [350:47:40<62:58:48,  5.80s/it]
 83%|████████▎ | 185923/225000 [350:47:44<58:18:22,  5.37s/it]
 83%|████████▎ | 185924/225000 [350:47:49<54:46:50,  5.05s/it]
 83%|████████▎ | 185925/225000 [350:47:53<53:29:17,  4.93s/it]
 83%|████████▎ | 185926/225000 [350:47:58<52:28:35,  4.83s/it]
 83%|████████▎ | 185927/225000 [350:48:02<50:47:04,  4.68s/it]
 83%|████████▎ | 185928/225000 [350:48:07<49:08:11,  4.53s/it]
 83%|████████▎ | 185929/225000 [350:48:19<73:34:49,  6.78s/it]
 83%|████████▎ | 185930/225000 [350:48:23<67:01:15,  6.18s/it]
 83%|████████▎ | 185931/225000 [350:48:27<57:44:34,  5.32s/it]
 83%|████████▎ | 185932/225000 [350:48:35<66:57:08,  6.17s/it]
 83%|████████▎ | 185933/225000 [350:48:39<60:17:07,  5.56s/it]
 83%|████████▎ | 185934/225000 [350:48:43<56:03:48,  5.17s/it]
 83%|████████▎ | 185935/225000 [350:48:47<51:48:26,  4.77s/it]
 83%|████████▎ | 185936/225000 [350:48:54<60:07:56,  5.54s/it]
 83%|████████▎ | 185937/225000 [350:49:01<62:54:59,  5.80s/it]
 83%|████████▎ | 185938/225000 [350:49:05<59:19:56,  5.47s/it]
 83%|████████▎ | 185939/225000 [350:49:10<54:41:57,  5.04s/it]
 83%|████████▎ | 185940/225000 [350:49:15<56:27:26,  5.20s/it]
 83%|████████▎ | 185941/225000 [350:49:24<69:38:08,  6.42s/it]
 83%|████████▎ | 185942/225000 [350:49:28<60:14:05,  5.55s/it]
 83%|████████▎ | 185943/225000 [350:49:34<63:03:41,  5.81s/it]
 83%|████████▎ | 185944/225000 [350:49:39<58:35:50,  5.40s/it]
 83%|████████▎ | 185945/225000 [350:49:43<55:18:34,  5.10s/it]
 83%|████████▎ | 185946/225000 [350:49:47<52:52:23,  4.87s/it]
 83%|████████▎ | 185947/225000 [350:49:52<50:58:08,  4.70s/it]
 83%|████████▎ | 185948/225000 [350:50:03<71:28:07,  6.59s/it]
 83%|████████▎ | 185949/225000 [350:50:07<62:48:10,  5.79s/it]
 83%|████████▎ | 185950/225000 [350:50:10<55:31:36,  5.12s/it]
                                                              
{'loss': '0.7596', 'grad_norm': '1.167', 'learning_rate': '3.978e-06', 'epoch': '0.8264', 'num_input_tokens_seen': 582028536, 'train_runtime': '1.263e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 185950/225000 [350:50:10<55:31:36,  5.12s/it]
 83%|████████▎ | 185951/225000 [350:50:20<69:18:07,  6.39s/it]
 83%|████████▎ | 185952/225000 [350:50:23<59:21:46,  5.47s/it]
 83%|████████▎ | 185953/225000 [350:50:27<55:06:30,  5.08s/it]
 83%|████████▎ | 185954/225000 [350:50:36<69:07:17,  6.37s/it]
 83%|████████▎ | 185955/225000 [350:50:40<60:34:17,  5.58s/it]
 83%|████████▎ | 185956/225000 [350:50:44<53:10:49,  4.90s/it]
 83%|████████▎ | 185957/225000 [350:50:52<65:49:57,  6.07s/it]
 83%|████████▎ | 185958/225000 [350:50:56<57:40:38,  5.32s/it]
 83%|████████▎ | 185959/225000 [350:51:03<63:58:49,  5.90s/it]
 83%|████████▎ | 185960/225000 [350:51:07<58:08:03,  5.36s/it]
 83%|████████▎ | 185961/225000 [350:51:11<53:24:57,  4.93s/it]
 83%|████████▎ | 185962/225000 [350:51:19<63:22:06,  5.84s/it]
 83%|████████▎ | 185963/225000 [350:51:23<56:45:22,  5.23s/it]
 83%|████████▎ | 185964/225000 [350:51:26<51:05:41,  4.71s/it]
 83%|████████▎ | 185965/225000 [350:51:35<63:33:30,  5.86s/it]
 83%|████████▎ | 185966/225000 [350:51:41<65:19:50,  6.03s/it]
 83%|████████▎ | 185967/225000 [350:51:46<60:28:04,  5.58s/it]
 83%|████████▎ | 185968/225000 [350:51:51<57:47:01,  5.33s/it]
 83%|████████▎ | 185969/225000 [350:51:55<53:05:47,  4.90s/it]
 83%|████████▎ | 185970/225000 [350:52:00<53:16:57,  4.91s/it]
 83%|████████▎ | 185971/225000 [350:52:04<52:19:52,  4.83s/it]
 83%|████████▎ | 185972/225000 [350:52:09<51:23:46,  4.74s/it]
 83%|████████▎ | 185973/225000 [350:52:14<51:43:06,  4.77s/it]
 83%|████████▎ | 185974/225000 [350:52:19<52:46:10,  4.87s/it]
 83%|████████▎ | 185975/225000 [350:52:30<72:57:24,  6.73s/it]
 83%|████████▎ | 185976/225000 [350:52:33<62:38:55,  5.78s/it]
 83%|████████▎ | 185977/225000 [350:52:37<55:16:02,  5.10s/it]
 83%|████████▎ | 185978/225000 [350:52:44<62:04:15,  5.73s/it]
 83%|████████▎ | 185979/225000 [350:52:52<69:49:58,  6.44s/it]
 83%|████████▎ | 185980/225000 [350:52:56<60:48:53,  5.61s/it]
 83%|████████▎ | 185981/225000 [350:53:01<58:10:43,  5.37s/it]
 83%|████████▎ | 185982/225000 [350:53:09<68:15:35,  6.30s/it]
 83%|████████▎ | 185983/225000 [350:53:12<58:58:54,  5.44s/it]
 83%|████████▎ | 185984/225000 [350:53:19<63:07:19,  5.82s/it]
 83%|████████▎ | 185985/225000 [350:53:24<59:27:13,  5.49s/it]
 83%|████████▎ | 185986/225000 [350:53:29<56:47:10,  5.24s/it]
 83%|████████▎ | 185987/225000 [350:53:33<53:12:13,  4.91s/it]
 83%|████████▎ | 185988/225000 [350:53:37<52:02:25,  4.80s/it]
 83%|████████▎ | 185989/225000 [350:53:42<51:06:16,  4.72s/it]
 83%|████████▎ | 185990/225000 [350:53:47<51:38:27,  4.77s/it]
 83%|████████▎ | 185991/225000 [350:53:52<52:10:00,  4.81s/it]
 83%|████████▎ | 185992/225000 [350:53:57<54:43:09,  5.05s/it]
 83%|████████▎ | 185993/225000 [350:54:08<74:21:02,  6.86s/it]
 83%|████████▎ | 185994/225000 [350:54:12<63:10:40,  5.83s/it]
 83%|████████▎ | 185995/225000 [350:54:20<71:28:47,  6.60s/it]
 83%|████████▎ | 185996/225000 [350:54:25<64:37:46,  5.97s/it]
 83%|████████▎ | 185997/225000 [350:54:31<67:33:24,  6.24s/it]
 83%|████████▎ | 185998/225000 [350:54:38<68:19:39,  6.31s/it]
 83%|████████▎ | 185999/225000 [350:54:41<58:17:23,  5.38s/it]
 83%|████████▎ | 186000/225000 [350:54:49<65:16:36,  6.03s/it]
                                                              
{'loss': '0.7577', 'grad_norm': '1.244', 'learning_rate': '3.968e-06', 'epoch': '0.8267', 'num_input_tokens_seen': 582157848, 'train_runtime': '1.263e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186000/225000 [350:54:49<65:16:36,  6.03s/it]
 83%|████████▎ | 186001/225000 [350:54:52<57:21:47,  5.30s/it]
 83%|████████▎ | 186002/225000 [350:54:57<56:12:55,  5.19s/it]
 83%|████████▎ | 186003/225000 [350:55:02<56:05:25,  5.18s/it]
 83%|████████▎ | 186004/225000 [350:55:07<54:59:07,  5.08s/it]
 83%|████████▎ | 186005/225000 [350:55:12<54:53:13,  5.07s/it]
 83%|████████▎ | 186006/225000 [350:55:17<55:31:30,  5.13s/it]
 83%|████████▎ | 186007/225000 [350:55:23<55:50:08,  5.15s/it]
 83%|████████▎ | 186008/225000 [350:55:28<56:23:13,  5.21s/it]
 83%|████████▎ | 186009/225000 [350:55:33<55:09:03,  5.09s/it]
 83%|████████▎ | 186010/225000 [350:55:38<54:18:51,  5.01s/it]
 83%|████████▎ | 186011/225000 [350:55:43<54:40:15,  5.05s/it]
 83%|████████▎ | 186012/225000 [350:55:55<78:55:43,  7.29s/it]
 83%|████████▎ | 186013/225000 [350:55:59<66:16:28,  6.12s/it]
 83%|████████▎ | 186014/225000 [350:56:06<70:54:34,  6.55s/it]
 83%|████████▎ | 186015/225000 [350:56:10<60:46:13,  5.61s/it]
 83%|████████▎ | 186016/225000 [350:56:15<60:28:09,  5.58s/it]
 83%|████████▎ | 186017/225000 [350:56:21<60:32:14,  5.59s/it]
 83%|████████▎ | 186018/225000 [350:56:26<60:21:02,  5.57s/it]
 83%|████████▎ | 186019/225000 [350:56:31<58:20:09,  5.39s/it]
 83%|████████▎ | 186020/225000 [350:56:37<59:03:54,  5.45s/it]
 83%|████████▎ | 186021/225000 [350:56:43<60:56:46,  5.63s/it]
 83%|████████▎ | 186022/225000 [350:56:52<70:28:14,  6.51s/it]
 83%|████████▎ | 186023/225000 [350:56:55<61:47:59,  5.71s/it]
 83%|████████▎ | 186024/225000 [350:57:02<66:03:26,  6.10s/it]
 83%|████████▎ | 186025/225000 [350:57:09<69:12:02,  6.39s/it]
 83%|████████▎ | 186026/225000 [350:57:14<62:10:10,  5.74s/it]
 83%|████████▎ | 186027/225000 [350:57:19<59:50:45,  5.53s/it]
 83%|████████▎ | 186028/225000 [350:57:24<60:05:51,  5.55s/it]
 83%|████████▎ | 186029/225000 [350:57:30<59:55:01,  5.53s/it]
 83%|████████▎ | 186030/225000 [350:57:35<58:36:26,  5.41s/it]
 83%|████████▎ | 186031/225000 [350:57:40<58:26:20,  5.40s/it]
 83%|████████▎ | 186032/225000 [350:57:46<57:52:31,  5.35s/it]
 83%|████████▎ | 186033/225000 [350:57:51<58:08:49,  5.37s/it]
 83%|████████▎ | 186034/225000 [350:57:56<56:40:55,  5.24s/it]
 83%|████████▎ | 186035/225000 [350:58:01<57:24:37,  5.30s/it]
 83%|████████▎ | 186036/225000 [350:58:07<57:55:39,  5.35s/it]
 83%|████████▎ | 186037/225000 [350:58:12<58:07:03,  5.37s/it]
 83%|████████▎ | 186038/225000 [350:58:17<57:15:58,  5.29s/it]
 83%|████████▎ | 186039/225000 [350:58:23<58:33:05,  5.41s/it]
 83%|████████▎ | 186040/225000 [350:58:28<58:14:09,  5.38s/it]
 83%|████████▎ | 186041/225000 [350:58:34<58:51:38,  5.44s/it]
 83%|████████▎ | 186042/225000 [350:58:40<59:25:08,  5.49s/it]
 83%|████████▎ | 186043/225000 [350:58:50<76:29:08,  7.07s/it]
 83%|████████▎ | 186044/225000 [350:58:54<66:03:17,  6.10s/it]
 83%|████████▎ | 186045/225000 [350:58:58<57:48:13,  5.34s/it]
 83%|████████▎ | 186046/225000 [350:59:05<65:15:30,  6.03s/it]
 83%|████████▎ | 186047/225000 [350:59:11<63:47:43,  5.90s/it]
 83%|████████▎ | 186048/225000 [350:59:14<56:09:05,  5.19s/it]
 83%|████████▎ | 186049/225000 [350:59:21<62:02:15,  5.73s/it]
 83%|████████▎ | 186050/225000 [350:59:27<62:24:57,  5.77s/it]
                                                              
{'loss': '0.7605', 'grad_norm': '1.08', 'learning_rate': '3.958e-06', 'epoch': '0.8269', 'num_input_tokens_seen': 582288088, 'train_runtime': '1.264e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186050/225000 [350:59:27<62:24:57,  5.77s/it]
 83%|████████▎ | 186051/225000 [350:59:31<56:12:50,  5.20s/it]
 83%|████████▎ | 186052/225000 [350:59:36<54:21:52,  5.02s/it]
 83%|████████▎ | 186053/225000 [350:59:40<51:49:39,  4.79s/it]
 83%|████████▎ | 186054/225000 [350:59:45<52:01:04,  4.81s/it]
 83%|████████▎ | 186055/225000 [350:59:49<51:09:48,  4.73s/it]
 83%|████████▎ | 186056/225000 [350:59:56<57:08:32,  5.28s/it]
 83%|████████▎ | 186057/225000 [351:00:06<73:16:33,  6.77s/it]
 83%|████████▎ | 186058/225000 [351:00:09<60:38:22,  5.61s/it]
 83%|████████▎ | 186059/225000 [351:00:17<68:47:23,  6.36s/it]
 83%|████████▎ | 186060/225000 [351:00:21<59:44:18,  5.52s/it]
 83%|████████▎ | 186061/225000 [351:00:26<59:10:01,  5.47s/it]
 83%|████████▎ | 186062/225000 [351:00:34<66:26:07,  6.14s/it]
 83%|████████▎ | 186063/225000 [351:00:38<60:13:15,  5.57s/it]
 83%|████████▎ | 186064/225000 [351:00:42<54:49:55,  5.07s/it]
 83%|████████▎ | 186065/225000 [351:00:52<70:11:22,  6.49s/it]
 83%|████████▎ | 186066/225000 [351:00:55<60:41:25,  5.61s/it]
 83%|████████▎ | 186067/225000 [351:01:01<60:12:43,  5.57s/it]
 83%|████████▎ | 186068/225000 [351:01:06<60:12:59,  5.57s/it]
 83%|████████▎ | 186069/225000 [351:01:12<60:30:55,  5.60s/it]
 83%|████████▎ | 186070/225000 [351:01:20<68:49:28,  6.36s/it]
 83%|████████▎ | 186071/225000 [351:01:24<60:39:46,  5.61s/it]
 83%|████████▎ | 186072/225000 [351:01:31<64:22:54,  5.95s/it]
 83%|████████▎ | 186073/225000 [351:01:37<63:30:27,  5.87s/it]
 83%|████████▎ | 186074/225000 [351:01:41<58:58:57,  5.45s/it]
 83%|████████▎ | 186075/225000 [351:01:45<55:31:46,  5.14s/it]
 83%|████████▎ | 186076/225000 [351:01:50<54:04:20,  5.00s/it]
 83%|████████▎ | 186077/225000 [351:01:55<52:36:34,  4.87s/it]
 83%|████████▎ | 186078/225000 [351:01:59<51:17:25,  4.74s/it]
 83%|████████▎ | 186079/225000 [351:02:10<70:03:17,  6.48s/it]
 83%|████████▎ | 186080/225000 [351:02:16<68:24:30,  6.33s/it]
 83%|████████▎ | 186081/225000 [351:02:20<62:11:55,  5.75s/it]
 83%|████████▎ | 186082/225000 [351:02:28<70:03:23,  6.48s/it]
 83%|████████▎ | 186083/225000 [351:02:32<60:21:38,  5.58s/it]
 83%|████████▎ | 186084/225000 [351:02:35<53:44:29,  4.97s/it]
 83%|████████▎ | 186085/225000 [351:02:44<64:44:40,  5.99s/it]
 83%|████████▎ | 186086/225000 [351:02:48<59:36:58,  5.52s/it]
 83%|████████▎ | 186087/225000 [351:02:53<56:33:27,  5.23s/it]
 83%|████████▎ | 186088/225000 [351:02:57<54:30:38,  5.04s/it]
 83%|████████▎ | 186089/225000 [351:03:01<52:12:45,  4.83s/it]
 83%|████████▎ | 186090/225000 [351:03:06<50:30:57,  4.67s/it]
 83%|████████▎ | 186091/225000 [351:03:15<65:58:49,  6.10s/it]
 83%|████████▎ | 186092/225000 [351:03:19<59:55:40,  5.54s/it]
 83%|████████▎ | 186093/225000 [351:03:25<58:38:05,  5.43s/it]
 83%|████████▎ | 186094/225000 [351:03:29<56:06:50,  5.19s/it]
 83%|████████▎ | 186095/225000 [351:03:34<54:54:12,  5.08s/it]
 83%|████████▎ | 186096/225000 [351:03:39<55:34:43,  5.14s/it]
 83%|████████▎ | 186097/225000 [351:03:45<55:28:15,  5.13s/it]
 83%|████████▎ | 186098/225000 [351:03:50<55:35:56,  5.15s/it]
 83%|████████▎ | 186099/225000 [351:03:55<55:19:58,  5.12s/it]
 83%|████████▎ | 186100/225000 [351:04:00<54:46:19,  5.07s/it]
                                                              
{'loss': '0.7626', 'grad_norm': '1.129', 'learning_rate': '3.948e-06', 'epoch': '0.8271', 'num_input_tokens_seen': 582418640, 'train_runtime': '1.264e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186100/225000 [351:04:00<54:46:19,  5.07s/it]
 83%|████████▎ | 186101/225000 [351:04:05<55:29:49,  5.14s/it]
 83%|████████▎ | 186102/225000 [351:04:15<71:54:59,  6.66s/it]
 83%|████████▎ | 186103/225000 [351:04:22<73:41:23,  6.82s/it]
 83%|████████▎ | 186104/225000 [351:04:27<65:15:39,  6.04s/it]
 83%|████████▎ | 186105/225000 [351:04:34<70:22:44,  6.51s/it]
 83%|████████▎ | 186106/225000 [351:04:39<63:09:20,  5.85s/it]
 83%|████████▎ | 186107/225000 [351:04:43<59:57:12,  5.55s/it]
 83%|████████▎ | 186108/225000 [351:04:48<57:12:06,  5.29s/it]
 83%|████████▎ | 186109/225000 [351:04:52<54:22:21,  5.03s/it]
 83%|████████▎ | 186110/225000 [351:04:57<52:47:35,  4.89s/it]
 83%|████████▎ | 186111/225000 [351:05:08<73:36:10,  6.81s/it]
 83%|████████▎ | 186112/225000 [351:05:12<62:45:27,  5.81s/it]
 83%|████████▎ | 186113/225000 [351:05:15<54:56:58,  5.09s/it]
 83%|████████▎ | 186114/225000 [351:05:26<74:19:51,  6.88s/it]
 83%|████████▎ | 186115/225000 [351:05:30<62:33:21,  5.79s/it]
 83%|████████▎ | 186116/225000 [351:05:36<63:58:59,  5.92s/it]
 83%|████████▎ | 186117/225000 [351:05:39<56:26:40,  5.23s/it]
 83%|████████▎ | 186118/225000 [351:05:49<70:24:40,  6.52s/it]
 83%|████████▎ | 186119/225000 [351:05:53<63:18:32,  5.86s/it]
 83%|████████▎ | 186120/225000 [351:05:57<58:03:45,  5.38s/it]
 83%|████████▎ | 186121/225000 [351:06:02<53:54:48,  4.99s/it]
 83%|████████▎ | 186122/225000 [351:06:06<51:47:32,  4.80s/it]
 83%|████████▎ | 186123/225000 [351:06:15<66:26:45,  6.15s/it]
 83%|████████▎ | 186124/225000 [351:06:21<64:51:34,  6.01s/it]
 83%|████████▎ | 186125/225000 [351:06:26<62:06:12,  5.75s/it]
 83%|████████▎ | 186126/225000 [351:06:33<65:12:17,  6.04s/it]
 83%|████████▎ | 186127/225000 [351:06:37<58:54:38,  5.46s/it]
 83%|████████▎ | 186128/225000 [351:06:44<63:51:17,  5.91s/it]
 83%|████████▎ | 186129/225000 [351:06:47<55:52:31,  5.17s/it]
 83%|████████▎ | 186130/225000 [351:06:57<69:20:21,  6.42s/it]
 83%|████████▎ | 186131/225000 [351:07:00<59:44:04,  5.53s/it]
 83%|████████▎ | 186132/225000 [351:07:04<54:58:15,  5.09s/it]
 83%|████████▎ | 186133/225000 [351:07:13<68:21:36,  6.33s/it]
 83%|████████▎ | 186134/225000 [351:07:17<59:23:56,  5.50s/it]
 83%|████████▎ | 186135/225000 [351:07:21<53:46:38,  4.98s/it]
 83%|████████▎ | 186136/225000 [351:07:24<49:16:10,  4.56s/it]
 83%|████████▎ | 186137/225000 [351:07:34<64:54:47,  6.01s/it]
 83%|████████▎ | 186138/225000 [351:07:39<61:34:46,  5.70s/it]
 83%|████████▎ | 186139/225000 [351:07:44<59:24:05,  5.50s/it]
 83%|████████▎ | 186140/225000 [351:07:49<58:25:13,  5.41s/it]
 83%|████████▎ | 186141/225000 [351:07:54<57:48:25,  5.36s/it]
 83%|████████▎ | 186142/225000 [351:07:59<57:17:19,  5.31s/it]
 83%|████████▎ | 186143/225000 [351:08:04<55:29:50,  5.14s/it]
 83%|████████▎ | 186144/225000 [351:08:09<55:28:20,  5.14s/it]
 83%|████████▎ | 186145/225000 [351:08:14<55:06:38,  5.11s/it]
 83%|████████▎ | 186146/225000 [351:08:19<54:09:45,  5.02s/it]
 83%|████████▎ | 186147/225000 [351:08:24<53:36:57,  4.97s/it]
 83%|████████▎ | 186148/225000 [351:08:29<53:31:42,  4.96s/it]
 83%|████████▎ | 186149/225000 [351:08:34<53:33:02,  4.96s/it]
 83%|████████▎ | 186150/225000 [351:08:39<55:07:06,  5.11s/it]
                                                              
{'loss': '0.759', 'grad_norm': '1.166', 'learning_rate': '3.938e-06', 'epoch': '0.8273', 'num_input_tokens_seen': 582549144, 'train_runtime': '1.264e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186150/225000 [351:08:39<55:07:06,  5.11s/it]
 83%|████████▎ | 186151/225000 [351:08:44<54:33:38,  5.06s/it]
 83%|████████▎ | 186152/225000 [351:08:57<79:41:01,  7.38s/it]
 83%|████████▎ | 186153/225000 [351:09:01<69:20:20,  6.43s/it]
 83%|████████▎ | 186154/225000 [351:09:05<60:58:27,  5.65s/it]
 83%|████████▎ | 186155/225000 [351:09:13<68:25:02,  6.34s/it]
 83%|████████▎ | 186156/225000 [351:09:19<68:28:19,  6.35s/it]
 83%|████████▎ | 186157/225000 [351:09:23<61:10:38,  5.67s/it]
 83%|████████▎ | 186158/225000 [351:09:27<55:29:54,  5.14s/it]
 83%|████████▎ | 186159/225000 [351:09:31<52:07:31,  4.83s/it]
 83%|████████▎ | 186160/225000 [351:09:36<52:16:06,  4.84s/it]
 83%|████████▎ | 186161/225000 [351:09:41<50:10:21,  4.65s/it]
 83%|████████▎ | 186162/225000 [351:09:45<49:16:25,  4.57s/it]
 83%|████████▎ | 186163/225000 [351:09:54<63:25:24,  5.88s/it]
 83%|████████▎ | 186164/225000 [351:10:02<71:03:16,  6.59s/it]
 83%|████████▎ | 186165/225000 [351:10:06<62:09:46,  5.76s/it]
 83%|████████▎ | 186166/225000 [351:10:10<55:32:47,  5.15s/it]
 83%|████████▎ | 186167/225000 [351:10:19<70:05:50,  6.50s/it]
 83%|████████▎ | 186168/225000 [351:10:24<64:20:37,  5.97s/it]
 83%|████████▎ | 186169/225000 [351:10:28<59:03:46,  5.48s/it]
 83%|████████▎ | 186170/225000 [351:10:33<55:28:43,  5.14s/it]
 83%|████████▎ | 186171/225000 [351:10:37<53:19:41,  4.94s/it]
 83%|████████▎ | 186172/225000 [351:10:42<52:15:08,  4.84s/it]
 83%|████████▎ | 186173/225000 [351:10:46<51:12:12,  4.75s/it]
 83%|████████▎ | 186174/225000 [351:10:58<72:34:40,  6.73s/it]
 83%|████████▎ | 186175/225000 [351:11:03<67:42:42,  6.28s/it]
 83%|████████▎ | 186176/225000 [351:11:11<75:06:22,  6.96s/it]
 83%|████████▎ | 186177/225000 [351:11:15<64:38:24,  5.99s/it]
 83%|████████▎ | 186178/225000 [351:11:19<57:53:23,  5.37s/it]
 83%|████████▎ | 186179/225000 [351:11:28<70:53:10,  6.57s/it]
 83%|████████▎ | 186180/225000 [351:11:32<61:02:56,  5.66s/it]
 83%|████████▎ | 186181/225000 [351:11:35<53:45:56,  4.99s/it]
 83%|████████▎ | 186182/225000 [351:11:41<54:34:27,  5.06s/it]
 83%|████████▎ | 186183/225000 [351:11:49<64:37:49,  5.99s/it]
 83%|████████▎ | 186184/225000 [351:11:52<56:22:04,  5.23s/it]
 83%|████████▎ | 186185/225000 [351:11:57<54:43:59,  5.08s/it]
 83%|████████▎ | 186186/225000 [351:12:05<65:36:09,  6.08s/it]
 83%|████████▎ | 186187/225000 [351:12:09<57:27:08,  5.33s/it]
 83%|████████▎ | 186188/225000 [351:12:14<56:44:07,  5.26s/it]
 83%|████████▎ | 186189/225000 [351:12:19<56:48:08,  5.27s/it]
 83%|████████▎ | 186190/225000 [351:12:25<57:43:45,  5.35s/it]
 83%|████████▎ | 186191/225000 [351:12:30<57:18:16,  5.32s/it]
 83%|████████▎ | 186192/225000 [351:12:35<56:25:26,  5.23s/it]
 83%|████████▎ | 186193/225000 [351:12:40<56:34:02,  5.25s/it]
 83%|████████▎ | 186194/225000 [351:12:46<56:20:18,  5.23s/it]
 83%|████████▎ | 186195/225000 [351:12:54<64:46:15,  6.01s/it]
 83%|████████▎ | 186196/225000 [351:12:59<61:59:48,  5.75s/it]
 83%|████████▎ | 186197/225000 [351:13:04<60:38:04,  5.63s/it]
 83%|████████▎ | 186198/225000 [351:13:13<70:37:57,  6.55s/it]
 83%|████████▎ | 186199/225000 [351:13:16<61:39:11,  5.72s/it]
 83%|████████▎ | 186200/225000 [351:13:20<55:35:58,  5.16s/it]
                                                              
{'loss': '0.7611', 'grad_norm': '1.25', 'learning_rate': '3.928e-06', 'epoch': '0.8276', 'num_input_tokens_seen': 582679616, 'train_runtime': '1.264e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186200/225000 [351:13:20<55:35:58,  5.16s/it]
 83%|████████▎ | 186201/225000 [351:13:31<74:29:51,  6.91s/it]
 83%|████████▎ | 186202/225000 [351:13:35<63:10:36,  5.86s/it]
 83%|████████▎ | 186203/225000 [351:13:38<56:10:44,  5.21s/it]
 83%|████████▎ | 186204/225000 [351:13:46<63:46:39,  5.92s/it]
 83%|████████▎ | 186205/225000 [351:13:49<55:43:27,  5.17s/it]
 83%|████████▎ | 186206/225000 [351:13:56<59:43:53,  5.54s/it]
 83%|████████▎ | 186207/225000 [351:14:02<61:50:11,  5.74s/it]
 83%|████████▎ | 186208/225000 [351:14:06<56:54:43,  5.28s/it]
 83%|████████▎ | 186209/225000 [351:14:11<55:11:53,  5.12s/it]
 83%|████████▎ | 186210/225000 [351:14:16<55:33:10,  5.16s/it]
 83%|████████▎ | 186211/225000 [351:14:21<55:51:48,  5.18s/it]
 83%|████████▎ | 186212/225000 [351:14:27<55:29:08,  5.15s/it]
 83%|████████▎ | 186213/225000 [351:14:32<55:34:53,  5.16s/it]
 83%|████████▎ | 186214/225000 [351:14:37<55:31:58,  5.15s/it]
 83%|████████▎ | 186215/225000 [351:14:42<55:24:42,  5.14s/it]
 83%|████████▎ | 186216/225000 [351:14:47<54:48:35,  5.09s/it]
 83%|████████▎ | 186217/225000 [351:14:52<54:49:17,  5.09s/it]
 83%|████████▎ | 186218/225000 [351:14:57<55:28:37,  5.15s/it]
 83%|████████▎ | 186219/225000 [351:15:03<58:27:32,  5.43s/it]
 83%|████████▎ | 186220/225000 [351:15:15<78:20:15,  7.27s/it]
 83%|████████▎ | 186221/225000 [351:15:18<65:59:31,  6.13s/it]
 83%|████████▎ | 186222/225000 [351:15:22<58:45:12,  5.45s/it]
 83%|████████▎ | 186223/225000 [351:15:28<59:09:28,  5.49s/it]
 83%|████████▎ | 186224/225000 [351:15:33<57:39:05,  5.35s/it]
 83%|████████▎ | 186225/225000 [351:15:41<66:27:42,  6.17s/it]
 83%|████████▎ | 186226/225000 [351:15:49<71:38:34,  6.65s/it]
 83%|████████▎ | 186227/225000 [351:15:53<63:59:48,  5.94s/it]
 83%|████████▎ | 186228/225000 [351:15:58<59:22:39,  5.51s/it]
 83%|████████▎ | 186229/225000 [351:16:02<54:42:27,  5.08s/it]
 83%|████████▎ | 186230/225000 [351:16:06<52:13:43,  4.85s/it]
 83%|████████▎ | 186231/225000 [351:16:11<52:57:04,  4.92s/it]
 83%|████████▎ | 186232/225000 [351:16:23<74:09:01,  6.89s/it]
 83%|████████▎ | 186233/225000 [351:16:26<62:26:25,  5.80s/it]
 83%|████████▎ | 186234/225000 [351:16:29<54:52:24,  5.10s/it]
 83%|████████▎ | 186235/225000 [351:16:36<61:24:43,  5.70s/it]
 83%|████████▎ | 186236/225000 [351:16:41<58:14:47,  5.41s/it]
 83%|████████▎ | 186237/225000 [351:16:46<55:20:12,  5.14s/it]
 83%|████████▎ | 186238/225000 [351:16:50<53:24:42,  4.96s/it]
 83%|████████▎ | 186239/225000 [351:17:01<73:32:03,  6.83s/it]
 83%|████████▎ | 186240/225000 [351:17:06<65:31:16,  6.09s/it]
 83%|████████▎ | 186241/225000 [351:17:10<60:51:16,  5.65s/it]
 83%|████████▎ | 186242/225000 [351:17:15<57:09:29,  5.31s/it]
 83%|████████▎ | 186243/225000 [351:17:20<57:44:01,  5.36s/it]
 83%|████████▎ | 186244/225000 [351:17:25<55:33:11,  5.16s/it]
 83%|████████▎ | 186245/225000 [351:17:30<55:06:50,  5.12s/it]
 83%|████████▎ | 186246/225000 [351:17:35<54:21:04,  5.05s/it]
 83%|████████▎ | 186247/225000 [351:17:41<58:20:08,  5.42s/it]
 83%|████████▎ | 186248/225000 [351:17:54<82:09:42,  7.63s/it]
 83%|████████▎ | 186249/225000 [351:17:57<67:42:38,  6.29s/it]
 83%|████████▎ | 186250/225000 [351:18:03<65:40:19,  6.10s/it]
                                                              
{'loss': '0.7574', 'grad_norm': '1.288', 'learning_rate': '3.919e-06', 'epoch': '0.8278', 'num_input_tokens_seen': 582810416, 'train_runtime': '1.265e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186250/225000 [351:18:03<65:40:19,  6.10s/it]
 83%|████████▎ | 186251/225000 [351:18:09<65:55:32,  6.12s/it]
 83%|████████▎ | 186252/225000 [351:18:13<58:53:58,  5.47s/it]
 83%|████████▎ | 186253/225000 [351:18:17<55:19:02,  5.14s/it]
 83%|████████▎ | 186254/225000 [351:18:22<53:04:45,  4.93s/it]
 83%|████████▎ | 186255/225000 [351:18:26<50:43:51,  4.71s/it]
 83%|████████▎ | 186256/225000 [351:18:37<70:41:36,  6.57s/it]
 83%|████████▎ | 186257/225000 [351:18:41<63:31:32,  5.90s/it]
 83%|████████▎ | 186258/225000 [351:18:45<56:04:56,  5.21s/it]
 83%|████████▎ | 186259/225000 [351:18:52<62:59:17,  5.85s/it]
 83%|████████▎ | 186260/225000 [351:18:56<56:12:25,  5.22s/it]
 83%|████████▎ | 186261/225000 [351:19:00<52:21:21,  4.87s/it]
 83%|████████▎ | 186262/225000 [351:19:09<67:14:22,  6.25s/it]
 83%|████████▎ | 186263/225000 [351:19:13<59:16:55,  5.51s/it]
 83%|████████▎ | 186264/225000 [351:19:17<54:24:04,  5.06s/it]
 83%|████████▎ | 186265/225000 [351:19:22<52:48:20,  4.91s/it]
 83%|████████▎ | 186266/225000 [351:19:26<51:18:46,  4.77s/it]
 83%|████████▎ | 186267/225000 [351:19:31<51:53:45,  4.82s/it]
 83%|████████▎ | 186268/225000 [351:19:36<51:14:56,  4.76s/it]
 83%|████████▎ | 186269/225000 [351:19:48<73:49:08,  6.86s/it]
 83%|████████▎ | 186270/225000 [351:19:52<66:49:55,  6.21s/it]
 83%|████████▎ | 186271/225000 [351:19:56<59:26:57,  5.53s/it]
 83%|████████▎ | 186272/225000 [351:20:03<64:36:30,  6.01s/it]
 83%|████████▎ | 186273/225000 [351:20:08<58:54:22,  5.48s/it]
 83%|████████▎ | 186274/225000 [351:20:11<53:14:14,  4.95s/it]
 83%|████████▎ | 186275/225000 [351:20:16<52:15:22,  4.86s/it]
 83%|████████▎ | 186276/225000 [351:20:21<51:42:55,  4.81s/it]
 83%|████████▎ | 186277/225000 [351:20:25<51:21:25,  4.77s/it]
 83%|████████▎ | 186278/225000 [351:20:30<51:46:20,  4.81s/it]
 83%|████████▎ | 186279/225000 [351:20:40<67:47:36,  6.30s/it]
 83%|████████▎ | 186280/225000 [351:20:47<69:56:45,  6.50s/it]
 83%|████████▎ | 186281/225000 [351:20:50<60:10:20,  5.59s/it]
 83%|████████▎ | 186282/225000 [351:21:00<73:28:51,  6.83s/it]
 83%|████████▎ | 186283/225000 [351:21:04<62:27:13,  5.81s/it]
 83%|████████▎ | 186284/225000 [351:21:12<70:50:27,  6.59s/it]
 83%|████████▎ | 186285/225000 [351:21:16<61:44:45,  5.74s/it]
 83%|████████▎ | 186286/225000 [351:21:20<55:39:16,  5.18s/it]
 83%|████████▎ | 186287/225000 [351:21:29<68:21:47,  6.36s/it]
 83%|████████▎ | 186288/225000 [351:21:32<59:59:49,  5.58s/it]
 83%|████████▎ | 186289/225000 [351:21:36<54:08:49,  5.04s/it]
 83%|████████▎ | 186290/225000 [351:21:45<65:24:33,  6.08s/it]
 83%|████████▎ | 186291/225000 [351:21:49<59:47:01,  5.56s/it]
 83%|████████▎ | 186292/225000 [351:21:53<55:22:43,  5.15s/it]
 83%|████████▎ | 186293/225000 [351:22:01<63:58:03,  5.95s/it]
 83%|████████▎ | 186294/225000 [351:22:07<62:25:08,  5.81s/it]
 83%|████████▎ | 186295/225000 [351:22:13<64:31:19,  6.00s/it]
 83%|████████▎ | 186296/225000 [351:22:17<56:58:36,  5.30s/it]
 83%|████████▎ | 186297/225000 [351:22:24<64:08:57,  5.97s/it]
 83%|████████▎ | 186298/225000 [351:22:29<59:24:07,  5.53s/it]
 83%|████████▎ | 186299/225000 [351:22:33<56:53:01,  5.29s/it]
 83%|████████▎ | 186300/225000 [351:22:38<55:26:52,  5.16s/it]
                                                              
{'loss': '0.7747', 'grad_norm': '1.172', 'learning_rate': '3.909e-06', 'epoch': '0.828', 'num_input_tokens_seen': 582944304, 'train_runtime': '1.265e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186300/225000 [351:22:38<55:26:52,  5.16s/it]
 83%|████████▎ | 186301/225000 [351:22:44<56:16:40,  5.24s/it]
 83%|████████▎ | 186302/225000 [351:22:49<55:25:54,  5.16s/it]
 83%|████████▎ | 186303/225000 [351:22:54<54:54:06,  5.11s/it]
 83%|████████▎ | 186304/225000 [351:22:59<54:16:15,  5.05s/it]
 83%|████████▎ | 186305/225000 [351:23:08<68:28:37,  6.37s/it]
 83%|████████▎ | 186306/225000 [351:23:16<72:56:01,  6.79s/it]
 83%|████████▎ | 186307/225000 [351:23:19<61:58:20,  5.77s/it]
 83%|████████▎ | 186308/225000 [351:23:24<57:38:32,  5.36s/it]
 83%|████████▎ | 186309/225000 [351:23:33<71:03:35,  6.61s/it]
 83%|████████▎ | 186310/225000 [351:23:41<75:26:54,  7.02s/it]
 83%|████████▎ | 186311/225000 [351:23:46<69:00:46,  6.42s/it]
 83%|████████▎ | 186312/225000 [351:23:51<63:46:45,  5.93s/it]
 83%|████████▎ | 186313/225000 [351:23:57<62:38:59,  5.83s/it]
 83%|████████▎ | 186314/225000 [351:24:01<58:59:37,  5.49s/it]
 83%|████████▎ | 186315/225000 [351:24:10<68:36:04,  6.38s/it]
 83%|████████▎ | 186316/225000 [351:24:14<61:34:48,  5.73s/it]
 83%|████████▎ | 186317/225000 [351:24:18<57:55:24,  5.39s/it]
 83%|████████▎ | 186318/225000 [351:24:30<77:56:27,  7.25s/it]
 83%|████████▎ | 186319/225000 [351:24:33<65:00:42,  6.05s/it]
 83%|████████▎ | 186320/225000 [351:24:41<70:23:27,  6.55s/it]
 83%|████████▎ | 186321/225000 [351:24:45<62:33:28,  5.82s/it]
 83%|████████▎ | 186322/225000 [351:24:56<78:37:42,  7.32s/it]
 83%|████████▎ | 186323/225000 [351:25:01<71:51:43,  6.69s/it]
 83%|████████▎ | 186324/225000 [351:25:09<76:20:08,  7.11s/it]
 83%|████████▎ | 186325/225000 [351:25:14<69:00:50,  6.42s/it]
 83%|████████▎ | 186326/225000 [351:25:19<63:26:20,  5.91s/it]
 83%|████████▎ | 186327/225000 [351:25:24<60:05:12,  5.59s/it]
 83%|████████▎ | 186328/225000 [351:25:28<57:33:02,  5.36s/it]
 83%|████████▎ | 186329/225000 [351:25:33<56:11:57,  5.23s/it]
 83%|████████▎ | 186330/225000 [351:25:38<54:16:31,  5.05s/it]
 83%|████████▎ | 186331/225000 [351:25:43<55:21:06,  5.15s/it]
 83%|████████▎ | 186332/225000 [351:25:52<66:33:52,  6.20s/it]
 83%|████████▎ | 186333/225000 [351:26:00<72:36:29,  6.76s/it]
 83%|████████▎ | 186334/225000 [351:26:03<59:54:52,  5.58s/it]
 83%|████████▎ | 186335/225000 [351:26:12<71:36:28,  6.67s/it]
 83%|████████▎ | 186336/225000 [351:26:17<66:37:23,  6.20s/it]
 83%|████████▎ | 186337/225000 [351:26:22<63:19:38,  5.90s/it]
 83%|████████▎ | 186338/225000 [351:26:28<63:05:46,  5.88s/it]
 83%|████████▎ | 186339/225000 [351:26:33<60:05:02,  5.59s/it]
 83%|████████▎ | 186340/225000 [351:26:38<58:12:49,  5.42s/it]
 83%|████████▎ | 186341/225000 [351:26:43<56:48:39,  5.29s/it]
 83%|████████▎ | 186342/225000 [351:26:48<56:02:42,  5.22s/it]
 83%|████████▎ | 186343/225000 [351:26:53<54:37:05,  5.09s/it]
 83%|████████▎ | 186344/225000 [351:26:58<55:20:27,  5.15s/it]
 83%|████████▎ | 186345/225000 [351:27:03<54:14:10,  5.05s/it]
 83%|████████▎ | 186346/225000 [351:27:13<69:43:19,  6.49s/it]
 83%|████████▎ | 186347/225000 [351:27:17<61:06:31,  5.69s/it]
 83%|████████▎ | 186348/225000 [351:27:24<66:42:36,  6.21s/it]
 83%|████████▎ | 186349/225000 [351:27:32<69:54:17,  6.51s/it]
 83%|████████▎ | 186350/225000 [351:27:36<62:43:50,  5.84s/it]
                                                              
{'loss': '0.789', 'grad_norm': '5.894', 'learning_rate': '3.899e-06', 'epoch': '0.8282', 'num_input_tokens_seen': 583083840, 'train_runtime': '1.265e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186350/225000 [351:27:36<62:43:50,  5.84s/it]
 83%|████████▎ | 186351/225000 [351:27:40<57:51:48,  5.39s/it]
 83%|████████▎ | 186352/225000 [351:27:45<55:24:35,  5.16s/it]
 83%|████████▎ | 186353/225000 [351:27:50<54:11:50,  5.05s/it]
 83%|████████▎ | 186354/225000 [351:27:55<57:06:44,  5.32s/it]
 83%|████████▎ | 186355/225000 [351:28:04<66:53:12,  6.23s/it]
 83%|████████▎ | 186356/225000 [351:28:12<73:50:59,  6.88s/it]
 83%|████████▎ | 186357/225000 [351:28:15<62:07:22,  5.79s/it]
 83%|████████▎ | 186358/225000 [351:28:22<62:55:30,  5.86s/it]
 83%|████████▎ | 186359/225000 [351:28:26<58:05:52,  5.41s/it]
 83%|████████▎ | 186360/225000 [351:28:31<57:35:49,  5.37s/it]
 83%|████████▎ | 186361/225000 [351:28:46<88:15:10,  8.22s/it]
 83%|████████▎ | 186362/225000 [351:28:50<75:22:43,  7.02s/it]
 83%|████████▎ | 186363/225000 [351:28:54<63:16:28,  5.90s/it]
 83%|████████▎ | 186364/225000 [351:29:00<66:28:42,  6.19s/it]
 83%|████████▎ | 186365/225000 [351:29:06<64:00:56,  5.96s/it]
 83%|████████▎ | 186366/225000 [351:29:10<56:42:22,  5.28s/it]
 83%|████████▎ | 186367/225000 [351:29:19<70:14:32,  6.55s/it]
 83%|████████▎ | 186368/225000 [351:29:23<62:34:03,  5.83s/it]
 83%|████████▎ | 186369/225000 [351:29:27<56:26:24,  5.26s/it]
 83%|████████▎ | 186370/225000 [351:29:31<52:05:16,  4.85s/it]
 83%|████████▎ | 186371/225000 [351:29:40<66:33:46,  6.20s/it]
 83%|████████▎ | 186372/225000 [351:29:44<58:14:19,  5.43s/it]
 83%|████████▎ | 186373/225000 [351:29:50<61:38:43,  5.75s/it]
 83%|████████▎ | 186374/225000 [351:29:56<60:11:47,  5.61s/it]
 83%|████████▎ | 186375/225000 [351:29:59<54:01:42,  5.04s/it]
 83%|████████▎ | 186376/225000 [351:30:07<61:21:40,  5.72s/it]
 83%|████████▎ | 186377/225000 [351:30:10<54:33:57,  5.09s/it]
 83%|████████▎ | 186378/225000 [351:30:14<50:23:54,  4.70s/it]
 83%|████████▎ | 186379/225000 [351:30:19<49:51:37,  4.65s/it]
 83%|████████▎ | 186380/225000 [351:30:30<70:34:29,  6.58s/it]
 83%|████████▎ | 186381/225000 [351:30:34<62:31:32,  5.83s/it]
 83%|████████▎ | 186382/225000 [351:30:40<61:55:19,  5.77s/it]
 83%|████████▎ | 186383/225000 [351:30:48<69:31:01,  6.48s/it]
 83%|████████▎ | 186384/225000 [351:30:52<61:49:15,  5.76s/it]
 83%|████████▎ | 186385/225000 [351:30:57<59:48:16,  5.58s/it]
 83%|████████▎ | 186386/225000 [351:31:02<57:24:11,  5.35s/it]
 83%|████████▎ | 186387/225000 [351:31:11<70:04:53,  6.53s/it]
 83%|████████▎ | 186388/225000 [351:31:16<65:22:18,  6.09s/it]
 83%|████████▎ | 186389/225000 [351:31:22<65:49:05,  6.14s/it]
 83%|████████▎ | 186390/225000 [351:31:26<59:20:38,  5.53s/it]
 83%|████████▎ | 186391/225000 [351:31:30<53:35:01,  5.00s/it]
 83%|████████▎ | 186392/225000 [351:31:35<53:04:50,  4.95s/it]
 83%|████████▎ | 186393/225000 [351:31:45<70:23:39,  6.56s/it]
 83%|████████▎ | 186394/225000 [351:31:49<60:41:17,  5.66s/it]
 83%|████████▎ | 186395/225000 [351:31:57<68:08:12,  6.35s/it]
 83%|████████▎ | 186396/225000 [351:32:01<61:33:25,  5.74s/it]
 83%|████████▎ | 186397/225000 [351:32:05<54:43:13,  5.10s/it]
 83%|████████▎ | 186398/225000 [351:32:12<61:45:52,  5.76s/it]
 83%|████████▎ | 186399/225000 [351:32:16<56:37:08,  5.28s/it]
 83%|████████▎ | 186400/225000 [351:32:20<52:42:18,  4.92s/it]
                                                              
{'loss': '0.796', 'grad_norm': '1.079', 'learning_rate': '3.889e-06', 'epoch': '0.8284', 'num_input_tokens_seen': 583219240, 'train_runtime': '1.266e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186400/225000 [351:32:20<52:42:18,  4.92s/it]
 83%|████████▎ | 186401/225000 [351:32:31<71:45:15,  6.69s/it]
 83%|████████▎ | 186402/225000 [351:32:36<65:03:30,  6.07s/it]
 83%|████████▎ | 186403/225000 [351:32:40<59:56:29,  5.59s/it]
 83%|████████▎ | 186404/225000 [351:32:46<60:30:45,  5.64s/it]
 83%|████████▎ | 186405/225000 [351:32:50<56:49:36,  5.30s/it]
 83%|████████▎ | 186406/225000 [351:33:01<72:35:59,  6.77s/it]
 83%|████████▎ | 186407/225000 [351:33:06<66:54:20,  6.24s/it]
 83%|████████▎ | 186408/225000 [351:33:13<69:36:24,  6.49s/it]
 83%|████████▎ | 186409/225000 [351:33:18<64:17:56,  6.00s/it]
 83%|████████▎ | 186410/225000 [351:33:23<62:10:31,  5.80s/it]
 83%|████████▎ | 186411/225000 [351:33:28<59:34:14,  5.56s/it]
 83%|████████▎ | 186412/225000 [351:33:33<57:45:21,  5.39s/it]
 83%|████████▎ | 186413/225000 [351:33:38<56:36:03,  5.28s/it]
 83%|████████▎ | 186414/225000 [351:33:43<57:04:34,  5.33s/it]
 83%|████████▎ | 186415/225000 [351:33:49<56:34:16,  5.28s/it]
 83%|████████▎ | 186416/225000 [351:33:53<54:59:01,  5.13s/it]
 83%|████████▎ | 186417/225000 [351:34:02<67:00:50,  6.25s/it]
 83%|████████▎ | 186418/225000 [351:34:07<62:46:33,  5.86s/it]
 83%|████████▎ | 186419/225000 [351:34:12<58:29:41,  5.46s/it]
 83%|████████▎ | 186420/225000 [351:34:17<57:29:46,  5.37s/it]
 83%|████████▎ | 186421/225000 [351:34:22<56:50:29,  5.30s/it]
 83%|████████▎ | 186422/225000 [351:34:27<56:50:21,  5.30s/it]
 83%|████████▎ | 186423/225000 [351:34:32<54:39:46,  5.10s/it]
 83%|████████▎ | 186424/225000 [351:34:41<68:37:37,  6.40s/it]
 83%|████████▎ | 186425/225000 [351:34:49<72:46:44,  6.79s/it]
 83%|████████▎ | 186426/225000 [351:34:53<62:18:55,  5.82s/it]
 83%|████████▎ | 186427/225000 [351:34:56<55:39:31,  5.19s/it]
 83%|████████▎ | 186428/225000 [351:35:01<52:33:16,  4.91s/it]
 83%|████████▎ | 186429/225000 [351:35:12<74:07:15,  6.92s/it]
 83%|████████▎ | 186430/225000 [351:35:16<62:29:15,  5.83s/it]
 83%|████████▎ | 186431/225000 [351:35:19<55:10:12,  5.15s/it]
 83%|████████▎ | 186432/225000 [351:35:24<54:23:36,  5.08s/it]
 83%|████████▎ | 186433/225000 [351:35:29<53:03:15,  4.95s/it]
 83%|████████▎ | 186434/225000 [351:35:33<51:42:16,  4.83s/it]
 83%|████████▎ | 186435/225000 [351:35:38<51:00:52,  4.76s/it]
 83%|████████▎ | 186436/225000 [351:35:49<71:57:39,  6.72s/it]
 83%|████████▎ | 186437/225000 [351:35:55<70:39:10,  6.60s/it]
 83%|████████▎ | 186438/225000 [351:36:04<76:41:57,  7.16s/it]
 83%|████████▎ | 186439/225000 [351:36:12<79:45:05,  7.45s/it]
 83%|████████▎ | 186440/225000 [351:36:15<66:46:07,  6.23s/it]
 83%|████████▎ | 186441/225000 [351:36:19<58:39:18,  5.48s/it]
 83%|████████▎ | 186442/225000 [351:36:23<55:00:27,  5.14s/it]
 83%|████████▎ | 186443/225000 [351:36:33<70:40:32,  6.60s/it]
 83%|████████▎ | 186444/225000 [351:36:38<65:02:10,  6.07s/it]
 83%|████████▎ | 186445/225000 [351:36:42<58:04:11,  5.42s/it]
 83%|████████▎ | 186446/225000 [351:36:46<54:29:08,  5.09s/it]
 83%|████████▎ | 186447/225000 [351:36:51<52:38:26,  4.92s/it]
 83%|████████▎ | 186448/225000 [351:36:58<59:23:23,  5.55s/it]
 83%|████████▎ | 186449/225000 [351:37:08<74:00:50,  6.91s/it]
 83%|████████▎ | 186450/225000 [351:37:12<64:03:52,  5.98s/it]
                                                              
{'loss': '0.7893', 'grad_norm': '1.145', 'learning_rate': '3.879e-06', 'epoch': '0.8287', 'num_input_tokens_seen': 583356528, 'train_runtime': '1.266e+06', 'train_tokens_per_second': '460.8'}

 83%|████████▎ | 186450/225000 [351:37:12<64:03:52,  5.98s/it]
 83%|████████▎ | 186451/225000 [351:37:16<56:29:42,  5.28s/it]
 83%|████████▎ | 186452/225000 [351:37:24<67:17:34,  6.28s/it]
 83%|████████▎ | 186453/225000 [351:37:28<60:19:43,  5.63s/it]
 83%|████████▎ | 186454/225000 [351:37:41<81:41:38,  7.63s/it]
 83%|████████▎ | 186455/225000 [351:37:44<68:42:30,  6.42s/it]
 83%|████████▎ | 186456/225000 [351:37:48<59:15:45,  5.54s/it]
 83%|████████▎ | 186457/225000 [351:37:53<57:28:36,  5.37s/it]
 83%|████████▎ | 186458/225000 [351:37:58<56:46:06,  5.30s/it]
 83%|████████▎ | 186459/225000 [351:38:03<54:55:11,  5.13s/it]
 83%|████████▎ | 186460/225000 [351:38:14<74:57:18,  7.00s/it]
 83%|████████▎ | 186461/225000 [351:38:18<66:40:13,  6.23s/it]
 83%|████████▎ | 186462/225000 [351:38:22<59:27:30,  5.55s/it]
 83%|████████▎ | 186463/225000 [351:38:27<55:41:16,  5.20s/it]
 83%|████████▎ | 186464/225000 [351:38:32<56:34:53,  5.29s/it]
 83%|████████▎ | 186465/225000 [351:38:39<63:10:17,  5.90s/it]
 83%|████████▎ | 186466/225000 [351:38:51<80:12:03,  7.49s/it]
 83%|████████▎ | 186467/225000 [351:38:55<70:57:06,  6.63s/it]
 83%|████████▎ | 186468/225000 [351:39:01<68:54:53,  6.44s/it]
 83%|████████▎ | 186469/225000 [351:39:05<60:59:17,  5.70s/it]
 83%|████████▎ | 186470/225000 [351:39:09<55:32:54,  5.19s/it]
 83%|████████▎ | 186471/225000 [351:39:13<51:14:38,  4.79s/it]
 83%|████████▎ | 186472/225000 [351:39:23<67:48:00,  6.34s/it]
 83%|████████▎ | 186473/225000 [351:39:27<60:30:20,  5.65s/it]
 83%|████████▎ | 186474/225000 [351:39:31<54:23:43,  5.08s/it]
 83%|████████▎ | 186475/225000 [351:39:40<68:05:28,  6.36s/it]
 83%|████████▎ | 186476/225000 [351:39:46<65:34:19,  6.13s/it]
 83%|████████▎ | 186477/225000 [351:39:52<65:06:58,  6.09s/it]
 83%|████████▎ | 186478/225000 [351:40:00<71:16:51,  6.66s/it]
 83%|████████▎ | 186479/225000 [351:40:06<69:44:57,  6.52s/it]
 83%|████████▎ | 186480/225000 [351:40:13<72:20:37,  6.76s/it]
 83%|████████▎ | 186481/225000 [351:40:22<79:40:58,  7.45s/it]
 83%|████████▎ | 186482/225000 [351:40:26<68:54:58,  6.44s/it]
 83%|████████▎ | 186483/225000 [351:40:31<63:49:16,  5.97s/it]
 83%|████████▎ | 186484/225000 [351:40:36<60:21:30,  5.64s/it]
 83%|████████▎ | 186485/225000 [351:40:41<58:16:47,  5.45s/it]
 83%|████████▎ | 186486/225000 [351:40:48<62:39:56,  5.86s/it]
 83%|████████▎ | 186487/225000 [351:40:57<72:10:35,  6.75s/it]
 83%|████████▎ | 186488/225000 [351:41:02<65:33:07,  6.13s/it]
 83%|████████▎ | 186489/225000 [351:41:06<61:30:52,  5.75s/it]
 83%|████████▎ | 186490/225000 [351:41:11<56:40:24,  5.30s/it]
 83%|████████▎ | 186491/225000 [351:41:16<56:30:17,  5.28s/it]
 83%|████████▎ | 186492/225000 [351:41:20<54:07:22,  5.06s/it]
 83%|████████▎ | 186493/225000 [351:41:27<59:07:17,  5.53s/it]
 83%|████████▎ | 186494/225000 [351:41:39<78:45:03,  7.36s/it]
 83%|████████▎ | 186495/225000 [351:41:47<82:36:22,  7.72s/it]
 83%|████████▎ | 186496/225000 [351:41:55<81:40:29,  7.64s/it]
 83%|████████▎ | 186497/225000 [351:42:00<73:57:29,  6.92s/it]
 83%|████████▎ | 186498/225000 [351:42:04<65:43:49,  6.15s/it]
 83%|████████▎ | 186499/225000 [351:42:13<74:32:52,  6.97s/it]
 83%|████████▎ | 186500/225000 [351:42:18<66:36:49,  6.23s/it]
                                                              
{'loss': '0.8029', 'grad_norm': '1.217', 'learning_rate': '3.87e-06', 'epoch': '0.8289', 'num_input_tokens_seen': 583501584, 'train_runtime': '1.266e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186500/225000 [351:42:18<66:36:49,  6.23s/it]
 83%|████████▎ | 186501/225000 [351:42:26<72:01:47,  6.74s/it]
 83%|████████▎ | 186502/225000 [351:42:29<60:55:24,  5.70s/it]
 83%|████████▎ | 186503/225000 [351:42:37<68:58:17,  6.45s/it]
 83%|████████▎ | 186504/225000 [351:42:45<72:25:34,  6.77s/it]
 83%|████████▎ | 186505/225000 [351:42:48<61:35:18,  5.76s/it]
 83%|████████▎ | 186506/225000 [351:42:55<65:15:55,  6.10s/it]
 83%|████████▎ | 186507/225000 [351:43:01<65:15:20,  6.10s/it]
 83%|████████▎ | 186508/225000 [351:43:06<63:06:58,  5.90s/it]
 83%|████████▎ | 186509/225000 [351:43:10<55:59:39,  5.24s/it]
 83%|████████▎ | 186510/225000 [351:43:18<63:22:31,  5.93s/it]
 83%|████████▎ | 186511/225000 [351:43:22<57:05:54,  5.34s/it]
 83%|████████▎ | 186512/225000 [351:43:26<53:14:46,  4.98s/it]
 83%|████████▎ | 186513/225000 [351:43:33<60:37:06,  5.67s/it]
 83%|████████▎ | 186514/225000 [351:43:42<69:45:29,  6.53s/it]
 83%|████████▎ | 186515/225000 [351:43:45<60:12:42,  5.63s/it]
 83%|████████▎ | 186516/225000 [351:43:49<53:58:31,  5.05s/it]
 83%|████████▎ | 186517/225000 [351:43:58<68:15:22,  6.39s/it]
 83%|████████▎ | 186518/225000 [351:44:02<59:44:13,  5.59s/it]
 83%|████████▎ | 186519/225000 [351:44:05<52:46:02,  4.94s/it]
 83%|████████▎ | 186520/225000 [351:44:11<53:23:20,  4.99s/it]
 83%|████████▎ | 186521/225000 [351:44:22<72:25:45,  6.78s/it]
 83%|████████▎ | 186522/225000 [351:44:25<63:28:47,  5.94s/it]
 83%|████████▎ | 186523/225000 [351:44:30<58:51:54,  5.51s/it]
 83%|████████▎ | 186524/225000 [351:44:38<65:46:38,  6.15s/it]
 83%|████████▎ | 186525/225000 [351:44:44<66:33:10,  6.23s/it]
 83%|████████▎ | 186526/225000 [351:44:51<67:27:25,  6.31s/it]
 83%|████████▎ | 186527/225000 [351:44:54<59:00:18,  5.52s/it]
 83%|████████▎ | 186528/225000 [351:45:03<70:47:51,  6.62s/it]
 83%|████████▎ | 186529/225000 [351:45:08<65:36:36,  6.14s/it]
 83%|████████▎ | 186530/225000 [351:45:15<67:43:54,  6.34s/it]
 83%|████████▎ | 186531/225000 [351:45:19<59:23:13,  5.56s/it]
 83%|████████▎ | 186532/225000 [351:45:24<56:11:04,  5.26s/it]
 83%|████████▎ | 186533/225000 [351:45:30<59:49:47,  5.60s/it]
 83%|████████▎ | 186534/225000 [351:45:34<54:02:44,  5.06s/it]
 83%|████████▎ | 186535/225000 [351:45:38<52:42:16,  4.93s/it]
 83%|████████▎ | 186536/225000 [351:45:43<53:09:24,  4.98s/it]
 83%|████████▎ | 186537/225000 [351:45:49<54:30:04,  5.10s/it]
 83%|████████▎ | 186538/225000 [351:45:54<55:54:54,  5.23s/it]
 83%|████████▎ | 186539/225000 [351:46:06<76:41:25,  7.18s/it]
 83%|████████▎ | 186540/225000 [351:46:13<74:14:16,  6.95s/it]
 83%|████████▎ | 186541/225000 [351:46:17<66:04:47,  6.19s/it]
 83%|████████▎ | 186542/225000 [351:46:22<61:01:33,  5.71s/it]
 83%|████████▎ | 186543/225000 [351:46:26<57:21:39,  5.37s/it]
 83%|████████▎ | 186544/225000 [351:46:36<72:19:50,  6.77s/it]
 83%|████████▎ | 186545/225000 [351:46:43<71:18:45,  6.68s/it]
 83%|████████▎ | 186546/225000 [351:46:46<61:17:36,  5.74s/it]
 83%|████████▎ | 186547/225000 [351:46:53<64:39:24,  6.05s/it]
 83%|████████▎ | 186548/225000 [351:47:00<66:19:32,  6.21s/it]
 83%|████████▎ | 186549/225000 [351:47:04<61:27:24,  5.75s/it]
 83%|████████▎ | 186550/225000 [351:47:08<55:33:55,  5.20s/it]
                                                              
{'loss': '0.8126', 'grad_norm': '1.352', 'learning_rate': '3.86e-06', 'epoch': '0.8291', 'num_input_tokens_seen': 583641136, 'train_runtime': '1.266e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186550/225000 [351:47:08<55:33:55,  5.20s/it]
 83%|████████▎ | 186551/225000 [351:47:13<53:15:35,  4.99s/it]
 83%|████████▎ | 186552/225000 [351:47:17<52:05:03,  4.88s/it]
 83%|████████▎ | 186553/225000 [351:47:21<49:22:21,  4.62s/it]
 83%|████████▎ | 186554/225000 [351:47:25<48:11:59,  4.51s/it]
 83%|████████▎ | 186555/225000 [351:47:34<61:59:39,  5.81s/it]
 83%|████████▎ | 186556/225000 [351:47:42<66:47:29,  6.25s/it]
 83%|████████▎ | 186557/225000 [351:47:45<56:45:12,  5.31s/it]
 83%|████████▎ | 186558/225000 [351:47:56<75:32:54,  7.07s/it]
 83%|████████▎ | 186559/225000 [351:48:04<78:25:33,  7.34s/it]
 83%|████████▎ | 186560/225000 [351:48:09<71:35:01,  6.70s/it]
 83%|████████▎ | 186561/225000 [351:48:16<72:33:40,  6.80s/it]
 83%|████████▎ | 186562/225000 [351:48:22<68:50:02,  6.45s/it]
 83%|████████▎ | 186563/225000 [351:48:27<65:32:42,  6.14s/it]
 83%|████████▎ | 186564/225000 [351:48:31<59:24:53,  5.56s/it]
 83%|████████▎ | 186565/225000 [351:48:36<57:41:43,  5.40s/it]
 83%|████████▎ | 186566/225000 [351:48:42<58:12:13,  5.45s/it]
 83%|████████▎ | 186567/225000 [351:48:51<69:28:52,  6.51s/it]
 83%|████████▎ | 186568/225000 [351:48:56<66:03:36,  6.19s/it]
 83%|████████▎ | 186569/225000 [351:49:01<60:00:47,  5.62s/it]
 83%|████████▎ | 186570/225000 [351:49:04<53:59:13,  5.06s/it]
 83%|████████▎ | 186571/225000 [351:49:14<69:55:40,  6.55s/it]
 83%|████████▎ | 186572/225000 [351:49:19<63:20:47,  5.93s/it]
 83%|████████▎ | 186573/225000 [351:49:23<57:53:32,  5.42s/it]
 83%|████████▎ | 186574/225000 [351:49:28<55:11:51,  5.17s/it]
 83%|████████▎ | 186575/225000 [351:49:43<86:44:09,  8.13s/it]
 83%|████████▎ | 186576/225000 [351:49:47<72:40:35,  6.81s/it]
 83%|████████▎ | 186577/225000 [351:49:51<63:48:14,  5.98s/it]
 83%|████████▎ | 186578/225000 [351:49:57<65:45:04,  6.16s/it]
 83%|████████▎ | 186579/225000 [351:50:02<60:19:23,  5.65s/it]
 83%|████████▎ | 186580/225000 [351:50:06<57:20:51,  5.37s/it]
 83%|████████▎ | 186581/225000 [351:50:13<61:48:16,  5.79s/it]
 83%|████████▎ | 186582/225000 [351:50:19<63:23:42,  5.94s/it]
 83%|████████▎ | 186583/225000 [351:50:30<79:06:48,  7.41s/it]
 83%|████████▎ | 186584/225000 [351:50:35<69:16:58,  6.49s/it]
 83%|████████▎ | 186585/225000 [351:50:42<72:25:18,  6.79s/it]
 83%|████████▎ | 186586/225000 [351:50:47<65:33:08,  6.14s/it]
 83%|████████▎ | 186587/225000 [351:50:51<59:15:24,  5.55s/it]
 83%|████████▎ | 186588/225000 [351:50:56<58:07:18,  5.45s/it]
 83%|████████▎ | 186589/225000 [351:51:01<57:34:10,  5.40s/it]
 83%|████████▎ | 186590/225000 [351:51:11<72:02:08,  6.75s/it]
 83%|████████▎ | 186591/225000 [351:51:15<63:00:05,  5.91s/it]
 83%|████████▎ | 186592/225000 [351:51:19<55:22:02,  5.19s/it]
 83%|████████▎ | 186593/225000 [351:51:27<65:35:17,  6.15s/it]
 83%|████████▎ | 186594/225000 [351:51:31<58:53:29,  5.52s/it]
 83%|████████▎ | 186595/225000 [351:51:36<56:42:52,  5.32s/it]
 83%|████████▎ | 186596/225000 [351:51:42<57:30:37,  5.39s/it]
 83%|████████▎ | 186597/225000 [351:51:46<54:17:32,  5.09s/it]
 83%|████████▎ | 186598/225000 [351:52:00<82:17:17,  7.71s/it]
 83%|████████▎ | 186599/225000 [351:52:03<68:44:13,  6.44s/it]
 83%|████████▎ | 186600/225000 [351:52:13<79:41:56,  7.47s/it]
                                                              
{'loss': '0.8372', 'grad_norm': '1.14', 'learning_rate': '3.85e-06', 'epoch': '0.8293', 'num_input_tokens_seen': 583784944, 'train_runtime': '1.267e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186600/225000 [351:52:13<79:41:56,  7.47s/it]
 83%|████████▎ | 186601/225000 [351:52:18<71:32:04,  6.71s/it]
 83%|████████▎ | 186602/225000 [351:52:21<60:08:43,  5.64s/it]
 83%|████████▎ | 186603/225000 [351:52:29<67:33:24,  6.33s/it]
 83%|████████▎ | 186604/225000 [351:52:34<61:11:52,  5.74s/it]
 83%|████████▎ | 186605/225000 [351:52:41<67:46:19,  6.35s/it]
 83%|████████▎ | 186606/225000 [351:52:50<73:52:47,  6.93s/it]
 83%|████████▎ | 186607/225000 [351:52:54<65:29:37,  6.14s/it]
 83%|████████▎ | 186608/225000 [351:52:58<58:58:20,  5.53s/it]
 83%|████████▎ | 186609/225000 [351:53:02<54:52:28,  5.15s/it]
 83%|████████▎ | 186610/225000 [351:53:06<51:13:33,  4.80s/it]
 83%|████████▎ | 186611/225000 [351:53:11<50:09:59,  4.70s/it]
 83%|████████▎ | 186612/225000 [351:53:20<65:26:16,  6.14s/it]
 83%|████████▎ | 186613/225000 [351:53:29<73:35:50,  6.90s/it]
 83%|████████▎ | 186614/225000 [351:53:33<64:03:05,  6.01s/it]
 83%|████████▎ | 186615/225000 [351:53:37<57:31:45,  5.40s/it]
 83%|████████▎ | 186616/225000 [351:53:41<54:15:03,  5.09s/it]
 83%|████████▎ | 186617/225000 [351:53:48<61:25:14,  5.76s/it]
 83%|████████▎ | 186618/225000 [351:53:56<66:43:22,  6.26s/it]
 83%|████████▎ | 186619/225000 [351:54:02<65:42:53,  6.16s/it]
 83%|████████▎ | 186620/225000 [351:54:07<62:13:57,  5.84s/it]
 83%|████████▎ | 186621/225000 [351:54:11<57:08:22,  5.36s/it]
 83%|████████▎ | 186622/225000 [351:54:16<55:53:10,  5.24s/it]
 83%|████████▎ | 186623/225000 [351:54:27<74:12:38,  6.96s/it]
 83%|████████▎ | 186624/225000 [351:54:36<79:33:33,  7.46s/it]
 83%|████████▎ | 186625/225000 [351:54:40<68:13:06,  6.40s/it]
 83%|████████▎ | 186626/225000 [351:54:49<77:07:21,  7.24s/it]
 83%|████████▎ | 186627/225000 [351:54:52<64:17:55,  6.03s/it]
 83%|████████▎ | 186628/225000 [351:55:01<72:47:02,  6.83s/it]
 83%|████████▎ | 186629/225000 [351:55:04<61:14:47,  5.75s/it]
 83%|████████▎ | 186630/225000 [351:55:08<56:16:20,  5.28s/it]
 83%|████████▎ | 186631/225000 [351:55:15<59:57:38,  5.63s/it]
 83%|████████▎ | 186632/225000 [351:55:19<56:57:26,  5.34s/it]
 83%|████████▎ | 186633/225000 [351:55:29<69:48:57,  6.55s/it]
 83%|████████▎ | 186634/225000 [351:55:34<66:02:55,  6.20s/it]
 83%|████████▎ | 186635/225000 [351:55:42<72:25:35,  6.80s/it]
 83%|████████▎ | 186636/225000 [351:55:48<70:17:39,  6.60s/it]
 83%|████████▎ | 186637/225000 [351:55:53<64:20:12,  6.04s/it]
 83%|████████▎ | 186638/225000 [351:56:02<72:04:39,  6.76s/it]
 83%|████████▎ | 186639/225000 [351:56:06<64:56:40,  6.09s/it]
 83%|████████▎ | 186640/225000 [351:56:10<58:47:30,  5.52s/it]
 83%|████████▎ | 186641/225000 [351:56:18<64:50:05,  6.08s/it]
 83%|████████▎ | 186642/225000 [351:56:22<59:12:29,  5.56s/it]
 83%|████████▎ | 186643/225000 [351:56:34<78:58:00,  7.41s/it]
 83%|████████▎ | 186644/225000 [351:56:39<72:10:13,  6.77s/it]
 83%|████████▎ | 186645/225000 [351:56:43<62:00:07,  5.82s/it]
 83%|████████▎ | 186646/225000 [351:56:51<69:08:23,  6.49s/it]
 83%|████████▎ | 186647/225000 [351:56:58<72:14:19,  6.78s/it]
 83%|████████▎ | 186648/225000 [351:57:06<74:15:49,  6.97s/it]
 83%|████████▎ | 186649/225000 [351:57:09<63:06:08,  5.92s/it]
 83%|████████▎ | 186650/225000 [351:57:14<61:01:04,  5.73s/it]
                                                              
{'loss': '0.8028', 'grad_norm': '1.217', 'learning_rate': '3.841e-06', 'epoch': '0.8296', 'num_input_tokens_seen': 583929056, 'train_runtime': '1.267e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186650/225000 [351:57:14<61:01:04,  5.73s/it]
 83%|████████▎ | 186651/225000 [351:57:21<65:14:19,  6.12s/it]
 83%|████████▎ | 186652/225000 [351:57:25<56:39:04,  5.32s/it]
 83%|████████▎ | 186653/225000 [351:57:33<66:16:43,  6.22s/it]
 83%|████████▎ | 186654/225000 [351:57:37<59:05:15,  5.55s/it]
 83%|████████▎ | 186655/225000 [351:57:48<75:21:21,  7.07s/it]
 83%|████████▎ | 186656/225000 [351:57:51<64:44:52,  6.08s/it]
 83%|████████▎ | 186657/225000 [351:57:58<66:31:33,  6.25s/it]
 83%|████████▎ | 186658/225000 [351:58:02<59:55:58,  5.63s/it]
 83%|████████▎ | 186659/225000 [351:58:16<85:29:25,  8.03s/it]
 83%|████████▎ | 186660/225000 [351:58:20<72:43:38,  6.83s/it]
 83%|████████▎ | 186661/225000 [351:58:24<63:34:43,  5.97s/it]
 83%|████████▎ | 186662/225000 [351:58:32<70:07:57,  6.59s/it]
 83%|████████▎ | 186663/225000 [351:58:37<64:02:47,  6.01s/it]
 83%|████████▎ | 186664/225000 [351:58:42<62:05:54,  5.83s/it]
 83%|████████▎ | 186665/225000 [351:58:48<61:27:54,  5.77s/it]
 83%|████████▎ | 186666/225000 [351:58:59<79:49:55,  7.50s/it]
 83%|████████▎ | 186667/225000 [351:59:04<69:52:09,  6.56s/it]
 83%|████████▎ | 186668/225000 [351:59:08<62:13:02,  5.84s/it]
 83%|████████▎ | 186669/225000 [351:59:16<68:29:26,  6.43s/it]
 83%|████████▎ | 186670/225000 [351:59:22<69:44:37,  6.55s/it]
 83%|████████▎ | 186671/225000 [351:59:29<69:20:01,  6.51s/it]
 83%|████████▎ | 186672/225000 [351:59:36<72:30:18,  6.81s/it]
 83%|████████▎ | 186673/225000 [351:59:45<77:03:19,  7.24s/it]
 83%|████████▎ | 186674/225000 [351:59:49<67:09:39,  6.31s/it]
 83%|████████▎ | 186675/225000 [351:59:53<60:48:39,  5.71s/it]
 83%|████████▎ | 186676/225000 [351:59:58<59:29:41,  5.59s/it]
 83%|████████▎ | 186677/225000 [352:00:03<56:28:22,  5.30s/it]
 83%|████████▎ | 186678/225000 [352:00:10<62:39:36,  5.89s/it]
 83%|████████▎ | 186679/225000 [352:00:18<69:13:47,  6.50s/it]
 83%|████████▎ | 186680/225000 [352:00:24<65:44:44,  6.18s/it]
 83%|████████▎ | 186681/225000 [352:00:33<77:42:04,  7.30s/it]
 83%|████████▎ | 186682/225000 [352:00:37<66:44:09,  6.27s/it]
 83%|████████▎ | 186683/225000 [352:00:43<63:56:15,  6.01s/it]
 83%|████████▎ | 186684/225000 [352:00:47<59:00:16,  5.54s/it]
 83%|████████▎ | 186685/225000 [352:00:52<56:43:51,  5.33s/it]
 83%|████████▎ | 186686/225000 [352:00:57<56:32:50,  5.31s/it]
 83%|████████▎ | 186687/225000 [352:01:02<56:07:33,  5.27s/it]
 83%|████████▎ | 186688/225000 [352:01:16<83:46:14,  7.87s/it]
 83%|████████▎ | 186689/225000 [352:01:27<93:31:09,  8.79s/it]
 83%|████████▎ | 186690/225000 [352:01:32<81:19:20,  7.64s/it]
 83%|████████▎ | 186691/225000 [352:01:39<79:35:25,  7.48s/it]
 83%|████████▎ | 186692/225000 [352:01:47<79:38:21,  7.48s/it]
 83%|████████▎ | 186693/225000 [352:01:53<76:08:52,  7.16s/it]
 83%|████████▎ | 186694/225000 [352:01:57<65:59:09,  6.20s/it]
 83%|████████▎ | 186695/225000 [352:02:02<60:59:15,  5.73s/it]
 83%|████████▎ | 186696/225000 [352:02:14<80:52:52,  7.60s/it]
 83%|████████▎ | 186697/225000 [352:02:18<71:28:53,  6.72s/it]
 83%|████████▎ | 186698/225000 [352:02:22<60:50:42,  5.72s/it]
 83%|████████▎ | 186699/225000 [352:02:26<55:33:43,  5.22s/it]
 83%|████████▎ | 186700/225000 [352:02:37<73:42:59,  6.93s/it]
                                                              
{'loss': '0.8454', 'grad_norm': '1.218', 'learning_rate': '3.831e-06', 'epoch': '0.8298', 'num_input_tokens_seen': 584081224, 'train_runtime': '1.267e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186700/225000 [352:02:37<73:42:59,  6.93s/it]
 83%|████████▎ | 186701/225000 [352:02:41<65:09:32,  6.12s/it]
 83%|████████▎ | 186702/225000 [352:02:52<81:53:03,  7.70s/it]
 83%|████████▎ | 186703/225000 [352:02:56<68:57:18,  6.48s/it]
 83%|████████▎ | 186704/225000 [352:03:01<65:01:18,  6.11s/it]
 83%|████████▎ | 186705/225000 [352:03:10<71:45:31,  6.75s/it]
 83%|████████▎ | 186706/225000 [352:03:13<60:36:23,  5.70s/it]
 83%|████████▎ | 186707/225000 [352:03:23<74:38:50,  7.02s/it]
 83%|████████▎ | 186708/225000 [352:03:27<66:11:55,  6.22s/it]
 83%|████████▎ | 186709/225000 [352:03:35<69:19:56,  6.52s/it]
 83%|████████▎ | 186710/225000 [352:03:41<67:44:00,  6.37s/it]
 83%|████████▎ | 186711/225000 [352:03:46<65:28:08,  6.16s/it]
 83%|████████▎ | 186712/225000 [352:03:54<69:11:56,  6.51s/it]
 83%|████████▎ | 186713/225000 [352:03:58<61:54:03,  5.82s/it]
 83%|████████▎ | 186714/225000 [352:04:05<67:35:58,  6.36s/it]
 83%|████████▎ | 186715/225000 [352:04:09<58:32:20,  5.50s/it]
 83%|████████▎ | 186716/225000 [352:04:14<55:57:57,  5.26s/it]
 83%|████████▎ | 186717/225000 [352:04:21<61:41:29,  5.80s/it]
 83%|████████▎ | 186718/225000 [352:04:25<56:26:06,  5.31s/it]
 83%|████████▎ | 186719/225000 [352:04:29<52:12:41,  4.91s/it]
 83%|████████▎ | 186720/225000 [352:04:33<51:27:25,  4.84s/it]
 83%|████████▎ | 186721/225000 [352:04:39<52:28:20,  4.93s/it]
 83%|████████▎ | 186722/225000 [352:04:48<67:50:02,  6.38s/it]
 83%|████████▎ | 186723/225000 [352:04:54<66:30:54,  6.26s/it]
 83%|████████▎ | 186724/225000 [352:04:58<57:51:03,  5.44s/it]
 83%|████████▎ | 186725/225000 [352:05:01<50:48:45,  4.78s/it]
 83%|████████▎ | 186726/225000 [352:05:06<50:42:25,  4.77s/it]
 83%|████████▎ | 186727/225000 [352:05:13<58:33:56,  5.51s/it]
 83%|████████▎ | 186728/225000 [352:05:20<63:07:21,  5.94s/it]
 83%|████████▎ | 186729/225000 [352:05:25<59:18:14,  5.58s/it]
 83%|████████▎ | 186730/225000 [352:05:30<58:00:43,  5.46s/it]
 83%|████████▎ | 186731/225000 [352:05:35<57:07:39,  5.37s/it]
 83%|████████▎ | 186732/225000 [352:05:41<58:15:47,  5.48s/it]
 83%|████████▎ | 186733/225000 [352:05:45<55:34:50,  5.23s/it]
 83%|████████▎ | 186734/225000 [352:05:57<76:00:42,  7.15s/it]
 83%|████████▎ | 186735/225000 [352:06:04<74:36:57,  7.02s/it]
 83%|████████▎ | 186736/225000 [352:06:14<85:03:13,  8.00s/it]
 83%|████████▎ | 186737/225000 [352:06:19<76:04:18,  7.16s/it]
 83%|████████▎ | 186738/225000 [352:06:25<71:48:48,  6.76s/it]
 83%|████████▎ | 186739/225000 [352:06:39<95:14:00,  8.96s/it]
 83%|████████▎ | 186740/225000 [352:06:43<79:50:28,  7.51s/it]
 83%|████████▎ | 186741/225000 [352:06:47<65:59:47,  6.21s/it]
 83%|████████▎ | 186742/225000 [352:06:50<57:55:29,  5.45s/it]
 83%|████████▎ | 186743/225000 [352:07:00<72:32:06,  6.83s/it]
 83%|████████▎ | 186744/225000 [352:07:03<60:42:39,  5.71s/it]
 83%|████████▎ | 186745/225000 [352:07:08<56:32:21,  5.32s/it]
 83%|████████▎ | 186746/225000 [352:07:12<53:45:47,  5.06s/it]
 83%|████████▎ | 186747/225000 [352:07:17<52:17:49,  4.92s/it]
 83%|████████▎ | 186748/225000 [352:07:21<49:49:09,  4.69s/it]
 83%|████████▎ | 186749/225000 [352:07:31<66:01:17,  6.21s/it]
 83%|████████▎ | 186750/225000 [352:07:36<63:02:23,  5.93s/it]
                                                              
{'loss': '0.8359', 'grad_norm': '1.173', 'learning_rate': '3.821e-06', 'epoch': '0.83', 'num_input_tokens_seen': 584228720, 'train_runtime': '1.268e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186750/225000 [352:07:36<63:02:23,  5.93s/it]
 83%|████████▎ | 186751/225000 [352:07:41<58:43:09,  5.53s/it]
 83%|████████▎ | 186752/225000 [352:07:46<57:16:52,  5.39s/it]
 83%|████████▎ | 186753/225000 [352:07:51<57:59:33,  5.46s/it]
 83%|████████▎ | 186754/225000 [352:08:05<83:35:03,  7.87s/it]
 83%|████████▎ | 186755/225000 [352:08:11<77:47:17,  7.32s/it]
 83%|████████▎ | 186756/225000 [352:08:19<81:38:30,  7.69s/it]
 83%|████████▎ | 186757/225000 [352:08:23<67:34:21,  6.36s/it]
 83%|████████▎ | 186758/225000 [352:08:27<61:18:00,  5.77s/it]
 83%|████████▎ | 186759/225000 [352:08:36<72:20:05,  6.81s/it]
 83%|████████▎ | 186760/225000 [352:08:39<60:52:42,  5.73s/it]
 83%|████████▎ | 186761/225000 [352:08:46<63:39:21,  5.99s/it]
 83%|████████▎ | 186762/225000 [352:08:53<65:20:29,  6.15s/it]
 83%|████████▎ | 186763/225000 [352:09:00<68:47:04,  6.48s/it]
 83%|████████▎ | 186764/225000 [352:09:04<59:59:45,  5.65s/it]
 83%|████████▎ | 186765/225000 [352:09:10<62:03:11,  5.84s/it]
 83%|████████▎ | 186766/225000 [352:09:19<72:43:37,  6.85s/it]
 83%|████████▎ | 186767/225000 [352:09:24<65:46:32,  6.19s/it]
 83%|████████▎ | 186768/225000 [352:09:34<78:26:53,  7.39s/it]
 83%|████████▎ | 186769/225000 [352:09:39<72:34:38,  6.83s/it]
 83%|████████▎ | 186770/225000 [352:09:44<66:40:47,  6.28s/it]
 83%|████████▎ | 186771/225000 [352:09:53<73:30:14,  6.92s/it]
 83%|████████▎ | 186772/225000 [352:10:00<74:16:31,  6.99s/it]
 83%|████████▎ | 186773/225000 [352:10:04<65:50:37,  6.20s/it]
 83%|████████▎ | 186774/225000 [352:10:15<78:41:40,  7.41s/it]
 83%|████████▎ | 186775/225000 [352:10:18<67:07:19,  6.32s/it]
 83%|████████▎ | 186776/225000 [352:10:29<81:15:16,  7.65s/it]
 83%|████████▎ | 186777/225000 [352:10:34<71:17:10,  6.71s/it]
 83%|████████▎ | 186778/225000 [352:10:38<62:53:17,  5.92s/it]
 83%|████████▎ | 186779/225000 [352:10:43<60:50:50,  5.73s/it]
 83%|████████▎ | 186780/225000 [352:10:48<58:12:19,  5.48s/it]
 83%|████████▎ | 186781/225000 [352:10:52<55:17:12,  5.21s/it]
 83%|████████▎ | 186782/225000 [352:10:57<52:52:22,  4.98s/it]
 83%|████████▎ | 186783/225000 [352:11:13<89:06:11,  8.39s/it]
 83%|████████▎ | 186784/225000 [352:11:17<73:36:17,  6.93s/it]
 83%|████████▎ | 186785/225000 [352:11:24<75:20:16,  7.10s/it]
 83%|████████▎ | 186786/225000 [352:11:29<68:13:03,  6.43s/it]
 83%|████████▎ | 186787/225000 [352:11:38<74:53:04,  7.05s/it]
 83%|████████▎ | 186788/225000 [352:11:43<70:39:30,  6.66s/it]
 83%|████████▎ | 186789/225000 [352:11:49<66:46:11,  6.29s/it]
 83%|████████▎ | 186790/225000 [352:11:54<64:13:38,  6.05s/it]
 83%|████████▎ | 186791/225000 [352:12:01<64:55:48,  6.12s/it]
 83%|████████▎ | 186792/225000 [352:12:06<62:07:34,  5.85s/it]
 83%|████████▎ | 186793/225000 [352:12:20<88:25:58,  8.33s/it]
 83%|████████▎ | 186794/225000 [352:12:24<74:07:23,  6.98s/it]
 83%|████████▎ | 186795/225000 [352:12:33<81:16:25,  7.66s/it]
 83%|████████▎ | 186796/225000 [352:12:36<67:20:36,  6.35s/it]
 83%|████████▎ | 186797/225000 [352:12:40<58:24:39,  5.50s/it]
 83%|████████▎ | 186798/225000 [352:12:48<66:40:38,  6.28s/it]
 83%|████████▎ | 186799/225000 [352:12:53<63:32:25,  5.99s/it]
 83%|████████▎ | 186800/225000 [352:12:56<53:21:29,  5.03s/it]
                                                              
{'loss': '0.8513', 'grad_norm': '1.286', 'learning_rate': '3.812e-06', 'epoch': '0.8302', 'num_input_tokens_seen': 584379416, 'train_runtime': '1.268e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186800/225000 [352:12:56<53:21:29,  5.03s/it]
 83%|████████▎ | 186801/225000 [352:13:01<52:23:57,  4.94s/it]
 83%|████████▎ | 186802/225000 [352:13:06<52:08:59,  4.91s/it]
 83%|████████▎ | 186803/225000 [352:13:10<51:25:38,  4.85s/it]
 83%|████████▎ | 186804/225000 [352:13:20<66:00:44,  6.22s/it]
 83%|████████▎ | 186805/225000 [352:13:25<62:55:16,  5.93s/it]
 83%|████████▎ | 186806/225000 [352:13:29<57:35:33,  5.43s/it]
 83%|████████▎ | 186807/225000 [352:13:33<53:29:27,  5.04s/it]
 83%|████████▎ | 186808/225000 [352:13:39<54:02:13,  5.09s/it]
 83%|████████▎ | 186809/225000 [352:13:47<64:57:14,  6.12s/it]
 83%|████████▎ | 186810/225000 [352:13:52<61:09:43,  5.77s/it]
 83%|████████▎ | 186811/225000 [352:13:57<59:16:10,  5.59s/it]
 83%|████████▎ | 186812/225000 [352:14:02<57:49:39,  5.45s/it]
 83%|████████▎ | 186813/225000 [352:14:08<58:07:14,  5.48s/it]
 83%|████████▎ | 186814/225000 [352:14:12<54:10:30,  5.11s/it]
 83%|████████▎ | 186815/225000 [352:14:24<77:04:45,  7.27s/it]
 83%|████████▎ | 186816/225000 [352:14:28<65:27:10,  6.17s/it]
 83%|████████▎ | 186817/225000 [352:14:32<59:48:03,  5.64s/it]
 83%|████████▎ | 186818/225000 [352:14:37<57:15:50,  5.40s/it]
 83%|████████▎ | 186819/225000 [352:14:46<69:19:54,  6.54s/it]
 83%|████████▎ | 186820/225000 [352:14:53<68:08:48,  6.43s/it]
 83%|████████▎ | 186821/225000 [352:14:56<59:56:36,  5.65s/it]
 83%|████████▎ | 186822/225000 [352:15:06<71:50:50,  6.77s/it]
 83%|████████▎ | 186823/225000 [352:15:11<65:57:51,  6.22s/it]
 83%|████████▎ | 186824/225000 [352:15:18<67:52:22,  6.40s/it]
 83%|████████▎ | 186825/225000 [352:15:29<85:01:41,  8.02s/it]
 83%|████████▎ | 186826/225000 [352:15:36<79:48:20,  7.53s/it]
 83%|████████▎ | 186827/225000 [352:15:51<102:36:58,  9.68s/it]
 83%|████████▎ | 186828/225000 [352:15:54<83:09:18,  7.84s/it] 
 83%|████████▎ | 186829/225000 [352:15:58<70:00:31,  6.60s/it]
 83%|████████▎ | 186830/225000 [352:16:05<73:31:02,  6.93s/it]
 83%|████████▎ | 186831/225000 [352:16:10<66:14:39,  6.25s/it]
 83%|████████▎ | 186832/225000 [352:16:13<56:14:59,  5.31s/it]
 83%|████████▎ | 186833/225000 [352:16:18<53:42:32,  5.07s/it]
 83%|████████▎ | 186834/225000 [352:16:24<55:56:27,  5.28s/it]
 83%|████████▎ | 186835/225000 [352:16:32<67:27:25,  6.36s/it]
 83%|████████▎ | 186836/225000 [352:16:36<57:32:12,  5.43s/it]
 83%|████████▎ | 186837/225000 [352:16:44<66:28:35,  6.27s/it]
 83%|████████▎ | 186838/225000 [352:16:50<64:52:28,  6.12s/it]
 83%|████████▎ | 186839/225000 [352:16:54<57:51:27,  5.46s/it]
 83%|████████▎ | 186840/225000 [352:16:59<57:55:45,  5.47s/it]
 83%|████████▎ | 186841/225000 [352:17:03<54:15:07,  5.12s/it]
 83%|████████▎ | 186842/225000 [352:17:08<52:59:18,  5.00s/it]
 83%|████████▎ | 186843/225000 [352:17:19<71:49:21,  6.78s/it]
 83%|████████▎ | 186844/225000 [352:17:24<65:52:21,  6.22s/it]
 83%|████████▎ | 186845/225000 [352:17:28<60:39:35,  5.72s/it]
 83%|████████▎ | 186846/225000 [352:17:33<57:12:33,  5.40s/it]
 83%|████████▎ | 186847/225000 [352:17:39<58:27:21,  5.52s/it]
 83%|████████▎ | 186848/225000 [352:17:44<55:54:23,  5.28s/it]
 83%|████████▎ | 186849/225000 [352:17:49<55:21:12,  5.22s/it]
 83%|████████▎ | 186850/225000 [352:17:58<67:29:14,  6.37s/it]
                                                              
{'loss': '0.8539', 'grad_norm': '1.163', 'learning_rate': '3.802e-06', 'epoch': '0.8304', 'num_input_tokens_seen': 584527008, 'train_runtime': '1.268e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186850/225000 [352:17:58<67:29:14,  6.37s/it]
 83%|████████▎ | 186851/225000 [352:18:03<63:30:17,  5.99s/it]
 83%|████████▎ | 186852/225000 [352:18:11<70:34:43,  6.66s/it]
 83%|████████▎ | 186853/225000 [352:18:16<65:58:50,  6.23s/it]
 83%|████████▎ | 186854/225000 [352:18:33<98:27:23,  9.29s/it]
 83%|████████▎ | 186855/225000 [352:18:36<80:15:15,  7.57s/it]
 83%|████████▎ | 186856/225000 [352:18:41<69:56:02,  6.60s/it]
 83%|████████▎ | 186857/225000 [352:18:46<65:26:06,  6.18s/it]
 83%|████████▎ | 186858/225000 [352:18:51<62:39:07,  5.91s/it]
 83%|████████▎ | 186859/225000 [352:18:59<69:48:05,  6.59s/it]
 83%|████████▎ | 186860/225000 [352:19:05<67:11:56,  6.34s/it]
 83%|████████▎ | 186861/225000 [352:19:11<65:44:15,  6.21s/it]
 83%|████████▎ | 186862/225000 [352:19:24<87:32:02,  8.26s/it]
 83%|████████▎ | 186863/225000 [352:19:27<69:47:08,  6.59s/it]
 83%|████████▎ | 186864/225000 [352:19:34<71:51:14,  6.78s/it]
 83%|████████▎ | 186865/225000 [352:19:40<69:04:42,  6.52s/it]
 83%|████████▎ | 186866/225000 [352:19:50<79:07:18,  7.47s/it]
 83%|████████▎ | 186867/225000 [352:19:57<77:57:03,  7.36s/it]
 83%|████████▎ | 186868/225000 [352:20:00<66:25:25,  6.27s/it]
 83%|████████▎ | 186869/225000 [352:20:09<75:08:31,  7.09s/it]
 83%|████████▎ | 186870/225000 [352:20:13<64:47:48,  6.12s/it]
 83%|████████▎ | 186871/225000 [352:20:21<69:29:15,  6.56s/it]
 83%|████████▎ | 186872/225000 [352:20:29<74:25:11,  7.03s/it]
 83%|████████▎ | 186873/225000 [352:20:33<64:51:46,  6.12s/it]
 83%|████████▎ | 186874/225000 [352:20:37<57:01:32,  5.38s/it]
 83%|████████▎ | 186875/225000 [352:20:47<71:53:39,  6.79s/it]
 83%|████████▎ | 186876/225000 [352:20:50<59:39:22,  5.63s/it]
 83%|████████▎ | 186877/225000 [352:20:56<62:43:08,  5.92s/it]
 83%|████████▎ | 186878/225000 [352:21:03<64:14:22,  6.07s/it]
 83%|████████▎ | 186879/225000 [352:21:06<55:14:30,  5.22s/it]
 83%|████████▎ | 186880/225000 [352:21:17<73:13:31,  6.92s/it]
 83%|████████▎ | 186881/225000 [352:21:20<62:10:43,  5.87s/it]
 83%|████████▎ | 186882/225000 [352:21:33<85:02:12,  8.03s/it]
 83%|████████▎ | 186883/225000 [352:21:40<81:47:31,  7.72s/it]
 83%|████████▎ | 186884/225000 [352:21:53<99:17:58,  9.38s/it]
 83%|████████▎ | 186885/225000 [352:21:57<81:42:23,  7.72s/it]
 83%|████████▎ | 186886/225000 [352:22:02<70:35:17,  6.67s/it]
 83%|████████▎ | 186887/225000 [352:22:06<65:01:30,  6.14s/it]
 83%|████████▎ | 186888/225000 [352:22:14<70:22:50,  6.65s/it]
 83%|████████▎ | 186889/225000 [352:22:23<77:29:06,  7.32s/it]
 83%|████████▎ | 186890/225000 [352:22:32<81:46:31,  7.72s/it]
 83%|████████▎ | 186891/225000 [352:22:38<76:00:57,  7.18s/it]
 83%|████████▎ | 186892/225000 [352:22:48<86:11:15,  8.14s/it]
 83%|████████▎ | 186893/225000 [352:22:57<87:08:27,  8.23s/it]
 83%|████████▎ | 186894/225000 [352:23:01<76:35:31,  7.24s/it]
 83%|████████▎ | 186895/225000 [352:23:12<87:35:37,  8.28s/it]
 83%|████████▎ | 186896/225000 [352:23:21<88:30:56,  8.36s/it]
 83%|████████▎ | 186897/225000 [352:23:29<86:39:01,  8.19s/it]
 83%|████████▎ | 186898/225000 [352:23:33<73:54:49,  6.98s/it]
 83%|████████▎ | 186899/225000 [352:23:37<66:06:41,  6.25s/it]
 83%|████████▎ | 186900/225000 [352:23:43<63:19:12,  5.98s/it]
                                                              
{'loss': '0.8548', 'grad_norm': '1.09', 'learning_rate': '3.792e-06', 'epoch': '0.8307', 'num_input_tokens_seen': 584687904, 'train_runtime': '1.269e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186900/225000 [352:23:43<63:19:12,  5.98s/it]
 83%|████████▎ | 186901/225000 [352:23:55<82:32:31,  7.80s/it]
 83%|████████▎ | 186902/225000 [352:23:59<70:26:06,  6.66s/it]
 83%|████████▎ | 186903/225000 [352:24:06<73:31:53,  6.95s/it]
 83%|████████▎ | 186904/225000 [352:24:10<62:52:32,  5.94s/it]
 83%|████████▎ | 186905/225000 [352:24:22<81:04:00,  7.66s/it]
 83%|████████▎ | 186906/225000 [352:24:25<66:33:13,  6.29s/it]
 83%|████████▎ | 186907/225000 [352:24:31<67:29:46,  6.38s/it]
 83%|████████▎ | 186908/225000 [352:24:37<64:07:57,  6.06s/it]
 83%|████████▎ | 186909/225000 [352:24:45<70:50:18,  6.69s/it]
 83%|████████▎ | 186910/225000 [352:24:55<81:11:00,  7.67s/it]
 83%|████████▎ | 186911/225000 [352:24:59<71:56:01,  6.80s/it]
 83%|████████▎ | 186912/225000 [352:25:05<68:41:01,  6.49s/it]
 83%|████████▎ | 186913/225000 [352:25:13<73:27:22,  6.94s/it]
 83%|████████▎ | 186914/225000 [352:25:16<61:15:38,  5.79s/it]
 83%|████████▎ | 186915/225000 [352:25:22<62:15:18,  5.88s/it]
 83%|████████▎ | 186916/225000 [352:25:35<84:35:42,  8.00s/it]
 83%|████████▎ | 186917/225000 [352:25:40<73:23:49,  6.94s/it]
 83%|████████▎ | 186918/225000 [352:25:51<86:51:19,  8.21s/it]
 83%|████████▎ | 186919/225000 [352:25:54<71:54:22,  6.80s/it]
 83%|████████▎ | 186920/225000 [352:26:01<71:14:01,  6.73s/it]
 83%|████████▎ | 186921/225000 [352:26:12<83:33:44,  7.90s/it]
 83%|████████▎ | 186922/225000 [352:26:20<85:27:12,  8.08s/it]
 83%|████████▎ | 186923/225000 [352:26:28<83:07:14,  7.86s/it]
 83%|████████▎ | 186924/225000 [352:26:31<70:39:23,  6.68s/it]
 83%|████████▎ | 186925/225000 [352:26:38<69:16:32,  6.55s/it]
 83%|████████▎ | 186926/225000 [352:26:46<75:24:17,  7.13s/it]
 83%|████████▎ | 186927/225000 [352:26:56<84:10:03,  7.96s/it]
 83%|████████▎ | 186928/225000 [352:27:00<72:54:35,  6.89s/it]
 83%|████████▎ | 186929/225000 [352:27:07<70:54:56,  6.71s/it]
 83%|████████▎ | 186930/225000 [352:27:24<104:46:15,  9.91s/it]
 83%|████████▎ | 186931/225000 [352:27:27<83:24:15,  7.89s/it] 
 83%|████████▎ | 186932/225000 [352:27:35<84:14:50,  7.97s/it]
 83%|████████▎ | 186933/225000 [352:27:42<78:25:24,  7.42s/it]
 83%|████████▎ | 186934/225000 [352:27:46<67:52:22,  6.42s/it]
 83%|████████▎ | 186935/225000 [352:27:50<62:36:59,  5.92s/it]
 83%|████████▎ | 186936/225000 [352:28:01<77:08:48,  7.30s/it]
 83%|████████▎ | 186937/225000 [352:28:04<63:50:24,  6.04s/it]
 83%|████████▎ | 186938/225000 [352:28:17<85:55:53,  8.13s/it]
 83%|████████▎ | 186939/225000 [352:28:20<70:58:26,  6.71s/it]
 83%|████████▎ | 186940/225000 [352:28:27<71:01:05,  6.72s/it]
 83%|████████▎ | 186941/225000 [352:28:39<86:06:08,  8.14s/it]
 83%|████████▎ | 186942/225000 [352:28:44<76:04:07,  7.20s/it]
 83%|████████▎ | 186943/225000 [352:28:48<66:07:28,  6.26s/it]
 83%|████████▎ | 186944/225000 [352:28:52<59:39:09,  5.64s/it]
 83%|████████▎ | 186945/225000 [352:29:02<73:10:27,  6.92s/it]
 83%|████████▎ | 186946/225000 [352:29:06<65:25:13,  6.19s/it]
 83%|████████▎ | 186947/225000 [352:29:16<75:11:00,  7.11s/it]
 83%|████████▎ | 186948/225000 [352:29:22<72:49:20,  6.89s/it]
 83%|████████▎ | 186949/225000 [352:29:26<64:13:43,  6.08s/it]
 83%|████████▎ | 186950/225000 [352:29:30<57:46:16,  5.47s/it]
                                                              
{'loss': '0.8569', 'grad_norm': '1.347', 'learning_rate': '3.783e-06', 'epoch': '0.8309', 'num_input_tokens_seen': 584858176, 'train_runtime': '1.269e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 186950/225000 [352:29:30<57:46:16,  5.47s/it]
 83%|████████▎ | 186951/225000 [352:29:34<53:40:28,  5.08s/it]
 83%|████████▎ | 186952/225000 [352:29:41<57:55:26,  5.48s/it]
 83%|████████▎ | 186953/225000 [352:29:48<64:21:19,  6.09s/it]
 83%|████████▎ | 186954/225000 [352:29:55<65:26:53,  6.19s/it]
 83%|████████▎ | 186955/225000 [352:29:58<57:17:57,  5.42s/it]
 83%|████████▎ | 186956/225000 [352:30:09<73:42:28,  6.97s/it]
 83%|████████▎ | 186957/225000 [352:30:16<72:57:15,  6.90s/it]
 83%|████████▎ | 186958/225000 [352:30:24<76:10:24,  7.21s/it]
 83%|████████▎ | 186959/225000 [352:30:27<64:03:10,  6.06s/it]
 83%|████████▎ | 186960/225000 [352:30:31<59:03:46,  5.59s/it]
 83%|████████▎ | 186961/225000 [352:30:38<60:55:37,  5.77s/it]
 83%|████████▎ | 186962/225000 [352:30:46<70:09:15,  6.64s/it]
 83%|████████▎ | 186963/225000 [352:30:50<60:42:23,  5.75s/it]
 83%|████████▎ | 186964/225000 [352:30:59<70:28:28,  6.67s/it]
 83%|████████▎ | 186965/225000 [352:31:03<61:32:00,  5.82s/it]
 83%|████████▎ | 186966/225000 [352:31:12<71:45:07,  6.79s/it]
 83%|████████▎ | 186967/225000 [352:31:17<66:22:21,  6.28s/it]
 83%|████████▎ | 186968/225000 [352:31:24<69:46:53,  6.61s/it]
 83%|████████▎ | 186969/225000 [352:31:29<63:24:58,  6.00s/it]
 83%|████████▎ | 186970/225000 [352:31:37<69:41:47,  6.60s/it]
 83%|████████▎ | 186971/225000 [352:31:41<61:47:28,  5.85s/it]
 83%|████████▎ | 186972/225000 [352:31:49<68:29:40,  6.48s/it]
 83%|████████▎ | 186973/225000 [352:31:54<65:54:21,  6.24s/it]
 83%|████████▎ | 186974/225000 [352:32:01<66:23:39,  6.29s/it]
 83%|████████▎ | 186975/225000 [352:32:06<62:14:17,  5.89s/it]
 83%|████████▎ | 186976/225000 [352:32:11<60:58:17,  5.77s/it]
 83%|████████▎ | 186977/225000 [352:32:16<57:37:42,  5.46s/it]
 83%|████████▎ | 186978/225000 [352:32:26<71:18:25,  6.75s/it]
 83%|████████▎ | 186979/225000 [352:32:33<74:00:45,  7.01s/it]
 83%|████████▎ | 186980/225000 [352:32:38<66:31:24,  6.30s/it]
 83%|████████▎ | 186981/225000 [352:32:42<60:30:55,  5.73s/it]
 83%|████████▎ | 186982/225000 [352:32:47<56:06:54,  5.31s/it]
 83%|████████▎ | 186983/225000 [352:32:55<65:44:17,  6.23s/it]
 83%|████████▎ | 186984/225000 [352:33:02<68:44:44,  6.51s/it]
 83%|████████▎ | 186985/225000 [352:33:08<65:21:46,  6.19s/it]
 83%|████████▎ | 186986/225000 [352:33:17<74:38:08,  7.07s/it]
 83%|████████▎ | 186987/225000 [352:33:22<68:54:44,  6.53s/it]
 83%|████████▎ | 186988/225000 [352:33:26<60:28:55,  5.73s/it]
 83%|████████▎ | 186989/225000 [352:33:35<70:28:41,  6.67s/it]
 83%|████████▎ | 186990/225000 [352:33:46<83:58:58,  7.95s/it]
 83%|████████▎ | 186991/225000 [352:33:50<71:04:41,  6.73s/it]
 83%|████████▎ | 186992/225000 [352:33:56<69:20:50,  6.57s/it]
 83%|████████▎ | 186993/225000 [352:34:02<66:32:25,  6.30s/it]
 83%|████████▎ | 186994/225000 [352:34:05<58:30:03,  5.54s/it]
 83%|████████▎ | 186995/225000 [352:34:12<63:08:54,  5.98s/it]
 83%|████████▎ | 186996/225000 [352:34:20<66:51:12,  6.33s/it]
 83%|████████▎ | 186997/225000 [352:34:32<86:05:17,  8.16s/it]
 83%|████████▎ | 186998/225000 [352:34:36<74:02:14,  7.01s/it]
 83%|████████▎ | 186999/225000 [352:34:43<72:21:57,  6.86s/it]
 83%|████████▎ | 187000/225000 [352:34:49<70:43:22,  6.70s/it]
                                                              
{'loss': '0.8267', 'grad_norm': '1.007', 'learning_rate': '3.773e-06', 'epoch': '0.8311', 'num_input_tokens_seen': 585016576, 'train_runtime': '1.269e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187000/225000 [352:34:49<70:43:22,  6.70s/it]
 83%|████████▎ | 187001/225000 [352:34:55<69:22:37,  6.57s/it]
 83%|████████▎ | 187002/225000 [352:35:00<61:43:14,  5.85s/it]
 83%|████████▎ | 187003/225000 [352:35:09<74:18:06,  7.04s/it]
 83%|████████▎ | 187004/225000 [352:35:17<77:41:34,  7.36s/it]
 83%|████████▎ | 187005/225000 [352:35:21<65:55:01,  6.25s/it]
 83%|████████▎ | 187006/225000 [352:35:32<81:34:03,  7.73s/it]
 83%|████████▎ | 187007/225000 [352:35:37<70:27:41,  6.68s/it]
 83%|████████▎ | 187008/225000 [352:35:50<93:17:50,  8.84s/it]
 83%|████████▎ | 187009/225000 [352:35:55<80:32:57,  7.63s/it]
 83%|████████▎ | 187010/225000 [352:36:03<80:01:45,  7.58s/it]
 83%|████████▎ | 187011/225000 [352:36:14<91:55:22,  8.71s/it]
 83%|████████▎ | 187012/225000 [352:36:17<74:19:42,  7.04s/it]
 83%|████████▎ | 187013/225000 [352:36:27<83:18:16,  7.89s/it]
 83%|████████▎ | 187014/225000 [352:36:31<70:00:43,  6.64s/it]
 83%|████████▎ | 187015/225000 [352:36:35<63:05:49,  5.98s/it]
 83%|████████▎ | 187016/225000 [352:36:41<61:14:32,  5.80s/it]
 83%|████████▎ | 187017/225000 [352:36:45<57:13:45,  5.42s/it]
 83%|████████▎ | 187018/225000 [352:36:50<54:25:05,  5.16s/it]
 83%|████████▎ | 187019/225000 [352:37:00<71:34:23,  6.78s/it]
 83%|████████▎ | 187020/225000 [352:37:05<65:28:28,  6.21s/it]
 83%|████████▎ | 187021/225000 [352:37:10<60:25:19,  5.73s/it]
 83%|████████▎ | 187022/225000 [352:37:14<56:55:43,  5.40s/it]
 83%|████████▎ | 187023/225000 [352:37:19<55:59:34,  5.31s/it]
 83%|████████▎ | 187024/225000 [352:37:24<52:34:01,  4.98s/it]
 83%|████████▎ | 187025/225000 [352:37:38<82:36:06,  7.83s/it]
 83%|████████▎ | 187026/225000 [352:37:43<72:40:12,  6.89s/it]
 83%|████████▎ | 187027/225000 [352:37:46<61:24:21,  5.82s/it]
 83%|████████▎ | 187028/225000 [352:37:54<66:57:32,  6.35s/it]
 83%|████████▎ | 187029/225000 [352:38:06<87:10:41,  8.27s/it]
 83%|████████▎ | 187030/225000 [352:38:10<72:38:41,  6.89s/it]
 83%|████████▎ | 187031/225000 [352:38:17<72:29:41,  6.87s/it]
 83%|████████▎ | 187032/225000 [352:38:21<64:53:18,  6.15s/it]
 83%|████████▎ | 187033/225000 [352:38:30<71:19:08,  6.76s/it]
 83%|████████▎ | 187034/225000 [352:38:34<62:40:11,  5.94s/it]
 83%|████████▎ | 187035/225000 [352:38:44<77:39:34,  7.36s/it]
 83%|████████▎ | 187036/225000 [352:38:49<67:51:49,  6.44s/it]
 83%|████████▎ | 187037/225000 [352:38:55<67:15:25,  6.38s/it]
 83%|████████▎ | 187038/225000 [352:38:59<61:38:00,  5.84s/it]
 83%|████████▎ | 187039/225000 [352:39:04<57:51:31,  5.49s/it]
 83%|████████▎ | 187040/225000 [352:39:14<71:22:00,  6.77s/it]
 83%|████████▎ | 187041/225000 [352:39:18<64:01:50,  6.07s/it]
 83%|████████▎ | 187042/225000 [352:39:28<73:59:22,  7.02s/it]
 83%|████████▎ | 187043/225000 [352:39:33<69:05:08,  6.55s/it]
 83%|████████▎ | 187044/225000 [352:39:41<74:33:28,  7.07s/it]
 83%|████████▎ | 187045/225000 [352:39:47<69:00:44,  6.55s/it]
 83%|████████▎ | 187046/225000 [352:39:50<59:53:55,  5.68s/it]
 83%|████████▎ | 187047/225000 [352:39:54<54:16:32,  5.15s/it]
 83%|████████▎ | 187048/225000 [352:40:05<73:13:25,  6.95s/it]
 83%|████████▎ | 187049/225000 [352:40:12<72:35:56,  6.89s/it]
 83%|████████▎ | 187050/225000 [352:40:23<84:09:31,  7.98s/it]
                                                              
{'loss': '0.8544', 'grad_norm': '0.98', 'learning_rate': '3.763e-06', 'epoch': '0.8313', 'num_input_tokens_seen': 585180664, 'train_runtime': '1.27e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187050/225000 [352:40:23<84:09:31,  7.98s/it]
 83%|████████▎ | 187051/225000 [352:40:30<82:44:12,  7.85s/it]
 83%|████████▎ | 187052/225000 [352:40:34<69:37:27,  6.61s/it]
 83%|████████▎ | 187053/225000 [352:40:38<61:26:35,  5.83s/it]
 83%|████████▎ | 187054/225000 [352:40:45<65:14:25,  6.19s/it]
 83%|████████▎ | 187055/225000 [352:40:55<77:20:48,  7.34s/it]
 83%|████████▎ | 187056/225000 [352:41:01<73:44:38,  7.00s/it]
 83%|████████▎ | 187057/225000 [352:41:06<66:13:57,  6.28s/it]
 83%|████████▎ | 187058/225000 [352:41:10<60:10:47,  5.71s/it]
 83%|████████▎ | 187059/225000 [352:41:17<65:07:52,  6.18s/it]
 83%|████████▎ | 187060/225000 [352:41:30<86:22:43,  8.20s/it]
 83%|████████▎ | 187061/225000 [352:41:35<74:53:24,  7.11s/it]
 83%|████████▎ | 187062/225000 [352:41:39<65:10:37,  6.18s/it]
 83%|████████▎ | 187063/225000 [352:41:44<61:20:46,  5.82s/it]
 83%|████████▎ | 187064/225000 [352:41:52<68:46:25,  6.53s/it]
 83%|████████▎ | 187065/225000 [352:42:00<74:07:57,  7.04s/it]
 83%|████████▎ | 187066/225000 [352:42:11<86:15:29,  8.19s/it]
 83%|████████▎ | 187067/225000 [352:42:20<87:32:56,  8.31s/it]
 83%|████████▎ | 187068/225000 [352:42:24<73:35:50,  6.98s/it]
 83%|████████▎ | 187069/225000 [352:42:30<72:42:03,  6.90s/it]
 83%|████████▎ | 187070/225000 [352:42:35<65:25:26,  6.21s/it]
 83%|████████▎ | 187071/225000 [352:42:40<60:27:16,  5.74s/it]
 83%|████████▎ | 187072/225000 [352:42:45<60:37:17,  5.75s/it]
 83%|████████▎ | 187073/225000 [352:42:50<57:11:21,  5.43s/it]
 83%|████████▎ | 187074/225000 [352:43:03<80:50:22,  7.67s/it]
 83%|████████▎ | 187075/225000 [352:43:07<70:48:26,  6.72s/it]
 83%|████████▎ | 187076/225000 [352:43:20<88:38:41,  8.41s/it]
 83%|████████▎ | 187077/225000 [352:43:24<74:58:25,  7.12s/it]
 83%|████████▎ | 187078/225000 [352:43:30<72:32:36,  6.89s/it]
 83%|████████▎ | 187079/225000 [352:43:40<80:34:30,  7.65s/it]
 83%|████████▎ | 187080/225000 [352:43:44<69:59:04,  6.64s/it]
 83%|████████▎ | 187081/225000 [352:43:53<77:00:53,  7.31s/it]
 83%|████████▎ | 187082/225000 [352:43:56<64:42:39,  6.14s/it]
 83%|████████▎ | 187083/225000 [352:44:01<61:34:21,  5.85s/it]
 83%|████████▎ | 187084/225000 [352:44:08<64:48:01,  6.15s/it]
 83%|████████▎ | 187085/225000 [352:44:11<55:26:24,  5.26s/it]
 83%|████████▎ | 187086/225000 [352:44:19<61:36:05,  5.85s/it]
 83%|████████▎ | 187087/225000 [352:44:24<60:25:56,  5.74s/it]
 83%|████████▎ | 187088/225000 [352:44:30<59:59:04,  5.70s/it]
 83%|████████▎ | 187089/225000 [352:44:34<56:03:32,  5.32s/it]
 83%|████████▎ | 187090/225000 [352:44:38<52:37:26,  5.00s/it]
 83%|████████▎ | 187091/225000 [352:44:43<50:41:38,  4.81s/it]
 83%|████████▎ | 187092/225000 [352:44:49<53:37:36,  5.09s/it]
 83%|████████▎ | 187093/225000 [352:45:00<72:15:25,  6.86s/it]
 83%|████████▎ | 187094/225000 [352:45:07<72:42:03,  6.90s/it]
 83%|████████▎ | 187095/225000 [352:45:11<64:37:13,  6.14s/it]
 83%|████████▎ | 187096/225000 [352:45:25<88:06:59,  8.37s/it]
 83%|████████▎ | 187097/225000 [352:45:33<87:12:11,  8.28s/it]
 83%|████████▎ | 187098/225000 [352:45:46<102:21:09,  9.72s/it]
 83%|████████▎ | 187099/225000 [352:45:52<92:10:43,  8.76s/it] 
 83%|████████▎ | 187100/225000 [352:46:03<99:06:33,  9.41s/it]
                                                              
{'loss': '0.8748', 'grad_norm': '1.22', 'learning_rate': '3.754e-06', 'epoch': '0.8316', 'num_input_tokens_seen': 585343720, 'train_runtime': '1.27e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187100/225000 [352:46:03<99:06:33,  9.41s/it]
 83%|████████▎ | 187101/225000 [352:46:07<82:48:38,  7.87s/it]
 83%|████████▎ | 187102/225000 [352:46:13<74:41:26,  7.10s/it]
 83%|████████▎ | 187103/225000 [352:46:23<84:03:10,  7.98s/it]
 83%|████████▎ | 187104/225000 [352:46:26<69:51:14,  6.64s/it]
 83%|████████▎ | 187105/225000 [352:46:34<73:19:13,  6.97s/it]
 83%|████████▎ | 187106/225000 [352:46:46<88:41:28,  8.43s/it]
 83%|████████▎ | 187107/225000 [352:46:50<74:36:27,  7.09s/it]
 83%|████████▎ | 187108/225000 [352:46:54<66:29:29,  6.32s/it]
 83%|████████▎ | 187109/225000 [352:46:58<59:18:37,  5.64s/it]
 83%|████████▎ | 187110/225000 [352:47:04<58:27:21,  5.55s/it]
 83%|████████▎ | 187111/225000 [352:47:12<66:15:28,  6.30s/it]
 83%|████████▎ | 187112/225000 [352:47:17<64:17:59,  6.11s/it]
 83%|████████▎ | 187113/225000 [352:47:26<70:47:54,  6.73s/it]
 83%|████████▎ | 187114/225000 [352:47:32<68:27:16,  6.50s/it]
 83%|████████▎ | 187115/225000 [352:47:42<79:30:30,  7.56s/it]
 83%|████████▎ | 187116/225000 [352:47:45<65:41:58,  6.24s/it]
 83%|████████▎ | 187117/225000 [352:47:49<58:45:14,  5.58s/it]
 83%|████████▎ | 187118/225000 [352:47:53<55:01:34,  5.23s/it]
 83%|████████▎ | 187119/225000 [352:48:03<69:39:47,  6.62s/it]
 83%|████████▎ | 187120/225000 [352:48:10<69:18:23,  6.59s/it]
 83%|████████▎ | 187121/225000 [352:48:21<85:15:23,  8.10s/it]
 83%|████████▎ | 187122/225000 [352:48:26<73:25:59,  6.98s/it]
 83%|████████▎ | 187123/225000 [352:48:37<87:24:26,  8.31s/it]
 83%|████████▎ | 187124/225000 [352:48:41<72:32:42,  6.90s/it]
 83%|████████▎ | 187125/225000 [352:48:45<65:16:29,  6.20s/it]
 83%|████████▎ | 187126/225000 [352:48:52<67:46:02,  6.44s/it]
 83%|████████▎ | 187127/225000 [352:48:56<58:57:52,  5.60s/it]
 83%|████████▎ | 187128/225000 [352:49:08<79:04:28,  7.52s/it]
 83%|████████▎ | 187129/225000 [352:49:11<66:14:32,  6.30s/it]
 83%|████████▎ | 187130/225000 [352:49:22<80:48:44,  7.68s/it]
 83%|████████▎ | 187131/225000 [352:49:28<73:36:12,  7.00s/it]
 83%|████████▎ | 187132/225000 [352:49:34<71:31:16,  6.80s/it]
 83%|████████▎ | 187133/225000 [352:49:38<63:56:00,  6.08s/it]
 83%|████████▎ | 187134/225000 [352:49:43<60:19:35,  5.74s/it]
 83%|████████▎ | 187135/225000 [352:49:49<62:04:32,  5.90s/it]
 83%|████████▎ | 187136/225000 [352:49:55<59:53:31,  5.69s/it]
 83%|████████▎ | 187137/225000 [352:50:07<80:17:41,  7.63s/it]
 83%|████████▎ | 187138/225000 [352:50:14<78:07:40,  7.43s/it]
 83%|████████▎ | 187139/225000 [352:50:23<84:40:23,  8.05s/it]
 83%|████████▎ | 187140/225000 [352:50:27<69:44:01,  6.63s/it]
 83%|████████▎ | 187141/225000 [352:50:31<61:59:26,  5.89s/it]
 83%|████████▎ | 187142/225000 [352:50:40<72:49:44,  6.93s/it]
 83%|████████▎ | 187143/225000 [352:50:54<93:42:04,  8.91s/it]
 83%|████████▎ | 187144/225000 [352:50:58<80:12:35,  7.63s/it]
 83%|████████▎ | 187145/225000 [352:51:08<87:17:59,  8.30s/it]
 83%|████████▎ | 187146/225000 [352:51:13<77:27:14,  7.37s/it]
 83%|████████▎ | 187147/225000 [352:51:20<76:08:43,  7.24s/it]
 83%|████████▎ | 187148/225000 [352:51:24<66:08:11,  6.29s/it]
 83%|████████▎ | 187149/225000 [352:51:33<73:47:17,  7.02s/it]
 83%|████████▎ | 187150/225000 [352:51:46<93:06:46,  8.86s/it]
                                                              
{'loss': '0.8894', 'grad_norm': '1.364', 'learning_rate': '3.744e-06', 'epoch': '0.8318', 'num_input_tokens_seen': 585513896, 'train_runtime': '1.27e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187150/225000 [352:51:46<93:06:46,  8.86s/it]
 83%|████████▎ | 187151/225000 [352:51:56<96:07:05,  9.14s/it]
 83%|████████▎ | 187152/225000 [352:52:04<92:07:18,  8.76s/it]
 83%|████████▎ | 187153/225000 [352:52:09<79:10:41,  7.53s/it]
 83%|████████▎ | 187154/225000 [352:52:14<73:49:50,  7.02s/it]
 83%|████████▎ | 187155/225000 [352:52:21<71:31:55,  6.80s/it]
 83%|████████▎ | 187156/225000 [352:52:25<63:49:03,  6.07s/it]
 83%|████████▎ | 187157/225000 [352:52:34<73:02:16,  6.95s/it]
 83%|████████▎ | 187158/225000 [352:52:43<79:28:14,  7.56s/it]
 83%|████████▎ | 187159/225000 [352:52:49<73:42:46,  7.01s/it]
 83%|████████▎ | 187160/225000 [352:52:55<72:21:07,  6.88s/it]
 83%|████████▎ | 187161/225000 [352:53:01<68:40:19,  6.53s/it]
 83%|████████▎ | 187162/225000 [352:53:10<75:31:03,  7.18s/it]
 83%|████████▎ | 187163/225000 [352:53:19<81:57:53,  7.80s/it]
 83%|████████▎ | 187164/225000 [352:53:24<73:28:43,  6.99s/it]
 83%|████████▎ | 187165/225000 [352:53:32<76:52:19,  7.31s/it]
 83%|████████▎ | 187166/225000 [352:53:39<73:56:08,  7.04s/it]
 83%|████████▎ | 187167/225000 [352:53:43<66:04:00,  6.29s/it]
 83%|████████▎ | 187168/225000 [352:53:48<60:54:45,  5.80s/it]
 83%|████████▎ | 187169/225000 [352:53:53<59:07:33,  5.63s/it]
 83%|████████▎ | 187170/225000 [352:53:57<54:20:14,  5.17s/it]
 83%|████████▎ | 187171/225000 [352:54:06<64:36:29,  6.15s/it]
 83%|████████▎ | 187172/225000 [352:54:16<79:28:43,  7.56s/it]
 83%|████████▎ | 187173/225000 [352:54:21<68:57:00,  6.56s/it]
 83%|████████▎ | 187174/225000 [352:54:28<70:42:15,  6.73s/it]
 83%|████████▎ | 187175/225000 [352:54:32<61:48:23,  5.88s/it]
 83%|████████▎ | 187176/225000 [352:54:39<67:34:55,  6.43s/it]
 83%|████████▎ | 187177/225000 [352:54:48<73:30:55,  7.00s/it]
 83%|████████▎ | 187178/225000 [352:54:57<80:24:34,  7.65s/it]
 83%|████████▎ | 187179/225000 [352:55:02<70:57:17,  6.75s/it]
 83%|████████▎ | 187180/225000 [352:55:10<74:49:33,  7.12s/it]
 83%|████████▎ | 187181/225000 [352:55:13<64:14:59,  6.12s/it]
 83%|████████▎ | 187182/225000 [352:55:18<58:27:33,  5.56s/it]
 83%|████████▎ | 187183/225000 [352:55:23<57:30:07,  5.47s/it]
 83%|████████▎ | 187184/225000 [352:55:30<63:59:30,  6.09s/it]
 83%|████████▎ | 187185/225000 [352:55:39<73:15:20,  6.97s/it]
 83%|████████▎ | 187186/225000 [352:55:49<82:41:12,  7.87s/it]
 83%|████████▎ | 187187/225000 [352:55:56<77:20:16,  7.36s/it]
 83%|████████▎ | 187188/225000 [352:56:01<72:00:34,  6.86s/it]
 83%|████████▎ | 187189/225000 [352:56:10<76:38:51,  7.30s/it]
 83%|████████▎ | 187190/225000 [352:56:15<70:03:15,  6.67s/it]
 83%|████████▎ | 187191/225000 [352:56:26<85:06:20,  8.10s/it]
 83%|████████▎ | 187192/225000 [352:56:30<71:21:05,  6.79s/it]
 83%|████████▎ | 187193/225000 [352:56:43<89:51:49,  8.56s/it]
 83%|████████▎ | 187194/225000 [352:56:49<82:34:39,  7.86s/it]
 83%|████████▎ | 187195/225000 [352:56:54<75:27:54,  7.19s/it]
 83%|████████▎ | 187196/225000 [352:57:00<70:22:56,  6.70s/it]
 83%|████████▎ | 187197/225000 [352:57:08<74:58:44,  7.14s/it]
 83%|████████▎ | 187198/225000 [352:57:14<70:45:48,  6.74s/it]
 83%|████████▎ | 187199/225000 [352:57:26<88:13:01,  8.40s/it]
 83%|████████▎ | 187200/225000 [352:57:30<73:21:25,  6.99s/it]
                                                              
{'loss': '0.8953', 'grad_norm': '1.067', 'learning_rate': '3.734e-06', 'epoch': '0.832', 'num_input_tokens_seen': 585681376, 'train_runtime': '1.271e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187200/225000 [352:57:30<73:21:25,  6.99s/it]
 83%|████████▎ | 187201/225000 [352:57:39<79:28:53,  7.57s/it]
 83%|████████▎ | 187202/225000 [352:57:47<80:04:33,  7.63s/it]
 83%|████████▎ | 187203/225000 [352:57:51<69:05:47,  6.58s/it]
 83%|████████▎ | 187204/225000 [352:57:56<65:03:13,  6.20s/it]
 83%|████████▎ | 187205/225000 [352:58:01<60:58:07,  5.81s/it]
 83%|████████▎ | 187206/225000 [352:58:05<55:19:10,  5.27s/it]
 83%|████████▎ | 187207/225000 [352:58:14<66:33:23,  6.34s/it]
 83%|████████▎ | 187208/225000 [352:58:19<61:42:30,  5.88s/it]
 83%|████████▎ | 187209/225000 [352:58:33<89:19:38,  8.51s/it]
 83%|████████▎ | 187210/225000 [352:58:37<73:37:42,  7.01s/it]
 83%|████████▎ | 187211/225000 [352:58:40<63:04:56,  6.01s/it]
 83%|████████▎ | 187212/225000 [352:58:50<75:06:10,  7.15s/it]
 83%|████████▎ | 187213/225000 [352:58:56<71:17:36,  6.79s/it]
 83%|████████▎ | 187214/225000 [352:59:01<65:41:52,  6.26s/it]
 83%|████████▎ | 187215/225000 [352:59:12<79:52:29,  7.61s/it]
 83%|████████▎ | 187216/225000 [352:59:17<70:42:17,  6.74s/it]
 83%|████████▎ | 187217/225000 [352:59:20<59:12:42,  5.64s/it]
 83%|████████▎ | 187218/225000 [352:59:30<72:42:29,  6.93s/it]
 83%|████████▎ | 187219/225000 [352:59:36<71:06:54,  6.78s/it]
 83%|████████▎ | 187220/225000 [352:59:41<65:56:53,  6.28s/it]
 83%|████████▎ | 187221/225000 [352:59:49<69:15:02,  6.60s/it]
 83%|████████▎ | 187222/225000 [352:59:55<69:37:45,  6.64s/it]
 83%|████████▎ | 187223/225000 [353:00:05<79:57:01,  7.62s/it]
 83%|████████▎ | 187224/225000 [353:00:18<94:32:29,  9.01s/it]
 83%|████████▎ | 187225/225000 [353:00:26<91:52:02,  8.76s/it]
 83%|████████▎ | 187226/225000 [353:00:31<80:23:37,  7.66s/it]
 83%|████████▎ | 187227/225000 [353:00:35<68:47:29,  6.56s/it]
 83%|████████▎ | 187228/225000 [353:00:43<75:32:50,  7.20s/it]
 83%|████████▎ | 187229/225000 [353:00:50<73:47:15,  7.03s/it]
 83%|████████▎ | 187230/225000 [353:00:55<65:40:49,  6.26s/it]
 83%|████████▎ | 187231/225000 [353:01:03<73:31:42,  7.01s/it]
 83%|████████▎ | 187232/225000 [353:01:07<63:45:26,  6.08s/it]
 83%|████████▎ | 187233/225000 [353:01:19<81:36:41,  7.78s/it]
 83%|████████▎ | 187234/225000 [353:01:24<74:05:00,  7.06s/it]
 83%|████████▎ | 187235/225000 [353:01:30<71:05:34,  6.78s/it]
 83%|████████▎ | 187236/225000 [353:01:36<66:22:02,  6.33s/it]
 83%|████████▎ | 187237/225000 [353:01:40<59:12:32,  5.64s/it]
 83%|████████▎ | 187238/225000 [353:01:49<69:45:33,  6.65s/it]
 83%|████████▎ | 187239/225000 [353:01:55<67:45:40,  6.46s/it]
 83%|████████▎ | 187240/225000 [353:02:00<62:46:33,  5.99s/it]
 83%|████████▎ | 187241/225000 [353:02:12<82:24:11,  7.86s/it]
 83%|████████▎ | 187242/225000 [353:02:15<67:30:45,  6.44s/it]
 83%|████████▎ | 187243/225000 [353:02:19<60:07:36,  5.73s/it]
 83%|████████▎ | 187244/225000 [353:02:26<63:47:41,  6.08s/it]
 83%|████████▎ | 187245/225000 [353:02:32<61:56:19,  5.91s/it]
 83%|████████▎ | 187246/225000 [353:02:37<59:01:17,  5.63s/it]
 83%|████████▎ | 187247/225000 [353:02:43<61:13:13,  5.84s/it]
 83%|████████▎ | 187248/225000 [353:02:48<58:44:36,  5.60s/it]
 83%|████████▎ | 187249/225000 [353:03:00<80:43:22,  7.70s/it]
 83%|████████▎ | 187250/225000 [353:03:06<74:41:30,  7.12s/it]
                                                              
{'loss': '0.852', 'grad_norm': '1.117', 'learning_rate': '3.725e-06', 'epoch': '0.8322', 'num_input_tokens_seen': 585845480, 'train_runtime': '1.271e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187250/225000 [353:03:06<74:41:30,  7.12s/it]
 83%|████████▎ | 187251/225000 [353:03:12<70:22:49,  6.71s/it]
 83%|████████▎ | 187252/225000 [353:03:17<65:02:36,  6.20s/it]
 83%|████████▎ | 187253/225000 [353:03:22<61:44:07,  5.89s/it]
 83%|████████▎ | 187254/225000 [353:03:27<57:37:12,  5.50s/it]
 83%|████████▎ | 187255/225000 [353:03:37<71:01:17,  6.77s/it]
 83%|████████▎ | 187256/225000 [353:03:42<66:01:16,  6.30s/it]
 83%|████████▎ | 187257/225000 [353:03:48<66:27:44,  6.34s/it]
 83%|████████▎ | 187258/225000 [353:03:56<69:46:22,  6.66s/it]
 83%|████████▎ | 187259/225000 [353:03:59<61:06:39,  5.83s/it]
 83%|████████▎ | 187260/225000 [353:04:03<54:12:40,  5.17s/it]
 83%|████████▎ | 187261/225000 [353:04:14<72:28:12,  6.91s/it]
 83%|████████▎ | 187262/225000 [353:04:19<65:55:43,  6.29s/it]
 83%|████████▎ | 187263/225000 [353:04:22<57:06:34,  5.45s/it]
 83%|████████▎ | 187264/225000 [353:04:27<54:57:25,  5.24s/it]
 83%|████████▎ | 187265/225000 [353:04:32<52:29:28,  5.01s/it]
 83%|████████▎ | 187266/225000 [353:04:49<90:08:07,  8.60s/it]
 83%|████████▎ | 187267/225000 [353:04:53<75:51:26,  7.24s/it]
 83%|████████▎ | 187268/225000 [353:05:00<76:07:57,  7.26s/it]
 83%|████████▎ | 187269/225000 [353:05:07<74:20:38,  7.09s/it]
 83%|████████▎ | 187270/225000 [353:05:12<68:17:10,  6.52s/it]
 83%|████████▎ | 187271/225000 [353:05:17<63:19:38,  6.04s/it]
 83%|████████▎ | 187272/225000 [353:05:28<78:58:21,  7.54s/it]
 83%|████████▎ | 187273/225000 [353:05:32<67:24:00,  6.43s/it]
 83%|████████▎ | 187274/225000 [353:05:38<68:22:21,  6.52s/it]
 83%|████████▎ | 187275/225000 [353:05:44<64:00:56,  6.11s/it]
 83%|████████▎ | 187276/225000 [353:05:48<58:48:31,  5.61s/it]
 83%|████████▎ | 187277/225000 [353:05:52<55:22:18,  5.28s/it]
 83%|████████▎ | 187278/225000 [353:05:58<55:59:01,  5.34s/it]
 83%|████████▎ | 187279/225000 [353:06:03<53:54:37,  5.15s/it]
 83%|████████▎ | 187280/225000 [353:06:12<68:32:23,  6.54s/it]
 83%|████████▎ | 187281/225000 [353:06:17<63:23:04,  6.05s/it]
 83%|████████▎ | 187282/225000 [353:06:25<69:17:31,  6.61s/it]
 83%|████████▎ | 187283/225000 [353:06:29<60:14:34,  5.75s/it]
 83%|████████▎ | 187284/225000 [353:06:34<58:28:01,  5.58s/it]
 83%|████████▎ | 187285/225000 [353:06:41<60:58:25,  5.82s/it]
 83%|████████▎ | 187286/225000 [353:06:49<68:04:17,  6.50s/it]
 83%|████████▎ | 187287/225000 [353:06:52<58:37:19,  5.60s/it]
 83%|████████▎ | 187288/225000 [353:07:02<71:27:27,  6.82s/it]
 83%|████████▎ | 187289/225000 [353:07:09<71:41:37,  6.84s/it]
 83%|████████▎ | 187290/225000 [353:07:18<79:20:09,  7.57s/it]
 83%|████████▎ | 187291/225000 [353:07:26<79:57:49,  7.63s/it]
 83%|████████▎ | 187292/225000 [353:07:31<72:15:14,  6.90s/it]
 83%|████████▎ | 187293/225000 [353:07:40<80:17:02,  7.66s/it]
 83%|████████▎ | 187294/225000 [353:07:47<77:43:33,  7.42s/it]
 83%|████████▎ | 187295/225000 [353:07:52<68:23:55,  6.53s/it]
 83%|████████▎ | 187296/225000 [353:08:00<74:57:39,  7.16s/it]
 83%|████████▎ | 187297/225000 [353:08:11<87:28:30,  8.35s/it]
 83%|████████▎ | 187298/225000 [353:08:16<75:37:15,  7.22s/it]
 83%|████████▎ | 187299/225000 [353:08:21<67:51:55,  6.48s/it]
 83%|████████▎ | 187300/225000 [353:08:26<62:22:06,  5.96s/it]
                                                              
{'loss': '0.8656', 'grad_norm': '1.34', 'learning_rate': '3.715e-06', 'epoch': '0.8324', 'num_input_tokens_seen': 586004264, 'train_runtime': '1.271e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187300/225000 [353:08:26<62:22:06,  5.96s/it]
 83%|████████▎ | 187301/225000 [353:08:30<57:34:07,  5.50s/it]
 83%|████████▎ | 187302/225000 [353:08:35<56:09:52,  5.36s/it]
 83%|████████▎ | 187303/225000 [353:08:40<55:10:02,  5.27s/it]
 83%|████████▎ | 187304/225000 [353:08:51<73:56:12,  7.06s/it]
 83%|████████▎ | 187305/225000 [353:08:55<64:27:50,  6.16s/it]
 83%|████████▎ | 187306/225000 [353:09:00<58:46:47,  5.61s/it]
 83%|████████▎ | 187307/225000 [353:09:03<52:44:04,  5.04s/it]
 83%|████████▎ | 187308/225000 [353:09:16<75:17:44,  7.19s/it]
 83%|████████▎ | 187309/225000 [353:09:22<71:13:02,  6.80s/it]
 83%|████████▎ | 187310/225000 [353:09:27<68:33:52,  6.55s/it]
 83%|████████▎ | 187311/225000 [353:09:35<72:03:08,  6.88s/it]
 83%|████████▎ | 187312/225000 [353:09:40<65:56:47,  6.30s/it]
 83%|████████▎ | 187313/225000 [353:09:58<102:23:01,  9.78s/it]
 83%|████████▎ | 187314/225000 [353:10:01<80:53:42,  7.73s/it] 
 83%|████████▎ | 187315/225000 [353:10:08<79:02:36,  7.55s/it]
 83%|████████▎ | 187316/225000 [353:10:11<64:35:39,  6.17s/it]
 83%|████████▎ | 187317/225000 [353:10:18<68:09:14,  6.51s/it]
 83%|████████▎ | 187318/225000 [353:10:27<73:42:31,  7.04s/it]
 83%|████████▎ | 187319/225000 [353:10:31<65:54:31,  6.30s/it]
 83%|████████▎ | 187320/225000 [353:10:38<69:02:11,  6.60s/it]
 83%|████████▎ | 187321/225000 [353:10:42<59:45:59,  5.71s/it]
 83%|████████▎ | 187322/225000 [353:10:50<67:07:49,  6.41s/it]
 83%|████████▎ | 187323/225000 [353:10:59<75:43:18,  7.24s/it]
 83%|████████▎ | 187324/225000 [353:11:10<87:38:55,  8.37s/it]
 83%|████████▎ | 187325/225000 [353:11:16<80:26:32,  7.69s/it]
 83%|████████▎ | 187326/225000 [353:11:23<77:24:07,  7.40s/it]
 83%|████████▎ | 187327/225000 [353:11:36<94:26:22,  9.02s/it]
 83%|████████▎ | 187328/225000 [353:11:40<78:20:45,  7.49s/it]
 83%|████████▎ | 187329/225000 [353:11:44<69:15:51,  6.62s/it]
 83%|████████▎ | 187330/225000 [353:11:50<64:32:30,  6.17s/it]
 83%|████████▎ | 187331/225000 [353:11:58<70:28:01,  6.73s/it]
 83%|████████▎ | 187332/225000 [353:12:08<81:14:57,  7.77s/it]
 83%|████████▎ | 187333/225000 [353:12:11<66:05:56,  6.32s/it]
 83%|████████▎ | 187334/225000 [353:12:21<77:35:15,  7.42s/it]
 83%|████████▎ | 187335/225000 [353:12:25<68:44:15,  6.57s/it]
 83%|████████▎ | 187336/225000 [353:12:29<61:00:08,  5.83s/it]
 83%|████████▎ | 187337/225000 [353:12:42<81:08:15,  7.76s/it]
 83%|████████▎ | 187338/225000 [353:12:50<83:12:44,  7.95s/it]
 83%|████████▎ | 187339/225000 [353:12:55<72:29:46,  6.93s/it]
 83%|████████▎ | 187340/225000 [353:13:05<82:25:31,  7.88s/it]
 83%|████████▎ | 187341/225000 [353:13:08<68:44:13,  6.57s/it]
 83%|████████▎ | 187342/225000 [353:13:12<59:48:27,  5.72s/it]
 83%|████████▎ | 187343/225000 [353:13:21<71:17:54,  6.82s/it]
 83%|████████▎ | 187344/225000 [353:13:31<80:50:25,  7.73s/it]
 83%|████████▎ | 187345/225000 [353:13:36<72:37:53,  6.94s/it]
 83%|████████▎ | 187346/225000 [353:13:41<64:43:01,  6.19s/it]
 83%|████████▎ | 187347/225000 [353:13:45<57:44:29,  5.52s/it]
 83%|████████▎ | 187348/225000 [353:13:55<71:54:32,  6.88s/it]
 83%|████████▎ | 187349/225000 [353:13:59<63:52:02,  6.11s/it]
 83%|████████▎ | 187350/225000 [353:14:09<76:13:17,  7.29s/it]
                                                              
{'loss': '0.8477', 'grad_norm': '1.131', 'learning_rate': '3.706e-06', 'epoch': '0.8327', 'num_input_tokens_seen': 586166032, 'train_runtime': '1.272e+06', 'train_tokens_per_second': '460.9'}

 83%|████████▎ | 187350/225000 [353:14:09<76:13:17,  7.29s/it]
 83%|████████▎ | 187351/225000 [353:14:19<85:42:53,  8.20s/it]
 83%|████████▎ | 187352/225000 [353:14:24<73:50:35,  7.06s/it]
 83%|████████▎ | 187353/225000 [353:14:32<77:38:25,  7.42s/it]
 83%|████████▎ | 187354/225000 [353:14:41<81:39:35,  7.81s/it]
 83%|████████▎ | 187355/225000 [353:14:44<67:13:37,  6.43s/it]
 83%|████████▎ | 187356/225000 [353:14:52<71:36:48,  6.85s/it]
 83%|████████▎ | 187357/225000 [353:14:55<60:52:31,  5.82s/it]
 83%|████████▎ | 187358/225000 [353:15:00<58:56:40,  5.64s/it]
 83%|████████▎ | 187359/225000 [353:15:08<64:09:40,  6.14s/it]
 83%|████████▎ | 187360/225000 [353:15:11<56:28:42,  5.40s/it]
 83%|████████▎ | 187361/225000 [353:15:19<62:14:19,  5.95s/it]
 83%|████████▎ | 187362/225000 [353:15:24<59:36:16,  5.70s/it]
 83%|████████▎ | 187363/225000 [353:15:28<55:58:24,  5.35s/it]
 83%|████████▎ | 187364/225000 [353:15:33<54:04:49,  5.17s/it]
 83%|████████▎ | 187365/225000 [353:15:38<53:26:25,  5.11s/it]
 83%|████████▎ | 187366/225000 [353:15:44<55:02:58,  5.27s/it]
 83%|████████▎ | 187367/225000 [353:15:48<51:43:52,  4.95s/it]
 83%|████████▎ | 187368/225000 [353:15:53<51:07:14,  4.89s/it]
 83%|████████▎ | 187369/225000 [353:16:06<79:10:49,  7.57s/it]
 83%|████████▎ | 187370/225000 [353:16:13<76:54:02,  7.36s/it]
 83%|████████▎ | 187371/225000 [353:16:20<73:58:57,  7.08s/it]
 83%|████████▎ | 187372/225000 [353:16:27<73:12:16,  7.00s/it]
 83%|████████▎ | 187373/225000 [353:16:32<67:02:23,  6.41s/it]
 83%|████████▎ | 187374/225000 [353:16:43<83:31:36,  7.99s/it]
 83%|████████▎ | 187375/225000 [353:16:54<92:31:11,  8.85s/it]
 83%|████████▎ | 187376/225000 [353:16:58<76:53:51,  7.36s/it]
 83%|████████▎ | 187377/225000 [353:17:05<75:50:32,  7.26s/it]
 83%|████████▎ | 187378/225000 [353:17:11<71:10:42,  6.81s/it]
 83%|████████▎ | 187379/225000 [353:17:16<65:56:55,  6.31s/it]
 83%|████████▎ | 187380/225000 [353:17:27<81:00:13,  7.75s/it]
 83%|████████▎ | 187381/225000 [353:17:35<82:04:36,  7.85s/it]
 83%|████████▎ | 187382/225000 [353:17:39<70:00:01,  6.70s/it]
 83%|████████▎ | 187383/225000 [353:17:50<81:46:46,  7.83s/it]
 83%|████████▎ | 187384/225000 [353:17:53<68:28:33,  6.55s/it]
 83%|████████▎ | 187385/225000 [353:18:01<70:47:59,  6.78s/it]
 83%|████████▎ | 187386/225000 [353:18:07<69:51:06,  6.69s/it]
 83%|████████▎ | 187387/225000 [353:18:13<68:40:02,  6.57s/it]
 83%|████████▎ | 187388/225000 [353:18:18<62:54:56,  6.02s/it]
 83%|████████▎ | 187389/225000 [353:18:22<55:32:30,  5.32s/it]
 83%|████████▎ | 187390/225000 [353:18:27<55:14:35,  5.29s/it]
 83%|████████▎ | 187391/225000 [353:18:34<59:27:40,  5.69s/it]
 83%|████████▎ | 187392/225000 [353:18:37<53:54:21,  5.16s/it]
 83%|████████▎ | 187393/225000 [353:18:46<63:39:24,  6.09s/it]
 83%|████████▎ | 187394/225000 [353:18:52<63:19:50,  6.06s/it]
 83%|████████▎ | 187395/225000 [353:19:03<80:34:47,  7.71s/it]
 83%|████████▎ | 187396/225000 [353:19:07<67:22:45,  6.45s/it]
 83%|████████▎ | 187397/225000 [353:19:13<65:25:23,  6.26s/it]
 83%|████████▎ | 187398/225000 [353:19:17<58:46:30,  5.63s/it]
 83%|████████▎ | 187399/225000 [353:19:22<57:15:49,  5.48s/it]
 83%|████████▎ | 187400/225000 [353:19:32<71:18:06,  6.83s/it]
                                                              
{'loss': '0.8724', 'grad_norm': '1.153', 'learning_rate': '3.696e-06', 'epoch': '0.8329', 'num_input_tokens_seen': 586322944, 'train_runtime': '1.272e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187400/225000 [353:19:32<71:18:06,  6.83s/it]
 83%|████████▎ | 187401/225000 [353:19:36<62:11:00,  5.95s/it]
 83%|████████▎ | 187402/225000 [353:19:51<92:24:51,  8.85s/it]
 83%|████████▎ | 187403/225000 [353:19:57<81:48:12,  7.83s/it]
 83%|████████▎ | 187404/225000 [353:20:14<112:08:46, 10.74s/it]
 83%|████████▎ | 187405/225000 [353:20:17<88:10:14,  8.44s/it] 
 83%|████████▎ | 187406/225000 [353:20:23<77:58:44,  7.47s/it]
 83%|████████▎ | 187407/225000 [353:20:28<70:27:51,  6.75s/it]
 83%|████████▎ | 187408/225000 [353:20:35<73:22:12,  7.03s/it]
 83%|████████▎ | 187409/225000 [353:20:39<62:38:23,  6.00s/it]
 83%|████████▎ | 187410/225000 [353:20:49<74:52:10,  7.17s/it]
 83%|████████▎ | 187411/225000 [353:20:55<71:27:59,  6.84s/it]
 83%|████████▎ | 187412/225000 [353:20:59<62:29:51,  5.99s/it]
 83%|████████▎ | 187413/225000 [353:21:05<61:51:10,  5.92s/it]
 83%|████████▎ | 187414/225000 [353:21:12<64:56:44,  6.22s/it]
 83%|████████▎ | 187415/225000 [353:21:23<79:51:25,  7.65s/it]
 83%|████████▎ | 187416/225000 [353:21:26<67:10:52,  6.43s/it]
 83%|████████▎ | 187417/225000 [353:21:30<58:07:17,  5.57s/it]
 83%|████████▎ | 187418/225000 [353:21:39<69:57:25,  6.70s/it]
 83%|████████▎ | 187419/225000 [353:21:44<63:09:27,  6.05s/it]
 83%|████████▎ | 187420/225000 [353:21:55<78:44:34,  7.54s/it]
 83%|████████▎ | 187421/225000 [353:22:00<71:57:16,  6.89s/it]
 83%|████████▎ | 187422/225000 [353:22:09<76:47:09,  7.36s/it]
 83%|████████▎ | 187423/225000 [353:22:13<68:12:17,  6.53s/it]
 83%|████████▎ | 187424/225000 [353:22:21<71:42:04,  6.87s/it]
 83%|████████▎ | 187425/225000 [353:22:27<69:14:55,  6.63s/it]
 83%|████████▎ | 187426/225000 [353:22:31<60:49:23,  5.83s/it]
 83%|████████▎ | 187427/225000 [353:22:37<62:55:16,  6.03s/it]
 83%|████████▎ | 187428/225000 [353:22:42<57:08:31,  5.48s/it]
 83%|████████▎ | 187429/225000 [353:22:55<81:03:14,  7.77s/it]
 83%|████████▎ | 187430/225000 [353:22:59<68:53:16,  6.60s/it]
 83%|████████▎ | 187431/225000 [353:23:03<63:30:02,  6.08s/it]
 83%|████████▎ | 187432/225000 [353:23:08<59:40:20,  5.72s/it]
 83%|████████▎ | 187433/225000 [353:23:17<68:42:06,  6.58s/it]
 83%|████████▎ | 187434/225000 [353:23:22<64:37:43,  6.19s/it]
 83%|████████▎ | 187435/225000 [353:23:30<68:29:48,  6.56s/it]
 83%|████████▎ | 187436/225000 [353:23:34<60:46:26,  5.82s/it]
 83%|████████▎ | 187437/225000 [353:23:39<59:38:44,  5.72s/it]
 83%|████████▎ | 187438/225000 [353:23:47<67:13:56,  6.44s/it]
 83%|████████▎ | 187439/225000 [353:23:52<62:44:17,  6.01s/it]
 83%|████████▎ | 187440/225000 [353:24:00<67:39:23,  6.48s/it]
 83%|████████▎ | 187441/225000 [353:24:04<58:59:24,  5.65s/it]
 83%|████████▎ | 187442/225000 [353:24:16<79:07:02,  7.58s/it]
 83%|████████▎ | 187443/225000 [353:24:21<73:17:01,  7.02s/it]
 83%|████████▎ | 187444/225000 [353:24:26<66:42:43,  6.39s/it]
 83%|████████▎ | 187445/225000 [353:24:38<84:22:02,  8.09s/it]
 83%|████████▎ | 187446/225000 [353:24:42<69:10:23,  6.63s/it]
 83%|████████▎ | 187447/225000 [353:24:50<74:13:45,  7.12s/it]
 83%|████████▎ | 187448/225000 [353:24:54<64:46:59,  6.21s/it]
 83%|████████▎ | 187449/225000 [353:25:00<65:47:31,  6.31s/it]
 83%|████████▎ | 187450/225000 [353:25:06<62:51:58,  6.03s/it]
                                                              
{'loss': '0.8886', 'grad_norm': '1.207', 'learning_rate': '3.686e-06', 'epoch': '0.8331', 'num_input_tokens_seen': 586486720, 'train_runtime': '1.272e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187450/225000 [353:25:06<62:51:58,  6.03s/it]
 83%|████████▎ | 187451/225000 [353:25:14<69:31:04,  6.67s/it]
 83%|████████▎ | 187452/225000 [353:25:18<59:51:00,  5.74s/it]
 83%|████████▎ | 187453/225000 [353:25:23<59:05:09,  5.67s/it]
 83%|████████▎ | 187454/225000 [353:25:31<66:30:13,  6.38s/it]
 83%|████████▎ | 187455/225000 [353:25:34<56:16:00,  5.40s/it]
 83%|████████▎ | 187456/225000 [353:25:45<73:30:01,  7.05s/it]
 83%|████████▎ | 187457/225000 [353:25:55<81:50:37,  7.85s/it]
 83%|████████▎ | 187458/225000 [353:25:59<69:04:00,  6.62s/it]
 83%|████████▎ | 187459/225000 [353:26:04<65:31:52,  6.28s/it]
 83%|████████▎ | 187460/225000 [353:26:19<91:32:05,  8.78s/it]
 83%|████████▎ | 187461/225000 [353:26:26<86:58:41,  8.34s/it]
 83%|████████▎ | 187462/225000 [353:26:31<75:52:20,  7.28s/it]
 83%|████████▎ | 187463/225000 [353:26:37<72:56:04,  6.99s/it]
 83%|████████▎ | 187464/225000 [353:26:41<64:35:42,  6.20s/it]
 83%|████████▎ | 187465/225000 [353:26:46<57:56:55,  5.56s/it]
 83%|████████▎ | 187466/225000 [353:26:54<67:03:48,  6.43s/it]
 83%|████████▎ | 187467/225000 [353:27:00<66:47:09,  6.41s/it]
 83%|████████▎ | 187468/225000 [353:27:06<63:12:36,  6.06s/it]
 83%|████████▎ | 187469/225000 [353:27:12<63:38:15,  6.10s/it]
 83%|████████▎ | 187470/225000 [353:27:15<55:22:06,  5.31s/it]
 83%|████████▎ | 187471/225000 [353:27:20<52:06:20,  5.00s/it]
 83%|████████▎ | 187472/225000 [353:27:26<57:44:22,  5.54s/it]
 83%|████████▎ | 187473/225000 [353:27:32<57:51:39,  5.55s/it]
 83%|████████▎ | 187474/225000 [353:27:37<56:54:25,  5.46s/it]
 83%|████████▎ | 187475/225000 [353:27:52<86:14:30,  8.27s/it]
 83%|████████▎ | 187476/225000 [353:27:56<71:47:58,  6.89s/it]
 83%|████████▎ | 187477/225000 [353:28:06<84:11:21,  8.08s/it]
 83%|████████▎ | 187478/225000 [353:28:15<85:57:08,  8.25s/it]
 83%|████████▎ | 187479/225000 [353:28:19<71:07:23,  6.82s/it]
 83%|████████▎ | 187480/225000 [353:28:23<63:00:39,  6.05s/it]
 83%|████████▎ | 187481/225000 [353:28:33<75:46:23,  7.27s/it]
 83%|████████▎ | 187482/225000 [353:28:38<67:19:53,  6.46s/it]
 83%|████████▎ | 187483/225000 [353:28:45<71:37:39,  6.87s/it]
 83%|████████▎ | 187484/225000 [353:28:52<71:39:08,  6.88s/it]
 83%|████████▎ | 187485/225000 [353:28:57<64:38:40,  6.20s/it]
 83%|████████▎ | 187486/225000 [353:29:06<74:18:07,  7.13s/it]
 83%|████████▎ | 187487/225000 [353:29:11<66:46:43,  6.41s/it]
 83%|████████▎ | 187488/225000 [353:29:21<77:49:33,  7.47s/it]
 83%|████████▎ | 187489/225000 [353:29:29<79:08:57,  7.60s/it]
 83%|████████▎ | 187490/225000 [353:29:37<81:14:38,  7.80s/it]
 83%|████████▎ | 187491/225000 [353:29:46<85:39:37,  8.22s/it]
 83%|████████▎ | 187492/225000 [353:29:52<76:33:03,  7.35s/it]
 83%|████████▎ | 187493/225000 [353:30:04<93:00:01,  8.93s/it]
 83%|████████▎ | 187494/225000 [353:30:08<76:06:15,  7.30s/it]
 83%|████████▎ | 187495/225000 [353:30:11<64:30:03,  6.19s/it]
 83%|████████▎ | 187496/225000 [353:30:19<68:31:26,  6.58s/it]
 83%|████████▎ | 187497/225000 [353:30:25<66:32:57,  6.39s/it]
 83%|████████▎ | 187498/225000 [353:30:29<60:54:41,  5.85s/it]
 83%|████████▎ | 187499/225000 [353:30:37<65:30:08,  6.29s/it]
 83%|████████▎ | 187500/225000 [353:30:46<74:50:02,  7.18s/it]
                                                              
{'loss': '0.891', 'grad_norm': '1.304', 'learning_rate': '3.677e-06', 'epoch': '0.8333', 'num_input_tokens_seen': 586653576, 'train_runtime': '1.273e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187500/225000 [353:30:46<74:50:02,  7.18s/it]
 83%|████████▎ | 187501/225000 [353:30:50<64:07:24,  6.16s/it]
 83%|████████▎ | 187502/225000 [353:30:57<69:19:59,  6.66s/it]
 83%|████████▎ | 187503/225000 [353:31:08<82:50:25,  7.95s/it]
 83%|████████▎ | 187504/225000 [353:31:13<71:48:29,  6.89s/it]
 83%|████████▎ | 187505/225000 [353:31:22<77:24:26,  7.43s/it]
 83%|████████▎ | 187506/225000 [353:31:25<65:50:34,  6.32s/it]
 83%|████████▎ | 187507/225000 [353:31:31<63:06:53,  6.06s/it]
 83%|████████▎ | 187508/225000 [353:31:37<62:20:00,  5.99s/it]
 83%|████████▎ | 187509/225000 [353:31:45<68:31:51,  6.58s/it]
 83%|████████▎ | 187510/225000 [353:31:49<60:41:42,  5.83s/it]
 83%|████████▎ | 187511/225000 [353:31:53<57:24:40,  5.51s/it]
 83%|████████▎ | 187512/225000 [353:31:58<55:35:00,  5.34s/it]
 83%|████████▎ | 187513/225000 [353:32:11<77:16:01,  7.42s/it]
 83%|████████▎ | 187514/225000 [353:32:14<66:02:47,  6.34s/it]
 83%|████████▎ | 187515/225000 [353:32:23<72:18:27,  6.94s/it]
 83%|████████▎ | 187516/225000 [353:32:27<63:28:19,  6.10s/it]
 83%|████████▎ | 187517/225000 [353:32:34<67:56:35,  6.53s/it]
 83%|████████▎ | 187518/225000 [353:32:43<73:10:43,  7.03s/it]
 83%|████████▎ | 187519/225000 [353:32:52<81:03:50,  7.79s/it]
 83%|████████▎ | 187520/225000 [353:32:56<69:44:39,  6.70s/it]
 83%|████████▎ | 187521/225000 [353:33:02<67:57:58,  6.53s/it]
 83%|████████▎ | 187522/225000 [353:33:13<79:17:26,  7.62s/it]
 83%|████████▎ | 187523/225000 [353:33:18<71:55:31,  6.91s/it]
 83%|████████▎ | 187524/225000 [353:33:23<64:53:01,  6.23s/it]
 83%|████████▎ | 187525/225000 [353:33:26<56:44:18,  5.45s/it]
 83%|████████▎ | 187526/225000 [353:33:35<68:22:06,  6.57s/it]
 83%|████████▎ | 187527/225000 [353:33:42<68:45:06,  6.60s/it]
 83%|████████▎ | 187528/225000 [353:33:50<73:42:20,  7.08s/it]
 83%|████████▎ | 187529/225000 [353:33:58<75:07:21,  7.22s/it]
 83%|████████▎ | 187530/225000 [353:34:04<73:07:26,  7.03s/it]
 83%|████████▎ | 187531/225000 [353:34:15<84:42:45,  8.14s/it]
 83%|████████▎ | 187532/225000 [353:34:19<72:14:07,  6.94s/it]
 83%|████████▎ | 187533/225000 [353:34:25<67:36:04,  6.50s/it]
 83%|████████▎ | 187534/225000 [353:34:35<79:38:35,  7.65s/it]
 83%|████████▎ | 187535/225000 [353:34:39<67:57:52,  6.53s/it]
 83%|████████▎ | 187536/225000 [353:34:44<64:29:45,  6.20s/it]
 83%|████████▎ | 187537/225000 [353:34:55<77:05:58,  7.41s/it]
 83%|████████▎ | 187538/225000 [353:35:05<85:15:28,  8.19s/it]
 83%|████████▎ | 187539/225000 [353:35:10<77:11:13,  7.42s/it]
 83%|████████▎ | 187540/225000 [353:35:19<81:43:54,  7.85s/it]
 83%|████████▎ | 187541/225000 [353:35:24<72:43:25,  6.99s/it]
 83%|████████▎ | 187542/225000 [353:35:33<78:07:29,  7.51s/it]
 83%|████████▎ | 187543/225000 [353:35:41<80:29:56,  7.74s/it]
 83%|████████▎ | 187544/225000 [353:35:45<67:58:11,  6.53s/it]
 83%|████████▎ | 187545/225000 [353:35:50<64:05:36,  6.16s/it]
 83%|████████▎ | 187546/225000 [353:35:58<70:53:08,  6.81s/it]
 83%|████████▎ | 187547/225000 [353:36:03<64:46:14,  6.23s/it]
 83%|████████▎ | 187548/225000 [353:36:08<59:34:39,  5.73s/it]
 83%|████████▎ | 187549/225000 [353:36:11<53:11:33,  5.11s/it]
 83%|████████▎ | 187550/225000 [353:36:15<48:30:18,  4.66s/it]
                                                              
{'loss': '0.929', 'grad_norm': '1.632', 'learning_rate': '3.667e-06', 'epoch': '0.8336', 'num_input_tokens_seen': 586813624, 'train_runtime': '1.273e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187550/225000 [353:36:15<48:30:18,  4.66s/it]
 83%|████████▎ | 187551/225000 [353:36:20<49:29:26,  4.76s/it]
 83%|████████▎ | 187552/225000 [353:36:25<48:43:15,  4.68s/it]
 83%|████████▎ | 187553/225000 [353:36:39<79:30:44,  7.64s/it]
 83%|████████▎ | 187554/225000 [353:36:43<66:15:49,  6.37s/it]
 83%|████████▎ | 187555/225000 [353:36:53<78:20:27,  7.53s/it]
 83%|████████▎ | 187556/225000 [353:37:02<84:05:49,  8.09s/it]
 83%|████████▎ | 187557/225000 [353:37:07<74:28:20,  7.16s/it]
 83%|████████▎ | 187558/225000 [353:37:15<77:37:46,  7.46s/it]
 83%|████████▎ | 187559/225000 [353:37:19<66:33:51,  6.40s/it]
 83%|████████▎ | 187560/225000 [353:37:24<61:35:55,  5.92s/it]
 83%|████████▎ | 187561/225000 [353:37:35<76:44:45,  7.38s/it]
 83%|████████▎ | 187562/225000 [353:37:40<68:27:37,  6.58s/it]
 83%|████████▎ | 187563/225000 [353:37:46<66:38:44,  6.41s/it]
 83%|████████▎ | 187564/225000 [353:37:51<62:55:17,  6.05s/it]
 83%|████████▎ | 187565/225000 [353:38:01<76:04:59,  7.32s/it]
 83%|████████▎ | 187566/225000 [353:38:06<68:54:33,  6.63s/it]
 83%|████████▎ | 187567/225000 [353:38:12<67:36:19,  6.50s/it]
 83%|████████▎ | 187568/225000 [353:38:23<81:41:14,  7.86s/it]
 83%|████████▎ | 187569/225000 [353:38:30<78:10:17,  7.52s/it]
 83%|████████▎ | 187570/225000 [353:38:35<69:47:43,  6.71s/it]
 83%|████████▎ | 187571/225000 [353:38:43<74:20:06,  7.15s/it]
 83%|████████▎ | 187572/225000 [353:38:48<66:28:18,  6.39s/it]
 83%|████████▎ | 187573/225000 [353:38:51<57:04:27,  5.49s/it]
 83%|████████▎ | 187574/225000 [353:39:00<66:35:34,  6.41s/it]
 83%|████████▎ | 187575/225000 [353:39:07<68:34:56,  6.60s/it]
 83%|████████▎ | 187576/225000 [353:39:13<69:21:20,  6.67s/it]
 83%|████████▎ | 187577/225000 [353:39:20<68:33:48,  6.60s/it]
 83%|████████▎ | 187578/225000 [353:39:28<71:55:53,  6.92s/it]
 83%|████████▎ | 187579/225000 [353:39:38<82:20:50,  7.92s/it]
 83%|████████▎ | 187580/225000 [353:39:47<86:17:58,  8.30s/it]
 83%|████████▎ | 187581/225000 [353:39:52<75:34:52,  7.27s/it]
 83%|████████▎ | 187582/225000 [353:39:56<66:07:46,  6.36s/it]
 83%|████████▎ | 187583/225000 [353:40:05<73:04:05,  7.03s/it]
 83%|████████▎ | 187584/225000 [353:40:12<73:07:16,  7.04s/it]
 83%|████████▎ | 187585/225000 [353:40:19<73:39:45,  7.09s/it]
 83%|████████▎ | 187586/225000 [353:40:25<70:08:40,  6.75s/it]
 83%|████████▎ | 187587/225000 [353:40:36<84:51:26,  8.17s/it]
 83%|████████▎ | 187588/225000 [353:40:44<82:26:02,  7.93s/it]
 83%|████████▎ | 187589/225000 [353:40:49<74:19:12,  7.15s/it]
 83%|████████▎ | 187590/225000 [353:40:54<68:37:09,  6.60s/it]
 83%|████████▎ | 187591/225000 [353:41:06<83:53:06,  8.07s/it]
 83%|████████▎ | 187592/225000 [353:41:11<73:56:32,  7.12s/it]
 83%|████████▎ | 187593/225000 [353:41:21<84:12:40,  8.10s/it]
 83%|████████▎ | 187594/225000 [353:41:30<86:14:30,  8.30s/it]
 83%|████████▎ | 187595/225000 [353:41:36<80:25:09,  7.74s/it]
 83%|████████▎ | 187596/225000 [353:41:40<68:35:14,  6.60s/it]
 83%|████████▎ | 187597/225000 [353:41:48<72:16:22,  6.96s/it]
 83%|████████▎ | 187598/225000 [353:41:57<76:57:34,  7.41s/it]
 83%|████████▎ | 187599/225000 [353:42:01<66:06:42,  6.36s/it]
 83%|████████▎ | 187600/225000 [353:42:11<79:37:56,  7.67s/it]
                                                              
{'loss': '0.9463', 'grad_norm': '1.396', 'learning_rate': '3.658e-06', 'epoch': '0.8338', 'num_input_tokens_seen': 586989656, 'train_runtime': '1.273e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187600/225000 [353:42:11<79:37:56,  7.67s/it]
 83%|████████▎ | 187601/225000 [353:42:16<71:51:07,  6.92s/it]
 83%|████████▎ | 187602/225000 [353:42:21<64:36:55,  6.22s/it]
 83%|████████▎ | 187603/225000 [353:42:26<60:46:15,  5.85s/it]
 83%|████████▎ | 187604/225000 [353:42:34<68:32:40,  6.60s/it]
 83%|████████▎ | 187605/225000 [353:42:40<65:24:51,  6.30s/it]
 83%|████████▎ | 187606/225000 [353:42:46<63:35:52,  6.12s/it]
 83%|████████▎ | 187607/225000 [353:42:50<59:29:40,  5.73s/it]
 83%|████████▎ | 187608/225000 [353:43:04<83:12:22,  8.01s/it]
 83%|████████▎ | 187609/225000 [353:43:13<87:01:06,  8.38s/it]
 83%|████████▎ | 187610/225000 [353:43:19<78:28:12,  7.56s/it]
 83%|████████▎ | 187611/225000 [353:43:22<65:52:54,  6.34s/it]
 83%|████████▎ | 187612/225000 [353:43:26<56:41:41,  5.46s/it]
 83%|████████▎ | 187613/225000 [353:43:35<70:01:51,  6.74s/it]
 83%|████████▎ | 187614/225000 [353:43:43<72:35:36,  6.99s/it]
 83%|████████▎ | 187615/225000 [353:43:52<80:12:58,  7.72s/it]
 83%|████████▎ | 187616/225000 [353:43:58<72:47:32,  7.01s/it]
 83%|████████▎ | 187617/225000 [353:44:11<92:21:34,  8.89s/it]
 83%|████████▎ | 187618/225000 [353:44:22<99:37:26,  9.59s/it]
 83%|████████▎ | 187619/225000 [353:44:31<95:50:59,  9.23s/it]
 83%|████████▎ | 187620/225000 [353:44:35<79:44:14,  7.68s/it]
 83%|████████▎ | 187621/225000 [353:44:39<70:54:37,  6.83s/it]
 83%|████████▎ | 187622/225000 [353:44:47<71:38:32,  6.90s/it]
 83%|████████▎ | 187623/225000 [353:44:54<74:18:44,  7.16s/it]
 83%|████████▎ | 187624/225000 [353:45:06<87:32:40,  8.43s/it]
 83%|████████▎ | 187625/225000 [353:45:11<77:20:55,  7.45s/it]
 83%|████████▎ | 187626/225000 [353:45:17<71:52:02,  6.92s/it]
 83%|████████▎ | 187627/225000 [353:45:21<64:57:30,  6.26s/it]
 83%|████████▎ | 187628/225000 [353:45:31<76:39:55,  7.39s/it]
 83%|████████▎ | 187629/225000 [353:45:36<68:43:07,  6.62s/it]
 83%|████████▎ | 187630/225000 [353:45:43<69:36:25,  6.71s/it]
 83%|████████▎ | 187631/225000 [353:45:47<62:02:40,  5.98s/it]
 83%|████████▎ | 187632/225000 [353:45:54<63:16:28,  6.10s/it]
 83%|████████▎ | 187633/225000 [353:45:59<61:29:20,  5.92s/it]
 83%|████████▎ | 187634/225000 [353:46:12<83:22:38,  8.03s/it]
 83%|████████▎ | 187635/225000 [353:46:20<82:17:23,  7.93s/it]
 83%|████████▎ | 187636/225000 [353:46:23<68:46:21,  6.63s/it]
 83%|████████▎ | 187637/225000 [353:46:31<72:29:14,  6.98s/it]
 83%|████████▎ | 187638/225000 [353:46:36<65:46:10,  6.34s/it]
 83%|████████▎ | 187639/225000 [353:46:40<58:22:44,  5.63s/it]
 83%|████████▎ | 187640/225000 [353:46:53<81:01:16,  7.81s/it]
 83%|████████▎ | 187641/225000 [353:46:57<70:20:42,  6.78s/it]
 83%|████████▎ | 187642/225000 [353:47:03<67:46:23,  6.53s/it]
 83%|████████▎ | 187643/225000 [353:47:10<68:09:08,  6.57s/it]
 83%|████████▎ | 187644/225000 [353:47:16<68:03:12,  6.56s/it]
 83%|████████▎ | 187645/225000 [353:47:20<58:52:14,  5.67s/it]
 83%|████████▎ | 187646/225000 [353:47:25<55:59:13,  5.40s/it]
 83%|████████▎ | 187647/225000 [353:47:29<52:00:43,  5.01s/it]
 83%|████████▎ | 187648/225000 [353:47:40<70:40:32,  6.81s/it]
 83%|████████▎ | 187649/225000 [353:47:45<65:44:20,  6.34s/it]
 83%|████████▎ | 187650/225000 [353:47:54<73:27:55,  7.08s/it]
                                                              
{'loss': '0.9267', 'grad_norm': '1.326', 'learning_rate': '3.648e-06', 'epoch': '0.834', 'num_input_tokens_seen': 587158176, 'train_runtime': '1.274e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187650/225000 [353:47:54<73:27:55,  7.08s/it]
 83%|████████▎ | 187651/225000 [353:47:59<66:07:07,  6.37s/it]
 83%|████████▎ | 187652/225000 [353:48:02<57:38:15,  5.56s/it]
 83%|████████▎ | 187653/225000 [353:48:06<51:59:42,  5.01s/it]
 83%|████████▎ | 187654/225000 [353:48:11<51:39:00,  4.98s/it]
 83%|████████▎ | 187655/225000 [353:48:28<89:00:58,  8.58s/it]
 83%|████████▎ | 187656/225000 [353:48:33<76:55:35,  7.42s/it]
 83%|████████▎ | 187657/225000 [353:48:40<75:48:36,  7.31s/it]
 83%|████████▎ | 187658/225000 [353:48:53<95:12:56,  9.18s/it]
 83%|████████▎ | 187659/225000 [353:48:59<83:52:42,  8.09s/it]
 83%|████████▎ | 187660/225000 [353:49:07<83:28:13,  8.05s/it]
 83%|████████▎ | 187661/225000 [353:49:11<70:39:05,  6.81s/it]
 83%|████████▎ | 187662/225000 [353:49:20<78:50:04,  7.60s/it]
 83%|████████▎ | 187663/225000 [353:49:24<66:26:48,  6.41s/it]
 83%|████████▎ | 187664/225000 [353:49:35<82:42:04,  7.97s/it]
 83%|████████▎ | 187665/225000 [353:49:39<69:59:49,  6.75s/it]
 83%|████████▎ | 187666/225000 [353:49:49<79:18:02,  7.65s/it]
 83%|████████▎ | 187667/225000 [353:49:53<66:40:35,  6.43s/it]
 83%|████████▎ | 187668/225000 [353:50:01<71:53:08,  6.93s/it]
 83%|████████▎ | 187669/225000 [353:50:13<88:21:47,  8.52s/it]
 83%|████████▎ | 187670/225000 [353:50:17<73:26:16,  7.08s/it]
 83%|████████▎ | 187671/225000 [353:50:20<63:14:23,  6.10s/it]
 83%|████████▎ | 187672/225000 [353:50:27<63:45:07,  6.15s/it]
 83%|████████▎ | 187673/225000 [353:50:33<62:53:38,  6.07s/it]
 83%|████████▎ | 187674/225000 [353:50:38<59:30:55,  5.74s/it]
 83%|████████▎ | 187675/225000 [353:50:46<69:16:53,  6.68s/it]
 83%|████████▎ | 187676/225000 [353:50:51<61:49:54,  5.96s/it]
 83%|████████▎ | 187677/225000 [353:50:56<59:07:38,  5.70s/it]
 83%|████████▎ | 187678/225000 [353:51:09<82:04:42,  7.92s/it]
 83%|████████▎ | 187679/225000 [353:51:14<73:42:12,  7.11s/it]
 83%|████████▎ | 187680/225000 [353:51:22<75:08:04,  7.25s/it]
 83%|████████▎ | 187681/225000 [353:51:25<64:11:55,  6.19s/it]
 83%|████████▎ | 187682/225000 [353:51:30<60:00:43,  5.79s/it]
 83%|████████▎ | 187683/225000 [353:51:39<68:52:16,  6.64s/it]
 83%|████████▎ | 187684/225000 [353:51:43<59:51:36,  5.77s/it]
 83%|████████▎ | 187685/225000 [353:51:48<57:56:29,  5.59s/it]
 83%|████████▎ | 187686/225000 [353:51:52<53:52:55,  5.20s/it]
 83%|████████▎ | 187687/225000 [353:52:05<76:44:04,  7.40s/it]
 83%|████████▎ | 187688/225000 [353:52:10<68:40:29,  6.63s/it]
 83%|████████▎ | 187689/225000 [353:52:20<82:07:03,  7.92s/it]
 83%|████████▎ | 187690/225000 [353:52:26<75:01:20,  7.24s/it]
 83%|████████▎ | 187691/225000 [353:52:33<75:22:55,  7.27s/it]
 83%|████████▎ | 187692/225000 [353:52:41<76:14:44,  7.36s/it]
 83%|████████▎ | 187693/225000 [353:52:45<66:58:27,  6.46s/it]
 83%|████████▎ | 187694/225000 [353:52:52<67:56:37,  6.56s/it]
 83%|████████▎ | 187695/225000 [353:53:02<79:30:14,  7.67s/it]
 83%|████████▎ | 187696/225000 [353:53:12<84:02:55,  8.11s/it]
 83%|████████▎ | 187697/225000 [353:53:16<73:14:16,  7.07s/it]
 83%|████████▎ | 187698/225000 [353:53:23<71:30:49,  6.90s/it]
 83%|████████▎ | 187699/225000 [353:53:27<63:28:28,  6.13s/it]
 83%|████████▎ | 187700/225000 [353:53:31<57:13:49,  5.52s/it]
                                                              
{'loss': '0.9096', 'grad_norm': '1.296', 'learning_rate': '3.639e-06', 'epoch': '0.8342', 'num_input_tokens_seen': 587326040, 'train_runtime': '1.274e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187700/225000 [353:53:31<57:13:49,  5.52s/it]
 83%|████████▎ | 187701/225000 [353:53:36<55:44:08,  5.38s/it]
 83%|████████▎ | 187702/225000 [353:53:42<56:24:38,  5.44s/it]
 83%|████████▎ | 187703/225000 [353:53:54<76:40:20,  7.40s/it]
 83%|████████▎ | 187704/225000 [353:54:01<76:46:24,  7.41s/it]
 83%|████████▎ | 187705/225000 [353:54:09<78:11:39,  7.55s/it]
 83%|████████▎ | 187706/225000 [353:54:17<79:37:19,  7.69s/it]
 83%|████████▎ | 187707/225000 [353:54:23<72:57:44,  7.04s/it]
 83%|████████▎ | 187708/225000 [353:54:35<91:02:49,  8.79s/it]
 83%|████████▎ | 187709/225000 [353:54:42<85:04:14,  8.21s/it]
 83%|████████▎ | 187710/225000 [353:54:52<88:09:43,  8.51s/it]
 83%|████████▎ | 187711/225000 [353:54:57<79:35:00,  7.68s/it]
 83%|████████▎ | 187712/225000 [353:55:04<75:27:31,  7.29s/it]
 83%|████████▎ | 187713/225000 [353:55:15<88:43:20,  8.57s/it]
 83%|████████▎ | 187714/225000 [353:55:24<87:58:40,  8.49s/it]
 83%|████████▎ | 187715/225000 [353:55:29<77:56:26,  7.53s/it]
 83%|████████▎ | 187716/225000 [353:55:36<78:16:14,  7.56s/it]
 83%|████████▎ | 187717/225000 [353:55:40<67:18:33,  6.50s/it]
 83%|████████▎ | 187718/225000 [353:55:46<64:42:22,  6.25s/it]
 83%|████████▎ | 187719/225000 [353:55:50<57:42:23,  5.57s/it]
 83%|████████▎ | 187720/225000 [353:55:55<55:44:12,  5.38s/it]
 83%|████████▎ | 187721/225000 [353:56:00<53:25:55,  5.16s/it]
 83%|████████▎ | 187722/225000 [353:56:05<53:59:29,  5.21s/it]
 83%|████████▎ | 187723/225000 [353:56:19<81:43:31,  7.89s/it]
 83%|████████▎ | 187724/225000 [353:56:24<71:43:17,  6.93s/it]
 83%|████████▎ | 187725/225000 [353:56:29<64:38:39,  6.24s/it]
 83%|████████▎ | 187726/225000 [353:56:32<56:56:37,  5.50s/it]
 83%|████████▎ | 187727/225000 [353:56:40<64:22:54,  6.22s/it]
 83%|████████▎ | 187728/225000 [353:56:50<76:04:29,  7.35s/it]
 83%|████████▎ | 187729/225000 [353:56:59<81:02:34,  7.83s/it]
 83%|████████▎ | 187730/225000 [353:57:05<76:09:55,  7.36s/it]
 83%|████████▎ | 187731/225000 [353:57:17<88:34:57,  8.56s/it]
 83%|████████▎ | 187732/225000 [353:57:21<75:55:19,  7.33s/it]
 83%|████████▎ | 187733/225000 [353:57:29<76:39:22,  7.41s/it]
 83%|████████▎ | 187734/225000 [353:57:36<76:02:04,  7.35s/it]
 83%|████████▎ | 187735/225000 [353:57:39<64:06:29,  6.19s/it]
 83%|████████▎ | 187736/225000 [353:57:54<89:01:57,  8.60s/it]
 83%|████████▎ | 187737/225000 [353:57:57<74:08:16,  7.16s/it]
 83%|████████▎ | 187738/225000 [353:58:04<71:00:02,  6.86s/it]
 83%|████████▎ | 187739/225000 [353:58:11<72:44:49,  7.03s/it]
 83%|████████▎ | 187740/225000 [353:58:16<65:11:11,  6.30s/it]
 83%|████████▎ | 187741/225000 [353:58:21<61:10:25,  5.91s/it]
 83%|████████▎ | 187742/225000 [353:58:26<58:51:39,  5.69s/it]
 83%|████████▎ | 187743/225000 [353:58:37<75:22:17,  7.28s/it]
 83%|████████▎ | 187744/225000 [353:58:44<76:30:35,  7.39s/it]
 83%|████████▎ | 187745/225000 [353:58:53<80:36:04,  7.79s/it]
 83%|████████▎ | 187746/225000 [353:58:57<67:39:30,  6.54s/it]
 83%|████████▎ | 187747/225000 [353:59:02<62:43:34,  6.06s/it]
 83%|████████▎ | 187748/225000 [353:59:12<75:36:24,  7.31s/it]
 83%|████████▎ | 187749/225000 [353:59:17<70:00:50,  6.77s/it]
 83%|████████▎ | 187750/225000 [353:59:22<62:15:53,  6.02s/it]
                                                              
{'loss': '0.956', 'grad_norm': '1.315', 'learning_rate': '3.629e-06', 'epoch': '0.8344', 'num_input_tokens_seen': 587500792, 'train_runtime': '1.274e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187750/225000 [353:59:22<62:15:53,  6.02s/it]
 83%|████████▎ | 187751/225000 [353:59:30<70:27:42,  6.81s/it]
 83%|████████▎ | 187752/225000 [353:59:36<67:57:06,  6.57s/it]
 83%|████████▎ | 187753/225000 [353:59:41<63:09:57,  6.11s/it]
 83%|████████▎ | 187754/225000 [353:59:53<79:37:42,  7.70s/it]
 83%|████████▎ | 187755/225000 [353:59:57<67:48:36,  6.55s/it]
 83%|████████▎ | 187756/225000 [354:00:04<68:45:42,  6.65s/it]
 83%|████████▎ | 187757/225000 [354:00:10<68:55:16,  6.66s/it]
 83%|████████▎ | 187758/225000 [354:00:19<74:25:01,  7.19s/it]
 83%|████████▎ | 187759/225000 [354:00:23<64:08:55,  6.20s/it]
 83%|████████▎ | 187760/225000 [354:00:27<58:47:30,  5.68s/it]
 83%|████████▎ | 187761/225000 [354:00:32<56:05:17,  5.42s/it]
 83%|████████▎ | 187762/225000 [354:00:38<58:31:23,  5.66s/it]
 83%|████████▎ | 187763/225000 [354:00:44<60:45:22,  5.87s/it]
 83%|████████▎ | 187764/225000 [354:00:49<55:39:21,  5.38s/it]
 83%|████████▎ | 187765/225000 [354:01:01<77:40:13,  7.51s/it]
 83%|████████▎ | 187766/225000 [354:01:07<72:00:33,  6.96s/it]
 83%|████████▎ | 187767/225000 [354:01:14<71:01:01,  6.87s/it]
 83%|████████▎ | 187768/225000 [354:01:18<62:51:13,  6.08s/it]
 83%|████████▎ | 187769/225000 [354:01:29<78:21:59,  7.58s/it]
 83%|████████▎ | 187770/225000 [354:01:38<82:02:17,  7.93s/it]
 83%|████████▎ | 187771/225000 [354:01:42<70:13:57,  6.79s/it]
 83%|████████▎ | 187772/225000 [354:01:51<79:09:46,  7.66s/it]
 83%|████████▎ | 187773/225000 [354:02:01<86:36:00,  8.37s/it]
 83%|████████▎ | 187774/225000 [354:02:12<93:14:37,  9.02s/it]
 83%|████████▎ | 187775/225000 [354:02:17<80:04:07,  7.74s/it]
 83%|████████▎ | 187776/225000 [354:02:22<70:48:08,  6.85s/it]
 83%|████████▎ | 187777/225000 [354:02:27<66:27:17,  6.43s/it]
 83%|████████▎ | 187778/225000 [354:02:30<57:13:45,  5.54s/it]
 83%|████████▎ | 187779/225000 [354:02:40<70:38:17,  6.83s/it]
 83%|████████▎ | 187780/225000 [354:02:45<63:12:59,  6.11s/it]
 83%|████████▎ | 187781/225000 [354:02:51<62:26:24,  6.04s/it]
 83%|████████▎ | 187782/225000 [354:03:02<78:28:01,  7.59s/it]
 83%|████████▎ | 187783/225000 [354:03:05<65:57:59,  6.38s/it]
 83%|████████▎ | 187784/225000 [354:03:15<75:11:46,  7.27s/it]
 83%|████████▎ | 187785/225000 [354:03:19<67:21:16,  6.52s/it]
 83%|████████▎ | 187786/225000 [354:03:25<65:02:25,  6.29s/it]
 83%|████████▎ | 187787/225000 [354:03:29<58:29:56,  5.66s/it]
 83%|████████▎ | 187788/225000 [354:03:36<60:28:49,  5.85s/it]
 83%|████████▎ | 187789/225000 [354:03:40<56:31:14,  5.47s/it]
 83%|████████▎ | 187790/225000 [354:03:49<67:33:58,  6.54s/it]
 83%|████████▎ | 187791/225000 [354:03:59<76:26:53,  7.40s/it]
 83%|████████▎ | 187792/225000 [354:04:12<94:02:12,  9.10s/it]
 83%|████████▎ | 187793/225000 [354:04:15<76:44:33,  7.43s/it]
 83%|████████▎ | 187794/225000 [354:04:22<75:46:03,  7.33s/it]
 83%|████████▎ | 187795/225000 [354:04:34<90:12:03,  8.73s/it]
 83%|████████▎ | 187796/225000 [354:04:40<81:36:46,  7.90s/it]
 83%|████████▎ | 187797/225000 [354:04:45<72:04:57,  6.98s/it]
 83%|████████▎ | 187798/225000 [354:04:50<64:59:04,  6.29s/it]
 83%|████████▎ | 187799/225000 [354:04:55<60:47:43,  5.88s/it]
 83%|████████▎ | 187800/225000 [354:05:07<79:54:32,  7.73s/it]
                                                              
{'loss': '0.9552', 'grad_norm': '1.467', 'learning_rate': '3.62e-06', 'epoch': '0.8347', 'num_input_tokens_seen': 587667624, 'train_runtime': '1.275e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187800/225000 [354:05:07<79:54:32,  7.73s/it]
 83%|████████▎ | 187801/225000 [354:05:12<72:04:57,  6.98s/it]
 83%|████████▎ | 187802/225000 [354:05:17<64:42:06,  6.26s/it]
 83%|████████▎ | 187803/225000 [354:05:21<59:55:15,  5.80s/it]
 83%|████████▎ | 187804/225000 [354:05:26<56:24:54,  5.46s/it]
 83%|████████▎ | 187805/225000 [354:05:36<70:16:59,  6.80s/it]
 83%|████████▎ | 187806/225000 [354:05:41<66:10:22,  6.40s/it]
 83%|████████▎ | 187807/225000 [354:05:48<65:14:26,  6.31s/it]
 83%|████████▎ | 187808/225000 [354:05:53<64:03:01,  6.20s/it]
 83%|████████▎ | 187809/225000 [354:06:01<69:33:02,  6.73s/it]
 83%|████████▎ | 187810/225000 [354:06:07<65:42:21,  6.36s/it]
 83%|████████▎ | 187811/225000 [354:06:11<59:47:55,  5.79s/it]
 83%|████████▎ | 187812/225000 [354:06:19<66:33:27,  6.44s/it]
 83%|████████▎ | 187813/225000 [354:06:25<63:28:40,  6.15s/it]
 83%|████████▎ | 187814/225000 [354:06:31<63:39:44,  6.16s/it]
 83%|████████▎ | 187815/225000 [354:06:35<57:33:16,  5.57s/it]
 83%|████████▎ | 187816/225000 [354:06:44<65:59:24,  6.39s/it]
 83%|████████▎ | 187817/225000 [354:06:48<60:56:43,  5.90s/it]
 83%|████████▎ | 187818/225000 [354:06:54<61:19:01,  5.94s/it]
 83%|████████▎ | 187819/225000 [354:07:00<59:38:57,  5.78s/it]
 83%|████████▎ | 187820/225000 [354:07:13<83:14:04,  8.06s/it]
 83%|████████▎ | 187821/225000 [354:07:20<79:41:55,  7.72s/it]
 83%|████████▎ | 187822/225000 [354:07:24<67:53:40,  6.57s/it]
 83%|████████▎ | 187823/225000 [354:07:32<71:49:44,  6.96s/it]
 83%|████████▎ | 187824/225000 [354:07:36<64:05:32,  6.21s/it]
 83%|████████▎ | 187825/225000 [354:07:49<83:01:49,  8.04s/it]
 83%|████████▎ | 187826/225000 [354:07:52<69:05:04,  6.69s/it]
 83%|████████▎ | 187827/225000 [354:08:00<73:53:31,  7.16s/it]
 83%|████████▎ | 187828/225000 [354:08:06<68:35:15,  6.64s/it]
 83%|████████▎ | 187829/225000 [354:08:14<74:39:16,  7.23s/it]
 83%|████████▎ | 187830/225000 [354:08:26<87:36:50,  8.49s/it]
 83%|████████▎ | 187831/225000 [354:08:31<78:11:45,  7.57s/it]
 83%|████████▎ | 187832/225000 [354:08:37<72:13:09,  6.99s/it]
 83%|████████▎ | 187833/225000 [354:08:47<81:02:12,  7.85s/it]
 83%|████████▎ | 187834/225000 [354:08:51<70:04:05,  6.79s/it]
 83%|████████▎ | 187835/225000 [354:09:01<78:49:22,  7.64s/it]
 83%|████████▎ | 187836/225000 [354:09:05<67:10:38,  6.51s/it]
 83%|████████▎ | 187837/225000 [354:09:13<74:24:32,  7.21s/it]
 83%|████████▎ | 187838/225000 [354:09:21<77:14:45,  7.48s/it]
 83%|████████▎ | 187839/225000 [354:09:28<73:20:19,  7.10s/it]
 83%|████████▎ | 187840/225000 [354:09:35<73:24:00,  7.11s/it]
 83%|████████▎ | 187841/225000 [354:09:40<65:52:10,  6.38s/it]
 83%|████████▎ | 187842/225000 [354:09:49<74:39:39,  7.23s/it]
 83%|████████▎ | 187843/225000 [354:09:58<80:11:53,  7.77s/it]
 83%|████████▎ | 187844/225000 [354:10:02<69:59:01,  6.78s/it]
 83%|████████▎ | 187845/225000 [354:10:11<75:46:32,  7.34s/it]
 83%|████████▎ | 187846/225000 [354:10:17<71:13:08,  6.90s/it]
 83%|████████▎ | 187847/225000 [354:10:22<67:08:11,  6.51s/it]
 83%|████████▎ | 187848/225000 [354:10:31<75:21:37,  7.30s/it]
 83%|████████▎ | 187849/225000 [354:10:35<65:01:13,  6.30s/it]
 83%|████████▎ | 187850/225000 [354:10:41<61:40:36,  5.98s/it]
                                                              
{'loss': '0.9696', 'grad_norm': '1.107', 'learning_rate': '3.61e-06', 'epoch': '0.8349', 'num_input_tokens_seen': 587832944, 'train_runtime': '1.275e+06', 'train_tokens_per_second': '461'}

 83%|████████▎ | 187850/225000 [354:10:41<61:40:36,  5.98s/it]
 83%|████████▎ | 187851/225000 [354:10:45<55:16:28,  5.36s/it]
 83%|████████▎ | 187852/225000 [354:10:56<74:00:16,  7.17s/it]
 83%|████████▎ | 187853/225000 [354:11:02<68:59:14,  6.69s/it]
 83%|████████▎ | 187854/225000 [354:11:08<67:51:50,  6.58s/it]
 83%|████████▎ | 187855/225000 [354:11:11<58:24:52,  5.66s/it]
 83%|████████▎ | 187856/225000 [354:11:17<58:23:46,  5.66s/it]
 83%|████████▎ | 187857/225000 [354:11:27<70:54:32,  6.87s/it]
 83%|████████▎ | 187858/225000 [354:11:32<65:01:37,  6.30s/it]
 83%|████████▎ | 187859/225000 [354:11:37<60:29:30,  5.86s/it]
 83%|████████▎ | 187860/225000 [354:11:40<53:50:21,  5.22s/it]
 83%|████████▎ | 187861/225000 [354:11:51<69:48:04,  6.77s/it]
 83%|████████▎ | 187862/225000 [354:11:55<61:46:54,  5.99s/it]
 83%|████████▎ | 187863/225000 [354:12:03<68:26:14,  6.63s/it]
 83%|████████▎ | 187864/225000 [354:12:17<92:33:50,  8.97s/it]
 83%|████████▎ | 187865/225000 [354:12:21<75:35:53,  7.33s/it]
 83%|████████▎ | 187866/225000 [354:12:27<72:14:11,  7.00s/it]
 83%|████████▎ | 187867/225000 [354:12:33<69:14:13,  6.71s/it]
 83%|████████▎ | 187868/225000 [354:12:44<83:04:03,  8.05s/it]
 83%|████████▎ | 187869/225000 [354:12:50<76:26:03,  7.41s/it]
 83%|████████▎ | 187870/225000 [354:13:00<84:24:45,  8.18s/it]
 83%|████████▎ | 187871/225000 [354:13:10<89:48:02,  8.71s/it]
 83%|████████▎ | 187872/225000 [354:13:23<103:49:56, 10.07s/it]
 83%|████████▎ | 187873/225000 [354:13:34<104:14:22, 10.11s/it]
 83%|████████▎ | 187874/225000 [354:13:40<93:57:39,  9.11s/it] 
 84%|████████▎ | 187875/225000 [354:13:45<78:28:10,  7.61s/it]
 84%|████████▎ | 187876/225000 [354:13:49<67:24:27,  6.54s/it]
 84%|████████▎ | 187877/225000 [354:14:00<83:24:07,  8.09s/it]
 84%|████████▎ | 187878/225000 [354:14:05<73:17:53,  7.11s/it]
 84%|████████▎ | 187879/225000 [354:14:08<61:32:25,  5.97s/it]
 84%|████████▎ | 187880/225000 [354:14:16<67:41:58,  6.57s/it]
 84%|████████▎ | 187881/225000 [354:14:23<66:49:44,  6.48s/it]
 84%|████████▎ | 187882/225000 [354:14:26<57:24:53,  5.57s/it]
 84%|████████▎ | 187883/225000 [354:14:38<77:36:34,  7.53s/it]
 84%|████████▎ | 187884/225000 [354:14:42<67:13:03,  6.52s/it]
 84%|████████▎ | 187885/225000 [354:14:47<60:53:37,  5.91s/it]
 84%|████████▎ | 187886/225000 [354:14:52<57:20:25,  5.56s/it]
 84%|████████▎ | 187887/225000 [354:14:57<57:54:13,  5.62s/it]
 84%|████████▎ | 187888/225000 [354:15:07<70:24:55,  6.83s/it]
 84%|████████▎ | 187889/225000 [354:15:16<78:38:57,  7.63s/it]
 84%|████████▎ | 187890/225000 [354:15:21<70:00:03,  6.79s/it]
 84%|████████▎ | 187891/225000 [354:15:33<84:43:25,  8.22s/it]
 84%|████████▎ | 187892/225000 [354:15:43<90:47:21,  8.81s/it]
 84%|████████▎ | 187893/225000 [354:15:50<86:24:20,  8.38s/it]
 84%|████████▎ | 187894/225000 [354:15:56<79:07:00,  7.68s/it]
 84%|████████▎ | 187895/225000 [354:16:08<89:37:13,  8.70s/it]
 84%|████████▎ | 187896/225000 [354:16:11<74:47:45,  7.26s/it]
 84%|████████▎ | 187897/225000 [354:16:17<69:14:30,  6.72s/it]
 84%|████████▎ | 187898/225000 [354:16:24<69:48:27,  6.77s/it]
 84%|████████▎ | 187899/225000 [354:16:29<66:09:49,  6.42s/it]
 84%|████████▎ | 187900/225000 [354:16:43<88:40:14,  8.60s/it]
                                                              
{'loss': '0.9464', 'grad_norm': '1.086', 'learning_rate': '3.601e-06', 'epoch': '0.8351', 'num_input_tokens_seen': 588010024, 'train_runtime': '1.275e+06', 'train_tokens_per_second': '461'}

 84%|████████▎ | 187900/225000 [354:16:43<88:40:14,  8.60s/it]
 84%|████████▎ | 187901/225000 [354:16:48<76:31:16,  7.43s/it]
 84%|████████▎ | 187902/225000 [354:17:01<94:56:08,  9.21s/it]
 84%|████████▎ | 187903/225000 [354:17:06<80:33:09,  7.82s/it]
 84%|████████▎ | 187904/225000 [354:17:09<67:19:34,  6.53s/it]
 84%|████████▎ | 187905/225000 [354:17:20<81:37:00,  7.92s/it]
 84%|████████▎ | 187906/225000 [354:17:26<74:06:56,  7.19s/it]
 84%|████████▎ | 187907/225000 [354:17:29<62:32:27,  6.07s/it]
 84%|████████▎ | 187908/225000 [354:17:40<77:47:40,  7.55s/it]
 84%|████████▎ | 187909/225000 [354:17:44<65:03:54,  6.32s/it]
 84%|████████▎ | 187910/225000 [354:17:53<72:36:06,  7.05s/it]
 84%|████████▎ | 187911/225000 [354:17:57<65:51:09,  6.39s/it]
 84%|████████▎ | 187912/225000 [354:18:04<67:32:03,  6.56s/it]
 84%|████████▎ | 187913/225000 [354:18:13<73:06:01,  7.10s/it]
 84%|████████▎ | 187914/225000 [354:18:18<67:10:15,  6.52s/it]
 84%|████████▎ | 187915/225000 [354:18:30<84:29:15,  8.20s/it]
 84%|████████▎ | 187916/225000 [354:18:36<77:01:38,  7.48s/it]
 84%|████████▎ | 187917/225000 [354:18:42<73:37:16,  7.15s/it]
 84%|████████▎ | 187918/225000 [354:18:53<84:53:22,  8.24s/it]
 84%|████████▎ | 187919/225000 [354:19:01<83:48:20,  8.14s/it]
 84%|████████▎ | 187920/225000 [354:19:07<77:35:46,  7.53s/it]
 84%|████████▎ | 187921/225000 [354:19:11<68:15:42,  6.63s/it]
 84%|████████▎ | 187922/225000 [354:19:23<84:24:21,  8.20s/it]
 84%|████████▎ | 187923/225000 [354:19:30<80:48:47,  7.85s/it]
 84%|████████▎ | 187924/225000 [354:19:38<78:45:53,  7.65s/it]
 84%|████████▎ | 187925/225000 [354:19:41<66:37:14,  6.47s/it]
 84%|████████▎ | 187926/225000 [354:19:46<59:46:28,  5.80s/it]
 84%|████████▎ | 187927/225000 [354:19:50<57:10:44,  5.55s/it]
 84%|████████▎ | 187928/225000 [354:19:59<66:24:27,  6.45s/it]
 84%|████████▎ | 187929/225000 [354:20:08<74:15:12,  7.21s/it]
 84%|████████▎ | 187930/225000 [354:20:18<82:28:27,  8.01s/it]
 84%|████████▎ | 187931/225000 [354:20:22<71:14:17,  6.92s/it]
 84%|████████▎ | 187932/225000 [354:20:27<64:28:19,  6.26s/it]
 84%|████████▎ | 187933/225000 [354:20:32<59:12:54,  5.75s/it]
 84%|████████▎ | 187934/225000 [354:20:42<74:32:29,  7.24s/it]
 84%|████████▎ | 187935/225000 [354:20:46<64:38:11,  6.28s/it]
 84%|████████▎ | 187936/225000 [354:20:57<79:14:07,  7.70s/it]
 84%|████████▎ | 187937/225000 [354:21:06<82:33:06,  8.02s/it]
 84%|████████▎ | 187938/225000 [354:21:11<73:04:15,  7.10s/it]
 84%|████████▎ | 187939/225000 [354:21:21<82:13:22,  7.99s/it]
 84%|████████▎ | 187940/225000 [354:21:26<71:16:36,  6.92s/it]
 84%|████████▎ | 187941/225000 [354:21:30<63:58:29,  6.21s/it]
 84%|████████▎ | 187942/225000 [354:21:34<57:54:27,  5.63s/it]
 84%|████████▎ | 187943/225000 [354:21:46<75:31:46,  7.34s/it]
 84%|████████▎ | 187944/225000 [354:21:51<68:00:03,  6.61s/it]
 84%|████████▎ | 187945/225000 [354:22:01<81:01:05,  7.87s/it]
 84%|████████▎ | 187946/225000 [354:22:09<81:02:34,  7.87s/it]
 84%|████████▎ | 187947/225000 [354:22:15<74:49:14,  7.27s/it]
 84%|████████▎ | 187948/225000 [354:22:25<83:05:45,  8.07s/it]
 84%|████████▎ | 187949/225000 [354:22:29<69:38:07,  6.77s/it]
 84%|████████▎ | 187950/225000 [354:22:42<89:19:07,  8.68s/it]
                                                              
{'loss': '0.9817', 'grad_norm': '1.088', 'learning_rate': '3.592e-06', 'epoch': '0.8353', 'num_input_tokens_seen': 588187984, 'train_runtime': '1.276e+06', 'train_tokens_per_second': '461'}

 84%|████████▎ | 187950/225000 [354:22:42<89:19:07,  8.68s/it]
 84%|████████▎ | 187951/225000 [354:22:52<94:20:50,  9.17s/it]
 84%|████████▎ | 187952/225000 [354:23:00<91:19:22,  8.87s/it]
 84%|████████▎ | 187953/225000 [354:23:04<75:58:31,  7.38s/it]
 84%|████████▎ | 187954/225000 [354:23:09<66:10:56,  6.43s/it]
 84%|████████▎ | 187955/225000 [354:23:13<60:03:24,  5.84s/it]
 84%|████████▎ | 187956/225000 [354:23:19<60:14:41,  5.85s/it]
 84%|████████▎ | 187957/225000 [354:23:28<71:17:11,  6.93s/it]
 84%|████████▎ | 187958/225000 [354:23:37<76:11:18,  7.40s/it]
 84%|████████▎ | 187959/225000 [354:23:47<84:38:06,  8.23s/it]
 84%|████████▎ | 187960/225000 [354:23:50<70:02:27,  6.81s/it]
 84%|████████▎ | 187961/225000 [354:23:58<71:16:38,  6.93s/it]
 84%|████████▎ | 187962/225000 [354:24:03<65:08:37,  6.33s/it]
 84%|████████▎ | 187963/225000 [354:24:08<61:38:22,  5.99s/it]
 84%|████████▎ | 187964/225000 [354:24:17<69:54:47,  6.80s/it]
 84%|████████▎ | 187965/225000 [354:24:23<68:36:15,  6.67s/it]
 84%|████████▎ | 187966/225000 [354:24:31<73:18:15,  7.13s/it]
 84%|████████▎ | 187967/225000 [354:24:35<64:03:15,  6.23s/it]
 84%|████████▎ | 187968/225000 [354:24:41<62:18:50,  6.06s/it]
 84%|████████▎ | 187969/225000 [354:24:47<62:18:07,  6.06s/it]
 84%|████████▎ | 187970/225000 [354:24:53<63:07:51,  6.14s/it]
 84%|████████▎ | 187971/225000 [354:25:01<67:40:39,  6.58s/it]
 84%|████████▎ | 187972/225000 [354:25:06<64:10:04,  6.24s/it]
 84%|████████▎ | 187973/225000 [354:25:19<84:34:14,  8.22s/it]
 84%|████████▎ | 187974/225000 [354:25:29<90:45:56,  8.83s/it]
 84%|████████▎ | 187975/225000 [354:25:35<80:32:36,  7.83s/it]
 84%|████████▎ | 187976/225000 [354:25:41<74:10:11,  7.21s/it]
 84%|████████▎ | 187977/225000 [354:25:50<80:39:13,  7.84s/it]
 84%|████████▎ | 187978/225000 [354:25:56<73:57:33,  7.19s/it]
 84%|████████▎ | 187979/225000 [354:25:59<63:04:35,  6.13s/it]
 84%|████████▎ | 187980/225000 [354:26:10<75:33:14,  7.35s/it]
 84%|████████▎ | 187981/225000 [354:26:14<65:31:21,  6.37s/it]
 84%|████████▎ | 187982/225000 [354:26:22<71:16:49,  6.93s/it]
 84%|████████▎ | 187983/225000 [354:26:27<66:29:15,  6.47s/it]
 84%|████████▎ | 187984/225000 [354:26:33<64:39:23,  6.29s/it]
 84%|████████▎ | 187985/225000 [354:26:42<73:33:38,  7.15s/it]
 84%|████████▎ | 187986/225000 [354:26:47<67:10:49,  6.53s/it]
 84%|████████▎ | 187987/225000 [354:26:52<60:36:47,  5.90s/it]
 84%|████████▎ | 187988/225000 [354:27:02<73:40:47,  7.17s/it]
 84%|████████▎ | 187989/225000 [354:27:09<73:16:40,  7.13s/it]
 84%|████████▎ | 187990/225000 [354:27:17<75:50:44,  7.38s/it]
 84%|████████▎ | 187991/225000 [354:27:21<64:53:08,  6.31s/it]
 84%|████████▎ | 187992/225000 [354:27:29<69:48:15,  6.79s/it]
 84%|████████▎ | 187993/225000 [354:27:35<67:32:09,  6.57s/it]
 84%|████████▎ | 187994/225000 [354:27:39<60:38:32,  5.90s/it]
 84%|████████▎ | 187995/225000 [354:27:49<74:41:11,  7.27s/it]
 84%|████████▎ | 187996/225000 [354:27:53<62:07:40,  6.04s/it]
 84%|████████▎ | 187997/225000 [354:28:02<72:30:10,  7.05s/it]
 84%|████████▎ | 187998/225000 [354:28:06<62:05:17,  6.04s/it]
 84%|████████▎ | 187999/225000 [354:28:13<66:28:55,  6.47s/it]
 84%|████████▎ | 188000/225000 [354:28:24<79:46:04,  7.76s/it]
                                                              
{'loss': '0.9723', 'grad_norm': '1.421', 'learning_rate': '3.582e-06', 'epoch': '0.8356', 'num_input_tokens_seen': 588357824, 'train_runtime': '1.276e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188000/225000 [354:28:24<79:46:04,  7.76s/it]
 84%|████████▎ | 188001/225000 [354:28:28<67:04:42,  6.53s/it]
 84%|████████▎ | 188002/225000 [354:28:38<79:11:54,  7.71s/it]
 84%|████████▎ | 188003/225000 [354:28:45<77:57:35,  7.59s/it]
 84%|████████▎ | 188004/225000 [354:28:51<72:39:51,  7.07s/it]
 84%|████████▎ | 188005/225000 [354:28:58<71:05:57,  6.92s/it]
 84%|████████▎ | 188006/225000 [354:29:12<92:46:58,  9.03s/it]
 84%|████████▎ | 188007/225000 [354:29:19<87:38:42,  8.53s/it]
 84%|████████▎ | 188008/225000 [354:29:24<77:09:24,  7.51s/it]
 84%|████████▎ | 188009/225000 [354:29:33<80:48:06,  7.86s/it]
 84%|████████▎ | 188010/225000 [354:29:37<69:36:59,  6.78s/it]
 84%|████████▎ | 188011/225000 [354:29:44<69:05:16,  6.72s/it]
 84%|████████▎ | 188012/225000 [354:29:50<68:29:16,  6.67s/it]
 84%|████████▎ | 188013/225000 [354:29:59<74:59:10,  7.30s/it]
 84%|████████▎ | 188014/225000 [354:30:04<68:30:43,  6.67s/it]
 84%|████████▎ | 188015/225000 [354:30:13<75:02:49,  7.30s/it]
 84%|████████▎ | 188016/225000 [354:30:18<67:34:33,  6.58s/it]
 84%|████████▎ | 188017/225000 [354:30:25<67:38:16,  6.58s/it]
 84%|████████▎ | 188018/225000 [354:30:32<71:32:47,  6.96s/it]
 84%|████████▎ | 188019/225000 [354:30:43<82:36:54,  8.04s/it]
 84%|████████▎ | 188020/225000 [354:30:48<73:56:23,  7.20s/it]
 84%|████████▎ | 188021/225000 [354:30:52<63:39:37,  6.20s/it]
 84%|████████▎ | 188022/225000 [354:30:59<66:41:06,  6.49s/it]
 84%|████████▎ | 188023/225000 [354:31:06<66:28:35,  6.47s/it]
 84%|████████▎ | 188024/225000 [354:31:11<63:17:16,  6.16s/it]
 84%|████████▎ | 188025/225000 [354:31:19<69:09:33,  6.73s/it]
 84%|████████▎ | 188026/225000 [354:31:26<69:03:32,  6.72s/it]
 84%|████████▎ | 188027/225000 [354:31:29<58:38:42,  5.71s/it]
 84%|████████▎ | 188028/225000 [354:31:37<66:23:46,  6.47s/it]
 84%|████████▎ | 188029/225000 [354:31:41<58:21:04,  5.68s/it]
 84%|████████▎ | 188030/225000 [354:31:50<66:47:47,  6.50s/it]
 84%|████████▎ | 188031/225000 [354:31:56<66:11:02,  6.44s/it]
 84%|████████▎ | 188032/225000 [354:32:02<65:44:26,  6.40s/it]
 84%|████████▎ | 188033/225000 [354:32:07<60:44:37,  5.92s/it]
 84%|████████▎ | 188034/225000 [354:32:11<55:09:13,  5.37s/it]
 84%|████████▎ | 188035/225000 [354:32:16<54:46:39,  5.33s/it]
 84%|████████▎ | 188036/225000 [354:32:32<86:53:03,  8.46s/it]
 84%|████████▎ | 188037/225000 [354:32:36<71:40:30,  6.98s/it]
 84%|████████▎ | 188038/225000 [354:32:43<71:58:50,  7.01s/it]
 84%|████████▎ | 188039/225000 [354:32:47<63:23:53,  6.17s/it]
 84%|████████▎ | 188040/225000 [354:32:55<68:07:35,  6.64s/it]
 84%|████████▎ | 188041/225000 [354:32:59<61:29:01,  5.99s/it]
 84%|████████▎ | 188042/225000 [354:33:09<72:28:55,  7.06s/it]
 84%|████████▎ | 188043/225000 [354:33:14<65:22:19,  6.37s/it]
 84%|████████▎ | 188044/225000 [354:33:22<70:22:30,  6.86s/it]
 84%|████████▎ | 188045/225000 [354:33:25<59:48:13,  5.83s/it]
 84%|████████▎ | 188046/225000 [354:33:34<69:32:33,  6.77s/it]
 84%|████████▎ | 188047/225000 [354:33:39<62:45:51,  6.11s/it]
 84%|████████▎ | 188048/225000 [354:33:48<72:45:50,  7.09s/it]
 84%|████████▎ | 188049/225000 [354:33:57<77:50:18,  7.58s/it]
 84%|████████▎ | 188050/225000 [354:34:01<68:09:00,  6.64s/it]
                                                              
{'loss': '1.027', 'grad_norm': '1.747', 'learning_rate': '3.573e-06', 'epoch': '0.8358', 'num_input_tokens_seen': 588526000, 'train_runtime': '1.276e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188050/225000 [354:34:01<68:09:00,  6.64s/it]
 84%|████████▎ | 188051/225000 [354:34:12<82:37:28,  8.05s/it]
 84%|████████▎ | 188052/225000 [354:34:17<71:32:03,  6.97s/it]
 84%|████████▎ | 188053/225000 [354:34:22<66:03:04,  6.44s/it]
 84%|████████▎ | 188054/225000 [354:34:30<71:58:26,  7.01s/it]
 84%|████████▎ | 188055/225000 [354:34:38<72:40:05,  7.08s/it]
 84%|████████▎ | 188056/225000 [354:34:43<66:18:44,  6.46s/it]
 84%|████████▎ | 188057/225000 [354:34:47<60:40:03,  5.91s/it]
 84%|████████▎ | 188058/225000 [354:35:03<89:46:22,  8.75s/it]
 84%|████████▎ | 188059/225000 [354:35:09<81:32:28,  7.95s/it]
 84%|████████▎ | 188060/225000 [354:35:12<66:58:52,  6.53s/it]
 84%|████████▎ | 188061/225000 [354:35:17<60:55:29,  5.94s/it]
 84%|████████▎ | 188062/225000 [354:35:23<62:14:18,  6.07s/it]
 84%|████████▎ | 188063/225000 [354:35:32<72:21:36,  7.05s/it]
 84%|████████▎ | 188064/225000 [354:35:38<68:32:44,  6.68s/it]
 84%|████████▎ | 188065/225000 [354:35:43<62:22:34,  6.08s/it]
 84%|████████▎ | 188066/225000 [354:35:48<60:34:08,  5.90s/it]
 84%|████████▎ | 188067/225000 [354:36:04<91:22:49,  8.91s/it]
 84%|████████▎ | 188068/225000 [354:36:11<86:29:09,  8.43s/it]
 84%|████████▎ | 188069/225000 [354:36:17<77:15:27,  7.53s/it]
 84%|████████▎ | 188070/225000 [354:36:26<82:23:05,  8.03s/it]
 84%|████████▎ | 188071/225000 [354:36:30<71:06:40,  6.93s/it]
 84%|████████▎ | 188072/225000 [354:36:37<70:44:49,  6.90s/it]
 84%|████████▎ | 188073/225000 [354:36:47<79:29:46,  7.75s/it]
 84%|████████▎ | 188074/225000 [354:36:53<74:43:29,  7.29s/it]
 84%|████████▎ | 188075/225000 [354:37:02<78:56:13,  7.70s/it]
 84%|████████▎ | 188076/225000 [354:37:07<69:46:12,  6.80s/it]
 84%|████████▎ | 188077/225000 [354:37:12<64:45:51,  6.31s/it]
 84%|████████▎ | 188078/225000 [354:37:22<78:11:18,  7.62s/it]
 84%|████████▎ | 188079/225000 [354:37:28<73:09:56,  7.13s/it]
 84%|████████▎ | 188080/225000 [354:37:33<66:40:31,  6.50s/it]
 84%|████████▎ | 188081/225000 [354:37:40<66:43:13,  6.51s/it]
 84%|████████▎ | 188082/225000 [354:37:53<85:29:02,  8.34s/it]
 84%|████████▎ | 188083/225000 [354:37:56<70:37:41,  6.89s/it]
 84%|████████▎ | 188084/225000 [354:38:03<69:57:43,  6.82s/it]
 84%|████████▎ | 188085/225000 [354:38:07<61:28:56,  6.00s/it]
 84%|████████▎ | 188086/225000 [354:38:15<67:42:09,  6.60s/it]
 84%|████████▎ | 188087/225000 [354:38:22<69:14:34,  6.75s/it]
 84%|████████▎ | 188088/225000 [354:38:28<67:10:03,  6.55s/it]
 84%|████████▎ | 188089/225000 [354:38:36<71:46:01,  7.00s/it]
 84%|████████▎ | 188090/225000 [354:38:43<72:14:18,  7.05s/it]
 84%|████████▎ | 188091/225000 [354:38:53<80:49:15,  7.88s/it]
 84%|████████▎ | 188092/225000 [354:39:02<82:49:02,  8.08s/it]
 84%|████████▎ | 188093/225000 [354:39:18<107:10:38, 10.45s/it]
 84%|████████▎ | 188094/225000 [354:39:21<86:50:08,  8.47s/it] 
 84%|████████▎ | 188095/225000 [354:39:30<88:35:13,  8.64s/it]
 84%|████████▎ | 188096/225000 [354:39:43<99:18:50,  9.69s/it]
 84%|████████▎ | 188097/225000 [354:39:48<84:55:11,  8.28s/it]
 84%|████████▎ | 188098/225000 [354:39:54<80:28:31,  7.85s/it]
 84%|████████▎ | 188099/225000 [354:40:01<75:29:40,  7.37s/it]
 84%|████████▎ | 188100/225000 [354:40:13<89:53:37,  8.77s/it]
                                                              
{'loss': '1.031', 'grad_norm': '1.132', 'learning_rate': '3.563e-06', 'epoch': '0.836', 'num_input_tokens_seen': 588707168, 'train_runtime': '1.277e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188100/225000 [354:40:13<89:53:37,  8.77s/it]
 84%|████████▎ | 188101/225000 [354:40:17<76:56:44,  7.51s/it]
 84%|████████▎ | 188102/225000 [354:40:21<65:59:56,  6.44s/it]
 84%|████████▎ | 188103/225000 [354:40:25<58:17:02,  5.69s/it]
 84%|████████▎ | 188104/225000 [354:40:29<53:24:40,  5.21s/it]
 84%|████████▎ | 188105/225000 [354:40:41<74:27:15,  7.26s/it]
 84%|████████▎ | 188106/225000 [354:40:47<70:02:55,  6.84s/it]
 84%|████████▎ | 188107/225000 [354:40:56<76:16:21,  7.44s/it]
 84%|████████▎ | 188108/225000 [354:41:06<82:37:30,  8.06s/it]
 84%|████████▎ | 188109/225000 [354:41:13<81:55:58,  8.00s/it]
 84%|████████▎ | 188110/225000 [354:41:24<90:43:48,  8.85s/it]
 84%|████████▎ | 188111/225000 [354:41:27<73:17:01,  7.15s/it]
 84%|████████▎ | 188112/225000 [354:41:32<64:42:16,  6.31s/it]
 84%|████████▎ | 188113/225000 [354:41:43<80:28:12,  7.85s/it]
 84%|████████▎ | 188114/225000 [354:41:50<76:19:03,  7.45s/it]
 84%|████████▎ | 188115/225000 [354:42:00<86:24:44,  8.43s/it]
 84%|████████▎ | 188116/225000 [354:42:09<86:45:59,  8.47s/it]
 84%|████████▎ | 188117/225000 [354:42:18<89:12:10,  8.71s/it]
 84%|████████▎ | 188118/225000 [354:42:23<76:12:48,  7.44s/it]
 84%|████████▎ | 188119/225000 [354:42:27<65:49:30,  6.43s/it]
 84%|████████▎ | 188120/225000 [354:42:34<67:09:30,  6.56s/it]
 84%|████████▎ | 188121/225000 [354:42:41<70:11:56,  6.85s/it]
 84%|████████▎ | 188122/225000 [354:42:50<75:06:10,  7.33s/it]
 84%|████████▎ | 188123/225000 [354:43:00<84:38:05,  8.26s/it]
 84%|████████▎ | 188124/225000 [354:43:05<73:52:49,  7.21s/it]
 84%|████████▎ | 188125/225000 [354:43:10<66:01:59,  6.45s/it]
 84%|████████▎ | 188126/225000 [354:43:14<60:48:35,  5.94s/it]
 84%|████████▎ | 188127/225000 [354:43:23<70:01:56,  6.84s/it]
 84%|████████▎ | 188128/225000 [354:43:32<76:03:18,  7.43s/it]
 84%|████████▎ | 188129/225000 [354:43:37<69:45:32,  6.81s/it]
 84%|████████▎ | 188130/225000 [354:43:43<65:41:43,  6.41s/it]
 84%|████████▎ | 188131/225000 [354:43:54<79:23:34,  7.75s/it]
 84%|████████▎ | 188132/225000 [354:43:58<67:37:04,  6.60s/it]
 84%|████████▎ | 188133/225000 [354:44:04<67:14:37,  6.57s/it]
 84%|████████▎ | 188134/225000 [354:44:10<64:11:58,  6.27s/it]
 84%|████████▎ | 188135/225000 [354:44:22<83:05:15,  8.11s/it]
 84%|████████▎ | 188136/225000 [354:44:27<74:01:09,  7.23s/it]
 84%|████████▎ | 188137/225000 [354:44:32<66:23:49,  6.48s/it]
 84%|████████▎ | 188138/225000 [354:44:37<61:03:37,  5.96s/it]
 84%|████████▎ | 188139/225000 [354:44:41<55:26:51,  5.42s/it]
 84%|████████▎ | 188140/225000 [354:44:53<75:13:42,  7.35s/it]
 84%|████████▎ | 188141/225000 [354:44:58<67:26:33,  6.59s/it]
 84%|████████▎ | 188142/225000 [354:45:02<61:44:55,  6.03s/it]
 84%|████████▎ | 188143/225000 [354:45:08<59:12:44,  5.78s/it]
 84%|████████▎ | 188144/225000 [354:45:18<73:46:54,  7.21s/it]
 84%|████████▎ | 188145/225000 [354:45:26<75:02:39,  7.33s/it]
 84%|████████▎ | 188146/225000 [354:45:32<71:49:06,  7.02s/it]
 84%|████████▎ | 188147/225000 [354:45:37<67:10:51,  6.56s/it]
 84%|████████▎ | 188148/225000 [354:45:42<60:25:37,  5.90s/it]
 84%|████████▎ | 188149/225000 [354:45:46<54:06:20,  5.29s/it]
 84%|████████▎ | 188150/225000 [354:45:55<67:03:17,  6.55s/it]
                                                              
{'loss': '1.062', 'grad_norm': '1.348', 'learning_rate': '3.554e-06', 'epoch': '0.8362', 'num_input_tokens_seen': 588884056, 'train_runtime': '1.277e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188150/225000 [354:45:55<67:03:17,  6.55s/it]
 84%|████████▎ | 188151/225000 [354:46:02<69:00:13,  6.74s/it]
 84%|████████▎ | 188152/225000 [354:46:11<75:32:17,  7.38s/it]
 84%|████████▎ | 188153/225000 [354:46:16<67:55:49,  6.64s/it]
 84%|████████▎ | 188154/225000 [354:46:22<64:07:37,  6.27s/it]
 84%|████████▎ | 188155/225000 [354:46:31<74:25:49,  7.27s/it]
 84%|████████▎ | 188156/225000 [354:46:44<90:50:50,  8.88s/it]
 84%|████████▎ | 188157/225000 [354:46:49<80:32:16,  7.87s/it]
 84%|████████▎ | 188158/225000 [354:46:56<76:13:05,  7.45s/it]
 84%|████████▎ | 188159/225000 [354:47:05<81:08:21,  7.93s/it]
 84%|████████▎ | 188160/225000 [354:47:12<78:44:36,  7.69s/it]
 84%|████████▎ | 188161/225000 [354:47:17<69:14:14,  6.77s/it]
 84%|████████▎ | 188162/225000 [354:47:22<64:28:34,  6.30s/it]
 84%|████████▎ | 188163/225000 [354:47:35<84:31:21,  8.26s/it]
 84%|████████▎ | 188164/225000 [354:47:38<70:43:51,  6.91s/it]
 84%|████████▎ | 188165/225000 [354:47:46<71:25:03,  6.98s/it]
 84%|████████▎ | 188166/225000 [354:48:01<97:42:02,  9.55s/it]
 84%|████████▎ | 188167/225000 [354:48:08<88:26:19,  8.64s/it]
 84%|████████▎ | 188168/225000 [354:48:13<77:43:53,  7.60s/it]
 84%|████████▎ | 188169/225000 [354:48:25<92:14:56,  9.02s/it]
 84%|████████▎ | 188170/225000 [354:48:30<81:07:33,  7.93s/it]
 84%|████████▎ | 188171/225000 [354:48:41<87:52:03,  8.59s/it]
 84%|████████▎ | 188172/225000 [354:48:47<81:41:32,  7.99s/it]
 84%|████████▎ | 188173/225000 [354:48:51<70:13:02,  6.86s/it]
 84%|████████▎ | 188174/225000 [354:48:58<70:20:27,  6.88s/it]
 84%|████████▎ | 188175/225000 [354:49:08<77:52:08,  7.61s/it]
 84%|████████▎ | 188176/225000 [354:49:12<66:50:22,  6.53s/it]
 84%|████████▎ | 188177/225000 [354:49:17<61:47:45,  6.04s/it]
 84%|████████▎ | 188178/225000 [354:49:26<72:58:06,  7.13s/it]
 84%|████████▎ | 188179/225000 [354:49:31<65:54:04,  6.44s/it]
 84%|████████▎ | 188180/225000 [354:49:43<82:05:26,  8.03s/it]
 84%|████████▎ | 188181/225000 [354:49:51<81:26:14,  7.96s/it]
 84%|████████▎ | 188182/225000 [354:49:55<71:03:46,  6.95s/it]
 84%|████████▎ | 188183/225000 [354:50:02<69:29:15,  6.79s/it]
 84%|████████▎ | 188184/225000 [354:50:10<74:32:13,  7.29s/it]
 84%|████████▎ | 188185/225000 [354:50:14<64:10:33,  6.28s/it]
 84%|████████▎ | 188186/225000 [354:50:21<67:33:13,  6.61s/it]
 84%|████████▎ | 188187/225000 [354:50:27<63:48:20,  6.24s/it]
 84%|████████▎ | 188188/225000 [354:50:37<76:02:59,  7.44s/it]
 84%|████████▎ | 188189/225000 [354:50:46<81:12:14,  7.94s/it]
 84%|████████▎ | 188190/225000 [354:51:00<97:57:38,  9.58s/it]
 84%|████████▎ | 188191/225000 [354:51:04<80:54:07,  7.91s/it]
 84%|████████▎ | 188192/225000 [354:51:07<68:21:04,  6.69s/it]
 84%|████████▎ | 188193/225000 [354:51:13<65:00:59,  6.36s/it]
 84%|████████▎ | 188194/225000 [354:51:23<74:54:05,  7.33s/it]
 84%|████████▎ | 188195/225000 [354:51:30<76:30:51,  7.48s/it]
 84%|████████▎ | 188196/225000 [354:51:35<67:25:57,  6.60s/it]
 84%|████████▎ | 188197/225000 [354:51:46<80:29:51,  7.87s/it]
 84%|████████▎ | 188198/225000 [354:51:58<93:43:47,  9.17s/it]
 84%|████████▎ | 188199/225000 [354:52:01<75:27:41,  7.38s/it]
 84%|████████▎ | 188200/225000 [354:52:05<64:20:24,  6.29s/it]
                                                              
{'loss': '1.024', 'grad_norm': '1.145', 'learning_rate': '3.545e-06', 'epoch': '0.8364', 'num_input_tokens_seen': 589068008, 'train_runtime': '1.278e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188200/225000 [354:52:05<64:20:24,  6.29s/it]
 84%|████████▎ | 188201/225000 [354:52:18<84:30:54,  8.27s/it]
 84%|████████▎ | 188202/225000 [354:52:22<72:02:10,  7.05s/it]
 84%|████████▎ | 188203/225000 [354:52:31<79:32:08,  7.78s/it]
 84%|████████▎ | 188204/225000 [354:52:36<69:00:42,  6.75s/it]
 84%|████████▎ | 188205/225000 [354:52:45<75:38:52,  7.40s/it]
 84%|████████▎ | 188206/225000 [354:52:50<69:22:35,  6.79s/it]
 84%|████████▎ | 188207/225000 [354:52:56<67:31:55,  6.61s/it]
 84%|████████▎ | 188208/225000 [354:53:01<61:54:55,  6.06s/it]
 84%|████████▎ | 188209/225000 [354:53:13<80:08:24,  7.84s/it]
 84%|████████▎ | 188210/225000 [354:53:20<76:12:46,  7.46s/it]
 84%|████████▎ | 188211/225000 [354:53:28<77:37:28,  7.60s/it]
 84%|████████▎ | 188212/225000 [354:53:38<84:56:49,  8.31s/it]
 84%|████████▎ | 188213/225000 [354:53:43<77:28:51,  7.58s/it]
 84%|████████▎ | 188214/225000 [354:53:49<70:32:12,  6.90s/it]
 84%|████████▎ | 188215/225000 [354:53:53<62:50:14,  6.15s/it]
 84%|████████▎ | 188216/225000 [354:53:58<59:31:57,  5.83s/it]
 84%|████████▎ | 188217/225000 [354:54:03<57:44:05,  5.65s/it]
 84%|████████▎ | 188218/225000 [354:54:11<62:20:04,  6.10s/it]
 84%|████████▎ | 188219/225000 [354:54:22<78:28:51,  7.68s/it]
 84%|████████▎ | 188220/225000 [354:54:28<72:40:30,  7.11s/it]
 84%|████████▎ | 188221/225000 [354:54:33<66:06:50,  6.47s/it]
 84%|████████▎ | 188222/225000 [354:54:43<76:45:10,  7.51s/it]
 84%|████████▎ | 188223/225000 [354:54:48<71:29:59,  7.00s/it]
 84%|████████▎ | 188224/225000 [354:54:54<66:04:32,  6.47s/it]
 84%|████████▎ | 188225/225000 [354:54:59<61:47:16,  6.05s/it]
 84%|████████▎ | 188226/225000 [354:55:08<71:37:31,  7.01s/it]
 84%|████████▎ | 188227/225000 [354:55:15<71:01:37,  6.95s/it]
 84%|████████▎ | 188228/225000 [354:55:25<81:03:48,  7.94s/it]
 84%|████████▎ | 188229/225000 [354:55:29<69:03:37,  6.76s/it]
 84%|████████▎ | 188230/225000 [354:55:37<71:32:59,  7.01s/it]
 84%|████████▎ | 188231/225000 [354:55:41<61:59:05,  6.07s/it]
 84%|████████▎ | 188232/225000 [354:55:45<57:50:41,  5.66s/it]
 84%|████████▎ | 188233/225000 [354:55:50<53:37:07,  5.25s/it]
 84%|████████▎ | 188234/225000 [354:55:54<51:15:00,  5.02s/it]
 84%|████████▎ | 188235/225000 [354:56:06<71:21:48,  6.99s/it]
 84%|████████▎ | 188236/225000 [354:56:11<67:37:17,  6.62s/it]
 84%|████████▎ | 188237/225000 [354:56:19<70:06:42,  6.87s/it]
 84%|████████▎ | 188238/225000 [354:56:25<67:19:32,  6.59s/it]
 84%|████████▎ | 188239/225000 [354:56:36<81:59:28,  8.03s/it]
 84%|████████▎ | 188240/225000 [354:56:41<71:44:56,  7.03s/it]
 84%|████████▎ | 188241/225000 [354:56:50<77:34:16,  7.60s/it]
 84%|████████▎ | 188242/225000 [354:56:54<67:59:24,  6.66s/it]
 84%|████████▎ | 188243/225000 [354:56:59<61:21:39,  6.01s/it]
 84%|████████▎ | 188244/225000 [354:57:04<57:57:00,  5.68s/it]
 84%|████████▎ | 188245/225000 [354:57:16<77:51:08,  7.63s/it]
 84%|████████▎ | 188246/225000 [354:57:20<68:33:00,  6.71s/it]
 84%|████████▎ | 188247/225000 [354:57:25<63:12:51,  6.19s/it]
 84%|████████▎ | 188248/225000 [354:57:30<57:42:39,  5.65s/it]
 84%|████████▎ | 188249/225000 [354:57:34<54:28:34,  5.34s/it]
 84%|████████▎ | 188250/225000 [354:57:43<66:02:02,  6.47s/it]
                                                              
{'loss': '1.036', 'grad_norm': '1.135', 'learning_rate': '3.535e-06', 'epoch': '0.8367', 'num_input_tokens_seen': 589236192, 'train_runtime': '1.278e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188250/225000 [354:57:43<66:02:02,  6.47s/it]
 84%|████████▎ | 188251/225000 [354:57:49<61:47:58,  6.05s/it]
 84%|████████▎ | 188252/225000 [354:58:04<89:03:36,  8.72s/it]
 84%|████████▎ | 188253/225000 [354:58:08<75:48:37,  7.43s/it]
 84%|████████▎ | 188254/225000 [354:58:14<71:22:52,  6.99s/it]
 84%|████████▎ | 188255/225000 [354:58:21<72:11:24,  7.07s/it]
 84%|████████▎ | 188256/225000 [354:58:33<85:46:11,  8.40s/it]
 84%|████████▎ | 188257/225000 [354:58:41<84:05:00,  8.24s/it]
 84%|████████▎ | 188258/225000 [354:58:51<89:33:14,  8.77s/it]
 84%|████████▎ | 188259/225000 [354:59:03<101:09:47,  9.91s/it]
 84%|████████▎ | 188260/225000 [354:59:07<81:21:28,  7.97s/it] 
 84%|████████▎ | 188261/225000 [354:59:11<69:51:09,  6.84s/it]
 84%|████████▎ | 188262/225000 [354:59:20<75:50:37,  7.43s/it]
 84%|████████▎ | 188263/225000 [354:59:32<91:29:12,  8.97s/it]
 84%|████████▎ | 188264/225000 [354:59:39<85:38:19,  8.39s/it]
 84%|████████▎ | 188265/225000 [354:59:44<75:03:32,  7.36s/it]
 84%|████████▎ | 188266/225000 [354:59:50<69:05:52,  6.77s/it]
 84%|████████▎ | 188267/225000 [354:59:55<65:44:07,  6.44s/it]
 84%|████████▎ | 188268/225000 [355:00:10<90:11:36,  8.84s/it]
 84%|████████▎ | 188269/225000 [355:00:20<93:42:35,  9.18s/it]
 84%|████████▎ | 188270/225000 [355:00:25<81:27:14,  7.98s/it]
 84%|████████▎ | 188271/225000 [355:00:35<89:02:44,  8.73s/it]
 84%|████████▎ | 188272/225000 [355:00:45<91:50:50,  9.00s/it]
 84%|████████▎ | 188273/225000 [355:00:50<78:58:35,  7.74s/it]
 84%|████████▎ | 188274/225000 [355:00:55<72:30:54,  7.11s/it]
 84%|████████▎ | 188275/225000 [355:01:06<82:52:00,  8.12s/it]
 84%|████████▎ | 188276/225000 [355:01:11<73:53:56,  7.24s/it]
 84%|████████▎ | 188277/225000 [355:01:16<67:30:44,  6.62s/it]
 84%|████████▎ | 188278/225000 [355:01:29<85:53:14,  8.42s/it]
 84%|████████▎ | 188279/225000 [355:01:35<78:30:25,  7.70s/it]
 84%|████████▎ | 188280/225000 [355:01:38<66:01:39,  6.47s/it]
 84%|████████▎ | 188281/225000 [355:01:42<57:43:22,  5.66s/it]
 84%|████████▎ | 188282/225000 [355:01:50<63:08:15,  6.19s/it]
 84%|████████▎ | 188283/225000 [355:01:56<62:51:04,  6.16s/it]
 84%|████████▎ | 188284/225000 [355:02:08<80:12:37,  7.86s/it]
 84%|████████▎ | 188285/225000 [355:02:12<68:42:54,  6.74s/it]
 84%|████████▎ | 188286/225000 [355:02:16<61:20:11,  6.01s/it]
 84%|████████▎ | 188287/225000 [355:02:27<76:19:45,  7.48s/it]
 84%|████████▎ | 188288/225000 [355:02:33<73:15:21,  7.18s/it]
 84%|████████▎ | 188289/225000 [355:02:43<81:21:56,  7.98s/it]
 84%|████████▎ | 188290/225000 [355:02:50<77:11:28,  7.57s/it]
 84%|████████▎ | 188291/225000 [355:02:54<68:16:58,  6.70s/it]
 84%|████████▎ | 188292/225000 [355:03:00<64:15:09,  6.30s/it]
 84%|████████▎ | 188293/225000 [355:03:05<59:55:01,  5.88s/it]
 84%|████████▎ | 188294/225000 [355:03:10<58:18:36,  5.72s/it]
 84%|████████▎ | 188295/225000 [355:03:21<74:08:30,  7.27s/it]
 84%|████████▎ | 188296/225000 [355:03:26<66:29:53,  6.52s/it]
 84%|████████▎ | 188297/225000 [355:03:34<70:39:31,  6.93s/it]
 84%|████████▎ | 188298/225000 [355:03:39<67:16:06,  6.60s/it]
 84%|████████▎ | 188299/225000 [355:03:47<68:39:22,  6.73s/it]
 84%|████████▎ | 188300/225000 [355:03:52<65:31:11,  6.43s/it]
                                                              
{'loss': '1.049', 'grad_norm': '1.205', 'learning_rate': '3.526e-06', 'epoch': '0.8369', 'num_input_tokens_seen': 589418104, 'train_runtime': '1.278e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188300/225000 [355:03:52<65:31:11,  6.43s/it]
 84%|████████▎ | 188301/225000 [355:03:58<64:57:38,  6.37s/it]
 84%|████████▎ | 188302/225000 [355:04:05<65:29:54,  6.43s/it]
 84%|████████▎ | 188303/225000 [355:04:16<79:02:39,  7.75s/it]
 84%|████████▎ | 188304/225000 [355:04:25<81:52:23,  8.03s/it]
 84%|████████▎ | 188305/225000 [355:04:30<73:56:12,  7.25s/it]
 84%|████████▎ | 188306/225000 [355:04:36<70:02:21,  6.87s/it]
 84%|████████▎ | 188307/225000 [355:04:46<80:20:22,  7.88s/it]
 84%|████████▎ | 188308/225000 [355:04:55<84:22:19,  8.28s/it]
 84%|████████▎ | 188309/225000 [355:04:59<68:39:28,  6.74s/it]
 84%|████████▎ | 188310/225000 [355:05:11<86:13:54,  8.46s/it]
 84%|████████▎ | 188311/225000 [355:05:15<73:44:03,  7.23s/it]
 84%|████████▎ | 188312/225000 [355:05:19<62:30:44,  6.13s/it]
 84%|████████▎ | 188313/225000 [355:05:26<65:21:52,  6.41s/it]
 84%|████████▎ | 188314/225000 [355:05:34<69:17:35,  6.80s/it]
 84%|████████▎ | 188315/225000 [355:05:47<87:42:01,  8.61s/it]
 84%|████████▎ | 188316/225000 [355:05:52<78:32:31,  7.71s/it]
 84%|████████▎ | 188317/225000 [355:05:57<69:35:32,  6.83s/it]
 84%|████████▎ | 188318/225000 [355:06:06<76:26:06,  7.50s/it]
 84%|████████▎ | 188319/225000 [355:06:16<84:49:59,  8.33s/it]
 84%|████████▎ | 188320/225000 [355:06:22<77:01:13,  7.56s/it]
 84%|████████▎ | 188321/225000 [355:06:28<71:36:49,  7.03s/it]
 84%|████████▎ | 188322/225000 [355:06:36<76:15:59,  7.49s/it]
 84%|████████▎ | 188323/225000 [355:06:41<67:54:42,  6.67s/it]
 84%|████████▎ | 188324/225000 [355:06:48<69:28:24,  6.82s/it]
 84%|████████▎ | 188325/225000 [355:06:52<59:40:31,  5.86s/it]
 84%|████████▎ | 188326/225000 [355:07:01<69:59:50,  6.87s/it]
 84%|████████▎ | 188327/225000 [355:07:05<60:14:07,  5.91s/it]
 84%|████████▎ | 188328/225000 [355:07:10<59:12:41,  5.81s/it]
 84%|████████▎ | 188329/225000 [355:07:18<64:08:03,  6.30s/it]
 84%|████████▎ | 188330/225000 [355:07:25<66:33:56,  6.53s/it]
 84%|████████▎ | 188331/225000 [355:07:32<67:41:01,  6.64s/it]
 84%|████████▎ | 188332/225000 [355:07:41<75:52:53,  7.45s/it]
 84%|████████▎ | 188333/225000 [355:07:48<73:31:46,  7.22s/it]
 84%|████████▎ | 188334/225000 [355:07:52<62:35:21,  6.15s/it]
 84%|████████▎ | 188335/225000 [355:08:00<71:14:13,  6.99s/it]
 84%|████████▎ | 188336/225000 [355:08:06<65:28:51,  6.43s/it]
 84%|████████▎ | 188337/225000 [355:08:17<81:53:48,  8.04s/it]
 84%|████████▎ | 188338/225000 [355:08:23<74:54:40,  7.36s/it]
 84%|████████▎ | 188339/225000 [355:08:30<72:05:10,  7.08s/it]
 84%|████████▎ | 188340/225000 [355:08:35<68:06:01,  6.69s/it]
 84%|████████▎ | 188341/225000 [355:08:48<85:44:07,  8.42s/it]
 84%|████████▎ | 188342/225000 [355:09:04<107:59:18, 10.61s/it]
 84%|████████▎ | 188343/225000 [355:09:13<104:55:35, 10.30s/it]
 84%|████████▎ | 188344/225000 [355:09:17<84:09:27,  8.27s/it] 
 84%|████████▎ | 188345/225000 [355:09:22<76:39:42,  7.53s/it]
 84%|████████▎ | 188346/225000 [355:09:27<68:49:10,  6.76s/it]
 84%|████████▎ | 188347/225000 [355:09:36<74:18:17,  7.30s/it]
 84%|████████▎ | 188348/225000 [355:09:42<69:12:01,  6.80s/it]
 84%|████████▎ | 188349/225000 [355:09:49<71:30:51,  7.02s/it]
 84%|████████▎ | 188350/225000 [355:09:55<66:25:07,  6.52s/it]
                                                              
{'loss': '1.043', 'grad_norm': '1.756', 'learning_rate': '3.516e-06', 'epoch': '0.8371', 'num_input_tokens_seen': 589596280, 'train_runtime': '1.279e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188350/225000 [355:09:55<66:25:07,  6.52s/it]
 84%|████████▎ | 188351/225000 [355:09:58<58:09:14,  5.71s/it]
 84%|████████▎ | 188352/225000 [355:10:10<77:32:28,  7.62s/it]
 84%|████████▎ | 188353/225000 [355:10:20<84:12:52,  8.27s/it]
 84%|████████▎ | 188354/225000 [355:10:24<69:15:19,  6.80s/it]
 84%|████████▎ | 188355/225000 [355:10:29<66:15:29,  6.51s/it]
 84%|████████▎ | 188356/225000 [355:10:35<64:06:23,  6.30s/it]
 84%|████████▎ | 188357/225000 [355:10:40<60:02:19,  5.90s/it]
 84%|████████▎ | 188358/225000 [355:10:47<61:33:37,  6.05s/it]
 84%|████████▎ | 188359/225000 [355:11:01<87:09:24,  8.56s/it]
 84%|████████▎ | 188360/225000 [355:11:05<73:00:39,  7.17s/it]
 84%|████████▎ | 188361/225000 [355:11:16<85:26:57,  8.40s/it]
 84%|████████▎ | 188362/225000 [355:11:19<69:40:02,  6.85s/it]
 84%|████████▎ | 188363/225000 [355:11:25<65:20:39,  6.42s/it]
 84%|████████▎ | 188364/225000 [355:11:36<80:10:19,  7.88s/it]
 84%|████████▎ | 188365/225000 [355:11:42<73:29:32,  7.22s/it]
 84%|████████▎ | 188366/225000 [355:11:47<67:12:42,  6.60s/it]
 84%|████████▎ | 188367/225000 [355:11:58<82:05:07,  8.07s/it]
 84%|████████▎ | 188368/225000 [355:12:06<81:30:54,  8.01s/it]
 84%|████████▎ | 188369/225000 [355:12:10<69:36:15,  6.84s/it]
 84%|████████▎ | 188370/225000 [355:12:18<71:20:20,  7.01s/it]
 84%|████████▎ | 188371/225000 [355:12:22<63:09:12,  6.21s/it]
 84%|████████▎ | 188372/225000 [355:12:26<56:39:50,  5.57s/it]
 84%|████████▎ | 188373/225000 [355:12:32<57:14:33,  5.63s/it]
 84%|████████▎ | 188374/225000 [355:12:43<72:32:17,  7.13s/it]
 84%|████████▎ | 188375/225000 [355:12:53<81:02:10,  7.97s/it]
 84%|████████▎ | 188376/225000 [355:13:00<78:03:26,  7.67s/it]
 84%|████████▎ | 188377/225000 [355:13:04<67:16:04,  6.61s/it]
 84%|████████▎ | 188378/225000 [355:13:14<77:25:04,  7.61s/it]
 84%|████████▎ | 188379/225000 [355:13:18<67:22:19,  6.62s/it]
 84%|████████▎ | 188380/225000 [355:13:23<61:09:28,  6.01s/it]
 84%|████████▎ | 188381/225000 [355:13:34<76:18:28,  7.50s/it]
 84%|████████▎ | 188382/225000 [355:13:38<67:52:50,  6.67s/it]
 84%|████████▎ | 188383/225000 [355:13:42<59:00:13,  5.80s/it]
 84%|████████▎ | 188384/225000 [355:13:54<76:51:56,  7.56s/it]
 84%|████████▎ | 188385/225000 [355:14:01<76:42:25,  7.54s/it]
 84%|████████▎ | 188386/225000 [355:14:07<70:29:33,  6.93s/it]
 84%|████████▎ | 188387/225000 [355:14:11<61:44:42,  6.07s/it]
 84%|████████▎ | 188388/225000 [355:14:21<74:11:42,  7.30s/it]
 84%|████████▎ | 188389/225000 [355:14:28<72:24:03,  7.12s/it]
 84%|████████▎ | 188390/225000 [355:14:31<61:02:41,  6.00s/it]
 84%|████████▎ | 188391/225000 [355:14:40<71:31:32,  7.03s/it]
 84%|████████▎ | 188392/225000 [355:14:46<67:29:15,  6.64s/it]
 84%|████████▎ | 188393/225000 [355:14:55<73:59:55,  7.28s/it]
 84%|████████▎ | 188394/225000 [355:15:00<67:55:38,  6.68s/it]
 84%|████████▎ | 188395/225000 [355:15:04<59:52:28,  5.89s/it]
 84%|████████▎ | 188396/225000 [355:15:10<58:45:44,  5.78s/it]
 84%|████████▎ | 188397/225000 [355:15:16<58:38:57,  5.77s/it]
 84%|████████▎ | 188398/225000 [355:15:27<76:17:55,  7.50s/it]
 84%|████████▎ | 188399/225000 [355:15:37<82:14:16,  8.09s/it]
 84%|████████▎ | 188400/225000 [355:15:43<76:31:15,  7.53s/it]
                                                              
{'loss': '1.035', 'grad_norm': '1.167', 'learning_rate': '3.507e-06', 'epoch': '0.8373', 'num_input_tokens_seen': 589774504, 'train_runtime': '1.279e+06', 'train_tokens_per_second': '461.1'}

 84%|████████▎ | 188400/225000 [355:15:43<76:31:15,  7.53s/it]
 84%|████████▎ | 188401/225000 [355:15:47<66:47:27,  6.57s/it]
 84%|████████▎ | 188402/225000 [355:15:58<79:00:21,  7.77s/it]
 84%|████████▎ | 188403/225000 [355:16:02<69:28:35,  6.83s/it]
 84%|████████▎ | 188404/225000 [355:16:10<72:28:18,  7.13s/it]
 84%|████████▎ | 188405/225000 [355:16:21<82:59:29,  8.16s/it]
 84%|████████▎ | 188406/225000 [355:16:33<94:07:16,  9.26s/it]
 84%|████████▎ | 188407/225000 [355:16:37<80:32:27,  7.92s/it]
 84%|████████▎ | 188408/225000 [355:16:43<74:16:00,  7.31s/it]
 84%|████████▎ | 188409/225000 [355:16:51<76:51:35,  7.56s/it]
 84%|████████▎ | 188410/225000 [355:16:56<69:07:36,  6.80s/it]
 84%|████████▎ | 188411/225000 [355:17:01<62:45:05,  6.17s/it]
 84%|████████▎ | 188412/225000 [355:17:12<77:06:59,  7.59s/it]
 84%|████████▎ | 188413/225000 [355:17:16<66:03:11,  6.50s/it]
 84%|████████▎ | 188414/225000 [355:17:22<65:39:19,  6.46s/it]
 84%|████████▎ | 188415/225000 [355:17:27<61:26:04,  6.05s/it]
 84%|████████▎ | 188416/225000 [355:17:37<73:28:44,  7.23s/it]
 84%|████████▎ | 188417/225000 [355:17:48<83:49:26,  8.25s/it]
 84%|████████▎ | 188418/225000 [355:17:53<72:39:25,  7.15s/it]
 84%|████████▎ | 188419/225000 [355:17:57<63:29:32,  6.25s/it]
 84%|████████▎ | 188420/225000 [355:18:06<71:48:04,  7.07s/it]
 84%|████████▎ | 188421/225000 [355:18:15<77:53:31,  7.67s/it]
 84%|████████▎ | 188422/225000 [355:18:18<65:36:43,  6.46s/it]
 84%|████████▎ | 188423/225000 [355:18:25<66:52:32,  6.58s/it]
 84%|████████▎ | 188424/225000 [355:18:31<63:06:34,  6.21s/it]
 84%|████████▎ | 188425/225000 [355:18:42<77:25:52,  7.62s/it]
 84%|████████▎ | 188426/225000 [355:18:53<89:42:04,  8.83s/it]
 84%|████████▎ | 188427/225000 [355:18:58<78:10:24,  7.69s/it]
 84%|████████▎ | 188428/225000 [355:19:06<77:48:16,  7.66s/it]
 84%|████████▎ | 188429/225000 [355:19:17<87:18:27,  8.59s/it]
 84%|████████▎ | 188430/225000 [355:19:23<79:56:04,  7.87s/it]
 84%|████████▎ | 188431/225000 [355:19:27<70:14:42,  6.92s/it]
 84%|████████▎ | 188432/225000 [355:19:34<68:04:15,  6.70s/it]
 84%|████████▎ | 188433/225000 [355:19:38<60:05:12,  5.92s/it]
 84%|████████▎ | 188434/225000 [355:19:45<64:16:35,  6.33s/it]
 84%|████████▎ | 188435/225000 [355:19:54<72:18:48,  7.12s/it]
 84%|████████▎ | 188436/225000 [355:19:59<67:18:02,  6.63s/it]
 84%|████████▎ | 188437/225000 [355:20:08<71:51:00,  7.07s/it]
 84%|████████▍ | 188438/225000 [355:20:12<64:47:01,  6.38s/it]
 84%|████████▍ | 188439/225000 [355:20:18<61:25:50,  6.05s/it]
 84%|████████▍ | 188440/225000 [355:20:30<81:21:11,  8.01s/it]
 84%|████████▍ | 188441/225000 [355:20:36<74:29:21,  7.34s/it]
 84%|████████▍ | 188442/225000 [355:20:40<65:33:50,  6.46s/it]
 84%|████████▍ | 188443/225000 [355:20:46<63:43:43,  6.28s/it]
 84%|████████▍ | 188444/225000 [355:20:51<57:45:49,  5.69s/it]
 84%|████████▍ | 188445/225000 [355:20:57<58:56:38,  5.80s/it]
 84%|████████▍ | 188446/225000 [355:21:01<54:27:47,  5.36s/it]
 84%|████████▍ | 188447/225000 [355:21:13<74:00:58,  7.29s/it]
 84%|████████▍ | 188448/225000 [355:21:20<74:25:50,  7.33s/it]
 84%|████████▍ | 188449/225000 [355:21:26<70:52:33,  6.98s/it]
 84%|████████▍ | 188450/225000 [355:21:37<82:00:10,  8.08s/it]
                                                              
{'loss': '1.074', 'grad_norm': '1.144', 'learning_rate': '3.498e-06', 'epoch': '0.8376', 'num_input_tokens_seen': 589951432, 'train_runtime': '1.279e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188450/225000 [355:21:37<82:00:10,  8.08s/it]
 84%|████████▍ | 188451/225000 [355:21:42<72:10:05,  7.11s/it]
 84%|████████▍ | 188452/225000 [355:21:53<84:18:16,  8.30s/it]
 84%|████████▍ | 188453/225000 [355:22:03<89:34:05,  8.82s/it]
 84%|████████▍ | 188454/225000 [355:22:10<82:50:35,  8.16s/it]
 84%|████████▍ | 188455/225000 [355:22:13<69:18:22,  6.83s/it]
 84%|████████▍ | 188456/225000 [355:22:23<77:34:30,  7.64s/it]
 84%|████████▍ | 188457/225000 [355:22:26<64:58:59,  6.40s/it]
 84%|████████▍ | 188458/225000 [355:22:36<75:52:39,  7.48s/it]
 84%|████████▍ | 188459/225000 [355:22:46<82:02:17,  8.08s/it]
 84%|████████▍ | 188460/225000 [355:22:51<72:01:04,  7.10s/it]
 84%|████████▍ | 188461/225000 [355:22:59<76:33:45,  7.54s/it]
 84%|████████▍ | 188462/225000 [355:23:03<64:15:12,  6.33s/it]
 84%|████████▍ | 188463/225000 [355:23:13<76:12:50,  7.51s/it]
 84%|████████▍ | 188464/225000 [355:23:20<73:33:20,  7.25s/it]
 84%|████████▍ | 188465/225000 [355:23:25<67:07:00,  6.61s/it]
 84%|████████▍ | 188466/225000 [355:23:30<64:23:18,  6.34s/it]
 84%|████████▍ | 188467/225000 [355:23:35<59:28:07,  5.86s/it]
 84%|████████▍ | 188468/225000 [355:23:47<76:29:48,  7.54s/it]
 84%|████████▍ | 188469/225000 [355:23:53<73:27:41,  7.24s/it]
 84%|████████▍ | 188470/225000 [355:23:58<64:53:30,  6.40s/it]
 84%|████████▍ | 188471/225000 [355:24:04<64:03:29,  6.31s/it]
 84%|████████▍ | 188472/225000 [355:24:09<59:40:56,  5.88s/it]
 84%|████████▍ | 188473/225000 [355:24:23<86:29:19,  8.52s/it]
 84%|████████▍ | 188474/225000 [355:24:28<75:18:41,  7.42s/it]
 84%|████████▍ | 188475/225000 [355:24:37<80:41:46,  7.95s/it]
 84%|████████▍ | 188476/225000 [355:24:46<82:54:37,  8.17s/it]
 84%|████████▍ | 188477/225000 [355:24:53<80:08:36,  7.90s/it]
 84%|████████▍ | 188478/225000 [355:25:03<85:01:22,  8.38s/it]
 84%|████████▍ | 188479/225000 [355:25:07<73:51:20,  7.28s/it]
 84%|████████▍ | 188480/225000 [355:25:14<72:47:08,  7.17s/it]
 84%|████████▍ | 188481/225000 [355:25:19<64:51:58,  6.39s/it]
 84%|████████▍ | 188482/225000 [355:25:29<76:09:01,  7.51s/it]
 84%|████████▍ | 188483/225000 [355:25:33<65:05:19,  6.42s/it]
 84%|████████▍ | 188484/225000 [355:25:37<56:58:17,  5.62s/it]
 84%|████████▍ | 188485/225000 [355:25:47<69:58:16,  6.90s/it]
 84%|████████▍ | 188486/225000 [355:25:50<59:33:46,  5.87s/it]
 84%|████████▍ | 188487/225000 [355:25:55<55:24:20,  5.46s/it]
 84%|████████▍ | 188488/225000 [355:26:09<84:09:56,  8.30s/it]
 84%|████████▍ | 188489/225000 [355:26:14<73:31:27,  7.25s/it]
 84%|████████▍ | 188490/225000 [355:26:20<70:01:51,  6.91s/it]
 84%|████████▍ | 188491/225000 [355:26:26<66:28:30,  6.55s/it]
 84%|████████▍ | 188492/225000 [355:26:31<60:44:07,  5.99s/it]
 84%|████████▍ | 188493/225000 [355:26:42<75:15:50,  7.42s/it]
 84%|████████▍ | 188494/225000 [355:26:45<63:39:55,  6.28s/it]
 84%|████████▍ | 188495/225000 [355:26:54<72:42:47,  7.17s/it]
 84%|████████▍ | 188496/225000 [355:26:59<64:20:20,  6.35s/it]
 84%|████████▍ | 188497/225000 [355:27:03<58:44:59,  5.79s/it]
 84%|████████▍ | 188498/225000 [355:27:07<52:22:50,  5.17s/it]
 84%|████████▍ | 188499/225000 [355:27:17<68:24:43,  6.75s/it]
 84%|████████▍ | 188500/225000 [355:27:26<74:00:17,  7.30s/it]
                                                              
{'loss': '1.034', 'grad_norm': '1.863', 'learning_rate': '3.489e-06', 'epoch': '0.8378', 'num_input_tokens_seen': 590127640, 'train_runtime': '1.28e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188500/225000 [355:27:26<74:00:17,  7.30s/it]
 84%|████████▍ | 188501/225000 [355:27:34<76:42:07,  7.57s/it]
 84%|████████▍ | 188502/225000 [355:27:39<69:03:01,  6.81s/it]
 84%|████████▍ | 188503/225000 [355:27:46<68:02:05,  6.71s/it]
 84%|████████▍ | 188504/225000 [355:27:54<73:11:51,  7.22s/it]
 84%|████████▍ | 188505/225000 [355:27:59<64:33:42,  6.37s/it]
 84%|████████▍ | 188506/225000 [355:28:05<63:30:01,  6.26s/it]
 84%|████████▍ | 188507/225000 [355:28:17<83:38:07,  8.25s/it]
 84%|████████▍ | 188508/225000 [355:28:22<71:55:18,  7.10s/it]
 84%|████████▍ | 188509/225000 [355:28:28<70:16:40,  6.93s/it]
 84%|████████▍ | 188510/225000 [355:28:38<77:40:17,  7.66s/it]
 84%|████████▍ | 188511/225000 [355:28:50<90:07:08,  8.89s/it]
 84%|████████▍ | 188512/225000 [355:28:59<90:26:37,  8.92s/it]
 84%|████████▍ | 188513/225000 [355:29:05<82:33:29,  8.15s/it]
 84%|████████▍ | 188514/225000 [355:29:09<71:36:47,  7.07s/it]
 84%|████████▍ | 188515/225000 [355:29:17<74:04:36,  7.31s/it]
 84%|████████▍ | 188516/225000 [355:29:23<69:24:05,  6.85s/it]
 84%|████████▍ | 188517/225000 [355:29:28<64:21:36,  6.35s/it]
 84%|████████▍ | 188518/225000 [355:29:34<62:53:08,  6.21s/it]
 84%|████████▍ | 188519/225000 [355:29:46<79:53:34,  7.88s/it]
 84%|████████▍ | 188520/225000 [355:29:52<73:26:37,  7.25s/it]
 84%|████████▍ | 188521/225000 [355:29:56<64:41:54,  6.38s/it]
 84%|████████▍ | 188522/225000 [355:30:10<86:52:45,  8.57s/it]
 84%|████████▍ | 188523/225000 [355:30:15<75:35:26,  7.46s/it]
 84%|████████▍ | 188524/225000 [355:30:20<68:40:55,  6.78s/it]
 84%|████████▍ | 188525/225000 [355:30:26<68:18:15,  6.74s/it]
 84%|████████▍ | 188526/225000 [355:30:41<93:19:33,  9.21s/it]
 84%|████████▍ | 188527/225000 [355:30:46<80:36:03,  7.96s/it]
 84%|████████▍ | 188528/225000 [355:30:50<68:05:22,  6.72s/it]
 84%|████████▍ | 188529/225000 [355:30:58<71:56:04,  7.10s/it]
 84%|████████▍ | 188530/225000 [355:31:03<65:28:23,  6.46s/it]
 84%|████████▍ | 188531/225000 [355:31:08<61:38:49,  6.09s/it]
 84%|████████▍ | 188532/225000 [355:31:13<58:22:54,  5.76s/it]
 84%|████████▍ | 188533/225000 [355:31:18<54:24:01,  5.37s/it]
 84%|████████▍ | 188534/225000 [355:31:29<72:56:46,  7.20s/it]
 84%|████████▍ | 188535/225000 [355:31:36<70:32:13,  6.96s/it]
 84%|████████▍ | 188536/225000 [355:31:41<63:45:36,  6.29s/it]
 84%|████████▍ | 188537/225000 [355:31:45<58:49:27,  5.81s/it]
 84%|████████▍ | 188538/225000 [355:31:52<63:14:16,  6.24s/it]
 84%|████████▍ | 188539/225000 [355:32:05<83:07:49,  8.21s/it]
 84%|████████▍ | 188540/225000 [355:32:15<88:13:56,  8.71s/it]
 84%|████████▍ | 188541/225000 [355:32:20<77:31:58,  7.66s/it]
 84%|████████▍ | 188542/225000 [355:32:31<86:18:15,  8.52s/it]
 84%|████████▍ | 188543/225000 [355:32:38<81:10:28,  8.02s/it]
 84%|████████▍ | 188544/225000 [355:32:46<80:38:04,  7.96s/it]
 84%|████████▍ | 188545/225000 [355:32:51<73:01:56,  7.21s/it]
 84%|████████▍ | 188546/225000 [355:32:56<65:02:01,  6.42s/it]
 84%|████████▍ | 188547/225000 [355:33:02<66:10:11,  6.53s/it]
 84%|████████▍ | 188548/225000 [355:33:13<78:33:22,  7.76s/it]
 84%|████████▍ | 188549/225000 [355:33:18<69:10:28,  6.83s/it]
 84%|████████▍ | 188550/225000 [355:33:24<66:05:23,  6.53s/it]
                                                              
{'loss': '1.045', 'grad_norm': '1.293', 'learning_rate': '3.479e-06', 'epoch': '0.838', 'num_input_tokens_seen': 590308448, 'train_runtime': '1.28e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188550/225000 [355:33:24<66:05:23,  6.53s/it]
 84%|████████▍ | 188551/225000 [355:33:34<79:00:51,  7.80s/it]
 84%|████████▍ | 188552/225000 [355:33:39<68:50:28,  6.80s/it]
 84%|████████▍ | 188553/225000 [355:33:47<71:53:13,  7.10s/it]
 84%|████████▍ | 188554/225000 [355:33:52<67:24:21,  6.66s/it]
 84%|████████▍ | 188555/225000 [355:33:59<66:36:28,  6.58s/it]
 84%|████████▍ | 188556/225000 [355:34:06<68:09:12,  6.73s/it]
 84%|████████▍ | 188557/225000 [355:34:10<62:21:13,  6.16s/it]
 84%|████████▍ | 188558/225000 [355:34:16<61:41:39,  6.09s/it]
 84%|████████▍ | 188559/225000 [355:34:21<57:09:17,  5.65s/it]
 84%|████████▍ | 188560/225000 [355:34:27<57:00:42,  5.63s/it]
 84%|████████▍ | 188561/225000 [355:34:32<56:01:16,  5.53s/it]
 84%|████████▍ | 188562/225000 [355:34:44<74:54:45,  7.40s/it]
 84%|████████▍ | 188563/225000 [355:34:49<67:25:13,  6.66s/it]
 84%|████████▍ | 188564/225000 [355:34:53<61:48:40,  6.11s/it]
 84%|████████▍ | 188565/225000 [355:34:59<59:53:46,  5.92s/it]
 84%|████████▍ | 188566/225000 [355:35:04<58:24:09,  5.77s/it]
 84%|████████▍ | 188567/225000 [355:35:16<74:57:39,  7.41s/it]
 84%|████████▍ | 188568/225000 [355:35:21<68:22:00,  6.76s/it]
 84%|████████▍ | 188569/225000 [355:35:25<60:07:05,  5.94s/it]
 84%|████████▍ | 188570/225000 [355:35:31<59:54:05,  5.92s/it]
 84%|████████▍ | 188571/225000 [355:35:37<60:10:13,  5.95s/it]
 84%|████████▍ | 188572/225000 [355:35:49<79:46:07,  7.88s/it]
 84%|████████▍ | 188573/225000 [355:35:54<72:04:54,  7.12s/it]
 84%|████████▍ | 188574/225000 [355:36:06<84:05:56,  8.31s/it]
 84%|████████▍ | 188575/225000 [355:36:10<73:25:34,  7.26s/it]
 84%|████████▍ | 188576/225000 [355:36:19<78:26:56,  7.75s/it]
 84%|████████▍ | 188577/225000 [355:36:28<81:43:32,  8.08s/it]
 84%|████████▍ | 188578/225000 [355:36:34<75:46:30,  7.49s/it]
 84%|████████▍ | 188579/225000 [355:36:45<86:58:53,  8.60s/it]
 84%|████████▍ | 188580/225000 [355:36:52<79:32:12,  7.86s/it]
 84%|████████▍ | 188581/225000 [355:37:07<102:43:27, 10.15s/it]
 84%|████████▍ | 188582/225000 [355:37:13<90:18:33,  8.93s/it] 
 84%|████████▍ | 188583/225000 [355:37:18<77:48:56,  7.69s/it]
 84%|████████▍ | 188584/225000 [355:37:27<80:49:19,  7.99s/it]
 84%|████████▍ | 188585/225000 [355:37:31<69:30:34,  6.87s/it]
 84%|████████▍ | 188586/225000 [355:37:39<72:53:14,  7.21s/it]
 84%|████████▍ | 188587/225000 [355:37:43<63:05:24,  6.24s/it]
 84%|████████▍ | 188588/225000 [355:37:47<56:47:02,  5.61s/it]
 84%|████████▍ | 188589/225000 [355:37:52<53:23:13,  5.28s/it]
 84%|████████▍ | 188590/225000 [355:37:59<59:21:12,  5.87s/it]
 84%|████████▍ | 188591/225000 [355:38:13<85:07:36,  8.42s/it]
 84%|████████▍ | 188592/225000 [355:38:17<71:30:53,  7.07s/it]
 84%|████████▍ | 188593/225000 [355:38:28<82:12:40,  8.13s/it]
 84%|████████▍ | 188594/225000 [355:38:32<69:45:50,  6.90s/it]
 84%|████████▍ | 188595/225000 [355:38:38<67:31:46,  6.68s/it]
 84%|████████▍ | 188596/225000 [355:38:42<61:22:10,  6.07s/it]
 84%|████████▍ | 188597/225000 [355:38:47<56:31:35,  5.59s/it]
 84%|████████▍ | 188598/225000 [355:38:58<73:25:23,  7.26s/it]
 84%|████████▍ | 188599/225000 [355:39:05<73:36:59,  7.28s/it]
 84%|████████▍ | 188600/225000 [355:39:13<74:50:01,  7.40s/it]
                                                              
{'loss': '1.094', 'grad_norm': '1.116', 'learning_rate': '3.47e-06', 'epoch': '0.8382', 'num_input_tokens_seen': 590485144, 'train_runtime': '1.28e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188600/225000 [355:39:13<74:50:01,  7.40s/it]
 84%|████████▍ | 188601/225000 [355:39:23<82:49:57,  8.19s/it]
 84%|████████▍ | 188602/225000 [355:39:28<74:09:33,  7.33s/it]
 84%|████████▍ | 188603/225000 [355:39:38<82:09:22,  8.13s/it]
 84%|████████▍ | 188604/225000 [355:39:42<68:35:44,  6.78s/it]
 84%|████████▍ | 188605/225000 [355:39:53<81:49:56,  8.09s/it]
 84%|████████▍ | 188606/225000 [355:40:03<86:45:06,  8.58s/it]
 84%|████████▍ | 188607/225000 [355:40:08<75:09:16,  7.43s/it]
 84%|████████▍ | 188608/225000 [355:40:14<70:17:19,  6.95s/it]
 84%|████████▍ | 188609/225000 [355:40:22<74:05:06,  7.33s/it]
 84%|████████▍ | 188610/225000 [355:40:29<72:18:29,  7.15s/it]
 84%|████████▍ | 188611/225000 [355:40:36<71:56:00,  7.12s/it]
 84%|████████▍ | 188612/225000 [355:40:41<67:37:17,  6.69s/it]
 84%|████████▍ | 188613/225000 [355:40:52<81:05:06,  8.02s/it]
 84%|████████▍ | 188614/225000 [355:40:56<68:34:32,  6.78s/it]
 84%|████████▍ | 188615/225000 [355:41:00<60:25:45,  5.98s/it]
 84%|████████▍ | 188616/225000 [355:41:12<77:04:46,  7.63s/it]
 84%|████████▍ | 188617/225000 [355:41:20<77:35:09,  7.68s/it]
 84%|████████▍ | 188618/225000 [355:41:25<69:52:36,  6.91s/it]
 84%|████████▍ | 188619/225000 [355:41:30<64:16:21,  6.36s/it]
 84%|████████▍ | 188620/225000 [355:41:42<83:07:40,  8.23s/it]
 84%|████████▍ | 188621/225000 [355:41:52<88:22:29,  8.75s/it]
 84%|████████▍ | 188622/225000 [355:41:56<73:50:31,  7.31s/it]
 84%|████████▍ | 188623/225000 [355:42:01<66:22:45,  6.57s/it]
 84%|████████▍ | 188624/225000 [355:42:05<57:14:43,  5.67s/it]
 84%|████████▍ | 188625/225000 [355:42:17<77:37:45,  7.68s/it]
 84%|████████▍ | 188626/225000 [355:42:23<71:47:25,  7.11s/it]
 84%|████████▍ | 188627/225000 [355:42:34<84:32:48,  8.37s/it]
 84%|████████▍ | 188628/225000 [355:42:46<93:25:45,  9.25s/it]
 84%|████████▍ | 188629/225000 [355:42:49<75:38:16,  7.49s/it]
 84%|████████▍ | 188630/225000 [355:42:55<70:34:01,  6.98s/it]
 84%|████████▍ | 188631/225000 [355:43:03<75:46:11,  7.50s/it]
 84%|████████▍ | 188632/225000 [355:43:10<71:44:34,  7.10s/it]
 84%|████████▍ | 188633/225000 [355:43:22<87:09:35,  8.63s/it]
 84%|████████▍ | 188634/225000 [355:43:32<92:00:02,  9.11s/it]
 84%|████████▍ | 188635/225000 [355:43:37<78:30:17,  7.77s/it]
 84%|████████▍ | 188636/225000 [355:43:44<77:46:27,  7.70s/it]
 84%|████████▍ | 188637/225000 [355:43:51<74:41:29,  7.39s/it]
 84%|████████▍ | 188638/225000 [355:43:57<71:49:10,  7.11s/it]
 84%|████████▍ | 188639/225000 [355:44:02<65:19:36,  6.47s/it]
 84%|████████▍ | 188640/225000 [355:44:11<71:34:22,  7.09s/it]
 84%|████████▍ | 188641/225000 [355:44:22<83:02:22,  8.22s/it]
 84%|████████▍ | 188642/225000 [355:44:30<82:50:53,  8.20s/it]
 84%|████████▍ | 188643/225000 [355:44:42<93:57:26,  9.30s/it]
 84%|████████▍ | 188644/225000 [355:44:46<79:10:04,  7.84s/it]
 84%|████████▍ | 188645/225000 [355:44:54<77:51:23,  7.71s/it]
 84%|████████▍ | 188646/225000 [355:44:59<70:40:52,  7.00s/it]
 84%|████████▍ | 188647/225000 [355:45:08<77:05:07,  7.63s/it]
 84%|████████▍ | 188648/225000 [355:45:13<69:16:24,  6.86s/it]
 84%|████████▍ | 188649/225000 [355:45:20<68:15:23,  6.76s/it]
 84%|████████▍ | 188650/225000 [355:45:25<62:57:05,  6.23s/it]
                                                              
{'loss': '1.063', 'grad_norm': '1.115', 'learning_rate': '3.461e-06', 'epoch': '0.8384', 'num_input_tokens_seen': 590670368, 'train_runtime': '1.281e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188650/225000 [355:45:25<62:57:05,  6.23s/it]
 84%|████████▍ | 188651/225000 [355:45:29<57:42:07,  5.71s/it]
 84%|████████▍ | 188652/225000 [355:45:33<53:14:50,  5.27s/it]
 84%|████████▍ | 188653/225000 [355:45:39<54:21:21,  5.38s/it]
 84%|████████▍ | 188654/225000 [355:45:54<82:17:39,  8.15s/it]
 84%|████████▍ | 188655/225000 [355:46:00<78:22:06,  7.76s/it]
 84%|████████▍ | 188656/225000 [355:46:09<82:11:46,  8.14s/it]
 84%|████████▍ | 188657/225000 [355:46:15<74:41:34,  7.40s/it]
 84%|████████▍ | 188658/225000 [355:46:20<68:09:19,  6.75s/it]
 84%|████████▍ | 188659/225000 [355:46:29<73:44:16,  7.30s/it]
 84%|████████▍ | 188660/225000 [355:46:38<77:41:24,  7.70s/it]
 84%|████████▍ | 188661/225000 [355:46:47<84:26:57,  8.37s/it]
 84%|████████▍ | 188662/225000 [355:46:54<79:11:42,  7.85s/it]
 84%|████████▍ | 188663/225000 [355:46:59<69:41:05,  6.90s/it]
 84%|████████▍ | 188664/225000 [355:47:07<73:39:16,  7.30s/it]
 84%|████████▍ | 188665/225000 [355:47:17<80:58:12,  8.02s/it]
 84%|████████▍ | 188666/225000 [355:47:23<75:16:58,  7.46s/it]
 84%|████████▍ | 188667/225000 [355:47:32<81:21:59,  8.06s/it]
 84%|████████▍ | 188668/225000 [355:47:38<74:26:21,  7.38s/it]
 84%|████████▍ | 188669/225000 [355:47:46<76:45:55,  7.61s/it]
 84%|████████▍ | 188670/225000 [355:47:55<80:02:13,  7.93s/it]
 84%|████████▍ | 188671/225000 [355:47:59<68:31:25,  6.79s/it]
 84%|████████▍ | 188672/225000 [355:48:07<71:05:13,  7.04s/it]
 84%|████████▍ | 188673/225000 [355:48:10<60:02:13,  5.95s/it]
 84%|████████▍ | 188674/225000 [355:48:15<56:52:40,  5.64s/it]
 84%|████████▍ | 188675/225000 [355:48:24<66:19:22,  6.57s/it]
 84%|████████▍ | 188676/225000 [355:48:32<70:15:24,  6.96s/it]
 84%|████████▍ | 188677/225000 [355:48:37<66:28:23,  6.59s/it]
 84%|████████▍ | 188678/225000 [355:48:55<99:04:32,  9.82s/it]
 84%|████████▍ | 188679/225000 [355:49:06<102:44:41, 10.18s/it]
 84%|████████▍ | 188680/225000 [355:49:09<83:03:04,  8.23s/it] 
 84%|████████▍ | 188681/225000 [355:49:20<89:48:30,  8.90s/it]
 84%|████████▍ | 188682/225000 [355:49:24<74:01:28,  7.34s/it]
 84%|████████▍ | 188683/225000 [355:49:35<85:02:06,  8.43s/it]
 84%|████████▍ | 188684/225000 [355:49:44<88:04:17,  8.73s/it]
 84%|████████▍ | 188685/225000 [355:49:49<76:15:16,  7.56s/it]
 84%|████████▍ | 188686/225000 [355:49:54<67:50:00,  6.72s/it]
 84%|████████▍ | 188687/225000 [355:50:10<96:35:55,  9.58s/it]
 84%|████████▍ | 188688/225000 [355:50:18<91:14:20,  9.05s/it]
 84%|████████▍ | 188689/225000 [355:50:24<83:44:36,  8.30s/it]
 84%|████████▍ | 188690/225000 [355:50:33<83:59:13,  8.33s/it]
 84%|████████▍ | 188691/225000 [355:50:38<74:47:39,  7.42s/it]
 84%|████████▍ | 188692/225000 [355:50:50<89:54:12,  8.91s/it]
 84%|████████▍ | 188693/225000 [355:50:54<75:28:33,  7.48s/it]
 84%|████████▍ | 188694/225000 [355:51:04<80:42:33,  8.00s/it]
 84%|████████▍ | 188695/225000 [355:51:09<72:43:12,  7.21s/it]
 84%|████████▍ | 188696/225000 [355:51:18<77:49:21,  7.72s/it]
 84%|████████▍ | 188697/225000 [355:51:23<69:25:49,  6.89s/it]
 84%|████████▍ | 188698/225000 [355:51:33<78:49:39,  7.82s/it]
 84%|████████▍ | 188699/225000 [355:51:42<82:34:10,  8.19s/it]
 84%|████████▍ | 188700/225000 [355:51:47<74:21:01,  7.37s/it]
                                                              
{'loss': '1.087', 'grad_norm': '1.163', 'learning_rate': '3.452e-06', 'epoch': '0.8387', 'num_input_tokens_seen': 590856536, 'train_runtime': '1.281e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188700/225000 [355:51:47<74:21:01,  7.37s/it]
 84%|████████▍ | 188701/225000 [355:51:57<81:04:13,  8.04s/it]
 84%|████████▍ | 188702/225000 [355:52:04<77:44:19,  7.71s/it]
 84%|████████▍ | 188703/225000 [355:52:08<68:04:35,  6.75s/it]
 84%|████████▍ | 188704/225000 [355:52:19<79:11:11,  7.85s/it]
 84%|████████▍ | 188705/225000 [355:52:26<77:03:59,  7.64s/it]
 84%|████████▍ | 188706/225000 [355:52:34<78:02:32,  7.74s/it]
 84%|████████▍ | 188707/225000 [355:52:43<83:12:45,  8.25s/it]
 84%|████████▍ | 188708/225000 [355:52:48<73:24:51,  7.28s/it]
 84%|████████▍ | 188709/225000 [355:52:54<68:48:21,  6.83s/it]
 84%|████████▍ | 188710/225000 [355:53:00<66:14:32,  6.57s/it]
 84%|████████▍ | 188711/225000 [355:53:10<75:26:06,  7.48s/it]
 84%|████████▍ | 188712/225000 [355:53:19<79:52:34,  7.92s/it]
 84%|████████▍ | 188713/225000 [355:53:23<69:33:28,  6.90s/it]
 84%|████████▍ | 188714/225000 [355:53:29<66:08:12,  6.56s/it]
 84%|████████▍ | 188715/225000 [355:53:33<58:52:22,  5.84s/it]
 84%|████████▍ | 188716/225000 [355:53:37<53:45:09,  5.33s/it]
 84%|████████▍ | 188717/225000 [355:53:51<77:32:00,  7.69s/it]
 84%|████████▍ | 188718/225000 [355:53:59<79:56:36,  7.93s/it]
 84%|████████▍ | 188719/225000 [355:54:11<92:24:43,  9.17s/it]
 84%|████████▍ | 188720/225000 [355:54:17<81:26:58,  8.08s/it]
 84%|████████▍ | 188721/225000 [355:54:23<77:27:28,  7.69s/it]
 84%|████████▍ | 188722/225000 [355:54:35<90:28:45,  8.98s/it]
 84%|████████▍ | 188723/225000 [355:54:40<76:53:08,  7.63s/it]
 84%|████████▍ | 188724/225000 [355:54:44<67:30:21,  6.70s/it]
 84%|████████▍ | 188725/225000 [355:55:03<103:50:22, 10.31s/it]
 84%|████████▍ | 188726/225000 [355:55:09<90:21:28,  8.97s/it] 
 84%|████████▍ | 188727/225000 [355:55:14<77:31:55,  7.69s/it]
 84%|████████▍ | 188728/225000 [355:55:27<93:56:32,  9.32s/it]
 84%|████████▍ | 188729/225000 [355:55:38<98:35:44,  9.79s/it]
 84%|████████▍ | 188730/225000 [355:55:41<80:24:42,  7.98s/it]
 84%|████████▍ | 188731/225000 [355:55:46<69:49:06,  6.93s/it]
 84%|████████▍ | 188732/225000 [355:55:55<76:22:03,  7.58s/it]
 84%|████████▍ | 188733/225000 [355:56:02<73:51:01,  7.33s/it]
 84%|████████▍ | 188734/225000 [355:56:12<82:58:48,  8.24s/it]
 84%|████████▍ | 188735/225000 [355:56:18<76:01:04,  7.55s/it]
 84%|████████▍ | 188736/225000 [355:56:27<81:21:31,  8.08s/it]
 84%|████████▍ | 188737/225000 [355:56:35<81:01:42,  8.04s/it]
 84%|████████▍ | 188738/225000 [355:56:45<86:05:12,  8.55s/it]
 84%|████████▍ | 188739/225000 [355:56:50<76:07:49,  7.56s/it]
 84%|████████▍ | 188740/225000 [355:56:58<75:31:20,  7.50s/it]
 84%|████████▍ | 188741/225000 [355:57:08<84:35:37,  8.40s/it]
 84%|████████▍ | 188742/225000 [355:57:16<81:40:46,  8.11s/it]
 84%|████████▍ | 188743/225000 [355:57:29<97:16:04,  9.66s/it]
 84%|████████▍ | 188744/225000 [355:57:32<78:31:01,  7.80s/it]
 84%|████████▍ | 188745/225000 [355:57:39<75:57:03,  7.54s/it]
 84%|████████▍ | 188746/225000 [355:57:47<76:21:56,  7.58s/it]
 84%|████████▍ | 188747/225000 [355:57:54<73:51:27,  7.33s/it]
 84%|████████▍ | 188748/225000 [355:58:02<76:03:01,  7.55s/it]
 84%|████████▍ | 188749/225000 [355:58:08<72:56:22,  7.24s/it]
 84%|████████▍ | 188750/225000 [355:58:14<67:33:57,  6.71s/it]
                                                              
{'loss': '1.075', 'grad_norm': '1.417', 'learning_rate': '3.443e-06', 'epoch': '0.8389', 'num_input_tokens_seen': 591048936, 'train_runtime': '1.281e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188750/225000 [355:58:14<67:33:57,  6.71s/it]
 84%|████████▍ | 188751/225000 [355:58:23<73:59:12,  7.35s/it]
 84%|████████▍ | 188752/225000 [355:58:26<62:09:28,  6.17s/it]
 84%|████████▍ | 188753/225000 [355:58:30<56:24:53,  5.60s/it]
 84%|████████▍ | 188754/225000 [355:58:36<55:35:05,  5.52s/it]
 84%|████████▍ | 188755/225000 [355:58:48<74:58:56,  7.45s/it]
 84%|████████▍ | 188756/225000 [355:58:51<63:02:49,  6.26s/it]
 84%|████████▍ | 188757/225000 [355:59:02<76:42:35,  7.62s/it]
 84%|████████▍ | 188758/225000 [355:59:06<67:26:16,  6.70s/it]
 84%|████████▍ | 188759/225000 [355:59:14<69:04:54,  6.86s/it]
 84%|████████▍ | 188760/225000 [355:59:25<82:05:49,  8.16s/it]
 84%|████████▍ | 188761/225000 [355:59:35<87:50:26,  8.73s/it]
 84%|████████▍ | 188762/225000 [355:59:38<72:12:55,  7.17s/it]
 84%|████████▍ | 188763/225000 [355:59:44<67:43:18,  6.73s/it]
 84%|████████▍ | 188764/225000 [355:59:54<76:37:04,  7.61s/it]
 84%|████████▍ | 188765/225000 [356:00:04<83:59:52,  8.35s/it]
 84%|████████▍ | 188766/225000 [356:00:09<75:22:31,  7.49s/it]
 84%|████████▍ | 188767/225000 [356:00:16<72:54:33,  7.24s/it]
 84%|████████▍ | 188768/225000 [356:00:23<73:16:45,  7.28s/it]
 84%|████████▍ | 188769/225000 [356:00:28<65:31:00,  6.51s/it]
 84%|████████▍ | 188770/225000 [356:00:33<59:34:42,  5.92s/it]
 84%|████████▍ | 188771/225000 [356:00:46<80:57:54,  8.05s/it]
 84%|████████▍ | 188772/225000 [356:00:52<74:51:47,  7.44s/it]
 84%|████████▍ | 188773/225000 [356:00:58<70:23:01,  6.99s/it]
 84%|████████▍ | 188774/225000 [356:01:04<67:24:53,  6.70s/it]
 84%|████████▍ | 188775/225000 [356:01:14<77:59:48,  7.75s/it]
 84%|████████▍ | 188776/225000 [356:01:22<77:52:26,  7.74s/it]
 84%|████████▍ | 188777/225000 [356:01:28<73:43:22,  7.33s/it]
 84%|████████▍ | 188778/225000 [356:01:33<67:05:14,  6.67s/it]
 84%|████████▍ | 188779/225000 [356:01:42<73:52:46,  7.34s/it]
 84%|████████▍ | 188780/225000 [356:01:47<67:28:30,  6.71s/it]
 84%|████████▍ | 188781/225000 [356:01:51<59:36:09,  5.92s/it]
 84%|████████▍ | 188782/225000 [356:02:01<71:38:53,  7.12s/it]
 84%|████████▍ | 188783/225000 [356:02:05<62:27:10,  6.21s/it]
 84%|████████▍ | 188784/225000 [356:02:11<61:33:54,  6.12s/it]
 84%|████████▍ | 188785/225000 [356:02:21<72:52:12,  7.24s/it]
 84%|████████▍ | 188786/225000 [356:02:28<71:26:56,  7.10s/it]
 84%|████████▍ | 188787/225000 [356:02:40<86:41:42,  8.62s/it]
 84%|████████▍ | 188788/225000 [356:02:47<82:10:21,  8.17s/it]
 84%|████████▍ | 188789/225000 [356:02:53<74:49:36,  7.44s/it]
 84%|████████▍ | 188790/225000 [356:03:07<94:24:52,  9.39s/it]
 84%|████████▍ | 188791/225000 [356:03:10<76:56:23,  7.65s/it]
 84%|████████▍ | 188792/225000 [356:03:17<74:52:14,  7.44s/it]
 84%|████████▍ | 188793/225000 [356:03:28<84:56:52,  8.45s/it]
 84%|████████▍ | 188794/225000 [356:03:31<69:07:18,  6.87s/it]
 84%|████████▍ | 188795/225000 [356:03:39<72:29:10,  7.21s/it]
 84%|████████▍ | 188796/225000 [356:03:45<67:14:37,  6.69s/it]
 84%|████████▍ | 188797/225000 [356:03:53<70:37:24,  7.02s/it]
 84%|████████▍ | 188798/225000 [356:03:56<60:40:32,  6.03s/it]
 84%|████████▍ | 188799/225000 [356:04:06<73:06:02,  7.27s/it]
 84%|████████▍ | 188800/225000 [356:04:15<75:33:50,  7.51s/it]
                                                              
{'loss': '1.058', 'grad_norm': '1.14', 'learning_rate': '3.433e-06', 'epoch': '0.8391', 'num_input_tokens_seen': 591225008, 'train_runtime': '1.282e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188800/225000 [356:04:15<75:33:50,  7.51s/it]
 84%|████████▍ | 188801/225000 [356:04:21<72:04:23,  7.17s/it]
 84%|████████▍ | 188802/225000 [356:04:28<70:47:20,  7.04s/it]
 84%|████████▍ | 188803/225000 [356:04:38<80:13:24,  7.98s/it]
 84%|████████▍ | 188804/225000 [356:04:42<68:50:04,  6.85s/it]
 84%|████████▍ | 188805/225000 [356:04:52<78:29:20,  7.81s/it]
 84%|████████▍ | 188806/225000 [356:04:56<67:34:12,  6.72s/it]
 84%|████████▍ | 188807/225000 [356:05:02<64:25:04,  6.41s/it]
 84%|████████▍ | 188808/225000 [356:05:07<59:50:22,  5.95s/it]
 84%|████████▍ | 188809/225000 [356:05:12<58:17:05,  5.80s/it]
 84%|████████▍ | 188810/225000 [356:05:24<76:03:17,  7.57s/it]
 84%|████████▍ | 188811/225000 [356:05:35<87:52:55,  8.74s/it]
 84%|████████▍ | 188812/225000 [356:05:40<74:31:42,  7.41s/it]
 84%|████████▍ | 188813/225000 [356:05:48<77:41:28,  7.73s/it]
 84%|████████▍ | 188814/225000 [356:05:53<69:43:44,  6.94s/it]
 84%|████████▍ | 188815/225000 [356:06:05<84:00:50,  8.36s/it]
 84%|████████▍ | 188816/225000 [356:06:15<88:41:55,  8.82s/it]
 84%|████████▍ | 188817/225000 [356:06:19<74:46:27,  7.44s/it]
 84%|████████▍ | 188818/225000 [356:06:25<70:01:30,  6.97s/it]
 84%|████████▍ | 188819/225000 [356:06:33<73:21:25,  7.30s/it]
 84%|████████▍ | 188820/225000 [356:06:40<72:31:44,  7.22s/it]
 84%|████████▍ | 188821/225000 [356:06:48<74:43:31,  7.44s/it]
 84%|████████▍ | 188822/225000 [356:06:55<72:33:36,  7.22s/it]
 84%|████████▍ | 188823/225000 [356:07:08<90:05:12,  8.96s/it]
 84%|████████▍ | 188824/225000 [356:07:11<74:03:10,  7.37s/it]
 84%|████████▍ | 188825/225000 [356:07:17<67:30:08,  6.72s/it]
 84%|████████▍ | 188826/225000 [356:07:21<59:43:37,  5.94s/it]
 84%|████████▍ | 188827/225000 [356:07:33<77:17:15,  7.69s/it]
 84%|████████▍ | 188828/225000 [356:07:41<80:55:37,  8.05s/it]
 84%|████████▍ | 188829/225000 [356:07:51<84:01:21,  8.36s/it]
 84%|████████▍ | 188830/225000 [356:07:57<77:52:54,  7.75s/it]
 84%|████████▍ | 188831/225000 [356:08:04<77:25:36,  7.71s/it]
 84%|████████▍ | 188832/225000 [356:08:11<73:18:16,  7.30s/it]
 84%|████████▍ | 188833/225000 [356:08:16<68:00:22,  6.77s/it]
 84%|████████▍ | 188834/225000 [356:08:21<62:29:59,  6.22s/it]
 84%|████████▍ | 188835/225000 [356:08:30<70:01:44,  6.97s/it]
 84%|████████▍ | 188836/225000 [356:08:41<81:15:50,  8.09s/it]
 84%|████████▍ | 188837/225000 [356:08:45<70:09:39,  6.98s/it]
 84%|████████▍ | 188838/225000 [356:08:51<66:05:04,  6.58s/it]
 84%|████████▍ | 188839/225000 [356:08:55<58:31:36,  5.83s/it]
 84%|████████▍ | 188840/225000 [356:09:06<75:36:37,  7.53s/it]
 84%|████████▍ | 188841/225000 [356:09:15<77:46:00,  7.74s/it]
 84%|████████▍ | 188842/225000 [356:09:19<67:44:34,  6.74s/it]
 84%|████████▍ | 188843/225000 [356:09:34<91:53:06,  9.15s/it]
 84%|████████▍ | 188844/225000 [356:09:38<77:16:56,  7.69s/it]
 84%|████████▍ | 188845/225000 [356:09:42<66:56:06,  6.66s/it]
 84%|████████▍ | 188846/225000 [356:09:53<77:47:43,  7.75s/it]
 84%|████████▍ | 188847/225000 [356:09:58<69:25:53,  6.91s/it]
 84%|████████▍ | 188848/225000 [356:10:03<66:32:45,  6.63s/it]
 84%|████████▍ | 188849/225000 [356:10:10<65:46:06,  6.55s/it]
 84%|████████▍ | 188850/225000 [356:10:23<84:51:06,  8.45s/it]
                                                              
{'loss': '1.102', 'grad_norm': '1.363', 'learning_rate': '3.424e-06', 'epoch': '0.8393', 'num_input_tokens_seen': 591410824, 'train_runtime': '1.282e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188850/225000 [356:10:23<84:51:06,  8.45s/it]
 84%|████████▍ | 188851/225000 [356:10:29<77:06:46,  7.68s/it]
 84%|████████▍ | 188852/225000 [356:10:36<75:53:15,  7.56s/it]
 84%|████████▍ | 188853/225000 [356:10:45<79:07:09,  7.88s/it]
 84%|████████▍ | 188854/225000 [356:10:51<75:10:49,  7.49s/it]
 84%|████████▍ | 188855/225000 [356:11:03<89:56:15,  8.96s/it]
 84%|████████▍ | 188856/225000 [356:11:10<81:48:50,  8.15s/it]
 84%|████████▍ | 188857/225000 [356:11:18<82:47:56,  8.25s/it]
 84%|████████▍ | 188858/225000 [356:11:23<73:40:07,  7.34s/it]
 84%|████████▍ | 188859/225000 [356:11:33<81:20:45,  8.10s/it]
 84%|████████▍ | 188860/225000 [356:11:43<85:02:23,  8.47s/it]
 84%|████████▍ | 188861/225000 [356:11:50<82:27:03,  8.21s/it]
 84%|████████▍ | 188862/225000 [356:11:55<72:24:43,  7.21s/it]
 84%|████████▍ | 188863/225000 [356:11:59<63:43:11,  6.35s/it]
 84%|████████▍ | 188864/225000 [356:12:05<62:15:03,  6.20s/it]
 84%|████████▍ | 188865/225000 [356:12:16<75:37:39,  7.53s/it]
 84%|████████▍ | 188866/225000 [356:12:21<67:05:38,  6.68s/it]
 84%|████████▍ | 188867/225000 [356:12:27<67:12:46,  6.70s/it]
 84%|████████▍ | 188868/225000 [356:12:37<74:26:51,  7.42s/it]
 84%|████████▍ | 188869/225000 [356:12:48<88:01:52,  8.77s/it]
 84%|████████▍ | 188870/225000 [356:12:57<87:25:36,  8.71s/it]
 84%|████████▍ | 188871/225000 [356:13:07<89:49:08,  8.95s/it]
 84%|████████▍ | 188872/225000 [356:13:12<78:10:44,  7.79s/it]
 84%|████████▍ | 188873/225000 [356:13:18<75:15:03,  7.50s/it]
 84%|████████▍ | 188874/225000 [356:13:22<63:15:53,  6.30s/it]
 84%|████████▍ | 188875/225000 [356:13:35<84:48:24,  8.45s/it]
 84%|████████▍ | 188876/225000 [356:13:40<74:16:51,  7.40s/it]
 84%|████████▍ | 188877/225000 [356:13:50<80:28:23,  8.02s/it]
 84%|████████▍ | 188878/225000 [356:13:53<66:28:52,  6.63s/it]
 84%|████████▍ | 188879/225000 [356:14:07<86:49:13,  8.65s/it]
 84%|████████▍ | 188880/225000 [356:14:11<74:25:39,  7.42s/it]
 84%|████████▍ | 188881/225000 [356:14:15<63:46:15,  6.36s/it]
 84%|████████▍ | 188882/225000 [356:14:19<55:35:49,  5.54s/it]
 84%|████████▍ | 188883/225000 [356:14:28<68:14:23,  6.80s/it]
 84%|████████▍ | 188884/225000 [356:14:35<67:08:56,  6.69s/it]
 84%|████████▍ | 188885/225000 [356:14:41<64:56:10,  6.47s/it]
 84%|████████▍ | 188886/225000 [356:14:45<58:33:34,  5.84s/it]
 84%|████████▍ | 188887/225000 [356:14:56<72:19:20,  7.21s/it]
 84%|████████▍ | 188888/225000 [356:15:01<65:38:57,  6.54s/it]
 84%|████████▍ | 188889/225000 [356:15:14<85:44:55,  8.55s/it]
 84%|████████▍ | 188890/225000 [356:15:19<76:48:02,  7.66s/it]
 84%|████████▍ | 188891/225000 [356:15:23<65:40:44,  6.55s/it]
 84%|████████▍ | 188892/225000 [356:15:30<67:06:46,  6.69s/it]
 84%|████████▍ | 188893/225000 [356:15:40<75:18:24,  7.51s/it]
 84%|████████▍ | 188894/225000 [356:15:47<75:13:20,  7.50s/it]
 84%|████████▍ | 188895/225000 [356:15:54<72:29:38,  7.23s/it]
 84%|████████▍ | 188896/225000 [356:16:04<81:14:27,  8.10s/it]
 84%|████████▍ | 188897/225000 [356:16:12<80:39:52,  8.04s/it]
 84%|████████▍ | 188898/225000 [356:16:19<79:28:21,  7.92s/it]
 84%|████████▍ | 188899/225000 [356:16:28<82:03:31,  8.18s/it]
 84%|████████▍ | 188900/225000 [356:16:35<77:57:54,  7.77s/it]
                                                              
{'loss': '1.101', 'grad_norm': '1.529', 'learning_rate': '3.415e-06', 'epoch': '0.8396', 'num_input_tokens_seen': 591593000, 'train_runtime': '1.283e+06', 'train_tokens_per_second': '461.2'}

 84%|████████▍ | 188900/225000 [356:16:35<77:57:54,  7.77s/it]
 84%|████████▍ | 188901/225000 [356:16:40<70:44:37,  7.05s/it]
 84%|████████▍ | 188902/225000 [356:16:44<59:04:47,  5.89s/it]
 84%|████████▍ | 188903/225000 [356:16:48<53:46:45,  5.36s/it]
 84%|████████▍ | 188904/225000 [356:16:55<60:15:40,  6.01s/it]
 84%|████████▍ | 188905/225000 [356:17:03<66:08:56,  6.60s/it]
 84%|████████▍ | 188906/225000 [356:17:08<61:26:25,  6.13s/it]
 84%|████████▍ | 188907/225000 [356:17:12<55:26:41,  5.53s/it]
 84%|████████▍ | 188908/225000 [356:17:26<78:57:15,  7.88s/it]
 84%|████████▍ | 188909/225000 [356:17:39<96:05:51,  9.59s/it]
 84%|████████▍ | 188910/225000 [356:17:43<79:42:56,  7.95s/it]
 84%|████████▍ | 188911/225000 [356:17:48<70:18:34,  7.01s/it]
 84%|████████▍ | 188912/225000 [356:18:01<88:35:13,  8.84s/it]
 84%|████████▍ | 188913/225000 [356:18:06<75:49:07,  7.56s/it]
 84%|████████▍ | 188914/225000 [356:18:11<68:13:17,  6.81s/it]
 84%|████████▍ | 188915/225000 [356:18:20<75:39:42,  7.55s/it]
 84%|████████▍ | 188916/225000 [356:18:25<68:22:16,  6.82s/it]
 84%|████████▍ | 188917/225000 [356:18:35<77:09:52,  7.70s/it]
 84%|████████▍ | 188918/225000 [356:18:40<69:08:17,  6.90s/it]
 84%|████████▍ | 188919/225000 [356:18:57<97:23:12,  9.72s/it]
 84%|████████▍ | 188920/225000 [356:19:01<80:25:54,  8.03s/it]
 84%|████████▍ | 188921/225000 [356:19:10<84:53:45,  8.47s/it]
 84%|████████▍ | 188922/225000 [356:19:19<85:13:18,  8.50s/it]
 84%|████████▍ | 188923/225000 [356:19:23<71:37:01,  7.15s/it]
 84%|████████▍ | 188924/225000 [356:19:29<68:33:59,  6.84s/it]
 84%|████████▍ | 188925/225000 [356:19:35<66:50:30,  6.67s/it]
 84%|████████▍ | 188926/225000 [356:19:48<84:11:20,  8.40s/it]
 84%|████████▍ | 188927/225000 [356:19:52<73:43:16,  7.36s/it]
 84%|████████▍ | 188928/225000 [356:20:00<74:01:14,  7.39s/it]
 84%|████████▍ | 188929/225000 [356:20:13<91:00:49,  9.08s/it]
 84%|████████▍ | 188930/225000 [356:20:18<77:40:05,  7.75s/it]
 84%|████████▍ | 188931/225000 [356:20:30<90:14:09,  9.01s/it]
 84%|████████▍ | 188932/225000 [356:20:33<73:53:20,  7.37s/it]
 84%|████████▍ | 188933/225000 [356:20:44<84:31:08,  8.44s/it]
 84%|████████▍ | 188934/225000 [356:20:49<72:59:35,  7.29s/it]
 84%|████████▍ | 188935/225000 [356:20:55<69:57:53,  6.98s/it]
 84%|████████▍ | 188936/225000 [356:21:00<63:18:00,  6.32s/it]
 84%|████████▍ | 188937/225000 [356:21:09<72:43:32,  7.26s/it]
 84%|████████▍ | 188938/225000 [356:21:18<77:20:28,  7.72s/it]
 84%|████████▍ | 188939/225000 [356:21:24<71:27:47,  7.13s/it]
 84%|████████▍ | 188940/225000 [356:21:33<77:51:50,  7.77s/it]
 84%|████████▍ | 188941/225000 [356:21:37<65:32:22,  6.54s/it]
 84%|████████▍ | 188942/225000 [356:21:44<68:57:57,  6.89s/it]
 84%|████████▍ | 188943/225000 [356:21:55<79:35:58,  7.95s/it]
 84%|████████▍ | 188944/225000 [356:21:59<68:38:41,  6.85s/it]
 84%|████████▍ | 188945/225000 [356:22:10<79:49:25,  7.97s/it]
 84%|████████▍ | 188946/225000 [356:22:14<67:52:50,  6.78s/it]
 84%|████████▍ | 188947/225000 [356:22:22<71:47:36,  7.17s/it]
 84%|████████▍ | 188948/225000 [356:22:27<66:39:59,  6.66s/it]
 84%|████████▍ | 188949/225000 [356:22:36<74:47:56,  7.47s/it]
 84%|████████▍ | 188950/225000 [356:22:41<65:37:24,  6.55s/it]
                                                              
{'loss': '1.111', 'grad_norm': '1.389', 'learning_rate': '3.406e-06', 'epoch': '0.8398', 'num_input_tokens_seen': 591784040, 'train_runtime': '1.283e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 188950/225000 [356:22:41<65:37:24,  6.55s/it]
 84%|████████▍ | 188951/225000 [356:22:49<69:07:25,  6.90s/it]
 84%|████████▍ | 188952/225000 [356:22:52<59:53:37,  5.98s/it]
 84%|████████▍ | 188953/225000 [356:22:57<55:10:43,  5.51s/it]
 84%|████████▍ | 188954/225000 [356:23:03<58:03:52,  5.80s/it]
 84%|████████▍ | 188955/225000 [356:23:16<78:43:53,  7.86s/it]
 84%|████████▍ | 188956/225000 [356:23:20<66:32:31,  6.65s/it]
 84%|████████▍ | 188957/225000 [356:23:32<84:27:27,  8.44s/it]
 84%|████████▍ | 188958/225000 [356:23:36<71:06:45,  7.10s/it]
 84%|████████▍ | 188959/225000 [356:23:52<96:02:19,  9.59s/it]
 84%|████████▍ | 188960/225000 [356:24:02<97:30:39,  9.74s/it]
 84%|████████▍ | 188961/225000 [356:24:11<96:10:09,  9.61s/it]
 84%|████████▍ | 188962/225000 [356:24:24<105:53:14, 10.58s/it]
 84%|████████▍ | 188963/225000 [356:24:31<93:37:49,  9.35s/it] 
 84%|████████▍ | 188964/225000 [356:24:41<96:03:14,  9.60s/it]
 84%|████████▍ | 188965/225000 [356:24:44<77:32:57,  7.75s/it]
 84%|████████▍ | 188966/225000 [356:24:53<79:28:00,  7.94s/it]
 84%|████████▍ | 188967/225000 [356:24:58<71:37:28,  7.16s/it]
 84%|████████▍ | 188968/225000 [356:25:07<78:45:52,  7.87s/it]
 84%|████████▍ | 188969/225000 [356:25:13<73:04:17,  7.30s/it]
 84%|████████▍ | 188970/225000 [356:25:23<81:21:28,  8.13s/it]
 84%|████████▍ | 188971/225000 [356:25:31<80:32:48,  8.05s/it]
 84%|████████▍ | 188972/225000 [356:25:39<78:42:39,  7.86s/it]
 84%|████████▍ | 188973/225000 [356:25:44<71:52:22,  7.18s/it]
 84%|████████▍ | 188974/225000 [356:25:50<65:56:40,  6.59s/it]
 84%|████████▍ | 188975/225000 [356:25:58<72:13:14,  7.22s/it]
 84%|████████▍ | 188976/225000 [356:26:04<67:06:00,  6.71s/it]
 84%|████████▍ | 188977/225000 [356:26:10<67:03:44,  6.70s/it]
 84%|████████▍ | 188978/225000 [356:26:22<81:52:25,  8.18s/it]
 84%|████████▍ | 188979/225000 [356:26:27<73:19:37,  7.33s/it]
 84%|████████▍ | 188980/225000 [356:26:35<73:33:46,  7.35s/it]
 84%|████████▍ | 188981/225000 [356:26:40<68:30:32,  6.85s/it]
 84%|████████▍ | 188982/225000 [356:26:50<76:07:45,  7.61s/it]
 84%|████████▍ | 188983/225000 [356:27:00<83:28:47,  8.34s/it]
 84%|████████▍ | 188984/225000 [356:27:04<70:04:20,  7.00s/it]
 84%|████████▍ | 188985/225000 [356:27:12<74:28:15,  7.44s/it]
 84%|████████▍ | 188986/225000 [356:27:17<65:50:41,  6.58s/it]
 84%|████████▍ | 188987/225000 [356:27:25<70:33:15,  7.05s/it]
 84%|████████▍ | 188988/225000 [356:27:33<73:01:06,  7.30s/it]
 84%|████████▍ | 188989/225000 [356:27:36<61:38:35,  6.16s/it]
 84%|████████▍ | 188990/225000 [356:27:44<64:44:47,  6.47s/it]
 84%|████████▍ | 188991/225000 [356:27:47<56:26:14,  5.64s/it]
 84%|████████▍ | 188992/225000 [356:27:52<54:33:59,  5.46s/it]
 84%|████████▍ | 188993/225000 [356:27:56<49:37:14,  4.96s/it]
 84%|████████▍ | 188994/225000 [356:28:11<79:38:03,  7.96s/it]
 84%|████████▍ | 188995/225000 [356:28:21<84:51:53,  8.49s/it]
 84%|████████▍ | 188996/225000 [356:28:29<85:33:54,  8.56s/it]
 84%|████████▍ | 188997/225000 [356:28:35<75:58:49,  7.60s/it]
 84%|████████▍ | 188998/225000 [356:28:40<69:02:07,  6.90s/it]
 84%|████████▍ | 188999/225000 [356:28:45<64:17:22,  6.43s/it]
 84%|████████▍ | 189000/225000 [356:28:51<62:29:33,  6.25s/it]
                                                              
{'loss': '1.085', 'grad_norm': '1.169', 'learning_rate': '3.397e-06', 'epoch': '0.84', 'num_input_tokens_seen': 591970456, 'train_runtime': '1.283e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189000/225000 [356:28:51<62:29:33,  6.25s/it]
 84%|████████▍ | 189001/225000 [356:28:56<58:45:51,  5.88s/it]
 84%|████████▍ | 189002/225000 [356:29:03<61:33:32,  6.16s/it]
 84%|████████▍ | 189003/225000 [356:29:08<58:19:18,  5.83s/it]
 84%|████████▍ | 189004/225000 [356:29:23<83:50:38,  8.39s/it]
 84%|████████▍ | 189005/225000 [356:29:27<73:02:19,  7.30s/it]
 84%|████████▍ | 189006/225000 [356:29:31<62:40:00,  6.27s/it]
 84%|████████▍ | 189007/225000 [356:29:44<81:43:13,  8.17s/it]
 84%|████████▍ | 189008/225000 [356:29:49<71:55:47,  7.19s/it]
 84%|████████▍ | 189009/225000 [356:29:58<78:34:11,  7.86s/it]
 84%|████████▍ | 189010/225000 [356:30:03<69:38:35,  6.97s/it]
 84%|████████▍ | 189011/225000 [356:30:10<68:22:21,  6.84s/it]
 84%|████████▍ | 189012/225000 [356:30:22<84:52:34,  8.49s/it]
 84%|████████▍ | 189013/225000 [356:30:28<79:03:09,  7.91s/it]
 84%|████████▍ | 189014/225000 [356:30:38<85:36:57,  8.56s/it]
 84%|████████▍ | 189015/225000 [356:30:42<70:34:33,  7.06s/it]
 84%|████████▍ | 189016/225000 [356:30:50<73:22:05,  7.34s/it]
 84%|████████▍ | 189017/225000 [356:30:55<65:03:09,  6.51s/it]
 84%|████████▍ | 189018/225000 [356:31:04<72:44:32,  7.28s/it]
 84%|████████▍ | 189019/225000 [356:31:08<64:49:28,  6.49s/it]
 84%|████████▍ | 189020/225000 [356:31:12<56:14:41,  5.63s/it]
 84%|████████▍ | 189021/225000 [356:31:22<68:50:45,  6.89s/it]
 84%|████████▍ | 189022/225000 [356:31:30<73:21:41,  7.34s/it]
 84%|████████▍ | 189023/225000 [356:31:34<63:27:55,  6.35s/it]
 84%|████████▍ | 189024/225000 [356:31:44<72:42:13,  7.28s/it]
 84%|████████▍ | 189025/225000 [356:31:49<66:23:09,  6.64s/it]
 84%|████████▍ | 189026/225000 [356:32:01<83:50:22,  8.39s/it]
 84%|████████▍ | 189027/225000 [356:32:06<72:57:21,  7.30s/it]
 84%|████████▍ | 189028/225000 [356:32:17<82:54:39,  8.30s/it]
 84%|████████▍ | 189029/225000 [356:32:27<90:16:12,  9.03s/it]
 84%|████████▍ | 189030/225000 [356:32:31<75:21:08,  7.54s/it]
 84%|████████▍ | 189031/225000 [356:32:41<79:48:52,  7.99s/it]
 84%|████████▍ | 189032/225000 [356:32:49<82:37:32,  8.27s/it]
 84%|████████▍ | 189033/225000 [356:33:00<88:31:12,  8.86s/it]
 84%|████████▍ | 189034/225000 [356:33:07<85:12:53,  8.53s/it]
 84%|████████▍ | 189035/225000 [356:33:17<88:29:01,  8.86s/it]
 84%|████████▍ | 189036/225000 [356:33:22<77:14:06,  7.73s/it]
 84%|████████▍ | 189037/225000 [356:33:31<80:10:14,  8.03s/it]
 84%|████████▍ | 189038/225000 [356:33:37<75:13:46,  7.53s/it]
 84%|████████▍ | 189039/225000 [356:33:44<73:39:47,  7.37s/it]
 84%|████████▍ | 189040/225000 [356:33:59<96:44:57,  9.69s/it]
 84%|████████▍ | 189041/225000 [356:34:04<81:07:05,  8.12s/it]
 84%|████████▍ | 189042/225000 [356:34:10<75:33:34,  7.56s/it]
 84%|████████▍ | 189043/225000 [356:34:18<75:52:23,  7.60s/it]
 84%|████████▍ | 189044/225000 [356:34:22<66:17:37,  6.64s/it]
 84%|████████▍ | 189045/225000 [356:34:36<87:57:23,  8.81s/it]
 84%|████████▍ | 189046/225000 [356:34:46<90:19:36,  9.04s/it]
 84%|████████▍ | 189047/225000 [356:34:51<79:24:41,  7.95s/it]
 84%|████████▍ | 189048/225000 [356:34:56<71:00:04,  7.11s/it]
 84%|████████▍ | 189049/225000 [356:35:01<64:46:58,  6.49s/it]
 84%|████████▍ | 189050/225000 [356:35:14<82:17:22,  8.24s/it]
                                                              
{'loss': '1.098', 'grad_norm': '1.332', 'learning_rate': '3.387e-06', 'epoch': '0.8402', 'num_input_tokens_seen': 592164528, 'train_runtime': '1.284e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189050/225000 [356:35:14<82:17:22,  8.24s/it]
 84%|████████▍ | 189051/225000 [356:35:21<80:45:45,  8.09s/it]
 84%|████████▍ | 189052/225000 [356:35:29<79:35:10,  7.97s/it]
 84%|████████▍ | 189053/225000 [356:35:39<84:38:38,  8.48s/it]
 84%|████████▍ | 189054/225000 [356:35:44<76:45:09,  7.69s/it]
 84%|████████▍ | 189055/225000 [356:35:53<80:41:25,  8.08s/it]
 84%|████████▍ | 189056/225000 [356:36:01<79:11:14,  7.93s/it]
 84%|████████▍ | 189057/225000 [356:36:07<73:51:05,  7.40s/it]
 84%|████████▍ | 189058/225000 [356:36:11<63:50:12,  6.39s/it]
 84%|████████▍ | 189059/225000 [356:36:15<56:54:51,  5.70s/it]
 84%|████████▍ | 189060/225000 [356:36:25<68:30:11,  6.86s/it]
 84%|████████▍ | 189061/225000 [356:36:31<64:50:58,  6.50s/it]
 84%|████████▍ | 189062/225000 [356:36:36<62:00:35,  6.21s/it]
 84%|████████▍ | 189063/225000 [356:36:45<71:08:35,  7.13s/it]
 84%|████████▍ | 189064/225000 [356:36:55<80:11:07,  8.03s/it]
 84%|████████▍ | 189065/225000 [356:37:00<70:11:21,  7.03s/it]
 84%|████████▍ | 189066/225000 [356:37:05<62:39:27,  6.28s/it]
 84%|████████▍ | 189067/225000 [356:37:11<61:47:26,  6.19s/it]
 84%|████████▍ | 189068/225000 [356:37:16<58:15:54,  5.84s/it]
 84%|████████▍ | 189069/225000 [356:37:30<82:20:29,  8.25s/it]
 84%|████████▍ | 189070/225000 [356:37:33<69:20:51,  6.95s/it]
 84%|████████▍ | 189071/225000 [356:37:37<59:46:30,  5.99s/it]
 84%|████████▍ | 189072/225000 [356:37:42<55:49:22,  5.59s/it]
 84%|████████▍ | 189073/225000 [356:37:49<61:38:02,  6.18s/it]
 84%|████████▍ | 189074/225000 [356:37:58<67:54:19,  6.80s/it]
 84%|████████▍ | 189075/225000 [356:38:04<65:04:18,  6.52s/it]
 84%|████████▍ | 189076/225000 [356:38:09<60:25:11,  6.05s/it]
 84%|████████▍ | 189077/225000 [356:38:14<58:23:24,  5.85s/it]
 84%|████████▍ | 189078/225000 [356:38:23<69:07:35,  6.93s/it]
 84%|████████▍ | 189079/225000 [356:38:38<92:04:45,  9.23s/it]
 84%|████████▍ | 189080/225000 [356:38:44<82:57:57,  8.32s/it]
 84%|████████▍ | 189081/225000 [356:38:49<73:12:21,  7.34s/it]
 84%|████████▍ | 189082/225000 [356:38:57<74:25:19,  7.46s/it]
 84%|████████▍ | 189083/225000 [356:39:01<64:36:40,  6.48s/it]
 84%|████████▍ | 189084/225000 [356:39:05<56:29:03,  5.66s/it]
 84%|████████▍ | 189085/225000 [356:39:16<72:29:37,  7.27s/it]
 84%|████████▍ | 189086/225000 [356:39:21<65:32:38,  6.57s/it]
 84%|████████▍ | 189087/225000 [356:39:30<72:30:00,  7.27s/it]
 84%|████████▍ | 189088/225000 [356:39:38<75:11:56,  7.54s/it]
 84%|████████▍ | 189089/225000 [356:39:44<71:46:46,  7.20s/it]
 84%|████████▍ | 189090/225000 [356:39:52<72:25:56,  7.26s/it]
 84%|████████▍ | 189091/225000 [356:39:57<67:42:36,  6.79s/it]
 84%|████████▍ | 189092/225000 [356:40:03<64:30:32,  6.47s/it]
 84%|████████▍ | 189093/225000 [356:40:10<67:10:57,  6.74s/it]
 84%|████████▍ | 189094/225000 [356:40:20<75:28:33,  7.57s/it]
 84%|████████▍ | 189095/225000 [356:40:26<72:12:16,  7.24s/it]
 84%|████████▍ | 189096/225000 [356:40:33<71:26:35,  7.16s/it]
 84%|████████▍ | 189097/225000 [356:40:38<62:10:46,  6.23s/it]
 84%|████████▍ | 189098/225000 [356:40:47<73:03:55,  7.33s/it]
 84%|████████▍ | 189099/225000 [356:40:53<66:49:22,  6.70s/it]
 84%|████████▍ | 189100/225000 [356:41:05<84:44:00,  8.50s/it]
                                                              
{'loss': '1.109', 'grad_norm': '1.346', 'learning_rate': '3.378e-06', 'epoch': '0.8404', 'num_input_tokens_seen': 592343560, 'train_runtime': '1.284e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189100/225000 [356:41:05<84:44:00,  8.50s/it]
 84%|████████▍ | 189101/225000 [356:41:12<79:25:31,  7.96s/it]
 84%|████████▍ | 189102/225000 [356:41:19<75:52:28,  7.61s/it]
 84%|████████▍ | 189103/225000 [356:41:24<68:29:43,  6.87s/it]
 84%|████████▍ | 189104/225000 [356:41:32<71:19:48,  7.15s/it]
 84%|████████▍ | 189105/225000 [356:41:41<77:26:27,  7.77s/it]
 84%|████████▍ | 189106/225000 [356:41:54<92:54:41,  9.32s/it]
 84%|████████▍ | 189107/225000 [356:41:58<78:17:40,  7.85s/it]
 84%|████████▍ | 189108/225000 [356:42:04<70:49:25,  7.10s/it]
 84%|████████▍ | 189109/225000 [356:42:14<80:41:29,  8.09s/it]
 84%|████████▍ | 189110/225000 [356:42:19<70:01:45,  7.02s/it]
 84%|████████▍ | 189111/225000 [356:42:23<62:14:44,  6.24s/it]
 84%|████████▍ | 189112/225000 [356:42:31<66:29:15,  6.67s/it]
 84%|████████▍ | 189113/225000 [356:42:41<76:48:50,  7.71s/it]
 84%|████████▍ | 189114/225000 [356:42:48<75:56:57,  7.62s/it]
 84%|████████▍ | 189115/225000 [356:42:53<68:17:37,  6.85s/it]
 84%|████████▍ | 189116/225000 [356:43:04<78:34:51,  7.88s/it]
 84%|████████▍ | 189117/225000 [356:43:08<69:26:21,  6.97s/it]
 84%|████████▍ | 189118/225000 [356:43:17<72:57:50,  7.32s/it]
 84%|████████▍ | 189119/225000 [356:43:23<71:35:51,  7.18s/it]
 84%|████████▍ | 189120/225000 [356:43:33<78:03:59,  7.83s/it]
 84%|████████▍ | 189121/225000 [356:43:39<73:04:41,  7.33s/it]
 84%|████████▍ | 189122/225000 [356:43:45<70:18:59,  7.06s/it]
 84%|████████▍ | 189123/225000 [356:43:51<65:14:05,  6.55s/it]
 84%|████████▍ | 189124/225000 [356:43:59<69:25:02,  6.97s/it]
 84%|████████▍ | 189125/225000 [356:44:09<78:05:52,  7.84s/it]
 84%|████████▍ | 189126/225000 [356:44:12<66:14:46,  6.65s/it]
 84%|████████▍ | 189127/225000 [356:44:26<86:18:08,  8.66s/it]
 84%|████████▍ | 189128/225000 [356:44:34<85:16:16,  8.56s/it]
 84%|████████▍ | 189129/225000 [356:44:41<81:23:44,  8.17s/it]
 84%|████████▍ | 189130/225000 [356:44:49<79:37:57,  7.99s/it]
 84%|████████▍ | 189131/225000 [356:45:01<90:19:57,  9.07s/it]
 84%|████████▍ | 189132/225000 [356:45:08<85:41:05,  8.60s/it]
 84%|████████▍ | 189133/225000 [356:45:14<77:58:58,  7.83s/it]
 84%|████████▍ | 189134/225000 [356:45:18<65:39:49,  6.59s/it]
 84%|████████▍ | 189135/225000 [356:45:23<60:33:26,  6.08s/it]
 84%|████████▍ | 189136/225000 [356:45:32<69:22:27,  6.96s/it]
 84%|████████▍ | 189137/225000 [356:45:38<66:39:05,  6.69s/it]
 84%|████████▍ | 189138/225000 [356:45:43<63:14:36,  6.35s/it]
 84%|████████▍ | 189139/225000 [356:45:48<58:34:01,  5.88s/it]
 84%|████████▍ | 189140/225000 [356:45:55<62:01:06,  6.23s/it]
 84%|████████▍ | 189141/225000 [356:46:01<62:23:43,  6.26s/it]
 84%|████████▍ | 189142/225000 [356:46:08<63:03:30,  6.33s/it]
 84%|████████▍ | 189143/225000 [356:46:19<77:21:23,  7.77s/it]
 84%|████████▍ | 189144/225000 [356:46:25<72:29:30,  7.28s/it]
 84%|████████▍ | 189145/225000 [356:46:31<67:59:21,  6.83s/it]
 84%|████████▍ | 189146/225000 [356:46:36<62:12:44,  6.25s/it]
 84%|████████▍ | 189147/225000 [356:46:44<67:10:53,  6.75s/it]
 84%|████████▍ | 189148/225000 [356:46:47<57:49:25,  5.81s/it]
 84%|████████▍ | 189149/225000 [356:46:59<75:06:00,  7.54s/it]
 84%|████████▍ | 189150/225000 [356:47:06<73:52:21,  7.42s/it]
                                                              
{'loss': '1.174', 'grad_norm': '1.684', 'learning_rate': '3.369e-06', 'epoch': '0.8407', 'num_input_tokens_seen': 592528120, 'train_runtime': '1.284e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189150/225000 [356:47:06<73:52:21,  7.42s/it]
 84%|████████▍ | 189151/225000 [356:47:14<75:54:05,  7.62s/it]
 84%|████████▍ | 189152/225000 [356:47:23<79:58:22,  8.03s/it]
 84%|████████▍ | 189153/225000 [356:47:28<71:46:28,  7.21s/it]
 84%|████████▍ | 189154/225000 [356:47:35<68:27:24,  6.88s/it]
 84%|████████▍ | 189155/225000 [356:47:41<66:25:03,  6.67s/it]
 84%|████████▍ | 189156/225000 [356:47:48<67:27:31,  6.78s/it]
 84%|████████▍ | 189157/225000 [356:47:59<81:30:22,  8.19s/it]
 84%|████████▍ | 189158/225000 [356:48:08<83:18:14,  8.37s/it]
 84%|████████▍ | 189159/225000 [356:48:16<80:45:17,  8.11s/it]
 84%|████████▍ | 189160/225000 [356:48:21<73:39:14,  7.40s/it]
 84%|████████▍ | 189161/225000 [356:48:29<74:40:31,  7.50s/it]
 84%|████████▍ | 189162/225000 [356:48:36<72:51:29,  7.32s/it]
 84%|████████▍ | 189163/225000 [356:48:41<65:04:59,  6.54s/it]
 84%|████████▍ | 189164/225000 [356:48:52<80:45:57,  8.11s/it]
 84%|████████▍ | 189165/225000 [356:49:00<79:01:04,  7.94s/it]
 84%|████████▍ | 189166/225000 [356:49:07<75:16:38,  7.56s/it]
 84%|████████▍ | 189167/225000 [356:49:15<78:16:58,  7.86s/it]
 84%|████████▍ | 189168/225000 [356:49:23<78:12:54,  7.86s/it]
 84%|████████▍ | 189169/225000 [356:49:31<79:08:23,  7.95s/it]
 84%|████████▍ | 189170/225000 [356:49:35<67:29:38,  6.78s/it]
 84%|████████▍ | 189171/225000 [356:49:41<65:46:51,  6.61s/it]
 84%|████████▍ | 189172/225000 [356:49:54<83:59:43,  8.44s/it]
 84%|████████▍ | 189173/225000 [356:49:59<72:02:18,  7.24s/it]
 84%|████████▍ | 189174/225000 [356:50:07<76:43:24,  7.71s/it]
 84%|████████▍ | 189175/225000 [356:50:13<71:33:04,  7.19s/it]
 84%|████████▍ | 189176/225000 [356:50:20<70:57:48,  7.13s/it]
 84%|████████▍ | 189177/225000 [356:50:26<65:06:38,  6.54s/it]
 84%|████████▍ | 189178/225000 [356:50:39<85:53:57,  8.63s/it]
 84%|████████▍ | 189179/225000 [356:50:47<82:38:00,  8.30s/it]
 84%|████████▍ | 189180/225000 [356:50:58<92:53:55,  9.34s/it]
 84%|████████▍ | 189181/225000 [356:51:04<80:24:58,  8.08s/it]
 84%|████████▍ | 189182/225000 [356:51:17<96:06:33,  9.66s/it]
 84%|████████▍ | 189183/225000 [356:51:22<81:11:31,  8.16s/it]
 84%|████████▍ | 189184/225000 [356:51:27<72:33:57,  7.29s/it]
 84%|████████▍ | 189185/225000 [356:51:35<75:03:45,  7.55s/it]
 84%|████████▍ | 189186/225000 [356:51:44<79:32:31,  8.00s/it]
 84%|████████▍ | 189187/225000 [356:51:51<75:55:13,  7.63s/it]
 84%|████████▍ | 189188/225000 [356:51:58<73:40:51,  7.41s/it]
 84%|████████▍ | 189189/225000 [356:52:06<77:18:17,  7.77s/it]
 84%|████████▍ | 189190/225000 [356:52:12<72:14:25,  7.26s/it]
 84%|████████▍ | 189191/225000 [356:52:20<74:08:40,  7.45s/it]
 84%|████████▍ | 189192/225000 [356:52:31<83:40:19,  8.41s/it]
 84%|████████▍ | 189193/225000 [356:52:40<85:31:09,  8.60s/it]
 84%|████████▍ | 189194/225000 [356:52:48<84:42:39,  8.52s/it]
 84%|████████▍ | 189195/225000 [356:53:00<94:35:48,  9.51s/it]
 84%|████████▍ | 189196/225000 [356:53:05<80:07:16,  8.06s/it]
 84%|████████▍ | 189197/225000 [356:53:10<72:52:43,  7.33s/it]
 84%|████████▍ | 189198/225000 [356:53:15<65:54:17,  6.63s/it]
 84%|████████▍ | 189199/225000 [356:53:28<84:13:31,  8.47s/it]
 84%|████████▍ | 189200/225000 [356:53:35<79:19:54,  7.98s/it]
                                                              
{'loss': '1.156', 'grad_norm': '1.273', 'learning_rate': '3.36e-06', 'epoch': '0.8409', 'num_input_tokens_seen': 592724920, 'train_runtime': '1.285e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189200/225000 [356:53:35<79:19:54,  7.98s/it]
 84%|████████▍ | 189201/225000 [356:53:44<81:23:58,  8.19s/it]
 84%|████████▍ | 189202/225000 [356:53:48<71:09:56,  7.16s/it]
 84%|████████▍ | 189203/225000 [356:53:54<66:13:57,  6.66s/it]
 84%|████████▍ | 189204/225000 [356:54:05<78:57:57,  7.94s/it]
 84%|████████▍ | 189205/225000 [356:54:09<67:13:42,  6.76s/it]
 84%|████████▍ | 189206/225000 [356:54:18<74:30:22,  7.49s/it]
 84%|████████▍ | 189207/225000 [356:54:26<75:36:46,  7.61s/it]
 84%|████████▍ | 189208/225000 [356:54:31<68:22:12,  6.88s/it]
 84%|████████▍ | 189209/225000 [356:54:38<69:21:49,  6.98s/it]
 84%|████████▍ | 189210/225000 [356:54:50<84:24:28,  8.49s/it]
 84%|████████▍ | 189211/225000 [356:54:59<84:11:59,  8.47s/it]
 84%|████████▍ | 189212/225000 [356:55:06<79:35:25,  8.01s/it]
 84%|████████▍ | 189213/225000 [356:55:17<88:07:58,  8.87s/it]
 84%|████████▍ | 189214/225000 [356:55:22<78:29:00,  7.90s/it]
 84%|████████▍ | 189215/225000 [356:55:29<74:48:50,  7.53s/it]
 84%|████████▍ | 189216/225000 [356:55:43<94:49:29,  9.54s/it]
 84%|████████▍ | 189217/225000 [356:55:50<88:31:55,  8.91s/it]
 84%|████████▍ | 189218/225000 [356:55:59<88:14:37,  8.88s/it]
 84%|████████▍ | 189219/225000 [356:56:05<78:28:54,  7.90s/it]
 84%|████████▍ | 189220/225000 [356:56:10<68:53:16,  6.93s/it]
 84%|████████▍ | 189221/225000 [356:56:20<77:48:02,  7.83s/it]
 84%|████████▍ | 189222/225000 [356:56:26<74:49:27,  7.53s/it]
 84%|████████▍ | 189223/225000 [356:56:34<75:39:25,  7.61s/it]
 84%|████████▍ | 189224/225000 [356:56:40<69:39:35,  7.01s/it]
 84%|████████▍ | 189225/225000 [356:56:46<66:25:03,  6.68s/it]
 84%|████████▍ | 189226/225000 [356:56:51<63:48:32,  6.42s/it]
 84%|████████▍ | 189227/225000 [356:56:57<60:53:00,  6.13s/it]
 84%|████████▍ | 189228/225000 [356:57:06<68:22:02,  6.88s/it]
 84%|████████▍ | 189229/225000 [356:57:17<80:29:48,  8.10s/it]
 84%|████████▍ | 189230/225000 [356:57:21<70:38:20,  7.11s/it]
 84%|████████▍ | 189231/225000 [356:57:33<84:08:39,  8.47s/it]
 84%|████████▍ | 189232/225000 [356:57:38<74:23:01,  7.49s/it]
 84%|████████▍ | 189233/225000 [356:57:49<84:56:29,  8.55s/it]
 84%|████████▍ | 189234/225000 [356:57:55<75:28:43,  7.60s/it]
 84%|████████▍ | 189235/225000 [356:58:05<83:34:55,  8.41s/it]
 84%|████████▍ | 189236/225000 [356:58:09<70:05:15,  7.06s/it]
 84%|████████▍ | 189237/225000 [356:58:15<66:23:01,  6.68s/it]
 84%|████████▍ | 189238/225000 [356:58:20<63:32:40,  6.40s/it]
 84%|████████▍ | 189239/225000 [356:58:27<65:29:36,  6.59s/it]
 84%|████████▍ | 189240/225000 [356:58:35<67:37:44,  6.81s/it]
 84%|████████▍ | 189241/225000 [356:58:43<73:29:07,  7.40s/it]
 84%|████████▍ | 189242/225000 [356:58:53<79:49:48,  8.04s/it]
 84%|████████▍ | 189243/225000 [356:58:58<71:03:57,  7.15s/it]
 84%|████████▍ | 189244/225000 [356:59:12<90:44:18,  9.14s/it]
 84%|████████▍ | 189245/225000 [356:59:18<80:30:15,  8.11s/it]
 84%|████████▍ | 189246/225000 [356:59:28<88:30:21,  8.91s/it]
 84%|████████▍ | 189247/225000 [356:59:35<80:54:48,  8.15s/it]
 84%|████████▍ | 189248/225000 [356:59:43<82:22:00,  8.29s/it]
 84%|████████▍ | 189249/225000 [356:59:49<74:11:22,  7.47s/it]
 84%|████████▍ | 189250/225000 [356:59:54<67:39:28,  6.81s/it]
                                                              
{'loss': '1.136', 'grad_norm': '1.423', 'learning_rate': '3.351e-06', 'epoch': '0.8411', 'num_input_tokens_seen': 592916960, 'train_runtime': '1.285e+06', 'train_tokens_per_second': '461.3'}

 84%|████████▍ | 189250/225000 [356:59:54<67:39:28,  6.81s/it]
 84%|████████▍ | 189251/225000 [356:59:59<63:08:14,  6.36s/it]
 84%|████████▍ | 189252/225000 [357:00:10<74:12:35,  7.47s/it]
 84%|████████▍ | 189253/225000 [357:00:14<66:03:15,  6.65s/it]
 84%|████████▍ | 189254/225000 [357:00:22<68:35:55,  6.91s/it]
 84%|████████▍ | 189255/225000 [357:00:28<67:56:51,  6.84s/it]
 84%|████████▍ | 189256/225000 [357:00:33<62:02:16,  6.25s/it]
 84%|████████▍ | 189257/225000 [357:00:38<57:31:58,  5.79s/it]
 84%|████████▍ | 189258/225000 [357:00:47<67:23:29,  6.79s/it]
 84%|████████▍ | 189259/225000 [357:00:52<61:24:55,  6.19s/it]
 84%|████████▍ | 189260/225000 [357:01:05<81:04:21,  8.17s/it]
 84%|████████▍ | 189261/225000 [357:01:13<80:35:39,  8.12s/it]
 84%|████████▍ | 189262/225000 [357:01:19<76:27:46,  7.70s/it]
 84%|████████▍ | 189263/225000 [357:01:30<86:12:00,  8.68s/it]
 84%|████████▍ | 189264/225000 [357:01:35<75:20:12,  7.59s/it]
 84%|████████▍ | 189265/225000 [357:01:46<85:20:11,  8.60s/it]
 84%|████████▍ | 189266/225000 [357:01:53<79:34:16,  8.02s/it]
 84%|████████▍ | 189267/225000 [357:01:59<72:45:03,  7.33s/it]
 84%|████████▍ | 189268/225000 [357:02:11<87:57:38,  8.86s/it]
 84%|████████▍ | 189269/225000 [357:02:15<72:53:31,  7.34s/it]
 84%|████████▍ | 189270/225000 [357:02:22<71:47:49,  7.23s/it]
 84%|████████▍ | 189271/225000 [357:02:34<85:03:16,  8.57s/it]
 84%|████████▍ | 189272/225000 [357:02:39<76:01:29,  7.66s/it]
 84%|████████▍ | 189273/225000 [357:02:46<74:23:42,  7.50s/it]
 84%|████████▍ | 189274/225000 [357:02:52<68:02:36,  6.86s/it]
 84%|████████▍ | 189275/225000 [357:03:00<70:54:09,  7.14s/it]
 84%|████████▍ | 189276/225000 [357:03:04<64:19:59,  6.48s/it]
 84%|████████▍ | 189277/225000 [357:03:10<62:39:20,  6.31s/it]
 84%|████████▍ | 189278/225000 [357:03:22<78:00:56,  7.86s/it]
 84%|████████▍ | 189279/225000 [357:03:26<65:51:57,  6.64s/it]
 84%|████████▍ | 189280/225000 [357:03:31<62:10:20,  6.27s/it]
 84%|████████▍ | 189281/225000 [357:03:38<64:23:04,  6.49s/it]
 84%|████████▍ | 189282/225000 [357:03:50<80:24:51,  8.10s/it]
 84%|████████▍ | 189283/225000 [357:03:55<70:13:59,  7.08s/it]
 84%|████████▍ | 189284/225000 [357:04:00<66:08:45,  6.67s/it]
 84%|████████▍ | 189285/225000 [357:04:06<61:57:32,  6.25s/it]
 84%|████████▍ | 189286/225000 [357:04:20<85:31:50,  8.62s/it]
 84%|████████▍ | 189287/225000 [357:04:27<82:11:15,  8.28s/it]
 84%|████████▍ | 189288/225000 [357:04:35<80:10:00,  8.08s/it]
 84%|████████▍ | 189289/225000 [357:04:42<76:35:11,  7.72s/it]
 84%|████████▍ | 189290/225000 [357:04:51<80:07:27,  8.08s/it]
 84%|████████▍ | 189291/225000 [357:04:57<75:09:13,  7.58s/it]
 84%|████████▍ | 189292/225000 [357:05:02<66:35:15,  6.71s/it]
 84%|████████▍ | 189293/225000 [357:05:06<60:43:38,  6.12s/it]
 84%|████████▍ | 189294/225000 [357:05:17<74:29:26,  7.51s/it]
 84%|████████▍ | 189295/225000 [357:05:25<75:04:09,  7.57s/it]
 84%|████████▍ | 189296/225000 [357:05:32<74:36:40,  7.52s/it]
 84%|████████▍ | 189297/225000 [357:05:43<83:39:23,  8.44s/it]
 84%|████████▍ | 189298/225000 [357:05:50<80:41:42,  8.14s/it]
 84%|████████▍ | 189299/225000 [357:05:58<78:12:22,  7.89s/it]
 84%|████████▍ | 189300/225000 [357:06:02<67:10:47,  6.77s/it]
                                                              
{'loss': '1.129', 'grad_norm': '1.486', 'learning_rate': '3.342e-06', 'epoch': '0.8413', 'num_input_tokens_seen': 593108264, 'train_runtime': '1.286e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189300/225000 [357:06:02<67:10:47,  6.77s/it]
 84%|████████▍ | 189301/225000 [357:06:11<73:53:30,  7.45s/it]
 84%|████████▍ | 189302/225000 [357:06:16<67:37:38,  6.82s/it]
 84%|████████▍ | 189303/225000 [357:06:21<60:24:57,  6.09s/it]
 84%|████████▍ | 189304/225000 [357:06:32<74:41:26,  7.53s/it]
 84%|████████▍ | 189305/225000 [357:06:36<65:20:05,  6.59s/it]
 84%|████████▍ | 189306/225000 [357:06:40<57:36:28,  5.81s/it]
 84%|████████▍ | 189307/225000 [357:06:51<72:05:36,  7.27s/it]
 84%|████████▍ | 189308/225000 [357:06:57<68:25:50,  6.90s/it]
 84%|████████▍ | 189309/225000 [357:07:06<74:39:40,  7.53s/it]
 84%|████████▍ | 189310/225000 [357:07:10<64:10:38,  6.47s/it]
 84%|████████▍ | 189311/225000 [357:07:23<84:55:59,  8.57s/it]
 84%|████████▍ | 189312/225000 [357:07:28<72:44:24,  7.34s/it]
 84%|████████▍ | 189313/225000 [357:07:32<65:09:01,  6.57s/it]
 84%|████████▍ | 189314/225000 [357:07:37<59:26:00,  6.00s/it]
 84%|████████▍ | 189315/225000 [357:07:45<66:23:05,  6.70s/it]
 84%|████████▍ | 189316/225000 [357:07:57<82:20:45,  8.31s/it]
 84%|████████▍ | 189317/225000 [357:08:07<86:58:12,  8.77s/it]
 84%|████████▍ | 189318/225000 [357:08:12<74:56:19,  7.56s/it]
 84%|████████▍ | 189319/225000 [357:08:22<82:55:59,  8.37s/it]
 84%|████████▍ | 189320/225000 [357:08:32<85:51:31,  8.66s/it]
 84%|████████▍ | 189321/225000 [357:08:37<75:57:31,  7.66s/it]
 84%|████████▍ | 189322/225000 [357:08:42<69:06:47,  6.97s/it]
 84%|████████▍ | 189323/225000 [357:08:53<79:36:18,  8.03s/it]
 84%|████████▍ | 189324/225000 [357:09:02<83:36:56,  8.44s/it]
 84%|████████▍ | 189325/225000 [357:09:06<70:40:01,  7.13s/it]
 84%|████████▍ | 189326/225000 [357:09:12<65:46:40,  6.64s/it]
 84%|████████▍ | 189327/225000 [357:09:20<72:01:08,  7.27s/it]
 84%|████████▍ | 189328/225000 [357:09:30<77:26:57,  7.82s/it]
 84%|████████▍ | 189329/225000 [357:09:36<74:01:37,  7.47s/it]
 84%|████████▍ | 189330/225000 [357:09:43<70:56:22,  7.16s/it]
 84%|████████▍ | 189331/225000 [357:09:48<64:19:04,  6.49s/it]
 84%|████████▍ | 189332/225000 [357:09:56<70:11:37,  7.08s/it]
 84%|████████▍ | 189333/225000 [357:10:02<68:07:20,  6.88s/it]
 84%|████████▍ | 189334/225000 [357:10:09<66:07:34,  6.67s/it]
 84%|████████▍ | 189335/225000 [357:10:19<75:45:31,  7.65s/it]
 84%|████████▍ | 189336/225000 [357:10:28<81:11:07,  8.20s/it]
 84%|████████▍ | 189337/225000 [357:10:32<69:38:04,  7.03s/it]
 84%|████████▍ | 189338/225000 [357:10:38<66:48:40,  6.74s/it]
 84%|████████▍ | 189339/225000 [357:10:49<79:34:12,  8.03s/it]
 84%|████████▍ | 189340/225000 [357:10:55<70:54:38,  7.16s/it]
 84%|████████▍ | 189341/225000 [357:11:03<75:14:36,  7.60s/it]
 84%|████████▍ | 189342/225000 [357:11:08<65:48:28,  6.64s/it]
 84%|████████▍ | 189343/225000 [357:11:18<78:08:41,  7.89s/it]
 84%|████████▍ | 189344/225000 [357:11:24<69:49:01,  7.05s/it]
 84%|████████▍ | 189345/225000 [357:11:31<71:20:41,  7.20s/it]
 84%|████████▍ | 189346/225000 [357:11:35<62:52:17,  6.35s/it]
 84%|████████▍ | 189347/225000 [357:11:45<71:47:15,  7.25s/it]
 84%|████████▍ | 189348/225000 [357:11:51<68:11:31,  6.89s/it]
 84%|████████▍ | 189349/225000 [357:11:56<63:36:13,  6.42s/it]
 84%|████████▍ | 189350/225000 [357:12:01<59:13:22,  5.98s/it]
                                                              
{'loss': '1.12', 'grad_norm': '1.254', 'learning_rate': '3.333e-06', 'epoch': '0.8416', 'num_input_tokens_seen': 593294840, 'train_runtime': '1.286e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189350/225000 [357:12:01<59:13:22,  5.98s/it]
 84%|████████▍ | 189351/225000 [357:12:12<74:53:39,  7.56s/it]
 84%|████████▍ | 189352/225000 [357:12:20<76:31:59,  7.73s/it]
 84%|████████▍ | 189353/225000 [357:12:25<68:10:14,  6.88s/it]
 84%|████████▍ | 189354/225000 [357:12:34<74:09:13,  7.49s/it]
 84%|████████▍ | 189355/225000 [357:12:43<78:58:55,  7.98s/it]
 84%|████████▍ | 189356/225000 [357:12:53<83:44:35,  8.46s/it]
 84%|████████▍ | 189357/225000 [357:13:02<84:31:48,  8.54s/it]
 84%|████████▍ | 189358/225000 [357:13:08<78:11:18,  7.90s/it]
 84%|████████▍ | 189359/225000 [357:13:18<83:38:35,  8.45s/it]
 84%|████████▍ | 189360/225000 [357:13:26<83:33:43,  8.44s/it]
 84%|████████▍ | 189361/225000 [357:13:32<75:06:51,  7.59s/it]
 84%|████████▍ | 189362/225000 [357:13:43<85:27:06,  8.63s/it]
 84%|████████▍ | 189363/225000 [357:13:48<74:05:10,  7.48s/it]
 84%|████████▍ | 189364/225000 [357:13:54<70:42:26,  7.14s/it]
 84%|████████▍ | 189365/225000 [357:14:05<81:50:18,  8.27s/it]
 84%|████████▍ | 189366/225000 [357:14:11<74:48:31,  7.56s/it]
 84%|████████▍ | 189367/225000 [357:14:21<81:44:38,  8.26s/it]
 84%|████████▍ | 189368/225000 [357:14:27<77:03:12,  7.78s/it]
 84%|████████▍ | 189369/225000 [357:14:36<79:09:45,  8.00s/it]
 84%|████████▍ | 189370/225000 [357:14:46<84:53:32,  8.58s/it]
 84%|████████▍ | 189371/225000 [357:14:54<82:16:05,  8.31s/it]
 84%|████████▍ | 189372/225000 [357:15:01<80:13:24,  8.11s/it]
 84%|████████▍ | 189373/225000 [357:15:09<79:20:38,  8.02s/it]
 84%|████████▍ | 189374/225000 [357:15:14<71:24:37,  7.22s/it]
 84%|████████▍ | 189375/225000 [357:15:20<66:30:49,  6.72s/it]
 84%|████████▍ | 189376/225000 [357:15:29<74:06:55,  7.49s/it]
 84%|████████▍ | 189377/225000 [357:15:33<64:25:03,  6.51s/it]
 84%|████████▍ | 189378/225000 [357:15:39<60:10:43,  6.08s/it]
 84%|████████▍ | 189379/225000 [357:15:46<63:31:15,  6.42s/it]
 84%|████████▍ | 189380/225000 [357:15:56<75:49:30,  7.66s/it]
 84%|████████▍ | 189381/225000 [357:16:02<70:06:34,  7.09s/it]
 84%|████████▍ | 189382/225000 [357:16:15<86:13:05,  8.71s/it]
 84%|████████▍ | 189383/225000 [357:16:19<72:45:16,  7.35s/it]
 84%|████████▍ | 189384/225000 [357:16:24<66:07:02,  6.68s/it]
 84%|████████▍ | 189385/225000 [357:16:33<73:33:34,  7.44s/it]
 84%|████████▍ | 189386/225000 [357:16:42<78:42:05,  7.96s/it]
 84%|████████▍ | 189387/225000 [357:16:46<67:22:06,  6.81s/it]
 84%|████████▍ | 189388/225000 [357:16:51<61:30:48,  6.22s/it]
 84%|████████▍ | 189389/225000 [357:17:00<69:22:11,  7.01s/it]
 84%|████████▍ | 189390/225000 [357:17:10<77:44:27,  7.86s/it]
 84%|████████▍ | 189391/225000 [357:17:18<78:28:13,  7.93s/it]
 84%|████████▍ | 189392/225000 [357:17:26<80:06:00,  8.10s/it]
 84%|████████▍ | 189393/225000 [357:17:33<75:48:01,  7.66s/it]
 84%|████████▍ | 189394/225000 [357:17:38<68:46:12,  6.95s/it]
 84%|████████▍ | 189395/225000 [357:17:48<77:02:53,  7.79s/it]
 84%|████████▍ | 189396/225000 [357:17:54<69:59:51,  7.08s/it]
 84%|████████▍ | 189397/225000 [357:17:59<64:38:53,  6.54s/it]
 84%|████████▍ | 189398/225000 [357:18:07<70:47:05,  7.16s/it]
 84%|████████▍ | 189399/225000 [357:18:15<72:47:04,  7.36s/it]
 84%|████████▍ | 189400/225000 [357:18:26<82:15:39,  8.32s/it]
                                                              
{'loss': '1.139', 'grad_norm': '1.559', 'learning_rate': '3.323e-06', 'epoch': '0.8418', 'num_input_tokens_seen': 593487552, 'train_runtime': '1.286e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189400/225000 [357:18:26<82:15:39,  8.32s/it]
 84%|████████▍ | 189401/225000 [357:18:32<75:58:52,  7.68s/it]
 84%|████████▍ | 189402/225000 [357:18:44<88:43:25,  8.97s/it]
 84%|████████▍ | 189403/225000 [357:18:52<86:10:39,  8.72s/it]
 84%|████████▍ | 189404/225000 [357:18:58<78:10:39,  7.91s/it]
 84%|████████▍ | 189405/225000 [357:19:04<71:38:14,  7.25s/it]
 84%|████████▍ | 189406/225000 [357:19:10<68:44:16,  6.95s/it]
 84%|████████▍ | 189407/225000 [357:19:25<90:59:10,  9.20s/it]
 84%|████████▍ | 189408/225000 [357:19:29<77:04:02,  7.80s/it]
 84%|████████▍ | 189409/225000 [357:19:35<72:55:52,  7.38s/it]
 84%|████████▍ | 189410/225000 [357:19:40<64:44:15,  6.55s/it]
 84%|████████▍ | 189411/225000 [357:19:46<63:08:57,  6.39s/it]
 84%|████████▍ | 189412/225000 [357:19:52<61:19:00,  6.20s/it]
 84%|████████▍ | 189413/225000 [357:20:07<86:58:43,  8.80s/it]
 84%|████████▍ | 189414/225000 [357:20:11<74:41:38,  7.56s/it]
 84%|████████▍ | 189415/225000 [357:20:16<66:46:32,  6.76s/it]
 84%|████████▍ | 189416/225000 [357:20:27<79:34:04,  8.05s/it]
 84%|████████▍ | 189417/225000 [357:20:37<84:34:38,  8.56s/it]
 84%|████████▍ | 189418/225000 [357:20:49<93:07:07,  9.42s/it]
 84%|████████▍ | 189419/225000 [357:20:53<77:51:37,  7.88s/it]
 84%|████████▍ | 189420/225000 [357:20:58<70:44:38,  7.16s/it]
 84%|████████▍ | 189421/225000 [357:21:05<70:20:08,  7.12s/it]
 84%|████████▍ | 189422/225000 [357:21:15<77:11:30,  7.81s/it]
 84%|████████▍ | 189423/225000 [357:21:20<70:08:37,  7.10s/it]
 84%|████████▍ | 189424/225000 [357:21:24<61:50:00,  6.26s/it]
 84%|████████▍ | 189425/225000 [357:21:34<71:09:27,  7.20s/it]
 84%|████████▍ | 189426/225000 [357:21:42<73:17:35,  7.42s/it]
 84%|████████▍ | 189427/225000 [357:21:47<65:36:28,  6.64s/it]
 84%|████████▍ | 189428/225000 [357:21:57<76:31:42,  7.74s/it]
 84%|████████▍ | 189429/225000 [357:22:02<69:28:07,  7.03s/it]
 84%|████████▍ | 189430/225000 [357:22:11<75:50:27,  7.68s/it]
 84%|████████▍ | 189431/225000 [357:22:18<72:17:13,  7.32s/it]
 84%|████████▍ | 189432/225000 [357:22:24<69:57:06,  7.08s/it]
 84%|████████▍ | 189433/225000 [357:22:35<81:00:57,  8.20s/it]
 84%|████████▍ | 189434/225000 [357:22:43<80:47:25,  8.18s/it]
 84%|████████▍ | 189435/225000 [357:22:54<87:34:49,  8.87s/it]
 84%|████████▍ | 189436/225000 [357:23:00<80:08:13,  8.11s/it]
 84%|████████▍ | 189437/225000 [357:23:05<71:25:44,  7.23s/it]
 84%|████████▍ | 189438/225000 [357:23:10<64:25:24,  6.52s/it]
 84%|████████▍ | 189439/225000 [357:23:15<60:25:52,  6.12s/it]
 84%|████████▍ | 189440/225000 [357:23:28<79:11:59,  8.02s/it]
 84%|████████▍ | 189441/225000 [357:23:40<90:04:41,  9.12s/it]
 84%|████████▍ | 189442/225000 [357:23:50<93:07:23,  9.43s/it]
 84%|████████▍ | 189443/225000 [357:23:55<80:24:03,  8.14s/it]
 84%|████████▍ | 189444/225000 [357:24:03<80:35:29,  8.16s/it]
 84%|████████▍ | 189445/225000 [357:24:07<68:42:18,  6.96s/it]
 84%|████████▍ | 189446/225000 [357:24:12<63:16:49,  6.41s/it]
 84%|████████▍ | 189447/225000 [357:24:24<77:59:46,  7.90s/it]
 84%|████████▍ | 189448/225000 [357:24:29<68:45:12,  6.96s/it]
 84%|████████▍ | 189449/225000 [357:24:35<66:38:06,  6.75s/it]
 84%|████████▍ | 189450/225000 [357:24:40<61:35:16,  6.24s/it]
                                                              
{'loss': '1.163', 'grad_norm': '1.54', 'learning_rate': '3.314e-06', 'epoch': '0.842', 'num_input_tokens_seen': 593681744, 'train_runtime': '1.287e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189450/225000 [357:24:40<61:35:16,  6.24s/it]
 84%|████████▍ | 189451/225000 [357:24:53<82:30:33,  8.36s/it]
 84%|████████▍ | 189452/225000 [357:24:57<69:21:26,  7.02s/it]
 84%|████████▍ | 189453/225000 [357:25:03<64:44:37,  6.56s/it]
 84%|████████▍ | 189454/225000 [357:25:09<65:55:24,  6.68s/it]
 84%|████████▍ | 189455/225000 [357:25:23<87:41:19,  8.88s/it]
 84%|████████▍ | 189456/225000 [357:25:29<76:59:37,  7.80s/it]
 84%|████████▍ | 189457/225000 [357:25:36<75:57:14,  7.69s/it]
 84%|████████▍ | 189458/225000 [357:25:44<75:07:29,  7.61s/it]
 84%|████████▍ | 189459/225000 [357:25:52<76:25:53,  7.74s/it]
 84%|████████▍ | 189460/225000 [357:25:57<69:33:38,  7.05s/it]
 84%|████████▍ | 189461/225000 [357:26:11<88:29:49,  8.96s/it]
 84%|████████▍ | 189462/225000 [357:26:18<84:01:39,  8.51s/it]
 84%|████████▍ | 189463/225000 [357:26:22<71:45:29,  7.27s/it]
 84%|████████▍ | 189464/225000 [357:26:33<80:57:28,  8.20s/it]
 84%|████████▍ | 189465/225000 [357:26:41<80:15:41,  8.13s/it]
 84%|████████▍ | 189466/225000 [357:26:48<77:23:55,  7.84s/it]
 84%|████████▍ | 189467/225000 [357:26:54<73:05:44,  7.41s/it]
 84%|████████▍ | 189468/225000 [357:27:01<71:15:12,  7.22s/it]
 84%|████████▍ | 189469/225000 [357:27:07<68:49:10,  6.97s/it]
 84%|████████▍ | 189470/225000 [357:27:16<73:49:23,  7.48s/it]
 84%|████████▍ | 189471/225000 [357:27:23<72:07:32,  7.31s/it]
 84%|████████▍ | 189472/225000 [357:27:28<66:03:11,  6.69s/it]
 84%|████████▍ | 189473/225000 [357:27:33<60:22:30,  6.12s/it]
 84%|████████▍ | 189474/225000 [357:27:42<69:30:50,  7.04s/it]
 84%|████████▍ | 189475/225000 [357:27:50<70:27:31,  7.14s/it]
 84%|████████▍ | 189476/225000 [357:27:57<71:47:48,  7.28s/it]
 84%|████████▍ | 189477/225000 [357:28:07<78:37:39,  7.97s/it]
 84%|████████▍ | 189478/225000 [357:28:17<85:41:58,  8.69s/it]
 84%|████████▍ | 189479/225000 [357:28:24<80:38:44,  8.17s/it]
 84%|████████▍ | 189480/225000 [357:28:30<73:11:58,  7.42s/it]
 84%|████████▍ | 189481/225000 [357:28:35<67:38:42,  6.86s/it]
 84%|████████▍ | 189482/225000 [357:28:47<82:00:41,  8.31s/it]
 84%|████████▍ | 189483/225000 [357:28:54<78:15:02,  7.93s/it]
 84%|████████▍ | 189484/225000 [357:28:59<68:03:17,  6.90s/it]
 84%|████████▍ | 189485/225000 [357:29:10<80:12:10,  8.13s/it]
 84%|████████▍ | 189486/225000 [357:29:15<71:36:15,  7.26s/it]
 84%|████████▍ | 189487/225000 [357:29:20<66:35:58,  6.75s/it]
 84%|████████▍ | 189488/225000 [357:29:32<82:20:25,  8.35s/it]
 84%|████████▍ | 189489/225000 [357:29:38<73:04:27,  7.41s/it]
 84%|████████▍ | 189490/225000 [357:29:48<81:52:34,  8.30s/it]
 84%|████████▍ | 189491/225000 [357:29:54<73:59:07,  7.50s/it]
 84%|████████▍ | 189492/225000 [357:29:58<64:39:43,  6.56s/it]
 84%|████████▍ | 189493/225000 [357:30:07<71:01:27,  7.20s/it]
 84%|████████▍ | 189494/225000 [357:30:11<62:10:15,  6.30s/it]
 84%|████████▍ | 189495/225000 [357:30:20<70:27:40,  7.14s/it]
 84%|████████▍ | 189496/225000 [357:30:25<63:30:05,  6.44s/it]
 84%|████████▍ | 189497/225000 [357:30:29<57:14:40,  5.80s/it]
 84%|████████▍ | 189498/225000 [357:30:41<74:56:20,  7.60s/it]
 84%|████████▍ | 189499/225000 [357:30:47<70:05:52,  7.11s/it]
 84%|████████▍ | 189500/225000 [357:30:58<83:07:58,  8.43s/it]
                                                              
{'loss': '1.142', 'grad_norm': '1.576', 'learning_rate': '3.305e-06', 'epoch': '0.8422', 'num_input_tokens_seen': 593869976, 'train_runtime': '1.287e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189500/225000 [357:30:58<83:07:58,  8.43s/it]
 84%|████████▍ | 189501/225000 [357:31:04<74:27:47,  7.55s/it]
 84%|████████▍ | 189502/225000 [357:31:09<66:56:54,  6.79s/it]
 84%|████████▍ | 189503/225000 [357:31:16<68:16:46,  6.92s/it]
 84%|████████▍ | 189504/225000 [357:31:24<69:57:26,  7.10s/it]
 84%|████████▍ | 189505/225000 [357:31:30<67:58:28,  6.89s/it]
 84%|████████▍ | 189506/225000 [357:31:48<99:19:42, 10.07s/it]
 84%|████████▍ | 189507/225000 [357:31:50<77:32:37,  7.87s/it]
 84%|████████▍ | 189508/225000 [357:32:03<92:49:53,  9.42s/it]
 84%|████████▍ | 189509/225000 [357:32:09<81:58:48,  8.32s/it]
 84%|████████▍ | 189510/225000 [357:32:16<77:04:51,  7.82s/it]
 84%|████████▍ | 189511/225000 [357:32:21<69:59:25,  7.10s/it]
 84%|████████▍ | 189512/225000 [357:32:28<68:42:05,  6.97s/it]
 84%|████████▍ | 189513/225000 [357:32:34<66:48:31,  6.78s/it]
 84%|████████▍ | 189514/225000 [357:32:46<80:21:27,  8.15s/it]
 84%|████████▍ | 189515/225000 [357:32:52<75:42:21,  7.68s/it]
 84%|████████▍ | 189516/225000 [357:33:01<80:39:03,  8.18s/it]
 84%|████████▍ | 189517/225000 [357:33:11<83:17:57,  8.45s/it]
 84%|████████▍ | 189518/225000 [357:33:17<78:34:07,  7.97s/it]
 84%|████████▍ | 189519/225000 [357:33:24<74:04:42,  7.52s/it]
 84%|████████▍ | 189520/225000 [357:33:32<77:18:26,  7.84s/it]
 84%|████████▍ | 189521/225000 [357:33:41<77:58:25,  7.91s/it]
 84%|████████▍ | 189522/225000 [357:33:47<73:36:47,  7.47s/it]
 84%|████████▍ | 189523/225000 [357:33:59<85:42:59,  8.70s/it]
 84%|████████▍ | 189524/225000 [357:34:07<84:43:06,  8.60s/it]
 84%|████████▍ | 189525/225000 [357:34:14<80:17:43,  8.15s/it]
 84%|████████▍ | 189526/225000 [357:34:18<68:28:05,  6.95s/it]
 84%|████████▍ | 189527/225000 [357:34:24<64:44:06,  6.57s/it]
 84%|████████▍ | 189528/225000 [357:34:34<75:57:52,  7.71s/it]
 84%|████████▍ | 189529/225000 [357:34:41<72:12:33,  7.33s/it]
 84%|████████▍ | 189530/225000 [357:34:46<66:49:50,  6.78s/it]
 84%|████████▍ | 189531/225000 [357:34:55<73:59:01,  7.51s/it]
 84%|████████▍ | 189532/225000 [357:35:00<64:30:59,  6.55s/it]
 84%|████████▍ | 189533/225000 [357:35:04<56:36:27,  5.75s/it]
 84%|████████▍ | 189534/225000 [357:35:11<62:16:51,  6.32s/it]
 84%|████████▍ | 189535/225000 [357:35:20<70:30:48,  7.16s/it]
 84%|████████▍ | 189536/225000 [357:35:25<62:17:05,  6.32s/it]
 84%|████████▍ | 189537/225000 [357:35:36<77:49:26,  7.90s/it]
 84%|████████▍ | 189538/225000 [357:35:44<76:13:12,  7.74s/it]
 84%|████████▍ | 189539/225000 [357:35:52<79:24:34,  8.06s/it]
 84%|████████▍ | 189540/225000 [357:35:57<69:08:35,  7.02s/it]
 84%|████████▍ | 189541/225000 [357:36:03<65:01:59,  6.60s/it]
 84%|████████▍ | 189542/225000 [357:36:14<80:22:11,  8.16s/it]
 84%|████████▍ | 189543/225000 [357:36:26<90:31:02,  9.19s/it]
 84%|████████▍ | 189544/225000 [357:36:34<86:39:54,  8.80s/it]
 84%|████████▍ | 189545/225000 [357:36:41<81:19:22,  8.26s/it]
 84%|████████▍ | 189546/225000 [357:36:49<80:34:57,  8.18s/it]
 84%|████████▍ | 189547/225000 [357:36:58<84:33:00,  8.59s/it]
 84%|████████▍ | 189548/225000 [357:37:08<88:18:09,  8.97s/it]
 84%|████████▍ | 189549/225000 [357:37:14<80:01:51,  8.13s/it]
 84%|████████▍ | 189550/225000 [357:37:23<82:24:21,  8.37s/it]
                                                              
{'loss': '1.156', 'grad_norm': '1.482', 'learning_rate': '3.296e-06', 'epoch': '0.8424', 'num_input_tokens_seen': 594070576, 'train_runtime': '1.287e+06', 'train_tokens_per_second': '461.4'}

 84%|████████▍ | 189550/225000 [357:37:23<82:24:21,  8.37s/it]
 84%|████████▍ | 189551/225000 [357:37:28<71:19:28,  7.24s/it]
 84%|████████▍ | 189552/225000 [357:37:32<63:05:53,  6.41s/it]
 84%|████████▍ | 189553/225000 [357:37:39<62:29:05,  6.35s/it]
 84%|████████▍ | 189554/225000 [357:37:45<62:45:02,  6.37s/it]
 84%|████████▍ | 189555/225000 [357:37:51<62:20:24,  6.33s/it]
 84%|████████▍ | 189556/225000 [357:38:04<80:12:20,  8.15s/it]
 84%|████████▍ | 189557/225000 [357:38:07<67:14:52,  6.83s/it]
 84%|████████▍ | 189558/225000 [357:38:15<68:53:40,  7.00s/it]
 84%|████████▍ | 189559/225000 [357:38:27<83:25:51,  8.47s/it]
 84%|████████▍ | 189560/225000 [357:38:36<85:19:35,  8.67s/it]
 84%|████████▍ | 189561/225000 [357:38:41<73:50:57,  7.50s/it]
 84%|████████▍ | 189562/225000 [357:38:52<83:53:43,  8.52s/it]
 84%|████████▍ | 189563/225000 [357:38:57<75:03:24,  7.62s/it]
 84%|████████▍ | 189564/225000 [357:39:12<97:41:09,  9.92s/it]
 84%|████████▍ | 189565/225000 [357:39:20<91:20:36,  9.28s/it]
 84%|████████▍ | 189566/225000 [357:39:30<91:34:33,  9.30s/it]
 84%|████████▍ | 189567/225000 [357:39:35<80:26:11,  8.17s/it]
 84%|████████▍ | 189568/225000 [357:39:47<90:49:06,  9.23s/it]
 84%|████████▍ | 189569/225000 [357:39:51<75:55:26,  7.71s/it]
 84%|████████▍ | 189570/225000 [357:39:59<76:34:33,  7.78s/it]
 84%|████████▍ | 189571/225000 [357:40:04<69:43:26,  7.08s/it]
 84%|████████▍ | 189572/225000 [357:40:12<69:53:21,  7.10s/it]
 84%|████████▍ | 189573/225000 [357:40:20<73:23:52,  7.46s/it]
 84%|████████▍ | 189574/225000 [357:40:30<81:31:03,  8.28s/it]
 84%|████████▍ | 189575/225000 [357:40:39<83:20:32,  8.47s/it]
 84%|████████▍ | 189576/225000 [357:40:51<95:10:53,  9.67s/it]
 84%|████████▍ | 189577/225000 [357:40:59<88:41:51,  9.01s/it]
 84%|████████▍ | 189578/225000 [357:41:11<96:38:58,  9.82s/it]
 84%|████████▍ | 189579/225000 [357:41:16<85:07:40,  8.65s/it]
 84%|████████▍ | 189580/225000 [357:41:21<73:44:54,  7.50s/it]
 84%|████████▍ | 189581/225000 [357:41:29<73:16:01,  7.45s/it]
 84%|████████▍ | 189582/225000 [357:41:39<81:51:43,  8.32s/it]
 84%|████████▍ | 189583/225000 [357:41:51<91:20:50,  9.29s/it]
 84%|████████▍ | 189584/225000 [357:41:56<80:40:24,  8.20s/it]
 84%|████████▍ | 189585/225000 [357:42:05<83:48:43,  8.52s/it]
 84%|████████▍ | 189586/225000 [357:42:13<79:33:48,  8.09s/it]
 84%|████████▍ | 189587/225000 [357:42:20<78:13:43,  7.95s/it]
 84%|████████▍ | 189588/225000 [357:42:25<68:50:26,  7.00s/it]
 84%|████████▍ | 189589/225000 [357:42:32<70:13:29,  7.14s/it]
 84%|████████▍ | 189590/225000 [357:42:40<72:12:23,  7.34s/it]
 84%|████████▍ | 189591/225000 [357:42:54<92:10:18,  9.37s/it]
 84%|████████▍ | 189592/225000 [357:43:02<88:12:22,  8.97s/it]
 84%|████████▍ | 189593/225000 [357:43:06<73:49:42,  7.51s/it]
 84%|████████▍ | 189594/225000 [357:43:11<65:58:54,  6.71s/it]
 84%|████████▍ | 189595/225000 [357:43:22<76:22:25,  7.77s/it]
 84%|████████▍ | 189596/225000 [357:43:27<69:42:28,  7.09s/it]
 84%|████████▍ | 189597/225000 [357:43:36<75:29:25,  7.68s/it]
 84%|████████▍ | 189598/225000 [357:43:42<69:17:13,  7.05s/it]
 84%|████████▍ | 189599/225000 [357:43:47<65:29:16,  6.66s/it]
 84%|████████▍ | 189600/225000 [357:43:53<61:21:18,  6.24s/it]
                                                              
{'loss': '1.152', 'grad_norm': '1.473', 'learning_rate': '3.287e-06', 'epoch': '0.8427', 'num_input_tokens_seen': 594274408, 'train_runtime': '1.288e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189600/225000 [357:43:53<61:21:18,  6.24s/it]
 84%|████████▍ | 189601/225000 [357:43:59<61:36:23,  6.27s/it]
 84%|████████▍ | 189602/225000 [357:44:06<63:57:22,  6.50s/it]
 84%|████████▍ | 189603/225000 [357:44:15<72:34:20,  7.38s/it]
 84%|████████▍ | 189604/225000 [357:44:21<67:27:04,  6.86s/it]
 84%|████████▍ | 189605/225000 [357:44:31<77:07:50,  7.84s/it]
 84%|████████▍ | 189606/225000 [357:44:34<63:26:56,  6.45s/it]
 84%|████████▍ | 189607/225000 [357:44:41<64:53:48,  6.60s/it]
 84%|████████▍ | 189608/225000 [357:44:47<62:58:34,  6.41s/it]
 84%|████████▍ | 189609/225000 [357:44:56<69:46:49,  7.10s/it]
 84%|████████▍ | 189610/225000 [357:45:07<80:06:07,  8.15s/it]
 84%|████████▍ | 189611/225000 [357:45:14<77:30:19,  7.88s/it]
 84%|████████▍ | 189612/225000 [357:45:25<85:25:47,  8.69s/it]
 84%|████████▍ | 189613/225000 [357:45:37<95:25:19,  9.71s/it]
 84%|████████▍ | 189614/225000 [357:45:44<88:11:26,  8.97s/it]
 84%|████████▍ | 189615/225000 [357:45:52<87:02:05,  8.85s/it]
 84%|████████▍ | 189616/225000 [357:45:58<76:38:59,  7.80s/it]
 84%|████████▍ | 189617/225000 [357:46:06<77:12:45,  7.86s/it]
 84%|████████▍ | 189618/225000 [357:46:10<65:19:18,  6.65s/it]
 84%|████████▍ | 189619/225000 [357:46:18<70:57:38,  7.22s/it]
 84%|████████▍ | 189620/225000 [357:46:22<60:52:01,  6.19s/it]
 84%|████████▍ | 189621/225000 [357:46:27<58:06:32,  5.91s/it]
 84%|████████▍ | 189622/225000 [357:46:36<65:19:18,  6.65s/it]
 84%|████████▍ | 189623/225000 [357:46:43<68:42:51,  6.99s/it]
 84%|████████▍ | 189624/225000 [357:46:49<65:28:13,  6.66s/it]
 84%|████████▍ | 189625/225000 [357:46:56<66:47:38,  6.80s/it]
 84%|████████▍ | 189626/225000 [357:47:07<77:57:53,  7.93s/it]
 84%|████████▍ | 189627/225000 [357:47:12<68:27:22,  6.97s/it]
 84%|████████▍ | 189628/225000 [357:47:21<74:24:48,  7.57s/it]
 84%|████████▍ | 189629/225000 [357:47:27<71:12:06,  7.25s/it]
 84%|████████▍ | 189630/225000 [357:47:32<64:19:51,  6.55s/it]
 84%|████████▍ | 189631/225000 [357:47:43<76:26:49,  7.78s/it]
 84%|████████▍ | 189632/225000 [357:47:50<75:02:50,  7.64s/it]
 84%|████████▍ | 189633/225000 [357:47:55<68:04:32,  6.93s/it]
 84%|████████▍ | 189634/225000 [357:48:04<72:43:21,  7.40s/it]
 84%|████████▍ | 189635/225000 [357:48:12<73:57:54,  7.53s/it]
 84%|████████▍ | 189636/225000 [357:48:20<75:38:11,  7.70s/it]
 84%|████████▍ | 189637/225000 [357:48:29<81:08:31,  8.26s/it]
 84%|████████▍ | 189638/225000 [357:48:38<81:17:28,  8.28s/it]
 84%|████████▍ | 189639/225000 [357:48:45<77:11:51,  7.86s/it]
 84%|████████▍ | 189640/225000 [357:48:50<71:28:07,  7.28s/it]
 84%|████████▍ | 189641/225000 [357:48:58<71:25:05,  7.27s/it]
 84%|████████▍ | 189642/225000 [357:49:09<83:44:06,  8.53s/it]
 84%|████████▍ | 189643/225000 [357:49:14<72:13:07,  7.35s/it]
 84%|████████▍ | 189644/225000 [357:49:23<78:45:54,  8.02s/it]
 84%|████████▍ | 189645/225000 [357:49:28<68:30:17,  6.98s/it]
 84%|████████▍ | 189646/225000 [357:49:38<78:52:53,  8.03s/it]
 84%|████████▍ | 189647/225000 [357:49:47<79:24:11,  8.09s/it]
 84%|████████▍ | 189648/225000 [357:49:56<82:11:53,  8.37s/it]
 84%|████████▍ | 189649/225000 [357:50:04<82:54:41,  8.44s/it]
 84%|████████▍ | 189650/225000 [357:50:08<70:31:59,  7.18s/it]
                                                              
{'loss': '1.129', 'grad_norm': '1.145', 'learning_rate': '3.278e-06', 'epoch': '0.8429', 'num_input_tokens_seen': 594465752, 'train_runtime': '1.288e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189650/225000 [357:50:08<70:31:59,  7.18s/it]
 84%|████████▍ | 189651/225000 [357:50:16<70:40:17,  7.20s/it]
 84%|████████▍ | 189652/225000 [357:50:23<70:00:57,  7.13s/it]
 84%|████████▍ | 189653/225000 [357:50:35<85:37:51,  8.72s/it]
 84%|████████▍ | 189654/225000 [357:50:41<76:06:59,  7.75s/it]
 84%|████████▍ | 189655/225000 [357:50:47<71:12:17,  7.25s/it]
 84%|████████▍ | 189656/225000 [357:50:52<65:38:08,  6.69s/it]
 84%|████████▍ | 189657/225000 [357:50:58<63:50:37,  6.50s/it]
 84%|████████▍ | 189658/225000 [357:51:05<64:26:07,  6.56s/it]
 84%|████████▍ | 189659/225000 [357:51:19<87:44:16,  8.94s/it]
 84%|████████▍ | 189660/225000 [357:51:24<76:17:04,  7.77s/it]
 84%|████████▍ | 189661/225000 [357:51:34<83:00:59,  8.46s/it]
 84%|████████▍ | 189662/225000 [357:51:38<70:05:49,  7.14s/it]
 84%|████████▍ | 189663/225000 [357:51:51<85:39:04,  8.73s/it]
 84%|████████▍ | 189664/225000 [357:51:56<74:57:48,  7.64s/it]
 84%|████████▍ | 189665/225000 [357:52:07<83:23:34,  8.50s/it]
 84%|████████▍ | 189666/225000 [357:52:11<72:25:10,  7.38s/it]
 84%|████████▍ | 189667/225000 [357:52:16<64:27:16,  6.57s/it]
 84%|████████▍ | 189668/225000 [357:52:23<65:03:46,  6.63s/it]
 84%|████████▍ | 189669/225000 [357:52:35<81:08:47,  8.27s/it]
 84%|████████▍ | 189670/225000 [357:52:41<74:12:27,  7.56s/it]
 84%|████████▍ | 189671/225000 [357:52:52<84:07:28,  8.57s/it]
 84%|████████▍ | 189672/225000 [357:52:56<70:48:52,  7.22s/it]
 84%|████████▍ | 189673/225000 [357:53:05<76:12:00,  7.77s/it]
 84%|████████▍ | 189674/225000 [357:53:10<70:06:40,  7.14s/it]
 84%|████████▍ | 189675/225000 [357:53:18<70:20:02,  7.17s/it]
 84%|████████▍ | 189676/225000 [357:53:25<69:41:10,  7.10s/it]
 84%|████████▍ | 189677/225000 [357:53:31<66:10:50,  6.74s/it]
 84%|████████▍ | 189678/225000 [357:53:43<83:16:12,  8.49s/it]
 84%|████████▍ | 189679/225000 [357:53:52<85:30:17,  8.71s/it]
 84%|████████▍ | 189680/225000 [357:53:59<79:13:00,  8.07s/it]
 84%|████████▍ | 189681/225000 [357:54:06<75:29:22,  7.69s/it]
 84%|████████▍ | 189682/225000 [357:54:11<69:29:14,  7.08s/it]
 84%|████████▍ | 189683/225000 [357:54:18<69:31:03,  7.09s/it]
 84%|████████▍ | 189684/225000 [357:54:32<87:57:49,  8.97s/it]
 84%|████████▍ | 189685/225000 [357:54:37<76:54:53,  7.84s/it]
 84%|████████▍ | 189686/225000 [357:54:45<76:11:05,  7.77s/it]
 84%|████████▍ | 189687/225000 [357:54:51<72:47:30,  7.42s/it]
 84%|████████▍ | 189688/225000 [357:54:57<67:57:44,  6.93s/it]
 84%|████████▍ | 189689/225000 [357:55:07<75:56:09,  7.74s/it]
 84%|████████▍ | 189690/225000 [357:55:16<80:45:57,  8.23s/it]
 84%|████████▍ | 189691/225000 [357:55:21<71:20:20,  7.27s/it]
 84%|████████▍ | 189692/225000 [357:55:30<75:37:34,  7.71s/it]
 84%|████████▍ | 189693/225000 [357:55:35<68:54:37,  7.03s/it]
 84%|████████▍ | 189694/225000 [357:55:42<67:50:08,  6.92s/it]
 84%|████████▍ | 189695/225000 [357:55:50<72:01:33,  7.34s/it]
 84%|████████▍ | 189696/225000 [357:56:01<81:28:47,  8.31s/it]
 84%|████████▍ | 189697/225000 [357:56:11<87:53:23,  8.96s/it]
 84%|████████▍ | 189698/225000 [357:56:21<89:42:41,  9.15s/it]
 84%|████████▍ | 189699/225000 [357:56:26<76:45:39,  7.83s/it]
 84%|████████▍ | 189700/225000 [357:56:32<72:39:30,  7.41s/it]
                                                              
{'loss': '1.173', 'grad_norm': '1.048', 'learning_rate': '3.269e-06', 'epoch': '0.8431', 'num_input_tokens_seen': 594664120, 'train_runtime': '1.289e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189700/225000 [357:56:32<72:39:30,  7.41s/it]
 84%|████████▍ | 189701/225000 [357:56:39<70:16:20,  7.17s/it]
 84%|████████▍ | 189702/225000 [357:56:48<77:26:24,  7.90s/it]
 84%|████████▍ | 189703/225000 [357:56:56<77:54:47,  7.95s/it]
 84%|████████▍ | 189704/225000 [357:57:00<64:55:52,  6.62s/it]
 84%|████████▍ | 189705/225000 [357:57:09<72:48:37,  7.43s/it]
 84%|████████▍ | 189706/225000 [357:57:19<79:49:50,  8.14s/it]
 84%|████████▍ | 189707/225000 [357:57:27<80:43:31,  8.23s/it]
 84%|████████▍ | 189708/225000 [357:57:34<76:11:45,  7.77s/it]
 84%|████████▍ | 189709/225000 [357:57:42<77:01:04,  7.86s/it]
 84%|████████▍ | 189710/225000 [357:57:50<77:22:34,  7.89s/it]
 84%|████████▍ | 189711/225000 [357:57:59<79:04:37,  8.07s/it]
 84%|████████▍ | 189712/225000 [357:58:05<74:34:10,  7.61s/it]
 84%|████████▍ | 189713/225000 [357:58:13<74:28:04,  7.60s/it]
 84%|████████▍ | 189714/225000 [357:58:20<72:51:56,  7.43s/it]
 84%|████████▍ | 189715/225000 [357:58:29<76:54:48,  7.85s/it]
 84%|████████▍ | 189716/225000 [357:58:37<78:54:24,  8.05s/it]
 84%|████████▍ | 189717/225000 [357:58:46<80:38:19,  8.23s/it]
 84%|████████▍ | 189718/225000 [357:58:52<76:05:10,  7.76s/it]
 84%|████████▍ | 189719/225000 [357:58:58<70:38:00,  7.21s/it]
 84%|████████▍ | 189720/225000 [357:59:05<67:47:06,  6.92s/it]
 84%|████████▍ | 189721/225000 [357:59:11<65:38:35,  6.70s/it]
 84%|████████▍ | 189722/225000 [357:59:16<61:38:46,  6.29s/it]
 84%|████████▍ | 189723/225000 [357:59:22<61:16:09,  6.25s/it]
 84%|████████▍ | 189724/225000 [357:59:27<57:00:45,  5.82s/it]
 84%|████████▍ | 189725/225000 [357:59:39<73:58:08,  7.55s/it]
 84%|████████▍ | 189726/225000 [357:59:47<76:14:06,  7.78s/it]
 84%|████████▍ | 189727/225000 [357:59:54<72:31:37,  7.40s/it]
 84%|████████▍ | 189728/225000 [358:00:01<71:57:47,  7.34s/it]
 84%|████████▍ | 189729/225000 [358:00:06<66:43:15,  6.81s/it]
 84%|████████▍ | 189730/225000 [358:00:18<81:20:36,  8.30s/it]
 84%|████████▍ | 189731/225000 [358:00:22<69:08:54,  7.06s/it]
 84%|████████▍ | 189732/225000 [358:00:28<64:10:54,  6.55s/it]
 84%|████████▍ | 189733/225000 [358:00:40<82:11:41,  8.39s/it]
 84%|████████▍ | 189734/225000 [358:00:48<80:01:03,  8.17s/it]
 84%|████████▍ | 189735/225000 [358:00:55<76:22:46,  7.80s/it]
 84%|████████▍ | 189736/225000 [358:01:00<69:26:44,  7.09s/it]
 84%|████████▍ | 189737/225000 [358:01:11<79:26:54,  8.11s/it]
 84%|████████▍ | 189738/225000 [358:01:15<67:23:52,  6.88s/it]
 84%|████████▍ | 189739/225000 [358:01:25<78:04:34,  7.97s/it]
 84%|████████▍ | 189740/225000 [358:01:33<77:42:19,  7.93s/it]
 84%|████████▍ | 189741/225000 [358:01:43<82:55:21,  8.47s/it]
 84%|████████▍ | 189742/225000 [358:01:49<76:36:23,  7.82s/it]
 84%|████████▍ | 189743/225000 [358:01:54<68:14:46,  6.97s/it]
 84%|████████▍ | 189744/225000 [358:02:04<75:20:39,  7.69s/it]
 84%|████████▍ | 189745/225000 [358:02:14<84:06:01,  8.59s/it]
 84%|████████▍ | 189746/225000 [358:02:18<71:19:37,  7.28s/it]
 84%|████████▍ | 189747/225000 [358:02:23<63:20:14,  6.47s/it]
 84%|████████▍ | 189748/225000 [358:02:33<74:12:45,  7.58s/it]
 84%|████████▍ | 189749/225000 [358:02:42<78:03:47,  7.97s/it]
 84%|████████▍ | 189750/225000 [358:02:47<69:13:21,  7.07s/it]
                                                              
{'loss': '1.154', 'grad_norm': '1.394', 'learning_rate': '3.26e-06', 'epoch': '0.8433', 'num_input_tokens_seen': 594857864, 'train_runtime': '1.289e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189750/225000 [358:02:47<69:13:21,  7.07s/it]
 84%|████████▍ | 189751/225000 [358:02:53<66:04:43,  6.75s/it]
 84%|████████▍ | 189752/225000 [358:02:58<61:28:30,  6.28s/it]
 84%|████████▍ | 189753/225000 [358:03:15<91:21:28,  9.33s/it]
 84%|████████▍ | 189754/225000 [358:03:23<88:33:48,  9.05s/it]
 84%|████████▍ | 189755/225000 [358:03:29<79:10:52,  8.09s/it]
 84%|████████▍ | 189756/225000 [358:03:37<77:42:02,  7.94s/it]
 84%|████████▍ | 189757/225000 [358:03:45<80:30:31,  8.22s/it]
 84%|████████▍ | 189758/225000 [358:03:56<88:12:53,  9.01s/it]
 84%|████████▍ | 189759/225000 [358:04:03<82:45:42,  8.45s/it]
 84%|████████▍ | 189760/225000 [358:04:08<72:06:42,  7.37s/it]
 84%|████████▍ | 189761/225000 [358:04:19<83:08:38,  8.49s/it]
 84%|████████▍ | 189762/225000 [358:04:26<76:53:00,  7.85s/it]
 84%|████████▍ | 189763/225000 [358:04:33<74:24:22,  7.60s/it]
 84%|████████▍ | 189764/225000 [358:04:38<68:42:59,  7.02s/it]
 84%|████████▍ | 189765/225000 [358:04:52<89:01:53,  9.10s/it]
 84%|████████▍ | 189766/225000 [358:04:58<78:23:00,  8.01s/it]
 84%|████████▍ | 189767/225000 [358:05:02<67:37:36,  6.91s/it]
 84%|████████▍ | 189768/225000 [358:05:08<63:32:54,  6.49s/it]
 84%|████████▍ | 189769/225000 [358:05:20<80:34:45,  8.23s/it]
 84%|████████▍ | 189770/225000 [358:05:24<67:41:13,  6.92s/it]
 84%|████████▍ | 189771/225000 [358:05:33<73:16:51,  7.49s/it]
 84%|████████▍ | 189772/225000 [358:05:38<66:06:22,  6.76s/it]
 84%|████████▍ | 189773/225000 [358:05:49<80:48:20,  8.26s/it]
 84%|████████▍ | 189774/225000 [358:06:00<86:25:57,  8.83s/it]
 84%|████████▍ | 189775/225000 [358:06:04<73:33:36,  7.52s/it]
 84%|████████▍ | 189776/225000 [358:06:12<75:01:34,  7.67s/it]
 84%|████████▍ | 189777/225000 [358:06:19<72:55:45,  7.45s/it]
 84%|████████▍ | 189778/225000 [358:06:24<65:50:04,  6.73s/it]
 84%|████████▍ | 189779/225000 [358:06:30<62:53:28,  6.43s/it]
 84%|████████▍ | 189780/225000 [358:06:37<66:36:42,  6.81s/it]
 84%|████████▍ | 189781/225000 [358:06:44<65:06:45,  6.66s/it]
 84%|████████▍ | 189782/225000 [358:06:49<61:32:14,  6.29s/it]
 84%|████████▍ | 189783/225000 [358:06:56<61:32:43,  6.29s/it]
 84%|████████▍ | 189784/225000 [358:07:05<70:35:06,  7.22s/it]
 84%|████████▍ | 189785/225000 [358:07:20<94:27:41,  9.66s/it]
 84%|████████▍ | 189786/225000 [358:07:31<99:01:54, 10.12s/it]
 84%|████████▍ | 189787/225000 [358:07:36<82:21:38,  8.42s/it]
 84%|████████▍ | 189788/225000 [358:07:45<85:17:27,  8.72s/it]
 84%|████████▍ | 189789/225000 [358:07:52<78:30:41,  8.03s/it]
 84%|████████▍ | 189790/225000 [358:07:59<77:29:32,  7.92s/it]
 84%|████████▍ | 189791/225000 [358:08:03<65:15:51,  6.67s/it]
 84%|████████▍ | 189792/225000 [358:08:12<71:04:15,  7.27s/it]
 84%|████████▍ | 189793/225000 [358:08:17<66:14:03,  6.77s/it]
 84%|████████▍ | 189794/225000 [358:08:29<80:00:43,  8.18s/it]
 84%|████████▍ | 189795/225000 [358:08:35<73:21:08,  7.50s/it]
 84%|████████▍ | 189796/225000 [358:08:42<72:56:17,  7.46s/it]
 84%|████████▍ | 189797/225000 [358:08:49<70:18:19,  7.19s/it]
 84%|████████▍ | 189798/225000 [358:09:02<88:13:58,  9.02s/it]
 84%|████████▍ | 189799/225000 [358:09:07<75:30:52,  7.72s/it]
 84%|████████▍ | 189800/225000 [358:09:17<83:12:02,  8.51s/it]
                                                              
{'loss': '1.16', 'grad_norm': '1.123', 'learning_rate': '3.251e-06', 'epoch': '0.8436', 'num_input_tokens_seen': 595054936, 'train_runtime': '1.289e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189800/225000 [358:09:17<83:12:02,  8.51s/it]
 84%|████████▍ | 189801/225000 [358:09:22<73:32:25,  7.52s/it]
 84%|████████▍ | 189802/225000 [358:09:27<65:50:35,  6.73s/it]
 84%|████████▍ | 189803/225000 [358:09:31<57:58:32,  5.93s/it]
 84%|████████▍ | 189804/225000 [358:09:44<77:09:37,  7.89s/it]
 84%|████████▍ | 189805/225000 [358:09:48<67:22:13,  6.89s/it]
 84%|████████▍ | 189806/225000 [358:09:56<70:44:39,  7.24s/it]
 84%|████████▍ | 189807/225000 [358:10:02<65:43:50,  6.72s/it]
 84%|████████▍ | 189808/225000 [358:10:06<57:46:24,  5.91s/it]
 84%|████████▍ | 189809/225000 [358:10:20<82:36:46,  8.45s/it]
 84%|████████▍ | 189810/225000 [358:10:29<83:43:30,  8.57s/it]
 84%|████████▍ | 189811/225000 [358:10:33<69:42:15,  7.13s/it]
 84%|████████▍ | 189812/225000 [358:10:38<62:37:27,  6.41s/it]
 84%|████████▍ | 189813/225000 [358:10:52<85:37:08,  8.76s/it]
 84%|████████▍ | 189814/225000 [358:10:58<77:01:46,  7.88s/it]
 84%|████████▍ | 189815/225000 [358:11:04<73:37:53,  7.53s/it]
 84%|████████▍ | 189816/225000 [358:11:10<67:30:20,  6.91s/it]
 84%|████████▍ | 189817/225000 [358:11:21<81:12:29,  8.31s/it]
 84%|████████▍ | 189818/225000 [358:11:29<79:01:41,  8.09s/it]
 84%|████████▍ | 189819/225000 [358:11:40<87:59:07,  9.00s/it]
 84%|████████▍ | 189820/225000 [358:11:47<81:43:30,  8.36s/it]
 84%|████████▍ | 189821/225000 [358:11:53<75:42:35,  7.75s/it]
 84%|████████▍ | 189822/225000 [358:12:03<80:50:43,  8.27s/it]
 84%|████████▍ | 189823/225000 [358:12:10<77:58:22,  7.98s/it]
 84%|████████▍ | 189824/225000 [358:12:15<70:04:13,  7.17s/it]
 84%|████████▍ | 189825/225000 [358:12:22<67:40:28,  6.93s/it]
 84%|████████▍ | 189826/225000 [358:12:29<68:26:01,  7.00s/it]
 84%|████████▍ | 189827/225000 [358:12:39<78:48:55,  8.07s/it]
 84%|████████▍ | 189828/225000 [358:12:43<66:19:30,  6.79s/it]
 84%|████████▍ | 189829/225000 [358:12:53<76:11:23,  7.80s/it]
 84%|████████▍ | 189830/225000 [358:12:59<69:41:06,  7.13s/it]
 84%|████████▍ | 189831/225000 [358:13:04<64:42:56,  6.62s/it]
 84%|████████▍ | 189832/225000 [358:13:19<89:08:31,  9.13s/it]
 84%|████████▍ | 189833/225000 [358:13:27<85:59:02,  8.80s/it]
 84%|████████▍ | 189834/225000 [358:13:33<76:14:33,  7.81s/it]
 84%|████████▍ | 189835/225000 [358:13:42<79:52:18,  8.18s/it]
 84%|████████▍ | 189836/225000 [358:13:49<75:34:04,  7.74s/it]
 84%|████████▍ | 189837/225000 [358:13:57<77:33:31,  7.94s/it]
 84%|████████▍ | 189838/225000 [358:14:05<77:19:46,  7.92s/it]
 84%|████████▍ | 189839/225000 [358:14:11<70:39:05,  7.23s/it]
 84%|████████▍ | 189840/225000 [358:14:15<62:13:09,  6.37s/it]
 84%|████████▍ | 189841/225000 [358:14:21<60:51:35,  6.23s/it]
 84%|████████▍ | 189842/225000 [358:14:37<88:47:25,  9.09s/it]
 84%|████████▍ | 189843/225000 [358:14:45<86:05:29,  8.82s/it]
 84%|████████▍ | 189844/225000 [358:14:53<85:10:16,  8.72s/it]
 84%|████████▍ | 189845/225000 [358:14:59<76:04:43,  7.79s/it]
 84%|████████▍ | 189846/225000 [358:15:06<74:35:04,  7.64s/it]
 84%|████████▍ | 189847/225000 [358:15:13<71:13:53,  7.29s/it]
 84%|████████▍ | 189848/225000 [358:15:22<78:20:10,  8.02s/it]
 84%|████████▍ | 189849/225000 [358:15:28<71:28:37,  7.32s/it]
 84%|████████▍ | 189850/225000 [358:15:38<78:11:00,  8.01s/it]
                                                              
{'loss': '1.163', 'grad_norm': '1.146', 'learning_rate': '3.242e-06', 'epoch': '0.8438', 'num_input_tokens_seen': 595255304, 'train_runtime': '1.29e+06', 'train_tokens_per_second': '461.5'}

 84%|████████▍ | 189850/225000 [358:15:38<78:11:00,  8.01s/it]
 84%|████████▍ | 189851/225000 [358:15:43<69:36:10,  7.13s/it]
 84%|████████▍ | 189852/225000 [358:15:49<65:46:59,  6.74s/it]
 84%|████████▍ | 189853/225000 [358:15:55<65:49:12,  6.74s/it]
 84%|████████▍ | 189854/225000 [358:16:00<59:43:39,  6.12s/it]
 84%|████████▍ | 189855/225000 [358:16:06<57:59:18,  5.94s/it]
 84%|████████▍ | 189856/225000 [358:16:16<70:34:31,  7.23s/it]
 84%|████████▍ | 189857/225000 [358:16:23<71:44:31,  7.35s/it]
 84%|████████▍ | 189858/225000 [358:16:30<69:14:38,  7.09s/it]
 84%|████████▍ | 189859/225000 [358:16:35<62:52:21,  6.44s/it]
 84%|████████▍ | 189860/225000 [358:16:45<74:56:43,  7.68s/it]
 84%|████████▍ | 189861/225000 [358:17:00<95:43:53,  9.81s/it]
 84%|████████▍ | 189862/225000 [358:17:05<81:40:03,  8.37s/it]
 84%|████████▍ | 189863/225000 [358:17:10<70:16:27,  7.20s/it]
 84%|████████▍ | 189864/225000 [358:17:15<66:05:28,  6.77s/it]
 84%|████████▍ | 189865/225000 [358:17:22<66:30:51,  6.82s/it]
 84%|████████▍ | 189866/225000 [358:17:28<62:02:28,  6.36s/it]
 84%|████████▍ | 189867/225000 [358:17:38<73:35:36,  7.54s/it]
 84%|████████▍ | 189868/225000 [358:17:43<67:39:46,  6.93s/it]
 84%|████████▍ | 189869/225000 [358:17:50<67:49:12,  6.95s/it]
 84%|████████▍ | 189870/225000 [358:17:57<65:56:57,  6.76s/it]
 84%|████████▍ | 189871/225000 [358:18:02<60:48:30,  6.23s/it]
 84%|████████▍ | 189872/225000 [358:18:15<82:21:25,  8.44s/it]
 84%|████████▍ | 189873/225000 [358:18:24<82:10:01,  8.42s/it]
 84%|████████▍ | 189874/225000 [358:18:29<72:22:09,  7.42s/it]
 84%|████████▍ | 189875/225000 [358:18:36<71:07:57,  7.29s/it]
 84%|████████▍ | 189876/225000 [358:18:42<69:17:23,  7.10s/it]
 84%|████████▍ | 189877/225000 [358:18:54<83:32:37,  8.56s/it]
 84%|████████▍ | 189878/225000 [358:19:00<75:35:09,  7.75s/it]
 84%|████████▍ | 189879/225000 [358:19:11<84:25:33,  8.65s/it]
 84%|████████▍ | 189880/225000 [358:19:19<82:47:03,  8.49s/it]
 84%|████████▍ | 189881/225000 [358:19:26<77:27:34,  7.94s/it]
 84%|████████▍ | 189882/225000 [358:19:30<65:33:07,  6.72s/it]
 84%|████████▍ | 189883/225000 [358:19:35<61:16:01,  6.28s/it]
 84%|████████▍ | 189884/225000 [358:19:44<68:22:21,  7.01s/it]
 84%|████████▍ | 189885/225000 [358:19:51<69:46:14,  7.15s/it]
 84%|████████▍ | 189886/225000 [358:20:00<75:21:49,  7.73s/it]
 84%|████████▍ | 189887/225000 [358:20:10<82:46:02,  8.49s/it]
 84%|████████▍ | 189888/225000 [358:20:19<82:20:45,  8.44s/it]
 84%|████████▍ | 189889/225000 [358:20:26<79:43:17,  8.17s/it]
 84%|████████▍ | 189890/225000 [358:20:36<83:26:29,  8.56s/it]
 84%|████████▍ | 189891/225000 [358:20:41<75:05:14,  7.70s/it]
 84%|████████▍ | 189892/225000 [358:20:47<68:23:09,  7.01s/it]
 84%|████████▍ | 189893/225000 [358:20:52<61:31:32,  6.31s/it]
 84%|████████▍ | 189894/225000 [358:20:58<60:38:33,  6.22s/it]
 84%|████████▍ | 189895/225000 [358:21:07<71:21:54,  7.32s/it]
 84%|████████▍ | 189896/225000 [358:21:17<77:38:49,  7.96s/it]
 84%|████████▍ | 189897/225000 [358:21:21<67:09:05,  6.89s/it]
 84%|████████▍ | 189898/225000 [358:21:28<66:39:45,  6.84s/it]
 84%|████████▍ | 189899/225000 [358:21:41<85:57:49,  8.82s/it]
 84%|████████▍ | 189900/225000 [358:21:49<82:15:47,  8.44s/it]
                                                              
{'loss': '1.193', 'grad_norm': '1.214', 'learning_rate': '3.233e-06', 'epoch': '0.844', 'num_input_tokens_seen': 595453680, 'train_runtime': '1.29e+06', 'train_tokens_per_second': '461.6'}

 84%|████████▍ | 189900/225000 [358:21:49<82:15:47,  8.44s/it]
 84%|████████▍ | 189901/225000 [358:22:04<101:33:50, 10.42s/it]
 84%|████████▍ | 189902/225000 [358:22:15<104:02:01, 10.67s/it]
 84%|████████▍ | 189903/225000 [358:22:26<105:03:00, 10.78s/it]
 84%|████████▍ | 189904/225000 [358:22:31<88:12:13,  9.05s/it] 
 84%|████████▍ | 189905/225000 [358:22:42<92:22:23,  9.48s/it]
 84%|████████▍ | 189906/225000 [358:22:52<94:37:59,  9.71s/it]
 84%|████████▍ | 189907/225000 [358:22:57<82:00:42,  8.41s/it]
 84%|████████▍ | 189908/225000 [358:23:04<77:05:47,  7.91s/it]
 84%|████████▍ | 189909/225000 [358:23:12<76:41:20,  7.87s/it]
 84%|████████▍ | 189910/225000 [358:23:18<70:02:35,  7.19s/it]
 84%|████████▍ | 189911/225000 [358:23:30<86:41:57,  8.90s/it]
 84%|████████▍ | 189912/225000 [358:23:34<71:22:39,  7.32s/it]
 84%|████████▍ | 189913/225000 [358:23:41<71:04:10,  7.29s/it]
 84%|████████▍ | 189914/225000 [358:23:49<72:47:25,  7.47s/it]
 84%|████████▍ | 189915/225000 [358:23:57<72:37:20,  7.45s/it]
 84%|████████▍ | 189916/225000 [358:24:01<63:55:45,  6.56s/it]
 84%|████████▍ | 189917/225000 [358:24:06<59:19:00,  6.09s/it]
 84%|████████▍ | 189918/225000 [358:24:11<56:19:25,  5.78s/it]
 84%|████████▍ | 189919/225000 [358:24:22<71:38:05,  7.35s/it]
 84%|████████▍ | 189920/225000 [358:24:31<77:26:57,  7.95s/it]
 84%|████████▍ | 189921/225000 [358:24:37<71:39:07,  7.35s/it]
 84%|████████▍ | 189922/225000 [358:24:41<59:48:32,  6.14s/it]
 84%|████████▍ | 189923/225000 [358:24:48<63:16:20,  6.49s/it]
 84%|████████▍ | 189924/225000 [358:24:53<59:15:34,  6.08s/it]
 84%|████████▍ | 189925/225000 [358:24:58<56:50:07,  5.83s/it]
 84%|████████▍ | 189926/225000 [358:25:12<80:03:52,  8.22s/it]
 84%|████████▍ | 189927/225000 [358:25:17<71:12:52,  7.31s/it]
 84%|████████▍ | 189928/225000 [358:25:24<68:21:04,  7.02s/it]
 84%|████████▍ | 189929/225000 [358:25:32<71:17:50,  7.32s/it]
 84%|████████▍ | 189930/225000 [358:25:37<64:01:06,  6.57s/it]
 84%|████████▍ | 189931/225000 [358:25:44<67:32:02,  6.93s/it]
 84%|████████▍ | 189932/225000 [358:25:55<77:55:30,  8.00s/it]
 84%|████████▍ | 189933/225000 [358:26:04<80:08:11,  8.23s/it]
 84%|████████▍ | 189934/225000 [358:26:12<79:32:43,  8.17s/it]
 84%|████████▍ | 189935/225000 [358:26:17<70:24:39,  7.23s/it]
 84%|████████▍ | 189936/225000 [358:26:27<78:49:41,  8.09s/it]
 84%|████████▍ | 189937/225000 [358:26:33<72:55:36,  7.49s/it]
 84%|████████▍ | 189938/225000 [358:26:44<82:56:59,  8.52s/it]
 84%|████████▍ | 189939/225000 [358:26:49<73:33:39,  7.55s/it]
 84%|████████▍ | 189940/225000 [358:26:55<68:18:34,  7.01s/it]
 84%|████████▍ | 189941/225000 [358:27:10<91:25:17,  9.39s/it]
 84%|████████▍ | 189942/225000 [358:27:17<86:17:16,  8.86s/it]
 84%|████████▍ | 189943/225000 [358:27:23<77:50:10,  7.99s/it]
 84%|████████▍ | 189944/225000 [358:27:30<72:23:57,  7.43s/it]
 84%|████████▍ | 189945/225000 [358:27:40<81:22:25,  8.36s/it]
 84%|████████▍ | 189946/225000 [358:27:47<77:47:33,  7.99s/it]
 84%|████████▍ | 189947/225000 [358:27:51<67:01:24,  6.88s/it]
 84%|████████▍ | 189948/225000 [358:27:59<69:23:29,  7.13s/it]
 84%|████████▍ | 189949/225000 [358:28:06<67:12:15,  6.90s/it]
 84%|████████▍ | 189950/225000 [358:28:19<85:32:08,  8.79s/it]
                                                              
{'loss': '1.141', 'grad_norm': '0.9793', 'learning_rate': '3.224e-06', 'epoch': '0.8442', 'num_input_tokens_seen': 595656976, 'train_runtime': '1.29e+06', 'train_tokens_per_second': '461.6'}

 84%|████████▍ | 189950/225000 [358:28:19<85:32:08,  8.79s/it]
 84%|████████▍ | 189951/225000 [358:28:27<83:21:55,  8.56s/it]
 84%|████████▍ | 189952/225000 [358:28:35<82:21:40,  8.46s/it]
 84%|████████▍ | 189953/225000 [358:28:42<77:02:57,  7.91s/it]
 84%|████████▍ | 189954/225000 [358:28:48<71:31:50,  7.35s/it]
 84%|████████▍ | 189955/225000 [358:28:54<68:55:50,  7.08s/it]
 84%|████████▍ | 189956/225000 [358:29:00<65:16:20,  6.71s/it]
 84%|████████▍ | 189957/225000 [358:29:06<63:31:51,  6.53s/it]
 84%|████████▍ | 189958/225000 [358:29:18<79:46:32,  8.20s/it]
 84%|████████▍ | 189959/225000 [358:29:22<68:01:39,  6.99s/it]
 84%|████████▍ | 189960/225000 [358:29:29<66:47:27,  6.86s/it]
 84%|████████▍ | 189961/225000 [358:29:38<73:15:28,  7.53s/it]
 84%|████████▍ | 189962/225000 [358:29:45<71:30:21,  7.35s/it]
 84%|████████▍ | 189963/225000 [358:29:53<73:20:43,  7.54s/it]
 84%|████████▍ | 189964/225000 [358:30:09<97:35:30, 10.03s/it]
 84%|████████▍ | 189965/225000 [358:30:20<100:55:34, 10.37s/it]
 84%|████████▍ | 189966/225000 [358:30:25<85:10:20,  8.75s/it] 
 84%|████████▍ | 189967/225000 [358:30:28<69:26:56,  7.14s/it]
 84%|████████▍ | 189968/225000 [358:30:39<81:24:18,  8.37s/it]
 84%|████████▍ | 189969/225000 [358:30:46<75:33:45,  7.77s/it]
 84%|████████▍ | 189970/225000 [358:30:56<81:10:27,  8.34s/it]
 84%|████████▍ | 189971/225000 [358:31:06<88:08:20,  9.06s/it]
 84%|████████▍ | 189972/225000 [358:31:13<82:50:09,  8.51s/it]
 84%|████████▍ | 189973/225000 [358:31:24<88:48:57,  9.13s/it]
 84%|████████▍ | 189974/225000 [358:31:33<88:04:43,  9.05s/it]
 84%|████████▍ | 189975/225000 [358:31:42<86:59:06,  8.94s/it]
 84%|████████▍ | 189976/225000 [358:31:47<76:25:09,  7.85s/it]
 84%|████████▍ | 189977/225000 [358:31:59<88:53:50,  9.14s/it]
 84%|████████▍ | 189978/225000 [358:32:04<75:55:42,  7.80s/it]
 84%|████████▍ | 189979/225000 [358:32:08<66:46:24,  6.86s/it]
 84%|████████▍ | 189980/225000 [358:32:17<71:54:11,  7.39s/it]
 84%|████████▍ | 189981/225000 [358:32:24<69:30:52,  7.15s/it]
 84%|████████▍ | 189982/225000 [358:32:34<78:20:55,  8.05s/it]
 84%|████████▍ | 189983/225000 [358:32:40<73:02:48,  7.51s/it]
 84%|████████▍ | 189984/225000 [358:32:49<76:44:20,  7.89s/it]
 84%|████████▍ | 189985/225000 [358:32:55<72:15:38,  7.43s/it]
 84%|████████▍ | 189986/225000 [358:33:01<66:52:43,  6.88s/it]
 84%|████████▍ | 189987/225000 [358:33:06<61:45:12,  6.35s/it]
 84%|████████▍ | 189988/225000 [358:33:20<83:15:20,  8.56s/it]
 84%|████████▍ | 189989/225000 [358:33:27<79:05:42,  8.13s/it]
 84%|████████▍ | 189990/225000 [358:33:36<81:04:11,  8.34s/it]
 84%|████████▍ | 189991/225000 [358:33:45<83:45:03,  8.61s/it]
 84%|████████▍ | 189992/225000 [358:33:55<87:56:46,  9.04s/it]
 84%|████████▍ | 189993/225000 [358:34:06<95:18:06,  9.80s/it]
 84%|████████▍ | 189994/225000 [358:34:11<81:29:02,  8.38s/it]
 84%|████████▍ | 189995/225000 [358:34:22<88:44:09,  9.13s/it]
 84%|████████▍ | 189996/225000 [358:34:27<74:21:47,  7.65s/it]
 84%|████████▍ | 189997/225000 [358:34:33<69:32:40,  7.15s/it]
 84%|████████▍ | 189998/225000 [358:34:40<70:34:01,  7.26s/it]
 84%|████████▍ | 189999/225000 [358:34:50<78:56:27,  8.12s/it]
 84%|████████▍ | 190000/225000 [358:34:56<73:14:14,  7.53s/it]
                                                              
{'loss': '1.167', 'grad_norm': '1.332', 'learning_rate': '3.215e-06', 'epoch': '0.8444', 'num_input_tokens_seen': 595869504, 'train_runtime': '1.291e+06', 'train_tokens_per_second': '461.6'}

 84%|████████▍ | 190000/225000 [358:34:56<73:14:14,  7.53s/it]
 84%|████████▍ | 190001/225000 [358:35:07<81:58:29,  8.43s/it]
 84%|████████▍ | 190002/225000 [358:35:13<74:30:32,  7.66s/it]
 84%|████████▍ | 190003/225000 [358:35:25<88:14:21,  9.08s/it]
 84%|████████▍ | 190004/225000 [358:35:32<82:42:20,  8.51s/it]
 84%|████████▍ | 190005/225000 [358:35:38<74:55:35,  7.71s/it]
 84%|████████▍ | 190006/225000 [358:35:45<71:29:33,  7.35s/it]
 84%|████████▍ | 190007/225000 [358:35:50<66:52:30,  6.88s/it]
 84%|████████▍ | 190008/225000 [358:35:57<67:03:47,  6.90s/it]
 84%|████████▍ | 190009/225000 [358:36:10<84:03:23,  8.65s/it]
 84%|████████▍ | 190010/225000 [358:36:16<76:39:01,  7.89s/it]
 84%|████████▍ | 190011/225000 [358:36:22<71:05:58,  7.32s/it]
 84%|████████▍ | 190012/225000 [358:36:29<69:13:26,  7.12s/it]
 84%|████████▍ | 190013/225000 [358:36:38<74:46:05,  7.69s/it]
 84%|████████▍ | 190014/225000 [358:36:43<67:40:03,  6.96s/it]
 84%|████████▍ | 190015/225000 [358:36:50<68:08:07,  7.01s/it]
 84%|████████▍ | 190016/225000 [358:36:56<63:27:39,  6.53s/it]
 84%|████████▍ | 190017/225000 [358:37:00<58:26:52,  6.01s/it]
 84%|████████▍ | 190018/225000 [358:37:16<87:05:09,  8.96s/it]
 84%|████████▍ | 190019/225000 [358:37:21<74:21:06,  7.65s/it]
 84%|████████▍ | 190020/225000 [358:37:26<67:25:35,  6.94s/it]
 84%|████████▍ | 190021/225000 [358:37:37<78:58:32,  8.13s/it]
 84%|████████▍ | 190022/225000 [358:37:41<65:46:48,  6.77s/it]
 84%|████████▍ | 190023/225000 [358:37:49<71:38:23,  7.37s/it]
 84%|████████▍ | 190024/225000 [358:37:58<73:46:46,  7.59s/it]
 84%|████████▍ | 190025/225000 [358:38:08<82:34:47,  8.50s/it]
 84%|████████▍ | 190026/225000 [358:38:14<75:02:49,  7.72s/it]
 84%|████████▍ | 190027/225000 [358:38:19<66:24:14,  6.84s/it]
 84%|████████▍ | 190028/225000 [358:38:34<89:19:31,  9.20s/it]
 84%|████████▍ | 190029/225000 [358:38:39<79:01:52,  8.14s/it]
 84%|████████▍ | 190030/225000 [358:38:47<79:22:53,  8.17s/it]
 84%|████████▍ | 190031/225000 [358:38:54<74:47:01,  7.70s/it]
 84%|████████▍ | 190032/225000 [358:38:59<66:14:02,  6.82s/it]
 84%|████████▍ | 190033/225000 [358:39:09<77:03:28,  7.93s/it]
 84%|████████▍ | 190034/225000 [358:39:15<71:02:48,  7.31s/it]
 84%|████████▍ | 190035/225000 [358:39:20<63:01:03,  6.49s/it]
 84%|████████▍ | 190036/225000 [358:39:30<73:11:41,  7.54s/it]
 84%|████████▍ | 190037/225000 [358:39:35<65:39:48,  6.76s/it]
 84%|████████▍ | 190038/225000 [358:39:41<62:42:46,  6.46s/it]
 84%|████████▍ | 190039/225000 [358:39:52<76:21:02,  7.86s/it]
 84%|████████▍ | 190040/225000 [358:40:00<77:56:07,  8.03s/it]
 84%|████████▍ | 190041/225000 [358:40:10<84:54:27,  8.74s/it]
 84%|████████▍ | 190042/225000 [358:40:21<89:04:35,  9.17s/it]
 84%|████████▍ | 190043/225000 [358:40:25<75:18:45,  7.76s/it]
 84%|████████▍ | 190044/225000 [358:40:31<70:34:48,  7.27s/it]
 84%|████████▍ | 190045/225000 [358:40:37<65:10:06,  6.71s/it]
 84%|████████▍ | 190046/225000 [358:40:48<80:08:34,  8.25s/it]
 84%|████████▍ | 190047/225000 [358:40:53<70:34:09,  7.27s/it]
 84%|████████▍ | 190048/225000 [358:40:58<62:56:23,  6.48s/it]
 84%|████████▍ | 190049/225000 [358:41:03<59:08:11,  6.09s/it]
 84%|████████▍ | 190050/225000 [358:41:11<62:25:38,  6.43s/it]
                                                              
{'loss': '1.18', 'grad_norm': '1.298', 'learning_rate': '3.206e-06', 'epoch': '0.8447', 'num_input_tokens_seen': 596065120, 'train_runtime': '1.291e+06', 'train_tokens_per_second': '461.6'}

 84%|████████▍ | 190050/225000 [358:41:11<62:25:38,  6.43s/it]
 84%|████████▍ | 190051/225000 [358:41:23<80:05:01,  8.25s/it]
 84%|████████▍ | 190052/225000 [358:41:29<74:37:00,  7.69s/it]
 84%|████████▍ | 190053/225000 [358:41:35<68:09:32,  7.02s/it]
 84%|████████▍ | 190054/225000 [358:41:43<72:50:00,  7.50s/it]
 84%|████████▍ | 190055/225000 [358:41:55<83:25:26,  8.59s/it]
 84%|████████▍ | 190056/225000 [358:41:59<71:49:37,  7.40s/it]
 84%|████████▍ | 190057/225000 [358:42:07<71:29:04,  7.36s/it]
 84%|████████▍ | 190058/225000 [358:42:13<70:13:13,  7.23s/it]
 84%|████████▍ | 190059/225000 [358:42:21<72:00:33,  7.42s/it]
 84%|████████▍ | 190060/225000 [358:42:28<70:15:39,  7.24s/it]
 84%|████████▍ | 190061/225000 [358:42:40<83:35:28,  8.61s/it]
 84%|████████▍ | 190062/225000 [358:42:45<74:40:07,  7.69s/it]
 84%|████████▍ | 190063/225000 [358:42:55<80:32:21,  8.30s/it]
 84%|████████▍ | 190064/225000 [358:43:02<75:07:10,  7.74s/it]
 84%|████████▍ | 190065/225000 [358:43:06<65:40:14,  6.77s/it]
 84%|████████▍ | 190066/225000 [358:43:13<66:21:29,  6.84s/it]
 84%|████████▍ | 190067/225000 [358:43:26<84:15:19,  8.68s/it]
 84%|████████▍ | 190068/225000 [358:43:34<82:31:21,  8.50s/it]
 84%|████████▍ | 190069/225000 [358:43:42<80:21:39,  8.28s/it]
 84%|████████▍ | 190070/225000 [358:43:50<78:29:19,  8.09s/it]
 84%|████████▍ | 190071/225000 [358:43:58<78:14:09,  8.06s/it]
 84%|████████▍ | 190072/225000 [358:44:08<83:41:43,  8.63s/it]
 84%|████████▍ | 190073/225000 [358:44:14<76:40:34,  7.90s/it]
 84%|████████▍ | 190074/225000 [358:44:20<72:54:23,  7.51s/it]
 84%|████████▍ | 190075/225000 [358:44:30<77:55:43,  8.03s/it]
 84%|████████▍ | 190076/225000 [358:44:35<69:54:58,  7.21s/it]
 84%|████████▍ | 190077/225000 [358:44:42<68:47:28,  7.09s/it]
 84%|████████▍ | 190078/225000 [358:44:54<83:11:32,  8.58s/it]
 84%|████████▍ | 190079/225000 [358:45:06<92:26:25,  9.53s/it]
 84%|████████▍ | 190080/225000 [358:45:10<79:11:29,  8.16s/it]
 84%|████████▍ | 190081/225000 [358:45:15<67:36:23,  6.97s/it]
 84%|████████▍ | 190082/225000 [358:45:21<65:25:35,  6.75s/it]
 84%|████████▍ | 190083/225000 [358:45:32<78:32:53,  8.10s/it]
 84%|████████▍ | 190084/225000 [358:45:37<69:01:26,  7.12s/it]
 84%|████████▍ | 190085/225000 [358:45:42<62:33:05,  6.45s/it]
 84%|████████▍ | 190086/225000 [358:45:50<67:56:43,  7.01s/it]
 84%|████████▍ | 190087/225000 [358:45:55<62:35:15,  6.45s/it]
 84%|████████▍ | 190088/225000 [358:46:12<91:58:10,  9.48s/it]
 84%|████████▍ | 190089/225000 [358:46:19<86:07:30,  8.88s/it]
 84%|████████▍ | 190090/225000 [358:46:34<101:36:23, 10.48s/it]
 84%|████████▍ | 190091/225000 [358:46:45<103:28:23, 10.67s/it]
 84%|████████▍ | 190092/225000 [358:46:50<86:36:06,  8.93s/it] 
 84%|████████▍ | 190093/225000 [358:46:57<83:34:15,  8.62s/it]
 84%|████████▍ | 190094/225000 [358:47:05<80:27:03,  8.30s/it]
 84%|████████▍ | 190095/225000 [358:47:14<83:56:35,  8.66s/it]
 84%|████████▍ | 190096/225000 [358:47:20<75:27:27,  7.78s/it]
 84%|████████▍ | 190097/225000 [358:47:28<73:57:55,  7.63s/it]
 84%|████████▍ | 190098/225000 [358:47:34<70:15:57,  7.25s/it]
 84%|████████▍ | 190099/225000 [358:47:39<63:49:48,  6.58s/it]
 84%|████████▍ | 190100/225000 [358:47:47<66:48:09,  6.89s/it]
                                                              
{'loss': '1.183', 'grad_norm': '1.405', 'learning_rate': '3.197e-06', 'epoch': '0.8449', 'num_input_tokens_seen': 596265880, 'train_runtime': '1.292e+06', 'train_tokens_per_second': '461.6'}

 84%|████████▍ | 190100/225000 [358:47:47<66:48:09,  6.89s/it]
 84%|████████▍ | 190101/225000 [358:47:59<81:49:34,  8.44s/it]
 84%|████████▍ | 190102/225000 [358:48:04<74:01:23,  7.64s/it]
 84%|████████▍ | 190103/225000 [358:48:13<77:58:28,  8.04s/it]
 84%|████████▍ | 190104/225000 [358:48:20<72:57:53,  7.53s/it]
 84%|████████▍ | 190105/225000 [358:48:24<64:47:24,  6.68s/it]
 84%|████████▍ | 190106/225000 [358:48:31<65:00:42,  6.71s/it]
 84%|████████▍ | 190107/225000 [358:48:41<74:48:37,  7.72s/it]
 84%|████████▍ | 190108/225000 [358:48:49<76:02:20,  7.85s/it]
 84%|████████▍ | 190109/225000 [358:48:59<81:39:44,  8.43s/it]
 84%|████████▍ | 190110/225000 [358:49:05<74:13:24,  7.66s/it]
 84%|████████▍ | 190111/225000 [358:49:11<68:10:46,  7.04s/it]
 84%|████████▍ | 190112/225000 [358:49:20<76:09:14,  7.86s/it]
 84%|████████▍ | 190113/225000 [358:49:31<83:58:01,  8.66s/it]
 84%|████████▍ | 190114/225000 [358:49:37<76:27:34,  7.89s/it]
 84%|████████▍ | 190115/225000 [358:49:42<68:00:04,  7.02s/it]
 84%|████████▍ | 190116/225000 [358:49:50<70:33:09,  7.28s/it]
 84%|████████▍ | 190117/225000 [358:50:00<78:01:52,  8.05s/it]
 84%|████████▍ | 190118/225000 [358:50:06<71:49:43,  7.41s/it]
 84%|████████▍ | 190119/225000 [358:50:11<66:55:19,  6.91s/it]
 84%|████████▍ | 190120/225000 [358:50:22<79:10:02,  8.17s/it]
 84%|████████▍ | 190121/225000 [358:50:33<84:42:32,  8.74s/it]
 84%|████████▍ | 190122/225000 [358:50:42<86:51:52,  8.97s/it]
 84%|████████▍ | 190123/225000 [358:50:49<81:03:11,  8.37s/it]
 84%|████████▍ | 190124/225000 [358:50:56<76:21:05,  7.88s/it]
 84%|████████▍ | 190125/225000 [358:51:07<86:36:45,  8.94s/it]
 85%|████████▍ | 190126/225000 [358:51:13<77:08:04,  7.96s/it]
 85%|████████▍ | 190127/225000 [358:51:23<83:14:33,  8.59s/it]
 85%|████████▍ | 190128/225000 [358:51:28<72:43:53,  7.51s/it]
 85%|████████▍ | 190129/225000 [358:51:33<64:45:36,  6.69s/it]
 85%|████████▍ | 190130/225000 [358:51:43<75:32:23,  7.80s/it]
 85%|████████▍ | 190131/225000 [358:51:50<72:05:59,  7.44s/it]
 85%|████████▍ | 190132/225000 [358:51:57<71:09:27,  7.35s/it]
 85%|████████▍ | 190133/225000 [358:52:02<63:52:02,  6.59s/it]
 85%|████████▍ | 190134/225000 [358:52:07<60:16:05,  6.22s/it]
 85%|████████▍ | 190135/225000 [358:52:12<58:09:42,  6.01s/it]
 85%|████████▍ | 190136/225000 [358:52:17<54:57:24,  5.67s/it]
 85%|████████▍ | 190137/225000 [358:52:28<69:02:53,  7.13s/it]
 85%|████████▍ | 190138/225000 [358:52:33<63:57:29,  6.60s/it]
 85%|████████▍ | 190139/225000 [358:52:42<71:05:17,  7.34s/it]
 85%|████████▍ | 190140/225000 [358:52:49<68:18:25,  7.05s/it]
 85%|████████▍ | 190141/225000 [358:52:55<65:00:09,  6.71s/it]
 85%|████████▍ | 190142/225000 [358:53:07<81:07:34,  8.38s/it]
 85%|████████▍ | 190143/225000 [358:53:15<80:24:12,  8.30s/it]
 85%|████████▍ | 190144/225000 [358:53:24<82:17:04,  8.50s/it]
 85%|████████▍ | 190145/225000 [358:53:28<68:31:48,  7.08s/it]
 85%|████████▍ | 190146/225000 [358:53:36<72:08:52,  7.45s/it]
 85%|████████▍ | 190147/225000 [358:53:40<62:51:14,  6.49s/it]
 85%|████████▍ | 190148/225000 [358:53:52<77:56:57,  8.05s/it]
 85%|████████▍ | 190149/225000 [358:53:58<70:53:39,  7.32s/it]
 85%|████████▍ | 190150/225000 [358:54:03<65:13:11,  6.74s/it]
                                                              
{'loss': '1.18', 'grad_norm': '1', 'learning_rate': '3.188e-06', 'epoch': '0.8451', 'num_input_tokens_seen': 596459848, 'train_runtime': '1.292e+06', 'train_tokens_per_second': '461.6'}

 85%|████████▍ | 190150/225000 [358:54:03<65:13:11,  6.74s/it]
 85%|████████▍ | 190151/225000 [358:54:09<61:41:24,  6.37s/it]
 85%|████████▍ | 190152/225000 [358:54:16<64:19:59,  6.65s/it]
 85%|████████▍ | 190153/225000 [358:54:23<65:28:44,  6.76s/it]
 85%|████████▍ | 190154/225000 [358:54:38<89:31:57,  9.25s/it]
 85%|████████▍ | 190155/225000 [358:54:44<81:18:41,  8.40s/it]
 85%|████████▍ | 190156/225000 [358:54:49<70:42:03,  7.30s/it]
 85%|████████▍ | 190157/225000 [358:55:03<90:14:07,  9.32s/it]
 85%|████████▍ | 190158/225000 [358:55:17<102:09:31, 10.56s/it]
 85%|████████▍ | 190159/225000 [358:55:21<85:18:41,  8.81s/it] 
 85%|████████▍ | 190160/225000 [358:55:25<71:13:37,  7.36s/it]
 85%|████████▍ | 190161/225000 [358:55:33<71:01:12,  7.34s/it]
 85%|████████▍ | 190162/225000 [358:55:39<67:02:05,  6.93s/it]
 85%|████████▍ | 190163/225000 [358:55:49<76:18:49,  7.89s/it]
 85%|████████▍ | 190164/225000 [358:55:53<66:12:58,  6.84s/it]
 85%|████████▍ | 190165/225000 [358:56:03<75:47:06,  7.83s/it]
 85%|████████▍ | 190166/225000 [358:56:09<70:19:00,  7.27s/it]
 85%|████████▍ | 190167/225000 [358:56:18<74:21:21,  7.68s/it]
 85%|████████▍ | 190168/225000 [358:56:33<97:35:57, 10.09s/it]
 85%|████████▍ | 190169/225000 [358:56:38<80:49:07,  8.35s/it]
 85%|████████▍ | 190170/225000 [358:56:46<81:44:16,  8.45s/it]
 85%|████████▍ | 190171/225000 [358:56:58<91:19:10,  9.44s/it]
 85%|████████▍ | 190172/225000 [358:57:03<78:09:00,  8.08s/it]
 85%|████████▍ | 190173/225000 [358:57:14<87:33:58,  9.05s/it]
 85%|████████▍ | 190174/225000 [358:57:19<75:16:06,  7.78s/it]
 85%|████████▍ | 190175/225000 [358:57:29<80:32:26,  8.33s/it]
 85%|████████▍ | 190176/225000 [358:57:38<83:26:29,  8.63s/it]
 85%|████████▍ | 190177/225000 [358:57:43<71:44:51,  7.42s/it]
 85%|████████▍ | 190178/225000 [358:57:51<74:48:27,  7.73s/it]
 85%|████████▍ | 190179/225000 [358:57:59<74:02:19,  7.65s/it]
 85%|████████▍ | 190180/225000 [358:58:03<65:18:52,  6.75s/it]
 85%|████████▍ | 190181/225000 [358:58:12<71:18:37,  7.37s/it]
 85%|████████▍ | 190182/225000 [358:58:18<67:02:05,  6.93s/it]
 85%|████████▍ | 190183/225000 [358:58:26<68:58:10,  7.13s/it]
 85%|████████▍ | 190184/225000 [358:58:30<61:06:30,  6.32s/it]
 85%|████████▍ | 190185/225000 [358:58:35<56:21:48,  5.83s/it]
 85%|████████▍ | 190186/225000 [358:58:47<73:48:02,  7.63s/it]
 85%|████████▍ | 190187/225000 [358:58:56<78:10:36,  8.08s/it]
 85%|████████▍ | 190188/225000 [358:59:03<75:17:48,  7.79s/it]
 85%|████████▍ | 190189/225000 [358:59:14<85:36:32,  8.85s/it]
 85%|████████▍ | 190190/225000 [358:59:21<80:49:23,  8.36s/it]
 85%|████████▍ | 190191/225000 [358:59:26<70:01:00,  7.24s/it]
 85%|████████▍ | 190192/225000 [358:59:33<67:51:45,  7.02s/it]
 85%|████████▍ | 190193/225000 [358:59:46<85:06:26,  8.80s/it]
 85%|████████▍ | 190194/225000 [358:59:53<81:58:41,  8.48s/it]
 85%|████████▍ | 190195/225000 [359:00:02<81:22:15,  8.42s/it]
 85%|████████▍ | 190196/225000 [359:00:07<71:39:22,  7.41s/it]
 85%|████████▍ | 190197/225000 [359:00:14<70:45:07,  7.32s/it]
 85%|████████▍ | 190198/225000 [359:00:23<76:58:11,  7.96s/it]
 85%|████████▍ | 190199/225000 [359:00:36<89:45:27,  9.29s/it]
 85%|████████▍ | 190200/225000 [359:00:41<78:39:29,  8.14s/it]
                                                              
{'loss': '1.175', 'grad_norm': '1.277', 'learning_rate': '3.179e-06', 'epoch': '0.8453', 'num_input_tokens_seen': 596663488, 'train_runtime': '1.292e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190200/225000 [359:00:41<78:39:29,  8.14s/it]
 85%|████████▍ | 190201/225000 [359:00:49<78:13:45,  8.09s/it]
 85%|████████▍ | 190202/225000 [359:00:56<75:04:25,  7.77s/it]
 85%|████████▍ | 190203/225000 [359:01:01<68:34:32,  7.09s/it]
 85%|████████▍ | 190204/225000 [359:01:15<87:32:18,  9.06s/it]
 85%|████████▍ | 190205/225000 [359:01:20<76:50:33,  7.95s/it]
 85%|████████▍ | 190206/225000 [359:01:27<73:14:37,  7.58s/it]
 85%|████████▍ | 190207/225000 [359:01:38<83:57:42,  8.69s/it]
 85%|████████▍ | 190208/225000 [359:01:47<83:10:37,  8.61s/it]
 85%|████████▍ | 190209/225000 [359:01:58<89:16:06,  9.24s/it]
 85%|████████▍ | 190210/225000 [359:02:05<83:19:18,  8.62s/it]
 85%|████████▍ | 190211/225000 [359:02:11<76:17:38,  7.89s/it]
 85%|████████▍ | 190212/225000 [359:02:18<73:25:47,  7.60s/it]
 85%|████████▍ | 190213/225000 [359:02:24<68:24:47,  7.08s/it]
 85%|████████▍ | 190214/225000 [359:02:28<60:39:17,  6.28s/it]
 85%|████████▍ | 190215/225000 [359:02:39<73:29:05,  7.61s/it]
 85%|████████▍ | 190216/225000 [359:02:44<65:12:05,  6.75s/it]
 85%|████████▍ | 190217/225000 [359:02:53<72:49:04,  7.54s/it]
 85%|████████▍ | 190218/225000 [359:02:58<66:41:57,  6.90s/it]
 85%|████████▍ | 190219/225000 [359:03:03<61:03:05,  6.32s/it]
 85%|████████▍ | 190220/225000 [359:03:09<59:04:41,  6.12s/it]
 85%|████████▍ | 190221/225000 [359:03:22<78:24:14,  8.12s/it]
 85%|████████▍ | 190222/225000 [359:03:28<72:27:52,  7.50s/it]
 85%|████████▍ | 190223/225000 [359:03:34<68:30:25,  7.09s/it]
 85%|████████▍ | 190224/225000 [359:03:41<67:43:24,  7.01s/it]
 85%|████████▍ | 190225/225000 [359:03:46<62:30:02,  6.47s/it]
 85%|████████▍ | 190226/225000 [359:03:52<61:17:16,  6.34s/it]
 85%|████████▍ | 190227/225000 [359:04:06<82:23:23,  8.53s/it]
 85%|████████▍ | 190228/225000 [359:04:13<77:46:58,  8.05s/it]
 85%|████████▍ | 190229/225000 [359:04:22<81:04:30,  8.39s/it]
 85%|████████▍ | 190230/225000 [359:04:27<73:00:14,  7.56s/it]
 85%|████████▍ | 190231/225000 [359:04:34<69:55:02,  7.24s/it]
 85%|████████▍ | 190232/225000 [359:04:45<81:26:26,  8.43s/it]
 85%|████████▍ | 190233/225000 [359:04:55<84:53:01,  8.79s/it]
 85%|████████▍ | 190234/225000 [359:05:02<79:22:32,  8.22s/it]
 85%|████████▍ | 190235/225000 [359:05:10<80:30:14,  8.34s/it]
 85%|████████▍ | 190236/225000 [359:05:20<83:09:53,  8.61s/it]
 85%|████████▍ | 190237/225000 [359:05:31<91:39:20,  9.49s/it]
 85%|████████▍ | 190238/225000 [359:05:37<82:23:15,  8.53s/it]
 85%|████████▍ | 190239/225000 [359:05:43<74:12:11,  7.68s/it]
 85%|████████▍ | 190240/225000 [359:05:48<67:16:19,  6.97s/it]
 85%|████████▍ | 190241/225000 [359:05:55<66:53:52,  6.93s/it]
 85%|████████▍ | 190242/225000 [359:06:03<68:54:14,  7.14s/it]
 85%|████████▍ | 190243/225000 [359:06:11<72:04:45,  7.47s/it]
 85%|████████▍ | 190244/225000 [359:06:22<82:35:16,  8.55s/it]
 85%|████████▍ | 190245/225000 [359:06:29<78:59:10,  8.18s/it]
 85%|████████▍ | 190246/225000 [359:06:38<79:08:48,  8.20s/it]
 85%|████████▍ | 190247/225000 [359:06:43<69:38:45,  7.21s/it]
 85%|████████▍ | 190248/225000 [359:06:50<68:39:20,  7.11s/it]
 85%|████████▍ | 190249/225000 [359:07:03<88:13:46,  9.14s/it]
 85%|████████▍ | 190250/225000 [359:07:10<81:54:01,  8.48s/it]
                                                              
{'loss': '1.187', 'grad_norm': '1.063', 'learning_rate': '3.17e-06', 'epoch': '0.8456', 'num_input_tokens_seen': 596860464, 'train_runtime': '1.293e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190250/225000 [359:07:10<81:54:01,  8.48s/it]
 85%|████████▍ | 190251/225000 [359:07:21<87:42:54,  9.09s/it]
 85%|████████▍ | 190252/225000 [359:07:27<80:23:33,  8.33s/it]
 85%|████████▍ | 190253/225000 [359:07:37<83:52:52,  8.69s/it]
 85%|████████▍ | 190254/225000 [359:07:46<86:18:54,  8.94s/it]
 85%|████████▍ | 190255/225000 [359:07:52<75:25:08,  7.81s/it]
 85%|████████▍ | 190256/225000 [359:08:03<85:15:58,  8.83s/it]
 85%|████████▍ | 190257/225000 [359:08:07<70:44:45,  7.33s/it]
 85%|████████▍ | 190258/225000 [359:08:18<83:33:04,  8.66s/it]
 85%|████████▍ | 190259/225000 [359:08:24<75:34:24,  7.83s/it]
 85%|████████▍ | 190260/225000 [359:08:32<74:00:12,  7.67s/it]
 85%|████████▍ | 190261/225000 [359:08:42<81:00:00,  8.39s/it]
 85%|████████▍ | 190262/225000 [359:08:47<70:38:32,  7.32s/it]
 85%|████████▍ | 190263/225000 [359:08:54<71:30:19,  7.41s/it]
 85%|████████▍ | 190264/225000 [359:08:59<63:24:25,  6.57s/it]
 85%|████████▍ | 190265/225000 [359:09:10<76:58:37,  7.98s/it]
 85%|████████▍ | 190266/225000 [359:09:17<75:19:47,  7.81s/it]
 85%|████████▍ | 190267/225000 [359:09:26<77:28:32,  8.03s/it]
 85%|████████▍ | 190268/225000 [359:09:37<84:41:07,  8.78s/it]
 85%|████████▍ | 190269/225000 [359:09:41<72:58:23,  7.56s/it]
 85%|████████▍ | 190270/225000 [359:09:49<74:49:17,  7.76s/it]
 85%|████████▍ | 190271/225000 [359:09:54<66:18:37,  6.87s/it]
 85%|████████▍ | 190272/225000 [359:10:03<72:26:34,  7.51s/it]
 85%|████████▍ | 190273/225000 [359:10:13<79:24:38,  8.23s/it]
 85%|████████▍ | 190274/225000 [359:10:20<76:15:38,  7.91s/it]
 85%|████████▍ | 190275/225000 [359:10:27<72:57:32,  7.56s/it]
 85%|████████▍ | 190276/225000 [359:10:34<70:08:50,  7.27s/it]
 85%|████████▍ | 190277/225000 [359:10:43<76:29:57,  7.93s/it]
 85%|████████▍ | 190278/225000 [359:10:48<68:09:47,  7.07s/it]
 85%|████████▍ | 190279/225000 [359:11:01<85:10:59,  8.83s/it]
 85%|████████▍ | 190280/225000 [359:11:06<73:37:33,  7.63s/it]
 85%|████████▍ | 190281/225000 [359:11:13<73:08:47,  7.58s/it]
 85%|████████▍ | 190282/225000 [359:11:25<84:22:06,  8.75s/it]
 85%|████████▍ | 190283/225000 [359:11:32<78:36:15,  8.15s/it]
 85%|████████▍ | 190284/225000 [359:11:42<84:16:20,  8.74s/it]
 85%|████████▍ | 190285/225000 [359:11:49<79:59:15,  8.29s/it]
 85%|████████▍ | 190286/225000 [359:11:56<75:57:44,  7.88s/it]
 85%|████████▍ | 190287/225000 [359:12:00<64:45:40,  6.72s/it]
 85%|████████▍ | 190288/225000 [359:12:06<63:45:25,  6.61s/it]
 85%|████████▍ | 190289/225000 [359:12:17<76:24:34,  7.92s/it]
 85%|████████▍ | 190290/225000 [359:12:21<65:21:37,  6.78s/it]
 85%|████████▍ | 190291/225000 [359:12:30<70:24:54,  7.30s/it]
 85%|████████▍ | 190292/225000 [359:12:41<81:26:47,  8.45s/it]
 85%|████████▍ | 190293/225000 [359:12:49<81:04:19,  8.41s/it]
 85%|████████▍ | 190294/225000 [359:12:54<69:38:06,  7.22s/it]
 85%|████████▍ | 190295/225000 [359:13:00<66:37:54,  6.91s/it]
 85%|████████▍ | 190296/225000 [359:13:06<65:10:10,  6.76s/it]
 85%|████████▍ | 190297/225000 [359:13:16<72:07:06,  7.48s/it]
 85%|████████▍ | 190298/225000 [359:13:28<85:58:10,  8.92s/it]
 85%|████████▍ | 190299/225000 [359:13:37<87:59:02,  9.13s/it]
 85%|████████▍ | 190300/225000 [359:13:44<79:07:22,  8.21s/it]
                                                              
{'loss': '1.183', 'grad_norm': '0.9653', 'learning_rate': '3.162e-06', 'epoch': '0.8458', 'num_input_tokens_seen': 597064600, 'train_runtime': '1.293e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190300/225000 [359:13:44<79:07:22,  8.21s/it]
 85%|████████▍ | 190301/225000 [359:13:56<92:43:12,  9.62s/it]
 85%|████████▍ | 190302/225000 [359:14:03<82:29:24,  8.56s/it]
 85%|████████▍ | 190303/225000 [359:14:16<96:54:52, 10.06s/it]
 85%|████████▍ | 190304/225000 [359:14:22<85:33:12,  8.88s/it]
 85%|████████▍ | 190305/225000 [359:14:28<75:36:29,  7.85s/it]
 85%|████████▍ | 190306/225000 [359:14:35<72:46:21,  7.55s/it]
 85%|████████▍ | 190307/225000 [359:14:48<88:56:26,  9.23s/it]
 85%|████████▍ | 190308/225000 [359:14:52<73:32:35,  7.63s/it]
 85%|████████▍ | 190309/225000 [359:14:57<67:52:00,  7.04s/it]
 85%|████████▍ | 190310/225000 [359:15:06<73:45:13,  7.65s/it]
 85%|████████▍ | 190311/225000 [359:15:13<72:05:44,  7.48s/it]
 85%|████████▍ | 190312/225000 [359:15:20<70:25:56,  7.31s/it]
 85%|████████▍ | 190313/225000 [359:15:26<66:49:02,  6.93s/it]
 85%|████████▍ | 190314/225000 [359:15:32<63:09:49,  6.56s/it]
 85%|████████▍ | 190315/225000 [359:15:37<57:22:34,  5.96s/it]
 85%|████████▍ | 190316/225000 [359:15:53<86:19:59,  8.96s/it]
 85%|████████▍ | 190317/225000 [359:15:58<77:11:02,  8.01s/it]
 85%|████████▍ | 190318/225000 [359:16:04<71:07:41,  7.38s/it]
 85%|████████▍ | 190319/225000 [359:16:09<64:25:11,  6.69s/it]
 85%|████████▍ | 190320/225000 [359:16:25<91:08:19,  9.46s/it]
 85%|████████▍ | 190321/225000 [359:16:30<77:54:25,  8.09s/it]
 85%|████████▍ | 190322/225000 [359:16:44<93:43:31,  9.73s/it]
 85%|████████▍ | 190323/225000 [359:16:49<80:54:55,  8.40s/it]
 85%|████████▍ | 190324/225000 [359:16:58<81:34:34,  8.47s/it]
 85%|████████▍ | 190325/225000 [359:17:04<76:52:39,  7.98s/it]
 85%|████████▍ | 190326/225000 [359:17:14<80:35:29,  8.37s/it]
 85%|████████▍ | 190327/225000 [359:17:26<92:28:32,  9.60s/it]
 85%|████████▍ | 190328/225000 [359:17:31<78:43:05,  8.17s/it]
 85%|████████▍ | 190329/225000 [359:17:39<78:58:01,  8.20s/it]
 85%|████████▍ | 190330/225000 [359:17:48<81:45:06,  8.49s/it]
 85%|████████▍ | 190331/225000 [359:18:00<89:54:46,  9.34s/it]
 85%|████████▍ | 190332/225000 [359:18:07<84:42:53,  8.80s/it]
 85%|████████▍ | 190333/225000 [359:18:18<91:11:41,  9.47s/it]
 85%|████████▍ | 190334/225000 [359:18:25<83:12:53,  8.64s/it]
 85%|████████▍ | 190335/225000 [359:18:34<83:24:05,  8.66s/it]
 85%|████████▍ | 190336/225000 [359:18:41<79:08:53,  8.22s/it]
 85%|████████▍ | 190337/225000 [359:18:48<74:16:11,  7.71s/it]
 85%|████████▍ | 190338/225000 [359:18:55<72:37:25,  7.54s/it]
 85%|████████▍ | 190339/225000 [359:19:08<90:38:25,  9.41s/it]
 85%|████████▍ | 190340/225000 [359:19:16<84:47:59,  8.81s/it]
 85%|████████▍ | 190341/225000 [359:19:23<79:09:06,  8.22s/it]
 85%|████████▍ | 190342/225000 [359:19:30<77:07:00,  8.01s/it]
 85%|████████▍ | 190343/225000 [359:19:39<78:02:43,  8.11s/it]
 85%|████████▍ | 190344/225000 [359:19:47<78:31:44,  8.16s/it]
 85%|████████▍ | 190345/225000 [359:19:57<84:16:17,  8.75s/it]
 85%|████████▍ | 190346/225000 [359:20:02<74:21:13,  7.72s/it]
 85%|████████▍ | 190347/225000 [359:20:09<72:30:14,  7.53s/it]
 85%|████████▍ | 190348/225000 [359:20:14<63:42:10,  6.62s/it]
 85%|████████▍ | 190349/225000 [359:20:23<72:01:05,  7.48s/it]
 85%|████████▍ | 190350/225000 [359:20:30<69:03:54,  7.18s/it]
                                                              
{'loss': '1.208', 'grad_norm': '1.229', 'learning_rate': '3.153e-06', 'epoch': '0.846', 'num_input_tokens_seen': 597273000, 'train_runtime': '1.294e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190350/225000 [359:20:30<69:03:54,  7.18s/it]
 85%|████████▍ | 190351/225000 [359:20:41<81:54:08,  8.51s/it]
 85%|████████▍ | 190352/225000 [359:20:47<72:30:59,  7.53s/it]
 85%|████████▍ | 190353/225000 [359:20:57<81:22:34,  8.46s/it]
 85%|████████▍ | 190354/225000 [359:21:10<92:55:09,  9.66s/it]
 85%|████████▍ | 190355/225000 [359:21:13<75:23:10,  7.83s/it]
 85%|████████▍ | 190356/225000 [359:21:20<71:12:34,  7.40s/it]
 85%|████████▍ | 190357/225000 [359:21:28<74:44:31,  7.77s/it]
 85%|████████▍ | 190358/225000 [359:21:36<75:02:50,  7.80s/it]
 85%|████████▍ | 190359/225000 [359:21:49<88:00:45,  9.15s/it]
 85%|████████▍ | 190360/225000 [359:22:01<97:34:04, 10.14s/it]
 85%|████████▍ | 190361/225000 [359:22:05<80:39:39,  8.38s/it]
 85%|████████▍ | 190362/225000 [359:22:12<74:55:32,  7.79s/it]
 85%|████████▍ | 190363/225000 [359:22:17<69:06:20,  7.18s/it]
 85%|████████▍ | 190364/225000 [359:22:28<78:51:55,  8.20s/it]
 85%|████████▍ | 190365/225000 [359:22:35<74:39:44,  7.76s/it]
 85%|████████▍ | 190366/225000 [359:22:42<73:46:39,  7.67s/it]
 85%|████████▍ | 190367/225000 [359:22:49<71:25:01,  7.42s/it]
 85%|████████▍ | 190368/225000 [359:23:02<87:23:01,  9.08s/it]
 85%|████████▍ | 190369/225000 [359:23:08<77:05:26,  8.01s/it]
 85%|████████▍ | 190370/225000 [359:23:16<77:51:01,  8.09s/it]
 85%|████████▍ | 190371/225000 [359:23:24<77:41:35,  8.08s/it]
 85%|████████▍ | 190372/225000 [359:23:36<89:39:03,  9.32s/it]
 85%|████████▍ | 190373/225000 [359:23:47<92:57:28,  9.66s/it]
 85%|████████▍ | 190374/225000 [359:23:54<87:26:55,  9.09s/it]
 85%|████████▍ | 190375/225000 [359:24:06<96:00:16,  9.98s/it]
 85%|████████▍ | 190376/225000 [359:24:11<80:12:42,  8.34s/it]
 85%|████████▍ | 190377/225000 [359:24:15<68:35:17,  7.13s/it]
 85%|████████▍ | 190378/225000 [359:24:23<69:21:05,  7.21s/it]
 85%|████████▍ | 190379/225000 [359:24:32<75:18:37,  7.83s/it]
 85%|████████▍ | 190380/225000 [359:24:37<67:33:41,  7.03s/it]
 85%|████████▍ | 190381/225000 [359:24:47<75:13:04,  7.82s/it]
 85%|████████▍ | 190382/225000 [359:24:56<79:15:02,  8.24s/it]
 85%|████████▍ | 190383/225000 [359:25:02<72:26:07,  7.53s/it]
 85%|████████▍ | 190384/225000 [359:25:09<71:19:38,  7.42s/it]
 85%|████████▍ | 190385/225000 [359:25:13<62:07:01,  6.46s/it]
 85%|████████▍ | 190386/225000 [359:25:29<88:04:38,  9.16s/it]
 85%|████████▍ | 190387/225000 [359:25:33<75:39:42,  7.87s/it]
 85%|████████▍ | 190388/225000 [359:25:39<69:20:19,  7.21s/it]
 85%|████████▍ | 190389/225000 [359:25:45<65:49:11,  6.85s/it]
 85%|████████▍ | 190390/225000 [359:25:51<63:38:13,  6.62s/it]
 85%|████████▍ | 190391/225000 [359:26:04<80:34:01,  8.38s/it]
 85%|████████▍ | 190392/225000 [359:26:12<80:24:30,  8.36s/it]
 85%|████████▍ | 190393/225000 [359:26:18<74:22:27,  7.74s/it]
 85%|████████▍ | 190394/225000 [359:26:24<68:02:29,  7.08s/it]
 85%|████████▍ | 190395/225000 [359:26:31<67:12:49,  6.99s/it]
 85%|████████▍ | 190396/225000 [359:26:37<63:58:05,  6.65s/it]
 85%|████████▍ | 190397/225000 [359:26:48<76:37:36,  7.97s/it]
 85%|████████▍ | 190398/225000 [359:26:55<74:06:25,  7.71s/it]
 85%|████████▍ | 190399/225000 [359:27:04<79:16:46,  8.25s/it]
 85%|████████▍ | 190400/225000 [359:27:10<71:30:33,  7.44s/it]
                                                              
{'loss': '1.203', 'grad_norm': '1.588', 'learning_rate': '3.144e-06', 'epoch': '0.8462', 'num_input_tokens_seen': 597478144, 'train_runtime': '1.294e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190400/225000 [359:27:10<71:30:33,  7.44s/it]
 85%|████████▍ | 190401/225000 [359:27:25<93:04:48,  9.68s/it]
 85%|████████▍ | 190402/225000 [359:27:29<79:02:47,  8.22s/it]
 85%|████████▍ | 190403/225000 [359:27:35<70:19:02,  7.32s/it]
 85%|████████▍ | 190404/225000 [359:27:40<63:52:36,  6.65s/it]
 85%|████████▍ | 190405/225000 [359:27:53<82:48:38,  8.62s/it]
 85%|████████▍ | 190406/225000 [359:28:01<81:11:02,  8.45s/it]
 85%|████████▍ | 190407/225000 [359:28:07<74:54:28,  7.80s/it]
 85%|████████▍ | 190408/225000 [359:28:15<75:29:49,  7.86s/it]
 85%|████████▍ | 190409/225000 [359:28:20<66:53:35,  6.96s/it]
 85%|████████▍ | 190410/225000 [359:28:37<96:37:55, 10.06s/it]
 85%|████████▍ | 190411/225000 [359:28:46<91:23:30,  9.51s/it]
 85%|████████▍ | 190412/225000 [359:28:52<82:53:27,  8.63s/it]
 85%|████████▍ | 190413/225000 [359:28:58<74:47:06,  7.78s/it]
 85%|████████▍ | 190414/225000 [359:29:08<81:20:07,  8.47s/it]
 85%|████████▍ | 190415/225000 [359:29:12<68:33:52,  7.14s/it]
 85%|████████▍ | 190416/225000 [359:29:22<75:01:23,  7.81s/it]
 85%|████████▍ | 190417/225000 [359:29:29<73:31:01,  7.65s/it]
 85%|████████▍ | 190418/225000 [359:29:36<71:32:13,  7.45s/it]
 85%|████████▍ | 190419/225000 [359:29:47<82:38:33,  8.60s/it]
 85%|████████▍ | 190420/225000 [359:29:55<80:26:39,  8.37s/it]
 85%|████████▍ | 190421/225000 [359:30:02<75:35:39,  7.87s/it]
 85%|████████▍ | 190422/225000 [359:30:14<88:22:21,  9.20s/it]
 85%|████████▍ | 190423/225000 [359:30:23<86:45:25,  9.03s/it]
 85%|████████▍ | 190424/225000 [359:30:31<84:28:27,  8.80s/it]
 85%|████████▍ | 190425/225000 [359:30:35<71:52:48,  7.48s/it]
 85%|████████▍ | 190426/225000 [359:30:45<78:45:12,  8.20s/it]
 85%|████████▍ | 190427/225000 [359:30:50<69:21:46,  7.22s/it]
 85%|████████▍ | 190428/225000 [359:31:03<87:07:28,  9.07s/it]
 85%|████████▍ | 190429/225000 [359:31:10<79:33:00,  8.28s/it]
 85%|████████▍ | 190430/225000 [359:31:19<83:04:16,  8.65s/it]
 85%|████████▍ | 190431/225000 [359:31:27<81:05:49,  8.45s/it]
 85%|████████▍ | 190432/225000 [359:31:36<81:03:20,  8.44s/it]
 85%|████████▍ | 190433/225000 [359:31:42<73:38:50,  7.67s/it]
 85%|████████▍ | 190434/225000 [359:31:50<74:48:47,  7.79s/it]
 85%|████████▍ | 190435/225000 [359:31:56<69:31:26,  7.24s/it]
 85%|████████▍ | 190436/225000 [359:32:03<69:57:27,  7.29s/it]
 85%|████████▍ | 190437/225000 [359:32:08<64:02:38,  6.67s/it]
 85%|████████▍ | 190438/225000 [359:32:20<77:08:30,  8.04s/it]
 85%|████████▍ | 190439/225000 [359:32:28<79:37:08,  8.29s/it]
 85%|████████▍ | 190440/225000 [359:32:39<85:50:24,  8.94s/it]
 85%|████████▍ | 190441/225000 [359:32:44<74:06:49,  7.72s/it]
 85%|████████▍ | 190442/225000 [359:32:51<72:07:31,  7.51s/it]
 85%|████████▍ | 190443/225000 [359:32:59<74:16:07,  7.74s/it]
 85%|████████▍ | 190444/225000 [359:33:11<85:31:56,  8.91s/it]
 85%|████████▍ | 190445/225000 [359:33:17<76:40:13,  7.99s/it]
 85%|████████▍ | 190446/225000 [359:33:24<76:37:40,  7.98s/it]
 85%|████████▍ | 190447/225000 [359:33:31<72:15:34,  7.53s/it]
 85%|████████▍ | 190448/225000 [359:33:46<93:34:32,  9.75s/it]
 85%|████████▍ | 190449/225000 [359:33:51<80:55:51,  8.43s/it]
 85%|████████▍ | 190450/225000 [359:33:57<71:59:00,  7.50s/it]
                                                              
{'loss': '1.2', 'grad_norm': '1.414', 'learning_rate': '3.135e-06', 'epoch': '0.8464', 'num_input_tokens_seen': 597685696, 'train_runtime': '1.294e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190450/225000 [359:33:57<71:59:00,  7.50s/it]
 85%|████████▍ | 190451/225000 [359:34:04<71:28:25,  7.45s/it]
 85%|████████▍ | 190452/225000 [359:34:10<67:20:41,  7.02s/it]
 85%|████████▍ | 190453/225000 [359:34:23<84:13:11,  8.78s/it]
 85%|████████▍ | 190454/225000 [359:34:29<76:49:25,  8.01s/it]
 85%|████████▍ | 190455/225000 [359:34:42<91:14:24,  9.51s/it]
 85%|████████▍ | 190456/225000 [359:34:47<79:27:22,  8.28s/it]
 85%|████████▍ | 190457/225000 [359:34:58<85:54:32,  8.95s/it]
 85%|████████▍ | 190458/225000 [359:35:03<73:15:35,  7.64s/it]
 85%|████████▍ | 190459/225000 [359:35:09<70:15:59,  7.32s/it]
 85%|████████▍ | 190460/225000 [359:35:18<74:37:23,  7.78s/it]
 85%|████████▍ | 190461/225000 [359:35:23<65:40:17,  6.84s/it]
 85%|████████▍ | 190462/225000 [359:35:35<83:03:58,  8.66s/it]
 85%|████████▍ | 190463/225000 [359:35:40<71:25:21,  7.44s/it]
 85%|████████▍ | 190464/225000 [359:35:45<64:32:58,  6.73s/it]
 85%|████████▍ | 190465/225000 [359:35:49<57:20:55,  5.98s/it]
 85%|████████▍ | 190466/225000 [359:36:02<75:16:23,  7.85s/it]
 85%|████████▍ | 190467/225000 [359:36:13<84:28:25,  8.81s/it]
 85%|████████▍ | 190468/225000 [359:36:17<71:38:41,  7.47s/it]
 85%|████████▍ | 190469/225000 [359:36:25<73:55:09,  7.71s/it]
 85%|████████▍ | 190470/225000 [359:36:34<78:10:14,  8.15s/it]
 85%|████████▍ | 190471/225000 [359:36:45<83:52:52,  8.75s/it]
 85%|████████▍ | 190472/225000 [359:36:51<77:16:30,  8.06s/it]
 85%|████████▍ | 190473/225000 [359:37:03<87:29:03,  9.12s/it]
 85%|████████▍ | 190474/225000 [359:37:09<80:06:58,  8.35s/it]
 85%|████████▍ | 190475/225000 [359:37:15<71:40:07,  7.47s/it]
 85%|████████▍ | 190476/225000 [359:37:20<65:41:27,  6.85s/it]
 85%|████████▍ | 190477/225000 [359:37:28<68:21:14,  7.13s/it]
 85%|████████▍ | 190478/225000 [359:37:34<66:14:04,  6.91s/it]
 85%|████████▍ | 190479/225000 [359:37:41<66:05:31,  6.89s/it]
 85%|████████▍ | 190480/225000 [359:37:47<62:32:38,  6.52s/it]
 85%|████████▍ | 190481/225000 [359:37:53<62:41:37,  6.54s/it]
 85%|████████▍ | 190482/225000 [359:37:59<61:24:58,  6.41s/it]
 85%|████████▍ | 190483/225000 [359:38:16<90:18:28,  9.42s/it]
 85%|████████▍ | 190484/225000 [359:38:25<89:29:16,  9.33s/it]
 85%|████████▍ | 190485/225000 [359:38:34<89:36:57,  9.35s/it]
 85%|████████▍ | 190486/225000 [359:38:41<81:28:52,  8.50s/it]
 85%|████████▍ | 190487/225000 [359:38:47<73:36:25,  7.68s/it]
 85%|████████▍ | 190488/225000 [359:38:57<82:39:29,  8.62s/it]
 85%|████████▍ | 190489/225000 [359:39:04<77:34:34,  8.09s/it]
 85%|████████▍ | 190490/225000 [359:39:14<81:10:10,  8.47s/it]
 85%|████████▍ | 190491/225000 [359:39:24<87:15:33,  9.10s/it]
 85%|████████▍ | 190492/225000 [359:39:29<75:06:16,  7.84s/it]
 85%|████████▍ | 190493/225000 [359:39:35<69:10:43,  7.22s/it]
 85%|████████▍ | 190494/225000 [359:39:42<67:30:17,  7.04s/it]
 85%|████████▍ | 190495/225000 [359:39:53<81:22:02,  8.49s/it]
 85%|████████▍ | 190496/225000 [359:40:02<81:21:16,  8.49s/it]
 85%|████████▍ | 190497/225000 [359:40:12<85:59:04,  8.97s/it]
 85%|████████▍ | 190498/225000 [359:40:18<77:44:32,  8.11s/it]
 85%|████████▍ | 190499/225000 [359:40:25<74:59:00,  7.82s/it]
 85%|████████▍ | 190500/225000 [359:40:39<91:17:49,  9.53s/it]
                                                              
{'loss': '1.222', 'grad_norm': '1.156', 'learning_rate': '3.126e-06', 'epoch': '0.8467', 'num_input_tokens_seen': 597888672, 'train_runtime': '1.295e+06', 'train_tokens_per_second': '461.7'}

 85%|████████▍ | 190500/225000 [359:40:39<91:17:49,  9.53s/it]
 85%|████████▍ | 190501/225000 [359:40:43<75:45:41,  7.91s/it]
 85%|████████▍ | 190502/225000 [359:40:55<89:04:28,  9.30s/it]
 85%|████████▍ | 190503/225000 [359:41:00<75:01:22,  7.83s/it]
 85%|████████▍ | 190504/225000 [359:41:04<65:06:01,  6.79s/it]
 85%|████████▍ | 190505/225000 [359:41:13<70:53:10,  7.40s/it]
 85%|████████▍ | 190506/225000 [359:41:23<78:48:37,  8.23s/it]
 85%|████████▍ | 190507/225000 [359:41:28<68:00:21,  7.10s/it]
 85%|████████▍ | 190508/225000 [359:41:35<69:24:46,  7.24s/it]
 85%|████████▍ | 190509/225000 [359:41:42<68:00:35,  7.10s/it]
 85%|████████▍ | 190510/225000 [359:41:50<72:12:54,  7.54s/it]
 85%|████████▍ | 190511/225000 [359:42:00<78:14:39,  8.17s/it]
 85%|████████▍ | 190512/225000 [359:42:09<79:22:40,  8.29s/it]
 85%|████████▍ | 190513/225000 [359:42:17<78:31:32,  8.20s/it]
 85%|████████▍ | 190514/225000 [359:42:24<76:47:43,  8.02s/it]
 85%|████████▍ | 190515/225000 [359:42:30<69:03:56,  7.21s/it]
 85%|████████▍ | 190516/225000 [359:42:38<71:22:07,  7.45s/it]
 85%|████████▍ | 190517/225000 [359:42:48<80:54:37,  8.45s/it]
 85%|████████▍ | 190518/225000 [359:42:53<69:32:11,  7.26s/it]
 85%|████████▍ | 190519/225000 [359:42:59<67:31:37,  7.05s/it]
 85%|████████▍ | 190520/225000 [359:43:06<66:57:42,  6.99s/it]
 85%|████████▍ | 190521/225000 [359:43:21<89:34:19,  9.35s/it]
 85%|████████▍ | 190522/225000 [359:43:29<86:27:07,  9.03s/it]
 85%|████████▍ | 190523/225000 [359:43:38<86:27:13,  9.03s/it]
 85%|████████▍ | 190524/225000 [359:43:45<80:16:22,  8.38s/it]
 85%|████████▍ | 190525/225000 [359:43:52<76:27:21,  7.98s/it]
 85%|████████▍ | 190526/225000 [359:43:59<72:24:49,  7.56s/it]
 85%|████████▍ | 190527/225000 [359:44:07<74:00:34,  7.73s/it]
 85%|████████▍ | 190528/225000 [359:44:12<66:50:56,  6.98s/it]
 85%|████████▍ | 190529/225000 [359:44:24<81:14:37,  8.48s/it]
 85%|████████▍ | 190530/225000 [359:44:32<78:16:05,  8.17s/it]
 85%|████████▍ | 190531/225000 [359:44:37<70:20:35,  7.35s/it]
 85%|████████▍ | 190532/225000 [359:44:44<68:02:57,  7.11s/it]
 85%|████████▍ | 190533/225000 [359:44:52<70:27:50,  7.36s/it]
 85%|████████▍ | 190534/225000 [359:45:01<76:37:10,  8.00s/it]
 85%|████████▍ | 190535/225000 [359:45:11<81:50:45,  8.55s/it]
 85%|████████▍ | 190536/225000 [359:45:18<76:34:11,  8.00s/it]
 85%|████████▍ | 190537/225000 [359:45:24<71:01:16,  7.42s/it]
 85%|████████▍ | 190538/225000 [359:45:33<76:17:19,  7.97s/it]
 85%|████████▍ | 190539/225000 [359:45:38<67:11:37,  7.02s/it]
 85%|████████▍ | 190540/225000 [359:45:48<76:41:04,  8.01s/it]
 85%|████████▍ | 190541/225000 [359:45:57<80:22:00,  8.40s/it]
 85%|████████▍ | 190542/225000 [359:46:07<83:01:46,  8.67s/it]
 85%|████████▍ | 190543/225000 [359:46:14<77:57:39,  8.15s/it]
 85%|████████▍ | 190544/225000 [359:46:19<68:35:11,  7.17s/it]
 85%|████████▍ | 190545/225000 [359:46:26<70:12:33,  7.34s/it]
 85%|████████▍ | 190546/225000 [359:46:32<64:38:35,  6.75s/it]
 85%|████████▍ | 190547/225000 [359:46:39<65:28:43,  6.84s/it]
 85%|████████▍ | 190548/225000 [359:46:45<62:35:49,  6.54s/it]
 85%|████████▍ | 190549/225000 [359:46:55<73:12:01,  7.65s/it]
 85%|████████▍ | 190550/225000 [359:47:04<77:57:29,  8.15s/it]
                                                              
{'loss': '1.229', 'grad_norm': '1.021', 'learning_rate': '3.117e-06', 'epoch': '0.8469', 'num_input_tokens_seen': 598091432, 'train_runtime': '1.295e+06', 'train_tokens_per_second': '461.8'}

 85%|████████▍ | 190550/225000 [359:47:04<77:57:29,  8.15s/it]
 85%|████████▍ | 190551/225000 [359:47:08<66:00:10,  6.90s/it]
 85%|████████▍ | 190552/225000 [359:47:19<78:27:18,  8.20s/it]
 85%|████████▍ | 190553/225000 [359:47:30<84:29:02,  8.83s/it]
 85%|████████▍ | 190554/225000 [359:47:36<77:12:27,  8.07s/it]
 85%|████████▍ | 190555/225000 [359:47:46<83:30:02,  8.73s/it]
 85%|████████▍ | 190556/225000 [359:47:50<69:34:22,  7.27s/it]
 85%|████████▍ | 190557/225000 [359:47:58<71:23:10,  7.46s/it]
 85%|████████▍ | 190558/225000 [359:48:04<67:30:42,  7.06s/it]
 85%|████████▍ | 190559/225000 [359:48:10<64:01:48,  6.69s/it]
 85%|████████▍ | 190560/225000 [359:48:21<76:33:37,  8.00s/it]
 85%|████████▍ | 190561/225000 [359:48:28<73:33:09,  7.69s/it]
 85%|████████▍ | 190562/225000 [359:48:35<71:24:04,  7.46s/it]
 85%|████████▍ | 190563/225000 [359:48:40<65:18:32,  6.83s/it]
 85%|████████▍ | 190564/225000 [359:48:47<65:42:23,  6.87s/it]
 85%|████████▍ | 190565/225000 [359:48:54<65:21:05,  6.83s/it]
 85%|████████▍ | 190566/225000 [359:49:10<90:50:05,  9.50s/it]
 85%|████████▍ | 190567/225000 [359:49:16<81:49:10,  8.55s/it]
 85%|████████▍ | 190568/225000 [359:49:23<77:32:37,  8.11s/it]
 85%|████████▍ | 190569/225000 [359:49:28<67:34:34,  7.07s/it]
 85%|████████▍ | 190570/225000 [359:49:39<80:11:13,  8.38s/it]
 85%|████████▍ | 190571/225000 [359:49:43<66:49:33,  6.99s/it]
 85%|████████▍ | 190572/225000 [359:49:53<76:13:49,  7.97s/it]
 85%|████████▍ | 190573/225000 [359:49:59<70:48:38,  7.40s/it]
 85%|████████▍ | 190574/225000 [359:50:08<74:15:56,  7.77s/it]
 85%|████████▍ | 190575/225000 [359:50:17<78:14:05,  8.18s/it]
 85%|████████▍ | 190576/225000 [359:50:24<73:23:19,  7.67s/it]
 85%|████████▍ | 190577/225000 [359:50:29<67:55:02,  7.10s/it]
 85%|████████▍ | 190578/225000 [359:50:36<66:54:15,  7.00s/it]
 85%|████████▍ | 190579/225000 [359:50:43<67:51:25,  7.10s/it]
 85%|████████▍ | 190580/225000 [359:50:54<78:46:16,  8.24s/it]
 85%|████████▍ | 190581/225000 [359:51:09<98:06:13, 10.26s/it]
 85%|████████▍ | 190582/225000 [359:51:17<91:55:35,  9.62s/it]
 85%|████████▍ | 190583/225000 [359:51:21<75:54:09,  7.94s/it]
 85%|████████▍ | 190584/225000 [359:51:31<79:20:31,  8.30s/it]
 85%|████████▍ | 190585/225000 [359:51:36<70:15:27,  7.35s/it]
 85%|████████▍ | 190586/225000 [359:51:41<65:39:43,  6.87s/it]
 85%|████████▍ | 190587/225000 [359:51:48<64:15:46,  6.72s/it]
 85%|████████▍ | 190588/225000 [359:51:59<77:06:40,  8.07s/it]
 85%|████████▍ | 190589/225000 [359:52:03<65:44:55,  6.88s/it]
 85%|████████▍ | 190590/225000 [359:52:13<73:22:53,  7.68s/it]
 85%|████████▍ | 190591/225000 [359:52:17<64:58:49,  6.80s/it]
 85%|████████▍ | 190592/225000 [359:52:25<67:35:13,  7.07s/it]
 85%|████████▍ | 190593/225000 [359:52:34<72:40:14,  7.60s/it]
 85%|████████▍ | 190594/225000 [359:52:45<83:34:22,  8.74s/it]
 85%|████████▍ | 190595/225000 [359:52:53<79:13:21,  8.29s/it]
 85%|████████▍ | 190596/225000 [359:53:01<79:33:11,  8.32s/it]
 85%|████████▍ | 190597/225000 [359:53:05<68:29:52,  7.17s/it]
 85%|████████▍ | 190598/225000 [359:53:14<71:35:18,  7.49s/it]
 85%|████████▍ | 190599/225000 [359:53:20<68:46:59,  7.20s/it]
 85%|████████▍ | 190600/225000 [359:53:25<62:26:24,  6.53s/it]
                                                              
{'loss': '1.193', 'grad_norm': '1.402', 'learning_rate': '3.108e-06', 'epoch': '0.8471', 'num_input_tokens_seen': 598291784, 'train_runtime': '1.296e+06', 'train_tokens_per_second': '461.8'}

 85%|████████▍ | 190600/225000 [359:53:25<62:26:24,  6.53s/it]
 85%|████████▍ | 190601/225000 [359:53:30<58:23:30,  6.11s/it]
 85%|████████▍ | 190602/225000 [359:53:43<77:21:17,  8.10s/it]
 85%|████████▍ | 190603/225000 [359:53:50<73:21:05,  7.68s/it]
 85%|████████▍ | 190604/225000 [359:53:55<65:51:04,  6.89s/it]
 85%|████████▍ | 190605/225000 [359:54:05<75:44:09,  7.93s/it]
 85%|████████▍ | 190606/225000 [359:54:10<66:30:20,  6.96s/it]
 85%|████████▍ | 190607/225000 [359:54:17<66:10:14,  6.93s/it]
 85%|████████▍ | 190608/225000 [359:54:26<72:16:20,  7.57s/it]
 85%|████████▍ | 190609/225000 [359:54:31<66:43:49,  6.99s/it]
 85%|████████▍ | 190610/225000 [359:54:39<69:05:57,  7.23s/it]
 85%|████████▍ | 190611/225000 [359:54:52<85:03:44,  8.90s/it]
 85%|████████▍ | 190612/225000 [359:54:57<73:09:02,  7.66s/it]
 85%|████████▍ | 190613/225000 [359:55:07<80:58:38,  8.48s/it]
 85%|████████▍ | 190614/225000 [359:55:16<81:22:35,  8.52s/it]
 85%|████████▍ | 190615/225000 [359:55:25<82:27:54,  8.63s/it]
 85%|████████▍ | 190616/225000 [359:55:36<91:36:32,  9.59s/it]
 85%|████████▍ | 190617/225000 [359:55:41<78:14:11,  8.19s/it]
 85%|████████▍ | 190618/225000 [359:55:50<79:43:37,  8.35s/it]
 85%|████████▍ | 190619/225000 [359:56:02<89:36:47,  9.38s/it]
 85%|████████▍ | 190620/225000 [359:56:13<94:35:38,  9.91s/it]
 85%|████████▍ | 190621/225000 [359:56:19<82:38:20,  8.65s/it]
 85%|████████▍ | 190622/225000 [359:56:23<69:28:25,  7.28s/it]
 85%|████████▍ | 190623/225000 [359:56:31<71:07:24,  7.45s/it]
 85%|████████▍ | 190624/225000 [359:56:38<70:43:56,  7.41s/it]
 85%|████████▍ | 190625/225000 [359:56:50<83:43:33,  8.77s/it]
 85%|████████▍ | 190626/225000 [359:57:00<87:11:05,  9.13s/it]
 85%|████████▍ | 190627/225000 [359:57:09<86:45:21,  9.09s/it]
 85%|████████▍ | 190628/225000 [359:57:13<73:52:09,  7.74s/it]
 85%|████████▍ | 190629/225000 [359:57:22<74:55:09,  7.85s/it]
 85%|████████▍ | 190630/225000 [359:57:26<64:37:52,  6.77s/it]
 85%|████████▍ | 190631/225000 [359:57:34<69:34:41,  7.29s/it]
 85%|████████▍ | 190632/225000 [359:57:38<58:55:16,  6.17s/it]
 85%|████████▍ | 190633/225000 [359:57:47<67:54:26,  7.11s/it]
 85%|████████▍ | 190634/225000 [359:57:54<66:58:30,  7.02s/it]
 85%|████████▍ | 190635/225000 [359:58:04<74:42:18,  7.83s/it]
 85%|████████▍ | 190636/225000 [359:58:12<75:56:03,  7.95s/it]
 85%|████████▍ | 190637/225000 [359:58:18<69:59:53,  7.33s/it]
 85%|████████▍ | 190638/225000 [359:58:25<69:39:58,  7.30s/it]
 85%|████████▍ | 190639/225000 [359:58:33<72:36:16,  7.61s/it]
 85%|████████▍ | 190640/225000 [359:58:42<74:02:29,  7.76s/it]
 85%|████████▍ | 190641/225000 [359:58:46<65:13:41,  6.83s/it]
 85%|████████▍ | 190642/225000 [359:58:55<71:07:02,  7.45s/it]
 85%|████████▍ | 190643/225000 [359:59:00<62:46:24,  6.58s/it]
 85%|████████▍ | 190644/225000 [359:59:09<70:12:09,  7.36s/it]
 85%|████████▍ | 190645/225000 [359:59:18<74:20:55,  7.79s/it]
 85%|████████▍ | 190646/225000 [359:59:22<64:54:17,  6.80s/it]
 85%|████████▍ | 190647/225000 [359:59:34<80:11:08,  8.40s/it]
 85%|████████▍ | 190648/225000 [359:59:41<76:05:13,  7.97s/it]
 85%|████████▍ | 190649/225000 [359:59:54<89:20:40,  9.36s/it]
 85%|████████▍ | 190650/225000 [360:00:04<90:56:30,  9.53s/it]
                                                              
{'loss': '1.201', 'grad_norm': '1.155', 'learning_rate': '3.1e-06', 'epoch': '0.8473', 'num_input_tokens_seen': 598495272, 'train_runtime': '1.296e+06', 'train_tokens_per_second': '461.8'}

 85%|████████▍ | 190650/225000 [360:00:04<90:56:30,  9.53s/it]
 85%|████████▍ | 190651/225000 [360:00:10<82:01:15,  8.60s/it]
 85%|████████▍ | 190652/225000 [360:00:19<84:03:59,  8.81s/it]
 85%|████████▍ | 190653/225000 [360:00:25<74:30:04,  7.81s/it]
 85%|████████▍ | 190654/225000 [360:00:32<73:24:33,  7.69s/it]
 85%|████████▍ | 190655/225000 [360:00:38<68:30:57,  7.18s/it]
 85%|████████▍ | 190656/225000 [360:00:45<68:16:26,  7.16s/it]
 85%|████████▍ | 190657/225000 [360:00:50<59:57:54,  6.29s/it]
 85%|████████▍ | 190658/225000 [360:00:55<56:39:18,  5.94s/it]
 85%|████████▍ | 190659/225000 [360:01:08<77:15:27,  8.10s/it]
 85%|████████▍ | 190660/225000 [360:01:15<75:28:37,  7.91s/it]
 85%|████████▍ | 190661/225000 [360:01:20<64:36:31,  6.77s/it]
 85%|████████▍ | 190662/225000 [360:01:25<61:14:05,  6.42s/it]
 85%|████████▍ | 190663/225000 [360:01:35<71:01:04,  7.45s/it]
 85%|████████▍ | 190664/225000 [360:01:40<62:39:49,  6.57s/it]
 85%|████████▍ | 190665/225000 [360:01:51<77:52:40,  8.17s/it]
 85%|████████▍ | 190666/225000 [360:01:56<68:33:57,  7.19s/it]
 85%|████████▍ | 190667/225000 [360:02:02<63:26:49,  6.65s/it]
 85%|████████▍ | 190668/225000 [360:02:17<87:51:34,  9.21s/it]
 85%|████████▍ | 190669/225000 [360:02:26<86:18:47,  9.05s/it]
 85%|████████▍ | 190670/225000 [360:02:35<87:25:29,  9.17s/it]
 85%|████████▍ | 190671/225000 [360:02:44<85:27:06,  8.96s/it]
 85%|████████▍ | 190672/225000 [360:02:53<85:44:43,  8.99s/it]
 85%|████████▍ | 190673/225000 [360:02:59<77:17:51,  8.11s/it]
 85%|████████▍ | 190674/225000 [360:03:06<75:59:45,  7.97s/it]
 85%|████████▍ | 190675/225000 [360:03:11<66:54:45,  7.02s/it]
 85%|████████▍ | 190676/225000 [360:03:23<80:28:46,  8.44s/it]
 85%|████████▍ | 190677/225000 [360:03:30<75:40:30,  7.94s/it]
 85%|████████▍ | 190678/225000 [360:03:41<85:05:13,  8.92s/it]
 85%|████████▍ | 190679/225000 [360:03:46<75:01:16,  7.87s/it]
 85%|████████▍ | 190680/225000 [360:03:56<80:05:46,  8.40s/it]
 85%|████████▍ | 190681/225000 [360:04:02<74:35:58,  7.83s/it]
 85%|████████▍ | 190682/225000 [360:04:09<70:45:18,  7.42s/it]
 85%|████████▍ | 190683/225000 [360:04:21<83:39:14,  8.78s/it]
 85%|████████▍ | 190684/225000 [360:04:28<78:14:32,  8.21s/it]
 85%|████████▍ | 190685/225000 [360:04:34<73:36:56,  7.72s/it]
 85%|████████▍ | 190686/225000 [360:04:41<71:49:40,  7.54s/it]
 85%|████████▍ | 190687/225000 [360:04:48<68:39:31,  7.20s/it]
 85%|████████▍ | 190688/225000 [360:04:54<64:42:38,  6.79s/it]
 85%|████████▍ | 190689/225000 [360:05:00<63:34:38,  6.67s/it]
 85%|████████▍ | 190690/225000 [360:05:05<58:35:40,  6.15s/it]
 85%|████████▍ | 190691/225000 [360:05:10<56:26:13,  5.92s/it]
 85%|████████▍ | 190692/225000 [360:05:15<52:14:44,  5.48s/it]
 85%|████████▍ | 190693/225000 [360:05:29<77:20:36,  8.12s/it]
 85%|████████▍ | 190694/225000 [360:05:38<78:56:00,  8.28s/it]
 85%|████████▍ | 190695/225000 [360:05:43<69:03:56,  7.25s/it]
 85%|████████▍ | 190696/225000 [360:05:54<80:48:36,  8.48s/it]
 85%|████████▍ | 190697/225000 [360:06:04<84:23:37,  8.86s/it]
 85%|████████▍ | 190698/225000 [360:06:09<73:45:41,  7.74s/it]
 85%|████████▍ | 190699/225000 [360:06:15<69:43:44,  7.32s/it]
 85%|████████▍ | 190700/225000 [360:06:26<81:08:57,  8.52s/it]
                                                              
{'loss': '1.207', 'grad_norm': '1.308', 'learning_rate': '3.091e-06', 'epoch': '0.8476', 'num_input_tokens_seen': 598695520, 'train_runtime': '1.296e+06', 'train_tokens_per_second': '461.8'}

 85%|████████▍ | 190700/225000 [360:06:26<81:08:57,  8.52s/it]
 85%|████████▍ | 190701/225000 [360:06:30<67:46:36,  7.11s/it]
 85%|████████▍ | 190702/225000 [360:06:40<74:38:30,  7.83s/it]
 85%|████████▍ | 190703/225000 [360:06:48<76:27:37,  8.03s/it]
 85%|████████▍ | 190704/225000 [360:06:57<77:43:32,  8.16s/it]
 85%|████████▍ | 190705/225000 [360:07:05<79:13:52,  8.32s/it]
 85%|████████▍ | 190706/225000 [360:07:12<74:30:47,  7.82s/it]
 85%|████████▍ | 190707/225000 [360:07:17<66:58:36,  7.03s/it]
 85%|████████▍ | 190708/225000 [360:07:22<61:16:04,  6.43s/it]
 85%|████████▍ | 190709/225000 [360:07:28<58:17:03,  6.12s/it]
 85%|████████▍ | 190710/225000 [360:07:35<62:47:56,  6.59s/it]
 85%|████████▍ | 190711/225000 [360:07:43<66:46:50,  7.01s/it]
 85%|████████▍ | 190712/225000 [360:07:50<66:14:34,  6.96s/it]
 85%|████████▍ | 190713/225000 [360:08:00<74:44:49,  7.85s/it]
 85%|████████▍ | 190714/225000 [360:08:11<84:40:53,  8.89s/it]
 85%|████████▍ | 190715/225000 [360:08:17<74:04:57,  7.78s/it]
 85%|████████▍ | 190716/225000 [360:08:28<83:48:51,  8.80s/it]
 85%|████████▍ | 190717/225000 [360:08:33<74:34:41,  7.83s/it]
 85%|████████▍ | 190718/225000 [360:08:40<70:06:09,  7.36s/it]
 85%|████████▍ | 190719/225000 [360:08:46<67:40:39,  7.11s/it]
 85%|████████▍ | 190720/225000 [360:08:55<72:18:11,  7.59s/it]
 85%|████████▍ | 190721/225000 [360:09:02<72:17:16,  7.59s/it]
 85%|████████▍ | 190722/225000 [360:09:16<88:58:14,  9.34s/it]
 85%|████████▍ | 190723/225000 [360:09:27<93:05:09,  9.78s/it]
 85%|████████▍ | 190724/225000 [360:09:35<90:09:18,  9.47s/it]
 85%|████████▍ | 190725/225000 [360:09:44<87:30:14,  9.19s/it]
 85%|████████▍ | 190726/225000 [360:09:49<75:26:31,  7.92s/it]
 85%|████████▍ | 190727/225000 [360:09:54<66:43:09,  7.01s/it]
 85%|████████▍ | 190728/225000 [360:10:06<82:25:24,  8.66s/it]
 85%|████████▍ | 190729/225000 [360:10:12<73:56:39,  7.77s/it]
 85%|████████▍ | 190730/225000 [360:10:23<84:09:25,  8.84s/it]
 85%|████████▍ | 190731/225000 [360:10:31<81:25:41,  8.55s/it]
 85%|████████▍ | 190732/225000 [360:10:38<75:28:01,  7.93s/it]
 85%|████████▍ | 190733/225000 [360:10:44<70:47:57,  7.44s/it]
 85%|████████▍ | 190734/225000 [360:10:53<75:36:37,  7.94s/it]
 85%|████████▍ | 190735/225000 [360:11:01<75:08:36,  7.89s/it]
 85%|████████▍ | 190736/225000 [360:11:05<64:15:33,  6.75s/it]
 85%|████████▍ | 190737/225000 [360:11:10<60:04:22,  6.31s/it]
 85%|████████▍ | 190738/225000 [360:11:21<73:13:44,  7.69s/it]
 85%|████████▍ | 190739/225000 [360:11:25<61:14:45,  6.44s/it]
 85%|████████▍ | 190740/225000 [360:11:33<67:34:36,  7.10s/it]
 85%|████████▍ | 190741/225000 [360:11:40<67:11:47,  7.06s/it]
 85%|████████▍ | 190742/225000 [360:11:53<82:13:05,  8.64s/it]
 85%|████████▍ | 190743/225000 [360:12:00<79:16:59,  8.33s/it]
 85%|████████▍ | 190744/225000 [360:12:07<75:41:25,  7.95s/it]
 85%|████████▍ | 190745/225000 [360:12:11<64:45:41,  6.81s/it]
 85%|████████▍ | 190746/225000 [360:12:25<83:25:46,  8.77s/it]
 85%|████████▍ | 190747/225000 [360:12:33<80:55:47,  8.51s/it]
 85%|████████▍ | 190748/225000 [360:12:43<85:39:54,  9.00s/it]
 85%|████████▍ | 190749/225000 [360:12:50<79:07:54,  8.32s/it]
 85%|████████▍ | 190750/225000 [360:12:56<74:23:21,  7.82s/it]
                                                              
{'loss': '1.222', 'grad_norm': '1.198', 'learning_rate': '3.082e-06', 'epoch': '0.8478', 'num_input_tokens_seen': 598898232, 'train_runtime': '1.297e+06', 'train_tokens_per_second': '461.8'}

 85%|████████▍ | 190750/225000 [360:12:56<74:23:21,  7.82s/it]
 85%|████████▍ | 190751/225000 [360:13:01<66:19:20,  6.97s/it]
 85%|████████▍ | 190752/225000 [360:13:08<66:53:59,  7.03s/it]
 85%|████████▍ | 190753/225000 [360:13:21<81:55:55,  8.61s/it]
 85%|████████▍ | 190754/225000 [360:13:32<89:55:23,  9.45s/it]
 85%|████████▍ | 190755/225000 [360:13:39<83:57:56,  8.83s/it]
 85%|████████▍ | 190756/225000 [360:13:46<76:50:54,  8.08s/it]
 85%|████████▍ | 190757/225000 [360:13:59<90:02:56,  9.47s/it]
 85%|████████▍ | 190758/225000 [360:14:07<87:04:12,  9.15s/it]
 85%|████████▍ | 190759/225000 [360:14:18<93:38:20,  9.84s/it]
 85%|████████▍ | 190760/225000 [360:14:29<95:04:11, 10.00s/it]
 85%|████████▍ | 190761/225000 [360:14:34<82:41:17,  8.69s/it]
 85%|████████▍ | 190762/225000 [360:14:42<80:20:22,  8.45s/it]
 85%|████████▍ | 190763/225000 [360:14:49<76:34:17,  8.05s/it]
 85%|████████▍ | 190764/225000 [360:14:56<73:39:20,  7.75s/it]
 85%|████████▍ | 190765/225000 [360:15:05<76:47:51,  8.08s/it]
 85%|████████▍ | 190766/225000 [360:15:10<66:27:04,  6.99s/it]
 85%|████████▍ | 190767/225000 [360:15:14<60:01:21,  6.31s/it]
 85%|████████▍ | 190768/225000 [360:15:25<72:34:45,  7.63s/it]
 85%|████████▍ | 190769/225000 [360:15:31<68:50:59,  7.24s/it]
 85%|████████▍ | 190770/225000 [360:15:40<72:35:06,  7.63s/it]
 85%|████████▍ | 190771/225000 [360:15:46<67:04:49,  7.06s/it]
 85%|████████▍ | 190772/225000 [360:15:53<67:45:15,  7.13s/it]
 85%|████████▍ | 190773/225000 [360:16:04<77:19:51,  8.13s/it]
 85%|████████▍ | 190774/225000 [360:16:16<90:18:43,  9.50s/it]
 85%|████████▍ | 190775/225000 [360:16:21<78:09:09,  8.22s/it]
 85%|████████▍ | 190776/225000 [360:16:31<82:53:43,  8.72s/it]
 85%|████████▍ | 190777/225000 [360:16:40<81:52:34,  8.61s/it]
 85%|████████▍ | 190778/225000 [360:16:46<75:51:40,  7.98s/it]
 85%|████████▍ | 190779/225000 [360:16:53<72:06:19,  7.59s/it]
 85%|████████▍ | 190780/225000 [360:17:02<76:01:02,  8.00s/it]
 85%|████████▍ | 190781/225000 [360:17:08<69:43:02,  7.33s/it]
 85%|████████▍ | 190782/225000 [360:17:15<69:46:25,  7.34s/it]
 85%|████████▍ | 190783/225000 [360:17:26<81:42:04,  8.60s/it]
 85%|████████▍ | 190784/225000 [360:17:36<84:41:36,  8.91s/it]
 85%|████████▍ | 190785/225000 [360:17:50<99:07:33, 10.43s/it]
 85%|████████▍ | 190786/225000 [360:17:54<81:29:17,  8.57s/it]
 85%|████████▍ | 190787/225000 [360:18:02<78:33:33,  8.27s/it]
 85%|████████▍ | 190788/225000 [360:18:10<76:51:11,  8.09s/it]
 85%|████████▍ | 190789/225000 [360:18:18<77:10:27,  8.12s/it]
 85%|████████▍ | 190790/225000 [360:18:25<75:33:50,  7.95s/it]
 85%|████████▍ | 190791/225000 [360:18:33<74:52:51,  7.88s/it]
 85%|████████▍ | 190792/225000 [360:18:38<66:18:53,  6.98s/it]
 85%|████████▍ | 190793/225000 [360:18:44<64:15:18,  6.76s/it]
 85%|████████▍ | 190794/225000 [360:18:49<60:04:57,  6.32s/it]
 85%|████████▍ | 190795/225000 [360:19:03<81:19:06,  8.56s/it]
 85%|████████▍ | 190796/225000 [360:19:08<71:38:40,  7.54s/it]
 85%|████████▍ | 190797/225000 [360:19:17<74:31:06,  7.84s/it]
 85%|████████▍ | 190798/225000 [360:19:22<66:34:44,  7.01s/it]
 85%|████████▍ | 190799/225000 [360:19:28<64:51:16,  6.83s/it]
 85%|████████▍ | 190800/225000 [360:19:34<60:19:47,  6.35s/it]
                                                              
{'loss': '1.202', 'grad_norm': '1.076', 'learning_rate': '3.073e-06', 'epoch': '0.848', 'num_input_tokens_seen': 599105368, 'train_runtime': '1.297e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 190800/225000 [360:19:34<60:19:47,  6.35s/it]
 85%|████████▍ | 190801/225000 [360:19:42<67:10:10,  7.07s/it]
 85%|████████▍ | 190802/225000 [360:19:46<58:23:28,  6.15s/it]
 85%|████████▍ | 190803/225000 [360:20:00<78:20:04,  8.25s/it]
 85%|████████▍ | 190804/225000 [360:20:09<82:17:17,  8.66s/it]
 85%|████████▍ | 190805/225000 [360:20:16<76:02:30,  8.01s/it]
 85%|████████▍ | 190806/225000 [360:20:22<71:07:59,  7.49s/it]
 85%|████████▍ | 190807/225000 [360:20:28<67:26:54,  7.10s/it]
 85%|████████▍ | 190808/225000 [360:20:36<70:26:30,  7.42s/it]
 85%|████████▍ | 190809/225000 [360:20:47<81:03:22,  8.53s/it]
 85%|████████▍ | 190810/225000 [360:20:53<72:36:59,  7.65s/it]
 85%|████████▍ | 190811/225000 [360:21:08<93:23:07,  9.83s/it]
 85%|████████▍ | 190812/225000 [360:21:14<83:10:48,  8.76s/it]
 85%|████████▍ | 190813/225000 [360:21:21<76:25:52,  8.05s/it]
 85%|████████▍ | 190814/225000 [360:21:36<96:55:35, 10.21s/it]
 85%|████████▍ | 190815/225000 [360:21:41<83:33:01,  8.80s/it]
 85%|████████▍ | 190816/225000 [360:21:52<89:00:09,  9.37s/it]
 85%|████████▍ | 190817/225000 [360:21:58<79:16:09,  8.35s/it]
 85%|████████▍ | 190818/225000 [360:22:06<78:13:50,  8.24s/it]
 85%|████████▍ | 190819/225000 [360:22:15<81:21:38,  8.57s/it]
 85%|████████▍ | 190820/225000 [360:22:23<78:20:40,  8.25s/it]
 85%|████████▍ | 190821/225000 [360:22:30<75:36:52,  7.96s/it]
 85%|████████▍ | 190822/225000 [360:22:37<71:23:27,  7.52s/it]
 85%|████████▍ | 190823/225000 [360:22:48<81:15:37,  8.56s/it]
 85%|████████▍ | 190824/225000 [360:22:54<76:15:26,  8.03s/it]
 85%|████████▍ | 190825/225000 [360:23:01<70:57:21,  7.47s/it]
 85%|████████▍ | 190826/225000 [360:23:15<90:56:23,  9.58s/it]
 85%|████████▍ | 190827/225000 [360:23:19<74:36:54,  7.86s/it]
 85%|████████▍ | 190828/225000 [360:23:31<86:09:08,  9.08s/it]
 85%|████████▍ | 190829/225000 [360:23:38<80:02:40,  8.43s/it]
 85%|████████▍ | 190830/225000 [360:23:48<84:47:21,  8.93s/it]
 85%|████████▍ | 190831/225000 [360:23:53<74:21:10,  7.83s/it]
 85%|████████▍ | 190832/225000 [360:24:00<71:25:06,  7.52s/it]
 85%|████████▍ | 190833/225000 [360:24:09<74:26:19,  7.84s/it]
 85%|████████▍ | 190834/225000 [360:24:15<69:13:57,  7.29s/it]
 85%|████████▍ | 190835/225000 [360:24:29<88:54:25,  9.37s/it]
 85%|████████▍ | 190836/225000 [360:24:34<75:57:25,  8.00s/it]
 85%|████████▍ | 190837/225000 [360:24:42<77:37:48,  8.18s/it]
 85%|████████▍ | 190838/225000 [360:24:49<73:58:39,  7.80s/it]
 85%|████████▍ | 190839/225000 [360:25:01<86:16:23,  9.09s/it]
 85%|████████▍ | 190840/225000 [360:25:06<73:06:10,  7.70s/it]
 85%|████████▍ | 190841/225000 [360:25:12<68:44:25,  7.24s/it]
 85%|████████▍ | 190842/225000 [360:25:18<65:10:46,  6.87s/it]
 85%|████████▍ | 190843/225000 [360:25:25<64:42:06,  6.82s/it]
 85%|████████▍ | 190844/225000 [360:25:35<75:04:40,  7.91s/it]
 85%|████████▍ | 190845/225000 [360:25:43<76:16:22,  8.04s/it]
 85%|████████▍ | 190846/225000 [360:25:50<72:19:24,  7.62s/it]
 85%|████████▍ | 190847/225000 [360:25:55<65:05:05,  6.86s/it]
 85%|████████▍ | 190848/225000 [360:26:01<62:01:55,  6.54s/it]
 85%|████████▍ | 190849/225000 [360:26:14<80:02:48,  8.44s/it]
 85%|████████▍ | 190850/225000 [360:26:19<70:39:33,  7.45s/it]
                                                              
{'loss': '1.234', 'grad_norm': '1.122', 'learning_rate': '3.064e-06', 'epoch': '0.8482', 'num_input_tokens_seen': 599313376, 'train_runtime': '1.298e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 190850/225000 [360:26:19<70:39:33,  7.45s/it]
 85%|████████▍ | 190851/225000 [360:26:25<65:56:03,  6.95s/it]
 85%|████████▍ | 190852/225000 [360:26:33<69:58:36,  7.38s/it]
 85%|████████▍ | 190853/225000 [360:26:43<76:59:54,  8.12s/it]
 85%|████████▍ | 190854/225000 [360:26:48<68:21:46,  7.21s/it]
 85%|████████▍ | 190855/225000 [360:26:56<69:39:17,  7.34s/it]
 85%|████████▍ | 190856/225000 [360:27:05<74:33:09,  7.86s/it]
 85%|████████▍ | 190857/225000 [360:27:14<79:59:03,  8.43s/it]
 85%|████████▍ | 190858/225000 [360:27:22<77:04:41,  8.13s/it]
 85%|████████▍ | 190859/225000 [360:27:29<73:00:33,  7.70s/it]
 85%|████████▍ | 190860/225000 [360:27:35<69:33:39,  7.34s/it]
 85%|████████▍ | 190861/225000 [360:27:43<70:23:16,  7.42s/it]
 85%|████████▍ | 190862/225000 [360:27:50<69:21:48,  7.31s/it]
 85%|████████▍ | 190863/225000 [360:27:56<67:47:58,  7.15s/it]
 85%|████████▍ | 190864/225000 [360:28:06<75:04:11,  7.92s/it]
 85%|████████▍ | 190865/225000 [360:28:19<88:33:30,  9.34s/it]
 85%|████████▍ | 190866/225000 [360:28:29<92:10:14,  9.72s/it]
 85%|████████▍ | 190867/225000 [360:28:34<76:21:18,  8.05s/it]
 85%|████████▍ | 190868/225000 [360:28:44<83:59:58,  8.86s/it]
 85%|████████▍ | 190869/225000 [360:28:54<85:14:21,  8.99s/it]
 85%|████████▍ | 190870/225000 [360:28:59<75:09:44,  7.93s/it]
 85%|████████▍ | 190871/225000 [360:29:09<80:09:54,  8.46s/it]
 85%|████████▍ | 190872/225000 [360:29:22<92:14:30,  9.73s/it]
 85%|████████▍ | 190873/225000 [360:29:31<90:26:44,  9.54s/it]
 85%|████████▍ | 190874/225000 [360:29:39<87:21:02,  9.21s/it]
 85%|████████▍ | 190875/225000 [360:29:45<76:41:29,  8.09s/it]
 85%|████████▍ | 190876/225000 [360:29:51<71:06:07,  7.50s/it]
 85%|████████▍ | 190877/225000 [360:30:03<85:23:01,  9.01s/it]
 85%|████████▍ | 190878/225000 [360:30:10<78:43:56,  8.31s/it]
 85%|████████▍ | 190879/225000 [360:30:17<75:46:43,  8.00s/it]
 85%|████████▍ | 190880/225000 [360:30:26<78:06:30,  8.24s/it]
 85%|████████▍ | 190881/225000 [360:30:31<68:42:32,  7.25s/it]
 85%|████████▍ | 190882/225000 [360:30:41<76:55:27,  8.12s/it]
 85%|████████▍ | 190883/225000 [360:30:48<74:52:38,  7.90s/it]
 85%|████████▍ | 190884/225000 [360:30:56<74:57:55,  7.91s/it]
 85%|████████▍ | 190885/225000 [360:31:02<67:23:43,  7.11s/it]
 85%|████████▍ | 190886/225000 [360:31:15<85:55:08,  9.07s/it]
 85%|████████▍ | 190887/225000 [360:31:21<77:34:50,  8.19s/it]
 85%|████████▍ | 190888/225000 [360:31:30<79:48:34,  8.42s/it]
 85%|████████▍ | 190889/225000 [360:31:35<67:51:29,  7.16s/it]
 85%|████████▍ | 190890/225000 [360:31:43<72:50:56,  7.69s/it]
 85%|████████▍ | 190891/225000 [360:31:49<67:59:56,  7.18s/it]
 85%|████████▍ | 190892/225000 [360:31:56<66:00:28,  6.97s/it]
 85%|████████▍ | 190893/225000 [360:32:03<66:49:40,  7.05s/it]
 85%|████████▍ | 190894/225000 [360:32:12<70:28:07,  7.44s/it]
 85%|████████▍ | 190895/225000 [360:32:25<87:58:59,  9.29s/it]
 85%|████████▍ | 190896/225000 [360:32:33<84:53:19,  8.96s/it]
 85%|████████▍ | 190897/225000 [360:32:40<77:52:14,  8.22s/it]
 85%|████████▍ | 190898/225000 [360:32:45<68:52:15,  7.27s/it]
 85%|████████▍ | 190899/225000 [360:32:52<67:10:53,  7.09s/it]
 85%|████████▍ | 190900/225000 [360:32:57<63:42:47,  6.73s/it]
                                                              
{'loss': '1.232', 'grad_norm': '1.412', 'learning_rate': '3.056e-06', 'epoch': '0.8484', 'num_input_tokens_seen': 599522904, 'train_runtime': '1.298e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 190900/225000 [360:32:57<63:42:47,  6.73s/it]
 85%|████████▍ | 190901/225000 [360:33:12<84:42:31,  8.94s/it]
 85%|████████▍ | 190902/225000 [360:33:16<71:47:01,  7.58s/it]
 85%|████████▍ | 190903/225000 [360:33:27<82:44:55,  8.74s/it]
 85%|████████▍ | 190904/225000 [360:33:38<88:52:50,  9.38s/it]
 85%|████████▍ | 190905/225000 [360:33:47<86:35:19,  9.14s/it]
 85%|████████▍ | 190906/225000 [360:33:53<77:51:04,  8.22s/it]
 85%|████████▍ | 190907/225000 [360:33:58<67:53:42,  7.17s/it]
 85%|████████▍ | 190908/225000 [360:34:09<80:48:36,  8.53s/it]
 85%|████████▍ | 190909/225000 [360:34:15<71:54:37,  7.59s/it]
 85%|████████▍ | 190910/225000 [360:34:20<64:11:18,  6.78s/it]
 85%|████████▍ | 190911/225000 [360:34:31<77:31:22,  8.19s/it]
 85%|████████▍ | 190912/225000 [360:34:36<69:23:36,  7.33s/it]
 85%|████████▍ | 190913/225000 [360:34:43<68:24:19,  7.22s/it]
 85%|████████▍ | 190914/225000 [360:34:50<65:39:13,  6.93s/it]
 85%|████████▍ | 190915/225000 [360:35:00<76:11:42,  8.05s/it]
 85%|████████▍ | 190916/225000 [360:35:08<74:02:18,  7.82s/it]
 85%|████████▍ | 190917/225000 [360:35:14<69:28:09,  7.34s/it]
 85%|████████▍ | 190918/225000 [360:35:20<66:39:13,  7.04s/it]
 85%|████████▍ | 190919/225000 [360:35:26<63:20:17,  6.69s/it]
 85%|████████▍ | 190920/225000 [360:35:36<71:37:09,  7.57s/it]
 85%|████████▍ | 190921/225000 [360:35:47<81:43:13,  8.63s/it]
 85%|████████▍ | 190922/225000 [360:35:56<84:31:46,  8.93s/it]
 85%|████████▍ | 190923/225000 [360:36:08<91:36:30,  9.68s/it]
 85%|████████▍ | 190924/225000 [360:36:15<84:27:07,  8.92s/it]
 85%|████████▍ | 190925/225000 [360:36:20<72:27:10,  7.65s/it]
 85%|████████▍ | 190926/225000 [360:36:28<75:31:08,  7.98s/it]
 85%|████████▍ | 190927/225000 [360:36:36<75:28:13,  7.97s/it]
 85%|████████▍ | 190928/225000 [360:36:41<67:22:19,  7.12s/it]
 85%|████████▍ | 190929/225000 [360:36:52<77:54:46,  8.23s/it]
 85%|████████▍ | 190930/225000 [360:36:57<68:42:06,  7.26s/it]
 85%|████████▍ | 190931/225000 [360:37:05<68:44:54,  7.26s/it]
 85%|████████▍ | 190932/225000 [360:37:09<60:38:17,  6.41s/it]
 85%|████████▍ | 190933/225000 [360:37:19<71:32:42,  7.56s/it]
 85%|████████▍ | 190934/225000 [360:37:24<64:34:23,  6.82s/it]
 85%|████████▍ | 190935/225000 [360:37:31<64:47:38,  6.85s/it]
 85%|████████▍ | 190936/225000 [360:37:39<67:51:35,  7.17s/it]
 85%|████████▍ | 190937/225000 [360:37:53<86:25:20,  9.13s/it]
 85%|████████▍ | 190938/225000 [360:38:02<85:46:46,  9.07s/it]
 85%|████████▍ | 190939/225000 [360:38:08<77:07:54,  8.15s/it]
 85%|████████▍ | 190940/225000 [360:38:24<98:35:33, 10.42s/it]
 85%|████████▍ | 190941/225000 [360:38:32<92:24:31,  9.77s/it]
 85%|████████▍ | 190942/225000 [360:38:41<90:21:07,  9.55s/it]
 85%|████████▍ | 190943/225000 [360:38:46<77:55:24,  8.24s/it]
 85%|████████▍ | 190944/225000 [360:38:53<75:22:26,  7.97s/it]
 85%|████████▍ | 190945/225000 [360:39:01<75:51:59,  8.02s/it]
 85%|████████▍ | 190946/225000 [360:39:16<93:52:09,  9.92s/it]
 85%|████████▍ | 190947/225000 [360:39:25<91:45:21,  9.70s/it]
 85%|████████▍ | 190948/225000 [360:39:30<77:31:58,  8.20s/it]
 85%|████████▍ | 190949/225000 [360:39:39<79:26:53,  8.40s/it]
 85%|████████▍ | 190950/225000 [360:39:49<84:31:12,  8.94s/it]
                                                              
{'loss': '1.243', 'grad_norm': '1.435', 'learning_rate': '3.047e-06', 'epoch': '0.8487', 'num_input_tokens_seen': 599734576, 'train_runtime': '1.298e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 190950/225000 [360:39:49<84:31:12,  8.94s/it]
 85%|████████▍ | 190951/225000 [360:39:56<78:38:26,  8.31s/it]
 85%|████████▍ | 190952/225000 [360:40:01<71:24:25,  7.55s/it]
 85%|████████▍ | 190953/225000 [360:40:10<75:19:40,  7.96s/it]
 85%|████████▍ | 190954/225000 [360:40:17<71:03:03,  7.51s/it]
 85%|████████▍ | 190955/225000 [360:40:27<78:59:46,  8.35s/it]
 85%|████████▍ | 190956/225000 [360:40:32<70:07:23,  7.42s/it]
 85%|████████▍ | 190957/225000 [360:40:37<62:15:54,  6.58s/it]
 85%|████████▍ | 190958/225000 [360:40:45<65:10:13,  6.89s/it]
 85%|████████▍ | 190959/225000 [360:40:57<80:03:29,  8.47s/it]
 85%|████████▍ | 190960/225000 [360:41:07<84:09:59,  8.90s/it]
 85%|████████▍ | 190961/225000 [360:41:11<72:19:07,  7.65s/it]
 85%|████████▍ | 190962/225000 [360:41:19<73:29:08,  7.77s/it]
 85%|████████▍ | 190963/225000 [360:41:23<63:02:58,  6.67s/it]
 85%|████████▍ | 190964/225000 [360:41:32<69:37:48,  7.36s/it]
 85%|████████▍ | 190965/225000 [360:41:37<62:55:04,  6.66s/it]
 85%|████████▍ | 190966/225000 [360:41:43<60:21:13,  6.38s/it]
 85%|████████▍ | 190967/225000 [360:41:51<65:13:24,  6.90s/it]
 85%|████████▍ | 190968/225000 [360:42:02<75:36:51,  8.00s/it]
 85%|████████▍ | 190969/225000 [360:42:11<80:02:08,  8.47s/it]
 85%|████████▍ | 190970/225000 [360:42:18<75:54:50,  8.03s/it]
 85%|████████▍ | 190971/225000 [360:42:28<79:33:09,  8.42s/it]
 85%|████████▍ | 190972/225000 [360:42:36<79:47:42,  8.44s/it]
 85%|████████▍ | 190973/225000 [360:42:45<79:30:38,  8.41s/it]
 85%|████████▍ | 190974/225000 [360:42:53<78:16:34,  8.28s/it]
 85%|████████▍ | 190975/225000 [360:42:57<68:09:38,  7.21s/it]
 85%|████████▍ | 190976/225000 [360:43:08<77:54:11,  8.24s/it]
 85%|████████▍ | 190977/225000 [360:43:13<69:32:21,  7.36s/it]
 85%|████████▍ | 190978/225000 [360:43:20<68:27:15,  7.24s/it]
 85%|████████▍ | 190979/225000 [360:43:29<71:19:43,  7.55s/it]
 85%|████████▍ | 190980/225000 [360:43:35<69:28:36,  7.35s/it]
 85%|████████▍ | 190981/225000 [360:43:42<66:51:50,  7.08s/it]
 85%|████████▍ | 190982/225000 [360:43:54<80:41:59,  8.54s/it]
 85%|████████▍ | 190983/225000 [360:44:01<78:06:19,  8.27s/it]
 85%|████████▍ | 190984/225000 [360:44:10<79:42:25,  8.44s/it]
 85%|████████▍ | 190985/225000 [360:44:15<69:27:53,  7.35s/it]
 85%|████████▍ | 190986/225000 [360:44:21<64:21:30,  6.81s/it]
 85%|████████▍ | 190987/225000 [360:44:28<66:45:32,  7.07s/it]
 85%|████████▍ | 190988/225000 [360:44:38<74:44:07,  7.91s/it]
 85%|████████▍ | 190989/225000 [360:44:42<64:17:22,  6.80s/it]
 85%|████████▍ | 190990/225000 [360:44:53<74:31:54,  7.89s/it]
 85%|████████▍ | 190991/225000 [360:45:02<77:25:28,  8.20s/it]
 85%|████████▍ | 190992/225000 [360:45:10<76:34:35,  8.11s/it]
 85%|████████▍ | 190993/225000 [360:45:22<87:35:34,  9.27s/it]
 85%|████████▍ | 190994/225000 [360:45:29<83:10:10,  8.80s/it]
 85%|████████▍ | 190995/225000 [360:45:43<97:27:44, 10.32s/it]
 85%|████████▍ | 190996/225000 [360:45:50<87:27:15,  9.26s/it]
 85%|████████▍ | 190997/225000 [360:45:55<76:37:49,  8.11s/it]
 85%|████████▍ | 190998/225000 [360:46:02<71:10:54,  7.54s/it]
 85%|████████▍ | 190999/225000 [360:46:15<87:09:58,  9.23s/it]
 85%|████████▍ | 191000/225000 [360:46:21<79:32:00,  8.42s/it]
                                                              
{'loss': '1.233', 'grad_norm': '1.064', 'learning_rate': '3.038e-06', 'epoch': '0.8489', 'num_input_tokens_seen': 599938576, 'train_runtime': '1.299e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 191000/225000 [360:46:21<79:32:00,  8.42s/it]
 85%|████████▍ | 191001/225000 [360:46:33<88:02:17,  9.32s/it]
 85%|████████▍ | 191002/225000 [360:46:38<77:24:43,  8.20s/it]
 85%|████████▍ | 191003/225000 [360:46:47<78:28:52,  8.31s/it]
 85%|████████▍ | 191004/225000 [360:46:56<79:44:44,  8.44s/it]
 85%|████████▍ | 191005/225000 [360:47:04<80:08:56,  8.49s/it]
 85%|████████▍ | 191006/225000 [360:47:12<77:39:40,  8.22s/it]
 85%|████████▍ | 191007/225000 [360:47:21<80:13:28,  8.50s/it]
 85%|████████▍ | 191008/225000 [360:47:28<74:55:18,  7.93s/it]
 85%|████████▍ | 191009/225000 [360:47:35<74:41:59,  7.91s/it]
 85%|████████▍ | 191010/225000 [360:47:42<71:35:33,  7.58s/it]
 85%|████████▍ | 191011/225000 [360:47:48<66:13:49,  7.01s/it]
 85%|████████▍ | 191012/225000 [360:47:55<65:49:09,  6.97s/it]
 85%|████████▍ | 191013/225000 [360:48:06<78:52:16,  8.35s/it]
 85%|████████▍ | 191014/225000 [360:48:10<64:25:28,  6.82s/it]
 85%|████████▍ | 191015/225000 [360:48:17<65:12:44,  6.91s/it]
 85%|████████▍ | 191016/225000 [360:48:24<66:08:03,  7.01s/it]
 85%|████████▍ | 191017/225000 [360:48:30<64:42:57,  6.86s/it]
 85%|████████▍ | 191018/225000 [360:48:37<64:58:02,  6.88s/it]
 85%|████████▍ | 191019/225000 [360:48:43<60:57:09,  6.46s/it]
 85%|████████▍ | 191020/225000 [360:48:51<64:28:01,  6.83s/it]
 85%|████████▍ | 191021/225000 [360:48:56<59:23:21,  6.29s/it]
 85%|████████▍ | 191022/225000 [360:49:06<70:14:58,  7.44s/it]
 85%|████████▍ | 191023/225000 [360:49:14<72:31:26,  7.68s/it]
 85%|████████▍ | 191024/225000 [360:49:21<71:21:25,  7.56s/it]
 85%|████████▍ | 191025/225000 [360:49:28<69:14:43,  7.34s/it]
 85%|████████▍ | 191026/225000 [360:49:34<64:31:33,  6.84s/it]
 85%|████████▍ | 191027/225000 [360:49:41<66:01:04,  7.00s/it]
 85%|████████▍ | 191028/225000 [360:49:57<90:40:42,  9.61s/it]
 85%|████████▍ | 191029/225000 [360:50:09<98:06:22, 10.40s/it]
 85%|████████▍ | 191030/225000 [360:50:17<92:25:48,  9.80s/it]
 85%|████████▍ | 191031/225000 [360:50:25<85:22:07,  9.05s/it]
 85%|████████▍ | 191032/225000 [360:50:29<72:08:01,  7.64s/it]
 85%|████████▍ | 191033/225000 [360:50:38<76:46:36,  8.14s/it]
 85%|████████▍ | 191034/225000 [360:50:45<72:08:39,  7.65s/it]
 85%|████████▍ | 191035/225000 [360:50:51<68:15:25,  7.23s/it]
 85%|████████▍ | 191036/225000 [360:51:02<78:10:31,  8.29s/it]
 85%|████████▍ | 191037/225000 [360:51:11<80:05:25,  8.49s/it]
 85%|████████▍ | 191038/225000 [360:51:17<72:23:03,  7.67s/it]
 85%|████████▍ | 191039/225000 [360:51:26<77:35:46,  8.23s/it]
 85%|████████▍ | 191040/225000 [360:51:33<73:24:36,  7.78s/it]
 85%|████████▍ | 191041/225000 [360:51:39<68:21:50,  7.25s/it]
 85%|████████▍ | 191042/225000 [360:51:53<89:02:03,  9.44s/it]
 85%|████████▍ | 191043/225000 [360:52:02<87:07:08,  9.24s/it]
 85%|████████▍ | 191044/225000 [360:52:12<87:59:59,  9.33s/it]
 85%|████████▍ | 191045/225000 [360:52:18<79:23:38,  8.42s/it]
 85%|████████▍ | 191046/225000 [360:52:31<91:43:59,  9.73s/it]
 85%|████████▍ | 191047/225000 [360:52:41<93:26:37,  9.91s/it]
 85%|████████▍ | 191048/225000 [360:52:47<81:52:43,  8.68s/it]
 85%|████████▍ | 191049/225000 [360:52:58<88:55:41,  9.43s/it]
 85%|████████▍ | 191050/225000 [360:53:04<79:03:12,  8.38s/it]
                                                              
{'loss': '1.224', 'grad_norm': '1.236', 'learning_rate': '3.029e-06', 'epoch': '0.8491', 'num_input_tokens_seen': 600147888, 'train_runtime': '1.299e+06', 'train_tokens_per_second': '461.9'}

 85%|████████▍ | 191050/225000 [360:53:04<79:03:12,  8.38s/it]
 85%|████████▍ | 191051/225000 [360:53:11<75:32:21,  8.01s/it]
 85%|████████▍ | 191052/225000 [360:53:24<90:07:00,  9.56s/it]
 85%|████████▍ | 191053/225000 [360:53:31<81:37:31,  8.66s/it]
 85%|████████▍ | 191054/225000 [360:53:37<73:47:18,  7.83s/it]
 85%|████████▍ | 191055/225000 [360:53:42<64:44:49,  6.87s/it]
 85%|████████▍ | 191056/225000 [360:53:47<60:49:05,  6.45s/it]
 85%|████████▍ | 191057/225000 [360:53:53<60:26:56,  6.41s/it]
 85%|████████▍ | 191058/225000 [360:54:07<79:47:31,  8.46s/it]
 85%|████████▍ | 191059/225000 [360:54:12<70:55:58,  7.52s/it]
 85%|████████▍ | 191060/225000 [360:54:18<66:57:06,  7.10s/it]
 85%|████████▍ | 191061/225000 [360:54:30<79:41:00,  8.45s/it]
 85%|████████▍ | 191062/225000 [360:54:41<88:08:30,  9.35s/it]
 85%|████████▍ | 191063/225000 [360:54:47<77:41:54,  8.24s/it]
 85%|████████▍ | 191064/225000 [360:54:53<73:03:55,  7.75s/it]
 85%|████████▍ | 191065/225000 [360:54:59<66:22:06,  7.04s/it]
 85%|████████▍ | 191066/225000 [360:55:04<60:35:09,  6.43s/it]
 85%|████████▍ | 191067/225000 [360:55:12<65:01:33,  6.90s/it]
 85%|████████▍ | 191068/225000 [360:55:22<75:57:05,  8.06s/it]
 85%|████████▍ | 191069/225000 [360:55:33<82:28:37,  8.75s/it]
 85%|████████▍ | 191070/225000 [360:55:37<70:02:27,  7.43s/it]
 85%|████████▍ | 191071/225000 [360:55:42<63:00:31,  6.69s/it]
 85%|████████▍ | 191072/225000 [360:55:55<80:45:15,  8.57s/it]
 85%|████████▍ | 191073/225000 [360:56:01<73:18:18,  7.78s/it]
 85%|████████▍ | 191074/225000 [360:56:08<71:44:47,  7.61s/it]
 85%|████████▍ | 191075/225000 [360:56:15<68:12:06,  7.24s/it]
 85%|████████▍ | 191076/225000 [360:56:25<78:01:32,  8.28s/it]
 85%|████████▍ | 191077/225000 [360:56:32<72:58:13,  7.74s/it]
 85%|████████▍ | 191078/225000 [360:56:42<81:09:06,  8.61s/it]
 85%|████████▍ | 191079/225000 [360:56:48<72:35:11,  7.70s/it]
 85%|████████▍ | 191080/225000 [360:56:53<64:35:25,  6.86s/it]
 85%|████████▍ | 191081/225000 [360:57:00<64:38:16,  6.86s/it]
 85%|████████▍ | 191082/225000 [360:57:05<59:16:50,  6.29s/it]
 85%|████████▍ | 191083/225000 [360:57:21<88:00:20,  9.34s/it]
 85%|████████▍ | 191084/225000 [360:57:29<83:52:10,  8.90s/it]
 85%|████████▍ | 191085/225000 [360:57:37<82:18:19,  8.74s/it]
 85%|████████▍ | 191086/225000 [360:57:41<68:46:16,  7.30s/it]
 85%|████████▍ | 191087/225000 [360:57:50<71:11:09,  7.56s/it]
 85%|████████▍ | 191088/225000 [360:57:57<72:01:41,  7.65s/it]
 85%|████████▍ | 191089/225000 [360:58:04<70:07:00,  7.44s/it]
 85%|████████▍ | 191090/225000 [360:58:09<62:54:44,  6.68s/it]
 85%|████████▍ | 191091/225000 [360:58:19<72:36:42,  7.71s/it]
 85%|████████▍ | 191092/225000 [360:58:29<79:11:51,  8.41s/it]
 85%|████████▍ | 191093/225000 [360:58:38<78:20:59,  8.32s/it]
 85%|████████▍ | 191094/225000 [360:58:42<67:23:27,  7.16s/it]
 85%|████████▍ | 191095/225000 [360:58:53<77:51:35,  8.27s/it]
 85%|████████▍ | 191096/225000 [360:58:57<66:10:39,  7.03s/it]
 85%|████████▍ | 191097/225000 [360:59:06<71:06:24,  7.55s/it]
 85%|████████▍ | 191098/225000 [360:59:13<71:41:11,  7.61s/it]
 85%|████████▍ | 191099/225000 [360:59:23<78:03:26,  8.29s/it]
 85%|████████▍ | 191100/225000 [360:59:35<87:02:27,  9.24s/it]
                                                              
{'loss': '1.217', 'grad_norm': '1.145', 'learning_rate': '3.021e-06', 'epoch': '0.8493', 'num_input_tokens_seen': 600350296, 'train_runtime': '1.3e+06', 'train_tokens_per_second': '462'}

 85%|████████▍ | 191100/225000 [360:59:35<87:02:27,  9.24s/it]
 85%|████████▍ | 191101/225000 [360:59:40<74:21:45,  7.90s/it]
 85%|████████▍ | 191102/225000 [360:59:50<80:36:30,  8.56s/it]
 85%|████████▍ | 191103/225000 [360:59:59<82:34:04,  8.77s/it]
 85%|████████▍ | 191104/225000 [361:00:10<88:42:26,  9.42s/it]
 85%|████████▍ | 191105/225000 [361:00:18<86:16:40,  9.16s/it]
 85%|████████▍ | 191106/225000 [361:00:24<74:56:25,  7.96s/it]
 85%|████████▍ | 191107/225000 [361:00:32<76:48:05,  8.16s/it]
 85%|████████▍ | 191108/225000 [361:00:40<75:53:16,  8.06s/it]
 85%|████████▍ | 191109/225000 [361:00:44<65:25:19,  6.95s/it]
 85%|████████▍ | 191110/225000 [361:00:51<64:00:46,  6.80s/it]
 85%|████████▍ | 191111/225000 [361:01:03<78:06:56,  8.30s/it]
 85%|████████▍ | 191112/225000 [361:01:11<76:58:44,  8.18s/it]
 85%|████████▍ | 191113/225000 [361:01:17<72:18:05,  7.68s/it]
 85%|████████▍ | 191114/225000 [361:01:23<66:58:53,  7.12s/it]
 85%|████████▍ | 191115/225000 [361:01:30<66:50:07,  7.10s/it]
 85%|████████▍ | 191116/225000 [361:01:44<85:04:56,  9.04s/it]
 85%|████████▍ | 191117/225000 [361:01:48<72:51:43,  7.74s/it]
 85%|████████▍ | 191118/225000 [361:01:59<82:21:07,  8.75s/it]
 85%|████████▍ | 191119/225000 [361:02:05<73:20:35,  7.79s/it]
 85%|████████▍ | 191120/225000 [361:02:09<62:42:30,  6.66s/it]
 85%|████████▍ | 191121/225000 [361:02:22<79:49:47,  8.48s/it]
 85%|████████▍ | 191122/225000 [361:02:29<76:39:22,  8.15s/it]
 85%|████████▍ | 191123/225000 [361:02:36<72:27:05,  7.70s/it]
 85%|████████▍ | 191124/225000 [361:02:46<80:13:50,  8.53s/it]
 85%|████████▍ | 191125/225000 [361:02:52<72:23:52,  7.69s/it]
 85%|████████▍ | 191126/225000 [361:02:58<67:05:15,  7.13s/it]
 85%|████████▍ | 191127/225000 [361:03:05<67:01:39,  7.12s/it]
 85%|████████▍ | 191128/225000 [361:03:16<77:31:04,  8.24s/it]
 85%|████████▍ | 191129/225000 [361:03:25<80:07:42,  8.52s/it]
 85%|████████▍ | 191130/225000 [361:03:35<85:21:26,  9.07s/it]
 85%|████████▍ | 191131/225000 [361:03:40<72:34:48,  7.71s/it]
 85%|████████▍ | 191132/225000 [361:03:46<69:24:49,  7.38s/it]
 85%|████████▍ | 191133/225000 [361:03:53<68:41:59,  7.30s/it]
 85%|████████▍ | 191134/225000 [361:03:59<64:18:45,  6.84s/it]
 85%|████████▍ | 191135/225000 [361:04:10<76:03:47,  8.09s/it]
 85%|████████▍ | 191136/225000 [361:04:18<74:15:24,  7.89s/it]
 85%|████████▍ | 191137/225000 [361:04:28<80:07:03,  8.52s/it]
 85%|████████▍ | 191138/225000 [361:04:40<90:18:00,  9.60s/it]
 85%|████████▍ | 191139/225000 [361:04:49<89:35:35,  9.53s/it]
 85%|████████▍ | 191140/225000 [361:04:55<79:40:38,  8.47s/it]
 85%|████████▍ | 191141/225000 [361:05:03<77:04:33,  8.19s/it]
 85%|████████▍ | 191142/225000 [361:05:09<71:51:42,  7.64s/it]
 85%|████████▍ | 191143/225000 [361:05:20<80:48:16,  8.59s/it]
 85%|████████▍ | 191144/225000 [361:05:27<76:34:43,  8.14s/it]
 85%|████████▍ | 191145/225000 [361:05:34<74:41:05,  7.94s/it]
 85%|████████▍ | 191146/225000 [361:05:44<78:57:50,  8.40s/it]
 85%|████████▍ | 191147/225000 [361:05:53<82:02:47,  8.73s/it]
 85%|████████▍ | 191148/225000 [361:06:04<87:24:17,  9.30s/it]
 85%|████████▍ | 191149/225000 [361:06:15<92:18:32,  9.82s/it]
 85%|████████▍ | 191150/225000 [361:06:23<87:07:17,  9.27s/it]
                                                              
{'loss': '1.244', 'grad_norm': '1.517', 'learning_rate': '3.012e-06', 'epoch': '0.8496', 'num_input_tokens_seen': 600560808, 'train_runtime': '1.3e+06', 'train_tokens_per_second': '462'}

 85%|████████▍ | 191150/225000 [361:06:23<87:07:17,  9.27s/it]
 85%|████████▍ | 191151/225000 [361:06:29<77:30:18,  8.24s/it]
 85%|████████▍ | 191152/225000 [361:06:39<82:51:50,  8.81s/it]
 85%|████████▍ | 191153/225000 [361:06:47<81:20:46,  8.65s/it]
 85%|████████▍ | 191154/225000 [361:06:56<82:57:29,  8.82s/it]
 85%|████████▍ | 191155/225000 [361:07:07<86:58:18,  9.25s/it]
 85%|████████▍ | 191156/225000 [361:07:14<81:45:10,  8.70s/it]
 85%|████████▍ | 191157/225000 [361:07:19<71:24:46,  7.60s/it]
 85%|████████▍ | 191158/225000 [361:07:25<65:51:40,  7.01s/it]
 85%|████████▍ | 191159/225000 [361:07:36<76:35:38,  8.15s/it]
 85%|████████▍ | 191160/225000 [361:07:43<75:13:43,  8.00s/it]
 85%|████████▍ | 191161/225000 [361:07:49<70:09:52,  7.46s/it]
 85%|████████▍ | 191162/225000 [361:08:02<85:27:54,  9.09s/it]
 85%|████████▍ | 191163/225000 [361:08:12<85:56:45,  9.14s/it]
 85%|████████▍ | 191164/225000 [361:08:23<93:02:01,  9.90s/it]
 85%|████████▍ | 191165/225000 [361:08:34<95:05:58, 10.12s/it]
 85%|████████▍ | 191166/225000 [361:08:38<78:46:38,  8.38s/it]
 85%|████████▍ | 191167/225000 [361:08:48<82:30:55,  8.78s/it]
 85%|████████▍ | 191168/225000 [361:08:57<82:40:31,  8.80s/it]
 85%|████████▍ | 191169/225000 [361:09:05<79:56:06,  8.51s/it]
 85%|████████▍ | 191170/225000 [361:09:10<70:46:47,  7.53s/it]
 85%|████████▍ | 191171/225000 [361:09:21<80:18:20,  8.55s/it]
 85%|████████▍ | 191172/225000 [361:09:32<89:16:09,  9.50s/it]
 85%|████████▍ | 191173/225000 [361:09:39<80:18:43,  8.55s/it]
 85%|████████▍ | 191174/225000 [361:09:46<75:36:45,  8.05s/it]
 85%|████████▍ | 191175/225000 [361:09:50<65:25:08,  6.96s/it]
 85%|████████▍ | 191176/225000 [361:09:56<63:31:58,  6.76s/it]
 85%|████████▍ | 191177/225000 [361:10:04<65:06:26,  6.93s/it]
 85%|████████▍ | 191178/225000 [361:10:10<63:46:55,  6.79s/it]
 85%|████████▍ | 191179/225000 [361:10:17<62:51:59,  6.69s/it]
 85%|████████▍ | 191180/225000 [361:10:29<79:08:42,  8.42s/it]
 85%|████████▍ | 191181/225000 [361:10:36<73:27:49,  7.82s/it]
 85%|████████▍ | 191182/225000 [361:10:47<84:37:41,  9.01s/it]
 85%|████████▍ | 191183/225000 [361:10:53<76:12:43,  8.11s/it]
 85%|████████▍ | 191184/225000 [361:11:01<73:53:31,  7.87s/it]
 85%|████████▍ | 191185/225000 [361:11:12<84:12:09,  8.96s/it]
 85%|████████▍ | 191186/225000 [361:11:23<90:38:33,  9.65s/it]
 85%|████████▍ | 191187/225000 [361:11:35<95:18:52, 10.15s/it]
 85%|████████▍ | 191188/225000 [361:11:42<86:46:20,  9.24s/it]
 85%|████████▍ | 191189/225000 [361:11:49<80:31:01,  8.57s/it]
 85%|████████▍ | 191190/225000 [361:11:55<73:00:30,  7.77s/it]
 85%|████████▍ | 191191/225000 [361:12:05<80:40:44,  8.59s/it]
 85%|████████▍ | 191192/225000 [361:12:13<79:07:14,  8.43s/it]
 85%|████████▍ | 191193/225000 [361:12:19<71:47:40,  7.65s/it]
 85%|████████▍ | 191194/225000 [361:12:31<83:50:48,  8.93s/it]
 85%|████████▍ | 191195/225000 [361:12:35<69:20:26,  7.38s/it]
 85%|████████▍ | 191196/225000 [361:12:42<68:40:08,  7.31s/it]
 85%|████████▍ | 191197/225000 [361:12:52<75:11:40,  8.01s/it]
 85%|████████▍ | 191198/225000 [361:12:57<68:41:52,  7.32s/it]
 85%|████████▍ | 191199/225000 [361:13:07<73:56:46,  7.88s/it]
 85%|████████▍ | 191200/225000 [361:13:14<73:28:49,  7.83s/it]
                                                              
{'loss': '1.235', 'grad_norm': '1.432', 'learning_rate': '3.003e-06', 'epoch': '0.8498', 'num_input_tokens_seen': 600771384, 'train_runtime': '1.3e+06', 'train_tokens_per_second': '462'}

 85%|████████▍ | 191200/225000 [361:13:14<73:28:49,  7.83s/it]
 85%|████████▍ | 191201/225000 [361:13:21<70:32:55,  7.51s/it]
 85%|████████▍ | 191202/225000 [361:13:26<63:52:04,  6.80s/it]
 85%|████████▍ | 191203/225000 [361:13:38<78:05:08,  8.32s/it]
 85%|████████▍ | 191204/225000 [361:13:46<78:03:51,  8.32s/it]
 85%|████████▍ | 191205/225000 [361:13:55<79:57:31,  8.52s/it]
 85%|████████▍ | 191206/225000 [361:14:02<74:47:17,  7.97s/it]
 85%|████████▍ | 191207/225000 [361:14:08<70:03:39,  7.46s/it]
 85%|████████▍ | 191208/225000 [361:14:15<67:09:08,  7.15s/it]
 85%|████████▍ | 191209/225000 [361:14:23<69:29:40,  7.40s/it]
 85%|████████▍ | 191210/225000 [361:14:29<67:02:50,  7.14s/it]
 85%|████████▍ | 191211/225000 [361:14:42<84:14:38,  8.98s/it]
 85%|████████▍ | 191212/225000 [361:14:50<80:21:19,  8.56s/it]
 85%|████████▍ | 191213/225000 [361:14:56<71:54:34,  7.66s/it]
 85%|████████▍ | 191214/225000 [361:15:02<68:06:15,  7.26s/it]
 85%|████████▍ | 191215/225000 [361:15:12<74:49:25,  7.97s/it]
 85%|████████▍ | 191216/225000 [361:15:25<89:35:08,  9.55s/it]
 85%|████████▍ | 191217/225000 [361:15:29<75:15:30,  8.02s/it]
 85%|████████▍ | 191218/225000 [361:15:36<72:49:51,  7.76s/it]
 85%|████████▍ | 191219/225000 [361:15:42<66:17:38,  7.06s/it]
 85%|████████▍ | 191220/225000 [361:15:49<66:03:30,  7.04s/it]
 85%|████████▍ | 191221/225000 [361:15:55<64:29:22,  6.87s/it]
 85%|████████▍ | 191222/225000 [361:16:10<86:42:07,  9.24s/it]
 85%|████████▍ | 191223/225000 [361:16:21<91:04:58,  9.71s/it]
 85%|████████▍ | 191224/225000 [361:16:33<97:46:57, 10.42s/it]
 85%|████████▍ | 191225/225000 [361:16:39<86:05:02,  9.18s/it]
 85%|████████▍ | 191226/225000 [361:16:47<81:08:10,  8.65s/it]
 85%|████████▍ | 191227/225000 [361:17:02<98:41:54, 10.52s/it]
 85%|████████▍ | 191228/225000 [361:17:09<90:26:44,  9.64s/it]
 85%|████████▍ | 191229/225000 [361:17:17<86:02:23,  9.17s/it]
 85%|████████▍ | 191230/225000 [361:17:23<76:27:04,  8.15s/it]
 85%|████████▍ | 191231/225000 [361:17:28<68:48:24,  7.34s/it]
 85%|████████▍ | 191232/225000 [361:17:34<64:57:41,  6.93s/it]
 85%|████████▍ | 191233/225000 [361:17:42<65:33:45,  6.99s/it]
 85%|████████▍ | 191234/225000 [361:17:47<60:32:01,  6.45s/it]
 85%|████████▍ | 191235/225000 [361:17:55<64:28:24,  6.87s/it]
 85%|████████▍ | 191236/225000 [361:18:09<85:11:52,  9.08s/it]
 85%|████████▍ | 191237/225000 [361:18:15<77:51:23,  8.30s/it]
 85%|████████▍ | 191238/225000 [361:18:21<70:05:43,  7.47s/it]
 85%|████████▍ | 191239/225000 [361:18:31<78:19:43,  8.35s/it]
 85%|████████▍ | 191240/225000 [361:18:44<92:02:50,  9.82s/it]
 85%|████████▍ | 191241/225000 [361:18:54<91:40:59,  9.78s/it]
 85%|████████▍ | 191242/225000 [361:18:59<79:11:23,  8.44s/it]
 85%|████████▍ | 191243/225000 [361:19:06<73:26:24,  7.83s/it]
 85%|████████▍ | 191244/225000 [361:19:17<81:19:54,  8.67s/it]
 85%|████████▍ | 191245/225000 [361:19:26<83:12:08,  8.87s/it]
 85%|████████▍ | 191246/225000 [361:19:31<72:25:50,  7.73s/it]
 85%|████████▍ | 191247/225000 [361:19:38<70:50:05,  7.56s/it]
 85%|████████▍ | 191248/225000 [361:19:46<70:50:43,  7.56s/it]
 85%|████████▍ | 191249/225000 [361:19:50<60:44:59,  6.48s/it]
 85%|████████▌ | 191250/225000 [361:20:02<77:11:39,  8.23s/it]
                                                              
{'loss': '1.217', 'grad_norm': '1.097', 'learning_rate': '2.995e-06', 'epoch': '0.85', 'num_input_tokens_seen': 600980128, 'train_runtime': '1.301e+06', 'train_tokens_per_second': '462'}

 85%|████████▌ | 191250/225000 [361:20:02<77:11:39,  8.23s/it]
 85%|████████▌ | 191251/225000 [361:20:07<69:15:15,  7.39s/it]
 85%|████████▌ | 191252/225000 [361:20:14<66:58:15,  7.14s/it]
 85%|████████▌ | 191253/225000 [361:20:21<66:37:44,  7.11s/it]
 85%|████████▌ | 191254/225000 [361:20:33<80:48:39,  8.62s/it]
 85%|████████▌ | 191255/225000 [361:20:41<78:05:31,  8.33s/it]
 85%|████████▌ | 191256/225000 [361:20:49<77:22:24,  8.25s/it]
 85%|████████▌ | 191257/225000 [361:20:55<71:28:39,  7.63s/it]
 85%|████████▌ | 191258/225000 [361:21:00<64:26:25,  6.88s/it]
 85%|████████▌ | 191259/225000 [361:21:08<67:57:56,  7.25s/it]
 85%|████████▌ | 191260/225000 [361:21:15<65:41:20,  7.01s/it]
 85%|████████▌ | 191261/225000 [361:21:28<83:10:02,  8.87s/it]
 85%|████████▌ | 191262/225000 [361:21:32<70:25:36,  7.51s/it]
 85%|████████▌ | 191263/225000 [361:21:45<85:14:16,  9.10s/it]
 85%|████████▌ | 191264/225000 [361:21:51<77:22:36,  8.26s/it]
 85%|████████▌ | 191265/225000 [361:21:56<66:42:41,  7.12s/it]
 85%|████████▌ | 191266/225000 [361:22:03<67:31:53,  7.21s/it]
 85%|████████▌ | 191267/225000 [361:22:10<67:16:23,  7.18s/it]
 85%|████████▌ | 191268/225000 [361:22:20<75:16:46,  8.03s/it]
 85%|████████▌ | 191269/225000 [361:22:24<63:36:25,  6.79s/it]
 85%|████████▌ | 191270/225000 [361:22:36<77:53:03,  8.31s/it]
 85%|████████▌ | 191271/225000 [361:22:42<70:33:37,  7.53s/it]
 85%|████████▌ | 191272/225000 [361:22:52<77:33:30,  8.28s/it]
 85%|████████▌ | 191273/225000 [361:22:58<72:33:15,  7.74s/it]
 85%|████████▌ | 191274/225000 [361:23:13<92:06:48,  9.83s/it]
 85%|████████▌ | 191275/225000 [361:23:18<79:22:48,  8.47s/it]
 85%|████████▌ | 191276/225000 [361:23:29<84:59:59,  9.07s/it]
 85%|████████▌ | 191277/225000 [361:23:39<88:44:47,  9.47s/it]
 85%|████████▌ | 191278/225000 [361:23:44<75:40:21,  8.08s/it]
 85%|████████▌ | 191279/225000 [361:23:49<67:28:36,  7.20s/it]
 85%|████████▌ | 191280/225000 [361:24:01<81:13:59,  8.67s/it]
 85%|████████▌ | 191281/225000 [361:24:12<85:56:48,  9.18s/it]
 85%|████████▌ | 191282/225000 [361:24:17<75:36:46,  8.07s/it]
 85%|████████▌ | 191283/225000 [361:24:23<70:35:22,  7.54s/it]
 85%|████████▌ | 191284/225000 [361:24:31<70:09:13,  7.49s/it]
 85%|████████▌ | 191285/225000 [361:24:38<69:50:05,  7.46s/it]
 85%|████████▌ | 191286/225000 [361:24:49<78:09:27,  8.35s/it]
 85%|████████▌ | 191287/225000 [361:25:00<87:32:46,  9.35s/it]
 85%|████████▌ | 191288/225000 [361:25:09<85:28:24,  9.13s/it]
 85%|████████▌ | 191289/225000 [361:25:17<81:30:00,  8.70s/it]
 85%|████████▌ | 191290/225000 [361:25:22<73:10:31,  7.81s/it]
 85%|████████▌ | 191291/225000 [361:25:27<63:52:51,  6.82s/it]
 85%|████████▌ | 191292/225000 [361:25:39<79:48:40,  8.52s/it]
 85%|████████▌ | 191293/225000 [361:25:44<69:41:20,  7.44s/it]
 85%|████████▌ | 191294/225000 [361:25:55<78:42:14,  8.41s/it]
 85%|████████▌ | 191295/225000 [361:26:00<68:34:02,  7.32s/it]
 85%|████████▌ | 191296/225000 [361:26:10<75:36:28,  8.08s/it]
 85%|████████▌ | 191297/225000 [361:26:17<72:58:40,  7.80s/it]
 85%|████████▌ | 191298/225000 [361:26:23<67:50:55,  7.25s/it]
 85%|████████▌ | 191299/225000 [361:26:34<78:58:43,  8.44s/it]
 85%|████████▌ | 191300/225000 [361:26:41<76:25:16,  8.16s/it]
                                                              
{'loss': '1.243', 'grad_norm': '1.83', 'learning_rate': '2.986e-06', 'epoch': '0.8502', 'num_input_tokens_seen': 601186504, 'train_runtime': '1.301e+06', 'train_tokens_per_second': '462'}

 85%|████████▌ | 191300/225000 [361:26:41<76:25:16,  8.16s/it]
 85%|████████▌ | 191301/225000 [361:26:49<74:35:36,  7.97s/it]
 85%|████████▌ | 191302/225000 [361:26:54<67:44:02,  7.24s/it]
 85%|████████▌ | 191303/225000 [361:27:06<78:30:07,  8.39s/it]
 85%|████████▌ | 191304/225000 [361:27:17<86:01:14,  9.19s/it]
 85%|████████▌ | 191305/225000 [361:27:23<77:45:14,  8.31s/it]
 85%|████████▌ | 191306/225000 [361:27:29<72:35:16,  7.76s/it]
 85%|████████▌ | 191307/225000 [361:27:35<66:56:50,  7.15s/it]
 85%|████████▌ | 191308/225000 [361:27:44<72:23:54,  7.74s/it]
 85%|████████▌ | 191309/225000 [361:27:55<80:25:36,  8.59s/it]
 85%|████████▌ | 191310/225000 [361:28:03<79:14:12,  8.47s/it]
 85%|████████▌ | 191311/225000 [361:28:13<82:31:06,  8.82s/it]
 85%|████████▌ | 191312/225000 [361:28:16<68:38:52,  7.34s/it]
 85%|████████▌ | 191313/225000 [361:28:32<92:03:44,  9.84s/it]
 85%|████████▌ | 191314/225000 [361:28:41<88:57:48,  9.51s/it]
 85%|████████▌ | 191315/225000 [361:28:49<84:27:35,  9.03s/it]
 85%|████████▌ | 191316/225000 [361:28:55<77:51:14,  8.32s/it]
 85%|████████▌ | 191317/225000 [361:29:02<74:01:15,  7.91s/it]
 85%|████████▌ | 191318/225000 [361:29:14<85:01:00,  9.09s/it]
 85%|████████▌ | 191319/225000 [361:29:24<85:38:40,  9.15s/it]
 85%|████████▌ | 191320/225000 [361:29:30<79:17:35,  8.48s/it]
 85%|████████▌ | 191321/225000 [361:29:40<81:49:16,  8.75s/it]
 85%|████████▌ | 191322/225000 [361:29:50<84:45:21,  9.06s/it]
 85%|████████▌ | 191323/225000 [361:29:55<73:51:26,  7.90s/it]
 85%|████████▌ | 191324/225000 [361:30:04<77:06:04,  8.24s/it]
 85%|████████▌ | 191325/225000 [361:30:14<82:32:21,  8.82s/it]
 85%|████████▌ | 191326/225000 [361:30:25<88:56:12,  9.51s/it]
 85%|████████▌ | 191327/225000 [361:30:28<71:08:17,  7.61s/it]
 85%|████████▌ | 191328/225000 [361:30:36<72:50:26,  7.79s/it]
 85%|████████▌ | 191329/225000 [361:30:47<79:52:43,  8.54s/it]
 85%|████████▌ | 191330/225000 [361:30:58<88:08:33,  9.42s/it]
 85%|████████▌ | 191331/225000 [361:31:04<76:30:58,  8.18s/it]
 85%|████████▌ | 191332/225000 [361:31:10<70:21:11,  7.52s/it]
 85%|████████▌ | 191333/225000 [361:31:19<74:33:22,  7.97s/it]
 85%|████████▌ | 191334/225000 [361:31:27<77:08:48,  8.25s/it]
 85%|████████▌ | 191335/225000 [361:31:36<78:06:02,  8.35s/it]
 85%|████████▌ | 191336/225000 [361:31:41<69:56:19,  7.48s/it]
 85%|████████▌ | 191337/225000 [361:31:46<62:59:09,  6.74s/it]
 85%|████████▌ | 191338/225000 [361:31:53<61:37:55,  6.59s/it]
 85%|████████▌ | 191339/225000 [361:32:04<74:46:44,  8.00s/it]
 85%|████████▌ | 191340/225000 [361:32:11<71:59:46,  7.70s/it]
 85%|████████▌ | 191341/225000 [361:32:17<68:17:03,  7.30s/it]
 85%|████████▌ | 191342/225000 [361:32:28<76:50:03,  8.22s/it]
 85%|████████▌ | 191343/225000 [361:32:38<82:12:45,  8.79s/it]
 85%|████████▌ | 191344/225000 [361:32:44<73:38:07,  7.88s/it]
 85%|████████▌ | 191345/225000 [361:32:50<70:20:03,  7.52s/it]
 85%|████████▌ | 191346/225000 [361:33:00<75:28:10,  8.07s/it]
 85%|████████▌ | 191347/225000 [361:33:10<83:03:48,  8.89s/it]
 85%|████████▌ | 191348/225000 [361:33:16<73:22:07,  7.85s/it]
 85%|████████▌ | 191349/225000 [361:33:27<81:13:36,  8.69s/it]
 85%|████████▌ | 191350/225000 [361:33:32<73:04:39,  7.82s/it]
                                                              
{'loss': '1.249', 'grad_norm': '1.17', 'learning_rate': '2.977e-06', 'epoch': '0.8504', 'num_input_tokens_seen': 601404208, 'train_runtime': '1.302e+06', 'train_tokens_per_second': '462'}

 85%|████████▌ | 191350/225000 [361:33:32<73:04:39,  7.82s/it]
 85%|████████▌ | 191351/225000 [361:33:39<68:55:40,  7.37s/it]
 85%|████████▌ | 191352/225000 [361:33:48<75:07:33,  8.04s/it]
 85%|████████▌ | 191353/225000 [361:33:57<76:16:42,  8.16s/it]
 85%|████████▌ | 191354/225000 [361:34:02<68:06:14,  7.29s/it]
 85%|████████▌ | 191355/225000 [361:34:08<63:32:54,  6.80s/it]
 85%|████████▌ | 191356/225000 [361:34:24<91:04:40,  9.75s/it]
 85%|████████▌ | 191357/225000 [361:34:29<76:19:51,  8.17s/it]
 85%|████████▌ | 191358/225000 [361:34:35<71:48:43,  7.68s/it]
 85%|████████▌ | 191359/225000 [361:34:41<66:00:10,  7.06s/it]
 85%|████████▌ | 191360/225000 [361:34:54<84:16:30,  9.02s/it]
 85%|████████▌ | 191361/225000 [361:35:01<77:24:54,  8.28s/it]
 85%|████████▌ | 191362/225000 [361:35:11<82:16:19,  8.80s/it]
 85%|████████▌ | 191363/225000 [361:35:18<77:19:04,  8.27s/it]
 85%|████████▌ | 191364/225000 [361:35:24<69:46:28,  7.47s/it]
 85%|████████▌ | 191365/225000 [361:35:33<75:39:50,  8.10s/it]
 85%|████████▌ | 191366/225000 [361:35:41<74:55:24,  8.02s/it]
 85%|████████▌ | 191367/225000 [361:35:47<68:55:44,  7.38s/it]
 85%|████████▌ | 191368/225000 [361:35:53<64:36:52,  6.92s/it]
 85%|████████▌ | 191369/225000 [361:36:01<68:06:45,  7.29s/it]
 85%|████████▌ | 191370/225000 [361:36:16<89:19:49,  9.56s/it]
 85%|████████▌ | 191371/225000 [361:36:26<92:16:01,  9.88s/it]
 85%|████████▌ | 191372/225000 [361:36:33<82:24:50,  8.82s/it]
 85%|████████▌ | 191373/225000 [361:36:39<74:38:28,  7.99s/it]
 85%|████████▌ | 191374/225000 [361:36:48<77:17:04,  8.27s/it]
 85%|████████▌ | 191375/225000 [361:36:55<73:41:00,  7.89s/it]
 85%|████████▌ | 191376/225000 [361:37:04<76:24:14,  8.18s/it]
 85%|████████▌ | 191377/225000 [361:37:16<88:08:13,  9.44s/it]
 85%|████████▌ | 191378/225000 [361:37:21<74:21:25,  7.96s/it]
 85%|████████▌ | 191379/225000 [361:37:30<79:38:13,  8.53s/it]
 85%|████████▌ | 191380/225000 [361:37:38<76:16:12,  8.17s/it]
 85%|████████▌ | 191381/225000 [361:37:45<74:44:05,  8.00s/it]
 85%|████████▌ | 191382/225000 [361:37:51<67:16:26,  7.20s/it]
 85%|████████▌ | 191383/225000 [361:38:03<80:25:02,  8.61s/it]
 85%|████████▌ | 191384/225000 [361:38:09<74:04:59,  7.93s/it]
 85%|████████▌ | 191385/225000 [361:38:23<92:25:12,  9.90s/it]
 85%|████████▌ | 191386/225000 [361:38:33<91:48:21,  9.83s/it]
 85%|████████▌ | 191387/225000 [361:38:39<81:06:14,  8.69s/it]
 85%|████████▌ | 191388/225000 [361:38:50<86:36:20,  9.28s/it]
 85%|████████▌ | 191389/225000 [361:38:57<80:03:06,  8.57s/it]
 85%|████████▌ | 191390/225000 [361:39:03<72:55:59,  7.81s/it]
 85%|████████▌ | 191391/225000 [361:39:10<72:04:23,  7.72s/it]
 85%|████████▌ | 191392/225000 [361:39:22<83:05:37,  8.90s/it]
 85%|████████▌ | 191393/225000 [361:39:31<83:15:49,  8.92s/it]
 85%|████████▌ | 191394/225000 [361:39:37<75:03:12,  8.04s/it]
 85%|████████▌ | 191395/225000 [361:39:42<66:39:42,  7.14s/it]
 85%|████████▌ | 191396/225000 [361:39:47<61:12:34,  6.56s/it]
 85%|████████▌ | 191397/225000 [361:39:57<71:17:53,  7.64s/it]
 85%|████████▌ | 191398/225000 [361:40:05<72:09:25,  7.73s/it]
 85%|████████▌ | 191399/225000 [361:40:12<69:14:24,  7.42s/it]
 85%|████████▌ | 191400/225000 [361:40:20<71:49:13,  7.70s/it]
                                                              
{'loss': '1.248', 'grad_norm': '1.419', 'learning_rate': '2.969e-06', 'epoch': '0.8507', 'num_input_tokens_seen': 601613976, 'train_runtime': '1.302e+06', 'train_tokens_per_second': '462.1'}

 85%|████████▌ | 191400/225000 [361:40:20<71:49:13,  7.70s/it]
 85%|████████▌ | 191401/225000 [361:40:28<71:48:20,  7.69s/it]
 85%|████████▌ | 191402/225000 [361:40:35<71:13:30,  7.63s/it]
 85%|████████▌ | 191403/225000 [361:40:41<66:37:49,  7.14s/it]
 85%|████████▌ | 191404/225000 [361:40:46<59:35:23,  6.39s/it]
 85%|████████▌ | 191405/225000 [361:40:52<57:59:24,  6.21s/it]
 85%|████████▌ | 191406/225000 [361:41:01<65:02:05,  6.97s/it]
 85%|████████▌ | 191407/225000 [361:41:06<60:27:18,  6.48s/it]
 85%|████████▌ | 191408/225000 [361:41:13<61:28:30,  6.59s/it]
 85%|████████▌ | 191409/225000 [361:41:19<60:06:45,  6.44s/it]
 85%|████████▌ | 191410/225000 [361:41:25<58:55:28,  6.32s/it]
 85%|████████▌ | 191411/225000 [361:41:38<78:16:25,  8.39s/it]
 85%|████████▌ | 191412/225000 [361:41:52<92:37:33,  9.93s/it]
 85%|████████▌ | 191413/225000 [361:41:57<79:33:02,  8.53s/it]
 85%|████████▌ | 191414/225000 [361:42:05<77:18:25,  8.29s/it]
 85%|████████▌ | 191415/225000 [361:42:08<64:42:14,  6.94s/it]
 85%|████████▌ | 191416/225000 [361:42:21<81:26:43,  8.73s/it]
 85%|████████▌ | 191417/225000 [361:42:32<87:29:57,  9.38s/it]
 85%|████████▌ | 191418/225000 [361:42:41<86:29:59,  9.27s/it]
 85%|████████▌ | 191419/225000 [361:42:48<79:44:06,  8.55s/it]
 85%|████████▌ | 191420/225000 [361:42:53<70:55:29,  7.60s/it]
 85%|████████▌ | 191421/225000 [361:43:07<86:43:21,  9.30s/it]
 85%|████████▌ | 191422/225000 [361:43:12<76:51:21,  8.24s/it]
 85%|████████▌ | 191423/225000 [361:43:19<71:05:09,  7.62s/it]
 85%|████████▌ | 191424/225000 [361:43:26<69:08:10,  7.41s/it]
 85%|████████▌ | 191425/225000 [361:43:38<82:53:10,  8.89s/it]
 85%|████████▌ | 191426/225000 [361:43:49<89:22:55,  9.58s/it]
 85%|████████▌ | 191427/225000 [361:44:00<92:26:06,  9.91s/it]
 85%|████████▌ | 191428/225000 [361:44:06<82:20:40,  8.83s/it]
 85%|████████▌ | 191429/225000 [361:44:13<76:52:16,  8.24s/it]
 85%|████████▌ | 191430/225000 [361:44:19<70:47:57,  7.59s/it]
 85%|████████▌ | 191431/225000 [361:44:31<83:51:15,  8.99s/it]
 85%|████████▌ | 191432/225000 [361:44:36<71:00:24,  7.62s/it]
 85%|████████▌ | 191433/225000 [361:44:45<75:53:12,  8.14s/it]
 85%|████████▌ | 191434/225000 [361:44:51<69:03:44,  7.41s/it]
 85%|████████▌ | 191435/225000 [361:45:03<83:07:56,  8.92s/it]
 85%|████████▌ | 191436/225000 [361:45:10<76:48:16,  8.24s/it]
 85%|████████▌ | 191437/225000 [361:45:17<73:39:11,  7.90s/it]
 85%|████████▌ | 191438/225000 [361:45:26<78:02:18,  8.37s/it]
 85%|████████▌ | 191439/225000 [361:45:33<72:28:27,  7.77s/it]
 85%|████████▌ | 191440/225000 [361:45:42<77:30:47,  8.31s/it]
 85%|████████▌ | 191441/225000 [361:45:46<65:39:09,  7.04s/it]
 85%|████████▌ | 191442/225000 [361:45:58<79:13:44,  8.50s/it]
 85%|████████▌ | 191443/225000 [361:46:06<76:13:56,  8.18s/it]
 85%|████████▌ | 191444/225000 [361:46:12<70:11:28,  7.53s/it]
 85%|████████▌ | 191445/225000 [361:46:20<72:58:02,  7.83s/it]
 85%|████████▌ | 191446/225000 [361:46:29<74:49:14,  8.03s/it]
 85%|████████▌ | 191447/225000 [361:46:41<85:03:02,  9.13s/it]
 85%|████████▌ | 191448/225000 [361:46:48<79:02:05,  8.48s/it]
 85%|████████▌ | 191449/225000 [361:46:55<75:56:19,  8.15s/it]
 85%|████████▌ | 191450/225000 [361:47:01<69:55:45,  7.50s/it]
                                                              
{'loss': '1.27', 'grad_norm': '1.192', 'learning_rate': '2.96e-06', 'epoch': '0.8509', 'num_input_tokens_seen': 601828632, 'train_runtime': '1.302e+06', 'train_tokens_per_second': '462.1'}

 85%|████████▌ | 191450/225000 [361:47:01<69:55:45,  7.50s/it]
 85%|████████▌ | 191451/225000 [361:47:16<90:06:05,  9.67s/it]
 85%|████████▌ | 191452/225000 [361:47:22<79:44:03,  8.56s/it]
 85%|████████▌ | 191453/225000 [361:47:29<77:55:18,  8.36s/it]
 85%|████████▌ | 191454/225000 [361:47:40<85:00:07,  9.12s/it]
 85%|████████▌ | 191455/225000 [361:47:46<73:56:38,  7.94s/it]
 85%|████████▌ | 191456/225000 [361:47:52<68:53:08,  7.39s/it]
 85%|████████▌ | 191457/225000 [361:47:59<69:13:33,  7.43s/it]
 85%|████████▌ | 191458/225000 [361:48:10<77:24:53,  8.31s/it]
 85%|████████▌ | 191459/225000 [361:48:16<72:49:15,  7.82s/it]
 85%|████████▌ | 191460/225000 [361:48:21<63:42:07,  6.84s/it]
 85%|████████▌ | 191461/225000 [361:48:26<60:38:37,  6.51s/it]
 85%|████████▌ | 191462/225000 [361:48:32<56:27:53,  6.06s/it]
 85%|████████▌ | 191463/225000 [361:48:42<69:45:06,  7.49s/it]
 85%|████████▌ | 191464/225000 [361:48:54<81:46:43,  8.78s/it]
 85%|████████▌ | 191465/225000 [361:48:59<69:37:32,  7.47s/it]
 85%|████████▌ | 191466/225000 [361:49:10<80:30:32,  8.64s/it]
 85%|████████▌ | 191467/225000 [361:49:17<76:42:58,  8.24s/it]
 85%|████████▌ | 191468/225000 [361:49:24<73:20:28,  7.87s/it]
 85%|████████▌ | 191469/225000 [361:49:31<69:16:15,  7.44s/it]
 85%|████████▌ | 191470/225000 [361:49:39<72:24:20,  7.77s/it]
 85%|████████▌ | 191471/225000 [361:49:44<63:16:12,  6.79s/it]
 85%|████████▌ | 191472/225000 [361:49:55<76:17:44,  8.19s/it]
 85%|████████▌ | 191473/225000 [361:50:00<67:26:29,  7.24s/it]
 85%|████████▌ | 191474/225000 [361:50:08<69:32:38,  7.47s/it]
 85%|████████▌ | 191475/225000 [361:50:20<80:23:52,  8.63s/it]
 85%|████████▌ | 191476/225000 [361:50:26<73:59:26,  7.95s/it]
 85%|████████▌ | 191477/225000 [361:50:36<79:41:27,  8.56s/it]
 85%|████████▌ | 191478/225000 [361:50:44<79:06:53,  8.50s/it]
 85%|████████▌ | 191479/225000 [361:50:54<81:40:28,  8.77s/it]
 85%|████████▌ | 191480/225000 [361:51:01<76:49:19,  8.25s/it]
 85%|████████▌ | 191481/225000 [361:51:13<89:07:50,  9.57s/it]
 85%|████████▌ | 191482/225000 [361:51:22<85:45:14,  9.21s/it]
 85%|████████▌ | 191483/225000 [361:51:31<84:38:19,  9.09s/it]
 85%|████████▌ | 191484/225000 [361:51:37<78:03:42,  8.38s/it]
 85%|████████▌ | 191485/225000 [361:51:44<72:51:51,  7.83s/it]
 85%|████████▌ | 191486/225000 [361:51:49<65:48:45,  7.07s/it]
 85%|████████▌ | 191487/225000 [361:52:01<79:45:35,  8.57s/it]
 85%|████████▌ | 191488/225000 [361:52:12<87:15:06,  9.37s/it]
 85%|████████▌ | 191489/225000 [361:52:20<83:22:48,  8.96s/it]
 85%|████████▌ | 191490/225000 [361:52:25<71:15:34,  7.66s/it]
 85%|████████▌ | 191491/225000 [361:52:32<70:49:55,  7.61s/it]
 85%|████████▌ | 191492/225000 [361:52:40<69:18:43,  7.45s/it]
 85%|████████▌ | 191493/225000 [361:52:47<68:21:21,  7.34s/it]
 85%|████████▌ | 191494/225000 [361:52:52<62:17:43,  6.69s/it]
 85%|████████▌ | 191495/225000 [361:52:58<61:29:02,  6.61s/it]
 85%|████████▌ | 191496/225000 [361:53:08<71:28:47,  7.68s/it]
 85%|████████▌ | 191497/225000 [361:53:19<79:17:14,  8.52s/it]
 85%|████████▌ | 191498/225000 [361:53:24<70:52:24,  7.62s/it]
 85%|████████▌ | 191499/225000 [361:53:31<68:28:25,  7.36s/it]
 85%|████████▌ | 191500/225000 [361:53:45<85:40:57,  9.21s/it]
                                                              
{'loss': '1.237', 'grad_norm': '1.093', 'learning_rate': '2.951e-06', 'epoch': '0.8511', 'num_input_tokens_seen': 602036312, 'train_runtime': '1.303e+06', 'train_tokens_per_second': '462.1'}

 85%|████████▌ | 191500/225000 [361:53:45<85:40:57,  9.21s/it]
 85%|████████▌ | 191501/225000 [361:53:49<71:09:51,  7.65s/it]
 85%|████████▌ | 191502/225000 [361:53:55<68:30:26,  7.36s/it]
 85%|████████▌ | 191503/225000 [361:54:04<72:56:39,  7.84s/it]
 85%|████████▌ | 191504/225000 [361:54:16<83:23:44,  8.96s/it]
 85%|████████▌ | 191505/225000 [361:54:24<79:47:45,  8.58s/it]
 85%|████████▌ | 191506/225000 [361:54:29<69:57:34,  7.52s/it]
 85%|████████▌ | 191507/225000 [361:54:39<77:04:17,  8.28s/it]
 85%|████████▌ | 191508/225000 [361:54:48<78:44:39,  8.46s/it]
 85%|████████▌ | 191509/225000 [361:54:55<74:53:17,  8.05s/it]
 85%|████████▌ | 191510/225000 [361:54:59<65:02:48,  6.99s/it]
 85%|████████▌ | 191511/225000 [361:55:12<80:59:57,  8.71s/it]
 85%|████████▌ | 191512/225000 [361:55:20<79:43:29,  8.57s/it]
 85%|████████▌ | 191513/225000 [361:55:26<71:22:30,  7.67s/it]
 85%|████████▌ | 191514/225000 [361:55:32<67:29:47,  7.26s/it]
 85%|████████▌ | 191515/225000 [361:55:40<68:58:39,  7.42s/it]
 85%|████████▌ | 191516/225000 [361:55:46<66:00:53,  7.10s/it]
 85%|████████▌ | 191517/225000 [361:55:51<58:52:14,  6.33s/it]
 85%|████████▌ | 191518/225000 [361:55:57<59:47:33,  6.43s/it]
 85%|████████▌ | 191519/225000 [361:56:08<72:10:05,  7.76s/it]
 85%|████████▌ | 191520/225000 [361:56:15<69:54:06,  7.52s/it]
 85%|████████▌ | 191521/225000 [361:56:26<78:04:51,  8.40s/it]
 85%|████████▌ | 191522/225000 [361:56:42<100:53:01, 10.85s/it]
 85%|████████▌ | 191523/225000 [361:56:46<81:42:29,  8.79s/it] 
 85%|████████▌ | 191524/225000 [361:56:53<76:22:56,  8.21s/it]
 85%|████████▌ | 191525/225000 [361:56:58<67:23:05,  7.25s/it]
 85%|████████▌ | 191526/225000 [361:57:11<82:02:16,  8.82s/it]
 85%|████████▌ | 191527/225000 [361:57:14<67:47:13,  7.29s/it]
 85%|████████▌ | 191528/225000 [361:57:23<71:11:20,  7.66s/it]
 85%|████████▌ | 191529/225000 [361:57:31<73:12:39,  7.87s/it]
 85%|████████▌ | 191530/225000 [361:57:38<70:39:11,  7.60s/it]
 85%|████████▌ | 191531/225000 [361:57:49<79:28:54,  8.55s/it]
 85%|████████▌ | 191532/225000 [361:57:55<71:35:23,  7.70s/it]
 85%|████████▌ | 191533/225000 [361:58:04<75:01:17,  8.07s/it]
 85%|████████▌ | 191534/225000 [361:58:10<70:48:23,  7.62s/it]
 85%|████████▌ | 191535/225000 [361:58:15<63:12:11,  6.80s/it]
 85%|████████▌ | 191536/225000 [361:58:23<66:52:42,  7.19s/it]
 85%|████████▌ | 191537/225000 [361:58:35<79:13:33,  8.52s/it]
 85%|████████▌ | 191538/225000 [361:58:46<85:52:52,  9.24s/it]
 85%|████████▌ | 191539/225000 [361:58:50<71:26:11,  7.69s/it]
 85%|████████▌ | 191540/225000 [361:58:59<75:14:44,  8.10s/it]
 85%|████████▌ | 191541/225000 [361:59:12<89:51:17,  9.67s/it]
 85%|████████▌ | 191542/225000 [361:59:19<81:34:11,  8.78s/it]
 85%|████████▌ | 191543/225000 [361:59:23<70:05:24,  7.54s/it]
 85%|████████▌ | 191544/225000 [361:59:36<82:38:45,  8.89s/it]
 85%|████████▌ | 191545/225000 [361:59:41<73:24:30,  7.90s/it]
 85%|████████▌ | 191546/225000 [361:59:47<69:06:58,  7.44s/it]
 85%|████████▌ | 191547/225000 [361:59:54<67:32:59,  7.27s/it]
 85%|████████▌ | 191548/225000 [362:00:02<68:15:58,  7.35s/it]
 85%|████████▌ | 191549/225000 [362:00:14<81:40:40,  8.79s/it]
 85%|████████▌ | 191550/225000 [362:00:20<73:27:13,  7.91s/it]
                                                              
{'loss': '1.232', 'grad_norm': '1.205', 'learning_rate': '2.943e-06', 'epoch': '0.8513', 'num_input_tokens_seen': 602243144, 'train_runtime': '1.303e+06', 'train_tokens_per_second': '462.1'}

 85%|████████▌ | 191550/225000 [362:00:20<73:27:13,  7.91s/it]
 85%|████████▌ | 191551/225000 [362:00:25<66:29:47,  7.16s/it]
 85%|████████▌ | 191552/225000 [362:00:32<63:57:15,  6.88s/it]
 85%|████████▌ | 191553/225000 [362:00:37<60:56:09,  6.56s/it]
 85%|████████▌ | 191554/225000 [362:00:44<60:13:28,  6.48s/it]
 85%|████████▌ | 191555/225000 [362:00:58<82:28:36,  8.88s/it]
 85%|████████▌ | 191556/225000 [362:01:06<80:03:22,  8.62s/it]
 85%|████████▌ | 191557/225000 [362:01:18<88:12:30,  9.50s/it]
 85%|████████▌ | 191558/225000 [362:01:23<75:32:56,  8.13s/it]
 85%|████████▌ | 191559/225000 [362:01:30<72:42:15,  7.83s/it]
 85%|████████▌ | 191560/225000 [362:01:41<81:15:54,  8.75s/it]
 85%|████████▌ | 191561/225000 [362:01:45<68:46:18,  7.40s/it]
 85%|████████▌ | 191562/225000 [362:01:54<74:50:16,  8.06s/it]
 85%|████████▌ | 191563/225000 [362:02:01<69:31:24,  7.49s/it]
 85%|████████▌ | 191564/225000 [362:02:08<68:10:24,  7.34s/it]
 85%|████████▌ | 191565/225000 [362:02:13<62:27:54,  6.73s/it]
 85%|████████▌ | 191566/225000 [362:02:20<62:21:06,  6.71s/it]
 85%|████████▌ | 191567/225000 [362:02:32<78:52:41,  8.49s/it]
 85%|████████▌ | 191568/225000 [362:02:36<66:37:37,  7.17s/it]
 85%|████████▌ | 191569/225000 [362:02:47<76:20:38,  8.22s/it]
 85%|████████▌ | 191570/225000 [362:02:59<85:43:22,  9.23s/it]
 85%|████████▌ | 191571/225000 [362:03:06<79:42:13,  8.58s/it]
 85%|████████▌ | 191572/225000 [362:03:15<82:36:52,  8.90s/it]
 85%|████████▌ | 191573/225000 [362:03:25<85:11:40,  9.18s/it]
 85%|████████▌ | 191574/225000 [362:03:30<73:47:15,  7.95s/it]
 85%|████████▌ | 191575/225000 [362:03:36<67:35:26,  7.28s/it]
 85%|████████▌ | 191576/225000 [362:03:41<61:46:41,  6.65s/it]
 85%|████████▌ | 191577/225000 [362:03:47<60:43:57,  6.54s/it]
 85%|████████▌ | 191578/225000 [362:04:00<77:52:28,  8.39s/it]
 85%|████████▌ | 191579/225000 [362:04:04<65:55:37,  7.10s/it]
 85%|████████▌ | 191580/225000 [362:04:11<65:10:36,  7.02s/it]
 85%|████████▌ | 191581/225000 [362:04:18<66:22:37,  7.15s/it]
 85%|████████▌ | 191582/225000 [362:04:32<82:52:18,  8.93s/it]
 85%|████████▌ | 191583/225000 [362:04:35<68:54:14,  7.42s/it]
 85%|████████▌ | 191584/225000 [362:04:39<59:30:20,  6.41s/it]
 85%|████████▌ | 191585/225000 [362:04:49<67:29:37,  7.27s/it]
 85%|████████▌ | 191586/225000 [362:05:02<82:47:23,  8.92s/it]
 85%|████████▌ | 191587/225000 [362:05:10<81:40:08,  8.80s/it]
 85%|████████▌ | 191588/225000 [362:05:14<69:21:32,  7.47s/it]
 85%|████████▌ | 191589/225000 [362:05:22<68:40:27,  7.40s/it]
 85%|████████▌ | 191590/225000 [362:05:34<82:22:03,  8.88s/it]
 85%|████████▌ | 191591/225000 [362:05:41<77:52:00,  8.39s/it]
 85%|████████▌ | 191592/225000 [362:05:51<80:21:03,  8.66s/it]
 85%|████████▌ | 191593/225000 [362:05:56<71:17:32,  7.68s/it]
 85%|████████▌ | 191594/225000 [362:06:04<72:27:44,  7.81s/it]
 85%|████████▌ | 191595/225000 [362:06:13<76:31:37,  8.25s/it]
 85%|████████▌ | 191596/225000 [362:06:19<69:31:58,  7.49s/it]
 85%|████████▌ | 191597/225000 [362:06:27<72:05:10,  7.77s/it]
 85%|████████▌ | 191598/225000 [362:06:34<69:44:05,  7.52s/it]
 85%|████████▌ | 191599/225000 [362:06:46<82:08:07,  8.85s/it]
 85%|████████▌ | 191600/225000 [362:06:54<78:24:41,  8.45s/it]
                                                              
{'loss': '1.218', 'grad_norm': '1.426', 'learning_rate': '2.934e-06', 'epoch': '0.8516', 'num_input_tokens_seen': 602448752, 'train_runtime': '1.304e+06', 'train_tokens_per_second': '462.1'}

 85%|████████▌ | 191600/225000 [362:06:54<78:24:41,  8.45s/it]
 85%|████████▌ | 191601/225000 [362:07:02<76:52:22,  8.29s/it]
 85%|████████▌ | 191602/225000 [362:07:09<73:44:57,  7.95s/it]
 85%|████████▌ | 191603/225000 [362:07:14<65:32:13,  7.06s/it]
 85%|████████▌ | 191604/225000 [362:07:22<68:00:33,  7.33s/it]
 85%|████████▌ | 191605/225000 [362:07:34<81:29:25,  8.78s/it]
 85%|████████▌ | 191606/225000 [362:07:44<84:05:52,  9.07s/it]
 85%|████████▌ | 191607/225000 [362:07:51<80:21:12,  8.66s/it]
 85%|████████▌ | 191608/225000 [362:08:05<92:37:27,  9.99s/it]
 85%|████████▌ | 191609/225000 [362:08:09<77:00:45,  8.30s/it]
 85%|████████▌ | 191610/225000 [362:08:18<79:26:36,  8.57s/it]
 85%|████████▌ | 191611/225000 [362:08:22<67:43:59,  7.30s/it]
 85%|████████▌ | 191612/225000 [362:08:31<70:40:17,  7.62s/it]
 85%|████████▌ | 191613/225000 [362:08:39<73:18:37,  7.90s/it]
 85%|████████▌ | 191614/225000 [362:08:46<70:08:22,  7.56s/it]
 85%|████████▌ | 191615/225000 [362:08:53<67:54:43,  7.32s/it]
 85%|████████▌ | 191616/225000 [362:08:59<65:32:43,  7.07s/it]
 85%|████████▌ | 191617/225000 [362:09:06<64:33:01,  6.96s/it]
 85%|████████▌ | 191618/225000 [362:09:13<63:08:52,  6.81s/it]
 85%|████████▌ | 191619/225000 [362:09:30<91:25:31,  9.86s/it]
 85%|████████▌ | 191620/225000 [362:09:38<86:39:35,  9.35s/it]
 85%|████████▌ | 191621/225000 [362:09:45<80:02:07,  8.63s/it]
 85%|████████▌ | 191622/225000 [362:09:53<78:45:24,  8.49s/it]
 85%|████████▌ | 191623/225000 [362:09:58<69:47:28,  7.53s/it]
 85%|████████▌ | 191624/225000 [362:10:07<72:11:51,  7.79s/it]
 85%|████████▌ | 191625/225000 [362:10:13<67:40:41,  7.30s/it]
 85%|████████▌ | 191626/225000 [362:10:19<64:15:55,  6.93s/it]
 85%|████████▌ | 191627/225000 [362:10:25<62:22:57,  6.73s/it]
 85%|████████▌ | 191628/225000 [362:10:39<83:24:20,  9.00s/it]
 85%|████████▌ | 191629/225000 [362:10:48<81:15:37,  8.77s/it]
 85%|████████▌ | 191630/225000 [362:10:55<76:50:55,  8.29s/it]
 85%|████████▌ | 191631/225000 [362:11:06<84:52:34,  9.16s/it]
 85%|████████▌ | 191632/225000 [362:11:12<76:36:53,  8.27s/it]
 85%|████████▌ | 191633/225000 [362:11:21<78:49:35,  8.50s/it]
 85%|████████▌ | 191634/225000 [362:11:26<68:34:55,  7.40s/it]
 85%|████████▌ | 191635/225000 [362:11:32<64:09:07,  6.92s/it]
 85%|████████▌ | 191636/225000 [362:11:44<80:03:39,  8.64s/it]
 85%|████████▌ | 191637/225000 [362:11:51<74:16:18,  8.01s/it]
 85%|████████▌ | 191638/225000 [362:12:01<79:12:40,  8.55s/it]
 85%|████████▌ | 191639/225000 [362:12:08<75:02:44,  8.10s/it]
 85%|████████▌ | 191640/225000 [362:12:16<74:10:11,  8.00s/it]
 85%|████████▌ | 191641/225000 [362:12:21<67:52:27,  7.32s/it]
 85%|████████▌ | 191642/225000 [362:12:29<70:11:23,  7.57s/it]
 85%|████████▌ | 191643/225000 [362:12:35<63:52:08,  6.89s/it]
 85%|████████▌ | 191644/225000 [362:12:49<84:18:38,  9.10s/it]
 85%|████████▌ | 191645/225000 [362:12:53<71:10:31,  7.68s/it]
 85%|████████▌ | 191646/225000 [362:13:03<76:13:13,  8.23s/it]
 85%|████████▌ | 191647/225000 [362:13:07<65:42:41,  7.09s/it]
 85%|████████▌ | 191648/225000 [362:13:14<63:58:27,  6.91s/it]
 85%|████████▌ | 191649/225000 [362:13:18<57:14:09,  6.18s/it]
 85%|████████▌ | 191650/225000 [362:13:26<62:34:51,  6.76s/it]
                                                              
{'loss': '1.282', 'grad_norm': '1.413', 'learning_rate': '2.926e-06', 'epoch': '0.8518', 'num_input_tokens_seen': 602655992, 'train_runtime': '1.304e+06', 'train_tokens_per_second': '462.2'}

 85%|████████▌ | 191650/225000 [362:13:26<62:34:51,  6.76s/it]
 85%|████████▌ | 191651/225000 [362:13:33<62:03:58,  6.70s/it]
 85%|████████▌ | 191652/225000 [362:13:45<76:19:29,  8.24s/it]
 85%|████████▌ | 191653/225000 [362:13:51<70:06:24,  7.57s/it]
 85%|████████▌ | 191654/225000 [362:14:00<75:52:41,  8.19s/it]
 85%|████████▌ | 191655/225000 [362:14:09<77:45:13,  8.39s/it]
 85%|████████▌ | 191656/225000 [362:14:21<86:44:40,  9.37s/it]
 85%|████████▌ | 191657/225000 [362:14:30<84:42:34,  9.15s/it]
 85%|████████▌ | 191658/225000 [362:14:35<73:46:42,  7.97s/it]
 85%|████████▌ | 191659/225000 [362:14:44<77:47:51,  8.40s/it]
 85%|████████▌ | 191660/225000 [362:14:51<71:57:04,  7.77s/it]
 85%|████████▌ | 191661/225000 [362:14:57<68:42:46,  7.42s/it]
 85%|████████▌ | 191662/225000 [362:15:08<77:16:21,  8.34s/it]
 85%|████████▌ | 191663/225000 [362:15:15<75:12:11,  8.12s/it]
 85%|████████▌ | 191664/225000 [362:15:25<80:39:33,  8.71s/it]
 85%|████████▌ | 191665/225000 [362:15:35<83:22:26,  9.00s/it]
 85%|████████▌ | 191666/225000 [362:15:42<77:44:44,  8.40s/it]
 85%|████████▌ | 191667/225000 [362:15:48<72:21:06,  7.81s/it]
 85%|████████▌ | 191668/225000 [362:15:56<72:02:10,  7.78s/it]
 85%|████████▌ | 191669/225000 [362:16:05<75:06:52,  8.11s/it]
 85%|████████▌ | 191670/225000 [362:16:15<79:56:55,  8.64s/it]
 85%|████████▌ | 191671/225000 [362:16:23<77:58:17,  8.42s/it]
 85%|████████▌ | 191672/225000 [362:16:29<71:16:17,  7.70s/it]
 85%|████████▌ | 191673/225000 [362:16:39<78:09:49,  8.44s/it]
 85%|████████▌ | 191674/225000 [362:16:50<85:39:20,  9.25s/it]
 85%|████████▌ | 191675/225000 [362:16:57<80:22:58,  8.68s/it]
 85%|████████▌ | 191676/225000 [362:17:06<79:06:50,  8.55s/it]
 85%|████████▌ | 191677/225000 [362:17:14<77:49:18,  8.41s/it]
 85%|████████▌ | 191678/225000 [362:17:19<67:44:13,  7.32s/it]
 85%|████████▌ | 191679/225000 [362:17:24<61:30:54,  6.65s/it]
 85%|████████▌ | 191680/225000 [362:17:31<64:12:07,  6.94s/it]
 85%|████████▌ | 191681/225000 [362:17:40<68:10:53,  7.37s/it]
 85%|████████▌ | 191682/225000 [362:17:46<66:21:28,  7.17s/it]
 85%|████████▌ | 191683/225000 [362:17:55<69:53:31,  7.55s/it]
 85%|████████▌ | 191684/225000 [362:18:07<83:27:19,  9.02s/it]
 85%|████████▌ | 191685/225000 [362:18:17<84:34:51,  9.14s/it]
 85%|████████▌ | 191686/225000 [362:18:24<79:07:26,  8.55s/it]
 85%|████████▌ | 191687/225000 [362:18:38<93:25:48, 10.10s/it]
 85%|████████▌ | 191688/225000 [362:18:44<83:20:43,  9.01s/it]
 85%|████████▌ | 191689/225000 [362:18:51<79:07:48,  8.55s/it]
 85%|████████▌ | 191690/225000 [362:19:03<88:31:16,  9.57s/it]
 85%|████████▌ | 191691/225000 [362:19:09<78:25:07,  8.48s/it]
 85%|████████▌ | 191692/225000 [362:19:23<91:28:17,  9.89s/it]
 85%|████████▌ | 191693/225000 [362:19:32<88:57:39,  9.62s/it]
 85%|████████▌ | 191694/225000 [362:19:40<84:34:19,  9.14s/it]
 85%|████████▌ | 191695/225000 [362:19:46<76:30:13,  8.27s/it]
 85%|████████▌ | 191696/225000 [362:19:55<78:00:03,  8.43s/it]
 85%|████████▌ | 191697/225000 [362:20:06<85:29:39,  9.24s/it]
 85%|████████▌ | 191698/225000 [362:20:11<75:06:24,  8.12s/it]
 85%|████████▌ | 191699/225000 [362:20:20<76:23:11,  8.26s/it]
 85%|████████▌ | 191700/225000 [362:20:31<85:09:40,  9.21s/it]
                                                              
{'loss': '1.274', 'grad_norm': '1.238', 'learning_rate': '2.917e-06', 'epoch': '0.852', 'num_input_tokens_seen': 602876264, 'train_runtime': '1.304e+06', 'train_tokens_per_second': '462.2'}

 85%|████████▌ | 191700/225000 [362:20:31<85:09:40,  9.21s/it]
 85%|████████▌ | 191701/225000 [362:20:36<72:05:31,  7.79s/it]
 85%|████████▌ | 191702/225000 [362:20:43<71:16:10,  7.71s/it]
 85%|████████▌ | 191703/225000 [362:20:49<65:33:18,  7.09s/it]
 85%|████████▌ | 191704/225000 [362:20:56<64:47:30,  7.01s/it]
 85%|████████▌ | 191705/225000 [362:21:03<65:59:08,  7.13s/it]
 85%|████████▌ | 191706/225000 [362:21:14<75:57:23,  8.21s/it]
 85%|████████▌ | 191707/225000 [362:21:20<71:00:27,  7.68s/it]
 85%|████████▌ | 191708/225000 [362:21:28<70:28:20,  7.62s/it]
 85%|████████▌ | 191709/225000 [362:21:34<67:19:55,  7.28s/it]
 85%|████████▌ | 191710/225000 [362:21:45<76:30:39,  8.27s/it]
 85%|████████▌ | 191711/225000 [362:21:55<81:31:08,  8.82s/it]
 85%|████████▌ | 191712/225000 [362:22:01<73:37:19,  7.96s/it]
 85%|████████▌ | 191713/225000 [362:22:12<82:04:53,  8.88s/it]
 85%|████████▌ | 191714/225000 [362:22:19<78:32:49,  8.50s/it]
 85%|████████▌ | 191715/225000 [362:22:28<78:48:17,  8.52s/it]
 85%|████████▌ | 191716/225000 [362:22:34<70:15:36,  7.60s/it]
 85%|████████▌ | 191717/225000 [362:22:42<72:24:05,  7.83s/it]
 85%|████████▌ | 191718/225000 [362:22:52<77:57:04,  8.43s/it]
 85%|████████▌ | 191719/225000 [362:23:01<79:17:17,  8.58s/it]
 85%|████████▌ | 191720/225000 [362:23:07<74:21:07,  8.04s/it]
 85%|████████▌ | 191721/225000 [362:23:17<78:01:55,  8.44s/it]
 85%|████████▌ | 191722/225000 [362:23:24<73:38:57,  7.97s/it]
 85%|████████▌ | 191723/225000 [362:23:29<66:51:26,  7.23s/it]
 85%|████████▌ | 191724/225000 [362:23:39<73:17:49,  7.93s/it]
 85%|████████▌ | 191725/225000 [362:23:49<79:07:41,  8.56s/it]
 85%|████████▌ | 191726/225000 [362:23:59<82:40:28,  8.94s/it]
 85%|████████▌ | 191727/225000 [362:24:05<75:04:13,  8.12s/it]
 85%|████████▌ | 191728/225000 [362:24:16<82:37:19,  8.94s/it]
 85%|████████▌ | 191729/225000 [362:24:21<73:29:20,  7.95s/it]
 85%|████████▌ | 191730/225000 [362:24:26<64:10:20,  6.94s/it]
 85%|████████▌ | 191731/225000 [362:24:34<68:38:32,  7.43s/it]
 85%|████████▌ | 191732/225000 [362:24:44<75:34:43,  8.18s/it]
 85%|████████▌ | 191733/225000 [362:24:50<68:42:45,  7.44s/it]
 85%|████████▌ | 191734/225000 [362:24:58<68:53:56,  7.46s/it]
 85%|████████▌ | 191735/225000 [362:25:07<74:03:50,  8.02s/it]
 85%|████████▌ | 191736/225000 [362:25:19<84:58:11,  9.20s/it]
 85%|████████▌ | 191737/225000 [362:25:29<86:49:28,  9.40s/it]
 85%|████████▌ | 191738/225000 [362:25:37<84:26:13,  9.14s/it]
 85%|████████▌ | 191739/225000 [362:25:43<74:00:13,  8.01s/it]
 85%|████████▌ | 191740/225000 [362:25:48<65:29:51,  7.09s/it]
 85%|████████▌ | 191741/225000 [362:25:53<62:08:59,  6.73s/it]
 85%|████████▌ | 191742/225000 [362:26:07<81:03:46,  8.77s/it]
 85%|████████▌ | 191743/225000 [362:26:12<70:00:33,  7.58s/it]
 85%|████████▌ | 191744/225000 [362:26:20<71:59:02,  7.79s/it]
 85%|████████▌ | 191745/225000 [362:26:27<69:09:47,  7.49s/it]
 85%|████████▌ | 191746/225000 [362:26:33<65:48:21,  7.12s/it]
 85%|████████▌ | 191747/225000 [362:26:44<76:07:29,  8.24s/it]
 85%|████████▌ | 191748/225000 [362:26:56<86:27:54,  9.36s/it]
 85%|████████▌ | 191749/225000 [362:27:02<77:13:38,  8.36s/it]
 85%|████████▌ | 191750/225000 [362:27:08<71:51:00,  7.78s/it]
                                                              
{'loss': '1.243', 'grad_norm': '1.276', 'learning_rate': '2.909e-06', 'epoch': '0.8522', 'num_input_tokens_seen': 603084824, 'train_runtime': '1.305e+06', 'train_tokens_per_second': '462.2'}

 85%|████████▌ | 191750/225000 [362:27:08<71:51:00,  7.78s/it]
 85%|████████▌ | 191751/225000 [362:27:15<69:03:40,  7.48s/it]
 85%|████████▌ | 191752/225000 [362:27:28<83:19:04,  9.02s/it]
 85%|████████▌ | 191753/225000 [362:27:33<71:32:36,  7.75s/it]
 85%|████████▌ | 191754/225000 [362:27:40<70:38:17,  7.65s/it]
 85%|████████▌ | 191755/225000 [362:27:49<74:44:45,  8.09s/it]
 85%|████████▌ | 191756/225000 [362:27:59<79:56:00,  8.66s/it]
 85%|████████▌ | 191757/225000 [362:28:05<73:11:54,  7.93s/it]
 85%|████████▌ | 191758/225000 [362:28:14<76:16:04,  8.26s/it]
 85%|████████▌ | 191759/225000 [362:28:19<66:11:01,  7.17s/it]
 85%|████████▌ | 191760/225000 [362:28:25<64:14:20,  6.96s/it]
 85%|████████▌ | 191761/225000 [362:28:31<59:08:08,  6.40s/it]
 85%|████████▌ | 191762/225000 [362:28:35<53:21:42,  5.78s/it]
 85%|████████▌ | 191763/225000 [362:28:46<68:08:53,  7.38s/it]
 85%|████████▌ | 191764/225000 [362:28:54<70:54:13,  7.68s/it]
 85%|████████▌ | 191765/225000 [362:29:09<90:33:34,  9.81s/it]
 85%|████████▌ | 191766/225000 [362:29:13<74:58:23,  8.12s/it]
 85%|████████▌ | 191767/225000 [362:29:22<76:16:39,  8.26s/it]
 85%|████████▌ | 191768/225000 [362:29:30<76:52:28,  8.33s/it]
 85%|████████▌ | 191769/225000 [362:29:37<72:06:50,  7.81s/it]
 85%|████████▌ | 191770/225000 [362:29:43<66:20:49,  7.19s/it]
 85%|████████▌ | 191771/225000 [362:29:51<70:32:38,  7.64s/it]
 85%|████████▌ | 191772/225000 [362:30:02<77:15:36,  8.37s/it]
 85%|████████▌ | 191773/225000 [362:30:07<70:27:11,  7.63s/it]
 85%|████████▌ | 191774/225000 [362:30:13<65:07:44,  7.06s/it]
 85%|████████▌ | 191775/225000 [362:30:27<83:08:45,  9.01s/it]
 85%|████████▌ | 191776/225000 [362:30:34<77:49:03,  8.43s/it]
 85%|████████▌ | 191777/225000 [362:30:48<93:32:40, 10.14s/it]
 85%|████████▌ | 191778/225000 [362:30:55<85:51:37,  9.30s/it]
 85%|████████▌ | 191779/225000 [362:31:01<74:56:06,  8.12s/it]
 85%|████████▌ | 191780/225000 [362:31:11<81:20:05,  8.81s/it]
 85%|████████▌ | 191781/225000 [362:31:16<70:26:34,  7.63s/it]
 85%|████████▌ | 191782/225000 [362:31:27<80:23:03,  8.71s/it]
 85%|████████▌ | 191783/225000 [362:31:34<73:46:02,  7.99s/it]
 85%|████████▌ | 191784/225000 [362:31:45<83:05:50,  9.01s/it]
 85%|████████▌ | 191785/225000 [362:31:53<79:25:10,  8.61s/it]
 85%|████████▌ | 191786/225000 [362:32:00<74:51:59,  8.11s/it]
 85%|████████▌ | 191787/225000 [362:32:06<70:54:51,  7.69s/it]
 85%|████████▌ | 191788/225000 [362:32:19<84:58:07,  9.21s/it]
 85%|████████▌ | 191789/225000 [362:32:25<74:54:17,  8.12s/it]
 85%|████████▌ | 191790/225000 [362:32:34<77:33:07,  8.41s/it]
 85%|████████▌ | 191791/225000 [362:32:41<73:28:05,  7.96s/it]
 85%|████████▌ | 191792/225000 [362:32:46<65:49:22,  7.14s/it]
 85%|████████▌ | 191793/225000 [362:32:52<62:00:55,  6.72s/it]
 85%|████████▌ | 191794/225000 [362:33:05<81:05:30,  8.79s/it]
 85%|████████▌ | 191795/225000 [362:33:10<69:56:42,  7.58s/it]
 85%|████████▌ | 191796/225000 [362:33:18<71:04:49,  7.71s/it]
 85%|████████▌ | 191797/225000 [362:33:23<64:41:11,  7.01s/it]
 85%|████████▌ | 191798/225000 [362:33:31<66:59:55,  7.26s/it]
 85%|████████▌ | 191799/225000 [362:33:36<61:06:29,  6.63s/it]
 85%|████████▌ | 191800/225000 [362:33:50<80:51:22,  8.77s/it]
                                                              
{'loss': '1.257', 'grad_norm': '1.208', 'learning_rate': '2.9e-06', 'epoch': '0.8524', 'num_input_tokens_seen': 603294160, 'train_runtime': '1.305e+06', 'train_tokens_per_second': '462.2'}

 85%|████████▌ | 191800/225000 [362:33:50<80:51:22,  8.77s/it]
 85%|████████▌ | 191801/225000 [362:33:58<78:55:34,  8.56s/it]
 85%|████████▌ | 191802/225000 [362:34:05<74:30:32,  8.08s/it]
 85%|████████▌ | 191803/225000 [362:34:19<90:59:22,  9.87s/it]
 85%|████████▌ | 191804/225000 [362:34:27<84:31:51,  9.17s/it]
 85%|████████▌ | 191805/225000 [362:34:35<81:27:00,  8.83s/it]
 85%|████████▌ | 191806/225000 [362:34:43<79:48:38,  8.66s/it]
 85%|████████▌ | 191807/225000 [362:34:51<79:21:16,  8.61s/it]
 85%|████████▌ | 191808/225000 [362:35:00<79:38:24,  8.64s/it]
 85%|████████▌ | 191809/225000 [362:35:06<72:09:38,  7.83s/it]
 85%|████████▌ | 191810/225000 [362:35:12<68:02:57,  7.38s/it]
 85%|████████▌ | 191811/225000 [362:35:21<70:17:39,  7.62s/it]
 85%|████████▌ | 191812/225000 [362:35:30<74:25:14,  8.07s/it]
 85%|████████▌ | 191813/225000 [362:35:40<80:07:54,  8.69s/it]
 85%|████████▌ | 191814/225000 [362:35:47<77:04:27,  8.36s/it]
 85%|████████▌ | 191815/225000 [362:35:55<74:32:02,  8.09s/it]
 85%|████████▌ | 191816/225000 [362:36:04<78:04:33,  8.47s/it]
 85%|████████▌ | 191817/225000 [362:36:11<74:18:43,  8.06s/it]
 85%|████████▌ | 191818/225000 [362:36:23<84:58:42,  9.22s/it]
 85%|████████▌ | 191819/225000 [362:36:30<79:04:23,  8.58s/it]
 85%|████████▌ | 191820/225000 [362:36:43<89:49:54,  9.75s/it]
 85%|████████▌ | 191821/225000 [362:36:52<86:48:00,  9.42s/it]
 85%|████████▌ | 191822/225000 [362:37:01<85:59:41,  9.33s/it]
 85%|████████▌ | 191823/225000 [362:37:10<86:21:32,  9.37s/it]
 85%|████████▌ | 191824/225000 [362:37:22<94:16:48, 10.23s/it]
 85%|████████▌ | 191825/225000 [362:37:30<87:54:15,  9.54s/it]
 85%|████████▌ | 191826/225000 [362:37:35<74:52:03,  8.12s/it]
 85%|████████▌ | 191827/225000 [362:37:45<79:07:39,  8.59s/it]
 85%|████████▌ | 191828/225000 [362:37:52<74:44:56,  8.11s/it]
 85%|████████▌ | 191829/225000 [362:37:57<65:49:59,  7.14s/it]
 85%|████████▌ | 191830/225000 [362:38:02<60:24:02,  6.56s/it]
 85%|████████▌ | 191831/225000 [362:38:12<71:40:53,  7.78s/it]
 85%|████████▌ | 191832/225000 [362:38:21<73:12:46,  7.95s/it]
 85%|████████▌ | 191833/225000 [362:38:27<67:08:51,  7.29s/it]
 85%|████████▌ | 191834/225000 [362:38:38<78:59:02,  8.57s/it]
 85%|████████▌ | 191835/225000 [362:38:43<69:36:54,  7.56s/it]
 85%|████████▌ | 191836/225000 [362:38:55<81:03:11,  8.80s/it]
 85%|████████▌ | 191837/225000 [362:39:01<72:35:15,  7.88s/it]
 85%|████████▌ | 191838/225000 [362:39:10<75:00:08,  8.14s/it]
 85%|████████▌ | 191839/225000 [362:39:23<88:48:45,  9.64s/it]
 85%|████████▌ | 191840/225000 [362:39:31<85:42:38,  9.31s/it]
 85%|████████▌ | 191841/225000 [362:39:36<73:11:05,  7.95s/it]
 85%|████████▌ | 191842/225000 [362:39:43<71:41:19,  7.78s/it]
 85%|████████▌ | 191843/225000 [362:39:50<68:06:06,  7.39s/it]
 85%|████████▌ | 191844/225000 [362:40:00<75:00:58,  8.15s/it]
 85%|████████▌ | 191845/225000 [362:40:10<80:27:38,  8.74s/it]
 85%|████████▌ | 191846/225000 [362:40:17<76:50:16,  8.34s/it]
 85%|████████▌ | 191847/225000 [362:40:31<92:52:39, 10.09s/it]
 85%|████████▌ | 191848/225000 [362:40:36<77:51:11,  8.45s/it]
 85%|████████▌ | 191849/225000 [362:40:49<89:34:42,  9.73s/it]
 85%|████████▌ | 191850/225000 [362:40:58<89:11:20,  9.69s/it]
                                                              
{'loss': '1.259', 'grad_norm': '1.165', 'learning_rate': '2.892e-06', 'epoch': '0.8527', 'num_input_tokens_seen': 603514456, 'train_runtime': '1.306e+06', 'train_tokens_per_second': '462.2'}

 85%|████████▌ | 191850/225000 [362:40:58<89:11:20,  9.69s/it]
 85%|████████▌ | 191851/225000 [362:41:07<85:23:46,  9.27s/it]
 85%|████████▌ | 191852/225000 [362:41:11<72:35:15,  7.88s/it]
 85%|████████▌ | 191853/225000 [362:41:20<75:16:31,  8.18s/it]
 85%|████████▌ | 191854/225000 [362:41:31<83:32:13,  9.07s/it]
 85%|████████▌ | 191855/225000 [362:41:37<74:09:33,  8.05s/it]
 85%|████████▌ | 191856/225000 [362:41:54<97:28:55, 10.59s/it]
 85%|████████▌ | 191857/225000 [362:42:05<99:51:01, 10.85s/it]
 85%|████████▌ | 191858/225000 [362:42:09<80:04:38,  8.70s/it]
 85%|████████▌ | 191859/225000 [362:42:18<82:20:09,  8.94s/it]
 85%|████████▌ | 191860/225000 [362:42:26<80:01:51,  8.69s/it]
 85%|████████▌ | 191861/225000 [362:42:38<89:13:35,  9.69s/it]
 85%|████████▌ | 191862/225000 [362:42:43<74:55:16,  8.14s/it]
 85%|████████▌ | 191863/225000 [362:42:53<79:57:21,  8.69s/it]
 85%|████████▌ | 191864/225000 [362:42:58<71:08:21,  7.73s/it]
 85%|████████▌ | 191865/225000 [362:43:05<69:23:08,  7.54s/it]
 85%|████████▌ | 191866/225000 [362:43:12<67:10:49,  7.30s/it]
 85%|████████▌ | 191867/225000 [362:43:21<70:16:23,  7.64s/it]
 85%|████████▌ | 191868/225000 [362:43:28<69:53:28,  7.59s/it]
 85%|████████▌ | 191869/225000 [362:43:34<66:26:40,  7.22s/it]
 85%|████████▌ | 191870/225000 [362:43:41<63:44:25,  6.93s/it]
 85%|████████▌ | 191871/225000 [362:43:56<87:43:18,  9.53s/it]
 85%|████████▌ | 191872/225000 [362:44:01<75:00:55,  8.15s/it]
 85%|████████▌ | 191873/225000 [362:44:08<70:33:26,  7.67s/it]
 85%|████████▌ | 191874/225000 [362:44:19<79:18:58,  8.62s/it]
 85%|████████▌ | 191875/225000 [362:44:23<68:15:37,  7.42s/it]
 85%|████████▌ | 191876/225000 [362:44:34<76:44:39,  8.34s/it]
 85%|████████▌ | 191877/225000 [362:44:41<73:30:22,  7.99s/it]
 85%|████████▌ | 191878/225000 [362:44:50<75:39:55,  8.22s/it]
 85%|████████▌ | 191879/225000 [362:44:55<67:36:33,  7.35s/it]
 85%|████████▌ | 191880/225000 [362:45:00<62:11:46,  6.76s/it]
 85%|████████▌ | 191881/225000 [362:45:07<62:44:28,  6.82s/it]
 85%|████████▌ | 191882/225000 [362:45:19<77:36:19,  8.44s/it]
 85%|████████▌ | 191883/225000 [362:45:28<79:05:49,  8.60s/it]
 85%|████████▌ | 191884/225000 [362:45:32<66:00:52,  7.18s/it]
 85%|████████▌ | 191885/225000 [362:45:39<64:27:45,  7.01s/it]
 85%|████████▌ | 191886/225000 [362:45:45<61:50:37,  6.72s/it]
 85%|████████▌ | 191887/225000 [362:45:51<59:08:20,  6.43s/it]
 85%|████████▌ | 191888/225000 [362:46:06<84:28:10,  9.18s/it]
 85%|████████▌ | 191889/225000 [362:46:13<76:46:53,  8.35s/it]
 85%|████████▌ | 191890/225000 [362:46:18<67:55:04,  7.38s/it]
 85%|████████▌ | 191891/225000 [362:46:23<61:30:55,  6.69s/it]
 85%|████████▌ | 191892/225000 [362:46:36<78:08:27,  8.50s/it]
 85%|████████▌ | 191893/225000 [362:46:41<68:55:24,  7.49s/it]
 85%|████████▌ | 191894/225000 [362:46:52<78:59:30,  8.59s/it]
 85%|████████▌ | 191895/225000 [362:46:57<68:55:11,  7.49s/it]
 85%|████████▌ | 191896/225000 [362:47:06<72:07:40,  7.84s/it]
 85%|████████▌ | 191897/225000 [362:47:15<77:43:12,  8.45s/it]
 85%|████████▌ | 191898/225000 [362:47:24<79:09:33,  8.61s/it]
 85%|████████▌ | 191899/225000 [362:47:30<69:36:30,  7.57s/it]
 85%|████████▌ | 191900/225000 [362:47:35<64:12:02,  6.98s/it]
                                                              
{'loss': '1.248', 'grad_norm': '1.301', 'learning_rate': '2.883e-06', 'epoch': '0.8529', 'num_input_tokens_seen': 603726632, 'train_runtime': '1.306e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 191900/225000 [362:47:35<64:12:02,  6.98s/it]
 85%|████████▌ | 191901/225000 [362:47:45<73:15:54,  7.97s/it]
 85%|████████▌ | 191902/225000 [362:47:58<85:34:38,  9.31s/it]
 85%|████████▌ | 191903/225000 [362:48:09<90:59:46,  9.90s/it]
 85%|████████▌ | 191904/225000 [362:48:14<78:23:22,  8.53s/it]
 85%|████████▌ | 191905/225000 [362:48:20<69:11:42,  7.53s/it]
 85%|████████▌ | 191906/225000 [362:48:33<85:34:09,  9.31s/it]
 85%|████████▌ | 191907/225000 [362:48:39<75:54:41,  8.26s/it]
 85%|████████▌ | 191908/225000 [362:48:50<83:51:58,  9.12s/it]
 85%|████████▌ | 191909/225000 [362:48:58<81:30:26,  8.87s/it]
 85%|████████▌ | 191910/225000 [362:49:07<82:08:57,  8.94s/it]
 85%|████████▌ | 191911/225000 [362:49:13<72:04:19,  7.84s/it]
 85%|████████▌ | 191912/225000 [362:49:19<67:10:18,  7.31s/it]
 85%|████████▌ | 191913/225000 [362:49:30<78:10:59,  8.51s/it]
 85%|████████▌ | 191914/225000 [362:49:40<80:55:15,  8.80s/it]
 85%|████████▌ | 191915/225000 [362:49:50<86:14:01,  9.38s/it]
 85%|████████▌ | 191916/225000 [362:49:58<82:20:53,  8.96s/it]
 85%|████████▌ | 191917/225000 [362:50:03<70:05:40,  7.63s/it]
 85%|████████▌ | 191918/225000 [362:50:12<73:54:17,  8.04s/it]
 85%|████████▌ | 191919/225000 [362:50:18<68:43:21,  7.48s/it]
 85%|████████▌ | 191920/225000 [362:50:23<61:18:33,  6.67s/it]
 85%|████████▌ | 191921/225000 [362:50:29<61:10:27,  6.66s/it]
 85%|████████▌ | 191922/225000 [362:50:44<82:18:05,  8.96s/it]
 85%|████████▌ | 191923/225000 [362:50:48<69:28:54,  7.56s/it]
 85%|████████▌ | 191924/225000 [362:50:59<78:05:15,  8.50s/it]
 85%|████████▌ | 191925/225000 [362:51:04<69:19:05,  7.54s/it]
 85%|████████▌ | 191926/225000 [362:51:11<67:54:59,  7.39s/it]
 85%|████████▌ | 191927/225000 [362:51:18<67:43:47,  7.37s/it]
 85%|████████▌ | 191928/225000 [362:51:26<69:07:59,  7.53s/it]
 85%|████████▌ | 191929/225000 [362:51:32<63:55:43,  6.96s/it]
 85%|████████▌ | 191930/225000 [362:51:38<62:09:53,  6.77s/it]
 85%|████████▌ | 191931/225000 [362:51:53<83:41:53,  9.11s/it]
 85%|████████▌ | 191932/225000 [362:51:59<76:11:49,  8.30s/it]
 85%|████████▌ | 191933/225000 [362:52:07<74:43:34,  8.14s/it]
 85%|████████▌ | 191934/225000 [362:52:17<79:13:03,  8.62s/it]
 85%|████████▌ | 191935/225000 [362:52:24<74:36:19,  8.12s/it]
 85%|████████▌ | 191936/225000 [362:52:31<72:45:07,  7.92s/it]
 85%|████████▌ | 191937/225000 [362:52:38<70:21:56,  7.66s/it]
 85%|████████▌ | 191938/225000 [362:52:47<73:51:45,  8.04s/it]
 85%|████████▌ | 191939/225000 [362:52:59<84:06:24,  9.16s/it]
 85%|████████▌ | 191940/225000 [362:53:04<72:07:30,  7.85s/it]
 85%|████████▌ | 191941/225000 [362:53:13<77:27:51,  8.44s/it]
 85%|████████▌ | 191942/225000 [362:53:19<68:28:26,  7.46s/it]
 85%|████████▌ | 191943/225000 [362:53:28<73:32:21,  8.01s/it]
 85%|████████▌ | 191944/225000 [362:53:34<68:11:37,  7.43s/it]
 85%|████████▌ | 191945/225000 [362:53:44<75:12:05,  8.19s/it]
 85%|████████▌ | 191946/225000 [362:53:48<64:30:35,  7.03s/it]
 85%|████████▌ | 191947/225000 [362:53:59<74:10:56,  8.08s/it]
 85%|████████▌ | 191948/225000 [362:54:08<76:06:55,  8.29s/it]
 85%|████████▌ | 191949/225000 [362:54:16<76:50:57,  8.37s/it]
 85%|████████▌ | 191950/225000 [362:54:21<67:31:57,  7.36s/it]
                                                              
{'loss': '1.254', 'grad_norm': '1.453', 'learning_rate': '2.874e-06', 'epoch': '0.8531', 'num_input_tokens_seen': 603939808, 'train_runtime': '1.306e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 191950/225000 [362:54:21<67:31:57,  7.36s/it]
 85%|████████▌ | 191951/225000 [362:54:29<68:11:21,  7.43s/it]
 85%|████████▌ | 191952/225000 [362:54:42<84:23:06,  9.19s/it]
 85%|████████▌ | 191953/225000 [362:54:46<70:16:58,  7.66s/it]
 85%|████████▌ | 191954/225000 [362:54:54<70:31:11,  7.68s/it]
 85%|████████▌ | 191955/225000 [362:55:05<80:21:27,  8.75s/it]
 85%|████████▌ | 191956/225000 [362:55:13<76:42:19,  8.36s/it]
 85%|████████▌ | 191957/225000 [362:55:24<85:21:43,  9.30s/it]
 85%|████████▌ | 191958/225000 [362:55:31<79:16:41,  8.64s/it]
 85%|████████▌ | 191959/225000 [362:55:38<74:25:21,  8.11s/it]
 85%|████████▌ | 191960/225000 [362:55:44<68:40:24,  7.48s/it]
 85%|████████▌ | 191961/225000 [362:55:54<74:11:33,  8.08s/it]
 85%|████████▌ | 191962/225000 [362:56:01<72:48:07,  7.93s/it]
 85%|████████▌ | 191963/225000 [362:56:05<62:37:45,  6.82s/it]
 85%|████████▌ | 191964/225000 [362:56:11<59:50:31,  6.52s/it]
 85%|████████▌ | 191965/225000 [362:56:28<88:41:45,  9.67s/it]
 85%|████████▌ | 191966/225000 [362:56:36<84:02:10,  9.16s/it]
 85%|████████▌ | 191967/225000 [362:56:44<81:04:44,  8.84s/it]
 85%|████████▌ | 191968/225000 [362:56:50<73:46:33,  8.04s/it]
 85%|████████▌ | 191969/225000 [362:57:00<78:32:10,  8.56s/it]
 85%|████████▌ | 191970/225000 [362:57:09<78:42:07,  8.58s/it]
 85%|████████▌ | 191971/225000 [362:57:17<77:58:27,  8.50s/it]
 85%|████████▌ | 191972/225000 [362:57:30<89:11:55,  9.72s/it]
 85%|████████▌ | 191973/225000 [362:57:36<80:15:36,  8.75s/it]
 85%|████████▌ | 191974/225000 [362:57:42<71:12:14,  7.76s/it]
 85%|████████▌ | 191975/225000 [362:57:48<68:29:56,  7.47s/it]
 85%|████████▌ | 191976/225000 [362:57:56<68:18:48,  7.45s/it]
 85%|████████▌ | 191977/225000 [362:58:07<78:02:55,  8.51s/it]
 85%|████████▌ | 191978/225000 [362:58:15<77:32:40,  8.45s/it]
 85%|████████▌ | 191979/225000 [362:58:19<65:44:16,  7.17s/it]
 85%|████████▌ | 191980/225000 [362:58:26<63:40:02,  6.94s/it]
 85%|████████▌ | 191981/225000 [362:58:35<71:01:50,  7.74s/it]
 85%|████████▌ | 191982/225000 [362:58:42<68:51:47,  7.51s/it]
 85%|████████▌ | 191983/225000 [362:58:48<64:46:42,  7.06s/it]
 85%|████████▌ | 191984/225000 [362:58:55<64:29:14,  7.03s/it]
 85%|████████▌ | 191985/225000 [362:59:00<58:36:48,  6.39s/it]
 85%|████████▌ | 191986/225000 [362:59:14<78:47:31,  8.59s/it]
 85%|████████▌ | 191987/225000 [362:59:24<82:00:37,  8.94s/it]
 85%|████████▌ | 191988/225000 [362:59:30<75:36:45,  8.25s/it]
 85%|████████▌ | 191989/225000 [362:59:41<81:53:05,  8.93s/it]
 85%|████████▌ | 191990/225000 [362:59:46<70:30:33,  7.69s/it]
 85%|████████▌ | 191991/225000 [362:59:59<86:15:41,  9.41s/it]
 85%|████████▌ | 191992/225000 [363:00:06<79:11:16,  8.64s/it]
 85%|████████▌ | 191993/225000 [363:00:13<75:49:52,  8.27s/it]
 85%|████████▌ | 191994/225000 [363:00:24<82:34:26,  9.01s/it]
 85%|████████▌ | 191995/225000 [363:00:32<79:47:10,  8.70s/it]
 85%|████████▌ | 191996/225000 [363:00:37<69:59:27,  7.63s/it]
 85%|████████▌ | 191997/225000 [363:00:47<75:08:44,  8.20s/it]
 85%|████████▌ | 191998/225000 [363:01:02<94:21:14, 10.29s/it]
 85%|████████▌ | 191999/225000 [363:01:07<79:21:16,  8.66s/it]
 85%|████████▌ | 192000/225000 [363:01:14<77:06:05,  8.41s/it]
                                                              
{'loss': '1.254', 'grad_norm': '1.013', 'learning_rate': '2.866e-06', 'epoch': '0.8533', 'num_input_tokens_seen': 604154592, 'train_runtime': '1.307e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 192000/225000 [363:01:14<77:06:05,  8.41s/it]
 85%|████████▌ | 192001/225000 [363:01:26<85:21:15,  9.31s/it]
 85%|████████▌ | 192002/225000 [363:01:37<89:00:45,  9.71s/it]
 85%|████████▌ | 192003/225000 [363:01:41<74:54:09,  8.17s/it]
 85%|████████▌ | 192004/225000 [363:01:51<79:04:18,  8.63s/it]
 85%|████████▌ | 192005/225000 [363:01:59<77:34:47,  8.46s/it]
 85%|████████▌ | 192006/225000 [363:02:06<72:31:59,  7.91s/it]
 85%|████████▌ | 192007/225000 [363:02:15<76:01:37,  8.30s/it]
 85%|████████▌ | 192008/225000 [363:02:22<73:25:14,  8.01s/it]
 85%|████████▌ | 192009/225000 [363:02:27<63:40:56,  6.95s/it]
 85%|████████▌ | 192010/225000 [363:02:37<73:39:05,  8.04s/it]
 85%|████████▌ | 192011/225000 [363:02:42<65:31:23,  7.15s/it]
 85%|████████▌ | 192012/225000 [363:02:50<67:52:19,  7.41s/it]
 85%|████████▌ | 192013/225000 [363:03:00<74:07:27,  8.09s/it]
 85%|████████▌ | 192014/225000 [363:03:04<62:28:21,  6.82s/it]
 85%|████████▌ | 192015/225000 [363:03:14<72:03:28,  7.86s/it]
 85%|████████▌ | 192016/225000 [363:03:22<71:30:36,  7.80s/it]
 85%|████████▌ | 192017/225000 [363:03:29<69:40:25,  7.60s/it]
 85%|████████▌ | 192018/225000 [363:03:38<73:42:01,  8.04s/it]
 85%|████████▌ | 192019/225000 [363:03:46<72:50:30,  7.95s/it]
 85%|████████▌ | 192020/225000 [363:03:58<84:21:46,  9.21s/it]
 85%|████████▌ | 192021/225000 [363:04:04<76:06:46,  8.31s/it]
 85%|████████▌ | 192022/225000 [363:04:14<81:57:52,  8.95s/it]
 85%|████████▌ | 192023/225000 [363:04:23<81:12:45,  8.87s/it]
 85%|████████▌ | 192024/225000 [363:04:35<89:08:33,  9.73s/it]
 85%|████████▌ | 192025/225000 [363:04:39<73:29:10,  8.02s/it]
 85%|████████▌ | 192026/225000 [363:04:51<84:06:42,  9.18s/it]
 85%|████████▌ | 192027/225000 [363:04:56<73:43:28,  8.05s/it]
 85%|████████▌ | 192028/225000 [363:05:06<79:57:47,  8.73s/it]
 85%|████████▌ | 192029/225000 [363:05:12<72:06:10,  7.87s/it]
 85%|████████▌ | 192030/225000 [363:05:18<64:37:36,  7.06s/it]
 85%|████████▌ | 192031/225000 [363:05:29<77:42:24,  8.49s/it]
 85%|████████▌ | 192032/225000 [363:05:35<69:05:13,  7.54s/it]
 85%|████████▌ | 192033/225000 [363:05:46<78:38:39,  8.59s/it]
 85%|████████▌ | 192034/225000 [363:05:57<85:57:48,  9.39s/it]
 85%|████████▌ | 192035/225000 [363:06:07<89:00:51,  9.72s/it]
 85%|████████▌ | 192036/225000 [363:06:13<76:48:04,  8.39s/it]
 85%|████████▌ | 192037/225000 [363:06:20<72:33:07,  7.92s/it]
 85%|████████▌ | 192038/225000 [363:06:30<79:49:44,  8.72s/it]
 85%|████████▌ | 192039/225000 [363:06:37<74:52:58,  8.18s/it]
 85%|████████▌ | 192040/225000 [363:06:43<68:16:20,  7.46s/it]
 85%|████████▌ | 192041/225000 [363:06:49<65:08:17,  7.11s/it]
 85%|████████▌ | 192042/225000 [363:06:54<59:26:29,  6.49s/it]
 85%|████████▌ | 192043/225000 [363:07:06<73:45:47,  8.06s/it]
 85%|████████▌ | 192044/225000 [363:07:15<76:23:54,  8.35s/it]
 85%|████████▌ | 192045/225000 [363:07:23<74:17:30,  8.12s/it]
 85%|████████▌ | 192046/225000 [363:07:35<86:00:29,  9.40s/it]
 85%|████████▌ | 192047/225000 [363:07:41<76:04:14,  8.31s/it]
 85%|████████▌ | 192048/225000 [363:07:51<81:37:20,  8.92s/it]
 85%|████████▌ | 192049/225000 [363:07:56<70:47:26,  7.73s/it]
 85%|████████▌ | 192050/225000 [363:08:02<66:13:06,  7.23s/it]
                                                              
{'loss': '1.268', 'grad_norm': '1.427', 'learning_rate': '2.858e-06', 'epoch': '0.8536', 'num_input_tokens_seen': 604367664, 'train_runtime': '1.307e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 192050/225000 [363:08:02<66:13:06,  7.23s/it]
 85%|████████▌ | 192051/225000 [363:08:07<61:06:33,  6.68s/it]
 85%|████████▌ | 192052/225000 [363:08:13<59:08:38,  6.46s/it]
 85%|████████▌ | 192053/225000 [363:08:25<73:27:43,  8.03s/it]
 85%|████████▌ | 192054/225000 [363:08:41<94:41:10, 10.35s/it]
 85%|████████▌ | 192055/225000 [363:08:46<80:46:57,  8.83s/it]
 85%|████████▌ | 192056/225000 [363:08:53<76:46:20,  8.39s/it]
 85%|████████▌ | 192057/225000 [363:09:01<73:57:04,  8.08s/it]
 85%|████████▌ | 192058/225000 [363:09:06<67:15:02,  7.35s/it]
 85%|████████▌ | 192059/225000 [363:09:13<64:28:08,  7.05s/it]
 85%|████████▌ | 192060/225000 [363:09:22<71:22:32,  7.80s/it]
 85%|████████▌ | 192061/225000 [363:09:32<76:02:35,  8.31s/it]
 85%|████████▌ | 192062/225000 [363:09:41<77:46:09,  8.50s/it]
 85%|████████▌ | 192063/225000 [363:09:47<72:37:06,  7.94s/it]
 85%|████████▌ | 192064/225000 [363:09:56<73:59:10,  8.09s/it]
 85%|████████▌ | 192065/225000 [363:10:05<75:44:05,  8.28s/it]
 85%|████████▌ | 192066/225000 [363:10:12<72:44:20,  7.95s/it]
 85%|████████▌ | 192067/225000 [363:10:17<64:22:23,  7.04s/it]
 85%|████████▌ | 192068/225000 [363:10:26<71:09:31,  7.78s/it]
 85%|████████▌ | 192069/225000 [363:10:38<82:19:28,  9.00s/it]
 85%|████████▌ | 192070/225000 [363:10:42<69:45:27,  7.63s/it]
 85%|████████▌ | 192071/225000 [363:10:56<86:56:55,  9.51s/it]
 85%|████████▌ | 192072/225000 [363:11:07<89:49:37,  9.82s/it]
 85%|████████▌ | 192073/225000 [363:11:13<78:49:09,  8.62s/it]
 85%|████████▌ | 192074/225000 [363:11:19<71:02:24,  7.77s/it]
 85%|████████▌ | 192075/225000 [363:11:23<62:38:09,  6.85s/it]
 85%|████████▌ | 192076/225000 [363:11:32<67:34:34,  7.39s/it]
 85%|████████▌ | 192077/225000 [363:11:41<71:49:44,  7.85s/it]
 85%|████████▌ | 192078/225000 [363:11:51<79:36:55,  8.71s/it]
 85%|████████▌ | 192079/225000 [363:11:57<71:16:44,  7.79s/it]
 85%|████████▌ | 192080/225000 [363:12:07<75:38:03,  8.27s/it]
 85%|████████▌ | 192081/225000 [363:12:15<75:50:15,  8.29s/it]
 85%|████████▌ | 192082/225000 [363:12:23<76:12:39,  8.33s/it]
 85%|████████▌ | 192083/225000 [363:12:30<71:44:20,  7.85s/it]
 85%|████████▌ | 192084/225000 [363:12:37<69:43:59,  7.63s/it]
 85%|████████▌ | 192085/225000 [363:12:49<80:52:36,  8.85s/it]
 85%|████████▌ | 192086/225000 [363:12:55<72:44:21,  7.96s/it]
 85%|████████▌ | 192087/225000 [363:13:05<80:05:22,  8.76s/it]
 85%|████████▌ | 192088/225000 [363:13:14<79:49:31,  8.73s/it]
 85%|████████▌ | 192089/225000 [363:13:25<86:03:51,  9.41s/it]
 85%|████████▌ | 192090/225000 [363:13:33<81:15:57,  8.89s/it]
 85%|████████▌ | 192091/225000 [363:13:37<69:15:51,  7.58s/it]
 85%|████████▌ | 192092/225000 [363:13:46<73:24:19,  8.03s/it]
 85%|████████▌ | 192093/225000 [363:14:07<107:03:22, 11.71s/it]
 85%|████████▌ | 192094/225000 [363:14:13<91:49:27, 10.05s/it] 
 85%|████████▌ | 192095/225000 [363:14:18<79:50:21,  8.73s/it]
 85%|████████▌ | 192096/225000 [363:14:25<75:08:55,  8.22s/it]
 85%|████████▌ | 192097/225000 [363:14:33<73:33:52,  8.05s/it]
 85%|████████▌ | 192098/225000 [363:14:45<85:26:53,  9.35s/it]
 85%|████████▌ | 192099/225000 [363:14:52<76:45:37,  8.40s/it]
 85%|████████▌ | 192100/225000 [363:15:01<78:45:24,  8.62s/it]
                                                              
{'loss': '1.27', 'grad_norm': '1.216', 'learning_rate': '2.849e-06', 'epoch': '0.8538', 'num_input_tokens_seen': 604582488, 'train_runtime': '1.308e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 192100/225000 [363:15:01<78:45:24,  8.62s/it]
 85%|████████▌ | 192101/225000 [363:15:07<71:50:41,  7.86s/it]
 85%|████████▌ | 192102/225000 [363:15:13<67:08:21,  7.35s/it]
 85%|████████▌ | 192103/225000 [363:15:20<64:53:41,  7.10s/it]
 85%|████████▌ | 192104/225000 [363:15:31<78:04:44,  8.54s/it]
 85%|████████▌ | 192105/225000 [363:15:39<73:59:15,  8.10s/it]
 85%|████████▌ | 192106/225000 [363:15:49<80:07:46,  8.77s/it]
 85%|████████▌ | 192107/225000 [363:15:55<72:57:36,  7.99s/it]
 85%|████████▌ | 192108/225000 [363:16:08<86:47:41,  9.50s/it]
 85%|████████▌ | 192109/225000 [363:16:16<81:18:40,  8.90s/it]
 85%|████████▌ | 192110/225000 [363:16:20<68:25:37,  7.49s/it]
 85%|████████▌ | 192111/225000 [363:16:27<69:06:23,  7.56s/it]
 85%|████████▌ | 192112/225000 [363:16:33<62:13:40,  6.81s/it]
 85%|████████▌ | 192113/225000 [363:16:38<58:24:42,  6.39s/it]
 85%|████████▌ | 192114/225000 [363:16:46<62:05:08,  6.80s/it]
 85%|████████▌ | 192115/225000 [363:16:57<73:13:29,  8.02s/it]
 85%|████████▌ | 192116/225000 [363:17:06<76:14:11,  8.35s/it]
 85%|████████▌ | 192117/225000 [363:17:12<71:16:24,  7.80s/it]
 85%|████████▌ | 192118/225000 [363:17:25<85:34:12,  9.37s/it]
 85%|████████▌ | 192119/225000 [363:17:30<73:15:16,  8.02s/it]
 85%|████████▌ | 192120/225000 [363:17:40<77:17:43,  8.46s/it]
 85%|████████▌ | 192121/225000 [363:17:46<72:39:58,  7.96s/it]
 85%|████████▌ | 192122/225000 [363:17:52<65:01:37,  7.12s/it]
 85%|████████▌ | 192123/225000 [363:18:04<78:29:58,  8.60s/it]
 85%|████████▌ | 192124/225000 [363:18:10<72:42:28,  7.96s/it]
 85%|████████▌ | 192125/225000 [363:18:15<64:50:15,  7.10s/it]
 85%|████████▌ | 192126/225000 [363:18:25<71:52:27,  7.87s/it]
 85%|████████▌ | 192127/225000 [363:18:38<86:17:09,  9.45s/it]
 85%|████████▌ | 192128/225000 [363:18:46<82:11:21,  9.00s/it]
 85%|████████▌ | 192129/225000 [363:18:51<70:49:57,  7.76s/it]
 85%|████████▌ | 192130/225000 [363:19:01<77:48:22,  8.52s/it]
 85%|████████▌ | 192131/225000 [363:19:06<67:03:34,  7.34s/it]
 85%|████████▌ | 192132/225000 [363:19:17<76:58:00,  8.43s/it]
 85%|████████▌ | 192133/225000 [363:19:26<80:11:44,  8.78s/it]
 85%|████████▌ | 192134/225000 [363:19:34<76:57:35,  8.43s/it]
 85%|████████▌ | 192135/225000 [363:19:46<85:51:44,  9.41s/it]
 85%|████████▌ | 192136/225000 [363:19:51<76:17:38,  8.36s/it]
 85%|████████▌ | 192137/225000 [363:19:59<74:37:19,  8.17s/it]
 85%|████████▌ | 192138/225000 [363:20:11<84:48:38,  9.29s/it]
 85%|████████▌ | 192139/225000 [363:20:16<73:11:47,  8.02s/it]
 85%|████████▌ | 192140/225000 [363:20:26<79:21:02,  8.69s/it]
 85%|████████▌ | 192141/225000 [363:20:33<74:03:20,  8.11s/it]
 85%|████████▌ | 192142/225000 [363:20:42<76:52:45,  8.42s/it]
 85%|████████▌ | 192143/225000 [363:20:51<76:53:09,  8.42s/it]
 85%|████████▌ | 192144/225000 [363:20:57<69:54:20,  7.66s/it]
 85%|████████▌ | 192145/225000 [363:21:04<68:23:12,  7.49s/it]
 85%|████████▌ | 192146/225000 [363:21:11<68:57:52,  7.56s/it]
 85%|████████▌ | 192147/225000 [363:21:18<66:30:14,  7.29s/it]
 85%|████████▌ | 192148/225000 [363:21:30<79:14:19,  8.68s/it]
 85%|████████▌ | 192149/225000 [363:21:36<72:08:57,  7.91s/it]
 85%|████████▌ | 192150/225000 [363:21:43<69:38:09,  7.63s/it]
                                                              
{'loss': '1.266', 'grad_norm': '1.43', 'learning_rate': '2.841e-06', 'epoch': '0.854', 'num_input_tokens_seen': 604800544, 'train_runtime': '1.308e+06', 'train_tokens_per_second': '462.3'}

 85%|████████▌ | 192150/225000 [363:21:43<69:38:09,  7.63s/it]
 85%|████████▌ | 192151/225000 [363:21:53<75:54:46,  8.32s/it]
 85%|████████▌ | 192152/225000 [363:21:58<65:34:56,  7.19s/it]
 85%|████████▌ | 192153/225000 [363:22:03<60:37:45,  6.64s/it]
 85%|████████▌ | 192154/225000 [363:22:14<73:06:40,  8.01s/it]
 85%|████████▌ | 192155/225000 [363:22:24<77:06:29,  8.45s/it]
 85%|████████▌ | 192156/225000 [363:22:29<69:21:31,  7.60s/it]
 85%|████████▌ | 192157/225000 [363:22:35<64:49:35,  7.11s/it]
 85%|████████▌ | 192158/225000 [363:22:46<75:08:37,  8.24s/it]
 85%|████████▌ | 192159/225000 [363:22:51<67:03:31,  7.35s/it]
 85%|████████▌ | 192160/225000 [363:22:56<58:50:42,  6.45s/it]
 85%|████████▌ | 192161/225000 [363:23:08<74:40:02,  8.19s/it]
 85%|████████▌ | 192162/225000 [363:23:18<80:33:11,  8.83s/it]
 85%|████████▌ | 192163/225000 [363:23:24<73:21:23,  8.04s/it]
 85%|████████▌ | 192164/225000 [363:23:31<69:56:04,  7.67s/it]
 85%|████████▌ | 192165/225000 [363:23:41<76:10:19,  8.35s/it]
 85%|████████▌ | 192166/225000 [363:23:48<70:58:22,  7.78s/it]
 85%|████████▌ | 192167/225000 [363:23:54<66:52:01,  7.33s/it]
 85%|████████▌ | 192168/225000 [363:24:07<81:51:07,  8.98s/it]
 85%|████████▌ | 192169/225000 [363:24:12<71:50:46,  7.88s/it]
 85%|████████▌ | 192170/225000 [363:24:21<73:44:07,  8.09s/it]
 85%|████████▌ | 192171/225000 [363:24:36<92:48:57, 10.18s/it]
 85%|████████▌ | 192172/225000 [363:24:40<75:57:03,  8.33s/it]
 85%|████████▌ | 192173/225000 [363:24:50<81:48:34,  8.97s/it]
 85%|████████▌ | 192174/225000 [363:24:56<72:14:00,  7.92s/it]
 85%|████████▌ | 192175/225000 [363:25:07<82:41:23,  9.07s/it]
 85%|████████▌ | 192176/225000 [363:25:16<80:15:37,  8.80s/it]
 85%|████████▌ | 192177/225000 [363:25:23<76:04:59,  8.34s/it]
 85%|████████▌ | 192178/225000 [363:25:34<83:15:27,  9.13s/it]
 85%|████████▌ | 192179/225000 [363:25:45<89:44:12,  9.84s/it]
 85%|████████▌ | 192180/225000 [363:25:52<82:18:33,  9.03s/it]
 85%|████████▌ | 192181/225000 [363:25:58<73:33:54,  8.07s/it]
 85%|████████▌ | 192182/225000 [363:26:08<77:02:52,  8.45s/it]
 85%|████████▌ | 192183/225000 [363:26:15<73:02:53,  8.01s/it]
 85%|████████▌ | 192184/225000 [363:26:22<70:02:20,  7.68s/it]
 85%|████████▌ | 192185/225000 [363:26:28<67:29:31,  7.40s/it]
 85%|████████▌ | 192186/225000 [363:26:36<68:23:24,  7.50s/it]
 85%|████████▌ | 192187/225000 [363:26:41<60:28:22,  6.63s/it]
 85%|████████▌ | 192188/225000 [363:26:49<64:49:15,  7.11s/it]
 85%|████████▌ | 192189/225000 [363:26:55<62:40:47,  6.88s/it]
 85%|████████▌ | 192190/225000 [363:27:10<85:37:09,  9.39s/it]
 85%|████████▌ | 192191/225000 [363:27:19<82:34:09,  9.06s/it]
 85%|████████▌ | 192192/225000 [363:27:28<82:12:58,  9.02s/it]
 85%|████████▌ | 192193/225000 [363:27:36<79:30:42,  8.73s/it]
 85%|████████▌ | 192194/225000 [363:27:43<76:51:56,  8.43s/it]
 85%|████████▌ | 192195/225000 [363:27:50<72:09:42,  7.92s/it]
 85%|████████▌ | 192196/225000 [363:27:56<66:37:10,  7.31s/it]
 85%|████████▌ | 192197/225000 [363:28:07<75:55:58,  8.33s/it]
 85%|████████▌ | 192198/225000 [363:28:15<74:53:20,  8.22s/it]
 85%|████████▌ | 192199/225000 [363:28:25<79:47:26,  8.76s/it]
 85%|████████▌ | 192200/225000 [363:28:31<71:45:25,  7.88s/it]
                                                              
{'loss': '1.27', 'grad_norm': '1.472', 'learning_rate': '2.832e-06', 'epoch': '0.8542', 'num_input_tokens_seen': 605014760, 'train_runtime': '1.309e+06', 'train_tokens_per_second': '462.4'}

 85%|████████▌ | 192200/225000 [363:28:31<71:45:25,  7.88s/it]
 85%|████████▌ | 192201/225000 [363:28:40<75:42:39,  8.31s/it]
 85%|████████▌ | 192202/225000 [363:28:49<77:08:35,  8.47s/it]
 85%|████████▌ | 192203/225000 [363:28:58<78:28:23,  8.61s/it]
 85%|████████▌ | 192204/225000 [363:29:05<74:03:59,  8.13s/it]
 85%|████████▌ | 192205/225000 [363:29:10<67:33:52,  7.42s/it]
 85%|████████▌ | 192206/225000 [363:29:16<62:12:55,  6.83s/it]
 85%|████████▌ | 192207/225000 [363:29:29<79:15:20,  8.70s/it]
 85%|████████▌ | 192208/225000 [363:29:36<73:49:34,  8.10s/it]
 85%|████████▌ | 192209/225000 [363:29:45<77:30:36,  8.51s/it]
 85%|████████▌ | 192210/225000 [363:29:50<67:05:59,  7.37s/it]
 85%|████████▌ | 192211/225000 [363:29:55<61:00:27,  6.70s/it]
 85%|████████▌ | 192212/225000 [363:30:08<77:36:15,  8.52s/it]
 85%|████████▌ | 192213/225000 [363:30:17<78:46:03,  8.65s/it]
 85%|████████▌ | 192214/225000 [363:30:24<74:36:34,  8.19s/it]
 85%|████████▌ | 192215/225000 [363:30:28<63:47:50,  7.01s/it]
 85%|████████▌ | 192216/225000 [363:30:43<85:45:52,  9.42s/it]
 85%|████████▌ | 192217/225000 [363:30:47<70:18:24,  7.72s/it]
 85%|████████▌ | 192218/225000 [363:30:55<70:01:13,  7.69s/it]
 85%|████████▌ | 192219/225000 [363:31:04<75:05:32,  8.25s/it]
 85%|████████▌ | 192220/225000 [363:31:19<93:47:10, 10.30s/it]
 85%|████████▌ | 192221/225000 [363:31:26<85:15:57,  9.36s/it]
 85%|████████▌ | 192222/225000 [363:31:32<76:27:11,  8.40s/it]
 85%|████████▌ | 192223/225000 [363:31:38<69:10:19,  7.60s/it]
 85%|████████▌ | 192224/225000 [363:31:48<76:26:48,  8.40s/it]
 85%|████████▌ | 192225/225000 [363:31:55<71:55:20,  7.90s/it]
 85%|████████▌ | 192226/225000 [363:32:00<62:59:32,  6.92s/it]
 85%|████████▌ | 192227/225000 [363:32:05<59:20:53,  6.52s/it]
 85%|████████▌ | 192228/225000 [363:32:13<61:12:28,  6.72s/it]
 85%|████████▌ | 192229/225000 [363:32:19<59:35:01,  6.55s/it]
 85%|████████▌ | 192230/225000 [363:32:29<69:55:05,  7.68s/it]
 85%|████████▌ | 192231/225000 [363:32:41<80:42:44,  8.87s/it]
 85%|████████▌ | 192232/225000 [363:32:46<71:28:02,  7.85s/it]
 85%|████████▌ | 192233/225000 [363:32:50<60:03:54,  6.60s/it]
 85%|████████▌ | 192234/225000 [363:33:07<89:50:08,  9.87s/it]
 85%|████████▌ | 192235/225000 [363:33:17<88:41:35,  9.75s/it]
 85%|████████▌ | 192236/225000 [363:33:26<86:25:10,  9.50s/it]
 85%|████████▌ | 192237/225000 [363:33:34<83:15:06,  9.15s/it]
 85%|████████▌ | 192238/225000 [363:33:41<76:13:39,  8.38s/it]
 85%|████████▌ | 192239/225000 [363:33:47<71:13:43,  7.83s/it]
 85%|████████▌ | 192240/225000 [363:33:53<66:47:50,  7.34s/it]
 85%|████████▌ | 192241/225000 [363:34:05<78:15:39,  8.60s/it]
 85%|████████▌ | 192242/225000 [363:34:11<71:37:12,  7.87s/it]
 85%|████████▌ | 192243/225000 [363:34:21<77:16:15,  8.49s/it]
 85%|████████▌ | 192244/225000 [363:34:27<69:08:49,  7.60s/it]
 85%|████████▌ | 192245/225000 [363:34:32<62:37:57,  6.88s/it]
 85%|████████▌ | 192246/225000 [363:34:37<57:58:48,  6.37s/it]
 85%|████████▌ | 192247/225000 [363:34:44<60:29:40,  6.65s/it]
 85%|████████▌ | 192248/225000 [363:34:50<58:05:18,  6.38s/it]
 85%|████████▌ | 192249/225000 [363:35:00<69:07:47,  7.60s/it]
 85%|████████▌ | 192250/225000 [363:35:08<70:10:02,  7.71s/it]
                                                              
{'loss': '1.252', 'grad_norm': '1.578', 'learning_rate': '2.824e-06', 'epoch': '0.8544', 'num_input_tokens_seen': 605221808, 'train_runtime': '1.309e+06', 'train_tokens_per_second': '462.4'}

 85%|████████▌ | 192250/225000 [363:35:08<70:10:02,  7.71s/it]
 85%|████████▌ | 192251/225000 [363:35:17<73:39:59,  8.10s/it]
 85%|████████▌ | 192252/225000 [363:35:23<65:35:30,  7.21s/it]
 85%|████████▌ | 192253/225000 [363:35:28<61:35:31,  6.77s/it]
 85%|████████▌ | 192254/225000 [363:35:38<69:34:42,  7.65s/it]
 85%|████████▌ | 192255/225000 [363:35:46<69:29:11,  7.64s/it]
 85%|████████▌ | 192256/225000 [363:35:49<58:41:53,  6.45s/it]
 85%|████████▌ | 192257/225000 [363:35:59<67:30:41,  7.42s/it]
 85%|████████▌ | 192258/225000 [363:36:08<71:25:37,  7.85s/it]
 85%|████████▌ | 192259/225000 [363:36:17<74:13:26,  8.16s/it]
 85%|████████▌ | 192260/225000 [363:36:21<64:01:12,  7.04s/it]
 85%|████████▌ | 192261/225000 [363:36:30<68:06:32,  7.49s/it]
 85%|████████▌ | 192262/225000 [363:36:34<60:37:14,  6.67s/it]
 85%|████████▌ | 192263/225000 [363:36:45<70:10:29,  7.72s/it]
 85%|████████▌ | 192264/225000 [363:36:52<70:31:49,  7.76s/it]
 85%|████████▌ | 192265/225000 [363:37:00<69:37:06,  7.66s/it]
 85%|████████▌ | 192266/225000 [363:37:09<73:04:12,  8.04s/it]
 85%|████████▌ | 192267/225000 [363:37:22<86:48:25,  9.55s/it]
 85%|████████▌ | 192268/225000 [363:37:29<81:23:22,  8.95s/it]
 85%|████████▌ | 192269/225000 [363:37:39<83:30:53,  9.19s/it]
 85%|████████▌ | 192270/225000 [363:37:47<80:38:35,  8.87s/it]
 85%|████████▌ | 192271/225000 [363:37:53<71:50:01,  7.90s/it]
 85%|████████▌ | 192272/225000 [363:37:59<65:44:51,  7.23s/it]
 85%|████████▌ | 192273/225000 [363:38:10<76:25:15,  8.41s/it]
 85%|████████▌ | 192274/225000 [363:38:15<68:53:49,  7.58s/it]
 85%|████████▌ | 192275/225000 [363:38:20<60:37:11,  6.67s/it]
 85%|████████▌ | 192276/225000 [363:38:27<62:32:59,  6.88s/it]
 85%|████████▌ | 192277/225000 [363:38:35<65:22:54,  7.19s/it]
 85%|████████▌ | 192278/225000 [363:38:47<76:45:38,  8.45s/it]
 85%|████████▌ | 192279/225000 [363:38:56<80:21:11,  8.84s/it]
 85%|████████▌ | 192280/225000 [363:39:08<88:06:05,  9.69s/it]
 85%|████████▌ | 192281/225000 [363:39:15<79:55:10,  8.79s/it]
 85%|████████▌ | 192282/225000 [363:39:22<75:25:47,  8.30s/it]
 85%|████████▌ | 192283/225000 [363:39:32<81:29:59,  8.97s/it]
 85%|████████▌ | 192284/225000 [363:39:38<70:56:53,  7.81s/it]
 85%|████████▌ | 192285/225000 [363:39:50<84:12:21,  9.27s/it]
 85%|████████▌ | 192286/225000 [363:39:55<73:02:21,  8.04s/it]
 85%|████████▌ | 192287/225000 [363:40:00<62:36:28,  6.89s/it]
 85%|████████▌ | 192288/225000 [363:40:18<93:27:20, 10.28s/it]
 85%|████████▌ | 192289/225000 [363:40:22<78:12:41,  8.61s/it]
 85%|████████▌ | 192290/225000 [363:40:34<86:31:29,  9.52s/it]
 85%|████████▌ | 192291/225000 [363:40:41<79:21:42,  8.73s/it]
 85%|████████▌ | 192292/225000 [363:40:45<67:01:26,  7.38s/it]
 85%|████████▌ | 192293/225000 [363:40:55<73:00:23,  8.04s/it]
 85%|████████▌ | 192294/225000 [363:41:03<72:59:50,  8.03s/it]
 85%|████████▌ | 192295/225000 [363:41:11<73:04:46,  8.04s/it]
 85%|████████▌ | 192296/225000 [363:41:22<80:00:15,  8.81s/it]
 85%|████████▌ | 192297/225000 [363:41:29<76:10:10,  8.38s/it]
 85%|████████▌ | 192298/225000 [363:41:38<77:41:12,  8.55s/it]
 85%|████████▌ | 192299/225000 [363:41:45<72:38:42,  8.00s/it]
 85%|████████▌ | 192300/225000 [363:41:50<66:32:28,  7.33s/it]
                                                              
{'loss': '1.264', 'grad_norm': '1.392', 'learning_rate': '2.815e-06', 'epoch': '0.8547', 'num_input_tokens_seen': 605435640, 'train_runtime': '1.309e+06', 'train_tokens_per_second': '462.4'}

 85%|████████▌ | 192300/225000 [363:41:50<66:32:28,  7.33s/it]
 85%|████████▌ | 192301/225000 [363:42:00<74:00:25,  8.15s/it]
 85%|████████▌ | 192302/225000 [363:42:13<85:04:24,  9.37s/it]
 85%|████████▌ | 192303/225000 [363:42:18<74:18:43,  8.18s/it]
 85%|████████▌ | 192304/225000 [363:42:30<85:16:55,  9.39s/it]
 85%|████████▌ | 192305/225000 [363:42:42<91:18:22, 10.05s/it]
 85%|████████▌ | 192306/225000 [363:42:48<81:46:04,  9.00s/it]
 85%|████████▌ | 192307/225000 [363:42:56<78:49:14,  8.68s/it]
 85%|████████▌ | 192308/225000 [363:43:04<76:17:34,  8.40s/it]
 85%|████████▌ | 192309/225000 [363:43:13<77:07:32,  8.49s/it]
 85%|████████▌ | 192310/225000 [363:43:23<81:32:11,  8.98s/it]
 85%|████████▌ | 192311/225000 [363:43:29<72:30:37,  7.99s/it]
 85%|████████▌ | 192312/225000 [363:43:39<80:09:52,  8.83s/it]
 85%|████████▌ | 192313/225000 [363:43:47<76:26:09,  8.42s/it]
 85%|████████▌ | 192314/225000 [363:43:52<67:09:21,  7.40s/it]
 85%|████████▌ | 192315/225000 [363:44:04<81:09:33,  8.94s/it]
 85%|████████▌ | 192316/225000 [363:44:12<77:14:58,  8.51s/it]
 85%|████████▌ | 192317/225000 [363:44:21<78:00:27,  8.59s/it]
 85%|████████▌ | 192318/225000 [363:44:30<80:55:53,  8.91s/it]
 85%|████████▌ | 192319/225000 [363:44:37<73:59:56,  8.15s/it]
 85%|████████▌ | 192320/225000 [363:44:44<71:41:18,  7.90s/it]
 85%|████████▌ | 192321/225000 [363:44:49<64:42:16,  7.13s/it]
 85%|████████▌ | 192322/225000 [363:44:58<69:47:01,  7.69s/it]
 85%|████████▌ | 192323/225000 [363:45:12<86:36:39,  9.54s/it]
 85%|████████▌ | 192324/225000 [363:45:20<80:36:02,  8.88s/it]
 85%|████████▌ | 192325/225000 [363:45:29<81:26:19,  8.97s/it]
 85%|████████▌ | 192326/225000 [363:45:36<76:23:58,  8.42s/it]
 85%|████████▌ | 192327/225000 [363:45:45<77:26:49,  8.53s/it]
 85%|████████▌ | 192328/225000 [363:45:52<75:03:19,  8.27s/it]
 85%|████████▌ | 192329/225000 [363:46:03<81:24:33,  8.97s/it]
 85%|████████▌ | 192330/225000 [363:46:12<81:59:44,  9.04s/it]
 85%|████████▌ | 192331/225000 [363:46:19<76:06:09,  8.39s/it]
 85%|████████▌ | 192332/225000 [363:46:27<74:23:07,  8.20s/it]
 85%|████████▌ | 192333/225000 [363:46:33<69:37:48,  7.67s/it]
 85%|████████▌ | 192334/225000 [363:46:40<66:25:56,  7.32s/it]
 85%|████████▌ | 192335/225000 [363:46:49<72:01:01,  7.94s/it]
 85%|████████▌ | 192336/225000 [363:47:01<84:12:51,  9.28s/it]
 85%|████████▌ | 192337/225000 [363:47:06<72:13:17,  7.96s/it]
 85%|████████▌ | 192338/225000 [363:47:13<68:55:39,  7.60s/it]
 85%|████████▌ | 192339/225000 [363:47:22<73:00:04,  8.05s/it]
 85%|████████▌ | 192340/225000 [363:47:30<71:18:37,  7.86s/it]
 85%|████████▌ | 192341/225000 [363:47:40<76:53:15,  8.48s/it]
 85%|████████▌ | 192342/225000 [363:47:46<70:13:49,  7.74s/it]
 85%|████████▌ | 192343/225000 [363:47:57<80:41:02,  8.89s/it]
 85%|████████▌ | 192344/225000 [363:48:04<76:21:55,  8.42s/it]
 85%|████████▌ | 192345/225000 [363:48:15<82:09:12,  9.06s/it]
 85%|████████▌ | 192346/225000 [363:48:24<80:45:58,  8.90s/it]
 85%|████████▌ | 192347/225000 [363:48:34<86:02:03,  9.49s/it]
 85%|████████▌ | 192348/225000 [363:48:41<77:31:31,  8.55s/it]
 85%|████████▌ | 192349/225000 [363:48:56<95:23:42, 10.52s/it]
 85%|████████▌ | 192350/225000 [363:49:02<83:18:51,  9.19s/it]
                                                              
{'loss': '1.268', 'grad_norm': '1.362', 'learning_rate': '2.807e-06', 'epoch': '0.8549', 'num_input_tokens_seen': 605656624, 'train_runtime': '1.31e+06', 'train_tokens_per_second': '462.4'}

 85%|████████▌ | 192350/225000 [363:49:02<83:18:51,  9.19s/it]
 85%|████████▌ | 192351/225000 [363:49:09<76:43:25,  8.46s/it]
 85%|████████▌ | 192352/225000 [363:49:14<67:43:21,  7.47s/it]
 85%|████████▌ | 192353/225000 [363:49:24<74:14:54,  8.19s/it]
 85%|████████▌ | 192354/225000 [363:49:29<67:06:30,  7.40s/it]
 85%|████████▌ | 192355/225000 [363:49:42<80:19:39,  8.86s/it]
 85%|████████▌ | 192356/225000 [363:49:48<75:12:07,  8.29s/it]
 85%|████████▌ | 192357/225000 [363:49:56<72:43:21,  8.02s/it]
 85%|████████▌ | 192358/225000 [363:50:03<70:46:29,  7.81s/it]
 85%|████████▌ | 192359/225000 [363:50:08<63:51:13,  7.04s/it]
 85%|████████▌ | 192360/225000 [363:50:22<82:44:21,  9.13s/it]
 85%|████████▌ | 192361/225000 [363:50:29<74:33:25,  8.22s/it]
 85%|████████▌ | 192362/225000 [363:50:38<78:38:04,  8.67s/it]
 85%|████████▌ | 192363/225000 [363:50:47<78:29:17,  8.66s/it]
 85%|████████▌ | 192364/225000 [363:50:54<75:05:51,  8.28s/it]
 85%|████████▌ | 192365/225000 [363:51:03<75:05:21,  8.28s/it]
 85%|████████▌ | 192366/225000 [363:51:13<80:57:55,  8.93s/it]
 85%|████████▌ | 192367/225000 [363:51:21<79:04:47,  8.72s/it]
 85%|████████▌ | 192368/225000 [363:51:27<70:28:24,  7.77s/it]
 85%|████████▌ | 192369/225000 [363:51:34<69:44:30,  7.69s/it]
 85%|████████▌ | 192370/225000 [363:51:44<74:19:00,  8.20s/it]
 85%|████████▌ | 192371/225000 [363:51:50<70:24:18,  7.77s/it]
 85%|████████▌ | 192372/225000 [363:52:01<77:44:38,  8.58s/it]
 85%|████████▌ | 192373/225000 [363:52:08<74:49:13,  8.26s/it]
 85%|████████▌ | 192374/225000 [363:52:20<82:57:50,  9.15s/it]
 86%|████████▌ | 192375/225000 [363:52:27<78:01:14,  8.61s/it]
 86%|████████▌ | 192376/225000 [363:52:34<74:04:46,  8.17s/it]
 86%|████████▌ | 192377/225000 [363:52:44<78:59:07,  8.72s/it]
 86%|████████▌ | 192378/225000 [363:52:55<84:04:00,  9.28s/it]
 86%|████████▌ | 192379/225000 [363:53:02<78:02:40,  8.61s/it]
 86%|████████▌ | 192380/225000 [363:53:10<76:05:06,  8.40s/it]
 86%|████████▌ | 192381/225000 [363:53:19<77:18:05,  8.53s/it]
 86%|████████▌ | 192382/225000 [363:53:25<70:56:33,  7.83s/it]
 86%|████████▌ | 192383/225000 [363:53:37<82:46:23,  9.14s/it]
 86%|████████▌ | 192384/225000 [363:53:49<90:01:57,  9.94s/it]
 86%|████████▌ | 192385/225000 [363:53:57<86:10:38,  9.51s/it]
 86%|████████▌ | 192386/225000 [363:54:06<83:50:50,  9.26s/it]
 86%|████████▌ | 192387/225000 [363:54:14<81:53:07,  9.04s/it]
 86%|████████▌ | 192388/225000 [363:54:22<77:19:20,  8.54s/it]
 86%|████████▌ | 192389/225000 [363:54:28<70:08:36,  7.74s/it]
 86%|████████▌ | 192390/225000 [363:54:34<66:07:00,  7.30s/it]
 86%|████████▌ | 192391/225000 [363:54:39<58:52:00,  6.50s/it]
 86%|████████▌ | 192392/225000 [363:54:50<71:50:36,  7.93s/it]
 86%|████████▌ | 192393/225000 [363:55:01<80:57:56,  8.94s/it]
 86%|████████▌ | 192394/225000 [363:55:10<79:36:33,  8.79s/it]
 86%|████████▌ | 192395/225000 [363:55:18<79:08:13,  8.74s/it]
 86%|████████▌ | 192396/225000 [363:55:25<74:58:53,  8.28s/it]
 86%|████████▌ | 192397/225000 [363:55:32<71:08:13,  7.85s/it]
 86%|████████▌ | 192398/225000 [363:55:39<67:15:21,  7.43s/it]
 86%|████████▌ | 192399/225000 [363:55:46<66:20:00,  7.32s/it]
 86%|████████▌ | 192400/225000 [363:56:01<88:27:54,  9.77s/it]
                                                              
{'loss': '1.251', 'grad_norm': '1.435', 'learning_rate': '2.798e-06', 'epoch': '0.8551', 'num_input_tokens_seen': 605874960, 'train_runtime': '1.31e+06', 'train_tokens_per_second': '462.4'}

 86%|████████▌ | 192400/225000 [363:56:01<88:27:54,  9.77s/it]
 86%|████████▌ | 192401/225000 [363:56:09<82:58:49,  9.16s/it]
 86%|████████▌ | 192402/225000 [363:56:15<75:02:54,  8.29s/it]
 86%|████████▌ | 192403/225000 [363:56:23<74:12:55,  8.20s/it]
 86%|████████▌ | 192404/225000 [363:56:30<69:00:03,  7.62s/it]
 86%|████████▌ | 192405/225000 [363:56:37<69:03:46,  7.63s/it]
 86%|████████▌ | 192406/225000 [363:56:44<65:35:14,  7.24s/it]
 86%|████████▌ | 192407/225000 [363:56:52<69:00:44,  7.62s/it]
 86%|████████▌ | 192408/225000 [363:57:04<81:09:23,  8.96s/it]
 86%|████████▌ | 192409/225000 [363:57:15<85:24:38,  9.43s/it]
 86%|████████▌ | 192410/225000 [363:57:20<74:07:23,  8.19s/it]
 86%|████████▌ | 192411/225000 [363:57:27<71:05:12,  7.85s/it]
 86%|████████▌ | 192412/225000 [363:57:37<77:13:36,  8.53s/it]
 86%|████████▌ | 192413/225000 [363:57:50<88:50:11,  9.81s/it]
 86%|████████▌ | 192414/225000 [363:57:56<79:31:13,  8.79s/it]
 86%|████████▌ | 192415/225000 [363:58:03<74:11:15,  8.20s/it]
 86%|████████▌ | 192416/225000 [363:58:09<68:37:58,  7.58s/it]
 86%|████████▌ | 192417/225000 [363:58:13<58:56:30,  6.51s/it]
 86%|████████▌ | 192418/225000 [363:58:19<56:39:00,  6.26s/it]
 86%|████████▌ | 192419/225000 [363:58:25<56:59:42,  6.30s/it]
 86%|████████▌ | 192420/225000 [363:58:38<73:40:33,  8.14s/it]
 86%|████████▌ | 192421/225000 [363:58:44<69:08:05,  7.64s/it]
 86%|████████▌ | 192422/225000 [363:58:56<79:29:18,  8.78s/it]
 86%|████████▌ | 192423/225000 [363:59:02<72:39:13,  8.03s/it]
 86%|████████▌ | 192424/225000 [363:59:12<76:48:33,  8.49s/it]
 86%|████████▌ | 192425/225000 [363:59:23<84:11:13,  9.30s/it]
 86%|████████▌ | 192426/225000 [363:59:33<86:39:31,  9.58s/it]
 86%|████████▌ | 192427/225000 [363:59:39<76:04:20,  8.41s/it]
 86%|████████▌ | 192428/225000 [363:59:51<85:55:24,  9.50s/it]
 86%|████████▌ | 192429/225000 [363:59:59<82:58:12,  9.17s/it]
 86%|████████▌ | 192430/225000 [364:00:05<73:45:25,  8.15s/it]
 86%|████████▌ | 192431/225000 [364:00:10<66:23:56,  7.34s/it]
 86%|████████▌ | 192432/225000 [364:00:17<64:08:36,  7.09s/it]
 86%|████████▌ | 192433/225000 [364:00:26<68:30:41,  7.57s/it]
 86%|████████▌ | 192434/225000 [364:00:37<79:30:31,  8.79s/it]
 86%|████████▌ | 192435/225000 [364:00:45<77:21:34,  8.55s/it]
 86%|████████▌ | 192436/225000 [364:00:51<70:24:42,  7.78s/it]
 86%|████████▌ | 192437/225000 [364:00:57<64:12:13,  7.10s/it]
 86%|████████▌ | 192438/225000 [364:01:08<75:19:16,  8.33s/it]
 86%|████████▌ | 192439/225000 [364:01:15<71:16:13,  7.88s/it]
 86%|████████▌ | 192440/225000 [364:01:25<77:28:33,  8.57s/it]
 86%|████████▌ | 192441/225000 [364:01:32<74:57:57,  8.29s/it]
 86%|████████▌ | 192442/225000 [364:01:43<80:30:51,  8.90s/it]
 86%|████████▌ | 192443/225000 [364:01:47<68:18:15,  7.55s/it]
 86%|████████▌ | 192444/225000 [364:01:59<80:28:08,  8.90s/it]
 86%|████████▌ | 192445/225000 [364:02:10<84:21:18,  9.33s/it]
 86%|████████▌ | 192446/225000 [364:02:21<89:58:08,  9.95s/it]
 86%|████████▌ | 192447/225000 [364:02:26<77:39:55,  8.59s/it]
 86%|████████▌ | 192448/225000 [364:02:40<92:28:05, 10.23s/it]
 86%|████████▌ | 192449/225000 [364:02:53<97:50:18, 10.82s/it]
 86%|████████▌ | 192450/225000 [364:02:58<81:56:26,  9.06s/it]
                                                              
{'loss': '1.275', 'grad_norm': '1.312', 'learning_rate': '2.79e-06', 'epoch': '0.8553', 'num_input_tokens_seen': 606094848, 'train_runtime': '1.311e+06', 'train_tokens_per_second': '462.5'}

 86%|████████▌ | 192450/225000 [364:02:58<81:56:26,  9.06s/it]
 86%|████████▌ | 192451/225000 [364:03:03<71:00:40,  7.85s/it]
 86%|████████▌ | 192452/225000 [364:03:09<67:01:47,  7.41s/it]
 86%|████████▌ | 192453/225000 [364:03:17<67:33:43,  7.47s/it]
 86%|████████▌ | 192454/225000 [364:03:23<65:07:29,  7.20s/it]
 86%|████████▌ | 192455/225000 [364:03:29<60:16:30,  6.67s/it]
 86%|████████▌ | 192456/225000 [364:03:41<74:27:10,  8.24s/it]
 86%|████████▌ | 192457/225000 [364:03:50<77:37:14,  8.59s/it]
 86%|████████▌ | 192458/225000 [364:03:56<69:52:19,  7.73s/it]
 86%|████████▌ | 192459/225000 [364:04:08<81:12:27,  8.98s/it]
 86%|████████▌ | 192460/225000 [364:04:14<73:05:43,  8.09s/it]
 86%|████████▌ | 192461/225000 [364:04:21<71:02:17,  7.86s/it]
 86%|████████▌ | 192462/225000 [364:04:34<84:39:41,  9.37s/it]
 86%|████████▌ | 192463/225000 [364:04:42<80:45:15,  8.93s/it]
 86%|████████▌ | 192464/225000 [364:04:47<71:25:50,  7.90s/it]
 86%|████████▌ | 192465/225000 [364:04:53<66:04:46,  7.31s/it]
 86%|████████▌ | 192466/225000 [364:04:59<62:31:54,  6.92s/it]
 86%|████████▌ | 192467/225000 [364:05:10<74:15:31,  8.22s/it]
 86%|████████▌ | 192468/225000 [364:05:19<74:33:23,  8.25s/it]
 86%|████████▌ | 192469/225000 [364:05:30<82:28:55,  9.13s/it]
 86%|████████▌ | 192470/225000 [364:05:37<77:12:36,  8.54s/it]
 86%|████████▌ | 192471/225000 [364:05:46<77:41:42,  8.60s/it]
 86%|████████▌ | 192472/225000 [364:05:56<81:09:54,  8.98s/it]
 86%|████████▌ | 192473/225000 [364:06:03<75:27:46,  8.35s/it]
 86%|████████▌ | 192474/225000 [364:06:10<73:14:45,  8.11s/it]
 86%|████████▌ | 192475/225000 [364:06:17<71:11:27,  7.88s/it]
 86%|████████▌ | 192476/225000 [364:06:23<63:53:00,  7.07s/it]
 86%|████████▌ | 192477/225000 [364:06:35<78:09:15,  8.65s/it]
 86%|████████▌ | 192478/225000 [364:06:45<80:40:41,  8.93s/it]
 86%|████████▌ | 192479/225000 [364:06:50<70:44:41,  7.83s/it]
 86%|████████▌ | 192480/225000 [364:06:59<74:38:03,  8.26s/it]
 86%|████████▌ | 192481/225000 [364:07:05<67:40:39,  7.49s/it]
 86%|████████▌ | 192482/225000 [364:07:15<74:48:25,  8.28s/it]
 86%|████████▌ | 192483/225000 [364:07:20<66:08:45,  7.32s/it]
 86%|████████▌ | 192484/225000 [364:07:27<65:35:58,  7.26s/it]
 86%|████████▌ | 192485/225000 [364:07:32<58:11:14,  6.44s/it]
 86%|████████▌ | 192486/225000 [364:07:41<65:27:11,  7.25s/it]
 86%|████████▌ | 192487/225000 [364:07:50<70:55:03,  7.85s/it]
 86%|████████▌ | 192488/225000 [364:07:55<64:19:46,  7.12s/it]
 86%|████████▌ | 192489/225000 [364:08:05<71:57:30,  7.97s/it]
 86%|████████▌ | 192490/225000 [364:08:14<73:39:07,  8.16s/it]
 86%|████████▌ | 192491/225000 [364:08:18<62:42:54,  6.94s/it]
 86%|████████▌ | 192492/225000 [364:08:32<82:04:40,  9.09s/it]
 86%|████████▌ | 192493/225000 [364:08:38<72:58:29,  8.08s/it]
 86%|████████▌ | 192494/225000 [364:08:51<85:53:38,  9.51s/it]
 86%|████████▌ | 192495/225000 [364:08:55<71:35:47,  7.93s/it]
 86%|████████▌ | 192496/225000 [364:09:04<73:40:06,  8.16s/it]
 86%|████████▌ | 192497/225000 [364:09:09<65:51:56,  7.30s/it]
 86%|████████▌ | 192498/225000 [364:09:18<71:17:12,  7.90s/it]
 86%|████████▌ | 192499/225000 [364:09:23<63:22:57,  7.02s/it]
 86%|████████▌ | 192500/225000 [364:09:29<60:02:13,  6.65s/it]
                                                              
{'loss': '1.262', 'grad_norm': '1.721', 'learning_rate': '2.782e-06', 'epoch': '0.8556', 'num_input_tokens_seen': 606304576, 'train_runtime': '1.311e+06', 'train_tokens_per_second': '462.5'}

 86%|████████▌ | 192500/225000 [364:09:29<60:02:13,  6.65s/it]
 86%|████████▌ | 192501/225000 [364:09:40<70:59:49,  7.86s/it]
 86%|████████▌ | 192502/225000 [364:09:49<75:18:23,  8.34s/it]
 86%|████████▌ | 192503/225000 [364:09:59<78:12:09,  8.66s/it]
 86%|████████▌ | 192504/225000 [364:10:04<69:12:50,  7.67s/it]
 86%|████████▌ | 192505/225000 [364:10:11<68:09:49,  7.55s/it]
 86%|████████▌ | 192506/225000 [364:10:19<69:33:23,  7.71s/it]
 86%|████████▌ | 192507/225000 [364:10:27<68:55:20,  7.64s/it]
 86%|████████▌ | 192508/225000 [364:10:37<77:06:18,  8.54s/it]
 86%|████████▌ | 192509/225000 [364:10:44<70:52:27,  7.85s/it]
 86%|████████▌ | 192510/225000 [364:10:49<64:00:50,  7.09s/it]
 86%|████████▌ | 192511/225000 [364:10:55<59:39:20,  6.61s/it]
 86%|████████▌ | 192512/225000 [364:11:01<58:06:09,  6.44s/it]
 86%|████████▌ | 192513/225000 [364:11:06<54:17:13,  6.02s/it]
 86%|████████▌ | 192514/225000 [364:11:18<70:59:17,  7.87s/it]
 86%|████████▌ | 192515/225000 [364:11:31<84:26:54,  9.36s/it]
 86%|████████▌ | 192516/225000 [364:11:37<77:23:04,  8.58s/it]
 86%|████████▌ | 192517/225000 [364:11:44<71:05:02,  7.88s/it]
 86%|████████▌ | 192518/225000 [364:11:49<65:05:27,  7.21s/it]
 86%|████████▌ | 192519/225000 [364:12:00<73:55:27,  8.19s/it]
 86%|████████▌ | 192520/225000 [364:12:13<88:31:30,  9.81s/it]
 86%|████████▌ | 192521/225000 [364:12:21<83:06:35,  9.21s/it]
 86%|████████▌ | 192522/225000 [364:12:28<76:56:27,  8.53s/it]
 86%|████████▌ | 192523/225000 [364:12:36<76:00:59,  8.43s/it]
 86%|████████▌ | 192524/225000 [364:12:46<78:42:37,  8.73s/it]
 86%|████████▌ | 192525/225000 [364:12:57<84:55:19,  9.41s/it]
 86%|████████▌ | 192526/225000 [364:13:05<81:11:49,  9.00s/it]
 86%|████████▌ | 192527/225000 [364:13:16<86:51:20,  9.63s/it]
 86%|████████▌ | 192528/225000 [364:13:21<76:01:51,  8.43s/it]
 86%|████████▌ | 192529/225000 [364:13:29<74:51:02,  8.30s/it]
 86%|████████▌ | 192530/225000 [364:13:36<69:45:57,  7.74s/it]
 86%|████████▌ | 192531/225000 [364:13:44<69:47:11,  7.74s/it]
 86%|████████▌ | 192532/225000 [364:13:57<85:07:38,  9.44s/it]
 86%|████████▌ | 192533/225000 [364:14:02<72:22:50,  8.03s/it]
 86%|████████▌ | 192534/225000 [364:14:11<76:16:59,  8.46s/it]
 86%|████████▌ | 192535/225000 [364:14:21<80:00:43,  8.87s/it]
 86%|████████▌ | 192536/225000 [364:14:26<68:25:11,  7.59s/it]
 86%|████████▌ | 192537/225000 [364:14:40<85:51:26,  9.52s/it]
 86%|████████▌ | 192538/225000 [364:14:46<78:03:29,  8.66s/it]
 86%|████████▌ | 192539/225000 [364:14:55<77:10:41,  8.56s/it]
 86%|████████▌ | 192540/225000 [364:15:00<68:26:03,  7.59s/it]
 86%|████████▌ | 192541/225000 [364:15:11<76:55:18,  8.53s/it]
 86%|████████▌ | 192542/225000 [364:15:21<80:30:10,  8.93s/it]
 86%|████████▌ | 192543/225000 [364:15:27<72:59:53,  8.10s/it]
 86%|████████▌ | 192544/225000 [364:15:35<72:18:19,  8.02s/it]
 86%|████████▌ | 192545/225000 [364:15:42<70:08:38,  7.78s/it]
 86%|████████▌ | 192546/225000 [364:15:55<84:04:25,  9.33s/it]
 86%|████████▌ | 192547/225000 [364:15:59<71:32:46,  7.94s/it]
 86%|████████▌ | 192548/225000 [364:16:06<68:38:54,  7.62s/it]
 86%|████████▌ | 192549/225000 [364:16:18<80:03:37,  8.88s/it]
 86%|████████▌ | 192550/225000 [364:16:25<75:24:07,  8.37s/it]
                                                              
{'loss': '1.272', 'grad_norm': '1.332', 'learning_rate': '2.773e-06', 'epoch': '0.8558', 'num_input_tokens_seen': 606528096, 'train_runtime': '1.311e+06', 'train_tokens_per_second': '462.5'}

 86%|████████▌ | 192550/225000 [364:16:25<75:24:07,  8.37s/it]
 86%|████████▌ | 192551/225000 [364:16:33<73:00:46,  8.10s/it]
 86%|████████▌ | 192552/225000 [364:16:37<63:02:34,  6.99s/it]
 86%|████████▌ | 192553/225000 [364:16:47<69:38:21,  7.73s/it]
 86%|████████▌ | 192554/225000 [364:16:54<68:09:51,  7.56s/it]
 86%|████████▌ | 192555/225000 [364:17:01<66:42:33,  7.40s/it]
 86%|████████▌ | 192556/225000 [364:17:05<58:33:49,  6.50s/it]
 86%|████████▌ | 192557/225000 [364:17:15<68:35:59,  7.61s/it]
 86%|████████▌ | 192558/225000 [364:17:25<74:19:19,  8.25s/it]
 86%|████████▌ | 192559/225000 [364:17:37<83:07:37,  9.22s/it]
 86%|████████▌ | 192560/225000 [364:17:42<72:09:48,  8.01s/it]
 86%|████████▌ | 192561/225000 [364:17:48<67:05:09,  7.45s/it]
 86%|████████▌ | 192562/225000 [364:18:00<80:14:25,  8.91s/it]
 86%|████████▌ | 192563/225000 [364:18:11<86:11:18,  9.57s/it]
 86%|████████▌ | 192564/225000 [364:18:17<75:21:44,  8.36s/it]
 86%|████████▌ | 192565/225000 [364:18:29<84:46:06,  9.41s/it]
 86%|████████▌ | 192566/225000 [364:18:35<76:07:43,  8.45s/it]
 86%|████████▌ | 192567/225000 [364:18:41<68:10:31,  7.57s/it]
 86%|████████▌ | 192568/225000 [364:18:47<66:34:07,  7.39s/it]
 86%|████████▌ | 192569/225000 [364:19:00<79:11:29,  8.79s/it]
 86%|████████▌ | 192570/225000 [364:19:08<79:19:06,  8.81s/it]
 86%|████████▌ | 192571/225000 [364:19:16<76:23:58,  8.48s/it]
 86%|████████▌ | 192572/225000 [364:19:23<72:33:26,  8.05s/it]
 86%|████████▌ | 192573/225000 [364:19:35<83:25:34,  9.26s/it]
 86%|████████▌ | 192574/225000 [364:19:39<69:18:39,  7.70s/it]
 86%|████████▌ | 192575/225000 [364:19:48<73:20:24,  8.14s/it]
 86%|████████▌ | 192576/225000 [364:19:56<71:51:39,  7.98s/it]
 86%|████████▌ | 192577/225000 [364:20:04<70:30:35,  7.83s/it]
 86%|████████▌ | 192578/225000 [364:20:14<78:27:30,  8.71s/it]
 86%|████████▌ | 192579/225000 [364:20:19<68:22:05,  7.59s/it]
 86%|████████▌ | 192580/225000 [364:20:24<61:42:27,  6.85s/it]
 86%|████████▌ | 192581/225000 [364:20:34<69:43:49,  7.74s/it]
 86%|████████▌ | 192582/225000 [364:20:42<69:40:28,  7.74s/it]
 86%|████████▌ | 192583/225000 [364:20:55<84:14:29,  9.36s/it]
 86%|████████▌ | 192584/225000 [364:21:06<88:09:52,  9.79s/it]
 86%|████████▌ | 192585/225000 [364:21:15<85:16:09,  9.47s/it]
 86%|████████▌ | 192586/225000 [364:21:22<79:12:45,  8.80s/it]
 86%|████████▌ | 192587/225000 [364:21:32<83:39:14,  9.29s/it]
 86%|████████▌ | 192588/225000 [364:21:42<83:59:01,  9.33s/it]
 86%|████████▌ | 192589/225000 [364:21:50<82:13:14,  9.13s/it]
 86%|████████▌ | 192590/225000 [364:21:59<80:12:06,  8.91s/it]
 86%|████████▌ | 192591/225000 [364:22:04<70:11:23,  7.80s/it]
 86%|████████▌ | 192592/225000 [364:22:14<75:02:01,  8.34s/it]
 86%|████████▌ | 192593/225000 [364:22:23<79:19:35,  8.81s/it]
 86%|████████▌ | 192594/225000 [364:22:33<82:26:32,  9.16s/it]
 86%|████████▌ | 192595/225000 [364:22:39<73:02:59,  8.12s/it]
 86%|████████▌ | 192596/225000 [364:22:51<83:20:13,  9.26s/it]
 86%|████████▌ | 192597/225000 [364:23:01<86:13:27,  9.58s/it]
 86%|████████▌ | 192598/225000 [364:23:07<76:23:36,  8.49s/it]
 86%|████████▌ | 192599/225000 [364:23:14<70:23:27,  7.82s/it]
 86%|████████▌ | 192600/225000 [364:23:28<87:43:30,  9.75s/it]
                                                              
{'loss': '1.278', 'grad_norm': '1.095', 'learning_rate': '2.765e-06', 'epoch': '0.856', 'num_input_tokens_seen': 606744496, 'train_runtime': '1.312e+06', 'train_tokens_per_second': '462.5'}

 86%|████████▌ | 192600/225000 [364:23:28<87:43:30,  9.75s/it]
 86%|████████▌ | 192601/225000 [364:23:32<71:29:47,  7.94s/it]
 86%|████████▌ | 192602/225000 [364:23:42<78:05:56,  8.68s/it]
 86%|████████▌ | 192603/225000 [364:23:49<73:29:52,  8.17s/it]
 86%|████████▌ | 192604/225000 [364:23:57<73:07:27,  8.13s/it]
 86%|████████▌ | 192605/225000 [364:24:04<69:06:37,  7.68s/it]
 86%|████████▌ | 192606/225000 [364:24:17<84:41:06,  9.41s/it]
 86%|████████▌ | 192607/225000 [364:24:22<71:37:17,  7.96s/it]
 86%|████████▌ | 192608/225000 [364:24:26<63:00:30,  7.00s/it]
 86%|████████▌ | 192609/225000 [364:24:34<63:46:02,  7.09s/it]
 86%|████████▌ | 192610/225000 [364:24:40<61:25:00,  6.83s/it]
 86%|████████▌ | 192611/225000 [364:24:53<78:36:15,  8.74s/it]
 86%|████████▌ | 192612/225000 [364:25:01<75:15:14,  8.36s/it]
 86%|████████▌ | 192613/225000 [364:25:10<76:47:41,  8.54s/it]
 86%|████████▌ | 192614/225000 [364:25:16<70:01:49,  7.78s/it]
 86%|████████▌ | 192615/225000 [364:25:21<63:00:17,  7.00s/it]
 86%|████████▌ | 192616/225000 [364:25:26<57:30:59,  6.39s/it]
 86%|████████▌ | 192617/225000 [364:25:36<69:04:44,  7.68s/it]
 86%|████████▌ | 192618/225000 [364:25:44<68:58:29,  7.67s/it]
 86%|████████▌ | 192619/225000 [364:25:55<78:06:54,  8.68s/it]
 86%|████████▌ | 192620/225000 [364:26:02<73:12:54,  8.14s/it]
 86%|████████▌ | 192621/225000 [364:26:07<63:55:58,  7.11s/it]
 86%|████████▌ | 192622/225000 [364:26:13<62:05:07,  6.90s/it]
 86%|████████▌ | 192623/225000 [364:26:25<76:34:37,  8.51s/it]
 86%|████████▌ | 192624/225000 [364:26:39<91:31:47, 10.18s/it]
 86%|████████▌ | 192625/225000 [364:26:46<81:26:29,  9.06s/it]
 86%|████████▌ | 192626/225000 [364:26:53<77:31:39,  8.62s/it]
 86%|████████▌ | 192627/225000 [364:26:58<66:02:05,  7.34s/it]
 86%|████████▌ | 192628/225000 [364:27:03<60:10:04,  6.69s/it]
 86%|████████▌ | 192629/225000 [364:27:11<64:48:00,  7.21s/it]
 86%|████████▌ | 192630/225000 [364:27:20<69:42:10,  7.75s/it]
 86%|████████▌ | 192631/225000 [364:27:27<65:38:43,  7.30s/it]
 86%|████████▌ | 192632/225000 [364:27:34<65:00:50,  7.23s/it]
 86%|████████▌ | 192633/225000 [364:27:45<74:55:02,  8.33s/it]
 86%|████████▌ | 192634/225000 [364:27:56<83:33:50,  9.29s/it]
 86%|████████▌ | 192635/225000 [364:28:04<79:50:01,  8.88s/it]
 86%|████████▌ | 192636/225000 [364:28:10<71:36:17,  7.96s/it]
 86%|████████▌ | 192637/225000 [364:28:22<82:27:27,  9.17s/it]
 86%|████████▌ | 192638/225000 [364:28:29<76:00:30,  8.46s/it]
 86%|████████▌ | 192639/225000 [364:28:35<69:18:47,  7.71s/it]
 86%|████████▌ | 192640/225000 [364:28:44<74:37:49,  8.30s/it]
 86%|████████▌ | 192641/225000 [364:28:49<65:58:03,  7.34s/it]
 86%|████████▌ | 192642/225000 [364:28:57<65:22:24,  7.27s/it]
 86%|████████▌ | 192643/225000 [364:29:07<72:55:14,  8.11s/it]
 86%|████████▌ | 192644/225000 [364:29:23<93:56:09, 10.45s/it]
 86%|████████▌ | 192645/225000 [364:29:27<78:40:43,  8.75s/it]
 86%|████████▌ | 192646/225000 [364:29:34<73:58:28,  8.23s/it]
 86%|████████▌ | 192647/225000 [364:29:40<68:19:42,  7.60s/it]
 86%|████████▌ | 192648/225000 [364:29:49<71:56:12,  8.00s/it]
 86%|████████▌ | 192649/225000 [364:30:03<87:23:15,  9.72s/it]
 86%|████████▌ | 192650/225000 [364:30:10<79:02:15,  8.80s/it]
                                                              
{'loss': '1.234', 'grad_norm': '1.244', 'learning_rate': '2.757e-06', 'epoch': '0.8562', 'num_input_tokens_seen': 606958640, 'train_runtime': '1.312e+06', 'train_tokens_per_second': '462.5'}

 86%|████████▌ | 192650/225000 [364:30:10<79:02:15,  8.80s/it]
 86%|████████▌ | 192651/225000 [364:30:18<78:18:59,  8.72s/it]
 86%|████████▌ | 192652/225000 [364:30:25<73:36:09,  8.19s/it]
 86%|████████▌ | 192653/225000 [364:30:34<73:50:28,  8.22s/it]
 86%|████████▌ | 192654/225000 [364:30:39<65:32:27,  7.29s/it]
 86%|████████▌ | 192655/225000 [364:30:56<93:14:37, 10.38s/it]
 86%|████████▌ | 192656/225000 [364:31:04<85:49:54,  9.55s/it]
 86%|████████▌ | 192657/225000 [364:31:13<84:55:09,  9.45s/it]
 86%|████████▌ | 192658/225000 [364:31:20<77:19:55,  8.61s/it]
 86%|████████▌ | 192659/225000 [364:31:26<71:41:36,  7.98s/it]
 86%|████████▌ | 192660/225000 [364:31:33<67:06:05,  7.47s/it]
 86%|████████▌ | 192661/225000 [364:31:43<74:39:34,  8.31s/it]
 86%|████████▌ | 192662/225000 [364:31:59<95:36:53, 10.64s/it]
 86%|████████▌ | 192663/225000 [364:32:05<83:31:29,  9.30s/it]
 86%|████████▌ | 192664/225000 [364:32:10<71:57:50,  8.01s/it]
 86%|████████▌ | 192665/225000 [364:32:19<74:58:10,  8.35s/it]
 86%|████████▌ | 192666/225000 [364:32:30<82:39:44,  9.20s/it]
 86%|████████▌ | 192667/225000 [364:32:40<84:55:38,  9.46s/it]
 86%|████████▌ | 192668/225000 [364:32:47<76:27:56,  8.51s/it]
 86%|████████▌ | 192669/225000 [364:32:57<81:41:43,  9.10s/it]
 86%|████████▌ | 192670/225000 [364:33:09<88:46:13,  9.88s/it]
 86%|████████▌ | 192671/225000 [364:33:14<76:38:31,  8.53s/it]
 86%|████████▌ | 192672/225000 [364:33:22<73:20:21,  8.17s/it]
 86%|████████▌ | 192673/225000 [364:33:27<65:52:18,  7.34s/it]
 86%|████████▌ | 192674/225000 [364:33:38<74:19:31,  8.28s/it]
 86%|████████▌ | 192675/225000 [364:33:45<73:19:22,  8.17s/it]
 86%|████████▌ | 192676/225000 [364:33:52<70:07:59,  7.81s/it]
 86%|████████▌ | 192677/225000 [364:33:59<66:10:31,  7.37s/it]
 86%|████████▌ | 192678/225000 [364:34:11<80:24:38,  8.96s/it]
 86%|████████▌ | 192679/225000 [364:34:18<74:57:33,  8.35s/it]
 86%|████████▌ | 192680/225000 [364:34:24<68:03:08,  7.58s/it]
 86%|████████▌ | 192681/225000 [364:34:35<75:46:48,  8.44s/it]
 86%|████████▌ | 192682/225000 [364:34:45<80:51:49,  9.01s/it]
 86%|████████▌ | 192683/225000 [364:34:53<79:10:25,  8.82s/it]
 86%|████████▌ | 192684/225000 [364:34:58<67:09:35,  7.48s/it]
 86%|████████▌ | 192685/225000 [364:35:02<58:26:12,  6.51s/it]
 86%|████████▌ | 192686/225000 [364:35:13<71:57:16,  8.02s/it]
 86%|████████▌ | 192687/225000 [364:35:17<60:25:06,  6.73s/it]
 86%|████████▌ | 192688/225000 [364:35:26<66:32:20,  7.41s/it]
 86%|████████▌ | 192689/225000 [364:35:33<65:44:45,  7.33s/it]
 86%|████████▌ | 192690/225000 [364:35:43<71:05:29,  7.92s/it]
 86%|████████▌ | 192691/225000 [364:35:50<69:55:00,  7.79s/it]
 86%|████████▌ | 192692/225000 [364:36:00<76:00:47,  8.47s/it]
 86%|████████▌ | 192693/225000 [364:36:08<74:03:12,  8.25s/it]
 86%|████████▌ | 192694/225000 [364:36:16<74:19:37,  8.28s/it]
 86%|████████▌ | 192695/225000 [364:36:25<74:20:16,  8.28s/it]
 86%|████████▌ | 192696/225000 [364:36:32<71:42:46,  7.99s/it]
 86%|████████▌ | 192697/225000 [364:36:44<83:53:56,  9.35s/it]
 86%|████████▌ | 192698/225000 [364:36:50<73:42:34,  8.21s/it]
 86%|████████▌ | 192699/225000 [364:37:00<78:37:27,  8.76s/it]
 86%|████████▌ | 192700/225000 [364:37:10<82:28:17,  9.19s/it]
                                                              
{'loss': '1.284', 'grad_norm': '1.122', 'learning_rate': '2.748e-06', 'epoch': '0.8564', 'num_input_tokens_seen': 607177176, 'train_runtime': '1.313e+06', 'train_tokens_per_second': '462.6'}

 86%|████████▌ | 192700/225000 [364:37:10<82:28:17,  9.19s/it]
 86%|████████▌ | 192701/225000 [364:37:19<82:02:29,  9.14s/it]
 86%|████████▌ | 192702/225000 [364:37:27<78:02:03,  8.70s/it]
 86%|████████▌ | 192703/225000 [364:37:33<71:12:32,  7.94s/it]
 86%|████████▌ | 192704/225000 [364:37:45<80:48:59,  9.01s/it]
 86%|████████▌ | 192705/225000 [364:37:50<71:57:33,  8.02s/it]
 86%|████████▌ | 192706/225000 [364:38:02<81:20:21,  9.07s/it]
 86%|████████▌ | 192707/225000 [364:38:10<80:16:50,  8.95s/it]
 86%|████████▌ | 192708/225000 [364:38:15<69:18:18,  7.73s/it]
 86%|████████▌ | 192709/225000 [364:38:21<64:02:24,  7.14s/it]
 86%|████████▌ | 192710/225000 [364:38:28<64:45:04,  7.22s/it]
 86%|████████▌ | 192711/225000 [364:38:41<79:41:31,  8.89s/it]
 86%|████████▌ | 192712/225000 [364:38:50<78:41:34,  8.77s/it]
 86%|████████▌ | 192713/225000 [364:38:55<70:17:34,  7.84s/it]
 86%|████████▌ | 192714/225000 [364:39:06<79:00:18,  8.81s/it]
 86%|████████▌ | 192715/225000 [364:39:14<76:44:01,  8.56s/it]
 86%|████████▌ | 192716/225000 [364:39:25<81:59:46,  9.14s/it]
 86%|████████▌ | 192717/225000 [364:39:31<72:29:31,  8.08s/it]
 86%|████████▌ | 192718/225000 [364:39:41<78:51:01,  8.79s/it]
 86%|████████▌ | 192719/225000 [364:39:48<73:18:01,  8.17s/it]
 86%|████████▌ | 192720/225000 [364:39:59<82:00:28,  9.15s/it]
 86%|████████▌ | 192721/225000 [364:40:07<78:01:13,  8.70s/it]
 86%|████████▌ | 192722/225000 [364:40:13<70:02:46,  7.81s/it]
 86%|████████▌ | 192723/225000 [364:40:20<69:54:54,  7.80s/it]
 86%|████████▌ | 192724/225000 [364:40:29<71:19:32,  7.96s/it]
 86%|████████▌ | 192725/225000 [364:40:44<90:24:12, 10.08s/it]
 86%|████████▌ | 192726/225000 [364:40:52<85:59:46,  9.59s/it]
 86%|████████▌ | 192727/225000 [364:40:59<78:39:25,  8.77s/it]
 86%|████████▌ | 192728/225000 [364:41:08<78:00:21,  8.70s/it]
 86%|████████▌ | 192729/225000 [364:41:12<66:15:50,  7.39s/it]
 86%|████████▌ | 192730/225000 [364:41:24<78:20:43,  8.74s/it]
 86%|████████▌ | 192731/225000 [364:41:33<80:58:42,  9.03s/it]
 86%|████████▌ | 192732/225000 [364:41:43<82:30:02,  9.20s/it]
 86%|████████▌ | 192733/225000 [364:41:56<91:32:35, 10.21s/it]
 86%|████████▌ | 192734/225000 [364:42:05<89:54:27, 10.03s/it]
 86%|████████▌ | 192735/225000 [364:42:17<93:08:54, 10.39s/it]
 86%|████████▌ | 192736/225000 [364:42:22<80:11:35,  8.95s/it]
 86%|████████▌ | 192737/225000 [364:42:34<87:32:15,  9.77s/it]
 86%|████████▌ | 192738/225000 [364:42:40<78:07:05,  8.72s/it]
 86%|████████▌ | 192739/225000 [364:42:46<70:15:59,  7.84s/it]
 86%|████████▌ | 192740/225000 [364:42:55<73:08:31,  8.16s/it]
 86%|████████▌ | 192741/225000 [364:43:04<76:51:45,  8.58s/it]
 86%|████████▌ | 192742/225000 [364:43:12<73:25:56,  8.20s/it]
 86%|████████▌ | 192743/225000 [364:43:22<80:09:04,  8.95s/it]
 86%|████████▌ | 192744/225000 [364:43:31<78:13:42,  8.73s/it]
 86%|████████▌ | 192745/225000 [364:43:44<92:20:38, 10.31s/it]
 86%|████████▌ | 192746/225000 [364:43:50<79:12:56,  8.84s/it]
 86%|████████▌ | 192747/225000 [364:44:00<81:17:11,  9.07s/it]
 86%|████████▌ | 192748/225000 [364:44:06<75:02:13,  8.38s/it]
 86%|████████▌ | 192749/225000 [364:44:13<70:25:45,  7.86s/it]
 86%|████████▌ | 192750/225000 [364:44:20<68:08:05,  7.61s/it]
                                                              
{'loss': '1.267', 'grad_norm': '1.11', 'learning_rate': '2.74e-06', 'epoch': '0.8567', 'num_input_tokens_seen': 607402592, 'train_runtime': '1.313e+06', 'train_tokens_per_second': '462.6'}

 86%|████████▌ | 192750/225000 [364:44:20<68:08:05,  7.61s/it]
 86%|████████▌ | 192751/225000 [364:44:25<60:22:43,  6.74s/it]
 86%|████████▌ | 192752/225000 [364:44:37<74:04:08,  8.27s/it]
 86%|████████▌ | 192753/225000 [364:44:43<68:35:30,  7.66s/it]
 86%|████████▌ | 192754/225000 [364:44:51<69:56:56,  7.81s/it]
 86%|████████▌ | 192755/225000 [364:44:59<71:28:08,  7.98s/it]
 86%|████████▌ | 192756/225000 [364:45:09<76:23:21,  8.53s/it]
 86%|████████▌ | 192757/225000 [364:45:13<64:39:10,  7.22s/it]
 86%|████████▌ | 192758/225000 [364:45:26<80:42:10,  9.01s/it]
 86%|████████▌ | 192759/225000 [364:45:33<74:01:41,  8.27s/it]
 86%|████████▌ | 192760/225000 [364:45:44<80:15:20,  8.96s/it]
 86%|████████▌ | 192761/225000 [364:45:48<68:21:33,  7.63s/it]
 86%|████████▌ | 192762/225000 [364:45:53<60:27:46,  6.75s/it]
 86%|████████▌ | 192763/225000 [364:46:02<66:17:05,  7.40s/it]
 86%|████████▌ | 192764/225000 [364:46:08<62:03:06,  6.93s/it]
 86%|████████▌ | 192765/225000 [364:46:18<72:18:29,  8.08s/it]
 86%|████████▌ | 192766/225000 [364:46:24<66:56:28,  7.48s/it]
 86%|████████▌ | 192767/225000 [364:46:32<67:14:45,  7.51s/it]
 86%|████████▌ | 192768/225000 [364:46:43<76:01:03,  8.49s/it]
 86%|████████▌ | 192769/225000 [364:46:50<71:56:24,  8.04s/it]
 86%|████████▌ | 192770/225000 [364:46:59<75:38:31,  8.45s/it]
 86%|████████▌ | 192771/225000 [364:47:07<74:59:52,  8.38s/it]
 86%|████████▌ | 192772/225000 [364:47:13<67:32:15,  7.54s/it]
 86%|████████▌ | 192773/225000 [364:47:21<67:59:19,  7.59s/it]
 86%|████████▌ | 192774/225000 [364:47:29<70:20:39,  7.86s/it]
 86%|████████▌ | 192775/225000 [364:47:41<80:35:50,  9.00s/it]
 86%|████████▌ | 192776/225000 [364:47:49<79:26:57,  8.88s/it]
 86%|████████▌ | 192777/225000 [364:47:56<73:36:43,  8.22s/it]
 86%|████████▌ | 192778/225000 [364:48:11<91:30:58, 10.22s/it]
 86%|████████▌ | 192779/225000 [364:48:15<74:00:57,  8.27s/it]
 86%|████████▌ | 192780/225000 [364:48:19<62:39:56,  7.00s/it]
 86%|████████▌ | 192781/225000 [364:48:24<56:51:43,  6.35s/it]
 86%|████████▌ | 192782/225000 [364:48:34<66:51:39,  7.47s/it]
 86%|████████▌ | 192783/225000 [364:48:45<77:13:03,  8.63s/it]
 86%|████████▌ | 192784/225000 [364:48:51<68:58:42,  7.71s/it]
 86%|████████▌ | 192785/225000 [364:48:59<71:31:02,  7.99s/it]
 86%|████████▌ | 192786/225000 [364:49:12<84:31:07,  9.45s/it]
 86%|████████▌ | 192787/225000 [364:49:20<79:35:34,  8.89s/it]
 86%|████████▌ | 192788/225000 [364:49:30<83:39:50,  9.35s/it]
 86%|████████▌ | 192789/225000 [364:49:39<83:06:39,  9.29s/it]
 86%|████████▌ | 192790/225000 [364:49:47<79:42:52,  8.91s/it]
 86%|████████▌ | 192791/225000 [364:49:56<79:11:32,  8.85s/it]
 86%|████████▌ | 192792/225000 [364:50:04<77:08:18,  8.62s/it]
 86%|████████▌ | 192793/225000 [364:50:15<84:10:30,  9.41s/it]
 86%|████████▌ | 192794/225000 [364:50:22<75:58:46,  8.49s/it]
 86%|████████▌ | 192795/225000 [364:50:26<64:30:46,  7.21s/it]
 86%|████████▌ | 192796/225000 [364:50:35<70:41:10,  7.90s/it]
 86%|████████▌ | 192797/225000 [364:50:43<70:07:22,  7.84s/it]
 86%|████████▌ | 192798/225000 [364:50:49<66:23:05,  7.42s/it]
 86%|████████▌ | 192799/225000 [364:50:57<67:21:55,  7.53s/it]
 86%|████████▌ | 192800/225000 [364:51:08<75:47:05,  8.47s/it]
                                                              
{'loss': '1.269', 'grad_norm': '1.329', 'learning_rate': '2.732e-06', 'epoch': '0.8569', 'num_input_tokens_seen': 607616696, 'train_runtime': '1.313e+06', 'train_tokens_per_second': '462.6'}

 86%|████████▌ | 192800/225000 [364:51:08<75:47:05,  8.47s/it]
 86%|████████▌ | 192801/225000 [364:51:18<80:27:21,  9.00s/it]
 86%|████████▌ | 192802/225000 [364:51:24<72:48:38,  8.14s/it]
 86%|████████▌ | 192803/225000 [364:51:33<74:51:56,  8.37s/it]
 86%|████████▌ | 192804/225000 [364:51:39<67:36:03,  7.56s/it]
 86%|████████▌ | 192805/225000 [364:51:53<85:33:16,  9.57s/it]
 86%|████████▌ | 192806/225000 [364:52:01<81:38:05,  9.13s/it]
 86%|████████▌ | 192807/225000 [364:52:06<69:20:04,  7.75s/it]
 86%|████████▌ | 192808/225000 [364:52:14<69:33:32,  7.78s/it]
 86%|████████▌ | 192809/225000 [364:52:24<76:39:28,  8.57s/it]
 86%|████████▌ | 192810/225000 [364:52:29<66:25:48,  7.43s/it]
 86%|████████▌ | 192811/225000 [364:52:35<63:48:54,  7.14s/it]
 86%|████████▌ | 192812/225000 [364:52:44<69:02:43,  7.72s/it]
 86%|████████▌ | 192813/225000 [364:52:50<62:54:00,  7.04s/it]
 86%|████████▌ | 192814/225000 [364:52:57<63:30:19,  7.10s/it]
 86%|████████▌ | 192815/225000 [364:53:07<70:44:31,  7.91s/it]
 86%|████████▌ | 192816/225000 [364:53:13<67:17:36,  7.53s/it]
 86%|████████▌ | 192817/225000 [364:53:24<75:35:01,  8.45s/it]
 86%|████████▌ | 192818/225000 [364:53:33<76:05:20,  8.51s/it]
 86%|████████▌ | 192819/225000 [364:53:37<64:27:43,  7.21s/it]
 86%|████████▌ | 192820/225000 [364:53:48<73:38:21,  8.24s/it]
 86%|████████▌ | 192821/225000 [364:53:53<65:12:26,  7.30s/it]
 86%|████████▌ | 192822/225000 [364:54:02<70:09:33,  7.85s/it]
 86%|████████▌ | 192823/225000 [364:54:14<81:36:55,  9.13s/it]
 86%|████████▌ | 192824/225000 [364:54:22<78:45:05,  8.81s/it]
 86%|████████▌ | 192825/225000 [364:54:30<77:03:13,  8.62s/it]
 86%|████████▌ | 192826/225000 [364:54:37<71:14:48,  7.97s/it]
 86%|████████▌ | 192827/225000 [364:54:49<83:55:42,  9.39s/it]
 86%|████████▌ | 192828/225000 [364:54:55<73:09:02,  8.19s/it]
 86%|████████▌ | 192829/225000 [364:55:01<68:45:20,  7.69s/it]
 86%|████████▌ | 192830/225000 [364:55:07<62:50:10,  7.03s/it]
 86%|████████▌ | 192831/225000 [364:55:12<58:04:21,  6.50s/it]
 86%|████████▌ | 192832/225000 [364:55:28<82:35:22,  9.24s/it]
 86%|████████▌ | 192833/225000 [364:55:34<76:16:26,  8.54s/it]
 86%|████████▌ | 192834/225000 [364:55:44<78:39:17,  8.80s/it]
 86%|████████▌ | 192835/225000 [364:55:54<81:13:20,  9.09s/it]
 86%|████████▌ | 192836/225000 [364:55:59<71:29:55,  8.00s/it]
 86%|████████▌ | 192837/225000 [364:56:06<67:24:59,  7.55s/it]
 86%|████████▌ | 192838/225000 [364:56:11<60:51:50,  6.81s/it]
 86%|████████▌ | 192839/225000 [364:56:22<73:56:10,  8.28s/it]
 86%|████████▌ | 192840/225000 [364:56:33<79:55:32,  8.95s/it]
 86%|████████▌ | 192841/225000 [364:56:38<69:47:06,  7.81s/it]
 86%|████████▌ | 192842/225000 [364:56:48<74:35:18,  8.35s/it]
 86%|████████▌ | 192843/225000 [364:56:52<63:58:33,  7.16s/it]
 86%|████████▌ | 192844/225000 [364:57:00<66:10:27,  7.41s/it]
 86%|████████▌ | 192845/225000 [364:57:12<77:40:36,  8.70s/it]
 86%|████████▌ | 192846/225000 [364:57:17<68:03:09,  7.62s/it]
 86%|████████▌ | 192847/225000 [364:57:23<64:18:56,  7.20s/it]
 86%|████████▌ | 192848/225000 [364:57:38<85:58:16,  9.63s/it]
 86%|████████▌ | 192849/225000 [364:57:42<70:19:01,  7.87s/it]
 86%|████████▌ | 192850/225000 [364:57:51<73:30:48,  8.23s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.117', 'learning_rate': '2.723e-06', 'epoch': '0.8571', 'num_input_tokens_seen': 607828408, 'train_runtime': '1.314e+06', 'train_tokens_per_second': '462.6'}

 86%|████████▌ | 192850/225000 [364:57:51<73:30:48,  8.23s/it]
 86%|████████▌ | 192851/225000 [364:57:56<63:30:50,  7.11s/it]
 86%|████████▌ | 192852/225000 [364:58:07<74:51:11,  8.38s/it]
 86%|████████▌ | 192853/225000 [364:58:22<91:37:47, 10.26s/it]
 86%|████████▌ | 192854/225000 [364:58:29<84:25:12,  9.45s/it]
 86%|████████▌ | 192855/225000 [364:58:38<82:11:03,  9.20s/it]
 86%|████████▌ | 192856/225000 [364:58:48<83:47:50,  9.38s/it]
 86%|████████▌ | 192857/225000 [364:58:55<77:31:59,  8.68s/it]
 86%|████████▌ | 192858/225000 [364:59:01<70:50:23,  7.93s/it]
 86%|████████▌ | 192859/225000 [364:59:12<78:11:19,  8.76s/it]
 86%|████████▌ | 192860/225000 [364:59:22<81:22:40,  9.12s/it]
 86%|████████▌ | 192861/225000 [364:59:28<74:41:38,  8.37s/it]
 86%|████████▌ | 192862/225000 [364:59:35<69:07:50,  7.74s/it]
 86%|████████▌ | 192863/225000 [364:59:43<71:41:41,  8.03s/it]
 86%|████████▌ | 192864/225000 [364:59:54<80:04:44,  8.97s/it]
 86%|████████▌ | 192865/225000 [365:00:03<78:52:20,  8.84s/it]
 86%|████████▌ | 192866/225000 [365:00:11<76:08:31,  8.53s/it]
 86%|████████▌ | 192867/225000 [365:00:21<80:53:16,  9.06s/it]
 86%|████████▌ | 192868/225000 [365:00:32<85:25:54,  9.57s/it]
 86%|████████▌ | 192869/225000 [365:00:44<92:20:03, 10.35s/it]
 86%|████████▌ | 192870/225000 [365:00:52<85:59:47,  9.64s/it]
 86%|████████▌ | 192871/225000 [365:01:02<87:16:05,  9.78s/it]
 86%|████████▌ | 192872/225000 [365:01:12<89:00:17,  9.97s/it]
 86%|████████▌ | 192873/225000 [365:01:22<87:24:18,  9.79s/it]
 86%|████████▌ | 192874/225000 [365:01:28<78:21:17,  8.78s/it]
 86%|████████▌ | 192875/225000 [365:01:37<79:12:44,  8.88s/it]
 86%|████████▌ | 192876/225000 [365:01:45<74:59:36,  8.40s/it]
 86%|████████▌ | 192877/225000 [365:01:51<70:41:14,  7.92s/it]
 86%|████████▌ | 192878/225000 [365:02:06<87:13:45,  9.78s/it]
 86%|████████▌ | 192879/225000 [365:02:12<78:09:32,  8.76s/it]
 86%|████████▌ | 192880/225000 [365:02:22<82:03:48,  9.20s/it]
 86%|████████▌ | 192881/225000 [365:02:29<76:04:24,  8.53s/it]
 86%|████████▌ | 192882/225000 [365:02:37<73:09:58,  8.20s/it]
 86%|████████▌ | 192883/225000 [365:02:43<69:19:41,  7.77s/it]
 86%|████████▌ | 192884/225000 [365:02:53<73:57:33,  8.29s/it]
 86%|████████▌ | 192885/225000 [365:03:00<70:25:08,  7.89s/it]
 86%|████████▌ | 192886/225000 [365:03:05<64:15:47,  7.20s/it]
 86%|████████▌ | 192887/225000 [365:03:11<60:13:25,  6.75s/it]
 86%|████████▌ | 192888/225000 [365:03:24<76:52:10,  8.62s/it]
 86%|████████▌ | 192889/225000 [365:03:31<71:21:15,  8.00s/it]
 86%|████████▌ | 192890/225000 [365:03:39<72:05:29,  8.08s/it]
 86%|████████▌ | 192891/225000 [365:03:46<69:43:21,  7.82s/it]
 86%|████████▌ | 192892/225000 [365:03:54<69:55:21,  7.84s/it]
 86%|████████▌ | 192893/225000 [365:04:00<64:52:16,  7.27s/it]
 86%|████████▌ | 192894/225000 [365:04:12<78:36:57,  8.82s/it]
 86%|████████▌ | 192895/225000 [365:04:20<76:05:46,  8.53s/it]
 86%|████████▌ | 192896/225000 [365:04:29<76:41:51,  8.60s/it]
 86%|████████▌ | 192897/225000 [365:04:35<69:19:36,  7.77s/it]
 86%|████████▌ | 192898/225000 [365:04:42<67:20:40,  7.55s/it]
 86%|████████▌ | 192899/225000 [365:04:48<63:53:23,  7.17s/it]
 86%|████████▌ | 192900/225000 [365:04:54<60:24:20,  6.77s/it]
                                                              
{'loss': '1.268', 'grad_norm': '1.607', 'learning_rate': '2.715e-06', 'epoch': '0.8573', 'num_input_tokens_seen': 608053288, 'train_runtime': '1.314e+06', 'train_tokens_per_second': '462.6'}

 86%|████████▌ | 192900/225000 [365:04:54<60:24:20,  6.77s/it]
 86%|████████▌ | 192901/225000 [365:05:08<80:01:29,  8.98s/it]
 86%|████████▌ | 192902/225000 [365:05:14<72:17:09,  8.11s/it]
 86%|████████▌ | 192903/225000 [365:05:21<69:06:28,  7.75s/it]
 86%|████████▌ | 192904/225000 [365:05:35<84:32:27,  9.48s/it]
 86%|████████▌ | 192905/225000 [365:05:39<71:30:52,  8.02s/it]
 86%|████████▌ | 192906/225000 [365:05:48<73:44:01,  8.27s/it]
 86%|████████▌ | 192907/225000 [365:06:03<92:02:40, 10.33s/it]
 86%|████████▌ | 192908/225000 [365:06:08<77:55:07,  8.74s/it]
 86%|████████▌ | 192909/225000 [365:06:16<74:52:16,  8.40s/it]
 86%|████████▌ | 192910/225000 [365:06:25<78:02:04,  8.75s/it]
 86%|████████▌ | 192911/225000 [365:06:35<78:59:28,  8.86s/it]
 86%|████████▌ | 192912/225000 [365:06:42<75:05:18,  8.42s/it]
 86%|████████▌ | 192913/225000 [365:06:47<66:45:37,  7.49s/it]
 86%|████████▌ | 192914/225000 [365:06:59<77:41:55,  8.72s/it]
 86%|████████▌ | 192915/225000 [365:07:05<69:48:58,  7.83s/it]
 86%|████████▌ | 192916/225000 [365:07:20<88:52:59,  9.97s/it]
 86%|████████▌ | 192917/225000 [365:07:25<76:28:02,  8.58s/it]
 86%|████████▌ | 192918/225000 [365:07:32<71:14:09,  7.99s/it]
 86%|████████▌ | 192919/225000 [365:07:42<78:56:59,  8.86s/it]
 86%|████████▌ | 192920/225000 [365:07:51<78:52:10,  8.85s/it]
 86%|████████▌ | 192921/225000 [365:07:57<69:28:12,  7.80s/it]
 86%|████████▌ | 192922/225000 [365:08:03<65:20:49,  7.33s/it]
 86%|████████▌ | 192923/225000 [365:08:15<77:55:47,  8.75s/it]
 86%|████████▌ | 192924/225000 [365:08:21<71:45:38,  8.05s/it]
 86%|████████▌ | 192925/225000 [365:08:32<79:57:28,  8.97s/it]
 86%|████████▌ | 192926/225000 [365:08:39<73:38:33,  8.27s/it]
 86%|████████▌ | 192927/225000 [365:08:49<77:02:27,  8.65s/it]
 86%|████████▌ | 192928/225000 [365:09:00<83:28:00,  9.37s/it]
 86%|████████▌ | 192929/225000 [365:09:08<79:29:08,  8.92s/it]
 86%|████████▌ | 192930/225000 [365:09:13<71:02:46,  7.98s/it]
 86%|████████▌ | 192931/225000 [365:09:24<79:08:19,  8.88s/it]
 86%|████████▌ | 192932/225000 [365:09:36<86:41:39,  9.73s/it]
 86%|████████▌ | 192933/225000 [365:09:47<90:02:54, 10.11s/it]
 86%|████████▌ | 192934/225000 [365:09:58<92:01:21, 10.33s/it]
 86%|████████▌ | 192935/225000 [365:10:05<83:41:04,  9.40s/it]
 86%|████████▌ | 192936/225000 [365:10:12<77:31:32,  8.70s/it]
 86%|████████▌ | 192937/225000 [365:10:17<68:33:56,  7.70s/it]
 86%|████████▌ | 192938/225000 [365:10:28<76:35:56,  8.60s/it]
 86%|████████▌ | 192939/225000 [365:10:39<82:36:55,  9.28s/it]
 86%|████████▌ | 192940/225000 [365:10:48<82:32:49,  9.27s/it]
 86%|████████▌ | 192941/225000 [365:10:56<78:28:18,  8.81s/it]
 86%|████████▌ | 192942/225000 [365:11:03<74:30:00,  8.37s/it]
 86%|████████▌ | 192943/225000 [365:11:14<81:24:37,  9.14s/it]
 86%|████████▌ | 192944/225000 [365:11:21<74:43:00,  8.39s/it]
 86%|████████▌ | 192945/225000 [365:11:30<76:48:56,  8.63s/it]
 86%|████████▌ | 192946/225000 [365:11:38<76:05:24,  8.55s/it]
 86%|████████▌ | 192947/225000 [365:11:44<68:32:16,  7.70s/it]
 86%|████████▌ | 192948/225000 [365:11:55<77:57:07,  8.76s/it]
 86%|████████▌ | 192949/225000 [365:12:06<82:57:58,  9.32s/it]
 86%|████████▌ | 192950/225000 [365:12:12<74:20:30,  8.35s/it]
                                                              
{'loss': '1.263', 'grad_norm': '1.247', 'learning_rate': '2.707e-06', 'epoch': '0.8576', 'num_input_tokens_seen': 608275448, 'train_runtime': '1.315e+06', 'train_tokens_per_second': '462.7'}

 86%|████████▌ | 192950/225000 [365:12:12<74:20:30,  8.35s/it]
 86%|████████▌ | 192951/225000 [365:12:26<89:59:19, 10.11s/it]
 86%|████████▌ | 192952/225000 [365:12:34<83:27:36,  9.38s/it]
 86%|████████▌ | 192953/225000 [365:12:46<89:57:45, 10.11s/it]
 86%|████████▌ | 192954/225000 [365:12:54<84:33:17,  9.50s/it]
 86%|████████▌ | 192955/225000 [365:13:00<74:43:06,  8.39s/it]
 86%|████████▌ | 192956/225000 [365:13:11<83:39:46,  9.40s/it]
 86%|████████▌ | 192957/225000 [365:13:21<84:56:31,  9.54s/it]
 86%|████████▌ | 192958/225000 [365:13:31<85:50:38,  9.64s/it]
 86%|████████▌ | 192959/225000 [365:13:39<81:29:48,  9.16s/it]
 86%|████████▌ | 192960/225000 [365:13:47<77:09:48,  8.67s/it]
 86%|████████▌ | 192961/225000 [365:13:53<71:16:13,  8.01s/it]
 86%|████████▌ | 192962/225000 [365:14:01<71:32:43,  8.04s/it]
 86%|████████▌ | 192963/225000 [365:14:09<69:19:50,  7.79s/it]
 86%|████████▌ | 192964/225000 [365:14:17<72:11:05,  8.11s/it]
 86%|████████▌ | 192965/225000 [365:14:27<76:57:42,  8.65s/it]
 86%|████████▌ | 192966/225000 [365:14:38<81:45:29,  9.19s/it]
 86%|████████▌ | 192967/225000 [365:14:46<77:57:40,  8.76s/it]
 86%|████████▌ | 192968/225000 [365:14:58<86:33:28,  9.73s/it]
 86%|████████▌ | 192969/225000 [365:15:04<77:40:08,  8.73s/it]
 86%|████████▌ | 192970/225000 [365:15:13<77:49:06,  8.75s/it]
 86%|████████▌ | 192971/225000 [365:15:20<74:53:14,  8.42s/it]
 86%|████████▌ | 192972/225000 [365:15:27<68:58:55,  7.75s/it]
 86%|████████▌ | 192973/225000 [365:15:35<70:52:00,  7.97s/it]
 86%|████████▌ | 192974/225000 [365:15:44<73:19:21,  8.24s/it]
 86%|████████▌ | 192975/225000 [365:15:52<72:31:05,  8.15s/it]
 86%|████████▌ | 192976/225000 [365:16:00<72:03:49,  8.10s/it]
 86%|████████▌ | 192977/225000 [365:16:11<80:22:04,  9.03s/it]
 86%|████████▌ | 192978/225000 [365:16:21<82:17:19,  9.25s/it]
 86%|████████▌ | 192979/225000 [365:16:33<89:22:31, 10.05s/it]
 86%|████████▌ | 192980/225000 [365:16:43<90:14:49, 10.15s/it]
 86%|████████▌ | 192981/225000 [365:16:51<85:17:57,  9.59s/it]
 86%|████████▌ | 192982/225000 [365:16:59<78:42:26,  8.85s/it]
 86%|████████▌ | 192983/225000 [365:17:04<70:07:08,  7.88s/it]
 86%|████████▌ | 192984/225000 [365:17:09<63:02:53,  7.09s/it]
 86%|████████▌ | 192985/225000 [365:17:14<57:23:19,  6.45s/it]
 86%|████████▌ | 192986/225000 [365:17:31<84:53:19,  9.55s/it]
 86%|████████▌ | 192987/225000 [365:17:41<84:28:36,  9.50s/it]
 86%|████████▌ | 192988/225000 [365:17:53<91:56:44, 10.34s/it]
 86%|████████▌ | 192989/225000 [365:17:59<79:35:41,  8.95s/it]
 86%|████████▌ | 192990/225000 [365:18:07<77:03:24,  8.67s/it]
 86%|████████▌ | 192991/225000 [365:18:15<76:42:02,  8.63s/it]
 86%|████████▌ | 192992/225000 [365:18:21<68:56:24,  7.75s/it]
 86%|████████▌ | 192993/225000 [365:18:26<62:01:22,  6.98s/it]
 86%|████████▌ | 192994/225000 [365:18:35<68:51:54,  7.75s/it]
 86%|████████▌ | 192995/225000 [365:18:45<72:55:39,  8.20s/it]
 86%|████████▌ | 192996/225000 [365:18:56<80:39:58,  9.07s/it]
 86%|████████▌ | 192997/225000 [365:19:01<70:22:34,  7.92s/it]
 86%|████████▌ | 192998/225000 [365:19:12<78:10:10,  8.79s/it]
 86%|████████▌ | 192999/225000 [365:19:24<88:07:01,  9.91s/it]
 86%|████████▌ | 193000/225000 [365:19:31<78:37:49,  8.85s/it]
                                                              
{'loss': '1.278', 'grad_norm': '1.313', 'learning_rate': '2.699e-06', 'epoch': '0.8578', 'num_input_tokens_seen': 608499648, 'train_runtime': '1.315e+06', 'train_tokens_per_second': '462.7'}

 86%|████████▌ | 193000/225000 [365:19:31<78:37:49,  8.85s/it]
 86%|████████▌ | 193001/225000 [365:19:38<73:39:08,  8.29s/it]
 86%|████████▌ | 193002/225000 [365:19:45<71:57:27,  8.10s/it]
 86%|████████▌ | 193003/225000 [365:19:53<71:06:39,  8.00s/it]
 86%|████████▌ | 193004/225000 [365:20:06<83:53:12,  9.44s/it]
 86%|████████▌ | 193005/225000 [365:20:12<75:38:42,  8.51s/it]
 86%|████████▌ | 193006/225000 [365:20:23<82:00:53,  9.23s/it]
 86%|████████▌ | 193007/225000 [365:20:28<71:25:04,  8.04s/it]
 86%|████████▌ | 193008/225000 [365:20:38<75:25:13,  8.49s/it]
 86%|████████▌ | 193009/225000 [365:20:43<66:03:21,  7.43s/it]
 86%|████████▌ | 193010/225000 [365:20:51<67:34:14,  7.60s/it]
 86%|████████▌ | 193011/225000 [365:20:56<61:11:42,  6.89s/it]
 86%|████████▌ | 193012/225000 [365:21:08<74:10:15,  8.35s/it]
 86%|████████▌ | 193013/225000 [365:21:15<69:58:15,  7.87s/it]
 86%|████████▌ | 193014/225000 [365:21:25<76:38:10,  8.63s/it]
 86%|████████▌ | 193015/225000 [365:21:30<67:30:29,  7.60s/it]
 86%|████████▌ | 193016/225000 [365:21:41<75:31:01,  8.50s/it]
 86%|████████▌ | 193017/225000 [365:21:49<73:06:46,  8.23s/it]
 86%|████████▌ | 193018/225000 [365:22:02<88:22:49,  9.95s/it]
 86%|████████▌ | 193019/225000 [365:22:13<90:24:22, 10.18s/it]
 86%|████████▌ | 193020/225000 [365:22:21<83:13:10,  9.37s/it]
 86%|████████▌ | 193021/225000 [365:22:29<80:52:44,  9.10s/it]
 86%|████████▌ | 193022/225000 [365:22:39<81:39:14,  9.19s/it]
 86%|████████▌ | 193023/225000 [365:22:46<76:36:56,  8.63s/it]
 86%|████████▌ | 193024/225000 [365:22:52<70:39:13,  7.95s/it]
 86%|████████▌ | 193025/225000 [365:23:01<71:31:23,  8.05s/it]
 86%|████████▌ | 193026/225000 [365:23:07<68:33:13,  7.72s/it]
 86%|████████▌ | 193027/225000 [365:23:15<67:31:20,  7.60s/it]
 86%|████████▌ | 193028/225000 [365:23:29<85:23:16,  9.61s/it]
 86%|████████▌ | 193029/225000 [365:23:35<74:55:37,  8.44s/it]
 86%|████████▌ | 193030/225000 [365:23:41<67:58:43,  7.65s/it]
 86%|████████▌ | 193031/225000 [365:23:55<86:34:42,  9.75s/it]
 86%|████████▌ | 193032/225000 [365:24:06<87:51:58,  9.89s/it]
 86%|████████▌ | 193033/225000 [365:24:12<78:54:17,  8.89s/it]
 86%|████████▌ | 193034/225000 [365:24:18<70:45:19,  7.97s/it]
 86%|████████▌ | 193035/225000 [365:24:28<76:46:09,  8.65s/it]
 86%|████████▌ | 193036/225000 [365:24:36<74:31:50,  8.39s/it]
 86%|████████▌ | 193037/225000 [365:24:41<66:56:15,  7.54s/it]
 86%|████████▌ | 193038/225000 [365:24:49<66:17:15,  7.47s/it]
 86%|████████▌ | 193039/225000 [365:24:55<63:03:26,  7.10s/it]
 86%|████████▌ | 193040/225000 [365:25:06<72:32:38,  8.17s/it]
 86%|████████▌ | 193041/225000 [365:25:14<73:42:46,  8.30s/it]
 86%|████████▌ | 193042/225000 [365:25:25<80:04:17,  9.02s/it]
 86%|████████▌ | 193043/225000 [365:25:31<72:41:03,  8.19s/it]
 86%|████████▌ | 193044/225000 [365:25:38<68:01:49,  7.66s/it]
 86%|████████▌ | 193045/225000 [365:25:50<80:47:32,  9.10s/it]
 86%|████████▌ | 193046/225000 [365:25:55<69:48:32,  7.86s/it]
 86%|████████▌ | 193047/225000 [365:26:00<61:44:36,  6.96s/it]
 86%|████████▌ | 193048/225000 [365:26:09<66:48:32,  7.53s/it]
 86%|████████▌ | 193049/225000 [365:26:23<84:08:18,  9.48s/it]
 86%|████████▌ | 193050/225000 [365:26:28<72:19:05,  8.15s/it]
                                                              
{'loss': '1.27', 'grad_norm': '1.316', 'learning_rate': '2.69e-06', 'epoch': '0.858', 'num_input_tokens_seen': 608719896, 'train_runtime': '1.316e+06', 'train_tokens_per_second': '462.7'}

 86%|████████▌ | 193050/225000 [365:26:28<72:19:05,  8.15s/it]
 86%|████████▌ | 193051/225000 [365:26:35<69:57:03,  7.88s/it]
 86%|████████▌ | 193052/225000 [365:26:48<83:54:19,  9.45s/it]
 86%|████████▌ | 193053/225000 [365:26:55<75:23:11,  8.50s/it]
 86%|████████▌ | 193054/225000 [365:27:09<91:48:34, 10.35s/it]
 86%|████████▌ | 193055/225000 [365:27:13<74:29:49,  8.40s/it]
 86%|████████▌ | 193056/225000 [365:27:23<78:14:09,  8.82s/it]
 86%|████████▌ | 193057/225000 [365:27:30<73:25:45,  8.28s/it]
 86%|████████▌ | 193058/225000 [365:27:37<69:36:28,  7.85s/it]
 86%|████████▌ | 193059/225000 [365:27:47<77:07:51,  8.69s/it]
 86%|████████▌ | 193060/225000 [365:27:55<73:58:00,  8.34s/it]
 86%|████████▌ | 193061/225000 [365:28:04<76:27:58,  8.62s/it]
 86%|████████▌ | 193062/225000 [365:28:10<69:57:40,  7.89s/it]
 86%|████████▌ | 193063/225000 [365:28:18<69:46:35,  7.87s/it]
 86%|████████▌ | 193064/225000 [365:28:25<67:44:35,  7.64s/it]
 86%|████████▌ | 193065/225000 [365:28:30<61:05:42,  6.89s/it]
 86%|████████▌ | 193066/225000 [365:28:39<65:14:01,  7.35s/it]
 86%|████████▌ | 193067/225000 [365:28:47<67:50:45,  7.65s/it]
 86%|████████▌ | 193068/225000 [365:29:00<82:50:41,  9.34s/it]
 86%|████████▌ | 193069/225000 [365:29:08<77:53:17,  8.78s/it]
 86%|████████▌ | 193070/225000 [365:29:18<81:45:08,  9.22s/it]
 86%|████████▌ | 193071/225000 [365:29:29<87:09:38,  9.83s/it]
 86%|████████▌ | 193072/225000 [365:29:41<91:51:55, 10.36s/it]
 86%|████████▌ | 193073/225000 [365:29:48<83:13:58,  9.39s/it]
 86%|████████▌ | 193074/225000 [365:29:53<72:13:05,  8.14s/it]
 86%|████████▌ | 193075/225000 [365:30:01<70:17:00,  7.93s/it]
 86%|████████▌ | 193076/225000 [365:30:08<68:56:13,  7.77s/it]
 86%|████████▌ | 193077/225000 [365:30:15<66:08:20,  7.46s/it]
 86%|████████▌ | 193078/225000 [365:30:28<80:16:32,  9.05s/it]
 86%|████████▌ | 193079/225000 [365:30:32<68:59:48,  7.78s/it]
 86%|████████▌ | 193080/225000 [365:30:44<79:46:36,  9.00s/it]
 86%|████████▌ | 193081/225000 [365:30:50<70:32:45,  7.96s/it]
 86%|████████▌ | 193082/225000 [365:31:03<84:01:31,  9.48s/it]
 86%|████████▌ | 193083/225000 [365:31:09<75:39:21,  8.53s/it]
 86%|████████▌ | 193084/225000 [365:31:16<71:28:09,  8.06s/it]
 86%|████████▌ | 193085/225000 [365:31:26<76:36:08,  8.64s/it]
 86%|████████▌ | 193086/225000 [365:31:33<70:34:08,  7.96s/it]
 86%|████████▌ | 193087/225000 [365:31:42<74:54:40,  8.45s/it]
 86%|████████▌ | 193088/225000 [365:31:57<91:00:32, 10.27s/it]
 86%|████████▌ | 193089/225000 [365:32:02<77:31:49,  8.75s/it]
 86%|████████▌ | 193090/225000 [365:32:13<84:53:30,  9.58s/it]
 86%|████████▌ | 193091/225000 [365:32:20<77:35:09,  8.75s/it]
 86%|████████▌ | 193092/225000 [365:32:26<70:52:46,  8.00s/it]
 86%|████████▌ | 193093/225000 [365:32:37<77:08:38,  8.70s/it]
 86%|████████▌ | 193094/225000 [365:32:42<68:01:42,  7.68s/it]
 86%|████████▌ | 193095/225000 [365:32:56<85:49:31,  9.68s/it]
 86%|████████▌ | 193096/225000 [365:33:03<78:48:56,  8.89s/it]
 86%|████████▌ | 193097/225000 [365:33:13<80:40:26,  9.10s/it]
 86%|████████▌ | 193098/225000 [365:33:17<67:12:44,  7.58s/it]
 86%|████████▌ | 193099/225000 [365:33:25<67:43:58,  7.64s/it]
 86%|████████▌ | 193100/225000 [365:33:32<66:48:20,  7.54s/it]
                                                              
{'loss': '1.267', 'grad_norm': '1.405', 'learning_rate': '2.682e-06', 'epoch': '0.8582', 'num_input_tokens_seen': 608942456, 'train_runtime': '1.316e+06', 'train_tokens_per_second': '462.7'}

 86%|████████▌ | 193100/225000 [365:33:32<66:48:20,  7.54s/it]
 86%|████████▌ | 193101/225000 [365:33:44<79:12:08,  8.94s/it]
 86%|████████▌ | 193102/225000 [365:33:51<71:57:51,  8.12s/it]
 86%|████████▌ | 193103/225000 [365:33:55<63:05:43,  7.12s/it]
 86%|████████▌ | 193104/225000 [365:34:01<59:36:50,  6.73s/it]
 86%|████████▌ | 193105/225000 [365:34:07<58:18:42,  6.58s/it]
 86%|████████▌ | 193106/225000 [365:34:20<75:06:10,  8.48s/it]
 86%|████████▌ | 193107/225000 [365:34:30<77:08:12,  8.71s/it]
 86%|████████▌ | 193108/225000 [365:34:35<69:37:13,  7.86s/it]
 86%|████████▌ | 193109/225000 [365:34:43<67:38:24,  7.64s/it]
 86%|████████▌ | 193110/225000 [365:34:57<86:13:24,  9.73s/it]
 86%|████████▌ | 193111/225000 [365:35:06<82:28:31,  9.31s/it]
 86%|████████▌ | 193112/225000 [365:35:13<77:47:11,  8.78s/it]
 86%|████████▌ | 193113/225000 [365:35:24<84:00:56,  9.49s/it]
 86%|████████▌ | 193114/225000 [365:35:31<77:56:12,  8.80s/it]
 86%|████████▌ | 193115/225000 [365:35:37<69:45:13,  7.88s/it]
 86%|████████▌ | 193116/225000 [365:35:48<76:30:09,  8.64s/it]
 86%|████████▌ | 193117/225000 [365:35:54<70:51:45,  8.00s/it]
 86%|████████▌ | 193118/225000 [365:36:00<66:06:48,  7.47s/it]
 86%|████████▌ | 193119/225000 [365:36:11<75:09:58,  8.49s/it]
 86%|████████▌ | 193120/225000 [365:36:20<75:48:16,  8.56s/it]
 86%|████████▌ | 193121/225000 [365:36:32<84:44:00,  9.57s/it]
 86%|████████▌ | 193122/225000 [365:36:40<82:10:47,  9.28s/it]
 86%|████████▌ | 193123/225000 [365:36:46<73:00:29,  8.25s/it]
 86%|████████▌ | 193124/225000 [365:36:52<67:21:34,  7.61s/it]
 86%|████████▌ | 193125/225000 [365:37:06<84:41:40,  9.57s/it]
 86%|████████▌ | 193126/225000 [365:37:14<78:30:56,  8.87s/it]
 86%|████████▌ | 193127/225000 [365:37:21<75:22:21,  8.51s/it]
 86%|████████▌ | 193128/225000 [365:37:27<66:33:06,  7.52s/it]
 86%|████████▌ | 193129/225000 [365:37:38<76:38:37,  8.66s/it]
 86%|████████▌ | 193130/225000 [365:37:46<74:10:36,  8.38s/it]
 86%|████████▌ | 193131/225000 [365:37:54<73:33:36,  8.31s/it]
 86%|████████▌ | 193132/225000 [365:38:02<73:56:34,  8.35s/it]
 86%|████████▌ | 193133/225000 [365:38:13<79:00:37,  8.93s/it]
 86%|████████▌ | 193134/225000 [365:38:21<78:29:35,  8.87s/it]
 86%|████████▌ | 193135/225000 [365:38:26<68:27:17,  7.73s/it]
 86%|████████▌ | 193136/225000 [365:38:34<68:00:56,  7.68s/it]
 86%|████████▌ | 193137/225000 [365:38:46<79:48:36,  9.02s/it]
 86%|████████▌ | 193138/225000 [365:38:51<69:40:11,  7.87s/it]
 86%|████████▌ | 193139/225000 [365:39:00<71:22:43,  8.07s/it]
 86%|████████▌ | 193140/225000 [365:39:09<75:16:38,  8.51s/it]
 86%|████████▌ | 193141/225000 [365:39:22<86:30:38,  9.78s/it]
 86%|████████▌ | 193142/225000 [365:39:27<74:58:18,  8.47s/it]
 86%|████████▌ | 193143/225000 [365:39:33<68:30:34,  7.74s/it]
 86%|████████▌ | 193144/225000 [365:39:48<86:54:00,  9.82s/it]
 86%|████████▌ | 193145/225000 [365:39:55<78:48:33,  8.91s/it]
 86%|████████▌ | 193146/225000 [365:40:01<70:26:29,  7.96s/it]
 86%|████████▌ | 193147/225000 [365:40:08<68:24:15,  7.73s/it]
 86%|████████▌ | 193148/225000 [365:40:16<70:04:33,  7.92s/it]
 86%|████████▌ | 193149/225000 [365:40:23<67:20:05,  7.61s/it]
 86%|████████▌ | 193150/225000 [365:40:37<85:05:27,  9.62s/it]
                                                              
{'loss': '1.282', 'grad_norm': '1.184', 'learning_rate': '2.674e-06', 'epoch': '0.8584', 'num_input_tokens_seen': 609162080, 'train_runtime': '1.316e+06', 'train_tokens_per_second': '462.7'}

 86%|████████▌ | 193150/225000 [365:40:37<85:05:27,  9.62s/it]
 86%|████████▌ | 193151/225000 [365:40:44<76:52:39,  8.69s/it]
 86%|████████▌ | 193152/225000 [365:40:54<81:04:07,  9.16s/it]
 86%|████████▌ | 193153/225000 [365:41:02<78:29:40,  8.87s/it]
 86%|████████▌ | 193154/225000 [365:41:13<83:26:00,  9.43s/it]
 86%|████████▌ | 193155/225000 [365:41:23<85:42:05,  9.69s/it]
 86%|████████▌ | 193156/225000 [365:41:33<84:17:40,  9.53s/it]
 86%|████████▌ | 193157/225000 [365:41:40<77:55:35,  8.81s/it]
 86%|████████▌ | 193158/225000 [365:41:48<77:31:45,  8.77s/it]
 86%|████████▌ | 193159/225000 [365:41:58<79:40:52,  9.01s/it]
 86%|████████▌ | 193160/225000 [365:42:06<78:21:02,  8.86s/it]
 86%|████████▌ | 193161/225000 [365:42:14<75:52:53,  8.58s/it]
 86%|████████▌ | 193162/225000 [365:42:26<83:37:35,  9.46s/it]
 86%|████████▌ | 193163/225000 [365:42:30<69:45:08,  7.89s/it]
 86%|████████▌ | 193164/225000 [365:42:40<74:52:26,  8.47s/it]
 86%|████████▌ | 193165/225000 [365:42:52<83:08:54,  9.40s/it]
 86%|████████▌ | 193166/225000 [365:42:59<78:12:40,  8.84s/it]
 86%|████████▌ | 193167/225000 [365:43:08<78:28:19,  8.87s/it]
 86%|████████▌ | 193168/225000 [365:43:16<75:08:11,  8.50s/it]
 86%|████████▌ | 193169/225000 [365:43:21<66:46:16,  7.55s/it]
 86%|████████▌ | 193170/225000 [365:43:26<58:55:59,  6.67s/it]
 86%|████████▌ | 193171/225000 [365:43:37<72:47:09,  8.23s/it]
 86%|████████▌ | 193172/225000 [365:43:43<66:23:11,  7.51s/it]
 86%|████████▌ | 193173/225000 [365:43:50<63:22:56,  7.17s/it]
 86%|████████▌ | 193174/225000 [365:43:56<61:41:48,  6.98s/it]
 86%|████████▌ | 193175/225000 [365:44:03<61:29:33,  6.96s/it]
 86%|████████▌ | 193176/225000 [365:44:15<75:25:50,  8.53s/it]
 86%|████████▌ | 193177/225000 [365:44:22<71:02:03,  8.04s/it]
 86%|████████▌ | 193178/225000 [365:44:28<64:21:34,  7.28s/it]
 86%|████████▌ | 193179/225000 [365:44:35<63:18:10,  7.16s/it]
 86%|████████▌ | 193180/225000 [365:44:42<63:18:01,  7.16s/it]
 86%|████████▌ | 193181/225000 [365:44:56<81:24:27,  9.21s/it]
 86%|████████▌ | 193182/225000 [365:45:01<71:12:32,  8.06s/it]
 86%|████████▌ | 193183/225000 [365:45:13<81:31:39,  9.22s/it]
 86%|████████▌ | 193184/225000 [365:45:18<70:58:34,  8.03s/it]
 86%|████████▌ | 193185/225000 [365:45:27<71:27:41,  8.09s/it]
 86%|████████▌ | 193186/225000 [365:45:36<75:55:38,  8.59s/it]
 86%|████████▌ | 193187/225000 [365:45:43<71:46:20,  8.12s/it]
 86%|████████▌ | 193188/225000 [365:45:53<76:20:10,  8.64s/it]
 86%|████████▌ | 193189/225000 [365:46:01<72:55:39,  8.25s/it]
 86%|████████▌ | 193190/225000 [365:46:10<75:52:12,  8.59s/it]
 86%|████████▌ | 193191/225000 [365:46:15<67:14:40,  7.61s/it]
 86%|████████▌ | 193192/225000 [365:46:25<72:11:55,  8.17s/it]
 86%|████████▌ | 193193/225000 [365:46:30<65:44:52,  7.44s/it]
 86%|████████▌ | 193194/225000 [365:46:41<74:24:32,  8.42s/it]
 86%|████████▌ | 193195/225000 [365:46:47<66:46:37,  7.56s/it]
 86%|████████▌ | 193196/225000 [365:46:52<60:57:56,  6.90s/it]
 86%|████████▌ | 193197/225000 [365:47:01<67:26:58,  7.64s/it]
 86%|████████▌ | 193198/225000 [365:47:10<70:45:58,  8.01s/it]
 86%|████████▌ | 193199/225000 [365:47:17<66:02:36,  7.48s/it]
 86%|████████▌ | 193200/225000 [365:47:23<63:22:09,  7.17s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.135', 'learning_rate': '2.666e-06', 'epoch': '0.8587', 'num_input_tokens_seen': 609380816, 'train_runtime': '1.317e+06', 'train_tokens_per_second': '462.8'}

 86%|████████▌ | 193200/225000 [365:47:23<63:22:09,  7.17s/it]
 86%|████████▌ | 193201/225000 [365:47:33<70:39:15,  8.00s/it]
 86%|████████▌ | 193202/225000 [365:47:44<78:09:15,  8.85s/it]
 86%|████████▌ | 193203/225000 [365:47:49<69:44:53,  7.90s/it]
 86%|████████▌ | 193204/225000 [365:47:56<65:59:09,  7.47s/it]
 86%|████████▌ | 193205/225000 [365:48:05<69:38:19,  7.88s/it]
 86%|████████▌ | 193206/225000 [365:48:16<79:03:23,  8.95s/it]
 86%|████████▌ | 193207/225000 [365:48:24<76:45:42,  8.69s/it]
 86%|████████▌ | 193208/225000 [365:48:36<83:50:12,  9.49s/it]
 86%|████████▌ | 193209/225000 [365:48:42<75:56:29,  8.60s/it]
 86%|████████▌ | 193210/225000 [365:48:53<81:31:21,  9.23s/it]
 86%|████████▌ | 193211/225000 [365:49:00<77:06:38,  8.73s/it]
 86%|████████▌ | 193212/225000 [365:49:12<85:28:05,  9.68s/it]
 86%|████████▌ | 193213/225000 [365:49:16<70:26:18,  7.98s/it]
 86%|████████▌ | 193214/225000 [365:49:24<69:42:06,  7.89s/it]
 86%|████████▌ | 193215/225000 [365:49:35<77:32:46,  8.78s/it]
 86%|████████▌ | 193216/225000 [365:49:42<71:58:04,  8.15s/it]
 86%|████████▌ | 193217/225000 [365:49:50<72:22:44,  8.20s/it]
 86%|████████▌ | 193218/225000 [365:49:58<71:02:36,  8.05s/it]
 86%|████████▌ | 193219/225000 [365:50:02<62:09:49,  7.04s/it]
 86%|████████▌ | 193220/225000 [365:50:12<70:30:20,  7.99s/it]
 86%|████████▌ | 193221/225000 [365:50:18<65:16:31,  7.39s/it]
 86%|████████▌ | 193222/225000 [365:50:26<65:56:00,  7.47s/it]
 86%|████████▌ | 193223/225000 [365:50:38<76:35:32,  8.68s/it]
 86%|████████▌ | 193224/225000 [365:50:45<72:20:11,  8.20s/it]
 86%|████████▌ | 193225/225000 [365:50:54<76:38:30,  8.68s/it]
 86%|████████▌ | 193226/225000 [365:51:00<68:28:57,  7.76s/it]
 86%|████████▌ | 193227/225000 [365:51:06<64:39:47,  7.33s/it]
 86%|████████▌ | 193228/225000 [365:51:14<65:00:52,  7.37s/it]
 86%|████████▌ | 193229/225000 [365:51:27<79:33:31,  9.01s/it]
 86%|████████▌ | 193230/225000 [365:51:36<80:48:05,  9.16s/it]
 86%|████████▌ | 193231/225000 [365:51:42<71:30:14,  8.10s/it]
 86%|████████▌ | 193232/225000 [365:51:47<63:52:28,  7.24s/it]
 86%|████████▌ | 193233/225000 [365:51:58<74:48:25,  8.48s/it]
 86%|████████▌ | 193234/225000 [365:52:09<80:57:20,  9.17s/it]
 86%|████████▌ | 193235/225000 [365:52:15<73:00:03,  8.27s/it]
 86%|████████▌ | 193236/225000 [365:52:25<77:31:31,  8.79s/it]
 86%|████████▌ | 193237/225000 [365:52:31<69:14:15,  7.85s/it]
 86%|████████▌ | 193238/225000 [365:52:38<67:46:09,  7.68s/it]
 86%|████████▌ | 193239/225000 [365:52:47<69:25:15,  7.87s/it]
 86%|████████▌ | 193240/225000 [365:52:53<66:14:12,  7.51s/it]
 86%|████████▌ | 193241/225000 [365:53:06<80:49:53,  9.16s/it]
 86%|████████▌ | 193242/225000 [365:53:17<85:38:31,  9.71s/it]
 86%|████████▌ | 193243/225000 [365:53:29<89:55:47, 10.19s/it]
 86%|████████▌ | 193244/225000 [365:53:35<81:00:36,  9.18s/it]
 86%|████████▌ | 193245/225000 [365:53:44<78:16:23,  8.87s/it]
 86%|████████▌ | 193246/225000 [365:53:55<84:44:47,  9.61s/it]
 86%|████████▌ | 193247/225000 [365:54:01<76:20:39,  8.66s/it]
 86%|████████▌ | 193248/225000 [365:54:09<72:22:08,  8.21s/it]
 86%|████████▌ | 193249/225000 [365:54:15<67:24:47,  7.64s/it]
 86%|████████▌ | 193250/225000 [365:54:26<76:33:54,  8.68s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.198', 'learning_rate': '2.657e-06', 'epoch': '0.8589', 'num_input_tokens_seen': 609600736, 'train_runtime': '1.317e+06', 'train_tokens_per_second': '462.8'}

 86%|████████▌ | 193250/225000 [365:54:26<76:33:54,  8.68s/it]
 86%|████████▌ | 193251/225000 [365:54:36<79:31:19,  9.02s/it]
 86%|████████▌ | 193252/225000 [365:54:43<74:52:28,  8.49s/it]
 86%|████████▌ | 193253/225000 [365:54:52<75:45:21,  8.59s/it]
 86%|████████▌ | 193254/225000 [365:55:02<80:35:54,  9.14s/it]
 86%|████████▌ | 193255/225000 [365:55:09<73:31:18,  8.34s/it]
 86%|████████▌ | 193256/225000 [365:55:14<66:32:52,  7.55s/it]
 86%|████████▌ | 193257/225000 [365:55:21<64:46:58,  7.35s/it]
 86%|████████▌ | 193258/225000 [365:55:35<80:45:46,  9.16s/it]
 86%|████████▌ | 193259/225000 [365:55:44<81:24:10,  9.23s/it]
 86%|████████▌ | 193260/225000 [365:55:53<80:38:02,  9.15s/it]
 86%|████████▌ | 193261/225000 [365:56:03<83:14:57,  9.44s/it]
 86%|████████▌ | 193262/225000 [365:56:10<75:00:14,  8.51s/it]
 86%|████████▌ | 193263/225000 [365:56:20<78:52:44,  8.95s/it]
 86%|████████▌ | 193264/225000 [365:56:27<76:02:26,  8.63s/it]
 86%|████████▌ | 193265/225000 [365:56:35<73:10:58,  8.30s/it]
 86%|████████▌ | 193266/225000 [365:56:42<70:32:48,  8.00s/it]
 86%|████████▌ | 193267/225000 [365:56:52<74:41:48,  8.47s/it]
 86%|████████▌ | 193268/225000 [365:56:58<68:42:02,  7.79s/it]
 86%|████████▌ | 193269/225000 [365:57:04<62:55:34,  7.14s/it]
 86%|████████▌ | 193270/225000 [365:57:09<58:06:10,  6.59s/it]
 86%|████████▌ | 193271/225000 [365:57:20<71:08:48,  8.07s/it]
 86%|████████▌ | 193272/225000 [365:57:30<75:22:28,  8.55s/it]
 86%|████████▌ | 193273/225000 [365:57:40<79:16:59,  9.00s/it]
 86%|████████▌ | 193274/225000 [365:57:48<75:32:14,  8.57s/it]
 86%|████████▌ | 193275/225000 [365:57:58<79:41:21,  9.04s/it]
 86%|████████▌ | 193276/225000 [365:58:08<82:36:29,  9.37s/it]
 86%|████████▌ | 193277/225000 [365:58:13<71:37:40,  8.13s/it]
 86%|████████▌ | 193278/225000 [365:58:20<66:40:29,  7.57s/it]
 86%|████████▌ | 193279/225000 [365:58:32<80:30:13,  9.14s/it]
 86%|████████▌ | 193280/225000 [365:58:45<88:55:17, 10.09s/it]
 86%|████████▌ | 193281/225000 [365:58:50<76:30:39,  8.68s/it]
 86%|████████▌ | 193282/225000 [365:58:56<69:14:47,  7.86s/it]
 86%|████████▌ | 193283/225000 [365:59:07<78:26:44,  8.90s/it]
 86%|████████▌ | 193284/225000 [365:59:16<78:47:10,  8.94s/it]
 86%|████████▌ | 193285/225000 [365:59:22<69:36:49,  7.90s/it]
 86%|████████▌ | 193286/225000 [365:59:29<67:24:18,  7.65s/it]
 86%|████████▌ | 193287/225000 [365:59:38<70:57:59,  8.06s/it]
 86%|████████▌ | 193288/225000 [365:59:43<64:08:09,  7.28s/it]
 86%|████████▌ | 193289/225000 [365:59:50<61:21:42,  6.97s/it]
 86%|████████▌ | 193290/225000 [365:59:56<59:14:30,  6.73s/it]
 86%|████████▌ | 193291/225000 [366:00:03<61:44:47,  7.01s/it]
 86%|████████▌ | 193292/225000 [366:00:16<75:51:14,  8.61s/it]
 86%|████████▌ | 193293/225000 [366:00:23<71:51:52,  8.16s/it]
 86%|████████▌ | 193294/225000 [366:00:36<84:27:46,  9.59s/it]
 86%|████████▌ | 193295/225000 [366:00:41<71:32:00,  8.12s/it]
 86%|████████▌ | 193296/225000 [366:00:48<68:59:03,  7.83s/it]
 86%|████████▌ | 193297/225000 [366:00:58<74:54:56,  8.51s/it]
 86%|████████▌ | 193298/225000 [366:01:02<64:25:43,  7.32s/it]
 86%|████████▌ | 193299/225000 [366:01:09<62:48:40,  7.13s/it]
 86%|████████▌ | 193300/225000 [366:01:15<60:01:57,  6.82s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.223', 'learning_rate': '2.649e-06', 'epoch': '0.8591', 'num_input_tokens_seen': 609821120, 'train_runtime': '1.318e+06', 'train_tokens_per_second': '462.8'}

 86%|████████▌ | 193300/225000 [366:01:15<60:01:57,  6.82s/it]
 86%|████████▌ | 193301/225000 [366:01:28<75:21:37,  8.56s/it]
 86%|████████▌ | 193302/225000 [366:01:40<86:26:16,  9.82s/it]
 86%|████████▌ | 193303/225000 [366:01:48<80:33:35,  9.15s/it]
 86%|████████▌ | 193304/225000 [366:01:56<77:55:43,  8.85s/it]
 86%|████████▌ | 193305/225000 [366:02:06<80:52:37,  9.19s/it]
 86%|████████▌ | 193306/225000 [366:02:13<73:39:15,  8.37s/it]
 86%|████████▌ | 193307/225000 [366:02:20<70:56:03,  8.06s/it]
 86%|████████▌ | 193308/225000 [366:02:31<77:43:27,  8.83s/it]
 86%|████████▌ | 193309/225000 [366:02:39<75:19:54,  8.56s/it]
 86%|████████▌ | 193310/225000 [366:02:45<71:06:42,  8.08s/it]
 86%|████████▌ | 193311/225000 [366:02:58<83:40:52,  9.51s/it]
 86%|████████▌ | 193312/225000 [366:03:09<86:00:05,  9.77s/it]
 86%|████████▌ | 193313/225000 [366:03:26<105:45:35, 12.02s/it]
 86%|████████▌ | 193314/225000 [366:03:37<102:46:48, 11.68s/it]
 86%|████████▌ | 193315/225000 [366:03:45<94:40:35, 10.76s/it] 
 86%|████████▌ | 193316/225000 [366:03:53<86:08:13,  9.79s/it]
 86%|████████▌ | 193317/225000 [366:04:03<87:25:18,  9.93s/it]
 86%|████████▌ | 193318/225000 [366:04:11<81:52:09,  9.30s/it]
 86%|████████▌ | 193319/225000 [366:04:21<83:35:31,  9.50s/it]
 86%|████████▌ | 193320/225000 [366:04:29<78:20:09,  8.90s/it]
 86%|████████▌ | 193321/225000 [366:04:38<79:23:20,  9.02s/it]
 86%|████████▌ | 193322/225000 [366:04:45<73:59:39,  8.41s/it]
 86%|████████▌ | 193323/225000 [366:04:50<66:36:52,  7.57s/it]
 86%|████████▌ | 193324/225000 [366:05:06<88:35:36, 10.07s/it]
 86%|████████▌ | 193325/225000 [366:05:15<85:15:53,  9.69s/it]
 86%|████████▌ | 193326/225000 [366:05:24<82:30:54,  9.38s/it]
 86%|████████▌ | 193327/225000 [366:05:32<80:15:21,  9.12s/it]
 86%|████████▌ | 193328/225000 [366:05:38<69:57:17,  7.95s/it]
 86%|████████▌ | 193329/225000 [366:05:47<74:35:29,  8.48s/it]
 86%|████████▌ | 193330/225000 [366:05:53<66:11:06,  7.52s/it]
 86%|████████▌ | 193331/225000 [366:06:01<67:54:49,  7.72s/it]
 86%|████████▌ | 193332/225000 [366:06:11<73:27:05,  8.35s/it]
 86%|████████▌ | 193333/225000 [366:06:17<68:36:23,  7.80s/it]
 86%|████████▌ | 193334/225000 [366:06:26<72:46:05,  8.27s/it]
 86%|████████▌ | 193335/225000 [366:06:31<63:27:41,  7.21s/it]
 86%|████████▌ | 193336/225000 [366:06:42<73:15:51,  8.33s/it]
 86%|████████▌ | 193337/225000 [366:06:53<79:15:35,  9.01s/it]
 86%|████████▌ | 193338/225000 [366:07:01<76:44:57,  8.73s/it]
 86%|████████▌ | 193339/225000 [366:07:07<70:37:56,  8.03s/it]
 86%|████████▌ | 193340/225000 [366:07:14<66:13:33,  7.53s/it]
 86%|████████▌ | 193341/225000 [366:07:19<61:37:40,  7.01s/it]
 86%|████████▌ | 193342/225000 [366:07:25<59:02:05,  6.71s/it]
 86%|████████▌ | 193343/225000 [366:07:40<79:52:23,  9.08s/it]
 86%|████████▌ | 193344/225000 [366:07:49<79:17:22,  9.02s/it]
 86%|████████▌ | 193345/225000 [366:07:55<71:44:52,  8.16s/it]
 86%|████████▌ | 193346/225000 [366:08:01<65:41:02,  7.47s/it]
 86%|████████▌ | 193347/225000 [366:08:14<80:21:30,  9.14s/it]
 86%|████████▌ | 193348/225000 [366:08:25<85:51:26,  9.77s/it]
 86%|████████▌ | 193349/225000 [366:08:38<94:36:02, 10.76s/it]
 86%|████████▌ | 193350/225000 [366:08:42<76:49:27,  8.74s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.213', 'learning_rate': '2.641e-06', 'epoch': '0.8593', 'num_input_tokens_seen': 610052128, 'train_runtime': '1.318e+06', 'train_tokens_per_second': '462.8'}

 86%|████████▌ | 193350/225000 [366:08:42<76:49:27,  8.74s/it]
 86%|████████▌ | 193351/225000 [366:08:53<83:22:50,  9.48s/it]
 86%|████████▌ | 193352/225000 [366:09:03<82:33:31,  9.39s/it]
 86%|████████▌ | 193353/225000 [366:09:12<82:59:01,  9.44s/it]
 86%|████████▌ | 193354/225000 [366:09:18<74:23:52,  8.46s/it]
 86%|████████▌ | 193355/225000 [366:09:29<79:00:23,  8.99s/it]
 86%|████████▌ | 193356/225000 [366:09:38<78:53:56,  8.98s/it]
 86%|████████▌ | 193357/225000 [366:09:45<74:47:43,  8.51s/it]
 86%|████████▌ | 193358/225000 [366:09:53<74:05:22,  8.43s/it]
 86%|████████▌ | 193359/225000 [366:10:00<70:44:23,  8.05s/it]
 86%|████████▌ | 193360/225000 [366:10:06<65:04:36,  7.40s/it]
 86%|████████▌ | 193361/225000 [366:10:16<72:30:48,  8.25s/it]
 86%|████████▌ | 193362/225000 [366:10:26<75:13:57,  8.56s/it]
 86%|████████▌ | 193363/225000 [366:10:33<72:24:51,  8.24s/it]
 86%|████████▌ | 193364/225000 [366:10:40<67:57:25,  7.73s/it]
 86%|████████▌ | 193365/225000 [366:10:47<66:05:22,  7.52s/it]
 86%|████████▌ | 193366/225000 [366:10:53<61:23:27,  6.99s/it]
 86%|████████▌ | 193367/225000 [366:11:01<64:44:07,  7.37s/it]
 86%|████████▌ | 193368/225000 [366:11:07<61:54:34,  7.05s/it]
 86%|████████▌ | 193369/225000 [366:11:21<80:55:22,  9.21s/it]
 86%|████████▌ | 193370/225000 [366:11:28<74:07:08,  8.44s/it]
 86%|████████▌ | 193371/225000 [366:11:33<64:23:05,  7.33s/it]
 86%|████████▌ | 193372/225000 [366:11:38<58:31:25,  6.66s/it]
 86%|████████▌ | 193373/225000 [366:11:53<80:36:27,  9.18s/it]
 86%|████████▌ | 193374/225000 [366:12:01<77:15:50,  8.79s/it]
 86%|████████▌ | 193375/225000 [366:12:07<70:22:14,  8.01s/it]
 86%|████████▌ | 193376/225000 [366:12:13<64:37:10,  7.36s/it]
 86%|████████▌ | 193377/225000 [366:12:24<75:14:43,  8.57s/it]
 86%|████████▌ | 193378/225000 [366:12:34<79:17:41,  9.03s/it]
 86%|████████▌ | 193379/225000 [366:12:41<72:54:39,  8.30s/it]
 86%|████████▌ | 193380/225000 [366:12:52<80:51:40,  9.21s/it]
 86%|████████▌ | 193381/225000 [366:12:59<73:22:40,  8.35s/it]
 86%|████████▌ | 193382/225000 [366:13:06<71:09:45,  8.10s/it]
 86%|████████▌ | 193383/225000 [366:13:16<75:34:10,  8.60s/it]
 86%|████████▌ | 193384/225000 [366:13:21<66:54:09,  7.62s/it]
 86%|████████▌ | 193385/225000 [366:13:30<70:52:56,  8.07s/it]
 86%|████████▌ | 193386/225000 [366:13:40<74:50:34,  8.52s/it]
 86%|████████▌ | 193387/225000 [366:13:44<62:28:05,  7.11s/it]
 86%|████████▌ | 193388/225000 [366:13:53<67:07:09,  7.64s/it]
 86%|████████▌ | 193389/225000 [366:14:01<68:52:24,  7.84s/it]
 86%|████████▌ | 193390/225000 [366:14:12<78:41:30,  8.96s/it]
 86%|████████▌ | 193391/225000 [366:14:18<68:39:21,  7.82s/it]
 86%|████████▌ | 193392/225000 [366:14:29<78:21:01,  8.92s/it]
 86%|████████▌ | 193393/225000 [366:14:36<73:42:43,  8.40s/it]
 86%|████████▌ | 193394/225000 [366:14:51<89:04:02, 10.14s/it]
 86%|████████▌ | 193395/225000 [366:15:04<98:01:47, 11.17s/it]
 86%|████████▌ | 193396/225000 [366:15:10<83:39:49,  9.53s/it]
 86%|████████▌ | 193397/225000 [366:15:15<71:35:27,  8.16s/it]
 86%|████████▌ | 193398/225000 [366:15:21<66:40:20,  7.60s/it]
 86%|████████▌ | 193399/225000 [366:15:35<84:25:22,  9.62s/it]
 86%|████████▌ | 193400/225000 [366:15:43<80:26:22,  9.16s/it]
                                                              
{'loss': '1.253', 'grad_norm': '1.203', 'learning_rate': '2.633e-06', 'epoch': '0.8596', 'num_input_tokens_seen': 610274352, 'train_runtime': '1.319e+06', 'train_tokens_per_second': '462.8'}

 86%|████████▌ | 193400/225000 [366:15:43<80:26:22,  9.16s/it]
 86%|████████▌ | 193401/225000 [366:15:55<85:26:06,  9.73s/it]
 86%|████████▌ | 193402/225000 [366:16:00<75:13:52,  8.57s/it]
 86%|████████▌ | 193403/225000 [366:16:07<69:28:29,  7.92s/it]
 86%|████████▌ | 193404/225000 [366:16:18<77:37:15,  8.84s/it]
 86%|████████▌ | 193405/225000 [366:16:28<81:58:53,  9.34s/it]
 86%|████████▌ | 193406/225000 [366:16:32<66:59:30,  7.63s/it]
 86%|████████▌ | 193407/225000 [366:16:43<76:11:43,  8.68s/it]
 86%|████████▌ | 193408/225000 [366:16:50<71:43:03,  8.17s/it]
 86%|████████▌ | 193409/225000 [366:17:00<77:14:07,  8.80s/it]
 86%|████████▌ | 193410/225000 [366:17:07<70:39:33,  8.05s/it]
 86%|████████▌ | 193411/225000 [366:17:19<82:04:10,  9.35s/it]
 86%|████████▌ | 193412/225000 [366:17:26<75:49:00,  8.64s/it]
 86%|████████▌ | 193413/225000 [366:17:33<72:20:41,  8.25s/it]
 86%|████████▌ | 193414/225000 [366:17:40<68:17:14,  7.78s/it]
 86%|████████▌ | 193415/225000 [366:17:47<65:40:00,  7.48s/it]
 86%|████████▌ | 193416/225000 [366:18:02<86:07:56,  9.82s/it]
 86%|████████▌ | 193417/225000 [366:18:06<71:12:59,  8.12s/it]
 86%|████████▌ | 193418/225000 [366:18:14<69:24:18,  7.91s/it]
 86%|████████▌ | 193419/225000 [366:18:21<68:01:38,  7.75s/it]
 86%|████████▌ | 193420/225000 [366:18:29<69:14:13,  7.89s/it]
 86%|████████▌ | 193421/225000 [366:18:36<65:12:12,  7.43s/it]
 86%|████████▌ | 193422/225000 [366:18:49<80:01:59,  9.12s/it]
 86%|████████▌ | 193423/225000 [366:18:55<72:06:26,  8.22s/it]
 86%|████████▌ | 193424/225000 [366:19:04<74:56:19,  8.54s/it]
 86%|████████▌ | 193425/225000 [366:19:12<73:44:35,  8.41s/it]
 86%|████████▌ | 193426/225000 [366:19:23<81:21:05,  9.28s/it]
 86%|████████▌ | 193427/225000 [366:19:30<72:56:26,  8.32s/it]
 86%|████████▌ | 193428/225000 [366:19:39<75:18:57,  8.59s/it]
 86%|████████▌ | 193429/225000 [366:19:45<68:34:02,  7.82s/it]
 86%|████████▌ | 193430/225000 [366:19:58<83:19:13,  9.50s/it]
 86%|████████▌ | 193431/225000 [366:20:02<68:55:04,  7.86s/it]
 86%|████████▌ | 193432/225000 [366:20:09<65:08:35,  7.43s/it]
 86%|████████▌ | 193433/225000 [366:20:18<70:09:55,  8.00s/it]
 86%|████████▌ | 193434/225000 [366:20:23<63:31:04,  7.24s/it]
 86%|████████▌ | 193435/225000 [366:20:34<72:51:48,  8.31s/it]
 86%|████████▌ | 193436/225000 [366:20:41<69:08:16,  7.89s/it]
 86%|████████▌ | 193437/225000 [366:20:47<62:30:31,  7.13s/it]
 86%|████████▌ | 193438/225000 [366:20:58<73:55:54,  8.43s/it]
 86%|████████▌ | 193439/225000 [366:21:03<63:34:30,  7.25s/it]
 86%|████████▌ | 193440/225000 [366:21:11<66:54:40,  7.63s/it]
 86%|████████▌ | 193441/225000 [366:21:23<77:09:19,  8.80s/it]
 86%|████████▌ | 193442/225000 [366:21:32<78:24:42,  8.94s/it]
 86%|████████▌ | 193443/225000 [366:21:38<71:35:13,  8.17s/it]
 86%|████████▌ | 193444/225000 [366:21:45<67:41:30,  7.72s/it]
 86%|████████▌ | 193445/225000 [366:21:53<69:40:23,  7.95s/it]
 86%|████████▌ | 193446/225000 [366:22:06<83:18:16,  9.50s/it]
 86%|████████▌ | 193447/225000 [366:22:19<90:03:11, 10.27s/it]
 86%|████████▌ | 193448/225000 [366:22:24<76:16:36,  8.70s/it]
 86%|████████▌ | 193449/225000 [366:22:33<77:45:56,  8.87s/it]
 86%|████████▌ | 193450/225000 [366:22:40<72:28:28,  8.27s/it]
                                                              
{'loss': '1.26', 'grad_norm': '1.189', 'learning_rate': '2.625e-06', 'epoch': '0.8598', 'num_input_tokens_seen': 610491344, 'train_runtime': '1.319e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193450/225000 [366:22:40<72:28:28,  8.27s/it]
 86%|████████▌ | 193451/225000 [366:22:50<78:39:12,  8.98s/it]
 86%|████████▌ | 193452/225000 [366:22:54<65:33:30,  7.48s/it]
 86%|████████▌ | 193453/225000 [366:23:04<72:12:26,  8.24s/it]
 86%|████████▌ | 193454/225000 [366:23:10<65:34:07,  7.48s/it]
 86%|████████▌ | 193455/225000 [366:23:18<67:06:30,  7.66s/it]
 86%|████████▌ | 193456/225000 [366:23:30<78:29:04,  8.96s/it]
 86%|████████▌ | 193457/225000 [366:23:34<66:16:44,  7.56s/it]
 86%|████████▌ | 193458/225000 [366:23:47<79:26:41,  9.07s/it]
 86%|████████▌ | 193459/225000 [366:23:55<76:51:58,  8.77s/it]
 86%|████████▌ | 193460/225000 [366:24:04<76:52:12,  8.77s/it]
 86%|████████▌ | 193461/225000 [366:24:12<74:14:28,  8.47s/it]
 86%|████████▌ | 193462/225000 [366:24:26<88:44:43, 10.13s/it]
 86%|████████▌ | 193463/225000 [366:24:32<78:23:25,  8.95s/it]
 86%|████████▌ | 193464/225000 [366:24:40<76:26:59,  8.73s/it]
 86%|████████▌ | 193465/225000 [366:24:52<85:09:52,  9.72s/it]
 86%|████████▌ | 193466/225000 [366:25:00<80:37:16,  9.20s/it]
 86%|████████▌ | 193467/225000 [366:25:08<76:29:23,  8.73s/it]
 86%|████████▌ | 193468/225000 [366:25:18<79:49:40,  9.11s/it]
 86%|████████▌ | 193469/225000 [366:25:23<70:21:01,  8.03s/it]
 86%|████████▌ | 193470/225000 [366:25:31<68:26:58,  7.82s/it]
 86%|████████▌ | 193471/225000 [366:25:40<73:21:35,  8.38s/it]
 86%|████████▌ | 193472/225000 [366:25:50<75:43:12,  8.65s/it]
 86%|████████▌ | 193473/225000 [366:25:56<70:29:59,  8.05s/it]
 86%|████████▌ | 193474/225000 [366:26:02<65:47:34,  7.51s/it]
 86%|████████▌ | 193475/225000 [366:26:16<80:59:41,  9.25s/it]
 86%|████████▌ | 193476/225000 [366:26:19<65:49:30,  7.52s/it]
 86%|████████▌ | 193477/225000 [366:26:27<67:14:15,  7.68s/it]
 86%|████████▌ | 193478/225000 [366:26:40<80:56:37,  9.24s/it]
 86%|████████▌ | 193479/225000 [366:26:52<86:38:03,  9.89s/it]
 86%|████████▌ | 193480/225000 [366:27:00<83:48:31,  9.57s/it]
 86%|████████▌ | 193481/225000 [366:27:09<81:15:19,  9.28s/it]
 86%|████████▌ | 193482/225000 [366:27:14<69:36:28,  7.95s/it]
 86%|████████▌ | 193483/225000 [366:27:22<70:21:50,  8.04s/it]
 86%|████████▌ | 193484/225000 [366:27:32<74:19:03,  8.49s/it]
 86%|████████▌ | 193485/225000 [366:27:38<68:09:54,  7.79s/it]
 86%|████████▌ | 193486/225000 [366:27:50<79:25:30,  9.07s/it]
 86%|████████▌ | 193487/225000 [366:27:58<77:17:01,  8.83s/it]
 86%|████████▌ | 193488/225000 [366:28:04<68:38:47,  7.84s/it]
 86%|████████▌ | 193489/225000 [366:28:16<79:24:38,  9.07s/it]
 86%|████████▌ | 193490/225000 [366:28:22<72:54:38,  8.33s/it]
 86%|████████▌ | 193491/225000 [366:28:28<67:13:13,  7.68s/it]
 86%|████████▌ | 193492/225000 [366:28:39<75:17:19,  8.60s/it]
 86%|████████▌ | 193493/225000 [366:28:52<87:30:23, 10.00s/it]
 86%|████████▌ | 193494/225000 [366:28:58<76:13:00,  8.71s/it]
 86%|████████▌ | 193495/225000 [366:29:09<81:19:23,  9.29s/it]
 86%|████████▌ | 193496/225000 [366:29:17<78:16:59,  8.95s/it]
 86%|████████▌ | 193497/225000 [366:29:25<76:58:36,  8.80s/it]
 86%|████████▌ | 193498/225000 [366:29:33<74:13:54,  8.48s/it]
 86%|████████▌ | 193499/225000 [366:29:40<70:14:47,  8.03s/it]
 86%|████████▌ | 193500/225000 [366:29:51<76:53:49,  8.79s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.306', 'learning_rate': '2.617e-06', 'epoch': '0.86', 'num_input_tokens_seen': 610709904, 'train_runtime': '1.319e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193500/225000 [366:29:51<76:53:49,  8.79s/it]
 86%|████████▌ | 193501/225000 [366:29:56<69:18:23,  7.92s/it]
 86%|████████▌ | 193502/225000 [366:30:07<77:25:15,  8.85s/it]
 86%|████████▌ | 193503/225000 [366:30:16<77:40:39,  8.88s/it]
 86%|████████▌ | 193504/225000 [366:30:27<80:54:36,  9.25s/it]
 86%|████████▌ | 193505/225000 [366:30:31<69:37:49,  7.96s/it]
 86%|████████▌ | 193506/225000 [366:30:38<65:43:20,  7.51s/it]
 86%|████████▌ | 193507/225000 [366:30:45<64:46:32,  7.40s/it]
 86%|████████▌ | 193508/225000 [366:30:52<62:43:53,  7.17s/it]
 86%|████████▌ | 193509/225000 [366:30:57<58:45:17,  6.72s/it]
 86%|████████▌ | 193510/225000 [366:31:12<79:55:12,  9.14s/it]
 86%|████████▌ | 193511/225000 [366:31:19<75:00:34,  8.58s/it]
 86%|████████▌ | 193512/225000 [366:31:26<69:12:13,  7.91s/it]
 86%|████████▌ | 193513/225000 [366:31:32<65:04:11,  7.44s/it]
 86%|████████▌ | 193514/225000 [366:31:40<66:22:35,  7.59s/it]
 86%|████████▌ | 193515/225000 [366:31:45<60:20:03,  6.90s/it]
 86%|████████▌ | 193516/225000 [366:31:52<58:28:52,  6.69s/it]
 86%|████████▌ | 193517/225000 [366:32:07<82:16:01,  9.41s/it]
 86%|████████▌ | 193518/225000 [366:32:13<71:24:10,  8.16s/it]
 86%|████████▌ | 193519/225000 [366:32:23<77:29:55,  8.86s/it]
 86%|████████▌ | 193520/225000 [366:32:27<65:31:28,  7.49s/it]
 86%|████████▌ | 193521/225000 [366:32:38<74:43:20,  8.55s/it]
 86%|████████▌ | 193522/225000 [366:32:46<72:48:57,  8.33s/it]
 86%|████████▌ | 193523/225000 [366:32:55<74:56:58,  8.57s/it]
 86%|████████▌ | 193524/225000 [366:33:01<67:34:17,  7.73s/it]
 86%|████████▌ | 193525/225000 [366:33:13<78:47:40,  9.01s/it]
 86%|████████▌ | 193526/225000 [366:33:22<77:25:53,  8.86s/it]
 86%|████████▌ | 193527/225000 [366:33:27<68:29:47,  7.83s/it]
 86%|████████▌ | 193528/225000 [366:33:37<73:05:22,  8.36s/it]
 86%|████████▌ | 193529/225000 [366:33:44<69:58:41,  8.00s/it]
 86%|████████▌ | 193530/225000 [366:33:49<61:57:09,  7.09s/it]
 86%|████████▌ | 193531/225000 [366:33:55<59:10:29,  6.77s/it]
 86%|████████▌ | 193532/225000 [366:34:08<77:14:42,  8.84s/it]
 86%|████████▌ | 193533/225000 [366:34:16<73:36:13,  8.42s/it]
 86%|████████▌ | 193534/225000 [366:34:24<73:33:54,  8.42s/it]
 86%|████████▌ | 193535/225000 [366:34:29<63:47:09,  7.30s/it]
 86%|████████▌ | 193536/225000 [366:34:44<82:42:04,  9.46s/it]
 86%|████████▌ | 193537/225000 [366:34:50<75:48:40,  8.67s/it]
 86%|████████▌ | 193538/225000 [366:35:00<77:52:13,  8.91s/it]
 86%|████████▌ | 193539/225000 [366:35:04<64:45:18,  7.41s/it]
 86%|████████▌ | 193540/225000 [366:35:12<68:08:06,  7.80s/it]
 86%|████████▌ | 193541/225000 [366:35:20<66:26:02,  7.60s/it]
 86%|████████▌ | 193542/225000 [366:35:28<69:33:34,  7.96s/it]
 86%|████████▌ | 193543/225000 [366:35:39<76:51:14,  8.80s/it]
 86%|████████▌ | 193544/225000 [366:35:43<65:02:08,  7.44s/it]
 86%|████████▌ | 193545/225000 [366:35:52<67:10:25,  7.69s/it]
 86%|████████▌ | 193546/225000 [366:36:03<75:30:45,  8.64s/it]
 86%|████████▌ | 193547/225000 [366:36:13<79:18:57,  9.08s/it]
 86%|████████▌ | 193548/225000 [366:36:22<79:49:39,  9.14s/it]
 86%|████████▌ | 193549/225000 [366:36:32<82:34:42,  9.45s/it]
 86%|████████▌ | 193550/225000 [366:36:40<78:33:24,  8.99s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.128', 'learning_rate': '2.608e-06', 'epoch': '0.8602', 'num_input_tokens_seen': 610925208, 'train_runtime': '1.32e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193550/225000 [366:36:40<78:33:24,  8.99s/it]
 86%|████████▌ | 193551/225000 [366:36:50<80:15:15,  9.19s/it]
 86%|████████▌ | 193552/225000 [366:36:57<74:21:42,  8.51s/it]
 86%|████████▌ | 193553/225000 [366:37:02<66:26:38,  7.61s/it]
 86%|████████▌ | 193554/225000 [366:37:07<58:41:38,  6.72s/it]
 86%|████████▌ | 193555/225000 [366:37:20<76:01:43,  8.70s/it]
 86%|████████▌ | 193556/225000 [366:37:27<72:18:12,  8.28s/it]
 86%|████████▌ | 193557/225000 [366:37:33<66:23:56,  7.60s/it]
 86%|████████▌ | 193558/225000 [366:37:40<63:13:06,  7.24s/it]
 86%|████████▌ | 193559/225000 [366:37:51<74:27:59,  8.53s/it]
 86%|████████▌ | 193560/225000 [366:38:00<75:01:11,  8.59s/it]
 86%|████████▌ | 193561/225000 [366:38:11<80:26:54,  9.21s/it]
 86%|████████▌ | 193562/225000 [366:38:19<78:23:37,  8.98s/it]
 86%|████████▌ | 193563/225000 [366:38:24<68:44:47,  7.87s/it]
 86%|████████▌ | 193564/225000 [366:38:30<62:34:43,  7.17s/it]
 86%|████████▌ | 193565/225000 [366:38:44<79:46:52,  9.14s/it]
 86%|████████▌ | 193566/225000 [366:38:53<79:59:53,  9.16s/it]
 86%|████████▌ | 193567/225000 [366:39:04<84:30:42,  9.68s/it]
 86%|████████▌ | 193568/225000 [366:39:11<78:29:17,  8.99s/it]
 86%|████████▌ | 193569/225000 [366:39:23<85:57:12,  9.84s/it]
 86%|████████▌ | 193570/225000 [366:39:30<78:30:38,  8.99s/it]
 86%|████████▌ | 193571/225000 [366:39:37<74:09:27,  8.49s/it]
 86%|████████▌ | 193572/225000 [366:39:43<68:02:18,  7.79s/it]
 86%|████████▌ | 193573/225000 [366:39:52<70:14:07,  8.05s/it]
 86%|████████▌ | 193574/225000 [366:39:57<61:38:57,  7.06s/it]
 86%|████████▌ | 193575/225000 [366:40:07<68:34:39,  7.86s/it]
 86%|████████▌ | 193576/225000 [366:40:16<72:40:19,  8.33s/it]
 86%|████████▌ | 193577/225000 [366:40:26<76:25:38,  8.76s/it]
 86%|████████▌ | 193578/225000 [366:40:33<73:02:07,  8.37s/it]
 86%|████████▌ | 193579/225000 [366:40:47<87:41:48, 10.05s/it]
 86%|████████▌ | 193580/225000 [366:40:55<83:04:32,  9.52s/it]
 86%|████████▌ | 193581/225000 [366:41:01<71:44:56,  8.22s/it]
 86%|████████▌ | 193582/225000 [366:41:08<68:55:26,  7.90s/it]
 86%|████████▌ | 193583/225000 [366:41:13<62:20:46,  7.14s/it]
 86%|████████▌ | 193584/225000 [366:41:25<73:23:41,  8.41s/it]
 86%|████████▌ | 193585/225000 [366:41:32<71:29:30,  8.19s/it]
 86%|████████▌ | 193586/225000 [366:41:40<70:24:38,  8.07s/it]
 86%|████████▌ | 193587/225000 [366:41:46<65:21:05,  7.49s/it]
 86%|████████▌ | 193588/225000 [366:41:58<75:38:45,  8.67s/it]
 86%|████████▌ | 193589/225000 [366:42:05<71:11:25,  8.16s/it]
 86%|████████▌ | 193590/225000 [366:42:11<66:36:12,  7.63s/it]
 86%|████████▌ | 193591/225000 [366:42:18<64:01:10,  7.34s/it]
 86%|████████▌ | 193592/225000 [366:42:32<81:42:25,  9.37s/it]
 86%|████████▌ | 193593/225000 [366:42:40<78:46:23,  9.03s/it]
 86%|████████▌ | 193594/225000 [366:42:45<68:50:44,  7.89s/it]
 86%|████████▌ | 193595/225000 [366:42:55<74:19:20,  8.52s/it]
 86%|████████▌ | 193596/225000 [366:43:02<69:49:30,  8.00s/it]
 86%|████████▌ | 193597/225000 [366:43:14<79:41:11,  9.14s/it]
 86%|████████▌ | 193598/225000 [366:43:23<80:19:59,  9.21s/it]
 86%|████████▌ | 193599/225000 [366:43:28<69:20:58,  7.95s/it]
 86%|████████▌ | 193600/225000 [366:43:39<76:57:04,  8.82s/it]
                                                              
{'loss': '1.246', 'grad_norm': '1.469', 'learning_rate': '2.6e-06', 'epoch': '0.8604', 'num_input_tokens_seen': 611141856, 'train_runtime': '1.32e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193600/225000 [366:43:39<76:57:04,  8.82s/it]
 86%|████████▌ | 193601/225000 [366:43:47<75:20:27,  8.64s/it]
 86%|████████▌ | 193602/225000 [366:43:59<84:35:38,  9.70s/it]
 86%|████████▌ | 193603/225000 [366:44:10<86:01:00,  9.86s/it]
 86%|████████▌ | 193604/225000 [366:44:18<82:03:42,  9.41s/it]
 86%|████████▌ | 193605/225000 [366:44:26<77:31:41,  8.89s/it]
 86%|████████▌ | 193606/225000 [366:44:33<73:17:56,  8.41s/it]
 86%|████████▌ | 193607/225000 [366:44:44<79:11:47,  9.08s/it]
 86%|████████▌ | 193608/225000 [366:44:50<71:01:43,  8.15s/it]
 86%|████████▌ | 193609/225000 [366:44:56<66:19:02,  7.61s/it]
 86%|████████▌ | 193610/225000 [366:45:08<77:34:42,  8.90s/it]
 86%|████████▌ | 193611/225000 [366:45:13<67:00:15,  7.68s/it]
 86%|████████▌ | 193612/225000 [366:45:21<67:54:40,  7.79s/it]
 86%|████████▌ | 193613/225000 [366:45:27<65:09:38,  7.47s/it]
 86%|████████▌ | 193614/225000 [366:45:39<77:06:28,  8.84s/it]
 86%|████████▌ | 193615/225000 [366:45:44<64:38:28,  7.41s/it]
 86%|████████▌ | 193616/225000 [366:45:53<68:45:46,  7.89s/it]
 86%|████████▌ | 193617/225000 [366:45:59<65:43:46,  7.54s/it]
 86%|████████▌ | 193618/225000 [366:46:06<64:15:24,  7.37s/it]
 86%|████████▌ | 193619/225000 [366:46:20<80:15:05,  9.21s/it]
 86%|████████▌ | 193620/225000 [366:46:30<82:19:54,  9.45s/it]
 86%|████████▌ | 193621/225000 [366:46:36<72:38:33,  8.33s/it]
 86%|████████▌ | 193622/225000 [366:46:45<74:50:07,  8.59s/it]
 86%|████████▌ | 193623/225000 [366:46:51<68:17:46,  7.84s/it]
 86%|████████▌ | 193624/225000 [366:47:03<80:17:36,  9.21s/it]
 86%|████████▌ | 193625/225000 [366:47:13<82:21:47,  9.45s/it]
 86%|████████▌ | 193626/225000 [366:47:19<72:46:59,  8.35s/it]
 86%|████████▌ | 193627/225000 [366:47:25<66:08:14,  7.59s/it]
 86%|████████▌ | 193628/225000 [366:47:37<77:10:56,  8.86s/it]
 86%|████████▌ | 193629/225000 [366:47:41<66:12:14,  7.60s/it]
 86%|████████▌ | 193630/225000 [366:47:55<82:15:04,  9.44s/it]
 86%|████████▌ | 193631/225000 [366:48:02<75:39:02,  8.68s/it]
 86%|████████▌ | 193632/225000 [366:48:08<68:50:22,  7.90s/it]
 86%|████████▌ | 193633/225000 [366:48:19<76:52:16,  8.82s/it]
 86%|████████▌ | 193634/225000 [366:48:25<68:19:15,  7.84s/it]
 86%|████████▌ | 193635/225000 [366:48:32<67:53:42,  7.79s/it]
 86%|████████▌ | 193636/225000 [366:48:43<74:40:43,  8.57s/it]
 86%|████████▌ | 193637/225000 [366:48:53<80:21:11,  9.22s/it]
 86%|████████▌ | 193638/225000 [366:48:59<70:39:04,  8.11s/it]
 86%|████████▌ | 193639/225000 [366:49:06<66:59:13,  7.69s/it]
 86%|████████▌ | 193640/225000 [366:49:12<64:42:08,  7.43s/it]
 86%|████████▌ | 193641/225000 [366:49:22<70:35:12,  8.10s/it]
 86%|████████▌ | 193642/225000 [366:49:29<68:18:49,  7.84s/it]
 86%|████████▌ | 193643/225000 [366:49:35<62:08:14,  7.13s/it]
 86%|████████▌ | 193644/225000 [366:49:45<71:21:25,  8.19s/it]
 86%|████████▌ | 193645/225000 [366:49:50<60:53:47,  6.99s/it]
 86%|████████▌ | 193646/225000 [366:49:59<68:16:43,  7.84s/it]
 86%|████████▌ | 193647/225000 [366:50:06<63:54:07,  7.34s/it]
 86%|████████▌ | 193648/225000 [366:50:17<73:59:17,  8.50s/it]
 86%|████████▌ | 193649/225000 [366:50:22<64:42:05,  7.43s/it]
 86%|████████▌ | 193650/225000 [366:50:32<71:09:15,  8.17s/it]
                                                              
{'loss': '1.27', 'grad_norm': '1.197', 'learning_rate': '2.592e-06', 'epoch': '0.8607', 'num_input_tokens_seen': 611359880, 'train_runtime': '1.321e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193650/225000 [366:50:32<71:09:15,  8.17s/it]
 86%|████████▌ | 193651/225000 [366:50:41<75:00:07,  8.61s/it]
 86%|████████▌ | 193652/225000 [366:50:47<68:41:03,  7.89s/it]
 86%|████████▌ | 193653/225000 [366:50:54<65:34:08,  7.53s/it]
 86%|████████▌ | 193654/225000 [366:51:05<74:53:44,  8.60s/it]
 86%|████████▌ | 193655/225000 [366:51:12<68:52:02,  7.91s/it]
 86%|████████▌ | 193656/225000 [366:51:25<83:32:48,  9.60s/it]
 86%|████████▌ | 193657/225000 [366:51:33<80:03:08,  9.19s/it]
 86%|████████▌ | 193658/225000 [366:51:39<71:09:52,  8.17s/it]
 86%|████████▌ | 193659/225000 [366:51:46<67:21:46,  7.74s/it]
 86%|████████▌ | 193660/225000 [366:51:51<59:44:04,  6.86s/it]
 86%|████████▌ | 193661/225000 [366:51:58<59:36:22,  6.85s/it]
 86%|████████▌ | 193662/225000 [366:52:10<74:27:50,  8.55s/it]
 86%|████████▌ | 193663/225000 [366:52:21<81:17:50,  9.34s/it]
 86%|████████▌ | 193664/225000 [366:52:29<76:55:12,  8.84s/it]
 86%|████████▌ | 193665/225000 [366:52:38<78:23:25,  9.01s/it]
 86%|████████▌ | 193666/225000 [366:52:49<83:18:44,  9.57s/it]
 86%|████████▌ | 193667/225000 [366:52:58<81:01:39,  9.31s/it]
 86%|████████▌ | 193668/225000 [366:53:05<75:21:29,  8.66s/it]
 86%|████████▌ | 193669/225000 [366:53:09<63:52:54,  7.34s/it]
 86%|████████▌ | 193670/225000 [366:53:19<69:02:30,  7.93s/it]
 86%|████████▌ | 193671/225000 [366:53:25<65:07:04,  7.48s/it]
 86%|████████▌ | 193672/225000 [366:53:32<63:09:12,  7.26s/it]
 86%|████████▌ | 193673/225000 [366:53:44<76:07:49,  8.75s/it]
 86%|████████▌ | 193674/225000 [366:53:49<65:16:50,  7.50s/it]
 86%|████████▌ | 193675/225000 [366:53:56<65:00:34,  7.47s/it]
 86%|████████▌ | 193676/225000 [366:54:07<74:50:58,  8.60s/it]
 86%|████████▌ | 193677/225000 [366:54:14<69:55:07,  8.04s/it]
 86%|████████▌ | 193678/225000 [366:54:26<80:22:17,  9.24s/it]
 86%|████████▌ | 193679/225000 [366:54:32<72:46:56,  8.37s/it]
 86%|████████▌ | 193680/225000 [366:54:43<77:45:06,  8.94s/it]
 86%|████████▌ | 193681/225000 [366:54:48<69:21:49,  7.97s/it]
 86%|████████▌ | 193682/225000 [366:55:02<85:07:10,  9.78s/it]
 86%|████████▌ | 193683/225000 [366:55:09<76:10:18,  8.76s/it]
 86%|████████▌ | 193684/225000 [366:55:23<90:44:58, 10.43s/it]
 86%|████████▌ | 193685/225000 [366:55:29<78:44:44,  9.05s/it]
 86%|████████▌ | 193686/225000 [366:55:40<82:59:47,  9.54s/it]
 86%|████████▌ | 193687/225000 [366:55:46<74:00:02,  8.51s/it]
 86%|████████▌ | 193688/225000 [366:55:56<77:38:44,  8.93s/it]
 86%|████████▌ | 193689/225000 [366:56:03<74:18:25,  8.54s/it]
 86%|████████▌ | 193690/225000 [366:56:12<74:13:25,  8.53s/it]
 86%|████████▌ | 193691/225000 [366:56:19<70:32:09,  8.11s/it]
 86%|████████▌ | 193692/225000 [366:56:28<72:42:50,  8.36s/it]
 86%|████████▌ | 193693/225000 [366:56:36<71:18:15,  8.20s/it]
 86%|████████▌ | 193694/225000 [366:56:44<71:20:55,  8.20s/it]
 86%|████████▌ | 193695/225000 [366:56:50<67:21:18,  7.75s/it]
 86%|████████▌ | 193696/225000 [366:56:57<64:13:06,  7.39s/it]
 86%|████████▌ | 193697/225000 [366:57:04<61:55:00,  7.12s/it]
 86%|████████▌ | 193698/225000 [366:57:09<57:21:49,  6.60s/it]
 86%|████████▌ | 193699/225000 [366:57:15<55:18:02,  6.36s/it]
 86%|████████▌ | 193700/225000 [366:57:27<71:50:51,  8.26s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.16', 'learning_rate': '2.584e-06', 'epoch': '0.8609', 'num_input_tokens_seen': 611578120, 'train_runtime': '1.321e+06', 'train_tokens_per_second': '462.9'}

 86%|████████▌ | 193700/225000 [366:57:27<71:50:51,  8.26s/it]
 86%|████████▌ | 193701/225000 [366:57:37<74:07:52,  8.53s/it]
 86%|████████▌ | 193702/225000 [366:57:42<66:49:14,  7.69s/it]
 86%|████████▌ | 193703/225000 [366:57:48<62:14:21,  7.16s/it]
 86%|████████▌ | 193704/225000 [366:57:56<62:57:43,  7.24s/it]
 86%|████████▌ | 193705/225000 [366:58:08<76:29:08,  8.80s/it]
 86%|████████▌ | 193706/225000 [366:58:16<73:11:40,  8.42s/it]
 86%|████████▌ | 193707/225000 [366:58:22<67:25:16,  7.76s/it]
 86%|████████▌ | 193708/225000 [366:58:29<64:54:05,  7.47s/it]
 86%|████████▌ | 193709/225000 [366:58:36<65:16:14,  7.51s/it]
 86%|████████▌ | 193710/225000 [366:58:50<80:33:03,  9.27s/it]
 86%|████████▌ | 193711/225000 [366:58:57<74:40:14,  8.59s/it]
 86%|████████▌ | 193712/225000 [366:59:11<90:31:33, 10.42s/it]
 86%|████████▌ | 193713/225000 [366:59:16<76:15:18,  8.77s/it]
 86%|████████▌ | 193714/225000 [366:59:28<84:35:29,  9.73s/it]
 86%|████████▌ | 193715/225000 [366:59:36<79:59:09,  9.20s/it]
 86%|████████▌ | 193716/225000 [366:59:48<85:44:40,  9.87s/it]
 86%|████████▌ | 193717/225000 [366:59:53<74:15:58,  8.55s/it]
 86%|████████▌ | 193718/225000 [366:59:59<68:17:32,  7.86s/it]
 86%|████████▌ | 193719/225000 [367:00:07<68:41:18,  7.91s/it]
 86%|████████▌ | 193720/225000 [367:00:15<67:48:09,  7.80s/it]
 86%|████████▌ | 193721/225000 [367:00:27<79:14:26,  9.12s/it]
 86%|████████▌ | 193722/225000 [367:00:32<69:14:47,  7.97s/it]
 86%|████████▌ | 193723/225000 [367:00:45<82:46:24,  9.53s/it]
 86%|████████▌ | 193724/225000 [367:00:51<71:36:20,  8.24s/it]
 86%|████████▌ | 193725/225000 [367:00:56<62:58:16,  7.25s/it]
 86%|████████▌ | 193726/225000 [367:01:02<60:40:18,  6.98s/it]
 86%|████████▌ | 193727/225000 [367:01:09<60:51:33,  7.01s/it]
 86%|████████▌ | 193728/225000 [367:01:14<56:03:14,  6.45s/it]
 86%|████████▌ | 193729/225000 [367:01:29<77:49:48,  8.96s/it]
 86%|████████▌ | 193730/225000 [367:01:39<79:40:25,  9.17s/it]
 86%|████████▌ | 193731/225000 [367:01:46<73:40:18,  8.48s/it]
 86%|████████▌ | 193732/225000 [367:01:55<75:31:35,  8.70s/it]
 86%|████████▌ | 193733/225000 [367:02:01<69:21:11,  7.99s/it]
 86%|████████▌ | 193734/225000 [367:02:07<63:49:42,  7.35s/it]
 86%|████████▌ | 193735/225000 [367:02:19<75:11:00,  8.66s/it]
 86%|████████▌ | 193736/225000 [367:02:30<82:21:24,  9.48s/it]
 86%|████████▌ | 193737/225000 [367:02:36<73:38:51,  8.48s/it]
 86%|████████▌ | 193738/225000 [367:02:41<64:17:24,  7.40s/it]
 86%|████████▌ | 193739/225000 [367:02:55<81:43:34,  9.41s/it]
 86%|████████▌ | 193740/225000 [367:03:02<74:25:09,  8.57s/it]
 86%|████████▌ | 193741/225000 [367:03:09<69:32:36,  8.01s/it]
 86%|████████▌ | 193742/225000 [367:03:17<71:03:37,  8.18s/it]
 86%|████████▌ | 193743/225000 [367:03:25<69:25:41,  8.00s/it]
 86%|████████▌ | 193744/225000 [367:03:33<70:35:42,  8.13s/it]
 86%|████████▌ | 193745/225000 [367:03:39<65:41:58,  7.57s/it]
 86%|████████▌ | 193746/225000 [367:03:48<69:09:01,  7.97s/it]
 86%|████████▌ | 193747/225000 [367:03:55<65:58:15,  7.60s/it]
 86%|████████▌ | 193748/225000 [367:04:05<70:54:56,  8.17s/it]
 86%|████████▌ | 193749/225000 [367:04:13<70:42:12,  8.14s/it]
 86%|████████▌ | 193750/225000 [367:04:23<76:30:43,  8.81s/it]
                                                              
{'loss': '1.277', 'grad_norm': '1.22', 'learning_rate': '2.576e-06', 'epoch': '0.8611', 'num_input_tokens_seen': 611796216, 'train_runtime': '1.321e+06', 'train_tokens_per_second': '463'}

 86%|████████▌ | 193750/225000 [367:04:23<76:30:43,  8.81s/it]
 86%|████████▌ | 193751/225000 [367:04:32<76:41:56,  8.84s/it]
 86%|████████▌ | 193752/225000 [367:04:40<74:15:44,  8.56s/it]
 86%|████████▌ | 193753/225000 [367:04:47<71:27:49,  8.23s/it]
 86%|████████▌ | 193754/225000 [367:04:53<65:48:44,  7.58s/it]
 86%|████████▌ | 193755/225000 [367:05:05<77:17:55,  8.91s/it]
 86%|████████▌ | 193756/225000 [367:05:15<79:29:26,  9.16s/it]
 86%|████████▌ | 193757/225000 [367:05:20<67:36:46,  7.79s/it]
 86%|████████▌ | 193758/225000 [367:05:29<72:50:32,  8.39s/it]
 86%|████████▌ | 193759/225000 [367:05:36<67:57:33,  7.83s/it]
 86%|████████▌ | 193760/225000 [367:05:44<67:35:12,  7.79s/it]
 86%|████████▌ | 193761/225000 [367:05:52<69:10:53,  7.97s/it]
 86%|████████▌ | 193762/225000 [367:06:00<68:36:27,  7.91s/it]
 86%|████████▌ | 193763/225000 [367:06:07<66:53:32,  7.71s/it]
 86%|████████▌ | 193764/225000 [367:06:17<72:06:31,  8.31s/it]
 86%|████████▌ | 193765/225000 [367:06:26<75:05:12,  8.65s/it]
 86%|████████▌ | 193766/225000 [367:06:31<65:54:24,  7.60s/it]
 86%|████████▌ | 193767/225000 [367:06:43<76:29:31,  8.82s/it]
 86%|████████▌ | 193768/225000 [367:06:50<71:51:30,  8.28s/it]
 86%|████████▌ | 193769/225000 [367:06:58<69:58:37,  8.07s/it]
 86%|████████▌ | 193770/225000 [367:07:02<61:28:18,  7.09s/it]
 86%|████████▌ | 193771/225000 [367:07:10<63:44:40,  7.35s/it]
 86%|████████▌ | 193772/225000 [367:07:17<62:22:41,  7.19s/it]
 86%|████████▌ | 193773/225000 [367:07:28<71:14:35,  8.21s/it]
 86%|████████▌ | 193774/225000 [367:07:38<77:20:05,  8.92s/it]
 86%|████████▌ | 193775/225000 [367:07:49<80:47:51,  9.32s/it]
 86%|████████▌ | 193776/225000 [367:07:54<69:25:01,  8.00s/it]
 86%|████████▌ | 193777/225000 [367:08:03<71:55:04,  8.29s/it]
 86%|████████▌ | 193778/225000 [367:08:13<77:04:52,  8.89s/it]
 86%|████████▌ | 193779/225000 [367:08:20<73:01:27,  8.42s/it]
 86%|████████▌ | 193780/225000 [367:08:25<64:36:09,  7.45s/it]
 86%|████████▌ | 193781/225000 [367:08:37<75:26:00,  8.70s/it]
 86%|████████▌ | 193782/225000 [367:08:45<72:50:52,  8.40s/it]
 86%|████████▌ | 193783/225000 [367:08:55<77:09:48,  8.90s/it]
 86%|████████▌ | 193784/225000 [367:09:04<77:45:38,  8.97s/it]
 86%|████████▌ | 193785/225000 [367:09:13<77:23:00,  8.92s/it]
 86%|████████▌ | 193786/225000 [367:09:22<78:21:20,  9.04s/it]
 86%|████████▌ | 193787/225000 [367:09:29<72:55:20,  8.41s/it]
 86%|████████▌ | 193788/225000 [367:09:35<65:42:01,  7.58s/it]
 86%|████████▌ | 193789/225000 [367:09:48<79:54:29,  9.22s/it]
 86%|████████▌ | 193790/225000 [367:09:54<73:13:46,  8.45s/it]
 86%|████████▌ | 193791/225000 [367:10:06<81:11:37,  9.37s/it]
 86%|████████▌ | 193792/225000 [367:10:10<68:17:31,  7.88s/it]
 86%|████████▌ | 193793/225000 [367:10:17<66:35:48,  7.68s/it]
 86%|████████▌ | 193794/225000 [367:10:30<79:54:50,  9.22s/it]
 86%|████████▌ | 193795/225000 [367:10:37<74:35:38,  8.61s/it]
 86%|████████▌ | 193796/225000 [367:10:45<73:18:05,  8.46s/it]
 86%|████████▌ | 193797/225000 [367:10:51<66:16:25,  7.65s/it]
 86%|████████▌ | 193798/225000 [367:11:01<71:50:26,  8.29s/it]
 86%|████████▌ | 193799/225000 [367:11:07<65:32:40,  7.56s/it]
 86%|████████▌ | 193800/225000 [367:11:12<58:43:45,  6.78s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.674', 'learning_rate': '2.568e-06', 'epoch': '0.8613', 'num_input_tokens_seen': 612013560, 'train_runtime': '1.322e+06', 'train_tokens_per_second': '463'}

 86%|████████▌ | 193800/225000 [367:11:12<58:43:45,  6.78s/it]
 86%|████████▌ | 193801/225000 [367:11:22<66:45:52,  7.70s/it]
 86%|████████▌ | 193802/225000 [367:11:30<67:07:49,  7.75s/it]
 86%|████████▌ | 193803/225000 [367:11:41<76:51:27,  8.87s/it]
 86%|████████▌ | 193804/225000 [367:11:47<69:53:21,  8.07s/it]
 86%|████████▌ | 193805/225000 [367:11:57<75:07:59,  8.67s/it]
 86%|████████▌ | 193806/225000 [367:12:03<67:15:27,  7.76s/it]
 86%|████████▌ | 193807/225000 [367:12:17<84:44:01,  9.78s/it]
 86%|████████▌ | 193808/225000 [367:12:27<83:21:19,  9.62s/it]
 86%|████████▌ | 193809/225000 [367:12:38<88:30:03, 10.21s/it]
 86%|████████▌ | 193810/225000 [367:12:45<80:29:56,  9.29s/it]
 86%|████████▌ | 193811/225000 [367:12:54<79:13:10,  9.14s/it]
 86%|████████▌ | 193812/225000 [367:13:02<76:06:45,  8.79s/it]
 86%|████████▌ | 193813/225000 [367:13:09<71:02:31,  8.20s/it]
 86%|████████▌ | 193814/225000 [367:13:20<77:32:33,  8.95s/it]
 86%|████████▌ | 193815/225000 [367:13:27<73:59:31,  8.54s/it]
 86%|████████▌ | 193816/225000 [367:13:33<65:50:26,  7.60s/it]
 86%|████████▌ | 193817/225000 [367:13:41<68:10:46,  7.87s/it]
 86%|████████▌ | 193818/225000 [367:13:54<82:02:43,  9.47s/it]
 86%|████████▌ | 193819/225000 [367:14:05<84:40:36,  9.78s/it]
 86%|████████▌ | 193820/225000 [367:14:12<78:58:46,  9.12s/it]
 86%|████████▌ | 193821/225000 [367:14:20<73:36:25,  8.50s/it]
 86%|████████▌ | 193822/225000 [367:14:27<71:28:29,  8.25s/it]
 86%|████████▌ | 193823/225000 [367:14:38<77:38:36,  8.97s/it]
 86%|████████▌ | 193824/225000 [367:14:47<78:59:19,  9.12s/it]
 86%|████████▌ | 193825/225000 [367:14:54<73:23:22,  8.47s/it]
 86%|████████▌ | 193826/225000 [367:15:03<74:31:11,  8.61s/it]
 86%|████████▌ | 193827/225000 [367:15:08<64:01:56,  7.39s/it]
 86%|████████▌ | 193828/225000 [367:15:15<64:00:30,  7.39s/it]
 86%|████████▌ | 193829/225000 [367:15:30<83:15:43,  9.62s/it]
 86%|████████▌ | 193830/225000 [367:15:36<73:44:52,  8.52s/it]
 86%|████████▌ | 193831/225000 [367:15:46<77:58:58,  9.01s/it]
 86%|████████▌ | 193832/225000 [367:15:56<81:15:41,  9.39s/it]
 86%|████████▌ | 193833/225000 [367:16:03<73:51:27,  8.53s/it]
 86%|████████▌ | 193834/225000 [367:16:10<70:51:08,  8.18s/it]
 86%|████████▌ | 193835/225000 [367:16:19<73:38:19,  8.51s/it]
 86%|████████▌ | 193836/225000 [367:16:27<70:51:23,  8.19s/it]
 86%|████████▌ | 193837/225000 [367:16:33<65:14:58,  7.54s/it]
 86%|████████▌ | 193838/225000 [367:16:43<72:20:46,  8.36s/it]
 86%|████████▌ | 193839/225000 [367:16:51<69:37:24,  8.04s/it]
 86%|████████▌ | 193840/225000 [367:16:57<65:12:50,  7.53s/it]
 86%|████████▌ | 193841/225000 [367:17:07<70:57:46,  8.20s/it]
 86%|████████▌ | 193842/225000 [367:17:17<77:39:18,  8.97s/it]
 86%|████████▌ | 193843/225000 [367:17:25<73:33:54,  8.50s/it]
 86%|████████▌ | 193844/225000 [367:17:30<65:19:52,  7.55s/it]
 86%|████████▌ | 193845/225000 [367:17:43<79:50:53,  9.23s/it]
 86%|████████▌ | 193846/225000 [367:17:56<89:10:11, 10.30s/it]
 86%|████████▌ | 193847/225000 [367:18:00<73:45:37,  8.52s/it]
 86%|████████▌ | 193848/225000 [367:18:13<82:52:34,  9.58s/it]
 86%|████████▌ | 193849/225000 [367:18:18<72:46:08,  8.41s/it]
 86%|████████▌ | 193850/225000 [367:18:31<83:45:58,  9.68s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.165', 'learning_rate': '2.56e-06', 'epoch': '0.8616', 'num_input_tokens_seen': 612243120, 'train_runtime': '1.322e+06', 'train_tokens_per_second': '463'}

 86%|████████▌ | 193850/225000 [367:18:31<83:45:58,  9.68s/it]
 86%|████████▌ | 193851/225000 [367:18:38<78:04:36,  9.02s/it]
 86%|████████▌ | 193852/225000 [367:18:47<77:50:40,  9.00s/it]
 86%|████████▌ | 193853/225000 [367:18:53<68:31:47,  7.92s/it]
 86%|████████▌ | 193854/225000 [367:18:58<61:38:37,  7.13s/it]
 86%|████████▌ | 193855/225000 [367:19:09<71:41:13,  8.29s/it]
 86%|████████▌ | 193856/225000 [367:19:20<78:45:06,  9.10s/it]
 86%|████████▌ | 193857/225000 [367:19:25<68:45:11,  7.95s/it]
 86%|████████▌ | 193858/225000 [367:19:38<80:04:14,  9.26s/it]
 86%|████████▌ | 193859/225000 [367:19:43<70:11:18,  8.11s/it]
 86%|████████▌ | 193860/225000 [367:19:49<64:34:24,  7.47s/it]
 86%|████████▌ | 193861/225000 [367:20:01<75:45:37,  8.76s/it]
 86%|████████▌ | 193862/225000 [367:20:12<81:46:06,  9.45s/it]
 86%|████████▌ | 193863/225000 [367:20:19<76:09:55,  8.81s/it]
 86%|████████▌ | 193864/225000 [367:20:28<75:53:58,  8.78s/it]
 86%|████████▌ | 193865/225000 [367:20:38<80:55:31,  9.36s/it]
 86%|████████▌ | 193866/225000 [367:20:47<79:22:44,  9.18s/it]
 86%|████████▌ | 193867/225000 [367:20:52<68:15:35,  7.89s/it]
 86%|████████▌ | 193868/225000 [367:20:59<65:30:35,  7.58s/it]
 86%|████████▌ | 193869/225000 [367:21:15<88:21:40, 10.22s/it]
 86%|████████▌ | 193870/225000 [367:21:20<74:50:58,  8.66s/it]
 86%|████████▌ | 193871/225000 [367:21:30<77:40:40,  8.98s/it]
 86%|████████▌ | 193872/225000 [367:21:37<71:01:47,  8.21s/it]
 86%|████████▌ | 193873/225000 [367:21:44<68:07:09,  7.88s/it]
 86%|████████▌ | 193874/225000 [367:21:51<66:12:17,  7.66s/it]
 86%|████████▌ | 193875/225000 [367:21:57<62:50:44,  7.27s/it]
 86%|████████▌ | 193876/225000 [367:22:09<74:33:16,  8.62s/it]
 86%|████████▌ | 193877/225000 [367:22:12<61:14:09,  7.08s/it]
 86%|████████▌ | 193878/225000 [367:22:22<66:33:07,  7.70s/it]
 86%|████████▌ | 193879/225000 [367:22:30<69:05:30,  7.99s/it]
 86%|████████▌ | 193880/225000 [367:22:38<68:47:19,  7.96s/it]
 86%|████████▌ | 193881/225000 [367:22:49<76:29:34,  8.85s/it]
 86%|████████▌ | 193882/225000 [367:22:55<69:10:39,  8.00s/it]
 86%|████████▌ | 193883/225000 [367:23:03<69:13:06,  8.01s/it]
 86%|████████▌ | 193884/225000 [367:23:13<75:08:57,  8.69s/it]
 86%|████████▌ | 193885/225000 [367:23:21<71:14:06,  8.24s/it]
 86%|████████▌ | 193886/225000 [367:23:27<65:54:35,  7.63s/it]
 86%|████████▌ | 193887/225000 [367:23:33<62:46:46,  7.26s/it]
 86%|████████▌ | 193888/225000 [367:23:39<59:19:18,  6.86s/it]
 86%|████████▌ | 193889/225000 [367:23:46<60:05:39,  6.95s/it]
 86%|████████▌ | 193890/225000 [367:24:00<77:18:06,  8.95s/it]
 86%|████████▌ | 193891/225000 [367:24:04<66:10:56,  7.66s/it]
 86%|████████▌ | 193892/225000 [367:24:17<79:16:25,  9.17s/it]
 86%|████████▌ | 193893/225000 [367:24:27<80:04:06,  9.27s/it]
 86%|████████▌ | 193894/225000 [367:24:38<84:22:36,  9.77s/it]
 86%|████████▌ | 193895/225000 [367:24:43<73:37:11,  8.52s/it]
 86%|████████▌ | 193896/225000 [367:24:56<85:50:29,  9.94s/it]
 86%|████████▌ | 193897/225000 [367:25:02<75:03:42,  8.69s/it]
 86%|████████▌ | 193898/225000 [367:25:16<87:13:14, 10.10s/it]
 86%|████████▌ | 193899/225000 [367:25:26<87:53:40, 10.17s/it]
 86%|████████▌ | 193900/225000 [367:25:30<72:35:50,  8.40s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.471', 'learning_rate': '2.552e-06', 'epoch': '0.8618', 'num_input_tokens_seen': 612466976, 'train_runtime': '1.323e+06', 'train_tokens_per_second': '463'}

 86%|████████▌ | 193900/225000 [367:25:30<72:35:50,  8.40s/it]
 86%|████████▌ | 193901/225000 [367:25:36<64:41:07,  7.49s/it]
 86%|████████▌ | 193902/225000 [367:25:46<72:33:20,  8.40s/it]
 86%|████████▌ | 193903/225000 [367:25:53<69:47:36,  8.08s/it]
 86%|████████▌ | 193904/225000 [367:26:09<89:16:09, 10.33s/it]
 86%|████████▌ | 193905/225000 [367:26:13<73:53:04,  8.55s/it]
 86%|████████▌ | 193906/225000 [367:26:23<76:13:59,  8.83s/it]
 86%|████████▌ | 193907/225000 [367:26:31<73:56:43,  8.56s/it]
 86%|████████▌ | 193908/225000 [367:26:36<65:14:09,  7.55s/it]
 86%|████████▌ | 193909/225000 [367:26:43<64:24:21,  7.46s/it]
 86%|████████▌ | 193910/225000 [367:26:48<58:13:12,  6.74s/it]
 86%|████████▌ | 193911/225000 [367:26:54<55:43:53,  6.45s/it]
 86%|████████▌ | 193912/225000 [367:27:01<57:20:36,  6.64s/it]
 86%|████████▌ | 193913/225000 [367:27:08<58:11:34,  6.74s/it]
 86%|████████▌ | 193914/225000 [367:27:15<57:18:47,  6.64s/it]
 86%|████████▌ | 193915/225000 [367:27:25<66:44:35,  7.73s/it]
 86%|████████▌ | 193916/225000 [367:27:35<73:21:25,  8.50s/it]
 86%|████████▌ | 193917/225000 [367:27:45<75:43:09,  8.77s/it]
 86%|████████▌ | 193918/225000 [367:27:51<69:58:25,  8.10s/it]
 86%|████████▌ | 193919/225000 [367:27:58<67:43:37,  7.84s/it]
 86%|████████▌ | 193920/225000 [367:28:09<75:01:34,  8.69s/it]
 86%|████████▌ | 193921/225000 [367:28:14<65:55:07,  7.64s/it]
 86%|████████▌ | 193922/225000 [367:28:26<76:23:55,  8.85s/it]
 86%|████████▌ | 193923/225000 [367:28:36<78:44:18,  9.12s/it]
 86%|████████▌ | 193924/225000 [367:28:40<66:45:22,  7.73s/it]
 86%|████████▌ | 193925/225000 [367:28:51<75:08:54,  8.71s/it]
 86%|████████▌ | 193926/225000 [367:28:59<74:09:19,  8.59s/it]
 86%|████████▌ | 193927/225000 [367:29:04<62:55:13,  7.29s/it]
 86%|████████▌ | 193928/225000 [367:29:16<74:38:33,  8.65s/it]
 86%|████████▌ | 193929/225000 [367:29:25<77:35:58,  8.99s/it]
 86%|████████▌ | 193930/225000 [367:29:32<70:53:33,  8.21s/it]
 86%|████████▌ | 193931/225000 [367:29:40<71:04:58,  8.24s/it]
 86%|████████▌ | 193932/225000 [367:29:52<80:14:37,  9.30s/it]
 86%|████████▌ | 193933/225000 [367:30:00<78:24:46,  9.09s/it]
 86%|████████▌ | 193934/225000 [367:30:11<83:20:39,  9.66s/it]
 86%|████████▌ | 193935/225000 [367:30:18<74:27:13,  8.63s/it]
 86%|████████▌ | 193936/225000 [367:30:26<74:44:49,  8.66s/it]
 86%|████████▌ | 193937/225000 [367:30:33<69:51:19,  8.10s/it]
 86%|████████▌ | 193938/225000 [367:30:40<66:26:12,  7.70s/it]
 86%|████████▌ | 193939/225000 [367:30:51<75:37:56,  8.77s/it]
 86%|████████▌ | 193940/225000 [367:31:04<85:04:19,  9.86s/it]
 86%|████████▌ | 193941/225000 [367:31:10<76:59:33,  8.92s/it]
 86%|████████▌ | 193942/225000 [367:31:23<85:53:52,  9.96s/it]
 86%|████████▌ | 193943/225000 [367:31:32<85:18:42,  9.89s/it]
 86%|████████▌ | 193944/225000 [367:31:39<76:51:48,  8.91s/it]
 86%|████████▌ | 193945/225000 [367:31:45<69:55:07,  8.11s/it]
 86%|████████▌ | 193946/225000 [367:31:57<80:02:12,  9.28s/it]
 86%|████████▌ | 193947/225000 [367:32:03<70:02:53,  8.12s/it]
 86%|████████▌ | 193948/225000 [367:32:11<69:33:50,  8.06s/it]
 86%|████████▌ | 193949/225000 [367:32:21<75:36:42,  8.77s/it]
 86%|████████▌ | 193950/225000 [367:32:33<82:59:39,  9.62s/it]
                                                              
{'loss': '1.292', 'grad_norm': '1.224', 'learning_rate': '2.544e-06', 'epoch': '0.862', 'num_input_tokens_seen': 612692872, 'train_runtime': '1.323e+06', 'train_tokens_per_second': '463.1'}

 86%|████████▌ | 193950/225000 [367:32:33<82:59:39,  9.62s/it]
 86%|████████▌ | 193951/225000 [367:32:38<70:45:06,  8.20s/it]
 86%|████████▌ | 193952/225000 [367:32:45<68:28:23,  7.94s/it]
 86%|████████▌ | 193953/225000 [367:32:53<69:59:18,  8.12s/it]
 86%|████████▌ | 193954/225000 [367:33:05<79:24:23,  9.21s/it]
 86%|████████▌ | 193955/225000 [367:33:12<73:19:59,  8.50s/it]
 86%|████████▌ | 193956/225000 [367:33:22<77:52:56,  9.03s/it]
 86%|████████▌ | 193957/225000 [367:33:32<78:53:15,  9.15s/it]
 86%|████████▌ | 193958/225000 [367:33:44<86:08:31,  9.99s/it]
 86%|████████▌ | 193959/225000 [367:33:55<89:34:05, 10.39s/it]
 86%|████████▌ | 193960/225000 [367:34:05<89:45:54, 10.41s/it]
 86%|████████▌ | 193961/225000 [367:34:12<80:55:17,  9.39s/it]
 86%|████████▌ | 193962/225000 [367:34:21<78:41:57,  9.13s/it]
 86%|████████▌ | 193963/225000 [367:34:29<77:03:30,  8.94s/it]
 86%|████████▌ | 193964/225000 [367:34:42<85:54:02,  9.96s/it]
 86%|████████▌ | 193965/225000 [367:34:49<78:59:54,  9.16s/it]
 86%|████████▌ | 193966/225000 [367:34:58<79:15:39,  9.19s/it]
 86%|████████▌ | 193967/225000 [367:35:05<71:51:15,  8.34s/it]
 86%|████████▌ | 193968/225000 [367:35:17<83:10:37,  9.65s/it]
 86%|████████▌ | 193969/225000 [367:35:26<81:08:52,  9.41s/it]
 86%|████████▌ | 193970/225000 [367:35:34<77:45:09,  9.02s/it]
 86%|████████▌ | 193971/225000 [367:35:45<81:52:55,  9.50s/it]
 86%|████████▌ | 193972/225000 [367:35:51<73:24:16,  8.52s/it]
 86%|████████▌ | 193973/225000 [367:35:58<69:11:39,  8.03s/it]
 86%|████████▌ | 193974/225000 [367:36:03<62:26:48,  7.25s/it]
 86%|████████▌ | 193975/225000 [367:36:17<77:46:35,  9.02s/it]
 86%|████████▌ | 193976/225000 [367:36:23<70:43:47,  8.21s/it]
 86%|████████▌ | 193977/225000 [367:36:35<80:45:56,  9.37s/it]
 86%|████████▌ | 193978/225000 [367:36:47<86:56:45, 10.09s/it]
 86%|████████▌ | 193979/225000 [367:36:53<76:25:32,  8.87s/it]
 86%|████████▌ | 193980/225000 [367:37:02<76:00:53,  8.82s/it]
 86%|████████▌ | 193981/225000 [367:37:09<73:00:50,  8.47s/it]
 86%|████████▌ | 193982/225000 [367:37:20<79:09:00,  9.19s/it]
 86%|████████▌ | 193983/225000 [367:37:33<87:43:23, 10.18s/it]
 86%|████████▌ | 193984/225000 [367:37:37<73:15:10,  8.50s/it]
 86%|████████▌ | 193985/225000 [367:37:48<78:05:57,  9.07s/it]
 86%|████████▌ | 193986/225000 [367:37:55<73:15:59,  8.50s/it]
 86%|████████▌ | 193987/225000 [367:38:00<64:52:08,  7.53s/it]
 86%|████████▌ | 193988/225000 [367:38:06<59:50:09,  6.95s/it]
 86%|████████▌ | 193989/225000 [367:38:19<76:54:18,  8.93s/it]
 86%|████████▌ | 193990/225000 [367:38:30<82:33:44,  9.58s/it]
 86%|████████▌ | 193991/225000 [367:38:37<75:51:31,  8.81s/it]
 86%|████████▌ | 193992/225000 [367:38:46<75:59:04,  8.82s/it]
 86%|████████▌ | 193993/225000 [367:38:51<66:32:19,  7.73s/it]
 86%|████████▌ | 193994/225000 [367:38:58<63:58:49,  7.43s/it]
 86%|████████▌ | 193995/225000 [367:39:10<76:32:16,  8.89s/it]
 86%|████████▌ | 193996/225000 [367:39:20<77:47:37,  9.03s/it]
 86%|████████▌ | 193997/225000 [367:39:26<71:50:57,  8.34s/it]
 86%|████████▌ | 193998/225000 [367:39:33<66:06:13,  7.68s/it]
 86%|████████▌ | 193999/225000 [367:39:44<77:03:23,  8.95s/it]
 86%|████████▌ | 194000/225000 [367:39:53<76:21:55,  8.87s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.072', 'learning_rate': '2.536e-06', 'epoch': '0.8622', 'num_input_tokens_seen': 612924024, 'train_runtime': '1.324e+06', 'train_tokens_per_second': '463.1'}

 86%|████████▌ | 194000/225000 [367:39:53<76:21:55,  8.87s/it]
 86%|████████▌ | 194001/225000 [367:40:01<74:20:55,  8.63s/it]
 86%|████████▌ | 194002/225000 [367:40:11<77:33:14,  9.01s/it]
 86%|████████▌ | 194003/225000 [367:40:22<81:32:46,  9.47s/it]
 86%|████████▌ | 194004/225000 [367:40:27<69:46:22,  8.10s/it]
 86%|████████▌ | 194005/225000 [367:40:40<83:18:41,  9.68s/it]
 86%|████████▌ | 194006/225000 [367:40:47<77:24:51,  8.99s/it]
 86%|████████▌ | 194007/225000 [367:40:54<70:23:39,  8.18s/it]
 86%|████████▌ | 194008/225000 [367:40:59<64:34:23,  7.50s/it]
 86%|████████▌ | 194009/225000 [367:41:09<69:23:08,  8.06s/it]
 86%|████████▌ | 194010/225000 [367:41:16<66:52:53,  7.77s/it]
 86%|████████▌ | 194011/225000 [367:41:27<75:59:37,  8.83s/it]
 86%|████████▌ | 194012/225000 [367:41:35<74:04:45,  8.61s/it]
 86%|████████▌ | 194013/225000 [367:41:42<69:19:34,  8.05s/it]
 86%|████████▌ | 194014/225000 [367:41:48<64:30:01,  7.49s/it]
 86%|████████▌ | 194015/225000 [367:41:58<70:02:57,  8.14s/it]
 86%|████████▌ | 194016/225000 [367:42:11<83:23:06,  9.69s/it]
 86%|████████▌ | 194017/225000 [367:42:17<72:03:41,  8.37s/it]
 86%|████████▌ | 194018/225000 [367:42:22<65:10:37,  7.57s/it]
 86%|████████▌ | 194019/225000 [367:42:29<63:42:38,  7.40s/it]
 86%|████████▌ | 194020/225000 [367:42:40<71:39:58,  8.33s/it]
 86%|████████▌ | 194021/225000 [367:42:47<69:33:34,  8.08s/it]
 86%|████████▌ | 194022/225000 [367:43:01<83:06:59,  9.66s/it]
 86%|████████▌ | 194023/225000 [367:43:07<75:30:46,  8.78s/it]
 86%|████████▌ | 194024/225000 [367:43:20<86:47:34, 10.09s/it]
 86%|████████▌ | 194025/225000 [367:43:26<75:09:00,  8.73s/it]
 86%|████████▌ | 194026/225000 [367:43:33<69:43:12,  8.10s/it]
 86%|████████▌ | 194027/225000 [367:43:44<76:53:53,  8.94s/it]
 86%|████████▌ | 194028/225000 [367:43:53<79:09:36,  9.20s/it]
 86%|████████▌ | 194029/225000 [367:43:59<69:36:56,  8.09s/it]
 86%|████████▌ | 194030/225000 [367:44:08<73:36:59,  8.56s/it]
 86%|████████▌ | 194031/225000 [367:44:16<69:53:46,  8.13s/it]
 86%|████████▌ | 194032/225000 [367:44:25<73:59:34,  8.60s/it]
 86%|████████▌ | 194033/225000 [367:44:33<71:54:22,  8.36s/it]
 86%|████████▌ | 194034/225000 [367:44:44<77:19:07,  8.99s/it]
 86%|████████▌ | 194035/225000 [367:44:48<66:13:33,  7.70s/it]
 86%|████████▌ | 194036/225000 [367:44:57<68:30:31,  7.97s/it]
 86%|████████▌ | 194037/225000 [367:45:03<63:52:25,  7.43s/it]
 86%|████████▌ | 194038/225000 [367:45:08<57:39:13,  6.70s/it]
 86%|████████▌ | 194039/225000 [367:45:15<57:11:38,  6.65s/it]
 86%|████████▌ | 194040/225000 [367:45:24<65:01:08,  7.56s/it]
 86%|████████▌ | 194041/225000 [367:45:37<79:12:04,  9.21s/it]
 86%|████████▌ | 194042/225000 [367:45:42<66:19:29,  7.71s/it]
 86%|████████▌ | 194043/225000 [367:45:46<58:37:57,  6.82s/it]
 86%|████████▌ | 194044/225000 [367:46:00<76:16:17,  8.87s/it]
 86%|████████▌ | 194045/225000 [367:46:09<77:46:23,  9.04s/it]
 86%|████████▌ | 194046/225000 [367:46:15<70:15:49,  8.17s/it]
 86%|████████▌ | 194047/225000 [367:46:21<63:29:14,  7.38s/it]
 86%|████████▌ | 194048/225000 [367:46:34<77:35:23,  9.02s/it]
 86%|████████▌ | 194049/225000 [367:46:40<70:53:40,  8.25s/it]
 86%|████████▌ | 194050/225000 [367:46:49<71:51:06,  8.36s/it]
                                                              
{'loss': '1.28', 'grad_norm': '1.406', 'learning_rate': '2.528e-06', 'epoch': '0.8624', 'num_input_tokens_seen': 613146744, 'train_runtime': '1.324e+06', 'train_tokens_per_second': '463.1'}

 86%|████████▌ | 194050/225000 [367:46:49<71:51:06,  8.36s/it]
 86%|████████▌ | 194051/225000 [367:46:54<63:29:18,  7.38s/it]
 86%|████████▌ | 194052/225000 [367:47:05<73:48:49,  8.59s/it]
 86%|████████▌ | 194053/225000 [367:47:13<70:02:52,  8.15s/it]
 86%|████████▌ | 194054/225000 [367:47:19<66:08:59,  7.70s/it]
 86%|████████▌ | 194055/225000 [367:47:32<78:13:06,  9.10s/it]
 86%|████████▌ | 194056/225000 [367:47:41<78:54:40,  9.18s/it]
 86%|████████▌ | 194057/225000 [367:47:49<76:12:02,  8.87s/it]
 86%|████████▌ | 194058/225000 [367:47:57<72:52:52,  8.48s/it]
 86%|████████▌ | 194059/225000 [367:48:02<66:01:04,  7.68s/it]
 86%|████████▌ | 194060/225000 [367:48:09<63:27:28,  7.38s/it]
 86%|████████▌ | 194061/225000 [367:48:16<62:31:45,  7.28s/it]
 86%|████████▌ | 194062/225000 [367:48:31<82:28:50,  9.60s/it]
 86%|████████▋ | 194063/225000 [367:48:39<78:36:32,  9.15s/it]
 86%|████████▋ | 194064/225000 [367:48:48<78:40:21,  9.16s/it]
 86%|████████▋ | 194065/225000 [367:49:00<84:19:23,  9.81s/it]
 86%|████████▋ | 194066/225000 [367:49:07<76:28:03,  8.90s/it]
 86%|████████▋ | 194067/225000 [367:49:15<75:35:06,  8.80s/it]
 86%|████████▋ | 194068/225000 [367:49:23<73:11:47,  8.52s/it]
 86%|████████▋ | 194069/225000 [367:49:29<67:16:53,  7.83s/it]
 86%|████████▋ | 194070/225000 [367:49:35<62:42:21,  7.30s/it]
 86%|████████▋ | 194071/225000 [367:49:46<71:46:38,  8.35s/it]
 86%|████████▋ | 194072/225000 [367:49:51<63:13:34,  7.36s/it]
 86%|████████▋ | 194073/225000 [367:50:02<71:05:22,  8.28s/it]
 86%|████████▋ | 194074/225000 [367:50:07<64:08:10,  7.47s/it]
 86%|████████▋ | 194075/225000 [367:50:18<73:56:51,  8.61s/it]
 86%|████████▋ | 194076/225000 [367:50:25<68:52:47,  8.02s/it]
 86%|████████▋ | 194077/225000 [367:50:32<66:01:44,  7.69s/it]
 86%|████████▋ | 194078/225000 [367:50:43<75:03:44,  8.74s/it]
 86%|████████▋ | 194079/225000 [367:50:47<62:03:47,  7.23s/it]
 86%|████████▋ | 194080/225000 [367:50:56<66:18:48,  7.72s/it]
 86%|████████▋ | 194081/225000 [367:51:01<59:09:53,  6.89s/it]
 86%|████████▋ | 194082/225000 [367:51:11<67:27:07,  7.85s/it]
 86%|████████▋ | 194083/225000 [367:51:18<66:23:48,  7.73s/it]
 86%|████████▋ | 194084/225000 [367:51:28<72:17:49,  8.42s/it]
 86%|████████▋ | 194085/225000 [367:51:38<76:42:43,  8.93s/it]
 86%|████████▋ | 194086/225000 [367:51:46<74:08:29,  8.63s/it]
 86%|████████▋ | 194087/225000 [367:51:53<68:49:39,  8.02s/it]
 86%|████████▋ | 194088/225000 [367:51:58<60:06:17,  7.00s/it]
 86%|████████▋ | 194089/225000 [367:52:06<64:23:53,  7.50s/it]
 86%|████████▋ | 194090/225000 [367:52:13<63:16:42,  7.37s/it]
 86%|████████▋ | 194091/225000 [367:52:18<57:38:54,  6.71s/it]
 86%|████████▋ | 194092/225000 [367:52:28<63:47:48,  7.43s/it]
 86%|████████▋ | 194093/225000 [367:52:42<82:15:10,  9.58s/it]
 86%|████████▋ | 194094/225000 [367:52:49<74:26:11,  8.67s/it]
 86%|████████▋ | 194095/225000 [367:52:55<67:14:30,  7.83s/it]
 86%|████████▋ | 194096/225000 [367:53:07<79:23:43,  9.25s/it]
 86%|████████▋ | 194097/225000 [367:53:18<82:40:08,  9.63s/it]
 86%|████████▋ | 194098/225000 [367:53:22<69:29:39,  8.10s/it]
 86%|████████▋ | 194099/225000 [367:53:33<76:48:02,  8.95s/it]
 86%|████████▋ | 194100/225000 [367:53:39<69:02:02,  8.04s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.51', 'learning_rate': '2.52e-06', 'epoch': '0.8627', 'num_input_tokens_seen': 613366352, 'train_runtime': '1.324e+06', 'train_tokens_per_second': '463.1'}

 86%|████████▋ | 194100/225000 [367:53:39<69:02:02,  8.04s/it]
 86%|████████▋ | 194101/225000 [367:53:48<72:35:43,  8.46s/it]
 86%|████████▋ | 194102/225000 [367:54:01<82:40:39,  9.63s/it]
 86%|████████▋ | 194103/225000 [367:54:08<76:34:48,  8.92s/it]
 86%|████████▋ | 194104/225000 [367:54:13<66:41:15,  7.77s/it]
 86%|████████▋ | 194105/225000 [367:54:22<70:22:57,  8.20s/it]
 86%|████████▋ | 194106/225000 [367:54:33<77:25:23,  9.02s/it]
 86%|████████▋ | 194107/225000 [367:54:39<69:02:20,  8.05s/it]
 86%|████████▋ | 194108/225000 [367:54:51<80:11:54,  9.35s/it]
 86%|████████▋ | 194109/225000 [367:54:59<76:19:46,  8.90s/it]
 86%|████████▋ | 194110/225000 [367:55:05<68:40:34,  8.00s/it]
 86%|████████▋ | 194111/225000 [367:55:16<75:27:55,  8.80s/it]
 86%|████████▋ | 194112/225000 [367:55:25<76:25:02,  8.91s/it]
 86%|████████▋ | 194113/225000 [367:55:32<70:49:42,  8.26s/it]
 86%|████████▋ | 194114/225000 [367:55:41<72:08:03,  8.41s/it]
 86%|████████▋ | 194115/225000 [367:55:48<69:13:49,  8.07s/it]
 86%|████████▋ | 194116/225000 [367:55:54<64:06:50,  7.47s/it]
 86%|████████▋ | 194117/225000 [367:55:59<58:35:22,  6.83s/it]
 86%|████████▋ | 194118/225000 [367:56:08<62:22:56,  7.27s/it]
 86%|████████▋ | 194119/225000 [367:56:16<66:18:31,  7.73s/it]
 86%|████████▋ | 194120/225000 [367:56:23<64:26:30,  7.51s/it]
 86%|████████▋ | 194121/225000 [367:56:32<68:00:33,  7.93s/it]
 86%|████████▋ | 194122/225000 [367:56:43<75:34:27,  8.81s/it]
 86%|████████▋ | 194123/225000 [367:56:50<69:35:38,  8.11s/it]
 86%|████████▋ | 194124/225000 [367:56:58<70:42:50,  8.24s/it]
 86%|████████▋ | 194125/225000 [367:57:10<78:48:38,  9.19s/it]
 86%|████████▋ | 194126/225000 [367:57:16<72:50:34,  8.49s/it]
 86%|████████▋ | 194127/225000 [367:57:27<77:12:50,  9.00s/it]
 86%|████████▋ | 194128/225000 [367:57:36<77:33:25,  9.04s/it]
 86%|████████▋ | 194129/225000 [367:57:42<69:43:59,  8.13s/it]
 86%|████████▋ | 194130/225000 [367:57:51<71:36:56,  8.35s/it]
 86%|████████▋ | 194131/225000 [367:57:58<68:00:43,  7.93s/it]
 86%|████████▋ | 194132/225000 [367:58:06<68:18:30,  7.97s/it]
 86%|████████▋ | 194133/225000 [367:58:18<80:03:00,  9.34s/it]
 86%|████████▋ | 194134/225000 [367:58:29<84:32:10,  9.86s/it]
 86%|████████▋ | 194135/225000 [367:58:34<70:59:48,  8.28s/it]
 86%|████████▋ | 194136/225000 [367:58:42<71:31:19,  8.34s/it]
 86%|████████▋ | 194137/225000 [367:58:51<72:05:17,  8.41s/it]
 86%|████████▋ | 194138/225000 [367:59:02<79:11:49,  9.24s/it]
 86%|████████▋ | 194139/225000 [367:59:07<67:18:27,  7.85s/it]
 86%|████████▋ | 194140/225000 [367:59:14<65:40:11,  7.66s/it]
 86%|████████▋ | 194141/225000 [367:59:27<78:42:11,  9.18s/it]
 86%|████████▋ | 194142/225000 [367:59:32<70:12:55,  8.19s/it]
 86%|████████▋ | 194143/225000 [367:59:39<65:08:52,  7.60s/it]
 86%|████████▋ | 194144/225000 [367:59:45<62:41:20,  7.31s/it]
 86%|████████▋ | 194145/225000 [367:59:57<72:37:13,  8.47s/it]
 86%|████████▋ | 194146/225000 [368:00:06<76:26:04,  8.92s/it]
 86%|████████▋ | 194147/225000 [368:00:17<80:02:55,  9.34s/it]
 86%|████████▋ | 194148/225000 [368:00:25<77:54:02,  9.09s/it]
 86%|████████▋ | 194149/225000 [368:00:32<70:45:50,  8.26s/it]
 86%|████████▋ | 194150/225000 [368:00:41<72:28:43,  8.46s/it]
                                                              
{'loss': '1.279', 'grad_norm': '1.429', 'learning_rate': '2.512e-06', 'epoch': '0.8629', 'num_input_tokens_seen': 613588600, 'train_runtime': '1.325e+06', 'train_tokens_per_second': '463.1'}

 86%|████████▋ | 194150/225000 [368:00:41<72:28:43,  8.46s/it]
 86%|████████▋ | 194151/225000 [368:00:48<70:57:22,  8.28s/it]
 86%|████████▋ | 194152/225000 [368:00:59<77:50:38,  9.08s/it]
 86%|████████▋ | 194153/225000 [368:01:13<89:16:16, 10.42s/it]
 86%|████████▋ | 194154/225000 [368:01:17<73:42:15,  8.60s/it]
 86%|████████▋ | 194155/225000 [368:01:27<76:25:28,  8.92s/it]
 86%|████████▋ | 194156/225000 [368:01:34<70:41:36,  8.25s/it]
 86%|████████▋ | 194157/225000 [368:01:45<77:54:12,  9.09s/it]
 86%|████████▋ | 194158/225000 [368:01:50<68:49:20,  8.03s/it]
 86%|████████▋ | 194159/225000 [368:02:00<74:22:37,  8.68s/it]
 86%|████████▋ | 194160/225000 [368:02:08<72:18:26,  8.44s/it]
 86%|████████▋ | 194161/225000 [368:02:19<78:47:35,  9.20s/it]
 86%|████████▋ | 194162/225000 [368:02:25<70:19:30,  8.21s/it]
 86%|████████▋ | 194163/225000 [368:02:33<70:06:56,  8.19s/it]
 86%|████████▋ | 194164/225000 [368:02:43<74:05:08,  8.65s/it]
 86%|████████▋ | 194165/225000 [368:02:53<76:29:02,  8.93s/it]
 86%|████████▋ | 194166/225000 [368:03:01<75:33:58,  8.82s/it]
 86%|████████▋ | 194167/225000 [368:03:09<73:53:55,  8.63s/it]
 86%|████████▋ | 194168/225000 [368:03:19<76:43:33,  8.96s/it]
 86%|████████▋ | 194169/225000 [368:03:28<75:15:13,  8.79s/it]
 86%|████████▋ | 194170/225000 [368:03:34<68:51:05,  8.04s/it]
 86%|████████▋ | 194171/225000 [368:03:46<80:09:46,  9.36s/it]
 86%|████████▋ | 194172/225000 [368:03:53<72:52:59,  8.51s/it]
 86%|████████▋ | 194173/225000 [368:04:00<69:02:40,  8.06s/it]
 86%|████████▋ | 194174/225000 [368:04:13<81:35:58,  9.53s/it]
 86%|████████▋ | 194175/225000 [368:04:20<76:23:09,  8.92s/it]
 86%|████████▋ | 194176/225000 [368:04:26<67:42:01,  7.91s/it]
 86%|████████▋ | 194177/225000 [368:04:39<81:01:38,  9.46s/it]
 86%|████████▋ | 194178/225000 [368:04:45<72:24:20,  8.46s/it]
 86%|████████▋ | 194179/225000 [368:04:51<67:18:12,  7.86s/it]
 86%|████████▋ | 194180/225000 [368:05:00<69:18:53,  8.10s/it]
 86%|████████▋ | 194181/225000 [368:05:13<81:17:50,  9.50s/it]
 86%|████████▋ | 194182/225000 [368:05:20<74:22:01,  8.69s/it]
 86%|████████▋ | 194183/225000 [368:05:26<68:21:24,  7.99s/it]
 86%|████████▋ | 194184/225000 [368:05:41<87:08:35, 10.18s/it]
 86%|████████▋ | 194185/225000 [368:05:46<73:10:30,  8.55s/it]
 86%|████████▋ | 194186/225000 [368:05:56<77:35:39,  9.07s/it]
 86%|████████▋ | 194187/225000 [368:06:04<75:12:46,  8.79s/it]
 86%|████████▋ | 194188/225000 [368:06:11<70:32:44,  8.24s/it]
 86%|████████▋ | 194189/225000 [368:06:19<69:11:01,  8.08s/it]
 86%|████████▋ | 194190/225000 [368:06:25<62:19:52,  7.28s/it]
 86%|████████▋ | 194191/225000 [368:06:33<64:30:48,  7.54s/it]
 86%|████████▋ | 194192/225000 [368:06:41<66:36:53,  7.78s/it]
 86%|████████▋ | 194193/225000 [368:06:46<60:08:32,  7.03s/it]
 86%|████████▋ | 194194/225000 [368:06:58<72:54:27,  8.52s/it]
 86%|████████▋ | 194195/225000 [368:07:09<77:39:20,  9.08s/it]
 86%|████████▋ | 194196/225000 [368:07:15<71:20:35,  8.34s/it]
 86%|████████▋ | 194197/225000 [368:07:23<69:01:23,  8.07s/it]
 86%|████████▋ | 194198/225000 [368:07:29<65:06:59,  7.61s/it]
 86%|████████▋ | 194199/225000 [368:07:35<61:06:51,  7.14s/it]
 86%|████████▋ | 194200/225000 [368:07:43<62:57:57,  7.36s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.139', 'learning_rate': '2.504e-06', 'epoch': '0.8631', 'num_input_tokens_seen': 613815072, 'train_runtime': '1.325e+06', 'train_tokens_per_second': '463.2'}

 86%|████████▋ | 194200/225000 [368:07:43<62:57:57,  7.36s/it]
 86%|████████▋ | 194201/225000 [368:07:54<70:50:59,  8.28s/it]
 86%|████████▋ | 194202/225000 [368:08:00<65:14:33,  7.63s/it]
 86%|████████▋ | 194203/225000 [368:08:13<80:53:47,  9.46s/it]
 86%|████████▋ | 194204/225000 [368:08:19<69:46:40,  8.16s/it]
 86%|████████▋ | 194205/225000 [368:08:26<67:24:37,  7.88s/it]
 86%|████████▋ | 194206/225000 [368:08:36<74:08:04,  8.67s/it]
 86%|████████▋ | 194207/225000 [368:08:42<65:20:33,  7.64s/it]
 86%|████████▋ | 194208/225000 [368:08:51<71:05:56,  8.31s/it]
 86%|████████▋ | 194209/225000 [368:08:56<62:18:17,  7.28s/it]
 86%|████████▋ | 194210/225000 [368:09:05<64:46:49,  7.57s/it]
 86%|████████▋ | 194211/225000 [368:09:14<68:32:57,  8.02s/it]
 86%|████████▋ | 194212/225000 [368:09:23<72:17:30,  8.45s/it]
 86%|████████▋ | 194213/225000 [368:09:28<62:02:43,  7.26s/it]
 86%|████████▋ | 194214/225000 [368:09:40<74:26:28,  8.70s/it]
 86%|████████▋ | 194215/225000 [368:09:48<72:55:57,  8.53s/it]
 86%|████████▋ | 194216/225000 [368:09:58<76:35:34,  8.96s/it]
 86%|████████▋ | 194217/225000 [368:10:08<78:44:49,  9.21s/it]
 86%|████████▋ | 194218/225000 [368:10:14<71:18:51,  8.34s/it]
 86%|████████▋ | 194219/225000 [368:10:19<63:34:27,  7.44s/it]
 86%|████████▋ | 194220/225000 [368:10:25<59:51:30,  7.00s/it]
 86%|████████▋ | 194221/225000 [368:10:41<81:33:09,  9.54s/it]
 86%|████████▋ | 194222/225000 [368:10:49<78:32:00,  9.19s/it]
 86%|████████▋ | 194223/225000 [368:10:57<74:37:58,  8.73s/it]
 86%|████████▋ | 194224/225000 [368:11:03<68:15:19,  7.98s/it]
 86%|████████▋ | 194225/225000 [368:11:09<64:28:43,  7.54s/it]
 86%|████████▋ | 194226/225000 [368:11:16<62:26:57,  7.31s/it]
 86%|████████▋ | 194227/225000 [368:11:27<71:22:09,  8.35s/it]
 86%|████████▋ | 194228/225000 [368:11:43<92:19:43, 10.80s/it]
 86%|████████▋ | 194229/225000 [368:11:49<79:01:15,  9.24s/it]
 86%|████████▋ | 194230/225000 [368:11:56<73:26:17,  8.59s/it]
 86%|████████▋ | 194231/225000 [368:12:06<76:38:22,  8.97s/it]
 86%|████████▋ | 194232/225000 [368:12:13<72:04:08,  8.43s/it]
 86%|████████▋ | 194233/225000 [368:12:20<67:29:28,  7.90s/it]
 86%|████████▋ | 194234/225000 [368:12:29<69:46:21,  8.16s/it]
 86%|████████▋ | 194235/225000 [368:12:40<77:34:34,  9.08s/it]
 86%|████████▋ | 194236/225000 [368:12:46<69:47:36,  8.17s/it]
 86%|████████▋ | 194237/225000 [368:12:53<67:36:29,  7.91s/it]
 86%|████████▋ | 194238/225000 [368:13:00<63:33:36,  7.44s/it]
 86%|████████▋ | 194239/225000 [368:13:07<62:40:37,  7.34s/it]
 86%|████████▋ | 194240/225000 [368:13:13<59:52:20,  7.01s/it]
 86%|████████▋ | 194241/225000 [368:13:28<80:20:41,  9.40s/it]
 86%|████████▋ | 194242/225000 [368:13:40<86:20:38, 10.11s/it]
 86%|████████▋ | 194243/225000 [368:13:44<72:07:55,  8.44s/it]
 86%|████████▋ | 194244/225000 [368:13:56<81:15:13,  9.51s/it]
 86%|████████▋ | 194245/225000 [368:14:02<71:25:21,  8.36s/it]
 86%|████████▋ | 194246/225000 [368:14:10<72:07:53,  8.44s/it]
 86%|████████▋ | 194247/225000 [368:14:16<64:08:49,  7.51s/it]
 86%|████████▋ | 194248/225000 [368:14:24<65:34:04,  7.68s/it]
 86%|████████▋ | 194249/225000 [368:14:31<63:26:13,  7.43s/it]
 86%|████████▋ | 194250/225000 [368:14:39<64:30:18,  7.55s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.171', 'learning_rate': '2.496e-06', 'epoch': '0.8633', 'num_input_tokens_seen': 614028448, 'train_runtime': '1.326e+06', 'train_tokens_per_second': '463.2'}

 86%|████████▋ | 194250/225000 [368:14:39<64:30:18,  7.55s/it]
 86%|████████▋ | 194251/225000 [368:14:48<68:37:04,  8.03s/it]
 86%|████████▋ | 194252/225000 [368:14:59<77:06:00,  9.03s/it]
 86%|████████▋ | 194253/225000 [368:15:09<79:44:03,  9.34s/it]
 86%|████████▋ | 194254/225000 [368:15:16<73:57:25,  8.66s/it]
 86%|████████▋ | 194255/225000 [368:15:26<75:48:26,  8.88s/it]
 86%|████████▋ | 194256/225000 [368:15:33<71:35:30,  8.38s/it]
 86%|████████▋ | 194257/225000 [368:15:40<67:43:50,  7.93s/it]
 86%|████████▋ | 194258/225000 [368:15:49<71:09:41,  8.33s/it]
 86%|████████▋ | 194259/225000 [368:15:58<73:02:42,  8.55s/it]
 86%|████████▋ | 194260/225000 [368:16:08<77:58:16,  9.13s/it]
 86%|████████▋ | 194261/225000 [368:16:18<79:29:00,  9.31s/it]
 86%|████████▋ | 194262/225000 [368:16:31<88:23:13, 10.35s/it]
 86%|████████▋ | 194263/225000 [368:16:37<78:06:42,  9.15s/it]
 86%|████████▋ | 194264/225000 [368:16:44<71:15:35,  8.35s/it]
 86%|████████▋ | 194265/225000 [368:16:55<79:00:16,  9.25s/it]
 86%|████████▋ | 194266/225000 [368:16:59<65:21:10,  7.66s/it]
 86%|████████▋ | 194267/225000 [368:17:11<75:06:13,  8.80s/it]
 86%|████████▋ | 194268/225000 [368:17:17<69:08:30,  8.10s/it]
 86%|████████▋ | 194269/225000 [368:17:28<76:39:06,  8.98s/it]
 86%|████████▋ | 194270/225000 [368:17:36<74:32:49,  8.73s/it]
 86%|████████▋ | 194271/225000 [368:17:47<79:09:26,  9.27s/it]
 86%|████████▋ | 194272/225000 [368:17:54<72:44:14,  8.52s/it]
 86%|████████▋ | 194273/225000 [368:18:05<79:10:30,  9.28s/it]
 86%|████████▋ | 194274/225000 [368:18:15<82:13:46,  9.63s/it]
 86%|████████▋ | 194275/225000 [368:18:23<77:00:20,  9.02s/it]
 86%|████████▋ | 194276/225000 [368:18:36<89:00:54, 10.43s/it]
 86%|████████▋ | 194277/225000 [368:18:44<81:43:41,  9.58s/it]
 86%|████████▋ | 194278/225000 [368:18:52<78:29:45,  9.20s/it]
 86%|████████▋ | 194279/225000 [368:19:00<73:32:41,  8.62s/it]
 86%|████████▋ | 194280/225000 [368:19:06<67:59:05,  7.97s/it]
 86%|████████▋ | 194281/225000 [368:19:12<63:24:24,  7.43s/it]
 86%|████████▋ | 194282/225000 [368:19:17<58:01:19,  6.80s/it]
 86%|████████▋ | 194283/225000 [368:19:23<55:59:10,  6.56s/it]
 86%|████████▋ | 194284/225000 [368:19:37<73:00:12,  8.56s/it]
 86%|████████▋ | 194285/225000 [368:19:50<84:33:46,  9.91s/it]
 86%|████████▋ | 194286/225000 [368:19:58<79:12:48,  9.28s/it]
 86%|████████▋ | 194287/225000 [368:20:07<80:37:15,  9.45s/it]
 86%|████████▋ | 194288/225000 [368:20:15<75:23:03,  8.84s/it]
 86%|████████▋ | 194289/225000 [368:20:20<67:12:45,  7.88s/it]
 86%|████████▋ | 194290/225000 [368:20:32<77:39:52,  9.10s/it]
 86%|████████▋ | 194291/225000 [368:20:38<67:42:50,  7.94s/it]
 86%|████████▋ | 194292/225000 [368:20:46<69:06:59,  8.10s/it]
 86%|████████▋ | 194293/225000 [368:20:55<72:05:33,  8.45s/it]
 86%|████████▋ | 194294/225000 [368:21:08<81:58:17,  9.61s/it]
 86%|████████▋ | 194295/225000 [368:21:17<81:18:11,  9.53s/it]
 86%|████████▋ | 194296/225000 [368:21:24<73:50:06,  8.66s/it]
 86%|████████▋ | 194297/225000 [368:21:32<71:42:55,  8.41s/it]
 86%|████████▋ | 194298/225000 [368:21:41<75:33:47,  8.86s/it]
 86%|████████▋ | 194299/225000 [368:21:55<86:42:49, 10.17s/it]
 86%|████████▋ | 194300/225000 [368:22:00<73:09:41,  8.58s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.158', 'learning_rate': '2.488e-06', 'epoch': '0.8636', 'num_input_tokens_seen': 614254624, 'train_runtime': '1.326e+06', 'train_tokens_per_second': '463.2'}

 86%|████████▋ | 194300/225000 [368:22:00<73:09:41,  8.58s/it]
 86%|████████▋ | 194301/225000 [368:22:10<78:51:50,  9.25s/it]
 86%|████████▋ | 194302/225000 [368:22:19<77:20:37,  9.07s/it]
 86%|████████▋ | 194303/225000 [368:22:29<81:02:28,  9.50s/it]
 86%|████████▋ | 194304/225000 [368:22:35<70:43:45,  8.30s/it]
 86%|████████▋ | 194305/225000 [368:22:42<68:45:59,  8.07s/it]
 86%|████████▋ | 194306/225000 [368:22:52<72:00:02,  8.44s/it]
 86%|████████▋ | 194307/225000 [368:22:58<67:04:35,  7.87s/it]
 86%|████████▋ | 194308/225000 [368:23:05<63:32:52,  7.45s/it]
 86%|████████▋ | 194309/225000 [368:23:14<66:40:33,  7.82s/it]
 86%|████████▋ | 194310/225000 [368:23:22<69:20:39,  8.13s/it]
 86%|████████▋ | 194311/225000 [368:23:29<65:26:02,  7.68s/it]
 86%|████████▋ | 194312/225000 [368:23:35<62:13:23,  7.30s/it]
 86%|████████▋ | 194313/225000 [368:23:43<64:08:12,  7.52s/it]
 86%|████████▋ | 194314/225000 [368:23:57<80:10:59,  9.41s/it]
 86%|████████▋ | 194315/225000 [368:24:02<68:11:48,  8.00s/it]
 86%|████████▋ | 194316/225000 [368:24:15<81:18:49,  9.54s/it]
 86%|████████▋ | 194317/225000 [368:24:23<76:32:24,  8.98s/it]
 86%|████████▋ | 194318/225000 [368:24:32<76:28:39,  8.97s/it]
 86%|████████▋ | 194319/225000 [368:24:36<63:43:19,  7.48s/it]
 86%|████████▋ | 194320/225000 [368:24:48<76:10:44,  8.94s/it]
 86%|████████▋ | 194321/225000 [368:24:56<73:54:33,  8.67s/it]
 86%|████████▋ | 194322/225000 [368:25:03<69:16:23,  8.13s/it]
 86%|████████▋ | 194323/225000 [368:25:09<65:04:41,  7.64s/it]
 86%|████████▋ | 194324/225000 [368:25:18<68:35:31,  8.05s/it]
 86%|████████▋ | 194325/225000 [368:25:26<67:28:23,  7.92s/it]
 86%|████████▋ | 194326/225000 [368:25:32<62:08:31,  7.29s/it]
 86%|████████▋ | 194327/225000 [368:25:42<69:06:59,  8.11s/it]
 86%|████████▋ | 194328/225000 [368:25:47<62:19:07,  7.31s/it]
 86%|████████▋ | 194329/225000 [368:25:59<73:53:56,  8.67s/it]
 86%|████████▋ | 194330/225000 [368:26:07<70:47:27,  8.31s/it]
 86%|████████▋ | 194331/225000 [368:26:15<71:56:29,  8.44s/it]
 86%|████████▋ | 194332/225000 [368:26:24<70:56:46,  8.33s/it]
 86%|████████▋ | 194333/225000 [368:26:38<86:10:35, 10.12s/it]
 86%|████████▋ | 194334/225000 [368:26:48<87:13:06, 10.24s/it]
 86%|████████▋ | 194335/225000 [368:27:01<93:32:41, 10.98s/it]
 86%|████████▋ | 194336/225000 [368:27:06<79:08:05,  9.29s/it]
 86%|████████▋ | 194337/225000 [368:27:18<83:48:27,  9.84s/it]
 86%|████████▋ | 194338/225000 [368:27:27<83:21:47,  9.79s/it]
 86%|████████▋ | 194339/225000 [368:27:34<76:18:48,  8.96s/it]
 86%|████████▋ | 194340/225000 [368:27:42<72:56:36,  8.56s/it]
 86%|████████▋ | 194341/225000 [368:27:52<75:53:38,  8.91s/it]
 86%|████████▋ | 194342/225000 [368:27:58<70:36:31,  8.29s/it]
 86%|████████▋ | 194343/225000 [368:28:04<63:55:49,  7.51s/it]
 86%|████████▋ | 194344/225000 [368:28:12<64:21:23,  7.56s/it]
 86%|████████▋ | 194345/225000 [368:28:20<65:00:16,  7.63s/it]
 86%|████████▋ | 194346/225000 [368:28:31<74:35:29,  8.76s/it]
 86%|████████▋ | 194347/225000 [368:28:37<67:05:53,  7.88s/it]
 86%|████████▋ | 194348/225000 [368:28:46<70:31:55,  8.28s/it]
 86%|████████▋ | 194349/225000 [368:28:52<65:45:15,  7.72s/it]
 86%|████████▋ | 194350/225000 [368:28:58<60:02:24,  7.05s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.397', 'learning_rate': '2.48e-06', 'epoch': '0.8638', 'num_input_tokens_seen': 614479432, 'train_runtime': '1.327e+06', 'train_tokens_per_second': '463.2'}

 86%|████████▋ | 194350/225000 [368:28:58<60:02:24,  7.05s/it]
 86%|████████▋ | 194351/225000 [368:29:07<64:49:06,  7.61s/it]
 86%|████████▋ | 194352/225000 [368:29:15<66:54:43,  7.86s/it]
 86%|████████▋ | 194353/225000 [368:29:30<84:41:27,  9.95s/it]
 86%|████████▋ | 194354/225000 [368:29:41<85:54:42, 10.09s/it]
 86%|████████▋ | 194355/225000 [368:29:45<72:29:21,  8.52s/it]
 86%|████████▋ | 194356/225000 [368:29:57<80:28:38,  9.45s/it]
 86%|████████▋ | 194357/225000 [368:30:06<80:06:01,  9.41s/it]
 86%|████████▋ | 194358/225000 [368:30:12<69:33:44,  8.17s/it]
 86%|████████▋ | 194359/225000 [368:30:17<62:46:00,  7.37s/it]
 86%|████████▋ | 194360/225000 [368:30:31<78:30:56,  9.23s/it]
 86%|████████▋ | 194361/225000 [368:30:37<72:10:59,  8.48s/it]
 86%|████████▋ | 194362/225000 [368:30:44<68:26:25,  8.04s/it]
 86%|████████▋ | 194363/225000 [368:30:51<65:12:19,  7.66s/it]
 86%|████████▋ | 194364/225000 [368:31:06<84:39:33,  9.95s/it]
 86%|████████▋ | 194365/225000 [368:31:14<78:21:04,  9.21s/it]
 86%|████████▋ | 194366/225000 [368:31:23<78:10:44,  9.19s/it]
 86%|████████▋ | 194367/225000 [368:31:29<68:47:48,  8.09s/it]
 86%|████████▋ | 194368/225000 [368:31:35<63:33:16,  7.47s/it]
 86%|████████▋ | 194369/225000 [368:31:39<55:27:59,  6.52s/it]
 86%|████████▋ | 194370/225000 [368:31:52<71:00:59,  8.35s/it]
 86%|████████▋ | 194371/225000 [368:31:59<68:56:00,  8.10s/it]
 86%|████████▋ | 194372/225000 [368:32:05<63:44:48,  7.49s/it]
 86%|████████▋ | 194373/225000 [368:32:11<59:40:45,  7.01s/it]
 86%|████████▋ | 194374/225000 [368:32:24<73:36:33,  8.65s/it]
 86%|████████▋ | 194375/225000 [368:32:33<76:20:31,  8.97s/it]
 86%|████████▋ | 194376/225000 [368:32:40<69:32:48,  8.18s/it]
 86%|████████▋ | 194377/225000 [368:32:49<71:58:02,  8.46s/it]
 86%|████████▋ | 194378/225000 [368:32:58<73:04:29,  8.59s/it]
 86%|████████▋ | 194379/225000 [368:33:05<70:12:12,  8.25s/it]
 86%|████████▋ | 194380/225000 [368:33:11<63:27:07,  7.46s/it]
 86%|████████▋ | 194381/225000 [368:33:22<72:19:08,  8.50s/it]
 86%|████████▋ | 194382/225000 [368:33:29<68:28:38,  8.05s/it]
 86%|████████▋ | 194383/225000 [368:33:33<60:16:46,  7.09s/it]
 86%|████████▋ | 194384/225000 [368:33:40<58:46:00,  6.91s/it]
 86%|████████▋ | 194385/225000 [368:33:50<67:17:41,  7.91s/it]
 86%|████████▋ | 194386/225000 [368:33:56<61:36:30,  7.24s/it]
 86%|████████▋ | 194387/225000 [368:34:02<58:08:34,  6.84s/it]
 86%|████████▋ | 194388/225000 [368:34:08<57:38:11,  6.78s/it]
 86%|████████▋ | 194389/225000 [368:34:17<63:22:03,  7.45s/it]
 86%|████████▋ | 194390/225000 [368:34:26<65:35:42,  7.71s/it]
 86%|████████▋ | 194391/225000 [368:34:38<77:43:45,  9.14s/it]
 86%|████████▋ | 194392/225000 [368:34:44<70:17:59,  8.27s/it]
 86%|████████▋ | 194393/225000 [368:34:49<60:00:28,  7.06s/it]
 86%|████████▋ | 194394/225000 [368:34:57<61:59:37,  7.29s/it]
 86%|████████▋ | 194395/225000 [368:35:10<78:51:42,  9.28s/it]
 86%|████████▋ | 194396/225000 [368:35:20<79:03:43,  9.30s/it]
 86%|████████▋ | 194397/225000 [368:35:28<75:20:59,  8.86s/it]
 86%|████████▋ | 194398/225000 [368:35:34<69:48:02,  8.21s/it]
 86%|████████▋ | 194399/225000 [368:35:41<67:01:58,  7.89s/it]
 86%|████████▋ | 194400/225000 [368:35:50<68:43:25,  8.09s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.695', 'learning_rate': '2.472e-06', 'epoch': '0.864', 'num_input_tokens_seen': 614698328, 'train_runtime': '1.327e+06', 'train_tokens_per_second': '463.2'}

 86%|████████▋ | 194400/225000 [368:35:50<68:43:25,  8.09s/it]
 86%|████████▋ | 194401/225000 [368:35:56<64:29:12,  7.59s/it]
 86%|████████▋ | 194402/225000 [368:36:07<70:50:58,  8.34s/it]
 86%|████████▋ | 194403/225000 [368:36:13<66:11:32,  7.79s/it]
 86%|████████▋ | 194404/225000 [368:36:19<61:36:08,  7.25s/it]
 86%|████████▋ | 194405/225000 [368:36:29<67:54:54,  7.99s/it]
 86%|████████▋ | 194406/225000 [368:36:43<84:15:20,  9.91s/it]
 86%|████████▋ | 194407/225000 [368:36:49<72:56:57,  8.58s/it]
 86%|████████▋ | 194408/225000 [368:37:02<85:56:54, 10.11s/it]
 86%|████████▋ | 194409/225000 [368:37:09<77:51:22,  9.16s/it]
 86%|████████▋ | 194410/225000 [368:37:18<77:34:04,  9.13s/it]
 86%|████████▋ | 194411/225000 [368:37:24<68:27:36,  8.06s/it]
 86%|████████▋ | 194412/225000 [368:37:30<63:50:28,  7.51s/it]
 86%|████████▋ | 194413/225000 [368:37:35<57:59:09,  6.82s/it]
 86%|████████▋ | 194414/225000 [368:37:40<52:50:42,  6.22s/it]
 86%|████████▋ | 194415/225000 [368:37:56<77:37:53,  9.14s/it]
 86%|████████▋ | 194416/225000 [368:38:05<76:54:51,  9.05s/it]
 86%|████████▋ | 194417/225000 [368:38:15<79:04:29,  9.31s/it]
 86%|████████▋ | 194418/225000 [368:38:21<72:06:50,  8.49s/it]
 86%|████████▋ | 194419/225000 [368:38:28<67:50:20,  7.99s/it]
 86%|████████▋ | 194420/225000 [368:38:33<59:47:46,  7.04s/it]
 86%|████████▋ | 194421/225000 [368:38:45<71:55:32,  8.47s/it]
 86%|████████▋ | 194422/225000 [368:38:51<65:54:21,  7.76s/it]
 86%|████████▋ | 194423/225000 [368:39:01<71:48:11,  8.45s/it]
 86%|████████▋ | 194424/225000 [368:39:13<80:32:42,  9.48s/it]
 86%|████████▋ | 194425/225000 [368:39:21<76:53:01,  9.05s/it]
 86%|████████▋ | 194426/225000 [368:39:28<71:06:10,  8.37s/it]
 86%|████████▋ | 194427/225000 [368:39:34<65:20:51,  7.69s/it]
 86%|████████▋ | 194428/225000 [368:39:48<82:58:27,  9.77s/it]
 86%|████████▋ | 194429/225000 [368:39:55<75:09:11,  8.85s/it]
 86%|████████▋ | 194430/225000 [368:40:01<68:38:29,  8.08s/it]
 86%|████████▋ | 194431/225000 [368:40:06<60:33:17,  7.13s/it]
 86%|████████▋ | 194432/225000 [368:40:13<60:29:34,  7.12s/it]
 86%|████████▋ | 194433/225000 [368:40:20<59:34:52,  7.02s/it]
 86%|████████▋ | 194434/225000 [368:40:33<74:33:27,  8.78s/it]
 86%|████████▋ | 194435/225000 [368:40:41<71:28:31,  8.42s/it]
 86%|████████▋ | 194436/225000 [368:40:49<71:10:25,  8.38s/it]
 86%|████████▋ | 194437/225000 [368:40:56<68:14:59,  8.04s/it]
 86%|████████▋ | 194438/225000 [368:41:04<67:57:39,  8.01s/it]
 86%|████████▋ | 194439/225000 [368:41:09<60:16:50,  7.10s/it]
 86%|████████▋ | 194440/225000 [368:41:21<71:49:08,  8.46s/it]
 86%|████████▋ | 194441/225000 [368:41:27<65:46:06,  7.75s/it]
 86%|████████▋ | 194442/225000 [368:41:34<63:52:52,  7.53s/it]
 86%|████████▋ | 194443/225000 [368:41:49<83:59:10,  9.89s/it]
 86%|████████▋ | 194444/225000 [368:41:54<71:37:34,  8.44s/it]
 86%|████████▋ | 194445/225000 [368:42:04<73:39:46,  8.68s/it]
 86%|████████▋ | 194446/225000 [368:42:10<67:30:41,  7.95s/it]
 86%|████████▋ | 194447/225000 [368:42:19<70:12:26,  8.27s/it]
 86%|████████▋ | 194448/225000 [368:42:30<78:03:17,  9.20s/it]
 86%|████████▋ | 194449/225000 [368:42:36<68:49:35,  8.11s/it]
 86%|████████▋ | 194450/225000 [368:42:42<64:58:46,  7.66s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.195', 'learning_rate': '2.464e-06', 'epoch': '0.8642', 'num_input_tokens_seen': 614919528, 'train_runtime': '1.327e+06', 'train_tokens_per_second': '463.3'}

 86%|████████▋ | 194450/225000 [368:42:42<64:58:46,  7.66s/it]
 86%|████████▋ | 194451/225000 [368:42:53<72:34:20,  8.55s/it]
 86%|████████▋ | 194452/225000 [368:42:59<65:56:11,  7.77s/it]
 86%|████████▋ | 194453/225000 [368:43:06<64:14:45,  7.57s/it]
 86%|████████▋ | 194454/225000 [368:43:14<65:42:29,  7.74s/it]
 86%|████████▋ | 194455/225000 [368:43:20<61:09:43,  7.21s/it]
 86%|████████▋ | 194456/225000 [368:43:32<73:01:00,  8.61s/it]
 86%|████████▋ | 194457/225000 [368:43:38<66:27:22,  7.83s/it]
 86%|████████▋ | 194458/225000 [368:43:46<66:35:18,  7.85s/it]
 86%|████████▋ | 194459/225000 [368:44:00<82:26:54,  9.72s/it]
 86%|████████▋ | 194460/225000 [368:44:09<81:13:40,  9.57s/it]
 86%|████████▋ | 194461/225000 [368:44:14<68:02:42,  8.02s/it]
 86%|████████▋ | 194462/225000 [368:44:23<71:25:58,  8.42s/it]
 86%|████████▋ | 194463/225000 [368:44:30<66:59:25,  7.90s/it]
 86%|████████▋ | 194464/225000 [368:44:40<72:08:29,  8.51s/it]
 86%|████████▋ | 194465/225000 [368:44:51<78:17:46,  9.23s/it]
 86%|████████▋ | 194466/225000 [368:45:01<80:24:45,  9.48s/it]
 86%|████████▋ | 194467/225000 [368:45:09<78:14:18,  9.22s/it]
 86%|████████▋ | 194468/225000 [368:45:17<73:07:50,  8.62s/it]
 86%|████████▋ | 194469/225000 [368:45:22<65:13:59,  7.69s/it]
 86%|████████▋ | 194470/225000 [368:45:31<68:27:43,  8.07s/it]
 86%|████████▋ | 194471/225000 [368:45:36<61:50:03,  7.29s/it]
 86%|████████▋ | 194472/225000 [368:45:45<65:20:07,  7.70s/it]
 86%|████████▋ | 194473/225000 [368:45:52<63:28:46,  7.49s/it]
 86%|████████▋ | 194474/225000 [368:46:03<71:45:09,  8.46s/it]
 86%|████████▋ | 194475/225000 [368:46:09<64:50:56,  7.65s/it]
 86%|████████▋ | 194476/225000 [368:46:16<63:47:29,  7.52s/it]
 86%|████████▋ | 194477/225000 [368:46:25<68:35:16,  8.09s/it]
 86%|████████▋ | 194478/225000 [368:46:41<87:28:47, 10.32s/it]
 86%|████████▋ | 194479/225000 [368:46:47<76:44:27,  9.05s/it]
 86%|████████▋ | 194480/225000 [368:46:54<72:47:49,  8.59s/it]
 86%|████████▋ | 194481/225000 [368:47:03<72:41:05,  8.57s/it]
 86%|████████▋ | 194482/225000 [368:47:13<75:51:49,  8.95s/it]
 86%|████████▋ | 194483/225000 [368:47:27<88:52:48, 10.48s/it]
 86%|████████▋ | 194484/225000 [368:47:34<81:27:31,  9.61s/it]
 86%|████████▋ | 194485/225000 [368:47:41<75:02:09,  8.85s/it]
 86%|████████▋ | 194486/225000 [368:47:47<66:28:17,  7.84s/it]
 86%|████████▋ | 194487/225000 [368:47:52<60:09:22,  7.10s/it]
 86%|████████▋ | 194488/225000 [368:48:05<74:23:39,  8.78s/it]
 86%|████████▋ | 194489/225000 [368:48:11<67:47:15,  8.00s/it]
 86%|████████▋ | 194490/225000 [368:48:21<73:25:06,  8.66s/it]
 86%|████████▋ | 194491/225000 [368:48:28<68:50:52,  8.12s/it]
 86%|████████▋ | 194492/225000 [368:48:37<71:38:04,  8.45s/it]
 86%|████████▋ | 194493/225000 [368:48:42<61:28:39,  7.25s/it]
 86%|████████▋ | 194494/225000 [368:48:49<61:31:22,  7.26s/it]
 86%|████████▋ | 194495/225000 [368:49:01<72:06:45,  8.51s/it]
 86%|████████▋ | 194496/225000 [368:49:08<70:18:31,  8.30s/it]
 86%|████████▋ | 194497/225000 [368:49:16<69:06:32,  8.16s/it]
 86%|████████▋ | 194498/225000 [368:49:24<68:30:41,  8.09s/it]
 86%|████████▋ | 194499/225000 [368:49:36<78:08:57,  9.22s/it]
 86%|████████▋ | 194500/225000 [368:49:43<73:02:48,  8.62s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.125', 'learning_rate': '2.456e-06', 'epoch': '0.8644', 'num_input_tokens_seen': 615145608, 'train_runtime': '1.328e+06', 'train_tokens_per_second': '463.3'}

 86%|████████▋ | 194500/225000 [368:49:43<73:02:48,  8.62s/it]
 86%|████████▋ | 194501/225000 [368:49:49<65:33:13,  7.74s/it]
 86%|████████▋ | 194502/225000 [368:49:55<60:52:06,  7.18s/it]
 86%|████████▋ | 194503/225000 [368:50:07<74:20:25,  8.78s/it]
 86%|████████▋ | 194504/225000 [368:50:18<78:17:14,  9.24s/it]
 86%|████████▋ | 194505/225000 [368:50:22<66:58:38,  7.91s/it]
 86%|████████▋ | 194506/225000 [368:50:28<60:51:19,  7.18s/it]
 86%|████████▋ | 194507/225000 [368:50:42<78:21:37,  9.25s/it]
 86%|████████▋ | 194508/225000 [368:50:50<74:29:42,  8.80s/it]
 86%|████████▋ | 194509/225000 [368:50:54<64:09:37,  7.58s/it]
 86%|████████▋ | 194510/225000 [368:51:02<64:12:28,  7.58s/it]
 86%|████████▋ | 194511/225000 [368:51:09<63:18:52,  7.48s/it]
 86%|████████▋ | 194512/225000 [368:51:16<61:21:23,  7.24s/it]
 86%|████████▋ | 194513/225000 [368:51:30<78:53:36,  9.32s/it]
 86%|████████▋ | 194514/225000 [368:51:39<77:10:55,  9.11s/it]
 86%|████████▋ | 194515/225000 [368:51:46<72:09:48,  8.52s/it]
 86%|████████▋ | 194516/225000 [368:51:54<71:59:37,  8.50s/it]
 86%|████████▋ | 194517/225000 [368:52:00<65:08:27,  7.69s/it]
 86%|████████▋ | 194518/225000 [368:52:10<69:45:05,  8.24s/it]
 86%|████████▋ | 194519/225000 [368:52:18<69:23:45,  8.20s/it]
 86%|████████▋ | 194520/225000 [368:52:28<73:27:30,  8.68s/it]
 86%|████████▋ | 194521/225000 [368:52:33<65:20:00,  7.72s/it]
 86%|████████▋ | 194522/225000 [368:52:42<67:51:19,  8.01s/it]
 86%|████████▋ | 194523/225000 [368:52:50<67:44:27,  8.00s/it]
 86%|████████▋ | 194524/225000 [368:52:59<71:58:34,  8.50s/it]
 86%|████████▋ | 194525/225000 [368:53:09<74:16:01,  8.77s/it]
 86%|████████▋ | 194526/225000 [368:53:16<70:10:23,  8.29s/it]
 86%|████████▋ | 194527/225000 [368:53:24<69:32:34,  8.22s/it]
 86%|████████▋ | 194528/225000 [368:53:34<75:02:39,  8.87s/it]
 86%|████████▋ | 194529/225000 [368:53:41<69:29:45,  8.21s/it]
 86%|████████▋ | 194530/225000 [368:53:49<68:59:21,  8.15s/it]
 86%|████████▋ | 194531/225000 [368:53:55<62:04:14,  7.33s/it]
 86%|████████▋ | 194532/225000 [368:54:01<58:36:44,  6.93s/it]
 86%|████████▋ | 194533/225000 [368:54:10<64:19:04,  7.60s/it]
 86%|████████▋ | 194534/225000 [368:54:16<61:36:05,  7.28s/it]
 86%|████████▋ | 194535/225000 [368:54:24<61:36:51,  7.28s/it]
 86%|████████▋ | 194536/225000 [368:54:31<62:46:02,  7.42s/it]
 86%|████████▋ | 194537/225000 [368:54:43<74:25:21,  8.79s/it]
 86%|████████▋ | 194538/225000 [368:54:53<76:54:21,  9.09s/it]
 86%|████████▋ | 194539/225000 [368:54:59<68:13:31,  8.06s/it]
 86%|████████▋ | 194540/225000 [368:55:09<73:25:48,  8.68s/it]
 86%|████████▋ | 194541/225000 [368:55:15<68:02:14,  8.04s/it]
 86%|████████▋ | 194542/225000 [368:55:23<65:45:44,  7.77s/it]
 86%|████████▋ | 194543/225000 [368:55:31<67:00:19,  7.92s/it]
 86%|████████▋ | 194544/225000 [368:55:40<70:21:47,  8.32s/it]
 86%|████████▋ | 194545/225000 [368:55:46<64:59:34,  7.68s/it]
 86%|████████▋ | 194546/225000 [368:55:56<70:07:52,  8.29s/it]
 86%|████████▋ | 194547/225000 [368:56:06<73:22:30,  8.67s/it]
 86%|████████▋ | 194548/225000 [368:56:16<76:54:19,  9.09s/it]
 86%|████████▋ | 194549/225000 [368:56:25<77:15:52,  9.13s/it]
 86%|████████▋ | 194550/225000 [368:56:31<69:06:11,  8.17s/it]
                                                              
{'loss': '1.282', 'grad_norm': '1.356', 'learning_rate': '2.449e-06', 'epoch': '0.8647', 'num_input_tokens_seen': 615359424, 'train_runtime': '1.328e+06', 'train_tokens_per_second': '463.3'}

 86%|████████▋ | 194550/225000 [368:56:31<69:06:11,  8.17s/it]
 86%|████████▋ | 194551/225000 [368:56:38<65:37:11,  7.76s/it]
 86%|████████▋ | 194552/225000 [368:56:53<84:00:27,  9.93s/it]
 86%|████████▋ | 194553/225000 [368:57:02<81:45:45,  9.67s/it]
 86%|████████▋ | 194554/225000 [368:57:08<72:54:35,  8.62s/it]
 86%|████████▋ | 194555/225000 [368:57:15<68:39:35,  8.12s/it]
 86%|████████▋ | 194556/225000 [368:57:27<80:26:16,  9.51s/it]
 86%|████████▋ | 194557/225000 [368:57:35<75:10:17,  8.89s/it]
 86%|████████▋ | 194558/225000 [368:57:42<70:59:55,  8.40s/it]
 86%|████████▋ | 194559/225000 [368:57:48<64:35:11,  7.64s/it]
 86%|████████▋ | 194560/225000 [368:58:04<85:27:52, 10.11s/it]
 86%|████████▋ | 194561/225000 [368:58:15<87:03:19, 10.30s/it]
 86%|████████▋ | 194562/225000 [368:58:19<71:58:10,  8.51s/it]
 86%|████████▋ | 194563/225000 [368:58:30<78:51:12,  9.33s/it]
 86%|████████▋ | 194564/225000 [368:58:37<72:14:28,  8.54s/it]
 86%|████████▋ | 194565/225000 [368:58:46<74:31:11,  8.81s/it]
 86%|████████▋ | 194566/225000 [368:58:54<72:23:18,  8.56s/it]
 86%|████████▋ | 194567/225000 [368:59:03<72:58:16,  8.63s/it]
 86%|████████▋ | 194568/225000 [368:59:11<70:57:20,  8.39s/it]
 86%|████████▋ | 194569/225000 [368:59:20<71:46:53,  8.49s/it]
 86%|████████▋ | 194570/225000 [368:59:26<66:04:13,  7.82s/it]
 86%|████████▋ | 194571/225000 [368:59:36<72:45:07,  8.61s/it]
 86%|████████▋ | 194572/225000 [368:59:41<62:40:10,  7.41s/it]
 86%|████████▋ | 194573/225000 [368:59:52<71:13:36,  8.43s/it]
 86%|████████▋ | 194574/225000 [369:00:01<72:55:17,  8.63s/it]
 86%|████████▋ | 194575/225000 [369:00:09<71:04:09,  8.41s/it]
 86%|████████▋ | 194576/225000 [369:00:17<70:24:11,  8.33s/it]
 86%|████████▋ | 194577/225000 [369:00:21<58:29:16,  6.92s/it]
 86%|████████▋ | 194578/225000 [369:00:34<74:23:40,  8.80s/it]
 86%|████████▋ | 194579/225000 [369:00:41<70:00:45,  8.29s/it]
 86%|████████▋ | 194580/225000 [369:00:46<61:26:54,  7.27s/it]
 86%|████████▋ | 194581/225000 [369:00:57<70:46:55,  8.38s/it]
 86%|████████▋ | 194582/225000 [369:01:07<76:29:14,  9.05s/it]
 86%|████████▋ | 194583/225000 [369:01:16<75:17:58,  8.91s/it]
 86%|████████▋ | 194584/225000 [369:01:22<68:40:05,  8.13s/it]
 86%|████████▋ | 194585/225000 [369:01:32<72:10:17,  8.54s/it]
 86%|████████▋ | 194586/225000 [369:01:42<77:07:16,  9.13s/it]
 86%|████████▋ | 194587/225000 [369:01:49<70:32:06,  8.35s/it]
 86%|████████▋ | 194588/225000 [369:01:59<74:19:39,  8.80s/it]
 86%|████████▋ | 194589/225000 [369:02:06<70:08:57,  8.30s/it]
 86%|████████▋ | 194590/225000 [369:02:14<70:01:02,  8.29s/it]
 86%|████████▋ | 194591/225000 [369:02:23<71:17:43,  8.44s/it]
 86%|████████▋ | 194592/225000 [369:02:30<68:21:18,  8.09s/it]
 86%|████████▋ | 194593/225000 [369:02:34<57:48:27,  6.84s/it]
 86%|████████▋ | 194594/225000 [369:02:40<55:00:58,  6.51s/it]
 86%|████████▋ | 194595/225000 [369:02:47<57:19:22,  6.79s/it]
 86%|████████▋ | 194596/225000 [369:02:57<66:10:00,  7.83s/it]
 86%|████████▋ | 194597/225000 [369:03:04<62:09:17,  7.36s/it]
 86%|████████▋ | 194598/225000 [369:03:14<69:12:18,  8.19s/it]
 86%|████████▋ | 194599/225000 [369:03:22<68:03:40,  8.06s/it]
 86%|████████▋ | 194600/225000 [369:03:28<65:04:14,  7.71s/it]
                                                              
{'loss': '1.269', 'grad_norm': '1.25', 'learning_rate': '2.441e-06', 'epoch': '0.8649', 'num_input_tokens_seen': 615574752, 'train_runtime': '1.329e+06', 'train_tokens_per_second': '463.3'}

 86%|████████▋ | 194600/225000 [369:03:28<65:04:14,  7.71s/it]
 86%|████████▋ | 194601/225000 [369:03:35<61:28:49,  7.28s/it]
 86%|████████▋ | 194602/225000 [369:03:41<59:07:03,  7.00s/it]
 86%|████████▋ | 194603/225000 [369:03:47<55:35:23,  6.58s/it]
 86%|████████▋ | 194604/225000 [369:03:59<70:37:09,  8.36s/it]
 86%|████████▋ | 194605/225000 [369:04:07<68:19:36,  8.09s/it]
 86%|████████▋ | 194606/225000 [369:04:12<62:07:43,  7.36s/it]
 86%|████████▋ | 194607/225000 [369:04:25<75:37:09,  8.96s/it]
 86%|████████▋ | 194608/225000 [369:04:29<64:09:41,  7.60s/it]
 86%|████████▋ | 194609/225000 [369:04:40<71:23:26,  8.46s/it]
 86%|████████▋ | 194610/225000 [369:04:47<66:58:25,  7.93s/it]
 86%|████████▋ | 194611/225000 [369:04:55<67:57:46,  8.05s/it]
 86%|████████▋ | 194612/225000 [369:05:02<64:59:27,  7.70s/it]
 86%|████████▋ | 194613/225000 [369:05:12<70:32:40,  8.36s/it]
 86%|████████▋ | 194614/225000 [369:05:21<73:30:22,  8.71s/it]
 86%|████████▋ | 194615/225000 [369:05:30<72:56:43,  8.64s/it]
 86%|████████▋ | 194616/225000 [369:05:34<62:50:29,  7.45s/it]
 86%|████████▋ | 194617/225000 [369:05:45<71:46:32,  8.50s/it]
 86%|████████▋ | 194618/225000 [369:05:52<65:50:31,  7.80s/it]
 86%|████████▋ | 194619/225000 [369:05:58<62:32:46,  7.41s/it]
 86%|████████▋ | 194620/225000 [369:06:14<83:34:26,  9.90s/it]
 86%|████████▋ | 194621/225000 [369:06:21<75:35:47,  8.96s/it]
 86%|████████▋ | 194622/225000 [369:06:30<76:37:46,  9.08s/it]
 86%|████████▋ | 194623/225000 [369:06:38<75:23:51,  8.94s/it]
 86%|████████▋ | 194624/225000 [369:06:49<79:39:52,  9.44s/it]
 86%|████████▋ | 194625/225000 [369:07:00<83:45:12,  9.93s/it]
 87%|████████▋ | 194626/225000 [369:07:05<72:06:51,  8.55s/it]
 87%|████████▋ | 194627/225000 [369:07:14<72:52:12,  8.64s/it]
 87%|████████▋ | 194628/225000 [369:07:24<75:37:43,  8.96s/it]
 87%|████████▋ | 194629/225000 [369:07:30<68:15:20,  8.09s/it]
 87%|████████▋ | 194630/225000 [369:07:37<64:50:20,  7.69s/it]
 87%|████████▋ | 194631/225000 [369:07:43<60:03:17,  7.12s/it]
 87%|████████▋ | 194632/225000 [369:07:52<65:04:10,  7.71s/it]
 87%|████████▋ | 194633/225000 [369:08:02<71:23:08,  8.46s/it]
 87%|████████▋ | 194634/225000 [369:08:07<61:41:30,  7.31s/it]
 87%|████████▋ | 194635/225000 [369:08:16<66:45:38,  7.91s/it]
 87%|████████▋ | 194636/225000 [369:08:26<71:03:44,  8.43s/it]
 87%|████████▋ | 194637/225000 [369:08:31<62:50:58,  7.45s/it]
 87%|████████▋ | 194638/225000 [369:08:35<55:26:11,  6.57s/it]
 87%|████████▋ | 194639/225000 [369:08:47<67:35:01,  8.01s/it]
 87%|████████▋ | 194640/225000 [369:08:52<60:59:54,  7.23s/it]
 87%|████████▋ | 194641/225000 [369:08:59<59:51:36,  7.10s/it]
 87%|████████▋ | 194642/225000 [369:09:04<55:19:35,  6.56s/it]
 87%|████████▋ | 194643/225000 [369:09:11<55:38:07,  6.60s/it]
 87%|████████▋ | 194644/225000 [369:09:22<66:16:50,  7.86s/it]
 87%|████████▋ | 194645/225000 [369:09:27<59:08:19,  7.01s/it]
 87%|████████▋ | 194646/225000 [369:09:35<62:29:25,  7.41s/it]
 87%|████████▋ | 194647/225000 [369:09:41<58:40:48,  6.96s/it]
 87%|████████▋ | 194648/225000 [369:09:49<60:39:30,  7.19s/it]
 87%|████████▋ | 194649/225000 [369:09:59<67:41:43,  8.03s/it]
 87%|████████▋ | 194650/225000 [369:10:04<62:11:28,  7.38s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.158', 'learning_rate': '2.433e-06', 'epoch': '0.8651', 'num_input_tokens_seen': 615791152, 'train_runtime': '1.329e+06', 'train_tokens_per_second': '463.3'}

 87%|████████▋ | 194650/225000 [369:10:04<62:11:28,  7.38s/it]
 87%|████████▋ | 194651/225000 [369:10:10<57:26:52,  6.81s/it]
 87%|████████▋ | 194652/225000 [369:10:26<79:50:31,  9.47s/it]
 87%|████████▋ | 194653/225000 [369:10:33<74:17:34,  8.81s/it]
 87%|████████▋ | 194654/225000 [369:10:40<68:40:31,  8.15s/it]
 87%|████████▋ | 194655/225000 [369:10:47<67:04:48,  7.96s/it]
 87%|████████▋ | 194656/225000 [369:11:04<90:20:38, 10.72s/it]
 87%|████████▋ | 194657/225000 [369:11:12<81:57:33,  9.72s/it]
 87%|████████▋ | 194658/225000 [369:11:22<82:35:26,  9.80s/it]
 87%|████████▋ | 194659/225000 [369:11:33<87:52:14, 10.43s/it]
 87%|████████▋ | 194660/225000 [369:11:38<73:46:16,  8.75s/it]
 87%|████████▋ | 194661/225000 [369:11:49<78:39:00,  9.33s/it]
 87%|████████▋ | 194662/225000 [369:11:58<79:04:19,  9.38s/it]
 87%|████████▋ | 194663/225000 [369:12:05<71:58:58,  8.54s/it]
 87%|████████▋ | 194664/225000 [369:12:14<73:07:49,  8.68s/it]
 87%|████████▋ | 194665/225000 [369:12:22<71:45:47,  8.52s/it]
 87%|████████▋ | 194666/225000 [369:12:29<66:24:21,  7.88s/it]
 87%|████████▋ | 194667/225000 [369:12:36<65:45:06,  7.80s/it]
 87%|████████▋ | 194668/225000 [369:12:46<71:30:13,  8.49s/it]
 87%|████████▋ | 194669/225000 [369:12:53<67:55:42,  8.06s/it]
 87%|████████▋ | 194670/225000 [369:13:03<72:20:33,  8.59s/it]
 87%|████████▋ | 194671/225000 [369:13:10<67:14:53,  7.98s/it]
 87%|████████▋ | 194672/225000 [369:13:18<67:22:46,  8.00s/it]
 87%|████████▋ | 194673/225000 [369:13:27<69:54:31,  8.30s/it]
 87%|████████▋ | 194674/225000 [369:13:33<65:35:54,  7.79s/it]
 87%|████████▋ | 194675/225000 [369:13:40<63:09:28,  7.50s/it]
 87%|████████▋ | 194676/225000 [369:13:50<69:37:51,  8.27s/it]
 87%|████████▋ | 194677/225000 [369:13:58<69:19:24,  8.23s/it]
 87%|████████▋ | 194678/225000 [369:14:05<65:30:29,  7.78s/it]
 87%|████████▋ | 194679/225000 [369:14:12<63:53:21,  7.59s/it]
 87%|████████▋ | 194680/225000 [369:14:21<66:06:50,  7.85s/it]
 87%|████████▋ | 194681/225000 [369:14:28<65:43:12,  7.80s/it]
 87%|████████▋ | 194682/225000 [369:14:40<75:09:27,  8.92s/it]
 87%|████████▋ | 194683/225000 [369:14:48<72:39:39,  8.63s/it]
 87%|████████▋ | 194684/225000 [369:14:56<70:28:17,  8.37s/it]
 87%|████████▋ | 194685/225000 [369:15:05<73:16:55,  8.70s/it]
 87%|████████▋ | 194686/225000 [369:15:11<65:40:05,  7.80s/it]
 87%|████████▋ | 194687/225000 [369:15:24<80:14:26,  9.53s/it]
 87%|████████▋ | 194688/225000 [369:15:31<71:33:57,  8.50s/it]
 87%|████████▋ | 194689/225000 [369:15:36<64:05:36,  7.61s/it]
 87%|████████▋ | 194690/225000 [369:15:47<71:25:10,  8.48s/it]
 87%|████████▋ | 194691/225000 [369:15:52<63:07:29,  7.50s/it]
 87%|████████▋ | 194692/225000 [369:16:05<78:27:50,  9.32s/it]
 87%|████████▋ | 194693/225000 [369:16:13<73:46:01,  8.76s/it]
 87%|████████▋ | 194694/225000 [369:16:18<65:56:33,  7.83s/it]
 87%|████████▋ | 194695/225000 [369:16:26<65:01:40,  7.72s/it]
 87%|████████▋ | 194696/225000 [369:16:41<83:57:51,  9.97s/it]
 87%|████████▋ | 194697/225000 [369:16:50<82:11:02,  9.76s/it]
 87%|████████▋ | 194698/225000 [369:16:56<71:21:07,  8.48s/it]
 87%|████████▋ | 194699/225000 [369:17:05<73:40:15,  8.75s/it]
 87%|████████▋ | 194700/225000 [369:17:14<73:28:22,  8.73s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.171', 'learning_rate': '2.425e-06', 'epoch': '0.8653', 'num_input_tokens_seen': 616011712, 'train_runtime': '1.329e+06', 'train_tokens_per_second': '463.4'}

 87%|████████▋ | 194700/225000 [369:17:14<73:28:22,  8.73s/it]
 87%|████████▋ | 194701/225000 [369:17:20<65:26:40,  7.78s/it]
 87%|████████▋ | 194702/225000 [369:17:29<68:30:03,  8.14s/it]
 87%|████████▋ | 194703/225000 [369:17:35<63:22:59,  7.53s/it]
 87%|████████▋ | 194704/225000 [369:17:44<69:00:23,  8.20s/it]
 87%|████████▋ | 194705/225000 [369:17:54<73:12:03,  8.70s/it]
 87%|████████▋ | 194706/225000 [369:18:02<71:20:19,  8.48s/it]
 87%|████████▋ | 194707/225000 [369:18:09<66:47:38,  7.94s/it]
 87%|████████▋ | 194708/225000 [369:18:24<84:51:36, 10.09s/it]
 87%|████████▋ | 194709/225000 [369:18:30<75:14:47,  8.94s/it]
 87%|████████▋ | 194710/225000 [369:18:44<86:59:45, 10.34s/it]
 87%|████████▋ | 194711/225000 [369:18:53<84:32:18, 10.05s/it]
 87%|████████▋ | 194712/225000 [369:19:01<78:08:41,  9.29s/it]
 87%|████████▋ | 194713/225000 [369:19:10<77:38:35,  9.23s/it]
 87%|████████▋ | 194714/225000 [369:19:18<76:11:11,  9.06s/it]
 87%|████████▋ | 194715/225000 [369:19:25<69:01:48,  8.21s/it]
 87%|████████▋ | 194716/225000 [369:19:37<78:34:50,  9.34s/it]
 87%|████████▋ | 194717/225000 [369:19:43<70:29:56,  8.38s/it]
 87%|████████▋ | 194718/225000 [369:19:51<70:59:55,  8.44s/it]
 87%|████████▋ | 194719/225000 [369:20:00<72:31:59,  8.62s/it]
 87%|████████▋ | 194720/225000 [369:20:07<67:34:54,  8.03s/it]
 87%|████████▋ | 194721/225000 [369:20:13<61:02:32,  7.26s/it]
 87%|████████▋ | 194722/225000 [369:20:20<60:45:21,  7.22s/it]
 87%|████████▋ | 194723/225000 [369:20:35<79:57:23,  9.51s/it]
 87%|████████▋ | 194724/225000 [369:20:41<73:11:52,  8.70s/it]
 87%|████████▋ | 194725/225000 [369:20:53<80:50:40,  9.61s/it]
 87%|████████▋ | 194726/225000 [369:20:58<69:36:51,  8.28s/it]
 87%|████████▋ | 194727/225000 [369:21:08<73:06:26,  8.69s/it]
 87%|████████▋ | 194728/225000 [369:21:13<64:08:46,  7.63s/it]
 87%|████████▋ | 194729/225000 [369:21:20<62:19:32,  7.41s/it]
 87%|████████▋ | 194730/225000 [369:21:31<71:19:14,  8.48s/it]
 87%|████████▋ | 194731/225000 [369:21:36<61:30:58,  7.32s/it]
 87%|████████▋ | 194732/225000 [369:21:48<75:07:44,  8.94s/it]
 87%|████████▋ | 194733/225000 [369:21:56<71:46:15,  8.54s/it]
 87%|████████▋ | 194734/225000 [369:22:06<75:13:19,  8.95s/it]
 87%|████████▋ | 194735/225000 [369:22:14<72:37:13,  8.64s/it]
 87%|████████▋ | 194736/225000 [369:22:20<66:27:36,  7.91s/it]
 87%|████████▋ | 194737/225000 [369:22:29<70:18:00,  8.36s/it]
 87%|████████▋ | 194738/225000 [369:22:34<61:47:38,  7.35s/it]
 87%|████████▋ | 194739/225000 [369:22:45<69:34:21,  8.28s/it]
 87%|████████▋ | 194740/225000 [369:22:51<65:21:38,  7.78s/it]
 87%|████████▋ | 194741/225000 [369:23:00<68:15:59,  8.12s/it]
 87%|████████▋ | 194742/225000 [369:23:06<63:11:44,  7.52s/it]
 87%|████████▋ | 194743/225000 [369:23:13<61:33:14,  7.32s/it]
 87%|████████▋ | 194744/225000 [369:23:21<63:16:24,  7.53s/it]
 87%|████████▋ | 194745/225000 [369:23:31<68:50:50,  8.19s/it]
 87%|████████▋ | 194746/225000 [369:23:48<90:18:16, 10.75s/it]
 87%|████████▋ | 194747/225000 [369:23:59<92:03:25, 10.95s/it]
 87%|████████▋ | 194748/225000 [369:24:05<78:30:02,  9.34s/it]
 87%|████████▋ | 194749/225000 [369:24:12<73:12:02,  8.71s/it]
 87%|████████▋ | 194750/225000 [369:24:20<71:24:05,  8.50s/it]
                                                              
{'loss': '1.271', 'grad_norm': '1.285', 'learning_rate': '2.417e-06', 'epoch': '0.8656', 'num_input_tokens_seen': 616236112, 'train_runtime': '1.33e+06', 'train_tokens_per_second': '463.4'}

 87%|████████▋ | 194750/225000 [369:24:20<71:24:05,  8.50s/it]
 87%|████████▋ | 194751/225000 [369:24:26<64:39:46,  7.70s/it]
 87%|████████▋ | 194752/225000 [369:24:41<82:51:52,  9.86s/it]
 87%|████████▋ | 194753/225000 [369:24:47<73:26:09,  8.74s/it]
 87%|████████▋ | 194754/225000 [369:24:53<67:56:44,  8.09s/it]
 87%|████████▋ | 194755/225000 [369:25:05<76:44:33,  9.13s/it]
 87%|████████▋ | 194756/225000 [369:25:11<69:53:42,  8.32s/it]
 87%|████████▋ | 194757/225000 [369:25:20<69:47:33,  8.31s/it]
 87%|████████▋ | 194758/225000 [369:25:31<77:42:11,  9.25s/it]
 87%|████████▋ | 194759/225000 [369:25:40<77:33:06,  9.23s/it]
 87%|████████▋ | 194760/225000 [369:25:50<79:32:23,  9.47s/it]
 87%|████████▋ | 194761/225000 [369:25:55<66:32:14,  7.92s/it]
 87%|████████▋ | 194762/225000 [369:26:03<68:41:30,  8.18s/it]
 87%|████████▋ | 194763/225000 [369:26:12<68:27:02,  8.15s/it]
 87%|████████▋ | 194764/225000 [369:26:18<65:17:21,  7.77s/it]
 87%|████████▋ | 194765/225000 [369:26:31<76:19:54,  9.09s/it]
 87%|████████▋ | 194766/225000 [369:26:40<77:40:42,  9.25s/it]
 87%|████████▋ | 194767/225000 [369:26:50<77:52:31,  9.27s/it]
 87%|████████▋ | 194768/225000 [369:26:55<67:44:59,  8.07s/it]
 87%|████████▋ | 194769/225000 [369:27:06<74:38:40,  8.89s/it]
 87%|████████▋ | 194770/225000 [369:27:15<75:59:54,  9.05s/it]
 87%|████████▋ | 194771/225000 [369:27:22<70:06:41,  8.35s/it]
 87%|████████▋ | 194772/225000 [369:27:27<61:36:29,  7.34s/it]
 87%|████████▋ | 194773/225000 [369:27:39<73:14:25,  8.72s/it]
 87%|████████▋ | 194774/225000 [369:27:47<72:57:24,  8.69s/it]
 87%|████████▋ | 194775/225000 [369:27:54<67:30:56,  8.04s/it]
 87%|████████▋ | 194776/225000 [369:28:06<77:13:10,  9.20s/it]
 87%|████████▋ | 194777/225000 [369:28:15<76:33:41,  9.12s/it]
 87%|████████▋ | 194778/225000 [369:28:21<69:09:03,  8.24s/it]
 87%|████████▋ | 194779/225000 [369:28:30<72:31:49,  8.64s/it]
 87%|████████▋ | 194780/225000 [369:28:40<74:59:58,  8.93s/it]
 87%|████████▋ | 194781/225000 [369:28:48<71:54:57,  8.57s/it]
 87%|████████▋ | 194782/225000 [369:28:55<68:07:53,  8.12s/it]
 87%|████████▋ | 194783/225000 [369:29:03<67:26:00,  8.03s/it]
 87%|████████▋ | 194784/225000 [369:29:14<74:50:54,  8.92s/it]
 87%|████████▋ | 194785/225000 [369:29:23<74:58:42,  8.93s/it]
 87%|████████▋ | 194786/225000 [369:29:32<75:13:24,  8.96s/it]
 87%|████████▋ | 194787/225000 [369:29:37<65:17:03,  7.78s/it]
 87%|████████▋ | 194788/225000 [369:29:48<73:30:50,  8.76s/it]
 87%|████████▋ | 194789/225000 [369:29:59<79:54:53,  9.52s/it]
 87%|████████▋ | 194790/225000 [369:30:08<79:12:04,  9.44s/it]
 87%|████████▋ | 194791/225000 [369:30:18<80:14:26,  9.56s/it]
 87%|████████▋ | 194792/225000 [369:30:26<75:12:10,  8.96s/it]
 87%|████████▋ | 194793/225000 [369:30:35<75:40:32,  9.02s/it]
 87%|████████▋ | 194794/225000 [369:30:43<72:38:44,  8.66s/it]
 87%|████████▋ | 194795/225000 [369:30:51<71:37:15,  8.54s/it]
 87%|████████▋ | 194796/225000 [369:31:01<75:00:41,  8.94s/it]
 87%|████████▋ | 194797/225000 [369:31:07<67:51:05,  8.09s/it]
 87%|████████▋ | 194798/225000 [369:31:14<65:25:02,  7.80s/it]
 87%|████████▋ | 194799/225000 [369:31:21<64:30:39,  7.69s/it]
 87%|████████▋ | 194800/225000 [369:31:30<67:04:13,  8.00s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.404', 'learning_rate': '2.409e-06', 'epoch': '0.8658', 'num_input_tokens_seen': 616461464, 'train_runtime': '1.33e+06', 'train_tokens_per_second': '463.4'}

 87%|████████▋ | 194800/225000 [369:31:30<67:04:13,  8.00s/it]
 87%|████████▋ | 194801/225000 [369:31:40<71:04:29,  8.47s/it]
 87%|████████▋ | 194802/225000 [369:31:48<69:31:18,  8.29s/it]
 87%|████████▋ | 194803/225000 [369:31:55<66:53:03,  7.97s/it]
 87%|████████▋ | 194804/225000 [369:32:01<62:32:36,  7.46s/it]
 87%|████████▋ | 194805/225000 [369:32:08<61:39:10,  7.35s/it]
 87%|████████▋ | 194806/225000 [369:32:15<61:11:10,  7.30s/it]
 87%|████████▋ | 194807/225000 [369:32:20<55:52:52,  6.66s/it]
 87%|████████▋ | 194808/225000 [369:32:31<65:20:54,  7.79s/it]
 87%|████████▋ | 194809/225000 [369:32:42<74:22:52,  8.87s/it]
 87%|████████▋ | 194810/225000 [369:32:50<70:36:56,  8.42s/it]
 87%|████████▋ | 194811/225000 [369:32:59<73:58:20,  8.82s/it]
 87%|████████▋ | 194812/225000 [369:33:05<64:46:57,  7.73s/it]
 87%|████████▋ | 194813/225000 [369:33:17<76:35:47,  9.13s/it]
 87%|████████▋ | 194814/225000 [369:33:28<82:19:44,  9.82s/it]
 87%|████████▋ | 194815/225000 [369:33:39<83:09:18,  9.92s/it]
 87%|████████▋ | 194816/225000 [369:33:44<70:45:50,  8.44s/it]
 87%|████████▋ | 194817/225000 [369:33:51<69:05:35,  8.24s/it]
 87%|████████▋ | 194818/225000 [369:33:57<62:08:59,  7.41s/it]
 87%|████████▋ | 194819/225000 [369:34:05<62:51:16,  7.50s/it]
 87%|████████▋ | 194820/225000 [369:34:13<65:39:03,  7.83s/it]
 87%|████████▋ | 194821/225000 [369:34:26<79:10:27,  9.44s/it]
 87%|████████▋ | 194822/225000 [369:34:35<77:37:37,  9.26s/it]
 87%|████████▋ | 194823/225000 [369:34:40<66:11:26,  7.90s/it]
 87%|████████▋ | 194824/225000 [369:34:51<74:35:26,  8.90s/it]
 87%|████████▋ | 194825/225000 [369:34:59<70:50:11,  8.45s/it]
 87%|████████▋ | 194826/225000 [369:35:08<72:41:51,  8.67s/it]
 87%|████████▋ | 194827/225000 [369:35:11<59:31:41,  7.10s/it]
 87%|████████▋ | 194828/225000 [369:35:22<68:51:33,  8.22s/it]
 87%|████████▋ | 194829/225000 [369:35:26<58:18:17,  6.96s/it]
 87%|████████▋ | 194830/225000 [369:35:38<70:45:03,  8.44s/it]
 87%|████████▋ | 194831/225000 [369:35:46<70:28:52,  8.41s/it]
 87%|████████▋ | 194832/225000 [369:35:51<60:37:22,  7.23s/it]
 87%|████████▋ | 194833/225000 [369:35:58<61:29:34,  7.34s/it]
 87%|████████▋ | 194834/225000 [369:36:04<58:01:14,  6.92s/it]
 87%|████████▋ | 194835/225000 [369:36:16<69:03:24,  8.24s/it]
 87%|████████▋ | 194836/225000 [369:36:25<70:56:02,  8.47s/it]
 87%|████████▋ | 194837/225000 [369:36:30<62:12:57,  7.43s/it]
 87%|████████▋ | 194838/225000 [369:36:40<68:45:40,  8.21s/it]
 87%|████████▋ | 194839/225000 [369:36:52<79:44:50,  9.52s/it]
 87%|████████▋ | 194840/225000 [369:37:03<83:24:33,  9.96s/it]
 87%|████████▋ | 194841/225000 [369:37:13<83:11:47,  9.93s/it]
 87%|████████▋ | 194842/225000 [369:37:19<72:24:53,  8.64s/it]
 87%|████████▋ | 194843/225000 [369:37:29<75:21:40,  9.00s/it]
 87%|████████▋ | 194844/225000 [369:37:36<71:12:48,  8.50s/it]
 87%|████████▋ | 194845/225000 [369:37:45<73:35:11,  8.78s/it]
 87%|████████▋ | 194846/225000 [369:37:53<70:03:55,  8.36s/it]
 87%|████████▋ | 194847/225000 [369:38:04<77:30:58,  9.25s/it]
 87%|████████▋ | 194848/225000 [369:38:10<68:21:07,  8.16s/it]
 87%|████████▋ | 194849/225000 [369:38:14<59:13:24,  7.07s/it]
 87%|████████▋ | 194850/225000 [369:38:22<59:55:55,  7.16s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.564', 'learning_rate': '2.401e-06', 'epoch': '0.866', 'num_input_tokens_seen': 616679704, 'train_runtime': '1.331e+06', 'train_tokens_per_second': '463.4'}

 87%|████████▋ | 194850/225000 [369:38:22<59:55:55,  7.16s/it]
 87%|████████▋ | 194851/225000 [369:38:32<69:12:40,  8.26s/it]
 87%|████████▋ | 194852/225000 [369:38:41<69:51:38,  8.34s/it]
 87%|████████▋ | 194853/225000 [369:38:52<76:19:18,  9.11s/it]
 87%|████████▋ | 194854/225000 [369:38:56<64:29:41,  7.70s/it]
 87%|████████▋ | 194855/225000 [369:39:06<70:11:12,  8.38s/it]
 87%|████████▋ | 194856/225000 [369:39:19<80:44:19,  9.64s/it]
 87%|████████▋ | 194857/225000 [369:39:25<72:04:36,  8.61s/it]
 87%|████████▋ | 194858/225000 [369:39:30<62:50:48,  7.51s/it]
 87%|████████▋ | 194859/225000 [369:39:44<78:19:35,  9.36s/it]
 87%|████████▋ | 194860/225000 [369:39:50<72:02:04,  8.60s/it]
 87%|████████▋ | 194861/225000 [369:39:57<67:38:51,  8.08s/it]
 87%|████████▋ | 194862/225000 [369:40:03<62:52:44,  7.51s/it]
 87%|████████▋ | 194863/225000 [369:40:16<76:05:12,  9.09s/it]
 87%|████████▋ | 194864/225000 [369:40:24<73:21:32,  8.76s/it]
 87%|████████▋ | 194865/225000 [369:40:32<71:59:49,  8.60s/it]
 87%|████████▋ | 194866/225000 [369:40:43<76:56:20,  9.19s/it]
 87%|████████▋ | 194867/225000 [369:40:50<71:26:54,  8.54s/it]
 87%|████████▋ | 194868/225000 [369:40:56<64:03:37,  7.65s/it]
 87%|████████▋ | 194869/225000 [369:41:07<73:53:20,  8.83s/it]
 87%|████████▋ | 194870/225000 [369:41:16<73:54:17,  8.83s/it]
 87%|████████▋ | 194871/225000 [369:41:30<85:59:29, 10.27s/it]
 87%|████████▋ | 194872/225000 [369:41:38<81:23:55,  9.73s/it]
 87%|████████▋ | 194873/225000 [369:41:48<81:58:28,  9.80s/it]
 87%|████████▋ | 194874/225000 [369:41:57<78:48:22,  9.42s/it]
 87%|████████▋ | 194875/225000 [369:42:10<88:52:30, 10.62s/it]
 87%|████████▋ | 194876/225000 [369:42:17<79:45:22,  9.53s/it]
 87%|████████▋ | 194877/225000 [369:42:25<76:44:13,  9.17s/it]
 87%|████████▋ | 194878/225000 [369:42:33<71:50:34,  8.59s/it]
 87%|████████▋ | 194879/225000 [369:42:46<83:54:24, 10.03s/it]
 87%|████████▋ | 194880/225000 [369:42:53<75:44:33,  9.05s/it]
 87%|████████▋ | 194881/225000 [369:43:03<78:46:31,  9.42s/it]
 87%|████████▋ | 194882/225000 [369:43:09<69:11:11,  8.27s/it]
 87%|████████▋ | 194883/225000 [369:43:18<73:10:55,  8.75s/it]
 87%|████████▋ | 194884/225000 [369:43:27<73:21:02,  8.77s/it]
 87%|████████▋ | 194885/225000 [369:43:32<63:17:29,  7.57s/it]
 87%|████████▋ | 194886/225000 [369:43:38<60:08:47,  7.19s/it]
 87%|████████▋ | 194887/225000 [369:43:44<57:29:01,  6.87s/it]
 87%|████████▋ | 194888/225000 [369:43:55<65:32:17,  7.84s/it]
 87%|████████▋ | 194889/225000 [369:44:10<83:36:45, 10.00s/it]
 87%|████████▋ | 194890/225000 [369:44:22<88:50:24, 10.62s/it]
 87%|████████▋ | 194891/225000 [369:44:31<84:38:32, 10.12s/it]
 87%|████████▋ | 194892/225000 [369:44:36<73:32:19,  8.79s/it]
 87%|████████▋ | 194893/225000 [369:44:42<65:13:40,  7.80s/it]
 87%|████████▋ | 194894/225000 [369:44:55<77:46:42,  9.30s/it]
 87%|████████▋ | 194895/225000 [369:45:07<85:25:16, 10.21s/it]
 87%|████████▋ | 194896/225000 [369:45:17<84:01:10, 10.05s/it]
 87%|████████▋ | 194897/225000 [369:45:26<81:31:04,  9.75s/it]
 87%|████████▋ | 194898/225000 [369:45:34<79:06:33,  9.46s/it]
 87%|████████▋ | 194899/225000 [369:45:44<79:24:16,  9.50s/it]
 87%|████████▋ | 194900/225000 [369:45:52<75:06:22,  8.98s/it]
                                                              
{'loss': '1.266', 'grad_norm': '1.275', 'learning_rate': '2.394e-06', 'epoch': '0.8662', 'num_input_tokens_seen': 616907688, 'train_runtime': '1.331e+06', 'train_tokens_per_second': '463.4'}

 87%|████████▋ | 194900/225000 [369:45:52<75:06:22,  8.98s/it]
 87%|████████▋ | 194901/225000 [369:45:58<67:42:43,  8.10s/it]
 87%|████████▋ | 194902/225000 [369:46:09<75:55:24,  9.08s/it]
 87%|████████▋ | 194903/225000 [369:46:18<74:42:09,  8.94s/it]
 87%|████████▋ | 194904/225000 [369:46:25<71:04:07,  8.50s/it]
 87%|████████▋ | 194905/225000 [369:46:37<77:57:44,  9.33s/it]
 87%|████████▋ | 194906/225000 [369:46:46<77:53:15,  9.32s/it]
 87%|████████▋ | 194907/225000 [369:46:52<69:22:15,  8.30s/it]
 87%|████████▋ | 194908/225000 [369:47:00<68:27:19,  8.19s/it]
 87%|████████▋ | 194909/225000 [369:47:06<63:15:23,  7.57s/it]
 87%|████████▋ | 194910/225000 [369:47:17<73:12:18,  8.76s/it]
 87%|████████▋ | 194911/225000 [369:47:23<66:32:54,  7.96s/it]
 87%|████████▋ | 194912/225000 [369:47:35<75:40:44,  9.05s/it]
 87%|████████▋ | 194913/225000 [369:47:42<69:55:32,  8.37s/it]
 87%|████████▋ | 194914/225000 [369:47:48<63:34:19,  7.61s/it]
 87%|████████▋ | 194915/225000 [369:47:54<61:28:30,  7.36s/it]
 87%|████████▋ | 194916/225000 [369:48:03<65:18:38,  7.82s/it]
 87%|████████▋ | 194917/225000 [369:48:18<82:10:11,  9.83s/it]
 87%|████████▋ | 194918/225000 [369:48:29<84:40:20, 10.13s/it]
 87%|████████▋ | 194919/225000 [369:48:34<73:28:24,  8.79s/it]
 87%|████████▋ | 194920/225000 [369:48:40<66:19:23,  7.94s/it]
 87%|████████▋ | 194921/225000 [369:48:45<58:43:08,  7.03s/it]
 87%|████████▋ | 194922/225000 [369:48:51<56:21:42,  6.75s/it]
 87%|████████▋ | 194923/225000 [369:48:57<53:25:48,  6.40s/it]
 87%|████████▋ | 194924/225000 [369:49:12<75:55:13,  9.09s/it]
 87%|████████▋ | 194925/225000 [369:49:19<69:28:55,  8.32s/it]
 87%|████████▋ | 194926/225000 [369:49:25<64:16:11,  7.69s/it]
 87%|████████▋ | 194927/225000 [369:49:33<66:05:59,  7.91s/it]
 87%|████████▋ | 194928/225000 [369:49:40<63:56:59,  7.66s/it]
 87%|████████▋ | 194929/225000 [369:49:51<71:52:07,  8.60s/it]
 87%|████████▋ | 194930/225000 [369:50:03<79:14:10,  9.49s/it]
 87%|████████▋ | 194931/225000 [369:50:15<85:33:41, 10.24s/it]
 87%|████████▋ | 194932/225000 [369:50:22<77:34:44,  9.29s/it]
 87%|████████▋ | 194933/225000 [369:50:30<75:21:25,  9.02s/it]
 87%|████████▋ | 194934/225000 [369:50:40<78:12:50,  9.37s/it]
 87%|████████▋ | 194935/225000 [369:50:45<66:45:17,  7.99s/it]
 87%|████████▋ | 194936/225000 [369:50:56<73:37:39,  8.82s/it]
 87%|████████▋ | 194937/225000 [369:51:03<69:06:01,  8.27s/it]
 87%|████████▋ | 194938/225000 [369:51:12<71:55:23,  8.61s/it]
 87%|████████▋ | 194939/225000 [369:51:22<74:49:17,  8.96s/it]
 87%|████████▋ | 194940/225000 [369:51:30<72:04:57,  8.63s/it]
 87%|████████▋ | 194941/225000 [369:51:38<69:26:33,  8.32s/it]
 87%|████████▋ | 194942/225000 [369:51:47<71:53:47,  8.61s/it]
 87%|████████▋ | 194943/225000 [369:51:54<68:40:24,  8.23s/it]
 87%|████████▋ | 194944/225000 [369:52:05<74:32:23,  8.93s/it]
 87%|████████▋ | 194945/225000 [369:52:19<87:38:27, 10.50s/it]
 87%|████████▋ | 194946/225000 [369:52:24<73:15:57,  8.78s/it]
 87%|████████▋ | 194947/225000 [369:52:34<75:46:44,  9.08s/it]
 87%|████████▋ | 194948/225000 [369:52:40<69:30:56,  8.33s/it]
 87%|████████▋ | 194949/225000 [369:52:52<78:57:39,  9.46s/it]
 87%|████████▋ | 194950/225000 [369:52:58<70:44:14,  8.47s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.293', 'learning_rate': '2.386e-06', 'epoch': '0.8664', 'num_input_tokens_seen': 617133064, 'train_runtime': '1.332e+06', 'train_tokens_per_second': '463.5'}

 87%|████████▋ | 194950/225000 [369:52:58<70:44:14,  8.47s/it]
 87%|████████▋ | 194951/225000 [369:53:05<65:22:55,  7.83s/it]
 87%|████████▋ | 194952/225000 [369:53:15<70:43:34,  8.47s/it]
 87%|████████▋ | 194953/225000 [369:53:28<83:16:46,  9.98s/it]
 87%|████████▋ | 194954/225000 [369:53:35<75:51:39,  9.09s/it]
 87%|████████▋ | 194955/225000 [369:53:46<79:48:50,  9.56s/it]
 87%|████████▋ | 194956/225000 [369:53:50<65:39:49,  7.87s/it]
 87%|████████▋ | 194957/225000 [369:54:00<72:29:35,  8.69s/it]
 87%|████████▋ | 194958/225000 [369:54:09<72:19:00,  8.67s/it]
 87%|████████▋ | 194959/225000 [369:54:14<64:10:24,  7.69s/it]
 87%|████████▋ | 194960/225000 [369:54:24<70:07:57,  8.40s/it]
 87%|████████▋ | 194961/225000 [369:54:32<68:45:49,  8.24s/it]
 87%|████████▋ | 194962/225000 [369:54:38<63:24:42,  7.60s/it]
 87%|████████▋ | 194963/225000 [369:54:49<71:31:23,  8.57s/it]
 87%|████████▋ | 194964/225000 [369:54:58<72:48:35,  8.73s/it]
 87%|████████▋ | 194965/225000 [369:55:05<68:03:15,  8.16s/it]
 87%|████████▋ | 194966/225000 [369:55:13<66:05:38,  7.92s/it]
 87%|████████▋ | 194967/225000 [369:55:25<76:32:27,  9.17s/it]
 87%|████████▋ | 194968/225000 [369:55:33<74:09:08,  8.89s/it]
 87%|████████▋ | 194969/225000 [369:55:47<86:00:57, 10.31s/it]
 87%|████████▋ | 194970/225000 [369:55:54<78:41:57,  9.43s/it]
 87%|████████▋ | 194971/225000 [369:56:00<70:20:10,  8.43s/it]
 87%|████████▋ | 194972/225000 [369:56:07<67:06:11,  8.04s/it]
 87%|████████▋ | 194973/225000 [369:56:18<74:52:51,  8.98s/it]
 87%|████████▋ | 194974/225000 [369:56:24<65:32:43,  7.86s/it]
 87%|████████▋ | 194975/225000 [369:56:30<60:52:46,  7.30s/it]
 87%|████████▋ | 194976/225000 [369:56:41<70:15:36,  8.42s/it]
 87%|████████▋ | 194977/225000 [369:56:49<71:05:28,  8.52s/it]
 87%|████████▋ | 194978/225000 [369:56:53<60:07:24,  7.21s/it]
 87%|████████▋ | 194979/225000 [369:56:59<56:43:46,  6.80s/it]
 87%|████████▋ | 194980/225000 [369:57:08<62:16:04,  7.47s/it]
 87%|████████▋ | 194981/225000 [369:57:21<74:16:39,  8.91s/it]
 87%|████████▋ | 194982/225000 [369:57:32<79:16:08,  9.51s/it]
 87%|████████▋ | 194983/225000 [369:57:38<72:47:59,  8.73s/it]
 87%|████████▋ | 194984/225000 [369:57:45<66:10:54,  7.94s/it]
 87%|████████▋ | 194985/225000 [369:57:50<60:45:14,  7.29s/it]
 87%|████████▋ | 194986/225000 [369:58:06<80:30:30,  9.66s/it]
 87%|████████▋ | 194987/225000 [369:58:15<79:08:43,  9.49s/it]
 87%|████████▋ | 194988/225000 [369:58:20<67:56:29,  8.15s/it]
 87%|████████▋ | 194989/225000 [369:58:26<64:07:03,  7.69s/it]
 87%|████████▋ | 194990/225000 [369:58:36<68:21:22,  8.20s/it]
 87%|████████▋ | 194991/225000 [369:58:41<60:54:31,  7.31s/it]
 87%|████████▋ | 194992/225000 [369:58:53<73:01:31,  8.76s/it]
 87%|████████▋ | 194993/225000 [369:59:02<73:56:52,  8.87s/it]
 87%|████████▋ | 194994/225000 [369:59:10<70:19:23,  8.44s/it]
 87%|████████▋ | 194995/225000 [369:59:15<63:33:52,  7.63s/it]
 87%|████████▋ | 194996/225000 [369:59:22<62:24:50,  7.49s/it]
 87%|████████▋ | 194997/225000 [369:59:34<71:48:34,  8.62s/it]
 87%|████████▋ | 194998/225000 [369:59:39<63:02:31,  7.56s/it]
 87%|████████▋ | 194999/225000 [369:59:50<71:22:14,  8.56s/it]
 87%|████████▋ | 195000/225000 [369:59:55<64:11:18,  7.70s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.224', 'learning_rate': '2.378e-06', 'epoch': '0.8667', 'num_input_tokens_seen': 617354168, 'train_runtime': '1.332e+06', 'train_tokens_per_second': '463.5'}

 87%|████████▋ | 195000/225000 [369:59:55<64:11:18,  7.70s/it]
 87%|████████▋ | 195001/225000 [370:00:06<72:11:41,  8.66s/it]
 87%|████████▋ | 195002/225000 [370:00:12<63:53:07,  7.67s/it]
 87%|████████▋ | 195003/225000 [370:00:17<56:58:33,  6.84s/it]
 87%|████████▋ | 195004/225000 [370:00:26<63:51:51,  7.66s/it]
 87%|████████▋ | 195005/225000 [370:00:33<61:20:23,  7.36s/it]
 87%|████████▋ | 195006/225000 [370:00:42<66:06:26,  7.93s/it]
 87%|████████▋ | 195007/225000 [370:00:49<63:58:59,  7.68s/it]
 87%|████████▋ | 195008/225000 [370:00:56<61:05:38,  7.33s/it]
 87%|████████▋ | 195009/225000 [370:01:06<68:47:58,  8.26s/it]
 87%|████████▋ | 195010/225000 [370:01:14<68:22:14,  8.21s/it]
 87%|████████▋ | 195011/225000 [370:01:24<72:56:00,  8.76s/it]
 87%|████████▋ | 195012/225000 [370:01:30<64:57:14,  7.80s/it]
 87%|████████▋ | 195013/225000 [370:01:40<71:41:12,  8.61s/it]
 87%|████████▋ | 195014/225000 [370:01:51<77:46:23,  9.34s/it]
 87%|████████▋ | 195015/225000 [370:01:56<65:21:28,  7.85s/it]
 87%|████████▋ | 195016/225000 [370:02:07<74:54:41,  8.99s/it]
 87%|████████▋ | 195017/225000 [370:02:12<64:19:47,  7.72s/it]
 87%|████████▋ | 195018/225000 [370:02:19<62:03:24,  7.45s/it]
 87%|████████▋ | 195019/225000 [370:02:30<71:57:29,  8.64s/it]
 87%|████████▋ | 195020/225000 [370:02:40<75:15:25,  9.04s/it]
 87%|████████▋ | 195021/225000 [370:02:45<65:28:40,  7.86s/it]
 87%|████████▋ | 195022/225000 [370:02:51<59:13:21,  7.11s/it]
 87%|████████▋ | 195023/225000 [370:02:59<62:36:16,  7.52s/it]
 87%|████████▋ | 195024/225000 [370:03:08<66:19:43,  7.97s/it]
 87%|████████▋ | 195025/225000 [370:03:17<67:22:03,  8.09s/it]
 87%|████████▋ | 195026/225000 [370:03:26<71:29:09,  8.59s/it]
 87%|████████▋ | 195027/225000 [370:03:37<77:19:15,  9.29s/it]
 87%|████████▋ | 195028/225000 [370:03:45<72:37:36,  8.72s/it]
 87%|████████▋ | 195029/225000 [370:03:52<68:12:46,  8.19s/it]
 87%|████████▋ | 195030/225000 [370:03:59<65:36:55,  7.88s/it]
 87%|████████▋ | 195031/225000 [370:04:05<61:50:44,  7.43s/it]
 87%|████████▋ | 195032/225000 [370:04:11<58:18:54,  7.01s/it]
 87%|████████▋ | 195033/225000 [370:04:27<81:16:09,  9.76s/it]
 87%|████████▋ | 195034/225000 [370:04:39<86:18:09, 10.37s/it]
 87%|████████▋ | 195035/225000 [370:04:46<77:04:53,  9.26s/it]
 87%|████████▋ | 195036/225000 [370:04:52<70:07:39,  8.43s/it]
 87%|████████▋ | 195037/225000 [370:05:01<71:31:36,  8.59s/it]
 87%|████████▋ | 195038/225000 [370:05:12<77:32:51,  9.32s/it]
 87%|████████▋ | 195039/225000 [370:05:21<76:10:00,  9.15s/it]
 87%|████████▋ | 195040/225000 [370:05:33<82:43:26,  9.94s/it]
 87%|████████▋ | 195041/225000 [370:05:39<73:10:55,  8.79s/it]
 87%|████████▋ | 195042/225000 [370:05:45<66:21:41,  7.97s/it]
 87%|████████▋ | 195043/225000 [370:05:54<67:56:47,  8.17s/it]
 87%|████████▋ | 195044/225000 [370:06:03<69:47:13,  8.39s/it]
 87%|████████▋ | 195045/225000 [370:06:07<61:01:10,  7.33s/it]
 87%|████████▋ | 195046/225000 [370:06:16<65:04:37,  7.82s/it]
 87%|████████▋ | 195047/225000 [370:06:24<63:28:49,  7.63s/it]
 87%|████████▋ | 195048/225000 [370:06:35<73:43:42,  8.86s/it]
 87%|████████▋ | 195049/225000 [370:06:43<70:09:54,  8.43s/it]
 87%|████████▋ | 195050/225000 [370:06:49<64:56:50,  7.81s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.365', 'learning_rate': '2.37e-06', 'epoch': '0.8669', 'num_input_tokens_seen': 617576648, 'train_runtime': '1.332e+06', 'train_tokens_per_second': '463.5'}

 87%|████████▋ | 195050/225000 [370:06:49<64:56:50,  7.81s/it]
 87%|████████▋ | 195051/225000 [370:06:57<65:26:51,  7.87s/it]
 87%|████████▋ | 195052/225000 [370:07:09<75:19:01,  9.05s/it]
 87%|████████▋ | 195053/225000 [370:07:20<79:27:00,  9.55s/it]
 87%|████████▋ | 195054/225000 [370:07:25<68:34:34,  8.24s/it]
 87%|████████▋ | 195055/225000 [370:07:30<59:51:57,  7.20s/it]
 87%|████████▋ | 195056/225000 [370:07:36<56:49:51,  6.83s/it]
 87%|████████▋ | 195057/225000 [370:07:45<62:25:41,  7.51s/it]
 87%|████████▋ | 195058/225000 [370:08:00<81:59:42,  9.86s/it]
 87%|████████▋ | 195059/225000 [370:08:05<69:05:58,  8.31s/it]
 87%|████████▋ | 195060/225000 [370:08:12<65:36:08,  7.89s/it]
 87%|████████▋ | 195061/225000 [370:08:21<70:27:52,  8.47s/it]
 87%|████████▋ | 195062/225000 [370:08:30<70:22:46,  8.46s/it]
 87%|████████▋ | 195063/225000 [370:08:41<77:38:52,  9.34s/it]
 87%|████████▋ | 195064/225000 [370:08:49<74:04:36,  8.91s/it]
 87%|████████▋ | 195065/225000 [370:08:58<73:41:54,  8.86s/it]
 87%|████████▋ | 195066/225000 [370:09:03<63:55:14,  7.69s/it]
 87%|████████▋ | 195067/225000 [370:09:14<73:38:36,  8.86s/it]
 87%|████████▋ | 195068/225000 [370:09:20<65:37:05,  7.89s/it]
 87%|████████▋ | 195069/225000 [370:09:30<71:39:45,  8.62s/it]
 87%|████████▋ | 195070/225000 [370:09:41<76:54:27,  9.25s/it]
 87%|████████▋ | 195071/225000 [370:09:46<66:08:29,  7.96s/it]
 87%|████████▋ | 195072/225000 [370:09:55<69:15:17,  8.33s/it]
 87%|████████▋ | 195073/225000 [370:10:04<70:18:58,  8.46s/it]
 87%|████████▋ | 195074/225000 [370:10:12<68:40:41,  8.26s/it]
 87%|████████▋ | 195075/225000 [370:10:16<57:48:03,  6.95s/it]
 87%|████████▋ | 195076/225000 [370:10:27<68:54:27,  8.29s/it]
 87%|████████▋ | 195077/225000 [370:10:32<61:12:14,  7.36s/it]
 87%|████████▋ | 195078/225000 [370:10:38<56:44:59,  6.83s/it]
 87%|████████▋ | 195079/225000 [370:10:53<77:02:55,  9.27s/it]
 87%|████████▋ | 195080/225000 [370:10:58<67:05:08,  8.07s/it]
 87%|████████▋ | 195081/225000 [370:11:06<67:37:41,  8.14s/it]
 87%|████████▋ | 195082/225000 [370:11:19<78:57:38,  9.50s/it]
 87%|████████▋ | 195083/225000 [370:11:26<71:55:41,  8.66s/it]
 87%|████████▋ | 195084/225000 [370:11:33<68:59:04,  8.30s/it]
 87%|████████▋ | 195085/225000 [370:11:40<64:00:54,  7.70s/it]
 87%|████████▋ | 195086/225000 [370:11:46<61:53:23,  7.45s/it]
 87%|████████▋ | 195087/225000 [370:11:55<64:40:50,  7.78s/it]
 87%|████████▋ | 195088/225000 [370:12:04<67:46:13,  8.16s/it]
 87%|████████▋ | 195089/225000 [370:12:10<62:16:34,  7.50s/it]
 87%|████████▋ | 195090/225000 [370:12:20<68:05:36,  8.20s/it]
 87%|████████▋ | 195091/225000 [370:12:26<63:52:08,  7.69s/it]
 87%|████████▋ | 195092/225000 [370:12:39<75:02:56,  9.03s/it]
 87%|████████▋ | 195093/225000 [370:12:45<67:48:31,  8.16s/it]
 87%|████████▋ | 195094/225000 [370:12:51<63:56:26,  7.70s/it]
 87%|████████▋ | 195095/225000 [370:13:04<75:15:53,  9.06s/it]
 87%|████████▋ | 195096/225000 [370:13:13<77:03:38,  9.28s/it]
 87%|████████▋ | 195097/225000 [370:13:22<75:37:34,  9.10s/it]
 87%|████████▋ | 195098/225000 [370:13:29<70:29:49,  8.49s/it]
 87%|████████▋ | 195099/225000 [370:13:38<70:45:49,  8.52s/it]
 87%|████████▋ | 195100/225000 [370:13:45<68:43:23,  8.27s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.018', 'learning_rate': '2.363e-06', 'epoch': '0.8671', 'num_input_tokens_seen': 617796144, 'train_runtime': '1.333e+06', 'train_tokens_per_second': '463.5'}

 87%|████████▋ | 195100/225000 [370:13:45<68:43:23,  8.27s/it]
 87%|████████▋ | 195101/225000 [370:13:55<72:01:45,  8.67s/it]
 87%|████████▋ | 195102/225000 [370:14:01<65:35:17,  7.90s/it]
 87%|████████▋ | 195103/225000 [370:14:09<64:45:01,  7.80s/it]
 87%|████████▋ | 195104/225000 [370:14:18<69:36:24,  8.38s/it]
 87%|████████▋ | 195105/225000 [370:14:27<70:24:39,  8.48s/it]
 87%|████████▋ | 195106/225000 [370:14:38<76:25:53,  9.20s/it]
 87%|████████▋ | 195107/225000 [370:14:46<73:27:16,  8.85s/it]
 87%|████████▋ | 195108/225000 [370:14:55<74:18:03,  8.95s/it]
 87%|████████▋ | 195109/225000 [370:15:06<77:49:30,  9.37s/it]
 87%|████████▋ | 195110/225000 [370:15:10<65:04:39,  7.84s/it]
 87%|████████▋ | 195111/225000 [370:15:15<59:11:08,  7.13s/it]
 87%|████████▋ | 195112/225000 [370:15:22<57:30:40,  6.93s/it]
 87%|████████▋ | 195113/225000 [370:15:27<53:28:20,  6.44s/it]
 87%|████████▋ | 195114/225000 [370:15:39<66:17:05,  7.98s/it]
 87%|████████▋ | 195115/225000 [370:15:50<74:37:05,  8.99s/it]
 87%|████████▋ | 195116/225000 [370:15:54<62:47:18,  7.56s/it]
 87%|████████▋ | 195117/225000 [370:16:04<69:30:52,  8.37s/it]
 87%|████████▋ | 195118/225000 [370:16:15<75:11:44,  9.06s/it]
 87%|████████▋ | 195119/225000 [370:16:22<68:56:14,  8.31s/it]
 87%|████████▋ | 195120/225000 [370:16:32<73:40:53,  8.88s/it]
 87%|████████▋ | 195121/225000 [370:16:39<69:21:51,  8.36s/it]
 87%|████████▋ | 195122/225000 [370:16:50<77:09:34,  9.30s/it]
 87%|████████▋ | 195123/225000 [370:16:57<69:13:57,  8.34s/it]
 87%|████████▋ | 195124/225000 [370:17:09<79:42:14,  9.60s/it]
 87%|████████▋ | 195125/225000 [370:17:14<67:27:44,  8.13s/it]
 87%|████████▋ | 195126/225000 [370:17:23<69:53:50,  8.42s/it]
 87%|████████▋ | 195127/225000 [370:17:29<65:14:47,  7.86s/it]
 87%|████████▋ | 195128/225000 [370:17:45<84:59:49, 10.24s/it]
 87%|████████▋ | 195129/225000 [370:17:51<72:29:24,  8.74s/it]
 87%|████████▋ | 195130/225000 [370:18:08<93:27:58, 11.26s/it]
 87%|████████▋ | 195131/225000 [370:18:13<79:27:23,  9.58s/it]
 87%|████████▋ | 195132/225000 [370:18:20<72:32:17,  8.74s/it]
 87%|████████▋ | 195133/225000 [370:18:28<71:17:46,  8.59s/it]
 87%|████████▋ | 195134/225000 [370:18:37<71:14:40,  8.59s/it]
 87%|████████▋ | 195135/225000 [370:18:43<64:37:32,  7.79s/it]
 87%|████████▋ | 195136/225000 [370:18:57<80:16:02,  9.68s/it]
 87%|████████▋ | 195137/225000 [370:19:01<66:14:52,  7.99s/it]
 87%|████████▋ | 195138/225000 [370:19:09<67:05:23,  8.09s/it]
 87%|████████▋ | 195139/225000 [370:19:18<68:35:26,  8.27s/it]
 87%|████████▋ | 195140/225000 [370:19:28<73:09:15,  8.82s/it]
 87%|████████▋ | 195141/225000 [370:19:37<74:32:22,  8.99s/it]
 87%|████████▋ | 195142/225000 [370:19:44<69:22:30,  8.36s/it]
 87%|████████▋ | 195143/225000 [370:19:51<64:24:12,  7.77s/it]
 87%|████████▋ | 195144/225000 [370:19:58<62:18:40,  7.51s/it]
 87%|████████▋ | 195145/225000 [370:20:11<77:01:00,  9.29s/it]
 87%|████████▋ | 195146/225000 [370:20:17<67:19:48,  8.12s/it]
 87%|████████▋ | 195147/225000 [370:20:21<58:54:54,  7.10s/it]
 87%|████████▋ | 195148/225000 [370:20:31<65:37:18,  7.91s/it]
 87%|████████▋ | 195149/225000 [370:20:39<66:48:40,  8.06s/it]
 87%|████████▋ | 195150/225000 [370:20:46<63:01:05,  7.60s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.221', 'learning_rate': '2.355e-06', 'epoch': '0.8673', 'num_input_tokens_seen': 618024952, 'train_runtime': '1.333e+06', 'train_tokens_per_second': '463.5'}

 87%|████████▋ | 195150/225000 [370:20:46<63:01:05,  7.60s/it]
 87%|████████▋ | 195151/225000 [370:20:52<59:56:58,  7.23s/it]
 87%|████████▋ | 195152/225000 [370:21:04<71:48:05,  8.66s/it]
 87%|████████▋ | 195153/225000 [370:21:11<65:45:18,  7.93s/it]
 87%|████████▋ | 195154/225000 [370:21:15<56:05:48,  6.77s/it]
 87%|████████▋ | 195155/225000 [370:21:30<78:23:26,  9.46s/it]
 87%|████████▋ | 195156/225000 [370:21:39<75:22:47,  9.09s/it]
 87%|████████▋ | 195157/225000 [370:21:42<61:58:01,  7.48s/it]
 87%|████████▋ | 195158/225000 [370:21:50<63:26:04,  7.65s/it]
 87%|████████▋ | 195159/225000 [370:21:57<59:55:29,  7.23s/it]
 87%|████████▋ | 195160/225000 [370:22:04<59:51:53,  7.22s/it]
 87%|████████▋ | 195161/225000 [370:22:20<80:57:39,  9.77s/it]
 87%|████████▋ | 195162/225000 [370:22:29<79:04:37,  9.54s/it]
 87%|████████▋ | 195163/225000 [370:22:37<76:43:44,  9.26s/it]
 87%|████████▋ | 195164/225000 [370:22:43<67:08:52,  8.10s/it]
 87%|████████▋ | 195165/225000 [370:22:50<65:37:22,  7.92s/it]
 87%|████████▋ | 195166/225000 [370:22:59<68:06:05,  8.22s/it]
 87%|████████▋ | 195167/225000 [370:23:09<71:39:49,  8.65s/it]
 87%|████████▋ | 195168/225000 [370:23:15<67:00:35,  8.09s/it]
 87%|████████▋ | 195169/225000 [370:23:27<76:45:03,  9.26s/it]
 87%|████████▋ | 195170/225000 [370:23:35<72:35:24,  8.76s/it]
 87%|████████▋ | 195171/225000 [370:23:45<76:51:30,  9.28s/it]
 87%|████████▋ | 195172/225000 [370:23:51<68:14:12,  8.24s/it]
 87%|████████▋ | 195173/225000 [370:23:59<67:26:03,  8.14s/it]
 87%|████████▋ | 195174/225000 [370:24:10<74:40:07,  9.01s/it]
 87%|████████▋ | 195175/225000 [370:24:20<75:51:41,  9.16s/it]
 87%|████████▋ | 195176/225000 [370:24:29<75:28:23,  9.11s/it]
 87%|████████▋ | 195177/225000 [370:24:36<71:06:13,  8.58s/it]
 87%|████████▋ | 195178/225000 [370:24:42<65:41:05,  7.93s/it]
 87%|████████▋ | 195179/225000 [370:24:49<63:09:59,  7.63s/it]
 87%|████████▋ | 195180/225000 [370:24:55<58:17:46,  7.04s/it]
 87%|████████▋ | 195181/225000 [370:25:09<75:04:12,  9.06s/it]
 87%|████████▋ | 195182/225000 [370:25:14<64:29:08,  7.79s/it]
 87%|████████▋ | 195183/225000 [370:25:22<66:16:09,  8.00s/it]
 87%|████████▋ | 195184/225000 [370:25:30<65:41:39,  7.93s/it]
 87%|████████▋ | 195185/225000 [370:25:41<73:16:06,  8.85s/it]
 87%|████████▋ | 195186/225000 [370:25:52<78:02:28,  9.42s/it]
 87%|████████▋ | 195187/225000 [370:25:58<71:24:47,  8.62s/it]
 87%|████████▋ | 195188/225000 [370:26:10<79:41:40,  9.62s/it]
 87%|████████▋ | 195189/225000 [370:26:17<71:43:27,  8.66s/it]
 87%|████████▋ | 195190/225000 [370:26:23<64:43:34,  7.82s/it]
 87%|████████▋ | 195191/225000 [370:26:39<86:59:58, 10.51s/it]
 87%|████████▋ | 195192/225000 [370:26:49<85:22:49, 10.31s/it]
 87%|████████▋ | 195193/225000 [370:26:54<70:51:36,  8.56s/it]
 87%|████████▋ | 195194/225000 [370:27:01<67:40:35,  8.17s/it]
 87%|████████▋ | 195195/225000 [370:27:14<80:08:29,  9.68s/it]
 87%|████████▋ | 195196/225000 [370:27:20<69:41:37,  8.42s/it]
 87%|████████▋ | 195197/225000 [370:27:26<65:17:11,  7.89s/it]
 87%|████████▋ | 195198/225000 [370:27:32<59:14:27,  7.16s/it]
 87%|████████▋ | 195199/225000 [370:27:39<58:48:34,  7.10s/it]
 87%|████████▋ | 195200/225000 [370:27:46<58:23:49,  7.05s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.123', 'learning_rate': '2.347e-06', 'epoch': '0.8676', 'num_input_tokens_seen': 618249088, 'train_runtime': '1.334e+06', 'train_tokens_per_second': '463.6'}

 87%|████████▋ | 195200/225000 [370:27:46<58:23:49,  7.05s/it]
 87%|████████▋ | 195201/225000 [370:27:52<56:55:05,  6.88s/it]
 87%|████████▋ | 195202/225000 [370:28:08<78:48:21,  9.52s/it]
 87%|████████▋ | 195203/225000 [370:28:12<65:17:12,  7.89s/it]
 87%|████████▋ | 195204/225000 [370:28:21<68:05:45,  8.23s/it]
 87%|████████▋ | 195205/225000 [370:28:28<63:59:07,  7.73s/it]
 87%|████████▋ | 195206/225000 [370:28:35<63:59:19,  7.73s/it]
 87%|████████▋ | 195207/225000 [370:28:48<75:42:02,  9.15s/it]
 87%|████████▋ | 195208/225000 [370:28:57<76:14:25,  9.21s/it]
 87%|████████▋ | 195209/225000 [370:29:05<72:09:27,  8.72s/it]
 87%|████████▋ | 195210/225000 [370:29:13<72:07:37,  8.72s/it]
 87%|████████▋ | 195211/225000 [370:29:20<66:57:34,  8.09s/it]
 87%|████████▋ | 195212/225000 [370:29:30<71:50:20,  8.68s/it]
 87%|████████▋ | 195213/225000 [370:29:39<71:43:16,  8.67s/it]
 87%|████████▋ | 195214/225000 [370:29:46<68:00:01,  8.22s/it]
 87%|████████▋ | 195215/225000 [370:29:52<64:02:52,  7.74s/it]
 87%|████████▋ | 195216/225000 [370:29:58<58:56:47,  7.12s/it]
 87%|████████▋ | 195217/225000 [370:30:03<54:08:57,  6.55s/it]
 87%|████████▋ | 195218/225000 [370:30:15<66:05:07,  7.99s/it]
 87%|████████▋ | 195219/225000 [370:30:21<60:55:17,  7.36s/it]
 87%|████████▋ | 195220/225000 [370:30:34<76:27:08,  9.24s/it]
 87%|████████▋ | 195221/225000 [370:30:40<66:51:47,  8.08s/it]
 87%|████████▋ | 195222/225000 [370:30:52<76:34:32,  9.26s/it]
 87%|████████▋ | 195223/225000 [370:30:57<67:58:52,  8.22s/it]
 87%|████████▋ | 195224/225000 [370:31:07<72:08:51,  8.72s/it]
 87%|████████▋ | 195225/225000 [370:31:20<82:58:59, 10.03s/it]
 87%|████████▋ | 195226/225000 [370:31:32<85:47:45, 10.37s/it]
 87%|████████▋ | 195227/225000 [370:31:38<74:45:18,  9.04s/it]
 87%|████████▋ | 195228/225000 [370:31:46<74:19:08,  8.99s/it]
 87%|████████▋ | 195229/225000 [370:31:56<75:18:37,  9.11s/it]
 87%|████████▋ | 195230/225000 [370:32:03<71:01:54,  8.59s/it]
 87%|████████▋ | 195231/225000 [370:32:09<64:39:41,  7.82s/it]
 87%|████████▋ | 195232/225000 [370:32:15<60:39:15,  7.34s/it]
 87%|████████▋ | 195233/225000 [370:32:24<62:38:47,  7.58s/it]
 87%|████████▋ | 195234/225000 [370:32:30<59:27:46,  7.19s/it]
 87%|████████▋ | 195235/225000 [370:32:45<78:35:39,  9.51s/it]
 87%|████████▋ | 195236/225000 [370:32:50<68:51:09,  8.33s/it]
 87%|████████▋ | 195237/225000 [370:32:58<67:06:55,  8.12s/it]
 87%|████████▋ | 195238/225000 [370:33:05<64:47:18,  7.84s/it]
 87%|████████▋ | 195239/225000 [370:33:17<75:38:19,  9.15s/it]
 87%|████████▋ | 195240/225000 [370:33:25<71:25:37,  8.64s/it]
 87%|████████▋ | 195241/225000 [370:33:35<76:37:25,  9.27s/it]
 87%|████████▋ | 195242/225000 [370:33:42<69:57:02,  8.46s/it]
 87%|████████▋ | 195243/225000 [370:33:49<64:59:49,  7.86s/it]
 87%|████████▋ | 195244/225000 [370:33:54<57:48:01,  6.99s/it]
 87%|████████▋ | 195245/225000 [370:34:09<78:35:37,  9.51s/it]
 87%|████████▋ | 195246/225000 [370:34:20<83:11:58, 10.07s/it]
 87%|████████▋ | 195247/225000 [370:34:28<78:22:35,  9.48s/it]
 87%|████████▋ | 195248/225000 [370:34:34<68:07:21,  8.24s/it]
 87%|████████▋ | 195249/225000 [370:34:40<63:15:53,  7.66s/it]
 87%|████████▋ | 195250/225000 [370:34:55<81:24:05,  9.85s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.084', 'learning_rate': '2.339e-06', 'epoch': '0.8678', 'num_input_tokens_seen': 618472912, 'train_runtime': '1.334e+06', 'train_tokens_per_second': '463.6'}

 87%|████████▋ | 195250/225000 [370:34:55<81:24:05,  9.85s/it]
 87%|████████▋ | 195251/225000 [370:35:05<81:00:28,  9.80s/it]
 87%|████████▋ | 195252/225000 [370:35:10<69:57:27,  8.47s/it]
 87%|████████▋ | 195253/225000 [370:35:22<79:24:50,  9.61s/it]
 87%|████████▋ | 195254/225000 [370:35:27<66:16:32,  8.02s/it]
 87%|████████▋ | 195255/225000 [370:35:36<68:29:55,  8.29s/it]
 87%|████████▋ | 195256/225000 [370:35:49<80:43:31,  9.77s/it]
 87%|████████▋ | 195257/225000 [370:35:59<82:05:40,  9.94s/it]
 87%|████████▋ | 195258/225000 [370:36:04<68:29:27,  8.29s/it]
 87%|████████▋ | 195259/225000 [370:36:12<67:58:35,  8.23s/it]
 87%|████████▋ | 195260/225000 [370:36:19<66:58:50,  8.11s/it]
 87%|████████▋ | 195261/225000 [370:36:29<71:48:23,  8.69s/it]
 87%|████████▋ | 195262/225000 [370:36:40<76:57:14,  9.32s/it]
 87%|████████▋ | 195263/225000 [370:36:45<65:36:39,  7.94s/it]
 87%|████████▋ | 195264/225000 [370:36:53<65:36:43,  7.94s/it]
 87%|████████▋ | 195265/225000 [370:36:57<56:16:39,  6.81s/it]
 87%|████████▋ | 195266/225000 [370:37:06<62:11:08,  7.53s/it]
 87%|████████▋ | 195267/225000 [370:37:13<59:19:57,  7.18s/it]
 87%|████████▋ | 195268/225000 [370:37:22<65:43:54,  7.96s/it]
 87%|████████▋ | 195269/225000 [370:37:35<77:21:18,  9.37s/it]
 87%|████████▋ | 195270/225000 [370:37:42<71:43:21,  8.68s/it]
 87%|████████▋ | 195271/225000 [370:37:48<65:05:48,  7.88s/it]
 87%|████████▋ | 195272/225000 [370:37:59<71:29:13,  8.66s/it]
 87%|████████▋ | 195273/225000 [370:38:07<69:34:05,  8.42s/it]
 87%|████████▋ | 195274/225000 [370:38:14<68:08:22,  8.25s/it]
 87%|████████▋ | 195275/225000 [370:38:24<72:11:50,  8.74s/it]
 87%|████████▋ | 195276/225000 [370:38:32<69:18:29,  8.39s/it]
 87%|████████▋ | 195277/225000 [370:38:41<70:29:31,  8.54s/it]
 87%|████████▋ | 195278/225000 [370:38:51<74:43:22,  9.05s/it]
 87%|████████▋ | 195279/225000 [370:38:58<70:09:48,  8.50s/it]
 87%|████████▋ | 195280/225000 [370:39:04<62:52:16,  7.62s/it]
 87%|████████▋ | 195281/225000 [370:39:13<67:08:19,  8.13s/it]
 87%|████████▋ | 195282/225000 [370:39:17<57:46:39,  7.00s/it]
 87%|████████▋ | 195283/225000 [370:39:27<64:58:25,  7.87s/it]
 87%|████████▋ | 195284/225000 [370:39:32<55:53:51,  6.77s/it]
 87%|████████▋ | 195285/225000 [370:39:39<56:50:55,  6.89s/it]
 87%|████████▋ | 195286/225000 [370:39:46<57:29:00,  6.96s/it]
 87%|████████▋ | 195287/225000 [370:39:54<59:29:30,  7.21s/it]
 87%|████████▋ | 195288/225000 [370:40:09<79:20:55,  9.61s/it]
 87%|████████▋ | 195289/225000 [370:40:15<70:08:54,  8.50s/it]
 87%|████████▋ | 195290/225000 [370:40:21<65:32:57,  7.94s/it]
 87%|████████▋ | 195291/225000 [370:40:33<75:34:43,  9.16s/it]
 87%|████████▋ | 195292/225000 [370:40:45<81:45:36,  9.91s/it]
 87%|████████▋ | 195293/225000 [370:40:52<74:36:39,  9.04s/it]
 87%|████████▋ | 195294/225000 [370:41:01<75:11:45,  9.11s/it]
 87%|████████▋ | 195295/225000 [370:41:08<68:34:49,  8.31s/it]
 87%|████████▋ | 195296/225000 [370:41:21<81:30:47,  9.88s/it]
 87%|████████▋ | 195297/225000 [370:41:26<68:39:57,  8.32s/it]
 87%|████████▋ | 195298/225000 [370:41:37<75:24:25,  9.14s/it]
 87%|████████▋ | 195299/225000 [370:41:48<80:02:26,  9.70s/it]
 87%|████████▋ | 195300/225000 [370:41:52<66:40:15,  8.08s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.213', 'learning_rate': '2.332e-06', 'epoch': '0.868', 'num_input_tokens_seen': 618695352, 'train_runtime': '1.335e+06', 'train_tokens_per_second': '463.6'}

 87%|████████▋ | 195300/225000 [370:41:52<66:40:15,  8.08s/it]
 87%|████████▋ | 195301/225000 [370:42:01<66:55:05,  8.11s/it]
 87%|████████▋ | 195302/225000 [370:42:13<78:04:38,  9.46s/it]
 87%|████████▋ | 195303/225000 [370:42:19<67:46:32,  8.22s/it]
 87%|████████▋ | 195304/225000 [370:42:29<73:23:16,  8.90s/it]
 87%|████████▋ | 195305/225000 [370:42:36<68:14:15,  8.27s/it]
 87%|████████▋ | 195306/225000 [370:42:47<75:10:03,  9.11s/it]
 87%|████████▋ | 195307/225000 [370:42:56<76:02:27,  9.22s/it]
 87%|████████▋ | 195308/225000 [370:43:06<78:04:19,  9.47s/it]
 87%|████████▋ | 195309/225000 [370:43:12<68:38:19,  8.32s/it]
 87%|████████▋ | 195310/225000 [370:43:23<75:26:35,  9.15s/it]
 87%|████████▋ | 195311/225000 [370:43:33<76:26:55,  9.27s/it]
 87%|████████▋ | 195312/225000 [370:43:38<67:34:56,  8.20s/it]
 87%|████████▋ | 195313/225000 [370:43:45<63:45:37,  7.73s/it]
 87%|████████▋ | 195314/225000 [370:43:51<60:01:06,  7.28s/it]
 87%|████████▋ | 195315/225000 [370:44:01<65:59:18,  8.00s/it]
 87%|████████▋ | 195316/225000 [370:44:11<71:49:55,  8.71s/it]
 87%|████████▋ | 195317/225000 [370:44:19<70:14:12,  8.52s/it]
 87%|████████▋ | 195318/225000 [370:44:24<60:45:05,  7.37s/it]
 87%|████████▋ | 195319/225000 [370:44:29<55:30:55,  6.73s/it]
 87%|████████▋ | 195320/225000 [370:44:36<55:49:06,  6.77s/it]
 87%|████████▋ | 195321/225000 [370:44:45<61:57:01,  7.51s/it]
 87%|████████▋ | 195322/225000 [370:44:52<59:46:10,  7.25s/it]
 87%|████████▋ | 195323/225000 [370:45:02<65:35:23,  7.96s/it]
 87%|████████▋ | 195324/225000 [370:45:13<73:02:47,  8.86s/it]
 87%|████████▋ | 195325/225000 [370:45:21<71:47:37,  8.71s/it]
 87%|████████▋ | 195326/225000 [370:45:27<65:14:32,  7.92s/it]
 87%|████████▋ | 195327/225000 [370:45:38<73:02:32,  8.86s/it]
 87%|████████▋ | 195328/225000 [370:45:49<78:45:41,  9.56s/it]
 87%|████████▋ | 195329/225000 [370:45:55<68:51:14,  8.35s/it]
 87%|████████▋ | 195330/225000 [370:46:04<71:47:55,  8.71s/it]
 87%|████████▋ | 195331/225000 [370:46:15<76:10:15,  9.24s/it]
 87%|████████▋ | 195332/225000 [370:46:22<69:51:41,  8.48s/it]
 87%|████████▋ | 195333/225000 [370:46:29<68:30:33,  8.31s/it]
 87%|████████▋ | 195334/225000 [370:46:37<66:45:44,  8.10s/it]
 87%|████████▋ | 195335/225000 [370:46:46<68:07:59,  8.27s/it]
 87%|████████▋ | 195336/225000 [370:46:56<72:26:25,  8.79s/it]
 87%|████████▋ | 195337/225000 [370:47:04<72:02:24,  8.74s/it]
 87%|████████▋ | 195338/225000 [370:47:18<83:40:13, 10.15s/it]
 87%|████████▋ | 195339/225000 [370:47:25<75:16:36,  9.14s/it]
 87%|████████▋ | 195340/225000 [370:47:32<71:38:50,  8.70s/it]
 87%|████████▋ | 195341/225000 [370:47:44<80:05:07,  9.72s/it]
 87%|████████▋ | 195342/225000 [370:47:51<71:42:56,  8.71s/it]
 87%|████████▋ | 195343/225000 [370:47:57<65:56:32,  8.00s/it]
 87%|████████▋ | 195344/225000 [370:48:09<75:29:30,  9.16s/it]
 87%|████████▋ | 195345/225000 [370:48:16<70:41:01,  8.58s/it]
 87%|████████▋ | 195346/225000 [370:48:22<64:53:08,  7.88s/it]
 87%|████████▋ | 195347/225000 [370:48:29<61:24:14,  7.45s/it]
 87%|████████▋ | 195348/225000 [370:48:37<61:53:41,  7.51s/it]
 87%|████████▋ | 195349/225000 [370:48:49<74:38:49,  9.06s/it]
 87%|████████▋ | 195350/225000 [370:48:59<76:29:23,  9.29s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.182', 'learning_rate': '2.324e-06', 'epoch': '0.8682', 'num_input_tokens_seen': 618923112, 'train_runtime': '1.335e+06', 'train_tokens_per_second': '463.6'}

 87%|████████▋ | 195350/225000 [370:48:59<76:29:23,  9.29s/it]
 87%|████████▋ | 195351/225000 [370:49:09<77:04:58,  9.36s/it]
 87%|████████▋ | 195352/225000 [370:49:15<70:39:24,  8.58s/it]
 87%|████████▋ | 195353/225000 [370:49:26<76:53:07,  9.34s/it]
 87%|████████▋ | 195354/225000 [370:49:31<64:55:00,  7.88s/it]
 87%|████████▋ | 195355/225000 [370:49:40<67:43:03,  8.22s/it]
 87%|████████▋ | 195356/225000 [370:49:46<62:45:16,  7.62s/it]
 87%|████████▋ | 195357/225000 [370:49:55<65:25:07,  7.94s/it]
 87%|████████▋ | 195358/225000 [370:50:02<62:57:27,  7.65s/it]
 87%|████████▋ | 195359/225000 [370:50:11<67:43:30,  8.23s/it]
 87%|████████▋ | 195360/225000 [370:50:19<67:17:12,  8.17s/it]
 87%|████████▋ | 195361/225000 [370:50:28<67:08:57,  8.16s/it]
 87%|████████▋ | 195362/225000 [370:50:34<64:07:24,  7.79s/it]
 87%|████████▋ | 195363/225000 [370:50:43<65:18:48,  7.93s/it]
 87%|████████▋ | 195364/225000 [370:50:58<82:26:32, 10.01s/it]
 87%|████████▋ | 195365/225000 [370:51:06<78:53:50,  9.58s/it]
 87%|████████▋ | 195366/225000 [370:51:12<70:27:33,  8.56s/it]
 87%|████████▋ | 195367/225000 [370:51:20<67:46:11,  8.23s/it]
 87%|████████▋ | 195368/225000 [370:51:31<75:07:01,  9.13s/it]
 87%|████████▋ | 195369/225000 [370:51:39<71:30:22,  8.69s/it]
 87%|████████▋ | 195370/225000 [370:51:50<79:01:08,  9.60s/it]
 87%|████████▋ | 195371/225000 [370:51:57<70:41:12,  8.59s/it]
 87%|████████▋ | 195372/225000 [370:52:05<70:13:18,  8.53s/it]
 87%|████████▋ | 195373/225000 [370:52:16<76:57:25,  9.35s/it]
 87%|████████▋ | 195374/225000 [370:52:21<64:26:33,  7.83s/it]
 87%|████████▋ | 195375/225000 [370:52:28<64:24:15,  7.83s/it]
 87%|████████▋ | 195376/225000 [370:52:32<54:36:32,  6.64s/it]
 87%|████████▋ | 195377/225000 [370:52:41<59:42:00,  7.26s/it]
 87%|████████▋ | 195378/225000 [370:52:48<58:58:35,  7.17s/it]
 87%|████████▋ | 195379/225000 [370:52:55<59:45:32,  7.26s/it]
 87%|████████▋ | 195380/225000 [370:53:05<66:36:42,  8.10s/it]
 87%|████████▋ | 195381/225000 [370:53:15<70:41:36,  8.59s/it]
 87%|████████▋ | 195382/225000 [370:53:25<72:37:57,  8.83s/it]
 87%|████████▋ | 195383/225000 [370:53:33<71:10:37,  8.65s/it]
 87%|████████▋ | 195384/225000 [370:53:42<73:27:23,  8.93s/it]
 87%|████████▋ | 195385/225000 [370:53:49<66:41:26,  8.11s/it]
 87%|████████▋ | 195386/225000 [370:54:00<74:29:14,  9.05s/it]
 87%|████████▋ | 195387/225000 [370:54:07<70:46:59,  8.60s/it]
 87%|████████▋ | 195388/225000 [370:54:12<60:15:50,  7.33s/it]
 87%|████████▋ | 195389/225000 [370:54:20<62:13:44,  7.57s/it]
 87%|████████▋ | 195390/225000 [370:54:30<69:31:40,  8.45s/it]
 87%|████████▋ | 195391/225000 [370:54:38<66:24:24,  8.07s/it]
 87%|████████▋ | 195392/225000 [370:54:46<67:40:02,  8.23s/it]
 87%|████████▋ | 195393/225000 [370:54:55<68:47:31,  8.36s/it]
 87%|████████▋ | 195394/225000 [370:55:06<76:46:57,  9.34s/it]
 87%|████████▋ | 195395/225000 [370:55:12<68:12:13,  8.29s/it]
 87%|████████▋ | 195396/225000 [370:55:22<72:28:26,  8.81s/it]
 87%|████████▋ | 195397/225000 [370:55:31<71:43:48,  8.72s/it]
 87%|████████▋ | 195398/225000 [370:55:41<74:56:43,  9.11s/it]
 87%|████████▋ | 195399/225000 [370:55:48<68:53:12,  8.38s/it]
 87%|████████▋ | 195400/225000 [370:55:53<60:27:22,  7.35s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.615', 'learning_rate': '2.316e-06', 'epoch': '0.8684', 'num_input_tokens_seen': 619144584, 'train_runtime': '1.335e+06', 'train_tokens_per_second': '463.7'}

 87%|████████▋ | 195400/225000 [370:55:53<60:27:22,  7.35s/it]
 87%|████████▋ | 195401/225000 [370:56:03<68:50:08,  8.37s/it]
 87%|████████▋ | 195402/225000 [370:56:15<77:47:57,  9.46s/it]
 87%|████████▋ | 195403/225000 [370:56:24<76:44:30,  9.33s/it]
 87%|████████▋ | 195404/225000 [370:56:29<65:27:51,  7.96s/it]
 87%|████████▋ | 195405/225000 [370:56:34<58:42:19,  7.14s/it]
 87%|████████▋ | 195406/225000 [370:56:46<70:43:02,  8.60s/it]
 87%|████████▋ | 195407/225000 [370:56:52<64:31:39,  7.85s/it]
 87%|████████▋ | 195408/225000 [370:57:02<69:21:00,  8.44s/it]
 87%|████████▋ | 195409/225000 [370:57:12<73:43:29,  8.97s/it]
 87%|████████▋ | 195410/225000 [370:57:22<75:35:23,  9.20s/it]
 87%|████████▋ | 195411/225000 [370:57:29<70:06:26,  8.53s/it]
 87%|████████▋ | 195412/225000 [370:57:39<72:55:09,  8.87s/it]
 87%|████████▋ | 195413/225000 [370:57:45<66:18:45,  8.07s/it]
 87%|████████▋ | 195414/225000 [370:57:52<64:49:12,  7.89s/it]
 87%|████████▋ | 195415/225000 [370:58:00<63:24:58,  7.72s/it]
 87%|████████▋ | 195416/225000 [370:58:06<60:19:11,  7.34s/it]
 87%|████████▋ | 195417/225000 [370:58:23<83:39:12, 10.18s/it]
 87%|████████▋ | 195418/225000 [370:58:30<75:33:24,  9.19s/it]
 87%|████████▋ | 195419/225000 [370:58:41<79:03:50,  9.62s/it]
 87%|████████▋ | 195420/225000 [370:58:51<81:24:24,  9.91s/it]
 87%|████████▋ | 195421/225000 [370:58:59<76:38:42,  9.33s/it]
 87%|████████▋ | 195422/225000 [370:59:08<76:06:10,  9.26s/it]
 87%|████████▋ | 195423/225000 [370:59:21<85:44:01, 10.44s/it]
 87%|████████▋ | 195424/225000 [370:59:32<85:38:05, 10.42s/it]
 87%|████████▋ | 195425/225000 [370:59:38<74:19:11,  9.05s/it]
 87%|████████▋ | 195426/225000 [370:59:43<64:53:53,  7.90s/it]
 87%|████████▋ | 195427/225000 [370:59:48<58:05:41,  7.07s/it]
 87%|████████▋ | 195428/225000 [370:59:55<59:00:07,  7.18s/it]
 87%|████████▋ | 195429/225000 [371:00:05<65:13:56,  7.94s/it]
 87%|████████▋ | 195430/225000 [371:00:10<57:40:38,  7.02s/it]
 87%|████████▋ | 195431/225000 [371:00:20<65:33:19,  7.98s/it]
 87%|████████▋ | 195432/225000 [371:00:27<63:26:48,  7.72s/it]
 87%|████████▋ | 195433/225000 [371:00:34<60:45:37,  7.40s/it]
 87%|████████▋ | 195434/225000 [371:00:48<77:08:56,  9.39s/it]
 87%|████████▋ | 195435/225000 [371:00:54<69:02:08,  8.41s/it]
 87%|████████▋ | 195436/225000 [371:01:03<69:18:46,  8.44s/it]
 87%|████████▋ | 195437/225000 [371:01:13<73:05:48,  8.90s/it]
 87%|████████▋ | 195438/225000 [371:01:28<89:59:03, 10.96s/it]
 87%|████████▋ | 195439/225000 [371:01:33<74:56:46,  9.13s/it]
 87%|████████▋ | 195440/225000 [371:01:39<66:45:10,  8.13s/it]
 87%|████████▋ | 195441/225000 [371:01:50<74:16:52,  9.05s/it]
 87%|████████▋ | 195442/225000 [371:02:00<75:31:34,  9.20s/it]
 87%|████████▋ | 195443/225000 [371:02:04<64:05:47,  7.81s/it]
 87%|████████▋ | 195444/225000 [371:02:12<64:29:35,  7.86s/it]
 87%|████████▋ | 195445/225000 [371:02:23<72:13:01,  8.80s/it]
 87%|████████▋ | 195446/225000 [371:02:33<74:53:55,  9.12s/it]
 87%|████████▋ | 195447/225000 [371:02:43<75:48:42,  9.24s/it]
 87%|████████▋ | 195448/225000 [371:02:49<68:33:01,  8.35s/it]
 87%|████████▋ | 195449/225000 [371:02:58<71:23:33,  8.70s/it]
 87%|████████▋ | 195450/225000 [371:03:07<69:57:40,  8.52s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.268', 'learning_rate': '2.309e-06', 'epoch': '0.8687', 'num_input_tokens_seen': 619376104, 'train_runtime': '1.336e+06', 'train_tokens_per_second': '463.7'}

 87%|████████▋ | 195450/225000 [371:03:07<69:57:40,  8.52s/it]
 87%|████████▋ | 195451/225000 [371:03:22<86:10:39, 10.50s/it]
 87%|████████▋ | 195452/225000 [371:03:29<79:03:27,  9.63s/it]
 87%|████████▋ | 195453/225000 [371:03:40<80:46:40,  9.84s/it]
 87%|████████▋ | 195454/225000 [371:03:48<77:32:35,  9.45s/it]
 87%|████████▋ | 195455/225000 [371:03:55<70:12:15,  8.55s/it]
 87%|████████▋ | 195456/225000 [371:04:07<79:17:00,  9.66s/it]
 87%|████████▋ | 195457/225000 [371:04:14<73:48:11,  8.99s/it]
 87%|████████▋ | 195458/225000 [371:04:26<80:07:09,  9.76s/it]
 87%|████████▋ | 195459/225000 [371:04:33<72:34:44,  8.84s/it]
 87%|████████▋ | 195460/225000 [371:04:38<63:37:56,  7.75s/it]
 87%|████████▋ | 195461/225000 [371:04:46<65:11:27,  7.94s/it]
 87%|████████▋ | 195462/225000 [371:04:56<70:33:46,  8.60s/it]
 87%|████████▋ | 195463/225000 [371:05:10<83:25:11, 10.17s/it]
 87%|████████▋ | 195464/225000 [371:05:16<72:53:53,  8.89s/it]
 87%|████████▋ | 195465/225000 [371:05:24<69:31:02,  8.47s/it]
 87%|████████▋ | 195466/225000 [371:05:31<67:26:00,  8.22s/it]
 87%|████████▋ | 195467/225000 [371:05:42<73:50:21,  9.00s/it]
 87%|████████▋ | 195468/225000 [371:05:49<69:31:29,  8.48s/it]
 87%|████████▋ | 195469/225000 [371:05:58<70:55:21,  8.65s/it]
 87%|████████▋ | 195470/225000 [371:06:07<71:07:12,  8.67s/it]
 87%|████████▋ | 195471/225000 [371:06:13<63:25:25,  7.73s/it]
 87%|████████▋ | 195472/225000 [371:06:17<56:07:13,  6.84s/it]
 87%|████████▋ | 195473/225000 [371:06:29<68:39:03,  8.37s/it]
 87%|████████▋ | 195474/225000 [371:06:36<65:02:21,  7.93s/it]
 87%|████████▋ | 195475/225000 [371:06:44<64:06:16,  7.82s/it]
 87%|████████▋ | 195476/225000 [371:06:51<61:54:59,  7.55s/it]
 87%|████████▋ | 195477/225000 [371:06:57<59:22:42,  7.24s/it]
 87%|████████▋ | 195478/225000 [371:07:05<60:02:19,  7.32s/it]
 87%|████████▋ | 195479/225000 [371:07:22<84:42:20, 10.33s/it]
 87%|████████▋ | 195480/225000 [371:07:28<73:44:25,  8.99s/it]
 87%|████████▋ | 195481/225000 [371:07:34<67:38:37,  8.25s/it]
 87%|████████▋ | 195482/225000 [371:07:45<73:29:00,  8.96s/it]
 87%|████████▋ | 195483/225000 [371:07:50<64:46:28,  7.90s/it]
 87%|████████▋ | 195484/225000 [371:07:59<66:42:18,  8.14s/it]
 87%|████████▋ | 195485/225000 [371:08:06<62:22:14,  7.61s/it]
 87%|████████▋ | 195486/225000 [371:08:16<68:54:30,  8.41s/it]
 87%|████████▋ | 195487/225000 [371:08:30<83:40:05, 10.21s/it]
 87%|████████▋ | 195488/225000 [371:08:36<72:29:36,  8.84s/it]
 87%|████████▋ | 195489/225000 [371:08:43<67:09:50,  8.19s/it]
 87%|████████▋ | 195490/225000 [371:08:53<72:31:38,  8.85s/it]
 87%|████████▋ | 195491/225000 [371:08:59<66:24:34,  8.10s/it]
 87%|████████▋ | 195492/225000 [371:09:14<82:17:51, 10.04s/it]
 87%|████████▋ | 195493/225000 [371:09:24<83:36:24, 10.20s/it]
 87%|████████▋ | 195494/225000 [371:09:32<77:46:18,  9.49s/it]
 87%|████████▋ | 195495/225000 [371:09:38<69:31:43,  8.48s/it]
 87%|████████▋ | 195496/225000 [371:09:47<68:53:22,  8.41s/it]
 87%|████████▋ | 195497/225000 [371:09:54<66:24:50,  8.10s/it]
 87%|████████▋ | 195498/225000 [371:10:02<64:55:50,  7.92s/it]
 87%|████████▋ | 195499/225000 [371:10:09<64:36:02,  7.88s/it]
 87%|████████▋ | 195500/225000 [371:10:19<68:59:13,  8.42s/it]
                                                              
{'loss': '1.284', 'grad_norm': '1.451', 'learning_rate': '2.301e-06', 'epoch': '0.8689', 'num_input_tokens_seen': 619603816, 'train_runtime': '1.336e+06', 'train_tokens_per_second': '463.7'}

 87%|████████▋ | 195500/225000 [371:10:19<68:59:13,  8.42s/it]
 87%|████████▋ | 195501/225000 [371:10:27<67:59:17,  8.30s/it]
 87%|████████▋ | 195502/225000 [371:10:34<64:09:07,  7.83s/it]
 87%|████████▋ | 195503/225000 [371:10:39<57:49:02,  7.06s/it]
 87%|████████▋ | 195504/225000 [371:10:48<61:58:53,  7.56s/it]
 87%|████████▋ | 195505/225000 [371:11:00<74:18:39,  9.07s/it]
 87%|████████▋ | 195506/225000 [371:11:06<65:53:53,  8.04s/it]
 87%|████████▋ | 195507/225000 [371:11:19<77:46:03,  9.49s/it]
 87%|████████▋ | 195508/225000 [371:11:26<71:02:06,  8.67s/it]
 87%|████████▋ | 195509/225000 [371:11:33<67:27:33,  8.23s/it]
 87%|████████▋ | 195510/225000 [371:11:37<58:39:56,  7.16s/it]
 87%|████████▋ | 195511/225000 [371:11:49<68:46:31,  8.40s/it]
 87%|████████▋ | 195512/225000 [371:11:57<69:36:36,  8.50s/it]
 87%|████████▋ | 195513/225000 [371:12:07<71:45:27,  8.76s/it]
 87%|████████▋ | 195514/225000 [371:12:12<62:54:53,  7.68s/it]
 87%|████████▋ | 195515/225000 [371:12:24<72:55:23,  8.90s/it]
 87%|████████▋ | 195516/225000 [371:12:29<64:38:22,  7.89s/it]
 87%|████████▋ | 195517/225000 [371:12:42<76:22:00,  9.32s/it]
 87%|████████▋ | 195518/225000 [371:12:52<77:43:14,  9.49s/it]
 87%|████████▋ | 195519/225000 [371:12:57<68:11:34,  8.33s/it]
 87%|████████▋ | 195520/225000 [371:13:04<64:59:32,  7.94s/it]
 87%|████████▋ | 195521/225000 [371:13:16<73:42:31,  9.00s/it]
 87%|████████▋ | 195522/225000 [371:13:21<64:14:41,  7.85s/it]
 87%|████████▋ | 195523/225000 [371:13:34<75:42:23,  9.25s/it]
 87%|████████▋ | 195524/225000 [371:13:41<71:32:37,  8.74s/it]
 87%|████████▋ | 195525/225000 [371:13:47<65:10:46,  7.96s/it]
 87%|████████▋ | 195526/225000 [371:14:01<79:13:16,  9.68s/it]
 87%|████████▋ | 195527/225000 [371:14:11<79:33:53,  9.72s/it]
 87%|████████▋ | 195528/225000 [371:14:16<69:01:13,  8.43s/it]
 87%|████████▋ | 195529/225000 [371:14:22<61:19:38,  7.49s/it]
 87%|████████▋ | 195530/225000 [371:14:32<67:26:27,  8.24s/it]
 87%|████████▋ | 195531/225000 [371:14:39<65:34:52,  8.01s/it]
 87%|████████▋ | 195532/225000 [371:14:48<68:20:46,  8.35s/it]
 87%|████████▋ | 195533/225000 [371:14:56<66:49:18,  8.16s/it]
 87%|████████▋ | 195534/225000 [371:15:04<67:28:07,  8.24s/it]
 87%|████████▋ | 195535/225000 [371:15:13<67:25:38,  8.24s/it]
 87%|████████▋ | 195536/225000 [371:15:24<76:25:55,  9.34s/it]
 87%|████████▋ | 195537/225000 [371:15:31<69:40:19,  8.51s/it]
 87%|████████▋ | 195538/225000 [371:15:43<77:48:47,  9.51s/it]
 87%|████████▋ | 195539/225000 [371:15:48<67:21:07,  8.23s/it]
 87%|████████▋ | 195540/225000 [371:15:57<67:57:52,  8.31s/it]
 87%|████████▋ | 195541/225000 [371:16:08<74:43:49,  9.13s/it]
 87%|████████▋ | 195542/225000 [371:16:16<72:23:53,  8.85s/it]
 87%|████████▋ | 195543/225000 [371:16:28<80:48:03,  9.87s/it]
 87%|████████▋ | 195544/225000 [371:16:34<70:37:12,  8.63s/it]
 87%|████████▋ | 195545/225000 [371:16:39<62:15:13,  7.61s/it]
 87%|████████▋ | 195546/225000 [371:16:51<73:25:27,  8.97s/it]
 87%|████████▋ | 195547/225000 [371:17:00<72:10:39,  8.82s/it]
 87%|████████▋ | 195548/225000 [371:17:11<79:28:33,  9.71s/it]
 87%|████████▋ | 195549/225000 [371:17:18<71:52:05,  8.78s/it]
 87%|████████▋ | 195550/225000 [371:17:28<73:30:58,  8.99s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.329', 'learning_rate': '2.293e-06', 'epoch': '0.8691', 'num_input_tokens_seen': 619829232, 'train_runtime': '1.337e+06', 'train_tokens_per_second': '463.7'}

 87%|████████▋ | 195550/225000 [371:17:28<73:30:58,  8.99s/it]
 87%|████████▋ | 195551/225000 [371:17:35<70:45:02,  8.65s/it]
 87%|████████▋ | 195552/225000 [371:17:41<64:21:47,  7.87s/it]
 87%|████████▋ | 195553/225000 [371:17:48<61:25:40,  7.51s/it]
 87%|████████▋ | 195554/225000 [371:18:02<77:02:36,  9.42s/it]
 87%|████████▋ | 195555/225000 [371:18:08<69:17:33,  8.47s/it]
 87%|████████▋ | 195556/225000 [371:18:18<73:34:49,  9.00s/it]
 87%|████████▋ | 195557/225000 [371:18:25<66:19:24,  8.11s/it]
 87%|████████▋ | 195558/225000 [371:18:35<72:08:18,  8.82s/it]
 87%|████████▋ | 195559/225000 [371:18:40<63:57:22,  7.82s/it]
 87%|████████▋ | 195560/225000 [371:18:54<77:35:06,  9.49s/it]
 87%|████████▋ | 195561/225000 [371:19:01<71:52:47,  8.79s/it]
 87%|████████▋ | 195562/225000 [371:19:08<68:23:21,  8.36s/it]
 87%|████████▋ | 195563/225000 [371:19:16<66:30:32,  8.13s/it]
 87%|████████▋ | 195564/225000 [371:19:25<69:02:38,  8.44s/it]
 87%|████████▋ | 195565/225000 [371:19:34<70:15:21,  8.59s/it]
 87%|████████▋ | 195566/225000 [371:19:41<66:43:39,  8.16s/it]
 87%|████████▋ | 195567/225000 [371:19:47<60:38:29,  7.42s/it]
 87%|████████▋ | 195568/225000 [371:19:56<64:05:38,  7.84s/it]
 87%|████████▋ | 195569/225000 [371:20:09<76:43:35,  9.39s/it]
 87%|████████▋ | 195570/225000 [371:20:20<80:59:35,  9.91s/it]
 87%|████████▋ | 195571/225000 [371:20:26<72:29:07,  8.87s/it]
 87%|████████▋ | 195572/225000 [371:20:35<71:02:38,  8.69s/it]
 87%|████████▋ | 195573/225000 [371:20:45<75:47:15,  9.27s/it]
 87%|████████▋ | 195574/225000 [371:20:52<69:15:39,  8.47s/it]
 87%|████████▋ | 195575/225000 [371:21:01<71:15:10,  8.72s/it]
 87%|████████▋ | 195576/225000 [371:21:11<73:48:40,  9.03s/it]
 87%|████████▋ | 195577/225000 [371:21:21<76:28:24,  9.36s/it]
 87%|████████▋ | 195578/225000 [371:21:26<66:39:08,  8.16s/it]
 87%|████████▋ | 195579/225000 [371:21:37<71:44:50,  8.78s/it]
 87%|████████▋ | 195580/225000 [371:21:43<65:46:11,  8.05s/it]
 87%|████████▋ | 195581/225000 [371:21:48<58:33:24,  7.17s/it]
 87%|████████▋ | 195582/225000 [371:21:58<64:19:46,  7.87s/it]
 87%|████████▋ | 195583/225000 [371:22:11<77:48:24,  9.52s/it]
 87%|████████▋ | 195584/225000 [371:22:21<79:35:30,  9.74s/it]
 87%|████████▋ | 195585/225000 [371:22:31<79:48:35,  9.77s/it]
 87%|████████▋ | 195586/225000 [371:22:39<74:17:28,  9.09s/it]
 87%|████████▋ | 195587/225000 [371:22:44<65:29:05,  8.02s/it]
 87%|████████▋ | 195588/225000 [371:22:50<61:00:47,  7.47s/it]
 87%|████████▋ | 195589/225000 [371:23:05<77:53:19,  9.53s/it]
 87%|████████▋ | 195590/225000 [371:23:12<72:01:58,  8.82s/it]
 87%|████████▋ | 195591/225000 [371:23:18<64:37:52,  7.91s/it]
 87%|████████▋ | 195592/225000 [371:23:30<76:21:18,  9.35s/it]
 87%|████████▋ | 195593/225000 [371:23:37<68:59:45,  8.45s/it]
 87%|████████▋ | 195594/225000 [371:23:44<67:07:37,  8.22s/it]
 87%|████████▋ | 195595/225000 [371:23:58<81:31:17,  9.98s/it]
 87%|████████▋ | 195596/225000 [371:24:06<74:40:46,  9.14s/it]
 87%|████████▋ | 195597/225000 [371:24:15<74:50:44,  9.16s/it]
 87%|████████▋ | 195598/225000 [371:24:21<68:22:45,  8.37s/it]
 87%|████████▋ | 195599/225000 [371:24:32<74:37:05,  9.14s/it]
 87%|████████▋ | 195600/225000 [371:24:42<75:53:49,  9.29s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.18', 'learning_rate': '2.286e-06', 'epoch': '0.8693', 'num_input_tokens_seen': 620061488, 'train_runtime': '1.337e+06', 'train_tokens_per_second': '463.7'}

 87%|████████▋ | 195600/225000 [371:24:42<75:53:49,  9.29s/it]
 87%|████████▋ | 195601/225000 [371:24:48<67:10:51,  8.23s/it]
 87%|████████▋ | 195602/225000 [371:24:57<71:04:03,  8.70s/it]
 87%|████████▋ | 195603/225000 [371:25:03<63:24:20,  7.76s/it]
 87%|████████▋ | 195604/225000 [371:25:13<69:02:33,  8.46s/it]
 87%|████████▋ | 195605/225000 [371:25:25<78:17:41,  9.59s/it]
 87%|████████▋ | 195606/225000 [371:25:34<77:14:56,  9.46s/it]
 87%|████████▋ | 195607/225000 [371:25:40<68:47:01,  8.42s/it]
 87%|████████▋ | 195608/225000 [371:25:45<60:02:45,  7.35s/it]
 87%|████████▋ | 195609/225000 [371:25:50<54:43:17,  6.70s/it]
 87%|████████▋ | 195610/225000 [371:25:56<52:45:11,  6.46s/it]
 87%|████████▋ | 195611/225000 [371:26:08<66:35:16,  8.16s/it]
 87%|████████▋ | 195612/225000 [371:26:16<64:10:26,  7.86s/it]
 87%|████████▋ | 195613/225000 [371:26:20<55:59:20,  6.86s/it]
 87%|████████▋ | 195614/225000 [371:26:27<55:47:44,  6.84s/it]
 87%|████████▋ | 195615/225000 [371:26:33<53:47:04,  6.59s/it]
 87%|████████▋ | 195616/225000 [371:26:41<57:54:41,  7.10s/it]
 87%|████████▋ | 195617/225000 [371:26:54<71:28:16,  8.76s/it]
 87%|████████▋ | 195618/225000 [371:27:02<70:20:18,  8.62s/it]
 87%|████████▋ | 195619/225000 [371:27:08<63:39:47,  7.80s/it]
 87%|████████▋ | 195620/225000 [371:27:14<58:11:44,  7.13s/it]
 87%|████████▋ | 195621/225000 [371:27:22<61:43:46,  7.56s/it]
 87%|████████▋ | 195622/225000 [371:27:30<62:53:03,  7.71s/it]
 87%|████████▋ | 195623/225000 [371:27:45<79:04:48,  9.69s/it]
 87%|████████▋ | 195624/225000 [371:27:52<73:00:34,  8.95s/it]
 87%|████████▋ | 195625/225000 [371:28:07<87:10:32, 10.68s/it]
 87%|████████▋ | 195626/225000 [371:28:14<80:08:40,  9.82s/it]
 87%|████████▋ | 195627/225000 [371:28:26<84:21:04, 10.34s/it]
 87%|████████▋ | 195628/225000 [371:28:32<73:51:01,  9.05s/it]
 87%|████████▋ | 195629/225000 [371:28:39<68:57:35,  8.45s/it]
 87%|████████▋ | 195630/225000 [371:28:48<71:18:21,  8.74s/it]
 87%|████████▋ | 195631/225000 [371:28:55<65:43:16,  8.06s/it]
 87%|████████▋ | 195632/225000 [371:29:01<61:49:41,  7.58s/it]
 87%|████████▋ | 195633/225000 [371:29:12<70:06:31,  8.59s/it]
 87%|████████▋ | 195634/225000 [371:29:24<77:34:18,  9.51s/it]
 87%|████████▋ | 195635/225000 [371:29:32<73:15:08,  8.98s/it]
 87%|████████▋ | 195636/225000 [371:29:38<67:56:07,  8.33s/it]
 87%|████████▋ | 195637/225000 [371:29:48<69:39:54,  8.54s/it]
 87%|████████▋ | 195638/225000 [371:29:53<61:18:33,  7.52s/it]
 87%|████████▋ | 195639/225000 [371:29:59<57:25:51,  7.04s/it]
 87%|████████▋ | 195640/225000 [371:30:11<71:18:36,  8.74s/it]
 87%|████████▋ | 195641/225000 [371:30:20<72:21:24,  8.87s/it]
 87%|████████▋ | 195642/225000 [371:30:29<71:45:00,  8.80s/it]
 87%|████████▋ | 195643/225000 [371:30:40<76:45:12,  9.41s/it]
 87%|████████▋ | 195644/225000 [371:30:46<67:41:59,  8.30s/it]
 87%|████████▋ | 195645/225000 [371:30:52<62:54:41,  7.72s/it]
 87%|████████▋ | 195646/225000 [371:31:01<65:01:04,  7.97s/it]
 87%|████████▋ | 195647/225000 [371:31:09<66:23:51,  8.14s/it]
 87%|████████▋ | 195648/225000 [371:31:22<76:52:14,  9.43s/it]
 87%|████████▋ | 195649/225000 [371:31:27<67:56:06,  8.33s/it]
 87%|████████▋ | 195650/225000 [371:31:40<79:28:41,  9.75s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.118', 'learning_rate': '2.278e-06', 'epoch': '0.8696', 'num_input_tokens_seen': 620283104, 'train_runtime': '1.338e+06', 'train_tokens_per_second': '463.8'}

 87%|████████▋ | 195650/225000 [371:31:40<79:28:41,  9.75s/it]
 87%|████████▋ | 195651/225000 [371:31:46<69:16:25,  8.50s/it]
 87%|████████▋ | 195652/225000 [371:31:53<65:50:07,  8.08s/it]
 87%|████████▋ | 195653/225000 [371:31:59<60:12:37,  7.39s/it]
 87%|████████▋ | 195654/225000 [371:32:04<55:05:22,  6.76s/it]
 87%|████████▋ | 195655/225000 [371:32:17<71:06:47,  8.72s/it]
 87%|████████▋ | 195656/225000 [371:32:23<63:28:17,  7.79s/it]
 87%|████████▋ | 195657/225000 [371:32:34<71:08:15,  8.73s/it]
 87%|████████▋ | 195658/225000 [371:32:44<73:18:14,  8.99s/it]
 87%|████████▋ | 195659/225000 [371:32:52<70:46:11,  8.68s/it]
 87%|████████▋ | 195660/225000 [371:33:00<71:04:44,  8.72s/it]
 87%|████████▋ | 195661/225000 [371:33:10<72:20:40,  8.88s/it]
 87%|████████▋ | 195662/225000 [371:33:19<74:28:25,  9.14s/it]
 87%|████████▋ | 195663/225000 [371:33:25<65:12:02,  8.00s/it]
 87%|████████▋ | 195664/225000 [371:33:36<72:49:35,  8.94s/it]
 87%|████████▋ | 195665/225000 [371:33:43<68:04:13,  8.35s/it]
 87%|████████▋ | 195666/225000 [371:33:48<60:58:54,  7.48s/it]
 87%|████████▋ | 195667/225000 [371:33:54<57:03:01,  7.00s/it]
 87%|████████▋ | 195668/225000 [371:34:05<65:43:35,  8.07s/it]
 87%|████████▋ | 195669/225000 [371:34:09<56:33:56,  6.94s/it]
 87%|████████▋ | 195670/225000 [371:34:20<66:11:26,  8.12s/it]
 87%|████████▋ | 195671/225000 [371:34:29<69:51:14,  8.57s/it]
 87%|████████▋ | 195672/225000 [371:34:40<74:23:01,  9.13s/it]
 87%|████████▋ | 195673/225000 [371:34:50<76:36:00,  9.40s/it]
 87%|████████▋ | 195674/225000 [371:34:57<71:47:10,  8.81s/it]
 87%|████████▋ | 195675/225000 [371:35:04<65:11:33,  8.00s/it]
 87%|████████▋ | 195676/225000 [371:35:10<60:19:55,  7.41s/it]
 87%|████████▋ | 195677/225000 [371:35:17<60:21:35,  7.41s/it]
 87%|████████▋ | 195678/225000 [371:35:31<75:56:35,  9.32s/it]
 87%|████████▋ | 195679/225000 [371:35:35<64:34:14,  7.93s/it]
 87%|████████▋ | 195680/225000 [371:35:41<58:31:14,  7.19s/it]
 87%|████████▋ | 195681/225000 [371:35:51<66:23:13,  8.15s/it]
 87%|████████▋ | 195682/225000 [371:35:59<64:16:33,  7.89s/it]
 87%|████████▋ | 195683/225000 [371:36:06<62:28:03,  7.67s/it]
 87%|████████▋ | 195684/225000 [371:36:13<60:48:56,  7.47s/it]
 87%|████████▋ | 195685/225000 [371:36:19<56:45:28,  6.97s/it]
 87%|████████▋ | 195686/225000 [371:36:32<73:06:57,  8.98s/it]
 87%|████████▋ | 195687/225000 [371:36:45<82:07:18, 10.09s/it]
 87%|████████▋ | 195688/225000 [371:36:51<72:54:45,  8.95s/it]
 87%|████████▋ | 195689/225000 [371:36:59<69:14:26,  8.50s/it]
 87%|████████▋ | 195690/225000 [371:37:08<72:22:00,  8.89s/it]
 87%|████████▋ | 195691/225000 [371:37:15<66:41:03,  8.19s/it]
 87%|████████▋ | 195692/225000 [371:37:22<63:45:29,  7.83s/it]
 87%|████████▋ | 195693/225000 [371:37:28<58:12:29,  7.15s/it]
 87%|████████▋ | 195694/225000 [371:37:45<84:13:21, 10.35s/it]
 87%|████████▋ | 195695/225000 [371:37:51<72:16:23,  8.88s/it]
 87%|████████▋ | 195696/225000 [371:38:03<81:06:20,  9.96s/it]
 87%|████████▋ | 195697/225000 [371:38:12<77:49:09,  9.56s/it]
 87%|████████▋ | 195698/225000 [371:38:19<72:11:48,  8.87s/it]
 87%|████████▋ | 195699/225000 [371:38:29<75:04:56,  9.22s/it]
 87%|████████▋ | 195700/225000 [371:38:37<71:11:23,  8.75s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.516', 'learning_rate': '2.27e-06', 'epoch': '0.8698', 'num_input_tokens_seen': 620509784, 'train_runtime': '1.338e+06', 'train_tokens_per_second': '463.8'}

 87%|████████▋ | 195700/225000 [371:38:37<71:11:23,  8.75s/it]
 87%|████████▋ | 195701/225000 [371:38:44<68:16:41,  8.39s/it]
 87%|████████▋ | 195702/225000 [371:38:51<65:09:26,  8.01s/it]
 87%|████████▋ | 195703/225000 [371:39:05<79:06:29,  9.72s/it]
 87%|████████▋ | 195704/225000 [371:39:12<72:56:54,  8.96s/it]
 87%|████████▋ | 195705/225000 [371:39:20<70:12:35,  8.63s/it]
 87%|████████▋ | 195706/225000 [371:39:27<65:42:58,  8.08s/it]
 87%|████████▋ | 195707/225000 [371:39:38<72:59:56,  8.97s/it]
 87%|████████▋ | 195708/225000 [371:39:43<63:27:23,  7.80s/it]
 87%|████████▋ | 195709/225000 [371:39:55<73:19:33,  9.01s/it]
 87%|████████▋ | 195710/225000 [371:40:01<65:05:04,  8.00s/it]
 87%|████████▋ | 195711/225000 [371:40:08<64:38:31,  7.95s/it]
 87%|████████▋ | 195712/225000 [371:40:16<62:48:52,  7.72s/it]
 87%|████████▋ | 195713/225000 [371:40:26<70:04:16,  8.61s/it]
 87%|████████▋ | 195714/225000 [371:40:34<67:38:30,  8.31s/it]
 87%|████████▋ | 195715/225000 [371:40:42<67:17:08,  8.27s/it]
 87%|████████▋ | 195716/225000 [371:40:51<68:34:51,  8.43s/it]
 87%|████████▋ | 195717/225000 [371:41:03<77:38:37,  9.55s/it]
 87%|████████▋ | 195718/225000 [371:41:09<68:45:57,  8.45s/it]
 87%|████████▋ | 195719/225000 [371:41:15<62:36:53,  7.70s/it]
 87%|████████▋ | 195720/225000 [371:41:31<83:26:11, 10.26s/it]
 87%|████████▋ | 195721/225000 [371:41:43<86:32:44, 10.64s/it]
 87%|████████▋ | 195722/225000 [371:41:52<84:22:13, 10.37s/it]
 87%|████████▋ | 195723/225000 [371:42:06<90:57:24, 11.18s/it]
 87%|████████▋ | 195724/225000 [371:42:11<76:29:16,  9.41s/it]
 87%|████████▋ | 195725/225000 [371:42:20<75:57:59,  9.34s/it]
 87%|████████▋ | 195726/225000 [371:42:26<68:36:19,  8.44s/it]
 87%|████████▋ | 195727/225000 [371:42:32<62:27:35,  7.68s/it]
 87%|████████▋ | 195728/225000 [371:42:39<60:41:23,  7.46s/it]
 87%|████████▋ | 195729/225000 [371:42:51<70:48:55,  8.71s/it]
 87%|████████▋ | 195730/225000 [371:42:57<64:44:48,  7.96s/it]
 87%|████████▋ | 195731/225000 [371:43:04<62:35:52,  7.70s/it]
 87%|████████▋ | 195732/225000 [371:43:15<70:40:56,  8.69s/it]
 87%|████████▋ | 195733/225000 [371:43:24<70:49:53,  8.71s/it]
 87%|████████▋ | 195734/225000 [371:43:29<61:41:30,  7.59s/it]
 87%|████████▋ | 195735/225000 [371:43:41<72:14:29,  8.89s/it]
 87%|████████▋ | 195736/225000 [371:43:51<74:46:44,  9.20s/it]
 87%|████████▋ | 195737/225000 [371:43:58<70:14:27,  8.64s/it]
 87%|████████▋ | 195738/225000 [371:44:05<65:45:17,  8.09s/it]
 87%|████████▋ | 195739/225000 [371:44:11<61:30:24,  7.57s/it]
 87%|████████▋ | 195740/225000 [371:44:18<59:26:27,  7.31s/it]
 87%|████████▋ | 195741/225000 [371:44:30<71:30:47,  8.80s/it]
 87%|████████▋ | 195742/225000 [371:44:45<85:20:51, 10.50s/it]
 87%|████████▋ | 195743/225000 [371:44:54<83:28:45, 10.27s/it]
 87%|████████▋ | 195744/225000 [371:45:03<78:44:04,  9.69s/it]
 87%|████████▋ | 195745/225000 [371:45:18<91:42:53, 11.29s/it]
 87%|████████▋ | 195746/225000 [371:45:28<88:16:10, 10.86s/it]
 87%|████████▋ | 195747/225000 [371:45:36<82:24:36, 10.14s/it]
 87%|████████▋ | 195748/225000 [371:45:41<69:49:38,  8.59s/it]
 87%|████████▋ | 195749/225000 [371:45:49<67:44:35,  8.34s/it]
 87%|████████▋ | 195750/225000 [371:45:59<71:14:45,  8.77s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.155', 'learning_rate': '2.263e-06', 'epoch': '0.87', 'num_input_tokens_seen': 620737104, 'train_runtime': '1.338e+06', 'train_tokens_per_second': '463.8'}

 87%|████████▋ | 195750/225000 [371:45:59<71:14:45,  8.77s/it]
 87%|████████▋ | 195751/225000 [371:46:12<81:32:00, 10.04s/it]
 87%|████████▋ | 195752/225000 [371:46:18<73:08:17,  9.00s/it]
 87%|████████▋ | 195753/225000 [371:46:26<70:58:30,  8.74s/it]
 87%|████████▋ | 195754/225000 [371:46:33<66:28:49,  8.18s/it]
 87%|████████▋ | 195755/225000 [371:46:41<64:31:25,  7.94s/it]
 87%|████████▋ | 195756/225000 [371:46:54<77:48:51,  9.58s/it]
 87%|████████▋ | 195757/225000 [371:47:01<72:51:45,  8.97s/it]
 87%|████████▋ | 195758/225000 [371:47:07<65:15:12,  8.03s/it]
 87%|████████▋ | 195759/225000 [371:47:15<63:37:54,  7.83s/it]
 87%|████████▋ | 195760/225000 [371:47:21<60:36:57,  7.46s/it]
 87%|████████▋ | 195761/225000 [371:47:27<56:07:44,  6.91s/it]
 87%|████████▋ | 195762/225000 [371:47:36<61:00:11,  7.51s/it]
 87%|████████▋ | 195763/225000 [371:47:41<55:41:28,  6.86s/it]
 87%|████████▋ | 195764/225000 [371:47:56<74:00:52,  9.11s/it]
 87%|████████▋ | 195765/225000 [371:48:04<72:41:25,  8.95s/it]
 87%|████████▋ | 195766/225000 [371:48:09<63:17:42,  7.79s/it]
 87%|████████▋ | 195767/225000 [371:48:19<69:04:15,  8.51s/it]
 87%|████████▋ | 195768/225000 [371:48:29<71:03:54,  8.75s/it]
 87%|████████▋ | 195769/225000 [371:48:36<68:14:28,  8.40s/it]
 87%|████████▋ | 195770/225000 [371:48:44<66:38:30,  8.21s/it]
 87%|████████▋ | 195771/225000 [371:48:57<77:26:54,  9.54s/it]
 87%|████████▋ | 195772/225000 [371:49:07<79:33:28,  9.80s/it]
 87%|████████▋ | 195773/225000 [371:49:15<75:42:03,  9.32s/it]
 87%|████████▋ | 195774/225000 [371:49:21<67:44:50,  8.34s/it]
 87%|████████▋ | 195775/225000 [371:49:28<64:24:44,  7.93s/it]
 87%|████████▋ | 195776/225000 [371:49:37<65:02:51,  8.01s/it]
 87%|████████▋ | 195777/225000 [371:49:47<69:57:32,  8.62s/it]
 87%|████████▋ | 195778/225000 [371:49:55<69:49:56,  8.60s/it]
 87%|████████▋ | 195779/225000 [371:50:05<72:07:50,  8.89s/it]
 87%|████████▋ | 195780/225000 [371:50:14<73:39:19,  9.07s/it]
 87%|████████▋ | 195781/225000 [371:50:23<74:06:45,  9.13s/it]
 87%|████████▋ | 195782/225000 [371:50:32<72:55:43,  8.99s/it]
 87%|████████▋ | 195783/225000 [371:50:44<80:40:06,  9.94s/it]
 87%|████████▋ | 195784/225000 [371:50:51<73:20:32,  9.04s/it]
 87%|████████▋ | 195785/225000 [371:50:56<63:45:59,  7.86s/it]
 87%|████████▋ | 195786/225000 [371:51:04<64:02:24,  7.89s/it]
 87%|████████▋ | 195787/225000 [371:51:13<65:28:12,  8.07s/it]
 87%|████████▋ | 195788/225000 [371:51:18<57:37:06,  7.10s/it]
 87%|████████▋ | 195789/225000 [371:51:25<58:13:25,  7.18s/it]
 87%|████████▋ | 195790/225000 [371:51:39<74:52:13,  9.23s/it]
 87%|████████▋ | 195791/225000 [371:51:46<70:32:54,  8.70s/it]
 87%|████████▋ | 195792/225000 [371:51:53<65:00:36,  8.01s/it]
 87%|████████▋ | 195793/225000 [371:52:04<72:49:06,  8.98s/it]
 87%|████████▋ | 195794/225000 [371:52:15<77:21:30,  9.54s/it]
 87%|████████▋ | 195795/225000 [371:52:24<77:19:06,  9.53s/it]
 87%|████████▋ | 195796/225000 [371:52:33<75:06:32,  9.26s/it]
 87%|████████▋ | 195797/225000 [371:52:42<74:00:44,  9.12s/it]
 87%|████████▋ | 195798/225000 [371:52:53<79:47:28,  9.84s/it]
 87%|████████▋ | 195799/225000 [371:53:00<72:00:51,  8.88s/it]
 87%|████████▋ | 195800/225000 [371:53:05<62:44:16,  7.73s/it]
                                                              
{'loss': '1.354', 'grad_norm': '1.521', 'learning_rate': '2.255e-06', 'epoch': '0.8702', 'num_input_tokens_seen': 620965920, 'train_runtime': '1.339e+06', 'train_tokens_per_second': '463.8'}

 87%|████████▋ | 195800/225000 [371:53:05<62:44:16,  7.73s/it]
 87%|████████▋ | 195801/225000 [371:53:17<74:10:45,  9.15s/it]
 87%|████████▋ | 195802/225000 [371:53:26<72:31:42,  8.94s/it]
 87%|████████▋ | 195803/225000 [371:53:37<77:44:11,  9.58s/it]
 87%|████████▋ | 195804/225000 [371:53:43<68:03:37,  8.39s/it]
 87%|████████▋ | 195805/225000 [371:53:50<66:11:41,  8.16s/it]
 87%|████████▋ | 195806/225000 [371:54:01<73:35:42,  9.08s/it]
 87%|████████▋ | 195807/225000 [371:54:17<88:28:26, 10.91s/it]
 87%|████████▋ | 195808/225000 [371:54:24<80:01:11,  9.87s/it]
 87%|████████▋ | 195809/225000 [371:54:34<80:18:00,  9.90s/it]
 87%|████████▋ | 195810/225000 [371:54:44<79:44:08,  9.83s/it]
 87%|████████▋ | 195811/225000 [371:54:51<73:12:29,  9.03s/it]
 87%|████████▋ | 195812/225000 [371:54:58<67:48:23,  8.36s/it]
 87%|████████▋ | 195813/225000 [371:55:12<83:10:25, 10.26s/it]
 87%|████████▋ | 195814/225000 [371:55:22<81:39:13, 10.07s/it]
 87%|████████▋ | 195815/225000 [371:55:31<78:48:37,  9.72s/it]
 87%|████████▋ | 195816/225000 [371:55:38<71:35:35,  8.83s/it]
 87%|████████▋ | 195817/225000 [371:55:46<69:45:02,  8.60s/it]
 87%|████████▋ | 195818/225000 [371:56:00<83:56:12, 10.35s/it]
 87%|████████▋ | 195819/225000 [371:56:08<77:32:19,  9.57s/it]
 87%|████████▋ | 195820/225000 [371:56:17<75:50:26,  9.36s/it]
 87%|████████▋ | 195821/225000 [371:56:24<71:18:06,  8.80s/it]
 87%|████████▋ | 195822/225000 [371:56:37<79:39:32,  9.83s/it]
 87%|████████▋ | 195823/225000 [371:56:45<75:15:21,  9.29s/it]
 87%|████████▋ | 195824/225000 [371:56:50<66:37:35,  8.22s/it]
 87%|████████▋ | 195825/225000 [371:56:58<66:27:51,  8.20s/it]
 87%|████████▋ | 195826/225000 [371:57:06<64:28:54,  7.96s/it]
 87%|████████▋ | 195827/225000 [371:57:12<61:06:00,  7.54s/it]
 87%|████████▋ | 195828/225000 [371:57:25<73:04:06,  9.02s/it]
 87%|████████▋ | 195829/225000 [371:57:30<64:23:55,  7.95s/it]
 87%|████████▋ | 195830/225000 [371:57:37<62:02:11,  7.66s/it]
 87%|████████▋ | 195831/225000 [371:57:47<67:28:21,  8.33s/it]
 87%|████████▋ | 195832/225000 [371:57:55<65:45:16,  8.12s/it]
 87%|████████▋ | 195833/225000 [371:58:03<66:39:49,  8.23s/it]
 87%|████████▋ | 195834/225000 [371:58:07<56:02:26,  6.92s/it]
 87%|████████▋ | 195835/225000 [371:58:17<63:02:32,  7.78s/it]
 87%|████████▋ | 195836/225000 [371:58:25<63:27:22,  7.83s/it]
 87%|████████▋ | 195837/225000 [371:58:31<58:30:06,  7.22s/it]
 87%|████████▋ | 195838/225000 [371:58:38<59:41:09,  7.37s/it]
 87%|████████▋ | 195839/225000 [371:58:44<54:25:50,  6.72s/it]
 87%|████████▋ | 195840/225000 [371:58:54<62:29:03,  7.71s/it]
 87%|████████▋ | 195841/225000 [371:59:02<63:10:15,  7.80s/it]
 87%|████████▋ | 195842/225000 [371:59:09<62:19:10,  7.69s/it]
 87%|████████▋ | 195843/225000 [371:59:17<61:43:09,  7.62s/it]
 87%|████████▋ | 195844/225000 [371:59:25<64:38:50,  7.98s/it]
 87%|████████▋ | 195845/225000 [371:59:38<75:53:31,  9.37s/it]
 87%|████████▋ | 195846/225000 [371:59:44<67:09:26,  8.29s/it]
 87%|████████▋ | 195847/225000 [371:59:53<68:47:09,  8.49s/it]
 87%|████████▋ | 195848/225000 [371:59:59<63:44:54,  7.87s/it]
 87%|████████▋ | 195849/225000 [372:00:08<65:09:29,  8.05s/it]
 87%|████████▋ | 195850/225000 [372:00:14<60:14:59,  7.44s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.272', 'learning_rate': '2.248e-06', 'epoch': '0.8704', 'num_input_tokens_seen': 621192952, 'train_runtime': '1.339e+06', 'train_tokens_per_second': '463.8'}

 87%|████████▋ | 195850/225000 [372:00:14<60:14:59,  7.44s/it]
 87%|████████▋ | 195851/225000 [372:00:20<57:14:36,  7.07s/it]
 87%|████████▋ | 195852/225000 [372:00:34<73:43:19,  9.11s/it]
 87%|████████▋ | 195853/225000 [372:00:44<77:21:13,  9.55s/it]
 87%|████████▋ | 195854/225000 [372:00:49<65:46:36,  8.12s/it]
 87%|████████▋ | 195855/225000 [372:00:57<65:35:53,  8.10s/it]
 87%|████████▋ | 195856/225000 [372:01:04<61:33:10,  7.60s/it]
 87%|████████▋ | 195857/225000 [372:01:11<59:56:59,  7.41s/it]
 87%|████████▋ | 195858/225000 [372:01:16<54:48:50,  6.77s/it]
 87%|████████▋ | 195859/225000 [372:01:29<69:32:40,  8.59s/it]
 87%|████████▋ | 195860/225000 [372:01:37<68:06:07,  8.41s/it]
 87%|████████▋ | 195861/225000 [372:01:45<68:42:03,  8.49s/it]
 87%|████████▋ | 195862/225000 [372:01:54<69:54:39,  8.64s/it]
 87%|████████▋ | 195863/225000 [372:02:02<67:07:59,  8.29s/it]
 87%|████████▋ | 195864/225000 [372:02:09<64:54:12,  8.02s/it]
 87%|████████▋ | 195865/225000 [372:02:15<60:17:18,  7.45s/it]
 87%|████████▋ | 195866/225000 [372:02:22<59:16:49,  7.33s/it]
 87%|████████▋ | 195867/225000 [372:02:31<62:11:09,  7.68s/it]
 87%|████████▋ | 195868/225000 [372:02:44<76:31:14,  9.46s/it]
 87%|████████▋ | 195869/225000 [372:02:56<81:47:27, 10.11s/it]
 87%|████████▋ | 195870/225000 [372:03:04<76:18:13,  9.43s/it]
 87%|████████▋ | 195871/225000 [372:03:13<75:12:48,  9.30s/it]
 87%|████████▋ | 195872/225000 [372:03:19<68:40:10,  8.49s/it]
 87%|████████▋ | 195873/225000 [372:03:33<80:33:10,  9.96s/it]
 87%|████████▋ | 195874/225000 [372:03:40<73:15:47,  9.06s/it]
 87%|████████▋ | 195875/225000 [372:03:51<77:59:08,  9.64s/it]
 87%|████████▋ | 195876/225000 [372:03:57<70:09:57,  8.67s/it]
 87%|████████▋ | 195877/225000 [372:04:04<64:52:09,  8.02s/it]
 87%|████████▋ | 195878/225000 [372:04:12<65:58:22,  8.16s/it]
 87%|████████▋ | 195879/225000 [372:04:19<63:30:37,  7.85s/it]
 87%|████████▋ | 195880/225000 [372:04:25<57:28:30,  7.11s/it]
 87%|████████▋ | 195881/225000 [372:04:31<56:01:27,  6.93s/it]
 87%|████████▋ | 195882/225000 [372:04:38<56:17:02,  6.96s/it]
 87%|████████▋ | 195883/225000 [372:04:49<65:48:31,  8.14s/it]
 87%|████████▋ | 195884/225000 [372:04:56<62:04:00,  7.67s/it]
 87%|████████▋ | 195885/225000 [372:05:11<80:19:08,  9.93s/it]
 87%|████████▋ | 195886/225000 [372:05:18<73:42:59,  9.12s/it]
 87%|████████▋ | 195887/225000 [372:05:28<75:25:20,  9.33s/it]
 87%|████████▋ | 195888/225000 [372:05:37<74:12:46,  9.18s/it]
 87%|████████▋ | 195889/225000 [372:05:43<68:05:38,  8.42s/it]
 87%|████████▋ | 195890/225000 [372:05:53<70:59:05,  8.78s/it]
 87%|████████▋ | 195891/225000 [372:05:58<62:48:07,  7.77s/it]
 87%|████████▋ | 195892/225000 [372:06:06<61:45:11,  7.64s/it]
 87%|████████▋ | 195893/225000 [372:06:14<63:25:28,  7.84s/it]
 87%|████████▋ | 195894/225000 [372:06:29<80:55:18, 10.01s/it]
 87%|████████▋ | 195895/225000 [372:06:38<78:09:28,  9.67s/it]
 87%|████████▋ | 195896/225000 [372:06:45<71:21:09,  8.83s/it]
 87%|████████▋ | 195897/225000 [372:06:49<60:33:38,  7.49s/it]
 87%|████████▋ | 195898/225000 [372:07:03<76:35:15,  9.47s/it]
 87%|████████▋ | 195899/225000 [372:07:11<73:11:48,  9.05s/it]
 87%|████████▋ | 195900/225000 [372:07:19<70:17:08,  8.70s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.299', 'learning_rate': '2.24e-06', 'epoch': '0.8707', 'num_input_tokens_seen': 621417672, 'train_runtime': '1.34e+06', 'train_tokens_per_second': '463.9'}

 87%|████████▋ | 195900/225000 [372:07:19<70:17:08,  8.70s/it]
 87%|████████▋ | 195901/225000 [372:07:23<59:09:04,  7.32s/it]
 87%|████████▋ | 195902/225000 [372:07:36<71:37:45,  8.86s/it]
 87%|████████▋ | 195903/225000 [372:07:47<77:23:45,  9.58s/it]
 87%|████████▋ | 195904/225000 [372:07:53<68:40:11,  8.50s/it]
 87%|████████▋ | 195905/225000 [372:08:05<76:53:06,  9.51s/it]
 87%|████████▋ | 195906/225000 [372:08:14<76:23:24,  9.45s/it]
 87%|████████▋ | 195907/225000 [372:08:21<69:54:31,  8.65s/it]
 87%|████████▋ | 195908/225000 [372:08:27<62:13:22,  7.70s/it]
 87%|████████▋ | 195909/225000 [372:08:33<59:45:13,  7.39s/it]
 87%|████████▋ | 195910/225000 [372:08:39<56:00:33,  6.93s/it]
 87%|████████▋ | 195911/225000 [372:08:46<56:30:56,  6.99s/it]
 87%|████████▋ | 195912/225000 [372:08:54<59:22:51,  7.35s/it]
 87%|████████▋ | 195913/225000 [372:09:09<77:58:56,  9.65s/it]
 87%|████████▋ | 195914/225000 [372:09:18<74:53:27,  9.27s/it]
 87%|████████▋ | 195915/225000 [372:09:24<68:27:23,  8.47s/it]
 87%|████████▋ | 195916/225000 [372:09:38<80:38:37,  9.98s/it]
 87%|████████▋ | 195917/225000 [372:09:47<77:14:13,  9.56s/it]
 87%|████████▋ | 195918/225000 [372:09:56<76:50:23,  9.51s/it]
 87%|████████▋ | 195919/225000 [372:10:06<78:29:39,  9.72s/it]
 87%|████████▋ | 195920/225000 [372:10:12<69:56:09,  8.66s/it]
 87%|████████▋ | 195921/225000 [372:10:23<75:56:15,  9.40s/it]
 87%|████████▋ | 195922/225000 [372:10:30<69:42:50,  8.63s/it]
 87%|████████▋ | 195923/225000 [372:10:37<65:57:29,  8.17s/it]
 87%|████████▋ | 195924/225000 [372:10:44<62:56:08,  7.79s/it]
 87%|████████▋ | 195925/225000 [372:10:54<68:40:28,  8.50s/it]
 87%|████████▋ | 195926/225000 [372:11:02<65:18:54,  8.09s/it]
 87%|████████▋ | 195927/225000 [372:11:13<72:47:12,  9.01s/it]
 87%|████████▋ | 195928/225000 [372:11:23<75:20:53,  9.33s/it]
 87%|████████▋ | 195929/225000 [372:11:32<75:52:13,  9.40s/it]
 87%|████████▋ | 195930/225000 [372:11:38<66:52:51,  8.28s/it]
 87%|████████▋ | 195931/225000 [372:11:45<63:20:50,  7.85s/it]
 87%|████████▋ | 195932/225000 [372:11:53<64:04:33,  7.94s/it]
 87%|████████▋ | 195933/225000 [372:12:00<61:15:37,  7.59s/it]
 87%|████████▋ | 195934/225000 [372:12:10<67:06:58,  8.31s/it]
 87%|████████▋ | 195935/225000 [372:12:15<60:47:13,  7.53s/it]
 87%|████████▋ | 195936/225000 [372:12:25<65:21:50,  8.10s/it]
 87%|████████▋ | 195937/225000 [372:12:36<73:00:41,  9.04s/it]
 87%|████████▋ | 195938/225000 [372:12:45<73:25:12,  9.09s/it]
 87%|████████▋ | 195939/225000 [372:12:54<71:06:06,  8.81s/it]
 87%|████████▋ | 195940/225000 [372:13:00<66:20:35,  8.22s/it]
 87%|████████▋ | 195941/225000 [372:13:08<65:41:11,  8.14s/it]
 87%|████████▋ | 195942/225000 [372:13:16<63:39:03,  7.89s/it]
 87%|████████▋ | 195943/225000 [372:13:23<63:30:09,  7.87s/it]
 87%|████████▋ | 195944/225000 [372:13:32<66:08:36,  8.20s/it]
 87%|████████▋ | 195945/225000 [372:13:45<77:22:48,  9.59s/it]
 87%|████████▋ | 195946/225000 [372:13:50<66:47:24,  8.28s/it]
 87%|████████▋ | 195947/225000 [372:14:00<69:24:25,  8.60s/it]
 87%|████████▋ | 195948/225000 [372:14:06<63:12:04,  7.83s/it]
 87%|████████▋ | 195949/225000 [372:14:21<80:30:35,  9.98s/it]
 87%|████████▋ | 195950/225000 [372:14:27<71:18:48,  8.84s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.244', 'learning_rate': '2.233e-06', 'epoch': '0.8709', 'num_input_tokens_seen': 621640648, 'train_runtime': '1.34e+06', 'train_tokens_per_second': '463.9'}

 87%|████████▋ | 195950/225000 [372:14:27<71:18:48,  8.84s/it]
 87%|████████▋ | 195951/225000 [372:14:35<69:16:32,  8.59s/it]
 87%|████████▋ | 195952/225000 [372:14:43<68:45:30,  8.52s/it]
 87%|████████▋ | 195953/225000 [372:14:49<61:38:26,  7.64s/it]
 87%|████████▋ | 195954/225000 [372:14:58<64:02:08,  7.94s/it]
 87%|████████▋ | 195955/225000 [372:15:03<57:56:31,  7.18s/it]
 87%|████████▋ | 195956/225000 [372:15:09<54:05:00,  6.70s/it]
 87%|████████▋ | 195957/225000 [372:15:19<63:21:59,  7.85s/it]
 87%|████████▋ | 195958/225000 [372:15:30<70:43:58,  8.77s/it]
 87%|████████▋ | 195959/225000 [372:15:41<76:01:44,  9.42s/it]
 87%|████████▋ | 195960/225000 [372:15:48<70:50:01,  8.78s/it]
 87%|████████▋ | 195961/225000 [372:15:55<65:54:02,  8.17s/it]
 87%|████████▋ | 195962/225000 [372:16:02<61:53:08,  7.67s/it]
 87%|████████▋ | 195963/225000 [372:16:14<73:22:40,  9.10s/it]
 87%|████████▋ | 195964/225000 [372:16:23<72:49:36,  9.03s/it]
 87%|████████▋ | 195965/225000 [372:16:29<66:14:05,  8.21s/it]
 87%|████████▋ | 195966/225000 [372:16:36<62:40:32,  7.77s/it]
 87%|████████▋ | 195967/225000 [372:16:47<71:25:06,  8.86s/it]
 87%|████████▋ | 195968/225000 [372:16:57<73:17:32,  9.09s/it]
 87%|████████▋ | 195969/225000 [372:17:06<73:37:17,  9.13s/it]
 87%|████████▋ | 195970/225000 [372:17:21<86:46:23, 10.76s/it]
 87%|████████▋ | 195971/225000 [372:17:28<77:25:13,  9.60s/it]
 87%|████████▋ | 195972/225000 [372:17:37<77:30:06,  9.61s/it]
 87%|████████▋ | 195973/225000 [372:17:44<70:29:08,  8.74s/it]
 87%|████████▋ | 195974/225000 [372:17:50<64:06:23,  7.95s/it]
 87%|████████▋ | 195975/225000 [372:17:58<64:19:33,  7.98s/it]
 87%|████████▋ | 195976/225000 [372:18:04<60:13:57,  7.47s/it]
 87%|████████▋ | 195977/225000 [372:18:16<70:17:14,  8.72s/it]
 87%|████████▋ | 195978/225000 [372:18:24<68:15:11,  8.47s/it]
 87%|████████▋ | 195979/225000 [372:18:32<68:35:21,  8.51s/it]
 87%|████████▋ | 195980/225000 [372:18:39<64:45:25,  8.03s/it]
 87%|████████▋ | 195981/225000 [372:18:45<59:38:11,  7.40s/it]
 87%|████████▋ | 195982/225000 [372:18:57<70:29:21,  8.74s/it]
 87%|████████▋ | 195983/225000 [372:19:04<66:04:56,  8.20s/it]
 87%|████████▋ | 195984/225000 [372:19:11<63:27:34,  7.87s/it]
 87%|████████▋ | 195985/225000 [372:19:23<72:56:48,  9.05s/it]
 87%|████████▋ | 195986/225000 [372:19:29<64:58:33,  8.06s/it]
 87%|████████▋ | 195987/225000 [372:19:44<81:13:26, 10.08s/it]
 87%|████████▋ | 195988/225000 [372:19:52<77:10:29,  9.58s/it]
 87%|████████▋ | 195989/225000 [372:20:01<75:16:27,  9.34s/it]
 87%|████████▋ | 195990/225000 [372:20:10<74:32:57,  9.25s/it]
 87%|████████▋ | 195991/225000 [372:20:14<62:24:14,  7.74s/it]
 87%|████████▋ | 195992/225000 [372:20:24<66:44:15,  8.28s/it]
 87%|████████▋ | 195993/225000 [372:20:33<68:31:26,  8.50s/it]
 87%|████████▋ | 195994/225000 [372:20:42<69:43:44,  8.65s/it]
 87%|████████▋ | 195995/225000 [372:20:49<67:05:35,  8.33s/it]
 87%|████████▋ | 195996/225000 [372:20:57<66:35:58,  8.27s/it]
 87%|████████▋ | 195997/225000 [372:21:03<59:34:29,  7.39s/it]
 87%|████████▋ | 195998/225000 [372:21:08<54:55:24,  6.82s/it]
 87%|████████▋ | 195999/225000 [372:21:15<54:23:09,  6.75s/it]
 87%|████████▋ | 196000/225000 [372:21:22<54:39:46,  6.79s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.549', 'learning_rate': '2.225e-06', 'epoch': '0.8711', 'num_input_tokens_seen': 621862768, 'train_runtime': '1.34e+06', 'train_tokens_per_second': '463.9'}

 87%|████████▋ | 196000/225000 [372:21:22<54:39:46,  6.79s/it]
 87%|████████▋ | 196001/225000 [372:21:41<84:33:09, 10.50s/it]
 87%|████████▋ | 196002/225000 [372:21:47<73:58:13,  9.18s/it]
 87%|████████▋ | 196003/225000 [372:21:54<68:17:26,  8.48s/it]
 87%|████████▋ | 196004/225000 [372:22:05<74:05:34,  9.20s/it]
 87%|████████▋ | 196005/225000 [372:22:10<64:39:50,  8.03s/it]
 87%|████████▋ | 196006/225000 [372:22:20<70:01:39,  8.69s/it]
 87%|████████▋ | 196007/225000 [372:22:27<64:35:15,  8.02s/it]
 87%|████████▋ | 196008/225000 [372:22:32<58:55:36,  7.32s/it]
 87%|████████▋ | 196009/225000 [372:22:38<54:15:33,  6.74s/it]
 87%|████████▋ | 196010/225000 [372:22:44<53:23:47,  6.63s/it]
 87%|████████▋ | 196011/225000 [372:22:49<49:40:48,  6.17s/it]
 87%|████████▋ | 196012/225000 [372:23:06<75:46:20,  9.41s/it]
 87%|████████▋ | 196013/225000 [372:23:10<62:53:04,  7.81s/it]
 87%|████████▋ | 196014/225000 [372:23:16<57:47:36,  7.18s/it]
 87%|████████▋ | 196015/225000 [372:23:28<69:15:13,  8.60s/it]
 87%|████████▋ | 196016/225000 [372:23:36<68:33:11,  8.51s/it]
 87%|████████▋ | 196017/225000 [372:23:51<82:51:45, 10.29s/it]
 87%|████████▋ | 196018/225000 [372:24:01<84:00:45, 10.44s/it]
 87%|████████▋ | 196019/225000 [372:24:09<76:14:25,  9.47s/it]
 87%|████████▋ | 196020/225000 [372:24:16<71:51:35,  8.93s/it]
 87%|████████▋ | 196021/225000 [372:24:27<77:07:40,  9.58s/it]
 87%|████████▋ | 196022/225000 [372:24:35<72:42:38,  9.03s/it]
 87%|████████▋ | 196023/225000 [372:24:44<71:47:51,  8.92s/it]
 87%|████████▋ | 196024/225000 [372:24:49<63:21:59,  7.87s/it]
 87%|████████▋ | 196025/225000 [372:24:56<61:08:16,  7.60s/it]
 87%|████████▋ | 196026/225000 [372:25:10<76:13:15,  9.47s/it]
 87%|████████▋ | 196027/225000 [372:25:17<69:46:46,  8.67s/it]
 87%|████████▋ | 196028/225000 [372:25:23<63:47:22,  7.93s/it]
 87%|████████▋ | 196029/225000 [372:25:29<59:32:24,  7.40s/it]
 87%|████████▋ | 196030/225000 [372:25:44<78:07:44,  9.71s/it]
 87%|████████▋ | 196031/225000 [372:25:52<73:08:55,  9.09s/it]
 87%|████████▋ | 196032/225000 [372:25:58<67:04:03,  8.33s/it]
 87%|████████▋ | 196033/225000 [372:26:04<61:19:35,  7.62s/it]
 87%|████████▋ | 196034/225000 [372:26:14<66:06:36,  8.22s/it]
 87%|████████▋ | 196035/225000 [372:26:28<79:07:28,  9.83s/it]
 87%|████████▋ | 196036/225000 [372:26:37<78:04:52,  9.70s/it]
 87%|████████▋ | 196037/225000 [372:26:43<70:05:19,  8.71s/it]
 87%|████████▋ | 196038/225000 [372:26:51<66:24:37,  8.25s/it]
 87%|████████▋ | 196039/225000 [372:27:00<69:50:30,  8.68s/it]
 87%|████████▋ | 196040/225000 [372:27:08<66:30:30,  8.27s/it]
 87%|████████▋ | 196041/225000 [372:27:20<76:06:07,  9.46s/it]
 87%|████████▋ | 196042/225000 [372:27:31<80:58:55, 10.07s/it]
 87%|████████▋ | 196043/225000 [372:27:37<71:23:37,  8.88s/it]
 87%|████████▋ | 196044/225000 [372:27:44<65:48:17,  8.18s/it]
 87%|████████▋ | 196045/225000 [372:27:50<61:55:49,  7.70s/it]
 87%|████████▋ | 196046/225000 [372:28:00<67:01:50,  8.33s/it]
 87%|████████▋ | 196047/225000 [372:28:08<65:54:05,  8.19s/it]
 87%|████████▋ | 196048/225000 [372:28:16<64:57:20,  8.08s/it]
 87%|████████▋ | 196049/225000 [372:28:31<81:54:25, 10.18s/it]
 87%|████████▋ | 196050/225000 [372:28:40<78:44:52,  9.79s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.1', 'learning_rate': '2.217e-06', 'epoch': '0.8713', 'num_input_tokens_seen': 622086512, 'train_runtime': '1.341e+06', 'train_tokens_per_second': '463.9'}

 87%|████████▋ | 196050/225000 [372:28:40<78:44:52,  9.79s/it]
 87%|████████▋ | 196051/225000 [372:28:46<70:06:56,  8.72s/it]
 87%|████████▋ | 196052/225000 [372:28:58<76:46:13,  9.55s/it]
 87%|████████▋ | 196053/225000 [372:29:04<68:09:09,  8.48s/it]
 87%|████████▋ | 196054/225000 [372:29:14<71:56:00,  8.95s/it]
 87%|████████▋ | 196055/225000 [372:29:22<70:34:57,  8.78s/it]
 87%|████████▋ | 196056/225000 [372:29:35<81:06:18, 10.09s/it]
 87%|████████▋ | 196057/225000 [372:29:46<82:31:03, 10.26s/it]
 87%|████████▋ | 196058/225000 [372:29:51<70:52:54,  8.82s/it]
 87%|████████▋ | 196059/225000 [372:30:02<76:09:30,  9.47s/it]
 87%|████████▋ | 196060/225000 [372:30:06<62:09:54,  7.73s/it]
 87%|████████▋ | 196061/225000 [372:30:11<54:43:37,  6.81s/it]
 87%|████████▋ | 196062/225000 [372:30:22<65:50:56,  8.19s/it]
 87%|████████▋ | 196063/225000 [372:30:32<70:57:57,  8.83s/it]
 87%|████████▋ | 196064/225000 [372:30:40<69:10:21,  8.61s/it]
 87%|████████▋ | 196065/225000 [372:30:46<61:02:13,  7.59s/it]
 87%|████████▋ | 196066/225000 [372:30:52<58:12:17,  7.24s/it]
 87%|████████▋ | 196067/225000 [372:31:05<72:40:34,  9.04s/it]
 87%|████████▋ | 196068/225000 [372:31:11<65:35:31,  8.16s/it]
 87%|████████▋ | 196069/225000 [372:31:17<58:24:10,  7.27s/it]
 87%|████████▋ | 196070/225000 [372:31:24<58:22:09,  7.26s/it]
 87%|████████▋ | 196071/225000 [372:31:40<79:43:31,  9.92s/it]
 87%|████████▋ | 196072/225000 [372:31:45<68:30:01,  8.52s/it]
 87%|████████▋ | 196073/225000 [372:31:51<62:48:42,  7.82s/it]
 87%|████████▋ | 196074/225000 [372:32:00<64:18:21,  8.00s/it]
 87%|████████▋ | 196075/225000 [372:32:12<75:00:15,  9.34s/it]
 87%|████████▋ | 196076/225000 [372:32:21<72:28:02,  9.02s/it]
 87%|████████▋ | 196077/225000 [372:32:27<65:29:30,  8.15s/it]
 87%|████████▋ | 196078/225000 [372:32:32<58:57:49,  7.34s/it]
 87%|████████▋ | 196079/225000 [372:32:40<59:37:38,  7.42s/it]
 87%|████████▋ | 196080/225000 [372:32:47<58:00:49,  7.22s/it]
 87%|████████▋ | 196081/225000 [372:32:56<64:05:25,  7.98s/it]
 87%|████████▋ | 196082/225000 [372:33:06<69:16:24,  8.62s/it]
 87%|████████▋ | 196083/225000 [372:33:14<67:42:09,  8.43s/it]
 87%|████████▋ | 196084/225000 [372:33:20<60:12:13,  7.50s/it]
 87%|████████▋ | 196085/225000 [372:33:27<58:43:14,  7.31s/it]
 87%|████████▋ | 196086/225000 [372:33:34<58:54:55,  7.34s/it]
 87%|████████▋ | 196087/225000 [372:33:44<64:22:27,  8.02s/it]
 87%|████████▋ | 196088/225000 [372:33:55<71:52:13,  8.95s/it]
 87%|████████▋ | 196089/225000 [372:34:07<79:54:08,  9.95s/it]
 87%|████████▋ | 196090/225000 [372:34:13<69:20:14,  8.63s/it]
 87%|████████▋ | 196091/225000 [372:34:23<73:09:04,  9.11s/it]
 87%|████████▋ | 196092/225000 [372:34:31<71:31:24,  8.91s/it]
 87%|████████▋ | 196093/225000 [372:34:38<67:17:27,  8.38s/it]
 87%|████████▋ | 196094/225000 [372:34:52<80:22:32, 10.01s/it]
 87%|████████▋ | 196095/225000 [372:35:01<77:50:58,  9.70s/it]
 87%|████████▋ | 196096/225000 [372:35:07<68:05:45,  8.48s/it]
 87%|████████▋ | 196097/225000 [372:35:13<61:26:40,  7.65s/it]
 87%|████████▋ | 196098/225000 [372:35:20<60:14:05,  7.50s/it]
 87%|████████▋ | 196099/225000 [372:35:29<64:57:00,  8.09s/it]
 87%|████████▋ | 196100/225000 [372:35:39<70:18:57,  8.76s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.061', 'learning_rate': '2.21e-06', 'epoch': '0.8716', 'num_input_tokens_seen': 622314848, 'train_runtime': '1.341e+06', 'train_tokens_per_second': '463.9'}

 87%|████████▋ | 196100/225000 [372:35:39<70:18:57,  8.76s/it]
 87%|████████▋ | 196101/225000 [372:35:47<66:53:45,  8.33s/it]
 87%|████████▋ | 196102/225000 [372:35:53<62:42:57,  7.81s/it]
 87%|████████▋ | 196103/225000 [372:36:01<63:05:27,  7.86s/it]
 87%|████████▋ | 196104/225000 [372:36:08<59:07:27,  7.37s/it]
 87%|████████▋ | 196105/225000 [372:36:14<56:05:07,  6.99s/it]
 87%|████████▋ | 196106/225000 [372:36:21<57:13:43,  7.13s/it]
 87%|████████▋ | 196107/225000 [372:36:33<69:40:46,  8.68s/it]
 87%|████████▋ | 196108/225000 [372:36:41<67:41:28,  8.43s/it]
 87%|████████▋ | 196109/225000 [372:36:49<66:53:45,  8.34s/it]
 87%|████████▋ | 196110/225000 [372:37:01<75:30:50,  9.41s/it]
 87%|████████▋ | 196111/225000 [372:37:08<69:54:49,  8.71s/it]
 87%|████████▋ | 196112/225000 [372:37:19<75:19:48,  9.39s/it]
 87%|████████▋ | 196113/225000 [372:37:25<65:49:10,  8.20s/it]
 87%|████████▋ | 196114/225000 [372:37:36<72:20:55,  9.02s/it]
 87%|████████▋ | 196115/225000 [372:37:41<63:54:52,  7.97s/it]
 87%|████████▋ | 196116/225000 [372:37:49<62:15:52,  7.76s/it]
 87%|████████▋ | 196117/225000 [372:38:01<73:47:16,  9.20s/it]
 87%|████████▋ | 196118/225000 [372:38:07<64:44:28,  8.07s/it]
 87%|████████▋ | 196119/225000 [372:38:14<62:16:12,  7.76s/it]
 87%|████████▋ | 196120/225000 [372:38:21<60:53:42,  7.59s/it]
 87%|████████▋ | 196121/225000 [372:38:34<74:38:25,  9.30s/it]
 87%|████████▋ | 196122/225000 [372:38:43<72:51:24,  9.08s/it]
 87%|████████▋ | 196123/225000 [372:38:49<66:11:19,  8.25s/it]
 87%|████████▋ | 196124/225000 [372:39:01<75:53:08,  9.46s/it]
 87%|████████▋ | 196125/225000 [372:39:08<69:39:06,  8.68s/it]
 87%|████████▋ | 196126/225000 [372:39:13<61:36:10,  7.68s/it]
 87%|████████▋ | 196127/225000 [372:39:23<65:41:07,  8.19s/it]
 87%|████████▋ | 196128/225000 [372:39:32<67:36:43,  8.43s/it]
 87%|████████▋ | 196129/225000 [372:39:40<67:41:13,  8.44s/it]
 87%|████████▋ | 196130/225000 [372:39:45<57:53:50,  7.22s/it]
 87%|████████▋ | 196131/225000 [372:39:50<53:26:15,  6.66s/it]
 87%|████████▋ | 196132/225000 [372:39:58<57:12:26,  7.13s/it]
 87%|████████▋ | 196133/225000 [372:40:06<58:54:27,  7.35s/it]
 87%|████████▋ | 196134/225000 [372:40:12<55:05:54,  6.87s/it]
 87%|████████▋ | 196135/225000 [372:40:27<75:13:21,  9.38s/it]
 87%|████████▋ | 196136/225000 [372:40:36<74:22:06,  9.28s/it]
 87%|████████▋ | 196137/225000 [372:40:46<75:35:07,  9.43s/it]
 87%|████████▋ | 196138/225000 [372:40:55<75:08:50,  9.37s/it]
 87%|████████▋ | 196139/225000 [372:41:03<72:10:02,  9.00s/it]
 87%|████████▋ | 196140/225000 [372:41:10<66:40:27,  8.32s/it]
 87%|████████▋ | 196141/225000 [372:41:23<76:52:55,  9.59s/it]
 87%|████████▋ | 196142/225000 [372:41:27<64:40:52,  8.07s/it]
 87%|████████▋ | 196143/225000 [372:41:38<71:28:17,  8.92s/it]
 87%|████████▋ | 196144/225000 [372:41:46<70:14:58,  8.76s/it]
 87%|████████▋ | 196145/225000 [372:41:52<62:14:32,  7.77s/it]
 87%|████████▋ | 196146/225000 [372:42:05<75:10:54,  9.38s/it]
 87%|████████▋ | 196147/225000 [372:42:15<75:39:19,  9.44s/it]
 87%|████████▋ | 196148/225000 [372:42:25<77:07:13,  9.62s/it]
 87%|████████▋ | 196149/225000 [372:42:32<71:08:48,  8.88s/it]
 87%|████████▋ | 196150/225000 [372:42:40<69:04:25,  8.62s/it]
                                                              
{'loss': '1.286', 'grad_norm': '1.159', 'learning_rate': '2.202e-06', 'epoch': '0.8718', 'num_input_tokens_seen': 622540464, 'train_runtime': '1.342e+06', 'train_tokens_per_second': '464'}

 87%|████████▋ | 196150/225000 [372:42:40<69:04:25,  8.62s/it]
 87%|████████▋ | 196151/225000 [372:42:51<75:51:58,  9.47s/it]
 87%|████████▋ | 196152/225000 [372:42:57<66:08:10,  8.25s/it]
 87%|████████▋ | 196153/225000 [372:43:04<65:02:46,  8.12s/it]
 87%|████████▋ | 196154/225000 [372:43:11<60:44:44,  7.58s/it]
 87%|████████▋ | 196155/225000 [372:43:23<72:21:53,  9.03s/it]
 87%|████████▋ | 196156/225000 [372:43:29<65:25:05,  8.16s/it]
 87%|████████▋ | 196157/225000 [372:43:43<79:21:27,  9.90s/it]
 87%|████████▋ | 196158/225000 [372:43:50<72:53:51,  9.10s/it]
 87%|████████▋ | 196159/225000 [372:43:57<66:04:35,  8.25s/it]
 87%|████████▋ | 196160/225000 [372:44:03<61:12:37,  7.64s/it]
 87%|████████▋ | 196161/225000 [372:44:15<70:38:51,  8.82s/it]
 87%|████████▋ | 196162/225000 [372:44:22<67:41:37,  8.45s/it]
 87%|████████▋ | 196163/225000 [372:44:27<59:49:55,  7.47s/it]
 87%|████████▋ | 196164/225000 [372:44:41<74:50:06,  9.34s/it]
 87%|████████▋ | 196165/225000 [372:44:47<67:52:26,  8.47s/it]
 87%|████████▋ | 196166/225000 [372:44:57<70:12:11,  8.77s/it]
 87%|████████▋ | 196167/225000 [372:45:02<61:05:25,  7.63s/it]
 87%|████████▋ | 196168/225000 [372:45:14<71:48:46,  8.97s/it]
 87%|████████▋ | 196169/225000 [372:45:20<63:42:18,  7.95s/it]
 87%|████████▋ | 196170/225000 [372:45:30<70:20:06,  8.78s/it]
 87%|████████▋ | 196171/225000 [372:45:42<77:07:10,  9.63s/it]
 87%|████████▋ | 196172/225000 [372:45:47<65:26:31,  8.17s/it]
 87%|████████▋ | 196173/225000 [372:45:52<59:37:12,  7.45s/it]
 87%|████████▋ | 196174/225000 [372:46:06<75:02:23,  9.37s/it]
 87%|████████▋ | 196175/225000 [372:46:16<77:06:27,  9.63s/it]
 87%|████████▋ | 196176/225000 [372:46:23<70:09:47,  8.76s/it]
 87%|████████▋ | 196177/225000 [372:46:32<70:37:52,  8.82s/it]
 87%|████████▋ | 196178/225000 [372:46:41<71:00:34,  8.87s/it]
 87%|████████▋ | 196179/225000 [372:46:48<66:52:17,  8.35s/it]
 87%|████████▋ | 196180/225000 [372:46:57<66:55:49,  8.36s/it]
 87%|████████▋ | 196181/225000 [372:47:05<66:29:51,  8.31s/it]
 87%|████████▋ | 196182/225000 [372:47:12<63:06:48,  7.88s/it]
 87%|████████▋ | 196183/225000 [372:47:19<61:32:52,  7.69s/it]
 87%|████████▋ | 196184/225000 [372:47:26<60:58:07,  7.62s/it]
 87%|████████▋ | 196185/225000 [372:47:35<62:18:54,  7.79s/it]
 87%|████████▋ | 196186/225000 [372:47:41<57:42:16,  7.21s/it]
 87%|████████▋ | 196187/225000 [372:47:51<65:20:03,  8.16s/it]
 87%|████████▋ | 196188/225000 [372:48:03<74:41:52,  9.33s/it]
 87%|████████▋ | 196189/225000 [372:48:07<62:11:04,  7.77s/it]
 87%|████████▋ | 196190/225000 [372:48:21<77:10:32,  9.64s/it]
 87%|████████▋ | 196191/225000 [372:48:31<77:44:17,  9.71s/it]
 87%|████████▋ | 196192/225000 [372:48:38<71:22:38,  8.92s/it]
 87%|████████▋ | 196193/225000 [372:48:43<62:05:25,  7.76s/it]
 87%|████████▋ | 196194/225000 [372:48:50<60:42:00,  7.59s/it]
 87%|████████▋ | 196195/225000 [372:49:01<67:58:16,  8.49s/it]
 87%|████████▋ | 196196/225000 [372:49:10<69:44:54,  8.72s/it]
 87%|████████▋ | 196197/225000 [372:49:16<61:53:11,  7.74s/it]
 87%|████████▋ | 196198/225000 [372:49:26<68:17:04,  8.53s/it]
 87%|████████▋ | 196199/225000 [372:49:34<65:53:14,  8.24s/it]
 87%|████████▋ | 196200/225000 [372:49:40<62:42:31,  7.84s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.246', 'learning_rate': '2.195e-06', 'epoch': '0.872', 'num_input_tokens_seen': 622765776, 'train_runtime': '1.342e+06', 'train_tokens_per_second': '464'}

 87%|████████▋ | 196200/225000 [372:49:40<62:42:31,  7.84s/it]
 87%|████████▋ | 196201/225000 [372:49:47<58:47:19,  7.35s/it]
 87%|████████▋ | 196202/225000 [372:49:59<71:33:12,  8.94s/it]
 87%|████████▋ | 196203/225000 [372:50:09<74:19:42,  9.29s/it]
 87%|████████▋ | 196204/225000 [372:50:16<68:08:11,  8.52s/it]
 87%|████████▋ | 196205/225000 [372:50:23<64:38:41,  8.08s/it]
 87%|████████▋ | 196206/225000 [372:50:29<59:23:37,  7.43s/it]
 87%|████████▋ | 196207/225000 [372:50:36<58:09:07,  7.27s/it]
 87%|████████▋ | 196208/225000 [372:50:42<55:55:15,  6.99s/it]
 87%|████████▋ | 196209/225000 [372:51:00<81:35:36, 10.20s/it]
 87%|████████▋ | 196210/225000 [372:51:07<73:37:22,  9.21s/it]
 87%|████████▋ | 196211/225000 [372:51:14<69:18:15,  8.67s/it]
 87%|████████▋ | 196212/225000 [372:51:25<74:57:06,  9.37s/it]
 87%|████████▋ | 196213/225000 [372:51:32<67:30:37,  8.44s/it]
 87%|████████▋ | 196214/225000 [372:51:45<80:29:02, 10.07s/it]
 87%|████████▋ | 196215/225000 [372:51:52<71:28:51,  8.94s/it]
 87%|████████▋ | 196216/225000 [372:52:03<77:20:16,  9.67s/it]
 87%|████████▋ | 196217/225000 [372:52:09<67:47:06,  8.48s/it]
 87%|████████▋ | 196218/225000 [372:52:19<72:36:59,  9.08s/it]
 87%|████████▋ | 196219/225000 [372:52:29<74:58:23,  9.38s/it]
 87%|████████▋ | 196220/225000 [372:52:35<65:25:33,  8.18s/it]
 87%|████████▋ | 196221/225000 [372:52:41<60:27:18,  7.56s/it]
 87%|████████▋ | 196222/225000 [372:52:52<69:56:14,  8.75s/it]
 87%|████████▋ | 196223/225000 [372:53:00<67:39:50,  8.46s/it]
 87%|████████▋ | 196224/225000 [372:53:06<60:53:56,  7.62s/it]
 87%|████████▋ | 196225/225000 [372:53:11<53:44:49,  6.72s/it]
 87%|████████▋ | 196226/225000 [372:53:23<66:47:49,  8.36s/it]
 87%|████████▋ | 196227/225000 [372:53:30<64:36:47,  8.08s/it]
 87%|████████▋ | 196228/225000 [372:53:39<66:37:06,  8.34s/it]
 87%|████████▋ | 196229/225000 [372:53:48<67:21:30,  8.43s/it]
 87%|████████▋ | 196230/225000 [372:53:57<69:08:29,  8.65s/it]
 87%|████████▋ | 196231/225000 [372:54:04<65:45:28,  8.23s/it]
 87%|████████▋ | 196232/225000 [372:54:12<65:40:58,  8.22s/it]
 87%|████████▋ | 196233/225000 [372:54:22<69:32:39,  8.70s/it]
 87%|████████▋ | 196234/225000 [372:54:28<63:52:30,  7.99s/it]
 87%|████████▋ | 196235/225000 [372:54:37<66:04:17,  8.27s/it]
 87%|████████▋ | 196236/225000 [372:54:45<65:22:50,  8.18s/it]
 87%|████████▋ | 196237/225000 [372:54:56<70:20:22,  8.80s/it]
 87%|████████▋ | 196238/225000 [372:55:05<72:38:49,  9.09s/it]
 87%|████████▋ | 196239/225000 [372:55:13<69:14:06,  8.67s/it]
 87%|████████▋ | 196240/225000 [372:55:22<69:03:36,  8.64s/it]
 87%|████████▋ | 196241/225000 [372:55:30<67:10:33,  8.41s/it]
 87%|████████▋ | 196242/225000 [372:55:36<62:16:54,  7.80s/it]
 87%|████████▋ | 196243/225000 [372:55:44<64:00:30,  8.01s/it]
 87%|████████▋ | 196244/225000 [372:55:49<56:54:40,  7.12s/it]
 87%|████████▋ | 196245/225000 [372:55:59<62:57:59,  7.88s/it]
 87%|████████▋ | 196246/225000 [372:56:05<58:44:50,  7.36s/it]
 87%|████████▋ | 196247/225000 [372:56:14<61:35:53,  7.71s/it]
 87%|████████▋ | 196248/225000 [372:56:23<66:01:55,  8.27s/it]
 87%|████████▋ | 196249/225000 [372:56:30<62:35:53,  7.84s/it]
 87%|████████▋ | 196250/225000 [372:56:38<62:23:09,  7.81s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.292', 'learning_rate': '2.187e-06', 'epoch': '0.8722', 'num_input_tokens_seen': 622987512, 'train_runtime': '1.343e+06', 'train_tokens_per_second': '464'}

 87%|████████▋ | 196250/225000 [372:56:38<62:23:09,  7.81s/it]
 87%|████████▋ | 196251/225000 [372:56:47<65:44:11,  8.23s/it]
 87%|████████▋ | 196252/225000 [372:56:56<66:53:45,  8.38s/it]
 87%|████████▋ | 196253/225000 [372:57:08<76:30:02,  9.58s/it]
 87%|████████▋ | 196254/225000 [372:57:16<72:35:03,  9.09s/it]
 87%|████████▋ | 196255/225000 [372:57:26<75:11:11,  9.42s/it]
 87%|████████▋ | 196256/225000 [372:57:35<74:14:16,  9.30s/it]
 87%|████████▋ | 196257/225000 [372:57:41<65:41:42,  8.23s/it]
 87%|████████▋ | 196258/225000 [372:57:47<60:59:50,  7.64s/it]
 87%|████████▋ | 196259/225000 [372:58:00<72:06:43,  9.03s/it]
 87%|████████▋ | 196260/225000 [372:58:08<71:13:26,  8.92s/it]
 87%|████████▋ | 196261/225000 [372:58:12<59:36:17,  7.47s/it]
 87%|████████▋ | 196262/225000 [372:58:21<61:44:16,  7.73s/it]
 87%|████████▋ | 196263/225000 [372:58:28<61:38:32,  7.72s/it]
 87%|████████▋ | 196264/225000 [372:58:35<58:57:44,  7.39s/it]
 87%|████████▋ | 196265/225000 [372:58:40<53:15:02,  6.67s/it]
 87%|████████▋ | 196266/225000 [372:58:52<65:12:47,  8.17s/it]
 87%|████████▋ | 196267/225000 [372:59:00<65:08:43,  8.16s/it]
 87%|████████▋ | 196268/225000 [372:59:09<67:14:53,  8.43s/it]
 87%|████████▋ | 196269/225000 [372:59:17<65:21:12,  8.19s/it]
 87%|████████▋ | 196270/225000 [372:59:28<72:45:58,  9.12s/it]
 87%|████████▋ | 196271/225000 [372:59:32<62:04:36,  7.78s/it]
 87%|████████▋ | 196272/225000 [372:59:41<64:41:06,  8.11s/it]
 87%|████████▋ | 196273/225000 [372:59:49<63:37:44,  7.97s/it]
 87%|████████▋ | 196274/225000 [372:59:57<63:16:42,  7.93s/it]
 87%|████████▋ | 196275/225000 [373:00:08<71:44:22,  8.99s/it]
 87%|████████▋ | 196276/225000 [373:00:17<71:17:14,  8.93s/it]
 87%|████████▋ | 196277/225000 [373:00:24<67:24:45,  8.45s/it]
 87%|████████▋ | 196278/225000 [373:00:31<64:05:25,  8.03s/it]
 87%|████████▋ | 196279/225000 [373:00:40<64:40:55,  8.11s/it]
 87%|████████▋ | 196280/225000 [373:00:49<68:30:26,  8.59s/it]
 87%|████████▋ | 196281/225000 [373:00:56<63:32:49,  7.97s/it]
 87%|████████▋ | 196282/225000 [373:01:02<59:14:58,  7.43s/it]
 87%|████████▋ | 196283/225000 [373:01:09<58:39:09,  7.35s/it]
 87%|████████▋ | 196284/225000 [373:01:16<56:29:07,  7.08s/it]
 87%|████████▋ | 196285/225000 [373:01:31<76:55:02,  9.64s/it]
 87%|████████▋ | 196286/225000 [373:01:38<68:48:46,  8.63s/it]
 87%|████████▋ | 196287/225000 [373:01:48<73:40:17,  9.24s/it]
 87%|████████▋ | 196288/225000 [373:01:57<73:21:25,  9.20s/it]
 87%|████████▋ | 196289/225000 [373:02:07<74:24:36,  9.33s/it]
 87%|████████▋ | 196290/225000 [373:02:15<71:05:24,  8.91s/it]
 87%|████████▋ | 196291/225000 [373:02:23<69:36:22,  8.73s/it]
 87%|████████▋ | 196292/225000 [373:02:31<67:24:00,  8.45s/it]
 87%|████████▋ | 196293/225000 [373:02:43<76:12:55,  9.56s/it]
 87%|████████▋ | 196294/225000 [373:02:54<79:33:03,  9.98s/it]
 87%|████████▋ | 196295/225000 [373:03:03<76:16:54,  9.57s/it]
 87%|████████▋ | 196296/225000 [373:03:08<66:12:27,  8.30s/it]
 87%|████████▋ | 196297/225000 [373:03:21<77:31:28,  9.72s/it]
 87%|████████▋ | 196298/225000 [373:03:26<65:03:15,  8.16s/it]
 87%|████████▋ | 196299/225000 [373:03:35<67:52:52,  8.51s/it]
 87%|████████▋ | 196300/225000 [373:03:42<63:35:55,  7.98s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.226', 'learning_rate': '2.18e-06', 'epoch': '0.8724', 'num_input_tokens_seen': 623207952, 'train_runtime': '1.343e+06', 'train_tokens_per_second': '464'}

 87%|████████▋ | 196300/225000 [373:03:42<63:35:55,  7.98s/it]
 87%|████████▋ | 196301/225000 [373:03:51<65:37:35,  8.23s/it]
 87%|████████▋ | 196302/225000 [373:03:59<65:38:53,  8.24s/it]
 87%|████████▋ | 196303/225000 [373:04:07<65:53:39,  8.27s/it]
 87%|████████▋ | 196304/225000 [373:04:17<68:33:19,  8.60s/it]
 87%|████████▋ | 196305/225000 [373:04:25<68:02:43,  8.54s/it]
 87%|████████▋ | 196306/225000 [373:04:30<60:39:49,  7.61s/it]
 87%|████████▋ | 196307/225000 [373:04:38<59:36:19,  7.48s/it]
 87%|████████▋ | 196308/225000 [373:04:45<58:35:58,  7.35s/it]
 87%|████████▋ | 196309/225000 [373:04:57<70:48:04,  8.88s/it]
 87%|████████▋ | 196310/225000 [373:05:02<61:49:45,  7.76s/it]
 87%|████████▋ | 196311/225000 [373:05:11<65:16:15,  8.19s/it]
 87%|████████▋ | 196312/225000 [373:05:19<64:44:36,  8.12s/it]
 87%|████████▋ | 196313/225000 [373:05:26<61:51:00,  7.76s/it]
 87%|████████▋ | 196314/225000 [373:05:35<63:05:30,  7.92s/it]
 87%|████████▋ | 196315/225000 [373:05:42<62:07:42,  7.80s/it]
 87%|████████▋ | 196316/225000 [373:05:51<65:34:54,  8.23s/it]
 87%|████████▋ | 196317/225000 [373:06:00<65:30:38,  8.22s/it]
 87%|████████▋ | 196318/225000 [373:06:15<81:40:20, 10.25s/it]
 87%|████████▋ | 196319/225000 [373:06:21<73:36:27,  9.24s/it]
 87%|████████▋ | 196320/225000 [373:06:29<69:34:34,  8.73s/it]
 87%|████████▋ | 196321/225000 [373:06:39<72:36:23,  9.11s/it]
 87%|████████▋ | 196322/225000 [373:06:47<68:55:46,  8.65s/it]
 87%|████████▋ | 196323/225000 [373:06:55<68:31:44,  8.60s/it]
 87%|████████▋ | 196324/225000 [373:07:06<74:59:43,  9.41s/it]
 87%|████████▋ | 196325/225000 [373:07:14<70:19:44,  8.83s/it]
 87%|████████▋ | 196326/225000 [373:07:25<76:42:44,  9.63s/it]
 87%|████████▋ | 196327/225000 [373:07:31<67:41:27,  8.50s/it]
 87%|████████▋ | 196328/225000 [373:07:45<81:12:26, 10.20s/it]
 87%|████████▋ | 196329/225000 [373:07:54<78:11:54,  9.82s/it]
 87%|████████▋ | 196330/225000 [373:08:01<69:39:04,  8.75s/it]
 87%|████████▋ | 196331/225000 [373:08:07<63:12:07,  7.94s/it]
 87%|████████▋ | 196332/225000 [373:08:16<66:28:33,  8.35s/it]
 87%|████████▋ | 196333/225000 [373:08:21<58:51:32,  7.39s/it]
 87%|████████▋ | 196334/225000 [373:08:30<61:43:32,  7.75s/it]
 87%|████████▋ | 196335/225000 [373:08:34<53:33:04,  6.73s/it]
 87%|████████▋ | 196336/225000 [373:08:45<63:39:21,  7.99s/it]
 87%|████████▋ | 196337/225000 [373:08:55<69:08:20,  8.68s/it]
 87%|████████▋ | 196338/225000 [373:09:01<61:17:58,  7.70s/it]
 87%|████████▋ | 196339/225000 [373:09:07<57:51:25,  7.27s/it]
 87%|████████▋ | 196340/225000 [373:09:20<72:37:37,  9.12s/it]
 87%|████████▋ | 196341/225000 [373:09:25<63:06:38,  7.93s/it]
 87%|████████▋ | 196342/225000 [373:09:38<73:48:18,  9.27s/it]
 87%|████████▋ | 196343/225000 [373:09:45<68:20:13,  8.58s/it]
 87%|████████▋ | 196344/225000 [373:09:51<63:27:04,  7.97s/it]
 87%|████████▋ | 196345/225000 [373:10:02<69:11:13,  8.69s/it]
 87%|████████▋ | 196346/225000 [373:10:11<71:21:53,  8.97s/it]
 87%|████████▋ | 196347/225000 [373:10:19<68:48:37,  8.65s/it]
 87%|████████▋ | 196348/225000 [373:10:27<66:40:45,  8.38s/it]
 87%|████████▋ | 196349/225000 [373:10:39<74:08:11,  9.32s/it]
 87%|████████▋ | 196350/225000 [373:10:49<76:02:31,  9.56s/it]
                                                              
{'loss': '1.292', 'grad_norm': '1.216', 'learning_rate': '2.173e-06', 'epoch': '0.8727', 'num_input_tokens_seen': 623438032, 'train_runtime': '1.343e+06', 'train_tokens_per_second': '464.1'}

 87%|████████▋ | 196350/225000 [373:10:49<76:02:31,  9.56s/it]
 87%|████████▋ | 196351/225000 [373:10:55<67:26:35,  8.47s/it]
 87%|████████▋ | 196352/225000 [373:11:00<59:43:28,  7.51s/it]
 87%|████████▋ | 196353/225000 [373:11:06<56:57:27,  7.16s/it]
 87%|████████▋ | 196354/225000 [373:11:20<72:25:57,  9.10s/it]
 87%|████████▋ | 196355/225000 [373:11:27<67:15:38,  8.45s/it]
 87%|████████▋ | 196356/225000 [373:11:37<71:45:25,  9.02s/it]
 87%|████████▋ | 196357/225000 [373:11:44<67:39:15,  8.50s/it]
 87%|████████▋ | 196358/225000 [373:11:50<61:21:18,  7.71s/it]
 87%|████████▋ | 196359/225000 [373:12:01<68:53:08,  8.66s/it]
 87%|████████▋ | 196360/225000 [373:12:06<60:18:40,  7.58s/it]
 87%|████████▋ | 196361/225000 [373:12:12<55:41:24,  7.00s/it]
 87%|████████▋ | 196362/225000 [373:12:21<61:53:30,  7.78s/it]
 87%|████████▋ | 196363/225000 [373:12:32<68:19:38,  8.59s/it]
 87%|████████▋ | 196364/225000 [373:12:38<61:28:44,  7.73s/it]
 87%|████████▋ | 196365/225000 [373:12:43<56:49:04,  7.14s/it]
 87%|████████▋ | 196366/225000 [373:12:58<74:21:03,  9.35s/it]
 87%|████████▋ | 196367/225000 [373:13:06<70:35:02,  8.87s/it]
 87%|████████▋ | 196368/225000 [373:13:12<63:23:23,  7.97s/it]
 87%|████████▋ | 196369/225000 [373:13:22<68:59:16,  8.67s/it]
 87%|████████▋ | 196370/225000 [373:13:29<64:50:58,  8.15s/it]
 87%|████████▋ | 196371/225000 [373:13:40<72:27:52,  9.11s/it]
 87%|████████▋ | 196372/225000 [373:13:49<71:38:27,  9.01s/it]
 87%|████████▋ | 196373/225000 [373:13:55<65:43:58,  8.27s/it]
 87%|████████▋ | 196374/225000 [373:14:02<61:27:23,  7.73s/it]
 87%|████████▋ | 196375/225000 [373:14:10<62:06:05,  7.81s/it]
 87%|████████▋ | 196376/225000 [373:14:18<62:19:48,  7.84s/it]
 87%|████████▋ | 196377/225000 [373:14:31<74:40:59,  9.39s/it]
 87%|████████▋ | 196378/225000 [373:14:37<68:03:57,  8.56s/it]
 87%|████████▋ | 196379/225000 [373:14:43<61:43:50,  7.76s/it]
 87%|████████▋ | 196380/225000 [373:14:51<62:32:04,  7.87s/it]
 87%|████████▋ | 196381/225000 [373:14:59<62:39:43,  7.88s/it]
 87%|████████▋ | 196382/225000 [373:15:09<66:19:11,  8.34s/it]
 87%|████████▋ | 196383/225000 [373:15:16<64:34:04,  8.12s/it]
 87%|████████▋ | 196384/225000 [373:15:23<60:24:55,  7.60s/it]
 87%|████████▋ | 196385/225000 [373:15:32<65:07:01,  8.19s/it]
 87%|████████▋ | 196386/225000 [373:15:45<76:09:10,  9.58s/it]
 87%|████████▋ | 196387/225000 [373:15:56<78:12:03,  9.84s/it]
 87%|████████▋ | 196388/225000 [373:16:03<72:26:45,  9.12s/it]
 87%|████████▋ | 196389/225000 [373:16:08<61:34:19,  7.75s/it]
 87%|████████▋ | 196390/225000 [373:16:14<58:00:53,  7.30s/it]
 87%|████████▋ | 196391/225000 [373:16:26<68:45:45,  8.65s/it]
 87%|████████▋ | 196392/225000 [373:16:37<75:20:27,  9.48s/it]
 87%|████████▋ | 196393/225000 [373:16:49<81:52:45, 10.30s/it]
 87%|████████▋ | 196394/225000 [373:16:58<77:41:58,  9.78s/it]
 87%|████████▋ | 196395/225000 [373:17:06<73:30:57,  9.25s/it]
 87%|████████▋ | 196396/225000 [373:17:17<78:10:01,  9.84s/it]
 87%|████████▋ | 196397/225000 [373:17:26<74:48:00,  9.41s/it]
 87%|████████▋ | 196398/225000 [373:17:34<73:17:52,  9.23s/it]
 87%|████████▋ | 196399/225000 [373:17:45<76:28:01,  9.62s/it]
 87%|████████▋ | 196400/225000 [373:17:50<65:55:55,  8.30s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.338', 'learning_rate': '2.165e-06', 'epoch': '0.8729', 'num_input_tokens_seen': 623661672, 'train_runtime': '1.344e+06', 'train_tokens_per_second': '464.1'}

 87%|████████▋ | 196400/225000 [373:17:50<65:55:55,  8.30s/it]
 87%|████████▋ | 196401/225000 [373:17:56<60:07:39,  7.57s/it]
 87%|████████▋ | 196402/225000 [373:18:08<71:09:00,  8.96s/it]
 87%|████████▋ | 196403/225000 [373:18:17<71:17:31,  8.97s/it]
 87%|████████▋ | 196404/225000 [373:18:28<76:40:47,  9.65s/it]
 87%|████████▋ | 196405/225000 [373:18:36<72:04:20,  9.07s/it]
 87%|████████▋ | 196406/225000 [373:18:47<76:09:04,  9.59s/it]
 87%|████████▋ | 196407/225000 [373:18:58<80:04:30, 10.08s/it]
 87%|████████▋ | 196408/225000 [373:19:04<69:18:34,  8.73s/it]
 87%|████████▋ | 196409/225000 [373:19:09<60:46:16,  7.65s/it]
 87%|████████▋ | 196410/225000 [373:19:15<56:27:43,  7.11s/it]
 87%|████████▋ | 196411/225000 [373:19:23<59:02:23,  7.43s/it]
 87%|████████▋ | 196412/225000 [373:19:29<56:40:55,  7.14s/it]
 87%|████████▋ | 196413/225000 [373:19:34<51:48:51,  6.53s/it]
 87%|████████▋ | 196414/225000 [373:19:44<58:07:25,  7.32s/it]
 87%|████████▋ | 196415/225000 [373:19:51<58:59:24,  7.43s/it]
 87%|████████▋ | 196416/225000 [373:20:03<68:43:22,  8.66s/it]
 87%|████████▋ | 196417/225000 [373:20:08<61:28:26,  7.74s/it]
 87%|████████▋ | 196418/225000 [373:20:19<69:15:35,  8.72s/it]
 87%|████████▋ | 196419/225000 [373:20:33<79:43:26, 10.04s/it]
 87%|████████▋ | 196420/225000 [373:20:39<70:21:02,  8.86s/it]
 87%|████████▋ | 196421/225000 [373:20:48<70:52:10,  8.93s/it]
 87%|████████▋ | 196422/225000 [373:21:01<81:00:44, 10.21s/it]
 87%|████████▋ | 196423/225000 [373:21:08<72:49:01,  9.17s/it]
 87%|████████▋ | 196424/225000 [373:21:17<72:28:35,  9.13s/it]
 87%|████████▋ | 196425/225000 [373:21:21<59:55:48,  7.55s/it]
 87%|████████▋ | 196426/225000 [373:21:30<65:20:34,  8.23s/it]
 87%|████████▋ | 196427/225000 [373:21:37<61:42:56,  7.78s/it]
 87%|████████▋ | 196428/225000 [373:21:45<60:56:07,  7.68s/it]
 87%|████████▋ | 196429/225000 [373:21:49<53:14:51,  6.71s/it]
 87%|████████▋ | 196430/225000 [373:22:02<68:02:54,  8.57s/it]
 87%|████████▋ | 196431/225000 [373:22:14<76:52:44,  9.69s/it]
 87%|████████▋ | 196432/225000 [373:22:21<70:46:51,  8.92s/it]
 87%|████████▋ | 196433/225000 [373:22:30<70:50:14,  8.93s/it]
 87%|████████▋ | 196434/225000 [373:22:38<68:26:12,  8.62s/it]
 87%|████████▋ | 196435/225000 [373:22:47<69:02:56,  8.70s/it]
 87%|████████▋ | 196436/225000 [373:22:59<76:25:46,  9.63s/it]
 87%|████████▋ | 196437/225000 [373:23:10<79:32:46, 10.03s/it]
 87%|████████▋ | 196438/225000 [373:23:18<74:00:14,  9.33s/it]
 87%|████████▋ | 196439/225000 [373:23:27<73:21:17,  9.25s/it]
 87%|████████▋ | 196440/225000 [373:23:34<68:44:48,  8.67s/it]
 87%|████████▋ | 196441/225000 [373:23:42<68:01:42,  8.58s/it]
 87%|████████▋ | 196442/225000 [373:23:50<65:40:26,  8.28s/it]
 87%|████████▋ | 196443/225000 [373:23:56<60:08:05,  7.58s/it]
 87%|████████▋ | 196444/225000 [373:24:07<67:37:57,  8.53s/it]
 87%|████████▋ | 196445/225000 [373:24:13<62:37:32,  7.90s/it]
 87%|████████▋ | 196446/225000 [373:24:20<59:53:25,  7.55s/it]
 87%|████████▋ | 196447/225000 [373:24:25<55:35:48,  7.01s/it]
 87%|████████▋ | 196448/225000 [373:24:33<56:57:27,  7.18s/it]
 87%|████████▋ | 196449/225000 [373:24:45<68:53:00,  8.69s/it]
 87%|████████▋ | 196450/225000 [373:24:56<74:54:28,  9.45s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.452', 'learning_rate': '2.158e-06', 'epoch': '0.8731', 'num_input_tokens_seen': 623885712, 'train_runtime': '1.344e+06', 'train_tokens_per_second': '464.1'}

 87%|████████▋ | 196450/225000 [373:24:56<74:54:28,  9.45s/it]
 87%|████████▋ | 196451/225000 [373:25:02<66:28:23,  8.38s/it]
 87%|████████▋ | 196452/225000 [373:25:12<68:44:29,  8.67s/it]
 87%|████████▋ | 196453/225000 [373:25:19<65:40:31,  8.28s/it]
 87%|████████▋ | 196454/225000 [373:25:29<68:45:09,  8.67s/it]
 87%|████████▋ | 196455/225000 [373:25:34<61:32:44,  7.76s/it]
 87%|████████▋ | 196456/225000 [373:25:47<73:35:48,  9.28s/it]
 87%|████████▋ | 196457/225000 [373:25:55<70:16:18,  8.86s/it]
 87%|████████▋ | 196458/225000 [373:26:02<65:48:06,  8.30s/it]
 87%|████████▋ | 196459/225000 [373:26:09<63:01:54,  7.95s/it]
 87%|████████▋ | 196460/225000 [373:26:16<60:51:58,  7.68s/it]
 87%|████████▋ | 196461/225000 [373:26:25<64:39:23,  8.16s/it]
 87%|████████▋ | 196462/225000 [373:26:31<57:47:11,  7.29s/it]
 87%|████████▋ | 196463/225000 [373:26:39<60:59:00,  7.69s/it]
 87%|████████▋ | 196464/225000 [373:26:47<62:06:07,  7.83s/it]
 87%|████████▋ | 196465/225000 [373:26:56<63:52:04,  8.06s/it]
 87%|████████▋ | 196466/225000 [373:27:00<54:47:48,  6.91s/it]
 87%|████████▋ | 196467/225000 [373:27:11<63:52:33,  8.06s/it]
 87%|████████▋ | 196468/225000 [373:27:22<70:16:19,  8.87s/it]
 87%|████████▋ | 196469/225000 [373:27:27<61:55:55,  7.81s/it]
 87%|████████▋ | 196470/225000 [373:27:36<64:44:11,  8.17s/it]
 87%|████████▋ | 196471/225000 [373:27:42<59:38:35,  7.53s/it]
 87%|████████▋ | 196472/225000 [373:27:51<61:48:33,  7.80s/it]
 87%|████████▋ | 196473/225000 [373:27:58<61:38:02,  7.78s/it]
 87%|████████▋ | 196474/225000 [373:28:10<69:39:25,  8.79s/it]
 87%|████████▋ | 196475/225000 [373:28:17<65:39:45,  8.29s/it]
 87%|████████▋ | 196476/225000 [373:28:23<62:02:05,  7.83s/it]
 87%|████████▋ | 196477/225000 [373:28:30<59:37:42,  7.53s/it]
 87%|████████▋ | 196478/225000 [373:28:36<55:57:25,  7.06s/it]
 87%|████████▋ | 196479/225000 [373:28:50<72:49:12,  9.19s/it]
 87%|████████▋ | 196480/225000 [373:28:56<63:44:20,  8.05s/it]
 87%|████████▋ | 196481/225000 [373:29:06<69:42:14,  8.80s/it]
 87%|████████▋ | 196482/225000 [373:29:11<61:01:30,  7.70s/it]
 87%|████████▋ | 196483/225000 [373:29:21<65:41:19,  8.29s/it]
 87%|████████▋ | 196484/225000 [373:29:28<62:26:30,  7.88s/it]
 87%|████████▋ | 196485/225000 [373:29:43<79:12:28, 10.00s/it]
 87%|████████▋ | 196486/225000 [373:29:51<74:13:53,  9.37s/it]
 87%|████████▋ | 196487/225000 [373:30:04<83:37:37, 10.56s/it]
 87%|████████▋ | 196488/225000 [373:30:11<73:41:43,  9.30s/it]
 87%|████████▋ | 196489/225000 [373:30:21<76:23:49,  9.65s/it]
 87%|████████▋ | 196490/225000 [373:30:31<77:22:07,  9.77s/it]
 87%|████████▋ | 196491/225000 [373:30:40<76:28:02,  9.66s/it]
 87%|████████▋ | 196492/225000 [373:30:50<76:43:01,  9.69s/it]
 87%|████████▋ | 196493/225000 [373:30:59<74:14:36,  9.38s/it]
 87%|████████▋ | 196494/225000 [373:31:06<68:00:43,  8.59s/it]
 87%|████████▋ | 196495/225000 [373:31:13<65:34:53,  8.28s/it]
 87%|████████▋ | 196496/225000 [373:31:19<60:01:44,  7.58s/it]
 87%|████████▋ | 196497/225000 [373:31:26<57:51:24,  7.31s/it]
 87%|████████▋ | 196498/225000 [373:31:33<57:43:07,  7.29s/it]
 87%|████████▋ | 196499/225000 [373:31:45<69:33:44,  8.79s/it]
 87%|████████▋ | 196500/225000 [373:31:58<78:56:47,  9.97s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.059', 'learning_rate': '2.15e-06', 'epoch': '0.8733', 'num_input_tokens_seen': 624110264, 'train_runtime': '1.345e+06', 'train_tokens_per_second': '464.1'}

 87%|████████▋ | 196500/225000 [373:31:58<78:56:47,  9.97s/it]
 87%|████████▋ | 196501/225000 [373:32:06<73:27:24,  9.28s/it]
 87%|████████▋ | 196502/225000 [373:32:20<86:17:56, 10.90s/it]
 87%|████████▋ | 196503/225000 [373:32:29<80:58:05, 10.23s/it]
 87%|████████▋ | 196504/225000 [373:32:36<72:46:13,  9.19s/it]
 87%|████████▋ | 196505/225000 [373:32:48<79:47:06, 10.08s/it]
 87%|████████▋ | 196506/225000 [373:33:00<84:12:49, 10.64s/it]
 87%|████████▋ | 196507/225000 [373:33:09<79:36:31, 10.06s/it]
 87%|████████▋ | 196508/225000 [373:33:15<70:33:57,  8.92s/it]
 87%|████████▋ | 196509/225000 [373:33:28<80:39:26, 10.19s/it]
 87%|████████▋ | 196510/225000 [373:33:33<68:02:11,  8.60s/it]
 87%|████████▋ | 196511/225000 [373:33:42<70:00:28,  8.85s/it]
 87%|████████▋ | 196512/225000 [373:33:49<64:07:39,  8.10s/it]
 87%|████████▋ | 196513/225000 [373:33:53<54:46:57,  6.92s/it]
 87%|████████▋ | 196514/225000 [373:34:03<62:41:37,  7.92s/it]
 87%|████████▋ | 196515/225000 [373:34:09<58:31:41,  7.40s/it]
 87%|████████▋ | 196516/225000 [373:34:22<71:34:09,  9.05s/it]
 87%|████████▋ | 196517/225000 [373:34:28<63:14:04,  7.99s/it]
 87%|████████▋ | 196518/225000 [373:34:37<66:49:10,  8.45s/it]
 87%|████████▋ | 196519/225000 [373:34:46<68:19:07,  8.64s/it]
 87%|████████▋ | 196520/225000 [373:34:51<58:05:42,  7.34s/it]
 87%|████████▋ | 196521/225000 [373:34:57<55:14:32,  6.98s/it]
 87%|████████▋ | 196522/225000 [373:35:10<70:26:30,  8.90s/it]
 87%|████████▋ | 196523/225000 [373:35:17<64:55:19,  8.21s/it]
 87%|████████▋ | 196524/225000 [373:35:27<69:31:10,  8.79s/it]
 87%|████████▋ | 196525/225000 [373:35:35<68:51:07,  8.70s/it]
 87%|████████▋ | 196526/225000 [373:35:50<81:55:26, 10.36s/it]
 87%|████████▋ | 196527/225000 [373:35:55<70:07:16,  8.87s/it]
 87%|████████▋ | 196528/225000 [373:36:06<76:10:04,  9.63s/it]
 87%|████████▋ | 196529/225000 [373:36:13<68:15:37,  8.63s/it]
 87%|████████▋ | 196530/225000 [373:36:19<63:22:55,  8.01s/it]
 87%|████████▋ | 196531/225000 [373:36:24<56:09:25,  7.10s/it]
 87%|████████▋ | 196532/225000 [373:36:35<65:20:28,  8.26s/it]
 87%|████████▋ | 196533/225000 [373:36:40<56:24:15,  7.13s/it]
 87%|████████▋ | 196534/225000 [373:36:50<63:47:14,  8.07s/it]
 87%|████████▋ | 196535/225000 [373:36:57<61:07:31,  7.73s/it]
 87%|████████▋ | 196536/225000 [373:37:08<69:40:53,  8.81s/it]
 87%|████████▋ | 196537/225000 [373:37:14<61:44:50,  7.81s/it]
 87%|████████▋ | 196538/225000 [373:37:24<66:19:55,  8.39s/it]
 87%|████████▋ | 196539/225000 [373:37:29<59:58:51,  7.59s/it]
 87%|████████▋ | 196540/225000 [373:37:40<67:57:43,  8.60s/it]
 87%|████████▋ | 196541/225000 [373:37:45<59:00:57,  7.47s/it]
 87%|████████▋ | 196542/225000 [373:37:54<62:49:11,  7.95s/it]
 87%|████████▋ | 196543/225000 [373:38:05<69:56:32,  8.85s/it]
 87%|████████▋ | 196544/225000 [373:38:11<63:55:09,  8.09s/it]
 87%|████████▋ | 196545/225000 [373:38:18<60:47:31,  7.69s/it]
 87%|████████▋ | 196546/225000 [373:38:25<57:56:00,  7.33s/it]
 87%|████████▋ | 196547/225000 [373:38:32<58:39:52,  7.42s/it]
 87%|████████▋ | 196548/225000 [373:38:46<72:53:17,  9.22s/it]
 87%|████████▋ | 196549/225000 [373:38:51<63:45:04,  8.07s/it]
 87%|████████▋ | 196550/225000 [373:38:59<63:11:03,  8.00s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.389', 'learning_rate': '2.143e-06', 'epoch': '0.8736', 'num_input_tokens_seen': 624333192, 'train_runtime': '1.345e+06', 'train_tokens_per_second': '464.1'}

 87%|████████▋ | 196550/225000 [373:38:59<63:11:03,  8.00s/it]
 87%|████████▋ | 196551/225000 [373:39:08<65:18:50,  8.26s/it]
 87%|████████▋ | 196552/225000 [373:39:15<62:42:09,  7.93s/it]
 87%|████████▋ | 196553/225000 [373:39:23<63:18:59,  8.01s/it]
 87%|████████▋ | 196554/225000 [373:39:33<68:11:47,  8.63s/it]
 87%|████████▋ | 196555/225000 [373:39:38<59:35:23,  7.54s/it]
 87%|████████▋ | 196556/225000 [373:39:43<53:46:55,  6.81s/it]
 87%|████████▋ | 196557/225000 [373:39:48<49:37:40,  6.28s/it]
 87%|████████▋ | 196558/225000 [373:40:00<62:59:53,  7.97s/it]
 87%|████████▋ | 196559/225000 [373:40:05<56:32:50,  7.16s/it]
 87%|████████▋ | 196560/225000 [373:40:11<53:42:54,  6.80s/it]
 87%|████████▋ | 196561/225000 [373:40:24<67:39:12,  8.56s/it]
 87%|████████▋ | 196562/225000 [373:40:34<69:47:19,  8.83s/it]
 87%|████████▋ | 196563/225000 [373:40:40<62:51:08,  7.96s/it]
 87%|████████▋ | 196564/225000 [373:40:53<74:50:30,  9.47s/it]
 87%|████████▋ | 196565/225000 [373:41:03<76:02:44,  9.63s/it]
 87%|████████▋ | 196566/225000 [373:41:12<76:47:35,  9.72s/it]
 87%|████████▋ | 196567/225000 [373:41:24<80:00:17, 10.13s/it]
 87%|████████▋ | 196568/225000 [373:41:36<85:13:40, 10.79s/it]
 87%|████████▋ | 196569/225000 [373:41:46<82:45:57, 10.48s/it]
 87%|████████▋ | 196570/225000 [373:41:54<77:24:26,  9.80s/it]
 87%|████████▋ | 196571/225000 [373:41:59<65:59:32,  8.36s/it]
 87%|████████▋ | 196572/225000 [373:42:12<77:50:33,  9.86s/it]
 87%|████████▋ | 196573/225000 [373:42:22<77:05:53,  9.76s/it]
 87%|████████▋ | 196574/225000 [373:42:32<78:19:33,  9.92s/it]
 87%|████████▋ | 196575/225000 [373:42:41<75:16:42,  9.53s/it]
 87%|████████▋ | 196576/225000 [373:42:47<68:44:52,  8.71s/it]
 87%|████████▋ | 196577/225000 [373:43:00<77:58:43,  9.88s/it]
 87%|████████▋ | 196578/225000 [373:43:07<70:39:09,  8.95s/it]
 87%|████████▋ | 196579/225000 [373:43:15<68:03:01,  8.62s/it]
 87%|████████▋ | 196580/225000 [373:43:27<75:56:16,  9.62s/it]
 87%|████████▋ | 196581/225000 [373:43:32<64:43:03,  8.20s/it]
 87%|████████▋ | 196582/225000 [373:43:36<55:22:38,  7.02s/it]
 87%|████████▋ | 196583/225000 [373:43:47<65:32:33,  8.30s/it]
 87%|████████▋ | 196584/225000 [373:43:54<61:53:59,  7.84s/it]
 87%|████████▋ | 196585/225000 [373:44:00<58:12:26,  7.37s/it]
 87%|████████▋ | 196586/225000 [373:44:14<72:55:18,  9.24s/it]
 87%|████████▋ | 196587/225000 [373:44:19<62:44:55,  7.95s/it]
 87%|████████▋ | 196588/225000 [373:44:28<64:56:00,  8.23s/it]
 87%|████████▋ | 196589/225000 [373:44:39<72:36:48,  9.20s/it]
 87%|████████▋ | 196590/225000 [373:44:49<75:06:48,  9.52s/it]
 87%|████████▋ | 196591/225000 [373:44:59<74:55:39,  9.49s/it]
 87%|████████▋ | 196592/225000 [373:45:05<67:00:56,  8.49s/it]
 87%|████████▋ | 196593/225000 [373:45:15<70:30:02,  8.93s/it]
 87%|████████▋ | 196594/225000 [373:45:19<59:52:28,  7.59s/it]
 87%|████████▋ | 196595/225000 [373:45:28<62:13:53,  7.89s/it]
 87%|████████▋ | 196596/225000 [373:45:36<62:28:51,  7.92s/it]
 87%|████████▋ | 196597/225000 [373:45:46<68:51:47,  8.73s/it]
 87%|████████▋ | 196598/225000 [373:45:53<63:27:56,  8.04s/it]
 87%|████████▋ | 196599/225000 [373:46:01<62:59:24,  7.98s/it]
 87%|████████▋ | 196600/225000 [373:46:06<57:05:17,  7.24s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.327', 'learning_rate': '2.135e-06', 'epoch': '0.8738', 'num_input_tokens_seen': 624568968, 'train_runtime': '1.346e+06', 'train_tokens_per_second': '464.2'}

 87%|████████▋ | 196600/225000 [373:46:06<57:05:17,  7.24s/it]
 87%|████████▋ | 196601/225000 [373:46:16<62:58:22,  7.98s/it]
 87%|████████▋ | 196602/225000 [373:46:33<83:12:45, 10.55s/it]
 87%|████████▋ | 196603/225000 [373:46:39<74:38:31,  9.46s/it]
 87%|████████▋ | 196604/225000 [373:46:47<69:15:31,  8.78s/it]
 87%|████████▋ | 196605/225000 [373:46:58<74:32:42,  9.45s/it]
 87%|████████▋ | 196606/225000 [373:47:05<70:18:26,  8.91s/it]
 87%|████████▋ | 196607/225000 [373:47:12<65:22:51,  8.29s/it]
 87%|████████▋ | 196608/225000 [373:47:25<77:18:01,  9.80s/it]
 87%|████████▋ | 196609/225000 [373:47:32<69:46:58,  8.85s/it]
 87%|████████▋ | 196610/225000 [373:47:41<70:50:01,  8.98s/it]
 87%|████████▋ | 196611/225000 [373:47:47<61:44:39,  7.83s/it]
 87%|████████▋ | 196612/225000 [373:47:54<60:20:27,  7.65s/it]
 87%|████████▋ | 196613/225000 [373:48:05<68:34:26,  8.70s/it]
 87%|████████▋ | 196614/225000 [373:48:14<70:42:11,  8.97s/it]
 87%|████████▋ | 196615/225000 [373:48:23<70:43:55,  8.97s/it]
 87%|████████▋ | 196616/225000 [373:48:32<69:52:16,  8.86s/it]
 87%|████████▋ | 196617/225000 [373:48:38<62:48:44,  7.97s/it]
 87%|████████▋ | 196618/225000 [373:48:45<61:15:07,  7.77s/it]
 87%|████████▋ | 196619/225000 [373:48:55<67:01:45,  8.50s/it]
 87%|████████▋ | 196620/225000 [373:49:01<60:01:01,  7.61s/it]
 87%|████████▋ | 196621/225000 [373:49:11<66:35:52,  8.45s/it]
 87%|████████▋ | 196622/225000 [373:49:19<65:13:08,  8.27s/it]
 87%|████████▋ | 196623/225000 [373:49:29<68:17:06,  8.66s/it]
 87%|████████▋ | 196624/225000 [373:49:35<62:23:33,  7.92s/it]
 87%|████████▋ | 196625/225000 [373:49:42<61:05:31,  7.75s/it]
 87%|████████▋ | 196626/225000 [373:49:51<63:18:54,  8.03s/it]
 87%|████████▋ | 196627/225000 [373:50:03<73:08:36,  9.28s/it]
 87%|████████▋ | 196628/225000 [373:50:15<79:14:17, 10.05s/it]
 87%|████████▋ | 196629/225000 [373:50:21<69:44:05,  8.85s/it]
 87%|████████▋ | 196630/225000 [373:50:33<77:01:07,  9.77s/it]
 87%|████████▋ | 196631/225000 [373:50:40<70:00:15,  8.88s/it]
 87%|████████▋ | 196632/225000 [373:50:49<70:45:09,  8.98s/it]
 87%|████████▋ | 196633/225000 [373:50:55<64:16:05,  8.16s/it]
 87%|████████▋ | 196634/225000 [373:51:00<57:04:48,  7.24s/it]
 87%|████████▋ | 196635/225000 [373:51:10<62:07:27,  7.88s/it]
 87%|████████▋ | 196636/225000 [373:51:16<58:58:55,  7.49s/it]
 87%|████████▋ | 196637/225000 [373:51:22<53:56:39,  6.85s/it]
 87%|████████▋ | 196638/225000 [373:51:28<52:14:16,  6.63s/it]
 87%|████████▋ | 196639/225000 [373:51:38<60:17:27,  7.65s/it]
 87%|████████▋ | 196640/225000 [373:51:45<59:37:30,  7.57s/it]
 87%|████████▋ | 196641/225000 [373:51:51<56:03:21,  7.12s/it]
 87%|████████▋ | 196642/225000 [373:51:58<56:06:42,  7.12s/it]
 87%|████████▋ | 196643/225000 [373:52:08<62:54:06,  7.99s/it]
 87%|████████▋ | 196644/225000 [373:52:16<61:20:20,  7.79s/it]
 87%|████████▋ | 196645/225000 [373:52:27<68:29:41,  8.70s/it]
 87%|████████▋ | 196646/225000 [373:52:36<70:10:06,  8.91s/it]
 87%|████████▋ | 196647/225000 [373:52:45<71:16:06,  9.05s/it]
 87%|████████▋ | 196648/225000 [373:52:53<67:20:23,  8.55s/it]
 87%|████████▋ | 196649/225000 [373:52:59<60:55:12,  7.74s/it]
 87%|████████▋ | 196650/225000 [373:53:06<59:18:32,  7.53s/it]
                                                              
{'loss': '1.275', 'grad_norm': '1.228', 'learning_rate': '2.128e-06', 'epoch': '0.874', 'num_input_tokens_seen': 624795704, 'train_runtime': '1.346e+06', 'train_tokens_per_second': '464.2'}

 87%|████████▋ | 196650/225000 [373:53:06<59:18:32,  7.53s/it]
 87%|████████▋ | 196651/225000 [373:53:11<53:48:58,  6.83s/it]
 87%|████████▋ | 196652/225000 [373:53:17<53:08:05,  6.75s/it]
 87%|████████▋ | 196653/225000 [373:53:31<70:02:48,  8.90s/it]
 87%|████████▋ | 196654/225000 [373:53:38<65:45:20,  8.35s/it]
 87%|████████▋ | 196655/225000 [373:53:46<64:28:41,  8.19s/it]
 87%|████████▋ | 196656/225000 [373:53:54<64:29:40,  8.19s/it]
 87%|████████▋ | 196657/225000 [373:54:01<60:35:02,  7.70s/it]
 87%|████████▋ | 196658/225000 [373:54:15<74:23:43,  9.45s/it]
 87%|████████▋ | 196659/225000 [373:54:26<78:23:02,  9.96s/it]
 87%|████████▋ | 196660/225000 [373:54:34<74:55:57,  9.52s/it]
 87%|████████▋ | 196661/225000 [373:54:40<66:39:11,  8.47s/it]
 87%|████████▋ | 196662/225000 [373:54:52<75:06:42,  9.54s/it]
 87%|████████▋ | 196663/225000 [373:54:59<68:39:13,  8.72s/it]
 87%|████████▋ | 196664/225000 [373:55:06<65:36:30,  8.34s/it]
 87%|████████▋ | 196665/225000 [373:55:14<63:12:28,  8.03s/it]
 87%|████████▋ | 196666/225000 [373:55:20<57:50:53,  7.35s/it]
 87%|████████▋ | 196667/225000 [373:55:26<55:10:01,  7.01s/it]
 87%|████████▋ | 196668/225000 [373:55:34<58:54:29,  7.49s/it]
 87%|████████▋ | 196669/225000 [373:55:44<64:10:25,  8.15s/it]
 87%|████████▋ | 196670/225000 [373:55:51<61:42:11,  7.84s/it]
 87%|████████▋ | 196671/225000 [373:55:57<55:59:22,  7.12s/it]
 87%|████████▋ | 196672/225000 [373:56:06<62:22:17,  7.93s/it]
 87%|████████▋ | 196673/225000 [373:56:12<57:03:18,  7.25s/it]
 87%|████████▋ | 196674/225000 [373:56:24<68:25:29,  8.70s/it]
 87%|████████▋ | 196675/225000 [373:56:34<70:12:19,  8.92s/it]
 87%|████████▋ | 196676/225000 [373:56:39<61:42:46,  7.84s/it]
 87%|████████▋ | 196677/225000 [373:56:45<58:03:21,  7.38s/it]
 87%|████████▋ | 196678/225000 [373:56:58<70:04:54,  8.91s/it]
 87%|████████▋ | 196679/225000 [373:57:12<81:40:44, 10.38s/it]
 87%|████████▋ | 196680/225000 [373:57:20<77:56:02,  9.91s/it]
 87%|████████▋ | 196681/225000 [373:57:31<79:56:55, 10.16s/it]
 87%|████████▋ | 196682/225000 [373:57:37<69:44:53,  8.87s/it]
 87%|████████▋ | 196683/225000 [373:57:46<71:21:29,  9.07s/it]
 87%|████████▋ | 196684/225000 [373:57:53<65:17:00,  8.30s/it]
 87%|████████▋ | 196685/225000 [373:58:03<69:07:44,  8.79s/it]
 87%|████████▋ | 196686/225000 [373:58:11<68:27:41,  8.70s/it]
 87%|████████▋ | 196687/225000 [373:58:17<61:04:37,  7.77s/it]
 87%|████████▋ | 196688/225000 [373:58:25<61:09:13,  7.78s/it]
 87%|████████▋ | 196689/225000 [373:58:32<59:12:48,  7.53s/it]
 87%|████████▋ | 196690/225000 [373:58:45<71:36:08,  9.11s/it]
 87%|████████▋ | 196691/225000 [373:58:55<75:04:11,  9.55s/it]
 87%|████████▋ | 196692/225000 [373:59:03<71:38:21,  9.11s/it]
 87%|████████▋ | 196693/225000 [373:59:17<82:30:09, 10.49s/it]
 87%|████████▋ | 196694/225000 [373:59:26<78:05:42,  9.93s/it]
 87%|████████▋ | 196695/225000 [373:59:32<70:12:11,  8.93s/it]
 87%|████████▋ | 196696/225000 [373:59:43<74:48:03,  9.51s/it]
 87%|████████▋ | 196697/225000 [373:59:51<72:01:13,  9.16s/it]
 87%|████████▋ | 196698/225000 [373:59:57<63:16:23,  8.05s/it]
 87%|████████▋ | 196699/225000 [374:00:08<69:43:16,  8.87s/it]
 87%|████████▋ | 196700/225000 [374:00:15<66:20:44,  8.44s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.344', 'learning_rate': '2.121e-06', 'epoch': '0.8742', 'num_input_tokens_seen': 625027944, 'train_runtime': '1.346e+06', 'train_tokens_per_second': '464.2'}

 87%|████████▋ | 196700/225000 [374:00:15<66:20:44,  8.44s/it]
 87%|████████▋ | 196701/225000 [374:00:20<58:54:10,  7.49s/it]
 87%|████████▋ | 196702/225000 [374:00:30<64:29:56,  8.21s/it]
 87%|████████▋ | 196703/225000 [374:00:37<62:07:36,  7.90s/it]
 87%|████████▋ | 196704/225000 [374:00:47<65:23:17,  8.32s/it]
 87%|████████▋ | 196705/225000 [374:00:58<73:11:08,  9.31s/it]
 87%|████████▋ | 196706/225000 [374:01:04<65:11:51,  8.30s/it]
 87%|████████▋ | 196707/225000 [374:01:12<64:08:28,  8.16s/it]
 87%|████████▋ | 196708/225000 [374:01:24<73:09:34,  9.31s/it]
 87%|████████▋ | 196709/225000 [374:01:29<64:04:16,  8.15s/it]
 87%|████████▋ | 196710/225000 [374:01:35<58:57:17,  7.50s/it]
 87%|████████▋ | 196711/225000 [374:01:43<60:06:34,  7.65s/it]
 87%|████████▋ | 196712/225000 [374:01:58<75:46:56,  9.64s/it]
 87%|████████▋ | 196713/225000 [374:02:04<68:39:22,  8.74s/it]
 87%|████████▋ | 196714/225000 [374:02:14<71:33:38,  9.11s/it]
 87%|████████▋ | 196715/225000 [374:02:21<65:04:08,  8.28s/it]
 87%|████████▋ | 196716/225000 [374:02:36<80:49:17, 10.29s/it]
 87%|████████▋ | 196717/225000 [374:02:45<78:04:53,  9.94s/it]
 87%|████████▋ | 196718/225000 [374:02:51<69:44:39,  8.88s/it]
 87%|████████▋ | 196719/225000 [374:02:57<61:57:22,  7.89s/it]
 87%|████████▋ | 196720/225000 [374:03:08<68:53:40,  8.77s/it]
 87%|████████▋ | 196721/225000 [374:03:17<69:24:57,  8.84s/it]
 87%|████████▋ | 196722/225000 [374:03:22<61:52:46,  7.88s/it]
 87%|████████▋ | 196723/225000 [374:03:29<58:07:41,  7.40s/it]
 87%|████████▋ | 196724/225000 [374:03:37<60:13:30,  7.67s/it]
 87%|████████▋ | 196725/225000 [374:03:49<69:53:34,  8.90s/it]
 87%|████████▋ | 196726/225000 [374:03:57<68:07:22,  8.67s/it]
 87%|████████▋ | 196727/225000 [374:04:08<74:46:47,  9.52s/it]
 87%|████████▋ | 196728/225000 [374:04:14<66:41:07,  8.49s/it]
 87%|████████▋ | 196729/225000 [374:04:26<73:45:13,  9.39s/it]
 87%|████████▋ | 196730/225000 [374:04:29<59:59:12,  7.64s/it]
 87%|████████▋ | 196731/225000 [374:04:39<64:00:09,  8.15s/it]
 87%|████████▋ | 196732/225000 [374:04:46<61:24:25,  7.82s/it]
 87%|████████▋ | 196733/225000 [374:04:54<62:59:27,  8.02s/it]
 87%|████████▋ | 196734/225000 [374:05:00<57:45:37,  7.36s/it]
 87%|████████▋ | 196735/225000 [374:05:06<53:57:54,  6.87s/it]
 87%|████████▋ | 196736/225000 [374:05:12<52:58:29,  6.75s/it]
 87%|████████▋ | 196737/225000 [374:05:18<51:19:53,  6.54s/it]
 87%|████████▋ | 196738/225000 [374:05:34<71:59:22,  9.17s/it]
 87%|████████▋ | 196739/225000 [374:05:44<74:07:02,  9.44s/it]
 87%|████████▋ | 196740/225000 [374:05:49<63:56:33,  8.15s/it]
 87%|████████▋ | 196741/225000 [374:06:02<75:29:33,  9.62s/it]
 87%|████████▋ | 196742/225000 [374:06:11<74:09:23,  9.45s/it]
 87%|████████▋ | 196743/225000 [374:06:20<73:17:37,  9.34s/it]
 87%|████████▋ | 196744/225000 [374:06:26<66:13:09,  8.44s/it]
 87%|████████▋ | 196745/225000 [374:06:32<60:38:12,  7.73s/it]
 87%|████████▋ | 196746/225000 [374:06:44<69:18:14,  8.83s/it]
 87%|████████▋ | 196747/225000 [374:06:50<62:52:31,  8.01s/it]
 87%|████████▋ | 196748/225000 [374:06:59<64:21:02,  8.20s/it]
 87%|████████▋ | 196749/225000 [374:07:05<59:57:12,  7.64s/it]
 87%|████████▋ | 196750/225000 [374:07:12<57:50:11,  7.37s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.399', 'learning_rate': '2.113e-06', 'epoch': '0.8744', 'num_input_tokens_seen': 625256752, 'train_runtime': '1.347e+06', 'train_tokens_per_second': '464.2'}

 87%|████████▋ | 196750/225000 [374:07:12<57:50:11,  7.37s/it]
 87%|████████▋ | 196751/225000 [374:07:22<64:45:28,  8.25s/it]
 87%|████████▋ | 196752/225000 [374:07:33<70:37:37,  9.00s/it]
 87%|████████▋ | 196753/225000 [374:07:38<61:05:21,  7.79s/it]
 87%|████████▋ | 196754/225000 [374:07:43<55:19:03,  7.05s/it]
 87%|████████▋ | 196755/225000 [374:07:55<66:33:23,  8.48s/it]
 87%|████████▋ | 196756/225000 [374:08:04<68:02:05,  8.67s/it]
 87%|████████▋ | 196757/225000 [374:08:12<66:36:23,  8.49s/it]
 87%|████████▋ | 196758/225000 [374:08:22<69:23:58,  8.85s/it]
 87%|████████▋ | 196759/225000 [374:08:28<63:26:04,  8.09s/it]
 87%|████████▋ | 196760/225000 [374:08:41<74:46:28,  9.53s/it]
 87%|████████▋ | 196761/225000 [374:08:52<79:05:21, 10.08s/it]
 87%|████████▋ | 196762/225000 [374:08:58<69:25:04,  8.85s/it]
 87%|████████▋ | 196763/225000 [374:09:06<67:29:38,  8.60s/it]
 87%|████████▋ | 196764/225000 [374:09:11<58:57:22,  7.52s/it]
 87%|████████▋ | 196765/225000 [374:09:17<54:48:35,  6.99s/it]
 87%|████████▋ | 196766/225000 [374:09:30<68:29:52,  8.73s/it]
 87%|████████▋ | 196767/225000 [374:09:35<60:05:55,  7.66s/it]
 87%|████████▋ | 196768/225000 [374:09:45<65:46:38,  8.39s/it]
 87%|████████▋ | 196769/225000 [374:09:53<65:46:49,  8.39s/it]
 87%|████████▋ | 196770/225000 [374:10:03<68:38:33,  8.75s/it]
 87%|████████▋ | 196771/225000 [374:10:12<70:08:33,  8.95s/it]
 87%|████████▋ | 196772/225000 [374:10:20<66:23:55,  8.47s/it]
 87%|████████▋ | 196773/225000 [374:10:29<67:06:24,  8.56s/it]
 87%|████████▋ | 196774/225000 [374:10:36<63:58:10,  8.16s/it]
 87%|████████▋ | 196775/225000 [374:10:45<66:31:58,  8.49s/it]
 87%|████████▋ | 196776/225000 [374:10:53<65:11:08,  8.31s/it]
 87%|████████▋ | 196777/225000 [374:10:59<59:24:24,  7.58s/it]
 87%|████████▋ | 196778/225000 [374:11:04<53:55:49,  6.88s/it]
 87%|████████▋ | 196779/225000 [374:11:11<53:23:43,  6.81s/it]
 87%|████████▋ | 196780/225000 [374:11:22<64:30:44,  8.23s/it]
 87%|████████▋ | 196781/225000 [374:11:31<66:43:43,  8.51s/it]
 87%|████████▋ | 196782/225000 [374:11:39<64:16:03,  8.20s/it]
 87%|████████▋ | 196783/225000 [374:11:46<61:39:49,  7.87s/it]
 87%|████████▋ | 196784/225000 [374:11:54<61:56:08,  7.90s/it]
 87%|████████▋ | 196785/225000 [374:12:01<58:48:53,  7.50s/it]
 87%|████████▋ | 196786/225000 [374:12:16<76:52:10,  9.81s/it]
 87%|████████▋ | 196787/225000 [374:12:22<69:24:37,  8.86s/it]
 87%|████████▋ | 196788/225000 [374:12:29<64:21:23,  8.21s/it]
 87%|████████▋ | 196789/225000 [374:12:40<70:08:51,  8.95s/it]
 87%|████████▋ | 196790/225000 [374:12:51<74:50:23,  9.55s/it]
 87%|████████▋ | 196791/225000 [374:12:58<69:22:16,  8.85s/it]
 87%|████████▋ | 196792/225000 [374:13:04<63:32:15,  8.11s/it]
 87%|████████▋ | 196793/225000 [374:13:13<65:31:33,  8.36s/it]
 87%|████████▋ | 196794/225000 [374:13:24<71:34:23,  9.14s/it]
 87%|████████▋ | 196795/225000 [374:13:33<70:42:23,  9.02s/it]
 87%|████████▋ | 196796/225000 [374:13:39<62:53:56,  8.03s/it]
 87%|████████▋ | 196797/225000 [374:13:45<59:24:04,  7.58s/it]
 87%|████████▋ | 196798/225000 [374:13:54<61:13:42,  7.82s/it]
 87%|████████▋ | 196799/225000 [374:14:05<70:06:48,  8.95s/it]
 87%|████████▋ | 196800/225000 [374:14:11<62:22:28,  7.96s/it]
                                                              
{'loss': '1.329', 'grad_norm': '1.398', 'learning_rate': '2.106e-06', 'epoch': '0.8747', 'num_input_tokens_seen': 625478928, 'train_runtime': '1.347e+06', 'train_tokens_per_second': '464.3'}

 87%|████████▋ | 196800/225000 [374:14:11<62:22:28,  7.96s/it]
 87%|████████▋ | 196801/225000 [374:14:20<65:03:42,  8.31s/it]
 87%|████████▋ | 196802/225000 [374:14:31<72:10:21,  9.21s/it]
 87%|████████▋ | 196803/225000 [374:14:38<66:43:04,  8.52s/it]
 87%|████████▋ | 196804/225000 [374:14:51<76:34:11,  9.78s/it]
 87%|████████▋ | 196805/225000 [374:14:58<71:07:51,  9.08s/it]
 87%|████████▋ | 196806/225000 [374:15:04<62:56:58,  8.04s/it]
 87%|████████▋ | 196807/225000 [374:15:13<66:02:03,  8.43s/it]
 87%|████████▋ | 196808/225000 [374:15:24<71:43:51,  9.16s/it]
 87%|████████▋ | 196809/225000 [374:15:32<68:38:46,  8.77s/it]
 87%|████████▋ | 196810/225000 [374:15:42<71:27:17,  9.13s/it]
 87%|████████▋ | 196811/225000 [374:15:52<73:25:51,  9.38s/it]
 87%|████████▋ | 196812/225000 [374:15:58<65:53:08,  8.41s/it]
 87%|████████▋ | 196813/225000 [374:16:11<76:41:48,  9.80s/it]
 87%|████████▋ | 196814/225000 [374:16:16<65:48:53,  8.41s/it]
 87%|████████▋ | 196815/225000 [374:16:21<58:15:19,  7.44s/it]
 87%|████████▋ | 196816/225000 [374:16:28<56:03:09,  7.16s/it]
 87%|████████▋ | 196817/225000 [374:16:35<56:27:18,  7.21s/it]
 87%|████████▋ | 196818/225000 [374:16:44<60:34:48,  7.74s/it]
 87%|████████▋ | 196819/225000 [374:16:54<64:21:01,  8.22s/it]
 87%|████████▋ | 196820/225000 [374:17:00<59:42:19,  7.63s/it]
 87%|████████▋ | 196821/225000 [374:17:10<65:59:14,  8.43s/it]
 87%|████████▋ | 196822/225000 [374:17:24<78:18:00, 10.00s/it]
 87%|████████▋ | 196823/225000 [374:17:30<70:00:49,  8.95s/it]
 87%|████████▋ | 196824/225000 [374:17:41<75:00:24,  9.58s/it]
 87%|████████▋ | 196825/225000 [374:17:53<79:57:40, 10.22s/it]
 87%|████████▋ | 196826/225000 [374:18:00<71:08:13,  9.09s/it]
 87%|████████▋ | 196827/225000 [374:18:13<82:15:29, 10.51s/it]
 87%|████████▋ | 196828/225000 [374:18:18<68:59:35,  8.82s/it]
 87%|████████▋ | 196829/225000 [374:18:27<68:23:21,  8.74s/it]
 87%|████████▋ | 196830/225000 [374:18:33<62:59:01,  8.05s/it]
 87%|████████▋ | 196831/225000 [374:18:39<56:35:25,  7.23s/it]
 87%|████████▋ | 196832/225000 [374:18:50<65:59:08,  8.43s/it]
 87%|████████▋ | 196833/225000 [374:18:59<68:16:52,  8.73s/it]
 87%|████████▋ | 196834/225000 [374:19:08<69:36:37,  8.90s/it]
 87%|████████▋ | 196835/225000 [374:19:19<74:36:20,  9.54s/it]
 87%|████████▋ | 196836/225000 [374:19:27<70:42:45,  9.04s/it]
 87%|████████▋ | 196837/225000 [374:19:33<61:59:11,  7.92s/it]
 87%|████████▋ | 196838/225000 [374:19:48<78:39:23, 10.05s/it]
 87%|████████▋ | 196839/225000 [374:19:54<70:54:05,  9.06s/it]
 87%|████████▋ | 196840/225000 [374:20:03<68:50:54,  8.80s/it]
 87%|████████▋ | 196841/225000 [374:20:11<67:35:03,  8.64s/it]
 87%|████████▋ | 196842/225000 [374:20:17<60:42:13,  7.76s/it]
 87%|████████▋ | 196843/225000 [374:20:24<58:47:36,  7.52s/it]
 87%|████████▋ | 196844/225000 [374:20:33<62:28:44,  7.99s/it]
 87%|████████▋ | 196845/225000 [374:20:43<68:57:10,  8.82s/it]
 87%|████████▋ | 196846/225000 [374:20:55<74:40:24,  9.55s/it]
 87%|████████▋ | 196847/225000 [374:21:00<65:44:28,  8.41s/it]
 87%|████████▋ | 196848/225000 [374:21:07<60:21:51,  7.72s/it]
 87%|████████▋ | 196849/225000 [374:21:19<71:20:34,  9.12s/it]
 87%|████████▋ | 196850/225000 [374:21:27<68:33:35,  8.77s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.151', 'learning_rate': '2.099e-06', 'epoch': '0.8749', 'num_input_tokens_seen': 625708304, 'train_runtime': '1.348e+06', 'train_tokens_per_second': '464.3'}

 87%|████████▋ | 196850/225000 [374:21:27<68:33:35,  8.77s/it]
 87%|████████▋ | 196851/225000 [374:21:36<69:13:26,  8.85s/it]
 87%|████████▋ | 196852/225000 [374:21:45<68:46:44,  8.80s/it]
 87%|████████▋ | 196853/225000 [374:21:55<71:48:45,  9.18s/it]
 87%|████████▋ | 196854/225000 [374:22:01<64:45:43,  8.28s/it]
 87%|████████▋ | 196855/225000 [374:22:14<75:08:45,  9.61s/it]
 87%|████████▋ | 196856/225000 [374:22:21<70:26:08,  9.01s/it]
 87%|████████▋ | 196857/225000 [374:22:27<62:04:18,  7.94s/it]
 87%|████████▋ | 196858/225000 [374:22:33<59:06:32,  7.56s/it]
 87%|████████▋ | 196859/225000 [374:22:44<67:16:38,  8.61s/it]
 87%|████████▋ | 196860/225000 [374:22:54<69:30:13,  8.89s/it]
 87%|████████▋ | 196861/225000 [374:23:04<73:21:30,  9.39s/it]
 87%|████████▋ | 196862/225000 [374:23:11<67:39:13,  8.66s/it]
 87%|████████▋ | 196863/225000 [374:23:16<58:26:47,  7.48s/it]
 87%|████████▋ | 196864/225000 [374:23:29<70:34:04,  9.03s/it]
 87%|████████▋ | 196865/225000 [374:23:34<62:35:01,  8.01s/it]
 87%|████████▋ | 196866/225000 [374:23:42<61:24:04,  7.86s/it]
 87%|████████▋ | 196867/225000 [374:23:48<58:23:24,  7.47s/it]
 87%|████████▋ | 196868/225000 [374:23:56<59:14:32,  7.58s/it]
 87%|████████▋ | 196869/225000 [374:24:03<56:10:44,  7.19s/it]
 87%|████████▋ | 196870/225000 [374:24:18<74:45:15,  9.57s/it]
 87%|████████▋ | 196871/225000 [374:24:23<65:19:08,  8.36s/it]
 87%|████████▋ | 196872/225000 [374:24:31<62:46:17,  8.03s/it]
 87%|████████▋ | 196873/225000 [374:24:42<72:00:57,  9.22s/it]
 87%|████████▋ | 196874/225000 [374:24:49<65:49:21,  8.43s/it]
 88%|████████▊ | 196875/225000 [374:24:56<61:23:17,  7.86s/it]
 88%|████████▊ | 196876/225000 [374:25:00<54:17:46,  6.95s/it]
 88%|████████▊ | 196877/225000 [374:25:12<65:33:28,  8.39s/it]
 88%|████████▊ | 196878/225000 [374:25:18<58:44:47,  7.52s/it]
 88%|████████▊ | 196879/225000 [374:25:29<67:08:28,  8.60s/it]
 88%|████████▊ | 196880/225000 [374:25:36<64:00:07,  8.19s/it]
 88%|████████▊ | 196881/225000 [374:25:42<58:46:47,  7.53s/it]
 88%|████████▊ | 196882/225000 [374:25:53<67:08:21,  8.60s/it]
 88%|████████▊ | 196883/225000 [374:26:01<65:13:23,  8.35s/it]
 88%|████████▊ | 196884/225000 [374:26:06<56:56:56,  7.29s/it]
 88%|████████▊ | 196885/225000 [374:26:17<66:15:01,  8.48s/it]
 88%|████████▊ | 196886/225000 [374:26:26<68:23:10,  8.76s/it]
 88%|████████▊ | 196887/225000 [374:26:35<68:15:45,  8.74s/it]
 88%|████████▊ | 196888/225000 [374:26:43<66:20:16,  8.50s/it]
 88%|████████▊ | 196889/225000 [374:26:48<58:22:13,  7.48s/it]
 88%|████████▊ | 196890/225000 [374:27:03<75:33:33,  9.68s/it]
 88%|████████▊ | 196891/225000 [374:27:11<71:55:10,  9.21s/it]
 88%|████████▊ | 196892/225000 [374:27:20<70:15:57,  9.00s/it]
 88%|████████▊ | 196893/225000 [374:27:26<64:59:50,  8.32s/it]
 88%|████████▊ | 196894/225000 [374:27:34<64:26:41,  8.25s/it]
 88%|████████▊ | 196895/225000 [374:27:40<58:00:53,  7.43s/it]
 88%|████████▊ | 196896/225000 [374:27:54<74:20:20,  9.52s/it]
 88%|████████▊ | 196897/225000 [374:28:04<73:58:40,  9.48s/it]
 88%|████████▊ | 196898/225000 [374:28:10<66:01:49,  8.46s/it]
 88%|████████▊ | 196899/225000 [374:28:20<69:26:31,  8.90s/it]
 88%|████████▊ | 196900/225000 [374:28:30<72:39:44,  9.31s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.152', 'learning_rate': '2.091e-06', 'epoch': '0.8751', 'num_input_tokens_seen': 625935520, 'train_runtime': '1.348e+06', 'train_tokens_per_second': '464.3'}

 88%|████████▊ | 196900/225000 [374:28:30<72:39:44,  9.31s/it]
 88%|████████▊ | 196901/225000 [374:28:41<75:54:41,  9.73s/it]
 88%|████████▊ | 196902/225000 [374:28:55<87:46:30, 11.25s/it]
 88%|████████▊ | 196903/225000 [374:29:04<82:16:18, 10.54s/it]
 88%|████████▊ | 196904/225000 [374:29:10<71:12:41,  9.12s/it]
 88%|████████▊ | 196905/225000 [374:29:19<69:49:45,  8.95s/it]
 88%|████████▊ | 196906/225000 [374:29:24<61:33:16,  7.89s/it]
 88%|████████▊ | 196907/225000 [374:29:32<62:41:33,  8.03s/it]
 88%|████████▊ | 196908/225000 [374:29:40<62:28:04,  8.01s/it]
 88%|████████▊ | 196909/225000 [374:29:46<57:14:13,  7.34s/it]
 88%|████████▊ | 196910/225000 [374:30:02<76:12:47,  9.77s/it]
 88%|████████▊ | 196911/225000 [374:30:12<77:28:48,  9.93s/it]
 88%|████████▊ | 196912/225000 [374:30:18<68:36:54,  8.79s/it]
 88%|████████▊ | 196913/225000 [374:30:30<75:35:28,  9.69s/it]
 88%|████████▊ | 196914/225000 [374:30:37<70:29:12,  9.03s/it]
 88%|████████▊ | 196915/225000 [374:30:46<70:26:07,  9.03s/it]
 88%|████████▊ | 196916/225000 [374:30:52<61:35:48,  7.90s/it]
 88%|████████▊ | 196917/225000 [374:31:03<70:48:09,  9.08s/it]
 88%|████████▊ | 196918/225000 [374:31:09<62:34:38,  8.02s/it]
 88%|████████▊ | 196919/225000 [374:31:17<62:19:30,  7.99s/it]
 88%|████████▊ | 196920/225000 [374:31:30<73:08:12,  9.38s/it]
 88%|████████▊ | 196921/225000 [374:31:39<74:21:37,  9.53s/it]
 88%|████████▊ | 196922/225000 [374:31:49<73:56:17,  9.48s/it]
 88%|████████▊ | 196923/225000 [374:32:00<78:39:01, 10.08s/it]
 88%|████████▊ | 196924/225000 [374:32:05<66:29:45,  8.53s/it]
 88%|████████▊ | 196925/225000 [374:32:18<76:00:06,  9.75s/it]
 88%|████████▊ | 196926/225000 [374:32:24<66:38:52,  8.55s/it]
 88%|████████▊ | 196927/225000 [374:32:30<61:43:01,  7.91s/it]
 88%|████████▊ | 196928/225000 [374:32:36<58:18:42,  7.48s/it]
 88%|████████▊ | 196929/225000 [374:32:44<57:41:18,  7.40s/it]
 88%|████████▊ | 196930/225000 [374:32:53<62:00:46,  7.95s/it]
 88%|████████▊ | 196931/225000 [374:33:04<68:45:33,  8.82s/it]
 88%|████████▊ | 196932/225000 [374:33:12<68:16:55,  8.76s/it]
 88%|████████▊ | 196933/225000 [374:33:20<65:38:36,  8.42s/it]
 88%|████████▊ | 196934/225000 [374:33:28<65:39:05,  8.42s/it]
 88%|████████▊ | 196935/225000 [374:33:42<77:11:00,  9.90s/it]
 88%|████████▊ | 196936/225000 [374:33:49<71:22:09,  9.16s/it]
 88%|████████▊ | 196937/225000 [374:33:56<66:44:18,  8.56s/it]
 88%|████████▊ | 196938/225000 [374:34:05<67:39:14,  8.68s/it]
 88%|████████▊ | 196939/225000 [374:34:16<72:51:14,  9.35s/it]
 88%|████████▊ | 196940/225000 [374:34:21<61:31:12,  7.89s/it]
 88%|████████▊ | 196941/225000 [374:34:31<66:57:12,  8.59s/it]
 88%|████████▊ | 196942/225000 [374:34:40<69:02:33,  8.86s/it]
 88%|████████▊ | 196943/225000 [374:34:52<75:10:51,  9.65s/it]
 88%|████████▊ | 196944/225000 [374:35:04<80:27:04, 10.32s/it]
 88%|████████▊ | 196945/225000 [374:35:15<82:06:12, 10.54s/it]
 88%|████████▊ | 196946/225000 [374:35:19<67:40:51,  8.69s/it]
 88%|████████▊ | 196947/225000 [374:35:26<64:11:30,  8.24s/it]
 88%|████████▊ | 196948/225000 [374:35:35<64:33:44,  8.29s/it]
 88%|████████▊ | 196949/225000 [374:35:43<64:09:58,  8.23s/it]
 88%|████████▊ | 196950/225000 [374:35:51<64:05:29,  8.23s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.207', 'learning_rate': '2.084e-06', 'epoch': '0.8753', 'num_input_tokens_seen': 626167616, 'train_runtime': '1.349e+06', 'train_tokens_per_second': '464.3'}

 88%|████████▊ | 196950/225000 [374:35:51<64:05:29,  8.23s/it]
 88%|████████▊ | 196951/225000 [374:35:59<63:24:27,  8.14s/it]
 88%|████████▊ | 196952/225000 [374:36:11<71:26:01,  9.17s/it]
 88%|████████▊ | 196953/225000 [374:36:15<60:33:06,  7.77s/it]
 88%|████████▊ | 196954/225000 [374:36:25<65:41:08,  8.43s/it]
 88%|████████▊ | 196955/225000 [374:36:33<64:11:19,  8.24s/it]
 88%|████████▊ | 196956/225000 [374:36:40<61:45:09,  7.93s/it]
 88%|████████▊ | 196957/225000 [374:36:46<55:59:10,  7.19s/it]
 88%|████████▊ | 196958/225000 [374:36:53<57:29:16,  7.38s/it]
 88%|████████▊ | 196959/225000 [374:37:08<75:30:25,  9.69s/it]
 88%|████████▊ | 196960/225000 [374:37:16<70:27:17,  9.05s/it]
 88%|████████▊ | 196961/225000 [374:37:25<69:23:34,  8.91s/it]
 88%|████████▊ | 196962/225000 [374:37:31<63:37:55,  8.17s/it]
 88%|████████▊ | 196963/225000 [374:37:38<60:39:14,  7.79s/it]
 88%|████████▊ | 196964/225000 [374:37:46<60:39:36,  7.79s/it]
 88%|████████▊ | 196965/225000 [374:37:55<63:31:45,  8.16s/it]
 88%|████████▊ | 196966/225000 [374:38:05<68:40:47,  8.82s/it]
 88%|████████▊ | 196967/225000 [374:38:16<72:37:45,  9.33s/it]
 88%|████████▊ | 196968/225000 [374:38:20<60:06:16,  7.72s/it]
 88%|████████▊ | 196969/225000 [374:38:28<61:39:22,  7.92s/it]
 88%|████████▊ | 196970/225000 [374:38:32<52:50:45,  6.79s/it]
 88%|████████▊ | 196971/225000 [374:38:41<58:24:16,  7.50s/it]
 88%|████████▊ | 196972/225000 [374:38:47<53:36:50,  6.89s/it]
 88%|████████▊ | 196973/225000 [374:38:54<54:01:55,  6.94s/it]
 88%|████████▊ | 196974/225000 [374:39:01<55:26:58,  7.12s/it]
 88%|████████▊ | 196975/225000 [374:39:13<65:18:25,  8.39s/it]
 88%|████████▊ | 196976/225000 [374:39:18<59:19:04,  7.62s/it]
 88%|████████▊ | 196977/225000 [374:39:24<54:45:03,  7.03s/it]
 88%|████████▊ | 196978/225000 [374:39:30<52:21:08,  6.73s/it]
 88%|████████▊ | 196979/225000 [374:39:44<68:49:53,  8.84s/it]
 88%|████████▊ | 196980/225000 [374:39:49<60:54:22,  7.83s/it]
 88%|████████▊ | 196981/225000 [374:40:00<66:27:00,  8.54s/it]
 88%|████████▊ | 196982/225000 [374:40:08<65:34:36,  8.43s/it]
 88%|████████▊ | 196983/225000 [374:40:19<71:25:02,  9.18s/it]
 88%|████████▊ | 196984/225000 [374:40:24<61:25:28,  7.89s/it]
 88%|████████▊ | 196985/225000 [374:40:30<56:55:09,  7.31s/it]
 88%|████████▊ | 196986/225000 [374:40:45<76:22:10,  9.81s/it]
 88%|████████▊ | 196987/225000 [374:40:55<75:34:09,  9.71s/it]
 88%|████████▊ | 196988/225000 [374:41:04<74:00:49,  9.51s/it]
 88%|████████▊ | 196989/225000 [374:41:12<70:51:31,  9.11s/it]
 88%|████████▊ | 196990/225000 [374:41:17<61:51:24,  7.95s/it]
 88%|████████▊ | 196991/225000 [374:41:26<63:24:02,  8.15s/it]
 88%|████████▊ | 196992/225000 [374:41:33<61:49:14,  7.95s/it]
 88%|████████▊ | 196993/225000 [374:41:47<75:56:25,  9.76s/it]
 88%|████████▊ | 196994/225000 [374:41:52<65:21:25,  8.40s/it]
 88%|████████▊ | 196995/225000 [374:42:07<80:31:48, 10.35s/it]
 88%|████████▊ | 196996/225000 [374:42:15<73:32:47,  9.45s/it]
 88%|████████▊ | 196997/225000 [374:42:21<65:53:16,  8.47s/it]
 88%|████████▊ | 196998/225000 [374:42:28<62:53:55,  8.09s/it]
 88%|████████▊ | 196999/225000 [374:42:39<69:48:34,  8.98s/it]
 88%|████████▊ | 197000/225000 [374:42:45<62:16:38,  8.01s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.967', 'learning_rate': '2.077e-06', 'epoch': '0.8756', 'num_input_tokens_seen': 626387936, 'train_runtime': '1.349e+06', 'train_tokens_per_second': '464.3'}

 88%|████████▊ | 197000/225000 [374:42:45<62:16:38,  8.01s/it]
 88%|████████▊ | 197001/225000 [374:42:54<64:30:31,  8.29s/it]
 88%|████████▊ | 197002/225000 [374:42:59<57:36:59,  7.41s/it]
 88%|████████▊ | 197003/225000 [374:43:08<61:34:26,  7.92s/it]
 88%|████████▊ | 197004/225000 [374:43:15<57:36:43,  7.41s/it]
 88%|████████▊ | 197005/225000 [374:43:21<55:19:04,  7.11s/it]
 88%|████████▊ | 197006/225000 [374:43:31<63:16:03,  8.14s/it]
 88%|████████▊ | 197007/225000 [374:43:38<59:50:26,  7.70s/it]
 88%|████████▊ | 197008/225000 [374:43:47<63:02:59,  8.11s/it]
 88%|████████▊ | 197009/225000 [374:43:54<60:46:51,  7.82s/it]
 88%|████████▊ | 197010/225000 [374:44:01<57:50:01,  7.44s/it]
 88%|████████▊ | 197011/225000 [374:44:12<66:40:10,  8.58s/it]
 88%|████████▊ | 197012/225000 [374:44:23<71:05:15,  9.14s/it]
 88%|████████▊ | 197013/225000 [374:44:28<63:09:45,  8.12s/it]
 88%|████████▊ | 197014/225000 [374:44:35<59:49:57,  7.70s/it]
 88%|████████▊ | 197015/225000 [374:44:41<55:44:59,  7.17s/it]
 88%|████████▊ | 197016/225000 [374:44:51<61:39:20,  7.93s/it]
 88%|████████▊ | 197017/225000 [374:44:59<62:57:12,  8.10s/it]
 88%|████████▊ | 197018/225000 [374:45:07<61:26:05,  7.90s/it]
 88%|████████▊ | 197019/225000 [374:45:11<54:21:13,  6.99s/it]
 88%|████████▊ | 197020/225000 [374:45:25<70:33:47,  9.08s/it]
 88%|████████▊ | 197021/225000 [374:45:39<81:47:00, 10.52s/it]
 88%|████████▊ | 197022/225000 [374:45:45<69:52:26,  8.99s/it]
 88%|████████▊ | 197023/225000 [374:45:57<77:32:19,  9.98s/it]
 88%|████████▊ | 197024/225000 [374:46:04<70:46:49,  9.11s/it]
 88%|████████▊ | 197025/225000 [374:46:08<58:43:20,  7.56s/it]
 88%|████████▊ | 197026/225000 [374:46:20<69:25:59,  8.94s/it]
 88%|████████▊ | 197027/225000 [374:46:26<61:08:07,  7.87s/it]
 88%|████████▊ | 197028/225000 [374:46:31<56:10:53,  7.23s/it]
 88%|████████▊ | 197029/225000 [374:46:41<61:52:33,  7.96s/it]
 88%|████████▊ | 197030/225000 [374:46:50<64:53:19,  8.35s/it]
 88%|████████▊ | 197031/225000 [374:46:58<63:30:16,  8.17s/it]
 88%|████████▊ | 197032/225000 [374:47:10<72:42:14,  9.36s/it]
 88%|████████▊ | 197033/225000 [374:47:14<60:32:22,  7.79s/it]
 88%|████████▊ | 197034/225000 [374:47:20<56:03:48,  7.22s/it]
 88%|████████▊ | 197035/225000 [374:47:27<56:22:05,  7.26s/it]
 88%|████████▊ | 197036/225000 [374:47:34<54:38:33,  7.03s/it]
 88%|████████▊ | 197037/225000 [374:47:50<76:12:26,  9.81s/it]
 88%|████████▊ | 197038/225000 [374:47:58<71:59:26,  9.27s/it]
 88%|████████▊ | 197039/225000 [374:48:03<62:28:06,  8.04s/it]
 88%|████████▊ | 197040/225000 [374:48:12<64:32:40,  8.31s/it]
 88%|████████▊ | 197041/225000 [374:48:20<62:55:10,  8.10s/it]
 88%|████████▊ | 197042/225000 [374:48:26<57:23:31,  7.39s/it]
 88%|████████▊ | 197043/225000 [374:48:30<50:04:26,  6.45s/it]
 88%|████████▊ | 197044/225000 [374:48:39<55:29:24,  7.15s/it]
 88%|████████▊ | 197045/225000 [374:48:49<62:36:01,  8.06s/it]
 88%|████████▊ | 197046/225000 [374:49:01<70:41:38,  9.10s/it]
 88%|████████▊ | 197047/225000 [374:49:09<70:21:15,  9.06s/it]
 88%|████████▊ | 197048/225000 [374:49:17<67:43:48,  8.72s/it]
 88%|████████▊ | 197049/225000 [374:49:26<67:57:55,  8.75s/it]
 88%|████████▊ | 197050/225000 [374:49:32<60:08:00,  7.75s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.302', 'learning_rate': '2.07e-06', 'epoch': '0.8758', 'num_input_tokens_seen': 626609128, 'train_runtime': '1.349e+06', 'train_tokens_per_second': '464.4'}

 88%|████████▊ | 197050/225000 [374:49:32<60:08:00,  7.75s/it]
 88%|████████▊ | 197051/225000 [374:49:40<60:42:24,  7.82s/it]
 88%|████████▊ | 197052/225000 [374:49:49<64:54:55,  8.36s/it]
 88%|████████▊ | 197053/225000 [374:49:57<63:21:52,  8.16s/it]
 88%|████████▊ | 197054/225000 [374:50:05<62:16:30,  8.02s/it]
 88%|████████▊ | 197055/225000 [374:50:17<71:29:15,  9.21s/it]
 88%|████████▊ | 197056/225000 [374:50:24<66:39:13,  8.59s/it]
 88%|████████▊ | 197057/225000 [374:50:31<62:23:42,  8.04s/it]
 88%|████████▊ | 197058/225000 [374:50:36<56:18:40,  7.26s/it]
 88%|████████▊ | 197059/225000 [374:50:43<56:41:48,  7.30s/it]
 88%|████████▊ | 197060/225000 [374:50:53<61:13:16,  7.89s/it]
 88%|████████▊ | 197061/225000 [374:51:08<78:09:47, 10.07s/it]
 88%|████████▊ | 197062/225000 [374:51:17<77:11:47,  9.95s/it]
 88%|████████▊ | 197063/225000 [374:51:25<71:35:54,  9.23s/it]
 88%|████████▊ | 197064/225000 [374:51:36<75:32:05,  9.73s/it]
 88%|████████▊ | 197065/225000 [374:51:47<79:25:31, 10.24s/it]
 88%|████████▊ | 197066/225000 [374:51:56<76:24:21,  9.85s/it]
 88%|████████▊ | 197067/225000 [374:52:02<67:41:20,  8.72s/it]
 88%|████████▊ | 197068/225000 [374:52:11<66:49:57,  8.61s/it]
 88%|████████▊ | 197069/225000 [374:52:21<70:47:07,  9.12s/it]
 88%|████████▊ | 197070/225000 [374:52:29<68:55:48,  8.88s/it]
 88%|████████▊ | 197071/225000 [374:52:37<66:09:38,  8.53s/it]
 88%|████████▊ | 197072/225000 [374:52:47<69:35:02,  8.97s/it]
 88%|████████▊ | 197073/225000 [374:52:53<63:21:04,  8.17s/it]
 88%|████████▊ | 197074/225000 [374:53:03<66:45:45,  8.61s/it]
 88%|████████▊ | 197075/225000 [374:53:12<68:01:50,  8.77s/it]
 88%|████████▊ | 197076/225000 [374:53:19<62:33:11,  8.06s/it]
 88%|████████▊ | 197077/225000 [374:53:27<64:01:57,  8.26s/it]
 88%|████████▊ | 197078/225000 [374:53:36<65:14:45,  8.41s/it]
 88%|████████▊ | 197079/225000 [374:53:46<68:43:06,  8.86s/it]
 88%|████████▊ | 197080/225000 [374:53:53<64:42:25,  8.34s/it]
 88%|████████▊ | 197081/225000 [374:54:00<61:52:19,  7.98s/it]
 88%|████████▊ | 197082/225000 [374:54:16<79:16:37, 10.22s/it]
 88%|████████▊ | 197083/225000 [374:54:28<84:22:05, 10.88s/it]
 88%|████████▊ | 197084/225000 [374:54:33<70:19:43,  9.07s/it]
 88%|████████▊ | 197085/225000 [374:54:45<77:40:48, 10.02s/it]
 88%|████████▊ | 197086/225000 [374:54:50<65:35:27,  8.46s/it]
 88%|████████▊ | 197087/225000 [374:54:56<59:48:24,  7.71s/it]
 88%|████████▊ | 197088/225000 [374:55:02<55:25:02,  7.15s/it]
 88%|████████▊ | 197089/225000 [374:55:11<60:34:04,  7.81s/it]
 88%|████████▊ | 197090/225000 [374:55:18<57:21:29,  7.40s/it]
 88%|████████▊ | 197091/225000 [374:55:33<75:17:33,  9.71s/it]
 88%|████████▊ | 197092/225000 [374:55:40<69:23:55,  8.95s/it]
 88%|████████▊ | 197093/225000 [374:55:52<75:42:37,  9.77s/it]
 88%|████████▊ | 197094/225000 [374:55:59<71:16:04,  9.19s/it]
 88%|████████▊ | 197095/225000 [374:56:07<67:12:46,  8.67s/it]
 88%|████████▊ | 197096/225000 [374:56:13<60:54:19,  7.86s/it]
 88%|████████▊ | 197097/225000 [374:56:19<56:25:45,  7.28s/it]
 88%|████████▊ | 197098/225000 [374:56:26<56:15:28,  7.26s/it]
 88%|████████▊ | 197099/225000 [374:56:44<81:10:09, 10.47s/it]
 88%|████████▊ | 197100/225000 [374:56:52<75:37:31,  9.76s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.026', 'learning_rate': '2.062e-06', 'epoch': '0.876', 'num_input_tokens_seen': 626838672, 'train_runtime': '1.35e+06', 'train_tokens_per_second': '464.4'}

 88%|████████▊ | 197100/225000 [374:56:52<75:37:31,  9.76s/it]
 88%|████████▊ | 197101/225000 [374:57:00<70:35:22,  9.11s/it]
 88%|████████▊ | 197102/225000 [374:57:07<66:57:55,  8.64s/it]
 88%|████████▊ | 197103/225000 [374:57:15<66:18:49,  8.56s/it]
 88%|████████▊ | 197104/225000 [374:57:30<79:14:50, 10.23s/it]
 88%|████████▊ | 197105/225000 [374:57:36<70:10:46,  9.06s/it]
 88%|████████▊ | 197106/225000 [374:57:45<69:54:19,  9.02s/it]
 88%|████████▊ | 197107/225000 [374:57:50<59:50:33,  7.72s/it]
 88%|████████▊ | 197108/225000 [374:57:55<55:33:53,  7.17s/it]
 88%|████████▊ | 197109/225000 [374:58:03<56:25:39,  7.28s/it]
 88%|████████▊ | 197110/225000 [374:58:08<52:05:35,  6.72s/it]
 88%|████████▊ | 197111/225000 [374:58:15<51:36:08,  6.66s/it]
 88%|████████▊ | 197112/225000 [374:58:21<49:55:07,  6.44s/it]
 88%|████████▊ | 197113/225000 [374:58:31<58:03:23,  7.49s/it]
 88%|████████▊ | 197114/225000 [374:58:43<69:32:07,  8.98s/it]
 88%|████████▊ | 197115/225000 [374:58:49<63:01:07,  8.14s/it]
 88%|████████▊ | 197116/225000 [374:58:57<61:32:09,  7.94s/it]
 88%|████████▊ | 197117/225000 [374:59:08<68:35:40,  8.86s/it]
 88%|████████▊ | 197118/225000 [374:59:15<64:55:16,  8.38s/it]
 88%|████████▊ | 197119/225000 [374:59:25<67:22:53,  8.70s/it]
 88%|████████▊ | 197120/225000 [374:59:30<60:46:31,  7.85s/it]
 88%|████████▊ | 197121/225000 [374:59:37<56:34:09,  7.30s/it]
 88%|████████▊ | 197122/225000 [374:59:50<71:15:23,  9.20s/it]
 88%|████████▊ | 197123/225000 [374:59:58<67:41:23,  8.74s/it]
 88%|████████▊ | 197124/225000 [375:00:06<65:22:11,  8.44s/it]
 88%|████████▊ | 197125/225000 [375:00:11<58:22:35,  7.54s/it]
 88%|████████▊ | 197126/225000 [375:00:23<67:53:32,  8.77s/it]
 88%|████████▊ | 197127/225000 [375:00:31<67:19:17,  8.70s/it]
 88%|████████▊ | 197128/225000 [375:00:39<64:25:41,  8.32s/it]
 88%|████████▊ | 197129/225000 [375:00:45<60:36:36,  7.83s/it]
 88%|████████▊ | 197130/225000 [375:00:53<60:15:32,  7.78s/it]
 88%|████████▊ | 197131/225000 [375:00:59<56:14:08,  7.26s/it]
 88%|████████▊ | 197132/225000 [375:01:09<63:11:29,  8.16s/it]
 88%|████████▊ | 197133/225000 [375:01:22<74:52:05,  9.67s/it]
 88%|████████▊ | 197134/225000 [375:01:28<66:17:07,  8.56s/it]
 88%|████████▊ | 197135/225000 [375:01:36<62:58:57,  8.14s/it]
 88%|████████▊ | 197136/225000 [375:01:47<69:55:36,  9.03s/it]
 88%|████████▊ | 197137/225000 [375:01:52<61:36:24,  7.96s/it]
 88%|████████▊ | 197138/225000 [375:01:59<59:25:27,  7.68s/it]
 88%|████████▊ | 197139/225000 [375:02:04<53:51:31,  6.96s/it]
 88%|████████▊ | 197140/225000 [375:02:12<54:53:52,  7.09s/it]
 88%|████████▊ | 197141/225000 [375:02:18<52:27:28,  6.78s/it]
 88%|████████▊ | 197142/225000 [375:02:25<53:54:56,  6.97s/it]
 88%|████████▊ | 197143/225000 [375:02:34<58:45:40,  7.59s/it]
 88%|████████▊ | 197144/225000 [375:02:43<60:58:22,  7.88s/it]
 88%|████████▊ | 197145/225000 [375:02:49<57:21:25,  7.41s/it]
 88%|████████▊ | 197146/225000 [375:02:56<56:03:21,  7.24s/it]
 88%|████████▊ | 197147/225000 [375:03:02<53:32:48,  6.92s/it]
 88%|████████▊ | 197148/225000 [375:03:16<69:48:46,  9.02s/it]
 88%|████████▊ | 197149/225000 [375:03:25<70:20:42,  9.09s/it]
 88%|████████▊ | 197150/225000 [375:03:33<65:45:08,  8.50s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.344', 'learning_rate': '2.055e-06', 'epoch': '0.8762', 'num_input_tokens_seen': 627060784, 'train_runtime': '1.35e+06', 'train_tokens_per_second': '464.4'}

 88%|████████▊ | 197150/225000 [375:03:33<65:45:08,  8.50s/it]
 88%|████████▊ | 197151/225000 [375:03:42<68:19:21,  8.83s/it]
 88%|████████▊ | 197152/225000 [375:03:50<65:42:16,  8.49s/it]
 88%|████████▊ | 197153/225000 [375:03:56<59:18:51,  7.67s/it]
 88%|████████▊ | 197154/225000 [375:04:03<57:33:01,  7.44s/it]
 88%|████████▊ | 197155/225000 [375:04:14<66:57:07,  8.66s/it]
 88%|████████▊ | 197156/225000 [375:04:21<62:41:41,  8.11s/it]
 88%|████████▊ | 197157/225000 [375:04:27<58:21:46,  7.55s/it]
 88%|████████▊ | 197158/225000 [375:04:34<56:07:47,  7.26s/it]
 88%|████████▊ | 197159/225000 [375:04:41<55:55:50,  7.23s/it]
 88%|████████▊ | 197160/225000 [375:04:48<54:59:24,  7.11s/it]
 88%|████████▊ | 197161/225000 [375:04:57<59:48:16,  7.73s/it]
 88%|████████▊ | 197162/225000 [375:05:03<56:55:51,  7.36s/it]
 88%|████████▊ | 197163/225000 [375:05:12<59:56:54,  7.75s/it]
 88%|████████▊ | 197164/225000 [375:05:21<63:45:48,  8.25s/it]
 88%|████████▊ | 197165/225000 [375:05:31<67:14:10,  8.70s/it]
 88%|████████▊ | 197166/225000 [375:05:41<68:50:04,  8.90s/it]
 88%|████████▊ | 197167/225000 [375:05:51<72:13:53,  9.34s/it]
 88%|████████▊ | 197168/225000 [375:05:56<63:15:04,  8.18s/it]
 88%|████████▊ | 197169/225000 [375:06:06<67:15:47,  8.70s/it]
 88%|████████▊ | 197170/225000 [375:06:14<63:49:39,  8.26s/it]
 88%|████████▊ | 197171/225000 [375:06:25<70:10:02,  9.08s/it]
 88%|████████▊ | 197172/225000 [375:06:33<69:44:59,  9.02s/it]
 88%|████████▊ | 197173/225000 [375:06:44<73:37:47,  9.53s/it]
 88%|████████▊ | 197174/225000 [375:06:53<71:47:43,  9.29s/it]
 88%|████████▊ | 197175/225000 [375:06:59<64:08:58,  8.30s/it]
 88%|████████▊ | 197176/225000 [375:07:06<62:21:07,  8.07s/it]
 88%|████████▊ | 197177/225000 [375:07:14<61:33:47,  7.97s/it]
 88%|████████▊ | 197178/225000 [375:07:22<60:57:39,  7.89s/it]
 88%|████████▊ | 197179/225000 [375:07:29<58:35:39,  7.58s/it]
 88%|████████▊ | 197180/225000 [375:07:40<66:44:32,  8.64s/it]
 88%|████████▊ | 197181/225000 [375:07:51<72:28:54,  9.38s/it]
 88%|████████▊ | 197182/225000 [375:07:55<61:17:48,  7.93s/it]
 88%|████████▊ | 197183/225000 [375:08:05<64:29:12,  8.35s/it]
 88%|████████▊ | 197184/225000 [375:08:15<68:17:34,  8.84s/it]
 88%|████████▊ | 197185/225000 [375:08:24<69:35:42,  9.01s/it]
 88%|████████▊ | 197186/225000 [375:08:35<72:47:38,  9.42s/it]
 88%|████████▊ | 197187/225000 [375:08:40<63:34:03,  8.23s/it]
 88%|████████▊ | 197188/225000 [375:08:51<69:08:14,  8.95s/it]
 88%|████████▊ | 197189/225000 [375:08:57<62:21:27,  8.07s/it]
 88%|████████▊ | 197190/225000 [375:09:03<59:01:49,  7.64s/it]
 88%|████████▊ | 197191/225000 [375:09:11<59:34:27,  7.71s/it]
 88%|████████▊ | 197192/225000 [375:09:16<53:40:56,  6.95s/it]
 88%|████████▊ | 197193/225000 [375:09:25<58:46:11,  7.61s/it]
 88%|████████▊ | 197194/225000 [375:09:36<64:32:08,  8.36s/it]
 88%|████████▊ | 197195/225000 [375:09:46<70:08:01,  9.08s/it]
 88%|████████▊ | 197196/225000 [375:09:51<60:31:14,  7.84s/it]
 88%|████████▊ | 197197/225000 [375:10:03<69:08:57,  8.95s/it]
 88%|████████▊ | 197198/225000 [375:10:12<70:03:49,  9.07s/it]
 88%|████████▊ | 197199/225000 [375:10:18<62:13:34,  8.06s/it]
 88%|████████▊ | 197200/225000 [375:10:26<63:13:46,  8.19s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.452', 'learning_rate': '2.048e-06', 'epoch': '0.8764', 'num_input_tokens_seen': 627282880, 'train_runtime': '1.351e+06', 'train_tokens_per_second': '464.4'}

 88%|████████▊ | 197200/225000 [375:10:26<63:13:46,  8.19s/it]
 88%|████████▊ | 197201/225000 [375:10:32<57:08:07,  7.40s/it]
 88%|████████▊ | 197202/225000 [375:10:39<56:52:31,  7.37s/it]
 88%|████████▊ | 197203/225000 [375:10:45<52:57:36,  6.86s/it]
 88%|████████▊ | 197204/225000 [375:10:57<64:43:09,  8.38s/it]
 88%|████████▊ | 197205/225000 [375:11:07<68:57:39,  8.93s/it]
 88%|████████▊ | 197206/225000 [375:11:12<58:51:53,  7.62s/it]
 88%|████████▊ | 197207/225000 [375:11:19<58:15:21,  7.55s/it]
 88%|████████▊ | 197208/225000 [375:11:25<54:37:58,  7.08s/it]
 88%|████████▊ | 197209/225000 [375:11:34<59:03:45,  7.65s/it]
 88%|████████▊ | 197210/225000 [375:11:39<53:14:34,  6.90s/it]
 88%|████████▊ | 197211/225000 [375:11:46<54:02:08,  7.00s/it]
 88%|████████▊ | 197212/225000 [375:11:57<61:51:41,  8.01s/it]
 88%|████████▊ | 197213/225000 [375:12:10<73:37:44,  9.54s/it]
 88%|████████▊ | 197214/225000 [375:12:15<63:56:57,  8.29s/it]
 88%|████████▊ | 197215/225000 [375:12:27<71:47:33,  9.30s/it]
 88%|████████▊ | 197216/225000 [375:12:37<73:51:01,  9.57s/it]
 88%|████████▊ | 197217/225000 [375:12:44<68:35:35,  8.89s/it]
 88%|████████▊ | 197218/225000 [375:12:51<63:16:47,  8.20s/it]
 88%|████████▊ | 197219/225000 [375:13:01<67:36:20,  8.76s/it]
 88%|████████▊ | 197220/225000 [375:13:07<61:23:03,  7.95s/it]
 88%|████████▊ | 197221/225000 [375:13:12<55:08:39,  7.15s/it]
 88%|████████▊ | 197222/225000 [375:13:24<66:10:36,  8.58s/it]
 88%|████████▊ | 197223/225000 [375:13:30<59:58:03,  7.77s/it]
 88%|████████▊ | 197224/225000 [375:13:41<66:02:38,  8.56s/it]
 88%|████████▊ | 197225/225000 [375:13:48<63:34:04,  8.24s/it]
 88%|████████▊ | 197226/225000 [375:13:58<67:17:04,  8.72s/it]
 88%|████████▊ | 197227/225000 [375:14:05<64:30:51,  8.36s/it]
 88%|████████▊ | 197228/225000 [375:14:11<57:24:59,  7.44s/it]
 88%|████████▊ | 197229/225000 [375:14:17<54:56:44,  7.12s/it]
 88%|████████▊ | 197230/225000 [375:14:26<58:03:03,  7.53s/it]
 88%|████████▊ | 197231/225000 [375:14:36<65:57:36,  8.55s/it]
 88%|████████▊ | 197232/225000 [375:14:44<63:52:37,  8.28s/it]
 88%|████████▊ | 197233/225000 [375:14:53<65:51:19,  8.54s/it]
 88%|████████▊ | 197234/225000 [375:15:03<68:42:31,  8.91s/it]
 88%|████████▊ | 197235/225000 [375:15:12<67:53:00,  8.80s/it]
 88%|████████▊ | 197236/225000 [375:15:18<62:10:41,  8.06s/it]
 88%|████████▊ | 197237/225000 [375:15:25<59:10:46,  7.67s/it]
 88%|████████▊ | 197238/225000 [375:15:32<58:31:44,  7.59s/it]
 88%|████████▊ | 197239/225000 [375:15:40<58:48:52,  7.63s/it]
 88%|████████▊ | 197240/225000 [375:15:47<58:02:20,  7.53s/it]
 88%|████████▊ | 197241/225000 [375:15:56<60:51:11,  7.89s/it]
 88%|████████▊ | 197242/225000 [375:16:10<75:10:02,  9.75s/it]
 88%|████████▊ | 197243/225000 [375:16:20<74:55:03,  9.72s/it]
 88%|████████▊ | 197244/225000 [375:16:27<70:23:06,  9.13s/it]
 88%|████████▊ | 197245/225000 [375:16:38<74:37:05,  9.68s/it]
 88%|████████▊ | 197246/225000 [375:16:43<62:50:49,  8.15s/it]
 88%|████████▊ | 197247/225000 [375:16:54<69:03:48,  8.96s/it]
 88%|████████▊ | 197248/225000 [375:17:01<65:56:18,  8.55s/it]
 88%|████████▊ | 197249/225000 [375:17:12<71:53:27,  9.33s/it]
 88%|████████▊ | 197250/225000 [375:17:23<75:49:00,  9.84s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.001', 'learning_rate': '2.04e-06', 'epoch': '0.8767', 'num_input_tokens_seen': 627506504, 'train_runtime': '1.351e+06', 'train_tokens_per_second': '464.5'}

 88%|████████▊ | 197250/225000 [375:17:23<75:49:00,  9.84s/it]
 88%|████████▊ | 197251/225000 [375:17:31<71:21:53,  9.26s/it]
 88%|████████▊ | 197252/225000 [375:17:39<68:23:30,  8.87s/it]
 88%|████████▊ | 197253/225000 [375:17:47<65:41:04,  8.52s/it]
 88%|████████▊ | 197254/225000 [375:17:56<67:09:41,  8.71s/it]
 88%|████████▊ | 197255/225000 [375:18:04<64:05:18,  8.32s/it]
 88%|████████▊ | 197256/225000 [375:18:09<57:37:43,  7.48s/it]
 88%|████████▊ | 197257/225000 [375:18:19<62:50:52,  8.16s/it]
 88%|████████▊ | 197258/225000 [375:18:31<71:06:08,  9.23s/it]
 88%|████████▊ | 197259/225000 [375:18:38<66:13:47,  8.59s/it]
 88%|████████▊ | 197260/225000 [375:18:48<71:13:00,  9.24s/it]
 88%|████████▊ | 197261/225000 [375:18:53<59:57:32,  7.78s/it]
 88%|████████▊ | 197262/225000 [375:19:05<69:11:29,  8.98s/it]
 88%|████████▊ | 197263/225000 [375:19:13<68:09:32,  8.85s/it]
 88%|████████▊ | 197264/225000 [375:19:18<58:54:07,  7.65s/it]
 88%|████████▊ | 197265/225000 [375:19:28<64:31:35,  8.38s/it]
 88%|████████▊ | 197266/225000 [375:19:37<66:08:55,  8.59s/it]
 88%|████████▊ | 197267/225000 [375:19:49<73:50:04,  9.58s/it]
 88%|████████▊ | 197268/225000 [375:19:56<68:34:17,  8.90s/it]
 88%|████████▊ | 197269/225000 [375:20:05<68:22:52,  8.88s/it]
 88%|████████▊ | 197270/225000 [375:20:12<64:40:06,  8.40s/it]
 88%|████████▊ | 197271/225000 [375:20:21<64:05:44,  8.32s/it]
 88%|████████▊ | 197272/225000 [375:20:30<65:41:32,  8.53s/it]
 88%|████████▊ | 197273/225000 [375:20:39<67:49:28,  8.81s/it]
 88%|████████▊ | 197274/225000 [375:20:45<60:05:03,  7.80s/it]
 88%|████████▊ | 197275/225000 [375:20:56<67:29:47,  8.76s/it]
 88%|████████▊ | 197276/225000 [375:21:07<73:55:17,  9.60s/it]
 88%|████████▊ | 197277/225000 [375:21:16<72:54:55,  9.47s/it]
 88%|████████▊ | 197278/225000 [375:21:25<71:34:43,  9.30s/it]
 88%|████████▊ | 197279/225000 [375:21:33<68:13:10,  8.86s/it]
 88%|████████▊ | 197280/225000 [375:21:39<61:05:21,  7.93s/it]
 88%|████████▊ | 197281/225000 [375:21:51<72:09:13,  9.37s/it]
 88%|████████▊ | 197282/225000 [375:22:01<72:39:52,  9.44s/it]
 88%|████████▊ | 197283/225000 [375:22:05<60:14:23,  7.82s/it]
 88%|████████▊ | 197284/225000 [375:22:15<65:09:26,  8.46s/it]
 88%|████████▊ | 197285/225000 [375:22:24<65:37:36,  8.52s/it]
 88%|████████▊ | 197286/225000 [375:22:35<71:32:21,  9.29s/it]
 88%|████████▊ | 197287/225000 [375:22:41<65:12:08,  8.47s/it]
 88%|████████▊ | 197288/225000 [375:22:48<60:22:06,  7.84s/it]
 88%|████████▊ | 197289/225000 [375:22:54<56:38:52,  7.36s/it]
 88%|████████▊ | 197290/225000 [375:23:01<55:00:40,  7.15s/it]
 88%|████████▊ | 197291/225000 [375:23:07<53:48:09,  6.99s/it]
 88%|████████▊ | 197292/225000 [375:23:21<68:13:12,  8.86s/it]
 88%|████████▊ | 197293/225000 [375:23:25<58:54:48,  7.65s/it]
 88%|████████▊ | 197294/225000 [375:23:37<67:13:15,  8.73s/it]
 88%|████████▊ | 197295/225000 [375:23:42<58:56:36,  7.66s/it]
 88%|████████▊ | 197296/225000 [375:23:49<57:08:56,  7.43s/it]
 88%|████████▊ | 197297/225000 [375:23:55<55:35:33,  7.22s/it]
 88%|████████▊ | 197298/225000 [375:24:03<56:33:29,  7.35s/it]
 88%|████████▊ | 197299/225000 [375:24:09<52:49:16,  6.86s/it]
 88%|████████▊ | 197300/225000 [375:24:23<69:47:32,  9.07s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.135', 'learning_rate': '2.033e-06', 'epoch': '0.8769', 'num_input_tokens_seen': 627738752, 'train_runtime': '1.351e+06', 'train_tokens_per_second': '464.5'}

 88%|████████▊ | 197300/225000 [375:24:23<69:47:32,  9.07s/it]
 88%|████████▊ | 197301/225000 [375:24:32<69:40:36,  9.06s/it]
 88%|████████▊ | 197302/225000 [375:24:45<78:42:45, 10.23s/it]
 88%|████████▊ | 197303/225000 [375:24:51<68:30:01,  8.90s/it]
 88%|████████▊ | 197304/225000 [375:24:56<60:16:09,  7.83s/it]
 88%|████████▊ | 197305/225000 [375:25:07<66:28:43,  8.64s/it]
 88%|████████▊ | 197306/225000 [375:25:15<66:42:31,  8.67s/it]
 88%|████████▊ | 197307/225000 [375:25:29<77:57:55, 10.14s/it]
 88%|████████▊ | 197308/225000 [375:25:33<64:19:30,  8.36s/it]
 88%|████████▊ | 197309/225000 [375:25:39<57:49:30,  7.52s/it]
 88%|████████▊ | 197310/225000 [375:25:49<64:53:57,  8.44s/it]
 88%|████████▊ | 197311/225000 [375:25:58<65:53:37,  8.57s/it]
 88%|████████▊ | 197312/225000 [375:26:07<66:07:19,  8.60s/it]
 88%|████████▊ | 197313/225000 [375:26:15<64:06:58,  8.34s/it]
 88%|████████▊ | 197314/225000 [375:26:21<59:28:28,  7.73s/it]
 88%|████████▊ | 197315/225000 [375:26:29<59:30:49,  7.74s/it]
 88%|████████▊ | 197316/225000 [375:26:38<63:39:47,  8.28s/it]
 88%|████████▊ | 197317/225000 [375:26:45<60:10:09,  7.82s/it]
 88%|████████▊ | 197318/225000 [375:26:56<66:44:00,  8.68s/it]
 88%|████████▊ | 197319/225000 [375:27:01<58:42:58,  7.64s/it]
 88%|████████▊ | 197320/225000 [375:27:12<67:13:31,  8.74s/it]
 88%|████████▊ | 197321/225000 [375:27:20<65:22:28,  8.50s/it]
 88%|████████▊ | 197322/225000 [375:27:25<57:35:00,  7.49s/it]
 88%|████████▊ | 197323/225000 [375:27:37<67:33:36,  8.79s/it]
 88%|████████▊ | 197324/225000 [375:27:48<72:23:56,  9.42s/it]
 88%|████████▊ | 197325/225000 [375:28:01<80:18:53, 10.45s/it]
 88%|████████▊ | 197326/225000 [375:28:10<77:06:12, 10.03s/it]
 88%|████████▊ | 197327/225000 [375:28:16<67:43:03,  8.81s/it]
 88%|████████▊ | 197328/225000 [375:28:22<61:17:13,  7.97s/it]
 88%|████████▊ | 197329/225000 [375:28:32<66:51:18,  8.70s/it]
 88%|████████▊ | 197330/225000 [375:28:40<64:27:05,  8.39s/it]
 88%|████████▊ | 197331/225000 [375:28:45<57:12:36,  7.44s/it]
 88%|████████▊ | 197332/225000 [375:28:52<55:59:56,  7.29s/it]
 88%|████████▊ | 197333/225000 [375:29:04<67:31:49,  8.79s/it]
 88%|████████▊ | 197334/225000 [375:29:12<64:02:04,  8.33s/it]
 88%|████████▊ | 197335/225000 [375:29:18<60:39:05,  7.89s/it]
 88%|████████▊ | 197336/225000 [375:29:25<57:57:36,  7.54s/it]
 88%|████████▊ | 197337/225000 [375:29:37<67:17:11,  8.76s/it]
 88%|████████▊ | 197338/225000 [375:29:48<72:09:03,  9.39s/it]
 88%|████████▊ | 197339/225000 [375:29:58<75:21:39,  9.81s/it]
 88%|████████▊ | 197340/225000 [375:30:03<63:51:31,  8.31s/it]
 88%|████████▊ | 197341/225000 [375:30:12<65:30:31,  8.53s/it]
 88%|████████▊ | 197342/225000 [375:30:17<57:22:23,  7.47s/it]
 88%|████████▊ | 197343/225000 [375:30:26<61:00:32,  7.94s/it]
 88%|████████▊ | 197344/225000 [375:30:37<67:00:17,  8.72s/it]
 88%|████████▊ | 197345/225000 [375:30:42<58:01:42,  7.55s/it]
 88%|████████▊ | 197346/225000 [375:30:48<56:06:34,  7.30s/it]
 88%|████████▊ | 197347/225000 [375:30:56<57:51:40,  7.53s/it]
 88%|████████▊ | 197348/225000 [375:31:07<65:30:41,  8.53s/it]
 88%|████████▊ | 197349/225000 [375:31:16<66:57:02,  8.72s/it]
 88%|████████▊ | 197350/225000 [375:31:24<63:42:46,  8.30s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.406', 'learning_rate': '2.026e-06', 'epoch': '0.8771', 'num_input_tokens_seen': 627967984, 'train_runtime': '1.352e+06', 'train_tokens_per_second': '464.5'}

 88%|████████▊ | 197350/225000 [375:31:24<63:42:46,  8.30s/it]
 88%|████████▊ | 197351/225000 [375:31:32<63:38:05,  8.29s/it]
 88%|████████▊ | 197352/225000 [375:31:37<56:16:54,  7.33s/it]
 88%|████████▊ | 197353/225000 [375:31:44<54:16:50,  7.07s/it]
 88%|████████▊ | 197354/225000 [375:31:57<68:45:44,  8.95s/it]
 88%|████████▊ | 197355/225000 [375:32:01<58:13:00,  7.58s/it]
 88%|████████▊ | 197356/225000 [375:32:15<71:48:07,  9.35s/it]
 88%|████████▊ | 197357/225000 [375:32:25<73:25:50,  9.56s/it]
 88%|████████▊ | 197358/225000 [375:32:32<68:08:09,  8.87s/it]
 88%|████████▊ | 197359/225000 [375:32:43<71:49:12,  9.35s/it]
 88%|████████▊ | 197360/225000 [375:32:53<75:06:55,  9.78s/it]
 88%|████████▊ | 197361/225000 [375:32:57<61:56:45,  8.07s/it]
 88%|████████▊ | 197362/225000 [375:33:05<61:48:23,  8.05s/it]
 88%|████████▊ | 197363/225000 [375:33:12<58:53:02,  7.67s/it]
 88%|████████▊ | 197364/225000 [375:33:26<72:46:02,  9.48s/it]
 88%|████████▊ | 197365/225000 [375:33:33<67:28:20,  8.79s/it]
 88%|████████▊ | 197366/225000 [375:33:44<72:07:25,  9.40s/it]
 88%|████████▊ | 197367/225000 [375:33:52<68:35:18,  8.94s/it]
 88%|████████▊ | 197368/225000 [375:33:59<64:47:00,  8.44s/it]
 88%|████████▊ | 197369/225000 [375:34:07<63:48:49,  8.31s/it]
 88%|████████▊ | 197370/225000 [375:34:15<61:54:20,  8.07s/it]
 88%|████████▊ | 197371/225000 [375:34:27<72:23:20,  9.43s/it]
 88%|████████▊ | 197372/225000 [375:34:32<62:05:27,  8.09s/it]
 88%|████████▊ | 197373/225000 [375:34:39<59:18:45,  7.73s/it]
 88%|████████▊ | 197374/225000 [375:34:50<65:51:50,  8.58s/it]
 88%|████████▊ | 197375/225000 [375:34:58<64:49:57,  8.45s/it]
 88%|████████▊ | 197376/225000 [375:35:06<64:51:19,  8.45s/it]
 88%|████████▊ | 197377/225000 [375:35:16<67:14:36,  8.76s/it]
 88%|████████▊ | 197378/225000 [375:35:21<60:09:56,  7.84s/it]
 88%|████████▊ | 197379/225000 [375:35:26<53:01:21,  6.91s/it]
 88%|████████▊ | 197380/225000 [375:35:34<55:24:29,  7.22s/it]
 88%|████████▊ | 197381/225000 [375:35:46<65:31:51,  8.54s/it]
 88%|████████▊ | 197382/225000 [375:35:56<70:16:15,  9.16s/it]
 88%|████████▊ | 197383/225000 [375:36:01<60:00:33,  7.82s/it]
 88%|████████▊ | 197384/225000 [375:36:09<60:14:23,  7.85s/it]
 88%|████████▊ | 197385/225000 [375:36:18<63:01:12,  8.22s/it]
 88%|████████▊ | 197386/225000 [375:36:26<62:47:29,  8.19s/it]
 88%|████████▊ | 197387/225000 [375:36:33<58:53:10,  7.68s/it]
 88%|████████▊ | 197388/225000 [375:36:39<54:56:32,  7.16s/it]
 88%|████████▊ | 197389/225000 [375:36:53<71:28:05,  9.32s/it]
 88%|████████▊ | 197390/225000 [375:37:03<72:48:00,  9.49s/it]
 88%|████████▊ | 197391/225000 [375:37:09<64:16:42,  8.38s/it]
 88%|████████▊ | 197392/225000 [375:37:18<67:44:11,  8.83s/it]
 88%|████████▊ | 197393/225000 [375:37:25<63:00:50,  8.22s/it]
 88%|████████▊ | 197394/225000 [375:37:32<60:20:45,  7.87s/it]
 88%|████████▊ | 197395/225000 [375:37:38<54:36:55,  7.12s/it]
 88%|████████▊ | 197396/225000 [375:37:52<72:15:35,  9.42s/it]
 88%|████████▊ | 197397/225000 [375:38:04<77:23:41, 10.09s/it]
 88%|████████▊ | 197398/225000 [375:38:14<76:42:08, 10.00s/it]
 88%|████████▊ | 197399/225000 [375:38:24<76:00:16,  9.91s/it]
 88%|████████▊ | 197400/225000 [375:38:30<66:49:01,  8.72s/it]
                                                              
{'loss': '1.275', 'grad_norm': '1.257', 'learning_rate': '2.019e-06', 'epoch': '0.8773', 'num_input_tokens_seen': 628192928, 'train_runtime': '1.352e+06', 'train_tokens_per_second': '464.5'}

 88%|████████▊ | 197400/225000 [375:38:30<66:49:01,  8.72s/it]
 88%|████████▊ | 197401/225000 [375:38:36<61:14:32,  7.99s/it]
 88%|████████▊ | 197402/225000 [375:38:49<73:09:06,  9.54s/it]
 88%|████████▊ | 197403/225000 [375:38:59<74:47:17,  9.76s/it]
 88%|████████▊ | 197404/225000 [375:39:09<74:16:50,  9.69s/it]
 88%|████████▊ | 197405/225000 [375:39:13<62:07:03,  8.10s/it]
 88%|████████▊ | 197406/225000 [375:39:21<62:19:49,  8.13s/it]
 88%|████████▊ | 197407/225000 [375:39:30<64:00:09,  8.35s/it]
 88%|████████▊ | 197408/225000 [375:39:38<61:50:57,  8.07s/it]
 88%|████████▊ | 197409/225000 [375:39:48<68:02:39,  8.88s/it]
 88%|████████▊ | 197410/225000 [375:40:00<74:04:14,  9.66s/it]
 88%|████████▊ | 197411/225000 [375:40:07<67:46:49,  8.84s/it]
 88%|████████▊ | 197412/225000 [375:40:14<63:49:13,  8.33s/it]
 88%|████████▊ | 197413/225000 [375:40:24<68:23:25,  8.92s/it]
 88%|████████▊ | 197414/225000 [375:40:37<76:54:54, 10.04s/it]
 88%|████████▊ | 197415/225000 [375:40:44<70:14:36,  9.17s/it]
 88%|████████▊ | 197416/225000 [375:40:52<66:51:01,  8.72s/it]
 88%|████████▊ | 197417/225000 [375:40:56<55:51:08,  7.29s/it]
 88%|████████▊ | 197418/225000 [375:41:02<54:07:10,  7.06s/it]
 88%|████████▊ | 197419/225000 [375:41:15<68:09:29,  8.90s/it]
 88%|████████▊ | 197420/225000 [375:41:28<76:53:45, 10.04s/it]
 88%|████████▊ | 197421/225000 [375:41:37<74:10:45,  9.68s/it]
 88%|████████▊ | 197422/225000 [375:41:49<80:10:00, 10.46s/it]
 88%|████████▊ | 197423/225000 [375:41:57<72:49:10,  9.51s/it]
 88%|████████▊ | 197424/225000 [375:42:08<76:57:17, 10.05s/it]
 88%|████████▊ | 197425/225000 [375:42:13<65:44:02,  8.58s/it]
 88%|████████▊ | 197426/225000 [375:42:24<70:14:59,  9.17s/it]
 88%|████████▊ | 197427/225000 [375:42:28<60:06:48,  7.85s/it]
 88%|████████▊ | 197428/225000 [375:42:39<66:47:52,  8.72s/it]
 88%|████████▊ | 197429/225000 [375:42:46<63:38:54,  8.31s/it]
 88%|████████▊ | 197430/225000 [375:42:53<58:57:37,  7.70s/it]
 88%|████████▊ | 197431/225000 [375:42:58<52:56:29,  6.91s/it]
 88%|████████▊ | 197432/225000 [375:43:12<69:11:07,  9.03s/it]
 88%|████████▊ | 197433/225000 [375:43:19<65:52:05,  8.60s/it]
 88%|████████▊ | 197434/225000 [375:43:28<65:24:47,  8.54s/it]
 88%|████████▊ | 197435/225000 [375:43:35<63:28:15,  8.29s/it]
 88%|████████▊ | 197436/225000 [375:43:45<66:17:26,  8.66s/it]
 88%|████████▊ | 197437/225000 [375:43:56<71:04:59,  9.28s/it]
 88%|████████▊ | 197438/225000 [375:44:08<77:03:17, 10.06s/it]
 88%|████████▊ | 197439/225000 [375:44:18<76:52:39, 10.04s/it]
 88%|████████▊ | 197440/225000 [375:44:23<66:59:22,  8.75s/it]
 88%|████████▊ | 197441/225000 [375:44:30<61:19:31,  8.01s/it]
 88%|████████▊ | 197442/225000 [375:44:39<63:17:55,  8.27s/it]
 88%|████████▊ | 197443/225000 [375:44:49<68:24:24,  8.94s/it]
 88%|████████▊ | 197444/225000 [375:44:53<56:41:46,  7.41s/it]
 88%|████████▊ | 197445/225000 [375:45:04<65:11:56,  8.52s/it]
 88%|████████▊ | 197446/225000 [375:45:12<64:42:50,  8.46s/it]
 88%|████████▊ | 197447/225000 [375:45:26<77:37:47, 10.14s/it]
 88%|████████▊ | 197448/225000 [375:45:35<73:13:40,  9.57s/it]
 88%|████████▊ | 197449/225000 [375:45:40<62:57:53,  8.23s/it]
 88%|████████▊ | 197450/225000 [375:45:47<61:11:42,  8.00s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.203', 'learning_rate': '2.012e-06', 'epoch': '0.8776', 'num_input_tokens_seen': 628426000, 'train_runtime': '1.353e+06', 'train_tokens_per_second': '464.6'}

 88%|████████▊ | 197450/225000 [375:45:47<61:11:42,  8.00s/it]
 88%|████████▊ | 197451/225000 [375:45:58<68:12:02,  8.91s/it]
 88%|████████▊ | 197452/225000 [375:46:09<73:33:26,  9.61s/it]
 88%|████████▊ | 197453/225000 [375:46:20<75:38:27,  9.89s/it]
 88%|████████▊ | 197454/225000 [375:46:28<71:56:14,  9.40s/it]
 88%|████████▊ | 197455/225000 [375:46:34<64:19:29,  8.41s/it]
 88%|████████▊ | 197456/225000 [375:46:42<63:41:26,  8.32s/it]
 88%|████████▊ | 197457/225000 [375:46:51<63:11:02,  8.26s/it]
 88%|████████▊ | 197458/225000 [375:47:00<65:17:52,  8.54s/it]
 88%|████████▊ | 197459/225000 [375:47:08<64:35:04,  8.44s/it]
 88%|████████▊ | 197460/225000 [375:47:19<70:19:10,  9.19s/it]
 88%|████████▊ | 197461/225000 [375:47:24<60:53:00,  7.96s/it]
 88%|████████▊ | 197462/225000 [375:47:34<65:56:47,  8.62s/it]
 88%|████████▊ | 197463/225000 [375:47:42<63:04:21,  8.25s/it]
 88%|████████▊ | 197464/225000 [375:47:53<70:15:36,  9.19s/it]
 88%|████████▊ | 197465/225000 [375:47:59<62:30:43,  8.17s/it]
 88%|████████▊ | 197466/225000 [375:48:03<53:33:12,  7.00s/it]
 88%|████████▊ | 197467/225000 [375:48:15<64:55:27,  8.49s/it]
 88%|████████▊ | 197468/225000 [375:48:19<55:38:35,  7.28s/it]
 88%|████████▊ | 197469/225000 [375:48:30<63:56:01,  8.36s/it]
 88%|████████▊ | 197470/225000 [375:48:37<60:03:30,  7.85s/it]
 88%|████████▊ | 197471/225000 [375:48:48<66:40:16,  8.72s/it]
 88%|████████▊ | 197472/225000 [375:48:55<62:23:22,  8.16s/it]
 88%|████████▊ | 197473/225000 [375:49:03<62:55:16,  8.23s/it]
 88%|████████▊ | 197474/225000 [375:49:08<55:19:07,  7.23s/it]
 88%|████████▊ | 197475/225000 [375:49:18<61:11:22,  8.00s/it]
 88%|████████▊ | 197476/225000 [375:49:23<55:20:00,  7.24s/it]
 88%|████████▊ | 197477/225000 [375:49:29<52:15:43,  6.84s/it]
 88%|████████▊ | 197478/225000 [375:49:35<50:05:29,  6.55s/it]
 88%|████████▊ | 197479/225000 [375:49:49<68:21:44,  8.94s/it]
 88%|████████▊ | 197480/225000 [375:50:02<76:40:05, 10.03s/it]
 88%|████████▊ | 197481/225000 [375:50:07<64:25:44,  8.43s/it]
 88%|████████▊ | 197482/225000 [375:50:14<61:39:33,  8.07s/it]
 88%|████████▊ | 197483/225000 [375:50:20<56:10:32,  7.35s/it]
 88%|████████▊ | 197484/225000 [375:50:24<50:31:03,  6.61s/it]
 88%|████████▊ | 197485/225000 [375:50:39<68:57:08,  9.02s/it]
 88%|████████▊ | 197486/225000 [375:50:46<63:05:54,  8.26s/it]
 88%|████████▊ | 197487/225000 [375:50:54<64:00:25,  8.38s/it]
 88%|████████▊ | 197488/225000 [375:51:06<71:03:52,  9.30s/it]
 88%|████████▊ | 197489/225000 [375:51:11<62:07:32,  8.13s/it]
 88%|████████▊ | 197490/225000 [375:51:22<67:43:01,  8.86s/it]
 88%|████████▊ | 197491/225000 [375:51:27<60:36:06,  7.93s/it]
 88%|████████▊ | 197492/225000 [375:51:39<67:56:20,  8.89s/it]
 88%|████████▊ | 197493/225000 [375:51:44<59:57:54,  7.85s/it]
 88%|████████▊ | 197494/225000 [375:51:50<55:28:42,  7.26s/it]
 88%|████████▊ | 197495/225000 [375:52:03<69:47:33,  9.13s/it]
 88%|████████▊ | 197496/225000 [375:52:11<66:12:46,  8.67s/it]
 88%|████████▊ | 197497/225000 [375:52:17<60:23:43,  7.91s/it]
 88%|████████▊ | 197498/225000 [375:52:28<68:01:20,  8.90s/it]
 88%|████████▊ | 197499/225000 [375:52:35<63:55:35,  8.37s/it]
 88%|████████▊ | 197500/225000 [375:52:42<60:32:47,  7.93s/it]
                                                              
{'loss': '1.284', 'grad_norm': '1.632', 'learning_rate': '2.005e-06', 'epoch': '0.8778', 'num_input_tokens_seen': 628652296, 'train_runtime': '1.353e+06', 'train_tokens_per_second': '464.6'}

 88%|████████▊ | 197500/225000 [375:52:42<60:32:47,  7.93s/it]
 88%|████████▊ | 197501/225000 [375:52:50<60:20:15,  7.90s/it]
 88%|████████▊ | 197502/225000 [375:52:58<60:46:31,  7.96s/it]
 88%|████████▊ | 197503/225000 [375:53:11<71:07:50,  9.31s/it]
 88%|████████▊ | 197504/225000 [375:53:14<57:48:27,  7.57s/it]
 88%|████████▊ | 197505/225000 [375:53:23<60:22:31,  7.91s/it]
 88%|████████▊ | 197506/225000 [375:53:31<61:56:16,  8.11s/it]
 88%|████████▊ | 197507/225000 [375:53:39<59:49:14,  7.83s/it]
 88%|████████▊ | 197508/225000 [375:53:49<66:23:34,  8.69s/it]
 88%|████████▊ | 197509/225000 [375:53:55<59:42:13,  7.82s/it]
 88%|████████▊ | 197510/225000 [375:54:04<62:54:09,  8.24s/it]
 88%|████████▊ | 197511/225000 [375:54:14<65:36:59,  8.59s/it]
 88%|████████▊ | 197512/225000 [375:54:19<58:35:09,  7.67s/it]
 88%|████████▊ | 197513/225000 [375:54:26<55:24:07,  7.26s/it]
 88%|████████▊ | 197514/225000 [375:54:36<61:52:58,  8.11s/it]
 88%|████████▊ | 197515/225000 [375:54:41<55:00:48,  7.21s/it]
 88%|████████▊ | 197516/225000 [375:54:48<54:26:52,  7.13s/it]
 88%|████████▊ | 197517/225000 [375:54:59<64:15:37,  8.42s/it]
 88%|████████▊ | 197518/225000 [375:55:06<60:09:28,  7.88s/it]
 88%|████████▊ | 197519/225000 [375:55:13<57:36:36,  7.55s/it]
 88%|████████▊ | 197520/225000 [375:55:22<62:44:41,  8.22s/it]
 88%|████████▊ | 197521/225000 [375:55:29<58:31:41,  7.67s/it]
 88%|████████▊ | 197522/225000 [375:55:36<56:56:27,  7.46s/it]
 88%|████████▊ | 197523/225000 [375:55:42<54:49:31,  7.18s/it]
 88%|████████▊ | 197524/225000 [375:55:47<49:55:44,  6.54s/it]
 88%|████████▊ | 197525/225000 [375:56:00<63:55:56,  8.38s/it]
 88%|████████▊ | 197526/225000 [375:56:06<57:46:18,  7.57s/it]
 88%|████████▊ | 197527/225000 [375:56:17<67:31:09,  8.85s/it]
 88%|████████▊ | 197528/225000 [375:56:26<67:12:43,  8.81s/it]
 88%|████████▊ | 197529/225000 [375:56:34<64:53:23,  8.50s/it]
 88%|████████▊ | 197530/225000 [375:56:38<55:25:51,  7.26s/it]
 88%|████████▊ | 197531/225000 [375:56:43<50:14:50,  6.59s/it]
 88%|████████▊ | 197532/225000 [375:56:51<52:50:56,  6.93s/it]
 88%|████████▊ | 197533/225000 [375:56:58<51:54:16,  6.80s/it]
 88%|████████▊ | 197534/225000 [375:57:07<58:27:29,  7.66s/it]
 88%|████████▊ | 197535/225000 [375:57:14<57:15:28,  7.51s/it]
 88%|████████▊ | 197536/225000 [375:57:31<77:29:14, 10.16s/it]
 88%|████████▊ | 197537/225000 [375:57:36<65:55:36,  8.64s/it]
 88%|████████▊ | 197538/225000 [375:57:41<58:46:06,  7.70s/it]
 88%|████████▊ | 197539/225000 [375:57:54<70:47:44,  9.28s/it]
 88%|████████▊ | 197540/225000 [375:58:03<68:34:26,  8.99s/it]
 88%|████████▊ | 197541/225000 [375:58:08<60:29:50,  7.93s/it]
 88%|████████▊ | 197542/225000 [375:58:14<55:27:50,  7.27s/it]
 88%|████████▊ | 197543/225000 [375:58:27<68:08:17,  8.93s/it]
 88%|████████▊ | 197544/225000 [375:58:35<66:46:28,  8.76s/it]
 88%|████████▊ | 197545/225000 [375:58:47<74:51:26,  9.82s/it]
 88%|████████▊ | 197546/225000 [375:58:58<76:02:16,  9.97s/it]
 88%|████████▊ | 197547/225000 [375:59:06<72:34:57,  9.52s/it]
 88%|████████▊ | 197548/225000 [375:59:16<74:08:59,  9.72s/it]
 88%|████████▊ | 197549/225000 [375:59:23<68:26:43,  8.98s/it]
 88%|████████▊ | 197550/225000 [375:59:30<63:25:03,  8.32s/it]
                                                              
{'loss': '1.274', 'grad_norm': '1.403', 'learning_rate': '1.997e-06', 'epoch': '0.878', 'num_input_tokens_seen': 628878352, 'train_runtime': '1.354e+06', 'train_tokens_per_second': '464.6'}

 88%|████████▊ | 197550/225000 [375:59:30<63:25:03,  8.32s/it]
 88%|████████▊ | 197551/225000 [375:59:46<80:07:46, 10.51s/it]
 88%|████████▊ | 197552/225000 [375:59:51<68:22:01,  8.97s/it]
 88%|████████▊ | 197553/225000 [376:00:02<71:22:20,  9.36s/it]
 88%|████████▊ | 197554/225000 [376:00:08<63:56:46,  8.39s/it]
 88%|████████▊ | 197555/225000 [376:00:13<57:23:38,  7.53s/it]
 88%|████████▊ | 197556/225000 [376:00:24<65:11:09,  8.55s/it]
 88%|████████▊ | 197557/225000 [376:00:29<57:45:16,  7.58s/it]
 88%|████████▊ | 197558/225000 [376:00:34<51:37:11,  6.77s/it]
 88%|████████▊ | 197559/225000 [376:00:44<58:54:02,  7.73s/it]
 88%|████████▊ | 197560/225000 [376:00:55<66:44:51,  8.76s/it]
 88%|████████▊ | 197561/225000 [376:01:03<63:35:38,  8.34s/it]
 88%|████████▊ | 197562/225000 [376:01:13<68:57:16,  9.05s/it]
 88%|████████▊ | 197563/225000 [376:01:21<65:01:15,  8.53s/it]
 88%|████████▊ | 197564/225000 [376:01:33<72:42:43,  9.54s/it]
 88%|████████▊ | 197565/225000 [376:01:41<69:21:36,  9.10s/it]
 88%|████████▊ | 197566/225000 [376:01:47<61:59:13,  8.13s/it]
 88%|████████▊ | 197567/225000 [376:01:53<58:30:04,  7.68s/it]
 88%|████████▊ | 197568/225000 [376:02:00<55:27:48,  7.28s/it]
 88%|████████▊ | 197569/225000 [376:02:05<50:11:08,  6.59s/it]
 88%|████████▊ | 197570/225000 [376:02:12<52:12:25,  6.85s/it]
 88%|████████▊ | 197571/225000 [376:02:25<65:59:52,  8.66s/it]
 88%|████████▊ | 197572/225000 [376:02:34<65:57:37,  8.66s/it]
 88%|████████▊ | 197573/225000 [376:02:44<69:51:47,  9.17s/it]
 88%|████████▊ | 197574/225000 [376:02:55<74:17:56,  9.75s/it]
 88%|████████▊ | 197575/225000 [376:03:02<68:03:15,  8.93s/it]
 88%|████████▊ | 197576/225000 [376:03:16<78:29:36, 10.30s/it]
 88%|████████▊ | 197577/225000 [376:03:21<66:27:06,  8.72s/it]
 88%|████████▊ | 197578/225000 [376:03:27<61:45:09,  8.11s/it]
 88%|████████▊ | 197579/225000 [376:03:35<61:01:44,  8.01s/it]
 88%|████████▊ | 197580/225000 [376:03:51<79:27:38, 10.43s/it]
 88%|████████▊ | 197581/225000 [376:04:01<78:20:56, 10.29s/it]
 88%|████████▊ | 197582/225000 [376:04:08<71:38:49,  9.41s/it]
 88%|████████▊ | 197583/225000 [376:04:23<83:14:18, 10.93s/it]
 88%|████████▊ | 197584/225000 [376:04:28<69:10:20,  9.08s/it]
 88%|████████▊ | 197585/225000 [376:04:38<71:21:04,  9.37s/it]
 88%|████████▊ | 197586/225000 [376:04:43<62:00:45,  8.14s/it]
 88%|████████▊ | 197587/225000 [376:04:54<67:35:33,  8.88s/it]
 88%|████████▊ | 197588/225000 [376:05:04<70:05:17,  9.20s/it]
 88%|████████▊ | 197589/225000 [376:05:11<65:10:50,  8.56s/it]
 88%|████████▊ | 197590/225000 [376:05:18<61:37:03,  8.09s/it]
 88%|████████▊ | 197591/225000 [376:05:28<66:33:38,  8.74s/it]
 88%|████████▊ | 197592/225000 [376:05:34<60:35:19,  7.96s/it]
 88%|████████▊ | 197593/225000 [376:05:42<60:36:00,  7.96s/it]
 88%|████████▊ | 197594/225000 [376:05:51<63:25:14,  8.33s/it]
 88%|████████▊ | 197595/225000 [376:05:58<59:53:19,  7.87s/it]
 88%|████████▊ | 197596/225000 [376:06:06<59:05:56,  7.76s/it]
 88%|████████▊ | 197597/225000 [376:06:20<73:18:56,  9.63s/it]
 88%|████████▊ | 197598/225000 [376:06:26<65:02:09,  8.54s/it]
 88%|████████▊ | 197599/225000 [376:06:31<59:06:32,  7.77s/it]
 88%|████████▊ | 197600/225000 [376:06:47<76:07:42, 10.00s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.125', 'learning_rate': '1.99e-06', 'epoch': '0.8782', 'num_input_tokens_seen': 629115072, 'train_runtime': '1.354e+06', 'train_tokens_per_second': '464.6'}

 88%|████████▊ | 197600/225000 [376:06:47<76:07:42, 10.00s/it]
 88%|████████▊ | 197601/225000 [376:06:54<69:49:45,  9.17s/it]
 88%|████████▊ | 197602/225000 [376:07:00<63:10:14,  8.30s/it]
 88%|████████▊ | 197603/225000 [376:07:08<61:12:30,  8.04s/it]
 88%|████████▊ | 197604/225000 [376:07:22<76:40:26, 10.08s/it]
 88%|████████▊ | 197605/225000 [376:07:31<72:03:44,  9.47s/it]
 88%|████████▊ | 197606/225000 [376:07:43<78:29:50, 10.32s/it]
 88%|████████▊ | 197607/225000 [376:07:50<70:47:19,  9.30s/it]
 88%|████████▊ | 197608/225000 [376:08:03<79:05:03, 10.39s/it]
 88%|████████▊ | 197609/225000 [376:08:13<78:35:25, 10.33s/it]
 88%|████████▊ | 197610/225000 [376:08:24<80:30:32, 10.58s/it]
 88%|████████▊ | 197611/225000 [376:08:30<70:22:32,  9.25s/it]
 88%|████████▊ | 197612/225000 [376:08:35<61:10:08,  8.04s/it]
 88%|████████▊ | 197613/225000 [376:08:46<67:08:04,  8.82s/it]
 88%|████████▊ | 197614/225000 [376:08:52<59:37:54,  7.84s/it]
 88%|████████▊ | 197615/225000 [376:09:05<71:57:54,  9.46s/it]
 88%|████████▊ | 197616/225000 [376:09:13<68:35:43,  9.02s/it]
 88%|████████▊ | 197617/225000 [376:09:17<58:36:52,  7.71s/it]
 88%|████████▊ | 197618/225000 [376:09:27<62:12:21,  8.18s/it]
 88%|████████▊ | 197619/225000 [376:09:34<59:55:17,  7.88s/it]
 88%|████████▊ | 197620/225000 [376:09:40<55:49:45,  7.34s/it]
 88%|████████▊ | 197621/225000 [376:09:46<52:11:08,  6.86s/it]
 88%|████████▊ | 197622/225000 [376:09:59<67:24:28,  8.86s/it]
 88%|████████▊ | 197623/225000 [376:10:08<67:49:45,  8.92s/it]
 88%|████████▊ | 197624/225000 [376:10:19<72:56:14,  9.59s/it]
 88%|████████▊ | 197625/225000 [376:10:27<69:14:48,  9.11s/it]
 88%|████████▊ | 197626/225000 [376:10:33<61:12:05,  8.05s/it]
 88%|████████▊ | 197627/225000 [376:10:40<58:31:46,  7.70s/it]
 88%|████████▊ | 197628/225000 [376:10:49<62:17:09,  8.19s/it]
 88%|████████▊ | 197629/225000 [376:10:59<66:54:05,  8.80s/it]
 88%|████████▊ | 197630/225000 [376:11:08<67:07:51,  8.83s/it]
 88%|████████▊ | 197631/225000 [376:11:15<62:05:37,  8.17s/it]
 88%|████████▊ | 197632/225000 [376:11:23<61:27:32,  8.08s/it]
 88%|████████▊ | 197633/225000 [376:11:39<80:28:32, 10.59s/it]
 88%|████████▊ | 197634/225000 [376:11:45<69:42:46,  9.17s/it]
 88%|████████▊ | 197635/225000 [376:11:52<65:27:51,  8.61s/it]
 88%|████████▊ | 197636/225000 [376:12:01<64:20:06,  8.46s/it]
 88%|████████▊ | 197637/225000 [376:12:12<70:18:27,  9.25s/it]
 88%|████████▊ | 197638/225000 [376:12:18<62:47:29,  8.26s/it]
 88%|████████▊ | 197639/225000 [376:12:26<62:58:01,  8.28s/it]
 88%|████████▊ | 197640/225000 [376:12:39<72:53:12,  9.59s/it]
 88%|████████▊ | 197641/225000 [376:12:45<64:55:28,  8.54s/it]
 88%|████████▊ | 197642/225000 [376:12:53<64:10:06,  8.44s/it]
 88%|████████▊ | 197643/225000 [376:12:59<57:51:11,  7.61s/it]
 88%|████████▊ | 197644/225000 [376:13:09<64:18:55,  8.46s/it]
 88%|████████▊ | 197645/225000 [376:13:21<72:44:49,  9.57s/it]
 88%|████████▊ | 197646/225000 [376:13:26<62:18:26,  8.20s/it]
 88%|████████▊ | 197647/225000 [376:13:34<62:09:29,  8.18s/it]
 88%|████████▊ | 197648/225000 [376:13:43<63:45:25,  8.39s/it]
 88%|████████▊ | 197649/225000 [376:13:50<60:17:55,  7.94s/it]
 88%|████████▊ | 197650/225000 [376:13:56<56:10:59,  7.40s/it]
                                                              
{'loss': '1.265', 'grad_norm': '1.171', 'learning_rate': '1.983e-06', 'epoch': '0.8784', 'num_input_tokens_seen': 629348600, 'train_runtime': '1.354e+06', 'train_tokens_per_second': '464.7'}

 88%|████████▊ | 197650/225000 [376:13:56<56:10:59,  7.40s/it]
 88%|████████▊ | 197651/225000 [376:14:05<58:53:29,  7.75s/it]
 88%|████████▊ | 197652/225000 [376:14:11<55:36:36,  7.32s/it]
 88%|████████▊ | 197653/225000 [376:14:18<55:10:17,  7.26s/it]
 88%|████████▊ | 197654/225000 [376:14:28<61:29:28,  8.10s/it]
 88%|████████▊ | 197655/225000 [376:14:40<68:36:34,  9.03s/it]
 88%|████████▊ | 197656/225000 [376:14:47<64:55:58,  8.55s/it]
 88%|████████▊ | 197657/225000 [376:14:58<70:19:28,  9.26s/it]
 88%|████████▊ | 197658/225000 [376:15:03<60:21:01,  7.95s/it]
 88%|████████▊ | 197659/225000 [376:15:12<63:34:14,  8.37s/it]
 88%|████████▊ | 197660/225000 [376:15:21<63:43:12,  8.39s/it]
 88%|████████▊ | 197661/225000 [376:15:27<59:04:18,  7.78s/it]
 88%|████████▊ | 197662/225000 [376:15:34<58:23:58,  7.69s/it]
 88%|████████▊ | 197663/225000 [376:15:40<54:35:15,  7.19s/it]
 88%|████████▊ | 197664/225000 [376:15:48<55:09:42,  7.26s/it]
 88%|████████▊ | 197665/225000 [376:15:59<63:25:12,  8.35s/it]
 88%|████████▊ | 197666/225000 [376:16:09<67:59:26,  8.95s/it]
 88%|████████▊ | 197667/225000 [376:16:17<64:44:19,  8.53s/it]
 88%|████████▊ | 197668/225000 [376:16:23<59:14:50,  7.80s/it]
 88%|████████▊ | 197669/225000 [376:16:29<55:33:37,  7.32s/it]
 88%|████████▊ | 197670/225000 [376:16:44<73:15:09,  9.65s/it]
 88%|████████▊ | 197671/225000 [376:16:52<69:40:33,  9.18s/it]
 88%|████████▊ | 197672/225000 [376:16:59<63:58:24,  8.43s/it]
 88%|████████▊ | 197673/225000 [376:17:04<56:44:55,  7.48s/it]
 88%|████████▊ | 197674/225000 [376:17:15<64:23:21,  8.48s/it]
 88%|████████▊ | 197675/225000 [376:17:26<69:20:58,  9.14s/it]
 88%|████████▊ | 197676/225000 [376:17:32<62:50:38,  8.28s/it]
 88%|████████▊ | 197677/225000 [376:17:38<57:00:13,  7.51s/it]
 88%|████████▊ | 197678/225000 [376:17:49<65:56:02,  8.69s/it]
 88%|████████▊ | 197679/225000 [376:17:56<61:13:32,  8.07s/it]
 88%|████████▊ | 197680/225000 [376:18:02<58:09:43,  7.66s/it]
 88%|████████▊ | 197681/225000 [376:18:14<66:55:04,  8.82s/it]
 88%|████████▊ | 197682/225000 [376:18:25<71:17:45,  9.40s/it]
 88%|████████▊ | 197683/225000 [376:18:29<60:50:47,  8.02s/it]
 88%|████████▊ | 197684/225000 [376:18:36<56:56:15,  7.50s/it]
 88%|████████▊ | 197685/225000 [376:18:48<67:41:40,  8.92s/it]
 88%|████████▊ | 197686/225000 [376:18:53<58:39:24,  7.73s/it]
 88%|████████▊ | 197687/225000 [376:19:06<70:00:49,  9.23s/it]
 88%|████████▊ | 197688/225000 [376:19:14<67:21:17,  8.88s/it]
 88%|████████▊ | 197689/225000 [376:19:22<65:45:49,  8.67s/it]
 88%|████████▊ | 197690/225000 [376:19:30<64:35:39,  8.51s/it]
 88%|████████▊ | 197691/225000 [376:19:43<74:18:30,  9.80s/it]
 88%|████████▊ | 197692/225000 [376:19:53<74:37:40,  9.84s/it]
 88%|████████▊ | 197693/225000 [376:20:03<74:45:12,  9.86s/it]
 88%|████████▊ | 197694/225000 [376:20:13<75:00:42,  9.89s/it]
 88%|████████▊ | 197695/225000 [376:20:24<77:45:08, 10.25s/it]
 88%|████████▊ | 197696/225000 [376:20:32<73:04:05,  9.63s/it]
 88%|████████▊ | 197697/225000 [376:20:40<68:53:42,  9.08s/it]
 88%|████████▊ | 197698/225000 [376:20:46<61:43:21,  8.14s/it]
 88%|████████▊ | 197699/225000 [376:20:53<60:18:35,  7.95s/it]
 88%|████████▊ | 197700/225000 [376:20:59<56:09:51,  7.41s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.311', 'learning_rate': '1.976e-06', 'epoch': '0.8787', 'num_input_tokens_seen': 629577632, 'train_runtime': '1.355e+06', 'train_tokens_per_second': '464.7'}

 88%|████████▊ | 197700/225000 [376:20:59<56:09:51,  7.41s/it]
 88%|████████▊ | 197701/225000 [376:21:07<56:25:03,  7.44s/it]
 88%|████████▊ | 197702/225000 [376:21:13<52:39:38,  6.94s/it]
 88%|████████▊ | 197703/225000 [376:21:28<72:10:46,  9.52s/it]
 88%|████████▊ | 197704/225000 [376:21:36<69:39:39,  9.19s/it]
 88%|████████▊ | 197705/225000 [376:21:44<65:52:38,  8.69s/it]
 88%|████████▊ | 197706/225000 [376:21:49<56:59:36,  7.52s/it]
 88%|████████▊ | 197707/225000 [376:22:00<64:29:33,  8.51s/it]
 88%|████████▊ | 197708/225000 [376:22:08<65:08:03,  8.59s/it]
 88%|████████▊ | 197709/225000 [376:22:23<77:52:34, 10.27s/it]
 88%|████████▊ | 197710/225000 [376:22:31<72:38:27,  9.58s/it]
 88%|████████▊ | 197711/225000 [376:22:42<76:18:51, 10.07s/it]
 88%|████████▊ | 197712/225000 [376:22:51<74:57:13,  9.89s/it]
 88%|████████▊ | 197713/225000 [376:23:01<75:05:01,  9.91s/it]
 88%|████████▊ | 197714/225000 [376:23:06<64:37:52,  8.53s/it]
 88%|████████▊ | 197715/225000 [376:23:14<62:42:02,  8.27s/it]
 88%|████████▊ | 197716/225000 [376:23:24<67:17:23,  8.88s/it]
 88%|████████▊ | 197717/225000 [376:23:31<62:24:38,  8.24s/it]
 88%|████████▊ | 197718/225000 [376:23:38<59:10:02,  7.81s/it]
 88%|████████▊ | 197719/225000 [376:23:45<57:43:15,  7.62s/it]
 88%|████████▊ | 197720/225000 [376:23:58<70:24:24,  9.29s/it]
 88%|████████▊ | 197721/225000 [376:24:02<58:30:40,  7.72s/it]
 88%|████████▊ | 197722/225000 [376:24:13<64:30:21,  8.51s/it]
 88%|████████▊ | 197723/225000 [376:24:22<65:30:17,  8.65s/it]
 88%|████████▊ | 197724/225000 [376:24:28<59:03:16,  7.79s/it]
 88%|████████▊ | 197725/225000 [376:24:32<51:46:19,  6.83s/it]
 88%|████████▊ | 197726/225000 [376:24:44<62:32:59,  8.26s/it]
 88%|████████▊ | 197727/225000 [376:24:48<54:40:39,  7.22s/it]
 88%|████████▊ | 197728/225000 [376:24:53<49:32:07,  6.54s/it]
 88%|████████▊ | 197729/225000 [376:25:00<50:03:30,  6.61s/it]
 88%|████████▊ | 197730/225000 [376:25:06<49:06:02,  6.48s/it]
 88%|████████▊ | 197731/225000 [376:25:12<45:57:23,  6.07s/it]
 88%|████████▊ | 197732/225000 [376:25:24<61:12:33,  8.08s/it]
 88%|████████▊ | 197733/225000 [376:25:30<55:47:25,  7.37s/it]
 88%|████████▊ | 197734/225000 [376:25:36<53:31:31,  7.07s/it]
 88%|████████▊ | 197735/225000 [376:25:49<67:03:19,  8.85s/it]
 88%|████████▊ | 197736/225000 [376:25:54<57:36:27,  7.61s/it]
 88%|████████▊ | 197737/225000 [376:26:06<66:36:24,  8.80s/it]
 88%|████████▊ | 197738/225000 [376:26:15<68:19:53,  9.02s/it]
 88%|████████▊ | 197739/225000 [376:26:25<70:01:47,  9.25s/it]
 88%|████████▊ | 197740/225000 [376:26:34<69:28:21,  9.17s/it]
 88%|████████▊ | 197741/225000 [376:26:43<69:12:06,  9.14s/it]
 88%|████████▊ | 197742/225000 [376:26:47<58:03:08,  7.67s/it]
 88%|████████▊ | 197743/225000 [376:26:57<62:40:06,  8.28s/it]
 88%|████████▊ | 197744/225000 [376:27:06<64:39:55,  8.54s/it]
 88%|████████▊ | 197745/225000 [376:27:19<75:19:48,  9.95s/it]
 88%|████████▊ | 197746/225000 [376:27:26<68:42:51,  9.08s/it]
 88%|████████▊ | 197747/225000 [376:27:34<64:58:38,  8.58s/it]
 88%|████████▊ | 197748/225000 [376:27:40<59:53:27,  7.91s/it]
 88%|████████▊ | 197749/225000 [376:27:49<62:47:53,  8.30s/it]
 88%|████████▊ | 197750/225000 [376:27:57<60:32:07,  8.00s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.332', 'learning_rate': '1.969e-06', 'epoch': '0.8789', 'num_input_tokens_seen': 629806584, 'train_runtime': '1.355e+06', 'train_tokens_per_second': '464.7'}

 88%|████████▊ | 197750/225000 [376:27:57<60:32:07,  8.00s/it]
 88%|████████▊ | 197751/225000 [376:28:12<76:13:18, 10.07s/it]
 88%|████████▊ | 197752/225000 [376:28:21<75:13:57,  9.94s/it]
 88%|████████▊ | 197753/225000 [376:28:29<70:49:47,  9.36s/it]
 88%|████████▊ | 197754/225000 [376:28:42<77:38:33, 10.26s/it]
 88%|████████▊ | 197755/225000 [376:28:55<84:14:37, 11.13s/it]
 88%|████████▊ | 197756/225000 [376:29:04<79:04:40, 10.45s/it]
 88%|████████▊ | 197757/225000 [376:29:08<65:00:36,  8.59s/it]
 88%|████████▊ | 197758/225000 [376:29:18<69:10:26,  9.14s/it]
 88%|████████▊ | 197759/225000 [376:29:24<62:11:45,  8.22s/it]
 88%|████████▊ | 197760/225000 [376:29:31<58:15:42,  7.70s/it]
 88%|████████▊ | 197761/225000 [376:29:39<59:50:03,  7.91s/it]
 88%|████████▊ | 197762/225000 [376:29:48<62:40:12,  8.28s/it]
 88%|████████▊ | 197763/225000 [376:29:57<62:25:38,  8.25s/it]
 88%|████████▊ | 197764/225000 [376:30:05<62:13:07,  8.22s/it]
 88%|████████▊ | 197765/225000 [376:30:18<73:01:05,  9.65s/it]
 88%|████████▊ | 197766/225000 [376:30:24<66:18:28,  8.77s/it]
 88%|████████▊ | 197767/225000 [376:30:30<59:38:30,  7.88s/it]
 88%|████████▊ | 197768/225000 [376:30:40<63:52:45,  8.44s/it]
 88%|████████▊ | 197769/225000 [376:30:48<62:52:12,  8.31s/it]
 88%|████████▊ | 197770/225000 [376:30:58<65:56:26,  8.72s/it]
 88%|████████▊ | 197771/225000 [376:31:05<63:50:38,  8.44s/it]
 88%|████████▊ | 197772/225000 [376:31:11<57:02:55,  7.54s/it]
 88%|████████▊ | 197773/225000 [376:31:22<64:42:57,  8.56s/it]
 88%|████████▊ | 197774/225000 [376:31:32<69:09:53,  9.15s/it]
 88%|████████▊ | 197775/225000 [376:31:37<59:08:02,  7.82s/it]
 88%|████████▊ | 197776/225000 [376:31:45<60:11:24,  7.96s/it]
 88%|████████▊ | 197777/225000 [376:31:53<58:58:06,  7.80s/it]
 88%|████████▊ | 197778/225000 [376:32:03<65:02:30,  8.60s/it]
 88%|████████▊ | 197779/225000 [376:32:13<67:15:37,  8.90s/it]
 88%|████████▊ | 197780/225000 [376:32:20<62:22:04,  8.25s/it]
 88%|████████▊ | 197781/225000 [376:32:31<70:21:28,  9.31s/it]
 88%|████████▊ | 197782/225000 [376:32:39<67:43:05,  8.96s/it]
 88%|████████▊ | 197783/225000 [376:32:44<57:50:41,  7.65s/it]
 88%|████████▊ | 197784/225000 [376:32:50<54:27:56,  7.20s/it]
 88%|████████▊ | 197785/225000 [376:33:03<67:20:37,  8.91s/it]
 88%|████████▊ | 197786/225000 [376:33:13<69:46:32,  9.23s/it]
 88%|████████▊ | 197787/225000 [376:33:19<61:04:54,  8.08s/it]
 88%|████████▊ | 197788/225000 [376:33:25<57:00:07,  7.54s/it]
 88%|████████▊ | 197789/225000 [376:33:36<65:57:15,  8.73s/it]
 88%|████████▊ | 197790/225000 [376:33:45<66:25:02,  8.79s/it]
 88%|████████▊ | 197791/225000 [376:33:55<67:40:04,  8.95s/it]
 88%|████████▊ | 197792/225000 [376:34:04<69:49:38,  9.24s/it]
 88%|████████▊ | 197793/225000 [376:34:09<59:01:10,  7.81s/it]
 88%|████████▊ | 197794/225000 [376:34:16<58:01:35,  7.68s/it]
 88%|████████▊ | 197795/225000 [376:34:25<61:25:40,  8.13s/it]
 88%|████████▊ | 197796/225000 [376:34:32<56:45:51,  7.51s/it]
 88%|████████▊ | 197797/225000 [376:34:38<54:58:56,  7.28s/it]
 88%|████████▊ | 197798/225000 [376:34:45<52:46:01,  6.98s/it]
 88%|████████▊ | 197799/225000 [376:35:00<72:54:49,  9.65s/it]
 88%|████████▊ | 197800/225000 [376:35:11<75:32:56, 10.00s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.086', 'learning_rate': '1.962e-06', 'epoch': '0.8791', 'num_input_tokens_seen': 630036520, 'train_runtime': '1.356e+06', 'train_tokens_per_second': '464.7'}

 88%|████████▊ | 197800/225000 [376:35:11<75:32:56, 10.00s/it]
 88%|████████▊ | 197801/225000 [376:35:18<68:22:51,  9.05s/it]
 88%|████████▊ | 197802/225000 [376:35:26<66:37:16,  8.82s/it]
 88%|████████▊ | 197803/225000 [376:35:36<68:03:46,  9.01s/it]
 88%|████████▊ | 197804/225000 [376:35:43<64:28:10,  8.53s/it]
 88%|████████▊ | 197805/225000 [376:35:55<72:52:19,  9.65s/it]
 88%|████████▊ | 197806/225000 [376:36:01<63:15:27,  8.37s/it]
 88%|████████▊ | 197807/225000 [376:36:09<61:37:03,  8.16s/it]
 88%|████████▊ | 197808/225000 [376:36:16<59:14:57,  7.84s/it]
 88%|████████▊ | 197809/225000 [376:36:27<67:25:03,  8.93s/it]
 88%|████████▊ | 197810/225000 [376:36:35<65:51:06,  8.72s/it]
 88%|████████▊ | 197811/225000 [376:36:45<68:30:36,  9.07s/it]
 88%|████████▊ | 197812/225000 [376:36:50<58:47:14,  7.78s/it]
 88%|████████▊ | 197813/225000 [376:37:01<66:43:25,  8.84s/it]
 88%|████████▊ | 197814/225000 [376:37:11<68:49:48,  9.11s/it]
 88%|████████▊ | 197815/225000 [376:37:17<61:26:40,  8.14s/it]
 88%|████████▊ | 197816/225000 [376:37:25<60:33:21,  8.02s/it]
 88%|████████▊ | 197817/225000 [376:37:39<73:59:21,  9.80s/it]
 88%|████████▊ | 197818/225000 [376:37:43<62:39:28,  8.30s/it]
 88%|████████▊ | 197819/225000 [376:37:54<68:01:53,  9.01s/it]
 88%|████████▊ | 197820/225000 [376:38:03<67:20:52,  8.92s/it]
 88%|████████▊ | 197821/225000 [376:38:14<72:04:08,  9.55s/it]
 88%|████████▊ | 197822/225000 [376:38:23<72:07:00,  9.55s/it]
 88%|████████▊ | 197823/225000 [376:38:29<63:01:32,  8.35s/it]
 88%|████████▊ | 197824/225000 [376:38:37<62:30:23,  8.28s/it]
 88%|████████▊ | 197825/225000 [376:38:45<60:57:47,  8.08s/it]
 88%|████████▊ | 197826/225000 [376:38:53<62:04:59,  8.22s/it]
 88%|████████▊ | 197827/225000 [376:39:02<62:34:13,  8.29s/it]
 88%|████████▊ | 197828/225000 [376:39:17<77:45:50, 10.30s/it]
 88%|████████▊ | 197829/225000 [376:39:24<70:06:58,  9.29s/it]
 88%|████████▊ | 197830/225000 [376:39:31<66:00:40,  8.75s/it]
 88%|████████▊ | 197831/225000 [376:39:38<62:16:49,  8.25s/it]
 88%|████████▊ | 197832/225000 [376:39:46<61:28:06,  8.15s/it]
 88%|████████▊ | 197833/225000 [376:40:01<76:09:45, 10.09s/it]
 88%|████████▊ | 197834/225000 [376:40:09<71:34:15,  9.48s/it]
 88%|████████▊ | 197835/225000 [376:40:17<68:15:08,  9.05s/it]
 88%|████████▊ | 197836/225000 [376:40:25<65:29:59,  8.68s/it]
 88%|████████▊ | 197837/225000 [376:40:32<62:40:26,  8.31s/it]
 88%|████████▊ | 197838/225000 [376:40:39<59:33:46,  7.89s/it]
 88%|████████▊ | 197839/225000 [376:40:46<58:12:28,  7.72s/it]
 88%|████████▊ | 197840/225000 [376:40:59<69:00:01,  9.15s/it]
 88%|████████▊ | 197841/225000 [376:41:05<62:58:56,  8.35s/it]
 88%|████████▊ | 197842/225000 [376:41:12<58:32:16,  7.76s/it]
 88%|████████▊ | 197843/225000 [376:41:20<59:28:43,  7.88s/it]
 88%|████████▊ | 197844/225000 [376:41:29<62:13:12,  8.25s/it]
 88%|████████▊ | 197845/225000 [376:41:39<67:13:38,  8.91s/it]
 88%|████████▊ | 197846/225000 [376:41:45<60:10:00,  7.98s/it]
 88%|████████▊ | 197847/225000 [376:41:53<59:05:37,  7.83s/it]
 88%|████████▊ | 197848/225000 [376:41:59<55:13:19,  7.32s/it]
 88%|████████▊ | 197849/225000 [376:42:05<53:50:11,  7.14s/it]
 88%|████████▊ | 197850/225000 [376:42:18<65:28:48,  8.68s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.219', 'learning_rate': '1.955e-06', 'epoch': '0.8793', 'num_input_tokens_seen': 630269560, 'train_runtime': '1.356e+06', 'train_tokens_per_second': '464.8'}

 88%|████████▊ | 197850/225000 [376:42:18<65:28:48,  8.68s/it]
 88%|████████▊ | 197851/225000 [376:42:25<63:08:21,  8.37s/it]
 88%|████████▊ | 197852/225000 [376:42:36<67:46:16,  8.99s/it]
 88%|████████▊ | 197853/225000 [376:42:50<79:04:13, 10.49s/it]
 88%|████████▊ | 197854/225000 [376:42:57<70:42:16,  9.38s/it]
 88%|████████▊ | 197855/225000 [376:43:06<71:27:37,  9.48s/it]
 88%|████████▊ | 197856/225000 [376:43:15<69:46:20,  9.25s/it]
 88%|████████▊ | 197857/225000 [376:43:26<74:22:43,  9.86s/it]
 88%|████████▊ | 197858/225000 [376:43:32<65:21:32,  8.67s/it]
 88%|████████▊ | 197859/225000 [376:43:41<66:30:59,  8.82s/it]
 88%|████████▊ | 197860/225000 [376:43:49<64:13:46,  8.52s/it]
 88%|████████▊ | 197861/225000 [376:43:58<64:36:15,  8.57s/it]
 88%|████████▊ | 197862/225000 [376:44:09<69:37:56,  9.24s/it]
 88%|████████▊ | 197863/225000 [376:44:20<74:06:44,  9.83s/it]
 88%|████████▊ | 197864/225000 [376:44:25<64:27:04,  8.55s/it]
 88%|████████▊ | 197865/225000 [376:44:32<59:58:31,  7.96s/it]
 88%|████████▊ | 197866/225000 [376:44:42<63:31:42,  8.43s/it]
 88%|████████▊ | 197867/225000 [376:44:48<58:01:06,  7.70s/it]
 88%|████████▊ | 197868/225000 [376:44:56<59:34:57,  7.91s/it]
 88%|████████▊ | 197869/225000 [376:45:08<69:14:25,  9.19s/it]
 88%|████████▊ | 197870/225000 [376:45:16<66:27:54,  8.82s/it]
 88%|████████▊ | 197871/225000 [376:45:22<58:43:52,  7.79s/it]
 88%|████████▊ | 197872/225000 [376:45:27<53:07:59,  7.05s/it]
 88%|████████▊ | 197873/225000 [376:45:35<56:28:58,  7.50s/it]
 88%|████████▊ | 197874/225000 [376:45:46<63:54:13,  8.48s/it]
 88%|████████▊ | 197875/225000 [376:45:53<61:11:28,  8.12s/it]
 88%|████████▊ | 197876/225000 [376:46:01<60:59:58,  8.10s/it]
 88%|████████▊ | 197877/225000 [376:46:09<60:20:50,  8.01s/it]
 88%|████████▊ | 197878/225000 [376:46:17<60:15:29,  8.00s/it]
 88%|████████▊ | 197879/225000 [376:46:29<69:08:47,  9.18s/it]
 88%|████████▊ | 197880/225000 [376:46:35<62:16:44,  8.27s/it]
 88%|████████▊ | 197881/225000 [376:46:43<60:36:31,  8.05s/it]
 88%|████████▊ | 197882/225000 [376:46:48<54:26:36,  7.23s/it]
 88%|████████▊ | 197883/225000 [376:46:56<56:08:13,  7.45s/it]
 88%|████████▊ | 197884/225000 [376:47:08<66:27:43,  8.82s/it]
 88%|████████▊ | 197885/225000 [376:47:14<60:06:31,  7.98s/it]
 88%|████████▊ | 197886/225000 [376:47:26<69:28:49,  9.23s/it]
 88%|████████▊ | 197887/225000 [376:47:34<65:17:36,  8.67s/it]
 88%|████████▊ | 197888/225000 [376:47:42<63:44:22,  8.46s/it]
 88%|████████▊ | 197889/225000 [376:47:54<73:23:33,  9.75s/it]
 88%|████████▊ | 197890/225000 [376:48:02<69:14:47,  9.20s/it]
 88%|████████▊ | 197891/225000 [376:48:11<67:46:26,  9.00s/it]
 88%|████████▊ | 197892/225000 [376:48:27<84:40:40, 11.25s/it]
 88%|████████▊ | 197893/225000 [376:48:32<70:18:58,  9.34s/it]
 88%|████████▊ | 197894/225000 [376:48:40<65:55:06,  8.75s/it]
 88%|████████▊ | 197895/225000 [376:48:52<72:58:17,  9.69s/it]
 88%|████████▊ | 197896/225000 [376:48:58<66:51:19,  8.88s/it]
 88%|████████▊ | 197897/225000 [376:49:05<62:11:15,  8.26s/it]
 88%|████████▊ | 197898/225000 [376:49:20<77:25:13, 10.28s/it]
 88%|████████▊ | 197899/225000 [376:49:29<73:25:20,  9.75s/it]
 88%|████████▊ | 197900/225000 [376:49:35<64:17:59,  8.54s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.503', 'learning_rate': '1.948e-06', 'epoch': '0.8796', 'num_input_tokens_seen': 630507440, 'train_runtime': '1.357e+06', 'train_tokens_per_second': '464.8'}

 88%|████████▊ | 197900/225000 [376:49:35<64:17:59,  8.54s/it]
 88%|████████▊ | 197901/225000 [376:49:43<64:14:44,  8.53s/it]
 88%|████████▊ | 197902/225000 [376:49:55<72:41:40,  9.66s/it]
 88%|████████▊ | 197903/225000 [376:50:03<68:07:37,  9.05s/it]
 88%|████████▊ | 197904/225000 [376:50:14<73:29:10,  9.76s/it]
 88%|████████▊ | 197905/225000 [376:50:20<64:08:43,  8.52s/it]
 88%|████████▊ | 197906/225000 [376:50:31<69:53:01,  9.29s/it]
 88%|████████▊ | 197907/225000 [376:50:38<63:26:25,  8.43s/it]
 88%|████████▊ | 197908/225000 [376:50:44<59:52:20,  7.96s/it]
 88%|████████▊ | 197909/225000 [376:50:56<67:04:59,  8.91s/it]
 88%|████████▊ | 197910/225000 [376:51:07<72:57:31,  9.70s/it]
 88%|████████▊ | 197911/225000 [376:51:19<77:28:58, 10.30s/it]
 88%|████████▊ | 197912/225000 [376:51:30<80:12:47, 10.66s/it]
 88%|████████▊ | 197913/225000 [376:51:42<82:56:33, 11.02s/it]
 88%|████████▊ | 197914/225000 [376:51:50<75:32:30, 10.04s/it]
 88%|████████▊ | 197915/225000 [376:51:59<74:09:20,  9.86s/it]
 88%|████████▊ | 197916/225000 [376:52:05<65:53:27,  8.76s/it]
 88%|████████▊ | 197917/225000 [376:52:11<57:40:54,  7.67s/it]
 88%|████████▊ | 197918/225000 [376:52:26<74:41:23,  9.93s/it]
 88%|████████▊ | 197919/225000 [376:52:30<62:11:04,  8.27s/it]
 88%|████████▊ | 197920/225000 [376:52:39<62:53:06,  8.36s/it]
 88%|████████▊ | 197921/225000 [376:52:47<62:15:07,  8.28s/it]
 88%|████████▊ | 197922/225000 [376:52:58<69:07:43,  9.19s/it]
 88%|████████▊ | 197923/225000 [376:53:05<63:04:54,  8.39s/it]
 88%|████████▊ | 197924/225000 [376:53:12<60:35:23,  8.06s/it]
 88%|████████▊ | 197925/225000 [376:53:20<59:35:49,  7.92s/it]
 88%|████████▊ | 197926/225000 [376:53:28<59:44:51,  7.94s/it]
 88%|████████▊ | 197927/225000 [376:53:37<63:02:54,  8.38s/it]
 88%|████████▊ | 197928/225000 [376:53:52<78:32:08, 10.44s/it]
 88%|████████▊ | 197929/225000 [376:53:58<67:22:43,  8.96s/it]
 88%|████████▊ | 197930/225000 [376:54:08<69:45:41,  9.28s/it]
 88%|████████▊ | 197931/225000 [376:54:13<60:12:10,  8.01s/it]
 88%|████████▊ | 197932/225000 [376:54:22<63:30:47,  8.45s/it]
 88%|████████▊ | 197933/225000 [376:54:28<56:30:01,  7.51s/it]
 88%|████████▊ | 197934/225000 [376:54:37<59:35:14,  7.93s/it]
 88%|████████▊ | 197935/225000 [376:54:52<77:33:45, 10.32s/it]
 88%|████████▊ | 197936/225000 [376:54:57<64:28:19,  8.58s/it]
 88%|████████▊ | 197937/225000 [376:55:10<74:24:20,  9.90s/it]
 88%|████████▊ | 197938/225000 [376:55:21<77:50:23, 10.35s/it]
 88%|████████▊ | 197939/225000 [376:55:26<64:35:47,  8.59s/it]
 88%|████████▊ | 197940/225000 [376:55:32<60:03:21,  7.99s/it]
 88%|████████▊ | 197941/225000 [376:55:40<59:47:49,  7.96s/it]
 88%|████████▊ | 197942/225000 [376:55:47<57:27:57,  7.65s/it]
 88%|████████▊ | 197943/225000 [376:55:56<59:04:10,  7.86s/it]
 88%|████████▊ | 197944/225000 [376:56:05<62:04:57,  8.26s/it]
 88%|████████▊ | 197945/225000 [376:56:12<59:28:22,  7.91s/it]
 88%|████████▊ | 197946/225000 [376:56:21<61:43:40,  8.21s/it]
 88%|████████▊ | 197947/225000 [376:56:28<60:14:35,  8.02s/it]
 88%|████████▊ | 197948/225000 [376:56:40<68:17:36,  9.09s/it]
 88%|████████▊ | 197949/225000 [376:56:46<62:01:36,  8.25s/it]
 88%|████████▊ | 197950/225000 [376:56:54<61:42:31,  8.21s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.225', 'learning_rate': '1.941e-06', 'epoch': '0.8798', 'num_input_tokens_seen': 630740424, 'train_runtime': '1.357e+06', 'train_tokens_per_second': '464.8'}

 88%|████████▊ | 197950/225000 [376:56:54<61:42:31,  8.21s/it]
 88%|████████▊ | 197951/225000 [376:57:00<57:04:44,  7.60s/it]
 88%|████████▊ | 197952/225000 [376:57:07<54:12:34,  7.22s/it]
 88%|████████▊ | 197953/225000 [376:57:20<66:53:17,  8.90s/it]
 88%|████████▊ | 197954/225000 [376:57:27<62:41:28,  8.34s/it]
 88%|████████▊ | 197955/225000 [376:57:33<57:17:10,  7.63s/it]
 88%|████████▊ | 197956/225000 [376:57:40<57:34:14,  7.66s/it]
 88%|████████▊ | 197957/225000 [376:57:50<62:55:30,  8.38s/it]
 88%|████████▊ | 197958/225000 [376:58:03<72:39:38,  9.67s/it]
 88%|████████▊ | 197959/225000 [376:58:09<63:01:32,  8.39s/it]
 88%|████████▊ | 197960/225000 [376:58:19<66:41:12,  8.88s/it]
 88%|████████▊ | 197961/225000 [376:58:24<58:31:28,  7.79s/it]
 88%|████████▊ | 197962/225000 [376:58:32<59:46:31,  7.96s/it]
 88%|████████▊ | 197963/225000 [376:58:42<63:15:38,  8.42s/it]
 88%|████████▊ | 197964/225000 [376:58:51<64:21:34,  8.57s/it]
 88%|████████▊ | 197965/225000 [376:59:00<67:02:25,  8.93s/it]
 88%|████████▊ | 197966/225000 [376:59:07<61:11:58,  8.15s/it]
 88%|████████▊ | 197967/225000 [376:59:18<67:21:29,  8.97s/it]
 88%|████████▊ | 197968/225000 [376:59:26<66:39:51,  8.88s/it]
 88%|████████▊ | 197969/225000 [376:59:33<62:45:38,  8.36s/it]
 88%|████████▊ | 197970/225000 [376:59:41<61:39:45,  8.21s/it]
 88%|████████▊ | 197971/225000 [376:59:48<57:22:27,  7.64s/it]
 88%|████████▊ | 197972/225000 [376:59:57<60:56:50,  8.12s/it]
 88%|████████▊ | 197973/225000 [377:00:03<56:20:58,  7.51s/it]
 88%|████████▊ | 197974/225000 [377:00:15<66:28:17,  8.85s/it]
 88%|████████▊ | 197975/225000 [377:00:23<65:26:24,  8.72s/it]
 88%|████████▊ | 197976/225000 [377:00:32<65:44:23,  8.76s/it]
 88%|████████▊ | 197977/225000 [377:00:39<61:14:37,  8.16s/it]
 88%|████████▊ | 197978/225000 [377:00:49<65:14:42,  8.69s/it]
 88%|████████▊ | 197979/225000 [377:01:02<75:55:21, 10.12s/it]
 88%|████████▊ | 197980/225000 [377:01:12<75:22:00, 10.04s/it]
 88%|████████▊ | 197981/225000 [377:01:20<69:27:39,  9.25s/it]
 88%|████████▊ | 197982/225000 [377:01:25<60:54:48,  8.12s/it]
 88%|████████▊ | 197983/225000 [377:01:29<52:18:52,  6.97s/it]
 88%|████████▊ | 197984/225000 [377:01:34<47:31:31,  6.33s/it]
 88%|████████▊ | 197985/225000 [377:01:50<68:56:57,  9.19s/it]
 88%|████████▊ | 197986/225000 [377:01:57<63:34:14,  8.47s/it]
 88%|████████▊ | 197987/225000 [377:02:06<65:43:57,  8.76s/it]
 88%|████████▊ | 197988/225000 [377:02:16<68:00:12,  9.06s/it]
 88%|████████▊ | 197989/225000 [377:02:22<61:33:04,  8.20s/it]
 88%|████████▊ | 197990/225000 [377:02:32<64:59:38,  8.66s/it]
 88%|████████▊ | 197991/225000 [377:02:38<60:14:42,  8.03s/it]
 88%|████████▊ | 197992/225000 [377:02:46<58:19:08,  7.77s/it]
 88%|████████▊ | 197993/225000 [377:02:59<70:19:40,  9.37s/it]
 88%|████████▊ | 197994/225000 [377:03:03<58:22:12,  7.78s/it]
 88%|████████▊ | 197995/225000 [377:03:13<62:42:11,  8.36s/it]
 88%|████████▊ | 197996/225000 [377:03:23<68:35:24,  9.14s/it]
 88%|████████▊ | 197997/225000 [377:03:35<73:35:32,  9.81s/it]
 88%|████████▊ | 197998/225000 [377:03:39<61:11:12,  8.16s/it]
 88%|████████▊ | 197999/225000 [377:03:47<59:37:41,  7.95s/it]
 88%|████████▊ | 198000/225000 [377:03:59<69:13:07,  9.23s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.263', 'learning_rate': '1.933e-06', 'epoch': '0.88', 'num_input_tokens_seen': 630969656, 'train_runtime': '1.357e+06', 'train_tokens_per_second': '464.8'}

 88%|████████▊ | 198000/225000 [377:03:59<69:13:07,  9.23s/it]
 88%|████████▊ | 198001/225000 [377:04:04<60:31:19,  8.07s/it]
 88%|████████▊ | 198002/225000 [377:04:14<64:15:20,  8.57s/it]
 88%|████████▊ | 198003/225000 [377:04:22<64:12:25,  8.56s/it]
 88%|████████▊ | 198004/225000 [377:04:35<72:06:55,  9.62s/it]
 88%|████████▊ | 198005/225000 [377:04:42<66:49:49,  8.91s/it]
 88%|████████▊ | 198006/225000 [377:04:54<73:26:25,  9.79s/it]
 88%|████████▊ | 198007/225000 [377:05:02<69:46:47,  9.31s/it]
 88%|████████▊ | 198008/225000 [377:05:08<61:46:56,  8.24s/it]
 88%|████████▊ | 198009/225000 [377:05:15<60:02:14,  8.01s/it]
 88%|████████▊ | 198010/225000 [377:05:22<57:02:49,  7.61s/it]
 88%|████████▊ | 198011/225000 [377:05:36<72:57:54,  9.73s/it]
 88%|████████▊ | 198012/225000 [377:05:42<62:31:23,  8.34s/it]
 88%|████████▊ | 198013/225000 [377:05:52<67:32:05,  9.01s/it]
 88%|████████▊ | 198014/225000 [377:06:01<67:26:59,  9.00s/it]
 88%|████████▊ | 198015/225000 [377:06:10<66:53:42,  8.92s/it]
 88%|████████▊ | 198016/225000 [377:06:17<63:15:23,  8.44s/it]
 88%|████████▊ | 198017/225000 [377:06:24<60:26:30,  8.06s/it]
 88%|████████▊ | 198018/225000 [377:06:38<72:22:06,  9.66s/it]
 88%|████████▊ | 198019/225000 [377:06:46<68:57:04,  9.20s/it]
 88%|████████▊ | 198020/225000 [377:06:55<69:23:43,  9.26s/it]
 88%|████████▊ | 198021/225000 [377:07:01<61:27:11,  8.20s/it]
 88%|████████▊ | 198022/225000 [377:07:08<58:13:40,  7.77s/it]
 88%|████████▊ | 198023/225000 [377:07:16<59:31:58,  7.94s/it]
 88%|████████▊ | 198024/225000 [377:07:31<74:13:03,  9.90s/it]
 88%|████████▊ | 198025/225000 [377:07:37<66:17:17,  8.85s/it]
 88%|████████▊ | 198026/225000 [377:07:43<59:49:52,  7.99s/it]
 88%|████████▊ | 198027/225000 [377:07:54<66:17:52,  8.85s/it]
 88%|████████▊ | 198028/225000 [377:07:58<57:00:21,  7.61s/it]
 88%|████████▊ | 198029/225000 [377:08:10<65:14:11,  8.71s/it]
 88%|████████▊ | 198030/225000 [377:08:18<64:53:00,  8.66s/it]
 88%|████████▊ | 198031/225000 [377:08:25<60:29:49,  8.08s/it]
 88%|████████▊ | 198032/225000 [377:08:31<55:35:03,  7.42s/it]
 88%|████████▊ | 198033/225000 [377:08:39<57:13:21,  7.64s/it]
 88%|████████▊ | 198034/225000 [377:08:46<55:46:56,  7.45s/it]
 88%|████████▊ | 198035/225000 [377:09:03<76:55:29, 10.27s/it]
 88%|████████▊ | 198036/225000 [377:09:07<63:03:07,  8.42s/it]
 88%|████████▊ | 198037/225000 [377:09:19<71:45:13,  9.58s/it]
 88%|████████▊ | 198038/225000 [377:09:27<68:36:06,  9.16s/it]
 88%|████████▊ | 198039/225000 [377:09:37<68:35:48,  9.16s/it]
 88%|████████▊ | 198040/225000 [377:09:44<64:37:18,  8.63s/it]
 88%|████████▊ | 198041/225000 [377:09:51<61:04:02,  8.15s/it]
 88%|████████▊ | 198042/225000 [377:10:01<65:22:14,  8.73s/it]
 88%|████████▊ | 198043/225000 [377:10:08<60:23:33,  8.07s/it]
 88%|████████▊ | 198044/225000 [377:10:13<55:16:52,  7.38s/it]
 88%|████████▊ | 198045/225000 [377:10:28<70:29:37,  9.41s/it]
 88%|████████▊ | 198046/225000 [377:10:39<74:19:47,  9.93s/it]
 88%|████████▊ | 198047/225000 [377:10:51<79:53:46, 10.67s/it]
 88%|████████▊ | 198048/225000 [377:10:58<70:50:59,  9.46s/it]
 88%|████████▊ | 198049/225000 [377:11:06<67:50:07,  9.06s/it]
 88%|████████▊ | 198050/225000 [377:11:10<57:20:19,  7.66s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.419', 'learning_rate': '1.926e-06', 'epoch': '0.8802', 'num_input_tokens_seen': 631201032, 'train_runtime': '1.358e+06', 'train_tokens_per_second': '464.8'}

 88%|████████▊ | 198050/225000 [377:11:10<57:20:19,  7.66s/it]
 88%|████████▊ | 198051/225000 [377:11:19<58:58:43,  7.88s/it]
 88%|████████▊ | 198052/225000 [377:11:25<55:21:40,  7.40s/it]
 88%|████████▊ | 198053/225000 [377:11:32<54:03:39,  7.22s/it]
 88%|████████▊ | 198054/225000 [377:11:42<61:16:01,  8.19s/it]
 88%|████████▊ | 198055/225000 [377:11:55<72:24:06,  9.67s/it]
 88%|████████▊ | 198056/225000 [377:12:03<68:40:14,  9.18s/it]
 88%|████████▊ | 198057/225000 [377:12:15<74:59:39, 10.02s/it]
 88%|████████▊ | 198058/225000 [377:12:27<77:52:20, 10.41s/it]
 88%|████████▊ | 198059/225000 [377:12:33<68:55:14,  9.21s/it]
 88%|████████▊ | 198060/225000 [377:12:41<66:45:25,  8.92s/it]
 88%|████████▊ | 198061/225000 [377:12:48<60:37:50,  8.10s/it]
 88%|████████▊ | 198062/225000 [377:12:55<59:01:29,  7.89s/it]
 88%|████████▊ | 198063/225000 [377:13:02<56:57:04,  7.61s/it]
 88%|████████▊ | 198064/225000 [377:13:16<71:30:12,  9.56s/it]
 88%|████████▊ | 198065/225000 [377:13:21<60:18:47,  8.06s/it]
 88%|████████▊ | 198066/225000 [377:13:29<62:03:17,  8.29s/it]
 88%|████████▊ | 198067/225000 [377:13:36<57:23:02,  7.67s/it]
 88%|████████▊ | 198068/225000 [377:13:46<63:37:06,  8.50s/it]
 88%|████████▊ | 198069/225000 [377:13:53<59:53:57,  8.01s/it]
 88%|████████▊ | 198070/225000 [377:13:59<55:24:42,  7.41s/it]
 88%|████████▊ | 198071/225000 [377:14:04<50:39:32,  6.77s/it]
 88%|████████▊ | 198072/225000 [377:14:15<59:57:45,  8.02s/it]
 88%|████████▊ | 198073/225000 [377:14:26<66:09:56,  8.85s/it]
 88%|████████▊ | 198074/225000 [377:14:35<67:30:52,  9.03s/it]
 88%|████████▊ | 198075/225000 [377:14:43<64:37:23,  8.64s/it]
 88%|████████▊ | 198076/225000 [377:14:56<74:09:33,  9.92s/it]
 88%|████████▊ | 198077/225000 [377:15:01<62:12:54,  8.32s/it]
 88%|████████▊ | 198078/225000 [377:15:09<62:35:40,  8.37s/it]
 88%|████████▊ | 198079/225000 [377:15:16<59:42:00,  7.98s/it]
 88%|████████▊ | 198080/225000 [377:15:25<60:52:28,  8.14s/it]
 88%|████████▊ | 198081/225000 [377:15:31<57:19:22,  7.67s/it]
 88%|████████▊ | 198082/225000 [377:15:37<52:36:03,  7.03s/it]
 88%|████████▊ | 198083/225000 [377:15:45<54:43:08,  7.32s/it]
 88%|████████▊ | 198084/225000 [377:15:55<61:29:34,  8.22s/it]
 88%|████████▊ | 198085/225000 [377:16:03<60:44:26,  8.12s/it]
 88%|████████▊ | 198086/225000 [377:16:10<57:34:48,  7.70s/it]
 88%|████████▊ | 198087/225000 [377:16:20<62:20:04,  8.34s/it]
 88%|████████▊ | 198088/225000 [377:16:25<56:07:34,  7.51s/it]
 88%|████████▊ | 198089/225000 [377:16:37<65:52:51,  8.81s/it]
 88%|████████▊ | 198090/225000 [377:16:43<58:40:46,  7.85s/it]
 88%|████████▊ | 198091/225000 [377:16:53<64:21:20,  8.61s/it]
 88%|████████▊ | 198092/225000 [377:17:03<68:02:44,  9.10s/it]
 88%|████████▊ | 198093/225000 [377:17:11<64:28:14,  8.63s/it]
 88%|████████▊ | 198094/225000 [377:17:21<69:18:12,  9.27s/it]
 88%|████████▊ | 198095/225000 [377:17:28<63:25:08,  8.49s/it]
 88%|████████▊ | 198096/225000 [377:17:38<66:49:51,  8.94s/it]
 88%|████████▊ | 198097/225000 [377:17:48<69:22:02,  9.28s/it]
 88%|████████▊ | 198098/225000 [377:17:55<62:43:33,  8.39s/it]
 88%|████████▊ | 198099/225000 [377:18:04<65:02:47,  8.70s/it]
 88%|████████▊ | 198100/225000 [377:18:14<67:56:10,  9.09s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.292', 'learning_rate': '1.919e-06', 'epoch': '0.8804', 'num_input_tokens_seen': 631429728, 'train_runtime': '1.358e+06', 'train_tokens_per_second': '464.9'}

 88%|████████▊ | 198100/225000 [377:18:14<67:56:10,  9.09s/it]
 88%|████████▊ | 198101/225000 [377:18:22<66:00:03,  8.83s/it]
 88%|████████▊ | 198102/225000 [377:18:32<67:35:22,  9.05s/it]
 88%|████████▊ | 198103/225000 [377:18:37<58:42:01,  7.86s/it]
 88%|████████▊ | 198104/225000 [377:18:46<61:19:07,  8.21s/it]
 88%|████████▊ | 198105/225000 [377:18:56<66:13:48,  8.87s/it]
 88%|████████▊ | 198106/225000 [377:19:02<59:16:58,  7.94s/it]
 88%|████████▊ | 198107/225000 [377:19:09<56:58:14,  7.63s/it]
 88%|████████▊ | 198108/225000 [377:19:20<64:17:10,  8.61s/it]
 88%|████████▊ | 198109/225000 [377:19:24<55:17:17,  7.40s/it]
 88%|████████▊ | 198110/225000 [377:19:32<56:47:39,  7.60s/it]
 88%|████████▊ | 198111/225000 [377:19:41<59:24:34,  7.95s/it]
 88%|████████▊ | 198112/225000 [377:19:50<62:21:01,  8.35s/it]
 88%|████████▊ | 198113/225000 [377:19:59<63:25:02,  8.49s/it]
 88%|████████▊ | 198114/225000 [377:20:04<55:42:21,  7.46s/it]
 88%|████████▊ | 198115/225000 [377:20:11<52:54:43,  7.09s/it]
 88%|████████▊ | 198116/225000 [377:20:21<59:14:38,  7.93s/it]
 88%|████████▊ | 198117/225000 [377:20:30<61:51:05,  8.28s/it]
 88%|████████▊ | 198118/225000 [377:20:37<60:36:47,  8.12s/it]
 88%|████████▊ | 198119/225000 [377:20:46<61:43:49,  8.27s/it]
 88%|████████▊ | 198120/225000 [377:20:52<56:43:47,  7.60s/it]
 88%|████████▊ | 198121/225000 [377:21:01<59:59:20,  8.03s/it]
 88%|████████▊ | 198122/225000 [377:21:08<58:11:47,  7.79s/it]
 88%|████████▊ | 198123/225000 [377:21:14<52:53:16,  7.08s/it]
 88%|████████▊ | 198124/225000 [377:21:31<75:02:48, 10.05s/it]
 88%|████████▊ | 198125/225000 [377:21:36<65:30:00,  8.77s/it]
 88%|████████▊ | 198126/225000 [377:21:42<58:44:54,  7.87s/it]
 88%|████████▊ | 198127/225000 [377:21:53<66:16:59,  8.88s/it]
 88%|████████▊ | 198128/225000 [377:22:03<67:55:32,  9.10s/it]
 88%|████████▊ | 198129/225000 [377:22:11<64:58:11,  8.70s/it]
 88%|████████▊ | 198130/225000 [377:22:17<59:37:28,  7.99s/it]
 88%|████████▊ | 198131/225000 [377:22:22<52:42:00,  7.06s/it]
 88%|████████▊ | 198132/225000 [377:22:34<63:55:30,  8.57s/it]
 88%|████████▊ | 198133/225000 [377:22:41<60:41:37,  8.13s/it]
 88%|████████▊ | 198134/225000 [377:22:53<69:45:14,  9.35s/it]
 88%|████████▊ | 198135/225000 [377:23:01<64:39:15,  8.66s/it]
 88%|████████▊ | 198136/225000 [377:23:09<64:06:38,  8.59s/it]
 88%|████████▊ | 198137/225000 [377:23:16<61:19:04,  8.22s/it]
 88%|████████▊ | 198138/225000 [377:23:21<54:07:14,  7.25s/it]
 88%|████████▊ | 198139/225000 [377:23:35<68:39:53,  9.20s/it]
 88%|████████▊ | 198140/225000 [377:23:41<62:26:19,  8.37s/it]
 88%|████████▊ | 198141/225000 [377:23:52<66:11:59,  8.87s/it]
 88%|████████▊ | 198142/225000 [377:24:00<64:25:17,  8.63s/it]
 88%|████████▊ | 198143/225000 [377:24:07<60:57:36,  8.17s/it]
 88%|████████▊ | 198144/225000 [377:24:18<68:20:30,  9.16s/it]
 88%|████████▊ | 198145/225000 [377:24:23<58:48:46,  7.88s/it]
 88%|████████▊ | 198146/225000 [377:24:35<68:20:59,  9.16s/it]
 88%|████████▊ | 198147/225000 [377:24:45<69:13:24,  9.28s/it]
 88%|████████▊ | 198148/225000 [377:24:53<66:15:00,  8.88s/it]
 88%|████████▊ | 198149/225000 [377:24:58<58:55:05,  7.90s/it]
 88%|████████▊ | 198150/225000 [377:25:09<64:56:15,  8.71s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.242', 'learning_rate': '1.912e-06', 'epoch': '0.8807', 'num_input_tokens_seen': 631657880, 'train_runtime': '1.359e+06', 'train_tokens_per_second': '464.9'}

 88%|████████▊ | 198150/225000 [377:25:09<64:56:15,  8.71s/it]
 88%|████████▊ | 198151/225000 [377:25:18<66:32:51,  8.92s/it]
 88%|████████▊ | 198152/225000 [377:25:23<56:48:40,  7.62s/it]
 88%|████████▊ | 198153/225000 [377:25:31<57:22:13,  7.69s/it]
 88%|████████▊ | 198154/225000 [377:25:39<57:26:47,  7.70s/it]
 88%|████████▊ | 198155/225000 [377:25:43<50:59:24,  6.84s/it]
 88%|████████▊ | 198156/225000 [377:25:51<53:52:34,  7.23s/it]
 88%|████████▊ | 198157/225000 [377:26:09<76:57:45, 10.32s/it]
 88%|████████▊ | 198158/225000 [377:26:17<72:44:05,  9.76s/it]
 88%|████████▊ | 198159/225000 [377:26:23<62:35:24,  8.39s/it]
 88%|████████▊ | 198160/225000 [377:26:29<57:58:32,  7.78s/it]
 88%|████████▊ | 198161/225000 [377:26:39<62:13:11,  8.35s/it]
 88%|████████▊ | 198162/225000 [377:26:49<66:03:54,  8.86s/it]
 88%|████████▊ | 198163/225000 [377:26:57<64:47:38,  8.69s/it]
 88%|████████▊ | 198164/225000 [377:27:05<62:20:09,  8.36s/it]
 88%|████████▊ | 198165/225000 [377:27:17<70:52:35,  9.51s/it]
 88%|████████▊ | 198166/225000 [377:27:22<61:16:17,  8.22s/it]
 88%|████████▊ | 198167/225000 [377:27:31<63:58:02,  8.58s/it]
 88%|████████▊ | 198168/225000 [377:27:37<56:57:20,  7.64s/it]
 88%|████████▊ | 198169/225000 [377:27:47<62:13:27,  8.35s/it]
 88%|████████▊ | 198170/225000 [377:27:56<64:08:30,  8.61s/it]
 88%|████████▊ | 198171/225000 [377:28:04<63:34:36,  8.53s/it]
 88%|████████▊ | 198172/225000 [377:28:10<57:06:33,  7.66s/it]
 88%|████████▊ | 198173/225000 [377:28:17<54:47:24,  7.35s/it]
 88%|████████▊ | 198174/225000 [377:28:26<58:47:32,  7.89s/it]
 88%|████████▊ | 198175/225000 [377:28:36<64:26:59,  8.65s/it]
 88%|████████▊ | 198176/225000 [377:28:44<61:24:07,  8.24s/it]
 88%|████████▊ | 198177/225000 [377:28:50<57:01:51,  7.65s/it]
 88%|████████▊ | 198178/225000 [377:29:02<67:56:37,  9.12s/it]
 88%|████████▊ | 198179/225000 [377:29:09<62:05:55,  8.34s/it]
 88%|████████▊ | 198180/225000 [377:29:15<57:52:20,  7.77s/it]
 88%|████████▊ | 198181/225000 [377:29:24<59:50:17,  8.03s/it]
 88%|████████▊ | 198182/225000 [377:29:37<70:10:23,  9.42s/it]
 88%|████████▊ | 198183/225000 [377:29:41<59:20:03,  7.97s/it]
 88%|████████▊ | 198184/225000 [377:29:50<60:57:36,  8.18s/it]
 88%|████████▊ | 198185/225000 [377:29:57<58:33:18,  7.86s/it]
 88%|████████▊ | 198186/225000 [377:30:09<68:50:15,  9.24s/it]
 88%|████████▊ | 198187/225000 [377:30:17<65:57:21,  8.86s/it]
 88%|████████▊ | 198188/225000 [377:30:23<58:27:07,  7.85s/it]
 88%|████████▊ | 198189/225000 [377:30:30<57:36:28,  7.74s/it]
 88%|████████▊ | 198190/225000 [377:30:38<56:26:13,  7.58s/it]
 88%|████████▊ | 198191/225000 [377:30:52<72:28:57,  9.73s/it]
 88%|████████▊ | 198192/225000 [377:31:02<71:35:59,  9.62s/it]
 88%|████████▊ | 198193/225000 [377:31:06<59:37:48,  8.01s/it]
 88%|████████▊ | 198194/225000 [377:31:16<63:32:52,  8.53s/it]
 88%|████████▊ | 198195/225000 [377:31:22<58:37:55,  7.87s/it]
 88%|████████▊ | 198196/225000 [377:31:33<66:10:57,  8.89s/it]
 88%|████████▊ | 198197/225000 [377:31:42<65:48:01,  8.84s/it]
 88%|████████▊ | 198198/225000 [377:31:48<58:26:27,  7.85s/it]
 88%|████████▊ | 198199/225000 [377:32:00<69:01:40,  9.27s/it]
 88%|████████▊ | 198200/225000 [377:32:10<69:33:35,  9.34s/it]
                                                              
{'loss': '1.336', 'grad_norm': '1.52', 'learning_rate': '1.905e-06', 'epoch': '0.8809', 'num_input_tokens_seen': 631881016, 'train_runtime': '1.359e+06', 'train_tokens_per_second': '464.9'}

 88%|████████▊ | 198200/225000 [377:32:10<69:33:35,  9.34s/it]
 88%|████████▊ | 198201/225000 [377:32:15<60:29:42,  8.13s/it]
 88%|████████▊ | 198202/225000 [377:32:24<61:49:04,  8.30s/it]
 88%|████████▊ | 198203/225000 [377:32:34<66:34:15,  8.94s/it]
 88%|████████▊ | 198204/225000 [377:32:46<74:00:15,  9.94s/it]
 88%|████████▊ | 198205/225000 [377:32:52<64:10:22,  8.62s/it]
 88%|████████▊ | 198206/225000 [377:33:03<69:09:00,  9.29s/it]
 88%|████████▊ | 198207/225000 [377:33:12<68:48:39,  9.25s/it]
 88%|████████▊ | 198208/225000 [377:33:21<68:49:35,  9.25s/it]
 88%|████████▊ | 198209/225000 [377:33:31<70:27:24,  9.47s/it]
 88%|████████▊ | 198210/225000 [377:33:38<64:06:40,  8.62s/it]
 88%|████████▊ | 198211/225000 [377:33:47<66:05:28,  8.88s/it]
 88%|████████▊ | 198212/225000 [377:33:57<67:48:14,  9.11s/it]
 88%|████████▊ | 198213/225000 [377:34:05<65:27:11,  8.80s/it]
 88%|████████▊ | 198214/225000 [377:34:11<59:34:12,  8.01s/it]
 88%|████████▊ | 198215/225000 [377:34:17<55:12:17,  7.42s/it]
 88%|████████▊ | 198216/225000 [377:34:22<49:53:51,  6.71s/it]
 88%|████████▊ | 198217/225000 [377:34:33<58:24:57,  7.85s/it]
 88%|████████▊ | 198218/225000 [377:34:43<64:07:02,  8.62s/it]
 88%|████████▊ | 198219/225000 [377:34:48<56:01:58,  7.53s/it]
 88%|████████▊ | 198220/225000 [377:34:53<50:51:34,  6.84s/it]
 88%|████████▊ | 198221/225000 [377:35:09<71:26:48,  9.60s/it]
 88%|████████▊ | 198222/225000 [377:35:22<77:29:38, 10.42s/it]
 88%|████████▊ | 198223/225000 [377:35:29<69:16:50,  9.31s/it]
 88%|████████▊ | 198224/225000 [377:35:37<67:07:21,  9.02s/it]
 88%|████████▊ | 198225/225000 [377:35:43<60:39:47,  8.16s/it]
 88%|████████▊ | 198226/225000 [377:35:50<58:40:54,  7.89s/it]
 88%|████████▊ | 198227/225000 [377:35:57<55:56:56,  7.52s/it]
 88%|████████▊ | 198228/225000 [377:36:13<73:56:40,  9.94s/it]
 88%|████████▊ | 198229/225000 [377:36:17<61:39:10,  8.29s/it]
 88%|████████▊ | 198230/225000 [377:36:27<65:52:25,  8.86s/it]
 88%|████████▊ | 198231/225000 [377:36:33<59:36:17,  8.02s/it]
 88%|████████▊ | 198232/225000 [377:36:41<58:39:24,  7.89s/it]
 88%|████████▊ | 198233/225000 [377:36:51<64:17:08,  8.65s/it]
 88%|████████▊ | 198234/225000 [377:36:57<58:00:00,  7.80s/it]
 88%|████████▊ | 198235/225000 [377:37:09<67:31:05,  9.08s/it]
 88%|████████▊ | 198236/225000 [377:37:17<63:44:56,  8.57s/it]
 88%|████████▊ | 198237/225000 [377:37:22<56:39:30,  7.62s/it]
 88%|████████▊ | 198238/225000 [377:37:28<53:36:35,  7.21s/it]
 88%|████████▊ | 198239/225000 [377:37:39<60:42:38,  8.17s/it]
 88%|████████▊ | 198240/225000 [377:37:48<64:05:49,  8.62s/it]
 88%|████████▊ | 198241/225000 [377:37:57<64:06:33,  8.62s/it]
 88%|████████▊ | 198242/225000 [377:38:08<70:45:08,  9.52s/it]
 88%|████████▊ | 198243/225000 [377:38:17<68:42:34,  9.24s/it]
 88%|████████▊ | 198244/225000 [377:38:28<72:19:07,  9.73s/it]
 88%|████████▊ | 198245/225000 [377:38:35<66:26:51,  8.94s/it]
 88%|████████▊ | 198246/225000 [377:38:42<62:50:25,  8.46s/it]
 88%|████████▊ | 198247/225000 [377:38:49<58:38:43,  7.89s/it]
 88%|████████▊ | 198248/225000 [377:39:02<69:12:35,  9.31s/it]
 88%|████████▊ | 198249/225000 [377:39:12<71:31:41,  9.63s/it]
 88%|████████▊ | 198250/225000 [377:39:18<63:31:38,  8.55s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.191', 'learning_rate': '1.898e-06', 'epoch': '0.8811', 'num_input_tokens_seen': 632110688, 'train_runtime': '1.36e+06', 'train_tokens_per_second': '464.9'}

 88%|████████▊ | 198250/225000 [377:39:18<63:31:38,  8.55s/it]
 88%|████████▊ | 198251/225000 [377:39:24<57:50:01,  7.78s/it]
 88%|████████▊ | 198252/225000 [377:39:36<67:12:13,  9.04s/it]
 88%|████████▊ | 198253/225000 [377:39:41<58:34:43,  7.88s/it]
 88%|████████▊ | 198254/225000 [377:39:46<52:56:09,  7.13s/it]
 88%|████████▊ | 198255/225000 [377:39:52<50:08:42,  6.75s/it]
 88%|████████▊ | 198256/225000 [377:40:07<66:50:57,  9.00s/it]
 88%|████████▊ | 198257/225000 [377:40:13<60:52:07,  8.19s/it]
 88%|████████▊ | 198258/225000 [377:40:24<66:30:54,  8.95s/it]
 88%|████████▊ | 198259/225000 [377:40:30<61:25:24,  8.27s/it]
 88%|████████▊ | 198260/225000 [377:40:37<58:20:35,  7.85s/it]
 88%|████████▊ | 198261/225000 [377:40:53<75:05:42, 10.11s/it]
 88%|████████▊ | 198262/225000 [377:40:58<64:41:14,  8.71s/it]
 88%|████████▊ | 198263/225000 [377:41:03<57:09:33,  7.70s/it]
 88%|████████▊ | 198264/225000 [377:41:09<53:06:37,  7.15s/it]
 88%|████████▊ | 198265/225000 [377:41:16<51:49:18,  6.98s/it]
 88%|████████▊ | 198266/225000 [377:41:29<66:18:25,  8.93s/it]
 88%|████████▊ | 198267/225000 [377:41:36<61:21:00,  8.26s/it]
 88%|████████▊ | 198268/225000 [377:41:51<76:06:55, 10.25s/it]
 88%|████████▊ | 198269/225000 [377:41:57<67:35:12,  9.10s/it]
 88%|████████▊ | 198270/225000 [377:42:05<65:10:52,  8.78s/it]
 88%|████████▊ | 198271/225000 [377:42:14<63:56:33,  8.61s/it]
 88%|████████▊ | 198272/225000 [377:42:18<55:36:38,  7.49s/it]
 88%|████████▊ | 198273/225000 [377:42:32<68:35:20,  9.24s/it]
 88%|████████▊ | 198274/225000 [377:42:37<60:44:13,  8.18s/it]
 88%|████████▊ | 198275/225000 [377:42:46<61:58:43,  8.35s/it]
 88%|████████▊ | 198276/225000 [377:42:56<65:28:40,  8.82s/it]
 88%|████████▊ | 198277/225000 [377:43:02<58:31:06,  7.88s/it]
 88%|████████▊ | 198278/225000 [377:43:11<60:34:37,  8.16s/it]
 88%|████████▊ | 198279/225000 [377:43:20<63:23:56,  8.54s/it]
 88%|████████▊ | 198280/225000 [377:43:29<65:00:53,  8.76s/it]
 88%|████████▊ | 198281/225000 [377:43:38<63:45:56,  8.59s/it]
 88%|████████▊ | 198282/225000 [377:43:45<61:44:24,  8.32s/it]
 88%|████████▊ | 198283/225000 [377:43:57<70:19:46,  9.48s/it]
 88%|████████▊ | 198284/225000 [377:44:02<60:06:07,  8.10s/it]
 88%|████████▊ | 198285/225000 [377:44:10<58:39:23,  7.90s/it]
 88%|████████▊ | 198286/225000 [377:44:17<57:40:23,  7.77s/it]
 88%|████████▊ | 198287/225000 [377:44:27<63:01:06,  8.49s/it]
 88%|████████▊ | 198288/225000 [377:44:34<58:34:49,  7.89s/it]
 88%|████████▊ | 198289/225000 [377:44:40<55:39:48,  7.50s/it]
 88%|████████▊ | 198290/225000 [377:44:48<55:11:59,  7.44s/it]
 88%|████████▊ | 198291/225000 [377:44:57<59:44:08,  8.05s/it]
 88%|████████▊ | 198292/225000 [377:45:03<53:43:01,  7.24s/it]
 88%|████████▊ | 198293/225000 [377:45:09<52:25:43,  7.07s/it]
 88%|████████▊ | 198294/225000 [377:45:19<59:19:54,  8.00s/it]
 88%|████████▊ | 198295/225000 [377:45:26<55:25:02,  7.47s/it]
 88%|████████▊ | 198296/225000 [377:45:32<53:28:08,  7.21s/it]
 88%|████████▊ | 198297/225000 [377:45:42<58:40:22,  7.91s/it]
 88%|████████▊ | 198298/225000 [377:45:50<60:28:16,  8.15s/it]
 88%|████████▊ | 198299/225000 [377:46:04<72:12:07,  9.73s/it]
 88%|████████▊ | 198300/225000 [377:46:13<71:31:18,  9.64s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.242', 'learning_rate': '1.892e-06', 'epoch': '0.8813', 'num_input_tokens_seen': 632338336, 'train_runtime': '1.36e+06', 'train_tokens_per_second': '465'}

 88%|████████▊ | 198300/225000 [377:46:13<71:31:18,  9.64s/it]
 88%|████████▊ | 198301/225000 [377:46:19<62:07:59,  8.38s/it]
 88%|████████▊ | 198302/225000 [377:46:32<71:59:15,  9.71s/it]
 88%|████████▊ | 198303/225000 [377:46:37<62:39:00,  8.45s/it]
 88%|████████▊ | 198304/225000 [377:46:42<55:45:07,  7.52s/it]
 88%|████████▊ | 198305/225000 [377:46:47<49:57:43,  6.74s/it]
 88%|████████▊ | 198306/225000 [377:46:56<54:26:36,  7.34s/it]
 88%|████████▊ | 198307/225000 [377:47:04<55:10:31,  7.44s/it]
 88%|████████▊ | 198308/225000 [377:47:19<71:30:18,  9.64s/it]
 88%|████████▊ | 198309/225000 [377:47:24<61:52:13,  8.34s/it]
 88%|████████▊ | 198310/225000 [377:47:31<59:47:27,  8.06s/it]
 88%|████████▊ | 198311/225000 [377:47:40<61:43:47,  8.33s/it]
 88%|████████▊ | 198312/225000 [377:47:50<64:32:22,  8.71s/it]
 88%|████████▊ | 198313/225000 [377:47:57<61:19:00,  8.27s/it]
 88%|████████▊ | 198314/225000 [377:48:07<65:39:29,  8.86s/it]
 88%|████████▊ | 198315/225000 [377:48:16<65:53:55,  8.89s/it]
 88%|████████▊ | 198316/225000 [377:48:22<59:55:43,  8.09s/it]
 88%|████████▊ | 198317/225000 [377:48:34<66:48:13,  9.01s/it]
 88%|████████▊ | 198318/225000 [377:48:41<62:24:04,  8.42s/it]
 88%|████████▊ | 198319/225000 [377:48:51<66:05:27,  8.92s/it]
 88%|████████▊ | 198320/225000 [377:48:58<63:24:28,  8.56s/it]
 88%|████████▊ | 198321/225000 [377:49:07<62:44:09,  8.47s/it]
 88%|████████▊ | 198322/225000 [377:49:13<58:55:01,  7.95s/it]
 88%|████████▊ | 198323/225000 [377:49:19<52:44:33,  7.12s/it]
 88%|████████▊ | 198324/225000 [377:49:25<51:03:54,  6.89s/it]
 88%|████████▊ | 198325/225000 [377:49:42<72:51:03,  9.83s/it]
 88%|████████▊ | 198326/225000 [377:49:50<69:53:27,  9.43s/it]
 88%|████████▊ | 198327/225000 [377:49:57<63:46:59,  8.61s/it]
 88%|████████▊ | 198328/225000 [377:50:05<63:34:04,  8.58s/it]
 88%|████████▊ | 198329/225000 [377:50:13<61:24:41,  8.29s/it]
 88%|████████▊ | 198330/225000 [377:50:28<75:34:30, 10.20s/it]
 88%|████████▊ | 198331/225000 [377:50:34<66:51:58,  9.03s/it]
 88%|████████▊ | 198332/225000 [377:50:46<74:02:01,  9.99s/it]
 88%|████████▊ | 198333/225000 [377:50:55<71:02:21,  9.59s/it]
 88%|████████▊ | 198334/225000 [377:51:01<62:43:57,  8.47s/it]
 88%|████████▊ | 198335/225000 [377:51:09<62:05:37,  8.38s/it]
 88%|████████▊ | 198336/225000 [377:51:22<72:47:57,  9.83s/it]
 88%|████████▊ | 198337/225000 [377:51:28<63:01:52,  8.51s/it]
 88%|████████▊ | 198338/225000 [377:51:34<57:40:57,  7.79s/it]
 88%|████████▊ | 198339/225000 [377:51:43<61:18:12,  8.28s/it]
 88%|████████▊ | 198340/225000 [377:51:49<55:08:01,  7.44s/it]
 88%|████████▊ | 198341/225000 [377:51:55<52:07:51,  7.04s/it]
 88%|████████▊ | 198342/225000 [377:52:00<48:57:09,  6.61s/it]
 88%|████████▊ | 198343/225000 [377:52:08<50:48:06,  6.86s/it]
 88%|████████▊ | 198344/225000 [377:52:13<48:17:10,  6.52s/it]
 88%|████████▊ | 198345/225000 [377:52:23<55:25:58,  7.49s/it]
 88%|████████▊ | 198346/225000 [377:52:36<66:12:30,  8.94s/it]
 88%|████████▊ | 198347/225000 [377:52:45<68:27:19,  9.25s/it]
 88%|████████▊ | 198348/225000 [377:52:54<66:20:31,  8.96s/it]
 88%|████████▊ | 198349/225000 [377:53:04<68:07:14,  9.20s/it]
 88%|████████▊ | 198350/225000 [377:53:11<63:33:34,  8.59s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.153', 'learning_rate': '1.885e-06', 'epoch': '0.8816', 'num_input_tokens_seen': 632563584, 'train_runtime': '1.36e+06', 'train_tokens_per_second': '465'}

 88%|████████▊ | 198350/225000 [377:53:11<63:33:34,  8.59s/it]
 88%|████████▊ | 198351/225000 [377:53:23<71:25:04,  9.65s/it]
 88%|████████▊ | 198352/225000 [377:53:30<66:38:01,  9.00s/it]
 88%|████████▊ | 198353/225000 [377:53:40<68:23:19,  9.24s/it]
 88%|████████▊ | 198354/225000 [377:53:45<58:36:51,  7.92s/it]
 88%|████████▊ | 198355/225000 [377:53:57<66:46:02,  9.02s/it]
 88%|████████▊ | 198356/225000 [377:54:03<60:33:58,  8.18s/it]
 88%|████████▊ | 198357/225000 [377:54:09<55:49:56,  7.54s/it]
 88%|████████▊ | 198358/225000 [377:54:17<58:00:28,  7.84s/it]
 88%|████████▊ | 198359/225000 [377:54:28<64:54:44,  8.77s/it]
 88%|████████▊ | 198360/225000 [377:54:36<63:37:27,  8.60s/it]
 88%|████████▊ | 198361/225000 [377:54:46<66:43:43,  9.02s/it]
 88%|████████▊ | 198362/225000 [377:54:54<64:21:39,  8.70s/it]
 88%|████████▊ | 198363/225000 [377:55:02<62:19:24,  8.42s/it]
 88%|████████▊ | 198364/225000 [377:55:06<52:05:58,  7.04s/it]
 88%|████████▊ | 198365/225000 [377:55:15<56:57:01,  7.70s/it]
 88%|████████▊ | 198366/225000 [377:55:26<63:25:23,  8.57s/it]
 88%|████████▊ | 198367/225000 [377:55:30<53:50:39,  7.28s/it]
 88%|████████▊ | 198368/225000 [377:55:39<58:09:49,  7.86s/it]
 88%|████████▊ | 198369/225000 [377:55:48<59:15:47,  8.01s/it]
 88%|████████▊ | 198370/225000 [377:55:58<64:03:08,  8.66s/it]
 88%|████████▊ | 198371/225000 [377:56:05<60:43:49,  8.21s/it]
 88%|████████▊ | 198372/225000 [377:56:12<58:48:15,  7.95s/it]
 88%|████████▊ | 198373/225000 [377:56:24<66:10:31,  8.95s/it]
 88%|████████▊ | 198374/225000 [377:56:30<60:36:36,  8.19s/it]
 88%|████████▊ | 198375/225000 [377:56:38<60:46:09,  8.22s/it]
 88%|████████▊ | 198376/225000 [377:56:47<62:40:11,  8.47s/it]
 88%|████████▊ | 198377/225000 [377:57:03<79:08:00, 10.70s/it]
 88%|████████▊ | 198378/225000 [377:57:10<69:32:59,  9.40s/it]
 88%|████████▊ | 198379/225000 [377:57:18<66:34:28,  9.00s/it]
 88%|████████▊ | 198380/225000 [377:57:26<64:07:27,  8.67s/it]
 88%|████████▊ | 198381/225000 [377:57:40<77:14:20, 10.45s/it]
 88%|████████▊ | 198382/225000 [377:57:47<69:29:10,  9.40s/it]
 88%|████████▊ | 198383/225000 [377:57:58<71:42:17,  9.70s/it]
 88%|████████▊ | 198384/225000 [377:58:06<69:03:46,  9.34s/it]
 88%|████████▊ | 198385/225000 [377:58:14<65:29:01,  8.86s/it]
 88%|████████▊ | 198386/225000 [377:58:21<60:46:48,  8.22s/it]
 88%|████████▊ | 198387/225000 [377:58:26<54:44:59,  7.41s/it]
 88%|████████▊ | 198388/225000 [377:58:32<50:52:25,  6.88s/it]
 88%|████████▊ | 198389/225000 [377:58:42<57:37:29,  7.80s/it]
 88%|████████▊ | 198390/225000 [377:58:49<56:57:48,  7.71s/it]
 88%|████████▊ | 198391/225000 [377:58:53<49:26:09,  6.69s/it]
 88%|████████▊ | 198392/225000 [377:59:04<57:32:53,  7.79s/it]
 88%|████████▊ | 198393/225000 [377:59:12<58:41:26,  7.94s/it]
 88%|████████▊ | 198394/225000 [377:59:22<63:22:22,  8.57s/it]
 88%|████████▊ | 198395/225000 [377:59:27<55:39:46,  7.53s/it]
 88%|████████▊ | 198396/225000 [377:59:35<55:07:45,  7.46s/it]
 88%|████████▊ | 198397/225000 [377:59:48<67:46:29,  9.17s/it]
 88%|████████▊ | 198398/225000 [377:59:54<60:42:49,  8.22s/it]
 88%|████████▊ | 198399/225000 [378:00:02<61:37:55,  8.34s/it]
 88%|████████▊ | 198400/225000 [378:00:11<62:04:37,  8.40s/it]
                                                              
{'loss': '1.329', 'grad_norm': '1.382', 'learning_rate': '1.878e-06', 'epoch': '0.8818', 'num_input_tokens_seen': 632786984, 'train_runtime': '1.361e+06', 'train_tokens_per_second': '465'}

 88%|████████▊ | 198400/225000 [378:00:11<62:04:37,  8.40s/it]
 88%|████████▊ | 198401/225000 [378:00:19<62:25:19,  8.45s/it]
 88%|████████▊ | 198402/225000 [378:00:24<54:20:31,  7.36s/it]
 88%|████████▊ | 198403/225000 [378:00:32<54:22:45,  7.36s/it]
 88%|████████▊ | 198404/225000 [378:00:45<66:48:04,  9.04s/it]
 88%|████████▊ | 198405/225000 [378:00:53<64:53:05,  8.78s/it]
 88%|████████▊ | 198406/225000 [378:01:01<62:55:15,  8.52s/it]
 88%|████████▊ | 198407/225000 [378:01:09<62:24:45,  8.45s/it]
 88%|████████▊ | 198408/225000 [378:01:20<68:44:57,  9.31s/it]
 88%|████████▊ | 198409/225000 [378:01:32<74:19:59, 10.06s/it]
 88%|████████▊ | 198410/225000 [378:01:38<64:36:54,  8.75s/it]
 88%|████████▊ | 198411/225000 [378:01:50<71:40:54,  9.71s/it]
 88%|████████▊ | 198412/225000 [378:01:56<63:39:36,  8.62s/it]
 88%|████████▊ | 198413/225000 [378:02:07<68:51:24,  9.32s/it]
 88%|████████▊ | 198414/225000 [378:02:19<75:25:47, 10.21s/it]
 88%|████████▊ | 198415/225000 [378:02:26<67:29:20,  9.14s/it]
 88%|████████▊ | 198416/225000 [378:02:38<73:39:52,  9.98s/it]
 88%|████████▊ | 198417/225000 [378:02:44<66:24:13,  8.99s/it]
 88%|████████▊ | 198418/225000 [378:02:51<60:49:50,  8.24s/it]
 88%|████████▊ | 198419/225000 [378:02:58<58:31:05,  7.93s/it]
 88%|████████▊ | 198420/225000 [378:03:10<68:22:35,  9.26s/it]
 88%|████████▊ | 198421/225000 [378:03:19<67:26:27,  9.13s/it]
 88%|████████▊ | 198422/225000 [378:03:25<59:08:14,  8.01s/it]
 88%|████████▊ | 198423/225000 [378:03:38<70:33:52,  9.56s/it]
 88%|████████▊ | 198424/225000 [378:03:45<65:57:23,  8.93s/it]
 88%|████████▊ | 198425/225000 [378:03:54<64:28:49,  8.73s/it]
 88%|████████▊ | 198426/225000 [378:04:00<59:44:02,  8.09s/it]
 88%|████████▊ | 198427/225000 [378:04:07<57:12:48,  7.75s/it]
 88%|████████▊ | 198428/225000 [378:04:13<53:34:07,  7.26s/it]
 88%|████████▊ | 198429/225000 [378:04:24<60:32:39,  8.20s/it]
 88%|████████▊ | 198430/225000 [378:04:35<67:22:06,  9.13s/it]
 88%|████████▊ | 198431/225000 [378:04:42<62:50:18,  8.51s/it]
 88%|████████▊ | 198432/225000 [378:04:49<59:39:23,  8.08s/it]
 88%|████████▊ | 198433/225000 [378:04:56<57:36:07,  7.81s/it]
 88%|████████▊ | 198434/225000 [378:05:06<63:02:44,  8.54s/it]
 88%|████████▊ | 198435/225000 [378:05:16<64:55:25,  8.80s/it]
 88%|████████▊ | 198436/225000 [378:05:27<69:14:14,  9.38s/it]
 88%|████████▊ | 198437/225000 [378:05:37<70:26:38,  9.55s/it]
 88%|████████▊ | 198438/225000 [378:05:45<68:41:33,  9.31s/it]
 88%|████████▊ | 198439/225000 [378:05:55<70:04:41,  9.50s/it]
 88%|████████▊ | 198440/225000 [378:06:01<62:17:23,  8.44s/it]
 88%|████████▊ | 198441/225000 [378:06:09<59:44:23,  8.10s/it]
 88%|████████▊ | 198442/225000 [378:06:23<72:54:42,  9.88s/it]
 88%|████████▊ | 198443/225000 [378:06:29<64:21:06,  8.72s/it]
 88%|████████▊ | 198444/225000 [378:06:40<69:47:27,  9.46s/it]
 88%|████████▊ | 198445/225000 [378:06:45<59:24:02,  8.05s/it]
 88%|████████▊ | 198446/225000 [378:06:51<55:28:54,  7.52s/it]
 88%|████████▊ | 198447/225000 [378:06:58<55:10:11,  7.48s/it]
 88%|████████▊ | 198448/225000 [378:07:05<54:34:45,  7.40s/it]
 88%|████████▊ | 198449/225000 [378:07:18<65:37:33,  8.90s/it]
 88%|████████▊ | 198450/225000 [378:07:27<65:49:33,  8.93s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.352', 'learning_rate': '1.871e-06', 'epoch': '0.882', 'num_input_tokens_seen': 633020336, 'train_runtime': '1.361e+06', 'train_tokens_per_second': '465'}

 88%|████████▊ | 198450/225000 [378:07:27<65:49:33,  8.93s/it]
 88%|████████▊ | 198451/225000 [378:07:33<59:15:08,  8.03s/it]
 88%|████████▊ | 198452/225000 [378:07:42<62:05:14,  8.42s/it]
 88%|████████▊ | 198453/225000 [378:07:50<60:17:34,  8.18s/it]
 88%|████████▊ | 198454/225000 [378:07:57<57:40:21,  7.82s/it]
 88%|████████▊ | 198455/225000 [378:08:07<63:49:28,  8.66s/it]
 88%|████████▊ | 198456/225000 [378:08:12<54:47:59,  7.43s/it]
 88%|████████▊ | 198457/225000 [378:08:21<57:48:01,  7.84s/it]
 88%|████████▊ | 198458/225000 [378:08:30<61:40:25,  8.37s/it]
 88%|████████▊ | 198459/225000 [378:08:40<65:33:09,  8.89s/it]
 88%|████████▊ | 198460/225000 [378:08:46<58:07:49,  7.89s/it]
 88%|████████▊ | 198461/225000 [378:08:52<53:26:04,  7.25s/it]
 88%|████████▊ | 198462/225000 [378:09:00<55:40:50,  7.55s/it]
 88%|████████▊ | 198463/225000 [378:09:05<51:15:59,  6.95s/it]
 88%|████████▊ | 198464/225000 [378:09:18<64:33:39,  8.76s/it]
 88%|████████▊ | 198465/225000 [378:09:27<63:15:54,  8.58s/it]
 88%|████████▊ | 198466/225000 [378:09:33<57:34:24,  7.81s/it]
 88%|████████▊ | 198467/225000 [378:09:44<66:19:57,  9.00s/it]
 88%|████████▊ | 198468/225000 [378:09:52<64:21:24,  8.73s/it]
 88%|████████▊ | 198469/225000 [378:10:01<64:46:33,  8.79s/it]
 88%|████████▊ | 198470/225000 [378:10:14<72:57:45,  9.90s/it]
 88%|████████▊ | 198471/225000 [378:10:19<62:17:01,  8.45s/it]
 88%|████████▊ | 198472/225000 [378:10:25<56:33:00,  7.67s/it]
 88%|████████▊ | 198473/225000 [378:10:38<67:48:40,  9.20s/it]
 88%|████████▊ | 198474/225000 [378:10:43<58:26:21,  7.93s/it]
 88%|████████▊ | 198475/225000 [378:10:53<64:15:34,  8.72s/it]
 88%|████████▊ | 198476/225000 [378:11:00<59:55:07,  8.13s/it]
 88%|████████▊ | 198477/225000 [378:11:04<50:29:18,  6.85s/it]
 88%|████████▊ | 198478/225000 [378:11:14<58:55:27,  8.00s/it]
 88%|████████▊ | 198479/225000 [378:11:25<64:18:16,  8.73s/it]
 88%|████████▊ | 198480/225000 [378:11:29<54:00:53,  7.33s/it]
 88%|████████▊ | 198481/225000 [378:11:38<58:07:48,  7.89s/it]
 88%|████████▊ | 198482/225000 [378:11:48<61:33:54,  8.36s/it]
 88%|████████▊ | 198483/225000 [378:11:56<60:47:22,  8.25s/it]
 88%|████████▊ | 198484/225000 [378:12:04<60:18:58,  8.19s/it]
 88%|████████▊ | 198485/225000 [378:12:12<61:42:34,  8.38s/it]
 88%|████████▊ | 198486/225000 [378:12:21<62:39:12,  8.51s/it]
 88%|████████▊ | 198487/225000 [378:12:30<62:53:14,  8.54s/it]
 88%|████████▊ | 198488/225000 [378:12:35<54:56:41,  7.46s/it]
 88%|████████▊ | 198489/225000 [378:12:41<51:00:50,  6.93s/it]
 88%|████████▊ | 198490/225000 [378:12:54<65:54:06,  8.95s/it]
 88%|████████▊ | 198491/225000 [378:13:01<61:02:08,  8.29s/it]
 88%|████████▊ | 198492/225000 [378:13:11<65:59:18,  8.96s/it]
 88%|████████▊ | 198493/225000 [378:13:24<73:43:49, 10.01s/it]
 88%|████████▊ | 198494/225000 [378:13:33<70:41:23,  9.60s/it]
 88%|████████▊ | 198495/225000 [378:13:38<62:06:40,  8.44s/it]
 88%|████████▊ | 198496/225000 [378:13:48<65:07:48,  8.85s/it]
 88%|████████▊ | 198497/225000 [378:14:02<75:48:49, 10.30s/it]
 88%|████████▊ | 198498/225000 [378:14:06<62:05:16,  8.43s/it]
 88%|████████▊ | 198499/225000 [378:14:15<63:25:17,  8.62s/it]
 88%|████████▊ | 198500/225000 [378:14:22<60:31:15,  8.22s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.893', 'learning_rate': '1.864e-06', 'epoch': '0.8822', 'num_input_tokens_seen': 633244168, 'train_runtime': '1.362e+06', 'train_tokens_per_second': '465.1'}

 88%|████████▊ | 198500/225000 [378:14:22<60:31:15,  8.22s/it]
 88%|████████▊ | 198501/225000 [378:14:35<70:18:49,  9.55s/it]
 88%|████████▊ | 198502/225000 [378:14:40<61:27:14,  8.35s/it]
 88%|████████▊ | 198503/225000 [378:14:50<65:12:20,  8.86s/it]
 88%|████████▊ | 198504/225000 [378:14:59<64:13:33,  8.73s/it]
 88%|████████▊ | 198505/225000 [378:15:06<59:54:28,  8.14s/it]
 88%|████████▊ | 198506/225000 [378:15:12<55:14:13,  7.51s/it]
 88%|████████▊ | 198507/225000 [378:15:18<53:32:05,  7.27s/it]
 88%|████████▊ | 198508/225000 [378:15:25<51:02:56,  6.94s/it]
 88%|████████▊ | 198509/225000 [378:15:39<67:25:11,  9.16s/it]
 88%|████████▊ | 198510/225000 [378:15:47<65:27:10,  8.90s/it]
 88%|████████▊ | 198511/225000 [378:15:52<56:30:42,  7.68s/it]
 88%|████████▊ | 198512/225000 [378:16:00<57:11:50,  7.77s/it]
 88%|████████▊ | 198513/225000 [378:16:07<55:31:45,  7.55s/it]
 88%|████████▊ | 198514/225000 [378:16:19<64:45:29,  8.80s/it]
 88%|████████▊ | 198515/225000 [378:16:28<65:12:32,  8.86s/it]
 88%|████████▊ | 198516/225000 [378:16:38<67:35:47,  9.19s/it]
 88%|████████▊ | 198517/225000 [378:16:46<64:38:21,  8.79s/it]
 88%|████████▊ | 198518/225000 [378:16:58<73:13:28,  9.95s/it]
 88%|████████▊ | 198519/225000 [378:17:08<73:35:28, 10.00s/it]
 88%|████████▊ | 198520/225000 [378:17:15<66:46:02,  9.08s/it]
 88%|████████▊ | 198521/225000 [378:17:25<68:30:15,  9.31s/it]
 88%|████████▊ | 198522/225000 [378:17:32<62:24:19,  8.48s/it]
 88%|████████▊ | 198523/225000 [378:17:44<71:52:23,  9.77s/it]
 88%|████████▊ | 198524/225000 [378:17:54<72:00:31,  9.79s/it]
 88%|████████▊ | 198525/225000 [378:18:01<65:31:20,  8.91s/it]
 88%|████████▊ | 198526/225000 [378:18:12<70:16:28,  9.56s/it]
 88%|████████▊ | 198527/225000 [378:18:22<71:33:26,  9.73s/it]
 88%|████████▊ | 198528/225000 [378:18:32<71:02:42,  9.66s/it]
 88%|████████▊ | 198529/225000 [378:18:42<72:27:23,  9.85s/it]
 88%|████████▊ | 198530/225000 [378:18:48<63:33:39,  8.64s/it]
 88%|████████▊ | 198531/225000 [378:18:56<61:44:44,  8.40s/it]
 88%|████████▊ | 198532/225000 [378:19:01<54:21:09,  7.39s/it]
 88%|████████▊ | 198533/225000 [378:19:12<62:50:00,  8.55s/it]
 88%|████████▊ | 198534/225000 [378:19:20<60:21:06,  8.21s/it]
 88%|████████▊ | 198535/225000 [378:19:28<61:30:46,  8.37s/it]
 88%|████████▊ | 198536/225000 [378:19:34<55:19:52,  7.53s/it]
 88%|████████▊ | 198537/225000 [378:19:46<64:57:28,  8.84s/it]
 88%|████████▊ | 198538/225000 [378:19:56<68:16:15,  9.29s/it]
 88%|████████▊ | 198539/225000 [378:20:06<70:30:30,  9.59s/it]
 88%|████████▊ | 198540/225000 [378:20:11<60:25:07,  8.22s/it]
 88%|████████▊ | 198541/225000 [378:20:25<73:14:32,  9.97s/it]
 88%|████████▊ | 198542/225000 [378:20:33<68:31:33,  9.32s/it]
 88%|████████▊ | 198543/225000 [378:20:41<64:54:06,  8.83s/it]
 88%|████████▊ | 198544/225000 [378:20:48<61:15:30,  8.34s/it]
 88%|████████▊ | 198545/225000 [378:20:54<56:30:18,  7.69s/it]
 88%|████████▊ | 198546/225000 [378:21:02<56:25:43,  7.68s/it]
 88%|████████▊ | 198547/225000 [378:21:07<51:35:27,  7.02s/it]
 88%|████████▊ | 198548/225000 [378:21:15<52:18:32,  7.12s/it]
 88%|████████▊ | 198549/225000 [378:21:28<65:03:37,  8.85s/it]
 88%|████████▊ | 198550/225000 [378:21:34<59:18:52,  8.07s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.428', 'learning_rate': '1.857e-06', 'epoch': '0.8824', 'num_input_tokens_seen': 633475016, 'train_runtime': '1.362e+06', 'train_tokens_per_second': '465.1'}

 88%|████████▊ | 198550/225000 [378:21:34<59:18:52,  8.07s/it]
 88%|████████▊ | 198551/225000 [378:21:39<51:54:26,  7.07s/it]
 88%|████████▊ | 198552/225000 [378:21:50<60:22:41,  8.22s/it]
 88%|████████▊ | 198553/225000 [378:21:56<55:25:54,  7.55s/it]
 88%|████████▊ | 198554/225000 [378:22:03<56:22:38,  7.67s/it]
 88%|████████▊ | 198555/225000 [378:22:17<69:25:59,  9.45s/it]
 88%|████████▊ | 198556/225000 [378:22:24<64:01:32,  8.72s/it]
 88%|████████▊ | 198557/225000 [378:22:33<63:24:03,  8.63s/it]
 88%|████████▊ | 198558/225000 [378:22:41<63:37:06,  8.66s/it]
 88%|████████▊ | 198559/225000 [378:22:49<61:05:51,  8.32s/it]
 88%|████████▊ | 198560/225000 [378:22:59<65:56:22,  8.98s/it]
 88%|████████▊ | 198561/225000 [378:23:08<65:02:46,  8.86s/it]
 88%|████████▊ | 198562/225000 [378:23:17<65:38:19,  8.94s/it]
 88%|████████▊ | 198563/225000 [378:23:22<57:29:54,  7.83s/it]
 88%|████████▊ | 198564/225000 [378:23:30<56:22:55,  7.68s/it]
 88%|████████▊ | 198565/225000 [378:23:41<63:42:23,  8.68s/it]
 88%|████████▊ | 198566/225000 [378:23:50<65:25:19,  8.91s/it]
 88%|████████▊ | 198567/225000 [378:23:56<58:03:26,  7.91s/it]
 88%|████████▊ | 198568/225000 [378:24:06<63:35:49,  8.66s/it]
 88%|████████▊ | 198569/225000 [378:24:12<56:55:16,  7.75s/it]
 88%|████████▊ | 198570/225000 [378:24:19<56:00:47,  7.63s/it]
 88%|████████▊ | 198571/225000 [378:24:28<59:59:53,  8.17s/it]
 88%|████████▊ | 198572/225000 [378:24:39<66:04:41,  9.00s/it]
 88%|████████▊ | 198573/225000 [378:24:46<60:55:15,  8.30s/it]
 88%|████████▊ | 198574/225000 [378:24:53<57:51:41,  7.88s/it]
 88%|████████▊ | 198575/225000 [378:25:07<71:54:41,  9.80s/it]
 88%|████████▊ | 198576/225000 [378:25:15<67:10:43,  9.15s/it]
 88%|████████▊ | 198577/225000 [378:25:23<64:56:49,  8.85s/it]
 88%|████████▊ | 198578/225000 [378:25:29<59:31:10,  8.11s/it]
 88%|████████▊ | 198579/225000 [378:25:43<71:30:13,  9.74s/it]
 88%|████████▊ | 198580/225000 [378:25:49<62:24:17,  8.50s/it]
 88%|████████▊ | 198581/225000 [378:26:00<68:51:14,  9.38s/it]
 88%|████████▊ | 198582/225000 [378:26:06<60:53:25,  8.30s/it]
 88%|████████▊ | 198583/225000 [378:26:15<64:06:08,  8.74s/it]
 88%|████████▊ | 198584/225000 [378:26:21<57:37:25,  7.85s/it]
 88%|████████▊ | 198585/225000 [378:26:30<60:27:02,  8.24s/it]
 88%|████████▊ | 198586/225000 [378:26:44<72:33:12,  9.89s/it]
 88%|████████▊ | 198587/225000 [378:26:55<74:43:40, 10.19s/it]
 88%|████████▊ | 198588/225000 [378:27:00<64:12:25,  8.75s/it]
 88%|████████▊ | 198589/225000 [378:27:10<65:27:19,  8.92s/it]
 88%|████████▊ | 198590/225000 [378:27:15<57:46:56,  7.88s/it]
 88%|████████▊ | 198591/225000 [378:27:23<56:49:04,  7.75s/it]
 88%|████████▊ | 198592/225000 [378:27:35<67:42:50,  9.23s/it]
 88%|████████▊ | 198593/225000 [378:27:40<57:17:46,  7.81s/it]
 88%|████████▊ | 198594/225000 [378:27:49<60:05:36,  8.19s/it]
 88%|████████▊ | 198595/225000 [378:28:01<67:48:25,  9.24s/it]
 88%|████████▊ | 198596/225000 [378:28:05<57:08:05,  7.79s/it]
 88%|████████▊ | 198597/225000 [378:28:13<58:03:41,  7.92s/it]
 88%|████████▊ | 198598/225000 [378:28:18<51:24:11,  7.01s/it]
 88%|████████▊ | 198599/225000 [378:28:27<55:21:52,  7.55s/it]
 88%|████████▊ | 198600/225000 [378:28:33<51:55:41,  7.08s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.094', 'learning_rate': '1.85e-06', 'epoch': '0.8827', 'num_input_tokens_seen': 633704072, 'train_runtime': '1.363e+06', 'train_tokens_per_second': '465.1'}

 88%|████████▊ | 198600/225000 [378:28:33<51:55:41,  7.08s/it]
 88%|████████▊ | 198601/225000 [378:28:44<60:41:51,  8.28s/it]
 88%|████████▊ | 198602/225000 [378:28:49<54:08:56,  7.38s/it]
 88%|████████▊ | 198603/225000 [378:29:02<66:17:29,  9.04s/it]
 88%|████████▊ | 198604/225000 [378:29:07<56:43:50,  7.74s/it]
 88%|████████▊ | 198605/225000 [378:29:13<52:54:58,  7.22s/it]
 88%|████████▊ | 198606/225000 [378:29:18<48:30:15,  6.62s/it]
 88%|████████▊ | 198607/225000 [378:29:26<51:15:12,  6.99s/it]
 88%|████████▊ | 198608/225000 [378:29:41<68:09:59,  9.30s/it]
 88%|████████▊ | 198609/225000 [378:29:46<59:46:30,  8.15s/it]
 88%|████████▊ | 198610/225000 [378:29:57<66:40:30,  9.10s/it]
 88%|████████▊ | 198611/225000 [378:30:07<68:26:22,  9.34s/it]
 88%|████████▊ | 198612/225000 [378:30:15<65:34:42,  8.95s/it]
 88%|████████▊ | 198613/225000 [378:30:24<65:52:43,  8.99s/it]
 88%|████████▊ | 198614/225000 [378:30:33<64:46:47,  8.84s/it]
 88%|████████▊ | 198615/225000 [378:30:41<63:42:48,  8.69s/it]
 88%|████████▊ | 198616/225000 [378:30:52<67:29:29,  9.21s/it]
 88%|████████▊ | 198617/225000 [378:30:58<60:59:47,  8.32s/it]
 88%|████████▊ | 198618/225000 [378:31:04<57:02:29,  7.78s/it]
 88%|████████▊ | 198619/225000 [378:31:10<51:59:11,  7.09s/it]
 88%|████████▊ | 198620/225000 [378:31:23<64:35:28,  8.81s/it]
 88%|████████▊ | 198621/225000 [378:31:31<63:50:55,  8.71s/it]
 88%|████████▊ | 198622/225000 [378:31:36<54:49:46,  7.48s/it]
 88%|████████▊ | 198623/225000 [378:31:45<59:22:16,  8.10s/it]
 88%|████████▊ | 198624/225000 [378:31:52<57:04:08,  7.79s/it]
 88%|████████▊ | 198625/225000 [378:32:01<58:46:57,  8.02s/it]
 88%|████████▊ | 198626/225000 [378:32:06<51:41:39,  7.06s/it]
 88%|████████▊ | 198627/225000 [378:32:14<54:24:39,  7.43s/it]
 88%|████████▊ | 198628/225000 [378:32:20<50:05:16,  6.84s/it]
 88%|████████▊ | 198629/225000 [378:32:25<47:34:11,  6.49s/it]
 88%|████████▊ | 198630/225000 [378:32:40<65:36:48,  8.96s/it]
 88%|████████▊ | 198631/225000 [378:32:46<59:46:14,  8.16s/it]
 88%|████████▊ | 198632/225000 [378:32:58<68:09:31,  9.31s/it]
 88%|████████▊ | 198633/225000 [378:33:04<59:12:22,  8.08s/it]
 88%|████████▊ | 198634/225000 [378:33:12<60:12:56,  8.22s/it]
 88%|████████▊ | 198635/225000 [378:33:18<54:14:47,  7.41s/it]
 88%|████████▊ | 198636/225000 [378:33:28<61:16:24,  8.37s/it]
 88%|████████▊ | 198637/225000 [378:33:35<58:46:44,  8.03s/it]
 88%|████████▊ | 198638/225000 [378:33:44<60:30:25,  8.26s/it]
 88%|████████▊ | 198639/225000 [378:33:52<60:20:42,  8.24s/it]
 88%|████████▊ | 198640/225000 [378:33:58<54:00:32,  7.38s/it]
 88%|████████▊ | 198641/225000 [378:34:04<50:54:04,  6.95s/it]
 88%|████████▊ | 198642/225000 [378:34:16<63:31:22,  8.68s/it]
 88%|████████▊ | 198643/225000 [378:34:22<57:14:36,  7.82s/it]
 88%|████████▊ | 198644/225000 [378:34:31<58:55:34,  8.05s/it]
 88%|████████▊ | 198645/225000 [378:34:41<63:34:12,  8.68s/it]
 88%|████████▊ | 198646/225000 [378:34:51<65:35:00,  8.96s/it]
 88%|████████▊ | 198647/225000 [378:34:57<60:21:21,  8.25s/it]
 88%|████████▊ | 198648/225000 [378:35:10<70:18:35,  9.61s/it]
 88%|████████▊ | 198649/225000 [378:35:18<66:21:20,  9.07s/it]
 88%|████████▊ | 198650/225000 [378:35:30<72:58:40,  9.97s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.146', 'learning_rate': '1.843e-06', 'epoch': '0.8829', 'num_input_tokens_seen': 633926304, 'train_runtime': '1.363e+06', 'train_tokens_per_second': '465.1'}

 88%|████████▊ | 198650/225000 [378:35:30<72:58:40,  9.97s/it]
 88%|████████▊ | 198651/225000 [378:35:35<61:34:28,  8.41s/it]
 88%|████████▊ | 198652/225000 [378:35:42<58:17:27,  7.96s/it]
 88%|████████▊ | 198653/225000 [378:35:50<59:20:42,  8.11s/it]
 88%|████████▊ | 198654/225000 [378:36:02<67:22:29,  9.21s/it]
 88%|████████▊ | 198655/225000 [378:36:09<63:21:37,  8.66s/it]
 88%|████████▊ | 198656/225000 [378:36:18<62:58:22,  8.61s/it]
 88%|████████▊ | 198657/225000 [378:36:22<53:41:01,  7.34s/it]
 88%|████████▊ | 198658/225000 [378:36:32<60:02:02,  8.20s/it]
 88%|████████▊ | 198659/225000 [378:36:43<64:48:31,  8.86s/it]
 88%|████████▊ | 198660/225000 [378:36:52<65:09:23,  8.91s/it]
 88%|████████▊ | 198661/225000 [378:36:57<57:42:01,  7.89s/it]
 88%|████████▊ | 198662/225000 [378:37:07<62:50:07,  8.59s/it]
 88%|████████▊ | 198663/225000 [378:37:14<59:21:34,  8.11s/it]
 88%|████████▊ | 198664/225000 [378:37:21<57:04:53,  7.80s/it]
 88%|████████▊ | 198665/225000 [378:37:31<61:31:55,  8.41s/it]
 88%|████████▊ | 198666/225000 [378:37:43<68:59:43,  9.43s/it]
 88%|████████▊ | 198667/225000 [378:37:49<61:27:18,  8.40s/it]
 88%|████████▊ | 198668/225000 [378:37:54<52:56:33,  7.24s/it]
 88%|████████▊ | 198669/225000 [378:38:05<61:12:42,  8.37s/it]
 88%|████████▊ | 198670/225000 [378:38:12<59:30:39,  8.14s/it]
 88%|████████▊ | 198671/225000 [378:38:18<54:57:49,  7.52s/it]
 88%|████████▊ | 198672/225000 [378:38:24<51:13:53,  7.01s/it]
 88%|████████▊ | 198673/225000 [378:38:31<50:50:03,  6.95s/it]
 88%|████████▊ | 198674/225000 [378:38:39<53:30:04,  7.32s/it]
 88%|████████▊ | 198675/225000 [378:38:54<69:19:32,  9.48s/it]
 88%|████████▊ | 198676/225000 [378:39:01<65:47:17,  9.00s/it]
 88%|████████▊ | 198677/225000 [378:39:12<69:53:13,  9.56s/it]
 88%|████████▊ | 198678/225000 [378:39:19<63:47:18,  8.72s/it]
 88%|████████▊ | 198679/225000 [378:39:25<57:42:35,  7.89s/it]
 88%|████████▊ | 198680/225000 [378:39:34<60:50:45,  8.32s/it]
 88%|████████▊ | 198681/225000 [378:39:43<60:23:37,  8.26s/it]
 88%|████████▊ | 198682/225000 [378:39:53<65:58:35,  9.02s/it]
 88%|████████▊ | 198683/225000 [378:40:05<72:46:57,  9.96s/it]
 88%|████████▊ | 198684/225000 [378:40:14<69:24:57,  9.50s/it]
 88%|████████▊ | 198685/225000 [378:40:23<68:54:31,  9.43s/it]
 88%|████████▊ | 198686/225000 [378:40:35<74:31:43, 10.20s/it]
 88%|████████▊ | 198687/225000 [378:40:46<75:06:05, 10.27s/it]
 88%|████████▊ | 198688/225000 [378:40:52<67:37:19,  9.25s/it]
 88%|████████▊ | 198689/225000 [378:41:02<68:28:33,  9.37s/it]
 88%|████████▊ | 198690/225000 [378:41:13<71:32:28,  9.79s/it]
 88%|████████▊ | 198691/225000 [378:41:19<62:52:49,  8.60s/it]
 88%|████████▊ | 198692/225000 [378:41:25<58:15:47,  7.97s/it]
 88%|████████▊ | 198693/225000 [378:41:37<66:31:30,  9.10s/it]
 88%|████████▊ | 198694/225000 [378:41:44<61:21:42,  8.40s/it]
 88%|████████▊ | 198695/225000 [378:41:50<55:59:02,  7.66s/it]
 88%|████████▊ | 198696/225000 [378:42:00<62:28:14,  8.55s/it]
 88%|████████▊ | 198697/225000 [378:42:11<66:28:29,  9.10s/it]
 88%|████████▊ | 198698/225000 [378:42:21<69:50:57,  9.56s/it]
 88%|████████▊ | 198699/225000 [378:42:28<62:44:44,  8.59s/it]
 88%|████████▊ | 198700/225000 [378:42:35<59:43:37,  8.18s/it]
                                                              
{'loss': '1.304', 'grad_norm': '0.9072', 'learning_rate': '1.836e-06', 'epoch': '0.8831', 'num_input_tokens_seen': 634150392, 'train_runtime': '1.363e+06', 'train_tokens_per_second': '465.1'}

 88%|████████▊ | 198700/225000 [378:42:35<59:43:37,  8.18s/it]
 88%|████████▊ | 198701/225000 [378:42:46<65:46:01,  9.00s/it]
 88%|████████▊ | 198702/225000 [378:42:51<56:37:16,  7.75s/it]
 88%|████████▊ | 198703/225000 [378:43:04<69:49:05,  9.56s/it]
 88%|████████▊ | 198704/225000 [378:43:09<60:06:59,  8.23s/it]
 88%|████████▊ | 198705/225000 [378:43:20<64:13:56,  8.79s/it]
 88%|████████▊ | 198706/225000 [378:43:29<66:02:52,  9.04s/it]
 88%|████████▊ | 198707/225000 [378:43:35<58:41:30,  8.04s/it]
 88%|████████▊ | 198708/225000 [378:43:46<66:22:36,  9.09s/it]
 88%|████████▊ | 198709/225000 [378:43:54<62:44:36,  8.59s/it]
 88%|████████▊ | 198710/225000 [378:44:05<67:10:34,  9.20s/it]
 88%|████████▊ | 198711/225000 [378:44:13<64:43:00,  8.86s/it]
 88%|████████▊ | 198712/225000 [378:44:18<58:08:10,  7.96s/it]
 88%|████████▊ | 198713/225000 [378:44:30<66:17:08,  9.08s/it]
 88%|████████▊ | 198714/225000 [378:44:38<64:03:27,  8.77s/it]
 88%|████████▊ | 198715/225000 [378:44:48<65:37:13,  8.99s/it]
 88%|████████▊ | 198716/225000 [378:44:56<64:39:46,  8.86s/it]
 88%|████████▊ | 198717/225000 [378:45:03<60:47:37,  8.33s/it]
 88%|████████▊ | 198718/225000 [378:45:16<70:54:51,  9.71s/it]
 88%|████████▊ | 198719/225000 [378:45:23<63:19:53,  8.68s/it]
 88%|████████▊ | 198720/225000 [378:45:35<71:39:39,  9.82s/it]
 88%|████████▊ | 198721/225000 [378:45:44<68:48:19,  9.43s/it]
 88%|████████▊ | 198722/225000 [378:45:50<61:49:33,  8.47s/it]
 88%|████████▊ | 198723/225000 [378:46:04<74:54:53, 10.26s/it]
 88%|████████▊ | 198724/225000 [378:46:11<68:01:39,  9.32s/it]
 88%|████████▊ | 198725/225000 [378:46:19<63:23:42,  8.69s/it]
 88%|████████▊ | 198726/225000 [378:46:26<60:40:26,  8.31s/it]
 88%|████████▊ | 198727/225000 [378:46:36<63:53:55,  8.76s/it]
 88%|████████▊ | 198728/225000 [378:46:43<60:42:26,  8.32s/it]
 88%|████████▊ | 198729/225000 [378:46:50<57:20:51,  7.86s/it]
 88%|████████▊ | 198730/225000 [378:47:01<64:15:20,  8.81s/it]
 88%|████████▊ | 198731/225000 [378:47:10<63:55:33,  8.76s/it]
 88%|████████▊ | 198732/225000 [378:47:15<56:08:37,  7.69s/it]
 88%|████████▊ | 198733/225000 [378:47:26<63:36:44,  8.72s/it]
 88%|████████▊ | 198734/225000 [378:47:31<55:55:15,  7.66s/it]
 88%|████████▊ | 198735/225000 [378:47:36<49:44:40,  6.82s/it]
 88%|████████▊ | 198736/225000 [378:47:47<58:07:45,  7.97s/it]
 88%|████████▊ | 198737/225000 [378:47:58<66:03:11,  9.05s/it]
 88%|████████▊ | 198738/225000 [378:48:09<69:17:54,  9.50s/it]
 88%|████████▊ | 198739/225000 [378:48:15<62:18:45,  8.54s/it]
 88%|████████▊ | 198740/225000 [378:48:20<55:33:13,  7.62s/it]
 88%|████████▊ | 198741/225000 [378:48:30<60:28:41,  8.29s/it]
 88%|████████▊ | 198742/225000 [378:48:37<57:45:54,  7.92s/it]
 88%|████████▊ | 198743/225000 [378:48:42<50:08:03,  6.87s/it]
 88%|████████▊ | 198744/225000 [378:48:53<59:06:14,  8.10s/it]
 88%|████████▊ | 198745/225000 [378:49:01<58:36:36,  8.04s/it]
 88%|████████▊ | 198746/225000 [378:49:08<56:19:49,  7.72s/it]
 88%|████████▊ | 198747/225000 [378:49:18<61:47:40,  8.47s/it]
 88%|████████▊ | 198748/225000 [378:49:26<61:42:21,  8.46s/it]
 88%|████████▊ | 198749/225000 [378:49:37<67:26:44,  9.25s/it]
 88%|████████▊ | 198750/225000 [378:49:43<60:07:55,  8.25s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.405', 'learning_rate': '1.829e-06', 'epoch': '0.8833', 'num_input_tokens_seen': 634382008, 'train_runtime': '1.364e+06', 'train_tokens_per_second': '465.2'}

 88%|████████▊ | 198750/225000 [378:49:43<60:07:55,  8.25s/it]
 88%|████████▊ | 198751/225000 [378:49:51<59:34:38,  8.17s/it]
 88%|████████▊ | 198752/225000 [378:49:58<56:14:07,  7.71s/it]
 88%|████████▊ | 198753/225000 [378:50:08<61:36:47,  8.45s/it]
 88%|████████▊ | 198754/225000 [378:50:20<68:33:47,  9.40s/it]
 88%|████████▊ | 198755/225000 [378:50:30<70:13:07,  9.63s/it]
 88%|████████▊ | 198756/225000 [378:50:36<61:54:03,  8.49s/it]
 88%|████████▊ | 198757/225000 [378:50:43<59:31:35,  8.17s/it]
 88%|████████▊ | 198758/225000 [378:50:52<61:45:38,  8.47s/it]
 88%|████████▊ | 198759/225000 [378:50:58<54:44:28,  7.51s/it]
 88%|████████▊ | 198760/225000 [378:51:06<57:01:24,  7.82s/it]
 88%|████████▊ | 198761/225000 [378:51:20<69:36:05,  9.55s/it]
 88%|████████▊ | 198762/225000 [378:51:25<60:52:29,  8.35s/it]
 88%|████████▊ | 198763/225000 [378:51:32<58:19:15,  8.00s/it]
 88%|████████▊ | 198764/225000 [378:51:42<62:12:02,  8.53s/it]
 88%|████████▊ | 198765/225000 [378:51:54<69:23:45,  9.52s/it]
 88%|████████▊ | 198766/225000 [378:52:02<65:57:51,  9.05s/it]
 88%|████████▊ | 198767/225000 [378:52:14<73:11:24, 10.04s/it]
 88%|████████▊ | 198768/225000 [378:52:22<67:40:44,  9.29s/it]
 88%|████████▊ | 198769/225000 [378:52:30<64:08:40,  8.80s/it]
 88%|████████▊ | 198770/225000 [378:52:36<59:29:59,  8.17s/it]
 88%|████████▊ | 198771/225000 [378:52:44<58:02:57,  7.97s/it]
 88%|████████▊ | 198772/225000 [378:52:51<56:55:38,  7.81s/it]
 88%|████████▊ | 198773/225000 [378:53:01<60:19:50,  8.28s/it]
 88%|████████▊ | 198774/225000 [378:53:07<57:08:19,  7.84s/it]
 88%|████████▊ | 198775/225000 [378:53:16<57:51:04,  7.94s/it]
 88%|████████▊ | 198776/225000 [378:53:23<57:36:46,  7.91s/it]
 88%|████████▊ | 198777/225000 [378:53:31<57:04:52,  7.84s/it]
 88%|████████▊ | 198778/225000 [378:53:37<52:47:44,  7.25s/it]
 88%|████████▊ | 198779/225000 [378:53:48<62:05:14,  8.52s/it]
 88%|████████▊ | 198780/225000 [378:53:59<67:24:13,  9.25s/it]
 88%|████████▊ | 198781/225000 [378:54:06<61:41:01,  8.47s/it]
 88%|████████▊ | 198782/225000 [378:54:18<68:37:15,  9.42s/it]
 88%|████████▊ | 198783/225000 [378:54:25<64:48:53,  8.90s/it]
 88%|████████▊ | 198784/225000 [378:54:38<72:43:14,  9.99s/it]
 88%|████████▊ | 198785/225000 [378:54:43<62:08:05,  8.53s/it]
 88%|████████▊ | 198786/225000 [378:54:56<71:14:57,  9.78s/it]
 88%|████████▊ | 198787/225000 [378:55:06<72:39:18,  9.98s/it]
 88%|████████▊ | 198788/225000 [378:55:17<74:27:53, 10.23s/it]
 88%|████████▊ | 198789/225000 [378:55:24<68:08:15,  9.36s/it]
 88%|████████▊ | 198790/225000 [378:55:34<68:15:11,  9.37s/it]
 88%|████████▊ | 198791/225000 [378:55:44<69:59:58,  9.61s/it]
 88%|████████▊ | 198792/225000 [378:55:47<56:14:15,  7.72s/it]
 88%|████████▊ | 198793/225000 [378:55:58<62:48:33,  8.63s/it]
 88%|████████▊ | 198794/225000 [378:56:05<59:02:16,  8.11s/it]
 88%|████████▊ | 198795/225000 [378:56:12<56:33:25,  7.77s/it]
 88%|████████▊ | 198796/225000 [378:56:17<50:34:57,  6.95s/it]
 88%|████████▊ | 198797/225000 [378:56:23<48:32:33,  6.67s/it]
 88%|████████▊ | 198798/225000 [378:56:30<48:31:45,  6.67s/it]
 88%|████████▊ | 198799/225000 [378:56:42<60:48:43,  8.36s/it]
 88%|████████▊ | 198800/225000 [378:56:49<57:34:06,  7.91s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.157', 'learning_rate': '1.822e-06', 'epoch': '0.8836', 'num_input_tokens_seen': 634606432, 'train_runtime': '1.364e+06', 'train_tokens_per_second': '465.2'}

 88%|████████▊ | 198800/225000 [378:56:49<57:34:06,  7.91s/it]
 88%|████████▊ | 198801/225000 [378:56:57<57:22:58,  7.88s/it]
 88%|████████▊ | 198802/225000 [378:57:09<67:46:43,  9.31s/it]
 88%|████████▊ | 198803/225000 [378:57:18<66:18:15,  9.11s/it]
 88%|████████▊ | 198804/225000 [378:57:24<60:27:20,  8.31s/it]
 88%|████████▊ | 198805/225000 [378:57:35<66:09:31,  9.09s/it]
 88%|████████▊ | 198806/225000 [378:57:44<65:05:00,  8.94s/it]
 88%|████████▊ | 198807/225000 [378:57:51<60:20:36,  8.29s/it]
 88%|████████▊ | 198808/225000 [378:57:59<60:53:16,  8.37s/it]
 88%|████████▊ | 198809/225000 [378:58:05<55:56:24,  7.69s/it]
 88%|████████▊ | 198810/225000 [378:58:11<52:32:22,  7.22s/it]
 88%|████████▊ | 198811/225000 [378:58:18<51:56:24,  7.14s/it]
 88%|████████▊ | 198812/225000 [378:58:25<50:38:49,  6.96s/it]
 88%|████████▊ | 198813/225000 [378:58:34<55:10:31,  7.59s/it]
 88%|████████▊ | 198814/225000 [378:58:42<56:23:33,  7.75s/it]
 88%|████████▊ | 198815/225000 [378:58:52<60:48:45,  8.36s/it]
 88%|████████▊ | 198816/225000 [378:59:04<69:11:59,  9.51s/it]
 88%|████████▊ | 198817/225000 [378:59:15<72:41:36,  9.99s/it]
 88%|████████▊ | 198818/225000 [378:59:23<68:39:19,  9.44s/it]
 88%|████████▊ | 198819/225000 [378:59:29<59:46:37,  8.22s/it]
 88%|████████▊ | 198820/225000 [378:59:34<53:04:14,  7.30s/it]
 88%|████████▊ | 198821/225000 [378:59:46<63:59:19,  8.80s/it]
 88%|████████▊ | 198822/225000 [378:59:51<56:02:54,  7.71s/it]
 88%|████████▊ | 198823/225000 [378:59:59<55:35:47,  7.65s/it]
 88%|████████▊ | 198824/225000 [379:00:12<66:51:21,  9.19s/it]
 88%|████████▊ | 198825/225000 [379:00:18<61:28:20,  8.45s/it]
 88%|████████▊ | 198826/225000 [379:00:24<55:33:25,  7.64s/it]
 88%|████████▊ | 198827/225000 [379:00:36<65:14:30,  8.97s/it]
 88%|████████▊ | 198828/225000 [379:00:41<56:33:41,  7.78s/it]
 88%|████████▊ | 198829/225000 [379:00:53<66:10:00,  9.10s/it]
 88%|████████▊ | 198830/225000 [379:01:03<67:06:57,  9.23s/it]
 88%|████████▊ | 198831/225000 [379:01:11<64:21:53,  8.85s/it]
 88%|████████▊ | 198832/225000 [379:01:18<61:18:57,  8.44s/it]
 88%|████████▊ | 198833/225000 [379:01:26<58:56:11,  8.11s/it]
 88%|████████▊ | 198834/225000 [379:01:32<54:45:37,  7.53s/it]
 88%|████████▊ | 198835/225000 [379:01:45<66:29:35,  9.15s/it]
 88%|████████▊ | 198836/225000 [379:01:54<65:55:58,  9.07s/it]
 88%|████████▊ | 198837/225000 [379:02:00<59:22:42,  8.17s/it]
 88%|████████▊ | 198838/225000 [379:02:13<69:51:28,  9.61s/it]
 88%|████████▊ | 198839/225000 [379:02:20<65:36:30,  9.03s/it]
 88%|████████▊ | 198840/225000 [379:02:29<65:16:34,  8.98s/it]
 88%|████████▊ | 198841/225000 [379:02:35<57:45:36,  7.95s/it]
 88%|████████▊ | 198842/225000 [379:02:40<51:42:47,  7.12s/it]
 88%|████████▊ | 198843/225000 [379:02:55<70:02:53,  9.64s/it]
 88%|████████▊ | 198844/225000 [379:03:02<63:58:48,  8.81s/it]
 88%|████████▊ | 198845/225000 [379:03:12<65:40:20,  9.04s/it]
 88%|████████▊ | 198846/225000 [379:03:18<59:26:34,  8.18s/it]
 88%|████████▊ | 198847/225000 [379:03:29<66:30:02,  9.15s/it]
 88%|████████▊ | 198848/225000 [379:03:36<61:42:53,  8.50s/it]
 88%|████████▊ | 198849/225000 [379:03:44<60:44:09,  8.36s/it]
 88%|████████▊ | 198850/225000 [379:03:52<59:37:32,  8.21s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.064', 'learning_rate': '1.816e-06', 'epoch': '0.8838', 'num_input_tokens_seen': 634821920, 'train_runtime': '1.365e+06', 'train_tokens_per_second': '465.2'}

 88%|████████▊ | 198850/225000 [379:03:52<59:37:32,  8.21s/it]
 88%|████████▊ | 198851/225000 [379:04:03<64:36:33,  8.89s/it]
 88%|████████▊ | 198852/225000 [379:04:14<70:30:22,  9.71s/it]
 88%|████████▊ | 198853/225000 [379:04:26<74:48:43, 10.30s/it]
 88%|████████▊ | 198854/225000 [379:04:30<61:50:02,  8.51s/it]
 88%|████████▊ | 198855/225000 [379:04:39<62:47:59,  8.65s/it]
 88%|████████▊ | 198856/225000 [379:04:48<62:31:32,  8.61s/it]
 88%|████████▊ | 198857/225000 [379:04:56<61:08:31,  8.42s/it]
 88%|████████▊ | 198858/225000 [379:05:02<55:44:46,  7.68s/it]
 88%|████████▊ | 198859/225000 [379:05:08<53:08:05,  7.32s/it]
 88%|████████▊ | 198860/225000 [379:05:15<50:55:38,  7.01s/it]
 88%|████████▊ | 198861/225000 [379:05:21<48:32:23,  6.69s/it]
 88%|████████▊ | 198862/225000 [379:05:28<50:00:25,  6.89s/it]
 88%|████████▊ | 198863/225000 [379:05:36<52:19:41,  7.21s/it]
 88%|████████▊ | 198864/225000 [379:05:47<61:37:11,  8.49s/it]
 88%|████████▊ | 198865/225000 [379:05:58<65:29:08,  9.02s/it]
 88%|████████▊ | 198866/225000 [379:06:04<60:34:39,  8.34s/it]
 88%|████████▊ | 198867/225000 [379:06:10<54:57:35,  7.57s/it]
 88%|████████▊ | 198868/225000 [379:06:20<60:48:10,  8.38s/it]
 88%|████████▊ | 198869/225000 [379:06:31<65:35:44,  9.04s/it]
 88%|████████▊ | 198870/225000 [379:06:39<63:56:22,  8.81s/it]
 88%|████████▊ | 198871/225000 [379:06:50<67:58:47,  9.37s/it]
 88%|████████▊ | 198872/225000 [379:06:54<55:44:05,  7.68s/it]
 88%|████████▊ | 198873/225000 [379:07:06<65:06:39,  8.97s/it]
 88%|████████▊ | 198874/225000 [379:07:19<74:29:01, 10.26s/it]
 88%|████████▊ | 198875/225000 [379:07:32<79:50:12, 11.00s/it]
 88%|████████▊ | 198876/225000 [379:07:39<72:46:04, 10.03s/it]
 88%|████████▊ | 198877/225000 [379:07:53<79:36:38, 10.97s/it]
 88%|████████▊ | 198878/225000 [379:07:57<65:59:18,  9.09s/it]
 88%|████████▊ | 198879/225000 [379:08:11<75:15:52, 10.37s/it]
 88%|████████▊ | 198880/225000 [379:08:19<70:25:24,  9.71s/it]
 88%|████████▊ | 198881/225000 [379:08:28<68:23:04,  9.43s/it]
 88%|████████▊ | 198882/225000 [379:08:39<73:31:02, 10.13s/it]
 88%|████████▊ | 198883/225000 [379:08:48<69:16:43,  9.55s/it]
 88%|████████▊ | 198884/225000 [379:08:53<60:31:44,  8.34s/it]
 88%|████████▊ | 198885/225000 [379:08:58<53:20:28,  7.35s/it]
 88%|████████▊ | 198886/225000 [379:09:05<51:51:23,  7.15s/it]
 88%|████████▊ | 198887/225000 [379:09:11<49:01:47,  6.76s/it]
 88%|████████▊ | 198888/225000 [379:09:25<66:27:11,  9.16s/it]
 88%|████████▊ | 198889/225000 [379:09:34<64:48:51,  8.94s/it]
 88%|████████▊ | 198890/225000 [379:09:44<67:17:40,  9.28s/it]
 88%|████████▊ | 198891/225000 [379:09:50<59:24:35,  8.19s/it]
 88%|████████▊ | 198892/225000 [379:09:56<55:22:36,  7.64s/it]
 88%|████████▊ | 198893/225000 [379:10:03<54:35:27,  7.53s/it]
 88%|████████▊ | 198894/225000 [379:10:12<57:05:04,  7.87s/it]
 88%|████████▊ | 198895/225000 [379:10:19<54:44:32,  7.55s/it]
 88%|████████▊ | 198896/225000 [379:10:34<71:09:26,  9.81s/it]
 88%|████████▊ | 198897/225000 [379:10:42<68:44:19,  9.48s/it]
 88%|████████▊ | 198898/225000 [379:10:50<63:53:05,  8.81s/it]
 88%|████████▊ | 198899/225000 [379:11:01<69:06:52,  9.53s/it]
 88%|████████▊ | 198900/225000 [379:11:07<62:20:12,  8.60s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.137', 'learning_rate': '1.809e-06', 'epoch': '0.884', 'num_input_tokens_seen': 635052936, 'train_runtime': '1.365e+06', 'train_tokens_per_second': '465.2'}

 88%|████████▊ | 198900/225000 [379:11:07<62:20:12,  8.60s/it]
 88%|████████▊ | 198901/225000 [379:11:18<67:48:05,  9.35s/it]
 88%|████████▊ | 198902/225000 [379:11:25<62:35:20,  8.63s/it]
 88%|████████▊ | 198903/225000 [379:11:36<66:44:09,  9.21s/it]
 88%|████████▊ | 198904/225000 [379:11:43<61:14:51,  8.45s/it]
 88%|████████▊ | 198905/225000 [379:11:51<60:25:59,  8.34s/it]
 88%|████████▊ | 198906/225000 [379:12:03<68:11:09,  9.41s/it]
 88%|████████▊ | 198907/225000 [379:12:10<63:42:03,  8.79s/it]
 88%|████████▊ | 198908/225000 [379:12:21<68:52:00,  9.50s/it]
 88%|████████▊ | 198909/225000 [379:12:26<58:53:41,  8.13s/it]
 88%|████████▊ | 198910/225000 [379:12:38<68:18:30,  9.43s/it]
 88%|████████▊ | 198911/225000 [379:12:52<77:03:43, 10.63s/it]
 88%|████████▊ | 198912/225000 [379:12:57<64:26:14,  8.89s/it]
 88%|████████▊ | 198913/225000 [379:13:03<57:38:52,  7.96s/it]
 88%|████████▊ | 198914/225000 [379:13:15<67:55:53,  9.37s/it]
 88%|████████▊ | 198915/225000 [379:13:21<60:19:10,  8.32s/it]
 88%|████████▊ | 198916/225000 [379:13:28<58:10:10,  8.03s/it]
 88%|████████▊ | 198917/225000 [379:13:37<59:37:04,  8.23s/it]
 88%|████████▊ | 198918/225000 [379:13:42<51:53:24,  7.16s/it]
 88%|████████▊ | 198919/225000 [379:13:48<50:27:32,  6.96s/it]
 88%|████████▊ | 198920/225000 [379:13:57<53:13:46,  7.35s/it]
 88%|████████▊ | 198921/225000 [379:14:10<65:59:57,  9.11s/it]
 88%|████████▊ | 198922/225000 [379:14:19<66:23:22,  9.16s/it]
 88%|████████▊ | 198923/225000 [379:14:29<69:04:31,  9.54s/it]
 88%|████████▊ | 198924/225000 [379:14:40<71:53:01,  9.92s/it]
 88%|████████▊ | 198925/225000 [379:14:52<75:03:42, 10.36s/it]
 88%|████████▊ | 198926/225000 [379:14:58<66:10:35,  9.14s/it]
 88%|████████▊ | 198927/225000 [379:15:07<66:46:50,  9.22s/it]
 88%|████████▊ | 198928/225000 [379:15:19<70:58:39,  9.80s/it]
 88%|████████▊ | 198929/225000 [379:15:24<62:06:35,  8.58s/it]
 88%|████████▊ | 198930/225000 [379:15:34<65:01:17,  8.98s/it]
 88%|████████▊ | 198931/225000 [379:15:42<62:10:26,  8.59s/it]
 88%|████████▊ | 198932/225000 [379:15:48<56:30:51,  7.80s/it]
 88%|████████▊ | 198933/225000 [379:15:56<56:59:01,  7.87s/it]
 88%|████████▊ | 198934/225000 [379:16:09<68:30:25,  9.46s/it]
 88%|████████▊ | 198935/225000 [379:16:15<60:27:20,  8.35s/it]
 88%|████████▊ | 198936/225000 [379:16:20<54:40:20,  7.55s/it]
 88%|████████▊ | 198937/225000 [379:16:32<64:07:59,  8.86s/it]
 88%|████████▊ | 198938/225000 [379:16:44<71:12:08,  9.84s/it]
 88%|████████▊ | 198939/225000 [379:16:51<64:14:15,  8.87s/it]
 88%|████████▊ | 198940/225000 [379:17:01<66:00:28,  9.12s/it]
 88%|████████▊ | 198941/225000 [379:17:10<65:45:43,  9.08s/it]
 88%|████████▊ | 198942/225000 [379:17:14<54:47:05,  7.57s/it]
 88%|████████▊ | 198943/225000 [379:17:25<63:14:25,  8.74s/it]
 88%|████████▊ | 198944/225000 [379:17:35<65:24:07,  9.04s/it]
 88%|████████▊ | 198945/225000 [379:17:41<58:23:46,  8.07s/it]
 88%|████████▊ | 198946/225000 [379:17:51<62:55:35,  8.69s/it]
 88%|████████▊ | 198947/225000 [379:18:06<77:08:53, 10.66s/it]
 88%|████████▊ | 198948/225000 [379:18:14<69:51:12,  9.65s/it]
 88%|████████▊ | 198949/225000 [379:18:24<72:23:07, 10.00s/it]
 88%|████████▊ | 198950/225000 [379:18:30<63:14:12,  8.74s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.439', 'learning_rate': '1.802e-06', 'epoch': '0.8842', 'num_input_tokens_seen': 635289632, 'train_runtime': '1.366e+06', 'train_tokens_per_second': '465.2'}

 88%|████████▊ | 198950/225000 [379:18:30<63:14:12,  8.74s/it]
 88%|████████▊ | 198951/225000 [379:18:36<57:01:44,  7.88s/it]
 88%|████████▊ | 198952/225000 [379:18:47<63:05:22,  8.72s/it]
 88%|████████▊ | 198953/225000 [379:18:57<66:28:41,  9.19s/it]
 88%|████████▊ | 198954/225000 [379:19:05<63:50:44,  8.82s/it]
 88%|████████▊ | 198955/225000 [379:19:14<63:27:49,  8.77s/it]
 88%|████████▊ | 198956/225000 [379:19:21<61:01:18,  8.43s/it]
 88%|████████▊ | 198957/225000 [379:19:32<65:22:49,  9.04s/it]
 88%|████████▊ | 198958/225000 [379:19:42<67:07:23,  9.28s/it]
 88%|████████▊ | 198959/225000 [379:19:48<61:51:33,  8.55s/it]
 88%|████████▊ | 198960/225000 [379:19:56<59:18:56,  8.20s/it]
 88%|████████▊ | 198961/225000 [379:20:02<54:22:29,  7.52s/it]
 88%|████████▊ | 198962/225000 [379:20:08<51:14:01,  7.08s/it]
 88%|████████▊ | 198963/225000 [379:20:18<58:42:33,  8.12s/it]
 88%|████████▊ | 198964/225000 [379:20:24<53:40:05,  7.42s/it]
 88%|████████▊ | 198965/225000 [379:20:30<50:46:59,  7.02s/it]
 88%|████████▊ | 198966/225000 [379:20:37<51:19:25,  7.10s/it]
 88%|████████▊ | 198967/225000 [379:20:46<55:31:50,  7.68s/it]
 88%|████████▊ | 198968/225000 [379:21:00<67:14:55,  9.30s/it]
 88%|████████▊ | 198969/225000 [379:21:07<62:50:41,  8.69s/it]
 88%|████████▊ | 198970/225000 [379:21:19<71:16:22,  9.86s/it]
 88%|████████▊ | 198971/225000 [379:21:24<59:05:59,  8.17s/it]
 88%|████████▊ | 198972/225000 [379:21:32<59:46:20,  8.27s/it]
 88%|████████▊ | 198973/225000 [379:21:42<63:35:43,  8.80s/it]
 88%|████████▊ | 198974/225000 [379:21:51<63:49:53,  8.83s/it]
 88%|████████▊ | 198975/225000 [379:21:59<60:55:16,  8.43s/it]
 88%|████████▊ | 198976/225000 [379:22:03<52:54:45,  7.32s/it]
 88%|████████▊ | 198977/225000 [379:22:11<52:52:59,  7.32s/it]
 88%|████████▊ | 198978/225000 [379:22:24<66:37:58,  9.22s/it]
 88%|████████▊ | 198979/225000 [379:22:34<66:38:47,  9.22s/it]
 88%|████████▊ | 198980/225000 [379:22:39<59:26:17,  8.22s/it]
 88%|████████▊ | 198981/225000 [379:22:49<63:11:15,  8.74s/it]
 88%|████████▊ | 198982/225000 [379:22:55<57:13:44,  7.92s/it]
 88%|████████▊ | 198983/225000 [379:23:03<55:41:09,  7.71s/it]
 88%|████████▊ | 198984/225000 [379:23:15<64:59:37,  8.99s/it]
 88%|████████▊ | 198985/225000 [379:23:24<66:25:15,  9.19s/it]
 88%|████████▊ | 198986/225000 [379:23:31<61:22:32,  8.49s/it]
 88%|████████▊ | 198987/225000 [379:23:37<56:23:08,  7.80s/it]
 88%|████████▊ | 198988/225000 [379:23:44<54:35:31,  7.56s/it]
 88%|████████▊ | 198989/225000 [379:23:54<58:56:12,  8.16s/it]
 88%|████████▊ | 198990/225000 [379:24:04<62:29:36,  8.65s/it]
 88%|████████▊ | 198991/225000 [379:24:11<59:50:57,  8.28s/it]
 88%|████████▊ | 198992/225000 [379:24:22<65:59:52,  9.14s/it]
 88%|████████▊ | 198993/225000 [379:24:31<64:26:33,  8.92s/it]
 88%|████████▊ | 198994/225000 [379:24:42<69:41:08,  9.65s/it]
 88%|████████▊ | 198995/225000 [379:24:46<58:23:18,  8.08s/it]
 88%|████████▊ | 198996/225000 [379:24:59<68:13:30,  9.45s/it]
 88%|████████▊ | 198997/225000 [379:25:05<60:22:33,  8.36s/it]
 88%|████████▊ | 198998/225000 [379:25:12<56:56:04,  7.88s/it]
 88%|████████▊ | 198999/225000 [379:25:21<59:39:03,  8.26s/it]
 88%|████████▊ | 199000/225000 [379:25:30<60:58:57,  8.44s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.397', 'learning_rate': '1.795e-06', 'epoch': '0.8844', 'num_input_tokens_seen': 635509088, 'train_runtime': '1.366e+06', 'train_tokens_per_second': '465.3'}

 88%|████████▊ | 199000/225000 [379:25:30<60:58:57,  8.44s/it]
 88%|████████▊ | 199001/225000 [379:25:37<59:42:08,  8.27s/it]
 88%|████████▊ | 199002/225000 [379:25:51<70:33:26,  9.77s/it]
 88%|████████▊ | 199003/225000 [379:25:56<60:08:54,  8.33s/it]
 88%|████████▊ | 199004/225000 [379:26:06<65:04:16,  9.01s/it]
 88%|████████▊ | 199005/225000 [379:26:11<56:39:35,  7.85s/it]
 88%|████████▊ | 199006/225000 [379:26:20<58:55:34,  8.16s/it]
 88%|████████▊ | 199007/225000 [379:26:28<58:10:43,  8.06s/it]
 88%|████████▊ | 199008/225000 [379:26:35<56:30:01,  7.83s/it]
 88%|████████▊ | 199009/225000 [379:26:43<57:00:44,  7.90s/it]
 88%|████████▊ | 199010/225000 [379:26:50<55:03:46,  7.63s/it]
 88%|████████▊ | 199011/225000 [379:26:59<56:02:57,  7.76s/it]
 88%|████████▊ | 199012/225000 [379:27:07<56:54:17,  7.88s/it]
 88%|████████▊ | 199013/225000 [379:27:17<61:44:26,  8.55s/it]
 88%|████████▊ | 199014/225000 [379:27:26<62:07:15,  8.61s/it]
 88%|████████▊ | 199015/225000 [379:27:32<56:29:04,  7.83s/it]
 88%|████████▊ | 199016/225000 [379:27:44<65:28:31,  9.07s/it]
 88%|████████▊ | 199017/225000 [379:27:50<60:09:47,  8.34s/it]
 88%|████████▊ | 199018/225000 [379:27:58<59:20:05,  8.22s/it]
 88%|████████▊ | 199019/225000 [379:28:03<52:59:19,  7.34s/it]
 88%|████████▊ | 199020/225000 [379:28:11<52:34:15,  7.28s/it]
 88%|████████▊ | 199021/225000 [379:28:17<50:33:50,  7.01s/it]
 88%|████████▊ | 199022/225000 [379:28:23<49:14:02,  6.82s/it]
 88%|████████▊ | 199023/225000 [379:28:30<49:05:01,  6.80s/it]
 88%|████████▊ | 199024/225000 [379:28:46<68:26:41,  9.49s/it]
 88%|████████▊ | 199025/225000 [379:28:53<63:33:37,  8.81s/it]
 88%|████████▊ | 199026/225000 [379:28:58<55:34:03,  7.70s/it]
 88%|████████▊ | 199027/225000 [379:29:07<57:48:46,  8.01s/it]
 88%|████████▊ | 199028/225000 [379:29:18<63:36:04,  8.82s/it]
 88%|████████▊ | 199029/225000 [379:29:28<66:00:23,  9.15s/it]
 88%|████████▊ | 199030/225000 [379:29:34<61:06:53,  8.47s/it]
 88%|████████▊ | 199031/225000 [379:29:43<61:39:28,  8.55s/it]
 88%|████████▊ | 199032/225000 [379:29:56<70:38:31,  9.79s/it]
 88%|████████▊ | 199033/225000 [379:30:00<58:26:33,  8.10s/it]
 88%|████████▊ | 199034/225000 [379:30:12<67:15:31,  9.32s/it]
 88%|████████▊ | 199035/225000 [379:30:18<58:59:26,  8.18s/it]
 88%|████████▊ | 199036/225000 [379:30:25<56:29:41,  7.83s/it]
 88%|████████▊ | 199037/225000 [379:30:30<51:10:53,  7.10s/it]
 88%|████████▊ | 199038/225000 [379:30:42<61:20:52,  8.51s/it]
 88%|████████▊ | 199039/225000 [379:30:50<61:22:25,  8.51s/it]
 88%|████████▊ | 199040/225000 [379:30:58<58:23:21,  8.10s/it]
 88%|████████▊ | 199041/225000 [379:31:08<64:23:39,  8.93s/it]
 88%|████████▊ | 199042/225000 [379:31:14<57:58:37,  8.04s/it]
 88%|████████▊ | 199043/225000 [379:31:24<62:05:08,  8.61s/it]
 88%|████████▊ | 199044/225000 [379:31:32<60:48:35,  8.43s/it]
 88%|████████▊ | 199045/225000 [379:31:41<61:47:41,  8.57s/it]
 88%|████████▊ | 199046/225000 [379:31:46<54:21:37,  7.54s/it]
 88%|████████▊ | 199047/225000 [379:31:56<59:35:14,  8.27s/it]
 88%|████████▊ | 199048/225000 [379:32:05<61:08:05,  8.48s/it]
 88%|████████▊ | 199049/225000 [379:32:12<57:22:43,  7.96s/it]
 88%|████████▊ | 199050/225000 [379:32:26<70:56:01,  9.84s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.014', 'learning_rate': '1.788e-06', 'epoch': '0.8847', 'num_input_tokens_seen': 635738832, 'train_runtime': '1.366e+06', 'train_tokens_per_second': '465.3'}

 88%|████████▊ | 199050/225000 [379:32:26<70:56:01,  9.84s/it]
 88%|████████▊ | 199051/225000 [379:32:34<66:00:13,  9.16s/it]
 88%|████████▊ | 199052/225000 [379:32:42<64:20:07,  8.93s/it]
 88%|████████▊ | 199053/225000 [379:32:52<67:14:44,  9.33s/it]
 88%|████████▊ | 199054/225000 [379:33:00<63:06:02,  8.76s/it]
 88%|████████▊ | 199055/225000 [379:33:05<55:13:28,  7.66s/it]
 88%|████████▊ | 199056/225000 [379:33:11<52:37:23,  7.30s/it]
 88%|████████▊ | 199057/225000 [379:33:23<62:11:43,  8.63s/it]
 88%|████████▊ | 199058/225000 [379:33:31<59:43:31,  8.29s/it]
 88%|████████▊ | 199059/225000 [379:33:40<61:57:52,  8.60s/it]
 88%|████████▊ | 199060/225000 [379:33:47<59:00:20,  8.19s/it]
 88%|████████▊ | 199061/225000 [379:33:54<56:50:06,  7.89s/it]
 88%|████████▊ | 199062/225000 [379:34:08<68:24:21,  9.49s/it]
 88%|████████▊ | 199063/225000 [379:34:15<63:47:33,  8.85s/it]
 88%|████████▊ | 199064/225000 [379:34:27<69:32:04,  9.65s/it]
 88%|████████▊ | 199065/225000 [379:34:33<62:16:30,  8.64s/it]
 88%|████████▊ | 199066/225000 [379:34:39<57:49:57,  8.03s/it]
 88%|████████▊ | 199067/225000 [379:34:46<54:34:47,  7.58s/it]
 88%|████████▊ | 199068/225000 [379:35:02<71:57:05,  9.99s/it]
 88%|████████▊ | 199069/225000 [379:35:07<62:04:10,  8.62s/it]
 88%|████████▊ | 199070/225000 [379:35:19<68:25:32,  9.50s/it]
 88%|████████▊ | 199071/225000 [379:35:25<61:14:18,  8.50s/it]
 88%|████████▊ | 199072/225000 [379:35:32<58:39:03,  8.14s/it]
 88%|████████▊ | 199073/225000 [379:35:39<55:57:15,  7.77s/it]
 88%|████████▊ | 199074/225000 [379:35:54<72:13:21, 10.03s/it]
 88%|████████▊ | 199075/225000 [379:36:00<63:38:21,  8.84s/it]
 88%|████████▊ | 199076/225000 [379:36:07<59:39:52,  8.29s/it]
 88%|████████▊ | 199077/225000 [379:36:18<65:56:30,  9.16s/it]
 88%|████████▊ | 199078/225000 [379:36:31<73:23:55, 10.19s/it]
 88%|████████▊ | 199079/225000 [379:36:37<63:10:52,  8.77s/it]
 88%|████████▊ | 199080/225000 [379:36:43<57:18:42,  7.96s/it]
 88%|████████▊ | 199081/225000 [379:36:48<51:56:54,  7.22s/it]
 88%|████████▊ | 199082/225000 [379:37:01<64:10:20,  8.91s/it]
 88%|████████▊ | 199083/225000 [379:37:11<66:31:18,  9.24s/it]
 88%|████████▊ | 199084/225000 [379:37:17<58:38:34,  8.15s/it]
 88%|████████▊ | 199085/225000 [379:37:23<55:45:19,  7.75s/it]
 88%|████████▊ | 199086/225000 [379:37:27<47:13:06,  6.56s/it]
 88%|████████▊ | 199087/225000 [379:37:34<47:02:52,  6.54s/it]
 88%|████████▊ | 199088/225000 [379:37:48<64:49:07,  9.01s/it]
 88%|████████▊ | 199089/225000 [379:37:55<58:45:38,  8.16s/it]
 88%|████████▊ | 199090/225000 [379:38:07<68:14:12,  9.48s/it]
 88%|████████▊ | 199091/225000 [379:38:14<62:56:30,  8.75s/it]
 88%|████████▊ | 199092/225000 [379:38:21<58:52:07,  8.18s/it]
 88%|████████▊ | 199093/225000 [379:38:27<54:29:58,  7.57s/it]
 88%|████████▊ | 199094/225000 [379:38:33<50:25:13,  7.01s/it]
 88%|████████▊ | 199095/225000 [379:38:40<51:02:14,  7.09s/it]
 88%|████████▊ | 199096/225000 [379:38:54<66:36:38,  9.26s/it]
 88%|████████▊ | 199097/225000 [379:39:02<63:27:55,  8.82s/it]
 88%|████████▊ | 199098/225000 [379:39:08<57:00:46,  7.92s/it]
 88%|████████▊ | 199099/225000 [379:39:15<53:58:54,  7.50s/it]
 88%|████████▊ | 199100/225000 [379:39:28<66:21:42,  9.22s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.206', 'learning_rate': '1.782e-06', 'epoch': '0.8849', 'num_input_tokens_seen': 635966408, 'train_runtime': '1.367e+06', 'train_tokens_per_second': '465.3'}

 88%|████████▊ | 199100/225000 [379:39:28<66:21:42,  9.22s/it]
 88%|████████▊ | 199101/225000 [379:39:33<58:24:31,  8.12s/it]
 88%|████████▊ | 199102/225000 [379:39:43<60:56:09,  8.47s/it]
 88%|████████▊ | 199103/225000 [379:39:50<57:52:47,  8.05s/it]
 88%|████████▊ | 199104/225000 [379:39:59<59:26:34,  8.26s/it]
 88%|████████▊ | 199105/225000 [379:40:08<62:01:32,  8.62s/it]
 88%|████████▊ | 199106/225000 [379:40:17<62:18:42,  8.66s/it]
 88%|████████▊ | 199107/225000 [379:40:27<64:41:27,  8.99s/it]
 88%|████████▊ | 199108/225000 [379:40:33<58:49:44,  8.18s/it]
 88%|████████▊ | 199109/225000 [379:40:38<52:56:06,  7.36s/it]
 88%|████████▊ | 199110/225000 [379:40:44<50:24:21,  7.01s/it]
 88%|████████▊ | 199111/225000 [379:40:54<55:47:50,  7.76s/it]
 88%|████████▊ | 199112/225000 [379:41:06<64:54:22,  9.03s/it]
 88%|████████▊ | 199113/225000 [379:41:14<63:01:41,  8.77s/it]
 88%|████████▊ | 199114/225000 [379:41:24<66:31:09,  9.25s/it]
 88%|████████▊ | 199115/225000 [379:41:35<69:03:55,  9.61s/it]
 88%|████████▊ | 199116/225000 [379:41:40<59:23:53,  8.26s/it]
 88%|████████▊ | 199117/225000 [379:41:52<66:42:19,  9.28s/it]
 88%|████████▊ | 199118/225000 [379:41:59<62:22:36,  8.68s/it]
 88%|████████▊ | 199119/225000 [379:42:05<55:57:09,  7.78s/it]
 88%|████████▊ | 199120/225000 [379:42:20<71:56:44, 10.01s/it]
 88%|████████▊ | 199121/225000 [379:42:24<58:41:56,  8.17s/it]
 88%|████████▊ | 199122/225000 [379:42:36<66:39:21,  9.27s/it]
 88%|████████▊ | 199123/225000 [379:42:42<60:14:03,  8.38s/it]
 88%|████████▊ | 199124/225000 [379:42:53<65:41:56,  9.14s/it]
 88%|████████▊ | 199125/225000 [379:43:02<66:38:39,  9.27s/it]
 89%|████████▊ | 199126/225000 [379:43:10<63:13:21,  8.80s/it]
 89%|████████▊ | 199127/225000 [379:43:20<65:18:11,  9.09s/it]
 89%|████████▊ | 199128/225000 [379:43:31<70:01:48,  9.74s/it]
 89%|████████▊ | 199129/225000 [379:43:38<63:35:01,  8.85s/it]
 89%|████████▊ | 199130/225000 [379:43:43<56:19:23,  7.84s/it]
 89%|████████▊ | 199131/225000 [379:43:51<56:44:36,  7.90s/it]
 89%|████████▊ | 199132/225000 [379:43:57<52:14:24,  7.27s/it]
 89%|████████▊ | 199133/225000 [379:44:06<55:43:32,  7.76s/it]
 89%|████████▊ | 199134/225000 [379:44:17<61:41:46,  8.59s/it]
 89%|████████▊ | 199135/225000 [379:44:22<55:06:22,  7.67s/it]
 89%|████████▊ | 199136/225000 [379:44:28<52:13:06,  7.27s/it]
 89%|████████▊ | 199137/225000 [379:44:42<65:38:50,  9.14s/it]
 89%|████████▊ | 199138/225000 [379:44:50<64:11:15,  8.93s/it]
 89%|████████▊ | 199139/225000 [379:44:58<60:25:23,  8.41s/it]
 89%|████████▊ | 199140/225000 [379:45:05<58:21:11,  8.12s/it]
 89%|████████▊ | 199141/225000 [379:45:12<56:17:26,  7.84s/it]
 89%|████████▊ | 199142/225000 [379:45:21<57:29:08,  8.00s/it]
 89%|████████▊ | 199143/225000 [379:45:32<65:24:21,  9.11s/it]
 89%|████████▊ | 199144/225000 [379:45:39<59:18:18,  8.26s/it]
 89%|████████▊ | 199145/225000 [379:45:47<59:25:51,  8.28s/it]
 89%|████████▊ | 199146/225000 [379:45:53<53:49:46,  7.50s/it]
 89%|████████▊ | 199147/225000 [379:46:02<57:39:23,  8.03s/it]
 89%|████████▊ | 199148/225000 [379:46:13<64:11:48,  8.94s/it]
 89%|████████▊ | 199149/225000 [379:46:19<58:51:02,  8.20s/it]
 89%|████████▊ | 199150/225000 [379:46:26<56:01:03,  7.80s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.088', 'learning_rate': '1.775e-06', 'epoch': '0.8851', 'num_input_tokens_seen': 636200696, 'train_runtime': '1.367e+06', 'train_tokens_per_second': '465.3'}

 89%|████████▊ | 199150/225000 [379:46:26<56:01:03,  7.80s/it]
 89%|████████▊ | 199151/225000 [379:46:39<65:57:23,  9.19s/it]
 89%|████████▊ | 199152/225000 [379:46:43<56:33:34,  7.88s/it]
 89%|████████▊ | 199153/225000 [379:46:53<59:16:13,  8.26s/it]
 89%|████████▊ | 199154/225000 [379:47:00<58:24:00,  8.13s/it]
 89%|████████▊ | 199155/225000 [379:47:08<57:28:18,  8.01s/it]
 89%|████████▊ | 199156/225000 [379:47:19<64:30:27,  8.99s/it]
 89%|████████▊ | 199157/225000 [379:47:26<60:09:32,  8.38s/it]
 89%|████████▊ | 199158/225000 [379:47:38<66:54:49,  9.32s/it]
 89%|████████▊ | 199159/225000 [379:47:46<64:27:30,  8.98s/it]
 89%|████████▊ | 199160/225000 [379:47:52<58:27:42,  8.14s/it]
 89%|████████▊ | 199161/225000 [379:48:02<62:25:18,  8.70s/it]
 89%|████████▊ | 199162/225000 [379:48:08<56:18:24,  7.85s/it]
 89%|████████▊ | 199163/225000 [379:48:18<60:57:49,  8.49s/it]
 89%|████████▊ | 199164/225000 [379:48:26<58:28:40,  8.15s/it]
 89%|████████▊ | 199165/225000 [379:48:34<60:02:38,  8.37s/it]
 89%|████████▊ | 199166/225000 [379:48:45<65:27:42,  9.12s/it]
 89%|████████▊ | 199167/225000 [379:48:53<62:45:46,  8.75s/it]
 89%|████████▊ | 199168/225000 [379:49:01<60:48:00,  8.47s/it]
 89%|████████▊ | 199169/225000 [379:49:07<55:25:22,  7.72s/it]
 89%|████████▊ | 199170/225000 [379:49:21<69:28:48,  9.68s/it]
 89%|████████▊ | 199171/225000 [379:49:26<59:06:04,  8.24s/it]
 89%|████████▊ | 199172/225000 [379:49:39<68:12:04,  9.51s/it]
 89%|████████▊ | 199173/225000 [379:49:44<59:56:01,  8.35s/it]
 89%|████████▊ | 199174/225000 [379:49:52<59:36:15,  8.31s/it]
 89%|████████▊ | 199175/225000 [379:50:04<66:22:57,  9.25s/it]
 89%|████████▊ | 199176/225000 [379:50:11<61:27:01,  8.57s/it]
 89%|████████▊ | 199177/225000 [379:50:18<58:26:00,  8.15s/it]
 89%|████████▊ | 199178/225000 [379:50:25<55:00:56,  7.67s/it]
 89%|████████▊ | 199179/225000 [379:50:30<50:29:32,  7.04s/it]
 89%|████████▊ | 199180/225000 [379:50:43<63:59:04,  8.92s/it]
 89%|████████▊ | 199181/225000 [379:50:52<62:55:56,  8.77s/it]
 89%|████████▊ | 199182/225000 [379:50:58<56:29:50,  7.88s/it]
 89%|████████▊ | 199183/225000 [379:51:02<49:28:56,  6.90s/it]
 89%|████████▊ | 199184/225000 [379:51:15<62:26:20,  8.71s/it]
 89%|████████▊ | 199185/225000 [379:51:22<58:46:49,  8.20s/it]
 89%|████████▊ | 199186/225000 [379:51:31<59:21:01,  8.28s/it]
 89%|████████▊ | 199187/225000 [379:51:41<63:18:30,  8.83s/it]
 89%|████████▊ | 199188/225000 [379:51:51<65:33:30,  9.14s/it]
 89%|████████▊ | 199189/225000 [379:51:57<59:08:31,  8.25s/it]
 89%|████████▊ | 199190/225000 [379:52:04<56:34:40,  7.89s/it]
 89%|████████▊ | 199191/225000 [379:52:09<49:58:25,  6.97s/it]
 89%|████████▊ | 199192/225000 [379:52:21<60:40:40,  8.46s/it]
 89%|████████▊ | 199193/225000 [379:52:31<65:35:00,  9.15s/it]
 89%|████████▊ | 199194/225000 [379:52:38<59:11:09,  8.26s/it]
 89%|████████▊ | 199195/225000 [379:52:44<54:38:13,  7.62s/it]
 89%|████████▊ | 199196/225000 [379:52:53<58:22:51,  8.14s/it]
 89%|████████▊ | 199197/225000 [379:53:07<70:24:40,  9.82s/it]
 89%|████████▊ | 199198/225000 [379:53:13<63:09:59,  8.81s/it]
 89%|████████▊ | 199199/225000 [379:53:28<75:58:42, 10.60s/it]
 89%|████████▊ | 199200/225000 [379:53:33<63:49:28,  8.91s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.373', 'learning_rate': '1.768e-06', 'epoch': '0.8853', 'num_input_tokens_seen': 636428392, 'train_runtime': '1.368e+06', 'train_tokens_per_second': '465.4'}

 89%|████████▊ | 199200/225000 [379:53:33<63:49:28,  8.91s/it]
 89%|████████▊ | 199201/225000 [379:53:38<56:18:47,  7.86s/it]
 89%|████████▊ | 199202/225000 [379:53:52<68:15:12,  9.52s/it]
 89%|████████▊ | 199203/225000 [379:53:59<63:50:48,  8.91s/it]
 89%|████████▊ | 199204/225000 [379:54:08<62:59:40,  8.79s/it]
 89%|████████▊ | 199205/225000 [379:54:15<59:21:02,  8.28s/it]
 89%|████████▊ | 199206/225000 [379:54:25<62:19:10,  8.70s/it]
 89%|████████▊ | 199207/225000 [379:54:32<59:17:08,  8.27s/it]
 89%|████████▊ | 199208/225000 [379:54:46<71:54:38, 10.04s/it]
 89%|████████▊ | 199209/225000 [379:54:51<61:55:00,  8.64s/it]
 89%|████████▊ | 199210/225000 [379:54:57<54:50:32,  7.66s/it]
 89%|████████▊ | 199211/225000 [379:55:07<61:25:00,  8.57s/it]
 89%|████████▊ | 199212/225000 [379:55:15<58:22:10,  8.15s/it]
 89%|████████▊ | 199213/225000 [379:55:21<54:49:40,  7.65s/it]
 89%|████████▊ | 199214/225000 [379:55:34<66:17:37,  9.26s/it]
 89%|████████▊ | 199215/225000 [379:55:39<57:09:22,  7.98s/it]
 89%|████████▊ | 199216/225000 [379:55:44<51:21:09,  7.17s/it]
 89%|████████▊ | 199217/225000 [379:56:00<68:37:42,  9.58s/it]
 89%|████████▊ | 199218/225000 [379:56:09<68:57:47,  9.63s/it]
 89%|████████▊ | 199219/225000 [379:56:15<59:23:13,  8.29s/it]
 89%|████████▊ | 199220/225000 [379:56:21<55:29:31,  7.75s/it]
 89%|████████▊ | 199221/225000 [379:56:28<53:01:45,  7.41s/it]
 89%|████████▊ | 199222/225000 [379:56:34<51:30:48,  7.19s/it]
 89%|████████▊ | 199223/225000 [379:56:50<70:25:36,  9.84s/it]
 89%|████████▊ | 199224/225000 [379:57:03<75:57:57, 10.61s/it]
 89%|████████▊ | 199225/225000 [379:57:08<63:52:36,  8.92s/it]
 89%|████████▊ | 199226/225000 [379:57:19<69:45:30,  9.74s/it]
 89%|████████▊ | 199227/225000 [379:57:27<64:42:49,  9.04s/it]
 89%|████████▊ | 199228/225000 [379:57:36<64:39:39,  9.03s/it]
 89%|████████▊ | 199229/225000 [379:57:44<62:21:20,  8.71s/it]
 89%|████████▊ | 199230/225000 [379:57:53<63:43:14,  8.90s/it]
 89%|████████▊ | 199231/225000 [379:57:58<54:29:42,  7.61s/it]
 89%|████████▊ | 199232/225000 [379:58:06<55:21:18,  7.73s/it]
 89%|████████▊ | 199233/225000 [379:58:18<64:46:56,  9.05s/it]
 89%|████████▊ | 199234/225000 [379:58:22<53:30:21,  7.48s/it]
 89%|████████▊ | 199235/225000 [379:58:29<52:57:52,  7.40s/it]
 89%|████████▊ | 199236/225000 [379:58:40<61:46:50,  8.63s/it]
 89%|████████▊ | 199237/225000 [379:58:47<57:49:00,  8.08s/it]
 89%|████████▊ | 199238/225000 [379:58:56<59:29:43,  8.31s/it]
 89%|████████▊ | 199239/225000 [379:59:10<72:03:20, 10.07s/it]
 89%|████████▊ | 199240/225000 [379:59:19<70:19:29,  9.83s/it]
 89%|████████▊ | 199241/225000 [379:59:26<63:14:27,  8.84s/it]
 89%|████████▊ | 199242/225000 [379:59:35<64:06:16,  8.96s/it]
 89%|████████▊ | 199243/225000 [379:59:48<71:27:27,  9.99s/it]
 89%|████████▊ | 199244/225000 [379:59:53<60:40:14,  8.48s/it]
 89%|████████▊ | 199245/225000 [380:00:03<64:05:43,  8.96s/it]
 89%|████████▊ | 199246/225000 [380:00:10<60:25:44,  8.45s/it]
 89%|████████▊ | 199247/225000 [380:00:21<67:00:49,  9.37s/it]
 89%|████████▊ | 199248/225000 [380:00:30<65:42:58,  9.19s/it]
 89%|████████▊ | 199249/225000 [380:00:41<69:26:34,  9.71s/it]
 89%|████████▊ | 199250/225000 [380:00:49<66:24:55,  9.29s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.231', 'learning_rate': '1.761e-06', 'epoch': '0.8856', 'num_input_tokens_seen': 636657376, 'train_runtime': '1.368e+06', 'train_tokens_per_second': '465.4'}

 89%|████████▊ | 199250/225000 [380:00:49<66:24:55,  9.29s/it]
 89%|████████▊ | 199251/225000 [380:00:54<57:17:23,  8.01s/it]
 89%|████████▊ | 199252/225000 [380:01:00<51:04:03,  7.14s/it]
 89%|████████▊ | 199253/225000 [380:01:05<48:11:59,  6.74s/it]
 89%|████████▊ | 199254/225000 [380:01:11<46:29:49,  6.50s/it]
 89%|████████▊ | 199255/225000 [380:01:24<58:50:53,  8.23s/it]
 89%|████████▊ | 199256/225000 [380:01:32<59:01:43,  8.25s/it]
 89%|████████▊ | 199257/225000 [380:01:42<63:30:40,  8.88s/it]
 89%|████████▊ | 199258/225000 [380:01:49<59:44:25,  8.35s/it]
 89%|████████▊ | 199259/225000 [380:01:54<51:59:55,  7.27s/it]
 89%|████████▊ | 199260/225000 [380:02:10<69:48:51,  9.76s/it]
 89%|████████▊ | 199261/225000 [380:02:17<64:40:38,  9.05s/it]
 89%|████████▊ | 199262/225000 [380:02:24<60:23:25,  8.45s/it]
 89%|████████▊ | 199263/225000 [380:02:35<65:07:17,  9.11s/it]
 89%|████████▊ | 199264/225000 [380:02:46<70:09:51,  9.81s/it]
 89%|████████▊ | 199265/225000 [380:02:54<65:29:58,  9.16s/it]
 89%|████████▊ | 199266/225000 [380:03:01<61:31:07,  8.61s/it]
 89%|████████▊ | 199267/225000 [380:03:07<55:52:22,  7.82s/it]
 89%|████████▊ | 199268/225000 [380:03:13<51:36:54,  7.22s/it]
 89%|████████▊ | 199269/225000 [380:03:27<65:58:29,  9.23s/it]
 89%|████████▊ | 199270/225000 [380:03:37<68:20:13,  9.56s/it]
 89%|████████▊ | 199271/225000 [380:03:45<64:24:34,  9.01s/it]
 89%|████████▊ | 199272/225000 [380:03:52<60:18:53,  8.44s/it]
 89%|████████▊ | 199273/225000 [380:03:59<58:09:14,  8.14s/it]
 89%|████████▊ | 199274/225000 [380:04:07<56:10:52,  7.86s/it]
 89%|████████▊ | 199275/225000 [380:04:22<72:43:09, 10.18s/it]
 89%|████████▊ | 199276/225000 [380:04:28<64:07:20,  8.97s/it]
 89%|████████▊ | 199277/225000 [380:04:35<58:10:20,  8.14s/it]
 89%|████████▊ | 199278/225000 [380:04:41<53:39:22,  7.51s/it]
 89%|████████▊ | 199279/225000 [380:04:56<70:41:18,  9.89s/it]
 89%|████████▊ | 199280/225000 [380:05:04<66:01:05,  9.24s/it]
 89%|████████▊ | 199281/225000 [380:05:08<55:39:44,  7.79s/it]
 89%|████████▊ | 199282/225000 [380:05:18<59:14:17,  8.29s/it]
 89%|████████▊ | 199283/225000 [380:05:29<65:16:51,  9.14s/it]
 89%|████████▊ | 199284/225000 [380:05:38<65:44:21,  9.20s/it]
 89%|████████▊ | 199285/225000 [380:05:45<61:26:24,  8.60s/it]
 89%|████████▊ | 199286/225000 [380:05:51<55:48:19,  7.81s/it]
 89%|████████▊ | 199287/225000 [380:06:00<58:12:01,  8.15s/it]
 89%|████████▊ | 199288/225000 [380:06:07<54:20:00,  7.61s/it]
 89%|████████▊ | 199289/225000 [380:06:16<58:41:27,  8.22s/it]
 89%|████████▊ | 199290/225000 [380:06:28<65:45:25,  9.21s/it]
 89%|████████▊ | 199291/225000 [380:06:34<58:50:01,  8.24s/it]
 89%|████████▊ | 199292/225000 [380:06:47<69:07:26,  9.68s/it]
 89%|████████▊ | 199293/225000 [380:06:58<71:19:34,  9.99s/it]
 89%|████████▊ | 199294/225000 [380:07:06<68:28:13,  9.59s/it]
 89%|████████▊ | 199295/225000 [380:07:12<60:33:45,  8.48s/it]
 89%|████████▊ | 199296/225000 [380:07:18<55:54:15,  7.83s/it]
 89%|████████▊ | 199297/225000 [380:07:25<53:57:41,  7.56s/it]
 89%|████████▊ | 199298/225000 [380:07:32<52:52:32,  7.41s/it]
 89%|████████▊ | 199299/225000 [380:07:38<49:40:39,  6.96s/it]
 89%|████████▊ | 199300/225000 [380:07:50<60:51:17,  8.52s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.435', 'learning_rate': '1.755e-06', 'epoch': '0.8858', 'num_input_tokens_seen': 636883432, 'train_runtime': '1.368e+06', 'train_tokens_per_second': '465.4'}

 89%|████████▊ | 199300/225000 [380:07:50<60:51:17,  8.52s/it]
 89%|████████▊ | 199301/225000 [380:08:04<72:04:50, 10.10s/it]
 89%|████████▊ | 199302/225000 [380:08:10<63:52:49,  8.95s/it]
 89%|████████▊ | 199303/225000 [380:08:17<58:02:48,  8.13s/it]
 89%|████████▊ | 199304/225000 [380:08:23<53:28:01,  7.49s/it]
 89%|████████▊ | 199305/225000 [380:08:30<52:35:21,  7.37s/it]
 89%|████████▊ | 199306/225000 [380:08:42<61:58:25,  8.68s/it]
 89%|████████▊ | 199307/225000 [380:08:51<63:06:33,  8.84s/it]
 89%|████████▊ | 199308/225000 [380:09:01<65:29:42,  9.18s/it]
 89%|████████▊ | 199309/225000 [380:09:08<61:06:55,  8.56s/it]
 89%|████████▊ | 199310/225000 [380:09:14<56:10:56,  7.87s/it]
 89%|████████▊ | 199311/225000 [380:09:27<65:59:27,  9.25s/it]
 89%|████████▊ | 199312/225000 [380:09:36<66:46:20,  9.36s/it]
 89%|████████▊ | 199313/225000 [380:09:41<56:55:54,  7.98s/it]
 89%|████████▊ | 199314/225000 [380:09:54<67:58:53,  9.53s/it]
 89%|████████▊ | 199315/225000 [380:10:03<66:15:49,  9.29s/it]
 89%|████████▊ | 199316/225000 [380:10:14<70:03:19,  9.82s/it]
 89%|████████▊ | 199317/225000 [380:10:19<60:02:02,  8.42s/it]
 89%|████████▊ | 199318/225000 [380:10:26<57:53:13,  8.11s/it]
 89%|████████▊ | 199319/225000 [380:10:34<55:59:16,  7.85s/it]
 89%|████████▊ | 199320/225000 [380:10:44<62:00:58,  8.69s/it]
 89%|████████▊ | 199321/225000 [380:10:54<63:12:50,  8.86s/it]
 89%|████████▊ | 199322/225000 [380:10:58<52:53:48,  7.42s/it]
 89%|████████▊ | 199323/225000 [380:11:05<52:14:08,  7.32s/it]
 89%|████████▊ | 199324/225000 [380:11:11<50:29:53,  7.08s/it]
 89%|████████▊ | 199325/225000 [380:11:20<54:37:16,  7.66s/it]
 89%|████████▊ | 199326/225000 [380:11:28<55:46:21,  7.82s/it]
 89%|████████▊ | 199327/225000 [380:11:36<55:05:26,  7.73s/it]
 89%|████████▊ | 199328/225000 [380:11:43<54:13:37,  7.60s/it]
 89%|████████▊ | 199329/225000 [380:11:56<65:10:11,  9.14s/it]
 89%|████████▊ | 199330/225000 [380:12:03<59:34:35,  8.36s/it]
 89%|████████▊ | 199331/225000 [380:12:08<54:00:07,  7.57s/it]
 89%|████████▊ | 199332/225000 [380:12:20<63:58:27,  8.97s/it]
 89%|████████▊ | 199333/225000 [380:12:30<64:29:05,  9.04s/it]
 89%|████████▊ | 199334/225000 [380:12:36<57:54:47,  8.12s/it]
 89%|████████▊ | 199335/225000 [380:12:43<56:22:55,  7.91s/it]
 89%|████████▊ | 199336/225000 [380:12:56<66:23:54,  9.31s/it]
 89%|████████▊ | 199337/225000 [380:13:01<58:48:33,  8.25s/it]
 89%|████████▊ | 199338/225000 [380:13:13<64:58:30,  9.12s/it]
 89%|████████▊ | 199339/225000 [380:13:21<63:44:07,  8.94s/it]
 89%|████████▊ | 199340/225000 [380:13:30<62:54:18,  8.83s/it]
 89%|████████▊ | 199341/225000 [380:13:41<68:17:21,  9.58s/it]
 89%|████████▊ | 199342/225000 [380:13:46<57:44:42,  8.10s/it]
 89%|████████▊ | 199343/225000 [380:13:53<55:05:39,  7.73s/it]
 89%|████████▊ | 199344/225000 [380:14:00<54:52:49,  7.70s/it]
 89%|████████▊ | 199345/225000 [380:14:07<53:02:59,  7.44s/it]
 89%|████████▊ | 199346/225000 [380:14:13<49:35:19,  6.96s/it]
 89%|████████▊ | 199347/225000 [380:14:23<56:37:50,  7.95s/it]
 89%|████████▊ | 199348/225000 [380:14:31<56:39:35,  7.95s/it]
 89%|████████▊ | 199349/225000 [380:14:48<75:26:03, 10.59s/it]
 89%|████████▊ | 199350/225000 [380:14:54<65:38:45,  9.21s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.369', 'learning_rate': '1.748e-06', 'epoch': '0.886', 'num_input_tokens_seen': 637108736, 'train_runtime': '1.369e+06', 'train_tokens_per_second': '465.4'}

 89%|████████▊ | 199350/225000 [380:14:54<65:38:45,  9.21s/it]
 89%|████████▊ | 199351/225000 [380:15:03<66:30:30,  9.33s/it]
 89%|████████▊ | 199352/225000 [380:15:12<63:57:47,  8.98s/it]
 89%|████████▊ | 199353/225000 [380:15:18<57:30:51,  8.07s/it]
 89%|████████▊ | 199354/225000 [380:15:28<63:27:41,  8.91s/it]
 89%|████████▊ | 199355/225000 [380:15:36<61:05:03,  8.57s/it]
 89%|████████▊ | 199356/225000 [380:15:43<56:54:15,  7.99s/it]
 89%|████████▊ | 199357/225000 [380:15:51<56:39:33,  7.95s/it]
 89%|████████▊ | 199358/225000 [380:16:00<58:57:39,  8.28s/it]
 89%|████████▊ | 199359/225000 [380:16:06<55:42:00,  7.82s/it]
 89%|████████▊ | 199360/225000 [380:16:14<54:39:46,  7.67s/it]
 89%|████████▊ | 199361/225000 [380:16:23<57:18:35,  8.05s/it]
 89%|████████▊ | 199362/225000 [380:16:33<62:55:26,  8.84s/it]
 89%|████████▊ | 199363/225000 [380:16:43<63:35:44,  8.93s/it]
 89%|████████▊ | 199364/225000 [380:16:48<56:44:41,  7.97s/it]
 89%|████████▊ | 199365/225000 [380:17:00<63:58:00,  8.98s/it]
 89%|████████▊ | 199366/225000 [380:17:08<61:39:37,  8.66s/it]
 89%|████████▊ | 199367/225000 [380:17:16<60:43:21,  8.53s/it]
 89%|████████▊ | 199368/225000 [380:17:25<62:52:53,  8.83s/it]
 89%|████████▊ | 199369/225000 [380:17:32<58:54:28,  8.27s/it]
 89%|████████▊ | 199370/225000 [380:17:47<73:03:27, 10.26s/it]
 89%|████████▊ | 199371/225000 [380:17:55<67:08:27,  9.43s/it]
 89%|████████▊ | 199372/225000 [380:18:01<60:17:21,  8.47s/it]
 89%|████████▊ | 199373/225000 [380:18:10<61:36:17,  8.65s/it]
 89%|████████▊ | 199374/225000 [380:18:17<58:54:34,  8.28s/it]
 89%|████████▊ | 199375/225000 [380:18:28<64:05:02,  9.00s/it]
 89%|████████▊ | 199376/225000 [380:18:36<61:54:33,  8.70s/it]
 89%|████████▊ | 199377/225000 [380:18:46<64:14:00,  9.02s/it]
 89%|████████▊ | 199378/225000 [380:18:53<60:08:42,  8.45s/it]
 89%|████████▊ | 199379/225000 [380:19:05<68:43:32,  9.66s/it]
 89%|████████▊ | 199380/225000 [380:19:13<64:48:54,  9.11s/it]
 89%|████████▊ | 199381/225000 [380:19:20<58:49:33,  8.27s/it]
 89%|████████▊ | 199382/225000 [380:19:30<64:34:14,  9.07s/it]
 89%|████████▊ | 199383/225000 [380:19:40<65:08:59,  9.16s/it]
 89%|████████▊ | 199384/225000 [380:19:46<58:06:56,  8.17s/it]
 89%|████████▊ | 199385/225000 [380:19:58<67:24:00,  9.47s/it]
 89%|████████▊ | 199386/225000 [380:20:05<62:34:17,  8.79s/it]
 89%|████████▊ | 199387/225000 [380:20:11<55:18:14,  7.77s/it]
 89%|████████▊ | 199388/225000 [380:20:23<65:23:16,  9.19s/it]
 89%|████████▊ | 199389/225000 [380:20:33<65:25:20,  9.20s/it]
 89%|████████▊ | 199390/225000 [380:20:38<56:47:33,  7.98s/it]
 89%|████████▊ | 199391/225000 [380:20:44<53:07:30,  7.47s/it]
 89%|████████▊ | 199392/225000 [380:20:55<60:40:25,  8.53s/it]
 89%|████████▊ | 199393/225000 [380:21:04<60:59:08,  8.57s/it]
 89%|████████▊ | 199394/225000 [380:21:12<61:24:11,  8.63s/it]
 89%|████████▊ | 199395/225000 [380:21:18<55:54:44,  7.86s/it]
 89%|████████▊ | 199396/225000 [380:21:26<55:53:51,  7.86s/it]
 89%|████████▊ | 199397/225000 [380:21:34<55:24:19,  7.79s/it]
 89%|████████▊ | 199398/225000 [380:21:43<58:49:26,  8.27s/it]
 89%|████████▊ | 199399/225000 [380:21:48<51:39:33,  7.26s/it]
 89%|████████▊ | 199400/225000 [380:21:56<51:56:58,  7.31s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.455', 'learning_rate': '1.741e-06', 'epoch': '0.8862', 'num_input_tokens_seen': 637340672, 'train_runtime': '1.369e+06', 'train_tokens_per_second': '465.4'}

 89%|████████▊ | 199400/225000 [380:21:56<51:56:58,  7.31s/it]
 89%|████████▊ | 199401/225000 [380:22:05<56:49:42,  7.99s/it]
 89%|████████▊ | 199402/225000 [380:22:11<51:56:15,  7.30s/it]
 89%|████████▊ | 199403/225000 [380:22:21<57:06:34,  8.03s/it]
 89%|████████▊ | 199404/225000 [380:22:34<67:25:17,  9.48s/it]
 89%|████████▊ | 199405/225000 [380:22:39<59:12:51,  8.33s/it]
 89%|████████▊ | 199406/225000 [380:22:45<54:06:07,  7.61s/it]
 89%|████████▊ | 199407/225000 [380:22:58<64:23:50,  9.06s/it]
 89%|████████▊ | 199408/225000 [380:23:09<70:02:37,  9.85s/it]
 89%|████████▊ | 199409/225000 [380:23:15<61:39:51,  8.67s/it]
 89%|████████▊ | 199410/225000 [380:23:22<57:42:15,  8.12s/it]
 89%|████████▊ | 199411/225000 [380:23:33<64:02:48,  9.01s/it]
 89%|████████▊ | 199412/225000 [380:23:42<62:49:19,  8.84s/it]
 89%|████████▊ | 199413/225000 [380:23:47<56:36:18,  7.96s/it]
 89%|████████▊ | 199414/225000 [380:23:53<51:25:08,  7.23s/it]
 89%|████████▊ | 199415/225000 [380:24:04<58:57:34,  8.30s/it]
 89%|████████▊ | 199416/225000 [380:24:15<65:55:37,  9.28s/it]
 89%|████████▊ | 199417/225000 [380:24:21<57:46:20,  8.13s/it]
 89%|████████▊ | 199418/225000 [380:24:28<56:38:08,  7.97s/it]
 89%|████████▊ | 199419/225000 [380:24:35<53:51:01,  7.58s/it]
 89%|████████▊ | 199420/225000 [380:24:44<57:28:40,  8.09s/it]
 89%|████████▊ | 199421/225000 [380:24:52<55:53:30,  7.87s/it]
 89%|████████▊ | 199422/225000 [380:24:57<50:15:03,  7.07s/it]
 89%|████████▊ | 199423/225000 [380:25:07<57:44:32,  8.13s/it]
 89%|████████▊ | 199424/225000 [380:25:16<57:56:29,  8.16s/it]
 89%|████████▊ | 199425/225000 [380:25:20<50:44:51,  7.14s/it]
 89%|████████▊ | 199426/225000 [380:25:26<47:04:25,  6.63s/it]
 89%|████████▊ | 199427/225000 [380:25:40<62:20:30,  8.78s/it]
 89%|████████▊ | 199428/225000 [380:25:47<59:56:11,  8.44s/it]
 89%|████████▊ | 199429/225000 [380:26:00<67:57:27,  9.57s/it]
 89%|████████▊ | 199430/225000 [380:26:07<63:39:37,  8.96s/it]
 89%|████████▊ | 199431/225000 [380:26:16<64:24:57,  9.07s/it]
 89%|████████▊ | 199432/225000 [380:26:27<68:35:36,  9.66s/it]
 89%|████████▊ | 199433/225000 [380:26:37<69:08:14,  9.73s/it]
 89%|████████▊ | 199434/225000 [380:26:43<60:02:41,  8.46s/it]
 89%|████████▊ | 199435/225000 [380:26:51<59:29:17,  8.38s/it]
 89%|████████▊ | 199436/225000 [380:27:03<67:36:58,  9.52s/it]
 89%|████████▊ | 199437/225000 [380:27:08<56:53:09,  8.01s/it]
 89%|████████▊ | 199438/225000 [380:27:17<58:36:41,  8.25s/it]
 89%|████████▊ | 199439/225000 [380:27:21<50:57:50,  7.18s/it]
 89%|████████▊ | 199440/225000 [380:27:36<66:56:49,  9.43s/it]
 89%|████████▊ | 199441/225000 [380:27:45<66:21:05,  9.35s/it]
 89%|████████▊ | 199442/225000 [380:27:53<62:43:58,  8.84s/it]
 89%|████████▊ | 199443/225000 [380:28:02<63:24:00,  8.93s/it]
 89%|████████▊ | 199444/225000 [380:28:09<60:33:24,  8.53s/it]
 89%|████████▊ | 199445/225000 [380:28:17<58:56:08,  8.30s/it]
 89%|████████▊ | 199446/225000 [380:28:30<68:34:39,  9.66s/it]
 89%|████████▊ | 199447/225000 [380:28:36<61:47:13,  8.70s/it]
 89%|████████▊ | 199448/225000 [380:28:42<54:20:53,  7.66s/it]
 89%|████████▊ | 199449/225000 [380:28:49<54:26:24,  7.67s/it]
 89%|████████▊ | 199450/225000 [380:29:01<63:31:47,  8.95s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.223', 'learning_rate': '1.735e-06', 'epoch': '0.8864', 'num_input_tokens_seen': 637565760, 'train_runtime': '1.37e+06', 'train_tokens_per_second': '465.5'}

 89%|████████▊ | 199450/225000 [380:29:01<63:31:47,  8.95s/it]
 89%|████████▊ | 199451/225000 [380:29:09<60:35:02,  8.54s/it]
 89%|████████▊ | 199452/225000 [380:29:17<59:51:01,  8.43s/it]
 89%|████████▊ | 199453/225000 [380:29:24<56:09:13,  7.91s/it]
 89%|████████▊ | 199454/225000 [380:29:29<51:24:43,  7.25s/it]
 89%|████████▊ | 199455/225000 [380:29:36<50:08:47,  7.07s/it]
 89%|████████▊ | 199456/225000 [380:29:43<49:16:36,  6.94s/it]
 89%|████████▊ | 199457/225000 [380:29:57<63:40:04,  8.97s/it]
 89%|████████▊ | 199458/225000 [380:30:03<58:33:56,  8.25s/it]
 89%|████████▊ | 199459/225000 [380:30:10<55:14:54,  7.79s/it]
 89%|████████▊ | 199460/225000 [380:30:20<59:24:31,  8.37s/it]
 89%|████████▊ | 199461/225000 [380:30:31<65:11:57,  9.19s/it]
 89%|████████▊ | 199462/225000 [380:30:37<59:42:08,  8.42s/it]
 89%|████████▊ | 199463/225000 [380:30:47<63:18:13,  8.92s/it]
 89%|████████▊ | 199464/225000 [380:30:55<60:40:52,  8.55s/it]
 89%|████████▊ | 199465/225000 [380:31:04<61:42:37,  8.70s/it]
 89%|████████▊ | 199466/225000 [380:31:11<58:48:46,  8.29s/it]
 89%|████████▊ | 199467/225000 [380:31:20<59:24:54,  8.38s/it]
 89%|████████▊ | 199468/225000 [380:31:29<60:22:37,  8.51s/it]
 89%|████████▊ | 199469/225000 [380:31:35<55:35:00,  7.84s/it]
 89%|████████▊ | 199470/225000 [380:31:43<56:27:19,  7.96s/it]
 89%|████████▊ | 199471/225000 [380:31:56<66:49:11,  9.42s/it]
 89%|████████▊ | 199472/225000 [380:32:02<59:48:18,  8.43s/it]
 89%|████████▊ | 199473/225000 [380:32:09<55:16:07,  7.79s/it]
 89%|████████▊ | 199474/225000 [380:32:16<55:11:24,  7.78s/it]
 89%|████████▊ | 199475/225000 [380:32:22<51:31:08,  7.27s/it]
 89%|████████▊ | 199476/225000 [380:32:39<72:10:17, 10.18s/it]
 89%|████████▊ | 199477/225000 [380:32:47<66:13:05,  9.34s/it]
 89%|████████▊ | 199478/225000 [380:32:53<60:24:52,  8.52s/it]
 89%|████████▊ | 199479/225000 [380:33:03<62:28:24,  8.81s/it]
 89%|████████▊ | 199480/225000 [380:33:08<53:53:24,  7.60s/it]
 89%|████████▊ | 199481/225000 [380:33:19<61:51:32,  8.73s/it]
 89%|████████▊ | 199482/225000 [380:33:29<63:55:25,  9.02s/it]
 89%|████████▊ | 199483/225000 [380:33:42<73:17:30, 10.34s/it]
 89%|████████▊ | 199484/225000 [380:33:47<61:46:41,  8.72s/it]
 89%|████████▊ | 199485/225000 [380:33:57<63:37:02,  8.98s/it]
 89%|████████▊ | 199486/225000 [380:34:04<59:46:40,  8.43s/it]
 89%|████████▊ | 199487/225000 [380:34:12<59:04:45,  8.34s/it]
 89%|████████▊ | 199488/225000 [380:34:21<60:14:20,  8.50s/it]
 89%|████████▊ | 199489/225000 [380:34:26<53:47:42,  7.59s/it]
 89%|████████▊ | 199490/225000 [380:34:33<52:25:40,  7.40s/it]
 89%|████████▊ | 199491/225000 [380:34:47<65:02:44,  9.18s/it]
 89%|████████▊ | 199492/225000 [380:34:55<63:11:25,  8.92s/it]
 89%|████████▊ | 199493/225000 [380:35:01<57:14:15,  8.08s/it]
 89%|████████▊ | 199494/225000 [380:35:08<55:22:03,  7.81s/it]
 89%|████████▊ | 199495/225000 [380:35:14<51:46:32,  7.31s/it]
 89%|████████▊ | 199496/225000 [380:35:26<61:24:12,  8.67s/it]
 89%|████████▊ | 199497/225000 [380:35:32<54:36:04,  7.71s/it]
 89%|████████▊ | 199498/225000 [380:35:41<58:58:32,  8.33s/it]
 89%|████████▊ | 199499/225000 [380:35:51<61:28:48,  8.68s/it]
 89%|████████▊ | 199500/225000 [380:35:58<58:00:00,  8.19s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.285', 'learning_rate': '1.728e-06', 'epoch': '0.8867', 'num_input_tokens_seen': 637795192, 'train_runtime': '1.37e+06', 'train_tokens_per_second': '465.5'}

 89%|████████▊ | 199500/225000 [380:35:58<58:00:00,  8.19s/it]
 89%|████████▊ | 199501/225000 [380:36:06<58:11:33,  8.22s/it]
 89%|████████▊ | 199502/225000 [380:36:14<57:15:45,  8.08s/it]
 89%|████████▊ | 199503/225000 [380:36:21<54:41:41,  7.72s/it]
 89%|████████▊ | 199504/225000 [380:36:27<50:22:19,  7.11s/it]
 89%|████████▊ | 199505/225000 [380:36:32<47:25:32,  6.70s/it]
 89%|████████▊ | 199506/225000 [380:36:47<65:15:40,  9.22s/it]
 89%|████████▊ | 199507/225000 [380:36:54<59:50:09,  8.45s/it]
 89%|████████▊ | 199508/225000 [380:37:04<62:37:17,  8.84s/it]
 89%|████████▊ | 199509/225000 [380:37:11<59:33:55,  8.41s/it]
 89%|████████▊ | 199510/225000 [380:37:18<56:41:58,  8.01s/it]
 89%|████████▊ | 199511/225000 [380:37:24<52:15:14,  7.38s/it]
 89%|████████▊ | 199512/225000 [380:37:33<54:30:06,  7.70s/it]
 89%|████████▊ | 199513/225000 [380:37:50<75:05:06, 10.61s/it]
 89%|████████▊ | 199514/225000 [380:37:59<71:36:58, 10.12s/it]
 89%|████████▊ | 199515/225000 [380:38:04<61:36:41,  8.70s/it]
 89%|████████▊ | 199516/225000 [380:38:09<53:48:57,  7.60s/it]
 89%|████████▊ | 199517/225000 [380:38:22<64:23:34,  9.10s/it]
 89%|████████▊ | 199518/225000 [380:38:27<56:09:57,  7.93s/it]
 89%|████████▊ | 199519/225000 [380:38:38<63:11:46,  8.93s/it]
 89%|████████▊ | 199520/225000 [380:38:45<57:16:55,  8.09s/it]
 89%|████████▊ | 199521/225000 [380:38:50<52:25:16,  7.41s/it]
 89%|████████▊ | 199522/225000 [380:39:04<65:15:25,  9.22s/it]
 89%|████████▊ | 199523/225000 [380:39:08<53:30:12,  7.56s/it]
 89%|████████▊ | 199524/225000 [380:39:14<51:30:40,  7.28s/it]
 89%|████████▊ | 199525/225000 [380:39:30<69:24:02,  9.81s/it]
 89%|████████▊ | 199526/225000 [380:39:37<62:57:54,  8.90s/it]
 89%|████████▊ | 199527/225000 [380:39:47<65:48:39,  9.30s/it]
 89%|████████▊ | 199528/225000 [380:39:51<54:55:33,  7.76s/it]
 89%|████████▊ | 199529/225000 [380:39:56<48:52:01,  6.91s/it]
 89%|████████▊ | 199530/225000 [380:40:08<60:12:36,  8.51s/it]
 89%|████████▊ | 199531/225000 [380:40:14<54:51:38,  7.75s/it]
 89%|████████▊ | 199532/225000 [380:40:24<59:22:21,  8.39s/it]
 89%|████████▊ | 199533/225000 [380:40:32<57:14:32,  8.09s/it]
 89%|████████▊ | 199534/225000 [380:40:39<54:57:25,  7.77s/it]
 89%|████████▊ | 199535/225000 [380:40:46<54:39:16,  7.73s/it]
 89%|████████▊ | 199536/225000 [380:40:54<54:23:54,  7.69s/it]
 89%|████████▊ | 199537/225000 [380:41:00<50:21:31,  7.12s/it]
 89%|████████▊ | 199538/225000 [380:41:15<68:14:40,  9.65s/it]
 89%|████████▊ | 199539/225000 [380:41:22<62:19:39,  8.81s/it]
 89%|████████▊ | 199540/225000 [380:41:30<61:15:03,  8.66s/it]
 89%|████████▊ | 199541/225000 [380:41:38<58:24:12,  8.26s/it]
 89%|████████▊ | 199542/225000 [380:41:46<59:31:22,  8.42s/it]
 89%|████████▊ | 199543/225000 [380:41:56<62:02:02,  8.77s/it]
 89%|████████▊ | 199544/225000 [380:42:04<59:36:37,  8.43s/it]
 89%|████████▊ | 199545/225000 [380:42:16<67:35:39,  9.56s/it]
 89%|████████▊ | 199546/225000 [380:42:22<61:01:41,  8.63s/it]
 89%|████████▊ | 199547/225000 [380:42:27<53:37:28,  7.58s/it]
 89%|████████▊ | 199548/225000 [380:42:43<70:50:45, 10.02s/it]
 89%|████████▊ | 199549/225000 [380:42:54<72:27:50, 10.25s/it]
 89%|████████▊ | 199550/225000 [380:43:05<74:25:07, 10.53s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.72', 'learning_rate': '1.721e-06', 'epoch': '0.8869', 'num_input_tokens_seen': 638023272, 'train_runtime': '1.371e+06', 'train_tokens_per_second': '465.5'}

 89%|████████▊ | 199550/225000 [380:43:05<74:25:07, 10.53s/it]
 89%|████████▊ | 199551/225000 [380:43:12<67:24:55,  9.54s/it]
 89%|████████▊ | 199552/225000 [380:43:22<67:48:56,  9.59s/it]
 89%|████████▊ | 199553/225000 [380:43:31<66:00:50,  9.34s/it]
 89%|████████▊ | 199554/225000 [380:43:35<55:14:09,  7.81s/it]
 89%|████████▊ | 199555/225000 [380:43:48<65:38:35,  9.29s/it]
 89%|████████▊ | 199556/225000 [380:43:55<61:46:25,  8.74s/it]
 89%|████████▊ | 199557/225000 [380:44:03<59:53:49,  8.47s/it]
 89%|████████▊ | 199558/225000 [380:44:17<70:25:27,  9.96s/it]
 89%|████████▊ | 199559/225000 [380:44:22<60:00:11,  8.49s/it]
 89%|████████▊ | 199560/225000 [380:44:31<62:22:03,  8.83s/it]
 89%|████████▊ | 199561/225000 [380:44:37<56:55:52,  8.06s/it]
 89%|████████▊ | 199562/225000 [380:44:51<67:39:16,  9.57s/it]
 89%|████████▊ | 199563/225000 [380:45:03<73:34:20, 10.41s/it]
 89%|████████▊ | 199564/225000 [380:45:12<71:22:32, 10.10s/it]
 89%|████████▊ | 199565/225000 [380:45:21<68:50:55,  9.74s/it]
 89%|████████▊ | 199566/225000 [380:45:27<59:39:53,  8.45s/it]
 89%|████████▊ | 199567/225000 [380:45:35<59:33:58,  8.43s/it]
 89%|████████▊ | 199568/225000 [380:45:48<68:45:05,  9.73s/it]
 89%|████████▊ | 199569/225000 [380:45:54<61:44:37,  8.74s/it]
 89%|████████▊ | 199570/225000 [380:46:04<63:27:21,  8.98s/it]
 89%|████████▊ | 199571/225000 [380:46:10<57:45:05,  8.18s/it]
 89%|████████▊ | 199572/225000 [380:46:19<58:22:29,  8.26s/it]
 89%|████████▊ | 199573/225000 [380:46:29<62:00:21,  8.78s/it]
 89%|████████▊ | 199574/225000 [380:46:37<60:24:24,  8.55s/it]
 89%|████████▊ | 199575/225000 [380:46:42<53:56:24,  7.64s/it]
 89%|████████▊ | 199576/225000 [380:46:52<59:42:57,  8.46s/it]
 89%|████████▊ | 199577/225000 [380:46:59<55:01:13,  7.79s/it]
 89%|████████▊ | 199578/225000 [380:47:09<60:05:32,  8.51s/it]
 89%|████████▊ | 199579/225000 [380:47:20<64:41:09,  9.16s/it]
 89%|████████▊ | 199580/225000 [380:47:26<58:59:33,  8.35s/it]
 89%|████████▊ | 199581/225000 [380:47:32<54:33:55,  7.73s/it]
 89%|████████▊ | 199582/225000 [380:47:43<61:32:14,  8.72s/it]
 89%|████████▊ | 199583/225000 [380:47:49<55:01:01,  7.79s/it]
 89%|████████▊ | 199584/225000 [380:48:01<64:09:52,  9.09s/it]
 89%|████████▊ | 199585/225000 [380:48:07<56:43:11,  8.03s/it]
 89%|████████▊ | 199586/225000 [380:48:15<58:22:09,  8.27s/it]
 89%|████████▊ | 199587/225000 [380:48:28<66:54:54,  9.48s/it]
 89%|████████▊ | 199588/225000 [380:48:32<55:34:08,  7.87s/it]
 89%|████████▊ | 199589/225000 [380:48:39<54:42:32,  7.75s/it]
 89%|████████▊ | 199590/225000 [380:48:48<55:51:55,  7.91s/it]
 89%|████████▊ | 199591/225000 [380:48:53<50:43:58,  7.19s/it]
 89%|████████▊ | 199592/225000 [380:49:02<54:30:22,  7.72s/it]
 89%|████████▊ | 199593/225000 [380:49:15<66:28:31,  9.42s/it]
 89%|████████▊ | 199594/225000 [380:49:20<57:10:40,  8.10s/it]
 89%|████████▊ | 199595/225000 [380:49:26<51:11:16,  7.25s/it]
 89%|████████▊ | 199596/225000 [380:49:34<53:10:40,  7.54s/it]
 89%|████████▊ | 199597/225000 [380:49:42<53:48:23,  7.63s/it]
 89%|████████▊ | 199598/225000 [380:49:50<55:16:26,  7.83s/it]
 89%|████████▊ | 199599/225000 [380:49:58<55:40:34,  7.89s/it]
 89%|████████▊ | 199600/225000 [380:50:05<52:58:27,  7.51s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.592', 'learning_rate': '1.715e-06', 'epoch': '0.8871', 'num_input_tokens_seen': 638249528, 'train_runtime': '1.371e+06', 'train_tokens_per_second': '465.5'}

 89%|████████▊ | 199600/225000 [380:50:05<52:58:27,  7.51s/it]
 89%|████████▊ | 199601/225000 [380:50:12<52:24:51,  7.43s/it]
 89%|████████▊ | 199602/225000 [380:50:25<64:56:58,  9.21s/it]
 89%|████████▊ | 199603/225000 [380:50:31<58:13:41,  8.25s/it]
 89%|████████▊ | 199604/225000 [380:50:37<52:59:04,  7.51s/it]
 89%|████████▊ | 199605/225000 [380:50:50<64:13:47,  9.11s/it]
 89%|████████▊ | 199606/225000 [380:50:58<63:01:05,  8.93s/it]
 89%|████████▊ | 199607/225000 [380:51:11<71:12:47, 10.10s/it]
 89%|████████▊ | 199608/225000 [380:51:16<60:30:00,  8.58s/it]
 89%|████████▊ | 199609/225000 [380:51:31<74:25:00, 10.55s/it]
 89%|████████▊ | 199610/225000 [380:51:38<66:53:31,  9.48s/it]
 89%|████████▊ | 199611/225000 [380:51:49<68:02:11,  9.65s/it]
 89%|████████▊ | 199612/225000 [380:51:53<57:14:53,  8.12s/it]
 89%|████████▊ | 199613/225000 [380:52:05<64:42:24,  9.18s/it]
 89%|████████▊ | 199614/225000 [380:52:15<66:18:23,  9.40s/it]
 89%|████████▊ | 199615/225000 [380:52:24<66:48:10,  9.47s/it]
 89%|████████▊ | 199616/225000 [380:52:31<61:27:09,  8.72s/it]
 89%|████████▊ | 199617/225000 [380:52:37<54:15:41,  7.70s/it]
 89%|████████▊ | 199618/225000 [380:52:43<51:05:49,  7.25s/it]
 89%|████████▊ | 199619/225000 [380:52:54<60:21:24,  8.56s/it]
 89%|████████▊ | 199620/225000 [380:53:02<57:33:20,  8.16s/it]
 89%|████████▊ | 199621/225000 [380:53:09<55:05:47,  7.82s/it]
 89%|████████▊ | 199622/225000 [380:53:16<54:49:10,  7.78s/it]
 89%|████████▊ | 199623/225000 [380:53:29<65:59:05,  9.36s/it]
 89%|████████▊ | 199624/225000 [380:53:36<59:18:48,  8.41s/it]
 89%|████████▊ | 199625/225000 [380:53:44<58:43:39,  8.33s/it]
 89%|████████▊ | 199626/225000 [380:53:56<67:30:33,  9.58s/it]
 89%|████████▊ | 199627/225000 [380:54:02<59:59:43,  8.51s/it]
 89%|████████▊ | 199628/225000 [380:54:10<58:29:23,  8.30s/it]
 89%|████████▊ | 199629/225000 [380:54:16<53:33:07,  7.60s/it]
 89%|████████▊ | 199630/225000 [380:54:28<63:10:10,  8.96s/it]
 89%|████████▊ | 199631/225000 [380:54:35<59:09:19,  8.39s/it]
 89%|████████▊ | 199632/225000 [380:54:42<54:51:56,  7.79s/it]
 89%|████████▊ | 199633/225000 [380:54:50<55:32:20,  7.88s/it]
 89%|████████▊ | 199634/225000 [380:55:00<60:23:01,  8.57s/it]
 89%|████████▊ | 199635/225000 [380:55:06<54:37:22,  7.75s/it]
 89%|████████▊ | 199636/225000 [380:55:14<56:42:43,  8.05s/it]
 89%|████████▊ | 199637/225000 [380:55:22<54:42:32,  7.77s/it]
 89%|████████▊ | 199638/225000 [380:55:34<64:28:32,  9.15s/it]
 89%|████████▊ | 199639/225000 [380:55:45<69:00:33,  9.80s/it]
 89%|████████▊ | 199640/225000 [380:55:54<66:52:02,  9.49s/it]
 89%|████████▊ | 199641/225000 [380:56:00<60:21:31,  8.57s/it]
 89%|████████▊ | 199642/225000 [380:56:07<55:25:28,  7.87s/it]
 89%|████████▊ | 199643/225000 [380:56:18<62:46:22,  8.91s/it]
 89%|████████▊ | 199644/225000 [380:56:29<68:01:50,  9.66s/it]
 89%|████████▊ | 199645/225000 [380:56:41<72:40:37, 10.32s/it]
 89%|████████▊ | 199646/225000 [380:56:50<68:50:38,  9.78s/it]
 89%|████████▊ | 199647/225000 [380:56:56<60:35:45,  8.60s/it]
 89%|████████▊ | 199648/225000 [380:57:05<61:49:57,  8.78s/it]
 89%|████████▊ | 199649/225000 [380:57:13<60:08:57,  8.54s/it]
 89%|████████▊ | 199650/225000 [380:57:18<52:58:38,  7.52s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.066', 'learning_rate': '1.708e-06', 'epoch': '0.8873', 'num_input_tokens_seen': 638481352, 'train_runtime': '1.371e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▊ | 199650/225000 [380:57:18<52:58:38,  7.52s/it]
 89%|████████▊ | 199651/225000 [380:57:29<59:51:01,  8.50s/it]
 89%|████████▊ | 199652/225000 [380:57:38<60:45:53,  8.63s/it]
 89%|████████▊ | 199653/225000 [380:57:43<52:56:20,  7.52s/it]
 89%|████████▊ | 199654/225000 [380:57:49<49:33:23,  7.04s/it]
 89%|████████▊ | 199655/225000 [380:57:56<50:41:05,  7.20s/it]
 89%|████████▊ | 199656/225000 [380:58:07<58:46:13,  8.35s/it]
 89%|████████▊ | 199657/225000 [380:58:14<55:26:00,  7.87s/it]
 89%|████████▊ | 199658/225000 [380:58:25<61:43:31,  8.77s/it]
 89%|████████▊ | 199659/225000 [380:58:29<52:42:54,  7.49s/it]
 89%|████████▊ | 199660/225000 [380:58:36<51:31:23,  7.32s/it]
 89%|████████▊ | 199661/225000 [380:58:45<54:04:21,  7.68s/it]
 89%|████████▊ | 199662/225000 [380:58:55<59:11:25,  8.41s/it]
 89%|████████▊ | 199663/225000 [380:59:08<68:21:07,  9.71s/it]
 89%|████████▊ | 199664/225000 [380:59:12<57:36:31,  8.19s/it]
 89%|████████▊ | 199665/225000 [380:59:19<55:31:54,  7.89s/it]
 89%|████████▊ | 199666/225000 [380:59:32<65:24:59,  9.30s/it]
 89%|████████▊ | 199667/225000 [380:59:39<60:31:00,  8.60s/it]
 89%|████████▊ | 199668/225000 [380:59:53<71:17:48, 10.13s/it]
 89%|████████▊ | 199669/225000 [381:00:02<69:31:57,  9.88s/it]
 89%|████████▊ | 199670/225000 [381:00:10<66:23:26,  9.44s/it]
 89%|████████▊ | 199671/225000 [381:00:17<60:41:20,  8.63s/it]
 89%|████████▊ | 199672/225000 [381:00:26<61:57:11,  8.81s/it]
 89%|████████▊ | 199673/225000 [381:00:36<64:28:03,  9.16s/it]
 89%|████████▊ | 199674/225000 [381:00:41<55:01:07,  7.82s/it]
 89%|████████▊ | 199675/225000 [381:00:52<61:14:16,  8.71s/it]
 89%|████████▊ | 199676/225000 [381:00:56<52:39:23,  7.49s/it]
 89%|████████▊ | 199677/225000 [381:01:04<52:33:33,  7.47s/it]
 89%|████████▊ | 199678/225000 [381:01:11<51:50:43,  7.37s/it]
 89%|████████▊ | 199679/225000 [381:01:18<50:10:30,  7.13s/it]
 89%|████████▊ | 199680/225000 [381:01:29<59:42:22,  8.49s/it]
 89%|████████▊ | 199681/225000 [381:01:41<66:15:26,  9.42s/it]
 89%|████████▊ | 199682/225000 [381:01:49<63:34:40,  9.04s/it]
 89%|████████▊ | 199683/225000 [381:01:58<63:15:59,  9.00s/it]
 89%|████████▊ | 199684/225000 [381:02:04<56:57:00,  8.10s/it]
 89%|████████▊ | 199685/225000 [381:02:15<64:19:08,  9.15s/it]
 89%|████████▊ | 199686/225000 [381:02:26<68:05:11,  9.68s/it]
 89%|████████▊ | 199687/225000 [381:02:35<64:50:45,  9.22s/it]
 89%|████████▉ | 199688/225000 [381:02:42<61:47:15,  8.79s/it]
 89%|████████▉ | 199689/225000 [381:02:48<55:57:36,  7.96s/it]
 89%|████████▉ | 199690/225000 [381:03:00<63:15:34,  9.00s/it]
 89%|████████▉ | 199691/225000 [381:03:05<55:13:55,  7.86s/it]
 89%|████████▉ | 199692/225000 [381:03:11<52:22:04,  7.45s/it]
 89%|████████▉ | 199693/225000 [381:03:24<63:36:42,  9.05s/it]
 89%|████████▉ | 199694/225000 [381:03:32<61:25:17,  8.74s/it]
 89%|████████▉ | 199695/225000 [381:03:43<65:45:01,  9.35s/it]
 89%|████████▉ | 199696/225000 [381:03:54<69:46:33,  9.93s/it]
 89%|████████▉ | 199697/225000 [381:03:58<56:17:24,  8.01s/it]
 89%|████████▉ | 199698/225000 [381:04:08<61:41:52,  8.78s/it]
 89%|████████▉ | 199699/225000 [381:04:17<60:31:34,  8.61s/it]
 89%|████████▉ | 199700/225000 [381:04:22<54:14:04,  7.72s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.278', 'learning_rate': '1.701e-06', 'epoch': '0.8876', 'num_input_tokens_seen': 638702744, 'train_runtime': '1.372e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 199700/225000 [381:04:22<54:14:04,  7.72s/it]
 89%|████████▉ | 199701/225000 [381:04:32<57:53:03,  8.24s/it]
 89%|████████▉ | 199702/225000 [381:04:40<57:28:35,  8.18s/it]
 89%|████████▉ | 199703/225000 [381:04:44<50:09:35,  7.14s/it]
 89%|████████▉ | 199704/225000 [381:04:52<52:04:30,  7.41s/it]
 89%|████████▉ | 199705/225000 [381:05:00<51:52:19,  7.38s/it]
 89%|████████▉ | 199706/225000 [381:05:06<50:15:04,  7.15s/it]
 89%|████████▉ | 199707/225000 [381:05:13<49:29:27,  7.04s/it]
 89%|████████▉ | 199708/225000 [381:05:20<49:36:15,  7.06s/it]
 89%|████████▉ | 199709/225000 [381:05:35<65:32:04,  9.33s/it]
 89%|████████▉ | 199710/225000 [381:05:40<57:35:33,  8.20s/it]
 89%|████████▉ | 199711/225000 [381:05:48<55:32:14,  7.91s/it]
 89%|████████▉ | 199712/225000 [381:05:56<55:54:27,  7.96s/it]
 89%|████████▉ | 199713/225000 [381:06:07<62:36:28,  8.91s/it]
 89%|████████▉ | 199714/225000 [381:06:18<67:42:18,  9.64s/it]
 89%|████████▉ | 199715/225000 [381:06:29<69:03:56,  9.83s/it]
 89%|████████▉ | 199716/225000 [381:06:35<61:57:24,  8.82s/it]
 89%|████████▉ | 199717/225000 [381:06:45<64:40:17,  9.21s/it]
 89%|████████▉ | 199718/225000 [381:06:50<55:40:10,  7.93s/it]
 89%|████████▉ | 199719/225000 [381:06:59<58:01:44,  8.26s/it]
 89%|████████▉ | 199720/225000 [381:07:12<68:18:22,  9.73s/it]
 89%|████████▉ | 199721/225000 [381:07:18<59:04:12,  8.41s/it]
 89%|████████▉ | 199722/225000 [381:07:27<62:03:30,  8.84s/it]
 89%|████████▉ | 199723/225000 [381:07:33<54:14:20,  7.72s/it]
 89%|████████▉ | 199724/225000 [381:07:42<57:36:08,  8.20s/it]
 89%|████████▉ | 199725/225000 [381:07:51<60:06:50,  8.56s/it]
 89%|████████▉ | 199726/225000 [381:07:58<56:46:16,  8.09s/it]
 89%|████████▉ | 199727/225000 [381:08:08<59:31:01,  8.48s/it]
 89%|████████▉ | 199728/225000 [381:08:14<54:33:18,  7.77s/it]
 89%|████████▉ | 199729/225000 [381:08:23<57:32:01,  8.20s/it]
 89%|████████▉ | 199730/225000 [381:08:34<62:59:35,  8.97s/it]
 89%|████████▉ | 199731/225000 [381:08:41<58:28:43,  8.33s/it]
 89%|████████▉ | 199732/225000 [381:08:51<63:17:24,  9.02s/it]
 89%|████████▉ | 199733/225000 [381:09:02<67:53:46,  9.67s/it]
 89%|████████▉ | 199734/225000 [381:09:11<66:31:52,  9.48s/it]
 89%|████████▉ | 199735/225000 [381:09:18<61:08:14,  8.71s/it]
 89%|████████▉ | 199736/225000 [381:09:28<63:42:26,  9.08s/it]
 89%|████████▉ | 199737/225000 [381:09:36<60:48:10,  8.66s/it]
 89%|████████▉ | 199738/225000 [381:09:46<62:47:40,  8.95s/it]
 89%|████████▉ | 199739/225000 [381:09:51<55:30:33,  7.91s/it]
 89%|████████▉ | 199740/225000 [381:10:01<60:14:30,  8.59s/it]
 89%|████████▉ | 199741/225000 [381:10:08<56:28:50,  8.05s/it]
 89%|████████▉ | 199742/225000 [381:10:15<53:11:57,  7.58s/it]
 89%|████████▉ | 199743/225000 [381:10:20<48:25:38,  6.90s/it]
 89%|████████▉ | 199744/225000 [381:10:28<50:12:09,  7.16s/it]
 89%|████████▉ | 199745/225000 [381:10:35<51:11:51,  7.30s/it]
 89%|████████▉ | 199746/225000 [381:10:49<65:03:18,  9.27s/it]
 89%|████████▉ | 199747/225000 [381:11:01<70:34:13, 10.06s/it]
 89%|████████▉ | 199748/225000 [381:11:08<64:04:26,  9.13s/it]
 89%|████████▉ | 199749/225000 [381:11:14<56:57:19,  8.12s/it]
 89%|████████▉ | 199750/225000 [381:11:22<56:28:44,  8.05s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.308', 'learning_rate': '1.695e-06', 'epoch': '0.8878', 'num_input_tokens_seen': 638927800, 'train_runtime': '1.372e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 199750/225000 [381:11:22<56:28:44,  8.05s/it]
 89%|████████▉ | 199751/225000 [381:11:28<53:50:01,  7.68s/it]
 89%|████████▉ | 199752/225000 [381:11:39<59:40:49,  8.51s/it]
 89%|████████▉ | 199753/225000 [381:11:45<53:42:25,  7.66s/it]
 89%|████████▉ | 199754/225000 [381:11:55<59:57:29,  8.55s/it]
 89%|████████▉ | 199755/225000 [381:12:07<66:40:09,  9.51s/it]
 89%|████████▉ | 199756/225000 [381:12:14<62:23:10,  8.90s/it]
 89%|████████▉ | 199757/225000 [381:12:20<56:30:13,  8.06s/it]
 89%|████████▉ | 199758/225000 [381:12:26<51:30:51,  7.35s/it]
 89%|████████▉ | 199759/225000 [381:12:37<59:50:09,  8.53s/it]
 89%|████████▉ | 199760/225000 [381:12:47<61:26:07,  8.76s/it]
 89%|████████▉ | 199761/225000 [381:12:58<66:26:35,  9.48s/it]
 89%|████████▉ | 199762/225000 [381:13:05<62:08:49,  8.86s/it]
 89%|████████▉ | 199763/225000 [381:13:11<56:09:04,  8.01s/it]
 89%|████████▉ | 199764/225000 [381:13:21<60:25:43,  8.62s/it]
 89%|████████▉ | 199765/225000 [381:13:28<55:32:53,  7.92s/it]
 89%|████████▉ | 199766/225000 [381:13:36<56:22:02,  8.04s/it]
 89%|████████▉ | 199767/225000 [381:13:44<55:35:18,  7.93s/it]
 89%|████████▉ | 199768/225000 [381:13:55<61:58:04,  8.84s/it]
 89%|████████▉ | 199769/225000 [381:14:00<55:33:36,  7.93s/it]
 89%|████████▉ | 199770/225000 [381:14:07<51:56:13,  7.41s/it]
 89%|████████▉ | 199771/225000 [381:14:12<48:03:34,  6.86s/it]
 89%|████████▉ | 199772/225000 [381:14:26<63:36:13,  9.08s/it]
 89%|████████▉ | 199773/225000 [381:14:34<59:33:27,  8.50s/it]
 89%|████████▉ | 199774/225000 [381:14:44<63:20:36,  9.04s/it]
 89%|████████▉ | 199775/225000 [381:14:53<63:17:31,  9.03s/it]
 89%|████████▉ | 199776/225000 [381:14:58<54:36:36,  7.79s/it]
 89%|████████▉ | 199777/225000 [381:15:07<56:51:21,  8.11s/it]
 89%|████████▉ | 199778/225000 [381:15:14<54:18:03,  7.75s/it]
 89%|████████▉ | 199779/225000 [381:15:21<52:27:53,  7.49s/it]
 89%|████████▉ | 199780/225000 [381:15:34<64:21:11,  9.19s/it]
 89%|████████▉ | 199781/225000 [381:15:40<59:10:57,  8.45s/it]
 89%|████████▉ | 199782/225000 [381:15:50<60:37:08,  8.65s/it]
 89%|████████▉ | 199783/225000 [381:16:00<63:41:21,  9.09s/it]
 89%|████████▉ | 199784/225000 [381:16:07<60:30:05,  8.64s/it]
 89%|████████▉ | 199785/225000 [381:16:17<63:14:22,  9.03s/it]
 89%|████████▉ | 199786/225000 [381:16:28<67:11:54,  9.59s/it]
 89%|████████▉ | 199787/225000 [381:16:33<58:27:58,  8.35s/it]
 89%|████████▉ | 199788/225000 [381:16:41<56:57:16,  8.13s/it]
 89%|████████▉ | 199789/225000 [381:16:49<55:37:59,  7.94s/it]
 89%|████████▉ | 199790/225000 [381:16:57<57:00:13,  8.14s/it]
 89%|████████▉ | 199791/225000 [381:17:07<60:27:48,  8.63s/it]
 89%|████████▉ | 199792/225000 [381:17:14<57:45:34,  8.25s/it]
 89%|████████▉ | 199793/225000 [381:17:22<56:54:29,  8.13s/it]
 89%|████████▉ | 199794/225000 [381:17:29<53:14:20,  7.60s/it]
 89%|████████▉ | 199795/225000 [381:17:36<53:00:20,  7.57s/it]
 89%|████████▉ | 199796/225000 [381:17:44<54:30:41,  7.79s/it]
 89%|████████▉ | 199797/225000 [381:17:53<56:25:57,  8.06s/it]
 89%|████████▉ | 199798/225000 [381:18:02<59:12:59,  8.46s/it]
 89%|████████▉ | 199799/225000 [381:18:10<58:16:31,  8.32s/it]
 89%|████████▉ | 199800/225000 [381:18:15<50:55:18,  7.27s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.509', 'learning_rate': '1.688e-06', 'epoch': '0.888', 'num_input_tokens_seen': 639153768, 'train_runtime': '1.373e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 199800/225000 [381:18:15<50:55:18,  7.27s/it]
 89%|████████▉ | 199801/225000 [381:18:29<63:40:39,  9.10s/it]
 89%|████████▉ | 199802/225000 [381:18:36<59:04:14,  8.44s/it]
 89%|████████▉ | 199803/225000 [381:18:45<62:00:52,  8.86s/it]
 89%|████████▉ | 199804/225000 [381:18:52<57:13:37,  8.18s/it]
 89%|████████▉ | 199805/225000 [381:19:00<57:35:33,  8.23s/it]
 89%|████████▉ | 199806/225000 [381:19:10<60:54:43,  8.70s/it]
 89%|████████▉ | 199807/225000 [381:19:17<56:44:48,  8.11s/it]
 89%|████████▉ | 199808/225000 [381:19:22<50:47:13,  7.26s/it]
 89%|████████▉ | 199809/225000 [381:19:27<44:49:04,  6.40s/it]
 89%|████████▉ | 199810/225000 [381:19:32<43:49:55,  6.26s/it]
 89%|████████▉ | 199811/225000 [381:19:47<60:25:44,  8.64s/it]
 89%|████████▉ | 199812/225000 [381:19:53<54:54:16,  7.85s/it]
 89%|████████▉ | 199813/225000 [381:19:59<51:58:47,  7.43s/it]
 89%|████████▉ | 199814/225000 [381:20:09<56:28:48,  8.07s/it]
 89%|████████▉ | 199815/225000 [381:20:16<53:50:33,  7.70s/it]
 89%|████████▉ | 199816/225000 [381:20:23<52:21:57,  7.49s/it]
 89%|████████▉ | 199817/225000 [381:20:33<58:31:07,  8.37s/it]
 89%|████████▉ | 199818/225000 [381:20:44<64:47:57,  9.26s/it]
 89%|████████▉ | 199819/225000 [381:20:55<67:36:08,  9.66s/it]
 89%|████████▉ | 199820/225000 [381:21:04<67:24:58,  9.64s/it]
 89%|████████▉ | 199821/225000 [381:21:11<60:34:54,  8.66s/it]
 89%|████████▉ | 199822/225000 [381:21:22<65:40:44,  9.39s/it]
 89%|████████▉ | 199823/225000 [381:21:29<61:10:27,  8.75s/it]
 89%|████████▉ | 199824/225000 [381:21:40<66:00:55,  9.44s/it]
 89%|████████▉ | 199825/225000 [381:21:46<57:53:51,  8.28s/it]
 89%|████████▉ | 199826/225000 [381:21:55<60:33:14,  8.66s/it]
 89%|████████▉ | 199827/225000 [381:22:03<59:16:44,  8.48s/it]
 89%|████████▉ | 199828/225000 [381:22:08<50:20:29,  7.20s/it]
 89%|████████▉ | 199829/225000 [381:22:17<54:51:19,  7.85s/it]
 89%|████████▉ | 199830/225000 [381:22:23<50:22:19,  7.20s/it]
 89%|████████▉ | 199831/225000 [381:22:31<52:29:11,  7.51s/it]
 89%|████████▉ | 199832/225000 [381:22:40<55:12:57,  7.90s/it]
 89%|████████▉ | 199833/225000 [381:22:49<57:20:44,  8.20s/it]
 89%|████████▉ | 199834/225000 [381:22:54<51:10:57,  7.32s/it]
 89%|████████▉ | 199835/225000 [381:23:04<57:56:50,  8.29s/it]
 89%|████████▉ | 199836/225000 [381:23:10<52:44:41,  7.55s/it]
 89%|████████▉ | 199837/225000 [381:23:18<53:23:15,  7.64s/it]
 89%|████████▉ | 199838/225000 [381:23:28<58:41:28,  8.40s/it]
 89%|████████▉ | 199839/225000 [381:23:38<60:32:01,  8.66s/it]
 89%|████████▉ | 199840/225000 [381:23:47<61:20:47,  8.78s/it]
 89%|████████▉ | 199841/225000 [381:23:54<58:54:53,  8.43s/it]
 89%|████████▉ | 199842/225000 [381:24:01<55:44:25,  7.98s/it]
 89%|████████▉ | 199843/225000 [381:24:08<52:52:06,  7.57s/it]
 89%|████████▉ | 199844/225000 [381:24:18<58:16:49,  8.34s/it]
 89%|████████▉ | 199845/225000 [381:24:30<65:28:30,  9.37s/it]
 89%|████████▉ | 199846/225000 [381:24:37<60:46:17,  8.70s/it]
 89%|████████▉ | 199847/225000 [381:24:44<57:03:08,  8.17s/it]
 89%|████████▉ | 199848/225000 [381:24:56<66:04:21,  9.46s/it]
 89%|████████▉ | 199849/225000 [381:25:05<63:49:45,  9.14s/it]
 89%|████████▉ | 199850/225000 [381:25:11<59:04:11,  8.46s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.529', 'learning_rate': '1.681e-06', 'epoch': '0.8882', 'num_input_tokens_seen': 639376960, 'train_runtime': '1.373e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 199850/225000 [381:25:11<59:04:11,  8.46s/it]
 89%|████████▉ | 199851/225000 [381:25:18<55:10:54,  7.90s/it]
 89%|████████▉ | 199852/225000 [381:25:30<64:28:32,  9.23s/it]
 89%|████████▉ | 199853/225000 [381:25:38<60:48:56,  8.71s/it]
 89%|████████▉ | 199854/225000 [381:25:46<59:24:07,  8.50s/it]
 89%|████████▉ | 199855/225000 [381:25:55<60:29:55,  8.66s/it]
 89%|████████▉ | 199856/225000 [381:26:05<64:30:07,  9.24s/it]
 89%|████████▉ | 199857/225000 [381:26:12<57:51:34,  8.28s/it]
 89%|████████▉ | 199858/225000 [381:26:19<55:03:56,  7.88s/it]
 89%|████████▉ | 199859/225000 [381:26:31<63:59:08,  9.16s/it]
 89%|████████▉ | 199860/225000 [381:26:36<56:25:32,  8.08s/it]
 89%|████████▉ | 199861/225000 [381:26:45<57:31:42,  8.24s/it]
 89%|████████▉ | 199862/225000 [381:26:53<58:18:54,  8.35s/it]
 89%|████████▉ | 199863/225000 [381:27:03<60:35:25,  8.68s/it]
 89%|████████▉ | 199864/225000 [381:27:12<61:47:54,  8.85s/it]
 89%|████████▉ | 199865/225000 [381:27:18<55:29:59,  7.95s/it]
 89%|████████▉ | 199866/225000 [381:27:27<58:05:49,  8.32s/it]
 89%|████████▉ | 199867/225000 [381:27:36<59:14:23,  8.49s/it]
 89%|████████▉ | 199868/225000 [381:27:47<65:04:53,  9.32s/it]
 89%|████████▉ | 199869/225000 [381:27:55<60:46:18,  8.71s/it]
 89%|████████▉ | 199870/225000 [381:28:02<58:51:46,  8.43s/it]
 89%|████████▉ | 199871/225000 [381:28:08<53:20:52,  7.64s/it]
 89%|████████▉ | 199872/225000 [381:28:14<48:55:07,  7.01s/it]
 89%|████████▉ | 199873/225000 [381:28:21<50:17:18,  7.20s/it]
 89%|████████▉ | 199874/225000 [381:28:35<64:41:42,  9.27s/it]
 89%|████████▉ | 199875/225000 [381:28:42<58:12:44,  8.34s/it]
 89%|████████▉ | 199876/225000 [381:28:51<60:43:39,  8.70s/it]
 89%|████████▉ | 199877/225000 [381:28:58<56:30:06,  8.10s/it]
 89%|████████▉ | 199878/225000 [381:29:09<62:05:16,  8.90s/it]
 89%|████████▉ | 199879/225000 [381:29:19<65:02:04,  9.32s/it]
 89%|████████▉ | 199880/225000 [381:29:25<57:50:40,  8.29s/it]
 89%|████████▉ | 199881/225000 [381:29:35<61:24:06,  8.80s/it]
 89%|████████▉ | 199882/225000 [381:29:40<54:17:30,  7.78s/it]
 89%|████████▉ | 199883/225000 [381:29:49<56:52:58,  8.15s/it]
 89%|████████▉ | 199884/225000 [381:29:58<58:15:34,  8.35s/it]
 89%|████████▉ | 199885/225000 [381:30:10<66:38:45,  9.55s/it]
 89%|████████▉ | 199886/225000 [381:30:18<62:37:30,  8.98s/it]
 89%|████████▉ | 199887/225000 [381:30:24<56:12:21,  8.06s/it]
 89%|████████▉ | 199888/225000 [381:30:31<53:42:13,  7.70s/it]
 89%|████████▉ | 199889/225000 [381:30:41<59:05:29,  8.47s/it]
 89%|████████▉ | 199890/225000 [381:30:48<56:35:14,  8.11s/it]
 89%|████████▉ | 199891/225000 [381:30:56<54:38:27,  7.83s/it]
 89%|████████▉ | 199892/225000 [381:31:02<52:04:10,  7.47s/it]
 89%|████████▉ | 199893/225000 [381:31:11<55:09:55,  7.91s/it]
 89%|████████▉ | 199894/225000 [381:31:26<68:46:57,  9.86s/it]
 89%|████████▉ | 199895/225000 [381:31:33<63:51:46,  9.16s/it]
 89%|████████▉ | 199896/225000 [381:31:42<62:42:35,  8.99s/it]
 89%|████████▉ | 199897/225000 [381:31:49<59:50:49,  8.58s/it]
 89%|████████▉ | 199898/225000 [381:31:58<59:35:14,  8.55s/it]
 89%|████████▉ | 199899/225000 [381:32:03<52:30:14,  7.53s/it]
 89%|████████▉ | 199900/225000 [381:32:11<54:01:17,  7.75s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.329', 'learning_rate': '1.675e-06', 'epoch': '0.8884', 'num_input_tokens_seen': 639603528, 'train_runtime': '1.374e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 199900/225000 [381:32:11<54:01:17,  7.75s/it]
 89%|████████▉ | 199901/225000 [381:32:19<54:05:04,  7.76s/it]
 89%|████████▉ | 199902/225000 [381:32:25<51:24:15,  7.37s/it]
 89%|████████▉ | 199903/225000 [381:32:41<69:09:05,  9.92s/it]
 89%|████████▉ | 199904/225000 [381:32:50<66:37:54,  9.56s/it]
 89%|████████▉ | 199905/225000 [381:33:01<68:45:37,  9.86s/it]
 89%|████████▉ | 199906/225000 [381:33:07<61:26:46,  8.82s/it]
 89%|████████▉ | 199907/225000 [381:33:15<59:02:43,  8.47s/it]
 89%|████████▉ | 199908/225000 [381:33:26<66:08:50,  9.49s/it]
 89%|████████▉ | 199909/225000 [381:33:32<56:56:12,  8.17s/it]
 89%|████████▉ | 199910/225000 [381:33:44<64:58:41,  9.32s/it]
 89%|████████▉ | 199911/225000 [381:33:54<67:19:27,  9.66s/it]
 89%|████████▉ | 199912/225000 [381:33:58<55:31:49,  7.97s/it]
 89%|████████▉ | 199913/225000 [381:34:12<67:04:38,  9.63s/it]
 89%|████████▉ | 199914/225000 [381:34:18<59:34:01,  8.55s/it]
 89%|████████▉ | 199915/225000 [381:34:25<58:07:35,  8.34s/it]
 89%|████████▉ | 199916/225000 [381:34:30<50:30:20,  7.25s/it]
 89%|████████▉ | 199917/225000 [381:34:34<43:25:12,  6.23s/it]
 89%|████████▉ | 199918/225000 [381:34:46<55:47:39,  8.01s/it]
 89%|████████▉ | 199919/225000 [381:34:54<55:13:39,  7.93s/it]
 89%|████████▉ | 199920/225000 [381:34:59<49:53:00,  7.16s/it]
 89%|████████▉ | 199921/225000 [381:35:08<52:58:48,  7.61s/it]
 89%|████████▉ | 199922/225000 [381:35:15<52:14:58,  7.50s/it]
 89%|████████▉ | 199923/225000 [381:35:20<46:48:43,  6.72s/it]
 89%|████████▉ | 199924/225000 [381:35:26<44:31:32,  6.39s/it]
 89%|████████▉ | 199925/225000 [381:35:33<46:16:55,  6.64s/it]
 89%|████████▉ | 199926/225000 [381:35:43<54:03:27,  7.76s/it]
 89%|████████▉ | 199927/225000 [381:35:53<58:32:37,  8.41s/it]
 89%|████████▉ | 199928/225000 [381:35:58<51:22:27,  7.38s/it]
 89%|████████▉ | 199929/225000 [381:36:04<48:51:49,  7.02s/it]
 89%|████████▉ | 199930/225000 [381:36:14<53:36:06,  7.70s/it]
 89%|████████▉ | 199931/225000 [381:36:21<52:18:43,  7.51s/it]
 89%|████████▉ | 199932/225000 [381:36:26<47:43:06,  6.85s/it]
 89%|████████▉ | 199933/225000 [381:36:33<48:43:25,  7.00s/it]
 89%|████████▉ | 199934/225000 [381:36:40<48:08:21,  6.91s/it]
 89%|████████▉ | 199935/225000 [381:36:53<59:44:40,  8.58s/it]
 89%|████████▉ | 199936/225000 [381:37:02<61:39:39,  8.86s/it]
 89%|████████▉ | 199937/225000 [381:37:13<67:07:09,  9.64s/it]
 89%|████████▉ | 199938/225000 [381:37:24<68:11:13,  9.79s/it]
 89%|████████▉ | 199939/225000 [381:37:29<59:19:20,  8.52s/it]
 89%|████████▉ | 199940/225000 [381:37:36<55:20:15,  7.95s/it]
 89%|████████▉ | 199941/225000 [381:37:41<50:29:42,  7.25s/it]
 89%|████████▉ | 199942/225000 [381:37:54<61:16:23,  8.80s/it]
 89%|████████▉ | 199943/225000 [381:38:04<63:03:25,  9.06s/it]
 89%|████████▉ | 199944/225000 [381:38:10<57:21:27,  8.24s/it]
 89%|████████▉ | 199945/225000 [381:38:20<61:58:44,  8.91s/it]
 89%|████████▉ | 199946/225000 [381:38:29<61:32:40,  8.84s/it]
 89%|████████▉ | 199947/225000 [381:38:34<53:40:33,  7.71s/it]
 89%|████████▉ | 199948/225000 [381:38:46<62:45:12,  9.02s/it]
 89%|████████▉ | 199949/225000 [381:38:54<60:46:36,  8.73s/it]
 89%|████████▉ | 199950/225000 [381:39:01<57:02:36,  8.20s/it]
                                                              
{'loss': '1.283', 'grad_norm': '1.385', 'learning_rate': '1.668e-06', 'epoch': '0.8887', 'num_input_tokens_seen': 639823424, 'train_runtime': '1.374e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 199950/225000 [381:39:01<57:02:36,  8.20s/it]
 89%|████████▉ | 199951/225000 [381:39:08<54:02:15,  7.77s/it]
 89%|████████▉ | 199952/225000 [381:39:15<52:02:10,  7.48s/it]
 89%|████████▉ | 199953/225000 [381:39:26<59:46:48,  8.59s/it]
 89%|████████▉ | 199954/225000 [381:39:35<61:16:08,  8.81s/it]
 89%|████████▉ | 199955/225000 [381:39:39<51:40:42,  7.43s/it]
 89%|████████▉ | 199956/225000 [381:39:50<58:44:07,  8.44s/it]
 89%|████████▉ | 199957/225000 [381:40:02<65:32:05,  9.42s/it]
 89%|████████▉ | 199958/225000 [381:40:15<72:07:16, 10.37s/it]
 89%|████████▉ | 199959/225000 [381:40:20<61:20:49,  8.82s/it]
 89%|████████▉ | 199960/225000 [381:40:30<63:45:36,  9.17s/it]
 89%|████████▉ | 199961/225000 [381:40:39<64:33:48,  9.28s/it]
 89%|████████▉ | 199962/225000 [381:40:46<58:39:07,  8.43s/it]
 89%|████████▉ | 199963/225000 [381:40:51<51:16:52,  7.37s/it]
 89%|████████▉ | 199964/225000 [381:40:59<53:33:45,  7.70s/it]
 89%|████████▉ | 199965/225000 [381:41:06<51:20:25,  7.38s/it]
 89%|████████▉ | 199966/225000 [381:41:15<54:35:29,  7.85s/it]
 89%|████████▉ | 199967/225000 [381:41:22<53:15:47,  7.66s/it]
 89%|████████▉ | 199968/225000 [381:41:30<54:21:21,  7.82s/it]
 89%|████████▉ | 199969/225000 [381:41:39<57:14:23,  8.23s/it]
 89%|████████▉ | 199970/225000 [381:41:48<58:54:01,  8.47s/it]
 89%|████████▉ | 199971/225000 [381:41:55<55:26:32,  7.97s/it]
 89%|████████▉ | 199972/225000 [381:42:05<59:53:35,  8.61s/it]
 89%|████████▉ | 199973/225000 [381:42:12<55:22:22,  7.97s/it]
 89%|████████▉ | 199974/225000 [381:42:19<53:33:23,  7.70s/it]
 89%|████████▉ | 199975/225000 [381:42:29<57:59:39,  8.34s/it]
 89%|████████▉ | 199976/225000 [381:42:41<65:49:05,  9.47s/it]
 89%|████████▉ | 199977/225000 [381:42:50<66:14:55,  9.53s/it]
 89%|████████▉ | 199978/225000 [381:42:56<59:02:59,  8.50s/it]
 89%|████████▉ | 199979/225000 [381:43:05<59:06:48,  8.51s/it]
 89%|████████▉ | 199980/225000 [381:43:16<63:33:25,  9.14s/it]
 89%|████████▉ | 199981/225000 [381:43:25<64:18:28,  9.25s/it]
 89%|████████▉ | 199982/225000 [381:43:34<64:11:56,  9.24s/it]
 89%|████████▉ | 199983/225000 [381:43:44<66:07:51,  9.52s/it]
 89%|████████▉ | 199984/225000 [381:43:52<62:40:06,  9.02s/it]
 89%|████████▉ | 199985/225000 [381:44:02<63:12:54,  9.10s/it]
 89%|████████▉ | 199986/225000 [381:44:11<64:28:32,  9.28s/it]
 89%|████████▉ | 199987/225000 [381:44:22<68:08:16,  9.81s/it]
 89%|████████▉ | 199988/225000 [381:44:28<60:06:33,  8.65s/it]
 89%|████████▉ | 199989/225000 [381:44:37<59:42:13,  8.59s/it]
 89%|████████▉ | 199990/225000 [381:44:43<53:56:39,  7.76s/it]
 89%|████████▉ | 199991/225000 [381:44:53<59:46:04,  8.60s/it]
 89%|████████▉ | 199992/225000 [381:45:03<62:13:26,  8.96s/it]
 89%|████████▉ | 199993/225000 [381:45:08<53:43:00,  7.73s/it]
 89%|████████▉ | 199994/225000 [381:45:18<58:22:01,  8.40s/it]
 89%|████████▉ | 199995/225000 [381:45:23<52:30:40,  7.56s/it]
 89%|████████▉ | 199996/225000 [381:45:31<52:04:45,  7.50s/it]
 89%|████████▉ | 199997/225000 [381:45:40<56:19:10,  8.11s/it]
 89%|████████▉ | 199998/225000 [381:45:50<60:34:52,  8.72s/it]
 89%|████████▉ | 199999/225000 [381:45:57<56:48:36,  8.18s/it]
 89%|████████▉ | 200000/225000 [381:46:04<53:57:43,  7.77s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.078', 'learning_rate': '1.662e-06', 'epoch': '0.8889', 'num_input_tokens_seen': 640050552, 'train_runtime': '1.374e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 200000/225000 [381:46:04<53:57:43,  7.77s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                              
{'eval_loss': '1.213', 'eval_runtime': '412', 'eval_samples_per_second': '1.214', 'eval_steps_per_second': '0.607', 'epoch': '0.8889', 'num_input_tokens_seen': 640050552}

 89%|████████▉ | 200000/225000 [381:52:56<53:57:43,  7.77s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-200000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}


 89%|████████▉ | 200001/225000 [381:53:02<908:09:50, 130.78s/it]
 89%|████████▉ | 200002/225000 [381:53:08<648:47:44, 93.43s/it] 
 89%|████████▉ | 200003/225000 [381:53:14<465:51:49, 67.09s/it]
 89%|████████▉ | 200004/225000 [381:53:21<339:54:42, 48.96s/it]
 89%|████████▉ | 200005/225000 [381:53:27<251:00:20, 36.15s/it]
 89%|████████▉ | 200006/225000 [381:53:40<203:16:00, 29.28s/it]
 89%|████████▉ | 200007/225000 [381:53:46<155:07:53, 22.35s/it]
 89%|████████▉ | 200008/225000 [381:53:53<123:36:37, 17.81s/it]
 89%|████████▉ | 200009/225000 [381:54:00<100:46:44, 14.52s/it]
 89%|████████▉ | 200010/225000 [381:54:07<84:08:20, 12.12s/it] 
 89%|████████▉ | 200011/225000 [381:54:20<86:33:29, 12.47s/it]
 89%|████████▉ | 200012/225000 [381:54:27<75:50:47, 10.93s/it]
 89%|████████▉ | 200013/225000 [381:54:39<76:38:55, 11.04s/it]
 89%|████████▉ | 200014/225000 [381:54:52<80:46:07, 11.64s/it]
 89%|████████▉ | 200015/225000 [381:54:58<70:31:15, 10.16s/it]
 89%|████████▉ | 200016/225000 [381:55:04<61:14:35,  8.82s/it]
 89%|████████▉ | 200017/225000 [381:55:13<62:16:10,  8.97s/it]
 89%|████████▉ | 200018/225000 [381:55:23<62:53:21,  9.06s/it]
 89%|████████▉ | 200019/225000 [381:55:28<54:41:11,  7.88s/it]
 89%|████████▉ | 200020/225000 [381:55:34<50:33:11,  7.29s/it]
 89%|████████▉ | 200021/225000 [381:55:42<53:03:25,  7.65s/it]
 89%|████████▉ | 200022/225000 [381:55:52<56:43:36,  8.18s/it]
 89%|████████▉ | 200023/225000 [381:55:58<52:44:23,  7.60s/it]
 89%|████████▉ | 200024/225000 [381:56:11<64:31:56,  9.30s/it]
 89%|████████▉ | 200025/225000 [381:56:20<62:39:19,  9.03s/it]
 89%|████████▉ | 200026/225000 [381:56:25<54:33:51,  7.87s/it]
 89%|████████▉ | 200027/225000 [381:56:35<59:33:06,  8.58s/it]
 89%|████████▉ | 200028/225000 [381:56:46<65:09:32,  9.39s/it]
 89%|████████▉ | 200029/225000 [381:56:53<59:31:27,  8.58s/it]
 89%|████████▉ | 200030/225000 [381:56:59<53:29:02,  7.71s/it]
 89%|████████▉ | 200031/225000 [381:57:08<56:01:28,  8.08s/it]
 89%|████████▉ | 200032/225000 [381:57:15<53:51:08,  7.76s/it]
 89%|████████▉ | 200033/225000 [381:57:22<53:10:15,  7.67s/it]
 89%|████████▉ | 200034/225000 [381:57:34<61:45:36,  8.91s/it]
 89%|████████▉ | 200035/225000 [381:57:39<54:04:31,  7.80s/it]
 89%|████████▉ | 200036/225000 [381:57:53<67:46:14,  9.77s/it]
 89%|████████▉ | 200037/225000 [381:58:03<67:43:57,  9.77s/it]
 89%|████████▉ | 200038/225000 [381:58:13<68:25:47,  9.87s/it]
 89%|████████▉ | 200039/225000 [381:58:19<60:36:07,  8.74s/it]
 89%|████████▉ | 200040/225000 [381:58:24<52:15:45,  7.54s/it]
 89%|████████▉ | 200041/225000 [381:58:31<51:38:44,  7.45s/it]
 89%|████████▉ | 200042/225000 [381:58:38<48:59:28,  7.07s/it]
 89%|████████▉ | 200043/225000 [381:58:51<62:29:47,  9.02s/it]
 89%|████████▉ | 200044/225000 [381:58:59<59:45:47,  8.62s/it]
 89%|████████▉ | 200045/225000 [381:59:09<61:58:11,  8.94s/it]
 89%|████████▉ | 200046/225000 [381:59:17<61:11:46,  8.83s/it]
 89%|████████▉ | 200047/225000 [381:59:22<53:47:15,  7.76s/it]
 89%|████████▉ | 200048/225000 [381:59:33<60:19:51,  8.70s/it]
 89%|████████▉ | 200049/225000 [381:59:46<68:58:23,  9.95s/it]
 89%|████████▉ | 200050/225000 [381:59:55<66:20:40,  9.57s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.404', 'learning_rate': '1.655e-06', 'epoch': '0.8891', 'num_input_tokens_seen': 640279640, 'train_runtime': '1.375e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 200050/225000 [381:59:55<66:20:40,  9.57s/it]
 89%|████████▉ | 200051/225000 [382:00:05<67:22:53,  9.72s/it]
 89%|████████▉ | 200052/225000 [382:00:16<70:04:48, 10.11s/it]
 89%|████████▉ | 200053/225000 [382:00:23<63:55:57,  9.23s/it]
 89%|████████▉ | 200054/225000 [382:00:28<55:24:51,  8.00s/it]
 89%|████████▉ | 200055/225000 [382:00:35<53:38:18,  7.74s/it]
 89%|████████▉ | 200056/225000 [382:00:42<52:17:58,  7.55s/it]
 89%|████████▉ | 200057/225000 [382:00:49<50:02:05,  7.22s/it]
 89%|████████▉ | 200058/225000 [382:00:55<48:00:10,  6.93s/it]
 89%|████████▉ | 200059/225000 [382:01:09<63:11:18,  9.12s/it]
 89%|████████▉ | 200060/225000 [382:01:14<54:43:14,  7.90s/it]
 89%|████████▉ | 200061/225000 [382:01:21<52:33:50,  7.59s/it]
 89%|████████▉ | 200062/225000 [382:01:29<53:36:16,  7.74s/it]
 89%|████████▉ | 200063/225000 [382:01:37<54:02:08,  7.80s/it]
 89%|████████▉ | 200064/225000 [382:01:48<59:01:59,  8.52s/it]
 89%|████████▉ | 200065/225000 [382:01:54<54:05:42,  7.81s/it]
 89%|████████▉ | 200066/225000 [382:02:04<59:53:47,  8.65s/it]
 89%|████████▉ | 200067/225000 [382:02:14<61:24:33,  8.87s/it]
 89%|████████▉ | 200068/225000 [382:02:21<59:02:15,  8.52s/it]
 89%|████████▉ | 200069/225000 [382:02:28<55:53:00,  8.07s/it]
 89%|████████▉ | 200070/225000 [382:02:35<52:01:04,  7.51s/it]
 89%|████████▉ | 200071/225000 [382:02:47<61:51:37,  8.93s/it]
 89%|████████▉ | 200072/225000 [382:02:52<54:13:27,  7.83s/it]
 89%|████████▉ | 200073/225000 [382:03:02<58:48:54,  8.49s/it]
 89%|████████▉ | 200074/225000 [382:03:10<58:13:22,  8.41s/it]
 89%|████████▉ | 200075/225000 [382:03:15<49:33:36,  7.16s/it]
 89%|████████▉ | 200076/225000 [382:03:26<58:36:19,  8.46s/it]
 89%|████████▉ | 200077/225000 [382:03:31<51:10:57,  7.39s/it]
 89%|████████▉ | 200078/225000 [382:03:44<61:57:36,  8.95s/it]
 89%|████████▉ | 200079/225000 [382:03:50<56:57:15,  8.23s/it]
 89%|████████▉ | 200080/225000 [382:03:57<55:04:47,  7.96s/it]
 89%|████████▉ | 200081/225000 [382:04:08<60:26:48,  8.73s/it]
 89%|████████▉ | 200082/225000 [382:04:14<54:04:54,  7.81s/it]
 89%|████████▉ | 200083/225000 [382:04:20<50:36:57,  7.31s/it]
 89%|████████▉ | 200084/225000 [382:04:27<50:38:47,  7.32s/it]
 89%|████████▉ | 200085/225000 [382:04:34<50:18:57,  7.27s/it]
 89%|████████▉ | 200086/225000 [382:04:49<65:20:25,  9.44s/it]
 89%|████████▉ | 200087/225000 [382:04:54<57:08:32,  8.26s/it]
 89%|████████▉ | 200088/225000 [382:05:05<61:20:17,  8.86s/it]
 89%|████████▉ | 200089/225000 [382:05:12<59:05:36,  8.54s/it]
 89%|████████▉ | 200090/225000 [382:05:20<57:18:39,  8.28s/it]
 89%|████████▉ | 200091/225000 [382:05:29<57:44:24,  8.34s/it]
 89%|████████▉ | 200092/225000 [382:05:37<57:23:35,  8.30s/it]
 89%|████████▉ | 200093/225000 [382:05:45<57:44:59,  8.35s/it]
 89%|████████▉ | 200094/225000 [382:05:52<55:34:24,  8.03s/it]
 89%|████████▉ | 200095/225000 [382:05:59<51:54:34,  7.50s/it]
 89%|████████▉ | 200096/225000 [382:06:12<63:30:13,  9.18s/it]
 89%|████████▉ | 200097/225000 [382:06:18<56:24:02,  8.15s/it]
 89%|████████▉ | 200098/225000 [382:06:29<62:43:59,  9.07s/it]
 89%|████████▉ | 200099/225000 [382:06:38<62:41:14,  9.06s/it]
 89%|████████▉ | 200100/225000 [382:06:50<68:31:19,  9.91s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.24', 'learning_rate': '1.649e-06', 'epoch': '0.8893', 'num_input_tokens_seen': 640505312, 'train_runtime': '1.376e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 200100/225000 [382:06:50<68:31:19,  9.91s/it]
 89%|████████▉ | 200101/225000 [382:07:00<68:44:48,  9.94s/it]
 89%|████████▉ | 200102/225000 [382:07:05<59:55:34,  8.66s/it]
 89%|████████▉ | 200103/225000 [382:07:10<52:05:04,  7.53s/it]
 89%|████████▉ | 200104/225000 [382:07:18<51:27:42,  7.44s/it]
 89%|████████▉ | 200105/225000 [382:07:27<55:34:27,  8.04s/it]
 89%|████████▉ | 200106/225000 [382:07:38<61:16:26,  8.86s/it]
 89%|████████▉ | 200107/225000 [382:07:50<68:18:39,  9.88s/it]
 89%|████████▉ | 200108/225000 [382:07:58<64:54:34,  9.39s/it]
 89%|████████▉ | 200109/225000 [382:08:06<62:01:31,  8.97s/it]
 89%|████████▉ | 200110/225000 [382:08:21<74:12:53, 10.73s/it]
 89%|████████▉ | 200111/225000 [382:08:26<62:28:05,  9.04s/it]
 89%|████████▉ | 200112/225000 [382:08:33<56:59:37,  8.24s/it]
 89%|████████▉ | 200113/225000 [382:08:39<53:05:27,  7.68s/it]
 89%|████████▉ | 200114/225000 [382:08:54<68:53:14,  9.97s/it]
 89%|████████▉ | 200115/225000 [382:09:08<76:15:30, 11.03s/it]
 89%|████████▉ | 200116/225000 [382:09:16<69:34:45, 10.07s/it]
 89%|████████▉ | 200117/225000 [382:09:29<75:36:48, 10.94s/it]
 89%|████████▉ | 200118/225000 [382:09:36<69:09:32, 10.01s/it]
 89%|████████▉ | 200119/225000 [382:09:42<60:30:32,  8.75s/it]
 89%|████████▉ | 200120/225000 [382:09:55<68:06:34,  9.86s/it]
 89%|████████▉ | 200121/225000 [382:10:03<64:48:40,  9.38s/it]
 89%|████████▉ | 200122/225000 [382:10:09<58:00:22,  8.39s/it]
 89%|████████▉ | 200123/225000 [382:10:15<53:02:04,  7.67s/it]
 89%|████████▉ | 200124/225000 [382:10:21<49:54:11,  7.22s/it]
 89%|████████▉ | 200125/225000 [382:10:33<59:43:55,  8.64s/it]
 89%|████████▉ | 200126/225000 [382:10:39<54:54:54,  7.95s/it]
 89%|████████▉ | 200127/225000 [382:10:45<50:22:48,  7.29s/it]
 89%|████████▉ | 200128/225000 [382:10:56<58:17:34,  8.44s/it]
 89%|████████▉ | 200129/225000 [382:11:06<60:35:01,  8.77s/it]
 89%|████████▉ | 200130/225000 [382:11:13<57:12:05,  8.28s/it]
 89%|████████▉ | 200131/225000 [382:11:20<54:08:57,  7.84s/it]
 89%|████████▉ | 200132/225000 [382:11:27<52:41:26,  7.63s/it]
 89%|████████▉ | 200133/225000 [382:11:38<59:29:12,  8.61s/it]
 89%|████████▉ | 200134/225000 [382:11:48<62:09:23,  9.00s/it]
 89%|████████▉ | 200135/225000 [382:11:55<58:56:54,  8.53s/it]
 89%|████████▉ | 200136/225000 [382:12:02<56:08:51,  8.13s/it]
 89%|████████▉ | 200137/225000 [382:12:15<66:21:45,  9.61s/it]
 89%|████████▉ | 200138/225000 [382:12:26<68:33:54,  9.93s/it]
 89%|████████▉ | 200139/225000 [382:12:32<60:56:57,  8.83s/it]
 89%|████████▉ | 200140/225000 [382:12:40<58:21:50,  8.45s/it]
 89%|████████▉ | 200141/225000 [382:12:49<58:39:24,  8.49s/it]
 89%|████████▉ | 200142/225000 [382:12:57<58:08:08,  8.42s/it]
 89%|████████▉ | 200143/225000 [382:13:02<51:01:04,  7.39s/it]
 89%|████████▉ | 200144/225000 [382:13:08<49:37:52,  7.19s/it]
 89%|████████▉ | 200145/225000 [382:13:18<54:19:36,  7.87s/it]
 89%|████████▉ | 200146/225000 [382:13:26<54:35:35,  7.91s/it]
 89%|████████▉ | 200147/225000 [382:13:36<59:28:18,  8.61s/it]
 89%|████████▉ | 200148/225000 [382:13:51<72:10:55, 10.46s/it]
 89%|████████▉ | 200149/225000 [382:14:02<73:49:14, 10.69s/it]
 89%|████████▉ | 200150/225000 [382:14:07<61:16:19,  8.88s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.268', 'learning_rate': '1.642e-06', 'epoch': '0.8896', 'num_input_tokens_seen': 640744856, 'train_runtime': '1.376e+06', 'train_tokens_per_second': '465.6'}

 89%|████████▉ | 200150/225000 [382:14:07<61:16:19,  8.88s/it]
 89%|████████▉ | 200151/225000 [382:14:14<57:30:38,  8.33s/it]
 89%|████████▉ | 200152/225000 [382:14:20<53:47:21,  7.79s/it]
 89%|████████▉ | 200153/225000 [382:14:29<54:40:04,  7.92s/it]
 89%|████████▉ | 200154/225000 [382:14:41<64:22:07,  9.33s/it]
 89%|████████▉ | 200155/225000 [382:14:46<55:44:32,  8.08s/it]
 89%|████████▉ | 200156/225000 [382:14:52<51:31:14,  7.47s/it]
 89%|████████▉ | 200157/225000 [382:15:04<58:54:56,  8.54s/it]
 89%|████████▉ | 200158/225000 [382:15:11<57:00:55,  8.26s/it]
 89%|████████▉ | 200159/225000 [382:15:21<59:33:30,  8.63s/it]
 89%|████████▉ | 200160/225000 [382:15:26<51:49:39,  7.51s/it]
 89%|████████▉ | 200161/225000 [382:15:32<48:44:06,  7.06s/it]
 89%|████████▉ | 200162/225000 [382:15:43<57:58:06,  8.40s/it]
 89%|████████▉ | 200163/225000 [382:15:51<57:01:59,  8.27s/it]
 89%|████████▉ | 200164/225000 [382:15:58<53:46:39,  7.80s/it]
 89%|████████▉ | 200165/225000 [382:16:03<48:29:11,  7.03s/it]
 89%|████████▉ | 200166/225000 [382:16:13<54:34:05,  7.91s/it]
 89%|████████▉ | 200167/225000 [382:16:20<52:37:08,  7.63s/it]
 89%|████████▉ | 200168/225000 [382:16:26<49:03:30,  7.11s/it]
 89%|████████▉ | 200169/225000 [382:16:32<47:51:49,  6.94s/it]
 89%|████████▉ | 200170/225000 [382:16:39<47:35:47,  6.90s/it]
 89%|████████▉ | 200171/225000 [382:16:45<45:34:01,  6.61s/it]
 89%|████████▉ | 200172/225000 [382:16:54<49:55:23,  7.24s/it]
 89%|████████▉ | 200173/225000 [382:17:09<65:35:58,  9.51s/it]
 89%|████████▉ | 200174/225000 [382:17:18<65:29:48,  9.50s/it]
 89%|████████▉ | 200175/225000 [382:17:24<59:02:50,  8.56s/it]
 89%|████████▉ | 200176/225000 [382:17:37<67:59:09,  9.86s/it]
 89%|████████▉ | 200177/225000 [382:17:47<67:08:00,  9.74s/it]
 89%|████████▉ | 200178/225000 [382:17:55<63:21:13,  9.19s/it]
 89%|████████▉ | 200179/225000 [382:18:01<58:19:41,  8.46s/it]
 89%|████████▉ | 200180/225000 [382:18:09<56:40:50,  8.22s/it]
 89%|████████▉ | 200181/225000 [382:18:15<52:38:06,  7.63s/it]
 89%|████████▉ | 200182/225000 [382:18:28<62:43:26,  9.10s/it]
 89%|████████▉ | 200183/225000 [382:18:39<66:51:52,  9.70s/it]
 89%|████████▉ | 200184/225000 [382:18:48<66:02:25,  9.58s/it]
 89%|████████▉ | 200185/225000 [382:18:53<56:52:30,  8.25s/it]
 89%|████████▉ | 200186/225000 [382:19:01<56:10:38,  8.15s/it]
 89%|████████▉ | 200187/225000 [382:19:15<68:00:27,  9.87s/it]
 89%|████████▉ | 200188/225000 [382:19:24<66:20:00,  9.62s/it]
 89%|████████▉ | 200189/225000 [382:19:31<61:15:22,  8.89s/it]
 89%|████████▉ | 200190/225000 [382:19:41<62:14:02,  9.03s/it]
 89%|████████▉ | 200191/225000 [382:19:49<60:29:40,  8.78s/it]
 89%|████████▉ | 200192/225000 [382:19:57<58:18:32,  8.46s/it]
 89%|████████▉ | 200193/225000 [382:20:05<58:17:26,  8.46s/it]
 89%|████████▉ | 200194/225000 [382:20:13<57:57:03,  8.41s/it]
 89%|████████▉ | 200195/225000 [382:20:20<53:49:58,  7.81s/it]
 89%|████████▉ | 200196/225000 [382:20:32<61:53:28,  8.98s/it]
 89%|████████▉ | 200197/225000 [382:20:37<54:32:38,  7.92s/it]
 89%|████████▉ | 200198/225000 [382:20:44<52:36:12,  7.64s/it]
 89%|████████▉ | 200199/225000 [382:20:51<51:35:24,  7.49s/it]
 89%|████████▉ | 200200/225000 [382:21:02<59:09:29,  8.59s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.11', 'learning_rate': '1.636e-06', 'epoch': '0.8898', 'num_input_tokens_seen': 640969792, 'train_runtime': '1.376e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 200200/225000 [382:21:02<59:09:29,  8.59s/it]
 89%|████████▉ | 200201/225000 [382:21:15<67:10:22,  9.75s/it]
 89%|████████▉ | 200202/225000 [382:21:23<63:10:18,  9.17s/it]
 89%|████████▉ | 200203/225000 [382:21:30<59:49:36,  8.69s/it]
 89%|████████▉ | 200204/225000 [382:21:39<59:39:18,  8.66s/it]
 89%|████████▉ | 200205/225000 [382:21:50<64:40:16,  9.39s/it]
 89%|████████▉ | 200206/225000 [382:21:58<61:41:41,  8.96s/it]
 89%|████████▉ | 200207/225000 [382:22:10<68:59:38, 10.02s/it]
 89%|████████▉ | 200208/225000 [382:22:16<60:58:06,  8.85s/it]
 89%|████████▉ | 200209/225000 [382:22:26<62:09:33,  9.03s/it]
 89%|████████▉ | 200210/225000 [382:22:36<63:56:21,  9.29s/it]
 89%|████████▉ | 200211/225000 [382:22:44<61:11:36,  8.89s/it]
 89%|████████▉ | 200212/225000 [382:22:48<52:15:40,  7.59s/it]
 89%|████████▉ | 200213/225000 [382:22:55<50:30:51,  7.34s/it]
 89%|████████▉ | 200214/225000 [382:23:09<64:49:20,  9.42s/it]
 89%|████████▉ | 200215/225000 [382:23:15<56:54:02,  8.26s/it]
 89%|████████▉ | 200216/225000 [382:23:26<63:07:57,  9.17s/it]
 89%|████████▉ | 200217/225000 [382:23:35<62:19:51,  9.05s/it]
 89%|████████▉ | 200218/225000 [382:23:42<57:59:10,  8.42s/it]
 89%|████████▉ | 200219/225000 [382:23:48<52:21:24,  7.61s/it]
 89%|████████▉ | 200220/225000 [382:23:57<56:55:58,  8.27s/it]
 89%|████████▉ | 200221/225000 [382:24:10<66:39:35,  9.68s/it]
 89%|████████▉ | 200222/225000 [382:24:20<66:46:46,  9.70s/it]
 89%|████████▉ | 200223/225000 [382:24:26<57:51:42,  8.41s/it]
 89%|████████▉ | 200224/225000 [382:24:31<51:43:27,  7.52s/it]
 89%|████████▉ | 200225/225000 [382:24:40<54:16:39,  7.89s/it]
 89%|████████▉ | 200226/225000 [382:24:52<62:36:59,  9.10s/it]
 89%|████████▉ | 200227/225000 [382:24:59<58:23:08,  8.48s/it]
 89%|████████▉ | 200228/225000 [382:25:05<53:23:16,  7.76s/it]
 89%|████████▉ | 200229/225000 [382:25:11<50:33:52,  7.35s/it]
 89%|████████▉ | 200230/225000 [382:25:19<51:38:32,  7.51s/it]
 89%|████████▉ | 200231/225000 [382:25:25<47:42:33,  6.93s/it]
 89%|████████▉ | 200232/225000 [382:25:36<55:53:43,  8.12s/it]
 89%|████████▉ | 200233/225000 [382:25:44<56:57:12,  8.28s/it]
 89%|████████▉ | 200234/225000 [382:25:55<62:00:03,  9.01s/it]
 89%|████████▉ | 200235/225000 [382:26:00<54:13:52,  7.88s/it]
 89%|████████▉ | 200236/225000 [382:26:10<58:35:41,  8.52s/it]
 89%|████████▉ | 200237/225000 [382:26:15<50:46:58,  7.38s/it]
 89%|████████▉ | 200238/225000 [382:26:23<53:10:01,  7.73s/it]
 89%|████████▉ | 200239/225000 [382:26:39<70:16:46, 10.22s/it]
 89%|████████▉ | 200240/225000 [382:26:47<65:17:07,  9.49s/it]
 89%|████████▉ | 200241/225000 [382:26:54<60:28:03,  8.79s/it]
 89%|████████▉ | 200242/225000 [382:27:06<66:02:50,  9.60s/it]
 89%|████████▉ | 200243/225000 [382:27:13<61:14:47,  8.91s/it]
 89%|████████▉ | 200244/225000 [382:27:23<62:38:04,  9.11s/it]
 89%|████████▉ | 200245/225000 [382:27:32<63:39:49,  9.26s/it]
 89%|████████▉ | 200246/225000 [382:27:41<61:53:56,  9.00s/it]
 89%|████████▉ | 200247/225000 [382:27:50<63:15:52,  9.20s/it]
 89%|████████▉ | 200248/225000 [382:27:57<56:55:47,  8.28s/it]
 89%|████████▉ | 200249/225000 [382:28:05<56:45:06,  8.25s/it]
 89%|████████▉ | 200250/225000 [382:28:14<59:31:34,  8.66s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.343', 'learning_rate': '1.629e-06', 'epoch': '0.89', 'num_input_tokens_seen': 641199256, 'train_runtime': '1.377e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 200250/225000 [382:28:14<59:31:34,  8.66s/it]
 89%|████████▉ | 200251/225000 [382:28:23<60:22:14,  8.78s/it]
 89%|████████▉ | 200252/225000 [382:28:34<64:23:06,  9.37s/it]
 89%|████████▉ | 200253/225000 [382:28:43<64:09:13,  9.33s/it]
 89%|████████▉ | 200254/225000 [382:28:50<59:17:09,  8.62s/it]
 89%|████████▉ | 200255/225000 [382:29:01<64:09:06,  9.33s/it]
 89%|████████▉ | 200256/225000 [382:29:07<55:38:31,  8.10s/it]
 89%|████████▉ | 200257/225000 [382:29:13<51:13:02,  7.45s/it]
 89%|████████▉ | 200258/225000 [382:29:24<59:24:51,  8.64s/it]
 89%|████████▉ | 200259/225000 [382:29:30<53:16:04,  7.75s/it]
 89%|████████▉ | 200260/225000 [382:29:42<63:48:28,  9.28s/it]
 89%|████████▉ | 200261/225000 [382:29:52<63:25:34,  9.23s/it]
 89%|████████▉ | 200262/225000 [382:29:58<57:32:32,  8.37s/it]
 89%|████████▉ | 200263/225000 [382:30:05<54:43:10,  7.96s/it]
 89%|████████▉ | 200264/225000 [382:30:15<58:23:39,  8.50s/it]
 89%|████████▉ | 200265/225000 [382:30:19<50:26:57,  7.34s/it]
 89%|████████▉ | 200266/225000 [382:30:29<55:52:36,  8.13s/it]
 89%|████████▉ | 200267/225000 [382:30:41<63:48:10,  9.29s/it]
 89%|████████▉ | 200268/225000 [382:30:49<60:08:03,  8.75s/it]
 89%|████████▉ | 200269/225000 [382:30:54<52:01:50,  7.57s/it]
 89%|████████▉ | 200270/225000 [382:31:03<56:13:02,  8.18s/it]
 89%|████████▉ | 200271/225000 [382:31:12<57:45:17,  8.41s/it]
 89%|████████▉ | 200272/225000 [382:31:20<56:21:52,  8.21s/it]
 89%|████████▉ | 200273/225000 [382:31:26<51:39:20,  7.52s/it]
 89%|████████▉ | 200274/225000 [382:31:31<47:42:33,  6.95s/it]
 89%|████████▉ | 200275/225000 [382:31:37<45:39:49,  6.65s/it]
 89%|████████▉ | 200276/225000 [382:31:43<44:02:00,  6.41s/it]
 89%|████████▉ | 200277/225000 [382:31:53<51:30:15,  7.50s/it]
 89%|████████▉ | 200278/225000 [382:32:04<57:37:36,  8.39s/it]
 89%|████████▉ | 200279/225000 [382:32:13<59:51:47,  8.72s/it]
 89%|████████▉ | 200280/225000 [382:32:21<57:55:25,  8.44s/it]
 89%|████████▉ | 200281/225000 [382:32:26<50:27:48,  7.35s/it]
 89%|████████▉ | 200282/225000 [382:32:36<55:23:04,  8.07s/it]
 89%|████████▉ | 200283/225000 [382:32:47<62:09:38,  9.05s/it]
 89%|████████▉ | 200284/225000 [382:32:59<69:19:41, 10.10s/it]
 89%|████████▉ | 200285/225000 [382:33:05<59:42:12,  8.70s/it]
 89%|████████▉ | 200286/225000 [382:33:14<60:05:08,  8.75s/it]
 89%|████████▉ | 200287/225000 [382:33:23<60:12:31,  8.77s/it]
 89%|████████▉ | 200288/225000 [382:33:29<55:56:55,  8.15s/it]
 89%|████████▉ | 200289/225000 [382:33:39<58:27:31,  8.52s/it]
 89%|████████▉ | 200290/225000 [382:33:46<56:57:45,  8.30s/it]
 89%|████████▉ | 200291/225000 [382:33:53<52:55:38,  7.71s/it]
 89%|████████▉ | 200292/225000 [382:34:06<65:00:22,  9.47s/it]
 89%|████████▉ | 200293/225000 [382:34:13<59:19:07,  8.64s/it]
 89%|████████▉ | 200294/225000 [382:34:25<66:39:36,  9.71s/it]
 89%|████████▉ | 200295/225000 [382:34:36<67:54:56,  9.90s/it]
 89%|████████▉ | 200296/225000 [382:34:42<59:46:47,  8.71s/it]
 89%|████████▉ | 200297/225000 [382:34:51<60:43:16,  8.85s/it]
 89%|████████▉ | 200298/225000 [382:34:55<50:34:27,  7.37s/it]
 89%|████████▉ | 200299/225000 [382:35:04<54:18:15,  7.91s/it]
 89%|████████▉ | 200300/225000 [382:35:10<50:42:47,  7.39s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.102', 'learning_rate': '1.623e-06', 'epoch': '0.8902', 'num_input_tokens_seen': 641426440, 'train_runtime': '1.377e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 200300/225000 [382:35:10<50:42:47,  7.39s/it]
 89%|████████▉ | 200301/225000 [382:35:21<57:25:34,  8.37s/it]
 89%|████████▉ | 200302/225000 [382:35:26<51:15:04,  7.47s/it]
 89%|████████▉ | 200303/225000 [382:35:31<46:03:36,  6.71s/it]
 89%|████████▉ | 200304/225000 [382:35:44<59:05:16,  8.61s/it]
 89%|████████▉ | 200305/225000 [382:35:51<55:15:59,  8.06s/it]
 89%|████████▉ | 200306/225000 [382:35:56<49:49:27,  7.26s/it]
 89%|████████▉ | 200307/225000 [382:36:03<48:27:16,  7.06s/it]
 89%|████████▉ | 200308/225000 [382:36:15<58:43:51,  8.56s/it]
 89%|████████▉ | 200309/225000 [382:36:25<62:34:05,  9.12s/it]
 89%|████████▉ | 200310/225000 [382:36:31<56:34:17,  8.25s/it]
 89%|████████▉ | 200311/225000 [382:36:38<52:53:36,  7.71s/it]
 89%|████████▉ | 200312/225000 [382:36:46<53:01:14,  7.73s/it]
 89%|████████▉ | 200313/225000 [382:36:59<64:10:30,  9.36s/it]
 89%|████████▉ | 200314/225000 [382:37:06<58:56:59,  8.60s/it]
 89%|████████▉ | 200315/225000 [382:37:14<57:30:00,  8.39s/it]
 89%|████████▉ | 200316/225000 [382:37:22<58:23:45,  8.52s/it]
 89%|████████▉ | 200317/225000 [382:37:31<57:45:43,  8.42s/it]
 89%|████████▉ | 200318/225000 [382:37:36<50:53:22,  7.42s/it]
 89%|████████▉ | 200319/225000 [382:37:48<61:55:47,  9.03s/it]
 89%|████████▉ | 200320/225000 [382:37:56<59:31:41,  8.68s/it]
 89%|████████▉ | 200321/225000 [382:38:05<60:22:28,  8.81s/it]
 89%|████████▉ | 200322/225000 [382:38:12<55:55:11,  8.16s/it]
 89%|████████▉ | 200323/225000 [382:38:20<55:11:37,  8.05s/it]
 89%|████████▉ | 200324/225000 [382:38:27<52:55:19,  7.72s/it]
 89%|████████▉ | 200325/225000 [382:38:40<63:36:20,  9.28s/it]
 89%|████████▉ | 200326/225000 [382:38:46<57:16:40,  8.36s/it]
 89%|████████▉ | 200327/225000 [382:38:54<57:04:16,  8.33s/it]
 89%|████████▉ | 200328/225000 [382:39:02<55:36:10,  8.11s/it]
 89%|████████▉ | 200329/225000 [382:39:09<53:23:27,  7.79s/it]
 89%|████████▉ | 200330/225000 [382:39:22<63:59:27,  9.34s/it]
 89%|████████▉ | 200331/225000 [382:39:33<66:55:17,  9.77s/it]
 89%|████████▉ | 200332/225000 [382:39:42<66:56:34,  9.77s/it]
 89%|████████▉ | 200333/225000 [382:39:48<57:27:11,  8.38s/it]
 89%|████████▉ | 200334/225000 [382:39:57<59:14:25,  8.65s/it]
 89%|████████▉ | 200335/225000 [382:40:08<65:06:27,  9.50s/it]
 89%|████████▉ | 200336/225000 [382:40:13<55:05:43,  8.04s/it]
 89%|████████▉ | 200337/225000 [382:40:19<51:59:01,  7.59s/it]
 89%|████████▉ | 200338/225000 [382:40:25<47:27:51,  6.93s/it]
 89%|████████▉ | 200339/225000 [382:40:41<65:27:10,  9.55s/it]
 89%|████████▉ | 200340/225000 [382:40:47<58:20:10,  8.52s/it]
 89%|████████▉ | 200341/225000 [382:40:54<55:49:47,  8.15s/it]
 89%|████████▉ | 200342/225000 [382:41:01<54:39:36,  7.98s/it]
 89%|████████▉ | 200343/225000 [382:41:11<57:39:48,  8.42s/it]
 89%|████████▉ | 200344/225000 [382:41:22<64:00:12,  9.35s/it]
 89%|████████▉ | 200345/225000 [382:41:31<61:52:40,  9.04s/it]
 89%|████████▉ | 200346/225000 [382:41:40<62:12:54,  9.08s/it]
 89%|████████▉ | 200347/225000 [382:41:47<57:13:17,  8.36s/it]
 89%|████████▉ | 200348/225000 [382:41:53<53:11:48,  7.77s/it]
 89%|████████▉ | 200349/225000 [382:42:02<55:56:13,  8.17s/it]
 89%|████████▉ | 200350/225000 [382:42:08<51:34:14,  7.53s/it]
                                                              
{'loss': '1.29', 'grad_norm': '2.247', 'learning_rate': '1.616e-06', 'epoch': '0.8904', 'num_input_tokens_seen': 641645960, 'train_runtime': '1.378e+06', 'train_tokens_per_second': '465.7'}

 89%|████████▉ | 200350/225000 [382:42:08<51:34:14,  7.53s/it]
 89%|████████▉ | 200351/225000 [382:42:18<55:41:37,  8.13s/it]
 89%|████████▉ | 200352/225000 [382:42:24<52:08:23,  7.62s/it]
 89%|████████▉ | 200353/225000 [382:42:30<47:43:23,  6.97s/it]
 89%|████████▉ | 200354/225000 [382:42:35<44:43:16,  6.53s/it]
 89%|████████▉ | 200355/225000 [382:42:49<59:36:00,  8.71s/it]
 89%|████████▉ | 200356/225000 [382:42:58<60:53:53,  8.90s/it]
 89%|████████▉ | 200357/225000 [382:43:06<58:09:00,  8.49s/it]
 89%|████████▉ | 200358/225000 [382:43:13<54:37:46,  7.98s/it]
 89%|████████▉ | 200359/225000 [382:43:20<52:37:23,  7.69s/it]
 89%|████████▉ | 200360/225000 [382:43:30<59:11:23,  8.65s/it]
 89%|████████▉ | 200361/225000 [382:43:41<63:39:36,  9.30s/it]
 89%|████████▉ | 200362/225000 [382:43:48<58:11:37,  8.50s/it]
 89%|████████▉ | 200363/225000 [382:43:57<58:39:24,  8.57s/it]
 89%|████████▉ | 200364/225000 [382:44:02<52:50:08,  7.72s/it]
 89%|████████▉ | 200365/225000 [382:44:16<64:39:24,  9.45s/it]
 89%|████████▉ | 200366/225000 [382:44:22<57:55:15,  8.46s/it]
 89%|████████▉ | 200367/225000 [382:44:29<54:33:06,  7.97s/it]
 89%|████████▉ | 200368/225000 [382:44:43<67:01:42,  9.80s/it]
 89%|████████▉ | 200369/225000 [382:44:51<62:50:07,  9.18s/it]
 89%|████████▉ | 200370/225000 [382:44:57<56:27:45,  8.25s/it]
 89%|████████▉ | 200371/225000 [382:45:03<53:02:52,  7.75s/it]
 89%|████████▉ | 200372/225000 [382:45:15<61:57:00,  9.06s/it]
 89%|████████▉ | 200373/225000 [382:45:21<54:56:47,  8.03s/it]
 89%|████████▉ | 200374/225000 [382:45:31<59:35:47,  8.71s/it]
 89%|████████▉ | 200375/225000 [382:45:36<51:33:36,  7.54s/it]
 89%|████████▉ | 200376/225000 [382:45:49<62:17:38,  9.11s/it]
 89%|████████▉ | 200377/225000 [382:45:56<57:17:23,  8.38s/it]
 89%|████████▉ | 200378/225000 [382:46:03<56:08:05,  8.21s/it]
 89%|████████▉ | 200379/225000 [382:46:14<61:30:24,  8.99s/it]
 89%|████████▉ | 200380/225000 [382:46:25<64:30:32,  9.43s/it]
 89%|████████▉ | 200381/225000 [382:46:33<62:10:29,  9.09s/it]
 89%|████████▉ | 200382/225000 [382:46:42<62:34:15,  9.15s/it]
 89%|████████▉ | 200383/225000 [382:46:49<56:37:19,  8.28s/it]
 89%|████████▉ | 200384/225000 [382:46:58<58:22:39,  8.54s/it]
 89%|████████▉ | 200385/225000 [382:47:06<57:46:08,  8.45s/it]
 89%|████████▉ | 200386/225000 [382:47:17<62:19:18,  9.12s/it]
 89%|████████▉ | 200387/225000 [382:47:25<60:49:43,  8.90s/it]
 89%|████████▉ | 200388/225000 [382:47:30<52:45:59,  7.72s/it]
 89%|████████▉ | 200389/225000 [382:47:41<59:14:49,  8.67s/it]
 89%|████████▉ | 200390/225000 [382:47:47<53:18:15,  7.80s/it]
 89%|████████▉ | 200391/225000 [382:47:58<59:58:11,  8.77s/it]
 89%|████████▉ | 200392/225000 [382:48:06<58:39:40,  8.58s/it]
 89%|████████▉ | 200393/225000 [382:48:12<54:32:27,  7.98s/it]
 89%|████████▉ | 200394/225000 [382:48:25<65:05:23,  9.52s/it]
 89%|████████▉ | 200395/225000 [382:48:36<67:26:55,  9.87s/it]
 89%|████████▉ | 200396/225000 [382:48:42<59:38:31,  8.73s/it]
 89%|████████▉ | 200397/225000 [382:48:53<64:04:53,  9.38s/it]
 89%|████████▉ | 200398/225000 [382:49:04<67:56:18,  9.94s/it]
 89%|████████▉ | 200399/225000 [382:49:14<67:15:18,  9.84s/it]
 89%|████████▉ | 200400/225000 [382:49:22<63:04:21,  9.23s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.696', 'learning_rate': '1.61e-06', 'epoch': '0.8907', 'num_input_tokens_seen': 641878960, 'train_runtime': '1.378e+06', 'train_tokens_per_second': '465.8'}

 89%|████████▉ | 200400/225000 [382:49:22<63:04:21,  9.23s/it]
 89%|████████▉ | 200401/225000 [382:49:33<67:52:43,  9.93s/it]
 89%|████████▉ | 200402/225000 [382:49:44<70:07:13, 10.26s/it]
 89%|████████▉ | 200403/225000 [382:49:55<70:10:03, 10.27s/it]
 89%|████████▉ | 200404/225000 [382:50:03<66:02:45,  9.67s/it]
 89%|████████▉ | 200405/225000 [382:50:10<60:18:13,  8.83s/it]
 89%|████████▉ | 200406/225000 [382:50:18<59:51:32,  8.76s/it]
 89%|████████▉ | 200407/225000 [382:50:25<56:01:52,  8.20s/it]
 89%|████████▉ | 200408/225000 [382:50:32<52:16:48,  7.65s/it]
 89%|████████▉ | 200409/225000 [382:50:46<65:55:51,  9.65s/it]
 89%|████████▉ | 200410/225000 [382:50:55<65:10:22,  9.54s/it]
 89%|████████▉ | 200411/225000 [382:51:08<71:16:03, 10.43s/it]
 89%|████████▉ | 200412/225000 [382:51:15<64:50:04,  9.49s/it]
 89%|████████▉ | 200413/225000 [382:51:24<64:28:10,  9.44s/it]
 89%|████████▉ | 200414/225000 [382:51:38<72:36:57, 10.63s/it]
 89%|████████▉ | 200415/225000 [382:51:50<74:49:24, 10.96s/it]
 89%|████████▉ | 200416/225000 [382:51:56<65:06:22,  9.53s/it]
 89%|████████▉ | 200417/225000 [382:52:08<70:06:26, 10.27s/it]
 89%|████████▉ | 200418/225000 [382:52:17<68:15:03, 10.00s/it]
 89%|████████▉ | 200419/225000 [382:52:24<61:59:16,  9.08s/it]
 89%|████████▉ | 200420/225000 [382:52:30<56:25:19,  8.26s/it]
 89%|████████▉ | 200421/225000 [382:52:37<52:50:51,  7.74s/it]
 89%|████████▉ | 200422/225000 [382:52:45<53:13:02,  7.79s/it]
 89%|████████▉ | 200423/225000 [382:52:57<63:05:57,  9.24s/it]
 89%|████████▉ | 200424/225000 [382:53:05<59:05:14,  8.66s/it]
 89%|████████▉ | 200425/225000 [382:53:17<66:17:03,  9.71s/it]
 89%|████████▉ | 200426/225000 [382:53:23<59:15:02,  8.68s/it]
 89%|████████▉ | 200427/225000 [382:53:32<59:04:40,  8.66s/it]
 89%|████████▉ | 200428/225000 [382:53:39<55:27:56,  8.13s/it]
 89%|████████▉ | 200429/225000 [382:53:48<57:35:50,  8.44s/it]
 89%|████████▉ | 200430/225000 [382:53:58<60:21:49,  8.84s/it]
 89%|████████▉ | 200431/225000 [382:54:04<54:28:36,  7.98s/it]
 89%|████████▉ | 200432/225000 [382:54:17<64:37:27,  9.47s/it]
 89%|████████▉ | 200433/225000 [382:54:24<61:14:43,  8.97s/it]
 89%|████████▉ | 200434/225000 [382:54:38<70:11:36, 10.29s/it]
 89%|████████▉ | 200435/225000 [382:54:49<72:52:31, 10.68s/it]
 89%|████████▉ | 200436/225000 [382:54:55<63:23:46,  9.29s/it]
 89%|████████▉ | 200437/225000 [382:55:02<58:35:17,  8.59s/it]
 89%|████████▉ | 200438/225000 [382:55:11<59:53:33,  8.78s/it]
 89%|████████▉ | 200439/225000 [382:55:24<68:18:40, 10.01s/it]
 89%|████████▉ | 200440/225000 [382:55:30<60:13:48,  8.83s/it]
 89%|████████▉ | 200441/225000 [382:55:36<54:03:23,  7.92s/it]
 89%|████████▉ | 200442/225000 [382:55:46<58:29:47,  8.58s/it]
 89%|████████▉ | 200443/225000 [382:55:58<64:57:16,  9.52s/it]
 89%|████████▉ | 200444/225000 [382:56:07<64:21:58,  9.44s/it]
 89%|████████▉ | 200445/225000 [382:56:15<60:05:48,  8.81s/it]
 89%|████████▉ | 200446/225000 [382:56:21<54:28:15,  7.99s/it]
 89%|████████▉ | 200447/225000 [382:56:26<49:10:02,  7.21s/it]
 89%|████████▉ | 200448/225000 [382:56:31<45:13:55,  6.63s/it]
 89%|████████▉ | 200449/225000 [382:56:38<45:32:03,  6.68s/it]
 89%|████████▉ | 200450/225000 [382:56:46<48:08:12,  7.06s/it]
                                                              
{'loss': '1.273', 'grad_norm': '1.555', 'learning_rate': '1.603e-06', 'epoch': '0.8909', 'num_input_tokens_seen': 642116184, 'train_runtime': '1.379e+06', 'train_tokens_per_second': '465.8'}

 89%|████████▉ | 200450/225000 [382:56:46<48:08:12,  7.06s/it]
 89%|████████▉ | 200451/225000 [382:56:57<55:18:49,  8.11s/it]
 89%|████████▉ | 200452/225000 [382:57:02<49:16:57,  7.23s/it]
 89%|████████▉ | 200453/225000 [382:57:07<45:39:42,  6.70s/it]
 89%|████████▉ | 200454/225000 [382:57:12<40:28:04,  5.94s/it]
 89%|████████▉ | 200455/225000 [382:57:25<56:03:39,  8.22s/it]
 89%|████████▉ | 200456/225000 [382:57:32<54:21:38,  7.97s/it]
 89%|████████▉ | 200457/225000 [382:57:39<51:18:48,  7.53s/it]
 89%|████████▉ | 200458/225000 [382:57:46<49:30:29,  7.26s/it]
 89%|████████▉ | 200459/225000 [382:57:56<56:49:13,  8.34s/it]
 89%|████████▉ | 200460/225000 [382:58:02<51:20:41,  7.53s/it]
 89%|████████▉ | 200461/225000 [382:58:10<51:19:06,  7.53s/it]
 89%|████████▉ | 200462/225000 [382:58:16<49:00:13,  7.19s/it]
 89%|████████▉ | 200463/225000 [382:58:30<62:31:26,  9.17s/it]
 89%|████████▉ | 200464/225000 [382:58:40<63:39:19,  9.34s/it]
 89%|████████▉ | 200465/225000 [382:58:46<58:17:31,  8.55s/it]
 89%|████████▉ | 200466/225000 [382:58:56<61:06:10,  8.97s/it]
 89%|████████▉ | 200467/225000 [382:59:03<56:34:28,  8.30s/it]
 89%|████████▉ | 200468/225000 [382:59:12<57:50:27,  8.49s/it]
 89%|████████▉ | 200469/225000 [382:59:21<59:26:15,  8.72s/it]
 89%|████████▉ | 200470/225000 [382:59:34<67:03:39,  9.84s/it]
 89%|████████▉ | 200471/225000 [382:59:42<64:49:15,  9.51s/it]
 89%|████████▉ | 200472/225000 [382:59:50<60:14:02,  8.84s/it]
 89%|████████▉ | 200473/225000 [382:59:56<55:18:00,  8.12s/it]
 89%|████████▉ | 200474/225000 [383:00:08<62:15:14,  9.14s/it]
 89%|████████▉ | 200475/225000 [383:00:14<56:59:14,  8.37s/it]
 89%|████████▉ | 200476/225000 [383:00:24<60:05:42,  8.82s/it]
 89%|████████▉ | 200477/225000 [383:00:32<57:47:16,  8.48s/it]
 89%|████████▉ | 200478/225000 [383:00:43<64:32:43,  9.48s/it]
 89%|████████▉ | 200479/225000 [383:00:52<62:14:00,  9.14s/it]
 89%|████████▉ | 200480/225000 [383:00:58<55:31:58,  8.15s/it]
 89%|████████▉ | 200481/225000 [383:01:10<63:37:02,  9.34s/it]
 89%|████████▉ | 200482/225000 [383:01:17<59:35:35,  8.75s/it]
 89%|████████▉ | 200483/225000 [383:01:27<62:42:08,  9.21s/it]
 89%|████████▉ | 200484/225000 [383:01:34<58:04:27,  8.53s/it]
 89%|████████▉ | 200485/225000 [383:01:40<51:23:05,  7.55s/it]
 89%|████████▉ | 200486/225000 [383:01:50<57:18:16,  8.42s/it]
 89%|████████▉ | 200487/225000 [383:01:59<58:20:50,  8.57s/it]
 89%|████████▉ | 200488/225000 [383:02:07<58:05:38,  8.53s/it]
 89%|████████▉ | 200489/225000 [383:02:15<55:23:18,  8.14s/it]
 89%|████████▉ | 200490/225000 [383:02:26<61:06:41,  8.98s/it]
 89%|████████▉ | 200491/225000 [383:02:34<60:23:55,  8.87s/it]
 89%|████████▉ | 200492/225000 [383:02:42<58:23:26,  8.58s/it]
 89%|████████▉ | 200493/225000 [383:02:47<50:30:08,  7.42s/it]
 89%|████████▉ | 200494/225000 [383:02:56<54:05:04,  7.95s/it]
 89%|████████▉ | 200495/225000 [383:03:04<53:56:42,  7.93s/it]
 89%|████████▉ | 200496/225000 [383:03:10<49:47:51,  7.32s/it]
 89%|████████▉ | 200497/225000 [383:03:18<52:14:01,  7.67s/it]
 89%|████████▉ | 200498/225000 [383:03:25<50:44:18,  7.45s/it]
 89%|████████▉ | 200499/225000 [383:03:31<46:31:01,  6.83s/it]
 89%|████████▉ | 200500/225000 [383:03:42<56:16:00,  8.27s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.582', 'learning_rate': '1.597e-06', 'epoch': '0.8911', 'num_input_tokens_seen': 642339280, 'train_runtime': '1.379e+06', 'train_tokens_per_second': '465.8'}

 89%|████████▉ | 200500/225000 [383:03:42<56:16:00,  8.27s/it]
 89%|████████▉ | 200501/225000 [383:03:51<57:46:58,  8.49s/it]
 89%|████████▉ | 200502/225000 [383:03:59<55:22:44,  8.14s/it]
 89%|████████▉ | 200503/225000 [383:04:09<60:08:22,  8.84s/it]
 89%|████████▉ | 200504/225000 [383:04:17<58:51:43,  8.65s/it]
 89%|████████▉ | 200505/225000 [383:04:26<60:02:07,  8.82s/it]
 89%|████████▉ | 200506/225000 [383:04:31<51:59:34,  7.64s/it]
 89%|████████▉ | 200507/225000 [383:04:40<53:33:20,  7.87s/it]
 89%|████████▉ | 200508/225000 [383:04:52<61:30:27,  9.04s/it]
 89%|████████▉ | 200509/225000 [383:05:03<65:28:11,  9.62s/it]
 89%|████████▉ | 200510/225000 [383:05:07<55:49:43,  8.21s/it]
 89%|████████▉ | 200511/225000 [383:05:17<58:20:41,  8.58s/it]
 89%|████████▉ | 200512/225000 [383:05:25<56:53:57,  8.36s/it]
 89%|████████▉ | 200513/225000 [383:05:32<54:02:09,  7.94s/it]
 89%|████████▉ | 200514/225000 [383:05:39<52:48:00,  7.76s/it]
 89%|████████▉ | 200515/225000 [383:05:52<63:23:39,  9.32s/it]
 89%|████████▉ | 200516/225000 [383:05:56<53:35:10,  7.88s/it]
 89%|████████▉ | 200517/225000 [383:06:07<58:45:00,  8.64s/it]
 89%|████████▉ | 200518/225000 [383:06:15<56:53:52,  8.37s/it]
 89%|████████▉ | 200519/225000 [383:06:22<55:27:30,  8.16s/it]
 89%|████████▉ | 200520/225000 [383:06:33<60:18:59,  8.87s/it]
 89%|████████▉ | 200521/225000 [383:06:41<57:51:16,  8.51s/it]
 89%|████████▉ | 200522/225000 [383:06:46<51:22:49,  7.56s/it]
 89%|████████▉ | 200523/225000 [383:06:54<52:54:14,  7.78s/it]
 89%|████████▉ | 200524/225000 [383:07:02<52:05:33,  7.66s/it]
 89%|████████▉ | 200525/225000 [383:07:08<50:37:04,  7.45s/it]
 89%|████████▉ | 200526/225000 [383:07:17<53:01:56,  7.80s/it]
 89%|████████▉ | 200527/225000 [383:07:27<57:14:57,  8.42s/it]
 89%|████████▉ | 200528/225000 [383:07:35<55:46:55,  8.21s/it]
 89%|████████▉ | 200529/225000 [383:07:45<59:21:10,  8.73s/it]
 89%|████████▉ | 200530/225000 [383:07:52<57:26:49,  8.45s/it]
 89%|████████▉ | 200531/225000 [383:08:02<59:49:10,  8.80s/it]
 89%|████████▉ | 200532/225000 [383:08:16<70:07:46, 10.32s/it]
 89%|████████▉ | 200533/225000 [383:08:21<60:28:24,  8.90s/it]
 89%|████████▉ | 200534/225000 [383:08:28<54:53:35,  8.08s/it]
 89%|████████▉ | 200535/225000 [383:08:36<54:57:23,  8.09s/it]
 89%|████████▉ | 200536/225000 [383:08:42<51:25:25,  7.57s/it]
 89%|████████▉ | 200537/225000 [383:08:48<47:22:09,  6.97s/it]
 89%|████████▉ | 200538/225000 [383:09:01<59:59:06,  8.83s/it]
 89%|████████▉ | 200539/225000 [383:09:08<56:55:08,  8.38s/it]
 89%|████████▉ | 200540/225000 [383:09:16<55:32:57,  8.18s/it]
 89%|████████▉ | 200541/225000 [383:09:23<53:25:27,  7.86s/it]
 89%|████████▉ | 200542/225000 [383:09:30<51:14:56,  7.54s/it]
 89%|████████▉ | 200543/225000 [383:09:36<49:01:16,  7.22s/it]
 89%|████████▉ | 200544/225000 [383:09:47<56:40:34,  8.34s/it]
 89%|████████▉ | 200545/225000 [383:09:52<50:06:51,  7.38s/it]
 89%|████████▉ | 200546/225000 [383:10:04<58:50:17,  8.66s/it]
 89%|████████▉ | 200547/225000 [383:10:12<57:44:37,  8.50s/it]
 89%|████████▉ | 200548/225000 [383:10:21<57:27:26,  8.46s/it]
 89%|████████▉ | 200549/225000 [383:10:27<52:26:21,  7.72s/it]
 89%|████████▉ | 200550/225000 [383:10:33<50:25:43,  7.43s/it]
                                                              
{'loss': '1.287', 'grad_norm': '1.322', 'learning_rate': '1.591e-06', 'epoch': '0.8913', 'num_input_tokens_seen': 642561096, 'train_runtime': '1.379e+06', 'train_tokens_per_second': '465.8'}

 89%|████████▉ | 200550/225000 [383:10:33<50:25:43,  7.43s/it]
 89%|████████▉ | 200551/225000 [383:10:41<50:47:41,  7.48s/it]
 89%|████████▉ | 200552/225000 [383:10:54<62:05:18,  9.14s/it]
 89%|████████▉ | 200553/225000 [383:11:03<61:37:37,  9.08s/it]
 89%|████████▉ | 200554/225000 [383:11:12<62:30:18,  9.20s/it]
 89%|████████▉ | 200555/225000 [383:11:20<58:28:42,  8.61s/it]
 89%|████████▉ | 200556/225000 [383:11:28<58:22:25,  8.60s/it]
 89%|████████▉ | 200557/225000 [383:11:35<54:25:25,  8.02s/it]
 89%|████████▉ | 200558/225000 [383:11:43<54:01:20,  7.96s/it]
 89%|████████▉ | 200559/225000 [383:11:55<62:14:43,  9.17s/it]
 89%|████████▉ | 200560/225000 [383:12:03<59:54:28,  8.82s/it]
 89%|████████▉ | 200561/225000 [383:12:08<53:04:27,  7.82s/it]
 89%|████████▉ | 200562/225000 [383:12:13<46:57:06,  6.92s/it]
 89%|████████▉ | 200563/225000 [383:12:25<58:33:55,  8.63s/it]
 89%|████████▉ | 200564/225000 [383:12:31<51:18:05,  7.56s/it]
 89%|████████▉ | 200565/225000 [383:12:41<57:13:27,  8.43s/it]
 89%|████████▉ | 200566/225000 [383:12:50<58:01:27,  8.55s/it]
 89%|████████▉ | 200567/225000 [383:12:58<57:46:38,  8.51s/it]
 89%|████████▉ | 200568/225000 [383:13:08<59:16:10,  8.73s/it]
 89%|████████▉ | 200569/225000 [383:13:16<58:01:59,  8.55s/it]
 89%|████████▉ | 200570/225000 [383:13:21<50:37:24,  7.46s/it]
 89%|████████▉ | 200571/225000 [383:13:31<57:26:23,  8.46s/it]
 89%|████████▉ | 200572/225000 [383:13:38<53:50:51,  7.94s/it]
 89%|████████▉ | 200573/225000 [383:13:44<49:38:01,  7.31s/it]
 89%|████████▉ | 200574/225000 [383:13:57<60:49:45,  8.97s/it]
 89%|████████▉ | 200575/225000 [383:14:06<61:20:41,  9.04s/it]
 89%|████████▉ | 200576/225000 [383:14:13<58:08:59,  8.57s/it]
 89%|████████▉ | 200577/225000 [383:14:19<52:53:28,  7.80s/it]
 89%|████████▉ | 200578/225000 [383:14:26<51:01:54,  7.52s/it]
 89%|████████▉ | 200579/225000 [383:14:41<66:27:59,  9.80s/it]
 89%|████████▉ | 200580/225000 [383:14:47<58:25:53,  8.61s/it]
 89%|████████▉ | 200581/225000 [383:14:58<62:57:27,  9.28s/it]
 89%|████████▉ | 200582/225000 [383:15:09<65:23:57,  9.64s/it]
 89%|████████▉ | 200583/225000 [383:15:13<55:34:46,  8.19s/it]
 89%|████████▉ | 200584/225000 [383:15:22<57:01:45,  8.41s/it]
 89%|████████▉ | 200585/225000 [383:15:32<58:36:26,  8.64s/it]
 89%|████████▉ | 200586/225000 [383:15:45<67:35:18,  9.97s/it]
 89%|████████▉ | 200587/225000 [383:15:51<59:51:50,  8.83s/it]
 89%|████████▉ | 200588/225000 [383:15:59<58:47:44,  8.67s/it]
 89%|████████▉ | 200589/225000 [383:16:14<70:40:09, 10.42s/it]
 89%|████████▉ | 200590/225000 [383:16:26<74:04:20, 10.92s/it]
 89%|████████▉ | 200591/225000 [383:16:34<68:07:17, 10.05s/it]
 89%|████████▉ | 200592/225000 [383:16:43<67:40:42,  9.98s/it]
 89%|████████▉ | 200593/225000 [383:16:50<61:25:06,  9.06s/it]
 89%|████████▉ | 200594/225000 [383:16:57<57:00:49,  8.41s/it]
 89%|████████▉ | 200595/225000 [383:17:02<50:14:22,  7.41s/it]
 89%|████████▉ | 200596/225000 [383:17:14<58:51:34,  8.68s/it]
 89%|████████▉ | 200597/225000 [383:17:21<55:27:18,  8.18s/it]
 89%|████████▉ | 200598/225000 [383:17:28<53:51:20,  7.95s/it]
 89%|████████▉ | 200599/225000 [383:17:34<49:05:35,  7.24s/it]
 89%|████████▉ | 200600/225000 [383:17:48<62:56:56,  9.29s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.168', 'learning_rate': '1.584e-06', 'epoch': '0.8916', 'num_input_tokens_seen': 642795000, 'train_runtime': '1.38e+06', 'train_tokens_per_second': '465.8'}

 89%|████████▉ | 200600/225000 [383:17:48<62:56:56,  9.29s/it]
 89%|████████▉ | 200601/225000 [383:17:57<62:18:44,  9.19s/it]
 89%|████████▉ | 200602/225000 [383:18:01<52:31:33,  7.75s/it]
 89%|████████▉ | 200603/225000 [383:18:11<56:11:32,  8.29s/it]
 89%|████████▉ | 200604/225000 [383:18:18<54:27:52,  8.04s/it]
 89%|████████▉ | 200605/225000 [383:18:27<55:27:30,  8.18s/it]
 89%|████████▉ | 200606/225000 [383:18:33<50:36:24,  7.47s/it]
 89%|████████▉ | 200607/225000 [383:18:38<46:51:45,  6.92s/it]
 89%|████████▉ | 200608/225000 [383:18:46<47:48:24,  7.06s/it]
 89%|████████▉ | 200609/225000 [383:18:53<47:17:08,  6.98s/it]
 89%|████████▉ | 200610/225000 [383:19:01<49:23:18,  7.29s/it]
 89%|████████▉ | 200611/225000 [383:19:13<58:54:27,  8.70s/it]
 89%|████████▉ | 200612/225000 [383:19:19<54:55:54,  8.11s/it]
 89%|████████▉ | 200613/225000 [383:19:28<55:09:42,  8.14s/it]
 89%|████████▉ | 200614/225000 [383:19:33<49:02:58,  7.24s/it]
 89%|████████▉ | 200615/225000 [383:19:43<54:23:23,  8.03s/it]
 89%|████████▉ | 200616/225000 [383:19:54<60:48:43,  8.98s/it]
 89%|████████▉ | 200617/225000 [383:19:59<53:26:26,  7.89s/it]
 89%|████████▉ | 200618/225000 [383:20:06<51:44:40,  7.64s/it]
 89%|████████▉ | 200619/225000 [383:20:20<65:17:41,  9.64s/it]
 89%|████████▉ | 200620/225000 [383:20:28<60:30:58,  8.94s/it]
 89%|████████▉ | 200621/225000 [383:20:35<58:05:15,  8.58s/it]
 89%|████████▉ | 200622/225000 [383:20:47<63:36:49,  9.39s/it]
 89%|████████▉ | 200623/225000 [383:20:57<64:20:29,  9.50s/it]
 89%|████████▉ | 200624/225000 [383:21:07<66:17:51,  9.79s/it]
 89%|████████▉ | 200625/225000 [383:21:18<67:46:48, 10.01s/it]
 89%|████████▉ | 200626/225000 [383:21:28<67:54:37, 10.03s/it]
 89%|████████▉ | 200627/225000 [383:21:36<63:44:38,  9.42s/it]
 89%|████████▉ | 200628/225000 [383:21:43<58:46:53,  8.68s/it]
 89%|████████▉ | 200629/225000 [383:21:56<68:18:46, 10.09s/it]
 89%|████████▉ | 200630/225000 [383:22:03<62:16:55,  9.20s/it]
 89%|████████▉ | 200631/225000 [383:22:08<54:07:43,  8.00s/it]
 89%|████████▉ | 200632/225000 [383:22:16<53:36:48,  7.92s/it]
 89%|████████▉ | 200633/225000 [383:22:26<57:47:46,  8.54s/it]
 89%|████████▉ | 200634/225000 [383:22:33<54:16:33,  8.02s/it]
 89%|████████▉ | 200635/225000 [383:22:41<53:45:08,  7.94s/it]
 89%|████████▉ | 200636/225000 [383:22:46<49:25:02,  7.30s/it]
 89%|████████▉ | 200637/225000 [383:22:59<59:23:01,  8.77s/it]
 89%|████████▉ | 200638/225000 [383:23:07<59:39:20,  8.82s/it]
 89%|████████▉ | 200639/225000 [383:23:16<58:43:24,  8.68s/it]
 89%|████████▉ | 200640/225000 [383:23:21<52:01:02,  7.69s/it]
 89%|████████▉ | 200641/225000 [383:23:32<58:49:16,  8.69s/it]
 89%|████████▉ | 200642/225000 [383:23:44<65:26:18,  9.67s/it]
 89%|████████▉ | 200643/225000 [383:23:50<58:23:44,  8.63s/it]
 89%|████████▉ | 200644/225000 [383:23:56<52:39:12,  7.78s/it]
 89%|████████▉ | 200645/225000 [383:24:03<49:45:03,  7.35s/it]
 89%|████████▉ | 200646/225000 [383:24:10<48:59:27,  7.24s/it]
 89%|████████▉ | 200647/225000 [383:24:17<50:12:44,  7.42s/it]
 89%|████████▉ | 200648/225000 [383:24:32<65:28:39,  9.68s/it]
 89%|████████▉ | 200649/225000 [383:24:41<63:12:50,  9.35s/it]
 89%|████████▉ | 200650/225000 [383:24:51<65:19:41,  9.66s/it]
                                                              
{'loss': '1.313', 'grad_norm': '0.9404', 'learning_rate': '1.578e-06', 'epoch': '0.8918', 'num_input_tokens_seen': 643026464, 'train_runtime': '1.38e+06', 'train_tokens_per_second': '465.9'}

 89%|████████▉ | 200650/225000 [383:24:51<65:19:41,  9.66s/it]
 89%|████████▉ | 200651/225000 [383:25:00<63:49:06,  9.44s/it]
 89%|████████▉ | 200652/225000 [383:25:12<68:12:30, 10.09s/it]
 89%|████████▉ | 200653/225000 [383:25:22<68:51:06, 10.18s/it]
 89%|████████▉ | 200654/225000 [383:25:29<61:58:14,  9.16s/it]
 89%|████████▉ | 200655/225000 [383:25:41<67:34:31,  9.99s/it]
 89%|████████▉ | 200656/225000 [383:25:52<69:26:25, 10.27s/it]
 89%|████████▉ | 200657/225000 [383:26:01<67:26:26,  9.97s/it]
 89%|████████▉ | 200658/225000 [383:26:06<57:32:53,  8.51s/it]
 89%|████████▉ | 200659/225000 [383:26:15<57:46:28,  8.54s/it]
 89%|████████▉ | 200660/225000 [383:26:22<54:48:52,  8.11s/it]
 89%|████████▉ | 200661/225000 [383:26:34<62:41:06,  9.27s/it]
 89%|████████▉ | 200662/225000 [383:26:43<62:54:48,  9.31s/it]
 89%|████████▉ | 200663/225000 [383:26:52<60:47:07,  8.99s/it]
 89%|████████▉ | 200664/225000 [383:27:07<73:44:38, 10.91s/it]
 89%|████████▉ | 200665/225000 [383:27:12<61:16:13,  9.06s/it]
 89%|████████▉ | 200666/225000 [383:27:20<59:24:15,  8.79s/it]
 89%|████████▉ | 200667/225000 [383:27:25<51:47:23,  7.66s/it]
 89%|████████▉ | 200668/225000 [383:27:31<48:29:37,  7.17s/it]
 89%|████████▉ | 200669/225000 [383:27:45<63:08:58,  9.34s/it]
 89%|████████▉ | 200670/225000 [383:27:53<59:22:57,  8.79s/it]
 89%|████████▉ | 200671/225000 [383:28:00<55:51:39,  8.27s/it]
 89%|████████▉ | 200672/225000 [383:28:07<54:33:33,  8.07s/it]
 89%|████████▉ | 200673/225000 [383:28:17<57:56:42,  8.57s/it]
 89%|████████▉ | 200674/225000 [383:28:24<53:47:10,  7.96s/it]
 89%|████████▉ | 200675/225000 [383:28:31<52:55:55,  7.83s/it]
 89%|████████▉ | 200676/225000 [383:28:45<65:09:47,  9.64s/it]
 89%|████████▉ | 200677/225000 [383:28:54<62:37:19,  9.27s/it]
 89%|████████▉ | 200678/225000 [383:29:01<59:28:49,  8.80s/it]
 89%|████████▉ | 200679/225000 [383:29:07<54:14:50,  8.03s/it]
 89%|████████▉ | 200680/225000 [383:29:14<51:55:26,  7.69s/it]
 89%|████████▉ | 200681/225000 [383:29:21<49:47:18,  7.37s/it]
 89%|████████▉ | 200682/225000 [383:29:35<63:05:39,  9.34s/it]
 89%|████████▉ | 200683/225000 [383:29:45<64:12:34,  9.51s/it]
 89%|████████▉ | 200684/225000 [383:30:00<75:05:40, 11.12s/it]
 89%|████████▉ | 200685/225000 [383:30:07<68:10:03, 10.09s/it]
 89%|████████▉ | 200686/225000 [383:30:18<68:15:49, 10.11s/it]
 89%|████████▉ | 200687/225000 [383:30:25<63:00:21,  9.33s/it]
 89%|████████▉ | 200688/225000 [383:30:34<61:14:53,  9.07s/it]
 89%|████████▉ | 200689/225000 [383:30:42<60:23:28,  8.94s/it]
 89%|████████▉ | 200690/225000 [383:30:48<54:24:25,  8.06s/it]
 89%|████████▉ | 200691/225000 [383:30:57<55:32:58,  8.23s/it]
 89%|████████▉ | 200692/225000 [383:31:10<65:06:28,  9.64s/it]
 89%|████████▉ | 200693/225000 [383:31:14<54:18:42,  8.04s/it]
 89%|████████▉ | 200694/225000 [383:31:24<58:13:37,  8.62s/it]
 89%|████████▉ | 200695/225000 [383:31:29<50:26:25,  7.47s/it]
 89%|████████▉ | 200696/225000 [383:31:39<56:15:16,  8.33s/it]
 89%|████████▉ | 200697/225000 [383:31:45<51:03:06,  7.56s/it]
 89%|████████▉ | 200698/225000 [383:31:54<55:02:45,  8.15s/it]
 89%|████████▉ | 200699/225000 [383:32:04<57:35:50,  8.53s/it]
 89%|████████▉ | 200700/225000 [383:32:12<57:45:47,  8.56s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.392', 'learning_rate': '1.571e-06', 'epoch': '0.892', 'num_input_tokens_seen': 643263728, 'train_runtime': '1.381e+06', 'train_tokens_per_second': '465.9'}

 89%|████████▉ | 200700/225000 [383:32:12<57:45:47,  8.56s/it]
 89%|████████▉ | 200701/225000 [383:32:20<56:19:51,  8.35s/it]
 89%|████████▉ | 200702/225000 [383:32:28<55:11:22,  8.18s/it]
 89%|████████▉ | 200703/225000 [383:32:33<48:59:10,  7.26s/it]
 89%|████████▉ | 200704/225000 [383:32:41<50:38:35,  7.50s/it]
 89%|████████▉ | 200705/225000 [383:32:53<58:40:30,  8.69s/it]
 89%|████████▉ | 200706/225000 [383:33:05<65:59:15,  9.78s/it]
 89%|████████▉ | 200707/225000 [383:33:11<57:52:40,  8.58s/it]
 89%|████████▉ | 200708/225000 [383:33:23<65:39:52,  9.73s/it]
 89%|████████▉ | 200709/225000 [383:33:29<57:25:44,  8.51s/it]
 89%|████████▉ | 200710/225000 [383:33:41<64:10:59,  9.51s/it]
 89%|████████▉ | 200711/225000 [383:33:51<65:20:28,  9.68s/it]
 89%|████████▉ | 200712/225000 [383:33:58<60:00:47,  8.90s/it]
 89%|████████▉ | 200713/225000 [383:34:10<65:26:04,  9.70s/it]
 89%|████████▉ | 200714/225000 [383:34:17<60:50:53,  9.02s/it]
 89%|████████▉ | 200715/225000 [383:34:24<56:17:52,  8.35s/it]
 89%|████████▉ | 200716/225000 [383:34:32<55:16:39,  8.19s/it]
 89%|████████▉ | 200717/225000 [383:34:43<61:24:52,  9.10s/it]
 89%|████████▉ | 200718/225000 [383:34:51<59:13:21,  8.78s/it]
 89%|████████▉ | 200719/225000 [383:34:59<57:33:02,  8.53s/it]
 89%|████████▉ | 200720/225000 [383:35:09<60:24:06,  8.96s/it]
 89%|████████▉ | 200721/225000 [383:35:17<59:06:11,  8.76s/it]
 89%|████████▉ | 200722/225000 [383:35:27<61:26:20,  9.11s/it]
 89%|████████▉ | 200723/225000 [383:35:37<64:13:49,  9.52s/it]
 89%|████████▉ | 200724/225000 [383:35:44<58:43:04,  8.71s/it]
 89%|████████▉ | 200725/225000 [383:35:55<63:44:13,  9.45s/it]
 89%|████████▉ | 200726/225000 [383:36:05<64:28:16,  9.56s/it]
 89%|████████▉ | 200727/225000 [383:36:13<59:49:59,  8.87s/it]
 89%|████████▉ | 200728/225000 [383:36:20<56:37:05,  8.40s/it]
 89%|████████▉ | 200729/225000 [383:36:30<61:15:39,  9.09s/it]
 89%|████████▉ | 200730/225000 [383:36:40<61:47:10,  9.16s/it]
 89%|████████▉ | 200731/225000 [383:36:47<56:45:32,  8.42s/it]
 89%|████████▉ | 200732/225000 [383:36:53<53:49:27,  7.98s/it]
 89%|████████▉ | 200733/225000 [383:37:01<53:15:26,  7.90s/it]
 89%|████████▉ | 200734/225000 [383:37:06<47:18:46,  7.02s/it]
 89%|████████▉ | 200735/225000 [383:37:12<44:49:22,  6.65s/it]
 89%|████████▉ | 200736/225000 [383:37:27<61:23:08,  9.11s/it]
 89%|████████▉ | 200737/225000 [383:37:34<56:56:11,  8.45s/it]
 89%|████████▉ | 200738/225000 [383:37:39<50:57:24,  7.56s/it]
 89%|████████▉ | 200739/225000 [383:37:50<57:49:17,  8.58s/it]
 89%|████████▉ | 200740/225000 [383:38:03<65:37:35,  9.74s/it]
 89%|████████▉ | 200741/225000 [383:38:11<63:57:25,  9.49s/it]
 89%|████████▉ | 200742/225000 [383:38:21<63:53:23,  9.48s/it]
 89%|████████▉ | 200743/225000 [383:38:29<60:25:50,  8.97s/it]
 89%|████████▉ | 200744/225000 [383:38:37<59:55:38,  8.89s/it]
 89%|████████▉ | 200745/225000 [383:38:45<56:42:47,  8.42s/it]
 89%|████████▉ | 200746/225000 [383:38:51<51:35:34,  7.66s/it]
 89%|████████▉ | 200747/225000 [383:38:58<51:27:30,  7.64s/it]
 89%|████████▉ | 200748/225000 [383:39:07<54:16:49,  8.06s/it]
 89%|████████▉ | 200749/225000 [383:39:15<53:14:55,  7.90s/it]
 89%|████████▉ | 200750/225000 [383:39:22<51:56:56,  7.71s/it]
                                                              
{'loss': '1.346', 'grad_norm': '1.146', 'learning_rate': '1.565e-06', 'epoch': '0.8922', 'num_input_tokens_seen': 643496680, 'train_runtime': '1.381e+06', 'train_tokens_per_second': '465.9'}

 89%|████████▉ | 200750/225000 [383:39:22<51:56:56,  7.71s/it]
 89%|████████▉ | 200751/225000 [383:39:34<61:09:38,  9.08s/it]
 89%|████████▉ | 200752/225000 [383:39:45<63:41:08,  9.46s/it]
 89%|████████▉ | 200753/225000 [383:39:50<55:52:23,  8.30s/it]
 89%|████████▉ | 200754/225000 [383:39:59<56:03:06,  8.32s/it]
 89%|████████▉ | 200755/225000 [383:40:13<68:45:06, 10.21s/it]
 89%|████████▉ | 200756/225000 [383:40:19<60:09:06,  8.93s/it]
 89%|████████▉ | 200757/225000 [383:40:27<58:23:47,  8.67s/it]
 89%|████████▉ | 200758/225000 [383:40:34<53:42:12,  7.98s/it]
 89%|████████▉ | 200759/225000 [383:40:39<49:13:36,  7.31s/it]
 89%|████████▉ | 200760/225000 [383:40:49<52:58:54,  7.87s/it]
 89%|████████▉ | 200761/225000 [383:40:57<54:04:26,  8.03s/it]
 89%|████████▉ | 200762/225000 [383:41:08<61:02:38,  9.07s/it]
 89%|████████▉ | 200763/225000 [383:41:16<57:35:57,  8.56s/it]
 89%|████████▉ | 200764/225000 [383:41:22<52:44:26,  7.83s/it]
 89%|████████▉ | 200765/225000 [383:41:33<58:34:17,  8.70s/it]
 89%|████████▉ | 200766/225000 [383:41:42<59:19:30,  8.81s/it]
 89%|████████▉ | 200767/225000 [383:41:50<59:06:32,  8.78s/it]
 89%|████████▉ | 200768/225000 [383:42:00<60:17:57,  8.96s/it]
 89%|████████▉ | 200769/225000 [383:42:08<57:47:26,  8.59s/it]
 89%|████████▉ | 200770/225000 [383:42:13<51:02:52,  7.58s/it]
 89%|████████▉ | 200771/225000 [383:42:26<62:36:34,  9.30s/it]
 89%|████████▉ | 200772/225000 [383:42:32<56:18:27,  8.37s/it]
 89%|████████▉ | 200773/225000 [383:42:40<54:13:57,  8.06s/it]
 89%|████████▉ | 200774/225000 [383:42:53<65:54:05,  9.79s/it]
 89%|████████▉ | 200775/225000 [383:43:00<58:41:04,  8.72s/it]
 89%|████████▉ | 200776/225000 [383:43:12<65:48:17,  9.78s/it]
 89%|████████▉ | 200777/225000 [383:43:23<68:55:15, 10.24s/it]
 89%|████████▉ | 200778/225000 [383:43:32<66:42:06,  9.91s/it]
 89%|████████▉ | 200779/225000 [383:43:39<59:51:38,  8.90s/it]
 89%|████████▉ | 200780/225000 [383:43:45<54:30:38,  8.10s/it]
 89%|████████▉ | 200781/225000 [383:43:56<60:52:21,  9.05s/it]
 89%|████████▉ | 200782/225000 [383:44:04<57:05:26,  8.49s/it]
 89%|████████▉ | 200783/225000 [383:44:11<55:14:11,  8.21s/it]
 89%|████████▉ | 200784/225000 [383:44:22<59:50:01,  8.89s/it]
 89%|████████▉ | 200785/225000 [383:44:26<51:04:26,  7.59s/it]
 89%|████████▉ | 200786/225000 [383:44:40<64:09:48,  9.54s/it]
 89%|████████▉ | 200787/225000 [383:44:47<57:39:38,  8.57s/it]
 89%|████████▉ | 200788/225000 [383:44:54<55:36:52,  8.27s/it]
 89%|████████▉ | 200789/225000 [383:44:59<49:35:28,  7.37s/it]
 89%|████████▉ | 200790/225000 [383:45:04<43:14:49,  6.43s/it]
 89%|████████▉ | 200791/225000 [383:45:17<56:54:43,  8.46s/it]
 89%|████████▉ | 200792/225000 [383:45:24<54:18:52,  8.08s/it]
 89%|████████▉ | 200793/225000 [383:45:29<48:26:10,  7.20s/it]
 89%|████████▉ | 200794/225000 [383:45:36<47:10:03,  7.01s/it]
 89%|████████▉ | 200795/225000 [383:45:49<58:43:02,  8.73s/it]
 89%|████████▉ | 200796/225000 [383:45:58<59:51:41,  8.90s/it]
 89%|████████▉ | 200797/225000 [383:46:05<56:10:14,  8.35s/it]
 89%|████████▉ | 200798/225000 [383:46:13<54:51:21,  8.16s/it]
 89%|████████▉ | 200799/225000 [383:46:19<50:36:59,  7.53s/it]
 89%|████████▉ | 200800/225000 [383:46:26<50:12:48,  7.47s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.071', 'learning_rate': '1.559e-06', 'epoch': '0.8924', 'num_input_tokens_seen': 643727448, 'train_runtime': '1.382e+06', 'train_tokens_per_second': '465.9'}

 89%|████████▉ | 200800/225000 [383:46:26<50:12:48,  7.47s/it]
 89%|████████▉ | 200801/225000 [383:46:31<45:21:31,  6.75s/it]
 89%|████████▉ | 200802/225000 [383:46:38<46:04:31,  6.85s/it]
 89%|████████▉ | 200803/225000 [383:46:48<51:08:31,  7.61s/it]
 89%|████████▉ | 200804/225000 [383:47:01<63:28:57,  9.45s/it]
 89%|████████▉ | 200805/225000 [383:47:07<56:12:59,  8.36s/it]
 89%|████████▉ | 200806/225000 [383:47:14<53:47:53,  8.01s/it]
 89%|████████▉ | 200807/225000 [383:47:22<52:12:43,  7.77s/it]
 89%|████████▉ | 200808/225000 [383:47:33<60:30:38,  9.00s/it]
 89%|████████▉ | 200809/225000 [383:47:43<62:34:07,  9.31s/it]
 89%|████████▉ | 200810/225000 [383:47:50<56:31:59,  8.41s/it]
 89%|████████▉ | 200811/225000 [383:47:57<54:52:00,  8.17s/it]
 89%|████████▉ | 200812/225000 [383:48:05<53:48:18,  8.01s/it]
 89%|████████▉ | 200813/225000 [383:48:16<60:30:50,  9.01s/it]
 89%|████████▉ | 200814/225000 [383:48:23<56:07:29,  8.35s/it]
 89%|████████▉ | 200815/225000 [383:48:32<56:40:32,  8.44s/it]
 89%|████████▉ | 200816/225000 [383:48:40<55:12:12,  8.22s/it]
 89%|████████▉ | 200817/225000 [383:48:52<63:25:26,  9.44s/it]
 89%|████████▉ | 200818/225000 [383:48:59<57:54:10,  8.62s/it]
 89%|████████▉ | 200819/225000 [383:49:07<58:17:53,  8.68s/it]
 89%|████████▉ | 200820/225000 [383:49:20<66:17:13,  9.87s/it]
 89%|████████▉ | 200821/225000 [383:49:26<59:21:16,  8.84s/it]
 89%|████████▉ | 200822/225000 [383:49:33<54:47:57,  8.16s/it]
 89%|████████▉ | 200823/225000 [383:49:44<60:07:19,  8.95s/it]
 89%|████████▉ | 200824/225000 [383:49:53<61:24:09,  9.14s/it]
 89%|████████▉ | 200825/225000 [383:50:05<65:35:24,  9.77s/it]
 89%|████████▉ | 200826/225000 [383:50:11<58:14:17,  8.67s/it]
 89%|████████▉ | 200827/225000 [383:50:21<61:49:39,  9.21s/it]
 89%|████████▉ | 200828/225000 [383:50:28<57:23:18,  8.55s/it]
 89%|████████▉ | 200829/225000 [383:50:36<56:03:20,  8.35s/it]
 89%|████████▉ | 200830/225000 [383:50:45<57:41:54,  8.59s/it]
 89%|████████▉ | 200831/225000 [383:50:55<60:34:44,  9.02s/it]
 89%|████████▉ | 200832/225000 [383:51:03<57:22:45,  8.55s/it]
 89%|████████▉ | 200833/225000 [383:51:12<58:26:29,  8.71s/it]
 89%|████████▉ | 200834/225000 [383:51:22<61:42:02,  9.19s/it]
 89%|████████▉ | 200835/225000 [383:51:27<53:34:03,  7.98s/it]
 89%|████████▉ | 200836/225000 [383:51:34<51:45:12,  7.71s/it]
 89%|████████▉ | 200837/225000 [383:51:41<49:33:11,  7.38s/it]
 89%|████████▉ | 200838/225000 [383:51:51<54:20:26,  8.10s/it]
 89%|████████▉ | 200839/225000 [383:52:01<58:23:25,  8.70s/it]
 89%|████████▉ | 200840/225000 [383:52:09<56:33:00,  8.43s/it]
 89%|████████▉ | 200841/225000 [383:52:17<57:01:54,  8.50s/it]
 89%|████████▉ | 200842/225000 [383:52:31<67:50:32, 10.11s/it]
 89%|████████▉ | 200843/225000 [383:52:36<56:21:08,  8.40s/it]
 89%|████████▉ | 200844/225000 [383:52:44<56:21:16,  8.40s/it]
 89%|████████▉ | 200845/225000 [383:52:51<53:52:26,  8.03s/it]
 89%|████████▉ | 200846/225000 [383:52:56<47:26:22,  7.07s/it]
 89%|████████▉ | 200847/225000 [383:53:04<50:14:29,  7.49s/it]
 89%|████████▉ | 200848/225000 [383:53:13<51:34:09,  7.69s/it]
 89%|████████▉ | 200849/225000 [383:53:27<65:14:49,  9.73s/it]
 89%|████████▉ | 200850/225000 [383:53:36<63:36:42,  9.48s/it]
                                                              
{'loss': '1.292', 'grad_norm': '1.162', 'learning_rate': '1.552e-06', 'epoch': '0.8927', 'num_input_tokens_seen': 643956248, 'train_runtime': '1.382e+06', 'train_tokens_per_second': '466'}

 89%|████████▉ | 200850/225000 [383:53:36<63:36:42,  9.48s/it]
 89%|████████▉ | 200851/225000 [383:53:41<55:14:58,  8.24s/it]
 89%|████████▉ | 200852/225000 [383:53:51<58:12:18,  8.68s/it]
 89%|████████▉ | 200853/225000 [383:54:02<62:06:50,  9.26s/it]
 89%|████████▉ | 200854/225000 [383:54:09<58:58:10,  8.79s/it]
 89%|████████▉ | 200855/225000 [383:54:21<64:19:32,  9.59s/it]
 89%|████████▉ | 200856/225000 [383:54:27<57:40:50,  8.60s/it]
 89%|████████▉ | 200857/225000 [383:54:36<58:29:47,  8.72s/it]
 89%|████████▉ | 200858/225000 [383:54:47<62:16:41,  9.29s/it]
 89%|████████▉ | 200859/225000 [383:54:54<57:30:05,  8.57s/it]
 89%|████████▉ | 200860/225000 [383:55:03<58:59:08,  8.80s/it]
 89%|████████▉ | 200861/225000 [383:55:11<57:06:12,  8.52s/it]
 89%|████████▉ | 200862/225000 [383:55:18<54:29:47,  8.13s/it]
 89%|████████▉ | 200863/225000 [383:55:23<49:06:37,  7.32s/it]
 89%|████████▉ | 200864/225000 [383:55:32<51:42:35,  7.71s/it]
 89%|████████▉ | 200865/225000 [383:55:42<56:30:50,  8.43s/it]
 89%|████████▉ | 200866/225000 [383:55:49<53:53:53,  8.04s/it]
 89%|████████▉ | 200867/225000 [383:56:02<62:57:35,  9.39s/it]
 89%|████████▉ | 200868/225000 [383:56:08<57:12:56,  8.54s/it]
 89%|████████▉ | 200869/225000 [383:56:18<60:11:31,  8.98s/it]
 89%|████████▉ | 200870/225000 [383:56:23<52:02:22,  7.76s/it]
 89%|████████▉ | 200871/225000 [383:56:30<50:00:58,  7.46s/it]
 89%|████████▉ | 200872/225000 [383:56:41<57:42:58,  8.61s/it]
 89%|████████▉ | 200873/225000 [383:56:52<61:41:12,  9.20s/it]
 89%|████████▉ | 200874/225000 [383:56:58<54:25:20,  8.12s/it]
 89%|████████▉ | 200875/225000 [383:57:04<51:35:44,  7.70s/it]
 89%|████████▉ | 200876/225000 [383:57:14<55:47:34,  8.33s/it]
 89%|████████▉ | 200877/225000 [383:57:23<57:46:12,  8.62s/it]
 89%|████████▉ | 200878/225000 [383:57:33<59:25:25,  8.87s/it]
 89%|████████▉ | 200879/225000 [383:57:39<54:21:33,  8.11s/it]
 89%|████████▉ | 200880/225000 [383:57:48<55:20:47,  8.26s/it]
 89%|████████▉ | 200881/225000 [383:57:58<60:02:04,  8.96s/it]
 89%|████████▉ | 200882/225000 [383:58:08<61:33:29,  9.19s/it]
 89%|████████▉ | 200883/225000 [383:58:17<62:01:01,  9.26s/it]
 89%|████████▉ | 200884/225000 [383:58:27<61:51:07,  9.23s/it]
 89%|████████▉ | 200885/225000 [383:58:32<54:48:20,  8.18s/it]
 89%|████████▉ | 200886/225000 [383:58:43<60:25:45,  9.02s/it]
 89%|████████▉ | 200887/225000 [383:58:50<54:45:34,  8.18s/it]
 89%|████████▉ | 200888/225000 [383:59:00<59:42:30,  8.91s/it]
 89%|████████▉ | 200889/225000 [383:59:08<57:35:08,  8.60s/it]
 89%|████████▉ | 200890/225000 [383:59:19<62:39:30,  9.36s/it]
 89%|████████▉ | 200891/225000 [383:59:32<68:38:18, 10.25s/it]
 89%|████████▉ | 200892/225000 [383:59:38<61:47:27,  9.23s/it]
 89%|████████▉ | 200893/225000 [383:59:45<57:19:15,  8.56s/it]
 89%|████████▉ | 200894/225000 [383:59:54<56:52:02,  8.49s/it]
 89%|████████▉ | 200895/225000 [384:00:03<58:55:23,  8.80s/it]
 89%|████████▉ | 200896/225000 [384:00:14<62:02:27,  9.27s/it]
 89%|████████▉ | 200897/225000 [384:00:24<63:45:45,  9.52s/it]
 89%|████████▉ | 200898/225000 [384:00:31<58:21:49,  8.72s/it]
 89%|████████▉ | 200899/225000 [384:00:40<59:26:51,  8.88s/it]
 89%|████████▉ | 200900/225000 [384:00:48<57:07:58,  8.53s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.182', 'learning_rate': '1.546e-06', 'epoch': '0.8929', 'num_input_tokens_seen': 644187944, 'train_runtime': '1.382e+06', 'train_tokens_per_second': '466'}

 89%|████████▉ | 200900/225000 [384:00:48<57:07:58,  8.53s/it]
 89%|████████▉ | 200901/225000 [384:00:54<53:31:29,  8.00s/it]
 89%|████████▉ | 200902/225000 [384:01:04<56:12:36,  8.40s/it]
 89%|████████▉ | 200903/225000 [384:01:18<67:42:03, 10.11s/it]
 89%|████████▉ | 200904/225000 [384:01:26<64:18:10,  9.61s/it]
 89%|████████▉ | 200905/225000 [384:01:38<68:47:56, 10.28s/it]
 89%|████████▉ | 200906/225000 [384:01:46<64:55:06,  9.70s/it]
 89%|████████▉ | 200907/225000 [384:01:52<57:16:37,  8.56s/it]
 89%|████████▉ | 200908/225000 [384:01:59<53:14:29,  7.96s/it]
 89%|████████▉ | 200909/225000 [384:02:04<48:19:36,  7.22s/it]
 89%|████████▉ | 200910/225000 [384:02:18<61:50:07,  9.24s/it]
 89%|████████▉ | 200911/225000 [384:02:26<58:08:06,  8.69s/it]
 89%|████████▉ | 200912/225000 [384:02:36<61:29:08,  9.19s/it]
 89%|████████▉ | 200913/225000 [384:02:43<56:22:57,  8.43s/it]
 89%|████████▉ | 200914/225000 [384:02:53<60:54:24,  9.10s/it]
 89%|████████▉ | 200915/225000 [384:02:59<54:00:08,  8.07s/it]
 89%|████████▉ | 200916/225000 [384:03:05<50:20:31,  7.52s/it]
 89%|████████▉ | 200917/225000 [384:03:19<61:50:59,  9.25s/it]
 89%|████████▉ | 200918/225000 [384:03:29<64:03:45,  9.58s/it]
 89%|████████▉ | 200919/225000 [384:03:36<58:37:15,  8.76s/it]
 89%|████████▉ | 200920/225000 [384:03:48<66:28:55,  9.94s/it]
 89%|████████▉ | 200921/225000 [384:03:54<57:35:50,  8.61s/it]
 89%|████████▉ | 200922/225000 [384:04:03<57:41:03,  8.62s/it]
 89%|████████▉ | 200923/225000 [384:04:13<61:53:40,  9.25s/it]
 89%|████████▉ | 200924/225000 [384:04:18<52:53:40,  7.91s/it]
 89%|████████▉ | 200925/225000 [384:04:27<55:19:32,  8.27s/it]
 89%|████████▉ | 200926/225000 [384:04:33<50:04:31,  7.49s/it]
 89%|████████▉ | 200927/225000 [384:04:39<47:54:13,  7.16s/it]
 89%|████████▉ | 200928/225000 [384:04:52<58:16:49,  8.72s/it]
 89%|████████▉ | 200929/225000 [384:04:59<54:53:38,  8.21s/it]
 89%|████████▉ | 200930/225000 [384:05:11<62:36:31,  9.36s/it]
 89%|████████▉ | 200931/225000 [384:05:19<60:53:31,  9.11s/it]
 89%|████████▉ | 200932/225000 [384:05:24<52:14:40,  7.81s/it]
 89%|████████▉ | 200933/225000 [384:05:31<49:55:42,  7.47s/it]
 89%|████████▉ | 200934/225000 [384:05:41<56:32:27,  8.46s/it]
 89%|████████▉ | 200935/225000 [384:05:48<52:03:57,  7.79s/it]
 89%|████████▉ | 200936/225000 [384:05:54<49:06:11,  7.35s/it]
 89%|████████▉ | 200937/225000 [384:06:02<49:48:33,  7.45s/it]
 89%|████████▉ | 200938/225000 [384:06:08<48:32:08,  7.26s/it]
 89%|████████▉ | 200939/225000 [384:06:14<45:34:57,  6.82s/it]
 89%|████████▉ | 200940/225000 [384:06:30<62:29:47,  9.35s/it]
 89%|████████▉ | 200941/225000 [384:06:36<57:20:57,  8.58s/it]
 89%|████████▉ | 200942/225000 [384:06:47<60:37:00,  9.07s/it]
 89%|████████▉ | 200943/225000 [384:06:56<60:52:22,  9.11s/it]
 89%|████████▉ | 200944/225000 [384:07:05<60:49:25,  9.10s/it]
 89%|████████▉ | 200945/225000 [384:07:13<58:53:30,  8.81s/it]
 89%|████████▉ | 200946/225000 [384:07:17<50:18:48,  7.53s/it]
 89%|████████▉ | 200947/225000 [384:07:23<46:18:02,  6.93s/it]
 89%|████████▉ | 200948/225000 [384:07:32<50:14:21,  7.52s/it]
 89%|████████▉ | 200949/225000 [384:07:40<52:02:18,  7.79s/it]
 89%|████████▉ | 200950/225000 [384:07:51<58:10:56,  8.71s/it]
                                                              
{'loss': '1.268', 'grad_norm': '1.135', 'learning_rate': '1.54e-06', 'epoch': '0.8931', 'num_input_tokens_seen': 644417048, 'train_runtime': '1.383e+06', 'train_tokens_per_second': '466'}

 89%|████████▉ | 200950/225000 [384:07:51<58:10:56,  8.71s/it]
 89%|████████▉ | 200951/225000 [384:07:59<56:17:54,  8.43s/it]
 89%|████████▉ | 200952/225000 [384:08:05<51:53:57,  7.77s/it]
 89%|████████▉ | 200953/225000 [384:08:17<60:46:50,  9.10s/it]
 89%|████████▉ | 200954/225000 [384:08:25<58:08:29,  8.70s/it]
 89%|████████▉ | 200955/225000 [384:08:33<57:07:18,  8.55s/it]
 89%|████████▉ | 200956/225000 [384:08:44<62:11:44,  9.31s/it]
 89%|████████▉ | 200957/225000 [384:08:51<56:03:26,  8.39s/it]
 89%|████████▉ | 200958/225000 [384:08:55<48:50:53,  7.31s/it]
 89%|████████▉ | 200959/225000 [384:09:02<48:05:52,  7.20s/it]
 89%|████████▉ | 200960/225000 [384:09:09<46:29:04,  6.96s/it]
 89%|████████▉ | 200961/225000 [384:09:17<49:17:19,  7.38s/it]
 89%|████████▉ | 200962/225000 [384:09:27<53:49:24,  8.06s/it]
 89%|████████▉ | 200963/225000 [384:09:38<60:09:58,  9.01s/it]
 89%|████████▉ | 200964/225000 [384:09:46<57:49:05,  8.66s/it]
 89%|████████▉ | 200965/225000 [384:09:53<54:05:15,  8.10s/it]
 89%|████████▉ | 200966/225000 [384:10:03<57:44:43,  8.65s/it]
 89%|████████▉ | 200967/225000 [384:10:11<56:30:57,  8.47s/it]
 89%|████████▉ | 200968/225000 [384:10:18<53:41:08,  8.04s/it]
 89%|████████▉ | 200969/225000 [384:10:26<54:36:34,  8.18s/it]
 89%|████████▉ | 200970/225000 [384:10:42<69:54:48, 10.47s/it]
 89%|████████▉ | 200971/225000 [384:10:51<66:55:13, 10.03s/it]
 89%|████████▉ | 200972/225000 [384:10:57<59:00:58,  8.84s/it]
 89%|████████▉ | 200973/225000 [384:11:04<55:22:32,  8.30s/it]
 89%|████████▉ | 200974/225000 [384:11:13<57:01:19,  8.54s/it]
 89%|████████▉ | 200975/225000 [384:11:19<51:41:57,  7.75s/it]
 89%|████████▉ | 200976/225000 [384:11:31<59:07:56,  8.86s/it]
 89%|████████▉ | 200977/225000 [384:11:39<57:44:33,  8.65s/it]
 89%|████████▉ | 200978/225000 [384:11:52<66:42:12, 10.00s/it]
 89%|████████▉ | 200979/225000 [384:11:59<61:05:50,  9.16s/it]
 89%|████████▉ | 200980/225000 [384:12:08<60:01:25,  9.00s/it]
 89%|████████▉ | 200981/225000 [384:12:19<63:40:23,  9.54s/it]
 89%|████████▉ | 200982/225000 [384:12:25<57:30:45,  8.62s/it]
 89%|████████▉ | 200983/225000 [384:12:34<57:47:39,  8.66s/it]
 89%|████████▉ | 200984/225000 [384:12:44<60:59:27,  9.14s/it]
 89%|████████▉ | 200985/225000 [384:12:50<54:06:41,  8.11s/it]
 89%|████████▉ | 200986/225000 [384:12:59<57:11:54,  8.57s/it]
 89%|████████▉ | 200987/225000 [384:13:04<48:38:10,  7.29s/it]
 89%|████████▉ | 200988/225000 [384:13:15<57:07:27,  8.56s/it]
 89%|████████▉ | 200989/225000 [384:13:23<55:20:26,  8.30s/it]
 89%|████████▉ | 200990/225000 [384:13:28<49:04:16,  7.36s/it]
 89%|████████▉ | 200991/225000 [384:13:44<65:16:22,  9.79s/it]
 89%|████████▉ | 200992/225000 [384:13:49<57:13:46,  8.58s/it]
 89%|████████▉ | 200993/225000 [384:14:00<60:59:21,  9.15s/it]
 89%|████████▉ | 200994/225000 [384:14:06<55:24:16,  8.31s/it]
 89%|████████▉ | 200995/225000 [384:14:13<52:45:43,  7.91s/it]
 89%|████████▉ | 200996/225000 [384:14:19<48:53:10,  7.33s/it]
 89%|████████▉ | 200997/225000 [384:14:29<53:07:31,  7.97s/it]
 89%|████████▉ | 200998/225000 [384:14:40<60:54:54,  9.14s/it]
 89%|████████▉ | 200999/225000 [384:14:49<58:54:32,  8.84s/it]
 89%|████████▉ | 201000/225000 [384:14:55<53:17:51,  7.99s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.315', 'learning_rate': '1.533e-06', 'epoch': '0.8933', 'num_input_tokens_seen': 644646744, 'train_runtime': '1.383e+06', 'train_tokens_per_second': '466'}

 89%|████████▉ | 201000/225000 [384:14:55<53:17:51,  7.99s/it]
 89%|████████▉ | 201001/225000 [384:15:01<51:07:26,  7.67s/it]
 89%|████████▉ | 201002/225000 [384:15:15<62:20:14,  9.35s/it]
 89%|████████▉ | 201003/225000 [384:15:21<55:20:03,  8.30s/it]
 89%|████████▉ | 201004/225000 [384:15:27<52:11:44,  7.83s/it]
 89%|████████▉ | 201005/225000 [384:15:36<53:20:29,  8.00s/it]
 89%|████████▉ | 201006/225000 [384:15:48<61:46:16,  9.27s/it]
 89%|████████▉ | 201007/225000 [384:15:55<56:56:51,  8.54s/it]
 89%|████████▉ | 201008/225000 [384:16:03<55:53:45,  8.39s/it]
 89%|████████▉ | 201009/225000 [384:16:12<57:09:27,  8.58s/it]
 89%|████████▉ | 201010/225000 [384:16:26<67:47:43, 10.17s/it]
 89%|████████▉ | 201011/225000 [384:16:34<63:27:09,  9.52s/it]
 89%|████████▉ | 201012/225000 [384:16:43<63:36:19,  9.55s/it]
 89%|████████▉ | 201013/225000 [384:16:48<53:06:32,  7.97s/it]
 89%|████████▉ | 201014/225000 [384:16:57<55:43:49,  8.36s/it]
 89%|████████▉ | 201015/225000 [384:17:11<67:08:24, 10.08s/it]
 89%|████████▉ | 201016/225000 [384:17:17<58:53:55,  8.84s/it]
 89%|████████▉ | 201017/225000 [384:17:31<69:24:37, 10.42s/it]
 89%|████████▉ | 201018/225000 [384:17:37<60:08:54,  9.03s/it]
 89%|████████▉ | 201019/225000 [384:17:46<59:36:26,  8.95s/it]
 89%|████████▉ | 201020/225000 [384:17:56<63:02:34,  9.46s/it]
 89%|████████▉ | 201021/225000 [384:18:03<57:23:56,  8.62s/it]
 89%|████████▉ | 201022/225000 [384:18:12<57:56:36,  8.70s/it]
 89%|████████▉ | 201023/225000 [384:18:21<58:58:39,  8.86s/it]
 89%|████████▉ | 201024/225000 [384:18:33<65:05:46,  9.77s/it]
 89%|████████▉ | 201025/225000 [384:18:39<57:03:34,  8.57s/it]
 89%|████████▉ | 201026/225000 [384:18:44<50:42:22,  7.61s/it]
 89%|████████▉ | 201027/225000 [384:18:51<49:38:19,  7.45s/it]
 89%|████████▉ | 201028/225000 [384:19:06<65:18:22,  9.81s/it]
 89%|████████▉ | 201029/225000 [384:19:14<61:11:55,  9.19s/it]
 89%|████████▉ | 201030/225000 [384:19:22<59:02:37,  8.87s/it]
 89%|████████▉ | 201031/225000 [384:19:31<58:34:36,  8.80s/it]
 89%|████████▉ | 201032/225000 [384:19:37<52:59:00,  7.96s/it]
 89%|████████▉ | 201033/225000 [384:19:49<60:10:26,  9.04s/it]
 89%|████████▉ | 201034/225000 [384:19:59<62:20:29,  9.36s/it]
 89%|████████▉ | 201035/225000 [384:20:05<56:13:53,  8.45s/it]
 89%|████████▉ | 201036/225000 [384:20:14<56:57:30,  8.56s/it]
 89%|████████▉ | 201037/225000 [384:20:30<72:53:43, 10.95s/it]
 89%|████████▉ | 201038/225000 [384:20:35<61:03:56,  9.17s/it]
 89%|████████▉ | 201039/225000 [384:20:43<58:24:31,  8.78s/it]
 89%|████████▉ | 201040/225000 [384:20:50<54:12:34,  8.15s/it]
 89%|████████▉ | 201041/225000 [384:20:59<57:01:15,  8.57s/it]
 89%|████████▉ | 201042/225000 [384:21:06<54:00:35,  8.12s/it]
 89%|████████▉ | 201043/225000 [384:21:15<55:09:45,  8.29s/it]
 89%|████████▉ | 201044/225000 [384:21:31<70:13:18, 10.55s/it]
 89%|████████▉ | 201045/225000 [384:21:36<59:22:08,  8.92s/it]
 89%|████████▉ | 201046/225000 [384:21:44<56:19:04,  8.46s/it]
 89%|████████▉ | 201047/225000 [384:21:49<49:43:59,  7.47s/it]
 89%|████████▉ | 201048/225000 [384:22:04<64:56:57,  9.76s/it]
 89%|████████▉ | 201049/225000 [384:22:11<60:10:28,  9.04s/it]
 89%|████████▉ | 201050/225000 [384:22:20<59:41:08,  8.97s/it]
                                                              
{'loss': '1.335', 'grad_norm': '1.158', 'learning_rate': '1.527e-06', 'epoch': '0.8936', 'num_input_tokens_seen': 644883312, 'train_runtime': '1.384e+06', 'train_tokens_per_second': '466'}

 89%|████████▉ | 201050/225000 [384:22:20<59:41:08,  8.97s/it]
 89%|████████▉ | 201051/225000 [384:22:27<55:30:02,  8.34s/it]
 89%|████████▉ | 201052/225000 [384:22:33<52:02:44,  7.82s/it]
 89%|████████▉ | 201053/225000 [384:22:40<50:05:27,  7.53s/it]
 89%|████████▉ | 201054/225000 [384:22:51<56:49:18,  8.54s/it]
 89%|████████▉ | 201055/225000 [384:22:56<48:40:26,  7.32s/it]
 89%|████████▉ | 201056/225000 [384:23:04<49:45:46,  7.48s/it]
 89%|████████▉ | 201057/225000 [384:23:18<64:14:53,  9.66s/it]
 89%|████████▉ | 201058/225000 [384:23:24<57:07:37,  8.59s/it]
 89%|████████▉ | 201059/225000 [384:23:33<57:32:15,  8.65s/it]
 89%|████████▉ | 201060/225000 [384:23:40<54:03:37,  8.13s/it]
 89%|████████▉ | 201061/225000 [384:23:55<67:18:08, 10.12s/it]
 89%|████████▉ | 201062/225000 [384:24:03<62:32:29,  9.41s/it]
 89%|████████▉ | 201063/225000 [384:24:09<57:21:02,  8.63s/it]
 89%|████████▉ | 201064/225000 [384:24:24<68:27:55, 10.30s/it]
 89%|████████▉ | 201065/225000 [384:24:31<62:03:38,  9.33s/it]
 89%|████████▉ | 201066/225000 [384:24:37<55:12:37,  8.30s/it]
 89%|████████▉ | 201067/225000 [384:24:46<58:22:11,  8.78s/it]
 89%|████████▉ | 201068/225000 [384:24:56<59:24:12,  8.94s/it]
 89%|████████▉ | 201069/225000 [384:25:05<59:58:35,  9.02s/it]
 89%|████████▉ | 201070/225000 [384:25:15<62:48:45,  9.45s/it]
 89%|████████▉ | 201071/225000 [384:25:24<61:12:11,  9.21s/it]
 89%|████████▉ | 201072/225000 [384:25:31<56:55:54,  8.57s/it]
 89%|████████▉ | 201073/225000 [384:25:41<59:23:45,  8.94s/it]
 89%|████████▉ | 201074/225000 [384:25:49<57:22:57,  8.63s/it]
 89%|████████▉ | 201075/225000 [384:25:54<51:14:07,  7.71s/it]
 89%|████████▉ | 201076/225000 [384:26:06<58:57:51,  8.87s/it]
 89%|████████▉ | 201077/225000 [384:26:12<53:57:38,  8.12s/it]
 89%|████████▉ | 201078/225000 [384:26:21<55:46:59,  8.39s/it]
 89%|████████▉ | 201079/225000 [384:26:26<48:22:18,  7.28s/it]
 89%|████████▉ | 201080/225000 [384:26:31<44:03:40,  6.63s/it]
 89%|████████▉ | 201081/225000 [384:26:38<43:39:54,  6.57s/it]
 89%|████████▉ | 201082/225000 [384:26:45<44:38:58,  6.72s/it]
 89%|████████▉ | 201083/225000 [384:26:59<60:26:14,  9.10s/it]
 89%|████████▉ | 201084/225000 [384:27:09<61:07:00,  9.20s/it]
 89%|████████▉ | 201085/225000 [384:27:14<54:11:30,  8.16s/it]
 89%|████████▉ | 201086/225000 [384:27:23<55:02:12,  8.29s/it]
 89%|████████▉ | 201087/225000 [384:27:32<56:42:14,  8.54s/it]
 89%|████████▉ | 201088/225000 [384:27:41<57:10:10,  8.61s/it]
 89%|████████▉ | 201089/225000 [384:27:50<58:24:49,  8.79s/it]
 89%|████████▉ | 201090/225000 [384:28:01<62:37:18,  9.43s/it]
 89%|████████▉ | 201091/225000 [384:28:11<62:37:40,  9.43s/it]
 89%|████████▉ | 201092/225000 [384:28:15<53:15:44,  8.02s/it]
 89%|████████▉ | 201093/225000 [384:28:22<50:52:20,  7.66s/it]
 89%|████████▉ | 201094/225000 [384:28:31<54:20:20,  8.18s/it]
 89%|████████▉ | 201095/225000 [384:28:46<66:17:13,  9.98s/it]
 89%|████████▉ | 201096/225000 [384:28:51<57:30:17,  8.66s/it]
 89%|████████▉ | 201097/225000 [384:28:56<50:19:29,  7.58s/it]
 89%|████████▉ | 201098/225000 [384:29:08<57:39:10,  8.68s/it]
 89%|████████▉ | 201099/225000 [384:29:14<53:54:35,  8.12s/it]
 89%|████████▉ | 201100/225000 [384:29:24<56:49:24,  8.56s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.347', 'learning_rate': '1.521e-06', 'epoch': '0.8938', 'num_input_tokens_seen': 645115272, 'train_runtime': '1.384e+06', 'train_tokens_per_second': '466.1'}

 89%|████████▉ | 201100/225000 [384:29:24<56:49:24,  8.56s/it]
 89%|████████▉ | 201101/225000 [384:29:31<53:35:29,  8.07s/it]
 89%|████████▉ | 201102/225000 [384:29:44<64:04:56,  9.65s/it]
 89%|████████▉ | 201103/225000 [384:29:52<60:06:37,  9.06s/it]
 89%|████████▉ | 201104/225000 [384:30:02<62:52:06,  9.47s/it]
 89%|████████▉ | 201105/225000 [384:30:10<59:44:44,  9.00s/it]
 89%|████████▉ | 201106/225000 [384:30:19<59:50:48,  9.02s/it]
 89%|████████▉ | 201107/225000 [384:30:26<54:56:45,  8.28s/it]
 89%|████████▉ | 201108/225000 [384:30:36<58:37:34,  8.83s/it]
 89%|████████▉ | 201109/225000 [384:30:42<53:54:52,  8.12s/it]
 89%|████████▉ | 201110/225000 [384:30:50<52:51:23,  7.96s/it]
 89%|████████▉ | 201111/225000 [384:30:56<48:45:52,  7.35s/it]
 89%|████████▉ | 201112/225000 [384:31:08<57:09:31,  8.61s/it]
 89%|████████▉ | 201113/225000 [384:31:14<53:02:19,  7.99s/it]
 89%|████████▉ | 201114/225000 [384:31:22<52:23:28,  7.90s/it]
 89%|████████▉ | 201115/225000 [384:31:29<50:21:16,  7.59s/it]
 89%|████████▉ | 201116/225000 [384:31:44<65:51:41,  9.93s/it]
 89%|████████▉ | 201117/225000 [384:31:52<63:02:05,  9.50s/it]
 89%|████████▉ | 201118/225000 [384:32:00<59:28:29,  8.97s/it]
 89%|████████▉ | 201119/225000 [384:32:13<66:14:59,  9.99s/it]
 89%|████████▉ | 201120/225000 [384:32:20<60:58:23,  9.19s/it]
 89%|████████▉ | 201121/225000 [384:32:28<57:51:58,  8.72s/it]
 89%|████████▉ | 201122/225000 [384:32:35<55:12:53,  8.32s/it]
 89%|████████▉ | 201123/225000 [384:32:47<62:01:48,  9.35s/it]
 89%|████████▉ | 201124/225000 [384:32:56<62:34:48,  9.44s/it]
 89%|████████▉ | 201125/225000 [384:33:06<63:24:41,  9.56s/it]
 89%|████████▉ | 201126/225000 [384:33:12<56:03:05,  8.45s/it]
 89%|████████▉ | 201127/225000 [384:33:17<49:36:46,  7.48s/it]
 89%|████████▉ | 201128/225000 [384:33:23<45:39:44,  6.89s/it]
 89%|████████▉ | 201129/225000 [384:33:34<53:40:41,  8.10s/it]
 89%|████████▉ | 201130/225000 [384:33:42<54:50:23,  8.27s/it]
 89%|████████▉ | 201131/225000 [384:33:50<53:55:28,  8.13s/it]
 89%|████████▉ | 201132/225000 [384:34:01<59:47:27,  9.02s/it]
 89%|████████▉ | 201133/225000 [384:34:07<52:34:49,  7.93s/it]
 89%|████████▉ | 201134/225000 [384:34:13<50:09:52,  7.57s/it]
 89%|████████▉ | 201135/225000 [384:34:27<63:05:06,  9.52s/it]
 89%|████████▉ | 201136/225000 [384:34:34<57:22:25,  8.66s/it]
 89%|████████▉ | 201137/225000 [384:34:43<57:11:18,  8.63s/it]
 89%|████████▉ | 201138/225000 [384:34:53<61:18:43,  9.25s/it]
 89%|████████▉ | 201139/225000 [384:34:59<54:05:39,  8.16s/it]
 89%|████████▉ | 201140/225000 [384:35:09<56:56:04,  8.59s/it]
 89%|████████▉ | 201141/225000 [384:35:17<57:08:47,  8.62s/it]
 89%|████████▉ | 201142/225000 [384:35:26<57:58:31,  8.75s/it]
 89%|████████▉ | 201143/225000 [384:35:35<57:33:09,  8.68s/it]
 89%|████████▉ | 201144/225000 [384:35:44<57:56:15,  8.74s/it]
 89%|████████▉ | 201145/225000 [384:35:56<64:11:51,  9.69s/it]
 89%|████████▉ | 201146/225000 [384:36:05<63:13:21,  9.54s/it]
 89%|████████▉ | 201147/225000 [384:36:11<57:20:03,  8.65s/it]
 89%|████████▉ | 201148/225000 [384:36:20<56:52:54,  8.59s/it]
 89%|████████▉ | 201149/225000 [384:36:34<68:16:20, 10.30s/it]
 89%|████████▉ | 201150/225000 [384:36:45<70:06:44, 10.58s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.123', 'learning_rate': '1.515e-06', 'epoch': '0.894', 'num_input_tokens_seen': 645347360, 'train_runtime': '1.385e+06', 'train_tokens_per_second': '466.1'}

 89%|████████▉ | 201150/225000 [384:36:45<70:06:44, 10.58s/it]
 89%|████████▉ | 201151/225000 [384:36:54<66:53:17, 10.10s/it]
 89%|████████▉ | 201152/225000 [384:37:00<58:24:54,  8.82s/it]
 89%|████████▉ | 201153/225000 [384:37:06<52:40:55,  7.95s/it]
 89%|████████▉ | 201154/225000 [384:37:14<52:03:59,  7.86s/it]
 89%|████████▉ | 201155/225000 [384:37:27<62:44:23,  9.47s/it]
 89%|████████▉ | 201156/225000 [384:37:34<57:27:06,  8.67s/it]
 89%|████████▉ | 201157/225000 [384:37:44<59:44:14,  9.02s/it]
 89%|████████▉ | 201158/225000 [384:37:53<61:15:44,  9.25s/it]
 89%|████████▉ | 201159/225000 [384:37:59<54:22:01,  8.21s/it]
 89%|████████▉ | 201160/225000 [384:38:04<47:20:32,  7.15s/it]
 89%|████████▉ | 201161/225000 [384:38:15<55:48:55,  8.43s/it]
 89%|████████▉ | 201162/225000 [384:38:23<54:02:36,  8.16s/it]
 89%|████████▉ | 201163/225000 [384:38:32<55:13:35,  8.34s/it]
 89%|████████▉ | 201164/225000 [384:38:38<51:07:41,  7.72s/it]
 89%|████████▉ | 201165/225000 [384:38:47<53:24:15,  8.07s/it]
 89%|████████▉ | 201166/225000 [384:38:55<54:06:10,  8.17s/it]
 89%|████████▉ | 201167/225000 [384:39:01<50:22:25,  7.61s/it]
 89%|████████▉ | 201168/225000 [384:39:08<48:06:33,  7.27s/it]
 89%|████████▉ | 201169/225000 [384:39:15<46:59:21,  7.10s/it]
 89%|████████▉ | 201170/225000 [384:39:27<57:21:28,  8.67s/it]
 89%|████████▉ | 201171/225000 [384:39:33<53:00:49,  8.01s/it]
 89%|████████▉ | 201172/225000 [384:39:43<56:42:39,  8.57s/it]
 89%|████████▉ | 201173/225000 [384:39:49<51:30:34,  7.78s/it]
 89%|████████▉ | 201174/225000 [384:40:00<57:59:32,  8.76s/it]
 89%|████████▉ | 201175/225000 [384:40:07<54:51:57,  8.29s/it]
 89%|████████▉ | 201176/225000 [384:40:16<54:31:10,  8.24s/it]
 89%|████████▉ | 201177/225000 [384:40:20<47:06:00,  7.12s/it]
 89%|████████▉ | 201178/225000 [384:40:29<50:26:36,  7.62s/it]
 89%|████████▉ | 201179/225000 [384:40:39<55:19:05,  8.36s/it]
 89%|████████▉ | 201180/225000 [384:40:48<56:19:18,  8.51s/it]
 89%|████████▉ | 201181/225000 [384:40:54<50:42:28,  7.66s/it]
 89%|████████▉ | 201182/225000 [384:41:05<58:25:59,  8.83s/it]
 89%|████████▉ | 201183/225000 [384:41:11<51:56:37,  7.85s/it]
 89%|████████▉ | 201184/225000 [384:41:17<49:01:56,  7.41s/it]
 89%|████████▉ | 201185/225000 [384:41:31<61:23:14,  9.28s/it]
 89%|████████▉ | 201186/225000 [384:41:42<64:58:09,  9.82s/it]
 89%|████████▉ | 201187/225000 [384:41:47<55:47:21,  8.43s/it]
 89%|████████▉ | 201188/225000 [384:41:55<55:59:23,  8.46s/it]
 89%|████████▉ | 201189/225000 [384:42:01<49:43:19,  7.52s/it]
 89%|████████▉ | 201190/225000 [384:42:13<58:57:25,  8.91s/it]
 89%|████████▉ | 201191/225000 [384:42:21<56:19:38,  8.52s/it]
 89%|████████▉ | 201192/225000 [384:42:30<57:58:41,  8.77s/it]
 89%|████████▉ | 201193/225000 [384:42:35<50:32:38,  7.64s/it]
 89%|████████▉ | 201194/225000 [384:42:48<61:55:37,  9.36s/it]
 89%|████████▉ | 201195/225000 [384:42:55<57:25:33,  8.68s/it]
 89%|████████▉ | 201196/225000 [384:43:07<64:06:15,  9.69s/it]
 89%|████████▉ | 201197/225000 [384:43:12<53:49:47,  8.14s/it]
 89%|████████▉ | 201198/225000 [384:43:21<55:37:07,  8.41s/it]
 89%|████████▉ | 201199/225000 [384:43:28<53:20:36,  8.07s/it]
 89%|████████▉ | 201200/225000 [384:43:36<52:39:59,  7.97s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.973', 'learning_rate': '1.508e-06', 'epoch': '0.8942', 'num_input_tokens_seen': 645575624, 'train_runtime': '1.385e+06', 'train_tokens_per_second': '466.1'}

 89%|████████▉ | 201200/225000 [384:43:36<52:39:59,  7.97s/it]
 89%|████████▉ | 201201/225000 [384:43:42<49:22:38,  7.47s/it]
 89%|████████▉ | 201202/225000 [384:43:51<52:17:12,  7.91s/it]
 89%|████████▉ | 201203/225000 [384:44:00<53:15:59,  8.06s/it]
 89%|████████▉ | 201204/225000 [384:44:05<47:39:46,  7.21s/it]
 89%|████████▉ | 201205/225000 [384:44:14<51:42:52,  7.82s/it]
 89%|████████▉ | 201206/225000 [384:44:19<46:03:17,  6.97s/it]
 89%|████████▉ | 201207/225000 [384:44:24<42:37:59,  6.45s/it]
 89%|████████▉ | 201208/225000 [384:44:30<40:56:46,  6.20s/it]
 89%|████████▉ | 201209/225000 [384:44:38<43:41:56,  6.61s/it]
 89%|████████▉ | 201210/225000 [384:44:51<57:37:49,  8.72s/it]
 89%|████████▉ | 201211/225000 [384:44:58<54:26:52,  8.24s/it]
 89%|████████▉ | 201212/225000 [384:45:02<45:59:23,  6.96s/it]
 89%|████████▉ | 201213/225000 [384:45:09<46:10:27,  6.99s/it]
 89%|████████▉ | 201214/225000 [384:45:17<47:58:23,  7.26s/it]
 89%|████████▉ | 201215/225000 [384:45:25<49:15:45,  7.46s/it]
 89%|████████▉ | 201216/225000 [384:45:36<56:29:02,  8.55s/it]
 89%|████████▉ | 201217/225000 [384:45:45<56:53:16,  8.61s/it]
 89%|████████▉ | 201218/225000 [384:46:00<68:59:29, 10.44s/it]
 89%|████████▉ | 201219/225000 [384:46:06<59:47:35,  9.05s/it]
 89%|████████▉ | 201220/225000 [384:46:16<62:49:26,  9.51s/it]
 89%|████████▉ | 201221/225000 [384:46:21<53:43:35,  8.13s/it]
 89%|████████▉ | 201222/225000 [384:46:29<52:53:43,  8.01s/it]
 89%|████████▉ | 201223/225000 [384:46:38<55:45:16,  8.44s/it]
 89%|████████▉ | 201224/225000 [384:46:43<48:20:57,  7.32s/it]
 89%|████████▉ | 201225/225000 [384:46:49<45:47:58,  6.93s/it]
 89%|████████▉ | 201226/225000 [384:47:02<57:34:08,  8.72s/it]
 89%|████████▉ | 201227/225000 [384:47:16<69:12:08, 10.48s/it]
 89%|████████▉ | 201228/225000 [384:47:25<65:43:45,  9.95s/it]
 89%|████████▉ | 201229/225000 [384:47:31<57:48:51,  8.76s/it]
 89%|████████▉ | 201230/225000 [384:47:36<50:28:50,  7.65s/it]
 89%|████████▉ | 201231/225000 [384:47:46<54:57:59,  8.33s/it]
 89%|████████▉ | 201232/225000 [384:47:52<51:16:18,  7.77s/it]
 89%|████████▉ | 201233/225000 [384:47:57<45:10:49,  6.84s/it]
 89%|████████▉ | 201234/225000 [384:48:11<58:29:26,  8.86s/it]
 89%|████████▉ | 201235/225000 [384:48:19<56:21:59,  8.54s/it]
 89%|████████▉ | 201236/225000 [384:48:29<59:22:12,  8.99s/it]
 89%|████████▉ | 201237/225000 [384:48:35<54:20:16,  8.23s/it]
 89%|████████▉ | 201238/225000 [384:48:45<57:33:40,  8.72s/it]
 89%|████████▉ | 201239/225000 [384:48:50<49:30:32,  7.50s/it]
 89%|████████▉ | 201240/225000 [384:49:01<58:01:06,  8.79s/it]
 89%|████████▉ | 201241/225000 [384:49:09<55:24:58,  8.40s/it]
 89%|████████▉ | 201242/225000 [384:49:19<58:47:00,  8.91s/it]
 89%|████████▉ | 201243/225000 [384:49:27<56:57:55,  8.63s/it]
 89%|████████▉ | 201244/225000 [384:49:38<61:23:41,  9.30s/it]
 89%|████████▉ | 201245/225000 [384:49:46<59:11:09,  8.97s/it]
 89%|████████▉ | 201246/225000 [384:49:54<57:56:21,  8.78s/it]
 89%|████████▉ | 201247/225000 [384:50:04<60:26:07,  9.16s/it]
 89%|████████▉ | 201248/225000 [384:50:12<57:39:23,  8.74s/it]
 89%|████████▉ | 201249/225000 [384:50:19<53:33:45,  8.12s/it]
 89%|████████▉ | 201250/225000 [384:50:25<50:22:54,  7.64s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.208', 'learning_rate': '1.502e-06', 'epoch': '0.8944', 'num_input_tokens_seen': 645796024, 'train_runtime': '1.385e+06', 'train_tokens_per_second': '466.1'}

 89%|████████▉ | 201250/225000 [384:50:25<50:22:54,  7.64s/it]
 89%|████████▉ | 201251/225000 [384:50:34<52:48:26,  8.00s/it]
 89%|████████▉ | 201252/225000 [384:50:40<49:20:03,  7.48s/it]
 89%|████████▉ | 201253/225000 [384:50:47<46:45:06,  7.09s/it]
 89%|████████▉ | 201254/225000 [384:51:01<60:24:05,  9.16s/it]
 89%|████████▉ | 201255/225000 [384:51:13<67:20:26, 10.21s/it]
 89%|████████▉ | 201256/225000 [384:51:21<63:17:08,  9.60s/it]
 89%|████████▉ | 201257/225000 [384:51:31<63:11:40,  9.58s/it]
 89%|████████▉ | 201258/225000 [384:51:43<67:02:51, 10.17s/it]
 89%|████████▉ | 201259/225000 [384:51:54<69:53:36, 10.60s/it]
 89%|████████▉ | 201260/225000 [384:52:03<67:06:58, 10.18s/it]
 89%|████████▉ | 201261/225000 [384:52:12<63:13:34,  9.59s/it]
 89%|████████▉ | 201262/225000 [384:52:20<61:35:00,  9.34s/it]
 89%|████████▉ | 201263/225000 [384:52:25<52:42:09,  7.99s/it]
 89%|████████▉ | 201264/225000 [384:52:34<53:46:08,  8.16s/it]
 89%|████████▉ | 201265/225000 [384:52:39<48:12:09,  7.31s/it]
 89%|████████▉ | 201266/225000 [384:52:51<57:38:57,  8.74s/it]
 89%|████████▉ | 201267/225000 [384:52:57<51:21:50,  7.79s/it]
 89%|████████▉ | 201268/225000 [384:53:05<51:37:58,  7.83s/it]
 89%|████████▉ | 201269/225000 [384:53:11<48:56:44,  7.43s/it]
 89%|████████▉ | 201270/225000 [384:53:18<48:51:27,  7.41s/it]
 89%|████████▉ | 201271/225000 [384:53:30<57:20:27,  8.70s/it]
 89%|████████▉ | 201272/225000 [384:53:36<50:59:18,  7.74s/it]
 89%|████████▉ | 201273/225000 [384:53:43<49:17:05,  7.48s/it]
 89%|████████▉ | 201274/225000 [384:53:58<65:42:18,  9.97s/it]
 89%|████████▉ | 201275/225000 [384:54:04<56:50:04,  8.62s/it]
 89%|████████▉ | 201276/225000 [384:54:14<60:54:10,  9.24s/it]
 89%|████████▉ | 201277/225000 [384:54:28<69:46:20, 10.59s/it]
 89%|████████▉ | 201278/225000 [384:54:39<69:18:08, 10.52s/it]
 89%|████████▉ | 201279/225000 [384:54:46<63:27:54,  9.63s/it]
 89%|████████▉ | 201280/225000 [384:54:51<54:42:21,  8.30s/it]
 89%|████████▉ | 201281/225000 [384:55:02<60:07:15,  9.12s/it]
 89%|████████▉ | 201282/225000 [384:55:07<51:12:57,  7.77s/it]
 89%|████████▉ | 201283/225000 [384:55:18<58:10:33,  8.83s/it]
 89%|████████▉ | 201284/225000 [384:55:26<55:06:51,  8.37s/it]
 89%|████████▉ | 201285/225000 [384:55:33<52:44:52,  8.01s/it]
 89%|████████▉ | 201286/225000 [384:55:46<62:54:52,  9.55s/it]
 89%|████████▉ | 201287/225000 [384:55:54<60:11:46,  9.14s/it]
 89%|████████▉ | 201288/225000 [384:56:00<54:37:40,  8.29s/it]
 89%|████████▉ | 201289/225000 [384:56:13<62:41:24,  9.52s/it]
 89%|████████▉ | 201290/225000 [384:56:18<54:02:35,  8.21s/it]
 89%|████████▉ | 201291/225000 [384:56:28<57:32:57,  8.74s/it]
 89%|████████▉ | 201292/225000 [384:56:34<52:53:33,  8.03s/it]
 89%|████████▉ | 201293/225000 [384:56:40<49:18:39,  7.49s/it]
 89%|████████▉ | 201294/225000 [384:56:56<64:29:03,  9.79s/it]
 89%|████████▉ | 201295/225000 [384:57:02<56:46:56,  8.62s/it]
 89%|████████▉ | 201296/225000 [384:57:11<58:08:43,  8.83s/it]
 89%|████████▉ | 201297/225000 [384:57:20<59:12:43,  8.99s/it]
 89%|████████▉ | 201298/225000 [384:57:29<57:54:17,  8.79s/it]
 89%|████████▉ | 201299/225000 [384:57:38<59:25:00,  9.02s/it]
 89%|████████▉ | 201300/225000 [384:57:48<60:48:11,  9.24s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.341', 'learning_rate': '1.496e-06', 'epoch': '0.8947', 'num_input_tokens_seen': 646033320, 'train_runtime': '1.386e+06', 'train_tokens_per_second': '466.2'}

 89%|████████▉ | 201300/225000 [384:57:48<60:48:11,  9.24s/it]
 89%|████████▉ | 201301/225000 [384:57:54<54:00:06,  8.20s/it]
 89%|████████▉ | 201302/225000 [384:58:06<61:22:01,  9.32s/it]
 89%|████████▉ | 201303/225000 [384:58:12<56:32:24,  8.59s/it]
 89%|████████▉ | 201304/225000 [384:58:23<60:26:51,  9.18s/it]
 89%|████████▉ | 201305/225000 [384:58:28<52:32:26,  7.98s/it]
 89%|████████▉ | 201306/225000 [384:58:39<57:33:16,  8.74s/it]
 89%|████████▉ | 201307/225000 [384:58:48<58:48:07,  8.93s/it]
 89%|████████▉ | 201308/225000 [384:58:55<55:38:29,  8.45s/it]
 89%|████████▉ | 201309/225000 [384:59:07<62:43:34,  9.53s/it]
 89%|████████▉ | 201310/225000 [384:59:13<54:26:45,  8.27s/it]
 89%|████████▉ | 201311/225000 [384:59:22<56:58:01,  8.66s/it]
 89%|████████▉ | 201312/225000 [384:59:31<56:52:08,  8.64s/it]
 89%|████████▉ | 201313/225000 [384:59:38<54:08:41,  8.23s/it]
 89%|████████▉ | 201314/225000 [384:59:44<50:13:00,  7.63s/it]
 89%|████████▉ | 201315/225000 [384:59:52<50:24:49,  7.66s/it]
 89%|████████▉ | 201316/225000 [385:00:06<61:36:46,  9.37s/it]
 89%|████████▉ | 201317/225000 [385:00:12<55:18:37,  8.41s/it]
 89%|████████▉ | 201318/225000 [385:00:18<50:32:32,  7.68s/it]
 89%|████████▉ | 201319/225000 [385:00:28<56:11:13,  8.54s/it]
 89%|████████▉ | 201320/225000 [385:00:38<58:30:11,  8.89s/it]
 89%|████████▉ | 201321/225000 [385:00:44<52:53:47,  8.04s/it]
 89%|████████▉ | 201322/225000 [385:00:52<51:58:41,  7.90s/it]
 89%|████████▉ | 201323/225000 [385:01:01<54:27:05,  8.28s/it]
 89%|████████▉ | 201324/225000 [385:01:08<52:09:18,  7.93s/it]
 89%|████████▉ | 201325/225000 [385:01:17<54:49:48,  8.34s/it]
 89%|████████▉ | 201326/225000 [385:01:25<53:21:27,  8.11s/it]
 89%|████████▉ | 201327/225000 [385:01:32<51:12:09,  7.79s/it]
 89%|████████▉ | 201328/225000 [385:01:38<48:52:05,  7.43s/it]
 89%|████████▉ | 201329/225000 [385:01:52<61:04:29,  9.29s/it]
 89%|████████▉ | 201330/225000 [385:01:59<56:20:01,  8.57s/it]
 89%|████████▉ | 201331/225000 [385:02:15<70:22:11, 10.70s/it]
 89%|████████▉ | 201332/225000 [385:02:20<60:43:37,  9.24s/it]
 89%|████████▉ | 201333/225000 [385:02:32<65:46:29, 10.01s/it]
 89%|████████▉ | 201334/225000 [385:02:44<69:16:25, 10.54s/it]
 89%|████████▉ | 201335/225000 [385:02:54<68:58:26, 10.49s/it]
 89%|████████▉ | 201336/225000 [385:03:03<65:37:29,  9.98s/it]
 89%|████████▉ | 201337/225000 [385:03:11<61:15:40,  9.32s/it]
 89%|████████▉ | 201338/225000 [385:03:20<60:56:47,  9.27s/it]
 89%|████████▉ | 201339/225000 [385:03:28<57:24:21,  8.73s/it]
 89%|████████▉ | 201340/225000 [385:03:40<64:00:34,  9.74s/it]
 89%|████████▉ | 201341/225000 [385:03:47<59:01:15,  8.98s/it]
 89%|████████▉ | 201342/225000 [385:03:53<52:51:38,  8.04s/it]
 89%|████████▉ | 201343/225000 [385:04:00<51:23:01,  7.82s/it]
 89%|████████▉ | 201344/225000 [385:04:06<48:15:51,  7.34s/it]
 89%|████████▉ | 201345/225000 [385:04:15<50:59:08,  7.76s/it]
 89%|████████▉ | 201346/225000 [385:04:25<56:02:26,  8.53s/it]
 89%|████████▉ | 201347/225000 [385:04:33<54:25:21,  8.28s/it]
 89%|████████▉ | 201348/225000 [385:04:40<51:09:23,  7.79s/it]
 89%|████████▉ | 201349/225000 [385:04:47<50:20:07,  7.66s/it]
 89%|████████▉ | 201350/225000 [385:04:53<46:11:16,  7.03s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.155', 'learning_rate': '1.49e-06', 'epoch': '0.8949', 'num_input_tokens_seen': 646268768, 'train_runtime': '1.386e+06', 'train_tokens_per_second': '466.2'}

 89%|████████▉ | 201350/225000 [385:04:53<46:11:16,  7.03s/it]
 89%|████████▉ | 201351/225000 [385:05:05<57:16:22,  8.72s/it]
 89%|████████▉ | 201352/225000 [385:05:11<52:23:16,  7.98s/it]
 89%|████████▉ | 201353/225000 [385:05:21<54:29:39,  8.30s/it]
 89%|████████▉ | 201354/225000 [385:05:33<62:41:09,  9.54s/it]
 89%|████████▉ | 201355/225000 [385:05:42<60:49:40,  9.26s/it]
 89%|████████▉ | 201356/225000 [385:05:50<59:38:23,  9.08s/it]
 89%|████████▉ | 201357/225000 [385:06:00<60:33:51,  9.22s/it]
 89%|████████▉ | 201358/225000 [385:06:07<55:58:29,  8.52s/it]
 89%|████████▉ | 201359/225000 [385:06:16<57:23:14,  8.74s/it]
 89%|████████▉ | 201360/225000 [385:06:25<57:14:36,  8.72s/it]
 89%|████████▉ | 201361/225000 [385:06:31<52:35:37,  8.01s/it]
 89%|████████▉ | 201362/225000 [385:06:40<54:14:56,  8.26s/it]
 89%|████████▉ | 201363/225000 [385:06:48<53:09:17,  8.10s/it]
 89%|████████▉ | 201364/225000 [385:06:55<51:31:55,  7.85s/it]
 89%|████████▉ | 201365/225000 [385:07:02<49:20:36,  7.52s/it]
 89%|████████▉ | 201366/225000 [385:07:08<47:57:38,  7.31s/it]
 89%|████████▉ | 201367/225000 [385:07:19<53:50:20,  8.20s/it]
 89%|████████▉ | 201368/225000 [385:07:26<51:43:14,  7.88s/it]
 89%|████████▉ | 201369/225000 [385:07:36<56:30:08,  8.61s/it]
 89%|████████▉ | 201370/225000 [385:07:46<58:16:55,  8.88s/it]
 89%|████████▉ | 201371/225000 [385:07:56<60:24:00,  9.20s/it]
 89%|████████▉ | 201372/225000 [385:08:03<56:38:40,  8.63s/it]
 89%|████████▉ | 201373/225000 [385:08:11<54:56:46,  8.37s/it]
 89%|████████▉ | 201374/225000 [385:08:22<61:41:40,  9.40s/it]
 90%|████████▉ | 201375/225000 [385:08:29<55:47:11,  8.50s/it]
 90%|████████▉ | 201376/225000 [385:08:36<52:50:28,  8.05s/it]
 90%|████████▉ | 201377/225000 [385:08:42<49:37:44,  7.56s/it]
 90%|████████▉ | 201378/225000 [385:08:48<45:25:46,  6.92s/it]
 90%|████████▉ | 201379/225000 [385:08:56<48:03:56,  7.33s/it]
 90%|████████▉ | 201380/225000 [385:09:14<68:36:32, 10.46s/it]
 90%|████████▉ | 201381/225000 [385:09:18<57:23:53,  8.75s/it]
 90%|████████▉ | 201382/225000 [385:09:29<59:57:56,  9.14s/it]
 90%|████████▉ | 201383/225000 [385:09:36<55:44:30,  8.50s/it]
 90%|████████▉ | 201384/225000 [385:09:41<50:34:23,  7.71s/it]
 90%|████████▉ | 201385/225000 [385:09:49<49:44:11,  7.58s/it]
 90%|████████▉ | 201386/225000 [385:09:55<46:29:33,  7.09s/it]
 90%|████████▉ | 201387/225000 [385:10:09<60:36:04,  9.24s/it]
 90%|████████▉ | 201388/225000 [385:10:16<57:07:24,  8.71s/it]
 90%|████████▉ | 201389/225000 [385:10:23<52:48:41,  8.05s/it]
 90%|████████▉ | 201390/225000 [385:10:31<52:10:34,  7.96s/it]
 90%|████████▉ | 201391/225000 [385:10:40<56:02:11,  8.54s/it]
 90%|████████▉ | 201392/225000 [385:10:50<57:15:33,  8.73s/it]
 90%|████████▉ | 201393/225000 [385:10:57<55:22:11,  8.44s/it]
 90%|████████▉ | 201394/225000 [385:11:05<53:48:10,  8.21s/it]
 90%|████████▉ | 201395/225000 [385:11:11<48:22:15,  7.38s/it]
 90%|████████▉ | 201396/225000 [385:11:20<53:25:16,  8.15s/it]
 90%|████████▉ | 201397/225000 [385:11:29<53:48:20,  8.21s/it]
 90%|████████▉ | 201398/225000 [385:11:34<47:51:28,  7.30s/it]
 90%|████████▉ | 201399/225000 [385:11:45<55:01:52,  8.39s/it]
 90%|████████▉ | 201400/225000 [385:11:55<58:36:21,  8.94s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.071', 'learning_rate': '1.483e-06', 'epoch': '0.8951', 'num_input_tokens_seen': 646496648, 'train_runtime': '1.387e+06', 'train_tokens_per_second': '466.2'}

 90%|████████▉ | 201400/225000 [385:11:55<58:36:21,  8.94s/it]
 90%|████████▉ | 201401/225000 [385:12:05<60:38:25,  9.25s/it]
 90%|████████▉ | 201402/225000 [385:12:12<55:02:36,  8.40s/it]
 90%|████████▉ | 201403/225000 [385:12:25<65:41:54, 10.02s/it]
 90%|████████▉ | 201404/225000 [385:12:31<57:50:25,  8.82s/it]
 90%|████████▉ | 201405/225000 [385:12:37<51:05:35,  7.80s/it]
 90%|████████▉ | 201406/225000 [385:12:42<45:58:31,  7.01s/it]
 90%|████████▉ | 201407/225000 [385:12:48<44:40:25,  6.82s/it]
 90%|████████▉ | 201408/225000 [385:13:01<56:23:26,  8.60s/it]
 90%|████████▉ | 201409/225000 [385:13:06<49:56:42,  7.62s/it]
 90%|████████▉ | 201410/225000 [385:13:18<57:11:15,  8.73s/it]
 90%|████████▉ | 201411/225000 [385:13:24<52:29:52,  8.01s/it]
 90%|████████▉ | 201412/225000 [385:13:34<56:40:07,  8.65s/it]
 90%|████████▉ | 201413/225000 [385:13:45<60:38:31,  9.26s/it]
 90%|████████▉ | 201414/225000 [385:13:50<51:38:47,  7.88s/it]
 90%|████████▉ | 201415/225000 [385:13:58<52:30:05,  8.01s/it]
 90%|████████▉ | 201416/225000 [385:14:05<50:54:56,  7.77s/it]
 90%|████████▉ | 201417/225000 [385:14:14<52:33:23,  8.02s/it]
 90%|████████▉ | 201418/225000 [385:14:25<58:55:48,  9.00s/it]
 90%|████████▉ | 201419/225000 [385:14:34<59:05:28,  9.02s/it]
 90%|████████▉ | 201420/225000 [385:14:41<54:33:00,  8.33s/it]
 90%|████████▉ | 201421/225000 [385:14:47<49:56:05,  7.62s/it]
 90%|████████▉ | 201422/225000 [385:14:59<58:17:29,  8.90s/it]
 90%|████████▉ | 201423/225000 [385:15:04<51:10:39,  7.81s/it]
 90%|████████▉ | 201424/225000 [385:15:10<47:10:57,  7.20s/it]
 90%|████████▉ | 201425/225000 [385:15:17<47:00:53,  7.18s/it]
 90%|████████▉ | 201426/225000 [385:15:29<56:36:18,  8.64s/it]
 90%|████████▉ | 201427/225000 [385:15:38<57:29:22,  8.78s/it]
 90%|████████▉ | 201428/225000 [385:15:45<53:14:14,  8.13s/it]
 90%|████████▉ | 201429/225000 [385:15:52<51:03:19,  7.80s/it]
 90%|████████▉ | 201430/225000 [385:16:00<51:15:35,  7.83s/it]
 90%|████████▉ | 201431/225000 [385:16:08<51:46:16,  7.91s/it]
 90%|████████▉ | 201432/225000 [385:16:14<49:34:22,  7.57s/it]
 90%|████████▉ | 201433/225000 [385:16:27<59:29:14,  9.09s/it]
 90%|████████▉ | 201434/225000 [385:16:33<52:34:34,  8.03s/it]
 90%|████████▉ | 201435/225000 [385:16:44<59:30:30,  9.09s/it]
 90%|████████▉ | 201436/225000 [385:16:51<55:06:39,  8.42s/it]
 90%|████████▉ | 201437/225000 [385:16:59<54:26:45,  8.32s/it]
 90%|████████▉ | 201438/225000 [385:17:06<52:30:00,  8.02s/it]
 90%|████████▉ | 201439/225000 [385:17:18<59:43:49,  9.13s/it]
 90%|████████▉ | 201440/225000 [385:17:30<65:08:59,  9.95s/it]
 90%|████████▉ | 201441/225000 [385:17:40<64:17:19,  9.82s/it]
 90%|████████▉ | 201442/225000 [385:17:52<68:51:27, 10.52s/it]
 90%|████████▉ | 201443/225000 [385:17:59<62:05:57,  9.49s/it]
 90%|████████▉ | 201444/225000 [385:18:10<64:43:51,  9.89s/it]
 90%|████████▉ | 201445/225000 [385:18:14<54:45:13,  8.37s/it]
 90%|████████▉ | 201446/225000 [385:18:20<49:41:29,  7.59s/it]
 90%|████████▉ | 201447/225000 [385:18:35<63:17:58,  9.68s/it]
 90%|████████▉ | 201448/225000 [385:18:41<56:07:03,  8.58s/it]
 90%|████████▉ | 201449/225000 [385:18:48<52:41:11,  8.05s/it]
 90%|████████▉ | 201450/225000 [385:18:54<49:45:23,  7.61s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.14', 'learning_rate': '1.477e-06', 'epoch': '0.8953', 'num_input_tokens_seen': 646725016, 'train_runtime': '1.387e+06', 'train_tokens_per_second': '466.2'}

 90%|████████▉ | 201450/225000 [385:18:54<49:45:23,  7.61s/it]
 90%|████████▉ | 201451/225000 [385:19:01<47:35:22,  7.28s/it]
 90%|████████▉ | 201452/225000 [385:19:12<56:34:38,  8.65s/it]
 90%|████████▉ | 201453/225000 [385:19:20<53:30:10,  8.18s/it]
 90%|████████▉ | 201454/225000 [385:19:28<54:03:20,  8.26s/it]
 90%|████████▉ | 201455/225000 [385:19:34<49:36:18,  7.58s/it]
 90%|████████▉ | 201456/225000 [385:19:44<53:46:20,  8.22s/it]
 90%|████████▉ | 201457/225000 [385:19:49<48:54:26,  7.48s/it]
 90%|████████▉ | 201458/225000 [385:19:57<48:39:23,  7.44s/it]
 90%|████████▉ | 201459/225000 [385:20:03<46:55:42,  7.18s/it]
 90%|████████▉ | 201460/225000 [385:20:19<64:07:57,  9.81s/it]
 90%|████████▉ | 201461/225000 [385:20:26<57:51:45,  8.85s/it]
 90%|████████▉ | 201462/225000 [385:20:35<57:58:52,  8.87s/it]
 90%|████████▉ | 201463/225000 [385:20:43<55:49:18,  8.54s/it]
 90%|████████▉ | 201464/225000 [385:20:51<56:01:40,  8.57s/it]
 90%|████████▉ | 201465/225000 [385:20:56<49:20:48,  7.55s/it]
 90%|████████▉ | 201466/225000 [385:21:08<57:27:16,  8.79s/it]
 90%|████████▉ | 201467/225000 [385:21:15<53:44:49,  8.22s/it]
 90%|████████▉ | 201468/225000 [385:21:24<54:31:57,  8.34s/it]
 90%|████████▉ | 201469/225000 [385:21:36<62:34:41,  9.57s/it]
 90%|████████▉ | 201470/225000 [385:21:42<56:07:40,  8.59s/it]
 90%|████████▉ | 201471/225000 [385:21:50<53:33:50,  8.20s/it]
 90%|████████▉ | 201472/225000 [385:22:06<68:48:54, 10.53s/it]
 90%|████████▉ | 201473/225000 [385:22:12<61:03:06,  9.34s/it]
 90%|████████▉ | 201474/225000 [385:22:18<54:32:50,  8.35s/it]
 90%|████████▉ | 201475/225000 [385:22:24<49:56:19,  7.64s/it]
 90%|████████▉ | 201476/225000 [385:22:32<50:24:24,  7.71s/it]
 90%|████████▉ | 201477/225000 [385:22:37<45:14:27,  6.92s/it]
 90%|████████▉ | 201478/225000 [385:22:45<47:29:15,  7.27s/it]
 90%|████████▉ | 201479/225000 [385:23:01<64:06:15,  9.81s/it]
 90%|████████▉ | 201480/225000 [385:23:08<58:33:48,  8.96s/it]
 90%|████████▉ | 201481/225000 [385:23:13<49:58:32,  7.65s/it]
 90%|████████▉ | 201482/225000 [385:23:19<48:05:36,  7.36s/it]
 90%|████████▉ | 201483/225000 [385:23:27<48:14:28,  7.38s/it]
 90%|████████▉ | 201484/225000 [385:23:34<47:42:45,  7.30s/it]
 90%|████████▉ | 201485/225000 [385:23:46<57:22:28,  8.78s/it]
 90%|████████▉ | 201486/225000 [385:23:57<61:18:11,  9.39s/it]
 90%|████████▉ | 201487/225000 [385:24:03<54:26:51,  8.34s/it]
 90%|████████▉ | 201488/225000 [385:24:11<54:06:01,  8.28s/it]
 90%|████████▉ | 201489/225000 [385:24:18<52:11:14,  7.99s/it]
 90%|████████▉ | 201490/225000 [385:24:25<49:56:57,  7.65s/it]
 90%|████████▉ | 201491/225000 [385:24:34<52:27:59,  8.03s/it]
 90%|████████▉ | 201492/225000 [385:24:42<51:49:54,  7.94s/it]
 90%|████████▉ | 201493/225000 [385:24:48<48:49:44,  7.48s/it]
 90%|████████▉ | 201494/225000 [385:24:58<53:02:58,  8.12s/it]
 90%|████████▉ | 201495/225000 [385:25:07<55:24:53,  8.49s/it]
 90%|████████▉ | 201496/225000 [385:25:13<50:39:48,  7.76s/it]
 90%|████████▉ | 201497/225000 [385:25:20<49:13:36,  7.54s/it]
 90%|████████▉ | 201498/225000 [385:25:34<61:06:18,  9.36s/it]
 90%|████████▉ | 201499/225000 [385:25:42<58:06:10,  8.90s/it]
 90%|████████▉ | 201500/225000 [385:25:48<53:06:48,  8.14s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.164', 'learning_rate': '1.471e-06', 'epoch': '0.8956', 'num_input_tokens_seen': 646953312, 'train_runtime': '1.388e+06', 'train_tokens_per_second': '466.3'}

 90%|████████▉ | 201500/225000 [385:25:48<53:06:48,  8.14s/it]
 90%|████████▉ | 201501/225000 [385:25:58<56:59:05,  8.73s/it]
 90%|████████▉ | 201502/225000 [385:26:08<60:16:21,  9.23s/it]
 90%|████████▉ | 201503/225000 [385:26:14<53:43:33,  8.23s/it]
 90%|████████▉ | 201504/225000 [385:26:25<58:09:11,  8.91s/it]
 90%|████████▉ | 201505/225000 [385:26:34<58:21:44,  8.94s/it]
 90%|████████▉ | 201506/225000 [385:26:41<55:19:57,  8.48s/it]
 90%|████████▉ | 201507/225000 [385:26:53<61:23:44,  9.41s/it]
 90%|████████▉ | 201508/225000 [385:27:00<56:09:26,  8.61s/it]
 90%|████████▉ | 201509/225000 [385:27:10<59:39:08,  9.14s/it]
 90%|████████▉ | 201510/225000 [385:27:16<52:56:32,  8.11s/it]
 90%|████████▉ | 201511/225000 [385:27:27<59:09:28,  9.07s/it]
 90%|████████▉ | 201512/225000 [385:27:37<60:40:36,  9.30s/it]
 90%|████████▉ | 201513/225000 [385:27:44<57:27:06,  8.81s/it]
 90%|████████▉ | 201514/225000 [385:27:51<52:44:09,  8.08s/it]
 90%|████████▉ | 201515/225000 [385:27:57<49:38:44,  7.61s/it]
 90%|████████▉ | 201516/225000 [385:28:13<66:02:01, 10.12s/it]
 90%|████████▉ | 201517/225000 [385:28:21<60:57:26,  9.34s/it]
 90%|████████▉ | 201518/225000 [385:28:29<58:11:22,  8.92s/it]
 90%|████████▉ | 201519/225000 [385:28:36<55:28:46,  8.51s/it]
 90%|████████▉ | 201520/225000 [385:28:44<53:30:10,  8.20s/it]
 90%|████████▉ | 201521/225000 [385:29:00<68:33:57, 10.51s/it]
 90%|████████▉ | 201522/225000 [385:29:05<57:49:07,  8.87s/it]
 90%|████████▉ | 201523/225000 [385:29:12<54:10:21,  8.31s/it]
 90%|████████▉ | 201524/225000 [385:29:23<59:53:07,  9.18s/it]
 90%|████████▉ | 201525/225000 [385:29:28<52:37:44,  8.07s/it]
 90%|████████▉ | 201526/225000 [385:29:33<46:06:17,  7.07s/it]
 90%|████████▉ | 201527/225000 [385:29:48<60:28:26,  9.27s/it]
 90%|████████▉ | 201528/225000 [385:29:56<58:57:57,  9.04s/it]
 90%|████████▉ | 201529/225000 [385:30:04<57:25:39,  8.81s/it]
 90%|████████▉ | 201530/225000 [385:30:15<61:23:30,  9.42s/it]
 90%|████████▉ | 201531/225000 [385:30:24<60:30:33,  9.28s/it]
 90%|████████▉ | 201532/225000 [385:30:30<53:12:22,  8.16s/it]
 90%|████████▉ | 201533/225000 [385:30:42<60:39:33,  9.31s/it]
 90%|████████▉ | 201534/225000 [385:30:52<62:57:41,  9.66s/it]
 90%|████████▉ | 201535/225000 [385:30:57<53:59:07,  8.28s/it]
 90%|████████▉ | 201536/225000 [385:31:09<60:27:29,  9.28s/it]
 90%|████████▉ | 201537/225000 [385:31:16<56:25:34,  8.66s/it]
 90%|████████▉ | 201538/225000 [385:31:25<56:47:50,  8.71s/it]
 90%|████████▉ | 201539/225000 [385:31:35<58:37:47,  9.00s/it]
 90%|████████▉ | 201540/225000 [385:31:47<65:38:21, 10.07s/it]
 90%|████████▉ | 201541/225000 [385:31:53<57:38:50,  8.85s/it]
 90%|████████▉ | 201542/225000 [385:32:02<56:55:53,  8.74s/it]
 90%|████████▉ | 201543/225000 [385:32:12<59:09:15,  9.08s/it]
 90%|████████▉ | 201544/225000 [385:32:18<53:19:55,  8.19s/it]
 90%|████████▉ | 201545/225000 [385:32:24<49:57:11,  7.67s/it]
 90%|████████▉ | 201546/225000 [385:32:35<55:21:16,  8.50s/it]
 90%|████████▉ | 201547/225000 [385:32:41<50:26:29,  7.74s/it]
 90%|████████▉ | 201548/225000 [385:32:51<56:19:53,  8.65s/it]
 90%|████████▉ | 201549/225000 [385:32:56<49:31:13,  7.60s/it]
 90%|████████▉ | 201550/225000 [385:33:07<55:42:03,  8.55s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.217', 'learning_rate': '1.465e-06', 'epoch': '0.8958', 'num_input_tokens_seen': 647185656, 'train_runtime': '1.388e+06', 'train_tokens_per_second': '466.3'}

 90%|████████▉ | 201550/225000 [385:33:07<55:42:03,  8.55s/it]
 90%|████████▉ | 201551/225000 [385:33:18<59:26:49,  9.13s/it]
 90%|████████▉ | 201552/225000 [385:33:27<60:47:22,  9.33s/it]
 90%|████████▉ | 201553/225000 [385:33:38<63:03:54,  9.68s/it]
 90%|████████▉ | 201554/225000 [385:33:44<56:35:00,  8.69s/it]
 90%|████████▉ | 201555/225000 [385:33:51<52:09:45,  8.01s/it]
 90%|████████▉ | 201556/225000 [385:34:02<57:28:55,  8.83s/it]
 90%|████████▉ | 201557/225000 [385:34:09<55:19:45,  8.50s/it]
 90%|████████▉ | 201558/225000 [385:34:16<51:49:28,  7.96s/it]
 90%|████████▉ | 201559/225000 [385:34:23<50:41:50,  7.79s/it]
 90%|████████▉ | 201560/225000 [385:34:31<49:59:16,  7.68s/it]
 90%|████████▉ | 201561/225000 [385:34:40<53:58:19,  8.29s/it]
 90%|████████▉ | 201562/225000 [385:34:50<57:08:55,  8.78s/it]
 90%|████████▉ | 201563/225000 [385:34:57<52:59:51,  8.14s/it]
 90%|████████▉ | 201564/225000 [385:35:09<59:40:47,  9.17s/it]
 90%|████████▉ | 201565/225000 [385:35:19<62:07:45,  9.54s/it]
 90%|████████▉ | 201566/225000 [385:35:24<52:57:05,  8.13s/it]
 90%|████████▉ | 201567/225000 [385:35:35<58:54:53,  9.05s/it]
 90%|████████▉ | 201568/225000 [385:35:40<51:41:01,  7.94s/it]
 90%|████████▉ | 201569/225000 [385:35:55<65:24:46, 10.05s/it]
 90%|████████▉ | 201570/225000 [385:36:01<56:02:46,  8.61s/it]
 90%|████████▉ | 201571/225000 [385:36:12<61:34:39,  9.46s/it]
 90%|████████▉ | 201572/225000 [385:36:18<54:25:33,  8.36s/it]
 90%|████████▉ | 201573/225000 [385:36:29<60:27:20,  9.29s/it]
 90%|████████▉ | 201574/225000 [385:36:34<51:43:33,  7.95s/it]
 90%|████████▉ | 201575/225000 [385:36:39<46:09:52,  7.09s/it]
 90%|████████▉ | 201576/225000 [385:36:51<55:33:41,  8.54s/it]
 90%|████████▉ | 201577/225000 [385:36:56<48:31:42,  7.46s/it]
 90%|████████▉ | 201578/225000 [385:37:07<54:34:50,  8.39s/it]
 90%|████████▉ | 201579/225000 [385:37:12<48:43:23,  7.49s/it]
 90%|████████▉ | 201580/225000 [385:37:20<49:16:43,  7.57s/it]
 90%|████████▉ | 201581/225000 [385:37:28<50:09:30,  7.71s/it]
 90%|████████▉ | 201582/225000 [385:37:39<57:08:27,  8.78s/it]
 90%|████████▉ | 201583/225000 [385:37:49<59:39:42,  9.17s/it]
 90%|████████▉ | 201584/225000 [385:37:59<60:53:45,  9.36s/it]
 90%|████████▉ | 201585/225000 [385:38:06<56:14:40,  8.65s/it]
 90%|████████▉ | 201586/225000 [385:38:13<52:51:41,  8.13s/it]
 90%|████████▉ | 201587/225000 [385:38:24<58:26:56,  8.99s/it]
 90%|████████▉ | 201588/225000 [385:38:32<57:29:28,  8.84s/it]
 90%|████████▉ | 201589/225000 [385:38:44<62:38:47,  9.63s/it]
 90%|████████▉ | 201590/225000 [385:38:53<61:35:08,  9.47s/it]
 90%|████████▉ | 201591/225000 [385:39:01<58:09:33,  8.94s/it]
 90%|████████▉ | 201592/225000 [385:39:11<61:34:00,  9.47s/it]
 90%|████████▉ | 201593/225000 [385:39:18<55:52:27,  8.59s/it]
 90%|████████▉ | 201594/225000 [385:39:28<57:59:26,  8.92s/it]
 90%|████████▉ | 201595/225000 [385:39:41<65:58:38, 10.15s/it]
 90%|████████▉ | 201596/225000 [385:39:47<58:11:46,  8.95s/it]
 90%|████████▉ | 201597/225000 [385:39:53<53:23:42,  8.21s/it]
 90%|████████▉ | 201598/225000 [385:40:06<62:04:45,  9.55s/it]
 90%|████████▉ | 201599/225000 [385:40:17<64:02:38,  9.85s/it]
 90%|████████▉ | 201600/225000 [385:40:27<64:33:16,  9.93s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.137', 'learning_rate': '1.459e-06', 'epoch': '0.896', 'num_input_tokens_seen': 647420688, 'train_runtime': '1.388e+06', 'train_tokens_per_second': '466.3'}

 90%|████████▉ | 201600/225000 [385:40:27<64:33:16,  9.93s/it]
 90%|████████▉ | 201601/225000 [385:40:38<68:18:54, 10.51s/it]
 90%|████████▉ | 201602/225000 [385:40:44<58:51:06,  9.05s/it]
 90%|████████▉ | 201603/225000 [385:40:55<62:13:26,  9.57s/it]
 90%|████████▉ | 201604/225000 [385:41:00<53:43:56,  8.27s/it]
 90%|████████▉ | 201605/225000 [385:41:10<56:06:19,  8.63s/it]
 90%|████████▉ | 201606/225000 [385:41:18<56:07:24,  8.64s/it]
 90%|████████▉ | 201607/225000 [385:41:29<59:36:40,  9.17s/it]
 90%|████████▉ | 201608/225000 [385:41:37<58:13:24,  8.96s/it]
 90%|████████▉ | 201609/225000 [385:41:43<51:14:49,  7.89s/it]
 90%|████████▉ | 201610/225000 [385:41:51<52:54:00,  8.14s/it]
 90%|████████▉ | 201611/225000 [385:42:02<58:48:42,  9.05s/it]
 90%|████████▉ | 201612/225000 [385:42:13<61:56:32,  9.53s/it]
 90%|████████▉ | 201613/225000 [385:42:19<54:33:50,  8.40s/it]
 90%|████████▉ | 201614/225000 [385:42:28<56:32:41,  8.70s/it]
 90%|████████▉ | 201615/225000 [385:42:36<55:29:59,  8.54s/it]
 90%|████████▉ | 201616/225000 [385:42:47<59:27:13,  9.15s/it]
 90%|████████▉ | 201617/225000 [385:42:57<61:43:27,  9.50s/it]
 90%|████████▉ | 201618/225000 [385:43:08<63:15:17,  9.74s/it]
 90%|████████▉ | 201619/225000 [385:43:14<57:32:54,  8.86s/it]
 90%|████████▉ | 201620/225000 [385:43:25<61:16:56,  9.44s/it]
 90%|████████▉ | 201621/225000 [385:43:34<60:18:12,  9.29s/it]
 90%|████████▉ | 201622/225000 [385:43:43<58:39:49,  9.03s/it]
 90%|████████▉ | 201623/225000 [385:43:52<59:58:35,  9.24s/it]
 90%|████████▉ | 201624/225000 [385:44:01<58:13:24,  8.97s/it]
 90%|████████▉ | 201625/225000 [385:44:08<54:12:17,  8.35s/it]
 90%|████████▉ | 201626/225000 [385:44:13<48:46:12,  7.51s/it]
 90%|████████▉ | 201627/225000 [385:44:27<60:13:21,  9.28s/it]
 90%|████████▉ | 201628/225000 [385:44:31<51:14:30,  7.89s/it]
 90%|████████▉ | 201629/225000 [385:44:40<53:15:17,  8.20s/it]
 90%|████████▉ | 201630/225000 [385:44:45<47:10:29,  7.27s/it]
 90%|████████▉ | 201631/225000 [385:44:52<46:49:22,  7.21s/it]
 90%|████████▉ | 201632/225000 [385:45:07<61:24:35,  9.46s/it]
 90%|████████▉ | 201633/225000 [385:45:15<58:01:59,  8.94s/it]
 90%|████████▉ | 201634/225000 [385:45:25<61:08:26,  9.42s/it]
 90%|████████▉ | 201635/225000 [385:45:30<52:45:21,  8.13s/it]
 90%|████████▉ | 201636/225000 [385:45:39<53:37:44,  8.26s/it]
 90%|████████▉ | 201637/225000 [385:45:51<61:27:05,  9.47s/it]
 90%|████████▉ | 201638/225000 [385:45:59<59:02:54,  9.10s/it]
 90%|████████▉ | 201639/225000 [385:46:07<55:29:26,  8.55s/it]
 90%|████████▉ | 201640/225000 [385:46:13<51:27:40,  7.93s/it]
 90%|████████▉ | 201641/225000 [385:46:22<52:47:45,  8.14s/it]
 90%|████████▉ | 201642/225000 [385:46:27<47:44:18,  7.36s/it]
 90%|████████▉ | 201643/225000 [385:46:33<43:55:45,  6.77s/it]
 90%|████████▉ | 201644/225000 [385:46:45<53:36:33,  8.26s/it]
 90%|████████▉ | 201645/225000 [385:46:56<59:04:22,  9.11s/it]
 90%|████████▉ | 201646/225000 [385:47:06<60:38:54,  9.35s/it]
 90%|████████▉ | 201647/225000 [385:47:16<62:30:31,  9.64s/it]
 90%|████████▉ | 201648/225000 [385:47:22<56:06:37,  8.65s/it]
 90%|████████▉ | 201649/225000 [385:47:30<55:17:16,  8.52s/it]
 90%|████████▉ | 201650/225000 [385:47:38<53:42:02,  8.28s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.407', 'learning_rate': '1.453e-06', 'epoch': '0.8962', 'num_input_tokens_seen': 647651216, 'train_runtime': '1.389e+06', 'train_tokens_per_second': '466.3'}

 90%|████████▉ | 201650/225000 [385:47:38<53:42:02,  8.28s/it]
 90%|████████▉ | 201651/225000 [385:47:45<50:37:20,  7.81s/it]
 90%|████████▉ | 201652/225000 [385:47:51<48:20:18,  7.45s/it]
 90%|████████▉ | 201653/225000 [385:48:04<58:43:07,  9.05s/it]
 90%|████████▉ | 201654/225000 [385:48:14<59:30:27,  9.18s/it]
 90%|████████▉ | 201655/225000 [385:48:24<62:27:57,  9.63s/it]
 90%|████████▉ | 201656/225000 [385:48:31<56:16:57,  8.68s/it]
 90%|████████▉ | 201657/225000 [385:48:37<51:34:55,  7.96s/it]
 90%|████████▉ | 201658/225000 [385:48:43<48:13:24,  7.44s/it]
 90%|████████▉ | 201659/225000 [385:48:55<56:35:29,  8.73s/it]
 90%|████████▉ | 201660/225000 [385:49:01<51:19:46,  7.92s/it]
 90%|████████▉ | 201661/225000 [385:49:14<61:20:53,  9.46s/it]
 90%|████████▉ | 201662/225000 [385:49:20<54:29:10,  8.40s/it]
 90%|████████▉ | 201663/225000 [385:49:30<57:26:59,  8.86s/it]
 90%|████████▉ | 201664/225000 [385:49:36<52:38:17,  8.12s/it]
 90%|████████▉ | 201665/225000 [385:49:43<49:06:08,  7.58s/it]
 90%|████████▉ | 201666/225000 [385:49:48<45:00:37,  6.94s/it]
 90%|████████▉ | 201667/225000 [385:49:56<46:47:56,  7.22s/it]
 90%|████████▉ | 201668/225000 [385:50:10<59:19:34,  9.15s/it]
 90%|████████▉ | 201669/225000 [385:50:15<52:19:08,  8.07s/it]
 90%|████████▉ | 201670/225000 [385:50:23<51:03:07,  7.88s/it]
 90%|████████▉ | 201671/225000 [385:50:33<55:06:00,  8.50s/it]
 90%|████████▉ | 201672/225000 [385:50:44<59:51:40,  9.24s/it]
 90%|████████▉ | 201673/225000 [385:50:51<55:46:19,  8.61s/it]
 90%|████████▉ | 201674/225000 [385:51:00<57:00:41,  8.80s/it]
 90%|████████▉ | 201675/225000 [385:51:08<55:46:28,  8.61s/it]
 90%|████████▉ | 201676/225000 [385:51:15<52:32:55,  8.11s/it]
 90%|████████▉ | 201677/225000 [385:51:22<50:51:58,  7.85s/it]
 90%|████████▉ | 201678/225000 [385:51:29<48:17:14,  7.45s/it]
 90%|████████▉ | 201679/225000 [385:51:36<47:53:39,  7.39s/it]
 90%|████████▉ | 201680/225000 [385:51:49<57:50:47,  8.93s/it]
 90%|████████▉ | 201681/225000 [385:51:55<53:32:09,  8.26s/it]
 90%|████████▉ | 201682/225000 [385:52:06<58:47:56,  9.08s/it]
 90%|████████▉ | 201683/225000 [385:52:14<56:24:34,  8.71s/it]
 90%|████████▉ | 201684/225000 [385:52:22<53:46:09,  8.30s/it]
 90%|████████▉ | 201685/225000 [385:52:33<59:40:10,  9.21s/it]
 90%|████████▉ | 201686/225000 [385:52:43<61:06:13,  9.44s/it]
 90%|████████▉ | 201687/225000 [385:52:50<57:27:04,  8.87s/it]
 90%|████████▉ | 201688/225000 [385:52:58<55:52:17,  8.63s/it]
 90%|████████▉ | 201689/225000 [385:53:05<51:11:49,  7.91s/it]
 90%|████████▉ | 201690/225000 [385:53:18<61:45:18,  9.54s/it]
 90%|████████▉ | 201691/225000 [385:53:25<57:35:52,  8.90s/it]
 90%|████████▉ | 201692/225000 [385:53:31<51:16:07,  7.92s/it]
 90%|████████▉ | 201693/225000 [385:53:40<53:42:19,  8.30s/it]
 90%|████████▉ | 201694/225000 [385:53:49<53:50:35,  8.32s/it]
 90%|████████▉ | 201695/225000 [385:53:56<52:22:15,  8.09s/it]
 90%|████████▉ | 201696/225000 [385:54:04<51:01:35,  7.88s/it]
 90%|████████▉ | 201697/225000 [385:54:08<45:06:37,  6.97s/it]
 90%|████████▉ | 201698/225000 [385:54:19<52:03:00,  8.04s/it]
 90%|████████▉ | 201699/225000 [385:54:28<53:06:52,  8.21s/it]
 90%|████████▉ | 201700/225000 [385:54:36<53:03:17,  8.20s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.128', 'learning_rate': '1.446e-06', 'epoch': '0.8964', 'num_input_tokens_seen': 647877584, 'train_runtime': '1.389e+06', 'train_tokens_per_second': '466.3'}

 90%|████████▉ | 201700/225000 [385:54:36<53:03:17,  8.20s/it]
 90%|████████▉ | 201701/225000 [385:54:42<50:10:08,  7.75s/it]
 90%|████████▉ | 201702/225000 [385:54:55<60:21:56,  9.33s/it]
 90%|████████▉ | 201703/225000 [385:55:01<53:36:35,  8.28s/it]
 90%|████████▉ | 201704/225000 [385:55:10<53:37:34,  8.29s/it]
 90%|████████▉ | 201705/225000 [385:55:16<49:44:16,  7.69s/it]
 90%|████████▉ | 201706/225000 [385:55:21<44:41:40,  6.91s/it]
 90%|████████▉ | 201707/225000 [385:55:25<40:00:18,  6.18s/it]
 90%|████████▉ | 201708/225000 [385:55:36<48:45:15,  7.54s/it]
 90%|████████▉ | 201709/225000 [385:55:44<48:36:08,  7.51s/it]
 90%|████████▉ | 201710/225000 [385:55:52<49:54:49,  7.72s/it]
 90%|████████▉ | 201711/225000 [385:55:59<49:15:00,  7.61s/it]
 90%|████████▉ | 201712/225000 [385:56:12<58:40:30,  9.07s/it]
 90%|████████▉ | 201713/225000 [385:56:19<55:25:50,  8.57s/it]
 90%|████████▉ | 201714/225000 [385:56:28<56:35:39,  8.75s/it]
 90%|████████▉ | 201715/225000 [385:56:35<52:04:27,  8.05s/it]
 90%|████████▉ | 201716/225000 [385:56:40<47:52:24,  7.40s/it]
 90%|████████▉ | 201717/225000 [385:56:47<46:21:49,  7.17s/it]
 90%|████████▉ | 201718/225000 [385:57:02<61:33:36,  9.52s/it]
 90%|████████▉ | 201719/225000 [385:57:07<53:03:08,  8.20s/it]
 90%|████████▉ | 201720/225000 [385:57:18<57:35:59,  8.91s/it]
 90%|████████▉ | 201721/225000 [385:57:24<51:48:18,  8.01s/it]
 90%|████████▉ | 201722/225000 [385:57:29<47:07:32,  7.29s/it]
 90%|████████▉ | 201723/225000 [385:57:40<54:24:30,  8.41s/it]
 90%|████████▉ | 201724/225000 [385:57:49<54:15:09,  8.39s/it]
 90%|████████▉ | 201725/225000 [385:57:59<58:15:58,  9.01s/it]
 90%|████████▉ | 201726/225000 [385:58:08<58:15:24,  9.01s/it]
 90%|████████▉ | 201727/225000 [385:58:21<65:35:34, 10.15s/it]
 90%|████████▉ | 201728/225000 [385:58:31<65:24:07, 10.12s/it]
 90%|████████▉ | 201729/225000 [385:58:38<60:04:36,  9.29s/it]
 90%|████████▉ | 201730/225000 [385:58:45<54:52:00,  8.49s/it]
 90%|████████▉ | 201731/225000 [385:58:50<48:06:30,  7.44s/it]
 90%|████████▉ | 201732/225000 [385:58:57<47:31:21,  7.35s/it]
 90%|████████▉ | 201733/225000 [385:59:08<54:25:23,  8.42s/it]
 90%|████████▉ | 201734/225000 [385:59:15<50:57:21,  7.88s/it]
 90%|████████▉ | 201735/225000 [385:59:25<55:28:27,  8.58s/it]
 90%|████████▉ | 201736/225000 [385:59:30<49:23:05,  7.64s/it]
 90%|████████▉ | 201737/225000 [385:59:40<52:54:36,  8.19s/it]
 90%|████████▉ | 201738/225000 [385:59:46<48:40:24,  7.53s/it]
 90%|████████▉ | 201739/225000 [385:59:56<53:46:05,  8.32s/it]
 90%|████████▉ | 201740/225000 [386:00:03<51:56:41,  8.04s/it]
 90%|████████▉ | 201741/225000 [386:00:10<49:11:56,  7.61s/it]
 90%|████████▉ | 201742/225000 [386:00:16<45:45:57,  7.08s/it]
 90%|████████▉ | 201743/225000 [386:00:25<49:52:22,  7.72s/it]
 90%|████████▉ | 201744/225000 [386:00:34<52:29:53,  8.13s/it]
 90%|████████▉ | 201745/225000 [386:00:38<43:51:05,  6.79s/it]
 90%|████████▉ | 201746/225000 [386:00:48<51:27:55,  7.97s/it]
 90%|████████▉ | 201747/225000 [386:00:53<45:22:27,  7.02s/it]
 90%|████████▉ | 201748/225000 [386:01:02<47:48:25,  7.40s/it]
 90%|████████▉ | 201749/225000 [386:01:08<45:07:44,  6.99s/it]
 90%|████████▉ | 201750/225000 [386:01:15<45:51:50,  7.10s/it]
                                                              
{'loss': '1.329', 'grad_norm': '1.305', 'learning_rate': '1.44e-06', 'epoch': '0.8967', 'num_input_tokens_seen': 648101208, 'train_runtime': '1.39e+06', 'train_tokens_per_second': '466.4'}

 90%|████████▉ | 201750/225000 [386:01:15<45:51:50,  7.10s/it]
 90%|████████▉ | 201751/225000 [386:01:23<47:09:19,  7.30s/it]
 90%|████████▉ | 201752/225000 [386:01:31<48:24:56,  7.50s/it]
 90%|████████▉ | 201753/225000 [386:01:39<49:01:53,  7.59s/it]
 90%|████████▉ | 201754/225000 [386:01:54<63:26:21,  9.82s/it]
 90%|████████▉ | 201755/225000 [386:02:00<56:32:14,  8.76s/it]
 90%|████████▉ | 201756/225000 [386:02:08<55:58:42,  8.67s/it]
 90%|████████▉ | 201757/225000 [386:02:17<56:16:19,  8.72s/it]
 90%|████████▉ | 201758/225000 [386:02:23<50:31:05,  7.82s/it]
 90%|████████▉ | 201759/225000 [386:02:30<49:43:09,  7.70s/it]
 90%|████████▉ | 201760/225000 [386:02:45<64:05:37,  9.93s/it]
 90%|████████▉ | 201761/225000 [386:02:54<61:38:13,  9.55s/it]
 90%|████████▉ | 201762/225000 [386:03:01<56:23:29,  8.74s/it]
 90%|████████▉ | 201763/225000 [386:03:08<53:45:19,  8.33s/it]
 90%|████████▉ | 201764/225000 [386:03:15<50:11:03,  7.78s/it]
 90%|████████▉ | 201765/225000 [386:03:22<48:23:37,  7.50s/it]
 90%|████████▉ | 201766/225000 [386:03:27<43:33:42,  6.75s/it]
 90%|████████▉ | 201767/225000 [386:03:36<48:23:56,  7.50s/it]
 90%|████████▉ | 201768/225000 [386:03:43<48:17:49,  7.48s/it]
 90%|████████▉ | 201769/225000 [386:03:54<54:43:21,  8.48s/it]
 90%|████████▉ | 201770/225000 [386:04:00<49:41:06,  7.70s/it]
 90%|████████▉ | 201771/225000 [386:04:10<54:09:11,  8.39s/it]
 90%|████████▉ | 201772/225000 [386:04:21<58:23:54,  9.05s/it]
 90%|████████▉ | 201773/225000 [386:04:26<51:13:15,  7.94s/it]
 90%|████████▉ | 201774/225000 [386:04:31<45:47:23,  7.10s/it]
 90%|████████▉ | 201775/225000 [386:04:42<52:21:15,  8.12s/it]
 90%|████████▉ | 201776/225000 [386:04:48<49:08:40,  7.62s/it]
 90%|████████▉ | 201777/225000 [386:05:02<61:02:52,  9.46s/it]
 90%|████████▉ | 201778/225000 [386:05:08<55:28:00,  8.60s/it]
 90%|████████▉ | 201779/225000 [386:05:17<54:35:58,  8.46s/it]
 90%|████████▉ | 201780/225000 [386:05:24<53:04:18,  8.23s/it]
 90%|████████▉ | 201781/225000 [386:05:36<59:23:14,  9.21s/it]
 90%|████████▉ | 201782/225000 [386:05:44<58:17:24,  9.04s/it]
 90%|████████▉ | 201783/225000 [386:05:52<55:48:51,  8.65s/it]
 90%|████████▉ | 201784/225000 [386:06:00<54:55:43,  8.52s/it]
 90%|████████▉ | 201785/225000 [386:06:10<56:18:57,  8.73s/it]
 90%|████████▉ | 201786/225000 [386:06:22<63:28:37,  9.84s/it]
 90%|████████▉ | 201787/225000 [386:06:33<64:53:58, 10.06s/it]
 90%|████████▉ | 201788/225000 [386:06:39<58:48:47,  9.12s/it]
 90%|████████▉ | 201789/225000 [386:06:47<56:41:04,  8.79s/it]
 90%|████████▉ | 201790/225000 [386:06:54<52:07:49,  8.09s/it]
 90%|████████▉ | 201791/225000 [386:07:01<50:03:06,  7.76s/it]
 90%|████████▉ | 201792/225000 [386:07:12<56:14:51,  8.73s/it]
 90%|████████▉ | 201793/225000 [386:07:24<62:35:33,  9.71s/it]
 90%|████████▉ | 201794/225000 [386:07:35<65:24:35, 10.15s/it]
 90%|████████▉ | 201795/225000 [386:07:45<64:02:22,  9.94s/it]
 90%|████████▉ | 201796/225000 [386:07:54<62:14:34,  9.66s/it]
 90%|████████▉ | 201797/225000 [386:08:00<55:22:25,  8.59s/it]
 90%|████████▉ | 201798/225000 [386:08:13<64:39:29, 10.03s/it]
 90%|████████▉ | 201799/225000 [386:08:20<58:02:57,  9.01s/it]
 90%|████████▉ | 201800/225000 [386:08:29<58:37:47,  9.10s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.526', 'learning_rate': '1.434e-06', 'epoch': '0.8969', 'num_input_tokens_seen': 648335240, 'train_runtime': '1.39e+06', 'train_tokens_per_second': '466.4'}

 90%|████████▉ | 201800/225000 [386:08:29<58:37:47,  9.10s/it]
 90%|████████▉ | 201801/225000 [386:08:41<64:10:15,  9.96s/it]
 90%|████████▉ | 201802/225000 [386:08:46<55:11:44,  8.57s/it]
 90%|████████▉ | 201803/225000 [386:08:53<51:57:45,  8.06s/it]
 90%|████████▉ | 201804/225000 [386:09:00<49:02:01,  7.61s/it]
 90%|████████▉ | 201805/225000 [386:09:12<57:23:38,  8.91s/it]
 90%|████████▉ | 201806/225000 [386:09:19<54:53:28,  8.52s/it]
 90%|████████▉ | 201807/225000 [386:09:30<58:48:23,  9.13s/it]
 90%|████████▉ | 201808/225000 [386:09:40<60:10:19,  9.34s/it]
 90%|████████▉ | 201809/225000 [386:09:47<56:36:53,  8.79s/it]
 90%|████████▉ | 201810/225000 [386:09:53<51:51:14,  8.05s/it]
 90%|████████▉ | 201811/225000 [386:10:03<55:02:54,  8.55s/it]
 90%|████████▉ | 201812/225000 [386:10:09<49:19:15,  7.66s/it]
 90%|████████▉ | 201813/225000 [386:10:20<55:46:07,  8.66s/it]
 90%|████████▉ | 201814/225000 [386:10:24<47:54:34,  7.44s/it]
 90%|████████▉ | 201815/225000 [386:10:34<52:48:09,  8.20s/it]
 90%|████████▉ | 201816/225000 [386:10:43<53:13:04,  8.26s/it]
 90%|████████▉ | 201817/225000 [386:10:52<54:56:53,  8.53s/it]
 90%|████████▉ | 201818/225000 [386:10:58<51:01:58,  7.93s/it]
 90%|████████▉ | 201819/225000 [386:11:09<56:12:31,  8.73s/it]
 90%|████████▉ | 201820/225000 [386:11:14<48:07:58,  7.48s/it]
 90%|████████▉ | 201821/225000 [386:11:22<49:12:21,  7.64s/it]
 90%|████████▉ | 201822/225000 [386:11:29<47:57:27,  7.45s/it]
 90%|████████▉ | 201823/225000 [386:11:40<55:19:25,  8.59s/it]
 90%|████████▉ | 201824/225000 [386:11:47<52:24:02,  8.14s/it]
 90%|████████▉ | 201825/225000 [386:11:54<50:04:12,  7.78s/it]
 90%|████████▉ | 201826/225000 [386:12:00<46:17:34,  7.19s/it]
 90%|████████▉ | 201827/225000 [386:12:09<50:00:12,  7.77s/it]
 90%|████████▉ | 201828/225000 [386:12:15<46:27:18,  7.22s/it]
 90%|████████▉ | 201829/225000 [386:12:26<53:25:15,  8.30s/it]
 90%|████████▉ | 201830/225000 [386:12:36<56:41:21,  8.81s/it]
 90%|████████▉ | 201831/225000 [386:12:46<60:20:27,  9.38s/it]
 90%|████████▉ | 201832/225000 [386:12:57<62:18:36,  9.68s/it]
 90%|████████▉ | 201833/225000 [386:13:03<55:46:01,  8.67s/it]
 90%|████████▉ | 201834/225000 [386:13:09<50:52:12,  7.91s/it]
 90%|████████▉ | 201835/225000 [386:13:15<46:44:48,  7.26s/it]
 90%|████████▉ | 201836/225000 [386:13:25<52:47:42,  8.21s/it]
 90%|████████▉ | 201837/225000 [386:13:36<58:21:11,  9.07s/it]
 90%|████████▉ | 201838/225000 [386:13:46<59:43:15,  9.28s/it]
 90%|████████▉ | 201839/225000 [386:13:53<55:10:10,  8.58s/it]
 90%|████████▉ | 201840/225000 [386:14:04<59:14:07,  9.21s/it]
 90%|████████▉ | 201841/225000 [386:14:15<64:02:27,  9.95s/it]
 90%|████████▉ | 201842/225000 [386:14:21<55:16:24,  8.59s/it]
 90%|████████▉ | 201843/225000 [386:14:30<56:11:22,  8.74s/it]
 90%|████████▉ | 201844/225000 [386:14:36<50:58:58,  7.93s/it]
 90%|████████▉ | 201845/225000 [386:14:42<47:46:51,  7.43s/it]
 90%|████████▉ | 201846/225000 [386:14:56<59:51:30,  9.31s/it]
 90%|████████▉ | 201847/225000 [386:15:06<61:50:04,  9.61s/it]
 90%|████████▉ | 201848/225000 [386:15:11<53:00:28,  8.24s/it]
 90%|████████▉ | 201849/225000 [386:15:18<49:57:52,  7.77s/it]
 90%|████████▉ | 201850/225000 [386:15:24<45:57:30,  7.15s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.257', 'learning_rate': '1.428e-06', 'epoch': '0.8971', 'num_input_tokens_seen': 648563336, 'train_runtime': '1.391e+06', 'train_tokens_per_second': '466.4'}

 90%|████████▉ | 201850/225000 [386:15:24<45:57:30,  7.15s/it]
 90%|████████▉ | 201851/225000 [386:15:32<48:54:57,  7.61s/it]
 90%|████████▉ | 201852/225000 [386:15:39<47:09:52,  7.34s/it]
 90%|████████▉ | 201853/225000 [386:15:52<57:47:05,  8.99s/it]
 90%|████████▉ | 201854/225000 [386:16:02<60:42:57,  9.44s/it]
 90%|████████▉ | 201855/225000 [386:16:06<49:29:39,  7.70s/it]
 90%|████████▉ | 201856/225000 [386:16:15<52:37:53,  8.19s/it]
 90%|████████▉ | 201857/225000 [386:16:27<58:29:48,  9.10s/it]
 90%|████████▉ | 201858/225000 [386:16:38<63:40:01,  9.90s/it]
 90%|████████▉ | 201859/225000 [386:16:45<56:34:49,  8.80s/it]
 90%|████████▉ | 201860/225000 [386:16:53<56:46:10,  8.83s/it]
 90%|████████▉ | 201861/225000 [386:16:59<50:53:44,  7.92s/it]
 90%|████████▉ | 201862/225000 [386:17:09<54:23:10,  8.46s/it]
 90%|████████▉ | 201863/225000 [386:17:14<47:50:36,  7.44s/it]
 90%|████████▉ | 201864/225000 [386:17:27<58:56:15,  9.17s/it]
 90%|████████▉ | 201865/225000 [386:17:33<53:15:25,  8.29s/it]
 90%|████████▉ | 201866/225000 [386:17:45<58:50:22,  9.16s/it]
 90%|████████▉ | 201867/225000 [386:17:55<60:27:44,  9.41s/it]
 90%|████████▉ | 201868/225000 [386:18:07<65:42:16, 10.23s/it]
 90%|████████▉ | 201869/225000 [386:18:12<55:43:34,  8.67s/it]
 90%|████████▉ | 201870/225000 [386:18:18<51:42:54,  8.05s/it]
 90%|████████▉ | 201871/225000 [386:18:24<47:10:30,  7.34s/it]
 90%|████████▉ | 201872/225000 [386:18:37<57:58:38,  9.02s/it]
 90%|████████▉ | 201873/225000 [386:18:45<55:33:48,  8.65s/it]
 90%|████████▉ | 201874/225000 [386:18:52<53:26:23,  8.32s/it]
 90%|████████▉ | 201875/225000 [386:19:00<52:18:35,  8.14s/it]
 90%|████████▉ | 201876/225000 [386:19:06<48:54:21,  7.61s/it]
 90%|████████▉ | 201877/225000 [386:19:18<55:51:29,  8.70s/it]
 90%|████████▉ | 201878/225000 [386:19:28<58:41:04,  9.14s/it]
 90%|████████▉ | 201879/225000 [386:19:38<61:13:23,  9.53s/it]
 90%|████████▉ | 201880/225000 [386:19:44<53:31:17,  8.33s/it]
 90%|████████▉ | 201881/225000 [386:19:52<52:24:24,  8.16s/it]
 90%|████████▉ | 201882/225000 [386:20:01<54:14:29,  8.45s/it]
 90%|████████▉ | 201883/225000 [386:20:08<52:44:58,  8.21s/it]
 90%|████████▉ | 201884/225000 [386:20:16<52:23:17,  8.16s/it]
 90%|████████▉ | 201885/225000 [386:20:24<51:07:38,  7.96s/it]
 90%|████████▉ | 201886/225000 [386:20:32<51:16:04,  7.98s/it]
 90%|████████▉ | 201887/225000 [386:20:39<49:03:27,  7.64s/it]
 90%|████████▉ | 201888/225000 [386:20:51<56:53:23,  8.86s/it]
 90%|████████▉ | 201889/225000 [386:20:56<50:59:16,  7.94s/it]
 90%|████████▉ | 201890/225000 [386:21:09<59:46:43,  9.31s/it]
 90%|████████▉ | 201891/225000 [386:21:17<57:50:41,  9.01s/it]
 90%|████████▉ | 201892/225000 [386:21:28<61:02:46,  9.51s/it]
 90%|████████▉ | 201893/225000 [386:21:35<56:36:37,  8.82s/it]
 90%|████████▉ | 201894/225000 [386:21:40<49:30:44,  7.71s/it]
 90%|████████▉ | 201895/225000 [386:21:45<44:40:51,  6.96s/it]
 90%|████████▉ | 201896/225000 [386:21:52<43:45:48,  6.82s/it]
 90%|████████▉ | 201897/225000 [386:22:06<58:21:13,  9.09s/it]
 90%|████████▉ | 201898/225000 [386:22:16<59:01:15,  9.20s/it]
 90%|████████▉ | 201899/225000 [386:22:23<55:32:38,  8.66s/it]
 90%|████████▉ | 201900/225000 [386:22:31<54:48:16,  8.54s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.099', 'learning_rate': '1.422e-06', 'epoch': '0.8973', 'num_input_tokens_seen': 648793328, 'train_runtime': '1.391e+06', 'train_tokens_per_second': '466.4'}

 90%|████████▉ | 201900/225000 [386:22:31<54:48:16,  8.54s/it]
 90%|████████▉ | 201901/225000 [386:22:39<53:04:40,  8.27s/it]
 90%|████████▉ | 201902/225000 [386:22:46<51:27:11,  8.02s/it]
 90%|████████▉ | 201903/225000 [386:22:53<49:11:30,  7.67s/it]
 90%|████████▉ | 201904/225000 [386:23:04<55:06:35,  8.59s/it]
 90%|████████▉ | 201905/225000 [386:23:10<50:32:08,  7.88s/it]
 90%|████████▉ | 201906/225000 [386:23:16<46:31:34,  7.25s/it]
 90%|████████▉ | 201907/225000 [386:23:23<46:29:26,  7.25s/it]
 90%|████████▉ | 201908/225000 [386:23:35<55:34:46,  8.66s/it]
 90%|████████▉ | 201909/225000 [386:23:45<56:54:35,  8.87s/it]
 90%|████████▉ | 201910/225000 [386:23:50<50:32:50,  7.88s/it]
 90%|████████▉ | 201911/225000 [386:23:58<50:02:39,  7.80s/it]
 90%|████████▉ | 201912/225000 [386:24:08<54:22:22,  8.48s/it]
 90%|████████▉ | 201913/225000 [386:24:14<49:05:47,  7.66s/it]
 90%|████████▉ | 201914/225000 [386:24:25<55:44:25,  8.69s/it]
 90%|████████▉ | 201915/225000 [386:24:34<56:36:53,  8.83s/it]
 90%|████████▉ | 201916/225000 [386:24:40<52:10:48,  8.14s/it]
 90%|████████▉ | 201917/225000 [386:24:46<47:39:18,  7.43s/it]
 90%|████████▉ | 201918/225000 [386:24:53<47:29:33,  7.41s/it]
 90%|████████▉ | 201919/225000 [386:25:08<60:35:52,  9.45s/it]
 90%|████████▉ | 201920/225000 [386:25:16<58:26:08,  9.11s/it]
 90%|████████▉ | 201921/225000 [386:25:27<61:55:08,  9.66s/it]
 90%|████████▉ | 201922/225000 [386:25:34<56:27:15,  8.81s/it]
 90%|████████▉ | 201923/225000 [386:25:45<61:50:12,  9.65s/it]
 90%|████████▉ | 201924/225000 [386:25:51<54:54:13,  8.57s/it]
 90%|████████▉ | 201925/225000 [386:25:59<52:24:54,  8.18s/it]
 90%|████████▉ | 201926/225000 [386:26:09<56:06:16,  8.75s/it]
 90%|████████▉ | 201927/225000 [386:26:22<64:46:45, 10.11s/it]
 90%|████████▉ | 201928/225000 [386:26:31<62:28:15,  9.75s/it]
 90%|████████▉ | 201929/225000 [386:26:39<59:06:28,  9.22s/it]
 90%|████████▉ | 201930/225000 [386:26:50<62:06:48,  9.69s/it]
 90%|████████▉ | 201931/225000 [386:26:58<60:01:32,  9.37s/it]
 90%|████████▉ | 201932/225000 [386:27:06<57:20:44,  8.95s/it]
 90%|████████▉ | 201933/225000 [386:27:15<56:04:52,  8.75s/it]
 90%|████████▉ | 201934/225000 [386:27:22<54:04:23,  8.44s/it]
 90%|████████▉ | 201935/225000 [386:27:28<48:30:13,  7.57s/it]
 90%|████████▉ | 201936/225000 [386:27:39<55:55:35,  8.73s/it]
 90%|████████▉ | 201937/225000 [386:27:46<51:27:20,  8.03s/it]
 90%|████████▉ | 201938/225000 [386:27:54<51:38:30,  8.06s/it]
 90%|████████▉ | 201939/225000 [386:27:59<46:32:01,  7.26s/it]
 90%|████████▉ | 201940/225000 [386:28:05<43:49:25,  6.84s/it]
 90%|████████▉ | 201941/225000 [386:28:17<53:04:07,  8.29s/it]
 90%|████████▉ | 201942/225000 [386:28:27<56:16:48,  8.79s/it]
 90%|████████▉ | 201943/225000 [386:28:34<52:57:16,  8.27s/it]
 90%|████████▉ | 201944/225000 [386:28:47<61:59:56,  9.68s/it]
 90%|████████▉ | 201945/225000 [386:28:56<61:36:40,  9.62s/it]
 90%|████████▉ | 201946/225000 [386:29:02<53:31:15,  8.36s/it]
 90%|████████▉ | 201947/225000 [386:29:12<58:00:55,  9.06s/it]
 90%|████████▉ | 201948/225000 [386:29:17<49:23:17,  7.71s/it]
 90%|████████▉ | 201949/225000 [386:29:26<52:57:17,  8.27s/it]
 90%|████████▉ | 201950/225000 [386:29:34<51:57:47,  8.12s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.207', 'learning_rate': '1.416e-06', 'epoch': '0.8976', 'num_input_tokens_seen': 649023712, 'train_runtime': '1.391e+06', 'train_tokens_per_second': '466.5'}

 90%|████████▉ | 201950/225000 [386:29:34<51:57:47,  8.12s/it]
 90%|████████▉ | 201951/225000 [386:29:42<51:17:41,  8.01s/it]
 90%|████████▉ | 201952/225000 [386:29:53<57:00:40,  8.90s/it]
 90%|████████▉ | 201953/225000 [386:29:58<49:50:17,  7.78s/it]
 90%|████████▉ | 201954/225000 [386:30:08<53:44:13,  8.39s/it]
 90%|████████▉ | 201955/225000 [386:30:13<47:00:59,  7.34s/it]
 90%|████████▉ | 201956/225000 [386:30:22<50:48:49,  7.94s/it]
 90%|████████▉ | 201957/225000 [386:30:29<49:07:36,  7.68s/it]
 90%|████████▉ | 201958/225000 [386:30:41<56:08:34,  8.77s/it]
 90%|████████▉ | 201959/225000 [386:30:52<62:01:16,  9.69s/it]
 90%|████████▉ | 201960/225000 [386:30:59<56:26:04,  8.82s/it]
 90%|████████▉ | 201961/225000 [386:31:10<59:50:41,  9.35s/it]
 90%|████████▉ | 201962/225000 [386:31:17<55:19:22,  8.64s/it]
 90%|████████▉ | 201963/225000 [386:31:25<55:10:29,  8.62s/it]
 90%|████████▉ | 201964/225000 [386:31:35<56:38:50,  8.85s/it]
 90%|████████▉ | 201965/225000 [386:31:45<60:02:17,  9.38s/it]
 90%|████████▉ | 201966/225000 [386:31:57<63:22:24,  9.90s/it]
 90%|████████▉ | 201967/225000 [386:32:02<55:01:10,  8.60s/it]
 90%|████████▉ | 201968/225000 [386:32:14<60:46:31,  9.50s/it]
 90%|████████▉ | 201969/225000 [386:32:24<62:48:50,  9.82s/it]
 90%|████████▉ | 201970/225000 [386:32:32<59:28:24,  9.30s/it]
 90%|████████▉ | 201971/225000 [386:32:41<59:03:18,  9.23s/it]
 90%|████████▉ | 201972/225000 [386:32:47<51:14:10,  8.01s/it]
 90%|████████▉ | 201973/225000 [386:32:54<50:25:44,  7.88s/it]
 90%|████████▉ | 201974/225000 [386:33:00<46:37:28,  7.29s/it]
 90%|████████▉ | 201975/225000 [386:33:06<44:27:56,  6.95s/it]
 90%|████████▉ | 201976/225000 [386:33:12<43:04:16,  6.73s/it]
 90%|████████▉ | 201977/225000 [386:33:19<43:01:42,  6.73s/it]
 90%|████████▉ | 201978/225000 [386:33:25<42:14:56,  6.61s/it]
 90%|████████▉ | 201979/225000 [386:33:33<43:32:02,  6.81s/it]
 90%|████████▉ | 201980/225000 [386:33:44<51:44:14,  8.09s/it]
 90%|████████▉ | 201981/225000 [386:33:48<43:46:00,  6.84s/it]
 90%|████████▉ | 201982/225000 [386:33:59<51:58:46,  8.13s/it]
 90%|████████▉ | 201983/225000 [386:34:10<57:07:35,  8.93s/it]
 90%|████████▉ | 201984/225000 [386:34:19<56:52:28,  8.90s/it]
 90%|████████▉ | 201985/225000 [386:34:24<51:12:09,  8.01s/it]
 90%|████████▉ | 201986/225000 [386:34:35<56:32:38,  8.84s/it]
 90%|████████▉ | 201987/225000 [386:34:46<59:36:52,  9.33s/it]
 90%|████████▉ | 201988/225000 [386:34:51<51:55:35,  8.12s/it]
 90%|████████▉ | 201989/225000 [386:34:59<52:00:33,  8.14s/it]
 90%|████████▉ | 201990/225000 [386:35:09<54:25:36,  8.52s/it]
 90%|████████▉ | 201991/225000 [386:35:13<45:58:23,  7.19s/it]
 90%|████████▉ | 201992/225000 [386:35:21<48:34:23,  7.60s/it]
 90%|████████▉ | 201993/225000 [386:35:31<52:50:54,  8.27s/it]
 90%|████████▉ | 201994/225000 [386:35:37<48:14:30,  7.55s/it]
 90%|████████▉ | 201995/225000 [386:35:47<52:10:31,  8.16s/it]
 90%|████████▉ | 201996/225000 [386:35:55<53:00:33,  8.30s/it]
 90%|████████▉ | 201997/225000 [386:36:04<53:40:36,  8.40s/it]
 90%|████████▉ | 201998/225000 [386:36:13<56:06:45,  8.78s/it]
 90%|████████▉ | 201999/225000 [386:36:22<54:49:05,  8.58s/it]
 90%|████████▉ | 202000/225000 [386:36:29<53:17:47,  8.34s/it]
                                                              
{'loss': '1.346', 'grad_norm': '1.278', 'learning_rate': '1.41e-06', 'epoch': '0.8978', 'num_input_tokens_seen': 649250008, 'train_runtime': '1.392e+06', 'train_tokens_per_second': '466.5'}

 90%|████████▉ | 202000/225000 [386:36:29<53:17:47,  8.34s/it]
 90%|████████▉ | 202001/225000 [386:36:37<51:08:00,  8.00s/it]
 90%|████████▉ | 202002/225000 [386:36:42<45:24:04,  7.11s/it]
 90%|████████▉ | 202003/225000 [386:36:55<56:54:09,  8.91s/it]
 90%|████████▉ | 202004/225000 [386:37:04<58:16:55,  9.12s/it]
 90%|████████▉ | 202005/225000 [386:37:12<56:06:18,  8.78s/it]
 90%|████████▉ | 202006/225000 [386:37:21<55:43:00,  8.72s/it]
 90%|████████▉ | 202007/225000 [386:37:28<53:34:23,  8.39s/it]
 90%|████████▉ | 202008/225000 [386:37:35<50:10:41,  7.86s/it]
 90%|████████▉ | 202009/225000 [386:37:42<47:31:17,  7.44s/it]
 90%|████████▉ | 202010/225000 [386:37:58<65:22:41, 10.24s/it]
 90%|████████▉ | 202011/225000 [386:38:05<59:07:22,  9.26s/it]
 90%|████████▉ | 202012/225000 [386:38:13<55:42:41,  8.72s/it]
 90%|████████▉ | 202013/225000 [386:38:19<51:15:10,  8.03s/it]
 90%|████████▉ | 202014/225000 [386:38:27<50:05:04,  7.84s/it]
 90%|████████▉ | 202015/225000 [386:38:39<58:40:39,  9.19s/it]
 90%|████████▉ | 202016/225000 [386:38:49<60:22:09,  9.46s/it]
 90%|████████▉ | 202017/225000 [386:39:01<64:15:18, 10.06s/it]
 90%|████████▉ | 202018/225000 [386:39:07<56:49:44,  8.90s/it]
 90%|████████▉ | 202019/225000 [386:39:13<50:56:38,  7.98s/it]
 90%|████████▉ | 202020/225000 [386:39:25<59:19:39,  9.29s/it]
 90%|████████▉ | 202021/225000 [386:39:30<51:49:37,  8.12s/it]
 90%|████████▉ | 202022/225000 [386:39:42<59:26:19,  9.31s/it]
 90%|████████▉ | 202023/225000 [386:39:49<53:51:06,  8.44s/it]
 90%|████████▉ | 202024/225000 [386:39:56<51:53:52,  8.13s/it]
 90%|████████▉ | 202025/225000 [386:40:07<56:40:09,  8.88s/it]
 90%|████████▉ | 202026/225000 [386:40:14<52:35:02,  8.24s/it]
 90%|████████▉ | 202027/225000 [386:40:24<57:27:53,  9.01s/it]
 90%|████████▉ | 202028/225000 [386:40:32<55:07:38,  8.64s/it]
 90%|████████▉ | 202029/225000 [386:40:37<48:50:24,  7.65s/it]
 90%|████████▉ | 202030/225000 [386:40:49<56:53:27,  8.92s/it]
 90%|████████▉ | 202031/225000 [386:40:58<56:39:09,  8.88s/it]
 90%|████████▉ | 202032/225000 [386:41:04<51:41:36,  8.10s/it]
 90%|████████▉ | 202033/225000 [386:41:11<49:01:20,  7.68s/it]
 90%|████████▉ | 202034/225000 [386:41:16<44:03:33,  6.91s/it]
 90%|████████▉ | 202035/225000 [386:41:23<44:02:03,  6.90s/it]
 90%|████████▉ | 202036/225000 [386:41:30<44:45:22,  7.02s/it]
 90%|████████▉ | 202037/225000 [386:41:43<55:11:00,  8.65s/it]
 90%|████████▉ | 202038/225000 [386:41:50<51:45:21,  8.11s/it]
 90%|████████▉ | 202039/225000 [386:42:01<57:54:23,  9.08s/it]
 90%|████████▉ | 202040/225000 [386:42:11<60:13:38,  9.44s/it]
 90%|████████▉ | 202041/225000 [386:42:23<64:49:11, 10.16s/it]
 90%|████████▉ | 202042/225000 [386:42:28<55:27:14,  8.70s/it]
 90%|████████▉ | 202043/225000 [386:42:39<59:06:11,  9.27s/it]
 90%|████████▉ | 202044/225000 [386:42:50<62:22:16,  9.78s/it]
 90%|████████▉ | 202045/225000 [386:42:57<57:00:06,  8.94s/it]
 90%|████████▉ | 202046/225000 [386:43:03<50:50:51,  7.97s/it]
 90%|████████▉ | 202047/225000 [386:43:13<56:10:22,  8.81s/it]
 90%|████████▉ | 202048/225000 [386:43:19<50:46:20,  7.96s/it]
 90%|████████▉ | 202049/225000 [386:43:32<60:21:19,  9.47s/it]
 90%|████████▉ | 202050/225000 [386:43:42<59:42:04,  9.36s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.414', 'learning_rate': '1.404e-06', 'epoch': '0.898', 'num_input_tokens_seen': 649486976, 'train_runtime': '1.392e+06', 'train_tokens_per_second': '466.5'}

 90%|████████▉ | 202050/225000 [386:43:42<59:42:04,  9.36s/it]
 90%|████████▉ | 202051/225000 [386:43:47<53:02:11,  8.32s/it]
 90%|████████▉ | 202052/225000 [386:43:57<56:04:54,  8.80s/it]
 90%|████████▉ | 202053/225000 [386:44:05<54:28:29,  8.55s/it]
 90%|████████▉ | 202054/225000 [386:44:17<60:39:46,  9.52s/it]
 90%|████████▉ | 202055/225000 [386:44:29<65:17:20, 10.24s/it]
 90%|████████▉ | 202056/225000 [386:44:37<61:29:30,  9.65s/it]
 90%|████████▉ | 202057/225000 [386:44:45<57:06:21,  8.96s/it]
 90%|████████▉ | 202058/225000 [386:44:51<52:11:21,  8.19s/it]
 90%|████████▉ | 202059/225000 [386:44:55<44:27:30,  6.98s/it]
 90%|████████▉ | 202060/225000 [386:45:05<50:14:48,  7.89s/it]
 90%|████████▉ | 202061/225000 [386:45:10<44:03:04,  6.91s/it]
 90%|████████▉ | 202062/225000 [386:45:19<48:11:16,  7.56s/it]
 90%|████████▉ | 202063/225000 [386:45:26<47:29:48,  7.45s/it]
 90%|████████▉ | 202064/225000 [386:45:33<45:23:55,  7.13s/it]
 90%|████████▉ | 202065/225000 [386:45:44<53:33:27,  8.41s/it]
 90%|████████▉ | 202066/225000 [386:45:51<51:42:36,  8.12s/it]
 90%|████████▉ | 202067/225000 [386:45:57<46:49:04,  7.35s/it]
 90%|████████▉ | 202068/225000 [386:46:08<54:11:11,  8.51s/it]
 90%|████████▉ | 202069/225000 [386:46:15<51:34:46,  8.10s/it]
 90%|████████▉ | 202070/225000 [386:46:21<47:41:07,  7.49s/it]
 90%|████████▉ | 202071/225000 [386:46:29<48:44:39,  7.65s/it]
 90%|████████▉ | 202072/225000 [386:46:37<49:34:00,  7.78s/it]
 90%|████████▉ | 202073/225000 [386:46:44<47:42:59,  7.49s/it]
 90%|████████▉ | 202074/225000 [386:46:55<54:44:03,  8.59s/it]
 90%|████████▉ | 202075/225000 [386:47:00<46:22:13,  7.28s/it]
 90%|████████▉ | 202076/225000 [386:47:11<54:18:10,  8.53s/it]
 90%|████████▉ | 202077/225000 [386:47:24<63:05:11,  9.91s/it]
 90%|████████▉ | 202078/225000 [386:47:33<60:59:27,  9.58s/it]
 90%|████████▉ | 202079/225000 [386:47:41<58:14:55,  9.15s/it]
 90%|████████▉ | 202080/225000 [386:47:46<49:43:34,  7.81s/it]
 90%|████████▉ | 202081/225000 [386:47:51<44:18:45,  6.96s/it]
 90%|████████▉ | 202082/225000 [386:47:57<42:21:25,  6.65s/it]
 90%|████████▉ | 202083/225000 [386:48:02<39:03:11,  6.13s/it]
 90%|████████▉ | 202084/225000 [386:48:14<50:28:50,  7.93s/it]
 90%|████████▉ | 202085/225000 [386:48:20<47:54:57,  7.53s/it]
 90%|████████▉ | 202086/225000 [386:48:31<52:59:30,  8.33s/it]
 90%|████████▉ | 202087/225000 [386:48:36<47:22:56,  7.44s/it]
 90%|████████▉ | 202088/225000 [386:48:45<50:42:42,  7.97s/it]
 90%|████████▉ | 202089/225000 [386:48:50<45:19:38,  7.12s/it]
 90%|████████▉ | 202090/225000 [386:49:01<52:08:53,  8.19s/it]
 90%|████████▉ | 202091/225000 [386:49:10<52:53:58,  8.31s/it]
 90%|████████▉ | 202092/225000 [386:49:20<56:22:20,  8.86s/it]
 90%|████████▉ | 202093/225000 [386:49:31<60:28:05,  9.50s/it]
 90%|████████▉ | 202094/225000 [386:49:39<57:19:41,  9.01s/it]
 90%|████████▉ | 202095/225000 [386:49:44<51:15:03,  8.06s/it]
 90%|████████▉ | 202096/225000 [386:49:56<58:15:46,  9.16s/it]
 90%|████████▉ | 202097/225000 [386:50:00<48:29:55,  7.62s/it]
 90%|████████▉ | 202098/225000 [386:50:10<52:45:07,  8.29s/it]
 90%|████████▉ | 202099/225000 [386:50:14<44:57:09,  7.07s/it]
 90%|████████▉ | 202100/225000 [386:50:21<43:27:11,  6.83s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.599', 'learning_rate': '1.398e-06', 'epoch': '0.8982', 'num_input_tokens_seen': 649704544, 'train_runtime': '1.393e+06', 'train_tokens_per_second': '466.5'}

 90%|████████▉ | 202100/225000 [386:50:21<43:27:11,  6.83s/it]
 90%|████████▉ | 202101/225000 [386:50:28<44:46:39,  7.04s/it]
 90%|████████▉ | 202102/225000 [386:50:41<55:42:24,  8.76s/it]
 90%|████████▉ | 202103/225000 [386:50:46<48:58:41,  7.70s/it]
 90%|████████▉ | 202104/225000 [386:50:53<46:35:55,  7.33s/it]
 90%|████████▉ | 202105/225000 [386:50:59<45:33:54,  7.16s/it]
 90%|████████▉ | 202106/225000 [386:51:10<52:46:58,  8.30s/it]
 90%|████████▉ | 202107/225000 [386:51:15<45:29:12,  7.15s/it]
 90%|████████▉ | 202108/225000 [386:51:26<53:47:18,  8.46s/it]
 90%|████████▉ | 202109/225000 [386:51:32<48:55:50,  7.70s/it]
 90%|████████▉ | 202110/225000 [386:51:39<46:28:00,  7.31s/it]
 90%|████████▉ | 202111/225000 [386:51:45<45:37:21,  7.18s/it]
 90%|████████▉ | 202112/225000 [386:51:57<54:27:22,  8.57s/it]
 90%|████████▉ | 202113/225000 [386:52:06<53:54:11,  8.48s/it]
 90%|████████▉ | 202114/225000 [386:52:14<53:02:05,  8.34s/it]
 90%|████████▉ | 202115/225000 [386:52:23<55:59:31,  8.81s/it]
 90%|████████▉ | 202116/225000 [386:52:30<51:07:52,  8.04s/it]
 90%|████████▉ | 202117/225000 [386:52:39<52:57:42,  8.33s/it]
 90%|████████▉ | 202118/225000 [386:52:46<50:57:30,  8.02s/it]
 90%|████████▉ | 202119/225000 [386:52:58<59:09:38,  9.31s/it]
 90%|████████▉ | 202120/225000 [386:53:07<58:32:38,  9.21s/it]
 90%|████████▉ | 202121/225000 [386:53:16<57:52:46,  9.11s/it]
 90%|████████▉ | 202122/225000 [386:53:24<55:32:32,  8.74s/it]
 90%|████████▉ | 202123/225000 [386:53:30<49:45:27,  7.83s/it]
 90%|████████▉ | 202124/225000 [386:53:44<61:51:14,  9.73s/it]
 90%|████████▉ | 202125/225000 [386:53:53<59:49:51,  9.42s/it]
 90%|████████▉ | 202126/225000 [386:53:59<54:48:07,  8.62s/it]
 90%|████████▉ | 202127/225000 [386:54:07<52:37:10,  8.28s/it]
 90%|████████▉ | 202128/225000 [386:54:14<51:08:46,  8.05s/it]
 90%|████████▉ | 202129/225000 [386:54:23<52:37:45,  8.28s/it]
 90%|████████▉ | 202130/225000 [386:54:30<49:22:27,  7.77s/it]
 90%|████████▉ | 202131/225000 [386:54:38<51:11:48,  8.06s/it]
 90%|████████▉ | 202132/225000 [386:54:51<60:03:44,  9.46s/it]
 90%|████████▉ | 202133/225000 [386:55:00<58:31:16,  9.21s/it]
 90%|████████▉ | 202134/225000 [386:55:07<54:26:07,  8.57s/it]
 90%|████████▉ | 202135/225000 [386:55:14<51:04:43,  8.04s/it]
 90%|████████▉ | 202136/225000 [386:55:22<51:20:35,  8.08s/it]
 90%|████████▉ | 202137/225000 [386:55:29<49:44:08,  7.83s/it]
 90%|████████▉ | 202138/225000 [386:55:43<61:34:04,  9.69s/it]
 90%|████████▉ | 202139/225000 [386:55:52<59:40:02,  9.40s/it]
 90%|████████▉ | 202140/225000 [386:56:02<61:15:22,  9.65s/it]
 90%|████████▉ | 202141/225000 [386:56:08<53:44:02,  8.46s/it]
 90%|████████▉ | 202142/225000 [386:56:15<51:27:16,  8.10s/it]
 90%|████████▉ | 202143/225000 [386:56:25<54:56:58,  8.65s/it]
 90%|████████▉ | 202144/225000 [386:56:35<58:11:36,  9.17s/it]
 90%|████████▉ | 202145/225000 [386:56:47<62:38:20,  9.87s/it]
 90%|████████▉ | 202146/225000 [386:56:57<63:21:11,  9.98s/it]
 90%|████████▉ | 202147/225000 [386:57:05<59:05:37,  9.31s/it]
 90%|████████▉ | 202148/225000 [386:57:14<59:30:31,  9.37s/it]
 90%|████████▉ | 202149/225000 [386:57:27<64:52:07, 10.22s/it]
 90%|████████▉ | 202150/225000 [386:57:34<59:34:00,  9.38s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.096', 'learning_rate': '1.392e-06', 'epoch': '0.8984', 'num_input_tokens_seen': 649933096, 'train_runtime': '1.393e+06', 'train_tokens_per_second': '466.6'}

 90%|████████▉ | 202150/225000 [386:57:34<59:34:00,  9.38s/it]
 90%|████████▉ | 202151/225000 [386:57:40<52:10:05,  8.22s/it]
 90%|████████▉ | 202152/225000 [386:57:51<58:33:01,  9.23s/it]
 90%|████████▉ | 202153/225000 [386:57:58<54:44:32,  8.63s/it]
 90%|████████▉ | 202154/225000 [386:58:05<50:45:07,  8.00s/it]
 90%|████████▉ | 202155/225000 [386:58:10<44:23:01,  6.99s/it]
 90%|████████▉ | 202156/225000 [386:58:19<49:13:31,  7.76s/it]
 90%|████████▉ | 202157/225000 [386:58:27<50:10:14,  7.91s/it]
 90%|████████▉ | 202158/225000 [386:58:38<54:49:55,  8.64s/it]
 90%|████████▉ | 202159/225000 [386:58:46<54:59:51,  8.67s/it]
 90%|████████▉ | 202160/225000 [386:58:53<51:18:23,  8.09s/it]
 90%|████████▉ | 202161/225000 [386:59:04<57:11:27,  9.01s/it]
 90%|████████▉ | 202162/225000 [386:59:11<52:35:20,  8.29s/it]
 90%|████████▉ | 202163/225000 [386:59:21<55:37:35,  8.77s/it]
 90%|████████▉ | 202164/225000 [386:59:29<54:03:30,  8.52s/it]
 90%|████████▉ | 202165/225000 [386:59:36<51:14:48,  8.08s/it]
 90%|████████▉ | 202166/225000 [386:59:45<54:08:26,  8.54s/it]
 90%|████████▉ | 202167/225000 [386:59:53<51:51:17,  8.18s/it]
 90%|████████▉ | 202168/225000 [387:00:00<49:51:07,  7.86s/it]
 90%|████████▉ | 202169/225000 [387:00:07<47:50:33,  7.54s/it]
 90%|████████▉ | 202170/225000 [387:00:20<58:26:28,  9.22s/it]
 90%|████████▉ | 202171/225000 [387:00:26<53:35:10,  8.45s/it]
 90%|████████▉ | 202172/225000 [387:00:34<51:10:48,  8.07s/it]
 90%|████████▉ | 202173/225000 [387:00:39<46:28:02,  7.33s/it]
 90%|████████▉ | 202174/225000 [387:00:49<50:12:44,  7.92s/it]
 90%|████████▉ | 202175/225000 [387:01:01<59:30:57,  9.39s/it]
 90%|████████▉ | 202176/225000 [387:01:07<52:51:36,  8.34s/it]
 90%|████████▉ | 202177/225000 [387:01:17<55:58:59,  8.83s/it]
 90%|████████▉ | 202178/225000 [387:01:23<50:40:56,  7.99s/it]
 90%|████████▉ | 202179/225000 [387:01:33<53:18:52,  8.41s/it]
 90%|████████▉ | 202180/225000 [387:01:41<52:19:18,  8.25s/it]
 90%|████████▉ | 202181/225000 [387:01:46<46:42:12,  7.37s/it]
 90%|████████▉ | 202182/225000 [387:01:53<46:54:11,  7.40s/it]
 90%|████████▉ | 202183/225000 [387:02:00<45:13:34,  7.14s/it]
 90%|████████▉ | 202184/225000 [387:02:14<58:00:36,  9.15s/it]
 90%|████████▉ | 202185/225000 [387:02:19<51:12:10,  8.08s/it]
 90%|████████▉ | 202186/225000 [387:02:24<44:26:51,  7.01s/it]
 90%|████████▉ | 202187/225000 [387:02:30<42:28:07,  6.70s/it]
 90%|████████▉ | 202188/225000 [387:02:39<47:50:45,  7.55s/it]
 90%|████████▉ | 202189/225000 [387:02:52<57:28:19,  9.07s/it]
 90%|████████▉ | 202190/225000 [387:02:57<49:39:18,  7.84s/it]
 90%|████████▉ | 202191/225000 [387:03:04<49:00:53,  7.74s/it]
 90%|████████▉ | 202192/225000 [387:03:14<52:50:38,  8.34s/it]
 90%|████████▉ | 202193/225000 [387:03:21<49:14:02,  7.77s/it]
 90%|████████▉ | 202194/225000 [387:03:28<48:42:06,  7.69s/it]
 90%|████████▉ | 202195/225000 [387:03:38<52:05:22,  8.22s/it]
 90%|████████▉ | 202196/225000 [387:03:45<50:46:36,  8.02s/it]
 90%|████████▉ | 202197/225000 [387:03:52<48:13:54,  7.61s/it]
 90%|████████▉ | 202198/225000 [387:03:59<47:33:32,  7.51s/it]
 90%|████████▉ | 202199/225000 [387:04:05<44:19:38,  7.00s/it]
 90%|████████▉ | 202200/225000 [387:04:20<59:08:19,  9.34s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.284', 'learning_rate': '1.386e-06', 'epoch': '0.8987', 'num_input_tokens_seen': 650157976, 'train_runtime': '1.393e+06', 'train_tokens_per_second': '466.6'}

 90%|████████▉ | 202200/225000 [387:04:20<59:08:19,  9.34s/it]
 90%|████████▉ | 202201/225000 [387:04:27<55:22:53,  8.74s/it]
 90%|████████▉ | 202202/225000 [387:04:37<57:26:41,  9.07s/it]
 90%|████████▉ | 202203/225000 [387:04:46<57:52:02,  9.14s/it]
 90%|████████▉ | 202204/225000 [387:04:52<52:05:31,  8.23s/it]
 90%|████████▉ | 202205/225000 [387:05:02<54:30:06,  8.61s/it]
 90%|████████▉ | 202206/225000 [387:05:07<48:53:10,  7.72s/it]
 90%|████████▉ | 202207/225000 [387:05:19<56:22:44,  8.90s/it]
 90%|████████▉ | 202208/225000 [387:05:25<50:39:04,  8.00s/it]
 90%|████████▉ | 202209/225000 [387:05:35<54:49:44,  8.66s/it]
 90%|████████▉ | 202210/225000 [387:05:43<52:35:49,  8.31s/it]
 90%|████████▉ | 202211/225000 [387:05:50<51:48:45,  8.18s/it]
 90%|████████▉ | 202212/225000 [387:05:58<49:53:10,  7.88s/it]
 90%|████████▉ | 202213/225000 [387:06:07<52:42:28,  8.33s/it]
 90%|████████▉ | 202214/225000 [387:06:17<56:04:26,  8.86s/it]
 90%|████████▉ | 202215/225000 [387:06:27<58:28:30,  9.24s/it]
 90%|████████▉ | 202216/225000 [387:06:37<60:05:51,  9.50s/it]
 90%|████████▉ | 202217/225000 [387:06:42<51:38:56,  8.16s/it]
 90%|████████▉ | 202218/225000 [387:06:55<59:19:12,  9.37s/it]
 90%|████████▉ | 202219/225000 [387:07:02<55:59:07,  8.85s/it]
 90%|████████▉ | 202220/225000 [387:07:10<54:20:13,  8.59s/it]
 90%|████████▉ | 202221/225000 [387:07:18<53:29:56,  8.45s/it]
 90%|████████▉ | 202222/225000 [387:07:26<52:31:32,  8.30s/it]
 90%|████████▉ | 202223/225000 [387:07:39<60:33:41,  9.57s/it]
 90%|████████▉ | 202224/225000 [387:07:49<62:07:21,  9.82s/it]
 90%|████████▉ | 202225/225000 [387:07:58<60:39:29,  9.59s/it]
 90%|████████▉ | 202226/225000 [387:08:07<59:01:06,  9.33s/it]
 90%|████████▉ | 202227/225000 [387:08:12<51:04:58,  8.08s/it]
 90%|████████▉ | 202228/225000 [387:08:19<48:03:42,  7.60s/it]
 90%|████████▉ | 202229/225000 [387:08:31<57:04:37,  9.02s/it]
 90%|████████▉ | 202230/225000 [387:08:40<56:36:05,  8.95s/it]
 90%|████████▉ | 202231/225000 [387:08:51<60:56:23,  9.64s/it]
 90%|████████▉ | 202232/225000 [387:08:58<56:50:41,  8.99s/it]
 90%|████████▉ | 202233/225000 [387:09:04<50:47:12,  8.03s/it]
 90%|████████▉ | 202234/225000 [387:09:14<53:13:19,  8.42s/it]
 90%|████████▉ | 202235/225000 [387:09:19<47:11:23,  7.46s/it]
 90%|████████▉ | 202236/225000 [387:09:25<45:31:30,  7.20s/it]
 90%|████████▉ | 202237/225000 [387:09:34<48:52:59,  7.73s/it]
 90%|████████▉ | 202238/225000 [387:09:48<60:43:49,  9.61s/it]
 90%|████████▉ | 202239/225000 [387:09:56<56:08:49,  8.88s/it]
 90%|████████▉ | 202240/225000 [387:10:03<52:37:14,  8.32s/it]
 90%|████████▉ | 202241/225000 [387:10:10<51:26:13,  8.14s/it]
 90%|████████▉ | 202242/225000 [387:10:22<59:06:25,  9.35s/it]
 90%|████████▉ | 202243/225000 [387:10:30<55:45:04,  8.82s/it]
 90%|████████▉ | 202244/225000 [387:10:37<51:24:45,  8.13s/it]
 90%|████████▉ | 202245/225000 [387:10:43<48:48:59,  7.72s/it]
 90%|████████▉ | 202246/225000 [387:10:54<53:40:50,  8.49s/it]
 90%|████████▉ | 202247/225000 [387:11:01<51:28:39,  8.14s/it]
 90%|████████▉ | 202248/225000 [387:11:15<63:22:19, 10.03s/it]
 90%|████████▉ | 202249/225000 [387:11:24<60:41:57,  9.60s/it]
 90%|████████▉ | 202250/225000 [387:11:29<52:31:12,  8.31s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.152', 'learning_rate': '1.38e-06', 'epoch': '0.8989', 'num_input_tokens_seen': 650390040, 'train_runtime': '1.394e+06', 'train_tokens_per_second': '466.6'}

 90%|████████▉ | 202250/225000 [387:11:29<52:31:12,  8.31s/it]
 90%|████████▉ | 202251/225000 [387:11:38<53:01:45,  8.39s/it]
 90%|████████▉ | 202252/225000 [387:11:45<50:51:21,  8.05s/it]
 90%|████████▉ | 202253/225000 [387:12:03<68:58:56, 10.92s/it]
 90%|████████▉ | 202254/225000 [387:12:09<60:44:16,  9.61s/it]
 90%|████████▉ | 202255/225000 [387:12:15<52:50:59,  8.36s/it]
 90%|████████▉ | 202256/225000 [387:12:23<52:59:33,  8.39s/it]
 90%|████████▉ | 202257/225000 [387:12:31<51:46:24,  8.20s/it]
 90%|████████▉ | 202258/225000 [387:12:41<56:18:39,  8.91s/it]
 90%|████████▉ | 202259/225000 [387:12:52<58:31:11,  9.26s/it]
 90%|████████▉ | 202260/225000 [387:13:01<58:50:59,  9.32s/it]
 90%|████████▉ | 202261/225000 [387:13:14<65:26:07, 10.36s/it]
 90%|████████▉ | 202262/225000 [387:13:18<54:37:13,  8.65s/it]
 90%|████████▉ | 202263/225000 [387:13:27<54:11:40,  8.58s/it]
 90%|████████▉ | 202264/225000 [387:13:34<52:17:21,  8.28s/it]
 90%|████████▉ | 202265/225000 [387:13:41<48:51:53,  7.74s/it]
 90%|████████▉ | 202266/225000 [387:13:48<46:39:16,  7.39s/it]
 90%|████████▉ | 202267/225000 [387:13:55<47:42:38,  7.56s/it]
 90%|████████▉ | 202268/225000 [387:14:03<48:13:16,  7.64s/it]
 90%|████████▉ | 202269/225000 [387:14:10<46:17:09,  7.33s/it]
 90%|████████▉ | 202270/225000 [387:14:23<57:32:37,  9.11s/it]
 90%|████████▉ | 202271/225000 [387:14:30<54:09:59,  8.58s/it]
 90%|████████▉ | 202272/225000 [387:14:37<49:26:52,  7.83s/it]
 90%|████████▉ | 202273/225000 [387:14:49<57:49:28,  9.16s/it]
 90%|████████▉ | 202274/225000 [387:14:56<54:46:33,  8.68s/it]
 90%|████████▉ | 202275/225000 [387:15:04<53:38:22,  8.50s/it]
 90%|████████▉ | 202276/225000 [387:15:12<52:12:44,  8.27s/it]
 90%|████████▉ | 202277/225000 [387:15:16<44:32:07,  7.06s/it]
 90%|████████▉ | 202278/225000 [387:15:26<49:35:04,  7.86s/it]
 90%|████████▉ | 202279/225000 [387:15:34<49:04:14,  7.77s/it]
 90%|████████▉ | 202280/225000 [387:15:42<50:37:27,  8.02s/it]
 90%|████████▉ | 202281/225000 [387:15:51<52:30:34,  8.32s/it]
 90%|████████▉ | 202282/225000 [387:16:03<59:20:48,  9.40s/it]
 90%|████████▉ | 202283/225000 [387:16:13<60:27:37,  9.58s/it]
 90%|████████▉ | 202284/225000 [387:16:20<54:31:33,  8.64s/it]
 90%|████████▉ | 202285/225000 [387:16:27<51:11:29,  8.11s/it]
 90%|████████▉ | 202286/225000 [387:16:39<60:08:00,  9.53s/it]
 90%|████████▉ | 202287/225000 [387:16:45<53:17:11,  8.45s/it]
 90%|████████▉ | 202288/225000 [387:16:51<47:42:17,  7.56s/it]
 90%|████████▉ | 202289/225000 [387:16:59<48:36:19,  7.70s/it]
 90%|████████▉ | 202290/225000 [387:17:13<59:55:24,  9.50s/it]
 90%|████████▉ | 202291/225000 [387:17:19<54:02:32,  8.57s/it]
 90%|████████▉ | 202292/225000 [387:17:25<49:32:55,  7.86s/it]
 90%|████████▉ | 202293/225000 [387:17:32<46:52:06,  7.43s/it]
 90%|████████▉ | 202294/225000 [387:17:43<54:00:26,  8.56s/it]
 90%|████████▉ | 202295/225000 [387:17:49<49:56:06,  7.92s/it]
 90%|████████▉ | 202296/225000 [387:17:58<51:29:24,  8.16s/it]
 90%|████████▉ | 202297/225000 [387:18:08<54:31:11,  8.65s/it]
 90%|████████▉ | 202298/225000 [387:18:15<50:58:23,  8.08s/it]
 90%|████████▉ | 202299/225000 [387:18:23<51:21:44,  8.15s/it]
 90%|████████▉ | 202300/225000 [387:18:34<56:35:58,  8.98s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.136', 'learning_rate': '1.374e-06', 'epoch': '0.8991', 'num_input_tokens_seen': 650621016, 'train_runtime': '1.394e+06', 'train_tokens_per_second': '466.6'}

 90%|████████▉ | 202300/225000 [387:18:34<56:35:58,  8.98s/it]
 90%|████████▉ | 202301/225000 [387:18:42<56:02:41,  8.89s/it]
 90%|████████▉ | 202302/225000 [387:18:52<56:50:55,  9.02s/it]
 90%|████████▉ | 202303/225000 [387:19:02<58:59:44,  9.36s/it]
 90%|████████▉ | 202304/225000 [387:19:12<60:17:22,  9.56s/it]
 90%|████████▉ | 202305/225000 [387:19:21<60:01:45,  9.52s/it]
 90%|████████▉ | 202306/225000 [387:19:32<61:40:26,  9.78s/it]
 90%|████████▉ | 202307/225000 [387:19:39<57:41:01,  9.15s/it]
 90%|████████▉ | 202308/225000 [387:19:53<66:13:16, 10.51s/it]
 90%|████████▉ | 202309/225000 [387:19:59<56:52:23,  9.02s/it]
 90%|████████▉ | 202310/225000 [387:20:08<57:14:47,  9.08s/it]
 90%|████████▉ | 202311/225000 [387:20:14<51:15:42,  8.13s/it]
 90%|████████▉ | 202312/225000 [387:20:20<47:37:05,  7.56s/it]
 90%|████████▉ | 202313/225000 [387:20:26<45:29:41,  7.22s/it]
 90%|████████▉ | 202314/225000 [387:20:36<50:50:27,  8.07s/it]
 90%|████████▉ | 202315/225000 [387:20:48<56:34:36,  8.98s/it]
 90%|████████▉ | 202316/225000 [387:20:55<54:26:23,  8.64s/it]
 90%|████████▉ | 202317/225000 [387:21:04<53:50:10,  8.54s/it]
 90%|████████▉ | 202318/225000 [387:21:10<50:20:45,  7.99s/it]
 90%|████████▉ | 202319/225000 [387:21:18<50:08:11,  7.96s/it]
 90%|████████▉ | 202320/225000 [387:21:28<54:10:41,  8.60s/it]
 90%|████████▉ | 202321/225000 [387:21:42<62:58:58, 10.00s/it]
 90%|████████▉ | 202322/225000 [387:21:50<60:22:36,  9.58s/it]
 90%|████████▉ | 202323/225000 [387:21:57<55:06:02,  8.75s/it]
 90%|████████▉ | 202324/225000 [387:22:03<50:23:19,  8.00s/it]
 90%|████████▉ | 202325/225000 [387:22:14<54:38:47,  8.68s/it]
 90%|████████▉ | 202326/225000 [387:22:22<53:44:40,  8.53s/it]
 90%|████████▉ | 202327/225000 [387:22:31<54:48:49,  8.70s/it]
 90%|████████▉ | 202328/225000 [387:22:41<57:15:01,  9.09s/it]
 90%|████████▉ | 202329/225000 [387:22:49<55:19:53,  8.79s/it]
 90%|████████▉ | 202330/225000 [387:22:57<54:26:53,  8.65s/it]
 90%|████████▉ | 202331/225000 [387:23:08<58:43:57,  9.33s/it]
 90%|████████▉ | 202332/225000 [387:23:18<58:54:00,  9.35s/it]
 90%|████████▉ | 202333/225000 [387:23:27<58:16:16,  9.25s/it]
 90%|████████▉ | 202334/225000 [387:23:31<49:44:49,  7.90s/it]
 90%|████████▉ | 202335/225000 [387:23:38<47:43:57,  7.58s/it]
 90%|████████▉ | 202336/225000 [387:23:46<47:33:45,  7.55s/it]
 90%|████████▉ | 202337/225000 [387:23:58<56:09:14,  8.92s/it]
 90%|████████▉ | 202338/225000 [387:24:06<54:05:01,  8.59s/it]
 90%|████████▉ | 202339/225000 [387:24:13<52:11:01,  8.29s/it]
 90%|████████▉ | 202340/225000 [387:24:21<51:21:17,  8.16s/it]
 90%|████████▉ | 202341/225000 [387:24:36<64:09:05, 10.19s/it]
 90%|████████▉ | 202342/225000 [387:24:45<61:20:21,  9.75s/it]
 90%|████████▉ | 202343/225000 [387:24:51<55:16:36,  8.78s/it]
 90%|████████▉ | 202344/225000 [387:24:57<49:39:24,  7.89s/it]
 90%|████████▉ | 202345/225000 [387:25:03<46:05:48,  7.33s/it]
 90%|████████▉ | 202346/225000 [387:25:19<62:32:14,  9.94s/it]
 90%|████████▉ | 202347/225000 [387:25:25<55:32:19,  8.83s/it]
 90%|████████▉ | 202348/225000 [387:25:36<59:46:38,  9.50s/it]
 90%|████████▉ | 202349/225000 [387:25:41<51:00:46,  8.11s/it]
 90%|████████▉ | 202350/225000 [387:25:53<58:11:30,  9.25s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.097', 'learning_rate': '1.368e-06', 'epoch': '0.8993', 'num_input_tokens_seen': 650852784, 'train_runtime': '1.395e+06', 'train_tokens_per_second': '466.6'}

 90%|████████▉ | 202350/225000 [387:25:53<58:11:30,  9.25s/it]
 90%|████████▉ | 202351/225000 [387:26:02<57:57:40,  9.21s/it]
 90%|████████▉ | 202352/225000 [387:26:11<57:34:33,  9.15s/it]
 90%|████████▉ | 202353/225000 [387:26:22<60:05:48,  9.55s/it]
 90%|████████▉ | 202354/225000 [387:26:29<55:48:40,  8.87s/it]
 90%|████████▉ | 202355/225000 [387:26:37<54:03:46,  8.59s/it]
 90%|████████▉ | 202356/225000 [387:26:43<48:55:30,  7.78s/it]
 90%|████████▉ | 202357/225000 [387:26:54<54:55:59,  8.73s/it]
 90%|████████▉ | 202358/225000 [387:27:04<58:21:03,  9.28s/it]
 90%|████████▉ | 202359/225000 [387:27:14<58:25:02,  9.29s/it]
 90%|████████▉ | 202360/225000 [387:27:20<52:30:33,  8.35s/it]
 90%|████████▉ | 202361/225000 [387:27:32<59:34:31,  9.47s/it]
 90%|████████▉ | 202362/225000 [387:27:40<55:57:30,  8.90s/it]
 90%|████████▉ | 202363/225000 [387:27:49<57:29:55,  9.14s/it]
 90%|████████▉ | 202364/225000 [387:27:56<53:42:30,  8.54s/it]
 90%|████████▉ | 202365/225000 [387:28:04<52:20:58,  8.33s/it]
 90%|████████▉ | 202366/225000 [387:28:13<52:47:15,  8.40s/it]
 90%|████████▉ | 202367/225000 [387:28:23<56:39:01,  9.01s/it]
 90%|████████▉ | 202368/225000 [387:28:31<53:44:27,  8.55s/it]
 90%|████████▉ | 202369/225000 [387:28:38<50:25:38,  8.02s/it]
 90%|████████▉ | 202370/225000 [387:28:45<50:02:46,  7.96s/it]
 90%|████████▉ | 202371/225000 [387:28:55<53:01:23,  8.44s/it]
 90%|████████▉ | 202372/225000 [387:29:04<53:54:45,  8.58s/it]
 90%|████████▉ | 202373/225000 [387:29:15<58:49:59,  9.36s/it]
 90%|████████▉ | 202374/225000 [387:29:23<55:43:02,  8.87s/it]
 90%|████████▉ | 202375/225000 [387:29:30<52:07:06,  8.29s/it]
 90%|████████▉ | 202376/225000 [387:29:42<60:10:50,  9.58s/it]
 90%|████████▉ | 202377/225000 [387:29:52<60:08:02,  9.57s/it]
 90%|████████▉ | 202378/225000 [387:29:57<52:37:24,  8.37s/it]
 90%|████████▉ | 202379/225000 [387:30:05<52:03:32,  8.28s/it]
 90%|████████▉ | 202380/225000 [387:30:22<66:53:05, 10.64s/it]
 90%|████████▉ | 202381/225000 [387:30:27<57:34:41,  9.16s/it]
 90%|████████▉ | 202382/225000 [387:30:38<59:35:12,  9.48s/it]
 90%|████████▉ | 202383/225000 [387:30:45<55:30:36,  8.84s/it]
 90%|████████▉ | 202384/225000 [387:30:51<50:02:38,  7.97s/it]
 90%|████████▉ | 202385/225000 [387:30:56<45:20:56,  7.22s/it]
 90%|████████▉ | 202386/225000 [387:31:03<45:23:52,  7.23s/it]
 90%|████████▉ | 202387/225000 [387:31:17<56:18:19,  8.96s/it]
 90%|████████▉ | 202388/225000 [387:31:25<55:30:30,  8.84s/it]
 90%|████████▉ | 202389/225000 [387:31:34<54:57:05,  8.75s/it]
 90%|████████▉ | 202390/225000 [387:31:40<49:51:27,  7.94s/it]
 90%|████████▉ | 202391/225000 [387:31:46<46:48:39,  7.45s/it]
 90%|████████▉ | 202392/225000 [387:31:53<46:29:32,  7.40s/it]
 90%|████████▉ | 202393/225000 [387:32:02<48:27:50,  7.72s/it]
 90%|████████▉ | 202394/225000 [387:32:15<58:54:15,  9.38s/it]
 90%|████████▉ | 202395/225000 [387:32:24<58:21:57,  9.30s/it]
 90%|████████▉ | 202396/225000 [387:32:30<51:42:07,  8.23s/it]
 90%|████████▉ | 202397/225000 [387:32:43<60:54:07,  9.70s/it]
 90%|████████▉ | 202398/225000 [387:32:51<58:32:58,  9.33s/it]
 90%|████████▉ | 202399/225000 [387:32:58<53:16:18,  8.49s/it]
 90%|████████▉ | 202400/225000 [387:33:09<57:42:23,  9.19s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.285', 'learning_rate': '1.362e-06', 'epoch': '0.8996', 'num_input_tokens_seen': 651091192, 'train_runtime': '1.395e+06', 'train_tokens_per_second': '466.7'}

 90%|████████▉ | 202400/225000 [387:33:09<57:42:23,  9.19s/it]
 90%|████████▉ | 202401/225000 [387:33:18<57:42:36,  9.19s/it]
 90%|████████▉ | 202402/225000 [387:33:25<54:09:24,  8.63s/it]
 90%|████████▉ | 202403/225000 [387:33:34<54:51:28,  8.74s/it]
 90%|████████▉ | 202404/225000 [387:33:46<60:50:49,  9.69s/it]
 90%|████████▉ | 202405/225000 [387:33:53<54:37:47,  8.70s/it]
 90%|████████▉ | 202406/225000 [387:33:59<50:02:51,  7.97s/it]
 90%|████████▉ | 202407/225000 [387:34:07<50:49:25,  8.10s/it]
 90%|████████▉ | 202408/225000 [387:34:21<61:08:27,  9.74s/it]
 90%|████████▉ | 202409/225000 [387:34:25<50:56:20,  8.12s/it]
 90%|████████▉ | 202410/225000 [387:34:34<52:16:38,  8.33s/it]
 90%|████████▉ | 202411/225000 [387:34:46<58:39:05,  9.35s/it]
 90%|████████▉ | 202412/225000 [387:34:55<58:39:10,  9.35s/it]
 90%|████████▉ | 202413/225000 [387:35:07<63:35:18, 10.13s/it]
 90%|████████▉ | 202414/225000 [387:35:12<54:44:49,  8.73s/it]
 90%|████████▉ | 202415/225000 [387:35:17<47:11:22,  7.52s/it]
 90%|████████▉ | 202416/225000 [387:35:23<44:40:25,  7.12s/it]
 90%|████████▉ | 202417/225000 [387:35:37<57:50:33,  9.22s/it]
 90%|████████▉ | 202418/225000 [387:35:44<52:14:03,  8.33s/it]
 90%|████████▉ | 202419/225000 [387:35:51<49:45:30,  7.93s/it]
 90%|████████▉ | 202420/225000 [387:36:04<59:17:22,  9.45s/it]
 90%|████████▉ | 202421/225000 [387:36:09<50:35:37,  8.07s/it]
 90%|████████▉ | 202422/225000 [387:36:16<50:00:28,  7.97s/it]
 90%|████████▉ | 202423/225000 [387:36:25<51:22:12,  8.19s/it]
 90%|████████▉ | 202424/225000 [387:36:32<48:32:54,  7.74s/it]
 90%|████████▉ | 202425/225000 [387:36:48<64:11:49, 10.24s/it]
 90%|████████▉ | 202426/225000 [387:36:55<57:51:26,  9.23s/it]
 90%|████████▉ | 202427/225000 [387:37:02<55:12:55,  8.81s/it]
 90%|████████▉ | 202428/225000 [387:37:12<57:31:56,  9.18s/it]
 90%|████████▉ | 202429/225000 [387:37:19<52:36:42,  8.39s/it]
 90%|████████▉ | 202430/225000 [387:37:31<59:10:38,  9.44s/it]
 90%|████████▉ | 202431/225000 [387:37:42<62:40:23, 10.00s/it]
 90%|████████▉ | 202432/225000 [387:37:52<62:12:33,  9.92s/it]
 90%|████████▉ | 202433/225000 [387:37:58<53:57:52,  8.61s/it]
 90%|████████▉ | 202434/225000 [387:38:04<50:45:56,  8.10s/it]
 90%|████████▉ | 202435/225000 [387:38:13<52:26:15,  8.37s/it]
 90%|████████▉ | 202436/225000 [387:38:26<59:58:33,  9.57s/it]
 90%|████████▉ | 202437/225000 [387:38:37<62:24:39,  9.96s/it]
 90%|████████▉ | 202438/225000 [387:38:43<56:32:03,  9.02s/it]
 90%|████████▉ | 202439/225000 [387:38:51<53:04:04,  8.47s/it]
 90%|████████▉ | 202440/225000 [387:38:56<47:56:21,  7.65s/it]
 90%|████████▉ | 202441/225000 [387:39:07<52:52:44,  8.44s/it]
 90%|████████▉ | 202442/225000 [387:39:13<49:46:18,  7.94s/it]
 90%|████████▉ | 202443/225000 [387:39:19<44:32:43,  7.11s/it]
 90%|████████▉ | 202444/225000 [387:39:25<42:34:25,  6.79s/it]
 90%|████████▉ | 202445/225000 [387:39:37<53:10:53,  8.49s/it]
 90%|████████▉ | 202446/225000 [387:39:47<55:32:32,  8.87s/it]
 90%|████████▉ | 202447/225000 [387:39:52<48:34:24,  7.75s/it]
 90%|████████▉ | 202448/225000 [387:40:03<54:07:17,  8.64s/it]
 90%|████████▉ | 202449/225000 [387:40:10<50:37:53,  8.08s/it]
 90%|████████▉ | 202450/225000 [387:40:19<53:22:16,  8.52s/it]
                                                              
{'loss': '1.333', 'grad_norm': '1.057', 'learning_rate': '1.356e-06', 'epoch': '0.8998', 'num_input_tokens_seen': 651324832, 'train_runtime': '1.396e+06', 'train_tokens_per_second': '466.7'}

 90%|████████▉ | 202450/225000 [387:40:19<53:22:16,  8.52s/it]
 90%|████████▉ | 202451/225000 [387:40:30<57:01:27,  9.10s/it]
 90%|████████▉ | 202452/225000 [387:40:38<55:57:03,  8.93s/it]
 90%|████████▉ | 202453/225000 [387:40:46<54:26:24,  8.69s/it]
 90%|████████▉ | 202454/225000 [387:40:56<55:34:09,  8.87s/it]
 90%|████████▉ | 202455/225000 [387:41:04<53:56:54,  8.61s/it]
 90%|████████▉ | 202456/225000 [387:41:11<51:25:48,  8.21s/it]
 90%|████████▉ | 202457/225000 [387:41:17<47:39:24,  7.61s/it]
 90%|████████▉ | 202458/225000 [387:41:29<56:01:46,  8.95s/it]
 90%|████████▉ | 202459/225000 [387:41:38<56:24:36,  9.01s/it]
 90%|████████▉ | 202460/225000 [387:41:47<55:05:32,  8.80s/it]
 90%|████████▉ | 202461/225000 [387:41:53<50:34:03,  8.08s/it]
 90%|████████▉ | 202462/225000 [387:42:05<58:18:01,  9.31s/it]
 90%|████████▉ | 202463/225000 [387:42:13<56:03:39,  8.96s/it]
 90%|████████▉ | 202464/225000 [387:42:25<61:16:13,  9.79s/it]
 90%|████████▉ | 202465/225000 [387:42:30<52:27:25,  8.38s/it]
 90%|████████▉ | 202466/225000 [387:42:41<56:25:22,  9.01s/it]
 90%|████████▉ | 202467/225000 [387:42:50<56:38:58,  9.05s/it]
 90%|████████▉ | 202468/225000 [387:42:55<50:14:23,  8.03s/it]
 90%|████████▉ | 202469/225000 [387:43:02<47:33:44,  7.60s/it]
 90%|████████▉ | 202470/225000 [387:43:15<58:43:47,  9.38s/it]
 90%|████████▉ | 202471/225000 [387:43:23<55:41:16,  8.90s/it]
 90%|████████▉ | 202472/225000 [387:43:29<50:41:14,  8.10s/it]
 90%|████████▉ | 202473/225000 [387:43:37<50:02:00,  8.00s/it]
 90%|████████▉ | 202474/225000 [387:43:45<48:43:11,  7.79s/it]
 90%|████████▉ | 202475/225000 [387:43:52<47:19:02,  7.56s/it]
 90%|████████▉ | 202476/225000 [387:44:01<51:11:17,  8.18s/it]
 90%|████████▉ | 202477/225000 [387:44:07<46:16:39,  7.40s/it]
 90%|████████▉ | 202478/225000 [387:44:19<55:01:48,  8.80s/it]
 90%|████████▉ | 202479/225000 [387:44:31<61:37:32,  9.85s/it]
 90%|████████▉ | 202480/225000 [387:44:38<55:19:04,  8.84s/it]
 90%|████████▉ | 202481/225000 [387:44:48<58:17:53,  9.32s/it]
 90%|████████▉ | 202482/225000 [387:44:55<53:56:19,  8.62s/it]
 90%|████████▉ | 202483/225000 [387:45:02<50:57:46,  8.15s/it]
 90%|████████▉ | 202484/225000 [387:45:08<45:57:57,  7.35s/it]
 90%|████████▉ | 202485/225000 [387:45:20<55:47:15,  8.92s/it]
 90%|████████▉ | 202486/225000 [387:45:28<53:49:17,  8.61s/it]
 90%|████████▉ | 202487/225000 [387:45:34<49:22:54,  7.90s/it]
 90%|████████▉ | 202488/225000 [387:45:43<51:48:39,  8.29s/it]
 90%|████████▉ | 202489/225000 [387:45:53<54:26:38,  8.71s/it]
 90%|████████▉ | 202490/225000 [387:46:02<53:47:50,  8.60s/it]
 90%|████████▉ | 202491/225000 [387:46:10<53:02:27,  8.48s/it]
 90%|████████▉ | 202492/225000 [387:46:16<48:38:34,  7.78s/it]
 90%|████████▉ | 202493/225000 [387:46:24<49:24:17,  7.90s/it]
 90%|████████▉ | 202494/225000 [387:46:31<48:15:03,  7.72s/it]
 90%|████████▉ | 202495/225000 [387:46:37<44:48:47,  7.17s/it]
 90%|████████▉ | 202496/225000 [387:46:53<61:29:24,  9.84s/it]
 90%|████████▉ | 202497/225000 [387:47:03<60:19:30,  9.65s/it]
 90%|████████▉ | 202498/225000 [387:47:14<63:29:42, 10.16s/it]
 90%|████████▉ | 202499/225000 [387:47:23<62:10:04,  9.95s/it]
 90%|█████████ | 202500/225000 [387:47:31<58:20:22,  9.33s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.318', 'learning_rate': '1.35e-06', 'epoch': '0.9', 'num_input_tokens_seen': 651555160, 'train_runtime': '1.396e+06', 'train_tokens_per_second': '466.7'}

 90%|█████████ | 202500/225000 [387:47:31<58:20:22,  9.33s/it]
 90%|█████████ | 202501/225000 [387:47:39<55:01:07,  8.80s/it]
 90%|█████████ | 202502/225000 [387:47:46<51:39:17,  8.27s/it]
 90%|█████████ | 202503/225000 [387:47:53<50:35:49,  8.10s/it]
 90%|█████████ | 202504/225000 [387:48:01<50:09:18,  8.03s/it]
 90%|█████████ | 202505/225000 [387:48:07<45:43:41,  7.32s/it]
 90%|█████████ | 202506/225000 [387:48:13<43:54:08,  7.03s/it]
 90%|█████████ | 202507/225000 [387:48:25<52:05:52,  8.34s/it]
 90%|█████████ | 202508/225000 [387:48:33<51:58:30,  8.32s/it]
 90%|█████████ | 202509/225000 [387:48:37<44:38:29,  7.15s/it]
 90%|█████████ | 202510/225000 [387:48:47<48:48:45,  7.81s/it]
 90%|█████████ | 202511/225000 [387:48:58<55:41:12,  8.91s/it]
 90%|█████████ | 202512/225000 [387:49:08<56:15:04,  9.01s/it]
 90%|█████████ | 202513/225000 [387:49:15<53:51:42,  8.62s/it]
 90%|█████████ | 202514/225000 [387:49:21<48:07:53,  7.71s/it]
 90%|█████████ | 202515/225000 [387:49:27<46:06:59,  7.38s/it]
 90%|█████████ | 202516/225000 [387:49:39<53:33:28,  8.58s/it]
 90%|█████████ | 202517/225000 [387:49:46<50:45:39,  8.13s/it]
 90%|█████████ | 202518/225000 [387:49:54<51:09:08,  8.19s/it]
 90%|█████████ | 202519/225000 [387:50:04<54:36:52,  8.75s/it]
 90%|█████████ | 202520/225000 [387:50:12<51:56:01,  8.32s/it]
 90%|█████████ | 202521/225000 [387:50:19<50:28:04,  8.08s/it]
 90%|█████████ | 202522/225000 [387:50:25<45:42:57,  7.32s/it]
 90%|█████████ | 202523/225000 [387:50:36<53:07:27,  8.51s/it]
 90%|█████████ | 202524/225000 [387:50:41<46:08:47,  7.39s/it]
 90%|█████████ | 202525/225000 [387:50:52<53:07:34,  8.51s/it]
 90%|█████████ | 202526/225000 [387:50:58<48:03:14,  7.70s/it]
 90%|█████████ | 202527/225000 [387:51:10<56:03:39,  8.98s/it]
 90%|█████████ | 202528/225000 [387:51:15<50:14:04,  8.05s/it]
 90%|█████████ | 202529/225000 [387:51:27<57:26:00,  9.20s/it]
 90%|█████████ | 202530/225000 [387:51:36<56:15:05,  9.01s/it]
 90%|█████████ | 202531/225000 [387:51:44<54:05:57,  8.67s/it]
 90%|█████████ | 202532/225000 [387:51:52<52:50:18,  8.47s/it]
 90%|█████████ | 202533/225000 [387:51:58<48:40:16,  7.80s/it]
 90%|█████████ | 202534/225000 [387:52:05<46:35:41,  7.47s/it]
 90%|█████████ | 202535/225000 [387:52:16<54:03:38,  8.66s/it]
 90%|█████████ | 202536/225000 [387:52:25<53:41:43,  8.61s/it]
 90%|█████████ | 202537/225000 [387:52:33<53:28:50,  8.57s/it]
 90%|█████████ | 202538/225000 [387:52:40<50:22:35,  8.07s/it]
 90%|█████████ | 202539/225000 [387:52:46<46:44:41,  7.49s/it]
 90%|█████████ | 202540/225000 [387:52:53<46:00:09,  7.37s/it]
 90%|█████████ | 202541/225000 [387:53:03<49:48:02,  7.98s/it]
 90%|█████████ | 202542/225000 [387:53:10<48:55:21,  7.84s/it]
 90%|█████████ | 202543/225000 [387:53:18<48:42:23,  7.81s/it]
 90%|█████████ | 202544/225000 [387:53:25<46:35:29,  7.47s/it]
 90%|█████████ | 202545/225000 [387:53:31<44:34:45,  7.15s/it]
 90%|█████████ | 202546/225000 [387:53:46<58:57:52,  9.45s/it]
 90%|█████████ | 202547/225000 [387:53:57<61:37:47,  9.88s/it]
 90%|█████████ | 202548/225000 [387:54:03<55:00:54,  8.82s/it]
 90%|█████████ | 202549/225000 [387:54:14<59:05:07,  9.47s/it]
 90%|█████████ | 202550/225000 [387:54:20<53:09:03,  8.52s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.467', 'learning_rate': '1.344e-06', 'epoch': '0.9002', 'num_input_tokens_seen': 651779256, 'train_runtime': '1.396e+06', 'train_tokens_per_second': '466.7'}

 90%|█████████ | 202550/225000 [387:54:20<53:09:03,  8.52s/it]
 90%|█████████ | 202551/225000 [387:54:35<64:44:28, 10.38s/it]
 90%|█████████ | 202552/225000 [387:54:42<58:48:23,  9.43s/it]
 90%|█████████ | 202553/225000 [387:54:52<58:36:56,  9.40s/it]
 90%|█████████ | 202554/225000 [387:55:00<56:49:00,  9.11s/it]
 90%|█████████ | 202555/225000 [387:55:07<53:21:04,  8.56s/it]
 90%|█████████ | 202556/225000 [387:55:14<49:56:23,  8.01s/it]
 90%|█████████ | 202557/225000 [387:55:30<65:06:31, 10.44s/it]
 90%|█████████ | 202558/225000 [387:55:36<56:12:31,  9.02s/it]
 90%|█████████ | 202559/225000 [387:55:43<52:52:15,  8.48s/it]
 90%|█████████ | 202560/225000 [387:55:50<49:45:48,  7.98s/it]
 90%|█████████ | 202561/225000 [387:55:55<44:47:05,  7.19s/it]
 90%|█████████ | 202562/225000 [387:56:08<55:05:56,  8.84s/it]
 90%|█████████ | 202563/225000 [387:56:17<54:36:53,  8.76s/it]
 90%|█████████ | 202564/225000 [387:56:24<51:19:24,  8.24s/it]
 90%|█████████ | 202565/225000 [387:56:31<49:30:33,  7.94s/it]
 90%|█████████ | 202566/225000 [387:56:40<51:10:05,  8.21s/it]
 90%|█████████ | 202567/225000 [387:56:45<45:18:20,  7.27s/it]
 90%|█████████ | 202568/225000 [387:56:51<43:34:09,  6.99s/it]
 90%|█████████ | 202569/225000 [387:57:04<55:32:43,  8.91s/it]
 90%|█████████ | 202570/225000 [387:57:09<47:50:14,  7.68s/it]
 90%|█████████ | 202571/225000 [387:57:19<50:49:12,  8.16s/it]
 90%|█████████ | 202572/225000 [387:57:24<46:15:43,  7.43s/it]
 90%|█████████ | 202573/225000 [387:57:32<46:07:08,  7.40s/it]
 90%|█████████ | 202574/225000 [387:57:40<48:46:19,  7.83s/it]
 90%|█████████ | 202575/225000 [387:57:46<43:43:37,  7.02s/it]
 90%|█████████ | 202576/225000 [387:57:52<42:25:42,  6.81s/it]
 90%|█████████ | 202577/225000 [387:57:58<41:22:51,  6.64s/it]
 90%|█████████ | 202578/225000 [387:58:09<48:58:25,  7.86s/it]
 90%|█████████ | 202579/225000 [387:58:16<47:31:02,  7.63s/it]
 90%|█████████ | 202580/225000 [387:58:21<42:27:14,  6.82s/it]
 90%|█████████ | 202581/225000 [387:58:27<40:45:00,  6.54s/it]
 90%|█████████ | 202582/225000 [387:58:34<41:17:37,  6.63s/it]
 90%|█████████ | 202583/225000 [387:58:45<50:24:34,  8.10s/it]
 90%|█████████ | 202584/225000 [387:58:55<53:49:27,  8.64s/it]
 90%|█████████ | 202585/225000 [387:59:02<50:14:02,  8.07s/it]
 90%|█████████ | 202586/225000 [387:59:08<47:00:02,  7.55s/it]
 90%|█████████ | 202587/225000 [387:59:20<55:54:56,  8.98s/it]
 90%|█████████ | 202588/225000 [387:59:31<59:11:08,  9.51s/it]
 90%|█████████ | 202589/225000 [387:59:38<53:42:11,  8.63s/it]
 90%|█████████ | 202590/225000 [387:59:45<51:53:22,  8.34s/it]
 90%|█████████ | 202591/225000 [387:59:52<48:13:11,  7.75s/it]
 90%|█████████ | 202592/225000 [388:00:05<59:24:02,  9.54s/it]
 90%|█████████ | 202593/225000 [388:00:16<61:05:51,  9.82s/it]
 90%|█████████ | 202594/225000 [388:00:24<56:55:46,  9.15s/it]
 90%|█████████ | 202595/225000 [388:00:28<48:06:08,  7.73s/it]
 90%|█████████ | 202596/225000 [388:00:35<47:09:35,  7.58s/it]
 90%|█████████ | 202597/225000 [388:00:45<52:10:00,  8.38s/it]
 90%|█████████ | 202598/225000 [388:00:56<55:29:55,  8.92s/it]
 90%|█████████ | 202599/225000 [388:01:03<53:12:58,  8.55s/it]
 90%|█████████ | 202600/225000 [388:01:14<56:26:35,  9.07s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.149', 'learning_rate': '1.338e-06', 'epoch': '0.9004', 'num_input_tokens_seen': 651999912, 'train_runtime': '1.397e+06', 'train_tokens_per_second': '466.8'}

 90%|█████████ | 202600/225000 [388:01:14<56:26:35,  9.07s/it]
 90%|█████████ | 202601/225000 [388:01:26<62:51:37, 10.10s/it]
 90%|█████████ | 202602/225000 [388:01:31<53:03:32,  8.53s/it]
 90%|█████████ | 202603/225000 [388:01:41<56:16:46,  9.05s/it]
 90%|█████████ | 202604/225000 [388:01:51<57:36:00,  9.26s/it]
 90%|█████████ | 202605/225000 [388:02:01<58:14:25,  9.36s/it]
 90%|█████████ | 202606/225000 [388:02:10<57:47:12,  9.29s/it]
 90%|█████████ | 202607/225000 [388:02:16<52:10:19,  8.39s/it]
 90%|█████████ | 202608/225000 [388:02:21<46:39:10,  7.50s/it]
 90%|█████████ | 202609/225000 [388:02:35<58:48:22,  9.45s/it]
 90%|█████████ | 202610/225000 [388:02:43<56:05:53,  9.02s/it]
 90%|█████████ | 202611/225000 [388:02:50<51:06:09,  8.22s/it]
 90%|█████████ | 202612/225000 [388:02:57<48:20:37,  7.77s/it]
 90%|█████████ | 202613/225000 [388:03:07<52:54:05,  8.51s/it]
 90%|█████████ | 202614/225000 [388:03:17<55:50:24,  8.98s/it]
 90%|█████████ | 202615/225000 [388:03:28<60:08:45,  9.67s/it]
 90%|█████████ | 202616/225000 [388:03:41<65:35:24, 10.55s/it]
 90%|█████████ | 202617/225000 [388:03:50<62:33:12, 10.06s/it]
 90%|█████████ | 202618/225000 [388:03:58<58:39:16,  9.43s/it]
 90%|█████████ | 202619/225000 [388:04:05<54:06:50,  8.70s/it]
 90%|█████████ | 202620/225000 [388:04:11<50:29:17,  8.12s/it]
 90%|█████████ | 202621/225000 [388:04:25<60:40:38,  9.76s/it]
 90%|█████████ | 202622/225000 [388:04:30<51:14:20,  8.24s/it]
 90%|█████████ | 202623/225000 [388:04:42<58:52:33,  9.47s/it]
 90%|█████████ | 202624/225000 [388:04:49<54:42:46,  8.80s/it]
 90%|█████████ | 202625/225000 [388:04:57<53:01:30,  8.53s/it]
 90%|█████████ | 202626/225000 [388:05:07<55:43:43,  8.97s/it]
 90%|█████████ | 202627/225000 [388:05:13<49:35:01,  7.98s/it]
 90%|█████████ | 202628/225000 [388:05:19<46:29:04,  7.48s/it]
 90%|█████████ | 202629/225000 [388:05:26<45:27:00,  7.31s/it]
 90%|█████████ | 202630/225000 [388:05:38<54:32:58,  8.78s/it]
 90%|█████████ | 202631/225000 [388:05:45<50:08:53,  8.07s/it]
 90%|█████████ | 202632/225000 [388:05:51<46:14:23,  7.44s/it]
 90%|█████████ | 202633/225000 [388:06:01<50:55:05,  8.20s/it]
 90%|█████████ | 202634/225000 [388:06:05<43:56:12,  7.07s/it]
 90%|█████████ | 202635/225000 [388:06:16<50:50:05,  8.18s/it]
 90%|█████████ | 202636/225000 [388:06:24<50:34:31,  8.14s/it]
 90%|█████████ | 202637/225000 [388:06:35<56:18:17,  9.06s/it]
 90%|█████████ | 202638/225000 [388:06:47<61:41:46,  9.93s/it]
 90%|█████████ | 202639/225000 [388:06:55<58:39:40,  9.44s/it]
 90%|█████████ | 202640/225000 [388:07:04<57:24:25,  9.24s/it]
 90%|█████████ | 202641/225000 [388:07:15<61:26:19,  9.89s/it]
 90%|█████████ | 202642/225000 [388:07:22<54:18:01,  8.74s/it]
 90%|█████████ | 202643/225000 [388:07:32<58:01:31,  9.34s/it]
 90%|█████████ | 202644/225000 [388:07:40<54:41:52,  8.81s/it]
 90%|█████████ | 202645/225000 [388:07:51<58:52:29,  9.48s/it]
 90%|█████████ | 202646/225000 [388:07:59<55:50:12,  8.99s/it]
 90%|█████████ | 202647/225000 [388:08:08<55:27:08,  8.93s/it]
 90%|█████████ | 202648/225000 [388:08:12<47:54:35,  7.72s/it]
 90%|█████████ | 202649/225000 [388:08:18<43:28:34,  7.00s/it]
 90%|█████████ | 202650/225000 [388:08:28<49:51:19,  8.03s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.335', 'learning_rate': '1.333e-06', 'epoch': '0.9007', 'num_input_tokens_seen': 652233512, 'train_runtime': '1.397e+06', 'train_tokens_per_second': '466.8'}

 90%|█████████ | 202650/225000 [388:08:28<49:51:19,  8.03s/it]
 90%|█████████ | 202651/225000 [388:08:36<49:54:36,  8.04s/it]
 90%|█████████ | 202652/225000 [388:08:42<45:11:35,  7.28s/it]
 90%|█████████ | 202653/225000 [388:08:54<53:48:52,  8.67s/it]
 90%|█████████ | 202654/225000 [388:09:03<55:26:23,  8.93s/it]
 90%|█████████ | 202655/225000 [388:09:13<56:52:29,  9.16s/it]
 90%|█████████ | 202656/225000 [388:09:18<49:38:13,  8.00s/it]
 90%|█████████ | 202657/225000 [388:09:29<54:28:45,  8.78s/it]
 90%|█████████ | 202658/225000 [388:09:37<52:37:49,  8.48s/it]
 90%|█████████ | 202659/225000 [388:09:44<50:43:24,  8.17s/it]
 90%|█████████ | 202660/225000 [388:09:50<46:47:10,  7.54s/it]
 90%|█████████ | 202661/225000 [388:10:04<57:56:28,  9.34s/it]
 90%|█████████ | 202662/225000 [388:10:11<54:50:34,  8.84s/it]
 90%|█████████ | 202663/225000 [388:10:19<51:54:52,  8.37s/it]
 90%|█████████ | 202664/225000 [388:10:25<48:01:40,  7.74s/it]
 90%|█████████ | 202665/225000 [388:10:38<57:10:26,  9.22s/it]
 90%|█████████ | 202666/225000 [388:10:49<61:36:47,  9.93s/it]
 90%|█████████ | 202667/225000 [388:10:58<59:20:49,  9.57s/it]
 90%|█████████ | 202668/225000 [388:11:06<57:02:04,  9.19s/it]
 90%|█████████ | 202669/225000 [388:11:18<62:40:39, 10.10s/it]
 90%|█████████ | 202670/225000 [388:11:26<58:46:54,  9.48s/it]
 90%|█████████ | 202671/225000 [388:11:33<52:32:28,  8.47s/it]
 90%|█████████ | 202672/225000 [388:11:40<49:51:02,  8.04s/it]
 90%|█████████ | 202673/225000 [388:11:46<47:17:55,  7.63s/it]
 90%|█████████ | 202674/225000 [388:11:54<47:57:40,  7.73s/it]
 90%|█████████ | 202675/225000 [388:12:08<59:09:48,  9.54s/it]
 90%|█████████ | 202676/225000 [388:12:18<59:34:01,  9.61s/it]
 90%|█████████ | 202677/225000 [388:12:26<56:29:27,  9.11s/it]
 90%|█████████ | 202678/225000 [388:12:32<50:57:42,  8.22s/it]
 90%|█████████ | 202679/225000 [388:12:37<45:02:04,  7.26s/it]
 90%|█████████ | 202680/225000 [388:12:48<52:29:08,  8.47s/it]
 90%|█████████ | 202681/225000 [388:12:57<52:34:57,  8.48s/it]
 90%|█████████ | 202682/225000 [388:13:02<47:19:11,  7.63s/it]
 90%|█████████ | 202683/225000 [388:13:14<54:50:51,  8.85s/it]
 90%|█████████ | 202684/225000 [388:13:22<53:39:54,  8.66s/it]
 90%|█████████ | 202685/225000 [388:13:27<46:34:51,  7.51s/it]
 90%|█████████ | 202686/225000 [388:13:38<52:16:18,  8.43s/it]
 90%|█████████ | 202687/225000 [388:13:43<47:01:53,  7.59s/it]
 90%|█████████ | 202688/225000 [388:13:51<47:49:17,  7.72s/it]
 90%|█████████ | 202689/225000 [388:13:57<44:41:02,  7.21s/it]
 90%|█████████ | 202690/225000 [388:14:12<57:53:35,  9.34s/it]
 90%|█████████ | 202691/225000 [388:14:18<52:24:48,  8.46s/it]
 90%|█████████ | 202692/225000 [388:14:25<50:14:40,  8.11s/it]
 90%|█████████ | 202693/225000 [388:14:35<53:05:33,  8.57s/it]
 90%|█████████ | 202694/225000 [388:14:42<49:37:55,  8.01s/it]
 90%|█████████ | 202695/225000 [388:14:53<55:42:12,  8.99s/it]
 90%|█████████ | 202696/225000 [388:15:02<56:13:15,  9.07s/it]
 90%|█████████ | 202697/225000 [388:15:13<59:10:07,  9.55s/it]
 90%|█████████ | 202698/225000 [388:15:22<58:19:41,  9.42s/it]
 90%|█████████ | 202699/225000 [388:15:29<54:51:13,  8.85s/it]
 90%|█████████ | 202700/225000 [388:15:38<54:08:35,  8.74s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.346', 'learning_rate': '1.327e-06', 'epoch': '0.9009', 'num_input_tokens_seen': 652462672, 'train_runtime': '1.398e+06', 'train_tokens_per_second': '466.8'}

 90%|█████████ | 202700/225000 [388:15:38<54:08:35,  8.74s/it]
 90%|█████████ | 202701/225000 [388:15:43<46:24:32,  7.49s/it]
 90%|█████████ | 202702/225000 [388:15:49<44:57:14,  7.26s/it]
 90%|█████████ | 202703/225000 [388:16:00<50:34:25,  8.17s/it]
 90%|█████████ | 202704/225000 [388:16:05<44:55:58,  7.26s/it]
 90%|█████████ | 202705/225000 [388:16:15<51:13:32,  8.27s/it]
 90%|█████████ | 202706/225000 [388:16:21<46:34:48,  7.52s/it]
 90%|█████████ | 202707/225000 [388:16:31<51:35:24,  8.33s/it]
 90%|█████████ | 202708/225000 [388:16:39<50:15:37,  8.12s/it]
 90%|█████████ | 202709/225000 [388:16:44<44:48:14,  7.24s/it]
 90%|█████████ | 202710/225000 [388:16:54<50:22:03,  8.13s/it]
 90%|█████████ | 202711/225000 [388:17:02<48:41:08,  7.86s/it]
 90%|█████████ | 202712/225000 [388:17:08<45:12:03,  7.30s/it]
 90%|█████████ | 202713/225000 [388:17:16<47:51:26,  7.73s/it]
 90%|█████████ | 202714/225000 [388:17:33<63:47:35, 10.30s/it]
 90%|█████████ | 202715/225000 [388:17:43<63:21:08, 10.23s/it]
 90%|█████████ | 202716/225000 [388:17:54<65:10:35, 10.53s/it]
 90%|█████████ | 202717/225000 [388:18:02<61:12:23,  9.89s/it]
 90%|█████████ | 202718/225000 [388:18:10<56:28:32,  9.12s/it]
 90%|█████████ | 202719/225000 [388:18:17<53:32:59,  8.65s/it]
 90%|█████████ | 202720/225000 [388:18:23<48:55:35,  7.91s/it]
 90%|█████████ | 202721/225000 [388:18:35<55:19:50,  8.94s/it]
 90%|█████████ | 202722/225000 [388:18:42<53:00:02,  8.56s/it]
 90%|█████████ | 202723/225000 [388:18:51<53:30:14,  8.65s/it]
 90%|█████████ | 202724/225000 [388:18:59<52:27:08,  8.48s/it]
 90%|█████████ | 202725/225000 [388:19:04<46:25:32,  7.50s/it]
 90%|█████████ | 202726/225000 [388:19:11<45:23:44,  7.34s/it]
 90%|█████████ | 202727/225000 [388:19:17<42:59:32,  6.95s/it]
 90%|█████████ | 202728/225000 [388:19:24<41:21:51,  6.69s/it]
 90%|█████████ | 202729/225000 [388:19:37<54:47:19,  8.86s/it]
 90%|█████████ | 202730/225000 [388:19:46<54:06:12,  8.75s/it]
 90%|█████████ | 202731/225000 [388:19:54<52:34:31,  8.50s/it]
 90%|█████████ | 202732/225000 [388:20:01<49:23:57,  7.99s/it]
 90%|█████████ | 202733/225000 [388:20:17<65:30:01, 10.59s/it]
 90%|█████████ | 202734/225000 [388:20:25<59:47:26,  9.67s/it]
 90%|█████████ | 202735/225000 [388:20:30<52:05:26,  8.42s/it]
 90%|█████████ | 202736/225000 [388:20:40<54:15:23,  8.77s/it]
 90%|█████████ | 202737/225000 [388:20:46<50:04:27,  8.10s/it]
 90%|█████████ | 202738/225000 [388:20:59<57:47:51,  9.35s/it]
 90%|█████████ | 202739/225000 [388:21:04<50:17:40,  8.13s/it]
 90%|█████████ | 202740/225000 [388:21:10<45:27:51,  7.35s/it]
 90%|█████████ | 202741/225000 [388:21:22<55:32:25,  8.98s/it]
 90%|█████████ | 202742/225000 [388:21:28<48:37:03,  7.86s/it]
 90%|█████████ | 202743/225000 [388:21:33<44:50:31,  7.25s/it]
 90%|█████████ | 202744/225000 [388:21:45<53:26:41,  8.64s/it]
 90%|█████████ | 202745/225000 [388:21:53<51:07:13,  8.27s/it]
 90%|█████████ | 202746/225000 [388:22:01<50:42:50,  8.20s/it]
 90%|█████████ | 202747/225000 [388:22:08<49:33:05,  8.02s/it]
 90%|█████████ | 202748/225000 [388:22:15<46:16:21,  7.49s/it]
 90%|█████████ | 202749/225000 [388:22:25<51:19:24,  8.30s/it]
 90%|█████████ | 202750/225000 [388:22:33<51:06:22,  8.27s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.668', 'learning_rate': '1.321e-06', 'epoch': '0.9011', 'num_input_tokens_seen': 652686928, 'train_runtime': '1.398e+06', 'train_tokens_per_second': '466.8'}

 90%|█████████ | 202750/225000 [388:22:33<51:06:22,  8.27s/it]
 90%|█████████ | 202751/225000 [388:22:38<45:43:21,  7.40s/it]
 90%|█████████ | 202752/225000 [388:22:49<51:18:23,  8.30s/it]
 90%|█████████ | 202753/225000 [388:22:55<48:16:49,  7.81s/it]
 90%|█████████ | 202754/225000 [388:23:07<54:20:41,  8.79s/it]
 90%|█████████ | 202755/225000 [388:23:13<50:17:56,  8.14s/it]
 90%|█████████ | 202756/225000 [388:23:19<46:06:58,  7.46s/it]
 90%|█████████ | 202757/225000 [388:23:26<46:01:36,  7.45s/it]
 90%|█████████ | 202758/225000 [388:23:38<53:35:55,  8.68s/it]
 90%|█████████ | 202759/225000 [388:23:47<54:53:08,  8.88s/it]
 90%|█████████ | 202760/225000 [388:23:56<55:16:45,  8.95s/it]
 90%|█████████ | 202761/225000 [388:24:02<48:44:49,  7.89s/it]
 90%|█████████ | 202762/225000 [388:24:09<46:47:43,  7.58s/it]
 90%|█████████ | 202763/225000 [388:24:15<45:08:26,  7.31s/it]
 90%|█████████ | 202764/225000 [388:24:30<58:06:08,  9.41s/it]
 90%|█████████ | 202765/225000 [388:24:35<51:07:54,  8.28s/it]
 90%|█████████ | 202766/225000 [388:24:44<50:52:05,  8.24s/it]
 90%|█████████ | 202767/225000 [388:24:53<53:54:46,  8.73s/it]
 90%|█████████ | 202768/225000 [388:25:00<49:33:00,  8.02s/it]
 90%|█████████ | 202769/225000 [388:25:06<46:09:42,  7.48s/it]
 90%|█████████ | 202770/225000 [388:25:13<45:10:37,  7.32s/it]
 90%|█████████ | 202771/225000 [388:25:24<51:24:16,  8.32s/it]
 90%|█████████ | 202772/225000 [388:25:38<63:34:29, 10.30s/it]
 90%|█████████ | 202773/225000 [388:25:47<61:00:22,  9.88s/it]
 90%|█████████ | 202774/225000 [388:25:54<54:52:21,  8.89s/it]
 90%|█████████ | 202775/225000 [388:26:06<61:05:33,  9.90s/it]
 90%|█████████ | 202776/225000 [388:26:15<59:50:00,  9.69s/it]
 90%|█████████ | 202777/225000 [388:26:21<52:03:28,  8.43s/it]
 90%|█████████ | 202778/225000 [388:26:31<55:56:04,  9.06s/it]
 90%|█████████ | 202779/225000 [388:26:38<51:41:05,  8.37s/it]
 90%|█████████ | 202780/225000 [388:26:51<59:15:33,  9.60s/it]
 90%|█████████ | 202781/225000 [388:26:59<57:33:17,  9.33s/it]
 90%|█████████ | 202782/225000 [388:27:07<54:14:27,  8.79s/it]
 90%|█████████ | 202783/225000 [388:27:13<50:09:27,  8.13s/it]
 90%|█████████ | 202784/225000 [388:27:20<46:33:53,  7.55s/it]
 90%|█████████ | 202785/225000 [388:27:26<44:11:44,  7.16s/it]
 90%|█████████ | 202786/225000 [388:27:33<43:57:34,  7.12s/it]
 90%|█████████ | 202787/225000 [388:27:47<56:51:36,  9.22s/it]
 90%|█████████ | 202788/225000 [388:27:55<54:38:02,  8.85s/it]
 90%|█████████ | 202789/225000 [388:28:00<47:40:40,  7.73s/it]
 90%|█████████ | 202790/225000 [388:28:10<51:56:49,  8.42s/it]
 90%|█████████ | 202791/225000 [388:28:15<45:50:34,  7.43s/it]
 90%|█████████ | 202792/225000 [388:28:28<55:49:12,  9.05s/it]
 90%|█████████ | 202793/225000 [388:28:35<51:39:35,  8.37s/it]
 90%|█████████ | 202794/225000 [388:28:45<55:10:38,  8.95s/it]
 90%|█████████ | 202795/225000 [388:28:52<51:00:22,  8.27s/it]
 90%|█████████ | 202796/225000 [388:29:00<50:25:21,  8.18s/it]
 90%|█████████ | 202797/225000 [388:29:09<52:02:12,  8.44s/it]
 90%|█████████ | 202798/225000 [388:29:17<51:19:25,  8.32s/it]
 90%|█████████ | 202799/225000 [388:29:28<56:23:18,  9.14s/it]
 90%|█████████ | 202800/225000 [388:29:35<52:44:44,  8.55s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.248', 'learning_rate': '1.315e-06', 'epoch': '0.9013', 'num_input_tokens_seen': 652914928, 'train_runtime': '1.399e+06', 'train_tokens_per_second': '466.8'}

 90%|█████████ | 202800/225000 [388:29:35<52:44:44,  8.55s/it]
 90%|█████████ | 202801/225000 [388:29:43<52:07:28,  8.45s/it]
 90%|█████████ | 202802/225000 [388:29:51<50:26:21,  8.18s/it]
 90%|█████████ | 202803/225000 [388:29:58<48:22:15,  7.85s/it]
 90%|█████████ | 202804/225000 [388:30:13<62:10:41, 10.08s/it]
 90%|█████████ | 202805/225000 [388:30:25<64:32:34, 10.47s/it]
 90%|█████████ | 202806/225000 [388:30:32<58:26:11,  9.48s/it]
 90%|█████████ | 202807/225000 [388:30:45<64:46:56, 10.51s/it]
 90%|█████████ | 202808/225000 [388:30:49<52:20:13,  8.49s/it]
 90%|█████████ | 202809/225000 [388:30:56<49:56:33,  8.10s/it]
 90%|█████████ | 202810/225000 [388:31:04<50:56:08,  8.26s/it]
 90%|█████████ | 202811/225000 [388:31:16<57:01:18,  9.25s/it]
 90%|█████████ | 202812/225000 [388:31:22<50:35:45,  8.21s/it]
 90%|█████████ | 202813/225000 [388:31:30<49:46:43,  8.08s/it]
 90%|█████████ | 202814/225000 [388:31:36<45:58:44,  7.46s/it]
 90%|█████████ | 202815/225000 [388:31:45<49:54:02,  8.10s/it]
 90%|█████████ | 202816/225000 [388:31:57<56:04:52,  9.10s/it]
 90%|█████████ | 202817/225000 [388:32:06<56:25:01,  9.16s/it]
 90%|█████████ | 202818/225000 [388:32:14<55:18:09,  8.98s/it]
 90%|█████████ | 202819/225000 [388:32:20<48:45:01,  7.91s/it]
 90%|█████████ | 202820/225000 [388:32:27<47:04:03,  7.64s/it]
 90%|█████████ | 202821/225000 [388:32:39<55:32:16,  9.01s/it]
 90%|█████████ | 202822/225000 [388:32:46<51:26:49,  8.35s/it]
 90%|█████████ | 202823/225000 [388:32:53<48:57:28,  7.95s/it]
 90%|█████████ | 202824/225000 [388:33:01<48:46:41,  7.92s/it]
 90%|█████████ | 202825/225000 [388:33:09<49:17:42,  8.00s/it]
 90%|█████████ | 202826/225000 [388:33:27<68:22:41, 11.10s/it]
 90%|█████████ | 202827/225000 [388:33:34<60:09:58,  9.77s/it]
 90%|█████████ | 202828/225000 [388:33:45<62:05:22, 10.08s/it]
 90%|█████████ | 202829/225000 [388:33:53<58:58:01,  9.57s/it]
 90%|█████████ | 202830/225000 [388:34:02<58:13:40,  9.46s/it]
 90%|█████████ | 202831/225000 [388:34:08<52:04:52,  8.46s/it]
 90%|█████████ | 202832/225000 [388:34:15<47:59:56,  7.79s/it]
 90%|█████████ | 202833/225000 [388:34:29<59:24:34,  9.65s/it]
 90%|█████████ | 202834/225000 [388:34:35<52:45:16,  8.57s/it]
 90%|█████████ | 202835/225000 [388:34:44<54:11:15,  8.80s/it]
 90%|█████████ | 202836/225000 [388:34:49<46:54:56,  7.62s/it]
 90%|█████████ | 202837/225000 [388:35:02<57:00:39,  9.26s/it]
 90%|█████████ | 202838/225000 [388:35:12<59:02:39,  9.59s/it]
 90%|█████████ | 202839/225000 [388:35:21<57:49:58,  9.39s/it]
 90%|█████████ | 202840/225000 [388:35:27<51:47:54,  8.41s/it]
 90%|█████████ | 202841/225000 [388:35:34<47:58:00,  7.79s/it]
 90%|█████████ | 202842/225000 [388:35:40<45:24:51,  7.38s/it]
 90%|█████████ | 202843/225000 [388:35:50<49:54:18,  8.11s/it]
 90%|█████████ | 202844/225000 [388:35:58<50:34:47,  8.22s/it]
 90%|█████████ | 202845/225000 [388:36:06<48:39:30,  7.91s/it]
 90%|█████████ | 202846/225000 [388:36:12<45:46:44,  7.44s/it]
 90%|█████████ | 202847/225000 [388:36:23<52:06:18,  8.47s/it]
 90%|█████████ | 202848/225000 [388:36:30<50:18:16,  8.18s/it]
 90%|█████████ | 202849/225000 [388:36:41<54:31:31,  8.86s/it]
 90%|█████████ | 202850/225000 [388:36:53<60:59:45,  9.91s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.205', 'learning_rate': '1.309e-06', 'epoch': '0.9016', 'num_input_tokens_seen': 653144672, 'train_runtime': '1.399e+06', 'train_tokens_per_second': '466.9'}

 90%|█████████ | 202850/225000 [388:36:53<60:59:45,  9.91s/it]
 90%|█████████ | 202851/225000 [388:37:00<55:39:22,  9.05s/it]
 90%|█████████ | 202852/225000 [388:37:06<49:12:08,  8.00s/it]
 90%|█████████ | 202853/225000 [388:37:17<55:03:29,  8.95s/it]
 90%|█████████ | 202854/225000 [388:37:25<53:27:40,  8.69s/it]
 90%|█████████ | 202855/225000 [388:37:31<47:53:51,  7.79s/it]
 90%|█████████ | 202856/225000 [388:37:38<46:17:56,  7.53s/it]
 90%|█████████ | 202857/225000 [388:37:46<48:12:51,  7.84s/it]
 90%|█████████ | 202858/225000 [388:37:56<51:19:20,  8.34s/it]
 90%|█████████ | 202859/225000 [388:38:05<53:24:49,  8.68s/it]
 90%|█████████ | 202860/225000 [388:38:15<55:05:12,  8.96s/it]
 90%|█████████ | 202861/225000 [388:38:29<64:12:32, 10.44s/it]
 90%|█████████ | 202862/225000 [388:38:38<62:44:53, 10.20s/it]
 90%|█████████ | 202863/225000 [388:38:43<53:12:13,  8.65s/it]
 90%|█████████ | 202864/225000 [388:38:52<52:29:31,  8.54s/it]
 90%|█████████ | 202865/225000 [388:38:59<50:24:01,  8.20s/it]
 90%|█████████ | 202866/225000 [388:39:13<60:24:27,  9.83s/it]
 90%|█████████ | 202867/225000 [388:39:23<61:06:46,  9.94s/it]
 90%|█████████ | 202868/225000 [388:39:29<54:46:09,  8.91s/it]
 90%|█████████ | 202869/225000 [388:39:38<54:42:09,  8.90s/it]
 90%|█████████ | 202870/225000 [388:39:49<58:52:38,  9.58s/it]
 90%|█████████ | 202871/225000 [388:39:59<58:39:05,  9.54s/it]
 90%|█████████ | 202872/225000 [388:40:05<51:32:42,  8.39s/it]
 90%|█████████ | 202873/225000 [388:40:16<58:00:54,  9.44s/it]
 90%|█████████ | 202874/225000 [388:40:21<49:49:01,  8.11s/it]
 90%|█████████ | 202875/225000 [388:40:30<50:33:44,  8.23s/it]
 90%|█████████ | 202876/225000 [388:40:42<57:41:11,  9.39s/it]
 90%|█████████ | 202877/225000 [388:40:48<51:32:04,  8.39s/it]
 90%|█████████ | 202878/225000 [388:40:59<56:28:56,  9.19s/it]
 90%|█████████ | 202879/225000 [388:41:06<51:20:03,  8.35s/it]
 90%|█████████ | 202880/225000 [388:41:17<57:24:19,  9.34s/it]
 90%|█████████ | 202881/225000 [388:41:22<48:06:48,  7.83s/it]
 90%|█████████ | 202882/225000 [388:41:27<43:37:37,  7.10s/it]
 90%|█████████ | 202883/225000 [388:41:35<45:44:52,  7.45s/it]
 90%|█████████ | 202884/225000 [388:41:45<49:57:24,  8.13s/it]
 90%|█████████ | 202885/225000 [388:41:52<47:35:50,  7.75s/it]
 90%|█████████ | 202886/225000 [388:41:57<43:47:47,  7.13s/it]
 90%|█████████ | 202887/225000 [388:42:07<47:21:49,  7.71s/it]
 90%|█████████ | 202888/225000 [388:42:13<44:46:10,  7.29s/it]
 90%|█████████ | 202889/225000 [388:42:18<40:36:02,  6.61s/it]
 90%|█████████ | 202890/225000 [388:42:23<37:08:44,  6.05s/it]
 90%|█████████ | 202891/225000 [388:42:34<46:23:45,  7.55s/it]
 90%|█████████ | 202892/225000 [388:42:47<56:14:46,  9.16s/it]
 90%|█████████ | 202893/225000 [388:42:53<51:06:15,  8.32s/it]
 90%|█████████ | 202894/225000 [388:43:04<55:54:32,  9.10s/it]
 90%|█████████ | 202895/225000 [388:43:12<53:54:29,  8.78s/it]
 90%|█████████ | 202896/225000 [388:43:22<56:25:54,  9.19s/it]
 90%|█████████ | 202897/225000 [388:43:28<49:54:34,  8.13s/it]
 90%|█████████ | 202898/225000 [388:43:35<47:31:46,  7.74s/it]
 90%|█████████ | 202899/225000 [388:43:42<46:18:35,  7.54s/it]
 90%|█████████ | 202900/225000 [388:43:48<44:36:44,  7.27s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.347', 'learning_rate': '1.303e-06', 'epoch': '0.9018', 'num_input_tokens_seen': 653373536, 'train_runtime': '1.399e+06', 'train_tokens_per_second': '466.9'}

 90%|█████████ | 202900/225000 [388:43:48<44:36:44,  7.27s/it]
 90%|█████████ | 202901/225000 [388:44:01<55:02:26,  8.97s/it]
 90%|█████████ | 202902/225000 [388:44:07<48:39:00,  7.93s/it]
 90%|█████████ | 202903/225000 [388:44:13<45:03:25,  7.34s/it]
 90%|█████████ | 202904/225000 [388:44:24<52:25:45,  8.54s/it]
 90%|█████████ | 202905/225000 [388:44:30<47:05:54,  7.67s/it]
 90%|█████████ | 202906/225000 [388:44:34<41:19:57,  6.73s/it]
 90%|█████████ | 202907/225000 [388:44:48<54:47:02,  8.93s/it]
 90%|█████████ | 202908/225000 [388:44:54<48:15:47,  7.86s/it]
 90%|█████████ | 202909/225000 [388:44:59<44:04:57,  7.18s/it]
 90%|█████████ | 202910/225000 [388:45:07<44:29:01,  7.25s/it]
 90%|█████████ | 202911/225000 [388:45:18<52:43:27,  8.59s/it]
 90%|█████████ | 202912/225000 [388:45:27<53:33:21,  8.73s/it]
 90%|█████████ | 202913/225000 [388:45:41<63:11:24, 10.30s/it]
 90%|█████████ | 202914/225000 [388:45:55<69:07:55, 11.27s/it]
 90%|█████████ | 202915/225000 [388:46:03<63:22:23, 10.33s/it]
 90%|█████████ | 202916/225000 [388:46:09<54:47:12,  8.93s/it]
 90%|█████████ | 202917/225000 [388:46:17<53:18:58,  8.69s/it]
 90%|█████████ | 202918/225000 [388:46:29<60:00:49,  9.78s/it]
 90%|█████████ | 202919/225000 [388:46:38<58:34:20,  9.55s/it]
 90%|█████████ | 202920/225000 [388:46:48<59:15:59,  9.66s/it]
 90%|█████████ | 202921/225000 [388:46:54<52:44:25,  8.60s/it]
 90%|█████████ | 202922/225000 [388:47:02<51:02:46,  8.32s/it]
 90%|█████████ | 202923/225000 [388:47:10<50:33:09,  8.24s/it]
 90%|█████████ | 202924/225000 [388:47:22<58:06:30,  9.48s/it]
 90%|█████████ | 202925/225000 [388:47:28<50:42:12,  8.27s/it]
 90%|█████████ | 202926/225000 [388:47:40<57:40:33,  9.41s/it]
 90%|█████████ | 202927/225000 [388:47:47<53:59:42,  8.81s/it]
 90%|█████████ | 202928/225000 [388:47:54<50:41:47,  8.27s/it]
 90%|█████████ | 202929/225000 [388:48:00<46:00:28,  7.50s/it]
 90%|█████████ | 202930/225000 [388:48:07<44:37:30,  7.28s/it]
 90%|█████████ | 202931/225000 [388:48:15<46:20:57,  7.56s/it]
 90%|█████████ | 202932/225000 [388:48:25<50:29:01,  8.24s/it]
 90%|█████████ | 202933/225000 [388:48:35<55:03:38,  8.98s/it]
 90%|█████████ | 202934/225000 [388:48:44<54:58:14,  8.97s/it]
 90%|█████████ | 202935/225000 [388:48:49<47:27:01,  7.74s/it]
 90%|█████████ | 202936/225000 [388:48:56<45:46:03,  7.47s/it]
 90%|█████████ | 202937/225000 [388:49:08<53:50:09,  8.78s/it]
 90%|█████████ | 202938/225000 [388:49:17<54:59:40,  8.97s/it]
 90%|█████████ | 202939/225000 [388:49:26<53:36:18,  8.75s/it]
 90%|█████████ | 202940/225000 [388:49:33<50:38:41,  8.26s/it]
 90%|█████████ | 202941/225000 [388:49:43<54:24:05,  8.88s/it]
 90%|█████████ | 202942/225000 [388:49:52<54:13:49,  8.85s/it]
 90%|█████████ | 202943/225000 [388:49:57<46:42:56,  7.62s/it]
 90%|█████████ | 202944/225000 [388:50:04<46:38:16,  7.61s/it]
 90%|█████████ | 202945/225000 [388:50:14<51:12:40,  8.36s/it]
 90%|█████████ | 202946/225000 [388:50:23<51:47:17,  8.45s/it]
 90%|█████████ | 202947/225000 [388:50:31<51:24:09,  8.39s/it]
 90%|█████████ | 202948/225000 [388:50:36<45:30:24,  7.43s/it]
 90%|█████████ | 202949/225000 [388:50:44<45:10:00,  7.37s/it]
 90%|█████████ | 202950/225000 [388:50:53<49:01:31,  8.00s/it]
                                                              
{'loss': '1.292', 'grad_norm': '0.9956', 'learning_rate': '1.297e-06', 'epoch': '0.902', 'num_input_tokens_seen': 653604776, 'train_runtime': '1.4e+06', 'train_tokens_per_second': '466.9'}

 90%|█████████ | 202950/225000 [388:50:53<49:01:31,  8.00s/it]
 90%|█████████ | 202951/225000 [388:51:03<53:02:01,  8.66s/it]
 90%|█████████ | 202952/225000 [388:51:13<54:18:35,  8.87s/it]
 90%|█████████ | 202953/225000 [388:51:23<57:51:26,  9.45s/it]
 90%|█████████ | 202954/225000 [388:51:32<56:47:48,  9.27s/it]
 90%|█████████ | 202955/225000 [388:51:39<52:43:58,  8.61s/it]
 90%|█████████ | 202956/225000 [388:51:46<48:40:56,  7.95s/it]
 90%|█████████ | 202957/225000 [388:51:54<48:58:30,  8.00s/it]
 90%|█████████ | 202958/225000 [388:52:06<57:19:14,  9.36s/it]
 90%|█████████ | 202959/225000 [388:52:17<59:46:31,  9.76s/it]
 90%|█████████ | 202960/225000 [388:52:27<59:20:45,  9.69s/it]
 90%|█████████ | 202961/225000 [388:52:31<49:42:29,  8.12s/it]
 90%|█████████ | 202962/225000 [388:52:40<51:27:54,  8.41s/it]
 90%|█████████ | 202963/225000 [388:52:45<44:12:02,  7.22s/it]
 90%|█████████ | 202964/225000 [388:52:56<52:24:39,  8.56s/it]
 90%|█████████ | 202965/225000 [388:53:07<55:40:34,  9.10s/it]
 90%|█████████ | 202966/225000 [388:53:11<46:28:19,  7.59s/it]
 90%|█████████ | 202967/225000 [388:53:23<55:29:28,  9.07s/it]
 90%|█████████ | 202968/225000 [388:53:29<49:31:00,  8.09s/it]
 90%|█████████ | 202969/225000 [388:53:39<53:23:59,  8.73s/it]
 90%|█████████ | 202970/225000 [388:53:50<56:18:14,  9.20s/it]
 90%|█████████ | 202971/225000 [388:53:56<52:05:43,  8.51s/it]
 90%|█████████ | 202972/225000 [388:54:03<48:22:45,  7.91s/it]
 90%|█████████ | 202973/225000 [388:54:09<44:41:48,  7.31s/it]
 90%|█████████ | 202974/225000 [388:54:18<48:55:08,  8.00s/it]
 90%|█████████ | 202975/225000 [388:54:25<45:35:14,  7.45s/it]
 90%|█████████ | 202976/225000 [388:54:34<48:59:53,  8.01s/it]
 90%|█████████ | 202977/225000 [388:54:46<56:48:48,  9.29s/it]
 90%|█████████ | 202978/225000 [388:54:52<50:26:09,  8.24s/it]
 90%|█████████ | 202979/225000 [388:55:00<50:44:07,  8.29s/it]
 90%|█████████ | 202980/225000 [388:55:08<49:11:56,  8.04s/it]
 90%|█████████ | 202981/225000 [388:55:15<46:59:04,  7.68s/it]
 90%|█████████ | 202982/225000 [388:55:29<59:44:17,  9.77s/it]
 90%|█████████ | 202983/225000 [388:55:40<62:12:14, 10.17s/it]
 90%|█████████ | 202984/225000 [388:55:50<60:04:10,  9.82s/it]
 90%|█████████ | 202985/225000 [388:56:01<62:37:58, 10.24s/it]
 90%|█████████ | 202986/225000 [388:56:09<59:32:26,  9.74s/it]
 90%|█████████ | 202987/225000 [388:56:17<56:33:38,  9.25s/it]
 90%|█████████ | 202988/225000 [388:56:27<56:39:18,  9.27s/it]
 90%|█████████ | 202989/225000 [388:56:37<58:47:55,  9.62s/it]
 90%|█████████ | 202990/225000 [388:56:50<63:50:57, 10.44s/it]
 90%|█████████ | 202991/225000 [388:57:00<63:10:07, 10.33s/it]
 90%|█████████ | 202992/225000 [388:57:10<63:17:38, 10.35s/it]
 90%|█████████ | 202993/225000 [388:57:17<56:44:42,  9.28s/it]
 90%|█████████ | 202994/225000 [388:57:25<54:40:36,  8.94s/it]
 90%|█████████ | 202995/225000 [388:57:29<46:25:32,  7.60s/it]
 90%|█████████ | 202996/225000 [388:57:39<50:50:19,  8.32s/it]
 90%|█████████ | 202997/225000 [388:57:48<51:11:20,  8.38s/it]
 90%|█████████ | 202998/225000 [388:57:52<43:50:36,  7.17s/it]
 90%|█████████ | 202999/225000 [388:58:01<47:11:48,  7.72s/it]
 90%|█████████ | 203000/225000 [388:58:08<44:46:57,  7.33s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.289', 'learning_rate': '1.292e-06', 'epoch': '0.9022', 'num_input_tokens_seen': 653837328, 'train_runtime': '1.4e+06', 'train_tokens_per_second': '466.9'}

 90%|█████████ | 203000/225000 [388:58:08<44:46:57,  7.33s/it]
 90%|█████████ | 203001/225000 [388:58:17<48:14:51,  7.90s/it]
 90%|█████████ | 203002/225000 [388:58:22<43:13:02,  7.07s/it]
 90%|█████████ | 203003/225000 [388:58:30<44:51:52,  7.34s/it]
 90%|█████████ | 203004/225000 [388:58:41<52:01:51,  8.52s/it]
 90%|█████████ | 203005/225000 [388:58:50<52:01:58,  8.52s/it]
 90%|█████████ | 203006/225000 [388:58:56<47:30:40,  7.78s/it]
 90%|█████████ | 203007/225000 [388:59:06<52:29:02,  8.59s/it]
 90%|█████████ | 203008/225000 [388:59:12<47:17:57,  7.74s/it]
 90%|█████████ | 203009/225000 [388:59:20<46:58:52,  7.69s/it]
 90%|█████████ | 203010/225000 [388:59:31<53:45:20,  8.80s/it]
 90%|█████████ | 203011/225000 [388:59:39<52:46:23,  8.64s/it]
 90%|█████████ | 203012/225000 [388:59:48<53:12:55,  8.71s/it]
 90%|█████████ | 203013/225000 [388:59:55<49:30:15,  8.11s/it]
 90%|█████████ | 203014/225000 [389:00:00<44:27:32,  7.28s/it]
 90%|█████████ | 203015/225000 [389:00:08<45:27:14,  7.44s/it]
 90%|█████████ | 203016/225000 [389:00:14<43:27:12,  7.12s/it]
 90%|█████████ | 203017/225000 [389:00:32<61:58:54, 10.15s/it]
 90%|█████████ | 203018/225000 [389:00:42<61:31:21, 10.08s/it]
 90%|█████████ | 203019/225000 [389:00:50<57:48:42,  9.47s/it]
 90%|█████████ | 203020/225000 [389:01:00<59:00:55,  9.67s/it]
 90%|█████████ | 203021/225000 [389:01:07<54:37:05,  8.95s/it]
 90%|█████████ | 203022/225000 [389:01:13<48:48:43,  8.00s/it]
 90%|█████████ | 203023/225000 [389:01:18<44:25:09,  7.28s/it]
 90%|█████████ | 203024/225000 [389:01:32<55:10:48,  9.04s/it]
 90%|█████████ | 203025/225000 [389:01:41<55:42:57,  9.13s/it]
 90%|█████████ | 203026/225000 [389:01:46<47:48:23,  7.83s/it]
 90%|█████████ | 203027/225000 [389:01:53<47:41:32,  7.81s/it]
 90%|█████████ | 203028/225000 [389:02:00<46:05:49,  7.55s/it]
 90%|█████████ | 203029/225000 [389:02:07<43:36:52,  7.15s/it]
 90%|█████████ | 203030/225000 [389:02:18<51:25:13,  8.43s/it]
 90%|█████████ | 203031/225000 [389:02:24<46:35:21,  7.63s/it]
 90%|█████████ | 203032/225000 [389:02:32<48:29:12,  7.95s/it]
 90%|█████████ | 203033/225000 [389:02:47<60:11:36,  9.86s/it]
 90%|█████████ | 203034/225000 [389:02:57<61:11:07, 10.03s/it]
 90%|█████████ | 203035/225000 [389:03:06<59:48:39,  9.80s/it]
 90%|█████████ | 203036/225000 [389:03:14<54:45:48,  8.98s/it]
 90%|█████████ | 203037/225000 [389:03:25<58:42:57,  9.62s/it]
 90%|█████████ | 203038/225000 [389:03:31<52:52:35,  8.67s/it]
 90%|█████████ | 203039/225000 [389:03:36<46:14:41,  7.58s/it]
 90%|█████████ | 203040/225000 [389:03:45<49:24:03,  8.10s/it]
 90%|█████████ | 203041/225000 [389:03:51<44:51:30,  7.35s/it]
 90%|█████████ | 203042/225000 [389:03:57<42:52:50,  7.03s/it]
 90%|█████████ | 203043/225000 [389:04:09<51:16:21,  8.41s/it]
 90%|█████████ | 203044/225000 [389:04:18<52:12:15,  8.56s/it]
 90%|█████████ | 203045/225000 [389:04:30<58:12:29,  9.54s/it]
 90%|█████████ | 203046/225000 [389:04:38<55:36:00,  9.12s/it]
 90%|█████████ | 203047/225000 [389:04:45<51:36:57,  8.46s/it]
 90%|█████████ | 203048/225000 [389:04:50<46:30:36,  7.63s/it]
 90%|█████████ | 203049/225000 [389:05:00<49:46:33,  8.16s/it]
 90%|█████████ | 203050/225000 [389:05:12<56:27:54,  9.26s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.164', 'learning_rate': '1.286e-06', 'epoch': '0.9024', 'num_input_tokens_seen': 654064392, 'train_runtime': '1.401e+06', 'train_tokens_per_second': '467'}

 90%|█████████ | 203050/225000 [389:05:12<56:27:54,  9.26s/it]
 90%|█████████ | 203051/225000 [389:05:19<53:13:47,  8.73s/it]
 90%|█████████ | 203052/225000 [389:05:26<49:01:55,  8.04s/it]
 90%|█████████ | 203053/225000 [389:05:32<46:27:41,  7.62s/it]
 90%|█████████ | 203054/225000 [389:05:45<56:31:55,  9.27s/it]
 90%|█████████ | 203055/225000 [389:05:53<53:07:58,  8.72s/it]
 90%|█████████ | 203056/225000 [389:05:57<45:22:09,  7.44s/it]
 90%|█████████ | 203057/225000 [389:06:08<51:30:30,  8.45s/it]
 90%|█████████ | 203058/225000 [389:06:20<57:32:12,  9.44s/it]
 90%|█████████ | 203059/225000 [389:06:30<58:06:25,  9.53s/it]
 90%|█████████ | 203060/225000 [389:06:34<48:58:50,  8.04s/it]
 90%|█████████ | 203061/225000 [389:06:41<47:05:29,  7.73s/it]
 90%|█████████ | 203062/225000 [389:06:51<51:33:49,  8.46s/it]
 90%|█████████ | 203063/225000 [389:07:04<59:30:41,  9.77s/it]
 90%|█████████ | 203064/225000 [389:07:16<62:58:22, 10.33s/it]
 90%|█████████ | 203065/225000 [389:07:23<56:32:46,  9.28s/it]
 90%|█████████ | 203066/225000 [389:07:29<51:27:26,  8.45s/it]
 90%|█████████ | 203067/225000 [389:07:40<56:28:54,  9.27s/it]
 90%|█████████ | 203068/225000 [389:07:46<49:36:47,  8.14s/it]
 90%|█████████ | 203069/225000 [389:07:54<49:28:44,  8.12s/it]
 90%|█████████ | 203070/225000 [389:08:11<65:26:50, 10.74s/it]
 90%|█████████ | 203071/225000 [389:08:16<55:49:17,  9.16s/it]
 90%|█████████ | 203072/225000 [389:08:25<55:19:13,  9.08s/it]
 90%|█████████ | 203073/225000 [389:08:35<56:13:47,  9.23s/it]
 90%|█████████ | 203074/225000 [389:08:40<48:45:02,  8.00s/it]
 90%|█████████ | 203075/225000 [389:08:48<49:52:36,  8.19s/it]
 90%|█████████ | 203076/225000 [389:08:59<54:00:29,  8.87s/it]
 90%|█████████ | 203077/225000 [389:09:09<55:49:11,  9.17s/it]
 90%|█████████ | 203078/225000 [389:09:20<60:30:42,  9.94s/it]
 90%|█████████ | 203079/225000 [389:09:28<56:02:05,  9.20s/it]
 90%|█████████ | 203080/225000 [389:09:36<53:14:38,  8.74s/it]
 90%|█████████ | 203081/225000 [389:09:42<47:57:38,  7.88s/it]
 90%|█████████ | 203082/225000 [389:09:54<56:04:48,  9.21s/it]
 90%|█████████ | 203083/225000 [389:10:01<52:12:58,  8.58s/it]
 90%|█████████ | 203084/225000 [389:10:07<47:20:47,  7.78s/it]
 90%|█████████ | 203085/225000 [389:10:17<52:25:04,  8.61s/it]
 90%|█████████ | 203086/225000 [389:10:29<57:57:43,  9.52s/it]
 90%|█████████ | 203087/225000 [389:10:39<59:02:22,  9.70s/it]
 90%|█████████ | 203088/225000 [389:10:51<62:41:01, 10.30s/it]
 90%|█████████ | 203089/225000 [389:10:58<56:36:18,  9.30s/it]
 90%|█████████ | 203090/225000 [389:11:05<52:27:54,  8.62s/it]
 90%|█████████ | 203091/225000 [389:11:17<59:17:09,  9.74s/it]
 90%|█████████ | 203092/225000 [389:11:26<57:31:56,  9.45s/it]
 90%|█████████ | 203093/225000 [389:11:35<56:52:34,  9.35s/it]
 90%|█████████ | 203094/225000 [389:11:42<52:58:16,  8.71s/it]
 90%|█████████ | 203095/225000 [389:11:49<50:12:09,  8.25s/it]
 90%|█████████ | 203096/225000 [389:12:03<58:59:16,  9.69s/it]
 90%|█████████ | 203097/225000 [389:12:07<50:16:53,  8.26s/it]
 90%|█████████ | 203098/225000 [389:12:14<46:14:17,  7.60s/it]
 90%|█████████ | 203099/225000 [389:12:24<50:56:55,  8.37s/it]
 90%|█████████ | 203100/225000 [389:12:31<49:09:52,  8.08s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.14', 'learning_rate': '1.28e-06', 'epoch': '0.9027', 'num_input_tokens_seen': 654298904, 'train_runtime': '1.401e+06', 'train_tokens_per_second': '467'}

 90%|█████████ | 203100/225000 [389:12:31<49:09:52,  8.08s/it]
 90%|█████████ | 203101/225000 [389:12:40<50:02:38,  8.23s/it]
 90%|█████████ | 203102/225000 [389:12:46<47:28:17,  7.80s/it]
 90%|█████████ | 203103/225000 [389:13:02<61:24:37, 10.10s/it]
 90%|█████████ | 203104/225000 [389:13:12<61:09:44, 10.06s/it]
 90%|█████████ | 203105/225000 [389:13:19<56:10:23,  9.24s/it]
 90%|█████████ | 203106/225000 [389:13:26<51:45:29,  8.51s/it]
 90%|█████████ | 203107/225000 [389:13:32<46:23:27,  7.63s/it]
 90%|█████████ | 203108/225000 [389:13:42<50:38:42,  8.33s/it]
 90%|█████████ | 203109/225000 [389:13:48<47:31:08,  7.81s/it]
 90%|█████████ | 203110/225000 [389:13:54<44:14:53,  7.28s/it]
 90%|█████████ | 203111/225000 [389:14:01<43:32:06,  7.16s/it]
 90%|█████████ | 203112/225000 [389:14:09<44:15:08,  7.28s/it]
 90%|█████████ | 203113/225000 [389:14:21<54:23:34,  8.95s/it]
 90%|█████████ | 203114/225000 [389:14:30<53:42:41,  8.83s/it]
 90%|█████████ | 203115/225000 [389:14:38<52:07:46,  8.58s/it]
 90%|█████████ | 203116/225000 [389:14:47<53:34:56,  8.81s/it]
 90%|█████████ | 203117/225000 [389:14:54<49:04:16,  8.07s/it]
 90%|█████████ | 203118/225000 [389:15:01<48:09:06,  7.92s/it]
 90%|█████████ | 203119/225000 [389:15:06<41:25:06,  6.81s/it]
 90%|█████████ | 203120/225000 [389:15:11<38:44:44,  6.37s/it]
 90%|█████████ | 203121/225000 [389:15:22<47:45:09,  7.86s/it]
 90%|█████████ | 203122/225000 [389:15:32<51:46:21,  8.52s/it]
 90%|█████████ | 203123/225000 [389:15:43<56:35:18,  9.31s/it]
 90%|█████████ | 203124/225000 [389:15:48<48:46:39,  8.03s/it]
 90%|█████████ | 203125/225000 [389:15:56<48:17:32,  7.95s/it]
 90%|█████████ | 203126/225000 [389:16:07<54:16:31,  8.93s/it]
 90%|█████████ | 203127/225000 [389:16:13<47:58:55,  7.90s/it]
 90%|█████████ | 203128/225000 [389:16:19<44:41:10,  7.36s/it]
 90%|█████████ | 203129/225000 [389:16:32<55:22:40,  9.12s/it]
 90%|█████████ | 203130/225000 [389:16:41<54:45:18,  9.01s/it]
 90%|█████████ | 203131/225000 [389:16:48<50:23:48,  8.30s/it]
 90%|█████████ | 203132/225000 [389:16:55<49:01:05,  8.07s/it]
 90%|█████████ | 203133/225000 [389:17:03<47:38:03,  7.84s/it]
 90%|█████████ | 203134/225000 [389:17:11<48:32:14,  7.99s/it]
 90%|█████████ | 203135/225000 [389:17:18<46:25:51,  7.64s/it]
 90%|█████████ | 203136/225000 [389:17:27<49:28:24,  8.15s/it]
 90%|█████████ | 203137/225000 [389:17:35<49:27:06,  8.14s/it]
 90%|█████████ | 203138/225000 [389:17:43<49:23:07,  8.13s/it]
 90%|█████████ | 203139/225000 [389:17:58<60:37:07,  9.98s/it]
 90%|█████████ | 203140/225000 [389:18:07<59:39:32,  9.82s/it]
 90%|█████████ | 203141/225000 [389:18:16<57:30:08,  9.47s/it]
 90%|█████████ | 203142/225000 [389:18:22<51:46:51,  8.53s/it]
 90%|█████████ | 203143/225000 [389:18:34<58:06:37,  9.57s/it]
 90%|█████████ | 203144/225000 [389:18:39<49:13:06,  8.11s/it]
 90%|█████████ | 203145/225000 [389:18:44<45:00:32,  7.41s/it]
 90%|█████████ | 203146/225000 [389:18:51<43:02:33,  7.09s/it]
 90%|█████████ | 203147/225000 [389:19:02<50:00:35,  8.24s/it]
 90%|█████████ | 203148/225000 [389:19:09<47:35:58,  7.84s/it]
 90%|█████████ | 203149/225000 [389:19:15<45:08:31,  7.44s/it]
 90%|█████████ | 203150/225000 [389:19:21<43:00:32,  7.09s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.22', 'learning_rate': '1.274e-06', 'epoch': '0.9029', 'num_input_tokens_seen': 654524824, 'train_runtime': '1.402e+06', 'train_tokens_per_second': '467'}

 90%|█████████ | 203150/225000 [389:19:21<43:00:32,  7.09s/it]
 90%|█████████ | 203151/225000 [389:19:28<42:52:10,  7.06s/it]
 90%|█████████ | 203152/225000 [389:19:33<38:51:20,  6.40s/it]
 90%|█████████ | 203153/225000 [389:19:44<46:21:10,  7.64s/it]
 90%|█████████ | 203154/225000 [389:19:53<48:21:40,  7.97s/it]
 90%|█████████ | 203155/225000 [389:20:02<51:38:22,  8.51s/it]
 90%|█████████ | 203156/225000 [389:20:10<50:12:45,  8.28s/it]
 90%|█████████ | 203157/225000 [389:20:19<52:00:00,  8.57s/it]
 90%|█████████ | 203158/225000 [389:20:29<54:37:25,  9.00s/it]
 90%|█████████ | 203159/225000 [389:20:39<56:29:31,  9.31s/it]
 90%|█████████ | 203160/225000 [389:20:49<56:24:34,  9.30s/it]
 90%|█████████ | 203161/225000 [389:20:58<56:51:56,  9.37s/it]
 90%|█████████ | 203162/225000 [389:21:10<61:48:38, 10.19s/it]
 90%|█████████ | 203163/225000 [389:21:17<55:13:48,  9.11s/it]
 90%|█████████ | 203164/225000 [389:21:22<48:22:29,  7.98s/it]
 90%|█████████ | 203165/225000 [389:21:30<47:35:39,  7.85s/it]
 90%|█████████ | 203166/225000 [389:21:37<46:52:08,  7.73s/it]
 90%|█████████ | 203167/225000 [389:21:53<61:17:56, 10.11s/it]
 90%|█████████ | 203168/225000 [389:21:59<54:20:13,  8.96s/it]
 90%|█████████ | 203169/225000 [389:22:08<53:57:07,  8.90s/it]
 90%|█████████ | 203170/225000 [389:22:17<54:27:06,  8.98s/it]
 90%|█████████ | 203171/225000 [389:22:25<52:31:03,  8.66s/it]
 90%|█████████ | 203172/225000 [389:22:30<46:23:51,  7.65s/it]
 90%|█████████ | 203173/225000 [389:22:36<43:31:46,  7.18s/it]
 90%|█████████ | 203174/225000 [389:22:49<54:15:59,  8.95s/it]
 90%|█████████ | 203175/225000 [389:23:00<56:55:11,  9.39s/it]
 90%|█████████ | 203176/225000 [389:23:04<47:38:06,  7.86s/it]
 90%|█████████ | 203177/225000 [389:23:09<42:49:29,  7.06s/it]
 90%|█████████ | 203178/225000 [389:23:21<50:31:16,  8.33s/it]
 90%|█████████ | 203179/225000 [389:23:31<54:05:01,  8.92s/it]
 90%|█████████ | 203180/225000 [389:23:43<60:08:08,  9.92s/it]
 90%|█████████ | 203181/225000 [389:23:50<55:09:42,  9.10s/it]
 90%|█████████ | 203182/225000 [389:24:00<56:59:19,  9.40s/it]
 90%|█████████ | 203183/225000 [389:24:12<60:56:04, 10.05s/it]
 90%|█████████ | 203184/225000 [389:24:21<59:19:30,  9.79s/it]
 90%|█████████ | 203185/225000 [389:24:31<60:08:12,  9.92s/it]
 90%|█████████ | 203186/225000 [389:24:36<50:58:31,  8.41s/it]
 90%|█████████ | 203187/225000 [389:24:47<55:33:00,  9.17s/it]
 90%|█████████ | 203188/225000 [389:24:55<52:40:49,  8.69s/it]
 90%|█████████ | 203189/225000 [389:25:03<51:06:19,  8.44s/it]
 90%|█████████ | 203190/225000 [389:25:14<56:14:11,  9.28s/it]
 90%|█████████ | 203191/225000 [389:25:20<50:45:52,  8.38s/it]
 90%|█████████ | 203192/225000 [389:25:32<56:10:25,  9.27s/it]
 90%|█████████ | 203193/225000 [389:25:38<50:37:32,  8.36s/it]
 90%|█████████ | 203194/225000 [389:25:45<48:17:34,  7.97s/it]
 90%|█████████ | 203195/225000 [389:25:54<50:04:09,  8.27s/it]
 90%|█████████ | 203196/225000 [389:26:05<54:56:37,  9.07s/it]
 90%|█████████ | 203197/225000 [389:26:09<46:43:53,  7.72s/it]
 90%|█████████ | 203198/225000 [389:26:18<48:49:07,  8.06s/it]
 90%|█████████ | 203199/225000 [389:26:25<46:12:51,  7.63s/it]
 90%|█████████ | 203200/225000 [389:26:37<53:47:53,  8.88s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.197', 'learning_rate': '1.269e-06', 'epoch': '0.9031', 'num_input_tokens_seen': 654760776, 'train_runtime': '1.402e+06', 'train_tokens_per_second': '467'}

 90%|█████████ | 203200/225000 [389:26:37<53:47:53,  8.88s/it]
 90%|█████████ | 203201/225000 [389:26:45<53:42:02,  8.87s/it]
 90%|█████████ | 203202/225000 [389:26:55<54:20:43,  8.98s/it]
 90%|█████████ | 203203/225000 [389:27:01<49:14:43,  8.13s/it]
 90%|█████████ | 203204/225000 [389:27:10<50:27:06,  8.33s/it]
 90%|█████████ | 203205/225000 [389:27:20<53:44:06,  8.88s/it]
 90%|█████████ | 203206/225000 [389:27:30<55:38:58,  9.19s/it]
 90%|█████████ | 203207/225000 [389:27:38<53:09:34,  8.78s/it]
 90%|█████████ | 203208/225000 [389:27:48<55:34:22,  9.18s/it]
 90%|█████████ | 203209/225000 [389:27:54<50:38:50,  8.37s/it]
 90%|█████████ | 203210/225000 [389:28:05<55:32:44,  9.18s/it]
 90%|█████████ | 203211/225000 [389:28:13<53:54:40,  8.91s/it]
 90%|█████████ | 203212/225000 [389:28:23<55:15:51,  9.13s/it]
 90%|█████████ | 203213/225000 [389:28:34<58:22:14,  9.64s/it]
 90%|█████████ | 203214/225000 [389:28:45<60:29:59, 10.00s/it]
 90%|█████████ | 203215/225000 [389:28:53<57:24:28,  9.49s/it]
 90%|█████████ | 203216/225000 [389:29:01<53:49:26,  8.89s/it]
 90%|█████████ | 203217/225000 [389:29:10<54:56:59,  9.08s/it]
 90%|█████████ | 203218/225000 [389:29:18<52:32:57,  8.69s/it]
 90%|█████████ | 203219/225000 [389:29:24<47:25:49,  7.84s/it]
 90%|█████████ | 203220/225000 [389:29:37<57:49:08,  9.56s/it]
 90%|█████████ | 203221/225000 [389:29:46<56:06:16,  9.27s/it]
 90%|█████████ | 203222/225000 [389:29:51<49:01:16,  8.10s/it]
 90%|█████████ | 203223/225000 [389:29:59<48:53:37,  8.08s/it]
 90%|█████████ | 203224/225000 [389:30:05<44:31:37,  7.36s/it]
 90%|█████████ | 203225/225000 [389:30:12<43:21:36,  7.17s/it]
 90%|█████████ | 203226/225000 [389:30:18<41:54:30,  6.93s/it]
 90%|█████████ | 203227/225000 [389:30:32<54:07:46,  8.95s/it]
 90%|█████████ | 203228/225000 [389:30:37<47:24:57,  7.84s/it]
 90%|█████████ | 203229/225000 [389:30:48<52:28:18,  8.68s/it]
 90%|█████████ | 203230/225000 [389:30:55<49:49:16,  8.24s/it]
 90%|█████████ | 203231/225000 [389:31:04<50:42:07,  8.38s/it]
 90%|█████████ | 203232/225000 [389:31:11<48:38:54,  8.05s/it]
 90%|█████████ | 203233/225000 [389:31:16<43:44:17,  7.23s/it]
 90%|█████████ | 203234/225000 [389:31:26<47:54:42,  7.92s/it]
 90%|█████████ | 203235/225000 [389:31:39<56:47:43,  9.39s/it]
 90%|█████████ | 203236/225000 [389:31:47<55:16:09,  9.14s/it]
 90%|█████████ | 203237/225000 [389:31:55<53:14:51,  8.81s/it]
 90%|█████████ | 203238/225000 [389:32:01<48:14:47,  7.98s/it]
 90%|█████████ | 203239/225000 [389:32:08<46:58:00,  7.77s/it]
 90%|█████████ | 203240/225000 [389:32:16<46:35:32,  7.71s/it]
 90%|█████████ | 203241/225000 [389:32:22<44:08:02,  7.30s/it]
 90%|█████████ | 203242/225000 [389:32:36<56:11:55,  9.30s/it]
 90%|█████████ | 203243/225000 [389:32:43<51:55:18,  8.59s/it]
 90%|█████████ | 203244/225000 [389:32:51<50:21:45,  8.33s/it]
 90%|█████████ | 203245/225000 [389:32:59<50:23:40,  8.34s/it]
 90%|█████████ | 203246/225000 [389:33:05<45:59:02,  7.61s/it]
 90%|█████████ | 203247/225000 [389:33:18<54:50:06,  9.07s/it]
 90%|█████████ | 203248/225000 [389:33:26<54:06:07,  8.95s/it]
 90%|█████████ | 203249/225000 [389:33:31<46:50:45,  7.75s/it]
 90%|█████████ | 203250/225000 [389:33:41<49:48:40,  8.24s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.441', 'learning_rate': '1.263e-06', 'epoch': '0.9033', 'num_input_tokens_seen': 654989224, 'train_runtime': '1.402e+06', 'train_tokens_per_second': '467'}

 90%|█████████ | 203250/225000 [389:33:41<49:48:40,  8.24s/it]
 90%|█████████ | 203251/225000 [389:33:48<47:35:43,  7.88s/it]
 90%|█████████ | 203252/225000 [389:33:56<48:46:27,  8.07s/it]
 90%|█████████ | 203253/225000 [389:34:05<49:12:13,  8.15s/it]
 90%|█████████ | 203254/225000 [389:34:13<49:47:37,  8.24s/it]
 90%|█████████ | 203255/225000 [389:34:22<51:38:57,  8.55s/it]
 90%|█████████ | 203256/225000 [389:34:29<48:53:36,  8.09s/it]
 90%|█████████ | 203257/225000 [389:34:36<46:12:23,  7.65s/it]
 90%|█████████ | 203258/225000 [389:34:48<53:29:32,  8.86s/it]
 90%|█████████ | 203259/225000 [389:34:53<47:12:52,  7.82s/it]
 90%|█████████ | 203260/225000 [389:35:05<54:07:32,  8.96s/it]
 90%|█████████ | 203261/225000 [389:35:10<47:13:28,  7.82s/it]
 90%|█████████ | 203262/225000 [389:35:17<45:21:56,  7.51s/it]
 90%|█████████ | 203263/225000 [389:35:29<53:22:41,  8.84s/it]
 90%|█████████ | 203264/225000 [389:35:35<48:47:49,  8.08s/it]
 90%|█████████ | 203265/225000 [389:35:41<44:36:30,  7.39s/it]
 90%|█████████ | 203266/225000 [389:35:52<51:18:20,  8.50s/it]
 90%|█████████ | 203267/225000 [389:35:59<49:32:16,  8.21s/it]
 90%|█████████ | 203268/225000 [389:36:05<44:31:28,  7.38s/it]
 90%|█████████ | 203269/225000 [389:36:15<49:12:32,  8.15s/it]
 90%|█████████ | 203270/225000 [389:36:21<45:59:46,  7.62s/it]
 90%|█████████ | 203271/225000 [389:36:35<56:54:20,  9.43s/it]
 90%|█████████ | 203272/225000 [389:36:41<50:40:32,  8.40s/it]
 90%|█████████ | 203273/225000 [389:36:50<51:35:54,  8.55s/it]
 90%|█████████ | 203274/225000 [389:37:01<56:17:45,  9.33s/it]
 90%|█████████ | 203275/225000 [389:37:11<57:27:27,  9.52s/it]
 90%|█████████ | 203276/225000 [389:37:16<49:38:23,  8.23s/it]
 90%|█████████ | 203277/225000 [389:37:23<47:34:47,  7.89s/it]
 90%|█████████ | 203278/225000 [389:37:33<51:14:44,  8.49s/it]
 90%|█████████ | 203279/225000 [389:37:38<45:28:42,  7.54s/it]
 90%|█████████ | 203280/225000 [389:37:47<48:09:51,  7.98s/it]
 90%|█████████ | 203281/225000 [389:37:55<48:20:25,  8.01s/it]
 90%|█████████ | 203282/225000 [389:38:01<44:51:07,  7.43s/it]
 90%|█████████ | 203283/225000 [389:38:10<47:36:34,  7.89s/it]
 90%|█████████ | 203284/225000 [389:38:17<44:59:40,  7.46s/it]
 90%|█████████ | 203285/225000 [389:38:29<54:19:56,  9.01s/it]
 90%|█████████ | 203286/225000 [389:38:37<52:07:46,  8.64s/it]
 90%|█████████ | 203287/225000 [389:38:46<52:21:18,  8.68s/it]
 90%|█████████ | 203288/225000 [389:38:52<47:22:30,  7.86s/it]
 90%|█████████ | 203289/225000 [389:39:06<58:11:29,  9.65s/it]
 90%|█████████ | 203290/225000 [389:39:12<52:54:18,  8.77s/it]
 90%|█████████ | 203291/225000 [389:39:23<56:03:48,  9.30s/it]
 90%|█████████ | 203292/225000 [389:39:30<50:59:55,  8.46s/it]
 90%|█████████ | 203293/225000 [389:39:37<49:08:50,  8.15s/it]
 90%|█████████ | 203294/225000 [389:39:44<46:42:42,  7.75s/it]
 90%|█████████ | 203295/225000 [389:39:50<44:11:08,  7.33s/it]
 90%|█████████ | 203296/225000 [389:40:02<51:32:36,  8.55s/it]
 90%|█████████ | 203297/225000 [389:40:09<48:54:54,  8.11s/it]
 90%|█████████ | 203298/225000 [389:40:17<48:46:38,  8.09s/it]
 90%|█████████ | 203299/225000 [389:40:29<57:08:18,  9.48s/it]
 90%|█████████ | 203300/225000 [389:40:36<52:01:04,  8.63s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.506', 'learning_rate': '1.257e-06', 'epoch': '0.9036', 'num_input_tokens_seen': 655217368, 'train_runtime': '1.403e+06', 'train_tokens_per_second': '467.1'}

 90%|█████████ | 203300/225000 [389:40:36<52:01:04,  8.63s/it]
 90%|█████████ | 203301/225000 [389:40:42<48:03:51,  7.97s/it]
 90%|█████████ | 203302/225000 [389:40:51<49:41:53,  8.25s/it]
 90%|█████████ | 203303/225000 [389:41:04<58:08:08,  9.65s/it]
 90%|█████████ | 203304/225000 [389:41:11<52:21:36,  8.69s/it]
 90%|█████████ | 203305/225000 [389:41:22<56:10:56,  9.32s/it]
 90%|█████████ | 203306/225000 [389:41:26<47:00:29,  7.80s/it]
 90%|█████████ | 203307/225000 [389:41:33<45:16:08,  7.51s/it]
 90%|█████████ | 203308/225000 [389:41:39<43:19:02,  7.19s/it]
 90%|█████████ | 203309/225000 [389:41:47<44:48:06,  7.44s/it]
 90%|█████████ | 203310/225000 [389:41:56<48:22:51,  8.03s/it]
 90%|█████████ | 203311/225000 [389:42:07<52:59:16,  8.80s/it]
 90%|█████████ | 203312/225000 [389:42:19<58:46:36,  9.76s/it]
 90%|█████████ | 203313/225000 [389:42:32<64:27:40, 10.70s/it]
 90%|█████████ | 203314/225000 [389:42:43<64:38:15, 10.73s/it]
 90%|█████████ | 203315/225000 [389:42:47<53:21:08,  8.86s/it]
 90%|█████████ | 203316/225000 [389:42:58<56:13:33,  9.33s/it]
 90%|█████████ | 203317/225000 [389:43:04<50:50:46,  8.44s/it]
 90%|█████████ | 203318/225000 [389:43:11<48:17:48,  8.02s/it]
 90%|█████████ | 203319/225000 [389:43:19<47:56:32,  7.96s/it]
 90%|█████████ | 203320/225000 [389:43:33<59:15:32,  9.84s/it]
 90%|█████████ | 203321/225000 [389:43:44<60:46:38, 10.09s/it]
 90%|█████████ | 203322/225000 [389:43:48<49:21:43,  8.20s/it]
 90%|█████████ | 203323/225000 [389:44:00<57:08:56,  9.49s/it]
 90%|█████████ | 203324/225000 [389:44:07<52:24:31,  8.70s/it]
 90%|█████████ | 203325/225000 [389:44:12<46:37:57,  7.75s/it]
 90%|█████████ | 203326/225000 [389:44:25<54:42:25,  9.09s/it]
 90%|█████████ | 203327/225000 [389:44:30<48:36:57,  8.08s/it]
 90%|█████████ | 203328/225000 [389:44:38<47:40:41,  7.92s/it]
 90%|█████████ | 203329/225000 [389:44:48<52:10:11,  8.67s/it]
 90%|█████████ | 203330/225000 [389:44:54<47:08:21,  7.83s/it]
 90%|█████████ | 203331/225000 [389:45:04<51:08:33,  8.50s/it]
 90%|█████████ | 203332/225000 [389:45:10<46:09:06,  7.67s/it]
 90%|█████████ | 203333/225000 [389:45:17<45:19:52,  7.53s/it]
 90%|█████████ | 203334/225000 [389:45:29<52:37:23,  8.74s/it]
 90%|█████████ | 203335/225000 [389:45:37<51:23:24,  8.54s/it]
 90%|█████████ | 203336/225000 [389:45:45<50:03:58,  8.32s/it]
 90%|█████████ | 203337/225000 [389:45:51<46:48:32,  7.78s/it]
 90%|█████████ | 203338/225000 [389:46:00<48:45:21,  8.10s/it]
 90%|█████████ | 203339/225000 [389:46:11<53:01:31,  8.81s/it]
 90%|█████████ | 203340/225000 [389:46:24<61:45:38, 10.26s/it]
 90%|█████████ | 203341/225000 [389:46:30<53:52:18,  8.95s/it]
 90%|█████████ | 203342/225000 [389:46:35<47:12:01,  7.85s/it]
 90%|█████████ | 203343/225000 [389:46:42<45:55:45,  7.63s/it]
 90%|█████████ | 203344/225000 [389:46:49<44:08:41,  7.34s/it]
 90%|█████████ | 203345/225000 [389:46:58<46:16:59,  7.69s/it]
 90%|█████████ | 203346/225000 [389:47:11<56:46:19,  9.44s/it]
 90%|█████████ | 203347/225000 [389:47:19<53:40:16,  8.92s/it]
 90%|█████████ | 203348/225000 [389:47:29<55:59:22,  9.31s/it]
 90%|█████████ | 203349/225000 [389:47:33<47:07:06,  7.83s/it]
 90%|█████████ | 203350/225000 [389:47:39<43:36:57,  7.25s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.317', 'learning_rate': '1.251e-06', 'epoch': '0.9038', 'num_input_tokens_seen': 655444864, 'train_runtime': '1.403e+06', 'train_tokens_per_second': '467.1'}

 90%|█████████ | 203350/225000 [389:47:39<43:36:57,  7.25s/it]
 90%|█████████ | 203351/225000 [389:47:51<51:55:50,  8.64s/it]
 90%|█████████ | 203352/225000 [389:48:01<54:17:58,  9.03s/it]
 90%|█████████ | 203353/225000 [389:48:07<48:19:42,  8.04s/it]
 90%|█████████ | 203354/225000 [389:48:13<44:43:41,  7.44s/it]
 90%|█████████ | 203355/225000 [389:48:20<44:05:07,  7.33s/it]
 90%|█████████ | 203356/225000 [389:48:31<50:15:25,  8.36s/it]
 90%|█████████ | 203357/225000 [389:48:45<60:06:18, 10.00s/it]
 90%|█████████ | 203358/225000 [389:48:55<60:35:48, 10.08s/it]
 90%|█████████ | 203359/225000 [389:49:02<54:38:47,  9.09s/it]
 90%|█████████ | 203360/225000 [389:49:08<50:20:43,  8.38s/it]
 90%|█████████ | 203361/225000 [389:49:15<46:30:52,  7.74s/it]
 90%|█████████ | 203362/225000 [389:49:21<44:51:56,  7.46s/it]
 90%|█████████ | 203363/225000 [389:49:32<50:19:56,  8.37s/it]
 90%|█████████ | 203364/225000 [389:49:41<52:19:12,  8.71s/it]
 90%|█████████ | 203365/225000 [389:49:51<53:31:50,  8.91s/it]
 90%|█████████ | 203366/225000 [389:50:01<55:13:51,  9.19s/it]
 90%|█████████ | 203367/225000 [389:50:07<49:44:57,  8.28s/it]
 90%|█████████ | 203368/225000 [389:50:14<48:05:18,  8.00s/it]
 90%|█████████ | 203369/225000 [389:50:20<44:12:45,  7.36s/it]
 90%|█████████ | 203370/225000 [389:50:27<42:47:11,  7.12s/it]
 90%|█████████ | 203371/225000 [389:50:36<47:47:12,  7.95s/it]
 90%|█████████ | 203372/225000 [389:50:46<51:02:05,  8.49s/it]
 90%|█████████ | 203373/225000 [389:50:51<44:14:18,  7.36s/it]
 90%|█████████ | 203374/225000 [389:51:01<48:41:58,  8.11s/it]
 90%|█████████ | 203375/225000 [389:51:09<48:51:59,  8.13s/it]
 90%|█████████ | 203376/225000 [389:51:17<48:02:29,  8.00s/it]
 90%|█████████ | 203377/225000 [389:51:24<47:07:22,  7.85s/it]
 90%|█████████ | 203378/225000 [389:51:36<53:56:36,  8.98s/it]
 90%|█████████ | 203379/225000 [389:51:47<57:05:30,  9.51s/it]
 90%|█████████ | 203380/225000 [389:51:54<52:31:14,  8.75s/it]
 90%|█████████ | 203381/225000 [389:52:07<60:47:28, 10.12s/it]
 90%|█████████ | 203382/225000 [389:52:18<62:40:45, 10.44s/it]
 90%|█████████ | 203383/225000 [389:52:28<61:29:25, 10.24s/it]
 90%|█████████ | 203384/225000 [389:52:38<61:36:57, 10.26s/it]
 90%|█████████ | 203385/225000 [389:52:45<55:06:28,  9.18s/it]
 90%|█████████ | 203386/225000 [389:52:53<53:40:42,  8.94s/it]
 90%|█████████ | 203387/225000 [389:53:01<52:01:23,  8.67s/it]
 90%|█████████ | 203388/225000 [389:53:16<63:22:46, 10.56s/it]
 90%|█████████ | 203389/225000 [389:53:24<59:17:24,  9.88s/it]
 90%|█████████ | 203390/225000 [389:53:33<56:59:38,  9.49s/it]
 90%|█████████ | 203391/225000 [389:53:38<49:26:17,  8.24s/it]
 90%|█████████ | 203392/225000 [389:53:46<48:37:40,  8.10s/it]
 90%|█████████ | 203393/225000 [389:53:53<46:47:38,  7.80s/it]
 90%|█████████ | 203394/225000 [389:53:58<40:56:14,  6.82s/it]
 90%|█████████ | 203395/225000 [389:54:03<38:29:34,  6.41s/it]
 90%|█████████ | 203396/225000 [389:54:15<48:35:26,  8.10s/it]
 90%|█████████ | 203397/225000 [389:54:20<42:15:29,  7.04s/it]
 90%|█████████ | 203398/225000 [389:54:27<43:22:26,  7.23s/it]
 90%|█████████ | 203399/225000 [389:54:36<45:27:51,  7.58s/it]
 90%|█████████ | 203400/225000 [389:54:42<43:31:49,  7.26s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.142', 'learning_rate': '1.246e-06', 'epoch': '0.904', 'num_input_tokens_seen': 655674712, 'train_runtime': '1.404e+06', 'train_tokens_per_second': '467.1'}

 90%|█████████ | 203400/225000 [389:54:42<43:31:49,  7.26s/it]
 90%|█████████ | 203401/225000 [389:54:51<46:38:53,  7.78s/it]
 90%|█████████ | 203402/225000 [389:54:58<45:12:10,  7.53s/it]
 90%|█████████ | 203403/225000 [389:55:07<47:52:21,  7.98s/it]
 90%|█████████ | 203404/225000 [389:55:19<54:13:33,  9.04s/it]
 90%|█████████ | 203405/225000 [389:55:26<50:41:51,  8.45s/it]
 90%|█████████ | 203406/225000 [389:55:34<49:27:43,  8.25s/it]
 90%|█████████ | 203407/225000 [389:55:46<57:25:13,  9.57s/it]
 90%|█████████ | 203408/225000 [389:55:57<58:35:59,  9.77s/it]
 90%|█████████ | 203409/225000 [389:56:06<57:39:19,  9.61s/it]
 90%|█████████ | 203410/225000 [389:56:13<52:54:09,  8.82s/it]
 90%|█████████ | 203411/225000 [389:56:18<46:01:56,  7.68s/it]
 90%|█████████ | 203412/225000 [389:56:26<47:01:45,  7.84s/it]
 90%|█████████ | 203413/225000 [389:56:32<43:50:12,  7.31s/it]
 90%|█████████ | 203414/225000 [389:56:40<45:27:10,  7.58s/it]
 90%|█████████ | 203415/225000 [389:56:55<58:56:35,  9.83s/it]
 90%|█████████ | 203416/225000 [389:57:04<57:31:00,  9.59s/it]
 90%|█████████ | 203417/225000 [389:57:10<51:03:21,  8.52s/it]
 90%|█████████ | 203418/225000 [389:57:20<52:46:02,  8.80s/it]
 90%|█████████ | 203419/225000 [389:57:28<51:15:09,  8.55s/it]
 90%|█████████ | 203420/225000 [389:57:37<53:03:57,  8.85s/it]
 90%|█████████ | 203421/225000 [389:57:44<49:37:16,  8.28s/it]
 90%|█████████ | 203422/225000 [389:57:53<50:55:08,  8.50s/it]
 90%|█████████ | 203423/225000 [389:58:08<61:27:59, 10.26s/it]
 90%|█████████ | 203424/225000 [389:58:18<61:07:01, 10.20s/it]
 90%|█████████ | 203425/225000 [389:58:24<54:21:56,  9.07s/it]
 90%|█████████ | 203426/225000 [389:58:32<52:03:00,  8.69s/it]
 90%|█████████ | 203427/225000 [389:58:39<48:35:15,  8.11s/it]
 90%|█████████ | 203428/225000 [389:58:49<52:37:24,  8.78s/it]
 90%|█████████ | 203429/225000 [389:58:57<51:44:47,  8.64s/it]
 90%|█████████ | 203430/225000 [389:59:08<55:31:07,  9.27s/it]
 90%|█████████ | 203431/225000 [389:59:14<50:11:16,  8.38s/it]
 90%|█████████ | 203432/225000 [389:59:27<56:58:38,  9.51s/it]
 90%|█████████ | 203433/225000 [389:59:38<60:11:49, 10.05s/it]
 90%|█████████ | 203434/225000 [389:59:46<56:06:42,  9.37s/it]
 90%|█████████ | 203435/225000 [389:59:52<51:23:06,  8.58s/it]
 90%|█████████ | 203436/225000 [390:00:00<48:47:03,  8.14s/it]
 90%|█████████ | 203437/225000 [390:00:14<60:40:04, 10.13s/it]
 90%|█████████ | 203438/225000 [390:00:25<61:32:01, 10.27s/it]
 90%|█████████ | 203439/225000 [390:00:30<52:59:44,  8.85s/it]
 90%|█████████ | 203440/225000 [390:00:37<49:32:31,  8.27s/it]
 90%|█████████ | 203441/225000 [390:00:48<53:48:47,  8.99s/it]
 90%|█████████ | 203442/225000 [390:00:58<54:59:14,  9.18s/it]
 90%|█████████ | 203443/225000 [390:01:04<50:16:36,  8.40s/it]
 90%|█████████ | 203444/225000 [390:01:11<47:03:23,  7.86s/it]
 90%|█████████ | 203445/225000 [390:01:19<47:15:27,  7.89s/it]
 90%|█████████ | 203446/225000 [390:01:25<43:53:56,  7.33s/it]
 90%|█████████ | 203447/225000 [390:01:36<51:24:26,  8.59s/it]
 90%|█████████ | 203448/225000 [390:01:46<52:24:37,  8.75s/it]
 90%|█████████ | 203449/225000 [390:01:52<47:33:17,  7.94s/it]
 90%|█████████ | 203450/225000 [390:01:59<46:53:22,  7.83s/it]
                                                              
{'loss': '1.321', 'grad_norm': '0.9681', 'learning_rate': '1.24e-06', 'epoch': '0.9042', 'num_input_tokens_seen': 655911544, 'train_runtime': '1.404e+06', 'train_tokens_per_second': '467.1'}

 90%|█████████ | 203450/225000 [390:01:59<46:53:22,  7.83s/it]
 90%|█████████ | 203451/225000 [390:02:07<47:38:09,  7.96s/it]
 90%|█████████ | 203452/225000 [390:02:13<43:19:53,  7.24s/it]
 90%|█████████ | 203453/225000 [390:02:20<43:41:24,  7.30s/it]
 90%|█████████ | 203454/225000 [390:02:28<43:59:15,  7.35s/it]
 90%|█████████ | 203455/225000 [390:02:42<55:45:44,  9.32s/it]
 90%|█████████ | 203456/225000 [390:02:49<51:44:48,  8.65s/it]
 90%|█████████ | 203457/225000 [390:02:56<48:51:00,  8.16s/it]
 90%|█████████ | 203458/225000 [390:03:06<52:17:37,  8.74s/it]
 90%|█████████ | 203459/225000 [390:03:17<55:39:30,  9.30s/it]
 90%|█████████ | 203460/225000 [390:03:22<48:28:40,  8.10s/it]
 90%|█████████ | 203461/225000 [390:03:33<54:38:30,  9.13s/it]
 90%|█████████ | 203462/225000 [390:03:40<50:25:09,  8.43s/it]
 90%|█████████ | 203463/225000 [390:03:47<47:39:27,  7.97s/it]
 90%|█████████ | 203464/225000 [390:03:58<52:54:42,  8.84s/it]
 90%|█████████ | 203465/225000 [390:04:09<55:59:04,  9.36s/it]
 90%|█████████ | 203466/225000 [390:04:13<47:22:23,  7.92s/it]
 90%|█████████ | 203467/225000 [390:04:23<51:28:54,  8.61s/it]
 90%|█████████ | 203468/225000 [390:04:30<48:27:03,  8.10s/it]
 90%|█████████ | 203469/225000 [390:04:37<46:45:00,  7.82s/it]
 90%|█████████ | 203470/225000 [390:04:52<58:31:54,  9.79s/it]
 90%|█████████ | 203471/225000 [390:04:57<50:09:13,  8.39s/it]
 90%|█████████ | 203472/225000 [390:05:05<50:11:42,  8.39s/it]
 90%|█████████ | 203473/225000 [390:05:11<45:17:20,  7.57s/it]
 90%|█████████ | 203474/225000 [390:05:22<51:08:27,  8.55s/it]
 90%|█████████ | 203475/225000 [390:05:29<48:24:22,  8.10s/it]
 90%|█████████ | 203476/225000 [390:05:35<44:59:28,  7.53s/it]
 90%|█████████ | 203477/225000 [390:05:45<50:11:21,  8.39s/it]
 90%|█████████ | 203478/225000 [390:05:51<45:17:59,  7.58s/it]
 90%|█████████ | 203479/225000 [390:05:59<44:59:05,  7.53s/it]
 90%|█████████ | 203480/225000 [390:06:04<41:50:26,  7.00s/it]
 90%|█████████ | 203481/225000 [390:06:15<48:49:22,  8.17s/it]
 90%|█████████ | 203482/225000 [390:06:22<46:23:20,  7.76s/it]
 90%|█████████ | 203483/225000 [390:06:28<43:15:32,  7.24s/it]
 90%|█████████ | 203484/225000 [390:06:36<44:31:10,  7.45s/it]
 90%|█████████ | 203485/225000 [390:06:42<42:09:16,  7.05s/it]
 90%|█████████ | 203486/225000 [390:06:52<47:05:31,  7.88s/it]
 90%|█████████ | 203487/225000 [390:07:02<50:41:20,  8.48s/it]
 90%|█████████ | 203488/225000 [390:07:14<57:54:46,  9.69s/it]
 90%|█████████ | 203489/225000 [390:07:25<59:33:43,  9.97s/it]
 90%|█████████ | 203490/225000 [390:07:31<53:13:37,  8.91s/it]
 90%|█████████ | 203491/225000 [390:07:41<55:04:12,  9.22s/it]
 90%|█████████ | 203492/225000 [390:07:49<52:59:23,  8.87s/it]
 90%|█████████ | 203493/225000 [390:08:00<55:51:44,  9.35s/it]
 90%|█████████ | 203494/225000 [390:08:08<53:40:00,  8.98s/it]
 90%|█████████ | 203495/225000 [390:08:19<57:11:14,  9.57s/it]
 90%|█████████ | 203496/225000 [390:08:24<49:07:55,  8.23s/it]
 90%|█████████ | 203497/225000 [390:08:34<51:36:54,  8.64s/it]
 90%|█████████ | 203498/225000 [390:08:44<54:21:35,  9.10s/it]
 90%|█████████ | 203499/225000 [390:08:51<50:29:17,  8.45s/it]
 90%|█████████ | 203500/225000 [390:08:59<50:03:52,  8.38s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.148', 'learning_rate': '1.234e-06', 'epoch': '0.9044', 'num_input_tokens_seen': 656139208, 'train_runtime': '1.405e+06', 'train_tokens_per_second': '467.2'}

 90%|█████████ | 203500/225000 [390:08:59<50:03:52,  8.38s/it]
 90%|█████████ | 203501/225000 [390:09:12<58:20:49,  9.77s/it]
 90%|█████████ | 203502/225000 [390:09:16<48:36:04,  8.14s/it]
 90%|█████████ | 203503/225000 [390:09:27<53:18:36,  8.93s/it]
 90%|█████████ | 203504/225000 [390:09:35<51:42:51,  8.66s/it]
 90%|█████████ | 203505/225000 [390:09:44<52:33:07,  8.80s/it]
 90%|█████████ | 203506/225000 [390:09:53<52:25:36,  8.78s/it]
 90%|█████████ | 203507/225000 [390:10:05<57:47:34,  9.68s/it]
 90%|█████████ | 203508/225000 [390:10:13<55:10:08,  9.24s/it]
 90%|█████████ | 203509/225000 [390:10:19<50:14:53,  8.42s/it]
 90%|█████████ | 203510/225000 [390:10:30<53:58:59,  9.04s/it]
 90%|█████████ | 203511/225000 [390:10:36<47:56:51,  8.03s/it]
 90%|█████████ | 203512/225000 [390:10:45<50:52:16,  8.52s/it]
 90%|█████████ | 203513/225000 [390:10:56<54:05:15,  9.06s/it]
 90%|█████████ | 203514/225000 [390:11:02<49:37:13,  8.31s/it]
 90%|█████████ | 203515/225000 [390:11:09<46:07:07,  7.73s/it]
 90%|█████████ | 203516/225000 [390:11:19<50:56:49,  8.54s/it]
 90%|█████████ | 203517/225000 [390:11:25<45:36:13,  7.64s/it]
 90%|█████████ | 203518/225000 [390:11:37<53:27:46,  8.96s/it]
 90%|█████████ | 203519/225000 [390:11:43<49:18:46,  8.26s/it]
 90%|█████████ | 203520/225000 [390:11:50<47:29:07,  7.96s/it]
 90%|█████████ | 203521/225000 [390:11:57<45:48:51,  7.68s/it]
 90%|█████████ | 203522/225000 [390:12:04<44:21:12,  7.43s/it]
 90%|█████████ | 203523/225000 [390:12:18<55:48:02,  9.35s/it]
 90%|█████████ | 203524/225000 [390:12:28<55:47:22,  9.35s/it]
 90%|█████████ | 203525/225000 [390:12:32<46:32:48,  7.80s/it]
 90%|█████████ | 203526/225000 [390:12:44<54:51:35,  9.20s/it]
 90%|█████████ | 203527/225000 [390:12:49<47:33:39,  7.97s/it]
 90%|█████████ | 203528/225000 [390:13:01<54:15:35,  9.10s/it]
 90%|█████████ | 203529/225000 [390:13:07<48:13:13,  8.09s/it]
 90%|█████████ | 203530/225000 [390:13:18<53:36:35,  8.99s/it]
 90%|█████████ | 203531/225000 [390:13:23<46:22:36,  7.78s/it]
 90%|█████████ | 203532/225000 [390:13:28<41:21:24,  6.94s/it]
 90%|█████████ | 203533/225000 [390:13:37<44:42:46,  7.50s/it]
 90%|█████████ | 203534/225000 [390:13:48<52:09:21,  8.75s/it]
 90%|█████████ | 203535/225000 [390:14:00<56:47:19,  9.52s/it]
 90%|█████████ | 203536/225000 [390:14:06<51:04:12,  8.57s/it]
 90%|█████████ | 203537/225000 [390:14:12<47:16:17,  7.93s/it]
 90%|█████████ | 203538/225000 [390:14:19<45:10:45,  7.58s/it]
 90%|█████████ | 203539/225000 [390:14:27<46:29:18,  7.80s/it]
 90%|█████████ | 203540/225000 [390:14:36<48:20:37,  8.11s/it]
 90%|█████████ | 203541/225000 [390:14:45<48:55:54,  8.21s/it]
 90%|█████████ | 203542/225000 [390:14:51<45:24:29,  7.62s/it]
 90%|█████████ | 203543/225000 [390:15:01<49:23:55,  8.29s/it]
 90%|█████████ | 203544/225000 [390:15:07<46:09:47,  7.75s/it]
 90%|█████████ | 203545/225000 [390:15:12<40:55:17,  6.87s/it]
 90%|█████████ | 203546/225000 [390:15:24<49:08:55,  8.25s/it]
 90%|█████████ | 203547/225000 [390:15:32<49:05:33,  8.24s/it]
 90%|█████████ | 203548/225000 [390:15:39<47:23:49,  7.95s/it]
 90%|█████████ | 203549/225000 [390:15:49<50:42:30,  8.51s/it]
 90%|█████████ | 203550/225000 [390:15:55<47:19:04,  7.94s/it]
                                                              
{'loss': '1.284', 'grad_norm': '1.307', 'learning_rate': '1.229e-06', 'epoch': '0.9047', 'num_input_tokens_seen': 656363472, 'train_runtime': '1.405e+06', 'train_tokens_per_second': '467.2'}

 90%|█████████ | 203550/225000 [390:15:55<47:19:04,  7.94s/it]
 90%|█████████ | 203551/225000 [390:16:02<44:04:40,  7.40s/it]
 90%|█████████ | 203552/225000 [390:16:08<42:27:22,  7.13s/it]
 90%|█████████ | 203553/225000 [390:16:19<48:22:37,  8.12s/it]
 90%|█████████ | 203554/225000 [390:16:29<51:45:59,  8.69s/it]
 90%|█████████ | 203555/225000 [390:16:36<49:40:06,  8.34s/it]
 90%|█████████ | 203556/225000 [390:16:45<49:56:46,  8.38s/it]
 90%|█████████ | 203557/225000 [390:16:50<44:05:21,  7.40s/it]
 90%|█████████ | 203558/225000 [390:17:00<49:11:35,  8.26s/it]
 90%|█████████ | 203559/225000 [390:17:06<46:11:46,  7.76s/it]
 90%|█████████ | 203560/225000 [390:17:12<42:24:42,  7.12s/it]
 90%|█████████ | 203561/225000 [390:17:21<44:49:25,  7.53s/it]
 90%|█████████ | 203562/225000 [390:17:35<57:02:59,  9.58s/it]
 90%|█████████ | 203563/225000 [390:17:43<54:14:16,  9.11s/it]
 90%|█████████ | 203564/225000 [390:17:51<52:48:57,  8.87s/it]
 90%|█████████ | 203565/225000 [390:17:59<50:54:49,  8.55s/it]
 90%|█████████ | 203566/225000 [390:18:06<47:49:33,  8.03s/it]
 90%|█████████ | 203567/225000 [390:18:19<57:27:32,  9.65s/it]
 90%|█████████ | 203568/225000 [390:18:31<61:52:19, 10.39s/it]
 90%|█████████ | 203569/225000 [390:18:39<56:46:55,  9.54s/it]
 90%|█████████ | 203570/225000 [390:18:49<57:44:46,  9.70s/it]
 90%|█████████ | 203571/225000 [390:18:55<51:42:57,  8.69s/it]
 90%|█████████ | 203572/225000 [390:19:02<48:40:40,  8.18s/it]
 90%|█████████ | 203573/225000 [390:19:12<51:26:22,  8.64s/it]
 90%|█████████ | 203574/225000 [390:19:23<55:27:37,  9.32s/it]
 90%|█████████ | 203575/225000 [390:19:34<58:23:42,  9.81s/it]
 90%|█████████ | 203576/225000 [390:19:43<57:07:48,  9.60s/it]
 90%|█████████ | 203577/225000 [390:19:50<52:30:03,  8.82s/it]
 90%|█████████ | 203578/225000 [390:19:58<51:24:57,  8.64s/it]
 90%|█████████ | 203579/225000 [390:20:05<47:38:22,  8.01s/it]
 90%|█████████ | 203580/225000 [390:20:17<54:51:22,  9.22s/it]
 90%|█████████ | 203581/225000 [390:20:24<51:52:06,  8.72s/it]
 90%|█████████ | 203582/225000 [390:20:35<55:17:05,  9.29s/it]
 90%|█████████ | 203583/225000 [390:20:47<60:37:18, 10.19s/it]
 90%|█████████ | 203584/225000 [390:20:56<57:38:23,  9.69s/it]
 90%|█████████ | 203585/225000 [390:21:04<54:18:27,  9.13s/it]
 90%|█████████ | 203586/225000 [390:21:16<59:54:08, 10.07s/it]
 90%|█████████ | 203587/225000 [390:21:25<57:49:07,  9.72s/it]
 90%|█████████ | 203588/225000 [390:21:31<51:52:42,  8.72s/it]
 90%|█████████ | 203589/225000 [390:21:42<55:06:53,  9.27s/it]
 90%|█████████ | 203590/225000 [390:21:52<57:21:17,  9.64s/it]
 90%|█████████ | 203591/225000 [390:21:59<52:03:17,  8.75s/it]
 90%|█████████ | 203592/225000 [390:22:06<48:26:02,  8.14s/it]
 90%|█████████ | 203593/225000 [390:22:13<46:31:13,  7.82s/it]
 90%|█████████ | 203594/225000 [390:22:20<46:00:58,  7.74s/it]
 90%|█████████ | 203595/225000 [390:22:35<57:57:46,  9.75s/it]
 90%|█████████ | 203596/225000 [390:22:43<56:03:18,  9.43s/it]
 90%|█████████ | 203597/225000 [390:22:50<51:02:31,  8.59s/it]
 90%|█████████ | 203598/225000 [390:22:58<50:19:22,  8.46s/it]
 90%|█████████ | 203599/225000 [390:23:10<56:37:19,  9.52s/it]
 90%|█████████ | 203600/225000 [390:23:17<51:21:57,  8.64s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.21', 'learning_rate': '1.223e-06', 'epoch': '0.9049', 'num_input_tokens_seen': 656598224, 'train_runtime': '1.405e+06', 'train_tokens_per_second': '467.2'}

 90%|█████████ | 203600/225000 [390:23:17<51:21:57,  8.64s/it]
 90%|█████████ | 203601/225000 [390:23:23<46:39:45,  7.85s/it]
 90%|█████████ | 203602/225000 [390:23:29<43:45:35,  7.36s/it]
 90%|█████████ | 203603/225000 [390:23:39<48:39:24,  8.19s/it]
 90%|█████████ | 203604/225000 [390:23:46<46:15:11,  7.78s/it]
 90%|█████████ | 203605/225000 [390:23:56<51:01:04,  8.58s/it]
 90%|█████████ | 203606/225000 [390:24:02<44:41:03,  7.52s/it]
 90%|█████████ | 203607/225000 [390:24:09<45:19:25,  7.63s/it]
 90%|█████████ | 203608/225000 [390:24:16<44:19:47,  7.46s/it]
 90%|█████████ | 203609/225000 [390:24:29<53:03:41,  8.93s/it]
 90%|█████████ | 203610/225000 [390:24:35<48:27:16,  8.16s/it]
 90%|█████████ | 203611/225000 [390:24:47<54:08:10,  9.11s/it]
 90%|█████████ | 203612/225000 [390:24:52<48:07:52,  8.10s/it]
 90%|█████████ | 203613/225000 [390:25:01<48:48:08,  8.21s/it]
 90%|█████████ | 203614/225000 [390:25:11<52:12:38,  8.79s/it]
 90%|█████████ | 203615/225000 [390:25:16<44:54:33,  7.56s/it]
 90%|█████████ | 203616/225000 [390:25:22<42:43:14,  7.19s/it]
 90%|█████████ | 203617/225000 [390:25:31<46:02:51,  7.75s/it]
 90%|█████████ | 203618/225000 [390:25:40<48:26:18,  8.16s/it]
 90%|█████████ | 203619/225000 [390:25:49<50:02:10,  8.42s/it]
 90%|█████████ | 203620/225000 [390:26:00<54:38:22,  9.20s/it]
 90%|█████████ | 203621/225000 [390:26:11<57:30:04,  9.68s/it]
 90%|█████████ | 203622/225000 [390:26:20<56:34:30,  9.53s/it]
 90%|█████████ | 203623/225000 [390:26:28<52:58:12,  8.92s/it]
 90%|█████████ | 203624/225000 [390:26:35<49:37:00,  8.36s/it]
 90%|█████████ | 203625/225000 [390:26:41<46:33:15,  7.84s/it]
 91%|█████████ | 203626/225000 [390:26:49<46:46:00,  7.88s/it]
 91%|█████████ | 203627/225000 [390:27:01<53:24:17,  9.00s/it]
 91%|█████████ | 203628/225000 [390:27:09<52:49:00,  8.90s/it]
 91%|█████████ | 203629/225000 [390:27:22<59:21:45, 10.00s/it]
 91%|█████████ | 203630/225000 [390:27:29<54:23:40,  9.16s/it]
 91%|█████████ | 203631/225000 [390:27:36<49:36:12,  8.36s/it]
 91%|█████████ | 203632/225000 [390:27:43<47:05:31,  7.93s/it]
 91%|█████████ | 203633/225000 [390:27:50<45:14:04,  7.62s/it]
 91%|█████████ | 203634/225000 [390:27:56<43:47:56,  7.38s/it]
 91%|█████████ | 203635/225000 [390:28:11<55:50:26,  9.41s/it]
 91%|█████████ | 203636/225000 [390:28:16<49:08:01,  8.28s/it]
 91%|█████████ | 203637/225000 [390:28:24<47:56:15,  8.08s/it]
 91%|█████████ | 203638/225000 [390:28:30<44:53:42,  7.57s/it]
 91%|█████████ | 203639/225000 [390:28:42<52:27:03,  8.84s/it]
 91%|█████████ | 203640/225000 [390:28:49<49:50:33,  8.40s/it]
 91%|█████████ | 203641/225000 [390:28:54<43:59:09,  7.41s/it]
 91%|█████████ | 203642/225000 [390:29:00<40:10:37,  6.77s/it]
 91%|█████████ | 203643/225000 [390:29:12<49:30:23,  8.34s/it]
 91%|█████████ | 203644/225000 [390:29:21<51:44:13,  8.72s/it]
 91%|█████████ | 203645/225000 [390:29:29<49:16:34,  8.31s/it]
 91%|█████████ | 203646/225000 [390:29:37<49:12:11,  8.29s/it]
 91%|█████████ | 203647/225000 [390:29:47<51:47:56,  8.73s/it]
 91%|█████████ | 203648/225000 [390:29:56<51:53:51,  8.75s/it]
 91%|█████████ | 203649/225000 [390:30:01<46:24:05,  7.82s/it]
 91%|█████████ | 203650/225000 [390:30:10<48:09:34,  8.12s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.381', 'learning_rate': '1.217e-06', 'epoch': '0.9051', 'num_input_tokens_seen': 656823840, 'train_runtime': '1.406e+06', 'train_tokens_per_second': '467.2'}

 91%|█████████ | 203650/225000 [390:30:10<48:09:34,  8.12s/it]
 91%|█████████ | 203651/225000 [390:30:18<48:41:13,  8.21s/it]
 91%|█████████ | 203652/225000 [390:30:30<54:48:03,  9.24s/it]
 91%|█████████ | 203653/225000 [390:30:37<50:37:28,  8.54s/it]
 91%|█████████ | 203654/225000 [390:30:44<47:31:40,  8.02s/it]
 91%|█████████ | 203655/225000 [390:30:52<48:03:58,  8.11s/it]
 91%|█████████ | 203656/225000 [390:30:56<41:08:49,  6.94s/it]
 91%|█████████ | 203657/225000 [390:31:05<45:04:08,  7.60s/it]
 91%|█████████ | 203658/225000 [390:31:15<47:44:33,  8.05s/it]
 91%|█████████ | 203659/225000 [390:31:24<49:48:12,  8.40s/it]
 91%|█████████ | 203660/225000 [390:31:30<45:19:01,  7.64s/it]
 91%|█████████ | 203661/225000 [390:31:44<56:34:09,  9.54s/it]
 91%|█████████ | 203662/225000 [390:31:52<54:38:13,  9.22s/it]
 91%|█████████ | 203663/225000 [390:32:01<53:44:04,  9.07s/it]
 91%|█████████ | 203664/225000 [390:32:12<57:42:31,  9.74s/it]
 91%|█████████ | 203665/225000 [390:32:23<59:25:17, 10.03s/it]
 91%|█████████ | 203666/225000 [390:32:31<55:52:36,  9.43s/it]
 91%|█████████ | 203667/225000 [390:32:38<51:36:22,  8.71s/it]
 91%|█████████ | 203668/225000 [390:32:45<49:24:49,  8.34s/it]
 91%|█████████ | 203669/225000 [390:32:52<45:38:10,  7.70s/it]
 91%|█████████ | 203670/225000 [390:33:03<51:45:46,  8.74s/it]
 91%|█████████ | 203671/225000 [390:33:11<51:28:23,  8.69s/it]
 91%|█████████ | 203672/225000 [390:33:25<59:43:31, 10.08s/it]
 91%|█████████ | 203673/225000 [390:33:29<49:39:10,  8.38s/it]
 91%|█████████ | 203674/225000 [390:33:40<53:40:58,  9.06s/it]
 91%|█████████ | 203675/225000 [390:33:46<48:12:52,  8.14s/it]
 91%|█████████ | 203676/225000 [390:33:56<52:44:51,  8.91s/it]
 91%|█████████ | 203677/225000 [390:34:01<45:48:56,  7.74s/it]
 91%|█████████ | 203678/225000 [390:34:14<53:59:59,  9.12s/it]
 91%|█████████ | 203679/225000 [390:34:25<58:08:20,  9.82s/it]
 91%|█████████ | 203680/225000 [390:34:30<49:25:33,  8.35s/it]
 91%|█████████ | 203681/225000 [390:34:38<48:19:48,  8.16s/it]
 91%|█████████ | 203682/225000 [390:34:48<52:11:46,  8.81s/it]
 91%|█████████ | 203683/225000 [390:34:56<50:21:33,  8.50s/it]
 91%|█████████ | 203684/225000 [390:35:08<57:08:33,  9.65s/it]
 91%|█████████ | 203685/225000 [390:35:15<52:39:48,  8.89s/it]
 91%|█████████ | 203686/225000 [390:35:24<52:16:35,  8.83s/it]
 91%|█████████ | 203687/225000 [390:35:35<55:11:34,  9.32s/it]
 91%|█████████ | 203688/225000 [390:35:42<51:19:28,  8.67s/it]
 91%|█████████ | 203689/225000 [390:35:48<48:03:12,  8.12s/it]
 91%|█████████ | 203690/225000 [390:35:59<52:08:19,  8.81s/it]
 91%|█████████ | 203691/225000 [390:36:04<45:36:46,  7.71s/it]
 91%|█████████ | 203692/225000 [390:36:10<42:15:19,  7.14s/it]
 91%|█████████ | 203693/225000 [390:36:17<42:29:29,  7.18s/it]
 91%|█████████ | 203694/225000 [390:36:23<39:46:35,  6.72s/it]
 91%|█████████ | 203695/225000 [390:36:35<48:54:01,  8.26s/it]
 91%|█████████ | 203696/225000 [390:36:43<48:18:29,  8.16s/it]
 91%|█████████ | 203697/225000 [390:36:50<46:22:07,  7.84s/it]
 91%|█████████ | 203698/225000 [390:36:56<44:14:11,  7.48s/it]
 91%|█████████ | 203699/225000 [390:37:11<56:19:43,  9.52s/it]
 91%|█████████ | 203700/225000 [390:37:17<51:30:48,  8.71s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.137', 'learning_rate': '1.212e-06', 'epoch': '0.9053', 'num_input_tokens_seen': 657056240, 'train_runtime': '1.406e+06', 'train_tokens_per_second': '467.2'}

 91%|█████████ | 203700/225000 [390:37:17<51:30:48,  8.71s/it]
 91%|█████████ | 203701/225000 [390:37:28<55:10:23,  9.33s/it]
 91%|█████████ | 203702/225000 [390:37:34<49:16:48,  8.33s/it]
 91%|█████████ | 203703/225000 [390:37:44<52:34:10,  8.89s/it]
 91%|█████████ | 203704/225000 [390:37:52<50:35:50,  8.55s/it]
 91%|█████████ | 203705/225000 [390:38:00<49:05:01,  8.30s/it]
 91%|█████████ | 203706/225000 [390:38:14<60:14:53, 10.19s/it]
 91%|█████████ | 203707/225000 [390:38:22<55:51:12,  9.44s/it]
 91%|█████████ | 203708/225000 [390:38:32<56:17:28,  9.52s/it]
 91%|█████████ | 203709/225000 [390:38:40<54:29:08,  9.21s/it]
 91%|█████████ | 203710/225000 [390:38:49<53:27:55,  9.04s/it]
 91%|█████████ | 203711/225000 [390:38:56<49:15:41,  8.33s/it]
 91%|█████████ | 203712/225000 [390:39:05<51:26:02,  8.70s/it]
 91%|█████████ | 203713/225000 [390:39:16<54:40:43,  9.25s/it]
 91%|█████████ | 203714/225000 [390:39:20<46:36:49,  7.88s/it]
 91%|█████████ | 203715/225000 [390:39:30<49:41:57,  8.41s/it]
 91%|█████████ | 203716/225000 [390:39:39<49:50:09,  8.43s/it]
 91%|█████████ | 203717/225000 [390:39:44<45:02:15,  7.62s/it]
 91%|█████████ | 203718/225000 [390:39:51<42:42:31,  7.22s/it]
 91%|█████████ | 203719/225000 [390:39:57<41:01:26,  6.94s/it]
 91%|█████████ | 203720/225000 [390:40:04<42:01:25,  7.11s/it]
 91%|█████████ | 203721/225000 [390:40:11<40:36:26,  6.87s/it]
 91%|█████████ | 203722/225000 [390:40:27<57:05:47,  9.66s/it]
 91%|█████████ | 203723/225000 [390:40:35<54:59:16,  9.30s/it]
 91%|█████████ | 203724/225000 [390:40:43<51:19:51,  8.69s/it]
 91%|█████████ | 203725/225000 [390:40:53<54:20:22,  9.19s/it]
 91%|█████████ | 203726/225000 [390:41:04<57:53:11,  9.80s/it]
 91%|█████████ | 203727/225000 [390:41:15<59:29:16, 10.07s/it]
 91%|█████████ | 203728/225000 [390:41:20<50:50:49,  8.61s/it]
 91%|█████████ | 203729/225000 [390:41:30<53:48:25,  9.11s/it]
 91%|█████████ | 203730/225000 [390:41:41<56:58:28,  9.64s/it]
 91%|█████████ | 203731/225000 [390:41:48<51:29:39,  8.72s/it]
 91%|█████████ | 203732/225000 [390:41:54<47:45:21,  8.08s/it]
 91%|█████████ | 203733/225000 [390:42:05<51:48:51,  8.77s/it]
 91%|█████████ | 203734/225000 [390:42:09<44:04:36,  7.46s/it]
 91%|█████████ | 203735/225000 [390:42:15<41:06:40,  6.96s/it]
 91%|█████████ | 203736/225000 [390:42:25<45:52:57,  7.77s/it]
 91%|█████████ | 203737/225000 [390:42:32<45:51:58,  7.77s/it]
 91%|█████████ | 203738/225000 [390:42:37<40:14:06,  6.81s/it]
 91%|█████████ | 203739/225000 [390:42:48<47:01:03,  7.96s/it]
 91%|█████████ | 203740/225000 [390:42:54<43:46:52,  7.41s/it]
 91%|█████████ | 203741/225000 [390:43:06<53:11:33,  9.01s/it]
 91%|█████████ | 203742/225000 [390:43:12<47:32:21,  8.05s/it]
 91%|█████████ | 203743/225000 [390:43:23<52:20:24,  8.86s/it]
 91%|█████████ | 203744/225000 [390:43:28<45:26:56,  7.70s/it]
 91%|█████████ | 203745/225000 [390:43:38<50:10:23,  8.50s/it]
 91%|█████████ | 203746/225000 [390:43:50<54:57:28,  9.31s/it]
 91%|█████████ | 203747/225000 [390:44:00<56:44:44,  9.61s/it]
 91%|█████████ | 203748/225000 [390:44:07<53:06:51,  9.00s/it]
 91%|█████████ | 203749/225000 [390:44:15<49:53:45,  8.45s/it]
 91%|█████████ | 203750/225000 [390:44:21<46:39:33,  7.90s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.365', 'learning_rate': '1.206e-06', 'epoch': '0.9056', 'num_input_tokens_seen': 657287048, 'train_runtime': '1.407e+06', 'train_tokens_per_second': '467.3'}

 91%|█████████ | 203750/225000 [390:44:21<46:39:33,  7.90s/it]
 91%|█████████ | 203751/225000 [390:44:29<46:29:08,  7.88s/it]
 91%|█████████ | 203752/225000 [390:44:41<54:12:57,  9.19s/it]
 91%|█████████ | 203753/225000 [390:44:48<50:24:21,  8.54s/it]
 91%|█████████ | 203754/225000 [390:45:01<57:21:04,  9.72s/it]
 91%|█████████ | 203755/225000 [390:45:06<48:56:43,  8.29s/it]
 91%|█████████ | 203756/225000 [390:45:15<50:09:10,  8.50s/it]
 91%|█████████ | 203757/225000 [390:45:27<57:34:13,  9.76s/it]
 91%|█████████ | 203758/225000 [390:45:35<54:24:54,  9.22s/it]
 91%|█████████ | 203759/225000 [390:45:42<49:05:34,  8.32s/it]
 91%|█████████ | 203760/225000 [390:45:51<50:23:23,  8.54s/it]
 91%|█████████ | 203761/225000 [390:45:58<48:35:05,  8.24s/it]
 91%|█████████ | 203762/225000 [390:46:03<42:08:41,  7.14s/it]
 91%|█████████ | 203763/225000 [390:46:16<53:09:09,  9.01s/it]
 91%|█████████ | 203764/225000 [390:46:23<50:12:51,  8.51s/it]
 91%|█████████ | 203765/225000 [390:46:32<50:24:26,  8.55s/it]
 91%|█████████ | 203766/225000 [390:46:40<48:58:56,  8.30s/it]
 91%|█████████ | 203767/225000 [390:46:48<49:31:48,  8.40s/it]
 91%|█████████ | 203768/225000 [390:46:56<47:19:24,  8.02s/it]
 91%|█████████ | 203769/225000 [390:47:04<48:12:18,  8.17s/it]
 91%|█████████ | 203770/225000 [390:47:15<52:08:05,  8.84s/it]
 91%|█████████ | 203771/225000 [390:47:20<46:31:04,  7.89s/it]
 91%|█████████ | 203772/225000 [390:47:25<40:45:23,  6.91s/it]
 91%|█████████ | 203773/225000 [390:47:31<38:35:49,  6.55s/it]
 91%|█████████ | 203774/225000 [390:47:48<58:01:48,  9.84s/it]
 91%|█████████ | 203775/225000 [390:47:56<55:23:40,  9.40s/it]
 91%|█████████ | 203776/225000 [390:48:03<49:37:55,  8.42s/it]
 91%|█████████ | 203777/225000 [390:48:08<43:55:33,  7.45s/it]
 91%|█████████ | 203778/225000 [390:48:21<53:37:13,  9.10s/it]
 91%|█████████ | 203779/225000 [390:48:26<46:41:58,  7.92s/it]
 91%|█████████ | 203780/225000 [390:48:32<43:08:08,  7.32s/it]
 91%|█████████ | 203781/225000 [390:48:38<40:47:34,  6.92s/it]
 91%|█████████ | 203782/225000 [390:48:51<52:40:27,  8.94s/it]
 91%|█████████ | 203783/225000 [390:49:02<56:02:50,  9.51s/it]
 91%|█████████ | 203784/225000 [390:49:08<49:32:43,  8.41s/it]
 91%|█████████ | 203785/225000 [390:49:14<45:14:25,  7.68s/it]
 91%|█████████ | 203786/225000 [390:49:27<54:01:43,  9.17s/it]
 91%|█████████ | 203787/225000 [390:49:37<56:18:47,  9.56s/it]
 91%|█████████ | 203788/225000 [390:49:44<50:42:59,  8.61s/it]
 91%|█████████ | 203789/225000 [390:49:59<62:43:01, 10.64s/it]
 91%|█████████ | 203790/225000 [390:50:04<53:10:59,  9.03s/it]
 91%|█████████ | 203791/225000 [390:50:11<48:22:48,  8.21s/it]
 91%|█████████ | 203792/225000 [390:50:18<47:02:38,  7.99s/it]
 91%|█████████ | 203793/225000 [390:50:32<57:29:13,  9.76s/it]
 91%|█████████ | 203794/225000 [390:50:38<51:07:41,  8.68s/it]
 91%|█████████ | 203795/225000 [390:50:49<55:54:15,  9.49s/it]
 91%|█████████ | 203796/225000 [390:50:56<50:48:04,  8.63s/it]
 91%|█████████ | 203797/225000 [390:51:03<47:33:12,  8.07s/it]
 91%|█████████ | 203798/225000 [390:51:12<49:50:54,  8.46s/it]
 91%|█████████ | 203799/225000 [390:51:18<45:19:10,  7.70s/it]
 91%|█████████ | 203800/225000 [390:51:30<52:41:44,  8.95s/it]
                                                              
{'loss': '1.298', 'grad_norm': '0.9385', 'learning_rate': '1.201e-06', 'epoch': '0.9058', 'num_input_tokens_seen': 657515736, 'train_runtime': '1.407e+06', 'train_tokens_per_second': '467.3'}

 91%|█████████ | 203800/225000 [390:51:30<52:41:44,  8.95s/it]
 91%|█████████ | 203801/225000 [390:51:45<62:39:12, 10.64s/it]
 91%|█████████ | 203802/225000 [390:51:56<63:58:34, 10.86s/it]
 91%|█████████ | 203803/225000 [390:52:03<57:27:19,  9.76s/it]
 91%|█████████ | 203804/225000 [390:52:10<51:49:37,  8.80s/it]
 91%|█████████ | 203805/225000 [390:52:21<56:00:15,  9.51s/it]
 91%|█████████ | 203806/225000 [390:52:28<51:18:28,  8.72s/it]
 91%|█████████ | 203807/225000 [390:52:34<46:40:18,  7.93s/it]
 91%|█████████ | 203808/225000 [390:52:40<43:54:40,  7.46s/it]
 91%|█████████ | 203809/225000 [390:52:57<60:43:58, 10.32s/it]
 91%|█████████ | 203810/225000 [390:53:03<52:05:42,  8.85s/it]
 91%|█████████ | 203811/225000 [390:53:08<46:29:43,  7.90s/it]
 91%|█████████ | 203812/225000 [390:53:14<42:36:59,  7.24s/it]
 91%|█████████ | 203813/225000 [390:53:25<49:55:25,  8.48s/it]
 91%|█████████ | 203814/225000 [390:53:36<53:01:16,  9.01s/it]
 91%|█████████ | 203815/225000 [390:53:42<48:00:21,  8.16s/it]
 91%|█████████ | 203816/225000 [390:53:50<48:22:50,  8.22s/it]
 91%|█████████ | 203817/225000 [390:54:00<51:41:48,  8.79s/it]
 91%|█████████ | 203818/225000 [390:54:08<49:33:47,  8.42s/it]
 91%|█████████ | 203819/225000 [390:54:14<45:21:05,  7.71s/it]
 91%|█████████ | 203820/225000 [390:54:24<50:11:00,  8.53s/it]
 91%|█████████ | 203821/225000 [390:54:29<43:50:53,  7.45s/it]
 91%|█████████ | 203822/225000 [390:54:39<47:45:56,  8.12s/it]
 91%|█████████ | 203823/225000 [390:54:45<43:22:07,  7.37s/it]
 91%|█████████ | 203824/225000 [390:54:50<40:28:29,  6.88s/it]
 91%|█████████ | 203825/225000 [390:55:00<46:04:02,  7.83s/it]
 91%|█████████ | 203826/225000 [390:55:08<45:56:57,  7.81s/it]
 91%|█████████ | 203827/225000 [390:55:18<48:55:51,  8.32s/it]
 91%|█████████ | 203828/225000 [390:55:26<49:42:04,  8.45s/it]
 91%|█████████ | 203829/225000 [390:55:39<57:29:40,  9.78s/it]
 91%|█████████ | 203830/225000 [390:55:49<57:10:39,  9.72s/it]
 91%|█████████ | 203831/225000 [390:55:56<52:19:54,  8.90s/it]
 91%|█████████ | 203832/225000 [390:56:07<56:36:12,  9.63s/it]
 91%|█████████ | 203833/225000 [390:56:14<50:51:20,  8.65s/it]
 91%|█████████ | 203834/225000 [390:56:24<54:17:09,  9.23s/it]
 91%|█████████ | 203835/225000 [390:56:31<49:53:42,  8.49s/it]
 91%|█████████ | 203836/225000 [390:56:40<51:08:58,  8.70s/it]
 91%|█████████ | 203837/225000 [390:56:50<52:43:52,  8.97s/it]
 91%|█████████ | 203838/225000 [390:56:58<52:05:18,  8.86s/it]
 91%|█████████ | 203839/225000 [390:57:04<46:38:38,  7.94s/it]
 91%|█████████ | 203840/225000 [390:57:10<43:19:57,  7.37s/it]
 91%|█████████ | 203841/225000 [390:57:21<50:05:25,  8.52s/it]
 91%|█████████ | 203842/225000 [390:57:28<47:25:26,  8.07s/it]
 91%|█████████ | 203843/225000 [390:57:39<51:32:51,  8.77s/it]
 91%|█████████ | 203844/225000 [390:57:44<45:22:19,  7.72s/it]
 91%|█████████ | 203845/225000 [390:57:53<47:26:11,  8.07s/it]
 91%|█████████ | 203846/225000 [390:57:59<43:10:54,  7.35s/it]
 91%|█████████ | 203847/225000 [390:58:07<44:27:49,  7.57s/it]
 91%|█████████ | 203848/225000 [390:58:12<41:21:42,  7.04s/it]
 91%|█████████ | 203849/225000 [390:58:25<50:36:59,  8.62s/it]
 91%|█████████ | 203850/225000 [390:58:32<48:07:26,  8.19s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.099', 'learning_rate': '1.195e-06', 'epoch': '0.906', 'num_input_tokens_seen': 657743544, 'train_runtime': '1.408e+06', 'train_tokens_per_second': '467.3'}

 91%|█████████ | 203850/225000 [390:58:32<48:07:26,  8.19s/it]
 91%|█████████ | 203851/225000 [390:58:43<53:00:12,  9.02s/it]
 91%|█████████ | 203852/225000 [390:58:50<49:32:14,  8.43s/it]
 91%|█████████ | 203853/225000 [390:59:04<59:27:54, 10.12s/it]
 91%|█████████ | 203854/225000 [390:59:12<55:48:54,  9.50s/it]
 91%|█████████ | 203855/225000 [390:59:22<57:22:08,  9.77s/it]
 91%|█████████ | 203856/225000 [390:59:30<53:02:36,  9.03s/it]
 91%|█████████ | 203857/225000 [390:59:39<54:13:10,  9.23s/it]
 91%|█████████ | 203858/225000 [390:59:46<50:15:07,  8.56s/it]
 91%|█████████ | 203859/225000 [390:59:52<44:52:24,  7.64s/it]
 91%|█████████ | 203860/225000 [391:00:06<56:09:21,  9.56s/it]
 91%|█████████ | 203861/225000 [391:00:14<53:05:25,  9.04s/it]
 91%|█████████ | 203862/225000 [391:00:19<46:42:00,  7.95s/it]
 91%|█████████ | 203863/225000 [391:00:27<47:09:10,  8.03s/it]
 91%|█████████ | 203864/225000 [391:00:39<52:47:34,  8.99s/it]
 91%|█████████ | 203865/225000 [391:00:46<50:04:55,  8.53s/it]
 91%|█████████ | 203866/225000 [391:00:57<53:29:13,  9.11s/it]
 91%|█████████ | 203867/225000 [391:01:03<47:55:04,  8.16s/it]
 91%|█████████ | 203868/225000 [391:01:07<41:16:00,  7.03s/it]
 91%|█████████ | 203869/225000 [391:01:18<47:58:30,  8.17s/it]
 91%|█████████ | 203870/225000 [391:01:22<41:25:28,  7.06s/it]
 91%|█████████ | 203871/225000 [391:01:29<40:59:24,  6.98s/it]
 91%|█████████ | 203872/225000 [391:01:43<52:28:54,  8.94s/it]
 91%|█████████ | 203873/225000 [391:01:49<48:54:04,  8.33s/it]
 91%|█████████ | 203874/225000 [391:01:58<49:03:30,  8.36s/it]
 91%|█████████ | 203875/225000 [391:02:03<43:30:44,  7.42s/it]
 91%|█████████ | 203876/225000 [391:02:14<49:53:51,  8.50s/it]
 91%|█████████ | 203877/225000 [391:02:19<44:14:39,  7.54s/it]
 91%|█████████ | 203878/225000 [391:02:26<42:06:32,  7.18s/it]
 91%|█████████ | 203879/225000 [391:02:43<60:15:00, 10.27s/it]
 91%|█████████ | 203880/225000 [391:02:53<59:09:02, 10.08s/it]
 91%|█████████ | 203881/225000 [391:03:00<53:54:25,  9.19s/it]
 91%|█████████ | 203882/225000 [391:03:07<50:44:00,  8.65s/it]
 91%|█████████ | 203883/225000 [391:03:12<43:54:28,  7.49s/it]
 91%|█████████ | 203884/225000 [391:03:23<49:54:04,  8.51s/it]
 91%|█████████ | 203885/225000 [391:03:28<44:20:01,  7.56s/it]
 91%|█████████ | 203886/225000 [391:03:39<50:34:04,  8.62s/it]
 91%|█████████ | 203887/225000 [391:03:48<50:15:03,  8.57s/it]
 91%|█████████ | 203888/225000 [391:03:56<49:24:59,  8.43s/it]
 91%|█████████ | 203889/225000 [391:04:03<46:17:57,  7.90s/it]
 91%|█████████ | 203890/225000 [391:04:09<43:51:35,  7.48s/it]
 91%|█████████ | 203891/225000 [391:04:17<44:28:39,  7.59s/it]
 91%|█████████ | 203892/225000 [391:04:33<59:25:05, 10.13s/it]
 91%|█████████ | 203893/225000 [391:04:41<56:19:31,  9.61s/it]
 91%|█████████ | 203894/225000 [391:04:52<58:30:07,  9.98s/it]
 91%|█████████ | 203895/225000 [391:05:01<56:03:31,  9.56s/it]
 91%|█████████ | 203896/225000 [391:05:08<51:24:08,  8.77s/it]
 91%|█████████ | 203897/225000 [391:05:20<57:48:33,  9.86s/it]
 91%|█████████ | 203898/225000 [391:05:28<53:56:54,  9.20s/it]
 91%|█████████ | 203899/225000 [391:05:32<45:10:07,  7.71s/it]
 91%|█████████ | 203900/225000 [391:05:40<44:50:52,  7.65s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.34', 'learning_rate': '1.189e-06', 'epoch': '0.9062', 'num_input_tokens_seen': 657972184, 'train_runtime': '1.408e+06', 'train_tokens_per_second': '467.3'}

 91%|█████████ | 203900/225000 [391:05:40<44:50:52,  7.65s/it]
 91%|█████████ | 203901/225000 [391:05:47<44:16:34,  7.55s/it]
 91%|█████████ | 203902/225000 [391:05:52<39:32:35,  6.75s/it]
 91%|█████████ | 203903/225000 [391:06:07<53:38:35,  9.15s/it]
 91%|█████████ | 203904/225000 [391:06:16<53:48:38,  9.18s/it]
 91%|█████████ | 203905/225000 [391:06:20<45:29:17,  7.76s/it]
 91%|█████████ | 203906/225000 [391:06:26<42:38:18,  7.28s/it]
 91%|█████████ | 203907/225000 [391:06:33<40:47:29,  6.96s/it]
 91%|█████████ | 203908/225000 [391:06:39<39:47:08,  6.79s/it]
 91%|█████████ | 203909/225000 [391:06:46<40:06:43,  6.85s/it]
 91%|█████████ | 203910/225000 [391:06:56<45:31:18,  7.77s/it]
 91%|█████████ | 203911/225000 [391:07:06<48:43:56,  8.32s/it]
 91%|█████████ | 203912/225000 [391:07:11<42:47:45,  7.31s/it]
 91%|█████████ | 203913/225000 [391:07:23<52:20:43,  8.94s/it]
 91%|█████████ | 203914/225000 [391:07:28<44:19:43,  7.57s/it]
 91%|█████████ | 203915/225000 [391:07:36<45:48:38,  7.82s/it]
 91%|█████████ | 203916/225000 [391:07:52<60:34:51, 10.34s/it]
 91%|█████████ | 203917/225000 [391:07:59<54:55:50,  9.38s/it]
 91%|█████████ | 203918/225000 [391:08:07<51:28:15,  8.79s/it]
 91%|█████████ | 203919/225000 [391:08:16<51:44:36,  8.84s/it]
 91%|█████████ | 203920/225000 [391:08:23<48:12:06,  8.23s/it]
 91%|█████████ | 203921/225000 [391:08:29<45:32:12,  7.78s/it]
 91%|█████████ | 203922/225000 [391:08:35<42:20:53,  7.23s/it]
 91%|█████████ | 203923/225000 [391:08:49<52:59:28,  9.05s/it]
 91%|█████████ | 203924/225000 [391:08:57<52:20:21,  8.94s/it]
 91%|█████████ | 203925/225000 [391:09:04<48:02:25,  8.21s/it]
 91%|█████████ | 203926/225000 [391:09:10<44:30:03,  7.60s/it]
 91%|█████████ | 203927/225000 [391:09:20<48:59:36,  8.37s/it]
 91%|█████████ | 203928/225000 [391:09:26<45:31:37,  7.78s/it]
 91%|█████████ | 203929/225000 [391:09:33<43:39:39,  7.46s/it]
 91%|█████████ | 203930/225000 [391:09:44<49:09:48,  8.40s/it]
 91%|█████████ | 203931/225000 [391:09:52<49:10:19,  8.40s/it]
 91%|█████████ | 203932/225000 [391:10:03<54:11:09,  9.26s/it]
 91%|█████████ | 203933/225000 [391:10:08<46:34:08,  7.96s/it]
 91%|█████████ | 203934/225000 [391:10:18<48:59:04,  8.37s/it]
 91%|█████████ | 203935/225000 [391:10:22<42:25:51,  7.25s/it]
 91%|█████████ | 203936/225000 [391:10:29<40:51:41,  6.98s/it]
 91%|█████████ | 203937/225000 [391:10:40<47:46:32,  8.17s/it]
 91%|█████████ | 203938/225000 [391:10:45<42:42:12,  7.30s/it]
 91%|█████████ | 203939/225000 [391:10:56<49:08:46,  8.40s/it]
 91%|█████████ | 203940/225000 [391:11:02<45:04:10,  7.70s/it]
 91%|█████████ | 203941/225000 [391:11:17<57:22:03,  9.81s/it]
 91%|█████████ | 203942/225000 [391:11:28<59:53:04, 10.24s/it]
 91%|█████████ | 203943/225000 [391:11:34<51:57:04,  8.88s/it]
 91%|█████████ | 203944/225000 [391:11:40<47:43:22,  8.16s/it]
 91%|█████████ | 203945/225000 [391:11:46<43:43:27,  7.48s/it]
 91%|█████████ | 203946/225000 [391:11:58<51:13:11,  8.76s/it]
 91%|█████████ | 203947/225000 [391:12:04<46:48:51,  8.01s/it]
 91%|█████████ | 203948/225000 [391:12:13<47:48:28,  8.18s/it]
 91%|█████████ | 203949/225000 [391:12:26<56:51:54,  9.72s/it]
 91%|█████████ | 203950/225000 [391:12:34<54:30:04,  9.32s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.53', 'learning_rate': '1.184e-06', 'epoch': '0.9064', 'num_input_tokens_seen': 658195616, 'train_runtime': '1.408e+06', 'train_tokens_per_second': '467.4'}

 91%|█████████ | 203950/225000 [391:12:34<54:30:04,  9.32s/it]
 91%|█████████ | 203951/225000 [391:12:41<49:04:28,  8.39s/it]
 91%|█████████ | 203952/225000 [391:12:52<54:11:45,  9.27s/it]
 91%|█████████ | 203953/225000 [391:12:59<50:38:31,  8.66s/it]
 91%|█████████ | 203954/225000 [391:13:10<55:19:54,  9.46s/it]
 91%|█████████ | 203955/225000 [391:13:16<47:51:45,  8.19s/it]
 91%|█████████ | 203956/225000 [391:13:24<48:50:39,  8.36s/it]
 91%|█████████ | 203957/225000 [391:13:36<54:31:57,  9.33s/it]
 91%|█████████ | 203958/225000 [391:13:44<52:41:50,  9.02s/it]
 91%|█████████ | 203959/225000 [391:13:51<48:25:00,  8.28s/it]
 91%|█████████ | 203960/225000 [391:14:00<49:18:02,  8.44s/it]
 91%|█████████ | 203961/225000 [391:14:09<50:32:22,  8.65s/it]
 91%|█████████ | 203962/225000 [391:14:16<47:58:29,  8.21s/it]
 91%|█████████ | 203963/225000 [391:14:23<46:28:24,  7.95s/it]
 91%|█████████ | 203964/225000 [391:14:32<47:27:50,  8.12s/it]
 91%|█████████ | 203965/225000 [391:14:42<50:55:37,  8.72s/it]
 91%|█████████ | 203966/225000 [391:14:46<43:21:11,  7.42s/it]
 91%|█████████ | 203967/225000 [391:14:56<46:57:54,  8.04s/it]
 91%|█████████ | 203968/225000 [391:15:02<43:36:20,  7.46s/it]
 91%|█████████ | 203969/225000 [391:15:13<50:48:09,  8.70s/it]
 91%|█████████ | 203970/225000 [391:15:20<47:01:32,  8.05s/it]
 91%|█████████ | 203971/225000 [391:15:33<54:58:00,  9.41s/it]
 91%|█████████ | 203972/225000 [391:15:39<48:56:10,  8.38s/it]
 91%|█████████ | 203973/225000 [391:15:45<45:59:05,  7.87s/it]
 91%|█████████ | 203974/225000 [391:15:51<41:55:50,  7.18s/it]
 91%|█████████ | 203975/225000 [391:16:03<51:16:36,  8.78s/it]
 91%|█████████ | 203976/225000 [391:16:15<56:01:10,  9.59s/it]
 91%|█████████ | 203977/225000 [391:16:23<53:06:45,  9.10s/it]
 91%|█████████ | 203978/225000 [391:16:28<45:42:00,  7.83s/it]
 91%|█████████ | 203979/225000 [391:16:39<51:31:45,  8.82s/it]
 91%|█████████ | 203980/225000 [391:16:48<51:23:45,  8.80s/it]
 91%|█████████ | 203981/225000 [391:16:53<44:53:51,  7.69s/it]
 91%|█████████ | 203982/225000 [391:17:02<47:31:18,  8.14s/it]
 91%|█████████ | 203983/225000 [391:17:09<46:40:04,  7.99s/it]
 91%|█████████ | 203984/225000 [391:17:17<45:01:27,  7.71s/it]
 91%|█████████ | 203985/225000 [391:17:29<53:16:30,  9.13s/it]
 91%|█████████ | 203986/225000 [391:17:37<51:10:06,  8.77s/it]
 91%|█████████ | 203987/225000 [391:17:45<50:05:09,  8.58s/it]
 91%|█████████ | 203988/225000 [391:17:53<48:37:18,  8.33s/it]
 91%|█████████ | 203989/225000 [391:18:00<46:51:48,  8.03s/it]
 91%|█████████ | 203990/225000 [391:18:07<44:26:14,  7.61s/it]
 91%|█████████ | 203991/225000 [391:18:18<50:00:30,  8.57s/it]
 91%|█████████ | 203992/225000 [391:18:25<48:37:30,  8.33s/it]
 91%|█████████ | 203993/225000 [391:18:36<52:11:52,  8.95s/it]
 91%|█████████ | 203994/225000 [391:18:43<49:08:47,  8.42s/it]
 91%|█████████ | 203995/225000 [391:18:55<55:45:27,  9.56s/it]
 91%|█████████ | 203996/225000 [391:19:05<55:27:51,  9.51s/it]
 91%|█████████ | 203997/225000 [391:19:14<55:26:12,  9.50s/it]
 91%|█████████ | 203998/225000 [391:19:22<53:08:17,  9.11s/it]
 91%|█████████ | 203999/225000 [391:19:30<51:00:50,  8.74s/it]
 91%|█████████ | 204000/225000 [391:19:40<53:45:49,  9.22s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.173', 'learning_rate': '1.178e-06', 'epoch': '0.9067', 'num_input_tokens_seen': 658427536, 'train_runtime': '1.409e+06', 'train_tokens_per_second': '467.4'}

 91%|█████████ | 204000/225000 [391:19:40<53:45:49,  9.22s/it]
 91%|█████████ | 204001/225000 [391:19:45<45:52:26,  7.86s/it]
 91%|█████████ | 204002/225000 [391:19:52<44:33:01,  7.64s/it]
 91%|█████████ | 204003/225000 [391:20:03<50:17:20,  8.62s/it]
 91%|█████████ | 204004/225000 [391:20:09<45:14:14,  7.76s/it]
 91%|█████████ | 204005/225000 [391:20:15<42:05:17,  7.22s/it]
 91%|█████████ | 204006/225000 [391:20:24<46:07:21,  7.91s/it]
 91%|█████████ | 204007/225000 [391:20:30<42:58:15,  7.37s/it]
 91%|█████████ | 204008/225000 [391:20:47<58:57:02, 10.11s/it]
 91%|█████████ | 204009/225000 [391:20:52<49:45:10,  8.53s/it]
 91%|█████████ | 204010/225000 [391:21:00<48:45:53,  8.36s/it]
 91%|█████████ | 204011/225000 [391:21:05<43:29:14,  7.46s/it]
 91%|█████████ | 204012/225000 [391:21:17<50:32:54,  8.67s/it]
 91%|█████████ | 204013/225000 [391:21:27<52:57:58,  9.09s/it]
 91%|█████████ | 204014/225000 [391:21:34<50:00:25,  8.58s/it]
 91%|█████████ | 204015/225000 [391:21:43<50:58:58,  8.75s/it]
 91%|█████████ | 204016/225000 [391:21:56<57:38:57,  9.89s/it]
 91%|█████████ | 204017/225000 [391:22:03<53:31:36,  9.18s/it]
 91%|█████████ | 204018/225000 [391:22:11<50:01:09,  8.58s/it]
 91%|█████████ | 204019/225000 [391:22:20<52:25:27,  9.00s/it]
 91%|█████████ | 204020/225000 [391:22:28<50:39:56,  8.69s/it]
 91%|█████████ | 204021/225000 [391:22:37<50:40:55,  8.70s/it]
 91%|█████████ | 204022/225000 [391:22:44<48:14:37,  8.28s/it]
 91%|█████████ | 204023/225000 [391:22:54<50:51:22,  8.73s/it]
 91%|█████████ | 204024/225000 [391:23:00<45:38:42,  7.83s/it]
 91%|█████████ | 204025/225000 [391:23:07<43:36:27,  7.48s/it]
 91%|█████████ | 204026/225000 [391:23:19<52:38:57,  9.04s/it]
 91%|█████████ | 204027/225000 [391:23:27<50:55:28,  8.74s/it]
 91%|█████████ | 204028/225000 [391:23:37<52:21:42,  8.99s/it]
 91%|█████████ | 204029/225000 [391:23:46<52:42:40,  9.05s/it]
 91%|█████████ | 204030/225000 [391:23:54<49:53:21,  8.56s/it]
 91%|█████████ | 204031/225000 [391:24:04<53:26:00,  9.17s/it]
 91%|█████████ | 204032/225000 [391:24:16<57:20:13,  9.84s/it]
 91%|█████████ | 204033/225000 [391:24:24<55:43:09,  9.57s/it]
 91%|█████████ | 204034/225000 [391:24:30<48:20:04,  8.30s/it]
 91%|█████████ | 204035/225000 [391:24:42<55:03:34,  9.45s/it]
 91%|█████████ | 204036/225000 [391:24:52<55:55:18,  9.60s/it]
 91%|█████████ | 204037/225000 [391:25:01<55:30:58,  9.53s/it]
 91%|█████████ | 204038/225000 [391:25:09<51:51:58,  8.91s/it]
 91%|█████████ | 204039/225000 [391:25:17<49:57:17,  8.58s/it]
 91%|█████████ | 204040/225000 [391:25:28<55:21:43,  9.51s/it]
 91%|█████████ | 204041/225000 [391:25:40<58:57:46, 10.13s/it]
 91%|█████████ | 204042/225000 [391:25:48<56:21:46,  9.68s/it]
 91%|█████████ | 204043/225000 [391:25:57<55:01:53,  9.45s/it]
 91%|█████████ | 204044/225000 [391:26:03<48:46:55,  8.38s/it]
 91%|█████████ | 204045/225000 [391:26:09<43:21:42,  7.45s/it]
 91%|█████████ | 204046/225000 [391:26:20<50:08:54,  8.62s/it]
 91%|█████████ | 204047/225000 [391:26:28<49:47:23,  8.55s/it]
 91%|█████████ | 204048/225000 [391:26:34<44:58:14,  7.73s/it]
 91%|█████████ | 204049/225000 [391:26:43<47:25:36,  8.15s/it]
 91%|█████████ | 204050/225000 [391:26:51<46:49:50,  8.05s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.514', 'learning_rate': '1.173e-06', 'epoch': '0.9069', 'num_input_tokens_seen': 658659424, 'train_runtime': '1.409e+06', 'train_tokens_per_second': '467.4'}

 91%|█████████ | 204050/225000 [391:26:51<46:49:50,  8.05s/it]
 91%|█████████ | 204051/225000 [391:26:58<44:12:35,  7.60s/it]
 91%|█████████ | 204052/225000 [391:27:10<51:55:28,  8.92s/it]
 91%|█████████ | 204053/225000 [391:27:15<46:23:24,  7.97s/it]
 91%|█████████ | 204054/225000 [391:27:22<44:37:10,  7.67s/it]
 91%|█████████ | 204055/225000 [391:27:30<45:28:46,  7.82s/it]
 91%|█████████ | 204056/225000 [391:27:44<54:39:59,  9.40s/it]
 91%|█████████ | 204057/225000 [391:27:49<47:42:20,  8.20s/it]
 91%|█████████ | 204058/225000 [391:28:00<51:53:04,  8.92s/it]
 91%|█████████ | 204059/225000 [391:28:08<50:31:05,  8.68s/it]
 91%|█████████ | 204060/225000 [391:28:15<47:58:48,  8.25s/it]
 91%|█████████ | 204061/225000 [391:28:24<48:49:52,  8.40s/it]
 91%|█████████ | 204062/225000 [391:28:31<47:45:49,  8.21s/it]
 91%|█████████ | 204063/225000 [391:28:38<45:24:48,  7.81s/it]
 91%|█████████ | 204064/225000 [391:28:45<43:12:49,  7.43s/it]
 91%|█████████ | 204065/225000 [391:28:51<41:05:54,  7.07s/it]
 91%|█████████ | 204066/225000 [391:29:04<51:35:10,  8.87s/it]
 91%|█████████ | 204067/225000 [391:29:11<48:16:48,  8.30s/it]
 91%|█████████ | 204068/225000 [391:29:20<49:21:50,  8.49s/it]
 91%|█████████ | 204069/225000 [391:29:35<61:16:36, 10.54s/it]
 91%|█████████ | 204070/225000 [391:29:45<59:29:47, 10.23s/it]
 91%|█████████ | 204071/225000 [391:29:50<51:17:56,  8.82s/it]
 91%|█████████ | 204072/225000 [391:29:58<48:31:53,  8.35s/it]
 91%|█████████ | 204073/225000 [391:30:15<64:11:04, 11.04s/it]
 91%|█████████ | 204074/225000 [391:30:20<52:55:29,  9.10s/it]
 91%|█████████ | 204075/225000 [391:30:24<45:15:28,  7.79s/it]
 91%|█████████ | 204076/225000 [391:30:36<51:17:11,  8.82s/it]
 91%|█████████ | 204077/225000 [391:30:42<46:40:40,  8.03s/it]
 91%|█████████ | 204078/225000 [391:30:50<47:05:26,  8.10s/it]
 91%|█████████ | 204079/225000 [391:30:58<46:43:43,  8.04s/it]
 91%|█████████ | 204080/225000 [391:31:06<46:27:53,  8.00s/it]
 91%|█████████ | 204081/225000 [391:31:21<59:57:04, 10.32s/it]
 91%|█████████ | 204082/225000 [391:31:26<49:45:26,  8.56s/it]
 91%|█████████ | 204083/225000 [391:31:36<51:47:26,  8.91s/it]
 91%|█████████ | 204084/225000 [391:31:45<53:05:08,  9.14s/it]
 91%|█████████ | 204085/225000 [391:31:55<53:06:14,  9.14s/it]
 91%|█████████ | 204086/225000 [391:32:01<49:01:48,  8.44s/it]
 91%|█████████ | 204087/225000 [391:32:14<55:46:47,  9.60s/it]
 91%|█████████ | 204088/225000 [391:32:22<53:48:21,  9.26s/it]
 91%|█████████ | 204089/225000 [391:32:28<47:14:16,  8.13s/it]
 91%|█████████ | 204090/225000 [391:32:34<43:40:46,  7.52s/it]
 91%|█████████ | 204091/225000 [391:32:43<47:13:19,  8.13s/it]
 91%|█████████ | 204092/225000 [391:32:51<45:53:20,  7.90s/it]
 91%|█████████ | 204093/225000 [391:32:59<46:29:51,  8.01s/it]
 91%|█████████ | 204094/225000 [391:33:07<47:11:37,  8.13s/it]
 91%|█████████ | 204095/225000 [391:33:18<51:05:02,  8.80s/it]
 91%|█████████ | 204096/225000 [391:33:29<54:55:30,  9.46s/it]
 91%|█████████ | 204097/225000 [391:33:36<51:30:55,  8.87s/it]
 91%|█████████ | 204098/225000 [391:33:43<48:26:44,  8.34s/it]
 91%|█████████ | 204099/225000 [391:33:50<46:33:14,  8.02s/it]
 91%|█████████ | 204100/225000 [391:33:56<42:00:00,  7.23s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.527', 'learning_rate': '1.167e-06', 'epoch': '0.9071', 'num_input_tokens_seen': 658890352, 'train_runtime': '1.41e+06', 'train_tokens_per_second': '467.4'}

 91%|█████████ | 204100/225000 [391:33:56<42:00:00,  7.23s/it]
 91%|█████████ | 204101/225000 [391:34:09<52:47:37,  9.09s/it]
 91%|█████████ | 204102/225000 [391:34:15<46:04:14,  7.94s/it]
 91%|█████████ | 204103/225000 [391:34:24<49:16:20,  8.49s/it]
 91%|█████████ | 204104/225000 [391:34:31<46:54:26,  8.08s/it]
 91%|█████████ | 204105/225000 [391:34:37<43:05:30,  7.42s/it]
 91%|█████████ | 204106/225000 [391:34:50<51:43:49,  8.91s/it]
 91%|█████████ | 204107/225000 [391:34:57<48:27:40,  8.35s/it]
 91%|█████████ | 204108/225000 [391:35:02<42:24:15,  7.31s/it]
 91%|█████████ | 204109/225000 [391:35:13<49:14:53,  8.49s/it]
 91%|█████████ | 204110/225000 [391:35:19<45:30:49,  7.84s/it]
 91%|█████████ | 204111/225000 [391:35:31<52:35:08,  9.06s/it]
 91%|█████████ | 204112/225000 [391:35:38<48:38:11,  8.38s/it]
 91%|█████████ | 204113/225000 [391:35:43<43:23:04,  7.48s/it]
 91%|█████████ | 204114/225000 [391:35:53<47:34:36,  8.20s/it]
 91%|█████████ | 204115/225000 [391:36:07<57:05:47,  9.84s/it]
 91%|█████████ | 204116/225000 [391:36:17<57:35:01,  9.93s/it]
 91%|█████████ | 204117/225000 [391:36:23<50:11:37,  8.65s/it]
 91%|█████████ | 204118/225000 [391:36:30<47:57:03,  8.27s/it]
 91%|█████████ | 204119/225000 [391:36:35<42:01:29,  7.25s/it]
 91%|█████████ | 204120/225000 [391:36:40<38:30:40,  6.64s/it]
 91%|█████████ | 204121/225000 [391:36:46<37:11:16,  6.41s/it]
 91%|█████████ | 204122/225000 [391:36:53<37:33:15,  6.48s/it]
 91%|█████████ | 204123/225000 [391:37:06<49:20:24,  8.51s/it]
 91%|█████████ | 204124/225000 [391:37:13<47:11:02,  8.14s/it]
 91%|█████████ | 204125/225000 [391:37:27<56:54:31,  9.81s/it]
 91%|█████████ | 204126/225000 [391:37:31<46:47:39,  8.07s/it]
 91%|█████████ | 204127/225000 [391:37:39<47:25:46,  8.18s/it]
 91%|█████████ | 204128/225000 [391:37:46<45:38:06,  7.87s/it]
 91%|█████████ | 204129/225000 [391:37:59<53:32:57,  9.24s/it]
 91%|█████████ | 204130/225000 [391:38:12<60:02:42, 10.36s/it]
 91%|█████████ | 204131/225000 [391:38:19<54:51:02,  9.46s/it]
 91%|█████████ | 204132/225000 [391:38:28<53:17:13,  9.19s/it]
 91%|█████████ | 204133/225000 [391:38:38<54:33:37,  9.41s/it]
 91%|█████████ | 204134/225000 [391:38:42<46:16:32,  7.98s/it]
 91%|█████████ | 204135/225000 [391:38:55<54:38:43,  9.43s/it]
 91%|█████████ | 204136/225000 [391:39:05<55:55:07,  9.65s/it]
 91%|█████████ | 204137/225000 [391:39:11<48:49:23,  8.42s/it]
 91%|█████████ | 204138/225000 [391:39:21<52:04:15,  8.99s/it]
 91%|█████████ | 204139/225000 [391:39:30<51:42:11,  8.92s/it]
 91%|█████████ | 204140/225000 [391:39:34<43:08:36,  7.45s/it]
 91%|█████████ | 204141/225000 [391:39:45<49:06:47,  8.48s/it]
 91%|█████████ | 204142/225000 [391:39:52<46:52:19,  8.09s/it]
 91%|█████████ | 204143/225000 [391:40:03<51:09:21,  8.83s/it]
 91%|█████████ | 204144/225000 [391:40:11<50:49:45,  8.77s/it]
 91%|█████████ | 204145/225000 [391:40:22<53:27:56,  9.23s/it]
 91%|█████████ | 204146/225000 [391:40:29<50:24:00,  8.70s/it]
 91%|█████████ | 204147/225000 [391:40:37<49:29:47,  8.54s/it]
 91%|█████████ | 204148/225000 [391:40:44<47:04:04,  8.13s/it]
 91%|█████████ | 204149/225000 [391:40:55<51:57:10,  8.97s/it]
 91%|█████████ | 204150/225000 [391:41:02<48:28:45,  8.37s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.283', 'learning_rate': '1.162e-06', 'epoch': '0.9073', 'num_input_tokens_seen': 659122992, 'train_runtime': '1.41e+06', 'train_tokens_per_second': '467.4'}

 91%|█████████ | 204150/225000 [391:41:02<48:28:45,  8.37s/it]
 91%|█████████ | 204151/225000 [391:41:11<48:35:27,  8.39s/it]
 91%|█████████ | 204152/225000 [391:41:17<44:49:33,  7.74s/it]
 91%|█████████ | 204153/225000 [391:41:34<61:02:09, 10.54s/it]
 91%|█████████ | 204154/225000 [391:41:40<52:39:18,  9.09s/it]
 91%|█████████ | 204155/225000 [391:41:50<55:33:11,  9.59s/it]
 91%|█████████ | 204156/225000 [391:41:56<48:41:39,  8.41s/it]
 91%|█████████ | 204157/225000 [391:42:09<56:10:10,  9.70s/it]
 91%|█████████ | 204158/225000 [391:42:14<48:49:19,  8.43s/it]
 91%|█████████ | 204159/225000 [391:42:26<54:41:36,  9.45s/it]
 91%|█████████ | 204160/225000 [391:42:36<55:59:08,  9.67s/it]
 91%|█████████ | 204161/225000 [391:42:45<54:24:15,  9.40s/it]
 91%|█████████ | 204162/225000 [391:42:54<54:05:07,  9.34s/it]
 91%|█████████ | 204163/225000 [391:43:04<54:51:55,  9.48s/it]
 91%|█████████ | 204164/225000 [391:43:09<47:15:07,  8.16s/it]
 91%|█████████ | 204165/225000 [391:43:19<49:43:51,  8.59s/it]
 91%|█████████ | 204166/225000 [391:43:26<46:42:49,  8.07s/it]
 91%|█████████ | 204167/225000 [391:43:38<54:03:21,  9.34s/it]
 91%|█████████ | 204168/225000 [391:43:48<55:40:57,  9.62s/it]
 91%|█████████ | 204169/225000 [391:43:57<54:37:10,  9.44s/it]
 91%|█████████ | 204170/225000 [391:44:02<46:44:30,  8.08s/it]
 91%|█████████ | 204171/225000 [391:44:11<48:06:50,  8.32s/it]
 91%|█████████ | 204172/225000 [391:44:16<42:56:19,  7.42s/it]
 91%|█████████ | 204173/225000 [391:44:24<43:36:28,  7.54s/it]
 91%|█████████ | 204174/225000 [391:44:33<45:03:40,  7.79s/it]
 91%|█████████ | 204175/225000 [391:44:40<45:13:00,  7.82s/it]
 91%|█████████ | 204176/225000 [391:44:48<44:01:44,  7.61s/it]
 91%|█████████ | 204177/225000 [391:44:57<47:48:20,  8.26s/it]
 91%|█████████ | 204178/225000 [391:45:09<53:32:20,  9.26s/it]
 91%|█████████ | 204179/225000 [391:45:15<48:39:03,  8.41s/it]
 91%|█████████ | 204180/225000 [391:45:20<42:53:15,  7.42s/it]
 91%|█████████ | 204181/225000 [391:45:27<40:41:09,  7.04s/it]
 91%|█████████ | 204182/225000 [391:45:46<62:54:25, 10.88s/it]
 91%|█████████ | 204183/225000 [391:45:57<62:31:37, 10.81s/it]
 91%|█████████ | 204184/225000 [391:46:04<55:12:09,  9.55s/it]
 91%|█████████ | 204185/225000 [391:46:17<61:11:38, 10.58s/it]
 91%|█████████ | 204186/225000 [391:46:23<53:37:25,  9.27s/it]
 91%|█████████ | 204187/225000 [391:46:33<55:49:45,  9.66s/it]
 91%|█████████ | 204188/225000 [391:46:40<49:59:23,  8.65s/it]
 91%|█████████ | 204189/225000 [391:46:45<44:16:22,  7.66s/it]
 91%|█████████ | 204190/225000 [391:46:52<42:32:33,  7.36s/it]
 91%|█████████ | 204191/225000 [391:47:04<51:38:12,  8.93s/it]
 91%|█████████ | 204192/225000 [391:47:10<45:23:44,  7.85s/it]
 91%|█████████ | 204193/225000 [391:47:19<47:19:25,  8.19s/it]
 91%|█████████ | 204194/225000 [391:47:28<49:40:31,  8.60s/it]
 91%|█████████ | 204195/225000 [391:47:35<47:00:57,  8.14s/it]
 91%|█████████ | 204196/225000 [391:47:49<55:58:39,  9.69s/it]
 91%|█████████ | 204197/225000 [391:47:55<50:47:45,  8.79s/it]
 91%|█████████ | 204198/225000 [391:48:04<51:26:34,  8.90s/it]
 91%|█████████ | 204199/225000 [391:48:13<50:56:45,  8.82s/it]
 91%|█████████ | 204200/225000 [391:48:18<44:49:40,  7.76s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.031', 'learning_rate': '1.156e-06', 'epoch': '0.9076', 'num_input_tokens_seen': 659355296, 'train_runtime': '1.41e+06', 'train_tokens_per_second': '467.5'}

 91%|█████████ | 204200/225000 [391:48:18<44:49:40,  7.76s/it]
 91%|█████████ | 204201/225000 [391:48:27<46:12:53,  8.00s/it]
 91%|█████████ | 204202/225000 [391:48:39<53:04:01,  9.19s/it]
 91%|█████████ | 204203/225000 [391:48:49<54:15:53,  9.39s/it]
 91%|█████████ | 204204/225000 [391:48:56<51:25:54,  8.90s/it]
 91%|█████████ | 204205/225000 [391:49:04<48:45:32,  8.44s/it]
 91%|█████████ | 204206/225000 [391:49:10<44:13:23,  7.66s/it]
 91%|█████████ | 204207/225000 [391:49:18<45:53:32,  7.95s/it]
 91%|█████████ | 204208/225000 [391:49:22<39:11:21,  6.79s/it]
 91%|█████████ | 204209/225000 [391:49:32<43:24:03,  7.51s/it]
 91%|█████████ | 204210/225000 [391:49:41<47:17:31,  8.19s/it]
 91%|█████████ | 204211/225000 [391:49:52<51:27:20,  8.91s/it]
 91%|█████████ | 204212/225000 [391:50:02<54:10:16,  9.38s/it]
 91%|█████████ | 204213/225000 [391:50:12<54:46:59,  9.49s/it]
 91%|█████████ | 204214/225000 [391:50:20<51:10:10,  8.86s/it]
 91%|█████████ | 204215/225000 [391:50:27<48:13:25,  8.35s/it]
 91%|█████████ | 204216/225000 [391:50:33<44:17:52,  7.67s/it]
 91%|█████████ | 204217/225000 [391:50:40<42:37:21,  7.38s/it]
 91%|█████████ | 204218/225000 [391:50:55<57:01:35,  9.88s/it]
 91%|█████████ | 204219/225000 [391:51:01<49:45:58,  8.62s/it]
 91%|█████████ | 204220/225000 [391:51:09<49:11:43,  8.52s/it]
 91%|█████████ | 204221/225000 [391:51:16<46:00:25,  7.97s/it]
 91%|█████████ | 204222/225000 [391:51:25<47:30:58,  8.23s/it]
 91%|█████████ | 204223/225000 [391:51:30<42:45:26,  7.41s/it]
 91%|█████████ | 204224/225000 [391:51:40<47:34:59,  8.25s/it]
 91%|█████████ | 204225/225000 [391:51:54<57:27:33,  9.96s/it]
 91%|█████████ | 204226/225000 [391:52:04<57:15:29,  9.92s/it]
 91%|█████████ | 204227/225000 [391:52:10<49:51:23,  8.64s/it]
 91%|█████████ | 204228/225000 [391:52:18<49:16:46,  8.54s/it]
 91%|█████████ | 204229/225000 [391:52:25<46:35:58,  8.08s/it]
 91%|█████████ | 204230/225000 [391:52:32<44:01:08,  7.63s/it]
 91%|█████████ | 204231/225000 [391:52:39<44:00:33,  7.63s/it]
 91%|█████████ | 204232/225000 [391:52:46<41:47:56,  7.25s/it]
 91%|█████████ | 204233/225000 [391:52:53<41:51:18,  7.26s/it]
 91%|█████████ | 204234/225000 [391:53:09<57:11:29,  9.91s/it]
 91%|█████████ | 204235/225000 [391:53:14<48:45:07,  8.45s/it]
 91%|█████████ | 204236/225000 [391:53:25<52:46:59,  9.15s/it]
 91%|█████████ | 204237/225000 [391:53:32<49:38:44,  8.61s/it]
 91%|█████████ | 204238/225000 [391:53:38<43:47:01,  7.59s/it]
 91%|█████████ | 204239/225000 [391:53:50<52:44:28,  9.15s/it]
 91%|█████████ | 204240/225000 [391:54:00<53:20:45,  9.25s/it]
 91%|█████████ | 204241/225000 [391:54:09<53:29:25,  9.28s/it]
 91%|█████████ | 204242/225000 [391:54:18<53:40:08,  9.31s/it]
 91%|█████████ | 204243/225000 [391:54:29<55:03:32,  9.55s/it]
 91%|█████████ | 204244/225000 [391:54:36<50:58:11,  8.84s/it]
 91%|█████████ | 204245/225000 [391:54:45<52:15:46,  9.07s/it]
 91%|█████████ | 204246/225000 [391:54:52<47:10:53,  8.18s/it]
 91%|█████████ | 204247/225000 [391:55:04<53:56:06,  9.36s/it]
 91%|█████████ | 204248/225000 [391:55:10<48:16:07,  8.37s/it]
 91%|█████████ | 204249/225000 [391:55:19<49:42:19,  8.62s/it]
 91%|█████████ | 204250/225000 [391:55:27<48:28:52,  8.41s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.618', 'learning_rate': '1.151e-06', 'epoch': '0.9078', 'num_input_tokens_seen': 659587824, 'train_runtime': '1.411e+06', 'train_tokens_per_second': '467.5'}

 91%|█████████ | 204250/225000 [391:55:27<48:28:52,  8.41s/it]
 91%|█████████ | 204251/225000 [391:55:37<51:02:39,  8.86s/it]
 91%|█████████ | 204252/225000 [391:55:44<48:31:50,  8.42s/it]
 91%|█████████ | 204253/225000 [391:55:52<47:01:48,  8.16s/it]
 91%|█████████ | 204254/225000 [391:56:00<46:48:00,  8.12s/it]
 91%|█████████ | 204255/225000 [391:56:12<54:36:43,  9.48s/it]
 91%|█████████ | 204256/225000 [391:56:19<49:49:47,  8.65s/it]
 91%|█████████ | 204257/225000 [391:56:24<43:44:32,  7.59s/it]
 91%|█████████ | 204258/225000 [391:56:39<56:10:34,  9.75s/it]
 91%|█████████ | 204259/225000 [391:56:44<47:25:49,  8.23s/it]
 91%|█████████ | 204260/225000 [391:56:57<55:25:55,  9.62s/it]
 91%|█████████ | 204261/225000 [391:57:02<48:33:57,  8.43s/it]
 91%|█████████ | 204262/225000 [391:57:10<47:45:26,  8.29s/it]
 91%|█████████ | 204263/225000 [391:57:17<44:48:46,  7.78s/it]
 91%|█████████ | 204264/225000 [391:57:25<45:29:24,  7.90s/it]
 91%|█████████ | 204265/225000 [391:57:33<45:49:41,  7.96s/it]
 91%|█████████ | 204266/225000 [391:57:47<56:53:06,  9.88s/it]
 91%|█████████ | 204267/225000 [391:57:56<53:56:56,  9.37s/it]
 91%|█████████ | 204268/225000 [391:58:00<45:44:36,  7.94s/it]
 91%|█████████ | 204269/225000 [391:58:09<47:11:02,  8.19s/it]
 91%|█████████ | 204270/225000 [391:58:14<41:46:09,  7.25s/it]
 91%|█████████ | 204271/225000 [391:58:20<39:57:04,  6.94s/it]
 91%|█████████ | 204272/225000 [391:58:28<40:43:54,  7.07s/it]
 91%|█████████ | 204273/225000 [391:58:39<48:11:31,  8.37s/it]
 91%|█████████ | 204274/225000 [391:58:45<44:07:55,  7.67s/it]
 91%|█████████ | 204275/225000 [391:58:54<46:55:07,  8.15s/it]
 91%|█████████ | 204276/225000 [391:59:06<53:15:24,  9.25s/it]
 91%|█████████ | 204277/225000 [391:59:12<48:16:09,  8.39s/it]
 91%|█████████ | 204278/225000 [391:59:18<43:22:59,  7.54s/it]
 91%|█████████ | 204279/225000 [391:59:26<44:22:59,  7.71s/it]
 91%|█████████ | 204280/225000 [391:59:39<52:59:59,  9.21s/it]
 91%|█████████ | 204281/225000 [391:59:44<45:12:49,  7.86s/it]
 91%|█████████ | 204282/225000 [391:59:52<46:07:37,  8.02s/it]
 91%|█████████ | 204283/225000 [391:59:58<42:43:44,  7.43s/it]
 91%|█████████ | 204284/225000 [392:00:11<52:36:38,  9.14s/it]
 91%|█████████ | 204285/225000 [392:00:19<49:37:22,  8.62s/it]
 91%|█████████ | 204286/225000 [392:00:31<56:00:45,  9.73s/it]
 91%|█████████ | 204287/225000 [392:00:41<56:00:54,  9.74s/it]
 91%|█████████ | 204288/225000 [392:00:45<46:58:21,  8.16s/it]
 91%|█████████ | 204289/225000 [392:00:54<48:14:56,  8.39s/it]
 91%|█████████ | 204290/225000 [392:01:02<48:23:28,  8.41s/it]
 91%|█████████ | 204291/225000 [392:01:11<48:24:58,  8.42s/it]
 91%|█████████ | 204292/225000 [392:01:23<54:05:00,  9.40s/it]
 91%|█████████ | 204293/225000 [392:01:30<50:28:29,  8.78s/it]
 91%|█████████ | 204294/225000 [392:01:37<46:51:59,  8.15s/it]
 91%|█████████ | 204295/225000 [392:01:44<45:17:48,  7.88s/it]
 91%|█████████ | 204296/225000 [392:01:54<49:35:17,  8.62s/it]
 91%|█████████ | 204297/225000 [392:02:01<45:45:48,  7.96s/it]
 91%|█████████ | 204298/225000 [392:02:06<41:37:12,  7.24s/it]
 91%|█████████ | 204299/225000 [392:02:15<44:56:50,  7.82s/it]
 91%|█████████ | 204300/225000 [392:02:28<53:54:56,  9.38s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.322', 'learning_rate': '1.145e-06', 'epoch': '0.908', 'num_input_tokens_seen': 659821072, 'train_runtime': '1.411e+06', 'train_tokens_per_second': '467.5'}

 91%|█████████ | 204300/225000 [392:02:28<53:54:56,  9.38s/it]
 91%|█████████ | 204301/225000 [392:02:35<48:58:22,  8.52s/it]
 91%|█████████ | 204302/225000 [392:02:40<43:55:44,  7.64s/it]
 91%|█████████ | 204303/225000 [392:02:52<49:59:30,  8.70s/it]
 91%|█████████ | 204304/225000 [392:02:59<48:19:28,  8.41s/it]
 91%|█████████ | 204305/225000 [392:03:05<43:22:08,  7.54s/it]
 91%|█████████ | 204306/225000 [392:03:13<45:06:34,  7.85s/it]
 91%|█████████ | 204307/225000 [392:03:27<54:30:57,  9.48s/it]
 91%|█████████ | 204308/225000 [392:03:36<53:27:02,  9.30s/it]
 91%|█████████ | 204309/225000 [392:03:43<50:00:04,  8.70s/it]
 91%|█████████ | 204310/225000 [392:03:48<43:25:34,  7.56s/it]
 91%|█████████ | 204311/225000 [392:03:58<48:19:05,  8.41s/it]
 91%|█████████ | 204312/225000 [392:04:06<47:46:10,  8.31s/it]
 91%|█████████ | 204313/225000 [392:04:13<44:47:37,  7.80s/it]
 91%|█████████ | 204314/225000 [392:04:19<41:58:48,  7.31s/it]
 91%|█████████ | 204315/225000 [392:04:26<40:36:19,  7.07s/it]
 91%|█████████ | 204316/225000 [392:04:37<48:15:51,  8.40s/it]
 91%|█████████ | 204317/225000 [392:04:42<42:48:13,  7.45s/it]
 91%|█████████ | 204318/225000 [392:04:54<49:47:28,  8.67s/it]
 91%|█████████ | 204319/225000 [392:05:01<48:00:18,  8.36s/it]
 91%|█████████ | 204320/225000 [392:05:10<47:35:13,  8.28s/it]
 91%|█████████ | 204321/225000 [392:05:22<54:20:20,  9.46s/it]
 91%|█████████ | 204322/225000 [392:05:26<45:19:49,  7.89s/it]
 91%|█████████ | 204323/225000 [392:05:35<46:38:52,  8.12s/it]
 91%|█████████ | 204324/225000 [392:05:40<42:25:39,  7.39s/it]
 91%|█████████ | 204325/225000 [392:05:49<45:32:56,  7.93s/it]
 91%|█████████ | 204326/225000 [392:05:59<48:32:09,  8.45s/it]
 91%|█████████ | 204327/225000 [392:06:06<45:27:21,  7.92s/it]
 91%|█████████ | 204328/225000 [392:06:14<46:32:44,  8.11s/it]
 91%|█████████ | 204329/225000 [392:06:23<46:56:20,  8.17s/it]
 91%|█████████ | 204330/225000 [392:06:32<48:53:41,  8.52s/it]
 91%|█████████ | 204331/225000 [392:06:39<46:06:34,  8.03s/it]
 91%|█████████ | 204332/225000 [392:06:45<42:59:35,  7.49s/it]
 91%|█████████ | 204333/225000 [392:06:51<39:38:42,  6.91s/it]
 91%|█████████ | 204334/225000 [392:07:02<46:37:21,  8.12s/it]
 91%|█████████ | 204335/225000 [392:07:12<49:41:18,  8.66s/it]
 91%|█████████ | 204336/225000 [392:07:20<48:53:23,  8.52s/it]
 91%|█████████ | 204337/225000 [392:07:29<49:32:03,  8.63s/it]
 91%|█████████ | 204338/225000 [392:07:38<50:52:10,  8.86s/it]
 91%|█████████ | 204339/225000 [392:07:43<44:39:14,  7.78s/it]
 91%|█████████ | 204340/225000 [392:07:50<43:18:56,  7.55s/it]
 91%|█████████ | 204341/225000 [392:08:00<47:23:52,  8.26s/it]
 91%|█████████ | 204342/225000 [392:08:07<44:28:08,  7.75s/it]
 91%|█████████ | 204343/225000 [392:08:18<50:38:38,  8.83s/it]
 91%|█████████ | 204344/225000 [392:08:31<58:15:42, 10.15s/it]
 91%|█████████ | 204345/225000 [392:08:40<54:49:16,  9.55s/it]
 91%|█████████ | 204346/225000 [392:08:48<52:25:57,  9.14s/it]
 91%|█████████ | 204347/225000 [392:09:01<60:25:08, 10.53s/it]
 91%|█████████ | 204348/225000 [392:09:15<65:06:15, 11.35s/it]
 91%|█████████ | 204349/225000 [392:09:19<53:27:03,  9.32s/it]
 91%|█████████ | 204350/225000 [392:09:30<56:27:44,  9.84s/it]
                                                              
{'loss': '1.331', 'grad_norm': '1.379', 'learning_rate': '1.14e-06', 'epoch': '0.9082', 'num_input_tokens_seen': 660049776, 'train_runtime': '1.412e+06', 'train_tokens_per_second': '467.5'}

 91%|█████████ | 204350/225000 [392:09:30<56:27:44,  9.84s/it]
 91%|█████████ | 204351/225000 [392:09:39<54:54:58,  9.57s/it]
 91%|█████████ | 204352/225000 [392:09:51<57:46:38, 10.07s/it]
 91%|█████████ | 204353/225000 [392:09:56<50:22:09,  8.78s/it]
 91%|█████████ | 204354/225000 [392:10:08<55:04:55,  9.60s/it]
 91%|█████████ | 204355/225000 [392:10:14<49:55:53,  8.71s/it]
 91%|█████████ | 204356/225000 [392:10:21<45:55:46,  8.01s/it]
 91%|█████████ | 204357/225000 [392:10:27<42:53:02,  7.48s/it]
 91%|█████████ | 204358/225000 [392:10:36<44:32:57,  7.77s/it]
 91%|█████████ | 204359/225000 [392:10:42<41:36:15,  7.26s/it]
 91%|█████████ | 204360/225000 [392:10:53<49:27:43,  8.63s/it]
 91%|█████████ | 204361/225000 [392:11:02<49:38:45,  8.66s/it]
 91%|█████████ | 204362/225000 [392:11:09<47:20:58,  8.26s/it]
 91%|█████████ | 204363/225000 [392:11:22<54:16:59,  9.47s/it]
 91%|█████████ | 204364/225000 [392:11:28<49:07:08,  8.57s/it]
 91%|█████████ | 204365/225000 [392:11:40<54:13:19,  9.46s/it]
 91%|█████████ | 204366/225000 [392:11:50<56:02:14,  9.78s/it]
 91%|█████████ | 204367/225000 [392:11:58<52:31:42,  9.17s/it]
 91%|█████████ | 204368/225000 [392:12:05<49:29:26,  8.64s/it]
 91%|█████████ | 204369/225000 [392:12:12<45:12:36,  7.89s/it]
 91%|█████████ | 204370/225000 [392:12:18<42:32:52,  7.42s/it]
 91%|█████████ | 204371/225000 [392:12:30<51:20:44,  8.96s/it]
 91%|█████████ | 204372/225000 [392:12:37<46:56:10,  8.19s/it]
 91%|█████████ | 204373/225000 [392:12:48<52:03:28,  9.09s/it]
 91%|█████████ | 204374/225000 [392:12:58<53:20:34,  9.31s/it]
 91%|█████████ | 204375/225000 [392:13:08<53:55:23,  9.41s/it]
 91%|█████████ | 204376/225000 [392:13:13<46:36:23,  8.14s/it]
 91%|█████████ | 204377/225000 [392:13:26<54:43:29,  9.55s/it]
 91%|█████████ | 204378/225000 [392:13:33<51:07:16,  8.92s/it]
 91%|█████████ | 204379/225000 [392:13:42<50:51:32,  8.88s/it]
 91%|█████████ | 204380/225000 [392:13:50<49:15:22,  8.60s/it]
 91%|█████████ | 204381/225000 [392:14:00<51:21:04,  8.97s/it]
 91%|█████████ | 204382/225000 [392:14:06<47:34:16,  8.31s/it]
 91%|█████████ | 204383/225000 [392:14:14<47:01:26,  8.21s/it]
 91%|█████████ | 204384/225000 [392:14:25<52:00:14,  9.08s/it]
 91%|█████████ | 204385/225000 [392:14:31<46:42:29,  8.16s/it]
 91%|█████████ | 204386/225000 [392:14:38<43:54:48,  7.67s/it]
 91%|█████████ | 204387/225000 [392:14:45<42:24:29,  7.41s/it]
 91%|█████████ | 204388/225000 [392:14:51<39:51:19,  6.96s/it]
 91%|█████████ | 204389/225000 [392:14:57<39:23:39,  6.88s/it]
 91%|█████████ | 204390/225000 [392:15:13<53:47:14,  9.40s/it]
 91%|█████████ | 204391/225000 [392:15:18<46:54:47,  8.19s/it]
 91%|█████████ | 204392/225000 [392:15:25<44:10:27,  7.72s/it]
 91%|█████████ | 204393/225000 [392:15:32<43:29:22,  7.60s/it]
 91%|█████████ | 204394/225000 [392:15:46<55:21:08,  9.67s/it]
 91%|█████████ | 204395/225000 [392:15:54<51:44:11,  9.04s/it]
 91%|█████████ | 204396/225000 [392:16:00<46:32:54,  8.13s/it]
 91%|█████████ | 204397/225000 [392:16:06<43:29:52,  7.60s/it]
 91%|█████████ | 204398/225000 [392:16:18<50:50:08,  8.88s/it]
 91%|█████████ | 204399/225000 [392:16:27<50:18:03,  8.79s/it]
 91%|█████████ | 204400/225000 [392:16:35<49:38:15,  8.67s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.44', 'learning_rate': '1.134e-06', 'epoch': '0.9084', 'num_input_tokens_seen': 660279576, 'train_runtime': '1.412e+06', 'train_tokens_per_second': '467.6'}

 91%|█████████ | 204400/225000 [392:16:35<49:38:15,  8.67s/it]
 91%|█████████ | 204401/225000 [392:16:40<42:06:24,  7.36s/it]
 91%|█████████ | 204402/225000 [392:16:45<39:43:15,  6.94s/it]
 91%|█████████ | 204403/225000 [392:16:52<39:03:00,  6.83s/it]
 91%|█████████ | 204404/225000 [392:17:08<54:46:42,  9.57s/it]
 91%|█████████ | 204405/225000 [392:17:14<48:44:46,  8.52s/it]
 91%|█████████ | 204406/225000 [392:17:27<56:09:03,  9.82s/it]
 91%|█████████ | 204407/225000 [392:17:34<51:54:38,  9.07s/it]
 91%|█████████ | 204408/225000 [392:17:45<55:19:58,  9.67s/it]
 91%|█████████ | 204409/225000 [392:17:53<52:34:18,  9.19s/it]
 91%|█████████ | 204410/225000 [392:17:58<45:11:20,  7.90s/it]
 91%|█████████ | 204411/225000 [392:18:09<49:13:29,  8.61s/it]
 91%|█████████ | 204412/225000 [392:18:17<48:27:46,  8.47s/it]
 91%|█████████ | 204413/225000 [392:18:23<44:44:27,  7.82s/it]
 91%|█████████ | 204414/225000 [392:18:33<49:03:27,  8.58s/it]
 91%|█████████ | 204415/225000 [392:18:41<47:45:35,  8.35s/it]
 91%|█████████ | 204416/225000 [392:18:48<44:36:58,  7.80s/it]
 91%|█████████ | 204417/225000 [392:18:54<41:13:08,  7.21s/it]
 91%|█████████ | 204418/225000 [392:18:59<38:59:29,  6.82s/it]
 91%|█████████ | 204419/225000 [392:19:07<40:18:08,  7.05s/it]
 91%|█████████ | 204420/225000 [392:19:20<51:06:54,  8.94s/it]
 91%|█████████ | 204421/225000 [392:19:25<43:32:49,  7.62s/it]
 91%|█████████ | 204422/225000 [392:19:32<42:19:53,  7.41s/it]
 91%|█████████ | 204423/225000 [392:19:44<50:39:27,  8.86s/it]
 91%|█████████ | 204424/225000 [392:19:52<48:30:47,  8.49s/it]
 91%|█████████ | 204425/225000 [392:20:03<53:09:51,  9.30s/it]
 91%|█████████ | 204426/225000 [392:20:11<51:46:51,  9.06s/it]
 91%|█████████ | 204427/225000 [392:20:19<49:37:28,  8.68s/it]
 91%|█████████ | 204428/225000 [392:20:31<54:23:59,  9.52s/it]
 91%|█████████ | 204429/225000 [392:20:36<46:41:37,  8.17s/it]
 91%|█████████ | 204430/225000 [392:20:49<54:59:53,  9.63s/it]
 91%|█████████ | 204431/225000 [392:20:55<49:17:38,  8.63s/it]
 91%|█████████ | 204432/225000 [392:21:06<53:32:46,  9.37s/it]
 91%|█████████ | 204433/225000 [392:21:15<51:59:49,  9.10s/it]
 91%|█████████ | 204434/225000 [392:21:20<45:54:09,  8.04s/it]
 91%|█████████ | 204435/225000 [392:21:28<45:40:00,  7.99s/it]
 91%|█████████ | 204436/225000 [392:21:32<39:28:26,  6.91s/it]
 91%|█████████ | 204437/225000 [392:21:47<53:09:40,  9.31s/it]
 91%|█████████ | 204438/225000 [392:21:58<55:15:20,  9.67s/it]
 91%|█████████ | 204439/225000 [392:22:03<47:49:58,  8.38s/it]
 91%|█████████ | 204440/225000 [392:22:10<45:27:24,  7.96s/it]
 91%|█████████ | 204441/225000 [392:22:19<46:23:56,  8.12s/it]
 91%|█████████ | 204442/225000 [392:22:25<43:57:36,  7.70s/it]
 91%|█████████ | 204443/225000 [392:22:36<48:30:59,  8.50s/it]
 91%|█████████ | 204444/225000 [392:22:41<42:39:28,  7.47s/it]
 91%|█████████ | 204445/225000 [392:22:46<37:55:36,  6.64s/it]
 91%|█████████ | 204446/225000 [392:23:00<50:56:20,  8.92s/it]
 91%|█████████ | 204447/225000 [392:23:04<43:33:14,  7.63s/it]
 91%|█████████ | 204448/225000 [392:23:10<40:30:10,  7.09s/it]
 91%|█████████ | 204449/225000 [392:23:20<45:43:51,  8.01s/it]
 91%|█████████ | 204450/225000 [392:23:27<42:45:24,  7.49s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.736', 'learning_rate': '1.129e-06', 'epoch': '0.9087', 'num_input_tokens_seen': 660507984, 'train_runtime': '1.413e+06', 'train_tokens_per_second': '467.6'}

 91%|█████████ | 204450/225000 [392:23:27<42:45:24,  7.49s/it]
 91%|█████████ | 204451/225000 [392:23:32<38:17:19,  6.71s/it]
 91%|█████████ | 204452/225000 [392:23:41<42:51:07,  7.51s/it]
 91%|█████████ | 204453/225000 [392:23:47<40:42:05,  7.13s/it]
 91%|█████████ | 204454/225000 [392:23:53<38:16:06,  6.71s/it]
 91%|█████████ | 204455/225000 [392:23:59<37:03:25,  6.49s/it]
 91%|█████████ | 204456/225000 [392:24:13<49:33:14,  8.68s/it]
 91%|█████████ | 204457/225000 [392:24:19<46:19:46,  8.12s/it]
 91%|█████████ | 204458/225000 [392:24:31<52:30:15,  9.20s/it]
 91%|█████████ | 204459/225000 [392:24:40<52:29:11,  9.20s/it]
 91%|█████████ | 204460/225000 [392:24:48<49:25:04,  8.66s/it]
 91%|█████████ | 204461/225000 [392:24:58<52:47:59,  9.25s/it]
 91%|█████████ | 204462/225000 [392:25:08<53:28:16,  9.37s/it]
 91%|█████████ | 204463/225000 [392:25:14<47:45:09,  8.37s/it]
 91%|█████████ | 204464/225000 [392:25:21<45:20:35,  7.95s/it]
 91%|█████████ | 204465/225000 [392:25:29<45:42:26,  8.01s/it]
 91%|█████████ | 204466/225000 [392:25:34<40:14:05,  7.05s/it]
 91%|█████████ | 204467/225000 [392:25:48<51:30:22,  9.03s/it]
 91%|█████████ | 204468/225000 [392:25:54<46:15:05,  8.11s/it]
 91%|█████████ | 204469/225000 [392:26:00<43:00:19,  7.54s/it]
 91%|█████████ | 204470/225000 [392:26:08<43:11:58,  7.58s/it]
 91%|█████████ | 204471/225000 [392:26:16<44:41:03,  7.84s/it]
 91%|█████████ | 204472/225000 [392:26:31<56:10:40,  9.85s/it]
 91%|█████████ | 204473/225000 [392:26:37<50:15:34,  8.81s/it]
 91%|█████████ | 204474/225000 [392:26:43<45:36:20,  8.00s/it]
 91%|█████████ | 204475/225000 [392:26:55<52:13:58,  9.16s/it]
 91%|█████████ | 204476/225000 [392:27:00<44:37:46,  7.83s/it]
 91%|█████████ | 204477/225000 [392:27:10<48:20:19,  8.48s/it]
 91%|█████████ | 204478/225000 [392:27:17<47:15:07,  8.29s/it]
 91%|█████████ | 204479/225000 [392:27:25<46:22:23,  8.14s/it]
 91%|█████████ | 204480/225000 [392:27:37<52:55:08,  9.28s/it]
 91%|█████████ | 204481/225000 [392:27:46<52:06:10,  9.14s/it]
 91%|█████████ | 204482/225000 [392:27:52<46:48:09,  8.21s/it]
 91%|█████████ | 204483/225000 [392:28:00<46:33:56,  8.17s/it]
 91%|█████████ | 204484/225000 [392:28:05<41:17:37,  7.25s/it]
 91%|█████████ | 204485/225000 [392:28:20<53:38:05,  9.41s/it]
 91%|█████████ | 204486/225000 [392:28:25<45:59:55,  8.07s/it]
 91%|█████████ | 204487/225000 [392:28:31<43:38:45,  7.66s/it]
 91%|█████████ | 204488/225000 [392:28:43<50:34:28,  8.88s/it]
 91%|█████████ | 204489/225000 [392:28:55<55:07:56,  9.68s/it]
 91%|█████████ | 204490/225000 [392:29:01<49:03:26,  8.61s/it]
 91%|█████████ | 204491/225000 [392:29:07<45:13:59,  7.94s/it]
 91%|█████████ | 204492/225000 [392:29:19<51:08:15,  8.98s/it]
 91%|█████████ | 204493/225000 [392:29:27<50:45:39,  8.91s/it]
 91%|█████████ | 204494/225000 [392:29:33<45:22:55,  7.97s/it]
 91%|█████████ | 204495/225000 [392:29:39<42:07:57,  7.40s/it]
 91%|█████████ | 204496/225000 [392:29:45<39:44:15,  6.98s/it]
 91%|█████████ | 204497/225000 [392:30:01<54:11:25,  9.51s/it]
 91%|█████████ | 204498/225000 [392:30:11<55:38:58,  9.77s/it]
 91%|█████████ | 204499/225000 [392:30:20<54:55:15,  9.64s/it]
 91%|█████████ | 204500/225000 [392:30:34<61:10:45, 10.74s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.335', 'learning_rate': '1.124e-06', 'epoch': '0.9089', 'num_input_tokens_seen': 660733240, 'train_runtime': '1.413e+06', 'train_tokens_per_second': '467.6'}

 91%|█████████ | 204500/225000 [392:30:34<61:10:45, 10.74s/it]
 91%|█████████ | 204501/225000 [392:30:40<53:19:03,  9.36s/it]
 91%|█████████ | 204502/225000 [392:30:48<51:12:06,  8.99s/it]
 91%|█████████ | 204503/225000 [392:30:55<48:17:49,  8.48s/it]
 91%|█████████ | 204504/225000 [392:31:05<51:02:38,  8.97s/it]
 91%|█████████ | 204505/225000 [392:31:12<46:39:03,  8.19s/it]
 91%|█████████ | 204506/225000 [392:31:18<42:56:01,  7.54s/it]
 91%|█████████ | 204507/225000 [392:31:26<44:30:47,  7.82s/it]
 91%|█████████ | 204508/225000 [392:31:38<51:31:24,  9.05s/it]
 91%|█████████ | 204509/225000 [392:31:45<47:23:22,  8.33s/it]
 91%|█████████ | 204510/225000 [392:31:54<48:30:24,  8.52s/it]
 91%|█████████ | 204511/225000 [392:32:00<45:29:12,  7.99s/it]
 91%|█████████ | 204512/225000 [392:32:07<43:40:50,  7.68s/it]
 91%|█████████ | 204513/225000 [392:32:14<42:22:23,  7.45s/it]
 91%|█████████ | 204514/225000 [392:32:27<51:39:44,  9.08s/it]
 91%|█████████ | 204515/225000 [392:32:36<50:58:06,  8.96s/it]
 91%|█████████ | 204516/225000 [392:32:41<45:19:22,  7.97s/it]
 91%|█████████ | 204517/225000 [392:32:53<51:30:29,  9.05s/it]
 91%|█████████ | 204518/225000 [392:33:05<56:33:11,  9.94s/it]
 91%|█████████ | 204519/225000 [392:33:10<47:35:28,  8.37s/it]
 91%|█████████ | 204520/225000 [392:33:15<42:58:49,  7.56s/it]
 91%|█████████ | 204521/225000 [392:33:25<45:44:09,  8.04s/it]
 91%|█████████ | 204522/225000 [392:33:38<55:22:00,  9.73s/it]
 91%|█████████ | 204523/225000 [392:33:47<52:53:53,  9.30s/it]
 91%|█████████ | 204524/225000 [392:34:00<59:50:43, 10.52s/it]
 91%|█████████ | 204525/225000 [392:34:11<61:01:37, 10.73s/it]
 91%|█████████ | 204526/225000 [392:34:17<52:14:20,  9.19s/it]
 91%|█████████ | 204527/225000 [392:34:27<54:17:57,  9.55s/it]
 91%|█████████ | 204528/225000 [392:34:35<51:18:36,  9.02s/it]
 91%|█████████ | 204529/225000 [392:34:41<45:36:32,  8.02s/it]
 91%|█████████ | 204530/225000 [392:34:53<52:41:20,  9.27s/it]
 91%|█████████ | 204531/225000 [392:35:00<48:28:42,  8.53s/it]
 91%|█████████ | 204532/225000 [392:35:11<53:19:09,  9.38s/it]
 91%|█████████ | 204533/225000 [392:35:19<50:34:28,  8.90s/it]
 91%|█████████ | 204534/225000 [392:35:25<46:55:35,  8.25s/it]
 91%|█████████ | 204535/225000 [392:35:39<56:29:21,  9.94s/it]
 91%|█████████ | 204536/225000 [392:35:47<53:12:22,  9.36s/it]
 91%|█████████ | 204537/225000 [392:35:57<54:09:44,  9.53s/it]
 91%|█████████ | 204538/225000 [392:36:03<47:54:37,  8.43s/it]
 91%|█████████ | 204539/225000 [392:36:11<47:06:07,  8.29s/it]
 91%|█████████ | 204540/225000 [392:36:17<43:35:38,  7.67s/it]
 91%|█████████ | 204541/225000 [392:36:25<43:05:15,  7.58s/it]
 91%|█████████ | 204542/225000 [392:36:35<48:04:17,  8.46s/it]
 91%|█████████ | 204543/225000 [392:36:43<46:20:12,  8.15s/it]
 91%|█████████ | 204544/225000 [392:36:52<47:58:58,  8.44s/it]
 91%|█████████ | 204545/225000 [392:37:01<48:40:57,  8.57s/it]
 91%|█████████ | 204546/225000 [392:37:07<44:58:10,  7.91s/it]
 91%|█████████ | 204547/225000 [392:37:17<48:44:36,  8.58s/it]
 91%|█████████ | 204548/225000 [392:37:25<48:04:42,  8.46s/it]
 91%|█████████ | 204549/225000 [392:37:32<44:50:44,  7.89s/it]
 91%|█████████ | 204550/225000 [392:37:49<61:09:17, 10.77s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.35', 'learning_rate': '1.118e-06', 'epoch': '0.9091', 'num_input_tokens_seen': 660969480, 'train_runtime': '1.413e+06', 'train_tokens_per_second': '467.6'}

 91%|█████████ | 204550/225000 [392:37:49<61:09:17, 10.77s/it]
 91%|█████████ | 204551/225000 [392:37:59<58:38:10, 10.32s/it]
 91%|█████████ | 204552/225000 [392:38:09<59:15:44, 10.43s/it]
 91%|█████████ | 204553/225000 [392:38:15<51:21:30,  9.04s/it]
 91%|█████████ | 204554/225000 [392:38:25<52:52:40,  9.31s/it]
 91%|█████████ | 204555/225000 [392:38:30<46:09:11,  8.13s/it]
 91%|█████████ | 204556/225000 [392:38:35<40:18:29,  7.10s/it]
 91%|█████████ | 204557/225000 [392:38:44<43:50:55,  7.72s/it]
 91%|█████████ | 204558/225000 [392:38:49<37:57:11,  6.68s/it]
 91%|█████████ | 204559/225000 [392:38:58<41:58:48,  7.39s/it]
 91%|█████████ | 204560/225000 [392:39:07<45:10:27,  7.96s/it]
 91%|█████████ | 204561/225000 [392:39:14<44:16:41,  7.80s/it]
 91%|█████████ | 204562/225000 [392:39:22<44:13:05,  7.79s/it]
 91%|█████████ | 204563/225000 [392:39:33<49:07:02,  8.65s/it]
 91%|█████████ | 204564/225000 [392:39:45<54:33:44,  9.61s/it]
 91%|█████████ | 204565/225000 [392:39:54<54:36:18,  9.62s/it]
 91%|█████████ | 204566/225000 [392:40:00<48:05:49,  8.47s/it]
 91%|█████████ | 204567/225000 [392:40:06<44:01:46,  7.76s/it]
 91%|█████████ | 204568/225000 [392:40:14<44:38:15,  7.86s/it]
 91%|█████████ | 204569/225000 [392:40:25<49:45:54,  8.77s/it]
 91%|█████████ | 204570/225000 [392:40:33<47:29:17,  8.37s/it]
 91%|█████████ | 204571/225000 [392:40:45<53:55:33,  9.50s/it]
 91%|█████████ | 204572/225000 [392:40:57<59:11:19, 10.43s/it]
 91%|█████████ | 204573/225000 [392:41:07<58:34:24, 10.32s/it]
 91%|█████████ | 204574/225000 [392:41:15<54:49:29,  9.66s/it]
 91%|█████████ | 204575/225000 [392:41:24<52:45:31,  9.30s/it]
 91%|█████████ | 204576/225000 [392:41:34<53:31:37,  9.43s/it]
 91%|█████████ | 204577/225000 [392:41:40<48:43:47,  8.59s/it]
 91%|█████████ | 204578/225000 [392:41:51<52:27:50,  9.25s/it]
 91%|█████████ | 204579/225000 [392:42:00<52:40:30,  9.29s/it]
 91%|█████████ | 204580/225000 [392:42:12<55:40:04,  9.81s/it]
 91%|█████████ | 204581/225000 [392:42:22<56:15:14,  9.92s/it]
 91%|█████████ | 204582/225000 [392:42:26<47:04:14,  8.30s/it]
 91%|█████████ | 204583/225000 [392:42:36<49:31:04,  8.73s/it]
 91%|█████████ | 204584/225000 [392:42:44<48:39:37,  8.58s/it]
 91%|█████████ | 204585/225000 [392:42:53<48:50:38,  8.61s/it]
 91%|█████████ | 204586/225000 [392:42:58<43:37:27,  7.69s/it]
 91%|█████████ | 204587/225000 [392:43:08<47:15:08,  8.33s/it]
 91%|█████████ | 204588/225000 [392:43:19<51:17:03,  9.04s/it]
 91%|█████████ | 204589/225000 [392:43:28<50:30:00,  8.91s/it]
 91%|█████████ | 204590/225000 [392:43:37<52:00:00,  9.17s/it]
 91%|█████████ | 204591/225000 [392:43:43<45:23:11,  8.01s/it]
 91%|█████████ | 204592/225000 [392:43:53<49:48:55,  8.79s/it]
 91%|█████████ | 204593/225000 [392:43:59<44:30:54,  7.85s/it]
 91%|█████████ | 204594/225000 [392:44:05<42:17:32,  7.46s/it]
 91%|█████████ | 204595/225000 [392:44:11<39:37:57,  6.99s/it]
 91%|█████████ | 204596/225000 [392:44:25<50:10:01,  8.85s/it]
 91%|█████████ | 204597/225000 [392:44:39<60:05:51, 10.60s/it]
 91%|█████████ | 204598/225000 [392:44:51<62:01:59, 10.95s/it]
 91%|█████████ | 204599/225000 [392:44:58<55:42:45,  9.83s/it]
 91%|█████████ | 204600/225000 [392:45:05<50:08:30,  8.85s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.123', 'learning_rate': '1.113e-06', 'epoch': '0.9093', 'num_input_tokens_seen': 661202904, 'train_runtime': '1.414e+06', 'train_tokens_per_second': '467.6'}

 91%|█████████ | 204600/225000 [392:45:05<50:08:30,  8.85s/it]
 91%|█████████ | 204601/225000 [392:45:12<47:29:10,  8.38s/it]
 91%|█████████ | 204602/225000 [392:45:24<53:11:15,  9.39s/it]
 91%|█████████ | 204603/225000 [392:45:30<48:32:55,  8.57s/it]
 91%|█████████ | 204604/225000 [392:45:42<53:26:07,  9.43s/it]
 91%|█████████ | 204605/225000 [392:45:51<53:11:42,  9.39s/it]
 91%|█████████ | 204606/225000 [392:46:02<55:59:11,  9.88s/it]
 91%|█████████ | 204607/225000 [392:46:08<49:09:43,  8.68s/it]
 91%|█████████ | 204608/225000 [392:46:20<53:54:03,  9.52s/it]
 91%|█████████ | 204609/225000 [392:46:28<52:20:13,  9.24s/it]
 91%|█████████ | 204610/225000 [392:46:39<55:33:48,  9.81s/it]
 91%|█████████ | 204611/225000 [392:46:46<50:24:31,  8.90s/it]
 91%|█████████ | 204612/225000 [392:46:55<50:48:15,  8.97s/it]
 91%|█████████ | 204613/225000 [392:47:02<46:58:50,  8.30s/it]
 91%|█████████ | 204614/225000 [392:47:07<41:47:44,  7.38s/it]
 91%|█████████ | 204615/225000 [392:47:20<50:19:18,  8.89s/it]
 91%|█████████ | 204616/225000 [392:47:29<51:34:25,  9.11s/it]
 91%|█████████ | 204617/225000 [392:47:40<53:51:20,  9.51s/it]
 91%|█████████ | 204618/225000 [392:47:47<49:45:26,  8.79s/it]
 91%|█████████ | 204619/225000 [392:47:51<42:54:08,  7.58s/it]
 91%|█████████ | 204620/225000 [392:48:05<52:53:28,  9.34s/it]
 91%|█████████ | 204621/225000 [392:48:10<45:59:28,  8.12s/it]
 91%|█████████ | 204622/225000 [392:48:18<45:44:52,  8.08s/it]
 91%|█████████ | 204623/225000 [392:48:29<50:37:14,  8.94s/it]
 91%|█████████ | 204624/225000 [392:48:36<46:26:08,  8.20s/it]
 91%|█████████ | 204625/225000 [392:48:47<52:19:57,  9.25s/it]
 91%|█████████ | 204626/225000 [392:48:55<50:15:30,  8.88s/it]
 91%|█████████ | 204627/225000 [392:49:03<47:22:17,  8.37s/it]
 91%|█████████ | 204628/225000 [392:49:08<43:15:53,  7.65s/it]
 91%|█████████ | 204629/225000 [392:49:15<41:51:40,  7.40s/it]
 91%|█████████ | 204630/225000 [392:49:23<41:59:48,  7.42s/it]
 91%|█████████ | 204631/225000 [392:49:32<44:33:18,  7.87s/it]
 91%|█████████ | 204632/225000 [392:49:38<42:28:30,  7.51s/it]
 91%|█████████ | 204633/225000 [392:49:52<52:12:42,  9.23s/it]
 91%|█████████ | 204634/225000 [392:49:58<46:57:44,  8.30s/it]
 91%|█████████ | 204635/225000 [392:50:04<43:43:25,  7.73s/it]
 91%|█████████ | 204636/225000 [392:50:11<42:25:36,  7.50s/it]
 91%|█████████ | 204637/225000 [392:50:22<48:54:44,  8.65s/it]
 91%|█████████ | 204638/225000 [392:50:33<51:37:20,  9.13s/it]
 91%|█████████ | 204639/225000 [392:50:41<50:18:25,  8.89s/it]
 91%|█████████ | 204640/225000 [392:50:51<51:38:43,  9.13s/it]
 91%|█████████ | 204641/225000 [392:51:02<55:04:52,  9.74s/it]
 91%|█████████ | 204642/225000 [392:51:12<56:34:50, 10.01s/it]
 91%|█████████ | 204643/225000 [392:51:20<51:34:14,  9.12s/it]
 91%|█████████ | 204644/225000 [392:51:31<55:16:58,  9.78s/it]
 91%|█████████ | 204645/225000 [392:51:39<51:51:08,  9.17s/it]
 91%|█████████ | 204646/225000 [392:51:45<46:19:20,  8.19s/it]
 91%|█████████ | 204647/225000 [392:51:51<43:41:29,  7.73s/it]
 91%|█████████ | 204648/225000 [392:52:00<44:56:12,  7.95s/it]
 91%|█████████ | 204649/225000 [392:52:09<47:22:12,  8.38s/it]
 91%|█████████ | 204650/225000 [392:52:17<46:52:57,  8.29s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.073', 'learning_rate': '1.107e-06', 'epoch': '0.9096', 'num_input_tokens_seen': 661439632, 'train_runtime': '1.414e+06', 'train_tokens_per_second': '467.7'}

 91%|█████████ | 204650/225000 [392:52:17<46:52:57,  8.29s/it]
 91%|█████████ | 204651/225000 [392:52:28<51:40:38,  9.14s/it]
 91%|█████████ | 204652/225000 [392:52:36<49:13:22,  8.71s/it]
 91%|█████████ | 204653/225000 [392:52:43<46:44:40,  8.27s/it]
 91%|█████████ | 204654/225000 [392:52:54<51:22:03,  9.09s/it]
 91%|█████████ | 204655/225000 [392:53:03<51:13:08,  9.06s/it]
 91%|█████████ | 204656/225000 [392:53:08<43:50:51,  7.76s/it]
 91%|█████████ | 204657/225000 [392:53:18<48:17:05,  8.54s/it]
 91%|█████████ | 204658/225000 [392:53:24<43:36:29,  7.72s/it]
 91%|█████████ | 204659/225000 [392:53:33<45:23:12,  8.03s/it]
 91%|█████████ | 204660/225000 [392:53:42<47:06:39,  8.34s/it]
 91%|█████████ | 204661/225000 [392:53:50<45:57:36,  8.13s/it]
 91%|█████████ | 204662/225000 [392:53:59<47:47:30,  8.46s/it]
 91%|█████████ | 204663/225000 [392:54:16<62:15:11, 11.02s/it]
 91%|█████████ | 204664/225000 [392:54:22<53:34:51,  9.49s/it]
 91%|█████████ | 204665/225000 [392:54:29<49:32:15,  8.77s/it]
 91%|█████████ | 204666/225000 [392:54:35<45:41:25,  8.09s/it]
 91%|█████████ | 204667/225000 [392:54:45<48:47:18,  8.64s/it]
 91%|█████████ | 204668/225000 [392:54:50<43:03:49,  7.62s/it]
 91%|█████████ | 204669/225000 [392:54:56<39:18:31,  6.96s/it]
 91%|█████████ | 204670/225000 [392:55:09<49:29:44,  8.76s/it]
 91%|█████████ | 204671/225000 [392:55:16<46:10:57,  8.18s/it]
 91%|█████████ | 204672/225000 [392:55:24<45:47:20,  8.11s/it]
 91%|█████████ | 204673/225000 [392:55:37<55:10:37,  9.77s/it]
 91%|█████████ | 204674/225000 [392:55:46<52:53:25,  9.37s/it]
 91%|█████████ | 204675/225000 [392:55:53<48:45:02,  8.63s/it]
 91%|█████████ | 204676/225000 [392:56:03<51:51:23,  9.19s/it]
 91%|█████████ | 204677/225000 [392:56:13<53:32:28,  9.48s/it]
 91%|█████████ | 204678/225000 [392:56:19<46:42:09,  8.27s/it]
 91%|█████████ | 204679/225000 [392:56:29<50:52:16,  9.01s/it]
 91%|█████████ | 204680/225000 [392:56:36<46:57:28,  8.32s/it]
 91%|█████████ | 204681/225000 [392:56:42<43:01:44,  7.62s/it]
 91%|█████████ | 204682/225000 [392:56:48<40:00:17,  7.09s/it]
 91%|█████████ | 204683/225000 [392:56:55<39:22:24,  6.98s/it]
 91%|█████████ | 204684/225000 [392:57:06<47:24:37,  8.40s/it]
 91%|█████████ | 204685/225000 [392:57:13<43:59:12,  7.79s/it]
 91%|█████████ | 204686/225000 [392:57:24<49:18:53,  8.74s/it]
 91%|█████████ | 204687/225000 [392:57:30<44:35:30,  7.90s/it]
 91%|█████████ | 204688/225000 [392:57:44<54:39:23,  9.69s/it]
 91%|█████████ | 204689/225000 [392:57:55<57:19:48, 10.16s/it]
 91%|█████████ | 204690/225000 [392:58:03<53:30:40,  9.49s/it]
 91%|█████████ | 204691/225000 [392:58:11<51:08:50,  9.07s/it]
 91%|█████████ | 204692/225000 [392:58:18<47:47:43,  8.47s/it]
 91%|█████████ | 204693/225000 [392:58:29<52:44:07,  9.35s/it]
 91%|█████████ | 204694/225000 [392:58:38<51:16:05,  9.09s/it]
 91%|█████████ | 204695/225000 [392:58:48<52:35:55,  9.33s/it]
 91%|█████████ | 204696/225000 [392:58:54<48:14:43,  8.55s/it]
 91%|█████████ | 204697/225000 [392:59:07<55:43:57,  9.88s/it]
 91%|█████████ | 204698/225000 [392:59:13<48:27:55,  8.59s/it]
 91%|█████████ | 204699/225000 [392:59:24<53:05:01,  9.41s/it]
 91%|█████████ | 204700/225000 [392:59:28<43:17:05,  7.68s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.009', 'learning_rate': '1.102e-06', 'epoch': '0.9098', 'num_input_tokens_seen': 661670072, 'train_runtime': '1.415e+06', 'train_tokens_per_second': '467.7'}

 91%|█████████ | 204700/225000 [392:59:28<43:17:05,  7.68s/it]
 91%|█████████ | 204701/225000 [392:59:34<39:49:22,  7.06s/it]
 91%|█████████ | 204702/225000 [392:59:41<41:11:44,  7.31s/it]
 91%|█████████ | 204703/225000 [392:59:48<39:43:48,  7.05s/it]
 91%|█████████ | 204704/225000 [392:59:54<38:18:18,  6.79s/it]
 91%|█████████ | 204705/225000 [393:00:04<44:17:22,  7.86s/it]
 91%|█████████ | 204706/225000 [393:00:12<43:30:10,  7.72s/it]
 91%|█████████ | 204707/225000 [393:00:25<52:23:14,  9.29s/it]
 91%|█████████ | 204708/225000 [393:00:30<46:07:47,  8.18s/it]
 91%|█████████ | 204709/225000 [393:00:36<41:07:42,  7.30s/it]
 91%|█████████ | 204710/225000 [393:00:46<47:06:05,  8.36s/it]
 91%|█████████ | 204711/225000 [393:00:54<45:04:07,  8.00s/it]
 91%|█████████ | 204712/225000 [393:01:02<46:01:11,  8.17s/it]
 91%|█████████ | 204713/225000 [393:01:09<44:20:23,  7.87s/it]
 91%|█████████ | 204714/225000 [393:01:17<44:32:04,  7.90s/it]
 91%|█████████ | 204715/225000 [393:01:32<55:44:56,  9.89s/it]
 91%|█████████ | 204716/225000 [393:01:40<53:38:34,  9.52s/it]
 91%|█████████ | 204717/225000 [393:01:45<45:25:10,  8.06s/it]
 91%|█████████ | 204718/225000 [393:01:56<50:13:19,  8.91s/it]
 91%|█████████ | 204719/225000 [393:02:05<50:42:55,  9.00s/it]
 91%|█████████ | 204720/225000 [393:02:12<47:09:21,  8.37s/it]
 91%|█████████ | 204721/225000 [393:02:20<45:52:55,  8.15s/it]
 91%|█████████ | 204722/225000 [393:02:31<51:07:31,  9.08s/it]
 91%|█████████ | 204723/225000 [393:02:41<52:42:09,  9.36s/it]
 91%|█████████ | 204724/225000 [393:02:47<47:18:22,  8.40s/it]
 91%|█████████ | 204725/225000 [393:02:59<53:30:01,  9.50s/it]
 91%|█████████ | 204726/225000 [393:03:07<51:14:06,  9.10s/it]
 91%|█████████ | 204727/225000 [393:03:15<48:35:30,  8.63s/it]
 91%|█████████ | 204728/225000 [393:03:24<49:59:51,  8.88s/it]
 91%|█████████ | 204729/225000 [393:03:31<46:33:04,  8.27s/it]
 91%|█████████ | 204730/225000 [393:03:47<58:57:11, 10.47s/it]
 91%|█████████ | 204731/225000 [393:03:52<49:55:55,  8.87s/it]
 91%|█████████ | 204732/225000 [393:04:03<53:40:49,  9.53s/it]
 91%|█████████ | 204733/225000 [393:04:10<48:43:34,  8.66s/it]
 91%|█████████ | 204734/225000 [393:04:15<43:15:43,  7.68s/it]
 91%|█████████ | 204735/225000 [393:04:28<51:16:11,  9.11s/it]
 91%|█████████ | 204736/225000 [393:04:38<52:59:12,  9.41s/it]
 91%|█████████ | 204737/225000 [393:04:47<52:50:03,  9.39s/it]
 91%|█████████ | 204738/225000 [393:04:58<55:01:24,  9.78s/it]
 91%|█████████ | 204739/225000 [393:05:05<51:37:44,  9.17s/it]
 91%|█████████ | 204740/225000 [393:05:13<48:44:47,  8.66s/it]
 91%|█████████ | 204741/225000 [393:05:21<48:20:31,  8.59s/it]
 91%|█████████ | 204742/225000 [393:05:27<43:16:42,  7.69s/it]
 91%|█████████ | 204743/225000 [393:05:33<39:59:09,  7.11s/it]
 91%|█████████ | 204744/225000 [393:05:38<37:11:27,  6.61s/it]
 91%|█████████ | 204745/225000 [393:05:49<43:53:42,  7.80s/it]
 91%|█████████ | 204746/225000 [393:06:00<49:54:26,  8.87s/it]
 91%|█████████ | 204747/225000 [393:06:07<47:21:10,  8.42s/it]
 91%|█████████ | 204748/225000 [393:06:23<58:43:51, 10.44s/it]
 91%|█████████ | 204749/225000 [393:06:29<52:38:02,  9.36s/it]
 91%|█████████ | 204750/225000 [393:06:40<54:10:20,  9.63s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.27', 'learning_rate': '1.097e-06', 'epoch': '0.91', 'num_input_tokens_seen': 661902808, 'train_runtime': '1.415e+06', 'train_tokens_per_second': '467.7'}

 91%|█████████ | 204750/225000 [393:06:40<54:10:20,  9.63s/it]
 91%|█████████ | 204751/225000 [393:06:48<51:53:03,  9.22s/it]
 91%|█████████ | 204752/225000 [393:06:55<48:58:11,  8.71s/it]
 91%|█████████ | 204753/225000 [393:07:05<50:12:06,  8.93s/it]
 91%|█████████ | 204754/225000 [393:07:18<57:40:52, 10.26s/it]
 91%|█████████ | 204755/225000 [393:07:22<47:03:52,  8.37s/it]
 91%|█████████ | 204756/225000 [393:07:29<43:42:10,  7.77s/it]
 91%|█████████ | 204757/225000 [393:07:37<44:23:36,  7.89s/it]
 91%|█████████ | 204758/225000 [393:07:44<43:20:32,  7.71s/it]
 91%|█████████ | 204759/225000 [393:07:52<44:17:31,  7.88s/it]
 91%|█████████ | 204760/225000 [393:07:59<42:32:35,  7.57s/it]
 91%|█████████ | 204761/225000 [393:08:06<41:04:02,  7.30s/it]
 91%|█████████ | 204762/225000 [393:08:13<40:00:55,  7.12s/it]
 91%|█████████ | 204763/225000 [393:08:21<41:57:36,  7.46s/it]
 91%|█████████ | 204764/225000 [393:08:28<40:45:37,  7.25s/it]
 91%|█████████ | 204765/225000 [393:08:34<39:24:34,  7.01s/it]
 91%|█████████ | 204766/225000 [393:08:47<48:55:17,  8.70s/it]
 91%|█████████ | 204767/225000 [393:08:54<47:19:26,  8.42s/it]
 91%|█████████ | 204768/225000 [393:09:06<52:28:53,  9.34s/it]
 91%|█████████ | 204769/225000 [393:09:12<46:28:09,  8.27s/it]
 91%|█████████ | 204770/225000 [393:09:24<52:32:42,  9.35s/it]
 91%|█████████ | 204771/225000 [393:09:28<44:24:45,  7.90s/it]
 91%|█████████ | 204772/225000 [393:09:37<46:01:10,  8.19s/it]
 91%|█████████ | 204773/225000 [393:09:45<46:37:48,  8.30s/it]
 91%|█████████ | 204774/225000 [393:09:55<48:13:07,  8.58s/it]
 91%|█████████ | 204775/225000 [393:10:02<45:52:52,  8.17s/it]
 91%|█████████ | 204776/225000 [393:10:11<46:39:22,  8.31s/it]
 91%|█████████ | 204777/225000 [393:10:20<48:09:46,  8.57s/it]
 91%|█████████ | 204778/225000 [393:10:25<43:21:53,  7.72s/it]
 91%|█████████ | 204779/225000 [393:10:30<38:45:50,  6.90s/it]
 91%|█████████ | 204780/225000 [393:10:39<42:14:04,  7.52s/it]
 91%|█████████ | 204781/225000 [393:10:50<46:53:18,  8.35s/it]
 91%|█████████ | 204782/225000 [393:10:59<49:01:01,  8.73s/it]
 91%|█████████ | 204783/225000 [393:11:05<44:31:20,  7.93s/it]
 91%|█████████ | 204784/225000 [393:11:14<46:18:41,  8.25s/it]
 91%|█████████ | 204785/225000 [393:11:25<50:31:42,  9.00s/it]
 91%|█████████ | 204786/225000 [393:11:32<46:41:45,  8.32s/it]
 91%|█████████ | 204787/225000 [393:11:42<48:56:24,  8.72s/it]
 91%|█████████ | 204788/225000 [393:11:49<47:02:01,  8.38s/it]
 91%|█████████ | 204789/225000 [393:12:00<51:02:30,  9.09s/it]
 91%|█████████ | 204790/225000 [393:12:06<45:45:43,  8.15s/it]
 91%|█████████ | 204791/225000 [393:12:18<51:48:54,  9.23s/it]
 91%|█████████ | 204792/225000 [393:12:24<46:44:20,  8.33s/it]
 91%|█████████ | 204793/225000 [393:12:28<40:25:54,  7.20s/it]
 91%|█████████ | 204794/225000 [393:12:37<42:58:45,  7.66s/it]
 91%|█████████ | 204795/225000 [393:12:47<47:03:39,  8.39s/it]
 91%|█████████ | 204796/225000 [393:12:58<51:38:29,  9.20s/it]
 91%|█████████ | 204797/225000 [393:13:08<51:54:18,  9.25s/it]
 91%|█████████ | 204798/225000 [393:13:17<52:41:18,  9.39s/it]
 91%|█████████ | 204799/225000 [393:13:22<45:18:08,  8.07s/it]
 91%|█████████ | 204800/225000 [393:13:28<41:06:26,  7.33s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.385', 'learning_rate': '1.091e-06', 'epoch': '0.9102', 'num_input_tokens_seen': 662124536, 'train_runtime': '1.416e+06', 'train_tokens_per_second': '467.7'}

 91%|█████████ | 204800/225000 [393:13:28<41:06:26,  7.33s/it]
 91%|█████████ | 204801/225000 [393:13:35<40:53:14,  7.29s/it]
 91%|█████████ | 204802/225000 [393:13:47<49:06:19,  8.75s/it]
 91%|█████████ | 204803/225000 [393:13:55<47:21:50,  8.44s/it]
 91%|█████████ | 204804/225000 [393:14:04<48:17:01,  8.61s/it]
 91%|█████████ | 204805/225000 [393:14:11<44:47:09,  7.98s/it]
 91%|█████████ | 204806/225000 [393:14:21<48:55:56,  8.72s/it]
 91%|█████████ | 204807/225000 [393:14:29<47:10:33,  8.41s/it]
 91%|█████████ | 204808/225000 [393:14:45<60:24:11, 10.77s/it]
 91%|█████████ | 204809/225000 [393:14:50<51:03:31,  9.10s/it]
 91%|█████████ | 204810/225000 [393:15:00<51:46:46,  9.23s/it]
 91%|█████████ | 204811/225000 [393:15:05<44:39:20,  7.96s/it]
 91%|█████████ | 204812/225000 [393:15:16<49:29:59,  8.83s/it]
 91%|█████████ | 204813/225000 [393:15:23<46:51:55,  8.36s/it]
 91%|█████████ | 204814/225000 [393:15:33<50:44:03,  9.05s/it]
 91%|█████████ | 204815/225000 [393:15:43<50:47:15,  9.06s/it]
 91%|█████████ | 204816/225000 [393:15:50<48:54:57,  8.72s/it]
 91%|█████████ | 204817/225000 [393:16:03<54:46:12,  9.77s/it]
 91%|█████████ | 204818/225000 [393:16:10<51:12:40,  9.13s/it]
 91%|█████████ | 204819/225000 [393:16:17<47:15:37,  8.43s/it]
 91%|█████████ | 204820/225000 [393:16:24<43:54:32,  7.83s/it]
 91%|█████████ | 204821/225000 [393:16:36<51:00:08,  9.10s/it]
 91%|█████████ | 204822/225000 [393:16:44<49:23:44,  8.81s/it]
 91%|█████████ | 204823/225000 [393:16:51<47:21:35,  8.45s/it]
 91%|█████████ | 204824/225000 [393:17:00<47:07:08,  8.41s/it]
 91%|█████████ | 204825/225000 [393:17:08<47:01:14,  8.39s/it]
 91%|█████████ | 204826/225000 [393:17:22<55:52:30,  9.97s/it]
 91%|█████████ | 204827/225000 [393:17:31<54:15:07,  9.68s/it]
 91%|█████████ | 204828/225000 [393:17:39<51:15:57,  9.15s/it]
 91%|█████████ | 204829/225000 [393:17:48<51:34:44,  9.21s/it]
 91%|█████████ | 204830/225000 [393:17:52<42:56:58,  7.67s/it]
 91%|█████████ | 204831/225000 [393:18:07<55:06:11,  9.84s/it]
 91%|█████████ | 204832/225000 [393:18:11<46:04:07,  8.22s/it]
 91%|█████████ | 204833/225000 [393:18:22<50:55:38,  9.09s/it]
 91%|█████████ | 204834/225000 [393:18:34<54:46:11,  9.78s/it]
 91%|█████████ | 204835/225000 [393:18:39<46:11:51,  8.25s/it]
 91%|█████████ | 204836/225000 [393:18:45<43:39:56,  7.80s/it]
 91%|█████████ | 204837/225000 [393:18:58<52:34:56,  9.39s/it]
 91%|█████████ | 204838/225000 [393:19:07<51:09:20,  9.13s/it]
 91%|█████████ | 204839/225000 [393:19:13<45:54:45,  8.20s/it]
 91%|█████████ | 204840/225000 [393:19:24<50:21:03,  8.99s/it]
 91%|█████████ | 204841/225000 [393:19:29<43:37:41,  7.79s/it]
 91%|█████████ | 204842/225000 [393:19:35<40:34:18,  7.25s/it]
 91%|█████████ | 204843/225000 [393:19:44<44:36:49,  7.97s/it]
 91%|█████████ | 204844/225000 [393:19:54<47:55:20,  8.56s/it]
 91%|█████████ | 204845/225000 [393:20:02<45:47:37,  8.18s/it]
 91%|█████████ | 204846/225000 [393:20:06<39:37:05,  7.08s/it]
 91%|█████████ | 204847/225000 [393:20:15<43:15:23,  7.73s/it]
 91%|█████████ | 204848/225000 [393:20:26<48:02:07,  8.58s/it]
 91%|█████████ | 204849/225000 [393:20:36<50:09:25,  8.96s/it]
 91%|█████████ | 204850/225000 [393:20:41<44:05:31,  7.88s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.341', 'learning_rate': '1.086e-06', 'epoch': '0.9104', 'num_input_tokens_seen': 662355872, 'train_runtime': '1.416e+06', 'train_tokens_per_second': '467.8'}

 91%|█████████ | 204850/225000 [393:20:41<44:05:31,  7.88s/it]
 91%|█████████ | 204851/225000 [393:20:47<40:44:11,  7.28s/it]
 91%|█████████ | 204852/225000 [393:20:59<47:59:03,  8.57s/it]
 91%|█████████ | 204853/225000 [393:21:05<43:38:15,  7.80s/it]
 91%|█████████ | 204854/225000 [393:21:12<42:54:56,  7.67s/it]
 91%|█████████ | 204855/225000 [393:21:20<43:42:19,  7.81s/it]
 91%|█████████ | 204856/225000 [393:21:28<43:51:56,  7.84s/it]
 91%|█████████ | 204857/225000 [393:21:40<51:09:47,  9.14s/it]
 91%|█████████ | 204858/225000 [393:21:45<43:13:58,  7.73s/it]
 91%|█████████ | 204859/225000 [393:21:54<45:51:00,  8.20s/it]
 91%|█████████ | 204860/225000 [393:22:01<44:39:05,  7.98s/it]
 91%|█████████ | 204861/225000 [393:22:09<44:08:16,  7.89s/it]
 91%|█████████ | 204862/225000 [393:22:19<46:56:31,  8.39s/it]
 91%|█████████ | 204863/225000 [393:22:31<53:48:37,  9.62s/it]
 91%|█████████ | 204864/225000 [393:22:38<49:36:56,  8.87s/it]
 91%|█████████ | 204865/225000 [393:22:51<55:37:41,  9.95s/it]
 91%|█████████ | 204866/225000 [393:23:00<54:01:46,  9.66s/it]
 91%|█████████ | 204867/225000 [393:23:07<50:25:44,  9.02s/it]
 91%|█████████ | 204868/225000 [393:23:16<49:24:54,  8.84s/it]
 91%|█████████ | 204869/225000 [393:23:30<57:56:10, 10.36s/it]
 91%|█████████ | 204870/225000 [393:23:37<52:31:38,  9.39s/it]
 91%|█████████ | 204871/225000 [393:23:44<48:18:45,  8.64s/it]
 91%|█████████ | 204872/225000 [393:23:55<52:16:15,  9.35s/it]
 91%|█████████ | 204873/225000 [393:24:03<51:08:37,  9.15s/it]
 91%|█████████ | 204874/225000 [393:24:10<46:18:26,  8.28s/it]
 91%|█████████ | 204875/225000 [393:24:18<47:15:13,  8.45s/it]
 91%|█████████ | 204876/225000 [393:24:24<42:02:49,  7.52s/it]
 91%|█████████ | 204877/225000 [393:24:39<55:22:56,  9.91s/it]
 91%|█████████ | 204878/225000 [393:24:45<48:55:54,  8.75s/it]
 91%|█████████ | 204879/225000 [393:24:53<46:29:15,  8.32s/it]
 91%|█████████ | 204880/225000 [393:25:08<58:24:00, 10.45s/it]
 91%|█████████ | 204881/225000 [393:25:12<48:19:37,  8.65s/it]
 91%|█████████ | 204882/225000 [393:25:22<50:20:00,  9.01s/it]
 91%|█████████ | 204883/225000 [393:25:31<49:46:48,  8.91s/it]
 91%|█████████ | 204884/225000 [393:25:42<53:22:27,  9.55s/it]
 91%|█████████ | 204885/225000 [393:25:53<56:07:49, 10.05s/it]
 91%|█████████ | 204886/225000 [393:26:01<52:33:26,  9.41s/it]
 91%|█████████ | 204887/225000 [393:26:07<45:58:09,  8.23s/it]
 91%|█████████ | 204888/225000 [393:26:17<49:41:55,  8.90s/it]
 91%|█████████ | 204889/225000 [393:26:28<52:39:18,  9.43s/it]
 91%|█████████ | 204890/225000 [393:26:39<55:07:55,  9.87s/it]
 91%|█████████ | 204891/225000 [393:26:46<50:30:21,  9.04s/it]
 91%|█████████ | 204892/225000 [393:26:58<56:02:43, 10.03s/it]
 91%|█████████ | 204893/225000 [393:27:03<47:16:38,  8.46s/it]
 91%|█████████ | 204894/225000 [393:27:12<48:50:30,  8.75s/it]
 91%|█████████ | 204895/225000 [393:27:22<50:09:36,  8.98s/it]
 91%|█████████ | 204896/225000 [393:27:26<42:31:37,  7.62s/it]
 91%|█████████ | 204897/225000 [393:27:36<45:48:10,  8.20s/it]
 91%|█████████ | 204898/225000 [393:27:42<42:35:45,  7.63s/it]
 91%|█████████ | 204899/225000 [393:27:55<51:00:30,  9.14s/it]
 91%|█████████ | 204900/225000 [393:28:04<50:35:36,  9.06s/it]
                                                              
{'loss': '1.301', 'grad_norm': '0.9173', 'learning_rate': '1.081e-06', 'epoch': '0.9107', 'num_input_tokens_seen': 662591888, 'train_runtime': '1.416e+06', 'train_tokens_per_second': '467.8'}

 91%|█████████ | 204900/225000 [393:28:04<50:35:36,  9.06s/it]
 91%|█████████ | 204901/225000 [393:28:12<49:56:53,  8.95s/it]
 91%|█████████ | 204902/225000 [393:28:21<48:49:50,  8.75s/it]
 91%|█████████ | 204903/225000 [393:28:30<49:34:00,  8.88s/it]
 91%|█████████ | 204904/225000 [393:28:41<53:30:44,  9.59s/it]
 91%|█████████ | 204905/225000 [393:28:53<57:00:09, 10.21s/it]
 91%|█████████ | 204906/225000 [393:28:59<50:56:49,  9.13s/it]
 91%|█████████ | 204907/225000 [393:29:06<46:11:58,  8.28s/it]
 91%|█████████ | 204908/225000 [393:29:12<43:19:56,  7.76s/it]
 91%|█████████ | 204909/225000 [393:29:18<40:23:38,  7.24s/it]
 91%|█████████ | 204910/225000 [393:29:35<56:40:30, 10.16s/it]
 91%|█████████ | 204911/225000 [393:29:43<53:13:42,  9.54s/it]
 91%|█████████ | 204912/225000 [393:29:53<54:21:24,  9.74s/it]
 91%|█████████ | 204913/225000 [393:30:02<52:36:25,  9.43s/it]
 91%|█████████ | 204914/225000 [393:30:12<54:09:17,  9.71s/it]
 91%|█████████ | 204915/225000 [393:30:24<57:27:23, 10.30s/it]
 91%|█████████ | 204916/225000 [393:30:30<49:29:32,  8.87s/it]
 91%|█████████ | 204917/225000 [393:30:40<51:56:29,  9.31s/it]
 91%|█████████ | 204918/225000 [393:30:47<48:01:08,  8.61s/it]
 91%|█████████ | 204919/225000 [393:30:56<48:41:32,  8.73s/it]
 91%|█████████ | 204920/225000 [393:31:07<53:14:28,  9.55s/it]
 91%|█████████ | 204921/225000 [393:31:13<45:52:09,  8.22s/it]
 91%|█████████ | 204922/225000 [393:31:19<42:21:09,  7.59s/it]
 91%|█████████ | 204923/225000 [393:31:30<48:17:33,  8.66s/it]
 91%|█████████ | 204924/225000 [393:31:39<48:55:20,  8.77s/it]
 91%|█████████ | 204925/225000 [393:31:47<47:00:36,  8.43s/it]
 91%|█████████ | 204926/225000 [393:31:53<42:52:41,  7.69s/it]
 91%|█████████ | 204927/225000 [393:32:02<45:52:51,  8.23s/it]
 91%|█████████ | 204928/225000 [393:32:10<45:33:47,  8.17s/it]
 91%|█████████ | 204929/225000 [393:32:16<42:22:01,  7.60s/it]
 91%|█████████ | 204930/225000 [393:32:22<38:39:28,  6.93s/it]
 91%|█████████ | 204931/225000 [393:32:28<38:07:57,  6.84s/it]
 91%|█████████ | 204932/225000 [393:32:42<49:21:27,  8.85s/it]
 91%|█████████ | 204933/225000 [393:32:48<45:26:36,  8.15s/it]
 91%|█████████ | 204934/225000 [393:32:54<41:23:50,  7.43s/it]
 91%|█████████ | 204935/225000 [393:33:04<45:25:05,  8.15s/it]
 91%|█████████ | 204936/225000 [393:33:12<44:29:34,  7.98s/it]
 91%|█████████ | 204937/225000 [393:33:19<44:17:18,  7.95s/it]
 91%|█████████ | 204938/225000 [393:33:32<52:52:03,  9.49s/it]
 91%|█████████ | 204939/225000 [393:33:39<48:35:49,  8.72s/it]
 91%|█████████ | 204940/225000 [393:33:46<45:51:01,  8.23s/it]
 91%|█████████ | 204941/225000 [393:33:53<42:43:42,  7.67s/it]
 91%|█████████ | 204942/225000 [393:34:03<47:17:31,  8.49s/it]
 91%|█████████ | 204943/225000 [393:34:11<46:43:23,  8.39s/it]
 91%|█████████ | 204944/225000 [393:34:21<49:01:28,  8.80s/it]
 91%|█████████ | 204945/225000 [393:34:28<45:34:45,  8.18s/it]
 91%|█████████ | 204946/225000 [393:34:40<52:13:08,  9.37s/it]
 91%|█████████ | 204947/225000 [393:34:51<54:50:19,  9.84s/it]
 91%|█████████ | 204948/225000 [393:35:01<55:50:58, 10.03s/it]
 91%|█████████ | 204949/225000 [393:35:10<52:37:59,  9.45s/it]
 91%|█████████ | 204950/225000 [393:35:18<50:16:01,  9.03s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.208', 'learning_rate': '1.075e-06', 'epoch': '0.9109', 'num_input_tokens_seen': 662824256, 'train_runtime': '1.417e+06', 'train_tokens_per_second': '467.8'}

 91%|█████████ | 204950/225000 [393:35:18<50:16:01,  9.03s/it]
 91%|█████████ | 204951/225000 [393:35:26<48:51:50,  8.77s/it]
 91%|█████████ | 204952/225000 [393:35:32<44:29:15,  7.99s/it]
 91%|█████████ | 204953/225000 [393:35:46<53:55:06,  9.68s/it]
 91%|█████████ | 204954/225000 [393:35:52<48:18:18,  8.67s/it]
 91%|█████████ | 204955/225000 [393:36:04<54:35:16,  9.80s/it]
 91%|█████████ | 204956/225000 [393:36:11<49:24:05,  8.87s/it]
 91%|█████████ | 204957/225000 [393:36:18<45:27:35,  8.17s/it]
 91%|█████████ | 204958/225000 [393:36:26<45:10:09,  8.11s/it]
 91%|█████████ | 204959/225000 [393:36:33<43:18:30,  7.78s/it]
 91%|█████████ | 204960/225000 [393:36:44<50:14:30,  9.03s/it]
 91%|█████████ | 204961/225000 [393:36:55<52:14:33,  9.39s/it]
 91%|█████████ | 204962/225000 [393:37:09<60:29:53, 10.87s/it]
 91%|█████████ | 204963/225000 [393:37:18<57:33:30, 10.34s/it]
 91%|█████████ | 204964/225000 [393:37:27<55:36:41,  9.99s/it]
 91%|█████████ | 204965/225000 [393:37:36<52:36:36,  9.45s/it]
 91%|█████████ | 204966/225000 [393:37:41<46:05:16,  8.28s/it]
 91%|█████████ | 204967/225000 [393:37:53<52:49:39,  9.49s/it]
 91%|█████████ | 204968/225000 [393:38:03<52:53:45,  9.51s/it]
 91%|█████████ | 204969/225000 [393:38:11<50:45:06,  9.12s/it]
 91%|█████████ | 204970/225000 [393:38:18<46:49:29,  8.42s/it]
 91%|█████████ | 204971/225000 [393:38:25<44:26:59,  7.99s/it]
 91%|█████████ | 204972/225000 [393:38:32<42:20:25,  7.61s/it]
 91%|█████████ | 204973/225000 [393:38:38<40:11:20,  7.22s/it]
 91%|█████████ | 204974/225000 [393:38:51<49:58:40,  8.98s/it]
 91%|█████████ | 204975/225000 [393:39:05<57:38:35, 10.36s/it]
 91%|█████████ | 204976/225000 [393:39:12<52:29:00,  9.44s/it]
 91%|█████████ | 204977/225000 [393:39:23<55:12:30,  9.93s/it]
 91%|█████████ | 204978/225000 [393:39:30<50:44:13,  9.12s/it]
 91%|█████████ | 204979/225000 [393:39:39<49:42:45,  8.94s/it]
 91%|█████████ | 204980/225000 [393:39:46<46:21:47,  8.34s/it]
 91%|█████████ | 204981/225000 [393:39:53<45:11:28,  8.13s/it]
 91%|█████████ | 204982/225000 [393:40:00<43:27:25,  7.82s/it]
 91%|█████████ | 204983/225000 [393:40:08<43:02:31,  7.74s/it]
 91%|█████████ | 204984/225000 [393:40:17<45:55:08,  8.26s/it]
 91%|█████████ | 204985/225000 [393:40:26<46:45:49,  8.41s/it]
 91%|█████████ | 204986/225000 [393:40:32<42:14:48,  7.60s/it]
 91%|█████████ | 204987/225000 [393:40:44<49:28:05,  8.90s/it]
 91%|█████████ | 204988/225000 [393:40:50<45:38:36,  8.21s/it]
 91%|█████████ | 204989/225000 [393:40:56<41:55:22,  7.54s/it]
 91%|█████████ | 204990/225000 [393:41:05<44:00:57,  7.92s/it]
 91%|█████████ | 204991/225000 [393:41:13<43:33:38,  7.84s/it]
 91%|█████████ | 204992/225000 [393:41:22<46:11:37,  8.31s/it]
 91%|█████████ | 204993/225000 [393:41:29<43:48:12,  7.88s/it]
 91%|█████████ | 204994/225000 [393:41:34<38:18:11,  6.89s/it]
 91%|█████████ | 204995/225000 [393:41:45<45:59:21,  8.28s/it]
 91%|█████████ | 204996/225000 [393:41:51<41:15:41,  7.43s/it]
 91%|█████████ | 204997/225000 [393:42:02<47:04:22,  8.47s/it]
 91%|█████████ | 204998/225000 [393:42:10<46:12:23,  8.32s/it]
 91%|█████████ | 204999/225000 [393:42:19<48:33:25,  8.74s/it]
 91%|█████████ | 205000/225000 [393:42:27<47:29:52,  8.55s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.296', 'learning_rate': '1.07e-06', 'epoch': '0.9111', 'num_input_tokens_seen': 663059944, 'train_runtime': '1.417e+06', 'train_tokens_per_second': '467.8'}

 91%|█████████ | 205000/225000 [393:42:27<47:29:52,  8.55s/it]
 91%|█████████ | 205001/225000 [393:42:34<44:27:01,  8.00s/it]
 91%|█████████ | 205002/225000 [393:42:48<53:54:54,  9.71s/it]
 91%|█████████ | 205003/225000 [393:42:58<54:27:01,  9.80s/it]
 91%|█████████ | 205004/225000 [393:43:07<53:34:47,  9.65s/it]
 91%|█████████ | 205005/225000 [393:43:12<46:05:07,  8.30s/it]
 91%|█████████ | 205006/225000 [393:43:22<47:51:57,  8.62s/it]
 91%|█████████ | 205007/225000 [393:43:29<45:34:15,  8.21s/it]
 91%|█████████ | 205008/225000 [393:43:34<40:45:21,  7.34s/it]
 91%|█████████ | 205009/225000 [393:43:41<40:03:57,  7.22s/it]
 91%|█████████ | 205010/225000 [393:43:50<43:10:35,  7.78s/it]
 91%|█████████ | 205011/225000 [393:44:05<54:58:58,  9.90s/it]
 91%|█████████ | 205012/225000 [393:44:14<52:59:43,  9.54s/it]
 91%|█████████ | 205013/225000 [393:44:24<54:19:10,  9.78s/it]
 91%|█████████ | 205014/225000 [393:44:30<47:28:20,  8.55s/it]
 91%|█████████ | 205015/225000 [393:44:38<46:05:45,  8.30s/it]
 91%|█████████ | 205016/225000 [393:44:45<45:21:24,  8.17s/it]
 91%|█████████ | 205017/225000 [393:44:53<45:08:55,  8.13s/it]
 91%|█████████ | 205018/225000 [393:45:04<48:56:34,  8.82s/it]
 91%|█████████ | 205019/225000 [393:45:10<43:55:13,  7.91s/it]
 91%|█████████ | 205020/225000 [393:45:16<42:07:58,  7.59s/it]
 91%|█████████ | 205021/225000 [393:45:28<49:11:04,  8.86s/it]
 91%|█████████ | 205022/225000 [393:45:36<46:32:15,  8.39s/it]
 91%|█████████ | 205023/225000 [393:45:45<47:47:50,  8.61s/it]
 91%|█████████ | 205024/225000 [393:45:55<50:11:39,  9.05s/it]
 91%|█████████ | 205025/225000 [393:46:00<43:39:41,  7.87s/it]
 91%|█████████ | 205026/225000 [393:46:11<48:23:24,  8.72s/it]
 91%|█████████ | 205027/225000 [393:46:22<52:00:09,  9.37s/it]
 91%|█████████ | 205028/225000 [393:46:31<52:39:44,  9.49s/it]
 91%|█████████ | 205029/225000 [393:46:38<47:49:21,  8.62s/it]
 91%|█████████ | 205030/225000 [393:46:44<43:15:40,  7.80s/it]
 91%|█████████ | 205031/225000 [393:46:55<49:40:05,  8.95s/it]
 91%|█████████ | 205032/225000 [393:47:03<47:45:18,  8.61s/it]
 91%|█████████ | 205033/225000 [393:47:10<44:40:23,  8.05s/it]
 91%|█████████ | 205034/225000 [393:47:17<43:33:10,  7.85s/it]
 91%|█████████ | 205035/225000 [393:47:27<46:12:24,  8.33s/it]
 91%|█████████ | 205036/225000 [393:47:41<56:34:36, 10.20s/it]
 91%|█████████ | 205037/225000 [393:47:50<54:16:52,  9.79s/it]
 91%|█████████ | 205038/225000 [393:47:56<47:25:35,  8.55s/it]
 91%|█████████ | 205039/225000 [393:48:00<39:29:05,  7.12s/it]
 91%|█████████ | 205040/225000 [393:48:11<46:20:20,  8.36s/it]
 91%|█████████ | 205041/225000 [393:48:18<44:11:32,  7.97s/it]
 91%|█████████ | 205042/225000 [393:48:28<47:33:04,  8.58s/it]
 91%|█████████ | 205043/225000 [393:48:35<45:38:03,  8.23s/it]
 91%|█████████ | 205044/225000 [393:48:44<46:18:56,  8.36s/it]
 91%|█████████ | 205045/225000 [393:48:54<48:57:07,  8.83s/it]
 91%|█████████ | 205046/225000 [393:49:01<46:00:05,  8.30s/it]
 91%|█████████ | 205047/225000 [393:49:09<44:58:13,  8.11s/it]
 91%|█████████ | 205048/225000 [393:49:15<42:01:03,  7.58s/it]
 91%|█████████ | 205049/225000 [393:49:27<48:41:36,  8.79s/it]
 91%|█████████ | 205050/225000 [393:49:36<48:58:47,  8.84s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.053', 'learning_rate': '1.065e-06', 'epoch': '0.9113', 'num_input_tokens_seen': 663288328, 'train_runtime': '1.418e+06', 'train_tokens_per_second': '467.8'}

 91%|█████████ | 205050/225000 [393:49:36<48:58:47,  8.84s/it]
 91%|█████████ | 205051/225000 [393:49:42<44:33:06,  8.04s/it]
 91%|█████████ | 205052/225000 [393:49:53<50:28:17,  9.11s/it]
 91%|█████████ | 205053/225000 [393:50:00<46:57:42,  8.48s/it]
 91%|█████████ | 205054/225000 [393:50:05<40:54:41,  7.38s/it]
 91%|█████████ | 205055/225000 [393:50:15<45:12:16,  8.16s/it]
 91%|█████████ | 205056/225000 [393:50:23<44:14:29,  7.99s/it]
 91%|█████████ | 205057/225000 [393:50:31<43:58:07,  7.94s/it]
 91%|█████████ | 205058/225000 [393:50:41<48:19:12,  8.72s/it]
 91%|█████████ | 205059/225000 [393:50:48<44:59:41,  8.12s/it]
 91%|█████████ | 205060/225000 [393:50:56<45:03:44,  8.14s/it]
 91%|█████████ | 205061/225000 [393:51:05<46:37:07,  8.42s/it]
 91%|█████████ | 205062/225000 [393:51:13<45:56:42,  8.30s/it]
 91%|█████████ | 205063/225000 [393:51:21<46:06:53,  8.33s/it]
 91%|█████████ | 205064/225000 [393:51:33<51:58:51,  9.39s/it]
 91%|█████████ | 205065/225000 [393:51:38<44:37:48,  8.06s/it]
 91%|█████████ | 205066/225000 [393:51:44<40:26:09,  7.30s/it]
 91%|█████████ | 205067/225000 [393:51:56<47:47:00,  8.63s/it]
 91%|█████████ | 205068/225000 [393:52:01<42:10:28,  7.62s/it]
 91%|█████████ | 205069/225000 [393:52:13<48:55:38,  8.84s/it]
 91%|█████████ | 205070/225000 [393:52:19<44:48:46,  8.09s/it]
 91%|█████████ | 205071/225000 [393:52:26<44:00:21,  7.95s/it]
 91%|█████████ | 205072/225000 [393:52:42<56:09:02, 10.14s/it]
 91%|█████████ | 205073/225000 [393:52:50<53:03:15,  9.58s/it]
 91%|█████████ | 205074/225000 [393:52:59<51:48:28,  9.36s/it]
 91%|█████████ | 205075/225000 [393:53:07<48:59:23,  8.85s/it]
 91%|█████████ | 205076/225000 [393:53:12<43:09:14,  7.80s/it]
 91%|█████████ | 205077/225000 [393:53:26<53:39:45,  9.70s/it]
 91%|█████████ | 205078/225000 [393:53:33<48:51:22,  8.83s/it]
 91%|█████████ | 205079/225000 [393:53:42<50:07:40,  9.06s/it]
 91%|█████████ | 205080/225000 [393:53:53<52:11:17,  9.43s/it]
 91%|█████████ | 205081/225000 [393:54:01<50:42:28,  9.16s/it]
 91%|█████████ | 205082/225000 [393:54:09<48:31:01,  8.77s/it]
 91%|█████████ | 205083/225000 [393:54:17<47:06:35,  8.52s/it]
 91%|█████████ | 205084/225000 [393:54:31<56:00:34, 10.12s/it]
 91%|█████████ | 205085/225000 [393:54:38<51:30:22,  9.31s/it]
 91%|█████████ | 205086/225000 [393:54:47<50:03:14,  9.05s/it]
 91%|█████████ | 205087/225000 [393:54:58<54:21:11,  9.83s/it]
 91%|█████████ | 205088/225000 [393:55:07<52:38:39,  9.52s/it]
 91%|█████████ | 205089/225000 [393:55:12<44:52:17,  8.11s/it]
 91%|█████████ | 205090/225000 [393:55:19<43:17:42,  7.83s/it]
 91%|█████████ | 205091/225000 [393:55:24<39:06:45,  7.07s/it]
 91%|█████████ | 205092/225000 [393:55:30<36:57:48,  6.68s/it]
 91%|█████████ | 205093/225000 [393:55:41<43:34:17,  7.88s/it]
 91%|█████████ | 205094/225000 [393:55:48<41:59:53,  7.60s/it]
 91%|█████████ | 205095/225000 [393:55:56<43:40:46,  7.90s/it]
 91%|█████████ | 205096/225000 [393:56:07<47:51:08,  8.65s/it]
 91%|█████████ | 205097/225000 [393:56:19<53:54:52,  9.75s/it]
 91%|█████████ | 205098/225000 [393:56:25<47:45:36,  8.64s/it]
 91%|█████████ | 205099/225000 [393:56:31<43:40:51,  7.90s/it]
 91%|█████████ | 205100/225000 [393:56:37<39:26:44,  7.14s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.544', 'learning_rate': '1.06e-06', 'epoch': '0.9116', 'num_input_tokens_seen': 663513008, 'train_runtime': '1.418e+06', 'train_tokens_per_second': '467.9'}

 91%|█████████ | 205100/225000 [393:56:37<39:26:44,  7.14s/it]
 91%|█████████ | 205101/225000 [393:56:51<50:45:54,  9.18s/it]
 91%|█████████ | 205102/225000 [393:56:56<44:42:31,  8.09s/it]
 91%|█████████ | 205103/225000 [393:57:02<40:33:52,  7.34s/it]
 91%|█████████ | 205104/225000 [393:57:12<44:29:21,  8.05s/it]
 91%|█████████ | 205105/225000 [393:57:20<44:27:00,  8.04s/it]
 91%|█████████ | 205106/225000 [393:57:29<46:30:39,  8.42s/it]
 91%|█████████ | 205107/225000 [393:57:36<44:25:27,  8.04s/it]
 91%|█████████ | 205108/225000 [393:57:46<47:12:35,  8.54s/it]
 91%|█████████ | 205109/225000 [393:57:56<49:50:56,  9.02s/it]
 91%|█████████ | 205110/225000 [393:58:08<55:42:00, 10.08s/it]
 91%|█████████ | 205111/225000 [393:58:17<53:30:08,  9.68s/it]
 91%|█████████ | 205112/225000 [393:58:26<51:16:39,  9.28s/it]
 91%|█████████ | 205113/225000 [393:58:31<45:33:18,  8.25s/it]
 91%|█████████ | 205114/225000 [393:58:42<48:49:24,  8.84s/it]
 91%|█████████ | 205115/225000 [393:58:48<44:23:41,  8.04s/it]
 91%|█████████ | 205116/225000 [393:58:55<42:52:09,  7.76s/it]
 91%|█████████ | 205117/225000 [393:59:10<54:50:58,  9.93s/it]
 91%|█████████ | 205118/225000 [393:59:16<48:56:13,  8.86s/it]
 91%|█████████ | 205119/225000 [393:59:26<51:08:56,  9.26s/it]
 91%|█████████ | 205120/225000 [393:59:34<47:56:41,  8.68s/it]
 91%|█████████ | 205121/225000 [393:59:48<56:27:54, 10.23s/it]
 91%|█████████ | 205122/225000 [393:59:55<52:20:20,  9.48s/it]
 91%|█████████ | 205123/225000 [394:00:02<47:48:38,  8.66s/it]
 91%|█████████ | 205124/225000 [394:00:06<40:41:17,  7.37s/it]
 91%|█████████ | 205125/225000 [394:00:22<54:32:02,  9.88s/it]
 91%|█████████ | 205126/225000 [394:00:29<50:03:31,  9.07s/it]
 91%|█████████ | 205127/225000 [394:00:41<53:59:56,  9.78s/it]
 91%|█████████ | 205128/225000 [394:00:48<50:05:36,  9.07s/it]
 91%|█████████ | 205129/225000 [394:00:57<50:14:33,  9.10s/it]
 91%|█████████ | 205130/225000 [394:01:09<53:58:06,  9.78s/it]
 91%|█████████ | 205131/225000 [394:01:15<47:43:24,  8.65s/it]
 91%|█████████ | 205132/225000 [394:01:25<50:28:30,  9.15s/it]
 91%|█████████ | 205133/225000 [394:01:32<47:23:26,  8.59s/it]
 91%|█████████ | 205134/225000 [394:01:41<46:43:48,  8.47s/it]
 91%|█████████ | 205135/225000 [394:01:48<44:46:29,  8.11s/it]
 91%|█████████ | 205136/225000 [394:01:54<41:16:55,  7.48s/it]
 91%|█████████ | 205137/225000 [394:02:04<45:31:20,  8.25s/it]
 91%|█████████ | 205138/225000 [394:02:10<41:19:50,  7.49s/it]
 91%|█████████ | 205139/225000 [394:02:19<44:38:25,  8.09s/it]
 91%|█████████ | 205140/225000 [394:02:28<46:31:29,  8.43s/it]
 91%|█████████ | 205141/225000 [394:02:37<46:44:12,  8.47s/it]
 91%|█████████ | 205142/225000 [394:02:44<45:01:37,  8.16s/it]
 91%|█████████ | 205143/225000 [394:02:56<50:19:17,  9.12s/it]
 91%|█████████ | 205144/225000 [394:03:10<58:58:00, 10.69s/it]
 91%|█████████ | 205145/225000 [394:03:18<55:01:11,  9.98s/it]
 91%|█████████ | 205146/225000 [394:03:24<47:43:00,  8.65s/it]
 91%|█████████ | 205147/225000 [394:03:34<49:48:35,  9.03s/it]
 91%|█████████ | 205148/225000 [394:03:39<44:11:06,  8.01s/it]
 91%|█████████ | 205149/225000 [394:03:50<48:34:50,  8.81s/it]
 91%|█████████ | 205150/225000 [394:04:00<50:47:11,  9.21s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.114', 'learning_rate': '1.054e-06', 'epoch': '0.9118', 'num_input_tokens_seen': 663744600, 'train_runtime': '1.419e+06', 'train_tokens_per_second': '467.9'}

 91%|█████████ | 205150/225000 [394:04:00<50:47:11,  9.21s/it]
 91%|█████████ | 205151/225000 [394:04:10<52:09:38,  9.46s/it]
 91%|█████████ | 205152/225000 [394:04:21<54:10:37,  9.83s/it]
 91%|█████████ | 205153/225000 [394:04:30<53:06:04,  9.63s/it]
 91%|█████████ | 205154/225000 [394:04:38<50:47:45,  9.21s/it]
 91%|█████████ | 205155/225000 [394:04:45<46:30:57,  8.44s/it]
 91%|█████████ | 205156/225000 [394:04:57<52:59:39,  9.61s/it]
 91%|█████████ | 205157/225000 [394:05:06<51:42:12,  9.38s/it]
 91%|█████████ | 205158/225000 [394:05:16<51:38:26,  9.37s/it]
 91%|█████████ | 205159/225000 [394:05:22<46:26:23,  8.43s/it]
 91%|█████████ | 205160/225000 [394:05:34<52:58:03,  9.61s/it]
 91%|█████████ | 205161/225000 [394:05:45<54:32:52,  9.90s/it]
 91%|█████████ | 205162/225000 [394:05:53<51:16:33,  9.31s/it]
 91%|█████████ | 205163/225000 [394:06:01<48:55:20,  8.88s/it]
 91%|█████████ | 205164/225000 [394:06:08<45:44:05,  8.30s/it]
 91%|█████████ | 205165/225000 [394:06:15<44:37:34,  8.10s/it]
 91%|█████████ | 205166/225000 [394:06:22<43:20:57,  7.87s/it]
 91%|█████████ | 205167/225000 [394:06:31<43:44:06,  7.94s/it]
 91%|█████████ | 205168/225000 [394:06:36<39:11:10,  7.11s/it]
 91%|█████████ | 205169/225000 [394:06:42<36:56:01,  6.70s/it]
 91%|█████████ | 205170/225000 [394:06:55<48:48:04,  8.86s/it]
 91%|█████████ | 205171/225000 [394:07:05<50:36:03,  9.19s/it]
 91%|█████████ | 205172/225000 [394:07:12<46:34:11,  8.46s/it]
 91%|█████████ | 205173/225000 [394:07:18<42:03:14,  7.64s/it]
 91%|█████████ | 205174/225000 [394:07:31<50:29:00,  9.17s/it]
 91%|█████████ | 205175/225000 [394:07:38<48:15:08,  8.76s/it]
 91%|█████████ | 205176/225000 [394:07:50<52:29:03,  9.53s/it]
 91%|█████████ | 205177/225000 [394:07:57<48:46:21,  8.86s/it]
 91%|█████████ | 205178/225000 [394:08:08<51:55:59,  9.43s/it]
 91%|█████████ | 205179/225000 [394:08:15<48:34:28,  8.82s/it]
 91%|█████████ | 205180/225000 [394:08:22<45:01:02,  8.18s/it]
 91%|█████████ | 205181/225000 [394:08:34<50:52:27,  9.24s/it]
 91%|█████████ | 205182/225000 [394:08:40<46:49:10,  8.50s/it]
 91%|█████████ | 205183/225000 [394:08:47<44:30:52,  8.09s/it]
 91%|█████████ | 205184/225000 [394:08:59<50:41:28,  9.21s/it]
 91%|█████████ | 205185/225000 [394:09:10<52:38:32,  9.56s/it]
 91%|█████████ | 205186/225000 [394:09:15<46:10:15,  8.39s/it]
 91%|█████████ | 205187/225000 [394:09:24<46:29:58,  8.45s/it]
 91%|█████████ | 205188/225000 [394:09:37<54:35:21,  9.92s/it]
 91%|█████████ | 205189/225000 [394:09:42<45:55:13,  8.34s/it]
 91%|█████████ | 205190/225000 [394:09:51<47:34:20,  8.65s/it]
 91%|█████████ | 205191/225000 [394:10:05<56:31:22, 10.27s/it]
 91%|█████████ | 205192/225000 [394:10:11<48:16:48,  8.77s/it]
 91%|█████████ | 205193/225000 [394:10:20<48:34:02,  8.83s/it]
 91%|█████████ | 205194/225000 [394:10:26<44:35:00,  8.10s/it]
 91%|█████████ | 205195/225000 [394:10:35<45:44:39,  8.32s/it]
 91%|█████████ | 205196/225000 [394:10:40<40:10:09,  7.30s/it]
 91%|█████████ | 205197/225000 [394:10:47<39:46:45,  7.23s/it]
 91%|█████████ | 205198/225000 [394:10:58<46:56:17,  8.53s/it]
 91%|█████████ | 205199/225000 [394:11:06<45:46:38,  8.32s/it]
 91%|█████████ | 205200/225000 [394:11:16<48:36:45,  8.84s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.18', 'learning_rate': '1.049e-06', 'epoch': '0.912', 'num_input_tokens_seen': 663973872, 'train_runtime': '1.419e+06', 'train_tokens_per_second': '467.9'}

 91%|█████████ | 205200/225000 [394:11:16<48:36:45,  8.84s/it]
 91%|█████████ | 205201/225000 [394:11:26<49:52:51,  9.07s/it]
 91%|█████████ | 205202/225000 [394:11:33<46:50:59,  8.52s/it]
 91%|█████████ | 205203/225000 [394:11:38<41:07:54,  7.48s/it]
 91%|█████████ | 205204/225000 [394:11:48<45:07:54,  8.21s/it]
 91%|█████████ | 205205/225000 [394:11:54<40:34:41,  7.38s/it]
 91%|█████████ | 205206/225000 [394:12:07<50:41:17,  9.22s/it]
 91%|█████████ | 205207/225000 [394:12:14<47:38:58,  8.67s/it]
 91%|█████████ | 205208/225000 [394:12:21<43:30:46,  7.91s/it]
 91%|█████████ | 205209/225000 [394:12:28<43:23:45,  7.89s/it]
 91%|█████████ | 205210/225000 [394:12:43<53:52:11,  9.80s/it]
 91%|█████████ | 205211/225000 [394:12:52<52:57:58,  9.64s/it]
 91%|█████████ | 205212/225000 [394:12:56<44:31:45,  8.10s/it]
 91%|█████████ | 205213/225000 [394:13:07<48:48:39,  8.88s/it]
 91%|█████████ | 205214/225000 [394:13:17<49:55:58,  9.09s/it]
 91%|█████████ | 205215/225000 [394:13:25<49:06:37,  8.94s/it]
 91%|█████████ | 205216/225000 [394:13:34<48:26:39,  8.82s/it]
 91%|█████████ | 205217/225000 [394:13:42<47:16:14,  8.60s/it]
 91%|█████████ | 205218/225000 [394:13:52<49:40:57,  9.04s/it]
 91%|█████████ | 205219/225000 [394:14:00<48:33:02,  8.84s/it]
 91%|█████████ | 205220/225000 [394:14:07<44:38:55,  8.13s/it]
 91%|█████████ | 205221/225000 [394:14:13<42:13:20,  7.68s/it]
 91%|█████████ | 205222/225000 [394:14:23<45:25:34,  8.27s/it]
 91%|█████████ | 205223/225000 [394:14:31<44:41:21,  8.13s/it]
 91%|█████████ | 205224/225000 [394:14:41<48:35:11,  8.84s/it]
 91%|█████████ | 205225/225000 [394:14:50<48:20:05,  8.80s/it]
 91%|█████████ | 205226/225000 [394:14:59<49:15:20,  8.97s/it]
 91%|█████████ | 205227/225000 [394:15:07<46:21:16,  8.44s/it]
 91%|█████████ | 205228/225000 [394:15:14<44:10:15,  8.04s/it]
 91%|█████████ | 205229/225000 [394:15:23<45:53:31,  8.36s/it]
 91%|█████████ | 205230/225000 [394:15:34<51:05:41,  9.30s/it]
 91%|█████████ | 205231/225000 [394:15:41<47:23:37,  8.63s/it]
 91%|█████████ | 205232/225000 [394:15:53<51:39:28,  9.41s/it]
 91%|█████████ | 205233/225000 [394:16:03<53:21:20,  9.72s/it]
 91%|█████████ | 205234/225000 [394:16:09<46:40:19,  8.50s/it]
 91%|█████████ | 205235/225000 [394:16:19<49:13:03,  8.96s/it]
 91%|█████████ | 205236/225000 [394:16:26<45:49:47,  8.35s/it]
 91%|█████████ | 205237/225000 [394:16:32<42:43:34,  7.78s/it]
 91%|█████████ | 205238/225000 [394:16:44<49:36:28,  9.04s/it]
 91%|█████████ | 205239/225000 [394:16:53<50:01:48,  9.11s/it]
 91%|█████████ | 205240/225000 [394:16:59<44:28:39,  8.10s/it]
 91%|█████████ | 205241/225000 [394:17:09<46:48:07,  8.53s/it]
 91%|█████████ | 205242/225000 [394:17:13<39:39:23,  7.23s/it]
 91%|█████████ | 205243/225000 [394:17:19<37:11:15,  6.78s/it]
 91%|█████████ | 205244/225000 [394:17:32<47:53:25,  8.73s/it]
 91%|█████████ | 205245/225000 [394:17:43<52:06:24,  9.50s/it]
 91%|█████████ | 205246/225000 [394:17:54<54:03:47,  9.85s/it]
 91%|█████████ | 205247/225000 [394:18:02<50:37:32,  9.23s/it]
 91%|█████████ | 205248/225000 [394:18:07<44:03:18,  8.03s/it]
 91%|█████████ | 205249/225000 [394:18:21<53:16:24,  9.71s/it]
 91%|█████████ | 205250/225000 [394:18:28<50:07:26,  9.14s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.07', 'learning_rate': '1.044e-06', 'epoch': '0.9122', 'num_input_tokens_seen': 664202352, 'train_runtime': '1.42e+06', 'train_tokens_per_second': '467.9'}

 91%|█████████ | 205250/225000 [394:18:28<50:07:26,  9.14s/it]
 91%|█████████ | 205251/225000 [394:18:38<50:40:42,  9.24s/it]
 91%|█████████ | 205252/225000 [394:18:44<45:46:18,  8.34s/it]
 91%|█████████ | 205253/225000 [394:18:56<51:41:16,  9.42s/it]
 91%|█████████ | 205254/225000 [394:19:05<50:33:59,  9.22s/it]
 91%|█████████ | 205255/225000 [394:19:13<49:15:35,  8.98s/it]
 91%|█████████ | 205256/225000 [394:19:19<44:08:14,  8.05s/it]
 91%|█████████ | 205257/225000 [394:19:25<40:04:48,  7.31s/it]
 91%|█████████ | 205258/225000 [394:19:39<51:57:45,  9.48s/it]
 91%|█████████ | 205259/225000 [394:19:45<46:43:47,  8.52s/it]
 91%|█████████ | 205260/225000 [394:19:53<45:32:24,  8.31s/it]
 91%|█████████ | 205261/225000 [394:20:07<54:09:34,  9.88s/it]
 91%|█████████ | 205262/225000 [394:20:16<53:03:11,  9.68s/it]
 91%|█████████ | 205263/225000 [394:20:22<47:31:49,  8.67s/it]
 91%|█████████ | 205264/225000 [394:20:31<47:09:35,  8.60s/it]
 91%|█████████ | 205265/225000 [394:20:38<44:30:14,  8.12s/it]
 91%|█████████ | 205266/225000 [394:20:44<41:15:15,  7.53s/it]
 91%|█████████ | 205267/225000 [394:20:54<45:30:34,  8.30s/it]
 91%|█████████ | 205268/225000 [394:21:05<49:42:34,  9.07s/it]
 91%|█████████ | 205269/225000 [394:21:10<43:34:16,  7.95s/it]
 91%|█████████ | 205270/225000 [394:21:22<49:19:22,  9.00s/it]
 91%|█████████ | 205271/225000 [394:21:27<43:42:27,  7.98s/it]
 91%|█████████ | 205272/225000 [394:21:35<42:49:05,  7.81s/it]
 91%|█████████ | 205273/225000 [394:21:42<41:36:56,  7.59s/it]
 91%|█████████ | 205274/225000 [394:21:54<49:29:52,  9.03s/it]
 91%|█████████ | 205275/225000 [394:22:03<48:59:31,  8.94s/it]
 91%|█████████ | 205276/225000 [394:22:13<50:33:15,  9.23s/it]
 91%|█████████ | 205277/225000 [394:22:24<54:01:29,  9.86s/it]
 91%|█████████ | 205278/225000 [394:22:30<48:02:49,  8.77s/it]
 91%|█████████ | 205279/225000 [394:22:44<55:41:01, 10.16s/it]
 91%|█████████ | 205280/225000 [394:22:50<48:49:12,  8.91s/it]
 91%|█████████ | 205281/225000 [394:22:59<49:03:53,  8.96s/it]
 91%|█████████ | 205282/225000 [394:23:04<42:09:50,  7.70s/it]
 91%|█████████ | 205283/225000 [394:23:12<43:02:24,  7.86s/it]
 91%|█████████ | 205284/225000 [394:23:18<41:03:03,  7.50s/it]
 91%|█████████ | 205285/225000 [394:23:32<51:07:53,  9.34s/it]
 91%|█████████ | 205286/225000 [394:23:41<49:42:51,  9.08s/it]
 91%|█████████ | 205287/225000 [394:23:49<49:26:14,  9.03s/it]
 91%|█████████ | 205288/225000 [394:23:56<45:18:02,  8.27s/it]
 91%|█████████ | 205289/225000 [394:24:07<49:20:54,  9.01s/it]
 91%|█████████ | 205290/225000 [394:24:14<46:05:23,  8.42s/it]
 91%|█████████ | 205291/225000 [394:24:23<47:17:18,  8.64s/it]
 91%|█████████ | 205292/225000 [394:24:33<50:30:59,  9.23s/it]
 91%|█████████ | 205293/225000 [394:24:40<46:28:07,  8.49s/it]
 91%|█████████ | 205294/225000 [394:24:53<53:30:36,  9.78s/it]
 91%|█████████ | 205295/225000 [394:24:59<46:57:55,  8.58s/it]
 91%|█████████ | 205296/225000 [394:25:11<53:27:43,  9.77s/it]
 91%|█████████ | 205297/225000 [394:25:21<53:41:00,  9.81s/it]
 91%|█████████ | 205298/225000 [394:25:29<50:58:43,  9.31s/it]
 91%|█████████ | 205299/225000 [394:25:38<49:29:31,  9.04s/it]
 91%|█████████ | 205300/225000 [394:25:45<46:23:33,  8.48s/it]
                                                              
{'loss': '1.321', 'grad_norm': '0.9822', 'learning_rate': '1.039e-06', 'epoch': '0.9124', 'num_input_tokens_seen': 664435256, 'train_runtime': '1.42e+06', 'train_tokens_per_second': '467.9'}

 91%|█████████ | 205300/225000 [394:25:45<46:23:33,  8.48s/it]
 91%|█████████ | 205301/225000 [394:25:58<53:20:56,  9.75s/it]
 91%|█████████ | 205302/225000 [394:26:08<54:05:40,  9.89s/it]
 91%|█████████ | 205303/225000 [394:26:16<50:44:24,  9.27s/it]
 91%|█████████ | 205304/225000 [394:26:25<50:59:35,  9.32s/it]
 91%|█████████ | 205305/225000 [394:26:32<47:17:51,  8.65s/it]
 91%|█████████ | 205306/225000 [394:26:40<45:50:19,  8.38s/it]
 91%|█████████ | 205307/225000 [394:26:53<52:47:27,  9.65s/it]
 91%|█████████ | 205308/225000 [394:27:02<51:36:33,  9.43s/it]
 91%|█████████ | 205309/225000 [394:27:10<49:40:30,  9.08s/it]
 91%|█████████ | 205310/225000 [394:27:14<41:41:55,  7.62s/it]
 91%|█████████ | 205311/225000 [394:27:22<42:15:20,  7.73s/it]
 91%|█████████ | 205312/225000 [394:27:31<43:50:46,  8.02s/it]
 91%|█████████▏| 205313/225000 [394:27:39<44:49:25,  8.20s/it]
 91%|█████████▏| 205314/225000 [394:27:49<47:42:31,  8.72s/it]
 91%|█████████▏| 205315/225000 [394:27:55<42:18:47,  7.74s/it]
 91%|█████████▏| 205316/225000 [394:28:01<39:57:57,  7.31s/it]
 91%|█████████▏| 205317/225000 [394:28:13<48:00:11,  8.78s/it]
 91%|█████████▏| 205318/225000 [394:28:19<42:50:55,  7.84s/it]
 91%|█████████▏| 205319/225000 [394:28:26<42:17:51,  7.74s/it]
 91%|█████████▏| 205320/225000 [394:28:32<39:35:01,  7.24s/it]
 91%|█████████▏| 205321/225000 [394:28:39<37:58:00,  6.95s/it]
 91%|█████████▏| 205322/225000 [394:28:49<43:53:16,  8.03s/it]
 91%|█████████▏| 205323/225000 [394:28:59<46:01:10,  8.42s/it]
 91%|█████████▏| 205324/225000 [394:29:08<48:18:25,  8.84s/it]
 91%|█████████▏| 205325/225000 [394:29:20<52:56:39,  9.69s/it]
 91%|█████████▏| 205326/225000 [394:29:31<54:07:54,  9.91s/it]
 91%|█████████▏| 205327/225000 [394:29:41<55:23:39, 10.14s/it]
 91%|█████████▏| 205328/225000 [394:29:47<48:20:28,  8.85s/it]
 91%|█████████▏| 205329/225000 [394:29:55<47:15:57,  8.65s/it]
 91%|█████████▏| 205330/225000 [394:30:06<50:49:41,  9.30s/it]
 91%|█████████▏| 205331/225000 [394:30:16<52:30:39,  9.61s/it]
 91%|█████████▏| 205332/225000 [394:30:23<47:28:20,  8.69s/it]
 91%|█████████▏| 205333/225000 [394:30:29<44:00:36,  8.06s/it]
 91%|█████████▏| 205334/225000 [394:30:41<50:01:06,  9.16s/it]
 91%|█████████▏| 205335/225000 [394:30:46<43:05:05,  7.89s/it]
 91%|█████████▏| 205336/225000 [394:30:53<42:07:00,  7.71s/it]
 91%|█████████▏| 205337/225000 [394:31:01<41:31:09,  7.60s/it]
 91%|█████████▏| 205338/225000 [394:31:14<49:56:45,  9.14s/it]
 91%|█████████▏| 205339/225000 [394:31:20<46:03:20,  8.43s/it]
 91%|█████████▏| 205340/225000 [394:31:25<39:51:16,  7.30s/it]
 91%|█████████▏| 205341/225000 [394:31:35<44:01:19,  8.06s/it]
 91%|█████████▏| 205342/225000 [394:31:43<44:10:59,  8.09s/it]
 91%|█████████▏| 205343/225000 [394:31:50<42:20:45,  7.76s/it]
 91%|█████████▏| 205344/225000 [394:32:03<50:35:14,  9.27s/it]
 91%|█████████▏| 205345/225000 [394:32:13<52:04:29,  9.54s/it]
 91%|█████████▏| 205346/225000 [394:32:18<44:36:31,  8.17s/it]
 91%|█████████▏| 205347/225000 [394:32:24<41:45:57,  7.65s/it]
 91%|█████████▏| 205348/225000 [394:32:36<47:51:43,  8.77s/it]
 91%|█████████▏| 205349/225000 [394:32:45<48:57:14,  8.97s/it]
 91%|█████████▏| 205350/225000 [394:32:55<50:37:05,  9.27s/it]
                                                              
{'loss': '1.322', 'grad_norm': '0.9909', 'learning_rate': '1.033e-06', 'epoch': '0.9127', 'num_input_tokens_seen': 664667040, 'train_runtime': '1.42e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205350/225000 [394:32:55<50:37:05,  9.27s/it]
 91%|█████████▏| 205351/225000 [394:33:02<46:17:57,  8.48s/it]
 91%|█████████▏| 205352/225000 [394:33:16<55:16:40, 10.13s/it]
 91%|█████████▏| 205353/225000 [394:33:21<47:41:19,  8.74s/it]
 91%|█████████▏| 205354/225000 [394:33:26<41:07:59,  7.54s/it]
 91%|█████████▏| 205355/225000 [394:33:37<47:25:19,  8.69s/it]
 91%|█████████▏| 205356/225000 [394:33:44<43:57:07,  8.05s/it]
 91%|█████████▏| 205357/225000 [394:33:51<42:37:34,  7.81s/it]
 91%|█████████▏| 205358/225000 [394:34:04<51:05:54,  9.37s/it]
 91%|█████████▏| 205359/225000 [394:34:14<52:26:05,  9.61s/it]
 91%|█████████▏| 205360/225000 [394:34:25<54:27:18,  9.98s/it]
 91%|█████████▏| 205361/225000 [394:34:31<47:02:32,  8.62s/it]
 91%|█████████▏| 205362/225000 [394:34:36<41:37:34,  7.63s/it]
 91%|█████████▏| 205363/225000 [394:34:43<40:45:08,  7.47s/it]
 91%|█████████▏| 205364/225000 [394:34:49<38:50:38,  7.12s/it]
 91%|█████████▏| 205365/225000 [394:34:55<37:12:20,  6.82s/it]
 91%|█████████▏| 205366/225000 [394:35:07<45:13:32,  8.29s/it]
 91%|█████████▏| 205367/225000 [394:35:24<58:35:07, 10.74s/it]
 91%|█████████▏| 205368/225000 [394:35:33<56:32:39, 10.37s/it]
 91%|█████████▏| 205369/225000 [394:35:41<51:57:51,  9.53s/it]
 91%|█████████▏| 205370/225000 [394:35:46<45:40:20,  8.38s/it]
 91%|█████████▏| 205371/225000 [394:35:51<40:00:30,  7.34s/it]
 91%|█████████▏| 205372/225000 [394:36:05<49:40:17,  9.11s/it]
 91%|█████████▏| 205373/225000 [394:36:12<47:19:00,  8.68s/it]
 91%|█████████▏| 205374/225000 [394:36:20<46:09:14,  8.47s/it]
 91%|█████████▏| 205375/225000 [394:36:26<42:29:32,  7.79s/it]
 91%|█████████▏| 205376/225000 [394:36:45<60:50:01, 11.16s/it]
 91%|█████████▏| 205377/225000 [394:36:52<52:57:32,  9.72s/it]
 91%|█████████▏| 205378/225000 [394:36:59<49:18:03,  9.05s/it]
 91%|█████████▏| 205379/225000 [394:37:13<56:24:13, 10.35s/it]
 91%|█████████▏| 205380/225000 [394:37:20<51:25:24,  9.44s/it]
 91%|█████████▏| 205381/225000 [394:37:26<45:29:36,  8.35s/it]
 91%|█████████▏| 205382/225000 [394:37:35<47:18:45,  8.68s/it]
 91%|█████████▏| 205383/225000 [394:37:42<43:53:50,  8.06s/it]
 91%|█████████▏| 205384/225000 [394:37:49<42:20:30,  7.77s/it]
 91%|█████████▏| 205385/225000 [394:38:00<47:59:26,  8.81s/it]
 91%|█████████▏| 205386/225000 [394:38:06<42:36:10,  7.82s/it]
 91%|█████████▏| 205387/225000 [394:38:11<39:15:17,  7.21s/it]
 91%|█████████▏| 205388/225000 [394:38:18<38:25:59,  7.05s/it]
 91%|█████████▏| 205389/225000 [394:38:24<36:45:16,  6.75s/it]
 91%|█████████▏| 205390/225000 [394:38:36<45:37:16,  8.38s/it]
 91%|█████████▏| 205391/225000 [394:38:43<42:09:36,  7.74s/it]
 91%|█████████▏| 205392/225000 [394:38:49<40:03:31,  7.35s/it]
 91%|█████████▏| 205393/225000 [394:38:55<37:01:28,  6.80s/it]
 91%|█████████▏| 205394/225000 [394:39:08<47:52:07,  8.79s/it]
 91%|█████████▏| 205395/225000 [394:39:14<42:42:57,  7.84s/it]
 91%|█████████▏| 205396/225000 [394:39:24<47:18:58,  8.69s/it]
 91%|█████████▏| 205397/225000 [394:39:31<43:25:38,  7.98s/it]
 91%|█████████▏| 205398/225000 [394:39:44<51:31:32,  9.46s/it]
 91%|█████████▏| 205399/225000 [394:39:51<48:40:58,  8.94s/it]
 91%|█████████▏| 205400/225000 [394:39:57<43:58:56,  8.08s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.326', 'learning_rate': '1.028e-06', 'epoch': '0.9129', 'num_input_tokens_seen': 664891616, 'train_runtime': '1.421e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205400/225000 [394:39:57<43:58:56,  8.08s/it]
 91%|█████████▏| 205401/225000 [394:40:04<41:07:12,  7.55s/it]
 91%|█████████▏| 205402/225000 [394:40:12<43:02:52,  7.91s/it]
 91%|█████████▏| 205403/225000 [394:40:27<53:24:38,  9.81s/it]
 91%|█████████▏| 205404/225000 [394:40:32<46:50:24,  8.61s/it]
 91%|█████████▏| 205405/225000 [394:40:41<47:12:08,  8.67s/it]
 91%|█████████▏| 205406/225000 [394:40:56<56:40:13, 10.41s/it]
 91%|█████████▏| 205407/225000 [394:41:02<49:29:37,  9.09s/it]
 91%|█████████▏| 205408/225000 [394:41:12<51:10:25,  9.40s/it]
 91%|█████████▏| 205409/225000 [394:41:25<57:26:09, 10.55s/it]
 91%|█████████▏| 205410/225000 [394:41:30<48:08:54,  8.85s/it]
 91%|█████████▏| 205411/225000 [394:41:39<47:55:13,  8.81s/it]
 91%|█████████▏| 205412/225000 [394:41:45<43:53:08,  8.07s/it]
 91%|█████████▏| 205413/225000 [394:41:54<45:27:55,  8.36s/it]
 91%|█████████▏| 205414/225000 [394:41:59<40:40:58,  7.48s/it]
 91%|█████████▏| 205415/225000 [394:42:10<45:10:08,  8.30s/it]
 91%|█████████▏| 205416/225000 [394:42:23<52:31:19,  9.65s/it]
 91%|█████████▏| 205417/225000 [394:42:28<46:25:17,  8.53s/it]
 91%|█████████▏| 205418/225000 [394:42:45<60:09:50, 11.06s/it]
 91%|█████████▏| 205419/225000 [394:42:51<51:35:42,  9.49s/it]
 91%|█████████▏| 205420/225000 [394:43:04<56:27:19, 10.38s/it]
 91%|█████████▏| 205421/225000 [394:43:10<50:27:17,  9.28s/it]
 91%|█████████▏| 205422/225000 [394:43:18<48:07:58,  8.85s/it]
 91%|█████████▏| 205423/225000 [394:43:29<50:51:38,  9.35s/it]
 91%|█████████▏| 205424/225000 [394:43:40<53:37:13,  9.86s/it]
 91%|█████████▏| 205425/225000 [394:43:48<51:28:34,  9.47s/it]
 91%|█████████▏| 205426/225000 [394:43:55<46:05:33,  8.48s/it]
 91%|█████████▏| 205427/225000 [394:44:02<43:38:22,  8.03s/it]
 91%|█████████▏| 205428/225000 [394:44:08<40:42:48,  7.49s/it]
 91%|█████████▏| 205429/225000 [394:44:13<37:48:38,  6.96s/it]
 91%|█████████▏| 205430/225000 [394:44:29<51:01:41,  9.39s/it]
 91%|█████████▏| 205431/225000 [394:44:34<44:52:08,  8.25s/it]
 91%|█████████▏| 205432/225000 [394:44:44<48:00:40,  8.83s/it]
 91%|█████████▏| 205433/225000 [394:44:50<43:23:07,  7.98s/it]
 91%|█████████▏| 205434/225000 [394:45:03<50:38:11,  9.32s/it]
 91%|█████████▏| 205435/225000 [394:45:11<49:24:10,  9.09s/it]
 91%|█████████▏| 205436/225000 [394:45:26<57:44:47, 10.63s/it]
 91%|█████████▏| 205437/225000 [394:45:29<45:30:11,  8.37s/it]
 91%|█████████▏| 205438/225000 [394:45:38<47:46:08,  8.79s/it]
 91%|█████████▏| 205439/225000 [394:45:46<45:30:20,  8.37s/it]
 91%|█████████▏| 205440/225000 [394:45:56<48:19:00,  8.89s/it]
 91%|█████████▏| 205441/225000 [394:46:04<47:30:26,  8.74s/it]
 91%|█████████▏| 205442/225000 [394:46:16<52:37:25,  9.69s/it]
 91%|█████████▏| 205443/225000 [394:46:24<49:53:47,  9.18s/it]
 91%|█████████▏| 205444/225000 [394:46:31<45:42:36,  8.41s/it]
 91%|█████████▏| 205445/225000 [394:46:38<44:03:10,  8.11s/it]
 91%|█████████▏| 205446/225000 [394:46:52<53:07:52,  9.78s/it]
 91%|█████████▏| 205447/225000 [394:46:59<49:35:12,  9.13s/it]
 91%|█████████▏| 205448/225000 [394:47:07<47:29:35,  8.74s/it]
 91%|█████████▏| 205449/225000 [394:47:19<51:34:30,  9.50s/it]
 91%|█████████▏| 205450/225000 [394:47:25<47:09:51,  8.68s/it]
                                                              
{'loss': '1.266', 'grad_norm': '1.068', 'learning_rate': '1.023e-06', 'epoch': '0.9131', 'num_input_tokens_seen': 665125552, 'train_runtime': '1.421e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205450/225000 [394:47:25<47:09:51,  8.68s/it]
 91%|█████████▏| 205451/225000 [394:47:32<43:50:15,  8.07s/it]
 91%|█████████▏| 205452/225000 [394:47:44<49:45:49,  9.16s/it]
 91%|█████████▏| 205453/225000 [394:47:51<46:55:27,  8.64s/it]
 91%|█████████▏| 205454/225000 [394:48:02<49:45:25,  9.16s/it]
 91%|█████████▏| 205455/225000 [394:48:07<44:00:14,  8.11s/it]
 91%|█████████▏| 205456/225000 [394:48:13<39:48:45,  7.33s/it]
 91%|█████████▏| 205457/225000 [394:48:27<51:47:46,  9.54s/it]
 91%|█████████▏| 205458/225000 [394:48:38<53:14:21,  9.81s/it]
 91%|█████████▏| 205459/225000 [394:48:50<56:27:54, 10.40s/it]
 91%|█████████▏| 205460/225000 [394:49:03<60:52:21, 11.22s/it]
 91%|█████████▏| 205461/225000 [394:49:07<49:40:07,  9.15s/it]
 91%|█████████▏| 205462/225000 [394:49:18<52:10:37,  9.61s/it]
 91%|█████████▏| 205463/225000 [394:49:32<59:21:21, 10.94s/it]
 91%|█████████▏| 205464/225000 [394:49:37<49:35:20,  9.14s/it]
 91%|█████████▏| 205465/225000 [394:49:47<52:11:51,  9.62s/it]
 91%|█████████▏| 205466/225000 [394:49:58<53:03:52,  9.78s/it]
 91%|█████████▏| 205467/225000 [394:50:09<54:59:23, 10.13s/it]
 91%|█████████▏| 205468/225000 [394:50:17<52:26:38,  9.67s/it]
 91%|█████████▏| 205469/225000 [394:50:25<50:06:48,  9.24s/it]
 91%|█████████▏| 205470/225000 [394:50:35<51:02:46,  9.41s/it]
 91%|█████████▏| 205471/225000 [394:50:40<43:43:04,  8.06s/it]
 91%|█████████▏| 205472/225000 [394:50:49<45:13:37,  8.34s/it]
 91%|█████████▏| 205473/225000 [394:50:56<42:50:21,  7.90s/it]
 91%|█████████▏| 205474/225000 [394:51:10<53:16:26,  9.82s/it]
 91%|█████████▏| 205475/225000 [394:51:21<54:19:28, 10.02s/it]
 91%|█████████▏| 205476/225000 [394:51:27<48:12:33,  8.89s/it]
 91%|█████████▏| 205477/225000 [394:51:37<49:28:38,  9.12s/it]
 91%|█████████▏| 205478/225000 [394:51:46<50:11:07,  9.25s/it]
 91%|█████████▏| 205479/225000 [394:51:53<45:45:12,  8.44s/it]
 91%|█████████▏| 205480/225000 [394:52:02<47:28:33,  8.76s/it]
 91%|█████████▏| 205481/225000 [394:52:13<50:12:44,  9.26s/it]
 91%|█████████▏| 205482/225000 [394:52:20<46:27:32,  8.57s/it]
 91%|█████████▏| 205483/225000 [394:52:33<54:04:52,  9.98s/it]
 91%|█████████▏| 205484/225000 [394:52:39<47:38:27,  8.79s/it]
 91%|█████████▏| 205485/225000 [394:52:48<48:41:37,  8.98s/it]
 91%|█████████▏| 205486/225000 [394:52:59<51:03:48,  9.42s/it]
 91%|█████████▏| 205487/225000 [394:53:06<48:01:28,  8.86s/it]
 91%|█████████▏| 205488/225000 [394:53:15<47:23:45,  8.74s/it]
 91%|█████████▏| 205489/225000 [394:53:23<45:53:23,  8.47s/it]
 91%|█████████▏| 205490/225000 [394:53:29<43:11:08,  7.97s/it]
 91%|█████████▏| 205491/225000 [394:53:37<42:50:52,  7.91s/it]
 91%|█████████▏| 205492/225000 [394:53:48<48:14:33,  8.90s/it]
 91%|█████████▏| 205493/225000 [394:53:56<45:58:25,  8.48s/it]
 91%|█████████▏| 205494/225000 [394:54:09<53:48:52,  9.93s/it]
 91%|█████████▏| 205495/225000 [394:54:16<48:23:19,  8.93s/it]
 91%|█████████▏| 205496/225000 [394:54:23<45:55:52,  8.48s/it]
 91%|█████████▏| 205497/225000 [394:54:36<53:28:40,  9.87s/it]
 91%|█████████▏| 205498/225000 [394:54:43<47:49:44,  8.83s/it]
 91%|█████████▏| 205499/225000 [394:54:55<53:06:05,  9.80s/it]
 91%|█████████▏| 205500/225000 [394:55:03<49:50:58,  9.20s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.118', 'learning_rate': '1.018e-06', 'epoch': '0.9133', 'num_input_tokens_seen': 665362600, 'train_runtime': '1.422e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205500/225000 [394:55:03<49:50:58,  9.20s/it]
 91%|█████████▏| 205501/225000 [394:55:10<46:21:18,  8.56s/it]
 91%|█████████▏| 205502/225000 [394:55:23<54:19:41, 10.03s/it]
 91%|█████████▏| 205503/225000 [394:55:31<49:58:47,  9.23s/it]
 91%|█████████▏| 205504/225000 [394:55:37<44:45:15,  8.26s/it]
 91%|█████████▏| 205505/225000 [394:55:44<43:35:59,  8.05s/it]
 91%|█████████▏| 205506/225000 [394:55:58<53:38:59,  9.91s/it]
 91%|█████████▏| 205507/225000 [394:56:11<58:32:22, 10.81s/it]
 91%|█████████▏| 205508/225000 [394:56:18<51:03:34,  9.43s/it]
 91%|█████████▏| 205509/225000 [394:56:27<51:11:50,  9.46s/it]
 91%|█████████▏| 205510/225000 [394:56:33<45:25:58,  8.39s/it]
 91%|█████████▏| 205511/225000 [394:56:42<46:22:36,  8.57s/it]
 91%|█████████▏| 205512/225000 [394:56:52<49:13:16,  9.09s/it]
 91%|█████████▏| 205513/225000 [394:56:58<44:09:08,  8.16s/it]
 91%|█████████▏| 205514/225000 [394:57:05<42:21:46,  7.83s/it]
 91%|█████████▏| 205515/225000 [394:57:17<48:44:48,  9.01s/it]
 91%|█████████▏| 205516/225000 [394:57:26<48:17:54,  8.92s/it]
 91%|█████████▏| 205517/225000 [394:57:34<46:26:13,  8.58s/it]
 91%|█████████▏| 205518/225000 [394:57:45<51:00:47,  9.43s/it]
 91%|█████████▏| 205519/225000 [394:57:53<48:15:05,  8.92s/it]
 91%|█████████▏| 205520/225000 [394:58:02<48:59:58,  9.06s/it]
 91%|█████████▏| 205521/225000 [394:58:09<45:41:41,  8.45s/it]
 91%|█████████▏| 205522/225000 [394:58:19<47:24:12,  8.76s/it]
 91%|█████████▏| 205523/225000 [394:58:25<43:56:45,  8.12s/it]
 91%|█████████▏| 205524/225000 [394:58:34<44:37:11,  8.25s/it]
 91%|█████████▏| 205525/225000 [394:58:44<47:34:18,  8.79s/it]
 91%|█████████▏| 205526/225000 [394:58:53<48:04:05,  8.89s/it]
 91%|█████████▏| 205527/225000 [394:59:02<48:04:56,  8.89s/it]
 91%|█████████▏| 205528/225000 [394:59:10<47:26:54,  8.77s/it]
 91%|█████████▏| 205529/225000 [394:59:20<49:37:06,  9.17s/it]
 91%|█████████▏| 205530/225000 [394:59:29<48:26:06,  8.96s/it]
 91%|█████████▏| 205531/225000 [394:59:43<57:00:29, 10.54s/it]
 91%|█████████▏| 205532/225000 [394:59:48<47:41:09,  8.82s/it]
 91%|█████████▏| 205533/225000 [394:59:56<45:44:40,  8.46s/it]
 91%|█████████▏| 205534/225000 [395:00:03<44:58:01,  8.32s/it]
 91%|█████████▏| 205535/225000 [395:00:16<51:36:23,  9.54s/it]
 91%|█████████▏| 205536/225000 [395:00:23<47:40:52,  8.82s/it]
 91%|█████████▏| 205537/225000 [395:00:28<42:06:09,  7.79s/it]
 91%|█████████▏| 205538/225000 [395:00:35<40:44:57,  7.54s/it]
 91%|█████████▏| 205539/225000 [395:00:42<39:11:20,  7.25s/it]
 91%|█████████▏| 205540/225000 [395:00:57<51:49:44,  9.59s/it]
 91%|█████████▏| 205541/225000 [395:01:03<46:41:48,  8.64s/it]
 91%|█████████▏| 205542/225000 [395:01:12<46:06:01,  8.53s/it]
 91%|█████████▏| 205543/225000 [395:01:22<48:30:11,  8.97s/it]
 91%|█████████▏| 205544/225000 [395:01:28<43:38:59,  8.08s/it]
 91%|█████████▏| 205545/225000 [395:01:39<49:16:06,  9.12s/it]
 91%|█████████▏| 205546/225000 [395:01:46<45:25:16,  8.41s/it]
 91%|█████████▏| 205547/225000 [395:01:53<43:20:35,  8.02s/it]
 91%|█████████▏| 205548/225000 [395:02:02<44:56:18,  8.32s/it]
 91%|█████████▏| 205549/225000 [395:02:09<42:27:03,  7.86s/it]
 91%|█████████▏| 205550/225000 [395:02:17<42:34:15,  7.88s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.137', 'learning_rate': '1.013e-06', 'epoch': '0.9136', 'num_input_tokens_seen': 665593632, 'train_runtime': '1.422e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205550/225000 [395:02:17<42:34:15,  7.88s/it]
 91%|█████████▏| 205551/225000 [395:02:26<44:21:24,  8.21s/it]
 91%|█████████▏| 205552/225000 [395:02:40<53:31:50,  9.91s/it]
 91%|█████████▏| 205553/225000 [395:02:45<45:32:49,  8.43s/it]
 91%|█████████▏| 205554/225000 [395:02:54<47:34:47,  8.81s/it]
 91%|█████████▏| 205555/225000 [395:03:09<57:25:16, 10.63s/it]
 91%|█████████▏| 205556/225000 [395:03:20<56:55:43, 10.54s/it]
 91%|█████████▏| 205557/225000 [395:03:29<54:34:52, 10.11s/it]
 91%|█████████▏| 205558/225000 [395:03:34<46:56:54,  8.69s/it]
 91%|█████████▏| 205559/225000 [395:03:43<47:28:55,  8.79s/it]
 91%|█████████▏| 205560/225000 [395:03:51<45:28:02,  8.42s/it]
 91%|█████████▏| 205561/225000 [395:03:59<45:22:43,  8.40s/it]
 91%|█████████▏| 205562/225000 [395:04:07<44:44:03,  8.28s/it]
 91%|█████████▏| 205563/225000 [395:04:20<52:18:19,  9.69s/it]
 91%|█████████▏| 205564/225000 [395:04:30<53:30:27,  9.91s/it]
 91%|█████████▏| 205565/225000 [395:04:37<48:30:32,  8.99s/it]
 91%|█████████▏| 205566/225000 [395:04:47<50:15:59,  9.31s/it]
 91%|█████████▏| 205567/225000 [395:04:58<51:51:58,  9.61s/it]
 91%|█████████▏| 205568/225000 [395:05:09<55:04:07, 10.20s/it]
 91%|█████████▏| 205569/225000 [395:05:16<48:50:05,  9.05s/it]
 91%|█████████▏| 205570/225000 [395:05:22<44:02:58,  8.16s/it]
 91%|█████████▏| 205571/225000 [395:05:31<46:17:02,  8.58s/it]
 91%|█████████▏| 205572/225000 [395:05:41<48:10:45,  8.93s/it]
 91%|█████████▏| 205573/225000 [395:05:47<43:27:48,  8.05s/it]
 91%|█████████▏| 205574/225000 [395:06:01<53:03:13,  9.83s/it]
 91%|█████████▏| 205575/225000 [395:06:11<54:09:42, 10.04s/it]
 91%|█████████▏| 205576/225000 [395:06:19<50:46:07,  9.41s/it]
 91%|█████████▏| 205577/225000 [395:06:28<49:37:51,  9.20s/it]
 91%|█████████▏| 205578/225000 [395:06:36<47:04:07,  8.72s/it]
 91%|█████████▏| 205579/225000 [395:06:42<43:40:00,  8.09s/it]
 91%|█████████▏| 205580/225000 [395:06:52<45:36:28,  8.45s/it]
 91%|█████████▏| 205581/225000 [395:06:59<44:25:35,  8.24s/it]
 91%|█████████▏| 205582/225000 [395:07:06<41:38:44,  7.72s/it]
 91%|█████████▏| 205583/225000 [395:07:13<40:33:30,  7.52s/it]
 91%|█████████▏| 205584/225000 [395:07:21<41:16:45,  7.65s/it]
 91%|█████████▏| 205585/225000 [395:07:26<37:58:56,  7.04s/it]
 91%|█████████▏| 205586/225000 [395:07:39<46:24:35,  8.61s/it]
 91%|█████████▏| 205587/225000 [395:07:44<40:16:58,  7.47s/it]
 91%|█████████▏| 205588/225000 [395:07:56<47:40:34,  8.84s/it]
 91%|█████████▏| 205589/225000 [395:08:08<52:38:16,  9.76s/it]
 91%|█████████▏| 205590/225000 [395:08:19<55:13:41, 10.24s/it]
 91%|█████████▏| 205591/225000 [395:08:25<48:33:11,  9.01s/it]
 91%|█████████▏| 205592/225000 [395:08:34<47:58:19,  8.90s/it]
 91%|█████████▏| 205593/225000 [395:08:39<42:08:29,  7.82s/it]
 91%|█████████▏| 205594/225000 [395:08:49<45:25:36,  8.43s/it]
 91%|█████████▏| 205595/225000 [395:08:57<45:29:03,  8.44s/it]
 91%|█████████▏| 205596/225000 [395:09:06<45:19:11,  8.41s/it]
 91%|█████████▏| 205597/225000 [395:09:15<47:10:32,  8.75s/it]
 91%|█████████▏| 205598/225000 [395:09:20<40:21:50,  7.49s/it]
 91%|█████████▏| 205599/225000 [395:09:28<40:55:24,  7.59s/it]
 91%|█████████▏| 205600/225000 [395:09:40<48:04:54,  8.92s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.551', 'learning_rate': '1.008e-06', 'epoch': '0.9138', 'num_input_tokens_seen': 665830200, 'train_runtime': '1.423e+06', 'train_tokens_per_second': '468'}

 91%|█████████▏| 205600/225000 [395:09:40<48:04:54,  8.92s/it]
 91%|█████████▏| 205601/225000 [395:09:50<50:21:09,  9.34s/it]
 91%|█████████▏| 205602/225000 [395:09:58<47:37:37,  8.84s/it]
 91%|█████████▏| 205603/225000 [395:10:06<47:04:44,  8.74s/it]
 91%|█████████▏| 205604/225000 [395:10:19<53:37:27,  9.95s/it]
 91%|█████████▏| 205605/225000 [395:10:27<51:12:04,  9.50s/it]
 91%|█████████▏| 205606/225000 [395:10:35<49:00:50,  9.10s/it]
 91%|█████████▏| 205607/225000 [395:10:45<49:53:54,  9.26s/it]
 91%|█████████▏| 205608/225000 [395:10:51<44:49:56,  8.32s/it]
 91%|█████████▏| 205609/225000 [395:10:57<40:02:43,  7.43s/it]
 91%|█████████▏| 205610/225000 [395:11:02<37:26:31,  6.95s/it]
 91%|█████████▏| 205611/225000 [395:11:09<37:26:20,  6.95s/it]
 91%|█████████▏| 205612/225000 [395:11:20<43:14:39,  8.03s/it]
 91%|█████████▏| 205613/225000 [395:11:31<48:57:46,  9.09s/it]
 91%|█████████▏| 205614/225000 [395:11:43<52:20:22,  9.72s/it]
 91%|█████████▏| 205615/225000 [395:11:48<45:03:12,  8.37s/it]
 91%|█████████▏| 205616/225000 [395:11:58<47:35:26,  8.84s/it]
 91%|█████████▏| 205617/225000 [395:12:04<42:50:27,  7.96s/it]
 91%|█████████▏| 205618/225000 [395:12:10<39:34:11,  7.35s/it]
 91%|█████████▏| 205619/225000 [395:12:16<37:37:44,  6.99s/it]
 91%|█████████▏| 205620/225000 [395:12:24<39:06:51,  7.27s/it]
 91%|█████████▏| 205621/225000 [395:12:30<37:18:47,  6.93s/it]
 91%|█████████▏| 205622/225000 [395:12:44<48:52:21,  9.08s/it]
 91%|█████████▏| 205623/225000 [395:12:52<47:30:20,  8.83s/it]
 91%|█████████▏| 205624/225000 [395:13:01<46:48:34,  8.70s/it]
 91%|█████████▏| 205625/225000 [395:13:08<43:57:35,  8.17s/it]
 91%|█████████▏| 205626/225000 [395:13:19<49:26:02,  9.19s/it]
 91%|█████████▏| 205627/225000 [395:13:24<42:59:22,  7.99s/it]
 91%|█████████▏| 205628/225000 [395:13:30<38:49:07,  7.21s/it]
 91%|█████████▏| 205629/225000 [395:13:42<47:23:34,  8.81s/it]
 91%|█████████▏| 205630/225000 [395:13:49<43:32:55,  8.09s/it]
 91%|█████████▏| 205631/225000 [395:13:57<43:15:49,  8.04s/it]
 91%|█████████▏| 205632/225000 [395:14:03<41:21:43,  7.69s/it]
 91%|█████████▏| 205633/225000 [395:14:12<43:32:30,  8.09s/it]
 91%|█████████▏| 205634/225000 [395:14:25<50:13:42,  9.34s/it]
 91%|█████████▏| 205635/225000 [395:14:37<55:47:55, 10.37s/it]
 91%|█████████▏| 205636/225000 [395:14:44<49:44:08,  9.25s/it]
 91%|█████████▏| 205637/225000 [395:14:54<51:32:40,  9.58s/it]
 91%|█████████▏| 205638/225000 [395:15:09<59:29:57, 11.06s/it]
 91%|█████████▏| 205639/225000 [395:15:16<52:15:44,  9.72s/it]
 91%|█████████▏| 205640/225000 [395:15:22<46:24:20,  8.63s/it]
 91%|█████████▏| 205641/225000 [395:15:27<41:00:25,  7.63s/it]
 91%|█████████▏| 205642/225000 [395:15:34<39:24:01,  7.33s/it]
 91%|█████████▏| 205643/225000 [395:15:43<42:06:04,  7.83s/it]
 91%|█████████▏| 205644/225000 [395:15:54<47:50:19,  8.90s/it]
 91%|█████████▏| 205645/225000 [395:15:59<41:58:04,  7.81s/it]
 91%|█████████▏| 205646/225000 [395:16:05<39:04:07,  7.27s/it]
 91%|█████████▏| 205647/225000 [395:16:19<50:00:37,  9.30s/it]
 91%|█████████▏| 205648/225000 [395:16:30<51:59:36,  9.67s/it]
 91%|█████████▏| 205649/225000 [395:16:40<53:29:47,  9.95s/it]
 91%|█████████▏| 205650/225000 [395:16:46<46:09:41,  8.59s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.282', 'learning_rate': '1.002e-06', 'epoch': '0.914', 'num_input_tokens_seen': 666056720, 'train_runtime': '1.423e+06', 'train_tokens_per_second': '468.1'}

 91%|█████████▏| 205650/225000 [395:16:46<46:09:41,  8.59s/it]
 91%|█████████▏| 205651/225000 [395:16:56<48:03:31,  8.94s/it]
 91%|█████████▏| 205652/225000 [395:17:02<44:18:37,  8.24s/it]
 91%|█████████▏| 205653/225000 [395:17:14<49:35:57,  9.23s/it]
 91%|█████████▏| 205654/225000 [395:17:20<44:52:35,  8.35s/it]
 91%|█████████▏| 205655/225000 [395:17:32<51:11:54,  9.53s/it]
 91%|█████████▏| 205656/225000 [395:17:38<44:59:31,  8.37s/it]
 91%|█████████▏| 205657/225000 [395:17:47<46:00:51,  8.56s/it]
 91%|█████████▏| 205658/225000 [395:17:56<46:48:29,  8.71s/it]
 91%|█████████▏| 205659/225000 [395:18:07<50:12:17,  9.34s/it]
 91%|█████████▏| 205660/225000 [395:18:12<43:12:02,  8.04s/it]
 91%|█████████▏| 205661/225000 [395:18:22<46:26:36,  8.65s/it]
 91%|█████████▏| 205662/225000 [395:18:30<45:12:31,  8.42s/it]
 91%|█████████▏| 205663/225000 [395:18:36<42:07:39,  7.84s/it]
 91%|█████████▏| 205664/225000 [395:18:43<39:32:27,  7.36s/it]
 91%|█████████▏| 205665/225000 [395:18:50<39:16:54,  7.31s/it]
 91%|█████████▏| 205666/225000 [395:18:58<41:33:20,  7.74s/it]
 91%|█████████▏| 205667/225000 [395:19:06<41:45:05,  7.77s/it]
 91%|█████████▏| 205668/225000 [395:19:18<47:37:04,  8.87s/it]
 91%|█████████▏| 205669/225000 [395:19:22<40:51:37,  7.61s/it]
 91%|█████████▏| 205670/225000 [395:19:28<38:06:23,  7.10s/it]
 91%|█████████▏| 205671/225000 [395:19:37<40:56:42,  7.63s/it]
 91%|█████████▏| 205672/225000 [395:19:44<40:24:52,  7.53s/it]
 91%|█████████▏| 205673/225000 [395:19:56<46:46:47,  8.71s/it]
 91%|█████████▏| 205674/225000 [395:20:03<44:15:36,  8.24s/it]
 91%|█████████▏| 205675/225000 [395:20:11<44:08:13,  8.22s/it]
 91%|█████████▏| 205676/225000 [395:20:26<54:20:23, 10.12s/it]
 91%|█████████▏| 205677/225000 [395:20:32<47:49:32,  8.91s/it]
 91%|█████████▏| 205678/225000 [395:20:38<43:32:48,  8.11s/it]
 91%|█████████▏| 205679/225000 [395:20:45<41:15:49,  7.69s/it]
 91%|█████████▏| 205680/225000 [395:20:56<46:42:22,  8.70s/it]
 91%|█████████▏| 205681/225000 [395:21:06<48:04:50,  8.96s/it]
 91%|█████████▏| 205682/225000 [395:21:14<48:07:00,  8.97s/it]
 91%|█████████▏| 205683/225000 [395:21:23<47:24:13,  8.83s/it]
 91%|█████████▏| 205684/225000 [395:21:33<49:45:31,  9.27s/it]
 91%|█████████▏| 205685/225000 [395:21:41<47:21:55,  8.83s/it]
 91%|█████████▏| 205686/225000 [395:21:49<45:55:07,  8.56s/it]
 91%|█████████▏| 205687/225000 [395:21:59<48:39:51,  9.07s/it]
 91%|█████████▏| 205688/225000 [395:22:05<42:47:31,  7.98s/it]
 91%|█████████▏| 205689/225000 [395:22:11<40:51:02,  7.62s/it]
 91%|█████████▏| 205690/225000 [395:22:20<42:14:08,  7.87s/it]
 91%|█████████▏| 205691/225000 [395:22:27<40:30:49,  7.55s/it]
 91%|█████████▏| 205692/225000 [395:22:36<42:25:46,  7.91s/it]
 91%|█████████▏| 205693/225000 [395:22:42<40:21:17,  7.52s/it]
 91%|█████████▏| 205694/225000 [395:22:56<50:42:45,  9.46s/it]
 91%|█████████▏| 205695/225000 [395:23:02<45:12:40,  8.43s/it]
 91%|█████████▏| 205696/225000 [395:23:09<42:43:02,  7.97s/it]
 91%|█████████▏| 205697/225000 [395:23:17<42:32:23,  7.93s/it]
 91%|█████████▏| 205698/225000 [395:23:26<44:13:19,  8.25s/it]
 91%|█████████▏| 205699/225000 [395:23:33<42:01:56,  7.84s/it]
 91%|█████████▏| 205700/225000 [395:23:40<40:37:11,  7.58s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.755', 'learning_rate': '9.974e-07', 'epoch': '0.9142', 'num_input_tokens_seen': 666279792, 'train_runtime': '1.423e+06', 'train_tokens_per_second': '468.1'}

 91%|█████████▏| 205700/225000 [395:23:40<40:37:11,  7.58s/it]
 91%|█████████▏| 205701/225000 [395:23:47<39:38:31,  7.39s/it]
 91%|█████████▏| 205702/225000 [395:23:54<39:51:56,  7.44s/it]
 91%|█████████▏| 205703/225000 [395:24:05<46:01:40,  8.59s/it]
 91%|█████████▏| 205704/225000 [395:24:14<46:02:33,  8.59s/it]
 91%|█████████▏| 205705/225000 [395:24:27<52:39:32,  9.82s/it]
 91%|█████████▏| 205706/225000 [395:24:32<45:52:59,  8.56s/it]
 91%|█████████▏| 205707/225000 [395:24:45<52:57:15,  9.88s/it]
 91%|█████████▏| 205708/225000 [395:24:57<55:12:26, 10.30s/it]
 91%|█████████▏| 205709/225000 [395:25:03<49:28:38,  9.23s/it]
 91%|█████████▏| 205710/225000 [395:25:09<43:55:10,  8.20s/it]
 91%|█████████▏| 205711/225000 [395:25:17<43:09:41,  8.06s/it]
 91%|█████████▏| 205712/225000 [395:25:27<46:35:20,  8.70s/it]
 91%|█████████▏| 205713/225000 [395:25:37<48:45:48,  9.10s/it]
 91%|█████████▏| 205714/225000 [395:25:49<52:52:39,  9.87s/it]
 91%|█████████▏| 205715/225000 [395:25:55<46:15:39,  8.64s/it]
 91%|█████████▏| 205716/225000 [395:26:03<46:37:44,  8.70s/it]
 91%|█████████▏| 205717/225000 [395:26:10<43:43:59,  8.16s/it]
 91%|█████████▏| 205718/225000 [395:26:19<44:03:58,  8.23s/it]
 91%|█████████▏| 205719/225000 [395:26:26<41:46:29,  7.80s/it]
 91%|█████████▏| 205720/225000 [395:26:37<48:22:59,  9.03s/it]
 91%|█████████▏| 205721/225000 [395:26:44<44:50:18,  8.37s/it]
 91%|█████████▏| 205722/225000 [395:26:50<41:15:31,  7.70s/it]
 91%|█████████▏| 205723/225000 [395:27:00<44:12:13,  8.26s/it]
 91%|█████████▏| 205724/225000 [395:27:12<50:43:16,  9.47s/it]
 91%|█████████▏| 205725/225000 [395:27:20<47:44:17,  8.92s/it]
 91%|█████████▏| 205726/225000 [395:27:28<46:13:19,  8.63s/it]
 91%|█████████▏| 205727/225000 [395:27:39<49:34:35,  9.26s/it]
 91%|█████████▏| 205728/225000 [395:27:47<48:48:09,  9.12s/it]
 91%|█████████▏| 205729/225000 [395:27:54<44:17:06,  8.27s/it]
 91%|█████████▏| 205730/225000 [395:28:02<45:11:24,  8.44s/it]
 91%|█████████▏| 205731/225000 [395:28:10<43:06:32,  8.05s/it]
 91%|█████████▏| 205732/225000 [395:28:16<39:53:16,  7.45s/it]
 91%|█████████▏| 205733/225000 [395:28:23<39:12:19,  7.33s/it]
 91%|█████████▏| 205734/225000 [395:28:31<41:11:57,  7.70s/it]
 91%|█████████▏| 205735/225000 [395:28:45<50:19:04,  9.40s/it]
 91%|█████████▏| 205736/225000 [395:28:50<44:32:44,  8.32s/it]
 91%|█████████▏| 205737/225000 [395:28:58<43:40:52,  8.16s/it]
 91%|█████████▏| 205738/225000 [395:29:05<41:41:18,  7.79s/it]
 91%|█████████▏| 205739/225000 [395:29:17<47:51:25,  8.94s/it]
 91%|█████████▏| 205740/225000 [395:29:26<48:37:51,  9.09s/it]
 91%|█████████▏| 205741/225000 [395:29:36<49:10:10,  9.19s/it]
 91%|█████████▏| 205742/225000 [395:29:43<45:58:44,  8.60s/it]
 91%|█████████▏| 205743/225000 [395:29:50<43:57:17,  8.22s/it]
 91%|█████████▏| 205744/225000 [395:29:59<44:20:07,  8.29s/it]
 91%|█████████▏| 205745/225000 [395:30:05<40:31:31,  7.58s/it]
 91%|█████████▏| 205746/225000 [395:30:11<38:16:23,  7.16s/it]
 91%|█████████▏| 205747/225000 [395:30:18<37:42:04,  7.05s/it]
 91%|█████████▏| 205748/225000 [395:30:27<42:09:24,  7.88s/it]
 91%|█████████▏| 205749/225000 [395:30:39<47:44:22,  8.93s/it]
 91%|█████████▏| 205750/225000 [395:30:49<49:48:59,  9.32s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.416', 'learning_rate': '9.923e-07', 'epoch': '0.9144', 'num_input_tokens_seen': 666510416, 'train_runtime': '1.424e+06', 'train_tokens_per_second': '468.1'}

 91%|█████████▏| 205750/225000 [395:30:49<49:48:59,  9.32s/it]
 91%|█████████▏| 205751/225000 [395:30:57<47:45:02,  8.93s/it]
 91%|█████████▏| 205752/225000 [395:31:03<43:28:46,  8.13s/it]
 91%|█████████▏| 205753/225000 [395:31:13<45:48:41,  8.57s/it]
 91%|█████████▏| 205754/225000 [395:31:22<47:08:33,  8.82s/it]
 91%|█████████▏| 205755/225000 [395:31:32<47:58:55,  8.98s/it]
 91%|█████████▏| 205756/225000 [395:31:37<42:38:56,  7.98s/it]
 91%|█████████▏| 205757/225000 [395:31:48<47:42:14,  8.92s/it]
 91%|█████████▏| 205758/225000 [395:31:58<49:11:39,  9.20s/it]
 91%|█████████▏| 205759/225000 [395:32:02<41:03:13,  7.68s/it]
 91%|█████████▏| 205760/225000 [395:32:12<44:01:07,  8.24s/it]
 91%|█████████▏| 205761/225000 [395:32:21<44:42:39,  8.37s/it]
 91%|█████████▏| 205762/225000 [395:32:28<43:28:05,  8.13s/it]
 91%|█████████▏| 205763/225000 [395:32:35<41:01:19,  7.68s/it]
 91%|█████████▏| 205764/225000 [395:32:43<42:37:09,  7.98s/it]
 91%|█████████▏| 205765/225000 [395:32:52<42:57:51,  8.04s/it]
 91%|█████████▏| 205766/225000 [395:33:01<44:16:29,  8.29s/it]
 91%|█████████▏| 205767/225000 [395:33:14<52:48:10,  9.88s/it]
 91%|█████████▏| 205768/225000 [395:33:24<52:38:41,  9.85s/it]
 91%|█████████▏| 205769/225000 [395:33:31<48:40:43,  9.11s/it]
 91%|█████████▏| 205770/225000 [395:33:37<42:56:00,  8.04s/it]
 91%|█████████▏| 205771/225000 [395:33:45<43:41:48,  8.18s/it]
 91%|█████████▏| 205772/225000 [395:33:55<45:27:06,  8.51s/it]
 91%|█████████▏| 205773/225000 [395:33:59<39:28:04,  7.39s/it]
 91%|█████████▏| 205774/225000 [395:34:05<36:57:46,  6.92s/it]
 91%|█████████▏| 205775/225000 [395:34:20<48:55:26,  9.16s/it]
 91%|█████████▏| 205776/225000 [395:34:26<44:54:38,  8.41s/it]
 91%|█████████▏| 205777/225000 [395:34:34<44:01:30,  8.24s/it]
 91%|█████████▏| 205778/225000 [395:34:41<42:05:47,  7.88s/it]
 91%|█████████▏| 205779/225000 [395:34:51<44:37:56,  8.36s/it]
 91%|█████████▏| 205780/225000 [395:35:00<46:34:00,  8.72s/it]
 91%|█████████▏| 205781/225000 [395:35:06<42:08:13,  7.89s/it]
 91%|█████████▏| 205782/225000 [395:35:16<45:45:46,  8.57s/it]
 91%|█████████▏| 205783/225000 [395:35:29<51:52:07,  9.72s/it]
 91%|█████████▏| 205784/225000 [395:35:38<51:04:57,  9.57s/it]
 91%|█████████▏| 205785/225000 [395:35:43<44:11:10,  8.28s/it]
 91%|█████████▏| 205786/225000 [395:35:50<42:00:56,  7.87s/it]
 91%|█████████▏| 205787/225000 [395:35:57<41:13:41,  7.73s/it]
 91%|█████████▏| 205788/225000 [395:36:06<41:48:34,  7.83s/it]
 91%|█████████▏| 205789/225000 [395:36:13<40:56:30,  7.67s/it]
 91%|█████████▏| 205790/225000 [395:36:21<42:03:27,  7.88s/it]
 91%|█████████▏| 205791/225000 [395:36:36<52:51:57,  9.91s/it]
 91%|█████████▏| 205792/225000 [395:36:46<53:14:33,  9.98s/it]
 91%|█████████▏| 205793/225000 [395:36:53<48:04:29,  9.01s/it]
 91%|█████████▏| 205794/225000 [395:37:02<48:51:20,  9.16s/it]
 91%|█████████▏| 205795/225000 [395:37:11<47:25:47,  8.89s/it]
 91%|█████████▏| 205796/225000 [395:37:24<55:04:46, 10.33s/it]
 91%|█████████▏| 205797/225000 [395:37:29<46:18:51,  8.68s/it]
 91%|█████████▏| 205798/225000 [395:37:38<46:57:41,  8.80s/it]
 91%|█████████▏| 205799/225000 [395:37:52<54:13:19, 10.17s/it]
 91%|█████████▏| 205800/225000 [395:38:00<52:03:52,  9.76s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.064', 'learning_rate': '9.872e-07', 'epoch': '0.9147', 'num_input_tokens_seen': 666742080, 'train_runtime': '1.424e+06', 'train_tokens_per_second': '468.1'}

 91%|█████████▏| 205800/225000 [395:38:00<52:03:52,  9.76s/it]
 91%|█████████▏| 205801/225000 [395:38:09<50:27:41,  9.46s/it]
 91%|█████████▏| 205802/225000 [395:38:19<51:15:53,  9.61s/it]
 91%|█████████▏| 205803/225000 [395:38:27<48:14:07,  9.05s/it]
 91%|█████████▏| 205804/225000 [395:38:39<53:00:30,  9.94s/it]
 91%|█████████▏| 205805/225000 [395:38:48<52:29:56,  9.85s/it]
 91%|█████████▏| 205806/225000 [395:38:59<52:58:42,  9.94s/it]
 91%|█████████▏| 205807/225000 [395:39:05<47:08:57,  8.84s/it]
 91%|█████████▏| 205808/225000 [395:39:11<43:11:32,  8.10s/it]
 91%|█████████▏| 205809/225000 [395:39:19<42:56:06,  8.05s/it]
 91%|█████████▏| 205810/225000 [395:39:27<42:32:22,  7.98s/it]
 91%|█████████▏| 205811/225000 [395:39:32<37:44:03,  7.08s/it]
 91%|█████████▏| 205812/225000 [395:39:43<43:45:15,  8.21s/it]
 91%|█████████▏| 205813/225000 [395:39:49<40:04:20,  7.52s/it]
 91%|█████████▏| 205814/225000 [395:39:54<36:15:04,  6.80s/it]
 91%|█████████▏| 205815/225000 [395:40:06<44:56:39,  8.43s/it]
 91%|█████████▏| 205816/225000 [395:40:14<44:23:06,  8.33s/it]
 91%|█████████▏| 205817/225000 [395:40:25<48:19:31,  9.07s/it]
 91%|█████████▏| 205818/225000 [395:40:37<53:28:29, 10.04s/it]
 91%|█████████▏| 205819/225000 [395:40:45<50:00:17,  9.39s/it]
 91%|█████████▏| 205820/225000 [395:40:55<50:12:12,  9.42s/it]
 91%|█████████▏| 205821/225000 [395:41:05<51:42:11,  9.70s/it]
 91%|█████████▏| 205822/225000 [395:41:10<43:45:43,  8.21s/it]
 91%|█████████▏| 205823/225000 [395:41:18<43:28:05,  8.16s/it]
 91%|█████████▏| 205824/225000 [395:41:25<41:27:25,  7.78s/it]
 91%|█████████▏| 205825/225000 [395:41:34<43:28:57,  8.16s/it]
 91%|█████████▏| 205826/225000 [395:41:40<41:00:12,  7.70s/it]
 91%|█████████▏| 205827/225000 [395:41:46<37:48:02,  7.10s/it]
 91%|█████████▏| 205828/225000 [395:41:52<36:11:59,  6.80s/it]
 91%|█████████▏| 205829/225000 [395:42:01<39:28:07,  7.41s/it]
 91%|█████████▏| 205830/225000 [395:42:09<39:51:13,  7.48s/it]
 91%|█████████▏| 205831/225000 [395:42:19<44:05:31,  8.28s/it]
 91%|█████████▏| 205832/225000 [395:42:24<38:55:45,  7.31s/it]
 91%|█████████▏| 205833/225000 [395:42:34<43:35:37,  8.19s/it]
 91%|█████████▏| 205834/225000 [395:42:47<51:41:48,  9.71s/it]
 91%|█████████▏| 205835/225000 [395:42:57<51:19:53,  9.64s/it]
 91%|█████████▏| 205836/225000 [395:43:03<45:43:54,  8.59s/it]
 91%|█████████▏| 205837/225000 [395:43:12<47:14:16,  8.87s/it]
 91%|█████████▏| 205838/225000 [395:43:19<43:02:50,  8.09s/it]
 91%|█████████▏| 205839/225000 [395:43:24<38:49:08,  7.29s/it]
 91%|█████████▏| 205840/225000 [395:43:38<48:33:49,  9.12s/it]
 91%|█████████▏| 205841/225000 [395:43:46<47:05:42,  8.85s/it]
 91%|█████████▏| 205842/225000 [395:43:54<46:02:11,  8.65s/it]
 91%|█████████▏| 205843/225000 [395:44:01<42:54:56,  8.06s/it]
 91%|█████████▏| 205844/225000 [395:44:11<46:56:57,  8.82s/it]
 91%|█████████▏| 205845/225000 [395:44:17<42:15:32,  7.94s/it]
 91%|█████████▏| 205846/225000 [395:44:30<49:32:42,  9.31s/it]
 91%|█████████▏| 205847/225000 [395:44:35<43:54:56,  8.25s/it]
 91%|█████████▏| 205848/225000 [395:44:45<46:02:52,  8.66s/it]
 91%|█████████▏| 205849/225000 [395:45:00<55:40:44, 10.47s/it]
 91%|█████████▏| 205850/225000 [395:45:06<49:13:33,  9.25s/it]
                                                              
{'loss': '1.341', 'grad_norm': '1.509', 'learning_rate': '9.821e-07', 'epoch': '0.9149', 'num_input_tokens_seen': 666975480, 'train_runtime': '1.425e+06', 'train_tokens_per_second': '468.1'}

 91%|█████████▏| 205850/225000 [395:45:06<49:13:33,  9.25s/it]
 91%|█████████▏| 205851/225000 [395:45:16<49:27:38,  9.30s/it]
 91%|█████████▏| 205852/225000 [395:45:30<57:35:06, 10.83s/it]
 91%|█████████▏| 205853/225000 [395:45:36<49:39:00,  9.34s/it]
 91%|█████████▏| 205854/225000 [395:45:42<44:40:28,  8.40s/it]
 91%|█████████▏| 205855/225000 [395:45:53<48:43:16,  9.16s/it]
 91%|█████████▏| 205856/225000 [395:46:00<45:26:04,  8.54s/it]
 91%|█████████▏| 205857/225000 [395:46:10<47:53:50,  9.01s/it]
 91%|█████████▏| 205858/225000 [395:46:17<45:05:46,  8.48s/it]
 91%|█████████▏| 205859/225000 [395:46:23<40:45:07,  7.66s/it]
 91%|█████████▏| 205860/225000 [395:46:31<40:52:27,  7.69s/it]
 91%|█████████▏| 205861/225000 [395:46:37<38:18:38,  7.21s/it]
 91%|█████████▏| 205862/225000 [395:46:44<38:46:55,  7.30s/it]
 91%|█████████▏| 205863/225000 [395:46:53<40:51:57,  7.69s/it]
 91%|█████████▏| 205864/225000 [395:47:00<39:12:47,  7.38s/it]
 91%|█████████▏| 205865/225000 [395:47:06<38:12:54,  7.19s/it]
 91%|█████████▏| 205866/225000 [395:47:23<52:34:13,  9.89s/it]
 91%|█████████▏| 205867/225000 [395:47:32<51:16:38,  9.65s/it]
 91%|█████████▏| 205868/225000 [395:47:40<48:44:44,  9.17s/it]
 91%|█████████▏| 205869/225000 [395:47:49<49:14:42,  9.27s/it]
 91%|█████████▏| 205870/225000 [395:47:57<47:03:57,  8.86s/it]
 91%|█████████▏| 205871/225000 [395:48:07<49:12:31,  9.26s/it]
 91%|█████████▏| 205872/225000 [395:48:14<44:58:34,  8.46s/it]
 91%|█████████▏| 205873/225000 [395:48:19<39:04:19,  7.35s/it]
 91%|█████████▏| 205874/225000 [395:48:34<50:54:04,  9.58s/it]
 92%|█████████▏| 205875/225000 [395:48:41<46:59:09,  8.84s/it]
 92%|█████████▏| 205876/225000 [395:48:49<45:25:16,  8.55s/it]
 92%|█████████▏| 205877/225000 [395:48:57<45:58:44,  8.66s/it]
 92%|█████████▏| 205878/225000 [395:49:08<48:24:01,  9.11s/it]
 92%|█████████▏| 205879/225000 [395:49:16<47:44:32,  8.99s/it]
 92%|█████████▏| 205880/225000 [395:49:23<43:40:28,  8.22s/it]
 92%|█████████▏| 205881/225000 [395:49:31<43:51:30,  8.26s/it]
 92%|█████████▏| 205882/225000 [395:49:39<43:47:18,  8.25s/it]
 92%|█████████▏| 205883/225000 [395:49:51<49:58:06,  9.41s/it]
 92%|█████████▏| 205884/225000 [395:49:59<47:48:04,  9.00s/it]
 92%|█████████▏| 205885/225000 [395:50:12<53:39:26, 10.11s/it]
 92%|█████████▏| 205886/225000 [395:50:17<44:33:11,  8.39s/it]
 92%|█████████▏| 205887/225000 [395:50:26<46:30:26,  8.76s/it]
 92%|█████████▏| 205888/225000 [395:50:32<41:40:13,  7.85s/it]
 92%|█████████▏| 205889/225000 [395:50:41<44:19:04,  8.35s/it]
 92%|█████████▏| 205890/225000 [395:50:52<48:37:37,  9.16s/it]
 92%|█████████▏| 205891/225000 [395:50:58<42:23:13,  7.99s/it]
 92%|█████████▏| 205892/225000 [395:51:03<38:42:25,  7.29s/it]
 92%|█████████▏| 205893/225000 [395:51:10<37:06:05,  6.99s/it]
 92%|█████████▏| 205894/225000 [395:51:16<36:35:05,  6.89s/it]
 92%|█████████▏| 205895/225000 [395:51:30<47:32:06,  8.96s/it]
 92%|█████████▏| 205896/225000 [395:51:37<43:46:01,  8.25s/it]
 92%|█████████▏| 205897/225000 [395:51:47<47:34:10,  8.96s/it]
 92%|█████████▏| 205898/225000 [395:51:53<41:31:23,  7.83s/it]
 92%|█████████▏| 205899/225000 [395:51:59<39:41:37,  7.48s/it]
 92%|█████████▏| 205900/225000 [395:52:10<45:43:26,  8.62s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.344', 'learning_rate': '9.77e-07', 'epoch': '0.9151', 'num_input_tokens_seen': 667206800, 'train_runtime': '1.425e+06', 'train_tokens_per_second': '468.2'}

 92%|█████████▏| 205900/225000 [395:52:10<45:43:26,  8.62s/it]
 92%|█████████▏| 205901/225000 [395:52:22<50:37:14,  9.54s/it]
 92%|█████████▏| 205902/225000 [395:52:27<43:45:36,  8.25s/it]
 92%|█████████▏| 205903/225000 [395:52:43<56:01:48, 10.56s/it]
 92%|█████████▏| 205904/225000 [395:52:52<53:06:30, 10.01s/it]
 92%|█████████▏| 205905/225000 [395:52:58<46:33:05,  8.78s/it]
 92%|█████████▏| 205906/225000 [395:53:06<45:23:51,  8.56s/it]
 92%|█████████▏| 205907/225000 [395:53:18<51:06:59,  9.64s/it]
 92%|█████████▏| 205908/225000 [395:53:26<48:25:02,  9.13s/it]
 92%|█████████▏| 205909/225000 [395:53:33<44:12:10,  8.34s/it]
 92%|█████████▏| 205910/225000 [395:53:41<44:46:34,  8.44s/it]
 92%|█████████▏| 205911/225000 [395:53:51<47:03:22,  8.87s/it]
 92%|█████████▏| 205912/225000 [395:54:02<50:47:27,  9.58s/it]
 92%|█████████▏| 205913/225000 [395:54:13<52:12:04,  9.85s/it]
 92%|█████████▏| 205914/225000 [395:54:18<45:21:56,  8.56s/it]
 92%|█████████▏| 205915/225000 [395:54:28<47:22:15,  8.94s/it]
 92%|█████████▏| 205916/225000 [395:54:40<51:26:43,  9.70s/it]
 92%|█████████▏| 205917/225000 [395:54:51<53:20:19, 10.06s/it]
 92%|█████████▏| 205918/225000 [395:54:55<45:02:32,  8.50s/it]
 92%|█████████▏| 205919/225000 [395:55:06<47:56:37,  9.05s/it]
 92%|█████████▏| 205920/225000 [395:55:13<44:47:57,  8.45s/it]
 92%|█████████▏| 205921/225000 [395:55:20<42:57:58,  8.11s/it]
 92%|█████████▏| 205922/225000 [395:55:28<41:45:36,  7.88s/it]
 92%|█████████▏| 205923/225000 [395:55:35<41:21:16,  7.80s/it]
 92%|█████████▏| 205924/225000 [395:55:42<39:50:22,  7.52s/it]
 92%|█████████▏| 205925/225000 [395:55:48<37:19:38,  7.04s/it]
 92%|█████████▏| 205926/225000 [395:56:04<51:35:05,  9.74s/it]
 92%|█████████▏| 205927/225000 [395:56:15<53:51:32, 10.17s/it]
 92%|█████████▏| 205928/225000 [395:56:22<49:17:55,  9.31s/it]
 92%|█████████▏| 205929/225000 [395:56:32<49:08:51,  9.28s/it]
 92%|█████████▏| 205930/225000 [395:56:37<42:29:34,  8.02s/it]
 92%|█████████▏| 205931/225000 [395:56:45<42:30:58,  8.03s/it]
 92%|█████████▏| 205932/225000 [395:56:51<39:46:45,  7.51s/it]
 92%|█████████▏| 205933/225000 [395:57:07<53:08:35, 10.03s/it]
 92%|█████████▏| 205934/225000 [395:57:15<49:18:28,  9.31s/it]
 92%|█████████▏| 205935/225000 [395:57:24<49:23:18,  9.33s/it]
 92%|█████████▏| 205936/225000 [395:57:30<43:31:09,  8.22s/it]
 92%|█████████▏| 205937/225000 [395:57:35<38:57:11,  7.36s/it]
 92%|█████████▏| 205938/225000 [395:57:45<43:23:16,  8.19s/it]
 92%|█████████▏| 205939/225000 [395:57:55<45:24:17,  8.58s/it]
 92%|█████████▏| 205940/225000 [395:57:59<38:53:07,  7.34s/it]
 92%|█████████▏| 205941/225000 [395:58:07<40:04:23,  7.57s/it]
 92%|█████████▏| 205942/225000 [395:58:14<38:42:27,  7.31s/it]
 92%|█████████▏| 205943/225000 [395:58:25<45:11:20,  8.54s/it]
 92%|█████████▏| 205944/225000 [395:58:32<42:17:47,  7.99s/it]
 92%|█████████▏| 205945/225000 [395:58:44<49:21:19,  9.32s/it]
 92%|█████████▏| 205946/225000 [395:58:54<50:32:56,  9.55s/it]
 92%|█████████▏| 205947/225000 [395:59:03<48:12:24,  9.11s/it]
 92%|█████████▏| 205948/225000 [395:59:10<45:07:50,  8.53s/it]
 92%|█████████▏| 205949/225000 [395:59:19<46:00:42,  8.69s/it]
 92%|█████████▏| 205950/225000 [395:59:24<40:35:46,  7.67s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.272', 'learning_rate': '9.72e-07', 'epoch': '0.9153', 'num_input_tokens_seen': 667436384, 'train_runtime': '1.426e+06', 'train_tokens_per_second': '468.2'}

 92%|█████████▏| 205950/225000 [395:59:24<40:35:46,  7.67s/it]
 92%|█████████▏| 205951/225000 [395:59:36<47:32:28,  8.98s/it]
 92%|█████████▏| 205952/225000 [395:59:42<42:53:08,  8.11s/it]
 92%|█████████▏| 205953/225000 [395:59:56<51:25:44,  9.72s/it]
 92%|█████████▏| 205954/225000 [396:00:00<43:13:16,  8.17s/it]
 92%|█████████▏| 205955/225000 [396:00:10<45:42:03,  8.64s/it]
 92%|█████████▏| 205956/225000 [396:00:16<41:08:20,  7.78s/it]
 92%|█████████▏| 205957/225000 [396:00:26<44:33:04,  8.42s/it]
 92%|█████████▏| 205958/225000 [396:00:33<42:35:37,  8.05s/it]
 92%|█████████▏| 205959/225000 [396:00:46<51:20:31,  9.71s/it]
 92%|█████████▏| 205960/225000 [396:00:54<48:09:38,  9.11s/it]
 92%|█████████▏| 205961/225000 [396:01:00<42:53:51,  8.11s/it]
 92%|█████████▏| 205962/225000 [396:01:12<48:28:39,  9.17s/it]
 92%|█████████▏| 205963/225000 [396:01:19<45:45:49,  8.65s/it]
 92%|█████████▏| 205964/225000 [396:01:30<49:02:40,  9.28s/it]
 92%|█████████▏| 205965/225000 [396:01:38<46:55:35,  8.88s/it]
 92%|█████████▏| 205966/225000 [396:01:46<46:07:23,  8.72s/it]
 92%|█████████▏| 205967/225000 [396:01:54<44:08:30,  8.35s/it]
 92%|█████████▏| 205968/225000 [396:02:06<50:43:36,  9.60s/it]
 92%|█████████▏| 205969/225000 [396:02:11<44:00:05,  8.32s/it]
 92%|█████████▏| 205970/225000 [396:02:18<40:48:06,  7.72s/it]
 92%|█████████▏| 205971/225000 [396:02:23<36:44:47,  6.95s/it]
 92%|█████████▏| 205972/225000 [396:02:34<43:13:47,  8.18s/it]
 92%|█████████▏| 205973/225000 [396:02:45<47:35:28,  9.00s/it]
 92%|█████████▏| 205974/225000 [396:02:53<46:00:44,  8.71s/it]
 92%|█████████▏| 205975/225000 [396:03:00<43:03:51,  8.15s/it]
 92%|█████████▏| 205976/225000 [396:03:16<55:40:40, 10.54s/it]
 92%|█████████▏| 205977/225000 [396:03:28<58:22:18, 11.05s/it]
 92%|█████████▏| 205978/225000 [396:03:37<55:03:31, 10.42s/it]
 92%|█████████▏| 205979/225000 [396:03:45<50:30:44,  9.56s/it]
 92%|█████████▏| 205980/225000 [396:03:56<53:02:29, 10.04s/it]
 92%|█████████▏| 205981/225000 [396:04:06<52:54:16, 10.01s/it]
 92%|█████████▏| 205982/225000 [396:04:12<46:30:13,  8.80s/it]
 92%|█████████▏| 205983/225000 [396:04:16<40:15:26,  7.62s/it]
 92%|█████████▏| 205984/225000 [396:04:27<45:09:47,  8.55s/it]
 92%|█████████▏| 205985/225000 [396:04:36<45:17:05,  8.57s/it]
 92%|█████████▏| 205986/225000 [396:04:47<48:54:07,  9.26s/it]
 92%|█████████▏| 205987/225000 [396:04:56<48:37:57,  9.21s/it]
 92%|█████████▏| 205988/225000 [396:05:03<44:51:47,  8.50s/it]
 92%|█████████▏| 205989/225000 [396:05:12<45:57:22,  8.70s/it]
 92%|█████████▏| 205990/225000 [396:05:17<40:12:54,  7.62s/it]
 92%|█████████▏| 205991/225000 [396:05:24<38:49:53,  7.35s/it]
 92%|█████████▏| 205992/225000 [396:05:41<54:33:32, 10.33s/it]
 92%|█████████▏| 205993/225000 [396:05:47<48:31:41,  9.19s/it]
 92%|█████████▏| 205994/225000 [396:05:55<45:35:25,  8.64s/it]
 92%|█████████▏| 205995/225000 [396:06:04<46:46:42,  8.86s/it]
 92%|█████████▏| 205996/225000 [396:06:10<42:34:31,  8.07s/it]
 92%|█████████▏| 205997/225000 [396:06:23<49:31:46,  9.38s/it]
 92%|█████████▏| 205998/225000 [396:06:29<44:58:34,  8.52s/it]
 92%|█████████▏| 205999/225000 [396:06:35<39:52:41,  7.56s/it]
 92%|█████████▏| 206000/225000 [396:06:46<45:13:36,  8.57s/it]
                                                              
{'loss': '1.278', 'grad_norm': '2.259', 'learning_rate': '9.669e-07', 'epoch': '0.9156', 'num_input_tokens_seen': 667675200, 'train_runtime': '1.426e+06', 'train_tokens_per_second': '468.2'}

 92%|█████████▏| 206000/225000 [396:06:46<45:13:36,  8.57s/it]
 92%|█████████▏| 206001/225000 [396:06:56<48:28:07,  9.18s/it]
 92%|█████████▏| 206002/225000 [396:07:04<45:41:38,  8.66s/it]
 92%|█████████▏| 206003/225000 [396:07:14<49:09:32,  9.32s/it]
 92%|█████████▏| 206004/225000 [396:07:20<43:03:09,  8.16s/it]
 92%|█████████▏| 206005/225000 [396:07:25<38:35:00,  7.31s/it]
 92%|█████████▏| 206006/225000 [396:07:32<37:09:36,  7.04s/it]
 92%|█████████▏| 206007/225000 [396:07:41<41:24:21,  7.85s/it]
 92%|█████████▏| 206008/225000 [396:07:54<48:33:59,  9.21s/it]
 92%|█████████▏| 206009/225000 [396:08:04<49:22:10,  9.36s/it]
 92%|█████████▏| 206010/225000 [396:08:09<43:44:56,  8.29s/it]
 92%|█████████▏| 206011/225000 [396:08:26<56:28:15, 10.71s/it]
 92%|█████████▏| 206012/225000 [396:08:36<55:27:16, 10.51s/it]
 92%|█████████▏| 206013/225000 [396:08:47<56:47:26, 10.77s/it]
 92%|█████████▏| 206014/225000 [396:08:57<54:49:15, 10.39s/it]
 92%|█████████▏| 206015/225000 [396:09:01<46:06:14,  8.74s/it]
 92%|█████████▏| 206016/225000 [396:09:11<47:49:26,  9.07s/it]
 92%|█████████▏| 206017/225000 [396:09:18<44:16:55,  8.40s/it]
 92%|█████████▏| 206018/225000 [396:09:26<42:40:32,  8.09s/it]
 92%|█████████▏| 206019/225000 [396:09:37<47:34:27,  9.02s/it]
 92%|█████████▏| 206020/225000 [396:09:43<43:09:57,  8.19s/it]
 92%|█████████▏| 206021/225000 [396:09:52<45:07:02,  8.56s/it]
 92%|█████████▏| 206022/225000 [396:10:03<49:01:46,  9.30s/it]
 92%|█████████▏| 206023/225000 [396:10:09<43:12:49,  8.20s/it]
 92%|█████████▏| 206024/225000 [396:10:18<44:44:47,  8.49s/it]
 92%|█████████▏| 206025/225000 [396:10:25<41:49:26,  7.94s/it]
 92%|█████████▏| 206026/225000 [396:10:33<42:12:40,  8.01s/it]
 92%|█████████▏| 206027/225000 [396:10:47<51:03:55,  9.69s/it]
 92%|█████████▏| 206028/225000 [396:10:56<50:36:35,  9.60s/it]
 92%|█████████▏| 206029/225000 [396:11:07<51:58:35,  9.86s/it]
 92%|█████████▏| 206030/225000 [396:11:15<49:40:30,  9.43s/it]
 92%|█████████▏| 206031/225000 [396:11:25<51:01:44,  9.68s/it]
 92%|█████████▏| 206032/225000 [396:11:33<47:44:18,  9.06s/it]
 92%|█████████▏| 206033/225000 [396:11:38<42:05:27,  7.99s/it]
 92%|█████████▏| 206034/225000 [396:11:49<45:36:20,  8.66s/it]
 92%|█████████▏| 206035/225000 [396:11:54<40:26:54,  7.68s/it]
 92%|█████████▏| 206036/225000 [396:12:06<47:06:29,  8.94s/it]
 92%|█████████▏| 206037/225000 [396:12:11<40:25:17,  7.67s/it]
 92%|█████████▏| 206038/225000 [396:12:21<44:35:57,  8.47s/it]
 92%|█████████▏| 206039/225000 [396:12:28<42:15:28,  8.02s/it]
 92%|█████████▏| 206040/225000 [396:12:35<41:17:02,  7.84s/it]
 92%|█████████▏| 206041/225000 [396:12:44<42:28:53,  8.07s/it]
 92%|█████████▏| 206042/225000 [396:12:56<48:28:52,  9.21s/it]
 92%|█████████▏| 206043/225000 [396:13:00<40:41:06,  7.73s/it]
 92%|█████████▏| 206044/225000 [396:13:11<45:46:06,  8.69s/it]
 92%|█████████▏| 206045/225000 [396:13:22<49:33:40,  9.41s/it]
 92%|█████████▏| 206046/225000 [396:13:37<58:08:51, 11.04s/it]
 92%|█████████▏| 206047/225000 [396:13:46<54:41:03, 10.39s/it]
 92%|█████████▏| 206048/225000 [396:13:54<51:48:29,  9.84s/it]
 92%|█████████▏| 206049/225000 [396:14:01<47:21:40,  9.00s/it]
 92%|█████████▏| 206050/225000 [396:14:07<42:26:59,  8.06s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.298', 'learning_rate': '9.619e-07', 'epoch': '0.9158', 'num_input_tokens_seen': 667910032, 'train_runtime': '1.426e+06', 'train_tokens_per_second': '468.2'}

 92%|█████████▏| 206050/225000 [396:14:07<42:26:59,  8.06s/it]
 92%|█████████▏| 206051/225000 [396:14:14<39:57:24,  7.59s/it]
 92%|█████████▏| 206052/225000 [396:14:20<37:24:25,  7.11s/it]
 92%|█████████▏| 206053/225000 [396:14:32<45:59:13,  8.74s/it]
 92%|█████████▏| 206054/225000 [396:14:42<47:38:11,  9.05s/it]
 92%|█████████▏| 206055/225000 [396:14:51<47:44:48,  9.07s/it]
 92%|█████████▏| 206056/225000 [396:14:55<39:38:03,  7.53s/it]
 92%|█████████▏| 206057/225000 [396:15:04<41:13:04,  7.83s/it]
 92%|█████████▏| 206058/225000 [396:15:12<41:44:35,  7.93s/it]
 92%|█████████▏| 206059/225000 [396:15:22<45:07:12,  8.58s/it]
 92%|█████████▏| 206060/225000 [396:15:31<46:01:34,  8.75s/it]
 92%|█████████▏| 206061/225000 [396:15:41<48:26:47,  9.21s/it]
 92%|█████████▏| 206062/225000 [396:15:47<42:24:21,  8.06s/it]
 92%|█████████▏| 206063/225000 [396:15:58<48:03:08,  9.13s/it]
 92%|█████████▏| 206064/225000 [396:16:02<39:47:30,  7.56s/it]
 92%|█████████▏| 206065/225000 [396:16:12<42:32:57,  8.09s/it]
 92%|█████████▏| 206066/225000 [396:16:19<40:50:20,  7.76s/it]
 92%|█████████▏| 206067/225000 [396:16:29<45:29:12,  8.65s/it]
 92%|█████████▏| 206068/225000 [396:16:34<39:20:35,  7.48s/it]
 92%|█████████▏| 206069/225000 [396:16:40<37:45:59,  7.18s/it]
 92%|█████████▏| 206070/225000 [396:16:55<49:43:46,  9.46s/it]
 92%|█████████▏| 206071/225000 [396:17:04<48:52:25,  9.30s/it]
 92%|█████████▏| 206072/225000 [396:17:09<41:44:32,  7.94s/it]
 92%|█████████▏| 206073/225000 [396:17:19<45:31:43,  8.66s/it]
 92%|█████████▏| 206074/225000 [396:17:24<39:18:34,  7.48s/it]
 92%|█████████▏| 206075/225000 [396:17:34<43:18:57,  8.24s/it]
 92%|█████████▏| 206076/225000 [396:17:43<43:47:03,  8.33s/it]
 92%|█████████▏| 206077/225000 [396:17:51<43:54:29,  8.35s/it]
 92%|█████████▏| 206078/225000 [396:17:56<39:14:04,  7.46s/it]
 92%|█████████▏| 206079/225000 [396:18:11<50:57:25,  9.70s/it]
 92%|█████████▏| 206080/225000 [396:18:18<45:42:11,  8.70s/it]
 92%|█████████▏| 206081/225000 [396:18:24<41:29:33,  7.90s/it]
 92%|█████████▏| 206082/225000 [396:18:29<37:48:09,  7.19s/it]
 92%|█████████▏| 206083/225000 [396:18:37<39:32:07,  7.52s/it]
 92%|█████████▏| 206084/225000 [396:18:54<53:38:30, 10.21s/it]
 92%|█████████▏| 206085/225000 [396:18:59<45:17:04,  8.62s/it]
 92%|█████████▏| 206086/225000 [396:19:11<51:33:18,  9.81s/it]
 92%|█████████▏| 206087/225000 [396:19:17<44:10:56,  8.41s/it]
 92%|█████████▏| 206088/225000 [396:19:27<46:37:54,  8.88s/it]
 92%|█████████▏| 206089/225000 [396:19:34<44:38:48,  8.50s/it]
 92%|█████████▏| 206090/225000 [396:19:45<48:43:11,  9.28s/it]
 92%|█████████▏| 206091/225000 [396:19:53<46:00:25,  8.76s/it]
 92%|█████████▏| 206092/225000 [396:19:59<42:07:05,  8.02s/it]
 92%|█████████▏| 206093/225000 [396:20:09<45:11:48,  8.61s/it]
 92%|█████████▏| 206094/225000 [396:20:19<46:50:22,  8.92s/it]
 92%|█████████▏| 206095/225000 [396:20:24<41:41:15,  7.94s/it]
 92%|█████████▏| 206096/225000 [396:20:31<39:55:41,  7.60s/it]
 92%|█████████▏| 206097/225000 [396:20:45<49:15:28,  9.38s/it]
 92%|█████████▏| 206098/225000 [396:20:51<45:03:37,  8.58s/it]
 92%|█████████▏| 206099/225000 [396:20:56<38:50:57,  7.40s/it]
 92%|█████████▏| 206100/225000 [396:21:01<35:35:50,  6.78s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.518', 'learning_rate': '9.569e-07', 'epoch': '0.916', 'num_input_tokens_seen': 668136632, 'train_runtime': '1.427e+06', 'train_tokens_per_second': '468.3'}

 92%|█████████▏| 206100/225000 [396:21:01<35:35:50,  6.78s/it]
 92%|█████████▏| 206101/225000 [396:21:14<44:41:42,  8.51s/it]
 92%|█████████▏| 206102/225000 [396:21:18<38:18:25,  7.30s/it]
 92%|█████████▏| 206103/225000 [396:21:27<39:40:54,  7.56s/it]
 92%|█████████▏| 206104/225000 [396:21:38<45:24:12,  8.65s/it]
 92%|█████████▏| 206105/225000 [396:21:46<45:16:10,  8.63s/it]
 92%|█████████▏| 206106/225000 [396:21:55<45:41:20,  8.71s/it]
 92%|█████████▏| 206107/225000 [396:22:05<47:13:07,  9.00s/it]
 92%|█████████▏| 206108/225000 [396:22:14<48:02:58,  9.16s/it]
 92%|█████████▏| 206109/225000 [396:22:21<44:32:56,  8.49s/it]
 92%|█████████▏| 206110/225000 [396:22:31<46:06:10,  8.79s/it]
 92%|█████████▏| 206111/225000 [396:22:42<50:09:11,  9.56s/it]
 92%|█████████▏| 206112/225000 [396:22:50<47:58:07,  9.14s/it]
 92%|█████████▏| 206113/225000 [396:22:58<45:40:55,  8.71s/it]
 92%|█████████▏| 206114/225000 [396:23:05<42:28:09,  8.10s/it]
 92%|█████████▏| 206115/225000 [396:23:12<41:40:21,  7.94s/it]
 92%|█████████▏| 206116/225000 [396:23:18<38:45:16,  7.39s/it]
 92%|█████████▏| 206117/225000 [396:23:33<50:17:41,  9.59s/it]
 92%|█████████▏| 206118/225000 [396:23:40<45:25:30,  8.66s/it]
 92%|█████████▏| 206119/225000 [396:23:47<43:26:24,  8.28s/it]
 92%|█████████▏| 206120/225000 [396:23:59<48:34:34,  9.26s/it]
 92%|█████████▏| 206121/225000 [396:24:08<48:03:44,  9.16s/it]
 92%|█████████▏| 206122/225000 [396:24:18<49:36:43,  9.46s/it]
 92%|█████████▏| 206123/225000 [396:24:24<44:16:26,  8.44s/it]
 92%|█████████▏| 206124/225000 [396:24:33<44:48:04,  8.54s/it]
 92%|█████████▏| 206125/225000 [396:24:41<44:07:23,  8.42s/it]
 92%|█████████▏| 206126/225000 [396:24:48<41:58:53,  8.01s/it]
 92%|█████████▏| 206127/225000 [396:24:52<36:23:14,  6.94s/it]
 92%|█████████▏| 206128/225000 [396:24:59<36:11:01,  6.90s/it]
 92%|█████████▏| 206129/225000 [396:25:14<48:18:23,  9.22s/it]
 92%|█████████▏| 206130/225000 [396:25:22<47:19:11,  9.03s/it]
 92%|█████████▏| 206131/225000 [396:25:33<49:38:28,  9.47s/it]
 92%|█████████▏| 206132/225000 [396:25:38<42:21:18,  8.08s/it]
 92%|█████████▏| 206133/225000 [396:25:49<47:48:03,  9.12s/it]
 92%|█████████▏| 206134/225000 [396:25:56<44:28:07,  8.49s/it]
 92%|█████████▏| 206135/225000 [396:26:06<46:20:59,  8.84s/it]
 92%|█████████▏| 206136/225000 [396:26:11<41:08:05,  7.85s/it]
 92%|█████████▏| 206137/225000 [396:26:20<41:55:42,  8.00s/it]
 92%|█████████▏| 206138/225000 [396:26:26<39:23:46,  7.52s/it]
 92%|█████████▏| 206139/225000 [396:26:32<36:22:26,  6.94s/it]
 92%|█████████▏| 206140/225000 [396:26:42<41:59:23,  8.02s/it]
 92%|█████████▏| 206141/225000 [396:26:51<43:30:12,  8.30s/it]
 92%|█████████▏| 206142/225000 [396:26:59<43:19:02,  8.27s/it]
 92%|█████████▏| 206143/225000 [396:27:07<42:54:36,  8.19s/it]
 92%|█████████▏| 206144/225000 [396:27:15<42:00:11,  8.02s/it]
 92%|█████████▏| 206145/225000 [396:27:26<46:06:31,  8.80s/it]
 92%|█████████▏| 206146/225000 [396:27:30<39:48:18,  7.60s/it]
 92%|█████████▏| 206147/225000 [396:27:43<47:55:44,  9.15s/it]
 92%|█████████▏| 206148/225000 [396:27:53<48:27:55,  9.25s/it]
 92%|█████████▏| 206149/225000 [396:27:59<43:56:42,  8.39s/it]
 92%|█████████▏| 206150/225000 [396:28:05<40:39:28,  7.76s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.664', 'learning_rate': '9.519e-07', 'epoch': '0.9162', 'num_input_tokens_seen': 668364928, 'train_runtime': '1.427e+06', 'train_tokens_per_second': '468.3'}

 92%|█████████▏| 206150/225000 [396:28:05<40:39:28,  7.76s/it]
 92%|█████████▏| 206151/225000 [396:28:13<40:17:54,  7.70s/it]
 92%|█████████▏| 206152/225000 [396:28:23<43:24:27,  8.29s/it]
 92%|█████████▏| 206153/225000 [396:28:27<37:53:06,  7.24s/it]
 92%|█████████▏| 206154/225000 [396:28:38<42:31:41,  8.12s/it]
 92%|█████████▏| 206155/225000 [396:28:46<43:00:48,  8.22s/it]
 92%|█████████▏| 206156/225000 [396:28:53<40:56:15,  7.82s/it]
 92%|█████████▏| 206157/225000 [396:29:03<43:55:46,  8.39s/it]
 92%|█████████▏| 206158/225000 [396:29:14<49:09:33,  9.39s/it]
 92%|█████████▏| 206159/225000 [396:29:27<53:42:54, 10.26s/it]
 92%|█████████▏| 206160/225000 [396:29:36<52:24:23, 10.01s/it]
 92%|█████████▏| 206161/225000 [396:29:44<49:33:47,  9.47s/it]
 92%|█████████▏| 206162/225000 [396:29:57<54:09:48, 10.35s/it]
 92%|█████████▏| 206163/225000 [396:30:06<52:09:42,  9.97s/it]
 92%|█████████▏| 206164/225000 [396:30:12<46:50:27,  8.95s/it]
 92%|█████████▏| 206165/225000 [396:30:19<43:11:04,  8.25s/it]
 92%|█████████▏| 206166/225000 [396:30:29<45:21:42,  8.67s/it]
 92%|█████████▏| 206167/225000 [396:30:40<50:22:08,  9.63s/it]
 92%|█████████▏| 206168/225000 [396:30:49<48:30:03,  9.27s/it]
 92%|█████████▏| 206169/225000 [396:30:55<43:14:10,  8.27s/it]
 92%|█████████▏| 206170/225000 [396:31:01<40:01:24,  7.65s/it]
 92%|█████████▏| 206171/225000 [396:31:10<42:34:29,  8.14s/it]
 92%|█████████▏| 206172/225000 [396:31:21<46:29:29,  8.89s/it]
 92%|█████████▏| 206173/225000 [396:31:27<42:13:04,  8.07s/it]
 92%|█████████▏| 206174/225000 [396:31:38<46:02:07,  8.80s/it]
 92%|█████████▏| 206175/225000 [396:31:43<39:54:15,  7.63s/it]
 92%|█████████▏| 206176/225000 [396:31:49<37:52:51,  7.24s/it]
 92%|█████████▏| 206177/225000 [396:31:57<38:35:44,  7.38s/it]
 92%|█████████▏| 206178/225000 [396:32:07<43:10:59,  8.26s/it]
 92%|█████████▏| 206179/225000 [396:32:17<46:14:23,  8.84s/it]
 92%|█████████▏| 206180/225000 [396:32:28<50:08:53,  9.59s/it]
 92%|█████████▏| 206181/225000 [396:32:37<48:11:58,  9.22s/it]
 92%|█████████▏| 206182/225000 [396:32:43<42:52:04,  8.20s/it]
 92%|█████████▏| 206183/225000 [396:32:48<39:08:44,  7.49s/it]
 92%|█████████▏| 206184/225000 [396:33:02<48:18:53,  9.24s/it]
 92%|█████████▏| 206185/225000 [396:33:09<45:48:10,  8.76s/it]
 92%|█████████▏| 206186/225000 [396:33:17<44:34:21,  8.53s/it]
 92%|█████████▏| 206187/225000 [396:33:25<43:04:55,  8.24s/it]
 92%|█████████▏| 206188/225000 [396:33:32<40:27:21,  7.74s/it]
 92%|█████████▏| 206189/225000 [396:33:37<36:37:36,  7.01s/it]
 92%|█████████▏| 206190/225000 [396:33:46<40:02:39,  7.66s/it]
 92%|█████████▏| 206191/225000 [396:33:55<42:00:17,  8.04s/it]
 92%|█████████▏| 206192/225000 [396:34:04<43:27:55,  8.32s/it]
 92%|█████████▏| 206193/225000 [396:34:11<41:19:55,  7.91s/it]
 92%|█████████▏| 206194/225000 [396:34:22<46:06:38,  8.83s/it]
 92%|█████████▏| 206195/225000 [396:34:33<50:24:37,  9.65s/it]
 92%|█████████▏| 206196/225000 [396:34:42<49:22:39,  9.45s/it]
 92%|█████████▏| 206197/225000 [396:34:53<51:55:01,  9.94s/it]
 92%|█████████▏| 206198/225000 [396:35:02<49:58:23,  9.57s/it]
 92%|█████████▏| 206199/225000 [396:35:10<47:43:04,  9.14s/it]
 92%|█████████▏| 206200/225000 [396:35:19<46:48:10,  8.96s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.337', 'learning_rate': '9.469e-07', 'epoch': '0.9164', 'num_input_tokens_seen': 668597216, 'train_runtime': '1.428e+06', 'train_tokens_per_second': '468.3'}

 92%|█████████▏| 206200/225000 [396:35:19<46:48:10,  8.96s/it]
 92%|█████████▏| 206201/225000 [396:35:26<43:31:16,  8.33s/it]
 92%|█████████▏| 206202/225000 [396:35:36<46:31:05,  8.91s/it]
 92%|█████████▏| 206203/225000 [396:35:45<46:29:44,  8.90s/it]
 92%|█████████▏| 206204/225000 [396:35:56<49:51:29,  9.55s/it]
 92%|█████████▏| 206205/225000 [396:36:04<46:49:44,  8.97s/it]
 92%|█████████▏| 206206/225000 [396:36:14<49:51:31,  9.55s/it]
 92%|█████████▏| 206207/225000 [396:36:24<50:11:07,  9.61s/it]
 92%|█████████▏| 206208/225000 [396:36:33<48:39:01,  9.32s/it]
 92%|█████████▏| 206209/225000 [396:36:41<46:10:42,  8.85s/it]
 92%|█████████▏| 206210/225000 [396:36:51<47:58:47,  9.19s/it]
 92%|█████████▏| 206211/225000 [396:36:57<43:32:17,  8.34s/it]
 92%|█████████▏| 206212/225000 [396:37:08<48:24:07,  9.27s/it]
 92%|█████████▏| 206213/225000 [396:37:18<48:40:48,  9.33s/it]
 92%|█████████▏| 206214/225000 [396:37:24<44:03:18,  8.44s/it]
 92%|█████████▏| 206215/225000 [396:37:32<42:34:18,  8.16s/it]
 92%|█████████▏| 206216/225000 [396:37:43<46:44:12,  8.96s/it]
 92%|█████████▏| 206217/225000 [396:37:52<46:45:50,  8.96s/it]
 92%|█████████▏| 206218/225000 [396:37:59<43:41:27,  8.37s/it]
 92%|█████████▏| 206219/225000 [396:38:05<40:24:34,  7.75s/it]
 92%|█████████▏| 206220/225000 [396:38:11<38:00:42,  7.29s/it]
 92%|█████████▏| 206221/225000 [396:38:19<39:50:20,  7.64s/it]
 92%|█████████▏| 206222/225000 [396:38:34<50:19:19,  9.65s/it]
 92%|█████████▏| 206223/225000 [396:38:42<48:37:07,  9.32s/it]
 92%|█████████▏| 206224/225000 [396:38:53<50:46:38,  9.74s/it]
 92%|█████████▏| 206225/225000 [396:39:04<52:49:06, 10.13s/it]
 92%|█████████▏| 206226/225000 [396:39:08<43:44:38,  8.39s/it]
 92%|█████████▏| 206227/225000 [396:39:17<44:14:35,  8.48s/it]
 92%|█████████▏| 206228/225000 [396:39:25<42:49:25,  8.21s/it]
 92%|█████████▏| 206229/225000 [396:39:30<38:27:48,  7.38s/it]
 92%|█████████▏| 206230/225000 [396:39:40<41:41:14,  8.00s/it]
 92%|█████████▏| 206231/225000 [396:39:51<46:55:54,  9.00s/it]
 92%|█████████▏| 206232/225000 [396:39:57<43:05:15,  8.26s/it]
 92%|█████████▏| 206233/225000 [396:40:06<44:02:55,  8.45s/it]
 92%|█████████▏| 206234/225000 [396:40:16<46:13:09,  8.87s/it]
 92%|█████████▏| 206235/225000 [396:40:24<45:10:03,  8.67s/it]
 92%|█████████▏| 206236/225000 [396:40:31<42:05:00,  8.07s/it]
 92%|█████████▏| 206237/225000 [396:40:38<39:30:48,  7.58s/it]
 92%|█████████▏| 206238/225000 [396:40:54<52:48:50, 10.13s/it]
 92%|█████████▏| 206239/225000 [396:41:01<48:24:27,  9.29s/it]
 92%|█████████▏| 206240/225000 [396:41:14<54:05:05, 10.38s/it]
 92%|█████████▏| 206241/225000 [396:41:24<53:48:36, 10.33s/it]
 92%|█████████▏| 206242/225000 [396:41:31<47:43:53,  9.16s/it]
 92%|█████████▏| 206243/225000 [396:41:41<50:35:56,  9.71s/it]
 92%|█████████▏| 206244/225000 [396:41:49<46:45:32,  8.97s/it]
 92%|█████████▏| 206245/225000 [396:41:55<43:11:14,  8.29s/it]
 92%|█████████▏| 206246/225000 [396:42:03<41:47:38,  8.02s/it]
 92%|█████████▏| 206247/225000 [396:42:13<44:49:35,  8.61s/it]
 92%|█████████▏| 206248/225000 [396:42:26<51:20:06,  9.86s/it]
 92%|█████████▏| 206249/225000 [396:42:33<47:26:33,  9.11s/it]
 92%|█████████▏| 206250/225000 [396:42:41<45:35:58,  8.76s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.041', 'learning_rate': '9.419e-07', 'epoch': '0.9167', 'num_input_tokens_seen': 668832768, 'train_runtime': '1.428e+06', 'train_tokens_per_second': '468.3'}

 92%|█████████▏| 206250/225000 [396:42:41<45:35:58,  8.76s/it]
 92%|█████████▏| 206251/225000 [396:42:48<43:38:53,  8.38s/it]
 92%|█████████▏| 206252/225000 [396:42:57<43:44:12,  8.40s/it]
 92%|█████████▏| 206253/225000 [396:43:10<51:04:18,  9.81s/it]
 92%|█████████▏| 206254/225000 [396:43:15<43:38:08,  8.38s/it]
 92%|█████████▏| 206255/225000 [396:43:22<41:40:48,  8.00s/it]
 92%|█████████▏| 206256/225000 [396:43:29<39:22:30,  7.56s/it]
 92%|█████████▏| 206257/225000 [396:43:37<40:10:18,  7.72s/it]
 92%|█████████▏| 206258/225000 [396:43:48<46:15:27,  8.89s/it]
 92%|█████████▏| 206259/225000 [396:43:54<41:11:30,  7.91s/it]
 92%|█████████▏| 206260/225000 [396:44:02<41:26:29,  7.96s/it]
 92%|█████████▏| 206261/225000 [396:44:10<42:04:34,  8.08s/it]
 92%|█████████▏| 206262/225000 [396:44:16<38:38:57,  7.43s/it]
 92%|█████████▏| 206263/225000 [396:44:28<44:35:25,  8.57s/it]
 92%|█████████▏| 206264/225000 [396:44:37<46:31:41,  8.94s/it]
 92%|█████████▏| 206265/225000 [396:44:46<45:59:21,  8.84s/it]
 92%|█████████▏| 206266/225000 [396:44:53<43:16:55,  8.32s/it]
 92%|█████████▏| 206267/225000 [396:44:59<39:24:42,  7.57s/it]
 92%|█████████▏| 206268/225000 [396:45:10<44:58:05,  8.64s/it]
 92%|█████████▏| 206269/225000 [396:45:16<41:27:41,  7.97s/it]
 92%|█████████▏| 206270/225000 [396:45:27<46:03:27,  8.85s/it]
 92%|█████████▏| 206271/225000 [396:45:32<40:05:16,  7.71s/it]
 92%|█████████▏| 206272/225000 [396:45:39<39:05:20,  7.51s/it]
 92%|█████████▏| 206273/225000 [396:45:52<47:11:53,  9.07s/it]
 92%|█████████▏| 206274/225000 [396:45:57<41:14:28,  7.93s/it]
 92%|█████████▏| 206275/225000 [396:46:08<45:30:55,  8.75s/it]
 92%|█████████▏| 206276/225000 [396:46:18<47:57:00,  9.22s/it]
 92%|█████████▏| 206277/225000 [396:46:28<48:06:13,  9.25s/it]
 92%|█████████▏| 206278/225000 [396:46:38<50:16:17,  9.67s/it]
 92%|█████████▏| 206279/225000 [396:46:47<49:15:11,  9.47s/it]
 92%|█████████▏| 206280/225000 [396:46:53<43:54:28,  8.44s/it]
 92%|█████████▏| 206281/225000 [396:47:01<43:13:14,  8.31s/it]
 92%|█████████▏| 206282/225000 [396:47:12<46:46:24,  9.00s/it]
 92%|█████████▏| 206283/225000 [396:47:20<45:29:27,  8.75s/it]
 92%|█████████▏| 206284/225000 [396:47:26<41:38:09,  8.01s/it]
 92%|█████████▏| 206285/225000 [396:47:34<40:12:30,  7.73s/it]
 92%|█████████▏| 206286/225000 [396:47:44<43:47:08,  8.42s/it]
 92%|█████████▏| 206287/225000 [396:47:57<51:11:01,  9.85s/it]
 92%|█████████▏| 206288/225000 [396:48:05<48:05:57,  9.25s/it]
 92%|█████████▏| 206289/225000 [396:48:20<56:54:38, 10.95s/it]
 92%|█████████▏| 206290/225000 [396:48:25<48:04:09,  9.25s/it]
 92%|█████████▏| 206291/225000 [396:48:33<46:31:53,  8.95s/it]
 92%|█████████▏| 206292/225000 [396:48:44<49:41:03,  9.56s/it]
 92%|█████████▏| 206293/225000 [396:48:52<47:53:36,  9.22s/it]
 92%|█████████▏| 206294/225000 [396:48:59<43:26:03,  8.36s/it]
 92%|█████████▏| 206295/225000 [396:49:10<47:41:33,  9.18s/it]
 92%|█████████▏| 206296/225000 [396:49:15<41:59:35,  8.08s/it]
 92%|█████████▏| 206297/225000 [396:49:28<48:23:33,  9.31s/it]
 92%|█████████▏| 206298/225000 [396:49:32<40:16:09,  7.75s/it]
 92%|█████████▏| 206299/225000 [396:49:41<43:11:11,  8.31s/it]
 92%|█████████▏| 206300/225000 [396:49:51<45:14:40,  8.71s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.627', 'learning_rate': '9.371e-07', 'epoch': '0.9169', 'num_input_tokens_seen': 669063960, 'train_runtime': '1.429e+06', 'train_tokens_per_second': '468.3'}

 92%|█████████▏| 206300/225000 [396:49:51<45:14:40,  8.71s/it]
 92%|█████████▏| 206301/225000 [396:49:58<42:03:52,  8.10s/it]
 92%|█████████▏| 206302/225000 [396:50:05<40:40:49,  7.83s/it]
 92%|█████████▏| 206303/225000 [396:50:17<47:14:13,  9.10s/it]
 92%|█████████▏| 206304/225000 [396:50:28<49:40:14,  9.56s/it]
 92%|█████████▏| 206305/225000 [396:50:35<45:40:11,  8.79s/it]
 92%|█████████▏| 206306/225000 [396:50:44<46:18:57,  8.92s/it]
 92%|█████████▏| 206307/225000 [396:50:53<46:20:23,  8.92s/it]
 92%|█████████▏| 206308/225000 [396:50:59<42:09:36,  8.12s/it]
 92%|█████████▏| 206309/225000 [396:51:09<44:58:13,  8.66s/it]
 92%|█████████▏| 206310/225000 [396:51:15<41:28:03,  7.99s/it]
 92%|█████████▏| 206311/225000 [396:51:24<42:18:42,  8.15s/it]
 92%|█████████▏| 206312/225000 [396:51:33<43:44:34,  8.43s/it]
 92%|█████████▏| 206313/225000 [396:51:42<44:46:03,  8.62s/it]
 92%|█████████▏| 206314/225000 [396:51:49<42:37:51,  8.21s/it]
 92%|█████████▏| 206315/225000 [396:51:56<40:05:16,  7.72s/it]
 92%|█████████▏| 206316/225000 [396:52:05<42:45:30,  8.24s/it]
 92%|█████████▏| 206317/225000 [396:52:13<41:24:00,  7.98s/it]
 92%|█████████▏| 206318/225000 [396:52:20<40:57:13,  7.89s/it]
 92%|█████████▏| 206319/225000 [396:52:26<37:59:39,  7.32s/it]
 92%|█████████▏| 206320/225000 [396:52:37<43:40:19,  8.42s/it]
 92%|█████████▏| 206321/225000 [396:52:47<45:56:27,  8.85s/it]
 92%|█████████▏| 206322/225000 [396:52:54<42:06:52,  8.12s/it]
 92%|█████████▏| 206323/225000 [396:53:07<50:12:18,  9.68s/it]
 92%|█████████▏| 206324/225000 [396:53:13<44:11:03,  8.52s/it]
 92%|█████████▏| 206325/225000 [396:53:19<40:34:50,  7.82s/it]
 92%|█████████▏| 206326/225000 [396:53:32<49:36:18,  9.56s/it]
 92%|█████████▏| 206327/225000 [396:53:40<45:43:40,  8.82s/it]
 92%|█████████▏| 206328/225000 [396:53:50<47:54:46,  9.24s/it]
 92%|█████████▏| 206329/225000 [396:53:58<46:39:42,  9.00s/it]
 92%|█████████▏| 206330/225000 [396:54:08<48:21:33,  9.32s/it]
 92%|█████████▏| 206331/225000 [396:54:16<46:30:04,  8.97s/it]
 92%|█████████▏| 206332/225000 [396:54:21<39:28:27,  7.61s/it]
 92%|█████████▏| 206333/225000 [396:54:27<37:48:12,  7.29s/it]
 92%|█████████▏| 206334/225000 [396:54:34<37:25:06,  7.22s/it]
 92%|█████████▏| 206335/225000 [396:54:42<37:56:19,  7.32s/it]
 92%|█████████▏| 206336/225000 [396:54:53<43:45:19,  8.44s/it]
 92%|█████████▏| 206337/225000 [396:55:03<46:43:41,  9.01s/it]
 92%|█████████▏| 206338/225000 [396:55:17<54:10:11, 10.45s/it]
 92%|█████████▏| 206339/225000 [396:55:27<53:05:40, 10.24s/it]
 92%|█████████▏| 206340/225000 [396:55:34<47:23:12,  9.14s/it]
 92%|█████████▏| 206341/225000 [396:55:42<45:42:59,  8.82s/it]
 92%|█████████▏| 206342/225000 [396:55:54<51:49:10, 10.00s/it]
 92%|█████████▏| 206343/225000 [396:56:04<51:40:59,  9.97s/it]
 92%|█████████▏| 206344/225000 [396:56:16<54:35:23, 10.53s/it]
 92%|█████████▏| 206345/225000 [396:56:24<50:22:17,  9.72s/it]
 92%|█████████▏| 206346/225000 [396:56:30<44:07:00,  8.51s/it]
 92%|█████████▏| 206347/225000 [396:56:41<49:09:23,  9.49s/it]
 92%|█████████▏| 206348/225000 [396:56:52<51:28:11,  9.93s/it]
 92%|█████████▏| 206349/225000 [396:57:04<53:28:49, 10.32s/it]
 92%|█████████▏| 206350/225000 [396:57:12<50:22:15,  9.72s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.199', 'learning_rate': '9.321e-07', 'epoch': '0.9171', 'num_input_tokens_seen': 669301784, 'train_runtime': '1.429e+06', 'train_tokens_per_second': '468.4'}

 92%|█████████▏| 206350/225000 [396:57:12<50:22:15,  9.72s/it]
 92%|█████████▏| 206351/225000 [396:57:20<47:58:53,  9.26s/it]
 92%|█████████▏| 206352/225000 [396:57:28<45:38:53,  8.81s/it]
 92%|█████████▏| 206353/225000 [396:57:37<46:49:49,  9.04s/it]
 92%|█████████▏| 206354/225000 [396:57:46<45:17:55,  8.75s/it]
 92%|█████████▏| 206355/225000 [396:57:51<39:26:45,  7.62s/it]
 92%|█████████▏| 206356/225000 [396:58:01<43:49:56,  8.46s/it]
 92%|█████████▏| 206357/225000 [396:58:11<47:00:05,  9.08s/it]
 92%|█████████▏| 206358/225000 [396:58:20<46:32:53,  8.99s/it]
 92%|█████████▏| 206359/225000 [396:58:27<42:34:50,  8.22s/it]
 92%|█████████▏| 206360/225000 [396:58:33<39:03:01,  7.54s/it]
 92%|█████████▏| 206361/225000 [396:58:40<38:24:55,  7.42s/it]
 92%|█████████▏| 206362/225000 [396:58:54<49:14:02,  9.51s/it]
 92%|█████████▏| 206363/225000 [396:59:03<48:44:20,  9.41s/it]
 92%|█████████▏| 206364/225000 [396:59:12<47:54:36,  9.25s/it]
 92%|█████████▏| 206365/225000 [396:59:19<44:34:51,  8.61s/it]
 92%|█████████▏| 206366/225000 [396:59:26<42:10:55,  8.15s/it]
 92%|█████████▏| 206367/225000 [396:59:34<40:46:08,  7.88s/it]
 92%|█████████▏| 206368/225000 [396:59:40<39:06:03,  7.55s/it]
 92%|█████████▏| 206369/225000 [396:59:54<49:05:30,  9.49s/it]
 92%|█████████▏| 206370/225000 [397:00:00<42:26:57,  8.20s/it]
 92%|█████████▏| 206371/225000 [397:00:10<45:50:10,  8.86s/it]
 92%|█████████▏| 206372/225000 [397:00:20<47:48:48,  9.24s/it]
 92%|█████████▏| 206373/225000 [397:00:28<45:01:54,  8.70s/it]
 92%|█████████▏| 206374/225000 [397:00:36<44:47:00,  8.66s/it]
 92%|█████████▏| 206375/225000 [397:00:46<46:09:56,  8.92s/it]
 92%|█████████▏| 206376/225000 [397:00:54<44:41:57,  8.64s/it]
 92%|█████████▏| 206377/225000 [397:01:01<43:14:36,  8.36s/it]
 92%|█████████▏| 206378/225000 [397:01:15<52:00:27, 10.05s/it]
 92%|█████████▏| 206379/225000 [397:01:19<42:39:00,  8.25s/it]
 92%|█████████▏| 206380/225000 [397:01:25<38:01:24,  7.35s/it]
 92%|█████████▏| 206381/225000 [397:01:36<44:43:33,  8.65s/it]
 92%|█████████▏| 206382/225000 [397:01:43<41:36:42,  8.05s/it]
 92%|█████████▏| 206383/225000 [397:01:50<39:41:25,  7.67s/it]
 92%|█████████▏| 206384/225000 [397:02:01<44:35:34,  8.62s/it]
 92%|█████████▏| 206385/225000 [397:02:07<40:28:07,  7.83s/it]
 92%|█████████▏| 206386/225000 [397:02:18<45:58:03,  8.89s/it]
 92%|█████████▏| 206387/225000 [397:02:30<51:27:39,  9.95s/it]
 92%|█████████▏| 206388/225000 [397:02:39<48:31:39,  9.39s/it]
 92%|█████████▏| 206389/225000 [397:02:49<50:19:01,  9.73s/it]
 92%|█████████▏| 206390/225000 [397:03:02<55:07:59, 10.67s/it]
 92%|█████████▏| 206391/225000 [397:03:10<51:01:25,  9.87s/it]
 92%|█████████▏| 206392/225000 [397:03:18<48:15:35,  9.34s/it]
 92%|█████████▏| 206393/225000 [397:03:25<43:55:49,  8.50s/it]
 92%|█████████▏| 206394/225000 [397:03:35<46:54:58,  9.08s/it]
 92%|█████████▏| 206395/225000 [397:03:44<46:36:46,  9.02s/it]
 92%|█████████▏| 206396/225000 [397:03:56<51:19:04,  9.93s/it]
 92%|█████████▏| 206397/225000 [397:04:02<44:47:01,  8.67s/it]
 92%|█████████▏| 206398/225000 [397:04:11<45:24:32,  8.79s/it]
 92%|█████████▏| 206399/225000 [397:04:25<54:26:03, 10.54s/it]
 92%|█████████▏| 206400/225000 [397:04:31<46:55:37,  9.08s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.909', 'learning_rate': '9.272e-07', 'epoch': '0.9173', 'num_input_tokens_seen': 669533600, 'train_runtime': '1.429e+06', 'train_tokens_per_second': '468.4'}

 92%|█████████▏| 206400/225000 [397:04:31<46:55:37,  9.08s/it]
 92%|█████████▏| 206401/225000 [397:04:41<47:58:29,  9.29s/it]
 92%|█████████▏| 206402/225000 [397:04:52<51:33:52,  9.98s/it]
 92%|█████████▏| 206403/225000 [397:04:57<43:08:07,  8.35s/it]
 92%|█████████▏| 206404/225000 [397:05:02<38:32:06,  7.46s/it]
 92%|█████████▏| 206405/225000 [397:05:09<37:24:43,  7.24s/it]
 92%|█████████▏| 206406/225000 [397:05:23<48:03:29,  9.30s/it]
 92%|█████████▏| 206407/225000 [397:05:32<47:39:26,  9.23s/it]
 92%|█████████▏| 206408/225000 [397:05:40<45:16:41,  8.77s/it]
 92%|█████████▏| 206409/225000 [397:05:48<44:34:10,  8.63s/it]
 92%|█████████▏| 206410/225000 [397:05:54<40:08:05,  7.77s/it]
 92%|█████████▏| 206411/225000 [397:06:04<44:10:59,  8.56s/it]
 92%|█████████▏| 206412/225000 [397:06:11<41:02:00,  7.95s/it]
 92%|█████████▏| 206413/225000 [397:06:24<49:28:10,  9.58s/it]
 92%|█████████▏| 206414/225000 [397:06:30<43:53:09,  8.50s/it]
 92%|█████████▏| 206415/225000 [397:06:37<41:53:41,  8.12s/it]
 92%|█████████▏| 206416/225000 [397:06:44<39:04:32,  7.57s/it]
 92%|█████████▏| 206417/225000 [397:06:54<42:59:56,  8.33s/it]
 92%|█████████▏| 206418/225000 [397:07:05<46:42:31,  9.05s/it]
 92%|█████████▏| 206419/225000 [397:07:18<53:42:03, 10.40s/it]
 92%|█████████▏| 206420/225000 [397:07:27<50:54:06,  9.86s/it]
 92%|█████████▏| 206421/225000 [397:07:33<45:21:34,  8.79s/it]
 92%|█████████▏| 206422/225000 [397:07:41<44:45:38,  8.67s/it]
 92%|█████████▏| 206423/225000 [397:07:52<46:53:35,  9.09s/it]
 92%|█████████▏| 206424/225000 [397:07:59<43:55:12,  8.51s/it]
 92%|█████████▏| 206425/225000 [397:08:06<42:22:08,  8.21s/it]
 92%|█████████▏| 206426/225000 [397:08:12<39:09:15,  7.59s/it]
 92%|█████████▏| 206427/225000 [397:08:20<38:41:13,  7.50s/it]
 92%|█████████▏| 206428/225000 [397:08:30<43:47:12,  8.49s/it]
 92%|█████████▏| 206429/225000 [397:08:35<38:23:20,  7.44s/it]
 92%|█████████▏| 206430/225000 [397:08:42<37:35:13,  7.29s/it]
 92%|█████████▏| 206431/225000 [397:08:52<40:30:29,  7.85s/it]
 92%|█████████▏| 206432/225000 [397:09:01<43:02:44,  8.35s/it]
 92%|█████████▏| 206433/225000 [397:09:06<38:33:19,  7.48s/it]
 92%|█████████▏| 206434/225000 [397:09:12<35:31:07,  6.89s/it]
 92%|█████████▏| 206435/225000 [397:09:23<42:19:39,  8.21s/it]
 92%|█████████▏| 206436/225000 [397:09:32<42:27:54,  8.23s/it]
 92%|█████████▏| 206437/225000 [397:09:41<44:37:53,  8.66s/it]
 92%|█████████▏| 206438/225000 [397:09:49<43:22:51,  8.41s/it]
 92%|█████████▏| 206439/225000 [397:09:55<38:55:08,  7.55s/it]
 92%|█████████▏| 206440/225000 [397:10:08<47:21:45,  9.19s/it]
 92%|█████████▏| 206441/225000 [397:10:13<40:55:00,  7.94s/it]
 92%|█████████▏| 206442/225000 [397:10:19<37:51:13,  7.34s/it]
 92%|█████████▏| 206443/225000 [397:10:31<45:59:34,  8.92s/it]
 92%|█████████▏| 206444/225000 [397:10:42<48:50:32,  9.48s/it]
 92%|█████████▏| 206445/225000 [397:10:46<40:41:12,  7.89s/it]
 92%|█████████▏| 206446/225000 [397:10:56<43:59:26,  8.54s/it]
 92%|█████████▏| 206447/225000 [397:11:01<37:49:22,  7.34s/it]
 92%|█████████▏| 206448/225000 [397:11:09<39:45:50,  7.72s/it]
 92%|█████████▏| 206449/225000 [397:11:16<37:58:32,  7.37s/it]
 92%|█████████▏| 206450/225000 [397:11:27<43:13:39,  8.39s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.233', 'learning_rate': '9.223e-07', 'epoch': '0.9176', 'num_input_tokens_seen': 669761128, 'train_runtime': '1.43e+06', 'train_tokens_per_second': '468.4'}

 92%|█████████▏| 206450/225000 [397:11:27<43:13:39,  8.39s/it]
 92%|█████████▏| 206451/225000 [397:11:34<42:12:04,  8.19s/it]
 92%|█████████▏| 206452/225000 [397:11:40<38:10:49,  7.41s/it]
 92%|█████████▏| 206453/225000 [397:11:48<38:51:13,  7.54s/it]
 92%|█████████▏| 206454/225000 [397:12:00<45:23:29,  8.81s/it]
 92%|█████████▏| 206455/225000 [397:12:09<46:07:33,  8.95s/it]
 92%|█████████▏| 206456/225000 [397:12:18<46:57:19,  9.12s/it]
 92%|█████████▏| 206457/225000 [397:12:28<47:56:18,  9.31s/it]
 92%|█████████▏| 206458/225000 [397:12:33<41:35:54,  8.08s/it]
 92%|█████████▏| 206459/225000 [397:12:40<39:20:38,  7.64s/it]
 92%|█████████▏| 206460/225000 [397:12:52<45:51:02,  8.90s/it]
 92%|█████████▏| 206461/225000 [397:12:57<39:30:30,  7.67s/it]
 92%|█████████▏| 206462/225000 [397:13:04<38:41:17,  7.51s/it]
 92%|█████████▏| 206463/225000 [397:13:11<37:55:42,  7.37s/it]
 92%|█████████▏| 206464/225000 [397:13:18<37:20:35,  7.25s/it]
 92%|█████████▏| 206465/225000 [397:13:29<44:16:27,  8.60s/it]
 92%|█████████▏| 206466/225000 [397:13:36<41:33:40,  8.07s/it]
 92%|█████████▏| 206467/225000 [397:13:49<48:31:23,  9.43s/it]
 92%|█████████▏| 206468/225000 [397:13:57<46:03:18,  8.95s/it]
 92%|█████████▏| 206469/225000 [397:14:01<38:44:15,  7.53s/it]
 92%|█████████▏| 206470/225000 [397:14:12<43:41:01,  8.49s/it]
 92%|█████████▏| 206471/225000 [397:14:22<45:49:29,  8.90s/it]
 92%|█████████▏| 206472/225000 [397:14:30<44:45:20,  8.70s/it]
 92%|█████████▏| 206473/225000 [397:14:37<43:09:06,  8.38s/it]
 92%|█████████▏| 206474/225000 [397:14:45<42:29:58,  8.26s/it]
 92%|█████████▏| 206475/225000 [397:14:58<49:07:27,  9.55s/it]
 92%|█████████▏| 206476/225000 [397:15:04<44:19:01,  8.61s/it]
 92%|█████████▏| 206477/225000 [397:15:11<41:44:34,  8.11s/it]
 92%|█████████▏| 206478/225000 [397:15:24<48:37:50,  9.45s/it]
 92%|█████████▏| 206479/225000 [397:15:31<44:42:28,  8.69s/it]
 92%|█████████▏| 206480/225000 [397:15:43<50:16:01,  9.77s/it]
 92%|█████████▏| 206481/225000 [397:15:49<44:20:09,  8.62s/it]
 92%|█████████▏| 206482/225000 [397:15:57<43:57:45,  8.55s/it]
 92%|█████████▏| 206483/225000 [397:16:05<42:39:45,  8.29s/it]
 92%|█████████▏| 206484/225000 [397:16:12<41:07:23,  8.00s/it]
 92%|█████████▏| 206485/225000 [397:16:22<44:05:50,  8.57s/it]
 92%|█████████▏| 206486/225000 [397:16:34<48:44:43,  9.48s/it]
 92%|█████████▏| 206487/225000 [397:16:43<48:45:31,  9.48s/it]
 92%|█████████▏| 206488/225000 [397:16:49<42:47:45,  8.32s/it]
 92%|█████████▏| 206489/225000 [397:16:58<44:14:59,  8.61s/it]
 92%|█████████▏| 206490/225000 [397:17:05<42:03:44,  8.18s/it]
 92%|█████████▏| 206491/225000 [397:17:13<40:54:07,  7.96s/it]
 92%|█████████▏| 206492/225000 [397:17:25<47:13:51,  9.19s/it]
 92%|█████████▏| 206493/225000 [397:17:30<40:06:03,  7.80s/it]
 92%|█████████▏| 206494/225000 [397:17:42<46:59:06,  9.14s/it]
 92%|█████████▏| 206495/225000 [397:17:48<41:42:57,  8.12s/it]
 92%|█████████▏| 206496/225000 [397:17:53<37:21:54,  7.27s/it]
 92%|█████████▏| 206497/225000 [397:18:00<36:41:51,  7.14s/it]
 92%|█████████▏| 206498/225000 [397:18:15<49:10:31,  9.57s/it]
 92%|█████████▏| 206499/225000 [397:18:24<47:57:54,  9.33s/it]
 92%|█████████▏| 206500/225000 [397:18:32<47:02:25,  9.15s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.198', 'learning_rate': '9.174e-07', 'epoch': '0.9178', 'num_input_tokens_seen': 669990736, 'train_runtime': '1.43e+06', 'train_tokens_per_second': '468.4'}

 92%|█████████▏| 206500/225000 [397:18:32<47:02:25,  9.15s/it]
 92%|█████████▏| 206501/225000 [397:18:39<43:25:12,  8.45s/it]
 92%|█████████▏| 206502/225000 [397:18:46<40:18:10,  7.84s/it]
 92%|█████████▏| 206503/225000 [397:19:01<52:00:16, 10.12s/it]
 92%|█████████▏| 206504/225000 [397:19:08<46:18:53,  9.01s/it]
 92%|█████████▏| 206505/225000 [397:19:15<44:13:01,  8.61s/it]
 92%|█████████▏| 206506/225000 [397:19:21<39:24:39,  7.67s/it]
 92%|█████████▏| 206507/225000 [397:19:27<37:38:34,  7.33s/it]
 92%|█████████▏| 206508/225000 [397:19:41<47:32:06,  9.25s/it]
 92%|█████████▏| 206509/225000 [397:19:47<42:31:29,  8.28s/it]
 92%|█████████▏| 206510/225000 [397:19:53<39:32:41,  7.70s/it]
 92%|█████████▏| 206511/225000 [397:20:01<39:27:22,  7.68s/it]
 92%|█████████▏| 206512/225000 [397:20:09<40:19:27,  7.85s/it]
 92%|█████████▏| 206513/225000 [397:20:17<39:41:47,  7.73s/it]
 92%|█████████▏| 206514/225000 [397:20:23<37:59:43,  7.40s/it]
 92%|█████████▏| 206515/225000 [397:20:31<38:18:30,  7.46s/it]
 92%|█████████▏| 206516/225000 [397:20:45<48:13:09,  9.39s/it]
 92%|█████████▏| 206517/225000 [397:20:54<48:31:21,  9.45s/it]
 92%|█████████▏| 206518/225000 [397:21:01<44:41:00,  8.70s/it]
 92%|█████████▏| 206519/225000 [397:21:08<42:10:16,  8.21s/it]
 92%|█████████▏| 206520/225000 [397:21:19<45:13:47,  8.81s/it]
 92%|█████████▏| 206521/225000 [397:21:31<51:08:04,  9.96s/it]
 92%|█████████▏| 206522/225000 [397:21:42<52:13:35, 10.18s/it]
 92%|█████████▏| 206523/225000 [397:21:49<47:38:03,  9.28s/it]
 92%|█████████▏| 206524/225000 [397:21:55<42:40:14,  8.31s/it]
 92%|█████████▏| 206525/225000 [397:22:09<51:31:19, 10.04s/it]
 92%|█████████▏| 206526/225000 [397:22:19<51:30:58, 10.04s/it]
 92%|█████████▏| 206527/225000 [397:22:30<52:54:31, 10.31s/it]
 92%|█████████▏| 206528/225000 [397:22:35<44:54:06,  8.75s/it]
 92%|█████████▏| 206529/225000 [397:22:45<46:58:46,  9.16s/it]
 92%|█████████▏| 206530/225000 [397:22:54<46:08:56,  8.99s/it]
 92%|█████████▏| 206531/225000 [397:23:01<43:44:53,  8.53s/it]
 92%|█████████▏| 206532/225000 [397:23:07<39:03:07,  7.61s/it]
 92%|█████████▏| 206533/225000 [397:23:15<40:17:11,  7.85s/it]
 92%|█████████▏| 206534/225000 [397:23:22<38:36:39,  7.53s/it]
 92%|█████████▏| 206535/225000 [397:23:28<36:35:48,  7.14s/it]
 92%|█████████▏| 206536/225000 [397:23:33<33:30:49,  6.53s/it]
 92%|█████████▏| 206537/225000 [397:23:45<41:23:25,  8.07s/it]
 92%|█████████▏| 206538/225000 [397:23:54<43:17:26,  8.44s/it]
 92%|█████████▏| 206539/225000 [397:23:58<35:56:03,  7.01s/it]
 92%|█████████▏| 206540/225000 [397:24:14<49:01:17,  9.56s/it]
 92%|█████████▏| 206541/225000 [397:24:21<45:27:37,  8.87s/it]
 92%|█████████▏| 206542/225000 [397:24:27<41:41:32,  8.13s/it]
 92%|█████████▏| 206543/225000 [397:24:37<44:06:24,  8.60s/it]
 92%|█████████▏| 206544/225000 [397:24:45<43:53:06,  8.56s/it]
 92%|█████████▏| 206545/225000 [397:24:53<43:03:19,  8.40s/it]
 92%|█████████▏| 206546/225000 [397:24:58<37:46:40,  7.37s/it]
 92%|█████████▏| 206547/225000 [397:25:05<36:32:26,  7.13s/it]
 92%|█████████▏| 206548/225000 [397:25:14<39:47:49,  7.76s/it]
 92%|█████████▏| 206549/225000 [397:25:21<37:53:16,  7.39s/it]
 92%|█████████▏| 206550/225000 [397:25:34<46:21:09,  9.04s/it]
                                                              
{'loss': '1.341', 'grad_norm': '1.302', 'learning_rate': '9.125e-07', 'epoch': '0.918', 'num_input_tokens_seen': 670218008, 'train_runtime': '1.431e+06', 'train_tokens_per_second': '468.4'}

 92%|█████████▏| 206550/225000 [397:25:34<46:21:09,  9.04s/it]
 92%|█████████▏| 206551/225000 [397:25:42<44:32:13,  8.69s/it]
 92%|█████████▏| 206552/225000 [397:25:49<42:51:37,  8.36s/it]
 92%|█████████▏| 206553/225000 [397:25:59<45:20:02,  8.85s/it]
 92%|█████████▏| 206554/225000 [397:26:10<48:27:18,  9.46s/it]
 92%|█████████▏| 206555/225000 [397:26:17<45:23:57,  8.86s/it]
 92%|█████████▏| 206556/225000 [397:26:29<49:42:10,  9.70s/it]
 92%|█████████▏| 206557/225000 [397:26:35<43:40:19,  8.52s/it]
 92%|█████████▏| 206558/225000 [397:26:46<48:15:36,  9.42s/it]
 92%|█████████▏| 206559/225000 [397:26:52<41:52:30,  8.17s/it]
 92%|█████████▏| 206560/225000 [397:27:03<46:12:24,  9.02s/it]
 92%|█████████▏| 206561/225000 [397:27:08<40:51:50,  7.98s/it]
 92%|█████████▏| 206562/225000 [397:27:18<43:27:13,  8.48s/it]
 92%|█████████▏| 206563/225000 [397:27:27<43:58:36,  8.59s/it]
 92%|█████████▏| 206564/225000 [397:27:36<44:53:51,  8.77s/it]
 92%|█████████▏| 206565/225000 [397:27:41<39:35:21,  7.73s/it]
 92%|█████████▏| 206566/225000 [397:27:48<37:55:54,  7.41s/it]
 92%|█████████▏| 206567/225000 [397:27:56<39:15:06,  7.67s/it]
 92%|█████████▏| 206568/225000 [397:28:06<42:39:35,  8.33s/it]
 92%|█████████▏| 206569/225000 [397:28:13<41:03:50,  8.02s/it]
 92%|█████████▏| 206570/225000 [397:28:20<38:37:18,  7.54s/it]
 92%|█████████▏| 206571/225000 [397:28:33<46:58:01,  9.17s/it]
 92%|█████████▏| 206572/225000 [397:28:41<44:58:38,  8.79s/it]
 92%|█████████▏| 206573/225000 [397:28:45<38:44:56,  7.57s/it]
 92%|█████████▏| 206574/225000 [397:28:55<41:42:26,  8.15s/it]
 92%|█████████▏| 206575/225000 [397:28:59<36:03:49,  7.05s/it]
 92%|█████████▏| 206576/225000 [397:29:14<47:42:42,  9.32s/it]
 92%|█████████▏| 206577/225000 [397:29:23<47:11:44,  9.22s/it]
 92%|█████████▏| 206578/225000 [397:29:29<42:50:46,  8.37s/it]
 92%|█████████▏| 206579/225000 [397:29:37<41:16:37,  8.07s/it]
 92%|█████████▏| 206580/225000 [397:29:44<39:59:49,  7.82s/it]
 92%|█████████▏| 206581/225000 [397:29:51<38:08:52,  7.46s/it]
 92%|█████████▏| 206582/225000 [397:29:57<35:53:12,  7.01s/it]
 92%|█████████▏| 206583/225000 [397:30:10<45:31:54,  8.90s/it]
 92%|█████████▏| 206584/225000 [397:30:17<42:24:13,  8.29s/it]
 92%|█████████▏| 206585/225000 [397:30:28<47:03:34,  9.20s/it]
 92%|█████████▏| 206586/225000 [397:30:36<44:40:19,  8.73s/it]
 92%|█████████▏| 206587/225000 [397:30:42<41:02:54,  8.03s/it]
 92%|█████████▏| 206588/225000 [397:30:52<44:16:29,  8.66s/it]
 92%|█████████▏| 206589/225000 [397:31:01<43:57:44,  8.60s/it]
 92%|█████████▏| 206590/225000 [397:31:09<43:23:46,  8.49s/it]
 92%|█████████▏| 206591/225000 [397:31:24<53:28:18, 10.46s/it]
 92%|█████████▏| 206592/225000 [397:31:32<50:16:14,  9.83s/it]
 92%|█████████▏| 206593/225000 [397:31:39<45:22:43,  8.88s/it]
 92%|█████████▏| 206594/225000 [397:31:49<47:26:37,  9.28s/it]
 92%|█████████▏| 206595/225000 [397:31:56<43:52:03,  8.58s/it]
 92%|█████████▏| 206596/225000 [397:32:02<40:16:37,  7.88s/it]
 92%|█████████▏| 206597/225000 [397:32:12<42:51:03,  8.38s/it]
 92%|█████████▏| 206598/225000 [397:32:27<53:34:18, 10.48s/it]
 92%|█████████▏| 206599/225000 [397:32:33<45:33:05,  8.91s/it]
 92%|█████████▏| 206600/225000 [397:32:45<50:45:46,  9.93s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.514', 'learning_rate': '9.076e-07', 'epoch': '0.9182', 'num_input_tokens_seen': 670444224, 'train_runtime': '1.431e+06', 'train_tokens_per_second': '468.5'}

 92%|█████████▏| 206600/225000 [397:32:45<50:45:46,  9.93s/it]
 92%|█████████▏| 206601/225000 [397:32:50<43:12:23,  8.45s/it]
 92%|█████████▏| 206602/225000 [397:33:01<47:08:11,  9.22s/it]
 92%|█████████▏| 206603/225000 [397:33:09<44:47:33,  8.77s/it]
 92%|█████████▏| 206604/225000 [397:33:16<43:20:57,  8.48s/it]
 92%|█████████▏| 206605/225000 [397:33:22<38:29:53,  7.53s/it]
 92%|█████████▏| 206606/225000 [397:33:28<36:33:09,  7.15s/it]
 92%|█████████▏| 206607/225000 [397:33:38<40:20:46,  7.90s/it]
 92%|█████████▏| 206608/225000 [397:33:44<37:37:15,  7.36s/it]
 92%|█████████▏| 206609/225000 [397:33:56<45:37:31,  8.93s/it]
 92%|█████████▏| 206610/225000 [397:34:02<41:23:29,  8.10s/it]
 92%|█████████▏| 206611/225000 [397:34:12<42:54:11,  8.40s/it]
 92%|█████████▏| 206612/225000 [397:34:17<39:00:11,  7.64s/it]
 92%|█████████▏| 206613/225000 [397:34:30<47:15:43,  9.25s/it]
 92%|█████████▏| 206614/225000 [397:34:42<51:03:35, 10.00s/it]
 92%|█████████▏| 206615/225000 [397:34:50<47:25:29,  9.29s/it]
 92%|█████████▏| 206616/225000 [397:34:56<42:09:23,  8.26s/it]
 92%|█████████▏| 206617/225000 [397:35:05<44:29:59,  8.71s/it]
 92%|█████████▏| 206618/225000 [397:35:16<47:35:21,  9.32s/it]
 92%|█████████▏| 206619/225000 [397:35:22<42:49:31,  8.39s/it]
 92%|█████████▏| 206620/225000 [397:35:35<49:09:19,  9.63s/it]
 92%|█████████▏| 206621/225000 [397:35:40<42:07:27,  8.25s/it]
 92%|█████████▏| 206622/225000 [397:35:46<38:30:15,  7.54s/it]
 92%|█████████▏| 206623/225000 [397:35:58<45:54:49,  8.99s/it]
 92%|█████████▏| 206624/225000 [397:36:08<47:29:29,  9.30s/it]
 92%|█████████▏| 206625/225000 [397:36:18<47:35:38,  9.32s/it]
 92%|█████████▏| 206626/225000 [397:36:28<49:04:00,  9.61s/it]
 92%|█████████▏| 206627/225000 [397:36:39<51:18:17, 10.05s/it]
 92%|█████████▏| 206628/225000 [397:36:53<57:18:18, 11.23s/it]
 92%|█████████▏| 206629/225000 [397:37:00<50:18:41,  9.86s/it]
 92%|█████████▏| 206630/225000 [397:37:05<42:59:24,  8.42s/it]
 92%|█████████▏| 206631/225000 [397:37:14<44:46:16,  8.77s/it]
 92%|█████████▏| 206632/225000 [397:37:21<41:12:53,  8.08s/it]
 92%|█████████▏| 206633/225000 [397:37:27<38:33:28,  7.56s/it]
 92%|█████████▏| 206634/225000 [397:37:38<44:13:40,  8.67s/it]
 92%|█████████▏| 206635/225000 [397:37:47<44:09:29,  8.66s/it]
 92%|█████████▏| 206636/225000 [397:37:59<48:47:41,  9.57s/it]
 92%|█████████▏| 206637/225000 [397:38:07<47:08:57,  9.24s/it]
 92%|█████████▏| 206638/225000 [397:38:15<45:41:10,  8.96s/it]
 92%|█████████▏| 206639/225000 [397:38:24<44:36:17,  8.75s/it]
 92%|█████████▏| 206640/225000 [397:38:39<54:54:49, 10.77s/it]
 92%|█████████▏| 206641/225000 [397:38:45<47:48:45,  9.38s/it]
 92%|█████████▏| 206642/225000 [397:38:57<51:33:18, 10.11s/it]
 92%|█████████▏| 206643/225000 [397:39:05<47:59:45,  9.41s/it]
 92%|█████████▏| 206644/225000 [397:39:11<43:05:04,  8.45s/it]
 92%|█████████▏| 206645/225000 [397:39:22<46:34:47,  9.14s/it]
 92%|█████████▏| 206646/225000 [397:39:29<43:30:22,  8.53s/it]
 92%|█████████▏| 206647/225000 [397:39:37<42:48:52,  8.40s/it]
 92%|█████████▏| 206648/225000 [397:39:46<43:14:23,  8.48s/it]
 92%|█████████▏| 206649/225000 [397:39:52<39:19:34,  7.71s/it]
 92%|█████████▏| 206650/225000 [397:40:02<43:21:01,  8.50s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.387', 'learning_rate': '9.027e-07', 'epoch': '0.9184', 'num_input_tokens_seen': 670680592, 'train_runtime': '1.432e+06', 'train_tokens_per_second': '468.5'}

 92%|█████████▏| 206650/225000 [397:40:02<43:21:01,  8.50s/it]
 92%|█████████▏| 206651/225000 [397:40:09<40:26:38,  7.93s/it]
 92%|█████████▏| 206652/225000 [397:40:16<40:17:25,  7.91s/it]
 92%|█████████▏| 206653/225000 [397:40:25<40:58:54,  8.04s/it]
 92%|█████████▏| 206654/225000 [397:40:31<38:04:40,  7.47s/it]
 92%|█████████▏| 206655/225000 [397:40:39<39:19:27,  7.72s/it]
 92%|█████████▏| 206656/225000 [397:40:48<40:23:01,  7.93s/it]
 92%|█████████▏| 206657/225000 [397:40:54<38:14:05,  7.50s/it]
 92%|█████████▏| 206658/225000 [397:41:03<39:32:31,  7.76s/it]
 92%|█████████▏| 206659/225000 [397:41:09<37:49:21,  7.42s/it]
 92%|█████████▏| 206660/225000 [397:41:24<49:38:41,  9.74s/it]
 92%|█████████▏| 206661/225000 [397:41:32<46:23:03,  9.11s/it]
 92%|█████████▏| 206662/225000 [397:41:41<46:59:29,  9.23s/it]
 92%|█████████▏| 206663/225000 [397:41:50<45:53:49,  9.01s/it]
 92%|█████████▏| 206664/225000 [397:42:00<46:55:03,  9.21s/it]
 92%|█████████▏| 206665/225000 [397:42:06<42:06:10,  8.27s/it]
 92%|█████████▏| 206666/225000 [397:42:15<42:56:26,  8.43s/it]
 92%|█████████▏| 206667/225000 [397:42:20<38:17:35,  7.52s/it]
 92%|█████████▏| 206668/225000 [397:42:32<44:43:20,  8.78s/it]
 92%|█████████▏| 206669/225000 [397:42:38<40:57:13,  8.04s/it]
 92%|█████████▏| 206670/225000 [397:42:45<39:01:52,  7.67s/it]
 92%|█████████▏| 206671/225000 [397:42:52<39:02:27,  7.67s/it]
 92%|█████████▏| 206672/225000 [397:43:04<44:50:16,  8.81s/it]
 92%|█████████▏| 206673/225000 [397:43:12<43:42:59,  8.59s/it]
 92%|█████████▏| 206674/225000 [397:43:23<47:15:31,  9.28s/it]
 92%|█████████▏| 206675/225000 [397:43:31<46:11:08,  9.07s/it]
 92%|█████████▏| 206676/225000 [397:43:38<42:44:02,  8.40s/it]
 92%|█████████▏| 206677/225000 [397:43:45<40:40:44,  7.99s/it]
 92%|█████████▏| 206678/225000 [397:43:51<36:29:54,  7.17s/it]
 92%|█████████▏| 206679/225000 [397:44:04<45:58:00,  9.03s/it]
 92%|█████████▏| 206680/225000 [397:44:14<47:35:07,  9.35s/it]
 92%|█████████▏| 206681/225000 [397:44:20<41:53:23,  8.23s/it]
 92%|█████████▏| 206682/225000 [397:44:31<46:33:24,  9.15s/it]
 92%|█████████▏| 206683/225000 [397:44:43<50:21:33,  9.90s/it]
 92%|█████████▏| 206684/225000 [397:44:47<42:30:40,  8.36s/it]
 92%|█████████▏| 206685/225000 [397:44:56<43:15:09,  8.50s/it]
 92%|█████████▏| 206686/225000 [397:45:02<38:29:31,  7.57s/it]
 92%|█████████▏| 206687/225000 [397:45:12<42:57:41,  8.45s/it]
 92%|█████████▏| 206688/225000 [397:45:21<43:44:10,  8.60s/it]
 92%|█████████▏| 206689/225000 [397:45:26<38:19:12,  7.53s/it]
 92%|█████████▏| 206690/225000 [397:45:35<41:03:00,  8.07s/it]
 92%|█████████▏| 206691/225000 [397:45:41<37:56:24,  7.46s/it]
 92%|█████████▏| 206692/225000 [397:45:51<41:35:10,  8.18s/it]
 92%|█████████▏| 206693/225000 [397:45:57<38:26:39,  7.56s/it]
 92%|█████████▏| 206694/225000 [397:46:08<42:28:23,  8.35s/it]
 92%|█████████▏| 206695/225000 [397:46:20<49:23:05,  9.71s/it]
 92%|█████████▏| 206696/225000 [397:46:34<54:53:33, 10.80s/it]
 92%|█████████▏| 206697/225000 [397:46:39<46:33:02,  9.16s/it]
 92%|█████████▏| 206698/225000 [397:46:48<45:43:45,  8.99s/it]
 92%|█████████▏| 206699/225000 [397:46:59<48:33:37,  9.55s/it]
 92%|█████████▏| 206700/225000 [397:47:04<42:28:02,  8.35s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.359', 'learning_rate': '8.979e-07', 'epoch': '0.9187', 'num_input_tokens_seen': 670910312, 'train_runtime': '1.432e+06', 'train_tokens_per_second': '468.5'}

 92%|█████████▏| 206700/225000 [397:47:04<42:28:02,  8.35s/it]
 92%|█████████▏| 206701/225000 [397:47:16<48:13:57,  9.49s/it]
 92%|█████████▏| 206702/225000 [397:47:26<49:06:23,  9.66s/it]
 92%|█████████▏| 206703/225000 [397:47:33<44:19:18,  8.72s/it]
 92%|█████████▏| 206704/225000 [397:47:41<44:04:08,  8.67s/it]
 92%|█████████▏| 206705/225000 [397:47:51<45:14:15,  8.90s/it]
 92%|█████████▏| 206706/225000 [397:48:01<46:33:36,  9.16s/it]
 92%|█████████▏| 206707/225000 [397:48:07<42:32:48,  8.37s/it]
 92%|█████████▏| 206708/225000 [397:48:20<49:22:24,  9.72s/it]
 92%|█████████▏| 206709/225000 [397:48:28<46:34:57,  9.17s/it]
 92%|█████████▏| 206710/225000 [397:48:34<41:33:23,  8.18s/it]
 92%|█████████▏| 206711/225000 [397:48:46<46:54:58,  9.23s/it]
 92%|█████████▏| 206712/225000 [397:48:54<46:12:49,  9.10s/it]
 92%|█████████▏| 206713/225000 [397:48:59<39:22:25,  7.75s/it]
 92%|█████████▏| 206714/225000 [397:49:05<36:48:18,  7.25s/it]
 92%|█████████▏| 206715/225000 [397:49:15<41:01:30,  8.08s/it]
 92%|█████████▏| 206716/225000 [397:49:25<44:43:08,  8.80s/it]
 92%|█████████▏| 206717/225000 [397:49:33<42:50:38,  8.44s/it]
 92%|█████████▏| 206718/225000 [397:49:44<46:14:49,  9.11s/it]
 92%|█████████▏| 206719/225000 [397:49:50<41:31:33,  8.18s/it]
 92%|█████████▏| 206720/225000 [397:49:57<39:59:52,  7.88s/it]
 92%|█████████▏| 206721/225000 [397:50:05<39:49:57,  7.84s/it]
 92%|█████████▏| 206722/225000 [397:50:11<38:12:15,  7.52s/it]
 92%|█████████▏| 206723/225000 [397:50:30<54:33:37, 10.75s/it]
 92%|█████████▏| 206724/225000 [397:50:39<52:10:35, 10.28s/it]
 92%|█████████▏| 206725/225000 [397:50:46<47:31:54,  9.36s/it]
 92%|█████████▏| 206726/225000 [397:50:53<43:19:10,  8.53s/it]
 92%|█████████▏| 206727/225000 [397:50:58<38:29:37,  7.58s/it]
 92%|█████████▏| 206728/225000 [397:51:05<37:42:01,  7.43s/it]
 92%|█████████▏| 206729/225000 [397:51:21<49:47:06,  9.81s/it]
 92%|█████████▏| 206730/225000 [397:51:29<47:06:42,  9.28s/it]
 92%|█████████▏| 206731/225000 [397:51:39<49:14:35,  9.70s/it]
 92%|█████████▏| 206732/225000 [397:51:45<43:03:52,  8.49s/it]
 92%|█████████▏| 206733/225000 [397:51:54<44:36:09,  8.79s/it]
 92%|█████████▏| 206734/225000 [397:52:03<43:51:22,  8.64s/it]
 92%|█████████▏| 206735/225000 [397:52:09<39:43:10,  7.83s/it]
 92%|█████████▏| 206736/225000 [397:52:21<46:22:45,  9.14s/it]
 92%|█████████▏| 206737/225000 [397:52:28<42:35:00,  8.39s/it]
 92%|█████████▏| 206738/225000 [397:52:33<37:25:17,  7.38s/it]
 92%|█████████▏| 206739/225000 [397:52:43<42:10:20,  8.31s/it]
 92%|█████████▏| 206740/225000 [397:52:52<43:37:47,  8.60s/it]
 92%|█████████▏| 206741/225000 [397:52:59<40:08:57,  7.92s/it]
 92%|█████████▏| 206742/225000 [397:53:07<41:03:13,  8.09s/it]
 92%|█████████▏| 206743/225000 [397:53:16<42:13:31,  8.33s/it]
 92%|█████████▏| 206744/225000 [397:53:24<41:39:49,  8.22s/it]
 92%|█████████▏| 206745/225000 [397:53:33<43:20:38,  8.55s/it]
 92%|█████████▏| 206746/225000 [397:53:43<44:43:42,  8.82s/it]
 92%|█████████▏| 206747/225000 [397:53:49<41:28:18,  8.18s/it]
 92%|█████████▏| 206748/225000 [397:53:57<39:54:56,  7.87s/it]
 92%|█████████▏| 206749/225000 [397:54:03<38:04:13,  7.51s/it]
 92%|█████████▏| 206750/225000 [397:54:14<43:21:09,  8.55s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.058', 'learning_rate': '8.93e-07', 'epoch': '0.9189', 'num_input_tokens_seen': 671146992, 'train_runtime': '1.432e+06', 'train_tokens_per_second': '468.5'}

 92%|█████████▏| 206750/225000 [397:54:14<43:21:09,  8.55s/it]
 92%|█████████▏| 206751/225000 [397:54:20<39:13:00,  7.74s/it]
 92%|█████████▏| 206752/225000 [397:54:32<44:53:39,  8.86s/it]
 92%|█████████▏| 206753/225000 [397:54:39<42:52:14,  8.46s/it]
 92%|█████████▏| 206754/225000 [397:54:47<41:32:34,  8.20s/it]
 92%|█████████▏| 206755/225000 [397:54:59<47:36:26,  9.39s/it]
 92%|█████████▏| 206756/225000 [397:55:09<49:14:24,  9.72s/it]
 92%|█████████▏| 206757/225000 [397:55:17<46:28:46,  9.17s/it]
 92%|█████████▏| 206758/225000 [397:55:23<42:05:09,  8.31s/it]
 92%|█████████▏| 206759/225000 [397:55:30<39:47:42,  7.85s/it]
 92%|█████████▏| 206760/225000 [397:55:40<43:08:26,  8.51s/it]
 92%|█████████▏| 206761/225000 [397:55:47<40:13:16,  7.94s/it]
 92%|█████████▏| 206762/225000 [397:55:59<46:31:17,  9.18s/it]
 92%|█████████▏| 206763/225000 [397:56:09<47:36:40,  9.40s/it]
 92%|█████████▏| 206764/225000 [397:56:15<42:23:57,  8.37s/it]
 92%|█████████▏| 206765/225000 [397:56:24<43:06:16,  8.51s/it]
 92%|█████████▏| 206766/225000 [397:56:30<40:00:17,  7.90s/it]
 92%|█████████▏| 206767/225000 [397:56:36<37:24:57,  7.39s/it]
 92%|█████████▏| 206768/225000 [397:56:43<36:38:05,  7.23s/it]
 92%|█████████▏| 206769/225000 [397:56:50<36:12:32,  7.15s/it]
 92%|█████████▏| 206770/225000 [397:57:06<49:54:14,  9.85s/it]
 92%|█████████▏| 206771/225000 [397:57:15<48:04:33,  9.49s/it]
 92%|█████████▏| 206772/225000 [397:57:25<48:29:20,  9.58s/it]
 92%|█████████▏| 206773/225000 [397:57:34<47:54:56,  9.46s/it]
 92%|█████████▏| 206774/225000 [397:57:43<47:10:55,  9.32s/it]
 92%|█████████▏| 206775/225000 [397:57:50<44:15:09,  8.74s/it]
 92%|█████████▏| 206776/225000 [397:57:59<43:49:23,  8.66s/it]
 92%|█████████▏| 206777/225000 [397:58:06<40:50:33,  8.07s/it]
 92%|█████████▏| 206778/225000 [397:58:14<41:02:47,  8.11s/it]
 92%|█████████▏| 206779/225000 [397:58:20<38:00:22,  7.51s/it]
 92%|█████████▏| 206780/225000 [397:58:28<39:19:39,  7.77s/it]
 92%|█████████▏| 206781/225000 [397:58:35<37:54:01,  7.49s/it]
 92%|█████████▏| 206782/225000 [397:58:46<43:45:13,  8.65s/it]
 92%|█████████▏| 206783/225000 [397:58:56<44:52:22,  8.87s/it]
 92%|█████████▏| 206784/225000 [397:59:01<38:44:10,  7.66s/it]
 92%|█████████▏| 206785/225000 [397:59:05<33:42:08,  6.66s/it]
 92%|█████████▏| 206786/225000 [397:59:12<33:51:04,  6.69s/it]
 92%|█████████▏| 206787/225000 [397:59:19<34:03:06,  6.73s/it]
 92%|█████████▏| 206788/225000 [397:59:30<41:57:39,  8.29s/it]
 92%|█████████▏| 206789/225000 [397:59:38<41:14:50,  8.15s/it]
 92%|█████████▏| 206790/225000 [397:59:49<45:19:56,  8.96s/it]
 92%|█████████▏| 206791/225000 [397:59:57<43:07:55,  8.53s/it]
 92%|█████████▏| 206792/225000 [398:00:03<39:20:04,  7.78s/it]
 92%|█████████▏| 206793/225000 [398:00:09<37:30:58,  7.42s/it]
 92%|█████████▏| 206794/225000 [398:00:18<39:24:35,  7.79s/it]
 92%|█████████▏| 206795/225000 [398:00:24<37:14:48,  7.37s/it]
 92%|█████████▏| 206796/225000 [398:00:31<36:58:54,  7.31s/it]
 92%|█████████▏| 206797/225000 [398:00:43<43:16:53,  8.56s/it]
 92%|█████████▏| 206798/225000 [398:00:51<42:10:23,  8.34s/it]
 92%|█████████▏| 206799/225000 [398:01:03<47:31:58,  9.40s/it]
 92%|█████████▏| 206800/225000 [398:01:13<48:41:14,  9.63s/it]
                                                              
{'loss': '1.342', 'grad_norm': '1.016', 'learning_rate': '8.882e-07', 'epoch': '0.9191', 'num_input_tokens_seen': 671373840, 'train_runtime': '1.433e+06', 'train_tokens_per_second': '468.6'}

 92%|█████████▏| 206800/225000 [398:01:13<48:41:14,  9.63s/it]
 92%|█████████▏| 206801/225000 [398:01:18<42:22:17,  8.38s/it]
 92%|█████████▏| 206802/225000 [398:01:29<46:11:00,  9.14s/it]
 92%|█████████▏| 206803/225000 [398:01:38<45:15:44,  8.95s/it]
 92%|█████████▏| 206804/225000 [398:01:46<44:15:33,  8.76s/it]
 92%|█████████▏| 206805/225000 [398:01:52<40:17:27,  7.97s/it]
 92%|█████████▏| 206806/225000 [398:02:01<41:39:16,  8.24s/it]
 92%|█████████▏| 206807/225000 [398:02:08<39:51:05,  7.89s/it]
 92%|█████████▏| 206808/225000 [398:02:21<46:58:45,  9.30s/it]
 92%|█████████▏| 206809/225000 [398:02:26<41:09:34,  8.15s/it]
 92%|█████████▏| 206810/225000 [398:02:37<45:52:31,  9.08s/it]
 92%|█████████▏| 206811/225000 [398:02:43<40:05:13,  7.93s/it]
 92%|█████████▏| 206812/225000 [398:02:50<39:22:10,  7.79s/it]
 92%|█████████▏| 206813/225000 [398:03:05<49:28:31,  9.79s/it]
 92%|█████████▏| 206814/225000 [398:03:10<42:08:43,  8.34s/it]
 92%|█████████▏| 206815/225000 [398:03:21<46:12:52,  9.15s/it]
 92%|█████████▏| 206816/225000 [398:03:26<40:24:46,  8.00s/it]
 92%|█████████▏| 206817/225000 [398:03:30<34:50:06,  6.90s/it]
 92%|█████████▏| 206818/225000 [398:03:44<45:37:09,  9.03s/it]
 92%|█████████▏| 206819/225000 [398:03:50<40:35:20,  8.04s/it]
 92%|█████████▏| 206820/225000 [398:03:57<39:36:54,  7.84s/it]
 92%|█████████▏| 206821/225000 [398:04:06<40:44:36,  8.07s/it]
 92%|█████████▏| 206822/225000 [398:04:17<44:54:56,  8.90s/it]
 92%|█████████▏| 206823/225000 [398:04:21<37:39:53,  7.46s/it]
 92%|█████████▏| 206824/225000 [398:04:31<42:01:59,  8.33s/it]
 92%|█████████▏| 206825/225000 [398:04:39<41:42:44,  8.26s/it]
 92%|█████████▏| 206826/225000 [398:04:46<39:22:00,  7.80s/it]
 92%|█████████▏| 206827/225000 [398:04:52<35:51:07,  7.10s/it]
 92%|█████████▏| 206828/225000 [398:04:58<35:04:08,  6.95s/it]
 92%|█████████▏| 206829/225000 [398:05:05<34:39:53,  6.87s/it]
 92%|█████████▏| 206830/225000 [398:05:18<44:19:11,  8.78s/it]
 92%|█████████▏| 206831/225000 [398:05:28<45:57:12,  9.11s/it]
 92%|█████████▏| 206832/225000 [398:05:33<39:51:37,  7.90s/it]
 92%|█████████▏| 206833/225000 [398:05:44<43:50:23,  8.69s/it]
 92%|█████████▏| 206834/225000 [398:05:54<45:53:01,  9.09s/it]
 92%|█████████▏| 206835/225000 [398:05:59<40:40:49,  8.06s/it]
 92%|█████████▏| 206836/225000 [398:06:12<48:33:08,  9.62s/it]
 92%|█████████▏| 206837/225000 [398:06:19<44:27:46,  8.81s/it]
 92%|█████████▏| 206838/225000 [398:06:28<43:38:51,  8.65s/it]
 92%|█████████▏| 206839/225000 [398:06:38<45:47:18,  9.08s/it]
 92%|█████████▏| 206840/225000 [398:06:42<37:54:35,  7.52s/it]
 92%|█████████▏| 206841/225000 [398:06:52<42:33:26,  8.44s/it]
 92%|█████████▏| 206842/225000 [398:06:59<39:49:28,  7.90s/it]
 92%|█████████▏| 206843/225000 [398:07:04<35:48:54,  7.10s/it]
 92%|█████████▏| 206844/225000 [398:07:18<46:30:29,  9.22s/it]
 92%|█████████▏| 206845/225000 [398:07:24<40:36:01,  8.05s/it]
 92%|█████████▏| 206846/225000 [398:07:33<42:56:57,  8.52s/it]
 92%|█████████▏| 206847/225000 [398:07:42<44:09:12,  8.76s/it]
 92%|█████████▏| 206848/225000 [398:07:50<41:31:51,  8.24s/it]
 92%|█████████▏| 206849/225000 [398:08:01<45:59:14,  9.12s/it]
 92%|█████████▏| 206850/225000 [398:08:11<48:02:12,  9.53s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.611', 'learning_rate': '8.834e-07', 'epoch': '0.9193', 'num_input_tokens_seen': 671595584, 'train_runtime': '1.433e+06', 'train_tokens_per_second': '468.6'}

 92%|█████████▏| 206850/225000 [398:08:11<48:02:12,  9.53s/it]
 92%|█████████▏| 206851/225000 [398:08:19<45:50:55,  9.09s/it]
 92%|█████████▏| 206852/225000 [398:08:25<41:24:58,  8.22s/it]
 92%|█████████▏| 206853/225000 [398:08:36<45:29:19,  9.02s/it]
 92%|█████████▏| 206854/225000 [398:08:43<41:39:12,  8.26s/it]
 92%|█████████▏| 206855/225000 [398:08:54<46:34:15,  9.24s/it]
 92%|█████████▏| 206856/225000 [398:08:59<39:57:52,  7.93s/it]
 92%|█████████▏| 206857/225000 [398:09:06<37:47:33,  7.50s/it]
 92%|█████████▏| 206858/225000 [398:09:21<48:52:10,  9.70s/it]
 92%|█████████▏| 206859/225000 [398:09:32<52:06:11, 10.34s/it]
 92%|█████████▏| 206860/225000 [398:09:40<47:52:13,  9.50s/it]
 92%|█████████▏| 206861/225000 [398:09:48<45:40:26,  9.06s/it]
 92%|█████████▏| 206862/225000 [398:09:54<41:42:56,  8.28s/it]
 92%|█████████▏| 206863/225000 [398:09:59<36:33:18,  7.26s/it]
 92%|█████████▏| 206864/225000 [398:10:09<39:41:35,  7.88s/it]
 92%|█████████▏| 206865/225000 [398:10:16<39:12:50,  7.78s/it]
 92%|█████████▏| 206866/225000 [398:10:23<37:04:53,  7.36s/it]
 92%|█████████▏| 206867/225000 [398:10:33<41:11:51,  8.18s/it]
 92%|█████████▏| 206868/225000 [398:10:45<48:00:49,  9.53s/it]
 92%|█████████▏| 206869/225000 [398:10:54<46:01:55,  9.14s/it]
 92%|█████████▏| 206870/225000 [398:11:02<44:15:32,  8.79s/it]
 92%|█████████▏| 206871/225000 [398:11:12<47:19:14,  9.40s/it]
 92%|█████████▏| 206872/225000 [398:11:22<47:23:05,  9.41s/it]
 92%|█████████▏| 206873/225000 [398:11:30<45:39:01,  9.07s/it]
 92%|█████████▏| 206874/225000 [398:11:40<47:17:43,  9.39s/it]
 92%|█████████▏| 206875/225000 [398:11:48<44:56:58,  8.93s/it]
 92%|█████████▏| 206876/225000 [398:11:57<45:22:47,  9.01s/it]
 92%|█████████▏| 206877/225000 [398:12:05<43:50:39,  8.71s/it]
 92%|█████████▏| 206878/225000 [398:12:12<41:00:51,  8.15s/it]
 92%|█████████▏| 206879/225000 [398:12:25<48:10:14,  9.57s/it]
 92%|█████████▏| 206880/225000 [398:12:33<45:23:41,  9.02s/it]
 92%|█████████▏| 206881/225000 [398:12:43<46:57:40,  9.33s/it]
 92%|█████████▏| 206882/225000 [398:12:55<51:56:57, 10.32s/it]
 92%|█████████▏| 206883/225000 [398:13:00<43:02:06,  8.55s/it]
 92%|█████████▏| 206884/225000 [398:13:09<43:23:59,  8.62s/it]
 92%|█████████▏| 206885/225000 [398:13:13<37:07:12,  7.38s/it]
 92%|█████████▏| 206886/225000 [398:13:22<40:01:37,  7.96s/it]
 92%|█████████▏| 206887/225000 [398:13:31<40:52:31,  8.12s/it]
 92%|█████████▏| 206888/225000 [398:13:38<38:49:30,  7.72s/it]
 92%|█████████▏| 206889/225000 [398:13:51<46:50:15,  9.31s/it]
 92%|█████████▏| 206890/225000 [398:14:01<48:15:30,  9.59s/it]
 92%|█████████▏| 206891/225000 [398:14:09<45:57:05,  9.13s/it]
 92%|█████████▏| 206892/225000 [398:14:18<45:02:24,  8.95s/it]
 92%|█████████▏| 206893/225000 [398:14:26<43:45:14,  8.70s/it]
 92%|█████████▏| 206894/225000 [398:14:35<45:12:39,  8.99s/it]
 92%|█████████▏| 206895/225000 [398:14:42<41:34:22,  8.27s/it]
 92%|█████████▏| 206896/225000 [398:14:57<51:33:17, 10.25s/it]
 92%|█████████▏| 206897/225000 [398:15:04<47:13:19,  9.39s/it]
 92%|█████████▏| 206898/225000 [398:15:12<45:12:25,  8.99s/it]
 92%|█████████▏| 206899/225000 [398:15:25<50:30:16, 10.04s/it]
 92%|█████████▏| 206900/225000 [398:15:32<46:18:12,  9.21s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.431', 'learning_rate': '8.786e-07', 'epoch': '0.9196', 'num_input_tokens_seen': 671833384, 'train_runtime': '1.434e+06', 'train_tokens_per_second': '468.6'}

 92%|█████████▏| 206900/225000 [398:15:32<46:18:12,  9.21s/it]
 92%|█████████▏| 206901/225000 [398:15:42<46:46:27,  9.30s/it]
 92%|█████████▏| 206902/225000 [398:15:50<44:56:40,  8.94s/it]
 92%|█████████▏| 206903/225000 [398:15:55<39:32:27,  7.87s/it]
 92%|█████████▏| 206904/225000 [398:16:09<48:11:01,  9.59s/it]
 92%|█████████▏| 206905/225000 [398:16:14<42:26:59,  8.45s/it]
 92%|█████████▏| 206906/225000 [398:16:22<41:51:38,  8.33s/it]
 92%|█████████▏| 206907/225000 [398:16:30<40:44:50,  8.11s/it]
 92%|█████████▏| 206908/225000 [398:16:44<50:00:10,  9.95s/it]
 92%|█████████▏| 206909/225000 [398:16:51<44:49:34,  8.92s/it]
 92%|█████████▏| 206910/225000 [398:16:57<41:17:40,  8.22s/it]
 92%|█████████▏| 206911/225000 [398:17:05<40:28:11,  8.05s/it]
 92%|█████████▏| 206912/225000 [398:17:20<50:30:28, 10.05s/it]
 92%|█████████▏| 206913/225000 [398:17:25<43:41:18,  8.70s/it]
 92%|█████████▏| 206914/225000 [398:17:32<41:09:11,  8.19s/it]
 92%|█████████▏| 206915/225000 [398:17:42<43:03:08,  8.57s/it]
 92%|█████████▏| 206916/225000 [398:17:51<44:34:27,  8.87s/it]
 92%|█████████▏| 206917/225000 [398:18:01<45:44:11,  9.11s/it]
 92%|█████████▏| 206918/225000 [398:18:06<38:57:15,  7.76s/it]
 92%|█████████▏| 206919/225000 [398:18:21<50:11:00,  9.99s/it]
 92%|█████████▏| 206920/225000 [398:18:31<50:33:06, 10.07s/it]
 92%|█████████▏| 206921/225000 [398:18:40<48:28:56,  9.65s/it]
 92%|█████████▏| 206922/225000 [398:18:49<47:28:04,  9.45s/it]
 92%|█████████▏| 206923/225000 [398:18:55<42:16:11,  8.42s/it]
 92%|█████████▏| 206924/225000 [398:19:01<38:41:54,  7.71s/it]
 92%|█████████▏| 206925/225000 [398:19:09<39:37:46,  7.89s/it]
 92%|█████████▏| 206926/225000 [398:19:16<37:32:55,  7.48s/it]
 92%|█████████▏| 206927/225000 [398:19:31<49:07:10,  9.78s/it]
 92%|█████████▏| 206928/225000 [398:19:37<44:29:34,  8.86s/it]
 92%|█████████▏| 206929/225000 [398:19:43<39:54:37,  7.95s/it]
 92%|█████████▏| 206930/225000 [398:19:48<35:08:03,  7.00s/it]
 92%|█████████▏| 206931/225000 [398:20:06<50:55:10, 10.15s/it]
 92%|█████████▏| 206932/225000 [398:20:14<47:39:47,  9.50s/it]
 92%|█████████▏| 206933/225000 [398:20:25<49:57:40,  9.96s/it]
 92%|█████████▏| 206934/225000 [398:20:31<45:16:34,  9.02s/it]
 92%|█████████▏| 206935/225000 [398:20:41<45:30:51,  9.07s/it]
 92%|█████████▏| 206936/225000 [398:20:50<45:23:30,  9.05s/it]
 92%|█████████▏| 206937/225000 [398:21:02<50:42:12, 10.11s/it]
 92%|█████████▏| 206938/225000 [398:21:09<45:50:23,  9.14s/it]
 92%|█████████▏| 206939/225000 [398:21:17<44:16:30,  8.83s/it]
 92%|█████████▏| 206940/225000 [398:21:24<41:13:27,  8.22s/it]
 92%|█████████▏| 206941/225000 [398:21:35<45:40:53,  9.11s/it]
 92%|█████████▏| 206942/225000 [398:21:50<54:08:04, 10.79s/it]
 92%|█████████▏| 206943/225000 [398:21:59<51:49:11, 10.33s/it]
 92%|█████████▏| 206944/225000 [398:22:06<46:02:49,  9.18s/it]
 92%|█████████▏| 206945/225000 [398:22:14<44:17:10,  8.83s/it]
 92%|█████████▏| 206946/225000 [398:22:20<40:49:18,  8.14s/it]
 92%|█████████▏| 206947/225000 [398:22:27<39:25:54,  7.86s/it]
 92%|█████████▏| 206948/225000 [398:22:43<51:30:24, 10.27s/it]
 92%|█████████▏| 206949/225000 [398:22:50<46:30:40,  9.28s/it]
 92%|█████████▏| 206950/225000 [398:22:56<41:31:50,  8.28s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.188', 'learning_rate': '8.738e-07', 'epoch': '0.9198', 'num_input_tokens_seen': 672076304, 'train_runtime': '1.434e+06', 'train_tokens_per_second': '468.6'}

 92%|█████████▏| 206950/225000 [398:22:56<41:31:50,  8.28s/it]
 92%|█████████▏| 206951/225000 [398:23:02<38:11:58,  7.62s/it]
 92%|█████████▏| 206952/225000 [398:23:14<43:58:58,  8.77s/it]
 92%|█████████▏| 206953/225000 [398:23:21<42:32:05,  8.48s/it]
 92%|█████████▏| 206954/225000 [398:23:29<40:23:28,  8.06s/it]
 92%|█████████▏| 206955/225000 [398:23:36<39:20:54,  7.85s/it]
 92%|█████████▏| 206956/225000 [398:23:42<36:53:40,  7.36s/it]
 92%|█████████▏| 206957/225000 [398:23:55<44:37:15,  8.90s/it]
 92%|█████████▏| 206958/225000 [398:24:06<49:01:19,  9.78s/it]
 92%|█████████▏| 206959/225000 [398:24:12<42:30:46,  8.48s/it]
 92%|█████████▏| 206960/225000 [398:24:20<41:23:57,  8.26s/it]
 92%|█████████▏| 206961/225000 [398:24:25<36:26:29,  7.27s/it]
 92%|█████████▏| 206962/225000 [398:24:31<34:59:52,  6.98s/it]
 92%|█████████▏| 206963/225000 [398:24:44<43:40:50,  8.72s/it]
 92%|█████████▏| 206964/225000 [398:24:50<40:43:06,  8.13s/it]
 92%|█████████▏| 206965/225000 [398:24:58<39:11:48,  7.82s/it]
 92%|█████████▏| 206966/225000 [398:25:06<40:11:41,  8.02s/it]
 92%|█████████▏| 206967/225000 [398:25:21<50:23:51, 10.06s/it]
 92%|█████████▏| 206968/225000 [398:25:27<44:24:12,  8.86s/it]
 92%|█████████▏| 206969/225000 [398:25:39<49:45:28,  9.93s/it]
 92%|█████████▏| 206970/225000 [398:25:48<47:59:11,  9.58s/it]
 92%|█████████▏| 206971/225000 [398:25:58<47:51:38,  9.56s/it]
 92%|█████████▏| 206972/225000 [398:26:05<44:16:52,  8.84s/it]
 92%|█████████▏| 206973/225000 [398:26:13<42:51:10,  8.56s/it]
 92%|█████████▏| 206974/225000 [398:26:22<43:44:38,  8.74s/it]
 92%|█████████▏| 206975/225000 [398:26:31<44:39:40,  8.92s/it]
 92%|█████████▏| 206976/225000 [398:26:41<46:39:08,  9.32s/it]
 92%|█████████▏| 206977/225000 [398:26:50<45:40:39,  9.12s/it]
 92%|█████████▏| 206978/225000 [398:26:58<44:00:28,  8.79s/it]
 92%|█████████▏| 206979/225000 [398:27:13<52:29:28, 10.49s/it]
 92%|█████████▏| 206980/225000 [398:27:23<52:47:36, 10.55s/it]
 92%|█████████▏| 206981/225000 [398:27:32<49:23:26,  9.87s/it]
 92%|█████████▏| 206982/225000 [398:27:43<51:07:14, 10.21s/it]
 92%|█████████▏| 206983/225000 [398:27:50<46:17:52,  9.25s/it]
 92%|█████████▏| 206984/225000 [398:28:01<49:17:15,  9.85s/it]
 92%|█████████▏| 206985/225000 [398:28:09<46:57:44,  9.38s/it]
 92%|█████████▏| 206986/225000 [398:28:14<40:08:38,  8.02s/it]
 92%|█████████▏| 206987/225000 [398:28:24<42:38:56,  8.52s/it]
 92%|█████████▏| 206988/225000 [398:28:29<38:31:40,  7.70s/it]
 92%|█████████▏| 206989/225000 [398:28:36<36:49:44,  7.36s/it]
 92%|█████████▏| 206990/225000 [398:28:42<35:16:13,  7.05s/it]
 92%|█████████▏| 206991/225000 [398:28:51<37:00:30,  7.40s/it]
 92%|█████████▏| 206992/225000 [398:28:56<34:05:16,  6.81s/it]
 92%|█████████▏| 206993/225000 [398:29:07<40:26:05,  8.08s/it]
 92%|█████████▏| 206994/225000 [398:29:15<40:19:17,  8.06s/it]
 92%|█████████▏| 206995/225000 [398:29:25<43:17:36,  8.66s/it]
 92%|█████████▏| 206996/225000 [398:29:30<37:06:47,  7.42s/it]
 92%|█████████▏| 206997/225000 [398:29:38<38:50:07,  7.77s/it]
 92%|█████████▏| 206998/225000 [398:29:46<38:58:51,  7.80s/it]
 92%|█████████▏| 206999/225000 [398:29:52<35:53:19,  7.18s/it]
 92%|█████████▏| 207000/225000 [398:30:05<45:15:22,  9.05s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.521', 'learning_rate': '8.69e-07', 'epoch': '0.92', 'num_input_tokens_seen': 672309888, 'train_runtime': '1.435e+06', 'train_tokens_per_second': '468.6'}

 92%|█████████▏| 207000/225000 [398:30:05<45:15:22,  9.05s/it]
 92%|█████████▏| 207001/225000 [398:30:14<44:37:23,  8.93s/it]
 92%|█████████▏| 207002/225000 [398:30:22<43:53:07,  8.78s/it]
 92%|█████████▏| 207003/225000 [398:30:28<39:59:48,  8.00s/it]
 92%|█████████▏| 207004/225000 [398:30:39<44:24:25,  8.88s/it]
 92%|█████████▏| 207005/225000 [398:30:47<42:09:22,  8.43s/it]
 92%|█████████▏| 207006/225000 [398:30:54<40:08:37,  8.03s/it]
 92%|█████████▏| 207007/225000 [398:31:02<40:19:51,  8.07s/it]
 92%|█████████▏| 207008/225000 [398:31:09<39:20:18,  7.87s/it]
 92%|█████████▏| 207009/225000 [398:31:25<50:40:08, 10.14s/it]
 92%|█████████▏| 207010/225000 [398:31:31<44:44:17,  8.95s/it]
 92%|█████████▏| 207011/225000 [398:31:40<44:49:46,  8.97s/it]
 92%|█████████▏| 207012/225000 [398:31:45<39:28:08,  7.90s/it]
 92%|█████████▏| 207013/225000 [398:31:57<45:15:45,  9.06s/it]
 92%|█████████▏| 207014/225000 [398:32:04<42:29:26,  8.50s/it]
 92%|█████████▏| 207015/225000 [398:32:10<38:18:41,  7.67s/it]
 92%|█████████▏| 207016/225000 [398:32:22<43:51:13,  8.78s/it]
 92%|█████████▏| 207017/225000 [398:32:27<38:14:46,  7.66s/it]
 92%|█████████▏| 207018/225000 [398:32:34<38:10:01,  7.64s/it]
 92%|█████████▏| 207019/225000 [398:32:41<36:19:59,  7.27s/it]
 92%|█████████▏| 207020/225000 [398:32:52<41:56:47,  8.40s/it]
 92%|█████████▏| 207021/225000 [398:32:59<40:44:03,  8.16s/it]
 92%|█████████▏| 207022/225000 [398:33:04<36:08:04,  7.24s/it]
 92%|█████████▏| 207023/225000 [398:33:09<32:54:45,  6.59s/it]
 92%|█████████▏| 207024/225000 [398:33:25<46:03:47,  9.22s/it]
 92%|█████████▏| 207025/225000 [398:33:33<44:13:58,  8.86s/it]
 92%|█████████▏| 207026/225000 [398:33:43<45:37:04,  9.14s/it]
 92%|█████████▏| 207027/225000 [398:33:53<47:15:51,  9.47s/it]
 92%|█████████▏| 207028/225000 [398:34:00<43:39:42,  8.75s/it]
 92%|█████████▏| 207029/225000 [398:34:05<38:52:07,  7.79s/it]
 92%|█████████▏| 207030/225000 [398:34:13<38:20:18,  7.68s/it]
 92%|█████████▏| 207031/225000 [398:34:18<34:27:03,  6.90s/it]
 92%|█████████▏| 207032/225000 [398:34:33<46:38:33,  9.35s/it]
 92%|█████████▏| 207033/225000 [398:34:40<43:21:06,  8.69s/it]
 92%|█████████▏| 207034/225000 [398:34:47<40:14:36,  8.06s/it]
 92%|█████████▏| 207035/225000 [398:34:55<40:10:11,  8.05s/it]
 92%|█████████▏| 207036/225000 [398:35:02<39:06:47,  7.84s/it]
 92%|█████████▏| 207037/225000 [398:35:21<55:53:10, 11.20s/it]
 92%|█████████▏| 207038/225000 [398:35:30<52:23:19, 10.50s/it]
 92%|█████████▏| 207039/225000 [398:35:34<43:10:32,  8.65s/it]
 92%|█████████▏| 207040/225000 [398:35:41<39:27:05,  7.91s/it]
 92%|█████████▏| 207041/225000 [398:35:50<42:24:00,  8.50s/it]
 92%|█████████▏| 207042/225000 [398:36:00<44:44:35,  8.97s/it]
 92%|█████████▏| 207043/225000 [398:36:07<40:31:45,  8.13s/it]
 92%|█████████▏| 207044/225000 [398:36:13<38:26:55,  7.71s/it]
 92%|█████████▏| 207045/225000 [398:36:27<48:01:05,  9.63s/it]
 92%|█████████▏| 207046/225000 [398:36:33<42:20:25,  8.49s/it]
 92%|█████████▏| 207047/225000 [398:36:40<39:00:59,  7.82s/it]
 92%|█████████▏| 207048/225000 [398:36:47<38:06:14,  7.64s/it]
 92%|█████████▏| 207049/225000 [398:36:53<36:23:54,  7.30s/it]
 92%|█████████▏| 207050/225000 [398:37:01<37:29:16,  7.52s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.141', 'learning_rate': '8.642e-07', 'epoch': '0.9202', 'num_input_tokens_seen': 672538816, 'train_runtime': '1.435e+06', 'train_tokens_per_second': '468.7'}

 92%|█████████▏| 207050/225000 [398:37:01<37:29:16,  7.52s/it]
 92%|█████████▏| 207051/225000 [398:37:09<37:40:41,  7.56s/it]
 92%|█████████▏| 207052/225000 [398:37:26<52:16:40, 10.49s/it]
 92%|█████████▏| 207053/225000 [398:37:36<51:09:43, 10.26s/it]
 92%|█████████▏| 207054/225000 [398:37:43<46:25:47,  9.31s/it]
 92%|█████████▏| 207055/225000 [398:37:50<42:13:23,  8.47s/it]
 92%|█████████▏| 207056/225000 [398:37:55<37:34:04,  7.54s/it]
 92%|█████████▏| 207057/225000 [398:38:03<37:34:38,  7.54s/it]
 92%|█████████▏| 207058/225000 [398:38:12<40:44:55,  8.18s/it]
 92%|█████████▏| 207059/225000 [398:38:20<40:05:21,  8.04s/it]
 92%|█████████▏| 207060/225000 [398:38:27<39:15:08,  7.88s/it]
 92%|█████████▏| 207061/225000 [398:38:36<39:49:29,  7.99s/it]
 92%|█████████▏| 207062/225000 [398:38:45<41:09:45,  8.26s/it]
 92%|█████████▏| 207063/225000 [398:38:55<43:43:34,  8.78s/it]
 92%|█████████▏| 207064/225000 [398:39:00<39:25:25,  7.91s/it]
 92%|█████████▏| 207065/225000 [398:39:06<35:22:29,  7.10s/it]
 92%|█████████▏| 207066/225000 [398:39:13<35:45:09,  7.18s/it]
 92%|█████████▏| 207067/225000 [398:39:27<45:39:20,  9.17s/it]
 92%|█████████▏| 207068/225000 [398:39:34<42:19:51,  8.50s/it]
 92%|█████████▏| 207069/225000 [398:39:43<43:53:50,  8.81s/it]
 92%|█████████▏| 207070/225000 [398:39:52<43:06:50,  8.66s/it]
 92%|█████████▏| 207071/225000 [398:40:01<43:52:20,  8.81s/it]
 92%|█████████▏| 207072/225000 [398:40:10<44:36:36,  8.96s/it]
 92%|█████████▏| 207073/225000 [398:40:18<43:17:54,  8.69s/it]
 92%|█████████▏| 207074/225000 [398:40:23<37:53:13,  7.61s/it]
 92%|█████████▏| 207075/225000 [398:40:35<43:47:21,  8.79s/it]
 92%|█████████▏| 207076/225000 [398:40:41<39:44:48,  7.98s/it]
 92%|█████████▏| 207077/225000 [398:40:51<42:28:09,  8.53s/it]
 92%|█████████▏| 207078/225000 [398:40:59<41:56:31,  8.42s/it]
 92%|█████████▏| 207079/225000 [398:41:06<39:27:07,  7.93s/it]
 92%|█████████▏| 207080/225000 [398:41:19<47:07:54,  9.47s/it]
 92%|█████████▏| 207081/225000 [398:41:29<48:46:22,  9.80s/it]
 92%|█████████▏| 207082/225000 [398:41:40<50:08:10, 10.07s/it]
 92%|█████████▏| 207083/225000 [398:41:46<44:37:16,  8.97s/it]
 92%|█████████▏| 207084/225000 [398:41:54<42:49:53,  8.61s/it]
 92%|█████████▏| 207085/225000 [398:42:01<40:54:00,  8.22s/it]
 92%|█████████▏| 207086/225000 [398:42:13<45:50:58,  9.21s/it]
 92%|█████████▏| 207087/225000 [398:42:19<40:50:25,  8.21s/it]
 92%|█████████▏| 207088/225000 [398:42:31<46:13:11,  9.29s/it]
 92%|█████████▏| 207089/225000 [398:42:39<45:11:58,  9.08s/it]
 92%|█████████▏| 207090/225000 [398:42:44<38:13:52,  7.68s/it]
 92%|█████████▏| 207091/225000 [398:42:52<39:00:11,  7.84s/it]
 92%|█████████▏| 207092/225000 [398:43:00<38:57:46,  7.83s/it]
 92%|█████████▏| 207093/225000 [398:43:05<35:47:07,  7.19s/it]
 92%|█████████▏| 207094/225000 [398:43:15<39:08:18,  7.87s/it]
 92%|█████████▏| 207095/225000 [398:43:27<45:06:06,  9.07s/it]
 92%|█████████▏| 207096/225000 [398:43:34<42:04:30,  8.46s/it]
 92%|█████████▏| 207097/225000 [398:43:41<40:03:08,  8.05s/it]
 92%|█████████▏| 207098/225000 [398:43:50<42:12:30,  8.49s/it]
 92%|█████████▏| 207099/225000 [398:44:02<47:10:57,  9.49s/it]
 92%|█████████▏| 207100/225000 [398:44:08<42:26:57,  8.54s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.448', 'learning_rate': '8.595e-07', 'epoch': '0.9204', 'num_input_tokens_seen': 672764176, 'train_runtime': '1.435e+06', 'train_tokens_per_second': '468.7'}

 92%|█████████▏| 207100/225000 [398:44:08<42:26:57,  8.54s/it]
 92%|█████████▏| 207101/225000 [398:44:17<42:29:51,  8.55s/it]
 92%|█████████▏| 207102/225000 [398:44:24<40:01:37,  8.05s/it]
 92%|█████████▏| 207103/225000 [398:44:30<37:08:48,  7.47s/it]
 92%|█████████▏| 207104/225000 [398:44:38<37:31:56,  7.55s/it]
 92%|█████████▏| 207105/225000 [398:44:51<46:16:45,  9.31s/it]
 92%|█████████▏| 207106/225000 [398:45:00<45:13:04,  9.10s/it]
 92%|█████████▏| 207107/225000 [398:45:09<45:28:50,  9.15s/it]
 92%|█████████▏| 207108/225000 [398:45:16<41:55:08,  8.43s/it]
 92%|█████████▏| 207109/225000 [398:45:24<41:45:22,  8.40s/it]
 92%|█████████▏| 207110/225000 [398:45:31<39:29:55,  7.95s/it]
 92%|█████████▏| 207111/225000 [398:45:45<48:55:07,  9.84s/it]
 92%|█████████▏| 207112/225000 [398:45:51<42:31:24,  8.56s/it]
 92%|█████████▏| 207113/225000 [398:45:56<37:07:09,  7.47s/it]
 92%|█████████▏| 207114/225000 [398:46:12<50:06:14, 10.08s/it]
 92%|█████████▏| 207115/225000 [398:46:23<51:06:21, 10.29s/it]
 92%|█████████▏| 207116/225000 [398:46:31<48:15:33,  9.71s/it]
 92%|█████████▏| 207117/225000 [398:46:40<46:52:24,  9.44s/it]
 92%|█████████▏| 207118/225000 [398:46:50<47:43:43,  9.61s/it]
 92%|█████████▏| 207119/225000 [398:46:56<41:41:42,  8.39s/it]
 92%|█████████▏| 207120/225000 [398:47:05<43:45:15,  8.81s/it]
 92%|█████████▏| 207121/225000 [398:47:16<46:08:57,  9.29s/it]
 92%|█████████▏| 207122/225000 [398:47:26<47:09:15,  9.50s/it]
 92%|█████████▏| 207123/225000 [398:47:32<42:15:21,  8.51s/it]
 92%|█████████▏| 207124/225000 [398:47:39<39:32:43,  7.96s/it]
 92%|█████████▏| 207125/225000 [398:47:50<44:56:11,  9.05s/it]
 92%|█████████▏| 207126/225000 [398:48:01<47:28:04,  9.56s/it]
 92%|█████████▏| 207127/225000 [398:48:09<45:11:58,  9.10s/it]
 92%|█████████▏| 207128/225000 [398:48:15<41:10:30,  8.29s/it]
 92%|█████████▏| 207129/225000 [398:48:23<40:18:10,  8.12s/it]
 92%|█████████▏| 207130/225000 [398:48:30<38:57:23,  7.85s/it]
 92%|█████████▏| 207131/225000 [398:48:42<45:18:23,  9.13s/it]
 92%|█████████▏| 207132/225000 [398:48:51<43:52:05,  8.84s/it]
 92%|█████████▏| 207133/225000 [398:48:59<43:34:44,  8.78s/it]
 92%|█████████▏| 207134/225000 [398:49:09<45:26:16,  9.16s/it]
 92%|█████████▏| 207135/225000 [398:49:17<43:33:47,  8.78s/it]
 92%|█████████▏| 207136/225000 [398:49:27<44:38:35,  9.00s/it]
 92%|█████████▏| 207137/225000 [398:49:32<39:45:59,  8.01s/it]
 92%|█████████▏| 207138/225000 [398:49:40<38:32:26,  7.77s/it]
 92%|█████████▏| 207139/225000 [398:49:49<41:26:05,  8.35s/it]
 92%|█████████▏| 207140/225000 [398:49:55<37:52:18,  7.63s/it]
 92%|█████████▏| 207141/225000 [398:50:03<38:16:06,  7.71s/it]
 92%|█████████▏| 207142/225000 [398:50:11<38:31:43,  7.77s/it]
 92%|█████████▏| 207143/225000 [398:50:22<43:07:52,  8.70s/it]
 92%|█████████▏| 207144/225000 [398:50:34<47:54:25,  9.66s/it]
 92%|█████████▏| 207145/225000 [398:50:40<42:14:07,  8.52s/it]
 92%|█████████▏| 207146/225000 [398:50:49<43:40:38,  8.81s/it]
 92%|█████████▏| 207147/225000 [398:50:56<40:23:19,  8.14s/it]
 92%|█████████▏| 207148/225000 [398:51:04<40:52:49,  8.24s/it]
 92%|█████████▏| 207149/225000 [398:51:12<40:53:54,  8.25s/it]
 92%|█████████▏| 207150/225000 [398:51:23<44:44:13,  9.02s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.365', 'learning_rate': '8.547e-07', 'epoch': '0.9207', 'num_input_tokens_seen': 672997624, 'train_runtime': '1.436e+06', 'train_tokens_per_second': '468.7'}

 92%|█████████▏| 207150/225000 [398:51:23<44:44:13,  9.02s/it]
 92%|█████████▏| 207151/225000 [398:51:32<44:16:43,  8.93s/it]
 92%|█████████▏| 207152/225000 [398:51:42<46:33:14,  9.39s/it]
 92%|█████████▏| 207153/225000 [398:51:48<40:35:01,  8.19s/it]
 92%|█████████▏| 207154/225000 [398:51:57<41:21:57,  8.34s/it]
 92%|█████████▏| 207155/225000 [398:52:03<39:00:54,  7.87s/it]
 92%|█████████▏| 207156/225000 [398:52:14<43:37:16,  8.80s/it]
 92%|█████████▏| 207157/225000 [398:52:21<40:22:01,  8.14s/it]
 92%|█████████▏| 207158/225000 [398:52:32<44:20:19,  8.95s/it]
 92%|█████████▏| 207159/225000 [398:52:41<44:19:59,  8.95s/it]
 92%|█████████▏| 207160/225000 [398:52:46<38:38:00,  7.80s/it]
 92%|█████████▏| 207161/225000 [398:52:56<42:11:03,  8.51s/it]
 92%|█████████▏| 207162/225000 [398:53:02<38:55:44,  7.86s/it]
 92%|█████████▏| 207163/225000 [398:53:18<50:22:34, 10.17s/it]
 92%|█████████▏| 207164/225000 [398:53:28<50:54:39, 10.28s/it]
 92%|█████████▏| 207165/225000 [398:53:33<42:34:56,  8.60s/it]
 92%|█████████▏| 207166/225000 [398:53:40<40:36:15,  8.20s/it]
 92%|█████████▏| 207167/225000 [398:53:48<39:33:36,  7.99s/it]
 92%|█████████▏| 207168/225000 [398:53:55<38:17:12,  7.73s/it]
 92%|█████████▏| 207169/225000 [398:54:08<46:01:18,  9.29s/it]
 92%|█████████▏| 207170/225000 [398:54:13<39:15:14,  7.93s/it]
 92%|█████████▏| 207171/225000 [398:54:25<45:22:10,  9.16s/it]
 92%|█████████▏| 207172/225000 [398:54:33<44:09:03,  8.92s/it]
 92%|█████████▏| 207173/225000 [398:54:44<46:58:14,  9.49s/it]
 92%|█████████▏| 207174/225000 [398:54:50<41:23:57,  8.36s/it]
 92%|█████████▏| 207175/225000 [398:54:55<36:39:40,  7.40s/it]
 92%|█████████▏| 207176/225000 [398:55:01<34:36:42,  6.99s/it]
 92%|█████████▏| 207177/225000 [398:55:08<35:27:11,  7.16s/it]
 92%|█████████▏| 207178/225000 [398:55:21<43:02:16,  8.69s/it]
 92%|█████████▏| 207179/225000 [398:55:30<43:39:02,  8.82s/it]
 92%|█████████▏| 207180/225000 [398:55:36<39:32:29,  7.99s/it]
 92%|█████████▏| 207181/225000 [398:55:42<37:20:57,  7.55s/it]
 92%|█████████▏| 207182/225000 [398:55:54<43:53:19,  8.87s/it]
 92%|█████████▏| 207183/225000 [398:55:59<37:45:53,  7.63s/it]
 92%|█████████▏| 207184/225000 [398:56:11<45:02:45,  9.10s/it]
 92%|█████████▏| 207185/225000 [398:56:18<40:42:16,  8.23s/it]
 92%|█████████▏| 207186/225000 [398:56:24<37:20:51,  7.55s/it]
 92%|█████████▏| 207187/225000 [398:56:36<43:47:12,  8.85s/it]
 92%|█████████▏| 207188/225000 [398:56:46<46:27:35,  9.39s/it]
 92%|█████████▏| 207189/225000 [398:56:54<44:45:10,  9.05s/it]
 92%|█████████▏| 207190/225000 [398:57:00<39:42:34,  8.03s/it]
 92%|█████████▏| 207191/225000 [398:57:14<49:06:09,  9.93s/it]
 92%|█████████▏| 207192/225000 [398:57:23<46:49:28,  9.47s/it]
 92%|█████████▏| 207193/225000 [398:57:28<41:01:38,  8.29s/it]
 92%|█████████▏| 207194/225000 [398:57:39<44:14:07,  8.94s/it]
 92%|█████████▏| 207195/225000 [398:57:45<40:41:13,  8.23s/it]
 92%|█████████▏| 207196/225000 [398:57:55<43:28:42,  8.79s/it]
 92%|█████████▏| 207197/225000 [398:58:03<41:56:11,  8.48s/it]
 92%|█████████▏| 207198/225000 [398:58:17<49:09:04,  9.94s/it]
 92%|█████████▏| 207199/225000 [398:58:25<47:00:02,  9.51s/it]
 92%|█████████▏| 207200/225000 [398:58:35<47:54:10,  9.69s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.279', 'learning_rate': '8.5e-07', 'epoch': '0.9209', 'num_input_tokens_seen': 673225168, 'train_runtime': '1.436e+06', 'train_tokens_per_second': '468.7'}

 92%|█████████▏| 207200/225000 [398:58:35<47:54:10,  9.69s/it]
 92%|█████████▏| 207201/225000 [398:58:41<42:34:22,  8.61s/it]
 92%|█████████▏| 207202/225000 [398:58:49<40:48:12,  8.25s/it]
 92%|█████████▏| 207203/225000 [398:59:01<46:56:54,  9.50s/it]
 92%|█████████▏| 207204/225000 [398:59:11<47:31:13,  9.61s/it]
 92%|█████████▏| 207205/225000 [398:59:21<48:06:33,  9.73s/it]
 92%|█████████▏| 207206/225000 [398:59:28<43:20:03,  8.77s/it]
 92%|█████████▏| 207207/225000 [398:59:39<47:23:23,  9.59s/it]
 92%|█████████▏| 207208/225000 [398:59:48<46:59:27,  9.51s/it]
 92%|█████████▏| 207209/225000 [398:59:56<43:33:05,  8.81s/it]
 92%|█████████▏| 207210/225000 [399:00:06<46:00:20,  9.31s/it]
 92%|█████████▏| 207211/225000 [399:00:11<39:15:10,  7.94s/it]
 92%|█████████▏| 207212/225000 [399:00:21<43:11:17,  8.74s/it]
 92%|█████████▏| 207213/225000 [399:00:32<45:43:56,  9.26s/it]
 92%|█████████▏| 207214/225000 [399:00:38<40:55:29,  8.28s/it]
 92%|█████████▏| 207215/225000 [399:00:46<40:27:39,  8.19s/it]
 92%|█████████▏| 207216/225000 [399:00:56<43:39:30,  8.84s/it]
 92%|█████████▏| 207217/225000 [399:01:06<44:23:52,  8.99s/it]
 92%|█████████▏| 207218/225000 [399:01:18<50:07:29, 10.15s/it]
 92%|█████████▏| 207219/225000 [399:01:23<42:27:24,  8.60s/it]
 92%|█████████▏| 207220/225000 [399:01:33<43:40:47,  8.84s/it]
 92%|█████████▏| 207221/225000 [399:01:43<45:07:52,  9.14s/it]
 92%|█████████▏| 207222/225000 [399:01:47<38:11:23,  7.73s/it]
 92%|█████████▏| 207223/225000 [399:01:55<38:15:52,  7.75s/it]
 92%|█████████▏| 207224/225000 [399:02:03<38:24:50,  7.78s/it]
 92%|█████████▏| 207225/225000 [399:02:10<38:07:09,  7.72s/it]
 92%|█████████▏| 207226/225000 [399:02:22<44:14:44,  8.96s/it]
 92%|█████████▏| 207227/225000 [399:02:30<42:37:27,  8.63s/it]
 92%|█████████▏| 207228/225000 [399:02:39<42:33:35,  8.62s/it]
 92%|█████████▏| 207229/225000 [399:02:44<38:33:17,  7.81s/it]
 92%|█████████▏| 207230/225000 [399:02:51<36:42:02,  7.44s/it]
 92%|█████████▏| 207231/225000 [399:02:59<38:09:33,  7.73s/it]
 92%|█████████▏| 207232/225000 [399:03:14<48:09:08,  9.76s/it]
 92%|█████████▏| 207233/225000 [399:03:23<47:13:33,  9.57s/it]
 92%|█████████▏| 207234/225000 [399:03:31<44:38:22,  9.05s/it]
 92%|█████████▏| 207235/225000 [399:03:40<44:11:12,  8.95s/it]
 92%|█████████▏| 207236/225000 [399:03:48<43:19:11,  8.78s/it]
 92%|█████████▏| 207237/225000 [399:03:58<45:48:01,  9.28s/it]
 92%|█████████▏| 207238/225000 [399:04:05<42:07:46,  8.54s/it]
 92%|█████████▏| 207239/225000 [399:04:10<37:03:03,  7.51s/it]
 92%|█████████▏| 207240/225000 [399:04:21<41:36:20,  8.43s/it]
 92%|█████████▏| 207241/225000 [399:04:31<43:13:56,  8.76s/it]
 92%|█████████▏| 207242/225000 [399:04:40<43:35:34,  8.84s/it]
 92%|█████████▏| 207243/225000 [399:04:51<47:24:42,  9.61s/it]
 92%|█████████▏| 207244/225000 [399:04:56<40:40:04,  8.25s/it]
 92%|█████████▏| 207245/225000 [399:05:06<43:36:36,  8.84s/it]
 92%|█████████▏| 207246/225000 [399:05:14<41:36:29,  8.44s/it]
 92%|█████████▏| 207247/225000 [399:05:23<42:57:25,  8.71s/it]
 92%|█████████▏| 207248/225000 [399:05:33<45:16:05,  9.18s/it]
 92%|█████████▏| 207249/225000 [399:05:39<40:38:35,  8.24s/it]
 92%|█████████▏| 207250/225000 [399:05:51<46:04:56,  9.35s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.313', 'learning_rate': '8.453e-07', 'epoch': '0.9211', 'num_input_tokens_seen': 673459864, 'train_runtime': '1.437e+06', 'train_tokens_per_second': '468.7'}

 92%|█████████▏| 207250/225000 [399:05:51<46:04:56,  9.35s/it]
 92%|█████████▏| 207251/225000 [399:05:58<41:45:29,  8.47s/it]
 92%|█████████▏| 207252/225000 [399:06:03<37:00:11,  7.51s/it]
 92%|█████████▏| 207253/225000 [399:06:16<44:44:35,  9.08s/it]
 92%|█████████▏| 207254/225000 [399:06:26<46:03:14,  9.34s/it]
 92%|█████████▏| 207255/225000 [399:06:36<47:16:21,  9.59s/it]
 92%|█████████▏| 207256/225000 [399:06:44<45:10:08,  9.16s/it]
 92%|█████████▏| 207257/225000 [399:06:56<49:29:40, 10.04s/it]
 92%|█████████▏| 207258/225000 [399:07:01<42:20:31,  8.59s/it]
 92%|█████████▏| 207259/225000 [399:07:09<41:06:21,  8.34s/it]
 92%|█████████▏| 207260/225000 [399:07:24<50:15:46, 10.20s/it]
 92%|█████████▏| 207261/225000 [399:07:34<50:34:45, 10.26s/it]
 92%|█████████▏| 207262/225000 [399:07:41<45:27:08,  9.22s/it]
 92%|█████████▏| 207263/225000 [399:07:50<44:51:34,  9.10s/it]
 92%|█████████▏| 207264/225000 [399:07:58<44:01:01,  8.93s/it]
 92%|█████████▏| 207265/225000 [399:08:08<44:46:21,  9.09s/it]
 92%|█████████▏| 207266/225000 [399:08:13<39:34:57,  8.04s/it]
 92%|█████████▏| 207267/225000 [399:08:22<40:20:46,  8.19s/it]
 92%|█████████▏| 207268/225000 [399:08:29<39:09:19,  7.95s/it]
 92%|█████████▏| 207269/225000 [399:08:36<37:21:50,  7.59s/it]
 92%|█████████▏| 207270/225000 [399:08:44<37:33:28,  7.63s/it]
 92%|█████████▏| 207271/225000 [399:09:01<51:15:04, 10.41s/it]
 92%|█████████▏| 207272/225000 [399:09:10<49:41:34, 10.09s/it]
 92%|█████████▏| 207273/225000 [399:09:18<47:01:47,  9.55s/it]
 92%|█████████▏| 207274/225000 [399:09:23<40:07:19,  8.15s/it]
 92%|█████████▏| 207275/225000 [399:09:31<40:11:38,  8.16s/it]
 92%|█████████▏| 207276/225000 [399:09:36<35:01:33,  7.11s/it]
 92%|█████████▏| 207277/225000 [399:09:45<37:35:33,  7.64s/it]
 92%|█████████▏| 207278/225000 [399:09:51<34:53:39,  7.09s/it]
 92%|█████████▏| 207279/225000 [399:09:59<36:46:40,  7.47s/it]
 92%|█████████▏| 207280/225000 [399:10:09<39:56:39,  8.12s/it]
 92%|█████████▏| 207281/225000 [399:10:16<38:39:38,  7.85s/it]
 92%|█████████▏| 207282/225000 [399:10:24<38:25:08,  7.81s/it]
 92%|█████████▏| 207283/225000 [399:10:32<39:07:09,  7.95s/it]
 92%|█████████▏| 207284/225000 [399:10:40<39:21:38,  8.00s/it]
 92%|█████████▏| 207285/225000 [399:10:53<46:26:50,  9.44s/it]
 92%|█████████▏| 207286/225000 [399:11:04<48:57:34,  9.95s/it]
 92%|█████████▏| 207287/225000 [399:11:11<45:24:49,  9.23s/it]
 92%|█████████▏| 207288/225000 [399:11:21<45:42:19,  9.29s/it]
 92%|█████████▏| 207289/225000 [399:11:35<52:35:03, 10.69s/it]
 92%|█████████▏| 207290/225000 [399:11:41<45:20:28,  9.22s/it]
 92%|█████████▏| 207291/225000 [399:11:46<39:51:23,  8.10s/it]
 92%|█████████▏| 207292/225000 [399:11:58<46:10:33,  9.39s/it]
 92%|█████████▏| 207293/225000 [399:12:08<47:03:20,  9.57s/it]
 92%|█████████▏| 207294/225000 [399:12:17<45:11:39,  9.19s/it]
 92%|█████████▏| 207295/225000 [399:12:27<47:17:36,  9.62s/it]
 92%|█████████▏| 207296/225000 [399:12:37<47:16:46,  9.61s/it]
 92%|█████████▏| 207297/225000 [399:12:42<39:56:16,  8.12s/it]
 92%|█████████▏| 207298/225000 [399:12:47<36:34:07,  7.44s/it]
 92%|█████████▏| 207299/225000 [399:12:53<33:19:59,  6.78s/it]
 92%|█████████▏| 207300/225000 [399:13:04<40:39:53,  8.27s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.969', 'learning_rate': '8.406e-07', 'epoch': '0.9213', 'num_input_tokens_seen': 673692624, 'train_runtime': '1.437e+06', 'train_tokens_per_second': '468.8'}

 92%|█████████▏| 207300/225000 [399:13:04<40:39:53,  8.27s/it]
 92%|█████████▏| 207301/225000 [399:13:13<41:19:15,  8.40s/it]
 92%|█████████▏| 207302/225000 [399:13:24<44:53:12,  9.13s/it]
 92%|█████████▏| 207303/225000 [399:13:34<46:28:43,  9.45s/it]
 92%|█████████▏| 207304/225000 [399:13:45<49:11:51, 10.01s/it]
 92%|█████████▏| 207305/225000 [399:13:57<51:02:16, 10.38s/it]
 92%|█████████▏| 207306/225000 [399:14:04<47:05:43,  9.58s/it]
 92%|█████████▏| 207307/225000 [399:14:17<51:01:36, 10.38s/it]
 92%|█████████▏| 207308/225000 [399:14:25<47:35:21,  9.68s/it]
 92%|█████████▏| 207309/225000 [399:14:34<46:28:50,  9.46s/it]
 92%|█████████▏| 207310/225000 [399:14:40<41:42:17,  8.49s/it]
 92%|█████████▏| 207311/225000 [399:14:50<43:31:56,  8.86s/it]
 92%|█████████▏| 207312/225000 [399:14:56<40:30:55,  8.25s/it]
 92%|█████████▏| 207313/225000 [399:15:04<40:09:52,  8.18s/it]
 92%|█████████▏| 207314/225000 [399:15:16<45:07:06,  9.18s/it]
 92%|█████████▏| 207315/225000 [399:15:26<46:06:22,  9.39s/it]
 92%|█████████▏| 207316/225000 [399:15:37<48:33:52,  9.89s/it]
 92%|█████████▏| 207317/225000 [399:15:47<48:34:25,  9.89s/it]
 92%|█████████▏| 207318/225000 [399:15:55<46:09:46,  9.40s/it]
 92%|█████████▏| 207319/225000 [399:16:03<44:28:07,  9.05s/it]
 92%|█████████▏| 207320/225000 [399:16:12<43:26:30,  8.85s/it]
 92%|█████████▏| 207321/225000 [399:16:22<45:39:49,  9.30s/it]
 92%|█████████▏| 207322/225000 [399:16:33<47:59:08,  9.77s/it]
 92%|█████████▏| 207323/225000 [399:16:40<43:18:16,  8.82s/it]
 92%|█████████▏| 207324/225000 [399:16:50<45:05:31,  9.18s/it]
 92%|█████████▏| 207325/225000 [399:16:57<42:18:43,  8.62s/it]
 92%|█████████▏| 207326/225000 [399:17:07<45:02:00,  9.17s/it]
 92%|█████████▏| 207327/225000 [399:17:18<47:58:25,  9.77s/it]
 92%|█████████▏| 207328/225000 [399:17:25<42:39:10,  8.69s/it]
 92%|█████████▏| 207329/225000 [399:17:34<44:01:35,  8.97s/it]
 92%|█████████▏| 207330/225000 [399:17:40<38:49:33,  7.91s/it]
 92%|█████████▏| 207331/225000 [399:17:46<36:33:42,  7.45s/it]
 92%|█████████▏| 207332/225000 [399:17:57<42:03:23,  8.57s/it]
 92%|█████████▏| 207333/225000 [399:18:09<46:16:47,  9.43s/it]
 92%|█████████▏| 207334/225000 [399:18:17<44:09:21,  9.00s/it]
 92%|█████████▏| 207335/225000 [399:18:28<48:02:06,  9.79s/it]
 92%|█████████▏| 207336/225000 [399:18:34<41:50:55,  8.53s/it]
 92%|█████████▏| 207337/225000 [399:18:40<38:52:46,  7.92s/it]
 92%|█████████▏| 207338/225000 [399:18:53<45:47:54,  9.33s/it]
 92%|█████████▏| 207339/225000 [399:19:00<41:55:02,  8.54s/it]
 92%|█████████▏| 207340/225000 [399:19:06<39:14:19,  8.00s/it]
 92%|█████████▏| 207341/225000 [399:19:17<42:23:54,  8.64s/it]
 92%|█████████▏| 207342/225000 [399:19:25<41:59:07,  8.56s/it]
 92%|█████████▏| 207343/225000 [399:19:34<43:02:35,  8.78s/it]
 92%|█████████▏| 207344/225000 [399:19:44<44:25:44,  9.06s/it]
 92%|█████████▏| 207345/225000 [399:19:54<46:25:39,  9.47s/it]
 92%|█████████▏| 207346/225000 [399:20:04<46:59:19,  9.58s/it]
 92%|█████████▏| 207347/225000 [399:20:12<43:56:41,  8.96s/it]
 92%|█████████▏| 207348/225000 [399:20:20<43:15:29,  8.82s/it]
 92%|█████████▏| 207349/225000 [399:20:27<39:34:16,  8.07s/it]
 92%|█████████▏| 207350/225000 [399:20:39<45:43:36,  9.33s/it]
                                                              
{'loss': '1.335', 'grad_norm': '1.093', 'learning_rate': '8.359e-07', 'epoch': '0.9216', 'num_input_tokens_seen': 673940608, 'train_runtime': '1.438e+06', 'train_tokens_per_second': '468.8'}

 92%|█████████▏| 207350/225000 [399:20:39<45:43:36,  9.33s/it]
 92%|█████████▏| 207351/225000 [399:20:44<39:13:37,  8.00s/it]
 92%|█████████▏| 207352/225000 [399:20:53<41:11:21,  8.40s/it]
 92%|█████████▏| 207353/225000 [399:21:02<42:18:38,  8.63s/it]
 92%|█████████▏| 207354/225000 [399:21:11<41:50:21,  8.54s/it]
 92%|█████████▏| 207355/225000 [399:21:20<43:37:41,  8.90s/it]
 92%|█████████▏| 207356/225000 [399:21:24<36:01:59,  7.35s/it]
 92%|█████████▏| 207357/225000 [399:21:31<34:57:35,  7.13s/it]
 92%|█████████▏| 207358/225000 [399:21:36<32:23:32,  6.61s/it]
 92%|█████████▏| 207359/225000 [399:21:43<32:27:53,  6.63s/it]
 92%|█████████▏| 207360/225000 [399:21:50<34:07:37,  6.96s/it]
 92%|█████████▏| 207361/225000 [399:22:01<39:52:00,  8.14s/it]
 92%|█████████▏| 207362/225000 [399:22:10<41:02:56,  8.38s/it]
 92%|█████████▏| 207363/225000 [399:22:19<41:18:54,  8.43s/it]
 92%|█████████▏| 207364/225000 [399:22:31<46:30:38,  9.49s/it]
 92%|█████████▏| 207365/225000 [399:22:36<40:38:31,  8.30s/it]
 92%|█████████▏| 207366/225000 [399:22:46<42:27:27,  8.67s/it]
 92%|█████████▏| 207367/225000 [399:22:55<42:50:10,  8.75s/it]
 92%|█████████▏| 207368/225000 [399:23:01<39:06:30,  7.98s/it]
 92%|█████████▏| 207369/225000 [399:23:07<36:12:32,  7.39s/it]
 92%|█████████▏| 207370/225000 [399:23:15<37:10:38,  7.59s/it]
 92%|█████████▏| 207371/225000 [399:23:27<42:57:01,  8.77s/it]
 92%|█████████▏| 207372/225000 [399:23:35<42:56:53,  8.77s/it]
 92%|█████████▏| 207373/225000 [399:23:41<37:43:09,  7.70s/it]
 92%|█████████▏| 207374/225000 [399:23:54<45:57:46,  9.39s/it]
 92%|█████████▏| 207375/225000 [399:24:03<45:06:32,  9.21s/it]
 92%|█████████▏| 207376/225000 [399:24:11<43:11:36,  8.82s/it]
 92%|█████████▏| 207377/225000 [399:24:20<44:04:30,  9.00s/it]
 92%|█████████▏| 207378/225000 [399:24:27<41:42:27,  8.52s/it]
 92%|█████████▏| 207379/225000 [399:24:35<40:46:37,  8.33s/it]
 92%|█████████▏| 207380/225000 [399:24:47<45:33:04,  9.31s/it]
 92%|█████████▏| 207381/225000 [399:24:56<45:29:46,  9.30s/it]
 92%|█████████▏| 207382/225000 [399:25:04<43:15:42,  8.84s/it]
 92%|█████████▏| 207383/225000 [399:25:13<43:14:05,  8.83s/it]
 92%|█████████▏| 207384/225000 [399:25:21<41:43:59,  8.53s/it]
 92%|█████████▏| 207385/225000 [399:25:35<49:39:34, 10.15s/it]
 92%|█████████▏| 207386/225000 [399:25:40<43:14:11,  8.84s/it]
 92%|█████████▏| 207387/225000 [399:25:50<44:44:15,  9.14s/it]
 92%|█████████▏| 207388/225000 [399:25:55<38:38:09,  7.90s/it]
 92%|█████████▏| 207389/225000 [399:26:05<42:02:25,  8.59s/it]
 92%|█████████▏| 207390/225000 [399:26:10<36:54:03,  7.54s/it]
 92%|█████████▏| 207391/225000 [399:26:16<33:47:04,  6.91s/it]
 92%|█████████▏| 207392/225000 [399:26:24<35:16:53,  7.21s/it]
 92%|█████████▏| 207393/225000 [399:26:34<39:33:08,  8.09s/it]
 92%|█████████▏| 207394/225000 [399:26:42<39:27:59,  8.07s/it]
 92%|█████████▏| 207395/225000 [399:26:50<39:16:14,  8.03s/it]
 92%|█████████▏| 207396/225000 [399:27:01<43:38:49,  8.93s/it]
 92%|█████████▏| 207397/225000 [399:27:09<41:45:06,  8.54s/it]
 92%|█████████▏| 207398/225000 [399:27:16<39:27:50,  8.07s/it]
 92%|█████████▏| 207399/225000 [399:27:23<39:02:32,  7.99s/it]
 92%|█████████▏| 207400/225000 [399:27:34<42:35:59,  8.71s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.244', 'learning_rate': '8.312e-07', 'epoch': '0.9218', 'num_input_tokens_seen': 674166576, 'train_runtime': '1.438e+06', 'train_tokens_per_second': '468.8'}

 92%|█████████▏| 207400/225000 [399:27:34<42:35:59,  8.71s/it]
 92%|█████████▏| 207401/225000 [399:27:40<39:19:22,  8.04s/it]
 92%|█████████▏| 207402/225000 [399:27:47<37:38:45,  7.70s/it]
 92%|█████████▏| 207403/225000 [399:28:00<45:57:52,  9.40s/it]
 92%|█████████▏| 207404/225000 [399:28:13<50:42:55, 10.38s/it]
 92%|█████████▏| 207405/225000 [399:28:21<47:30:36,  9.72s/it]
 92%|█████████▏| 207406/225000 [399:28:36<54:27:28, 11.14s/it]
 92%|█████████▏| 207407/225000 [399:28:41<46:22:11,  9.49s/it]
 92%|█████████▏| 207408/225000 [399:28:53<49:05:27, 10.05s/it]
 92%|█████████▏| 207409/225000 [399:29:00<44:46:18,  9.16s/it]
 92%|█████████▏| 207410/225000 [399:29:09<44:20:35,  9.08s/it]
 92%|█████████▏| 207411/225000 [399:29:21<49:31:57, 10.14s/it]
 92%|█████████▏| 207412/225000 [399:29:26<41:26:18,  8.48s/it]
 92%|█████████▏| 207413/225000 [399:29:39<47:34:46,  9.74s/it]
 92%|█████████▏| 207414/225000 [399:29:46<44:18:56,  9.07s/it]
 92%|█████████▏| 207415/225000 [399:29:52<39:32:41,  8.10s/it]
 92%|█████████▏| 207416/225000 [399:30:01<40:52:05,  8.37s/it]
 92%|█████████▏| 207417/225000 [399:30:07<37:36:19,  7.70s/it]
 92%|█████████▏| 207418/225000 [399:30:19<43:45:46,  8.96s/it]
 92%|█████████▏| 207419/225000 [399:30:25<39:50:29,  8.16s/it]
 92%|█████████▏| 207420/225000 [399:30:34<40:45:55,  8.35s/it]
 92%|█████████▏| 207421/225000 [399:30:41<38:05:03,  7.80s/it]
 92%|█████████▏| 207422/225000 [399:30:49<39:08:55,  8.02s/it]
 92%|█████████▏| 207423/225000 [399:31:03<47:29:52,  9.73s/it]
 92%|█████████▏| 207424/225000 [399:31:11<45:12:13,  9.26s/it]
 92%|█████████▏| 207425/225000 [399:31:19<43:20:46,  8.88s/it]
 92%|█████████▏| 207426/225000 [399:31:26<40:41:29,  8.34s/it]
 92%|█████████▏| 207427/225000 [399:31:36<43:24:49,  8.89s/it]
 92%|█████████▏| 207428/225000 [399:31:44<42:06:32,  8.63s/it]
 92%|█████████▏| 207429/225000 [399:31:54<43:47:26,  8.97s/it]
 92%|█████████▏| 207430/225000 [399:32:03<44:08:26,  9.04s/it]
 92%|█████████▏| 207431/225000 [399:32:09<38:58:29,  7.99s/it]
 92%|█████████▏| 207432/225000 [399:32:19<42:07:35,  8.63s/it]
 92%|█████████▏| 207433/225000 [399:32:26<39:42:25,  8.14s/it]
 92%|█████████▏| 207434/225000 [399:32:32<36:40:33,  7.52s/it]
 92%|█████████▏| 207435/225000 [399:32:46<45:41:42,  9.37s/it]
 92%|█████████▏| 207436/225000 [399:32:58<49:20:53, 10.11s/it]
 92%|█████████▏| 207437/225000 [399:33:02<41:17:27,  8.46s/it]
 92%|█████████▏| 207438/225000 [399:33:16<49:30:02, 10.15s/it]
 92%|█████████▏| 207439/225000 [399:33:24<45:42:21,  9.37s/it]
 92%|█████████▏| 207440/225000 [399:33:32<43:48:15,  8.98s/it]
 92%|█████████▏| 207441/225000 [399:33:39<41:27:08,  8.50s/it]
 92%|█████████▏| 207442/225000 [399:33:46<39:13:53,  8.04s/it]
 92%|█████████▏| 207443/225000 [399:33:58<45:27:08,  9.32s/it]
 92%|█████████▏| 207444/225000 [399:34:06<42:28:32,  8.71s/it]
 92%|█████████▏| 207445/225000 [399:34:17<45:46:48,  9.39s/it]
 92%|█████████▏| 207446/225000 [399:34:26<45:08:48,  9.26s/it]
 92%|█████████▏| 207447/225000 [399:34:33<42:20:43,  8.68s/it]
 92%|█████████▏| 207448/225000 [399:34:38<37:01:16,  7.59s/it]
 92%|█████████▏| 207449/225000 [399:34:47<38:30:37,  7.90s/it]
 92%|█████████▏| 207450/225000 [399:34:53<36:51:31,  7.56s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.233', 'learning_rate': '8.265e-07', 'epoch': '0.922', 'num_input_tokens_seen': 674404416, 'train_runtime': '1.438e+06', 'train_tokens_per_second': '468.8'}

 92%|█████████▏| 207450/225000 [399:34:53<36:51:31,  7.56s/it]
 92%|█████████▏| 207451/225000 [399:35:00<35:57:16,  7.38s/it]
 92%|█████████▏| 207452/225000 [399:35:13<42:50:46,  8.79s/it]
 92%|█████████▏| 207453/225000 [399:35:19<39:04:19,  8.02s/it]
 92%|█████████▏| 207454/225000 [399:35:25<36:35:15,  7.51s/it]
 92%|█████████▏| 207455/225000 [399:35:31<34:30:41,  7.08s/it]
 92%|█████████▏| 207456/225000 [399:35:41<38:01:12,  7.80s/it]
 92%|█████████▏| 207457/225000 [399:35:58<51:18:09, 10.53s/it]
 92%|█████████▏| 207458/225000 [399:36:03<43:30:27,  8.93s/it]
 92%|█████████▏| 207459/225000 [399:36:10<40:36:09,  8.33s/it]
 92%|█████████▏| 207460/225000 [399:36:16<38:13:11,  7.84s/it]
 92%|█████████▏| 207461/225000 [399:36:27<42:08:43,  8.65s/it]
 92%|█████████▏| 207462/225000 [399:36:34<39:35:57,  8.13s/it]
 92%|█████████▏| 207463/225000 [399:36:40<36:49:09,  7.56s/it]
 92%|█████████▏| 207464/225000 [399:36:48<38:06:52,  7.82s/it]
 92%|█████████▏| 207465/225000 [399:36:57<39:04:49,  8.02s/it]
 92%|█████████▏| 207466/225000 [399:37:04<37:02:25,  7.60s/it]
 92%|█████████▏| 207467/225000 [399:37:13<39:40:16,  8.15s/it]
 92%|█████████▏| 207468/225000 [399:37:19<36:14:34,  7.44s/it]
 92%|█████████▏| 207469/225000 [399:37:28<38:05:51,  7.82s/it]
 92%|█████████▏| 207470/225000 [399:37:37<40:35:30,  8.34s/it]
 92%|█████████▏| 207471/225000 [399:37:44<38:19:37,  7.87s/it]
 92%|█████████▏| 207472/225000 [399:37:52<39:07:47,  8.04s/it]
 92%|█████████▏| 207473/225000 [399:38:03<42:40:18,  8.76s/it]
 92%|█████████▏| 207474/225000 [399:38:08<37:45:01,  7.75s/it]
 92%|█████████▏| 207475/225000 [399:38:12<32:46:17,  6.73s/it]
 92%|█████████▏| 207476/225000 [399:38:22<36:29:48,  7.50s/it]
 92%|█████████▏| 207477/225000 [399:38:31<38:41:42,  7.95s/it]
 92%|█████████▏| 207478/225000 [399:38:41<42:31:20,  8.74s/it]
 92%|█████████▏| 207479/225000 [399:38:50<42:15:23,  8.68s/it]
 92%|█████████▏| 207480/225000 [399:38:58<41:48:35,  8.59s/it]
 92%|█████████▏| 207481/225000 [399:39:07<41:55:43,  8.62s/it]
 92%|█████████▏| 207482/225000 [399:39:16<42:20:36,  8.70s/it]
 92%|█████████▏| 207483/225000 [399:39:21<37:30:22,  7.71s/it]
 92%|█████████▏| 207484/225000 [399:39:37<49:02:34, 10.08s/it]
 92%|█████████▏| 207485/225000 [399:39:43<42:46:25,  8.79s/it]
 92%|█████████▏| 207486/225000 [399:39:53<44:25:36,  9.13s/it]
 92%|█████████▏| 207487/225000 [399:40:00<42:10:17,  8.67s/it]
 92%|█████████▏| 207488/225000 [399:40:12<46:34:28,  9.57s/it]
 92%|█████████▏| 207489/225000 [399:40:22<48:07:34,  9.89s/it]
 92%|█████████▏| 207490/225000 [399:40:32<47:40:56,  9.80s/it]
 92%|█████████▏| 207491/225000 [399:40:38<42:17:37,  8.70s/it]
 92%|█████████▏| 207492/225000 [399:40:44<38:42:46,  7.96s/it]
 92%|█████████▏| 207493/225000 [399:40:58<46:44:18,  9.61s/it]
 92%|█████████▏| 207494/225000 [399:41:05<42:55:27,  8.83s/it]
 92%|█████████▏| 207495/225000 [399:41:11<39:04:17,  8.04s/it]
 92%|█████████▏| 207496/225000 [399:41:20<40:18:26,  8.29s/it]
 92%|█████████▏| 207497/225000 [399:41:27<37:59:30,  7.81s/it]
 92%|█████████▏| 207498/225000 [399:41:32<34:20:17,  7.06s/it]
 92%|█████████▏| 207499/225000 [399:41:45<43:40:21,  8.98s/it]
 92%|█████████▏| 207500/225000 [399:41:55<45:00:52,  9.26s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.166', 'learning_rate': '8.219e-07', 'epoch': '0.9222', 'num_input_tokens_seen': 674633312, 'train_runtime': '1.439e+06', 'train_tokens_per_second': '468.8'}

 92%|█████████▏| 207500/225000 [399:41:55<45:00:52,  9.26s/it]
 92%|█████████▏| 207501/225000 [399:42:04<44:31:51,  9.16s/it]
 92%|█████████▏| 207502/225000 [399:42:12<42:04:44,  8.66s/it]
 92%|█████████▏| 207503/225000 [399:42:21<42:15:49,  8.70s/it]
 92%|█████████▏| 207504/225000 [399:42:27<39:16:51,  8.08s/it]
 92%|█████████▏| 207505/225000 [399:42:35<38:17:20,  7.88s/it]
 92%|█████████▏| 207506/225000 [399:42:42<37:15:46,  7.67s/it]
 92%|█████████▏| 207507/225000 [399:42:53<42:27:14,  8.74s/it]
 92%|█████████▏| 207508/225000 [399:42:59<38:38:58,  7.95s/it]
 92%|█████████▏| 207509/225000 [399:43:08<39:53:17,  8.21s/it]
 92%|█████████▏| 207510/225000 [399:43:21<47:16:47,  9.73s/it]
 92%|█████████▏| 207511/225000 [399:43:30<45:24:48,  9.35s/it]
 92%|█████████▏| 207512/225000 [399:43:41<47:41:29,  9.82s/it]
 92%|█████████▏| 207513/225000 [399:43:48<44:48:37,  9.22s/it]
 92%|█████████▏| 207514/225000 [399:43:57<43:54:01,  9.04s/it]
 92%|█████████▏| 207515/225000 [399:44:06<44:21:31,  9.13s/it]
 92%|█████████▏| 207516/225000 [399:44:12<38:51:03,  8.00s/it]
 92%|█████████▏| 207517/225000 [399:44:18<36:25:18,  7.50s/it]
 92%|█████████▏| 207518/225000 [399:44:28<39:41:14,  8.17s/it]
 92%|█████████▏| 207519/225000 [399:44:36<39:19:58,  8.10s/it]
 92%|█████████▏| 207520/225000 [399:44:41<35:28:32,  7.31s/it]
 92%|█████████▏| 207521/225000 [399:44:48<34:57:52,  7.20s/it]
 92%|█████████▏| 207522/225000 [399:44:57<38:02:53,  7.84s/it]
 92%|█████████▏| 207523/225000 [399:45:06<39:16:21,  8.09s/it]
 92%|█████████▏| 207524/225000 [399:45:16<41:31:48,  8.56s/it]
 92%|█████████▏| 207525/225000 [399:45:24<40:58:53,  8.44s/it]
 92%|█████████▏| 207526/225000 [399:45:30<37:01:33,  7.63s/it]
 92%|█████████▏| 207527/225000 [399:45:42<43:38:31,  8.99s/it]
 92%|█████████▏| 207528/225000 [399:45:50<42:22:11,  8.73s/it]
 92%|█████████▏| 207529/225000 [399:45:57<39:19:43,  8.10s/it]
 92%|█████████▏| 207530/225000 [399:46:08<44:04:26,  9.08s/it]
 92%|█████████▏| 207531/225000 [399:46:13<38:34:34,  7.95s/it]
 92%|█████████▏| 207532/225000 [399:46:19<35:19:30,  7.28s/it]
 92%|█████████▏| 207533/225000 [399:46:28<38:20:58,  7.90s/it]
 92%|█████████▏| 207534/225000 [399:46:35<36:58:41,  7.62s/it]
 92%|█████████▏| 207535/225000 [399:46:42<36:16:36,  7.48s/it]
 92%|█████████▏| 207536/225000 [399:46:52<38:37:53,  7.96s/it]
 92%|█████████▏| 207537/225000 [399:47:02<42:34:48,  8.78s/it]
 92%|█████████▏| 207538/225000 [399:47:10<41:09:03,  8.48s/it]
 92%|█████████▏| 207539/225000 [399:47:16<37:27:33,  7.72s/it]
 92%|█████████▏| 207540/225000 [399:47:31<48:40:59, 10.04s/it]
 92%|█████████▏| 207541/225000 [399:47:37<41:51:08,  8.63s/it]
 92%|█████████▏| 207542/225000 [399:47:44<39:33:11,  8.16s/it]
 92%|█████████▏| 207543/225000 [399:47:51<38:03:22,  7.85s/it]
 92%|█████████▏| 207544/225000 [399:47:59<39:01:34,  8.05s/it]
 92%|█████████▏| 207545/225000 [399:48:04<34:29:12,  7.11s/it]
 92%|█████████▏| 207546/225000 [399:48:10<31:40:55,  6.53s/it]
 92%|█████████▏| 207547/225000 [399:48:15<30:18:44,  6.25s/it]
 92%|█████████▏| 207548/225000 [399:48:27<38:08:44,  7.87s/it]
 92%|█████████▏| 207549/225000 [399:48:36<40:46:15,  8.41s/it]
 92%|█████████▏| 207550/225000 [399:48:44<39:21:29,  8.12s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.518', 'learning_rate': '8.173e-07', 'epoch': '0.9224', 'num_input_tokens_seen': 674855136, 'train_runtime': '1.439e+06', 'train_tokens_per_second': '468.9'}

 92%|█████████▏| 207550/225000 [399:48:44<39:21:29,  8.12s/it]
 92%|█████████▏| 207551/225000 [399:48:53<41:22:34,  8.54s/it]
 92%|█████████▏| 207552/225000 [399:49:01<40:29:55,  8.36s/it]
 92%|█████████▏| 207553/225000 [399:49:11<42:59:06,  8.87s/it]
 92%|█████████▏| 207554/225000 [399:49:20<42:29:41,  8.77s/it]
 92%|█████████▏| 207555/225000 [399:49:33<48:30:37, 10.01s/it]
 92%|█████████▏| 207556/225000 [399:49:42<47:50:04,  9.87s/it]
 92%|█████████▏| 207557/225000 [399:49:49<42:47:47,  8.83s/it]
 92%|█████████▏| 207558/225000 [399:49:58<43:26:54,  8.97s/it]
 92%|█████████▏| 207559/225000 [399:50:07<42:47:58,  8.83s/it]
 92%|█████████▏| 207560/225000 [399:50:14<41:07:09,  8.49s/it]
 92%|█████████▏| 207561/225000 [399:50:22<40:01:32,  8.26s/it]
 92%|█████████▏| 207562/225000 [399:50:35<46:55:13,  9.69s/it]
 92%|█████████▏| 207563/225000 [399:50:40<40:33:59,  8.38s/it]
 92%|█████████▏| 207564/225000 [399:50:47<37:38:35,  7.77s/it]
 92%|█████████▏| 207565/225000 [399:50:54<37:13:00,  7.68s/it]
 92%|█████████▏| 207566/225000 [399:51:06<43:49:05,  9.05s/it]
 92%|█████████▏| 207567/225000 [399:51:14<41:09:03,  8.50s/it]
 92%|█████████▏| 207568/225000 [399:51:23<41:53:57,  8.65s/it]
 92%|█████████▏| 207569/225000 [399:51:37<50:01:40, 10.33s/it]
 92%|█████████▏| 207570/225000 [399:51:45<46:44:29,  9.65s/it]
 92%|█████████▏| 207571/225000 [399:51:56<49:20:57, 10.19s/it]
 92%|█████████▏| 207572/225000 [399:52:08<50:55:34, 10.52s/it]
 92%|█████████▏| 207573/225000 [399:52:14<44:19:12,  9.16s/it]
 92%|█████████▏| 207574/225000 [399:52:21<41:06:49,  8.49s/it]
 92%|█████████▏| 207575/225000 [399:52:28<39:57:35,  8.26s/it]
 92%|█████████▏| 207576/225000 [399:52:41<46:33:52,  9.62s/it]
 92%|█████████▏| 207577/225000 [399:52:48<42:34:44,  8.80s/it]
 92%|█████████▏| 207578/225000 [399:52:53<36:56:20,  7.63s/it]
 92%|█████████▏| 207579/225000 [399:53:00<35:31:24,  7.34s/it]
 92%|█████████▏| 207580/225000 [399:53:13<43:45:08,  9.04s/it]
 92%|█████████▏| 207581/225000 [399:53:18<38:05:01,  7.87s/it]
 92%|█████████▏| 207582/225000 [399:53:29<42:34:44,  8.80s/it]
 92%|█████████▏| 207583/225000 [399:53:40<46:15:18,  9.56s/it]
 92%|█████████▏| 207584/225000 [399:53:49<45:56:37,  9.50s/it]
 92%|█████████▏| 207585/225000 [399:53:57<42:45:30,  8.84s/it]
 92%|█████████▏| 207586/225000 [399:54:03<38:49:47,  8.03s/it]
 92%|█████████▏| 207587/225000 [399:54:09<35:59:50,  7.44s/it]
 92%|█████████▏| 207588/225000 [399:54:15<34:35:33,  7.15s/it]
 92%|█████████▏| 207589/225000 [399:54:27<41:12:35,  8.52s/it]
 92%|█████████▏| 207590/225000 [399:54:34<39:05:12,  8.08s/it]
 92%|█████████▏| 207591/225000 [399:54:48<47:08:10,  9.75s/it]
 92%|█████████▏| 207592/225000 [399:54:55<43:18:18,  8.96s/it]
 92%|█████████▏| 207593/225000 [399:55:04<42:48:03,  8.85s/it]
 92%|█████████▏| 207594/225000 [399:55:11<41:02:07,  8.49s/it]
 92%|█████████▏| 207595/225000 [399:55:18<37:59:12,  7.86s/it]
 92%|█████████▏| 207596/225000 [399:55:29<42:57:10,  8.88s/it]
 92%|█████████▏| 207597/225000 [399:55:34<38:00:24,  7.86s/it]
 92%|█████████▏| 207598/225000 [399:55:41<36:14:26,  7.50s/it]
 92%|█████████▏| 207599/225000 [399:55:47<33:33:32,  6.94s/it]
 92%|█████████▏| 207600/225000 [399:55:54<34:49:12,  7.20s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.015', 'learning_rate': '8.126e-07', 'epoch': '0.9227', 'num_input_tokens_seen': 675085576, 'train_runtime': '1.44e+06', 'train_tokens_per_second': '468.9'}

 92%|█████████▏| 207600/225000 [399:55:54<34:49:12,  7.20s/it]
 92%|█████████▏| 207601/225000 [399:56:07<42:24:18,  8.77s/it]
 92%|█████████▏| 207602/225000 [399:56:14<39:51:08,  8.25s/it]
 92%|█████████▏| 207603/225000 [399:56:23<41:36:43,  8.61s/it]
 92%|█████████▏| 207604/225000 [399:56:36<47:31:23,  9.83s/it]
 92%|█████████▏| 207605/225000 [399:56:46<48:22:36, 10.01s/it]
 92%|█████████▏| 207606/225000 [399:56:56<47:40:54,  9.87s/it]
 92%|█████████▏| 207607/225000 [399:57:02<42:10:04,  8.73s/it]
 92%|█████████▏| 207608/225000 [399:57:09<40:09:19,  8.31s/it]
 92%|█████████▏| 207609/225000 [399:57:18<40:24:53,  8.37s/it]
 92%|█████████▏| 207610/225000 [399:57:30<45:28:24,  9.41s/it]
 92%|█████████▏| 207611/225000 [399:57:38<44:19:44,  9.18s/it]
 92%|█████████▏| 207612/225000 [399:57:49<45:47:49,  9.48s/it]
 92%|█████████▏| 207613/225000 [399:57:55<41:30:47,  8.60s/it]
 92%|█████████▏| 207614/225000 [399:58:05<43:31:41,  9.01s/it]
 92%|█████████▏| 207615/225000 [399:58:14<43:06:43,  8.93s/it]
 92%|█████████▏| 207616/225000 [399:58:21<40:58:08,  8.48s/it]
 92%|█████████▏| 207617/225000 [399:58:34<46:42:48,  9.67s/it]
 92%|█████████▏| 207618/225000 [399:58:40<42:18:38,  8.76s/it]
 92%|█████████▏| 207619/225000 [399:58:49<42:13:40,  8.75s/it]
 92%|█████████▏| 207620/225000 [399:58:59<44:04:32,  9.13s/it]
 92%|█████████▏| 207621/225000 [399:59:07<42:32:19,  8.81s/it]
 92%|█████████▏| 207622/225000 [399:59:16<42:31:33,  8.81s/it]
 92%|█████████▏| 207623/225000 [399:59:24<41:34:02,  8.61s/it]
 92%|█████████▏| 207624/225000 [399:59:30<38:09:20,  7.91s/it]
 92%|█████████▏| 207625/225000 [399:59:38<37:35:05,  7.79s/it]
 92%|█████████▏| 207626/225000 [399:59:51<45:23:54,  9.41s/it]
 92%|█████████▏| 207627/225000 [399:59:59<43:45:20,  9.07s/it]
 92%|█████████▏| 207628/225000 [400:00:09<45:07:01,  9.35s/it]
 92%|█████████▏| 207629/225000 [400:00:14<38:20:54,  7.95s/it]
 92%|█████████▏| 207630/225000 [400:00:20<35:20:54,  7.33s/it]
 92%|█████████▏| 207631/225000 [400:00:26<33:39:56,  6.98s/it]
 92%|█████████▏| 207632/225000 [400:00:33<33:25:13,  6.93s/it]
 92%|█████████▏| 207633/225000 [400:00:44<39:39:35,  8.22s/it]
 92%|█████████▏| 207634/225000 [400:00:55<43:37:30,  9.04s/it]
 92%|█████████▏| 207635/225000 [400:01:01<39:15:25,  8.14s/it]
 92%|█████████▏| 207636/225000 [400:01:08<36:55:06,  7.65s/it]
 92%|█████████▏| 207637/225000 [400:01:18<41:22:15,  8.58s/it]
 92%|█████████▏| 207638/225000 [400:01:28<42:58:00,  8.91s/it]
 92%|█████████▏| 207639/225000 [400:01:35<40:18:55,  8.36s/it]
 92%|█████████▏| 207640/225000 [400:01:44<40:39:19,  8.43s/it]
 92%|█████████▏| 207641/225000 [400:01:49<36:06:47,  7.49s/it]
 92%|█████████▏| 207642/225000 [400:01:58<37:42:22,  7.82s/it]
 92%|█████████▏| 207643/225000 [400:02:09<42:25:54,  8.80s/it]
 92%|█████████▏| 207644/225000 [400:02:19<45:04:29,  9.35s/it]
 92%|█████████▏| 207645/225000 [400:02:30<47:06:33,  9.77s/it]
 92%|█████████▏| 207646/225000 [400:02:41<49:10:44, 10.20s/it]
 92%|█████████▏| 207647/225000 [400:02:47<42:47:46,  8.88s/it]
 92%|█████████▏| 207648/225000 [400:02:59<47:54:49,  9.94s/it]
 92%|█████████▏| 207649/225000 [400:03:05<41:34:31,  8.63s/it]
 92%|█████████▏| 207650/225000 [400:03:12<38:42:06,  8.03s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.479', 'learning_rate': '8.08e-07', 'epoch': '0.9229', 'num_input_tokens_seen': 675318720, 'train_runtime': '1.44e+06', 'train_tokens_per_second': '468.9'}

 92%|█████████▏| 207650/225000 [400:03:12<38:42:06,  8.03s/it]
 92%|█████████▏| 207651/225000 [400:03:27<49:28:39, 10.27s/it]
 92%|█████████▏| 207652/225000 [400:03:36<48:03:22,  9.97s/it]
 92%|█████████▏| 207653/225000 [400:03:45<45:53:31,  9.52s/it]
 92%|█████████▏| 207654/225000 [400:03:50<40:07:59,  8.33s/it]
 92%|█████████▏| 207655/225000 [400:03:59<39:52:20,  8.28s/it]
 92%|█████████▏| 207656/225000 [400:04:11<45:13:38,  9.39s/it]
 92%|█████████▏| 207657/225000 [400:04:19<43:26:14,  9.02s/it]
 92%|█████████▏| 207658/225000 [400:04:28<43:22:45,  9.01s/it]
 92%|█████████▏| 207659/225000 [400:04:37<44:24:36,  9.22s/it]
 92%|█████████▏| 207660/225000 [400:04:45<42:37:55,  8.85s/it]
 92%|█████████▏| 207661/225000 [400:04:55<43:06:05,  8.95s/it]
 92%|█████████▏| 207662/225000 [400:05:04<42:58:42,  8.92s/it]
 92%|█████████▏| 207663/225000 [400:05:10<38:51:20,  8.07s/it]
 92%|█████████▏| 207664/225000 [400:05:22<45:23:37,  9.43s/it]
 92%|█████████▏| 207665/225000 [400:05:34<49:02:53, 10.19s/it]
 92%|█████████▏| 207666/225000 [400:05:41<43:46:37,  9.09s/it]
 92%|█████████▏| 207667/225000 [400:05:48<41:20:08,  8.59s/it]
 92%|█████████▏| 207668/225000 [400:05:54<37:51:17,  7.86s/it]
 92%|█████████▏| 207669/225000 [400:06:03<39:17:48,  8.16s/it]
 92%|█████████▏| 207670/225000 [400:06:08<35:15:54,  7.33s/it]
 92%|█████████▏| 207671/225000 [400:06:15<33:27:53,  6.95s/it]
 92%|█████████▏| 207672/225000 [400:06:26<39:44:20,  8.26s/it]
 92%|█████████▏| 207673/225000 [400:06:37<44:12:20,  9.18s/it]
 92%|█████████▏| 207674/225000 [400:06:45<42:27:10,  8.82s/it]
 92%|█████████▏| 207675/225000 [400:06:56<44:47:50,  9.31s/it]
 92%|█████████▏| 207676/225000 [400:07:07<47:39:29,  9.90s/it]
 92%|█████████▏| 207677/225000 [400:07:13<42:29:33,  8.83s/it]
 92%|█████████▏| 207678/225000 [400:07:19<38:01:54,  7.90s/it]
 92%|█████████▏| 207679/225000 [400:07:31<44:21:33,  9.22s/it]
 92%|█████████▏| 207680/225000 [400:07:36<38:14:39,  7.95s/it]
 92%|█████████▏| 207681/225000 [400:07:48<43:16:01,  8.99s/it]
 92%|█████████▏| 207682/225000 [400:07:55<40:23:27,  8.40s/it]
 92%|█████████▏| 207683/225000 [400:08:03<39:32:37,  8.22s/it]
 92%|█████████▏| 207684/225000 [400:08:13<42:42:59,  8.88s/it]
 92%|█████████▏| 207685/225000 [400:08:20<39:56:23,  8.30s/it]
 92%|█████████▏| 207686/225000 [400:08:26<36:59:39,  7.69s/it]
 92%|█████████▏| 207687/225000 [400:08:38<43:39:48,  9.08s/it]
 92%|█████████▏| 207688/225000 [400:08:49<45:20:20,  9.43s/it]
 92%|█████████▏| 207689/225000 [400:08:56<42:09:59,  8.77s/it]
 92%|█████████▏| 207690/225000 [400:09:02<38:02:24,  7.91s/it]
 92%|█████████▏| 207691/225000 [400:09:12<41:48:20,  8.69s/it]
 92%|█████████▏| 207692/225000 [400:09:22<43:04:06,  8.96s/it]
 92%|█████████▏| 207693/225000 [400:09:32<44:42:35,  9.30s/it]
 92%|█████████▏| 207694/225000 [400:09:38<39:46:01,  8.27s/it]
 92%|█████████▏| 207695/225000 [400:09:49<44:05:42,  9.17s/it]
 92%|█████████▏| 207696/225000 [400:09:59<44:51:17,  9.33s/it]
 92%|█████████▏| 207697/225000 [400:10:05<39:58:02,  8.32s/it]
 92%|█████████▏| 207698/225000 [400:10:14<40:48:10,  8.49s/it]
 92%|█████████▏| 207699/225000 [400:10:21<38:49:23,  8.08s/it]
 92%|█████████▏| 207700/225000 [400:10:28<37:27:01,  7.79s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.143', 'learning_rate': '8.034e-07', 'epoch': '0.9231', 'num_input_tokens_seen': 675551872, 'train_runtime': '1.441e+06', 'train_tokens_per_second': '468.9'}

 92%|█████████▏| 207700/225000 [400:10:28<37:27:01,  7.79s/it]
 92%|█████████▏| 207701/225000 [400:10:38<40:38:28,  8.46s/it]
 92%|█████████▏| 207702/225000 [400:10:49<43:45:00,  9.11s/it]
 92%|█████████▏| 207703/225000 [400:10:54<38:36:50,  8.04s/it]
 92%|█████████▏| 207704/225000 [400:11:05<42:20:16,  8.81s/it]
 92%|█████████▏| 207705/225000 [400:11:14<43:26:11,  9.04s/it]
 92%|█████████▏| 207706/225000 [400:11:26<47:17:54,  9.85s/it]
 92%|█████████▏| 207707/225000 [400:11:32<42:15:47,  8.80s/it]
 92%|█████████▏| 207708/225000 [400:11:38<37:31:31,  7.81s/it]
 92%|█████████▏| 207709/225000 [400:11:48<41:15:58,  8.59s/it]
 92%|█████████▏| 207710/225000 [400:11:56<39:44:40,  8.28s/it]
 92%|█████████▏| 207711/225000 [400:12:04<39:29:34,  8.22s/it]
 92%|█████████▏| 207712/225000 [400:12:12<38:44:30,  8.07s/it]
 92%|█████████▏| 207713/225000 [400:12:23<43:42:22,  9.10s/it]
 92%|█████████▏| 207714/225000 [400:12:32<43:08:44,  8.99s/it]
 92%|█████████▏| 207715/225000 [400:12:40<41:34:23,  8.66s/it]
 92%|█████████▏| 207716/225000 [400:12:46<38:17:33,  7.98s/it]
 92%|█████████▏| 207717/225000 [400:12:55<39:37:28,  8.25s/it]
 92%|█████████▏| 207718/225000 [400:13:02<37:10:48,  7.74s/it]
 92%|█████████▏| 207719/225000 [400:13:07<33:59:44,  7.08s/it]
 92%|█████████▏| 207720/225000 [400:13:22<45:32:54,  9.49s/it]
 92%|█████████▏| 207721/225000 [400:13:33<47:44:57,  9.95s/it]
 92%|█████████▏| 207722/225000 [400:13:41<43:51:11,  9.14s/it]
 92%|█████████▏| 207723/225000 [400:13:49<42:57:56,  8.95s/it]
 92%|█████████▏| 207724/225000 [400:13:55<38:17:53,  7.98s/it]
 92%|█████████▏| 207725/225000 [400:14:07<43:56:35,  9.16s/it]
 92%|█████████▏| 207726/225000 [400:14:16<44:11:00,  9.21s/it]
 92%|█████████▏| 207727/225000 [400:14:24<42:31:51,  8.86s/it]
 92%|█████████▏| 207728/225000 [400:14:34<44:00:14,  9.17s/it]
 92%|█████████▏| 207729/225000 [400:14:44<45:08:19,  9.41s/it]
 92%|█████████▏| 207730/225000 [400:14:50<40:35:19,  8.46s/it]
 92%|█████████▏| 207731/225000 [400:15:00<42:18:09,  8.82s/it]
 92%|█████████▏| 207732/225000 [400:15:07<39:38:59,  8.27s/it]
 92%|█████████▏| 207733/225000 [400:15:12<35:20:13,  7.37s/it]
 92%|█████████▏| 207734/225000 [400:15:23<40:57:51,  8.54s/it]
 92%|█████████▏| 207735/225000 [400:15:29<36:40:49,  7.65s/it]
 92%|█████████▏| 207736/225000 [400:15:35<34:17:33,  7.15s/it]
 92%|█████████▏| 207737/225000 [400:15:47<41:47:48,  8.72s/it]
 92%|█████████▏| 207738/225000 [400:15:52<35:29:53,  7.40s/it]
 92%|█████████▏| 207739/225000 [400:16:02<39:45:30,  8.29s/it]
 92%|█████████▏| 207740/225000 [400:16:11<41:14:21,  8.60s/it]
 92%|█████████▏| 207741/225000 [400:16:17<37:40:28,  7.86s/it]
 92%|█████████▏| 207742/225000 [400:16:23<34:51:31,  7.27s/it]
 92%|█████████▏| 207743/225000 [400:16:33<38:39:51,  8.07s/it]
 92%|█████████▏| 207744/225000 [400:16:47<46:10:58,  9.63s/it]
 92%|█████████▏| 207745/225000 [400:16:51<39:14:46,  8.19s/it]
 92%|█████████▏| 207746/225000 [400:16:58<36:53:36,  7.70s/it]
 92%|█████████▏| 207747/225000 [400:17:09<42:01:10,  8.77s/it]
 92%|█████████▏| 207748/225000 [400:17:17<41:13:38,  8.60s/it]
 92%|█████████▏| 207749/225000 [400:17:26<41:06:25,  8.58s/it]
 92%|█████████▏| 207750/225000 [400:17:34<40:36:20,  8.47s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.499', 'learning_rate': '7.988e-07', 'epoch': '0.9233', 'num_input_tokens_seen': 675781400, 'train_runtime': '1.441e+06', 'train_tokens_per_second': '468.9'}

 92%|█████████▏| 207750/225000 [400:17:34<40:36:20,  8.47s/it]
 92%|█████████▏| 207751/225000 [400:17:43<40:29:18,  8.45s/it]
 92%|█████████▏| 207752/225000 [400:17:50<38:47:06,  8.10s/it]
 92%|█████████▏| 207753/225000 [400:17:58<39:11:06,  8.18s/it]
 92%|█████████▏| 207754/225000 [400:18:09<42:15:32,  8.82s/it]
 92%|█████████▏| 207755/225000 [400:18:14<37:01:31,  7.73s/it]
 92%|█████████▏| 207756/225000 [400:18:22<38:07:08,  7.96s/it]
 92%|█████████▏| 207757/225000 [400:18:28<35:35:43,  7.43s/it]
 92%|█████████▏| 207758/225000 [400:18:37<37:00:36,  7.73s/it]
 92%|█████████▏| 207759/225000 [400:18:45<38:06:56,  7.96s/it]
 92%|█████████▏| 207760/225000 [400:18:55<41:14:17,  8.61s/it]
 92%|█████████▏| 207761/225000 [400:19:01<37:21:13,  7.80s/it]
 92%|█████████▏| 207762/225000 [400:19:07<34:40:07,  7.24s/it]
 92%|█████████▏| 207763/225000 [400:19:21<43:32:53,  9.10s/it]
 92%|█████████▏| 207764/225000 [400:19:27<39:53:02,  8.33s/it]
 92%|█████████▏| 207765/225000 [400:19:39<44:30:46,  9.30s/it]
 92%|█████████▏| 207766/225000 [400:19:46<41:09:55,  8.60s/it]
 92%|█████████▏| 207767/225000 [400:19:55<41:29:41,  8.67s/it]
 92%|█████████▏| 207768/225000 [400:20:05<44:05:27,  9.21s/it]
 92%|█████████▏| 207769/225000 [400:20:16<46:17:07,  9.67s/it]
 92%|█████████▏| 207770/225000 [400:20:26<47:20:47,  9.89s/it]
 92%|█████████▏| 207771/225000 [400:20:33<42:12:58,  8.82s/it]
 92%|█████████▏| 207772/225000 [400:20:40<40:22:30,  8.44s/it]
 92%|█████████▏| 207773/225000 [400:20:47<38:19:30,  8.01s/it]
 92%|█████████▏| 207774/225000 [400:20:57<41:43:28,  8.72s/it]
 92%|█████████▏| 207775/225000 [400:21:03<37:31:42,  7.84s/it]
 92%|█████████▏| 207776/225000 [400:21:12<39:26:02,  8.24s/it]
 92%|█████████▏| 207777/225000 [400:21:22<40:38:01,  8.49s/it]
 92%|█████████▏| 207778/225000 [400:21:28<37:18:41,  7.80s/it]
 92%|█████████▏| 207779/225000 [400:21:38<40:11:53,  8.40s/it]
 92%|█████████▏| 207780/225000 [400:21:43<35:42:25,  7.46s/it]
 92%|█████████▏| 207781/225000 [400:21:54<41:23:55,  8.66s/it]
 92%|█████████▏| 207782/225000 [400:22:02<40:21:47,  8.44s/it]
 92%|█████████▏| 207783/225000 [400:22:14<44:49:00,  9.37s/it]
 92%|█████████▏| 207784/225000 [400:22:23<44:42:19,  9.35s/it]
 92%|█████████▏| 207785/225000 [400:22:33<45:17:39,  9.47s/it]
 92%|█████████▏| 207786/225000 [400:22:38<39:14:02,  8.21s/it]
 92%|█████████▏| 207787/225000 [400:22:46<38:41:28,  8.09s/it]
 92%|█████████▏| 207788/225000 [400:22:53<36:54:43,  7.72s/it]
 92%|█████████▏| 207789/225000 [400:23:02<38:47:51,  8.12s/it]
 92%|█████████▏| 207790/225000 [400:23:10<39:07:09,  8.18s/it]
 92%|█████████▏| 207791/225000 [400:23:19<39:53:34,  8.35s/it]
 92%|█████████▏| 207792/225000 [400:23:25<36:33:44,  7.65s/it]
 92%|█████████▏| 207793/225000 [400:23:37<43:25:26,  9.09s/it]
 92%|█████████▏| 207794/225000 [400:23:44<39:25:42,  8.25s/it]
 92%|█████████▏| 207795/225000 [400:23:55<43:51:12,  9.18s/it]
 92%|█████████▏| 207796/225000 [400:24:00<38:06:37,  7.97s/it]
 92%|█████████▏| 207797/225000 [400:24:08<38:25:29,  8.04s/it]
 92%|█████████▏| 207798/225000 [400:24:21<45:36:18,  9.54s/it]
 92%|█████████▏| 207799/225000 [400:24:29<42:15:41,  8.84s/it]
 92%|█████████▏| 207800/225000 [400:24:33<36:24:09,  7.62s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.586', 'learning_rate': '7.943e-07', 'epoch': '0.9236', 'num_input_tokens_seen': 676006968, 'train_runtime': '1.441e+06', 'train_tokens_per_second': '469'}

 92%|█████████▏| 207800/225000 [400:24:33<36:24:09,  7.62s/it]
 92%|█████████▏| 207801/225000 [400:24:39<34:17:02,  7.18s/it]
 92%|█████████▏| 207802/225000 [400:24:46<33:00:32,  6.91s/it]
 92%|█████████▏| 207803/225000 [400:24:52<32:44:02,  6.85s/it]
 92%|█████████▏| 207804/225000 [400:24:59<32:38:27,  6.83s/it]
 92%|█████████▏| 207805/225000 [400:25:12<41:19:24,  8.65s/it]
 92%|█████████▏| 207806/225000 [400:25:18<37:35:27,  7.87s/it]
 92%|█████████▏| 207807/225000 [400:25:30<42:41:35,  8.94s/it]
 92%|█████████▏| 207808/225000 [400:25:40<44:21:55,  9.29s/it]
 92%|█████████▏| 207809/225000 [400:25:47<41:08:06,  8.61s/it]
 92%|█████████▏| 207810/225000 [400:25:59<45:58:56,  9.63s/it]
 92%|█████████▏| 207811/225000 [400:26:05<40:39:28,  8.52s/it]
 92%|█████████▏| 207812/225000 [400:26:14<41:54:44,  8.78s/it]
 92%|█████████▏| 207813/225000 [400:26:21<39:45:10,  8.33s/it]
 92%|█████████▏| 207814/225000 [400:26:31<41:03:41,  8.60s/it]
 92%|█████████▏| 207815/225000 [400:26:41<43:51:50,  9.19s/it]
 92%|█████████▏| 207816/225000 [400:26:53<47:24:49,  9.93s/it]
 92%|█████████▏| 207817/225000 [400:26:58<40:55:46,  8.58s/it]
 92%|█████████▏| 207818/225000 [400:27:08<42:28:14,  8.90s/it]
 92%|█████████▏| 207819/225000 [400:27:16<41:29:19,  8.69s/it]
 92%|█████████▏| 207820/225000 [400:27:31<50:01:16, 10.48s/it]
 92%|█████████▏| 207821/225000 [400:27:37<43:51:27,  9.19s/it]
 92%|█████████▏| 207822/225000 [400:27:43<39:20:17,  8.24s/it]
 92%|█████████▏| 207823/225000 [400:27:49<36:40:55,  7.69s/it]
 92%|█████████▏| 207824/225000 [400:28:05<47:53:15, 10.04s/it]
 92%|█████████▏| 207825/225000 [400:28:11<42:57:07,  9.00s/it]
 92%|█████████▏| 207826/225000 [400:28:20<42:45:45,  8.96s/it]
 92%|█████████▏| 207827/225000 [400:28:32<46:08:24,  9.67s/it]
 92%|█████████▏| 207828/225000 [400:28:37<40:35:57,  8.51s/it]
 92%|█████████▏| 207829/225000 [400:28:45<38:36:11,  8.09s/it]
 92%|█████████▏| 207830/225000 [400:28:51<36:54:50,  7.74s/it]
 92%|█████████▏| 207831/225000 [400:28:58<34:36:56,  7.26s/it]
 92%|█████████▏| 207832/225000 [400:29:04<33:41:14,  7.06s/it]
 92%|█████████▏| 207833/225000 [400:29:10<32:32:21,  6.82s/it]
 92%|█████████▏| 207834/225000 [400:29:21<37:36:49,  7.89s/it]
 92%|█████████▏| 207835/225000 [400:29:35<46:35:45,  9.77s/it]
 92%|█████████▏| 207836/225000 [400:29:45<46:21:32,  9.72s/it]
 92%|█████████▏| 207837/225000 [400:29:51<40:55:21,  8.58s/it]
 92%|█████████▏| 207838/225000 [400:30:02<45:24:40,  9.53s/it]
 92%|█████████▏| 207839/225000 [400:30:08<40:01:52,  8.40s/it]
 92%|█████████▏| 207840/225000 [400:30:14<36:49:56,  7.73s/it]
 92%|█████████▏| 207841/225000 [400:30:20<34:43:34,  7.29s/it]
 92%|█████████▏| 207842/225000 [400:30:30<38:16:19,  8.03s/it]
 92%|█████████▏| 207843/225000 [400:30:39<39:59:56,  8.39s/it]
 92%|█████████▏| 207844/225000 [400:30:47<38:21:49,  8.05s/it]
 92%|█████████▏| 207845/225000 [400:31:02<48:51:59, 10.25s/it]
 92%|█████████▏| 207846/225000 [400:31:06<40:14:16,  8.44s/it]
 92%|█████████▏| 207847/225000 [400:31:14<38:26:32,  8.07s/it]
 92%|█████████▏| 207848/225000 [400:31:24<42:02:27,  8.82s/it]
 92%|█████████▏| 207849/225000 [400:31:30<37:06:27,  7.79s/it]
 92%|█████████▏| 207850/225000 [400:31:45<47:43:24, 10.02s/it]
                                                              
{'loss': '1.303', 'grad_norm': '0.9239', 'learning_rate': '7.897e-07', 'epoch': '0.9238', 'num_input_tokens_seen': 676232920, 'train_runtime': '1.442e+06', 'train_tokens_per_second': '469'}

 92%|█████████▏| 207850/225000 [400:31:45<47:43:24, 10.02s/it]
 92%|█████████▏| 207851/225000 [400:31:52<44:16:30,  9.29s/it]
 92%|█████████▏| 207852/225000 [400:31:59<40:26:57,  8.49s/it]
 92%|█████████▏| 207853/225000 [400:32:07<40:09:15,  8.43s/it]
 92%|█████████▏| 207854/225000 [400:32:16<40:02:46,  8.41s/it]
 92%|█████████▏| 207855/225000 [400:32:27<44:57:08,  9.44s/it]
 92%|█████████▏| 207856/225000 [400:32:36<44:02:38,  9.25s/it]
 92%|█████████▏| 207857/225000 [400:32:44<41:46:40,  8.77s/it]
 92%|█████████▏| 207858/225000 [400:32:50<38:04:50,  8.00s/it]
 92%|█████████▏| 207859/225000 [400:33:07<50:42:58, 10.65s/it]
 92%|█████████▏| 207860/225000 [400:33:13<43:34:38,  9.15s/it]
 92%|█████████▏| 207861/225000 [400:33:22<43:56:49,  9.23s/it]
 92%|█████████▏| 207862/225000 [400:33:27<37:50:07,  7.95s/it]
 92%|█████████▏| 207863/225000 [400:33:37<40:28:25,  8.50s/it]
 92%|█████████▏| 207864/225000 [400:33:48<44:21:47,  9.32s/it]
 92%|█████████▏| 207865/225000 [400:33:54<40:00:12,  8.40s/it]
 92%|█████████▏| 207866/225000 [400:34:01<37:05:20,  7.79s/it]
 92%|█████████▏| 207867/225000 [400:34:06<34:12:16,  7.19s/it]
 92%|█████████▏| 207868/225000 [400:34:20<43:46:54,  9.20s/it]
 92%|█████████▏| 207869/225000 [400:34:26<39:19:07,  8.26s/it]
 92%|█████████▏| 207870/225000 [400:34:35<39:15:21,  8.25s/it]
 92%|█████████▏| 207871/225000 [400:34:41<36:01:27,  7.57s/it]
 92%|█████████▏| 207872/225000 [400:34:46<32:29:42,  6.83s/it]
 92%|█████████▏| 207873/225000 [400:34:59<42:18:12,  8.89s/it]
 92%|█████████▏| 207874/225000 [400:35:03<35:20:29,  7.43s/it]
 92%|█████████▏| 207875/225000 [400:35:16<42:33:58,  8.95s/it]
 92%|█████████▏| 207876/225000 [400:35:27<45:50:03,  9.64s/it]
 92%|█████████▏| 207877/225000 [400:35:33<40:46:50,  8.57s/it]
 92%|█████████▏| 207878/225000 [400:35:43<42:02:50,  8.84s/it]
 92%|█████████▏| 207879/225000 [400:35:52<43:15:25,  9.10s/it]
 92%|█████████▏| 207880/225000 [400:36:00<41:24:23,  8.71s/it]
 92%|█████████▏| 207881/225000 [400:36:07<39:22:13,  8.28s/it]
 92%|█████████▏| 207882/225000 [400:36:14<37:04:36,  7.80s/it]
 92%|█████████▏| 207883/225000 [400:36:21<35:47:48,  7.53s/it]
 92%|█████████▏| 207884/225000 [400:36:32<40:50:11,  8.59s/it]
 92%|█████████▏| 207885/225000 [400:36:39<38:45:31,  8.15s/it]
 92%|█████████▏| 207886/225000 [400:36:46<36:46:40,  7.74s/it]
 92%|█████████▏| 207887/225000 [400:36:54<36:51:50,  7.75s/it]
 92%|█████████▏| 207888/225000 [400:37:01<36:05:46,  7.59s/it]
 92%|█████████▏| 207889/225000 [400:37:12<40:56:31,  8.61s/it]
 92%|█████████▏| 207890/225000 [400:37:19<38:24:17,  8.08s/it]
 92%|█████████▏| 207891/225000 [400:37:30<42:46:02,  9.00s/it]
 92%|█████████▏| 207892/225000 [400:37:35<37:00:07,  7.79s/it]
 92%|█████████▏| 207893/225000 [400:37:41<35:04:49,  7.38s/it]
 92%|█████████▏| 207894/225000 [400:37:53<41:06:48,  8.65s/it]
 92%|█████████▏| 207895/225000 [400:37:59<36:46:56,  7.74s/it]
 92%|█████████▏| 207896/225000 [400:38:06<36:30:05,  7.68s/it]
 92%|█████████▏| 207897/225000 [400:38:14<36:02:23,  7.59s/it]
 92%|█████████▏| 207898/225000 [400:38:29<46:35:30,  9.81s/it]
 92%|█████████▏| 207899/225000 [400:38:35<41:22:15,  8.71s/it]
 92%|█████████▏| 207900/225000 [400:38:42<39:58:15,  8.41s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.143', 'learning_rate': '7.851e-07', 'epoch': '0.924', 'num_input_tokens_seen': 676459496, 'train_runtime': '1.442e+06', 'train_tokens_per_second': '469'}

 92%|█████████▏| 207900/225000 [400:38:42<39:58:15,  8.41s/it]
 92%|█████████▏| 207901/225000 [400:38:55<46:06:21,  9.71s/it]
 92%|█████████▏| 207902/225000 [400:39:06<47:35:03, 10.02s/it]
 92%|█████████▏| 207903/225000 [400:39:14<45:00:06,  9.48s/it]
 92%|█████████▏| 207904/225000 [400:39:20<40:24:24,  8.51s/it]
 92%|█████████▏| 207905/225000 [400:39:29<40:38:22,  8.56s/it]
 92%|█████████▏| 207906/225000 [400:39:37<39:44:06,  8.37s/it]
 92%|█████████▏| 207907/225000 [400:39:49<45:08:46,  9.51s/it]
 92%|█████████▏| 207908/225000 [400:39:56<41:26:05,  8.73s/it]
 92%|█████████▏| 207909/225000 [400:40:03<38:27:23,  8.10s/it]
 92%|█████████▏| 207910/225000 [400:40:13<41:23:41,  8.72s/it]
 92%|█████████▏| 207911/225000 [400:40:20<39:03:31,  8.23s/it]
 92%|█████████▏| 207912/225000 [400:40:25<34:45:24,  7.32s/it]
 92%|█████████▏| 207913/225000 [400:40:37<41:59:50,  8.85s/it]
 92%|█████████▏| 207914/225000 [400:40:49<45:33:27,  9.60s/it]
 92%|█████████▏| 207915/225000 [400:40:54<38:34:40,  8.13s/it]
 92%|█████████▏| 207916/225000 [400:41:01<37:24:21,  7.88s/it]
 92%|█████████▏| 207917/225000 [400:41:11<40:11:47,  8.47s/it]
 92%|█████████▏| 207918/225000 [400:41:21<43:22:59,  9.14s/it]
 92%|█████████▏| 207919/225000 [400:41:30<42:51:38,  9.03s/it]
 92%|█████████▏| 207920/225000 [400:41:35<36:59:43,  7.80s/it]
 92%|█████████▏| 207921/225000 [400:41:44<39:04:20,  8.24s/it]
 92%|█████████▏| 207922/225000 [400:41:53<38:57:20,  8.21s/it]
 92%|█████████▏| 207923/225000 [400:42:03<41:42:48,  8.79s/it]
 92%|█████████▏| 207924/225000 [400:42:11<40:45:27,  8.59s/it]
 92%|█████████▏| 207925/225000 [400:42:20<41:15:48,  8.70s/it]
 92%|█████████▏| 207926/225000 [400:42:28<41:08:46,  8.68s/it]
 92%|█████████▏| 207927/225000 [400:42:34<36:27:46,  7.69s/it]
 92%|█████████▏| 207928/225000 [400:42:40<34:12:15,  7.21s/it]
 92%|█████████▏| 207929/225000 [400:42:51<40:01:04,  8.44s/it]
 92%|█████████▏| 207930/225000 [400:42:56<34:59:25,  7.38s/it]
 92%|█████████▏| 207931/225000 [400:43:05<37:23:47,  7.89s/it]
 92%|█████████▏| 207932/225000 [400:43:13<37:00:10,  7.80s/it]
 92%|█████████▏| 207933/225000 [400:43:22<39:04:38,  8.24s/it]
 92%|█████████▏| 207934/225000 [400:43:31<40:51:33,  8.62s/it]
 92%|█████████▏| 207935/225000 [400:43:36<35:33:48,  7.50s/it]
 92%|█████████▏| 207936/225000 [400:43:50<44:48:45,  9.45s/it]
 92%|█████████▏| 207937/225000 [400:43:59<44:14:20,  9.33s/it]
 92%|█████████▏| 207938/225000 [400:44:05<38:45:37,  8.18s/it]
 92%|█████████▏| 207939/225000 [400:44:12<37:00:34,  7.81s/it]
 92%|█████████▏| 207940/225000 [400:44:16<31:27:34,  6.64s/it]
 92%|█████████▏| 207941/225000 [400:44:26<36:52:57,  7.78s/it]
 92%|█████████▏| 207942/225000 [400:44:34<36:15:05,  7.65s/it]
 92%|█████████▏| 207943/225000 [400:44:40<33:48:05,  7.13s/it]
 92%|█████████▏| 207944/225000 [400:44:46<32:43:06,  6.91s/it]
 92%|█████████▏| 207945/225000 [400:44:56<36:39:01,  7.74s/it]
 92%|█████████▏| 207946/225000 [400:45:06<40:44:18,  8.60s/it]
 92%|█████████▏| 207947/225000 [400:45:13<38:00:49,  8.02s/it]
 92%|█████████▏| 207948/225000 [400:45:18<33:21:47,  7.04s/it]
 92%|█████████▏| 207949/225000 [400:45:24<31:47:04,  6.71s/it]
 92%|█████████▏| 207950/225000 [400:45:37<41:14:40,  8.71s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.233', 'learning_rate': '7.806e-07', 'epoch': '0.9242', 'num_input_tokens_seen': 676685832, 'train_runtime': '1.443e+06', 'train_tokens_per_second': '469'}

 92%|█████████▏| 207950/225000 [400:45:37<41:14:40,  8.71s/it]
 92%|█████████▏| 207951/225000 [400:45:42<35:40:58,  7.53s/it]
 92%|█████████▏| 207952/225000 [400:45:49<36:01:52,  7.61s/it]
 92%|█████████▏| 207953/225000 [400:45:56<34:52:05,  7.36s/it]
 92%|█████████▏| 207954/225000 [400:46:09<42:01:41,  8.88s/it]
 92%|█████████▏| 207955/225000 [400:46:18<41:59:54,  8.87s/it]
 92%|█████████▏| 207956/225000 [400:46:24<38:10:22,  8.06s/it]
 92%|█████████▏| 207957/225000 [400:46:31<37:28:47,  7.92s/it]
 92%|█████████▏| 207958/225000 [400:46:44<44:24:46,  9.38s/it]
 92%|█████████▏| 207959/225000 [400:46:49<37:25:14,  7.91s/it]
 92%|█████████▏| 207960/225000 [400:47:01<43:52:51,  9.27s/it]
 92%|█████████▏| 207961/225000 [400:47:07<39:17:53,  8.30s/it]
 92%|█████████▏| 207962/225000 [400:47:13<35:57:49,  7.60s/it]
 92%|█████████▏| 207963/225000 [400:47:26<44:05:41,  9.32s/it]
 92%|█████████▏| 207964/225000 [400:47:34<42:25:17,  8.96s/it]
 92%|█████████▏| 207965/225000 [400:47:43<41:27:42,  8.76s/it]
 92%|█████████▏| 207966/225000 [400:47:54<44:43:11,  9.45s/it]
 92%|█████████▏| 207967/225000 [400:47:59<39:10:32,  8.28s/it]
 92%|█████████▏| 207968/225000 [400:48:06<37:10:21,  7.86s/it]
 92%|█████████▏| 207969/225000 [400:48:13<36:00:06,  7.61s/it]
 92%|█████████▏| 207970/225000 [400:48:20<34:46:50,  7.35s/it]
 92%|█████████▏| 207971/225000 [400:48:32<40:57:09,  8.66s/it]
 92%|█████████▏| 207972/225000 [400:48:39<38:39:58,  8.17s/it]
 92%|█████████▏| 207973/225000 [400:48:45<35:15:02,  7.45s/it]
 92%|█████████▏| 207974/225000 [400:48:58<43:15:06,  9.15s/it]
 92%|█████████▏| 207975/225000 [400:49:04<39:52:31,  8.43s/it]
 92%|█████████▏| 207976/225000 [400:49:11<36:38:11,  7.75s/it]
 92%|█████████▏| 207977/225000 [400:49:17<34:05:37,  7.21s/it]
 92%|█████████▏| 207978/225000 [400:49:28<40:49:19,  8.63s/it]
 92%|█████████▏| 207979/225000 [400:49:34<36:39:51,  7.75s/it]
 92%|█████████▏| 207980/225000 [400:49:41<34:55:47,  7.39s/it]
 92%|█████████▏| 207981/225000 [400:49:51<38:25:44,  8.13s/it]
 92%|█████████▏| 207982/225000 [400:49:59<39:07:19,  8.28s/it]
 92%|█████████▏| 207983/225000 [400:50:10<42:06:26,  8.91s/it]
 92%|█████████▏| 207984/225000 [400:50:18<41:03:01,  8.68s/it]
 92%|█████████▏| 207985/225000 [400:50:29<44:08:39,  9.34s/it]
 92%|█████████▏| 207986/225000 [400:50:35<40:15:27,  8.52s/it]
 92%|█████████▏| 207987/225000 [400:50:41<36:29:04,  7.72s/it]
 92%|█████████▏| 207988/225000 [400:50:55<45:19:50,  9.59s/it]
 92%|█████████▏| 207989/225000 [400:51:02<42:12:51,  8.93s/it]
 92%|█████████▏| 207990/225000 [400:51:09<38:23:16,  8.12s/it]
 92%|█████████▏| 207991/225000 [400:51:21<44:05:02,  9.33s/it]
 92%|█████████▏| 207992/225000 [400:51:31<45:23:54,  9.61s/it]
 92%|█████████▏| 207993/225000 [400:51:37<39:37:34,  8.39s/it]
 92%|█████████▏| 207994/225000 [400:51:42<35:33:27,  7.53s/it]
 92%|█████████▏| 207995/225000 [400:51:54<41:18:24,  8.74s/it]
 92%|█████████▏| 207996/225000 [400:52:00<37:13:49,  7.88s/it]
 92%|█████████▏| 207997/225000 [400:52:06<34:53:06,  7.39s/it]
 92%|█████████▏| 207998/225000 [400:52:19<42:24:36,  8.98s/it]
 92%|█████████▏| 207999/225000 [400:52:29<44:18:19,  9.38s/it]
 92%|█████████▏| 208000/225000 [400:52:38<43:37:05,  9.24s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.175', 'learning_rate': '7.761e-07', 'epoch': '0.9244', 'num_input_tokens_seen': 676913000, 'train_runtime': '1.443e+06', 'train_tokens_per_second': '469.1'}

 92%|█████████▏| 208000/225000 [400:52:38<43:37:05,  9.24s/it]
 92%|█████████▏| 208001/225000 [400:52:48<45:47:10,  9.70s/it]
 92%|█████████▏| 208002/225000 [400:52:56<42:48:41,  9.07s/it]
 92%|█████████▏| 208003/225000 [400:53:02<38:09:19,  8.08s/it]
 92%|█████████▏| 208004/225000 [400:53:08<34:55:57,  7.40s/it]
 92%|█████████▏| 208005/225000 [400:53:15<35:11:42,  7.46s/it]
 92%|█████████▏| 208006/225000 [400:53:28<42:05:09,  8.92s/it]
 92%|█████████▏| 208007/225000 [400:53:36<42:00:10,  8.90s/it]
 92%|█████████▏| 208008/225000 [400:53:46<42:46:27,  9.06s/it]
 92%|█████████▏| 208009/225000 [400:53:57<45:50:44,  9.71s/it]
 92%|█████████▏| 208010/225000 [400:54:05<43:01:45,  9.12s/it]
 92%|█████████▏| 208011/225000 [400:54:14<43:18:00,  9.18s/it]
 92%|█████████▏| 208012/225000 [400:54:20<38:11:26,  8.09s/it]
 92%|█████████▏| 208013/225000 [400:54:24<33:04:35,  7.01s/it]
 92%|█████████▏| 208014/225000 [400:54:37<40:40:01,  8.62s/it]
 92%|█████████▏| 208015/225000 [400:54:46<41:28:39,  8.79s/it]
 92%|█████████▏| 208016/225000 [400:54:52<37:18:13,  7.91s/it]
 92%|█████████▏| 208017/225000 [400:55:04<42:59:43,  9.11s/it]
 92%|█████████▏| 208018/225000 [400:55:14<44:21:16,  9.40s/it]
 92%|█████████▏| 208019/225000 [400:55:19<38:57:35,  8.26s/it]
 92%|█████████▏| 208020/225000 [400:55:30<42:25:34,  8.99s/it]
 92%|█████████▏| 208021/225000 [400:55:36<38:35:26,  8.18s/it]
 92%|█████████▏| 208022/225000 [400:55:46<40:33:54,  8.60s/it]
 92%|█████████▏| 208023/225000 [400:55:52<37:42:07,  7.99s/it]
 92%|█████████▏| 208024/225000 [400:56:01<38:17:48,  8.12s/it]
 92%|█████████▏| 208025/225000 [400:56:08<36:42:58,  7.79s/it]
 92%|█████████▏| 208026/225000 [400:56:14<34:41:29,  7.36s/it]
 92%|█████████▏| 208027/225000 [400:56:21<34:31:11,  7.32s/it]
 92%|█████████▏| 208028/225000 [400:56:33<40:50:35,  8.66s/it]
 92%|█████████▏| 208029/225000 [400:56:41<40:19:19,  8.55s/it]
 92%|█████████▏| 208030/225000 [400:56:53<44:23:00,  9.42s/it]
 92%|█████████▏| 208031/225000 [400:56:59<39:08:24,  8.30s/it]
 92%|█████████▏| 208032/225000 [400:57:11<44:56:34,  9.54s/it]
 92%|█████████▏| 208033/225000 [400:57:16<38:16:09,  8.12s/it]
 92%|█████████▏| 208034/225000 [400:57:28<43:29:03,  9.23s/it]
 92%|█████████▏| 208035/225000 [400:57:33<37:28:52,  7.95s/it]
 92%|█████████▏| 208036/225000 [400:57:43<41:18:08,  8.76s/it]
 92%|█████████▏| 208037/225000 [400:57:49<37:15:00,  7.91s/it]
 92%|█████████▏| 208038/225000 [400:57:56<36:00:55,  7.64s/it]
 92%|█████████▏| 208039/225000 [400:58:04<36:05:01,  7.66s/it]
 92%|█████████▏| 208040/225000 [400:58:10<34:20:09,  7.29s/it]
 92%|█████████▏| 208041/225000 [400:58:23<41:47:50,  8.87s/it]
 92%|█████████▏| 208042/225000 [400:58:31<40:30:02,  8.60s/it]
 92%|█████████▏| 208043/225000 [400:58:42<43:49:42,  9.30s/it]
 92%|█████████▏| 208044/225000 [400:58:51<43:05:11,  9.15s/it]
 92%|█████████▏| 208045/225000 [400:59:01<44:32:58,  9.46s/it]
 92%|█████████▏| 208046/225000 [400:59:07<40:18:11,  8.56s/it]
 92%|█████████▏| 208047/225000 [400:59:14<37:12:57,  7.90s/it]
 92%|█████████▏| 208048/225000 [400:59:20<35:42:08,  7.58s/it]
 92%|█████████▏| 208049/225000 [400:59:32<40:49:25,  8.67s/it]
 92%|█████████▏| 208050/225000 [400:59:40<40:41:43,  8.64s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.259', 'learning_rate': '7.716e-07', 'epoch': '0.9247', 'num_input_tokens_seen': 677144920, 'train_runtime': '1.444e+06', 'train_tokens_per_second': '469.1'}

 92%|█████████▏| 208050/225000 [400:59:40<40:41:43,  8.64s/it]
 92%|█████████▏| 208051/225000 [400:59:45<35:26:52,  7.53s/it]
 92%|█████████▏| 208052/225000 [400:59:53<36:19:59,  7.72s/it]
 92%|█████████▏| 208053/225000 [401:00:00<34:28:44,  7.32s/it]
 92%|█████████▏| 208054/225000 [401:00:11<39:37:00,  8.42s/it]
 92%|█████████▏| 208055/225000 [401:00:23<45:13:56,  9.61s/it]
 92%|█████████▏| 208056/225000 [401:00:33<45:52:10,  9.75s/it]
 92%|█████████▏| 208057/225000 [401:00:40<41:43:31,  8.87s/it]
 92%|█████████▏| 208058/225000 [401:00:48<40:26:28,  8.59s/it]
 92%|█████████▏| 208059/225000 [401:00:57<41:25:59,  8.80s/it]
 92%|█████████▏| 208060/225000 [401:01:04<38:55:30,  8.27s/it]
 92%|█████████▏| 208061/225000 [401:01:10<35:55:35,  7.64s/it]
 92%|█████████▏| 208062/225000 [401:01:22<40:51:34,  8.68s/it]
 92%|█████████▏| 208063/225000 [401:01:30<40:00:40,  8.50s/it]
 92%|█████████▏| 208064/225000 [401:01:39<41:52:03,  8.90s/it]
 92%|█████████▏| 208065/225000 [401:01:45<37:12:20,  7.91s/it]
 92%|█████████▏| 208066/225000 [401:01:58<44:20:21,  9.43s/it]
 92%|█████████▏| 208067/225000 [401:02:07<43:12:37,  9.19s/it]
 92%|█████████▏| 208068/225000 [401:02:17<44:29:57,  9.46s/it]
 92%|█████████▏| 208069/225000 [401:02:27<45:31:45,  9.68s/it]
 92%|█████████▏| 208070/225000 [401:02:36<44:36:13,  9.48s/it]
 92%|█████████▏| 208071/225000 [401:02:48<47:57:18, 10.20s/it]
 92%|█████████▏| 208072/225000 [401:02:56<45:03:04,  9.58s/it]
 92%|█████████▏| 208073/225000 [401:03:01<38:35:33,  8.21s/it]
 92%|█████████▏| 208074/225000 [401:03:11<40:47:31,  8.68s/it]
 92%|█████████▏| 208075/225000 [401:03:18<38:56:52,  8.28s/it]
 92%|█████████▏| 208076/225000 [401:03:26<38:48:45,  8.26s/it]
 92%|█████████▏| 208077/225000 [401:03:37<41:39:43,  8.86s/it]
 92%|█████████▏| 208078/225000 [401:03:42<37:07:58,  7.90s/it]
 92%|█████████▏| 208079/225000 [401:03:50<37:15:02,  7.93s/it]
 92%|█████████▏| 208080/225000 [401:03:57<36:00:27,  7.66s/it]
 92%|█████████▏| 208081/225000 [401:04:04<35:19:43,  7.52s/it]
 92%|█████████▏| 208082/225000 [401:04:13<37:12:38,  7.92s/it]
 92%|█████████▏| 208083/225000 [401:04:18<33:17:48,  7.09s/it]
 92%|█████████▏| 208084/225000 [401:04:26<34:19:54,  7.31s/it]
 92%|█████████▏| 208085/225000 [401:04:34<34:51:08,  7.42s/it]
 92%|█████████▏| 208086/225000 [401:04:42<36:03:23,  7.67s/it]
 92%|█████████▏| 208087/225000 [401:04:50<35:49:13,  7.62s/it]
 92%|█████████▏| 208088/225000 [401:04:59<38:40:45,  8.23s/it]
 92%|█████████▏| 208089/225000 [401:05:12<44:21:35,  9.44s/it]
 92%|█████████▏| 208090/225000 [401:05:21<44:19:25,  9.44s/it]
 92%|█████████▏| 208091/225000 [401:05:29<42:48:45,  9.11s/it]
 92%|█████████▏| 208092/225000 [401:05:35<38:06:03,  8.11s/it]
 92%|█████████▏| 208093/225000 [401:05:41<34:45:13,  7.40s/it]
 92%|█████████▏| 208094/225000 [401:05:52<40:34:01,  8.64s/it]
 92%|█████████▏| 208095/225000 [401:06:00<38:39:00,  8.23s/it]
 92%|█████████▏| 208096/225000 [401:06:09<40:15:39,  8.57s/it]
 92%|█████████▏| 208097/225000 [401:06:16<38:09:54,  8.13s/it]
 92%|█████████▏| 208098/225000 [401:06:27<41:59:54,  8.95s/it]
 92%|█████████▏| 208099/225000 [401:06:35<40:14:12,  8.57s/it]
 92%|█████████▏| 208100/225000 [401:06:47<45:52:57,  9.77s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.158', 'learning_rate': '7.671e-07', 'epoch': '0.9249', 'num_input_tokens_seen': 677376528, 'train_runtime': '1.444e+06', 'train_tokens_per_second': '469.1'}

 92%|█████████▏| 208100/225000 [401:06:47<45:52:57,  9.77s/it]
 92%|█████████▏| 208101/225000 [401:06:57<45:48:08,  9.76s/it]
 92%|█████████▏| 208102/225000 [401:07:04<42:05:56,  8.97s/it]
 92%|█████████▏| 208103/225000 [401:07:13<42:10:32,  8.99s/it]
 92%|█████████▏| 208104/225000 [401:07:20<38:39:40,  8.24s/it]
 92%|█████████▏| 208105/225000 [401:07:27<36:39:37,  7.81s/it]
 92%|█████████▏| 208106/225000 [401:07:38<42:22:58,  9.03s/it]
 92%|█████████▏| 208107/225000 [401:07:46<40:30:46,  8.63s/it]
 92%|█████████▏| 208108/225000 [401:07:53<37:58:14,  8.09s/it]
 92%|█████████▏| 208109/225000 [401:08:02<39:40:24,  8.46s/it]
 92%|█████████▏| 208110/225000 [401:08:13<43:10:23,  9.20s/it]
 92%|█████████▏| 208111/225000 [401:08:20<40:28:59,  8.63s/it]
 92%|█████████▏| 208112/225000 [401:08:31<43:36:26,  9.30s/it]
 92%|█████████▏| 208113/225000 [401:08:42<45:23:13,  9.68s/it]
 92%|█████████▏| 208114/225000 [401:08:50<43:10:01,  9.20s/it]
 92%|█████████▏| 208115/225000 [401:08:58<41:36:43,  8.87s/it]
 92%|█████████▏| 208116/225000 [401:09:11<47:58:05, 10.23s/it]
 92%|█████████▏| 208117/225000 [401:09:23<49:37:11, 10.58s/it]
 92%|█████████▏| 208118/225000 [401:09:28<42:30:32,  9.06s/it]
 92%|█████████▏| 208119/225000 [401:09:33<36:43:17,  7.83s/it]
 92%|█████████▏| 208120/225000 [401:09:39<33:49:23,  7.21s/it]
 92%|█████████▏| 208121/225000 [401:09:46<33:00:00,  7.04s/it]
 92%|█████████▏| 208122/225000 [401:09:56<38:01:10,  8.11s/it]
 92%|█████████▏| 208123/225000 [401:10:02<34:18:14,  7.32s/it]
 92%|█████████▏| 208124/225000 [401:10:16<43:18:10,  9.24s/it]
 92%|█████████▎| 208125/225000 [401:10:22<40:01:10,  8.54s/it]
 93%|█████████▎| 208126/225000 [401:10:30<38:05:46,  8.13s/it]
 93%|█████████▎| 208127/225000 [401:10:41<41:59:53,  8.96s/it]
 93%|█████████▎| 208128/225000 [401:10:47<38:02:40,  8.12s/it]
 93%|█████████▎| 208129/225000 [401:10:53<36:12:29,  7.73s/it]
 93%|█████████▎| 208130/225000 [401:11:03<38:17:28,  8.17s/it]
 93%|█████████▎| 208131/225000 [401:11:09<36:01:20,  7.69s/it]
 93%|█████████▎| 208132/225000 [401:11:18<38:08:49,  8.14s/it]
 93%|█████████▎| 208133/225000 [401:11:27<38:01:04,  8.11s/it]
 93%|█████████▎| 208134/225000 [401:11:32<33:59:04,  7.25s/it]
 93%|█████████▎| 208135/225000 [401:11:42<37:39:37,  8.04s/it]
 93%|█████████▎| 208136/225000 [401:11:51<40:03:03,  8.55s/it]
 93%|█████████▎| 208137/225000 [401:12:00<40:28:38,  8.64s/it]
 93%|█████████▎| 208138/225000 [401:12:08<38:48:48,  8.29s/it]
 93%|█████████▎| 208139/225000 [401:12:16<38:22:27,  8.19s/it]
 93%|█████████▎| 208140/225000 [401:12:23<36:59:47,  7.90s/it]
 93%|█████████▎| 208141/225000 [401:12:30<35:44:23,  7.63s/it]
 93%|█████████▎| 208142/225000 [401:12:39<38:04:02,  8.13s/it]
 93%|█████████▎| 208143/225000 [401:12:47<37:18:44,  7.97s/it]
 93%|█████████▎| 208144/225000 [401:12:55<37:19:45,  7.97s/it]
 93%|█████████▎| 208145/225000 [401:13:07<43:53:27,  9.37s/it]
 93%|█████████▎| 208146/225000 [401:13:17<43:40:05,  9.33s/it]
 93%|█████████▎| 208147/225000 [401:13:24<41:23:12,  8.84s/it]
 93%|█████████▎| 208148/225000 [401:13:31<38:36:56,  8.25s/it]
 93%|█████████▎| 208149/225000 [401:13:44<44:35:13,  9.53s/it]
 93%|█████████▎| 208150/225000 [401:13:52<43:27:54,  9.29s/it]
                                                              
{'loss': '1.337', 'grad_norm': '1.368', 'learning_rate': '7.626e-07', 'epoch': '0.9251', 'num_input_tokens_seen': 677604904, 'train_runtime': '1.444e+06', 'train_tokens_per_second': '469.1'}

 93%|█████████▎| 208150/225000 [401:13:52<43:27:54,  9.29s/it]
 93%|█████████▎| 208151/225000 [401:14:06<49:55:09, 10.67s/it]
 93%|█████████▎| 208152/225000 [401:14:15<46:39:40,  9.97s/it]
 93%|█████████▎| 208153/225000 [401:14:20<40:25:23,  8.64s/it]
 93%|█████████▎| 208154/225000 [401:14:29<40:50:03,  8.73s/it]
 93%|█████████▎| 208155/225000 [401:14:37<39:40:23,  8.48s/it]
 93%|█████████▎| 208156/225000 [401:14:50<45:33:16,  9.74s/it]
 93%|█████████▎| 208157/225000 [401:14:55<39:13:42,  8.38s/it]
 93%|█████████▎| 208158/225000 [401:15:03<38:19:58,  8.19s/it]
 93%|█████████▎| 208159/225000 [401:15:08<34:44:29,  7.43s/it]
 93%|█████████▎| 208160/225000 [401:15:16<34:39:41,  7.41s/it]
 93%|█████████▎| 208161/225000 [401:15:23<34:09:39,  7.30s/it]
 93%|█████████▎| 208162/225000 [401:15:32<37:22:51,  7.99s/it]
 93%|█████████▎| 208163/225000 [401:15:38<34:06:35,  7.29s/it]
 93%|█████████▎| 208164/225000 [401:15:45<34:17:43,  7.33s/it]
 93%|█████████▎| 208165/225000 [401:15:58<41:10:09,  8.80s/it]
 93%|█████████▎| 208166/225000 [401:16:05<38:43:14,  8.28s/it]
 93%|█████████▎| 208167/225000 [401:16:13<39:14:31,  8.39s/it]
 93%|█████████▎| 208168/225000 [401:16:19<35:50:03,  7.66s/it]
 93%|█████████▎| 208169/225000 [401:16:31<41:04:24,  8.79s/it]
 93%|█████████▎| 208170/225000 [401:16:40<41:28:14,  8.87s/it]
 93%|█████████▎| 208171/225000 [401:16:46<37:58:49,  8.12s/it]
 93%|█████████▎| 208172/225000 [401:16:55<39:30:43,  8.45s/it]
 93%|█████████▎| 208173/225000 [401:17:08<45:50:46,  9.81s/it]
 93%|█████████▎| 208174/225000 [401:17:17<44:40:49,  9.56s/it]
 93%|█████████▎| 208175/225000 [401:17:22<37:07:00,  7.94s/it]
 93%|█████████▎| 208176/225000 [401:17:32<40:51:01,  8.74s/it]
 93%|█████████▎| 208177/225000 [401:17:36<34:42:53,  7.43s/it]
 93%|█████████▎| 208178/225000 [401:17:47<39:04:59,  8.36s/it]
 93%|█████████▎| 208179/225000 [401:17:57<41:34:47,  8.90s/it]
 93%|█████████▎| 208180/225000 [401:18:07<43:14:21,  9.25s/it]
 93%|█████████▎| 208181/225000 [401:18:15<41:40:54,  8.92s/it]
 93%|█████████▎| 208182/225000 [401:18:22<37:47:32,  8.09s/it]
 93%|█████████▎| 208183/225000 [401:18:29<36:21:42,  7.78s/it]
 93%|█████████▎| 208184/225000 [401:18:42<44:24:59,  9.51s/it]
 93%|█████████▎| 208185/225000 [401:18:50<42:31:56,  9.11s/it]
 93%|█████████▎| 208186/225000 [401:18:56<37:06:37,  7.95s/it]
 93%|█████████▎| 208187/225000 [401:19:05<38:37:58,  8.27s/it]
 93%|█████████▎| 208188/225000 [401:19:19<47:37:06, 10.20s/it]
 93%|█████████▎| 208189/225000 [401:19:28<44:50:52,  9.60s/it]
 93%|█████████▎| 208190/225000 [401:19:38<46:17:53,  9.92s/it]
 93%|█████████▎| 208191/225000 [401:19:43<39:43:30,  8.51s/it]
 93%|█████████▎| 208192/225000 [401:19:54<42:02:30,  9.00s/it]
 93%|█████████▎| 208193/225000 [401:20:05<45:30:32,  9.75s/it]
 93%|█████████▎| 208194/225000 [401:20:09<37:41:48,  8.07s/it]
 93%|█████████▎| 208195/225000 [401:20:18<38:56:37,  8.34s/it]
 93%|█████████▎| 208196/225000 [401:20:28<41:13:53,  8.83s/it]
 93%|█████████▎| 208197/225000 [401:20:35<38:11:19,  8.18s/it]
 93%|█████████▎| 208198/225000 [401:20:45<40:40:36,  8.72s/it]
 93%|█████████▎| 208199/225000 [401:20:51<36:31:44,  7.83s/it]
 93%|█████████▎| 208200/225000 [401:20:59<37:54:09,  8.12s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.302', 'learning_rate': '7.581e-07', 'epoch': '0.9253', 'num_input_tokens_seen': 677834688, 'train_runtime': '1.445e+06', 'train_tokens_per_second': '469.1'}

 93%|█████████▎| 208200/225000 [401:20:59<37:54:09,  8.12s/it]
 93%|█████████▎| 208201/225000 [401:21:13<45:16:22,  9.70s/it]
 93%|█████████▎| 208202/225000 [401:21:18<38:55:15,  8.34s/it]
 93%|█████████▎| 208203/225000 [401:21:22<32:59:56,  7.07s/it]
 93%|█████████▎| 208204/225000 [401:21:33<37:58:12,  8.14s/it]
 93%|█████████▎| 208205/225000 [401:21:40<37:35:53,  8.06s/it]
 93%|█████████▎| 208206/225000 [401:21:49<38:52:11,  8.33s/it]
 93%|█████████▎| 208207/225000 [401:21:57<37:30:48,  8.04s/it]
 93%|█████████▎| 208208/225000 [401:22:07<40:40:34,  8.72s/it]
 93%|█████████▎| 208209/225000 [401:22:15<39:49:47,  8.54s/it]
 93%|█████████▎| 208210/225000 [401:22:27<44:13:50,  9.48s/it]
 93%|█████████▎| 208211/225000 [401:22:33<38:45:23,  8.31s/it]
 93%|█████████▎| 208212/225000 [401:22:46<46:06:35,  9.89s/it]
 93%|█████████▎| 208213/225000 [401:22:52<40:09:46,  8.61s/it]
 93%|█████████▎| 208214/225000 [401:23:03<43:31:32,  9.33s/it]
 93%|█████████▎| 208215/225000 [401:23:09<39:26:10,  8.46s/it]
 93%|█████████▎| 208216/225000 [401:23:21<44:05:07,  9.46s/it]
 93%|█████████▎| 208217/225000 [401:23:26<37:29:11,  8.04s/it]
 93%|█████████▎| 208218/225000 [401:23:35<39:02:19,  8.37s/it]
 93%|█████████▎| 208219/225000 [401:23:44<39:37:04,  8.50s/it]
 93%|█████████▎| 208220/225000 [401:23:53<40:20:35,  8.66s/it]
 93%|█████████▎| 208221/225000 [401:24:02<40:38:44,  8.72s/it]
 93%|█████████▎| 208222/225000 [401:24:10<40:34:11,  8.70s/it]
 93%|█████████▎| 208223/225000 [401:24:20<42:03:46,  9.03s/it]
 93%|█████████▎| 208224/225000 [401:24:27<39:37:27,  8.50s/it]
 93%|█████████▎| 208225/225000 [401:24:35<38:26:17,  8.25s/it]
 93%|█████████▎| 208226/225000 [401:24:48<45:23:05,  9.74s/it]
 93%|█████████▎| 208227/225000 [401:24:57<44:09:58,  9.48s/it]
 93%|█████████▎| 208228/225000 [401:25:05<41:57:06,  9.00s/it]
 93%|█████████▎| 208229/225000 [401:25:13<40:57:58,  8.79s/it]
 93%|█████████▎| 208230/225000 [401:25:19<36:32:56,  7.85s/it]
 93%|█████████▎| 208231/225000 [401:25:30<41:10:10,  8.84s/it]
 93%|█████████▎| 208232/225000 [401:25:44<47:46:31, 10.26s/it]
 93%|█████████▎| 208233/225000 [401:25:52<44:42:00,  9.60s/it]
 93%|█████████▎| 208234/225000 [401:25:58<40:22:33,  8.67s/it]
 93%|█████████▎| 208235/225000 [401:26:10<44:48:58,  9.62s/it]
 93%|█████████▎| 208236/225000 [401:26:17<41:14:27,  8.86s/it]
 93%|█████████▎| 208237/225000 [401:26:29<45:02:53,  9.67s/it]
 93%|█████████▎| 208238/225000 [401:26:34<39:15:39,  8.43s/it]
 93%|█████████▎| 208239/225000 [401:26:47<45:10:26,  9.70s/it]
 93%|█████████▎| 208240/225000 [401:26:57<45:14:48,  9.72s/it]
 93%|█████████▎| 208241/225000 [401:27:04<41:37:08,  8.94s/it]
 93%|█████████▎| 208242/225000 [401:27:16<47:01:01, 10.10s/it]
 93%|█████████▎| 208243/225000 [401:27:26<46:08:13,  9.91s/it]
 93%|█████████▎| 208244/225000 [401:27:32<40:52:40,  8.78s/it]
 93%|█████████▎| 208245/225000 [401:27:44<45:18:58,  9.74s/it]
 93%|█████████▎| 208246/225000 [401:27:53<44:09:22,  9.49s/it]
 93%|█████████▎| 208247/225000 [401:27:59<39:27:59,  8.48s/it]
 93%|█████████▎| 208248/225000 [401:28:08<39:47:44,  8.55s/it]
 93%|█████████▎| 208249/225000 [401:28:13<35:05:18,  7.54s/it]
 93%|█████████▎| 208250/225000 [401:28:24<39:35:05,  8.51s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.541', 'learning_rate': '7.537e-07', 'epoch': '0.9256', 'num_input_tokens_seen': 678072736, 'train_runtime': '1.445e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208250/225000 [401:28:24<39:35:05,  8.51s/it]
 93%|█████████▎| 208251/225000 [401:28:30<36:44:45,  7.90s/it]
 93%|█████████▎| 208252/225000 [401:28:38<36:12:14,  7.78s/it]
 93%|█████████▎| 208253/225000 [401:28:45<35:40:58,  7.67s/it]
 93%|█████████▎| 208254/225000 [401:29:02<47:53:19, 10.29s/it]
 93%|█████████▎| 208255/225000 [401:29:08<42:04:14,  9.04s/it]
 93%|█████████▎| 208256/225000 [401:29:15<39:44:10,  8.54s/it]
 93%|█████████▎| 208257/225000 [401:29:29<46:48:43, 10.07s/it]
 93%|█████████▎| 208258/225000 [401:29:38<45:33:54,  9.80s/it]
 93%|█████████▎| 208259/225000 [401:29:46<43:24:00,  9.33s/it]
 93%|█████████▎| 208260/225000 [401:29:53<39:33:31,  8.51s/it]
 93%|█████████▎| 208261/225000 [401:30:05<44:40:40,  9.61s/it]
 93%|█████████▎| 208262/225000 [401:30:13<42:40:17,  9.18s/it]
 93%|█████████▎| 208263/225000 [401:30:19<38:24:59,  8.26s/it]
 93%|█████████▎| 208264/225000 [401:30:29<40:27:21,  8.70s/it]
 93%|█████████▎| 208265/225000 [401:30:44<49:54:06, 10.73s/it]
 93%|█████████▎| 208266/225000 [401:30:50<42:29:32,  9.14s/it]
 93%|█████████▎| 208267/225000 [401:31:02<46:49:10, 10.07s/it]
 93%|█████████▎| 208268/225000 [401:31:07<40:00:17,  8.61s/it]
 93%|█████████▎| 208269/225000 [401:31:14<36:46:39,  7.91s/it]
 93%|█████████▎| 208270/225000 [401:31:20<34:30:09,  7.42s/it]
 93%|█████████▎| 208271/225000 [401:31:32<41:35:00,  8.95s/it]
 93%|█████████▎| 208272/225000 [401:31:41<40:40:46,  8.75s/it]
 93%|█████████▎| 208273/225000 [401:31:45<35:08:29,  7.56s/it]
 93%|█████████▎| 208274/225000 [401:31:55<38:39:25,  8.32s/it]
 93%|█████████▎| 208275/225000 [401:32:04<39:25:54,  8.49s/it]
 93%|█████████▎| 208276/225000 [401:32:10<35:34:47,  7.66s/it]
 93%|█████████▎| 208277/225000 [401:32:17<34:42:45,  7.47s/it]
 93%|█████████▎| 208278/225000 [401:32:30<42:04:19,  9.06s/it]
 93%|█████████▎| 208279/225000 [401:32:39<41:48:26,  9.00s/it]
 93%|█████████▎| 208280/225000 [401:32:49<43:58:20,  9.47s/it]
 93%|█████████▎| 208281/225000 [401:32:54<37:16:24,  8.03s/it]
 93%|█████████▎| 208282/225000 [401:33:01<36:08:16,  7.78s/it]
 93%|█████████▎| 208283/225000 [401:33:07<34:02:36,  7.33s/it]
 93%|█████████▎| 208284/225000 [401:33:13<31:57:51,  6.88s/it]
 93%|█████████▎| 208285/225000 [401:33:28<43:28:46,  9.36s/it]
 93%|█████████▎| 208286/225000 [401:33:35<39:40:04,  8.54s/it]
 93%|█████████▎| 208287/225000 [401:33:45<41:27:02,  8.93s/it]
 93%|█████████▎| 208288/225000 [401:33:56<43:51:17,  9.45s/it]
 93%|█████████▎| 208289/225000 [401:34:01<38:09:02,  8.22s/it]
 93%|█████████▎| 208290/225000 [401:34:12<41:54:42,  9.03s/it]
 93%|█████████▎| 208291/225000 [401:34:18<37:43:48,  8.13s/it]
 93%|█████████▎| 208292/225000 [401:34:24<35:26:34,  7.64s/it]
 93%|█████████▎| 208293/225000 [401:34:33<37:10:31,  8.01s/it]
 93%|█████████▎| 208294/225000 [401:34:42<38:08:20,  8.22s/it]
 93%|█████████▎| 208295/225000 [401:34:53<42:24:21,  9.14s/it]
 93%|█████████▎| 208296/225000 [401:34:58<36:01:47,  7.77s/it]
 93%|█████████▎| 208297/225000 [401:35:08<39:10:45,  8.44s/it]
 93%|█████████▎| 208298/225000 [401:35:17<40:45:23,  8.78s/it]
 93%|█████████▎| 208299/225000 [401:35:23<35:47:22,  7.71s/it]
 93%|█████████▎| 208300/225000 [401:35:31<36:56:55,  7.97s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.111', 'learning_rate': '7.493e-07', 'epoch': '0.9258', 'num_input_tokens_seen': 678303672, 'train_runtime': '1.446e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208300/225000 [401:35:31<36:56:55,  7.97s/it]
 93%|█████████▎| 208301/225000 [401:35:38<35:10:25,  7.58s/it]
 93%|█████████▎| 208302/225000 [401:35:45<34:09:20,  7.36s/it]
 93%|█████████▎| 208303/225000 [401:35:58<42:04:52,  9.07s/it]
 93%|█████████▎| 208304/225000 [401:36:04<38:12:42,  8.24s/it]
 93%|█████████▎| 208305/225000 [401:36:19<47:28:47, 10.24s/it]
 93%|█████████▎| 208306/225000 [401:36:26<43:19:07,  9.34s/it]
 93%|█████████▎| 208307/225000 [401:36:38<46:43:42, 10.08s/it]
 93%|█████████▎| 208308/225000 [401:36:45<42:32:06,  9.17s/it]
 93%|█████████▎| 208309/225000 [401:36:53<41:16:21,  8.90s/it]
 93%|█████████▎| 208310/225000 [401:37:03<42:38:17,  9.20s/it]
 93%|█████████▎| 208311/225000 [401:37:13<42:58:36,  9.27s/it]
 93%|█████████▎| 208312/225000 [401:37:21<41:11:49,  8.89s/it]
 93%|█████████▎| 208313/225000 [401:37:28<39:17:04,  8.48s/it]
 93%|█████████▎| 208314/225000 [401:37:39<42:41:41,  9.21s/it]
 93%|█████████▎| 208315/225000 [401:37:48<42:44:25,  9.22s/it]
 93%|█████████▎| 208316/225000 [401:37:58<43:13:45,  9.33s/it]
 93%|█████████▎| 208317/225000 [401:38:05<39:52:57,  8.61s/it]
 93%|█████████▎| 208318/225000 [401:38:17<45:09:22,  9.74s/it]
 93%|█████████▎| 208319/225000 [401:38:28<47:05:34, 10.16s/it]
 93%|█████████▎| 208320/225000 [401:38:35<42:05:45,  9.09s/it]
 93%|█████████▎| 208321/225000 [401:38:44<42:42:02,  9.22s/it]
 93%|█████████▎| 208322/225000 [401:38:53<41:48:28,  9.02s/it]
 93%|█████████▎| 208323/225000 [401:39:04<44:31:21,  9.61s/it]
 93%|█████████▎| 208324/225000 [401:39:12<42:38:23,  9.21s/it]
 93%|█████████▎| 208325/225000 [401:39:19<39:02:10,  8.43s/it]
 93%|█████████▎| 208326/225000 [401:39:25<35:19:04,  7.63s/it]
 93%|█████████▎| 208327/225000 [401:39:36<41:03:27,  8.87s/it]
 93%|█████████▎| 208328/225000 [401:39:47<43:08:47,  9.32s/it]
 93%|█████████▎| 208329/225000 [401:39:53<38:56:39,  8.41s/it]
 93%|█████████▎| 208330/225000 [401:40:02<40:02:06,  8.65s/it]
 93%|█████████▎| 208331/225000 [401:40:12<41:48:08,  9.03s/it]
 93%|█████████▎| 208332/225000 [401:40:21<41:30:21,  8.96s/it]
 93%|█████████▎| 208333/225000 [401:40:29<40:21:56,  8.72s/it]
 93%|█████████▎| 208334/225000 [401:40:36<38:16:56,  8.27s/it]
 93%|█████████▎| 208335/225000 [401:40:48<43:00:43,  9.29s/it]
 93%|█████████▎| 208336/225000 [401:40:56<41:05:06,  8.88s/it]
 93%|█████████▎| 208337/225000 [401:41:06<42:42:36,  9.23s/it]
 93%|█████████▎| 208338/225000 [401:41:15<41:40:22,  9.00s/it]
 93%|█████████▎| 208339/225000 [401:41:23<40:50:38,  8.83s/it]
 93%|█████████▎| 208340/225000 [401:41:30<38:15:16,  8.27s/it]
 93%|█████████▎| 208341/225000 [401:41:42<43:30:50,  9.40s/it]
 93%|█████████▎| 208342/225000 [401:41:52<43:46:43,  9.46s/it]
 93%|█████████▎| 208343/225000 [401:42:02<45:24:31,  9.81s/it]
 93%|█████████▎| 208344/225000 [401:42:07<38:04:44,  8.23s/it]
 93%|█████████▎| 208345/225000 [401:42:13<35:07:15,  7.59s/it]
 93%|█████████▎| 208346/225000 [401:42:25<41:10:17,  8.90s/it]
 93%|█████████▎| 208347/225000 [401:42:35<43:30:33,  9.41s/it]
 93%|█████████▎| 208348/225000 [401:42:44<43:01:32,  9.30s/it]
 93%|█████████▎| 208349/225000 [401:42:53<41:50:27,  9.05s/it]
 93%|█████████▎| 208350/225000 [401:42:59<38:00:45,  8.22s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.356', 'learning_rate': '7.449e-07', 'epoch': '0.926', 'num_input_tokens_seen': 678540344, 'train_runtime': '1.446e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208350/225000 [401:42:59<38:00:45,  8.22s/it]
 93%|█████████▎| 208351/225000 [401:43:16<49:45:23, 10.76s/it]
 93%|█████████▎| 208352/225000 [401:43:22<43:56:25,  9.50s/it]
 93%|█████████▎| 208353/225000 [401:43:32<43:58:14,  9.51s/it]
 93%|█████████▎| 208354/225000 [401:43:42<44:23:54,  9.60s/it]
 93%|█████████▎| 208355/225000 [401:43:50<41:59:43,  9.08s/it]
 93%|█████████▎| 208356/225000 [401:44:01<45:03:17,  9.75s/it]
 93%|█████████▎| 208357/225000 [401:44:07<40:01:20,  8.66s/it]
 93%|█████████▎| 208358/225000 [401:44:16<40:23:49,  8.74s/it]
 93%|█████████▎| 208359/225000 [401:44:24<39:25:40,  8.53s/it]
 93%|█████████▎| 208360/225000 [401:44:30<36:24:30,  7.88s/it]
 93%|█████████▎| 208361/225000 [401:44:43<42:32:30,  9.20s/it]
 93%|█████████▎| 208362/225000 [401:44:50<39:33:33,  8.56s/it]
 93%|█████████▎| 208363/225000 [401:44:57<37:19:44,  8.08s/it]
 93%|█████████▎| 208364/225000 [401:45:10<44:02:03,  9.53s/it]
 93%|█████████▎| 208365/225000 [401:45:16<40:18:14,  8.72s/it]
 93%|█████████▎| 208366/225000 [401:45:29<46:05:55,  9.98s/it]
 93%|█████████▎| 208367/225000 [401:45:38<43:58:56,  9.52s/it]
 93%|█████████▎| 208368/225000 [401:45:47<43:14:53,  9.36s/it]
 93%|█████████▎| 208369/225000 [401:45:57<44:48:04,  9.70s/it]
 93%|█████████▎| 208370/225000 [401:46:03<39:02:39,  8.45s/it]
 93%|█████████▎| 208371/225000 [401:46:09<35:22:45,  7.66s/it]
 93%|█████████▎| 208372/225000 [401:46:18<38:05:46,  8.25s/it]
 93%|█████████▎| 208373/225000 [401:46:28<40:29:16,  8.77s/it]
 93%|█████████▎| 208374/225000 [401:46:36<38:48:17,  8.40s/it]
 93%|█████████▎| 208375/225000 [401:46:43<36:59:56,  8.01s/it]
 93%|█████████▎| 208376/225000 [401:46:51<37:43:09,  8.17s/it]
 93%|█████████▎| 208377/225000 [401:46:57<33:59:27,  7.36s/it]
 93%|█████████▎| 208378/225000 [401:47:10<41:38:48,  9.02s/it]
 93%|█████████▎| 208379/225000 [401:47:18<40:51:05,  8.85s/it]
 93%|█████████▎| 208380/225000 [401:47:27<40:21:44,  8.74s/it]
 93%|█████████▎| 208381/225000 [401:47:34<38:03:26,  8.24s/it]
 93%|█████████▎| 208382/225000 [401:47:44<41:22:07,  8.96s/it]
 93%|█████████▎| 208383/225000 [401:47:53<40:30:13,  8.77s/it]
 93%|█████████▎| 208384/225000 [401:48:07<48:03:11, 10.41s/it]
 93%|█████████▎| 208385/225000 [401:48:13<42:14:26,  9.15s/it]
 93%|█████████▎| 208386/225000 [401:48:19<37:51:22,  8.20s/it]
 93%|█████████▎| 208387/225000 [401:48:36<49:30:53, 10.73s/it]
 93%|█████████▎| 208388/225000 [401:48:42<42:56:29,  9.31s/it]
 93%|█████████▎| 208389/225000 [401:48:49<40:13:22,  8.72s/it]
 93%|█████████▎| 208390/225000 [401:48:57<39:03:26,  8.47s/it]
 93%|█████████▎| 208391/225000 [401:49:04<36:30:18,  7.91s/it]
 93%|█████████▎| 208392/225000 [401:49:11<36:06:40,  7.83s/it]
 93%|█████████▎| 208393/225000 [401:49:18<34:48:08,  7.54s/it]
 93%|█████████▎| 208394/225000 [401:49:32<44:07:15,  9.56s/it]
 93%|█████████▎| 208395/225000 [401:49:41<42:44:12,  9.27s/it]
 93%|█████████▎| 208396/225000 [401:49:48<39:48:57,  8.63s/it]
 93%|█████████▎| 208397/225000 [401:50:03<47:46:02, 10.36s/it]
 93%|█████████▎| 208398/225000 [401:50:08<41:22:21,  8.97s/it]
 93%|█████████▎| 208399/225000 [401:50:15<37:46:51,  8.19s/it]
 93%|█████████▎| 208400/225000 [401:50:24<39:46:47,  8.63s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.511', 'learning_rate': '7.405e-07', 'epoch': '0.9262', 'num_input_tokens_seen': 678771456, 'train_runtime': '1.447e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208400/225000 [401:50:24<39:46:47,  8.63s/it]
 93%|█████████▎| 208401/225000 [401:50:31<37:42:23,  8.18s/it]
 93%|█████████▎| 208402/225000 [401:50:39<37:22:34,  8.11s/it]
 93%|█████████▎| 208403/225000 [401:50:47<36:18:24,  7.88s/it]
 93%|█████████▎| 208404/225000 [401:50:53<33:29:07,  7.26s/it]
 93%|█████████▎| 208405/225000 [401:51:03<37:22:18,  8.11s/it]
 93%|█████████▎| 208406/225000 [401:51:21<51:38:43, 11.20s/it]
 93%|█████████▎| 208407/225000 [401:51:27<44:13:52,  9.60s/it]
 93%|█████████▎| 208408/225000 [401:51:37<44:30:27,  9.66s/it]
 93%|█████████▎| 208409/225000 [401:51:46<44:09:55,  9.58s/it]
 93%|█████████▎| 208410/225000 [401:51:56<44:42:51,  9.70s/it]
 93%|█████████▎| 208411/225000 [401:52:05<43:08:42,  9.36s/it]
 93%|█████████▎| 208412/225000 [401:52:10<37:51:29,  8.22s/it]
 93%|█████████▎| 208413/225000 [401:52:20<40:05:44,  8.70s/it]
 93%|█████████▎| 208414/225000 [401:52:27<37:06:18,  8.05s/it]
 93%|█████████▎| 208415/225000 [401:52:39<42:34:47,  9.24s/it]
 93%|█████████▎| 208416/225000 [401:52:46<39:55:54,  8.67s/it]
 93%|█████████▎| 208417/225000 [401:52:52<36:09:00,  7.85s/it]
 93%|█████████▎| 208418/225000 [401:53:00<36:44:57,  7.98s/it]
 93%|█████████▎| 208419/225000 [401:53:12<41:37:18,  9.04s/it]
 93%|█████████▎| 208420/225000 [401:53:20<40:21:44,  8.76s/it]
 93%|█████████▎| 208421/225000 [401:53:30<42:15:53,  9.18s/it]
 93%|█████████▎| 208422/225000 [401:53:34<35:54:23,  7.80s/it]
 93%|█████████▎| 208423/225000 [401:53:43<37:13:26,  8.08s/it]
 93%|█████████▎| 208424/225000 [401:53:49<33:58:27,  7.38s/it]
 93%|█████████▎| 208425/225000 [401:53:55<32:28:02,  7.05s/it]
 93%|█████████▎| 208426/225000 [401:54:02<31:37:45,  6.87s/it]
 93%|█████████▎| 208427/225000 [401:54:15<40:56:56,  8.89s/it]
 93%|█████████▎| 208428/225000 [401:54:24<40:33:44,  8.81s/it]
 93%|█████████▎| 208429/225000 [401:54:30<37:08:09,  8.07s/it]
 93%|█████████▎| 208430/225000 [401:54:42<42:46:40,  9.29s/it]
 93%|█████████▎| 208431/225000 [401:54:53<44:30:02,  9.67s/it]
 93%|█████████▎| 208432/225000 [401:54:59<38:48:57,  8.43s/it]
 93%|█████████▎| 208433/225000 [401:55:06<37:14:04,  8.09s/it]
 93%|█████████▎| 208434/225000 [401:55:12<35:06:36,  7.63s/it]
 93%|█████████▎| 208435/225000 [401:55:21<36:39:18,  7.97s/it]
 93%|█████████▎| 208436/225000 [401:55:35<45:07:23,  9.81s/it]
 93%|█████████▎| 208437/225000 [401:55:45<44:57:00,  9.77s/it]
 93%|█████████▎| 208438/225000 [401:55:58<48:56:29, 10.64s/it]
 93%|█████████▎| 208439/225000 [401:56:05<44:35:40,  9.69s/it]
 93%|█████████▎| 208440/225000 [401:56:14<43:58:50,  9.56s/it]
 93%|█████████▎| 208441/225000 [401:56:22<41:49:34,  9.09s/it]
 93%|█████████▎| 208442/225000 [401:56:30<39:56:17,  8.68s/it]
 93%|█████████▎| 208443/225000 [401:56:41<42:53:24,  9.33s/it]
 93%|█████████▎| 208444/225000 [401:56:48<39:21:59,  8.56s/it]
 93%|█████████▎| 208445/225000 [401:57:00<44:40:56,  9.72s/it]
 93%|█████████▎| 208446/225000 [401:57:09<44:03:55,  9.58s/it]
 93%|█████████▎| 208447/225000 [401:57:19<44:12:28,  9.61s/it]
 93%|█████████▎| 208448/225000 [401:57:29<45:07:02,  9.81s/it]
 93%|█████████▎| 208449/225000 [401:57:36<40:55:59,  8.90s/it]
 93%|█████████▎| 208450/225000 [401:57:41<35:44:30,  7.77s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.476', 'learning_rate': '7.361e-07', 'epoch': '0.9264', 'num_input_tokens_seen': 679002704, 'train_runtime': '1.447e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208450/225000 [401:57:41<35:44:30,  7.77s/it]
 93%|█████████▎| 208451/225000 [401:57:47<33:40:56,  7.33s/it]
 93%|█████████▎| 208452/225000 [401:58:00<40:57:52,  8.91s/it]
 93%|█████████▎| 208453/225000 [401:58:07<37:39:44,  8.19s/it]
 93%|█████████▎| 208454/225000 [401:58:15<38:00:13,  8.27s/it]
 93%|█████████▎| 208455/225000 [401:58:24<38:46:22,  8.44s/it]
 93%|█████████▎| 208456/225000 [401:58:35<42:03:06,  9.15s/it]
 93%|█████████▎| 208457/225000 [401:58:42<39:48:17,  8.66s/it]
 93%|█████████▎| 208458/225000 [401:58:49<37:25:27,  8.14s/it]
 93%|█████████▎| 208459/225000 [401:58:55<34:53:59,  7.60s/it]
 93%|█████████▎| 208460/225000 [401:59:05<38:05:46,  8.29s/it]
 93%|█████████▎| 208461/225000 [401:59:17<42:00:54,  9.15s/it]
 93%|█████████▎| 208462/225000 [401:59:24<39:12:57,  8.54s/it]
 93%|█████████▎| 208463/225000 [401:59:32<39:28:14,  8.59s/it]
 93%|█████████▎| 208464/225000 [401:59:43<42:24:41,  9.23s/it]
 93%|█████████▎| 208465/225000 [401:59:49<37:53:36,  8.25s/it]
 93%|█████████▎| 208466/225000 [402:00:03<45:49:52,  9.98s/it]
 93%|█████████▎| 208467/225000 [402:00:09<40:36:19,  8.84s/it]
 93%|█████████▎| 208468/225000 [402:00:21<44:16:40,  9.64s/it]
 93%|█████████▎| 208469/225000 [402:00:27<39:44:33,  8.65s/it]
 93%|█████████▎| 208470/225000 [402:00:33<35:21:54,  7.70s/it]
 93%|█████████▎| 208471/225000 [402:00:40<35:18:28,  7.69s/it]
 93%|█████████▎| 208472/225000 [402:00:56<46:50:43, 10.20s/it]
 93%|█████████▎| 208473/225000 [402:01:03<41:56:21,  9.14s/it]
 93%|█████████▎| 208474/225000 [402:01:13<43:44:28,  9.53s/it]
 93%|█████████▎| 208475/225000 [402:01:24<45:47:23,  9.98s/it]
 93%|█████████▎| 208476/225000 [402:01:36<48:35:31, 10.59s/it]
 93%|█████████▎| 208477/225000 [402:01:50<52:13:45, 11.38s/it]
 93%|█████████▎| 208478/225000 [402:01:54<43:02:52,  9.38s/it]
 93%|█████████▎| 208479/225000 [402:02:02<41:05:40,  8.95s/it]
 93%|█████████▎| 208480/225000 [402:02:09<37:30:03,  8.17s/it]
 93%|█████████▎| 208481/225000 [402:02:15<35:31:02,  7.74s/it]
 93%|█████████▎| 208482/225000 [402:02:24<36:06:34,  7.87s/it]
 93%|█████████▎| 208483/225000 [402:02:39<46:23:24, 10.11s/it]
 93%|█████████▎| 208484/225000 [402:02:46<42:51:52,  9.34s/it]
 93%|█████████▎| 208485/225000 [402:02:53<39:23:28,  8.59s/it]
 93%|█████████▎| 208486/225000 [402:03:07<46:05:27, 10.05s/it]
 93%|█████████▎| 208487/225000 [402:03:18<47:10:13, 10.28s/it]
 93%|█████████▎| 208488/225000 [402:03:24<42:19:26,  9.23s/it]
 93%|█████████▎| 208489/225000 [402:03:35<44:14:00,  9.64s/it]
 93%|█████████▎| 208490/225000 [402:03:43<41:52:06,  9.13s/it]
 93%|█████████▎| 208491/225000 [402:03:50<39:28:22,  8.61s/it]
 93%|█████████▎| 208492/225000 [402:04:02<43:19:12,  9.45s/it]
 93%|█████████▎| 208493/225000 [402:04:12<44:03:20,  9.61s/it]
 93%|█████████▎| 208494/225000 [402:04:18<38:51:36,  8.48s/it]
 93%|█████████▎| 208495/225000 [402:04:23<34:53:36,  7.61s/it]
 93%|█████████▎| 208496/225000 [402:04:31<34:39:56,  7.56s/it]
 93%|█████████▎| 208497/225000 [402:04:44<42:54:14,  9.36s/it]
 93%|█████████▎| 208498/225000 [402:04:51<39:11:49,  8.55s/it]
 93%|█████████▎| 208499/225000 [402:04:59<38:32:54,  8.41s/it]
 93%|█████████▎| 208500/225000 [402:05:06<37:22:50,  8.16s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.39', 'learning_rate': '7.317e-07', 'epoch': '0.9267', 'num_input_tokens_seen': 679235872, 'train_runtime': '1.448e+06', 'train_tokens_per_second': '469.2'}

 93%|█████████▎| 208500/225000 [402:05:06<37:22:50,  8.16s/it]
 93%|█████████▎| 208501/225000 [402:05:12<34:06:27,  7.44s/it]
 93%|█████████▎| 208502/225000 [402:05:22<36:59:56,  8.07s/it]
 93%|█████████▎| 208503/225000 [402:05:33<41:04:16,  8.96s/it]
 93%|█████████▎| 208504/225000 [402:05:40<38:28:20,  8.40s/it]
 93%|█████████▎| 208505/225000 [402:05:49<39:14:16,  8.56s/it]
 93%|█████████▎| 208506/225000 [402:05:56<37:28:33,  8.18s/it]
 93%|█████████▎| 208507/225000 [402:06:04<37:21:01,  8.15s/it]
 93%|█████████▎| 208508/225000 [402:06:16<41:44:08,  9.11s/it]
 93%|█████████▎| 208509/225000 [402:06:24<40:36:52,  8.87s/it]
 93%|█████████▎| 208510/225000 [402:06:31<37:58:32,  8.29s/it]
 93%|█████████▎| 208511/225000 [402:06:43<43:11:39,  9.43s/it]
 93%|█████████▎| 208512/225000 [402:06:50<40:38:36,  8.87s/it]
 93%|█████████▎| 208513/225000 [402:07:00<41:47:46,  9.13s/it]
 93%|█████████▎| 208514/225000 [402:07:10<42:25:37,  9.26s/it]
 93%|█████████▎| 208515/225000 [402:07:22<46:48:19, 10.22s/it]
 93%|█████████▎| 208516/225000 [402:07:31<44:15:50,  9.67s/it]
 93%|█████████▎| 208517/225000 [402:07:38<41:01:38,  8.96s/it]
 93%|█████████▎| 208518/225000 [402:07:47<41:21:21,  9.03s/it]
 93%|█████████▎| 208519/225000 [402:07:53<36:27:59,  7.97s/it]
 93%|█████████▎| 208520/225000 [402:08:05<43:10:53,  9.43s/it]
 93%|█████████▎| 208521/225000 [402:08:14<42:07:37,  9.20s/it]
 93%|█████████▎| 208522/225000 [402:08:21<38:50:54,  8.49s/it]
 93%|█████████▎| 208523/225000 [402:08:28<36:33:16,  7.99s/it]
 93%|█████████▎| 208524/225000 [402:08:35<36:02:18,  7.87s/it]
 93%|█████████▎| 208525/225000 [402:08:43<35:45:49,  7.81s/it]
 93%|█████████▎| 208526/225000 [402:08:54<40:28:08,  8.84s/it]
 93%|█████████▎| 208527/225000 [402:09:01<37:04:39,  8.10s/it]
 93%|█████████▎| 208528/225000 [402:09:15<46:11:12, 10.09s/it]
 93%|█████████▎| 208529/225000 [402:09:22<41:46:19,  9.13s/it]
 93%|█████████▎| 208530/225000 [402:09:36<47:48:00, 10.45s/it]
 93%|█████████▎| 208531/225000 [402:09:41<40:46:40,  8.91s/it]
 93%|█████████▎| 208532/225000 [402:09:48<38:03:54,  8.32s/it]
 93%|█████████▎| 208533/225000 [402:09:54<35:16:42,  7.71s/it]
 93%|█████████▎| 208534/225000 [402:10:12<49:06:30, 10.74s/it]
 93%|█████████▎| 208535/225000 [402:10:21<47:06:52, 10.30s/it]
 93%|█████████▎| 208536/225000 [402:10:31<45:36:20,  9.97s/it]
 93%|█████████▎| 208537/225000 [402:10:39<43:54:29,  9.60s/it]
 93%|█████████▎| 208538/225000 [402:10:46<40:21:30,  8.83s/it]
 93%|█████████▎| 208539/225000 [402:10:56<41:17:59,  9.03s/it]
 93%|█████████▎| 208540/225000 [402:11:05<40:44:14,  8.91s/it]
 93%|█████████▎| 208541/225000 [402:11:11<37:27:24,  8.19s/it]
 93%|█████████▎| 208542/225000 [402:11:27<48:06:39, 10.52s/it]
 93%|█████████▎| 208543/225000 [402:11:36<45:26:06,  9.94s/it]
 93%|█████████▎| 208544/225000 [402:11:43<41:41:46,  9.12s/it]
 93%|█████████▎| 208545/225000 [402:11:53<42:38:22,  9.33s/it]
 93%|█████████▎| 208546/225000 [402:12:01<41:01:51,  8.98s/it]
 93%|█████████▎| 208547/225000 [402:12:08<38:25:10,  8.41s/it]
 93%|█████████▎| 208548/225000 [402:12:14<35:16:37,  7.72s/it]
 93%|█████████▎| 208549/225000 [402:12:20<33:19:00,  7.29s/it]
 93%|█████████▎| 208550/225000 [402:12:26<31:36:45,  6.92s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.764', 'learning_rate': '7.273e-07', 'epoch': '0.9269', 'num_input_tokens_seen': 679465808, 'train_runtime': '1.448e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208550/225000 [402:12:26<31:36:45,  6.92s/it]
 93%|█████████▎| 208551/225000 [402:12:37<36:20:14,  7.95s/it]
 93%|█████████▎| 208552/225000 [402:12:53<48:29:58, 10.62s/it]
 93%|█████████▎| 208553/225000 [402:12:59<40:53:02,  8.95s/it]
 93%|█████████▎| 208554/225000 [402:13:05<38:01:26,  8.32s/it]
 93%|█████████▎| 208555/225000 [402:13:12<35:12:01,  7.71s/it]
 93%|█████████▎| 208556/225000 [402:13:27<44:58:02,  9.84s/it]
 93%|█████████▎| 208557/225000 [402:13:31<37:18:37,  8.17s/it]
 93%|█████████▎| 208558/225000 [402:13:39<37:05:02,  8.12s/it]
 93%|█████████▎| 208559/225000 [402:13:53<44:47:01,  9.81s/it]
 93%|█████████▎| 208560/225000 [402:14:00<41:52:28,  9.17s/it]
 93%|█████████▎| 208561/225000 [402:14:05<36:29:36,  7.99s/it]
 93%|█████████▎| 208562/225000 [402:14:19<43:53:14,  9.61s/it]
 93%|█████████▎| 208563/225000 [402:14:26<40:59:45,  8.98s/it]
 93%|█████████▎| 208564/225000 [402:14:35<40:08:17,  8.79s/it]
 93%|█████████▎| 208565/225000 [402:14:42<37:44:23,  8.27s/it]
 93%|█████████▎| 208566/225000 [402:14:58<49:08:54, 10.77s/it]
 93%|█████████▎| 208567/225000 [402:15:02<40:05:18,  8.78s/it]
 93%|█████████▎| 208568/225000 [402:15:11<40:12:32,  8.81s/it]
 93%|█████████▎| 208569/225000 [402:15:21<41:07:03,  9.01s/it]
 93%|█████████▎| 208570/225000 [402:15:27<37:19:31,  8.18s/it]
 93%|█████████▎| 208571/225000 [402:15:37<39:05:04,  8.56s/it]
 93%|█████████▎| 208572/225000 [402:15:47<41:19:07,  9.05s/it]
 93%|█████████▎| 208573/225000 [402:15:51<34:54:15,  7.65s/it]
 93%|█████████▎| 208574/225000 [402:15:57<32:57:22,  7.22s/it]
 93%|█████████▎| 208575/225000 [402:16:10<40:59:46,  8.99s/it]
 93%|█████████▎| 208576/225000 [402:16:15<34:46:38,  7.62s/it]
 93%|█████████▎| 208577/225000 [402:16:21<33:10:49,  7.27s/it]
 93%|█████████▎| 208578/225000 [402:16:29<33:17:01,  7.30s/it]
 93%|█████████▎| 208579/225000 [402:16:36<32:40:37,  7.16s/it]
 93%|█████████▎| 208580/225000 [402:16:42<31:07:44,  6.82s/it]
 93%|█████████▎| 208581/225000 [402:16:56<41:09:08,  9.02s/it]
 93%|█████████▎| 208582/225000 [402:17:01<35:48:21,  7.85s/it]
 93%|█████████▎| 208583/225000 [402:17:07<34:03:03,  7.47s/it]
 93%|█████████▎| 208584/225000 [402:17:22<43:53:50,  9.63s/it]
 93%|█████████▎| 208585/225000 [402:17:27<38:06:34,  8.36s/it]
 93%|█████████▎| 208586/225000 [402:17:33<33:49:06,  7.42s/it]
 93%|█████████▎| 208587/225000 [402:17:43<37:37:51,  8.25s/it]
 93%|█████████▎| 208588/225000 [402:17:53<40:13:30,  8.82s/it]
 93%|█████████▎| 208589/225000 [402:18:04<42:47:32,  9.39s/it]
 93%|█████████▎| 208590/225000 [402:18:12<41:18:04,  9.06s/it]
 93%|█████████▎| 208591/225000 [402:18:20<40:14:02,  8.83s/it]
 93%|█████████▎| 208592/225000 [402:18:27<37:38:53,  8.26s/it]
 93%|█████████▎| 208593/225000 [402:18:41<44:44:14,  9.82s/it]
 93%|█████████▎| 208594/225000 [402:18:49<43:16:29,  9.50s/it]
 93%|█████████▎| 208595/225000 [402:18:59<43:09:57,  9.47s/it]
 93%|█████████▎| 208596/225000 [402:19:09<44:35:44,  9.79s/it]
 93%|█████████▎| 208597/225000 [402:19:15<39:07:33,  8.59s/it]
 93%|█████████▎| 208598/225000 [402:19:25<40:38:54,  8.92s/it]
 93%|█████████▎| 208599/225000 [402:19:38<46:01:35, 10.10s/it]
 93%|█████████▎| 208600/225000 [402:19:48<46:13:58, 10.15s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.347', 'learning_rate': '7.229e-07', 'epoch': '0.9271', 'num_input_tokens_seen': 679694152, 'train_runtime': '1.448e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208600/225000 [402:19:48<46:13:58, 10.15s/it]
 93%|█████████▎| 208601/225000 [402:19:58<45:49:57, 10.06s/it]
 93%|█████████▎| 208602/225000 [402:20:05<42:16:35,  9.28s/it]
 93%|█████████▎| 208603/225000 [402:20:12<39:05:26,  8.58s/it]
 93%|█████████▎| 208604/225000 [402:20:23<41:52:51,  9.20s/it]
 93%|█████████▎| 208605/225000 [402:20:32<41:41:34,  9.15s/it]
 93%|█████████▎| 208606/225000 [402:20:39<39:04:24,  8.58s/it]
 93%|█████████▎| 208607/225000 [402:20:53<45:52:34, 10.07s/it]
 93%|█████████▎| 208608/225000 [402:21:00<41:19:08,  9.07s/it]
 93%|█████████▎| 208609/225000 [402:21:11<45:07:12,  9.91s/it]
 93%|█████████▎| 208610/225000 [402:21:24<48:12:39, 10.59s/it]
 93%|█████████▎| 208611/225000 [402:21:33<46:32:12, 10.22s/it]
 93%|█████████▎| 208612/225000 [402:21:42<45:25:46,  9.98s/it]
 93%|█████████▎| 208613/225000 [402:21:50<42:33:04,  9.35s/it]
 93%|█████████▎| 208614/225000 [402:21:56<37:57:39,  8.34s/it]
 93%|█████████▎| 208615/225000 [402:22:04<37:24:38,  8.22s/it]
 93%|█████████▎| 208616/225000 [402:22:10<33:52:00,  7.44s/it]
 93%|█████████▎| 208617/225000 [402:22:20<37:19:34,  8.20s/it]
 93%|█████████▎| 208618/225000 [402:22:27<35:46:18,  7.86s/it]
 93%|█████████▎| 208619/225000 [402:22:38<39:58:28,  8.79s/it]
 93%|█████████▎| 208620/225000 [402:22:50<44:39:56,  9.82s/it]
 93%|█████████▎| 208621/225000 [402:23:02<47:48:38, 10.51s/it]
 93%|█████████▎| 208622/225000 [402:23:09<42:19:27,  9.30s/it]
 93%|█████████▎| 208623/225000 [402:23:19<43:53:41,  9.65s/it]
 93%|█████████▎| 208624/225000 [402:23:27<42:08:06,  9.26s/it]
 93%|█████████▎| 208625/225000 [402:23:36<41:10:52,  9.05s/it]
 93%|█████████▎| 208626/225000 [402:23:49<47:15:21, 10.39s/it]
 93%|█████████▎| 208627/225000 [402:23:58<44:18:50,  9.74s/it]
 93%|█████████▎| 208628/225000 [402:24:11<49:29:18, 10.88s/it]
 93%|█████████▎| 208629/225000 [402:24:18<44:04:01,  9.69s/it]
 93%|█████████▎| 208630/225000 [402:24:27<42:24:56,  9.33s/it]
 93%|█████████▎| 208631/225000 [402:24:38<44:32:43,  9.80s/it]
 93%|█████████▎| 208632/225000 [402:24:44<40:17:51,  8.86s/it]
 93%|█████████▎| 208633/225000 [402:24:52<39:05:55,  8.60s/it]
 93%|█████████▎| 208634/225000 [402:24:58<34:51:31,  7.67s/it]
 93%|█████████▎| 208635/225000 [402:25:10<40:46:54,  8.97s/it]
 93%|█████████▎| 208636/225000 [402:25:17<38:02:31,  8.37s/it]
 93%|█████████▎| 208637/225000 [402:25:24<36:51:44,  8.11s/it]
 93%|█████████▎| 208638/225000 [402:25:34<39:36:39,  8.72s/it]
 93%|█████████▎| 208639/225000 [402:25:46<43:45:58,  9.63s/it]
 93%|█████████▎| 208640/225000 [402:25:57<45:58:23, 10.12s/it]
 93%|█████████▎| 208641/225000 [402:26:04<41:00:03,  9.02s/it]
 93%|█████████▎| 208642/225000 [402:26:17<46:23:10, 10.21s/it]
 93%|█████████▎| 208643/225000 [402:26:25<43:11:47,  9.51s/it]
 93%|█████████▎| 208644/225000 [402:26:33<41:45:43,  9.19s/it]
 93%|█████████▎| 208645/225000 [402:26:42<41:17:48,  9.09s/it]
 93%|█████████▎| 208646/225000 [402:26:49<38:59:22,  8.58s/it]
 93%|█████████▎| 208647/225000 [402:26:58<39:14:00,  8.64s/it]
 93%|█████████▎| 208648/225000 [402:27:06<38:30:29,  8.48s/it]
 93%|█████████▎| 208649/225000 [402:27:19<44:05:03,  9.71s/it]
 93%|█████████▎| 208650/225000 [402:27:27<42:23:42,  9.33s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.264', 'learning_rate': '7.186e-07', 'epoch': '0.9273', 'num_input_tokens_seen': 679932240, 'train_runtime': '1.449e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208650/225000 [402:27:27<42:23:42,  9.33s/it]
 93%|█████████▎| 208651/225000 [402:27:32<36:00:17,  7.93s/it]
 93%|█████████▎| 208652/225000 [402:27:46<44:28:32,  9.79s/it]
 93%|█████████▎| 208653/225000 [402:27:57<45:43:33, 10.07s/it]
 93%|█████████▎| 208654/225000 [402:28:06<44:15:30,  9.75s/it]
 93%|█████████▎| 208655/225000 [402:28:15<44:15:16,  9.75s/it]
 93%|█████████▎| 208656/225000 [402:28:24<42:59:23,  9.47s/it]
 93%|█████████▎| 208657/225000 [402:28:38<49:20:57, 10.87s/it]
 93%|█████████▎| 208658/225000 [402:28:50<49:55:25, 11.00s/it]
 93%|█████████▎| 208659/225000 [402:28:59<47:28:12, 10.46s/it]
 93%|█████████▎| 208660/225000 [402:29:04<39:41:50,  8.75s/it]
 93%|█████████▎| 208661/225000 [402:29:12<38:39:40,  8.52s/it]
 93%|█████████▎| 208662/225000 [402:29:21<40:19:41,  8.89s/it]
 93%|█████████▎| 208663/225000 [402:29:30<39:48:06,  8.77s/it]
 93%|█████████▎| 208664/225000 [402:29:35<34:31:55,  7.61s/it]
 93%|█████████▎| 208665/225000 [402:29:41<32:57:21,  7.26s/it]
 93%|█████████▎| 208666/225000 [402:29:54<40:02:41,  8.83s/it]
 93%|█████████▎| 208667/225000 [402:29:59<35:29:44,  7.82s/it]
 93%|█████████▎| 208668/225000 [402:30:04<31:37:43,  6.97s/it]
 93%|█████████▎| 208669/225000 [402:30:14<35:27:11,  7.82s/it]
 93%|█████████▎| 208670/225000 [402:30:24<37:54:49,  8.36s/it]
 93%|█████████▎| 208671/225000 [402:30:34<41:19:17,  9.11s/it]
 93%|█████████▎| 208672/225000 [402:30:41<37:54:15,  8.36s/it]
 93%|█████████▎| 208673/225000 [402:30:47<34:09:31,  7.53s/it]
 93%|█████████▎| 208674/225000 [402:30:51<29:58:07,  6.61s/it]
 93%|█████████▎| 208675/225000 [402:31:05<39:41:26,  8.75s/it]
 93%|█████████▎| 208676/225000 [402:31:13<38:59:27,  8.60s/it]
 93%|█████████▎| 208677/225000 [402:31:20<35:57:28,  7.93s/it]
 93%|█████████▎| 208678/225000 [402:31:34<45:07:43,  9.95s/it]
 93%|█████████▎| 208679/225000 [402:31:43<43:43:44,  9.65s/it]
 93%|█████████▎| 208680/225000 [402:31:52<43:20:40,  9.56s/it]
 93%|█████████▎| 208681/225000 [402:32:03<45:03:07,  9.94s/it]
 93%|█████████▎| 208682/225000 [402:32:09<39:08:39,  8.64s/it]
 93%|█████████▎| 208683/225000 [402:32:17<38:12:01,  8.43s/it]
 93%|█████████▎| 208684/225000 [402:32:31<46:10:39, 10.19s/it]
 93%|█████████▎| 208685/225000 [402:32:37<40:22:54,  8.91s/it]
 93%|█████████▎| 208686/225000 [402:32:45<38:37:14,  8.52s/it]
 93%|█████████▎| 208687/225000 [402:32:51<35:56:58,  7.93s/it]
 93%|█████████▎| 208688/225000 [402:32:57<32:52:49,  7.26s/it]
 93%|█████████▎| 208689/225000 [402:33:03<31:13:01,  6.89s/it]
 93%|█████████▎| 208690/225000 [402:33:14<36:42:26,  8.10s/it]
 93%|█████████▎| 208691/225000 [402:33:28<45:28:12, 10.04s/it]
 93%|█████████▎| 208692/225000 [402:33:38<45:06:59,  9.96s/it]
 93%|█████████▎| 208693/225000 [402:33:43<38:42:54,  8.55s/it]
 93%|█████████▎| 208694/225000 [402:33:56<43:40:17,  9.64s/it]
 93%|█████████▎| 208695/225000 [402:34:09<48:51:53, 10.79s/it]
 93%|█████████▎| 208696/225000 [402:34:13<39:39:06,  8.76s/it]
 93%|█████████▎| 208697/225000 [402:34:23<40:45:18,  9.00s/it]
 93%|█████████▎| 208698/225000 [402:34:28<35:10:29,  7.77s/it]
 93%|█████████▎| 208699/225000 [402:34:37<37:42:47,  8.33s/it]
 93%|█████████▎| 208700/225000 [402:34:45<36:45:25,  8.12s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.291', 'learning_rate': '7.142e-07', 'epoch': '0.9276', 'num_input_tokens_seen': 680159368, 'train_runtime': '1.449e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208700/225000 [402:34:45<36:45:25,  8.12s/it]
 93%|█████████▎| 208701/225000 [402:34:56<41:23:40,  9.14s/it]
 93%|█████████▎| 208702/225000 [402:35:05<40:43:04,  8.99s/it]
 93%|█████████▎| 208703/225000 [402:35:14<40:14:52,  8.89s/it]
 93%|█████████▎| 208704/225000 [402:35:26<44:29:10,  9.83s/it]
 93%|█████████▎| 208705/225000 [402:35:33<40:40:24,  8.99s/it]
 93%|█████████▎| 208706/225000 [402:35:43<42:31:27,  9.40s/it]
 93%|█████████▎| 208707/225000 [402:35:48<36:59:06,  8.17s/it]
 93%|█████████▎| 208708/225000 [402:35:57<36:53:11,  8.15s/it]
 93%|█████████▎| 208709/225000 [402:36:09<42:48:18,  9.46s/it]
 93%|█████████▎| 208710/225000 [402:36:15<38:04:23,  8.41s/it]
 93%|█████████▎| 208711/225000 [402:36:27<42:21:40,  9.36s/it]
 93%|█████████▎| 208712/225000 [402:36:38<44:43:50,  9.89s/it]
 93%|█████████▎| 208713/225000 [402:36:50<47:21:51, 10.47s/it]
 93%|█████████▎| 208714/225000 [402:36:56<42:30:20,  9.40s/it]
 93%|█████████▎| 208715/225000 [402:37:09<47:03:50, 10.40s/it]
 93%|█████████▎| 208716/225000 [402:37:17<42:57:02,  9.50s/it]
 93%|█████████▎| 208717/225000 [402:37:27<44:50:42,  9.91s/it]
 93%|█████████▎| 208718/225000 [402:37:34<40:46:06,  9.01s/it]
 93%|█████████▎| 208719/225000 [402:37:43<40:07:42,  8.87s/it]
 93%|█████████▎| 208720/225000 [402:37:50<37:37:26,  8.32s/it]
 93%|█████████▎| 208721/225000 [402:37:57<36:02:31,  7.97s/it]
 93%|█████████▎| 208722/225000 [402:38:14<47:52:14, 10.59s/it]
 93%|█████████▎| 208723/225000 [402:38:22<44:49:48,  9.92s/it]
 93%|█████████▎| 208724/225000 [402:38:34<47:10:05, 10.43s/it]
 93%|█████████▎| 208725/225000 [402:38:40<41:34:45,  9.20s/it]
 93%|█████████▎| 208726/225000 [402:38:49<40:34:55,  8.98s/it]
 93%|█████████▎| 208727/225000 [402:38:57<40:02:44,  8.86s/it]
 93%|█████████▎| 208728/225000 [402:39:05<38:48:05,  8.58s/it]
 93%|█████████▎| 208729/225000 [402:39:14<38:53:37,  8.61s/it]
 93%|█████████▎| 208730/225000 [402:39:23<40:00:48,  8.85s/it]
 93%|█████████▎| 208731/225000 [402:39:30<36:55:38,  8.17s/it]
 93%|█████████▎| 208732/225000 [402:39:37<35:25:46,  7.84s/it]
 93%|█████████▎| 208733/225000 [402:39:49<41:19:19,  9.14s/it]
 93%|█████████▎| 208734/225000 [402:39:57<39:42:15,  8.79s/it]
 93%|█████████▎| 208735/225000 [402:40:02<35:13:22,  7.80s/it]
 93%|█████████▎| 208736/225000 [402:40:15<41:11:52,  9.12s/it]
 93%|█████████▎| 208737/225000 [402:40:20<35:40:36,  7.90s/it]
 93%|█████████▎| 208738/225000 [402:40:25<32:08:59,  7.12s/it]
 93%|█████████▎| 208739/225000 [402:40:30<29:57:55,  6.63s/it]
 93%|█████████▎| 208740/225000 [402:40:37<30:27:52,  6.74s/it]
 93%|█████████▎| 208741/225000 [402:40:48<35:19:01,  7.82s/it]
 93%|█████████▎| 208742/225000 [402:40:58<38:37:51,  8.55s/it]
 93%|█████████▎| 208743/225000 [402:41:06<37:41:40,  8.35s/it]
 93%|█████████▎| 208744/225000 [402:41:12<35:05:52,  7.77s/it]
 93%|█████████▎| 208745/225000 [402:41:18<32:28:40,  7.19s/it]
 93%|█████████▎| 208746/225000 [402:41:35<45:44:01, 10.13s/it]
 93%|█████████▎| 208747/225000 [402:41:39<37:51:49,  8.39s/it]
 93%|█████████▎| 208748/225000 [402:41:50<40:46:49,  9.03s/it]
 93%|█████████▎| 208749/225000 [402:41:56<35:56:29,  7.96s/it]
 93%|█████████▎| 208750/225000 [402:42:10<44:26:08,  9.84s/it]
                                                              
{'loss': '1.332', 'grad_norm': '1.237', 'learning_rate': '7.099e-07', 'epoch': '0.9278', 'num_input_tokens_seen': 680395848, 'train_runtime': '1.45e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208750/225000 [402:42:10<44:26:08,  9.84s/it]
 93%|█████████▎| 208751/225000 [402:42:16<39:34:49,  8.77s/it]
 93%|█████████▎| 208752/225000 [402:42:29<44:56:00,  9.96s/it]
 93%|█████████▎| 208753/225000 [402:42:35<39:29:32,  8.75s/it]
 93%|█████████▎| 208754/225000 [402:42:41<35:58:50,  7.97s/it]
 93%|█████████▎| 208755/225000 [402:42:55<44:46:30,  9.92s/it]
 93%|█████████▎| 208756/225000 [402:43:00<37:08:57,  8.23s/it]
 93%|█████████▎| 208757/225000 [402:43:09<38:30:23,  8.53s/it]
 93%|█████████▎| 208758/225000 [402:43:19<40:05:00,  8.88s/it]
 93%|█████████▎| 208759/225000 [402:43:25<37:06:06,  8.22s/it]
 93%|█████████▎| 208760/225000 [402:43:37<42:19:53,  9.38s/it]
 93%|█████████▎| 208761/225000 [402:43:45<39:36:49,  8.78s/it]
 93%|█████████▎| 208762/225000 [402:43:53<39:39:33,  8.79s/it]
 93%|█████████▎| 208763/225000 [402:44:02<39:00:45,  8.65s/it]
 93%|█████████▎| 208764/225000 [402:44:09<36:26:20,  8.08s/it]
 93%|█████████▎| 208765/225000 [402:44:14<33:26:58,  7.42s/it]
 93%|█████████▎| 208766/225000 [402:44:31<45:37:11, 10.12s/it]
 93%|█████████▎| 208767/225000 [402:44:37<40:54:37,  9.07s/it]
 93%|█████████▎| 208768/225000 [402:44:51<47:15:55, 10.48s/it]
 93%|█████████▎| 208769/225000 [402:44:58<41:32:43,  9.21s/it]
 93%|█████████▎| 208770/225000 [402:45:04<37:48:37,  8.39s/it]
 93%|█████████▎| 208771/225000 [402:45:10<34:27:24,  7.64s/it]
 93%|█████████▎| 208772/225000 [402:45:15<30:54:12,  6.86s/it]
 93%|█████████▎| 208773/225000 [402:45:28<39:50:04,  8.84s/it]
 93%|█████████▎| 208774/225000 [402:45:37<39:40:50,  8.80s/it]
 93%|█████████▎| 208775/225000 [402:45:45<38:47:10,  8.61s/it]
 93%|█████████▎| 208776/225000 [402:45:57<43:13:59,  9.59s/it]
 93%|█████████▎| 208777/225000 [402:46:04<39:57:01,  8.87s/it]
 93%|█████████▎| 208778/225000 [402:46:13<40:06:21,  8.90s/it]
 93%|█████████▎| 208779/225000 [402:46:19<35:26:20,  7.87s/it]
 93%|█████████▎| 208780/225000 [402:46:29<39:13:30,  8.71s/it]
 93%|█████████▎| 208781/225000 [402:46:37<37:57:28,  8.43s/it]
 93%|█████████▎| 208782/225000 [402:46:44<36:03:35,  8.00s/it]
 93%|█████████▎| 208783/225000 [402:46:55<39:26:25,  8.76s/it]
 93%|█████████▎| 208784/225000 [402:47:02<38:09:08,  8.47s/it]
 93%|█████████▎| 208785/225000 [402:47:10<37:21:16,  8.29s/it]
 93%|█████████▎| 208786/225000 [402:47:20<39:01:58,  8.67s/it]
 93%|█████████▎| 208787/225000 [402:47:27<36:14:14,  8.05s/it]
 93%|█████████▎| 208788/225000 [402:47:34<35:10:11,  7.81s/it]
 93%|█████████▎| 208789/225000 [402:47:42<35:09:05,  7.81s/it]
 93%|█████████▎| 208790/225000 [402:47:47<32:13:59,  7.16s/it]
 93%|█████████▎| 208791/225000 [402:48:02<42:54:48,  9.53s/it]
 93%|█████████▎| 208792/225000 [402:48:12<43:16:18,  9.61s/it]
 93%|█████████▎| 208793/225000 [402:48:16<35:59:14,  7.99s/it]
 93%|█████████▎| 208794/225000 [402:48:25<37:34:21,  8.35s/it]
 93%|█████████▎| 208795/225000 [402:48:38<42:48:41,  9.51s/it]
 93%|█████████▎| 208796/225000 [402:48:45<40:23:28,  8.97s/it]
 93%|█████████▎| 208797/225000 [402:48:52<36:35:18,  8.13s/it]
 93%|█████████▎| 208798/225000 [402:49:01<38:50:36,  8.63s/it]
 93%|█████████▎| 208799/225000 [402:49:13<42:52:32,  9.53s/it]
 93%|█████████▎| 208800/225000 [402:49:18<37:13:13,  8.27s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.358', 'learning_rate': '7.056e-07', 'epoch': '0.928', 'num_input_tokens_seen': 680628024, 'train_runtime': '1.45e+06', 'train_tokens_per_second': '469.3'}

 93%|█████████▎| 208800/225000 [402:49:18<37:13:13,  8.27s/it]
 93%|█████████▎| 208801/225000 [402:49:30<42:14:39,  9.39s/it]
 93%|█████████▎| 208802/225000 [402:49:37<38:18:10,  8.51s/it]
 93%|█████████▎| 208803/225000 [402:49:44<36:46:54,  8.18s/it]
 93%|█████████▎| 208804/225000 [402:49:53<37:47:27,  8.40s/it]
 93%|█████████▎| 208805/225000 [402:50:01<36:39:18,  8.15s/it]
 93%|█████████▎| 208806/225000 [402:50:07<34:15:16,  7.61s/it]
 93%|█████████▎| 208807/225000 [402:50:19<39:52:43,  8.87s/it]
 93%|█████████▎| 208808/225000 [402:50:26<37:48:54,  8.41s/it]
 93%|█████████▎| 208809/225000 [402:50:32<34:30:47,  7.67s/it]
 93%|█████████▎| 208810/225000 [402:50:43<38:40:28,  8.60s/it]
 93%|█████████▎| 208811/225000 [402:50:49<35:14:53,  7.84s/it]
 93%|█████████▎| 208812/225000 [402:51:00<39:15:55,  8.73s/it]
 93%|█████████▎| 208813/225000 [402:51:08<39:16:13,  8.73s/it]
 93%|█████████▎| 208814/225000 [402:51:16<37:06:23,  8.25s/it]
 93%|█████████▎| 208815/225000 [402:51:22<34:56:56,  7.77s/it]
 93%|█████████▎| 208816/225000 [402:51:33<39:28:53,  8.78s/it]
 93%|█████████▎| 208817/225000 [402:51:39<34:58:38,  7.78s/it]
 93%|█████████▎| 208818/225000 [402:51:44<31:48:16,  7.08s/it]
 93%|█████████▎| 208819/225000 [402:51:50<29:37:40,  6.59s/it]
 93%|█████████▎| 208820/225000 [402:52:01<36:27:57,  8.11s/it]
 93%|█████████▎| 208821/225000 [402:52:14<41:49:12,  9.31s/it]
 93%|█████████▎| 208822/225000 [402:52:23<42:06:17,  9.37s/it]
 93%|█████████▎| 208823/225000 [402:52:35<45:23:03, 10.10s/it]
 93%|█████████▎| 208824/225000 [402:52:43<42:41:17,  9.50s/it]
 93%|█████████▎| 208825/225000 [402:52:47<35:58:25,  8.01s/it]
 93%|█████████▎| 208826/225000 [402:52:58<38:44:24,  8.62s/it]
 93%|█████████▎| 208827/225000 [402:53:07<39:35:27,  8.81s/it]
 93%|█████████▎| 208828/225000 [402:53:13<36:36:43,  8.15s/it]
 93%|█████████▎| 208829/225000 [402:53:20<34:31:05,  7.68s/it]
 93%|█████████▎| 208830/225000 [402:53:25<31:27:50,  7.00s/it]
 93%|█████████▎| 208831/225000 [402:53:31<29:53:48,  6.66s/it]
 93%|█████████▎| 208832/225000 [402:53:45<39:47:20,  8.86s/it]
 93%|█████████▎| 208833/225000 [402:53:56<42:22:09,  9.43s/it]
 93%|█████████▎| 208834/225000 [402:54:03<39:27:22,  8.79s/it]
 93%|█████████▎| 208835/225000 [402:54:13<40:04:41,  8.93s/it]
 93%|█████████▎| 208836/225000 [402:54:23<41:50:43,  9.32s/it]
 93%|█████████▎| 208837/225000 [402:54:32<41:25:35,  9.23s/it]
 93%|█████████▎| 208838/225000 [402:54:45<46:13:04, 10.29s/it]
 93%|█████████▎| 208839/225000 [402:54:51<40:34:20,  9.04s/it]
 93%|█████████▎| 208840/225000 [402:54:58<38:16:44,  8.53s/it]
 93%|█████████▎| 208841/225000 [402:55:04<34:48:50,  7.76s/it]
 93%|█████████▎| 208842/225000 [402:55:19<45:14:59, 10.08s/it]
 93%|█████████▎| 208843/225000 [402:55:25<39:15:59,  8.75s/it]
 93%|█████████▎| 208844/225000 [402:55:33<38:15:01,  8.52s/it]
 93%|█████████▎| 208845/225000 [402:55:40<36:35:25,  8.15s/it]
 93%|█████████▎| 208846/225000 [402:55:47<34:00:34,  7.58s/it]
 93%|█████████▎| 208847/225000 [402:56:01<43:39:45,  9.73s/it]
 93%|█████████▎| 208848/225000 [402:56:08<39:54:24,  8.89s/it]
 93%|█████████▎| 208849/225000 [402:56:19<42:43:09,  9.52s/it]
 93%|█████████▎| 208850/225000 [402:56:30<43:39:37,  9.73s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.408', 'learning_rate': '7.013e-07', 'epoch': '0.9282', 'num_input_tokens_seen': 680856392, 'train_runtime': '1.451e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 208850/225000 [402:56:30<43:39:37,  9.73s/it]
 93%|█████████▎| 208851/225000 [402:56:37<40:54:34,  9.12s/it]
 93%|█████████▎| 208852/225000 [402:56:44<37:45:23,  8.42s/it]
 93%|█████████▎| 208853/225000 [402:56:53<38:32:28,  8.59s/it]
 93%|█████████▎| 208854/225000 [402:57:04<41:10:56,  9.18s/it]
 93%|█████████▎| 208855/225000 [402:57:11<38:22:01,  8.56s/it]
 93%|█████████▎| 208856/225000 [402:57:18<36:36:32,  8.16s/it]
 93%|█████████▎| 208857/225000 [402:57:27<37:27:42,  8.35s/it]
 93%|█████████▎| 208858/225000 [402:57:41<44:59:42, 10.03s/it]
 93%|█████████▎| 208859/225000 [402:57:50<43:54:20,  9.79s/it]
 93%|█████████▎| 208860/225000 [402:57:58<40:58:08,  9.14s/it]
 93%|█████████▎| 208861/225000 [402:58:10<45:01:50, 10.04s/it]
 93%|█████████▎| 208862/225000 [402:58:17<41:33:05,  9.27s/it]
 93%|█████████▎| 208863/225000 [402:58:28<43:50:08,  9.78s/it]
 93%|█████████▎| 208864/225000 [402:58:33<36:55:31,  8.24s/it]
 93%|█████████▎| 208865/225000 [402:58:45<42:23:03,  9.46s/it]
 93%|█████████▎| 208866/225000 [402:58:54<41:59:59,  9.37s/it]
 93%|█████████▎| 208867/225000 [402:59:00<37:26:29,  8.35s/it]
 93%|█████████▎| 208868/225000 [402:59:09<38:03:30,  8.49s/it]
 93%|█████████▎| 208869/225000 [402:59:14<33:49:26,  7.55s/it]
 93%|█████████▎| 208870/225000 [402:59:21<32:33:40,  7.27s/it]
 93%|█████████▎| 208871/225000 [402:59:28<31:37:28,  7.06s/it]
 93%|█████████▎| 208872/225000 [402:59:36<33:19:21,  7.44s/it]
 93%|█████████▎| 208873/225000 [402:59:43<33:11:19,  7.41s/it]
 93%|█████████▎| 208874/225000 [402:59:51<34:20:59,  7.67s/it]
 93%|█████████▎| 208875/225000 [403:00:05<41:36:47,  9.29s/it]
 93%|█████████▎| 208876/225000 [403:00:14<41:41:59,  9.31s/it]
 93%|█████████▎| 208877/225000 [403:00:19<36:07:03,  8.06s/it]
 93%|█████████▎| 208878/225000 [403:00:30<40:00:30,  8.93s/it]
 93%|█████████▎| 208879/225000 [403:00:37<37:59:48,  8.49s/it]
 93%|█████████▎| 208880/225000 [403:00:48<40:34:18,  9.06s/it]
 93%|█████████▎| 208881/225000 [403:00:55<38:09:51,  8.52s/it]
 93%|█████████▎| 208882/225000 [403:01:01<33:55:29,  7.58s/it]
 93%|█████████▎| 208883/225000 [403:01:13<39:56:05,  8.92s/it]
 93%|█████████▎| 208884/225000 [403:01:22<39:57:08,  8.92s/it]
 93%|█████████▎| 208885/225000 [403:01:30<39:33:02,  8.84s/it]
 93%|█████████▎| 208886/225000 [403:01:37<36:37:21,  8.18s/it]
 93%|█████████▎| 208887/225000 [403:01:44<35:24:15,  7.91s/it]
 93%|█████████▎| 208888/225000 [403:01:56<40:29:22,  9.05s/it]
 93%|█████████▎| 208889/225000 [403:02:05<40:44:16,  9.10s/it]
 93%|█████████▎| 208890/225000 [403:02:13<39:16:29,  8.78s/it]
 93%|█████████▎| 208891/225000 [403:02:22<39:06:36,  8.74s/it]
 93%|█████████▎| 208892/225000 [403:02:30<37:56:38,  8.48s/it]
 93%|█████████▎| 208893/225000 [403:02:41<42:20:14,  9.46s/it]
 93%|█████████▎| 208894/225000 [403:02:51<43:03:06,  9.62s/it]
 93%|█████████▎| 208895/225000 [403:03:00<41:22:58,  9.25s/it]
 93%|█████████▎| 208896/225000 [403:03:09<41:03:41,  9.18s/it]
 93%|█████████▎| 208897/225000 [403:03:18<41:16:56,  9.23s/it]
 93%|█████████▎| 208898/225000 [403:03:27<41:10:43,  9.21s/it]
 93%|█████████▎| 208899/225000 [403:03:33<36:44:08,  8.21s/it]
 93%|█████████▎| 208900/225000 [403:03:47<43:56:22,  9.83s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.11', 'learning_rate': '6.97e-07', 'epoch': '0.9284', 'num_input_tokens_seen': 681087232, 'train_runtime': '1.451e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 208900/225000 [403:03:47<43:56:22,  9.83s/it]
 93%|█████████▎| 208901/225000 [403:03:52<37:35:53,  8.41s/it]
 93%|█████████▎| 208902/225000 [403:03:57<33:40:15,  7.53s/it]
 93%|█████████▎| 208903/225000 [403:04:07<36:21:26,  8.13s/it]
 93%|█████████▎| 208904/225000 [403:04:15<36:46:57,  8.23s/it]
 93%|█████████▎| 208905/225000 [403:04:23<36:21:35,  8.13s/it]
 93%|█████████▎| 208906/225000 [403:04:31<35:43:14,  7.99s/it]
 93%|█████████▎| 208907/225000 [403:04:41<39:07:55,  8.75s/it]
 93%|█████████▎| 208908/225000 [403:04:51<40:15:54,  9.01s/it]
 93%|█████████▎| 208909/225000 [403:04:57<35:40:46,  7.98s/it]
 93%|█████████▎| 208910/225000 [403:05:02<31:54:49,  7.14s/it]
 93%|█████████▎| 208911/225000 [403:05:09<32:12:16,  7.21s/it]
 93%|█████████▎| 208912/225000 [403:05:27<46:19:08, 10.36s/it]
 93%|█████████▎| 208913/225000 [403:05:36<44:57:04, 10.06s/it]
 93%|█████████▎| 208914/225000 [403:05:46<44:01:11,  9.85s/it]
 93%|█████████▎| 208915/225000 [403:05:52<39:06:07,  8.75s/it]
 93%|█████████▎| 208916/225000 [403:06:00<39:02:24,  8.74s/it]
 93%|█████████▎| 208917/225000 [403:06:15<46:13:44, 10.35s/it]
 93%|█████████▎| 208918/225000 [403:06:21<41:15:30,  9.24s/it]
 93%|█████████▎| 208919/225000 [403:06:28<37:24:37,  8.37s/it]
 93%|█████████▎| 208920/225000 [403:06:38<40:30:41,  9.07s/it]
 93%|█████████▎| 208921/225000 [403:06:47<40:23:56,  9.05s/it]
 93%|█████████▎| 208922/225000 [403:06:57<41:57:13,  9.39s/it]
 93%|█████████▎| 208923/225000 [403:07:04<38:17:17,  8.57s/it]
 93%|█████████▎| 208924/225000 [403:07:11<35:27:55,  7.94s/it]
 93%|█████████▎| 208925/225000 [403:07:27<46:38:49, 10.45s/it]
 93%|█████████▎| 208926/225000 [403:07:34<42:41:38,  9.56s/it]
 93%|█████████▎| 208927/225000 [403:07:45<44:18:55,  9.93s/it]
 93%|█████████▎| 208928/225000 [403:07:56<45:53:40, 10.28s/it]
 93%|█████████▎| 208929/225000 [403:08:02<40:24:31,  9.05s/it]
 93%|█████████▎| 208930/225000 [403:08:10<37:57:11,  8.50s/it]
 93%|█████████▎| 208931/225000 [403:08:17<36:28:18,  8.17s/it]
 93%|█████████▎| 208932/225000 [403:08:27<38:35:51,  8.65s/it]
 93%|█████████▎| 208933/225000 [403:08:36<39:10:12,  8.78s/it]
 93%|█████████▎| 208934/225000 [403:08:44<38:54:45,  8.72s/it]
 93%|█████████▎| 208935/225000 [403:08:50<34:57:59,  7.84s/it]
 93%|█████████▎| 208936/225000 [403:09:01<39:29:07,  8.85s/it]
 93%|█████████▎| 208937/225000 [403:09:11<40:14:45,  9.02s/it]
 93%|█████████▎| 208938/225000 [403:09:21<41:22:33,  9.27s/it]
 93%|█████████▎| 208939/225000 [403:09:29<39:46:18,  8.91s/it]
 93%|█████████▎| 208940/225000 [403:09:38<40:37:00,  9.10s/it]
 93%|█████████▎| 208941/225000 [403:09:48<41:22:37,  9.28s/it]
 93%|█████████▎| 208942/225000 [403:09:53<35:09:56,  7.88s/it]
 93%|█████████▎| 208943/225000 [403:10:02<37:37:23,  8.44s/it]
 93%|█████████▎| 208944/225000 [403:10:10<36:19:11,  8.14s/it]
 93%|█████████▎| 208945/225000 [403:10:15<32:56:16,  7.39s/it]
 93%|█████████▎| 208946/225000 [403:10:31<43:18:36,  9.71s/it]
 93%|█████████▎| 208947/225000 [403:10:39<41:29:41,  9.31s/it]
 93%|█████████▎| 208948/225000 [403:10:48<41:39:42,  9.34s/it]
 93%|█████████▎| 208949/225000 [403:10:58<42:03:14,  9.43s/it]
 93%|█████████▎| 208950/225000 [403:11:08<42:27:54,  9.52s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.205', 'learning_rate': '6.927e-07', 'epoch': '0.9287', 'num_input_tokens_seen': 681319560, 'train_runtime': '1.451e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 208950/225000 [403:11:08<42:27:54,  9.52s/it]
 93%|█████████▎| 208951/225000 [403:11:17<41:26:39,  9.30s/it]
 93%|█████████▎| 208952/225000 [403:11:29<45:56:28, 10.31s/it]
 93%|█████████▎| 208953/225000 [403:11:37<42:48:11,  9.60s/it]
 93%|█████████▎| 208954/225000 [403:11:45<40:48:09,  9.15s/it]
 93%|█████████▎| 208955/225000 [403:11:51<36:21:31,  8.16s/it]
 93%|█████████▎| 208956/225000 [403:12:02<40:04:15,  8.99s/it]
 93%|█████████▎| 208957/225000 [403:12:09<37:49:43,  8.49s/it]
 93%|█████████▎| 208958/225000 [403:12:15<34:34:25,  7.76s/it]
 93%|█████████▎| 208959/225000 [403:12:22<33:23:48,  7.50s/it]
 93%|█████████▎| 208960/225000 [403:12:34<38:39:30,  8.68s/it]
 93%|█████████▎| 208961/225000 [403:12:46<43:49:56,  9.84s/it]
 93%|█████████▎| 208962/225000 [403:12:56<43:33:37,  9.78s/it]
 93%|█████████▎| 208963/225000 [403:13:03<40:16:03,  9.04s/it]
 93%|█████████▎| 208964/225000 [403:13:13<40:41:20,  9.13s/it]
 93%|█████████▎| 208965/225000 [403:13:26<46:07:02, 10.35s/it]
 93%|█████████▎| 208966/225000 [403:13:34<43:31:33,  9.77s/it]
 93%|█████████▎| 208967/225000 [403:13:42<41:28:59,  9.31s/it]
 93%|█████████▎| 208968/225000 [403:13:50<38:31:15,  8.65s/it]
 93%|█████████▎| 208969/225000 [403:13:59<39:15:29,  8.82s/it]
 93%|█████████▎| 208970/225000 [403:14:05<35:19:06,  7.93s/it]
 93%|█████████▎| 208971/225000 [403:14:20<45:40:29, 10.26s/it]
 93%|█████████▎| 208972/225000 [403:14:28<42:45:49,  9.61s/it]
 93%|█████████▎| 208973/225000 [403:14:35<38:13:57,  8.59s/it]
 93%|█████████▎| 208974/225000 [403:14:45<41:01:38,  9.22s/it]
 93%|█████████▎| 208975/225000 [403:14:53<39:17:47,  8.83s/it]
 93%|█████████▎| 208976/225000 [403:15:07<46:23:01, 10.42s/it]
 93%|█████████▎| 208977/225000 [403:15:17<45:18:41, 10.18s/it]
 93%|█████████▎| 208978/225000 [403:15:23<39:15:23,  8.82s/it]
 93%|█████████▎| 208979/225000 [403:15:35<44:26:38,  9.99s/it]
 93%|█████████▎| 208980/225000 [403:15:45<43:55:20,  9.87s/it]
 93%|█████████▎| 208981/225000 [403:15:54<42:34:56,  9.57s/it]
 93%|█████████▎| 208982/225000 [403:16:02<41:22:14,  9.30s/it]
 93%|█████████▎| 208983/225000 [403:16:12<41:29:15,  9.32s/it]
 93%|█████████▎| 208984/225000 [403:16:25<46:04:38, 10.36s/it]
 93%|█████████▎| 208985/225000 [403:16:32<42:03:48,  9.46s/it]
 93%|█████████▎| 208986/225000 [403:16:38<36:59:21,  8.32s/it]
 93%|█████████▎| 208987/225000 [403:16:48<39:33:43,  8.89s/it]
 93%|█████████▎| 208988/225000 [403:16:56<38:11:48,  8.59s/it]
 93%|█████████▎| 208989/225000 [403:17:05<39:46:47,  8.94s/it]
 93%|█████████▎| 208990/225000 [403:17:12<37:11:16,  8.36s/it]
 93%|█████████▎| 208991/225000 [403:17:22<39:10:22,  8.81s/it]
 93%|█████████▎| 208992/225000 [403:17:29<35:54:47,  8.08s/it]
 93%|█████████▎| 208993/225000 [403:17:34<32:30:22,  7.31s/it]
 93%|█████████▎| 208994/225000 [403:17:42<33:11:18,  7.46s/it]
 93%|█████████▎| 208995/225000 [403:17:49<32:32:18,  7.32s/it]
 93%|█████████▎| 208996/225000 [403:17:56<31:52:02,  7.17s/it]
 93%|█████████▎| 208997/225000 [403:18:05<34:21:11,  7.73s/it]
 93%|█████████▎| 208998/225000 [403:18:17<40:21:43,  9.08s/it]
 93%|█████████▎| 208999/225000 [403:18:25<39:05:29,  8.80s/it]
 93%|█████████▎| 209000/225000 [403:18:40<46:35:48, 10.48s/it]
                                                              
{'loss': '1.278', 'grad_norm': '1.711', 'learning_rate': '6.885e-07', 'epoch': '0.9289', 'num_input_tokens_seen': 681551128, 'train_runtime': '1.452e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 209000/225000 [403:18:40<46:35:48, 10.48s/it]
 93%|█████████▎| 209001/225000 [403:18:45<39:40:29,  8.93s/it]
 93%|█████████▎| 209002/225000 [403:18:52<37:25:51,  8.42s/it]
 93%|█████████▎| 209003/225000 [403:18:59<34:51:38,  7.85s/it]
 93%|█████████▎| 209004/225000 [403:19:13<43:18:04,  9.75s/it]
 93%|█████████▎| 209005/225000 [403:19:18<37:15:11,  8.38s/it]
 93%|█████████▎| 209006/225000 [403:19:26<37:01:46,  8.33s/it]
 93%|█████████▎| 209007/225000 [403:19:34<35:43:09,  8.04s/it]
 93%|█████████▎| 209008/225000 [403:19:44<38:14:44,  8.61s/it]
 93%|█████████▎| 209009/225000 [403:19:54<40:54:20,  9.21s/it]
 93%|█████████▎| 209010/225000 [403:19:59<35:33:27,  8.01s/it]
 93%|█████████▎| 209011/225000 [403:20:07<34:24:42,  7.75s/it]
 93%|█████████▎| 209012/225000 [403:20:16<36:45:10,  8.28s/it]
 93%|█████████▎| 209013/225000 [403:20:24<36:18:53,  8.18s/it]
 93%|█████████▎| 209014/225000 [403:20:32<35:25:07,  7.98s/it]
 93%|█████████▎| 209015/225000 [403:20:38<33:59:29,  7.66s/it]
 93%|█████████▎| 209016/225000 [403:20:49<38:03:55,  8.57s/it]
 93%|█████████▎| 209017/225000 [403:20:59<39:56:13,  9.00s/it]
 93%|█████████▎| 209018/225000 [403:21:05<36:06:57,  8.14s/it]
 93%|█████████▎| 209019/225000 [403:21:12<34:33:36,  7.79s/it]
 93%|█████████▎| 209020/225000 [403:21:19<33:03:16,  7.45s/it]
 93%|█████████▎| 209021/225000 [403:21:27<33:36:19,  7.57s/it]
 93%|█████████▎| 209022/225000 [403:21:37<37:37:32,  8.48s/it]
 93%|█████████▎| 209023/225000 [403:21:47<39:52:38,  8.99s/it]
 93%|█████████▎| 209024/225000 [403:21:57<40:15:43,  9.07s/it]
 93%|█████████▎| 209025/225000 [403:22:08<43:00:51,  9.69s/it]
 93%|█████████▎| 209026/225000 [403:22:21<47:34:47, 10.72s/it]
 93%|█████████▎| 209027/225000 [403:22:25<39:02:55,  8.80s/it]
 93%|█████████▎| 209028/225000 [403:22:31<34:52:55,  7.86s/it]
 93%|█████████▎| 209029/225000 [403:22:47<45:37:33, 10.28s/it]
 93%|█████████▎| 209030/225000 [403:22:54<41:17:33,  9.31s/it]
 93%|█████████▎| 209031/225000 [403:23:00<37:05:46,  8.36s/it]
 93%|█████████▎| 209032/225000 [403:23:08<37:01:51,  8.35s/it]
 93%|█████████▎| 209033/225000 [403:23:22<43:55:12,  9.90s/it]
 93%|█████████▎| 209034/225000 [403:23:29<39:33:50,  8.92s/it]
 93%|█████████▎| 209035/225000 [403:23:34<35:07:56,  7.92s/it]
 93%|█████████▎| 209036/225000 [403:23:42<34:38:48,  7.81s/it]
 93%|█████████▎| 209037/225000 [403:23:56<43:16:46,  9.76s/it]
 93%|█████████▎| 209038/225000 [403:24:06<43:00:17,  9.70s/it]
 93%|█████████▎| 209039/225000 [403:24:16<44:16:27,  9.99s/it]
 93%|█████████▎| 209040/225000 [403:24:27<45:40:27, 10.30s/it]
 93%|█████████▎| 209041/225000 [403:24:37<44:12:22,  9.97s/it]
 93%|█████████▎| 209042/225000 [403:24:45<42:00:27,  9.48s/it]
 93%|█████████▎| 209043/225000 [403:24:54<41:07:43,  9.28s/it]
 93%|█████████▎| 209044/225000 [403:25:03<40:47:29,  9.20s/it]
 93%|█████████▎| 209045/225000 [403:25:12<41:28:27,  9.36s/it]
 93%|█████████▎| 209046/225000 [403:25:21<40:44:24,  9.19s/it]
 93%|█████████▎| 209047/225000 [403:25:33<43:51:42,  9.90s/it]
 93%|█████████▎| 209048/225000 [403:25:38<37:21:51,  8.43s/it]
 93%|█████████▎| 209049/225000 [403:25:44<34:18:07,  7.74s/it]
 93%|█████████▎| 209050/225000 [403:25:51<33:48:24,  7.63s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.427', 'learning_rate': '6.842e-07', 'epoch': '0.9291', 'num_input_tokens_seen': 681781648, 'train_runtime': '1.452e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 209050/225000 [403:25:51<33:48:24,  7.63s/it]
 93%|█████████▎| 209051/225000 [403:26:04<41:12:58,  9.30s/it]
 93%|█████████▎| 209052/225000 [403:26:11<37:14:20,  8.41s/it]
 93%|█████████▎| 209053/225000 [403:26:20<38:30:58,  8.69s/it]
 93%|█████████▎| 209054/225000 [403:26:29<38:25:04,  8.67s/it]
 93%|█████████▎| 209055/225000 [403:26:42<44:00:51,  9.94s/it]
 93%|█████████▎| 209056/225000 [403:26:47<38:07:53,  8.61s/it]
 93%|█████████▎| 209057/225000 [403:26:57<40:00:07,  9.03s/it]
 93%|█████████▎| 209058/225000 [403:27:05<38:32:33,  8.70s/it]
 93%|█████████▎| 209059/225000 [403:27:11<35:06:21,  7.93s/it]
 93%|█████████▎| 209060/225000 [403:27:23<40:38:07,  9.18s/it]
 93%|█████████▎| 209061/225000 [403:27:36<45:13:44, 10.22s/it]
 93%|█████████▎| 209062/225000 [403:27:44<42:38:21,  9.63s/it]
 93%|█████████▎| 209063/225000 [403:27:55<43:46:12,  9.89s/it]
 93%|█████████▎| 209064/225000 [403:27:59<36:54:43,  8.34s/it]
 93%|█████████▎| 209065/225000 [403:28:08<37:38:10,  8.50s/it]
 93%|█████████▎| 209066/225000 [403:28:16<36:56:46,  8.35s/it]
 93%|█████████▎| 209067/225000 [403:28:28<41:23:28,  9.35s/it]
 93%|█████████▎| 209068/225000 [403:28:36<39:27:58,  8.92s/it]
 93%|█████████▎| 209069/225000 [403:28:52<48:40:46, 11.00s/it]
 93%|█████████▎| 209070/225000 [403:29:02<47:39:42, 10.77s/it]
 93%|█████████▎| 209071/225000 [403:29:08<41:24:05,  9.36s/it]
 93%|█████████▎| 209072/225000 [403:29:14<37:03:52,  8.38s/it]
 93%|█████████▎| 209073/225000 [403:29:20<33:21:35,  7.54s/it]
 93%|█████████▎| 209074/225000 [403:29:30<36:45:30,  8.31s/it]
 93%|█████████▎| 209075/225000 [403:29:38<35:55:57,  8.12s/it]
 93%|█████████▎| 209076/225000 [403:29:45<34:22:59,  7.77s/it]
 93%|█████████▎| 209077/225000 [403:29:57<41:15:45,  9.33s/it]
 93%|█████████▎| 209078/225000 [403:30:09<44:19:24, 10.02s/it]
 93%|█████████▎| 209079/225000 [403:30:18<42:34:43,  9.63s/it]
 93%|█████████▎| 209080/225000 [403:30:23<37:09:10,  8.40s/it]
 93%|█████████▎| 209081/225000 [403:30:33<39:05:22,  8.84s/it]
 93%|█████████▎| 209082/225000 [403:30:39<34:24:49,  7.78s/it]
 93%|█████████▎| 209083/225000 [403:30:49<37:30:57,  8.49s/it]
 93%|█████████▎| 209084/225000 [403:31:00<40:51:56,  9.24s/it]
 93%|█████████▎| 209085/225000 [403:31:05<35:54:22,  8.12s/it]
 93%|█████████▎| 209086/225000 [403:31:11<33:25:41,  7.56s/it]
 93%|█████████▎| 209087/225000 [403:31:23<39:12:54,  8.87s/it]
 93%|█████████▎| 209088/225000 [403:31:33<40:07:04,  9.08s/it]
 93%|█████████▎| 209089/225000 [403:31:41<38:43:14,  8.76s/it]
 93%|█████████▎| 209090/225000 [403:31:51<40:05:15,  9.07s/it]
 93%|█████████▎| 209091/225000 [403:31:56<34:50:36,  7.88s/it]
 93%|█████████▎| 209092/225000 [403:32:13<46:52:28, 10.61s/it]
 93%|█████████▎| 209093/225000 [403:32:19<40:30:44,  9.17s/it]
 93%|█████████▎| 209094/225000 [403:32:26<38:04:19,  8.62s/it]
 93%|█████████▎| 209095/225000 [403:32:38<43:10:18,  9.77s/it]
 93%|█████████▎| 209096/225000 [403:32:49<43:40:42,  9.89s/it]
 93%|█████████▎| 209097/225000 [403:32:55<38:36:18,  8.74s/it]
 93%|█████████▎| 209098/225000 [403:33:05<41:09:43,  9.32s/it]
 93%|█████████▎| 209099/225000 [403:33:11<35:57:23,  8.14s/it]
 93%|█████████▎| 209100/225000 [403:33:16<32:11:56,  7.29s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.577', 'learning_rate': '6.8e-07', 'epoch': '0.9293', 'num_input_tokens_seen': 682012512, 'train_runtime': '1.453e+06', 'train_tokens_per_second': '469.4'}

 93%|█████████▎| 209100/225000 [403:33:16<32:11:56,  7.29s/it]
 93%|█████████▎| 209101/225000 [403:33:28<39:03:42,  8.84s/it]
 93%|█████████▎| 209102/225000 [403:33:40<42:04:23,  9.53s/it]
 93%|█████████▎| 209103/225000 [403:33:48<41:10:50,  9.33s/it]
 93%|█████████▎| 209104/225000 [403:33:55<37:49:05,  8.56s/it]
 93%|█████████▎| 209105/225000 [403:34:01<33:37:46,  7.62s/it]
 93%|█████████▎| 209106/225000 [403:34:07<32:35:32,  7.38s/it]
 93%|█████████▎| 209107/225000 [403:34:20<39:57:54,  9.05s/it]
 93%|█████████▎| 209108/225000 [403:34:28<37:49:22,  8.57s/it]
 93%|█████████▎| 209109/225000 [403:34:40<42:13:28,  9.57s/it]
 93%|█████████▎| 209110/225000 [403:34:46<37:57:22,  8.60s/it]
 93%|█████████▎| 209111/225000 [403:34:54<37:06:39,  8.41s/it]
 93%|█████████▎| 209112/225000 [403:35:03<37:20:03,  8.46s/it]
 93%|█████████▎| 209113/225000 [403:35:16<43:35:35,  9.88s/it]
 93%|█████████▎| 209114/225000 [403:35:21<37:35:42,  8.52s/it]
 93%|█████████▎| 209115/225000 [403:35:27<33:27:08,  7.58s/it]
 93%|█████████▎| 209116/225000 [403:35:36<35:18:38,  8.00s/it]
 93%|█████████▎| 209117/225000 [403:35:46<37:53:35,  8.59s/it]
 93%|█████████▎| 209118/225000 [403:35:56<40:13:22,  9.12s/it]
 93%|█████████▎| 209119/225000 [403:36:05<40:02:55,  9.08s/it]
 93%|█████████▎| 209120/225000 [403:36:15<41:17:14,  9.36s/it]
 93%|█████████▎| 209121/225000 [403:36:23<39:06:55,  8.87s/it]
 93%|█████████▎| 209122/225000 [403:36:29<36:03:04,  8.17s/it]
 93%|█████████▎| 209123/225000 [403:36:41<40:59:38,  9.30s/it]
 93%|█████████▎| 209124/225000 [403:36:47<36:46:35,  8.34s/it]
 93%|█████████▎| 209125/225000 [403:36:59<41:32:14,  9.42s/it]
 93%|█████████▎| 209126/225000 [403:37:08<41:28:46,  9.41s/it]
 93%|█████████▎| 209127/225000 [403:37:18<40:57:35,  9.29s/it]
 93%|█████████▎| 209128/225000 [403:37:30<45:12:38, 10.25s/it]
 93%|█████████▎| 209129/225000 [403:37:36<40:03:28,  9.09s/it]
 93%|█████████▎| 209130/225000 [403:37:42<35:55:23,  8.15s/it]
 93%|█████████▎| 209131/225000 [403:37:57<44:17:27, 10.05s/it]
 93%|█████████▎| 209132/225000 [403:38:06<43:00:25,  9.76s/it]
 93%|█████████▎| 209133/225000 [403:38:13<39:12:55,  8.90s/it]
 93%|█████████▎| 209134/225000 [403:38:22<40:13:31,  9.13s/it]
 93%|█████████▎| 209135/225000 [403:38:32<41:26:14,  9.40s/it]
 93%|█████████▎| 209136/225000 [403:38:39<37:44:19,  8.56s/it]
 93%|█████████▎| 209137/225000 [403:38:46<36:08:35,  8.20s/it]
 93%|█████████▎| 209138/225000 [403:39:00<42:57:10,  9.75s/it]
 93%|█████████▎| 209139/225000 [403:39:03<34:47:27,  7.90s/it]
 93%|█████████▎| 209140/225000 [403:39:10<33:06:00,  7.51s/it]
 93%|█████████▎| 209141/225000 [403:39:17<31:58:49,  7.26s/it]
 93%|█████████▎| 209142/225000 [403:39:23<30:36:37,  6.95s/it]
 93%|█████████▎| 209143/225000 [403:39:30<31:24:58,  7.13s/it]
 93%|█████████▎| 209144/225000 [403:39:46<42:07:22,  9.56s/it]
 93%|█████████▎| 209145/225000 [403:39:52<38:06:55,  8.65s/it]
 93%|█████████▎| 209146/225000 [403:39:58<33:55:53,  7.70s/it]
 93%|█████████▎| 209147/225000 [403:40:09<39:03:43,  8.87s/it]
 93%|█████████▎| 209148/225000 [403:40:21<43:20:46,  9.84s/it]
 93%|█████████▎| 209149/225000 [403:40:28<38:34:20,  8.76s/it]
 93%|█████████▎| 209150/225000 [403:40:37<39:59:08,  9.08s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.106', 'learning_rate': '6.757e-07', 'epoch': '0.9296', 'num_input_tokens_seen': 682239280, 'train_runtime': '1.453e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209150/225000 [403:40:37<39:59:08,  9.08s/it]
 93%|█████████▎| 209151/225000 [403:40:43<35:44:20,  8.12s/it]
 93%|█████████▎| 209152/225000 [403:40:54<38:59:20,  8.86s/it]
 93%|█████████▎| 209153/225000 [403:41:00<35:53:57,  8.16s/it]
 93%|█████████▎| 209154/225000 [403:41:15<44:45:30, 10.17s/it]
 93%|█████████▎| 209155/225000 [403:41:23<41:37:22,  9.46s/it]
 93%|█████████▎| 209156/225000 [403:41:32<41:31:06,  9.43s/it]
 93%|█████████▎| 209157/225000 [403:41:43<42:31:04,  9.66s/it]
 93%|█████████▎| 209158/225000 [403:41:52<42:40:17,  9.70s/it]
 93%|█████████▎| 209159/225000 [403:41:58<37:29:42,  8.52s/it]
 93%|█████████▎| 209160/225000 [403:42:11<43:25:26,  9.87s/it]
 93%|█████████▎| 209161/225000 [403:42:17<38:30:24,  8.75s/it]
 93%|█████████▎| 209162/225000 [403:42:27<40:00:35,  9.09s/it]
 93%|█████████▎| 209163/225000 [403:42:32<33:46:52,  7.68s/it]
 93%|█████████▎| 209164/225000 [403:42:43<38:47:23,  8.82s/it]
 93%|█████████▎| 209165/225000 [403:42:56<44:30:01, 10.12s/it]
 93%|█████████▎| 209166/225000 [403:43:05<42:02:39,  9.56s/it]
 93%|█████████▎| 209167/225000 [403:43:11<37:34:37,  8.54s/it]
 93%|█████████▎| 209168/225000 [403:43:24<43:17:40,  9.84s/it]
 93%|█████████▎| 209169/225000 [403:43:29<37:43:37,  8.58s/it]
 93%|█████████▎| 209170/225000 [403:43:40<40:17:03,  9.16s/it]
 93%|█████████▎| 209171/225000 [403:43:45<35:28:47,  8.07s/it]
 93%|█████████▎| 209172/225000 [403:43:55<37:11:05,  8.46s/it]
 93%|█████████▎| 209173/225000 [403:44:04<38:46:35,  8.82s/it]
 93%|█████████▎| 209174/225000 [403:44:14<39:18:08,  8.94s/it]
 93%|█████████▎| 209175/225000 [403:44:23<39:49:52,  9.06s/it]
 93%|█████████▎| 209176/225000 [403:44:29<36:24:34,  8.28s/it]
 93%|█████████▎| 209177/225000 [403:44:35<33:33:05,  7.63s/it]
 93%|█████████▎| 209178/225000 [403:44:42<32:07:17,  7.31s/it]
 93%|█████████▎| 209179/225000 [403:44:55<39:28:41,  8.98s/it]
 93%|█████████▎| 209180/225000 [403:45:08<45:29:33, 10.35s/it]
 93%|█████████▎| 209181/225000 [403:45:13<38:21:07,  8.73s/it]
 93%|█████████▎| 209182/225000 [403:45:18<33:34:33,  7.64s/it]
 93%|█████████▎| 209183/225000 [403:45:29<37:18:58,  8.49s/it]
 93%|█████████▎| 209184/225000 [403:45:38<37:42:44,  8.58s/it]
 93%|█████████▎| 209185/225000 [403:45:44<34:47:36,  7.92s/it]
 93%|█████████▎| 209186/225000 [403:45:55<38:18:17,  8.72s/it]
 93%|█████████▎| 209187/225000 [403:46:02<35:45:42,  8.14s/it]
 93%|█████████▎| 209188/225000 [403:46:10<35:51:05,  8.16s/it]
 93%|█████████▎| 209189/225000 [403:46:20<39:14:44,  8.94s/it]
 93%|█████████▎| 209190/225000 [403:46:28<37:56:45,  8.64s/it]
 93%|█████████▎| 209191/225000 [403:46:40<41:54:49,  9.54s/it]
 93%|█████████▎| 209192/225000 [403:46:50<42:47:50,  9.75s/it]
 93%|█████████▎| 209193/225000 [403:46:56<37:46:53,  8.60s/it]
 93%|█████████▎| 209194/225000 [403:47:06<39:38:20,  9.03s/it]
 93%|█████████▎| 209195/225000 [403:47:17<41:31:40,  9.46s/it]
 93%|█████████▎| 209196/225000 [403:47:26<40:43:02,  9.28s/it]
 93%|█████████▎| 209197/225000 [403:47:34<39:01:38,  8.89s/it]
 93%|█████████▎| 209198/225000 [403:47:40<36:11:16,  8.24s/it]
 93%|█████████▎| 209199/225000 [403:47:49<37:19:15,  8.50s/it]
 93%|█████████▎| 209200/225000 [403:47:56<35:26:05,  8.07s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.295', 'learning_rate': '6.715e-07', 'epoch': '0.9298', 'num_input_tokens_seen': 682471496, 'train_runtime': '1.454e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209200/225000 [403:47:56<35:26:05,  8.07s/it]
 93%|█████████▎| 209201/225000 [403:48:05<36:16:36,  8.27s/it]
 93%|█████████▎| 209202/225000 [403:48:11<32:48:57,  7.48s/it]
 93%|█████████▎| 209203/225000 [403:48:23<38:28:58,  8.77s/it]
 93%|█████████▎| 209204/225000 [403:48:27<32:27:32,  7.40s/it]
 93%|█████████▎| 209205/225000 [403:48:34<32:34:00,  7.42s/it]
 93%|█████████▎| 209206/225000 [403:48:48<41:14:43,  9.40s/it]
 93%|█████████▎| 209207/225000 [403:48:59<43:31:30,  9.92s/it]
 93%|█████████▎| 209208/225000 [403:49:07<40:52:33,  9.32s/it]
 93%|█████████▎| 209209/225000 [403:49:17<41:26:35,  9.45s/it]
 93%|█████████▎| 209210/225000 [403:49:27<42:05:51,  9.60s/it]
 93%|█████████▎| 209211/225000 [403:49:38<44:20:56, 10.11s/it]
 93%|█████████▎| 209212/225000 [403:49:47<42:51:27,  9.77s/it]
 93%|█████████▎| 209213/225000 [403:49:56<41:16:36,  9.41s/it]
 93%|█████████▎| 209214/225000 [403:50:05<41:05:21,  9.37s/it]
 93%|█████████▎| 209215/225000 [403:50:11<36:17:29,  8.28s/it]
 93%|█████████▎| 209216/225000 [403:50:23<41:15:23,  9.41s/it]
 93%|█████████▎| 209217/225000 [403:50:38<48:38:10, 11.09s/it]
 93%|█████████▎| 209218/225000 [403:50:44<41:37:46,  9.50s/it]
 93%|█████████▎| 209219/225000 [403:50:50<37:19:00,  8.51s/it]
 93%|█████████▎| 209220/225000 [403:50:55<33:09:29,  7.56s/it]
 93%|█████████▎| 209221/225000 [403:51:01<30:21:46,  6.93s/it]
 93%|█████████▎| 209222/225000 [403:51:13<37:11:20,  8.49s/it]
 93%|█████████▎| 209223/225000 [403:51:20<35:04:13,  8.00s/it]
 93%|█████████▎| 209224/225000 [403:51:30<37:28:02,  8.55s/it]
 93%|█████████▎| 209225/225000 [403:51:41<41:04:44,  9.37s/it]
 93%|█████████▎| 209226/225000 [403:51:47<36:39:15,  8.37s/it]
 93%|█████████▎| 209227/225000 [403:51:57<38:48:13,  8.86s/it]
 93%|█████████▎| 209228/225000 [403:52:06<39:02:33,  8.91s/it]
 93%|█████████▎| 209229/225000 [403:52:15<39:14:06,  8.96s/it]
 93%|█████████▎| 209230/225000 [403:52:20<33:57:08,  7.75s/it]
 93%|█████████▎| 209231/225000 [403:52:26<32:13:59,  7.36s/it]
 93%|█████████▎| 209232/225000 [403:52:42<43:22:45,  9.90s/it]
 93%|█████████▎| 209233/225000 [403:52:54<45:31:20, 10.39s/it]
 93%|█████████▎| 209234/225000 [403:53:01<41:10:41,  9.40s/it]
 93%|█████████▎| 209235/225000 [403:53:13<45:15:26, 10.33s/it]
 93%|█████████▎| 209236/225000 [403:53:18<37:52:27,  8.65s/it]
 93%|█████████▎| 209237/225000 [403:53:26<36:47:52,  8.40s/it]
 93%|█████████▎| 209238/225000 [403:53:34<36:49:44,  8.41s/it]
 93%|█████████▎| 209239/225000 [403:53:43<37:30:10,  8.57s/it]
 93%|█████████▎| 209240/225000 [403:53:54<40:00:17,  9.14s/it]
 93%|█████████▎| 209241/225000 [403:54:00<36:26:45,  8.33s/it]
 93%|█████████▎| 209242/225000 [403:54:12<41:06:09,  9.39s/it]
 93%|█████████▎| 209243/225000 [403:54:21<40:03:17,  9.15s/it]
 93%|█████████▎| 209244/225000 [403:54:28<37:12:52,  8.50s/it]
 93%|█████████▎| 209245/225000 [403:54:40<42:46:08,  9.77s/it]
 93%|█████████▎| 209246/225000 [403:54:50<43:12:41,  9.87s/it]
 93%|█████████▎| 209247/225000 [403:54:57<39:13:39,  8.96s/it]
 93%|█████████▎| 209248/225000 [403:55:05<37:27:01,  8.56s/it]
 93%|█████████▎| 209249/225000 [403:55:15<39:51:10,  9.11s/it]
 93%|█████████▎| 209250/225000 [403:55:26<41:47:05,  9.55s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.334', 'learning_rate': '6.673e-07', 'epoch': '0.93', 'num_input_tokens_seen': 682706488, 'train_runtime': '1.454e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209250/225000 [403:55:26<41:47:05,  9.55s/it]
 93%|█████████▎| 209251/225000 [403:55:33<38:06:43,  8.71s/it]
 93%|█████████▎| 209252/225000 [403:55:38<33:42:25,  7.71s/it]
 93%|█████████▎| 209253/225000 [403:55:44<31:17:58,  7.16s/it]
 93%|█████████▎| 209254/225000 [403:55:54<35:47:38,  8.18s/it]
 93%|█████████▎| 209255/225000 [403:56:05<38:38:47,  8.84s/it]
 93%|█████████▎| 209256/225000 [403:56:11<34:53:30,  7.98s/it]
 93%|█████████▎| 209257/225000 [403:56:18<34:28:26,  7.88s/it]
 93%|█████████▎| 209258/225000 [403:56:32<41:29:41,  9.49s/it]
 93%|█████████▎| 209259/225000 [403:56:41<40:43:15,  9.31s/it]
 93%|█████████▎| 209260/225000 [403:56:49<39:12:41,  8.97s/it]
 93%|█████████▎| 209261/225000 [403:57:00<42:11:41,  9.65s/it]
 93%|█████████▎| 209262/225000 [403:57:08<40:30:47,  9.27s/it]
 93%|█████████▎| 209263/225000 [403:57:23<47:28:20, 10.86s/it]
 93%|█████████▎| 209264/225000 [403:57:30<42:19:59,  9.68s/it]
 93%|█████████▎| 209265/225000 [403:57:41<44:05:26, 10.09s/it]
 93%|█████████▎| 209266/225000 [403:57:48<39:48:22,  9.11s/it]
 93%|█████████▎| 209267/225000 [403:58:01<44:41:33, 10.23s/it]
 93%|█████████▎| 209268/225000 [403:58:07<39:11:05,  8.97s/it]
 93%|█████████▎| 209269/225000 [403:58:16<40:07:41,  9.18s/it]
 93%|█████████▎| 209270/225000 [403:58:21<34:52:07,  7.98s/it]
 93%|█████████▎| 209271/225000 [403:58:33<39:26:59,  9.03s/it]
 93%|█████████▎| 209272/225000 [403:58:41<38:28:14,  8.81s/it]
 93%|█████████▎| 209273/225000 [403:58:53<41:44:42,  9.56s/it]
 93%|█████████▎| 209274/225000 [403:58:58<36:08:21,  8.27s/it]
 93%|█████████▎| 209275/225000 [403:59:07<37:14:27,  8.53s/it]
 93%|█████████▎| 209276/225000 [403:59:14<35:31:55,  8.14s/it]
 93%|█████████▎| 209277/225000 [403:59:22<35:34:28,  8.15s/it]
 93%|█████████▎| 209278/225000 [403:59:34<40:30:51,  9.28s/it]
 93%|█████████▎| 209279/225000 [403:59:44<40:33:05,  9.29s/it]
 93%|█████████▎| 209280/225000 [403:59:53<40:37:51,  9.30s/it]
 93%|█████████▎| 209281/225000 [404:00:03<42:18:46,  9.69s/it]
 93%|█████████▎| 209282/225000 [404:00:09<37:06:59,  8.50s/it]
 93%|█████████▎| 209283/225000 [404:00:20<40:36:22,  9.30s/it]
 93%|█████████▎| 209284/225000 [404:00:30<41:16:51,  9.46s/it]
 93%|█████████▎| 209285/225000 [404:00:41<42:59:09,  9.85s/it]
 93%|█████████▎| 209286/225000 [404:00:49<40:55:47,  9.38s/it]
 93%|█████████▎| 209287/225000 [404:01:01<44:26:36, 10.18s/it]
 93%|█████████▎| 209288/225000 [404:01:06<37:39:19,  8.63s/it]
 93%|█████████▎| 209289/225000 [404:01:18<41:31:01,  9.51s/it]
 93%|█████████▎| 209290/225000 [404:01:27<40:29:15,  9.28s/it]
 93%|█████████▎| 209291/225000 [404:01:33<37:17:48,  8.55s/it]
 93%|█████████▎| 209292/225000 [404:01:39<33:00:17,  7.56s/it]
 93%|█████████▎| 209293/225000 [404:01:52<40:28:54,  9.28s/it]
 93%|█████████▎| 209294/225000 [404:02:03<42:51:38,  9.82s/it]
 93%|█████████▎| 209295/225000 [404:02:09<38:03:35,  8.72s/it]
 93%|█████████▎| 209296/225000 [404:02:16<35:38:16,  8.17s/it]
 93%|█████████▎| 209297/225000 [404:02:27<39:40:17,  9.09s/it]
 93%|█████████▎| 209298/225000 [404:02:38<41:32:51,  9.53s/it]
 93%|█████████▎| 209299/225000 [404:02:51<46:23:23, 10.64s/it]
 93%|█████████▎| 209300/225000 [404:02:57<40:31:56,  9.29s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.535', 'learning_rate': '6.631e-07', 'epoch': '0.9302', 'num_input_tokens_seen': 682938920, 'train_runtime': '1.455e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209300/225000 [404:02:57<40:31:56,  9.29s/it]
 93%|█████████▎| 209301/225000 [404:03:04<36:49:06,  8.44s/it]
 93%|█████████▎| 209302/225000 [404:03:20<46:58:05, 10.77s/it]
 93%|█████████▎| 209303/225000 [404:03:26<40:17:13,  9.24s/it]
 93%|█████████▎| 209304/225000 [404:03:33<38:17:44,  8.78s/it]
 93%|█████████▎| 209305/225000 [404:03:44<40:17:51,  9.24s/it]
 93%|█████████▎| 209306/225000 [404:03:53<40:51:57,  9.37s/it]
 93%|█████████▎| 209307/225000 [404:04:00<36:45:21,  8.43s/it]
 93%|█████████▎| 209308/225000 [404:04:06<33:52:31,  7.77s/it]
 93%|█████████▎| 209309/225000 [404:04:17<38:39:16,  8.87s/it]
 93%|█████████▎| 209310/225000 [404:04:26<37:58:32,  8.71s/it]
 93%|█████████▎| 209311/225000 [404:04:34<37:23:51,  8.58s/it]
 93%|█████████▎| 209312/225000 [404:04:43<38:42:23,  8.88s/it]
 93%|█████████▎| 209313/225000 [404:04:49<34:33:52,  7.93s/it]
 93%|█████████▎| 209314/225000 [404:05:00<38:20:23,  8.80s/it]
 93%|█████████▎| 209315/225000 [404:05:07<35:25:44,  8.13s/it]
 93%|█████████▎| 209316/225000 [404:05:12<31:32:27,  7.24s/it]
 93%|█████████▎| 209317/225000 [404:05:27<41:30:03,  9.53s/it]
 93%|█████████▎| 209318/225000 [404:05:37<42:37:02,  9.78s/it]
 93%|█████████▎| 209319/225000 [404:05:43<38:17:44,  8.79s/it]
 93%|█████████▎| 209320/225000 [404:05:53<38:59:54,  8.95s/it]
 93%|█████████▎| 209321/225000 [404:06:02<39:09:41,  8.99s/it]
 93%|█████████▎| 209322/225000 [404:06:10<37:50:58,  8.69s/it]
 93%|█████████▎| 209323/225000 [404:06:20<40:04:52,  9.20s/it]
 93%|█████████▎| 209324/225000 [404:06:25<33:51:43,  7.78s/it]
 93%|█████████▎| 209325/225000 [404:06:35<37:06:02,  8.52s/it]
 93%|█████████▎| 209326/225000 [404:06:42<35:08:05,  8.07s/it]
 93%|█████████▎| 209327/225000 [404:06:49<33:14:29,  7.64s/it]
 93%|█████████▎| 209328/225000 [404:06:58<34:58:59,  8.04s/it]
 93%|█████████▎| 209329/225000 [404:07:15<46:36:36, 10.71s/it]
 93%|█████████▎| 209330/225000 [404:07:19<38:28:46,  8.84s/it]
 93%|█████████▎| 209331/225000 [404:07:28<38:20:36,  8.81s/it]
 93%|█████████▎| 209332/225000 [404:07:36<37:34:02,  8.63s/it]
 93%|█████████▎| 209333/225000 [404:07:46<39:26:24,  9.06s/it]
 93%|█████████▎| 209334/225000 [404:07:52<35:56:00,  8.26s/it]
 93%|█████████▎| 209335/225000 [404:08:02<37:16:07,  8.56s/it]
 93%|█████████▎| 209336/225000 [404:08:17<46:02:38, 10.58s/it]
 93%|█████████▎| 209337/225000 [404:08:24<41:10:15,  9.46s/it]
 93%|█████████▎| 209338/225000 [404:08:36<44:30:04, 10.23s/it]
 93%|█████████▎| 209339/225000 [404:08:49<48:33:23, 11.16s/it]
 93%|█████████▎| 209340/225000 [404:08:58<44:54:39, 10.32s/it]
 93%|█████████▎| 209341/225000 [404:09:06<42:54:53,  9.87s/it]
 93%|█████████▎| 209342/225000 [404:09:12<37:58:01,  8.73s/it]
 93%|█████████▎| 209343/225000 [404:09:23<40:40:43,  9.35s/it]
 93%|█████████▎| 209344/225000 [404:09:32<39:54:06,  9.18s/it]
 93%|█████████▎| 209345/225000 [404:09:44<43:59:30, 10.12s/it]
 93%|█████████▎| 209346/225000 [404:09:55<44:50:14, 10.31s/it]
 93%|█████████▎| 209347/225000 [404:10:06<45:04:52, 10.37s/it]
 93%|█████████▎| 209348/225000 [404:10:12<39:30:28,  9.09s/it]
 93%|█████████▎| 209349/225000 [404:10:18<35:28:57,  8.16s/it]
 93%|█████████▎| 209350/225000 [404:10:31<41:52:10,  9.63s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.277', 'learning_rate': '6.59e-07', 'epoch': '0.9304', 'num_input_tokens_seen': 683170560, 'train_runtime': '1.455e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209350/225000 [404:10:31<41:52:10,  9.63s/it]
 93%|█████████▎| 209351/225000 [404:10:40<40:59:07,  9.43s/it]
 93%|█████████▎| 209352/225000 [404:10:44<34:00:45,  7.82s/it]
 93%|█████████▎| 209353/225000 [404:10:57<40:38:30,  9.35s/it]
 93%|█████████▎| 209354/225000 [404:11:04<37:48:16,  8.70s/it]
 93%|█████████▎| 209355/225000 [404:11:09<32:55:38,  7.58s/it]
 93%|█████████▎| 209356/225000 [404:11:16<33:01:55,  7.60s/it]
 93%|█████████▎| 209357/225000 [404:11:23<31:58:53,  7.36s/it]
 93%|█████████▎| 209358/225000 [404:11:35<37:34:52,  8.65s/it]
 93%|█████████▎| 209359/225000 [404:11:43<36:22:26,  8.37s/it]
 93%|█████████▎| 209360/225000 [404:11:49<33:35:25,  7.73s/it]
 93%|█████████▎| 209361/225000 [404:12:05<44:21:02, 10.21s/it]
 93%|█████████▎| 209362/225000 [404:12:17<46:47:19, 10.77s/it]
 93%|█████████▎| 209363/225000 [404:12:24<41:58:30,  9.66s/it]
 93%|█████████▎| 209364/225000 [404:12:33<40:52:32,  9.41s/it]
 93%|█████████▎| 209365/225000 [404:12:39<36:54:23,  8.50s/it]
 93%|█████████▎| 209366/225000 [404:12:48<37:49:17,  8.71s/it]
 93%|█████████▎| 209367/225000 [404:12:57<38:11:06,  8.79s/it]
 93%|█████████▎| 209368/225000 [404:13:09<42:17:13,  9.74s/it]
 93%|█████████▎| 209369/225000 [404:13:15<36:18:59,  8.36s/it]
 93%|█████████▎| 209370/225000 [404:13:22<34:59:01,  8.06s/it]
 93%|█████████▎| 209371/225000 [404:13:37<44:28:31, 10.24s/it]
 93%|█████████▎| 209372/225000 [404:13:43<38:48:03,  8.94s/it]
 93%|█████████▎| 209373/225000 [404:13:57<45:46:19, 10.54s/it]
 93%|█████████▎| 209374/225000 [404:14:03<39:55:55,  9.20s/it]
 93%|█████████▎| 209375/225000 [404:14:14<41:41:44,  9.61s/it]
 93%|█████████▎| 209376/225000 [404:14:19<35:43:39,  8.23s/it]
 93%|█████████▎| 209377/225000 [404:14:26<33:53:02,  7.81s/it]
 93%|█████████▎| 209378/225000 [404:14:36<37:10:15,  8.57s/it]
 93%|█████████▎| 209379/225000 [404:14:46<38:57:44,  8.98s/it]
 93%|█████████▎| 209380/225000 [404:14:55<38:13:07,  8.81s/it]
 93%|█████████▎| 209381/225000 [404:15:07<43:07:11,  9.94s/it]
 93%|█████████▎| 209382/225000 [404:15:12<36:02:50,  8.31s/it]
 93%|█████████▎| 209383/225000 [404:15:21<36:52:58,  8.50s/it]
 93%|█████████▎| 209384/225000 [404:15:30<38:36:02,  8.90s/it]
 93%|█████████▎| 209385/225000 [404:15:37<35:35:22,  8.21s/it]
 93%|█████████▎| 209386/225000 [404:15:46<36:22:11,  8.39s/it]
 93%|█████████▎| 209387/225000 [404:15:57<39:28:52,  9.10s/it]
 93%|█████████▎| 209388/225000 [404:16:06<39:40:57,  9.15s/it]
 93%|█████████▎| 209389/225000 [404:16:15<39:26:36,  9.10s/it]
 93%|█████████▎| 209390/225000 [404:16:24<39:04:32,  9.01s/it]
 93%|█████████▎| 209391/225000 [404:16:32<38:24:56,  8.86s/it]
 93%|█████████▎| 209392/225000 [404:16:40<36:36:47,  8.44s/it]
 93%|█████████▎| 209393/225000 [404:16:47<35:20:35,  8.15s/it]
 93%|█████████▎| 209394/225000 [404:16:58<39:20:26,  9.08s/it]
 93%|█████████▎| 209395/225000 [404:17:11<43:35:56, 10.06s/it]
 93%|█████████▎| 209396/225000 [404:17:20<42:53:13,  9.89s/it]
 93%|█████████▎| 209397/225000 [404:17:27<39:02:12,  9.01s/it]
 93%|█████████▎| 209398/225000 [404:17:34<36:07:01,  8.33s/it]
 93%|█████████▎| 209399/225000 [404:17:39<32:16:39,  7.45s/it]
 93%|█████████▎| 209400/225000 [404:17:53<40:33:16,  9.36s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.182', 'learning_rate': '6.548e-07', 'epoch': '0.9307', 'num_input_tokens_seen': 683396472, 'train_runtime': '1.455e+06', 'train_tokens_per_second': '469.5'}

 93%|█████████▎| 209400/225000 [404:17:53<40:33:16,  9.36s/it]
 93%|█████████▎| 209401/225000 [404:18:00<37:12:31,  8.59s/it]
 93%|█████████▎| 209402/225000 [404:18:11<40:35:09,  9.37s/it]
 93%|█████████▎| 209403/225000 [404:18:17<36:29:39,  8.42s/it]
 93%|█████████▎| 209404/225000 [404:18:25<35:03:35,  8.09s/it]
 93%|█████████▎| 209405/225000 [404:18:36<39:59:59,  9.23s/it]
 93%|█████████▎| 209406/225000 [404:18:47<41:45:25,  9.64s/it]
 93%|█████████▎| 209407/225000 [404:18:52<35:25:40,  8.18s/it]
 93%|█████████▎| 209408/225000 [404:19:02<37:27:49,  8.65s/it]
 93%|█████████▎| 209409/225000 [404:19:12<40:22:14,  9.32s/it]
 93%|█████████▎| 209410/225000 [404:19:24<42:38:27,  9.85s/it]
 93%|█████████▎| 209411/225000 [404:19:36<45:28:20, 10.50s/it]
 93%|█████████▎| 209412/225000 [404:19:45<44:03:17, 10.17s/it]
 93%|█████████▎| 209413/225000 [404:19:55<43:22:46, 10.02s/it]
 93%|█████████▎| 209414/225000 [404:20:05<43:21:32, 10.01s/it]
 93%|█████████▎| 209415/225000 [404:20:12<40:13:07,  9.29s/it]
 93%|█████████▎| 209416/225000 [404:20:18<36:00:59,  8.32s/it]
 93%|█████████▎| 209417/225000 [404:20:31<42:20:02,  9.78s/it]
 93%|█████████▎| 209418/225000 [404:20:43<43:59:51, 10.17s/it]
 93%|█████████▎| 209419/225000 [404:20:56<47:43:18, 11.03s/it]
 93%|█████████▎| 209420/225000 [404:21:01<40:00:40,  9.25s/it]
 93%|█████████▎| 209421/225000 [404:21:09<38:16:10,  8.84s/it]
 93%|█████████▎| 209422/225000 [404:21:22<43:52:36, 10.14s/it]
 93%|█████████▎| 209423/225000 [404:21:31<42:06:57,  9.73s/it]
 93%|█████████▎| 209424/225000 [404:21:39<40:52:21,  9.45s/it]
 93%|█████████▎| 209425/225000 [404:21:49<41:45:24,  9.65s/it]
 93%|█████████▎| 209426/225000 [404:22:00<43:15:37, 10.00s/it]
 93%|█████████▎| 209427/225000 [404:22:08<39:52:08,  9.22s/it]
 93%|█████████▎| 209428/225000 [404:22:17<39:42:49,  9.18s/it]
 93%|█████████▎| 209429/225000 [404:22:26<39:43:30,  9.18s/it]
 93%|█████████▎| 209430/225000 [404:22:31<33:50:27,  7.82s/it]
 93%|█████████▎| 209431/225000 [404:22:43<39:39:55,  9.17s/it]
 93%|█████████▎| 209432/225000 [404:22:48<34:33:16,  7.99s/it]
 93%|█████████▎| 209433/225000 [404:22:59<37:51:36,  8.76s/it]
 93%|█████████▎| 209434/225000 [404:23:10<41:10:59,  9.52s/it]
 93%|█████████▎| 209435/225000 [404:23:18<39:44:55,  9.19s/it]
 93%|█████████▎| 209436/225000 [404:23:26<37:17:28,  8.63s/it]
 93%|█████████▎| 209437/225000 [404:23:33<35:20:43,  8.18s/it]
 93%|█████████▎| 209438/225000 [404:23:42<36:44:01,  8.50s/it]
 93%|█████████▎| 209439/225000 [404:23:56<43:08:47,  9.98s/it]
 93%|█████████▎| 209440/225000 [404:24:03<40:15:09,  9.31s/it]
 93%|█████████▎| 209441/225000 [404:24:15<42:49:25,  9.91s/it]
 93%|█████████▎| 209442/225000 [404:24:22<39:38:02,  9.17s/it]
 93%|█████████▎| 209443/225000 [404:24:26<33:07:13,  7.66s/it]
 93%|█████████▎| 209444/225000 [404:24:35<34:56:18,  8.09s/it]
 93%|█████████▎| 209445/225000 [404:24:42<33:46:09,  7.82s/it]
 93%|█████████▎| 209446/225000 [404:24:54<38:01:21,  8.80s/it]
 93%|█████████▎| 209447/225000 [404:25:02<37:57:12,  8.78s/it]
 93%|█████████▎| 209448/225000 [404:25:17<46:03:45, 10.66s/it]
 93%|█████████▎| 209449/225000 [404:25:25<42:43:06,  9.89s/it]
 93%|█████████▎| 209450/225000 [404:25:32<39:05:05,  9.05s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.118', 'learning_rate': '6.507e-07', 'epoch': '0.9309', 'num_input_tokens_seen': 683633464, 'train_runtime': '1.456e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209450/225000 [404:25:32<39:05:05,  9.05s/it]
 93%|█████████▎| 209451/225000 [404:25:40<36:46:05,  8.51s/it]
 93%|█████████▎| 209452/225000 [404:25:45<32:02:02,  7.42s/it]
 93%|█████████▎| 209453/225000 [404:25:55<36:11:41,  8.38s/it]
 93%|█████████▎| 209454/225000 [404:26:09<43:06:33,  9.98s/it]
 93%|█████████▎| 209455/225000 [404:26:17<40:59:47,  9.49s/it]
 93%|█████████▎| 209456/225000 [404:26:22<35:08:20,  8.14s/it]
 93%|█████████▎| 209457/225000 [404:26:31<36:19:04,  8.41s/it]
 93%|█████████▎| 209458/225000 [404:26:47<45:31:46, 10.55s/it]
 93%|█████████▎| 209459/225000 [404:26:54<40:55:25,  9.48s/it]
 93%|█████████▎| 209460/225000 [404:26:59<35:17:49,  8.18s/it]
 93%|█████████▎| 209461/225000 [404:27:07<34:54:47,  8.09s/it]
 93%|█████████▎| 209462/225000 [404:27:15<34:56:54,  8.10s/it]
 93%|█████████▎| 209463/225000 [404:27:34<49:30:03, 11.47s/it]
 93%|█████████▎| 209464/225000 [404:27:40<41:59:19,  9.73s/it]
 93%|█████████▎| 209465/225000 [404:27:48<39:48:56,  9.23s/it]
 93%|█████████▎| 209466/225000 [404:27:58<40:41:19,  9.43s/it]
 93%|█████████▎| 209467/225000 [404:28:08<41:27:45,  9.61s/it]
 93%|█████████▎| 209468/225000 [404:28:18<42:25:10,  9.83s/it]
 93%|█████████▎| 209469/225000 [404:28:30<44:32:01, 10.32s/it]
 93%|█████████▎| 209470/225000 [404:28:40<44:40:27, 10.36s/it]
 93%|█████████▎| 209471/225000 [404:28:47<40:32:23,  9.40s/it]
 93%|█████████▎| 209472/225000 [404:28:55<38:07:33,  8.84s/it]
 93%|█████████▎| 209473/225000 [404:29:03<37:13:05,  8.63s/it]
 93%|█████████▎| 209474/225000 [404:29:19<46:02:38, 10.68s/it]
 93%|█████████▎| 209475/225000 [404:29:29<46:17:36, 10.73s/it]
 93%|█████████▎| 209476/225000 [404:29:36<40:43:18,  9.44s/it]
 93%|█████████▎| 209477/225000 [404:29:47<42:32:14,  9.86s/it]
 93%|█████████▎| 209478/225000 [404:29:59<45:51:20, 10.64s/it]
 93%|█████████▎| 209479/225000 [404:30:07<42:16:11,  9.80s/it]
 93%|█████████▎| 209480/225000 [404:30:16<41:04:28,  9.53s/it]
 93%|█████████▎| 209481/225000 [404:30:22<36:09:44,  8.39s/it]
 93%|█████████▎| 209482/225000 [404:30:30<36:34:35,  8.49s/it]
 93%|█████████▎| 209483/225000 [404:30:43<41:45:29,  9.69s/it]
 93%|█████████▎| 209484/225000 [404:30:50<38:12:29,  8.86s/it]
 93%|█████████▎| 209485/225000 [404:31:05<45:51:03, 10.64s/it]
 93%|█████████▎| 209486/225000 [404:31:11<40:02:40,  9.29s/it]
 93%|█████████▎| 209487/225000 [404:31:16<35:23:57,  8.21s/it]
 93%|█████████▎| 209488/225000 [404:31:23<33:09:52,  7.70s/it]
 93%|█████████▎| 209489/225000 [404:31:28<30:25:09,  7.06s/it]
 93%|█████████▎| 209490/225000 [404:31:39<35:13:37,  8.18s/it]
 93%|█████████▎| 209491/225000 [404:31:49<37:33:06,  8.72s/it]
 93%|█████████▎| 209492/225000 [404:31:55<34:21:21,  7.98s/it]
 93%|█████████▎| 209493/225000 [404:32:07<38:53:39,  9.03s/it]
 93%|█████████▎| 209494/225000 [404:32:18<40:57:07,  9.51s/it]
 93%|█████████▎| 209495/225000 [404:32:25<37:50:00,  8.78s/it]
 93%|█████████▎| 209496/225000 [404:32:35<40:10:20,  9.33s/it]
 93%|█████████▎| 209497/225000 [404:32:41<35:06:21,  8.15s/it]
 93%|█████████▎| 209498/225000 [404:32:49<35:52:38,  8.33s/it]
 93%|█████████▎| 209499/225000 [404:33:04<43:40:38, 10.14s/it]
 93%|█████████▎| 209500/225000 [404:33:10<38:13:46,  8.88s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.311', 'learning_rate': '6.465e-07', 'epoch': '0.9311', 'num_input_tokens_seen': 683870376, 'train_runtime': '1.456e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209500/225000 [404:33:10<38:13:46,  8.88s/it]
 93%|█████████▎| 209501/225000 [404:33:18<37:41:52,  8.76s/it]
 93%|█████████▎| 209502/225000 [404:33:25<34:56:59,  8.12s/it]
 93%|█████████▎| 209503/225000 [404:33:30<31:36:14,  7.34s/it]
 93%|█████████▎| 209504/225000 [404:33:45<41:39:59,  9.68s/it]
 93%|█████████▎| 209505/225000 [404:33:53<39:27:18,  9.17s/it]
 93%|█████████▎| 209506/225000 [404:34:06<44:10:04, 10.26s/it]
 93%|█████████▎| 209507/225000 [404:34:11<36:32:29,  8.49s/it]
 93%|█████████▎| 209508/225000 [404:34:22<39:59:21,  9.29s/it]
 93%|█████████▎| 209509/225000 [404:34:30<38:13:13,  8.88s/it]
 93%|█████████▎| 209510/225000 [404:34:38<36:53:41,  8.57s/it]
 93%|█████████▎| 209511/225000 [404:34:49<41:06:51,  9.56s/it]
 93%|█████████▎| 209512/225000 [404:34:54<34:09:56,  7.94s/it]
 93%|█████████▎| 209513/225000 [404:35:04<37:15:45,  8.66s/it]
 93%|█████████▎| 209514/225000 [404:35:12<36:23:22,  8.46s/it]
 93%|█████████▎| 209515/225000 [404:35:21<36:55:09,  8.58s/it]
 93%|█████████▎| 209516/225000 [404:35:31<38:42:14,  9.00s/it]
 93%|█████████▎| 209517/225000 [404:35:38<36:04:18,  8.39s/it]
 93%|█████████▎| 209518/225000 [404:35:46<35:43:51,  8.31s/it]
 93%|█████████▎| 209519/225000 [404:35:57<38:58:08,  9.06s/it]
 93%|█████████▎| 209520/225000 [404:36:03<35:26:34,  8.24s/it]
 93%|█████████▎| 209521/225000 [404:36:17<42:52:17,  9.97s/it]
 93%|█████████▎| 209522/225000 [404:36:27<42:47:08,  9.95s/it]
 93%|█████████▎| 209523/225000 [404:36:32<36:48:41,  8.56s/it]
 93%|█████████▎| 209524/225000 [404:36:39<33:54:17,  7.89s/it]
 93%|█████████▎| 209525/225000 [404:36:51<40:13:41,  9.36s/it]
 93%|█████████▎| 209526/225000 [404:37:00<39:57:50,  9.30s/it]
 93%|█████████▎| 209527/225000 [404:37:10<39:44:40,  9.25s/it]
 93%|█████████▎| 209528/225000 [404:37:16<36:38:00,  8.52s/it]
 93%|█████████▎| 209529/225000 [404:37:25<36:40:38,  8.53s/it]
 93%|█████████▎| 209530/225000 [404:37:33<35:38:32,  8.29s/it]
 93%|█████████▎| 209531/225000 [404:37:38<32:05:52,  7.47s/it]
 93%|█████████▎| 209532/225000 [404:37:50<37:18:47,  8.68s/it]
 93%|█████████▎| 209533/225000 [404:37:54<31:40:49,  7.37s/it]
 93%|█████████▎| 209534/225000 [404:38:03<33:46:48,  7.86s/it]
 93%|█████████▎| 209535/225000 [404:38:08<29:57:03,  6.97s/it]
 93%|█████████▎| 209536/225000 [404:38:17<31:59:53,  7.45s/it]
 93%|█████████▎| 209537/225000 [404:38:25<33:14:11,  7.74s/it]
 93%|█████████▎| 209538/225000 [404:38:33<34:03:50,  7.93s/it]
 93%|█████████▎| 209539/225000 [404:38:39<31:40:31,  7.38s/it]
 93%|█████████▎| 209540/225000 [404:38:58<46:02:40, 10.72s/it]
 93%|█████████▎| 209541/225000 [404:39:09<47:02:39, 10.96s/it]
 93%|█████████▎| 209542/225000 [404:39:20<45:53:38, 10.69s/it]
 93%|█████████▎| 209543/225000 [404:39:28<42:36:54,  9.93s/it]
 93%|█████████▎| 209544/225000 [404:39:36<40:21:39,  9.40s/it]
 93%|█████████▎| 209545/225000 [404:39:41<34:40:40,  8.08s/it]
 93%|█████████▎| 209546/225000 [404:39:49<34:55:32,  8.14s/it]
 93%|█████████▎| 209547/225000 [404:39:55<32:04:07,  7.47s/it]
 93%|█████████▎| 209548/225000 [404:40:09<41:02:18,  9.56s/it]
 93%|█████████▎| 209549/225000 [404:40:15<36:02:18,  8.40s/it]
 93%|█████████▎| 209550/225000 [404:40:23<35:20:20,  8.23s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.558', 'learning_rate': '6.424e-07', 'epoch': '0.9313', 'num_input_tokens_seen': 684099768, 'train_runtime': '1.457e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209550/225000 [404:40:23<35:20:20,  8.23s/it]
 93%|█████████▎| 209551/225000 [404:40:33<37:24:23,  8.72s/it]
 93%|█████████▎| 209552/225000 [404:40:40<35:25:05,  8.25s/it]
 93%|█████████▎| 209553/225000 [404:40:48<35:30:42,  8.28s/it]
 93%|█████████▎| 209554/225000 [404:40:55<33:54:56,  7.90s/it]
 93%|█████████▎| 209555/225000 [404:41:10<42:56:25, 10.01s/it]
 93%|█████████▎| 209556/225000 [404:41:24<48:07:07, 11.22s/it]
 93%|█████████▎| 209557/225000 [404:41:32<43:35:52, 10.16s/it]
 93%|█████████▎| 209558/225000 [404:41:43<44:48:17, 10.45s/it]
 93%|█████████▎| 209559/225000 [404:41:49<38:29:38,  8.97s/it]
 93%|█████████▎| 209560/225000 [404:41:54<33:18:33,  7.77s/it]
 93%|█████████▎| 209561/225000 [404:41:59<30:19:06,  7.07s/it]
 93%|█████████▎| 209562/225000 [404:42:13<38:37:06,  9.01s/it]
 93%|█████████▎| 209563/225000 [404:42:25<42:23:21,  9.89s/it]
 93%|█████████▎| 209564/225000 [404:42:31<38:06:32,  8.89s/it]
 93%|█████████▎| 209565/225000 [404:42:39<37:21:13,  8.71s/it]
 93%|█████████▎| 209566/225000 [404:42:47<35:16:11,  8.23s/it]
 93%|█████████▎| 209567/225000 [404:42:55<36:02:44,  8.41s/it]
 93%|█████████▎| 209568/225000 [404:43:07<39:56:49,  9.32s/it]
 93%|█████████▎| 209569/225000 [404:43:16<40:08:27,  9.36s/it]
 93%|█████████▎| 209570/225000 [404:43:26<40:51:15,  9.53s/it]
 93%|█████████▎| 209571/225000 [404:43:33<37:12:53,  8.68s/it]
 93%|█████████▎| 209572/225000 [404:43:42<37:50:38,  8.83s/it]
 93%|█████████▎| 209573/225000 [404:43:56<44:41:12, 10.43s/it]
 93%|█████████▎| 209574/225000 [404:44:02<38:41:40,  9.03s/it]
 93%|█████████▎| 209575/225000 [404:44:09<36:09:41,  8.44s/it]
 93%|█████████▎| 209576/225000 [404:44:20<39:28:59,  9.22s/it]
 93%|█████████▎| 209577/225000 [404:44:27<36:28:40,  8.51s/it]
 93%|█████████▎| 209578/225000 [404:44:34<34:33:26,  8.07s/it]
 93%|█████████▎| 209579/225000 [404:44:41<32:41:26,  7.63s/it]
 93%|█████████▎| 209580/225000 [404:44:54<40:23:39,  9.43s/it]
 93%|█████████▎| 209581/225000 [404:45:02<38:19:41,  8.95s/it]
 93%|█████████▎| 209582/225000 [404:45:11<37:51:24,  8.84s/it]
 93%|█████████▎| 209583/225000 [404:45:18<36:33:59,  8.54s/it]
 93%|█████████▎| 209584/225000 [404:45:26<35:04:18,  8.19s/it]
 93%|█████████▎| 209585/225000 [404:45:33<33:07:47,  7.74s/it]
 93%|█████████▎| 209586/225000 [404:45:43<37:01:54,  8.65s/it]
 93%|█████████▎| 209587/225000 [404:45:54<39:27:26,  9.22s/it]
 93%|█████████▎| 209588/225000 [404:46:04<40:04:18,  9.36s/it]
 93%|█████████▎| 209589/225000 [404:46:14<41:46:05,  9.76s/it]
 93%|█████████▎| 209590/225000 [404:46:21<38:14:07,  8.93s/it]
 93%|█████████▎| 209591/225000 [404:46:32<40:53:10,  9.55s/it]
 93%|█████████▎| 209592/225000 [404:46:39<37:30:20,  8.76s/it]
 93%|█████████▎| 209593/225000 [404:46:49<38:17:45,  8.95s/it]
 93%|█████████▎| 209594/225000 [404:47:03<45:04:15, 10.53s/it]
 93%|█████████▎| 209595/225000 [404:47:10<40:56:20,  9.57s/it]
 93%|█████████▎| 209596/225000 [404:47:20<41:34:02,  9.71s/it]
 93%|█████████▎| 209597/225000 [404:47:28<38:50:03,  9.08s/it]
 93%|█████████▎| 209598/225000 [404:47:38<40:30:35,  9.47s/it]
 93%|█████████▎| 209599/225000 [404:47:45<37:50:47,  8.85s/it]
 93%|█████████▎| 209600/225000 [404:47:51<34:11:56,  7.99s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.883', 'learning_rate': '6.383e-07', 'epoch': '0.9316', 'num_input_tokens_seen': 684333944, 'train_runtime': '1.457e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209600/225000 [404:47:51<34:11:56,  7.99s/it]
 93%|█████████▎| 209601/225000 [404:47:59<34:01:04,  7.95s/it]
 93%|█████████▎| 209602/225000 [404:48:07<33:50:55,  7.91s/it]
 93%|█████████▎| 209603/225000 [404:48:18<36:59:28,  8.65s/it]
 93%|█████████▎| 209604/225000 [404:48:29<40:02:29,  9.36s/it]
 93%|█████████▎| 209605/225000 [404:48:34<35:31:51,  8.31s/it]
 93%|█████████▎| 209606/225000 [404:48:40<32:22:25,  7.57s/it]
 93%|█████████▎| 209607/225000 [404:48:55<41:23:51,  9.68s/it]
 93%|█████████▎| 209608/225000 [404:49:01<37:18:29,  8.73s/it]
 93%|█████████▎| 209609/225000 [404:49:13<41:17:52,  9.66s/it]
 93%|█████████▎| 209610/225000 [404:49:20<37:54:37,  8.87s/it]
 93%|█████████▎| 209611/225000 [404:49:31<40:04:36,  9.38s/it]
 93%|█████████▎| 209612/225000 [404:49:37<36:23:39,  8.51s/it]
 93%|█████████▎| 209613/225000 [404:49:46<37:12:31,  8.71s/it]
 93%|█████████▎| 209614/225000 [404:49:56<38:06:37,  8.92s/it]
 93%|█████████▎| 209615/225000 [404:50:06<39:51:55,  9.33s/it]
 93%|█████████▎| 209616/225000 [404:50:16<40:03:44,  9.37s/it]
 93%|█████████▎| 209617/225000 [404:50:26<41:08:27,  9.63s/it]
 93%|█████████▎| 209618/225000 [404:50:40<46:32:21, 10.89s/it]
 93%|█████████▎| 209619/225000 [404:50:45<39:50:53,  9.33s/it]
 93%|█████████▎| 209620/225000 [404:50:52<36:22:19,  8.51s/it]
 93%|█████████▎| 209621/225000 [404:50:58<32:52:01,  7.69s/it]
 93%|█████████▎| 209622/225000 [404:51:10<38:47:01,  9.08s/it]
 93%|█████████▎| 209623/225000 [404:51:20<40:29:49,  9.48s/it]
 93%|█████████▎| 209624/225000 [404:51:26<34:58:49,  8.19s/it]
 93%|█████████▎| 209625/225000 [404:51:32<31:59:00,  7.49s/it]
 93%|█████████▎| 209626/225000 [404:51:40<33:06:38,  7.75s/it]
 93%|█████████▎| 209627/225000 [404:51:52<38:29:33,  9.01s/it]
 93%|█████████▎| 209628/225000 [404:52:00<37:07:00,  8.69s/it]
 93%|█████████▎| 209629/225000 [404:52:04<31:58:02,  7.49s/it]
 93%|█████████▎| 209630/225000 [404:52:15<35:17:34,  8.27s/it]
 93%|█████████▎| 209631/225000 [404:52:20<31:54:03,  7.47s/it]
 93%|█████████▎| 209632/225000 [404:52:30<34:21:29,  8.05s/it]
 93%|█████████▎| 209633/225000 [404:52:42<39:48:09,  9.32s/it]
 93%|█████████▎| 209634/225000 [404:52:51<39:34:19,  9.27s/it]
 93%|█████████▎| 209635/225000 [404:52:57<35:36:52,  8.34s/it]
 93%|█████████▎| 209636/225000 [404:53:08<38:27:37,  9.01s/it]
 93%|█████████▎| 209637/225000 [404:53:19<40:55:13,  9.59s/it]
 93%|█████████▎| 209638/225000 [404:53:26<37:36:53,  8.81s/it]
 93%|█████████▎| 209639/225000 [404:53:32<34:48:45,  8.16s/it]
 93%|█████████▎| 209640/225000 [404:53:45<41:08:41,  9.64s/it]
 93%|█████████▎| 209641/225000 [404:53:53<38:20:38,  8.99s/it]
 93%|█████████▎| 209642/225000 [404:54:06<43:33:58, 10.21s/it]
 93%|█████████▎| 209643/225000 [404:54:14<41:24:14,  9.71s/it]
 93%|█████████▎| 209644/225000 [404:54:26<44:04:08, 10.33s/it]
 93%|█████████▎| 209645/225000 [404:54:33<38:56:02,  9.13s/it]
 93%|█████████▎| 209646/225000 [404:54:42<39:53:05,  9.35s/it]
 93%|█████████▎| 209647/225000 [404:54:55<43:51:27, 10.28s/it]
 93%|█████████▎| 209648/225000 [404:55:05<43:47:22, 10.27s/it]
 93%|█████████▎| 209649/225000 [404:55:17<45:52:31, 10.76s/it]
 93%|█████████▎| 209650/225000 [404:55:28<46:10:17, 10.83s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.185', 'learning_rate': '6.342e-07', 'epoch': '0.9318', 'num_input_tokens_seen': 684568728, 'train_runtime': '1.458e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209650/225000 [404:55:28<46:10:17, 10.83s/it]
 93%|█████████▎| 209651/225000 [404:55:37<44:13:59, 10.37s/it]
 93%|█████████▎| 209652/225000 [404:55:44<39:33:11,  9.28s/it]
 93%|█████████▎| 209653/225000 [404:55:56<42:21:36,  9.94s/it]
 93%|█████████▎| 209654/225000 [404:56:02<38:01:08,  8.92s/it]
 93%|█████████▎| 209655/225000 [404:56:08<34:05:52,  8.00s/it]
 93%|█████████▎| 209656/225000 [404:56:18<36:32:28,  8.57s/it]
 93%|█████████▎| 209657/225000 [404:56:27<37:06:28,  8.71s/it]
 93%|█████████▎| 209658/225000 [404:56:37<38:18:29,  8.99s/it]
 93%|█████████▎| 209659/225000 [404:56:44<35:55:39,  8.43s/it]
 93%|█████████▎| 209660/225000 [404:56:51<34:03:49,  7.99s/it]
 93%|█████████▎| 209661/225000 [404:56:58<33:15:42,  7.81s/it]
 93%|█████████▎| 209662/225000 [404:57:06<33:48:25,  7.93s/it]
 93%|█████████▎| 209663/225000 [404:57:14<33:56:46,  7.97s/it]
 93%|█████████▎| 209664/225000 [404:57:24<35:36:25,  8.36s/it]
 93%|█████████▎| 209665/225000 [404:57:33<36:52:24,  8.66s/it]
 93%|█████████▎| 209666/225000 [404:57:40<34:45:18,  8.16s/it]
 93%|█████████▎| 209667/225000 [404:57:49<35:30:04,  8.34s/it]
 93%|█████████▎| 209668/225000 [404:57:54<31:56:26,  7.50s/it]
 93%|█████████▎| 209669/225000 [404:58:01<30:50:16,  7.24s/it]
 93%|█████████▎| 209670/225000 [404:58:08<30:06:35,  7.07s/it]
 93%|█████████▎| 209671/225000 [404:58:22<39:54:06,  9.37s/it]
 93%|█████████▎| 209672/225000 [404:58:31<39:34:12,  9.29s/it]
 93%|█████████▎| 209673/225000 [404:58:41<39:26:06,  9.26s/it]
 93%|█████████▎| 209674/225000 [404:58:47<35:37:58,  8.37s/it]
 93%|█████████▎| 209675/225000 [404:58:53<32:56:32,  7.74s/it]
 93%|█████████▎| 209676/225000 [404:59:07<40:30:42,  9.52s/it]
 93%|█████████▎| 209677/225000 [404:59:18<42:50:55, 10.07s/it]
 93%|█████████▎| 209678/225000 [404:59:23<36:23:45,  8.55s/it]
 93%|█████████▎| 209679/225000 [404:59:32<36:21:11,  8.54s/it]
 93%|█████████▎| 209680/225000 [404:59:40<36:10:29,  8.50s/it]
 93%|█████████▎| 209681/225000 [404:59:53<41:25:10,  9.73s/it]
 93%|█████████▎| 209682/225000 [405:00:04<42:54:28, 10.08s/it]
 93%|█████████▎| 209683/225000 [405:00:18<47:52:12, 11.25s/it]
 93%|█████████▎| 209684/225000 [405:00:22<39:31:12,  9.29s/it]
 93%|█████████▎| 209685/225000 [405:00:35<44:02:56, 10.35s/it]
 93%|█████████▎| 209686/225000 [405:00:42<39:59:43,  9.40s/it]
 93%|█████████▎| 209687/225000 [405:00:48<34:53:35,  8.20s/it]
 93%|█████████▎| 209688/225000 [405:00:57<35:41:40,  8.39s/it]
 93%|█████████▎| 209689/225000 [405:01:08<39:36:28,  9.31s/it]
 93%|█████████▎| 209690/225000 [405:01:17<38:49:11,  9.13s/it]
 93%|█████████▎| 209691/225000 [405:01:29<42:15:11,  9.94s/it]
 93%|█████████▎| 209692/225000 [405:01:38<41:59:15,  9.87s/it]
 93%|█████████▎| 209693/225000 [405:01:44<36:31:24,  8.59s/it]
 93%|█████████▎| 209694/225000 [405:01:54<38:19:29,  9.01s/it]
 93%|█████████▎| 209695/225000 [405:01:59<33:18:43,  7.84s/it]
 93%|█████████▎| 209696/225000 [405:02:04<30:24:25,  7.15s/it]
 93%|█████████▎| 209697/225000 [405:02:16<36:14:46,  8.53s/it]
 93%|█████████▎| 209698/225000 [405:02:23<33:59:55,  8.00s/it]
 93%|█████████▎| 209699/225000 [405:02:31<33:40:26,  7.92s/it]
 93%|█████████▎| 209700/225000 [405:02:43<39:01:04,  9.18s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.445', 'learning_rate': '6.301e-07', 'epoch': '0.932', 'num_input_tokens_seen': 684797376, 'train_runtime': '1.458e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209700/225000 [405:02:43<39:01:04,  9.18s/it]
 93%|█████████▎| 209701/225000 [405:02:49<35:33:20,  8.37s/it]
 93%|█████████▎| 209702/225000 [405:03:00<37:53:53,  8.92s/it]
 93%|█████████▎| 209703/225000 [405:03:06<35:07:00,  8.26s/it]
 93%|█████████▎| 209704/225000 [405:03:17<38:23:48,  9.04s/it]
 93%|█████████▎| 209705/225000 [405:03:24<35:04:29,  8.26s/it]
 93%|█████████▎| 209706/225000 [405:03:38<42:38:01, 10.04s/it]
 93%|█████████▎| 209707/225000 [405:03:47<41:18:10,  9.72s/it]
 93%|█████████▎| 209708/225000 [405:03:54<38:28:21,  9.06s/it]
 93%|█████████▎| 209709/225000 [405:04:04<39:08:53,  9.22s/it]
 93%|█████████▎| 209710/225000 [405:04:18<45:16:54, 10.66s/it]
 93%|█████████▎| 209711/225000 [405:04:31<47:50:30, 11.27s/it]
 93%|█████████▎| 209712/225000 [405:04:37<41:56:48,  9.88s/it]
 93%|█████████▎| 209713/225000 [405:04:47<42:12:49,  9.94s/it]
 93%|█████████▎| 209714/225000 [405:04:52<35:34:09,  8.38s/it]
 93%|█████████▎| 209715/225000 [405:05:01<37:00:21,  8.72s/it]
 93%|█████████▎| 209716/225000 [405:05:13<40:27:00,  9.53s/it]
 93%|█████████▎| 209717/225000 [405:05:21<38:51:33,  9.15s/it]
 93%|█████████▎| 209718/225000 [405:05:30<39:04:01,  9.20s/it]
 93%|█████████▎| 209719/225000 [405:05:41<40:59:42,  9.66s/it]
 93%|█████████▎| 209720/225000 [405:05:49<38:09:39,  8.99s/it]
 93%|█████████▎| 209721/225000 [405:05:56<35:48:18,  8.44s/it]
 93%|█████████▎| 209722/225000 [405:06:02<32:53:21,  7.75s/it]
 93%|█████████▎| 209723/225000 [405:06:08<30:25:04,  7.17s/it]
 93%|█████████▎| 209724/225000 [405:06:19<35:39:12,  8.40s/it]
 93%|█████████▎| 209725/225000 [405:06:32<41:13:13,  9.71s/it]
 93%|█████████▎| 209726/225000 [405:06:39<37:27:52,  8.83s/it]
 93%|█████████▎| 209727/225000 [405:06:51<42:33:46, 10.03s/it]
 93%|█████████▎| 209728/225000 [405:07:00<41:01:25,  9.67s/it]
 93%|█████████▎| 209729/225000 [405:07:07<36:47:02,  8.67s/it]
 93%|█████████▎| 209730/225000 [405:07:14<35:30:55,  8.37s/it]
 93%|█████████▎| 209731/225000 [405:07:22<34:27:21,  8.12s/it]
 93%|█████████▎| 209732/225000 [405:07:33<37:55:22,  8.94s/it]
 93%|█████████▎| 209733/225000 [405:07:39<34:56:30,  8.24s/it]
 93%|█████████▎| 209734/225000 [405:07:50<38:17:57,  9.03s/it]
 93%|█████████▎| 209735/225000 [405:08:01<40:20:43,  9.51s/it]
 93%|█████████▎| 209736/225000 [405:08:09<38:11:07,  9.01s/it]
 93%|█████████▎| 209737/225000 [405:08:18<38:29:23,  9.08s/it]
 93%|█████████▎| 209738/225000 [405:08:24<34:14:50,  8.08s/it]
 93%|█████████▎| 209739/225000 [405:08:30<32:09:54,  7.59s/it]
 93%|█████████▎| 209740/225000 [405:08:37<31:11:21,  7.36s/it]
 93%|█████████▎| 209741/225000 [405:08:43<29:44:54,  7.02s/it]
 93%|█████████▎| 209742/225000 [405:08:58<40:02:24,  9.45s/it]
 93%|█████████▎| 209743/225000 [405:09:03<34:18:20,  8.09s/it]
 93%|█████████▎| 209744/225000 [405:09:15<39:28:59,  9.32s/it]
 93%|█████████▎| 209745/225000 [405:09:21<35:08:59,  8.29s/it]
 93%|█████████▎| 209746/225000 [405:09:31<37:39:29,  8.89s/it]
 93%|█████████▎| 209747/225000 [405:09:43<40:48:27,  9.63s/it]
 93%|█████████▎| 209748/225000 [405:09:52<40:48:01,  9.63s/it]
 93%|█████████▎| 209749/225000 [405:10:00<38:22:51,  9.06s/it]
 93%|█████████▎| 209750/225000 [405:10:06<34:33:11,  8.16s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.302', 'learning_rate': '6.261e-07', 'epoch': '0.9322', 'num_input_tokens_seen': 685032456, 'train_runtime': '1.459e+06', 'train_tokens_per_second': '469.6'}

 93%|█████████▎| 209750/225000 [405:10:06<34:33:11,  8.16s/it]
 93%|█████████▎| 209751/225000 [405:10:15<35:32:10,  8.39s/it]
 93%|█████████▎| 209752/225000 [405:10:23<35:01:58,  8.27s/it]
 93%|█████████▎| 209753/225000 [405:10:34<38:49:04,  9.17s/it]
 93%|█████████▎| 209754/225000 [405:10:42<36:32:05,  8.63s/it]
 93%|█████████▎| 209755/225000 [405:10:50<35:56:01,  8.49s/it]
 93%|█████████▎| 209756/225000 [405:11:00<38:12:10,  9.02s/it]
 93%|█████████▎| 209757/225000 [405:11:14<44:07:31, 10.42s/it]
 93%|█████████▎| 209758/225000 [405:11:21<39:51:53,  9.42s/it]
 93%|█████████▎| 209759/225000 [405:11:32<42:31:25, 10.04s/it]
 93%|█████████▎| 209760/225000 [405:11:41<40:46:35,  9.63s/it]
 93%|█████████▎| 209761/225000 [405:11:49<38:49:55,  9.17s/it]
 93%|█████████▎| 209762/225000 [405:11:58<38:50:12,  9.18s/it]
 93%|█████████▎| 209763/225000 [405:12:04<34:38:58,  8.19s/it]
 93%|█████████▎| 209764/225000 [405:12:13<34:53:09,  8.24s/it]
 93%|█████████▎| 209765/225000 [405:12:18<31:31:59,  7.45s/it]
 93%|█████████▎| 209766/225000 [405:12:32<39:36:54,  9.36s/it]
 93%|█████████▎| 209767/225000 [405:12:38<35:39:48,  8.43s/it]
 93%|█████████▎| 209768/225000 [405:12:53<43:53:11, 10.37s/it]
 93%|█████████▎| 209769/225000 [405:13:05<45:07:13, 10.66s/it]
 93%|█████████▎| 209770/225000 [405:13:10<38:34:28,  9.12s/it]
 93%|█████████▎| 209771/225000 [405:13:22<42:05:50,  9.95s/it]
 93%|█████████▎| 209772/225000 [405:13:30<39:53:14,  9.43s/it]
 93%|█████████▎| 209773/225000 [405:13:37<36:38:41,  8.66s/it]
 93%|█████████▎| 209774/225000 [405:13:43<33:03:11,  7.81s/it]
 93%|█████████▎| 209775/225000 [405:13:49<30:30:12,  7.21s/it]
 93%|█████████▎| 209776/225000 [405:14:03<39:47:10,  9.41s/it]
 93%|█████████▎| 209777/225000 [405:14:13<40:21:26,  9.54s/it]
 93%|█████████▎| 209778/225000 [405:14:21<37:52:21,  8.96s/it]
 93%|█████████▎| 209779/225000 [405:14:28<35:17:31,  8.35s/it]
 93%|█████████▎| 209780/225000 [405:14:40<40:44:31,  9.64s/it]
 93%|█████████▎| 209781/225000 [405:14:50<40:14:39,  9.52s/it]
 93%|█████████▎| 209782/225000 [405:14:55<34:56:03,  8.26s/it]
 93%|█████████▎| 209783/225000 [405:15:02<33:13:28,  7.86s/it]
 93%|█████████▎| 209784/225000 [405:15:08<30:59:23,  7.33s/it]
 93%|█████████▎| 209785/225000 [405:15:14<29:40:04,  7.02s/it]
 93%|█████████▎| 209786/225000 [405:15:21<29:34:20,  7.00s/it]
 93%|█████████▎| 209787/225000 [405:15:27<28:33:14,  6.76s/it]
 93%|█████████▎| 209788/225000 [405:15:43<39:16:38,  9.30s/it]
 93%|█████████▎| 209789/225000 [405:15:54<42:20:07, 10.02s/it]
 93%|█████████▎| 209790/225000 [405:16:00<36:18:00,  8.59s/it]
 93%|█████████▎| 209791/225000 [405:16:12<41:08:51,  9.74s/it]
 93%|█████████▎| 209792/225000 [405:16:18<36:53:24,  8.73s/it]
 93%|█████████▎| 209793/225000 [405:16:25<34:25:38,  8.15s/it]
 93%|█████████▎| 209794/225000 [405:16:33<33:37:27,  7.96s/it]
 93%|█████████▎| 209795/225000 [405:16:48<42:45:57, 10.13s/it]
 93%|█████████▎| 209796/225000 [405:17:02<48:28:09, 11.48s/it]
 93%|█████████▎| 209797/225000 [405:17:07<40:05:01,  9.49s/it]
 93%|█████████▎| 209798/225000 [405:17:15<38:10:33,  9.04s/it]
 93%|█████████▎| 209799/225000 [405:17:32<47:31:24, 11.25s/it]
 93%|█████████▎| 209800/225000 [405:17:42<46:26:49, 11.00s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.143', 'learning_rate': '6.22e-07', 'epoch': '0.9324', 'num_input_tokens_seen': 685265624, 'train_runtime': '1.459e+06', 'train_tokens_per_second': '469.7'}

 93%|█████████▎| 209800/225000 [405:17:42<46:26:49, 11.00s/it]
 93%|█████████▎| 209801/225000 [405:17:49<41:21:46,  9.80s/it]
 93%|█████████▎| 209802/225000 [405:17:55<36:27:11,  8.63s/it]
 93%|█████████▎| 209803/225000 [405:18:03<35:50:30,  8.49s/it]
 93%|█████████▎| 209804/225000 [405:18:10<34:16:35,  8.12s/it]
 93%|█████████▎| 209805/225000 [405:18:16<31:20:15,  7.42s/it]
 93%|█████████▎| 209806/225000 [405:18:28<36:27:50,  8.64s/it]
 93%|█████████▎| 209807/225000 [405:18:40<40:39:45,  9.64s/it]
 93%|█████████▎| 209808/225000 [405:18:55<47:32:05, 11.26s/it]
 93%|█████████▎| 209809/225000 [405:19:00<40:20:20,  9.56s/it]
 93%|█████████▎| 209810/225000 [405:19:07<36:56:25,  8.75s/it]
 93%|█████████▎| 209811/225000 [405:19:15<36:12:26,  8.58s/it]
 93%|█████████▎| 209812/225000 [405:19:26<38:39:38,  9.16s/it]
 93%|█████████▎| 209813/225000 [405:19:31<33:52:27,  8.03s/it]
 93%|█████████▎| 209814/225000 [405:19:38<31:41:06,  7.51s/it]
 93%|█████████▎| 209815/225000 [405:19:50<37:31:09,  8.89s/it]
 93%|█████████▎| 209816/225000 [405:19:56<33:40:23,  7.98s/it]
 93%|█████████▎| 209817/225000 [405:20:06<37:04:40,  8.79s/it]
 93%|█████████▎| 209818/225000 [405:20:14<35:49:54,  8.50s/it]
 93%|█████████▎| 209819/225000 [405:20:20<32:44:57,  7.77s/it]
 93%|█████████▎| 209820/225000 [405:20:28<33:03:53,  7.84s/it]
 93%|█████████▎| 209821/225000 [405:20:44<42:47:21, 10.15s/it]
 93%|█████████▎| 209822/225000 [405:20:53<41:56:36,  9.95s/it]
 93%|█████████▎| 209823/225000 [405:21:02<40:38:20,  9.64s/it]
 93%|█████████▎| 209824/225000 [405:21:07<34:39:07,  8.22s/it]
 93%|█████████▎| 209825/225000 [405:21:17<36:47:41,  8.73s/it]
 93%|█████████▎| 209826/225000 [405:21:26<37:15:47,  8.84s/it]
 93%|█████████▎| 209827/225000 [405:21:34<36:17:06,  8.61s/it]
 93%|█████████▎| 209828/225000 [405:21:45<39:41:57,  9.42s/it]
 93%|█████████▎| 209829/225000 [405:21:51<35:01:12,  8.31s/it]
 93%|█████████▎| 209830/225000 [405:21:58<33:15:36,  7.89s/it]
 93%|█████████▎| 209831/225000 [405:22:11<39:21:59,  9.34s/it]
 93%|█████████▎| 209832/225000 [405:22:19<37:23:13,  8.87s/it]
 93%|█████████▎| 209833/225000 [405:22:24<33:07:26,  7.86s/it]
 93%|█████████▎| 209834/225000 [405:22:30<31:00:10,  7.36s/it]
 93%|█████████▎| 209835/225000 [405:22:42<36:30:51,  8.67s/it]
 93%|█████████▎| 209836/225000 [405:22:54<41:24:28,  9.83s/it]
 93%|█████████▎| 209837/225000 [405:23:06<42:59:37, 10.21s/it]
 93%|█████████▎| 209838/225000 [405:23:12<38:33:47,  9.16s/it]
 93%|█████████▎| 209839/225000 [405:23:18<34:10:27,  8.11s/it]
 93%|█████████▎| 209840/225000 [405:23:29<37:34:13,  8.92s/it]
 93%|█████████▎| 209841/225000 [405:23:38<37:52:10,  8.99s/it]
 93%|█████████▎| 209842/225000 [405:23:45<35:02:24,  8.32s/it]
 93%|█████████▎| 209843/225000 [405:23:50<30:56:47,  7.35s/it]
 93%|█████████▎| 209844/225000 [405:24:01<35:21:34,  8.40s/it]
 93%|█████████▎| 209845/225000 [405:24:07<32:28:53,  7.72s/it]
 93%|█████████▎| 209846/225000 [405:24:18<37:22:27,  8.88s/it]
 93%|█████████▎| 209847/225000 [405:24:27<37:33:54,  8.92s/it]
 93%|█████████▎| 209848/225000 [405:24:34<34:11:16,  8.12s/it]
 93%|█████████▎| 209849/225000 [405:24:40<32:30:51,  7.73s/it]
 93%|█████████▎| 209850/225000 [405:24:55<41:28:02,  9.85s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.221', 'learning_rate': '6.18e-07', 'epoch': '0.9327', 'num_input_tokens_seen': 685495368, 'train_runtime': '1.459e+06', 'train_tokens_per_second': '469.7'}

 93%|█████████▎| 209850/225000 [405:24:55<41:28:02,  9.85s/it]
 93%|█████████▎| 209851/225000 [405:25:02<37:03:54,  8.81s/it]
 93%|█████████▎| 209852/225000 [405:25:07<32:35:00,  7.74s/it]
 93%|█████████▎| 209853/225000 [405:25:18<37:06:59,  8.82s/it]
 93%|█████████▎| 209854/225000 [405:25:30<40:27:49,  9.62s/it]
 93%|█████████▎| 209855/225000 [405:25:35<35:10:26,  8.36s/it]
 93%|█████████▎| 209856/225000 [405:25:43<34:42:01,  8.25s/it]
 93%|█████████▎| 209857/225000 [405:25:48<30:35:39,  7.27s/it]
 93%|█████████▎| 209858/225000 [405:25:57<32:53:07,  7.82s/it]
 93%|█████████▎| 209859/225000 [405:26:05<33:25:17,  7.95s/it]
 93%|█████████▎| 209860/225000 [405:26:11<30:46:13,  7.32s/it]
 93%|█████████▎| 209861/225000 [405:26:17<28:15:10,  6.72s/it]
 93%|█████████▎| 209862/225000 [405:26:31<38:16:48,  9.10s/it]
 93%|█████████▎| 209863/225000 [405:26:41<38:36:09,  9.18s/it]
 93%|█████████▎| 209864/225000 [405:26:45<32:52:38,  7.82s/it]
 93%|█████████▎| 209865/225000 [405:26:52<31:06:11,  7.40s/it]
 93%|█████████▎| 209866/225000 [405:27:00<32:01:10,  7.62s/it]
 93%|█████████▎| 209867/225000 [405:27:13<39:14:58,  9.34s/it]
 93%|█████████▎| 209868/225000 [405:27:25<42:21:21, 10.08s/it]
 93%|█████████▎| 209869/225000 [405:27:30<36:05:35,  8.59s/it]
 93%|█████████▎| 209870/225000 [405:27:37<33:57:11,  8.08s/it]
 93%|█████████▎| 209871/225000 [405:27:48<37:36:11,  8.95s/it]
 93%|█████████▎| 209872/225000 [405:27:57<38:11:14,  9.09s/it]
 93%|█████████▎| 209873/225000 [405:28:02<32:49:59,  7.81s/it]
 93%|█████████▎| 209874/225000 [405:28:10<32:51:32,  7.82s/it]
 93%|█████████▎| 209875/225000 [405:28:19<33:42:46,  8.02s/it]
 93%|█████████▎| 209876/225000 [405:28:31<39:32:48,  9.41s/it]
 93%|█████████▎| 209877/225000 [405:28:42<41:35:52,  9.90s/it]
 93%|█████████▎| 209878/225000 [405:28:50<39:14:23,  9.34s/it]
 93%|█████████▎| 209879/225000 [405:28:58<37:25:00,  8.91s/it]
 93%|█████████▎| 209880/225000 [405:29:05<34:49:11,  8.29s/it]
 93%|█████████▎| 209881/225000 [405:29:12<32:37:42,  7.77s/it]
 93%|█████████▎| 209882/225000 [405:29:18<30:32:29,  7.27s/it]
 93%|█████████▎| 209883/225000 [405:29:27<33:39:13,  8.01s/it]
 93%|█████████▎| 209884/225000 [405:29:36<34:30:58,  8.22s/it]
 93%|█████████▎| 209885/225000 [405:29:47<38:17:52,  9.12s/it]
 93%|█████████▎| 209886/225000 [405:29:52<33:14:20,  7.92s/it]
 93%|█████████▎| 209887/225000 [405:30:00<32:16:26,  7.69s/it]
 93%|█████████▎| 209888/225000 [405:30:04<28:10:38,  6.71s/it]
 93%|█████████▎| 209889/225000 [405:30:10<27:29:50,  6.55s/it]
 93%|█████████▎| 209890/225000 [405:30:26<39:35:40,  9.43s/it]
 93%|█████████▎| 209891/225000 [405:30:32<34:33:24,  8.23s/it]
 93%|█████████▎| 209892/225000 [405:30:42<36:42:25,  8.75s/it]
 93%|█████████▎| 209893/225000 [405:30:51<37:28:36,  8.93s/it]
 93%|█████████▎| 209894/225000 [405:31:01<38:07:11,  9.08s/it]
 93%|█████████▎| 209895/225000 [405:31:08<35:59:59,  8.58s/it]
 93%|█████████▎| 209896/225000 [405:31:18<38:15:12,  9.12s/it]
 93%|█████████▎| 209897/225000 [405:31:25<35:10:33,  8.38s/it]
 93%|█████████▎| 209898/225000 [405:31:36<38:59:12,  9.29s/it]
 93%|█████████▎| 209899/225000 [405:31:42<33:57:04,  8.09s/it]
 93%|█████████▎| 209900/225000 [405:31:52<36:52:34,  8.79s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.477', 'learning_rate': '6.139e-07', 'epoch': '0.9329', 'num_input_tokens_seen': 685718272, 'train_runtime': '1.46e+06', 'train_tokens_per_second': '469.7'}

 93%|█████████▎| 209900/225000 [405:31:52<36:52:34,  8.79s/it]
 93%|█████████▎| 209901/225000 [405:31:58<33:33:07,  8.00s/it]
 93%|█████████▎| 209902/225000 [405:32:07<34:47:39,  8.30s/it]
 93%|█████████▎| 209903/225000 [405:32:14<32:50:38,  7.83s/it]
 93%|█████████▎| 209904/225000 [405:32:27<39:30:34,  9.42s/it]
 93%|█████████▎| 209905/225000 [405:32:38<40:46:50,  9.73s/it]
 93%|█████████▎| 209906/225000 [405:32:49<42:38:02, 10.17s/it]
 93%|█████████▎| 209907/225000 [405:32:54<36:14:07,  8.64s/it]
 93%|█████████▎| 209908/225000 [405:33:03<37:15:39,  8.89s/it]
 93%|█████████▎| 209909/225000 [405:33:12<37:02:28,  8.84s/it]
 93%|█████████▎| 209910/225000 [405:33:19<35:08:55,  8.39s/it]
 93%|█████████▎| 209911/225000 [405:33:28<34:52:13,  8.32s/it]
 93%|█████████▎| 209912/225000 [405:33:34<32:35:09,  7.78s/it]
 93%|█████████▎| 209913/225000 [405:33:43<34:31:44,  8.24s/it]
 93%|█████████▎| 209914/225000 [405:33:51<33:45:30,  8.06s/it]
 93%|█████████▎| 209915/225000 [405:33:54<27:56:56,  6.67s/it]
 93%|█████████▎| 209916/225000 [405:34:04<31:35:33,  7.54s/it]
 93%|█████████▎| 209917/225000 [405:34:13<33:02:04,  7.88s/it]
 93%|█████████▎| 209918/225000 [405:34:18<30:18:25,  7.23s/it]
 93%|█████████▎| 209919/225000 [405:34:27<31:44:00,  7.58s/it]
 93%|█████████▎| 209920/225000 [405:34:35<31:57:22,  7.63s/it]
 93%|█████████▎| 209921/225000 [405:34:43<32:57:22,  7.87s/it]
 93%|█████████▎| 209922/225000 [405:34:50<31:43:56,  7.58s/it]
 93%|█████████▎| 209923/225000 [405:35:00<34:39:55,  8.28s/it]
 93%|█████████▎| 209924/225000 [405:35:07<33:26:14,  7.98s/it]
 93%|█████████▎| 209925/225000 [405:35:15<32:51:00,  7.84s/it]
 93%|█████████▎| 209926/225000 [405:35:26<37:22:31,  8.93s/it]
 93%|█████████▎| 209927/225000 [405:35:35<37:37:07,  8.98s/it]
 93%|█████████▎| 209928/225000 [405:35:45<38:42:22,  9.25s/it]
 93%|█████████▎| 209929/225000 [405:35:51<34:49:02,  8.32s/it]
 93%|█████████▎| 209930/225000 [405:36:04<40:58:41,  9.79s/it]
 93%|█████████▎| 209931/225000 [405:36:15<41:29:21,  9.91s/it]
 93%|█████████▎| 209932/225000 [405:36:23<40:10:17,  9.60s/it]
 93%|█████████▎| 209933/225000 [405:36:29<35:11:56,  8.41s/it]
 93%|█████████▎| 209934/225000 [405:36:40<38:54:19,  9.30s/it]
 93%|█████████▎| 209935/225000 [405:36:46<34:39:25,  8.28s/it]
 93%|█████████▎| 209936/225000 [405:36:58<39:21:15,  9.40s/it]
 93%|█████████▎| 209937/225000 [405:37:08<39:23:14,  9.41s/it]
 93%|█████████▎| 209938/225000 [405:37:14<35:19:26,  8.44s/it]
 93%|█████████▎| 209939/225000 [405:37:20<32:09:14,  7.69s/it]
 93%|█████████▎| 209940/225000 [405:37:27<31:54:45,  7.63s/it]
 93%|█████████▎| 209941/225000 [405:37:37<34:56:32,  8.35s/it]
 93%|█████████▎| 209942/225000 [405:37:50<39:43:08,  9.50s/it]
 93%|█████████▎| 209943/225000 [405:37:58<38:52:32,  9.29s/it]
 93%|█████████▎| 209944/225000 [405:38:07<38:06:02,  9.11s/it]
 93%|█████████▎| 209945/225000 [405:38:18<40:32:14,  9.69s/it]
 93%|█████████▎| 209946/225000 [405:38:24<35:10:31,  8.41s/it]
 93%|█████████▎| 209947/225000 [405:38:38<43:06:40, 10.31s/it]
 93%|█████████▎| 209948/225000 [405:38:48<42:08:18, 10.08s/it]
 93%|█████████▎| 209949/225000 [405:38:54<36:59:29,  8.85s/it]
 93%|█████████▎| 209950/225000 [405:39:04<38:15:30,  9.15s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.41', 'learning_rate': '6.099e-07', 'epoch': '0.9331', 'num_input_tokens_seen': 685944664, 'train_runtime': '1.46e+06', 'train_tokens_per_second': '469.7'}

 93%|█████████▎| 209950/225000 [405:39:04<38:15:30,  9.15s/it]
 93%|█████████▎| 209951/225000 [405:39:10<35:02:24,  8.38s/it]
 93%|█████████▎| 209952/225000 [405:39:15<30:59:04,  7.41s/it]
 93%|█████████▎| 209953/225000 [405:39:27<35:56:56,  8.60s/it]
 93%|█████████▎| 209954/225000 [405:39:35<35:38:50,  8.53s/it]
 93%|█████████▎| 209955/225000 [405:39:43<34:59:08,  8.37s/it]
 93%|█████████▎| 209956/225000 [405:39:53<36:15:24,  8.68s/it]
 93%|█████████▎| 209957/225000 [405:39:58<32:18:33,  7.73s/it]
 93%|█████████▎| 209958/225000 [405:40:09<36:13:56,  8.67s/it]
 93%|█████████▎| 209959/225000 [405:40:15<32:31:19,  7.78s/it]
 93%|█████████▎| 209960/225000 [405:40:22<32:13:29,  7.71s/it]
 93%|█████████▎| 209961/225000 [405:40:28<30:06:29,  7.21s/it]
 93%|█████████▎| 209962/225000 [405:40:46<42:53:06, 10.27s/it]
 93%|█████████▎| 209963/225000 [405:40:52<37:39:46,  9.02s/it]
 93%|█████████▎| 209964/225000 [405:40:58<33:53:25,  8.11s/it]
 93%|█████████▎| 209965/225000 [405:41:04<31:39:59,  7.58s/it]
 93%|█████████▎| 209966/225000 [405:41:12<32:13:43,  7.72s/it]
 93%|█████████▎| 209967/225000 [405:41:26<40:11:55,  9.63s/it]
 93%|█████████▎| 209968/225000 [405:41:37<41:02:24,  9.83s/it]
 93%|█████████▎| 209969/225000 [405:41:45<39:14:02,  9.40s/it]
 93%|█████████▎| 209970/225000 [405:41:54<38:35:50,  9.24s/it]
 93%|█████████▎| 209971/225000 [405:42:05<41:15:41,  9.88s/it]
 93%|█████████▎| 209972/225000 [405:42:10<34:37:34,  8.29s/it]
 93%|█████████▎| 209973/225000 [405:42:22<39:28:28,  9.46s/it]
 93%|█████████▎| 209974/225000 [405:42:26<32:45:26,  7.85s/it]
 93%|█████████▎| 209975/225000 [405:42:35<33:57:57,  8.14s/it]
 93%|█████████▎| 209976/225000 [405:42:43<34:17:34,  8.22s/it]
 93%|█████████▎| 209977/225000 [405:42:53<36:46:44,  8.81s/it]
 93%|█████████▎| 209978/225000 [405:42:59<33:01:40,  7.92s/it]
 93%|█████████▎| 209979/225000 [405:43:06<31:19:42,  7.51s/it]
 93%|█████████▎| 209980/225000 [405:43:16<34:04:44,  8.17s/it]
 93%|█████████▎| 209981/225000 [405:43:28<39:55:51,  9.57s/it]
 93%|█████████▎| 209982/225000 [405:43:35<35:43:50,  8.57s/it]
 93%|█████████▎| 209983/225000 [405:43:44<36:38:58,  8.79s/it]
 93%|█████████▎| 209984/225000 [405:44:01<46:32:05, 11.16s/it]
 93%|█████████▎| 209985/225000 [405:44:07<40:30:17,  9.71s/it]
 93%|█████████▎| 209986/225000 [405:44:18<42:21:35, 10.16s/it]
 93%|█████████▎| 209987/225000 [405:44:25<38:11:48,  9.16s/it]
 93%|█████████▎| 209988/225000 [405:44:31<34:37:23,  8.30s/it]
 93%|█████████▎| 209989/225000 [405:44:37<31:24:18,  7.53s/it]
 93%|█████████▎| 209990/225000 [405:44:45<31:32:21,  7.56s/it]
 93%|█████████▎| 209991/225000 [405:44:56<36:49:59,  8.83s/it]
 93%|█████████▎| 209992/225000 [405:45:11<43:25:21, 10.42s/it]
 93%|█████████▎| 209993/225000 [405:45:20<41:56:39, 10.06s/it]
 93%|█████████▎| 209994/225000 [405:45:30<41:53:25, 10.05s/it]
 93%|█████████▎| 209995/225000 [405:45:36<37:16:21,  8.94s/it]
 93%|█████████▎| 209996/225000 [405:45:43<34:43:06,  8.33s/it]
 93%|█████████▎| 209997/225000 [405:45:49<31:27:49,  7.55s/it]
 93%|█████████▎| 209998/225000 [405:45:58<33:45:36,  8.10s/it]
 93%|█████████▎| 209999/225000 [405:46:06<33:58:07,  8.15s/it]
 93%|█████████▎| 210000/225000 [405:46:12<31:19:48,  7.52s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.094', 'learning_rate': '6.059e-07', 'epoch': '0.9333', 'num_input_tokens_seen': 686174568, 'train_runtime': '1.461e+06', 'train_tokens_per_second': '469.7'}

 93%|█████████▎| 210000/225000 [405:46:12<31:19:48,  7.52s/it]
 93%|█████████▎| 210001/225000 [405:46:25<37:13:37,  8.94s/it]
 93%|█████████▎| 210002/225000 [405:46:33<36:26:36,  8.75s/it]
 93%|█████████▎| 210003/225000 [405:46:41<34:58:47,  8.40s/it]
 93%|█████████▎| 210004/225000 [405:46:49<35:07:59,  8.43s/it]
 93%|█████████▎| 210005/225000 [405:46:55<32:28:40,  7.80s/it]
 93%|█████████▎| 210006/225000 [405:47:07<36:38:29,  8.80s/it]
 93%|█████████▎| 210007/225000 [405:47:14<34:33:36,  8.30s/it]
 93%|█████████▎| 210008/225000 [405:47:26<40:08:42,  9.64s/it]
 93%|█████████▎| 210009/225000 [405:47:34<37:14:09,  8.94s/it]
 93%|█████████▎| 210010/225000 [405:47:42<36:09:06,  8.68s/it]
 93%|█████████▎| 210011/225000 [405:47:47<31:30:26,  7.57s/it]
 93%|█████████▎| 210012/225000 [405:47:57<34:52:27,  8.38s/it]
 93%|█████████▎| 210013/225000 [405:48:06<35:28:07,  8.52s/it]
 93%|█████████▎| 210014/225000 [405:48:13<33:33:56,  8.06s/it]
 93%|█████████▎| 210015/225000 [405:48:22<34:28:35,  8.28s/it]
 93%|█████████▎| 210016/225000 [405:48:32<37:31:26,  9.02s/it]
 93%|█████████▎| 210017/225000 [405:48:40<35:27:10,  8.52s/it]
 93%|█████████▎| 210018/225000 [405:48:47<34:05:27,  8.19s/it]
 93%|█████████▎| 210019/225000 [405:48:58<37:51:41,  9.10s/it]
 93%|█████████▎| 210020/225000 [405:49:06<36:27:56,  8.76s/it]
 93%|█████████▎| 210021/225000 [405:49:18<40:06:01,  9.64s/it]
 93%|█████████▎| 210022/225000 [405:49:24<35:29:43,  8.53s/it]
 93%|█████████▎| 210023/225000 [405:49:30<32:35:28,  7.83s/it]
 93%|█████████▎| 210024/225000 [405:49:36<30:14:26,  7.27s/it]
 93%|█████████▎| 210025/225000 [405:49:43<29:04:55,  6.99s/it]
 93%|█████████▎| 210026/225000 [405:49:57<38:38:15,  9.29s/it]
 93%|█████████▎| 210027/225000 [405:50:04<35:48:37,  8.61s/it]
 93%|█████████▎| 210028/225000 [405:50:16<39:13:25,  9.43s/it]
 93%|█████████▎| 210029/225000 [405:50:21<34:09:11,  8.21s/it]
 93%|█████████▎| 210030/225000 [405:50:27<30:53:22,  7.43s/it]
 93%|█████████▎| 210031/225000 [405:50:39<36:42:45,  8.83s/it]
 93%|█████████▎| 210032/225000 [405:50:45<33:15:05,  8.00s/it]
 93%|█████████▎| 210033/225000 [405:50:54<34:30:40,  8.30s/it]
 93%|█████████▎| 210034/225000 [405:51:05<37:38:40,  9.06s/it]
 93%|█████████▎| 210035/225000 [405:51:12<35:06:26,  8.45s/it]
 93%|█████████▎| 210036/225000 [405:51:20<35:21:27,  8.51s/it]
 93%|█████████▎| 210037/225000 [405:51:27<32:38:31,  7.85s/it]
 93%|█████████▎| 210038/225000 [405:51:40<39:59:17,  9.62s/it]
 93%|█████████▎| 210039/225000 [405:51:50<40:11:41,  9.67s/it]
 93%|█████████▎| 210040/225000 [405:51:56<35:22:21,  8.51s/it]
 93%|█████████▎| 210041/225000 [405:52:01<31:43:09,  7.63s/it]
 93%|█████████▎| 210042/225000 [405:52:15<39:22:50,  9.48s/it]
 93%|█████████▎| 210043/225000 [405:52:26<41:03:37,  9.88s/it]
 93%|█████████▎| 210044/225000 [405:52:31<34:36:54,  8.33s/it]
 93%|█████████▎| 210045/225000 [405:52:40<35:53:28,  8.64s/it]
 93%|█████████▎| 210046/225000 [405:52:49<35:45:13,  8.61s/it]
 93%|█████████▎| 210047/225000 [405:53:00<39:40:07,  9.55s/it]
 93%|█████████▎| 210048/225000 [405:53:06<35:05:55,  8.45s/it]
 93%|█████████▎| 210049/225000 [405:53:18<39:22:32,  9.48s/it]
 93%|█████████▎| 210050/225000 [405:53:27<38:04:43,  9.17s/it]
                                                              
{'loss': '1.343', 'grad_norm': '1.601', 'learning_rate': '6.019e-07', 'epoch': '0.9336', 'num_input_tokens_seen': 686407016, 'train_runtime': '1.461e+06', 'train_tokens_per_second': '469.8'}

 93%|█████████▎| 210050/225000 [405:53:27<38:04:43,  9.17s/it]
 93%|█████████▎| 210051/225000 [405:53:34<35:41:07,  8.59s/it]
 93%|█████████▎| 210052/225000 [405:53:45<38:29:40,  9.27s/it]
 93%|█████████▎| 210053/225000 [405:53:50<33:37:18,  8.10s/it]
 93%|█████████▎| 210054/225000 [405:54:00<35:39:38,  8.59s/it]
 93%|█████████▎| 210055/225000 [405:54:06<32:39:59,  7.87s/it]
 93%|█████████▎| 210056/225000 [405:54:14<32:22:47,  7.80s/it]
 93%|█████████▎| 210057/225000 [405:54:19<29:04:20,  7.00s/it]
 93%|█████████▎| 210058/225000 [405:54:25<28:30:11,  6.87s/it]
 93%|█████████▎| 210059/225000 [405:54:34<30:56:35,  7.46s/it]
 93%|█████████▎| 210060/225000 [405:54:48<38:55:51,  9.38s/it]
 93%|█████████▎| 210061/225000 [405:54:57<37:54:01,  9.13s/it]
 93%|█████████▎| 210062/225000 [405:55:09<42:29:19, 10.24s/it]
 93%|█████████▎| 210063/225000 [405:55:16<37:55:04,  9.14s/it]
 93%|█████████▎| 210064/225000 [405:55:25<38:18:54,  9.24s/it]
 93%|█████████▎| 210065/225000 [405:55:33<36:05:20,  8.70s/it]
 93%|█████████▎| 210066/225000 [405:55:44<38:29:22,  9.28s/it]
 93%|█████████▎| 210067/225000 [405:55:50<34:52:37,  8.41s/it]
 93%|█████████▎| 210068/225000 [405:56:00<36:28:24,  8.79s/it]
 93%|█████████▎| 210069/225000 [405:56:04<31:08:32,  7.51s/it]
 93%|█████████▎| 210070/225000 [405:56:11<30:46:54,  7.42s/it]
 93%|█████████▎| 210071/225000 [405:56:19<31:30:06,  7.60s/it]
 93%|█████████▎| 210072/225000 [405:56:31<36:26:52,  8.79s/it]
 93%|█████████▎| 210073/225000 [405:56:38<34:46:28,  8.39s/it]
 93%|█████████▎| 210074/225000 [405:56:47<34:28:49,  8.32s/it]
 93%|█████████▎| 210075/225000 [405:56:56<36:06:32,  8.71s/it]
 93%|█████████▎| 210076/225000 [405:57:04<35:03:07,  8.46s/it]
 93%|█████████▎| 210077/225000 [405:57:13<36:14:08,  8.74s/it]
 93%|█████████▎| 210078/225000 [405:57:26<40:28:47,  9.77s/it]
 93%|█████████▎| 210079/225000 [405:57:35<39:47:36,  9.60s/it]
 93%|█████████▎| 210080/225000 [405:57:42<36:15:48,  8.75s/it]
 93%|█████████▎| 210081/225000 [405:57:49<35:12:30,  8.50s/it]
 93%|█████████▎| 210082/225000 [405:57:59<36:42:20,  8.86s/it]
 93%|█████████▎| 210083/225000 [405:58:09<37:38:27,  9.08s/it]
 93%|█████████▎| 210084/225000 [405:58:15<33:40:22,  8.13s/it]
 93%|█████████▎| 210085/225000 [405:58:25<36:51:52,  8.90s/it]
 93%|█████████▎| 210086/225000 [405:58:31<32:35:08,  7.87s/it]
 93%|█████████▎| 210087/225000 [405:58:41<34:52:21,  8.42s/it]
 93%|█████████▎| 210088/225000 [405:58:46<31:00:53,  7.49s/it]
 93%|█████████▎| 210089/225000 [405:58:57<35:21:21,  8.54s/it]
 93%|█████████▎| 210090/225000 [405:59:08<38:48:34,  9.37s/it]
 93%|█████████▎| 210091/225000 [405:59:13<32:59:14,  7.97s/it]
 93%|█████████▎| 210092/225000 [405:59:19<30:17:09,  7.31s/it]
 93%|█████████▎| 210093/225000 [405:59:25<29:41:26,  7.17s/it]
 93%|█████████▎| 210094/225000 [405:59:33<30:02:32,  7.26s/it]
 93%|█████████▎| 210095/225000 [405:59:48<39:18:44,  9.50s/it]
 93%|█████████▎| 210096/225000 [405:59:58<39:57:42,  9.65s/it]
 93%|█████████▎| 210097/225000 [406:00:08<40:59:08,  9.90s/it]
 93%|█████████▎| 210098/225000 [406:00:15<37:08:45,  8.97s/it]
 93%|█████████▎| 210099/225000 [406:00:21<33:09:16,  8.01s/it]
 93%|█████████▎| 210100/225000 [406:00:27<31:20:53,  7.57s/it]
                                                              
{'loss': '1.337', 'grad_norm': '1.31', 'learning_rate': '5.979e-07', 'epoch': '0.9338', 'num_input_tokens_seen': 686640376, 'train_runtime': '1.462e+06', 'train_tokens_per_second': '469.8'}

 93%|█████████▎| 210100/225000 [406:00:27<31:20:53,  7.57s/it]
 93%|█████████▎| 210101/225000 [406:00:39<36:55:20,  8.92s/it]
 93%|█████████▎| 210102/225000 [406:00:44<31:20:52,  7.58s/it]
 93%|█████████▎| 210103/225000 [406:00:55<36:06:04,  8.72s/it]
 93%|█████████▎| 210104/225000 [406:01:06<38:32:51,  9.32s/it]
 93%|█████████▎| 210105/225000 [406:01:13<36:12:07,  8.75s/it]
 93%|█████████▎| 210106/225000 [406:01:24<38:40:47,  9.35s/it]
 93%|█████████▎| 210107/225000 [406:01:31<36:08:05,  8.73s/it]
 93%|█████████▎| 210108/225000 [406:01:38<33:21:51,  8.07s/it]
 93%|█████████▎| 210109/225000 [406:01:46<32:53:48,  7.95s/it]
 93%|█████████▎| 210110/225000 [406:01:53<31:55:38,  7.72s/it]
 93%|█████████▎| 210111/225000 [406:02:07<40:24:09,  9.77s/it]
 93%|█████████▎| 210112/225000 [406:02:16<39:29:14,  9.55s/it]
 93%|█████████▎| 210113/225000 [406:02:22<34:43:09,  8.40s/it]
 93%|█████████▎| 210114/225000 [406:02:35<40:07:22,  9.70s/it]
 93%|█████████▎| 210115/225000 [406:02:42<36:41:53,  8.88s/it]
 93%|█████████▎| 210116/225000 [406:02:49<35:14:27,  8.52s/it]
 93%|█████████▎| 210117/225000 [406:02:56<33:10:06,  8.02s/it]
 93%|█████████▎| 210118/225000 [406:03:12<42:42:46, 10.33s/it]
 93%|█████████▎| 210119/225000 [406:03:21<40:36:56,  9.83s/it]
 93%|█████████▎| 210120/225000 [406:03:29<38:26:49,  9.30s/it]
 93%|█████████▎| 210121/225000 [406:03:35<34:21:26,  8.31s/it]
 93%|█████████▎| 210122/225000 [406:03:48<41:03:55,  9.94s/it]
 93%|█████████▎| 210123/225000 [406:03:53<34:42:58,  8.40s/it]
 93%|█████████▎| 210124/225000 [406:04:02<35:04:57,  8.49s/it]
 93%|█████████▎| 210125/225000 [406:04:10<34:46:35,  8.42s/it]
 93%|█████████▎| 210126/225000 [406:04:20<36:50:35,  8.92s/it]
 93%|█████████▎| 210127/225000 [406:04:27<33:42:18,  8.16s/it]
 93%|█████████▎| 210128/225000 [406:04:35<34:10:41,  8.27s/it]
 93%|█████████▎| 210129/225000 [406:04:44<35:13:51,  8.53s/it]
 93%|█████████▎| 210130/225000 [406:04:51<33:04:21,  8.01s/it]
 93%|█████████▎| 210131/225000 [406:04:58<31:32:50,  7.64s/it]
 93%|█████████▎| 210132/225000 [406:05:11<38:22:37,  9.29s/it]
 93%|█████████▎| 210133/225000 [406:05:21<39:46:09,  9.63s/it]
 93%|█████████▎| 210134/225000 [406:05:27<34:49:07,  8.43s/it]
 93%|█████████▎| 210135/225000 [406:05:34<32:18:50,  7.83s/it]
 93%|█████████▎| 210136/225000 [406:05:43<34:26:54,  8.34s/it]
 93%|█████████▎| 210137/225000 [406:05:50<32:16:42,  7.82s/it]
 93%|█████████▎| 210138/225000 [406:05:56<30:14:17,  7.32s/it]
 93%|█████████▎| 210139/225000 [406:06:05<32:44:33,  7.93s/it]
 93%|█████████▎| 210140/225000 [406:06:18<38:16:45,  9.27s/it]
 93%|█████████▎| 210141/225000 [406:06:25<36:20:06,  8.80s/it]
 93%|█████████▎| 210142/225000 [406:06:33<34:34:04,  8.38s/it]
 93%|█████████▎| 210143/225000 [406:06:42<36:05:21,  8.74s/it]
 93%|█████████▎| 210144/225000 [406:06:48<32:00:36,  7.76s/it]
 93%|█████████▎| 210145/225000 [406:06:55<30:47:43,  7.46s/it]
 93%|█████████▎| 210146/225000 [406:07:07<36:59:03,  8.96s/it]
 93%|█████████▎| 210147/225000 [406:07:16<36:45:22,  8.91s/it]
 93%|█████████▎| 210148/225000 [406:07:23<34:14:14,  8.30s/it]
 93%|█████████▎| 210149/225000 [406:07:33<36:12:42,  8.78s/it]
 93%|█████████▎| 210150/225000 [406:07:41<35:12:25,  8.54s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.03', 'learning_rate': '5.94e-07', 'epoch': '0.934', 'num_input_tokens_seen': 686873496, 'train_runtime': '1.462e+06', 'train_tokens_per_second': '469.8'}

 93%|█████████▎| 210150/225000 [406:07:41<35:12:25,  8.54s/it]
 93%|█████████▎| 210151/225000 [406:07:51<37:37:51,  9.12s/it]
 93%|█████████▎| 210152/225000 [406:07:59<35:52:55,  8.70s/it]
 93%|█████████▎| 210153/225000 [406:08:04<31:04:32,  7.54s/it]
 93%|█████████▎| 210154/225000 [406:08:09<28:00:58,  6.79s/it]
 93%|█████████▎| 210155/225000 [406:08:21<35:16:05,  8.55s/it]
 93%|█████████▎| 210156/225000 [406:08:29<33:49:08,  8.20s/it]
 93%|█████████▎| 210157/225000 [406:08:41<38:54:47,  9.44s/it]
 93%|█████████▎| 210158/225000 [406:08:47<34:37:22,  8.40s/it]
 93%|█████████▎| 210159/225000 [406:08:52<29:56:28,  7.26s/it]
 93%|█████████▎| 210160/225000 [406:09:03<34:48:30,  8.44s/it]
 93%|█████████▎| 210161/225000 [406:09:11<34:33:56,  8.39s/it]
 93%|█████████▎| 210162/225000 [406:09:18<32:33:43,  7.90s/it]
 93%|█████████▎| 210163/225000 [406:09:31<38:41:45,  9.39s/it]
 93%|█████████▎| 210164/225000 [406:09:39<37:43:55,  9.16s/it]
 93%|█████████▎| 210165/225000 [406:09:45<33:24:52,  8.11s/it]
 93%|█████████▎| 210166/225000 [406:10:00<41:57:43, 10.18s/it]
 93%|█████████▎| 210167/225000 [406:10:05<35:53:12,  8.71s/it]
 93%|█████████▎| 210168/225000 [406:10:11<32:15:35,  7.83s/it]
 93%|█████████▎| 210169/225000 [406:10:23<36:59:17,  8.98s/it]
 93%|█████████▎| 210170/225000 [406:10:29<33:15:31,  8.07s/it]
 93%|█████████▎| 210171/225000 [406:10:36<32:34:54,  7.91s/it]
 93%|█████████▎| 210172/225000 [406:10:42<30:28:10,  7.40s/it]
 93%|█████████▎| 210173/225000 [406:10:52<33:50:56,  8.22s/it]
 93%|█████████▎| 210174/225000 [406:11:01<34:30:16,  8.38s/it]
 93%|█████████▎| 210175/225000 [406:11:07<31:39:11,  7.69s/it]
 93%|█████████▎| 210176/225000 [406:11:13<28:38:39,  6.96s/it]
 93%|█████████▎| 210177/225000 [406:11:20<28:55:58,  7.03s/it]
 93%|█████████▎| 210178/225000 [406:11:27<28:44:57,  6.98s/it]
 93%|█████████▎| 210179/225000 [406:11:43<40:21:41,  9.80s/it]
 93%|█████████▎| 210180/225000 [406:11:51<37:46:07,  9.17s/it]
 93%|█████████▎| 210181/225000 [406:11:59<37:14:14,  9.05s/it]
 93%|█████████▎| 210182/225000 [406:12:07<34:53:08,  8.48s/it]
 93%|█████████▎| 210183/225000 [406:12:14<33:41:10,  8.18s/it]
 93%|█████████▎| 210184/225000 [406:12:23<34:45:25,  8.45s/it]
 93%|█████████▎| 210185/225000 [406:12:29<31:48:14,  7.73s/it]
 93%|█████████▎| 210186/225000 [406:12:37<32:13:09,  7.83s/it]
 93%|█████████▎| 210187/225000 [406:12:52<40:34:59,  9.86s/it]
 93%|█████████▎| 210188/225000 [406:13:00<38:25:01,  9.34s/it]
 93%|█████████▎| 210189/225000 [406:13:05<33:28:00,  8.13s/it]
 93%|█████████▎| 210190/225000 [406:13:16<36:34:31,  8.89s/it]
 93%|█████████▎| 210191/225000 [406:13:22<32:36:36,  7.93s/it]
 93%|█████████▎| 210192/225000 [406:13:33<36:45:34,  8.94s/it]
 93%|█████████▎| 210193/225000 [406:13:41<36:13:13,  8.81s/it]
 93%|█████████▎| 210194/225000 [406:13:49<34:34:37,  8.41s/it]
 93%|█████████▎| 210195/225000 [406:13:57<33:42:41,  8.20s/it]
 93%|█████████▎| 210196/225000 [406:14:13<44:00:41, 10.70s/it]
 93%|█████████▎| 210197/225000 [406:14:19<37:53:47,  9.22s/it]
 93%|█████████▎| 210198/225000 [406:14:28<37:28:56,  9.12s/it]
 93%|█████████▎| 210199/225000 [406:14:37<37:32:54,  9.13s/it]
 93%|█████████▎| 210200/225000 [406:14:48<39:36:57,  9.64s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.404', 'learning_rate': '5.9e-07', 'epoch': '0.9342', 'num_input_tokens_seen': 687101808, 'train_runtime': '1.462e+06', 'train_tokens_per_second': '469.8'}

 93%|█████████▎| 210200/225000 [406:14:48<39:36:57,  9.64s/it]
 93%|█████████▎| 210201/225000 [406:14:57<38:29:32,  9.36s/it]
 93%|█████████▎| 210202/225000 [406:15:06<38:22:59,  9.34s/it]
 93%|█████████▎| 210203/225000 [406:15:13<35:14:32,  8.57s/it]
 93%|█████████▎| 210204/225000 [406:15:19<32:12:35,  7.84s/it]
 93%|█████████▎| 210205/225000 [406:15:25<30:22:32,  7.39s/it]
 93%|█████████▎| 210206/225000 [406:15:31<28:41:12,  6.98s/it]
 93%|█████████▎| 210207/225000 [406:15:36<26:34:52,  6.47s/it]
 93%|█████████▎| 210208/225000 [406:15:48<32:52:31,  8.00s/it]
 93%|█████████▎| 210209/225000 [406:15:58<35:21:48,  8.61s/it]
 93%|█████████▎| 210210/225000 [406:16:06<34:26:54,  8.39s/it]
 93%|█████████▎| 210211/225000 [406:16:13<33:21:47,  8.12s/it]
 93%|█████████▎| 210212/225000 [406:16:19<30:32:10,  7.43s/it]
 93%|█████████▎| 210213/225000 [406:16:29<33:03:52,  8.05s/it]
 93%|█████████▎| 210214/225000 [406:16:39<35:39:59,  8.68s/it]
 93%|█████████▎| 210215/225000 [406:16:44<31:16:09,  7.61s/it]
 93%|█████████▎| 210216/225000 [406:16:50<29:16:24,  7.13s/it]
 93%|█████████▎| 210217/225000 [406:17:02<34:54:33,  8.50s/it]
 93%|█████████▎| 210218/225000 [406:17:07<31:38:17,  7.71s/it]
 93%|█████████▎| 210219/225000 [406:17:18<34:59:22,  8.52s/it]
 93%|█████████▎| 210220/225000 [406:17:27<36:06:42,  8.80s/it]
 93%|█████████▎| 210221/225000 [406:17:35<34:48:28,  8.48s/it]
 93%|█████████▎| 210222/225000 [406:17:41<31:46:04,  7.74s/it]
 93%|█████████▎| 210223/225000 [406:17:48<30:41:55,  7.48s/it]
 93%|█████████▎| 210224/225000 [406:18:01<37:39:08,  9.17s/it]
 93%|█████████▎| 210225/225000 [406:18:07<33:03:02,  8.05s/it]
 93%|█████████▎| 210226/225000 [406:18:13<31:28:36,  7.67s/it]
 93%|█████████▎| 210227/225000 [406:18:24<35:03:33,  8.54s/it]
 93%|█████████▎| 210228/225000 [406:18:31<33:07:54,  8.07s/it]
 93%|█████████▎| 210229/225000 [406:18:43<38:11:03,  9.31s/it]
 93%|█████████▎| 210230/225000 [406:18:49<34:17:16,  8.36s/it]
 93%|█████████▎| 210231/225000 [406:18:59<35:30:56,  8.66s/it]
 93%|█████████▎| 210232/225000 [406:19:09<38:12:02,  9.31s/it]
 93%|█████████▎| 210233/225000 [406:19:17<35:36:43,  8.68s/it]
 93%|█████████▎| 210234/225000 [406:19:25<35:35:16,  8.68s/it]
 93%|█████████▎| 210235/225000 [406:19:31<31:53:39,  7.78s/it]
 93%|█████████▎| 210236/225000 [406:19:43<36:43:47,  8.96s/it]
 93%|█████████▎| 210237/225000 [406:19:56<42:14:20, 10.30s/it]
 93%|█████████▎| 210238/225000 [406:20:06<42:17:20, 10.31s/it]
 93%|█████████▎| 210239/225000 [406:20:12<36:31:52,  8.91s/it]
 93%|█████████▎| 210240/225000 [406:20:21<36:20:49,  8.87s/it]
 93%|█████████▎| 210241/225000 [406:20:27<33:22:01,  8.14s/it]
 93%|█████████▎| 210242/225000 [406:20:38<36:42:40,  8.96s/it]
 93%|█████████▎| 210243/225000 [406:20:50<39:46:52,  9.70s/it]
 93%|█████████▎| 210244/225000 [406:20:55<34:46:19,  8.48s/it]
 93%|█████████▎| 210245/225000 [406:21:02<32:17:32,  7.88s/it]
 93%|█████████▎| 210246/225000 [406:21:13<36:47:20,  8.98s/it]
 93%|█████████▎| 210247/225000 [406:21:21<35:14:01,  8.60s/it]
 93%|█████████▎| 210248/225000 [406:21:30<35:40:43,  8.71s/it]
 93%|█████████▎| 210249/225000 [406:21:36<31:58:23,  7.80s/it]
 93%|█████████▎| 210250/225000 [406:21:42<30:12:59,  7.37s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.546', 'learning_rate': '5.861e-07', 'epoch': '0.9344', 'num_input_tokens_seen': 687327624, 'train_runtime': '1.463e+06', 'train_tokens_per_second': '469.8'}

 93%|█████████▎| 210250/225000 [406:21:42<30:12:59,  7.37s/it]
 93%|█████████▎| 210251/225000 [406:21:52<33:30:06,  8.18s/it]
 93%|█████████▎| 210252/225000 [406:21:58<30:13:30,  7.38s/it]
 93%|█████████▎| 210253/225000 [406:22:13<39:36:19,  9.67s/it]
 93%|█████████▎| 210254/225000 [406:22:19<36:15:17,  8.85s/it]
 93%|█████████▎| 210255/225000 [406:22:29<37:07:53,  9.07s/it]
 93%|█████████▎| 210256/225000 [406:22:34<31:32:59,  7.70s/it]
 93%|█████████▎| 210257/225000 [406:22:45<36:28:02,  8.90s/it]
 93%|█████████▎| 210258/225000 [406:22:49<29:33:33,  7.22s/it]
 93%|█████████▎| 210259/225000 [406:22:56<29:55:31,  7.31s/it]
 93%|█████████▎| 210260/225000 [406:23:04<30:44:00,  7.51s/it]
 93%|█████████▎| 210261/225000 [406:23:17<37:06:38,  9.06s/it]
 93%|█████████▎| 210262/225000 [406:23:25<35:38:20,  8.71s/it]
 93%|█████████▎| 210263/225000 [406:23:35<37:47:39,  9.23s/it]
 93%|█████████▎| 210264/225000 [406:23:41<33:38:37,  8.22s/it]
 93%|█████████▎| 210265/225000 [406:23:48<31:39:16,  7.73s/it]
 93%|█████████▎| 210266/225000 [406:23:54<30:30:25,  7.45s/it]
 93%|█████████▎| 210267/225000 [406:24:05<33:53:54,  8.28s/it]
 93%|█████████▎| 210268/225000 [406:24:11<31:15:08,  7.64s/it]
 93%|█████████▎| 210269/225000 [406:24:16<28:58:10,  7.08s/it]
 93%|█████████▎| 210270/225000 [406:24:23<28:28:46,  6.96s/it]
 93%|█████████▎| 210271/225000 [406:24:35<34:52:58,  8.53s/it]
 93%|█████████▎| 210272/225000 [406:24:46<37:24:53,  9.15s/it]
 93%|█████████▎| 210273/225000 [406:24:54<35:32:48,  8.69s/it]
 93%|█████████▎| 210274/225000 [406:24:59<31:27:38,  7.69s/it]
 93%|█████████▎| 210275/225000 [406:25:12<37:44:16,  9.23s/it]
 93%|█████████▎| 210276/225000 [406:25:19<35:34:33,  8.70s/it]
 93%|█████████▎| 210277/225000 [406:25:31<38:57:00,  9.52s/it]
 93%|█████████▎| 210278/225000 [406:25:42<41:05:24, 10.05s/it]
 93%|█████████▎| 210279/225000 [406:25:49<37:26:24,  9.16s/it]
 93%|█████████▎| 210280/225000 [406:25:58<37:22:14,  9.14s/it]
 93%|█████████▎| 210281/225000 [406:26:06<35:45:36,  8.75s/it]
 93%|█████████▎| 210282/225000 [406:26:14<35:06:49,  8.59s/it]
 93%|█████████▎| 210283/225000 [406:26:22<34:11:53,  8.37s/it]
 93%|█████████▎| 210284/225000 [406:26:31<34:56:54,  8.55s/it]
 93%|█████████▎| 210285/225000 [406:26:39<34:51:19,  8.53s/it]
 93%|█████████▎| 210286/225000 [406:26:47<33:40:51,  8.24s/it]
 93%|█████████▎| 210287/225000 [406:26:58<36:30:38,  8.93s/it]
 93%|█████████▎| 210288/225000 [406:27:05<34:51:38,  8.53s/it]
 93%|█████████▎| 210289/225000 [406:27:18<40:43:05,  9.96s/it]
 93%|█████████▎| 210290/225000 [406:27:23<34:11:23,  8.37s/it]
 93%|█████████▎| 210291/225000 [406:27:36<39:46:27,  9.73s/it]
 93%|█████████▎| 210292/225000 [406:27:44<38:08:17,  9.33s/it]
 93%|█████████▎| 210293/225000 [406:27:54<37:58:26,  9.30s/it]
 93%|█████████▎| 210294/225000 [406:28:05<39:59:50,  9.79s/it]
 93%|█████████▎| 210295/225000 [406:28:10<34:38:50,  8.48s/it]
 93%|█████████▎| 210296/225000 [406:28:14<29:45:58,  7.29s/it]
 93%|█████████▎| 210297/225000 [406:28:25<33:30:08,  8.20s/it]
 93%|█████████▎| 210298/225000 [406:28:32<31:57:06,  7.82s/it]
 93%|█████████▎| 210299/225000 [406:28:42<34:40:42,  8.49s/it]
 93%|█████████▎| 210300/225000 [406:28:51<36:05:45,  8.84s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.189', 'learning_rate': '5.822e-07', 'epoch': '0.9347', 'num_input_tokens_seen': 687557528, 'train_runtime': '1.463e+06', 'train_tokens_per_second': '469.9'}

 93%|█████████▎| 210300/225000 [406:28:51<36:05:45,  8.84s/it]
 93%|█████████▎| 210301/225000 [406:28:57<32:11:58,  7.89s/it]
 93%|█████████▎| 210302/225000 [406:29:09<36:41:09,  8.99s/it]
 93%|█████████▎| 210303/225000 [406:29:17<35:32:54,  8.71s/it]
 93%|█████████▎| 210304/225000 [406:29:26<36:34:24,  8.96s/it]
 93%|█████████▎| 210305/225000 [406:29:35<36:03:58,  8.84s/it]
 93%|█████████▎| 210306/225000 [406:29:41<33:06:49,  8.11s/it]
 93%|█████████▎| 210307/225000 [406:29:49<33:08:09,  8.12s/it]
 93%|█████████▎| 210308/225000 [406:29:54<29:17:10,  7.18s/it]
 93%|█████████▎| 210309/225000 [406:30:03<31:08:55,  7.63s/it]
 93%|█████████▎| 210310/225000 [406:30:14<35:04:11,  8.59s/it]
 93%|█████████▎| 210311/225000 [406:30:20<32:14:58,  7.90s/it]
 93%|█████████▎| 210312/225000 [406:30:26<30:13:46,  7.41s/it]
 93%|█████████▎| 210313/225000 [406:30:32<27:48:04,  6.81s/it]
 93%|█████████▎| 210314/225000 [406:30:45<35:19:50,  8.66s/it]
 93%|█████████▎| 210315/225000 [406:30:57<39:03:48,  9.58s/it]
 93%|█████████▎| 210316/225000 [406:31:03<34:55:57,  8.56s/it]
 93%|█████████▎| 210317/225000 [406:31:14<38:00:28,  9.32s/it]
 93%|█████████▎| 210318/225000 [406:31:23<37:14:15,  9.13s/it]
 93%|█████████▎| 210319/225000 [406:31:28<33:16:46,  8.16s/it]
 93%|█████████▎| 210320/225000 [406:31:41<39:04:13,  9.58s/it]
 93%|█████████▎| 210321/225000 [406:31:51<39:30:19,  9.69s/it]
 93%|█████████▎| 210322/225000 [406:31:56<33:14:12,  8.15s/it]
 93%|█████████▎| 210323/225000 [406:32:02<31:09:09,  7.64s/it]
 93%|█████████▎| 210324/225000 [406:32:12<34:00:31,  8.34s/it]
 93%|█████████▎| 210325/225000 [406:32:17<30:11:02,  7.40s/it]
 93%|█████████▎| 210326/225000 [406:32:24<28:57:30,  7.10s/it]
 93%|█████████▎| 210327/225000 [406:32:33<31:46:33,  7.80s/it]
 93%|█████████▎| 210328/225000 [406:32:39<29:26:37,  7.22s/it]
 93%|█████████▎| 210329/225000 [406:32:47<30:46:35,  7.55s/it]
 93%|█████████▎| 210330/225000 [406:32:59<35:44:15,  8.77s/it]
 93%|█████████▎| 210331/225000 [406:33:04<31:35:17,  7.75s/it]
 93%|█████████▎| 210332/225000 [406:33:14<34:18:26,  8.42s/it]
 93%|█████████▎| 210333/225000 [406:33:24<35:42:49,  8.77s/it]
 93%|█████████▎| 210334/225000 [406:33:31<33:16:44,  8.17s/it]
 93%|█████████▎| 210335/225000 [406:33:39<33:21:59,  8.19s/it]
 93%|█████████▎| 210336/225000 [406:33:47<33:07:57,  8.13s/it]
 93%|█████████▎| 210337/225000 [406:33:56<33:31:44,  8.23s/it]
 93%|█████████▎| 210338/225000 [406:34:04<34:01:53,  8.36s/it]
 93%|█████████▎| 210339/225000 [406:34:10<31:12:33,  7.66s/it]
 93%|█████████▎| 210340/225000 [406:34:24<38:38:45,  9.49s/it]
 93%|█████████▎| 210341/225000 [406:34:30<34:32:26,  8.48s/it]
 93%|█████████▎| 210342/225000 [406:34:37<33:00:17,  8.11s/it]
 93%|█████████▎| 210343/225000 [406:34:47<34:44:53,  8.53s/it]
 93%|█████████▎| 210344/225000 [406:34:57<36:12:27,  8.89s/it]
 93%|█████████▎| 210345/225000 [406:35:05<35:56:11,  8.83s/it]
 93%|█████████▎| 210346/225000 [406:35:13<35:04:14,  8.62s/it]
 93%|█████████▎| 210347/225000 [406:35:21<33:56:04,  8.34s/it]
 93%|█████████▎| 210348/225000 [406:35:29<33:10:59,  8.15s/it]
 93%|█████████▎| 210349/225000 [406:35:39<35:58:49,  8.84s/it]
 93%|█████████▎| 210350/225000 [406:35:47<35:07:43,  8.63s/it]
                                                              
{'loss': '1.292', 'grad_norm': '1.37', 'learning_rate': '5.783e-07', 'epoch': '0.9349', 'num_input_tokens_seen': 687775664, 'train_runtime': '1.464e+06', 'train_tokens_per_second': '469.9'}

 93%|█████████▎| 210350/225000 [406:35:47<35:07:43,  8.63s/it]
 93%|█████████▎| 210351/225000 [406:35:54<32:16:45,  7.93s/it]
 93%|█████████▎| 210352/225000 [406:36:06<37:23:57,  9.19s/it]
 93%|█████████▎| 210353/225000 [406:36:15<36:51:46,  9.06s/it]
 93%|█████████▎| 210354/225000 [406:36:21<33:20:33,  8.20s/it]
 93%|█████████▎| 210355/225000 [406:36:30<34:59:18,  8.60s/it]
 93%|█████████▎| 210356/225000 [406:36:42<38:44:47,  9.53s/it]
 93%|█████████▎| 210357/225000 [406:36:47<33:27:34,  8.23s/it]
 93%|█████████▎| 210358/225000 [406:37:00<39:05:30,  9.61s/it]
 93%|█████████▎| 210359/225000 [406:37:07<35:19:06,  8.68s/it]
 93%|█████████▎| 210360/225000 [406:37:17<37:19:15,  9.18s/it]
 93%|█████████▎| 210361/225000 [406:37:26<37:47:40,  9.29s/it]
 93%|█████████▎| 210362/225000 [406:37:36<38:21:02,  9.43s/it]
 93%|█████████▎| 210363/225000 [406:37:46<39:15:57,  9.66s/it]
 93%|█████████▎| 210364/225000 [406:37:54<36:31:57,  8.99s/it]
 93%|█████████▎| 210365/225000 [406:37:59<31:47:08,  7.82s/it]
 93%|█████████▎| 210366/225000 [406:38:11<37:18:27,  9.18s/it]
 93%|█████████▎| 210367/225000 [406:38:16<32:32:05,  8.00s/it]
 93%|█████████▎| 210368/225000 [406:38:26<34:36:30,  8.51s/it]
 93%|█████████▎| 210369/225000 [406:38:39<40:20:02,  9.92s/it]
 93%|█████████▎| 210370/225000 [406:38:47<37:44:57,  9.29s/it]
 93%|█████████▎| 210371/225000 [406:38:55<36:14:42,  8.92s/it]
 93%|█████████▎| 210372/225000 [406:39:05<37:44:06,  9.29s/it]
 93%|█████████▎| 210373/225000 [406:39:15<37:52:28,  9.32s/it]
 93%|█████████▎| 210374/225000 [406:39:22<34:49:03,  8.57s/it]
 94%|█████████▎| 210375/225000 [406:39:28<32:30:25,  8.00s/it]
 94%|█████████▎| 210376/225000 [406:39:39<35:25:47,  8.72s/it]
 94%|█████████▎| 210377/225000 [406:39:49<37:18:35,  9.19s/it]
 94%|█████████▎| 210378/225000 [406:39:58<36:44:03,  9.04s/it]
 94%|█████████▎| 210379/225000 [406:40:05<34:37:58,  8.53s/it]
 94%|█████████▎| 210380/225000 [406:40:10<30:26:40,  7.50s/it]
 94%|█████████▎| 210381/225000 [406:40:20<33:26:47,  8.24s/it]
 94%|█████████▎| 210382/225000 [406:40:28<33:27:39,  8.24s/it]
 94%|█████████▎| 210383/225000 [406:40:33<29:25:31,  7.25s/it]
 94%|█████████▎| 210384/225000 [406:40:42<31:40:02,  7.80s/it]
 94%|█████████▎| 210385/225000 [406:40:48<29:38:12,  7.30s/it]
 94%|█████████▎| 210386/225000 [406:40:58<32:27:57,  8.00s/it]
 94%|█████████▎| 210387/225000 [406:41:07<33:52:29,  8.35s/it]
 94%|█████████▎| 210388/225000 [406:41:15<32:40:48,  8.05s/it]
 94%|█████████▎| 210389/225000 [406:41:20<29:29:17,  7.27s/it]
 94%|█████████▎| 210390/225000 [406:41:26<28:22:51,  6.99s/it]
 94%|█████████▎| 210391/225000 [406:41:34<29:06:56,  7.17s/it]
 94%|█████████▎| 210392/225000 [406:41:41<29:23:21,  7.24s/it]
 94%|█████████▎| 210393/225000 [406:41:48<29:11:37,  7.20s/it]
 94%|█████████▎| 210394/225000 [406:42:01<35:48:39,  8.83s/it]
 94%|█████████▎| 210395/225000 [406:42:16<43:44:52, 10.78s/it]
 94%|█████████▎| 210396/225000 [406:42:23<38:33:24,  9.50s/it]
 94%|█████████▎| 210397/225000 [406:42:32<37:23:02,  9.22s/it]
 94%|█████████▎| 210398/225000 [406:42:44<41:15:43, 10.17s/it]
 94%|█████████▎| 210399/225000 [406:42:49<34:59:32,  8.63s/it]
 94%|█████████▎| 210400/225000 [406:43:01<39:31:57,  9.75s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.528', 'learning_rate': '5.744e-07', 'epoch': '0.9351', 'num_input_tokens_seen': 688010936, 'train_runtime': '1.464e+06', 'train_tokens_per_second': '469.9'}

 94%|█████████▎| 210400/225000 [406:43:01<39:31:57,  9.75s/it]
 94%|█████████▎| 210401/225000 [406:43:13<42:22:56, 10.45s/it]
 94%|█████████▎| 210402/225000 [406:43:26<45:27:14, 11.21s/it]
 94%|█████████▎| 210403/225000 [406:43:33<40:09:35,  9.90s/it]
 94%|█████████▎| 210404/225000 [406:43:41<38:02:45,  9.38s/it]
 94%|█████████▎| 210405/225000 [406:43:49<35:31:01,  8.76s/it]
 94%|█████████▎| 210406/225000 [406:43:55<32:58:55,  8.14s/it]
 94%|█████████▎| 210407/225000 [406:44:04<33:04:07,  8.16s/it]
 94%|█████████▎| 210408/225000 [406:44:16<37:58:40,  9.37s/it]
 94%|█████████▎| 210409/225000 [406:44:21<32:29:49,  8.02s/it]
 94%|█████████▎| 210410/225000 [406:44:27<30:33:29,  7.54s/it]
 94%|█████████▎| 210411/225000 [406:44:35<30:42:46,  7.58s/it]
 94%|█████████▎| 210412/225000 [406:44:44<33:18:27,  8.22s/it]
 94%|█████████▎| 210413/225000 [406:44:54<34:40:28,  8.56s/it]
 94%|█████████▎| 210414/225000 [406:45:01<33:25:39,  8.25s/it]
 94%|█████████▎| 210415/225000 [406:45:07<30:00:39,  7.41s/it]
 94%|█████████▎| 210416/225000 [406:45:13<28:45:17,  7.10s/it]
 94%|█████████▎| 210417/225000 [406:45:28<38:08:39,  9.42s/it]
 94%|█████████▎| 210418/225000 [406:45:36<36:26:22,  9.00s/it]
 94%|█████████▎| 210419/225000 [406:45:43<34:08:04,  8.43s/it]
 94%|█████████▎| 210420/225000 [406:45:50<32:00:45,  7.90s/it]
 94%|█████████▎| 210421/225000 [406:45:59<33:25:57,  8.26s/it]
 94%|█████████▎| 210422/225000 [406:46:06<32:00:34,  7.90s/it]
 94%|█████████▎| 210423/225000 [406:46:16<34:11:27,  8.44s/it]
 94%|█████████▎| 210424/225000 [406:46:25<35:26:29,  8.75s/it]
 94%|█████████▎| 210425/225000 [406:46:34<35:00:11,  8.65s/it]
 94%|█████████▎| 210426/225000 [406:46:46<39:03:16,  9.65s/it]
 94%|█████████▎| 210427/225000 [406:46:51<34:26:46,  8.51s/it]
 94%|█████████▎| 210428/225000 [406:47:02<37:15:50,  9.21s/it]
 94%|█████████▎| 210429/225000 [406:47:11<36:35:19,  9.04s/it]
 94%|█████████▎| 210430/225000 [406:47:20<36:14:31,  8.95s/it]
 94%|█████████▎| 210431/225000 [406:47:27<34:28:20,  8.52s/it]
 94%|█████████▎| 210432/225000 [406:47:38<37:44:29,  9.33s/it]
 94%|█████████▎| 210433/225000 [406:47:46<35:47:15,  8.84s/it]
 94%|█████████▎| 210434/225000 [406:47:53<33:41:18,  8.33s/it]
 94%|█████████▎| 210435/225000 [406:47:59<31:06:26,  7.69s/it]
 94%|█████████▎| 210436/225000 [406:48:11<36:00:57,  8.90s/it]
 94%|█████████▎| 210437/225000 [406:48:21<37:17:43,  9.22s/it]
 94%|█████████▎| 210438/225000 [406:48:26<32:36:11,  8.06s/it]
 94%|█████████▎| 210439/225000 [406:48:35<33:20:24,  8.24s/it]
 94%|█████████▎| 210440/225000 [406:48:46<36:48:25,  9.10s/it]
 94%|█████████▎| 210441/225000 [406:48:57<38:30:14,  9.52s/it]
 94%|█████████▎| 210442/225000 [406:49:02<33:40:01,  8.33s/it]
 94%|█████████▎| 210443/225000 [406:49:15<38:27:12,  9.51s/it]
 94%|█████████▎| 210444/225000 [406:49:20<34:06:56,  8.44s/it]
 94%|█████████▎| 210445/225000 [406:49:27<31:23:57,  7.77s/it]
 94%|█████████▎| 210446/225000 [406:49:35<32:07:12,  7.95s/it]
 94%|█████████▎| 210447/225000 [406:49:45<34:50:53,  8.62s/it]
 94%|█████████▎| 210448/225000 [406:49:51<31:51:26,  7.88s/it]
 94%|█████████▎| 210449/225000 [406:49:59<31:53:18,  7.89s/it]
 94%|█████████▎| 210450/225000 [406:50:08<32:47:10,  8.11s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.338', 'learning_rate': '5.705e-07', 'epoch': '0.9353', 'num_input_tokens_seen': 688240984, 'train_runtime': '1.465e+06', 'train_tokens_per_second': '469.9'}

 94%|█████████▎| 210450/225000 [406:50:08<32:47:10,  8.11s/it]
 94%|█████████▎| 210451/225000 [406:50:16<32:36:48,  8.07s/it]
 94%|█████████▎| 210452/225000 [406:50:28<38:03:59,  9.42s/it]
 94%|█████████▎| 210453/225000 [406:50:32<31:08:59,  7.71s/it]
 94%|█████████▎| 210454/225000 [406:50:43<35:06:12,  8.69s/it]
 94%|█████████▎| 210455/225000 [406:50:51<33:41:22,  8.34s/it]
 94%|█████████▎| 210456/225000 [406:50:59<33:16:59,  8.24s/it]
 94%|█████████▎| 210457/225000 [406:51:04<29:34:34,  7.32s/it]
 94%|█████████▎| 210458/225000 [406:51:17<36:58:27,  9.15s/it]
 94%|█████████▎| 210459/225000 [406:51:24<34:19:43,  8.50s/it]
 94%|█████████▎| 210460/225000 [406:51:31<32:27:43,  8.04s/it]
 94%|█████████▎| 210461/225000 [406:51:41<34:16:19,  8.49s/it]
 94%|█████████▎| 210462/225000 [406:51:56<42:18:47, 10.48s/it]
 94%|█████████▎| 210463/225000 [406:52:00<35:00:09,  8.67s/it]
 94%|█████████▎| 210464/225000 [406:52:06<31:22:50,  7.77s/it]
 94%|█████████▎| 210465/225000 [406:52:20<39:18:44,  9.74s/it]
 94%|█████████▎| 210466/225000 [406:52:27<35:44:47,  8.85s/it]
 94%|█████████▎| 210467/225000 [406:52:32<31:26:49,  7.79s/it]
 94%|█████████▎| 210468/225000 [406:52:46<39:00:03,  9.66s/it]
 94%|█████████▎| 210469/225000 [406:52:52<34:21:53,  8.51s/it]
 94%|█████████▎| 210470/225000 [406:53:01<34:22:36,  8.52s/it]
 94%|█████████▎| 210471/225000 [406:53:12<37:33:57,  9.31s/it]
 94%|█████████▎| 210472/225000 [406:53:23<39:20:32,  9.75s/it]
 94%|█████████▎| 210473/225000 [406:53:32<38:27:51,  9.53s/it]
 94%|█████████▎| 210474/225000 [406:53:37<33:31:00,  8.31s/it]
 94%|█████████▎| 210475/225000 [406:53:52<41:25:10, 10.27s/it]
 94%|█████████▎| 210476/225000 [406:54:02<41:38:42, 10.32s/it]
 94%|█████████▎| 210477/225000 [406:54:10<38:11:40,  9.47s/it]
 94%|█████████▎| 210478/225000 [406:54:16<33:43:30,  8.36s/it]
 94%|█████████▎| 210479/225000 [406:54:28<38:54:48,  9.65s/it]
 94%|█████████▎| 210480/225000 [406:54:33<32:57:54,  8.17s/it]
 94%|█████████▎| 210481/225000 [406:54:47<39:34:37,  9.81s/it]
 94%|█████████▎| 210482/225000 [406:54:52<33:46:15,  8.37s/it]
 94%|█████████▎| 210483/225000 [406:54:57<30:05:52,  7.46s/it]
 94%|█████████▎| 210484/225000 [406:55:04<29:48:59,  7.39s/it]
 94%|█████████▎| 210485/225000 [406:55:15<34:19:31,  8.51s/it]
 94%|█████████▎| 210486/225000 [406:55:22<31:30:05,  7.81s/it]
 94%|█████████▎| 210487/225000 [406:55:29<30:28:59,  7.56s/it]
 94%|█████████▎| 210488/225000 [406:55:35<29:17:16,  7.27s/it]
 94%|█████████▎| 210489/225000 [406:55:50<38:29:36,  9.55s/it]
 94%|█████████▎| 210490/225000 [406:55:57<35:24:05,  8.78s/it]
 94%|█████████▎| 210491/225000 [406:56:04<33:18:02,  8.26s/it]
 94%|█████████▎| 210492/225000 [406:56:10<30:41:26,  7.62s/it]
 94%|█████████▎| 210493/225000 [406:56:21<34:58:29,  8.68s/it]
 94%|█████████▎| 210494/225000 [406:56:28<32:11:26,  7.99s/it]
 94%|█████████▎| 210495/225000 [406:56:40<37:51:50,  9.40s/it]
 94%|█████████▎| 210496/225000 [406:56:48<35:14:57,  8.75s/it]
 94%|█████████▎| 210497/225000 [406:56:59<37:50:40,  9.39s/it]
 94%|█████████▎| 210498/225000 [406:57:05<34:14:18,  8.50s/it]
 94%|█████████▎| 210499/225000 [406:57:12<32:27:37,  8.06s/it]
 94%|█████████▎| 210500/225000 [406:57:19<31:37:59,  7.85s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.398', 'learning_rate': '5.667e-07', 'epoch': '0.9356', 'num_input_tokens_seen': 688468672, 'train_runtime': '1.465e+06', 'train_tokens_per_second': '469.9'}

 94%|█████████▎| 210500/225000 [406:57:19<31:37:59,  7.85s/it]
 94%|█████████▎| 210501/225000 [406:57:29<33:55:13,  8.42s/it]
 94%|█████████▎| 210502/225000 [406:57:38<34:01:55,  8.45s/it]
 94%|█████████▎| 210503/225000 [406:57:51<39:50:01,  9.89s/it]
 94%|█████████▎| 210504/225000 [406:57:55<33:08:31,  8.23s/it]
 94%|█████████▎| 210505/225000 [406:58:05<34:40:17,  8.61s/it]
 94%|█████████▎| 210506/225000 [406:58:10<30:21:17,  7.54s/it]
 94%|█████████▎| 210507/225000 [406:58:19<32:26:52,  8.06s/it]
 94%|█████████▎| 210508/225000 [406:58:24<29:13:51,  7.26s/it]
 94%|█████████▎| 210509/225000 [406:58:33<30:40:45,  7.62s/it]
 94%|█████████▎| 210510/225000 [406:58:47<38:34:25,  9.58s/it]
 94%|█████████▎| 210511/225000 [406:58:52<32:35:54,  8.10s/it]
 94%|█████████▎| 210512/225000 [406:59:03<35:49:27,  8.90s/it]
 94%|█████████▎| 210513/225000 [406:59:11<34:58:55,  8.69s/it]
 94%|█████████▎| 210514/225000 [406:59:21<37:06:44,  9.22s/it]
 94%|█████████▎| 210515/225000 [406:59:32<39:21:50,  9.78s/it]
 94%|█████████▎| 210516/225000 [406:59:42<38:49:57,  9.65s/it]
 94%|█████████▎| 210517/225000 [406:59:50<37:49:50,  9.40s/it]
 94%|█████████▎| 210518/225000 [407:00:01<39:09:50,  9.74s/it]
 94%|█████████▎| 210519/225000 [407:00:07<34:23:48,  8.55s/it]
 94%|█████████▎| 210520/225000 [407:00:22<42:46:20, 10.63s/it]
 94%|█████████▎| 210521/225000 [407:00:30<39:44:38,  9.88s/it]
 94%|█████████▎| 210522/225000 [407:00:35<33:36:03,  8.35s/it]
 94%|█████████▎| 210523/225000 [407:00:43<33:07:02,  8.24s/it]
 94%|█████████▎| 210524/225000 [407:00:50<31:45:48,  7.90s/it]
 94%|█████████▎| 210525/225000 [407:00:58<31:40:50,  7.88s/it]
 94%|█████████▎| 210526/225000 [407:01:11<37:20:26,  9.29s/it]
 94%|█████████▎| 210527/225000 [407:01:18<34:39:42,  8.62s/it]
 94%|█████████▎| 210528/225000 [407:01:26<34:31:11,  8.59s/it]
 94%|█████████▎| 210529/225000 [407:01:36<35:26:18,  8.82s/it]
 94%|█████████▎| 210530/225000 [407:01:41<31:54:14,  7.94s/it]
 94%|█████████▎| 210531/225000 [407:01:51<33:14:22,  8.27s/it]
 94%|█████████▎| 210532/225000 [407:02:01<35:59:43,  8.96s/it]
 94%|█████████▎| 210533/225000 [407:02:11<37:22:06,  9.30s/it]
 94%|█████████▎| 210534/225000 [407:02:17<33:30:01,  8.34s/it]
 94%|█████████▎| 210535/225000 [407:02:26<34:25:41,  8.57s/it]
 94%|█████████▎| 210536/225000 [407:02:33<31:46:07,  7.91s/it]
 94%|█████████▎| 210537/225000 [407:02:43<34:21:25,  8.55s/it]
 94%|█████████▎| 210538/225000 [407:02:51<33:58:38,  8.46s/it]
 94%|█████████▎| 210539/225000 [407:03:00<34:30:47,  8.59s/it]
 94%|█████████▎| 210540/225000 [407:03:06<31:27:12,  7.83s/it]
 94%|█████████▎| 210541/225000 [407:03:13<30:37:35,  7.63s/it]
 94%|█████████▎| 210542/225000 [407:03:24<34:20:05,  8.55s/it]
 94%|█████████▎| 210543/225000 [407:03:32<33:45:12,  8.41s/it]
 94%|█████████▎| 210544/225000 [407:03:39<32:35:47,  8.12s/it]
 94%|█████████▎| 210545/225000 [407:03:45<30:07:11,  7.50s/it]
 94%|█████████▎| 210546/225000 [407:03:53<29:53:58,  7.45s/it]
 94%|█████████▎| 210547/225000 [407:04:02<32:20:36,  8.06s/it]
 94%|█████████▎| 210548/225000 [407:04:14<36:48:32,  9.17s/it]
 94%|█████████▎| 210549/225000 [407:04:27<41:07:45, 10.25s/it]
 94%|█████████▎| 210550/225000 [407:04:37<41:30:01, 10.34s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.156', 'learning_rate': '5.628e-07', 'epoch': '0.9358', 'num_input_tokens_seen': 688696976, 'train_runtime': '1.465e+06', 'train_tokens_per_second': '469.9'}

 94%|█████████▎| 210550/225000 [407:04:37<41:30:01, 10.34s/it]
 94%|█████████▎| 210551/225000 [407:04:44<36:33:35,  9.11s/it]
 94%|█████████▎| 210552/225000 [407:04:51<35:04:09,  8.74s/it]
 94%|█████████▎| 210553/225000 [407:05:00<34:30:32,  8.60s/it]
 94%|█████████▎| 210554/225000 [407:05:08<33:42:58,  8.40s/it]
 94%|█████████▎| 210555/225000 [407:05:18<36:27:23,  9.09s/it]
 94%|█████████▎| 210556/225000 [407:05:31<40:35:26, 10.12s/it]
 94%|█████████▎| 210557/225000 [407:05:38<37:07:48,  9.25s/it]
 94%|█████████▎| 210558/225000 [407:05:51<41:52:02, 10.44s/it]
 94%|█████████▎| 210559/225000 [407:05:59<39:11:30,  9.77s/it]
 94%|█████████▎| 210560/225000 [407:06:09<39:23:33,  9.82s/it]
 94%|█████████▎| 210561/225000 [407:06:22<43:04:40, 10.74s/it]
 94%|█████████▎| 210562/225000 [407:06:30<39:09:20,  9.76s/it]
 94%|█████████▎| 210563/225000 [407:06:37<36:36:18,  9.13s/it]
 94%|█████████▎| 210564/225000 [407:06:46<36:01:10,  8.98s/it]
 94%|█████████▎| 210565/225000 [407:06:59<40:27:38, 10.09s/it]
 94%|█████████▎| 210566/225000 [407:07:04<34:43:14,  8.66s/it]
 94%|█████████▎| 210567/225000 [407:07:09<30:13:20,  7.54s/it]
 94%|█████████▎| 210568/225000 [407:07:17<30:22:31,  7.58s/it]
 94%|█████████▎| 210569/225000 [407:07:25<30:53:57,  7.71s/it]
 94%|█████████▎| 210570/225000 [407:07:37<36:11:14,  9.03s/it]
 94%|█████████▎| 210571/225000 [407:07:45<35:10:11,  8.77s/it]
 94%|█████████▎| 210572/225000 [407:07:52<33:10:39,  8.28s/it]
 94%|█████████▎| 210573/225000 [407:07:58<30:47:45,  7.68s/it]
 94%|█████████▎| 210574/225000 [407:08:10<35:48:00,  8.93s/it]
 94%|█████████▎| 210575/225000 [407:08:17<32:45:13,  8.17s/it]
 94%|█████████▎| 210576/225000 [407:08:28<37:04:32,  9.25s/it]
 94%|█████████▎| 210577/225000 [407:08:36<34:54:51,  8.71s/it]
 94%|█████████▎| 210578/225000 [407:08:48<39:25:00,  9.84s/it]
 94%|█████████▎| 210579/225000 [407:08:53<33:09:54,  8.28s/it]
 94%|█████████▎| 210580/225000 [407:08:59<30:55:44,  7.72s/it]
 94%|█████████▎| 210581/225000 [407:09:05<28:38:19,  7.15s/it]
 94%|█████████▎| 210582/225000 [407:09:12<28:32:29,  7.13s/it]
 94%|█████████▎| 210583/225000 [407:09:22<32:06:55,  8.02s/it]
 94%|█████████▎| 210584/225000 [407:09:31<32:20:34,  8.08s/it]
 94%|█████████▎| 210585/225000 [407:09:39<32:41:47,  8.17s/it]
 94%|█████████▎| 210586/225000 [407:09:47<32:51:42,  8.21s/it]
 94%|█████████▎| 210587/225000 [407:09:57<34:54:49,  8.72s/it]
 94%|█████████▎| 210588/225000 [407:10:05<34:25:22,  8.60s/it]
 94%|█████████▎| 210589/225000 [407:10:13<32:52:27,  8.21s/it]
 94%|█████████▎| 210590/225000 [407:10:19<29:58:08,  7.49s/it]
 94%|█████████▎| 210591/225000 [407:10:26<30:04:00,  7.51s/it]
 94%|█████████▎| 210592/225000 [407:10:34<30:48:46,  7.70s/it]
 94%|█████████▎| 210593/225000 [407:10:45<34:18:31,  8.57s/it]
 94%|█████████▎| 210594/225000 [407:10:54<34:59:18,  8.74s/it]
 94%|█████████▎| 210595/225000 [407:11:04<36:42:09,  9.17s/it]
 94%|█████████▎| 210596/225000 [407:11:13<35:53:30,  8.97s/it]
 94%|█████████▎| 210597/225000 [407:11:18<31:45:05,  7.94s/it]
 94%|█████████▎| 210598/225000 [407:11:24<29:18:40,  7.33s/it]
 94%|█████████▎| 210599/225000 [407:11:35<32:56:23,  8.23s/it]
 94%|█████████▎| 210600/225000 [407:11:47<38:24:08,  9.60s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.186', 'learning_rate': '5.59e-07', 'epoch': '0.936', 'num_input_tokens_seen': 688926624, 'train_runtime': '1.466e+06', 'train_tokens_per_second': '470'}

 94%|█████████▎| 210600/225000 [407:11:47<38:24:08,  9.60s/it]
 94%|█████████▎| 210601/225000 [407:11:59<40:20:15, 10.09s/it]
 94%|█████████▎| 210602/225000 [407:12:07<38:24:06,  9.60s/it]
 94%|█████████▎| 210603/225000 [407:12:13<33:47:36,  8.45s/it]
 94%|█████████▎| 210604/225000 [407:12:25<37:52:20,  9.47s/it]
 94%|█████████▎| 210605/225000 [407:12:34<37:49:59,  9.46s/it]
 94%|█████████▎| 210606/225000 [407:12:41<34:14:53,  8.57s/it]
 94%|█████████▎| 210607/225000 [407:12:51<36:32:02,  9.14s/it]
 94%|█████████▎| 210608/225000 [407:13:01<37:11:20,  9.30s/it]
 94%|█████████▎| 210609/225000 [407:13:08<35:08:09,  8.79s/it]
 94%|█████████▎| 210610/225000 [407:13:14<31:39:05,  7.92s/it]
 94%|█████████▎| 210611/225000 [407:13:20<29:36:53,  7.41s/it]
 94%|█████████▎| 210612/225000 [407:13:33<35:28:03,  8.87s/it]
 94%|█████████▎| 210613/225000 [407:13:39<32:31:57,  8.14s/it]
 94%|█████████▎| 210614/225000 [407:13:46<30:56:42,  7.74s/it]
 94%|█████████▎| 210615/225000 [407:13:53<29:47:31,  7.46s/it]
 94%|█████████▎| 210616/225000 [407:14:00<29:29:48,  7.38s/it]
 94%|█████████▎| 210617/225000 [407:14:07<28:54:52,  7.24s/it]
 94%|█████████▎| 210618/225000 [407:14:14<29:22:53,  7.35s/it]
 94%|█████████▎| 210619/225000 [407:14:23<30:18:42,  7.59s/it]
 94%|█████████▎| 210620/225000 [407:14:33<33:09:55,  8.30s/it]
 94%|█████████▎| 210621/225000 [407:14:47<40:29:48, 10.14s/it]
 94%|█████████▎| 210622/225000 [407:14:56<39:13:41,  9.82s/it]
 94%|█████████▎| 210623/225000 [407:15:03<35:22:27,  8.86s/it]
 94%|█████████▎| 210624/225000 [407:15:17<41:52:50, 10.49s/it]
 94%|█████████▎| 210625/225000 [407:15:24<37:16:49,  9.34s/it]
 94%|█████████▎| 210626/225000 [407:15:29<32:52:17,  8.23s/it]
 94%|█████████▎| 210627/225000 [407:15:35<29:21:20,  7.35s/it]
 94%|█████████▎| 210628/225000 [407:15:45<33:13:07,  8.32s/it]
 94%|█████████▎| 210629/225000 [407:15:52<31:49:14,  7.97s/it]
 94%|█████████▎| 210630/225000 [407:16:03<35:21:06,  8.86s/it]
 94%|█████████▎| 210631/225000 [407:16:11<33:48:01,  8.47s/it]
 94%|█████████▎| 210632/225000 [407:16:16<29:23:47,  7.37s/it]
 94%|█████████▎| 210633/225000 [407:16:28<35:40:48,  8.94s/it]
 94%|█████████▎| 210634/225000 [407:16:36<34:53:50,  8.75s/it]
 94%|█████████▎| 210635/225000 [407:16:43<32:37:33,  8.18s/it]
 94%|█████████▎| 210636/225000 [407:16:50<30:31:47,  7.65s/it]
 94%|█████████▎| 210637/225000 [407:16:59<31:53:43,  7.99s/it]
 94%|█████████▎| 210638/225000 [407:17:05<29:57:29,  7.51s/it]
 94%|█████████▎| 210639/225000 [407:17:19<37:39:56,  9.44s/it]
 94%|█████████▎| 210640/225000 [407:17:24<32:37:54,  8.18s/it]
 94%|█████████▎| 210641/225000 [407:17:35<35:31:34,  8.91s/it]
 94%|█████████▎| 210642/225000 [407:17:45<36:48:54,  9.23s/it]
 94%|█████████▎| 210643/225000 [407:17:56<38:52:21,  9.75s/it]
 94%|█████████▎| 210644/225000 [407:18:05<38:03:43,  9.54s/it]
 94%|█████████▎| 210645/225000 [407:18:11<33:58:15,  8.52s/it]
 94%|█████████▎| 210646/225000 [407:18:19<33:53:13,  8.50s/it]
 94%|█████████▎| 210647/225000 [407:18:25<30:16:38,  7.59s/it]
 94%|█████████▎| 210648/225000 [407:18:37<36:03:47,  9.05s/it]
 94%|█████████▎| 210649/225000 [407:18:44<32:55:50,  8.26s/it]
 94%|█████████▎| 210650/225000 [407:18:50<30:18:34,  7.60s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.361', 'learning_rate': '5.551e-07', 'epoch': '0.9362', 'num_input_tokens_seen': 689150528, 'train_runtime': '1.466e+06', 'train_tokens_per_second': '470'}

 94%|█████████▎| 210650/225000 [407:18:50<30:18:34,  7.60s/it]
 94%|█████████▎| 210651/225000 [407:19:02<36:14:10,  9.09s/it]
 94%|█████████▎| 210652/225000 [407:19:10<34:41:02,  8.70s/it]
 94%|█████████▎| 210653/225000 [407:19:19<34:44:26,  8.72s/it]
 94%|█████████▎| 210654/225000 [407:19:27<33:58:41,  8.53s/it]
 94%|█████████▎| 210655/225000 [407:19:37<35:19:22,  8.86s/it]
 94%|█████████▎| 210656/225000 [407:19:48<38:04:12,  9.55s/it]
 94%|█████████▎| 210657/225000 [407:19:56<36:55:54,  9.27s/it]
 94%|█████████▎| 210658/225000 [407:20:03<33:51:40,  8.50s/it]
 94%|█████████▎| 210659/225000 [407:20:10<31:31:28,  7.91s/it]
 94%|█████████▎| 210660/225000 [407:20:17<31:05:44,  7.81s/it]
 94%|█████████▎| 210661/225000 [407:20:25<31:10:29,  7.83s/it]
 94%|█████████▎| 210662/225000 [407:20:36<35:02:38,  8.80s/it]
 94%|█████████▎| 210663/225000 [407:20:45<34:41:01,  8.71s/it]
 94%|█████████▎| 210664/225000 [407:20:51<31:22:29,  7.88s/it]
 94%|█████████▎| 210665/225000 [407:21:02<35:11:41,  8.84s/it]
 94%|█████████▎| 210666/225000 [407:21:13<38:26:31,  9.65s/it]
 94%|█████████▎| 210667/225000 [407:21:20<35:22:39,  8.89s/it]
 94%|█████████▎| 210668/225000 [407:21:28<33:53:56,  8.51s/it]
 94%|█████████▎| 210669/225000 [407:21:37<35:04:05,  8.81s/it]
 94%|█████████▎| 210670/225000 [407:21:47<36:35:11,  9.19s/it]
 94%|█████████▎| 210671/225000 [407:21:52<31:15:46,  7.85s/it]
 94%|█████████▎| 210672/225000 [407:22:04<36:14:36,  9.11s/it]
 94%|█████████▎| 210673/225000 [407:22:13<35:20:34,  8.88s/it]
 94%|█████████▎| 210674/225000 [407:22:19<32:07:53,  8.07s/it]
 94%|█████████▎| 210675/225000 [407:22:26<30:36:47,  7.69s/it]
 94%|█████████▎| 210676/225000 [407:22:38<35:52:08,  9.01s/it]
 94%|█████████▎| 210677/225000 [407:22:47<35:41:55,  8.97s/it]
 94%|█████████▎| 210678/225000 [407:22:59<39:39:23,  9.97s/it]
 94%|█████████▎| 210679/225000 [407:23:06<36:12:51,  9.10s/it]
 94%|█████████▎| 210680/225000 [407:23:13<33:24:38,  8.40s/it]
 94%|█████████▎| 210681/225000 [407:23:19<30:47:48,  7.74s/it]
 94%|█████████▎| 210682/225000 [407:23:25<28:29:55,  7.17s/it]
 94%|█████████▎| 210683/225000 [407:23:33<29:39:57,  7.46s/it]
 94%|█████████▎| 210684/225000 [407:23:46<36:11:26,  9.10s/it]
 94%|█████████▎| 210685/225000 [407:23:51<31:13:28,  7.85s/it]
 94%|█████████▎| 210686/225000 [407:24:02<34:58:18,  8.80s/it]
 94%|█████████▎| 210687/225000 [407:24:07<30:40:55,  7.72s/it]
 94%|█████████▎| 210688/225000 [407:24:18<35:10:47,  8.85s/it]
 94%|█████████▎| 210689/225000 [407:24:24<30:55:44,  7.78s/it]
 94%|█████████▎| 210690/225000 [407:24:33<32:42:11,  8.23s/it]
 94%|█████████▎| 210691/225000 [407:24:44<36:17:57,  9.13s/it]
 94%|█████████▎| 210692/225000 [407:24:52<35:03:40,  8.82s/it]
 94%|█████████▎| 210693/225000 [407:25:02<36:25:52,  9.17s/it]
 94%|█████████▎| 210694/225000 [407:25:09<33:23:38,  8.40s/it]
 94%|█████████▎| 210695/225000 [407:25:16<31:18:04,  7.88s/it]
 94%|█████████▎| 210696/225000 [407:25:27<35:30:52,  8.94s/it]
 94%|█████████▎| 210697/225000 [407:25:36<36:05:15,  9.08s/it]
 94%|█████████▎| 210698/225000 [407:25:42<32:13:54,  8.11s/it]
 94%|█████████▎| 210699/225000 [407:25:57<40:35:11, 10.22s/it]
 94%|█████████▎| 210700/225000 [407:26:08<40:47:11, 10.27s/it]
                                                              
{'loss': '1.283', 'grad_norm': '1.06', 'learning_rate': '5.513e-07', 'epoch': '0.9364', 'num_input_tokens_seen': 689384968, 'train_runtime': '1.467e+06', 'train_tokens_per_second': '470'}

 94%|█████████▎| 210700/225000 [407:26:08<40:47:11, 10.27s/it]
 94%|█████████▎| 210701/225000 [407:26:20<42:57:48, 10.82s/it]
 94%|█████████▎| 210702/225000 [407:26:28<39:50:10, 10.03s/it]
 94%|█████████▎| 210703/225000 [407:26:37<38:32:48,  9.71s/it]
 94%|█████████▎| 210704/225000 [407:26:45<36:51:40,  9.28s/it]
 94%|█████████▎| 210705/225000 [407:26:51<32:42:13,  8.24s/it]
 94%|█████████▎| 210706/225000 [407:27:04<38:39:34,  9.74s/it]
 94%|█████████▎| 210707/225000 [407:27:13<37:45:57,  9.51s/it]
 94%|█████████▎| 210708/225000 [407:27:20<33:53:44,  8.54s/it]
 94%|█████████▎| 210709/225000 [407:27:32<39:01:05,  9.83s/it]
 94%|█████████▎| 210710/225000 [407:27:37<32:52:12,  8.28s/it]
 94%|█████████▎| 210711/225000 [407:27:47<34:28:56,  8.69s/it]
 94%|█████████▎| 210712/225000 [407:27:51<29:40:43,  7.48s/it]
 94%|█████████▎| 210713/225000 [407:28:06<37:35:13,  9.47s/it]
 94%|█████████▎| 210714/225000 [407:28:13<34:57:52,  8.81s/it]
 94%|█████████▎| 210715/225000 [407:28:27<40:52:40, 10.30s/it]
 94%|█████████▎| 210716/225000 [407:28:32<35:17:37,  8.90s/it]
 94%|█████████▎| 210717/225000 [407:28:37<30:41:51,  7.74s/it]
 94%|█████████▎| 210718/225000 [407:28:48<34:38:42,  8.73s/it]
 94%|█████████▎| 210719/225000 [407:28:55<32:43:17,  8.25s/it]
 94%|█████████▎| 210720/225000 [407:29:02<30:51:50,  7.78s/it]
 94%|█████████▎| 210721/225000 [407:29:08<29:09:30,  7.35s/it]
 94%|█████████▎| 210722/225000 [407:29:16<29:28:51,  7.43s/it]
 94%|█████████▎| 210723/225000 [407:29:32<39:56:29, 10.07s/it]
 94%|█████████▎| 210724/225000 [407:29:37<33:05:02,  8.34s/it]
 94%|█████████▎| 210725/225000 [407:29:49<37:42:27,  9.51s/it]
 94%|█████████▎| 210726/225000 [407:29:55<34:03:53,  8.59s/it]
 94%|█████████▎| 210727/225000 [407:30:03<33:26:48,  8.44s/it]
 94%|█████████▎| 210728/225000 [407:30:11<32:00:33,  8.07s/it]
 94%|█████████▎| 210729/225000 [407:30:24<38:53:25,  9.81s/it]
 94%|█████████▎| 210730/225000 [407:30:35<39:22:38,  9.93s/it]
 94%|█████████▎| 210731/225000 [407:30:43<38:01:07,  9.59s/it]
 94%|█████████▎| 210732/225000 [407:30:52<37:19:01,  9.42s/it]
 94%|█████████▎| 210733/225000 [407:31:02<37:43:56,  9.52s/it]
 94%|█████████▎| 210734/225000 [407:31:15<41:05:20, 10.37s/it]
 94%|█████████▎| 210735/225000 [407:31:20<35:35:19,  8.98s/it]
 94%|█████████▎| 210736/225000 [407:31:33<39:53:49, 10.07s/it]
 94%|█████████▎| 210737/225000 [407:31:39<35:24:27,  8.94s/it]
 94%|█████████▎| 210738/225000 [407:31:46<32:43:18,  8.26s/it]
 94%|█████████▎| 210739/225000 [407:31:52<30:27:39,  7.69s/it]
 94%|█████████▎| 210740/225000 [407:31:59<28:49:08,  7.28s/it]
 94%|█████████▎| 210741/225000 [407:32:06<29:05:21,  7.34s/it]
 94%|█████████▎| 210742/225000 [407:32:19<36:01:32,  9.10s/it]
 94%|█████████▎| 210743/225000 [407:32:31<39:36:09, 10.00s/it]
 94%|█████████▎| 210744/225000 [407:32:39<36:30:56,  9.22s/it]
 94%|█████████▎| 210745/225000 [407:32:44<31:53:27,  8.05s/it]
 94%|█████████▎| 210746/225000 [407:32:55<34:59:51,  8.84s/it]
 94%|█████████▎| 210747/225000 [407:33:03<34:21:13,  8.68s/it]
 94%|█████████▎| 210748/225000 [407:33:13<35:46:38,  9.04s/it]
 94%|█████████▎| 210749/225000 [407:33:20<33:54:55,  8.57s/it]
 94%|█████████▎| 210750/225000 [407:33:26<30:01:36,  7.59s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.171', 'learning_rate': '5.475e-07', 'epoch': '0.9367', 'num_input_tokens_seen': 689620736, 'train_runtime': '1.467e+06', 'train_tokens_per_second': '470'}

 94%|█████████▎| 210750/225000 [407:33:26<30:01:36,  7.59s/it]
 94%|█████████▎| 210751/225000 [407:33:31<27:33:26,  6.96s/it]
 94%|█████████▎| 210752/225000 [407:33:45<36:00:29,  9.10s/it]
 94%|█████████▎| 210753/225000 [407:33:58<39:44:54, 10.04s/it]
 94%|█████████▎| 210754/225000 [407:34:04<35:15:35,  8.91s/it]
 94%|█████████▎| 210755/225000 [407:34:11<33:06:03,  8.37s/it]
 94%|█████████▎| 210756/225000 [407:34:18<31:37:13,  7.99s/it]
 94%|█████████▎| 210757/225000 [407:34:25<30:40:29,  7.75s/it]
 94%|█████████▎| 210758/225000 [407:34:39<38:06:11,  9.63s/it]
 94%|█████████▎| 210759/225000 [407:34:46<34:20:21,  8.68s/it]
 94%|█████████▎| 210760/225000 [407:34:56<36:24:16,  9.20s/it]
 94%|█████████▎| 210761/225000 [407:35:03<33:39:36,  8.51s/it]
 94%|█████████▎| 210762/225000 [407:35:12<34:48:12,  8.80s/it]
 94%|█████████▎| 210763/225000 [407:35:19<32:08:22,  8.13s/it]
 94%|█████████▎| 210764/225000 [407:35:32<37:51:06,  9.57s/it]
 94%|█████████▎| 210765/225000 [407:35:39<34:59:26,  8.85s/it]
 94%|█████████▎| 210766/225000 [407:35:44<30:48:56,  7.79s/it]
 94%|█████████▎| 210767/225000 [407:35:52<30:23:53,  7.69s/it]
 94%|█████████▎| 210768/225000 [407:36:00<31:02:13,  7.85s/it]
 94%|█████████▎| 210769/225000 [407:36:09<31:40:00,  8.01s/it]
 94%|█████████▎| 210770/225000 [407:36:15<29:29:09,  7.46s/it]
 94%|█████████▎| 210771/225000 [407:36:27<35:31:52,  8.99s/it]
 94%|█████████▎| 210772/225000 [407:36:35<34:33:15,  8.74s/it]
 94%|█████████▎| 210773/225000 [407:36:48<38:36:13,  9.77s/it]
 94%|█████████▎| 210774/225000 [407:36:55<35:41:41,  9.03s/it]
 94%|█████████▎| 210775/225000 [407:37:03<34:27:51,  8.72s/it]
 94%|█████████▎| 210776/225000 [407:37:08<29:33:50,  7.48s/it]
 94%|█████████▎| 210777/225000 [407:37:14<28:29:48,  7.21s/it]
 94%|█████████▎| 210778/225000 [407:37:26<34:27:27,  8.72s/it]
 94%|█████████▎| 210779/225000 [407:37:35<34:03:24,  8.62s/it]
 94%|█████████▎| 210780/225000 [407:37:46<36:50:07,  9.33s/it]
 94%|█████████▎| 210781/225000 [407:37:53<33:59:50,  8.61s/it]
 94%|█████████▎| 210782/225000 [407:38:03<35:50:19,  9.07s/it]
 94%|█████████▎| 210783/225000 [407:38:08<31:27:33,  7.97s/it]
 94%|█████████▎| 210784/225000 [407:38:17<32:30:21,  8.23s/it]
 94%|█████████▎| 210785/225000 [407:38:24<30:49:20,  7.81s/it]
 94%|█████████▎| 210786/225000 [407:38:37<36:54:37,  9.35s/it]
 94%|█████████▎| 210787/225000 [407:38:43<33:36:20,  8.51s/it]
 94%|█████████▎| 210788/225000 [407:38:54<36:00:57,  9.12s/it]
 94%|█████████▎| 210789/225000 [407:39:00<32:12:13,  8.16s/it]
 94%|█████████▎| 210790/225000 [407:39:07<30:52:07,  7.82s/it]
 94%|█████████▎| 210791/225000 [407:39:17<34:08:46,  8.65s/it]
 94%|█████████▎| 210792/225000 [407:39:24<31:21:29,  7.95s/it]
 94%|█████████▎| 210793/225000 [407:39:34<34:19:17,  8.70s/it]
 94%|█████████▎| 210794/225000 [407:39:41<32:16:20,  8.18s/it]
 94%|█████████▎| 210795/225000 [407:39:54<37:22:00,  9.47s/it]
 94%|█████████▎| 210796/225000 [407:40:02<36:02:08,  9.13s/it]
 94%|█████████▎| 210797/225000 [407:40:15<40:56:01, 10.38s/it]
 94%|█████████▎| 210798/225000 [407:40:21<35:05:33,  8.90s/it]
 94%|█████████▎| 210799/225000 [407:40:31<37:00:36,  9.38s/it]
 94%|█████████▎| 210800/225000 [407:40:38<33:53:58,  8.59s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.329', 'learning_rate': '5.437e-07', 'epoch': '0.9369', 'num_input_tokens_seen': 689851944, 'train_runtime': '1.468e+06', 'train_tokens_per_second': '470'}

 94%|█████████▎| 210800/225000 [407:40:38<33:53:58,  8.59s/it]
 94%|█████████▎| 210801/225000 [407:40:45<31:59:10,  8.11s/it]
 94%|█████████▎| 210802/225000 [407:40:57<36:36:33,  9.28s/it]
 94%|█████████▎| 210803/225000 [407:41:03<33:20:50,  8.46s/it]
 94%|█████████▎| 210804/225000 [407:41:12<33:55:17,  8.60s/it]
 94%|█████████▎| 210805/225000 [407:41:24<37:19:12,  9.46s/it]
 94%|█████████▎| 210806/225000 [407:41:31<34:52:21,  8.84s/it]
 94%|█████████▎| 210807/225000 [407:41:42<36:52:15,  9.35s/it]
 94%|█████████▎| 210808/225000 [407:41:50<35:01:58,  8.89s/it]
 94%|█████████▎| 210809/225000 [407:41:54<29:48:28,  7.56s/it]
 94%|█████████▎| 210810/225000 [407:42:02<30:03:06,  7.62s/it]
 94%|█████████▎| 210811/225000 [407:42:12<32:48:42,  8.32s/it]
 94%|█████████▎| 210812/225000 [407:42:19<31:06:27,  7.89s/it]
 94%|█████████▎| 210813/225000 [407:42:26<30:28:44,  7.73s/it]
 94%|█████████▎| 210814/225000 [407:42:40<37:45:26,  9.58s/it]
 94%|█████████▎| 210815/225000 [407:42:46<34:04:07,  8.65s/it]
 94%|█████████▎| 210816/225000 [407:42:56<35:28:42,  9.00s/it]
 94%|█████████▎| 210817/225000 [407:43:03<32:18:28,  8.20s/it]
 94%|█████████▎| 210818/225000 [407:43:13<34:20:40,  8.72s/it]
 94%|█████████▎| 210819/225000 [407:43:20<32:50:50,  8.34s/it]
 94%|█████████▎| 210820/225000 [407:43:28<32:14:47,  8.19s/it]
 94%|█████████▎| 210821/225000 [407:43:35<30:51:15,  7.83s/it]
 94%|█████████▎| 210822/225000 [407:43:46<34:24:15,  8.74s/it]
 94%|█████████▎| 210823/225000 [407:43:55<35:41:59,  9.07s/it]
 94%|█████████▎| 210824/225000 [407:44:00<30:19:55,  7.70s/it]
 94%|█████████▎| 210825/225000 [407:44:09<31:36:45,  8.03s/it]
 94%|█████████▎| 210826/225000 [407:44:15<29:50:09,  7.58s/it]
 94%|█████████▎| 210827/225000 [407:44:29<36:52:01,  9.36s/it]
 94%|█████████▎| 210828/225000 [407:44:36<33:47:21,  8.58s/it]
 94%|█████████▎| 210829/225000 [407:44:42<30:45:58,  7.82s/it]
 94%|█████████▎| 210830/225000 [407:44:57<40:01:16, 10.17s/it]
 94%|█████████▎| 210831/225000 [407:45:02<33:43:55,  8.57s/it]
 94%|█████████▎| 210832/225000 [407:45:08<30:31:53,  7.76s/it]
 94%|█████████▎| 210833/225000 [407:45:14<28:05:33,  7.14s/it]
 94%|█████████▎| 210834/225000 [407:45:25<32:28:33,  8.25s/it]
 94%|█████████▎| 210835/225000 [407:45:31<30:34:27,  7.77s/it]
 94%|█████████▎| 210836/225000 [407:45:41<32:59:40,  8.39s/it]
 94%|█████████▎| 210837/225000 [407:45:47<30:22:21,  7.72s/it]
 94%|█████████▎| 210838/225000 [407:45:55<30:28:53,  7.75s/it]
 94%|█████████▎| 210839/225000 [407:46:05<32:56:22,  8.37s/it]
 94%|█████████▎| 210840/225000 [407:46:16<36:21:08,  9.24s/it]
 94%|█████████▎| 210841/225000 [407:46:28<39:42:25, 10.10s/it]
 94%|█████████▎| 210842/225000 [407:46:34<34:20:17,  8.73s/it]
 94%|█████████▎| 210843/225000 [407:46:41<32:22:39,  8.23s/it]
 94%|█████████▎| 210844/225000 [407:46:54<37:50:04,  9.62s/it]
 94%|█████████▎| 210845/225000 [407:47:01<35:03:50,  8.92s/it]
 94%|█████████▎| 210846/225000 [407:47:09<34:19:10,  8.73s/it]
 94%|█████████▎| 210847/225000 [407:47:17<32:54:56,  8.37s/it]
 94%|█████████▎| 210848/225000 [407:47:29<37:51:57,  9.63s/it]
 94%|█████████▎| 210849/225000 [407:47:38<36:49:51,  9.37s/it]
 94%|█████████▎| 210850/225000 [407:47:46<34:54:59,  8.88s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.069', 'learning_rate': '5.399e-07', 'epoch': '0.9371', 'num_input_tokens_seen': 690084440, 'train_runtime': '1.468e+06', 'train_tokens_per_second': '470.1'}

 94%|█████████▎| 210850/225000 [407:47:46<34:54:59,  8.88s/it]
 94%|█████████▎| 210851/225000 [407:47:52<31:59:20,  8.14s/it]
 94%|█████████▎| 210852/225000 [407:48:07<39:47:01, 10.12s/it]
 94%|█████████▎| 210853/225000 [407:48:13<34:20:15,  8.74s/it]
 94%|█████████▎| 210854/225000 [407:48:19<32:03:01,  8.16s/it]
 94%|█████████▎| 210855/225000 [407:48:31<36:28:38,  9.28s/it]
 94%|█████████▎| 210856/225000 [407:48:40<36:25:39,  9.27s/it]
 94%|█████████▎| 210857/225000 [407:48:49<35:22:03,  9.00s/it]
 94%|█████████▎| 210858/225000 [407:48:59<37:15:05,  9.48s/it]
 94%|█████████▎| 210859/225000 [407:49:05<32:57:42,  8.39s/it]
 94%|█████████▎| 210860/225000 [407:49:13<32:35:13,  8.30s/it]
 94%|█████████▎| 210861/225000 [407:49:25<36:06:08,  9.19s/it]
 94%|█████████▎| 210862/225000 [407:49:30<31:04:50,  7.91s/it]
 94%|█████████▎| 210863/225000 [407:49:39<32:40:09,  8.32s/it]
 94%|█████████▎| 210864/225000 [407:49:44<29:18:28,  7.46s/it]
 94%|█████████▎| 210865/225000 [407:49:50<27:18:13,  6.95s/it]
 94%|█████████▎| 210866/225000 [407:49:55<24:27:30,  6.23s/it]
 94%|█████████▎| 210867/225000 [407:50:06<30:29:27,  7.77s/it]
 94%|█████████▎| 210868/225000 [407:50:17<34:44:48,  8.85s/it]
 94%|█████████▎| 210869/225000 [407:50:28<37:20:54,  9.51s/it]
 94%|█████████▎| 210870/225000 [407:50:36<35:33:54,  9.06s/it]
 94%|█████████▎| 210871/225000 [407:50:44<33:27:12,  8.52s/it]
 94%|█████████▎| 210872/225000 [407:50:55<36:09:02,  9.21s/it]
 94%|█████████▎| 210873/225000 [407:50:59<31:09:43,  7.94s/it]
 94%|█████████▎| 210874/225000 [407:51:06<29:10:27,  7.44s/it]
 94%|█████████▎| 210875/225000 [407:51:11<26:42:43,  6.81s/it]
 94%|█████████▎| 210876/225000 [407:51:24<34:07:39,  8.70s/it]
 94%|█████████▎| 210877/225000 [407:51:30<30:51:05,  7.86s/it]
 94%|█████████▎| 210878/225000 [407:51:37<30:08:06,  7.68s/it]
 94%|█████████▎| 210879/225000 [407:51:44<29:20:24,  7.48s/it]
 94%|█████████▎| 210880/225000 [407:51:55<32:31:59,  8.29s/it]
 94%|█████████▎| 210881/225000 [407:52:02<30:58:40,  7.90s/it]
 94%|█████████▎| 210882/225000 [407:52:09<30:48:47,  7.86s/it]
 94%|█████████▎| 210883/225000 [407:52:24<38:57:10,  9.93s/it]
 94%|█████████▎| 210884/225000 [407:52:31<35:39:34,  9.09s/it]
 94%|█████████▎| 210885/225000 [407:52:44<40:25:34, 10.31s/it]
 94%|█████████▎| 210886/225000 [407:52:55<41:01:59, 10.47s/it]
 94%|█████████▎| 210887/225000 [407:53:00<34:51:08,  8.89s/it]
 94%|█████████▎| 210888/225000 [407:53:07<31:52:20,  8.13s/it]
 94%|█████████▎| 210889/225000 [407:53:16<33:24:02,  8.52s/it]
 94%|█████████▎| 210890/225000 [407:53:23<31:36:44,  8.07s/it]
 94%|█████████▎| 210891/225000 [407:53:30<30:20:33,  7.74s/it]
 94%|█████████▎| 210892/225000 [407:53:42<35:09:59,  8.97s/it]
 94%|█████████▎| 210893/225000 [407:53:46<29:40:08,  7.57s/it]
 94%|█████████▎| 210894/225000 [407:53:57<33:32:51,  8.56s/it]
 94%|█████████▎| 210895/225000 [407:54:05<32:08:28,  8.20s/it]
 94%|█████████▎| 210896/225000 [407:54:12<31:01:56,  7.92s/it]
 94%|█████████▎| 210897/225000 [407:54:17<28:12:27,  7.20s/it]
 94%|█████████▎| 210898/225000 [407:54:25<29:00:14,  7.40s/it]
 94%|█████████▎| 210899/225000 [407:54:31<27:05:19,  6.92s/it]
 94%|█████████▎| 210900/225000 [407:54:49<40:08:04, 10.25s/it]
                                                              
{'loss': '1.305', 'grad_norm': '2.41', 'learning_rate': '5.362e-07', 'epoch': '0.9373', 'num_input_tokens_seen': 690314088, 'train_runtime': '1.468e+06', 'train_tokens_per_second': '470.1'}

 94%|█████████▎| 210900/225000 [407:54:49<40:08:04, 10.25s/it]
 94%|█████████▎| 210901/225000 [407:54:56<35:57:05,  9.18s/it]
 94%|█████████▎| 210902/225000 [407:55:04<34:41:14,  8.86s/it]
 94%|█████████▎| 210903/225000 [407:55:15<37:20:56,  9.54s/it]
 94%|█████████▎| 210904/225000 [407:55:22<34:56:29,  8.92s/it]
 94%|█████████▎| 210905/225000 [407:55:32<35:54:50,  9.17s/it]
 94%|█████████▎| 210906/225000 [407:55:38<31:43:58,  8.11s/it]
 94%|█████████▎| 210907/225000 [407:55:48<33:58:53,  8.68s/it]
 94%|█████████▎| 210908/225000 [407:55:55<32:20:56,  8.26s/it]
 94%|█████████▎| 210909/225000 [407:56:07<36:10:30,  9.24s/it]
 94%|█████████▎| 210910/225000 [407:56:13<32:40:25,  8.35s/it]
 94%|█████████▎| 210911/225000 [407:56:18<28:55:26,  7.39s/it]
 94%|█████████▎| 210912/225000 [407:56:25<28:16:59,  7.23s/it]
 94%|█████████▎| 210913/225000 [407:56:34<30:16:20,  7.74s/it]
 94%|█████████▎| 210914/225000 [407:56:47<36:07:20,  9.23s/it]
 94%|█████████▎| 210915/225000 [407:56:55<34:51:27,  8.91s/it]
 94%|█████████▎| 210916/225000 [407:57:00<31:03:08,  7.94s/it]
 94%|█████████▎| 210917/225000 [407:57:07<29:26:34,  7.53s/it]
 94%|█████████▎| 210918/225000 [407:57:14<28:55:57,  7.40s/it]
 94%|█████████▎| 210919/225000 [407:57:27<35:32:20,  9.09s/it]
 94%|█████████▎| 210920/225000 [407:57:33<32:17:05,  8.25s/it]
 94%|█████████▎| 210921/225000 [407:57:41<31:05:35,  7.95s/it]
 94%|█████████▎| 210922/225000 [407:57:49<31:46:07,  8.12s/it]
 94%|█████████▎| 210923/225000 [407:58:04<39:50:54, 10.19s/it]
 94%|█████████▎| 210924/225000 [407:58:09<33:21:16,  8.53s/it]
 94%|█████████▎| 210925/225000 [407:58:17<33:26:54,  8.56s/it]
 94%|█████████▎| 210926/225000 [407:58:25<32:02:23,  8.20s/it]
 94%|█████████▎| 210927/225000 [407:58:31<30:01:25,  7.68s/it]
 94%|█████████▎| 210928/225000 [407:58:44<35:22:28,  9.05s/it]
 94%|█████████▎| 210929/225000 [407:58:55<37:52:19,  9.69s/it]
 94%|█████████▎| 210930/225000 [407:59:01<33:24:51,  8.55s/it]
 94%|█████████▎| 210931/225000 [407:59:16<41:41:33, 10.67s/it]
 94%|█████████▎| 210932/225000 [407:59:22<36:05:16,  9.23s/it]
 94%|█████████▎| 210933/225000 [407:59:32<36:47:06,  9.41s/it]
 94%|█████████▎| 210934/225000 [407:59:45<40:51:30, 10.46s/it]
 94%|█████████▎| 210935/225000 [407:59:56<41:07:28, 10.53s/it]
 94%|█████████▎| 210936/225000 [408:00:02<36:23:02,  9.31s/it]
 94%|█████████▎| 210937/225000 [408:00:12<36:46:49,  9.42s/it]
 94%|█████████▍| 210938/225000 [408:00:18<33:39:46,  8.62s/it]
 94%|█████████▍| 210939/225000 [408:00:25<31:02:48,  7.95s/it]
 94%|█████████▍| 210940/225000 [408:00:38<37:02:04,  9.48s/it]
 94%|█████████▍| 210941/225000 [408:00:48<38:06:53,  9.76s/it]
 94%|█████████▍| 210942/225000 [408:00:56<35:44:35,  9.15s/it]
 94%|█████████▍| 210943/225000 [408:01:02<31:39:36,  8.11s/it]
 94%|█████████▍| 210944/225000 [408:01:08<29:45:32,  7.62s/it]
 94%|█████████▍| 210945/225000 [408:01:20<35:10:10,  9.01s/it]
 94%|█████████▍| 210946/225000 [408:01:26<31:41:35,  8.12s/it]
 94%|█████████▍| 210947/225000 [408:01:34<30:52:39,  7.91s/it]
 94%|█████████▍| 210948/225000 [408:01:40<29:18:40,  7.51s/it]
 94%|█████████▍| 210949/225000 [408:01:47<28:32:12,  7.31s/it]
 94%|█████████▍| 210950/225000 [408:01:57<31:22:19,  8.04s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.351', 'learning_rate': '5.324e-07', 'epoch': '0.9376', 'num_input_tokens_seen': 690546544, 'train_runtime': '1.469e+06', 'train_tokens_per_second': '470.1'}

 94%|█████████▍| 210950/225000 [408:01:57<31:22:19,  8.04s/it]
 94%|█████████▍| 210951/225000 [408:02:05<31:32:37,  8.08s/it]
 94%|█████████▍| 210952/225000 [408:02:12<30:00:53,  7.69s/it]
 94%|█████████▍| 210953/225000 [408:02:22<32:46:26,  8.40s/it]
 94%|█████████▍| 210954/225000 [408:02:29<31:11:17,  7.99s/it]
 94%|█████████▍| 210955/225000 [408:02:39<32:57:51,  8.45s/it]
 94%|█████████▍| 210956/225000 [408:02:48<33:31:54,  8.60s/it]
 94%|█████████▍| 210957/225000 [408:02:55<31:38:16,  8.11s/it]
 94%|█████████▍| 210958/225000 [408:03:02<30:31:52,  7.83s/it]
 94%|█████████▍| 210959/225000 [408:03:11<32:02:40,  8.22s/it]
 94%|█████████▍| 210960/225000 [408:03:18<31:03:57,  7.97s/it]
 94%|█████████▍| 210961/225000 [408:03:26<30:41:49,  7.87s/it]
 94%|█████████▍| 210962/225000 [408:03:32<28:20:30,  7.27s/it]
 94%|█████████▍| 210963/225000 [408:03:46<36:20:30,  9.32s/it]
 94%|█████████▍| 210964/225000 [408:03:55<36:22:44,  9.33s/it]
 94%|█████████▍| 210965/225000 [408:04:01<32:50:19,  8.42s/it]
 94%|█████████▍| 210966/225000 [408:04:10<33:27:22,  8.58s/it]
 94%|█████████▍| 210967/225000 [408:04:20<34:10:44,  8.77s/it]
 94%|█████████▍| 210968/225000 [408:04:28<33:57:55,  8.71s/it]
 94%|█████████▍| 210969/225000 [408:04:37<34:05:03,  8.75s/it]
 94%|█████████▍| 210970/225000 [408:04:47<35:11:53,  9.03s/it]
 94%|█████████▍| 210971/225000 [408:04:57<36:09:26,  9.28s/it]
 94%|█████████▍| 210972/225000 [408:05:02<31:45:57,  8.15s/it]
 94%|█████████▍| 210973/225000 [408:05:10<31:21:41,  8.05s/it]
 94%|█████████▍| 210974/225000 [408:05:17<30:18:42,  7.78s/it]
 94%|█████████▍| 210975/225000 [408:05:25<30:55:17,  7.94s/it]
 94%|█████████▍| 210976/225000 [408:05:31<27:50:03,  7.15s/it]
 94%|█████████▍| 210977/225000 [408:05:41<31:06:27,  7.99s/it]
 94%|█████████▍| 210978/225000 [408:05:54<36:57:35,  9.49s/it]
 94%|█████████▍| 210979/225000 [408:05:59<32:02:50,  8.23s/it]
 94%|█████████▍| 210980/225000 [408:06:08<32:31:38,  8.35s/it]
 94%|█████████▍| 210981/225000 [408:06:17<33:48:12,  8.68s/it]
 94%|█████████▍| 210982/225000 [408:06:26<34:39:03,  8.90s/it]
 94%|█████████▍| 210983/225000 [408:06:35<34:27:17,  8.85s/it]
 94%|█████████▍| 210984/225000 [408:06:42<32:25:41,  8.33s/it]
 94%|█████████▍| 210985/225000 [408:06:50<31:15:24,  8.03s/it]
 94%|█████████▍| 210986/225000 [408:07:04<38:16:08,  9.83s/it]
 94%|█████████▍| 210987/225000 [408:07:13<37:53:40,  9.74s/it]
 94%|█████████▍| 210988/225000 [408:07:20<34:19:02,  8.82s/it]
 94%|█████████▍| 210989/225000 [408:07:27<32:32:09,  8.36s/it]
 94%|█████████▍| 210990/225000 [408:07:38<34:59:30,  8.99s/it]
 94%|█████████▍| 210991/225000 [408:07:46<34:42:28,  8.92s/it]
 94%|█████████▍| 210992/225000 [408:07:56<35:55:29,  9.23s/it]
 94%|█████████▍| 210993/225000 [408:08:01<31:09:19,  8.01s/it]
 94%|█████████▍| 210994/225000 [408:08:12<33:52:55,  8.71s/it]
 94%|█████████▍| 210995/225000 [408:08:18<30:51:03,  7.93s/it]
 94%|█████████▍| 210996/225000 [408:08:24<28:10:38,  7.24s/it]
 94%|█████████▍| 210997/225000 [408:08:31<28:03:56,  7.22s/it]
 94%|█████████▍| 210998/225000 [408:08:40<30:07:30,  7.75s/it]
 94%|█████████▍| 210999/225000 [408:08:45<26:57:09,  6.93s/it]
 94%|█████████▍| 211000/225000 [408:08:53<28:14:44,  7.26s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.755', 'learning_rate': '5.287e-07', 'epoch': '0.9378', 'num_input_tokens_seen': 690774184, 'train_runtime': '1.469e+06', 'train_tokens_per_second': '470.1'}

 94%|█████████▍| 211000/225000 [408:08:53<28:14:44,  7.26s/it]
 94%|█████████▍| 211001/225000 [408:09:06<35:10:07,  9.04s/it]
 94%|█████████▍| 211002/225000 [408:09:18<39:14:32, 10.09s/it]
 94%|█████████▍| 211003/225000 [408:09:25<35:23:19,  9.10s/it]
 94%|█████████▍| 211004/225000 [408:09:34<34:39:10,  8.91s/it]
 94%|█████████▍| 211005/225000 [408:09:43<35:00:28,  9.01s/it]
 94%|█████████▍| 211006/225000 [408:09:53<36:00:39,  9.26s/it]
 94%|█████████▍| 211007/225000 [408:10:03<36:44:11,  9.45s/it]
 94%|█████████▍| 211008/225000 [408:10:09<33:07:02,  8.52s/it]
 94%|█████████▍| 211009/225000 [408:10:16<31:51:24,  8.20s/it]
 94%|█████████▍| 211010/225000 [408:10:30<37:47:53,  9.73s/it]
 94%|█████████▍| 211011/225000 [408:10:36<33:33:00,  8.63s/it]
 94%|█████████▍| 211012/225000 [408:10:41<29:04:29,  7.48s/it]
 94%|█████████▍| 211013/225000 [408:10:52<33:51:36,  8.71s/it]
 94%|█████████▍| 211014/225000 [408:11:01<34:06:17,  8.78s/it]
 94%|█████████▍| 211015/225000 [408:11:06<28:55:56,  7.45s/it]
 94%|█████████▍| 211016/225000 [408:11:14<30:13:21,  7.78s/it]
 94%|█████████▍| 211017/225000 [408:11:22<30:45:49,  7.92s/it]
 94%|█████████▍| 211018/225000 [408:11:31<31:46:49,  8.18s/it]
 94%|█████████▍| 211019/225000 [408:11:41<34:03:40,  8.77s/it]
 94%|█████████▍| 211020/225000 [408:11:51<35:37:02,  9.17s/it]
 94%|█████████▍| 211021/225000 [408:11:57<31:43:33,  8.17s/it]
 94%|█████████▍| 211022/225000 [408:12:03<29:04:30,  7.49s/it]
 94%|█████████▍| 211023/225000 [408:12:09<27:05:54,  6.98s/it]
 94%|█████████▍| 211024/225000 [408:12:16<27:12:35,  7.01s/it]
 94%|█████████▍| 211025/225000 [408:12:34<39:43:31, 10.23s/it]
 94%|█████████▍| 211026/225000 [408:12:43<38:20:21,  9.88s/it]
 94%|█████████▍| 211027/225000 [408:12:50<35:30:18,  9.15s/it]
 94%|█████████▍| 211028/225000 [408:13:00<36:44:42,  9.47s/it]
 94%|█████████▍| 211029/225000 [408:13:05<31:03:01,  8.00s/it]
 94%|█████████▍| 211030/225000 [408:13:17<35:18:05,  9.10s/it]
 94%|█████████▍| 211031/225000 [408:13:26<35:50:17,  9.24s/it]
 94%|█████████▍| 211032/225000 [408:13:38<38:28:43,  9.92s/it]
 94%|█████████▍| 211033/225000 [408:13:46<36:38:08,  9.44s/it]
 94%|█████████▍| 211034/225000 [408:13:52<32:11:58,  8.30s/it]
 94%|█████████▍| 211035/225000 [408:13:59<30:54:57,  7.97s/it]
 94%|█████████▍| 211036/225000 [408:14:12<37:25:00,  9.65s/it]
 94%|█████████▍| 211037/225000 [408:14:21<35:33:01,  9.17s/it]
 94%|█████████▍| 211038/225000 [408:14:30<36:00:49,  9.29s/it]
 94%|█████████▍| 211039/225000 [408:14:41<38:12:03,  9.85s/it]
 94%|█████████▍| 211040/225000 [408:14:49<35:28:56,  9.15s/it]
 94%|█████████▍| 211041/225000 [408:14:55<32:09:24,  8.29s/it]
 94%|█████████▍| 211042/225000 [408:15:04<32:31:05,  8.39s/it]
 94%|█████████▍| 211043/225000 [408:15:13<33:55:01,  8.75s/it]
 94%|█████████▍| 211044/225000 [408:15:21<33:17:32,  8.59s/it]
 94%|█████████▍| 211045/225000 [408:15:28<30:35:20,  7.89s/it]
 94%|█████████▍| 211046/225000 [408:15:36<31:02:32,  8.01s/it]
 94%|█████████▍| 211047/225000 [408:15:42<28:48:11,  7.43s/it]
 94%|█████████▍| 211048/225000 [408:15:47<26:22:31,  6.81s/it]
 94%|█████████▍| 211049/225000 [408:16:01<34:44:13,  8.96s/it]
 94%|█████████▍| 211050/225000 [408:16:07<30:53:52,  7.97s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.578', 'learning_rate': '5.249e-07', 'epoch': '0.938', 'num_input_tokens_seen': 691007472, 'train_runtime': '1.47e+06', 'train_tokens_per_second': '470.1'}

 94%|█████████▍| 211050/225000 [408:16:07<30:53:52,  7.97s/it]
 94%|█████████▍| 211051/225000 [408:16:16<32:21:13,  8.35s/it]
 94%|█████████▍| 211052/225000 [408:16:25<32:47:01,  8.46s/it]
 94%|█████████▍| 211053/225000 [408:16:29<27:58:58,  7.22s/it]
 94%|█████████▍| 211054/225000 [408:16:40<31:51:19,  8.22s/it]
 94%|█████████▍| 211055/225000 [408:16:49<32:44:39,  8.45s/it]
 94%|█████████▍| 211056/225000 [408:16:56<31:38:08,  8.17s/it]
 94%|█████████▍| 211057/225000 [408:17:03<29:18:54,  7.57s/it]
 94%|█████████▍| 211058/225000 [408:17:12<31:51:33,  8.23s/it]
 94%|█████████▍| 211059/225000 [408:17:17<27:31:13,  7.11s/it]
 94%|█████████▍| 211060/225000 [408:17:29<33:56:18,  8.76s/it]
 94%|█████████▍| 211061/225000 [408:17:35<29:41:10,  7.67s/it]
 94%|█████████▍| 211062/225000 [408:17:39<25:51:49,  6.68s/it]
 94%|█████████▍| 211063/225000 [408:17:46<26:41:11,  6.89s/it]
 94%|█████████▍| 211064/225000 [408:17:57<30:40:41,  7.92s/it]
 94%|█████████▍| 211065/225000 [408:18:05<30:42:53,  7.93s/it]
 94%|█████████▍| 211066/225000 [408:18:12<30:11:02,  7.80s/it]
 94%|█████████▍| 211067/225000 [408:18:23<34:05:16,  8.81s/it]
 94%|█████████▍| 211068/225000 [408:18:33<35:08:59,  9.08s/it]
 94%|█████████▍| 211069/225000 [408:18:40<32:59:46,  8.53s/it]
 94%|█████████▍| 211070/225000 [408:18:47<30:59:56,  8.01s/it]
 94%|█████████▍| 211071/225000 [408:18:57<33:19:56,  8.61s/it]
 94%|█████████▍| 211072/225000 [408:19:06<34:01:22,  8.79s/it]
 94%|█████████▍| 211073/225000 [408:19:19<38:04:34,  9.84s/it]
 94%|█████████▍| 211074/225000 [408:19:25<34:27:08,  8.91s/it]
 94%|█████████▍| 211075/225000 [408:19:39<39:25:21, 10.19s/it]
 94%|█████████▍| 211076/225000 [408:19:47<37:57:12,  9.81s/it]
 94%|█████████▍| 211077/225000 [408:19:59<39:51:32, 10.31s/it]
 94%|█████████▍| 211078/225000 [408:20:10<41:09:10, 10.64s/it]
 94%|█████████▍| 211079/225000 [408:20:17<36:36:57,  9.47s/it]
 94%|█████████▍| 211080/225000 [408:20:24<33:19:32,  8.62s/it]
 94%|█████████▍| 211081/225000 [408:20:35<36:17:18,  9.39s/it]
 94%|█████████▍| 211082/225000 [408:20:42<34:15:58,  8.86s/it]
 94%|█████████▍| 211083/225000 [408:20:50<32:43:02,  8.46s/it]
 94%|█████████▍| 211084/225000 [408:20:55<28:33:18,  7.39s/it]
 94%|█████████▍| 211085/225000 [408:21:08<34:42:55,  8.98s/it]
 94%|█████████▍| 211086/225000 [408:21:13<30:28:40,  7.89s/it]
 94%|█████████▍| 211087/225000 [408:21:24<33:55:03,  8.78s/it]
 94%|█████████▍| 211088/225000 [408:21:33<33:54:33,  8.77s/it]
 94%|█████████▍| 211089/225000 [408:21:40<31:57:34,  8.27s/it]
 94%|█████████▍| 211090/225000 [408:21:47<31:13:34,  8.08s/it]
 94%|█████████▍| 211091/225000 [408:21:54<29:03:30,  7.52s/it]
 94%|█████████▍| 211092/225000 [408:22:00<27:30:20,  7.12s/it]
 94%|█████████▍| 211093/225000 [408:22:06<26:50:56,  6.95s/it]
 94%|█████████▍| 211094/225000 [408:22:20<35:08:37,  9.10s/it]
 94%|█████████▍| 211095/225000 [408:22:27<32:31:47,  8.42s/it]
 94%|█████████▍| 211096/225000 [408:22:35<31:42:15,  8.21s/it]
 94%|█████████▍| 211097/225000 [408:22:42<30:39:10,  7.94s/it]
 94%|█████████▍| 211098/225000 [408:22:49<29:39:46,  7.68s/it]
 94%|█████████▍| 211099/225000 [408:22:57<29:47:22,  7.71s/it]
 94%|█████████▍| 211100/225000 [408:23:11<36:56:22,  9.57s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.491', 'learning_rate': '5.212e-07', 'epoch': '0.9382', 'num_input_tokens_seen': 691236992, 'train_runtime': '1.47e+06', 'train_tokens_per_second': '470.2'}

 94%|█████████▍| 211100/225000 [408:23:11<36:56:22,  9.57s/it]
 94%|█████████▍| 211101/225000 [408:23:17<32:40:59,  8.47s/it]
 94%|█████████▍| 211102/225000 [408:23:27<34:06:07,  8.83s/it]
 94%|█████████▍| 211103/225000 [408:23:33<31:21:15,  8.12s/it]
 94%|█████████▍| 211104/225000 [408:23:45<35:45:23,  9.26s/it]
 94%|█████████▍| 211105/225000 [408:23:50<30:54:34,  8.01s/it]
 94%|█████████▍| 211106/225000 [408:23:55<26:55:46,  6.98s/it]
 94%|█████████▍| 211107/225000 [408:24:09<35:50:32,  9.29s/it]
 94%|█████████▍| 211108/225000 [408:24:14<30:11:27,  7.82s/it]
 94%|█████████▍| 211109/225000 [408:24:19<27:02:12,  7.01s/it]
 94%|█████████▍| 211110/225000 [408:24:30<31:47:29,  8.24s/it]
 94%|█████████▍| 211111/225000 [408:24:36<29:03:30,  7.53s/it]
 94%|█████████▍| 211112/225000 [408:24:45<31:22:11,  8.13s/it]
 94%|█████████▍| 211113/225000 [408:24:53<30:52:56,  8.01s/it]
 94%|█████████▍| 211114/225000 [408:25:05<35:42:40,  9.26s/it]
 94%|█████████▍| 211115/225000 [408:25:18<40:20:03, 10.46s/it]
 94%|█████████▍| 211116/225000 [408:25:26<37:24:12,  9.70s/it]
 94%|█████████▍| 211117/225000 [408:25:35<36:39:42,  9.51s/it]
 94%|█████████▍| 211118/225000 [408:25:43<33:50:39,  8.78s/it]
 94%|█████████▍| 211119/225000 [408:25:50<32:52:08,  8.52s/it]
 94%|█████████▍| 211120/225000 [408:26:01<34:58:49,  9.07s/it]
 94%|█████████▍| 211121/225000 [408:26:09<34:04:52,  8.84s/it]
 94%|█████████▍| 211122/225000 [408:26:24<41:19:54, 10.72s/it]
 94%|█████████▍| 211123/225000 [408:26:30<35:39:30,  9.25s/it]
 94%|█████████▍| 211124/225000 [408:26:39<35:11:29,  9.13s/it]
 94%|█████████▍| 211125/225000 [408:26:47<33:53:42,  8.79s/it]
 94%|█████████▍| 211126/225000 [408:26:57<35:50:47,  9.30s/it]
 94%|█████████▍| 211127/225000 [408:27:05<34:10:36,  8.87s/it]
 94%|█████████▍| 211128/225000 [408:27:15<34:38:58,  8.99s/it]
 94%|█████████▍| 211129/225000 [408:27:21<31:39:26,  8.22s/it]
 94%|█████████▍| 211130/225000 [408:27:27<29:31:23,  7.66s/it]
 94%|█████████▍| 211131/225000 [408:27:35<29:09:37,  7.57s/it]
 94%|█████████▍| 211132/225000 [408:27:46<33:29:35,  8.69s/it]
 94%|█████████▍| 211133/225000 [408:28:00<39:05:13, 10.15s/it]
 94%|█████████▍| 211134/225000 [408:28:06<35:22:39,  9.19s/it]
 94%|█████████▍| 211135/225000 [408:28:13<32:02:47,  8.32s/it]
 94%|█████████▍| 211136/225000 [408:28:18<28:07:31,  7.30s/it]
 94%|█████████▍| 211137/225000 [408:28:24<27:01:47,  7.02s/it]
 94%|█████████▍| 211138/225000 [408:28:39<36:42:02,  9.53s/it]
 94%|█████████▍| 211139/225000 [408:28:47<34:39:16,  9.00s/it]
 94%|█████████▍| 211140/225000 [408:28:54<31:54:40,  8.29s/it]
 94%|█████████▍| 211141/225000 [408:29:06<36:27:46,  9.47s/it]
 94%|█████████▍| 211142/225000 [408:29:17<37:50:05,  9.83s/it]
 94%|█████████▍| 211143/225000 [408:29:22<32:51:31,  8.54s/it]
 94%|█████████▍| 211144/225000 [408:29:35<37:48:24,  9.82s/it]
 94%|█████████▍| 211145/225000 [408:29:44<37:01:58,  9.62s/it]
 94%|█████████▍| 211146/225000 [408:29:53<36:19:46,  9.44s/it]
 94%|█████████▍| 211147/225000 [408:30:03<36:38:34,  9.52s/it]
 94%|█████████▍| 211148/225000 [408:30:10<33:39:56,  8.75s/it]
 94%|█████████▍| 211149/225000 [408:30:19<33:52:45,  8.81s/it]
 94%|█████████▍| 211150/225000 [408:30:30<36:05:55,  9.38s/it]
                                                              
{'loss': '1.293', 'grad_norm': '1.102', 'learning_rate': '5.175e-07', 'epoch': '0.9384', 'num_input_tokens_seen': 691470912, 'train_runtime': '1.471e+06', 'train_tokens_per_second': '470.2'}

 94%|█████████▍| 211150/225000 [408:30:30<36:05:55,  9.38s/it]
 94%|█████████▍| 211151/225000 [408:30:36<32:25:10,  8.43s/it]
 94%|█████████▍| 211152/225000 [408:30:43<30:54:56,  8.04s/it]
 94%|█████████▍| 211153/225000 [408:30:50<30:19:36,  7.88s/it]
 94%|█████████▍| 211154/225000 [408:30:56<27:59:09,  7.28s/it]
 94%|█████████▍| 211155/225000 [408:31:10<34:52:24,  9.07s/it]
 94%|█████████▍| 211156/225000 [408:31:16<32:07:28,  8.35s/it]
 94%|█████████▍| 211157/225000 [408:31:23<30:10:40,  7.85s/it]
 94%|█████████▍| 211158/225000 [408:31:39<39:17:51, 10.22s/it]
 94%|█████████▍| 211159/225000 [408:31:44<34:13:52,  8.90s/it]
 94%|█████████▍| 211160/225000 [408:31:50<29:58:59,  7.80s/it]
 94%|█████████▍| 211161/225000 [408:32:01<33:39:58,  8.76s/it]
 94%|█████████▍| 211162/225000 [408:32:10<34:34:50,  9.00s/it]
 94%|█████████▍| 211163/225000 [408:32:16<30:39:39,  7.98s/it]
 94%|█████████▍| 211164/225000 [408:32:22<28:13:50,  7.35s/it]
 94%|█████████▍| 211165/225000 [408:32:37<37:46:24,  9.83s/it]
 94%|█████████▍| 211166/225000 [408:32:46<36:55:40,  9.61s/it]
 94%|█████████▍| 211167/225000 [408:32:57<38:06:58,  9.92s/it]
 94%|█████████▍| 211168/225000 [408:33:05<35:18:02,  9.19s/it]
 94%|█████████▍| 211169/225000 [408:33:13<34:18:05,  8.93s/it]
 94%|█████████▍| 211170/225000 [408:33:18<29:48:42,  7.76s/it]
 94%|█████████▍| 211171/225000 [408:33:29<33:38:27,  8.76s/it]
 94%|█████████▍| 211172/225000 [408:33:38<34:19:28,  8.94s/it]
 94%|█████████▍| 211173/225000 [408:33:45<31:36:46,  8.23s/it]
 94%|█████████▍| 211174/225000 [408:33:52<30:11:25,  7.86s/it]
 94%|█████████▍| 211175/225000 [408:33:59<29:19:21,  7.64s/it]
 94%|█████████▍| 211176/225000 [408:34:13<37:00:34,  9.64s/it]
 94%|█████████▍| 211177/225000 [408:34:23<36:59:36,  9.63s/it]
 94%|█████████▍| 211178/225000 [408:34:34<38:29:58, 10.03s/it]
 94%|█████████▍| 211179/225000 [408:34:42<35:42:12,  9.30s/it]
 94%|█████████▍| 211180/225000 [408:34:49<33:34:55,  8.75s/it]
 94%|█████████▍| 211181/225000 [408:34:55<30:59:29,  8.07s/it]
 94%|█████████▍| 211182/225000 [408:35:02<29:14:35,  7.62s/it]
 94%|█████████▍| 211183/225000 [408:35:15<35:48:00,  9.33s/it]
 94%|█████████▍| 211184/225000 [408:35:25<36:07:42,  9.41s/it]
 94%|█████████▍| 211185/225000 [408:35:32<33:28:15,  8.72s/it]
 94%|█████████▍| 211186/225000 [408:35:40<32:18:10,  8.42s/it]
 94%|█████████▍| 211187/225000 [408:35:48<31:37:38,  8.24s/it]
 94%|█████████▍| 211188/225000 [408:36:02<38:37:57, 10.07s/it]
 94%|█████████▍| 211189/225000 [408:36:09<35:38:19,  9.29s/it]
 94%|█████████▍| 211190/225000 [408:36:19<35:56:30,  9.37s/it]
 94%|█████████▍| 211191/225000 [408:36:33<41:36:29, 10.85s/it]
 94%|█████████▍| 211192/225000 [408:36:39<35:18:30,  9.21s/it]
 94%|█████████▍| 211193/225000 [408:36:46<33:18:46,  8.69s/it]
 94%|█████████▍| 211194/225000 [408:36:55<33:55:01,  8.84s/it]
 94%|█████████▍| 211195/225000 [408:37:00<29:38:37,  7.73s/it]
 94%|█████████▍| 211196/225000 [408:37:13<35:30:12,  9.26s/it]
 94%|█████████▍| 211197/225000 [408:37:20<32:43:15,  8.53s/it]
 94%|█████████▍| 211198/225000 [408:37:26<29:34:37,  7.71s/it]
 94%|█████████▍| 211199/225000 [408:37:38<35:08:58,  9.17s/it]
 94%|█████████▍| 211200/225000 [408:37:47<33:53:41,  8.84s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.501', 'learning_rate': '5.138e-07', 'epoch': '0.9387', 'num_input_tokens_seen': 691702304, 'train_runtime': '1.471e+06', 'train_tokens_per_second': '470.2'}

 94%|█████████▍| 211200/225000 [408:37:47<33:53:41,  8.84s/it]
 94%|█████████▍| 211201/225000 [408:37:54<31:47:02,  8.29s/it]
 94%|█████████▍| 211202/225000 [408:38:02<31:46:03,  8.29s/it]
 94%|█████████▍| 211203/225000 [408:38:09<30:04:56,  7.85s/it]
 94%|█████████▍| 211204/225000 [408:38:19<32:54:23,  8.59s/it]
 94%|█████████▍| 211205/225000 [408:38:31<36:38:27,  9.56s/it]
 94%|█████████▍| 211206/225000 [408:38:39<34:37:20,  9.04s/it]
 94%|█████████▍| 211207/225000 [408:38:49<36:28:09,  9.52s/it]
 94%|█████████▍| 211208/225000 [408:38:55<31:42:16,  8.28s/it]
 94%|█████████▍| 211209/225000 [408:39:09<38:50:41, 10.14s/it]
 94%|█████████▍| 211210/225000 [408:39:14<33:10:13,  8.66s/it]
 94%|█████████▍| 211211/225000 [408:39:25<35:43:21,  9.33s/it]
 94%|█████████▍| 211212/225000 [408:39:35<36:16:50,  9.47s/it]
 94%|█████████▍| 211213/225000 [408:39:43<34:24:56,  8.99s/it]
 94%|█████████▍| 211214/225000 [408:39:55<38:15:55,  9.99s/it]
 94%|█████████▍| 211215/225000 [408:40:06<39:01:18, 10.19s/it]
 94%|█████████▍| 211216/225000 [408:40:13<35:34:48,  9.29s/it]
 94%|█████████▍| 211217/225000 [408:40:24<37:04:16,  9.68s/it]
 94%|█████████▍| 211218/225000 [408:40:29<32:32:24,  8.50s/it]
 94%|█████████▍| 211219/225000 [408:40:39<33:57:12,  8.87s/it]
 94%|█████████▍| 211220/225000 [408:40:46<31:21:31,  8.19s/it]
 94%|█████████▍| 211221/225000 [408:40:56<33:07:03,  8.65s/it]
 94%|█████████▍| 211222/225000 [408:41:01<29:30:58,  7.71s/it]
 94%|█████████▍| 211223/225000 [408:41:09<29:27:26,  7.70s/it]
 94%|█████████▍| 211224/225000 [408:41:20<33:11:35,  8.67s/it]
 94%|█████████▍| 211225/225000 [408:41:27<31:32:36,  8.24s/it]
 94%|█████████▍| 211226/225000 [408:41:32<28:04:01,  7.34s/it]
 94%|█████████▍| 211227/225000 [408:41:42<31:11:32,  8.15s/it]
 94%|█████████▍| 211228/225000 [408:41:53<33:48:14,  8.84s/it]
 94%|█████████▍| 211229/225000 [408:42:09<42:15:13, 11.05s/it]
 94%|█████████▍| 211230/225000 [408:42:17<39:24:14, 10.30s/it]
 94%|█████████▍| 211231/225000 [408:42:24<34:40:58,  9.07s/it]
 94%|█████████▍| 211232/225000 [408:42:38<41:03:21, 10.74s/it]
 94%|█████████▍| 211233/225000 [408:42:45<37:04:58,  9.70s/it]
 94%|█████████▍| 211234/225000 [408:42:57<38:51:22, 10.16s/it]
 94%|█████████▍| 211235/225000 [408:43:05<36:31:40,  9.55s/it]
 94%|█████████▍| 211236/225000 [408:43:11<32:31:37,  8.51s/it]
 94%|█████████▍| 211237/225000 [408:43:18<31:24:16,  8.21s/it]
 94%|█████████▍| 211238/225000 [408:43:26<30:20:48,  7.94s/it]
 94%|█████████▍| 211239/225000 [408:43:34<30:51:30,  8.07s/it]
 94%|█████████▍| 211240/225000 [408:43:41<29:46:35,  7.79s/it]
 94%|█████████▍| 211241/225000 [408:43:54<35:41:47,  9.34s/it]
 94%|█████████▍| 211242/225000 [408:44:05<37:10:14,  9.73s/it]
 94%|█████████▍| 211243/225000 [408:44:14<36:15:05,  9.49s/it]
 94%|█████████▍| 211244/225000 [408:44:19<31:07:24,  8.15s/it]
 94%|█████████▍| 211245/225000 [408:44:27<31:28:55,  8.24s/it]
 94%|█████████▍| 211246/225000 [408:44:38<34:44:21,  9.09s/it]
 94%|█████████▍| 211247/225000 [408:44:45<31:28:59,  8.24s/it]
 94%|█████████▍| 211248/225000 [408:44:51<29:32:14,  7.73s/it]
 94%|█████████▍| 211249/225000 [408:45:04<35:33:38,  9.31s/it]
 94%|█████████▍| 211250/225000 [408:45:12<33:34:49,  8.79s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.5', 'learning_rate': '5.101e-07', 'epoch': '0.9389', 'num_input_tokens_seen': 691940120, 'train_runtime': '1.472e+06', 'train_tokens_per_second': '470.2'}

 94%|█████████▍| 211250/225000 [408:45:12<33:34:49,  8.79s/it]
 94%|█████████▍| 211251/225000 [408:45:20<33:10:09,  8.68s/it]
 94%|█████████▍| 211252/225000 [408:45:26<29:28:53,  7.72s/it]
 94%|█████████▍| 211253/225000 [408:45:32<27:46:59,  7.28s/it]
 94%|█████████▍| 211254/225000 [408:45:38<27:05:32,  7.10s/it]
 94%|█████████▍| 211255/225000 [408:45:44<25:19:17,  6.63s/it]
 94%|█████████▍| 211256/225000 [408:45:59<35:18:26,  9.25s/it]
 94%|█████████▍| 211257/225000 [408:46:06<32:40:01,  8.56s/it]
 94%|█████████▍| 211258/225000 [408:46:13<30:20:00,  7.95s/it]
 94%|█████████▍| 211259/225000 [408:46:22<31:09:26,  8.16s/it]
 94%|█████████▍| 211260/225000 [408:46:33<35:20:37,  9.26s/it]
 94%|█████████▍| 211261/225000 [408:46:41<33:37:56,  8.81s/it]
 94%|█████████▍| 211262/225000 [408:46:54<38:38:26, 10.13s/it]
 94%|█████████▍| 211263/225000 [408:47:02<36:00:35,  9.44s/it]
 94%|█████████▍| 211264/225000 [408:47:10<33:56:43,  8.90s/it]
 94%|█████████▍| 211265/225000 [408:47:16<31:06:43,  8.15s/it]
 94%|█████████▍| 211266/225000 [408:47:25<31:43:50,  8.32s/it]
 94%|█████████▍| 211267/225000 [408:47:33<31:47:11,  8.33s/it]
 94%|█████████▍| 211268/225000 [408:47:40<29:26:02,  7.72s/it]
 94%|█████████▍| 211269/225000 [408:47:52<34:30:21,  9.05s/it]
 94%|█████████▍| 211270/225000 [408:47:56<29:24:35,  7.71s/it]
 94%|█████████▍| 211271/225000 [408:48:06<32:16:15,  8.46s/it]
 94%|█████████▍| 211272/225000 [408:48:16<33:41:59,  8.84s/it]
 94%|█████████▍| 211273/225000 [408:48:20<27:54:46,  7.32s/it]
 94%|█████████▍| 211274/225000 [408:48:26<26:30:07,  6.95s/it]
 94%|█████████▍| 211275/225000 [408:48:36<30:16:29,  7.94s/it]
 94%|█████████▍| 211276/225000 [408:48:42<27:38:41,  7.25s/it]
 94%|█████████▍| 211277/225000 [408:48:47<25:31:20,  6.70s/it]
 94%|█████████▍| 211278/225000 [408:48:59<31:36:34,  8.29s/it]
 94%|█████████▍| 211279/225000 [408:49:06<29:21:02,  7.70s/it]
 94%|█████████▍| 211280/225000 [408:49:10<25:54:27,  6.80s/it]
 94%|█████████▍| 211281/225000 [408:49:21<30:29:20,  8.00s/it]
 94%|█████████▍| 211282/225000 [408:49:30<31:33:08,  8.28s/it]
 94%|█████████▍| 211283/225000 [408:49:37<30:03:01,  7.89s/it]
 94%|█████████▍| 211284/225000 [408:49:47<32:02:58,  8.41s/it]
 94%|█████████▍| 211285/225000 [408:49:52<28:49:25,  7.57s/it]
 94%|█████████▍| 211286/225000 [408:50:01<29:54:19,  7.85s/it]
 94%|█████████▍| 211287/225000 [408:50:14<36:00:13,  9.45s/it]
 94%|█████████▍| 211288/225000 [408:50:21<32:40:40,  8.58s/it]
 94%|█████████▍| 211289/225000 [408:50:28<31:53:28,  8.37s/it]
 94%|█████████▍| 211290/225000 [408:50:39<34:40:53,  9.11s/it]
 94%|█████████▍| 211291/225000 [408:50:46<32:28:55,  8.53s/it]
 94%|█████████▍| 211292/225000 [408:50:54<31:01:31,  8.15s/it]
 94%|█████████▍| 211293/225000 [408:51:00<28:58:01,  7.61s/it]
 94%|█████████▍| 211294/225000 [408:51:13<35:01:02,  9.20s/it]
 94%|█████████▍| 211295/225000 [408:51:19<31:22:16,  8.24s/it]
 94%|█████████▍| 211296/225000 [408:51:28<31:49:09,  8.36s/it]
 94%|█████████▍| 211297/225000 [408:51:35<30:22:09,  7.98s/it]
 94%|█████████▍| 211298/225000 [408:51:42<29:10:35,  7.67s/it]
 94%|█████████▍| 211299/225000 [408:51:49<28:21:37,  7.45s/it]
 94%|█████████▍| 211300/225000 [408:51:56<28:43:34,  7.55s/it]
                                                              
{'loss': '1.283', 'grad_norm': '1.359', 'learning_rate': '5.065e-07', 'epoch': '0.9391', 'num_input_tokens_seen': 692162448, 'train_runtime': '1.472e+06', 'train_tokens_per_second': '470.2'}

 94%|█████████▍| 211300/225000 [408:51:56<28:43:34,  7.55s/it]
 94%|█████████▍| 211301/225000 [408:52:11<37:18:44,  9.81s/it]
 94%|█████████▍| 211302/225000 [408:52:15<30:37:30,  8.05s/it]
 94%|█████████▍| 211303/225000 [408:52:26<33:48:51,  8.89s/it]
 94%|█████████▍| 211304/225000 [408:52:34<32:50:35,  8.63s/it]
 94%|█████████▍| 211305/225000 [408:52:44<33:38:27,  8.84s/it]
 94%|█████████▍| 211306/225000 [408:52:52<33:24:48,  8.78s/it]
 94%|█████████▍| 211307/225000 [408:53:01<33:02:49,  8.69s/it]
 94%|█████████▍| 211308/225000 [408:53:11<35:08:22,  9.24s/it]
 94%|█████████▍| 211309/225000 [408:53:22<36:29:04,  9.59s/it]
 94%|█████████▍| 211310/225000 [408:53:33<38:49:17, 10.21s/it]
 94%|█████████▍| 211311/225000 [408:53:41<35:36:24,  9.36s/it]
 94%|█████████▍| 211312/225000 [408:53:47<32:38:14,  8.58s/it]
 94%|█████████▍| 211313/225000 [408:53:59<35:27:40,  9.33s/it]
 94%|█████████▍| 211314/225000 [408:54:06<33:28:51,  8.81s/it]
 94%|█████████▍| 211315/225000 [408:54:13<31:06:21,  8.18s/it]
 94%|█████████▍| 211316/225000 [408:54:22<32:21:43,  8.51s/it]
 94%|█████████▍| 211317/225000 [408:54:28<29:15:22,  7.70s/it]
 94%|█████████▍| 211318/225000 [408:54:38<32:25:14,  8.53s/it]
 94%|█████████▍| 211319/225000 [408:54:48<33:07:33,  8.72s/it]
 94%|█████████▍| 211320/225000 [408:54:55<31:46:11,  8.36s/it]
 94%|█████████▍| 211321/225000 [408:55:04<32:36:21,  8.58s/it]
 94%|█████████▍| 211322/225000 [408:55:12<31:14:01,  8.22s/it]
 94%|█████████▍| 211323/225000 [408:55:22<33:41:24,  8.87s/it]
 94%|█████████▍| 211324/225000 [408:55:28<30:24:50,  8.01s/it]
 94%|█████████▍| 211325/225000 [408:55:39<34:03:21,  8.97s/it]
 94%|█████████▍| 211326/225000 [408:55:45<30:25:48,  8.01s/it]
 94%|█████████▍| 211327/225000 [408:55:59<36:54:46,  9.72s/it]
 94%|█████████▍| 211328/225000 [408:56:10<38:42:16, 10.19s/it]
 94%|█████████▍| 211329/225000 [408:56:15<33:11:48,  8.74s/it]
 94%|█████████▍| 211330/225000 [408:56:25<34:32:52,  9.10s/it]
 94%|█████████▍| 211331/225000 [408:56:36<36:49:49,  9.70s/it]
 94%|█████████▍| 211332/225000 [408:56:44<34:38:55,  9.13s/it]
 94%|█████████▍| 211333/225000 [408:56:50<31:06:00,  8.19s/it]
 94%|█████████▍| 211334/225000 [408:56:57<29:13:40,  7.70s/it]
 94%|█████████▍| 211335/225000 [408:57:04<29:14:30,  7.70s/it]
 94%|█████████▍| 211336/225000 [408:57:15<32:31:40,  8.57s/it]
 94%|█████████▍| 211337/225000 [408:57:24<32:51:51,  8.66s/it]
 94%|█████████▍| 211338/225000 [408:57:29<29:19:06,  7.73s/it]
 94%|█████████▍| 211339/225000 [408:57:36<27:49:11,  7.33s/it]
 94%|█████████▍| 211340/225000 [408:57:45<29:32:49,  7.79s/it]
 94%|█████████▍| 211341/225000 [408:57:55<32:41:56,  8.62s/it]
 94%|█████████▍| 211342/225000 [408:58:04<32:28:03,  8.56s/it]
 94%|█████████▍| 211343/225000 [408:58:14<34:42:33,  9.15s/it]
 94%|█████████▍| 211344/225000 [408:58:22<33:08:13,  8.74s/it]
 94%|█████████▍| 211345/225000 [408:58:30<32:35:30,  8.59s/it]
 94%|█████████▍| 211346/225000 [408:58:39<32:34:03,  8.59s/it]
 94%|█████████▍| 211347/225000 [408:58:45<29:39:46,  7.82s/it]
 94%|█████████▍| 211348/225000 [408:58:53<30:04:55,  7.93s/it]
 94%|█████████▍| 211349/225000 [408:59:03<32:17:15,  8.51s/it]
 94%|█████████▍| 211350/225000 [408:59:09<29:29:01,  7.78s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.309', 'learning_rate': '5.028e-07', 'epoch': '0.9393', 'num_input_tokens_seen': 692390808, 'train_runtime': '1.472e+06', 'train_tokens_per_second': '470.3'}

 94%|█████████▍| 211350/225000 [408:59:09<29:29:01,  7.78s/it]
 94%|█████████▍| 211351/225000 [408:59:14<26:37:59,  7.02s/it]
 94%|█████████▍| 211352/225000 [408:59:22<27:25:52,  7.24s/it]
 94%|█████████▍| 211353/225000 [408:59:28<25:44:46,  6.79s/it]
 94%|█████████▍| 211354/225000 [408:59:34<25:03:01,  6.61s/it]
 94%|█████████▍| 211355/225000 [408:59:42<26:24:34,  6.97s/it]
 94%|█████████▍| 211356/225000 [408:59:55<33:59:11,  8.97s/it]
 94%|█████████▍| 211357/225000 [409:00:07<36:51:56,  9.73s/it]
 94%|█████████▍| 211358/225000 [409:00:13<32:55:27,  8.69s/it]
 94%|█████████▍| 211359/225000 [409:00:24<35:12:56,  9.29s/it]
 94%|█████████▍| 211360/225000 [409:00:31<32:51:11,  8.67s/it]
 94%|█████████▍| 211361/225000 [409:00:38<31:24:51,  8.29s/it]
 94%|█████████▍| 211362/225000 [409:00:50<34:53:30,  9.21s/it]
 94%|█████████▍| 211363/225000 [409:00:58<34:02:06,  8.98s/it]
 94%|█████████▍| 211364/225000 [409:01:08<35:07:03,  9.27s/it]
 94%|█████████▍| 211365/225000 [409:01:15<32:45:14,  8.65s/it]
 94%|█████████▍| 211366/225000 [409:01:22<30:25:48,  8.03s/it]
 94%|█████████▍| 211367/225000 [409:01:30<30:38:45,  8.09s/it]
 94%|█████████▍| 211368/225000 [409:01:43<36:34:16,  9.66s/it]
 94%|█████████▍| 211369/225000 [409:01:48<31:10:44,  8.23s/it]
 94%|█████████▍| 211370/225000 [409:01:56<30:49:46,  8.14s/it]
 94%|█████████▍| 211371/225000 [409:02:03<28:41:09,  7.58s/it]
 94%|█████████▍| 211372/225000 [409:02:09<27:48:15,  7.34s/it]
 94%|█████████▍| 211373/225000 [409:02:17<27:35:55,  7.29s/it]
 94%|█████████▍| 211374/225000 [409:02:25<29:16:06,  7.73s/it]
 94%|█████████▍| 211375/225000 [409:02:39<36:21:04,  9.60s/it]
 94%|█████████▍| 211376/225000 [409:02:47<33:51:30,  8.95s/it]
 94%|█████████▍| 211377/225000 [409:02:55<33:00:35,  8.72s/it]
 94%|█████████▍| 211378/225000 [409:03:03<32:10:14,  8.50s/it]
 94%|█████████▍| 211379/225000 [409:03:15<35:54:04,  9.49s/it]
 94%|█████████▍| 211380/225000 [409:03:21<32:33:01,  8.60s/it]
 94%|█████████▍| 211381/225000 [409:03:29<31:55:10,  8.44s/it]
 94%|█████████▍| 211382/225000 [409:03:33<27:03:43,  7.15s/it]
 94%|█████████▍| 211383/225000 [409:03:45<31:42:17,  8.38s/it]
 94%|█████████▍| 211384/225000 [409:03:51<29:32:31,  7.81s/it]
 94%|█████████▍| 211385/225000 [409:03:57<27:36:11,  7.30s/it]
 94%|█████████▍| 211386/225000 [409:04:04<27:28:15,  7.26s/it]
 94%|█████████▍| 211387/225000 [409:04:11<26:22:43,  6.98s/it]
 94%|█████████▍| 211388/225000 [409:04:17<25:11:36,  6.66s/it]
 94%|█████████▍| 211389/225000 [409:04:29<31:17:44,  8.28s/it]
 94%|█████████▍| 211390/225000 [409:04:38<31:54:02,  8.44s/it]
 94%|█████████▍| 211391/225000 [409:04:45<31:10:43,  8.25s/it]
 94%|█████████▍| 211392/225000 [409:04:52<29:53:29,  7.91s/it]
 94%|█████████▍| 211393/225000 [409:05:05<35:06:21,  9.29s/it]
 94%|█████████▍| 211394/225000 [409:05:16<36:54:13,  9.76s/it]
 94%|█████████▍| 211395/225000 [409:05:22<33:13:41,  8.79s/it]
 94%|█████████▍| 211396/225000 [409:05:29<30:42:18,  8.13s/it]
 94%|█████████▍| 211397/225000 [409:05:35<28:03:59,  7.43s/it]
 94%|█████████▍| 211398/225000 [409:05:41<26:57:38,  7.14s/it]
 94%|█████████▍| 211399/225000 [409:05:52<31:25:58,  8.32s/it]
 94%|█████████▍| 211400/225000 [409:05:58<28:11:22,  7.46s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.376', 'learning_rate': '4.992e-07', 'epoch': '0.9396', 'num_input_tokens_seen': 692615544, 'train_runtime': '1.473e+06', 'train_tokens_per_second': '470.3'}

 94%|█████████▍| 211400/225000 [409:05:58<28:11:22,  7.46s/it]
 94%|█████████▍| 211401/225000 [409:06:03<26:14:35,  6.95s/it]
 94%|█████████▍| 211402/225000 [409:06:15<31:34:08,  8.36s/it]
 94%|█████████▍| 211403/225000 [409:06:23<31:27:26,  8.33s/it]
 94%|█████████▍| 211404/225000 [409:06:30<29:23:51,  7.78s/it]
 94%|█████████▍| 211405/225000 [409:06:36<26:56:34,  7.13s/it]
 94%|█████████▍| 211406/225000 [409:06:43<27:45:35,  7.35s/it]
 94%|█████████▍| 211407/225000 [409:06:56<33:37:09,  8.90s/it]
 94%|█████████▍| 211408/225000 [409:07:03<31:55:04,  8.45s/it]
 94%|█████████▍| 211409/225000 [409:07:10<29:42:51,  7.87s/it]
 94%|█████████▍| 211410/225000 [409:07:16<27:30:43,  7.29s/it]
 94%|█████████▍| 211411/225000 [409:07:24<28:24:52,  7.53s/it]
 94%|█████████▍| 211412/225000 [409:07:35<32:56:41,  8.73s/it]
 94%|█████████▍| 211413/225000 [409:07:41<29:26:37,  7.80s/it]
 94%|█████████▍| 211414/225000 [409:07:49<29:58:16,  7.94s/it]
 94%|█████████▍| 211415/225000 [409:07:57<29:28:36,  7.81s/it]
 94%|█████████▍| 211416/225000 [409:08:03<28:07:45,  7.45s/it]
 94%|█████████▍| 211417/225000 [409:08:13<31:01:52,  8.22s/it]
 94%|█████████▍| 211418/225000 [409:08:22<31:26:27,  8.33s/it]
 94%|█████████▍| 211419/225000 [409:08:28<28:43:11,  7.61s/it]
 94%|█████████▍| 211420/225000 [409:08:35<27:37:21,  7.32s/it]
 94%|█████████▍| 211421/225000 [409:08:43<28:18:58,  7.51s/it]
 94%|█████████▍| 211422/225000 [409:08:54<32:28:56,  8.61s/it]
 94%|█████████▍| 211423/225000 [409:09:00<29:56:26,  7.94s/it]
 94%|█████████▍| 211424/225000 [409:09:11<33:04:03,  8.77s/it]
 94%|█████████▍| 211425/225000 [409:09:16<29:01:22,  7.70s/it]
 94%|█████████▍| 211426/225000 [409:09:26<32:07:51,  8.52s/it]
 94%|█████████▍| 211427/225000 [409:09:36<33:51:50,  8.98s/it]
 94%|█████████▍| 211428/225000 [409:09:47<35:48:07,  9.50s/it]
 94%|█████████▍| 211429/225000 [409:09:55<33:47:51,  8.97s/it]
 94%|█████████▍| 211430/225000 [409:10:02<31:39:54,  8.40s/it]
 94%|█████████▍| 211431/225000 [409:10:11<31:55:13,  8.47s/it]
 94%|█████████▍| 211432/225000 [409:10:25<38:24:47, 10.19s/it]
 94%|█████████▍| 211433/225000 [409:10:34<37:09:32,  9.86s/it]
 94%|█████████▍| 211434/225000 [409:10:40<33:07:02,  8.79s/it]
 94%|█████████▍| 211435/225000 [409:10:46<29:59:50,  7.96s/it]
 94%|█████████▍| 211436/225000 [409:10:52<27:22:11,  7.26s/it]
 94%|█████████▍| 211437/225000 [409:11:04<33:15:07,  8.83s/it]
 94%|█████████▍| 211438/225000 [409:11:12<31:41:16,  8.41s/it]
 94%|█████████▍| 211439/225000 [409:11:19<30:54:13,  8.20s/it]
 94%|█████████▍| 211440/225000 [409:11:29<32:25:25,  8.61s/it]
 94%|█████████▍| 211441/225000 [409:11:39<33:54:02,  9.00s/it]
 94%|█████████▍| 211442/225000 [409:11:48<33:40:45,  8.94s/it]
 94%|█████████▍| 211443/225000 [409:11:54<30:33:04,  8.11s/it]
 94%|█████████▍| 211444/225000 [409:12:04<32:48:23,  8.71s/it]
 94%|█████████▍| 211445/225000 [409:12:13<32:37:21,  8.66s/it]
 94%|█████████▍| 211446/225000 [409:12:18<29:18:20,  7.78s/it]
 94%|█████████▍| 211447/225000 [409:12:25<27:43:58,  7.37s/it]
 94%|█████████▍| 211448/225000 [409:12:33<28:54:07,  7.68s/it]
 94%|█████████▍| 211449/225000 [409:12:40<27:52:34,  7.41s/it]
 94%|█████████▍| 211450/225000 [409:12:47<27:53:30,  7.41s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.119', 'learning_rate': '4.956e-07', 'epoch': '0.9398', 'num_input_tokens_seen': 692835264, 'train_runtime': '1.473e+06', 'train_tokens_per_second': '470.3'}

 94%|█████████▍| 211450/225000 [409:12:47<27:53:30,  7.41s/it]
 94%|█████████▍| 211451/225000 [409:12:55<28:35:25,  7.60s/it]
 94%|█████████▍| 211452/225000 [409:13:09<35:29:20,  9.43s/it]
 94%|█████████▍| 211453/225000 [409:13:18<35:07:20,  9.33s/it]
 94%|█████████▍| 211454/225000 [409:13:24<31:39:31,  8.41s/it]
 94%|█████████▍| 211455/225000 [409:13:35<33:54:19,  9.01s/it]
 94%|█████████▍| 211456/225000 [409:13:46<36:33:03,  9.72s/it]
 94%|█████████▍| 211457/225000 [409:13:55<35:15:05,  9.37s/it]
 94%|█████████▍| 211458/225000 [409:14:04<34:34:52,  9.19s/it]
 94%|█████████▍| 211459/225000 [409:14:09<30:50:27,  8.20s/it]
 94%|█████████▍| 211460/225000 [409:14:15<28:00:36,  7.45s/it]
 94%|█████████▍| 211461/225000 [409:14:26<32:25:13,  8.62s/it]
 94%|█████████▍| 211462/225000 [409:14:31<27:24:42,  7.29s/it]
 94%|█████████▍| 211463/225000 [409:14:38<27:37:21,  7.35s/it]
 94%|█████████▍| 211464/225000 [409:14:50<32:46:08,  8.72s/it]
 94%|█████████▍| 211465/225000 [409:15:00<34:38:38,  9.21s/it]
 94%|█████████▍| 211466/225000 [409:15:09<34:28:29,  9.17s/it]
 94%|█████████▍| 211467/225000 [409:15:20<35:53:15,  9.55s/it]
 94%|█████████▍| 211468/225000 [409:15:27<33:17:23,  8.86s/it]
 94%|█████████▍| 211469/225000 [409:15:36<32:44:53,  8.71s/it]
 94%|█████████▍| 211470/225000 [409:15:43<31:48:14,  8.46s/it]
 94%|█████████▍| 211471/225000 [409:15:50<29:18:19,  7.80s/it]
 94%|█████████▍| 211472/225000 [409:16:01<33:50:18,  9.00s/it]
 94%|█████████▍| 211473/225000 [409:16:08<30:39:16,  8.16s/it]
 94%|█████████▍| 211474/225000 [409:16:16<30:17:46,  8.06s/it]
 94%|█████████▍| 211475/225000 [409:16:27<33:37:40,  8.95s/it]
 94%|█████████▍| 211476/225000 [409:16:39<38:00:46, 10.12s/it]
 94%|█████████▍| 211477/225000 [409:16:45<33:05:54,  8.81s/it]
 94%|█████████▍| 211478/225000 [409:16:52<31:16:04,  8.32s/it]
 94%|█████████▍| 211479/225000 [409:17:00<30:19:19,  8.07s/it]
 94%|█████████▍| 211480/225000 [409:17:07<29:33:51,  7.87s/it]
 94%|█████████▍| 211481/225000 [409:17:19<33:51:36,  9.02s/it]
 94%|█████████▍| 211482/225000 [409:17:27<32:48:26,  8.74s/it]
 94%|█████████▍| 211483/225000 [409:17:35<32:15:27,  8.59s/it]
 94%|█████████▍| 211484/225000 [409:17:44<32:54:05,  8.76s/it]
 94%|█████████▍| 211485/225000 [409:17:55<34:43:57,  9.25s/it]
 94%|█████████▍| 211486/225000 [409:18:04<34:11:24,  9.11s/it]
 94%|█████████▍| 211487/225000 [409:18:10<31:42:20,  8.45s/it]
 94%|█████████▍| 211488/225000 [409:18:23<36:38:59,  9.76s/it]
 94%|█████████▍| 211489/225000 [409:18:31<34:29:17,  9.19s/it]
 94%|█████████▍| 211490/225000 [409:18:37<31:14:11,  8.32s/it]
 94%|█████████▍| 211491/225000 [409:18:48<33:58:35,  9.05s/it]
 94%|█████████▍| 211492/225000 [409:18:56<32:40:35,  8.71s/it]
 94%|█████████▍| 211493/225000 [409:19:03<31:03:42,  8.28s/it]
 94%|█████████▍| 211494/225000 [409:19:18<37:41:19, 10.05s/it]
 94%|█████████▍| 211495/225000 [409:19:24<33:05:12,  8.82s/it]
 94%|█████████▍| 211496/225000 [409:19:33<34:21:41,  9.16s/it]
 94%|█████████▍| 211497/225000 [409:19:47<39:03:07, 10.41s/it]
 94%|█████████▍| 211498/225000 [409:19:52<33:17:08,  8.87s/it]
 94%|█████████▍| 211499/225000 [409:19:58<29:57:18,  7.99s/it]
 94%|█████████▍| 211500/225000 [409:20:06<29:56:04,  7.98s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.608', 'learning_rate': '4.919e-07', 'epoch': '0.94', 'num_input_tokens_seen': 693070864, 'train_runtime': '1.474e+06', 'train_tokens_per_second': '470.3'}

 94%|█████████▍| 211500/225000 [409:20:06<29:56:04,  7.98s/it]
 94%|█████████▍| 211501/225000 [409:20:14<29:37:09,  7.90s/it]
 94%|█████████▍| 211502/225000 [409:20:20<27:45:24,  7.40s/it]
 94%|█████████▍| 211503/225000 [409:20:27<27:17:04,  7.28s/it]
 94%|█████████▍| 211504/225000 [409:20:40<33:18:42,  8.89s/it]
 94%|█████████▍| 211505/225000 [409:20:49<33:55:42,  9.05s/it]
 94%|█████████▍| 211506/225000 [409:20:54<29:29:11,  7.87s/it]
 94%|█████████▍| 211507/225000 [409:21:01<28:20:40,  7.56s/it]
 94%|█████████▍| 211508/225000 [409:21:11<30:52:53,  8.24s/it]
 94%|█████████▍| 211509/225000 [409:21:23<35:44:35,  9.54s/it]
 94%|█████████▍| 211510/225000 [409:21:30<32:28:21,  8.67s/it]
 94%|█████████▍| 211511/225000 [409:21:39<33:25:23,  8.92s/it]
 94%|█████████▍| 211512/225000 [409:21:45<29:46:46,  7.95s/it]
 94%|█████████▍| 211513/225000 [409:21:52<28:30:40,  7.61s/it]
 94%|█████████▍| 211514/225000 [409:22:02<31:23:45,  8.38s/it]
 94%|█████████▍| 211515/225000 [409:22:06<26:14:09,  7.00s/it]
 94%|█████████▍| 211516/225000 [409:22:12<25:31:10,  6.81s/it]
 94%|█████████▍| 211517/225000 [409:22:23<29:34:14,  7.90s/it]
 94%|█████████▍| 211518/225000 [409:22:33<31:57:49,  8.54s/it]
 94%|█████████▍| 211519/225000 [409:22:42<33:12:30,  8.87s/it]
 94%|█████████▍| 211520/225000 [409:22:50<31:22:40,  8.38s/it]
 94%|█████████▍| 211521/225000 [409:22:57<29:59:15,  8.01s/it]
 94%|█████████▍| 211522/225000 [409:23:05<29:54:18,  7.99s/it]
 94%|█████████▍| 211523/225000 [409:23:12<29:01:28,  7.75s/it]
 94%|█████████▍| 211524/225000 [409:23:19<27:42:32,  7.40s/it]
 94%|█████████▍| 211525/225000 [409:23:35<38:05:24, 10.18s/it]
 94%|█████████▍| 211526/225000 [409:23:44<36:40:48,  9.80s/it]
 94%|█████████▍| 211527/225000 [409:23:51<33:51:52,  9.05s/it]
 94%|█████████▍| 211528/225000 [409:23:59<31:47:20,  8.49s/it]
 94%|█████████▍| 211529/225000 [409:24:10<34:41:22,  9.27s/it]
 94%|█████████▍| 211530/225000 [409:24:21<37:00:06,  9.89s/it]
 94%|█████████▍| 211531/225000 [409:24:27<32:29:00,  8.68s/it]
 94%|█████████▍| 211532/225000 [409:24:32<29:01:12,  7.76s/it]
 94%|█████████▍| 211533/225000 [409:24:47<36:51:33,  9.85s/it]
 94%|█████████▍| 211534/225000 [409:24:51<30:19:52,  8.11s/it]
 94%|█████████▍| 211535/225000 [409:24:56<26:27:47,  7.08s/it]
 94%|█████████▍| 211536/225000 [409:25:03<26:33:28,  7.10s/it]
 94%|█████████▍| 211537/225000 [409:25:10<26:41:33,  7.14s/it]
 94%|█████████▍| 211538/225000 [409:25:17<26:42:44,  7.14s/it]
 94%|█████████▍| 211539/225000 [409:25:29<31:49:11,  8.51s/it]
 94%|█████████▍| 211540/225000 [409:25:36<30:04:52,  8.05s/it]
 94%|█████████▍| 211541/225000 [409:25:46<32:07:32,  8.59s/it]
 94%|█████████▍| 211542/225000 [409:25:59<36:33:50,  9.78s/it]
 94%|█████████▍| 211543/225000 [409:26:07<34:56:07,  9.35s/it]
 94%|█████████▍| 211544/225000 [409:26:13<31:04:22,  8.31s/it]
 94%|█████████▍| 211545/225000 [409:26:22<32:18:23,  8.64s/it]
 94%|█████████▍| 211546/225000 [409:26:37<38:39:29, 10.34s/it]
 94%|█████████▍| 211547/225000 [409:26:41<32:17:41,  8.64s/it]
 94%|█████████▍| 211548/225000 [409:26:51<33:26:00,  8.95s/it]
 94%|█████████▍| 211549/225000 [409:26:59<32:22:39,  8.67s/it]
 94%|█████████▍| 211550/225000 [409:27:12<37:16:34,  9.98s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.253', 'learning_rate': '4.883e-07', 'epoch': '0.9402', 'num_input_tokens_seen': 693298040, 'train_runtime': '1.474e+06', 'train_tokens_per_second': '470.3'}

 94%|█████████▍| 211550/225000 [409:27:12<37:16:34,  9.98s/it]
 94%|█████████▍| 211551/225000 [409:27:19<33:30:15,  8.97s/it]
 94%|█████████▍| 211552/225000 [409:27:30<36:05:07,  9.66s/it]
 94%|█████████▍| 211553/225000 [409:27:39<35:45:10,  9.57s/it]
 94%|█████████▍| 211554/225000 [409:27:48<34:46:48,  9.31s/it]
 94%|█████████▍| 211555/225000 [409:28:02<40:13:23, 10.77s/it]
 94%|█████████▍| 211556/225000 [409:28:09<36:08:43,  9.68s/it]
 94%|█████████▍| 211557/225000 [409:28:17<34:18:13,  9.19s/it]
 94%|█████████▍| 211558/225000 [409:28:25<32:16:27,  8.64s/it]
 94%|█████████▍| 211559/225000 [409:28:33<31:38:14,  8.47s/it]
 94%|█████████▍| 211560/225000 [409:28:40<30:42:58,  8.23s/it]
 94%|█████████▍| 211561/225000 [409:28:48<29:51:02,  8.00s/it]
 94%|█████████▍| 211562/225000 [409:28:58<32:39:24,  8.75s/it]
 94%|█████████▍| 211563/225000 [409:29:07<32:40:39,  8.75s/it]
 94%|█████████▍| 211564/225000 [409:29:22<39:08:40, 10.49s/it]
 94%|█████████▍| 211565/225000 [409:29:29<36:08:22,  9.68s/it]
 94%|█████████▍| 211566/225000 [409:29:34<30:40:55,  8.22s/it]
 94%|█████████▍| 211567/225000 [409:29:44<31:54:19,  8.55s/it]
 94%|█████████▍| 211568/225000 [409:29:54<34:06:24,  9.14s/it]
 94%|█████████▍| 211569/225000 [409:30:03<33:37:58,  9.01s/it]
 94%|█████████▍| 211570/225000 [409:30:12<34:01:01,  9.12s/it]
 94%|█████████▍| 211571/225000 [409:30:24<37:01:50,  9.93s/it]
 94%|█████████▍| 211572/225000 [409:30:30<32:54:41,  8.82s/it]
 94%|█████████▍| 211573/225000 [409:30:35<28:21:36,  7.60s/it]
 94%|█████████▍| 211574/225000 [409:30:47<33:27:23,  8.97s/it]
 94%|█████████▍| 211575/225000 [409:30:54<31:27:43,  8.44s/it]
 94%|█████████▍| 211576/225000 [409:31:02<30:09:14,  8.09s/it]
 94%|█████████▍| 211577/225000 [409:31:09<28:53:27,  7.75s/it]
 94%|█████████▍| 211578/225000 [409:31:16<28:36:57,  7.68s/it]
 94%|█████████▍| 211579/225000 [409:31:22<26:11:53,  7.03s/it]
 94%|█████████▍| 211580/225000 [409:31:28<25:40:38,  6.89s/it]
 94%|█████████▍| 211581/225000 [409:31:44<36:10:45,  9.71s/it]
 94%|█████████▍| 211582/225000 [409:31:56<37:49:58, 10.15s/it]
 94%|█████████▍| 211583/225000 [409:32:01<32:56:35,  8.84s/it]
 94%|█████████▍| 211584/225000 [409:32:10<32:14:49,  8.65s/it]
 94%|█████████▍| 211585/225000 [409:32:23<38:06:03, 10.22s/it]
 94%|█████████▍| 211586/225000 [409:32:28<31:22:34,  8.42s/it]
 94%|█████████▍| 211587/225000 [409:32:33<28:18:51,  7.60s/it]
 94%|█████████▍| 211588/225000 [409:32:39<25:54:37,  6.95s/it]
 94%|█████████▍| 211589/225000 [409:32:47<26:46:33,  7.19s/it]
 94%|█████████▍| 211590/225000 [409:32:54<26:51:41,  7.21s/it]
 94%|█████████▍| 211591/225000 [409:33:06<32:03:14,  8.61s/it]
 94%|█████████▍| 211592/225000 [409:33:10<27:27:24,  7.37s/it]
 94%|█████████▍| 211593/225000 [409:33:22<32:05:45,  8.62s/it]
 94%|█████████▍| 211594/225000 [409:33:28<29:46:28,  8.00s/it]
 94%|█████████▍| 211595/225000 [409:33:34<27:43:31,  7.45s/it]
 94%|█████████▍| 211596/225000 [409:33:44<30:31:26,  8.20s/it]
 94%|█████████▍| 211597/225000 [409:33:54<32:15:16,  8.66s/it]
 94%|█████████▍| 211598/225000 [409:34:01<30:03:41,  8.07s/it]
 94%|█████████▍| 211599/225000 [409:34:08<29:15:46,  7.86s/it]
 94%|█████████▍| 211600/225000 [409:34:18<31:30:54,  8.47s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.211', 'learning_rate': '4.848e-07', 'epoch': '0.9404', 'num_input_tokens_seen': 693526640, 'train_runtime': '1.474e+06', 'train_tokens_per_second': '470.4'}

 94%|█████████▍| 211600/225000 [409:34:18<31:30:54,  8.47s/it]
 94%|█████████▍| 211601/225000 [409:34:28<33:43:39,  9.06s/it]
 94%|█████████▍| 211602/225000 [409:34:39<35:46:11,  9.61s/it]
 94%|█████████▍| 211603/225000 [409:34:48<34:32:36,  9.28s/it]
 94%|█████████▍| 211604/225000 [409:34:54<30:29:29,  8.19s/it]
 94%|█████████▍| 211605/225000 [409:35:02<30:34:29,  8.22s/it]
 94%|█████████▍| 211606/225000 [409:35:10<30:53:18,  8.30s/it]
 94%|█████████▍| 211607/225000 [409:35:25<38:15:31, 10.28s/it]
 94%|█████████▍| 211608/225000 [409:35:35<37:11:44, 10.00s/it]
 94%|█████████▍| 211609/225000 [409:35:40<32:36:14,  8.77s/it]
 94%|█████████▍| 211610/225000 [409:35:47<30:39:18,  8.24s/it]
 94%|█████████▍| 211611/225000 [409:36:01<36:14:05,  9.74s/it]
 94%|█████████▍| 211612/225000 [409:36:05<30:20:02,  8.16s/it]
 94%|█████████▍| 211613/225000 [409:36:15<32:17:23,  8.68s/it]
 94%|█████████▍| 211614/225000 [409:36:23<31:15:21,  8.41s/it]
 94%|█████████▍| 211615/225000 [409:36:28<28:03:51,  7.55s/it]
 94%|█████████▍| 211616/225000 [409:36:40<32:06:47,  8.64s/it]
 94%|█████████▍| 211617/225000 [409:36:45<28:06:24,  7.56s/it]
 94%|█████████▍| 211618/225000 [409:36:51<26:46:35,  7.20s/it]
 94%|█████████▍| 211619/225000 [409:37:03<31:39:41,  8.52s/it]
 94%|█████████▍| 211620/225000 [409:37:13<33:50:04,  9.10s/it]
 94%|█████████▍| 211621/225000 [409:37:23<35:08:04,  9.45s/it]
 94%|█████████▍| 211622/225000 [409:37:32<34:00:11,  9.15s/it]
 94%|█████████▍| 211623/225000 [409:37:40<32:53:16,  8.85s/it]
 94%|█████████▍| 211624/225000 [409:37:52<36:43:16,  9.88s/it]
 94%|█████████▍| 211625/225000 [409:38:02<36:24:26,  9.80s/it]
 94%|█████████▍| 211626/225000 [409:38:07<31:44:27,  8.54s/it]
 94%|█████████▍| 211627/225000 [409:38:15<30:50:11,  8.30s/it]
 94%|█████████▍| 211628/225000 [409:38:22<28:42:17,  7.73s/it]
 94%|█████████▍| 211629/225000 [409:38:35<34:45:35,  9.36s/it]
 94%|█████████▍| 211630/225000 [409:38:42<32:33:34,  8.77s/it]
 94%|█████████▍| 211631/225000 [409:38:52<33:22:18,  8.99s/it]
 94%|█████████▍| 211632/225000 [409:38:58<30:22:37,  8.18s/it]
 94%|█████████▍| 211633/225000 [409:39:10<34:18:09,  9.24s/it]
 94%|█████████▍| 211634/225000 [409:39:21<36:36:11,  9.86s/it]
 94%|█████████▍| 211635/225000 [409:39:26<31:32:22,  8.50s/it]
 94%|█████████▍| 211636/225000 [409:39:36<32:51:59,  8.85s/it]
 94%|█████████▍| 211637/225000 [409:39:43<31:12:02,  8.41s/it]
 94%|█████████▍| 211638/225000 [409:39:51<30:09:04,  8.12s/it]
 94%|█████████▍| 211639/225000 [409:40:00<31:46:29,  8.56s/it]
 94%|█████████▍| 211640/225000 [409:40:10<33:18:58,  8.98s/it]
 94%|█████████▍| 211641/225000 [409:40:16<29:32:06,  7.96s/it]
 94%|█████████▍| 211642/225000 [409:40:23<28:39:27,  7.72s/it]
 94%|█████████▍| 211643/225000 [409:40:29<26:57:35,  7.27s/it]
 94%|█████████▍| 211644/225000 [409:40:41<32:07:40,  8.66s/it]
 94%|█████████▍| 211645/225000 [409:40:47<29:07:25,  7.85s/it]
 94%|█████████▍| 211646/225000 [409:40:57<31:02:41,  8.37s/it]
 94%|█████████▍| 211647/225000 [409:41:06<32:34:18,  8.78s/it]
 94%|█████████▍| 211648/225000 [409:41:14<31:18:56,  8.44s/it]
 94%|█████████▍| 211649/225000 [409:41:20<28:26:18,  7.67s/it]
 94%|█████████▍| 211650/225000 [409:41:32<33:30:03,  9.03s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.18', 'learning_rate': '4.812e-07', 'epoch': '0.9407', 'num_input_tokens_seen': 693759248, 'train_runtime': '1.475e+06', 'train_tokens_per_second': '470.4'}

 94%|█████████▍| 211650/225000 [409:41:32<33:30:03,  9.03s/it]
 94%|█████████▍| 211651/225000 [409:41:39<30:46:04,  8.30s/it]
 94%|█████████▍| 211652/225000 [409:41:46<29:56:01,  8.07s/it]
 94%|█████████▍| 211653/225000 [409:41:52<27:04:48,  7.30s/it]
 94%|█████████▍| 211654/225000 [409:41:58<26:10:00,  7.06s/it]
 94%|█████████▍| 211655/225000 [409:42:09<30:11:57,  8.15s/it]
 94%|█████████▍| 211656/225000 [409:42:23<36:57:07,  9.97s/it]
 94%|█████████▍| 211657/225000 [409:42:32<35:36:55,  9.61s/it]
 94%|█████████▍| 211658/225000 [409:42:39<32:47:51,  8.85s/it]
 94%|█████████▍| 211659/225000 [409:42:50<35:41:14,  9.63s/it]
 94%|█████████▍| 211660/225000 [409:43:00<35:42:38,  9.64s/it]
 94%|█████████▍| 211661/225000 [409:43:08<34:10:27,  9.22s/it]
 94%|█████████▍| 211662/225000 [409:43:15<31:37:46,  8.54s/it]
 94%|█████████▍| 211663/225000 [409:43:21<28:58:28,  7.82s/it]
 94%|█████████▍| 211664/225000 [409:43:31<30:29:18,  8.23s/it]
 94%|█████████▍| 211665/225000 [409:43:41<32:24:30,  8.75s/it]
 94%|█████████▍| 211666/225000 [409:43:51<34:05:20,  9.20s/it]
 94%|█████████▍| 211667/225000 [409:43:57<30:51:53,  8.33s/it]
 94%|█████████▍| 211668/225000 [409:44:06<31:55:22,  8.62s/it]
 94%|█████████▍| 211669/225000 [409:44:15<31:37:27,  8.54s/it]
 94%|█████████▍| 211670/225000 [409:44:25<32:53:59,  8.89s/it]
 94%|█████████▍| 211671/225000 [409:44:33<32:09:33,  8.69s/it]
 94%|█████████▍| 211672/225000 [409:44:44<35:04:00,  9.47s/it]
 94%|█████████▍| 211673/225000 [409:44:52<33:50:09,  9.14s/it]
 94%|█████████▍| 211674/225000 [409:45:01<32:52:49,  8.88s/it]
 94%|█████████▍| 211675/225000 [409:45:09<31:57:44,  8.64s/it]
 94%|█████████▍| 211676/225000 [409:45:23<38:26:59, 10.39s/it]
 94%|█████████▍| 211677/225000 [409:45:30<34:05:00,  9.21s/it]
 94%|█████████▍| 211678/225000 [409:45:36<31:19:55,  8.47s/it]
 94%|█████████▍| 211679/225000 [409:45:42<27:50:58,  7.53s/it]
 94%|█████████▍| 211680/225000 [409:45:54<33:32:25,  9.06s/it]
 94%|█████████▍| 211681/225000 [409:46:03<33:03:56,  8.94s/it]
 94%|█████████▍| 211682/225000 [409:46:10<31:19:53,  8.47s/it]
 94%|█████████▍| 211683/225000 [409:46:17<28:45:51,  7.78s/it]
 94%|█████████▍| 211684/225000 [409:46:29<33:25:00,  9.03s/it]
 94%|█████████▍| 211685/225000 [409:46:34<29:01:16,  7.85s/it]
 94%|█████████▍| 211686/225000 [409:46:41<28:30:12,  7.71s/it]
 94%|█████████▍| 211687/225000 [409:46:51<31:31:57,  8.53s/it]
 94%|█████████▍| 211688/225000 [409:46:59<30:04:44,  8.13s/it]
 94%|█████████▍| 211689/225000 [409:47:09<32:37:27,  8.82s/it]
 94%|█████████▍| 211690/225000 [409:47:18<32:10:40,  8.70s/it]
 94%|█████████▍| 211691/225000 [409:47:25<30:21:59,  8.21s/it]
 94%|█████████▍| 211692/225000 [409:47:34<31:43:05,  8.58s/it]
 94%|█████████▍| 211693/225000 [409:47:40<28:54:22,  7.82s/it]
 94%|█████████▍| 211694/225000 [409:47:48<28:40:34,  7.76s/it]
 94%|█████████▍| 211695/225000 [409:48:01<35:14:44,  9.54s/it]
 94%|█████████▍| 211696/225000 [409:48:07<31:02:47,  8.40s/it]
 94%|█████████▍| 211697/225000 [409:48:16<31:03:20,  8.40s/it]
 94%|█████████▍| 211698/225000 [409:48:24<31:34:40,  8.55s/it]
 94%|█████████▍| 211699/225000 [409:48:36<34:53:15,  9.44s/it]
 94%|█████████▍| 211700/225000 [409:48:40<29:25:28,  7.96s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.316', 'learning_rate': '4.776e-07', 'epoch': '0.9409', 'num_input_tokens_seen': 693988752, 'train_runtime': '1.475e+06', 'train_tokens_per_second': '470.4'}

 94%|█████████▍| 211700/225000 [409:48:40<29:25:28,  7.96s/it]
 94%|█████████▍| 211701/225000 [409:48:56<38:16:38, 10.36s/it]
 94%|█████████▍| 211702/225000 [409:49:05<36:32:52,  9.89s/it]
 94%|█████████▍| 211703/225000 [409:49:12<32:46:57,  8.88s/it]
 94%|█████████▍| 211704/225000 [409:49:20<31:56:57,  8.65s/it]
 94%|█████████▍| 211705/225000 [409:49:27<30:35:51,  8.29s/it]
 94%|█████████▍| 211706/225000 [409:49:35<29:47:04,  8.07s/it]
 94%|█████████▍| 211707/225000 [409:49:49<36:39:01,  9.93s/it]
 94%|█████████▍| 211708/225000 [409:49:55<32:23:49,  8.77s/it]
 94%|█████████▍| 211709/225000 [409:50:05<33:10:25,  8.99s/it]
 94%|█████████▍| 211710/225000 [409:50:13<32:25:04,  8.78s/it]
 94%|█████████▍| 211711/225000 [409:50:25<35:31:13,  9.62s/it]
 94%|█████████▍| 211712/225000 [409:50:38<39:17:11, 10.64s/it]
 94%|█████████▍| 211713/225000 [409:50:43<33:00:44,  8.94s/it]
 94%|█████████▍| 211714/225000 [409:50:51<32:19:13,  8.76s/it]
 94%|█████████▍| 211715/225000 [409:51:00<33:12:02,  9.00s/it]
 94%|█████████▍| 211716/225000 [409:51:06<29:03:58,  7.88s/it]
 94%|█████████▍| 211717/225000 [409:51:12<27:30:23,  7.45s/it]
 94%|█████████▍| 211718/225000 [409:51:23<31:10:00,  8.45s/it]
 94%|█████████▍| 211719/225000 [409:51:28<27:39:34,  7.50s/it]
 94%|█████████▍| 211720/225000 [409:51:39<31:20:10,  8.49s/it]
 94%|█████████▍| 211721/225000 [409:51:47<30:15:14,  8.20s/it]
 94%|█████████▍| 211722/225000 [409:51:52<27:07:02,  7.35s/it]
 94%|█████████▍| 211723/225000 [409:52:05<33:09:46,  8.99s/it]
 94%|█████████▍| 211724/225000 [409:52:15<34:16:41,  9.30s/it]
 94%|█████████▍| 211725/225000 [409:52:21<31:06:42,  8.44s/it]
 94%|█████████▍| 211726/225000 [409:52:26<27:18:51,  7.41s/it]
 94%|█████████▍| 211727/225000 [409:52:33<26:31:17,  7.19s/it]
 94%|█████████▍| 211728/225000 [409:52:46<33:05:32,  8.98s/it]
 94%|█████████▍| 211729/225000 [409:52:53<31:22:24,  8.51s/it]
 94%|█████████▍| 211730/225000 [409:53:04<34:07:51,  9.26s/it]
 94%|█████████▍| 211731/225000 [409:53:17<37:23:58, 10.15s/it]
 94%|█████████▍| 211732/225000 [409:53:26<36:54:19, 10.01s/it]
 94%|█████████▍| 211733/225000 [409:53:37<37:17:54, 10.12s/it]
 94%|█████████▍| 211734/225000 [409:53:43<32:59:55,  8.95s/it]
 94%|█████████▍| 211735/225000 [409:53:56<37:04:18, 10.06s/it]
 94%|█████████▍| 211736/225000 [409:54:04<35:05:42,  9.53s/it]
 94%|█████████▍| 211737/225000 [409:54:11<32:31:09,  8.83s/it]
 94%|█████████▍| 211738/225000 [409:54:18<30:42:18,  8.33s/it]
 94%|█████████▍| 211739/225000 [409:54:29<33:51:09,  9.19s/it]
 94%|█████████▍| 211740/225000 [409:54:38<33:27:17,  9.08s/it]
 94%|█████████▍| 211741/225000 [409:54:45<30:25:07,  8.26s/it]
 94%|█████████▍| 211742/225000 [409:54:51<28:34:15,  7.76s/it]
 94%|█████████▍| 211743/225000 [409:55:01<30:55:41,  8.40s/it]
 94%|█████████▍| 211744/225000 [409:55:12<33:59:30,  9.23s/it]
 94%|█████████▍| 211745/225000 [409:55:20<32:04:32,  8.71s/it]
 94%|█████████▍| 211746/225000 [409:55:33<36:52:26, 10.02s/it]
 94%|█████████▍| 211747/225000 [409:55:39<32:54:58,  8.94s/it]
 94%|█████████▍| 211748/225000 [409:55:47<31:59:35,  8.69s/it]
 94%|█████████▍| 211749/225000 [409:55:56<31:49:23,  8.65s/it]
 94%|█████████▍| 211750/225000 [409:56:05<31:58:31,  8.69s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.184', 'learning_rate': '4.741e-07', 'epoch': '0.9411', 'num_input_tokens_seen': 694224704, 'train_runtime': '1.476e+06', 'train_tokens_per_second': '470.4'}

 94%|█████████▍| 211750/225000 [409:56:05<31:58:31,  8.69s/it]
 94%|█████████▍| 211751/225000 [409:56:17<35:28:08,  9.64s/it]
 94%|█████████▍| 211752/225000 [409:56:22<31:18:37,  8.51s/it]
 94%|█████████▍| 211753/225000 [409:56:31<31:28:13,  8.55s/it]
 94%|█████████▍| 211754/225000 [409:56:43<35:30:14,  9.65s/it]
 94%|█████████▍| 211755/225000 [409:56:48<30:32:01,  8.30s/it]
 94%|█████████▍| 211756/225000 [409:56:54<27:55:48,  7.59s/it]
 94%|█████████▍| 211757/225000 [409:57:01<26:35:52,  7.23s/it]
 94%|█████████▍| 211758/225000 [409:57:14<32:42:05,  8.89s/it]
 94%|█████████▍| 211759/225000 [409:57:24<34:06:47,  9.27s/it]
 94%|█████████▍| 211760/225000 [409:57:29<30:08:26,  8.20s/it]
 94%|█████████▍| 211761/225000 [409:57:36<28:08:40,  7.65s/it]
 94%|█████████▍| 211762/225000 [409:57:44<28:15:09,  7.68s/it]
 94%|█████████▍| 211763/225000 [409:57:49<25:40:44,  6.98s/it]
 94%|█████████▍| 211764/225000 [409:57:55<24:33:00,  6.68s/it]
 94%|█████████▍| 211765/225000 [409:58:09<32:47:47,  8.92s/it]
 94%|█████████▍| 211766/225000 [409:58:14<28:48:52,  7.84s/it]
 94%|█████████▍| 211767/225000 [409:58:20<26:14:22,  7.14s/it]
 94%|█████████▍| 211768/225000 [409:58:32<31:37:23,  8.60s/it]
 94%|█████████▍| 211769/225000 [409:58:39<29:35:32,  8.05s/it]
 94%|█████████▍| 211770/225000 [409:58:45<27:48:49,  7.57s/it]
 94%|█████████▍| 211771/225000 [409:58:49<24:22:45,  6.63s/it]
 94%|█████████▍| 211772/225000 [409:59:02<30:34:15,  8.32s/it]
 94%|█████████▍| 211773/225000 [409:59:07<26:43:36,  7.27s/it]
 94%|█████████▍| 211774/225000 [409:59:16<29:29:58,  8.03s/it]
 94%|█████████▍| 211775/225000 [409:59:22<26:34:07,  7.23s/it]
 94%|█████████▍| 211776/225000 [409:59:32<29:26:57,  8.02s/it]
 94%|█████████▍| 211777/225000 [409:59:40<30:08:58,  8.21s/it]
 94%|█████████▍| 211778/225000 [409:59:51<32:32:50,  8.86s/it]
 94%|█████████▍| 211779/225000 [409:59:57<30:09:03,  8.21s/it]
 94%|█████████▍| 211780/225000 [410:00:10<34:53:05,  9.50s/it]
 94%|█████████▍| 211781/225000 [410:00:20<35:45:43,  9.74s/it]
 94%|█████████▍| 211782/225000 [410:00:25<30:10:00,  8.22s/it]
 94%|█████████▍| 211783/225000 [410:00:34<31:35:08,  8.60s/it]
 94%|█████████▍| 211784/225000 [410:00:42<30:31:38,  8.32s/it]
 94%|█████████▍| 211785/225000 [410:00:52<31:58:11,  8.71s/it]
 94%|█████████▍| 211786/225000 [410:01:01<32:40:22,  8.90s/it]
 94%|█████████▍| 211787/225000 [410:01:07<29:15:13,  7.97s/it]
 94%|█████████▍| 211788/225000 [410:01:12<26:44:54,  7.29s/it]
 94%|█████████▍| 211789/225000 [410:01:18<24:51:48,  6.78s/it]
 94%|█████████▍| 211790/225000 [410:01:24<24:26:41,  6.66s/it]
 94%|█████████▍| 211791/225000 [410:01:38<32:20:27,  8.81s/it]
 94%|█████████▍| 211792/225000 [410:01:45<30:05:04,  8.20s/it]
 94%|█████████▍| 211793/225000 [410:01:52<28:21:22,  7.73s/it]
 94%|█████████▍| 211794/225000 [410:02:02<30:45:34,  8.39s/it]
 94%|█████████▍| 211795/225000 [410:02:07<28:05:03,  7.66s/it]
 94%|█████████▍| 211796/225000 [410:02:15<27:40:28,  7.55s/it]
 94%|█████████▍| 211797/225000 [410:02:22<27:29:26,  7.50s/it]
 94%|█████████▍| 211798/225000 [410:02:30<27:45:05,  7.57s/it]
 94%|█████████▍| 211799/225000 [410:02:44<34:29:59,  9.41s/it]
 94%|█████████▍| 211800/225000 [410:02:49<30:17:50,  8.26s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.558', 'learning_rate': '4.705e-07', 'epoch': '0.9413', 'num_input_tokens_seen': 694446592, 'train_runtime': '1.476e+06', 'train_tokens_per_second': '470.4'}

 94%|█████████▍| 211800/225000 [410:02:49<30:17:50,  8.26s/it]
 94%|█████████▍| 211801/225000 [410:03:00<32:54:44,  8.98s/it]
 94%|█████████▍| 211802/225000 [410:03:10<34:25:33,  9.39s/it]
 94%|█████████▍| 211803/225000 [410:03:17<31:09:11,  8.50s/it]
 94%|█████████▍| 211804/225000 [410:03:23<29:00:00,  7.91s/it]
 94%|█████████▍| 211805/225000 [410:03:33<31:34:57,  8.62s/it]
 94%|█████████▍| 211806/225000 [410:03:47<37:10:51, 10.14s/it]
 94%|█████████▍| 211807/225000 [410:03:53<32:32:28,  8.88s/it]
 94%|█████████▍| 211808/225000 [410:04:00<30:16:19,  8.26s/it]
 94%|█████████▍| 211809/225000 [410:04:05<26:17:17,  7.17s/it]
 94%|█████████▍| 211810/225000 [410:04:17<31:41:11,  8.65s/it]
 94%|█████████▍| 211811/225000 [410:04:26<32:22:21,  8.84s/it]
 94%|█████████▍| 211812/225000 [410:04:32<28:59:09,  7.91s/it]
 94%|█████████▍| 211813/225000 [410:04:41<30:36:44,  8.36s/it]
 94%|█████████▍| 211814/225000 [410:04:52<33:49:53,  9.24s/it]
 94%|█████████▍| 211815/225000 [410:05:01<33:40:09,  9.19s/it]
 94%|█████████▍| 211816/225000 [410:05:08<30:51:42,  8.43s/it]
 94%|█████████▍| 211817/225000 [410:05:15<28:43:30,  7.84s/it]
 94%|█████████▍| 211818/225000 [410:05:21<27:06:56,  7.41s/it]
 94%|█████████▍| 211819/225000 [410:05:34<33:08:51,  9.05s/it]
 94%|█████████▍| 211820/225000 [410:05:41<31:32:37,  8.62s/it]
 94%|█████████▍| 211821/225000 [410:05:51<32:56:02,  9.00s/it]
 94%|█████████▍| 211822/225000 [410:05:58<30:34:05,  8.35s/it]
 94%|█████████▍| 211823/225000 [410:06:06<30:12:11,  8.25s/it]
 94%|█████████▍| 211824/225000 [410:06:15<31:23:18,  8.58s/it]
 94%|█████████▍| 211825/225000 [410:06:23<29:52:37,  8.16s/it]
 94%|█████████▍| 211826/225000 [410:06:31<30:20:15,  8.29s/it]
 94%|█████████▍| 211827/225000 [410:06:40<30:57:29,  8.46s/it]
 94%|█████████▍| 211828/225000 [410:06:47<29:03:45,  7.94s/it]
 94%|█████████▍| 211829/225000 [410:06:53<27:37:21,  7.55s/it]
 94%|█████████▍| 211830/225000 [410:07:01<27:54:03,  7.63s/it]
 94%|█████████▍| 211831/225000 [410:07:07<26:07:46,  7.14s/it]
 94%|█████████▍| 211832/225000 [410:07:21<33:38:06,  9.20s/it]
 94%|█████████▍| 211833/225000 [410:07:27<30:16:32,  8.28s/it]
 94%|█████████▍| 211834/225000 [410:07:35<29:40:37,  8.11s/it]
 94%|█████████▍| 211835/225000 [410:07:42<28:08:08,  7.69s/it]
 94%|█████████▍| 211836/225000 [410:07:51<29:39:19,  8.11s/it]
 94%|█████████▍| 211837/225000 [410:07:59<29:18:36,  8.02s/it]
 94%|█████████▍| 211838/225000 [410:08:10<33:13:33,  9.09s/it]
 94%|█████████▍| 211839/225000 [410:08:15<28:52:55,  7.90s/it]
 94%|█████████▍| 211840/225000 [410:08:22<27:15:10,  7.46s/it]
 94%|█████████▍| 211841/225000 [410:08:33<31:05:06,  8.50s/it]
 94%|█████████▍| 211842/225000 [410:08:42<31:59:24,  8.75s/it]
 94%|█████████▍| 211843/225000 [410:08:52<32:43:25,  8.95s/it]
 94%|█████████▍| 211844/225000 [410:09:02<34:22:19,  9.41s/it]
 94%|█████████▍| 211845/225000 [410:09:07<29:02:45,  7.95s/it]
 94%|█████████▍| 211846/225000 [410:09:18<32:28:31,  8.89s/it]
 94%|█████████▍| 211847/225000 [410:09:24<29:23:53,  8.05s/it]
 94%|█████████▍| 211848/225000 [410:09:31<28:51:19,  7.90s/it]
 94%|█████████▍| 211849/225000 [410:09:37<26:45:32,  7.33s/it]
 94%|█████████▍| 211850/225000 [410:09:49<31:54:39,  8.74s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.635', 'learning_rate': '4.67e-07', 'epoch': '0.9416', 'num_input_tokens_seen': 694676848, 'train_runtime': '1.477e+06', 'train_tokens_per_second': '470.5'}

 94%|█████████▍| 211850/225000 [410:09:49<31:54:39,  8.74s/it]
 94%|█████████▍| 211851/225000 [410:09:56<29:33:45,  8.09s/it]
 94%|█████████▍| 211852/225000 [410:10:02<27:48:27,  7.61s/it]
 94%|█████████▍| 211853/225000 [410:10:12<30:26:29,  8.34s/it]
 94%|█████████▍| 211854/225000 [410:10:22<31:47:18,  8.71s/it]
 94%|█████████▍| 211855/225000 [410:10:30<30:35:56,  8.38s/it]
 94%|█████████▍| 211856/225000 [410:10:43<35:46:34,  9.80s/it]
 94%|█████████▍| 211857/225000 [410:10:52<35:30:25,  9.73s/it]
 94%|█████████▍| 211858/225000 [410:11:00<32:56:06,  9.02s/it]
 94%|█████████▍| 211859/225000 [410:11:17<41:35:34, 11.39s/it]
 94%|█████████▍| 211860/225000 [410:11:24<37:05:11, 10.16s/it]
 94%|█████████▍| 211861/225000 [410:11:34<36:44:38, 10.07s/it]
 94%|█████████▍| 211862/225000 [410:11:42<34:34:36,  9.47s/it]
 94%|█████████▍| 211863/225000 [410:11:53<35:52:47,  9.83s/it]
 94%|█████████▍| 211864/225000 [410:11:59<32:26:23,  8.89s/it]
 94%|█████████▍| 211865/225000 [410:12:06<29:40:51,  8.13s/it]
 94%|█████████▍| 211866/225000 [410:12:14<29:43:09,  8.15s/it]
 94%|█████████▍| 211867/225000 [410:12:24<32:15:27,  8.84s/it]
 94%|█████████▍| 211868/225000 [410:12:32<30:59:20,  8.50s/it]
 94%|█████████▍| 211869/225000 [410:12:38<28:10:11,  7.72s/it]
 94%|█████████▍| 211870/225000 [410:12:48<31:04:17,  8.52s/it]
 94%|█████████▍| 211871/225000 [410:12:55<28:39:41,  7.86s/it]
 94%|█████████▍| 211872/225000 [410:13:06<32:29:10,  8.91s/it]
 94%|█████████▍| 211873/225000 [410:13:12<29:44:48,  8.16s/it]
 94%|█████████▍| 211874/225000 [410:13:22<31:09:09,  8.54s/it]
 94%|█████████▍| 211875/225000 [410:13:28<29:05:27,  7.98s/it]
 94%|█████████▍| 211876/225000 [410:13:35<27:31:42,  7.55s/it]
 94%|█████████▍| 211877/225000 [410:13:49<34:23:25,  9.43s/it]
 94%|█████████▍| 211878/225000 [410:13:57<32:34:24,  8.94s/it]
 94%|█████████▍| 211879/225000 [410:14:02<29:18:41,  8.04s/it]
 94%|█████████▍| 211880/225000 [410:14:17<36:40:04, 10.06s/it]
 94%|█████████▍| 211881/225000 [410:14:24<33:29:00,  9.19s/it]
 94%|█████████▍| 211882/225000 [410:14:33<32:56:45,  9.04s/it]
 94%|█████████▍| 211883/225000 [410:14:38<28:25:36,  7.80s/it]
 94%|█████████▍| 211884/225000 [410:14:46<28:47:10,  7.90s/it]
 94%|█████████▍| 211885/225000 [410:14:57<32:27:28,  8.91s/it]
 94%|█████████▍| 211886/225000 [410:15:05<30:27:50,  8.36s/it]
 94%|█████████▍| 211887/225000 [410:15:13<30:57:40,  8.50s/it]
 94%|█████████▍| 211888/225000 [410:15:19<27:25:18,  7.53s/it]
 94%|█████████▍| 211889/225000 [410:15:28<29:51:52,  8.20s/it]
 94%|█████████▍| 211890/225000 [410:15:36<29:18:10,  8.05s/it]
 94%|█████████▍| 211891/225000 [410:15:42<27:01:28,  7.42s/it]
 94%|█████████▍| 211892/225000 [410:15:53<30:52:04,  8.48s/it]
 94%|█████████▍| 211893/225000 [410:16:05<34:23:21,  9.45s/it]
 94%|█████████▍| 211894/225000 [410:16:11<30:49:17,  8.47s/it]
 94%|█████████▍| 211895/225000 [410:16:18<29:54:18,  8.22s/it]
 94%|█████████▍| 211896/225000 [410:16:28<31:51:33,  8.75s/it]
 94%|█████████▍| 211897/225000 [410:16:37<31:53:38,  8.76s/it]
 94%|█████████▍| 211898/225000 [410:16:43<28:19:45,  7.78s/it]
 94%|█████████▍| 211899/225000 [410:16:50<27:27:12,  7.54s/it]
 94%|█████████▍| 211900/225000 [410:16:58<28:25:26,  7.81s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.095', 'learning_rate': '4.635e-07', 'epoch': '0.9418', 'num_input_tokens_seen': 694906792, 'train_runtime': '1.477e+06', 'train_tokens_per_second': '470.5'}

 94%|█████████▍| 211900/225000 [410:16:58<28:25:26,  7.81s/it]
 94%|█████████▍| 211901/225000 [410:17:05<27:11:51,  7.47s/it]
 94%|█████████▍| 211902/225000 [410:17:16<30:56:48,  8.51s/it]
 94%|█████████▍| 211903/225000 [410:17:20<26:20:05,  7.24s/it]
 94%|█████████▍| 211904/225000 [410:17:30<29:18:58,  8.06s/it]
 94%|█████████▍| 211905/225000 [410:17:37<28:28:05,  7.83s/it]
 94%|█████████▍| 211906/225000 [410:17:47<31:01:47,  8.53s/it]
 94%|█████████▍| 211907/225000 [410:17:57<31:57:30,  8.79s/it]
 94%|█████████▍| 211908/225000 [410:18:06<32:52:03,  9.04s/it]
 94%|█████████▍| 211909/225000 [410:18:16<33:04:12,  9.09s/it]
 94%|█████████▍| 211910/225000 [410:18:24<31:57:11,  8.79s/it]
 94%|█████████▍| 211911/225000 [410:18:31<30:12:52,  8.31s/it]
 94%|█████████▍| 211912/225000 [410:18:39<29:39:11,  8.16s/it]
 94%|█████████▍| 211913/225000 [410:18:49<32:14:04,  8.87s/it]
 94%|█████████▍| 211914/225000 [410:19:00<33:51:26,  9.31s/it]
 94%|█████████▍| 211915/225000 [410:19:09<34:07:54,  9.39s/it]
 94%|█████████▍| 211916/225000 [410:19:19<34:13:07,  9.42s/it]
 94%|█████████▍| 211917/225000 [410:19:26<32:25:52,  8.92s/it]
 94%|█████████▍| 211918/225000 [410:19:38<35:37:08,  9.80s/it]
 94%|█████████▍| 211919/225000 [410:19:47<34:35:39,  9.52s/it]
 94%|█████████▍| 211920/225000 [410:19:56<33:42:43,  9.28s/it]
 94%|█████████▍| 211921/225000 [410:20:04<32:35:22,  8.97s/it]
 94%|█████████▍| 211922/225000 [410:20:09<28:24:31,  7.82s/it]
 94%|█████████▍| 211923/225000 [410:20:23<35:17:22,  9.71s/it]
 94%|█████████▍| 211924/225000 [410:20:33<34:38:07,  9.54s/it]
 94%|█████████▍| 211925/225000 [410:20:39<31:32:56,  8.69s/it]
 94%|█████████▍| 211926/225000 [410:20:45<28:34:04,  7.87s/it]
 94%|█████████▍| 211927/225000 [410:20:51<26:16:01,  7.23s/it]
 94%|█████████▍| 211928/225000 [410:20:58<26:25:06,  7.28s/it]
 94%|█████████▍| 211929/225000 [410:21:09<30:32:18,  8.41s/it]
 94%|█████████▍| 211930/225000 [410:21:18<31:06:10,  8.57s/it]
 94%|█████████▍| 211931/225000 [410:21:23<27:06:23,  7.47s/it]
 94%|█████████▍| 211932/225000 [410:21:32<29:03:06,  8.00s/it]
 94%|█████████▍| 211933/225000 [410:21:45<33:43:09,  9.29s/it]
 94%|█████████▍| 211934/225000 [410:21:53<32:43:04,  9.01s/it]
 94%|█████████▍| 211935/225000 [410:22:04<34:11:26,  9.42s/it]
 94%|█████████▍| 211936/225000 [410:22:13<34:25:37,  9.49s/it]
 94%|█████████▍| 211937/225000 [410:22:19<30:20:26,  8.36s/it]
 94%|█████████▍| 211938/225000 [410:22:27<30:05:02,  8.29s/it]
 94%|█████████▍| 211939/225000 [410:22:41<36:02:43,  9.94s/it]
 94%|█████████▍| 211940/225000 [410:22:49<33:56:40,  9.36s/it]
 94%|█████████▍| 211941/225000 [410:22:55<30:04:32,  8.29s/it]
 94%|█████████▍| 211942/225000 [410:23:08<35:45:46,  9.86s/it]
 94%|█████████▍| 211943/225000 [410:23:18<36:15:25, 10.00s/it]
 94%|█████████▍| 211944/225000 [410:23:26<33:08:47,  9.14s/it]
 94%|█████████▍| 211945/225000 [410:23:32<30:12:35,  8.33s/it]
 94%|█████████▍| 211946/225000 [410:23:47<37:11:35, 10.26s/it]
 94%|█████████▍| 211947/225000 [410:23:54<33:37:22,  9.27s/it]
 94%|█████████▍| 211948/225000 [410:23:59<28:49:27,  7.95s/it]
 94%|█████████▍| 211949/225000 [410:24:10<32:58:49,  9.10s/it]
 94%|█████████▍| 211950/225000 [410:24:21<35:00:18,  9.66s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.141', 'learning_rate': '4.6e-07', 'epoch': '0.942', 'num_input_tokens_seen': 695142872, 'train_runtime': '1.477e+06', 'train_tokens_per_second': '470.5'}

 94%|█████████▍| 211950/225000 [410:24:21<35:00:18,  9.66s/it]
 94%|█████████▍| 211951/225000 [410:24:26<30:02:49,  8.29s/it]
 94%|█████████▍| 211952/225000 [410:24:33<28:10:25,  7.77s/it]
 94%|█████████▍| 211953/225000 [410:24:39<26:07:45,  7.21s/it]
 94%|█████████▍| 211954/225000 [410:24:47<26:57:41,  7.44s/it]
 94%|█████████▍| 211955/225000 [410:24:59<32:04:44,  8.85s/it]
 94%|█████████▍| 211956/225000 [410:25:06<30:10:08,  8.33s/it]
 94%|█████████▍| 211957/225000 [410:25:14<29:42:57,  8.20s/it]
 94%|█████████▍| 211958/225000 [410:25:27<34:32:20,  9.53s/it]
 94%|█████████▍| 211959/225000 [410:25:35<32:41:02,  9.02s/it]
 94%|█████████▍| 211960/225000 [410:25:38<26:56:02,  7.44s/it]
 94%|█████████▍| 211961/225000 [410:25:52<34:02:03,  9.40s/it]
 94%|█████████▍| 211962/225000 [410:26:01<32:48:51,  9.06s/it]
 94%|█████████▍| 211963/225000 [410:26:11<34:03:45,  9.41s/it]
 94%|█████████▍| 211964/225000 [410:26:19<32:38:43,  9.02s/it]
 94%|█████████▍| 211965/225000 [410:26:25<29:31:27,  8.15s/it]
 94%|█████████▍| 211966/225000 [410:26:32<28:23:40,  7.84s/it]
 94%|█████████▍| 211967/225000 [410:26:46<34:28:55,  9.52s/it]
 94%|█████████▍| 211968/225000 [410:26:51<30:30:53,  8.43s/it]
 94%|█████████▍| 211969/225000 [410:27:01<32:07:14,  8.87s/it]
 94%|█████████▍| 211970/225000 [410:27:14<36:25:55, 10.07s/it]
 94%|█████████▍| 211971/225000 [410:27:22<33:53:39,  9.37s/it]
 94%|█████████▍| 211972/225000 [410:27:29<31:23:25,  8.67s/it]
 94%|█████████▍| 211973/225000 [410:27:37<30:45:05,  8.50s/it]
 94%|█████████▍| 211974/225000 [410:27:45<29:49:28,  8.24s/it]
 94%|█████████▍| 211975/225000 [410:27:55<32:01:31,  8.85s/it]
 94%|█████████▍| 211976/225000 [410:28:02<30:08:49,  8.33s/it]
 94%|█████████▍| 211977/225000 [410:28:14<34:04:55,  9.42s/it]
 94%|█████████▍| 211978/225000 [410:28:21<31:46:52,  8.79s/it]
 94%|█████████▍| 211979/225000 [410:28:34<35:38:23,  9.85s/it]
 94%|█████████▍| 211980/225000 [410:28:42<33:41:00,  9.31s/it]
 94%|█████████▍| 211981/225000 [410:28:49<31:21:51,  8.67s/it]
 94%|█████████▍| 211982/225000 [410:28:57<30:22:45,  8.40s/it]
 94%|█████████▍| 211983/225000 [410:29:09<34:23:31,  9.51s/it]
 94%|█████████▍| 211984/225000 [410:29:18<34:24:08,  9.52s/it]
 94%|█████████▍| 211985/225000 [410:29:25<31:35:38,  8.74s/it]
 94%|█████████▍| 211986/225000 [410:29:33<30:28:39,  8.43s/it]
 94%|█████████▍| 211987/225000 [410:29:42<30:58:35,  8.57s/it]
 94%|█████████▍| 211988/225000 [410:29:50<30:10:41,  8.35s/it]
 94%|█████████▍| 211989/225000 [410:29:57<28:48:04,  7.97s/it]
 94%|█████████▍| 211990/225000 [410:30:08<32:40:26,  9.04s/it]
 94%|█████████▍| 211991/225000 [410:30:19<33:55:32,  9.39s/it]
 94%|█████████▍| 211992/225000 [410:30:25<31:04:19,  8.60s/it]
 94%|█████████▍| 211993/225000 [410:30:37<34:35:42,  9.58s/it]
 94%|█████████▍| 211994/225000 [410:30:45<33:04:01,  9.15s/it]
 94%|█████████▍| 211995/225000 [410:30:52<30:32:32,  8.45s/it]
 94%|█████████▍| 211996/225000 [410:30:59<29:01:42,  8.04s/it]
 94%|█████████▍| 211997/225000 [410:31:06<27:25:04,  7.59s/it]
 94%|█████████▍| 211998/225000 [410:31:16<30:39:50,  8.49s/it]
 94%|█████████▍| 211999/225000 [410:31:30<36:05:34,  9.99s/it]
 94%|█████████▍| 212000/225000 [410:31:39<35:35:14,  9.85s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.276', 'learning_rate': '4.565e-07', 'epoch': '0.9422', 'num_input_tokens_seen': 695380424, 'train_runtime': '1.478e+06', 'train_tokens_per_second': '470.5'}

 94%|█████████▍| 212000/225000 [410:31:39<35:35:14,  9.85s/it]
 94%|█████████▍| 212001/225000 [410:31:46<32:21:21,  8.96s/it]
 94%|█████████▍| 212002/225000 [410:31:53<29:30:55,  8.17s/it]
 94%|█████████▍| 212003/225000 [410:32:07<36:44:38, 10.18s/it]
 94%|█████████▍| 212004/225000 [410:32:17<36:21:28, 10.07s/it]
 94%|█████████▍| 212005/225000 [410:32:25<34:21:36,  9.52s/it]
 94%|█████████▍| 212006/225000 [410:32:34<33:03:17,  9.16s/it]
 94%|█████████▍| 212007/225000 [410:32:43<33:16:12,  9.22s/it]
 94%|█████████▍| 212008/225000 [410:32:52<32:57:21,  9.13s/it]
 94%|█████████▍| 212009/225000 [410:33:02<33:33:58,  9.30s/it]
 94%|█████████▍| 212010/225000 [410:33:07<29:05:00,  8.06s/it]
 94%|█████████▍| 212011/225000 [410:33:18<32:42:25,  9.07s/it]
 94%|█████████▍| 212012/225000 [410:33:31<36:33:11, 10.13s/it]
 94%|█████████▍| 212013/225000 [410:33:41<36:49:34, 10.21s/it]
 94%|█████████▍| 212014/225000 [410:33:53<38:09:04, 10.58s/it]
 94%|█████████▍| 212015/225000 [410:34:00<34:29:11,  9.56s/it]
 94%|█████████▍| 212016/225000 [410:34:06<31:04:29,  8.62s/it]
 94%|█████████▍| 212017/225000 [410:34:17<33:09:31,  9.19s/it]
 94%|█████████▍| 212018/225000 [410:34:25<31:45:05,  8.80s/it]
 94%|█████████▍| 212019/225000 [410:34:33<31:31:50,  8.74s/it]
 94%|█████████▍| 212020/225000 [410:34:42<31:33:28,  8.75s/it]
 94%|█████████▍| 212021/225000 [410:34:50<30:13:54,  8.39s/it]
 94%|█████████▍| 212022/225000 [410:34:58<30:34:15,  8.48s/it]
 94%|█████████▍| 212023/225000 [410:35:12<36:10:56, 10.04s/it]
 94%|█████████▍| 212024/225000 [410:35:19<32:21:25,  8.98s/it]
 94%|█████████▍| 212025/225000 [410:35:25<29:38:19,  8.22s/it]
 94%|█████████▍| 212026/225000 [410:35:40<36:38:41, 10.17s/it]
 94%|█████████▍| 212027/225000 [410:35:50<36:38:34, 10.17s/it]
 94%|█████████▍| 212028/225000 [410:35:56<32:28:45,  9.01s/it]
 94%|█████████▍| 212029/225000 [410:36:02<28:32:08,  7.92s/it]
 94%|█████████▍| 212030/225000 [410:36:07<25:22:07,  7.04s/it]
 94%|█████████▍| 212031/225000 [410:36:18<30:27:08,  8.45s/it]
 94%|█████████▍| 212032/225000 [410:36:28<31:28:01,  8.74s/it]
 94%|█████████▍| 212033/225000 [410:36:34<28:26:22,  7.90s/it]
 94%|█████████▍| 212034/225000 [410:36:46<32:42:08,  9.08s/it]
 94%|█████████▍| 212035/225000 [410:36:52<29:21:17,  8.15s/it]
 94%|█████████▍| 212036/225000 [410:37:01<30:50:54,  8.57s/it]
 94%|█████████▍| 212037/225000 [410:37:06<27:10:08,  7.55s/it]
 94%|█████████▍| 212038/225000 [410:37:13<25:49:25,  7.17s/it]
 94%|█████████▍| 212039/225000 [410:37:21<27:38:57,  7.68s/it]
 94%|█████████▍| 212040/225000 [410:37:28<26:35:22,  7.39s/it]
 94%|█████████▍| 212041/225000 [410:37:34<24:56:12,  6.93s/it]
 94%|█████████▍| 212042/225000 [410:37:45<29:17:51,  8.14s/it]
 94%|█████████▍| 212043/225000 [410:37:56<31:56:57,  8.88s/it]
 94%|█████████▍| 212044/225000 [410:38:10<38:27:47, 10.69s/it]
 94%|█████████▍| 212045/225000 [410:38:19<36:17:31, 10.09s/it]
 94%|█████████▍| 212046/225000 [410:38:26<32:56:31,  9.15s/it]
 94%|█████████▍| 212047/225000 [410:38:32<29:02:26,  8.07s/it]
 94%|█████████▍| 212048/225000 [410:38:42<31:36:47,  8.79s/it]
 94%|█████████▍| 212049/225000 [410:38:53<34:13:46,  9.51s/it]
 94%|█████████▍| 212050/225000 [410:38:59<30:34:19,  8.50s/it]
                                                              
{'loss': '1.278', 'grad_norm': '1.499', 'learning_rate': '4.531e-07', 'epoch': '0.9424', 'num_input_tokens_seen': 695610432, 'train_runtime': '1.478e+06', 'train_tokens_per_second': '470.5'}

 94%|█████████▍| 212050/225000 [410:38:59<30:34:19,  8.50s/it]
 94%|█████████▍| 212051/225000 [410:39:08<31:03:12,  8.63s/it]
 94%|█████████▍| 212052/225000 [410:39:14<28:17:11,  7.86s/it]
 94%|█████████▍| 212053/225000 [410:39:25<30:46:30,  8.56s/it]
 94%|█████████▍| 212054/225000 [410:39:34<31:14:45,  8.69s/it]
 94%|█████████▍| 212055/225000 [410:39:39<27:52:52,  7.75s/it]
 94%|█████████▍| 212056/225000 [410:39:50<31:36:06,  8.79s/it]
 94%|█████████▍| 212057/225000 [410:39:55<27:03:30,  7.53s/it]
 94%|█████████▍| 212058/225000 [410:40:04<28:43:44,  7.99s/it]
 94%|█████████▍| 212059/225000 [410:40:10<26:37:16,  7.41s/it]
 94%|█████████▍| 212060/225000 [410:40:20<28:54:41,  8.04s/it]
 94%|█████████▍| 212061/225000 [410:40:30<31:02:42,  8.64s/it]
 94%|█████████▍| 212062/225000 [410:40:35<27:43:20,  7.71s/it]
 94%|█████████▍| 212063/225000 [410:40:41<25:56:41,  7.22s/it]
 94%|█████████▍| 212064/225000 [410:40:56<34:29:47,  9.60s/it]
 94%|█████████▍| 212065/225000 [410:41:04<32:06:20,  8.94s/it]
 94%|█████████▍| 212066/225000 [410:41:11<29:44:56,  8.28s/it]
 94%|█████████▍| 212067/225000 [410:41:21<31:39:39,  8.81s/it]
 94%|█████████▍| 212068/225000 [410:41:31<33:29:19,  9.32s/it]
 94%|█████████▍| 212069/225000 [410:41:41<33:33:56,  9.34s/it]
 94%|█████████▍| 212070/225000 [410:41:48<31:18:56,  8.72s/it]
 94%|█████████▍| 212071/225000 [410:41:57<31:37:19,  8.80s/it]
 94%|█████████▍| 212072/225000 [410:42:05<31:19:40,  8.72s/it]
 94%|█████████▍| 212073/225000 [410:42:11<28:05:35,  7.82s/it]
 94%|█████████▍| 212074/225000 [410:42:22<31:47:06,  8.85s/it]
 94%|█████████▍| 212075/225000 [410:42:31<31:36:18,  8.80s/it]
 94%|█████████▍| 212076/225000 [410:42:38<29:51:47,  8.32s/it]
 94%|█████████▍| 212077/225000 [410:42:49<32:22:23,  9.02s/it]
 94%|█████████▍| 212078/225000 [410:42:57<31:49:13,  8.86s/it]
 94%|█████████▍| 212079/225000 [410:43:03<28:44:54,  8.01s/it]
 94%|█████████▍| 212080/225000 [410:43:15<32:54:09,  9.17s/it]
 94%|█████████▍| 212081/225000 [410:43:22<30:25:03,  8.48s/it]
 94%|█████████▍| 212082/225000 [410:43:28<27:42:06,  7.72s/it]
 94%|█████████▍| 212083/225000 [410:43:36<27:53:09,  7.77s/it]
 94%|█████████▍| 212084/225000 [410:43:44<28:03:42,  7.82s/it]
 94%|█████████▍| 212085/225000 [410:43:58<35:01:50,  9.76s/it]
 94%|█████████▍| 212086/225000 [410:44:06<32:58:02,  9.19s/it]
 94%|█████████▍| 212087/225000 [410:44:11<28:25:13,  7.92s/it]
 94%|█████████▍| 212088/225000 [410:44:21<30:52:41,  8.61s/it]
 94%|█████████▍| 212089/225000 [410:44:31<31:43:29,  8.85s/it]
 94%|█████████▍| 212090/225000 [410:44:39<31:10:26,  8.69s/it]
 94%|█████████▍| 212091/225000 [410:44:46<29:35:12,  8.25s/it]
 94%|█████████▍| 212092/225000 [410:44:56<31:42:15,  8.84s/it]
 94%|█████████▍| 212093/225000 [410:45:04<30:07:24,  8.40s/it]
 94%|█████████▍| 212094/225000 [410:45:13<31:11:24,  8.70s/it]
 94%|█████████▍| 212095/225000 [410:45:21<29:55:16,  8.35s/it]
 94%|█████████▍| 212096/225000 [410:45:27<28:02:12,  7.82s/it]
 94%|█████████▍| 212097/225000 [410:45:40<32:57:11,  9.19s/it]
 94%|█████████▍| 212098/225000 [410:45:45<29:05:18,  8.12s/it]
 94%|█████████▍| 212099/225000 [410:45:54<29:23:15,  8.20s/it]
 94%|█████████▍| 212100/225000 [410:46:09<37:22:39, 10.43s/it]
                                                              
{'loss': '1.307', 'grad_norm': '0.955', 'learning_rate': '4.496e-07', 'epoch': '0.9427', 'num_input_tokens_seen': 695842712, 'train_runtime': '1.479e+06', 'train_tokens_per_second': '470.6'}

 94%|█████████▍| 212100/225000 [410:46:09<37:22:39, 10.43s/it]
 94%|█████████▍| 212101/225000 [410:46:18<35:45:49,  9.98s/it]
 94%|█████████▍| 212102/225000 [410:46:25<32:25:48,  9.05s/it]
 94%|█████████▍| 212103/225000 [410:46:36<34:32:05,  9.64s/it]
 94%|█████████▍| 212104/225000 [410:46:43<31:36:49,  8.83s/it]
 94%|█████████▍| 212105/225000 [410:46:51<30:15:03,  8.45s/it]
 94%|█████████▍| 212106/225000 [410:47:02<33:52:36,  9.46s/it]
 94%|█████████▍| 212107/225000 [410:47:12<33:45:20,  9.43s/it]
 94%|█████████▍| 212108/225000 [410:47:20<32:38:51,  9.12s/it]
 94%|█████████▍| 212109/225000 [410:47:27<29:48:17,  8.32s/it]
 94%|█████████▍| 212110/225000 [410:47:33<28:12:12,  7.88s/it]
 94%|█████████▍| 212111/225000 [410:47:46<32:58:39,  9.21s/it]
 94%|█████████▍| 212112/225000 [410:47:55<32:26:10,  9.06s/it]
 94%|█████████▍| 212113/225000 [410:48:00<28:59:56,  8.10s/it]
 94%|█████████▍| 212114/225000 [410:48:07<27:03:48,  7.56s/it]
 94%|█████████▍| 212115/225000 [410:48:16<28:30:10,  7.96s/it]
 94%|█████████▍| 212116/225000 [410:48:27<31:47:32,  8.88s/it]
 94%|█████████▍| 212117/225000 [410:48:34<29:50:20,  8.34s/it]
 94%|█████████▍| 212118/225000 [410:48:42<29:35:16,  8.27s/it]
 94%|█████████▍| 212119/225000 [410:48:49<28:24:14,  7.94s/it]
 94%|█████████▍| 212120/225000 [410:49:02<33:32:48,  9.38s/it]
 94%|█████████▍| 212121/225000 [410:49:08<30:19:54,  8.48s/it]
 94%|█████████▍| 212122/225000 [410:49:14<27:20:17,  7.64s/it]
 94%|█████████▍| 212123/225000 [410:49:28<34:50:37,  9.74s/it]
 94%|█████████▍| 212124/225000 [410:49:33<29:40:47,  8.30s/it]
 94%|█████████▍| 212125/225000 [410:49:46<34:43:33,  9.71s/it]
 94%|█████████▍| 212126/225000 [410:49:52<30:17:01,  8.47s/it]
 94%|█████████▍| 212127/225000 [410:50:01<30:48:13,  8.61s/it]
 94%|█████████▍| 212128/225000 [410:50:10<31:17:20,  8.75s/it]
 94%|█████████▍| 212129/225000 [410:50:17<29:17:03,  8.19s/it]
 94%|█████████▍| 212130/225000 [410:50:27<30:59:08,  8.67s/it]
 94%|█████████▍| 212131/225000 [410:50:34<29:42:59,  8.31s/it]
 94%|█████████▍| 212132/225000 [410:50:47<34:28:20,  9.64s/it]
 94%|█████████▍| 212133/225000 [410:50:58<35:40:21,  9.98s/it]
 94%|█████████▍| 212134/225000 [410:51:03<30:32:49,  8.55s/it]
 94%|█████████▍| 212135/225000 [410:51:14<32:58:37,  9.23s/it]
 94%|█████████▍| 212136/225000 [410:51:21<31:11:54,  8.73s/it]
 94%|█████████▍| 212137/225000 [410:51:32<33:43:55,  9.44s/it]
 94%|█████████▍| 212138/225000 [410:51:38<30:03:47,  8.41s/it]
 94%|█████████▍| 212139/225000 [410:51:48<31:40:40,  8.87s/it]
 94%|█████████▍| 212140/225000 [410:51:55<29:15:21,  8.19s/it]
 94%|█████████▍| 212141/225000 [410:52:03<29:37:17,  8.29s/it]
 94%|█████████▍| 212142/225000 [410:52:17<35:00:23,  9.80s/it]
 94%|█████████▍| 212143/225000 [410:52:21<28:51:01,  8.08s/it]
 94%|█████████▍| 212144/225000 [410:52:31<31:24:14,  8.79s/it]
 94%|█████████▍| 212145/225000 [410:52:41<32:02:21,  8.97s/it]
 94%|█████████▍| 212146/225000 [410:52:52<34:41:25,  9.72s/it]
 94%|█████████▍| 212147/225000 [410:52:58<30:54:07,  8.66s/it]
 94%|█████████▍| 212148/225000 [410:53:11<35:44:26, 10.01s/it]
 94%|█████████▍| 212149/225000 [410:53:18<32:20:43,  9.06s/it]
 94%|█████████▍| 212150/225000 [410:53:27<32:20:35,  9.06s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.18', 'learning_rate': '4.462e-07', 'epoch': '0.9429', 'num_input_tokens_seen': 696082152, 'train_runtime': '1.479e+06', 'train_tokens_per_second': '470.6'}

 94%|█████████▍| 212150/225000 [410:53:27<32:20:35,  9.06s/it]
 94%|█████████▍| 212151/225000 [410:53:34<29:33:43,  8.28s/it]
 94%|█████████▍| 212152/225000 [410:53:41<28:56:37,  8.11s/it]
 94%|█████████▍| 212153/225000 [410:53:55<35:12:37,  9.87s/it]
 94%|█████████▍| 212154/225000 [410:54:02<32:01:06,  8.97s/it]
 94%|█████████▍| 212155/225000 [410:54:11<32:03:57,  8.99s/it]
 94%|█████████▍| 212156/225000 [410:54:19<30:59:42,  8.69s/it]
 94%|█████████▍| 212157/225000 [410:54:26<29:11:12,  8.18s/it]
 94%|█████████▍| 212158/225000 [410:54:37<31:29:57,  8.83s/it]
 94%|█████████▍| 212159/225000 [410:54:45<31:25:12,  8.81s/it]
 94%|█████████▍| 212160/225000 [410:54:55<31:43:13,  8.89s/it]
 94%|█████████▍| 212161/225000 [410:55:03<31:12:58,  8.75s/it]
 94%|█████████▍| 212162/225000 [410:55:11<30:06:41,  8.44s/it]
 94%|█████████▍| 212163/225000 [410:55:18<29:26:02,  8.25s/it]
 94%|█████████▍| 212164/225000 [410:55:24<26:46:34,  7.51s/it]
 94%|█████████▍| 212165/225000 [410:55:34<29:18:34,  8.22s/it]
 94%|█████████▍| 212166/225000 [410:55:44<30:42:25,  8.61s/it]
 94%|█████████▍| 212167/225000 [410:55:54<32:19:57,  9.07s/it]
 94%|█████████▍| 212168/225000 [410:55:59<28:05:44,  7.88s/it]
 94%|█████████▍| 212169/225000 [410:56:09<29:55:06,  8.39s/it]
 94%|█████████▍| 212170/225000 [410:56:17<30:17:09,  8.50s/it]
 94%|█████████▍| 212171/225000 [410:56:22<26:39:17,  7.48s/it]
 94%|█████████▍| 212172/225000 [410:56:37<34:12:25,  9.60s/it]
 94%|█████████▍| 212173/225000 [410:56:47<34:49:22,  9.77s/it]
 94%|█████████▍| 212174/225000 [410:56:54<31:18:04,  8.79s/it]
 94%|█████████▍| 212175/225000 [410:57:07<35:44:16, 10.03s/it]
 94%|█████████▍| 212176/225000 [410:57:16<34:50:20,  9.78s/it]
 94%|█████████▍| 212177/225000 [410:57:22<30:51:39,  8.66s/it]
 94%|█████████▍| 212178/225000 [410:57:30<30:24:15,  8.54s/it]
 94%|█████████▍| 212179/225000 [410:57:38<30:10:04,  8.47s/it]
 94%|█████████▍| 212180/225000 [410:57:45<28:39:21,  8.05s/it]
 94%|█████████▍| 212181/225000 [410:57:56<31:36:23,  8.88s/it]
 94%|█████████▍| 212182/225000 [410:58:05<31:13:11,  8.77s/it]
 94%|█████████▍| 212183/225000 [410:58:20<38:33:33, 10.83s/it]
 94%|█████████▍| 212184/225000 [410:58:25<31:28:11,  8.84s/it]
 94%|█████████▍| 212185/225000 [410:58:30<28:06:09,  7.89s/it]
 94%|█████████▍| 212186/225000 [410:58:43<33:04:57,  9.29s/it]
 94%|█████████▍| 212187/225000 [410:58:51<32:12:29,  9.05s/it]
 94%|█████████▍| 212188/225000 [410:59:02<33:58:10,  9.55s/it]
 94%|█████████▍| 212189/225000 [410:59:08<30:42:07,  8.63s/it]
 94%|█████████▍| 212190/225000 [410:59:15<28:58:09,  8.14s/it]
 94%|█████████▍| 212191/225000 [410:59:20<25:33:26,  7.18s/it]
 94%|█████████▍| 212192/225000 [410:59:29<27:24:19,  7.70s/it]
 94%|█████████▍| 212193/225000 [410:59:36<26:39:58,  7.50s/it]
 94%|█████████▍| 212194/225000 [410:59:51<34:36:43,  9.73s/it]
 94%|█████████▍| 212195/225000 [411:00:01<35:03:31,  9.86s/it]
 94%|█████████▍| 212196/225000 [411:00:09<32:29:46,  9.14s/it]
 94%|█████████▍| 212197/225000 [411:00:17<31:12:07,  8.77s/it]
 94%|█████████▍| 212198/225000 [411:00:30<36:00:40, 10.13s/it]
 94%|█████████▍| 212199/225000 [411:00:35<30:23:00,  8.54s/it]
 94%|█████████▍| 212200/225000 [411:00:42<29:11:24,  8.21s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.32', 'learning_rate': '4.428e-07', 'epoch': '0.9431', 'num_input_tokens_seen': 696318576, 'train_runtime': '1.48e+06', 'train_tokens_per_second': '470.6'}

 94%|█████████▍| 212200/225000 [411:00:42<29:11:24,  8.21s/it]
 94%|█████████▍| 212201/225000 [411:00:55<33:47:34,  9.51s/it]
 94%|█████████▍| 212202/225000 [411:01:00<29:29:13,  8.29s/it]
 94%|█████████▍| 212203/225000 [411:01:08<28:44:35,  8.09s/it]
 94%|█████████▍| 212204/225000 [411:01:22<34:56:00,  9.83s/it]
 94%|█████████▍| 212205/225000 [411:01:30<33:05:23,  9.31s/it]
 94%|█████████▍| 212206/225000 [411:01:38<31:32:23,  8.87s/it]
 94%|█████████▍| 212207/225000 [411:01:53<38:08:55, 10.74s/it]
 94%|█████████▍| 212208/225000 [411:02:03<37:37:47, 10.59s/it]
 94%|█████████▍| 212209/225000 [411:02:10<33:29:18,  9.43s/it]
 94%|█████████▍| 212210/225000 [411:02:20<34:41:49,  9.77s/it]
 94%|█████████▍| 212211/225000 [411:02:27<31:25:30,  8.85s/it]
 94%|█████████▍| 212212/225000 [411:02:33<28:12:12,  7.94s/it]
 94%|█████████▍| 212213/225000 [411:02:44<31:24:00,  8.84s/it]
 94%|█████████▍| 212214/225000 [411:02:52<30:34:41,  8.61s/it]
 94%|█████████▍| 212215/225000 [411:02:59<28:53:45,  8.14s/it]
 94%|█████████▍| 212216/225000 [411:03:13<34:49:49,  9.81s/it]
 94%|█████████▍| 212217/225000 [411:03:21<33:09:27,  9.34s/it]
 94%|█████████▍| 212218/225000 [411:03:32<35:09:19,  9.90s/it]
 94%|█████████▍| 212219/225000 [411:03:42<34:54:50,  9.83s/it]
 94%|█████████▍| 212220/225000 [411:03:50<33:25:39,  9.42s/it]
 94%|█████████▍| 212221/225000 [411:03:57<30:09:52,  8.50s/it]
 94%|█████████▍| 212222/225000 [411:04:04<29:05:23,  8.20s/it]
 94%|█████████▍| 212223/225000 [411:04:13<30:05:22,  8.48s/it]
 94%|█████████▍| 212224/225000 [411:04:23<31:24:31,  8.85s/it]
 94%|█████████▍| 212225/225000 [411:04:33<32:59:21,  9.30s/it]
 94%|█████████▍| 212226/225000 [411:04:43<33:39:45,  9.49s/it]
 94%|█████████▍| 212227/225000 [411:04:48<28:23:50,  8.00s/it]
 94%|█████████▍| 212228/225000 [411:04:55<28:02:48,  7.91s/it]
 94%|█████████▍| 212229/225000 [411:05:06<30:55:55,  8.72s/it]
 94%|█████████▍| 212230/225000 [411:05:13<28:40:06,  8.08s/it]
 94%|█████████▍| 212231/225000 [411:05:21<28:25:37,  8.01s/it]
 94%|█████████▍| 212232/225000 [411:05:26<25:59:10,  7.33s/it]
 94%|█████████▍| 212233/225000 [411:05:39<31:29:07,  8.88s/it]
 94%|█████████▍| 212234/225000 [411:05:44<27:55:27,  7.87s/it]
 94%|█████████▍| 212235/225000 [411:05:52<28:06:44,  7.93s/it]
 94%|█████████▍| 212236/225000 [411:05:59<27:08:53,  7.66s/it]
 94%|█████████▍| 212237/225000 [411:06:08<28:15:28,  7.97s/it]
 94%|█████████▍| 212238/225000 [411:06:20<31:59:50,  9.03s/it]
 94%|█████████▍| 212239/225000 [411:06:26<29:34:02,  8.34s/it]
 94%|█████████▍| 212240/225000 [411:06:40<35:41:30, 10.07s/it]
 94%|█████████▍| 212241/225000 [411:06:51<35:59:47, 10.16s/it]
 94%|█████████▍| 212242/225000 [411:06:55<29:46:43,  8.40s/it]
 94%|█████████▍| 212243/225000 [411:07:03<29:42:54,  8.39s/it]
 94%|█████████▍| 212244/225000 [411:07:12<30:18:24,  8.55s/it]
 94%|█████████▍| 212245/225000 [411:07:22<31:08:22,  8.79s/it]
 94%|█████████▍| 212246/225000 [411:07:29<29:59:10,  8.46s/it]
 94%|█████████▍| 212247/225000 [411:07:40<31:57:50,  9.02s/it]
 94%|█████████▍| 212248/225000 [411:07:51<33:51:24,  9.56s/it]
 94%|█████████▍| 212249/225000 [411:08:04<37:52:57, 10.70s/it]
 94%|█████████▍| 212250/225000 [411:08:11<34:32:11,  9.75s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.187', 'learning_rate': '4.393e-07', 'epoch': '0.9433', 'num_input_tokens_seen': 696559160, 'train_runtime': '1.48e+06', 'train_tokens_per_second': '470.6'}

 94%|█████████▍| 212250/225000 [411:08:11<34:32:11,  9.75s/it]
 94%|█████████▍| 212251/225000 [411:08:21<33:56:56,  9.59s/it]
 94%|█████████▍| 212252/225000 [411:08:26<29:50:57,  8.43s/it]
 94%|█████████▍| 212253/225000 [411:08:34<29:07:02,  8.22s/it]
 94%|█████████▍| 212254/225000 [411:08:40<26:06:47,  7.38s/it]
 94%|█████████▍| 212255/225000 [411:08:55<34:34:57,  9.77s/it]
 94%|█████████▍| 212256/225000 [411:09:02<31:34:12,  8.92s/it]
 94%|█████████▍| 212257/225000 [411:09:11<32:09:28,  9.08s/it]
 94%|█████████▍| 212258/225000 [411:09:22<33:59:35,  9.60s/it]
 94%|█████████▍| 212259/225000 [411:09:29<31:27:48,  8.89s/it]
 94%|█████████▍| 212260/225000 [411:09:40<33:23:25,  9.44s/it]
 94%|█████████▍| 212261/225000 [411:09:49<32:59:59,  9.33s/it]
 94%|█████████▍| 212262/225000 [411:09:56<30:27:43,  8.61s/it]
 94%|█████████▍| 212263/225000 [411:10:05<30:43:16,  8.68s/it]
 94%|█████████▍| 212264/225000 [411:10:12<28:34:09,  8.08s/it]
 94%|█████████▍| 212265/225000 [411:10:21<30:04:11,  8.50s/it]
 94%|█████████▍| 212266/225000 [411:10:29<29:51:45,  8.44s/it]
 94%|█████████▍| 212267/225000 [411:10:36<27:36:35,  7.81s/it]
 94%|█████████▍| 212268/225000 [411:10:41<24:55:06,  7.05s/it]
 94%|█████████▍| 212269/225000 [411:10:55<31:56:58,  9.03s/it]
 94%|█████████▍| 212270/225000 [411:11:04<32:18:35,  9.14s/it]
 94%|█████████▍| 212271/225000 [411:11:09<28:00:30,  7.92s/it]
 94%|█████████▍| 212272/225000 [411:11:17<28:21:39,  8.02s/it]
 94%|█████████▍| 212273/225000 [411:11:26<29:19:10,  8.29s/it]
 94%|█████████▍| 212274/225000 [411:11:36<31:22:33,  8.88s/it]
 94%|█████████▍| 212275/225000 [411:11:48<33:41:45,  9.53s/it]
 94%|█████████▍| 212276/225000 [411:11:57<33:18:15,  9.42s/it]
 94%|█████████▍| 212277/225000 [411:12:05<32:32:55,  9.21s/it]
 94%|█████████▍| 212278/225000 [411:12:13<31:07:04,  8.81s/it]
 94%|█████████▍| 212279/225000 [411:12:24<33:02:20,  9.35s/it]
 94%|█████████▍| 212280/225000 [411:12:31<30:23:16,  8.60s/it]
 94%|█████████▍| 212281/225000 [411:12:37<28:15:04,  8.00s/it]
 94%|█████████▍| 212282/225000 [411:12:50<33:08:45,  9.38s/it]
 94%|█████████▍| 212283/225000 [411:12:55<28:55:54,  8.19s/it]
 94%|█████████▍| 212284/225000 [411:13:01<26:40:46,  7.55s/it]
 94%|█████████▍| 212285/225000 [411:13:12<29:55:34,  8.47s/it]
 94%|█████████▍| 212286/225000 [411:13:17<26:17:16,  7.44s/it]
 94%|█████████▍| 212287/225000 [411:13:23<24:23:14,  6.91s/it]
 94%|█████████▍| 212288/225000 [411:13:28<22:06:28,  6.26s/it]
 94%|█████████▍| 212289/225000 [411:13:42<30:23:58,  8.61s/it]
 94%|█████████▍| 212290/225000 [411:13:53<33:29:31,  9.49s/it]
 94%|█████████▍| 212291/225000 [411:13:58<29:06:14,  8.24s/it]
 94%|█████████▍| 212292/225000 [411:14:06<28:04:31,  7.95s/it]
 94%|█████████▍| 212293/225000 [411:14:14<28:01:37,  7.94s/it]
 94%|█████████▍| 212294/225000 [411:14:21<27:45:45,  7.87s/it]
 94%|█████████▍| 212295/225000 [411:14:33<32:05:19,  9.09s/it]
 94%|█████████▍| 212296/225000 [411:14:41<31:02:20,  8.80s/it]
 94%|█████████▍| 212297/225000 [411:14:51<31:23:30,  8.90s/it]
 94%|█████████▍| 212298/225000 [411:15:00<32:18:56,  9.16s/it]
 94%|█████████▍| 212299/225000 [411:15:12<34:32:18,  9.79s/it]
 94%|█████████▍| 212300/225000 [411:15:19<31:35:37,  8.96s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.089', 'learning_rate': '4.36e-07', 'epoch': '0.9436', 'num_input_tokens_seen': 696790136, 'train_runtime': '1.481e+06', 'train_tokens_per_second': '470.6'}

 94%|█████████▍| 212300/225000 [411:15:19<31:35:37,  8.96s/it]
 94%|█████████▍| 212301/225000 [411:15:29<33:20:21,  9.45s/it]
 94%|█████████▍| 212302/225000 [411:15:34<28:38:16,  8.12s/it]
 94%|█████████▍| 212303/225000 [411:15:44<30:54:07,  8.76s/it]
 94%|█████████▍| 212304/225000 [411:15:56<33:45:28,  9.57s/it]
 94%|█████████▍| 212305/225000 [411:16:01<29:19:28,  8.32s/it]
 94%|█████████▍| 212306/225000 [411:16:13<32:31:57,  9.23s/it]
 94%|█████████▍| 212307/225000 [411:16:17<27:35:39,  7.83s/it]
 94%|█████████▍| 212308/225000 [411:16:28<31:02:39,  8.81s/it]
 94%|█████████▍| 212309/225000 [411:16:38<32:16:23,  9.15s/it]
 94%|█████████▍| 212310/225000 [411:16:47<31:33:24,  8.95s/it]
 94%|█████████▍| 212311/225000 [411:16:56<32:19:51,  9.17s/it]
 94%|█████████▍| 212312/225000 [411:17:04<30:41:42,  8.71s/it]
 94%|█████████▍| 212313/225000 [411:17:12<29:44:49,  8.44s/it]
 94%|█████████▍| 212314/225000 [411:17:18<27:41:51,  7.86s/it]
 94%|█████████▍| 212315/225000 [411:17:25<26:48:34,  7.61s/it]
 94%|█████████▍| 212316/225000 [411:17:36<30:23:46,  8.63s/it]
 94%|█████████▍| 212317/225000 [411:17:47<32:31:41,  9.23s/it]
 94%|█████████▍| 212318/225000 [411:17:53<29:21:38,  8.33s/it]
 94%|█████████▍| 212319/225000 [411:17:59<26:52:52,  7.63s/it]
 94%|█████████▍| 212320/225000 [411:18:06<25:34:54,  7.26s/it]
 94%|█████████▍| 212321/225000 [411:18:18<30:49:27,  8.75s/it]
 94%|█████████▍| 212322/225000 [411:18:26<30:11:18,  8.57s/it]
 94%|█████████▍| 212323/225000 [411:18:33<28:35:45,  8.12s/it]
 94%|█████████▍| 212324/225000 [411:18:48<35:16:20, 10.02s/it]
 94%|█████████▍| 212325/225000 [411:18:59<37:13:18, 10.57s/it]
 94%|█████████▍| 212326/225000 [411:19:09<35:46:29, 10.16s/it]
 94%|█████████▍| 212327/225000 [411:19:20<36:55:25, 10.49s/it]
 94%|█████████▍| 212328/225000 [411:19:28<34:28:02,  9.79s/it]
 94%|█████████▍| 212329/225000 [411:19:38<35:03:59,  9.96s/it]
 94%|█████████▍| 212330/225000 [411:19:51<37:58:51, 10.79s/it]
 94%|█████████▍| 212331/225000 [411:20:00<35:52:33, 10.19s/it]
 94%|█████████▍| 212332/225000 [411:20:05<30:24:40,  8.64s/it]
 94%|█████████▍| 212333/225000 [411:20:13<29:50:45,  8.48s/it]
 94%|█████████▍| 212334/225000 [411:20:22<30:25:57,  8.65s/it]
 94%|█████████▍| 212335/225000 [411:20:34<34:14:00,  9.73s/it]
 94%|█████████▍| 212336/225000 [411:20:41<30:48:19,  8.76s/it]
 94%|█████████▍| 212337/225000 [411:20:48<28:56:33,  8.23s/it]
 94%|█████████▍| 212338/225000 [411:21:02<35:23:28, 10.06s/it]
 94%|█████████▍| 212339/225000 [411:21:10<33:10:19,  9.43s/it]
 94%|█████████▍| 212340/225000 [411:21:17<30:34:31,  8.69s/it]
 94%|█████████▍| 212341/225000 [411:21:29<33:46:26,  9.60s/it]
 94%|█████████▍| 212342/225000 [411:21:33<28:20:50,  8.06s/it]
 94%|█████████▍| 212343/225000 [411:21:39<26:15:29,  7.47s/it]
 94%|█████████▍| 212344/225000 [411:21:45<24:11:05,  6.88s/it]
 94%|█████████▍| 212345/225000 [411:21:51<23:48:01,  6.77s/it]
 94%|█████████▍| 212346/225000 [411:22:03<29:01:29,  8.26s/it]
 94%|█████████▍| 212347/225000 [411:22:11<28:26:32,  8.09s/it]
 94%|█████████▍| 212348/225000 [411:22:19<28:49:50,  8.20s/it]
 94%|█████████▍| 212349/225000 [411:22:28<29:07:25,  8.29s/it]
 94%|█████████▍| 212350/225000 [411:22:40<32:44:51,  9.32s/it]
                                                              
{'loss': '1.299', 'grad_norm': '1.001', 'learning_rate': '4.326e-07', 'epoch': '0.9438', 'num_input_tokens_seen': 697025712, 'train_runtime': '1.481e+06', 'train_tokens_per_second': '470.7'}

 94%|█████████▍| 212350/225000 [411:22:40<32:44:51,  9.32s/it]
 94%|█████████▍| 212351/225000 [411:22:52<35:51:03, 10.20s/it]
 94%|█████████▍| 212352/225000 [411:22:58<31:17:33,  8.91s/it]
 94%|█████████▍| 212353/225000 [411:23:04<28:33:17,  8.13s/it]
 94%|█████████▍| 212354/225000 [411:23:17<34:10:49,  9.73s/it]
 94%|█████████▍| 212355/225000 [411:23:23<29:57:39,  8.53s/it]
 94%|█████████▍| 212356/225000 [411:23:31<28:54:34,  8.23s/it]
 94%|█████████▍| 212357/225000 [411:23:41<31:31:50,  8.98s/it]
 94%|█████████▍| 212358/225000 [411:23:53<33:56:14,  9.66s/it]
 94%|█████████▍| 212359/225000 [411:24:02<33:12:06,  9.46s/it]
 94%|█████████▍| 212360/225000 [411:24:12<33:40:02,  9.59s/it]
 94%|█████████▍| 212361/225000 [411:24:22<34:59:52,  9.97s/it]
 94%|█████████▍| 212362/225000 [411:24:29<31:08:04,  8.87s/it]
 94%|█████████▍| 212363/225000 [411:24:40<33:36:19,  9.57s/it]
 94%|█████████▍| 212364/225000 [411:24:50<33:59:22,  9.68s/it]
 94%|█████████▍| 212365/225000 [411:24:56<30:38:54,  8.73s/it]
 94%|█████████▍| 212366/225000 [411:25:07<32:11:13,  9.17s/it]
 94%|█████████▍| 212367/225000 [411:25:22<38:47:31, 11.05s/it]
 94%|█████████▍| 212368/225000 [411:25:31<36:51:45, 10.51s/it]
 94%|█████████▍| 212369/225000 [411:25:38<32:55:37,  9.38s/it]
 94%|█████████▍| 212370/225000 [411:25:46<31:40:20,  9.03s/it]
 94%|█████████▍| 212371/225000 [411:25:52<27:56:47,  7.97s/it]
 94%|█████████▍| 212372/225000 [411:26:01<29:03:08,  8.28s/it]
 94%|█████████▍| 212373/225000 [411:26:10<29:36:15,  8.44s/it]
 94%|█████████▍| 212374/225000 [411:26:22<34:02:13,  9.70s/it]
 94%|█████████▍| 212375/225000 [411:26:30<31:44:47,  9.05s/it]
 94%|█████████▍| 212376/225000 [411:26:34<26:54:22,  7.67s/it]
 94%|█████████▍| 212377/225000 [411:26:39<24:22:24,  6.95s/it]
 94%|█████████▍| 212378/225000 [411:26:45<22:59:58,  6.56s/it]
 94%|█████████▍| 212379/225000 [411:27:01<33:15:00,  9.48s/it]
 94%|█████████▍| 212380/225000 [411:27:12<34:27:22,  9.83s/it]
 94%|█████████▍| 212381/225000 [411:27:17<29:45:05,  8.49s/it]
 94%|█████████▍| 212382/225000 [411:27:31<35:26:58, 10.11s/it]
 94%|█████████▍| 212383/225000 [411:27:38<31:17:53,  8.93s/it]
 94%|█████████▍| 212384/225000 [411:27:47<32:14:36,  9.20s/it]
 94%|█████████▍| 212385/225000 [411:27:57<33:09:29,  9.46s/it]
 94%|█████████▍| 212386/225000 [411:28:06<32:11:40,  9.19s/it]
 94%|█████████▍| 212387/225000 [411:28:12<28:58:01,  8.27s/it]
 94%|█████████▍| 212388/225000 [411:28:22<30:14:56,  8.63s/it]
 94%|█████████▍| 212389/225000 [411:28:30<29:47:45,  8.51s/it]
 94%|█████████▍| 212390/225000 [411:28:43<34:59:50,  9.99s/it]
 94%|█████████▍| 212391/225000 [411:28:50<31:16:26,  8.93s/it]
 94%|█████████▍| 212392/225000 [411:28:55<27:14:51,  7.78s/it]
 94%|█████████▍| 212393/225000 [411:29:05<29:49:52,  8.52s/it]
 94%|█████████▍| 212394/225000 [411:29:14<30:20:36,  8.67s/it]
 94%|█████████▍| 212395/225000 [411:29:20<27:10:15,  7.76s/it]
 94%|█████████▍| 212396/225000 [411:29:29<29:18:22,  8.37s/it]
 94%|█████████▍| 212397/225000 [411:29:41<33:02:42,  9.44s/it]
 94%|█████████▍| 212398/225000 [411:29:48<29:45:21,  8.50s/it]
 94%|█████████▍| 212399/225000 [411:29:53<26:33:46,  7.59s/it]
 94%|█████████▍| 212400/225000 [411:30:06<31:56:15,  9.13s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.139', 'learning_rate': '4.292e-07', 'epoch': '0.944', 'num_input_tokens_seen': 697258016, 'train_runtime': '1.481e+06', 'train_tokens_per_second': '470.7'}

 94%|█████████▍| 212400/225000 [411:30:06<31:56:15,  9.13s/it]
 94%|█████████▍| 212401/225000 [411:30:14<30:53:10,  8.83s/it]
 94%|█████████▍| 212402/225000 [411:30:21<29:11:09,  8.34s/it]
 94%|█████████▍| 212403/225000 [411:30:33<32:21:47,  9.25s/it]
 94%|█████████▍| 212404/225000 [411:30:38<27:54:20,  7.98s/it]
 94%|█████████▍| 212405/225000 [411:30:47<29:25:41,  8.41s/it]
 94%|█████████▍| 212406/225000 [411:30:57<31:30:47,  9.01s/it]
 94%|█████████▍| 212407/225000 [411:31:03<27:50:09,  7.96s/it]
 94%|█████████▍| 212408/225000 [411:31:14<31:04:17,  8.88s/it]
 94%|█████████▍| 212409/225000 [411:31:18<26:03:00,  7.45s/it]
 94%|█████████▍| 212410/225000 [411:31:27<27:30:46,  7.87s/it]
 94%|█████████▍| 212411/225000 [411:31:31<24:00:55,  6.87s/it]
 94%|█████████▍| 212412/225000 [411:31:44<30:22:30,  8.69s/it]
 94%|█████████▍| 212413/225000 [411:31:53<30:17:13,  8.66s/it]
 94%|█████████▍| 212414/225000 [411:31:59<27:48:51,  7.96s/it]
 94%|█████████▍| 212415/225000 [411:32:06<26:14:49,  7.51s/it]
 94%|█████████▍| 212416/225000 [411:32:16<28:53:32,  8.27s/it]
 94%|█████████▍| 212417/225000 [411:32:24<28:38:17,  8.19s/it]
 94%|█████████▍| 212418/225000 [411:32:33<29:51:19,  8.54s/it]
 94%|█████████▍| 212419/225000 [411:32:41<29:07:11,  8.33s/it]
 94%|█████████▍| 212420/225000 [411:32:50<29:29:17,  8.44s/it]
 94%|█████████▍| 212421/225000 [411:33:02<33:33:16,  9.60s/it]
 94%|█████████▍| 212422/225000 [411:33:12<33:54:31,  9.71s/it]
 94%|█████████▍| 212423/225000 [411:33:19<31:01:34,  8.88s/it]
 94%|█████████▍| 212424/225000 [411:33:32<35:52:41, 10.27s/it]
 94%|█████████▍| 212425/225000 [411:33:38<30:27:23,  8.72s/it]
 94%|█████████▍| 212426/225000 [411:33:48<32:22:27,  9.27s/it]
 94%|█████████▍| 212427/225000 [411:34:00<35:36:37, 10.20s/it]
 94%|█████████▍| 212428/225000 [411:34:06<30:36:11,  8.76s/it]
 94%|█████████▍| 212429/225000 [411:34:12<27:52:30,  7.98s/it]
 94%|█████████▍| 212430/225000 [411:34:23<31:08:28,  8.92s/it]
 94%|█████████▍| 212431/225000 [411:34:32<30:43:22,  8.80s/it]
 94%|█████████▍| 212432/225000 [411:34:42<32:35:59,  9.34s/it]
 94%|█████████▍| 212433/225000 [411:34:53<34:11:35,  9.80s/it]
 94%|█████████▍| 212434/225000 [411:35:04<35:12:02, 10.08s/it]
 94%|█████████▍| 212435/225000 [411:35:12<33:25:54,  9.58s/it]
 94%|█████████▍| 212436/225000 [411:35:19<30:12:45,  8.66s/it]
 94%|█████████▍| 212437/225000 [411:35:33<36:10:44, 10.37s/it]
 94%|█████████▍| 212438/225000 [411:35:38<30:42:17,  8.80s/it]
 94%|█████████▍| 212439/225000 [411:35:47<30:27:45,  8.73s/it]
 94%|█████████▍| 212440/225000 [411:35:55<30:07:19,  8.63s/it]
 94%|█████████▍| 212441/225000 [411:36:01<26:49:05,  7.69s/it]
 94%|█████████▍| 212442/225000 [411:36:09<26:54:48,  7.72s/it]
 94%|█████████▍| 212443/225000 [411:36:14<24:07:24,  6.92s/it]
 94%|█████████▍| 212444/225000 [411:36:20<23:32:51,  6.75s/it]
 94%|█████████▍| 212445/225000 [411:36:31<27:37:57,  7.92s/it]
 94%|█████████▍| 212446/225000 [411:36:36<25:18:40,  7.26s/it]
 94%|█████████▍| 212447/225000 [411:36:45<27:12:02,  7.80s/it]
 94%|█████████▍| 212448/225000 [411:36:56<29:50:34,  8.56s/it]
 94%|█████████▍| 212449/225000 [411:37:00<25:35:43,  7.34s/it]
 94%|█████████▍| 212450/225000 [411:37:09<27:22:04,  7.85s/it]
                                                              
{'loss': '1.335', 'grad_norm': '1.261', 'learning_rate': '4.259e-07', 'epoch': '0.9442', 'num_input_tokens_seen': 697486352, 'train_runtime': '1.482e+06', 'train_tokens_per_second': '470.7'}

 94%|█████████▍| 212450/225000 [411:37:09<27:22:04,  7.85s/it]
 94%|█████████▍| 212451/225000 [411:37:17<27:47:05,  7.97s/it]
 94%|█████████▍| 212452/225000 [411:37:24<25:51:55,  7.42s/it]
 94%|█████████▍| 212453/225000 [411:37:29<24:12:32,  6.95s/it]
 94%|█████████▍| 212454/225000 [411:37:40<27:30:20,  7.89s/it]
 94%|█████████▍| 212455/225000 [411:37:51<30:55:08,  8.87s/it]
 94%|█████████▍| 212456/225000 [411:37:58<29:35:11,  8.49s/it]
 94%|█████████▍| 212457/225000 [411:38:09<31:59:12,  9.18s/it]
 94%|█████████▍| 212458/225000 [411:38:17<30:12:21,  8.67s/it]
 94%|█████████▍| 212459/225000 [411:38:23<28:21:48,  8.14s/it]
 94%|█████████▍| 212460/225000 [411:38:30<26:29:49,  7.61s/it]
 94%|█████████▍| 212461/225000 [411:38:39<27:45:13,  7.97s/it]
 94%|█████████▍| 212462/225000 [411:38:54<35:43:27, 10.26s/it]
 94%|█████████▍| 212463/225000 [411:39:02<32:56:46,  9.46s/it]
 94%|█████████▍| 212464/225000 [411:39:08<29:42:32,  8.53s/it]
 94%|█████████▍| 212465/225000 [411:39:18<31:20:54,  9.00s/it]
 94%|█████████▍| 212466/225000 [411:39:25<29:04:22,  8.35s/it]
 94%|█████████▍| 212467/225000 [411:39:36<31:28:42,  9.04s/it]
 94%|█████████▍| 212468/225000 [411:39:41<27:53:26,  8.01s/it]
 94%|█████████▍| 212469/225000 [411:39:48<25:57:25,  7.46s/it]
 94%|█████████▍| 212470/225000 [411:39:54<25:04:35,  7.20s/it]
 94%|█████████▍| 212471/225000 [411:40:12<35:38:40, 10.24s/it]
 94%|█████████▍| 212472/225000 [411:40:19<32:35:46,  9.37s/it]
 94%|█████████▍| 212473/225000 [411:40:28<32:44:15,  9.41s/it]
 94%|█████████▍| 212474/225000 [411:40:34<29:07:48,  8.37s/it]
 94%|█████████▍| 212475/225000 [411:40:43<29:58:03,  8.61s/it]
 94%|█████████▍| 212476/225000 [411:40:58<36:22:31, 10.46s/it]
 94%|█████████▍| 212477/225000 [411:41:10<37:58:40, 10.92s/it]
 94%|█████████▍| 212478/225000 [411:41:16<32:26:57,  9.33s/it]
 94%|█████████▍| 212479/225000 [411:41:24<31:22:00,  9.02s/it]
 94%|█████████▍| 212480/225000 [411:41:34<32:39:34,  9.39s/it]
 94%|█████████▍| 212481/225000 [411:41:43<31:24:04,  9.03s/it]
 94%|█████████▍| 212482/225000 [411:41:51<30:20:18,  8.72s/it]
 94%|█████████▍| 212483/225000 [411:41:58<29:05:11,  8.37s/it]
 94%|█████████▍| 212484/225000 [411:42:06<28:33:04,  8.21s/it]
 94%|█████████▍| 212485/225000 [411:42:18<32:15:06,  9.28s/it]
 94%|█████████▍| 212486/225000 [411:42:28<32:50:35,  9.45s/it]
 94%|█████████▍| 212487/225000 [411:42:35<31:07:37,  8.96s/it]
 94%|█████████▍| 212488/225000 [411:42:45<32:00:27,  9.21s/it]
 94%|█████████▍| 212489/225000 [411:42:54<31:43:50,  9.13s/it]
 94%|█████████▍| 212490/225000 [411:43:04<32:14:18,  9.28s/it]
 94%|█████████▍| 212491/225000 [411:43:11<30:30:50,  8.78s/it]
 94%|█████████▍| 212492/225000 [411:43:17<27:34:46,  7.94s/it]
 94%|█████████▍| 212493/225000 [411:43:27<29:23:54,  8.46s/it]
 94%|█████████▍| 212494/225000 [411:43:37<31:22:12,  9.03s/it]
 94%|█████████▍| 212495/225000 [411:43:43<28:01:25,  8.07s/it]
 94%|█████████▍| 212496/225000 [411:43:54<31:17:22,  9.01s/it]
 94%|█████████▍| 212497/225000 [411:44:00<27:16:19,  7.85s/it]
 94%|█████████▍| 212498/225000 [411:44:13<32:41:01,  9.41s/it]
 94%|█████████▍| 212499/225000 [411:44:22<32:44:34,  9.43s/it]
 94%|█████████▍| 212500/225000 [411:44:29<30:19:34,  8.73s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.127', 'learning_rate': '4.225e-07', 'epoch': '0.9444', 'num_input_tokens_seen': 697724128, 'train_runtime': '1.482e+06', 'train_tokens_per_second': '470.7'}

 94%|█████████▍| 212500/225000 [411:44:29<30:19:34,  8.73s/it]
 94%|█████████▍| 212501/225000 [411:44:40<32:39:42,  9.41s/it]
 94%|█████████▍| 212502/225000 [411:44:47<29:36:03,  8.53s/it]
 94%|█████████▍| 212503/225000 [411:44:53<27:31:01,  7.93s/it]
 94%|█████████▍| 212504/225000 [411:45:01<27:06:45,  7.81s/it]
 94%|█████████▍| 212505/225000 [411:45:08<26:36:16,  7.67s/it]
 94%|█████████▍| 212506/225000 [411:45:20<30:39:51,  8.84s/it]
 94%|█████████▍| 212507/225000 [411:45:29<31:24:04,  9.05s/it]
 94%|█████████▍| 212508/225000 [411:45:41<33:50:43,  9.75s/it]
 94%|█████████▍| 212509/225000 [411:45:48<31:15:20,  9.01s/it]
 94%|█████████▍| 212510/225000 [411:45:53<27:39:32,  7.97s/it]
 94%|█████████▍| 212511/225000 [411:46:04<30:37:34,  8.83s/it]
 94%|█████████▍| 212512/225000 [411:46:11<28:11:51,  8.13s/it]
 94%|█████████▍| 212513/225000 [411:46:18<27:48:45,  8.02s/it]
 94%|█████████▍| 212514/225000 [411:46:30<31:18:19,  9.03s/it]
 94%|█████████▍| 212515/225000 [411:46:35<27:10:38,  7.84s/it]
 94%|█████████▍| 212516/225000 [411:46:41<25:02:15,  7.22s/it]
 94%|█████████▍| 212517/225000 [411:46:56<33:26:20,  9.64s/it]
 94%|█████████▍| 212518/225000 [411:47:04<31:58:47,  9.22s/it]
 94%|█████████▍| 212519/225000 [411:47:10<28:00:42,  8.08s/it]
 94%|█████████▍| 212520/225000 [411:47:21<30:55:41,  8.92s/it]
 94%|█████████▍| 212521/225000 [411:47:31<32:37:21,  9.41s/it]
 94%|█████████▍| 212522/225000 [411:47:39<31:11:44,  9.00s/it]
 94%|█████████▍| 212523/225000 [411:47:45<27:54:49,  8.05s/it]
 94%|█████████▍| 212524/225000 [411:47:51<26:08:12,  7.54s/it]
 94%|█████████▍| 212525/225000 [411:48:00<26:52:40,  7.76s/it]
 94%|█████████▍| 212526/225000 [411:48:10<29:40:05,  8.56s/it]
 94%|█████████▍| 212527/225000 [411:48:24<35:00:35, 10.10s/it]
 94%|█████████▍| 212528/225000 [411:48:34<35:14:09, 10.17s/it]
 94%|█████████▍| 212529/225000 [411:48:40<31:10:01,  9.00s/it]
 94%|█████████▍| 212530/225000 [411:48:51<32:53:57,  9.50s/it]
 94%|█████████▍| 212531/225000 [411:48:58<30:41:52,  8.86s/it]
 94%|█████████▍| 212532/225000 [411:49:10<33:08:16,  9.57s/it]
 94%|█████████▍| 212533/225000 [411:49:18<31:39:15,  9.14s/it]
 94%|█████████▍| 212534/225000 [411:49:25<29:50:07,  8.62s/it]
 94%|█████████▍| 212535/225000 [411:49:36<32:37:07,  9.42s/it]
 94%|█████████▍| 212536/225000 [411:49:44<30:42:51,  8.87s/it]
 94%|█████████▍| 212537/225000 [411:49:52<30:08:18,  8.71s/it]
 94%|█████████▍| 212538/225000 [411:50:04<32:43:44,  9.45s/it]
 94%|█████████▍| 212539/225000 [411:50:13<33:03:31,  9.55s/it]
 94%|█████████▍| 212540/225000 [411:50:19<29:13:00,  8.44s/it]
 94%|█████████▍| 212541/225000 [411:50:24<25:37:46,  7.41s/it]
 94%|█████████▍| 212542/225000 [411:50:36<30:21:59,  8.78s/it]
 94%|█████████▍| 212543/225000 [411:50:48<33:36:27,  9.71s/it]
 94%|█████████▍| 212544/225000 [411:50:57<33:03:04,  9.55s/it]
 94%|█████████▍| 212545/225000 [411:51:03<29:26:59,  8.51s/it]
 94%|█████████▍| 212546/225000 [411:51:12<29:38:46,  8.57s/it]
 94%|█████████▍| 212547/225000 [411:51:20<28:56:45,  8.37s/it]
 94%|█████████▍| 212548/225000 [411:51:26<27:00:47,  7.81s/it]
 94%|█████████▍| 212549/225000 [411:51:32<24:25:20,  7.06s/it]
 94%|█████████▍| 212550/225000 [411:51:42<27:21:42,  7.91s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.882', 'learning_rate': '4.192e-07', 'epoch': '0.9447', 'num_input_tokens_seen': 697954936, 'train_runtime': '1.483e+06', 'train_tokens_per_second': '470.7'}

 94%|█████████▍| 212550/225000 [411:51:42<27:21:42,  7.91s/it]
 94%|█████████▍| 212551/225000 [411:51:52<29:29:35,  8.53s/it]
 94%|█████████▍| 212552/225000 [411:51:58<27:28:23,  7.95s/it]
 94%|█████████▍| 212553/225000 [411:52:07<28:41:40,  8.30s/it]
 94%|█████████▍| 212554/225000 [411:52:14<26:52:31,  7.77s/it]
 94%|█████████▍| 212555/225000 [411:52:27<32:03:20,  9.27s/it]
 94%|█████████▍| 212556/225000 [411:52:39<35:26:03, 10.25s/it]
 94%|█████████▍| 212557/225000 [411:52:44<30:07:07,  8.71s/it]
 94%|█████████▍| 212558/225000 [411:52:51<27:56:30,  8.08s/it]
 94%|█████████▍| 212559/225000 [411:52:57<25:55:51,  7.50s/it]
 94%|█████████▍| 212560/225000 [411:53:04<25:42:05,  7.44s/it]
 94%|█████████▍| 212561/225000 [411:53:15<29:26:12,  8.52s/it]
 94%|█████████▍| 212562/225000 [411:53:22<27:44:34,  8.03s/it]
 94%|█████████▍| 212563/225000 [411:53:35<32:15:52,  9.34s/it]
 94%|█████████▍| 212564/225000 [411:53:39<27:22:25,  7.92s/it]
 94%|█████████▍| 212565/225000 [411:53:53<33:09:10,  9.60s/it]
 94%|█████████▍| 212566/225000 [411:53:59<29:18:45,  8.49s/it]
 94%|█████████▍| 212567/225000 [411:54:11<33:10:10,  9.60s/it]
 94%|█████████▍| 212568/225000 [411:54:19<31:56:00,  9.25s/it]
 94%|█████████▍| 212569/225000 [411:54:30<33:14:29,  9.63s/it]
 94%|█████████▍| 212570/225000 [411:54:38<31:21:32,  9.08s/it]
 94%|█████████▍| 212571/225000 [411:54:47<31:50:42,  9.22s/it]
 94%|█████████▍| 212572/225000 [411:54:56<31:19:19,  9.07s/it]
 94%|█████████▍| 212573/225000 [411:55:01<27:41:07,  8.02s/it]
 94%|█████████▍| 212574/225000 [411:55:08<26:16:10,  7.61s/it]
 94%|█████████▍| 212575/225000 [411:55:20<30:35:17,  8.86s/it]
 94%|█████████▍| 212576/225000 [411:55:26<27:53:57,  8.08s/it]
 94%|█████████▍| 212577/225000 [411:55:37<31:07:08,  9.02s/it]
 94%|█████████▍| 212578/225000 [411:55:45<29:37:31,  8.59s/it]
 94%|█████████▍| 212579/225000 [411:55:51<26:33:56,  7.70s/it]
 94%|█████████▍| 212580/225000 [411:55:58<26:42:34,  7.74s/it]
 94%|█████████▍| 212581/225000 [411:56:05<25:25:00,  7.37s/it]
 94%|█████████▍| 212582/225000 [411:56:15<28:14:26,  8.19s/it]
 94%|█████████▍| 212583/225000 [411:56:24<29:02:59,  8.42s/it]
 94%|█████████▍| 212584/225000 [411:56:33<29:26:33,  8.54s/it]
 94%|█████████▍| 212585/225000 [411:56:40<28:18:17,  8.21s/it]
 94%|█████████▍| 212586/225000 [411:56:52<31:51:26,  9.24s/it]
 94%|█████████▍| 212587/225000 [411:57:00<30:48:29,  8.93s/it]
 94%|█████████▍| 212588/225000 [411:57:08<29:35:27,  8.58s/it]
 94%|█████████▍| 212589/225000 [411:57:17<30:04:29,  8.72s/it]
 94%|█████████▍| 212590/225000 [411:57:24<28:26:05,  8.25s/it]
 94%|█████████▍| 212591/225000 [411:57:32<27:57:32,  8.11s/it]
 94%|█████████▍| 212592/225000 [411:57:40<28:21:11,  8.23s/it]
 94%|█████████▍| 212593/225000 [411:57:54<34:28:02, 10.00s/it]
 94%|█████████▍| 212594/225000 [411:58:03<32:39:12,  9.48s/it]
 94%|█████████▍| 212595/225000 [411:58:10<29:53:51,  8.68s/it]
 94%|█████████▍| 212596/225000 [411:58:15<26:33:34,  7.71s/it]
 94%|█████████▍| 212597/225000 [411:58:25<28:43:44,  8.34s/it]
 94%|█████████▍| 212598/225000 [411:58:36<31:55:01,  9.26s/it]
 94%|█████████▍| 212599/225000 [411:58:45<31:31:56,  9.15s/it]
 94%|█████████▍| 212600/225000 [411:58:51<27:42:38,  8.05s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.478', 'learning_rate': '4.159e-07', 'epoch': '0.9449', 'num_input_tokens_seen': 698185496, 'train_runtime': '1.483e+06', 'train_tokens_per_second': '470.8'}

 94%|█████████▍| 212600/225000 [411:58:51<27:42:38,  8.05s/it]
 94%|█████████▍| 212601/225000 [411:59:02<30:54:27,  8.97s/it]
 94%|█████████▍| 212602/225000 [411:59:07<26:46:38,  7.78s/it]
 94%|█████████▍| 212603/225000 [411:59:19<31:45:44,  9.22s/it]
 94%|█████████▍| 212604/225000 [411:59:29<31:53:22,  9.26s/it]
 94%|█████████▍| 212605/225000 [411:59:41<34:48:34, 10.11s/it]
 94%|█████████▍| 212606/225000 [411:59:46<29:51:04,  8.67s/it]
 94%|█████████▍| 212607/225000 [412:00:00<34:54:46, 10.14s/it]
 94%|█████████▍| 212608/225000 [412:00:05<29:42:33,  8.63s/it]
 94%|█████████▍| 212609/225000 [412:00:16<31:56:56,  9.28s/it]
 94%|█████████▍| 212610/225000 [412:00:24<31:05:53,  9.04s/it]
 94%|█████████▍| 212611/225000 [412:00:30<28:00:21,  8.14s/it]
 94%|█████████▍| 212612/225000 [412:00:37<26:59:39,  7.84s/it]
 94%|█████████▍| 212613/225000 [412:00:44<26:03:09,  7.57s/it]
 94%|█████████▍| 212614/225000 [412:01:02<36:43:14, 10.67s/it]
 94%|█████████▍| 212615/225000 [412:01:10<33:27:14,  9.72s/it]
 94%|█████████▍| 212616/225000 [412:01:17<31:23:24,  9.13s/it]
 94%|█████████▍| 212617/225000 [412:01:25<30:05:56,  8.75s/it]
 94%|█████████▍| 212618/225000 [412:01:39<35:45:56, 10.40s/it]
 94%|█████████▍| 212619/225000 [412:01:49<34:44:32, 10.10s/it]
 94%|█████████▍| 212620/225000 [412:01:55<30:25:40,  8.85s/it]
 94%|█████████▍| 212621/225000 [412:02:05<31:26:50,  9.15s/it]
 94%|█████████▍| 212622/225000 [412:02:13<30:33:17,  8.89s/it]
 94%|█████████▍| 212623/225000 [412:02:19<27:38:39,  8.04s/it]
 94%|█████████▍| 212624/225000 [412:02:32<32:41:25,  9.51s/it]
 94%|█████████▍| 212625/225000 [412:02:40<31:40:57,  9.22s/it]
 95%|█████████▍| 212626/225000 [412:02:50<31:40:54,  9.22s/it]
 95%|█████████▍| 212627/225000 [412:02:55<27:58:02,  8.14s/it]
 95%|█████████▍| 212628/225000 [412:03:03<27:28:16,  7.99s/it]
 95%|█████████▍| 212629/225000 [412:03:17<33:34:53,  9.77s/it]
 95%|█████████▍| 212630/225000 [412:03:24<31:17:06,  9.10s/it]
 95%|█████████▍| 212631/225000 [412:03:31<28:28:42,  8.29s/it]
 95%|█████████▍| 212632/225000 [412:03:37<26:43:12,  7.78s/it]
 95%|█████████▍| 212633/225000 [412:03:45<26:15:30,  7.64s/it]
 95%|█████████▍| 212634/225000 [412:03:58<32:12:18,  9.38s/it]
 95%|█████████▍| 212635/225000 [412:04:05<29:11:49,  8.50s/it]
 95%|█████████▍| 212636/225000 [412:04:12<27:52:03,  8.11s/it]
 95%|█████████▍| 212637/225000 [412:04:24<32:12:52,  9.38s/it]
 95%|█████████▍| 212638/225000 [412:04:33<31:48:59,  9.27s/it]
 95%|█████████▍| 212639/225000 [412:04:44<33:20:46,  9.71s/it]
 95%|█████████▍| 212640/225000 [412:04:50<29:57:54,  8.73s/it]
 95%|█████████▍| 212641/225000 [412:04:58<29:07:01,  8.48s/it]
 95%|█████████▍| 212642/225000 [412:05:06<28:52:25,  8.41s/it]
 95%|█████████▍| 212643/225000 [412:05:14<27:43:41,  8.08s/it]
 95%|█████████▍| 212644/225000 [412:05:19<24:54:58,  7.26s/it]
 95%|█████████▍| 212645/225000 [412:05:25<23:28:43,  6.84s/it]
 95%|█████████▍| 212646/225000 [412:05:39<31:15:13,  9.11s/it]
 95%|█████████▍| 212647/225000 [412:05:51<34:19:56, 10.01s/it]
 95%|█████████▍| 212648/225000 [412:05:58<30:59:03,  9.03s/it]
 95%|█████████▍| 212649/225000 [412:06:04<27:48:07,  8.10s/it]
 95%|█████████▍| 212650/225000 [412:06:17<32:28:00,  9.46s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.068', 'learning_rate': '4.125e-07', 'epoch': '0.9451', 'num_input_tokens_seen': 698421928, 'train_runtime': '1.484e+06', 'train_tokens_per_second': '470.8'}

 95%|█████████▍| 212650/225000 [412:06:17<32:28:00,  9.46s/it]
 95%|█████████▍| 212651/225000 [412:06:20<26:31:36,  7.73s/it]
 95%|█████████▍| 212652/225000 [412:06:30<28:47:43,  8.40s/it]
 95%|█████████▍| 212653/225000 [412:06:37<26:58:27,  7.86s/it]
 95%|█████████▍| 212654/225000 [412:06:50<32:41:00,  9.53s/it]
 95%|█████████▍| 212655/225000 [412:06:58<31:08:51,  9.08s/it]
 95%|█████████▍| 212656/225000 [412:07:07<31:04:25,  9.06s/it]
 95%|█████████▍| 212657/225000 [412:07:17<31:37:58,  9.23s/it]
 95%|█████████▍| 212658/225000 [412:07:26<30:57:10,  9.03s/it]
 95%|█████████▍| 212659/225000 [412:07:32<27:45:22,  8.10s/it]
 95%|█████████▍| 212660/225000 [412:07:38<25:32:00,  7.45s/it]
 95%|█████████▍| 212661/225000 [412:07:47<27:43:31,  8.09s/it]
 95%|█████████▍| 212662/225000 [412:07:56<29:02:13,  8.47s/it]
 95%|█████████▍| 212663/225000 [412:08:03<27:11:17,  7.93s/it]
 95%|█████████▍| 212664/225000 [412:08:12<28:27:04,  8.30s/it]
 95%|█████████▍| 212665/225000 [412:08:19<26:32:23,  7.75s/it]
 95%|█████████▍| 212666/225000 [412:08:34<33:54:20,  9.90s/it]
 95%|█████████▍| 212667/225000 [412:08:40<30:08:14,  8.80s/it]
 95%|█████████▍| 212668/225000 [412:08:48<29:28:00,  8.60s/it]
 95%|█████████▍| 212669/225000 [412:08:56<28:55:53,  8.45s/it]
 95%|█████████▍| 212670/225000 [412:09:07<31:31:58,  9.21s/it]
 95%|█████████▍| 212671/225000 [412:09:14<28:59:43,  8.47s/it]
 95%|█████████▍| 212672/225000 [412:09:25<31:58:41,  9.34s/it]
 95%|█████████▍| 212673/225000 [412:09:29<26:44:32,  7.81s/it]
 95%|█████████▍| 212674/225000 [412:09:36<25:27:31,  7.44s/it]
 95%|█████████▍| 212675/225000 [412:09:48<30:17:38,  8.85s/it]
 95%|█████████▍| 212676/225000 [412:09:53<26:27:14,  7.73s/it]
 95%|█████████▍| 212677/225000 [412:10:09<34:36:06, 10.11s/it]
 95%|█████████▍| 212678/225000 [412:10:17<32:53:58,  9.61s/it]
 95%|█████████▍| 212679/225000 [412:10:23<28:58:49,  8.47s/it]
 95%|█████████▍| 212680/225000 [412:10:29<26:29:46,  7.74s/it]
 95%|█████████▍| 212681/225000 [412:10:36<25:22:58,  7.42s/it]
 95%|█████████▍| 212682/225000 [412:10:44<26:09:13,  7.64s/it]
 95%|█████████▍| 212683/225000 [412:10:53<27:06:04,  7.92s/it]
 95%|█████████▍| 212684/225000 [412:11:02<28:19:30,  8.28s/it]
 95%|█████████▍| 212685/225000 [412:11:09<27:26:35,  8.02s/it]
 95%|█████████▍| 212686/225000 [412:11:15<25:32:16,  7.47s/it]
 95%|█████████▍| 212687/225000 [412:11:26<28:49:28,  8.43s/it]
 95%|█████████▍| 212688/225000 [412:11:34<28:41:42,  8.39s/it]
 95%|█████████▍| 212689/225000 [412:11:42<28:01:31,  8.20s/it]
 95%|█████████▍| 212690/225000 [412:11:52<29:21:10,  8.58s/it]
 95%|█████████▍| 212691/225000 [412:12:00<29:16:42,  8.56s/it]
 95%|█████████▍| 212692/225000 [412:12:08<28:13:37,  8.26s/it]
 95%|█████████▍| 212693/225000 [412:12:14<26:39:10,  7.80s/it]
 95%|█████████▍| 212694/225000 [412:12:28<32:57:36,  9.64s/it]
 95%|█████████▍| 212695/225000 [412:12:34<28:51:16,  8.44s/it]
 95%|█████████▍| 212696/225000 [412:12:45<31:12:22,  9.13s/it]
 95%|█████████▍| 212697/225000 [412:12:57<34:23:33, 10.06s/it]
 95%|█████████▍| 212698/225000 [412:13:03<30:50:25,  9.02s/it]
 95%|█████████▍| 212699/225000 [412:13:13<31:50:08,  9.32s/it]
 95%|█████████▍| 212700/225000 [412:13:20<29:07:30,  8.52s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.659', 'learning_rate': '4.092e-07', 'epoch': '0.9453', 'num_input_tokens_seen': 698646576, 'train_runtime': '1.484e+06', 'train_tokens_per_second': '470.8'}

 95%|█████████▍| 212700/225000 [412:13:20<29:07:30,  8.52s/it]
 95%|█████████▍| 212701/225000 [412:13:26<25:58:58,  7.61s/it]
 95%|█████████▍| 212702/225000 [412:13:35<27:50:35,  8.15s/it]
 95%|█████████▍| 212703/225000 [412:13:44<28:35:45,  8.37s/it]
 95%|█████████▍| 212704/225000 [412:13:55<31:43:37,  9.29s/it]
 95%|█████████▍| 212705/225000 [412:14:09<35:52:23, 10.50s/it]
 95%|█████████▍| 212706/225000 [412:14:18<34:47:18, 10.19s/it]
 95%|█████████▍| 212707/225000 [412:14:25<31:22:42,  9.19s/it]
 95%|█████████▍| 212708/225000 [412:14:32<28:50:04,  8.44s/it]
 95%|█████████▍| 212709/225000 [412:14:44<32:53:35,  9.63s/it]
 95%|█████████▍| 212710/225000 [412:14:55<33:45:14,  9.89s/it]
 95%|█████████▍| 212711/225000 [412:15:00<28:40:05,  8.40s/it]
 95%|█████████▍| 212712/225000 [412:15:04<24:56:18,  7.31s/it]
 95%|█████████▍| 212713/225000 [412:15:17<30:04:07,  8.81s/it]
 95%|█████████▍| 212714/225000 [412:15:28<32:59:37,  9.67s/it]
 95%|█████████▍| 212715/225000 [412:15:36<31:15:05,  9.16s/it]
 95%|█████████▍| 212716/225000 [412:15:44<30:00:44,  8.80s/it]
 95%|█████████▍| 212717/225000 [412:15:51<28:01:44,  8.22s/it]
 95%|█████████▍| 212718/225000 [412:15:59<27:52:03,  8.17s/it]
 95%|█████████▍| 212719/225000 [412:16:16<36:44:52, 10.77s/it]
 95%|█████████▍| 212720/225000 [412:16:24<34:22:15, 10.08s/it]
 95%|█████████▍| 212721/225000 [412:16:32<32:14:25,  9.45s/it]
 95%|█████████▍| 212722/225000 [412:16:40<30:18:56,  8.89s/it]
 95%|█████████▍| 212723/225000 [412:16:52<33:48:28,  9.91s/it]
 95%|█████████▍| 212724/225000 [412:16:59<30:07:51,  8.84s/it]
 95%|█████████▍| 212725/225000 [412:17:05<27:23:15,  8.03s/it]
 95%|█████████▍| 212726/225000 [412:17:10<24:28:33,  7.18s/it]
 95%|█████████▍| 212727/225000 [412:17:22<29:24:07,  8.62s/it]
 95%|█████████▍| 212728/225000 [412:17:32<30:34:29,  8.97s/it]
 95%|█████████▍| 212729/225000 [412:17:40<30:16:32,  8.88s/it]
 95%|█████████▍| 212730/225000 [412:17:48<28:55:09,  8.48s/it]
 95%|█████████▍| 212731/225000 [412:17:57<29:19:38,  8.61s/it]
 95%|█████████▍| 212732/225000 [412:18:04<28:05:10,  8.24s/it]
 95%|█████████▍| 212733/225000 [412:18:17<32:19:17,  9.49s/it]
 95%|█████████▍| 212734/225000 [412:18:29<34:47:08, 10.21s/it]
 95%|█████████▍| 212735/225000 [412:18:40<36:31:12, 10.72s/it]
 95%|█████████▍| 212736/225000 [412:18:48<33:05:42,  9.71s/it]
 95%|█████████▍| 212737/225000 [412:18:59<34:41:22, 10.18s/it]
 95%|█████████▍| 212738/225000 [412:19:07<32:15:50,  9.47s/it]
 95%|█████████▍| 212739/225000 [412:19:14<30:18:18,  8.90s/it]
 95%|█████████▍| 212740/225000 [412:19:23<29:40:50,  8.72s/it]
 95%|█████████▍| 212741/225000 [412:19:34<32:37:50,  9.58s/it]
 95%|█████████▍| 212742/225000 [412:19:42<30:27:50,  8.95s/it]
 95%|█████████▍| 212743/225000 [412:19:48<27:57:16,  8.21s/it]
 95%|█████████▍| 212744/225000 [412:19:57<28:42:53,  8.43s/it]
 95%|█████████▍| 212745/225000 [412:20:08<30:44:20,  9.03s/it]
 95%|█████████▍| 212746/225000 [412:20:16<29:36:49,  8.70s/it]
 95%|█████████▍| 212747/225000 [412:20:25<30:14:14,  8.88s/it]
 95%|█████████▍| 212748/225000 [412:20:35<31:29:52,  9.26s/it]
 95%|█████████▍| 212749/225000 [412:20:41<28:10:42,  8.28s/it]
 95%|█████████▍| 212750/225000 [412:20:48<26:32:17,  7.80s/it]
                                                              
{'loss': '1.325', 'grad_norm': '1.374', 'learning_rate': '4.06e-07', 'epoch': '0.9456', 'num_input_tokens_seen': 698888544, 'train_runtime': '1.484e+06', 'train_tokens_per_second': '470.8'}

 95%|█████████▍| 212750/225000 [412:20:48<26:32:17,  7.80s/it]
 95%|█████████▍| 212751/225000 [412:20:59<29:49:37,  8.77s/it]
 95%|█████████▍| 212752/225000 [412:21:06<28:32:07,  8.39s/it]
 95%|█████████▍| 212753/225000 [412:21:14<27:30:14,  8.08s/it]
 95%|█████████▍| 212754/225000 [412:21:22<27:22:39,  8.05s/it]
 95%|█████████▍| 212755/225000 [412:21:29<26:25:16,  7.77s/it]
 95%|█████████▍| 212756/225000 [412:21:35<25:10:20,  7.40s/it]
 95%|█████████▍| 212757/225000 [412:21:41<23:34:59,  6.93s/it]
 95%|█████████▍| 212758/225000 [412:21:59<34:17:21, 10.08s/it]
 95%|█████████▍| 212759/225000 [412:22:05<30:46:23,  9.05s/it]
 95%|█████████▍| 212760/225000 [412:22:15<31:58:54,  9.41s/it]
 95%|█████████▍| 212761/225000 [412:22:20<27:34:03,  8.11s/it]
 95%|█████████▍| 212762/225000 [412:22:33<31:37:47,  9.30s/it]
 95%|█████████▍| 212763/225000 [412:22:41<30:16:35,  8.91s/it]
 95%|█████████▍| 212764/225000 [412:22:52<32:34:39,  9.58s/it]
 95%|█████████▍| 212765/225000 [412:22:58<29:00:04,  8.53s/it]
 95%|█████████▍| 212766/225000 [412:23:03<25:38:13,  7.54s/it]
 95%|█████████▍| 212767/225000 [412:23:15<30:23:34,  8.94s/it]
 95%|█████████▍| 212768/225000 [412:23:26<31:50:23,  9.37s/it]
 95%|█████████▍| 212769/225000 [412:23:37<33:50:54,  9.96s/it]
 95%|█████████▍| 212770/225000 [412:23:42<28:33:14,  8.41s/it]
 95%|█████████▍| 212771/225000 [412:23:49<27:13:18,  8.01s/it]
 95%|█████████▍| 212772/225000 [412:23:57<27:40:26,  8.15s/it]
 95%|█████████▍| 212773/225000 [412:24:03<25:34:24,  7.53s/it]
 95%|█████████▍| 212774/225000 [412:24:11<25:10:36,  7.41s/it]
 95%|█████████▍| 212775/225000 [412:24:24<31:24:09,  9.25s/it]
 95%|█████████▍| 212776/225000 [412:24:36<34:11:52, 10.07s/it]
 95%|█████████▍| 212777/225000 [412:24:41<28:27:58,  8.38s/it]
 95%|█████████▍| 212778/225000 [412:24:49<28:32:15,  8.41s/it]
 95%|█████████▍| 212779/225000 [412:24:54<24:46:44,  7.30s/it]
 95%|█████████▍| 212780/225000 [412:25:00<23:42:14,  6.98s/it]
 95%|█████████▍| 212781/225000 [412:25:12<29:09:01,  8.59s/it]
 95%|█████████▍| 212782/225000 [412:25:24<32:32:56,  9.59s/it]
 95%|█████████▍| 212783/225000 [412:25:30<28:51:14,  8.50s/it]
 95%|█████████▍| 212784/225000 [412:25:41<31:23:58,  9.25s/it]
 95%|█████████▍| 212785/225000 [412:25:53<33:57:27, 10.01s/it]
 95%|█████████▍| 212786/225000 [412:26:02<32:36:35,  9.61s/it]
 95%|█████████▍| 212787/225000 [412:26:07<28:28:12,  8.39s/it]
 95%|█████████▍| 212788/225000 [412:26:17<30:09:29,  8.89s/it]
 95%|█████████▍| 212789/225000 [412:26:22<26:03:17,  7.68s/it]
 95%|█████████▍| 212790/225000 [412:26:33<29:04:13,  8.57s/it]
 95%|█████████▍| 212791/225000 [412:26:36<23:58:22,  7.07s/it]
 95%|█████████▍| 212792/225000 [412:26:46<27:10:36,  8.01s/it]
 95%|█████████▍| 212793/225000 [412:26:52<24:13:59,  7.15s/it]
 95%|█████████▍| 212794/225000 [412:26:59<24:33:02,  7.24s/it]
 95%|█████████▍| 212795/225000 [412:27:10<28:45:58,  8.48s/it]
 95%|█████████▍| 212796/225000 [412:27:15<25:14:07,  7.44s/it]
 95%|█████████▍| 212797/225000 [412:27:25<27:31:09,  8.12s/it]
 95%|█████████▍| 212798/225000 [412:27:31<25:04:38,  7.40s/it]
 95%|█████████▍| 212799/225000 [412:27:37<24:08:41,  7.12s/it]
 95%|█████████▍| 212800/225000 [412:27:44<23:19:46,  6.88s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.652', 'learning_rate': '4.027e-07', 'epoch': '0.9458', 'num_input_tokens_seen': 699119592, 'train_runtime': '1.485e+06', 'train_tokens_per_second': '470.8'}

 95%|█████████▍| 212800/225000 [412:27:44<23:19:46,  6.88s/it]
 95%|█████████▍| 212801/225000 [412:27:57<29:22:34,  8.67s/it]
 95%|█████████▍| 212802/225000 [412:28:07<31:25:58,  9.28s/it]
 95%|█████████▍| 212803/225000 [412:28:13<28:11:41,  8.32s/it]
 95%|█████████▍| 212804/225000 [412:28:20<26:53:17,  7.94s/it]
 95%|█████████▍| 212805/225000 [412:28:34<32:22:55,  9.56s/it]
 95%|█████████▍| 212806/225000 [412:28:41<30:14:32,  8.93s/it]
 95%|█████████▍| 212807/225000 [412:28:49<29:09:41,  8.61s/it]
 95%|█████████▍| 212808/225000 [412:28:58<29:58:04,  8.85s/it]
 95%|█████████▍| 212809/225000 [412:29:07<29:12:50,  8.63s/it]
 95%|█████████▍| 212810/225000 [412:29:21<34:42:25, 10.25s/it]
 95%|█████████▍| 212811/225000 [412:29:27<31:10:02,  9.21s/it]
 95%|█████████▍| 212812/225000 [412:29:34<28:19:31,  8.37s/it]
 95%|█████████▍| 212813/225000 [412:29:42<28:18:26,  8.36s/it]
 95%|█████████▍| 212814/225000 [412:29:54<31:46:36,  9.39s/it]
 95%|█████████▍| 212815/225000 [412:30:04<32:19:08,  9.55s/it]
 95%|█████████▍| 212816/225000 [412:30:08<27:04:47,  8.00s/it]
 95%|█████████▍| 212817/225000 [412:30:15<25:41:10,  7.59s/it]
 95%|█████████▍| 212818/225000 [412:30:28<31:20:39,  9.26s/it]
 95%|█████████▍| 212819/225000 [412:30:36<29:49:33,  8.81s/it]
 95%|█████████▍| 212820/225000 [412:30:46<31:41:12,  9.37s/it]
 95%|█████████▍| 212821/225000 [412:30:57<33:19:11,  9.85s/it]
 95%|█████████▍| 212822/225000 [412:31:07<32:37:32,  9.64s/it]
 95%|█████████▍| 212823/225000 [412:31:13<29:29:01,  8.72s/it]
 95%|█████████▍| 212824/225000 [412:31:19<26:08:30,  7.73s/it]
 95%|█████████▍| 212825/225000 [412:31:26<26:21:12,  7.79s/it]
 95%|█████████▍| 212826/225000 [412:31:33<25:23:43,  7.51s/it]
 95%|█████████▍| 212827/225000 [412:31:47<31:45:22,  9.39s/it]
 95%|█████████▍| 212828/225000 [412:31:54<29:26:29,  8.71s/it]
 95%|█████████▍| 212829/225000 [412:32:03<29:38:49,  8.77s/it]
 95%|█████████▍| 212830/225000 [412:32:14<31:53:30,  9.43s/it]
 95%|█████████▍| 212831/225000 [412:32:21<28:59:45,  8.58s/it]
 95%|█████████▍| 212832/225000 [412:32:36<35:33:39, 10.52s/it]
 95%|█████████▍| 212833/225000 [412:32:42<31:22:40,  9.28s/it]
 95%|█████████▍| 212834/225000 [412:32:50<30:19:48,  8.97s/it]
 95%|█████████▍| 212835/225000 [412:33:00<31:11:26,  9.23s/it]
 95%|█████████▍| 212836/225000 [412:33:08<29:54:28,  8.85s/it]
 95%|█████████▍| 212837/225000 [412:33:16<28:24:03,  8.41s/it]
 95%|█████████▍| 212838/225000 [412:33:29<33:29:53,  9.92s/it]
 95%|█████████▍| 212839/225000 [412:33:35<29:18:59,  8.68s/it]
 95%|█████████▍| 212840/225000 [412:33:42<27:46:46,  8.22s/it]
 95%|█████████▍| 212841/225000 [412:33:49<26:28:17,  7.84s/it]
 95%|█████████▍| 212842/225000 [412:33:55<24:26:33,  7.24s/it]
 95%|█████████▍| 212843/225000 [412:34:08<30:12:39,  8.95s/it]
 95%|█████████▍| 212844/225000 [412:34:13<26:52:03,  7.96s/it]
 95%|█████████▍| 212845/225000 [412:34:26<31:44:51,  9.40s/it]
 95%|█████████▍| 212846/225000 [412:34:34<29:49:34,  8.83s/it]
 95%|█████████▍| 212847/225000 [412:34:44<31:50:40,  9.43s/it]
 95%|█████████▍| 212848/225000 [412:34:54<32:21:55,  9.59s/it]
 95%|█████████▍| 212849/225000 [412:35:01<29:30:05,  8.74s/it]
 95%|█████████▍| 212850/225000 [412:35:09<29:01:31,  8.60s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.404', 'learning_rate': '3.994e-07', 'epoch': '0.946', 'num_input_tokens_seen': 699351560, 'train_runtime': '1.485e+06', 'train_tokens_per_second': '470.8'}

 95%|█████████▍| 212850/225000 [412:35:09<29:01:31,  8.60s/it]
 95%|█████████▍| 212851/225000 [412:35:16<27:08:38,  8.04s/it]
 95%|█████████▍| 212852/225000 [412:35:23<25:41:08,  7.61s/it]
 95%|█████████▍| 212853/225000 [412:35:36<31:15:22,  9.26s/it]
 95%|█████████▍| 212854/225000 [412:35:47<33:09:03,  9.83s/it]
 95%|█████████▍| 212855/225000 [412:35:56<31:47:01,  9.42s/it]
 95%|█████████▍| 212856/225000 [412:36:01<27:48:14,  8.24s/it]
 95%|█████████▍| 212857/225000 [412:36:07<25:54:51,  7.68s/it]
 95%|█████████▍| 212858/225000 [412:36:13<23:38:48,  7.01s/it]
 95%|█████████▍| 212859/225000 [412:36:24<27:36:19,  8.19s/it]
 95%|█████████▍| 212860/225000 [412:36:33<29:09:22,  8.65s/it]
 95%|█████████▍| 212861/225000 [412:36:44<31:03:04,  9.21s/it]
 95%|█████████▍| 212862/225000 [412:36:52<30:14:15,  8.97s/it]
 95%|█████████▍| 212863/225000 [412:36:59<27:20:25,  8.11s/it]
 95%|█████████▍| 212864/225000 [412:37:04<24:34:55,  7.29s/it]
 95%|█████████▍| 212865/225000 [412:37:11<24:26:11,  7.25s/it]
 95%|█████████▍| 212866/225000 [412:37:28<34:06:09, 10.12s/it]
 95%|█████████▍| 212867/225000 [412:37:35<31:15:43,  9.28s/it]
 95%|█████████▍| 212868/225000 [412:37:43<29:55:40,  8.88s/it]
 95%|█████████▍| 212869/225000 [412:37:52<29:27:15,  8.74s/it]
 95%|█████████▍| 212870/225000 [412:38:02<31:34:47,  9.37s/it]
 95%|█████████▍| 212871/225000 [412:38:09<29:13:16,  8.67s/it]
 95%|█████████▍| 212872/225000 [412:38:18<29:00:47,  8.61s/it]
 95%|█████████▍| 212873/225000 [412:38:25<27:24:17,  8.14s/it]
 95%|█████████▍| 212874/225000 [412:38:35<29:10:59,  8.66s/it]
 95%|█████████▍| 212875/225000 [412:38:44<29:42:41,  8.82s/it]
 95%|█████████▍| 212876/225000 [412:38:53<29:24:14,  8.73s/it]
 95%|█████████▍| 212877/225000 [412:39:00<28:29:09,  8.46s/it]
 95%|█████████▍| 212878/225000 [412:39:06<25:19:19,  7.52s/it]
 95%|█████████▍| 212879/225000 [412:39:12<24:04:23,  7.15s/it]
 95%|█████████▍| 212880/225000 [412:39:19<23:28:03,  6.97s/it]
 95%|█████████▍| 212881/225000 [412:39:30<27:42:02,  8.23s/it]
 95%|█████████▍| 212882/225000 [412:39:38<27:41:36,  8.23s/it]
 95%|█████████▍| 212883/225000 [412:39:43<24:24:17,  7.25s/it]
 95%|█████████▍| 212884/225000 [412:39:53<27:28:43,  8.16s/it]
 95%|█████████▍| 212885/225000 [412:39:58<23:44:40,  7.06s/it]
 95%|█████████▍| 212886/225000 [412:40:11<29:58:24,  8.91s/it]
 95%|█████████▍| 212887/225000 [412:40:22<32:07:50,  9.55s/it]
 95%|█████████▍| 212888/225000 [412:40:26<26:52:03,  7.99s/it]
 95%|█████████▍| 212889/225000 [412:40:35<27:44:42,  8.25s/it]
 95%|█████████▍| 212890/225000 [412:40:46<30:01:54,  8.93s/it]
 95%|█████████▍| 212891/225000 [412:40:53<28:05:15,  8.35s/it]
 95%|█████████▍| 212892/225000 [412:40:58<24:53:06,  7.40s/it]
 95%|█████████▍| 212893/225000 [412:41:09<28:28:02,  8.46s/it]
 95%|█████████▍| 212894/225000 [412:41:19<30:34:23,  9.09s/it]
 95%|█████████▍| 212895/225000 [412:41:25<27:26:20,  8.16s/it]
 95%|█████████▍| 212896/225000 [412:41:34<28:07:07,  8.36s/it]
 95%|█████████▍| 212897/225000 [412:41:42<28:02:00,  8.34s/it]
 95%|█████████▍| 212898/225000 [412:41:48<25:43:04,  7.65s/it]
 95%|█████████▍| 212899/225000 [412:41:56<25:55:23,  7.71s/it]
 95%|█████████▍| 212900/225000 [412:42:07<28:38:55,  8.52s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.018', 'learning_rate': '3.962e-07', 'epoch': '0.9462', 'num_input_tokens_seen': 699577872, 'train_runtime': '1.486e+06', 'train_tokens_per_second': '470.9'}

 95%|█████████▍| 212900/225000 [412:42:07<28:38:55,  8.52s/it]
 95%|█████████▍| 212901/225000 [412:42:16<29:43:17,  8.84s/it]
 95%|█████████▍| 212902/225000 [412:42:22<26:17:12,  7.82s/it]
 95%|█████████▍| 212903/225000 [412:42:37<34:11:51, 10.18s/it]
 95%|█████████▍| 212904/225000 [412:42:48<34:39:08, 10.31s/it]
 95%|█████████▍| 212905/225000 [412:42:58<34:29:46, 10.27s/it]
 95%|█████████▍| 212906/225000 [412:43:04<29:34:55,  8.81s/it]
 95%|█████████▍| 212907/225000 [412:43:13<30:35:37,  9.11s/it]
 95%|█████████▍| 212908/225000 [412:43:22<29:46:41,  8.87s/it]
 95%|█████████▍| 212909/225000 [412:43:28<27:05:28,  8.07s/it]
 95%|█████████▍| 212910/225000 [412:43:40<31:21:14,  9.34s/it]
 95%|█████████▍| 212911/225000 [412:43:49<30:43:16,  9.15s/it]
 95%|█████████▍| 212912/225000 [412:44:00<32:22:25,  9.64s/it]
 95%|█████████▍| 212913/225000 [412:44:05<27:47:41,  8.28s/it]
 95%|█████████▍| 212914/225000 [412:44:12<26:23:55,  7.86s/it]
 95%|█████████▍| 212915/225000 [412:44:25<32:04:37,  9.56s/it]
 95%|█████████▍| 212916/225000 [412:44:29<26:36:36,  7.93s/it]
 95%|█████████▍| 212917/225000 [412:44:38<27:37:59,  8.23s/it]
 95%|█████████▍| 212918/225000 [412:44:44<25:29:55,  7.60s/it]
 95%|█████████▍| 212919/225000 [412:44:53<26:52:53,  8.01s/it]
 95%|█████████▍| 212920/225000 [412:45:04<29:28:40,  8.78s/it]
 95%|█████████▍| 212921/225000 [412:45:10<27:06:19,  8.08s/it]
 95%|█████████▍| 212922/225000 [412:45:18<27:02:46,  8.06s/it]
 95%|█████████▍| 212923/225000 [412:45:31<31:25:45,  9.37s/it]
 95%|█████████▍| 212924/225000 [412:45:41<32:18:03,  9.63s/it]
 95%|█████████▍| 212925/225000 [412:45:49<30:22:42,  9.06s/it]
 95%|█████████▍| 212926/225000 [412:45:54<26:48:38,  7.99s/it]
 95%|█████████▍| 212927/225000 [412:46:03<27:06:46,  8.08s/it]
 95%|█████████▍| 212928/225000 [412:46:16<31:57:21,  9.53s/it]
 95%|█████████▍| 212929/225000 [412:46:20<27:10:05,  8.10s/it]
 95%|█████████▍| 212930/225000 [412:46:26<25:03:04,  7.47s/it]
 95%|█████████▍| 212931/225000 [412:46:34<24:47:30,  7.40s/it]
 95%|█████████▍| 212932/225000 [412:46:44<27:58:34,  8.35s/it]
 95%|█████████▍| 212933/225000 [412:46:49<24:56:42,  7.44s/it]
 95%|█████████▍| 212934/225000 [412:46:55<22:46:58,  6.80s/it]
 95%|█████████▍| 212935/225000 [412:47:02<23:06:29,  6.90s/it]
 95%|█████████▍| 212936/225000 [412:47:06<20:48:05,  6.21s/it]
 95%|█████████▍| 212937/225000 [412:47:15<22:51:13,  6.82s/it]
 95%|█████████▍| 212938/225000 [412:47:31<32:04:40,  9.57s/it]
 95%|█████████▍| 212939/225000 [412:47:37<28:23:46,  8.48s/it]
 95%|█████████▍| 212940/225000 [412:47:43<26:12:45,  7.82s/it]
 95%|█████████▍| 212941/225000 [412:47:49<24:34:14,  7.34s/it]
 95%|█████████▍| 212942/225000 [412:47:58<26:26:38,  7.90s/it]
 95%|█████████▍| 212943/225000 [412:48:06<25:47:00,  7.70s/it]
 95%|█████████▍| 212944/225000 [412:48:12<24:42:50,  7.38s/it]
 95%|█████████▍| 212945/225000 [412:48:19<23:53:16,  7.13s/it]
 95%|█████████▍| 212946/225000 [412:48:35<32:38:47,  9.75s/it]
 95%|█████████▍| 212947/225000 [412:48:41<29:14:38,  8.73s/it]
 95%|█████████▍| 212948/225000 [412:48:49<28:05:32,  8.39s/it]
 95%|█████████▍| 212949/225000 [412:48:54<25:12:10,  7.53s/it]
 95%|█████████▍| 212950/225000 [412:49:05<28:22:17,  8.48s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.203', 'learning_rate': '3.93e-07', 'epoch': '0.9464', 'num_input_tokens_seen': 699806232, 'train_runtime': '1.486e+06', 'train_tokens_per_second': '470.9'}

 95%|█████████▍| 212950/225000 [412:49:05<28:22:17,  8.48s/it]
 95%|█████████▍| 212951/225000 [412:49:19<33:46:50, 10.09s/it]
 95%|█████████▍| 212952/225000 [412:49:28<33:06:48,  9.89s/it]
 95%|█████████▍| 212953/225000 [412:49:39<33:41:22, 10.07s/it]
 95%|█████████▍| 212954/225000 [412:49:46<30:56:47,  9.25s/it]
 95%|█████████▍| 212955/225000 [412:49:53<28:23:50,  8.49s/it]
 95%|█████████▍| 212956/225000 [412:50:02<29:28:21,  8.81s/it]
 95%|█████████▍| 212957/225000 [412:50:11<29:42:34,  8.88s/it]
 95%|█████████▍| 212958/225000 [412:50:18<27:34:15,  8.24s/it]
 95%|█████████▍| 212959/225000 [412:50:25<25:57:31,  7.76s/it]
 95%|█████████▍| 212960/225000 [412:50:31<25:06:59,  7.51s/it]
 95%|█████████▍| 212961/225000 [412:50:39<25:03:36,  7.49s/it]
 95%|█████████▍| 212962/225000 [412:50:46<24:36:46,  7.36s/it]
 95%|█████████▍| 212963/225000 [412:50:53<24:21:35,  7.29s/it]
 95%|█████████▍| 212964/225000 [412:51:01<24:33:45,  7.35s/it]
 95%|█████████▍| 212965/225000 [412:51:15<31:18:15,  9.36s/it]
 95%|█████████▍| 212966/225000 [412:51:25<32:33:05,  9.74s/it]
 95%|█████████▍| 212967/225000 [412:51:35<32:27:56,  9.71s/it]
 95%|█████████▍| 212968/225000 [412:51:40<28:05:21,  8.40s/it]
 95%|█████████▍| 212969/225000 [412:51:48<27:45:16,  8.30s/it]
 95%|█████████▍| 212970/225000 [412:51:57<28:07:54,  8.42s/it]
 95%|█████████▍| 212971/225000 [412:52:05<27:36:32,  8.26s/it]
 95%|█████████▍| 212972/225000 [412:52:13<27:51:20,  8.34s/it]
 95%|█████████▍| 212973/225000 [412:52:19<25:03:06,  7.50s/it]
 95%|█████████▍| 212974/225000 [412:52:33<31:11:18,  9.34s/it]
 95%|█████████▍| 212975/225000 [412:52:43<32:21:39,  9.69s/it]
 95%|█████████▍| 212976/225000 [412:52:52<31:15:05,  9.36s/it]
 95%|█████████▍| 212977/225000 [412:53:01<31:00:29,  9.28s/it]
 95%|█████████▍| 212978/225000 [412:53:08<29:02:34,  8.70s/it]
 95%|█████████▍| 212979/225000 [412:53:15<27:05:52,  8.12s/it]
 95%|█████████▍| 212980/225000 [412:53:26<29:44:35,  8.91s/it]
 95%|█████████▍| 212981/225000 [412:53:34<29:03:34,  8.70s/it]
 95%|█████████▍| 212982/225000 [412:53:46<32:15:32,  9.66s/it]
 95%|█████████▍| 212983/225000 [412:53:55<32:04:15,  9.61s/it]
 95%|█████████▍| 212984/225000 [412:54:01<27:45:25,  8.32s/it]
 95%|█████████▍| 212985/225000 [412:54:09<28:05:04,  8.41s/it]
 95%|█████████▍| 212986/225000 [412:54:16<26:55:46,  8.07s/it]
 95%|█████████▍| 212987/225000 [412:54:26<28:44:39,  8.61s/it]
 95%|█████████▍| 212988/225000 [412:54:32<26:02:40,  7.81s/it]
 95%|█████████▍| 212989/225000 [412:54:41<26:41:30,  8.00s/it]
 95%|█████████▍| 212990/225000 [412:54:48<25:35:00,  7.67s/it]
 95%|█████████▍| 212991/225000 [412:54:59<29:33:12,  8.86s/it]
 95%|█████████▍| 212992/225000 [412:55:08<29:28:18,  8.84s/it]
 95%|█████████▍| 212993/225000 [412:55:14<26:34:07,  7.97s/it]
 95%|█████████▍| 212994/225000 [412:55:18<22:40:32,  6.80s/it]
 95%|█████████▍| 212995/225000 [412:55:30<27:42:45,  8.31s/it]
 95%|█████████▍| 212996/225000 [412:55:41<30:24:48,  9.12s/it]
 95%|█████████▍| 212997/225000 [412:55:47<27:29:05,  8.24s/it]
 95%|█████████▍| 212998/225000 [412:55:57<29:24:24,  8.82s/it]
 95%|█████████▍| 212999/225000 [412:56:07<29:53:36,  8.97s/it]
 95%|█████████▍| 213000/225000 [412:56:17<31:15:57,  9.38s/it]
                                                              
{'loss': '1.286', 'grad_norm': '1.377', 'learning_rate': '3.897e-07', 'epoch': '0.9467', 'num_input_tokens_seen': 700034784, 'train_runtime': '1.487e+06', 'train_tokens_per_second': '470.9'}

 95%|█████████▍| 213000/225000 [412:56:17<31:15:57,  9.38s/it]
 95%|█████████▍| 213001/225000 [412:56:26<30:29:46,  9.15s/it]
 95%|█████████▍| 213002/225000 [412:56:37<33:04:27,  9.92s/it]
 95%|█████████▍| 213003/225000 [412:56:45<31:07:11,  9.34s/it]
 95%|█████████▍| 213004/225000 [412:56:57<33:12:28,  9.97s/it]
 95%|█████████▍| 213005/225000 [412:57:07<33:10:02,  9.95s/it]
 95%|█████████▍| 213006/225000 [412:57:15<31:09:42,  9.35s/it]
 95%|█████████▍| 213007/225000 [412:57:25<31:56:22,  9.59s/it]
 95%|█████████▍| 213008/225000 [412:57:33<30:31:50,  9.17s/it]
 95%|█████████▍| 213009/225000 [412:57:41<29:51:10,  8.96s/it]
 95%|█████████▍| 213010/225000 [412:57:50<29:15:01,  8.78s/it]
 95%|█████████▍| 213011/225000 [412:57:56<26:58:08,  8.10s/it]
 95%|█████████▍| 213012/225000 [412:58:03<25:51:13,  7.76s/it]
 95%|█████████▍| 213013/225000 [412:58:14<28:33:12,  8.58s/it]
 95%|█████████▍| 213014/225000 [412:58:20<26:33:45,  7.98s/it]
 95%|█████████▍| 213015/225000 [412:58:32<30:16:48,  9.10s/it]
 95%|█████████▍| 213016/225000 [412:58:40<29:38:13,  8.90s/it]
 95%|█████████▍| 213017/225000 [412:58:48<28:32:34,  8.58s/it]
 95%|█████████▍| 213018/225000 [412:59:01<32:52:44,  9.88s/it]
 95%|█████████▍| 213019/225000 [412:59:09<31:18:22,  9.41s/it]
 95%|█████████▍| 213020/225000 [412:59:16<28:17:30,  8.50s/it]
 95%|█████████▍| 213021/225000 [412:59:25<28:37:33,  8.60s/it]
 95%|█████████▍| 213022/225000 [412:59:37<32:00:34,  9.62s/it]
 95%|█████████▍| 213023/225000 [412:59:43<28:26:18,  8.55s/it]
 95%|█████████▍| 213024/225000 [412:59:53<29:53:26,  8.99s/it]
 95%|█████████▍| 213025/225000 [412:59:58<26:21:50,  7.93s/it]
 95%|█████████▍| 213026/225000 [413:00:03<23:33:14,  7.08s/it]
 95%|█████████▍| 213027/225000 [413:00:10<22:50:05,  6.87s/it]
 95%|█████████▍| 213028/225000 [413:00:25<30:57:27,  9.31s/it]
 95%|█████████▍| 213029/225000 [413:00:32<29:01:58,  8.73s/it]
 95%|█████████▍| 213030/225000 [413:00:40<28:37:04,  8.61s/it]
 95%|█████████▍| 213031/225000 [413:00:50<29:17:51,  8.81s/it]
 95%|█████████▍| 213032/225000 [413:00:58<28:22:24,  8.53s/it]
 95%|█████████▍| 213033/225000 [413:01:07<29:28:57,  8.87s/it]
 95%|█████████▍| 213034/225000 [413:01:16<29:46:22,  8.96s/it]
 95%|█████████▍| 213035/225000 [413:01:25<29:14:39,  8.80s/it]
 95%|█████████▍| 213036/225000 [413:01:32<27:11:51,  8.18s/it]
 95%|█████████▍| 213037/225000 [413:01:43<30:11:37,  9.09s/it]
 95%|█████████▍| 213038/225000 [413:01:53<31:33:53,  9.50s/it]
 95%|█████████▍| 213039/225000 [413:02:00<29:12:13,  8.79s/it]
 95%|█████████▍| 213040/225000 [413:02:07<27:07:05,  8.16s/it]
 95%|█████████▍| 213041/225000 [413:02:13<25:24:10,  7.65s/it]
 95%|█████████▍| 213042/225000 [413:02:24<27:52:24,  8.39s/it]
 95%|█████████▍| 213043/225000 [413:02:33<29:13:47,  8.80s/it]
 95%|█████████▍| 213044/225000 [413:02:43<30:31:11,  9.19s/it]
 95%|█████████▍| 213045/225000 [413:02:52<29:51:12,  8.99s/it]
 95%|█████████▍| 213046/225000 [413:03:00<28:46:13,  8.66s/it]
 95%|█████████▍| 213047/225000 [413:03:08<28:40:39,  8.64s/it]
 95%|█████████▍| 213048/225000 [413:03:18<29:38:06,  8.93s/it]
 95%|█████████▍| 213049/225000 [413:03:24<26:32:52,  8.00s/it]
 95%|█████████▍| 213050/225000 [413:03:31<26:09:31,  7.88s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.006', 'learning_rate': '3.865e-07', 'epoch': '0.9469', 'num_input_tokens_seen': 700269272, 'train_runtime': '1.487e+06', 'train_tokens_per_second': '470.9'}

 95%|█████████▍| 213050/225000 [413:03:31<26:09:31,  7.88s/it]
 95%|█████████▍| 213051/225000 [413:03:44<30:38:21,  9.23s/it]
 95%|█████████▍| 213052/225000 [413:03:50<27:52:42,  8.40s/it]
 95%|█████████▍| 213053/225000 [413:03:58<27:30:45,  8.29s/it]
 95%|█████████▍| 213054/225000 [413:04:04<24:43:22,  7.45s/it]
 95%|█████████▍| 213055/225000 [413:04:17<30:13:48,  9.11s/it]
 95%|█████████▍| 213056/225000 [413:04:25<28:57:17,  8.73s/it]
 95%|█████████▍| 213057/225000 [413:04:35<30:18:48,  9.14s/it]
 95%|█████████▍| 213058/225000 [413:04:41<27:44:36,  8.36s/it]
 95%|█████████▍| 213059/225000 [413:04:46<24:13:57,  7.31s/it]
 95%|█████████▍| 213060/225000 [413:04:52<22:33:43,  6.80s/it]
 95%|█████████▍| 213061/225000 [413:04:58<22:01:51,  6.64s/it]
 95%|█████████▍| 213062/225000 [413:05:10<27:21:53,  8.25s/it]
 95%|█████████▍| 213063/225000 [413:05:18<27:31:34,  8.30s/it]
 95%|█████████▍| 213064/225000 [413:05:28<29:08:23,  8.79s/it]
 95%|█████████▍| 213065/225000 [413:05:36<27:50:24,  8.40s/it]
 95%|█████████▍| 213066/225000 [413:05:41<24:17:17,  7.33s/it]
 95%|█████████▍| 213067/225000 [413:05:54<29:50:30,  9.00s/it]
 95%|█████████▍| 213068/225000 [413:06:00<27:40:58,  8.35s/it]
 95%|█████████▍| 213069/225000 [413:06:08<27:09:49,  8.20s/it]
 95%|█████████▍| 213070/225000 [413:06:20<30:58:28,  9.35s/it]
 95%|█████████▍| 213071/225000 [413:06:29<30:17:24,  9.14s/it]
 95%|█████████▍| 213072/225000 [413:06:41<33:17:43, 10.05s/it]
 95%|█████████▍| 213073/225000 [413:06:47<29:23:24,  8.87s/it]
 95%|█████████▍| 213074/225000 [413:07:00<33:12:00, 10.02s/it]
 95%|█████████▍| 213075/225000 [413:07:10<32:42:15,  9.87s/it]
 95%|█████████▍| 213076/225000 [413:07:16<29:01:36,  8.76s/it]
 95%|█████████▍| 213077/225000 [413:07:27<31:34:20,  9.53s/it]
 95%|█████████▍| 213078/225000 [413:07:36<30:38:31,  9.25s/it]
 95%|█████████▍| 213079/225000 [413:07:41<27:08:18,  8.20s/it]
 95%|█████████▍| 213080/225000 [413:07:50<27:54:28,  8.43s/it]
 95%|█████████▍| 213081/225000 [413:07:59<28:31:20,  8.61s/it]
 95%|█████████▍| 213082/225000 [413:08:07<27:59:04,  8.45s/it]
 95%|█████████▍| 213083/225000 [413:08:15<26:42:12,  8.07s/it]
 95%|█████████▍| 213084/225000 [413:08:25<28:45:47,  8.69s/it]
 95%|█████████▍| 213085/225000 [413:08:33<28:17:18,  8.55s/it]
 95%|█████████▍| 213086/225000 [413:08:39<25:31:33,  7.71s/it]
 95%|█████████▍| 213087/225000 [413:08:45<23:47:20,  7.19s/it]
 95%|█████████▍| 213088/225000 [413:08:57<29:17:02,  8.85s/it]
 95%|█████████▍| 213089/225000 [413:09:07<29:47:21,  9.00s/it]
 95%|█████████▍| 213090/225000 [413:09:17<31:09:35,  9.42s/it]
 95%|█████████▍| 213091/225000 [413:09:23<27:21:54,  8.27s/it]
 95%|█████████▍| 213092/225000 [413:09:29<25:40:11,  7.76s/it]
 95%|█████████▍| 213093/225000 [413:09:43<31:22:33,  9.49s/it]
 95%|█████████▍| 213094/225000 [413:09:49<27:42:58,  8.38s/it]
 95%|█████████▍| 213095/225000 [413:09:57<27:44:13,  8.39s/it]
 95%|█████████▍| 213096/225000 [413:10:05<27:05:02,  8.19s/it]
 95%|█████████▍| 213097/225000 [413:10:18<32:00:18,  9.68s/it]
 95%|█████████▍| 213098/225000 [413:10:23<27:20:24,  8.27s/it]
 95%|█████████▍| 213099/225000 [413:10:28<24:36:50,  7.45s/it]
 95%|█████████▍| 213100/225000 [413:10:35<23:50:20,  7.21s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.629', 'learning_rate': '3.833e-07', 'epoch': '0.9471', 'num_input_tokens_seen': 700498744, 'train_runtime': '1.487e+06', 'train_tokens_per_second': '470.9'}

 95%|█████████▍| 213100/225000 [413:10:35<23:50:20,  7.21s/it]
 95%|█████████▍| 213101/225000 [413:10:42<23:45:49,  7.19s/it]
 95%|█████████▍| 213102/225000 [413:10:53<26:51:49,  8.13s/it]
 95%|█████████▍| 213103/225000 [413:11:03<28:57:08,  8.76s/it]
 95%|█████████▍| 213104/225000 [413:11:16<33:06:27, 10.02s/it]
 95%|█████████▍| 213105/225000 [413:11:23<30:04:59,  9.10s/it]
 95%|█████████▍| 213106/225000 [413:11:32<29:58:26,  9.07s/it]
 95%|█████████▍| 213107/225000 [413:11:42<31:38:58,  9.58s/it]
 95%|█████████▍| 213108/225000 [413:11:54<33:47:54, 10.23s/it]
 95%|█████████▍| 213109/225000 [413:12:01<30:25:13,  9.21s/it]
 95%|█████████▍| 213110/225000 [413:12:12<32:26:15,  9.82s/it]
 95%|█████████▍| 213111/225000 [413:12:19<29:45:42,  9.01s/it]
 95%|█████████▍| 213112/225000 [413:12:28<29:14:21,  8.85s/it]
 95%|█████████▍| 213113/225000 [413:12:37<29:55:04,  9.06s/it]
 95%|█████████▍| 213114/225000 [413:12:47<29:54:51,  9.06s/it]
 95%|█████████▍| 213115/225000 [413:12:53<27:13:41,  8.25s/it]
 95%|█████████▍| 213116/225000 [413:13:06<32:31:38,  9.85s/it]
 95%|█████████▍| 213117/225000 [413:13:15<31:37:59,  9.58s/it]
 95%|█████████▍| 213118/225000 [413:13:23<29:56:54,  9.07s/it]
 95%|█████████▍| 213119/225000 [413:13:29<26:51:26,  8.14s/it]
 95%|█████████▍| 213120/225000 [413:13:38<27:12:13,  8.24s/it]
 95%|█████████▍| 213121/225000 [413:13:44<25:09:42,  7.63s/it]
 95%|█████████▍| 213122/225000 [413:14:01<34:23:21, 10.42s/it]
 95%|█████████▍| 213123/225000 [413:14:07<30:21:09,  9.20s/it]
 95%|█████████▍| 213124/225000 [413:14:17<31:05:15,  9.42s/it]
 95%|█████████▍| 213125/225000 [413:14:27<31:14:10,  9.47s/it]
 95%|█████████▍| 213126/225000 [413:14:34<29:23:00,  8.91s/it]
 95%|█████████▍| 213127/225000 [413:14:44<29:54:02,  9.07s/it]
 95%|█████████▍| 213128/225000 [413:14:54<31:25:28,  9.53s/it]
 95%|█████████▍| 213129/225000 [413:15:02<29:38:18,  8.99s/it]
 95%|█████████▍| 213130/225000 [413:15:15<33:43:00, 10.23s/it]
 95%|█████████▍| 213131/225000 [413:15:26<34:02:10, 10.32s/it]
 95%|█████████▍| 213132/225000 [413:15:35<32:34:14,  9.88s/it]
 95%|█████████▍| 213133/225000 [413:15:42<30:01:39,  9.11s/it]
 95%|█████████▍| 213134/225000 [413:15:49<27:32:54,  8.36s/it]
 95%|█████████▍| 213135/225000 [413:15:54<25:08:26,  7.63s/it]
 95%|█████████▍| 213136/225000 [413:16:04<27:25:09,  8.32s/it]
 95%|█████████▍| 213137/225000 [413:16:10<24:30:34,  7.44s/it]
 95%|█████████▍| 213138/225000 [413:16:20<26:46:30,  8.13s/it]
 95%|█████████▍| 213139/225000 [413:16:27<26:06:42,  7.93s/it]
 95%|█████████▍| 213140/225000 [413:16:33<23:54:44,  7.26s/it]
 95%|█████████▍| 213141/225000 [413:16:45<28:41:45,  8.71s/it]
 95%|█████████▍| 213142/225000 [413:16:50<25:03:44,  7.61s/it]
 95%|█████████▍| 213143/225000 [413:17:02<30:02:17,  9.12s/it]
 95%|█████████▍| 213144/225000 [413:17:13<31:46:14,  9.65s/it]
 95%|█████████▍| 213145/225000 [413:17:18<27:16:19,  8.28s/it]
 95%|█████████▍| 213146/225000 [413:17:25<25:33:20,  7.76s/it]
 95%|█████████▍| 213147/225000 [413:17:37<29:47:33,  9.05s/it]
 95%|█████████▍| 213148/225000 [413:17:45<29:01:09,  8.81s/it]
 95%|█████████▍| 213149/225000 [413:17:59<34:12:27, 10.39s/it]
 95%|█████████▍| 213150/225000 [413:18:05<29:02:56,  8.83s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.638', 'learning_rate': '3.802e-07', 'epoch': '0.9473', 'num_input_tokens_seen': 700734192, 'train_runtime': '1.488e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213150/225000 [413:18:05<29:02:56,  8.83s/it]
 95%|█████████▍| 213151/225000 [413:18:17<32:13:17,  9.79s/it]
 95%|█████████▍| 213152/225000 [413:18:25<31:02:19,  9.43s/it]
 95%|█████████▍| 213153/225000 [413:18:35<31:30:16,  9.57s/it]
 95%|█████████▍| 213154/225000 [413:18:42<28:46:48,  8.75s/it]
 95%|█████████▍| 213155/225000 [413:18:49<27:18:47,  8.30s/it]
 95%|█████████▍| 213156/225000 [413:18:59<28:36:32,  8.70s/it]
 95%|█████████▍| 213157/225000 [413:19:06<27:03:31,  8.23s/it]
 95%|█████████▍| 213158/225000 [413:19:13<26:07:15,  7.94s/it]
 95%|█████████▍| 213159/225000 [413:19:22<27:12:48,  8.27s/it]
 95%|█████████▍| 213160/225000 [413:19:31<28:02:57,  8.53s/it]
 95%|█████████▍| 213161/225000 [413:19:45<33:12:59, 10.10s/it]
 95%|█████████▍| 213162/225000 [413:19:53<30:34:34,  9.30s/it]
 95%|█████████▍| 213163/225000 [413:20:02<30:29:09,  9.27s/it]
 95%|█████████▍| 213164/225000 [413:20:09<28:04:36,  8.54s/it]
 95%|█████████▍| 213165/225000 [413:20:17<28:13:34,  8.59s/it]
 95%|█████████▍| 213166/225000 [413:20:29<31:36:25,  9.62s/it]
 95%|█████████▍| 213167/225000 [413:20:35<27:50:25,  8.47s/it]
 95%|█████████▍| 213168/225000 [413:20:47<31:25:48,  9.56s/it]
 95%|█████████▍| 213169/225000 [413:20:54<28:43:01,  8.74s/it]
 95%|█████████▍| 213170/225000 [413:21:05<30:52:12,  9.39s/it]
 95%|█████████▍| 213171/225000 [413:21:12<28:16:07,  8.60s/it]
 95%|█████████▍| 213172/225000 [413:21:21<29:03:36,  8.84s/it]
 95%|█████████▍| 213173/225000 [413:21:30<29:21:33,  8.94s/it]
 95%|█████████▍| 213174/225000 [413:21:38<28:18:28,  8.62s/it]
 95%|█████████▍| 213175/225000 [413:21:45<26:55:18,  8.20s/it]
 95%|█████████▍| 213176/225000 [413:21:53<25:58:06,  7.91s/it]
 95%|█████████▍| 213177/225000 [413:21:59<24:23:18,  7.43s/it]
 95%|█████████▍| 213178/225000 [413:22:08<25:37:44,  7.80s/it]
 95%|█████████▍| 213179/225000 [413:22:22<32:10:40,  9.80s/it]
 95%|█████████▍| 213180/225000 [413:22:31<31:10:56,  9.50s/it]
 95%|█████████▍| 213181/225000 [413:22:39<30:15:06,  9.21s/it]
 95%|█████████▍| 213182/225000 [413:22:47<28:34:07,  8.70s/it]
 95%|█████████▍| 213183/225000 [413:22:55<27:53:28,  8.50s/it]
 95%|█████████▍| 213184/225000 [413:23:03<27:26:51,  8.36s/it]
 95%|█████████▍| 213185/225000 [413:23:15<30:55:29,  9.42s/it]
 95%|█████████▍| 213186/225000 [413:23:22<28:33:46,  8.70s/it]
 95%|█████████▍| 213187/225000 [413:23:35<32:51:50, 10.02s/it]
 95%|█████████▍| 213188/225000 [413:23:44<31:42:57,  9.67s/it]
 95%|█████████▍| 213189/225000 [413:23:49<27:34:39,  8.41s/it]
 95%|█████████▍| 213190/225000 [413:23:58<27:40:08,  8.43s/it]
 95%|█████████▍| 213191/225000 [413:24:03<24:51:32,  7.58s/it]
 95%|█████████▍| 213192/225000 [413:24:13<26:39:58,  8.13s/it]
 95%|█████████▍| 213193/225000 [413:24:26<31:26:21,  9.59s/it]
 95%|█████████▍| 213194/225000 [413:24:30<26:23:51,  8.05s/it]
 95%|█████████▍| 213195/225000 [413:24:39<27:20:48,  8.34s/it]
 95%|█████████▍| 213196/225000 [413:24:48<27:26:32,  8.37s/it]
 95%|█████████▍| 213197/225000 [413:25:01<31:55:38,  9.74s/it]
 95%|█████████▍| 213198/225000 [413:25:08<29:33:40,  9.02s/it]
 95%|█████████▍| 213199/225000 [413:25:14<26:46:35,  8.17s/it]
 95%|█████████▍| 213200/225000 [413:25:25<29:11:37,  8.91s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.608', 'learning_rate': '3.77e-07', 'epoch': '0.9476', 'num_input_tokens_seen': 700964336, 'train_runtime': '1.488e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213200/225000 [413:25:25<29:11:37,  8.91s/it]
 95%|█████████▍| 213201/225000 [413:25:33<28:55:05,  8.82s/it]
 95%|█████████▍| 213202/225000 [413:25:40<26:46:38,  8.17s/it]
 95%|█████████▍| 213203/225000 [413:25:50<28:07:52,  8.58s/it]
 95%|█████████▍| 213204/225000 [413:26:03<32:30:22,  9.92s/it]
 95%|█████████▍| 213205/225000 [413:26:12<31:29:01,  9.61s/it]
 95%|█████████▍| 213206/225000 [413:26:17<27:40:47,  8.45s/it]
 95%|█████████▍| 213207/225000 [413:26:28<29:38:06,  9.05s/it]
 95%|█████████▍| 213208/225000 [413:26:35<27:49:07,  8.49s/it]
 95%|█████████▍| 213209/225000 [413:26:42<26:54:18,  8.21s/it]
 95%|█████████▍| 213210/225000 [413:26:54<30:26:40,  9.30s/it]
 95%|█████████▍| 213211/225000 [413:27:02<28:40:50,  8.76s/it]
 95%|█████████▍| 213212/225000 [413:27:10<27:42:41,  8.46s/it]
 95%|█████████▍| 213213/225000 [413:27:21<30:38:59,  9.36s/it]
 95%|█████████▍| 213214/225000 [413:27:29<29:26:04,  8.99s/it]
 95%|█████████▍| 213215/225000 [413:27:40<31:16:35,  9.55s/it]
 95%|█████████▍| 213216/225000 [413:27:47<28:21:22,  8.66s/it]
 95%|█████████▍| 213217/225000 [413:27:57<30:06:58,  9.20s/it]
 95%|█████████▍| 213218/225000 [413:28:06<29:26:53,  9.00s/it]
 95%|█████████▍| 213219/225000 [413:28:14<28:38:55,  8.75s/it]
 95%|█████████▍| 213220/225000 [413:28:20<25:55:47,  7.92s/it]
 95%|█████████▍| 213221/225000 [413:28:31<29:14:32,  8.94s/it]
 95%|█████████▍| 213222/225000 [413:28:42<30:59:12,  9.47s/it]
 95%|█████████▍| 213223/225000 [413:28:46<26:09:44,  8.00s/it]
 95%|█████████▍| 213224/225000 [413:28:57<29:16:44,  8.95s/it]
 95%|█████████▍| 213225/225000 [413:29:06<28:23:12,  8.68s/it]
 95%|█████████▍| 213226/225000 [413:29:12<25:50:58,  7.90s/it]
 95%|█████████▍| 213227/225000 [413:29:19<25:22:13,  7.76s/it]
 95%|█████████▍| 213228/225000 [413:29:35<33:24:53, 10.22s/it]
 95%|█████████▍| 213229/225000 [413:29:42<30:18:51,  9.27s/it]
 95%|█████████▍| 213230/225000 [413:29:50<29:11:22,  8.93s/it]
 95%|█████████▍| 213231/225000 [413:29:59<29:28:05,  9.01s/it]
 95%|█████████▍| 213232/225000 [413:30:08<29:04:45,  8.90s/it]
 95%|█████████▍| 213233/225000 [413:30:14<25:59:46,  7.95s/it]
 95%|█████████▍| 213234/225000 [413:30:20<24:17:31,  7.43s/it]
 95%|█████████▍| 213235/225000 [413:30:37<34:03:25, 10.42s/it]
 95%|█████████▍| 213236/225000 [413:30:46<31:49:32,  9.74s/it]
 95%|█████████▍| 213237/225000 [413:30:51<27:15:38,  8.34s/it]
 95%|█████████▍| 213238/225000 [413:30:56<24:04:51,  7.37s/it]
 95%|█████████▍| 213239/225000 [413:31:08<28:24:32,  8.70s/it]
 95%|█████████▍| 213240/225000 [413:31:17<29:31:47,  9.04s/it]
 95%|█████████▍| 213241/225000 [413:31:25<28:31:43,  8.73s/it]
 95%|█████████▍| 213242/225000 [413:31:39<33:18:58, 10.20s/it]
 95%|█████████▍| 213243/225000 [413:31:46<30:04:31,  9.21s/it]
 95%|█████████▍| 213244/225000 [413:31:52<26:49:15,  8.21s/it]
 95%|█████████▍| 213245/225000 [413:32:03<30:04:52,  9.21s/it]
 95%|█████████▍| 213246/225000 [413:32:09<26:53:16,  8.24s/it]
 95%|█████████▍| 213247/225000 [413:32:18<26:51:55,  8.23s/it]
 95%|█████████▍| 213248/225000 [413:32:27<28:19:32,  8.68s/it]
 95%|█████████▍| 213249/225000 [413:32:37<29:47:34,  9.13s/it]
 95%|█████████▍| 213250/225000 [413:32:45<28:25:32,  8.71s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.62', 'learning_rate': '3.738e-07', 'epoch': '0.9478', 'num_input_tokens_seen': 701192224, 'train_runtime': '1.489e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213250/225000 [413:32:45<28:25:32,  8.71s/it]
 95%|█████████▍| 213251/225000 [413:32:51<25:36:01,  7.84s/it]
 95%|█████████▍| 213252/225000 [413:33:00<26:59:24,  8.27s/it]
 95%|█████████▍| 213253/225000 [413:33:08<26:36:51,  8.16s/it]
 95%|█████████▍| 213254/225000 [413:33:16<25:56:16,  7.95s/it]
 95%|█████████▍| 213255/225000 [413:33:26<27:57:46,  8.57s/it]
 95%|█████████▍| 213256/225000 [413:33:32<26:01:52,  7.98s/it]
 95%|█████████▍| 213257/225000 [413:33:40<25:41:53,  7.88s/it]
 95%|█████████▍| 213258/225000 [413:33:48<25:53:18,  7.94s/it]
 95%|█████████▍| 213259/225000 [413:34:01<31:12:04,  9.57s/it]
 95%|█████████▍| 213260/225000 [413:34:12<32:04:05,  9.83s/it]
 95%|█████████▍| 213261/225000 [413:34:18<28:40:41,  8.79s/it]
 95%|█████████▍| 213262/225000 [413:34:25<26:48:44,  8.22s/it]
 95%|█████████▍| 213263/225000 [413:34:32<25:35:34,  7.85s/it]
 95%|█████████▍| 213264/225000 [413:34:38<23:31:47,  7.22s/it]
 95%|█████████▍| 213265/225000 [413:34:51<29:12:59,  8.96s/it]
 95%|█████████▍| 213266/225000 [413:35:02<30:58:50,  9.50s/it]
 95%|█████████▍| 213267/225000 [413:35:11<30:31:13,  9.36s/it]
 95%|█████████▍| 213268/225000 [413:35:17<27:24:36,  8.41s/it]
 95%|█████████▍| 213269/225000 [413:35:24<26:19:59,  8.08s/it]
 95%|█████████▍| 213270/225000 [413:35:35<29:02:48,  8.91s/it]
 95%|█████████▍| 213271/225000 [413:35:41<26:41:39,  8.19s/it]
 95%|█████████▍| 213272/225000 [413:35:53<29:30:49,  9.06s/it]
 95%|█████████▍| 213273/225000 [413:36:03<30:32:22,  9.38s/it]
 95%|█████████▍| 213274/225000 [413:36:11<29:48:46,  9.15s/it]
 95%|█████████▍| 213275/225000 [413:36:20<29:45:58,  9.14s/it]
 95%|█████████▍| 213276/225000 [413:36:26<26:46:09,  8.22s/it]
 95%|█████████▍| 213277/225000 [413:36:34<25:44:09,  7.90s/it]
 95%|█████████▍| 213278/225000 [413:36:48<32:17:16,  9.92s/it]
 95%|█████████▍| 213279/225000 [413:36:53<27:20:38,  8.40s/it]
 95%|█████████▍| 213280/225000 [413:37:02<27:47:10,  8.53s/it]
 95%|█████████▍| 213281/225000 [413:37:08<25:21:58,  7.79s/it]
 95%|█████████▍| 213282/225000 [413:37:17<26:34:43,  8.17s/it]
 95%|█████████▍| 213283/225000 [413:37:33<34:14:57, 10.52s/it]
 95%|█████████▍| 213284/225000 [413:37:39<29:35:45,  9.09s/it]
 95%|█████████▍| 213285/225000 [413:37:51<32:19:30,  9.93s/it]
 95%|█████████▍| 213286/225000 [413:38:00<31:41:53,  9.74s/it]
 95%|█████████▍| 213287/225000 [413:38:06<27:46:28,  8.54s/it]
 95%|█████████▍| 213288/225000 [413:38:13<26:41:45,  8.21s/it]
 95%|█████████▍| 213289/225000 [413:38:28<32:47:38, 10.08s/it]
 95%|█████████▍| 213290/225000 [413:38:37<31:54:00,  9.81s/it]
 95%|█████████▍| 213291/225000 [413:38:44<29:05:07,  8.94s/it]
 95%|█████████▍| 213292/225000 [413:38:55<31:15:34,  9.61s/it]
 95%|█████████▍| 213293/225000 [413:38:59<26:01:03,  8.00s/it]
 95%|█████████▍| 213294/225000 [413:39:08<26:59:51,  8.30s/it]
 95%|█████████▍| 213295/225000 [413:39:22<32:41:15, 10.05s/it]
 95%|█████████▍| 213296/225000 [413:39:28<28:39:51,  8.82s/it]
 95%|█████████▍| 213297/225000 [413:39:36<27:46:55,  8.55s/it]
 95%|█████████▍| 213298/225000 [413:39:46<29:25:54,  9.05s/it]
 95%|█████████▍| 213299/225000 [413:39:56<29:46:41,  9.16s/it]
 95%|█████████▍| 213300/225000 [413:40:07<31:56:49,  9.83s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.32', 'learning_rate': '3.707e-07', 'epoch': '0.948', 'num_input_tokens_seen': 701426488, 'train_runtime': '1.489e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213300/225000 [413:40:07<31:56:49,  9.83s/it]
 95%|█████████▍| 213301/225000 [413:40:15<29:52:27,  9.19s/it]
 95%|█████████▍| 213302/225000 [413:40:20<26:03:50,  8.02s/it]
 95%|█████████▍| 213303/225000 [413:40:28<25:45:15,  7.93s/it]
 95%|█████████▍| 213304/225000 [413:40:40<29:24:20,  9.05s/it]
 95%|█████████▍| 213305/225000 [413:40:48<29:01:46,  8.94s/it]
 95%|█████████▍| 213306/225000 [413:41:00<31:47:31,  9.79s/it]
 95%|█████████▍| 213307/225000 [413:41:05<27:33:31,  8.48s/it]
 95%|█████████▍| 213308/225000 [413:41:11<24:32:32,  7.56s/it]
 95%|█████████▍| 213309/225000 [413:41:23<29:22:51,  9.05s/it]
 95%|█████████▍| 213310/225000 [413:41:30<27:13:15,  8.38s/it]
 95%|█████████▍| 213311/225000 [413:41:43<31:50:24,  9.81s/it]
 95%|█████████▍| 213312/225000 [413:41:50<28:48:02,  8.87s/it]
 95%|█████████▍| 213313/225000 [413:41:55<24:51:27,  7.66s/it]
 95%|█████████▍| 213314/225000 [413:42:04<26:12:26,  8.07s/it]
 95%|█████████▍| 213315/225000 [413:42:12<25:57:10,  8.00s/it]
 95%|█████████▍| 213316/225000 [413:42:18<24:00:00,  7.39s/it]
 95%|█████████▍| 213317/225000 [413:42:27<26:14:26,  8.09s/it]
 95%|█████████▍| 213318/225000 [413:42:35<25:23:07,  7.82s/it]
 95%|█████████▍| 213319/225000 [413:42:46<29:20:12,  9.04s/it]
 95%|█████████▍| 213320/225000 [413:42:51<25:24:39,  7.83s/it]
 95%|█████████▍| 213321/225000 [413:43:01<27:15:41,  8.40s/it]
 95%|█████████▍| 213322/225000 [413:43:10<27:20:50,  8.43s/it]
 95%|█████████▍| 213323/225000 [413:43:20<29:11:20,  9.00s/it]
 95%|█████████▍| 213324/225000 [413:43:30<30:23:36,  9.37s/it]
 95%|█████████▍| 213325/225000 [413:43:40<30:22:47,  9.37s/it]
 95%|█████████▍| 213326/225000 [413:43:48<29:01:25,  8.95s/it]
 95%|█████████▍| 213327/225000 [413:43:54<26:17:10,  8.11s/it]
 95%|█████████▍| 213328/225000 [413:44:00<24:29:59,  7.56s/it]
 95%|█████████▍| 213329/225000 [413:44:07<23:40:24,  7.30s/it]
 95%|█████████▍| 213330/225000 [413:44:23<32:13:38,  9.94s/it]
 95%|█████████▍| 213331/225000 [413:44:34<33:33:53, 10.36s/it]
 95%|█████████▍| 213332/225000 [413:44:43<32:18:58,  9.97s/it]
 95%|█████████▍| 213333/225000 [413:44:51<30:28:37,  9.40s/it]
 95%|█████████▍| 213334/225000 [413:44:58<27:38:20,  8.53s/it]
 95%|█████████▍| 213335/225000 [413:45:10<31:15:04,  9.64s/it]
 95%|█████████▍| 213336/225000 [413:45:17<28:48:10,  8.89s/it]
 95%|█████████▍| 213337/225000 [413:45:24<26:48:49,  8.28s/it]
 95%|█████████▍| 213338/225000 [413:45:34<28:46:48,  8.88s/it]
 95%|█████████▍| 213339/225000 [413:45:47<32:07:22,  9.92s/it]
 95%|█████████▍| 213340/225000 [413:45:51<26:56:58,  8.32s/it]
 95%|█████████▍| 213341/225000 [413:45:57<24:51:00,  7.67s/it]
 95%|█████████▍| 213342/225000 [413:46:09<28:38:48,  8.85s/it]
 95%|█████████▍| 213343/225000 [413:46:16<26:44:33,  8.26s/it]
 95%|█████████▍| 213344/225000 [413:46:27<29:17:21,  9.05s/it]
 95%|█████████▍| 213345/225000 [413:46:34<27:18:20,  8.43s/it]
 95%|█████████▍| 213346/225000 [413:46:39<24:34:49,  7.59s/it]
 95%|█████████▍| 213347/225000 [413:46:45<22:27:53,  6.94s/it]
 95%|█████████▍| 213348/225000 [413:46:53<23:22:10,  7.22s/it]
 95%|█████████▍| 213349/225000 [413:47:00<23:47:32,  7.35s/it]
 95%|█████████▍| 213350/225000 [413:47:13<29:09:14,  9.01s/it]
                                                              
{'loss': '1.317', 'grad_norm': '1.648', 'learning_rate': '3.676e-07', 'epoch': '0.9482', 'num_input_tokens_seen': 701661136, 'train_runtime': '1.49e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213350/225000 [413:47:13<29:09:14,  9.01s/it]
 95%|█████████▍| 213351/225000 [413:47:24<31:14:12,  9.65s/it]
 95%|█████████▍| 213352/225000 [413:47:31<28:38:41,  8.85s/it]
 95%|█████████▍| 213353/225000 [413:47:47<35:06:16, 10.85s/it]
 95%|█████████▍| 213354/225000 [413:47:52<29:48:50,  9.22s/it]
 95%|█████████▍| 213355/225000 [413:48:02<30:25:01,  9.40s/it]
 95%|█████████▍| 213356/225000 [413:48:08<26:57:37,  8.34s/it]
 95%|█████████▍| 213357/225000 [413:48:14<24:44:01,  7.65s/it]
 95%|█████████▍| 213358/225000 [413:48:23<26:05:32,  8.07s/it]
 95%|█████████▍| 213359/225000 [413:48:30<24:57:42,  7.72s/it]
 95%|█████████▍| 213360/225000 [413:48:39<25:55:44,  8.02s/it]
 95%|█████████▍| 213361/225000 [413:48:49<27:54:55,  8.63s/it]
 95%|█████████▍| 213362/225000 [413:49:00<30:36:00,  9.47s/it]
 95%|█████████▍| 213363/225000 [413:49:07<28:13:15,  8.73s/it]
 95%|█████████▍| 213364/225000 [413:49:15<27:47:14,  8.60s/it]
 95%|█████████▍| 213365/225000 [413:49:22<26:14:17,  8.12s/it]
 95%|█████████▍| 213366/225000 [413:49:33<28:51:49,  8.93s/it]
 95%|█████████▍| 213367/225000 [413:49:42<28:15:36,  8.75s/it]
 95%|█████████▍| 213368/225000 [413:49:50<27:41:34,  8.57s/it]
 95%|█████████▍| 213369/225000 [413:49:58<27:10:22,  8.41s/it]
 95%|█████████▍| 213370/225000 [413:50:12<32:40:01, 10.11s/it]
 95%|█████████▍| 213371/225000 [413:50:22<32:12:13,  9.97s/it]
 95%|█████████▍| 213372/225000 [413:50:32<32:28:35, 10.05s/it]
 95%|█████████▍| 213373/225000 [413:50:39<29:17:35,  9.07s/it]
 95%|█████████▍| 213374/225000 [413:50:44<25:48:41,  7.99s/it]
 95%|█████████▍| 213375/225000 [413:50:55<28:45:31,  8.91s/it]
 95%|█████████▍| 213376/225000 [413:51:04<29:11:42,  9.04s/it]
 95%|█████████▍| 213377/225000 [413:51:09<24:53:09,  7.71s/it]
 95%|█████████▍| 213378/225000 [413:51:18<26:31:46,  8.22s/it]
 95%|█████████▍| 213379/225000 [413:51:26<26:00:14,  8.06s/it]
 95%|█████████▍| 213380/225000 [413:51:33<25:15:19,  7.82s/it]
 95%|█████████▍| 213381/225000 [413:51:43<26:36:53,  8.25s/it]
 95%|█████████▍| 213382/225000 [413:51:50<25:20:28,  7.85s/it]
 95%|█████████▍| 213383/225000 [413:51:59<26:28:46,  8.21s/it]
 95%|█████████▍| 213384/225000 [413:52:04<23:56:29,  7.42s/it]
 95%|█████████▍| 213385/225000 [413:52:14<26:13:03,  8.13s/it]
 95%|█████████▍| 213386/225000 [413:52:22<25:42:08,  7.97s/it]
 95%|█████████▍| 213387/225000 [413:52:27<23:08:56,  7.18s/it]
 95%|█████████▍| 213388/225000 [413:52:39<27:29:18,  8.52s/it]
 95%|█████████▍| 213389/225000 [413:52:49<29:28:13,  9.14s/it]
 95%|█████████▍| 213390/225000 [413:52:58<29:15:15,  9.07s/it]
 95%|█████████▍| 213391/225000 [413:53:05<26:55:40,  8.35s/it]
 95%|█████████▍| 213392/225000 [413:53:11<24:32:04,  7.61s/it]
 95%|█████████▍| 213393/225000 [413:53:19<24:59:46,  7.75s/it]
 95%|█████████▍| 213394/225000 [413:53:37<35:20:35, 10.96s/it]
 95%|█████████▍| 213395/225000 [413:53:48<35:17:58, 10.95s/it]
 95%|█████████▍| 213396/225000 [413:53:54<30:16:32,  9.39s/it]
 95%|█████████▍| 213397/225000 [413:54:05<31:59:59,  9.93s/it]
 95%|█████████▍| 213398/225000 [413:54:16<32:43:32, 10.15s/it]
 95%|█████████▍| 213399/225000 [413:54:26<32:47:52, 10.18s/it]
 95%|█████████▍| 213400/225000 [413:54:37<33:59:16, 10.55s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.214', 'learning_rate': '3.645e-07', 'epoch': '0.9484', 'num_input_tokens_seen': 701891704, 'train_runtime': '1.49e+06', 'train_tokens_per_second': '471'}

 95%|█████████▍| 213400/225000 [413:54:37<33:59:16, 10.55s/it]
 95%|█████████▍| 213401/225000 [413:54:41<27:43:43,  8.61s/it]
 95%|█████████▍| 213402/225000 [413:54:49<26:26:36,  8.21s/it]
 95%|█████████▍| 213403/225000 [413:54:54<24:10:43,  7.51s/it]
 95%|█████████▍| 213404/225000 [413:55:01<22:50:50,  7.09s/it]
 95%|█████████▍| 213405/225000 [413:55:13<28:15:27,  8.77s/it]
 95%|█████████▍| 213406/225000 [413:55:22<27:48:49,  8.64s/it]
 95%|█████████▍| 213407/225000 [413:55:28<25:55:03,  8.05s/it]
 95%|█████████▍| 213408/225000 [413:55:38<27:45:37,  8.62s/it]
 95%|█████████▍| 213409/225000 [413:55:48<28:55:12,  8.98s/it]
 95%|█████████▍| 213410/225000 [413:55:57<28:25:02,  8.83s/it]
 95%|█████████▍| 213411/225000 [413:56:09<32:19:15, 10.04s/it]
 95%|█████████▍| 213412/225000 [413:56:15<28:12:08,  8.76s/it]
 95%|█████████▍| 213413/225000 [413:56:25<29:16:52,  9.10s/it]
 95%|█████████▍| 213414/225000 [413:56:36<31:27:50,  9.78s/it]
 95%|█████████▍| 213415/225000 [413:56:43<28:46:30,  8.94s/it]
 95%|█████████▍| 213416/225000 [413:56:52<28:42:17,  8.92s/it]
 95%|█████████▍| 213417/225000 [413:56:58<25:44:48,  8.00s/it]
 95%|█████████▍| 213418/225000 [413:57:11<30:46:20,  9.56s/it]
 95%|█████████▍| 213419/225000 [413:57:20<29:40:21,  9.22s/it]
 95%|█████████▍| 213420/225000 [413:57:27<27:44:37,  8.62s/it]
 95%|█████████▍| 213421/225000 [413:57:32<24:05:37,  7.49s/it]
 95%|█████████▍| 213422/225000 [413:57:43<27:51:12,  8.66s/it]
 95%|█████████▍| 213423/225000 [413:57:48<24:04:38,  7.49s/it]
 95%|█████████▍| 213424/225000 [413:57:54<22:53:40,  7.12s/it]
 95%|█████████▍| 213425/225000 [413:58:10<31:10:51,  9.70s/it]
 95%|█████████▍| 213426/225000 [413:58:18<29:11:31,  9.08s/it]
 95%|█████████▍| 213427/225000 [413:58:28<30:53:07,  9.61s/it]
 95%|█████████▍| 213428/225000 [413:58:34<27:19:11,  8.50s/it]
 95%|█████████▍| 213429/225000 [413:58:46<29:53:35,  9.30s/it]
 95%|█████████▍| 213430/225000 [413:58:51<25:49:06,  8.03s/it]
 95%|█████████▍| 213431/225000 [413:59:07<34:14:06, 10.65s/it]
 95%|█████████▍| 213432/225000 [413:59:14<30:28:28,  9.48s/it]
 95%|█████████▍| 213433/225000 [413:59:23<29:37:29,  9.22s/it]
 95%|█████████▍| 213434/225000 [413:59:31<28:27:44,  8.86s/it]
 95%|█████████▍| 213435/225000 [413:59:39<27:32:28,  8.57s/it]
 95%|█████████▍| 213436/225000 [413:59:44<23:59:55,  7.47s/it]
 95%|█████████▍| 213437/225000 [413:59:56<29:08:56,  9.08s/it]
 95%|█████████▍| 213438/225000 [414:00:02<25:29:36,  7.94s/it]
 95%|█████████▍| 213439/225000 [414:00:08<23:41:48,  7.38s/it]
 95%|█████████▍| 213440/225000 [414:00:20<28:13:01,  8.79s/it]
 95%|█████████▍| 213441/225000 [414:00:28<27:58:25,  8.71s/it]
 95%|█████████▍| 213442/225000 [414:00:37<27:46:34,  8.65s/it]
 95%|█████████▍| 213443/225000 [414:00:46<27:51:14,  8.68s/it]
 95%|█████████▍| 213444/225000 [414:00:59<32:42:17, 10.19s/it]
 95%|█████████▍| 213445/225000 [414:01:06<29:27:42,  9.18s/it]
 95%|█████████▍| 213446/225000 [414:01:14<28:04:43,  8.75s/it]
 95%|█████████▍| 213447/225000 [414:01:27<32:20:16, 10.08s/it]
 95%|█████████▍| 213448/225000 [414:01:34<28:53:45,  9.00s/it]
 95%|█████████▍| 213449/225000 [414:01:39<25:36:34,  7.98s/it]
 95%|█████████▍| 213450/225000 [414:01:47<25:23:57,  7.92s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.304', 'learning_rate': '3.614e-07', 'epoch': '0.9487', 'num_input_tokens_seen': 702125936, 'train_runtime': '1.491e+06', 'train_tokens_per_second': '471.1'}

 95%|█████████▍| 213450/225000 [414:01:47<25:23:57,  7.92s/it]
 95%|█████████▍| 213451/225000 [414:01:53<23:08:10,  7.21s/it]
 95%|█████████▍| 213452/225000 [414:02:07<29:46:30,  9.28s/it]
 95%|█████████▍| 213453/225000 [414:02:11<25:19:27,  7.90s/it]
 95%|█████████▍| 213454/225000 [414:02:21<27:05:26,  8.45s/it]
 95%|█████████▍| 213455/225000 [414:02:27<24:37:42,  7.68s/it]
 95%|█████████▍| 213456/225000 [414:02:34<24:14:29,  7.56s/it]
 95%|█████████▍| 213457/225000 [414:02:44<25:59:06,  8.10s/it]
 95%|█████████▍| 213458/225000 [414:02:50<24:48:40,  7.74s/it]
 95%|█████████▍| 213459/225000 [414:03:00<26:14:16,  8.18s/it]
 95%|█████████▍| 213460/225000 [414:03:07<25:31:42,  7.96s/it]
 95%|█████████▍| 213461/225000 [414:03:17<26:58:48,  8.42s/it]
 95%|█████████▍| 213462/225000 [414:03:22<23:46:16,  7.42s/it]
 95%|█████████▍| 213463/225000 [414:03:29<23:59:57,  7.49s/it]
 95%|█████████▍| 213464/225000 [414:03:42<28:40:36,  8.95s/it]
 95%|█████████▍| 213465/225000 [414:03:56<33:32:44, 10.47s/it]
 95%|█████████▍| 213466/225000 [414:04:02<29:55:28,  9.34s/it]
 95%|█████████▍| 213467/225000 [414:04:10<28:11:32,  8.80s/it]
 95%|█████████▍| 213468/225000 [414:04:23<32:33:48, 10.17s/it]
 95%|█████████▍| 213469/225000 [414:04:29<28:34:18,  8.92s/it]
 95%|█████████▍| 213470/225000 [414:04:40<30:44:09,  9.60s/it]
 95%|█████████▍| 213471/225000 [414:04:50<31:01:32,  9.69s/it]
 95%|█████████▍| 213472/225000 [414:05:00<30:55:33,  9.66s/it]
 95%|█████████▍| 213473/225000 [414:05:07<28:09:31,  8.79s/it]
 95%|█████████▍| 213474/225000 [414:05:22<34:13:06, 10.69s/it]
 95%|█████████▍| 213475/225000 [414:05:30<31:45:49,  9.92s/it]
 95%|█████████▍| 213476/225000 [414:05:37<29:20:15,  9.16s/it]
 95%|█████████▍| 213477/225000 [414:05:44<27:11:57,  8.50s/it]
 95%|█████████▍| 213478/225000 [414:05:52<26:04:17,  8.15s/it]
 95%|█████████▍| 213479/225000 [414:06:02<27:45:13,  8.67s/it]
 95%|█████████▍| 213480/225000 [414:06:08<25:15:24,  7.89s/it]
 95%|█████████▍| 213481/225000 [414:06:15<24:25:12,  7.63s/it]
 95%|█████████▍| 213482/225000 [414:06:28<29:58:06,  9.37s/it]
 95%|█████████▍| 213483/225000 [414:06:33<25:45:28,  8.05s/it]
 95%|█████████▍| 213484/225000 [414:06:41<25:11:38,  7.88s/it]
 95%|█████████▍| 213485/225000 [414:06:51<27:41:50,  8.66s/it]
 95%|█████████▍| 213486/225000 [414:06:57<25:09:02,  7.86s/it]
 95%|█████████▍| 213487/225000 [414:07:06<26:14:05,  8.20s/it]
 95%|█████████▍| 213488/225000 [414:07:18<29:24:04,  9.19s/it]
 95%|█████████▍| 213489/225000 [414:07:22<25:06:28,  7.85s/it]
 95%|█████████▍| 213490/225000 [414:07:34<28:33:51,  8.93s/it]
 95%|█████████▍| 213491/225000 [414:07:44<30:20:38,  9.49s/it]
 95%|█████████▍| 213492/225000 [414:07:52<28:06:33,  8.79s/it]
 95%|█████████▍| 213493/225000 [414:07:56<24:16:44,  7.60s/it]
 95%|█████████▍| 213494/225000 [414:08:08<27:47:55,  8.70s/it]
 95%|█████████▍| 213495/225000 [414:08:16<27:18:44,  8.55s/it]
 95%|█████████▍| 213496/225000 [414:08:28<30:23:27,  9.51s/it]
 95%|█████████▍| 213497/225000 [414:08:35<28:30:22,  8.92s/it]
 95%|█████████▍| 213498/225000 [414:08:42<26:45:23,  8.37s/it]
 95%|█████████▍| 213499/225000 [414:08:49<25:29:45,  7.98s/it]
 95%|█████████▍| 213500/225000 [414:08:56<23:59:01,  7.51s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.609', 'learning_rate': '3.583e-07', 'epoch': '0.9489', 'num_input_tokens_seen': 702349696, 'train_runtime': '1.491e+06', 'train_tokens_per_second': '471.1'}

 95%|█████████▍| 213500/225000 [414:08:56<23:59:01,  7.51s/it]
 95%|█████████▍| 213501/225000 [414:09:01<22:06:10,  6.92s/it]
 95%|█████████▍| 213502/225000 [414:09:13<26:21:07,  8.25s/it]
 95%|█████████▍| 213503/225000 [414:09:19<24:10:59,  7.57s/it]
 95%|█████████▍| 213504/225000 [414:09:30<27:33:34,  8.63s/it]
 95%|█████████▍| 213505/225000 [414:09:38<26:55:00,  8.43s/it]
 95%|█████████▍| 213506/225000 [414:09:45<25:49:06,  8.09s/it]
 95%|█████████▍| 213507/225000 [414:09:51<24:07:51,  7.56s/it]
 95%|█████████▍| 213508/225000 [414:10:04<29:11:31,  9.14s/it]
 95%|█████████▍| 213509/225000 [414:10:11<26:44:39,  8.38s/it]
 95%|█████████▍| 213510/225000 [414:10:21<28:45:53,  9.01s/it]
 95%|█████████▍| 213511/225000 [414:10:31<29:06:41,  9.12s/it]
 95%|█████████▍| 213512/225000 [414:10:38<27:03:40,  8.48s/it]
 95%|█████████▍| 213513/225000 [414:10:44<25:03:04,  7.85s/it]
 95%|█████████▍| 213514/225000 [414:10:55<27:59:15,  8.77s/it]
 95%|█████████▍| 213515/225000 [414:11:04<28:29:01,  8.93s/it]
 95%|█████████▍| 213516/225000 [414:11:11<25:59:31,  8.15s/it]
 95%|█████████▍| 213517/225000 [414:11:19<26:28:13,  8.30s/it]
 95%|█████████▍| 213518/225000 [414:11:25<24:07:07,  7.56s/it]
 95%|█████████▍| 213519/225000 [414:11:34<25:31:00,  8.00s/it]
 95%|█████████▍| 213520/225000 [414:11:44<27:09:02,  8.51s/it]
 95%|█████████▍| 213521/225000 [414:11:55<29:44:16,  9.33s/it]
 95%|█████████▍| 213522/225000 [414:12:01<26:45:46,  8.39s/it]
 95%|█████████▍| 213523/225000 [414:12:14<30:44:06,  9.64s/it]
 95%|█████████▍| 213524/225000 [414:12:21<28:26:36,  8.92s/it]
 95%|█████████▍| 213525/225000 [414:12:29<27:58:05,  8.77s/it]
 95%|█████████▍| 213526/225000 [414:12:36<25:52:35,  8.12s/it]
 95%|█████████▍| 213527/225000 [414:12:47<28:11:00,  8.84s/it]
 95%|█████████▍| 213528/225000 [414:12:51<23:37:03,  7.41s/it]
 95%|█████████▍| 213529/225000 [414:12:58<23:57:12,  7.52s/it]
 95%|█████████▍| 213530/225000 [414:13:03<21:03:03,  6.61s/it]
 95%|█████████▍| 213531/225000 [414:13:16<27:22:52,  8.59s/it]
 95%|█████████▍| 213532/225000 [414:13:26<29:02:11,  9.12s/it]
 95%|█████████▍| 213533/225000 [414:13:36<29:49:21,  9.36s/it]
 95%|█████████▍| 213534/225000 [414:13:46<30:21:01,  9.53s/it]
 95%|█████████▍| 213535/225000 [414:13:54<28:29:42,  8.95s/it]
 95%|█████████▍| 213536/225000 [414:13:59<24:47:17,  7.78s/it]
 95%|█████████▍| 213537/225000 [414:14:06<23:59:39,  7.54s/it]
 95%|█████████▍| 213538/225000 [414:14:15<25:35:37,  8.04s/it]
 95%|█████████▍| 213539/225000 [414:14:22<24:52:34,  7.81s/it]
 95%|█████████▍| 213540/225000 [414:14:34<28:13:28,  8.87s/it]
 95%|█████████▍| 213541/225000 [414:14:41<27:07:55,  8.52s/it]
 95%|█████████▍| 213542/225000 [414:14:47<24:40:59,  7.76s/it]
 95%|█████████▍| 213543/225000 [414:14:55<24:24:49,  7.67s/it]
 95%|█████████▍| 213544/225000 [414:15:02<23:40:28,  7.44s/it]
 95%|█████████▍| 213545/225000 [414:15:12<26:32:04,  8.34s/it]
 95%|█████████▍| 213546/225000 [414:15:19<24:41:39,  7.76s/it]
 95%|█████████▍| 213547/225000 [414:15:25<23:11:08,  7.29s/it]
 95%|█████████▍| 213548/225000 [414:15:39<29:16:57,  9.21s/it]
 95%|█████████▍| 213549/225000 [414:15:45<26:31:23,  8.34s/it]
 95%|█████████▍| 213550/225000 [414:15:57<30:20:27,  9.54s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.342', 'learning_rate': '3.552e-07', 'epoch': '0.9491', 'num_input_tokens_seen': 702576168, 'train_runtime': '1.491e+06', 'train_tokens_per_second': '471.1'}

 95%|█████████▍| 213550/225000 [414:15:57<30:20:27,  9.54s/it]
 95%|█████████▍| 213551/225000 [414:16:03<26:19:32,  8.28s/it]
 95%|█████████▍| 213552/225000 [414:16:09<24:35:47,  7.73s/it]
 95%|█████████▍| 213553/225000 [414:16:24<31:42:15,  9.97s/it]
 95%|█████████▍| 213554/225000 [414:16:30<27:59:42,  8.81s/it]
 95%|█████████▍| 213555/225000 [414:16:36<25:06:01,  7.90s/it]
 95%|█████████▍| 213556/225000 [414:16:47<27:44:51,  8.73s/it]
 95%|█████████▍| 213557/225000 [414:16:52<24:50:39,  7.82s/it]
 95%|█████████▍| 213558/225000 [414:17:06<30:13:02,  9.51s/it]
 95%|█████████▍| 213559/225000 [414:17:13<28:12:40,  8.88s/it]
 95%|█████████▍| 213560/225000 [414:17:19<24:53:51,  7.83s/it]
 95%|█████████▍| 213561/225000 [414:17:24<22:12:25,  6.99s/it]
 95%|█████████▍| 213562/225000 [414:17:29<20:59:13,  6.61s/it]
 95%|█████████▍| 213563/225000 [414:17:35<20:13:33,  6.37s/it]
 95%|█████████▍| 213564/225000 [414:17:53<31:04:18,  9.78s/it]
 95%|█████████▍| 213565/225000 [414:18:05<33:31:08, 10.55s/it]
 95%|█████████▍| 213566/225000 [414:18:11<28:40:41,  9.03s/it]
 95%|█████████▍| 213567/225000 [414:18:17<26:25:51,  8.32s/it]
 95%|█████████▍| 213568/225000 [414:18:27<27:38:38,  8.71s/it]
 95%|█████████▍| 213569/225000 [414:18:37<29:01:47,  9.14s/it]
 95%|█████████▍| 213570/225000 [414:18:44<26:54:59,  8.48s/it]
 95%|█████████▍| 213571/225000 [414:18:51<24:58:20,  7.87s/it]
 95%|█████████▍| 213572/225000 [414:18:58<24:42:23,  7.78s/it]
 95%|█████████▍| 213573/225000 [414:19:06<24:21:37,  7.67s/it]
 95%|█████████▍| 213574/225000 [414:19:17<27:39:03,  8.71s/it]
 95%|█████████▍| 213575/225000 [414:19:25<27:30:01,  8.67s/it]
 95%|█████████▍| 213576/225000 [414:19:31<25:07:23,  7.92s/it]
 95%|█████████▍| 213577/225000 [414:19:41<26:43:38,  8.42s/it]
 95%|█████████▍| 213578/225000 [414:19:49<26:10:12,  8.25s/it]
 95%|█████████▍| 213579/225000 [414:20:01<29:43:30,  9.37s/it]
 95%|█████████▍| 213580/225000 [414:20:05<24:38:23,  7.77s/it]
 95%|█████████▍| 213581/225000 [414:20:15<26:41:41,  8.42s/it]
 95%|█████████▍| 213582/225000 [414:20:28<31:14:21,  9.85s/it]
 95%|█████████▍| 213583/225000 [414:20:35<28:42:40,  9.05s/it]
 95%|█████████▍| 213584/225000 [414:20:42<26:16:48,  8.29s/it]
 95%|█████████▍| 213585/225000 [414:20:51<27:38:52,  8.72s/it]
 95%|█████████▍| 213586/225000 [414:21:01<27:58:02,  8.82s/it]
 95%|█████████▍| 213587/225000 [414:21:07<25:40:50,  8.10s/it]
 95%|█████████▍| 213588/225000 [414:21:14<24:21:43,  7.69s/it]
 95%|█████████▍| 213589/225000 [414:21:20<23:14:21,  7.33s/it]
 95%|█████████▍| 213590/225000 [414:21:26<21:55:53,  6.92s/it]
 95%|█████████▍| 213591/225000 [414:21:40<28:52:14,  9.11s/it]
 95%|█████████▍| 213592/225000 [414:21:46<25:35:05,  8.07s/it]
 95%|█████████▍| 213593/225000 [414:21:54<25:07:27,  7.93s/it]
 95%|█████████▍| 213594/225000 [414:22:03<26:06:57,  8.24s/it]
 95%|█████████▍| 213595/225000 [414:22:10<25:28:34,  8.04s/it]
 95%|█████████▍| 213596/225000 [414:22:22<29:03:52,  9.18s/it]
 95%|█████████▍| 213597/225000 [414:22:27<25:20:59,  8.00s/it]
 95%|█████████▍| 213598/225000 [414:22:39<29:13:45,  9.23s/it]
 95%|█████████▍| 213599/225000 [414:22:45<25:32:43,  8.07s/it]
 95%|█████████▍| 213600/225000 [414:22:58<30:26:59,  9.62s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.164', 'learning_rate': '3.521e-07', 'epoch': '0.9493', 'num_input_tokens_seen': 702805088, 'train_runtime': '1.492e+06', 'train_tokens_per_second': '471.1'}

 95%|█████████▍| 213600/225000 [414:22:58<30:26:59,  9.62s/it]
 95%|█████████▍| 213601/225000 [414:23:07<30:02:45,  9.49s/it]
 95%|█████████▍| 213602/225000 [414:23:13<26:36:22,  8.40s/it]
 95%|█████████▍| 213603/225000 [414:23:24<29:24:14,  9.29s/it]
 95%|█████████▍| 213604/225000 [414:23:32<28:17:18,  8.94s/it]
 95%|█████████▍| 213605/225000 [414:23:42<28:58:50,  9.16s/it]
 95%|█████████▍| 213606/225000 [414:23:47<25:24:08,  8.03s/it]
 95%|█████████▍| 213607/225000 [414:23:54<24:16:54,  7.67s/it]
 95%|█████████▍| 213608/225000 [414:24:03<24:55:16,  7.88s/it]
 95%|█████████▍| 213609/225000 [414:24:16<30:26:21,  9.62s/it]
 95%|█████████▍| 213610/225000 [414:24:22<26:50:40,  8.48s/it]
 95%|█████████▍| 213611/225000 [414:24:28<23:59:40,  7.58s/it]
 95%|█████████▍| 213612/225000 [414:24:38<26:56:00,  8.51s/it]
 95%|█████████▍| 213613/225000 [414:24:46<26:13:55,  8.29s/it]
 95%|█████████▍| 213614/225000 [414:24:59<30:55:20,  9.78s/it]
 95%|█████████▍| 213615/225000 [414:25:12<33:46:46, 10.68s/it]
 95%|█████████▍| 213616/225000 [414:25:18<29:21:39,  9.28s/it]
 95%|█████████▍| 213617/225000 [414:25:26<27:39:42,  8.75s/it]
 95%|█████████▍| 213618/225000 [414:25:32<25:22:09,  8.02s/it]
 95%|█████████▍| 213619/225000 [414:25:43<27:43:02,  8.77s/it]
 95%|█████████▍| 213620/225000 [414:25:51<27:24:39,  8.67s/it]
 95%|█████████▍| 213621/225000 [414:26:05<32:50:59, 10.39s/it]
 95%|█████████▍| 213622/225000 [414:26:11<28:19:19,  8.96s/it]
 95%|█████████▍| 213623/225000 [414:26:24<31:48:03, 10.06s/it]
 95%|█████████▍| 213624/225000 [414:26:33<31:06:09,  9.84s/it]
 95%|█████████▍| 213625/225000 [414:26:40<28:08:43,  8.91s/it]
 95%|█████████▍| 213626/225000 [414:26:52<30:58:44,  9.81s/it]
 95%|█████████▍| 213627/225000 [414:26:59<29:03:58,  9.20s/it]
 95%|█████████▍| 213628/225000 [414:27:06<26:22:53,  8.35s/it]
 95%|█████████▍| 213629/225000 [414:27:12<24:27:40,  7.74s/it]
 95%|█████████▍| 213630/225000 [414:27:19<23:34:39,  7.47s/it]
 95%|█████████▍| 213631/225000 [414:27:30<27:13:08,  8.62s/it]
 95%|█████████▍| 213632/225000 [414:27:43<31:10:42,  9.87s/it]
 95%|█████████▍| 213633/225000 [414:27:50<28:43:25,  9.10s/it]
 95%|█████████▍| 213634/225000 [414:28:02<30:57:40,  9.81s/it]
 95%|█████████▍| 213635/225000 [414:28:09<28:20:00,  8.97s/it]
 95%|█████████▍| 213636/225000 [414:28:18<28:26:16,  9.01s/it]
 95%|█████████▍| 213637/225000 [414:28:29<30:11:48,  9.57s/it]
 95%|█████████▍| 213638/225000 [414:28:37<28:44:15,  9.11s/it]
 95%|█████████▍| 213639/225000 [414:28:47<29:58:39,  9.50s/it]
 95%|█████████▍| 213640/225000 [414:28:52<25:28:04,  8.07s/it]
 95%|█████████▍| 213641/225000 [414:29:03<28:43:24,  9.10s/it]
 95%|█████████▍| 213642/225000 [414:29:10<26:02:22,  8.25s/it]
 95%|█████████▍| 213643/225000 [414:29:17<25:24:24,  8.05s/it]
 95%|█████████▍| 213644/225000 [414:29:23<22:58:16,  7.28s/it]
 95%|█████████▍| 213645/225000 [414:29:30<22:59:10,  7.29s/it]
 95%|█████████▍| 213646/225000 [414:29:44<29:16:12,  9.28s/it]
 95%|█████████▍| 213647/225000 [414:29:54<29:42:09,  9.42s/it]
 95%|█████████▍| 213648/225000 [414:30:03<29:03:17,  9.21s/it]
 95%|█████████▍| 213649/225000 [414:30:10<27:02:04,  8.57s/it]
 95%|█████████▍| 213650/225000 [414:30:19<27:55:07,  8.86s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.394', 'learning_rate': '3.491e-07', 'epoch': '0.9496', 'num_input_tokens_seen': 703039632, 'train_runtime': '1.492e+06', 'train_tokens_per_second': '471.1'}

 95%|█████████▍| 213650/225000 [414:30:19<27:55:07,  8.86s/it]
 95%|█████████▍| 213651/225000 [414:30:27<26:32:35,  8.42s/it]
 95%|█████████▍| 213652/225000 [414:30:39<30:34:55,  9.70s/it]
 95%|█████████▍| 213653/225000 [414:30:48<29:32:32,  9.37s/it]
 95%|█████████▍| 213654/225000 [414:30:53<25:11:43,  7.99s/it]
 95%|█████████▍| 213655/225000 [414:31:05<29:29:38,  9.36s/it]
 95%|█████████▍| 213656/225000 [414:31:12<27:26:39,  8.71s/it]
 95%|█████████▍| 213657/225000 [414:31:20<26:38:03,  8.45s/it]
 95%|█████████▍| 213658/225000 [414:31:27<24:56:21,  7.92s/it]
 95%|█████████▍| 213659/225000 [414:31:37<26:59:45,  8.57s/it]
 95%|█████████▍| 213660/225000 [414:31:44<25:39:27,  8.15s/it]
 95%|█████████▍| 213661/225000 [414:31:57<30:22:07,  9.64s/it]
 95%|█████████▍| 213662/225000 [414:32:05<28:47:29,  9.14s/it]
 95%|█████████▍| 213663/225000 [414:32:11<25:35:12,  8.12s/it]
 95%|█████████▍| 213664/225000 [414:32:24<30:20:45,  9.64s/it]
 95%|█████████▍| 213665/225000 [414:32:32<28:39:13,  9.10s/it]
 95%|█████████▍| 213666/225000 [414:32:43<30:43:37,  9.76s/it]
 95%|█████████▍| 213667/225000 [414:32:50<27:40:25,  8.79s/it]
 95%|█████████▍| 213668/225000 [414:32:58<27:00:51,  8.58s/it]
 95%|█████████▍| 213669/225000 [414:33:04<24:48:48,  7.88s/it]
 95%|█████████▍| 213670/225000 [414:33:17<29:26:13,  9.35s/it]
 95%|█████████▍| 213671/225000 [414:33:26<29:28:27,  9.37s/it]
 95%|█████████▍| 213672/225000 [414:33:34<27:58:43,  8.89s/it]
 95%|█████████▍| 213673/225000 [414:33:45<30:05:29,  9.56s/it]
 95%|█████████▍| 213674/225000 [414:33:55<30:00:47,  9.54s/it]
 95%|█████████▍| 213675/225000 [414:34:00<26:14:29,  8.34s/it]
 95%|█████████▍| 213676/225000 [414:34:09<26:23:19,  8.39s/it]
 95%|█████████▍| 213677/225000 [414:34:19<27:40:44,  8.80s/it]
 95%|█████████▍| 213678/225000 [414:34:26<26:47:50,  8.52s/it]
 95%|█████████▍| 213679/225000 [414:34:34<26:03:49,  8.29s/it]
 95%|█████████▍| 213680/225000 [414:34:41<24:54:03,  7.92s/it]
 95%|█████████▍| 213681/225000 [414:34:48<23:44:20,  7.55s/it]
 95%|█████████▍| 213682/225000 [414:35:03<31:15:48,  9.94s/it]
 95%|█████████▍| 213683/225000 [414:35:10<27:53:02,  8.87s/it]
 95%|█████████▍| 213684/225000 [414:35:17<26:23:51,  8.40s/it]
 95%|█████████▍| 213685/225000 [414:35:30<30:44:57,  9.78s/it]
 95%|█████████▍| 213686/225000 [414:35:37<28:08:59,  8.96s/it]
 95%|█████████▍| 213687/225000 [414:35:42<24:11:18,  7.70s/it]
 95%|█████████▍| 213688/225000 [414:35:52<26:51:28,  8.55s/it]
 95%|█████████▍| 213689/225000 [414:36:00<25:46:36,  8.20s/it]
 95%|█████████▍| 213690/225000 [414:36:07<24:40:33,  7.85s/it]
 95%|█████████▍| 213691/225000 [414:36:13<23:20:55,  7.43s/it]
 95%|█████████▍| 213692/225000 [414:36:19<22:09:52,  7.06s/it]
 95%|█████████▍| 213693/225000 [414:36:36<31:00:04,  9.87s/it]
 95%|█████████▍| 213694/225000 [414:36:42<27:09:07,  8.65s/it]
 95%|█████████▍| 213695/225000 [414:36:49<25:30:40,  8.12s/it]
 95%|█████████▍| 213696/225000 [414:36:59<27:55:34,  8.89s/it]
 95%|█████████▍| 213697/225000 [414:37:09<28:52:03,  9.19s/it]
 95%|█████████▍| 213698/225000 [414:37:20<30:00:04,  9.56s/it]
 95%|█████████▍| 213699/225000 [414:37:24<25:22:10,  8.08s/it]
 95%|█████████▍| 213700/225000 [414:37:33<26:14:11,  8.36s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.531', 'learning_rate': '3.46e-07', 'epoch': '0.9498', 'num_input_tokens_seen': 703266264, 'train_runtime': '1.493e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▍| 213700/225000 [414:37:33<26:14:11,  8.36s/it]
 95%|█████████▍| 213701/225000 [414:37:44<28:39:28,  9.13s/it]
 95%|█████████▍| 213702/225000 [414:37:53<28:06:46,  8.96s/it]
 95%|█████████▍| 213703/225000 [414:37:59<26:02:26,  8.30s/it]
 95%|█████████▍| 213704/225000 [414:38:12<30:00:43,  9.56s/it]
 95%|█████████▍| 213705/225000 [414:38:17<25:26:53,  8.11s/it]
 95%|█████████▍| 213706/225000 [414:38:25<25:36:56,  8.17s/it]
 95%|█████████▍| 213707/225000 [414:38:34<26:21:12,  8.40s/it]
 95%|█████████▍| 213708/225000 [414:38:41<25:04:05,  7.99s/it]
 95%|█████████▍| 213709/225000 [414:38:50<25:33:00,  8.15s/it]
 95%|█████████▍| 213710/225000 [414:39:02<29:28:19,  9.40s/it]
 95%|█████████▍| 213711/225000 [414:39:08<26:00:09,  8.29s/it]
 95%|█████████▍| 213712/225000 [414:39:13<23:41:02,  7.55s/it]
 95%|█████████▍| 213713/225000 [414:39:19<21:49:11,  6.96s/it]
 95%|█████████▍| 213714/225000 [414:39:31<27:03:58,  8.63s/it]
 95%|█████████▍| 213715/225000 [414:39:41<27:35:37,  8.80s/it]
 95%|█████████▍| 213716/225000 [414:39:49<27:01:53,  8.62s/it]
 95%|█████████▍| 213717/225000 [414:40:03<32:01:32, 10.22s/it]
 95%|█████████▍| 213718/225000 [414:40:11<29:59:15,  9.57s/it]
 95%|█████████▍| 213719/225000 [414:40:17<26:37:06,  8.49s/it]
 95%|█████████▍| 213720/225000 [414:40:25<25:50:15,  8.25s/it]
 95%|█████████▍| 213721/225000 [414:40:31<24:36:03,  7.85s/it]
 95%|█████████▍| 213722/225000 [414:40:43<27:54:41,  8.91s/it]
 95%|█████████▍| 213723/225000 [414:40:50<26:33:26,  8.48s/it]
 95%|█████████▍| 213724/225000 [414:41:04<31:08:20,  9.94s/it]
 95%|█████████▍| 213725/225000 [414:41:10<28:09:58,  8.99s/it]
 95%|█████████▍| 213726/225000 [414:41:17<25:35:13,  8.17s/it]
 95%|█████████▍| 213727/225000 [414:41:27<27:15:28,  8.70s/it]
 95%|█████████▍| 213728/225000 [414:41:32<24:13:49,  7.74s/it]
 95%|█████████▍| 213729/225000 [414:41:39<23:20:44,  7.46s/it]
 95%|█████████▍| 213730/225000 [414:41:49<26:15:07,  8.39s/it]
 95%|█████████▍| 213731/225000 [414:41:59<27:07:40,  8.67s/it]
 95%|█████████▍| 213732/225000 [414:42:06<25:19:41,  8.09s/it]
 95%|█████████▍| 213733/225000 [414:42:19<30:32:39,  9.76s/it]
 95%|█████████▍| 213734/225000 [414:42:30<31:36:20, 10.10s/it]
 95%|█████████▍| 213735/225000 [414:42:35<26:20:28,  8.42s/it]
 95%|█████████▍| 213736/225000 [414:42:39<22:52:02,  7.31s/it]
 95%|█████████▍| 213737/225000 [414:42:51<27:06:13,  8.66s/it]
 95%|█████████▍| 213738/225000 [414:42:58<25:52:33,  8.27s/it]
 95%|█████████▍| 213739/225000 [414:43:06<24:59:22,  7.99s/it]
 95%|█████████▍| 213740/225000 [414:43:11<22:40:06,  7.25s/it]
 95%|█████████▍| 213741/225000 [414:43:23<26:23:28,  8.44s/it]
 95%|█████████▍| 213742/225000 [414:43:27<22:57:16,  7.34s/it]
 95%|█████████▍| 213743/225000 [414:43:38<26:11:48,  8.38s/it]
 95%|█████████▍| 213744/225000 [414:43:46<25:28:41,  8.15s/it]
 95%|█████████▍| 213745/225000 [414:43:52<23:44:02,  7.59s/it]
 95%|█████████▍| 213746/225000 [414:44:01<24:38:05,  7.88s/it]
 95%|█████████▍| 213747/225000 [414:44:08<24:19:34,  7.78s/it]
 95%|█████████▍| 213748/225000 [414:44:18<26:13:42,  8.39s/it]
 95%|█████████▍| 213749/225000 [414:44:29<28:56:25,  9.26s/it]
 95%|█████████▌| 213750/225000 [414:44:37<27:39:53,  8.85s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.735', 'learning_rate': '3.43e-07', 'epoch': '0.95', 'num_input_tokens_seen': 703487528, 'train_runtime': '1.493e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 213750/225000 [414:44:37<27:39:53,  8.85s/it]
 95%|█████████▌| 213751/225000 [414:44:49<30:03:08,  9.62s/it]
 95%|█████████▌| 213752/225000 [414:44:55<27:04:28,  8.67s/it]
 95%|█████████▌| 213753/225000 [414:45:01<24:13:52,  7.76s/it]
 95%|█████████▌| 213754/225000 [414:45:14<29:29:00,  9.44s/it]
 95%|█████████▌| 213755/225000 [414:45:18<24:48:44,  7.94s/it]
 95%|█████████▌| 213756/225000 [414:45:24<22:15:32,  7.13s/it]
 95%|█████████▌| 213757/225000 [414:45:35<26:35:59,  8.52s/it]
 95%|█████████▌| 213758/225000 [414:45:46<28:04:44,  8.99s/it]
 95%|█████████▌| 213759/225000 [414:45:54<27:53:26,  8.93s/it]
 95%|█████████▌| 213760/225000 [414:45:59<24:05:58,  7.72s/it]
 95%|█████████▌| 213761/225000 [414:46:10<26:56:50,  8.63s/it]
 95%|█████████▌| 213762/225000 [414:46:19<27:42:24,  8.88s/it]
 95%|█████████▌| 213763/225000 [414:46:34<33:13:49, 10.65s/it]
 95%|█████████▌| 213764/225000 [414:46:40<28:59:26,  9.29s/it]
 95%|█████████▌| 213765/225000 [414:46:52<30:59:37,  9.93s/it]
 95%|█████████▌| 213766/225000 [414:47:00<29:49:28,  9.56s/it]
 95%|█████████▌| 213767/225000 [414:47:06<26:26:39,  8.47s/it]
 95%|█████████▌| 213768/225000 [414:47:21<32:19:02, 10.36s/it]
 95%|█████████▌| 213769/225000 [414:47:30<31:06:20,  9.97s/it]
 95%|█████████▌| 213770/225000 [414:47:38<28:43:20,  9.21s/it]
 95%|█████████▌| 213771/225000 [414:47:46<27:46:46,  8.91s/it]
 95%|█████████▌| 213772/225000 [414:47:59<31:36:35, 10.13s/it]
 95%|█████████▌| 213773/225000 [414:48:04<26:57:09,  8.64s/it]
 95%|█████████▌| 213774/225000 [414:48:18<31:30:37, 10.10s/it]
 95%|█████████▌| 213775/225000 [414:48:27<30:49:27,  9.89s/it]
 95%|█████████▌| 213776/225000 [414:48:36<30:15:30,  9.71s/it]
 95%|█████████▌| 213777/225000 [414:48:43<27:07:45,  8.70s/it]
 95%|█████████▌| 213778/225000 [414:48:49<25:28:19,  8.17s/it]
 95%|█████████▌| 213779/225000 [414:48:56<23:33:38,  7.56s/it]
 95%|█████████▌| 213780/225000 [414:49:07<27:20:44,  8.77s/it]
 95%|█████████▌| 213781/225000 [414:49:13<24:16:25,  7.79s/it]
 95%|█████████▌| 213782/225000 [414:49:25<28:34:01,  9.17s/it]
 95%|█████████▌| 213783/225000 [414:49:34<27:55:00,  8.96s/it]
 95%|█████████▌| 213784/225000 [414:49:44<29:08:53,  9.36s/it]
 95%|█████████▌| 213785/225000 [414:49:49<25:21:35,  8.14s/it]
 95%|█████████▌| 213786/225000 [414:50:01<29:14:10,  9.39s/it]
 95%|█████████▌| 213787/225000 [414:50:10<28:42:30,  9.22s/it]
 95%|█████████▌| 213788/225000 [414:50:21<30:32:04,  9.80s/it]
 95%|█████████▌| 213789/225000 [414:50:29<28:45:45,  9.24s/it]
 95%|█████████▌| 213790/225000 [414:50:42<31:39:52, 10.17s/it]
 95%|█████████▌| 213791/225000 [414:50:54<33:49:47, 10.87s/it]
 95%|█████████▌| 213792/225000 [414:51:01<30:04:42,  9.66s/it]
 95%|█████████▌| 213793/225000 [414:51:06<25:17:14,  8.12s/it]
 95%|█████████▌| 213794/225000 [414:51:13<24:16:19,  7.80s/it]
 95%|█████████▌| 213795/225000 [414:51:20<24:05:42,  7.74s/it]
 95%|█████████▌| 213796/225000 [414:51:35<30:32:11,  9.81s/it]
 95%|█████████▌| 213797/225000 [414:51:42<27:42:51,  8.91s/it]
 95%|█████████▌| 213798/225000 [414:51:53<30:00:13,  9.64s/it]
 95%|█████████▌| 213799/225000 [414:51:59<26:39:54,  8.57s/it]
 95%|█████████▌| 213800/225000 [414:52:10<28:59:59,  9.32s/it]
                                                              
{'loss': '1.34', 'grad_norm': '1.399', 'learning_rate': '3.4e-07', 'epoch': '0.9502', 'num_input_tokens_seen': 703721824, 'train_runtime': '1.494e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 213800/225000 [414:52:10<28:59:59,  9.32s/it]
 95%|█████████▌| 213801/225000 [414:52:19<28:48:18,  9.26s/it]
 95%|█████████▌| 213802/225000 [414:52:27<27:10:06,  8.73s/it]
 95%|█████████▌| 213803/225000 [414:52:33<25:16:03,  8.12s/it]
 95%|█████████▌| 213804/225000 [414:52:43<26:17:30,  8.45s/it]
 95%|█████████▌| 213805/225000 [414:52:54<28:42:53,  9.23s/it]
 95%|█████████▌| 213806/225000 [414:53:03<28:28:52,  9.16s/it]
 95%|█████████▌| 213807/225000 [414:53:07<24:12:12,  7.78s/it]
 95%|█████████▌| 213808/225000 [414:53:15<24:00:49,  7.72s/it]
 95%|█████████▌| 213809/225000 [414:53:31<31:29:57, 10.13s/it]
 95%|█████████▌| 213810/225000 [414:53:37<28:04:23,  9.03s/it]
 95%|█████████▌| 213811/225000 [414:53:53<34:32:01, 11.11s/it]
 95%|█████████▌| 213812/225000 [414:53:59<30:09:46,  9.71s/it]
 95%|█████████▌| 213813/225000 [414:54:09<30:07:52,  9.70s/it]
 95%|█████████▌| 213814/225000 [414:54:14<25:19:44,  8.15s/it]
 95%|█████████▌| 213815/225000 [414:54:23<26:36:40,  8.57s/it]
 95%|█████████▌| 213816/225000 [414:54:33<28:07:01,  9.05s/it]
 95%|█████████▌| 213817/225000 [414:54:40<25:36:47,  8.25s/it]
 95%|█████████▌| 213818/225000 [414:54:51<27:55:45,  8.99s/it]
 95%|█████████▌| 213819/225000 [414:55:00<28:38:46,  9.22s/it]
 95%|█████████▌| 213820/225000 [414:55:06<25:34:49,  8.24s/it]
 95%|█████████▌| 213821/225000 [414:55:16<26:44:09,  8.61s/it]
 95%|█████████▌| 213822/225000 [414:55:25<27:47:47,  8.95s/it]
 95%|█████████▌| 213823/225000 [414:55:34<27:32:03,  8.87s/it]
 95%|█████████▌| 213824/225000 [414:55:43<27:46:50,  8.95s/it]
 95%|█████████▌| 213825/225000 [414:55:50<25:37:50,  8.26s/it]
 95%|█████████▌| 213826/225000 [414:55:57<24:58:17,  8.05s/it]
 95%|█████████▌| 213827/225000 [414:56:07<26:02:45,  8.39s/it]
 95%|█████████▌| 213828/225000 [414:56:20<30:17:55,  9.76s/it]
 95%|█████████▌| 213829/225000 [414:56:28<29:02:12,  9.36s/it]
 95%|█████████▌| 213830/225000 [414:56:35<26:50:29,  8.65s/it]
 95%|█████████▌| 213831/225000 [414:56:41<24:22:10,  7.85s/it]
 95%|█████████▌| 213832/225000 [414:56:52<27:15:10,  8.78s/it]
 95%|█████████▌| 213833/225000 [414:57:00<26:05:53,  8.41s/it]
 95%|█████████▌| 213834/225000 [414:57:11<29:04:45,  9.38s/it]
 95%|█████████▌| 213835/225000 [414:57:18<26:17:28,  8.48s/it]
 95%|█████████▌| 213836/225000 [414:57:30<30:05:28,  9.70s/it]
 95%|█████████▌| 213837/225000 [414:57:36<26:50:10,  8.65s/it]
 95%|█████████▌| 213838/225000 [414:57:48<30:03:23,  9.69s/it]
 95%|█████████▌| 213839/225000 [414:58:01<32:50:42, 10.59s/it]
 95%|█████████▌| 213840/225000 [414:58:10<31:30:39, 10.16s/it]
 95%|█████████▌| 213841/225000 [414:58:16<27:06:13,  8.74s/it]
 95%|█████████▌| 213842/225000 [414:58:31<32:55:44, 10.62s/it]
 95%|█████████▌| 213843/225000 [414:58:36<28:03:42,  9.05s/it]
 95%|█████████▌| 213844/225000 [414:58:46<29:02:32,  9.37s/it]
 95%|█████████▌| 213845/225000 [414:58:55<28:46:50,  9.29s/it]
 95%|█████████▌| 213846/225000 [414:59:08<31:49:30, 10.27s/it]
 95%|█████████▌| 213847/225000 [414:59:13<27:12:33,  8.78s/it]
 95%|█████████▌| 213848/225000 [414:59:22<27:24:11,  8.85s/it]
 95%|█████████▌| 213849/225000 [414:59:33<29:17:07,  9.45s/it]
 95%|█████████▌| 213850/225000 [414:59:38<24:52:37,  8.03s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.54', 'learning_rate': '3.37e-07', 'epoch': '0.9504', 'num_input_tokens_seen': 703960896, 'train_runtime': '1.494e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 213850/225000 [414:59:38<24:52:37,  8.03s/it]
 95%|█████████▌| 213851/225000 [414:59:49<27:57:08,  9.03s/it]
 95%|█████████▌| 213852/225000 [414:59:59<29:00:50,  9.37s/it]
 95%|█████████▌| 213853/225000 [415:00:09<29:29:10,  9.52s/it]
 95%|█████████▌| 213854/225000 [415:00:20<30:47:53,  9.95s/it]
 95%|█████████▌| 213855/225000 [415:00:28<28:32:55,  9.22s/it]
 95%|█████████▌| 213856/225000 [415:00:35<26:31:32,  8.57s/it]
 95%|█████████▌| 213857/225000 [415:00:42<25:11:54,  8.14s/it]
 95%|█████████▌| 213858/225000 [415:00:48<23:31:22,  7.60s/it]
 95%|█████████▌| 213859/225000 [415:00:59<26:46:32,  8.65s/it]
 95%|█████████▌| 213860/225000 [415:01:05<24:29:00,  7.91s/it]
 95%|█████████▌| 213861/225000 [415:01:13<23:46:33,  7.68s/it]
 95%|█████████▌| 213862/225000 [415:01:19<22:59:32,  7.43s/it]
 95%|█████████▌| 213863/225000 [415:01:34<29:12:12,  9.44s/it]
 95%|█████████▌| 213864/225000 [415:01:40<26:26:40,  8.55s/it]
 95%|█████████▌| 213865/225000 [415:01:49<26:40:42,  8.63s/it]
 95%|█████████▌| 213866/225000 [415:02:01<30:11:37,  9.76s/it]
 95%|█████████▌| 213867/225000 [415:02:09<27:53:28,  9.02s/it]
 95%|█████████▌| 213868/225000 [415:02:18<28:25:52,  9.19s/it]
 95%|█████████▌| 213869/225000 [415:02:33<34:01:32, 11.00s/it]
 95%|█████████▌| 213870/225000 [415:02:40<29:44:04,  9.62s/it]
 95%|█████████▌| 213871/225000 [415:02:51<30:46:10,  9.95s/it]
 95%|█████████▌| 213872/225000 [415:02:58<28:02:23,  9.07s/it]
 95%|█████████▌| 213873/225000 [415:03:09<30:28:09,  9.86s/it]
 95%|█████████▌| 213874/225000 [415:03:19<30:11:39,  9.77s/it]
 95%|█████████▌| 213875/225000 [415:03:26<27:28:49,  8.89s/it]
 95%|█████████▌| 213876/225000 [415:03:35<27:57:33,  9.05s/it]
 95%|█████████▌| 213877/225000 [415:03:41<24:59:12,  8.09s/it]
 95%|█████████▌| 213878/225000 [415:03:48<23:57:27,  7.75s/it]
 95%|█████████▌| 213879/225000 [415:04:00<27:51:45,  9.02s/it]
 95%|█████████▌| 213880/225000 [415:04:07<26:11:04,  8.48s/it]
 95%|█████████▌| 213881/225000 [415:04:15<25:33:41,  8.28s/it]
 95%|█████████▌| 213882/225000 [415:04:24<26:25:16,  8.56s/it]
 95%|█████████▌| 213883/225000 [415:04:30<23:56:31,  7.75s/it]
 95%|█████████▌| 213884/225000 [415:04:38<24:23:05,  7.90s/it]
 95%|█████████▌| 213885/225000 [415:04:50<28:13:48,  9.14s/it]
 95%|█████████▌| 213886/225000 [415:04:55<24:12:28,  7.84s/it]
 95%|█████████▌| 213887/225000 [415:05:04<25:28:34,  8.25s/it]
 95%|█████████▌| 213888/225000 [415:05:10<23:32:37,  7.63s/it]
 95%|█████████▌| 213889/225000 [415:05:17<22:12:11,  7.19s/it]
 95%|█████████▌| 213890/225000 [415:05:32<29:39:19,  9.61s/it]
 95%|█████████▌| 213891/225000 [415:05:41<29:03:13,  9.42s/it]
 95%|█████████▌| 213892/225000 [415:05:53<31:17:39, 10.14s/it]
 95%|█████████▌| 213893/225000 [415:05:57<25:44:42,  8.34s/it]
 95%|█████████▌| 213894/225000 [415:06:06<26:40:45,  8.65s/it]
 95%|█████████▌| 213895/225000 [415:06:14<25:55:30,  8.40s/it]
 95%|█████████▌| 213896/225000 [415:06:21<24:58:59,  8.10s/it]
 95%|█████████▌| 213897/225000 [415:06:28<23:41:16,  7.68s/it]
 95%|█████████▌| 213898/225000 [415:06:37<24:54:35,  8.08s/it]
 95%|█████████▌| 213899/225000 [415:06:47<26:58:48,  8.75s/it]
 95%|█████████▌| 213900/225000 [415:06:57<27:42:47,  8.99s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.936', 'learning_rate': '3.34e-07', 'epoch': '0.9507', 'num_input_tokens_seen': 704191888, 'train_runtime': '1.494e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 213900/225000 [415:06:57<27:42:47,  8.99s/it]
 95%|█████████▌| 213901/225000 [415:07:04<26:17:56,  8.53s/it]
 95%|█████████▌| 213902/225000 [415:07:14<27:44:46,  9.00s/it]
 95%|█████████▌| 213903/225000 [415:07:20<24:40:42,  8.01s/it]
 95%|█████████▌| 213904/225000 [415:07:26<22:35:43,  7.33s/it]
 95%|█████████▌| 213905/225000 [415:07:33<22:09:43,  7.19s/it]
 95%|█████████▌| 213906/225000 [415:07:48<29:45:25,  9.66s/it]
 95%|█████████▌| 213907/225000 [415:07:53<25:20:04,  8.22s/it]
 95%|█████████▌| 213908/225000 [415:08:04<28:03:27,  9.11s/it]
 95%|█████████▌| 213909/225000 [415:08:12<27:09:58,  8.82s/it]
 95%|█████████▌| 213910/225000 [415:08:26<31:41:31, 10.29s/it]
 95%|█████████▌| 213911/225000 [415:08:36<31:10:31, 10.12s/it]
 95%|█████████▌| 213912/225000 [415:08:44<29:06:51,  9.45s/it]
 95%|█████████▌| 213913/225000 [415:08:56<31:36:18, 10.26s/it]
 95%|█████████▌| 213914/225000 [415:09:06<31:45:12, 10.31s/it]
 95%|█████████▌| 213915/225000 [415:09:12<27:43:34,  9.00s/it]
 95%|█████████▌| 213916/225000 [415:09:25<31:13:44, 10.14s/it]
 95%|█████████▌| 213917/225000 [415:09:33<29:30:45,  9.59s/it]
 95%|█████████▌| 213918/225000 [415:09:41<27:58:02,  9.09s/it]
 95%|█████████▌| 213919/225000 [415:09:47<24:28:02,  7.95s/it]
 95%|█████████▌| 213920/225000 [415:10:00<29:41:28,  9.65s/it]
 95%|█████████▌| 213921/225000 [415:10:06<25:47:00,  8.38s/it]
 95%|█████████▌| 213922/225000 [415:10:12<24:05:58,  7.83s/it]
 95%|█████████▌| 213923/225000 [415:10:26<29:18:05,  9.52s/it]
 95%|█████████▌| 213924/225000 [415:10:36<30:14:33,  9.83s/it]
 95%|█████████▌| 213925/225000 [415:10:46<30:07:45,  9.79s/it]
 95%|█████████▌| 213926/225000 [415:10:58<32:36:37, 10.60s/it]
 95%|█████████▌| 213927/225000 [415:11:05<28:37:10,  9.30s/it]
 95%|█████████▌| 213928/225000 [415:11:14<28:38:55,  9.31s/it]
 95%|█████████▌| 213929/225000 [415:11:22<27:09:11,  8.83s/it]
 95%|█████████▌| 213930/225000 [415:11:27<23:58:58,  7.80s/it]
 95%|█████████▌| 213931/225000 [415:11:40<28:24:56,  9.24s/it]
 95%|█████████▌| 213932/225000 [415:11:45<25:11:34,  8.19s/it]
 95%|█████████▌| 213933/225000 [415:11:50<21:56:30,  7.14s/it]
 95%|█████████▌| 213934/225000 [415:12:01<25:15:09,  8.22s/it]
 95%|█████████▌| 213935/225000 [415:12:07<23:38:58,  7.69s/it]
 95%|█████████▌| 213936/225000 [415:12:20<28:34:25,  9.30s/it]
 95%|█████████▌| 213937/225000 [415:12:31<30:08:59,  9.81s/it]
 95%|█████████▌| 213938/225000 [415:12:41<29:36:53,  9.64s/it]
 95%|█████████▌| 213939/225000 [415:12:51<30:07:23,  9.80s/it]
 95%|█████████▌| 213940/225000 [415:12:57<26:47:42,  8.72s/it]
 95%|█████████▌| 213941/225000 [415:13:09<29:40:00,  9.66s/it]
 95%|█████████▌| 213942/225000 [415:13:16<26:58:24,  8.78s/it]
 95%|█████████▌| 213943/225000 [415:13:21<23:41:57,  7.72s/it]
 95%|█████████▌| 213944/225000 [415:13:29<23:56:39,  7.80s/it]
 95%|█████████▌| 213945/225000 [415:13:40<27:09:42,  8.85s/it]
 95%|█████████▌| 213946/225000 [415:13:49<27:38:39,  9.00s/it]
 95%|█████████▌| 213947/225000 [415:13:56<25:11:50,  8.21s/it]
 95%|█████████▌| 213948/225000 [415:14:06<27:00:10,  8.80s/it]
 95%|█████████▌| 213949/225000 [415:14:12<24:45:38,  8.07s/it]
 95%|█████████▌| 213950/225000 [415:14:24<28:13:57,  9.20s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.218', 'learning_rate': '3.311e-07', 'epoch': '0.9509', 'num_input_tokens_seen': 704424744, 'train_runtime': '1.495e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 213950/225000 [415:14:24<28:13:57,  9.20s/it]
 95%|█████████▌| 213951/225000 [415:14:29<24:23:28,  7.95s/it]
 95%|█████████▌| 213952/225000 [415:14:38<25:38:19,  8.35s/it]
 95%|█████████▌| 213953/225000 [415:14:45<24:14:20,  7.90s/it]
 95%|█████████▌| 213954/225000 [415:14:53<24:19:27,  7.93s/it]
 95%|█████████▌| 213955/225000 [415:15:04<26:36:00,  8.67s/it]
 95%|█████████▌| 213956/225000 [415:15:17<30:45:03, 10.02s/it]
 95%|█████████▌| 213957/225000 [415:15:27<30:34:09,  9.97s/it]
 95%|█████████▌| 213958/225000 [415:15:37<30:34:22,  9.97s/it]
 95%|█████████▌| 213959/225000 [415:15:44<27:46:16,  9.05s/it]
 95%|█████████▌| 213960/225000 [415:15:51<26:21:52,  8.60s/it]
 95%|█████████▌| 213961/225000 [415:16:03<29:47:29,  9.72s/it]
 95%|█████████▌| 213962/225000 [415:16:12<28:25:33,  9.27s/it]
 95%|█████████▌| 213963/225000 [415:16:25<32:22:37, 10.56s/it]
 95%|█████████▌| 213964/225000 [415:16:31<27:57:20,  9.12s/it]
 95%|█████████▌| 213965/225000 [415:16:41<28:27:48,  9.29s/it]
 95%|█████████▌| 213966/225000 [415:16:51<29:37:12,  9.66s/it]
 95%|█████████▌| 213967/225000 [415:17:01<29:44:58,  9.71s/it]
 95%|█████████▌| 213968/225000 [415:17:11<30:02:33,  9.80s/it]
 95%|█████████▌| 213969/225000 [415:17:18<26:56:27,  8.79s/it]
 95%|█████████▌| 213970/225000 [415:17:29<29:35:25,  9.66s/it]
 95%|█████████▌| 213971/225000 [415:17:37<27:33:53,  9.00s/it]
 95%|█████████▌| 213972/225000 [415:17:48<29:21:31,  9.58s/it]
 95%|█████████▌| 213973/225000 [415:17:59<31:14:36, 10.20s/it]
 95%|█████████▌| 213974/225000 [415:18:12<33:25:04, 10.91s/it]
 95%|█████████▌| 213975/225000 [415:18:20<30:39:03, 10.01s/it]
 95%|█████████▌| 213976/225000 [415:18:24<25:43:49,  8.40s/it]
 95%|█████████▌| 213977/225000 [415:18:33<25:39:45,  8.38s/it]
 95%|█████████▌| 213978/225000 [415:18:40<24:22:51,  7.96s/it]
 95%|█████████▌| 213979/225000 [415:18:48<24:17:01,  7.93s/it]
 95%|█████████▌| 213980/225000 [415:18:58<26:37:00,  8.70s/it]
 95%|█████████▌| 213981/225000 [415:19:08<27:40:10,  9.04s/it]
 95%|█████████▌| 213982/225000 [415:19:15<25:43:31,  8.41s/it]
 95%|█████████▌| 213983/225000 [415:19:21<23:26:17,  7.66s/it]
 95%|█████████▌| 213984/225000 [415:19:32<26:32:22,  8.67s/it]
 95%|█████████▌| 213985/225000 [415:19:39<25:01:26,  8.18s/it]
 95%|█████████▌| 213986/225000 [415:19:46<24:20:13,  7.95s/it]
 95%|█████████▌| 213987/225000 [415:19:54<23:42:53,  7.75s/it]
 95%|█████████▌| 213988/225000 [415:20:00<22:09:08,  7.24s/it]
 95%|█████████▌| 213989/225000 [415:20:13<27:48:28,  9.09s/it]
 95%|█████████▌| 213990/225000 [415:20:18<24:28:22,  8.00s/it]
 95%|█████████▌| 213991/225000 [415:20:26<23:41:53,  7.75s/it]
 95%|█████████▌| 213992/225000 [415:20:35<25:39:00,  8.39s/it]
 95%|█████████▌| 213993/225000 [415:20:46<27:29:01,  8.99s/it]
 95%|█████████▌| 213994/225000 [415:20:56<28:50:04,  9.43s/it]
 95%|█████████▌| 213995/225000 [415:21:07<30:07:39,  9.86s/it]
 95%|█████████▌| 213996/225000 [415:21:14<27:02:34,  8.85s/it]
 95%|█████████▌| 213997/225000 [415:21:21<25:54:52,  8.48s/it]
 95%|█████████▌| 213998/225000 [415:21:27<23:35:03,  7.72s/it]
 95%|█████████▌| 213999/225000 [415:21:41<28:53:29,  9.45s/it]
 95%|█████████▌| 214000/225000 [415:21:52<30:43:47, 10.06s/it]
                                                              
{'loss': '1.286', 'grad_norm': '1.18', 'learning_rate': '3.281e-07', 'epoch': '0.9511', 'num_input_tokens_seen': 704657760, 'train_runtime': '1.495e+06', 'train_tokens_per_second': '471.2'}

 95%|█████████▌| 214000/225000 [415:21:52<30:43:47, 10.06s/it]
 95%|█████████▌| 214001/225000 [415:22:02<30:52:04, 10.10s/it]
 95%|█████████▌| 214002/225000 [415:22:08<27:00:13,  8.84s/it]
 95%|█████████▌| 214003/225000 [415:22:15<25:23:09,  8.31s/it]
 95%|█████████▌| 214004/225000 [415:22:33<34:04:28, 11.16s/it]
 95%|█████████▌| 214005/225000 [415:22:39<29:26:38,  9.64s/it]
 95%|█████████▌| 214006/225000 [415:22:46<26:29:06,  8.67s/it]
 95%|█████████▌| 214007/225000 [415:22:59<30:31:45, 10.00s/it]
 95%|█████████▌| 214008/225000 [415:23:03<25:33:15,  8.37s/it]
 95%|█████████▌| 214009/225000 [415:23:12<25:51:51,  8.47s/it]
 95%|█████████▌| 214010/225000 [415:23:19<24:39:56,  8.08s/it]
 95%|█████████▌| 214011/225000 [415:23:25<22:22:36,  7.33s/it]
 95%|█████████▌| 214012/225000 [415:23:31<21:25:20,  7.02s/it]
 95%|█████████▌| 214013/225000 [415:23:37<20:41:40,  6.78s/it]
 95%|█████████▌| 214014/225000 [415:23:47<23:15:46,  7.62s/it]
 95%|█████████▌| 214015/225000 [415:24:02<29:52:58,  9.79s/it]
 95%|█████████▌| 214016/225000 [415:24:12<30:33:17, 10.01s/it]
 95%|█████████▌| 214017/225000 [415:24:23<31:29:21, 10.32s/it]
 95%|█████████▌| 214018/225000 [415:24:30<27:58:22,  9.17s/it]
 95%|█████████▌| 214019/225000 [415:24:41<29:27:46,  9.66s/it]
 95%|█████████▌| 214020/225000 [415:24:47<26:51:13,  8.80s/it]
 95%|█████████▌| 214021/225000 [415:24:58<28:21:56,  9.30s/it]
 95%|█████████▌| 214022/225000 [415:25:11<31:45:27, 10.41s/it]
 95%|█████████▌| 214023/225000 [415:25:17<27:36:14,  9.05s/it]
 95%|█████████▌| 214024/225000 [415:25:28<29:09:02,  9.56s/it]
 95%|█████████▌| 214025/225000 [415:25:33<25:47:42,  8.46s/it]
 95%|█████████▌| 214026/225000 [415:25:45<28:31:09,  9.36s/it]
 95%|█████████▌| 214027/225000 [415:25:53<27:04:48,  8.88s/it]
 95%|█████████▌| 214028/225000 [415:26:02<27:14:14,  8.94s/it]
 95%|█████████▌| 214029/225000 [415:26:08<24:36:18,  8.07s/it]
 95%|█████████▌| 214030/225000 [415:26:16<24:36:50,  8.08s/it]
 95%|█████████▌| 214031/225000 [415:26:26<26:51:00,  8.81s/it]
 95%|█████████▌| 214032/225000 [415:26:33<24:38:12,  8.09s/it]
 95%|█████████▌| 214033/225000 [415:26:44<27:49:10,  9.13s/it]
 95%|█████████▌| 214034/225000 [415:26:55<29:28:09,  9.67s/it]
 95%|█████████▌| 214035/225000 [415:27:02<27:08:47,  8.91s/it]
 95%|█████████▌| 214036/225000 [415:27:13<28:23:05,  9.32s/it]
 95%|█████████▌| 214037/225000 [415:27:20<26:08:24,  8.58s/it]
 95%|█████████▌| 214038/225000 [415:27:31<28:59:23,  9.52s/it]
 95%|█████████▌| 214039/225000 [415:27:38<26:50:22,  8.82s/it]
 95%|█████████▌| 214040/225000 [415:27:45<24:55:40,  8.19s/it]
 95%|█████████▌| 214041/225000 [415:27:52<23:28:36,  7.71s/it]
 95%|█████████▌| 214042/225000 [415:27:58<21:44:21,  7.14s/it]
 95%|█████████▌| 214043/225000 [415:28:08<25:03:36,  8.23s/it]
 95%|█████████▌| 214044/225000 [415:28:13<22:10:58,  7.29s/it]
 95%|█████████▌| 214045/225000 [415:28:26<26:51:59,  8.83s/it]
 95%|█████████▌| 214046/225000 [415:28:32<24:16:04,  7.98s/it]
 95%|█████████▌| 214047/225000 [415:28:39<23:43:28,  7.80s/it]
 95%|█████████▌| 214048/225000 [415:28:47<23:40:56,  7.78s/it]
 95%|█████████▌| 214049/225000 [415:28:57<25:28:22,  8.37s/it]
 95%|█████████▌| 214050/225000 [415:29:06<26:25:43,  8.69s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.27', 'learning_rate': '3.252e-07', 'epoch': '0.9513', 'num_input_tokens_seen': 704887576, 'train_runtime': '1.496e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214050/225000 [415:29:06<26:25:43,  8.69s/it]
 95%|█████████▌| 214051/225000 [415:29:20<31:03:55, 10.21s/it]
 95%|█████████▌| 214052/225000 [415:29:30<31:03:41, 10.21s/it]
 95%|█████████▌| 214053/225000 [415:29:38<29:12:24,  9.60s/it]
 95%|█████████▌| 214054/225000 [415:29:51<31:59:58, 10.52s/it]
 95%|█████████▌| 214055/225000 [415:30:03<32:59:44, 10.85s/it]
 95%|█████████▌| 214056/225000 [415:30:07<27:01:44,  8.89s/it]
 95%|█████████▌| 214057/225000 [415:30:17<28:16:12,  9.30s/it]
 95%|█████████▌| 214058/225000 [415:30:29<30:20:33,  9.98s/it]
 95%|█████████▌| 214059/225000 [415:30:35<26:47:27,  8.82s/it]
 95%|█████████▌| 214060/225000 [415:30:41<24:00:22,  7.90s/it]
 95%|█████████▌| 214061/225000 [415:30:55<30:18:28,  9.97s/it]
 95%|█████████▌| 214062/225000 [415:31:00<25:33:57,  8.41s/it]
 95%|█████████▌| 214063/225000 [415:31:10<26:27:17,  8.71s/it]
 95%|█████████▌| 214064/225000 [415:31:20<28:03:47,  9.24s/it]
 95%|█████████▌| 214065/225000 [415:31:25<24:26:03,  8.04s/it]
 95%|█████████▌| 214066/225000 [415:31:31<22:13:35,  7.32s/it]
 95%|█████████▌| 214067/225000 [415:31:41<24:51:08,  8.18s/it]
 95%|█████████▌| 214068/225000 [415:31:49<24:36:21,  8.10s/it]
 95%|█████████▌| 214069/225000 [415:31:57<24:33:42,  8.09s/it]
 95%|█████████▌| 214070/225000 [415:32:11<29:31:25,  9.72s/it]
 95%|█████████▌| 214071/225000 [415:32:17<26:45:49,  8.82s/it]
 95%|█████████▌| 214072/225000 [415:32:23<23:31:09,  7.75s/it]
 95%|█████████▌| 214073/225000 [415:32:35<27:21:31,  9.01s/it]
 95%|█████████▌| 214074/225000 [415:32:41<24:59:14,  8.23s/it]
 95%|█████████▌| 214075/225000 [415:32:53<28:21:07,  9.34s/it]
 95%|█████████▌| 214076/225000 [415:33:00<26:13:30,  8.64s/it]
 95%|█████████▌| 214077/225000 [415:33:12<29:28:28,  9.71s/it]
 95%|█████████▌| 214078/225000 [415:33:17<25:26:37,  8.39s/it]
 95%|█████████▌| 214079/225000 [415:33:24<23:42:37,  7.82s/it]
 95%|█████████▌| 214080/225000 [415:33:32<24:21:37,  8.03s/it]
 95%|█████████▌| 214081/225000 [415:33:44<27:08:52,  8.95s/it]
 95%|█████████▌| 214082/225000 [415:33:49<24:04:05,  7.94s/it]
 95%|█████████▌| 214083/225000 [415:33:56<23:33:00,  7.77s/it]
 95%|█████████▌| 214084/225000 [415:34:03<22:24:35,  7.39s/it]
 95%|█████████▌| 214085/225000 [415:34:09<21:08:49,  6.97s/it]
 95%|█████████▌| 214086/225000 [415:34:22<26:19:49,  8.69s/it]
 95%|█████████▌| 214087/225000 [415:34:34<29:28:28,  9.72s/it]
 95%|█████████▌| 214088/225000 [415:34:41<27:15:45,  8.99s/it]
 95%|█████████▌| 214089/225000 [415:34:54<30:49:08, 10.17s/it]
 95%|█████████▌| 214090/225000 [415:35:03<29:24:28,  9.70s/it]
 95%|█████████▌| 214091/225000 [415:35:11<28:19:17,  9.35s/it]
 95%|█████████▌| 214092/225000 [415:35:22<29:54:43,  9.87s/it]
 95%|█████████▌| 214093/225000 [415:35:28<26:07:33,  8.62s/it]
 95%|█████████▌| 214094/225000 [415:35:40<29:07:43,  9.62s/it]
 95%|█████████▌| 214095/225000 [415:35:50<29:40:49,  9.80s/it]
 95%|█████████▌| 214096/225000 [415:35:57<26:47:08,  8.84s/it]
 95%|█████████▌| 214097/225000 [415:36:09<30:14:34,  9.99s/it]
 95%|█████████▌| 214098/225000 [415:36:14<25:42:55,  8.49s/it]
 95%|█████████▌| 214099/225000 [415:36:25<27:57:24,  9.23s/it]
 95%|█████████▌| 214100/225000 [415:36:33<26:53:44,  8.88s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.156', 'learning_rate': '3.222e-07', 'epoch': '0.9516', 'num_input_tokens_seen': 705116504, 'train_runtime': '1.496e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214100/225000 [415:36:33<26:53:44,  8.88s/it]
 95%|█████████▌| 214101/225000 [415:36:43<27:28:58,  9.08s/it]
 95%|█████████▌| 214102/225000 [415:36:54<28:49:27,  9.52s/it]
 95%|█████████▌| 214103/225000 [415:37:03<28:58:48,  9.57s/it]
 95%|█████████▌| 214104/225000 [415:37:09<25:07:07,  8.30s/it]
 95%|█████████▌| 214105/225000 [415:37:15<23:23:29,  7.73s/it]
 95%|█████████▌| 214106/225000 [415:37:27<27:05:21,  8.95s/it]
 95%|█████████▌| 214107/225000 [415:37:32<24:05:56,  7.96s/it]
 95%|█████████▌| 214108/225000 [415:37:46<29:01:13,  9.59s/it]
 95%|█████████▌| 214109/225000 [415:37:52<25:44:05,  8.51s/it]
 95%|█████████▌| 214110/225000 [415:37:59<24:17:12,  8.03s/it]
 95%|█████████▌| 214111/225000 [415:38:10<27:39:51,  9.15s/it]
 95%|█████████▌| 214112/225000 [415:38:16<24:50:13,  8.21s/it]
 95%|█████████▌| 214113/225000 [415:38:23<23:03:23,  7.62s/it]
 95%|█████████▌| 214114/225000 [415:38:31<23:36:57,  7.81s/it]
 95%|█████████▌| 214115/225000 [415:38:40<24:56:36,  8.25s/it]
 95%|█████████▌| 214116/225000 [415:38:56<31:48:34, 10.52s/it]
 95%|█████████▌| 214117/225000 [415:39:02<27:43:46,  9.17s/it]
 95%|█████████▌| 214118/225000 [415:39:10<26:27:53,  8.76s/it]
 95%|█████████▌| 214119/225000 [415:39:21<28:25:23,  9.40s/it]
 95%|█████████▌| 214120/225000 [415:39:28<26:26:52,  8.75s/it]
 95%|█████████▌| 214121/225000 [415:39:42<31:03:38, 10.28s/it]
 95%|█████████▌| 214122/225000 [415:39:54<32:59:11, 10.92s/it]
 95%|█████████▌| 214123/225000 [415:40:03<31:21:00, 10.38s/it]
 95%|█████████▌| 214124/225000 [415:40:12<30:01:15,  9.94s/it]
 95%|█████████▌| 214125/225000 [415:40:25<32:42:33, 10.83s/it]
 95%|█████████▌| 214126/225000 [415:40:31<28:03:25,  9.29s/it]
 95%|█████████▌| 214127/225000 [415:40:41<28:51:42,  9.56s/it]
 95%|█████████▌| 214128/225000 [415:40:50<28:13:37,  9.35s/it]
 95%|█████████▌| 214129/225000 [415:41:03<31:26:24, 10.41s/it]
 95%|█████████▌| 214130/225000 [415:41:08<26:35:18,  8.81s/it]
 95%|█████████▌| 214131/225000 [415:41:14<23:52:04,  7.91s/it]
 95%|█████████▌| 214132/225000 [415:41:21<23:30:52,  7.79s/it]
 95%|█████████▌| 214133/225000 [415:41:32<26:35:40,  8.81s/it]
 95%|█████████▌| 214134/225000 [415:41:37<22:23:20,  7.42s/it]
 95%|█████████▌| 214135/225000 [415:41:47<24:47:52,  8.22s/it]
 95%|█████████▌| 214136/225000 [415:41:55<24:54:21,  8.25s/it]
 95%|█████████▌| 214137/225000 [415:42:00<22:09:29,  7.34s/it]
 95%|█████████▌| 214138/225000 [415:42:12<26:27:39,  8.77s/it]
 95%|█████████▌| 214139/225000 [415:42:17<22:21:29,  7.41s/it]
 95%|█████████▌| 214140/225000 [415:42:29<27:06:50,  8.99s/it]
 95%|█████████▌| 214141/225000 [415:42:39<27:46:47,  9.21s/it]
 95%|█████████▌| 214142/225000 [415:42:47<26:47:05,  8.88s/it]
 95%|█████████▌| 214143/225000 [415:42:56<26:40:04,  8.84s/it]
 95%|█████████▌| 214144/225000 [415:43:03<25:12:04,  8.36s/it]
 95%|█████████▌| 214145/225000 [415:43:11<24:59:11,  8.29s/it]
 95%|█████████▌| 214146/225000 [415:43:20<25:22:14,  8.41s/it]
 95%|█████████▌| 214147/225000 [415:43:35<31:11:42, 10.35s/it]
 95%|█████████▌| 214148/225000 [415:43:41<27:45:54,  9.21s/it]
 95%|█████████▌| 214149/225000 [415:43:47<24:54:30,  8.26s/it]
 95%|█████████▌| 214150/225000 [415:43:58<26:38:29,  8.84s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.17', 'learning_rate': '3.193e-07', 'epoch': '0.9518', 'num_input_tokens_seen': 705348712, 'train_runtime': '1.497e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214150/225000 [415:43:58<26:38:29,  8.84s/it]
 95%|█████████▌| 214151/225000 [415:44:06<26:08:36,  8.68s/it]
 95%|█████████▌| 214152/225000 [415:44:12<23:43:58,  7.88s/it]
 95%|█████████▌| 214153/225000 [415:44:20<23:58:08,  7.96s/it]
 95%|█████████▌| 214154/225000 [415:44:34<29:53:57,  9.92s/it]
 95%|█████████▌| 214155/225000 [415:44:42<27:31:44,  9.14s/it]
 95%|█████████▌| 214156/225000 [415:44:50<26:29:50,  8.80s/it]
 95%|█████████▌| 214157/225000 [415:44:56<24:23:01,  8.10s/it]
 95%|█████████▌| 214158/225000 [415:45:05<25:21:08,  8.42s/it]
 95%|█████████▌| 214159/225000 [415:45:13<24:21:01,  8.09s/it]
 95%|█████████▌| 214160/225000 [415:45:20<23:14:36,  7.72s/it]
 95%|█████████▌| 214161/225000 [415:45:29<24:45:01,  8.22s/it]
 95%|█████████▌| 214162/225000 [415:45:38<25:34:06,  8.49s/it]
 95%|█████████▌| 214163/225000 [415:45:48<26:43:43,  8.88s/it]
 95%|█████████▌| 214164/225000 [415:45:56<26:01:55,  8.65s/it]
 95%|█████████▌| 214165/225000 [415:46:04<25:24:07,  8.44s/it]
 95%|█████████▌| 214166/225000 [415:46:12<25:09:54,  8.36s/it]
 95%|█████████▌| 214167/225000 [415:46:20<25:01:22,  8.32s/it]
 95%|█████████▌| 214168/225000 [415:46:30<26:25:33,  8.78s/it]
 95%|█████████▌| 214169/225000 [415:46:40<27:20:29,  9.09s/it]
 95%|█████████▌| 214170/225000 [415:46:47<25:24:06,  8.44s/it]
 95%|█████████▌| 214171/225000 [415:46:56<25:48:29,  8.58s/it]
 95%|█████████▌| 214172/225000 [415:47:03<24:04:12,  8.00s/it]
 95%|█████████▌| 214173/225000 [415:47:11<24:41:51,  8.21s/it]
 95%|█████████▌| 214174/225000 [415:47:18<23:01:41,  7.66s/it]
 95%|█████████▌| 214175/225000 [415:47:29<26:24:26,  8.78s/it]
 95%|█████████▌| 214176/225000 [415:47:38<26:55:11,  8.95s/it]
 95%|█████████▌| 214177/225000 [415:47:44<24:18:54,  8.09s/it]
 95%|█████████▌| 214178/225000 [415:47:54<25:31:03,  8.49s/it]
 95%|█████████▌| 214179/225000 [415:48:04<27:20:39,  9.10s/it]
 95%|█████████▌| 214180/225000 [415:48:11<24:58:10,  8.31s/it]
 95%|█████████▌| 214181/225000 [415:48:18<24:18:34,  8.09s/it]
 95%|█████████▌| 214182/225000 [415:48:27<24:24:55,  8.12s/it]
 95%|█████████▌| 214183/225000 [415:48:34<23:21:40,  7.77s/it]
 95%|█████████▌| 214184/225000 [415:48:48<29:33:26,  9.84s/it]
 95%|█████████▌| 214185/225000 [415:48:55<26:51:35,  8.94s/it]
 95%|█████████▌| 214186/225000 [415:49:09<31:26:48, 10.47s/it]
 95%|█████████▌| 214187/225000 [415:49:19<30:45:03, 10.24s/it]
 95%|█████████▌| 214188/225000 [415:49:24<26:39:20,  8.88s/it]
 95%|█████████▌| 214189/225000 [415:49:36<28:51:52,  9.61s/it]
 95%|█████████▌| 214190/225000 [415:49:42<25:46:40,  8.58s/it]
 95%|█████████▌| 214191/225000 [415:49:50<24:54:28,  8.30s/it]
 95%|█████████▌| 214192/225000 [415:49:57<23:50:45,  7.94s/it]
 95%|█████████▌| 214193/225000 [415:50:04<23:30:46,  7.83s/it]
 95%|█████████▌| 214194/225000 [415:50:20<30:33:39, 10.18s/it]
 95%|█████████▌| 214195/225000 [415:50:31<30:57:10, 10.31s/it]
 95%|█████████▌| 214196/225000 [415:50:38<28:42:01,  9.56s/it]
 95%|█████████▌| 214197/225000 [415:50:44<25:25:16,  8.47s/it]
 95%|█████████▌| 214198/225000 [415:50:53<25:40:41,  8.56s/it]
 95%|█████████▌| 214199/225000 [415:50:59<23:02:15,  7.68s/it]
 95%|█████████▌| 214200/225000 [415:51:12<27:41:18,  9.23s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.128', 'learning_rate': '3.164e-07', 'epoch': '0.952', 'num_input_tokens_seen': 705575576, 'train_runtime': '1.497e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214200/225000 [415:51:12<27:41:18,  9.23s/it]
 95%|█████████▌| 214201/225000 [415:51:19<25:52:49,  8.63s/it]
 95%|█████████▌| 214202/225000 [415:51:28<26:05:05,  8.70s/it]
 95%|█████████▌| 214203/225000 [415:51:37<26:33:34,  8.86s/it]
 95%|█████████▌| 214204/225000 [415:51:43<24:29:55,  8.17s/it]
 95%|█████████▌| 214205/225000 [415:51:51<23:37:31,  7.88s/it]
 95%|█████████▌| 214206/225000 [415:52:00<24:46:42,  8.26s/it]
 95%|█████████▌| 214207/225000 [415:52:09<25:25:57,  8.48s/it]
 95%|█████████▌| 214208/225000 [415:52:19<27:03:05,  9.02s/it]
 95%|█████████▌| 214209/225000 [415:52:26<24:44:15,  8.25s/it]
 95%|█████████▌| 214210/225000 [415:52:37<27:45:21,  9.26s/it]
 95%|█████████▌| 214211/225000 [415:52:42<24:11:47,  8.07s/it]
 95%|█████████▌| 214212/225000 [415:52:51<25:03:45,  8.36s/it]
 95%|█████████▌| 214213/225000 [415:53:04<29:10:08,  9.73s/it]
 95%|█████████▌| 214214/225000 [415:53:12<26:56:53,  8.99s/it]
 95%|█████████▌| 214215/225000 [415:53:20<26:36:38,  8.88s/it]
 95%|█████████▌| 214216/225000 [415:53:27<24:14:06,  8.09s/it]
 95%|█████████▌| 214217/225000 [415:53:35<24:45:46,  8.27s/it]
 95%|█████████▌| 214218/225000 [415:53:43<23:56:29,  7.99s/it]
 95%|█████████▌| 214219/225000 [415:53:49<22:52:41,  7.64s/it]
 95%|█████████▌| 214220/225000 [415:54:00<25:37:49,  8.56s/it]
 95%|█████████▌| 214221/225000 [415:54:11<27:27:38,  9.17s/it]
 95%|█████████▌| 214222/225000 [415:54:23<29:51:16,  9.97s/it]
 95%|█████████▌| 214223/225000 [415:54:28<25:54:03,  8.65s/it]
 95%|█████████▌| 214224/225000 [415:54:36<25:37:29,  8.56s/it]
 95%|█████████▌| 214225/225000 [415:54:44<24:28:26,  8.18s/it]
 95%|█████████▌| 214226/225000 [415:54:50<22:44:07,  7.60s/it]
 95%|█████████▌| 214227/225000 [415:54:58<22:44:15,  7.60s/it]
 95%|█████████▌| 214228/225000 [415:55:10<27:17:45,  9.12s/it]
 95%|█████████▌| 214229/225000 [415:55:16<24:32:55,  8.20s/it]
 95%|█████████▌| 214230/225000 [415:55:23<23:26:27,  7.84s/it]
 95%|█████████▌| 214231/225000 [415:55:33<25:21:54,  8.48s/it]
 95%|█████████▌| 214232/225000 [415:55:47<30:15:48, 10.12s/it]
 95%|█████████▌| 214233/225000 [415:55:57<30:05:36, 10.06s/it]
 95%|█████████▌| 214234/225000 [415:56:04<27:08:44,  9.08s/it]
 95%|█████████▌| 214235/225000 [415:56:12<25:49:55,  8.64s/it]
 95%|█████████▌| 214236/225000 [415:56:19<24:45:45,  8.28s/it]
 95%|█████████▌| 214237/225000 [415:56:28<25:41:31,  8.59s/it]
 95%|█████████▌| 214238/225000 [415:56:36<24:59:20,  8.36s/it]
 95%|█████████▌| 214239/225000 [415:56:42<22:37:56,  7.57s/it]
 95%|█████████▌| 214240/225000 [415:56:48<21:09:19,  7.08s/it]
 95%|█████████▌| 214241/225000 [415:57:02<27:52:57,  9.33s/it]
 95%|█████████▌| 214242/225000 [415:57:08<24:44:12,  8.28s/it]
 95%|█████████▌| 214243/225000 [415:57:22<29:39:13,  9.92s/it]
 95%|█████████▌| 214244/225000 [415:57:28<26:11:17,  8.77s/it]
 95%|█████████▌| 214245/225000 [415:57:36<25:53:51,  8.67s/it]
 95%|█████████▌| 214246/225000 [415:57:48<28:46:24,  9.63s/it]
 95%|█████████▌| 214247/225000 [415:57:56<26:43:10,  8.95s/it]
 95%|█████████▌| 214248/225000 [415:58:05<26:37:59,  8.92s/it]
 95%|█████████▌| 214249/225000 [415:58:11<24:16:30,  8.13s/it]
 95%|█████████▌| 214250/225000 [415:58:24<28:52:00,  9.67s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.058', 'learning_rate': '3.135e-07', 'epoch': '0.9522', 'num_input_tokens_seen': 705804120, 'train_runtime': '1.498e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214250/225000 [415:58:24<28:52:00,  9.67s/it]
 95%|█████████▌| 214251/225000 [415:58:35<29:32:21,  9.89s/it]
 95%|█████████▌| 214252/225000 [415:58:43<28:34:33,  9.57s/it]
 95%|█████████▌| 214253/225000 [415:58:56<31:06:33, 10.42s/it]
 95%|█████████▌| 214254/225000 [415:59:04<29:02:55,  9.73s/it]
 95%|█████████▌| 214255/225000 [415:59:16<30:57:02, 10.37s/it]
 95%|█████████▌| 214256/225000 [415:59:27<31:18:35, 10.49s/it]
 95%|█████████▌| 214257/225000 [415:59:33<27:53:39,  9.35s/it]
 95%|█████████▌| 214258/225000 [415:59:39<24:44:02,  8.29s/it]
 95%|█████████▌| 214259/225000 [415:59:45<22:17:36,  7.47s/it]
 95%|█████████▌| 214260/225000 [415:59:52<22:07:17,  7.42s/it]
 95%|█████████▌| 214261/225000 [416:00:05<26:54:57,  9.02s/it]
 95%|█████████▌| 214262/225000 [416:00:13<26:28:21,  8.88s/it]
 95%|█████████▌| 214263/225000 [416:00:19<23:31:36,  7.89s/it]
 95%|█████████▌| 214264/225000 [416:00:24<20:58:58,  7.04s/it]
 95%|█████████▌| 214265/225000 [416:00:31<21:05:05,  7.07s/it]
 95%|█████████▌| 214266/225000 [416:00:44<26:09:07,  8.77s/it]
 95%|█████████▌| 214267/225000 [416:00:53<26:54:30,  9.03s/it]
 95%|█████████▌| 214268/225000 [416:01:00<24:48:31,  8.32s/it]
 95%|█████████▌| 214269/225000 [416:01:13<28:40:51,  9.62s/it]
 95%|█████████▌| 214270/225000 [416:01:21<27:44:21,  9.31s/it]
 95%|█████████▌| 214271/225000 [416:01:32<29:10:25,  9.79s/it]
 95%|█████████▌| 214272/225000 [416:01:37<24:52:30,  8.35s/it]
 95%|█████████▌| 214273/225000 [416:01:47<26:00:58,  8.73s/it]
 95%|█████████▌| 214274/225000 [416:01:56<26:23:15,  8.86s/it]
 95%|█████████▌| 214275/225000 [416:02:02<23:49:43,  8.00s/it]
 95%|█████████▌| 214276/225000 [416:02:12<25:26:32,  8.54s/it]
 95%|█████████▌| 214277/225000 [416:02:19<24:09:03,  8.11s/it]
 95%|█████████▌| 214278/225000 [416:02:23<20:44:44,  6.97s/it]
 95%|█████████▌| 214279/225000 [416:02:31<21:10:48,  7.11s/it]
 95%|█████████▌| 214280/225000 [416:02:36<19:55:02,  6.69s/it]
 95%|█████████▌| 214281/225000 [416:02:47<23:49:43,  8.00s/it]
 95%|█████████▌| 214282/225000 [416:02:56<24:36:06,  8.26s/it]
 95%|█████████▌| 214283/225000 [416:03:07<27:12:44,  9.14s/it]
 95%|█████████▌| 214284/225000 [416:03:16<26:23:15,  8.86s/it]
 95%|█████████▌| 214285/225000 [416:03:21<23:04:06,  7.75s/it]
 95%|█████████▌| 214286/225000 [416:03:28<22:47:48,  7.66s/it]
 95%|█████████▌| 214287/225000 [416:03:38<25:05:41,  8.43s/it]
 95%|█████████▌| 214288/225000 [416:03:46<24:24:24,  8.20s/it]
 95%|█████████▌| 214289/225000 [416:03:57<27:08:02,  9.12s/it]
 95%|█████████▌| 214290/225000 [416:04:04<25:02:35,  8.42s/it]
 95%|█████████▌| 214291/225000 [416:04:13<25:37:02,  8.61s/it]
 95%|█████████▌| 214292/225000 [416:04:19<23:13:43,  7.81s/it]
 95%|█████████▌| 214293/225000 [416:04:31<26:41:01,  8.97s/it]
 95%|█████████▌| 214294/225000 [416:04:41<27:58:51,  9.41s/it]
 95%|█████████▌| 214295/225000 [416:04:46<23:48:40,  8.01s/it]
 95%|█████████▌| 214296/225000 [416:05:00<29:02:53,  9.77s/it]
 95%|█████████▌| 214297/225000 [416:05:08<27:36:04,  9.28s/it]
 95%|█████████▌| 214298/225000 [416:05:17<27:06:14,  9.12s/it]
 95%|█████████▌| 214299/225000 [416:05:28<28:47:29,  9.69s/it]
 95%|█████████▌| 214300/225000 [416:05:36<27:33:54,  9.27s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.404', 'learning_rate': '3.107e-07', 'epoch': '0.9524', 'num_input_tokens_seen': 706030048, 'train_runtime': '1.498e+06', 'train_tokens_per_second': '471.3'}

 95%|█████████▌| 214300/225000 [416:05:36<27:33:54,  9.27s/it]
 95%|█████████▌| 214301/225000 [416:05:45<26:56:52,  9.07s/it]
 95%|█████████▌| 214302/225000 [416:05:54<26:58:37,  9.08s/it]
 95%|█████████▌| 214303/225000 [416:06:02<26:40:32,  8.98s/it]
 95%|█████████▌| 214304/225000 [416:06:12<27:07:19,  9.13s/it]
 95%|█████████▌| 214305/225000 [416:06:20<26:03:12,  8.77s/it]
 95%|█████████▌| 214306/225000 [416:06:26<23:31:12,  7.92s/it]
 95%|█████████▌| 214307/225000 [416:06:36<25:07:31,  8.46s/it]
 95%|█████████▌| 214308/225000 [416:06:45<25:46:13,  8.68s/it]
 95%|█████████▌| 214309/225000 [416:06:59<31:00:44, 10.44s/it]
 95%|█████████▌| 214310/225000 [416:07:09<30:34:19, 10.30s/it]
 95%|█████████▌| 214311/225000 [416:07:14<25:36:14,  8.62s/it]
 95%|█████████▌| 214312/225000 [416:07:26<28:38:25,  9.65s/it]
 95%|█████████▌| 214313/225000 [416:07:34<26:52:11,  9.05s/it]
 95%|█████████▌| 214314/225000 [416:07:43<27:12:12,  9.16s/it]
 95%|█████████▌| 214315/225000 [416:07:57<31:19:59, 10.56s/it]
 95%|█████████▌| 214316/225000 [416:08:05<29:31:33,  9.95s/it]
 95%|█████████▌| 214317/225000 [416:08:15<29:11:19,  9.84s/it]
 95%|█████████▌| 214318/225000 [416:08:24<28:35:38,  9.64s/it]
 95%|█████████▌| 214319/225000 [416:08:32<26:53:49,  9.07s/it]
 95%|█████████▌| 214320/225000 [416:08:39<25:14:27,  8.51s/it]
 95%|█████████▌| 214321/225000 [416:08:50<27:38:55,  9.32s/it]
 95%|█████████▌| 214322/225000 [416:09:02<29:52:03, 10.07s/it]
 95%|█████████▌| 214323/225000 [416:09:10<27:50:09,  9.39s/it]
 95%|█████████▌| 214324/225000 [416:09:24<31:35:53, 10.66s/it]
 95%|█████████▌| 214325/225000 [416:09:32<29:24:37,  9.92s/it]
 95%|█████████▌| 214326/225000 [416:09:47<33:46:26, 11.39s/it]
 95%|█████████▌| 214327/225000 [416:09:53<29:44:00, 10.03s/it]
 95%|█████████▌| 214328/225000 [416:10:01<27:53:34,  9.41s/it]
 95%|█████████▌| 214329/225000 [416:10:06<23:52:13,  8.05s/it]
 95%|█████████▌| 214330/225000 [416:10:17<26:37:16,  8.98s/it]
 95%|█████████▌| 214331/225000 [416:10:24<24:03:06,  8.12s/it]
 95%|█████████▌| 214332/225000 [416:10:35<26:35:50,  8.98s/it]
 95%|█████████▌| 214333/225000 [416:10:43<25:50:27,  8.72s/it]
 95%|█████████▌| 214334/225000 [416:10:54<27:54:23,  9.42s/it]
 95%|█████████▌| 214335/225000 [416:11:04<28:58:42,  9.78s/it]
 95%|█████████▌| 214336/225000 [416:11:13<28:05:07,  9.48s/it]
 95%|█████████▌| 214337/225000 [416:11:25<30:03:33, 10.15s/it]
 95%|█████████▌| 214338/225000 [416:11:33<28:13:26,  9.53s/it]
 95%|█████████▌| 214339/225000 [416:11:39<24:59:48,  8.44s/it]
 95%|█████████▌| 214340/225000 [416:11:47<24:45:29,  8.36s/it]
 95%|█████████▌| 214341/225000 [416:11:54<23:12:27,  7.84s/it]
 95%|█████████▌| 214342/225000 [416:12:00<21:51:09,  7.38s/it]
 95%|█████████▌| 214343/225000 [416:12:13<27:16:28,  9.21s/it]
 95%|█████████▌| 214344/225000 [416:12:18<23:07:30,  7.81s/it]
 95%|█████████▌| 214345/225000 [416:12:27<24:13:31,  8.19s/it]
 95%|█████████▌| 214346/225000 [416:12:36<25:17:58,  8.55s/it]
 95%|█████████▌| 214347/225000 [416:12:44<24:07:40,  8.15s/it]
 95%|█████████▌| 214348/225000 [416:12:59<30:17:19, 10.24s/it]
 95%|█████████▌| 214349/225000 [416:13:04<25:31:53,  8.63s/it]
 95%|█████████▌| 214350/225000 [416:13:10<23:52:45,  8.07s/it]
                                                              
{'loss': '1.323', 'grad_norm': '1.242', 'learning_rate': '3.078e-07', 'epoch': '0.9527', 'num_input_tokens_seen': 706269064, 'train_runtime': '1.498e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214350/225000 [416:13:10<23:52:45,  8.07s/it]
 95%|█████████▌| 214351/225000 [416:13:21<26:01:27,  8.80s/it]
 95%|█████████▌| 214352/225000 [416:13:30<26:26:09,  8.94s/it]
 95%|█████████▌| 214353/225000 [416:13:41<28:23:16,  9.60s/it]
 95%|█████████▌| 214354/225000 [416:13:52<29:21:26,  9.93s/it]
 95%|█████████▌| 214355/225000 [416:13:57<25:17:32,  8.55s/it]
 95%|█████████▌| 214356/225000 [416:14:05<24:30:19,  8.29s/it]
 95%|█████████▌| 214357/225000 [416:14:11<22:12:04,  7.51s/it]
 95%|█████████▌| 214358/225000 [416:14:20<23:52:20,  8.08s/it]
 95%|█████████▌| 214359/225000 [416:14:28<23:27:15,  7.93s/it]
 95%|█████████▌| 214360/225000 [416:14:35<23:07:43,  7.83s/it]
 95%|█████████▌| 214361/225000 [416:14:47<26:19:49,  8.91s/it]
 95%|█████████▌| 214362/225000 [416:14:52<22:45:55,  7.70s/it]
 95%|█████████▌| 214363/225000 [416:14:59<22:54:17,  7.75s/it]
 95%|█████████▌| 214364/225000 [416:15:12<27:03:00,  9.16s/it]
 95%|█████████▌| 214365/225000 [416:15:24<29:58:34, 10.15s/it]
 95%|█████████▌| 214366/225000 [416:15:33<29:05:06,  9.85s/it]
 95%|█████████▌| 214367/225000 [416:15:41<27:26:25,  9.29s/it]
 95%|█████████▌| 214368/225000 [416:15:53<29:21:23,  9.94s/it]
 95%|█████████▌| 214369/225000 [416:16:00<27:11:53,  9.21s/it]
 95%|█████████▌| 214370/225000 [416:16:07<25:07:10,  8.51s/it]
 95%|█████████▌| 214371/225000 [416:16:19<28:10:46,  9.54s/it]
 95%|█████████▌| 214372/225000 [416:16:28<27:45:25,  9.40s/it]
 95%|█████████▌| 214373/225000 [416:16:40<29:50:51, 10.11s/it]
 95%|█████████▌| 214374/225000 [416:16:46<26:14:11,  8.89s/it]
 95%|█████████▌| 214375/225000 [416:16:58<28:50:20,  9.77s/it]
 95%|█████████▌| 214376/225000 [416:17:04<25:14:52,  8.56s/it]
 95%|█████████▌| 214377/225000 [416:17:16<28:17:30,  9.59s/it]
 95%|█████████▌| 214378/225000 [416:17:25<28:04:33,  9.52s/it]
 95%|█████████▌| 214379/225000 [416:17:34<27:17:36,  9.25s/it]
 95%|█████████▌| 214380/225000 [416:17:39<23:27:13,  7.95s/it]
 95%|█████████▌| 214381/225000 [416:17:54<30:27:13, 10.32s/it]
 95%|█████████▌| 214382/225000 [416:18:06<31:26:28, 10.66s/it]
 95%|█████████▌| 214383/225000 [416:18:11<26:12:25,  8.89s/it]
 95%|█████████▌| 214384/225000 [416:18:22<28:33:48,  9.69s/it]
 95%|█████████▌| 214385/225000 [416:18:29<26:07:01,  8.86s/it]
 95%|█████████▌| 214386/225000 [416:18:40<28:13:32,  9.57s/it]
 95%|█████████▌| 214387/225000 [416:18:45<24:03:48,  8.16s/it]
 95%|█████████▌| 214388/225000 [416:18:58<27:45:29,  9.42s/it]
 95%|█████████▌| 214389/225000 [416:19:08<28:44:30,  9.75s/it]
 95%|█████████▌| 214390/225000 [416:19:17<27:45:47,  9.42s/it]
 95%|█████████▌| 214391/225000 [416:19:25<26:56:07,  9.14s/it]
 95%|█████████▌| 214392/225000 [416:19:38<29:56:03, 10.16s/it]
 95%|█████████▌| 214393/225000 [416:19:46<28:01:47,  9.51s/it]
 95%|█████████▌| 214394/225000 [416:19:59<31:29:41, 10.69s/it]
 95%|█████████▌| 214395/225000 [416:20:04<26:00:12,  8.83s/it]
 95%|█████████▌| 214396/225000 [416:20:10<23:21:58,  7.93s/it]
 95%|█████████▌| 214397/225000 [416:20:16<22:29:48,  7.64s/it]
 95%|█████████▌| 214398/225000 [416:20:24<21:59:22,  7.47s/it]
 95%|█████████▌| 214399/225000 [416:20:39<28:40:19,  9.74s/it]
 95%|█████████▌| 214400/225000 [416:20:49<28:57:39,  9.84s/it]
                                                              
{'loss': '1.296', 'grad_norm': '1.27', 'learning_rate': '3.05e-07', 'epoch': '0.9529', 'num_input_tokens_seen': 706507264, 'train_runtime': '1.499e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214400/225000 [416:20:49<28:57:39,  9.84s/it]
 95%|█████████▌| 214401/225000 [416:20:54<25:16:51,  8.59s/it]
 95%|█████████▌| 214402/225000 [416:21:05<27:04:02,  9.19s/it]
 95%|█████████▌| 214403/225000 [416:21:13<26:07:24,  8.87s/it]
 95%|█████████▌| 214404/225000 [416:21:22<25:47:16,  8.76s/it]
 95%|█████████▌| 214405/225000 [416:21:27<23:13:38,  7.89s/it]
 95%|█████████▌| 214406/225000 [416:21:36<23:28:08,  7.98s/it]
 95%|█████████▌| 214407/225000 [416:21:44<23:26:36,  7.97s/it]
 95%|█████████▌| 214408/225000 [416:21:50<21:48:19,  7.41s/it]
 95%|█████████▌| 214409/225000 [416:21:59<23:36:26,  8.02s/it]
 95%|█████████▌| 214410/225000 [416:22:08<24:09:09,  8.21s/it]
 95%|█████████▌| 214411/225000 [416:22:15<23:08:18,  7.87s/it]
 95%|█████████▌| 214412/225000 [416:22:22<22:56:14,  7.80s/it]
 95%|█████████▌| 214413/225000 [416:22:34<25:57:22,  8.83s/it]
 95%|█████████▌| 214414/225000 [416:22:43<26:21:40,  8.96s/it]
 95%|█████████▌| 214415/225000 [416:22:49<23:29:26,  7.99s/it]
 95%|█████████▌| 214416/225000 [416:22:58<24:14:33,  8.25s/it]
 95%|█████████▌| 214417/225000 [416:23:07<25:43:18,  8.75s/it]
 95%|█████████▌| 214418/225000 [416:23:16<25:35:35,  8.71s/it]
 95%|█████████▌| 214419/225000 [416:23:23<23:51:59,  8.12s/it]
 95%|█████████▌| 214420/225000 [416:23:32<24:57:43,  8.49s/it]
 95%|█████████▌| 214421/225000 [416:23:44<27:49:30,  9.47s/it]
 95%|█████████▌| 214422/225000 [416:23:54<28:47:14,  9.80s/it]
 95%|█████████▌| 214423/225000 [416:24:00<24:58:26,  8.50s/it]
 95%|█████████▌| 214424/225000 [416:24:09<25:38:35,  8.73s/it]
 95%|█████████▌| 214425/225000 [416:24:18<25:45:02,  8.77s/it]
 95%|█████████▌| 214426/225000 [416:24:24<23:18:32,  7.94s/it]
 95%|█████████▌| 214427/225000 [416:24:35<25:54:48,  8.82s/it]
 95%|█████████▌| 214428/225000 [416:24:44<26:24:27,  8.99s/it]
 95%|█████████▌| 214429/225000 [416:24:55<28:13:00,  9.61s/it]
 95%|█████████▌| 214430/225000 [416:25:06<29:30:23, 10.05s/it]
 95%|█████████▌| 214431/225000 [416:25:16<29:26:49, 10.03s/it]
 95%|█████████▌| 214432/225000 [416:25:21<24:45:52,  8.44s/it]
 95%|█████████▌| 214433/225000 [416:25:31<26:02:52,  8.87s/it]
 95%|█████████▌| 214434/225000 [416:25:42<27:57:28,  9.53s/it]
 95%|█████████▌| 214435/225000 [416:25:51<27:07:06,  9.24s/it]
 95%|█████████▌| 214436/225000 [416:26:00<27:09:12,  9.25s/it]
 95%|█████████▌| 214437/225000 [416:26:09<26:43:55,  9.11s/it]
 95%|█████████▌| 214438/225000 [416:26:16<25:03:55,  8.54s/it]
 95%|█████████▌| 214439/225000 [416:26:26<26:05:58,  8.90s/it]
 95%|█████████▌| 214440/225000 [416:26:31<22:54:17,  7.81s/it]
 95%|█████████▌| 214441/225000 [416:26:39<23:29:25,  8.01s/it]
 95%|█████████▌| 214442/225000 [416:26:49<24:56:13,  8.50s/it]
 95%|█████████▌| 214443/225000 [416:27:01<27:59:40,  9.55s/it]
 95%|█████████▌| 214444/225000 [416:27:08<25:58:10,  8.86s/it]
 95%|█████████▌| 214445/225000 [416:27:18<26:23:25,  9.00s/it]
 95%|█████████▌| 214446/225000 [416:27:27<26:54:25,  9.18s/it]
 95%|█████████▌| 214447/225000 [416:27:34<25:05:47,  8.56s/it]
 95%|█████████▌| 214448/225000 [416:27:39<22:02:11,  7.52s/it]
 95%|█████████▌| 214449/225000 [416:27:50<24:34:36,  8.39s/it]
 95%|█████████▌| 214450/225000 [416:27:55<21:54:18,  7.47s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.578', 'learning_rate': '3.022e-07', 'epoch': '0.9531', 'num_input_tokens_seen': 706735072, 'train_runtime': '1.499e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214450/225000 [416:27:55<21:54:18,  7.47s/it]
 95%|█████████▌| 214451/225000 [416:28:09<27:14:14,  9.30s/it]
 95%|█████████▌| 214452/225000 [416:28:17<26:24:39,  9.01s/it]
 95%|█████████▌| 214453/225000 [416:28:27<27:26:07,  9.36s/it]
 95%|█████████▌| 214454/225000 [416:28:34<25:22:27,  8.66s/it]
 95%|█████████▌| 214455/225000 [416:28:49<30:39:57, 10.47s/it]
 95%|█████████▌| 214456/225000 [416:28:57<28:42:48,  9.80s/it]
 95%|█████████▌| 214457/225000 [416:29:11<32:16:32, 11.02s/it]
 95%|█████████▌| 214458/225000 [416:29:17<28:09:29,  9.62s/it]
 95%|█████████▌| 214459/225000 [416:29:23<24:55:09,  8.51s/it]
 95%|█████████▌| 214460/225000 [416:29:29<22:41:41,  7.75s/it]
 95%|█████████▌| 214461/225000 [416:29:43<27:40:18,  9.45s/it]
 95%|█████████▌| 214462/225000 [416:29:51<26:38:36,  9.10s/it]
 95%|█████████▌| 214463/225000 [416:29:58<25:00:48,  8.55s/it]
 95%|█████████▌| 214464/225000 [416:30:07<25:16:12,  8.63s/it]
 95%|█████████▌| 214465/225000 [416:30:13<22:56:21,  7.84s/it]
 95%|█████████▌| 214466/225000 [416:30:25<26:03:53,  8.91s/it]
 95%|█████████▌| 214467/225000 [416:30:30<22:55:32,  7.84s/it]
 95%|█████████▌| 214468/225000 [416:30:40<25:20:22,  8.66s/it]
 95%|█████████▌| 214469/225000 [416:30:48<24:29:29,  8.37s/it]
 95%|█████████▌| 214470/225000 [416:30:56<23:53:07,  8.17s/it]
 95%|█████████▌| 214471/225000 [416:31:07<26:20:47,  9.01s/it]
 95%|█████████▌| 214472/225000 [416:31:14<24:53:26,  8.51s/it]
 95%|█████████▌| 214473/225000 [416:31:19<21:49:25,  7.46s/it]
 95%|█████████▌| 214474/225000 [416:31:28<23:18:26,  7.97s/it]
 95%|█████████▌| 214475/225000 [416:31:36<23:06:00,  7.90s/it]
 95%|█████████▌| 214476/225000 [416:31:45<24:22:12,  8.34s/it]
 95%|█████████▌| 214477/225000 [416:31:51<22:09:50,  7.58s/it]
 95%|█████████▌| 214478/225000 [416:31:58<21:00:22,  7.19s/it]
 95%|█████████▌| 214479/225000 [416:32:12<27:26:38,  9.39s/it]
 95%|█████████▌| 214480/225000 [416:32:18<24:41:45,  8.45s/it]
 95%|█████████▌| 214481/225000 [416:32:26<23:35:32,  8.07s/it]
 95%|█████████▌| 214482/225000 [416:32:36<26:07:20,  8.94s/it]
 95%|█████████▌| 214483/225000 [416:32:44<24:55:28,  8.53s/it]
 95%|█████████▌| 214484/225000 [416:32:49<21:34:46,  7.39s/it]
 95%|█████████▌| 214485/225000 [416:32:58<22:44:52,  7.79s/it]
 95%|█████████▌| 214486/225000 [416:33:12<28:27:49,  9.75s/it]
 95%|█████████▌| 214487/225000 [416:33:19<26:14:56,  8.99s/it]
 95%|█████████▌| 214488/225000 [416:33:27<25:18:39,  8.67s/it]
 95%|█████████▌| 214489/225000 [416:33:42<31:08:07, 10.66s/it]
 95%|█████████▌| 214490/225000 [416:33:48<26:27:40,  9.06s/it]
 95%|█████████▌| 214491/225000 [416:33:57<26:49:53,  9.19s/it]
 95%|█████████▌| 214492/225000 [416:34:05<26:01:52,  8.92s/it]
 95%|█████████▌| 214493/225000 [416:34:15<26:40:14,  9.14s/it]
 95%|█████████▌| 214494/225000 [416:34:22<24:34:27,  8.42s/it]
 95%|█████████▌| 214495/225000 [416:34:31<25:16:38,  8.66s/it]
 95%|█████████▌| 214496/225000 [416:34:39<24:22:36,  8.35s/it]
 95%|█████████▌| 214497/225000 [416:34:44<21:53:02,  7.50s/it]
 95%|█████████▌| 214498/225000 [416:34:51<21:24:40,  7.34s/it]
 95%|█████████▌| 214499/225000 [416:35:02<24:48:39,  8.51s/it]
 95%|█████████▌| 214500/225000 [416:35:14<27:36:57,  9.47s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.8', 'learning_rate': '2.993e-07', 'epoch': '0.9533', 'num_input_tokens_seen': 706962280, 'train_runtime': '1.5e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214500/225000 [416:35:14<27:36:57,  9.47s/it]
 95%|█████████▌| 214501/225000 [416:35:22<26:06:21,  8.95s/it]
 95%|█████████▌| 214502/225000 [416:35:32<27:29:41,  9.43s/it]
 95%|█████████▌| 214503/225000 [416:35:37<23:27:15,  8.04s/it]
 95%|█████████▌| 214504/225000 [416:35:44<22:30:29,  7.72s/it]
 95%|█████████▌| 214505/225000 [416:35:57<26:40:53,  9.15s/it]
 95%|█████████▌| 214506/225000 [416:36:05<26:18:16,  9.02s/it]
 95%|█████████▌| 214507/225000 [416:36:11<22:56:30,  7.87s/it]
 95%|█████████▌| 214508/225000 [416:36:20<24:04:46,  8.26s/it]
 95%|█████████▌| 214509/225000 [416:36:30<25:49:23,  8.86s/it]
 95%|█████████▌| 214510/225000 [416:36:39<25:36:57,  8.79s/it]
 95%|█████████▌| 214511/225000 [416:36:46<24:15:45,  8.33s/it]
 95%|█████████▌| 214512/225000 [416:37:03<32:09:26, 11.04s/it]
 95%|█████████▌| 214513/225000 [416:37:11<29:17:06, 10.05s/it]
 95%|█████████▌| 214514/225000 [416:37:18<26:29:40,  9.10s/it]
 95%|█████████▌| 214515/225000 [416:37:24<24:17:38,  8.34s/it]
 95%|█████████▌| 214516/225000 [416:37:32<23:14:25,  7.98s/it]
 95%|█████████▌| 214517/225000 [416:37:49<31:26:17, 10.80s/it]
 95%|█████████▌| 214518/225000 [416:37:54<26:09:46,  8.99s/it]
 95%|█████████▌| 214519/225000 [416:38:01<24:19:12,  8.35s/it]
 95%|█████████▌| 214520/225000 [416:38:08<23:55:28,  8.22s/it]
 95%|█████████▌| 214521/225000 [416:38:16<23:07:10,  7.94s/it]
 95%|█████████▌| 214522/225000 [416:38:29<27:42:06,  9.52s/it]
 95%|█████████▌| 214523/225000 [416:38:39<28:30:10,  9.79s/it]
 95%|█████████▌| 214524/225000 [416:38:47<26:35:32,  9.14s/it]
 95%|█████████▌| 214525/225000 [416:38:54<25:06:00,  8.63s/it]
 95%|█████████▌| 214526/225000 [416:39:03<25:16:15,  8.69s/it]
 95%|█████████▌| 214527/225000 [416:39:11<24:22:38,  8.38s/it]
 95%|█████████▌| 214528/225000 [416:39:19<24:11:19,  8.32s/it]
 95%|█████████▌| 214529/225000 [416:39:29<25:31:05,  8.77s/it]
 95%|█████████▌| 214530/225000 [416:39:35<23:08:26,  7.96s/it]
 95%|█████████▌| 214531/225000 [416:39:41<21:21:09,  7.34s/it]
 95%|█████████▌| 214532/225000 [416:39:49<21:40:04,  7.45s/it]
 95%|█████████▌| 214533/225000 [416:40:00<25:04:48,  8.63s/it]
 95%|█████████▌| 214534/225000 [416:40:12<28:24:40,  9.77s/it]
 95%|█████████▌| 214535/225000 [416:40:22<28:06:33,  9.67s/it]
 95%|█████████▌| 214536/225000 [416:40:30<26:39:27,  9.17s/it]
 95%|█████████▌| 214537/225000 [416:40:42<29:00:52,  9.98s/it]
 95%|█████████▌| 214538/225000 [416:40:51<28:08:17,  9.68s/it]
 95%|█████████▌| 214539/225000 [416:41:01<28:51:18,  9.93s/it]
 95%|█████████▌| 214540/225000 [416:41:08<26:06:26,  8.99s/it]
 95%|█████████▌| 214541/225000 [416:41:15<24:04:23,  8.29s/it]
 95%|█████████▌| 214542/225000 [416:41:23<23:54:59,  8.23s/it]
 95%|█████████▌| 214543/225000 [416:41:32<25:10:11,  8.67s/it]
 95%|█████████▌| 214544/225000 [416:41:37<21:53:51,  7.54s/it]
 95%|█████████▌| 214545/225000 [416:41:49<25:08:37,  8.66s/it]
 95%|█████████▌| 214546/225000 [416:41:56<24:24:29,  8.41s/it]
 95%|█████████▌| 214547/225000 [416:42:07<26:17:45,  9.06s/it]
 95%|█████████▌| 214548/225000 [416:42:16<26:06:14,  8.99s/it]
 95%|█████████▌| 214549/225000 [416:42:23<24:39:11,  8.49s/it]
 95%|█████████▌| 214550/225000 [416:42:33<26:02:02,  8.97s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.095', 'learning_rate': '2.965e-07', 'epoch': '0.9536', 'num_input_tokens_seen': 707194904, 'train_runtime': '1.5e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214550/225000 [416:42:33<26:02:02,  8.97s/it]
 95%|█████████▌| 214551/225000 [416:42:44<28:00:32,  9.65s/it]
 95%|█████████▌| 214552/225000 [416:42:54<27:58:21,  9.64s/it]
 95%|█████████▌| 214553/225000 [416:43:01<25:14:32,  8.70s/it]
 95%|█████████▌| 214554/225000 [416:43:10<25:41:25,  8.85s/it]
 95%|█████████▌| 214555/225000 [416:43:24<30:10:50, 10.40s/it]
 95%|█████████▌| 214556/225000 [416:43:32<28:36:36,  9.86s/it]
 95%|█████████▌| 214557/225000 [416:43:39<25:33:25,  8.81s/it]
 95%|█████████▌| 214558/225000 [416:43:49<27:00:46,  9.31s/it]
 95%|█████████▌| 214559/225000 [416:43:54<23:22:14,  8.06s/it]
 95%|█████████▌| 214560/225000 [416:44:10<29:50:48, 10.29s/it]
 95%|█████████▌| 214561/225000 [416:44:17<26:51:52,  9.26s/it]
 95%|█████████▌| 214562/225000 [416:44:28<28:46:41,  9.93s/it]
 95%|█████████▌| 214563/225000 [416:44:38<28:16:23,  9.75s/it]
 95%|█████████▌| 214564/225000 [416:44:45<26:27:21,  9.13s/it]
 95%|█████████▌| 214565/225000 [416:44:55<27:15:26,  9.40s/it]
 95%|█████████▌| 214566/225000 [416:45:07<29:38:14, 10.23s/it]
 95%|█████████▌| 214567/225000 [416:45:12<25:03:29,  8.65s/it]
 95%|█████████▌| 214568/225000 [416:45:19<22:59:19,  7.93s/it]
 95%|█████████▌| 214569/225000 [416:45:32<27:30:57,  9.50s/it]
 95%|█████████▌| 214570/225000 [416:45:42<28:15:40,  9.75s/it]
 95%|█████████▌| 214571/225000 [416:45:48<24:44:42,  8.54s/it]
 95%|█████████▌| 214572/225000 [416:45:58<25:42:58,  8.88s/it]
 95%|█████████▌| 214573/225000 [416:46:03<23:01:37,  7.95s/it]
 95%|█████████▌| 214574/225000 [416:46:10<21:33:50,  7.45s/it]
 95%|█████████▌| 214575/225000 [416:46:16<20:15:51,  7.00s/it]
 95%|█████████▌| 214576/225000 [416:46:23<20:28:59,  7.07s/it]
 95%|█████████▌| 214577/225000 [416:46:37<26:24:56,  9.12s/it]
 95%|█████████▌| 214578/225000 [416:46:52<31:57:47, 11.04s/it]
 95%|█████████▌| 214579/225000 [416:46:58<27:20:17,  9.44s/it]
 95%|█████████▌| 214580/225000 [416:47:06<25:49:01,  8.92s/it]
 95%|█████████▌| 214581/225000 [416:47:14<25:27:46,  8.80s/it]
 95%|█████████▌| 214582/225000 [416:47:18<21:06:57,  7.30s/it]
 95%|█████████▌| 214583/225000 [416:47:27<22:51:39,  7.90s/it]
 95%|█████████▌| 214584/225000 [416:47:38<25:13:44,  8.72s/it]
 95%|█████████▌| 214585/225000 [416:47:46<24:31:10,  8.48s/it]
 95%|█████████▌| 214586/225000 [416:47:53<23:43:14,  8.20s/it]
 95%|█████████▌| 214587/225000 [416:48:02<24:13:58,  8.38s/it]
 95%|█████████▌| 214588/225000 [416:48:10<23:52:34,  8.26s/it]
 95%|█████████▌| 214589/225000 [416:48:23<27:43:15,  9.59s/it]
 95%|█████████▌| 214590/225000 [416:48:33<28:27:45,  9.84s/it]
 95%|█████████▌| 214591/225000 [416:48:43<28:05:03,  9.71s/it]
 95%|█████████▌| 214592/225000 [416:48:54<29:28:34, 10.20s/it]
 95%|█████████▌| 214593/225000 [416:49:01<26:31:12,  9.17s/it]
 95%|█████████▌| 214594/225000 [416:49:09<25:39:54,  8.88s/it]
 95%|█████████▌| 214595/225000 [416:49:23<29:45:12, 10.29s/it]
 95%|█████████▌| 214596/225000 [416:49:28<25:49:28,  8.94s/it]
 95%|█████████▌| 214597/225000 [416:49:37<25:50:36,  8.94s/it]
 95%|█████████▌| 214598/225000 [416:49:45<24:57:06,  8.64s/it]
 95%|█████████▌| 214599/225000 [416:49:57<27:56:50,  9.67s/it]
 95%|█████████▌| 214600/225000 [416:50:04<25:35:27,  8.86s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.533', 'learning_rate': '2.937e-07', 'epoch': '0.9538', 'num_input_tokens_seen': 707428784, 'train_runtime': '1.501e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214600/225000 [416:50:04<25:35:27,  8.86s/it]
 95%|█████████▌| 214601/225000 [416:50:12<24:32:31,  8.50s/it]
 95%|█████████▌| 214602/225000 [416:50:19<22:58:34,  7.95s/it]
 95%|█████████▌| 214603/225000 [416:50:31<26:25:13,  9.15s/it]
 95%|█████████▌| 214604/225000 [416:50:37<24:06:40,  8.35s/it]
 95%|█████████▌| 214605/225000 [416:50:43<21:50:27,  7.56s/it]
 95%|█████████▌| 214606/225000 [416:50:55<26:12:49,  9.08s/it]
 95%|█████████▌| 214607/225000 [416:51:08<29:15:01, 10.13s/it]
 95%|█████████▌| 214608/225000 [416:51:15<26:48:00,  9.28s/it]
 95%|█████████▌| 214609/225000 [416:51:21<23:50:47,  8.26s/it]
 95%|█████████▌| 214610/225000 [416:51:29<23:28:59,  8.14s/it]
 95%|█████████▌| 214611/225000 [416:51:41<27:08:55,  9.41s/it]
 95%|█████████▌| 214612/225000 [416:51:50<26:04:06,  9.03s/it]
 95%|█████████▌| 214613/225000 [416:52:01<27:48:39,  9.64s/it]
 95%|█████████▌| 214614/225000 [416:52:09<26:43:52,  9.27s/it]
 95%|█████████▌| 214615/225000 [416:52:15<24:14:11,  8.40s/it]
 95%|█████████▌| 214616/225000 [416:52:21<21:48:32,  7.56s/it]
 95%|█████████▌| 214617/225000 [416:52:27<20:21:56,  7.06s/it]
 95%|█████████▌| 214618/225000 [416:52:35<21:10:11,  7.34s/it]
 95%|█████████▌| 214619/225000 [416:52:54<31:29:46, 10.92s/it]
 95%|█████████▌| 214620/225000 [416:52:59<26:26:51,  9.17s/it]
 95%|█████████▌| 214621/225000 [416:53:05<23:07:55,  8.02s/it]
 95%|█████████▌| 214622/225000 [416:53:17<26:34:25,  9.22s/it]
 95%|█████████▌| 214623/225000 [416:53:23<24:04:45,  8.35s/it]
 95%|█████████▌| 214624/225000 [416:53:31<23:44:46,  8.24s/it]
 95%|█████████▌| 214625/225000 [416:53:38<23:13:38,  8.06s/it]
 95%|█████████▌| 214626/225000 [416:53:54<30:00:18, 10.41s/it]
 95%|█████████▌| 214627/225000 [416:54:00<25:40:11,  8.91s/it]
 95%|█████████▌| 214628/225000 [416:54:05<22:31:48,  7.82s/it]
 95%|█████████▌| 214629/225000 [416:54:12<21:23:08,  7.42s/it]
 95%|█████████▌| 214630/225000 [416:54:27<28:05:55,  9.75s/it]
 95%|█████████▌| 214631/225000 [416:54:33<25:19:23,  8.79s/it]
 95%|█████████▌| 214632/225000 [416:54:45<27:27:58,  9.54s/it]
 95%|█████████▌| 214633/225000 [416:54:52<25:44:21,  8.94s/it]
 95%|█████████▌| 214634/225000 [416:54:57<22:29:04,  7.81s/it]
 95%|█████████▌| 214635/225000 [416:55:10<26:58:35,  9.37s/it]
 95%|█████████▌| 214636/225000 [416:55:19<26:42:01,  9.27s/it]
 95%|█████████▌| 214637/225000 [416:55:29<27:18:04,  9.48s/it]
 95%|█████████▌| 214638/225000 [416:55:37<25:51:34,  8.98s/it]
 95%|█████████▌| 214639/225000 [416:55:48<27:49:33,  9.67s/it]
 95%|█████████▌| 214640/225000 [416:55:52<22:42:09,  7.89s/it]
 95%|█████████▌| 214641/225000 [416:56:01<23:20:13,  8.11s/it]
 95%|█████████▌| 214642/225000 [416:56:09<23:23:02,  8.13s/it]
 95%|█████████▌| 214643/225000 [416:56:14<21:08:18,  7.35s/it]
 95%|█████████▌| 214644/225000 [416:56:23<21:55:38,  7.62s/it]
 95%|█████████▌| 214645/225000 [416:56:32<23:03:42,  8.02s/it]
 95%|█████████▌| 214646/225000 [416:56:43<26:18:46,  9.15s/it]
 95%|█████████▌| 214647/225000 [416:56:48<22:37:57,  7.87s/it]
 95%|█████████▌| 214648/225000 [416:56:55<21:35:14,  7.51s/it]
 95%|█████████▌| 214649/225000 [416:57:03<22:23:56,  7.79s/it]
 95%|█████████▌| 214650/225000 [416:57:09<20:20:08,  7.07s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.566', 'learning_rate': '2.909e-07', 'epoch': '0.954', 'num_input_tokens_seen': 707653240, 'train_runtime': '1.501e+06', 'train_tokens_per_second': '471.4'}

 95%|█████████▌| 214650/225000 [416:57:09<20:20:08,  7.07s/it]
 95%|█████████▌| 214651/225000 [416:57:21<24:52:39,  8.65s/it]
 95%|█████████▌| 214652/225000 [416:57:26<21:19:13,  7.42s/it]
 95%|█████████▌| 214653/225000 [416:57:34<21:42:54,  7.56s/it]
 95%|█████████▌| 214654/225000 [416:57:47<26:19:56,  9.16s/it]
 95%|█████████▌| 214655/225000 [416:57:56<26:14:49,  9.13s/it]
 95%|█████████▌| 214656/225000 [416:58:02<24:14:07,  8.43s/it]
 95%|█████████▌| 214657/225000 [416:58:12<25:15:14,  8.79s/it]
 95%|█████████▌| 214658/225000 [416:58:20<24:09:41,  8.41s/it]
 95%|█████████▌| 214659/225000 [416:58:30<25:50:36,  9.00s/it]
 95%|█████████▌| 214660/225000 [416:58:41<27:27:31,  9.56s/it]
 95%|█████████▌| 214661/225000 [416:58:52<28:28:53,  9.92s/it]
 95%|█████████▌| 214662/225000 [416:59:00<26:55:52,  9.38s/it]
 95%|█████████▌| 214663/225000 [416:59:07<25:08:18,  8.75s/it]
 95%|█████████▌| 214664/225000 [416:59:21<29:44:03, 10.36s/it]
 95%|█████████▌| 214665/225000 [416:59:27<26:13:56,  9.14s/it]
 95%|█████████▌| 214666/225000 [416:59:39<28:08:26,  9.80s/it]
 95%|█████████▌| 214667/225000 [416:59:48<27:58:05,  9.74s/it]
 95%|█████████▌| 214668/225000 [416:59:58<28:13:02,  9.83s/it]
 95%|█████████▌| 214669/225000 [417:00:08<27:59:53,  9.76s/it]
 95%|█████████▌| 214670/225000 [417:00:18<28:10:44,  9.82s/it]
 95%|█████████▌| 214671/225000 [417:00:25<26:13:57,  9.14s/it]
 95%|█████████▌| 214672/225000 [417:00:36<27:38:54,  9.64s/it]
 95%|█████████▌| 214673/225000 [417:00:41<23:24:39,  8.16s/it]
 95%|█████████▌| 214674/225000 [417:00:55<28:42:52, 10.01s/it]
 95%|█████████▌| 214675/225000 [417:01:04<27:29:09,  9.58s/it]
 95%|█████████▌| 214676/225000 [417:01:13<27:16:23,  9.51s/it]
 95%|█████████▌| 214677/225000 [417:01:20<24:51:02,  8.67s/it]
 95%|█████████▌| 214678/225000 [417:01:31<27:16:19,  9.51s/it]
 95%|█████████▌| 214679/225000 [417:01:37<24:10:42,  8.43s/it]
 95%|█████████▌| 214680/225000 [417:01:45<23:39:52,  8.26s/it]
 95%|█████████▌| 214681/225000 [417:01:55<25:07:34,  8.77s/it]
 95%|█████████▌| 214682/225000 [417:02:05<25:53:47,  9.04s/it]
 95%|█████████▌| 214683/225000 [417:02:12<24:32:26,  8.56s/it]
 95%|█████████▌| 214684/225000 [417:02:22<25:55:35,  9.05s/it]
 95%|█████████▌| 214685/225000 [417:02:28<22:38:36,  7.90s/it]
 95%|█████████▌| 214686/225000 [417:02:40<26:10:27,  9.14s/it]
 95%|█████████▌| 214687/225000 [417:02:46<23:38:11,  8.25s/it]
 95%|█████████▌| 214688/225000 [417:02:59<27:29:35,  9.60s/it]
 95%|█████████▌| 214689/225000 [417:03:05<24:20:00,  8.50s/it]
 95%|█████████▌| 214690/225000 [417:03:10<21:32:47,  7.52s/it]
 95%|█████████▌| 214691/225000 [417:03:26<28:56:58, 10.11s/it]
 95%|█████████▌| 214692/225000 [417:03:36<29:02:17, 10.14s/it]
 95%|█████████▌| 214693/225000 [417:03:44<27:24:59,  9.58s/it]
 95%|█████████▌| 214694/225000 [417:03:52<26:05:18,  9.11s/it]
 95%|█████████▌| 214695/225000 [417:04:00<24:33:57,  8.58s/it]
 95%|█████████▌| 214696/225000 [417:04:05<21:54:53,  7.66s/it]
 95%|█████████▌| 214697/225000 [417:04:11<20:36:17,  7.20s/it]
 95%|█████████▌| 214698/225000 [417:04:22<23:47:08,  8.31s/it]
 95%|█████████▌| 214699/225000 [417:04:29<22:15:36,  7.78s/it]
 95%|█████████▌| 214700/225000 [417:04:40<25:18:02,  8.84s/it]
                                                              
{'loss': '1.321', 'grad_norm': '1.048', 'learning_rate': '2.882e-07', 'epoch': '0.9542', 'num_input_tokens_seen': 707890048, 'train_runtime': '1.501e+06', 'train_tokens_per_second': '471.5'}

 95%|█████████▌| 214700/225000 [417:04:40<25:18:02,  8.84s/it]
 95%|█████████▌| 214701/225000 [417:04:51<27:20:35,  9.56s/it]
 95%|█████████▌| 214702/225000 [417:04:57<23:34:44,  8.24s/it]
 95%|█████████▌| 214703/225000 [417:05:03<22:16:40,  7.79s/it]
 95%|█████████▌| 214704/225000 [417:05:15<25:46:22,  9.01s/it]
 95%|█████████▌| 214705/225000 [417:05:23<25:05:46,  8.78s/it]
 95%|█████████▌| 214706/225000 [417:05:32<24:50:46,  8.69s/it]
 95%|█████████▌| 214707/225000 [417:05:40<24:23:46,  8.53s/it]
 95%|█████████▌| 214708/225000 [417:05:56<31:04:56, 10.87s/it]
 95%|█████████▌| 214709/225000 [417:06:03<27:42:41,  9.69s/it]
 95%|█████████▌| 214710/225000 [417:06:10<25:14:06,  8.83s/it]
 95%|█████████▌| 214711/225000 [417:06:17<23:16:19,  8.14s/it]
 95%|█████████▌| 214712/225000 [417:06:29<26:59:22,  9.44s/it]
 95%|█████████▌| 214713/225000 [417:06:41<28:38:29, 10.02s/it]
 95%|█████████▌| 214714/225000 [417:06:46<24:51:28,  8.70s/it]
 95%|█████████▌| 214715/225000 [417:06:51<21:55:38,  7.68s/it]
 95%|█████████▌| 214716/225000 [417:07:02<24:04:30,  8.43s/it]
 95%|█████████▌| 214717/225000 [417:07:11<25:00:58,  8.76s/it]
 95%|█████████▌| 214718/225000 [417:07:19<24:38:51,  8.63s/it]
 95%|█████████▌| 214719/225000 [417:07:26<22:42:57,  7.95s/it]
 95%|█████████▌| 214720/225000 [417:07:35<23:49:14,  8.34s/it]
 95%|█████████▌| 214721/225000 [417:07:44<23:56:50,  8.39s/it]
 95%|█████████▌| 214722/225000 [417:07:54<25:26:56,  8.91s/it]
 95%|█████████▌| 214723/225000 [417:08:04<26:30:00,  9.28s/it]
 95%|█████████▌| 214724/225000 [417:08:10<23:41:33,  8.30s/it]
 95%|█████████▌| 214725/225000 [417:08:18<23:38:02,  8.28s/it]
 95%|█████████▌| 214726/225000 [417:08:33<29:27:26, 10.32s/it]
 95%|█████████▌| 214727/225000 [417:08:41<27:43:36,  9.72s/it]
 95%|█████████▌| 214728/225000 [417:08:53<28:59:20, 10.16s/it]
 95%|█████████▌| 214729/225000 [417:09:00<26:10:13,  9.17s/it]
 95%|█████████▌| 214730/225000 [417:09:06<24:09:50,  8.47s/it]
 95%|█████████▌| 214731/225000 [417:09:20<28:11:28,  9.88s/it]
 95%|█████████▌| 214732/225000 [417:09:27<26:30:12,  9.29s/it]
 95%|█████████▌| 214733/225000 [417:09:35<25:15:19,  8.86s/it]
 95%|█████████▌| 214734/225000 [417:09:51<30:59:43, 10.87s/it]
 95%|█████████▌| 214735/225000 [417:09:59<28:49:26, 10.11s/it]
 95%|█████████▌| 214736/225000 [417:10:08<27:32:10,  9.66s/it]
 95%|█████████▌| 214737/225000 [417:10:14<24:31:09,  8.60s/it]
 95%|█████████▌| 214738/225000 [417:10:23<24:50:56,  8.72s/it]
 95%|█████████▌| 214739/225000 [417:10:34<26:45:34,  9.39s/it]
 95%|█████████▌| 214740/225000 [417:10:41<24:45:36,  8.69s/it]
 95%|█████████▌| 214741/225000 [417:10:51<25:41:28,  9.02s/it]
 95%|█████████▌| 214742/225000 [417:11:00<26:03:32,  9.15s/it]
 95%|█████████▌| 214743/225000 [417:11:10<26:19:12,  9.24s/it]
 95%|█████████▌| 214744/225000 [417:11:18<25:37:13,  8.99s/it]
 95%|█████████▌| 214745/225000 [417:11:24<22:47:44,  8.00s/it]
 95%|█████████▌| 214746/225000 [417:11:39<29:02:59, 10.20s/it]
 95%|█████████▌| 214747/225000 [417:11:48<28:01:01,  9.84s/it]
 95%|█████████▌| 214748/225000 [417:11:56<26:26:01,  9.28s/it]
 95%|█████████▌| 214749/225000 [417:12:02<23:37:28,  8.30s/it]
 95%|█████████▌| 214750/225000 [417:12:14<26:26:48,  9.29s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.18', 'learning_rate': '2.854e-07', 'epoch': '0.9544', 'num_input_tokens_seen': 708126192, 'train_runtime': '1.502e+06', 'train_tokens_per_second': '471.5'}

 95%|█████████▌| 214750/225000 [417:12:14<26:26:48,  9.29s/it]
 95%|█████████▌| 214751/225000 [417:12:19<23:20:05,  8.20s/it]
 95%|█████████▌| 214752/225000 [417:12:25<21:36:28,  7.59s/it]
 95%|█████████▌| 214753/225000 [417:12:33<21:20:07,  7.50s/it]
 95%|█████████▌| 214754/225000 [417:12:43<23:47:29,  8.36s/it]
 95%|█████████▌| 214755/225000 [417:12:51<23:27:22,  8.24s/it]
 95%|█████████▌| 214756/225000 [417:13:01<25:09:39,  8.84s/it]
 95%|█████████▌| 214757/225000 [417:13:08<23:37:54,  8.31s/it]
 95%|█████████▌| 214758/225000 [417:13:21<27:35:58,  9.70s/it]
 95%|█████████▌| 214759/225000 [417:13:28<24:46:00,  8.71s/it]
 95%|█████████▌| 214760/225000 [417:13:33<21:43:39,  7.64s/it]
 95%|█████████▌| 214761/225000 [417:13:44<24:26:00,  8.59s/it]
 95%|█████████▌| 214762/225000 [417:13:49<21:42:24,  7.63s/it]
 95%|█████████▌| 214763/225000 [417:13:56<20:58:55,  7.38s/it]
 95%|█████████▌| 214764/225000 [417:14:07<24:08:37,  8.49s/it]
 95%|█████████▌| 214765/225000 [417:14:14<22:37:51,  7.96s/it]
 95%|█████████▌| 214766/225000 [417:14:21<21:48:29,  7.67s/it]
 95%|█████████▌| 214767/225000 [417:14:28<21:43:57,  7.65s/it]
 95%|█████████▌| 214768/225000 [417:14:34<20:05:25,  7.07s/it]
 95%|█████████▌| 214769/225000 [417:14:47<25:11:09,  8.86s/it]
 95%|█████████▌| 214770/225000 [417:15:03<31:37:40, 11.13s/it]
 95%|█████████▌| 214771/225000 [417:15:12<29:31:19, 10.39s/it]
 95%|█████████▌| 214772/225000 [417:15:20<27:17:47,  9.61s/it]
 95%|█████████▌| 214773/225000 [417:15:28<26:10:46,  9.22s/it]
 95%|█████████▌| 214774/225000 [417:15:39<27:15:19,  9.60s/it]
 95%|█████████▌| 214775/225000 [417:15:47<25:54:16,  9.12s/it]
 95%|█████████▌| 214776/225000 [417:15:53<23:37:32,  8.32s/it]
 95%|█████████▌| 214777/225000 [417:16:07<28:20:31,  9.98s/it]
 95%|█████████▌| 214778/225000 [417:16:14<25:47:02,  9.08s/it]
 95%|█████████▌| 214779/225000 [417:16:25<27:21:04,  9.63s/it]
 95%|█████████▌| 214780/225000 [417:16:35<27:57:29,  9.85s/it]
 95%|█████████▌| 214781/225000 [417:16:41<24:05:10,  8.49s/it]
 95%|█████████▌| 214782/225000 [417:16:45<21:02:09,  7.41s/it]
 95%|█████████▌| 214783/225000 [417:16:52<20:39:28,  7.28s/it]
 95%|█████████▌| 214784/225000 [417:17:04<24:30:30,  8.64s/it]
 95%|█████████▌| 214785/225000 [417:17:16<27:00:36,  9.52s/it]
 95%|█████████▌| 214786/225000 [417:17:23<24:39:30,  8.69s/it]
 95%|█████████▌| 214787/225000 [417:17:31<24:10:30,  8.52s/it]
 95%|█████████▌| 214788/225000 [417:17:42<26:12:51,  9.24s/it]
 95%|█████████▌| 214789/225000 [417:17:48<23:33:51,  8.31s/it]
 95%|█████████▌| 214790/225000 [417:18:01<27:25:16,  9.67s/it]
 95%|█████████▌| 214791/225000 [417:18:07<24:54:18,  8.78s/it]
 95%|█████████▌| 214792/225000 [417:18:17<25:47:26,  9.10s/it]
 95%|█████████▌| 214793/225000 [417:18:27<26:40:17,  9.41s/it]
 95%|█████████▌| 214794/225000 [417:18:33<23:17:43,  8.22s/it]
 95%|█████████▌| 214795/225000 [417:18:39<21:35:22,  7.62s/it]
 95%|█████████▌| 214796/225000 [417:18:54<27:43:09,  9.78s/it]
 95%|█████████▌| 214797/225000 [417:19:00<25:04:47,  8.85s/it]
 95%|█████████▌| 214798/225000 [417:19:07<23:19:21,  8.23s/it]
 95%|█████████▌| 214799/225000 [417:19:15<22:31:11,  7.95s/it]
 95%|█████████▌| 214800/225000 [417:19:25<24:50:30,  8.77s/it]
                                                              
{'loss': '1.285', 'grad_norm': '1.461', 'learning_rate': '2.827e-07', 'epoch': '0.9547', 'num_input_tokens_seen': 708351424, 'train_runtime': '1.502e+06', 'train_tokens_per_second': '471.5'}

 95%|█████████▌| 214800/225000 [417:19:25<24:50:30,  8.77s/it]
 95%|█████████▌| 214801/225000 [417:19:37<27:01:17,  9.54s/it]
 95%|█████████▌| 214802/225000 [417:19:48<28:40:41, 10.12s/it]
 95%|█████████▌| 214803/225000 [417:19:56<26:27:46,  9.34s/it]
 95%|█████████▌| 214804/225000 [417:20:01<23:07:36,  8.17s/it]
 95%|█████████▌| 214805/225000 [417:20:12<25:39:48,  9.06s/it]
 95%|█████████▌| 214806/225000 [417:20:20<24:41:16,  8.72s/it]
 95%|█████████▌| 214807/225000 [417:20:29<24:57:47,  8.82s/it]
 95%|█████████▌| 214808/225000 [417:20:38<25:02:28,  8.85s/it]
 95%|█████████▌| 214809/225000 [417:20:45<23:14:14,  8.21s/it]
 95%|█████████▌| 214810/225000 [417:20:54<24:17:48,  8.58s/it]
 95%|█████████▌| 214811/225000 [417:21:06<27:13:02,  9.62s/it]
 95%|█████████▌| 214812/225000 [417:21:13<25:02:41,  8.85s/it]
 95%|█████████▌| 214813/225000 [417:21:19<22:37:08,  7.99s/it]
 95%|█████████▌| 214814/225000 [417:21:27<22:37:31,  8.00s/it]
 95%|█████████▌| 214815/225000 [417:21:38<25:01:40,  8.85s/it]
 95%|█████████▌| 214816/225000 [417:21:44<22:57:52,  8.12s/it]
 95%|█████████▌| 214817/225000 [417:21:57<26:22:44,  9.33s/it]
 95%|█████████▌| 214818/225000 [417:22:01<22:10:11,  7.84s/it]
 95%|█████████▌| 214819/225000 [417:22:15<27:08:07,  9.60s/it]
 95%|█████████▌| 214820/225000 [417:22:25<27:59:52,  9.90s/it]
 95%|█████████▌| 214821/225000 [417:22:35<28:02:09,  9.92s/it]
 95%|█████████▌| 214822/225000 [417:22:41<24:54:09,  8.81s/it]
 95%|█████████▌| 214823/225000 [417:22:47<22:24:03,  7.92s/it]
 95%|█████████▌| 214824/225000 [417:22:59<25:49:51,  9.14s/it]
 95%|█████████▌| 214825/225000 [417:23:06<23:38:51,  8.37s/it]
 95%|█████████▌| 214826/225000 [417:23:13<22:17:42,  7.89s/it]
 95%|█████████▌| 214827/225000 [417:23:27<27:29:38,  9.73s/it]
 95%|█████████▌| 214828/225000 [417:23:34<25:21:07,  8.97s/it]
 95%|█████████▌| 214829/225000 [417:23:40<22:44:20,  8.05s/it]
 95%|█████████▌| 214830/225000 [417:23:56<29:24:09, 10.41s/it]
 95%|█████████▌| 214831/225000 [417:24:10<33:02:14, 11.70s/it]
 95%|█████████▌| 214832/225000 [417:24:18<29:23:53, 10.41s/it]
 95%|█████████▌| 214833/225000 [417:24:31<31:23:00, 11.11s/it]
 95%|█████████▌| 214834/225000 [417:24:39<29:20:17, 10.39s/it]
 95%|█████████▌| 214835/225000 [417:24:47<27:19:11,  9.68s/it]
 95%|█████████▌| 214836/225000 [417:24:56<26:47:57,  9.49s/it]
 95%|█████████▌| 214837/225000 [417:25:08<28:41:07, 10.16s/it]
 95%|█████████▌| 214838/225000 [417:25:15<25:34:35,  9.06s/it]
 95%|█████████▌| 214839/225000 [417:25:28<28:52:36, 10.23s/it]
 95%|█████████▌| 214840/225000 [417:25:34<25:29:02,  9.03s/it]
 95%|█████████▌| 214841/225000 [417:25:40<23:06:19,  8.19s/it]
 95%|█████████▌| 214842/225000 [417:25:50<24:31:48,  8.69s/it]
 95%|█████████▌| 214843/225000 [417:25:57<23:38:02,  8.38s/it]
 95%|█████████▌| 214844/225000 [417:26:10<27:15:14,  9.66s/it]
 95%|█████████▌| 214845/225000 [417:26:23<29:43:52, 10.54s/it]
 95%|█████████▌| 214846/225000 [417:26:31<27:33:14,  9.77s/it]
 95%|█████████▌| 214847/225000 [417:26:43<29:20:56, 10.41s/it]
 95%|█████████▌| 214848/225000 [417:26:48<25:12:42,  8.94s/it]
 95%|█████████▌| 214849/225000 [417:26:58<25:46:07,  9.14s/it]
 95%|█████████▌| 214850/225000 [417:27:03<22:47:22,  8.08s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.708', 'learning_rate': '2.799e-07', 'epoch': '0.9549', 'num_input_tokens_seen': 708588040, 'train_runtime': '1.503e+06', 'train_tokens_per_second': '471.5'}

 95%|█████████▌| 214850/225000 [417:27:03<22:47:22,  8.08s/it]
 95%|█████████▌| 214851/225000 [417:27:16<26:26:32,  9.38s/it]
 95%|█████████▌| 214852/225000 [417:27:22<23:31:00,  8.34s/it]
 95%|█████████▌| 214853/225000 [417:27:26<20:19:47,  7.21s/it]
 95%|█████████▌| 214854/225000 [417:27:40<25:43:00,  9.12s/it]
 95%|█████████▌| 214855/225000 [417:27:47<23:41:33,  8.41s/it]
 95%|█████████▌| 214856/225000 [417:27:56<24:38:36,  8.75s/it]
 95%|█████████▌| 214857/225000 [417:28:08<27:40:42,  9.82s/it]
 95%|█████████▌| 214858/225000 [417:28:19<28:39:20, 10.17s/it]
 95%|█████████▌| 214859/225000 [417:28:26<25:15:15,  8.97s/it]
 95%|█████████▌| 214860/225000 [417:28:34<25:02:05,  8.89s/it]
 95%|█████████▌| 214861/225000 [417:28:45<26:13:16,  9.31s/it]
 95%|█████████▌| 214862/225000 [417:28:52<24:44:31,  8.79s/it]
 95%|█████████▌| 214863/225000 [417:29:00<23:48:33,  8.46s/it]
 95%|█████████▌| 214864/225000 [417:29:07<23:06:38,  8.21s/it]
 95%|█████████▌| 214865/225000 [417:29:19<25:36:28,  9.10s/it]
 95%|█████████▌| 214866/225000 [417:29:25<23:15:35,  8.26s/it]
 95%|█████████▌| 214867/225000 [417:29:34<24:05:23,  8.56s/it]
 95%|█████████▌| 214868/225000 [417:29:42<23:20:28,  8.29s/it]
 95%|█████████▌| 214869/225000 [417:29:50<22:51:44,  8.12s/it]
 95%|█████████▌| 214870/225000 [417:29:56<21:46:08,  7.74s/it]
 95%|█████████▌| 214871/225000 [417:30:00<18:38:37,  6.63s/it]
 95%|█████████▌| 214872/225000 [417:30:13<24:03:55,  8.55s/it]
 95%|█████████▌| 214873/225000 [417:30:19<21:28:54,  7.64s/it]
 95%|█████████▌| 214874/225000 [417:30:28<22:29:52,  8.00s/it]
 96%|█████████▌| 214875/225000 [417:30:34<20:40:35,  7.35s/it]
 96%|█████████▌| 214876/225000 [417:30:47<25:42:08,  9.14s/it]
 96%|█████████▌| 214877/225000 [417:30:52<22:17:08,  7.93s/it]
 96%|█████████▌| 214878/225000 [417:31:00<22:19:04,  7.94s/it]
 96%|█████████▌| 214879/225000 [417:31:16<29:14:37, 10.40s/it]
 96%|█████████▌| 214880/225000 [417:31:27<29:34:10, 10.52s/it]
 96%|█████████▌| 214881/225000 [417:31:34<26:54:21,  9.57s/it]
 96%|█████████▌| 214882/225000 [417:31:42<25:18:13,  9.00s/it]
 96%|█████████▌| 214883/225000 [417:31:47<22:04:00,  7.85s/it]
 96%|█████████▌| 214884/225000 [417:31:52<19:52:00,  7.07s/it]
 96%|█████████▌| 214885/225000 [417:32:08<27:02:21,  9.62s/it]
 96%|█████████▌| 214886/225000 [417:32:18<27:35:23,  9.82s/it]
 96%|█████████▌| 214887/225000 [417:32:26<25:25:00,  9.05s/it]
 96%|█████████▌| 214888/225000 [417:32:33<23:46:12,  8.46s/it]
 96%|█████████▌| 214889/225000 [417:32:45<27:11:44,  9.68s/it]
 96%|█████████▌| 214890/225000 [417:32:55<27:20:12,  9.73s/it]
 96%|█████████▌| 214891/225000 [417:33:01<24:07:34,  8.59s/it]
 96%|█████████▌| 214892/225000 [417:33:09<23:25:52,  8.35s/it]
 96%|█████████▌| 214893/225000 [417:33:16<22:09:16,  7.89s/it]
 96%|█████████▌| 214894/225000 [417:33:26<24:27:46,  8.71s/it]
 96%|█████████▌| 214895/225000 [417:33:33<23:03:06,  8.21s/it]
 96%|█████████▌| 214896/225000 [417:33:41<22:22:34,  7.97s/it]
 96%|█████████▌| 214897/225000 [417:33:52<25:37:30,  9.13s/it]
 96%|█████████▌| 214898/225000 [417:34:04<28:02:25,  9.99s/it]
 96%|█████████▌| 214899/225000 [417:34:11<24:44:35,  8.82s/it]
 96%|█████████▌| 214900/225000 [417:34:15<21:27:30,  7.65s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.573', 'learning_rate': '2.772e-07', 'epoch': '0.9551', 'num_input_tokens_seen': 708818632, 'train_runtime': '1.503e+06', 'train_tokens_per_second': '471.5'}

 96%|█████████▌| 214900/225000 [417:34:15<21:27:30,  7.65s/it]
 96%|█████████▌| 214901/225000 [417:34:27<24:38:35,  8.78s/it]
 96%|█████████▌| 214902/225000 [417:34:32<21:51:26,  7.79s/it]
 96%|█████████▌| 214903/225000 [417:34:41<22:42:04,  8.09s/it]
 96%|█████████▌| 214904/225000 [417:34:47<20:29:46,  7.31s/it]
 96%|█████████▌| 214905/225000 [417:35:00<25:48:56,  9.21s/it]
 96%|█████████▌| 214906/225000 [417:35:06<22:39:23,  8.08s/it]
 96%|█████████▌| 214907/225000 [417:35:18<26:26:55,  9.43s/it]
 96%|█████████▌| 214908/225000 [417:35:23<22:38:59,  8.08s/it]
 96%|█████████▌| 214909/225000 [417:35:35<26:00:52,  9.28s/it]
 96%|█████████▌| 214910/225000 [417:35:46<27:29:58,  9.81s/it]
 96%|█████████▌| 214911/225000 [417:35:54<25:50:54,  9.22s/it]
 96%|█████████▌| 214912/225000 [417:36:06<28:00:17,  9.99s/it]
 96%|█████████▌| 214913/225000 [417:36:11<23:51:16,  8.51s/it]
 96%|█████████▌| 214914/225000 [417:36:21<24:59:25,  8.92s/it]
 96%|█████████▌| 214915/225000 [417:36:27<22:23:30,  7.99s/it]
 96%|█████████▌| 214916/225000 [417:36:33<21:15:54,  7.59s/it]
 96%|█████████▌| 214917/225000 [417:36:41<21:12:33,  7.57s/it]
 96%|█████████▌| 214918/225000 [417:36:53<24:37:53,  8.80s/it]
 96%|█████████▌| 214919/225000 [417:37:00<23:39:47,  8.45s/it]
 96%|█████████▌| 214920/225000 [417:37:09<23:46:14,  8.49s/it]
 96%|█████████▌| 214921/225000 [417:37:21<26:33:58,  9.49s/it]
 96%|█████████▌| 214922/225000 [417:37:32<28:31:01, 10.19s/it]
 96%|█████████▌| 214923/225000 [417:37:46<30:58:08, 11.06s/it]
 96%|█████████▌| 214924/225000 [417:37:52<26:41:41,  9.54s/it]
 96%|█████████▌| 214925/225000 [417:37:59<24:37:55,  8.80s/it]
 96%|█████████▌| 214926/225000 [417:38:05<22:31:44,  8.05s/it]
 96%|█████████▌| 214927/225000 [417:38:12<21:28:12,  7.67s/it]
 96%|█████████▌| 214928/225000 [417:38:25<26:05:01,  9.32s/it]
 96%|█████████▌| 214929/225000 [417:38:35<26:49:23,  9.59s/it]
 96%|█████████▌| 214930/225000 [417:38:48<29:26:08, 10.52s/it]
 96%|█████████▌| 214931/225000 [417:38:57<28:09:21, 10.07s/it]
 96%|█████████▌| 214932/225000 [417:39:09<29:45:08, 10.64s/it]
 96%|█████████▌| 214933/225000 [417:39:18<28:12:37, 10.09s/it]
 96%|█████████▌| 214934/225000 [417:39:24<24:53:40,  8.90s/it]
 96%|█████████▌| 214935/225000 [417:39:31<23:44:05,  8.49s/it]
 96%|█████████▌| 214936/225000 [417:39:39<23:12:39,  8.30s/it]
 96%|█████████▌| 214937/225000 [417:39:46<22:23:52,  8.01s/it]
 96%|█████████▌| 214938/225000 [417:39:57<24:20:24,  8.71s/it]
 96%|█████████▌| 214939/225000 [417:40:05<24:01:05,  8.59s/it]
 96%|█████████▌| 214940/225000 [417:40:15<24:46:56,  8.87s/it]
 96%|█████████▌| 214941/225000 [417:40:21<23:01:35,  8.24s/it]
 96%|█████████▌| 214942/225000 [417:40:29<22:31:41,  8.06s/it]
 96%|█████████▌| 214943/225000 [417:40:35<20:59:04,  7.51s/it]
 96%|█████████▌| 214944/225000 [417:40:49<25:53:00,  9.27s/it]
 96%|█████████▌| 214945/225000 [417:40:55<23:50:36,  8.54s/it]
 96%|█████████▌| 214946/225000 [417:41:01<21:02:27,  7.53s/it]
 96%|█████████▌| 214947/225000 [417:41:14<25:58:35,  9.30s/it]
 96%|█████████▌| 214948/225000 [417:41:21<24:17:36,  8.70s/it]
 96%|█████████▌| 214949/225000 [417:41:28<22:18:07,  7.99s/it]
 96%|█████████▌| 214950/225000 [417:41:34<20:35:15,  7.37s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.381', 'learning_rate': '2.745e-07', 'epoch': '0.9553', 'num_input_tokens_seen': 709047616, 'train_runtime': '1.504e+06', 'train_tokens_per_second': '471.5'}

 96%|█████████▌| 214950/225000 [417:41:34<20:35:15,  7.37s/it]
 96%|█████████▌| 214951/225000 [417:41:47<25:34:45,  9.16s/it]
 96%|█████████▌| 214952/225000 [417:41:55<24:27:07,  8.76s/it]
 96%|█████████▌| 214953/225000 [417:42:04<24:52:53,  8.92s/it]
 96%|█████████▌| 214954/225000 [417:42:10<22:11:24,  7.95s/it]
 96%|█████████▌| 214955/225000 [417:42:17<21:46:19,  7.80s/it]
 96%|█████████▌| 214956/225000 [417:42:30<25:34:10,  9.16s/it]
 96%|█████████▌| 214957/225000 [417:42:35<22:11:31,  7.95s/it]
 96%|█████████▌| 214958/225000 [417:42:47<25:36:11,  9.18s/it]
 96%|█████████▌| 214959/225000 [417:42:53<23:16:10,  8.34s/it]
 96%|█████████▌| 214960/225000 [417:42:59<20:51:38,  7.48s/it]
 96%|█████████▌| 214961/225000 [417:43:09<23:38:20,  8.48s/it]
 96%|█████████▌| 214962/225000 [417:43:16<22:06:28,  7.93s/it]
 96%|█████████▌| 214963/225000 [417:43:27<24:53:50,  8.93s/it]
 96%|█████████▌| 214964/225000 [417:43:35<24:03:32,  8.63s/it]
 96%|█████████▌| 214965/225000 [417:43:44<24:22:39,  8.75s/it]
 96%|█████████▌| 214966/225000 [417:43:51<23:01:18,  8.26s/it]
 96%|█████████▌| 214967/225000 [417:44:00<23:39:20,  8.49s/it]
 96%|█████████▌| 214968/225000 [417:44:15<28:53:48, 10.37s/it]
 96%|█████████▌| 214969/225000 [417:44:23<26:19:40,  9.45s/it]
 96%|█████████▌| 214970/225000 [417:44:28<23:17:32,  8.36s/it]
 96%|█████████▌| 214971/225000 [417:44:37<23:54:56,  8.58s/it]
 96%|█████████▌| 214972/225000 [417:44:45<23:28:08,  8.43s/it]
 96%|█████████▌| 214973/225000 [417:44:57<25:44:52,  9.24s/it]
 96%|█████████▌| 214974/225000 [417:45:01<21:50:44,  7.84s/it]
 96%|█████████▌| 214975/225000 [417:45:10<22:25:23,  8.05s/it]
 96%|█████████▌| 214976/225000 [417:45:16<20:35:02,  7.39s/it]
 96%|█████████▌| 214977/225000 [417:45:21<19:19:03,  6.94s/it]
 96%|█████████▌| 214978/225000 [417:45:31<21:51:04,  7.85s/it]
 96%|█████████▌| 214979/225000 [417:45:38<20:33:00,  7.38s/it]
 96%|█████████▌| 214980/225000 [417:45:49<23:40:57,  8.51s/it]
 96%|█████████▌| 214981/225000 [417:45:59<24:39:25,  8.86s/it]
 96%|█████████▌| 214982/225000 [417:46:07<24:31:18,  8.81s/it]
 96%|█████████▌| 214983/225000 [417:46:15<23:53:19,  8.59s/it]
 96%|█████████▌| 214984/225000 [417:46:26<25:55:46,  9.32s/it]
 96%|█████████▌| 214985/225000 [417:46:33<23:43:01,  8.53s/it]
 96%|█████████▌| 214986/225000 [417:46:43<25:09:39,  9.05s/it]
 96%|█████████▌| 214987/225000 [417:46:53<26:00:50,  9.35s/it]
 96%|█████████▌| 214988/225000 [417:47:05<27:40:48,  9.95s/it]
 96%|█████████▌| 214989/225000 [417:47:14<27:12:37,  9.78s/it]
 96%|█████████▌| 214990/225000 [417:47:20<24:01:30,  8.64s/it]
 96%|█████████▌| 214991/225000 [417:47:29<24:34:20,  8.84s/it]
 96%|█████████▌| 214992/225000 [417:47:37<23:51:24,  8.58s/it]
 96%|█████████▌| 214993/225000 [417:47:44<22:31:31,  8.10s/it]
 96%|█████████▌| 214994/225000 [417:47:59<28:21:36, 10.20s/it]
 96%|█████████▌| 214995/225000 [417:48:05<24:40:30,  8.88s/it]
 96%|█████████▌| 214996/225000 [417:48:16<26:34:44,  9.56s/it]
 96%|█████████▌| 214997/225000 [417:48:27<27:10:57,  9.78s/it]
 96%|█████████▌| 214998/225000 [417:48:42<31:49:54, 11.46s/it]
 96%|█████████▌| 214999/225000 [417:48:46<25:55:17,  9.33s/it]
 96%|█████████▌| 215000/225000 [417:49:00<29:05:49, 10.47s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.361', 'learning_rate': '2.718e-07', 'epoch': '0.9556', 'num_input_tokens_seen': 709277184, 'train_runtime': '1.504e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215000/225000 [417:49:00<29:05:49, 10.47s/it]
 96%|█████████▌| 215001/225000 [417:49:07<26:55:26,  9.69s/it]
 96%|█████████▌| 215002/225000 [417:49:17<26:46:16,  9.64s/it]
 96%|█████████▌| 215003/225000 [417:49:22<23:02:23,  8.30s/it]
 96%|█████████▌| 215004/225000 [417:49:34<26:18:07,  9.47s/it]
 96%|█████████▌| 215005/225000 [417:49:46<27:51:31, 10.03s/it]
 96%|█████████▌| 215006/225000 [417:49:54<26:38:54,  9.60s/it]
 96%|█████████▌| 215007/225000 [417:50:02<25:14:48,  9.10s/it]
 96%|█████████▌| 215008/225000 [417:50:08<22:44:09,  8.19s/it]
 96%|█████████▌| 215009/225000 [417:50:16<22:23:06,  8.07s/it]
 96%|█████████▌| 215010/225000 [417:50:31<28:31:19, 10.28s/it]
 96%|█████████▌| 215011/225000 [417:50:38<25:05:27,  9.04s/it]
 96%|█████████▌| 215012/225000 [417:50:51<28:29:18, 10.27s/it]
 96%|█████████▌| 215013/225000 [417:51:00<27:15:59,  9.83s/it]
 96%|█████████▌| 215014/225000 [417:51:11<28:12:41, 10.17s/it]
 96%|█████████▌| 215015/225000 [417:51:17<25:13:24,  9.09s/it]
 96%|█████████▌| 215016/225000 [417:51:26<24:56:09,  8.99s/it]
 96%|█████████▌| 215017/225000 [417:51:33<23:07:02,  8.34s/it]
 96%|█████████▌| 215018/225000 [417:51:39<21:33:54,  7.78s/it]
 96%|█████████▌| 215019/225000 [417:51:45<19:38:01,  7.08s/it]
 96%|█████████▌| 215020/225000 [417:51:58<25:02:56,  9.04s/it]
 96%|█████████▌| 215021/225000 [417:52:07<24:33:38,  8.86s/it]
 96%|█████████▌| 215022/225000 [417:52:15<23:45:28,  8.57s/it]
 96%|█████████▌| 215023/225000 [417:52:22<22:57:15,  8.28s/it]
 96%|█████████▌| 215024/225000 [417:52:31<23:24:30,  8.45s/it]
 96%|█████████▌| 215025/225000 [417:52:44<27:23:41,  9.89s/it]
 96%|█████████▌| 215026/225000 [417:52:55<28:04:17, 10.13s/it]
 96%|█████████▌| 215027/225000 [417:53:00<23:25:43,  8.46s/it]
 96%|█████████▌| 215028/225000 [417:53:06<21:49:37,  7.88s/it]
 96%|█████████▌| 215029/225000 [417:53:17<24:05:51,  8.70s/it]
 96%|█████████▌| 215030/225000 [417:53:25<23:32:21,  8.50s/it]
 96%|█████████▌| 215031/225000 [417:53:33<23:14:13,  8.39s/it]
 96%|█████████▌| 215032/225000 [417:53:44<25:29:13,  9.20s/it]
 96%|█████████▌| 215033/225000 [417:53:53<25:13:42,  9.11s/it]
 96%|█████████▌| 215034/225000 [417:54:05<27:24:25,  9.90s/it]
 96%|█████████▌| 215035/225000 [417:54:14<27:08:47,  9.81s/it]
 96%|█████████▌| 215036/225000 [417:54:21<24:28:21,  8.84s/it]
 96%|█████████▌| 215037/225000 [417:54:27<22:20:17,  8.07s/it]
 96%|█████████▌| 215038/225000 [417:54:34<21:07:00,  7.63s/it]
 96%|█████████▌| 215039/225000 [417:54:47<26:08:03,  9.45s/it]
 96%|█████████▌| 215040/225000 [417:54:54<23:44:04,  8.58s/it]
 96%|█████████▌| 215041/225000 [417:55:00<21:39:55,  7.83s/it]
 96%|█████████▌| 215042/225000 [417:55:12<25:33:58,  9.24s/it]
 96%|█████████▌| 215043/225000 [417:55:21<24:41:10,  8.93s/it]
 96%|█████████▌| 215044/225000 [417:55:28<23:32:19,  8.51s/it]
 96%|█████████▌| 215045/225000 [417:55:40<26:13:13,  9.48s/it]
 96%|█████████▌| 215046/225000 [417:55:46<23:14:51,  8.41s/it]
 96%|█████████▌| 215047/225000 [417:55:52<21:21:39,  7.73s/it]
 96%|█████████▌| 215048/225000 [417:55:59<20:34:57,  7.45s/it]
 96%|█████████▌| 215049/225000 [417:56:08<22:07:49,  8.01s/it]
 96%|█████████▌| 215050/225000 [417:56:22<26:36:21,  9.63s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.327', 'learning_rate': '2.691e-07', 'epoch': '0.9558', 'num_input_tokens_seen': 709509360, 'train_runtime': '1.505e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215050/225000 [417:56:22<26:36:21,  9.63s/it]
 96%|█████████▌| 215051/225000 [417:56:27<22:54:35,  8.29s/it]
 96%|█████████▌| 215052/225000 [417:56:36<23:50:52,  8.63s/it]
 96%|█████████▌| 215053/225000 [417:56:42<21:28:46,  7.77s/it]
 96%|█████████▌| 215054/225000 [417:56:51<22:24:03,  8.11s/it]
 96%|█████████▌| 215055/225000 [417:57:03<25:36:53,  9.27s/it]
 96%|█████████▌| 215056/225000 [417:57:10<23:49:43,  8.63s/it]
 96%|█████████▌| 215057/225000 [417:57:16<21:42:29,  7.86s/it]
 96%|█████████▌| 215058/225000 [417:57:22<19:55:03,  7.21s/it]
 96%|█████████▌| 215059/225000 [417:57:27<18:40:19,  6.76s/it]
 96%|█████████▌| 215060/225000 [417:57:35<19:31:34,  7.07s/it]
 96%|█████████▌| 215061/225000 [417:57:50<26:04:59,  9.45s/it]
 96%|█████████▌| 215062/225000 [417:57:57<23:39:37,  8.57s/it]
 96%|█████████▌| 215063/225000 [417:58:05<23:13:30,  8.41s/it]
 96%|█████████▌| 215064/225000 [417:58:12<22:15:02,  8.06s/it]
 96%|█████████▌| 215065/225000 [417:58:29<29:33:55, 10.71s/it]
 96%|█████████▌| 215066/225000 [417:58:37<27:40:59, 10.03s/it]
 96%|█████████▌| 215067/225000 [417:58:47<27:45:15, 10.06s/it]
 96%|█████████▌| 215068/225000 [417:59:01<30:56:05, 11.21s/it]
 96%|█████████▌| 215069/225000 [417:59:08<27:12:03,  9.86s/it]
 96%|█████████▌| 215070/225000 [417:59:16<25:22:22,  9.20s/it]
 96%|█████████▌| 215071/225000 [417:59:28<27:59:50, 10.15s/it]
 96%|█████████▌| 215072/225000 [417:59:34<24:46:56,  8.99s/it]
 96%|█████████▌| 215073/225000 [417:59:41<23:07:54,  8.39s/it]
 96%|█████████▌| 215074/225000 [417:59:47<20:44:56,  7.53s/it]
 96%|█████████▌| 215075/225000 [418:00:01<25:59:02,  9.42s/it]
 96%|█████████▌| 215076/225000 [418:00:09<25:12:51,  9.15s/it]
 96%|█████████▌| 215077/225000 [418:00:19<25:28:37,  9.24s/it]
 96%|█████████▌| 215078/225000 [418:00:26<23:57:19,  8.69s/it]
 96%|█████████▌| 215079/225000 [418:00:32<21:30:13,  7.80s/it]
 96%|█████████▌| 215080/225000 [418:00:43<24:27:52,  8.88s/it]
 96%|█████████▌| 215081/225000 [418:00:50<22:25:38,  8.14s/it]
 96%|█████████▌| 215082/225000 [418:00:55<20:17:57,  7.37s/it]
 96%|█████████▌| 215083/225000 [418:01:03<20:20:38,  7.39s/it]
 96%|█████████▌| 215084/225000 [418:01:16<25:30:12,  9.26s/it]
 96%|█████████▌| 215085/225000 [418:01:23<23:46:52,  8.63s/it]
 96%|█████████▌| 215086/225000 [418:01:31<23:15:38,  8.45s/it]
 96%|█████████▌| 215087/225000 [418:01:42<24:38:14,  8.95s/it]
 96%|█████████▌| 215088/225000 [418:01:50<23:57:37,  8.70s/it]
 96%|█████████▌| 215089/225000 [418:01:59<24:14:12,  8.80s/it]
 96%|█████████▌| 215090/225000 [418:02:04<21:41:55,  7.88s/it]
 96%|█████████▌| 215091/225000 [418:02:12<21:44:18,  7.90s/it]
 96%|█████████▌| 215092/225000 [418:02:22<23:04:52,  8.39s/it]
 96%|█████████▌| 215093/225000 [418:02:31<23:20:34,  8.48s/it]
 96%|█████████▌| 215094/225000 [418:02:43<26:10:20,  9.51s/it]
 96%|█████████▌| 215095/225000 [418:02:50<24:47:27,  9.01s/it]
 96%|█████████▌| 215096/225000 [418:02:57<23:09:59,  8.42s/it]
 96%|█████████▌| 215097/225000 [418:03:04<21:59:28,  7.99s/it]
 96%|█████████▌| 215098/225000 [418:03:17<25:41:36,  9.34s/it]
 96%|█████████▌| 215099/225000 [418:03:25<24:40:02,  8.97s/it]
 96%|█████████▌| 215100/225000 [418:03:35<25:19:42,  9.21s/it]
                                                              
{'loss': '1.267', 'grad_norm': '1.369', 'learning_rate': '2.665e-07', 'epoch': '0.956', 'num_input_tokens_seen': 709735992, 'train_runtime': '1.505e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215100/225000 [418:03:35<25:19:42,  9.21s/it]
 96%|█████████▌| 215101/225000 [418:03:39<21:36:11,  7.86s/it]
 96%|█████████▌| 215102/225000 [418:03:50<23:33:46,  8.57s/it]
 96%|█████████▌| 215103/225000 [418:04:00<25:12:04,  9.17s/it]
 96%|█████████▌| 215104/225000 [418:04:08<24:00:16,  8.73s/it]
 96%|█████████▌| 215105/225000 [418:04:17<23:51:16,  8.68s/it]
 96%|█████████▌| 215106/225000 [418:04:30<27:58:32, 10.18s/it]
 96%|█████████▌| 215107/225000 [418:04:37<24:56:59,  9.08s/it]
 96%|█████████▌| 215108/225000 [418:04:44<23:40:59,  8.62s/it]
 96%|█████████▌| 215109/225000 [418:04:56<26:07:53,  9.51s/it]
 96%|█████████▌| 215110/225000 [418:05:02<23:07:54,  8.42s/it]
 96%|█████████▌| 215111/225000 [418:05:13<25:37:01,  9.33s/it]
 96%|█████████▌| 215112/225000 [418:05:25<27:25:03,  9.98s/it]
 96%|█████████▌| 215113/225000 [418:05:32<24:51:09,  9.05s/it]
 96%|█████████▌| 215114/225000 [418:05:42<26:10:56,  9.53s/it]
 96%|█████████▌| 215115/225000 [418:05:48<23:29:28,  8.56s/it]
 96%|█████████▌| 215116/225000 [418:06:00<25:35:42,  9.32s/it]
 96%|█████████▌| 215117/225000 [418:06:05<22:31:40,  8.21s/it]
 96%|█████████▌| 215118/225000 [418:06:14<23:15:37,  8.47s/it]
 96%|█████████▌| 215119/225000 [418:06:26<26:06:51,  9.51s/it]
 96%|█████████▌| 215120/225000 [418:06:33<23:50:36,  8.69s/it]
 96%|█████████▌| 215121/225000 [418:06:45<26:20:05,  9.60s/it]
 96%|█████████▌| 215122/225000 [418:06:54<26:08:16,  9.53s/it]
 96%|█████████▌| 215123/225000 [418:07:00<23:14:57,  8.47s/it]
 96%|█████████▌| 215124/225000 [418:07:12<26:16:48,  9.58s/it]
 96%|█████████▌| 215125/225000 [418:07:21<25:36:11,  9.33s/it]
 96%|█████████▌| 215126/225000 [418:07:27<23:13:53,  8.47s/it]
 96%|█████████▌| 215127/225000 [418:07:37<24:09:42,  8.81s/it]
 96%|█████████▌| 215128/225000 [418:07:41<20:16:04,  7.39s/it]
 96%|█████████▌| 215129/225000 [418:07:48<19:26:05,  7.09s/it]
 96%|█████████▌| 215130/225000 [418:07:54<18:38:03,  6.80s/it]
 96%|█████████▌| 215131/225000 [418:08:06<23:06:58,  8.43s/it]
 96%|█████████▌| 215132/225000 [418:08:12<21:05:21,  7.69s/it]
 96%|█████████▌| 215133/225000 [418:08:19<20:39:27,  7.54s/it]
 96%|█████████▌| 215134/225000 [418:08:32<25:10:04,  9.18s/it]
 96%|█████████▌| 215135/225000 [418:08:38<22:42:58,  8.29s/it]
 96%|█████████▌| 215136/225000 [418:08:46<22:25:25,  8.18s/it]
 96%|█████████▌| 215137/225000 [418:08:51<19:59:37,  7.30s/it]
 96%|█████████▌| 215138/225000 [418:08:57<18:46:39,  6.85s/it]
 96%|█████████▌| 215139/225000 [418:09:15<27:36:20, 10.08s/it]
 96%|█████████▌| 215140/225000 [418:09:21<24:05:26,  8.80s/it]
 96%|█████████▌| 215141/225000 [418:09:27<21:46:48,  7.95s/it]
 96%|█████████▌| 215142/225000 [418:09:33<20:15:38,  7.40s/it]
 96%|█████████▌| 215143/225000 [418:09:49<27:56:31, 10.21s/it]
 96%|█████████▌| 215144/225000 [418:09:54<22:59:49,  8.40s/it]
 96%|█████████▌| 215145/225000 [418:10:02<23:06:43,  8.44s/it]
 96%|█████████▌| 215146/225000 [418:10:10<22:40:43,  8.29s/it]
 96%|█████████▌| 215147/225000 [418:10:21<24:45:12,  9.04s/it]
 96%|█████████▌| 215148/225000 [418:10:26<21:50:10,  7.98s/it]
 96%|█████████▌| 215149/225000 [418:10:33<21:01:17,  7.68s/it]
 96%|█████████▌| 215150/225000 [418:10:39<18:54:52,  6.91s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.358', 'learning_rate': '2.638e-07', 'epoch': '0.9562', 'num_input_tokens_seen': 709963432, 'train_runtime': '1.505e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215150/225000 [418:10:39<18:54:52,  6.91s/it]
 96%|█████████▌| 215151/225000 [418:10:52<23:59:33,  8.77s/it]
 96%|█████████▌| 215152/225000 [418:11:00<23:24:26,  8.56s/it]
 96%|█████████▌| 215153/225000 [418:11:10<25:10:08,  9.20s/it]
 96%|█████████▌| 215154/225000 [418:11:19<25:02:07,  9.15s/it]
 96%|█████████▌| 215155/225000 [418:11:26<22:47:29,  8.33s/it]
 96%|█████████▌| 215156/225000 [418:11:31<20:18:35,  7.43s/it]
 96%|█████████▌| 215157/225000 [418:11:44<24:44:38,  9.05s/it]
 96%|█████████▌| 215158/225000 [418:11:49<21:05:39,  7.72s/it]
 96%|█████████▌| 215159/225000 [418:11:59<23:24:28,  8.56s/it]
 96%|█████████▌| 215160/225000 [418:12:05<21:32:59,  7.88s/it]
 96%|█████████▌| 215161/225000 [418:12:16<23:41:59,  8.67s/it]
 96%|█████████▌| 215162/225000 [418:12:22<21:40:15,  7.93s/it]
 96%|█████████▌| 215163/225000 [418:12:30<21:31:07,  7.88s/it]
 96%|█████████▌| 215164/225000 [418:12:43<25:22:33,  9.29s/it]
 96%|█████████▌| 215165/225000 [418:12:52<25:39:15,  9.39s/it]
 96%|█████████▌| 215166/225000 [418:12:59<23:15:29,  8.51s/it]
 96%|█████████▌| 215167/225000 [418:13:08<23:48:50,  8.72s/it]
 96%|█████████▌| 215168/225000 [418:13:14<22:01:03,  8.06s/it]
 96%|█████████▌| 215169/225000 [418:13:30<27:53:13, 10.21s/it]
 96%|█████████▌| 215170/225000 [418:13:36<25:02:26,  9.17s/it]
 96%|█████████▌| 215171/225000 [418:13:50<28:22:07, 10.39s/it]
 96%|█████████▌| 215172/225000 [418:13:57<25:46:46,  9.44s/it]
 96%|█████████▌| 215173/225000 [418:14:04<23:46:07,  8.71s/it]
 96%|█████████▌| 215174/225000 [418:14:19<29:30:04, 10.81s/it]
 96%|█████████▌| 215175/225000 [418:14:26<26:19:09,  9.64s/it]
 96%|█████████▌| 215176/225000 [418:14:32<23:10:25,  8.49s/it]
 96%|█████████▌| 215177/225000 [418:14:45<26:53:05,  9.85s/it]
 96%|█████████▌| 215178/225000 [418:14:57<28:04:48, 10.29s/it]
 96%|█████████▌| 215179/225000 [418:15:05<26:55:54,  9.87s/it]
 96%|█████████▌| 215180/225000 [418:15:11<23:27:47,  8.60s/it]
 96%|█████████▌| 215181/225000 [418:15:23<26:04:45,  9.56s/it]
 96%|█████████▌| 215182/225000 [418:15:33<26:24:48,  9.69s/it]
 96%|█████████▌| 215183/225000 [418:15:38<22:16:45,  8.17s/it]
 96%|█████████▌| 215184/225000 [418:15:46<22:19:35,  8.19s/it]
 96%|█████████▌| 215185/225000 [418:15:52<20:21:28,  7.47s/it]
 96%|█████████▌| 215186/225000 [418:15:58<19:32:09,  7.17s/it]
 96%|█████████▌| 215187/225000 [418:16:11<24:15:52,  8.90s/it]
 96%|█████████▌| 215188/225000 [418:16:17<22:10:27,  8.14s/it]
 96%|█████████▌| 215189/225000 [418:16:31<26:28:00,  9.71s/it]
 96%|█████████▌| 215190/225000 [418:16:41<26:59:42,  9.91s/it]
 96%|█████████▌| 215191/225000 [418:16:46<23:19:58,  8.56s/it]
 96%|█████████▌| 215192/225000 [418:16:56<24:01:30,  8.82s/it]
 96%|█████████▌| 215193/225000 [418:17:02<22:11:43,  8.15s/it]
 96%|█████████▌| 215194/225000 [418:17:16<26:20:10,  9.67s/it]
 96%|█████████▌| 215195/225000 [418:17:21<22:45:57,  8.36s/it]
 96%|█████████▌| 215196/225000 [418:17:33<25:56:28,  9.53s/it]
 96%|█████████▌| 215197/225000 [418:17:41<24:26:32,  8.98s/it]
 96%|█████████▌| 215198/225000 [418:17:48<22:34:29,  8.29s/it]
 96%|█████████▌| 215199/225000 [418:17:58<24:23:41,  8.96s/it]
 96%|█████████▌| 215200/225000 [418:18:06<23:23:24,  8.59s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.327', 'learning_rate': '2.612e-07', 'epoch': '0.9564', 'num_input_tokens_seen': 710196864, 'train_runtime': '1.506e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215200/225000 [418:18:06<23:23:24,  8.59s/it]
 96%|█████████▌| 215201/225000 [418:18:16<24:20:06,  8.94s/it]
 96%|█████████▌| 215202/225000 [418:18:24<23:37:36,  8.68s/it]
 96%|█████████▌| 215203/225000 [418:18:34<24:34:19,  9.03s/it]
 96%|█████████▌| 215204/225000 [418:18:39<21:50:27,  8.03s/it]
 96%|█████████▌| 215205/225000 [418:18:45<19:50:56,  7.30s/it]
 96%|█████████▌| 215206/225000 [418:18:56<23:10:27,  8.52s/it]
 96%|█████████▌| 215207/225000 [418:19:05<23:33:41,  8.66s/it]
 96%|█████████▌| 215208/225000 [418:19:15<24:19:50,  8.95s/it]
 96%|█████████▌| 215209/225000 [418:19:27<26:57:58,  9.92s/it]
 96%|█████████▌| 215210/225000 [418:19:33<23:59:03,  8.82s/it]
 96%|█████████▌| 215211/225000 [418:19:44<25:50:55,  9.51s/it]
 96%|█████████▌| 215212/225000 [418:19:54<25:55:02,  9.53s/it]
 96%|█████████▌| 215213/225000 [418:20:05<27:18:35, 10.05s/it]
 96%|█████████▌| 215214/225000 [418:20:12<24:38:42,  9.07s/it]
 96%|█████████▌| 215215/225000 [418:20:19<23:17:39,  8.57s/it]
 96%|█████████▌| 215216/225000 [418:20:31<26:03:03,  9.59s/it]
 96%|█████████▌| 215217/225000 [418:20:36<21:41:02,  7.98s/it]
 96%|█████████▌| 215218/225000 [418:20:46<23:31:14,  8.66s/it]
 96%|█████████▌| 215219/225000 [418:20:52<21:45:55,  8.01s/it]
 96%|█████████▌| 215220/225000 [418:21:03<23:39:07,  8.71s/it]
 96%|█████████▌| 215221/225000 [418:21:18<29:26:27, 10.84s/it]
 96%|█████████▌| 215222/225000 [418:21:25<25:36:51,  9.43s/it]
 96%|█████████▌| 215223/225000 [418:21:33<24:53:47,  9.17s/it]
 96%|█████████▌| 215224/225000 [418:21:39<22:35:44,  8.32s/it]
 96%|█████████▌| 215225/225000 [418:21:48<22:54:03,  8.43s/it]
 96%|█████████▌| 215226/225000 [418:21:54<21:02:00,  7.75s/it]
 96%|█████████▌| 215227/225000 [418:22:01<19:46:01,  7.28s/it]
 96%|█████████▌| 215228/225000 [418:22:15<25:41:34,  9.47s/it]
 96%|█████████▌| 215229/225000 [418:22:25<26:22:52,  9.72s/it]
 96%|█████████▌| 215230/225000 [418:22:33<24:43:17,  9.11s/it]
 96%|█████████▌| 215231/225000 [418:22:44<25:50:13,  9.52s/it]
 96%|█████████▌| 215232/225000 [418:22:54<26:49:53,  9.89s/it]
 96%|█████████▌| 215233/225000 [418:23:02<25:15:58,  9.31s/it]
 96%|█████████▌| 215234/225000 [418:23:14<27:06:28,  9.99s/it]
 96%|█████████▌| 215235/225000 [418:23:24<27:09:21, 10.01s/it]
 96%|█████████▌| 215236/225000 [418:23:34<27:30:23, 10.14s/it]
 96%|█████████▌| 215237/225000 [418:23:41<24:31:19,  9.04s/it]
 96%|█████████▌| 215238/225000 [418:23:49<23:49:58,  8.79s/it]
 96%|█████████▌| 215239/225000 [418:23:55<21:24:15,  7.89s/it]
 96%|█████████▌| 215240/225000 [418:24:04<22:14:06,  8.20s/it]
 96%|█████████▌| 215241/225000 [418:24:16<25:33:17,  9.43s/it]
 96%|█████████▌| 215242/225000 [418:24:26<25:54:36,  9.56s/it]
 96%|█████████▌| 215243/225000 [418:24:37<27:03:53,  9.99s/it]
 96%|█████████▌| 215244/225000 [418:24:44<24:28:32,  9.03s/it]
 96%|█████████▌| 215245/225000 [418:24:48<20:36:00,  7.60s/it]
 96%|█████████▌| 215246/225000 [418:24:55<19:57:17,  7.36s/it]
 96%|█████████▌| 215247/225000 [418:25:08<25:05:45,  9.26s/it]
 96%|█████████▌| 215248/225000 [418:25:14<21:49:55,  8.06s/it]
 96%|█████████▌| 215249/225000 [418:25:20<20:16:44,  7.49s/it]
 96%|█████████▌| 215250/225000 [418:25:33<24:51:35,  9.18s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.258', 'learning_rate': '2.585e-07', 'epoch': '0.9567', 'num_input_tokens_seen': 710426440, 'train_runtime': '1.506e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215250/225000 [418:25:33<24:51:35,  9.18s/it]
 96%|█████████▌| 215251/225000 [418:25:42<24:34:11,  9.07s/it]
 96%|█████████▌| 215252/225000 [418:25:53<26:17:06,  9.71s/it]
 96%|█████████▌| 215253/225000 [418:25:59<23:37:14,  8.72s/it]
 96%|█████████▌| 215254/225000 [418:26:07<23:00:46,  8.50s/it]
 96%|█████████▌| 215255/225000 [418:26:20<26:40:05,  9.85s/it]
 96%|█████████▌| 215256/225000 [418:26:28<24:34:18,  9.08s/it]
 96%|█████████▌| 215257/225000 [418:26:39<26:18:52,  9.72s/it]
 96%|█████████▌| 215258/225000 [418:26:54<30:34:54, 11.30s/it]
 96%|█████████▌| 215259/225000 [418:27:08<32:29:28, 12.01s/it]
 96%|█████████▌| 215260/225000 [418:27:14<28:15:30, 10.44s/it]
 96%|█████████▌| 215261/225000 [418:27:21<25:29:25,  9.42s/it]
 96%|█████████▌| 215262/225000 [418:27:30<24:52:41,  9.20s/it]
 96%|█████████▌| 215263/225000 [418:27:36<22:04:38,  8.16s/it]
 96%|█████████▌| 215264/225000 [418:27:41<19:49:03,  7.33s/it]
 96%|█████████▌| 215265/225000 [418:27:49<19:48:49,  7.33s/it]
 96%|█████████▌| 215266/225000 [418:27:56<19:48:26,  7.33s/it]
 96%|█████████▌| 215267/225000 [418:28:03<19:17:07,  7.13s/it]
 96%|█████████▌| 215268/225000 [418:28:16<24:40:23,  9.13s/it]
 96%|█████████▌| 215269/225000 [418:28:22<22:04:07,  8.16s/it]
 96%|█████████▌| 215270/225000 [418:28:31<22:24:59,  8.29s/it]
 96%|█████████▌| 215271/225000 [418:28:40<23:27:59,  8.68s/it]
 96%|█████████▌| 215272/225000 [418:28:45<20:16:00,  7.50s/it]
 96%|█████████▌| 215273/225000 [418:28:52<20:06:38,  7.44s/it]
 96%|█████████▌| 215274/225000 [418:29:05<24:08:56,  8.94s/it]
 96%|█████████▌| 215275/225000 [418:29:15<25:00:03,  9.25s/it]
 96%|█████████▌| 215276/225000 [418:29:32<31:00:12, 11.48s/it]
 96%|█████████▌| 215277/225000 [418:29:43<30:44:54, 11.38s/it]
 96%|█████████▌| 215278/225000 [418:29:53<29:46:38, 11.03s/it]
 96%|█████████▌| 215279/225000 [418:30:01<27:28:56, 10.18s/it]
 96%|█████████▌| 215280/225000 [418:30:08<25:02:55,  9.28s/it]
 96%|█████████▌| 215281/225000 [418:30:16<23:41:54,  8.78s/it]
 96%|█████████▌| 215282/225000 [418:30:26<24:25:53,  9.05s/it]
 96%|█████████▌| 215283/225000 [418:30:39<28:19:56, 10.50s/it]
 96%|█████████▌| 215284/225000 [418:30:51<29:09:27, 10.80s/it]
 96%|█████████▌| 215285/225000 [418:30:57<24:53:27,  9.22s/it]
 96%|█████████▌| 215286/225000 [418:31:07<25:30:25,  9.45s/it]
 96%|█████████▌| 215287/225000 [418:31:12<22:42:24,  8.42s/it]
 96%|█████████▌| 215288/225000 [418:31:24<24:50:22,  9.21s/it]
 96%|█████████▌| 215289/225000 [418:31:29<21:57:43,  8.14s/it]
 96%|█████████▌| 215290/225000 [418:31:38<22:11:54,  8.23s/it]
 96%|█████████▌| 215291/225000 [418:31:45<21:53:24,  8.12s/it]
 96%|█████████▌| 215292/225000 [418:31:54<22:05:18,  8.19s/it]
 96%|█████████▌| 215293/225000 [418:32:02<22:18:27,  8.27s/it]
 96%|█████████▌| 215294/225000 [418:32:10<22:13:03,  8.24s/it]
 96%|█████████▌| 215295/225000 [418:32:18<21:13:17,  7.87s/it]
 96%|█████████▌| 215296/225000 [418:32:24<19:42:20,  7.31s/it]
 96%|█████████▌| 215297/225000 [418:32:35<23:15:16,  8.63s/it]
 96%|█████████▌| 215298/225000 [418:32:48<26:28:10,  9.82s/it]
 96%|█████████▌| 215299/225000 [418:32:57<25:33:23,  9.48s/it]
 96%|█████████▌| 215300/225000 [418:33:06<25:17:03,  9.38s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.091', 'learning_rate': '2.559e-07', 'epoch': '0.9569', 'num_input_tokens_seen': 710660480, 'train_runtime': '1.507e+06', 'train_tokens_per_second': '471.6'}

 96%|█████████▌| 215300/225000 [418:33:06<25:17:03,  9.38s/it]
 96%|█████████▌| 215301/225000 [418:33:11<21:46:55,  8.08s/it]
 96%|█████████▌| 215302/225000 [418:33:23<25:28:34,  9.46s/it]
 96%|█████████▌| 215303/225000 [418:33:31<23:49:03,  8.84s/it]
 96%|█████████▌| 215304/225000 [418:33:45<28:20:43, 10.52s/it]
 96%|█████████▌| 215305/225000 [418:33:52<25:31:21,  9.48s/it]
 96%|█████████▌| 215306/225000 [418:34:06<28:43:45, 10.67s/it]
 96%|█████████▌| 215307/225000 [418:34:14<27:00:25, 10.03s/it]
 96%|█████████▌| 215308/225000 [418:34:23<25:39:31,  9.53s/it]
 96%|█████████▌| 215309/225000 [418:34:29<22:48:13,  8.47s/it]
 96%|█████████▌| 215310/225000 [418:34:36<21:58:32,  8.16s/it]
 96%|█████████▌| 215311/225000 [418:34:43<20:49:24,  7.74s/it]
 96%|█████████▌| 215312/225000 [418:34:54<23:55:24,  8.89s/it]
 96%|█████████▌| 215313/225000 [418:35:01<22:16:12,  8.28s/it]
 96%|█████████▌| 215314/225000 [418:35:08<20:47:23,  7.73s/it]
 96%|█████████▌| 215315/225000 [418:35:17<21:52:01,  8.13s/it]
 96%|█████████▌| 215316/225000 [418:35:34<28:59:45, 10.78s/it]
 96%|█████████▌| 215317/225000 [418:35:39<24:42:10,  9.18s/it]
 96%|█████████▌| 215318/225000 [418:35:49<24:54:51,  9.26s/it]
 96%|█████████▌| 215319/225000 [418:35:57<24:19:30,  9.05s/it]
 96%|█████████▌| 215320/225000 [418:36:12<28:48:05, 10.71s/it]
 96%|█████████▌| 215321/225000 [418:36:19<25:52:24,  9.62s/it]
 96%|█████████▌| 215322/225000 [418:36:25<23:09:50,  8.62s/it]
 96%|█████████▌| 215323/225000 [418:36:32<21:33:35,  8.02s/it]
 96%|█████████▌| 215324/225000 [418:36:44<25:01:15,  9.31s/it]
 96%|█████████▌| 215325/225000 [418:36:48<20:36:20,  7.67s/it]
 96%|█████████▌| 215326/225000 [418:36:58<22:37:20,  8.42s/it]
 96%|█████████▌| 215327/225000 [418:37:09<24:19:39,  9.05s/it]
 96%|█████████▌| 215328/225000 [418:37:18<24:52:54,  9.26s/it]
 96%|█████████▌| 215329/225000 [418:37:31<27:51:55, 10.37s/it]
 96%|█████████▌| 215330/225000 [418:37:45<30:18:34, 11.28s/it]
 96%|█████████▌| 215331/225000 [418:37:54<28:51:26, 10.74s/it]
 96%|█████████▌| 215332/225000 [418:38:00<25:16:15,  9.41s/it]
 96%|█████████▌| 215333/225000 [418:38:07<22:41:46,  8.45s/it]
 96%|█████████▌| 215334/225000 [418:38:13<21:18:00,  7.93s/it]
 96%|█████████▌| 215335/225000 [418:38:25<24:18:42,  9.06s/it]
 96%|█████████▌| 215336/225000 [418:38:37<26:13:27,  9.77s/it]
 96%|█████████▌| 215337/225000 [418:38:42<22:41:10,  8.45s/it]
 96%|█████████▌| 215338/225000 [418:38:48<20:35:50,  7.67s/it]
 96%|█████████▌| 215339/225000 [418:39:00<24:22:08,  9.08s/it]
 96%|█████████▌| 215340/225000 [418:39:06<21:37:02,  8.06s/it]
 96%|█████████▌| 215341/225000 [418:39:15<22:09:49,  8.26s/it]
 96%|█████████▌| 215342/225000 [418:39:23<22:19:31,  8.32s/it]
 96%|█████████▌| 215343/225000 [418:39:32<22:58:27,  8.56s/it]
 96%|█████████▌| 215344/225000 [418:39:39<21:51:44,  8.15s/it]
 96%|█████████▌| 215345/225000 [418:39:54<26:59:41, 10.07s/it]
 96%|█████████▌| 215346/225000 [418:40:00<23:45:26,  8.86s/it]
 96%|█████████▌| 215347/225000 [418:40:09<24:14:43,  9.04s/it]
 96%|█████████▌| 215348/225000 [418:40:20<25:30:00,  9.51s/it]
 96%|█████████▌| 215349/225000 [418:40:26<22:36:46,  8.44s/it]
 96%|█████████▌| 215350/225000 [418:40:32<20:42:10,  7.72s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.259', 'learning_rate': '2.533e-07', 'epoch': '0.9571', 'num_input_tokens_seen': 710901504, 'train_runtime': '1.507e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215350/225000 [418:40:32<20:42:10,  7.72s/it]
 96%|█████████▌| 215351/225000 [418:40:41<21:47:25,  8.13s/it]
 96%|█████████▌| 215352/225000 [418:40:52<23:54:17,  8.92s/it]
 96%|█████████▌| 215353/225000 [418:40:59<22:30:04,  8.40s/it]
 96%|█████████▌| 215354/225000 [418:41:09<23:44:36,  8.86s/it]
 96%|█████████▌| 215355/225000 [418:41:22<27:19:05, 10.20s/it]
 96%|█████████▌| 215356/225000 [418:41:28<23:36:41,  8.81s/it]
 96%|█████████▌| 215357/225000 [418:41:40<26:29:11,  9.89s/it]
 96%|█████████▌| 215358/225000 [418:41:46<23:24:16,  8.74s/it]
 96%|█████████▌| 215359/225000 [418:41:56<23:53:40,  8.92s/it]
 96%|█████████▌| 215360/225000 [418:42:01<20:46:42,  7.76s/it]
 96%|█████████▌| 215361/225000 [418:42:12<23:57:44,  8.95s/it]
 96%|█████████▌| 215362/225000 [418:42:23<25:30:07,  9.53s/it]
 96%|█████████▌| 215363/225000 [418:42:30<23:26:37,  8.76s/it]
 96%|█████████▌| 215364/225000 [418:42:38<22:24:14,  8.37s/it]
 96%|█████████▌| 215365/225000 [418:42:47<23:11:57,  8.67s/it]
 96%|█████████▌| 215366/225000 [418:42:58<24:47:54,  9.27s/it]
 96%|█████████▌| 215367/225000 [418:43:06<24:19:27,  9.09s/it]
 96%|█████████▌| 215368/225000 [418:43:18<26:08:10,  9.77s/it]
 96%|█████████▌| 215369/225000 [418:43:23<22:37:28,  8.46s/it]
 96%|█████████▌| 215370/225000 [418:43:29<20:48:02,  7.78s/it]
 96%|█████████▌| 215371/225000 [418:43:36<20:09:44,  7.54s/it]
 96%|█████████▌| 215372/225000 [418:43:43<19:19:53,  7.23s/it]
 96%|█████████▌| 215373/225000 [418:43:59<26:17:51,  9.83s/it]
 96%|█████████▌| 215374/225000 [418:44:05<23:04:39,  8.63s/it]
 96%|█████████▌| 215375/225000 [418:44:15<24:21:13,  9.11s/it]
 96%|█████████▌| 215376/225000 [418:44:28<27:25:40, 10.26s/it]
 96%|█████████▌| 215377/225000 [418:44:37<26:30:54,  9.92s/it]
 96%|█████████▌| 215378/225000 [418:44:42<22:23:17,  8.38s/it]
 96%|█████████▌| 215379/225000 [418:44:47<19:53:44,  7.44s/it]
 96%|█████████▌| 215380/225000 [418:44:54<19:28:58,  7.29s/it]
 96%|█████████▌| 215381/225000 [418:45:07<23:47:29,  8.90s/it]
 96%|█████████▌| 215382/225000 [418:45:14<22:54:17,  8.57s/it]
 96%|█████████▌| 215383/225000 [418:45:20<20:55:04,  7.83s/it]
 96%|█████████▌| 215384/225000 [418:45:27<19:48:58,  7.42s/it]
 96%|█████████▌| 215385/225000 [418:45:40<24:27:05,  9.16s/it]
 96%|█████████▌| 215386/225000 [418:45:49<23:56:01,  8.96s/it]
 96%|█████████▌| 215387/225000 [418:45:54<21:27:36,  8.04s/it]
 96%|█████████▌| 215388/225000 [418:46:08<25:41:42,  9.62s/it]
 96%|█████████▌| 215389/225000 [418:46:18<26:26:45,  9.91s/it]
 96%|█████████▌| 215390/225000 [418:46:27<25:26:04,  9.53s/it]
 96%|█████████▌| 215391/225000 [418:46:37<25:40:35,  9.62s/it]
 96%|█████████▌| 215392/225000 [418:46:49<27:55:57, 10.47s/it]
 96%|█████████▌| 215393/225000 [418:46:55<24:30:30,  9.18s/it]
 96%|█████████▌| 215394/225000 [418:47:07<26:02:20,  9.76s/it]
 96%|█████████▌| 215395/225000 [418:47:18<27:07:11, 10.16s/it]
 96%|█████████▌| 215396/225000 [418:47:27<26:42:02, 10.01s/it]
 96%|█████████▌| 215397/225000 [418:47:33<23:19:29,  8.74s/it]
 96%|█████████▌| 215398/225000 [418:47:38<20:26:16,  7.66s/it]
 96%|█████████▌| 215399/225000 [418:47:48<22:23:30,  8.40s/it]
 96%|█████████▌| 215400/225000 [418:47:54<20:14:15,  7.59s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.352', 'learning_rate': '2.507e-07', 'epoch': '0.9573', 'num_input_tokens_seen': 711131128, 'train_runtime': '1.508e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215400/225000 [418:47:54<20:14:15,  7.59s/it]
 96%|█████████▌| 215401/225000 [418:47:59<18:08:34,  6.80s/it]
 96%|█████████▌| 215402/225000 [418:48:11<22:35:54,  8.48s/it]
 96%|█████████▌| 215403/225000 [418:48:19<22:13:30,  8.34s/it]
 96%|█████████▌| 215404/225000 [418:48:26<20:30:26,  7.69s/it]
 96%|█████████▌| 215405/225000 [418:48:31<18:53:31,  7.09s/it]
 96%|█████████▌| 215406/225000 [418:48:44<23:32:24,  8.83s/it]
 96%|█████████▌| 215407/225000 [418:48:54<24:21:53,  9.14s/it]
 96%|█████████▌| 215408/225000 [418:49:04<25:03:39,  9.41s/it]
 96%|█████████▌| 215409/225000 [418:49:14<25:22:25,  9.52s/it]
 96%|█████████▌| 215410/225000 [418:49:26<27:38:40, 10.38s/it]
 96%|█████████▌| 215411/225000 [418:49:32<23:37:35,  8.87s/it]
 96%|█████████▌| 215412/225000 [418:49:42<25:02:13,  9.40s/it]
 96%|█████████▌| 215413/225000 [418:49:53<26:14:57,  9.86s/it]
 96%|█████████▌| 215414/225000 [418:50:00<23:46:13,  8.93s/it]
 96%|█████████▌| 215415/225000 [418:50:06<21:43:45,  8.16s/it]
 96%|█████████▌| 215416/225000 [418:50:19<24:59:33,  9.39s/it]
 96%|█████████▌| 215417/225000 [418:50:24<21:58:03,  8.25s/it]
 96%|█████████▌| 215418/225000 [418:50:31<20:58:55,  7.88s/it]
 96%|█████████▌| 215419/225000 [418:50:38<19:53:22,  7.47s/it]
 96%|█████████▌| 215420/225000 [418:50:52<25:18:40,  9.51s/it]
 96%|█████████▌| 215421/225000 [418:50:59<23:39:37,  8.89s/it]
 96%|█████████▌| 215422/225000 [418:51:14<28:17:15, 10.63s/it]
 96%|█████████▌| 215423/225000 [418:51:24<27:39:52, 10.40s/it]
 96%|█████████▌| 215424/225000 [418:51:30<23:53:57,  8.98s/it]
 96%|█████████▌| 215425/225000 [418:51:36<21:44:45,  8.18s/it]
 96%|█████████▌| 215426/225000 [418:51:42<20:11:40,  7.59s/it]
 96%|█████████▌| 215427/225000 [418:51:50<20:08:02,  7.57s/it]
 96%|█████████▌| 215428/225000 [418:52:06<27:18:01, 10.27s/it]
 96%|█████████▌| 215429/225000 [418:52:14<25:13:33,  9.49s/it]
 96%|█████████▌| 215430/225000 [418:52:22<23:59:37,  9.03s/it]
 96%|█████████▌| 215431/225000 [418:52:29<22:25:56,  8.44s/it]
 96%|█████████▌| 215432/225000 [418:52:36<21:39:50,  8.15s/it]
 96%|█████████▌| 215433/225000 [418:52:50<25:59:31,  9.78s/it]
 96%|█████████▌| 215434/225000 [418:52:58<24:31:18,  9.23s/it]
 96%|█████████▌| 215435/225000 [418:53:04<22:22:08,  8.42s/it]
 96%|█████████▌| 215436/225000 [418:53:15<23:47:42,  8.96s/it]
 96%|█████████▌| 215437/225000 [418:53:20<21:16:20,  8.01s/it]
 96%|█████████▌| 215438/225000 [418:53:27<19:54:10,  7.49s/it]
 96%|█████████▌| 215439/225000 [418:53:38<23:15:29,  8.76s/it]
 96%|█████████▌| 215440/225000 [418:53:46<22:20:56,  8.42s/it]
 96%|█████████▌| 215441/225000 [418:53:54<21:52:29,  8.24s/it]
 96%|█████████▌| 215442/225000 [418:54:02<21:51:33,  8.23s/it]
 96%|█████████▌| 215443/225000 [418:54:16<26:14:02,  9.88s/it]
 96%|█████████▌| 215444/225000 [418:54:22<23:20:52,  8.80s/it]
 96%|█████████▌| 215445/225000 [418:54:29<22:06:47,  8.33s/it]
 96%|█████████▌| 215446/225000 [418:54:39<23:24:28,  8.82s/it]
 96%|█████████▌| 215447/225000 [418:54:47<22:16:05,  8.39s/it]
 96%|█████████▌| 215448/225000 [418:54:58<24:15:36,  9.14s/it]
 96%|█████████▌| 215449/225000 [418:55:08<25:14:01,  9.51s/it]
 96%|█████████▌| 215450/225000 [418:55:14<22:34:40,  8.51s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.256', 'learning_rate': '2.482e-07', 'epoch': '0.9576', 'num_input_tokens_seen': 711358872, 'train_runtime': '1.508e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215450/225000 [418:55:14<22:34:40,  8.51s/it]
 96%|█████████▌| 215451/225000 [418:55:24<23:22:42,  8.81s/it]
 96%|█████████▌| 215452/225000 [418:55:35<25:10:26,  9.49s/it]
 96%|█████████▌| 215453/225000 [418:55:42<23:28:39,  8.85s/it]
 96%|█████████▌| 215454/225000 [418:55:54<25:31:19,  9.62s/it]
 96%|█████████▌| 215455/225000 [418:56:01<23:59:43,  9.05s/it]
 96%|█████████▌| 215456/225000 [418:56:14<26:56:28, 10.16s/it]
 96%|█████████▌| 215457/225000 [418:56:23<25:55:42,  9.78s/it]
 96%|█████████▌| 215458/225000 [418:56:36<28:13:14, 10.65s/it]
 96%|█████████▌| 215459/225000 [418:56:43<25:42:50,  9.70s/it]
 96%|█████████▌| 215460/225000 [418:56:50<23:40:39,  8.93s/it]
 96%|█████████▌| 215461/225000 [418:57:02<25:56:48,  9.79s/it]
 96%|█████████▌| 215462/225000 [418:57:11<25:05:59,  9.47s/it]
 96%|█████████▌| 215463/225000 [418:57:20<25:13:16,  9.52s/it]
 96%|█████████▌| 215464/225000 [418:57:26<21:44:19,  8.21s/it]
 96%|█████████▌| 215465/225000 [418:57:36<23:53:31,  9.02s/it]
 96%|█████████▌| 215466/225000 [418:57:43<22:13:39,  8.39s/it]
 96%|█████████▌| 215467/225000 [418:57:49<19:58:51,  7.55s/it]
 96%|█████████▌| 215468/225000 [418:57:55<18:54:55,  7.14s/it]
 96%|█████████▌| 215469/225000 [418:58:08<23:32:22,  8.89s/it]
 96%|█████████▌| 215470/225000 [418:58:17<23:08:43,  8.74s/it]
 96%|█████████▌| 215471/225000 [418:58:23<21:06:02,  7.97s/it]
 96%|█████████▌| 215472/225000 [418:58:28<19:02:46,  7.20s/it]
 96%|█████████▌| 215473/225000 [418:58:44<26:01:02,  9.83s/it]
 96%|█████████▌| 215474/225000 [418:58:53<25:02:51,  9.47s/it]
 96%|█████████▌| 215475/225000 [418:59:01<23:50:43,  9.01s/it]
 96%|█████████▌| 215476/225000 [418:59:11<25:12:34,  9.53s/it]
 96%|█████████▌| 215477/225000 [418:59:21<25:22:54,  9.60s/it]
 96%|█████████▌| 215478/225000 [418:59:30<25:11:34,  9.52s/it]
 96%|█████████▌| 215479/225000 [418:59:39<24:36:18,  9.30s/it]
 96%|█████████▌| 215480/225000 [418:59:48<24:12:48,  9.16s/it]
 96%|█████████▌| 215481/225000 [418:59:56<23:35:48,  8.92s/it]
 96%|█████████▌| 215482/225000 [419:00:06<24:20:13,  9.20s/it]
 96%|█████████▌| 215483/225000 [419:00:16<24:56:07,  9.43s/it]
 96%|█████████▌| 215484/225000 [419:00:28<26:35:33, 10.06s/it]
 96%|█████████▌| 215485/225000 [419:00:34<23:35:00,  8.92s/it]
 96%|█████████▌| 215486/225000 [419:00:39<20:32:16,  7.77s/it]
 96%|█████████▌| 215487/225000 [419:00:54<26:09:00,  9.90s/it]
 96%|█████████▌| 215488/225000 [419:01:06<27:29:44, 10.41s/it]
 96%|█████████▌| 215489/225000 [419:01:16<27:52:08, 10.55s/it]
 96%|█████████▌| 215490/225000 [419:01:24<25:45:36,  9.75s/it]
 96%|█████████▌| 215491/225000 [419:01:33<25:01:32,  9.47s/it]
 96%|█████████▌| 215492/225000 [419:01:44<25:48:24,  9.77s/it]
 96%|█████████▌| 215493/225000 [419:01:54<26:30:47, 10.04s/it]
 96%|█████████▌| 215494/225000 [419:02:03<25:45:05,  9.75s/it]
 96%|█████████▌| 215495/225000 [419:02:07<21:14:27,  8.05s/it]
 96%|█████████▌| 215496/225000 [419:02:17<22:08:18,  8.39s/it]
 96%|█████████▌| 215497/225000 [419:02:22<19:58:02,  7.56s/it]
 96%|█████████▌| 215498/225000 [419:02:32<21:19:04,  8.08s/it]
 96%|█████████▌| 215499/225000 [419:02:38<19:43:54,  7.48s/it]
 96%|█████████▌| 215500/225000 [419:02:45<19:44:25,  7.48s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.442', 'learning_rate': '2.456e-07', 'epoch': '0.9578', 'num_input_tokens_seen': 711590160, 'train_runtime': '1.509e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215500/225000 [419:02:45<19:44:25,  7.48s/it]
 96%|█████████▌| 215501/225000 [419:02:54<20:34:13,  7.80s/it]
 96%|█████████▌| 215502/225000 [419:03:08<25:50:45,  9.80s/it]
 96%|█████████▌| 215503/225000 [419:03:18<25:43:28,  9.75s/it]
 96%|█████████▌| 215504/225000 [419:03:26<24:41:29,  9.36s/it]
 96%|█████████▌| 215505/225000 [419:03:39<27:11:33, 10.31s/it]
 96%|█████████▌| 215506/225000 [419:03:45<24:10:13,  9.17s/it]
 96%|█████████▌| 215507/225000 [419:03:59<27:51:54, 10.57s/it]
 96%|█████████▌| 215508/225000 [419:04:07<25:35:40,  9.71s/it]
 96%|█████████▌| 215509/225000 [419:04:18<26:30:22, 10.05s/it]
 96%|█████████▌| 215510/225000 [419:04:26<25:27:50,  9.66s/it]
 96%|█████████▌| 215511/225000 [419:04:32<22:04:33,  8.38s/it]
 96%|█████████▌| 215512/225000 [419:04:39<21:10:24,  8.03s/it]
 96%|█████████▌| 215513/225000 [419:04:45<19:25:35,  7.37s/it]
 96%|█████████▌| 215514/225000 [419:04:59<25:09:05,  9.55s/it]
 96%|█████████▌| 215515/225000 [419:05:06<22:27:42,  8.53s/it]
 96%|█████████▌| 215516/225000 [419:05:12<21:04:02,  8.00s/it]
 96%|█████████▌| 215517/225000 [419:05:18<19:35:45,  7.44s/it]
 96%|█████████▌| 215518/225000 [419:05:33<25:25:13,  9.65s/it]
 96%|█████████▌| 215519/225000 [419:05:42<24:38:40,  9.36s/it]
 96%|█████████▌| 215520/225000 [419:05:53<26:00:36,  9.88s/it]
 96%|█████████▌| 215521/225000 [419:05:58<21:43:17,  8.25s/it]
 96%|█████████▌| 215522/225000 [419:06:10<25:07:51,  9.55s/it]
 96%|█████████▌| 215523/225000 [419:06:21<26:16:00,  9.98s/it]
 96%|█████████▌| 215524/225000 [419:06:27<22:42:12,  8.63s/it]
 96%|█████████▌| 215525/225000 [419:06:41<27:42:13, 10.53s/it]
 96%|█████████▌| 215526/225000 [419:06:48<24:46:02,  9.41s/it]
 96%|█████████▌| 215527/225000 [419:06:59<26:00:35,  9.88s/it]
 96%|█████████▌| 215528/225000 [419:07:06<23:09:51,  8.80s/it]
 96%|█████████▌| 215529/225000 [419:07:19<27:05:05, 10.30s/it]
 96%|█████████▌| 215530/225000 [419:07:26<23:54:58,  9.09s/it]
 96%|█████████▌| 215531/225000 [419:07:31<21:03:27,  8.01s/it]
 96%|█████████▌| 215532/225000 [419:07:45<25:29:09,  9.69s/it]
 96%|█████████▌| 215533/225000 [419:07:49<21:26:09,  8.15s/it]
 96%|█████████▌| 215534/225000 [419:08:00<23:06:42,  8.79s/it]
 96%|█████████▌| 215535/225000 [419:08:08<22:52:46,  8.70s/it]
 96%|█████████▌| 215536/225000 [419:08:17<22:53:16,  8.71s/it]
 96%|█████████▌| 215537/225000 [419:08:25<22:27:19,  8.54s/it]
 96%|█████████▌| 215538/225000 [419:08:32<21:26:13,  8.16s/it]
 96%|█████████▌| 215539/225000 [419:08:38<19:46:43,  7.53s/it]
 96%|█████████▌| 215540/225000 [419:08:46<20:15:38,  7.71s/it]
 96%|█████████▌| 215541/225000 [419:08:54<19:50:46,  7.55s/it]
 96%|█████████▌| 215542/225000 [419:09:06<23:34:56,  8.98s/it]
 96%|█████████▌| 215543/225000 [419:09:13<22:00:18,  8.38s/it]
 96%|█████████▌| 215544/225000 [419:09:25<24:47:52,  9.44s/it]
 96%|█████████▌| 215545/225000 [419:09:31<22:35:34,  8.60s/it]
 96%|█████████▌| 215546/225000 [419:09:43<24:44:55,  9.42s/it]
 96%|█████████▌| 215547/225000 [419:09:50<23:24:03,  8.91s/it]
 96%|█████████▌| 215548/225000 [419:09:57<21:48:32,  8.31s/it]
 96%|█████████▌| 215549/225000 [419:10:07<22:44:51,  8.66s/it]
 96%|█████████▌| 215550/225000 [419:10:24<29:15:12, 11.14s/it]
                                                              
{'loss': '1.313', 'grad_norm': '0.956', 'learning_rate': '2.431e-07', 'epoch': '0.958', 'num_input_tokens_seen': 711824904, 'train_runtime': '1.509e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215550/225000 [419:10:24<29:15:12, 11.14s/it]
 96%|█████████▌| 215551/225000 [419:10:31<26:10:07,  9.97s/it]
 96%|█████████▌| 215552/225000 [419:10:41<25:57:29,  9.89s/it]
 96%|█████████▌| 215553/225000 [419:10:48<23:52:56,  9.10s/it]
 96%|█████████▌| 215554/225000 [419:11:00<26:09:38,  9.97s/it]
 96%|█████████▌| 215555/225000 [419:11:08<24:34:34,  9.37s/it]
 96%|█████████▌| 215556/225000 [419:11:15<23:02:47,  8.79s/it]
 96%|█████████▌| 215557/225000 [419:11:24<22:52:52,  8.72s/it]
 96%|█████████▌| 215558/225000 [419:11:37<26:22:00, 10.05s/it]
 96%|█████████▌| 215559/225000 [419:11:48<26:40:55, 10.17s/it]
 96%|█████████▌| 215560/225000 [419:11:55<24:24:02,  9.31s/it]
 96%|█████████▌| 215561/225000 [419:12:06<26:01:14,  9.92s/it]
 96%|█████████▌| 215562/225000 [419:12:12<22:25:45,  8.56s/it]
 96%|█████████▌| 215563/225000 [419:12:19<21:16:30,  8.12s/it]
 96%|█████████▌| 215564/225000 [419:12:37<29:12:49, 11.15s/it]
 96%|█████████▌| 215565/225000 [419:12:42<24:06:33,  9.20s/it]
 96%|█████████▌| 215566/225000 [419:12:48<21:58:52,  8.39s/it]
 96%|█████████▌| 215567/225000 [419:12:58<23:04:24,  8.81s/it]
 96%|█████████▌| 215568/225000 [419:13:09<24:41:12,  9.42s/it]
 96%|█████████▌| 215569/225000 [419:13:17<24:07:47,  9.21s/it]
 96%|█████████▌| 215570/225000 [419:13:28<24:50:46,  9.49s/it]
 96%|█████████▌| 215571/225000 [419:13:34<22:32:36,  8.61s/it]
 96%|█████████▌| 215572/225000 [419:13:41<20:56:51,  8.00s/it]
 96%|█████████▌| 215573/225000 [419:13:49<21:20:56,  8.15s/it]
 96%|█████████▌| 215574/225000 [419:13:57<21:07:35,  8.07s/it]
 96%|█████████▌| 215575/225000 [419:14:04<19:54:46,  7.61s/it]
 96%|█████████▌| 215576/225000 [419:14:12<20:38:37,  7.89s/it]
 96%|█████████▌| 215577/225000 [419:14:28<26:34:47, 10.15s/it]
 96%|█████████▌| 215578/225000 [419:14:37<25:57:31,  9.92s/it]
 96%|█████████▌| 215579/225000 [419:14:46<25:35:53,  9.78s/it]
 96%|█████████▌| 215580/225000 [419:14:54<23:39:35,  9.04s/it]
 96%|█████████▌| 215581/225000 [419:15:08<27:34:08, 10.54s/it]
 96%|█████████▌| 215582/225000 [419:15:14<23:58:20,  9.16s/it]
 96%|█████████▌| 215583/225000 [419:15:18<20:30:16,  7.84s/it]
 96%|█████████▌| 215584/225000 [419:15:25<19:17:45,  7.38s/it]
 96%|█████████▌| 215585/225000 [419:15:36<22:05:05,  8.44s/it]
 96%|█████████▌| 215586/225000 [419:15:44<22:12:51,  8.50s/it]
 96%|█████████▌| 215587/225000 [419:15:53<22:35:57,  8.64s/it]
 96%|█████████▌| 215588/225000 [419:16:06<25:55:19,  9.91s/it]
 96%|█████████▌| 215589/225000 [419:16:11<22:14:51,  8.51s/it]
 96%|█████████▌| 215590/225000 [419:16:23<24:18:06,  9.30s/it]
 96%|█████████▌| 215591/225000 [419:16:28<21:34:47,  8.26s/it]
 96%|█████████▌| 215592/225000 [419:16:38<22:33:22,  8.63s/it]
 96%|█████████▌| 215593/225000 [419:16:43<20:00:47,  7.66s/it]
 96%|█████████▌| 215594/225000 [419:16:57<24:49:33,  9.50s/it]
 96%|█████████▌| 215595/225000 [419:17:03<22:07:36,  8.47s/it]
 96%|█████████▌| 215596/225000 [419:17:17<26:36:01, 10.18s/it]
 96%|█████████▌| 215597/225000 [419:17:23<23:16:02,  8.91s/it]
 96%|█████████▌| 215598/225000 [419:17:33<24:13:55,  9.28s/it]
 96%|█████████▌| 215599/225000 [419:17:43<24:24:20,  9.35s/it]
 96%|█████████▌| 215600/225000 [419:17:51<23:41:01,  9.07s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.362', 'learning_rate': '2.405e-07', 'epoch': '0.9582', 'num_input_tokens_seen': 712060536, 'train_runtime': '1.509e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215600/225000 [419:17:51<23:41:01,  9.07s/it]
 96%|█████████▌| 215601/225000 [419:18:02<25:20:03,  9.70s/it]
 96%|█████████▌| 215602/225000 [419:18:11<24:11:58,  9.27s/it]
 96%|█████████▌| 215603/225000 [419:18:15<20:10:44,  7.73s/it]
 96%|█████████▌| 215604/225000 [419:18:26<22:52:11,  8.76s/it]
 96%|█████████▌| 215605/225000 [419:18:33<21:08:00,  8.10s/it]
 96%|█████████▌| 215606/225000 [419:18:38<19:24:22,  7.44s/it]
 96%|█████████▌| 215607/225000 [419:18:50<22:55:12,  8.78s/it]
 96%|█████████▌| 215608/225000 [419:18:58<22:06:29,  8.47s/it]
 96%|█████████▌| 215609/225000 [419:19:04<19:50:51,  7.61s/it]
 96%|█████████▌| 215610/225000 [419:19:13<20:46:49,  7.97s/it]
 96%|█████████▌| 215611/225000 [419:19:26<24:42:39,  9.47s/it]
 96%|█████████▌| 215612/225000 [419:19:34<23:40:07,  9.08s/it]
 96%|█████████▌| 215613/225000 [419:19:47<26:50:44, 10.30s/it]
 96%|█████████▌| 215614/225000 [419:19:54<24:24:00,  9.36s/it]
 96%|█████████▌| 215615/225000 [419:20:03<23:45:01,  9.11s/it]
 96%|█████████▌| 215616/225000 [419:20:11<23:17:39,  8.94s/it]
 96%|█████████▌| 215617/225000 [419:20:20<23:04:17,  8.85s/it]
 96%|█████████▌| 215618/225000 [419:20:25<20:01:08,  7.68s/it]
 96%|█████████▌| 215619/225000 [419:20:36<22:58:58,  8.82s/it]
 96%|█████████▌| 215620/225000 [419:20:42<20:45:07,  7.96s/it]
 96%|█████████▌| 215621/225000 [419:20:47<18:24:15,  7.06s/it]
 96%|█████████▌| 215622/225000 [419:20:53<17:13:05,  6.61s/it]
 96%|█████████▌| 215623/225000 [419:21:08<23:41:55,  9.10s/it]
 96%|█████████▌| 215624/225000 [419:21:17<24:18:24,  9.33s/it]
 96%|█████████▌| 215625/225000 [419:21:29<26:06:15, 10.02s/it]
 96%|█████████▌| 215626/225000 [419:21:37<24:11:40,  9.29s/it]
 96%|█████████▌| 215627/225000 [419:21:43<21:49:14,  8.38s/it]
 96%|█████████▌| 215628/225000 [419:21:50<20:43:13,  7.96s/it]
 96%|█████████▌| 215629/225000 [419:22:05<26:20:33, 10.12s/it]
 96%|█████████▌| 215630/225000 [419:22:13<24:39:02,  9.47s/it]
 96%|█████████▌| 215631/225000 [419:22:18<21:32:32,  8.28s/it]
 96%|█████████▌| 215632/225000 [419:22:24<19:28:17,  7.48s/it]
 96%|█████████▌| 215633/225000 [419:22:38<24:17:53,  9.34s/it]
 96%|█████████▌| 215634/225000 [419:22:48<24:51:58,  9.56s/it]
 96%|█████████▌| 215635/225000 [419:23:02<28:21:16, 10.90s/it]
 96%|█████████▌| 215636/225000 [419:23:13<28:38:44, 11.01s/it]
 96%|█████████▌| 215637/225000 [419:23:19<24:22:17,  9.37s/it]
 96%|█████████▌| 215638/225000 [419:23:24<21:19:04,  8.20s/it]
 96%|█████████▌| 215639/225000 [419:23:31<20:36:26,  7.93s/it]
 96%|█████████▌| 215640/225000 [419:23:39<19:57:19,  7.68s/it]
 96%|█████████▌| 215641/225000 [419:23:49<22:25:36,  8.63s/it]
 96%|█████████▌| 215642/225000 [419:23:54<19:40:25,  7.57s/it]
 96%|█████████▌| 215643/225000 [419:24:03<20:41:53,  7.96s/it]
 96%|█████████▌| 215644/225000 [419:24:14<22:35:58,  8.70s/it]
 96%|█████████▌| 215645/225000 [419:24:19<19:44:50,  7.60s/it]
 96%|█████████▌| 215646/225000 [419:24:27<19:49:10,  7.63s/it]
 96%|█████████▌| 215647/225000 [419:24:38<22:58:35,  8.84s/it]
 96%|█████████▌| 215648/225000 [419:24:44<20:49:28,  8.02s/it]
 96%|█████████▌| 215649/225000 [419:24:58<25:08:45,  9.68s/it]
 96%|█████████▌| 215650/225000 [419:25:02<20:36:49,  7.94s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.337', 'learning_rate': '2.38e-07', 'epoch': '0.9584', 'num_input_tokens_seen': 712289520, 'train_runtime': '1.51e+06', 'train_tokens_per_second': '471.7'}

 96%|█████████▌| 215650/225000 [419:25:02<20:36:49,  7.94s/it]
 96%|█████████▌| 215651/225000 [419:25:11<21:24:43,  8.25s/it]
 96%|█████████▌| 215652/225000 [419:25:19<21:10:39,  8.16s/it]
 96%|█████████▌| 215653/225000 [419:25:28<21:52:25,  8.42s/it]
 96%|█████████▌| 215654/225000 [419:25:34<20:05:15,  7.74s/it]
 96%|█████████▌| 215655/225000 [419:25:47<24:40:47,  9.51s/it]
 96%|█████████▌| 215656/225000 [419:25:57<24:27:38,  9.42s/it]
 96%|█████████▌| 215657/225000 [419:26:02<21:35:45,  8.32s/it]
 96%|█████████▌| 215658/225000 [419:26:11<21:37:31,  8.33s/it]
 96%|█████████▌| 215659/225000 [419:26:17<20:05:23,  7.74s/it]
 96%|█████████▌| 215660/225000 [419:26:29<23:32:52,  9.08s/it]
 96%|█████████▌| 215661/225000 [419:26:35<20:48:11,  8.02s/it]
 96%|█████████▌| 215662/225000 [419:26:41<19:39:13,  7.58s/it]
 96%|█████████▌| 215663/225000 [419:26:51<20:49:56,  8.03s/it]
 96%|█████████▌| 215664/225000 [419:27:02<23:33:48,  9.09s/it]
 96%|█████████▌| 215665/225000 [419:27:11<23:48:27,  9.18s/it]
 96%|█████████▌| 215666/225000 [419:27:27<28:25:26, 10.96s/it]
 96%|█████████▌| 215667/225000 [419:27:34<25:38:25,  9.89s/it]
 96%|█████████▌| 215668/225000 [419:27:43<25:16:01,  9.75s/it]
 96%|█████████▌| 215669/225000 [419:27:53<24:53:52,  9.61s/it]
 96%|█████████▌| 215670/225000 [419:27:57<20:58:03,  8.09s/it]
 96%|█████████▌| 215671/225000 [419:28:12<25:57:18, 10.02s/it]
 96%|█████████▌| 215672/225000 [419:28:17<22:35:52,  8.72s/it]
 96%|█████████▌| 215673/225000 [419:28:23<19:59:37,  7.72s/it]
 96%|█████████▌| 215674/225000 [419:28:30<19:46:38,  7.63s/it]
 96%|█████████▌| 215675/225000 [419:28:37<18:51:48,  7.28s/it]
 96%|█████████▌| 215676/225000 [419:28:50<23:22:21,  9.02s/it]
 96%|█████████▌| 215677/225000 [419:28:55<20:45:40,  8.02s/it]
 96%|█████████▌| 215678/225000 [419:29:09<25:06:25,  9.70s/it]
 96%|█████████▌| 215679/225000 [419:29:17<24:00:38,  9.27s/it]
 96%|█████████▌| 215680/225000 [419:29:27<24:13:54,  9.36s/it]
 96%|█████████▌| 215681/225000 [419:29:35<23:26:58,  9.06s/it]
 96%|█████████▌| 215682/225000 [419:29:43<22:22:56,  8.65s/it]
 96%|█████████▌| 215683/225000 [419:29:54<24:15:57,  9.38s/it]
 96%|█████████▌| 215684/225000 [419:30:04<24:20:52,  9.41s/it]
 96%|█████████▌| 215685/225000 [419:30:16<26:57:16, 10.42s/it]
 96%|█████████▌| 215686/225000 [419:30:26<26:45:52, 10.34s/it]
 96%|█████████▌| 215687/225000 [419:30:35<25:11:00,  9.73s/it]
 96%|█████████▌| 215688/225000 [419:30:41<22:23:34,  8.66s/it]
 96%|█████████▌| 215689/225000 [419:30:51<23:50:24,  9.22s/it]
 96%|█████████▌| 215690/225000 [419:31:01<23:41:43,  9.16s/it]
 96%|█████████▌| 215691/225000 [419:31:09<23:21:58,  9.04s/it]
 96%|█████████▌| 215692/225000 [419:31:14<20:19:36,  7.86s/it]
 96%|█████████▌| 215693/225000 [419:31:22<20:03:41,  7.76s/it]
 96%|█████████▌| 215694/225000 [419:31:29<19:21:24,  7.49s/it]
 96%|█████████▌| 215695/225000 [419:31:36<18:57:43,  7.34s/it]
 96%|█████████▌| 215696/225000 [419:31:48<22:47:53,  8.82s/it]
 96%|█████████▌| 215697/225000 [419:31:55<21:16:28,  8.23s/it]
 96%|█████████▌| 215698/225000 [419:32:06<23:11:06,  8.97s/it]
 96%|█████████▌| 215699/225000 [419:32:12<21:03:44,  8.15s/it]
 96%|█████████▌| 215700/225000 [419:32:25<24:44:19,  9.58s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.643', 'learning_rate': '2.355e-07', 'epoch': '0.9587', 'num_input_tokens_seen': 712523048, 'train_runtime': '1.51e+06', 'train_tokens_per_second': '471.8'}

 96%|█████████▌| 215700/225000 [419:32:25<24:44:19,  9.58s/it]
 96%|█████████▌| 215701/225000 [419:32:32<22:48:39,  8.83s/it]
 96%|█████████▌| 215702/225000 [419:32:37<19:48:48,  7.67s/it]
 96%|█████████▌| 215703/225000 [419:32:50<23:47:53,  9.22s/it]
 96%|█████████▌| 215704/225000 [419:32:58<22:49:22,  8.84s/it]
 96%|█████████▌| 215705/225000 [419:33:06<22:39:05,  8.77s/it]
 96%|█████████▌| 215706/225000 [419:33:13<21:00:08,  8.14s/it]
 96%|█████████▌| 215707/225000 [419:33:21<21:15:38,  8.24s/it]
 96%|█████████▌| 215708/225000 [419:33:31<22:09:16,  8.58s/it]
 96%|█████████▌| 215709/225000 [419:33:36<19:40:39,  7.62s/it]
 96%|█████████▌| 215710/225000 [419:33:42<18:45:33,  7.27s/it]
 96%|█████████▌| 215711/225000 [419:33:51<20:00:19,  7.75s/it]
 96%|█████████▌| 215712/225000 [419:34:02<21:59:13,  8.52s/it]
 96%|█████████▌| 215713/225000 [419:34:09<21:10:56,  8.21s/it]
 96%|█████████▌| 215714/225000 [419:34:17<20:47:51,  8.06s/it]
 96%|█████████▌| 215715/225000 [419:34:27<22:03:20,  8.55s/it]
 96%|█████████▌| 215716/225000 [419:34:34<21:19:33,  8.27s/it]
 96%|█████████▌| 215717/225000 [419:34:46<23:52:54,  9.26s/it]
 96%|█████████▌| 215718/225000 [419:34:53<22:25:06,  8.69s/it]
 96%|█████████▌| 215719/225000 [419:35:01<21:29:58,  8.34s/it]
 96%|█████████▌| 215720/225000 [419:35:08<21:03:26,  8.17s/it]
 96%|█████████▌| 215721/225000 [419:35:14<19:17:22,  7.48s/it]
 96%|█████████▌| 215722/225000 [419:35:30<25:56:23, 10.07s/it]
 96%|█████████▌| 215723/225000 [419:35:36<22:23:57,  8.69s/it]
 96%|█████████▌| 215724/225000 [419:35:44<21:33:40,  8.37s/it]
 96%|█████████▌| 215725/225000 [419:35:56<24:22:15,  9.46s/it]
 96%|█████████▌| 215726/225000 [419:36:01<21:31:54,  8.36s/it]
 96%|█████████▌| 215727/225000 [419:36:10<22:03:00,  8.56s/it]
 96%|█████████▌| 215728/225000 [419:36:17<20:18:47,  7.89s/it]
 96%|█████████▌| 215729/225000 [419:36:28<22:54:43,  8.90s/it]
 96%|█████████▌| 215730/225000 [419:36:38<23:54:24,  9.28s/it]
 96%|█████████▌| 215731/225000 [419:36:42<19:50:51,  7.71s/it]
 96%|█████████▌| 215732/225000 [419:36:50<19:38:13,  7.63s/it]
 96%|█████████▌| 215733/225000 [419:36:56<18:20:46,  7.13s/it]
 96%|█████████▌| 215734/225000 [419:37:03<18:34:08,  7.21s/it]
 96%|█████████▌| 215735/225000 [419:37:13<20:27:45,  7.95s/it]
 96%|█████████▌| 215736/225000 [419:37:26<24:24:52,  9.49s/it]
 96%|█████████▌| 215737/225000 [419:37:34<23:47:44,  9.25s/it]
 96%|█████████▌| 215738/225000 [419:37:44<23:55:41,  9.30s/it]
 96%|█████████▌| 215739/225000 [419:37:52<23:14:31,  9.03s/it]
 96%|█████████▌| 215740/225000 [419:38:00<22:04:52,  8.58s/it]
 96%|█████████▌| 215741/225000 [419:38:06<20:38:13,  8.02s/it]
 96%|█████████▌| 215742/225000 [419:38:16<21:32:48,  8.38s/it]
 96%|█████████▌| 215743/225000 [419:38:24<21:19:27,  8.29s/it]
 96%|█████████▌| 215744/225000 [419:38:37<25:26:19,  9.89s/it]
 96%|█████████▌| 215745/225000 [419:38:46<24:04:47,  9.37s/it]
 96%|█████████▌| 215746/225000 [419:38:56<24:40:58,  9.60s/it]
 96%|█████████▌| 215747/225000 [419:39:01<21:08:05,  8.22s/it]
 96%|█████████▌| 215748/225000 [419:39:11<22:23:35,  8.71s/it]
 96%|█████████▌| 215749/225000 [419:39:22<24:45:56,  9.64s/it]
 96%|█████████▌| 215750/225000 [419:39:31<24:16:48,  9.45s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.337', 'learning_rate': '2.33e-07', 'epoch': '0.9589', 'num_input_tokens_seen': 712750112, 'train_runtime': '1.511e+06', 'train_tokens_per_second': '471.8'}

 96%|█████████▌| 215750/225000 [419:39:31<24:16:48,  9.45s/it]
 96%|█████████▌| 215751/225000 [419:39:40<24:02:55,  9.36s/it]
 96%|█████████▌| 215752/225000 [419:39:49<23:21:02,  9.09s/it]
 96%|█████████▌| 215753/225000 [419:39:59<24:03:20,  9.37s/it]
 96%|█████████▌| 215754/225000 [419:40:09<24:29:21,  9.54s/it]
 96%|█████████▌| 215755/225000 [419:40:20<25:55:01, 10.09s/it]
 96%|█████████▌| 215756/225000 [419:40:26<22:09:38,  8.63s/it]
 96%|█████████▌| 215757/225000 [419:40:37<24:23:18,  9.50s/it]
 96%|█████████▌| 215758/225000 [419:40:43<21:26:28,  8.35s/it]
 96%|█████████▌| 215759/225000 [419:40:54<23:34:36,  9.18s/it]
 96%|█████████▌| 215760/225000 [419:41:02<22:56:03,  8.94s/it]
 96%|█████████▌| 215761/225000 [419:41:08<20:44:01,  8.08s/it]
 96%|█████████▌| 215762/225000 [419:41:15<20:01:04,  7.80s/it]
 96%|█████████▌| 215763/225000 [419:41:23<19:40:16,  7.67s/it]
 96%|█████████▌| 215764/225000 [419:41:28<18:02:00,  7.03s/it]
 96%|█████████▌| 215765/225000 [419:41:35<18:08:56,  7.07s/it]
 96%|█████████▌| 215766/225000 [419:41:43<18:07:27,  7.07s/it]
 96%|█████████▌| 215767/225000 [419:41:48<17:00:52,  6.63s/it]
 96%|█████████▌| 215768/225000 [419:42:03<23:33:47,  9.19s/it]
 96%|█████████▌| 215769/225000 [419:42:10<21:57:45,  8.57s/it]
 96%|█████████▌| 215770/225000 [419:42:17<20:31:00,  8.00s/it]
 96%|█████████▌| 215771/225000 [419:42:28<22:51:30,  8.92s/it]
 96%|█████████▌| 215772/225000 [419:42:34<20:38:46,  8.05s/it]
 96%|█████████▌| 215773/225000 [419:42:41<19:30:54,  7.61s/it]
 96%|█████████▌| 215774/225000 [419:42:54<23:54:20,  9.33s/it]
 96%|█████████▌| 215775/225000 [419:43:06<25:42:57, 10.04s/it]
 96%|█████████▌| 215776/225000 [419:43:11<22:05:30,  8.62s/it]
 96%|█████████▌| 215777/225000 [419:43:22<23:36:43,  9.22s/it]
 96%|█████████▌| 215778/225000 [419:43:30<22:51:22,  8.92s/it]
 96%|█████████▌| 215779/225000 [419:43:36<20:43:39,  8.09s/it]
 96%|█████████▌| 215780/225000 [419:43:43<20:03:30,  7.83s/it]
 96%|█████████▌| 215781/225000 [419:44:00<26:30:44, 10.35s/it]
 96%|█████████▌| 215782/225000 [419:44:06<23:38:05,  9.23s/it]
 96%|█████████▌| 215783/225000 [419:44:16<23:50:02,  9.31s/it]
 96%|█████████▌| 215784/225000 [419:44:20<19:54:15,  7.78s/it]
 96%|█████████▌| 215785/225000 [419:44:29<20:40:08,  8.07s/it]
 96%|█████████▌| 215786/225000 [419:44:35<19:34:13,  7.65s/it]
 96%|█████████▌| 215787/225000 [419:44:42<18:53:13,  7.38s/it]
 96%|█████████▌| 215788/225000 [419:44:49<18:34:15,  7.26s/it]
 96%|█████████▌| 215789/225000 [419:45:07<26:41:43, 10.43s/it]
 96%|█████████▌| 215790/225000 [419:45:15<24:53:16,  9.73s/it]
 96%|█████████▌| 215791/225000 [419:45:23<23:18:22,  9.11s/it]
 96%|█████████▌| 215792/225000 [419:45:29<21:22:25,  8.36s/it]
 96%|█████████▌| 215793/225000 [419:45:41<24:21:56,  9.53s/it]
 96%|█████████▌| 215794/225000 [419:45:47<21:12:33,  8.29s/it]
 96%|█████████▌| 215795/225000 [419:45:57<22:30:23,  8.80s/it]
 96%|█████████▌| 215796/225000 [419:46:03<20:41:25,  8.09s/it]
 96%|█████████▌| 215797/225000 [419:46:14<22:35:50,  8.84s/it]
 96%|█████████▌| 215798/225000 [419:46:19<19:38:39,  7.69s/it]
 96%|█████████▌| 215799/225000 [419:46:33<24:40:40,  9.66s/it]
 96%|█████████▌| 215800/225000 [419:46:41<23:01:55,  9.01s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.098', 'learning_rate': '2.305e-07', 'epoch': '0.9591', 'num_input_tokens_seen': 712983256, 'train_runtime': '1.511e+06', 'train_tokens_per_second': '471.8'}

 96%|█████████▌| 215800/225000 [419:46:41<23:01:55,  9.01s/it]
 96%|█████████▌| 215801/225000 [419:46:48<21:52:33,  8.56s/it]
 96%|█████████▌| 215802/225000 [419:46:55<20:22:23,  7.97s/it]
 96%|█████████▌| 215803/225000 [419:47:05<22:14:17,  8.70s/it]
 96%|█████████▌| 215804/225000 [419:47:11<19:45:48,  7.74s/it]
 96%|█████████▌| 215805/225000 [419:47:23<23:25:48,  9.17s/it]
 96%|█████████▌| 215806/225000 [419:47:33<23:48:50,  9.32s/it]
 96%|█████████▌| 215807/225000 [419:47:42<23:28:24,  9.19s/it]
 96%|█████████▌| 215808/225000 [419:47:49<22:19:29,  8.74s/it]
 96%|█████████▌| 215809/225000 [419:47:57<21:15:01,  8.32s/it]
 96%|█████████▌| 215810/225000 [419:48:03<19:48:08,  7.76s/it]
 96%|█████████▌| 215811/225000 [419:48:13<21:23:31,  8.38s/it]
 96%|█████████▌| 215812/225000 [419:48:21<20:47:50,  8.15s/it]
 96%|█████████▌| 215813/225000 [419:48:31<22:48:39,  8.94s/it]
 96%|█████████▌| 215814/225000 [419:48:41<23:26:29,  9.19s/it]
 96%|█████████▌| 215815/225000 [419:48:48<21:15:46,  8.33s/it]
 96%|█████████▌| 215816/225000 [419:48:55<20:32:45,  8.05s/it]
 96%|█████████▌| 215817/225000 [419:49:03<20:22:15,  7.99s/it]
 96%|█████████▌| 215818/225000 [419:49:16<24:28:18,  9.59s/it]
 96%|█████████▌| 215819/225000 [419:49:26<24:56:53,  9.78s/it]
 96%|█████████▌| 215820/225000 [419:49:35<23:53:57,  9.37s/it]
 96%|█████████▌| 215821/225000 [419:49:43<23:03:46,  9.05s/it]
 96%|█████████▌| 215822/225000 [419:49:52<23:00:31,  9.03s/it]
 96%|█████████▌| 215823/225000 [419:50:04<25:25:12,  9.97s/it]
 96%|█████████▌| 215824/225000 [419:50:10<22:34:35,  8.86s/it]
 96%|█████████▌| 215825/225000 [419:50:20<23:15:20,  9.12s/it]
 96%|█████████▌| 215826/225000 [419:50:29<22:42:12,  8.91s/it]
 96%|█████████▌| 215827/225000 [419:50:36<21:22:11,  8.39s/it]
 96%|█████████▌| 215828/225000 [419:50:42<20:03:00,  7.87s/it]
 96%|█████████▌| 215829/225000 [419:50:54<22:40:29,  8.90s/it]
 96%|█████████▌| 215830/225000 [419:51:00<20:22:50,  8.00s/it]
 96%|█████████▌| 215831/225000 [419:51:08<20:27:30,  8.03s/it]
 96%|█████████▌| 215832/225000 [419:51:16<20:42:31,  8.13s/it]
 96%|█████████▌| 215833/225000 [419:51:20<17:25:47,  6.84s/it]
 96%|█████████▌| 215834/225000 [419:51:31<20:18:51,  7.98s/it]
 96%|█████████▌| 215835/225000 [419:51:39<20:43:32,  8.14s/it]
 96%|█████████▌| 215836/225000 [419:51:45<18:56:53,  7.44s/it]
 96%|█████████▌| 215837/225000 [419:51:56<21:51:30,  8.59s/it]
 96%|█████████▌| 215838/225000 [419:52:02<19:55:33,  7.83s/it]
 96%|█████████▌| 215839/225000 [419:52:08<18:23:58,  7.23s/it]
 96%|█████████▌| 215840/225000 [419:52:15<18:19:13,  7.20s/it]
 96%|█████████▌| 215841/225000 [419:52:24<19:48:04,  7.78s/it]
 96%|█████████▌| 215842/225000 [419:52:33<20:04:13,  7.89s/it]
 96%|█████████▌| 215843/225000 [419:52:42<21:40:07,  8.52s/it]
 96%|█████████▌| 215844/225000 [419:52:50<20:34:59,  8.09s/it]
 96%|█████████▌| 215845/225000 [419:52:55<18:31:52,  7.29s/it]
 96%|█████████▌| 215846/225000 [419:53:08<23:06:34,  9.09s/it]
 96%|█████████▌| 215847/225000 [419:53:13<19:30:13,  7.67s/it]
 96%|█████████▌| 215848/225000 [419:53:22<20:37:13,  8.11s/it]
 96%|█████████▌| 215849/225000 [419:53:33<22:36:14,  8.89s/it]
 96%|█████████▌| 215850/225000 [419:53:44<24:51:50,  9.78s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.146', 'learning_rate': '2.281e-07', 'epoch': '0.9593', 'num_input_tokens_seen': 713211120, 'train_runtime': '1.512e+06', 'train_tokens_per_second': '471.8'}

 96%|█████████▌| 215850/225000 [419:53:44<24:51:50,  9.78s/it]
 96%|█████████▌| 215851/225000 [419:53:57<27:13:53, 10.72s/it]
 96%|█████████▌| 215852/225000 [419:54:06<25:24:24, 10.00s/it]
 96%|█████████▌| 215853/225000 [419:54:14<24:03:35,  9.47s/it]
 96%|█████████▌| 215854/225000 [419:54:19<20:56:36,  8.24s/it]
 96%|█████████▌| 215855/225000 [419:54:31<23:47:59,  9.37s/it]
 96%|█████████▌| 215856/225000 [419:54:39<22:52:46,  9.01s/it]
 96%|█████████▌| 215857/225000 [419:54:49<23:39:12,  9.31s/it]
 96%|█████████▌| 215858/225000 [419:54:55<20:39:36,  8.14s/it]
 96%|█████████▌| 215859/225000 [419:55:05<21:56:42,  8.64s/it]
 96%|█████████▌| 215860/225000 [419:55:11<20:12:10,  7.96s/it]
 96%|█████████▌| 215861/225000 [419:55:18<19:26:25,  7.66s/it]
 96%|█████████▌| 215862/225000 [419:55:30<22:47:13,  8.98s/it]
 96%|█████████▌| 215863/225000 [419:55:38<22:04:27,  8.70s/it]
 96%|█████████▌| 215864/225000 [419:55:45<20:50:13,  8.21s/it]
 96%|█████████▌| 215865/225000 [419:55:52<20:07:27,  7.93s/it]
 96%|█████████▌| 215866/225000 [419:56:00<20:03:01,  7.90s/it]
 96%|█████████▌| 215867/225000 [419:56:08<19:53:34,  7.84s/it]
 96%|█████████▌| 215868/225000 [419:56:15<19:13:08,  7.58s/it]
 96%|█████████▌| 215869/225000 [419:56:27<22:49:27,  9.00s/it]
 96%|█████████▌| 215870/225000 [419:56:39<24:53:31,  9.82s/it]
 96%|█████████▌| 215871/225000 [419:56:46<23:01:38,  9.08s/it]
 96%|█████████▌| 215872/225000 [419:56:54<21:57:27,  8.66s/it]
 96%|█████████▌| 215873/225000 [419:57:05<23:34:34,  9.30s/it]
 96%|█████████▌| 215874/225000 [419:57:11<21:10:42,  8.35s/it]
 96%|█████████▌| 215875/225000 [419:57:24<24:53:55,  9.82s/it]
 96%|█████████▌| 215876/225000 [419:57:33<24:10:04,  9.54s/it]
 96%|█████████▌| 215877/225000 [419:57:45<26:00:46, 10.26s/it]
 96%|█████████▌| 215878/225000 [419:57:56<26:25:17, 10.43s/it]
 96%|█████████▌| 215879/225000 [419:58:02<23:30:51,  9.28s/it]
 96%|█████████▌| 215880/225000 [419:58:14<25:00:15,  9.87s/it]
 96%|█████████▌| 215881/225000 [419:58:18<20:47:18,  8.21s/it]
 96%|█████████▌| 215882/225000 [419:58:22<17:48:38,  7.03s/it]
 96%|█████████▌| 215883/225000 [419:58:35<22:21:37,  8.83s/it]
 96%|█████████▌| 215884/225000 [419:58:40<19:13:05,  7.59s/it]
 96%|█████████▌| 215885/225000 [419:58:51<21:43:52,  8.58s/it]
 96%|█████████▌| 215886/225000 [419:59:01<22:48:29,  9.01s/it]
 96%|█████████▌| 215887/225000 [419:59:07<20:33:17,  8.12s/it]
 96%|█████████▌| 215888/225000 [419:59:14<19:30:26,  7.71s/it]
 96%|█████████▌| 215889/225000 [419:59:24<21:49:10,  8.62s/it]
 96%|█████████▌| 215890/225000 [419:59:32<21:06:12,  8.34s/it]
 96%|█████████▌| 215891/225000 [419:59:42<22:20:48,  8.83s/it]
 96%|█████████▌| 215892/225000 [419:59:54<24:59:24,  9.88s/it]
 96%|█████████▌| 215893/225000 [419:59:59<20:46:54,  8.22s/it]
 96%|█████████▌| 215894/225000 [420:00:08<21:29:20,  8.50s/it]
 96%|█████████▌| 215895/225000 [420:00:13<19:03:08,  7.53s/it]
 96%|█████████▌| 215896/225000 [420:00:22<19:49:25,  7.84s/it]
 96%|█████████▌| 215897/225000 [420:00:27<18:10:51,  7.19s/it]
 96%|█████████▌| 215898/225000 [420:00:42<23:36:33,  9.34s/it]
 96%|█████████▌| 215899/225000 [420:00:48<21:06:32,  8.35s/it]
 96%|█████████▌| 215900/225000 [420:00:56<21:10:05,  8.37s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.177', 'learning_rate': '2.256e-07', 'epoch': '0.9596', 'num_input_tokens_seen': 713446200, 'train_runtime': '1.512e+06', 'train_tokens_per_second': '471.8'}

 96%|█████████▌| 215900/225000 [420:00:56<21:10:05,  8.37s/it]
 96%|█████████▌| 215901/225000 [420:01:01<18:22:37,  7.27s/it]
 96%|█████████▌| 215902/225000 [420:01:13<22:15:07,  8.80s/it]
 96%|█████████▌| 215903/225000 [420:01:22<22:01:12,  8.71s/it]
 96%|█████████▌| 215904/225000 [420:01:31<22:08:39,  8.76s/it]
 96%|█████████▌| 215905/225000 [420:01:37<20:10:02,  7.98s/it]
 96%|█████████▌| 215906/225000 [420:01:49<23:30:59,  9.31s/it]
 96%|█████████▌| 215907/225000 [420:01:55<20:44:44,  8.21s/it]
 96%|█████████▌| 215908/225000 [420:02:04<21:04:18,  8.34s/it]
 96%|█████████▌| 215909/225000 [420:02:11<20:11:17,  7.99s/it]
 96%|█████████▌| 215910/225000 [420:02:18<19:59:17,  7.92s/it]
 96%|█████████▌| 215911/225000 [420:02:30<22:54:00,  9.07s/it]
 96%|█████████▌| 215912/225000 [420:02:38<22:01:08,  8.72s/it]
 96%|█████████▌| 215913/225000 [420:02:46<21:16:50,  8.43s/it]
 96%|█████████▌| 215914/225000 [420:02:55<21:42:34,  8.60s/it]
 96%|█████████▌| 215915/225000 [420:03:01<19:59:11,  7.92s/it]
 96%|█████████▌| 215916/225000 [420:03:08<19:20:46,  7.67s/it]
 96%|█████████▌| 215917/225000 [420:03:26<26:35:14, 10.54s/it]
 96%|█████████▌| 215918/225000 [420:03:31<22:53:16,  9.07s/it]
 96%|█████████▌| 215919/225000 [420:03:41<23:35:24,  9.35s/it]
 96%|█████████▌| 215920/225000 [420:03:53<25:27:07, 10.09s/it]
 96%|█████████▌| 215921/225000 [420:03:58<21:46:32,  8.63s/it]
 96%|█████████▌| 215922/225000 [420:04:02<18:12:34,  7.22s/it]
 96%|█████████▌| 215923/225000 [420:04:08<17:19:08,  6.87s/it]
 96%|█████████▌| 215924/225000 [420:04:19<20:34:26,  8.16s/it]
 96%|█████████▌| 215925/225000 [420:04:26<19:34:45,  7.77s/it]
 96%|█████████▌| 215926/225000 [420:04:39<23:09:35,  9.19s/it]
 96%|█████████▌| 215927/225000 [420:04:46<21:50:58,  8.67s/it]
 96%|█████████▌| 215928/225000 [420:04:53<20:24:21,  8.10s/it]
 96%|█████████▌| 215929/225000 [420:04:59<18:39:00,  7.40s/it]
 96%|█████████▌| 215930/225000 [420:05:07<19:04:32,  7.57s/it]
 96%|█████████▌| 215931/225000 [420:05:13<18:28:08,  7.33s/it]
 96%|█████████▌| 215932/225000 [420:05:20<17:37:10,  7.00s/it]
 96%|█████████▌| 215933/225000 [420:05:30<19:53:49,  7.90s/it]
 96%|█████████▌| 215934/225000 [420:05:37<19:48:08,  7.86s/it]
 96%|█████████▌| 215935/225000 [420:05:51<23:55:31,  9.50s/it]
 96%|█████████▌| 215936/225000 [420:05:59<23:10:52,  9.21s/it]
 96%|█████████▌| 215937/225000 [420:06:10<24:16:21,  9.64s/it]
 96%|█████████▌| 215938/225000 [420:06:20<24:47:20,  9.85s/it]
 96%|█████████▌| 215939/225000 [420:06:26<21:19:19,  8.47s/it]
 96%|█████████▌| 215940/225000 [420:06:31<19:17:40,  7.67s/it]
 96%|█████████▌| 215941/225000 [420:06:43<22:29:51,  8.94s/it]
 96%|█████████▌| 215942/225000 [420:06:54<23:35:15,  9.37s/it]
 96%|█████████▌| 215943/225000 [420:06:59<20:13:01,  8.04s/it]
 96%|█████████▌| 215944/225000 [420:07:12<24:17:52,  9.66s/it]
 96%|█████████▌| 215945/225000 [420:07:19<21:57:49,  8.73s/it]
 96%|█████████▌| 215946/225000 [420:07:29<23:21:45,  9.29s/it]
 96%|█████████▌| 215947/225000 [420:07:35<20:49:05,  8.28s/it]
 96%|█████████▌| 215948/225000 [420:07:40<18:22:14,  7.31s/it]
 96%|█████████▌| 215949/225000 [420:07:50<20:31:49,  8.17s/it]
 96%|█████████▌| 215950/225000 [420:08:02<23:29:23,  9.34s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.207', 'learning_rate': '2.232e-07', 'epoch': '0.9598', 'num_input_tokens_seen': 713679968, 'train_runtime': '1.512e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 215950/225000 [420:08:02<23:29:23,  9.34s/it]
 96%|█████████▌| 215951/225000 [420:08:09<21:29:43,  8.55s/it]
 96%|█████████▌| 215952/225000 [420:08:20<23:30:42,  9.35s/it]
 96%|█████████▌| 215953/225000 [420:08:29<22:46:42,  9.06s/it]
 96%|█████████▌| 215954/225000 [420:08:37<22:03:12,  8.78s/it]
 96%|█████████▌| 215955/225000 [420:08:47<22:52:31,  9.10s/it]
 96%|█████████▌| 215956/225000 [420:08:58<24:25:34,  9.72s/it]
 96%|█████████▌| 215957/225000 [420:09:04<21:39:12,  8.62s/it]
 96%|█████████▌| 215958/225000 [420:09:10<19:51:03,  7.90s/it]
 96%|█████████▌| 215959/225000 [420:09:23<23:13:08,  9.25s/it]
 96%|█████████▌| 215960/225000 [420:09:27<19:49:54,  7.90s/it]
 96%|█████████▌| 215961/225000 [420:09:37<21:15:04,  8.46s/it]
 96%|█████████▌| 215962/225000 [420:09:45<21:05:34,  8.40s/it]
 96%|█████████▌| 215963/225000 [420:09:54<21:18:27,  8.49s/it]
 96%|█████████▌| 215964/225000 [420:10:01<20:17:51,  8.09s/it]
 96%|█████████▌| 215965/225000 [420:10:07<18:34:14,  7.40s/it]
 96%|█████████▌| 215966/225000 [420:10:18<21:39:09,  8.63s/it]
 96%|█████████▌| 215967/225000 [420:10:27<21:28:00,  8.56s/it]
 96%|█████████▌| 215968/225000 [420:10:31<18:28:12,  7.36s/it]
 96%|█████████▌| 215969/225000 [420:10:42<20:54:02,  8.33s/it]
 96%|█████████▌| 215970/225000 [420:10:49<20:07:35,  8.02s/it]
 96%|█████████▌| 215971/225000 [420:10:57<19:46:48,  7.89s/it]
 96%|█████████▌| 215972/225000 [420:11:04<18:55:36,  7.55s/it]
 96%|█████████▌| 215973/225000 [420:11:09<17:40:07,  7.05s/it]
 96%|█████████▌| 215974/225000 [420:11:19<19:24:38,  7.74s/it]
 96%|█████████▌| 215975/225000 [420:11:29<21:19:41,  8.51s/it]
 96%|█████████▌| 215976/225000 [420:11:39<22:14:54,  8.88s/it]
 96%|█████████▌| 215977/225000 [420:11:45<20:22:48,  8.13s/it]
 96%|█████████▌| 215978/225000 [420:11:57<23:20:58,  9.32s/it]
 96%|█████████▌| 215979/225000 [420:12:05<21:49:50,  8.71s/it]
 96%|█████████▌| 215980/225000 [420:12:12<21:08:41,  8.44s/it]
 96%|█████████▌| 215981/225000 [420:12:23<23:01:00,  9.19s/it]
 96%|█████████▌| 215982/225000 [420:12:32<22:39:23,  9.04s/it]
 96%|█████████▌| 215983/225000 [420:12:43<24:24:15,  9.74s/it]
 96%|█████████▌| 215984/225000 [420:12:53<23:55:20,  9.55s/it]
 96%|█████████▌| 215985/225000 [420:13:02<23:38:18,  9.44s/it]
 96%|█████████▌| 215986/225000 [420:13:08<21:15:26,  8.49s/it]
 96%|█████████▌| 215987/225000 [420:13:17<21:33:46,  8.61s/it]
 96%|█████████▌| 215988/225000 [420:13:28<23:27:44,  9.37s/it]
 96%|█████████▌| 215989/225000 [420:13:34<20:30:19,  8.19s/it]
 96%|█████████▌| 215990/225000 [420:13:40<19:15:08,  7.69s/it]
 96%|█████████▌| 215991/225000 [420:13:47<18:58:23,  7.58s/it]
 96%|█████████▌| 215992/225000 [420:13:54<18:00:25,  7.20s/it]
 96%|█████████▌| 215993/225000 [420:14:06<21:47:03,  8.71s/it]
 96%|█████████▌| 215994/225000 [420:14:13<20:55:52,  8.37s/it]
 96%|█████████▌| 215995/225000 [420:14:21<19:58:55,  7.99s/it]
 96%|█████████▌| 215996/225000 [420:14:28<19:42:57,  7.88s/it]
 96%|█████████▌| 215997/225000 [420:14:33<17:36:17,  7.04s/it]
 96%|█████████▌| 215998/225000 [420:14:48<23:38:30,  9.45s/it]
 96%|█████████▌| 215999/225000 [420:14:55<21:40:22,  8.67s/it]
 96%|█████████▌| 216000/225000 [420:15:06<23:36:08,  9.44s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.168', 'learning_rate': '2.207e-07', 'epoch': '0.96', 'num_input_tokens_seen': 713909080, 'train_runtime': '1.513e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 216000/225000 [420:15:06<23:36:08,  9.44s/it]
 96%|█████████▌| 216001/225000 [420:15:13<21:28:05,  8.59s/it]
 96%|█████████▌| 216002/225000 [420:15:19<19:48:06,  7.92s/it]
 96%|█████████▌| 216003/225000 [420:15:31<22:33:15,  9.02s/it]
 96%|█████████▌| 216004/225000 [420:15:37<20:29:20,  8.20s/it]
 96%|█████████▌| 216005/225000 [420:15:49<22:48:08,  9.13s/it]
 96%|█████████▌| 216006/225000 [420:15:56<21:34:18,  8.63s/it]
 96%|█████████▌| 216007/225000 [420:16:05<21:55:00,  8.77s/it]
 96%|█████████▌| 216008/225000 [420:16:13<21:13:27,  8.50s/it]
 96%|█████████▌| 216009/225000 [420:16:25<23:51:22,  9.55s/it]
 96%|█████████▌| 216010/225000 [420:16:31<20:51:22,  8.35s/it]
 96%|█████████▌| 216011/225000 [420:16:36<18:31:55,  7.42s/it]
 96%|█████████▌| 216012/225000 [420:16:47<21:20:51,  8.55s/it]
 96%|█████████▌| 216013/225000 [420:16:52<19:02:31,  7.63s/it]
 96%|█████████▌| 216014/225000 [420:17:02<20:35:40,  8.25s/it]
 96%|█████████▌| 216015/225000 [420:17:09<19:13:25,  7.70s/it]
 96%|█████████▌| 216016/225000 [420:17:15<18:20:32,  7.35s/it]
 96%|█████████▌| 216017/225000 [420:17:27<22:02:51,  8.84s/it]
 96%|█████████▌| 216018/225000 [420:17:38<23:39:10,  9.48s/it]
 96%|█████████▌| 216019/225000 [420:17:44<20:39:25,  8.28s/it]
 96%|█████████▌| 216020/225000 [420:17:51<19:34:55,  7.85s/it]
 96%|█████████▌| 216021/225000 [420:17:59<20:04:29,  8.05s/it]
 96%|█████████▌| 216022/225000 [420:18:08<20:14:39,  8.12s/it]
 96%|█████████▌| 216023/225000 [420:18:17<21:19:26,  8.55s/it]
 96%|█████████▌| 216024/225000 [420:18:23<19:38:02,  7.87s/it]
 96%|█████████▌| 216025/225000 [420:18:31<19:42:28,  7.91s/it]
 96%|█████████▌| 216026/225000 [420:18:37<18:02:10,  7.24s/it]
 96%|█████████▌| 216027/225000 [420:18:48<20:42:43,  8.31s/it]
 96%|█████████▌| 216028/225000 [420:18:56<20:21:47,  8.17s/it]
 96%|█████████▌| 216029/225000 [420:19:05<21:30:23,  8.63s/it]
 96%|█████████▌| 216030/225000 [420:19:17<23:46:11,  9.54s/it]
 96%|█████████▌| 216031/225000 [420:19:26<23:05:55,  9.27s/it]
 96%|█████████▌| 216032/225000 [420:19:34<22:04:01,  8.86s/it]
 96%|█████████▌| 216033/225000 [420:19:44<23:31:43,  9.45s/it]
 96%|█████████▌| 216034/225000 [420:19:52<22:19:13,  8.96s/it]
 96%|█████████▌| 216035/225000 [420:20:03<23:48:45,  9.56s/it]
 96%|█████████▌| 216036/225000 [420:20:09<20:57:01,  8.41s/it]
 96%|█████████▌| 216037/225000 [420:20:21<23:41:05,  9.51s/it]
 96%|█████████▌| 216038/225000 [420:20:33<25:09:03, 10.10s/it]
 96%|█████████▌| 216039/225000 [420:20:37<20:59:17,  8.43s/it]
 96%|█████████▌| 216040/225000 [420:20:46<20:59:37,  8.44s/it]
 96%|█████████▌| 216041/225000 [420:20:52<19:45:31,  7.94s/it]
 96%|█████████▌| 216042/225000 [420:21:00<19:13:47,  7.73s/it]
 96%|█████████▌| 216043/225000 [420:21:13<23:19:10,  9.37s/it]
 96%|█████████▌| 216044/225000 [420:21:21<22:23:56,  9.00s/it]
 96%|█████████▌| 216045/225000 [420:21:33<24:25:52,  9.82s/it]
 96%|█████████▌| 216046/225000 [420:21:41<23:06:59,  9.29s/it]
 96%|█████████▌| 216047/225000 [420:21:45<19:31:24,  7.85s/it]
 96%|█████████▌| 216048/225000 [420:21:55<20:43:09,  8.33s/it]
 96%|█████████▌| 216049/225000 [420:22:00<18:13:08,  7.33s/it]
 96%|█████████▌| 216050/225000 [420:22:12<21:44:33,  8.75s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.299', 'learning_rate': '2.183e-07', 'epoch': '0.9602', 'num_input_tokens_seen': 714141232, 'train_runtime': '1.513e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 216050/225000 [420:22:12<21:44:33,  8.75s/it]
 96%|█████████▌| 216051/225000 [420:22:20<21:24:43,  8.61s/it]
 96%|█████████▌| 216052/225000 [420:22:29<21:25:49,  8.62s/it]
 96%|█████████▌| 216053/225000 [420:22:38<21:56:31,  8.83s/it]
 96%|█████████▌| 216054/225000 [420:22:48<22:45:05,  9.16s/it]
 96%|█████████▌| 216055/225000 [420:23:00<25:02:06, 10.08s/it]
 96%|█████████▌| 216056/225000 [420:23:05<21:19:30,  8.58s/it]
 96%|█████████▌| 216057/225000 [420:23:14<21:09:49,  8.52s/it]
 96%|█████████▌| 216058/225000 [420:23:19<18:41:06,  7.52s/it]
 96%|█████████▌| 216059/225000 [420:23:27<19:08:02,  7.70s/it]
 96%|█████████▌| 216060/225000 [420:23:33<17:58:41,  7.24s/it]
 96%|█████████▌| 216061/225000 [420:23:44<20:53:56,  8.42s/it]
 96%|█████████▌| 216062/225000 [420:23:51<19:29:59,  7.85s/it]
 96%|█████████▌| 216063/225000 [420:23:56<17:47:01,  7.16s/it]
 96%|█████████▌| 216064/225000 [420:24:04<17:57:38,  7.24s/it]
 96%|█████████▌| 216065/225000 [420:24:15<20:42:53,  8.35s/it]
 96%|█████████▌| 216066/225000 [420:24:22<19:45:23,  7.96s/it]
 96%|█████████▌| 216067/225000 [420:24:29<19:38:21,  7.91s/it]
 96%|█████████▌| 216068/225000 [420:24:36<18:29:41,  7.45s/it]
 96%|█████████▌| 216069/225000 [420:24:46<20:23:45,  8.22s/it]
 96%|█████████▌| 216070/225000 [420:25:00<24:55:24, 10.05s/it]
 96%|█████████▌| 216071/225000 [420:25:08<23:26:05,  9.45s/it]
 96%|█████████▌| 216072/225000 [420:25:14<20:57:07,  8.45s/it]
 96%|█████████▌| 216073/225000 [420:25:27<24:09:16,  9.74s/it]
 96%|█████████▌| 216074/225000 [420:25:35<22:49:17,  9.20s/it]
 96%|█████████▌| 216075/225000 [420:25:45<23:06:54,  9.32s/it]
 96%|█████████▌| 216076/225000 [420:25:53<22:36:13,  9.12s/it]
 96%|█████████▌| 216077/225000 [420:26:02<22:00:24,  8.88s/it]
 96%|█████████▌| 216078/225000 [420:26:13<24:14:11,  9.78s/it]
 96%|█████████▌| 216079/225000 [420:26:19<21:24:15,  8.64s/it]
 96%|█████████▌| 216080/225000 [420:26:30<22:59:32,  9.28s/it]
 96%|█████████▌| 216081/225000 [420:26:37<21:28:10,  8.67s/it]
 96%|█████████▌| 216082/225000 [420:26:50<24:33:56,  9.92s/it]
 96%|█████████▌| 216083/225000 [420:26:57<21:48:50,  8.81s/it]
 96%|█████████▌| 216084/225000 [420:27:04<20:59:34,  8.48s/it]
 96%|█████████▌| 216085/225000 [420:27:13<21:06:01,  8.52s/it]
 96%|█████████▌| 216086/225000 [420:27:24<23:07:37,  9.34s/it]
 96%|█████████▌| 216087/225000 [420:27:32<22:14:31,  8.98s/it]
 96%|█████████▌| 216088/225000 [420:27:42<22:51:29,  9.23s/it]
 96%|█████████▌| 216089/225000 [420:27:49<21:16:45,  8.60s/it]
 96%|█████████▌| 216090/225000 [420:28:03<24:55:44, 10.07s/it]
 96%|█████████▌| 216091/225000 [420:28:09<22:15:13,  8.99s/it]
 96%|█████████▌| 216092/225000 [420:28:20<23:39:27,  9.56s/it]
 96%|█████████▌| 216093/225000 [420:28:26<20:51:29,  8.43s/it]
 96%|█████████▌| 216094/225000 [420:28:32<19:27:39,  7.87s/it]
 96%|█████████▌| 216095/225000 [420:28:40<19:24:25,  7.85s/it]
 96%|█████████▌| 216096/225000 [420:28:47<18:16:36,  7.39s/it]
 96%|█████████▌| 216097/225000 [420:29:01<23:12:43,  9.39s/it]
 96%|█████████▌| 216098/225000 [420:29:08<21:55:18,  8.87s/it]
 96%|█████████▌| 216099/225000 [420:29:16<21:28:34,  8.69s/it]
 96%|█████████▌| 216100/225000 [420:29:24<20:45:48,  8.40s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.455', 'learning_rate': '2.159e-07', 'epoch': '0.9604', 'num_input_tokens_seen': 714374312, 'train_runtime': '1.514e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 216100/225000 [420:29:24<20:45:48,  8.40s/it]
 96%|█████████▌| 216101/225000 [420:29:35<22:31:44,  9.11s/it]
 96%|█████████▌| 216102/225000 [420:29:45<23:31:26,  9.52s/it]
 96%|█████████▌| 216103/225000 [420:29:57<25:18:49, 10.24s/it]
 96%|█████████▌| 216104/225000 [420:30:07<24:35:19,  9.95s/it]
 96%|█████████▌| 216105/225000 [420:30:17<24:49:01, 10.04s/it]
 96%|█████████▌| 216106/225000 [420:30:25<23:11:44,  9.39s/it]
 96%|█████████▌| 216107/225000 [420:30:32<21:46:59,  8.82s/it]
 96%|█████████▌| 216108/225000 [420:30:41<21:27:00,  8.68s/it]
 96%|█████████▌| 216109/225000 [420:30:51<22:52:45,  9.26s/it]
 96%|█████████▌| 216110/225000 [420:31:01<23:35:00,  9.55s/it]
 96%|█████████▌| 216111/225000 [420:31:09<21:58:41,  8.90s/it]
 96%|█████████▌| 216112/225000 [420:31:15<19:45:05,  8.00s/it]
 96%|█████████▌| 216113/225000 [420:31:27<22:42:59,  9.20s/it]
 96%|█████████▌| 216114/225000 [420:31:33<20:53:00,  8.46s/it]
 96%|█████████▌| 216115/225000 [420:31:47<24:40:16, 10.00s/it]
 96%|█████████▌| 216116/225000 [420:31:54<22:31:42,  9.13s/it]
 96%|█████████▌| 216117/225000 [420:32:02<21:17:46,  8.63s/it]
 96%|█████████▌| 216118/225000 [420:32:11<21:29:12,  8.71s/it]
 96%|█████████▌| 216119/225000 [420:32:22<23:28:42,  9.52s/it]
 96%|█████████▌| 216120/225000 [420:32:30<22:10:25,  8.99s/it]
 96%|█████████▌| 216121/225000 [420:32:42<24:31:41,  9.94s/it]
 96%|█████████▌| 216122/225000 [420:32:47<20:58:10,  8.50s/it]
 96%|█████████▌| 216123/225000 [420:32:57<22:24:17,  9.09s/it]
 96%|█████████▌| 216124/225000 [420:33:06<21:38:43,  8.78s/it]
 96%|█████████▌| 216125/225000 [420:33:14<21:14:12,  8.61s/it]
 96%|█████████▌| 216126/225000 [420:33:22<20:37:19,  8.37s/it]
 96%|█████████▌| 216127/225000 [420:33:28<19:32:27,  7.93s/it]
 96%|█████████▌| 216128/225000 [420:33:42<23:24:39,  9.50s/it]
 96%|█████████▌| 216129/225000 [420:33:47<20:11:39,  8.20s/it]
 96%|█████████▌| 216130/225000 [420:33:57<21:21:16,  8.67s/it]
 96%|█████████▌| 216131/225000 [420:34:02<18:43:58,  7.60s/it]
 96%|█████████▌| 216132/225000 [420:34:14<22:05:40,  8.97s/it]
 96%|█████████▌| 216133/225000 [420:34:21<20:43:39,  8.42s/it]
 96%|█████████▌| 216134/225000 [420:34:27<18:40:46,  7.58s/it]
 96%|█████████▌| 216135/225000 [420:34:37<20:50:30,  8.46s/it]
 96%|█████████▌| 216136/225000 [420:34:45<20:37:01,  8.37s/it]
 96%|█████████▌| 216137/225000 [420:34:54<20:50:03,  8.46s/it]
 96%|█████████▌| 216138/225000 [420:35:01<19:44:48,  8.02s/it]
 96%|█████████▌| 216139/225000 [420:35:11<21:13:32,  8.62s/it]
 96%|█████████▌| 216140/225000 [420:35:16<18:13:07,  7.40s/it]
 96%|█████████▌| 216141/225000 [420:35:27<21:20:20,  8.67s/it]
 96%|█████████▌| 216142/225000 [420:35:32<18:24:14,  7.48s/it]
 96%|█████████▌| 216143/225000 [420:35:39<18:14:55,  7.42s/it]
 96%|█████████▌| 216144/225000 [420:35:46<17:53:25,  7.27s/it]
 96%|█████████▌| 216145/225000 [420:35:52<16:58:49,  6.90s/it]
 96%|█████████▌| 216146/225000 [420:36:07<23:02:53,  9.37s/it]
 96%|█████████▌| 216147/225000 [420:36:18<24:01:29,  9.77s/it]
 96%|█████████▌| 216148/225000 [420:36:25<22:15:55,  9.06s/it]
 96%|█████████▌| 216149/225000 [420:36:35<22:44:03,  9.25s/it]
 96%|█████████▌| 216150/225000 [420:36:44<22:20:14,  9.09s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.913', 'learning_rate': '2.135e-07', 'epoch': '0.9607', 'num_input_tokens_seen': 714606664, 'train_runtime': '1.514e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 216150/225000 [420:36:44<22:20:14,  9.09s/it]
 96%|█████████▌| 216151/225000 [420:36:53<22:19:23,  9.08s/it]
 96%|█████████▌| 216152/225000 [420:37:02<22:30:25,  9.16s/it]
 96%|█████████▌| 216153/225000 [420:37:10<21:38:28,  8.81s/it]
 96%|█████████▌| 216154/225000 [420:37:24<25:19:05, 10.30s/it]
 96%|█████████▌| 216155/225000 [420:37:33<24:29:59,  9.97s/it]
 96%|█████████▌| 216156/225000 [420:37:40<22:19:51,  9.09s/it]
 96%|█████████▌| 216157/225000 [420:37:48<21:30:27,  8.76s/it]
 96%|█████████▌| 216158/225000 [420:37:54<19:34:15,  7.97s/it]
 96%|█████████▌| 216159/225000 [420:38:07<23:22:48,  9.52s/it]
 96%|█████████▌| 216160/225000 [420:38:19<24:35:08, 10.01s/it]
 96%|█████████▌| 216161/225000 [420:38:24<21:21:50,  8.70s/it]
 96%|█████████▌| 216162/225000 [420:38:29<18:28:45,  7.53s/it]
 96%|█████████▌| 216163/225000 [420:38:40<20:53:07,  8.51s/it]
 96%|█████████▌| 216164/225000 [420:38:46<19:12:24,  7.83s/it]
 96%|█████████▌| 216165/225000 [420:38:52<18:04:06,  7.36s/it]
 96%|█████████▌| 216166/225000 [420:39:04<21:05:13,  8.59s/it]
 96%|█████████▌| 216167/225000 [420:39:16<23:28:17,  9.57s/it]
 96%|█████████▌| 216168/225000 [420:39:25<23:22:57,  9.53s/it]
 96%|█████████▌| 216169/225000 [420:39:36<24:19:11,  9.91s/it]
 96%|█████████▌| 216170/225000 [420:39:46<24:24:16,  9.95s/it]
 96%|█████████▌| 216171/225000 [420:39:55<23:56:18,  9.76s/it]
 96%|█████████▌| 216172/225000 [420:40:02<21:44:41,  8.87s/it]
 96%|█████████▌| 216173/225000 [420:40:08<19:42:43,  8.04s/it]
 96%|█████████▌| 216174/225000 [420:40:14<18:24:12,  7.51s/it]
 96%|█████████▌| 216175/225000 [420:40:25<20:45:43,  8.47s/it]
 96%|█████████▌| 216176/225000 [420:40:35<21:57:56,  8.96s/it]
 96%|█████████▌| 216177/225000 [420:40:46<23:29:44,  9.59s/it]
 96%|█████████▌| 216178/225000 [420:40:54<21:49:17,  8.90s/it]
 96%|█████████▌| 216179/225000 [420:41:02<21:48:47,  8.90s/it]
 96%|█████████▌| 216180/225000 [420:41:10<20:51:22,  8.51s/it]
 96%|█████████▌| 216181/225000 [420:41:17<19:59:04,  8.16s/it]
 96%|█████████▌| 216182/225000 [420:41:27<20:45:22,  8.47s/it]
 96%|█████████▌| 216183/225000 [420:41:39<23:23:30,  9.55s/it]
 96%|█████████▌| 216184/225000 [420:41:47<22:49:26,  9.32s/it]
 96%|█████████▌| 216185/225000 [420:41:55<21:36:35,  8.83s/it]
 96%|█████████▌| 216186/225000 [420:42:03<20:39:15,  8.44s/it]
 96%|█████████▌| 216187/225000 [420:42:14<22:29:46,  9.19s/it]
 96%|█████████▌| 216188/225000 [420:42:26<24:33:58, 10.04s/it]
 96%|█████████▌| 216189/225000 [420:42:30<20:42:01,  8.46s/it]
 96%|█████████▌| 216190/225000 [420:42:43<23:53:42,  9.76s/it]
 96%|█████████▌| 216191/225000 [420:42:51<22:39:53,  9.26s/it]
 96%|█████████▌| 216192/225000 [420:42:58<20:28:22,  8.37s/it]
 96%|█████████▌| 216193/225000 [420:43:05<19:27:24,  7.95s/it]
 96%|█████████▌| 216194/225000 [420:43:19<24:03:06,  9.83s/it]
 96%|█████████▌| 216195/225000 [420:43:24<20:55:24,  8.55s/it]
 96%|█████████▌| 216196/225000 [420:43:33<20:43:07,  8.47s/it]
 96%|█████████▌| 216197/225000 [420:43:42<21:08:20,  8.64s/it]
 96%|█████████▌| 216198/225000 [420:43:49<19:51:49,  8.12s/it]
 96%|█████████▌| 216199/225000 [420:44:03<24:45:58, 10.13s/it]
 96%|█████████▌| 216200/225000 [420:44:14<25:28:39, 10.42s/it]
                                                              
{'loss': '1.302', 'grad_norm': '1.181', 'learning_rate': '2.112e-07', 'epoch': '0.9609', 'num_input_tokens_seen': 714841800, 'train_runtime': '1.515e+06', 'train_tokens_per_second': '471.9'}

 96%|█████████▌| 216200/225000 [420:44:14<25:28:39, 10.42s/it]
 96%|█████████▌| 216201/225000 [420:44:24<24:45:18, 10.13s/it]
 96%|█████████▌| 216202/225000 [420:44:31<22:34:32,  9.24s/it]
 96%|█████████▌| 216203/225000 [420:44:38<21:01:50,  8.61s/it]
 96%|█████████▌| 216204/225000 [420:44:49<22:43:15,  9.30s/it]
 96%|█████████▌| 216205/225000 [420:44:57<21:30:58,  8.81s/it]
 96%|█████████▌| 216206/225000 [420:45:03<19:41:30,  8.06s/it]
 96%|█████████▌| 216207/225000 [420:45:07<16:47:24,  6.87s/it]
 96%|█████████▌| 216208/225000 [420:45:17<19:02:13,  7.79s/it]
 96%|█████████▌| 216209/225000 [420:45:24<18:28:47,  7.57s/it]
 96%|█████████▌| 216210/225000 [420:45:34<19:57:52,  8.18s/it]
 96%|█████████▌| 216211/225000 [420:45:46<23:12:57,  9.51s/it]
 96%|█████████▌| 216212/225000 [420:45:54<21:50:24,  8.95s/it]
 96%|█████████▌| 216213/225000 [420:46:03<21:48:47,  8.94s/it]
 96%|█████████▌| 216214/225000 [420:46:15<23:44:06,  9.73s/it]
 96%|█████████▌| 216215/225000 [420:46:19<19:54:10,  8.16s/it]
 96%|█████████▌| 216216/225000 [420:46:29<21:21:29,  8.75s/it]
 96%|█████████▌| 216217/225000 [420:46:34<18:44:17,  7.68s/it]
 96%|█████████▌| 216218/225000 [420:46:46<21:23:59,  8.77s/it]
 96%|█████████▌| 216219/225000 [420:46:54<20:43:34,  8.50s/it]
 96%|█████████▌| 216220/225000 [420:47:01<19:40:55,  8.07s/it]
 96%|█████████▌| 216221/225000 [420:47:11<21:25:49,  8.79s/it]
 96%|█████████▌| 216222/225000 [420:47:21<22:14:11,  9.12s/it]
 96%|█████████▌| 216223/225000 [420:47:26<19:29:42,  8.00s/it]
 96%|█████████▌| 216224/225000 [420:47:38<22:12:54,  9.11s/it]
 96%|█████████▌| 216225/225000 [420:47:48<23:06:05,  9.48s/it]
 96%|█████████▌| 216226/225000 [420:47:55<21:14:55,  8.72s/it]
 96%|█████████▌| 216227/225000 [420:48:05<22:11:30,  9.11s/it]
 96%|█████████▌| 216228/225000 [420:48:18<24:31:00, 10.06s/it]
 96%|█████████▌| 216229/225000 [420:48:26<23:01:12,  9.45s/it]
 96%|█████████▌| 216230/225000 [420:48:35<23:00:43,  9.45s/it]
 96%|█████████▌| 216231/225000 [420:48:42<21:24:55,  8.79s/it]
 96%|█████████▌| 216232/225000 [420:48:49<19:37:30,  8.06s/it]
 96%|█████████▌| 216233/225000 [420:48:56<19:04:09,  7.83s/it]
 96%|█████████▌| 216234/225000 [420:49:05<20:14:42,  8.31s/it]
 96%|█████████▌| 216235/225000 [420:49:14<20:35:51,  8.46s/it]
 96%|█████████▌| 216236/225000 [420:49:25<22:33:16,  9.26s/it]
 96%|█████████▌| 216237/225000 [420:49:32<20:46:02,  8.53s/it]
 96%|█████████▌| 216238/225000 [420:49:39<19:10:33,  7.88s/it]
 96%|█████████▌| 216239/225000 [420:49:52<23:11:50,  9.53s/it]
 96%|█████████▌| 216240/225000 [420:50:01<22:40:29,  9.32s/it]
 96%|█████████▌| 216241/225000 [420:50:07<20:35:38,  8.46s/it]
 96%|█████████▌| 216242/225000 [420:50:15<20:07:06,  8.27s/it]
 96%|█████████▌| 216243/225000 [420:50:31<25:35:23, 10.52s/it]
 96%|█████████▌| 216244/225000 [420:50:41<25:15:33, 10.39s/it]
 96%|█████████▌| 216245/225000 [420:50:47<21:49:16,  8.97s/it]
 96%|█████████▌| 216246/225000 [420:50:57<22:41:58,  9.33s/it]
 96%|█████████▌| 216247/225000 [420:51:09<24:37:38, 10.13s/it]
 96%|█████████▌| 216248/225000 [420:51:16<22:44:01,  9.35s/it]
 96%|█████████▌| 216249/225000 [420:51:22<19:52:38,  8.18s/it]
 96%|█████████▌| 216250/225000 [420:51:35<23:27:46,  9.65s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.122', 'learning_rate': '2.088e-07', 'epoch': '0.9611', 'num_input_tokens_seen': 715073416, 'train_runtime': '1.515e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216250/225000 [420:51:35<23:27:46,  9.65s/it]
 96%|█████████▌| 216251/225000 [420:51:44<23:07:43,  9.52s/it]
 96%|█████████▌| 216252/225000 [420:51:48<19:19:05,  7.95s/it]
 96%|█████████▌| 216253/225000 [420:51:58<20:19:03,  8.36s/it]
 96%|█████████▌| 216254/225000 [420:52:08<21:48:50,  8.98s/it]
 96%|█████████▌| 216255/225000 [420:52:14<19:34:57,  8.06s/it]
 96%|█████████▌| 216256/225000 [420:52:28<24:04:53,  9.91s/it]
 96%|█████████▌| 216257/225000 [420:52:35<21:51:18,  9.00s/it]
 96%|█████████▌| 216258/225000 [420:52:43<21:12:09,  8.73s/it]
 96%|█████████▌| 216259/225000 [420:52:56<24:08:24,  9.94s/it]
 96%|█████████▌| 216260/225000 [420:53:04<22:29:40,  9.27s/it]
 96%|█████████▌| 216261/225000 [420:53:10<20:29:32,  8.44s/it]
 96%|█████████▌| 216262/225000 [420:53:20<21:19:00,  8.78s/it]
 96%|█████████▌| 216263/225000 [420:53:28<20:52:51,  8.60s/it]
 96%|█████████▌| 216264/225000 [420:53:33<18:10:51,  7.49s/it]
 96%|█████████▌| 216265/225000 [420:53:43<20:21:40,  8.39s/it]
 96%|█████████▌| 216266/225000 [420:53:50<19:03:13,  7.85s/it]
 96%|█████████▌| 216267/225000 [420:54:02<22:04:24,  9.10s/it]
 96%|█████████▌| 216268/225000 [420:54:11<22:04:32,  9.10s/it]
 96%|█████████▌| 216269/225000 [420:54:20<22:01:05,  9.08s/it]
 96%|█████████▌| 216270/225000 [420:54:26<19:31:21,  8.05s/it]
 96%|█████████▌| 216271/225000 [420:54:32<18:19:29,  7.56s/it]
 96%|█████████▌| 216272/225000 [420:54:43<20:29:13,  8.45s/it]
 96%|█████████▌| 216273/225000 [420:54:53<22:01:19,  9.08s/it]
 96%|█████████▌| 216274/225000 [420:54:58<18:38:16,  7.69s/it]
 96%|█████████▌| 216275/225000 [420:55:08<20:50:30,  8.60s/it]
 96%|█████████▌| 216276/225000 [420:55:14<18:32:29,  7.65s/it]
 96%|█████████▌| 216277/225000 [420:55:20<17:43:02,  7.31s/it]
 96%|█████████▌| 216278/225000 [420:55:36<23:46:41,  9.81s/it]
 96%|█████████▌| 216279/225000 [420:55:42<21:04:51,  8.70s/it]
 96%|█████████▌| 216280/225000 [420:55:54<23:36:32,  9.75s/it]
 96%|█████████▌| 216281/225000 [420:56:05<24:04:12,  9.94s/it]
 96%|█████████▌| 216282/225000 [420:56:10<21:04:51,  8.71s/it]
 96%|█████████▌| 216283/225000 [420:56:19<21:04:12,  8.70s/it]
 96%|█████████▌| 216284/225000 [420:56:28<21:09:24,  8.74s/it]
 96%|█████████▌| 216285/225000 [420:56:41<24:03:55,  9.94s/it]
 96%|█████████▌| 216286/225000 [420:56:48<21:47:33,  9.00s/it]
 96%|█████████▌| 216287/225000 [420:56:58<23:12:01,  9.59s/it]
 96%|█████████▌| 216288/225000 [420:57:03<19:23:11,  8.01s/it]
 96%|█████████▌| 216289/225000 [420:57:11<19:40:59,  8.13s/it]
 96%|█████████▌| 216290/225000 [420:57:18<18:59:44,  7.85s/it]
 96%|█████████▌| 216291/225000 [420:57:28<20:14:29,  8.37s/it]
 96%|█████████▌| 216292/225000 [420:57:39<22:31:09,  9.31s/it]
 96%|█████████▌| 216293/225000 [420:57:47<21:30:43,  8.89s/it]
 96%|█████████▌| 216294/225000 [420:58:00<24:31:21, 10.14s/it]
 96%|█████████▌| 216295/225000 [420:58:08<22:40:10,  9.38s/it]
 96%|█████████▌| 216296/225000 [420:58:16<21:49:28,  9.03s/it]
 96%|█████████▌| 216297/225000 [420:58:27<23:17:41,  9.64s/it]
 96%|█████████▌| 216298/225000 [420:58:37<23:26:04,  9.69s/it]
 96%|█████████▌| 216299/225000 [420:58:47<23:21:49,  9.67s/it]
 96%|█████████▌| 216300/225000 [420:58:56<23:06:41,  9.56s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.272', 'learning_rate': '2.065e-07', 'epoch': '0.9613', 'num_input_tokens_seen': 715304656, 'train_runtime': '1.516e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216300/225000 [420:58:56<23:06:41,  9.56s/it]
 96%|█████████▌| 216301/225000 [420:59:05<22:18:52,  9.23s/it]
 96%|█████████▌| 216302/225000 [420:59:11<20:23:54,  8.44s/it]
 96%|█████████▌| 216303/225000 [420:59:22<22:15:58,  9.22s/it]
 96%|█████████▌| 216304/225000 [420:59:33<23:44:40,  9.83s/it]
 96%|█████████▌| 216305/225000 [420:59:42<23:00:06,  9.52s/it]
 96%|█████████▌| 216306/225000 [420:59:54<24:25:33, 10.11s/it]
 96%|█████████▌| 216307/225000 [421:00:00<21:22:43,  8.85s/it]
 96%|█████████▌| 216308/225000 [421:00:05<18:40:35,  7.74s/it]
 96%|█████████▌| 216309/225000 [421:00:10<16:43:32,  6.93s/it]
 96%|█████████▌| 216310/225000 [421:00:20<19:01:53,  7.88s/it]
 96%|█████████▌| 216311/225000 [421:00:32<22:10:14,  9.19s/it]
 96%|█████████▌| 216312/225000 [421:00:42<22:20:33,  9.26s/it]
 96%|█████████▌| 216313/225000 [421:00:47<19:50:49,  8.22s/it]
 96%|█████████▌| 216314/225000 [421:00:56<20:04:23,  8.32s/it]
 96%|█████████▌| 216315/225000 [421:01:05<20:44:53,  8.60s/it]
 96%|█████████▌| 216316/225000 [421:01:11<18:51:33,  7.82s/it]
 96%|█████████▌| 216317/225000 [421:01:22<21:00:29,  8.71s/it]
 96%|█████████▌| 216318/225000 [421:01:28<18:58:13,  7.87s/it]
 96%|█████████▌| 216319/225000 [421:01:41<22:56:00,  9.51s/it]
 96%|█████████▌| 216320/225000 [421:01:53<24:37:17, 10.21s/it]
 96%|█████████▌| 216321/225000 [421:02:02<23:42:56,  9.84s/it]
 96%|█████████▌| 216322/225000 [421:02:13<24:22:09, 10.11s/it]
 96%|█████████▌| 216323/225000 [421:02:22<24:04:43,  9.99s/it]
 96%|█████████▌| 216324/225000 [421:02:27<20:08:02,  8.35s/it]
 96%|█████████▌| 216325/225000 [421:02:37<21:07:14,  8.76s/it]
 96%|█████████▌| 216326/225000 [421:02:46<21:33:18,  8.95s/it]
 96%|█████████▌| 216327/225000 [421:02:52<19:02:54,  7.91s/it]
 96%|█████████▌| 216328/225000 [421:03:00<19:27:24,  8.08s/it]
 96%|█████████▌| 216329/225000 [421:03:10<20:37:04,  8.56s/it]
 96%|█████████▌| 216330/225000 [421:03:20<21:48:56,  9.06s/it]
 96%|█████████▌| 216331/225000 [421:03:26<19:42:49,  8.19s/it]
 96%|█████████▌| 216332/225000 [421:03:32<18:12:36,  7.56s/it]
 96%|█████████▌| 216333/225000 [421:03:46<22:20:49,  9.28s/it]
 96%|█████████▌| 216334/225000 [421:03:55<22:25:23,  9.31s/it]
 96%|█████████▌| 216335/225000 [421:04:08<24:47:06, 10.30s/it]
 96%|█████████▌| 216336/225000 [421:04:14<22:11:40,  9.22s/it]
 96%|█████████▌| 216337/225000 [421:04:25<23:06:03,  9.60s/it]
 96%|█████████▌| 216338/225000 [421:04:33<22:31:02,  9.36s/it]
 96%|█████████▌| 216339/225000 [421:04:41<21:15:37,  8.84s/it]
 96%|█████████▌| 216340/225000 [421:04:48<19:59:55,  8.31s/it]
 96%|█████████▌| 216341/225000 [421:04:54<18:05:42,  7.52s/it]
 96%|█████████▌| 216342/225000 [421:05:04<20:18:39,  8.45s/it]
 96%|█████████▌| 216343/225000 [421:05:10<17:52:10,  7.43s/it]
 96%|█████████▌| 216344/225000 [421:05:19<19:22:14,  8.06s/it]
 96%|█████████▌| 216345/225000 [421:05:31<22:14:45,  9.25s/it]
 96%|█████████▌| 216346/225000 [421:05:41<22:21:30,  9.30s/it]
 96%|█████████▌| 216347/225000 [421:05:50<22:26:11,  9.33s/it]
 96%|█████████▌| 216348/225000 [421:06:06<27:24:58, 11.41s/it]
 96%|█████████▌| 216349/225000 [421:06:14<24:33:29, 10.22s/it]
 96%|█████████▌| 216350/225000 [421:06:20<21:42:27,  9.03s/it]
                                                              
{'loss': '1.303', 'grad_norm': '1.383', 'learning_rate': '2.042e-07', 'epoch': '0.9616', 'num_input_tokens_seen': 715535440, 'train_runtime': '1.516e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216350/225000 [421:06:20<21:42:27,  9.03s/it]
 96%|█████████▌| 216351/225000 [421:06:26<19:20:21,  8.05s/it]
 96%|█████████▌| 216352/225000 [421:06:35<20:22:46,  8.48s/it]
 96%|█████████▌| 216353/225000 [421:06:46<22:13:54,  9.26s/it]
 96%|█████████▌| 216354/225000 [421:06:52<19:40:33,  8.19s/it]
 96%|█████████▌| 216355/225000 [421:07:02<21:15:58,  8.86s/it]
 96%|█████████▌| 216356/225000 [421:07:10<20:04:08,  8.36s/it]
 96%|█████████▌| 216357/225000 [421:07:17<19:31:38,  8.13s/it]
 96%|█████████▌| 216358/225000 [421:07:26<19:55:32,  8.30s/it]
 96%|█████████▌| 216359/225000 [421:07:36<21:05:18,  8.79s/it]
 96%|█████████▌| 216360/225000 [421:07:43<19:44:25,  8.23s/it]
 96%|█████████▌| 216361/225000 [421:07:49<18:24:00,  7.67s/it]
 96%|█████████▌| 216362/225000 [421:08:00<20:38:20,  8.60s/it]
 96%|█████████▌| 216363/225000 [421:08:08<20:32:30,  8.56s/it]
 96%|█████████▌| 216364/225000 [421:08:17<20:55:30,  8.72s/it]
 96%|█████████▌| 216365/225000 [421:08:25<20:12:36,  8.43s/it]
 96%|█████████▌| 216366/225000 [421:08:38<23:27:51,  9.78s/it]
 96%|█████████▌| 216367/225000 [421:08:43<20:00:36,  8.34s/it]
 96%|█████████▌| 216368/225000 [421:08:50<19:00:31,  7.93s/it]
 96%|█████████▌| 216369/225000 [421:09:00<20:45:52,  8.66s/it]
 96%|█████████▌| 216370/225000 [421:09:09<20:42:51,  8.64s/it]
 96%|█████████▌| 216371/225000 [421:09:14<17:51:30,  7.45s/it]
 96%|█████████▌| 216372/225000 [421:09:20<17:23:56,  7.26s/it]
 96%|█████████▌| 216373/225000 [421:09:33<21:31:04,  8.98s/it]
 96%|█████████▌| 216374/225000 [421:09:43<21:47:05,  9.09s/it]
 96%|█████████▌| 216375/225000 [421:09:52<21:33:06,  9.00s/it]
 96%|█████████▌| 216376/225000 [421:10:01<21:47:10,  9.09s/it]
 96%|█████████▌| 216377/225000 [421:10:09<21:13:53,  8.86s/it]
 96%|█████████▌| 216378/225000 [421:10:15<18:48:54,  7.86s/it]
 96%|█████████▌| 216379/225000 [421:10:22<18:18:28,  7.65s/it]
 96%|█████████▌| 216380/225000 [421:10:36<22:44:41,  9.50s/it]
 96%|█████████▌| 216381/225000 [421:10:43<21:07:40,  8.82s/it]
 96%|█████████▌| 216382/225000 [421:10:51<20:44:45,  8.67s/it]
 96%|█████████▌| 216383/225000 [421:11:03<23:06:05,  9.65s/it]
 96%|█████████▌| 216384/225000 [421:11:07<19:04:33,  7.97s/it]
 96%|█████████▌| 216385/225000 [421:11:17<20:10:02,  8.43s/it]
 96%|█████████▌| 216386/225000 [421:11:25<20:08:05,  8.41s/it]
 96%|█████████▌| 216387/225000 [421:11:34<20:30:31,  8.57s/it]
 96%|█████████▌| 216388/225000 [421:11:41<19:36:26,  8.20s/it]
 96%|█████████▌| 216389/225000 [421:11:48<18:24:41,  7.70s/it]
 96%|█████████▌| 216390/225000 [421:12:02<22:50:38,  9.55s/it]
 96%|█████████▌| 216391/225000 [421:12:10<21:47:34,  9.11s/it]
 96%|█████████▌| 216392/225000 [421:12:19<21:44:21,  9.09s/it]
 96%|█████████▌| 216393/225000 [421:12:27<21:20:41,  8.93s/it]
 96%|█████████▌| 216394/225000 [421:12:36<20:58:05,  8.77s/it]
 96%|█████████▌| 216395/225000 [421:12:43<19:51:20,  8.31s/it]
 96%|█████████▌| 216396/225000 [421:12:51<19:16:41,  8.07s/it]
 96%|█████████▌| 216397/225000 [421:13:00<19:52:25,  8.32s/it]
 96%|█████████▌| 216398/225000 [421:13:10<21:25:21,  8.97s/it]
 96%|█████████▌| 216399/225000 [421:13:18<21:00:12,  8.79s/it]
 96%|█████████▌| 216400/225000 [421:13:23<18:09:48,  7.60s/it]
                                                              
{'loss': '1.303', 'grad_norm': '2.214', 'learning_rate': '2.018e-07', 'epoch': '0.9618', 'num_input_tokens_seen': 715760792, 'train_runtime': '1.516e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216400/225000 [421:13:23<18:09:48,  7.60s/it]
 96%|█████████▌| 216401/225000 [421:13:36<22:04:40,  9.24s/it]
 96%|█████████▌| 216402/225000 [421:13:45<21:22:42,  8.95s/it]
 96%|█████████▌| 216403/225000 [421:13:52<20:39:40,  8.65s/it]
 96%|█████████▌| 216404/225000 [421:13:58<18:14:07,  7.64s/it]
 96%|█████████▌| 216405/225000 [421:14:12<22:38:24,  9.48s/it]
 96%|█████████▌| 216406/225000 [421:14:18<20:19:38,  8.52s/it]
 96%|█████████▌| 216407/225000 [421:14:30<23:04:06,  9.66s/it]
 96%|█████████▌| 216408/225000 [421:14:38<21:31:47,  9.02s/it]
 96%|█████████▌| 216409/225000 [421:14:45<20:18:12,  8.51s/it]
 96%|█████████▌| 216410/225000 [421:14:53<20:11:21,  8.46s/it]
 96%|█████████▌| 216411/225000 [421:15:00<19:10:20,  8.04s/it]
 96%|█████████▌| 216412/225000 [421:15:11<20:40:28,  8.67s/it]
 96%|█████████▌| 216413/225000 [421:15:16<18:44:43,  7.86s/it]
 96%|█████████▌| 216414/225000 [421:15:25<19:33:06,  8.20s/it]
 96%|█████████▌| 216415/225000 [421:15:33<19:02:41,  7.99s/it]
 96%|█████████▌| 216416/225000 [421:15:39<17:21:43,  7.28s/it]
 96%|█████████▌| 216417/225000 [421:15:47<18:28:27,  7.75s/it]
 96%|█████████▌| 216418/225000 [421:16:03<24:08:34, 10.13s/it]
 96%|█████████▌| 216419/225000 [421:16:12<23:15:54,  9.76s/it]
 96%|█████████▌| 216420/225000 [421:16:20<21:38:54,  9.08s/it]
 96%|█████████▌| 216421/225000 [421:16:28<21:12:17,  8.90s/it]
 96%|█████████▌| 216422/225000 [421:16:37<21:29:26,  9.02s/it]
 96%|█████████▌| 216423/225000 [421:16:46<21:03:44,  8.84s/it]
 96%|█████████▌| 216424/225000 [421:16:54<20:33:18,  8.63s/it]
 96%|█████████▌| 216425/225000 [421:17:06<23:00:39,  9.66s/it]
 96%|█████████▌| 216426/225000 [421:17:16<23:03:11,  9.68s/it]
 96%|█████████▌| 216427/225000 [421:17:21<20:07:48,  8.45s/it]
 96%|█████████▌| 216428/225000 [421:17:29<19:42:42,  8.28s/it]
 96%|█████████▌| 216429/225000 [421:17:35<18:04:30,  7.59s/it]
 96%|█████████▌| 216430/225000 [421:17:45<19:37:03,  8.24s/it]
 96%|█████████▌| 216431/225000 [421:17:57<22:29:31,  9.45s/it]
 96%|█████████▌| 216432/225000 [421:18:03<20:00:34,  8.41s/it]
 96%|█████████▌| 216433/225000 [421:18:13<21:07:24,  8.88s/it]
 96%|█████████▌| 216434/225000 [421:18:23<21:43:10,  9.13s/it]
 96%|█████████▌| 216435/225000 [421:18:33<22:25:18,  9.42s/it]
 96%|█████████▌| 216436/225000 [421:18:41<21:26:24,  9.01s/it]
 96%|█████████▌| 216437/225000 [421:18:50<21:13:07,  8.92s/it]
 96%|█████████▌| 216438/225000 [421:18:56<19:06:12,  8.03s/it]
 96%|█████████▌| 216439/225000 [421:19:09<22:47:46,  9.59s/it]
 96%|█████████▌| 216440/225000 [421:19:19<23:17:32,  9.80s/it]
 96%|█████████▌| 216441/225000 [421:19:25<20:23:42,  8.58s/it]
 96%|█████████▌| 216442/225000 [421:19:33<20:06:20,  8.46s/it]
 96%|█████████▌| 216443/225000 [421:19:41<19:29:19,  8.20s/it]
 96%|█████████▌| 216444/225000 [421:19:53<22:48:46,  9.60s/it]
 96%|█████████▌| 216445/225000 [421:20:03<22:38:14,  9.53s/it]
 96%|█████████▌| 216446/225000 [421:20:13<22:44:29,  9.57s/it]
 96%|█████████▌| 216447/225000 [421:20:22<22:28:50,  9.46s/it]
 96%|█████████▌| 216448/225000 [421:20:31<22:22:58,  9.42s/it]
 96%|█████████▌| 216449/225000 [421:20:44<24:48:53, 10.45s/it]
 96%|█████████▌| 216450/225000 [421:20:50<21:37:46,  9.11s/it]
                                                              
{'loss': '1.31', 'grad_norm': '1.08', 'learning_rate': '1.995e-07', 'epoch': '0.962', 'num_input_tokens_seen': 715992960, 'train_runtime': '1.517e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216450/225000 [421:20:50<21:37:46,  9.11s/it]
 96%|█████████▌| 216451/225000 [421:21:00<22:04:58,  9.30s/it]
 96%|█████████▌| 216452/225000 [421:21:06<20:01:40,  8.43s/it]
 96%|█████████▌| 216453/225000 [421:21:17<21:48:08,  9.18s/it]
 96%|█████████▌| 216454/225000 [421:21:28<22:55:15,  9.66s/it]
 96%|█████████▌| 216455/225000 [421:21:37<22:59:20,  9.69s/it]
 96%|█████████▌| 216456/225000 [421:21:44<20:32:00,  8.65s/it]
 96%|█████████▌| 216457/225000 [421:21:55<22:28:45,  9.47s/it]
 96%|█████████▌| 216458/225000 [421:22:04<22:22:31,  9.43s/it]
 96%|█████████▌| 216459/225000 [421:22:10<19:23:50,  8.18s/it]
 96%|█████████▌| 216460/225000 [421:22:18<19:21:40,  8.16s/it]
 96%|█████████▌| 216461/225000 [421:22:31<23:03:40,  9.72s/it]
 96%|█████████▌| 216462/225000 [421:22:37<20:00:24,  8.44s/it]
 96%|█████████▌| 216463/225000 [421:22:54<26:12:42, 11.05s/it]
 96%|█████████▌| 216464/225000 [421:23:01<23:31:17,  9.92s/it]
 96%|█████████▌| 216465/225000 [421:23:11<23:31:44,  9.92s/it]
 96%|█████████▌| 216466/225000 [421:23:20<22:55:21,  9.67s/it]
 96%|█████████▌| 216467/225000 [421:23:30<23:23:11,  9.87s/it]
 96%|█████████▌| 216468/225000 [421:23:37<21:04:18,  8.89s/it]
 96%|█████████▌| 216469/225000 [421:23:50<23:57:00, 10.11s/it]
 96%|█████████▌| 216470/225000 [421:23:56<21:09:52,  8.93s/it]
 96%|█████████▌| 216471/225000 [421:24:04<20:20:55,  8.59s/it]
 96%|█████████▌| 216472/225000 [421:24:13<20:38:30,  8.71s/it]
 96%|█████████▌| 216473/225000 [421:24:19<18:41:43,  7.89s/it]
 96%|█████████▌| 216474/225000 [421:24:32<22:34:53,  9.53s/it]
 96%|█████████▌| 216475/225000 [421:24:40<21:19:29,  9.01s/it]
 96%|█████████▌| 216476/225000 [421:24:46<18:49:05,  7.95s/it]
 96%|█████████▌| 216477/225000 [421:24:53<18:37:21,  7.87s/it]
 96%|█████████▌| 216478/225000 [421:24:59<17:25:32,  7.36s/it]
 96%|█████████▌| 216479/225000 [421:25:14<22:49:14,  9.64s/it]
 96%|█████████▌| 216480/225000 [421:25:19<19:34:43,  8.27s/it]
 96%|█████████▌| 216481/225000 [421:25:27<18:57:11,  8.01s/it]
 96%|█████████▌| 216482/225000 [421:25:32<16:57:05,  7.16s/it]
 96%|█████████▌| 216483/225000 [421:25:38<16:04:10,  6.79s/it]
 96%|█████████▌| 216484/225000 [421:25:50<19:42:44,  8.33s/it]
 96%|█████████▌| 216485/225000 [421:25:59<20:26:58,  8.65s/it]
 96%|█████████▌| 216486/225000 [421:26:09<21:06:20,  8.92s/it]
 96%|█████████▌| 216487/225000 [421:26:21<23:42:51, 10.03s/it]
 96%|█████████▌| 216488/225000 [421:26:27<20:21:02,  8.61s/it]
 96%|█████████▌| 216489/225000 [421:26:35<19:53:46,  8.42s/it]
 96%|█████████▌| 216490/225000 [421:26:43<19:32:57,  8.27s/it]
 96%|█████████▌| 216491/225000 [421:26:53<21:12:27,  8.97s/it]
 96%|█████████▌| 216492/225000 [421:27:01<20:05:58,  8.50s/it]
 96%|█████████▌| 216493/225000 [421:27:12<22:13:57,  9.41s/it]
 96%|█████████▌| 216494/225000 [421:27:20<20:59:20,  8.88s/it]
 96%|█████████▌| 216495/225000 [421:27:32<23:31:27,  9.96s/it]
 96%|█████████▌| 216496/225000 [421:27:44<24:52:53, 10.53s/it]
 96%|█████████▌| 216497/225000 [421:27:51<22:03:51,  9.34s/it]
 96%|█████████▌| 216498/225000 [421:27:58<20:35:40,  8.72s/it]
 96%|█████████▌| 216499/225000 [421:28:06<20:21:22,  8.62s/it]
 96%|█████████▌| 216500/225000 [421:28:15<20:00:21,  8.47s/it]
                                                              
{'loss': '1.332', 'grad_norm': '1.319', 'learning_rate': '1.972e-07', 'epoch': '0.9622', 'num_input_tokens_seen': 716229712, 'train_runtime': '1.517e+06', 'train_tokens_per_second': '472'}

 96%|█████████▌| 216500/225000 [421:28:15<20:00:21,  8.47s/it]
 96%|█████████▌| 216501/225000 [421:28:32<26:20:20, 11.16s/it]
 96%|█████████▌| 216502/225000 [421:28:38<22:47:43,  9.66s/it]
 96%|█████████▌| 216503/225000 [421:28:45<20:33:43,  8.71s/it]
 96%|█████████▌| 216504/225000 [421:28:51<19:04:27,  8.08s/it]
 96%|█████████▌| 216505/225000 [421:28:57<17:47:00,  7.54s/it]
 96%|█████████▌| 216506/225000 [421:29:14<23:48:11, 10.09s/it]
 96%|█████████▌| 216507/225000 [421:29:20<21:03:27,  8.93s/it]
 96%|█████████▌| 216508/225000 [421:29:27<19:46:27,  8.38s/it]
 96%|█████████▌| 216509/225000 [421:29:35<19:44:17,  8.37s/it]
 96%|█████████▌| 216510/225000 [421:29:44<20:16:51,  8.60s/it]
 96%|█████████▌| 216511/225000 [421:29:54<21:18:10,  9.03s/it]
 96%|█████████▌| 216512/225000 [421:30:08<24:14:37, 10.28s/it]
 96%|█████████▌| 216513/225000 [421:30:15<22:04:17,  9.36s/it]
 96%|█████████▌| 216514/225000 [421:30:21<20:00:59,  8.49s/it]
 96%|█████████▌| 216515/225000 [421:30:33<22:06:23,  9.38s/it]
 96%|█████████▌| 216516/225000 [421:30:41<21:41:54,  9.21s/it]
 96%|█████████▌| 216517/225000 [421:30:49<20:27:19,  8.68s/it]
 96%|█████████▌| 216518/225000 [421:30:58<20:46:55,  8.82s/it]
 96%|█████████▌| 216519/225000 [421:31:05<19:44:54,  8.38s/it]
 96%|█████████▌| 216520/225000 [421:31:17<21:56:19,  9.31s/it]
 96%|█████████▌| 216521/225000 [421:31:22<19:05:01,  8.10s/it]
 96%|█████████▌| 216522/225000 [421:31:31<19:36:26,  8.33s/it]
 96%|█████████▌| 216523/225000 [421:31:42<21:35:43,  9.17s/it]
 96%|█████████▌| 216524/225000 [421:31:53<22:34:54,  9.59s/it]
 96%|█████████▌| 216525/225000 [421:32:00<20:41:12,  8.79s/it]
 96%|█████████▌| 216526/225000 [421:32:08<20:37:12,  8.76s/it]
 96%|█████████▌| 216527/225000 [421:32:14<18:22:09,  7.80s/it]
 96%|█████████▌| 216528/225000 [421:32:28<23:05:10,  9.81s/it]
 96%|█████████▌| 216529/225000 [421:32:36<21:41:05,  9.22s/it]
 96%|█████████▌| 216530/225000 [421:32:43<19:58:07,  8.49s/it]
 96%|█████████▌| 216531/225000 [421:32:58<24:47:58, 10.54s/it]
 96%|█████████▌| 216532/225000 [421:33:06<22:58:16,  9.77s/it]
 96%|█████████▌| 216533/225000 [421:33:14<21:45:23,  9.25s/it]
 96%|█████████▌| 216534/225000 [421:33:28<25:00:08, 10.63s/it]
 96%|█████████▌| 216535/225000 [421:33:36<23:10:34,  9.86s/it]
 96%|█████████▌| 216536/225000 [421:33:42<20:34:53,  8.75s/it]
 96%|█████████▌| 216537/225000 [421:33:49<18:47:21,  7.99s/it]
 96%|█████████▌| 216538/225000 [421:33:58<19:53:22,  8.46s/it]
 96%|█████████▌| 216539/225000 [421:34:09<21:26:48,  9.13s/it]
 96%|█████████▌| 216540/225000 [421:34:19<22:27:34,  9.56s/it]
 96%|█████████▌| 216541/225000 [421:34:26<19:57:23,  8.49s/it]
 96%|█████████▌| 216542/225000 [421:34:30<17:18:11,  7.36s/it]
 96%|█████████▌| 216543/225000 [421:34:40<18:48:18,  8.01s/it]
 96%|█████████▌| 216544/225000 [421:34:48<18:49:23,  8.01s/it]
 96%|█████████▌| 216545/225000 [421:34:58<20:36:08,  8.77s/it]
 96%|█████████▌| 216546/225000 [421:35:04<18:26:04,  7.85s/it]
 96%|█████████▌| 216547/225000 [421:35:19<23:45:18, 10.12s/it]
 96%|█████████▌| 216548/225000 [421:35:27<22:08:58,  9.43s/it]
 96%|█████████▌| 216549/225000 [421:35:33<19:40:35,  8.38s/it]
 96%|█████████▌| 216550/225000 [421:35:41<19:20:58,  8.24s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.399', 'learning_rate': '1.949e-07', 'epoch': '0.9624', 'num_input_tokens_seen': 716458872, 'train_runtime': '1.518e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▌| 216550/225000 [421:35:41<19:20:58,  8.24s/it]
 96%|█████████▌| 216551/225000 [421:35:48<18:25:09,  7.85s/it]
 96%|█████████▌| 216552/225000 [421:36:04<24:06:34, 10.27s/it]
 96%|█████████▌| 216553/225000 [421:36:10<21:20:29,  9.10s/it]
 96%|█████████▌| 216554/225000 [421:36:18<20:18:58,  8.66s/it]
 96%|█████████▌| 216555/225000 [421:36:37<27:41:05, 11.80s/it]
 96%|█████████▌| 216556/225000 [421:36:43<23:19:39,  9.95s/it]
 96%|█████████▌| 216557/225000 [421:36:55<24:43:37, 10.54s/it]
 96%|█████████▌| 216558/225000 [421:37:01<21:49:09,  9.30s/it]
 96%|█████████▌| 216559/225000 [421:37:12<23:07:45,  9.86s/it]
 96%|█████████▌| 216560/225000 [421:37:23<23:40:10, 10.10s/it]
 96%|█████████▌| 216561/225000 [421:37:34<24:16:28, 10.36s/it]
 96%|█████████▌| 216562/225000 [421:37:40<21:17:17,  9.08s/it]
 96%|█████████▋| 216563/225000 [421:37:47<20:11:07,  8.61s/it]
 96%|█████████▋| 216564/225000 [421:37:57<20:37:32,  8.80s/it]
 96%|█████████▋| 216565/225000 [421:38:09<23:02:16,  9.83s/it]
 96%|█████████▋| 216566/225000 [421:38:16<21:19:02,  9.10s/it]
 96%|█████████▋| 216567/225000 [421:38:23<19:45:09,  8.43s/it]
 96%|█████████▋| 216568/225000 [421:38:31<19:03:28,  8.14s/it]
 96%|█████████▋| 216569/225000 [421:38:41<20:39:19,  8.82s/it]
 96%|█████████▋| 216570/225000 [421:38:55<24:25:42, 10.43s/it]
 96%|█████████▋| 216571/225000 [421:39:05<24:13:42, 10.35s/it]
 96%|█████████▋| 216572/225000 [421:39:11<20:42:28,  8.85s/it]
 96%|█████████▋| 216573/225000 [421:39:20<21:08:05,  9.03s/it]
 96%|█████████▋| 216574/225000 [421:39:28<20:19:26,  8.68s/it]
 96%|█████████▋| 216575/225000 [421:39:35<19:11:13,  8.20s/it]
 96%|█████████▋| 216576/225000 [421:39:42<18:15:27,  7.80s/it]
 96%|█████████▋| 216577/225000 [421:39:49<17:48:01,  7.61s/it]
 96%|█████████▋| 216578/225000 [421:40:00<19:52:11,  8.49s/it]
 96%|█████████▋| 216579/225000 [421:40:07<18:45:35,  8.02s/it]
 96%|█████████▋| 216580/225000 [421:40:16<19:40:33,  8.41s/it]
 96%|█████████▋| 216581/225000 [421:40:24<19:32:21,  8.36s/it]
 96%|█████████▋| 216582/225000 [421:40:33<19:36:14,  8.38s/it]
 96%|█████████▋| 216583/225000 [421:40:46<22:56:04,  9.81s/it]
 96%|█████████▋| 216584/225000 [421:40:55<22:32:08,  9.64s/it]
 96%|█████████▋| 216585/225000 [421:41:05<22:53:02,  9.79s/it]
 96%|█████████▋| 216586/225000 [421:41:14<22:29:47,  9.63s/it]
 96%|█████████▋| 216587/225000 [421:41:24<22:17:38,  9.54s/it]
 96%|█████████▋| 216588/225000 [421:41:29<19:30:22,  8.35s/it]
 96%|█████████▋| 216589/225000 [421:41:36<18:39:59,  7.99s/it]
 96%|█████████▋| 216590/225000 [421:41:48<21:22:21,  9.15s/it]
 96%|█████████▋| 216591/225000 [421:41:56<20:22:50,  8.73s/it]
 96%|█████████▋| 216592/225000 [421:42:03<19:00:59,  8.14s/it]
 96%|█████████▋| 216593/225000 [421:42:13<20:14:23,  8.67s/it]
 96%|█████████▋| 216594/225000 [421:42:22<20:51:47,  8.94s/it]
 96%|█████████▋| 216595/225000 [421:42:27<17:42:14,  7.58s/it]
 96%|█████████▋| 216596/225000 [421:42:40<21:44:18,  9.31s/it]
 96%|█████████▋| 216597/225000 [421:42:47<19:53:51,  8.52s/it]
 96%|█████████▋| 216598/225000 [421:42:58<21:30:04,  9.21s/it]
 96%|█████████▋| 216599/225000 [421:43:04<19:23:51,  8.31s/it]
 96%|█████████▋| 216600/225000 [421:43:13<20:03:02,  8.59s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.134', 'learning_rate': '1.927e-07', 'epoch': '0.9627', 'num_input_tokens_seen': 716690824, 'train_runtime': '1.518e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216600/225000 [421:43:13<20:03:02,  8.59s/it]
 96%|█████████▋| 216601/225000 [421:43:20<18:34:57,  7.96s/it]
 96%|█████████▋| 216602/225000 [421:43:34<22:55:09,  9.82s/it]
 96%|█████████▋| 216603/225000 [421:43:42<21:52:12,  9.38s/it]
 96%|█████████▋| 216604/225000 [421:43:54<23:27:39, 10.06s/it]
 96%|█████████▋| 216605/225000 [421:44:03<22:52:40,  9.81s/it]
 96%|█████████▋| 216606/225000 [421:44:10<21:04:51,  9.04s/it]
 96%|█████████▋| 216607/225000 [421:44:20<21:52:59,  9.39s/it]
 96%|█████████▋| 216608/225000 [421:44:30<21:46:59,  9.34s/it]
 96%|█████████▋| 216609/225000 [421:44:37<20:20:47,  8.73s/it]
 96%|█████████▋| 216610/225000 [421:44:46<20:16:26,  8.70s/it]
 96%|█████████▋| 216611/225000 [421:45:00<24:29:52, 10.51s/it]
 96%|█████████▋| 216612/225000 [421:45:07<21:51:58,  9.38s/it]
 96%|█████████▋| 216613/225000 [421:45:14<19:53:55,  8.54s/it]
 96%|█████████▋| 216614/225000 [421:45:21<19:01:27,  8.17s/it]
 96%|█████████▋| 216615/225000 [421:45:39<25:42:45, 11.04s/it]
 96%|█████████▋| 216616/225000 [421:45:45<22:30:44,  9.67s/it]
 96%|█████████▋| 216617/225000 [421:45:51<19:57:52,  8.57s/it]
 96%|█████████▋| 216618/225000 [421:46:04<22:56:35,  9.85s/it]
 96%|█████████▋| 216619/225000 [421:46:10<20:20:23,  8.74s/it]
 96%|█████████▋| 216620/225000 [421:46:17<18:57:56,  8.15s/it]
 96%|█████████▋| 216621/225000 [421:46:23<17:37:47,  7.57s/it]
 96%|█████████▋| 216622/225000 [421:46:35<20:48:33,  8.94s/it]
 96%|█████████▋| 216623/225000 [421:46:50<24:33:21, 10.55s/it]
 96%|█████████▋| 216624/225000 [421:46:54<20:25:55,  8.78s/it]
 96%|█████████▋| 216625/225000 [421:47:04<21:13:51,  9.13s/it]
 96%|█████████▋| 216626/225000 [421:47:11<19:58:33,  8.59s/it]
 96%|█████████▋| 216627/225000 [421:47:24<22:46:44,  9.79s/it]
 96%|█████████▋| 216628/225000 [421:47:33<22:07:26,  9.51s/it]
 96%|█████████▋| 216629/225000 [421:47:40<20:14:12,  8.70s/it]
 96%|█████████▋| 216630/225000 [421:47:48<20:15:27,  8.71s/it]
 96%|█████████▋| 216631/225000 [421:47:56<19:19:36,  8.31s/it]
 96%|█████████▋| 216632/225000 [421:48:15<26:52:15, 11.56s/it]
 96%|█████████▋| 216633/225000 [421:48:25<25:42:34, 11.06s/it]
 96%|█████████▋| 216634/225000 [421:48:33<23:58:48, 10.32s/it]
 96%|█████████▋| 216635/225000 [421:48:39<20:29:06,  8.82s/it]
 96%|█████████▋| 216636/225000 [421:48:49<21:24:26,  9.21s/it]
 96%|█████████▋| 216637/225000 [421:48:58<21:06:20,  9.09s/it]
 96%|█████████▋| 216638/225000 [421:49:04<19:07:50,  8.24s/it]
 96%|█████████▋| 216639/225000 [421:49:10<17:52:11,  7.69s/it]
 96%|█████████▋| 216640/225000 [421:49:21<19:42:08,  8.48s/it]
 96%|█████████▋| 216641/225000 [421:49:31<21:03:56,  9.07s/it]
 96%|█████████▋| 216642/225000 [421:49:41<21:23:48,  9.22s/it]
 96%|█████████▋| 216643/225000 [421:49:47<19:26:56,  8.38s/it]
 96%|█████████▋| 216644/225000 [421:49:52<17:08:45,  7.39s/it]
 96%|█████████▋| 216645/225000 [421:50:00<17:32:58,  7.56s/it]
 96%|█████████▋| 216646/225000 [421:50:08<17:48:43,  7.68s/it]
 96%|█████████▋| 216647/225000 [421:50:22<22:24:43,  9.66s/it]
 96%|█████████▋| 216648/225000 [421:50:33<23:01:19,  9.92s/it]
 96%|█████████▋| 216649/225000 [421:50:43<22:47:14,  9.82s/it]
 96%|█████████▋| 216650/225000 [421:50:53<23:24:26, 10.09s/it]
                                                              
{'loss': '1.322', 'grad_norm': '0.8767', 'learning_rate': '1.904e-07', 'epoch': '0.9629', 'num_input_tokens_seen': 716929480, 'train_runtime': '1.519e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216650/225000 [421:50:53<23:24:26, 10.09s/it]
 96%|█████████▋| 216651/225000 [421:51:02<22:46:52,  9.82s/it]
 96%|█████████▋| 216652/225000 [421:51:06<18:40:29,  8.05s/it]
 96%|█████████▋| 216653/225000 [421:51:17<20:40:12,  8.91s/it]
 96%|█████████▋| 216654/225000 [421:51:25<20:01:37,  8.64s/it]
 96%|█████████▋| 216655/225000 [421:51:34<19:57:11,  8.61s/it]
 96%|█████████▋| 216656/225000 [421:51:43<20:39:32,  8.91s/it]
 96%|█████████▋| 216657/225000 [421:51:52<20:15:59,  8.75s/it]
 96%|█████████▋| 216658/225000 [421:52:01<20:19:14,  8.77s/it]
 96%|█████████▋| 216659/225000 [421:52:08<19:31:22,  8.43s/it]
 96%|█████████▋| 216660/225000 [421:52:15<18:14:22,  7.87s/it]
 96%|█████████▋| 216661/225000 [421:52:21<16:59:06,  7.33s/it]
 96%|█████████▋| 216662/225000 [421:52:27<16:25:23,  7.09s/it]
 96%|█████████▋| 216663/225000 [421:52:39<19:27:21,  8.40s/it]
 96%|█████████▋| 216664/225000 [421:52:45<17:42:36,  7.65s/it]
 96%|█████████▋| 216665/225000 [421:52:52<17:18:51,  7.48s/it]
 96%|█████████▋| 216666/225000 [421:53:03<19:39:12,  8.49s/it]
 96%|█████████▋| 216667/225000 [421:53:14<21:23:26,  9.24s/it]
 96%|█████████▋| 216668/225000 [421:53:20<19:21:08,  8.36s/it]
 96%|█████████▋| 216669/225000 [421:53:31<21:06:54,  9.12s/it]
 96%|█████████▋| 216670/225000 [421:53:41<21:51:09,  9.44s/it]
 96%|█████████▋| 216671/225000 [421:53:53<23:41:26, 10.24s/it]
 96%|█████████▋| 216672/225000 [421:54:00<21:17:35,  9.20s/it]
 96%|█████████▋| 216673/225000 [421:54:05<18:32:47,  8.02s/it]
 96%|█████████▋| 216674/225000 [421:54:19<22:25:22,  9.70s/it]
 96%|█████████▋| 216675/225000 [421:54:25<19:48:16,  8.56s/it]
 96%|█████████▋| 216676/225000 [421:54:32<18:37:38,  8.06s/it]
 96%|█████████▋| 216677/225000 [421:54:37<16:42:31,  7.23s/it]
 96%|█████████▋| 216678/225000 [421:54:43<16:13:42,  7.02s/it]
 96%|█████████▋| 216679/225000 [421:54:53<17:42:42,  7.66s/it]
 96%|█████████▋| 216680/225000 [421:55:09<23:39:02, 10.23s/it]
 96%|█████████▋| 216681/225000 [421:55:20<24:29:39, 10.60s/it]
 96%|█████████▋| 216682/225000 [421:55:25<20:15:33,  8.77s/it]
 96%|█████████▋| 216683/225000 [421:55:31<18:15:10,  7.90s/it]
 96%|█████████▋| 216684/225000 [421:55:36<16:24:27,  7.10s/it]
 96%|█████████▋| 216685/225000 [421:55:44<17:11:56,  7.45s/it]
 96%|█████████▋| 216686/225000 [421:55:54<18:57:27,  8.21s/it]
 96%|█████████▋| 216687/225000 [421:56:04<20:14:22,  8.76s/it]
 96%|█████████▋| 216688/225000 [421:56:12<19:43:04,  8.54s/it]
 96%|█████████▋| 216689/225000 [421:56:20<19:20:13,  8.38s/it]
 96%|█████████▋| 216690/225000 [421:56:27<18:25:11,  7.98s/it]
 96%|█████████▋| 216691/225000 [421:56:41<22:20:46,  9.68s/it]
 96%|█████████▋| 216692/225000 [421:56:50<21:43:41,  9.42s/it]
 96%|█████████▋| 216693/225000 [421:56:58<20:36:08,  8.93s/it]
 96%|█████████▋| 216694/225000 [421:57:04<18:48:58,  8.16s/it]
 96%|█████████▋| 216695/225000 [421:57:12<18:27:34,  8.00s/it]
 96%|█████████▋| 216696/225000 [421:57:18<17:05:43,  7.41s/it]
 96%|█████████▋| 216697/225000 [421:57:29<19:35:55,  8.50s/it]
 96%|█████████▋| 216698/225000 [421:57:36<18:49:38,  8.16s/it]
 96%|█████████▋| 216699/225000 [421:57:40<16:14:07,  7.04s/it]
 96%|█████████▋| 216700/225000 [421:57:51<18:30:13,  8.03s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.389', 'learning_rate': '1.882e-07', 'epoch': '0.9631', 'num_input_tokens_seen': 717155376, 'train_runtime': '1.519e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216700/225000 [421:57:51<18:30:13,  8.03s/it]
 96%|█████████▋| 216701/225000 [421:57:55<16:11:17,  7.02s/it]
 96%|█████████▋| 216702/225000 [421:58:07<19:13:54,  8.34s/it]
 96%|█████████▋| 216703/225000 [421:58:18<20:58:13,  9.10s/it]
 96%|█████████▋| 216704/225000 [421:58:23<18:19:02,  7.95s/it]
 96%|█████████▋| 216705/225000 [421:58:33<19:40:51,  8.54s/it]
 96%|█████████▋| 216706/225000 [421:58:40<18:34:43,  8.06s/it]
 96%|█████████▋| 216707/225000 [421:58:47<17:58:51,  7.81s/it]
 96%|█████████▋| 216708/225000 [421:58:53<16:34:01,  7.19s/it]
 96%|█████████▋| 216709/225000 [421:59:04<19:06:14,  8.30s/it]
 96%|█████████▋| 216710/225000 [421:59:14<20:24:14,  8.86s/it]
 96%|█████████▋| 216711/225000 [421:59:24<21:03:36,  9.15s/it]
 96%|█████████▋| 216712/225000 [421:59:34<21:53:30,  9.51s/it]
 96%|█████████▋| 216713/225000 [421:59:39<18:55:25,  8.22s/it]
 96%|█████████▋| 216714/225000 [421:59:50<20:50:53,  9.06s/it]
 96%|█████████▋| 216715/225000 [422:00:01<22:16:53,  9.68s/it]
 96%|█████████▋| 216716/225000 [422:00:06<18:56:53,  8.23s/it]
 96%|█████████▋| 216717/225000 [422:00:12<17:00:57,  7.40s/it]
 96%|█████████▋| 216718/225000 [422:00:25<21:20:30,  9.28s/it]
 96%|█████████▋| 216719/225000 [422:00:30<17:56:14,  7.80s/it]
 96%|█████████▋| 216720/225000 [422:00:37<17:41:38,  7.69s/it]
 96%|█████████▋| 216721/225000 [422:00:53<23:39:22, 10.29s/it]
 96%|█████████▋| 216722/225000 [422:00:58<19:48:20,  8.61s/it]
 96%|█████████▋| 216723/225000 [422:01:04<17:31:54,  7.63s/it]
 96%|█████████▋| 216724/225000 [422:01:11<17:43:58,  7.71s/it]
 96%|█████████▋| 216725/225000 [422:01:26<22:25:26,  9.76s/it]
 96%|█████████▋| 216726/225000 [422:01:31<18:53:49,  8.22s/it]
 96%|█████████▋| 216727/225000 [422:01:38<18:12:48,  7.93s/it]
 96%|█████████▋| 216728/225000 [422:01:46<18:03:32,  7.86s/it]
 96%|█████████▋| 216729/225000 [422:01:57<20:50:57,  9.07s/it]
 96%|█████████▋| 216730/225000 [422:02:04<19:04:42,  8.30s/it]
 96%|█████████▋| 216731/225000 [422:02:15<21:03:17,  9.17s/it]
 96%|█████████▋| 216732/225000 [422:02:23<20:19:24,  8.85s/it]
 96%|█████████▋| 216733/225000 [422:02:34<21:30:38,  9.37s/it]
 96%|█████████▋| 216734/225000 [422:02:43<21:03:58,  9.17s/it]
 96%|█████████▋| 216735/225000 [422:02:50<20:12:49,  8.80s/it]
 96%|█████████▋| 216736/225000 [422:02:57<18:29:03,  8.05s/it]
 96%|█████████▋| 216737/225000 [422:03:11<22:33:59,  9.83s/it]
 96%|█████████▋| 216738/225000 [422:03:19<21:34:41,  9.40s/it]
 96%|█████████▋| 216739/225000 [422:03:27<20:24:44,  8.90s/it]
 96%|█████████▋| 216740/225000 [422:03:34<19:31:57,  8.51s/it]
 96%|█████████▋| 216741/225000 [422:03:44<20:31:42,  8.95s/it]
 96%|█████████▋| 216742/225000 [422:03:59<24:16:27, 10.58s/it]
 96%|█████████▋| 216743/225000 [422:04:05<21:11:37,  9.24s/it]
 96%|█████████▋| 216744/225000 [422:04:14<21:15:23,  9.27s/it]
 96%|█████████▋| 216745/225000 [422:04:19<18:02:59,  7.87s/it]
 96%|█████████▋| 216746/225000 [422:04:25<16:28:45,  7.19s/it]
 96%|█████████▋| 216747/225000 [422:04:34<17:53:29,  7.80s/it]
 96%|█████████▋| 216748/225000 [422:04:40<16:55:40,  7.38s/it]
 96%|█████████▋| 216749/225000 [422:04:49<17:44:04,  7.74s/it]
 96%|█████████▋| 216750/225000 [422:04:54<16:06:41,  7.03s/it]
                                                              
{'loss': '1.308', 'grad_norm': '1.513', 'learning_rate': '1.859e-07', 'epoch': '0.9633', 'num_input_tokens_seen': 717384928, 'train_runtime': '1.519e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216750/225000 [422:04:54<16:06:41,  7.03s/it]
 96%|█████████▋| 216751/225000 [422:05:08<20:58:47,  9.16s/it]
 96%|█████████▋| 216752/225000 [422:05:14<18:46:00,  8.19s/it]
 96%|█████████▋| 216753/225000 [422:05:25<20:45:21,  9.06s/it]
 96%|█████████▋| 216754/225000 [422:05:33<19:54:40,  8.69s/it]
 96%|█████████▋| 216755/225000 [422:05:44<21:12:29,  9.26s/it]
 96%|█████████▋| 216756/225000 [422:05:55<22:33:17,  9.85s/it]
 96%|█████████▋| 216757/225000 [422:06:06<23:25:53, 10.23s/it]
 96%|█████████▋| 216758/225000 [422:06:15<22:30:49,  9.83s/it]
 96%|█████████▋| 216759/225000 [422:06:22<20:48:12,  9.09s/it]
 96%|█████████▋| 216760/225000 [422:06:32<21:30:34,  9.40s/it]
 96%|█████████▋| 216761/225000 [422:06:41<20:38:59,  9.02s/it]
 96%|█████████▋| 216762/225000 [422:06:49<20:15:27,  8.85s/it]
 96%|█████████▋| 216763/225000 [422:06:57<19:55:10,  8.71s/it]
 96%|█████████▋| 216764/225000 [422:07:03<17:52:59,  7.82s/it]
 96%|█████████▋| 216765/225000 [422:07:09<16:28:36,  7.20s/it]
 96%|█████████▋| 216766/225000 [422:07:22<20:19:06,  8.88s/it]
 96%|█████████▋| 216767/225000 [422:07:29<19:10:19,  8.38s/it]
 96%|█████████▋| 216768/225000 [422:07:39<20:10:41,  8.82s/it]
 96%|█████████▋| 216769/225000 [422:07:53<23:56:37, 10.47s/it]
 96%|█████████▋| 216770/225000 [422:08:00<21:33:48,  9.43s/it]
 96%|█████████▋| 216771/225000 [422:08:09<20:54:54,  9.15s/it]
 96%|█████████▋| 216772/225000 [422:08:16<19:43:19,  8.63s/it]
 96%|█████████▋| 216773/225000 [422:08:26<20:40:18,  9.05s/it]
 96%|█████████▋| 216774/225000 [422:08:38<22:30:04,  9.85s/it]
 96%|█████████▋| 216775/225000 [422:08:45<20:37:57,  9.03s/it]
 96%|█████████▋| 216776/225000 [422:08:52<19:01:22,  8.33s/it]
 96%|█████████▋| 216777/225000 [422:09:01<20:07:55,  8.81s/it]
 96%|█████████▋| 216778/225000 [422:09:07<17:54:34,  7.84s/it]
 96%|█████████▋| 216779/225000 [422:09:16<18:47:36,  8.23s/it]
 96%|█████████▋| 216780/225000 [422:09:25<19:29:51,  8.54s/it]
 96%|█████████▋| 216781/225000 [422:09:32<18:11:42,  7.97s/it]
 96%|█████████▋| 216782/225000 [422:09:43<20:26:56,  8.96s/it]
 96%|█████████▋| 216783/225000 [422:09:56<23:15:11, 10.19s/it]
 96%|█████████▋| 216784/225000 [422:10:01<19:30:34,  8.55s/it]
 96%|█████████▋| 216785/225000 [422:10:11<20:27:35,  8.97s/it]
 96%|█████████▋| 216786/225000 [422:10:22<21:45:17,  9.53s/it]
 96%|█████████▋| 216787/225000 [422:10:33<22:56:24, 10.06s/it]
 96%|█████████▋| 216788/225000 [422:10:40<21:02:25,  9.22s/it]
 96%|█████████▋| 216789/225000 [422:10:48<19:48:26,  8.68s/it]
 96%|█████████▋| 216790/225000 [422:10:57<20:02:34,  8.79s/it]
 96%|█████████▋| 216791/225000 [422:11:03<18:13:18,  7.99s/it]
 96%|█████████▋| 216792/225000 [422:11:10<17:19:58,  7.60s/it]
 96%|█████████▋| 216793/225000 [422:11:21<19:48:20,  8.69s/it]
 96%|█████████▋| 216794/225000 [422:11:32<21:03:50,  9.24s/it]
 96%|█████████▋| 216795/225000 [422:11:38<19:08:13,  8.40s/it]
 96%|█████████▋| 216796/225000 [422:11:49<20:40:39,  9.07s/it]
 96%|█████████▋| 216797/225000 [422:11:56<19:31:09,  8.57s/it]
 96%|█████████▋| 216798/225000 [422:12:06<20:15:03,  8.89s/it]
 96%|█████████▋| 216799/225000 [422:12:12<18:35:34,  8.16s/it]
 96%|█████████▋| 216800/225000 [422:12:21<19:15:42,  8.46s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.434', 'learning_rate': '1.837e-07', 'epoch': '0.9636', 'num_input_tokens_seen': 717613024, 'train_runtime': '1.52e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216800/225000 [422:12:21<19:15:42,  8.46s/it]
 96%|█████████▋| 216801/225000 [422:12:29<19:01:02,  8.35s/it]
 96%|█████████▋| 216802/225000 [422:12:33<15:56:09,  7.00s/it]
 96%|█████████▋| 216803/225000 [422:12:41<16:43:29,  7.35s/it]
 96%|█████████▋| 216804/225000 [422:12:49<17:02:37,  7.49s/it]
 96%|█████████▋| 216805/225000 [422:13:04<22:00:23,  9.67s/it]
 96%|█████████▋| 216806/225000 [422:13:11<20:04:37,  8.82s/it]
 96%|█████████▋| 216807/225000 [422:13:21<21:21:12,  9.38s/it]
 96%|█████████▋| 216808/225000 [422:13:28<19:42:46,  8.66s/it]
 96%|█████████▋| 216809/225000 [422:13:39<21:01:59,  9.24s/it]
 96%|█████████▋| 216810/225000 [422:13:47<20:03:16,  8.82s/it]
 96%|█████████▋| 216811/225000 [422:13:54<19:13:29,  8.45s/it]
 96%|█████████▋| 216812/225000 [422:13:59<16:18:53,  7.17s/it]
 96%|█████████▋| 216813/225000 [422:14:10<19:27:18,  8.55s/it]
 96%|█████████▋| 216814/225000 [422:14:22<21:24:04,  9.41s/it]
 96%|█████████▋| 216815/225000 [422:14:27<18:32:15,  8.15s/it]
 96%|█████████▋| 216816/225000 [422:14:34<17:33:58,  7.73s/it]
 96%|█████████▋| 216817/225000 [422:14:46<20:38:36,  9.08s/it]
 96%|█████████▋| 216818/225000 [422:14:59<23:14:54, 10.23s/it]
 96%|█████████▋| 216819/225000 [422:15:04<19:46:01,  8.70s/it]
 96%|█████████▋| 216820/225000 [422:15:15<20:58:16,  9.23s/it]
 96%|█████████▋| 216821/225000 [422:15:21<19:06:56,  8.41s/it]
 96%|█████████▋| 216822/225000 [422:15:28<18:16:55,  8.05s/it]
 96%|█████████▋| 216823/225000 [422:15:35<17:36:58,  7.76s/it]
 96%|█████████▋| 216824/225000 [422:15:44<18:21:13,  8.08s/it]
 96%|█████████▋| 216825/225000 [422:15:57<21:42:12,  9.56s/it]
 96%|█████████▋| 216826/225000 [422:16:03<19:13:55,  8.47s/it]
 96%|█████████▋| 216827/225000 [422:16:11<18:41:41,  8.23s/it]
 96%|█████████▋| 216828/225000 [422:16:24<22:12:50,  9.79s/it]
 96%|█████████▋| 216829/225000 [422:16:34<22:00:46,  9.70s/it]
 96%|█████████▋| 216830/225000 [422:16:42<20:54:17,  9.21s/it]
 96%|█████████▋| 216831/225000 [422:16:59<26:04:53, 11.49s/it]
 96%|█████████▋| 216832/225000 [422:17:04<22:03:48,  9.72s/it]
 96%|█████████▋| 216833/225000 [422:17:13<21:43:28,  9.58s/it]
 96%|█████████▋| 216834/225000 [422:17:19<18:59:58,  8.38s/it]
 96%|█████████▋| 216835/225000 [422:17:25<17:15:45,  7.61s/it]
 96%|█████████▋| 216836/225000 [422:17:33<17:46:57,  7.84s/it]
 96%|█████████▋| 216837/225000 [422:17:44<19:51:32,  8.76s/it]
 96%|█████████▋| 216838/225000 [422:17:54<20:20:58,  8.98s/it]
 96%|█████████▋| 216839/225000 [422:17:59<18:07:10,  7.99s/it]
 96%|█████████▋| 216840/225000 [422:18:05<16:43:23,  7.38s/it]
 96%|█████████▋| 216841/225000 [422:18:22<22:56:21, 10.12s/it]
 96%|█████████▋| 216842/225000 [422:18:29<20:53:18,  9.22s/it]
 96%|█████████▋| 216843/225000 [422:18:41<23:01:32, 10.16s/it]
 96%|█████████▋| 216844/225000 [422:18:50<22:17:15,  9.84s/it]
 96%|█████████▋| 216845/225000 [422:18:58<20:44:39,  9.16s/it]
 96%|█████████▋| 216846/225000 [422:19:09<21:55:52,  9.68s/it]
 96%|█████████▋| 216847/225000 [422:19:13<18:13:28,  8.05s/it]
 96%|█████████▋| 216848/225000 [422:19:21<18:14:57,  8.06s/it]
 96%|█████████▋| 216849/225000 [422:19:26<16:27:12,  7.27s/it]
 96%|█████████▋| 216850/225000 [422:19:38<19:26:39,  8.59s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.525', 'learning_rate': '1.815e-07', 'epoch': '0.9638', 'num_input_tokens_seen': 717839320, 'train_runtime': '1.52e+06', 'train_tokens_per_second': '472.1'}

 96%|█████████▋| 216850/225000 [422:19:38<19:26:39,  8.59s/it]
 96%|█████████▋| 216851/225000 [422:19:43<16:40:14,  7.36s/it]
 96%|█████████▋| 216852/225000 [422:19:49<15:53:00,  7.02s/it]
 96%|█████████▋| 216853/225000 [422:19:55<15:05:54,  6.67s/it]
 96%|█████████▋| 216854/225000 [422:20:03<16:11:44,  7.16s/it]
 96%|█████████▋| 216855/225000 [422:20:14<18:49:23,  8.32s/it]
 96%|█████████▋| 216856/225000 [422:20:22<18:28:27,  8.17s/it]
 96%|█████████▋| 216857/225000 [422:20:30<18:25:45,  8.15s/it]
 96%|█████████▋| 216858/225000 [422:20:46<24:06:55, 10.66s/it]
 96%|█████████▋| 216859/225000 [422:20:57<23:56:17, 10.59s/it]
 96%|█████████▋| 216860/225000 [422:21:03<21:11:39,  9.37s/it]
 96%|█████████▋| 216861/225000 [422:21:10<19:21:55,  8.57s/it]
 96%|█████████▋| 216862/225000 [422:21:18<19:10:46,  8.48s/it]
 96%|█████████▋| 216863/225000 [422:21:28<20:02:27,  8.87s/it]
 96%|█████████▋| 216864/225000 [422:21:38<20:27:19,  9.05s/it]
 96%|█████████▋| 216865/225000 [422:21:43<18:11:33,  8.05s/it]
 96%|█████████▋| 216866/225000 [422:21:55<20:29:39,  9.07s/it]
 96%|█████████▋| 216867/225000 [422:22:01<18:19:36,  8.11s/it]
 96%|█████████▋| 216868/225000 [422:22:11<19:31:57,  8.65s/it]
 96%|█████████▋| 216869/225000 [422:22:17<18:18:40,  8.11s/it]
 96%|█████████▋| 216870/225000 [422:22:24<17:17:59,  7.66s/it]
 96%|█████████▋| 216871/225000 [422:22:38<21:51:43,  9.68s/it]
 96%|█████████▋| 216872/225000 [422:22:45<20:03:28,  8.88s/it]
 96%|█████████▋| 216873/225000 [422:22:54<19:35:51,  8.68s/it]
 96%|█████████▋| 216874/225000 [422:23:02<19:24:13,  8.60s/it]
 96%|█████████▋| 216875/225000 [422:23:18<24:01:47, 10.65s/it]
 96%|█████████▋| 216876/225000 [422:23:26<22:28:51,  9.96s/it]
 96%|█████████▋| 216877/225000 [422:23:33<20:38:45,  9.15s/it]
 96%|█████████▋| 216878/225000 [422:23:44<21:41:42,  9.62s/it]
 96%|█████████▋| 216879/225000 [422:23:55<22:31:00,  9.98s/it]
 96%|█████████▋| 216880/225000 [422:24:08<24:44:45, 10.97s/it]
 96%|█████████▋| 216881/225000 [422:24:14<21:24:05,  9.49s/it]
 96%|█████████▋| 216882/225000 [422:24:20<19:06:57,  8.48s/it]
 96%|█████████▋| 216883/225000 [422:24:25<16:41:56,  7.41s/it]
 96%|█████████▋| 216884/225000 [422:24:37<19:29:43,  8.65s/it]
 96%|█████████▋| 216885/225000 [422:24:48<21:27:25,  9.52s/it]
 96%|█████████▋| 216886/225000 [422:24:58<21:23:42,  9.49s/it]
 96%|█████████▋| 216887/225000 [422:25:06<20:27:34,  9.08s/it]
 96%|█████████▋| 216888/225000 [422:25:11<17:45:52,  7.88s/it]
 96%|█████████▋| 216889/225000 [422:25:17<16:39:40,  7.39s/it]
 96%|█████████▋| 216890/225000 [422:25:26<17:45:36,  7.88s/it]
 96%|█████████▋| 216891/225000 [422:25:37<19:41:40,  8.74s/it]
 96%|█████████▋| 216892/225000 [422:25:43<17:45:53,  7.89s/it]
 96%|█████████▋| 216893/225000 [422:25:51<17:48:52,  7.91s/it]
 96%|█████████▋| 216894/225000 [422:25:58<17:09:26,  7.62s/it]
 96%|█████████▋| 216895/225000 [422:26:05<16:53:53,  7.51s/it]
 96%|█████████▋| 216896/225000 [422:26:13<17:03:20,  7.58s/it]
 96%|█████████▋| 216897/225000 [422:26:23<19:19:29,  8.59s/it]
 96%|█████████▋| 216898/225000 [422:26:32<19:28:31,  8.65s/it]
 96%|█████████▋| 216899/225000 [422:26:45<21:58:59,  9.77s/it]
 96%|█████████▋| 216900/225000 [422:26:51<19:28:33,  8.66s/it]
                                                              
{'loss': '1.29', 'grad_norm': '1.308', 'learning_rate': '1.793e-07', 'epoch': '0.964', 'num_input_tokens_seen': 718070848, 'train_runtime': '1.521e+06', 'train_tokens_per_second': '472.2'}

 96%|█████████▋| 216900/225000 [422:26:51<19:28:33,  8.66s/it]
 96%|█████████▋| 216901/225000 [422:27:01<20:44:49,  9.22s/it]
 96%|█████████▋| 216902/225000 [422:27:10<20:13:13,  8.99s/it]
 96%|█████████▋| 216903/225000 [422:27:14<16:47:18,  7.46s/it]
 96%|█████████▋| 216904/225000 [422:27:18<14:55:14,  6.63s/it]
 96%|█████████▋| 216905/225000 [422:27:31<19:06:15,  8.50s/it]
 96%|█████████▋| 216906/225000 [422:27:40<19:37:32,  8.73s/it]
 96%|█████████▋| 216907/225000 [422:27:52<21:49:03,  9.71s/it]
 96%|█████████▋| 216908/225000 [422:28:02<22:03:36,  9.81s/it]
 96%|█████████▋| 216909/225000 [422:28:09<19:36:02,  8.72s/it]
 96%|█████████▋| 216910/225000 [422:28:23<23:41:33, 10.54s/it]
 96%|█████████▋| 216911/225000 [422:28:32<22:30:22, 10.02s/it]
 96%|█████████▋| 216912/225000 [422:28:39<20:08:23,  8.96s/it]
 96%|█████████▋| 216913/225000 [422:28:45<18:38:03,  8.30s/it]
 96%|█████████▋| 216914/225000 [422:28:52<17:26:34,  7.77s/it]
 96%|█████████▋| 216915/225000 [422:29:09<23:47:26, 10.59s/it]
 96%|█████████▋| 216916/225000 [422:29:16<21:24:33,  9.53s/it]
 96%|█████████▋| 216917/225000 [422:29:22<18:53:22,  8.41s/it]
 96%|█████████▋| 216918/225000 [422:29:34<20:56:57,  9.33s/it]
 96%|█████████▋| 216919/225000 [422:29:40<18:52:57,  8.41s/it]
 96%|█████████▋| 216920/225000 [422:29:48<18:28:33,  8.23s/it]
 96%|█████████▋| 216921/225000 [422:30:02<22:34:08, 10.06s/it]
 96%|█████████▋| 216922/225000 [422:30:08<19:41:54,  8.78s/it]
 96%|█████████▋| 216923/225000 [422:30:18<20:37:37,  9.19s/it]
 96%|█████████▋| 216924/225000 [422:30:28<21:09:12,  9.43s/it]
 96%|█████████▋| 216925/225000 [422:30:34<19:11:07,  8.55s/it]
 96%|█████████▋| 216926/225000 [422:30:44<19:50:19,  8.85s/it]
 96%|█████████▋| 216927/225000 [422:30:51<18:53:26,  8.42s/it]
 96%|█████████▋| 216928/225000 [422:31:05<22:18:44,  9.95s/it]
 96%|█████████▋| 216929/225000 [422:31:15<22:17:11,  9.94s/it]
 96%|█████████▋| 216930/225000 [422:31:23<21:10:23,  9.45s/it]
 96%|█████████▋| 216931/225000 [422:31:34<22:27:55, 10.02s/it]
 96%|█████████▋| 216932/225000 [422:31:40<19:32:19,  8.72s/it]
 96%|█████████▋| 216933/225000 [422:31:54<22:41:21, 10.13s/it]
 96%|█████████▋| 216934/225000 [422:32:04<23:10:00, 10.34s/it]
 96%|█████████▋| 216935/225000 [422:32:16<23:43:55, 10.59s/it]
 96%|█████████▋| 216936/225000 [422:32:20<19:32:11,  8.72s/it]
 96%|█████████▋| 216937/225000 [422:32:29<19:49:02,  8.85s/it]
 96%|█████████▋| 216938/225000 [422:32:34<16:53:18,  7.54s/it]
 96%|█████████▋| 216939/225000 [422:32:48<21:22:21,  9.54s/it]
 96%|█████████▋| 216940/225000 [422:32:55<19:34:34,  8.74s/it]
 96%|█████████▋| 216941/225000 [422:33:03<19:09:31,  8.56s/it]
 96%|█████████▋| 216942/225000 [422:33:13<20:21:23,  9.09s/it]
 96%|█████████▋| 216943/225000 [422:33:22<20:07:19,  8.99s/it]
 96%|█████████▋| 216944/225000 [422:33:33<21:44:28,  9.72s/it]
 96%|█████████▋| 216945/225000 [422:33:42<21:20:35,  9.54s/it]
 96%|█████████▋| 216946/225000 [422:33:50<19:45:54,  8.83s/it]
 96%|█████████▋| 216947/225000 [422:33:56<17:59:15,  8.04s/it]
 96%|█████████▋| 216948/225000 [422:34:09<21:29:33,  9.61s/it]
 96%|█████████▋| 216949/225000 [422:34:19<22:02:36,  9.86s/it]
 96%|█████████▋| 216950/225000 [422:34:29<21:40:19,  9.69s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.512', 'learning_rate': '1.772e-07', 'epoch': '0.9642', 'num_input_tokens_seen': 718307432, 'train_runtime': '1.521e+06', 'train_tokens_per_second': '472.2'}

 96%|█████████▋| 216950/225000 [422:34:29<21:40:19,  9.69s/it]
 96%|█████████▋| 216951/225000 [422:34:34<18:54:33,  8.46s/it]
 96%|█████████▋| 216952/225000 [422:34:44<19:53:43,  8.90s/it]
 96%|█████████▋| 216953/225000 [422:34:56<21:44:46,  9.73s/it]
 96%|█████████▋| 216954/225000 [422:35:02<19:01:06,  8.51s/it]
 96%|█████████▋| 216955/225000 [422:35:12<20:09:25,  9.02s/it]
 96%|█████████▋| 216956/225000 [422:35:23<21:45:00,  9.73s/it]
 96%|█████████▋| 216957/225000 [422:35:31<20:32:05,  9.19s/it]
 96%|█████████▋| 216958/225000 [422:35:37<18:10:05,  8.13s/it]
 96%|█████████▋| 216959/225000 [422:35:53<23:25:47, 10.49s/it]
 96%|█████████▋| 216960/225000 [422:36:01<22:12:00,  9.94s/it]
 96%|█████████▋| 216961/225000 [422:36:12<22:21:56, 10.02s/it]
 96%|█████████▋| 216962/225000 [422:36:20<21:20:33,  9.56s/it]
 96%|█████████▋| 216963/225000 [422:36:30<21:14:47,  9.52s/it]
 96%|█████████▋| 216964/225000 [422:36:35<18:22:43,  8.23s/it]
 96%|█████████▋| 216965/225000 [422:36:48<22:01:04,  9.86s/it]
 96%|█████████▋| 216966/225000 [422:36:55<20:02:28,  8.98s/it]
 96%|█████████▋| 216967/225000 [422:37:02<18:12:12,  8.16s/it]
 96%|█████████▋| 216968/225000 [422:37:08<16:47:52,  7.53s/it]
 96%|█████████▋| 216969/225000 [422:37:19<19:04:18,  8.55s/it]
 96%|█████████▋| 216970/225000 [422:37:26<18:15:42,  8.19s/it]
 96%|█████████▋| 216971/225000 [422:37:33<17:13:45,  7.73s/it]
 96%|█████████▋| 216972/225000 [422:37:47<21:34:47,  9.68s/it]
 96%|█████████▋| 216973/225000 [422:37:57<21:57:44,  9.85s/it]
 96%|█████████▋| 216974/225000 [422:38:03<19:29:57,  8.75s/it]
 96%|█████████▋| 216975/225000 [422:38:11<19:04:27,  8.56s/it]
 96%|█████████▋| 216976/225000 [422:38:26<22:55:08, 10.28s/it]
 96%|█████████▋| 216977/225000 [422:38:33<20:52:46,  9.37s/it]
 96%|█████████▋| 216978/225000 [422:38:45<22:59:47, 10.32s/it]
 96%|█████████▋| 216979/225000 [422:38:54<21:46:43,  9.77s/it]
 96%|█████████▋| 216980/225000 [422:39:04<21:43:51,  9.75s/it]
 96%|█████████▋| 216981/225000 [422:39:11<19:53:04,  8.93s/it]
 96%|█████████▋| 216982/225000 [422:39:21<20:40:43,  9.28s/it]
 96%|█████████▋| 216983/225000 [422:39:30<20:47:36,  9.34s/it]
 96%|█████████▋| 216984/225000 [422:39:44<23:43:52, 10.66s/it]
 96%|█████████▋| 216985/225000 [422:39:55<23:45:26, 10.67s/it]
 96%|█████████▋| 216986/225000 [422:40:06<24:16:49, 10.91s/it]
 96%|█████████▋| 216987/225000 [422:40:14<22:18:53, 10.03s/it]
 96%|█████████▋| 216988/225000 [422:40:22<21:11:13,  9.52s/it]
 96%|█████████▋| 216989/225000 [422:40:29<19:02:02,  8.55s/it]
 96%|█████████▋| 216990/225000 [422:40:35<17:28:47,  7.86s/it]
 96%|█████████▋| 216991/225000 [422:40:51<22:48:21, 10.25s/it]
 96%|█████████▋| 216992/225000 [422:40:58<20:53:51,  9.39s/it]
 96%|█████████▋| 216993/225000 [422:41:04<18:20:34,  8.25s/it]
 96%|█████████▋| 216994/225000 [422:41:12<18:11:43,  8.18s/it]
 96%|█████████▋| 216995/225000 [422:41:23<20:30:36,  9.22s/it]
 96%|█████████▋| 216996/225000 [422:41:32<20:09:17,  9.07s/it]
 96%|█████████▋| 216997/225000 [422:41:44<21:51:56,  9.84s/it]
 96%|█████████▋| 216998/225000 [422:41:54<22:25:44, 10.09s/it]
 96%|█████████▋| 216999/225000 [422:42:00<19:26:08,  8.74s/it]
 96%|█████████▋| 217000/225000 [422:42:09<19:18:12,  8.69s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.508', 'learning_rate': '1.75e-07', 'epoch': '0.9644', 'num_input_tokens_seen': 718547440, 'train_runtime': '1.522e+06', 'train_tokens_per_second': '472.2'}

 96%|█████████▋| 217000/225000 [422:42:09<19:18:12,  8.69s/it]
 96%|█████████▋| 217001/225000 [422:42:14<16:53:56,  7.61s/it]
 96%|█████████▋| 217002/225000 [422:42:24<18:33:14,  8.35s/it]
 96%|█████████▋| 217003/225000 [422:42:30<17:19:18,  7.80s/it]
 96%|█████████▋| 217004/225000 [422:42:40<18:16:10,  8.23s/it]
 96%|█████████▋| 217005/225000 [422:42:47<17:41:14,  7.96s/it]
 96%|█████████▋| 217006/225000 [422:42:55<18:00:20,  8.11s/it]
 96%|█████████▋| 217007/225000 [422:43:05<19:04:59,  8.59s/it]
 96%|█████████▋| 217008/225000 [422:43:16<20:57:20,  9.44s/it]
 96%|█████████▋| 217009/225000 [422:43:23<19:07:43,  8.62s/it]
 96%|█████████▋| 217010/225000 [422:43:31<18:23:42,  8.29s/it]
 96%|█████████▋| 217011/225000 [422:43:47<23:29:09, 10.58s/it]
 96%|█████████▋| 217012/225000 [422:43:57<23:26:15, 10.56s/it]
 96%|█████████▋| 217013/225000 [422:44:06<22:34:13, 10.17s/it]
 96%|█████████▋| 217014/225000 [422:44:12<19:26:47,  8.77s/it]
 96%|█████████▋| 217015/225000 [422:44:19<18:23:03,  8.29s/it]
 96%|█████████▋| 217016/225000 [422:44:31<20:59:58,  9.47s/it]
 96%|█████████▋| 217017/225000 [422:44:37<18:42:04,  8.43s/it]
 96%|█████████▋| 217018/225000 [422:44:45<18:21:48,  8.28s/it]
 96%|█████████▋| 217019/225000 [422:44:58<21:18:45,  9.61s/it]
 96%|█████████▋| 217020/225000 [422:45:07<21:14:07,  9.58s/it]
 96%|█████████▋| 217021/225000 [422:45:20<23:06:59, 10.43s/it]
 96%|█████████▋| 217022/225000 [422:45:27<20:43:12,  9.35s/it]
 96%|█████████▋| 217023/225000 [422:45:39<22:41:33, 10.24s/it]
 96%|█████████▋| 217024/225000 [422:45:47<21:16:57,  9.61s/it]
 96%|█████████▋| 217025/225000 [422:45:57<21:24:17,  9.66s/it]
 96%|█████████▋| 217026/225000 [422:46:04<19:46:33,  8.93s/it]
 96%|█████████▋| 217027/225000 [422:46:13<19:57:00,  9.01s/it]
 96%|█████████▋| 217028/225000 [422:46:26<22:10:16, 10.01s/it]
 96%|█████████▋| 217029/225000 [422:46:31<18:55:42,  8.55s/it]
 96%|█████████▋| 217030/225000 [422:46:40<19:17:05,  8.71s/it]
 96%|█████████▋| 217031/225000 [422:46:51<20:55:41,  9.45s/it]
 96%|█████████▋| 217032/225000 [422:46:57<18:19:27,  8.28s/it]
 96%|█████████▋| 217033/225000 [422:47:03<16:47:03,  7.58s/it]
 96%|█████████▋| 217034/225000 [422:47:10<16:43:57,  7.56s/it]
 96%|█████████▋| 217035/225000 [422:47:24<20:55:15,  9.46s/it]
 96%|█████████▋| 217036/225000 [422:47:29<18:11:57,  8.23s/it]
 96%|█████████▋| 217037/225000 [422:47:35<16:28:51,  7.45s/it]
 96%|█████████▋| 217038/225000 [422:47:44<17:16:17,  7.81s/it]
 96%|█████████▋| 217039/225000 [422:47:57<20:58:02,  9.48s/it]
 96%|█████████▋| 217040/225000 [422:48:09<22:27:42, 10.16s/it]
 96%|█████████▋| 217041/225000 [422:48:22<24:10:19, 10.93s/it]
 96%|█████████▋| 217042/225000 [422:48:30<22:18:13, 10.09s/it]
 96%|█████████▋| 217043/225000 [422:48:37<20:48:25,  9.41s/it]
 96%|█████████▋| 217044/225000 [422:48:49<22:11:07, 10.04s/it]
 96%|█████████▋| 217045/225000 [422:48:54<19:06:31,  8.65s/it]
 96%|█████████▋| 217046/225000 [422:49:09<23:21:17, 10.57s/it]
 96%|█████████▋| 217047/225000 [422:49:19<22:39:17, 10.25s/it]
 96%|█████████▋| 217048/225000 [422:49:24<19:24:48,  8.79s/it]
 96%|█████████▋| 217049/225000 [422:49:36<21:38:46,  9.80s/it]
 96%|█████████▋| 217050/225000 [422:49:44<20:25:02,  9.25s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.66', 'learning_rate': '1.728e-07', 'epoch': '0.9647', 'num_input_tokens_seen': 718781048, 'train_runtime': '1.522e+06', 'train_tokens_per_second': '472.2'}

 96%|█████████▋| 217050/225000 [422:49:44<20:25:02,  9.25s/it]
 96%|█████████▋| 217051/225000 [422:49:51<18:20:57,  8.31s/it]
 96%|█████████▋| 217052/225000 [422:50:03<20:51:00,  9.44s/it]
 96%|█████████▋| 217053/225000 [422:50:10<19:11:32,  8.69s/it]
 96%|█████████▋| 217054/225000 [422:50:17<18:02:02,  8.17s/it]
 96%|█████████▋| 217055/225000 [422:50:23<16:34:57,  7.51s/it]
 96%|█████████▋| 217056/225000 [422:50:34<19:03:34,  8.64s/it]
 96%|█████████▋| 217057/225000 [422:50:42<18:44:40,  8.50s/it]
 96%|█████████▋| 217058/225000 [422:50:47<16:39:32,  7.55s/it]
 96%|█████████▋| 217059/225000 [422:50:55<17:04:01,  7.74s/it]
 96%|█████████▋| 217060/225000 [422:51:03<16:47:24,  7.61s/it]
 96%|█████████▋| 217061/225000 [422:51:12<17:54:12,  8.12s/it]
 96%|█████████▋| 217062/225000 [422:51:22<19:00:04,  8.62s/it]
 96%|█████████▋| 217063/225000 [422:51:32<19:54:52,  9.03s/it]
 96%|█████████▋| 217064/225000 [422:51:42<20:49:27,  9.45s/it]
 96%|█████████▋| 217065/225000 [422:51:48<18:22:31,  8.34s/it]
 96%|█████████▋| 217066/225000 [422:52:00<20:38:10,  9.36s/it]
 96%|█████████▋| 217067/225000 [422:52:08<20:10:14,  9.15s/it]
 96%|█████████▋| 217068/225000 [422:52:18<20:06:19,  9.12s/it]
 96%|█████████▋| 217069/225000 [422:52:25<18:56:06,  8.59s/it]
 96%|█████████▋| 217070/225000 [422:52:37<21:26:55,  9.74s/it]
 96%|█████████▋| 217071/225000 [422:52:43<18:54:22,  8.58s/it]
 96%|█████████▋| 217072/225000 [422:52:54<20:13:10,  9.18s/it]
 96%|█████████▋| 217073/225000 [422:53:01<19:02:27,  8.65s/it]
 96%|█████████▋| 217074/225000 [422:53:13<20:59:57,  9.54s/it]
 96%|█████████▋| 217075/225000 [422:53:21<20:02:48,  9.11s/it]
 96%|█████████▋| 217076/225000 [422:53:29<19:26:41,  8.83s/it]
 96%|█████████▋| 217077/225000 [422:53:40<20:32:40,  9.33s/it]
 96%|█████████▋| 217078/225000 [422:53:45<17:57:22,  8.16s/it]
 96%|█████████▋| 217079/225000 [422:53:53<17:34:15,  7.99s/it]
 96%|█████████▋| 217080/225000 [422:54:01<17:40:02,  8.03s/it]
 96%|█████████▋| 217081/225000 [422:54:09<17:41:44,  8.04s/it]
 96%|█████████▋| 217082/225000 [422:54:17<17:49:56,  8.11s/it]
 96%|█████████▋| 217083/225000 [422:54:30<21:21:55,  9.72s/it]
 96%|█████████▋| 217084/225000 [422:54:36<18:40:13,  8.49s/it]
 96%|█████████▋| 217085/225000 [422:54:46<19:32:46,  8.89s/it]
 96%|█████████▋| 217086/225000 [422:54:52<17:45:45,  8.08s/it]
 96%|█████████▋| 217087/225000 [422:54:58<16:22:04,  7.45s/it]
 96%|█████████▋| 217088/225000 [422:55:06<16:25:59,  7.48s/it]
 96%|█████████▋| 217089/225000 [422:55:19<20:21:13,  9.26s/it]
 96%|█████████▋| 217090/225000 [422:55:28<19:57:13,  9.08s/it]
 96%|█████████▋| 217091/225000 [422:55:38<20:35:58,  9.38s/it]
 96%|█████████▋| 217092/225000 [422:55:46<19:58:42,  9.09s/it]
 96%|█████████▋| 217093/225000 [422:55:56<20:16:08,  9.23s/it]
 96%|█████████▋| 217094/225000 [422:56:08<22:03:04, 10.04s/it]
 96%|█████████▋| 217095/225000 [422:56:17<21:36:33,  9.84s/it]
 96%|█████████▋| 217096/225000 [422:56:24<19:31:44,  8.89s/it]
 96%|█████████▋| 217097/225000 [422:56:30<17:56:12,  8.17s/it]
 96%|█████████▋| 217098/225000 [422:56:38<17:52:48,  8.15s/it]
 96%|█████████▋| 217099/225000 [422:56:45<16:42:57,  7.62s/it]
 96%|█████████▋| 217100/225000 [422:56:54<17:49:16,  8.12s/it]
                                                              
{'loss': '1.284', 'grad_norm': '1.451', 'learning_rate': '1.707e-07', 'epoch': '0.9649', 'num_input_tokens_seen': 719009280, 'train_runtime': '1.523e+06', 'train_tokens_per_second': '472.2'}

 96%|█████████▋| 217100/225000 [422:56:54<17:49:16,  8.12s/it]
 96%|█████████▋| 217101/225000 [422:57:01<17:20:37,  7.90s/it]
 96%|█████████▋| 217102/225000 [422:57:15<21:13:41,  9.68s/it]
 96%|█████████▋| 217103/225000 [422:57:23<20:08:20,  9.18s/it]
 96%|█████████▋| 217104/225000 [422:57:33<20:26:54,  9.32s/it]
 96%|█████████▋| 217105/225000 [422:57:42<20:36:54,  9.40s/it]
 96%|█████████▋| 217106/225000 [422:57:53<21:12:30,  9.67s/it]
 96%|█████████▋| 217107/225000 [422:58:03<21:38:43,  9.87s/it]
 96%|█████████▋| 217108/225000 [422:58:09<19:11:00,  8.75s/it]
 96%|█████████▋| 217109/225000 [422:58:18<19:03:47,  8.70s/it]
 96%|█████████▋| 217110/225000 [422:58:31<21:40:22,  9.89s/it]
 96%|█████████▋| 217111/225000 [422:58:38<20:16:48,  9.25s/it]
 96%|█████████▋| 217112/225000 [422:58:47<20:04:19,  9.16s/it]
 96%|█████████▋| 217113/225000 [422:58:57<20:12:03,  9.22s/it]
 96%|█████████▋| 217114/225000 [422:59:04<18:53:30,  8.62s/it]
 96%|█████████▋| 217115/225000 [422:59:11<17:41:15,  8.08s/it]
 96%|█████████▋| 217116/225000 [422:59:20<18:48:48,  8.59s/it]
 96%|█████████▋| 217117/225000 [422:59:32<20:39:32,  9.43s/it]
 96%|█████████▋| 217118/225000 [422:59:40<19:38:38,  8.97s/it]
 96%|█████████▋| 217119/225000 [422:59:46<17:56:15,  8.19s/it]
 96%|█████████▋| 217120/225000 [423:00:00<21:28:42,  9.81s/it]
 96%|█████████▋| 217121/225000 [423:00:09<21:21:29,  9.76s/it]
 96%|█████████▋| 217122/225000 [423:00:15<18:58:42,  8.67s/it]
 96%|█████████▋| 217123/225000 [423:00:27<20:43:39,  9.47s/it]
 96%|█████████▋| 217124/225000 [423:00:31<16:58:44,  7.76s/it]
 96%|█████████▋| 217125/225000 [423:00:40<18:22:35,  8.40s/it]
 97%|█████████▋| 217126/225000 [423:00:47<16:58:43,  7.76s/it]
 97%|█████████▋| 217127/225000 [423:00:53<16:11:08,  7.40s/it]
 97%|█████████▋| 217128/225000 [423:01:04<18:11:18,  8.32s/it]
 97%|█████████▋| 217129/225000 [423:01:18<21:59:34, 10.06s/it]
 97%|█████████▋| 217130/225000 [423:01:27<21:39:58,  9.91s/it]
 97%|█████████▋| 217131/225000 [423:01:36<20:32:08,  9.39s/it]
 97%|█████████▋| 217132/225000 [423:01:44<20:08:33,  9.22s/it]
 97%|█████████▋| 217133/225000 [423:01:52<19:21:32,  8.86s/it]
 97%|█████████▋| 217134/225000 [423:02:06<22:36:27, 10.35s/it]
 97%|█████████▋| 217135/225000 [423:02:17<23:08:30, 10.59s/it]
 97%|█████████▋| 217136/225000 [423:02:26<21:30:28,  9.85s/it]
 97%|█████████▋| 217137/225000 [423:02:31<18:28:43,  8.46s/it]
 97%|█████████▋| 217138/225000 [423:02:36<16:22:00,  7.49s/it]
 97%|█████████▋| 217139/225000 [423:02:43<15:43:11,  7.20s/it]
 97%|█████████▋| 217140/225000 [423:02:56<19:56:18,  9.13s/it]
 97%|█████████▋| 217141/225000 [423:03:03<18:19:12,  8.39s/it]
 97%|█████████▋| 217142/225000 [423:03:15<20:41:49,  9.48s/it]
 97%|█████████▋| 217143/225000 [423:03:21<18:28:40,  8.47s/it]
 97%|█████████▋| 217144/225000 [423:03:33<20:30:32,  9.40s/it]
 97%|█████████▋| 217145/225000 [423:03:42<20:50:10,  9.55s/it]
 97%|█████████▋| 217146/225000 [423:03:48<18:21:51,  8.42s/it]
 97%|█████████▋| 217147/225000 [423:04:02<22:04:03, 10.12s/it]
 97%|█████████▋| 217148/225000 [423:04:11<21:00:13,  9.63s/it]
 97%|█████████▋| 217149/225000 [423:04:17<18:43:04,  8.58s/it]
 97%|█████████▋| 217150/225000 [423:04:26<18:49:30,  8.63s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.453', 'learning_rate': '1.686e-07', 'epoch': '0.9651', 'num_input_tokens_seen': 719243872, 'train_runtime': '1.523e+06', 'train_tokens_per_second': '472.2'}

 97%|█████████▋| 217150/225000 [423:04:26<18:49:30,  8.63s/it]
 97%|█████████▋| 217151/225000 [423:04:36<19:56:54,  9.15s/it]
 97%|█████████▋| 217152/225000 [423:04:46<20:20:43,  9.33s/it]
 97%|█████████▋| 217153/225000 [423:04:52<18:14:28,  8.37s/it]
 97%|█████████▋| 217154/225000 [423:05:00<17:52:03,  8.20s/it]
 97%|█████████▋| 217155/225000 [423:05:14<21:54:18, 10.05s/it]
 97%|█████████▋| 217156/225000 [423:05:25<22:34:38, 10.36s/it]
 97%|█████████▋| 217157/225000 [423:05:37<23:18:50, 10.70s/it]
 97%|█████████▋| 217158/225000 [423:05:46<22:29:43, 10.33s/it]
 97%|█████████▋| 217159/225000 [423:05:53<20:23:50,  9.36s/it]
 97%|█████████▋| 217160/225000 [423:06:05<21:39:26,  9.94s/it]
 97%|█████████▋| 217161/225000 [423:06:10<18:53:56,  8.68s/it]
 97%|█████████▋| 217162/225000 [423:06:23<21:42:59,  9.97s/it]
 97%|█████████▋| 217163/225000 [423:06:30<19:27:29,  8.94s/it]
 97%|█████████▋| 217164/225000 [423:06:40<20:19:38,  9.34s/it]
 97%|█████████▋| 217165/225000 [423:06:51<21:20:16,  9.80s/it]
 97%|█████████▋| 217166/225000 [423:07:01<21:47:56, 10.02s/it]
 97%|█████████▋| 217167/225000 [423:07:10<20:46:24,  9.55s/it]
 97%|█████████▋| 217168/225000 [423:07:18<19:36:59,  9.02s/it]
 97%|█████████▋| 217169/225000 [423:07:23<17:26:33,  8.02s/it]
 97%|█████████▋| 217170/225000 [423:07:31<17:09:11,  7.89s/it]
 97%|█████████▋| 217171/225000 [423:07:44<20:19:02,  9.34s/it]
 97%|█████████▋| 217172/225000 [423:07:54<20:58:27,  9.65s/it]
 97%|█████████▋| 217173/225000 [423:08:04<21:17:45,  9.80s/it]
 97%|█████████▋| 217174/225000 [423:08:11<19:26:23,  8.94s/it]
 97%|█████████▋| 217175/225000 [423:08:21<19:51:28,  9.14s/it]
 97%|█████████▋| 217176/225000 [423:08:31<20:31:34,  9.44s/it]
 97%|█████████▋| 217177/225000 [423:08:36<17:32:13,  8.07s/it]
 97%|█████████▋| 217178/225000 [423:08:43<17:15:54,  7.95s/it]
 97%|█████████▋| 217179/225000 [423:08:53<18:30:24,  8.52s/it]
 97%|█████████▋| 217180/225000 [423:08:58<16:09:36,  7.44s/it]
 97%|█████████▋| 217181/225000 [423:09:05<16:01:47,  7.38s/it]
 97%|█████████▋| 217182/225000 [423:09:13<16:00:11,  7.37s/it]
 97%|█████████▋| 217183/225000 [423:09:25<18:59:27,  8.75s/it]
 97%|█████████▋| 217184/225000 [423:09:32<17:49:52,  8.21s/it]
 97%|█████████▋| 217185/225000 [423:09:44<20:23:33,  9.39s/it]
 97%|█████████▋| 217186/225000 [423:09:51<18:55:53,  8.72s/it]
 97%|█████████▋| 217187/225000 [423:09:58<17:56:52,  8.27s/it]
 97%|█████████▋| 217188/225000 [423:10:10<20:13:41,  9.32s/it]
 97%|█████████▋| 217189/225000 [423:10:19<20:19:20,  9.37s/it]
 97%|█████████▋| 217190/225000 [423:10:35<24:06:01, 11.11s/it]
 97%|█████████▋| 217191/225000 [423:10:40<20:16:24,  9.35s/it]
 97%|█████████▋| 217192/225000 [423:10:45<17:48:38,  8.21s/it]
 97%|█████████▋| 217193/225000 [423:10:54<18:19:24,  8.45s/it]
 97%|█████████▋| 217194/225000 [423:11:03<18:11:02,  8.39s/it]
 97%|█████████▋| 217195/225000 [423:11:14<20:05:40,  9.27s/it]
 97%|█████████▋| 217196/225000 [423:11:20<18:08:35,  8.37s/it]
 97%|█████████▋| 217197/225000 [423:11:27<16:57:24,  7.82s/it]
 97%|█████████▋| 217198/225000 [423:11:33<16:04:44,  7.42s/it]
 97%|█████████▋| 217199/225000 [423:11:47<20:03:55,  9.26s/it]
 97%|█████████▋| 217200/225000 [423:11:54<18:26:13,  8.51s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.395', 'learning_rate': '1.665e-07', 'epoch': '0.9653', 'num_input_tokens_seen': 719477616, 'train_runtime': '1.524e+06', 'train_tokens_per_second': '472.2'}

 97%|█████████▋| 217200/225000 [423:11:54<18:26:13,  8.51s/it]
 97%|█████████▋| 217201/225000 [423:12:01<17:47:53,  8.22s/it]
 97%|█████████▋| 217202/225000 [423:12:09<17:52:09,  8.25s/it]
 97%|█████████▋| 217203/225000 [423:12:15<16:07:39,  7.45s/it]
 97%|█████████▋| 217204/225000 [423:12:27<18:57:35,  8.76s/it]
 97%|█████████▋| 217205/225000 [423:12:37<19:43:09,  9.11s/it]
 97%|█████████▋| 217206/225000 [423:12:50<22:24:48, 10.35s/it]
 97%|█████████▋| 217207/225000 [423:12:57<20:02:57,  9.26s/it]
 97%|█████████▋| 217208/225000 [423:13:05<19:24:45,  8.97s/it]
 97%|█████████▋| 217209/225000 [423:13:14<19:42:20,  9.11s/it]
 97%|█████████▋| 217210/225000 [423:13:21<18:19:27,  8.47s/it]
 97%|█████████▋| 217211/225000 [423:13:28<17:02:38,  7.88s/it]
 97%|█████████▋| 217212/225000 [423:13:34<15:49:43,  7.32s/it]
 97%|█████████▋| 217213/225000 [423:13:43<17:08:24,  7.92s/it]
 97%|█████████▋| 217214/225000 [423:13:56<20:33:01,  9.50s/it]
 97%|█████████▋| 217215/225000 [423:14:03<18:52:52,  8.73s/it]
 97%|█████████▋| 217216/225000 [423:14:13<19:22:45,  8.96s/it]
 97%|█████████▋| 217217/225000 [423:14:25<21:11:26,  9.80s/it]
 97%|█████████▋| 217218/225000 [423:14:30<18:07:56,  8.39s/it]
 97%|█████████▋| 217219/225000 [423:14:42<20:37:38,  9.54s/it]
 97%|█████████▋| 217220/225000 [423:14:53<21:45:47, 10.07s/it]
 97%|█████████▋| 217221/225000 [423:15:02<20:38:55,  9.56s/it]
 97%|█████████▋| 217222/225000 [423:15:08<18:23:49,  8.51s/it]
 97%|█████████▋| 217223/225000 [423:15:14<17:05:03,  7.91s/it]
 97%|█████████▋| 217224/225000 [423:15:21<16:29:11,  7.63s/it]
 97%|█████████▋| 217225/225000 [423:15:33<19:06:01,  8.84s/it]
 97%|█████████▋| 217226/225000 [423:15:40<17:44:56,  8.22s/it]
 97%|█████████▋| 217227/225000 [423:15:50<18:53:30,  8.75s/it]
 97%|█████████▋| 217228/225000 [423:15:59<19:35:05,  9.07s/it]
 97%|█████████▋| 217229/225000 [423:16:06<18:11:18,  8.43s/it]
 97%|█████████▋| 217230/225000 [423:16:19<21:10:18,  9.81s/it]
 97%|█████████▋| 217231/225000 [423:16:31<22:28:14, 10.41s/it]
 97%|█████████▋| 217232/225000 [423:16:37<19:27:27,  9.02s/it]
 97%|█████████▋| 217233/225000 [423:16:46<19:26:57,  9.01s/it]
 97%|█████████▋| 217234/225000 [423:16:58<21:07:08,  9.79s/it]
 97%|█████████▋| 217235/225000 [423:17:06<20:01:53,  9.29s/it]
 97%|█████████▋| 217236/225000 [423:17:14<19:31:26,  9.05s/it]
 97%|█████████▋| 217237/225000 [423:17:27<21:58:00, 10.19s/it]
 97%|█████████▋| 217238/225000 [423:17:36<20:54:44,  9.70s/it]
 97%|█████████▋| 217239/225000 [423:17:43<19:13:03,  8.91s/it]
 97%|█████████▋| 217240/225000 [423:17:49<17:17:57,  8.03s/it]
 97%|█████████▋| 217241/225000 [423:17:56<16:51:26,  7.82s/it]
 97%|█████████▋| 217242/225000 [423:18:04<17:11:06,  7.97s/it]
 97%|█████████▋| 217243/225000 [423:18:19<21:36:55, 10.03s/it]
 97%|█████████▋| 217244/225000 [423:18:26<19:20:55,  8.98s/it]
 97%|█████████▋| 217245/225000 [423:18:36<20:30:42,  9.52s/it]
 97%|█████████▋| 217246/225000 [423:18:44<18:55:48,  8.79s/it]
 97%|█████████▋| 217247/225000 [423:18:58<22:34:40, 10.48s/it]
 97%|█████████▋| 217248/225000 [423:19:04<19:44:52,  9.17s/it]
 97%|█████████▋| 217249/225000 [423:19:11<18:10:04,  8.44s/it]
 97%|█████████▋| 217250/225000 [423:19:18<17:10:34,  7.98s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.5', 'learning_rate': '1.644e-07', 'epoch': '0.9656', 'num_input_tokens_seen': 719707800, 'train_runtime': '1.524e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217250/225000 [423:19:18<17:10:34,  7.98s/it]
 97%|█████████▋| 217251/225000 [423:19:29<19:19:50,  8.98s/it]
 97%|█████████▋| 217252/225000 [423:19:38<19:01:34,  8.84s/it]
 97%|█████████▋| 217253/225000 [423:19:47<19:39:43,  9.14s/it]
 97%|█████████▋| 217254/225000 [423:19:55<18:34:13,  8.63s/it]
 97%|█████████▋| 217255/225000 [423:20:04<18:44:20,  8.71s/it]
 97%|█████████▋| 217256/225000 [423:20:10<17:07:32,  7.96s/it]
 97%|█████████▋| 217257/225000 [423:20:18<17:09:12,  7.98s/it]
 97%|█████████▋| 217258/225000 [423:20:27<17:41:50,  8.23s/it]
 97%|█████████▋| 217259/225000 [423:20:32<15:58:50,  7.43s/it]
 97%|█████████▋| 217260/225000 [423:20:43<17:46:02,  8.26s/it]
 97%|█████████▋| 217261/225000 [423:20:49<16:19:52,  7.60s/it]
 97%|█████████▋| 217262/225000 [423:20:55<15:32:40,  7.23s/it]
 97%|█████████▋| 217263/225000 [423:21:06<18:07:57,  8.44s/it]
 97%|█████████▋| 217264/225000 [423:21:17<19:43:56,  9.18s/it]
 97%|█████████▋| 217265/225000 [423:21:24<18:00:51,  8.38s/it]
 97%|█████████▋| 217266/225000 [423:21:29<16:15:01,  7.56s/it]
 97%|█████████▋| 217267/225000 [423:21:42<19:23:32,  9.03s/it]
 97%|█████████▋| 217268/225000 [423:21:47<17:09:15,  7.99s/it]
 97%|█████████▋| 217269/225000 [423:21:53<15:44:03,  7.33s/it]
 97%|█████████▋| 217270/225000 [423:22:07<20:01:57,  9.33s/it]
 97%|█████████▋| 217271/225000 [423:22:17<20:08:20,  9.38s/it]
 97%|█████████▋| 217272/225000 [423:22:23<18:10:32,  8.47s/it]
 97%|█████████▋| 217273/225000 [423:22:37<21:51:59, 10.19s/it]
 97%|█████████▋| 217274/225000 [423:22:45<20:34:05,  9.58s/it]
 97%|█████████▋| 217275/225000 [423:22:54<19:56:01,  9.29s/it]
 97%|█████████▋| 217276/225000 [423:23:02<19:02:56,  8.88s/it]
 97%|█████████▋| 217277/225000 [423:23:06<16:13:10,  7.56s/it]
 97%|█████████▋| 217278/225000 [423:23:17<18:09:37,  8.47s/it]
 97%|█████████▋| 217279/225000 [423:23:27<19:06:55,  8.91s/it]
 97%|█████████▋| 217280/225000 [423:23:35<18:18:24,  8.54s/it]
 97%|█████████▋| 217281/225000 [423:23:41<17:15:43,  8.05s/it]
 97%|█████████▋| 217282/225000 [423:23:49<16:46:43,  7.83s/it]
 97%|█████████▋| 217283/225000 [423:23:57<16:48:58,  7.84s/it]
 97%|█████████▋| 217284/225000 [423:24:11<21:17:11,  9.93s/it]
 97%|█████████▋| 217285/225000 [423:24:26<24:20:12, 11.36s/it]
 97%|█████████▋| 217286/225000 [423:24:31<20:04:01,  9.36s/it]
 97%|█████████▋| 217287/225000 [423:24:42<21:18:19,  9.94s/it]
 97%|█████████▋| 217288/225000 [423:24:52<21:21:39,  9.97s/it]
 97%|█████████▋| 217289/225000 [423:24:58<18:43:32,  8.74s/it]
 97%|█████████▋| 217290/225000 [423:25:08<19:43:50,  9.21s/it]
 97%|█████████▋| 217291/225000 [423:25:18<19:48:51,  9.25s/it]
 97%|█████████▋| 217292/225000 [423:25:24<17:35:42,  8.22s/it]
 97%|█████████▋| 217293/225000 [423:25:34<19:07:50,  8.94s/it]
 97%|█████████▋| 217294/225000 [423:25:43<18:59:20,  8.87s/it]
 97%|█████████▋| 217295/225000 [423:25:55<21:19:22,  9.96s/it]
 97%|█████████▋| 217296/225000 [423:26:06<21:30:48, 10.05s/it]
 97%|█████████▋| 217297/225000 [423:26:16<21:47:31, 10.18s/it]
 97%|█████████▋| 217298/225000 [423:26:22<19:10:59,  8.97s/it]
 97%|█████████▋| 217299/225000 [423:26:30<18:14:24,  8.53s/it]
 97%|█████████▋| 217300/225000 [423:26:36<17:00:47,  7.95s/it]
                                                              
{'loss': '1.324', 'grad_norm': '1.373', 'learning_rate': '1.623e-07', 'epoch': '0.9658', 'num_input_tokens_seen': 719934024, 'train_runtime': '1.524e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217300/225000 [423:26:36<17:00:47,  7.95s/it]
 97%|█████████▋| 217301/225000 [423:26:42<15:42:56,  7.35s/it]
 97%|█████████▋| 217302/225000 [423:26:53<17:47:15,  8.32s/it]
 97%|█████████▋| 217303/225000 [423:27:00<16:52:23,  7.89s/it]
 97%|█████████▋| 217304/225000 [423:27:10<18:21:26,  8.59s/it]
 97%|█████████▋| 217305/225000 [423:27:19<18:19:59,  8.58s/it]
 97%|█████████▋| 217306/225000 [423:27:28<18:54:37,  8.85s/it]
 97%|█████████▋| 217307/225000 [423:27:36<18:35:50,  8.70s/it]
 97%|█████████▋| 217308/225000 [423:27:43<17:31:42,  8.20s/it]
 97%|█████████▋| 217309/225000 [423:27:51<16:50:42,  7.88s/it]
 97%|█████████▋| 217310/225000 [423:28:01<18:13:29,  8.53s/it]
 97%|█████████▋| 217311/225000 [423:28:13<20:55:51,  9.80s/it]
 97%|█████████▋| 217312/225000 [423:28:27<23:15:01, 10.89s/it]
 97%|█████████▋| 217313/225000 [423:28:39<24:08:21, 11.30s/it]
 97%|█████████▋| 217314/225000 [423:28:44<20:20:51,  9.53s/it]
 97%|█████████▋| 217315/225000 [423:28:54<20:17:39,  9.51s/it]
 97%|█████████▋| 217316/225000 [423:29:02<19:12:40,  9.00s/it]
 97%|█████████▋| 217317/225000 [423:29:12<19:58:53,  9.36s/it]
 97%|█████████▋| 217318/225000 [423:29:22<20:39:59,  9.68s/it]
 97%|█████████▋| 217319/225000 [423:29:29<18:47:56,  8.81s/it]
 97%|█████████▋| 217320/225000 [423:29:37<18:08:29,  8.50s/it]
 97%|█████████▋| 217321/225000 [423:29:47<19:03:19,  8.93s/it]
 97%|█████████▋| 217322/225000 [423:29:52<16:31:21,  7.75s/it]
 97%|█████████▋| 217323/225000 [423:30:06<20:52:49,  9.79s/it]
 97%|█████████▋| 217324/225000 [423:30:11<17:36:02,  8.25s/it]
 97%|█████████▋| 217325/225000 [423:30:19<17:25:43,  8.18s/it]
 97%|█████████▋| 217326/225000 [423:30:24<15:35:48,  7.32s/it]
 97%|█████████▋| 217327/225000 [423:30:37<18:48:56,  8.83s/it]
 97%|█████████▋| 217328/225000 [423:30:45<18:19:06,  8.60s/it]
 97%|█████████▋| 217329/225000 [423:30:52<17:14:14,  8.09s/it]
 97%|█████████▋| 217330/225000 [423:30:59<16:44:09,  7.86s/it]
 97%|█████████▋| 217331/225000 [423:31:06<16:14:51,  7.63s/it]
 97%|█████████▋| 217332/225000 [423:31:20<20:17:55,  9.53s/it]
 97%|█████████▋| 217333/225000 [423:31:27<18:19:48,  8.61s/it]
 97%|█████████▋| 217334/225000 [423:31:39<21:04:09,  9.89s/it]
 97%|█████████▋| 217335/225000 [423:31:45<18:37:12,  8.75s/it]
 97%|█████████▋| 217336/225000 [423:31:51<16:32:11,  7.77s/it]
 97%|█████████▋| 217337/225000 [423:32:02<18:19:37,  8.61s/it]
 97%|█████████▋| 217338/225000 [423:32:10<18:14:50,  8.57s/it]
 97%|█████████▋| 217339/225000 [423:32:15<16:16:16,  7.65s/it]
 97%|█████████▋| 217340/225000 [423:32:22<15:44:34,  7.40s/it]
 97%|█████████▋| 217341/225000 [423:32:38<20:45:16,  9.76s/it]
 97%|█████████▋| 217342/225000 [423:32:47<20:14:49,  9.52s/it]
 97%|█████████▋| 217343/225000 [423:32:56<20:17:18,  9.54s/it]
 97%|█████████▋| 217344/225000 [423:33:06<20:48:40,  9.79s/it]
 97%|█████████▋| 217345/225000 [423:33:13<18:37:21,  8.76s/it]
 97%|█████████▋| 217346/225000 [423:33:24<20:22:15,  9.58s/it]
 97%|█████████▋| 217347/225000 [423:33:31<18:17:09,  8.60s/it]
 97%|█████████▋| 217348/225000 [423:33:38<17:35:31,  8.28s/it]
 97%|█████████▋| 217349/225000 [423:33:44<16:10:05,  7.61s/it]
 97%|█████████▋| 217350/225000 [423:33:50<15:13:56,  7.17s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.379', 'learning_rate': '1.602e-07', 'epoch': '0.966', 'num_input_tokens_seen': 720164032, 'train_runtime': '1.525e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217350/225000 [423:33:50<15:13:56,  7.17s/it]
 97%|█████████▋| 217351/225000 [423:34:03<18:27:56,  8.69s/it]
 97%|█████████▋| 217352/225000 [423:34:15<20:58:55,  9.88s/it]
 97%|█████████▋| 217353/225000 [423:34:21<18:37:29,  8.77s/it]
 97%|█████████▋| 217354/225000 [423:34:28<16:59:33,  8.00s/it]
 97%|█████████▋| 217355/225000 [423:34:35<16:24:08,  7.72s/it]
 97%|█████████▋| 217356/225000 [423:34:47<19:18:49,  9.10s/it]
 97%|█████████▋| 217357/225000 [423:34:53<17:22:52,  8.19s/it]
 97%|█████████▋| 217358/225000 [423:35:05<19:55:18,  9.38s/it]
 97%|█████████▋| 217359/225000 [423:35:13<18:59:15,  8.95s/it]
 97%|█████████▋| 217360/225000 [423:35:19<17:13:55,  8.12s/it]
 97%|█████████▋| 217361/225000 [423:35:27<16:37:17,  7.83s/it]
 97%|█████████▋| 217362/225000 [423:35:38<19:01:44,  8.97s/it]
 97%|█████████▋| 217363/225000 [423:35:45<17:37:09,  8.31s/it]
 97%|█████████▋| 217364/225000 [423:35:54<18:22:42,  8.66s/it]
 97%|█████████▋| 217365/225000 [423:35:59<15:57:59,  7.53s/it]
 97%|█████████▋| 217366/225000 [423:36:12<18:56:48,  8.93s/it]
 97%|█████████▋| 217367/225000 [423:36:23<20:15:09,  9.55s/it]
 97%|█████████▋| 217368/225000 [423:36:34<21:37:32, 10.20s/it]
 97%|█████████▋| 217369/225000 [423:36:43<20:53:03,  9.85s/it]
 97%|█████████▋| 217370/225000 [423:36:52<20:10:37,  9.52s/it]
 97%|█████████▋| 217371/225000 [423:37:01<19:51:24,  9.37s/it]
 97%|█████████▋| 217372/225000 [423:37:06<17:03:41,  8.05s/it]
 97%|█████████▋| 217373/225000 [423:37:12<15:45:01,  7.43s/it]
 97%|█████████▋| 217374/225000 [423:37:27<20:16:00,  9.57s/it]
 97%|█████████▋| 217375/225000 [423:37:37<21:01:26,  9.93s/it]
 97%|█████████▋| 217376/225000 [423:37:46<20:12:28,  9.54s/it]
 97%|█████████▋| 217377/225000 [423:37:53<18:50:19,  8.90s/it]
 97%|█████████▋| 217378/225000 [423:38:02<18:40:04,  8.82s/it]
 97%|█████████▋| 217379/225000 [423:38:10<17:55:46,  8.47s/it]
 97%|█████████▋| 217380/225000 [423:38:15<15:38:42,  7.39s/it]
 97%|█████████▋| 217381/225000 [423:38:22<15:25:55,  7.29s/it]
 97%|█████████▋| 217382/225000 [423:38:27<14:02:48,  6.64s/it]
 97%|█████████▋| 217383/225000 [423:38:43<20:01:35,  9.47s/it]
 97%|█████████▋| 217384/225000 [423:38:57<23:05:09, 10.91s/it]
 97%|█████████▋| 217385/225000 [423:39:08<23:04:22, 10.91s/it]
 97%|█████████▋| 217386/225000 [423:39:15<20:31:06,  9.70s/it]
 97%|█████████▋| 217387/225000 [423:39:25<21:05:05,  9.97s/it]
 97%|█████████▋| 217388/225000 [423:39:36<21:44:50, 10.29s/it]
 97%|█████████▋| 217389/225000 [423:39:45<20:36:30,  9.75s/it]
 97%|█████████▋| 217390/225000 [423:39:55<20:30:54,  9.70s/it]
 97%|█████████▋| 217391/225000 [423:40:05<20:42:09,  9.79s/it]
 97%|█████████▋| 217392/225000 [423:40:15<21:06:33,  9.99s/it]
 97%|█████████▋| 217393/225000 [423:40:23<19:48:22,  9.37s/it]
 97%|█████████▋| 217394/225000 [423:40:33<20:08:36,  9.53s/it]
 97%|█████████▋| 217395/225000 [423:40:38<17:18:35,  8.19s/it]
 97%|█████████▋| 217396/225000 [423:40:48<18:32:27,  8.78s/it]
 97%|█████████▋| 217397/225000 [423:40:55<17:29:51,  8.29s/it]
 97%|█████████▋| 217398/225000 [423:41:02<16:19:32,  7.73s/it]
 97%|█████████▋| 217399/225000 [423:41:17<21:01:06,  9.95s/it]
 97%|█████████▋| 217400/225000 [423:41:25<19:56:18,  9.44s/it]
                                                              
{'loss': '1.342', 'grad_norm': '1.146', 'learning_rate': '1.582e-07', 'epoch': '0.9662', 'num_input_tokens_seen': 720396872, 'train_runtime': '1.525e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217400/225000 [423:41:25<19:56:18,  9.44s/it]
 97%|█████████▋| 217401/225000 [423:41:30<17:14:29,  8.17s/it]
 97%|█████████▋| 217402/225000 [423:41:37<16:27:25,  7.80s/it]
 97%|█████████▋| 217403/225000 [423:41:44<15:54:58,  7.54s/it]
 97%|█████████▋| 217404/225000 [423:41:51<15:21:21,  7.28s/it]
 97%|█████████▋| 217405/225000 [423:41:57<14:49:03,  7.02s/it]
 97%|█████████▋| 217406/225000 [423:42:09<17:52:08,  8.47s/it]
 97%|█████████▋| 217407/225000 [423:42:15<16:15:24,  7.71s/it]
 97%|█████████▋| 217408/225000 [423:42:25<17:53:59,  8.49s/it]
 97%|█████████▋| 217409/225000 [423:42:32<16:59:45,  8.06s/it]
 97%|█████████▋| 217410/225000 [423:42:45<19:49:21,  9.40s/it]
 97%|█████████▋| 217411/225000 [423:42:54<19:41:56,  9.34s/it]
 97%|█████████▋| 217412/225000 [423:42:59<17:05:39,  8.11s/it]
 97%|█████████▋| 217413/225000 [423:43:12<19:51:14,  9.42s/it]
 97%|█████████▋| 217414/225000 [423:43:23<20:42:26,  9.83s/it]
 97%|█████████▋| 217415/225000 [423:43:32<20:16:33,  9.62s/it]
 97%|█████████▋| 217416/225000 [423:43:42<20:30:21,  9.73s/it]
 97%|█████████▋| 217417/225000 [423:43:51<20:15:34,  9.62s/it]
 97%|█████████▋| 217418/225000 [423:43:59<19:27:58,  9.24s/it]
 97%|█████████▋| 217419/225000 [423:44:06<18:01:56,  8.56s/it]
 97%|█████████▋| 217420/225000 [423:44:17<19:23:40,  9.21s/it]
 97%|█████████▋| 217421/225000 [423:44:25<18:44:03,  8.90s/it]
 97%|█████████▋| 217422/225000 [423:44:35<19:01:50,  9.04s/it]
 97%|█████████▋| 217423/225000 [423:44:41<17:03:30,  8.10s/it]
 97%|█████████▋| 217424/225000 [423:44:49<17:21:22,  8.25s/it]
 97%|█████████▋| 217425/225000 [423:44:56<16:14:49,  7.72s/it]
 97%|█████████▋| 217426/225000 [423:45:05<17:26:21,  8.29s/it]
 97%|█████████▋| 217427/225000 [423:45:14<17:37:36,  8.38s/it]
 97%|█████████▋| 217428/225000 [423:45:26<19:51:45,  9.44s/it]
 97%|█████████▋| 217429/225000 [423:45:35<19:43:15,  9.38s/it]
 97%|█████████▋| 217430/225000 [423:45:44<19:13:08,  9.14s/it]
 97%|█████████▋| 217431/225000 [423:45:51<18:25:51,  8.77s/it]
 97%|█████████▋| 217432/225000 [423:46:04<21:06:05, 10.04s/it]
 97%|█████████▋| 217433/225000 [423:46:10<18:11:45,  8.66s/it]
 97%|█████████▋| 217434/225000 [423:46:16<16:27:02,  7.83s/it]
 97%|█████████▋| 217435/225000 [423:46:31<21:03:21, 10.02s/it]
 97%|█████████▋| 217436/225000 [423:46:37<18:43:49,  8.91s/it]
 97%|█████████▋| 217437/225000 [423:46:47<18:56:34,  9.02s/it]
 97%|█████████▋| 217438/225000 [423:46:54<17:53:01,  8.51s/it]
 97%|█████████▋| 217439/225000 [423:47:03<18:27:28,  8.79s/it]
 97%|█████████▋| 217440/225000 [423:47:13<18:47:26,  8.95s/it]
 97%|█████████▋| 217441/225000 [423:47:24<20:01:58,  9.54s/it]
 97%|█████████▋| 217442/225000 [423:47:33<19:40:05,  9.37s/it]
 97%|█████████▋| 217443/225000 [423:47:39<17:54:39,  8.53s/it]
 97%|█████████▋| 217444/225000 [423:47:50<19:29:37,  9.29s/it]
 97%|█████████▋| 217445/225000 [423:47:57<18:04:17,  8.61s/it]
 97%|█████████▋| 217446/225000 [423:48:06<18:19:05,  8.73s/it]
 97%|█████████▋| 217447/225000 [423:48:12<16:32:43,  7.89s/it]
 97%|█████████▋| 217448/225000 [423:48:19<15:53:27,  7.58s/it]
 97%|█████████▋| 217449/225000 [423:48:27<16:25:00,  7.83s/it]
 97%|█████████▋| 217450/225000 [423:48:34<15:32:35,  7.41s/it]
                                                              
{'loss': '1.306', 'grad_norm': '1.41', 'learning_rate': '1.561e-07', 'epoch': '0.9664', 'num_input_tokens_seen': 720629312, 'train_runtime': '1.526e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217450/225000 [423:48:34<15:32:35,  7.41s/it]
 97%|█████████▋| 217451/225000 [423:48:46<18:31:57,  8.84s/it]
 97%|█████████▋| 217452/225000 [423:48:54<18:00:25,  8.59s/it]
 97%|█████████▋| 217453/225000 [423:49:04<18:38:10,  8.89s/it]
 97%|█████████▋| 217454/225000 [423:49:11<18:00:51,  8.59s/it]
 97%|█████████▋| 217455/225000 [423:49:16<15:43:18,  7.50s/it]
 97%|█████████▋| 217456/225000 [423:49:27<17:26:27,  8.32s/it]
 97%|█████████▋| 217457/225000 [423:49:41<21:26:48, 10.24s/it]
 97%|█████████▋| 217458/225000 [423:49:49<20:03:11,  9.57s/it]
 97%|█████████▋| 217459/225000 [423:49:56<18:14:20,  8.71s/it]
 97%|█████████▋| 217460/225000 [423:50:06<19:07:34,  9.13s/it]
 97%|█████████▋| 217461/225000 [423:50:16<19:23:04,  9.26s/it]
 97%|█████████▋| 217462/225000 [423:50:20<16:12:38,  7.74s/it]
 97%|█████████▋| 217463/225000 [423:50:32<18:38:26,  8.90s/it]
 97%|█████████▋| 217464/225000 [423:50:41<18:42:40,  8.94s/it]
 97%|█████████▋| 217465/225000 [423:50:50<19:13:55,  9.19s/it]
 97%|█████████▋| 217466/225000 [423:50:56<17:15:19,  8.25s/it]
 97%|█████████▋| 217467/225000 [423:51:06<17:57:13,  8.58s/it]
 97%|█████████▋| 217468/225000 [423:51:17<19:44:19,  9.43s/it]
 97%|█████████▋| 217469/225000 [423:51:25<18:31:48,  8.86s/it]
 97%|█████████▋| 217470/225000 [423:51:32<17:29:09,  8.36s/it]
 97%|█████████▋| 217471/225000 [423:51:39<16:47:08,  8.03s/it]
 97%|█████████▋| 217472/225000 [423:51:46<16:04:32,  7.69s/it]
 97%|█████████▋| 217473/225000 [423:51:53<15:35:10,  7.45s/it]
 97%|█████████▋| 217474/225000 [423:52:03<17:24:16,  8.33s/it]
 97%|█████████▋| 217475/225000 [423:52:14<19:10:15,  9.17s/it]
 97%|█████████▋| 217476/225000 [423:52:23<18:36:12,  8.90s/it]
 97%|█████████▋| 217477/225000 [423:52:34<19:47:51,  9.47s/it]
 97%|█████████▋| 217478/225000 [423:52:40<18:06:57,  8.67s/it]
 97%|█████████▋| 217479/225000 [423:52:52<20:13:06,  9.68s/it]
 97%|█████████▋| 217480/225000 [423:53:01<19:28:39,  9.32s/it]
 97%|█████████▋| 217481/225000 [423:53:07<17:42:18,  8.48s/it]
 97%|█████████▋| 217482/225000 [423:53:14<16:41:15,  7.99s/it]
 97%|█████████▋| 217483/225000 [423:53:28<20:03:05,  9.60s/it]
 97%|█████████▋| 217484/225000 [423:53:37<20:07:10,  9.64s/it]
 97%|█████████▋| 217485/225000 [423:53:50<21:51:24, 10.47s/it]
 97%|█████████▋| 217486/225000 [423:54:00<22:00:57, 10.55s/it]
 97%|█████████▋| 217487/225000 [423:54:05<18:33:12,  8.89s/it]
 97%|█████████▋| 217488/225000 [423:54:11<16:37:13,  7.97s/it]
 97%|█████████▋| 217489/225000 [423:54:18<15:59:35,  7.67s/it]
 97%|█████████▋| 217490/225000 [423:54:31<19:24:11,  9.30s/it]
 97%|█████████▋| 217491/225000 [423:54:40<18:43:18,  8.98s/it]
 97%|█████████▋| 217492/225000 [423:54:46<17:19:47,  8.31s/it]
 97%|█████████▋| 217493/225000 [423:54:56<17:58:34,  8.62s/it]
 97%|█████████▋| 217494/225000 [423:55:02<16:30:56,  7.92s/it]
 97%|█████████▋| 217495/225000 [423:55:11<17:13:26,  8.26s/it]
 97%|█████████▋| 217496/225000 [423:55:17<15:54:25,  7.63s/it]
 97%|█████████▋| 217497/225000 [423:55:26<16:43:20,  8.02s/it]
 97%|█████████▋| 217498/225000 [423:55:33<15:51:15,  7.61s/it]
 97%|█████████▋| 217499/225000 [423:55:49<21:05:10, 10.12s/it]
 97%|█████████▋| 217500/225000 [423:55:58<20:49:17,  9.99s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.356', 'learning_rate': '1.541e-07', 'epoch': '0.9667', 'num_input_tokens_seen': 720860264, 'train_runtime': '1.526e+06', 'train_tokens_per_second': '472.3'}

 97%|█████████▋| 217500/225000 [423:55:58<20:49:17,  9.99s/it]
 97%|█████████▋| 217501/225000 [423:56:05<18:32:28,  8.90s/it]
 97%|█████████▋| 217502/225000 [423:56:18<21:14:14, 10.20s/it]
 97%|█████████▋| 217503/225000 [423:56:24<18:36:54,  8.94s/it]
 97%|█████████▋| 217504/225000 [423:56:30<17:02:35,  8.19s/it]
 97%|█████████▋| 217505/225000 [423:56:37<15:50:00,  7.61s/it]
 97%|█████████▋| 217506/225000 [423:56:55<22:14:52, 10.69s/it]
 97%|█████████▋| 217507/225000 [423:57:00<19:05:02,  9.17s/it]
 97%|█████████▋| 217508/225000 [423:57:10<19:26:00,  9.34s/it]
 97%|█████████▋| 217509/225000 [423:57:18<18:25:56,  8.86s/it]
 97%|█████████▋| 217510/225000 [423:57:24<16:57:17,  8.15s/it]
 97%|█████████▋| 217511/225000 [423:57:37<19:41:10,  9.46s/it]
 97%|█████████▋| 217512/225000 [423:57:46<19:29:03,  9.37s/it]
 97%|█████████▋| 217513/225000 [423:57:53<17:57:08,  8.63s/it]
 97%|█████████▋| 217514/225000 [423:58:06<20:50:36, 10.02s/it]
 97%|█████████▋| 217515/225000 [423:58:14<19:24:40,  9.34s/it]
 97%|█████████▋| 217516/225000 [423:58:18<16:29:43,  7.93s/it]
 97%|█████████▋| 217517/225000 [423:58:29<18:05:32,  8.70s/it]
 97%|█████████▋| 217518/225000 [423:58:35<16:31:51,  7.95s/it]
 97%|█████████▋| 217519/225000 [423:58:41<15:22:20,  7.40s/it]
 97%|█████████▋| 217520/225000 [423:58:49<15:43:04,  7.56s/it]
 97%|█████████▋| 217521/225000 [423:58:56<15:18:19,  7.37s/it]
 97%|█████████▋| 217522/225000 [423:59:10<19:34:54,  9.43s/it]
 97%|█████████▋| 217523/225000 [423:59:22<20:41:43,  9.96s/it]
 97%|█████████▋| 217524/225000 [423:59:27<18:09:09,  8.74s/it]
 97%|█████████▋| 217525/225000 [423:59:35<17:27:14,  8.41s/it]
 97%|█████████▋| 217526/225000 [423:59:43<17:06:13,  8.24s/it]
 97%|█████████▋| 217527/225000 [423:59:57<20:32:47,  9.90s/it]
 97%|█████████▋| 217528/225000 [424:00:07<20:50:45, 10.04s/it]
 97%|█████████▋| 217529/225000 [424:00:14<18:43:58,  9.03s/it]
 97%|█████████▋| 217530/225000 [424:00:20<16:50:29,  8.12s/it]
 97%|█████████▋| 217531/225000 [424:00:28<16:58:29,  8.18s/it]
 97%|█████████▋| 217532/225000 [424:00:34<15:26:08,  7.44s/it]
 97%|█████████▋| 217533/225000 [424:00:46<18:28:07,  8.90s/it]
 97%|█████████▋| 217534/225000 [424:00:56<18:55:08,  9.12s/it]
 97%|█████████▋| 217535/225000 [424:01:03<17:36:35,  8.49s/it]
 97%|█████████▋| 217536/225000 [424:01:11<17:20:57,  8.37s/it]
 97%|█████████▋| 217537/225000 [424:01:24<20:18:08,  9.79s/it]
 97%|█████████▋| 217538/225000 [424:01:31<18:48:06,  9.07s/it]
 97%|█████████▋| 217539/225000 [424:01:42<19:59:41,  9.65s/it]
 97%|█████████▋| 217540/225000 [424:01:50<18:38:54,  9.00s/it]
 97%|█████████▋| 217541/225000 [424:01:59<18:46:32,  9.06s/it]
 97%|█████████▋| 217542/225000 [424:02:06<17:44:06,  8.56s/it]
 97%|█████████▋| 217543/225000 [424:02:20<20:38:51,  9.97s/it]
 97%|█████████▋| 217544/225000 [424:02:27<19:07:01,  9.23s/it]
 97%|█████████▋| 217545/225000 [424:02:34<17:43:24,  8.56s/it]
 97%|█████████▋| 217546/225000 [424:02:45<19:17:16,  9.32s/it]
 97%|█████████▋| 217547/225000 [424:02:55<19:27:03,  9.40s/it]
 97%|█████████▋| 217548/225000 [424:03:07<21:04:18, 10.18s/it]
 97%|█████████▋| 217549/225000 [424:03:15<19:33:16,  9.45s/it]
 97%|█████████▋| 217550/225000 [424:03:23<18:55:41,  9.15s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.284', 'learning_rate': '1.521e-07', 'epoch': '0.9669', 'num_input_tokens_seen': 721093736, 'train_runtime': '1.527e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217550/225000 [424:03:23<18:55:41,  9.15s/it]
 97%|█████████▋| 217551/225000 [424:03:36<21:24:06, 10.34s/it]
 97%|█████████▋| 217552/225000 [424:03:43<19:00:49,  9.19s/it]
 97%|█████████▋| 217553/225000 [424:03:54<20:07:09,  9.73s/it]
 97%|█████████▋| 217554/225000 [424:04:00<18:13:55,  8.81s/it]
 97%|█████████▋| 217555/225000 [424:04:07<16:40:56,  8.07s/it]
 97%|█████████▋| 217556/225000 [424:04:14<16:30:51,  7.99s/it]
 97%|█████████▋| 217557/225000 [424:04:23<16:58:33,  8.21s/it]
 97%|█████████▋| 217558/225000 [424:04:40<22:03:28, 10.67s/it]
 97%|█████████▋| 217559/225000 [424:04:48<20:24:50,  9.88s/it]
 97%|█████████▋| 217560/225000 [424:04:58<20:35:52,  9.97s/it]
 97%|█████████▋| 217561/225000 [424:05:06<19:38:33,  9.51s/it]
 97%|█████████▋| 217562/225000 [424:05:13<18:10:24,  8.80s/it]
 97%|█████████▋| 217563/225000 [424:05:23<18:55:56,  9.16s/it]
 97%|█████████▋| 217564/225000 [424:05:28<16:01:53,  7.76s/it]
 97%|█████████▋| 217565/225000 [424:05:37<17:09:41,  8.31s/it]
 97%|█████████▋| 217566/225000 [424:05:49<18:55:00,  9.16s/it]
 97%|█████████▋| 217567/225000 [424:05:56<18:04:56,  8.76s/it]
 97%|█████████▋| 217568/225000 [424:06:04<17:35:08,  8.52s/it]
 97%|█████████▋| 217569/225000 [424:06:10<16:00:33,  7.76s/it]
 97%|█████████▋| 217570/225000 [424:06:20<17:10:33,  8.32s/it]
 97%|█████████▋| 217571/225000 [424:06:30<18:25:45,  8.93s/it]
 97%|█████████▋| 217572/225000 [424:06:41<19:47:59,  9.60s/it]
 97%|█████████▋| 217573/225000 [424:06:47<17:09:44,  8.32s/it]
 97%|█████████▋| 217574/225000 [424:06:58<18:41:42,  9.06s/it]
 97%|█████████▋| 217575/225000 [424:07:05<17:36:19,  8.54s/it]
 97%|█████████▋| 217576/225000 [424:07:16<19:16:29,  9.35s/it]
 97%|█████████▋| 217577/225000 [424:07:29<21:39:08, 10.50s/it]
 97%|█████████▋| 217578/225000 [424:07:36<19:31:18,  9.47s/it]
 97%|█████████▋| 217579/225000 [424:07:43<17:57:55,  8.72s/it]
 97%|█████████▋| 217580/225000 [424:07:54<19:03:32,  9.25s/it]
 97%|█████████▋| 217581/225000 [424:08:02<18:25:22,  8.94s/it]
 97%|█████████▋| 217582/225000 [424:08:11<18:14:07,  8.85s/it]
 97%|█████████▋| 217583/225000 [424:08:19<17:44:09,  8.61s/it]
 97%|█████████▋| 217584/225000 [424:08:30<19:14:13,  9.34s/it]
 97%|█████████▋| 217585/225000 [424:08:39<19:02:04,  9.24s/it]
 97%|█████████▋| 217586/225000 [424:08:44<16:26:57,  7.99s/it]
 97%|█████████▋| 217587/225000 [424:08:53<16:51:48,  8.19s/it]
 97%|█████████▋| 217588/225000 [424:09:08<21:07:06, 10.26s/it]
 97%|█████████▋| 217589/225000 [424:09:13<17:56:54,  8.72s/it]
 97%|█████████▋| 217590/225000 [424:09:21<17:26:09,  8.47s/it]
 97%|█████████▋| 217591/225000 [424:09:31<18:53:31,  9.18s/it]
 97%|█████████▋| 217592/225000 [424:09:42<19:36:07,  9.53s/it]
 97%|█████████▋| 217593/225000 [424:09:51<19:38:28,  9.55s/it]
 97%|█████████▋| 217594/225000 [424:10:03<20:48:53, 10.12s/it]
 97%|█████████▋| 217595/225000 [424:10:10<18:45:30,  9.12s/it]
 97%|█████████▋| 217596/225000 [424:10:18<18:19:30,  8.91s/it]
 97%|█████████▋| 217597/225000 [424:10:27<18:09:52,  8.83s/it]
 97%|█████████▋| 217598/225000 [424:10:38<19:27:55,  9.47s/it]
 97%|█████████▋| 217599/225000 [424:10:47<19:21:33,  9.42s/it]
 97%|█████████▋| 217600/225000 [424:10:52<16:36:41,  8.08s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.906', 'learning_rate': '1.501e-07', 'epoch': '0.9671', 'num_input_tokens_seen': 721326176, 'train_runtime': '1.527e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217600/225000 [424:10:52<16:36:41,  8.08s/it]
 97%|█████████▋| 217601/225000 [424:11:02<17:41:06,  8.60s/it]
 97%|█████████▋| 217602/225000 [424:11:11<17:50:26,  8.68s/it]
 97%|█████████▋| 217603/225000 [424:11:19<17:35:52,  8.56s/it]
 97%|█████████▋| 217604/225000 [424:11:24<15:39:41,  7.62s/it]
 97%|█████████▋| 217605/225000 [424:11:36<18:11:35,  8.86s/it]
 97%|█████████▋| 217606/225000 [424:11:43<16:46:26,  8.17s/it]
 97%|█████████▋| 217607/225000 [424:11:55<19:35:38,  9.54s/it]
 97%|█████████▋| 217608/225000 [424:12:06<19:58:44,  9.73s/it]
 97%|█████████▋| 217609/225000 [424:12:12<18:07:11,  8.83s/it]
 97%|█████████▋| 217610/225000 [424:12:23<19:06:18,  9.31s/it]
 97%|█████████▋| 217611/225000 [424:12:29<17:17:50,  8.43s/it]
 97%|█████████▋| 217612/225000 [424:12:35<15:44:00,  7.67s/it]
 97%|█████████▋| 217613/225000 [424:12:48<19:17:18,  9.40s/it]
 97%|█████████▋| 217614/225000 [424:12:54<16:45:52,  8.17s/it]
 97%|█████████▋| 217615/225000 [424:13:00<15:45:51,  7.68s/it]
 97%|█████████▋| 217616/225000 [424:13:12<18:17:39,  8.92s/it]
 97%|█████████▋| 217617/225000 [424:13:21<18:06:54,  8.83s/it]
 97%|█████████▋| 217618/225000 [424:13:29<17:59:46,  8.78s/it]
 97%|█████████▋| 217619/225000 [424:13:37<17:08:13,  8.36s/it]
 97%|█████████▋| 217620/225000 [424:13:48<18:59:57,  9.27s/it]
 97%|█████████▋| 217621/225000 [424:14:00<20:45:22, 10.13s/it]
 97%|█████████▋| 217622/225000 [424:14:09<19:51:44,  9.69s/it]
 97%|█████████▋| 217623/225000 [424:14:19<19:58:17,  9.75s/it]
 97%|█████████▋| 217624/225000 [424:14:31<21:13:46, 10.36s/it]
 97%|█████████▋| 217625/225000 [424:14:38<19:43:19,  9.63s/it]
 97%|█████████▋| 217626/225000 [424:14:46<18:19:18,  8.94s/it]
 97%|█████████▋| 217627/225000 [424:14:53<17:12:16,  8.40s/it]
 97%|█████████▋| 217628/225000 [424:15:00<16:24:46,  8.01s/it]
 97%|█████████▋| 217629/225000 [424:15:10<17:41:46,  8.64s/it]
 97%|█████████▋| 217630/225000 [424:15:16<16:08:53,  7.89s/it]
 97%|█████████▋| 217631/225000 [424:15:23<15:37:01,  7.63s/it]
 97%|█████████▋| 217632/225000 [424:15:37<19:00:54,  9.29s/it]
 97%|█████████▋| 217633/225000 [424:15:45<18:30:57,  9.05s/it]
 97%|█████████▋| 217634/225000 [424:15:55<18:56:31,  9.26s/it]
 97%|█████████▋| 217635/225000 [424:16:02<17:26:05,  8.52s/it]
 97%|█████████▋| 217636/225000 [424:16:11<18:00:22,  8.80s/it]
 97%|█████████▋| 217637/225000 [424:16:16<15:48:04,  7.73s/it]
 97%|█████████▋| 217638/225000 [424:16:24<15:50:00,  7.74s/it]
 97%|█████████▋| 217639/225000 [424:16:32<16:01:46,  7.84s/it]
 97%|█████████▋| 217640/225000 [424:16:42<17:04:55,  8.36s/it]
 97%|█████████▋| 217641/225000 [424:16:50<17:10:18,  8.40s/it]
 97%|█████████▋| 217642/225000 [424:16:58<16:33:57,  8.11s/it]
 97%|█████████▋| 217643/225000 [424:17:07<17:21:46,  8.50s/it]
 97%|█████████▋| 217644/225000 [424:17:13<15:43:57,  7.70s/it]
 97%|█████████▋| 217645/225000 [424:17:19<14:41:24,  7.19s/it]
 97%|█████████▋| 217646/225000 [424:17:27<15:31:57,  7.60s/it]
 97%|█████████▋| 217647/225000 [424:17:38<17:17:44,  8.47s/it]
 97%|█████████▋| 217648/225000 [424:17:45<16:28:19,  8.07s/it]
 97%|█████████▋| 217649/225000 [424:17:54<17:09:14,  8.40s/it]
 97%|█████████▋| 217650/225000 [424:18:04<18:12:45,  8.92s/it]
                                                              
{'loss': '1.315', 'grad_norm': '1.306', 'learning_rate': '1.481e-07', 'epoch': '0.9673', 'num_input_tokens_seen': 721555032, 'train_runtime': '1.527e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217650/225000 [424:18:04<18:12:45,  8.92s/it]
 97%|█████████▋| 217651/225000 [424:18:13<18:03:03,  8.84s/it]
 97%|█████████▋| 217652/225000 [424:18:21<17:27:42,  8.56s/it]
 97%|█████████▋| 217653/225000 [424:18:29<17:08:15,  8.40s/it]
 97%|█████████▋| 217654/225000 [424:18:40<19:02:45,  9.33s/it]
 97%|█████████▋| 217655/225000 [424:18:47<17:10:10,  8.42s/it]
 97%|█████████▋| 217656/225000 [424:18:54<16:36:04,  8.14s/it]
 97%|█████████▋| 217657/225000 [424:19:07<19:22:53,  9.50s/it]
 97%|█████████▋| 217658/225000 [424:19:11<16:24:15,  8.04s/it]
 97%|█████████▋| 217659/225000 [424:19:25<19:49:00,  9.72s/it]
 97%|█████████▋| 217660/225000 [424:19:32<17:55:58,  8.80s/it]
 97%|█████████▋| 217661/225000 [424:19:43<19:34:04,  9.60s/it]
 97%|█████████▋| 217662/225000 [424:19:52<19:17:47,  9.47s/it]
 97%|█████████▋| 217663/225000 [424:19:57<16:01:59,  7.87s/it]
 97%|█████████▋| 217664/225000 [424:20:06<16:59:41,  8.34s/it]
 97%|█████████▋| 217665/225000 [424:20:15<17:12:24,  8.44s/it]
 97%|█████████▋| 217666/225000 [424:20:26<18:59:35,  9.32s/it]
 97%|█████████▋| 217667/225000 [424:20:37<20:04:13,  9.85s/it]
 97%|█████████▋| 217668/225000 [424:20:43<17:49:46,  8.75s/it]
 97%|█████████▋| 217669/225000 [424:20:56<20:13:16,  9.93s/it]
 97%|█████████▋| 217670/225000 [424:21:02<17:52:33,  8.78s/it]
 97%|█████████▋| 217671/225000 [424:21:11<18:01:53,  8.86s/it]
 97%|█████████▋| 217672/225000 [424:21:19<17:40:53,  8.69s/it]
 97%|█████████▋| 217673/225000 [424:21:24<15:05:08,  7.41s/it]
 97%|█████████▋| 217674/225000 [424:21:33<15:58:26,  7.85s/it]
 97%|█████████▋| 217675/225000 [424:21:43<17:11:41,  8.45s/it]
 97%|█████████▋| 217676/225000 [424:21:51<17:05:45,  8.40s/it]
 97%|█████████▋| 217677/225000 [424:21:58<16:14:06,  7.98s/it]
 97%|█████████▋| 217678/225000 [424:22:10<18:41:06,  9.19s/it]
 97%|█████████▋| 217679/225000 [424:22:20<19:08:36,  9.41s/it]
 97%|█████████▋| 217680/225000 [424:22:26<16:54:09,  8.31s/it]
 97%|█████████▋| 217681/225000 [424:22:33<16:26:54,  8.09s/it]
 97%|█████████▋| 217682/225000 [424:22:46<19:18:29,  9.50s/it]
 97%|█████████▋| 217683/225000 [424:22:54<18:17:43,  9.00s/it]
 97%|█████████▋| 217684/225000 [424:23:01<16:58:39,  8.35s/it]
 97%|█████████▋| 217685/225000 [424:23:09<16:54:05,  8.32s/it]
 97%|█████████▋| 217686/225000 [424:23:20<18:26:39,  9.08s/it]
 97%|█████████▋| 217687/225000 [424:23:28<17:42:26,  8.72s/it]
 97%|█████████▋| 217688/225000 [424:23:38<18:29:31,  9.10s/it]
 97%|█████████▋| 217689/225000 [424:23:44<16:34:50,  8.16s/it]
 97%|█████████▋| 217690/225000 [424:23:50<15:41:45,  7.73s/it]
 97%|█████████▋| 217691/225000 [424:23:57<15:05:43,  7.44s/it]
 97%|█████████▋| 217692/225000 [424:24:12<19:30:27,  9.61s/it]
 97%|█████████▋| 217693/225000 [424:24:18<17:47:31,  8.77s/it]
 97%|█████████▋| 217694/225000 [424:24:26<16:50:06,  8.30s/it]
 97%|█████████▋| 217695/225000 [424:24:37<18:56:08,  9.33s/it]
 97%|█████████▋| 217696/225000 [424:24:45<17:58:57,  8.86s/it]
 97%|█████████▋| 217697/225000 [424:24:51<15:54:35,  7.84s/it]
 97%|█████████▋| 217698/225000 [424:24:59<15:59:52,  7.89s/it]
 97%|█████████▋| 217699/225000 [424:25:06<15:40:22,  7.73s/it]
 97%|█████████▋| 217700/225000 [424:25:20<19:46:24,  9.75s/it]
                                                              
{'loss': '1.309', 'grad_norm': '1.083', 'learning_rate': '1.461e-07', 'epoch': '0.9676', 'num_input_tokens_seen': 721783352, 'train_runtime': '1.528e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217700/225000 [424:25:20<19:46:24,  9.75s/it]
 97%|█████████▋| 217701/225000 [424:25:28<18:39:16,  9.20s/it]
 97%|█████████▋| 217702/225000 [424:25:38<19:10:43,  9.46s/it]
 97%|█████████▋| 217703/225000 [424:25:45<17:34:34,  8.67s/it]
 97%|█████████▋| 217704/225000 [424:25:56<18:51:06,  9.30s/it]
 97%|█████████▋| 217705/225000 [424:26:01<16:01:07,  7.91s/it]
 97%|█████████▋| 217706/225000 [424:26:12<18:06:54,  8.94s/it]
 97%|█████████▋| 217707/225000 [424:26:19<16:58:51,  8.38s/it]
 97%|█████████▋| 217708/225000 [424:26:38<23:10:59, 11.45s/it]
 97%|█████████▋| 217709/225000 [424:26:43<19:33:34,  9.66s/it]
 97%|█████████▋| 217710/225000 [424:26:50<17:40:00,  8.72s/it]
 97%|█████████▋| 217711/225000 [424:26:56<16:24:25,  8.10s/it]
 97%|█████████▋| 217712/225000 [424:27:04<16:15:45,  8.03s/it]
 97%|█████████▋| 217713/225000 [424:27:13<16:58:54,  8.39s/it]
 97%|█████████▋| 217714/225000 [424:27:19<15:10:03,  7.49s/it]
 97%|█████████▋| 217715/225000 [424:27:25<14:05:29,  6.96s/it]
 97%|█████████▋| 217716/225000 [424:27:38<18:06:53,  8.95s/it]
 97%|█████████▋| 217717/225000 [424:27:45<17:04:22,  8.44s/it]
 97%|█████████▋| 217718/225000 [424:27:55<17:34:36,  8.69s/it]
 97%|█████████▋| 217719/225000 [424:28:05<18:39:25,  9.22s/it]
 97%|█████████▋| 217720/225000 [424:28:11<16:44:10,  8.28s/it]
 97%|█████████▋| 217721/225000 [424:28:26<20:28:15, 10.12s/it]
 97%|█████████▋| 217722/225000 [424:28:34<19:15:33,  9.53s/it]
 97%|█████████▋| 217723/225000 [424:28:40<17:28:11,  8.64s/it]
 97%|█████████▋| 217724/225000 [424:28:45<15:09:08,  7.50s/it]
 97%|█████████▋| 217725/225000 [424:28:59<18:55:01,  9.36s/it]
 97%|█████████▋| 217726/225000 [424:29:06<17:16:59,  8.55s/it]
 97%|█████████▋| 217727/225000 [424:29:15<17:49:17,  8.82s/it]
 97%|█████████▋| 217728/225000 [424:29:19<14:50:26,  7.35s/it]
 97%|█████████▋| 217729/225000 [424:29:25<13:50:45,  6.86s/it]
 97%|█████████▋| 217730/225000 [424:29:31<13:33:27,  6.71s/it]
 97%|█████████▋| 217731/225000 [424:29:42<15:52:49,  7.86s/it]
 97%|█████████▋| 217732/225000 [424:29:56<19:45:33,  9.79s/it]
 97%|█████████▋| 217733/225000 [424:30:01<17:01:14,  8.43s/it]
 97%|█████████▋| 217734/225000 [424:30:11<17:59:30,  8.91s/it]
 97%|█████████▋| 217735/225000 [424:30:19<17:36:11,  8.72s/it]
 97%|█████████▋| 217736/225000 [424:30:28<17:39:23,  8.75s/it]
 97%|█████████▋| 217737/225000 [424:30:38<18:10:06,  9.01s/it]
 97%|█████████▋| 217738/225000 [424:30:45<16:45:47,  8.31s/it]
 97%|█████████▋| 217739/225000 [424:30:53<17:03:42,  8.46s/it]
 97%|█████████▋| 217740/225000 [424:31:05<18:47:17,  9.32s/it]
 97%|█████████▋| 217741/225000 [424:31:11<16:46:05,  8.32s/it]
 97%|█████████▋| 217742/225000 [424:31:16<15:03:48,  7.47s/it]
 97%|█████████▋| 217743/225000 [424:31:28<17:33:35,  8.71s/it]
 97%|█████████▋| 217744/225000 [424:31:37<17:46:30,  8.82s/it]
 97%|█████████▋| 217745/225000 [424:31:45<17:26:03,  8.65s/it]
 97%|█████████▋| 217746/225000 [424:31:53<17:04:14,  8.47s/it]
 97%|█████████▋| 217747/225000 [424:32:05<18:48:30,  9.34s/it]
 97%|█████████▋| 217748/225000 [424:32:12<17:57:13,  8.91s/it]
 97%|█████████▋| 217749/225000 [424:32:19<16:27:10,  8.17s/it]
 97%|█████████▋| 217750/225000 [424:32:30<18:25:32,  9.15s/it]
                                                              
{'loss': '1.342', 'grad_norm': '1.324', 'learning_rate': '1.441e-07', 'epoch': '0.9678', 'num_input_tokens_seen': 722010472, 'train_runtime': '1.528e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217750/225000 [424:32:30<18:25:32,  9.15s/it]
 97%|█████████▋| 217751/225000 [424:32:44<21:25:41, 10.64s/it]
 97%|█████████▋| 217752/225000 [424:32:52<19:35:44,  9.73s/it]
 97%|█████████▋| 217753/225000 [424:32:57<16:39:39,  8.28s/it]
 97%|█████████▋| 217754/225000 [424:33:02<14:37:35,  7.27s/it]
 97%|█████████▋| 217755/225000 [424:33:14<17:31:47,  8.71s/it]
 97%|█████████▋| 217756/225000 [424:33:21<16:45:03,  8.32s/it]
 97%|█████████▋| 217757/225000 [424:33:33<18:37:02,  9.25s/it]
 97%|█████████▋| 217758/225000 [424:33:43<19:20:44,  9.62s/it]
 97%|█████████▋| 217759/225000 [424:33:50<17:52:01,  8.88s/it]
 97%|█████████▋| 217760/225000 [424:33:59<17:56:47,  8.92s/it]
 97%|█████████▋| 217761/225000 [424:34:06<16:45:44,  8.34s/it]
 97%|█████████▋| 217762/225000 [424:34:15<16:49:10,  8.37s/it]
 97%|█████████▋| 217763/225000 [424:34:31<21:13:52, 10.56s/it]
 97%|█████████▋| 217764/225000 [424:34:39<19:53:31,  9.90s/it]
 97%|█████████▋| 217765/225000 [424:34:47<18:39:19,  9.28s/it]
 97%|█████████▋| 217766/225000 [424:34:55<18:15:11,  9.08s/it]
 97%|█████████▋| 217767/225000 [424:35:01<16:18:29,  8.12s/it]
 97%|█████████▋| 217768/225000 [424:35:17<20:49:47, 10.37s/it]
 97%|█████████▋| 217769/225000 [424:35:23<18:24:50,  9.17s/it]
 97%|█████████▋| 217770/225000 [424:35:33<18:56:13,  9.43s/it]
 97%|█████████▋| 217771/225000 [424:35:46<20:41:21, 10.30s/it]
 97%|█████████▋| 217772/225000 [424:35:50<17:18:50,  8.62s/it]
 97%|█████████▋| 217773/225000 [424:35:58<16:46:59,  8.36s/it]
 97%|█████████▋| 217774/225000 [424:36:08<17:49:44,  8.88s/it]
 97%|█████████▋| 217775/225000 [424:36:18<18:26:04,  9.19s/it]
 97%|█████████▋| 217776/225000 [424:36:25<16:52:41,  8.41s/it]
 97%|█████████▋| 217777/225000 [424:36:34<17:35:36,  8.77s/it]
 97%|█████████▋| 217778/225000 [424:36:40<15:43:46,  7.84s/it]
 97%|█████████▋| 217779/225000 [424:36:52<18:11:23,  9.07s/it]
 97%|█████████▋| 217780/225000 [424:36:59<17:08:54,  8.55s/it]
 97%|█████████▋| 217781/225000 [424:37:08<17:18:19,  8.63s/it]
 97%|█████████▋| 217782/225000 [424:37:15<16:04:51,  8.02s/it]
 97%|█████████▋| 217783/225000 [424:37:22<15:27:10,  7.71s/it]
 97%|█████████▋| 217784/225000 [424:37:33<17:40:29,  8.82s/it]
 97%|█████████▋| 217785/225000 [424:37:39<15:49:19,  7.89s/it]
 97%|█████████▋| 217786/225000 [424:37:46<15:29:43,  7.73s/it]
 97%|█████████▋| 217787/225000 [424:37:56<16:59:03,  8.48s/it]
 97%|█████████▋| 217788/225000 [424:38:08<18:46:02,  9.37s/it]
 97%|█████████▋| 217789/225000 [424:38:13<16:20:07,  8.16s/it]
 97%|█████████▋| 217790/225000 [424:38:25<18:54:20,  9.44s/it]
 97%|█████████▋| 217791/225000 [424:38:35<18:58:16,  9.47s/it]
 97%|█████████▋| 217792/225000 [424:38:41<16:46:57,  8.38s/it]
 97%|█████████▋| 217793/225000 [424:38:53<18:55:03,  9.45s/it]
 97%|█████████▋| 217794/225000 [424:39:02<18:42:46,  9.35s/it]
 97%|█████████▋| 217795/225000 [424:39:11<18:48:11,  9.40s/it]
 97%|█████████▋| 217796/225000 [424:39:21<18:40:17,  9.33s/it]
 97%|█████████▋| 217797/225000 [424:39:25<15:56:42,  7.97s/it]
 97%|█████████▋| 217798/225000 [424:39:40<19:59:41,  9.99s/it]
 97%|█████████▋| 217799/225000 [424:39:45<17:00:06,  8.50s/it]
 97%|█████████▋| 217800/225000 [424:39:54<17:06:44,  8.56s/it]
                                                              
{'loss': '1.295', 'grad_norm': '1.235', 'learning_rate': '1.422e-07', 'epoch': '0.968', 'num_input_tokens_seen': 722242128, 'train_runtime': '1.529e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217800/225000 [424:39:54<17:06:44,  8.56s/it]
 97%|█████████▋| 217801/225000 [424:40:01<16:25:43,  8.22s/it]
 97%|█████████▋| 217802/225000 [424:40:12<17:42:42,  8.86s/it]
 97%|█████████▋| 217803/225000 [424:40:21<17:52:02,  8.94s/it]
 97%|█████████▋| 217804/225000 [424:40:26<15:33:38,  7.78s/it]
 97%|█████████▋| 217805/225000 [424:40:33<14:55:30,  7.47s/it]
 97%|█████████▋| 217806/225000 [424:40:39<14:24:19,  7.21s/it]
 97%|█████████▋| 217807/225000 [424:40:52<17:40:32,  8.85s/it]
 97%|█████████▋| 217808/225000 [424:40:58<15:49:13,  7.92s/it]
 97%|█████████▋| 217809/225000 [424:41:07<16:47:37,  8.41s/it]
 97%|█████████▋| 217810/225000 [424:41:17<17:47:06,  8.90s/it]
 97%|█████████▋| 217811/225000 [424:41:26<17:47:38,  8.91s/it]
 97%|█████████▋| 217812/225000 [424:41:33<16:40:27,  8.35s/it]
 97%|█████████▋| 217813/225000 [424:41:44<18:18:33,  9.17s/it]
 97%|█████████▋| 217814/225000 [424:41:55<18:58:22,  9.50s/it]
 97%|█████████▋| 217815/225000 [424:42:01<16:52:57,  8.46s/it]
 97%|█████████▋| 217816/225000 [424:42:11<17:58:41,  9.01s/it]
 97%|█████████▋| 217817/225000 [424:42:21<18:27:39,  9.25s/it]
 97%|█████████▋| 217818/225000 [424:42:31<19:20:12,  9.69s/it]
 97%|█████████▋| 217819/225000 [424:42:38<17:32:53,  8.80s/it]
 97%|█████████▋| 217820/225000 [424:42:49<19:04:14,  9.56s/it]
 97%|█████████▋| 217821/225000 [424:42:54<16:00:16,  8.03s/it]
 97%|█████████▋| 217822/225000 [424:43:03<16:43:40,  8.39s/it]
 97%|█████████▋| 217823/225000 [424:43:14<17:56:34,  9.00s/it]
 97%|█████████▋| 217824/225000 [424:43:22<17:50:46,  8.95s/it]
 97%|█████████▋| 217825/225000 [424:43:28<15:39:33,  7.86s/it]
 97%|█████████▋| 217826/225000 [424:43:35<15:31:46,  7.79s/it]
 97%|█████████▋| 217827/225000 [424:43:49<19:16:54,  9.68s/it]
 97%|█████████▋| 217828/225000 [424:43:56<17:40:30,  8.87s/it]
 97%|█████████▋| 217829/225000 [424:44:03<16:13:37,  8.15s/it]
 97%|█████████▋| 217830/225000 [424:44:14<18:17:58,  9.19s/it]
 97%|█████████▋| 217831/225000 [424:44:19<15:38:06,  7.85s/it]
 97%|█████████▋| 217832/225000 [424:44:25<14:20:50,  7.21s/it]
 97%|█████████▋| 217833/225000 [424:44:34<15:40:51,  7.88s/it]
 97%|█████████▋| 217834/225000 [424:44:44<16:35:13,  8.33s/it]
 97%|█████████▋| 217835/225000 [424:44:54<17:30:07,  8.79s/it]
 97%|█████████▋| 217836/225000 [424:45:02<17:03:14,  8.57s/it]
 97%|█████████▋| 217837/225000 [424:45:10<16:46:44,  8.43s/it]
 97%|█████████▋| 217838/225000 [424:45:20<18:08:08,  9.12s/it]
 97%|█████████▋| 217839/225000 [424:45:27<16:29:13,  8.29s/it]
 97%|█████████▋| 217840/225000 [424:45:36<16:55:55,  8.51s/it]
 97%|█████████▋| 217841/225000 [424:45:52<21:28:10, 10.80s/it]
 97%|█████████▋| 217842/225000 [424:46:00<19:57:24, 10.04s/it]
 97%|█████████▋| 217843/225000 [424:46:04<16:30:02,  8.30s/it]
 97%|█████████▋| 217844/225000 [424:46:10<15:07:19,  7.61s/it]
 97%|█████████▋| 217845/225000 [424:46:19<15:45:18,  7.93s/it]
 97%|█████████▋| 217846/225000 [424:46:28<16:07:18,  8.11s/it]
 97%|█████████▋| 217847/225000 [424:46:34<15:04:27,  7.59s/it]
 97%|█████████▋| 217848/225000 [424:46:41<14:32:27,  7.32s/it]
 97%|█████████▋| 217849/225000 [424:46:50<15:54:51,  8.01s/it]
 97%|█████████▋| 217850/225000 [424:47:00<16:39:21,  8.39s/it]
                                                              
{'loss': '1.329', 'grad_norm': '1.379', 'learning_rate': '1.402e-07', 'epoch': '0.9682', 'num_input_tokens_seen': 722472600, 'train_runtime': '1.529e+06', 'train_tokens_per_second': '472.4'}

 97%|█████████▋| 217850/225000 [424:47:00<16:39:21,  8.39s/it]
 97%|█████████▋| 217851/225000 [424:47:10<17:52:32,  9.00s/it]
 97%|█████████▋| 217852/225000 [424:47:18<17:24:55,  8.77s/it]
 97%|█████████▋| 217853/225000 [424:47:26<16:40:26,  8.40s/it]
 97%|█████████▋| 217854/225000 [424:47:36<17:56:25,  9.04s/it]
 97%|█████████▋| 217855/225000 [424:47:43<16:39:38,  8.39s/it]
 97%|█████████▋| 217856/225000 [424:47:51<16:13:48,  8.18s/it]
 97%|█████████▋| 217857/225000 [424:47:55<13:58:20,  7.04s/it]
 97%|█████████▋| 217858/225000 [424:48:05<15:15:41,  7.69s/it]
 97%|█████████▋| 217859/225000 [424:48:13<15:42:02,  7.92s/it]
 97%|█████████▋| 217860/225000 [424:48:24<17:28:39,  8.81s/it]
 97%|█████████▋| 217861/225000 [424:48:33<17:51:39,  9.01s/it]
 97%|█████████▋| 217862/225000 [424:48:42<17:25:55,  8.79s/it]
 97%|█████████▋| 217863/225000 [424:48:50<16:53:05,  8.52s/it]
 97%|█████████▋| 217864/225000 [424:48:58<17:07:16,  8.64s/it]
 97%|█████████▋| 217865/225000 [424:49:09<18:32:06,  9.35s/it]
 97%|█████████▋| 217866/225000 [424:49:17<17:14:33,  8.70s/it]
 97%|█████████▋| 217867/225000 [424:49:27<18:18:26,  9.24s/it]
 97%|█████████▋| 217868/225000 [424:49:37<18:30:40,  9.34s/it]
 97%|█████████▋| 217869/225000 [424:49:47<18:55:20,  9.55s/it]
 97%|█████████▋| 217870/225000 [424:49:53<16:42:41,  8.44s/it]
 97%|█████████▋| 217871/225000 [424:49:58<15:09:10,  7.65s/it]
 97%|█████████▋| 217872/225000 [424:50:09<16:55:42,  8.55s/it]
 97%|█████████▋| 217873/225000 [424:50:16<16:13:49,  8.20s/it]
 97%|█████████▋| 217874/225000 [424:50:21<14:05:03,  7.12s/it]
 97%|█████████▋| 217875/225000 [424:50:32<16:21:23,  8.26s/it]
 97%|█████████▋| 217876/225000 [424:50:46<19:45:09,  9.98s/it]
 97%|█████████▋| 217877/225000 [424:50:57<20:26:58, 10.34s/it]
 97%|█████████▋| 217878/225000 [424:51:01<16:46:28,  8.48s/it]
 97%|█████████▋| 217879/225000 [424:51:11<17:44:22,  8.97s/it]
 97%|█████████▋| 217880/225000 [424:51:18<16:11:07,  8.18s/it]
 97%|█████████▋| 217881/225000 [424:51:24<15:11:01,  7.68s/it]
 97%|█████████▋| 217882/225000 [424:51:37<18:15:14,  9.23s/it]
 97%|█████████▋| 217883/225000 [424:51:45<17:18:11,  8.75s/it]
 97%|█████████▋| 217884/225000 [424:51:56<19:01:51,  9.63s/it]
 97%|█████████▋| 217885/225000 [424:52:02<16:36:33,  8.40s/it]
 97%|█████████▋| 217886/225000 [424:52:08<15:01:45,  7.61s/it]
 97%|█████████▋| 217887/225000 [424:52:20<17:37:38,  8.92s/it]
 97%|█████████▋| 217888/225000 [424:52:28<17:17:35,  8.75s/it]
 97%|█████████▋| 217889/225000 [424:52:36<17:07:34,  8.67s/it]
 97%|█████████▋| 217890/225000 [424:52:42<14:58:33,  7.58s/it]
 97%|█████████▋| 217891/225000 [424:52:50<15:18:56,  7.76s/it]
 97%|█████████▋| 217892/225000 [424:52:57<15:04:57,  7.64s/it]
 97%|█████████▋| 217893/225000 [424:53:11<18:36:40,  9.43s/it]
 97%|█████████▋| 217894/225000 [424:53:17<16:49:20,  8.52s/it]
 97%|█████████▋| 217895/225000 [424:53:24<15:51:59,  8.04s/it]
 97%|█████████▋| 217896/225000 [424:53:32<15:33:36,  7.89s/it]
 97%|█████████▋| 217897/225000 [424:53:40<15:56:54,  8.08s/it]
 97%|█████████▋| 217898/225000 [424:53:51<17:52:34,  9.06s/it]
 97%|█████████▋| 217899/225000 [424:54:00<17:35:53,  8.92s/it]
 97%|█████████▋| 217900/225000 [424:54:05<15:32:33,  7.88s/it]
                                                              
{'loss': '1.336', 'grad_norm': '1.565', 'learning_rate': '1.383e-07', 'epoch': '0.9684', 'num_input_tokens_seen': 722696992, 'train_runtime': '1.53e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 217900/225000 [424:54:05<15:32:33,  7.88s/it]
 97%|█████████▋| 217901/225000 [424:54:12<14:49:56,  7.52s/it]
 97%|█████████▋| 217902/225000 [424:54:27<18:57:10,  9.61s/it]
 97%|█████████▋| 217903/225000 [424:54:32<16:20:32,  8.29s/it]
 97%|█████████▋| 217904/225000 [424:54:39<15:35:27,  7.91s/it]
 97%|█████████▋| 217905/225000 [424:54:52<18:31:02,  9.40s/it]
 97%|█████████▋| 217906/225000 [424:55:00<17:43:56,  9.00s/it]
 97%|█████████▋| 217907/225000 [424:55:08<17:17:01,  8.77s/it]
 97%|█████████▋| 217908/225000 [424:55:16<16:46:27,  8.51s/it]
 97%|█████████▋| 217909/225000 [424:55:26<17:40:40,  8.97s/it]
 97%|█████████▋| 217910/225000 [424:55:31<15:27:11,  7.85s/it]
 97%|█████████▋| 217911/225000 [424:55:39<15:23:40,  7.82s/it]
 97%|█████████▋| 217912/225000 [424:55:50<17:19:23,  8.80s/it]
 97%|█████████▋| 217913/225000 [424:55:59<17:16:09,  8.77s/it]
 97%|█████████▋| 217914/225000 [424:56:04<15:06:59,  7.68s/it]
 97%|█████████▋| 217915/225000 [424:56:17<18:25:26,  9.36s/it]
 97%|█████████▋| 217916/225000 [424:56:23<16:25:35,  8.35s/it]
 97%|█████████▋| 217917/225000 [424:56:35<18:19:36,  9.31s/it]
 97%|█████████▋| 217918/225000 [424:56:40<15:39:07,  7.96s/it]
 97%|█████████▋| 217919/225000 [424:56:46<14:36:07,  7.42s/it]
 97%|█████████▋| 217920/225000 [424:56:52<13:50:25,  7.04s/it]
 97%|█████████▋| 217921/225000 [424:57:02<15:36:43,  7.94s/it]
 97%|█████████▋| 217922/225000 [424:57:16<19:27:17,  9.90s/it]
 97%|█████████▋| 217923/225000 [424:57:25<18:37:12,  9.47s/it]
 97%|█████████▋| 217924/225000 [424:57:34<18:17:59,  9.31s/it]
 97%|█████████▋| 217925/225000 [424:57:47<20:49:58, 10.60s/it]
 97%|█████████▋| 217926/225000 [424:57:59<21:37:26, 11.00s/it]
 97%|█████████▋| 217927/225000 [424:58:09<21:04:24, 10.73s/it]
 97%|█████████▋| 217928/225000 [424:58:13<17:09:20,  8.73s/it]
 97%|█████████▋| 217929/225000 [424:58:26<19:13:58,  9.79s/it]
 97%|█████████▋| 217930/225000 [424:58:34<18:04:10,  9.20s/it]
 97%|█████████▋| 217931/225000 [424:58:41<16:52:13,  8.59s/it]
 97%|█████████▋| 217932/225000 [424:58:47<15:36:28,  7.95s/it]
 97%|█████████▋| 217933/225000 [424:58:53<14:19:48,  7.30s/it]
 97%|█████████▋| 217934/225000 [424:59:04<16:20:18,  8.32s/it]
 97%|█████████▋| 217935/225000 [424:59:13<17:08:07,  8.73s/it]
 97%|█████████▋| 217936/225000 [424:59:25<19:02:04,  9.70s/it]
 97%|█████████▋| 217937/225000 [424:59:36<19:48:57, 10.10s/it]
 97%|█████████▋| 217938/225000 [424:59:43<17:38:28,  8.99s/it]
 97%|█████████▋| 217939/225000 [424:59:51<17:27:23,  8.90s/it]
 97%|█████████▋| 217940/225000 [425:00:01<17:57:49,  9.16s/it]
 97%|█████████▋| 217941/225000 [425:00:09<17:11:52,  8.77s/it]
 97%|█████████▋| 217942/225000 [425:00:17<16:44:46,  8.54s/it]
 97%|█████████▋| 217943/225000 [425:00:28<18:09:57,  9.27s/it]
 97%|█████████▋| 217944/225000 [425:00:33<15:37:29,  7.97s/it]
 97%|█████████▋| 217945/225000 [425:00:44<17:11:24,  8.77s/it]
 97%|█████████▋| 217946/225000 [425:00:52<16:52:20,  8.61s/it]
 97%|█████████▋| 217947/225000 [425:01:01<17:05:12,  8.72s/it]
 97%|█████████▋| 217948/225000 [425:01:12<18:36:18,  9.50s/it]
 97%|█████████▋| 217949/225000 [425:01:17<15:52:42,  8.11s/it]
 97%|█████████▋| 217950/225000 [425:01:23<14:48:57,  7.57s/it]
                                                              
{'loss': '1.301', 'grad_norm': '1.066', 'learning_rate': '1.364e-07', 'epoch': '0.9687', 'num_input_tokens_seen': 722925896, 'train_runtime': '1.53e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 217950/225000 [425:01:23<14:48:57,  7.57s/it]
 97%|█████████▋| 217951/225000 [425:01:29<13:47:18,  7.04s/it]
 97%|█████████▋| 217952/225000 [425:01:36<13:44:23,  7.02s/it]
 97%|█████████▋| 217953/225000 [425:01:43<13:24:28,  6.85s/it]
 97%|█████████▋| 217954/225000 [425:01:56<17:10:35,  8.78s/it]
 97%|█████████▋| 217955/225000 [425:02:02<15:49:17,  8.08s/it]
 97%|█████████▋| 217956/225000 [425:02:13<17:06:20,  8.74s/it]
 97%|█████████▋| 217957/225000 [425:02:21<16:45:41,  8.57s/it]
 97%|█████████▋| 217958/225000 [425:02:31<17:49:39,  9.11s/it]
 97%|█████████▋| 217959/225000 [425:02:40<17:52:29,  9.14s/it]
 97%|█████████▋| 217960/225000 [425:02:52<19:09:27,  9.80s/it]
 97%|█████████▋| 217961/225000 [425:02:59<17:35:54,  9.00s/it]
 97%|█████████▋| 217962/225000 [425:03:14<20:58:18, 10.73s/it]
 97%|█████████▋| 217963/225000 [425:03:23<20:17:38, 10.38s/it]
 97%|█████████▋| 217964/225000 [425:03:30<17:56:36,  9.18s/it]
 97%|█████████▋| 217965/225000 [425:03:37<17:10:01,  8.78s/it]
 97%|█████████▋| 217966/225000 [425:03:44<15:50:37,  8.11s/it]
 97%|█████████▋| 217967/225000 [425:03:53<16:42:46,  8.55s/it]
 97%|█████████▋| 217968/225000 [425:04:05<18:21:33,  9.40s/it]
 97%|█████████▋| 217969/225000 [425:04:11<16:33:21,  8.48s/it]
 97%|█████████▋| 217970/225000 [425:04:21<17:06:14,  8.76s/it]
 97%|█████████▋| 217971/225000 [425:04:33<19:24:26,  9.94s/it]
 97%|█████████▋| 217972/225000 [425:04:44<19:50:45, 10.17s/it]
 97%|█████████▋| 217973/225000 [425:04:50<17:10:51,  8.80s/it]
 97%|█████████▋| 217974/225000 [425:04:58<17:03:18,  8.74s/it]
 97%|█████████▋| 217975/225000 [425:05:04<15:25:36,  7.91s/it]
 97%|█████████▋| 217976/225000 [425:05:12<15:40:15,  8.03s/it]
 97%|█████████▋| 217977/225000 [425:05:20<15:13:14,  7.80s/it]
 97%|█████████▋| 217978/225000 [425:05:26<14:03:13,  7.21s/it]
 97%|█████████▋| 217979/225000 [425:05:34<14:29:27,  7.43s/it]
 97%|█████████▋| 217980/225000 [425:05:47<18:05:37,  9.28s/it]
 97%|█████████▋| 217981/225000 [425:05:54<16:48:04,  8.62s/it]
 97%|█████████▋| 217982/225000 [425:06:08<19:34:08, 10.04s/it]
 97%|█████████▋| 217983/225000 [425:06:13<16:49:56,  8.64s/it]
 97%|█████████▋| 217984/225000 [425:06:18<14:59:38,  7.69s/it]
 97%|█████████▋| 217985/225000 [425:06:29<16:58:51,  8.71s/it]
 97%|█████████▋| 217986/225000 [425:06:35<15:19:05,  7.86s/it]
 97%|█████████▋| 217987/225000 [425:06:42<14:20:13,  7.36s/it]
 97%|█████████▋| 217988/225000 [425:06:49<14:19:40,  7.36s/it]
 97%|█████████▋| 217989/225000 [425:06:56<14:17:42,  7.34s/it]
 97%|█████████▋| 217990/225000 [425:07:03<13:49:31,  7.10s/it]
 97%|█████████▋| 217991/225000 [425:07:13<15:50:23,  8.14s/it]
 97%|█████████▋| 217992/225000 [425:07:19<14:39:57,  7.53s/it]
 97%|█████████▋| 217993/225000 [425:07:25<13:27:11,  6.91s/it]
 97%|█████████▋| 217994/225000 [425:07:32<13:20:48,  6.86s/it]
 97%|█████████▋| 217995/225000 [425:07:41<14:48:45,  7.61s/it]
 97%|█████████▋| 217996/225000 [425:07:49<15:00:23,  7.71s/it]
 97%|█████████▋| 217997/225000 [425:08:02<18:03:02,  9.28s/it]
 97%|█████████▋| 217998/225000 [425:08:15<20:13:12, 10.40s/it]
 97%|█████████▋| 217999/225000 [425:08:26<20:30:13, 10.54s/it]
 97%|█████████▋| 218000/225000 [425:08:35<19:39:40, 10.11s/it]
                                                              
{'loss': '1.297', 'grad_norm': '1.321', 'learning_rate': '1.345e-07', 'epoch': '0.9689', 'num_input_tokens_seen': 723154960, 'train_runtime': '1.531e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 218000/225000 [425:08:35<19:39:40, 10.11s/it]
 97%|█████████▋| 218001/225000 [425:08:43<18:44:54,  9.64s/it]
 97%|█████████▋| 218002/225000 [425:08:51<17:38:08,  9.07s/it]
 97%|█████████▋| 218003/225000 [425:08:58<16:19:41,  8.40s/it]
 97%|█████████▋| 218004/225000 [425:09:13<19:59:02, 10.28s/it]
 97%|█████████▋| 218005/225000 [425:09:19<17:44:19,  9.13s/it]
 97%|█████████▋| 218006/225000 [425:09:28<17:21:46,  8.94s/it]
 97%|█████████▋| 218007/225000 [425:09:35<16:24:31,  8.45s/it]
 97%|█████████▋| 218008/225000 [425:09:41<14:54:33,  7.68s/it]
 97%|█████████▋| 218009/225000 [425:09:49<14:58:10,  7.71s/it]
 97%|█████████▋| 218010/225000 [425:09:57<15:14:53,  7.85s/it]
 97%|█████████▋| 218011/225000 [425:10:08<17:20:03,  8.93s/it]
 97%|█████████▋| 218012/225000 [425:10:17<17:25:14,  8.97s/it]
 97%|█████████▋| 218013/225000 [425:10:25<16:48:18,  8.66s/it]
 97%|█████████▋| 218014/225000 [425:10:33<16:13:31,  8.36s/it]
 97%|█████████▋| 218015/225000 [425:10:43<17:03:04,  8.79s/it]
 97%|█████████▋| 218016/225000 [425:10:50<16:14:41,  8.37s/it]
 97%|█████████▋| 218017/225000 [425:11:01<17:35:07,  9.07s/it]
 97%|█████████▋| 218018/225000 [425:11:15<20:53:16, 10.77s/it]
 97%|█████████▋| 218019/225000 [425:11:23<18:56:41,  9.77s/it]
 97%|█████████▋| 218020/225000 [425:11:32<18:23:22,  9.48s/it]
 97%|█████████▋| 218021/225000 [425:11:38<16:36:55,  8.57s/it]
 97%|█████████▋| 218022/225000 [425:11:47<16:47:02,  8.66s/it]
 97%|█████████▋| 218023/225000 [425:11:54<15:36:47,  8.06s/it]
 97%|█████████▋| 218024/225000 [425:12:01<15:01:11,  7.75s/it]
 97%|█████████▋| 218025/225000 [425:12:16<19:32:14, 10.08s/it]
 97%|█████████▋| 218026/225000 [425:12:20<16:06:20,  8.31s/it]
 97%|█████████▋| 218027/225000 [425:12:29<16:25:29,  8.48s/it]
 97%|█████████▋| 218028/225000 [425:12:43<19:16:48,  9.96s/it]
 97%|█████████▋| 218029/225000 [425:12:48<16:41:19,  8.62s/it]
 97%|█████████▋| 218030/225000 [425:12:56<16:19:13,  8.43s/it]
 97%|█████████▋| 218031/225000 [425:13:03<15:20:56,  7.93s/it]
 97%|█████████▋| 218032/225000 [425:13:16<18:19:10,  9.46s/it]
 97%|█████████▋| 218033/225000 [425:13:25<18:14:46,  9.43s/it]
 97%|█████████▋| 218034/225000 [425:13:30<15:30:52,  8.02s/it]
 97%|█████████▋| 218035/225000 [425:13:41<17:06:32,  8.84s/it]
 97%|█████████▋| 218036/225000 [425:13:53<19:07:44,  9.89s/it]
 97%|█████████▋| 218037/225000 [425:14:01<18:05:40,  9.36s/it]
 97%|█████████▋| 218038/225000 [425:14:08<16:40:11,  8.62s/it]
 97%|█████████▋| 218039/225000 [425:14:16<16:05:37,  8.32s/it]
 97%|█████████▋| 218040/225000 [425:14:22<15:03:45,  7.79s/it]
 97%|█████████▋| 218041/225000 [425:14:38<19:44:38, 10.21s/it]
 97%|█████████▋| 218042/225000 [425:14:45<17:49:37,  9.22s/it]
 97%|█████████▋| 218043/225000 [425:14:57<19:18:58, 10.00s/it]
 97%|█████████▋| 218044/225000 [425:15:02<16:16:33,  8.42s/it]
 97%|█████████▋| 218045/225000 [425:15:11<16:56:46,  8.77s/it]
 97%|█████████▋| 218046/225000 [425:15:20<16:40:50,  8.64s/it]
 97%|█████████▋| 218047/225000 [425:15:32<18:52:05,  9.77s/it]
 97%|█████████▋| 218048/225000 [425:15:39<17:04:38,  8.84s/it]
 97%|█████████▋| 218049/225000 [425:15:50<18:20:57,  9.50s/it]
 97%|█████████▋| 218050/225000 [425:15:54<15:30:50,  8.04s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.409', 'learning_rate': '1.326e-07', 'epoch': '0.9691', 'num_input_tokens_seen': 723388080, 'train_runtime': '1.531e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 218050/225000 [425:15:54<15:30:50,  8.04s/it]
 97%|█████████▋| 218051/225000 [425:16:05<16:46:01,  8.69s/it]
 97%|█████████▋| 218052/225000 [425:16:11<15:24:37,  7.98s/it]
 97%|█████████▋| 218053/225000 [425:16:21<16:45:00,  8.68s/it]
 97%|█████████▋| 218054/225000 [425:16:33<18:36:08,  9.64s/it]
 97%|█████████▋| 218055/225000 [425:16:40<17:04:56,  8.85s/it]
 97%|█████████▋| 218056/225000 [425:16:52<18:59:58,  9.85s/it]
 97%|█████████▋| 218057/225000 [425:17:00<17:55:55,  9.30s/it]
 97%|█████████▋| 218058/225000 [425:17:07<16:09:16,  8.38s/it]
 97%|█████████▋| 218059/225000 [425:17:13<15:12:44,  7.89s/it]
 97%|█████████▋| 218060/225000 [425:17:21<14:51:50,  7.71s/it]
 97%|█████████▋| 218061/225000 [425:17:33<17:34:16,  9.12s/it]
 97%|█████████▋| 218062/225000 [425:17:43<18:11:44,  9.44s/it]
 97%|█████████▋| 218063/225000 [425:17:50<16:28:02,  8.55s/it]
 97%|█████████▋| 218064/225000 [425:17:57<16:03:26,  8.33s/it]
 97%|█████████▋| 218065/225000 [425:18:13<20:17:59, 10.54s/it]
 97%|█████████▋| 218066/225000 [425:18:19<17:41:20,  9.18s/it]
 97%|█████████▋| 218067/225000 [425:18:29<17:50:42,  9.27s/it]
 97%|█████████▋| 218068/225000 [425:18:39<18:23:09,  9.55s/it]
 97%|█████████▋| 218069/225000 [425:18:48<17:55:10,  9.31s/it]
 97%|█████████▋| 218070/225000 [425:18:55<16:52:10,  8.76s/it]
 97%|█████████▋| 218071/225000 [425:19:01<15:19:57,  7.97s/it]
 97%|█████████▋| 218072/225000 [425:19:10<15:34:40,  8.09s/it]
 97%|█████████▋| 218073/225000 [425:19:20<16:48:21,  8.73s/it]
 97%|█████████▋| 218074/225000 [425:19:29<17:03:14,  8.86s/it]
 97%|█████████▋| 218075/225000 [425:19:35<15:32:19,  8.08s/it]
 97%|█████████▋| 218076/225000 [425:19:46<17:08:30,  8.91s/it]
 97%|█████████▋| 218077/225000 [425:19:52<15:32:50,  8.08s/it]
 97%|█████████▋| 218078/225000 [425:20:00<15:28:23,  8.05s/it]
 97%|█████████▋| 218079/225000 [425:20:17<20:18:09, 10.56s/it]
 97%|█████████▋| 218080/225000 [425:20:26<19:40:08, 10.23s/it]
 97%|█████████▋| 218081/225000 [425:20:40<21:51:29, 11.37s/it]
 97%|█████████▋| 218082/225000 [425:20:45<18:20:43,  9.55s/it]
 97%|█████████▋| 218083/225000 [425:20:56<18:56:57,  9.86s/it]
 97%|█████████▋| 218084/225000 [425:21:01<16:25:17,  8.55s/it]
 97%|█████████▋| 218085/225000 [425:21:13<18:14:55,  9.50s/it]
 97%|█████████▋| 218086/225000 [425:21:21<17:09:01,  8.93s/it]
 97%|█████████▋| 218087/225000 [425:21:31<18:03:14,  9.40s/it]
 97%|█████████▋| 218088/225000 [425:21:36<15:29:21,  8.07s/it]
 97%|█████████▋| 218089/225000 [425:21:50<18:38:59,  9.71s/it]
 97%|█████████▋| 218090/225000 [425:21:56<16:34:23,  8.63s/it]
 97%|█████████▋| 218091/225000 [425:22:08<18:46:46,  9.79s/it]
 97%|█████████▋| 218092/225000 [425:22:16<17:22:00,  9.05s/it]
 97%|█████████▋| 218093/225000 [425:22:24<17:12:00,  8.96s/it]
 97%|█████████▋| 218094/225000 [425:22:35<18:04:04,  9.42s/it]
 97%|█████████▋| 218095/225000 [425:22:44<17:36:38,  9.18s/it]
 97%|█████████▋| 218096/225000 [425:22:52<17:01:48,  8.88s/it]
 97%|█████████▋| 218097/225000 [425:22:58<15:44:48,  8.21s/it]
 97%|█████████▋| 218098/225000 [425:23:12<18:49:48,  9.82s/it]
 97%|█████████▋| 218099/225000 [425:23:21<18:07:00,  9.45s/it]
 97%|█████████▋| 218100/225000 [425:23:27<16:29:12,  8.60s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.778', 'learning_rate': '1.307e-07', 'epoch': '0.9693', 'num_input_tokens_seen': 723625864, 'train_runtime': '1.531e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 218100/225000 [425:23:27<16:29:12,  8.60s/it]
 97%|█████████▋| 218101/225000 [425:23:32<14:28:34,  7.55s/it]
 97%|█████████▋| 218102/225000 [425:23:42<15:36:15,  8.14s/it]
 97%|█████████▋| 218103/225000 [425:23:47<13:52:59,  7.25s/it]
 97%|█████████▋| 218104/225000 [425:23:53<13:14:30,  6.91s/it]
 97%|█████████▋| 218105/225000 [425:24:02<14:22:53,  7.51s/it]
 97%|█████████▋| 218106/225000 [425:24:13<16:12:20,  8.46s/it]
 97%|█████████▋| 218107/225000 [425:24:20<15:29:27,  8.09s/it]
 97%|█████████▋| 218108/225000 [425:24:31<17:23:02,  9.08s/it]
 97%|█████████▋| 218109/225000 [425:24:37<15:38:38,  8.17s/it]
 97%|█████████▋| 218110/225000 [425:24:45<15:04:39,  7.88s/it]
 97%|█████████▋| 218111/225000 [425:24:52<15:00:17,  7.84s/it]
 97%|█████████▋| 218112/225000 [425:25:06<18:17:49,  9.56s/it]
 97%|█████████▋| 218113/225000 [425:25:11<15:39:12,  8.18s/it]
 97%|█████████▋| 218114/225000 [425:25:24<18:36:49,  9.73s/it]
 97%|█████████▋| 218115/225000 [425:25:29<15:48:07,  8.26s/it]
 97%|█████████▋| 218116/225000 [425:25:36<15:14:20,  7.97s/it]
 97%|█████████▋| 218117/225000 [425:25:47<17:00:12,  8.89s/it]
 97%|█████████▋| 218118/225000 [425:25:56<16:36:31,  8.69s/it]
 97%|█████████▋| 218119/225000 [425:26:04<16:38:18,  8.70s/it]
 97%|█████████▋| 218120/225000 [425:26:14<17:01:33,  8.91s/it]
 97%|█████████▋| 218121/225000 [425:26:25<18:36:37,  9.74s/it]
 97%|█████████▋| 218122/225000 [425:26:35<18:35:42,  9.73s/it]
 97%|█████████▋| 218123/225000 [425:26:41<16:28:50,  8.63s/it]
 97%|█████████▋| 218124/225000 [425:26:53<18:07:18,  9.49s/it]
 97%|█████████▋| 218125/225000 [425:27:00<16:55:21,  8.86s/it]
 97%|█████████▋| 218126/225000 [425:27:14<19:46:04, 10.35s/it]
 97%|█████████▋| 218127/225000 [425:27:20<17:07:32,  8.97s/it]
 97%|█████████▋| 218128/225000 [425:27:31<18:42:51,  9.80s/it]
 97%|█████████▋| 218129/225000 [425:27:41<18:38:46,  9.77s/it]
 97%|█████████▋| 218130/225000 [425:27:49<17:28:12,  9.15s/it]
 97%|█████████▋| 218131/225000 [425:27:54<15:27:20,  8.10s/it]
 97%|█████████▋| 218132/225000 [425:28:03<15:38:45,  8.20s/it]
 97%|█████████▋| 218133/225000 [425:28:09<14:25:31,  7.56s/it]
 97%|█████████▋| 218134/225000 [425:28:22<17:17:32,  9.07s/it]
 97%|█████████▋| 218135/225000 [425:28:30<16:48:50,  8.82s/it]
 97%|█████████▋| 218136/225000 [425:28:36<15:32:06,  8.15s/it]
 97%|█████████▋| 218137/225000 [425:28:46<16:36:39,  8.71s/it]
 97%|█████████▋| 218138/225000 [425:28:55<16:43:46,  8.78s/it]
 97%|█████████▋| 218139/225000 [425:29:03<16:17:16,  8.55s/it]
 97%|█████████▋| 218140/225000 [425:29:12<16:26:19,  8.63s/it]
 97%|█████████▋| 218141/225000 [425:29:24<18:16:26,  9.59s/it]
 97%|█████████▋| 218142/225000 [425:29:31<16:40:57,  8.76s/it]
 97%|█████████▋| 218143/225000 [425:29:42<18:17:45,  9.61s/it]
 97%|█████████▋| 218144/225000 [425:29:49<16:48:06,  8.82s/it]
 97%|█████████▋| 218145/225000 [425:29:59<17:26:40,  9.16s/it]
 97%|█████████▋| 218146/225000 [425:30:06<16:12:13,  8.51s/it]
 97%|█████████▋| 218147/225000 [425:30:12<14:32:44,  7.64s/it]
 97%|█████████▋| 218148/225000 [425:30:21<15:39:09,  8.22s/it]
 97%|█████████▋| 218149/225000 [425:30:30<16:02:47,  8.43s/it]
 97%|█████████▋| 218150/225000 [425:30:39<16:20:41,  8.59s/it]
                                                              
{'loss': '1.326', 'grad_norm': '1.177', 'learning_rate': '1.289e-07', 'epoch': '0.9696', 'num_input_tokens_seen': 723854480, 'train_runtime': '1.532e+06', 'train_tokens_per_second': '472.5'}

 97%|█████████▋| 218150/225000 [425:30:39<16:20:41,  8.59s/it]
 97%|█████████▋| 218151/225000 [425:30:47<16:01:55,  8.43s/it]
 97%|█████████▋| 218152/225000 [425:30:58<17:19:00,  9.10s/it]
 97%|█████████▋| 218153/225000 [425:31:07<17:18:28,  9.10s/it]
 97%|█████████▋| 218154/225000 [425:31:14<15:45:14,  8.28s/it]
 97%|█████████▋| 218155/225000 [425:31:21<15:25:34,  8.11s/it]
 97%|█████████▋| 218156/225000 [425:31:30<15:31:39,  8.17s/it]
 97%|█████████▋| 218157/225000 [425:31:39<16:18:42,  8.58s/it]
 97%|█████████▋| 218158/225000 [425:31:52<18:28:20,  9.72s/it]
 97%|█████████▋| 218159/225000 [425:31:58<16:43:14,  8.80s/it]
 97%|█████████▋| 218160/225000 [425:32:03<14:32:38,  7.65s/it]
 97%|█████████▋| 218161/225000 [425:32:17<18:03:38,  9.51s/it]
 97%|█████████▋| 218162/225000 [425:32:21<15:04:10,  7.93s/it]
 97%|█████████▋| 218163/225000 [425:32:35<18:27:20,  9.72s/it]
 97%|█████████▋| 218164/225000 [425:32:43<17:38:50,  9.29s/it]
 97%|█████████▋| 218165/225000 [425:32:49<15:42:03,  8.27s/it]
 97%|█████████▋| 218166/225000 [425:32:56<14:44:58,  7.77s/it]
 97%|█████████▋| 218167/225000 [425:33:03<14:10:11,  7.47s/it]
 97%|█████████▋| 218168/225000 [425:33:09<13:34:14,  7.15s/it]
 97%|█████████▋| 218169/225000 [425:33:18<14:51:56,  7.83s/it]
 97%|█████████▋| 218170/225000 [425:33:24<13:36:14,  7.17s/it]
 97%|█████████▋| 218171/225000 [425:33:31<13:18:52,  7.02s/it]
 97%|█████████▋| 218172/225000 [425:33:37<12:53:10,  6.79s/it]
 97%|█████████▋| 218173/225000 [425:33:45<13:33:07,  7.15s/it]
 97%|█████████▋| 218174/225000 [425:33:57<16:10:13,  8.53s/it]
 97%|█████████▋| 218175/225000 [425:34:05<16:08:05,  8.51s/it]
 97%|█████████▋| 218176/225000 [425:34:18<18:19:29,  9.67s/it]
 97%|█████████▋| 218177/225000 [425:34:23<16:01:47,  8.46s/it]
 97%|█████████▋| 218178/225000 [425:34:34<17:15:15,  9.11s/it]
 97%|█████████▋| 218179/225000 [425:34:41<16:12:34,  8.56s/it]
 97%|█████████▋| 218180/225000 [425:34:49<16:02:13,  8.47s/it]
 97%|█████████▋| 218181/225000 [425:34:59<16:28:15,  8.70s/it]
 97%|█████████▋| 218182/225000 [425:35:07<16:15:47,  8.59s/it]
 97%|█████████▋| 218183/225000 [425:35:15<16:08:40,  8.53s/it]
 97%|█████████▋| 218184/225000 [425:35:26<17:34:54,  9.29s/it]
 97%|█████████▋| 218185/225000 [425:35:34<16:22:36,  8.65s/it]
 97%|█████████▋| 218186/225000 [425:35:41<15:28:39,  8.18s/it]
 97%|█████████▋| 218187/225000 [425:35:50<15:57:48,  8.44s/it]
 97%|█████████▋| 218188/225000 [425:36:03<18:30:23,  9.78s/it]
 97%|█████████▋| 218189/225000 [425:36:10<16:58:17,  8.97s/it]
 97%|█████████▋| 218190/225000 [425:36:16<15:26:47,  8.17s/it]
 97%|█████████▋| 218191/225000 [425:36:25<15:51:14,  8.38s/it]
 97%|█████████▋| 218192/225000 [425:36:32<14:53:28,  7.87s/it]
 97%|█████████▋| 218193/225000 [425:36:40<15:09:09,  8.01s/it]
 97%|█████████▋| 218194/225000 [425:36:51<17:02:24,  9.01s/it]
 97%|█████████▋| 218195/225000 [425:37:03<18:26:29,  9.76s/it]
 97%|█████████▋| 218196/225000 [425:37:09<16:19:05,  8.63s/it]
 97%|█████████▋| 218197/225000 [425:37:14<14:26:51,  7.65s/it]
 97%|█████████▋| 218198/225000 [425:37:24<15:36:30,  8.26s/it]
 97%|█████████▋| 218199/225000 [425:37:33<16:00:34,  8.47s/it]
 97%|█████████▋| 218200/225000 [425:37:43<17:01:40,  9.01s/it]
                                                              
{'loss': '1.331', 'grad_norm': '1.268', 'learning_rate': '1.27e-07', 'epoch': '0.9698', 'num_input_tokens_seen': 724081432, 'train_runtime': '1.532e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218200/225000 [425:37:43<17:01:40,  9.01s/it]
 97%|█████████▋| 218201/225000 [425:37:50<16:02:37,  8.50s/it]
 97%|█████████▋| 218202/225000 [425:37:57<14:46:29,  7.82s/it]
 97%|█████████▋| 218203/225000 [425:38:04<14:36:10,  7.73s/it]
 97%|█████████▋| 218204/225000 [425:38:20<19:03:34, 10.10s/it]
 97%|█████████▋| 218205/225000 [425:38:27<17:29:23,  9.27s/it]
 97%|█████████▋| 218206/225000 [425:38:33<15:38:19,  8.29s/it]
 97%|█████████▋| 218207/225000 [425:38:41<15:33:27,  8.24s/it]
 97%|█████████▋| 218208/225000 [425:38:49<15:21:44,  8.14s/it]
 97%|█████████▋| 218209/225000 [425:39:03<18:53:57, 10.02s/it]
 97%|█████████▋| 218210/225000 [425:39:10<16:57:31,  8.99s/it]
 97%|█████████▋| 218211/225000 [425:39:17<15:35:10,  8.26s/it]
 97%|█████████▋| 218212/225000 [425:39:29<18:07:15,  9.61s/it]
 97%|█████████▋| 218213/225000 [425:39:40<18:29:31,  9.81s/it]
 97%|█████████▋| 218214/225000 [425:39:48<17:38:28,  9.36s/it]
 97%|█████████▋| 218215/225000 [425:40:02<20:29:44, 10.87s/it]
 97%|█████████▋| 218216/225000 [425:40:10<18:54:37, 10.03s/it]
 97%|█████████▋| 218217/225000 [425:40:17<17:02:02,  9.04s/it]
 97%|█████████▋| 218218/225000 [425:40:25<16:05:25,  8.54s/it]
 97%|█████████▋| 218219/225000 [425:40:31<15:04:26,  8.00s/it]
 97%|█████████▋| 218220/225000 [425:40:45<18:11:45,  9.66s/it]
 97%|█████████▋| 218221/225000 [425:40:51<15:58:06,  8.48s/it]
 97%|█████████▋| 218222/225000 [425:40:54<13:23:54,  7.12s/it]
 97%|█████████▋| 218223/225000 [425:41:06<15:51:10,  8.42s/it]
 97%|█████████▋| 218224/225000 [425:41:13<15:10:17,  8.06s/it]
 97%|█████████▋| 218225/225000 [425:41:23<15:59:04,  8.49s/it]
 97%|█████████▋| 218226/225000 [425:41:33<16:53:57,  8.98s/it]
 97%|█████████▋| 218227/225000 [425:41:40<15:55:33,  8.47s/it]
 97%|█████████▋| 218228/225000 [425:41:46<14:43:09,  7.82s/it]
 97%|█████████▋| 218229/225000 [425:41:59<17:20:28,  9.22s/it]
 97%|█████████▋| 218230/225000 [425:42:08<17:31:09,  9.32s/it]
 97%|█████████▋| 218231/225000 [425:42:21<19:29:58, 10.37s/it]
 97%|█████████▋| 218232/225000 [425:42:28<17:12:03,  9.15s/it]
 97%|█████████▋| 218233/225000 [425:42:41<19:26:59, 10.35s/it]
 97%|█████████▋| 218234/225000 [425:42:52<20:13:53, 10.76s/it]
 97%|█████████▋| 218235/225000 [425:43:02<19:33:24, 10.41s/it]
 97%|█████████▋| 218236/225000 [425:43:08<16:57:37,  9.03s/it]
 97%|█████████▋| 218237/225000 [425:43:13<15:03:41,  8.02s/it]
 97%|█████████▋| 218238/225000 [425:43:18<13:04:07,  6.96s/it]
 97%|█████████▋| 218239/225000 [425:43:31<16:37:09,  8.85s/it]
 97%|█████████▋| 218240/225000 [425:43:40<16:23:04,  8.73s/it]
 97%|█████████▋| 218241/225000 [425:43:48<16:18:12,  8.68s/it]
 97%|█████████▋| 218242/225000 [425:43:54<14:29:47,  7.72s/it]
 97%|█████████▋| 218243/225000 [425:44:06<16:47:46,  8.95s/it]
 97%|█████████▋| 218244/225000 [425:44:13<15:49:25,  8.43s/it]
 97%|█████████▋| 218245/225000 [425:44:19<14:48:08,  7.89s/it]
 97%|█████████▋| 218246/225000 [425:44:26<14:04:40,  7.50s/it]
 97%|█████████▋| 218247/225000 [425:44:32<13:06:01,  6.98s/it]
 97%|█████████▋| 218248/225000 [425:44:43<15:17:22,  8.15s/it]
 97%|█████████▋| 218249/225000 [425:44:48<13:59:36,  7.46s/it]
 97%|█████████▋| 218250/225000 [425:44:59<15:57:39,  8.51s/it]
                                                              
{'loss': '1.299', 'grad_norm': '0.9202', 'learning_rate': '1.252e-07', 'epoch': '0.97', 'num_input_tokens_seen': 724314320, 'train_runtime': '1.533e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218250/225000 [425:44:59<15:57:39,  8.51s/it]
 97%|█████████▋| 218251/225000 [425:45:12<18:30:47,  9.88s/it]
 97%|█████████▋| 218252/225000 [425:45:20<17:14:01,  9.19s/it]
 97%|█████████▋| 218253/225000 [425:45:32<18:45:31, 10.01s/it]
 97%|█████████▋| 218254/225000 [425:45:38<16:42:38,  8.92s/it]
 97%|█████████▋| 218255/225000 [425:45:46<16:14:34,  8.67s/it]
 97%|█████████▋| 218256/225000 [425:45:54<15:22:56,  8.21s/it]
 97%|█████████▋| 218257/225000 [425:46:00<14:17:30,  7.63s/it]
 97%|█████████▋| 218258/225000 [425:46:15<18:28:06,  9.86s/it]
 97%|█████████▋| 218259/225000 [425:46:26<19:02:23, 10.17s/it]
 97%|█████████▋| 218260/225000 [425:46:33<17:25:30,  9.31s/it]
 97%|█████████▋| 218261/225000 [425:46:40<15:53:49,  8.49s/it]
 97%|█████████▋| 218262/225000 [425:46:47<14:57:13,  7.99s/it]
 97%|█████████▋| 218263/225000 [425:46:56<15:31:24,  8.30s/it]
 97%|█████████▋| 218264/225000 [425:47:09<18:32:20,  9.91s/it]
 97%|█████████▋| 218265/225000 [425:47:18<18:02:20,  9.64s/it]
 97%|█████████▋| 218266/225000 [425:47:29<18:51:24, 10.08s/it]
 97%|█████████▋| 218267/225000 [425:47:36<17:09:17,  9.17s/it]
 97%|█████████▋| 218268/225000 [425:47:46<17:32:39,  9.38s/it]
 97%|█████████▋| 218269/225000 [425:47:53<16:02:32,  8.58s/it]
 97%|█████████▋| 218270/225000 [425:48:05<17:42:01,  9.47s/it]
 97%|█████████▋| 218271/225000 [425:48:13<17:14:36,  9.23s/it]
 97%|█████████▋| 218272/225000 [425:48:21<16:39:58,  8.92s/it]
 97%|█████████▋| 218273/225000 [425:48:33<17:57:12,  9.61s/it]
 97%|█████████▋| 218274/225000 [425:48:39<16:15:59,  8.71s/it]
 97%|█████████▋| 218275/225000 [425:48:46<15:22:44,  8.23s/it]
 97%|█████████▋| 218276/225000 [425:48:53<14:34:31,  7.80s/it]
 97%|█████████▋| 218277/225000 [425:49:06<17:40:09,  9.46s/it]
 97%|█████████▋| 218278/225000 [425:49:13<15:47:31,  8.46s/it]
 97%|█████████▋| 218279/225000 [425:49:24<17:28:58,  9.36s/it]
 97%|█████████▋| 218280/225000 [425:49:30<15:22:11,  8.23s/it]
 97%|█████████▋| 218281/225000 [425:49:34<13:02:29,  6.99s/it]
 97%|█████████▋| 218282/225000 [425:49:45<15:38:36,  8.38s/it]
 97%|█████████▋| 218283/225000 [425:49:54<15:50:47,  8.49s/it]
 97%|█████████▋| 218284/225000 [425:50:05<17:15:56,  9.25s/it]
 97%|█████████▋| 218285/225000 [425:50:15<17:32:01,  9.40s/it]
 97%|█████████▋| 218286/225000 [425:50:24<17:17:30,  9.27s/it]
 97%|█████████▋| 218287/225000 [425:50:33<17:02:12,  9.14s/it]
 97%|█████████▋| 218288/225000 [425:50:42<16:56:40,  9.09s/it]
 97%|█████████▋| 218289/225000 [425:50:49<15:46:27,  8.46s/it]
 97%|█████████▋| 218290/225000 [425:51:01<17:51:28,  9.58s/it]
 97%|█████████▋| 218291/225000 [425:51:12<18:51:27, 10.12s/it]
 97%|█████████▋| 218292/225000 [425:51:19<17:07:35,  9.19s/it]
 97%|█████████▋| 218293/225000 [425:51:29<17:40:22,  9.49s/it]
 97%|█████████▋| 218294/225000 [425:51:38<17:25:38,  9.36s/it]
 97%|█████████▋| 218295/225000 [425:51:45<15:40:41,  8.42s/it]
 97%|█████████▋| 218296/225000 [425:51:55<16:28:29,  8.85s/it]
 97%|█████████▋| 218297/225000 [425:52:00<14:22:54,  7.72s/it]
 97%|█████████▋| 218298/225000 [425:52:09<15:26:49,  8.30s/it]
 97%|█████████▋| 218299/225000 [425:52:16<14:22:32,  7.72s/it]
 97%|█████████▋| 218300/225000 [425:52:22<13:50:09,  7.43s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.18', 'learning_rate': '1.234e-07', 'epoch': '0.9702', 'num_input_tokens_seen': 724547736, 'train_runtime': '1.533e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218300/225000 [425:52:22<13:50:09,  7.43s/it]
 97%|█████████▋| 218301/225000 [425:52:37<17:43:37,  9.53s/it]
 97%|█████████▋| 218302/225000 [425:52:45<17:05:08,  9.18s/it]
 97%|█████████▋| 218303/225000 [425:52:51<15:18:55,  8.23s/it]
 97%|█████████▋| 218304/225000 [425:53:00<15:22:37,  8.27s/it]
 97%|█████████▋| 218305/225000 [425:53:10<16:41:56,  8.98s/it]
 97%|█████████▋| 218306/225000 [425:53:17<15:17:33,  8.22s/it]
 97%|█████████▋| 218307/225000 [425:53:29<17:31:02,  9.42s/it]
 97%|█████████▋| 218308/225000 [425:53:36<16:04:58,  8.65s/it]
 97%|█████████▋| 218309/225000 [425:53:43<15:16:07,  8.22s/it]
 97%|█████████▋| 218310/225000 [425:53:49<14:12:43,  7.65s/it]
 97%|█████████▋| 218311/225000 [425:54:00<15:57:24,  8.59s/it]
 97%|█████████▋| 218312/225000 [425:54:09<15:58:53,  8.60s/it]
 97%|█████████▋| 218313/225000 [425:54:19<16:55:45,  9.11s/it]
 97%|█████████▋| 218314/225000 [425:54:26<15:28:36,  8.33s/it]
 97%|█████████▋| 218315/225000 [425:54:38<18:00:05,  9.69s/it]
 97%|█████████▋| 218316/225000 [425:54:49<18:14:22,  9.82s/it]
 97%|█████████▋| 218317/225000 [425:54:53<15:19:21,  8.25s/it]
 97%|█████████▋| 218318/225000 [425:55:06<17:47:57,  9.59s/it]
 97%|█████████▋| 218319/225000 [425:55:15<17:29:52,  9.43s/it]
 97%|█████████▋| 218320/225000 [425:55:20<14:51:50,  8.01s/it]
 97%|█████████▋| 218321/225000 [425:55:28<15:00:55,  8.09s/it]
 97%|█████████▋| 218322/225000 [425:55:38<16:05:38,  8.68s/it]
 97%|█████████▋| 218323/225000 [425:55:46<15:30:11,  8.36s/it]
 97%|█████████▋| 218324/225000 [425:55:53<14:47:39,  7.98s/it]
 97%|█████████▋| 218325/225000 [425:56:01<15:10:18,  8.18s/it]
 97%|█████████▋| 218326/225000 [425:56:14<17:32:25,  9.46s/it]
 97%|█████████▋| 218327/225000 [425:56:28<20:06:22, 10.85s/it]
 97%|█████████▋| 218328/225000 [425:56:32<16:31:21,  8.92s/it]
 97%|█████████▋| 218329/225000 [425:56:43<17:45:27,  9.58s/it]
 97%|█████████▋| 218330/225000 [425:56:56<19:16:44, 10.41s/it]
 97%|█████████▋| 218331/225000 [425:57:02<16:45:57,  9.05s/it]
 97%|█████████▋| 218332/225000 [425:57:11<17:14:42,  9.31s/it]
 97%|█████████▋| 218333/225000 [425:57:17<15:05:47,  8.15s/it]
 97%|█████████▋| 218334/225000 [425:57:24<14:35:56,  7.88s/it]
 97%|█████████▋| 218335/225000 [425:57:36<16:42:52,  9.03s/it]
 97%|█████████▋| 218336/225000 [425:57:45<16:39:14,  9.00s/it]
 97%|█████████▋| 218337/225000 [425:57:52<15:24:49,  8.33s/it]
 97%|█████████▋| 218338/225000 [425:58:01<16:16:20,  8.79s/it]
 97%|█████████▋| 218339/225000 [425:58:09<15:45:50,  8.52s/it]
 97%|█████████▋| 218340/225000 [425:58:18<16:05:56,  8.70s/it]
 97%|█████████▋| 218341/225000 [425:58:26<15:40:47,  8.48s/it]
 97%|█████████▋| 218342/225000 [425:58:34<15:15:03,  8.25s/it]
 97%|█████████▋| 218343/225000 [425:58:45<16:26:48,  8.89s/it]
 97%|█████████▋| 218344/225000 [425:58:52<15:47:31,  8.54s/it]
 97%|█████████▋| 218345/225000 [425:58:59<14:51:17,  8.04s/it]
 97%|█████████▋| 218346/225000 [425:59:10<16:34:20,  8.97s/it]
 97%|█████████▋| 218347/225000 [425:59:16<14:47:46,  8.01s/it]
 97%|█████████▋| 218348/225000 [425:59:26<15:59:50,  8.66s/it]
 97%|█████████▋| 218349/225000 [425:59:40<19:07:02, 10.35s/it]
 97%|█████████▋| 218350/225000 [425:59:47<17:12:21,  9.31s/it]
                                                              
{'loss': '1.334', 'grad_norm': '1.233', 'learning_rate': '1.216e-07', 'epoch': '0.9704', 'num_input_tokens_seen': 724777384, 'train_runtime': '1.534e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218350/225000 [425:59:47<17:12:21,  9.31s/it]
 97%|█████████▋| 218351/225000 [425:59:56<17:00:03,  9.20s/it]
 97%|█████████▋| 218352/225000 [426:00:05<16:55:30,  9.17s/it]
 97%|█████████▋| 218353/225000 [426:00:14<16:28:18,  8.92s/it]
 97%|█████████▋| 218354/225000 [426:00:20<14:53:05,  8.06s/it]
 97%|█████████▋| 218355/225000 [426:00:32<16:54:47,  9.16s/it]
 97%|█████████▋| 218356/225000 [426:00:41<16:56:54,  9.18s/it]
 97%|█████████▋| 218357/225000 [426:00:50<16:45:39,  9.08s/it]
 97%|█████████▋| 218358/225000 [426:01:01<18:04:12,  9.79s/it]
 97%|█████████▋| 218359/225000 [426:01:11<17:58:47,  9.75s/it]
 97%|█████████▋| 218360/225000 [426:01:20<17:43:46,  9.61s/it]
 97%|█████████▋| 218361/225000 [426:01:30<17:43:36,  9.61s/it]
 97%|█████████▋| 218362/225000 [426:01:38<17:16:28,  9.37s/it]
 97%|█████████▋| 218363/225000 [426:01:45<15:30:20,  8.41s/it]
 97%|█████████▋| 218364/225000 [426:01:55<16:38:34,  9.03s/it]
 97%|█████████▋| 218365/225000 [426:02:01<14:45:47,  8.01s/it]
 97%|█████████▋| 218366/225000 [426:02:08<14:16:50,  7.75s/it]
 97%|█████████▋| 218367/225000 [426:02:14<13:41:01,  7.43s/it]
 97%|█████████▋| 218368/225000 [426:02:26<15:55:07,  8.64s/it]
 97%|█████████▋| 218369/225000 [426:02:35<16:12:30,  8.80s/it]
 97%|█████████▋| 218370/225000 [426:02:40<14:08:17,  7.68s/it]
 97%|█████████▋| 218371/225000 [426:02:49<14:32:23,  7.90s/it]
 97%|█████████▋| 218372/225000 [426:02:56<14:28:17,  7.86s/it]
 97%|█████████▋| 218373/225000 [426:03:13<19:03:40, 10.35s/it]
 97%|█████████▋| 218374/225000 [426:03:19<16:38:21,  9.04s/it]
 97%|█████████▋| 218375/225000 [426:03:26<15:33:00,  8.45s/it]
 97%|█████████▋| 218376/225000 [426:03:35<16:15:26,  8.84s/it]
 97%|█████████▋| 218377/225000 [426:03:44<16:24:25,  8.92s/it]
 97%|█████████▋| 218378/225000 [426:03:50<14:36:04,  7.94s/it]
 97%|█████████▋| 218379/225000 [426:03:56<13:40:59,  7.44s/it]
 97%|█████████▋| 218380/225000 [426:04:12<18:17:17,  9.95s/it]
 97%|█████████▋| 218381/225000 [426:04:18<15:58:56,  8.69s/it]
 97%|█████████▋| 218382/225000 [426:04:24<14:39:33,  7.97s/it]
 97%|█████████▋| 218383/225000 [426:04:36<16:45:38,  9.12s/it]
 97%|█████████▋| 218384/225000 [426:04:44<15:54:20,  8.65s/it]
 97%|█████████▋| 218385/225000 [426:04:51<14:57:46,  8.14s/it]
 97%|█████████▋| 218386/225000 [426:04:58<14:47:23,  8.05s/it]
 97%|█████████▋| 218387/225000 [426:05:07<15:01:26,  8.18s/it]
 97%|█████████▋| 218388/225000 [426:05:19<17:05:12,  9.30s/it]
 97%|█████████▋| 218389/225000 [426:05:28<16:58:51,  9.25s/it]
 97%|█████████▋| 218390/225000 [426:05:36<16:26:28,  8.95s/it]
 97%|█████████▋| 218391/225000 [426:05:41<14:06:11,  7.68s/it]
 97%|█████████▋| 218392/225000 [426:05:48<13:43:23,  7.48s/it]
 97%|█████████▋| 218393/225000 [426:05:58<15:05:16,  8.22s/it]
 97%|█████████▋| 218394/225000 [426:06:08<16:23:10,  8.93s/it]
 97%|█████████▋| 218395/225000 [426:06:22<18:42:47, 10.20s/it]
 97%|█████████▋| 218396/225000 [426:06:26<15:31:54,  8.47s/it]
 97%|█████████▋| 218397/225000 [426:06:33<14:53:48,  8.12s/it]
 97%|█████████▋| 218398/225000 [426:06:41<14:33:19,  7.94s/it]
 97%|█████████▋| 218399/225000 [426:06:47<13:36:45,  7.42s/it]
 97%|█████████▋| 218400/225000 [426:06:59<16:12:58,  8.85s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.343', 'learning_rate': '1.198e-07', 'epoch': '0.9707', 'num_input_tokens_seen': 725005928, 'train_runtime': '1.534e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218400/225000 [426:06:59<16:12:58,  8.85s/it]
 97%|█████████▋| 218401/225000 [426:07:08<16:17:25,  8.89s/it]
 97%|█████████▋| 218402/225000 [426:07:15<14:58:25,  8.17s/it]
 97%|█████████▋| 218403/225000 [426:07:22<14:24:20,  7.86s/it]
 97%|█████████▋| 218404/225000 [426:07:30<14:41:41,  8.02s/it]
 97%|█████████▋| 218405/225000 [426:07:46<18:44:14, 10.23s/it]
 97%|█████████▋| 218406/225000 [426:07:56<18:34:41, 10.14s/it]
 97%|█████████▋| 218407/225000 [426:08:05<18:21:36, 10.03s/it]
 97%|█████████▋| 218408/225000 [426:08:15<18:07:02,  9.89s/it]
 97%|█████████▋| 218409/225000 [426:08:22<16:26:04,  8.98s/it]
 97%|█████████▋| 218410/225000 [426:08:34<18:00:51,  9.84s/it]
 97%|█████████▋| 218411/225000 [426:08:40<16:13:25,  8.86s/it]
 97%|█████████▋| 218412/225000 [426:08:46<14:23:37,  7.87s/it]
 97%|█████████▋| 218413/225000 [426:08:58<16:53:33,  9.23s/it]
 97%|█████████▋| 218414/225000 [426:09:03<14:45:09,  8.06s/it]
 97%|█████████▋| 218415/225000 [426:09:10<13:41:22,  7.48s/it]
 97%|█████████▋| 218416/225000 [426:09:16<12:54:23,  7.06s/it]
 97%|█████████▋| 218417/225000 [426:09:28<16:02:21,  8.77s/it]
 97%|█████████▋| 218418/225000 [426:09:35<15:04:01,  8.24s/it]
 97%|█████████▋| 218419/225000 [426:09:47<16:50:36,  9.21s/it]
 97%|█████████▋| 218420/225000 [426:09:59<18:13:02,  9.97s/it]
 97%|█████████▋| 218421/225000 [426:10:05<16:03:32,  8.79s/it]
 97%|█████████▋| 218422/225000 [426:10:10<14:18:05,  7.83s/it]
 97%|█████████▋| 218423/225000 [426:10:19<14:33:03,  7.96s/it]
 97%|█████████▋| 218424/225000 [426:10:26<14:10:31,  7.76s/it]
 97%|█████████▋| 218425/225000 [426:10:40<17:55:54,  9.82s/it]
 97%|█████████▋| 218426/225000 [426:10:47<15:55:38,  8.72s/it]
 97%|█████████▋| 218427/225000 [426:10:52<14:17:50,  7.83s/it]
 97%|█████████▋| 218428/225000 [426:11:01<14:36:48,  8.01s/it]
 97%|█████████▋| 218429/225000 [426:11:07<13:34:37,  7.44s/it]
 97%|█████████▋| 218430/225000 [426:11:22<17:59:51,  9.86s/it]
 97%|█████████▋| 218431/225000 [426:11:29<16:01:56,  8.79s/it]
 97%|█████████▋| 218432/225000 [426:11:38<16:27:38,  9.02s/it]
 97%|█████████▋| 218433/225000 [426:11:50<17:58:42,  9.86s/it]
 97%|█████████▋| 218434/225000 [426:11:55<15:21:18,  8.42s/it]
 97%|█████████▋| 218435/225000 [426:12:05<16:07:38,  8.84s/it]
 97%|█████████▋| 218436/225000 [426:12:14<16:27:36,  9.03s/it]
 97%|█████████▋| 218437/225000 [426:12:27<18:14:17, 10.00s/it]
 97%|█████████▋| 218438/225000 [426:12:34<16:55:35,  9.29s/it]
 97%|█████████▋| 218439/225000 [426:12:41<15:45:23,  8.65s/it]
 97%|█████████▋| 218440/225000 [426:12:48<14:25:45,  7.92s/it]
 97%|█████████▋| 218441/225000 [426:12:58<15:55:34,  8.74s/it]
 97%|█████████▋| 218442/225000 [426:13:07<16:07:05,  8.85s/it]
 97%|█████████▋| 218443/225000 [426:13:17<16:22:21,  8.99s/it]
 97%|█████████▋| 218444/225000 [426:13:27<17:00:41,  9.34s/it]
 97%|█████████▋| 218445/225000 [426:13:33<14:58:06,  8.22s/it]
 97%|█████████▋| 218446/225000 [426:13:44<16:43:38,  9.19s/it]
 97%|█████████▋| 218447/225000 [426:13:53<16:42:45,  9.18s/it]
 97%|█████████▋| 218448/225000 [426:14:05<17:53:49,  9.83s/it]
 97%|█████████▋| 218449/225000 [426:14:14<17:45:13,  9.76s/it]
 97%|█████████▋| 218450/225000 [426:14:22<16:58:29,  9.33s/it]
                                                              
{'loss': '1.329', 'grad_norm': '1.113', 'learning_rate': '1.18e-07', 'epoch': '0.9709', 'num_input_tokens_seen': 725235896, 'train_runtime': '1.534e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218450/225000 [426:14:22<16:58:29,  9.33s/it]
 97%|█████████▋| 218451/225000 [426:14:35<18:36:46, 10.23s/it]
 97%|█████████▋| 218452/225000 [426:14:39<15:15:10,  8.39s/it]
 97%|█████████▋| 218453/225000 [426:14:44<13:34:39,  7.47s/it]
 97%|█████████▋| 218454/225000 [426:14:49<12:21:34,  6.80s/it]
 97%|█████████▋| 218455/225000 [426:14:56<12:27:07,  6.85s/it]
 97%|█████████▋| 218456/225000 [426:15:10<16:19:32,  8.98s/it]
 97%|█████████▋| 218457/225000 [426:15:20<16:29:11,  9.07s/it]
 97%|█████████▋| 218458/225000 [426:15:30<17:28:23,  9.62s/it]
 97%|█████████▋| 218459/225000 [426:15:41<18:05:42,  9.96s/it]
 97%|█████████▋| 218460/225000 [426:15:48<16:05:49,  8.86s/it]
 97%|█████████▋| 218461/225000 [426:15:55<15:18:28,  8.43s/it]
 97%|█████████▋| 218462/225000 [426:16:02<14:17:03,  7.87s/it]
 97%|█████████▋| 218463/225000 [426:16:10<14:24:28,  7.93s/it]
 97%|█████████▋| 218464/225000 [426:16:24<17:54:27,  9.86s/it]
 97%|█████████▋| 218465/225000 [426:16:33<17:12:39,  9.48s/it]
 97%|█████████▋| 218466/225000 [426:16:41<16:31:42,  9.11s/it]
 97%|█████████▋| 218467/225000 [426:16:46<14:27:00,  7.96s/it]
 97%|█████████▋| 218468/225000 [426:16:57<16:05:28,  8.87s/it]
 97%|█████████▋| 218469/225000 [426:17:09<17:36:23,  9.71s/it]
 97%|█████████▋| 218470/225000 [426:17:21<18:56:46, 10.45s/it]
 97%|█████████▋| 218471/225000 [426:17:30<18:11:02, 10.03s/it]
 97%|█████████▋| 218472/225000 [426:17:36<15:55:31,  8.78s/it]
 97%|█████████▋| 218473/225000 [426:17:41<14:09:46,  7.81s/it]
 97%|█████████▋| 218474/225000 [426:17:53<16:09:48,  8.92s/it]
 97%|█████████▋| 218475/225000 [426:18:03<16:41:23,  9.21s/it]
 97%|█████████▋| 218476/225000 [426:18:12<16:26:14,  9.07s/it]
 97%|█████████▋| 218477/225000 [426:18:25<19:01:06, 10.50s/it]
 97%|█████████▋| 218478/225000 [426:18:34<17:45:16,  9.80s/it]
 97%|█████████▋| 218479/225000 [426:18:44<18:20:38, 10.13s/it]
 97%|█████████▋| 218480/225000 [426:18:52<16:44:12,  9.24s/it]
 97%|█████████▋| 218481/225000 [426:19:05<18:55:14, 10.45s/it]
 97%|█████████▋| 218482/225000 [426:19:10<16:10:06,  8.93s/it]
 97%|█████████▋| 218483/225000 [426:19:18<15:31:27,  8.58s/it]
 97%|█████████▋| 218484/225000 [426:19:29<16:52:13,  9.32s/it]
 97%|█████████▋| 218485/225000 [426:19:36<15:49:09,  8.74s/it]
 97%|█████████▋| 218486/225000 [426:19:45<15:28:07,  8.55s/it]
 97%|█████████▋| 218487/225000 [426:19:50<13:57:43,  7.72s/it]
 97%|█████████▋| 218488/225000 [426:20:05<17:30:41,  9.68s/it]
 97%|█████████▋| 218489/225000 [426:20:13<17:04:12,  9.44s/it]
 97%|█████████▋| 218490/225000 [426:20:26<18:35:35, 10.28s/it]
 97%|█████████▋| 218491/225000 [426:20:33<17:08:12,  9.48s/it]
 97%|█████████▋| 218492/225000 [426:20:45<18:11:29, 10.06s/it]
 97%|█████████▋| 218493/225000 [426:20:52<16:36:32,  9.19s/it]
 97%|█████████▋| 218494/225000 [426:21:03<17:54:16,  9.91s/it]
 97%|█████████▋| 218495/225000 [426:21:11<16:32:25,  9.15s/it]
 97%|█████████▋| 218496/225000 [426:21:20<16:22:57,  9.07s/it]
 97%|█████████▋| 218497/225000 [426:21:31<17:25:49,  9.65s/it]
 97%|█████████▋| 218498/225000 [426:21:42<18:11:51, 10.08s/it]
 97%|█████████▋| 218499/225000 [426:21:48<16:01:14,  8.87s/it]
 97%|█████████▋| 218500/225000 [426:21:58<16:43:06,  9.26s/it]
                                                              
{'loss': '1.316', 'grad_norm': '1.665', 'learning_rate': '1.163e-07', 'epoch': '0.9711', 'num_input_tokens_seen': 725471664, 'train_runtime': '1.535e+06', 'train_tokens_per_second': '472.6'}

 97%|█████████▋| 218500/225000 [426:21:58<16:43:06,  9.26s/it]
 97%|█████████▋| 218501/225000 [426:22:04<15:08:12,  8.38s/it]
 97%|█████████▋| 218502/225000 [426:22:12<14:46:58,  8.19s/it]
 97%|█████████▋| 218503/225000 [426:22:21<14:59:36,  8.31s/it]
 97%|█████████▋| 218504/225000 [426:22:34<17:49:13,  9.88s/it]
 97%|█████████▋| 218505/225000 [426:22:45<18:09:43, 10.07s/it]
 97%|█████████▋| 218506/225000 [426:22:58<20:00:56, 11.10s/it]
 97%|█████████▋| 218507/225000 [426:23:07<18:50:06, 10.44s/it]
 97%|█████████▋| 218508/225000 [426:23:14<16:53:50,  9.37s/it]
 97%|█████████▋| 218509/225000 [426:23:25<17:45:31,  9.85s/it]
 97%|█████████▋| 218510/225000 [426:23:34<17:31:41,  9.72s/it]
 97%|█████████▋| 218511/225000 [426:23:42<16:34:22,  9.19s/it]
 97%|█████████▋| 218512/225000 [426:23:50<15:39:03,  8.68s/it]
 97%|█████████▋| 218513/225000 [426:24:00<16:33:17,  9.19s/it]
 97%|█████████▋| 218514/225000 [426:24:12<18:00:41, 10.00s/it]
 97%|█████████▋| 218515/225000 [426:24:22<17:56:29,  9.96s/it]
 97%|█████████▋| 218516/225000 [426:24:36<20:13:13, 11.23s/it]
 97%|█████████▋| 218517/225000 [426:24:46<19:24:39, 10.78s/it]
 97%|█████████▋| 218518/225000 [426:24:53<17:26:01,  9.68s/it]
 97%|█████████▋| 218519/225000 [426:25:02<16:48:07,  9.33s/it]
 97%|█████████▋| 218520/225000 [426:25:15<19:16:22, 10.71s/it]
 97%|█████████▋| 218521/225000 [426:25:21<16:30:14,  9.17s/it]
 97%|█████████▋| 218522/225000 [426:25:34<18:25:25, 10.24s/it]
 97%|█████████▋| 218523/225000 [426:25:41<17:01:52,  9.47s/it]
 97%|█████████▋| 218524/225000 [426:25:50<16:42:48,  9.29s/it]
 97%|█████████▋| 218525/225000 [426:25:56<14:32:56,  8.09s/it]
 97%|█████████▋| 218526/225000 [426:26:06<15:44:28,  8.75s/it]
 97%|█████████▋| 218527/225000 [426:26:15<16:06:10,  8.96s/it]
 97%|█████████▋| 218528/225000 [426:26:25<16:38:27,  9.26s/it]
 97%|█████████▋| 218529/225000 [426:26:33<15:45:35,  8.77s/it]
 97%|█████████▋| 218530/225000 [426:26:49<19:56:43, 11.10s/it]
 97%|█████████▋| 218531/225000 [426:26:59<18:59:46, 10.57s/it]
 97%|█████████▋| 218532/225000 [426:27:04<15:58:15,  8.89s/it]
 97%|█████████▋| 218533/225000 [426:27:12<15:24:12,  8.57s/it]
 97%|█████████▋| 218534/225000 [426:27:18<14:24:10,  8.02s/it]
 97%|█████████▋| 218535/225000 [426:27:26<14:08:58,  7.88s/it]
 97%|█████████▋| 218536/225000 [426:27:32<13:25:33,  7.48s/it]
 97%|█████████▋| 218537/225000 [426:27:47<17:30:07,  9.75s/it]
 97%|█████████▋| 218538/225000 [426:27:53<15:02:18,  8.38s/it]
 97%|█████████▋| 218539/225000 [426:28:01<14:59:41,  8.35s/it]
 97%|█████████▋| 218540/225000 [426:28:09<14:36:14,  8.14s/it]
 97%|█████████▋| 218541/225000 [426:28:17<14:41:11,  8.19s/it]
 97%|█████████▋| 218542/225000 [426:28:24<14:01:45,  7.82s/it]
 97%|█████████▋| 218543/225000 [426:28:29<12:50:25,  7.16s/it]
 97%|█████████▋| 218544/225000 [426:28:46<17:51:13,  9.96s/it]
 97%|█████████▋| 218545/225000 [426:28:59<19:20:00, 10.78s/it]
 97%|█████████▋| 218546/225000 [426:29:05<16:55:00,  9.44s/it]
 97%|█████████▋| 218547/225000 [426:29:13<16:21:09,  9.12s/it]
 97%|█████████▋| 218548/225000 [426:29:24<17:06:31,  9.55s/it]
 97%|█████████▋| 218549/225000 [426:29:34<17:34:07,  9.80s/it]
 97%|█████████▋| 218550/225000 [426:29:43<16:56:54,  9.46s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.243', 'learning_rate': '1.145e-07', 'epoch': '0.9713', 'num_input_tokens_seen': 725711416, 'train_runtime': '1.535e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218550/225000 [426:29:43<16:56:54,  9.46s/it]
 97%|█████████▋| 218551/225000 [426:29:52<16:51:45,  9.41s/it]
 97%|█████████▋| 218552/225000 [426:29:59<15:17:54,  8.54s/it]
 97%|█████████▋| 218553/225000 [426:30:08<15:56:06,  8.90s/it]
 97%|█████████▋| 218554/225000 [426:30:13<13:43:02,  7.66s/it]
 97%|█████████▋| 218555/225000 [426:30:23<14:55:30,  8.34s/it]
 97%|█████████▋| 218556/225000 [426:30:32<14:59:25,  8.37s/it]
 97%|█████████▋| 218557/225000 [426:30:38<13:43:32,  7.67s/it]
 97%|█████████▋| 218558/225000 [426:30:45<13:41:25,  7.65s/it]
 97%|█████████▋| 218559/225000 [426:30:56<15:13:48,  8.51s/it]
 97%|█████████▋| 218560/225000 [426:31:05<15:21:51,  8.59s/it]
 97%|█████████▋| 218561/225000 [426:31:13<15:20:03,  8.57s/it]
 97%|█████████▋| 218562/225000 [426:31:21<15:01:51,  8.41s/it]
 97%|█████████▋| 218563/225000 [426:31:28<14:25:41,  8.07s/it]
 97%|█████████▋| 218564/225000 [426:31:38<15:09:04,  8.47s/it]
 97%|█████████▋| 218565/225000 [426:31:42<12:50:25,  7.18s/it]
 97%|█████████▋| 218566/225000 [426:31:55<16:03:08,  8.98s/it]
 97%|█████████▋| 218567/225000 [426:32:02<14:55:43,  8.35s/it]
 97%|█████████▋| 218568/225000 [426:32:08<13:54:35,  7.79s/it]
 97%|█████████▋| 218569/225000 [426:32:15<13:25:24,  7.51s/it]
 97%|█████████▋| 218570/225000 [426:32:23<13:24:11,  7.50s/it]
 97%|█████████▋| 218571/225000 [426:32:37<17:07:32,  9.59s/it]
 97%|█████████▋| 218572/225000 [426:32:45<15:52:35,  8.89s/it]
 97%|█████████▋| 218573/225000 [426:32:51<14:29:24,  8.12s/it]
 97%|█████████▋| 218574/225000 [426:32:56<12:48:46,  7.18s/it]
 97%|█████████▋| 218575/225000 [426:33:02<12:07:37,  6.79s/it]
 97%|█████████▋| 218576/225000 [426:33:16<15:51:43,  8.89s/it]
 97%|█████████▋| 218577/225000 [426:33:21<13:53:19,  7.78s/it]
 97%|█████████▋| 218578/225000 [426:33:32<15:59:37,  8.97s/it]
 97%|█████████▋| 218579/225000 [426:33:41<16:00:51,  8.98s/it]
 97%|█████████▋| 218580/225000 [426:33:49<15:20:09,  8.60s/it]
 97%|█████████▋| 218581/225000 [426:33:57<14:44:09,  8.26s/it]
 97%|█████████▋| 218582/225000 [426:34:08<16:28:04,  9.24s/it]
 97%|█████████▋| 218583/225000 [426:34:13<14:09:30,  7.94s/it]
 97%|█████████▋| 218584/225000 [426:34:23<15:20:29,  8.61s/it]
 97%|█████████▋| 218585/225000 [426:34:29<13:43:11,  7.70s/it]
 97%|█████████▋| 218586/225000 [426:34:41<15:57:13,  8.95s/it]
 97%|█████████▋| 218587/225000 [426:34:46<14:14:48,  8.00s/it]
 97%|█████████▋| 218588/225000 [426:35:00<17:13:13,  9.67s/it]
 97%|█████████▋| 218589/225000 [426:35:06<15:27:59,  8.68s/it]
 97%|█████████▋| 218590/225000 [426:35:18<16:46:57,  9.43s/it]
 97%|█████████▋| 218591/225000 [426:35:25<15:38:31,  8.79s/it]
 97%|█████████▋| 218592/225000 [426:35:30<13:26:59,  7.56s/it]
 97%|█████████▋| 218593/225000 [426:35:35<12:06:50,  6.81s/it]
 97%|█████████▋| 218594/225000 [426:35:41<11:50:05,  6.65s/it]
 97%|█████████▋| 218595/225000 [426:35:55<15:44:34,  8.85s/it]
 97%|█████████▋| 218596/225000 [426:36:00<13:37:37,  7.66s/it]
 97%|█████████▋| 218597/225000 [426:36:06<12:58:56,  7.30s/it]
 97%|█████████▋| 218598/225000 [426:36:18<15:36:08,  8.77s/it]
 97%|█████████▋| 218599/225000 [426:36:24<13:41:32,  7.70s/it]
 97%|█████████▋| 218600/225000 [426:36:31<13:18:43,  7.49s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.516', 'learning_rate': '1.128e-07', 'epoch': '0.9716', 'num_input_tokens_seen': 725931704, 'train_runtime': '1.536e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218600/225000 [426:36:31<13:18:43,  7.49s/it]
 97%|█████████▋| 218601/225000 [426:36:43<16:04:47,  9.05s/it]
 97%|█████████▋| 218602/225000 [426:36:51<15:25:01,  8.67s/it]
 97%|█████████▋| 218603/225000 [426:36:57<13:39:54,  7.69s/it]
 97%|█████████▋| 218604/225000 [426:37:06<14:24:17,  8.11s/it]
 97%|█████████▋| 218605/225000 [426:37:11<13:06:06,  7.38s/it]
 97%|█████████▋| 218606/225000 [426:37:20<13:46:43,  7.76s/it]
 97%|█████████▋| 218607/225000 [426:37:27<13:18:16,  7.49s/it]
 97%|█████████▋| 218608/225000 [426:37:37<14:30:34,  8.17s/it]
 97%|█████████▋| 218609/225000 [426:37:49<17:01:22,  9.59s/it]
 97%|█████████▋| 218610/225000 [426:37:57<15:59:37,  9.01s/it]
 97%|█████████▋| 218611/225000 [426:38:02<13:58:39,  7.88s/it]
 97%|█████████▋| 218612/225000 [426:38:08<12:54:57,  7.28s/it]
 97%|█████████▋| 218613/225000 [426:38:17<13:54:16,  7.84s/it]
 97%|█████████▋| 218614/225000 [426:38:24<13:07:21,  7.40s/it]
 97%|█████████▋| 218615/225000 [426:38:33<14:21:52,  8.10s/it]
 97%|█████████▋| 218616/225000 [426:38:43<14:55:23,  8.42s/it]
 97%|█████████▋| 218617/225000 [426:38:53<15:48:50,  8.92s/it]
 97%|█████████▋| 218618/225000 [426:39:01<15:27:49,  8.72s/it]
 97%|█████████▋| 218619/225000 [426:39:07<13:59:11,  7.89s/it]
 97%|█████████▋| 218620/225000 [426:39:19<16:05:40,  9.08s/it]
 97%|█████████▋| 218621/225000 [426:39:35<19:42:01, 11.12s/it]
 97%|█████████▋| 218622/225000 [426:39:44<18:37:27, 10.51s/it]
 97%|█████████▋| 218623/225000 [426:39:53<18:07:14, 10.23s/it]
 97%|█████████▋| 218624/225000 [426:40:04<18:11:05, 10.27s/it]
 97%|█████████▋| 218625/225000 [426:40:15<18:33:12, 10.48s/it]
 97%|█████████▋| 218626/225000 [426:40:22<16:49:50,  9.51s/it]
 97%|█████████▋| 218627/225000 [426:40:29<15:25:47,  8.72s/it]
 97%|█████████▋| 218628/225000 [426:40:35<14:05:36,  7.96s/it]
 97%|█████████▋| 218629/225000 [426:40:42<13:33:34,  7.66s/it]
 97%|█████████▋| 218630/225000 [426:40:57<17:19:08,  9.79s/it]
 97%|█████████▋| 218631/225000 [426:41:04<15:59:25,  9.04s/it]
 97%|█████████▋| 218632/225000 [426:41:18<18:23:22, 10.40s/it]
 97%|█████████▋| 218633/225000 [426:41:23<15:43:58,  8.90s/it]
 97%|█████████▋| 218634/225000 [426:41:38<19:04:21, 10.79s/it]
 97%|█████████▋| 218635/225000 [426:41:45<16:48:00,  9.50s/it]
 97%|█████████▋| 218636/225000 [426:41:51<15:05:22,  8.54s/it]
 97%|█████████▋| 218637/225000 [426:42:02<16:29:31,  9.33s/it]
 97%|█████████▋| 218638/225000 [426:42:11<16:15:15,  9.20s/it]
 97%|█████████▋| 218639/225000 [426:42:15<13:22:04,  7.57s/it]
 97%|█████████▋| 218640/225000 [426:42:25<14:39:25,  8.30s/it]
 97%|█████████▋| 218641/225000 [426:42:35<15:43:52,  8.91s/it]
 97%|█████████▋| 218642/225000 [426:42:43<15:13:19,  8.62s/it]
 97%|█████████▋| 218643/225000 [426:42:50<14:20:41,  8.12s/it]
 97%|█████████▋| 218644/225000 [426:42:57<13:50:46,  7.84s/it]
 97%|█████████▋| 218645/225000 [426:43:09<16:00:51,  9.07s/it]
 97%|█████████▋| 218646/225000 [426:43:19<16:27:29,  9.32s/it]
 97%|█████████▋| 218647/225000 [426:43:28<16:00:39,  9.07s/it]
 97%|█████████▋| 218648/225000 [426:43:40<18:03:54, 10.24s/it]
 97%|█████████▋| 218649/225000 [426:43:54<19:33:57, 11.09s/it]
 97%|█████████▋| 218650/225000 [426:44:02<18:19:36, 10.39s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.167', 'learning_rate': '1.111e-07', 'epoch': '0.9718', 'num_input_tokens_seen': 726164856, 'train_runtime': '1.536e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218650/225000 [426:44:02<18:19:36, 10.39s/it]
 97%|█████████▋| 218651/225000 [426:44:08<15:50:56,  8.99s/it]
 97%|█████████▋| 218652/225000 [426:44:15<15:00:00,  8.51s/it]
 97%|█████████▋| 218653/225000 [426:44:24<15:02:11,  8.53s/it]
 97%|█████████▋| 218654/225000 [426:44:33<15:09:59,  8.60s/it]
 97%|█████████▋| 218655/225000 [426:44:42<15:22:53,  8.73s/it]
 97%|█████████▋| 218656/225000 [426:44:53<16:33:23,  9.40s/it]
 97%|█████████▋| 218657/225000 [426:44:58<14:33:35,  8.26s/it]
 97%|█████████▋| 218658/225000 [426:45:05<13:48:16,  7.84s/it]
 97%|█████████▋| 218659/225000 [426:45:11<12:46:35,  7.25s/it]
 97%|█████████▋| 218660/225000 [426:45:27<17:16:07,  9.81s/it]
 97%|█████████▋| 218661/225000 [426:45:35<16:27:03,  9.34s/it]
 97%|█████████▋| 218662/225000 [426:45:45<16:28:31,  9.36s/it]
 97%|█████████▋| 218663/225000 [426:45:54<16:37:31,  9.44s/it]
 97%|█████████▋| 218664/225000 [426:46:05<17:06:17,  9.72s/it]
 97%|█████████▋| 218665/225000 [426:46:17<18:35:21, 10.56s/it]
 97%|█████████▋| 218666/225000 [426:46:27<18:03:16, 10.26s/it]
 97%|█████████▋| 218667/225000 [426:46:34<16:39:05,  9.47s/it]
 97%|█████████▋| 218668/225000 [426:46:44<16:59:22,  9.66s/it]
 97%|█████████▋| 218669/225000 [426:46:50<14:48:09,  8.42s/it]
 97%|█████████▋| 218670/225000 [426:46:56<13:40:53,  7.78s/it]
 97%|█████████▋| 218671/225000 [426:47:08<16:00:09,  9.10s/it]
 97%|█████████▋| 218672/225000 [426:47:16<15:25:35,  8.78s/it]
 97%|█████████▋| 218673/225000 [426:47:25<15:25:56,  8.78s/it]
 97%|█████████▋| 218674/225000 [426:47:32<14:37:00,  8.32s/it]
 97%|█████████▋| 218675/225000 [426:47:38<13:24:39,  7.63s/it]
 97%|█████████▋| 218676/225000 [426:47:51<16:02:12,  9.13s/it]
 97%|█████████▋| 218677/225000 [426:48:01<16:41:36,  9.50s/it]
 97%|█████████▋| 218678/225000 [426:48:08<14:53:36,  8.48s/it]
 97%|█████████▋| 218679/225000 [426:48:15<14:30:08,  8.26s/it]
 97%|█████████▋| 218680/225000 [426:48:22<13:29:54,  7.69s/it]
 97%|█████████▋| 218681/225000 [426:48:28<12:43:10,  7.25s/it]
 97%|█████████▋| 218682/225000 [426:48:34<12:23:40,  7.06s/it]
 97%|█████████▋| 218683/225000 [426:48:48<15:35:53,  8.89s/it]
 97%|█████████▋| 218684/225000 [426:48:57<15:55:48,  9.08s/it]
 97%|█████████▋| 218685/225000 [426:49:03<14:26:32,  8.23s/it]
 97%|█████████▋| 218686/225000 [426:49:12<14:43:34,  8.40s/it]
 97%|█████████▋| 218687/225000 [426:49:18<13:17:00,  7.57s/it]
 97%|█████████▋| 218688/225000 [426:49:28<14:30:41,  8.28s/it]
 97%|█████████▋| 218689/225000 [426:49:36<14:14:38,  8.13s/it]
 97%|█████████▋| 218690/225000 [426:49:52<18:50:23, 10.75s/it]
 97%|█████████▋| 218691/225000 [426:50:02<18:22:38, 10.49s/it]
 97%|█████████▋| 218692/225000 [426:50:08<16:02:19,  9.15s/it]
 97%|█████████▋| 218693/225000 [426:50:16<15:13:31,  8.69s/it]
 97%|█████████▋| 218694/225000 [426:50:29<17:40:19, 10.09s/it]
 97%|█████████▋| 218695/225000 [426:50:35<15:15:12,  8.71s/it]
 97%|█████████▋| 218696/225000 [426:50:44<15:29:39,  8.85s/it]
 97%|█████████▋| 218697/225000 [426:50:50<14:03:59,  8.03s/it]
 97%|█████████▋| 218698/225000 [426:50:56<13:05:45,  7.48s/it]
 97%|█████████▋| 218699/225000 [426:51:09<15:40:10,  8.95s/it]
 97%|█████████▋| 218700/225000 [426:51:15<14:28:08,  8.27s/it]
                                                              
{'loss': '1.312', 'grad_norm': '1.31', 'learning_rate': '1.093e-07', 'epoch': '0.972', 'num_input_tokens_seen': 726395792, 'train_runtime': '1.537e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218700/225000 [426:51:15<14:28:08,  8.27s/it]
 97%|█████████▋| 218701/225000 [426:51:23<14:12:08,  8.12s/it]
 97%|█████████▋| 218702/225000 [426:51:31<14:18:58,  8.18s/it]
 97%|█████████▋| 218703/225000 [426:51:37<13:06:22,  7.49s/it]
 97%|█████████▋| 218704/225000 [426:51:48<14:32:27,  8.31s/it]
 97%|█████████▋| 218705/225000 [426:51:54<13:31:00,  7.73s/it]
 97%|█████████▋| 218706/225000 [426:52:05<15:05:29,  8.63s/it]
 97%|█████████▋| 218707/225000 [426:52:15<15:54:33,  9.10s/it]
 97%|█████████▋| 218708/225000 [426:52:24<15:42:33,  8.99s/it]
 97%|█████████▋| 218709/225000 [426:52:40<19:23:04, 11.09s/it]
 97%|█████████▋| 218710/225000 [426:52:49<18:23:07, 10.52s/it]
 97%|█████████▋| 218711/225000 [426:52:59<18:17:56, 10.47s/it]
 97%|█████████▋| 218712/225000 [426:53:10<18:30:22, 10.60s/it]
 97%|█████████▋| 218713/225000 [426:53:19<17:25:43,  9.98s/it]
 97%|█████████▋| 218714/225000 [426:53:25<15:46:36,  9.04s/it]
 97%|█████████▋| 218715/225000 [426:53:32<14:20:53,  8.22s/it]
 97%|█████████▋| 218716/225000 [426:53:40<14:14:50,  8.16s/it]
 97%|█████████▋| 218717/225000 [426:53:49<14:43:34,  8.44s/it]
 97%|█████████▋| 218718/225000 [426:53:56<14:18:10,  8.20s/it]
 97%|█████████▋| 218719/225000 [426:54:07<15:39:17,  8.97s/it]
 97%|█████████▋| 218720/225000 [426:54:21<17:57:36, 10.30s/it]
 97%|█████████▋| 218721/225000 [426:54:28<16:13:29,  9.30s/it]
 97%|█████████▋| 218722/225000 [426:54:41<18:18:59, 10.50s/it]
 97%|█████████▋| 218723/225000 [426:54:48<16:42:51,  9.59s/it]
 97%|█████████▋| 218724/225000 [426:54:53<13:59:05,  8.02s/it]
 97%|█████████▋| 218725/225000 [426:55:02<14:31:43,  8.34s/it]
 97%|█████████▋| 218726/225000 [426:55:09<13:45:14,  7.89s/it]
 97%|█████████▋| 218727/225000 [426:55:14<12:36:29,  7.24s/it]
 97%|█████████▋| 218728/225000 [426:55:25<14:21:38,  8.24s/it]
 97%|█████████▋| 218729/225000 [426:55:34<14:58:51,  8.60s/it]
 97%|█████████▋| 218730/225000 [426:55:40<13:15:15,  7.61s/it]
 97%|█████████▋| 218731/225000 [426:55:52<15:44:31,  9.04s/it]
 97%|█████████▋| 218732/225000 [426:56:01<15:44:10,  9.04s/it]
 97%|█████████▋| 218733/225000 [426:56:11<16:16:11,  9.35s/it]
 97%|█████████▋| 218734/225000 [426:56:20<16:13:01,  9.32s/it]
 97%|█████████▋| 218735/225000 [426:56:27<14:39:53,  8.43s/it]
 97%|█████████▋| 218736/225000 [426:56:36<15:21:24,  8.83s/it]
 97%|█████████▋| 218737/225000 [426:56:43<13:58:44,  8.04s/it]
 97%|█████████▋| 218738/225000 [426:56:49<13:07:36,  7.55s/it]
 97%|█████████▋| 218739/225000 [426:57:04<16:45:03,  9.63s/it]
 97%|█████████▋| 218740/225000 [426:57:18<19:04:28, 10.97s/it]
 97%|█████████▋| 218741/225000 [426:57:22<15:51:24,  9.12s/it]
 97%|█████████▋| 218742/225000 [426:57:29<14:36:43,  8.41s/it]
 97%|█████████▋| 218743/225000 [426:57:40<15:49:27,  9.10s/it]
 97%|█████████▋| 218744/225000 [426:57:45<13:52:21,  7.98s/it]
 97%|█████████▋| 218745/225000 [426:57:51<12:45:46,  7.35s/it]
 97%|█████████▋| 218746/225000 [426:58:02<14:46:13,  8.50s/it]
 97%|█████████▋| 218747/225000 [426:58:12<15:26:45,  8.89s/it]
 97%|█████████▋| 218748/225000 [426:58:22<15:49:49,  9.12s/it]
 97%|█████████▋| 218749/225000 [426:58:27<13:57:17,  8.04s/it]
 97%|█████████▋| 218750/225000 [426:58:33<12:47:02,  7.36s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.668', 'learning_rate': '1.076e-07', 'epoch': '0.9722', 'num_input_tokens_seen': 726626080, 'train_runtime': '1.537e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218750/225000 [426:58:33<12:47:02,  7.36s/it]
 97%|█████████▋| 218751/225000 [426:58:47<16:04:43,  9.26s/it]
 97%|█████████▋| 218752/225000 [426:58:57<16:30:04,  9.51s/it]
 97%|█████████▋| 218753/225000 [426:59:02<13:59:03,  8.06s/it]
 97%|█████████▋| 218754/225000 [426:59:10<14:22:30,  8.29s/it]
 97%|█████████▋| 218755/225000 [426:59:22<15:53:47,  9.16s/it]
 97%|█████████▋| 218756/225000 [426:59:29<14:55:54,  8.61s/it]
 97%|█████████▋| 218757/225000 [426:59:37<14:41:29,  8.47s/it]
 97%|█████████▋| 218758/225000 [426:59:44<13:58:44,  8.06s/it]
 97%|█████████▋| 218759/225000 [426:59:57<16:23:24,  9.45s/it]
 97%|█████████▋| 218760/225000 [427:00:03<14:24:09,  8.31s/it]
 97%|█████████▋| 218761/225000 [427:00:13<15:37:47,  9.02s/it]
 97%|█████████▋| 218762/225000 [427:00:21<15:14:32,  8.80s/it]
 97%|█████████▋| 218763/225000 [427:00:27<13:42:40,  7.91s/it]
 97%|█████████▋| 218764/225000 [427:00:40<16:01:39,  9.25s/it]
 97%|█████████▋| 218765/225000 [427:00:47<15:13:03,  8.79s/it]
 97%|█████████▋| 218766/225000 [427:00:55<14:29:03,  8.36s/it]
 97%|█████████▋| 218767/225000 [427:01:06<15:43:23,  9.08s/it]
 97%|█████████▋| 218768/225000 [427:01:12<14:34:37,  8.42s/it]
 97%|█████████▋| 218769/225000 [427:01:19<13:24:12,  7.74s/it]
 97%|█████████▋| 218770/225000 [427:01:28<14:30:54,  8.39s/it]
 97%|█████████▋| 218771/225000 [427:01:39<15:32:04,  8.98s/it]
 97%|█████████▋| 218772/225000 [427:01:47<14:55:13,  8.62s/it]
 97%|█████████▋| 218773/225000 [427:01:52<13:09:34,  7.61s/it]
 97%|█████████▋| 218774/225000 [427:02:05<16:12:23,  9.37s/it]
 97%|█████████▋| 218775/225000 [427:02:14<15:38:41,  9.05s/it]
 97%|█████████▋| 218776/225000 [427:02:23<15:37:28,  9.04s/it]
 97%|█████████▋| 218777/225000 [427:02:30<14:38:44,  8.47s/it]
 97%|█████████▋| 218778/225000 [427:02:40<15:42:01,  9.08s/it]
 97%|█████████▋| 218779/225000 [427:02:45<13:26:12,  7.78s/it]
 97%|█████████▋| 218780/225000 [427:02:57<15:38:43,  9.06s/it]
 97%|█████████▋| 218781/225000 [427:03:06<15:48:55,  9.16s/it]
 97%|█████████▋| 218782/225000 [427:03:14<15:09:05,  8.77s/it]
 97%|█████████▋| 218783/225000 [427:03:28<17:33:27, 10.17s/it]
 97%|█████████▋| 218784/225000 [427:03:35<16:12:43,  9.39s/it]
 97%|█████████▋| 218785/225000 [427:03:47<17:18:33, 10.03s/it]
 97%|█████████▋| 218786/225000 [427:03:57<17:16:25, 10.01s/it]
 97%|█████████▋| 218787/225000 [427:04:01<14:12:28,  8.23s/it]
 97%|█████████▋| 218788/225000 [427:04:08<13:34:11,  7.86s/it]
 97%|█████████▋| 218789/225000 [427:04:19<15:05:04,  8.74s/it]
 97%|█████████▋| 218790/225000 [427:04:26<14:29:15,  8.40s/it]
 97%|█████████▋| 218791/225000 [427:04:32<13:16:49,  7.70s/it]
 97%|█████████▋| 218792/225000 [427:04:39<12:42:54,  7.37s/it]
 97%|█████████▋| 218793/225000 [427:04:47<13:15:37,  7.69s/it]
 97%|█████████▋| 218794/225000 [427:05:03<17:33:56, 10.19s/it]
 97%|█████████▋| 218795/225000 [427:05:13<17:02:53,  9.89s/it]
 97%|█████████▋| 218796/225000 [427:05:20<15:46:57,  9.16s/it]
 97%|█████████▋| 218797/225000 [427:05:30<16:13:32,  9.42s/it]
 97%|█████████▋| 218798/225000 [427:05:38<15:11:24,  8.82s/it]
 97%|█████████▋| 218799/225000 [427:05:50<16:56:48,  9.84s/it]
 97%|█████████▋| 218800/225000 [427:05:58<16:11:49,  9.40s/it]
                                                              
{'loss': '1.335', 'grad_norm': '1.51', 'learning_rate': '1.06e-07', 'epoch': '0.9724', 'num_input_tokens_seen': 726858160, 'train_runtime': '1.538e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218800/225000 [427:05:58<16:11:49,  9.40s/it]
 97%|█████████▋| 218801/225000 [427:06:07<15:44:17,  9.14s/it]
 97%|█████████▋| 218802/225000 [427:06:17<16:15:24,  9.44s/it]
 97%|█████████▋| 218803/225000 [427:06:27<16:30:22,  9.59s/it]
 97%|█████████▋| 218804/225000 [427:06:31<13:59:00,  8.12s/it]
 97%|█████████▋| 218805/225000 [427:06:40<13:58:44,  8.12s/it]
 97%|█████████▋| 218806/225000 [427:06:51<15:35:19,  9.06s/it]
 97%|█████████▋| 218807/225000 [427:07:01<16:21:23,  9.51s/it]
 97%|█████████▋| 218808/225000 [427:07:07<14:34:25,  8.47s/it]
 97%|█████████▋| 218809/225000 [427:07:19<16:10:17,  9.40s/it]
 97%|█████████▋| 218810/225000 [427:07:28<15:54:02,  9.25s/it]
 97%|█████████▋| 218811/225000 [427:07:34<14:16:11,  8.30s/it]
 97%|█████████▋| 218812/225000 [427:07:40<13:18:38,  7.74s/it]
 97%|█████████▋| 218813/225000 [427:07:47<12:49:04,  7.46s/it]
 97%|█████████▋| 218814/225000 [427:08:00<15:21:34,  8.94s/it]
 97%|█████████▋| 218815/225000 [427:08:04<13:13:18,  7.70s/it]
 97%|█████████▋| 218816/225000 [427:08:14<14:24:52,  8.39s/it]
 97%|█████████▋| 218817/225000 [427:08:22<13:49:37,  8.05s/it]
 97%|█████████▋| 218818/225000 [427:08:29<13:37:03,  7.93s/it]
 97%|█████████▋| 218819/225000 [427:08:35<12:40:13,  7.38s/it]
 97%|█████████▋| 218820/225000 [427:08:53<17:52:41, 10.41s/it]
 97%|█████████▋| 218821/225000 [427:08:58<14:55:35,  8.70s/it]
 97%|█████████▋| 218822/225000 [427:09:11<17:18:28, 10.09s/it]
 97%|█████████▋| 218823/225000 [427:09:16<14:42:19,  8.57s/it]
 97%|█████████▋| 218824/225000 [427:09:24<14:25:22,  8.41s/it]
 97%|█████████▋| 218825/225000 [427:09:31<13:33:07,  7.90s/it]
 97%|█████████▋| 218826/225000 [427:09:36<12:26:32,  7.25s/it]
 97%|█████████▋| 218827/225000 [427:09:46<13:37:11,  7.94s/it]
 97%|█████████▋| 218828/225000 [427:09:54<13:27:20,  7.85s/it]
 97%|█████████▋| 218829/225000 [427:10:05<15:01:23,  8.76s/it]
 97%|█████████▋| 218830/225000 [427:10:11<13:57:29,  8.14s/it]
 97%|█████████▋| 218831/225000 [427:10:18<13:28:49,  7.87s/it]
 97%|█████████▋| 218832/225000 [427:10:33<16:52:10,  9.85s/it]
 97%|█████████▋| 218833/225000 [427:10:44<17:24:52, 10.17s/it]
 97%|█████████▋| 218834/225000 [427:10:50<15:13:22,  8.89s/it]
 97%|█████████▋| 218835/225000 [427:10:57<14:20:51,  8.38s/it]
 97%|█████████▋| 218836/225000 [427:11:07<15:02:08,  8.78s/it]
 97%|█████████▋| 218837/225000 [427:11:14<14:14:42,  8.32s/it]
 97%|█████████▋| 218838/225000 [427:11:22<14:12:18,  8.30s/it]
 97%|█████████▋| 218839/225000 [427:11:29<13:39:43,  7.98s/it]
 97%|█████████▋| 218840/225000 [427:11:40<14:47:23,  8.64s/it]
 97%|█████████▋| 218841/225000 [427:11:50<15:32:23,  9.08s/it]
 97%|█████████▋| 218842/225000 [427:12:00<16:00:41,  9.36s/it]
 97%|█████████▋| 218843/225000 [427:12:06<14:20:01,  8.38s/it]
 97%|█████████▋| 218844/225000 [427:12:11<12:27:59,  7.29s/it]
 97%|█████████▋| 218845/225000 [427:12:18<12:29:03,  7.30s/it]
 97%|█████████▋| 218846/225000 [427:12:29<14:34:06,  8.52s/it]
 97%|█████████▋| 218847/225000 [427:12:39<15:19:58,  8.97s/it]
 97%|█████████▋| 218848/225000 [427:12:46<14:08:26,  8.27s/it]
 97%|█████████▋| 218849/225000 [427:12:58<16:17:35,  9.54s/it]
 97%|█████████▋| 218850/225000 [427:13:10<17:20:29, 10.15s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.32', 'learning_rate': '1.043e-07', 'epoch': '0.9727', 'num_input_tokens_seen': 727083504, 'train_runtime': '1.538e+06', 'train_tokens_per_second': '472.7'}

 97%|█████████▋| 218850/225000 [427:13:10<17:20:29, 10.15s/it]
 97%|█████████▋| 218851/225000 [427:13:15<14:49:07,  8.68s/it]
 97%|█████████▋| 218852/225000 [427:13:24<15:02:30,  8.81s/it]
 97%|█████████▋| 218853/225000 [427:13:28<12:35:26,  7.37s/it]
 97%|█████████▋| 218854/225000 [427:13:36<12:53:25,  7.55s/it]
 97%|█████████▋| 218855/225000 [427:13:48<14:55:07,  8.74s/it]
 97%|█████████▋| 218856/225000 [427:13:59<16:00:19,  9.38s/it]
 97%|█████████▋| 218857/225000 [427:14:03<13:12:07,  7.74s/it]
 97%|█████████▋| 218858/225000 [427:14:12<14:12:45,  8.33s/it]
 97%|█████████▋| 218859/225000 [427:14:23<15:15:27,  8.94s/it]
 97%|█████████▋| 218860/225000 [427:14:31<15:11:33,  8.91s/it]
 97%|█████████▋| 218861/225000 [427:14:39<14:43:45,  8.64s/it]
 97%|█████████▋| 218862/225000 [427:14:47<14:00:25,  8.22s/it]
 97%|█████████▋| 218863/225000 [427:14:55<14:17:16,  8.38s/it]
 97%|█████████▋| 218864/225000 [427:15:06<15:27:13,  9.07s/it]
 97%|█████████▋| 218865/225000 [427:15:13<14:23:41,  8.45s/it]
 97%|█████████▋| 218866/225000 [427:15:19<13:18:07,  7.81s/it]
 97%|█████████▋| 218867/225000 [427:15:29<14:22:08,  8.43s/it]
 97%|█████████▋| 218868/225000 [427:15:36<13:18:46,  7.82s/it]
 97%|█████████▋| 218869/225000 [427:15:41<11:59:06,  7.04s/it]
 97%|█████████▋| 218870/225000 [427:15:51<13:22:32,  7.86s/it]
 97%|█████████▋| 218871/225000 [427:15:58<12:59:09,  7.63s/it]
 97%|█████████▋| 218872/225000 [427:16:05<12:47:24,  7.51s/it]
 97%|█████████▋| 218873/225000 [427:16:18<15:33:19,  9.14s/it]
 97%|█████████▋| 218874/225000 [427:16:26<15:12:37,  8.94s/it]
 97%|█████████▋| 218875/225000 [427:16:34<14:34:43,  8.57s/it]
 97%|█████████▋| 218876/225000 [427:16:50<18:23:28, 10.81s/it]
 97%|█████████▋| 218877/225000 [427:16:56<15:35:57,  9.17s/it]
 97%|█████████▋| 218878/225000 [427:17:06<16:00:07,  9.41s/it]
 97%|█████████▋| 218879/225000 [427:17:11<13:48:35,  8.12s/it]
 97%|█████████▋| 218880/225000 [427:17:19<13:48:08,  8.12s/it]
 97%|█████████▋| 218881/225000 [427:17:24<12:30:29,  7.36s/it]
 97%|█████████▋| 218882/225000 [427:17:41<17:19:50, 10.20s/it]
 97%|█████████▋| 218883/225000 [427:17:49<16:21:29,  9.63s/it]
 97%|█████████▋| 218884/225000 [427:18:03<18:22:54, 10.82s/it]
 97%|█████████▋| 218885/225000 [427:18:10<16:11:31,  9.53s/it]
 97%|█████████▋| 218886/225000 [427:18:21<17:00:08, 10.01s/it]
 97%|█████████▋| 218887/225000 [427:18:27<15:18:22,  9.01s/it]
 97%|█████████▋| 218888/225000 [427:18:33<13:36:32,  8.02s/it]
 97%|█████████▋| 218889/225000 [427:18:39<12:32:03,  7.38s/it]
 97%|█████████▋| 218890/225000 [427:18:45<11:45:01,  6.92s/it]
 97%|█████████▋| 218891/225000 [427:18:59<15:13:52,  8.98s/it]
 97%|█████████▋| 218892/225000 [427:19:05<13:46:23,  8.12s/it]
 97%|█████████▋| 218893/225000 [427:19:12<13:06:22,  7.73s/it]
 97%|█████████▋| 218894/225000 [427:19:25<16:08:58,  9.52s/it]
 97%|█████████▋| 218895/225000 [427:19:32<14:34:25,  8.59s/it]
 97%|█████████▋| 218896/225000 [427:19:40<14:10:36,  8.36s/it]
 97%|█████████▋| 218897/225000 [427:19:47<13:48:21,  8.14s/it]
 97%|█████████▋| 218898/225000 [427:19:55<13:30:43,  7.97s/it]
 97%|█████████▋| 218899/225000 [427:20:01<12:45:36,  7.53s/it]
 97%|█████████▋| 218900/225000 [427:20:07<12:04:27,  7.13s/it]
                                                              
{'loss': '1.291', 'grad_norm': '1.217', 'learning_rate': '1.026e-07', 'epoch': '0.9729', 'num_input_tokens_seen': 727314080, 'train_runtime': '1.538e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 218900/225000 [427:20:07<12:04:27,  7.13s/it]
 97%|█████████▋| 218901/225000 [427:20:15<12:32:15,  7.40s/it]
 97%|█████████▋| 218902/225000 [427:20:29<15:36:19,  9.21s/it]
 97%|█████████▋| 218903/225000 [427:20:37<15:12:48,  8.98s/it]
 97%|█████████▋| 218904/225000 [427:20:45<14:18:52,  8.45s/it]
 97%|█████████▋| 218905/225000 [427:20:50<12:50:58,  7.59s/it]
 97%|█████████▋| 218906/225000 [427:21:01<14:38:47,  8.65s/it]
 97%|█████████▋| 218907/225000 [427:21:08<13:34:45,  8.02s/it]
 97%|█████████▋| 218908/225000 [427:21:17<14:10:49,  8.38s/it]
 97%|█████████▋| 218909/225000 [427:21:27<14:56:51,  8.83s/it]
 97%|█████████▋| 218910/225000 [427:21:38<15:56:40,  9.43s/it]
 97%|█████████▋| 218911/225000 [427:21:44<14:09:43,  8.37s/it]
 97%|█████████▋| 218912/225000 [427:21:51<13:38:46,  8.07s/it]
 97%|█████████▋| 218913/225000 [427:22:07<17:30:24, 10.35s/it]
 97%|█████████▋| 218914/225000 [427:22:15<16:27:15,  9.73s/it]
 97%|█████████▋| 218915/225000 [427:22:22<14:52:50,  8.80s/it]
 97%|█████████▋| 218916/225000 [427:22:32<15:35:20,  9.22s/it]
 97%|█████████▋| 218917/225000 [427:22:37<13:26:54,  7.96s/it]
 97%|█████████▋| 218918/225000 [427:22:49<15:38:18,  9.26s/it]
 97%|█████████▋| 218919/225000 [427:23:01<16:53:05, 10.00s/it]
 97%|█████████▋| 218920/225000 [427:23:14<18:29:55, 10.95s/it]
 97%|█████████▋| 218921/225000 [427:23:20<15:56:07,  9.44s/it]
 97%|█████████▋| 218922/225000 [427:23:24<13:11:25,  7.81s/it]
 97%|█████████▋| 218923/225000 [427:23:33<13:57:11,  8.27s/it]
 97%|█████████▋| 218924/225000 [427:23:40<13:25:44,  7.96s/it]
 97%|█████████▋| 218925/225000 [427:23:49<13:55:48,  8.25s/it]
 97%|█████████▋| 218926/225000 [427:23:59<14:31:30,  8.61s/it]
 97%|█████████▋| 218927/225000 [427:24:05<13:06:06,  7.77s/it]
 97%|█████████▋| 218928/225000 [427:24:15<14:36:36,  8.66s/it]
 97%|█████████▋| 218929/225000 [427:24:25<14:58:39,  8.88s/it]
 97%|█████████▋| 218930/225000 [427:24:35<15:25:31,  9.15s/it]
 97%|█████████▋| 218931/225000 [427:24:43<15:03:30,  8.93s/it]
 97%|█████████▋| 218932/225000 [427:24:48<13:15:17,  7.86s/it]
 97%|█████████▋| 218933/225000 [427:24:53<11:46:27,  6.99s/it]
 97%|█████████▋| 218934/225000 [427:25:06<14:49:02,  8.79s/it]
 97%|█████████▋| 218935/225000 [427:25:11<12:31:49,  7.44s/it]
 97%|█████████▋| 218936/225000 [427:25:19<12:51:57,  7.64s/it]
 97%|█████████▋| 218937/225000 [427:25:25<12:11:00,  7.23s/it]
 97%|█████████▋| 218938/225000 [427:25:33<12:29:50,  7.42s/it]
 97%|█████████▋| 218939/225000 [427:25:47<15:43:42,  9.34s/it]
 97%|█████████▋| 218940/225000 [427:25:55<15:07:11,  8.98s/it]
 97%|█████████▋| 218941/225000 [427:26:02<14:13:13,  8.45s/it]
 97%|█████████▋| 218942/225000 [427:26:12<14:54:02,  8.85s/it]
 97%|█████████▋| 218943/225000 [427:26:22<15:35:53,  9.27s/it]
 97%|█████████▋| 218944/225000 [427:26:32<15:54:15,  9.45s/it]
 97%|█████████▋| 218945/225000 [427:26:39<14:28:58,  8.61s/it]
 97%|█████████▋| 218946/225000 [427:26:46<13:54:50,  8.27s/it]
 97%|█████████▋| 218947/225000 [427:26:56<14:47:51,  8.80s/it]
 97%|█████████▋| 218948/225000 [427:27:07<15:40:34,  9.32s/it]
 97%|█████████▋| 218949/225000 [427:27:14<14:34:55,  8.68s/it]
 97%|█████████▋| 218950/225000 [427:27:20<13:17:50,  7.91s/it]
                                                              
{'loss': '1.318', 'grad_norm': '1.4', 'learning_rate': '1.01e-07', 'epoch': '0.9731', 'num_input_tokens_seen': 727539328, 'train_runtime': '1.539e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 218950/225000 [427:27:20<13:17:50,  7.91s/it]
 97%|█████████▋| 218951/225000 [427:27:34<16:37:49,  9.90s/it]
 97%|█████████▋| 218952/225000 [427:27:46<17:13:28, 10.25s/it]
 97%|█████████▋| 218953/225000 [427:27:50<14:14:02,  8.47s/it]
 97%|█████████▋| 218954/225000 [427:27:56<12:52:27,  7.67s/it]
 97%|█████████▋| 218955/225000 [427:28:03<12:46:06,  7.60s/it]
 97%|█████████▋| 218956/225000 [427:28:17<16:03:52,  9.57s/it]
 97%|█████████▋| 218957/225000 [427:28:24<14:43:15,  8.77s/it]
 97%|█████████▋| 218958/225000 [427:28:33<14:56:49,  8.91s/it]
 97%|█████████▋| 218959/225000 [427:28:40<13:37:19,  8.12s/it]
 97%|█████████▋| 218960/225000 [427:28:48<13:28:51,  8.04s/it]
 97%|█████████▋| 218961/225000 [427:28:56<13:46:35,  8.21s/it]
 97%|█████████▋| 218962/225000 [427:29:04<13:42:50,  8.18s/it]
 97%|█████████▋| 218963/225000 [427:29:21<17:47:12, 10.61s/it]
 97%|█████████▋| 218964/225000 [427:29:30<17:02:00, 10.16s/it]
 97%|█████████▋| 218965/225000 [427:29:37<15:43:07,  9.38s/it]
 97%|█████████▋| 218966/225000 [427:29:44<14:34:15,  8.69s/it]
 97%|█████████▋| 218967/225000 [427:29:56<15:53:57,  9.49s/it]
 97%|█████████▋| 218968/225000 [427:30:01<13:46:45,  8.22s/it]
 97%|█████████▋| 218969/225000 [427:30:08<13:16:39,  7.93s/it]
 97%|█████████▋| 218970/225000 [427:30:21<15:52:53,  9.48s/it]
 97%|█████████▋| 218971/225000 [427:30:30<15:35:04,  9.31s/it]
 97%|█████████▋| 218972/225000 [427:30:40<16:03:47,  9.59s/it]
 97%|█████████▋| 218973/225000 [427:30:47<14:17:52,  8.54s/it]
 97%|█████████▋| 218974/225000 [427:30:57<15:22:53,  9.19s/it]
 97%|█████████▋| 218975/225000 [427:31:06<15:22:59,  9.19s/it]
 97%|█████████▋| 218976/225000 [427:31:13<14:02:00,  8.39s/it]
 97%|█████████▋| 218977/225000 [427:31:20<13:10:20,  7.87s/it]
 97%|█████████▋| 218978/225000 [427:31:36<17:41:08, 10.57s/it]
 97%|█████████▋| 218979/225000 [427:31:42<14:54:49,  8.92s/it]
 97%|█████████▋| 218980/225000 [427:31:48<13:26:38,  8.04s/it]
 97%|█████████▋| 218981/225000 [427:32:01<16:24:22,  9.81s/it]
 97%|█████████▋| 218982/225000 [427:32:08<14:40:02,  8.77s/it]
 97%|█████████▋| 218983/225000 [427:32:17<15:04:02,  9.01s/it]
 97%|█████████▋| 218984/225000 [427:32:27<15:12:40,  9.10s/it]
 97%|█████████▋| 218985/225000 [427:32:35<15:03:12,  9.01s/it]
 97%|█████████▋| 218986/225000 [427:32:45<15:13:31,  9.11s/it]
 97%|█████████▋| 218987/225000 [427:32:54<15:27:46,  9.26s/it]
 97%|█████████▋| 218988/225000 [427:33:03<15:19:56,  9.18s/it]
 97%|█████████▋| 218989/225000 [427:33:11<14:40:51,  8.79s/it]
 97%|█████████▋| 218990/225000 [427:33:22<15:37:49,  9.36s/it]
 97%|█████████▋| 218991/225000 [427:33:27<13:27:26,  8.06s/it]
 97%|█████████▋| 218992/225000 [427:33:36<14:07:41,  8.47s/it]
 97%|█████████▋| 218993/225000 [427:33:45<13:57:13,  8.36s/it]
 97%|█████████▋| 218994/225000 [427:33:54<14:25:44,  8.65s/it]
 97%|█████████▋| 218995/225000 [427:34:05<15:35:08,  9.34s/it]
 97%|█████████▋| 218996/225000 [427:34:13<14:49:29,  8.89s/it]
 97%|█████████▋| 218997/225000 [427:34:22<14:58:54,  8.98s/it]
 97%|█████████▋| 218998/225000 [427:34:29<14:00:57,  8.41s/it]
 97%|█████████▋| 218999/225000 [427:34:36<13:29:39,  8.10s/it]
 97%|█████████▋| 219000/225000 [427:34:51<16:34:18,  9.94s/it]
                                                              
{'loss': '1.32', 'grad_norm': '1.775', 'learning_rate': '9.934e-08', 'epoch': '0.9733', 'num_input_tokens_seen': 727777832, 'train_runtime': '1.539e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 219000/225000 [427:34:51<16:34:18,  9.94s/it]
 97%|█████████▋| 219001/225000 [427:34:58<15:10:31,  9.11s/it]
 97%|█████████▋| 219002/225000 [427:35:05<14:15:08,  8.55s/it]
 97%|█████████▋| 219003/225000 [427:35:11<13:11:34,  7.92s/it]
 97%|█████████▋| 219004/225000 [427:35:20<13:41:59,  8.23s/it]
 97%|█████████▋| 219005/225000 [427:35:36<17:10:10, 10.31s/it]
 97%|█████████▋| 219006/225000 [427:35:41<14:45:38,  8.87s/it]
 97%|█████████▋| 219007/225000 [427:35:55<17:06:55, 10.28s/it]
 97%|█████████▋| 219008/225000 [427:36:03<16:14:02,  9.75s/it]
 97%|█████████▋| 219009/225000 [427:36:10<14:37:53,  8.79s/it]
 97%|█████████▋| 219010/225000 [427:36:22<16:18:17,  9.80s/it]
 97%|█████████▋| 219011/225000 [427:36:33<16:45:30, 10.07s/it]
 97%|█████████▋| 219012/225000 [427:36:42<16:26:57,  9.89s/it]
 97%|█████████▋| 219013/225000 [427:36:50<15:25:53,  9.28s/it]
 97%|█████████▋| 219014/225000 [427:36:55<13:16:47,  7.99s/it]
 97%|█████████▋| 219015/225000 [427:37:01<12:35:19,  7.57s/it]
 97%|█████████▋| 219016/225000 [427:37:15<15:30:24,  9.33s/it]
 97%|█████████▋| 219017/225000 [427:37:24<15:21:30,  9.24s/it]
 97%|█████████▋| 219018/225000 [427:37:34<15:50:44,  9.54s/it]
 97%|█████████▋| 219019/225000 [427:37:38<13:11:41,  7.94s/it]
 97%|█████████▋| 219020/225000 [427:37:50<15:15:59,  9.19s/it]
 97%|█████████▋| 219021/225000 [427:37:59<14:59:42,  9.03s/it]
 97%|█████████▋| 219022/225000 [427:38:07<14:12:15,  8.55s/it]
 97%|█████████▋| 219023/225000 [427:38:18<15:32:11,  9.36s/it]
 97%|█████████▋| 219024/225000 [427:38:29<16:21:24,  9.85s/it]
 97%|█████████▋| 219025/225000 [427:38:36<14:57:27,  9.01s/it]
 97%|█████████▋| 219026/225000 [427:38:43<13:58:16,  8.42s/it]
 97%|█████████▋| 219027/225000 [427:38:50<13:23:57,  8.08s/it]
 97%|█████████▋| 219028/225000 [427:39:03<15:55:33,  9.60s/it]
 97%|█████████▋| 219029/225000 [427:39:12<15:32:31,  9.37s/it]
 97%|█████████▋| 219030/225000 [427:39:22<15:45:19,  9.50s/it]
 97%|█████████▋| 219031/225000 [427:39:32<15:53:09,  9.58s/it]
 97%|█████████▋| 219032/225000 [427:39:43<16:53:19, 10.19s/it]
 97%|█████████▋| 219033/225000 [427:39:51<15:49:23,  9.55s/it]
 97%|█████████▋| 219034/225000 [427:39:58<14:18:16,  8.63s/it]
 97%|█████████▋| 219035/225000 [427:40:07<14:28:55,  8.74s/it]
 97%|█████████▋| 219036/225000 [427:40:19<16:19:41,  9.86s/it]
 97%|█████████▋| 219037/225000 [427:40:28<15:49:22,  9.55s/it]
 97%|█████████▋| 219038/225000 [427:40:34<13:58:53,  8.44s/it]
 97%|█████████▋| 219039/225000 [427:40:42<13:48:03,  8.33s/it]
 97%|█████████▋| 219040/225000 [427:40:50<13:44:17,  8.30s/it]
 97%|█████████▋| 219041/225000 [427:40:57<13:00:16,  7.86s/it]
 97%|█████████▋| 219042/225000 [427:41:12<16:32:12,  9.99s/it]
 97%|█████████▋| 219043/225000 [427:41:17<14:00:34,  8.47s/it]
 97%|█████████▋| 219044/225000 [427:41:26<14:17:08,  8.63s/it]
 97%|█████████▋| 219045/225000 [427:41:31<12:25:57,  7.52s/it]
 97%|█████████▋| 219046/225000 [427:41:43<14:52:49,  9.00s/it]
 97%|█████████▋| 219047/225000 [427:41:51<14:06:53,  8.54s/it]
 97%|█████████▋| 219048/225000 [427:41:57<13:07:01,  7.93s/it]
 97%|█████████▋| 219049/225000 [427:42:05<12:45:39,  7.72s/it]
 97%|█████████▋| 219050/225000 [427:42:11<11:51:46,  7.18s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.366', 'learning_rate': '9.772e-08', 'epoch': '0.9736', 'num_input_tokens_seen': 728011032, 'train_runtime': '1.54e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 219050/225000 [427:42:11<11:51:46,  7.18s/it]
 97%|█████████▋| 219051/225000 [427:42:24<15:02:27,  9.10s/it]
 97%|█████████▋| 219052/225000 [427:42:29<12:59:25,  7.86s/it]
 97%|█████████▋| 219053/225000 [427:42:39<13:56:12,  8.44s/it]
 97%|█████████▋| 219054/225000 [427:42:44<12:20:42,  7.47s/it]
 97%|█████████▋| 219055/225000 [427:42:58<15:24:04,  9.33s/it]
 97%|█████████▋| 219056/225000 [427:43:07<15:32:51,  9.42s/it]
 97%|█████████▋| 219057/225000 [427:43:17<15:46:39,  9.56s/it]
 97%|█████████▋| 219058/225000 [427:43:25<14:46:29,  8.95s/it]
 97%|█████████▋| 219059/225000 [427:43:33<14:35:00,  8.84s/it]
 97%|█████████▋| 219060/225000 [427:43:39<12:51:47,  7.80s/it]
 97%|█████████▋| 219061/225000 [427:43:45<12:11:54,  7.39s/it]
 97%|█████████▋| 219062/225000 [427:43:58<14:47:17,  8.97s/it]
 97%|█████████▋| 219063/225000 [427:44:05<13:56:27,  8.45s/it]
 97%|█████████▋| 219064/225000 [427:44:17<15:48:41,  9.59s/it]
 97%|█████████▋| 219065/225000 [427:44:23<13:43:21,  8.32s/it]
 97%|█████████▋| 219066/225000 [427:44:28<12:22:46,  7.51s/it]
 97%|█████████▋| 219067/225000 [427:44:43<15:52:12,  9.63s/it]
 97%|█████████▋| 219068/225000 [427:44:52<15:32:38,  9.43s/it]
 97%|█████████▋| 219069/225000 [427:45:01<15:29:38,  9.40s/it]
 97%|█████████▋| 219070/225000 [427:45:07<13:50:50,  8.41s/it]
 97%|█████████▋| 219071/225000 [427:45:14<13:15:28,  8.05s/it]
 97%|█████████▋| 219072/225000 [427:45:22<13:07:53,  7.97s/it]
 97%|█████████▋| 219073/225000 [427:45:30<12:55:30,  7.85s/it]
 97%|█████████▋| 219074/225000 [427:45:42<14:51:37,  9.03s/it]
 97%|█████████▋| 219075/225000 [427:45:52<15:34:46,  9.47s/it]
 97%|█████████▋| 219076/225000 [427:45:56<12:56:03,  7.86s/it]
 97%|█████████▋| 219077/225000 [427:46:03<12:37:44,  7.68s/it]
 97%|█████████▋| 219078/225000 [427:46:16<14:54:05,  9.06s/it]
 97%|█████████▋| 219079/225000 [427:46:29<17:10:56, 10.45s/it]
 97%|█████████▋| 219080/225000 [427:46:35<14:53:24,  9.05s/it]
 97%|█████████▋| 219081/225000 [427:46:43<14:12:55,  8.65s/it]
 97%|█████████▋| 219082/225000 [427:46:57<16:54:50, 10.29s/it]
 97%|█████████▋| 219083/225000 [427:47:04<15:02:53,  9.16s/it]
 97%|█████████▋| 219084/225000 [427:47:17<16:59:12, 10.34s/it]
 97%|█████████▋| 219085/225000 [427:47:26<16:34:13, 10.09s/it]
 97%|█████████▋| 219086/225000 [427:47:35<15:55:49,  9.70s/it]
 97%|█████████▋| 219087/225000 [427:47:45<16:17:30,  9.92s/it]
 97%|█████████▋| 219088/225000 [427:47:52<14:26:14,  8.79s/it]
 97%|█████████▋| 219089/225000 [427:48:02<15:07:42,  9.21s/it]
 97%|█████████▋| 219090/225000 [427:48:07<13:24:32,  8.17s/it]
 97%|█████████▋| 219091/225000 [427:48:14<12:20:59,  7.52s/it]
 97%|█████████▋| 219092/225000 [427:48:27<15:19:44,  9.34s/it]
 97%|█████████▋| 219093/225000 [427:48:38<15:56:26,  9.71s/it]
 97%|█████████▋| 219094/225000 [427:48:46<15:23:59,  9.39s/it]
 97%|█████████▋| 219095/225000 [427:48:54<14:48:52,  9.03s/it]
 97%|█████████▋| 219096/225000 [427:49:01<13:31:40,  8.25s/it]
 97%|█████████▋| 219097/225000 [427:49:16<17:00:18, 10.37s/it]
 97%|█████████▋| 219098/225000 [427:49:25<16:15:15,  9.91s/it]
 97%|█████████▋| 219099/225000 [427:49:32<14:47:50,  9.03s/it]
 97%|█████████▋| 219100/225000 [427:49:46<17:07:04, 10.44s/it]
                                                              
{'loss': '1.334', 'grad_norm': '1.448', 'learning_rate': '9.611e-08', 'epoch': '0.9738', 'num_input_tokens_seen': 728249160, 'train_runtime': '1.54e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 219100/225000 [427:49:46<17:07:04, 10.44s/it]
 97%|█████████▋| 219101/225000 [427:49:52<15:13:02,  9.29s/it]
 97%|█████████▋| 219102/225000 [427:50:02<15:35:19,  9.51s/it]
 97%|█████████▋| 219103/225000 [427:50:11<15:06:26,  9.22s/it]
 97%|█████████▋| 219104/225000 [427:50:16<13:13:07,  8.07s/it]
 97%|█████████▋| 219105/225000 [427:50:24<13:11:58,  8.06s/it]
 97%|█████████▋| 219106/225000 [427:50:33<13:41:48,  8.37s/it]
 97%|█████████▋| 219107/225000 [427:50:43<14:10:55,  8.66s/it]
 97%|█████████▋| 219108/225000 [427:50:53<14:45:38,  9.02s/it]
 97%|█████████▋| 219109/225000 [427:51:02<14:50:05,  9.07s/it]
 97%|█████████▋| 219110/225000 [427:51:10<14:31:19,  8.88s/it]
 97%|█████████▋| 219111/225000 [427:51:15<12:26:37,  7.61s/it]
 97%|█████████▋| 219112/225000 [427:51:22<12:00:55,  7.35s/it]
 97%|█████████▋| 219113/225000 [427:51:33<14:08:52,  8.65s/it]
 97%|█████████▋| 219114/225000 [427:51:39<12:29:44,  7.64s/it]
 97%|█████████▋| 219115/225000 [427:51:44<11:31:00,  7.05s/it]
 97%|█████████▋| 219116/225000 [427:51:52<11:45:10,  7.19s/it]
 97%|█████████▋| 219117/225000 [427:51:59<11:42:48,  7.17s/it]
 97%|█████████▋| 219118/225000 [427:52:09<13:14:47,  8.11s/it]
 97%|█████████▋| 219119/225000 [427:52:15<11:57:15,  7.32s/it]
 97%|█████████▋| 219120/225000 [427:52:29<15:34:50,  9.54s/it]
 97%|█████████▋| 219121/225000 [427:52:36<13:59:56,  8.57s/it]
 97%|█████████▋| 219122/225000 [427:52:47<15:21:49,  9.41s/it]
 97%|█████████▋| 219123/225000 [427:53:00<17:17:31, 10.59s/it]
 97%|█████████▋| 219124/225000 [427:53:07<15:06:48,  9.26s/it]
 97%|█████████▋| 219125/225000 [427:53:14<14:18:59,  8.77s/it]
 97%|█████████▋| 219126/225000 [427:53:20<12:57:46,  7.94s/it]
 97%|█████████▋| 219127/225000 [427:53:31<14:16:43,  8.75s/it]
 97%|█████████▋| 219128/225000 [427:53:38<13:15:20,  8.13s/it]
 97%|█████████▋| 219129/225000 [427:53:44<12:36:20,  7.73s/it]
 97%|█████████▋| 219130/225000 [427:54:02<17:24:42, 10.68s/it]
 97%|█████████▋| 219131/225000 [427:54:12<16:55:04, 10.38s/it]
 97%|█████████▋| 219132/225000 [427:54:19<15:29:51,  9.51s/it]
 97%|█████████▋| 219133/225000 [427:54:27<14:37:55,  8.98s/it]
 97%|█████████▋| 219134/225000 [427:54:40<16:49:38, 10.33s/it]
 97%|█████████▋| 219135/225000 [427:54:47<15:07:03,  9.28s/it]
 97%|█████████▋| 219136/225000 [427:54:59<16:14:45,  9.97s/it]
 97%|█████████▋| 219137/225000 [427:55:07<15:26:06,  9.48s/it]
 97%|█████████▋| 219138/225000 [427:55:18<16:13:57,  9.97s/it]
 97%|█████████▋| 219139/225000 [427:55:29<16:27:17, 10.11s/it]
 97%|█████████▋| 219140/225000 [427:55:40<16:54:27, 10.39s/it]
 97%|█████████▋| 219141/225000 [427:55:46<14:51:58,  9.13s/it]
 97%|█████████▋| 219142/225000 [427:55:57<15:49:40,  9.73s/it]
 97%|█████████▋| 219143/225000 [427:56:07<16:06:42,  9.90s/it]
 97%|█████████▋| 219144/225000 [427:56:12<13:43:16,  8.44s/it]
 97%|█████████▋| 219145/225000 [427:56:24<15:07:07,  9.30s/it]
 97%|█████████▋| 219146/225000 [427:56:30<13:36:08,  8.36s/it]
 97%|█████████▋| 219147/225000 [427:56:38<13:27:38,  8.28s/it]
 97%|█████████▋| 219148/225000 [427:56:47<13:47:49,  8.49s/it]
 97%|█████████▋| 219149/225000 [427:56:54<13:03:37,  8.04s/it]
 97%|█████████▋| 219150/225000 [427:57:03<13:34:40,  8.36s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.245', 'learning_rate': '9.452e-08', 'epoch': '0.974', 'num_input_tokens_seen': 728481400, 'train_runtime': '1.541e+06', 'train_tokens_per_second': '472.8'}

 97%|█████████▋| 219150/225000 [427:57:03<13:34:40,  8.36s/it]
 97%|█████████▋| 219151/225000 [427:57:09<12:30:30,  7.70s/it]
 97%|█████████▋| 219152/225000 [427:57:23<15:30:55,  9.55s/it]
 97%|█████████▋| 219153/225000 [427:57:35<16:55:46, 10.42s/it]
 97%|█████████▋| 219154/225000 [427:57:44<15:48:24,  9.73s/it]
 97%|█████████▋| 219155/225000 [427:57:52<15:19:08,  9.44s/it]
 97%|█████████▋| 219156/225000 [427:58:04<16:17:29, 10.04s/it]
 97%|█████████▋| 219157/225000 [427:58:14<16:21:10, 10.08s/it]
 97%|█████████▋| 219158/225000 [427:58:21<14:48:16,  9.12s/it]
 97%|█████████▋| 219159/225000 [427:58:31<15:27:25,  9.53s/it]
 97%|█████████▋| 219160/225000 [427:58:43<16:31:40, 10.19s/it]
 97%|█████████▋| 219161/225000 [427:58:51<15:26:28,  9.52s/it]
 97%|█████████▋| 219162/225000 [427:58:59<14:47:09,  9.12s/it]
 97%|█████████▋| 219163/225000 [427:59:08<14:37:37,  9.02s/it]
 97%|█████████▋| 219164/225000 [427:59:17<14:32:50,  8.97s/it]
 97%|█████████▋| 219165/225000 [427:59:22<12:52:04,  7.94s/it]
 97%|█████████▋| 219166/225000 [427:59:34<14:36:33,  9.02s/it]
 97%|█████████▋| 219167/225000 [427:59:47<16:39:45, 10.28s/it]
 97%|█████████▋| 219168/225000 [427:59:55<15:41:43,  9.69s/it]
 97%|█████████▋| 219169/225000 [428:00:02<14:03:13,  8.68s/it]
 97%|█████████▋| 219170/225000 [428:00:09<13:18:18,  8.22s/it]
 97%|█████████▋| 219171/225000 [428:00:13<11:33:56,  7.14s/it]
 97%|█████████▋| 219172/225000 [428:00:20<11:15:46,  6.96s/it]
 97%|█████████▋| 219173/225000 [428:00:27<11:20:48,  7.01s/it]
 97%|█████████▋| 219174/225000 [428:00:33<10:56:08,  6.76s/it]
 97%|█████████▋| 219175/225000 [428:00:48<14:38:34,  9.05s/it]
 97%|█████████▋| 219176/225000 [428:00:57<14:36:59,  9.03s/it]
 97%|█████████▋| 219177/225000 [428:01:09<15:57:16,  9.86s/it]
 97%|█████████▋| 219178/225000 [428:01:16<14:44:45,  9.12s/it]
 97%|█████████▋| 219179/225000 [428:01:24<14:04:47,  8.71s/it]
 97%|█████████▋| 219180/225000 [428:01:30<12:43:14,  7.87s/it]
 97%|█████████▋| 219181/225000 [428:01:36<12:02:01,  7.44s/it]
 97%|█████████▋| 219182/225000 [428:01:42<11:26:49,  7.08s/it]
 97%|█████████▋| 219183/225000 [428:01:56<14:43:08,  9.11s/it]
 97%|█████████▋| 219184/225000 [428:02:09<16:20:54, 10.12s/it]
 97%|█████████▋| 219185/225000 [428:02:13<13:45:50,  8.52s/it]
 97%|█████████▋| 219186/225000 [428:02:23<14:13:43,  8.81s/it]
 97%|█████████▋| 219187/225000 [428:02:33<14:42:46,  9.11s/it]
 97%|█████████▋| 219188/225000 [428:02:43<15:22:45,  9.53s/it]
 97%|█████████▋| 219189/225000 [428:02:53<15:27:25,  9.58s/it]
 97%|█████████▋| 219190/225000 [428:03:03<15:43:17,  9.74s/it]
 97%|█████████▋| 219191/225000 [428:03:09<14:04:16,  8.72s/it]
 97%|█████████▋| 219192/225000 [428:03:16<13:09:20,  8.15s/it]
 97%|█████████▋| 219193/225000 [428:03:27<14:36:50,  9.06s/it]
 97%|█████████▋| 219194/225000 [428:03:37<14:49:00,  9.19s/it]
 97%|█████████▋| 219195/225000 [428:03:48<15:36:16,  9.68s/it]
 97%|█████████▋| 219196/225000 [428:03:53<13:25:48,  8.33s/it]
 97%|█████████▋| 219197/225000 [428:04:00<12:38:58,  7.85s/it]
 97%|█████████▋| 219198/225000 [428:04:08<12:45:50,  7.92s/it]
 97%|█████████▋| 219199/225000 [428:04:17<13:22:12,  8.30s/it]
 97%|█████████▋| 219200/225000 [428:04:29<15:04:24,  9.36s/it]
                                                              
{'loss': '1.332', 'grad_norm': '1.772', 'learning_rate': '9.294e-08', 'epoch': '0.9742', 'num_input_tokens_seen': 728716304, 'train_runtime': '1.541e+06', 'train_tokens_per_second': '472.9'}

 97%|█████████▋| 219200/225000 [428:04:29<15:04:24,  9.36s/it]
 97%|█████████▋| 219201/225000 [428:04:38<15:02:43,  9.34s/it]
 97%|█████████▋| 219202/225000 [428:04:44<13:24:27,  8.32s/it]
 97%|█████████▋| 219203/225000 [428:04:52<13:10:56,  8.19s/it]
 97%|█████████▋| 219204/225000 [428:05:02<14:00:31,  8.70s/it]
 97%|█████████▋| 219205/225000 [428:05:09<13:21:55,  8.30s/it]
 97%|█████████▋| 219206/225000 [428:05:15<12:29:13,  7.76s/it]
 97%|█████████▋| 219207/225000 [428:05:22<11:53:05,  7.39s/it]
 97%|█████████▋| 219208/225000 [428:05:38<16:14:16, 10.09s/it]
 97%|█████████▋| 219209/225000 [428:05:44<13:58:18,  8.69s/it]
 97%|█████████▋| 219210/225000 [428:05:54<14:41:07,  9.13s/it]
 97%|█████████▋| 219211/225000 [428:06:02<14:09:05,  8.80s/it]
 97%|█████████▋| 219212/225000 [428:06:11<14:05:24,  8.76s/it]
 97%|█████████▋| 219213/225000 [428:06:17<12:49:58,  7.98s/it]
 97%|█████████▋| 219214/225000 [428:06:26<13:34:24,  8.45s/it]
 97%|█████████▋| 219215/225000 [428:06:31<11:36:54,  7.23s/it]
 97%|█████████▋| 219216/225000 [428:06:48<16:15:49, 10.12s/it]
 97%|█████████▋| 219217/225000 [428:06:56<15:17:16,  9.52s/it]
 97%|█████████▋| 219218/225000 [428:07:03<14:16:07,  8.88s/it]
 97%|█████████▋| 219219/225000 [428:07:16<16:22:26, 10.20s/it]
 97%|█████████▋| 219220/225000 [428:07:23<14:42:46,  9.16s/it]
 97%|█████████▋| 219221/225000 [428:07:34<15:39:55,  9.76s/it]
 97%|█████████▋| 219222/225000 [428:07:48<17:21:55, 10.82s/it]
 97%|█████████▋| 219223/225000 [428:07:56<16:13:24, 10.11s/it]
 97%|█████████▋| 219224/225000 [428:08:04<15:02:47,  9.38s/it]
 97%|█████████▋| 219225/225000 [428:08:10<13:18:22,  8.29s/it]
 97%|█████████▋| 219226/225000 [428:08:20<14:08:37,  8.82s/it]
 97%|█████████▋| 219227/225000 [428:08:26<13:08:06,  8.19s/it]
 97%|█████████▋| 219228/225000 [428:08:33<12:25:35,  7.75s/it]
 97%|█████████▋| 219229/225000 [428:08:44<13:59:24,  8.73s/it]
 97%|█████████▋| 219230/225000 [428:08:50<12:42:41,  7.93s/it]
 97%|█████████▋| 219231/225000 [428:09:00<13:47:58,  8.61s/it]
 97%|█████████▋| 219232/225000 [428:09:09<13:54:03,  8.68s/it]
 97%|█████████▋| 219233/225000 [428:09:19<14:30:55,  9.06s/it]
 97%|█████████▋| 219234/225000 [428:09:28<14:21:57,  8.97s/it]
 97%|█████████▋| 219235/225000 [428:09:42<16:59:27, 10.61s/it]
 97%|█████████▋| 219236/225000 [428:09:49<15:01:08,  9.38s/it]
 97%|█████████▋| 219237/225000 [428:09:59<15:37:20,  9.76s/it]
 97%|█████████▋| 219238/225000 [428:10:12<16:51:37, 10.53s/it]
 97%|█████████▋| 219239/225000 [428:10:19<15:22:32,  9.61s/it]
 97%|█████████▋| 219240/225000 [428:10:27<14:35:31,  9.12s/it]
 97%|█████████▋| 219241/225000 [428:10:35<14:04:46,  8.80s/it]
 97%|█████████▋| 219242/225000 [428:10:41<12:29:29,  7.81s/it]
 97%|█████████▋| 219243/225000 [428:10:48<12:07:49,  7.59s/it]
 97%|█████████▋| 219244/225000 [428:11:01<14:50:50,  9.29s/it]
 97%|█████████▋| 219245/225000 [428:11:12<15:27:07,  9.67s/it]
 97%|█████████▋| 219246/225000 [428:11:20<14:42:03,  9.20s/it]
 97%|█████████▋| 219247/225000 [428:11:27<13:49:02,  8.65s/it]
 97%|█████████▋| 219248/225000 [428:11:37<14:31:24,  9.09s/it]
 97%|█████████▋| 219249/225000 [428:11:44<13:37:44,  8.53s/it]
 97%|█████████▋| 219250/225000 [428:11:54<14:09:52,  8.87s/it]
                                                              
{'loss': '1.288', 'grad_norm': '1.167', 'learning_rate': '9.137e-08', 'epoch': '0.9744', 'num_input_tokens_seen': 728952496, 'train_runtime': '1.542e+06', 'train_tokens_per_second': '472.9'}

 97%|█████████▋| 219250/225000 [428:11:54<14:09:52,  8.87s/it]
 97%|█████████▋| 219251/225000 [428:12:01<13:22:03,  8.37s/it]
 97%|█████████▋| 219252/225000 [428:12:09<13:02:06,  8.16s/it]
 97%|█████████▋| 219253/225000 [428:12:17<12:58:50,  8.13s/it]
 97%|█████████▋| 219254/225000 [428:12:29<14:51:39,  9.31s/it]
 97%|█████████▋| 219255/225000 [428:12:36<13:41:43,  8.58s/it]
 97%|█████████▋| 219256/225000 [428:12:41<11:59:14,  7.51s/it]
 97%|█████████▋| 219257/225000 [428:12:50<12:39:26,  7.93s/it]
 97%|█████████▋| 219258/225000 [428:13:03<15:13:47,  9.55s/it]
 97%|█████████▋| 219259/225000 [428:13:09<13:37:04,  8.54s/it]
 97%|█████████▋| 219260/225000 [428:13:15<12:08:40,  7.62s/it]
 97%|█████████▋| 219261/225000 [428:13:26<13:53:20,  8.71s/it]
 97%|█████████▋| 219262/225000 [428:13:33<13:01:39,  8.17s/it]
 97%|█████████▋| 219263/225000 [428:13:43<14:04:56,  8.84s/it]
 97%|█████████▋| 219264/225000 [428:13:51<13:26:38,  8.44s/it]
 97%|█████████▋| 219265/225000 [428:13:56<11:39:54,  7.32s/it]
 97%|█████████▋| 219266/225000 [428:14:07<13:27:18,  8.45s/it]
 97%|█████████▋| 219267/225000 [428:14:18<14:39:11,  9.20s/it]
 97%|█████████▋| 219268/225000 [428:14:26<14:05:08,  8.85s/it]
 97%|█████████▋| 219269/225000 [428:14:32<12:59:28,  8.16s/it]
 97%|█████████▋| 219270/225000 [428:14:38<12:01:17,  7.55s/it]
 97%|█████████▋| 219271/225000 [428:14:48<12:52:47,  8.09s/it]
 97%|█████████▋| 219272/225000 [428:14:57<13:24:32,  8.43s/it]
 97%|█████████▋| 219273/225000 [428:15:05<13:22:44,  8.41s/it]
 97%|█████████▋| 219274/225000 [428:15:19<15:46:26,  9.92s/it]
 97%|█████████▋| 219275/225000 [428:15:28<15:31:24,  9.76s/it]
 97%|█████████▋| 219276/225000 [428:15:35<14:02:15,  8.83s/it]
 97%|█████████▋| 219277/225000 [428:15:45<14:46:05,  9.29s/it]
 97%|█████████▋| 219278/225000 [428:15:54<14:28:39,  9.11s/it]
 97%|█████████▋| 219279/225000 [428:16:02<14:07:26,  8.89s/it]
 97%|█████████▋| 219280/225000 [428:16:12<14:24:55,  9.07s/it]
 97%|█████████▋| 219281/225000 [428:16:20<13:55:59,  8.77s/it]
 97%|█████████▋| 219282/225000 [428:16:28<13:32:27,  8.53s/it]
 97%|█████████▋| 219283/225000 [428:16:36<13:17:47,  8.37s/it]
 97%|█████████▋| 219284/225000 [428:16:44<13:02:46,  8.22s/it]
 97%|█████████▋| 219285/225000 [428:16:50<12:06:50,  7.63s/it]
 97%|█████████▋| 219286/225000 [428:17:02<14:04:57,  8.87s/it]
 97%|█████████▋| 219287/225000 [428:17:08<12:57:11,  8.16s/it]
 97%|█████████▋| 219288/225000 [428:17:19<14:12:20,  8.95s/it]
 97%|█████████▋| 219289/225000 [428:17:26<13:19:36,  8.40s/it]
 97%|█████████▋| 219290/225000 [428:17:36<14:11:13,  8.94s/it]
 97%|█████████▋| 219291/225000 [428:17:46<14:43:23,  9.28s/it]
 97%|█████████▋| 219292/225000 [428:17:50<12:13:03,  7.71s/it]
 97%|█████████▋| 219293/225000 [428:18:00<13:10:31,  8.31s/it]
 97%|█████████▋| 219294/225000 [428:18:07<12:21:39,  7.80s/it]
 97%|█████████▋| 219295/225000 [428:18:19<14:29:44,  9.15s/it]
 97%|█████████▋| 219296/225000 [428:18:28<14:15:09,  9.00s/it]
 97%|█████████▋| 219297/225000 [428:18:34<13:01:26,  8.22s/it]
 97%|█████████▋| 219298/225000 [428:18:39<11:34:39,  7.31s/it]
 97%|█████████▋| 219299/225000 [428:18:48<12:19:21,  7.78s/it]
 97%|█████████▋| 219300/225000 [428:19:01<14:56:33,  9.44s/it]
                                                              
{'loss': '1.319', 'grad_norm': '1.186', 'learning_rate': '8.982e-08', 'epoch': '0.9747', 'num_input_tokens_seen': 729179064, 'train_runtime': '1.542e+06', 'train_tokens_per_second': '472.9'}

 97%|█████████▋| 219300/225000 [428:19:01<14:56:33,  9.44s/it]
 97%|█████████▋| 219301/225000 [428:19:14<16:28:23, 10.41s/it]
 97%|█████████▋| 219302/225000 [428:19:20<14:13:41,  8.99s/it]
 97%|█████████▋| 219303/225000 [428:19:27<13:23:35,  8.46s/it]
 97%|█████████▋| 219304/225000 [428:19:37<14:01:58,  8.87s/it]
 97%|█████████▋| 219305/225000 [428:19:45<13:40:48,  8.65s/it]
 97%|█████████▋| 219306/225000 [428:19:49<11:22:45,  7.19s/it]
 97%|█████████▋| 219307/225000 [428:20:00<13:24:28,  8.48s/it]
 97%|█████████▋| 219308/225000 [428:20:11<14:15:27,  9.02s/it]
 97%|█████████▋| 219309/225000 [428:20:16<12:40:41,  8.02s/it]
 97%|█████████▋| 219310/225000 [428:20:23<12:07:34,  7.67s/it]
 97%|█████████▋| 219311/225000 [428:20:32<12:43:32,  8.05s/it]
 97%|█████████▋| 219312/225000 [428:20:40<12:47:48,  8.10s/it]
 97%|█████████▋| 219313/225000 [428:20:53<15:09:47,  9.60s/it]
 97%|█████████▋| 219314/225000 [428:21:01<14:26:38,  9.14s/it]
 97%|█████████▋| 219315/225000 [428:21:13<15:29:21,  9.81s/it]
 97%|█████████▋| 219316/225000 [428:21:19<14:01:00,  8.88s/it]
 97%|█████████▋| 219317/225000 [428:21:25<12:29:19,  7.91s/it]
 97%|█████████▋| 219318/225000 [428:21:35<13:37:06,  8.63s/it]
 97%|█████████▋| 219319/225000 [428:21:47<15:03:19,  9.54s/it]
 97%|█████████▋| 219320/225000 [428:21:53<13:10:30,  8.35s/it]
 97%|█████████▋| 219321/225000 [428:22:06<15:42:03,  9.95s/it]
 97%|█████████▋| 219322/225000 [428:22:17<15:51:28, 10.05s/it]
 97%|█████████▋| 219323/225000 [428:22:26<15:23:52,  9.76s/it]
 97%|█████████▋| 219324/225000 [428:22:33<14:02:01,  8.90s/it]
 97%|█████████▋| 219325/225000 [428:22:43<14:56:56,  9.48s/it]
 97%|█████████▋| 219326/225000 [428:22:51<14:13:22,  9.02s/it]
 97%|█████████▋| 219327/225000 [428:23:03<15:24:14,  9.78s/it]
 97%|█████████▋| 219328/225000 [428:23:13<15:43:34,  9.98s/it]
 97%|█████████▋| 219329/225000 [428:23:20<13:54:59,  8.83s/it]
 97%|█████████▋| 219330/225000 [428:23:25<12:06:14,  7.69s/it]
 97%|█████████▋| 219331/225000 [428:23:34<12:46:47,  8.12s/it]
 97%|█████████▋| 219332/225000 [428:23:48<15:39:23,  9.94s/it]
 97%|█████████▋| 219333/225000 [428:24:01<17:13:04, 10.94s/it]
 97%|█████████▋| 219334/225000 [428:24:10<16:05:55, 10.23s/it]
 97%|█████████▋| 219335/225000 [428:24:17<14:51:07,  9.44s/it]
 97%|█████████▋| 219336/225000 [428:24:24<13:38:05,  8.67s/it]
 97%|█████████▋| 219337/225000 [428:24:33<13:46:14,  8.75s/it]
 97%|█████████▋| 219338/225000 [428:24:41<13:16:21,  8.44s/it]
 97%|█████████▋| 219339/225000 [428:24:48<12:37:26,  8.03s/it]
 97%|█████████▋| 219340/225000 [428:24:55<12:03:57,  7.67s/it]
 97%|█████████▋| 219341/225000 [428:25:07<14:17:34,  9.09s/it]
 97%|█████████▋| 219342/225000 [428:25:12<12:13:14,  7.78s/it]
 97%|█████████▋| 219343/225000 [428:25:23<13:41:06,  8.71s/it]
 97%|█████████▋| 219344/225000 [428:25:29<12:32:39,  7.98s/it]
 97%|█████████▋| 219345/225000 [428:25:36<11:55:10,  7.59s/it]
 97%|█████████▋| 219346/225000 [428:25:49<14:33:51,  9.27s/it]
 97%|█████████▋| 219347/225000 [428:25:55<12:58:30,  8.26s/it]
 97%|█████████▋| 219348/225000 [428:26:03<12:45:46,  8.13s/it]
 97%|█████████▋| 219349/225000 [428:26:16<15:18:08,  9.75s/it]
 97%|█████████▋| 219350/225000 [428:26:22<13:26:54,  8.57s/it]
                                                              
{'loss': '1.337', 'grad_norm': '1.502', 'learning_rate': '8.828e-08', 'epoch': '0.9749', 'num_input_tokens_seen': 729410984, 'train_runtime': '1.542e+06', 'train_tokens_per_second': '472.9'}

 97%|█████████▋| 219350/225000 [428:26:22<13:26:54,  8.57s/it]
 97%|█████████▋| 219351/225000 [428:26:33<14:23:06,  9.17s/it]
 97%|█████████▋| 219352/225000 [428:26:39<12:55:40,  8.24s/it]
 97%|█████████▋| 219353/225000 [428:26:49<14:02:36,  8.95s/it]
 97%|█████████▋| 219354/225000 [428:27:01<15:21:26,  9.79s/it]
 97%|█████████▋| 219355/225000 [428:27:08<13:49:29,  8.82s/it]
 97%|█████████▋| 219356/225000 [428:27:16<13:32:53,  8.64s/it]
 97%|█████████▋| 219357/225000 [428:27:26<14:26:53,  9.22s/it]
 97%|█████████▋| 219358/225000 [428:27:31<12:16:53,  7.84s/it]
 97%|█████████▋| 219359/225000 [428:27:40<12:59:30,  8.29s/it]
 97%|█████████▋| 219360/225000 [428:27:51<14:16:33,  9.11s/it]
 97%|█████████▋| 219361/225000 [428:27:59<13:21:43,  8.53s/it]
 97%|█████████▋| 219362/225000 [428:28:04<12:08:34,  7.75s/it]
 97%|█████████▋| 219363/225000 [428:28:16<14:06:53,  9.01s/it]
 97%|█████████▋| 219364/225000 [428:28:22<12:22:42,  7.91s/it]
 97%|█████████▋| 219365/225000 [428:28:34<14:12:35,  9.08s/it]
 97%|█████████▋| 219366/225000 [428:28:42<13:48:51,  8.83s/it]
 97%|█████████▋| 219367/225000 [428:28:47<12:04:50,  7.72s/it]
 97%|█████████▋| 219368/225000 [428:28:54<11:48:52,  7.55s/it]
 97%|█████████▋| 219369/225000 [428:29:02<11:49:35,  7.56s/it]
 97%|█████████▋| 219370/225000 [428:29:13<13:41:07,  8.75s/it]
 97%|█████████▋| 219371/225000 [428:29:26<15:45:58, 10.08s/it]
 97%|█████████▋| 219372/225000 [428:29:34<14:24:38,  9.22s/it]
 97%|█████████▋| 219373/225000 [428:29:43<14:35:09,  9.33s/it]
 97%|█████████▋| 219374/225000 [428:29:48<12:38:43,  8.09s/it]
 98%|█████████▊| 219375/225000 [428:29:59<13:57:03,  8.93s/it]
 98%|█████████▊| 219376/225000 [428:30:06<12:58:51,  8.31s/it]
 98%|█████████▊| 219377/225000 [428:30:19<15:04:57,  9.66s/it]
 98%|█████████▊| 219378/225000 [428:30:29<15:14:23,  9.76s/it]
 98%|█████████▊| 219379/225000 [428:30:37<14:31:19,  9.30s/it]
 98%|█████████▊| 219380/225000 [428:30:44<13:25:16,  8.60s/it]
 98%|█████████▊| 219381/225000 [428:30:52<13:06:54,  8.40s/it]
 98%|█████████▊| 219382/225000 [428:31:01<13:17:25,  8.52s/it]
 98%|█████████▊| 219383/225000 [428:31:09<13:19:56,  8.54s/it]
 98%|█████████▊| 219384/225000 [428:31:20<14:23:31,  9.23s/it]
 98%|█████████▊| 219385/225000 [428:31:30<14:44:38,  9.45s/it]
 98%|█████████▊| 219386/225000 [428:31:37<13:24:01,  8.59s/it]
 98%|█████████▊| 219387/225000 [428:31:45<12:58:52,  8.33s/it]
 98%|█████████▊| 219388/225000 [428:31:51<12:00:33,  7.70s/it]
 98%|█████████▊| 219389/225000 [428:31:59<12:03:08,  7.73s/it]
 98%|█████████▊| 219390/225000 [428:32:15<15:52:37, 10.19s/it]
 98%|█████████▊| 219391/225000 [428:32:19<13:22:13,  8.58s/it]
 98%|█████████▊| 219392/225000 [428:32:33<15:32:59,  9.98s/it]
 98%|█████████▊| 219393/225000 [428:32:43<15:50:25, 10.17s/it]
 98%|█████████▊| 219394/225000 [428:32:48<13:30:23,  8.67s/it]
 98%|█████████▊| 219395/225000 [428:33:00<15:00:25,  9.64s/it]
 98%|█████████▊| 219396/225000 [428:33:07<13:27:52,  8.65s/it]
 98%|█████████▊| 219397/225000 [428:33:14<12:48:07,  8.23s/it]
 98%|█████████▊| 219398/225000 [428:33:24<13:41:30,  8.80s/it]
 98%|█████████▊| 219399/225000 [428:33:31<12:53:52,  8.29s/it]
 98%|█████████▊| 219400/225000 [428:33:37<11:51:42,  7.63s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.41', 'learning_rate': '8.675e-08', 'epoch': '0.9751', 'num_input_tokens_seen': 729641816, 'train_runtime': '1.543e+06', 'train_tokens_per_second': '472.9'}

 98%|█████████▊| 219400/225000 [428:33:37<11:51:42,  7.63s/it]
 98%|█████████▊| 219401/225000 [428:33:49<13:45:17,  8.84s/it]
 98%|█████████▊| 219402/225000 [428:33:56<12:56:07,  8.32s/it]
 98%|█████████▊| 219403/225000 [428:34:03<12:26:20,  8.00s/it]
 98%|█████████▊| 219404/225000 [428:34:11<12:07:53,  7.80s/it]
 98%|█████████▊| 219405/225000 [428:34:17<11:42:10,  7.53s/it]
 98%|█████████▊| 219406/225000 [428:34:29<13:41:01,  8.81s/it]
 98%|█████████▊| 219407/225000 [428:34:37<13:12:19,  8.50s/it]
 98%|█████████▊| 219408/225000 [428:34:47<13:45:51,  8.86s/it]
 98%|█████████▊| 219409/225000 [428:34:58<14:49:10,  9.54s/it]
 98%|█████████▊| 219410/225000 [428:35:03<12:47:12,  8.23s/it]
 98%|█████████▊| 219411/225000 [428:35:16<14:45:56,  9.51s/it]
 98%|█████████▊| 219412/225000 [428:35:22<13:15:20,  8.54s/it]
 98%|█████████▊| 219413/225000 [428:35:31<13:28:40,  8.68s/it]
 98%|█████████▊| 219414/225000 [428:35:38<12:54:14,  8.32s/it]
 98%|█████████▊| 219415/225000 [428:35:48<13:44:33,  8.86s/it]
 98%|█████████▊| 219416/225000 [428:35:54<12:24:42,  8.00s/it]
 98%|█████████▊| 219417/225000 [428:36:04<13:06:55,  8.46s/it]
 98%|█████████▊| 219418/225000 [428:36:16<15:01:16,  9.69s/it]
 98%|█████████▊| 219419/225000 [428:36:25<14:42:28,  9.49s/it]
 98%|█████████▊| 219420/225000 [428:36:32<13:24:47,  8.65s/it]
 98%|█████████▊| 219421/225000 [428:36:40<12:49:31,  8.28s/it]
 98%|█████████▊| 219422/225000 [428:36:50<13:50:23,  8.93s/it]
 98%|█████████▊| 219423/225000 [428:37:01<14:39:17,  9.46s/it]
 98%|█████████▊| 219424/225000 [428:37:10<14:36:08,  9.43s/it]
 98%|█████████▊| 219425/225000 [428:37:20<14:53:30,  9.62s/it]
 98%|█████████▊| 219426/225000 [428:37:26<13:06:44,  8.47s/it]
 98%|█████████▊| 219427/225000 [428:37:37<14:20:49,  9.27s/it]
 98%|█████████▊| 219428/225000 [428:37:48<14:57:08,  9.66s/it]
 98%|█████████▊| 219429/225000 [428:37:56<14:18:54,  9.25s/it]
 98%|█████████▊| 219430/225000 [428:38:05<14:02:08,  9.07s/it]
 98%|█████████▊| 219431/225000 [428:38:12<13:08:24,  8.49s/it]
 98%|█████████▊| 219432/225000 [428:38:25<15:16:54,  9.88s/it]
 98%|█████████▊| 219433/225000 [428:38:30<13:04:00,  8.45s/it]
 98%|█████████▊| 219434/225000 [428:38:40<13:33:57,  8.77s/it]
 98%|█████████▊| 219435/225000 [428:38:46<12:42:26,  8.22s/it]
 98%|█████████▊| 219436/225000 [428:38:55<12:38:24,  8.18s/it]
 98%|█████████▊| 219437/225000 [428:39:01<11:58:32,  7.75s/it]
 98%|█████████▊| 219438/225000 [428:39:09<11:57:02,  7.74s/it]
 98%|█████████▊| 219439/225000 [428:39:17<12:03:37,  7.81s/it]
 98%|█████████▊| 219440/225000 [428:39:24<11:32:10,  7.47s/it]
 98%|█████████▊| 219441/225000 [428:39:30<10:54:00,  7.06s/it]
 98%|█████████▊| 219442/225000 [428:39:39<11:58:22,  7.75s/it]
 98%|█████████▊| 219443/225000 [428:39:47<12:00:46,  7.78s/it]
 98%|█████████▊| 219444/225000 [428:39:54<11:50:15,  7.67s/it]
 98%|█████████▊| 219445/225000 [428:40:01<11:31:41,  7.47s/it]
 98%|█████████▊| 219446/225000 [428:40:09<11:29:24,  7.45s/it]
 98%|█████████▊| 219447/225000 [428:40:20<13:10:45,  8.54s/it]
 98%|█████████▊| 219448/225000 [428:40:26<12:03:48,  7.82s/it]
 98%|█████████▊| 219449/225000 [428:40:36<13:15:45,  8.60s/it]
 98%|█████████▊| 219450/225000 [428:40:47<14:03:39,  9.12s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.461', 'learning_rate': '8.524e-08', 'epoch': '0.9753', 'num_input_tokens_seen': 729865776, 'train_runtime': '1.543e+06', 'train_tokens_per_second': '472.9'}

 98%|█████████▊| 219450/225000 [428:40:47<14:03:39,  9.12s/it]
 98%|█████████▊| 219451/225000 [428:40:58<15:04:18,  9.78s/it]
 98%|█████████▊| 219452/225000 [428:41:06<14:24:10,  9.35s/it]
 98%|█████████▊| 219453/225000 [428:41:12<12:31:27,  8.13s/it]
 98%|█████████▊| 219454/225000 [428:41:24<14:23:30,  9.34s/it]
 98%|█████████▊| 219455/225000 [428:41:31<13:28:14,  8.75s/it]
 98%|█████████▊| 219456/225000 [428:41:40<13:27:39,  8.74s/it]
 98%|█████████▊| 219457/225000 [428:41:50<13:51:49,  9.00s/it]
 98%|█████████▊| 219458/225000 [428:41:57<13:06:21,  8.51s/it]
 98%|█████████▊| 219459/225000 [428:42:09<14:31:36,  9.44s/it]
 98%|█████████▊| 219460/225000 [428:42:15<13:20:59,  8.68s/it]
 98%|█████████▊| 219461/225000 [428:42:22<12:27:44,  8.10s/it]
 98%|█████████▊| 219462/225000 [428:42:29<11:39:55,  7.58s/it]
 98%|█████████▊| 219463/225000 [428:42:46<16:09:02, 10.50s/it]
 98%|█████████▊| 219464/225000 [428:42:55<15:37:16, 10.16s/it]
 98%|█████████▊| 219465/225000 [428:43:04<15:04:23,  9.80s/it]
 98%|█████████▊| 219466/225000 [428:43:11<13:40:27,  8.90s/it]
 98%|█████████▊| 219467/225000 [428:43:24<15:34:20, 10.13s/it]
 98%|█████████▊| 219468/225000 [428:43:31<14:14:48,  9.27s/it]
 98%|█████████▊| 219469/225000 [428:43:39<13:43:41,  8.94s/it]
 98%|█████████▊| 219470/225000 [428:43:45<12:24:21,  8.08s/it]
 98%|█████████▊| 219471/225000 [428:43:55<13:17:19,  8.65s/it]
 98%|█████████▊| 219472/225000 [428:44:04<13:05:10,  8.52s/it]
 98%|█████████▊| 219473/225000 [428:44:10<12:17:25,  8.01s/it]
 98%|█████████▊| 219474/225000 [428:44:24<14:53:34,  9.70s/it]
 98%|█████████▊| 219475/225000 [428:44:34<15:04:09,  9.82s/it]
 98%|█████████▊| 219476/225000 [428:44:40<13:18:16,  8.67s/it]
 98%|█████████▊| 219477/225000 [428:44:49<13:07:33,  8.56s/it]
 98%|█████████▊| 219478/225000 [428:44:56<12:28:53,  8.14s/it]
 98%|█████████▊| 219479/225000 [428:45:08<14:24:49,  9.40s/it]
 98%|█████████▊| 219480/225000 [428:45:14<13:00:12,  8.48s/it]
 98%|█████████▊| 219481/225000 [428:45:25<14:00:42,  9.14s/it]
 98%|█████████▊| 219482/225000 [428:45:32<13:09:57,  8.59s/it]
 98%|█████████▊| 219483/225000 [428:45:38<11:59:49,  7.83s/it]
 98%|█████████▊| 219484/225000 [428:45:47<12:26:49,  8.12s/it]
 98%|█████████▊| 219485/225000 [428:45:56<12:44:10,  8.31s/it]
 98%|█████████▊| 219486/225000 [428:46:03<11:56:30,  7.80s/it]
 98%|█████████▊| 219487/225000 [428:46:13<13:08:01,  8.58s/it]
 98%|█████████▊| 219488/225000 [428:46:24<14:09:18,  9.25s/it]
 98%|█████████▊| 219489/225000 [428:46:35<15:12:39,  9.94s/it]
 98%|█████████▊| 219490/225000 [428:46:42<13:45:31,  8.99s/it]
 98%|█████████▊| 219491/225000 [428:46:52<13:58:14,  9.13s/it]
 98%|█████████▊| 219492/225000 [428:46:58<12:41:23,  8.29s/it]
 98%|█████████▊| 219493/225000 [428:47:08<13:30:36,  8.83s/it]
 98%|█████████▊| 219494/225000 [428:47:17<13:35:47,  8.89s/it]
 98%|█████████▊| 219495/225000 [428:47:28<14:41:28,  9.61s/it]
 98%|█████████▊| 219496/225000 [428:47:33<12:29:55,  8.18s/it]
 98%|█████████▊| 219497/225000 [428:47:43<13:24:47,  8.77s/it]
 98%|█████████▊| 219498/225000 [428:47:55<14:37:02,  9.56s/it]
 98%|█████████▊| 219499/225000 [428:48:00<12:39:45,  8.29s/it]
 98%|█████████▊| 219500/225000 [428:48:09<13:02:14,  8.53s/it]
                                                              
{'loss': '1.332', 'grad_norm': '1.357', 'learning_rate': '8.374e-08', 'epoch': '0.9756', 'num_input_tokens_seen': 730101288, 'train_runtime': '1.544e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219500/225000 [428:48:09<13:02:14,  8.53s/it]
 98%|█████████▊| 219501/225000 [428:48:16<12:27:53,  8.16s/it]
 98%|█████████▊| 219502/225000 [428:48:21<10:51:37,  7.11s/it]
 98%|█████████▊| 219503/225000 [428:48:28<10:58:41,  7.19s/it]
 98%|█████████▊| 219504/225000 [428:48:40<12:57:27,  8.49s/it]
 98%|█████████▊| 219505/225000 [428:48:47<12:04:57,  7.92s/it]
 98%|█████████▊| 219506/225000 [428:48:53<11:28:40,  7.52s/it]
 98%|█████████▊| 219507/225000 [428:49:00<11:22:26,  7.45s/it]
 98%|█████████▊| 219508/225000 [428:49:11<12:37:53,  8.28s/it]
 98%|█████████▊| 219509/225000 [428:49:21<13:24:10,  8.79s/it]
 98%|█████████▊| 219510/225000 [428:49:31<14:06:21,  9.25s/it]
 98%|█████████▊| 219511/225000 [428:49:42<14:43:10,  9.65s/it]
 98%|█████████▊| 219512/225000 [428:49:52<15:02:33,  9.87s/it]
 98%|█████████▊| 219513/225000 [428:49:57<12:37:49,  8.29s/it]
 98%|█████████▊| 219514/225000 [428:50:01<11:03:30,  7.26s/it]
 98%|█████████▊| 219515/225000 [428:50:16<14:18:40,  9.39s/it]
 98%|█████████▊| 219516/225000 [428:50:25<14:13:08,  9.33s/it]
 98%|█████████▊| 219517/225000 [428:50:32<13:14:59,  8.70s/it]
 98%|█████████▊| 219518/225000 [428:50:39<12:29:12,  8.20s/it]
 98%|█████████▊| 219519/225000 [428:50:45<11:36:41,  7.63s/it]
 98%|█████████▊| 219520/225000 [428:51:00<14:50:40,  9.75s/it]
 98%|█████████▊| 219521/225000 [428:51:13<16:14:30, 10.67s/it]
 98%|█████████▊| 219522/225000 [428:51:22<15:41:30, 10.31s/it]
 98%|█████████▊| 219523/225000 [428:51:38<18:09:12, 11.93s/it]
 98%|█████████▊| 219524/225000 [428:51:46<16:09:57, 10.63s/it]
 98%|█████████▊| 219525/225000 [428:51:56<16:10:21, 10.63s/it]
 98%|█████████▊| 219526/225000 [428:52:04<14:42:53,  9.68s/it]
 98%|█████████▊| 219527/225000 [428:52:10<13:03:26,  8.59s/it]
 98%|█████████▊| 219528/225000 [428:52:24<15:45:45, 10.37s/it]
 98%|█████████▊| 219529/225000 [428:52:31<13:56:57,  9.18s/it]
 98%|█████████▊| 219530/225000 [428:52:42<14:54:47,  9.81s/it]
 98%|█████████▊| 219531/225000 [428:52:50<14:06:41,  9.29s/it]
 98%|█████████▊| 219532/225000 [428:52:59<13:54:12,  9.15s/it]
 98%|█████████▊| 219533/225000 [428:53:04<11:53:43,  7.83s/it]
 98%|█████████▊| 219534/225000 [428:53:17<14:18:07,  9.42s/it]
 98%|█████████▊| 219535/225000 [428:53:25<13:45:11,  9.06s/it]
 98%|█████████▊| 219536/225000 [428:53:32<12:49:23,  8.45s/it]
 98%|█████████▊| 219537/225000 [428:53:40<12:37:58,  8.32s/it]
 98%|█████████▊| 219538/225000 [428:53:51<13:56:00,  9.18s/it]
 98%|█████████▊| 219539/225000 [428:54:01<14:19:33,  9.44s/it]
 98%|█████████▊| 219540/225000 [428:54:08<13:11:12,  8.69s/it]
 98%|█████████▊| 219541/225000 [428:54:20<14:26:19,  9.52s/it]
 98%|█████████▊| 219542/225000 [428:54:25<12:14:39,  8.08s/it]
 98%|█████████▊| 219543/225000 [428:54:31<11:27:38,  7.56s/it]
 98%|█████████▊| 219544/225000 [428:54:41<12:37:33,  8.33s/it]
 98%|█████████▊| 219545/225000 [428:54:50<12:45:01,  8.41s/it]
 98%|█████████▊| 219546/225000 [428:54:57<12:08:03,  8.01s/it]
 98%|█████████▊| 219547/225000 [428:55:11<14:50:20,  9.80s/it]
 98%|█████████▊| 219548/225000 [428:55:17<13:06:57,  8.66s/it]
 98%|█████████▊| 219549/225000 [428:55:29<14:47:18,  9.77s/it]
 98%|█████████▊| 219550/225000 [428:55:41<15:39:04, 10.34s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.107', 'learning_rate': '8.225e-08', 'epoch': '0.9758', 'num_input_tokens_seen': 730334256, 'train_runtime': '1.544e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219550/225000 [428:55:41<15:39:04, 10.34s/it]
 98%|█████████▊| 219551/225000 [428:55:47<13:51:22,  9.15s/it]
 98%|█████████▊| 219552/225000 [428:55:58<14:27:39,  9.56s/it]
 98%|█████████▊| 219553/225000 [428:56:05<13:24:20,  8.86s/it]
 98%|█████████▊| 219554/225000 [428:56:14<13:19:58,  8.81s/it]
 98%|█████████▊| 219555/225000 [428:56:18<11:34:45,  7.66s/it]
 98%|█████████▊| 219556/225000 [428:56:23<10:12:41,  6.75s/it]
 98%|█████████▊| 219557/225000 [428:56:33<11:44:28,  7.77s/it]
 98%|█████████▊| 219558/225000 [428:56:44<13:11:32,  8.73s/it]
 98%|█████████▊| 219559/225000 [428:56:55<14:00:52,  9.27s/it]
 98%|█████████▊| 219560/225000 [428:57:01<12:33:25,  8.31s/it]
 98%|█████████▊| 219561/225000 [428:57:09<12:37:22,  8.35s/it]
 98%|█████████▊| 219562/225000 [428:57:17<12:21:00,  8.18s/it]
 98%|█████████▊| 219563/225000 [428:57:27<13:11:01,  8.73s/it]
 98%|█████████▊| 219564/225000 [428:57:34<12:24:28,  8.22s/it]
 98%|█████████▊| 219565/225000 [428:57:42<12:08:57,  8.05s/it]
 98%|█████████▊| 219566/225000 [428:57:53<13:37:23,  9.03s/it]
 98%|█████████▊| 219567/225000 [428:57:58<11:56:31,  7.91s/it]
 98%|█████████▊| 219568/225000 [428:58:08<12:43:01,  8.43s/it]
 98%|█████████▊| 219569/225000 [428:58:17<12:53:19,  8.54s/it]
 98%|█████████▊| 219570/225000 [428:58:27<13:41:30,  9.08s/it]
 98%|█████████▊| 219571/225000 [428:58:35<13:20:53,  8.85s/it]
 98%|█████████▊| 219572/225000 [428:58:42<12:12:31,  8.10s/it]
 98%|█████████▊| 219573/225000 [428:58:49<11:37:48,  7.71s/it]
 98%|█████████▊| 219574/225000 [428:58:55<11:15:05,  7.47s/it]
 98%|█████████▊| 219575/225000 [428:59:01<10:28:56,  6.96s/it]
 98%|█████████▊| 219576/225000 [428:59:13<12:49:01,  8.51s/it]
 98%|█████████▊| 219577/225000 [428:59:26<14:38:18,  9.72s/it]
 98%|█████████▊| 219578/225000 [428:59:36<14:51:30,  9.87s/it]
 98%|█████████▊| 219579/225000 [428:59:41<12:45:43,  8.48s/it]
 98%|█████████▊| 219580/225000 [428:59:49<12:25:31,  8.25s/it]
 98%|█████████▊| 219581/225000 [428:59:55<11:14:32,  7.47s/it]
 98%|█████████▊| 219582/225000 [429:00:05<12:22:13,  8.22s/it]
 98%|█████████▊| 219583/225000 [429:00:19<14:59:54,  9.97s/it]
 98%|█████████▊| 219584/225000 [429:00:24<12:38:23,  8.40s/it]
 98%|█████████▊| 219585/225000 [429:00:34<13:32:11,  9.00s/it]
 98%|█████████▊| 219586/225000 [429:00:43<13:44:18,  9.14s/it]
 98%|█████████▊| 219587/225000 [429:00:49<11:56:46,  7.94s/it]
 98%|█████████▊| 219588/225000 [429:00:55<11:24:36,  7.59s/it]
 98%|█████████▊| 219589/225000 [429:01:06<12:42:29,  8.45s/it]
 98%|█████████▊| 219590/225000 [429:01:19<14:53:03,  9.90s/it]
 98%|█████████▊| 219591/225000 [429:01:24<12:51:28,  8.56s/it]
 98%|█████████▊| 219592/225000 [429:01:37<14:48:08,  9.85s/it]
 98%|█████████▊| 219593/225000 [429:01:43<12:41:58,  8.46s/it]
 98%|█████████▊| 219594/225000 [429:01:58<16:01:32, 10.67s/it]
 98%|█████████▊| 219595/225000 [429:02:03<13:18:14,  8.86s/it]
 98%|█████████▊| 219596/225000 [429:02:14<14:13:01,  9.47s/it]
 98%|█████████▊| 219597/225000 [429:02:24<14:21:06,  9.56s/it]
 98%|█████████▊| 219598/225000 [429:02:34<14:48:10,  9.86s/it]
 98%|█████████▊| 219599/225000 [429:02:42<13:55:26,  9.28s/it]
 98%|█████████▊| 219600/225000 [429:02:50<13:18:11,  8.87s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.116', 'learning_rate': '8.078e-08', 'epoch': '0.976', 'num_input_tokens_seen': 730563480, 'train_runtime': '1.545e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219600/225000 [429:02:50<13:18:11,  8.87s/it]
 98%|█████████▊| 219601/225000 [429:02:59<13:31:08,  9.01s/it]
 98%|█████████▊| 219602/225000 [429:03:07<12:42:06,  8.47s/it]
 98%|█████████▊| 219603/225000 [429:03:15<12:47:57,  8.54s/it]
 98%|█████████▊| 219604/225000 [429:03:24<12:49:05,  8.55s/it]
 98%|█████████▊| 219605/225000 [429:03:31<12:08:41,  8.10s/it]
 98%|█████████▊| 219606/225000 [429:03:44<14:11:59,  9.48s/it]
 98%|█████████▊| 219607/225000 [429:03:54<14:31:27,  9.70s/it]
 98%|█████████▊| 219608/225000 [429:04:05<15:20:01, 10.24s/it]
 98%|█████████▊| 219609/225000 [429:04:13<14:02:47,  9.38s/it]
 98%|█████████▊| 219610/225000 [429:04:22<14:01:32,  9.37s/it]
 98%|█████████▊| 219611/225000 [429:04:28<12:31:30,  8.37s/it]
 98%|█████████▊| 219612/225000 [429:04:41<14:44:59,  9.86s/it]
 98%|█████████▊| 219613/225000 [429:04:48<13:07:06,  8.77s/it]
 98%|█████████▊| 219614/225000 [429:04:56<12:51:15,  8.59s/it]
 98%|█████████▊| 219615/225000 [429:05:07<14:08:55,  9.46s/it]
 98%|█████████▊| 219616/225000 [429:05:14<13:06:04,  8.76s/it]
 98%|█████████▊| 219617/225000 [429:05:25<14:00:25,  9.37s/it]
 98%|█████████▊| 219618/225000 [429:05:35<14:16:41,  9.55s/it]
 98%|█████████▊| 219619/225000 [429:05:47<15:10:23, 10.15s/it]
 98%|█████████▊| 219620/225000 [429:05:55<14:04:55,  9.42s/it]
 98%|█████████▊| 219621/225000 [429:06:05<14:41:42,  9.84s/it]
 98%|█████████▊| 219622/225000 [429:06:12<13:07:50,  8.79s/it]
 98%|█████████▊| 219623/225000 [429:06:19<12:41:27,  8.50s/it]
 98%|█████████▊| 219624/225000 [429:06:32<14:36:10,  9.78s/it]
 98%|█████████▊| 219625/225000 [429:06:40<13:41:31,  9.17s/it]
 98%|█████████▊| 219626/225000 [429:06:47<12:48:22,  8.58s/it]
 98%|█████████▊| 219627/225000 [429:06:54<11:53:21,  7.97s/it]
 98%|█████████▊| 219628/225000 [429:07:00<10:58:02,  7.35s/it]
 98%|█████████▊| 219629/225000 [429:07:17<15:22:09, 10.30s/it]
 98%|█████████▊| 219630/225000 [429:07:25<14:36:06,  9.79s/it]
 98%|█████████▊| 219631/225000 [429:07:41<16:59:31, 11.39s/it]
 98%|█████████▊| 219632/225000 [429:07:46<14:15:59,  9.57s/it]
 98%|█████████▊| 219633/225000 [429:07:54<13:24:42,  9.00s/it]
 98%|█████████▊| 219634/225000 [429:08:08<15:54:15, 10.67s/it]
 98%|█████████▊| 219635/225000 [429:08:17<15:16:56, 10.25s/it]
 98%|█████████▊| 219636/225000 [429:08:27<15:11:39, 10.20s/it]
 98%|█████████▊| 219637/225000 [429:08:36<14:14:39,  9.56s/it]
 98%|█████████▊| 219638/225000 [429:08:44<13:43:52,  9.22s/it]
 98%|█████████▊| 219639/225000 [429:08:49<11:59:53,  8.06s/it]
 98%|█████████▊| 219640/225000 [429:08:59<12:51:18,  8.63s/it]
 98%|█████████▊| 219641/225000 [429:09:06<11:47:44,  7.92s/it]
 98%|█████████▊| 219642/225000 [429:09:13<11:41:56,  7.86s/it]
 98%|█████████▊| 219643/225000 [429:09:24<12:56:13,  8.69s/it]
 98%|█████████▊| 219644/225000 [429:09:32<12:35:03,  8.46s/it]
 98%|█████████▊| 219645/225000 [429:09:39<11:59:13,  8.06s/it]
 98%|█████████▊| 219646/225000 [429:09:49<12:58:07,  8.72s/it]
 98%|█████████▊| 219647/225000 [429:09:59<13:33:31,  9.12s/it]
 98%|█████████▊| 219648/225000 [429:10:11<14:49:47,  9.98s/it]
 98%|█████████▊| 219649/225000 [429:10:20<14:08:56,  9.52s/it]
 98%|█████████▊| 219650/225000 [429:10:30<14:29:21,  9.75s/it]
                                                              
{'loss': '1.33', 'grad_norm': '1.007', 'learning_rate': '7.932e-08', 'epoch': '0.9762', 'num_input_tokens_seen': 730802752, 'train_runtime': '1.545e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219650/225000 [429:10:30<14:29:21,  9.75s/it]
 98%|█████████▊| 219651/225000 [429:10:37<13:15:24,  8.92s/it]
 98%|█████████▊| 219652/225000 [429:10:47<13:50:49,  9.32s/it]
 98%|█████████▊| 219653/225000 [429:10:54<12:48:41,  8.63s/it]
 98%|█████████▊| 219654/225000 [429:11:02<12:26:02,  8.37s/it]
 98%|█████████▊| 219655/225000 [429:11:10<12:06:22,  8.15s/it]
 98%|█████████▊| 219656/225000 [429:11:23<14:15:54,  9.61s/it]
 98%|█████████▊| 219657/225000 [429:11:32<14:06:07,  9.50s/it]
 98%|█████████▊| 219658/225000 [429:11:40<13:19:40,  8.98s/it]
 98%|█████████▊| 219659/225000 [429:11:49<13:30:31,  9.11s/it]
 98%|█████████▊| 219660/225000 [429:12:00<14:29:59,  9.78s/it]
 98%|█████████▊| 219661/225000 [429:12:10<14:29:12,  9.77s/it]
 98%|█████████▊| 219662/225000 [429:12:15<12:16:32,  8.28s/it]
 98%|█████████▊| 219663/225000 [429:12:25<13:12:49,  8.91s/it]
 98%|█████████▊| 219664/225000 [429:12:33<12:29:49,  8.43s/it]
 98%|█████████▊| 219665/225000 [429:12:40<11:51:18,  8.00s/it]
 98%|█████████▊| 219666/225000 [429:12:51<13:12:53,  8.92s/it]
 98%|█████████▊| 219667/225000 [429:13:03<14:40:50,  9.91s/it]
 98%|█████████▊| 219668/225000 [429:13:11<13:56:28,  9.41s/it]
 98%|█████████▊| 219669/225000 [429:13:18<12:47:01,  8.63s/it]
 98%|█████████▊| 219670/225000 [429:13:30<14:13:40,  9.61s/it]
 98%|█████████▊| 219671/225000 [429:13:38<13:32:32,  9.15s/it]
 98%|█████████▊| 219672/225000 [429:13:44<12:17:35,  8.31s/it]
 98%|█████████▊| 219673/225000 [429:13:59<14:57:08, 10.10s/it]
 98%|█████████▊| 219674/225000 [429:14:08<14:33:50,  9.84s/it]
 98%|█████████▊| 219675/225000 [429:14:21<15:51:00, 10.72s/it]
 98%|█████████▊| 219676/225000 [429:14:32<15:59:59, 10.82s/it]
 98%|█████████▊| 219677/225000 [429:14:44<16:40:13, 11.27s/it]
 98%|█████████▊| 219678/225000 [429:14:52<15:21:09, 10.39s/it]
 98%|█████████▊| 219679/225000 [429:14:57<12:56:49,  8.76s/it]
 98%|█████████▊| 219680/225000 [429:15:09<14:15:45,  9.65s/it]
 98%|█████████▊| 219681/225000 [429:15:17<13:37:22,  9.22s/it]
 98%|█████████▊| 219682/225000 [429:15:29<14:45:04,  9.99s/it]
 98%|█████████▊| 219683/225000 [429:15:38<14:17:22,  9.68s/it]
 98%|█████████▊| 219684/225000 [429:15:45<13:09:42,  8.91s/it]
 98%|█████████▊| 219685/225000 [429:15:58<15:09:52, 10.27s/it]
 98%|█████████▊| 219686/225000 [429:16:07<14:20:16,  9.71s/it]
 98%|█████████▊| 219687/225000 [429:16:15<13:28:06,  9.13s/it]
 98%|█████████▊| 219688/225000 [429:16:23<13:08:38,  8.91s/it]
 98%|█████████▊| 219689/225000 [429:16:31<12:49:24,  8.69s/it]
 98%|█████████▊| 219690/225000 [429:16:38<12:00:07,  8.14s/it]
 98%|█████████▊| 219691/225000 [429:16:45<11:26:42,  7.76s/it]
 98%|█████████▊| 219692/225000 [429:16:58<13:58:55,  9.48s/it]
 98%|█████████▊| 219693/225000 [429:17:05<12:40:48,  8.60s/it]
 98%|█████████▊| 219694/225000 [429:17:13<12:19:08,  8.36s/it]
 98%|█████████▊| 219695/225000 [429:17:20<12:00:10,  8.15s/it]
 98%|█████████▊| 219696/225000 [429:17:28<11:40:15,  7.92s/it]
 98%|█████████▊| 219697/225000 [429:17:40<13:21:36,  9.07s/it]
 98%|█████████▊| 219698/225000 [429:17:48<13:00:57,  8.84s/it]
 98%|█████████▊| 219699/225000 [429:17:55<12:22:33,  8.40s/it]
 98%|█████████▊| 219700/225000 [429:18:04<12:43:05,  8.64s/it]
                                                              
{'loss': '1.289', 'grad_norm': '1.229', 'learning_rate': '7.787e-08', 'epoch': '0.9764', 'num_input_tokens_seen': 731040448, 'train_runtime': '1.545e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219700/225000 [429:18:04<12:43:05,  8.64s/it]
 98%|█████████▊| 219701/225000 [429:18:17<14:36:32,  9.92s/it]
 98%|█████████▊| 219702/225000 [429:18:23<12:32:58,  8.53s/it]
 98%|█████████▊| 219703/225000 [429:18:32<12:58:11,  8.81s/it]
 98%|█████████▊| 219704/225000 [429:18:43<13:49:48,  9.40s/it]
 98%|█████████▊| 219705/225000 [429:18:50<12:54:20,  8.77s/it]
 98%|█████████▊| 219706/225000 [429:18:58<12:15:09,  8.33s/it]
 98%|█████████▊| 219707/225000 [429:19:06<12:06:56,  8.24s/it]
 98%|█████████▊| 219708/225000 [429:19:16<12:53:41,  8.77s/it]
 98%|█████████▊| 219709/225000 [429:19:22<11:50:32,  8.06s/it]
 98%|█████████▊| 219710/225000 [429:19:31<12:18:24,  8.38s/it]
 98%|█████████▊| 219711/225000 [429:19:41<13:11:59,  8.98s/it]
 98%|█████████▊| 219712/225000 [429:19:47<11:48:55,  8.04s/it]
 98%|█████████▊| 219713/225000 [429:19:58<12:47:18,  8.71s/it]
 98%|█████████▊| 219714/225000 [429:20:03<11:23:02,  7.75s/it]
 98%|█████████▊| 219715/225000 [429:20:14<12:37:16,  8.60s/it]
 98%|█████████▊| 219716/225000 [429:20:23<12:46:59,  8.71s/it]
 98%|█████████▊| 219717/225000 [429:20:33<13:20:20,  9.09s/it]
 98%|█████████▊| 219718/225000 [429:20:42<13:28:24,  9.18s/it]
 98%|█████████▊| 219719/225000 [429:20:55<14:55:18, 10.17s/it]
 98%|█████████▊| 219720/225000 [429:21:03<14:02:21,  9.57s/it]
 98%|█████████▊| 219721/225000 [429:21:13<14:20:21,  9.78s/it]
 98%|█████████▊| 219722/225000 [429:21:23<14:38:35,  9.99s/it]
 98%|█████████▊| 219723/225000 [429:21:29<12:38:12,  8.62s/it]
 98%|█████████▊| 219724/225000 [429:21:40<13:37:18,  9.29s/it]
 98%|█████████▊| 219725/225000 [429:21:47<12:48:16,  8.74s/it]
 98%|█████████▊| 219726/225000 [429:21:57<13:05:25,  8.94s/it]
 98%|█████████▊| 219727/225000 [429:22:05<12:41:15,  8.66s/it]
 98%|█████████▊| 219728/225000 [429:22:12<12:00:53,  8.20s/it]
 98%|█████████▊| 219729/225000 [429:22:22<12:58:12,  8.86s/it]
 98%|█████████▊| 219730/225000 [429:22:30<12:45:14,  8.71s/it]
 98%|█████████▊| 219731/225000 [429:22:40<12:57:57,  8.86s/it]
 98%|█████████▊| 219732/225000 [429:22:47<12:04:51,  8.26s/it]
 98%|█████████▊| 219733/225000 [429:22:53<11:22:43,  7.78s/it]
 98%|█████████▊| 219734/225000 [429:23:01<11:16:04,  7.70s/it]
 98%|█████████▊| 219735/225000 [429:23:11<12:15:05,  8.38s/it]
 98%|█████████▊| 219736/225000 [429:23:17<11:29:08,  7.86s/it]
 98%|█████████▊| 219737/225000 [429:23:24<10:52:07,  7.43s/it]
 98%|█████████▊| 219738/225000 [429:23:39<14:25:54,  9.87s/it]
 98%|█████████▊| 219739/225000 [429:23:47<13:24:51,  9.18s/it]
 98%|█████████▊| 219740/225000 [429:23:56<13:34:01,  9.29s/it]
 98%|█████████▊| 219741/225000 [429:24:02<11:57:30,  8.19s/it]
 98%|█████████▊| 219742/225000 [429:24:14<13:36:02,  9.31s/it]
 98%|█████████▊| 219743/225000 [429:24:20<12:21:40,  8.47s/it]
 98%|█████████▊| 219744/225000 [429:24:31<13:16:20,  9.09s/it]
 98%|█████████▊| 219745/225000 [429:24:38<12:24:44,  8.50s/it]
 98%|█████████▊| 219746/225000 [429:24:49<13:37:39,  9.34s/it]
 98%|█████████▊| 219747/225000 [429:24:57<13:04:12,  8.96s/it]
 98%|█████████▊| 219748/225000 [429:25:08<13:36:47,  9.33s/it]
 98%|█████████▊| 219749/225000 [429:25:14<12:06:18,  8.30s/it]
 98%|█████████▊| 219750/225000 [429:25:26<13:48:31,  9.47s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.423', 'learning_rate': '7.644e-08', 'epoch': '0.9767', 'num_input_tokens_seen': 731275144, 'train_runtime': '1.546e+06', 'train_tokens_per_second': '473'}

 98%|█████████▊| 219750/225000 [429:25:26<13:48:31,  9.47s/it]
 98%|█████████▊| 219751/225000 [429:25:32<12:24:15,  8.51s/it]
 98%|█████████▊| 219752/225000 [429:25:41<12:46:33,  8.76s/it]
 98%|█████████▊| 219753/225000 [429:25:48<12:00:30,  8.24s/it]
 98%|█████████▊| 219754/225000 [429:25:58<12:41:19,  8.71s/it]
 98%|█████████▊| 219755/225000 [429:26:07<12:51:45,  8.83s/it]
 98%|█████████▊| 219756/225000 [429:26:16<12:56:26,  8.88s/it]
 98%|█████████▊| 219757/225000 [429:26:28<14:00:47,  9.62s/it]
 98%|█████████▊| 219758/225000 [429:26:36<13:29:49,  9.27s/it]
 98%|█████████▊| 219759/225000 [429:26:47<14:05:45,  9.68s/it]
 98%|█████████▊| 219760/225000 [429:26:56<13:59:26,  9.61s/it]
 98%|█████████▊| 219761/225000 [429:27:01<11:58:19,  8.23s/it]
 98%|█████████▊| 219762/225000 [429:27:09<11:46:57,  8.10s/it]
 98%|█████████▊| 219763/225000 [429:27:17<11:43:58,  8.07s/it]
 98%|█████████▊| 219764/225000 [429:27:32<14:51:31, 10.22s/it]
 98%|█████████▊| 219765/225000 [429:27:42<14:47:12, 10.17s/it]
 98%|█████████▊| 219766/225000 [429:27:53<14:47:37, 10.18s/it]
 98%|█████████▊| 219767/225000 [429:28:00<13:48:33,  9.50s/it]
 98%|█████████▊| 219768/225000 [429:28:12<14:42:45, 10.12s/it]
 98%|█████████▊| 219769/225000 [429:28:21<14:23:49,  9.91s/it]
 98%|█████████▊| 219770/225000 [429:28:26<11:57:08,  8.23s/it]
 98%|█████████▊| 219771/225000 [429:28:35<12:26:41,  8.57s/it]
 98%|█████████▊| 219772/225000 [429:28:43<12:02:00,  8.29s/it]
 98%|█████████▊| 219773/225000 [429:28:51<11:50:08,  8.15s/it]
 98%|█████████▊| 219774/225000 [429:28:59<11:45:55,  8.10s/it]
 98%|█████████▊| 219775/225000 [429:29:08<12:31:28,  8.63s/it]
 98%|█████████▊| 219776/225000 [429:29:17<12:31:40,  8.63s/it]
 98%|█████████▊| 219777/225000 [429:29:24<11:46:20,  8.11s/it]
 98%|█████████▊| 219778/225000 [429:29:35<12:54:27,  8.90s/it]
 98%|█████████▊| 219779/225000 [429:29:43<12:27:21,  8.59s/it]
 98%|█████████▊| 219780/225000 [429:29:54<13:54:48,  9.60s/it]
 98%|█████████▊| 219781/225000 [429:30:04<13:44:41,  9.48s/it]
 98%|█████████▊| 219782/225000 [429:30:08<11:24:15,  7.87s/it]
 98%|█████████▊| 219783/225000 [429:30:17<12:10:01,  8.40s/it]
 98%|█████████▊| 219784/225000 [429:30:24<11:29:15,  7.93s/it]
 98%|█████████▊| 219785/225000 [429:30:34<12:03:31,  8.32s/it]
 98%|█████████▊| 219786/225000 [429:30:41<11:35:29,  8.00s/it]
 98%|█████████▊| 219787/225000 [429:30:48<11:24:52,  7.88s/it]
 98%|█████████▊| 219788/225000 [429:31:03<14:08:41,  9.77s/it]
 98%|█████████▊| 219789/225000 [429:31:08<12:14:43,  8.46s/it]
 98%|█████████▊| 219790/225000 [429:31:21<14:07:48,  9.76s/it]
 98%|█████████▊| 219791/225000 [429:31:30<13:47:30,  9.53s/it]
 98%|█████████▊| 219792/225000 [429:31:46<16:39:54, 11.52s/it]
 98%|█████████▊| 219793/225000 [429:31:51<14:01:12,  9.69s/it]
 98%|█████████▊| 219794/225000 [429:32:00<13:36:40,  9.41s/it]
 98%|█████████▊| 219795/225000 [429:32:12<14:45:10, 10.20s/it]
 98%|█████████▊| 219796/225000 [429:32:20<13:55:05,  9.63s/it]
 98%|█████████▊| 219797/225000 [429:32:26<12:16:26,  8.49s/it]
 98%|█████████▊| 219798/225000 [429:32:38<13:43:55,  9.50s/it]
 98%|█████████▊| 219799/225000 [429:32:50<14:41:52, 10.17s/it]
 98%|█████████▊| 219800/225000 [429:32:55<12:33:42,  8.70s/it]
                                                              
{'loss': '1.335', 'grad_norm': '1.288', 'learning_rate': '7.502e-08', 'epoch': '0.9769', 'num_input_tokens_seen': 731512968, 'train_runtime': '1.546e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 219800/225000 [429:32:55<12:33:42,  8.70s/it]
 98%|█████████▊| 219801/225000 [429:33:06<13:38:16,  9.44s/it]
 98%|█████████▊| 219802/225000 [429:33:15<13:21:04,  9.25s/it]
 98%|█████████▊| 219803/225000 [429:33:23<12:55:08,  8.95s/it]
 98%|█████████▊| 219804/225000 [429:33:29<11:25:00,  7.91s/it]
 98%|█████████▊| 219805/225000 [429:33:36<10:54:27,  7.56s/it]
 98%|█████████▊| 219806/225000 [429:33:42<10:33:21,  7.32s/it]
 98%|█████████▊| 219807/225000 [429:33:58<14:05:33,  9.77s/it]
 98%|█████████▊| 219808/225000 [429:34:07<13:55:30,  9.66s/it]
 98%|█████████▊| 219809/225000 [429:34:15<13:05:41,  9.08s/it]
 98%|█████████▊| 219810/225000 [429:34:21<11:37:38,  8.07s/it]
 98%|█████████▊| 219811/225000 [429:34:34<13:48:28,  9.58s/it]
 98%|█████████▊| 219812/225000 [429:34:41<12:52:40,  8.94s/it]
 98%|█████████▊| 219813/225000 [429:34:48<11:52:52,  8.25s/it]
 98%|█████████▊| 219814/225000 [429:34:56<11:40:45,  8.11s/it]
 98%|█████████▊| 219815/225000 [429:35:03<11:10:22,  7.76s/it]
 98%|█████████▊| 219816/225000 [429:35:11<11:19:47,  7.87s/it]
 98%|█████████▊| 219817/225000 [429:35:22<12:46:48,  8.88s/it]
 98%|█████████▊| 219818/225000 [429:35:28<11:40:25,  8.11s/it]
 98%|█████████▊| 219819/225000 [429:35:45<15:37:21, 10.86s/it]
 98%|█████████▊| 219820/225000 [429:35:54<14:40:33, 10.20s/it]
 98%|█████████▊| 219821/225000 [429:35:59<12:18:18,  8.55s/it]
 98%|█████████▊| 219822/225000 [429:36:10<13:24:59,  9.33s/it]
 98%|█████████▊| 219823/225000 [429:36:17<12:24:42,  8.63s/it]
 98%|█████████▊| 219824/225000 [429:36:24<11:43:05,  8.15s/it]
 98%|█████████▊| 219825/225000 [429:36:37<13:36:01,  9.46s/it]
 98%|█████████▊| 219826/225000 [429:36:43<12:05:53,  8.42s/it]
 98%|█████████▊| 219827/225000 [429:36:49<11:05:01,  7.71s/it]
 98%|█████████▊| 219828/225000 [429:36:59<12:09:43,  8.47s/it]
 98%|█████████▊| 219829/225000 [429:37:10<13:32:15,  9.42s/it]
 98%|█████████▊| 219830/225000 [429:37:17<12:12:31,  8.50s/it]
 98%|█████████▊| 219831/225000 [429:37:29<13:37:22,  9.49s/it]
 98%|█████████▊| 219832/225000 [429:37:38<13:38:40,  9.50s/it]
 98%|█████████▊| 219833/225000 [429:37:48<13:43:04,  9.56s/it]
 98%|█████████▊| 219834/225000 [429:37:58<14:02:17,  9.78s/it]
 98%|█████████▊| 219835/225000 [429:38:06<13:06:30,  9.14s/it]
 98%|█████████▊| 219836/225000 [429:38:17<14:08:51,  9.86s/it]
 98%|█████████▊| 219837/225000 [429:38:27<13:57:51,  9.74s/it]
 98%|█████████▊| 219838/225000 [429:38:32<11:55:41,  8.32s/it]
 98%|█████████▊| 219839/225000 [429:38:46<14:34:33, 10.17s/it]
 98%|█████████▊| 219840/225000 [429:38:53<13:15:51,  9.25s/it]
 98%|█████████▊| 219841/225000 [429:39:05<14:20:19, 10.01s/it]
 98%|█████████▊| 219842/225000 [429:39:12<13:09:14,  9.18s/it]
 98%|█████████▊| 219843/225000 [429:39:21<12:50:00,  8.96s/it]
 98%|█████████▊| 219844/225000 [429:39:29<12:17:33,  8.58s/it]
 98%|█████████▊| 219845/225000 [429:39:35<11:29:56,  8.03s/it]
 98%|█████████▊| 219846/225000 [429:39:42<10:47:54,  7.54s/it]
 98%|█████████▊| 219847/225000 [429:39:57<14:16:55,  9.98s/it]
 98%|█████████▊| 219848/225000 [429:40:07<14:03:25,  9.82s/it]
 98%|█████████▊| 219849/225000 [429:40:14<12:50:52,  8.98s/it]
 98%|█████████▊| 219850/225000 [429:40:20<11:40:37,  8.16s/it]
                                                              
{'loss': '1.313', 'grad_norm': '1.757', 'learning_rate': '7.361e-08', 'epoch': '0.9771', 'num_input_tokens_seen': 731751496, 'train_runtime': '1.547e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 219850/225000 [429:40:20<11:40:37,  8.16s/it]
 98%|█████████▊| 219851/225000 [429:40:26<10:52:58,  7.61s/it]
 98%|█████████▊| 219852/225000 [429:40:38<12:23:27,  8.66s/it]
 98%|█████████▊| 219853/225000 [429:40:47<12:46:51,  8.94s/it]
 98%|█████████▊| 219854/225000 [429:40:53<11:15:08,  7.87s/it]
 98%|█████████▊| 219855/225000 [429:41:03<12:17:30,  8.60s/it]
 98%|█████████▊| 219856/225000 [429:41:09<11:27:05,  8.01s/it]
 98%|█████████▊| 219857/225000 [429:41:17<11:15:33,  7.88s/it]
 98%|█████████▊| 219858/225000 [429:41:24<10:58:41,  7.69s/it]
 98%|█████████▊| 219859/225000 [429:41:31<10:36:18,  7.43s/it]
 98%|█████████▊| 219860/225000 [429:41:44<12:55:34,  9.05s/it]
 98%|█████████▊| 219861/225000 [429:41:49<11:19:07,  7.93s/it]
 98%|█████████▊| 219862/225000 [429:42:02<13:14:24,  9.28s/it]
 98%|█████████▊| 219863/225000 [429:42:10<12:40:24,  8.88s/it]
 98%|█████████▊| 219864/225000 [429:42:16<11:32:54,  8.09s/it]
 98%|█████████▊| 219865/225000 [429:42:29<13:43:53,  9.63s/it]
 98%|█████████▊| 219866/225000 [429:42:36<12:34:04,  8.81s/it]
 98%|█████████▊| 219867/225000 [429:42:44<12:18:43,  8.63s/it]
 98%|█████████▊| 219868/225000 [429:42:57<13:56:30,  9.78s/it]
 98%|█████████▊| 219869/225000 [429:43:05<13:13:14,  9.28s/it]
 98%|█████████▊| 219870/225000 [429:43:15<13:39:45,  9.59s/it]
 98%|█████████▊| 219871/225000 [429:43:26<14:07:05,  9.91s/it]
 98%|█████████▊| 219872/225000 [429:43:32<12:22:04,  8.68s/it]
 98%|█████████▊| 219873/225000 [429:43:38<11:25:48,  8.03s/it]
 98%|█████████▊| 219874/225000 [429:43:53<14:28:12, 10.16s/it]
 98%|█████████▊| 219875/225000 [429:43:59<12:44:45,  8.95s/it]
 98%|█████████▊| 219876/225000 [429:44:04<11:06:40,  7.81s/it]
 98%|█████████▊| 219877/225000 [429:44:15<12:21:28,  8.68s/it]
 98%|█████████▊| 219878/225000 [429:44:23<12:05:38,  8.50s/it]
 98%|█████████▊| 219879/225000 [429:44:29<11:06:26,  7.81s/it]
 98%|█████████▊| 219880/225000 [429:44:38<11:27:00,  8.05s/it]
 98%|█████████▊| 219881/225000 [429:44:49<12:33:06,  8.83s/it]
 98%|█████████▊| 219882/225000 [429:44:59<13:12:06,  9.29s/it]
 98%|█████████▊| 219883/225000 [429:45:05<11:42:12,  8.23s/it]
 98%|█████████▊| 219884/225000 [429:45:13<11:38:08,  8.19s/it]
 98%|█████████▊| 219885/225000 [429:45:20<10:58:05,  7.72s/it]
 98%|█████████▊| 219886/225000 [429:45:29<11:32:49,  8.13s/it]
 98%|█████████▊| 219887/225000 [429:45:42<13:36:05,  9.58s/it]
 98%|█████████▊| 219888/225000 [429:45:48<12:10:10,  8.57s/it]
 98%|█████████▊| 219889/225000 [429:45:53<10:50:43,  7.64s/it]
 98%|█████████▊| 219890/225000 [429:46:04<11:59:25,  8.45s/it]
 98%|█████████▊| 219891/225000 [429:46:11<11:32:27,  8.13s/it]
 98%|█████████▊| 219892/225000 [429:46:20<11:53:55,  8.39s/it]
 98%|█████████▊| 219893/225000 [429:46:26<10:59:59,  7.75s/it]
 98%|█████████▊| 219894/225000 [429:46:37<12:11:52,  8.60s/it]
 98%|█████████▊| 219895/225000 [429:46:43<11:04:49,  7.81s/it]
 98%|█████████▊| 219896/225000 [429:46:55<12:55:39,  9.12s/it]
 98%|█████████▊| 219897/225000 [429:47:05<13:28:37,  9.51s/it]
 98%|█████████▊| 219898/225000 [429:47:13<12:51:13,  9.07s/it]
 98%|█████████▊| 219899/225000 [429:47:20<11:47:09,  8.32s/it]
 98%|█████████▊| 219900/225000 [429:47:29<12:12:04,  8.61s/it]
                                                              
{'loss': '1.314', 'grad_norm': '1.553', 'learning_rate': '7.222e-08', 'epoch': '0.9773', 'num_input_tokens_seen': 731980800, 'train_runtime': '1.547e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 219900/225000 [429:47:29<12:12:04,  8.61s/it]
 98%|█████████▊| 219901/225000 [429:47:35<11:00:35,  7.77s/it]
 98%|█████████▊| 219902/225000 [429:47:46<12:16:08,  8.66s/it]
 98%|█████████▊| 219903/225000 [429:47:59<14:19:24, 10.12s/it]
 98%|█████████▊| 219904/225000 [429:48:05<12:33:10,  8.87s/it]
 98%|█████████▊| 219905/225000 [429:48:14<12:22:02,  8.74s/it]
 98%|█████████▊| 219906/225000 [429:48:19<10:59:59,  7.77s/it]
 98%|█████████▊| 219907/225000 [429:48:30<12:10:36,  8.61s/it]
 98%|█████████▊| 219908/225000 [429:48:41<13:23:44,  9.47s/it]
 98%|█████████▊| 219909/225000 [429:48:49<12:41:46,  8.98s/it]
 98%|█████████▊| 219910/225000 [429:48:59<13:14:15,  9.36s/it]
 98%|█████████▊| 219911/225000 [429:49:06<11:57:53,  8.46s/it]
 98%|█████████▊| 219912/225000 [429:49:17<12:56:50,  9.16s/it]
 98%|█████████▊| 219913/225000 [429:49:24<12:25:17,  8.79s/it]
 98%|█████████▊| 219914/225000 [429:49:29<10:31:59,  7.46s/it]
 98%|█████████▊| 219915/225000 [429:49:42<12:52:33,  9.12s/it]
 98%|█████████▊| 219916/225000 [429:49:50<12:26:08,  8.81s/it]
 98%|█████████▊| 219917/225000 [429:49:59<12:24:32,  8.79s/it]
 98%|█████████▊| 219918/225000 [429:50:10<13:23:22,  9.49s/it]
 98%|█████████▊| 219919/225000 [429:50:18<12:55:43,  9.16s/it]
 98%|█████████▊| 219920/225000 [429:50:25<11:46:53,  8.35s/it]
 98%|█████████▊| 219921/225000 [429:50:36<13:14:18,  9.38s/it]
 98%|█████████▊| 219922/225000 [429:50:45<12:56:01,  9.17s/it]
 98%|█████████▊| 219923/225000 [429:50:56<13:37:32,  9.66s/it]
 98%|█████████▊| 219924/225000 [429:51:05<13:14:39,  9.39s/it]
 98%|█████████▊| 219925/225000 [429:51:10<11:30:24,  8.16s/it]
 98%|█████████▊| 219926/225000 [429:51:21<12:41:27,  9.00s/it]
 98%|█████████▊| 219927/225000 [429:51:27<11:32:23,  8.19s/it]
 98%|█████████▊| 219928/225000 [429:51:37<12:02:49,  8.55s/it]
 98%|█████████▊| 219929/225000 [429:51:47<12:49:19,  9.10s/it]
 98%|█████████▊| 219930/225000 [429:51:56<12:43:12,  9.03s/it]
 98%|█████████▊| 219931/225000 [429:52:03<11:51:11,  8.42s/it]
 98%|█████████▊| 219932/225000 [429:52:10<11:10:38,  7.94s/it]
 98%|█████████▊| 219933/225000 [429:52:25<14:06:06, 10.02s/it]
 98%|█████████▊| 219934/225000 [429:52:33<13:15:40,  9.42s/it]
 98%|█████████▊| 219935/225000 [429:52:43<13:45:30,  9.78s/it]
 98%|█████████▊| 219936/225000 [429:52:48<11:49:03,  8.40s/it]
 98%|█████████▊| 219937/225000 [429:52:56<11:34:35,  8.23s/it]
 98%|█████████▊| 219938/225000 [429:53:08<13:07:32,  9.33s/it]
 98%|█████████▊| 219939/225000 [429:53:21<14:39:54, 10.43s/it]
 98%|█████████▊| 219940/225000 [429:53:28<13:21:55,  9.51s/it]
 98%|█████████▊| 219941/225000 [429:53:35<12:12:13,  8.68s/it]
 98%|█████████▊| 219942/225000 [429:53:43<11:56:33,  8.50s/it]
 98%|█████████▊| 219943/225000 [429:53:51<11:29:30,  8.18s/it]
 98%|█████████▊| 219944/225000 [429:53:58<11:03:46,  7.88s/it]
 98%|█████████▊| 219945/225000 [429:54:14<14:19:25, 10.20s/it]
 98%|█████████▊| 219946/225000 [429:54:23<14:00:40,  9.98s/it]
 98%|█████████▊| 219947/225000 [429:54:31<13:07:01,  9.35s/it]
 98%|█████████▊| 219948/225000 [429:54:41<13:18:41,  9.49s/it]
 98%|█████████▊| 219949/225000 [429:54:47<11:47:26,  8.40s/it]
 98%|█████████▊| 219950/225000 [429:54:58<12:57:33,  9.24s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.287', 'learning_rate': '7.084e-08', 'epoch': '0.9776', 'num_input_tokens_seen': 732214904, 'train_runtime': '1.548e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 219950/225000 [429:54:58<12:57:33,  9.24s/it]
 98%|█████████▊| 219951/225000 [429:55:04<11:31:18,  8.22s/it]
 98%|█████████▊| 219952/225000 [429:55:14<12:35:52,  8.98s/it]
 98%|█████████▊| 219953/225000 [429:55:20<11:20:37,  8.09s/it]
 98%|█████████▊| 219954/225000 [429:55:31<12:31:01,  8.93s/it]
 98%|█████████▊| 219955/225000 [429:55:41<12:45:44,  9.11s/it]
 98%|█████████▊| 219956/225000 [429:55:47<11:43:25,  8.37s/it]
 98%|█████████▊| 219957/225000 [429:55:55<11:32:14,  8.24s/it]
 98%|█████████▊| 219958/225000 [429:56:03<11:20:52,  8.10s/it]
 98%|█████████▊| 219959/225000 [429:56:14<12:30:58,  8.94s/it]
 98%|█████████▊| 219960/225000 [429:56:23<12:22:38,  8.84s/it]
 98%|█████████▊| 219961/225000 [429:56:34<13:23:15,  9.56s/it]
 98%|█████████▊| 219962/225000 [429:56:44<13:25:58,  9.60s/it]
 98%|█████████▊| 219963/225000 [429:56:49<11:52:52,  8.49s/it]
 98%|█████████▊| 219964/225000 [429:56:58<11:47:54,  8.43s/it]
 98%|█████████▊| 219965/225000 [429:57:04<10:41:04,  7.64s/it]
 98%|█████████▊| 219966/225000 [429:57:17<13:08:15,  9.40s/it]
 98%|█████████▊| 219967/225000 [429:57:22<11:13:28,  8.03s/it]
 98%|█████████▊| 219968/225000 [429:57:35<13:11:13,  9.43s/it]
 98%|█████████▊| 219969/225000 [429:57:40<11:29:16,  8.22s/it]
 98%|█████████▊| 219970/225000 [429:57:47<11:03:14,  7.91s/it]
 98%|█████████▊| 219971/225000 [429:57:56<11:23:58,  8.16s/it]
 98%|█████████▊| 219972/225000 [429:58:08<13:06:15,  9.38s/it]
 98%|█████████▊| 219973/225000 [429:58:15<11:57:18,  8.56s/it]
 98%|█████████▊| 219974/225000 [429:58:23<12:00:14,  8.60s/it]
 98%|█████████▊| 219975/225000 [429:58:31<11:35:11,  8.30s/it]
 98%|█████████▊| 219976/225000 [429:58:36<10:14:37,  7.34s/it]
 98%|█████████▊| 219977/225000 [429:58:43<10:10:58,  7.30s/it]
 98%|█████████▊| 219978/225000 [429:59:01<14:25:07, 10.34s/it]
 98%|█████████▊| 219979/225000 [429:59:12<14:53:06, 10.67s/it]
 98%|█████████▊| 219980/225000 [429:59:19<13:17:29,  9.53s/it]
 98%|█████████▊| 219981/225000 [429:59:26<12:01:31,  8.63s/it]
 98%|█████████▊| 219982/225000 [429:59:37<13:02:10,  9.35s/it]
 98%|█████████▊| 219983/225000 [429:59:44<12:11:34,  8.75s/it]
 98%|█████████▊| 219984/225000 [429:59:51<11:31:56,  8.28s/it]
 98%|█████████▊| 219985/225000 [430:00:05<13:54:12,  9.98s/it]
 98%|█████████▊| 219986/225000 [430:00:11<12:21:39,  8.88s/it]
 98%|█████████▊| 219987/225000 [430:00:17<10:57:05,  7.86s/it]
 98%|█████████▊| 219988/225000 [430:00:22<9:52:31,  7.09s/it] 
 98%|█████████▊| 219989/225000 [430:00:40<14:15:47, 10.25s/it]
 98%|█████████▊| 219990/225000 [430:00:50<14:15:40, 10.25s/it]
 98%|█████████▊| 219991/225000 [430:00:57<12:59:07,  9.33s/it]
 98%|█████████▊| 219992/225000 [430:01:03<11:30:55,  8.28s/it]
 98%|█████████▊| 219993/225000 [430:01:10<11:01:42,  7.93s/it]
 98%|█████████▊| 219994/225000 [430:01:16<10:05:08,  7.25s/it]
 98%|█████████▊| 219995/225000 [430:01:30<12:57:52,  9.33s/it]
 98%|█████████▊| 219996/225000 [430:01:39<12:41:38,  9.13s/it]
 98%|█████████▊| 219997/225000 [430:01:44<11:04:15,  7.97s/it]
 98%|█████████▊| 219998/225000 [430:01:56<12:40:01,  9.12s/it]
 98%|█████████▊| 219999/225000 [430:02:08<13:44:38,  9.89s/it]
 98%|█████████▊| 220000/225000 [430:02:17<13:46:38,  9.92s/it]
                                                              
{'loss': '1.307', 'grad_norm': '1.36', 'learning_rate': '6.947e-08', 'epoch': '0.9778', 'num_input_tokens_seen': 732442424, 'train_runtime': '1.548e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 220000/225000 [430:02:17<13:46:38,  9.92s/it]
 98%|█████████▊| 220001/225000 [430:02:22<11:30:59,  8.29s/it]
 98%|█████████▊| 220002/225000 [430:02:29<10:59:59,  7.92s/it]
 98%|█████████▊| 220003/225000 [430:02:37<11:00:51,  7.94s/it]
 98%|█████████▊| 220004/225000 [430:02:43<10:15:38,  7.39s/it]
 98%|█████████▊| 220005/225000 [430:02:51<10:24:03,  7.50s/it]
 98%|█████████▊| 220006/225000 [430:02:58<10:05:40,  7.28s/it]
 98%|█████████▊| 220007/225000 [430:03:05<9:57:53,  7.18s/it] 
 98%|█████████▊| 220008/225000 [430:03:19<12:53:18,  9.29s/it]
 98%|█████████▊| 220009/225000 [430:03:26<11:57:53,  8.63s/it]
 98%|█████████▊| 220010/225000 [430:03:36<12:43:20,  9.18s/it]
 98%|█████████▊| 220011/225000 [430:03:44<11:53:15,  8.58s/it]
 98%|█████████▊| 220012/225000 [430:03:54<12:30:19,  9.03s/it]
 98%|█████████▊| 220013/225000 [430:03:59<11:03:24,  7.98s/it]
 98%|█████████▊| 220014/225000 [430:04:12<12:56:33,  9.34s/it]
 98%|█████████▊| 220015/225000 [430:04:19<11:57:29,  8.64s/it]
 98%|█████████▊| 220016/225000 [430:04:24<10:46:39,  7.78s/it]
 98%|█████████▊| 220017/225000 [430:04:35<11:43:28,  8.47s/it]
 98%|█████████▊| 220018/225000 [430:04:42<11:15:16,  8.13s/it]
 98%|█████████▊| 220019/225000 [430:04:48<10:33:17,  7.63s/it]
 98%|█████████▊| 220020/225000 [430:04:56<10:25:24,  7.53s/it]
 98%|█████████▊| 220021/225000 [430:05:11<13:48:02,  9.98s/it]
 98%|█████████▊| 220022/225000 [430:05:21<13:40:53,  9.89s/it]
 98%|█████████▊| 220023/225000 [430:05:32<13:58:58, 10.11s/it]
 98%|█████████▊| 220024/225000 [430:05:37<12:09:26,  8.80s/it]
 98%|█████████▊| 220025/225000 [430:05:48<12:50:25,  9.29s/it]
 98%|█████████▊| 220026/225000 [430:05:55<11:53:44,  8.61s/it]
 98%|█████████▊| 220027/225000 [430:06:03<11:30:23,  8.33s/it]
 98%|█████████▊| 220028/225000 [430:06:13<12:26:28,  9.01s/it]
 98%|█████████▊| 220029/225000 [430:06:23<12:50:26,  9.30s/it]
 98%|█████████▊| 220030/225000 [430:06:29<11:28:17,  8.31s/it]
 98%|█████████▊| 220031/225000 [430:06:42<13:14:45,  9.60s/it]
 98%|█████████▊| 220032/225000 [430:06:51<13:08:13,  9.52s/it]
 98%|█████████▊| 220033/225000 [430:06:59<12:39:15,  9.17s/it]
 98%|█████████▊| 220034/225000 [430:07:06<11:24:35,  8.27s/it]
 98%|█████████▊| 220035/225000 [430:07:13<11:03:39,  8.02s/it]
 98%|█████████▊| 220036/225000 [430:07:20<10:35:58,  7.69s/it]
 98%|█████████▊| 220037/225000 [430:07:26<9:49:31,  7.13s/it] 
 98%|█████████▊| 220038/225000 [430:07:36<11:07:54,  8.08s/it]
 98%|█████████▊| 220039/225000 [430:07:42<10:27:55,  7.59s/it]
 98%|█████████▊| 220040/225000 [430:07:51<10:39:52,  7.74s/it]
 98%|█████████▊| 220041/225000 [430:07:59<10:57:40,  7.96s/it]
 98%|█████████▊| 220042/225000 [430:08:18<15:25:23, 11.20s/it]
 98%|█████████▊| 220043/225000 [430:08:27<14:32:27, 10.56s/it]
 98%|█████████▊| 220044/225000 [430:08:32<12:25:26,  9.02s/it]
 98%|█████████▊| 220045/225000 [430:08:42<12:30:38,  9.09s/it]
 98%|█████████▊| 220046/225000 [430:08:54<13:51:20, 10.07s/it]
 98%|█████████▊| 220047/225000 [430:09:01<12:36:58,  9.17s/it]
 98%|█████████▊| 220048/225000 [430:09:12<13:11:26,  9.59s/it]
 98%|█████████▊| 220049/225000 [430:09:20<12:41:28,  9.23s/it]
 98%|█████████▊| 220050/225000 [430:09:29<12:43:14,  9.25s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.184', 'learning_rate': '6.811e-08', 'epoch': '0.978', 'num_input_tokens_seen': 732673136, 'train_runtime': '1.549e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 220050/225000 [430:09:29<12:43:14,  9.25s/it]
 98%|█████████▊| 220051/225000 [430:09:35<11:19:29,  8.24s/it]
 98%|█████████▊| 220052/225000 [430:09:45<11:55:30,  8.68s/it]
 98%|█████████▊| 220053/225000 [430:09:50<10:18:19,  7.50s/it]
 98%|█████████▊| 220054/225000 [430:09:54<8:59:39,  6.55s/it] 
 98%|█████████▊| 220055/225000 [430:10:08<12:13:08,  8.90s/it]
 98%|█████████▊| 220056/225000 [430:10:18<12:40:16,  9.23s/it]
 98%|█████████▊| 220057/225000 [430:10:26<12:02:55,  8.78s/it]
 98%|█████████▊| 220058/225000 [430:10:32<10:58:20,  7.99s/it]
 98%|█████████▊| 220059/225000 [430:10:38<10:13:35,  7.45s/it]
 98%|█████████▊| 220060/225000 [430:10:44<9:32:05,  6.95s/it] 
 98%|█████████▊| 220061/225000 [430:10:52<9:51:52,  7.19s/it]
 98%|█████████▊| 220062/225000 [430:11:05<12:30:39,  9.12s/it]
 98%|█████████▊| 220063/225000 [430:11:21<14:57:05, 10.90s/it]
 98%|█████████▊| 220064/225000 [430:11:25<12:28:32,  9.10s/it]
 98%|█████████▊| 220065/225000 [430:11:36<12:52:28,  9.39s/it]
 98%|█████████▊| 220066/225000 [430:11:43<12:03:16,  8.80s/it]
 98%|█████████▊| 220067/225000 [430:11:52<12:16:22,  8.96s/it]
 98%|█████████▊| 220068/225000 [430:12:01<11:59:38,  8.75s/it]
 98%|█████████▊| 220069/225000 [430:12:10<12:16:03,  8.96s/it]
 98%|█████████▊| 220070/225000 [430:12:15<10:47:45,  7.88s/it]
 98%|█████████▊| 220071/225000 [430:12:22<10:08:01,  7.40s/it]
 98%|█████████▊| 220072/225000 [430:12:32<11:23:22,  8.32s/it]
 98%|█████████▊| 220073/225000 [430:12:40<11:16:02,  8.23s/it]
 98%|█████████▊| 220074/225000 [430:12:48<10:58:26,  8.02s/it]
 98%|█████████▊| 220075/225000 [430:13:01<13:00:52,  9.51s/it]
 98%|█████████▊| 220076/225000 [430:13:09<12:26:30,  9.10s/it]
 98%|█████████▊| 220077/225000 [430:13:19<12:50:39,  9.39s/it]
 98%|█████████▊| 220078/225000 [430:13:26<11:48:28,  8.64s/it]
 98%|█████████▊| 220079/225000 [430:13:34<11:41:29,  8.55s/it]
 98%|█████████▊| 220080/225000 [430:13:44<12:25:25,  9.09s/it]
 98%|█████████▊| 220081/225000 [430:13:51<11:16:12,  8.25s/it]
 98%|█████████▊| 220082/225000 [430:13:59<11:06:30,  8.13s/it]
 98%|█████████▊| 220083/225000 [430:14:05<10:17:31,  7.54s/it]
 98%|█████████▊| 220084/225000 [430:14:17<12:04:45,  8.85s/it]
 98%|█████████▊| 220085/225000 [430:14:24<11:29:00,  8.41s/it]
 98%|█████████▊| 220086/225000 [430:14:37<13:33:57,  9.94s/it]
 98%|█████████▊| 220087/225000 [430:14:43<11:50:16,  8.67s/it]
 98%|█████████▊| 220088/225000 [430:14:49<10:34:12,  7.75s/it]
 98%|█████████▊| 220089/225000 [430:14:57<10:43:17,  7.86s/it]
 98%|█████████▊| 220090/225000 [430:15:08<11:58:14,  8.78s/it]
 98%|█████████▊| 220091/225000 [430:15:14<10:53:49,  7.99s/it]
 98%|█████████▊| 220092/225000 [430:15:23<11:16:13,  8.27s/it]
 98%|█████████▊| 220093/225000 [430:15:31<11:01:02,  8.08s/it]
 98%|█████████▊| 220094/225000 [430:15:36<9:53:42,  7.26s/it] 
 98%|█████████▊| 220095/225000 [430:15:47<11:27:51,  8.41s/it]
 98%|█████████▊| 220096/225000 [430:15:57<11:54:23,  8.74s/it]
 98%|█████████▊| 220097/225000 [430:16:02<10:30:20,  7.71s/it]
 98%|█████████▊| 220098/225000 [430:16:10<10:47:07,  7.92s/it]
 98%|█████████▊| 220099/225000 [430:16:17<10:13:40,  7.51s/it]
 98%|█████████▊| 220100/225000 [430:16:29<12:03:27,  8.86s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.201', 'learning_rate': '6.677e-08', 'epoch': '0.9782', 'num_input_tokens_seen': 732898496, 'train_runtime': '1.549e+06', 'train_tokens_per_second': '473.1'}

 98%|█████████▊| 220100/225000 [430:16:29<12:03:27,  8.86s/it]
 98%|█████████▊| 220101/225000 [430:16:36<11:11:01,  8.22s/it]
 98%|█████████▊| 220102/225000 [430:16:46<12:11:14,  8.96s/it]
 98%|█████████▊| 220103/225000 [430:16:54<11:45:21,  8.64s/it]
 98%|█████████▊| 220104/225000 [430:17:01<10:56:52,  8.05s/it]
 98%|█████████▊| 220105/225000 [430:17:08<10:25:01,  7.66s/it]
 98%|█████████▊| 220106/225000 [430:17:12<9:17:32,  6.84s/it] 
 98%|█████████▊| 220107/225000 [430:17:27<12:31:26,  9.21s/it]
 98%|█████████▊| 220108/225000 [430:17:36<12:20:51,  9.09s/it]
 98%|█████████▊| 220109/225000 [430:17:43<11:22:44,  8.38s/it]
 98%|█████████▊| 220110/225000 [430:17:54<12:30:44,  9.21s/it]
 98%|█████████▊| 220111/225000 [430:18:05<13:26:39,  9.90s/it]
 98%|█████████▊| 220112/225000 [430:18:12<12:18:30,  9.07s/it]
 98%|█████████▊| 220113/225000 [430:18:21<12:16:22,  9.04s/it]
 98%|█████████▊| 220114/225000 [430:18:31<12:30:20,  9.21s/it]
 98%|█████████▊| 220115/225000 [430:18:41<12:54:36,  9.51s/it]
 98%|█████████▊| 220116/225000 [430:18:50<12:39:56,  9.34s/it]
 98%|█████████▊| 220117/225000 [430:18:59<12:34:27,  9.27s/it]
 98%|█████████▊| 220118/225000 [430:19:06<11:40:34,  8.61s/it]
 98%|█████████▊| 220119/225000 [430:19:15<11:38:01,  8.58s/it]
 98%|█████████▊| 220120/225000 [430:19:24<11:52:27,  8.76s/it]
 98%|█████████▊| 220121/225000 [430:19:30<10:39:27,  7.86s/it]
 98%|█████████▊| 220122/225000 [430:19:39<11:07:54,  8.22s/it]
 98%|█████████▊| 220123/225000 [430:19:46<10:47:41,  7.97s/it]
 98%|█████████▊| 220124/225000 [430:19:56<11:20:13,  8.37s/it]
 98%|█████████▊| 220125/225000 [430:20:02<10:38:06,  7.85s/it]
 98%|█████████▊| 220126/225000 [430:20:15<12:43:21,  9.40s/it]
 98%|█████████▊| 220127/225000 [430:20:24<12:24:50,  9.17s/it]
 98%|█████████▊| 220128/225000 [430:20:31<11:46:08,  8.70s/it]
 98%|█████████▊| 220129/225000 [430:20:44<13:12:53,  9.77s/it]
 98%|█████████▊| 220130/225000 [430:20:58<14:49:59, 10.97s/it]
 98%|█████████▊| 220131/225000 [430:21:09<15:03:43, 11.14s/it]
 98%|█████████▊| 220132/225000 [430:21:15<13:02:17,  9.64s/it]
 98%|█████████▊| 220133/225000 [430:21:22<11:49:45,  8.75s/it]
 98%|█████████▊| 220134/225000 [430:21:28<10:39:04,  7.88s/it]
 98%|█████████▊| 220135/225000 [430:21:34<10:04:07,  7.45s/it]
 98%|█████████▊| 220136/225000 [430:21:41<9:45:59,  7.23s/it] 
 98%|█████████▊| 220137/225000 [430:21:49<10:17:10,  7.61s/it]
 98%|█████████▊| 220138/225000 [430:21:56<9:58:43,  7.39s/it] 
 98%|█████████▊| 220139/225000 [430:22:05<10:30:24,  7.78s/it]
 98%|█████████▊| 220140/225000 [430:22:21<13:58:02, 10.35s/it]
 98%|█████████▊| 220141/225000 [430:22:27<12:03:26,  8.93s/it]
 98%|█████████▊| 220142/225000 [430:22:35<11:34:08,  8.57s/it]
 98%|█████████▊| 220143/225000 [430:22:42<11:11:10,  8.29s/it]
 98%|█████████▊| 220144/225000 [430:22:54<12:43:54,  9.44s/it]
 98%|█████████▊| 220145/225000 [430:23:03<12:12:28,  9.05s/it]
 98%|█████████▊| 220146/225000 [430:23:09<11:04:55,  8.22s/it]
 98%|█████████▊| 220147/225000 [430:23:15<10:11:19,  7.56s/it]
 98%|█████████▊| 220148/225000 [430:23:29<13:00:09,  9.65s/it]
 98%|█████████▊| 220149/225000 [430:23:37<12:01:44,  8.93s/it]
 98%|█████████▊| 220150/225000 [430:23:48<13:10:16,  9.78s/it]
                                                              
{'loss': '1.304', 'grad_norm': '1.152', 'learning_rate': '6.545e-08', 'epoch': '0.9784', 'num_input_tokens_seen': 733132760, 'train_runtime': '1.549e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220150/225000 [430:23:48<13:10:16,  9.78s/it]
 98%|█████████▊| 220151/225000 [430:23:56<12:28:46,  9.27s/it]
 98%|█████████▊| 220152/225000 [430:24:03<11:20:38,  8.42s/it]
 98%|█████████▊| 220153/225000 [430:24:14<12:23:36,  9.20s/it]
 98%|█████████▊| 220154/225000 [430:24:22<11:50:39,  8.80s/it]
 98%|█████████▊| 220155/225000 [430:24:33<12:42:52,  9.45s/it]
 98%|█████████▊| 220156/225000 [430:24:43<13:07:10,  9.75s/it]
 98%|█████████▊| 220157/225000 [430:24:49<11:23:55,  8.47s/it]
 98%|█████████▊| 220158/225000 [430:24:55<10:40:07,  7.93s/it]
 98%|█████████▊| 220159/225000 [430:25:03<10:24:57,  7.75s/it]
 98%|█████████▊| 220160/225000 [430:25:15<12:21:34,  9.19s/it]
 98%|█████████▊| 220161/225000 [430:25:21<11:07:11,  8.27s/it]
 98%|█████████▊| 220162/225000 [430:25:28<10:32:17,  7.84s/it]
 98%|█████████▊| 220163/225000 [430:25:40<12:03:30,  8.97s/it]
 98%|█████████▊| 220164/225000 [430:25:48<11:37:51,  8.66s/it]
 98%|█████████▊| 220165/225000 [430:25:54<10:30:10,  7.82s/it]
 98%|█████████▊| 220166/225000 [430:26:06<12:19:50,  9.18s/it]
 98%|█████████▊| 220167/225000 [430:26:11<10:44:18,  8.00s/it]
 98%|█████████▊| 220168/225000 [430:26:18<10:11:55,  7.60s/it]
 98%|█████████▊| 220169/225000 [430:26:26<10:32:18,  7.85s/it]
 98%|█████████▊| 220170/225000 [430:26:36<11:27:18,  8.54s/it]
 98%|█████████▊| 220171/225000 [430:26:47<12:05:11,  9.01s/it]
 98%|█████████▊| 220172/225000 [430:26:54<11:23:08,  8.49s/it]
 98%|█████████▊| 220173/225000 [430:27:00<10:22:08,  7.73s/it]
 98%|█████████▊| 220174/225000 [430:27:06<9:35:44,  7.16s/it] 
 98%|█████████▊| 220175/225000 [430:27:14<9:54:12,  7.39s/it]
 98%|█████████▊| 220176/225000 [430:27:20<9:34:12,  7.14s/it]
 98%|█████████▊| 220177/225000 [430:27:34<12:11:53,  9.11s/it]
 98%|█████████▊| 220178/225000 [430:27:40<11:02:08,  8.24s/it]
 98%|█████████▊| 220179/225000 [430:27:46<10:02:33,  7.50s/it]
 98%|█████████▊| 220180/225000 [430:27:52<9:30:49,  7.11s/it] 
 98%|█████████▊| 220181/225000 [430:28:05<11:45:45,  8.79s/it]
 98%|█████████▊| 220182/225000 [430:28:13<11:36:00,  8.67s/it]
 98%|█████████▊| 220183/225000 [430:28:20<10:53:38,  8.14s/it]
 98%|█████████▊| 220184/225000 [430:28:25<9:39:34,  7.22s/it] 
 98%|█████████▊| 220185/225000 [430:28:32<9:43:00,  7.26s/it]
 98%|█████████▊| 220186/225000 [430:28:48<13:05:24,  9.79s/it]
 98%|█████████▊| 220187/225000 [430:28:55<11:50:59,  8.86s/it]
 98%|█████████▊| 220188/225000 [430:29:02<11:04:05,  8.28s/it]
 98%|█████████▊| 220189/225000 [430:29:12<11:52:04,  8.88s/it]
 98%|█████████▊| 220190/225000 [430:29:24<12:57:33,  9.70s/it]
 98%|█████████▊| 220191/225000 [430:29:33<12:40:03,  9.48s/it]
 98%|█████████▊| 220192/225000 [430:29:47<14:28:17, 10.84s/it]
 98%|█████████▊| 220193/225000 [430:29:53<12:48:53,  9.60s/it]
 98%|█████████▊| 220194/225000 [430:30:01<11:55:28,  8.93s/it]
 98%|█████████▊| 220195/225000 [430:30:06<10:33:03,  7.91s/it]
 98%|█████████▊| 220196/225000 [430:30:19<12:35:15,  9.43s/it]
 98%|█████████▊| 220197/225000 [430:30:24<10:47:39,  8.09s/it]
 98%|█████████▊| 220198/225000 [430:30:31<10:18:04,  7.72s/it]
 98%|█████████▊| 220199/225000 [430:30:44<12:26:36,  9.33s/it]
 98%|█████████▊| 220200/225000 [430:30:50<11:13:41,  8.42s/it]
                                                              
{'loss': '1.297', 'grad_norm': '2.222', 'learning_rate': '6.413e-08', 'epoch': '0.9787', 'num_input_tokens_seen': 733356776, 'train_runtime': '1.55e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220200/225000 [430:30:50<11:13:41,  8.42s/it]
 98%|█████████▊| 220201/225000 [430:30:57<10:37:06,  7.97s/it]
 98%|█████████▊| 220202/225000 [430:31:04<10:12:38,  7.66s/it]
 98%|█████████▊| 220203/225000 [430:31:15<11:26:43,  8.59s/it]
 98%|█████████▊| 220204/225000 [430:31:28<13:22:55, 10.04s/it]
 98%|█████████▊| 220205/225000 [430:31:35<11:59:43,  9.01s/it]
 98%|█████████▊| 220206/225000 [430:31:44<11:59:49,  9.01s/it]
 98%|█████████▊| 220207/225000 [430:31:54<12:21:00,  9.28s/it]
 98%|█████████▊| 220208/225000 [430:32:04<12:35:17,  9.46s/it]
 98%|█████████▊| 220209/225000 [430:32:09<11:02:25,  8.30s/it]
 98%|█████████▊| 220210/225000 [430:32:18<11:09:45,  8.39s/it]
 98%|█████████▊| 220211/225000 [430:32:26<10:51:46,  8.17s/it]
 98%|█████████▊| 220212/225000 [430:32:38<12:35:56,  9.47s/it]
 98%|█████████▊| 220213/225000 [430:32:50<13:23:35, 10.07s/it]
 98%|█████████▊| 220214/225000 [430:32:58<12:33:25,  9.45s/it]
 98%|█████████▊| 220215/225000 [430:33:05<11:33:12,  8.69s/it]
 98%|█████████▊| 220216/225000 [430:33:17<13:05:47,  9.86s/it]
 98%|█████████▊| 220217/225000 [430:33:23<11:40:33,  8.79s/it]
 98%|█████████▊| 220218/225000 [430:33:38<13:49:03, 10.40s/it]
 98%|█████████▊| 220219/225000 [430:33:44<12:20:28,  9.29s/it]
 98%|█████████▊| 220220/225000 [430:34:00<14:43:26, 11.09s/it]
 98%|█████████▊| 220221/225000 [430:34:08<13:29:41, 10.17s/it]
 98%|█████████▊| 220222/225000 [430:34:16<12:40:15,  9.55s/it]
 98%|█████████▊| 220223/225000 [430:34:26<13:03:47,  9.84s/it]
 98%|█████████▊| 220224/225000 [430:34:35<12:25:36,  9.37s/it]
 98%|█████████▊| 220225/225000 [430:34:44<12:35:45,  9.50s/it]
 98%|█████████▊| 220226/225000 [430:34:52<11:58:32,  9.03s/it]
 98%|█████████▊| 220227/225000 [430:34:58<10:34:01,  7.97s/it]
 98%|█████████▊| 220228/225000 [430:35:06<10:31:58,  7.95s/it]
 98%|█████████▊| 220229/225000 [430:35:22<13:48:27, 10.42s/it]
 98%|█████████▊| 220230/225000 [430:35:29<12:25:18,  9.38s/it]
 98%|█████████▊| 220231/225000 [430:35:35<11:10:08,  8.43s/it]
 98%|█████████▊| 220232/225000 [430:35:42<10:45:34,  8.12s/it]
 98%|█████████▊| 220233/225000 [430:35:56<12:55:04,  9.76s/it]
 98%|█████████▊| 220234/225000 [430:36:01<11:05:03,  8.37s/it]
 98%|█████████▊| 220235/225000 [430:36:08<10:27:32,  7.90s/it]
 98%|█████████▊| 220236/225000 [430:36:16<10:33:48,  7.98s/it]
 98%|█████████▊| 220237/225000 [430:36:24<10:42:33,  8.09s/it]
 98%|█████████▊| 220238/225000 [430:36:31<10:04:33,  7.62s/it]
 98%|█████████▊| 220239/225000 [430:36:47<13:25:28, 10.15s/it]
 98%|█████████▊| 220240/225000 [430:36:55<12:33:12,  9.49s/it]
 98%|█████████▊| 220241/225000 [430:37:05<12:48:29,  9.69s/it]
 98%|█████████▊| 220242/225000 [430:37:13<12:00:46,  9.09s/it]
 98%|█████████▊| 220243/225000 [430:37:21<11:37:56,  8.80s/it]
 98%|█████████▊| 220244/225000 [430:37:36<14:01:25, 10.62s/it]
 98%|█████████▊| 220245/225000 [430:37:44<13:14:48, 10.03s/it]
 98%|█████████▊| 220246/225000 [430:37:51<11:42:00,  8.86s/it]
 98%|█████████▊| 220247/225000 [430:38:02<12:52:46,  9.76s/it]
 98%|█████████▊| 220248/225000 [430:38:10<11:57:25,  9.06s/it]
 98%|█████████▊| 220249/225000 [430:38:20<12:25:42,  9.42s/it]
 98%|█████████▊| 220250/225000 [430:38:30<12:45:48,  9.67s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.355', 'learning_rate': '6.283e-08', 'epoch': '0.9789', 'num_input_tokens_seen': 733593408, 'train_runtime': '1.55e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220250/225000 [430:38:30<12:45:48,  9.67s/it]
 98%|█████████▊| 220251/225000 [430:38:35<10:40:17,  8.09s/it]
 98%|█████████▊| 220252/225000 [430:38:42<10:17:22,  7.80s/it]
 98%|█████████▊| 220253/225000 [430:38:50<10:12:58,  7.75s/it]
 98%|█████████▊| 220254/225000 [430:39:05<13:07:14,  9.95s/it]
 98%|█████████▊| 220255/225000 [430:39:13<12:32:21,  9.51s/it]
 98%|█████████▊| 220256/225000 [430:39:20<11:38:29,  8.83s/it]
 98%|█████████▊| 220257/225000 [430:39:27<10:49:54,  8.22s/it]
 98%|█████████▊| 220258/225000 [430:39:36<11:01:43,  8.37s/it]
 98%|█████████▊| 220259/225000 [430:39:41<9:53:41,  7.51s/it] 
 98%|█████████▊| 220260/225000 [430:39:52<10:59:03,  8.34s/it]
 98%|█████████▊| 220261/225000 [430:40:02<11:49:14,  8.98s/it]
 98%|█████████▊| 220262/225000 [430:40:07<10:13:20,  7.77s/it]
 98%|█████████▊| 220263/225000 [430:40:17<10:57:31,  8.33s/it]
 98%|█████████▊| 220264/225000 [430:40:22<9:46:27,  7.43s/it] 
 98%|█████████▊| 220265/225000 [430:40:33<11:18:18,  8.60s/it]
 98%|█████████▊| 220266/225000 [430:40:41<10:49:42,  8.23s/it]
 98%|█████████▊| 220267/225000 [430:40:53<12:17:47,  9.35s/it]
 98%|█████████▊| 220268/225000 [430:41:00<11:21:25,  8.64s/it]
 98%|█████████▊| 220269/225000 [430:41:08<11:19:54,  8.62s/it]
 98%|█████████▊| 220270/225000 [430:41:21<12:54:23,  9.82s/it]
 98%|█████████▊| 220271/225000 [430:41:29<12:07:37,  9.23s/it]
 98%|█████████▊| 220272/225000 [430:41:37<11:33:28,  8.80s/it]
 98%|█████████▊| 220273/225000 [430:41:46<11:50:46,  9.02s/it]
 98%|█████████▊| 220274/225000 [430:41:52<10:32:09,  8.03s/it]
 98%|█████████▊| 220275/225000 [430:41:57<9:25:52,  7.19s/it] 
 98%|█████████▊| 220276/225000 [430:42:11<12:04:49,  9.21s/it]
 98%|█████████▊| 220277/225000 [430:42:23<13:20:19, 10.17s/it]
 98%|█████████▊| 220278/225000 [430:42:31<12:21:41,  9.42s/it]
 98%|█████████▊| 220279/225000 [430:42:43<13:18:20, 10.15s/it]
 98%|█████████▊| 220280/225000 [430:42:50<12:11:56,  9.30s/it]
 98%|█████████▊| 220281/225000 [430:42:58<11:29:08,  8.76s/it]
 98%|█████████▊| 220282/225000 [430:43:03<10:12:31,  7.79s/it]
 98%|█████████▊| 220283/225000 [430:43:10<9:58:42,  7.62s/it] 
 98%|█████████▊| 220284/225000 [430:43:22<11:30:55,  8.79s/it]
 98%|█████████▊| 220285/225000 [430:43:30<11:23:47,  8.70s/it]
 98%|█████████▊| 220286/225000 [430:43:40<11:47:07,  9.00s/it]
 98%|█████████▊| 220287/225000 [430:43:47<11:01:04,  8.42s/it]
 98%|█████████▊| 220288/225000 [430:43:57<11:25:13,  8.73s/it]
 98%|█████████▊| 220289/225000 [430:44:04<11:01:24,  8.42s/it]
 98%|█████████▊| 220290/225000 [430:44:12<10:35:13,  8.09s/it]
 98%|█████████▊| 220291/225000 [430:44:18<9:51:08,  7.53s/it] 
 98%|█████████▊| 220292/225000 [430:44:31<12:04:34,  9.23s/it]
 98%|█████████▊| 220293/225000 [430:44:39<11:40:32,  8.93s/it]
 98%|█████████▊| 220294/225000 [430:44:52<13:07:08, 10.04s/it]
 98%|█████████▊| 220295/225000 [430:44:57<11:21:10,  8.69s/it]
 98%|█████████▊| 220296/225000 [430:45:04<10:40:37,  8.17s/it]
 98%|█████████▊| 220297/225000 [430:45:13<10:51:49,  8.32s/it]
 98%|█████████▊| 220298/225000 [430:45:24<11:45:58,  9.01s/it]
 98%|█████████▊| 220299/225000 [430:45:28<10:05:32,  7.73s/it]
 98%|█████████▊| 220300/225000 [430:45:41<12:03:58,  9.24s/it]
                                                              
{'loss': '1.305', 'grad_norm': '1.29', 'learning_rate': '6.157e-08', 'epoch': '0.9791', 'num_input_tokens_seen': 733824672, 'train_runtime': '1.551e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220300/225000 [430:45:41<12:03:58,  9.24s/it]
 98%|█████████▊| 220301/225000 [430:45:47<10:33:16,  8.09s/it]
 98%|█████████▊| 220302/225000 [430:46:04<14:06:01, 10.80s/it]
 98%|█████████▊| 220303/225000 [430:46:13<13:20:13, 10.22s/it]
 98%|█████████▊| 220304/225000 [430:46:19<11:51:57,  9.10s/it]
 98%|█████████▊| 220305/225000 [430:46:31<12:50:11,  9.84s/it]
 98%|█████████▊| 220306/225000 [430:46:38<11:56:02,  9.15s/it]
 98%|█████████▊| 220307/225000 [430:46:47<11:45:30,  9.02s/it]
 98%|█████████▊| 220308/225000 [430:46:52<10:18:47,  7.91s/it]
 98%|█████████▊| 220309/225000 [430:47:02<11:02:48,  8.48s/it]
 98%|█████████▊| 220310/225000 [430:47:08<10:03:54,  7.73s/it]
 98%|█████████▊| 220311/225000 [430:47:14<9:28:21,  7.27s/it] 
 98%|█████████▊| 220312/225000 [430:47:22<9:44:04,  7.48s/it]
 98%|█████████▊| 220313/225000 [430:47:39<13:18:15, 10.22s/it]
 98%|█████████▊| 220314/225000 [430:47:53<14:49:35, 11.39s/it]
 98%|█████████▊| 220315/225000 [430:47:59<12:36:00,  9.68s/it]
 98%|█████████▊| 220316/225000 [430:48:07<12:01:18,  9.24s/it]
 98%|█████████▊| 220317/225000 [430:48:15<11:27:46,  8.81s/it]
 98%|█████████▊| 220318/225000 [430:48:24<11:37:07,  8.93s/it]
 98%|█████████▊| 220319/225000 [430:48:34<12:03:30,  9.27s/it]
 98%|█████████▊| 220320/225000 [430:48:40<10:40:22,  8.21s/it]
 98%|█████████▊| 220321/225000 [430:48:48<10:44:17,  8.26s/it]
 98%|█████████▊| 220322/225000 [430:49:00<12:17:34,  9.46s/it]
 98%|█████████▊| 220323/225000 [430:49:07<11:21:22,  8.74s/it]
 98%|█████████▊| 220324/225000 [430:49:17<11:45:26,  9.05s/it]
 98%|█████████▊| 220325/225000 [430:49:28<12:26:06,  9.58s/it]
 98%|█████████▊| 220326/225000 [430:49:35<11:17:45,  8.70s/it]
 98%|█████████▊| 220327/225000 [430:49:46<12:17:05,  9.46s/it]
 98%|█████████▊| 220328/225000 [430:49:58<13:24:24, 10.33s/it]
 98%|█████████▊| 220329/225000 [430:50:07<12:50:58,  9.90s/it]
 98%|█████████▊| 220330/225000 [430:50:16<12:33:29,  9.68s/it]
 98%|█████████▊| 220331/225000 [430:50:28<13:29:23, 10.40s/it]
 98%|█████████▊| 220332/225000 [430:50:40<14:03:09, 10.84s/it]
 98%|█████████▊| 220333/225000 [430:50:49<13:14:36, 10.22s/it]
 98%|█████████▊| 220334/225000 [430:50:58<12:49:58,  9.90s/it]
 98%|█████████▊| 220335/225000 [430:51:12<14:24:00, 11.11s/it]
 98%|█████████▊| 220336/225000 [430:51:18<12:29:25,  9.64s/it]
 98%|█████████▊| 220337/225000 [430:51:25<11:17:49,  8.72s/it]
 98%|█████████▊| 220338/225000 [430:51:34<11:23:31,  8.80s/it]
 98%|█████████▊| 220339/225000 [430:51:46<12:31:34,  9.67s/it]
 98%|█████████▊| 220340/225000 [430:51:51<11:01:20,  8.52s/it]
 98%|█████████▊| 220341/225000 [430:51:57<9:49:25,  7.59s/it] 
 98%|█████████▊| 220342/225000 [430:52:08<11:10:05,  8.63s/it]
 98%|█████████▊| 220343/225000 [430:52:15<10:24:23,  8.04s/it]
 98%|█████████▊| 220344/225000 [430:52:22<10:10:41,  7.87s/it]
 98%|█████████▊| 220345/225000 [430:52:31<10:38:01,  8.22s/it]
 98%|█████████▊| 220346/225000 [430:52:44<12:26:55,  9.63s/it]
 98%|█████████▊| 220347/225000 [430:52:55<12:50:23,  9.93s/it]
 98%|█████████▊| 220348/225000 [430:53:00<11:08:08,  8.62s/it]
 98%|█████████▊| 220349/225000 [430:53:09<11:15:25,  8.71s/it]
 98%|█████████▊| 220350/225000 [430:53:16<10:27:02,  8.09s/it]
                                                              
{'loss': '1.327', 'grad_norm': '1.636', 'learning_rate': '6.029e-08', 'epoch': '0.9793', 'num_input_tokens_seen': 734065872, 'train_runtime': '1.551e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220350/225000 [430:53:16<10:27:02,  8.09s/it]
 98%|█████████▊| 220351/225000 [430:53:23<10:12:10,  7.90s/it]
 98%|█████████▊| 220352/225000 [430:53:28<8:53:49,  6.89s/it] 
 98%|█████████▊| 220353/225000 [430:53:35<8:55:06,  6.91s/it]
 98%|█████████▊| 220354/225000 [430:53:43<9:16:14,  7.18s/it]
 98%|█████████▊| 220355/225000 [430:53:56<11:43:41,  9.09s/it]
 98%|█████████▊| 220356/225000 [430:54:05<11:47:38,  9.14s/it]
 98%|█████████▊| 220357/225000 [430:54:11<10:16:57,  7.97s/it]
 98%|█████████▊| 220358/225000 [430:54:22<11:40:59,  9.06s/it]
 98%|█████████▊| 220359/225000 [430:54:34<12:37:33,  9.79s/it]
 98%|█████████▊| 220360/225000 [430:54:40<11:07:07,  8.63s/it]
 98%|█████████▊| 220361/225000 [430:54:56<14:02:17, 10.89s/it]
 98%|█████████▊| 220362/225000 [430:55:05<13:24:25, 10.41s/it]
 98%|█████████▊| 220363/225000 [430:55:11<11:35:48,  9.00s/it]
 98%|█████████▊| 220364/225000 [430:55:18<10:45:45,  8.36s/it]
 98%|█████████▊| 220365/225000 [430:55:28<11:30:21,  8.94s/it]
 98%|█████████▊| 220366/225000 [430:55:35<10:56:01,  8.49s/it]
 98%|█████████▊| 220367/225000 [430:55:42<10:11:21,  7.92s/it]
 98%|█████████▊| 220368/225000 [430:55:55<12:10:24,  9.46s/it]
 98%|█████████▊| 220369/225000 [430:56:02<11:22:08,  8.84s/it]
 98%|█████████▊| 220370/225000 [430:56:14<12:25:56,  9.67s/it]
 98%|█████████▊| 220371/225000 [430:56:19<10:49:22,  8.42s/it]
 98%|█████████▊| 220372/225000 [430:56:30<11:39:06,  9.06s/it]
 98%|█████████▊| 220373/225000 [430:56:36<10:25:35,  8.11s/it]
 98%|█████████▊| 220374/225000 [430:56:42<9:27:33,  7.36s/it] 
 98%|█████████▊| 220375/225000 [430:56:51<10:06:15,  7.86s/it]
 98%|█████████▊| 220376/225000 [430:57:03<11:52:08,  9.24s/it]
 98%|█████████▊| 220377/225000 [430:57:09<10:41:01,  8.32s/it]
 98%|█████████▊| 220378/225000 [430:57:17<10:31:23,  8.20s/it]
 98%|█████████▊| 220379/225000 [430:57:28<11:35:57,  9.04s/it]
 98%|█████████▊| 220380/225000 [430:57:37<11:39:16,  9.08s/it]
 98%|█████████▊| 220381/225000 [430:57:46<11:27:23,  8.93s/it]
 98%|█████████▊| 220382/225000 [430:57:57<12:12:17,  9.51s/it]
 98%|█████████▊| 220383/225000 [430:58:09<13:05:37, 10.21s/it]
 98%|█████████▊| 220384/225000 [430:58:17<12:21:53,  9.64s/it]
 98%|█████████▊| 220385/225000 [430:58:25<11:46:02,  9.18s/it]
 98%|█████████▊| 220386/225000 [430:58:38<13:24:49, 10.47s/it]
 98%|█████████▊| 220387/225000 [430:58:45<11:50:29,  9.24s/it]
 98%|█████████▊| 220388/225000 [430:58:50<10:23:01,  8.11s/it]
 98%|█████████▊| 220389/225000 [430:59:02<11:36:48,  9.07s/it]
 98%|█████████▊| 220390/225000 [430:59:13<12:38:52,  9.88s/it]
 98%|█████████▊| 220391/225000 [430:59:23<12:21:41,  9.66s/it]
 98%|█████████▊| 220392/225000 [430:59:29<11:04:03,  8.65s/it]
 98%|█████████▊| 220393/225000 [430:59:44<13:26:13, 10.50s/it]
 98%|█████████▊| 220394/225000 [430:59:50<11:48:13,  9.23s/it]
 98%|█████████▊| 220395/225000 [431:00:02<12:47:15, 10.00s/it]
 98%|█████████▊| 220396/225000 [431:00:07<11:05:03,  8.67s/it]
 98%|█████████▊| 220397/225000 [431:00:18<11:56:12,  9.34s/it]
 98%|█████████▊| 220398/225000 [431:00:25<10:52:24,  8.51s/it]
 98%|█████████▊| 220399/225000 [431:00:34<11:07:24,  8.70s/it]
 98%|█████████▊| 220400/225000 [431:00:47<12:59:47, 10.17s/it]
                                                              
{'loss': '1.344', 'grad_norm': '4.019', 'learning_rate': '5.903e-08', 'epoch': '0.9796', 'num_input_tokens_seen': 734308328, 'train_runtime': '1.552e+06', 'train_tokens_per_second': '473.2'}

 98%|█████████▊| 220400/225000 [431:00:47<12:59:47, 10.17s/it]
 98%|█████████▊| 220401/225000 [431:00:55<12:02:49,  9.43s/it]
 98%|█████████▊| 220402/225000 [431:01:02<10:53:28,  8.53s/it]
 98%|█████████▊| 220403/225000 [431:01:08<10:12:45,  8.00s/it]
 98%|█████████▊| 220404/225000 [431:01:22<12:23:15,  9.70s/it]
 98%|█████████▊| 220405/225000 [431:01:31<12:13:13,  9.57s/it]
 98%|█████████▊| 220406/225000 [431:01:42<12:29:57,  9.79s/it]
 98%|█████████▊| 220407/225000 [431:01:51<12:26:04,  9.75s/it]
 98%|█████████▊| 220408/225000 [431:02:00<12:12:11,  9.57s/it]
 98%|█████████▊| 220409/225000 [431:02:06<10:43:15,  8.41s/it]
 98%|█████████▊| 220410/225000 [431:02:12<9:41:53,  7.61s/it] 
 98%|█████████▊| 220411/225000 [431:02:26<12:07:59,  9.52s/it]
 98%|█████████▊| 220412/225000 [431:02:32<10:48:14,  8.48s/it]
 98%|█████████▊| 220413/225000 [431:02:42<11:33:48,  9.08s/it]
 98%|█████████▊| 220414/225000 [431:02:50<11:00:09,  8.64s/it]
 98%|█████████▊| 220415/225000 [431:03:00<11:24:57,  8.96s/it]
 98%|█████████▊| 220416/225000 [431:03:08<11:09:28,  8.76s/it]
 98%|█████████▊| 220417/225000 [431:03:14<10:15:11,  8.05s/it]
 98%|█████████▊| 220418/225000 [431:03:22<9:59:05,  7.84s/it] 
 98%|█████████▊| 220419/225000 [431:03:28<9:14:16,  7.26s/it]
 98%|█████████▊| 220420/225000 [431:03:42<11:46:28,  9.26s/it]
 98%|█████████▊| 220421/225000 [431:03:51<11:49:51,  9.30s/it]
 98%|█████████▊| 220422/225000 [431:03:57<10:34:06,  8.31s/it]
 98%|█████████▊| 220423/225000 [431:04:03<9:40:21,  7.61s/it] 
 98%|█████████▊| 220424/225000 [431:04:10<9:29:51,  7.47s/it]
 98%|█████████▊| 220425/225000 [431:04:21<10:40:09,  8.40s/it]
 98%|█████████▊| 220426/225000 [431:04:27<10:01:00,  7.88s/it]
 98%|█████████▊| 220427/225000 [431:04:33<9:01:14,  7.10s/it] 
 98%|█████████▊| 220428/225000 [431:04:46<11:32:30,  9.09s/it]
 98%|█████████▊| 220429/225000 [431:04:54<10:58:33,  8.64s/it]
 98%|█████████▊| 220430/225000 [431:05:02<10:48:09,  8.51s/it]
 98%|█████████▊| 220431/225000 [431:05:17<13:06:17, 10.33s/it]
 98%|█████████▊| 220432/225000 [431:05:23<11:36:02,  9.14s/it]
 98%|█████████▊| 220433/225000 [431:05:30<10:39:30,  8.40s/it]
 98%|█████████▊| 220434/225000 [431:05:43<12:21:55,  9.75s/it]
 98%|█████████▊| 220435/225000 [431:05:50<11:24:58,  9.00s/it]
 98%|█████████▊| 220436/225000 [431:05:58<11:11:05,  8.82s/it]
 98%|█████████▊| 220437/225000 [431:06:07<11:13:04,  8.85s/it]
 98%|█████████▊| 220438/225000 [431:06:12<9:38:10,  7.60s/it] 
 98%|█████████▊| 220439/225000 [431:06:25<11:53:49,  9.39s/it]
 98%|█████████▊| 220440/225000 [431:06:32<10:47:18,  8.52s/it]
 98%|█████████▊| 220441/225000 [431:06:38<9:55:47,  7.84s/it] 
 98%|█████████▊| 220442/225000 [431:06:48<10:50:46,  8.57s/it]
 98%|█████████▊| 220443/225000 [431:06:57<10:40:43,  8.44s/it]
 98%|█████████▊| 220444/225000 [431:07:09<12:10:04,  9.61s/it]
 98%|█████████▊| 220445/225000 [431:07:18<11:58:46,  9.47s/it]
 98%|█████████▊| 220446/225000 [431:07:25<10:50:58,  8.58s/it]
 98%|█████████▊| 220447/225000 [431:07:31<10:08:21,  8.02s/it]
 98%|█████████▊| 220448/225000 [431:07:37<9:24:09,  7.44s/it] 
 98%|█████████▊| 220449/225000 [431:07:45<9:19:26,  7.38s/it]
 98%|█████████▊| 220450/225000 [431:07:51<8:49:23,  6.98s/it]
                                                             
{'loss': '1.32', 'grad_norm': '1.535', 'learning_rate': '5.779e-08', 'epoch': '0.9798', 'num_input_tokens_seen': 734540128, 'train_runtime': '1.552e+06', 'train_tokens_per_second': '473.3'}

 98%|█████████▊| 220450/225000 [431:07:51<8:49:23,  6.98s/it]
 98%|█████████▊| 220451/225000 [431:07:56<8:09:32,  6.46s/it]
 98%|█████████▊| 220452/225000 [431:08:08<10:06:16,  8.00s/it]
 98%|█████████▊| 220453/225000 [431:08:19<11:24:49,  9.04s/it]
 98%|█████████▊| 220454/225000 [431:08:25<10:09:07,  8.04s/it]
 98%|█████████▊| 220455/225000 [431:08:34<10:47:28,  8.55s/it]
 98%|█████████▊| 220456/225000 [431:08:45<11:28:12,  9.09s/it]
 98%|█████████▊| 220457/225000 [431:08:52<10:36:57,  8.41s/it]
 98%|█████████▊| 220458/225000 [431:08:58<9:52:42,  7.83s/it] 
 98%|█████████▊| 220459/225000 [431:09:05<9:24:45,  7.46s/it]
 98%|█████████▊| 220460/225000 [431:09:12<9:14:41,  7.33s/it]
 98%|█████████▊| 220461/225000 [431:09:21<10:02:26,  7.96s/it]
 98%|█████████▊| 220462/225000 [431:09:37<12:55:55, 10.26s/it]
 98%|█████████▊| 220463/225000 [431:09:45<12:08:28,  9.63s/it]
 98%|█████████▊| 220464/225000 [431:09:53<11:23:04,  9.04s/it]
 98%|█████████▊| 220465/225000 [431:10:01<11:04:27,  8.79s/it]
 98%|█████████▊| 220466/225000 [431:10:16<13:20:20, 10.59s/it]
 98%|█████████▊| 220467/225000 [431:10:25<13:03:12, 10.37s/it]
 98%|█████████▊| 220468/225000 [431:10:32<11:42:34,  9.30s/it]
 98%|█████████▊| 220469/225000 [431:10:40<11:02:44,  8.78s/it]
 98%|█████████▊| 220470/225000 [431:10:45<9:51:57,  7.84s/it] 
 98%|█████████▊| 220471/225000 [431:10:59<11:52:11,  9.44s/it]
 98%|█████████▊| 220472/225000 [431:11:10<12:45:17, 10.14s/it]
 98%|█████████▊| 220473/225000 [431:11:18<11:36:50,  9.24s/it]
 98%|█████████▊| 220474/225000 [431:11:25<10:46:02,  8.56s/it]
 98%|█████████▊| 220475/225000 [431:11:32<10:12:58,  8.13s/it]
 98%|█████████▊| 220476/225000 [431:11:38<9:36:54,  7.65s/it] 
 98%|█████████▊| 220477/225000 [431:11:48<10:22:50,  8.26s/it]
 98%|█████████▊| 220478/225000 [431:11:59<11:33:50,  9.21s/it]
 98%|█████████▊| 220479/225000 [431:12:07<11:00:42,  8.77s/it]
 98%|█████████▊| 220480/225000 [431:12:17<11:34:17,  9.22s/it]
 98%|█████████▊| 220481/225000 [431:12:31<13:05:59, 10.44s/it]
 98%|█████████▊| 220482/225000 [431:12:37<11:45:00,  9.36s/it]
 98%|█████████▊| 220483/225000 [431:12:50<12:47:02, 10.19s/it]
 98%|█████████▊| 220484/225000 [431:12:57<11:39:28,  9.29s/it]
 98%|█████████▊| 220485/225000 [431:13:08<12:23:23,  9.88s/it]
 98%|█████████▊| 220486/225000 [431:13:13<10:27:23,  8.34s/it]
 98%|█████████▊| 220487/225000 [431:13:23<11:06:08,  8.86s/it]
 98%|█████████▊| 220488/225000 [431:13:32<11:08:15,  8.89s/it]
 98%|█████████▊| 220489/225000 [431:13:43<12:11:55,  9.74s/it]
 98%|█████████▊| 220490/225000 [431:13:53<12:01:23,  9.60s/it]
 98%|█████████▊| 220491/225000 [431:14:04<12:40:05, 10.11s/it]
 98%|█████████▊| 220492/225000 [431:14:11<11:25:57,  9.13s/it]
 98%|█████████▊| 220493/225000 [431:14:16<10:04:28,  8.05s/it]
 98%|█████████▊| 220494/225000 [431:14:25<10:27:07,  8.35s/it]
 98%|█████████▊| 220495/225000 [431:14:37<11:40:20,  9.33s/it]
 98%|█████████▊| 220496/225000 [431:14:47<11:47:02,  9.42s/it]
 98%|█████████▊| 220497/225000 [431:15:02<14:04:50, 11.26s/it]
 98%|█████████▊| 220498/225000 [431:15:09<12:26:35,  9.95s/it]
 98%|█████████▊| 220499/225000 [431:15:19<12:30:54, 10.01s/it]
 98%|█████████▊| 220500/225000 [431:15:26<11:10:41,  8.94s/it]
                                                              
{'loss': '1.315', 'grad_norm': '2.692', 'learning_rate': '5.655e-08', 'epoch': '0.98', 'num_input_tokens_seen': 734781176, 'train_runtime': '1.553e+06', 'train_tokens_per_second': '473.3'}

 98%|█████████▊| 220500/225000 [431:15:26<11:10:41,  8.94s/it]
 98%|█████████▊| 220501/225000 [431:15:32<10:20:17,  8.27s/it]
 98%|█████████▊| 220502/225000 [431:15:39<9:49:32,  7.86s/it] 
 98%|█████████▊| 220503/225000 [431:15:48<10:15:28,  8.21s/it]
 98%|█████████▊| 220504/225000 [431:16:03<12:39:29, 10.14s/it]
 98%|█████████▊| 220505/225000 [431:16:11<12:01:57,  9.64s/it]
 98%|█████████▊| 220506/225000 [431:16:20<11:30:49,  9.22s/it]
 98%|█████████▊| 220507/225000 [431:16:29<11:39:33,  9.34s/it]
 98%|█████████▊| 220508/225000 [431:16:36<10:42:46,  8.59s/it]
 98%|█████████▊| 220509/225000 [431:16:47<11:26:33,  9.17s/it]
 98%|█████████▊| 220510/225000 [431:16:57<11:50:29,  9.49s/it]
 98%|█████████▊| 220511/225000 [431:17:03<10:28:45,  8.40s/it]
 98%|█████████▊| 220512/225000 [431:17:08<9:13:55,  7.41s/it] 
 98%|█████████▊| 220513/225000 [431:17:19<10:30:05,  8.43s/it]
 98%|█████████▊| 220514/225000 [431:17:27<10:21:15,  8.31s/it]
 98%|█████████▊| 220515/225000 [431:17:40<12:02:45,  9.67s/it]
 98%|█████████▊| 220516/225000 [431:17:46<10:58:35,  8.81s/it]
 98%|█████████▊| 220517/225000 [431:17:58<11:54:19,  9.56s/it]
 98%|█████████▊| 220518/225000 [431:18:05<11:09:13,  8.96s/it]
 98%|█████████▊| 220519/225000 [431:18:12<10:14:42,  8.23s/it]
 98%|█████████▊| 220520/225000 [431:18:22<10:49:25,  8.70s/it]
 98%|█████████▊| 220521/225000 [431:18:31<11:01:44,  8.86s/it]
 98%|█████████▊| 220522/225000 [431:18:40<11:19:17,  9.10s/it]
 98%|█████████▊| 220523/225000 [431:18:45<9:32:49,  7.68s/it] 
 98%|█████████▊| 220524/225000 [431:18:55<10:27:07,  8.41s/it]
 98%|█████████▊| 220525/225000 [431:19:06<11:35:08,  9.32s/it]
 98%|█████████▊| 220526/225000 [431:19:13<10:34:48,  8.51s/it]
 98%|█████████▊| 220527/225000 [431:19:22<10:34:38,  8.51s/it]
 98%|█████████▊| 220528/225000 [431:19:27<9:36:29,  7.73s/it] 
 98%|█████████▊| 220529/225000 [431:19:38<10:44:28,  8.65s/it]
 98%|█████████▊| 220530/225000 [431:19:48<11:00:28,  8.87s/it]
 98%|█████████▊| 220531/225000 [431:20:03<13:23:36, 10.79s/it]
 98%|█████████▊| 220532/225000 [431:20:11<12:12:24,  9.84s/it]
 98%|█████████▊| 220533/225000 [431:20:18<11:18:03,  9.11s/it]
 98%|█████████▊| 220534/225000 [431:20:23<9:47:53,  7.90s/it] 
 98%|█████████▊| 220535/225000 [431:20:34<10:46:53,  8.69s/it]
 98%|█████████▊| 220536/225000 [431:20:44<11:31:51,  9.30s/it]
 98%|█████████▊| 220537/225000 [431:20:55<11:53:57,  9.60s/it]
 98%|█████████▊| 220538/225000 [431:21:00<10:29:41,  8.47s/it]
 98%|█████████▊| 220539/225000 [431:21:11<11:08:37,  8.99s/it]
 98%|█████████▊| 220540/225000 [431:21:17<10:09:39,  8.20s/it]
 98%|█████████▊| 220541/225000 [431:21:23<9:20:33,  7.54s/it] 
 98%|█████████▊| 220542/225000 [431:21:33<10:17:31,  8.31s/it]
 98%|█████████▊| 220543/225000 [431:21:41<10:16:17,  8.30s/it]
 98%|█████████▊| 220544/225000 [431:21:49<10:09:37,  8.21s/it]
 98%|█████████▊| 220545/225000 [431:22:01<11:23:01,  9.20s/it]
 98%|█████████▊| 220546/225000 [431:22:12<12:16:52,  9.93s/it]
 98%|█████████▊| 220547/225000 [431:22:20<11:23:39,  9.21s/it]
 98%|█████████▊| 220548/225000 [431:22:31<12:05:34,  9.78s/it]
 98%|█████████▊| 220549/225000 [431:22:41<11:57:35,  9.67s/it]
 98%|█████████▊| 220550/225000 [431:22:48<11:05:19,  8.97s/it]
                                                              
{'loss': '1.317', 'grad_norm': '2.201', 'learning_rate': '5.533e-08', 'epoch': '0.9802', 'num_input_tokens_seen': 735020800, 'train_runtime': '1.553e+06', 'train_tokens_per_second': '473.3'}

 98%|█████████▊| 220550/225000 [431:22:48<11:05:19,  8.97s/it]
 98%|█████████▊| 220551/225000 [431:22:57<11:00:04,  8.90s/it]
 98%|█████████▊| 220552/225000 [431:23:08<11:58:50,  9.70s/it]
 98%|█████████▊| 220553/225000 [431:23:17<11:38:20,  9.42s/it]
 98%|█████████▊| 220554/225000 [431:23:33<14:00:15, 11.34s/it]
 98%|█████████▊| 220555/225000 [431:23:41<12:41:01, 10.27s/it]
 98%|█████████▊| 220556/225000 [431:23:47<11:23:58,  9.23s/it]
 98%|█████████▊| 220557/225000 [431:23:54<10:37:09,  8.60s/it]
 98%|█████████▊| 220558/225000 [431:24:02<10:03:24,  8.15s/it]
 98%|█████████▊| 220559/225000 [431:24:11<10:23:27,  8.42s/it]
 98%|█████████▊| 220560/225000 [431:24:18<9:52:01,  8.00s/it] 
 98%|█████████▊| 220561/225000 [431:24:37<14:03:59, 11.41s/it]
 98%|█████████▊| 220562/225000 [431:24:42<11:49:24,  9.59s/it]
 98%|█████████▊| 220563/225000 [431:24:47<10:05:26,  8.19s/it]
 98%|█████████▊| 220564/225000 [431:24:58<10:56:36,  8.88s/it]
 98%|█████████▊| 220565/225000 [431:25:04<10:03:40,  8.17s/it]
 98%|█████████▊| 220566/225000 [431:25:21<13:14:31, 10.75s/it]
 98%|█████████▊| 220567/225000 [431:25:27<11:34:18,  9.40s/it]
 98%|█████████▊| 220568/225000 [431:25:41<13:00:40, 10.57s/it]
 98%|█████████▊| 220569/225000 [431:25:50<12:26:29, 10.11s/it]
 98%|█████████▊| 220570/225000 [431:25:58<11:40:01,  9.48s/it]
 98%|█████████▊| 220571/225000 [431:26:11<13:03:32, 10.61s/it]
 98%|█████████▊| 220572/225000 [431:26:18<11:36:49,  9.44s/it]
 98%|█████████▊| 220573/225000 [431:26:23<10:08:08,  8.24s/it]
 98%|█████████▊| 220574/225000 [431:26:35<11:28:12,  9.33s/it]
 98%|█████████▊| 220575/225000 [431:26:45<11:42:21,  9.52s/it]
 98%|█████████▊| 220576/225000 [431:26:56<12:11:32,  9.92s/it]
 98%|█████████▊| 220577/225000 [431:27:05<11:45:51,  9.58s/it]
 98%|█████████▊| 220578/225000 [431:27:10<10:18:32,  8.39s/it]
 98%|█████████▊| 220579/225000 [431:27:16<9:23:16,  7.64s/it] 
 98%|█████████▊| 220580/225000 [431:27:25<9:50:44,  8.02s/it]
 98%|█████████▊| 220581/225000 [431:27:38<11:38:43,  9.49s/it]
 98%|█████████▊| 220582/225000 [431:27:47<11:25:53,  9.32s/it]
 98%|█████████▊| 220583/225000 [431:27:55<10:58:15,  8.94s/it]
 98%|█████████▊| 220584/225000 [431:28:03<10:43:56,  8.75s/it]
 98%|█████████▊| 220585/225000 [431:28:13<11:15:45,  9.18s/it]
 98%|█████████▊| 220586/225000 [431:28:21<10:38:08,  8.67s/it]
 98%|█████████▊| 220587/225000 [431:28:29<10:37:53,  8.67s/it]
 98%|█████████▊| 220588/225000 [431:28:36<9:47:16,  7.99s/it] 
 98%|█████████▊| 220589/225000 [431:28:42<9:01:22,  7.36s/it]
 98%|█████████▊| 220590/225000 [431:28:56<11:36:50,  9.48s/it]
 98%|█████████▊| 220591/225000 [431:29:03<10:26:47,  8.53s/it]
 98%|█████████▊| 220592/225000 [431:29:10<10:05:53,  8.25s/it]
 98%|█████████▊| 220593/225000 [431:29:17<9:46:18,  7.98s/it] 
 98%|█████████▊| 220594/225000 [431:29:32<12:21:25, 10.10s/it]
 98%|█████████▊| 220595/225000 [431:29:45<13:18:12, 10.87s/it]
 98%|█████████▊| 220596/225000 [431:29:53<12:06:17,  9.90s/it]
 98%|█████████▊| 220597/225000 [431:30:03<12:15:46, 10.03s/it]
 98%|█████████▊| 220598/225000 [431:30:10<11:13:23,  9.18s/it]
 98%|█████████▊| 220599/225000 [431:30:18<10:37:57,  8.70s/it]
 98%|█████████▊| 220600/225000 [431:30:27<10:39:12,  8.72s/it]
                                                              
{'loss': '1.322', 'grad_norm': '1.479', 'learning_rate': '5.412e-08', 'epoch': '0.9804', 'num_input_tokens_seen': 735264992, 'train_runtime': '1.553e+06', 'train_tokens_per_second': '473.3'}

 98%|█████████▊| 220600/225000 [431:30:27<10:39:12,  8.72s/it]
 98%|█████████▊| 220601/225000 [431:30:39<11:56:29,  9.77s/it]
 98%|█████████▊| 220602/225000 [431:30:48<11:38:22,  9.53s/it]
 98%|█████████▊| 220603/225000 [431:30:56<10:58:44,  8.99s/it]
 98%|█████████▊| 220604/225000 [431:31:02<10:08:19,  8.30s/it]
 98%|█████████▊| 220605/225000 [431:31:09<9:43:46,  7.97s/it] 
 98%|█████████▊| 220606/225000 [431:31:25<12:27:00, 10.20s/it]
 98%|█████████▊| 220607/225000 [431:31:35<12:14:47, 10.04s/it]
 98%|█████████▊| 220608/225000 [431:31:43<11:47:58,  9.67s/it]
 98%|█████████▊| 220609/225000 [431:31:55<12:36:52, 10.34s/it]
 98%|█████████▊| 220610/225000 [431:32:02<11:08:54,  9.14s/it]
 98%|█████████▊| 220611/225000 [431:32:11<11:15:00,  9.23s/it]
 98%|█████████▊| 220612/225000 [431:32:18<10:16:45,  8.43s/it]
 98%|█████████▊| 220613/225000 [431:32:28<10:54:35,  8.95s/it]
 98%|█████████▊| 220614/225000 [431:32:37<10:52:46,  8.93s/it]
 98%|█████████▊| 220615/225000 [431:32:48<11:39:16,  9.57s/it]
 98%|█████████▊| 220616/225000 [431:32:56<11:03:11,  9.08s/it]
 98%|█████████▊| 220617/225000 [431:33:08<12:06:26,  9.94s/it]
 98%|█████████▊| 220618/225000 [431:33:15<11:08:43,  9.16s/it]
 98%|█████████▊| 220619/225000 [431:33:24<11:04:36,  9.10s/it]
 98%|█████████▊| 220620/225000 [431:33:31<10:20:53,  8.51s/it]
 98%|█████████▊| 220621/225000 [431:33:39<10:07:21,  8.32s/it]
 98%|█████████▊| 220622/225000 [431:33:51<11:37:50,  9.56s/it]
 98%|█████████▊| 220623/225000 [431:34:02<12:03:07,  9.91s/it]
 98%|█████████▊| 220624/225000 [431:34:14<12:40:30, 10.43s/it]
 98%|█████████▊| 220625/225000 [431:34:19<10:49:18,  8.90s/it]
 98%|█████████▊| 220626/225000 [431:34:29<11:06:02,  9.14s/it]
 98%|█████████▊| 220627/225000 [431:34:37<10:40:47,  8.79s/it]
 98%|█████████▊| 220628/225000 [431:34:47<11:10:45,  9.21s/it]
 98%|█████████▊| 220629/225000 [431:34:58<11:47:06,  9.71s/it]
 98%|█████████▊| 220630/225000 [431:35:04<10:31:51,  8.68s/it]
 98%|█████████▊| 220631/225000 [431:35:14<10:57:54,  9.04s/it]
 98%|█████████▊| 220632/225000 [431:35:23<10:58:37,  9.05s/it]
 98%|█████████▊| 220633/225000 [431:35:33<11:12:30,  9.24s/it]
 98%|█████████▊| 220634/225000 [431:35:41<10:48:11,  8.91s/it]
 98%|█████████▊| 220635/225000 [431:35:51<11:11:16,  9.23s/it]
 98%|█████████▊| 220636/225000 [431:36:00<11:02:46,  9.11s/it]
 98%|█████████▊| 220637/225000 [431:36:09<11:08:13,  9.19s/it]
 98%|█████████▊| 220638/225000 [431:36:17<10:31:53,  8.69s/it]
 98%|█████████▊| 220639/225000 [431:36:28<11:41:12,  9.65s/it]
 98%|█████████▊| 220640/225000 [431:36:39<11:57:21,  9.87s/it]
 98%|█████████▊| 220641/225000 [431:36:44<10:14:20,  8.46s/it]
 98%|█████████▊| 220642/225000 [431:36:50<9:23:22,  7.76s/it] 
 98%|█████████▊| 220643/225000 [431:36:58<9:24:30,  7.77s/it]
 98%|█████████▊| 220644/225000 [431:37:08<10:20:39,  8.55s/it]
 98%|█████████▊| 220645/225000 [431:37:14<9:17:53,  7.69s/it] 
 98%|█████████▊| 220646/225000 [431:37:21<9:06:51,  7.54s/it]
 98%|█████████▊| 220647/225000 [431:37:31<10:06:38,  8.36s/it]
 98%|█████████▊| 220648/225000 [431:37:43<11:22:05,  9.40s/it]
 98%|█████████▊| 220649/225000 [431:37:49<10:06:27,  8.36s/it]
 98%|█████████▊| 220650/225000 [431:37:56<9:28:55,  7.85s/it] 
                                                             
{'loss': '1.336', 'grad_norm': '1.448', 'learning_rate': '5.293e-08', 'epoch': '0.9807', 'num_input_tokens_seen': 735507392, 'train_runtime': '1.554e+06', 'train_tokens_per_second': '473.3'}

 98%|█████████▊| 220650/225000 [431:37:56<9:28:55,  7.85s/it]
 98%|█████████▊| 220651/225000 [431:38:09<11:29:37,  9.51s/it]
 98%|█████████▊| 220652/225000 [431:38:18<11:19:46,  9.38s/it]
 98%|█████████▊| 220653/225000 [431:38:25<10:29:53,  8.69s/it]
 98%|█████████▊| 220654/225000 [431:38:35<10:39:31,  8.83s/it]
 98%|█████████▊| 220655/225000 [431:38:41<9:41:02,  8.02s/it] 
 98%|█████████▊| 220656/225000 [431:38:53<11:14:41,  9.32s/it]
 98%|█████████▊| 220657/225000 [431:38:58<9:32:41,  7.91s/it] 
 98%|█████████▊| 220658/225000 [431:39:06<9:43:33,  8.06s/it]
 98%|█████████▊| 220659/225000 [431:39:15<10:02:57,  8.33s/it]
 98%|█████████▊| 220660/225000 [431:39:22<9:41:48,  8.04s/it] 
 98%|█████████▊| 220661/225000 [431:39:35<11:22:48,  9.44s/it]
 98%|█████████▊| 220662/225000 [431:39:42<10:33:56,  8.77s/it]
 98%|█████████▊| 220663/225000 [431:39:48<9:20:36,  7.76s/it] 
 98%|█████████▊| 220664/225000 [431:40:01<11:12:07,  9.30s/it]
 98%|█████████▊| 220665/225000 [431:40:07<10:11:03,  8.46s/it]
 98%|█████████▊| 220666/225000 [431:40:14<9:44:29,  8.09s/it] 
 98%|█████████▊| 220667/225000 [431:40:27<11:33:02,  9.60s/it]
 98%|█████████▊| 220668/225000 [431:40:37<11:25:40,  9.50s/it]
 98%|█████████▊| 220669/225000 [431:40:49<12:15:18, 10.19s/it]
 98%|█████████▊| 220670/225000 [431:40:55<10:47:51,  8.98s/it]
 98%|█████████▊| 220671/225000 [431:41:05<11:22:35,  9.46s/it]
 98%|█████████▊| 220672/225000 [431:41:14<11:15:03,  9.36s/it]
 98%|█████████▊| 220673/225000 [431:41:22<10:32:43,  8.77s/it]
 98%|█████████▊| 220674/225000 [431:41:28<9:46:37,  8.14s/it] 
 98%|█████████▊| 220675/225000 [431:41:41<11:24:40,  9.50s/it]
 98%|█████████▊| 220676/225000 [431:41:47<9:56:52,  8.28s/it] 
 98%|█████████▊| 220677/225000 [431:41:57<10:40:25,  8.89s/it]
 98%|█████████▊| 220678/225000 [431:42:13<13:16:36, 11.06s/it]
 98%|█████████▊| 220679/225000 [431:42:20<11:44:44,  9.79s/it]
 98%|█████████▊| 220680/225000 [431:42:31<12:10:53, 10.15s/it]
 98%|█████████▊| 220681/225000 [431:42:38<11:17:18,  9.41s/it]
 98%|█████████▊| 220682/225000 [431:42:47<11:02:30,  9.21s/it]
 98%|█████████▊| 220683/225000 [431:42:56<10:57:41,  9.14s/it]
 98%|█████████▊| 220684/225000 [431:43:05<10:40:42,  8.91s/it]
 98%|█████████▊| 220685/225000 [431:43:13<10:21:13,  8.64s/it]
 98%|█████████▊| 220686/225000 [431:43:19<9:37:02,  8.03s/it] 
 98%|█████████▊| 220687/225000 [431:43:26<9:02:22,  7.55s/it]
 98%|█████████▊| 220688/225000 [431:43:42<12:04:52, 10.09s/it]
 98%|█████████▊| 220689/225000 [431:43:52<12:14:15, 10.22s/it]
 98%|█████████▊| 220690/225000 [431:44:01<11:37:29,  9.71s/it]
 98%|█████████▊| 220691/225000 [431:44:12<12:11:03, 10.18s/it]
 98%|█████████▊| 220692/225000 [431:44:24<13:01:27, 10.88s/it]
 98%|█████████▊| 220693/225000 [431:44:30<11:08:10,  9.31s/it]
 98%|█████████▊| 220694/225000 [431:44:37<10:16:19,  8.59s/it]
 98%|█████████▊| 220695/225000 [431:44:51<12:22:44, 10.35s/it]
 98%|█████████▊| 220696/225000 [431:45:02<12:22:49, 10.36s/it]
 98%|█████████▊| 220697/225000 [431:45:14<12:52:52, 10.78s/it]
 98%|█████████▊| 220698/225000 [431:45:22<11:57:21, 10.00s/it]
 98%|█████████▊| 220699/225000 [431:45:29<10:51:02,  9.08s/it]
 98%|█████████▊| 220700/225000 [431:45:36<10:22:25,  8.68s/it]
                                                              
{'loss': '1.298', 'grad_norm': '1.351', 'learning_rate': '5.175e-08', 'epoch': '0.9809', 'num_input_tokens_seen': 735756760, 'train_runtime': '1.554e+06', 'train_tokens_per_second': '473.4'}

 98%|█████████▊| 220700/225000 [431:45:36<10:22:25,  8.68s/it]
 98%|█████████▊| 220701/225000 [431:45:46<10:41:50,  8.96s/it]
 98%|█████████▊| 220702/225000 [431:45:59<12:15:53, 10.27s/it]
 98%|█████████▊| 220703/225000 [431:46:08<11:37:16,  9.74s/it]
 98%|█████████▊| 220704/225000 [431:46:14<10:18:09,  8.63s/it]
 98%|█████████▊| 220705/225000 [431:46:21<9:40:59,  8.12s/it] 
 98%|█████████▊| 220706/225000 [431:46:27<9:01:58,  7.57s/it]
 98%|█████████▊| 220707/225000 [431:46:39<10:33:07,  8.85s/it]
 98%|█████████▊| 220708/225000 [431:46:45<9:31:14,  7.99s/it] 
 98%|█████████▊| 220709/225000 [431:46:52<9:05:44,  7.63s/it]
 98%|█████████▊| 220710/225000 [431:47:00<9:08:09,  7.67s/it]
 98%|█████████▊| 220711/225000 [431:47:07<8:56:16,  7.50s/it]
 98%|█████████▊| 220712/225000 [431:47:15<9:20:17,  7.84s/it]
 98%|█████████▊| 220713/225000 [431:47:30<11:48:16,  9.91s/it]
 98%|█████████▊| 220714/225000 [431:47:39<11:19:54,  9.52s/it]
 98%|█████████▊| 220715/225000 [431:47:46<10:26:12,  8.77s/it]
 98%|█████████▊| 220716/225000 [431:47:52<9:28:57,  7.97s/it] 
 98%|█████████▊| 220717/225000 [431:48:04<11:07:33,  9.35s/it]
 98%|█████████▊| 220718/225000 [431:48:16<12:06:05, 10.17s/it]
 98%|█████████▊| 220719/225000 [431:48:27<12:08:10, 10.21s/it]
 98%|█████████▊| 220720/225000 [431:48:35<11:22:30,  9.57s/it]
 98%|█████████▊| 220721/225000 [431:48:44<11:10:36,  9.40s/it]
 98%|█████████▊| 220722/225000 [431:48:52<10:49:36,  9.11s/it]
 98%|█████████▊| 220723/225000 [431:48:59<9:57:26,  8.38s/it] 
 98%|█████████▊| 220724/225000 [431:49:07<9:47:07,  8.24s/it]
 98%|█████████▊| 220725/225000 [431:49:19<11:21:34,  9.57s/it]
 98%|█████████▊| 220726/225000 [431:49:30<11:39:25,  9.82s/it]
 98%|█████████▊| 220727/225000 [431:49:40<11:51:28,  9.99s/it]
 98%|█████████▊| 220728/225000 [431:49:47<10:40:19,  8.99s/it]
 98%|█████████▊| 220729/225000 [431:50:02<12:53:30, 10.87s/it]
 98%|█████████▊| 220730/225000 [431:50:11<12:08:01, 10.23s/it]
 98%|█████████▊| 220731/225000 [431:50:20<11:38:50,  9.82s/it]
 98%|█████████▊| 220732/225000 [431:50:26<10:25:51,  8.80s/it]
 98%|█████████▊| 220733/225000 [431:50:35<10:26:27,  8.81s/it]
 98%|█████████▊| 220734/225000 [431:50:49<12:18:13, 10.38s/it]
 98%|█████████▊| 220735/225000 [431:50:59<12:04:27, 10.19s/it]
 98%|█████████▊| 220736/225000 [431:51:05<10:42:23,  9.04s/it]
 98%|█████████▊| 220737/225000 [431:51:17<11:35:34,  9.79s/it]
 98%|█████████▊| 220738/225000 [431:51:25<11:05:17,  9.37s/it]
 98%|█████████▊| 220739/225000 [431:51:33<10:33:02,  8.91s/it]
 98%|█████████▊| 220740/225000 [431:51:46<11:53:21, 10.05s/it]
 98%|█████████▊| 220741/225000 [431:51:50<10:01:13,  8.47s/it]
 98%|█████████▊| 220742/225000 [431:51:57<9:15:21,  7.83s/it] 
 98%|█████████▊| 220743/225000 [431:52:04<9:08:09,  7.73s/it]
 98%|█████████▊| 220744/225000 [431:52:14<9:42:38,  8.21s/it]
 98%|█████████▊| 220745/225000 [431:52:20<9:13:07,  7.80s/it]
 98%|█████████▊| 220746/225000 [431:52:27<8:54:08,  7.53s/it]
 98%|█████████▊| 220747/225000 [431:52:35<9:00:57,  7.63s/it]
 98%|█████████▊| 220748/225000 [431:52:46<10:13:40,  8.66s/it]
 98%|█████████▊| 220749/225000 [431:52:54<9:49:29,  8.32s/it] 
 98%|█████████▊| 220750/225000 [431:53:00<8:54:13,  7.54s/it]
                                                             
{'loss': '1.286', 'grad_norm': '1.795', 'learning_rate': '5.058e-08', 'epoch': '0.9811', 'num_input_tokens_seen': 735998304, 'train_runtime': '1.555e+06', 'train_tokens_per_second': '473.4'}

 98%|█████████▊| 220750/225000 [431:53:00<8:54:13,  7.54s/it]
 98%|█████████▊| 220751/225000 [431:53:06<8:31:43,  7.23s/it]
 98%|█████████▊| 220752/225000 [431:53:20<11:02:26,  9.36s/it]
 98%|█████████▊| 220753/225000 [431:53:26<9:33:57,  8.11s/it] 
 98%|█████████▊| 220754/225000 [431:53:32<8:57:56,  7.60s/it]
 98%|█████████▊| 220755/225000 [431:53:45<10:49:26,  9.18s/it]
 98%|█████████▊| 220756/225000 [431:53:51<9:55:09,  8.41s/it] 
 98%|█████████▊| 220757/225000 [431:53:57<8:58:40,  7.62s/it]
 98%|█████████▊| 220758/225000 [431:54:12<11:20:28,  9.62s/it]
 98%|█████████▊| 220759/225000 [431:54:20<10:54:12,  9.26s/it]
 98%|█████████▊| 220760/225000 [431:54:25<9:16:43,  7.88s/it] 
 98%|█████████▊| 220761/225000 [431:54:37<10:44:57,  9.13s/it]
 98%|█████████▊| 220762/225000 [431:54:43<9:51:06,  8.37s/it] 
 98%|█████████▊| 220763/225000 [431:54:54<10:38:00,  9.03s/it]
 98%|█████████▊| 220764/225000 [431:55:06<11:42:00,  9.94s/it]
 98%|█████████▊| 220765/225000 [431:55:19<12:47:27, 10.87s/it]
 98%|█████████▊| 220766/225000 [431:55:25<11:05:14,  9.43s/it]
 98%|█████████▊| 220767/225000 [431:55:32<10:11:19,  8.67s/it]
 98%|█████████▊| 220768/225000 [431:55:39<9:31:37,  8.10s/it] 
 98%|█████████▊| 220769/225000 [431:55:48<10:04:51,  8.58s/it]
 98%|█████████▊| 220770/225000 [431:56:01<11:40:28,  9.94s/it]
 98%|█████████▊| 220771/225000 [431:56:08<10:39:26,  9.07s/it]
 98%|█████████▊| 220772/225000 [431:56:17<10:22:00,  8.83s/it]
 98%|█████████▊| 220773/225000 [431:56:24<9:43:24,  8.28s/it] 
 98%|█████████▊| 220774/225000 [431:56:36<11:14:37,  9.58s/it]
 98%|█████████▊| 220775/225000 [431:56:44<10:27:00,  8.90s/it]
 98%|█████████▊| 220776/225000 [431:56:52<10:12:03,  8.69s/it]
 98%|█████████▊| 220777/225000 [431:57:02<10:38:32,  9.07s/it]
 98%|█████████▊| 220778/225000 [431:57:09<10:00:39,  8.54s/it]
 98%|█████████▊| 220779/225000 [431:57:18<10:06:38,  8.62s/it]
 98%|█████████▊| 220780/225000 [431:57:26<9:58:38,  8.51s/it] 
 98%|█████████▊| 220781/225000 [431:57:39<11:19:08,  9.66s/it]
 98%|█████████▊| 220782/225000 [431:57:53<12:54:46, 11.02s/it]
 98%|█████████▊| 220783/225000 [431:58:04<12:51:11, 10.97s/it]
 98%|█████████▊| 220784/225000 [431:58:16<13:21:05, 11.40s/it]
 98%|█████████▊| 220785/225000 [431:58:21<11:04:09,  9.45s/it]
 98%|█████████▊| 220786/225000 [431:58:31<11:09:52,  9.54s/it]
 98%|█████████▊| 220787/225000 [431:58:38<10:31:56,  9.00s/it]
 98%|█████████▊| 220788/225000 [431:58:48<10:51:46,  9.28s/it]
 98%|█████████▊| 220789/225000 [431:58:54<9:40:24,  8.27s/it] 
 98%|█████████▊| 220790/225000 [431:59:03<10:00:01,  8.55s/it]
 98%|█████████▊| 220791/225000 [431:59:13<10:10:49,  8.71s/it]
 98%|█████████▊| 220792/225000 [431:59:20<9:46:38,  8.36s/it] 
 98%|█████████▊| 220793/225000 [431:59:29<9:48:21,  8.39s/it]
 98%|█████████▊| 220794/225000 [431:59:38<10:00:40,  8.57s/it]
 98%|█████████▊| 220795/225000 [431:59:47<10:18:32,  8.83s/it]
 98%|█████████▊| 220796/225000 [431:59:56<10:29:35,  8.99s/it]
 98%|█████████▊| 220797/225000 [432:00:02<9:23:57,  8.05s/it] 
 98%|█████████▊| 220798/225000 [432:00:09<8:57:19,  7.67s/it]
 98%|█████████▊| 220799/225000 [432:00:23<11:17:40,  9.68s/it]
 98%|█████████▊| 220800/225000 [432:00:29<9:59:59,  8.57s/it] 
                                                             
{'loss': '1.321', 'grad_norm': '1.677', 'learning_rate': '4.942e-08', 'epoch': '0.9813', 'num_input_tokens_seen': 736241056, 'train_runtime': '1.555e+06', 'train_tokens_per_second': '473.4'}

 98%|█████████▊| 220800/225000 [432:00:29<9:59:59,  8.57s/it]
 98%|█████████▊| 220801/225000 [432:00:39<10:24:24,  8.92s/it]
 98%|█████████▊| 220802/225000 [432:00:50<11:11:54,  9.60s/it]
 98%|█████████▊| 220803/225000 [432:01:03<12:11:42, 10.46s/it]
 98%|█████████▊| 220804/225000 [432:01:08<10:28:17,  8.98s/it]
 98%|█████████▊| 220805/225000 [432:01:17<10:19:19,  8.86s/it]
 98%|█████████▊| 220806/225000 [432:01:22<9:08:09,  7.84s/it] 
 98%|█████████▊| 220807/225000 [432:01:28<8:19:49,  7.15s/it]
 98%|█████████▊| 220808/225000 [432:01:41<10:25:14,  8.95s/it]
 98%|█████████▊| 220809/225000 [432:01:53<11:32:43,  9.92s/it]
 98%|█████████▊| 220810/225000 [432:02:02<11:10:10,  9.60s/it]
 98%|█████████▊| 220811/225000 [432:02:09<10:24:50,  8.95s/it]
 98%|█████████▊| 220812/225000 [432:02:16<9:33:30,  8.22s/it] 
 98%|█████████▊| 220813/225000 [432:02:23<9:09:01,  7.87s/it]
 98%|█████████▊| 220814/225000 [432:02:32<9:27:58,  8.14s/it]
 98%|█████████▊| 220815/225000 [432:02:40<9:27:05,  8.13s/it]
 98%|█████████▊| 220816/225000 [432:02:51<10:39:05,  9.16s/it]
 98%|█████████▊| 220817/225000 [432:02:58<9:50:58,  8.48s/it] 
 98%|█████████▊| 220818/225000 [432:03:10<10:59:38,  9.46s/it]
 98%|█████████▊| 220819/225000 [432:03:16<9:42:01,  8.35s/it] 
 98%|█████████▊| 220820/225000 [432:03:20<8:20:39,  7.19s/it]
 98%|█████████▊| 220821/225000 [432:03:33<10:19:30,  8.89s/it]
 98%|█████████▊| 220822/225000 [432:03:39<9:23:20,  8.09s/it] 
 98%|█████████▊| 220823/225000 [432:03:50<10:23:14,  8.95s/it]
 98%|█████████▊| 220824/225000 [432:03:58<9:53:01,  8.52s/it] 
 98%|█████████▊| 220825/225000 [432:04:07<9:57:33,  8.59s/it]
 98%|█████████▊| 220826/225000 [432:04:14<9:38:00,  8.31s/it]
 98%|█████████▊| 220827/225000 [432:04:20<8:40:05,  7.48s/it]
 98%|█████████▊| 220828/225000 [432:04:29<9:22:32,  8.09s/it]
 98%|█████████▊| 220829/225000 [432:04:34<8:20:42,  7.20s/it]
 98%|█████████▊| 220830/225000 [432:04:45<9:31:25,  8.22s/it]
 98%|█████████▊| 220831/225000 [432:04:52<8:59:03,  7.76s/it]
 98%|█████████▊| 220832/225000 [432:05:02<9:55:26,  8.57s/it]
 98%|█████████▊| 220833/225000 [432:05:14<11:09:40,  9.64s/it]
 98%|█████████▊| 220834/225000 [432:05:21<9:57:39,  8.61s/it] 
 98%|█████████▊| 220835/225000 [432:05:27<9:06:17,  7.87s/it]
 98%|█████████▊| 220836/225000 [432:05:40<11:01:21,  9.53s/it]
 98%|█████████▊| 220837/225000 [432:05:48<10:21:03,  8.95s/it]
 98%|█████████▊| 220838/225000 [432:05:54<9:21:39,  8.10s/it] 
 98%|█████████▊| 220839/225000 [432:06:01<8:54:30,  7.71s/it]
 98%|█████████▊| 220840/225000 [432:06:17<11:51:08, 10.26s/it]
 98%|█████████▊| 220841/225000 [432:06:25<11:14:04,  9.72s/it]
 98%|█████████▊| 220842/225000 [432:06:32<10:20:29,  8.95s/it]
 98%|█████████▊| 220843/225000 [432:06:46<11:46:57, 10.20s/it]
 98%|█████████▊| 220844/225000 [432:06:55<11:31:19,  9.98s/it]
 98%|█████████▊| 220845/225000 [432:07:04<11:13:26,  9.72s/it]
 98%|█████████▊| 220846/225000 [432:07:11<10:23:30,  9.01s/it]
 98%|█████████▊| 220847/225000 [432:07:23<11:18:52,  9.81s/it]
 98%|█████████▊| 220848/225000 [432:07:31<10:35:01,  9.18s/it]
 98%|█████████▊| 220849/225000 [432:07:36<9:01:42,  7.83s/it] 
 98%|█████████▊| 220850/225000 [432:07:50<11:18:10,  9.81s/it]
                                                              
{'loss': '1.311', 'grad_norm': '1.626', 'learning_rate': '4.828e-08', 'epoch': '0.9816', 'num_input_tokens_seen': 736479360, 'train_runtime': '1.556e+06', 'train_tokens_per_second': '473.4'}

 98%|█████████▊| 220850/225000 [432:07:50<11:18:10,  9.81s/it]
 98%|█████████▊| 220851/225000 [432:08:01<11:36:11, 10.07s/it]
 98%|█████████▊| 220852/225000 [432:08:11<11:47:41, 10.24s/it]
 98%|█████████▊| 220853/225000 [432:08:18<10:42:44,  9.30s/it]
 98%|█████████▊| 220854/225000 [432:08:26<10:13:58,  8.89s/it]
 98%|█████████▊| 220855/225000 [432:08:35<10:08:09,  8.80s/it]
 98%|█████████▊| 220856/225000 [432:08:45<10:37:56,  9.24s/it]
 98%|█████████▊| 220857/225000 [432:08:51<9:20:07,  8.11s/it] 
 98%|█████████▊| 220858/225000 [432:09:04<11:14:19,  9.77s/it]
 98%|█████████▊| 220859/225000 [432:09:12<10:36:02,  9.22s/it]
 98%|█████████▊| 220860/225000 [432:09:22<10:51:00,  9.43s/it]
 98%|█████████▊| 220861/225000 [432:09:28<9:29:28,  8.26s/it] 
 98%|█████████▊| 220862/225000 [432:09:40<10:55:37,  9.51s/it]
 98%|█████████▊| 220863/225000 [432:09:49<10:48:52,  9.41s/it]
 98%|█████████▊| 220864/225000 [432:09:55<9:29:13,  8.26s/it] 
 98%|█████████▊| 220865/225000 [432:10:05<10:01:55,  8.73s/it]
 98%|█████████▊| 220866/225000 [432:10:11<9:20:17,  8.13s/it] 
 98%|█████████▊| 220867/225000 [432:10:18<8:38:05,  7.52s/it]
 98%|█████████▊| 220868/225000 [432:10:26<8:47:24,  7.66s/it]
 98%|█████████▊| 220869/225000 [432:10:37<10:00:52,  8.73s/it]
 98%|█████████▊| 220870/225000 [432:10:45<9:48:42,  8.55s/it] 
 98%|█████████▊| 220871/225000 [432:10:52<9:25:14,  8.21s/it]
 98%|█████████▊| 220872/225000 [432:10:59<8:49:07,  7.69s/it]
 98%|█████████▊| 220873/225000 [432:11:13<11:03:41,  9.65s/it]
 98%|█████████▊| 220874/225000 [432:11:21<10:35:10,  9.24s/it]
 98%|█████████▊| 220875/225000 [432:11:30<10:22:30,  9.05s/it]
 98%|█████████▊| 220876/225000 [432:11:40<10:41:23,  9.33s/it]
 98%|█████████▊| 220877/225000 [432:11:48<10:08:41,  8.86s/it]
 98%|█████████▊| 220878/225000 [432:11:56<9:56:33,  8.68s/it] 
 98%|█████████▊| 220879/225000 [432:12:04<9:44:12,  8.51s/it]
 98%|█████████▊| 220880/225000 [432:12:12<9:30:33,  8.31s/it]
 98%|█████████▊| 220881/225000 [432:12:26<11:36:06, 10.14s/it]
 98%|█████████▊| 220882/225000 [432:12:35<10:58:45,  9.60s/it]
 98%|█████████▊| 220883/225000 [432:12:42<10:20:29,  9.04s/it]
 98%|█████████▊| 220884/225000 [432:12:52<10:35:06,  9.26s/it]
 98%|█████████▊| 220885/225000 [432:13:00<10:09:55,  8.89s/it]
 98%|█████████▊| 220886/225000 [432:13:10<10:39:12,  9.32s/it]
 98%|█████████▊| 220887/225000 [432:13:17<9:43:31,  8.51s/it] 
 98%|█████████▊| 220888/225000 [432:13:21<8:16:39,  7.25s/it]
 98%|█████████▊| 220889/225000 [432:13:29<8:26:24,  7.39s/it]
 98%|█████████▊| 220890/225000 [432:13:44<11:02:18,  9.67s/it]
 98%|█████████▊| 220891/225000 [432:13:54<11:03:13,  9.68s/it]
 98%|█████████▊| 220892/225000 [432:14:00<9:52:40,  8.66s/it] 
 98%|█████████▊| 220893/225000 [432:14:08<9:38:09,  8.45s/it]
 98%|█████████▊| 220894/225000 [432:14:14<8:49:17,  7.73s/it]
 98%|█████████▊| 220895/225000 [432:14:26<10:18:35,  9.04s/it]
 98%|█████████▊| 220896/225000 [432:14:41<12:19:48, 10.82s/it]
 98%|█████████▊| 220897/225000 [432:14:46<10:25:49,  9.15s/it]
 98%|█████████▊| 220898/225000 [432:14:53<9:33:51,  8.39s/it] 
 98%|█████████▊| 220899/225000 [432:15:04<10:28:57,  9.20s/it]
 98%|█████████▊| 220900/225000 [432:15:13<10:14:12,  8.99s/it]
                                                              
{'loss': '1.275', 'grad_norm': '1.54', 'learning_rate': '4.715e-08', 'epoch': '0.9818', 'num_input_tokens_seen': 736720528, 'train_runtime': '1.556e+06', 'train_tokens_per_second': '473.4'}

 98%|█████████▊| 220900/225000 [432:15:13<10:14:12,  8.99s/it]
 98%|█████████▊| 220901/225000 [432:15:19<9:16:47,  8.15s/it] 
 98%|█████████▊| 220902/225000 [432:15:28<9:37:11,  8.45s/it]
 98%|█████████▊| 220903/225000 [432:15:39<10:30:13,  9.23s/it]
 98%|█████████▊| 220904/225000 [432:15:50<11:00:02,  9.67s/it]
 98%|█████████▊| 220905/225000 [432:16:04<12:40:13, 11.14s/it]
 98%|█████████▊| 220906/225000 [432:16:11<11:15:45,  9.90s/it]
 98%|█████████▊| 220907/225000 [432:16:18<10:04:44,  8.86s/it]
 98%|█████████▊| 220908/225000 [432:16:23<8:58:52,  7.90s/it] 
 98%|█████████▊| 220909/225000 [432:16:31<8:43:23,  7.68s/it]
 98%|█████████▊| 220910/225000 [432:16:42<9:57:54,  8.77s/it]
 98%|█████████▊| 220911/225000 [432:16:48<9:07:25,  8.03s/it]
 98%|█████████▊| 220912/225000 [432:16:55<8:46:19,  7.72s/it]
 98%|█████████▊| 220913/225000 [432:17:07<10:15:52,  9.04s/it]
 98%|█████████▊| 220914/225000 [432:17:15<9:39:38,  8.51s/it] 
 98%|█████████▊| 220915/225000 [432:17:21<8:53:48,  7.84s/it]
 98%|█████████▊| 220916/225000 [432:17:36<11:17:34,  9.95s/it]
 98%|█████████▊| 220917/225000 [432:17:45<10:58:05,  9.67s/it]
 98%|█████████▊| 220918/225000 [432:17:54<10:55:49,  9.64s/it]
 98%|█████████▊| 220919/225000 [432:18:01<9:59:47,  8.82s/it] 
 98%|█████████▊| 220920/225000 [432:18:09<9:46:09,  8.62s/it]
 98%|█████████▊| 220921/225000 [432:18:13<8:11:08,  7.22s/it]
 98%|█████████▊| 220922/225000 [432:18:27<10:21:20,  9.14s/it]
 98%|█████████▊| 220923/225000 [432:18:33<9:11:36,  8.12s/it] 
 98%|█████████▊| 220924/225000 [432:18:44<10:09:13,  8.97s/it]
 98%|█████████▊| 220925/225000 [432:18:51<9:29:51,  8.39s/it] 
 98%|█████████▊| 220926/225000 [432:19:01<10:13:02,  9.03s/it]
 98%|█████████▊| 220927/225000 [432:19:13<11:06:44,  9.82s/it]
 98%|█████████▊| 220928/225000 [432:19:25<11:44:12, 10.38s/it]
 98%|█████████▊| 220929/225000 [432:19:34<11:15:46,  9.96s/it]
 98%|█████████▊| 220930/225000 [432:19:39<9:38:52,  8.53s/it] 
 98%|█████████▊| 220931/225000 [432:19:49<10:17:19,  9.10s/it]
 98%|█████████▊| 220932/225000 [432:20:00<10:49:24,  9.58s/it]
 98%|█████████▊| 220933/225000 [432:20:10<10:59:17,  9.73s/it]
 98%|█████████▊| 220934/225000 [432:20:16<9:42:02,  8.59s/it] 
 98%|█████████▊| 220935/225000 [432:20:24<9:26:24,  8.36s/it]
 98%|█████████▊| 220936/225000 [432:20:30<8:38:11,  7.65s/it]
 98%|█████████▊| 220937/225000 [432:20:36<8:17:12,  7.34s/it]
 98%|█████████▊| 220938/225000 [432:20:50<10:19:29,  9.15s/it]
 98%|█████████▊| 220939/225000 [432:21:02<11:14:30,  9.97s/it]
 98%|█████████▊| 220940/225000 [432:21:11<11:00:21,  9.76s/it]
 98%|█████████▊| 220941/225000 [432:21:15<9:04:45,  8.05s/it] 
 98%|█████████▊| 220942/225000 [432:21:23<9:03:18,  8.03s/it]
 98%|█████████▊| 220943/225000 [432:21:30<8:41:56,  7.72s/it]
 98%|█████████▊| 220944/225000 [432:21:42<10:03:35,  8.93s/it]
 98%|█████████▊| 220945/225000 [432:21:53<10:50:34,  9.63s/it]
 98%|█████████▊| 220946/225000 [432:22:04<11:13:17,  9.96s/it]
 98%|█████████▊| 220947/225000 [432:22:11<10:15:05,  9.11s/it]
 98%|█████████▊| 220948/225000 [432:22:20<10:12:29,  9.07s/it]
 98%|█████████▊| 220949/225000 [432:22:33<11:39:42, 10.36s/it]
 98%|█████████▊| 220950/225000 [432:22:40<10:25:55,  9.27s/it]
                                                              
{'loss': '1.294', 'grad_norm': '1.574', 'learning_rate': '4.604e-08', 'epoch': '0.982', 'num_input_tokens_seen': 736957224, 'train_runtime': '1.557e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 220950/225000 [432:22:40<10:25:55,  9.27s/it]
 98%|█████████▊| 220951/225000 [432:22:51<10:56:35,  9.73s/it]
 98%|█████████▊| 220952/225000 [432:22:58<10:15:23,  9.12s/it]
 98%|█████████▊| 220953/225000 [432:23:07<10:07:14,  9.00s/it]
 98%|█████████▊| 220954/225000 [432:23:17<10:21:25,  9.22s/it]
 98%|█████████▊| 220955/225000 [432:23:22<9:07:46,  8.13s/it] 
 98%|█████████▊| 220956/225000 [432:23:32<9:42:09,  8.64s/it]
 98%|█████████▊| 220957/225000 [432:23:44<10:55:50,  9.73s/it]
 98%|█████████▊| 220958/225000 [432:23:54<10:46:14,  9.59s/it]
 98%|█████████▊| 220959/225000 [432:23:58<9:05:36,  8.10s/it] 
 98%|█████████▊| 220960/225000 [432:24:10<10:08:28,  9.04s/it]
 98%|█████████▊| 220961/225000 [432:24:19<10:09:22,  9.05s/it]
 98%|█████████▊| 220962/225000 [432:24:32<11:42:33, 10.44s/it]
 98%|█████████▊| 220963/225000 [432:24:38<10:14:44,  9.14s/it]
 98%|█████████▊| 220964/225000 [432:24:48<10:14:22,  9.13s/it]
 98%|█████████▊| 220965/225000 [432:25:01<11:36:27, 10.36s/it]
 98%|█████████▊| 220966/225000 [432:25:10<11:14:36, 10.03s/it]
 98%|█████████▊| 220967/225000 [432:25:19<10:49:59,  9.67s/it]
 98%|█████████▊| 220968/225000 [432:25:26<10:01:47,  8.96s/it]
 98%|█████████▊| 220969/225000 [432:25:33<9:21:17,  8.35s/it] 
 98%|█████████▊| 220970/225000 [432:25:41<9:12:01,  8.22s/it]
 98%|█████████▊| 220971/225000 [432:25:48<8:41:35,  7.77s/it]
 98%|█████████▊| 220972/225000 [432:25:54<8:09:34,  7.29s/it]
 98%|█████████▊| 220973/225000 [432:26:07<10:07:07,  9.05s/it]
 98%|█████████▊| 220974/225000 [432:26:15<9:43:38,  8.70s/it] 
 98%|█████████▊| 220975/225000 [432:26:24<9:59:35,  8.94s/it]
 98%|█████████▊| 220976/225000 [432:26:31<9:05:25,  8.13s/it]
 98%|█████████▊| 220977/225000 [432:26:38<8:50:56,  7.92s/it]
 98%|█████████▊| 220978/225000 [432:26:45<8:38:09,  7.73s/it]
 98%|█████████▊| 220979/225000 [432:26:53<8:28:38,  7.59s/it]
 98%|█████████▊| 220980/225000 [432:27:02<8:54:42,  7.98s/it]
 98%|█████████▊| 220981/225000 [432:27:17<11:29:44, 10.30s/it]
 98%|█████████▊| 220982/225000 [432:27:25<10:36:57,  9.51s/it]
 98%|█████████▊| 220983/225000 [432:27:34<10:35:12,  9.49s/it]
 98%|█████████▊| 220984/225000 [432:27:45<10:51:01,  9.73s/it]
 98%|█████████▊| 220985/225000 [432:27:51<9:37:24,  8.63s/it] 
 98%|█████████▊| 220986/225000 [432:28:00<9:44:36,  8.74s/it]
 98%|█████████▊| 220987/225000 [432:28:08<9:32:45,  8.56s/it]
 98%|█████████▊| 220988/225000 [432:28:18<10:03:37,  9.03s/it]
 98%|█████████▊| 220989/225000 [432:28:26<9:37:35,  8.64s/it] 
 98%|█████████▊| 220990/225000 [432:28:34<9:27:53,  8.50s/it]
 98%|█████████▊| 220991/225000 [432:28:43<9:47:52,  8.80s/it]
 98%|█████████▊| 220992/225000 [432:28:50<8:58:08,  8.06s/it]
 98%|█████████▊| 220993/225000 [432:28:57<8:38:51,  7.77s/it]
 98%|█████████▊| 220994/225000 [432:29:03<8:00:49,  7.20s/it]
 98%|█████████▊| 220995/225000 [432:29:10<8:10:46,  7.35s/it]
 98%|█████████▊| 220996/225000 [432:29:22<9:28:13,  8.51s/it]
 98%|█████████▊| 220997/225000 [432:29:32<10:06:57,  9.10s/it]
 98%|█████████▊| 220998/225000 [432:29:41<10:06:00,  9.09s/it]
 98%|█████████▊| 220999/225000 [432:29:49<9:44:06,  8.76s/it] 
 98%|█████████▊| 221000/225000 [432:29:59<10:08:55,  9.13s/it]
                                                              
{'loss': '1.328', 'grad_norm': '1.692', 'learning_rate': '4.494e-08', 'epoch': '0.9822', 'num_input_tokens_seen': 737197880, 'train_runtime': '1.557e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 221000/225000 [432:29:59<10:08:55,  9.13s/it]
 98%|█████████▊| 221001/225000 [432:30:08<9:57:50,  8.97s/it] 
 98%|█████████▊| 221002/225000 [432:30:15<9:17:09,  8.36s/it]
 98%|█████████▊| 221003/225000 [432:30:28<10:48:17,  9.73s/it]
 98%|█████████▊| 221004/225000 [432:30:35<9:55:35,  8.94s/it] 
 98%|█████████▊| 221005/225000 [432:30:42<9:24:26,  8.48s/it]
 98%|█████████▊| 221006/225000 [432:30:56<11:13:37, 10.12s/it]
 98%|█████████▊| 221007/225000 [432:31:06<11:06:39, 10.02s/it]
 98%|█████████▊| 221008/225000 [432:31:14<10:31:36,  9.49s/it]
 98%|█████████▊| 221009/225000 [432:31:21<9:43:33,  8.77s/it] 
 98%|█████████▊| 221010/225000 [432:31:32<10:17:27,  9.28s/it]
 98%|█████████▊| 221011/225000 [432:31:39<9:38:42,  8.70s/it] 
 98%|█████████▊| 221012/225000 [432:31:49<10:03:54,  9.09s/it]
 98%|█████████▊| 221013/225000 [432:31:55<9:00:16,  8.13s/it] 
 98%|█████████▊| 221014/225000 [432:32:10<11:21:12, 10.25s/it]
 98%|█████████▊| 221015/225000 [432:32:20<11:06:54, 10.04s/it]
 98%|█████████▊| 221016/225000 [432:32:34<12:40:09, 11.45s/it]
 98%|█████████▊| 221017/225000 [432:32:45<12:25:30, 11.23s/it]
 98%|█████████▊| 221018/225000 [432:32:51<10:43:38,  9.70s/it]
 98%|█████████▊| 221019/225000 [432:32:58<9:53:50,  8.95s/it] 
 98%|█████████▊| 221020/225000 [432:33:05<9:06:39,  8.24s/it]
 98%|█████████▊| 221021/225000 [432:33:17<10:31:13,  9.52s/it]
 98%|█████████▊| 221022/225000 [432:33:27<10:34:41,  9.57s/it]
 98%|█████████▊| 221023/225000 [432:33:34<9:47:41,  8.87s/it] 
 98%|█████████▊| 221024/225000 [432:33:42<9:17:43,  8.42s/it]
 98%|█████████▊| 221025/225000 [432:33:53<10:09:15,  9.20s/it]
 98%|█████████▊| 221026/225000 [432:34:03<10:35:40,  9.60s/it]
 98%|█████████▊| 221027/225000 [432:34:08<9:04:34,  8.22s/it] 
 98%|█████████▊| 221028/225000 [432:34:16<9:01:11,  8.18s/it]
 98%|█████████▊| 221029/225000 [432:34:22<8:19:32,  7.55s/it]
 98%|█████████▊| 221030/225000 [432:34:34<9:29:39,  8.61s/it]
 98%|█████████▊| 221031/225000 [432:34:43<9:42:25,  8.80s/it]
 98%|█████████▊| 221032/225000 [432:34:50<9:18:46,  8.45s/it]
 98%|█████████▊| 221033/225000 [432:34:56<8:24:00,  7.62s/it]
 98%|█████████▊| 221034/225000 [432:35:09<10:00:13,  9.08s/it]
 98%|█████████▊| 221035/225000 [432:35:19<10:28:37,  9.51s/it]
 98%|█████████▊| 221036/225000 [432:35:27<9:51:58,  8.96s/it] 
 98%|█████████▊| 221037/225000 [432:35:33<9:06:29,  8.27s/it]
 98%|█████████▊| 221038/225000 [432:35:40<8:37:31,  7.84s/it]
 98%|█████████▊| 221039/225000 [432:35:48<8:40:44,  7.89s/it]
 98%|█████████▊| 221040/225000 [432:36:02<10:39:58,  9.70s/it]
 98%|█████████▊| 221041/225000 [432:36:10<10:00:09,  9.10s/it]
 98%|█████████▊| 221042/225000 [432:36:16<8:52:37,  8.07s/it] 
 98%|█████████▊| 221043/225000 [432:36:26<9:39:10,  8.78s/it]
 98%|█████████▊| 221044/225000 [432:36:35<9:37:31,  8.76s/it]
 98%|█████████▊| 221045/225000 [432:36:43<9:26:31,  8.59s/it]
 98%|█████████▊| 221046/225000 [432:36:53<9:48:45,  8.93s/it]
 98%|█████████▊| 221047/225000 [432:37:07<11:43:47, 10.68s/it]
 98%|█████████▊| 221048/225000 [432:37:18<11:38:28, 10.60s/it]
 98%|█████████▊| 221049/225000 [432:37:29<11:41:01, 10.65s/it]
 98%|█████████▊| 221050/225000 [432:37:34<9:54:34,  9.03s/it] 
                                                             
{'loss': '1.303', 'grad_norm': '2.477', 'learning_rate': '4.385e-08', 'epoch': '0.9824', 'num_input_tokens_seen': 737437336, 'train_runtime': '1.557e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 221050/225000 [432:37:34<9:54:34,  9.03s/it]
 98%|█████████▊| 221051/225000 [432:37:39<8:35:46,  7.84s/it]
 98%|█████████▊| 221052/225000 [432:37:45<8:04:45,  7.37s/it]
 98%|█████████▊| 221053/225000 [432:37:58<9:42:09,  8.85s/it]
 98%|█████████▊| 221054/225000 [432:38:07<9:57:40,  9.09s/it]
 98%|█████████▊| 221055/225000 [432:38:16<9:45:57,  8.91s/it]
 98%|█████████▊| 221056/225000 [432:38:21<8:26:23,  7.70s/it]
 98%|█████████▊| 221057/225000 [432:38:34<10:11:48,  9.31s/it]
 98%|█████████▊| 221058/225000 [432:38:39<9:01:52,  8.25s/it] 
 98%|█████████▊| 221059/225000 [432:38:46<8:35:10,  7.84s/it]
 98%|█████████▊| 221060/225000 [432:38:54<8:31:23,  7.79s/it]
 98%|█████████▊| 221061/225000 [432:39:10<11:11:15, 10.22s/it]
 98%|█████████▊| 221062/225000 [432:39:19<10:43:20,  9.80s/it]
 98%|█████████▊| 221063/225000 [432:39:28<10:32:23,  9.64s/it]
 98%|█████████▊| 221064/225000 [432:39:38<10:48:05,  9.88s/it]
 98%|█████████▊| 221065/225000 [432:39:45<9:42:41,  8.88s/it] 
 98%|█████████▊| 221066/225000 [432:39:51<8:51:48,  8.11s/it]
 98%|█████████▊| 221067/225000 [432:39:58<8:16:46,  7.58s/it]
 98%|█████████▊| 221068/225000 [432:40:06<8:27:50,  7.75s/it]
 98%|█████████▊| 221069/225000 [432:40:20<10:33:01,  9.66s/it]
 98%|█████████▊| 221070/225000 [432:40:28<9:55:59,  9.10s/it] 
 98%|█████████▊| 221071/225000 [432:40:36<9:48:10,  8.98s/it]
 98%|█████████▊| 221072/225000 [432:40:44<9:18:14,  8.53s/it]
 98%|█████████▊| 221073/225000 [432:40:48<8:02:02,  7.37s/it]
 98%|█████████▊| 221074/225000 [432:41:03<10:18:48,  9.46s/it]
 98%|█████████▊| 221075/225000 [432:41:10<9:33:37,  8.77s/it] 
 98%|█████████▊| 221076/225000 [432:41:21<10:20:42,  9.49s/it]
 98%|█████████▊| 221077/225000 [432:41:28<9:20:17,  8.57s/it] 
 98%|█████████▊| 221078/225000 [432:41:37<9:43:17,  8.92s/it]
 98%|█████████▊| 221079/225000 [432:41:47<10:01:29,  9.20s/it]
 98%|█████████▊| 221080/225000 [432:41:55<9:40:18,  8.88s/it] 
 98%|█████████▊| 221081/225000 [432:42:02<9:04:37,  8.34s/it]
 98%|█████████▊| 221082/225000 [432:42:11<9:08:18,  8.40s/it]
 98%|█████████▊| 221083/225000 [432:42:23<10:19:34,  9.49s/it]
 98%|█████████▊| 221084/225000 [432:42:35<11:13:42, 10.32s/it]
 98%|█████████▊| 221085/225000 [432:42:45<11:05:31, 10.20s/it]
 98%|█████████▊| 221086/225000 [432:42:50<9:27:49,  8.70s/it] 
 98%|█████████▊| 221087/225000 [432:42:59<9:18:27,  8.56s/it]
 98%|█████████▊| 221088/225000 [432:43:12<11:02:55, 10.17s/it]
 98%|█████████▊| 221089/225000 [432:43:17<9:09:42,  8.43s/it] 
 98%|█████████▊| 221090/225000 [432:43:32<11:20:24, 10.44s/it]
 98%|█████████▊| 221091/225000 [432:43:38<9:45:09,  8.98s/it] 
 98%|█████████▊| 221092/225000 [432:43:49<10:41:43,  9.85s/it]
 98%|█████████▊| 221093/225000 [432:44:00<10:51:42, 10.01s/it]
 98%|█████████▊| 221094/225000 [432:44:08<10:17:25,  9.48s/it]
 98%|█████████▊| 221095/225000 [432:44:16<9:46:27,  9.01s/it] 
 98%|█████████▊| 221096/225000 [432:44:21<8:37:19,  7.95s/it]
 98%|█████████▊| 221097/225000 [432:44:31<9:02:55,  8.35s/it]
 98%|█████████▊| 221098/225000 [432:44:36<7:53:46,  7.29s/it]
 98%|█████████▊| 221099/225000 [432:44:46<8:57:52,  8.27s/it]
 98%|█████████▊| 221100/225000 [432:45:00<10:52:40, 10.04s/it]
                                                              
{'loss': '1.3', 'grad_norm': '1.07', 'learning_rate': '4.278e-08', 'epoch': '0.9827', 'num_input_tokens_seen': 737678320, 'train_runtime': '1.558e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 221100/225000 [432:45:00<10:52:40, 10.04s/it]
 98%|█████████▊| 221101/225000 [432:45:08<10:11:39,  9.41s/it]
 98%|█████████▊| 221102/225000 [432:45:14<9:06:09,  8.41s/it] 
 98%|█████████▊| 221103/225000 [432:45:23<9:05:58,  8.41s/it]
 98%|█████████▊| 221104/225000 [432:45:34<9:59:09,  9.23s/it]
 98%|█████████▊| 221105/225000 [432:45:40<9:01:56,  8.35s/it]
 98%|█████████▊| 221106/225000 [432:45:51<9:43:21,  8.99s/it]
 98%|█████████▊| 221107/225000 [432:46:04<11:16:57, 10.43s/it]
 98%|█████████▊| 221108/225000 [432:46:13<10:35:14,  9.79s/it]
 98%|█████████▊| 221109/225000 [432:46:22<10:16:42,  9.51s/it]
 98%|█████████▊| 221110/225000 [432:46:28<9:24:00,  8.70s/it] 
 98%|█████████▊| 221111/225000 [432:46:34<8:28:27,  7.84s/it]
 98%|█████████▊| 221112/225000 [432:46:40<7:56:45,  7.36s/it]
 98%|█████████▊| 221113/225000 [432:46:55<10:07:59,  9.38s/it]
 98%|█████████▊| 221114/225000 [432:47:00<8:56:52,  8.29s/it] 
 98%|█████████▊| 221115/225000 [432:47:13<10:18:25,  9.55s/it]
 98%|█████████▊| 221116/225000 [432:47:24<10:43:31,  9.94s/it]
 98%|█████████▊| 221117/225000 [432:47:31<9:48:07,  9.09s/it] 
 98%|█████████▊| 221118/225000 [432:47:42<10:23:56,  9.64s/it]
 98%|█████████▊| 221119/225000 [432:47:51<10:11:31,  9.45s/it]
 98%|█████████▊| 221120/225000 [432:48:00<9:59:13,  9.27s/it] 
 98%|█████████▊| 221121/225000 [432:48:10<10:24:14,  9.66s/it]
 98%|█████████▊| 221122/225000 [432:48:17<9:39:17,  8.96s/it] 
 98%|█████████▊| 221123/225000 [432:48:26<9:22:16,  8.70s/it]
 98%|█████████▊| 221124/225000 [432:48:33<9:07:12,  8.47s/it]
 98%|█████████▊| 221125/225000 [432:48:41<8:40:16,  8.06s/it]
 98%|█████████▊| 221126/225000 [432:48:47<8:11:27,  7.61s/it]
 98%|█████████▊| 221127/225000 [432:48:58<9:15:53,  8.61s/it]
 98%|█████████▊| 221128/225000 [432:49:04<8:25:33,  7.83s/it]
 98%|█████████▊| 221129/225000 [432:49:13<8:40:11,  8.06s/it]
 98%|█████████▊| 221130/225000 [432:49:31<11:57:57, 11.13s/it]
 98%|█████████▊| 221131/225000 [432:49:40<11:23:04, 10.59s/it]
 98%|█████████▊| 221132/225000 [432:49:52<11:50:29, 11.02s/it]
 98%|█████████▊| 221133/225000 [432:50:00<10:53:16, 10.14s/it]
 98%|█████████▊| 221134/225000 [432:50:06<9:18:34,  8.67s/it] 
 98%|█████████▊| 221135/225000 [432:50:16<9:57:12,  9.27s/it]
 98%|█████████▊| 221136/225000 [432:50:24<9:32:49,  8.89s/it]
 98%|█████████▊| 221137/225000 [432:50:31<8:54:15,  8.30s/it]
 98%|█████████▊| 221138/225000 [432:50:37<8:10:43,  7.62s/it]
 98%|█████████▊| 221139/225000 [432:50:50<9:56:45,  9.27s/it]
 98%|█████████▊| 221140/225000 [432:50:58<9:33:11,  8.91s/it]
 98%|█████████▊| 221141/225000 [432:51:04<8:22:25,  7.81s/it]
 98%|█████████▊| 221142/225000 [432:51:10<7:58:33,  7.44s/it]
 98%|█████████▊| 221143/225000 [432:51:24<10:04:12,  9.40s/it]
 98%|█████████▊| 221144/225000 [432:51:29<8:37:27,  8.05s/it] 
 98%|█████████▊| 221145/225000 [432:51:40<9:38:27,  9.00s/it]
 98%|█████████▊| 221146/225000 [432:51:51<10:01:35,  9.37s/it]
 98%|█████████▊| 221147/225000 [432:51:57<9:01:10,  8.43s/it] 
 98%|█████████▊| 221148/225000 [432:52:07<9:39:32,  9.03s/it]
 98%|█████████▊| 221149/225000 [432:52:18<10:14:06,  9.57s/it]
 98%|█████████▊| 221150/225000 [432:52:28<10:26:17,  9.76s/it]
                                                              
{'loss': '1.312', 'grad_norm': '3.9', 'learning_rate': '4.171e-08', 'epoch': '0.9829', 'num_input_tokens_seen': 737915712, 'train_runtime': '1.558e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 221150/225000 [432:52:28<10:26:17,  9.76s/it]
 98%|█████████▊| 221151/225000 [432:52:36<9:45:48,  9.13s/it] 
 98%|█████████▊| 221152/225000 [432:52:44<9:22:21,  8.77s/it]
 98%|█████████▊| 221153/225000 [432:52:58<11:08:18, 10.42s/it]
 98%|█████████▊| 221154/225000 [432:53:09<11:22:43, 10.65s/it]
 98%|█████████▊| 221155/225000 [432:53:17<10:21:38,  9.70s/it]
 98%|█████████▊| 221156/225000 [432:53:28<10:41:43, 10.02s/it]
 98%|█████████▊| 221157/225000 [432:53:39<11:11:50, 10.49s/it]
 98%|█████████▊| 221158/225000 [432:53:45<9:41:30,  9.08s/it] 
 98%|█████████▊| 221159/225000 [432:53:54<9:38:02,  9.03s/it]
 98%|█████████▊| 221160/225000 [432:54:06<10:41:33, 10.02s/it]
 98%|█████████▊| 221161/225000 [432:54:12<9:10:19,  8.60s/it] 
 98%|█████████▊| 221162/225000 [432:54:22<9:53:18,  9.28s/it]
 98%|█████████▊| 221163/225000 [432:54:28<8:38:25,  8.11s/it]
 98%|█████████▊| 221164/225000 [432:54:34<8:07:17,  7.62s/it]
 98%|█████████▊| 221165/225000 [432:54:40<7:37:05,  7.15s/it]
 98%|█████████▊| 221166/225000 [432:54:48<7:50:49,  7.37s/it]
 98%|█████████▊| 221167/225000 [432:55:02<9:55:50,  9.33s/it]
 98%|█████████▊| 221168/225000 [432:55:10<9:24:15,  8.83s/it]
 98%|█████████▊| 221169/225000 [432:55:25<11:23:36, 10.71s/it]
 98%|█████████▊| 221170/225000 [432:55:33<10:36:04,  9.96s/it]
 98%|█████████▊| 221171/225000 [432:55:44<10:48:11, 10.16s/it]
 98%|█████████▊| 221172/225000 [432:55:50<9:34:38,  9.01s/it] 
 98%|█████████▊| 221173/225000 [432:55:59<9:33:13,  8.99s/it]
 98%|█████████▊| 221174/225000 [432:56:05<8:41:35,  8.18s/it]
 98%|█████████▊| 221175/225000 [432:56:11<8:04:05,  7.59s/it]
 98%|█████████▊| 221176/225000 [432:56:18<7:44:51,  7.29s/it]
 98%|█████████▊| 221177/225000 [432:56:26<8:03:21,  7.59s/it]
 98%|█████████▊| 221178/225000 [432:56:36<8:35:00,  8.08s/it]
 98%|█████████▊| 221179/225000 [432:56:50<10:26:58,  9.85s/it]
 98%|█████████▊| 221180/225000 [432:56:54<8:53:14,  8.38s/it] 
 98%|█████████▊| 221181/225000 [432:57:07<10:14:21,  9.65s/it]
 98%|█████████▊| 221182/225000 [432:57:11<8:29:46,  8.01s/it] 
 98%|█████████▊| 221183/225000 [432:57:25<10:19:58,  9.75s/it]
 98%|█████████▊| 221184/225000 [432:57:31<9:02:22,  8.53s/it] 
 98%|█████████▊| 221185/225000 [432:57:41<9:30:28,  8.97s/it]
 98%|█████████▊| 221186/225000 [432:57:50<9:33:29,  9.02s/it]
 98%|█████████▊| 221187/225000 [432:58:04<11:12:04, 10.58s/it]
 98%|█████████▊| 221188/225000 [432:58:13<10:44:44, 10.15s/it]
 98%|█████████▊| 221189/225000 [432:58:22<10:13:40,  9.66s/it]
 98%|█████████▊| 221190/225000 [432:58:30<9:53:42,  9.35s/it] 
 98%|█████████▊| 221191/225000 [432:58:43<10:54:31, 10.31s/it]
 98%|█████████▊| 221192/225000 [432:58:49<9:29:49,  8.98s/it] 
 98%|█████████▊| 221193/225000 [432:58:56<9:00:06,  8.51s/it]
 98%|█████████▊| 221194/225000 [432:59:04<8:36:46,  8.15s/it]
 98%|█████████▊| 221195/225000 [432:59:12<8:49:43,  8.35s/it]
 98%|█████████▊| 221196/225000 [432:59:24<9:58:56,  9.45s/it]
 98%|█████████▊| 221197/225000 [432:59:30<8:52:36,  8.40s/it]
 98%|█████████▊| 221198/225000 [432:59:43<10:11:02,  9.64s/it]
 98%|█████████▊| 221199/225000 [432:59:47<8:29:52,  8.05s/it] 
 98%|█████████▊| 221200/225000 [432:59:58<9:14:56,  8.76s/it]
                                                             
{'loss': '1.342', 'grad_norm': '1.568', 'learning_rate': '4.067e-08', 'epoch': '0.9831', 'num_input_tokens_seen': 738158968, 'train_runtime': '1.559e+06', 'train_tokens_per_second': '473.5'}

 98%|█████████▊| 221200/225000 [432:59:58<9:14:56,  8.76s/it]
 98%|█████████▊| 221201/225000 [433:00:07<9:29:03,  8.99s/it]
 98%|█████████▊| 221202/225000 [433:00:18<10:02:14,  9.51s/it]
 98%|█████████▊| 221203/225000 [433:00:25<9:07:41,  8.65s/it] 
 98%|█████████▊| 221204/225000 [433:00:31<8:29:57,  8.06s/it]
 98%|█████████▊| 221205/225000 [433:00:40<8:52:55,  8.43s/it]
 98%|█████████▊| 221206/225000 [433:00:52<9:59:33,  9.48s/it]
 98%|█████████▊| 221207/225000 [433:01:01<9:41:12,  9.19s/it]
 98%|█████████▊| 221208/225000 [433:01:16<11:30:51, 10.93s/it]
 98%|█████████▊| 221209/225000 [433:01:24<10:26:53,  9.92s/it]
 98%|█████████▊| 221210/225000 [433:01:32<10:00:45,  9.51s/it]
 98%|█████████▊| 221211/225000 [433:01:39<9:21:12,  8.89s/it] 
 98%|█████████▊| 221212/225000 [433:01:51<10:14:46,  9.74s/it]
 98%|█████████▊| 221213/225000 [433:02:00<9:48:47,  9.33s/it] 
 98%|█████████▊| 221214/225000 [433:02:05<8:41:33,  8.27s/it]
 98%|█████████▊| 221215/225000 [433:02:16<9:34:26,  9.11s/it]
 98%|█████████▊| 221216/225000 [433:02:28<10:26:18,  9.93s/it]
 98%|█████████▊| 221217/225000 [433:02:39<10:35:23, 10.08s/it]
 98%|█████████▊| 221218/225000 [433:02:45<9:24:33,  8.96s/it] 
 98%|█████████▊| 221219/225000 [433:02:57<10:11:44,  9.71s/it]
 98%|█████████▊| 221220/225000 [433:03:07<10:33:44, 10.06s/it]
 98%|█████████▊| 221221/225000 [433:03:14<9:30:52,  9.06s/it] 
 98%|█████████▊| 221222/225000 [433:03:22<9:03:40,  8.63s/it]
 98%|█████████▊| 221223/225000 [433:03:33<9:47:06,  9.33s/it]
 98%|█████████▊| 221224/225000 [433:03:40<9:11:16,  8.76s/it]
 98%|█████████▊| 221225/225000 [433:03:49<9:09:46,  8.74s/it]
 98%|█████████▊| 221226/225000 [433:03:58<9:23:58,  8.97s/it]
 98%|█████████▊| 221227/225000 [433:04:12<10:53:47, 10.40s/it]
 98%|█████████▊| 221228/225000 [433:04:16<8:53:37,  8.49s/it] 
 98%|█████████▊| 221229/225000 [433:04:28<9:48:38,  9.37s/it]
 98%|█████████▊| 221230/225000 [433:04:39<10:37:32, 10.15s/it]
 98%|█████████▊| 221231/225000 [433:04:44<8:46:21,  8.38s/it] 
 98%|█████████▊| 221232/225000 [433:04:51<8:26:19,  8.06s/it]
 98%|█████████▊| 221233/225000 [433:04:56<7:29:40,  7.16s/it]
 98%|█████████▊| 221234/225000 [433:05:08<8:57:16,  8.56s/it]
 98%|█████████▊| 221235/225000 [433:05:14<8:06:29,  7.75s/it]
 98%|█████████▊| 221236/225000 [433:05:28<10:06:35,  9.67s/it]
 98%|█████████▊| 221237/225000 [433:05:36<9:27:24,  9.05s/it] 
 98%|█████████▊| 221238/225000 [433:05:46<9:54:30,  9.48s/it]
 98%|█████████▊| 221239/225000 [433:05:52<8:54:34,  8.53s/it]
 98%|█████████▊| 221240/225000 [433:06:04<9:53:06,  9.46s/it]
 98%|█████████▊| 221241/225000 [433:06:13<9:46:11,  9.36s/it]
 98%|█████████▊| 221242/225000 [433:06:20<8:50:55,  8.48s/it]
 98%|█████████▊| 221243/225000 [433:06:28<8:53:33,  8.52s/it]
 98%|█████████▊| 221244/225000 [433:06:36<8:41:30,  8.33s/it]
 98%|█████████▊| 221245/225000 [433:06:45<9:02:16,  8.66s/it]
 98%|█████████▊| 221246/225000 [433:06:56<9:44:58,  9.35s/it]
 98%|█████████▊| 221247/225000 [433:07:08<10:18:54,  9.89s/it]
 98%|█████████▊| 221248/225000 [433:07:14<9:09:15,  8.78s/it] 
 98%|█████████▊| 221249/225000 [433:07:21<8:36:06,  8.26s/it]
 98%|█████████▊| 221250/225000 [433:07:28<8:24:54,  8.08s/it]
                                                             
{'loss': '1.297', 'grad_norm': '2.592', 'learning_rate': '3.963e-08', 'epoch': '0.9833', 'num_input_tokens_seen': 738401544, 'train_runtime': '1.559e+06', 'train_tokens_per_second': '473.6'}

 98%|█████████▊| 221250/225000 [433:07:28<8:24:54,  8.08s/it]
 98%|█████████▊| 221251/225000 [433:07:43<10:16:12,  9.86s/it]
 98%|█████████▊| 221252/225000 [433:07:49<9:12:51,  8.85s/it] 
 98%|█████████▊| 221253/225000 [433:07:59<9:34:12,  9.19s/it]
 98%|█████████▊| 221254/225000 [433:08:09<9:55:28,  9.54s/it]
 98%|█████████▊| 221255/225000 [433:08:21<10:27:30, 10.05s/it]
 98%|█████████▊| 221256/225000 [433:08:30<10:12:47,  9.82s/it]
 98%|█████████▊| 221257/225000 [433:08:39<10:06:58,  9.73s/it]
 98%|█████████▊| 221258/225000 [433:08:51<10:48:47, 10.40s/it]
 98%|█████████▊| 221259/225000 [433:08:59<9:51:56,  9.49s/it] 
 98%|█████████▊| 221260/225000 [433:09:05<8:48:33,  8.48s/it]
 98%|█████████▊| 221261/225000 [433:09:14<9:08:18,  8.80s/it]
 98%|█████████▊| 221262/225000 [433:09:21<8:23:36,  8.08s/it]
 98%|█████████▊| 221263/225000 [433:09:27<7:56:56,  7.66s/it]
 98%|█████████▊| 221264/225000 [433:09:41<9:43:29,  9.37s/it]
 98%|█████████▊| 221265/225000 [433:09:49<9:24:44,  9.07s/it]
 98%|█████████▊| 221266/225000 [433:10:01<10:08:42,  9.78s/it]
 98%|█████████▊| 221267/225000 [433:10:10<9:58:38,  9.62s/it] 
 98%|█████████▊| 221268/225000 [433:10:17<9:13:11,  8.89s/it]
 98%|█████████▊| 221269/225000 [433:10:26<9:11:44,  8.87s/it]
 98%|█████████▊| 221270/225000 [433:10:32<8:10:56,  7.90s/it]
 98%|█████████▊| 221271/225000 [433:10:38<7:44:03,  7.47s/it]
 98%|█████████▊| 221272/225000 [433:10:52<9:50:06,  9.50s/it]
 98%|█████████▊| 221273/225000 [433:11:03<10:10:46,  9.83s/it]
 98%|█████████▊| 221274/225000 [433:11:12<10:00:21,  9.67s/it]
 98%|█████████▊| 221275/225000 [433:11:22<10:01:17,  9.69s/it]
 98%|█████████▊| 221276/225000 [433:11:32<10:06:13,  9.77s/it]
 98%|█████████▊| 221277/225000 [433:11:41<9:47:05,  9.46s/it] 
 98%|█████████▊| 221278/225000 [433:11:47<8:54:28,  8.62s/it]
 98%|█████████▊| 221279/225000 [433:11:58<9:26:02,  9.13s/it]
 98%|█████████▊| 221280/225000 [433:12:03<8:08:58,  7.89s/it]
 98%|█████████▊| 221281/225000 [433:12:14<9:15:26,  8.96s/it]
 98%|█████████▊| 221282/225000 [433:12:20<8:29:35,  8.22s/it]
 98%|█████████▊| 221283/225000 [433:12:27<7:58:39,  7.73s/it]
 98%|█████████▊| 221284/225000 [433:12:40<9:42:42,  9.41s/it]
 98%|█████████▊| 221285/225000 [433:12:49<9:36:34,  9.31s/it]
 98%|█████████▊| 221286/225000 [433:12:55<8:32:23,  8.28s/it]
 98%|█████████▊| 221287/225000 [433:13:07<9:41:46,  9.40s/it]
 98%|█████████▊| 221288/225000 [433:13:16<9:29:36,  9.21s/it]
 98%|█████████▊| 221289/225000 [433:13:27<10:06:17,  9.80s/it]
 98%|█████████▊| 221290/225000 [433:13:38<10:31:31, 10.21s/it]
 98%|█████████▊| 221291/225000 [433:13:51<11:07:12, 10.79s/it]
 98%|█████████▊| 221292/225000 [433:13:59<10:29:26, 10.19s/it]
 98%|█████████▊| 221293/225000 [433:14:08<9:51:02,  9.57s/it] 
 98%|█████████▊| 221294/225000 [433:14:17<9:45:34,  9.48s/it]
 98%|█████████▊| 221295/225000 [433:14:23<8:53:43,  8.64s/it]
 98%|█████████▊| 221296/225000 [433:14:30<8:22:50,  8.15s/it]
 98%|█████████▊| 221297/225000 [433:14:38<8:03:06,  7.83s/it]
 98%|█████████▊| 221298/225000 [433:14:53<10:29:44, 10.21s/it]
 98%|█████████▊| 221299/225000 [433:14:59<9:03:02,  8.80s/it] 
 98%|█████████▊| 221300/225000 [433:15:06<8:34:47,  8.35s/it]
                                                             
{'loss': '1.319', 'grad_norm': '1.533', 'learning_rate': '3.861e-08', 'epoch': '0.9836', 'num_input_tokens_seen': 738651072, 'train_runtime': '1.56e+06', 'train_tokens_per_second': '473.6'}

 98%|█████████▊| 221300/225000 [433:15:06<8:34:47,  8.35s/it]
 98%|█████████▊| 221301/225000 [433:15:16<8:57:36,  8.72s/it]
 98%|█████████▊| 221302/225000 [433:15:26<9:28:38,  9.23s/it]
 98%|█████████▊| 221303/225000 [433:15:36<9:45:57,  9.51s/it]
 98%|█████████▊| 221304/225000 [433:15:42<8:34:14,  8.35s/it]
 98%|█████████▊| 221305/225000 [433:15:54<9:45:27,  9.51s/it]
 98%|█████████▊| 221306/225000 [433:16:01<8:55:52,  8.70s/it]
 98%|█████████▊| 221307/225000 [433:16:09<8:51:13,  8.63s/it]
 98%|█████████▊| 221308/225000 [433:16:18<8:58:58,  8.76s/it]
 98%|█████████▊| 221309/225000 [433:16:28<9:05:45,  8.87s/it]
 98%|█████████▊| 221310/225000 [433:16:35<8:42:42,  8.50s/it]
 98%|█████████▊| 221311/225000 [433:16:41<7:55:18,  7.73s/it]
 98%|█████████▊| 221312/225000 [433:16:57<10:17:00, 10.04s/it]
 98%|█████████▊| 221313/225000 [433:17:09<10:51:26, 10.60s/it]
 98%|█████████▊| 221314/225000 [433:17:14<9:20:21,  9.12s/it] 
 98%|█████████▊| 221315/225000 [433:17:23<9:09:59,  8.96s/it]
 98%|█████████▊| 221316/225000 [433:17:29<8:24:05,  8.21s/it]
 98%|█████████▊| 221317/225000 [433:17:42<9:55:32,  9.70s/it]
 98%|█████████▊| 221318/225000 [433:17:52<9:50:08,  9.62s/it]
 98%|█████████▊| 221319/225000 [433:18:01<9:41:24,  9.48s/it]
 98%|█████████▊| 221320/225000 [433:18:07<8:39:39,  8.47s/it]
 98%|█████████▊| 221321/225000 [433:18:15<8:36:38,  8.43s/it]
 98%|█████████▊| 221322/225000 [433:18:26<9:23:53,  9.20s/it]
 98%|█████████▊| 221323/225000 [433:18:31<8:06:10,  7.93s/it]
 98%|█████████▊| 221324/225000 [433:18:41<8:40:46,  8.50s/it]
 98%|█████████▊| 221325/225000 [433:18:51<8:56:00,  8.75s/it]
 98%|█████████▊| 221326/225000 [433:19:00<9:06:15,  8.92s/it]
 98%|█████████▊| 221327/225000 [433:19:08<8:51:53,  8.69s/it]
 98%|█████████▊| 221328/225000 [433:19:16<8:30:59,  8.35s/it]
 98%|█████████▊| 221329/225000 [433:19:30<10:26:59, 10.25s/it]
 98%|█████████▊| 221330/225000 [433:19:46<11:59:43, 11.77s/it]
 98%|█████████▊| 221331/225000 [433:19:53<10:41:47, 10.50s/it]
 98%|█████████▊| 221332/225000 [433:20:05<10:59:44, 10.79s/it]
 98%|█████████▊| 221333/225000 [433:20:16<11:14:55, 11.04s/it]
 98%|█████████▊| 221334/225000 [433:20:25<10:29:38, 10.31s/it]
 98%|█████████▊| 221335/225000 [433:20:33<9:59:16,  9.81s/it] 
 98%|█████████▊| 221336/225000 [433:20:40<9:07:29,  8.97s/it]
 98%|█████████▊| 221337/225000 [433:20:46<8:01:52,  7.89s/it]
 98%|█████████▊| 221338/225000 [433:20:55<8:15:54,  8.13s/it]
 98%|█████████▊| 221339/225000 [433:21:09<10:15:40, 10.09s/it]
 98%|█████████▊| 221340/225000 [433:21:17<9:24:36,  9.26s/it] 
 98%|█████████▊| 221341/225000 [433:21:24<8:56:27,  8.80s/it]
 98%|█████████▊| 221342/225000 [433:21:36<9:53:55,  9.74s/it]
 98%|█████████▊| 221343/225000 [433:21:43<9:02:06,  8.89s/it]
 98%|█████████▊| 221344/225000 [433:21:53<9:19:35,  9.18s/it]
 98%|█████████▊| 221345/225000 [433:22:00<8:45:51,  8.63s/it]
 98%|█████████▊| 221346/225000 [433:22:14<10:12:49, 10.06s/it]
 98%|█████████▊| 221347/225000 [433:22:21<9:17:54,  9.16s/it] 
 98%|█████████▊| 221348/225000 [433:22:27<8:27:47,  8.34s/it]
 98%|█████████▊| 221349/225000 [433:22:37<9:03:03,  8.92s/it]
 98%|█████████▊| 221350/225000 [433:22:44<8:18:14,  8.19s/it]
                                                             
{'loss': '1.303', 'grad_norm': '1.287', 'learning_rate': '3.76e-08', 'epoch': '0.9838', 'num_input_tokens_seen': 738894744, 'train_runtime': '1.56e+06', 'train_tokens_per_second': '473.6'}

 98%|█████████▊| 221350/225000 [433:22:44<8:18:14,  8.19s/it]
 98%|█████████▊| 221351/225000 [433:22:55<9:15:44,  9.14s/it]
 98%|█████████▊| 221352/225000 [433:23:04<9:00:33,  8.89s/it]
 98%|█████████▊| 221353/225000 [433:23:08<7:46:43,  7.68s/it]
 98%|█████████▊| 221354/225000 [433:23:20<8:51:21,  8.74s/it]
 98%|█████████▊| 221355/225000 [433:23:28<8:38:02,  8.53s/it]
 98%|█████████▊| 221356/225000 [433:23:35<8:10:03,  8.07s/it]
 98%|█████████▊| 221357/225000 [433:23:52<10:52:10, 10.74s/it]
 98%|█████████▊| 221358/225000 [433:23:58<9:29:55,  9.39s/it] 
 98%|█████████▊| 221359/225000 [433:24:09<9:54:17,  9.79s/it]
 98%|█████████▊| 221360/225000 [433:24:16<9:10:31,  9.07s/it]
 98%|█████████▊| 221361/225000 [433:24:21<8:00:18,  7.92s/it]
 98%|█████████▊| 221362/225000 [433:24:28<7:29:58,  7.42s/it]
 98%|█████████▊| 221363/225000 [433:24:33<7:02:08,  6.96s/it]
 98%|█████████▊| 221364/225000 [433:24:46<8:48:53,  8.73s/it]
 98%|█████████▊| 221365/225000 [433:24:55<8:51:19,  8.77s/it]
 98%|█████████▊| 221366/225000 [433:25:05<9:19:18,  9.23s/it]
 98%|█████████▊| 221367/225000 [433:25:19<10:43:13, 10.62s/it]
 98%|█████████▊| 221368/225000 [433:25:26<9:32:07,  9.45s/it] 
 98%|█████████▊| 221369/225000 [433:25:33<8:40:18,  8.60s/it]
 98%|█████████▊| 221370/225000 [433:25:43<9:17:37,  9.22s/it]
 98%|█████████▊| 221371/225000 [433:25:51<8:46:17,  8.70s/it]
 98%|█████████▊| 221372/225000 [433:25:58<8:20:55,  8.28s/it]
 98%|█████████▊| 221373/225000 [433:26:07<8:35:57,  8.54s/it]
 98%|█████████▊| 221374/225000 [433:26:15<8:16:07,  8.21s/it]
 98%|█████████▊| 221375/225000 [433:26:29<10:14:03, 10.16s/it]
 98%|█████████▊| 221376/225000 [433:26:36<9:08:39,  9.08s/it] 
 98%|█████████▊| 221377/225000 [433:26:43<8:32:11,  8.48s/it]
 98%|█████████▊| 221378/225000 [433:26:50<8:03:32,  8.01s/it]
 98%|█████████▊| 221379/225000 [433:27:00<8:45:39,  8.71s/it]
 98%|█████████▊| 221380/225000 [433:27:14<10:06:34, 10.05s/it]
 98%|█████████▊| 221381/225000 [433:27:21<9:19:21,  9.27s/it] 
 98%|█████████▊| 221382/225000 [433:27:30<9:13:18,  9.18s/it]
 98%|█████████▊| 221383/225000 [433:27:37<8:36:10,  8.56s/it]
 98%|█████████▊| 221384/225000 [433:27:44<8:07:39,  8.09s/it]
 98%|█████████▊| 221385/225000 [433:27:56<9:19:51,  9.29s/it]
 98%|█████████▊| 221386/225000 [433:28:07<9:43:52,  9.69s/it]
 98%|█████████▊| 221387/225000 [433:28:13<8:45:23,  8.73s/it]
 98%|█████████▊| 221388/225000 [433:28:19<7:54:28,  7.88s/it]
 98%|█████████▊| 221389/225000 [433:28:27<8:01:09,  7.99s/it]
 98%|█████████▊| 221390/225000 [433:28:36<8:08:03,  8.11s/it]
 98%|█████████▊| 221391/225000 [433:28:45<8:34:42,  8.56s/it]
 98%|█████████▊| 221392/225000 [433:28:52<8:04:19,  8.05s/it]
 98%|█████████▊| 221393/225000 [433:28:59<7:43:59,  7.72s/it]
 98%|█████████▊| 221394/225000 [433:29:08<8:06:25,  8.09s/it]
 98%|█████████▊| 221395/225000 [433:29:21<9:39:16,  9.64s/it]
 98%|█████████▊| 221396/225000 [433:29:34<10:26:47, 10.43s/it]
 98%|█████████▊| 221397/225000 [433:29:50<12:03:49, 12.05s/it]
 98%|█████████▊| 221398/225000 [433:29:56<10:25:53, 10.43s/it]
 98%|█████████▊| 221399/225000 [433:30:08<10:45:04, 10.75s/it]
 98%|█████████▊| 221400/225000 [433:30:17<10:12:38, 10.21s/it]
                                                              
{'loss': '1.334', 'grad_norm': '1.718', 'learning_rate': '3.661e-08', 'epoch': '0.984', 'num_input_tokens_seen': 739134456, 'train_runtime': '1.561e+06', 'train_tokens_per_second': '473.6'}

 98%|█████████▊| 221400/225000 [433:30:17<10:12:38, 10.21s/it]
 98%|█████████▊| 221401/225000 [433:30:24<9:28:34,  9.48s/it] 
 98%|█████████▊| 221402/225000 [433:30:32<9:03:28,  9.06s/it]
 98%|█████████▊| 221403/225000 [433:30:41<8:47:20,  8.80s/it]
 98%|█████████▊| 221404/225000 [433:30:47<8:08:22,  8.15s/it]
 98%|█████████▊| 221405/225000 [433:31:03<10:27:40, 10.48s/it]
 98%|█████████▊| 221406/225000 [433:31:13<10:16:22, 10.29s/it]
 98%|█████████▊| 221407/225000 [433:31:20<9:21:50,  9.38s/it] 
 98%|█████████▊| 221408/225000 [433:31:26<8:17:26,  8.31s/it]
 98%|█████████▊| 221409/225000 [433:31:39<9:44:41,  9.77s/it]
 98%|█████████▊| 221410/225000 [433:31:45<8:28:48,  8.50s/it]
 98%|█████████▊| 221411/225000 [433:31:54<8:41:02,  8.71s/it]
 98%|█████████▊| 221412/225000 [433:32:01<8:03:16,  8.08s/it]
 98%|█████████▊| 221413/225000 [433:32:10<8:17:16,  8.32s/it]
 98%|█████████▊| 221414/225000 [433:32:15<7:25:50,  7.46s/it]
 98%|█████████▊| 221415/225000 [433:32:24<7:54:27,  7.94s/it]
 98%|█████████▊| 221416/225000 [433:32:33<8:05:09,  8.12s/it]
 98%|█████████▊| 221417/225000 [433:32:42<8:30:33,  8.55s/it]
 98%|█████████▊| 221418/225000 [433:32:49<7:54:33,  7.95s/it]
 98%|█████████▊| 221419/225000 [433:32:57<7:52:13,  7.91s/it]
 98%|█████████▊| 221420/225000 [433:33:05<7:55:26,  7.97s/it]
 98%|█████████▊| 221421/225000 [433:33:14<8:28:01,  8.52s/it]
 98%|█████████▊| 221422/225000 [433:33:25<9:10:01,  9.22s/it]
 98%|█████████▊| 221423/225000 [433:33:32<8:17:11,  8.34s/it]
 98%|█████████▊| 221424/225000 [433:33:42<8:46:06,  8.83s/it]
 98%|█████████▊| 221425/225000 [433:33:51<9:04:49,  9.14s/it]
 98%|█████████▊| 221426/225000 [433:34:05<10:16:59, 10.36s/it]
 98%|█████████▊| 221427/225000 [433:34:13<9:45:44,  9.84s/it] 
 98%|█████████▊| 221428/225000 [433:34:19<8:36:40,  8.68s/it]
 98%|█████████▊| 221429/225000 [433:34:31<9:30:26,  9.58s/it]
 98%|█████████▊| 221430/225000 [433:34:39<9:10:56,  9.26s/it]
 98%|█████████▊| 221431/225000 [433:34:48<8:58:37,  9.06s/it]
 98%|█████████▊| 221432/225000 [433:34:55<8:20:42,  8.42s/it]
 98%|█████████▊| 221433/225000 [433:35:08<9:49:19,  9.91s/it]
 98%|█████████▊| 221434/225000 [433:35:15<8:59:04,  9.07s/it]
 98%|█████████▊| 221435/225000 [433:35:26<9:27:25,  9.55s/it]
 98%|█████████▊| 221436/225000 [433:35:40<10:38:00, 10.74s/it]
 98%|█████████▊| 221437/225000 [433:35:51<10:40:45, 10.79s/it]
 98%|█████████▊| 221438/225000 [433:36:00<10:23:02, 10.49s/it]
 98%|█████████▊| 221439/225000 [433:36:11<10:25:05, 10.53s/it]
 98%|█████████▊| 221440/225000 [433:36:23<10:44:18, 10.86s/it]
 98%|█████████▊| 221441/225000 [433:36:31<9:54:49, 10.03s/it] 
 98%|█████████▊| 221442/225000 [433:36:38<9:06:22,  9.21s/it]
 98%|█████████▊| 221443/225000 [433:36:47<9:03:46,  9.17s/it]
 98%|█████████▊| 221444/225000 [433:36:56<8:57:41,  9.07s/it]
 98%|█████████▊| 221445/225000 [433:37:04<8:36:11,  8.71s/it]
 98%|█████████▊| 221446/225000 [433:37:19<10:28:20, 10.61s/it]
 98%|█████████▊| 221447/225000 [433:37:28<10:07:21, 10.26s/it]
 98%|█████████▊| 221448/225000 [433:37:34<8:43:15,  8.84s/it] 
 98%|█████████▊| 221449/225000 [433:37:46<9:37:39,  9.76s/it]
 98%|█████████▊| 221450/225000 [433:37:52<8:31:10,  8.64s/it]
                                                             
{'loss': '1.324', 'grad_norm': '2.035', 'learning_rate': '3.563e-08', 'epoch': '0.9842', 'num_input_tokens_seen': 739376616, 'train_runtime': '1.561e+06', 'train_tokens_per_second': '473.6'}

 98%|█████████▊| 221450/225000 [433:37:52<8:31:10,  8.64s/it]
 98%|█████████▊| 221451/225000 [433:37:58<7:51:27,  7.97s/it]
 98%|█████████▊| 221452/225000 [433:38:09<8:49:23,  8.95s/it]
 98%|█████████▊| 221453/225000 [433:38:23<10:19:33, 10.48s/it]
 98%|█████████▊| 221454/225000 [433:38:30<9:12:26,  9.35s/it] 
 98%|█████████▊| 221455/225000 [433:38:37<8:34:19,  8.71s/it]
 98%|█████████▊| 221456/225000 [433:38:46<8:30:04,  8.64s/it]
 98%|█████████▊| 221457/225000 [433:38:54<8:27:32,  8.60s/it]
 98%|█████████▊| 221458/225000 [433:39:04<8:50:57,  8.99s/it]
 98%|█████████▊| 221459/225000 [433:39:21<11:06:52, 11.30s/it]
 98%|█████████▊| 221460/225000 [433:39:25<9:02:39,  9.20s/it] 
 98%|█████████▊| 221461/225000 [433:39:36<9:37:18,  9.79s/it]
 98%|█████████▊| 221462/225000 [433:39:41<8:05:07,  8.23s/it]
 98%|█████████▊| 221463/225000 [433:39:52<8:50:20,  9.00s/it]
 98%|█████████▊| 221464/225000 [433:39:59<8:17:00,  8.43s/it]
 98%|█████████▊| 221465/225000 [433:40:09<8:54:27,  9.07s/it]
 98%|█████████▊| 221466/225000 [433:40:16<8:16:47,  8.43s/it]
 98%|█████████▊| 221467/225000 [433:40:23<7:41:38,  7.84s/it]
 98%|█████████▊| 221468/225000 [433:40:29<7:21:00,  7.49s/it]
 98%|█████████▊| 221469/225000 [433:40:41<8:31:59,  8.70s/it]
 98%|█████████▊| 221470/225000 [433:40:51<8:48:31,  8.98s/it]
 98%|█████████▊| 221471/225000 [433:41:02<9:34:25,  9.77s/it]
 98%|█████████▊| 221472/225000 [433:41:11<9:21:25,  9.55s/it]
 98%|█████████▊| 221473/225000 [433:41:17<8:23:00,  8.56s/it]
 98%|█████████▊| 221474/225000 [433:41:24<7:53:16,  8.05s/it]
 98%|█████████▊| 221475/225000 [433:41:38<9:34:50,  9.78s/it]
 98%|█████████▊| 221476/225000 [433:41:44<8:21:39,  8.54s/it]
 98%|█████████▊| 221477/225000 [433:41:52<8:08:54,  8.33s/it]
 98%|█████████▊| 221478/225000 [433:42:03<9:09:47,  9.37s/it]
 98%|█████████▊| 221479/225000 [433:42:11<8:37:29,  8.82s/it]
 98%|█████████▊| 221480/225000 [433:42:22<9:10:26,  9.38s/it]
 98%|█████████▊| 221481/225000 [433:42:30<8:44:51,  8.95s/it]
 98%|█████████▊| 221482/225000 [433:42:38<8:26:37,  8.64s/it]
 98%|█████████▊| 221483/225000 [433:42:43<7:23:22,  7.56s/it]
 98%|█████████▊| 221484/225000 [433:42:55<8:46:12,  8.98s/it]
 98%|█████████▊| 221485/225000 [433:43:01<7:51:16,  8.04s/it]
 98%|█████████▊| 221486/225000 [433:43:08<7:43:18,  7.91s/it]
 98%|█████████▊| 221487/225000 [433:43:21<9:13:21,  9.45s/it]
 98%|█████████▊| 221488/225000 [433:43:27<7:57:42,  8.16s/it]
 98%|█████████▊| 221489/225000 [433:43:34<7:45:20,  7.95s/it]
 98%|█████████▊| 221490/225000 [433:43:43<8:05:28,  8.30s/it]
 98%|█████████▊| 221491/225000 [433:43:58<9:56:10, 10.19s/it]
 98%|█████████▊| 221492/225000 [433:44:03<8:27:54,  8.69s/it]
 98%|█████████▊| 221493/225000 [433:44:10<7:53:58,  8.11s/it]
 98%|█████████▊| 221494/225000 [433:44:21<8:55:13,  9.16s/it]
 98%|█████████▊| 221495/225000 [433:44:31<8:57:51,  9.21s/it]
 98%|█████████▊| 221496/225000 [433:44:42<9:38:20,  9.90s/it]
 98%|█████████▊| 221497/225000 [433:44:50<9:11:17,  9.44s/it]
 98%|█████████▊| 221498/225000 [433:45:01<9:34:23,  9.84s/it]
 98%|█████████▊| 221499/225000 [433:45:08<8:48:27,  9.06s/it]
 98%|█████████▊| 221500/225000 [433:45:16<8:28:01,  8.71s/it]
                                                             
{'loss': '1.294', 'grad_norm': '1.236', 'learning_rate': '3.466e-08', 'epoch': '0.9844', 'num_input_tokens_seen': 739613448, 'train_runtime': '1.562e+06', 'train_tokens_per_second': '473.7'}

 98%|█████████▊| 221500/225000 [433:45:16<8:28:01,  8.71s/it]
 98%|█████████▊| 221501/225000 [433:45:28<9:14:08,  9.50s/it]
 98%|█████████▊| 221502/225000 [433:45:41<10:22:18, 10.67s/it]
 98%|█████████▊| 221503/225000 [433:45:49<9:25:07,  9.70s/it] 
 98%|█████████▊| 221504/225000 [433:45:56<8:36:53,  8.87s/it]
 98%|█████████▊| 221505/225000 [433:46:02<7:50:24,  8.08s/it]
 98%|█████████▊| 221506/225000 [433:46:13<8:51:33,  9.13s/it]
 98%|█████████▊| 221507/225000 [433:46:21<8:34:04,  8.83s/it]
 98%|█████████▊| 221508/225000 [433:46:32<9:01:07,  9.30s/it]
 98%|█████████▊| 221509/225000 [433:46:37<7:46:00,  8.01s/it]
 98%|█████████▊| 221510/225000 [433:46:43<7:17:04,  7.51s/it]
 98%|█████████▊| 221511/225000 [433:46:52<7:45:55,  8.01s/it]
 98%|█████████▊| 221512/225000 [433:47:06<9:18:51,  9.61s/it]
 98%|█████████▊| 221513/225000 [433:47:12<8:23:34,  8.66s/it]
 98%|█████████▊| 221514/225000 [433:47:19<7:54:07,  8.16s/it]
 98%|█████████▊| 221515/225000 [433:47:31<8:50:51,  9.14s/it]
 98%|█████████▊| 221516/225000 [433:47:42<9:31:22,  9.84s/it]
 98%|█████████▊| 221517/225000 [433:47:49<8:39:18,  8.95s/it]
 98%|█████████▊| 221518/225000 [433:47:57<8:27:19,  8.74s/it]
 98%|█████████▊| 221519/225000 [433:48:06<8:25:19,  8.71s/it]
 98%|█████████▊| 221520/225000 [433:48:18<9:21:47,  9.69s/it]
 98%|█████████▊| 221521/225000 [433:48:25<8:46:15,  9.08s/it]
 98%|█████████▊| 221522/225000 [433:48:38<9:54:05, 10.25s/it]
 98%|█████████▊| 221523/225000 [433:48:48<9:37:58,  9.97s/it]
 98%|█████████▊| 221524/225000 [433:48:53<8:14:27,  8.53s/it]
 98%|█████████▊| 221525/225000 [433:49:00<7:49:21,  8.10s/it]
 98%|█████████▊| 221526/225000 [433:49:15<9:39:54, 10.02s/it]
 98%|█████████▊| 221527/225000 [433:49:22<9:02:18,  9.37s/it]
 98%|█████████▊| 221528/225000 [433:49:30<8:30:28,  8.82s/it]
 98%|█████████▊| 221529/225000 [433:49:45<10:16:19, 10.65s/it]
 98%|█████████▊| 221530/225000 [433:49:52<9:08:01,  9.48s/it] 
 98%|█████████▊| 221531/225000 [433:50:01<9:14:39,  9.59s/it]
 98%|█████████▊| 221532/225000 [433:50:08<8:28:06,  8.79s/it]
 98%|█████████▊| 221533/225000 [433:50:16<8:08:34,  8.46s/it]
 98%|█████████▊| 221534/225000 [433:50:24<7:59:17,  8.30s/it]
 98%|█████████▊| 221535/225000 [433:50:39<10:04:49, 10.47s/it]
 98%|█████████▊| 221536/225000 [433:50:52<10:42:30, 11.13s/it]
 98%|█████████▊| 221537/225000 [433:51:01<9:59:32, 10.39s/it] 
 98%|█████████▊| 221538/225000 [433:51:08<9:04:20,  9.43s/it]
 98%|█████████▊| 221539/225000 [433:51:14<8:06:23,  8.43s/it]
 98%|█████████▊| 221540/225000 [433:51:22<7:53:46,  8.22s/it]
 98%|█████████▊| 221541/225000 [433:51:27<6:59:17,  7.27s/it]
 98%|█████████▊| 221542/225000 [433:51:43<9:38:20, 10.03s/it]
 98%|█████████▊| 221543/225000 [433:51:47<7:50:07,  8.16s/it]
 98%|█████████▊| 221544/225000 [433:51:54<7:18:27,  7.61s/it]
 98%|█████████▊| 221545/225000 [433:52:08<9:18:24,  9.70s/it]
 98%|█████████▊| 221546/225000 [433:52:15<8:26:25,  8.80s/it]
 98%|█████████▊| 221547/225000 [433:52:25<8:42:42,  9.08s/it]
 98%|█████████▊| 221548/225000 [433:52:36<9:29:40,  9.90s/it]
 98%|█████████▊| 221549/225000 [433:52:46<9:21:42,  9.77s/it]
 98%|█████████▊| 221550/225000 [433:52:55<9:06:19,  9.50s/it]
                                                             
{'loss': '1.325', 'grad_norm': '1.486', 'learning_rate': '3.37e-08', 'epoch': '0.9847', 'num_input_tokens_seen': 739858568, 'train_runtime': '1.562e+06', 'train_tokens_per_second': '473.7'}

 98%|█████████▊| 221550/225000 [433:52:55<9:06:19,  9.50s/it]
 98%|█████████▊| 221551/225000 [433:53:07<9:48:50, 10.24s/it]
 98%|█████████▊| 221552/225000 [433:53:13<8:40:39,  9.06s/it]
 98%|█████████▊| 221553/225000 [433:53:18<7:40:04,  8.01s/it]
 98%|█████████▊| 221554/225000 [433:53:31<8:55:12,  9.32s/it]
 98%|█████████▊| 221555/225000 [433:53:38<8:14:41,  8.62s/it]
 98%|█████████▊| 221556/225000 [433:53:53<10:09:13, 10.61s/it]
 98%|█████████▊| 221557/225000 [433:53:59<8:41:54,  9.10s/it] 
 98%|█████████▊| 221558/225000 [433:54:11<9:29:27,  9.93s/it]
 98%|█████████▊| 221559/225000 [433:54:24<10:23:29, 10.87s/it]
 98%|█████████▊| 221560/225000 [433:54:30<9:03:26,  9.48s/it] 
 98%|█████████▊| 221561/225000 [433:54:36<8:04:05,  8.45s/it]
 98%|█████████▊| 221562/225000 [433:54:48<9:14:34,  9.68s/it]
 98%|█████████▊| 221563/225000 [433:54:56<8:35:33,  9.00s/it]
 98%|█████████▊| 221564/225000 [433:55:01<7:35:59,  7.96s/it]
 98%|█████████▊| 221565/225000 [433:55:07<6:58:47,  7.32s/it]
 98%|█████████▊| 221566/225000 [433:55:22<8:59:55,  9.43s/it]
 98%|█████████▊| 221567/225000 [433:55:34<9:44:07, 10.21s/it]
 98%|█████████▊| 221568/225000 [433:55:42<9:15:14,  9.71s/it]
 98%|█████████▊| 221569/225000 [433:55:48<8:17:43,  8.70s/it]
 98%|█████████▊| 221570/225000 [433:55:54<7:28:26,  7.84s/it]
 98%|█████████▊| 221571/225000 [433:56:12<10:13:26, 10.73s/it]
 98%|█████████▊| 221572/225000 [433:56:22<10:05:46, 10.60s/it]
 98%|█████████▊| 221573/225000 [433:56:29<8:56:50,  9.40s/it] 
 98%|█████████▊| 221574/225000 [433:56:35<7:58:01,  8.37s/it]
 98%|█████████▊| 221575/225000 [433:56:42<7:39:09,  8.04s/it]
 98%|█████████▊| 221576/225000 [433:56:48<7:08:12,  7.50s/it]
 98%|█████████▊| 221577/225000 [433:57:03<9:06:04,  9.57s/it]
 98%|█████████▊| 221578/225000 [433:57:10<8:35:40,  9.04s/it]
 98%|█████████▊| 221579/225000 [433:57:19<8:25:00,  8.86s/it]
 98%|█████████▊| 221580/225000 [433:57:27<8:10:21,  8.60s/it]
 98%|█████████▊| 221581/225000 [433:57:37<8:28:58,  8.93s/it]
 98%|█████████▊| 221582/225000 [433:57:43<7:48:07,  8.22s/it]
 98%|█████████▊| 221583/225000 [433:57:57<9:32:12, 10.05s/it]
 98%|█████████▊| 221584/225000 [433:58:04<8:26:58,  8.90s/it]
 98%|█████████▊| 221585/225000 [433:58:15<9:16:12,  9.77s/it]
 98%|█████████▊| 221586/225000 [433:58:24<8:49:14,  9.30s/it]
 98%|█████████▊| 221587/225000 [433:58:30<7:59:36,  8.43s/it]
 98%|█████████▊| 221588/225000 [433:58:38<7:45:05,  8.18s/it]
 98%|█████████▊| 221589/225000 [433:58:51<9:20:58,  9.87s/it]
 98%|█████████▊| 221590/225000 [433:59:02<9:35:52, 10.13s/it]
 98%|█████████▊| 221591/225000 [433:59:13<9:53:08, 10.44s/it]
 98%|█████████▊| 221592/225000 [433:59:20<8:45:26,  9.25s/it]
 98%|█████████▊| 221593/225000 [433:59:31<9:19:24,  9.85s/it]
 98%|█████████▊| 221594/225000 [433:59:40<9:01:56,  9.55s/it]
 98%|█████████▊| 221595/225000 [433:59:50<9:12:41,  9.74s/it]
 98%|█████████▊| 221596/225000 [433:59:58<8:33:07,  9.04s/it]
 98%|█████████▊| 221597/225000 [434:00:12<10:13:21, 10.81s/it]
 98%|█████████▊| 221598/225000 [434:00:18<8:51:34,  9.38s/it] 
 98%|█████████▊| 221599/225000 [434:00:26<8:22:39,  8.87s/it]
 98%|█████████▊| 221600/225000 [434:00:35<8:29:15,  8.99s/it]
                                                             
{'loss': '1.304', 'grad_norm': '1.42', 'learning_rate': '3.276e-08', 'epoch': '0.9849', 'num_input_tokens_seen': 740101840, 'train_runtime': '1.562e+06', 'train_tokens_per_second': '473.7'}

 98%|█████████▊| 221600/225000 [434:00:35<8:29:15,  8.99s/it]
 98%|█████████▊| 221601/225000 [434:00:42<7:41:42,  8.15s/it]
 98%|█████████▊| 221602/225000 [434:00:51<8:09:36,  8.65s/it]
 98%|█████████▊| 221603/225000 [434:00:55<6:50:34,  7.25s/it]
 98%|█████████▊| 221604/225000 [434:01:01<6:24:46,  6.80s/it]
 98%|█████████▊| 221605/225000 [434:01:11<7:14:34,  7.68s/it]
 98%|█████████▊| 221606/225000 [434:01:18<7:05:23,  7.52s/it]
 98%|█████████▊| 221607/225000 [434:01:29<8:10:26,  8.67s/it]
 98%|█████████▊| 221608/225000 [434:01:37<7:54:22,  8.39s/it]
 98%|█████████▊| 221609/225000 [434:01:44<7:36:00,  8.07s/it]
 98%|█████████▊| 221610/225000 [434:01:51<7:17:37,  7.75s/it]
 98%|█████████▊| 221611/225000 [434:02:05<9:04:11,  9.63s/it]
 98%|█████████▊| 221612/225000 [434:02:12<8:08:52,  8.66s/it]
 98%|█████████▊| 221613/225000 [434:02:17<7:13:57,  7.69s/it]
 98%|█████████▊| 221614/225000 [434:02:33<9:29:27, 10.09s/it]
 98%|█████████▊| 221615/225000 [434:02:40<8:37:36,  9.17s/it]
 98%|█████████▊| 221616/225000 [434:02:46<7:36:17,  8.09s/it]
 98%|█████████▊| 221617/225000 [434:02:58<8:50:30,  9.41s/it]
 98%|█████████▊| 221618/225000 [434:03:05<8:15:11,  8.79s/it]
 98%|█████████▊| 221619/225000 [434:03:14<8:11:49,  8.73s/it]
 98%|█████████▊| 221620/225000 [434:03:23<8:20:30,  8.88s/it]
 98%|█████████▊| 221621/225000 [434:03:35<9:14:19,  9.84s/it]
 98%|█████████▊| 221622/225000 [434:03:42<8:29:03,  9.04s/it]
 98%|█████████▊| 221623/225000 [434:03:54<9:10:23,  9.78s/it]
 98%|█████████▊| 221624/225000 [434:04:02<8:48:02,  9.38s/it]
 98%|█████████▊| 221625/225000 [434:04:14<9:22:19, 10.00s/it]
 99%|█████████▊| 221626/225000 [434:04:20<8:17:26,  8.85s/it]
 99%|█████████▊| 221627/225000 [434:04:29<8:21:33,  8.92s/it]
 99%|█████████▊| 221628/225000 [434:04:39<8:39:08,  9.24s/it]
 99%|█████████▊| 221629/225000 [434:04:46<8:03:39,  8.61s/it]
 99%|█████████▊| 221630/225000 [434:04:52<7:09:33,  7.65s/it]
 99%|█████████▊| 221631/225000 [434:05:03<8:06:42,  8.67s/it]
 99%|█████████▊| 221632/225000 [434:05:13<8:33:17,  9.14s/it]
 99%|█████████▊| 221633/225000 [434:05:21<8:08:50,  8.71s/it]
 99%|█████████▊| 221634/225000 [434:05:26<7:08:27,  7.64s/it]
 99%|█████████▊| 221635/225000 [434:05:40<8:59:13,  9.61s/it]
 99%|█████████▊| 221636/225000 [434:05:47<8:08:08,  8.71s/it]
 99%|█████████▊| 221637/225000 [434:05:55<8:01:50,  8.60s/it]
 99%|█████████▊| 221638/225000 [434:06:02<7:28:49,  8.01s/it]
 99%|█████████▊| 221639/225000 [434:06:12<8:11:42,  8.78s/it]
 99%|█████████▊| 221640/225000 [434:06:19<7:38:38,  8.19s/it]
 99%|█████████▊| 221641/225000 [434:06:32<9:05:41,  9.75s/it]
 99%|█████████▊| 221642/225000 [434:06:39<8:21:17,  8.96s/it]
 99%|█████████▊| 221643/225000 [434:06:51<9:03:07,  9.71s/it]
 99%|█████████▊| 221644/225000 [434:06:59<8:42:33,  9.34s/it]
 99%|█████████▊| 221645/225000 [434:07:07<8:09:00,  8.75s/it]
 99%|█████████▊| 221646/225000 [434:07:17<8:30:52,  9.14s/it]
 99%|█████████▊| 221647/225000 [434:07:24<8:05:51,  8.69s/it]
 99%|█████████▊| 221648/225000 [434:07:35<8:30:10,  9.13s/it]
 99%|█████████▊| 221649/225000 [434:07:42<8:04:01,  8.67s/it]
 99%|█████████▊| 221650/225000 [434:07:49<7:27:43,  8.02s/it]
                                                             
{'loss': '1.324', 'grad_norm': '2.1', 'learning_rate': '3.183e-08', 'epoch': '0.9851', 'num_input_tokens_seen': 740342864, 'train_runtime': '1.563e+06', 'train_tokens_per_second': '473.7'}

 99%|█████████▊| 221650/225000 [434:07:49<7:27:43,  8.02s/it]
 99%|█████████▊| 221651/225000 [434:08:00<8:29:21,  9.13s/it]
 99%|█████████▊| 221652/225000 [434:08:08<8:01:22,  8.63s/it]
 99%|█████████▊| 221653/225000 [434:08:20<9:06:07,  9.79s/it]
 99%|█████████▊| 221654/225000 [434:08:30<8:55:15,  9.60s/it]
 99%|█████████▊| 221655/225000 [434:08:37<8:17:15,  8.92s/it]
 99%|█████████▊| 221656/225000 [434:08:44<7:53:19,  8.49s/it]
 99%|█████████▊| 221657/225000 [434:08:53<8:02:58,  8.67s/it]
 99%|█████████▊| 221658/225000 [434:09:00<7:34:43,  8.16s/it]
 99%|█████████▊| 221659/225000 [434:09:06<6:48:10,  7.33s/it]
 99%|█████████▊| 221660/225000 [434:09:20<8:44:48,  9.43s/it]
 99%|█████████▊| 221661/225000 [434:09:31<9:06:06,  9.81s/it]
 99%|█████████▊| 221662/225000 [434:09:37<7:59:18,  8.62s/it]
 99%|█████████▊| 221663/225000 [434:09:46<8:05:39,  8.73s/it]
 99%|█████████▊| 221664/225000 [434:09:58<9:03:04,  9.77s/it]
 99%|█████████▊| 221665/225000 [434:10:04<7:59:10,  8.62s/it]
 99%|█████████▊| 221666/225000 [434:10:12<7:48:19,  8.43s/it]
 99%|█████████▊| 221667/225000 [434:10:20<7:51:32,  8.49s/it]
 99%|█████████▊| 221668/225000 [434:10:35<9:32:29, 10.31s/it]
 99%|█████████▊| 221669/225000 [434:10:47<10:06:40, 10.93s/it]
 99%|█████████▊| 221670/225000 [434:10:53<8:36:11,  9.30s/it] 
 99%|█████████▊| 221671/225000 [434:11:00<7:55:43,  8.57s/it]
 99%|█████████▊| 221672/225000 [434:11:11<8:42:13,  9.42s/it]
 99%|█████████▊| 221673/225000 [434:11:18<8:07:38,  8.79s/it]
 99%|█████████▊| 221674/225000 [434:11:26<7:50:42,  8.49s/it]
 99%|█████████▊| 221675/225000 [434:11:37<8:20:00,  9.02s/it]
 99%|█████████▊| 221676/225000 [434:11:43<7:43:55,  8.37s/it]
 99%|█████████▊| 221677/225000 [434:11:51<7:24:43,  8.03s/it]
 99%|█████████▊| 221678/225000 [434:11:57<6:52:19,  7.45s/it]
 99%|█████████▊| 221679/225000 [434:12:02<6:12:47,  6.74s/it]
 99%|█████████▊| 221680/225000 [434:12:11<7:01:53,  7.62s/it]
 99%|█████████▊| 221681/225000 [434:12:27<9:16:33, 10.06s/it]
 99%|█████████▊| 221682/225000 [434:12:37<9:18:30, 10.10s/it]
 99%|█████████▊| 221683/225000 [434:12:44<8:23:00,  9.10s/it]
 99%|█████████▊| 221684/225000 [434:12:55<8:47:22,  9.54s/it]
 99%|█████████▊| 221685/225000 [434:13:04<8:50:22,  9.60s/it]
 99%|█████████▊| 221686/225000 [434:13:13<8:32:08,  9.27s/it]
 99%|█████████▊| 221687/225000 [434:13:20<7:49:33,  8.50s/it]
 99%|█████████▊| 221688/225000 [434:13:34<9:24:37, 10.23s/it]
 99%|█████████▊| 221689/225000 [434:13:41<8:34:49,  9.33s/it]
 99%|█████████▊| 221690/225000 [434:13:48<7:46:57,  8.46s/it]
 99%|█████████▊| 221691/225000 [434:13:59<8:34:09,  9.32s/it]
 99%|█████████▊| 221692/225000 [434:14:07<8:09:18,  8.87s/it]
 99%|█████████▊| 221693/225000 [434:14:17<8:30:40,  9.27s/it]
 99%|█████████▊| 221694/225000 [434:14:28<8:52:47,  9.67s/it]
 99%|█████████▊| 221695/225000 [434:14:34<7:55:05,  8.63s/it]
 99%|█████████▊| 221696/225000 [434:14:40<7:19:03,  7.97s/it]
 99%|█████████▊| 221697/225000 [434:14:50<7:45:42,  8.46s/it]
 99%|█████████▊| 221698/225000 [434:15:07<10:13:41, 11.15s/it]
 99%|█████████▊| 221699/225000 [434:15:15<9:18:43, 10.16s/it] 
 99%|█████████▊| 221700/225000 [434:15:20<7:53:21,  8.61s/it]
                                                             
{'loss': '1.317', 'grad_norm': '1.521', 'learning_rate': '3.092e-08', 'epoch': '0.9853', 'num_input_tokens_seen': 740583184, 'train_runtime': '1.563e+06', 'train_tokens_per_second': '473.7'}

 99%|█████████▊| 221700/225000 [434:15:20<7:53:21,  8.61s/it]
 99%|█████████▊| 221701/225000 [434:15:32<8:43:10,  9.52s/it]
 99%|█████████▊| 221702/225000 [434:15:46<10:05:01, 11.01s/it]
 99%|█████████▊| 221703/225000 [434:15:52<8:32:00,  9.32s/it] 
 99%|█████████▊| 221704/225000 [434:16:00<8:10:20,  8.93s/it]
 99%|█████████▊| 221705/225000 [434:16:11<8:52:11,  9.69s/it]
 99%|█████████▊| 221706/225000 [434:16:18<8:12:43,  8.97s/it]
 99%|█████████▊| 221707/225000 [434:16:34<9:54:40, 10.84s/it]
 99%|█████████▊| 221708/225000 [434:16:40<8:48:40,  9.64s/it]
 99%|█████████▊| 221709/225000 [434:16:48<8:21:26,  9.14s/it]
 99%|█████████▊| 221710/225000 [434:16:58<8:29:30,  9.29s/it]
 99%|█████████▊| 221711/225000 [434:17:08<8:38:09,  9.45s/it]
 99%|█████████▊| 221712/225000 [434:17:19<9:03:23,  9.92s/it]
 99%|█████████▊| 221713/225000 [434:17:27<8:36:48,  9.43s/it]
 99%|█████████▊| 221714/225000 [434:17:36<8:28:26,  9.28s/it]
 99%|█████████▊| 221715/225000 [434:17:44<8:05:31,  8.87s/it]
 99%|█████████▊| 221716/225000 [434:17:50<7:26:23,  8.16s/it]
 99%|█████████▊| 221717/225000 [434:18:01<8:06:31,  8.89s/it]
 99%|█████████▊| 221718/225000 [434:18:10<8:11:38,  8.99s/it]
 99%|█████████▊| 221719/225000 [434:18:17<7:39:43,  8.41s/it]
 99%|█████████▊| 221720/225000 [434:18:28<8:21:34,  9.18s/it]
 99%|█████████▊| 221721/225000 [434:18:36<7:52:31,  8.65s/it]
 99%|█████████▊| 221722/225000 [434:18:41<7:03:52,  7.76s/it]
 99%|█████████▊| 221723/225000 [434:18:48<6:44:24,  7.40s/it]
 99%|█████████▊| 221724/225000 [434:18:56<6:47:01,  7.45s/it]
 99%|█████████▊| 221725/225000 [434:19:10<8:38:31,  9.50s/it]
 99%|█████████▊| 221726/225000 [434:19:21<8:59:00,  9.88s/it]
 99%|█████████▊| 221727/225000 [434:19:27<8:01:37,  8.83s/it]
 99%|█████████▊| 221728/225000 [434:19:34<7:36:16,  8.37s/it]
 99%|█████████▊| 221729/225000 [434:19:41<7:03:55,  7.78s/it]
 99%|█████████▊| 221730/225000 [434:19:57<9:19:44, 10.27s/it]
 99%|█████████▊| 221731/225000 [434:20:02<8:02:02,  8.85s/it]
 99%|█████████▊| 221732/225000 [434:20:13<8:31:37,  9.39s/it]
 99%|█████████▊| 221733/225000 [434:20:23<8:36:52,  9.49s/it]
 99%|█████████▊| 221734/225000 [434:20:31<8:19:54,  9.18s/it]
 99%|█████████▊| 221735/225000 [434:20:39<7:54:56,  8.73s/it]
 99%|█████████▊| 221736/225000 [434:20:50<8:33:25,  9.44s/it]
 99%|█████████▊| 221737/225000 [434:20:56<7:46:09,  8.57s/it]
 99%|█████████▊| 221738/225000 [434:21:01<6:44:45,  7.45s/it]
 99%|█████████▊| 221739/225000 [434:21:11<7:22:13,  8.14s/it]
 99%|█████████▊| 221740/225000 [434:21:22<8:10:12,  9.02s/it]
 99%|█████████▊| 221741/225000 [434:21:34<8:57:12,  9.89s/it]
 99%|█████████▊| 221742/225000 [434:21:44<9:01:38,  9.98s/it]
 99%|█████████▊| 221743/225000 [434:21:50<7:54:25,  8.74s/it]
 99%|█████████▊| 221744/225000 [434:21:56<7:01:22,  7.76s/it]
 99%|█████████▊| 221745/225000 [434:22:08<8:11:16,  9.06s/it]
 99%|█████████▊| 221746/225000 [434:22:20<9:13:41, 10.21s/it]
 99%|█████████▊| 221747/225000 [434:22:29<8:43:40,  9.66s/it]
 99%|█████████▊| 221748/225000 [434:22:43<9:48:31, 10.86s/it]
 99%|█████████▊| 221749/225000 [434:22:49<8:42:33,  9.64s/it]
 99%|█████████▊| 221750/225000 [434:22:59<8:49:35,  9.78s/it]
                                                             
{'loss': '1.334', 'grad_norm': '1.205', 'learning_rate': '3.002e-08', 'epoch': '0.9856', 'num_input_tokens_seen': 740829096, 'train_runtime': '1.564e+06', 'train_tokens_per_second': '473.7'}

 99%|█████████▊| 221750/225000 [434:22:59<8:49:35,  9.78s/it]
 99%|█████████▊| 221751/225000 [434:23:05<7:43:24,  8.56s/it]
 99%|█████████▊| 221752/225000 [434:23:13<7:25:29,  8.23s/it]
 99%|█████████▊| 221753/225000 [434:23:20<7:04:11,  7.84s/it]
 99%|█████████▊| 221754/225000 [434:23:35<9:03:55, 10.05s/it]
 99%|█████████▊| 221755/225000 [434:23:41<7:55:51,  8.80s/it]
 99%|█████████▊| 221756/225000 [434:23:47<7:15:39,  8.06s/it]
 99%|█████████▊| 221757/225000 [434:23:56<7:39:30,  8.50s/it]
 99%|█████████▊| 221758/225000 [434:24:08<8:24:11,  9.33s/it]
 99%|█████████▊| 221759/225000 [434:24:15<7:49:41,  8.70s/it]
 99%|█████████▊| 221760/225000 [434:24:22<7:26:23,  8.27s/it]
 99%|█████████▊| 221761/225000 [434:24:28<6:44:52,  7.50s/it]
 99%|█████████▊| 221762/225000 [434:24:35<6:29:42,  7.22s/it]
 99%|█████████▊| 221763/225000 [434:24:46<7:35:38,  8.45s/it]
 99%|█████████▊| 221764/225000 [434:24:57<8:23:47,  9.34s/it]
 99%|█████████▊| 221765/225000 [434:25:09<9:01:39, 10.05s/it]
 99%|█████████▊| 221766/225000 [434:25:15<7:53:40,  8.79s/it]
 99%|█████████▊| 221767/225000 [434:25:26<8:34:29,  9.55s/it]
 99%|█████████▊| 221768/225000 [434:25:34<8:03:00,  8.97s/it]
 99%|█████████▊| 221769/225000 [434:25:41<7:27:37,  8.31s/it]
 99%|█████████▊| 221770/225000 [434:25:56<9:20:39, 10.41s/it]
 99%|█████████▊| 221771/225000 [434:26:04<8:47:01,  9.79s/it]
 99%|█████████▊| 221772/225000 [434:26:11<8:03:43,  8.99s/it]
 99%|█████████▊| 221773/225000 [434:26:17<7:10:22,  8.00s/it]
 99%|█████████▊| 221774/225000 [434:26:25<7:13:22,  8.06s/it]
 99%|█████████▊| 221775/225000 [434:26:37<8:14:36,  9.20s/it]
 99%|█████████▊| 221776/225000 [434:26:50<9:15:05, 10.33s/it]
 99%|█████████▊| 221777/225000 [434:26:57<8:21:08,  9.33s/it]
 99%|█████████▊| 221778/225000 [434:27:05<7:54:58,  8.84s/it]
 99%|█████████▊| 221779/225000 [434:27:13<7:39:48,  8.57s/it]
 99%|█████████▊| 221780/225000 [434:27:18<6:48:05,  7.60s/it]
 99%|█████████▊| 221781/225000 [434:27:26<6:47:59,  7.60s/it]
 99%|█████████▊| 221782/225000 [434:27:35<7:23:23,  8.27s/it]
 99%|█████████▊| 221783/225000 [434:27:50<8:58:46, 10.05s/it]
 99%|█████████▊| 221784/225000 [434:28:00<8:57:58, 10.04s/it]
 99%|█████████▊| 221785/225000 [434:28:08<8:34:07,  9.59s/it]
 99%|█████████▊| 221786/225000 [434:28:16<8:11:55,  9.18s/it]
 99%|█████████▊| 221787/225000 [434:28:24<7:51:54,  8.81s/it]
 99%|█████████▊| 221788/225000 [434:28:31<7:10:06,  8.03s/it]
 99%|█████████▊| 221789/225000 [434:28:38<6:56:13,  7.78s/it]
 99%|█████████▊| 221790/225000 [434:28:52<8:45:24,  9.82s/it]
 99%|█████████▊| 221791/225000 [434:28:57<7:26:51,  8.35s/it]
 99%|█████████▊| 221792/225000 [434:29:05<7:09:50,  8.04s/it]
 99%|█████████▊| 221793/225000 [434:29:14<7:38:46,  8.58s/it]
 99%|█████████▊| 221794/225000 [434:29:23<7:34:31,  8.51s/it]
 99%|█████████▊| 221795/225000 [434:29:33<8:06:46,  9.11s/it]
 99%|█████████▊| 221796/225000 [434:29:42<7:56:35,  8.92s/it]
 99%|█████████▊| 221797/225000 [434:29:50<7:47:46,  8.76s/it]
 99%|█████████▊| 221798/225000 [434:29:56<6:54:45,  7.77s/it]
 99%|█████████▊| 221799/225000 [434:30:13<9:25:05, 10.59s/it]
 99%|█████████▊| 221800/225000 [434:30:23<9:20:39, 10.51s/it]
                                                             
{'loss': '1.307', 'grad_norm': '1.324', 'learning_rate': '2.913e-08', 'epoch': '0.9858', 'num_input_tokens_seen': 741067136, 'train_runtime': '1.564e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 221800/225000 [434:30:23<9:20:39, 10.51s/it]
 99%|█████████▊| 221801/225000 [434:30:33<9:08:35, 10.29s/it]
 99%|█████████▊| 221802/225000 [434:30:41<8:38:00,  9.72s/it]
 99%|█████████▊| 221803/225000 [434:30:51<8:40:31,  9.77s/it]
 99%|█████████▊| 221804/225000 [434:30:56<7:17:24,  8.21s/it]
 99%|█████████▊| 221805/225000 [434:31:07<8:06:55,  9.14s/it]
 99%|█████████▊| 221806/225000 [434:31:15<7:40:41,  8.65s/it]
 99%|█████████▊| 221807/225000 [434:31:21<7:04:37,  7.98s/it]
 99%|█████████▊| 221808/225000 [434:31:38<9:33:40, 10.78s/it]
 99%|█████████▊| 221809/225000 [434:31:45<8:23:12,  9.46s/it]
 99%|█████████▊| 221810/225000 [434:31:54<8:14:45,  9.31s/it]
 99%|█████████▊| 221811/225000 [434:32:07<9:17:30, 10.49s/it]
 99%|█████████▊| 221812/225000 [434:32:19<9:51:12, 11.13s/it]
 99%|█████████▊| 221813/225000 [434:32:25<8:15:29,  9.33s/it]
 99%|█████████▊| 221814/225000 [434:32:35<8:32:15,  9.65s/it]
 99%|█████████▊| 221815/225000 [434:32:42<7:51:12,  8.88s/it]
 99%|█████████▊| 221816/225000 [434:32:51<7:55:54,  8.97s/it]
 99%|█████████▊| 221817/225000 [434:32:58<7:26:25,  8.42s/it]
 99%|█████████▊| 221818/225000 [434:33:11<8:31:27,  9.64s/it]
 99%|█████████▊| 221819/225000 [434:33:19<7:59:19,  9.04s/it]
 99%|█████████▊| 221820/225000 [434:33:30<8:33:12,  9.68s/it]
 99%|█████████▊| 221821/225000 [434:33:36<7:31:57,  8.53s/it]
 99%|█████████▊| 221822/225000 [434:33:42<7:02:56,  7.99s/it]
 99%|█████████▊| 221823/225000 [434:33:53<7:53:20,  8.94s/it]
 99%|█████████▊| 221824/225000 [434:34:00<7:15:20,  8.22s/it]
 99%|█████████▊| 221825/225000 [434:34:11<7:54:31,  8.97s/it]
 99%|█████████▊| 221826/225000 [434:34:19<7:42:45,  8.75s/it]
 99%|█████████▊| 221827/225000 [434:34:26<7:08:11,  8.10s/it]
 99%|█████████▊| 221828/225000 [434:34:34<7:06:46,  8.07s/it]
 99%|█████████▊| 221829/225000 [434:34:40<6:46:13,  7.69s/it]
 99%|█████████▊| 221830/225000 [434:34:56<8:46:00,  9.96s/it]
 99%|█████████▊| 221831/225000 [434:35:06<8:58:06, 10.19s/it]
 99%|█████████▊| 221832/225000 [434:35:19<9:31:46, 10.83s/it]
 99%|█████████▊| 221833/225000 [434:35:26<8:33:12,  9.72s/it]
 99%|█████████▊| 221834/225000 [434:35:37<8:54:26, 10.13s/it]
 99%|█████████▊| 221835/225000 [434:35:45<8:22:35,  9.53s/it]
 99%|█████████▊| 221836/225000 [434:35:52<7:43:57,  8.80s/it]
 99%|█████████▊| 221837/225000 [434:36:06<9:06:48, 10.37s/it]
 99%|█████████▊| 221838/225000 [434:36:17<9:20:48, 10.64s/it]
 99%|█████████▊| 221839/225000 [434:36:30<9:46:51, 11.14s/it]
 99%|█████████▊| 221840/225000 [434:36:41<9:41:50, 11.05s/it]
 99%|█████████▊| 221841/225000 [434:36:48<8:43:47,  9.95s/it]
 99%|█████████▊| 221842/225000 [434:36:53<7:32:05,  8.59s/it]
 99%|█████████▊| 221843/225000 [434:37:00<7:01:01,  8.00s/it]
 99%|█████████▊| 221844/225000 [434:37:16<9:11:55, 10.49s/it]
 99%|█████████▊| 221845/225000 [434:37:24<8:35:05,  9.80s/it]
 99%|█████████▊| 221846/225000 [434:37:33<8:08:50,  9.30s/it]
 99%|█████████▊| 221847/225000 [434:37:37<6:53:50,  7.88s/it]
 99%|█████████▊| 221848/225000 [434:37:45<6:49:46,  7.80s/it]
 99%|█████████▊| 221849/225000 [434:38:00<8:41:45,  9.93s/it]
 99%|█████████▊| 221850/225000 [434:38:06<7:39:23,  8.75s/it]
                                                             
{'loss': '1.322', 'grad_norm': '1.64', 'learning_rate': '2.825e-08', 'epoch': '0.986', 'num_input_tokens_seen': 741314272, 'train_runtime': '1.565e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 221850/225000 [434:38:06<7:39:23,  8.75s/it]
 99%|█████████▊| 221851/225000 [434:38:12<7:04:51,  8.10s/it]
 99%|█████████▊| 221852/225000 [434:38:26<8:36:53,  9.85s/it]
 99%|█████████▊| 221853/225000 [434:38:34<8:09:35,  9.33s/it]
 99%|█████████▊| 221854/225000 [434:38:42<7:41:19,  8.80s/it]
 99%|█████████▊| 221855/225000 [434:38:55<8:54:13, 10.19s/it]
 99%|█████████▊| 221856/225000 [434:39:07<9:22:14, 10.73s/it]
 99%|█████████▊| 221857/225000 [434:39:18<9:28:01, 10.84s/it]
 99%|█████████▊| 221858/225000 [434:39:24<8:11:08,  9.38s/it]
 99%|█████████▊| 221859/225000 [434:39:33<8:02:19,  9.21s/it]
 99%|█████████▊| 221860/225000 [434:39:40<7:21:47,  8.44s/it]
 99%|█████████▊| 221861/225000 [434:39:50<7:44:01,  8.87s/it]
 99%|█████████▊| 221862/225000 [434:39:58<7:32:31,  8.65s/it]
 99%|█████████▊| 221863/225000 [434:40:03<6:37:05,  7.60s/it]
 99%|█████████▊| 221864/225000 [434:40:16<8:03:33,  9.25s/it]
 99%|█████████▊| 221865/225000 [434:40:26<8:14:26,  9.46s/it]
 99%|█████████▊| 221866/225000 [434:40:33<7:31:22,  8.64s/it]
 99%|█████████▊| 221867/225000 [434:40:48<9:07:37, 10.49s/it]
 99%|█████████▊| 221868/225000 [434:40:55<8:16:16,  9.51s/it]
 99%|█████████▊| 221869/225000 [434:41:05<8:20:45,  9.60s/it]
 99%|█████████▊| 221870/225000 [434:41:10<7:18:31,  8.41s/it]
 99%|█████████▊| 221871/225000 [434:41:17<6:55:52,  7.97s/it]
 99%|█████████▊| 221872/225000 [434:41:24<6:30:37,  7.49s/it]
 99%|█████████▊| 221873/225000 [434:41:39<8:30:47,  9.80s/it]
 99%|█████████▊| 221874/225000 [434:41:50<8:47:16, 10.12s/it]
 99%|█████████▊| 221875/225000 [434:41:57<8:01:51,  9.25s/it]
 99%|█████████▊| 221876/225000 [434:42:05<7:51:21,  9.05s/it]
 99%|█████████▊| 221877/225000 [434:42:13<7:35:46,  8.76s/it]
 99%|█████████▊| 221878/225000 [434:42:22<7:36:49,  8.78s/it]
 99%|█████████▊| 221879/225000 [434:42:30<7:17:31,  8.41s/it]
 99%|█████████▊| 221880/225000 [434:42:37<6:52:26,  7.93s/it]
 99%|█████████▊| 221881/225000 [434:42:49<7:56:32,  9.17s/it]
 99%|█████████▊| 221882/225000 [434:43:00<8:35:38,  9.92s/it]
 99%|█████████▊| 221883/225000 [434:43:10<8:27:20,  9.77s/it]
 99%|█████████▊| 221884/225000 [434:43:18<8:05:28,  9.35s/it]
 99%|█████████▊| 221885/225000 [434:43:28<8:16:47,  9.57s/it]
 99%|█████████▊| 221886/225000 [434:43:38<8:12:52,  9.50s/it]
 99%|█████████▊| 221887/225000 [434:43:46<8:02:11,  9.29s/it]
 99%|█████████▊| 221888/225000 [434:43:53<7:16:15,  8.41s/it]
 99%|█████████▊| 221889/225000 [434:44:04<7:53:48,  9.14s/it]
 99%|█████████▊| 221890/225000 [434:44:13<7:59:13,  9.25s/it]
 99%|█████████▊| 221891/225000 [434:44:20<7:23:40,  8.56s/it]
 99%|█████████▊| 221892/225000 [434:44:27<7:03:59,  8.19s/it]
 99%|█████████▊| 221893/225000 [434:44:35<6:48:16,  7.88s/it]
 99%|█████████▊| 221894/225000 [434:44:49<8:37:49, 10.00s/it]
 99%|█████████▊| 221895/225000 [434:44:58<8:18:08,  9.63s/it]
 99%|█████████▊| 221896/225000 [434:45:07<8:10:22,  9.48s/it]
 99%|█████████▊| 221897/225000 [434:45:14<7:30:02,  8.70s/it]
 99%|█████████▊| 221898/225000 [434:45:22<7:10:31,  8.33s/it]
 99%|█████████▊| 221899/225000 [434:45:35<8:24:00,  9.75s/it]
 99%|█████████▊| 221900/225000 [434:45:41<7:23:54,  8.59s/it]
                                                             
{'loss': '1.347', 'grad_norm': '1.517', 'learning_rate': '2.739e-08', 'epoch': '0.9862', 'num_input_tokens_seen': 741558776, 'train_runtime': '1.565e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 221900/225000 [434:45:41<7:23:54,  8.59s/it]
 99%|█████████▊| 221901/225000 [434:45:51<7:58:34,  9.27s/it]
 99%|█████████▊| 221902/225000 [434:46:03<8:27:03,  9.82s/it]
 99%|█████████▊| 221903/225000 [434:46:10<7:47:38,  9.06s/it]
 99%|█████████▊| 221904/225000 [434:46:18<7:40:16,  8.92s/it]
 99%|█████████▊| 221905/225000 [434:46:30<8:21:01,  9.71s/it]
 99%|█████████▊| 221906/225000 [434:46:37<7:35:57,  8.84s/it]
 99%|█████████▊| 221907/225000 [434:46:46<7:37:38,  8.88s/it]
 99%|█████████▊| 221908/225000 [434:46:52<7:00:57,  8.17s/it]
 99%|█████████▊| 221909/225000 [434:47:00<6:48:36,  7.93s/it]
 99%|█████████▊| 221910/225000 [434:47:08<6:49:17,  7.95s/it]
 99%|█████████▊| 221911/225000 [434:47:18<7:28:41,  8.72s/it]
 99%|█████████▊| 221912/225000 [434:47:24<6:50:51,  7.98s/it]
 99%|█████████▊| 221913/225000 [434:47:38<8:16:46,  9.66s/it]
 99%|█████████▊| 221914/225000 [434:47:47<8:13:12,  9.59s/it]
 99%|█████████▊| 221915/225000 [434:47:58<8:23:59,  9.80s/it]
 99%|█████████▊| 221916/225000 [434:48:11<9:16:25, 10.83s/it]
 99%|█████████▊| 221917/225000 [434:48:17<7:57:52,  9.30s/it]
 99%|█████████▊| 221918/225000 [434:48:25<7:48:11,  9.11s/it]
 99%|█████████▊| 221919/225000 [434:48:32<7:16:14,  8.50s/it]
 99%|█████████▊| 221920/225000 [434:48:43<7:52:30,  9.20s/it]
 99%|█████████▊| 221921/225000 [434:48:52<7:48:18,  9.13s/it]
 99%|█████████▊| 221922/225000 [434:49:00<7:33:48,  8.85s/it]
 99%|█████████▊| 221923/225000 [434:49:11<7:53:08,  9.23s/it]
 99%|█████████▊| 221924/225000 [434:49:19<7:45:51,  9.09s/it]
 99%|█████████▊| 221925/225000 [434:49:29<7:54:54,  9.27s/it]
 99%|█████████▊| 221926/225000 [434:49:37<7:41:02,  9.00s/it]
 99%|█████████▊| 221927/225000 [434:49:47<7:57:47,  9.33s/it]
 99%|█████████▊| 221928/225000 [434:49:55<7:33:27,  8.86s/it]
 99%|█████████▊| 221929/225000 [434:50:07<8:11:51,  9.61s/it]
 99%|█████████▊| 221930/225000 [434:50:14<7:32:26,  8.84s/it]
 99%|█████████▊| 221931/225000 [434:50:21<7:15:01,  8.50s/it]
 99%|█████████▊| 221932/225000 [434:50:29<7:01:14,  8.24s/it]
 99%|█████████▊| 221933/225000 [434:50:42<8:17:53,  9.74s/it]
 99%|█████████▊| 221934/225000 [434:50:52<8:23:40,  9.86s/it]
 99%|█████████▊| 221935/225000 [434:51:06<9:22:08, 11.00s/it]
 99%|█████████▊| 221936/225000 [434:51:13<8:24:52,  9.89s/it]
 99%|█████████▊| 221937/225000 [434:51:20<7:40:32,  9.02s/it]
 99%|█████████▊| 221938/225000 [434:51:28<7:16:45,  8.56s/it]
 99%|█████████▊| 221939/225000 [434:51:34<6:35:50,  7.76s/it]
 99%|█████████▊| 221940/225000 [434:51:49<8:25:51,  9.92s/it]
 99%|█████████▊| 221941/225000 [434:51:55<7:35:25,  8.93s/it]
 99%|█████████▊| 221942/225000 [434:52:06<8:08:33,  9.59s/it]
 99%|█████████▊| 221943/225000 [434:52:16<8:13:51,  9.69s/it]
 99%|█████████▊| 221944/225000 [434:52:25<7:53:37,  9.30s/it]
 99%|█████████▊| 221945/225000 [434:52:38<8:55:37, 10.52s/it]
 99%|█████████▊| 221946/225000 [434:52:46<8:07:52,  9.59s/it]
 99%|█████████▊| 221947/225000 [434:52:53<7:32:56,  8.90s/it]
 99%|█████████▊| 221948/225000 [434:53:01<7:29:00,  8.83s/it]
 99%|█████████▊| 221949/225000 [434:53:11<7:34:33,  8.94s/it]
 99%|█████████▊| 221950/225000 [434:53:18<7:14:19,  8.54s/it]
                                                             
{'loss': '1.31', 'grad_norm': '1.176', 'learning_rate': '2.654e-08', 'epoch': '0.9864', 'num_input_tokens_seen': 741801992, 'train_runtime': '1.566e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 221950/225000 [434:53:18<7:14:19,  8.54s/it]
 99%|█████████▊| 221951/225000 [434:53:29<7:40:29,  9.06s/it]
 99%|█████████▊| 221952/225000 [434:53:39<7:55:08,  9.35s/it]
 99%|█████████▊| 221953/225000 [434:53:44<7:01:28,  8.30s/it]
 99%|█████████▊| 221954/225000 [434:53:54<7:18:43,  8.64s/it]
 99%|█████████▊| 221955/225000 [434:54:06<8:16:09,  9.78s/it]
 99%|█████████▊| 221956/225000 [434:54:15<7:55:22,  9.37s/it]
 99%|█████████▊| 221957/225000 [434:54:24<7:51:04,  9.29s/it]
 99%|█████████▊| 221958/225000 [434:54:31<7:16:27,  8.61s/it]
 99%|█████████▊| 221959/225000 [434:54:36<6:24:06,  7.58s/it]
 99%|█████████▊| 221960/225000 [434:54:45<6:49:03,  8.07s/it]
 99%|█████████▊| 221961/225000 [434:54:53<6:40:54,  7.92s/it]
 99%|█████████▊| 221962/225000 [434:55:01<6:40:57,  7.92s/it]
 99%|█████████▊| 221963/225000 [434:55:07<6:17:26,  7.46s/it]
 99%|█████████▊| 221964/225000 [434:55:16<6:44:28,  7.99s/it]
 99%|█████████▊| 221965/225000 [434:55:29<7:52:11,  9.33s/it]
 99%|█████████▊| 221966/225000 [434:55:40<8:13:15,  9.75s/it]
 99%|█████████▊| 221967/225000 [434:55:46<7:18:36,  8.68s/it]
 99%|█████████▊| 221968/225000 [434:55:57<7:51:35,  9.33s/it]
 99%|█████████▊| 221969/225000 [434:56:05<7:33:18,  8.97s/it]
 99%|█████████▊| 221970/225000 [434:56:16<8:13:25,  9.77s/it]
 99%|█████████▊| 221971/225000 [434:56:22<7:08:22,  8.49s/it]
 99%|█████████▊| 221972/225000 [434:56:31<7:25:37,  8.83s/it]
 99%|█████████▊| 221973/225000 [434:56:38<6:48:05,  8.09s/it]
 99%|█████████▊| 221974/225000 [434:56:50<7:48:59,  9.30s/it]
 99%|█████████▊| 221975/225000 [434:56:58<7:29:31,  8.92s/it]
 99%|█████████▊| 221976/225000 [434:57:05<6:59:15,  8.32s/it]
 99%|█████████▊| 221977/225000 [434:57:11<6:19:51,  7.54s/it]
 99%|█████████▊| 221978/225000 [434:57:24<7:43:53,  9.21s/it]
 99%|█████████▊| 221979/225000 [434:57:33<7:37:27,  9.09s/it]
 99%|█████████▊| 221980/225000 [434:57:42<7:41:15,  9.16s/it]
 99%|█████████▊| 221981/225000 [434:57:49<7:14:53,  8.64s/it]
 99%|█████████▊| 221982/225000 [434:58:03<8:35:52, 10.26s/it]
 99%|█████████▊| 221983/225000 [434:58:12<8:14:07,  9.83s/it]
 99%|█████████▊| 221984/225000 [434:58:21<8:01:10,  9.57s/it]
 99%|█████████▊| 221985/225000 [434:58:29<7:33:22,  9.02s/it]
 99%|█████████▊| 221986/225000 [434:58:42<8:29:11, 10.14s/it]
 99%|█████████▊| 221987/225000 [434:58:51<8:13:19,  9.82s/it]
 99%|█████████▊| 221988/225000 [434:58:59<7:57:29,  9.51s/it]
 99%|█████████▊| 221989/225000 [434:59:07<7:31:49,  9.00s/it]
 99%|█████████▊| 221990/225000 [434:59:18<8:03:37,  9.64s/it]
 99%|█████████▊| 221991/225000 [434:59:27<7:52:11,  9.42s/it]
 99%|█████████▊| 221992/225000 [434:59:35<7:19:23,  8.76s/it]
 99%|█████████▊| 221993/225000 [434:59:44<7:28:42,  8.95s/it]
 99%|█████████▊| 221994/225000 [434:59:53<7:36:00,  9.10s/it]
 99%|█████████▊| 221995/225000 [435:00:04<7:55:49,  9.50s/it]
 99%|█████████▊| 221996/225000 [435:00:13<7:49:08,  9.37s/it]
 99%|█████████▊| 221997/225000 [435:00:20<7:12:08,  8.63s/it]
 99%|█████████▊| 221998/225000 [435:00:26<6:39:02,  7.98s/it]
 99%|█████████▊| 221999/225000 [435:00:37<7:26:09,  8.92s/it]
 99%|█████████▊| 222000/225000 [435:00:42<6:27:03,  7.74s/it]
                                                             
{'loss': '1.343', 'grad_norm': '1.232', 'learning_rate': '2.571e-08', 'epoch': '0.9867', 'num_input_tokens_seen': 742040800, 'train_runtime': '1.566e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 222000/225000 [435:00:42<6:27:03,  7.74s/it]
 99%|█████████▊| 222001/225000 [435:00:53<7:16:41,  8.74s/it]
 99%|█████████▊| 222002/225000 [435:01:03<7:27:53,  8.96s/it]
 99%|█████████▊| 222003/225000 [435:01:12<7:25:07,  8.91s/it]
 99%|█████████▊| 222004/225000 [435:01:20<7:15:53,  8.73s/it]
 99%|█████████▊| 222005/225000 [435:01:30<7:35:52,  9.13s/it]
 99%|█████████▊| 222006/225000 [435:01:36<6:54:31,  8.31s/it]
 99%|█████████▊| 222007/225000 [435:01:43<6:26:19,  7.74s/it]
 99%|█████████▊| 222008/225000 [435:01:48<5:51:42,  7.05s/it]
 99%|█████████▊| 222009/225000 [435:01:59<6:41:35,  8.06s/it]
 99%|█████████▊| 222010/225000 [435:02:05<6:16:57,  7.56s/it]
 99%|█████████▊| 222011/225000 [435:02:12<6:13:49,  7.50s/it]
 99%|█████████▊| 222012/225000 [435:02:20<6:07:03,  7.37s/it]
 99%|█████████▊| 222013/225000 [435:02:25<5:44:56,  6.93s/it]
 99%|█████████▊| 222014/225000 [435:02:38<7:13:43,  8.72s/it]
 99%|█████████▊| 222015/225000 [435:02:44<6:20:43,  7.65s/it]
 99%|█████████▊| 222016/225000 [435:02:55<7:13:26,  8.72s/it]
 99%|█████████▊| 222017/225000 [435:03:01<6:32:37,  7.90s/it]
 99%|█████████▊| 222018/225000 [435:03:08<6:24:47,  7.74s/it]
 99%|█████████▊| 222019/225000 [435:03:17<6:41:43,  8.09s/it]
 99%|█████████▊| 222020/225000 [435:03:31<8:12:45,  9.92s/it]
 99%|█████████▊| 222021/225000 [435:03:39<7:44:42,  9.36s/it]
 99%|█████████▊| 222022/225000 [435:03:48<7:33:16,  9.13s/it]
 99%|█████████▊| 222023/225000 [435:03:54<6:42:54,  8.12s/it]
 99%|█████████▊| 222024/225000 [435:04:09<8:34:07, 10.37s/it]
 99%|█████████▊| 222025/225000 [435:04:18<8:16:17, 10.01s/it]
 99%|█████████▊| 222026/225000 [435:04:33<9:23:01, 11.36s/it]
 99%|█████████▊| 222027/225000 [435:04:43<8:57:52, 10.86s/it]
 99%|█████████▊| 222028/225000 [435:04:49<7:55:45,  9.60s/it]
 99%|█████████▊| 222029/225000 [435:04:59<8:01:33,  9.73s/it]
 99%|█████████▊| 222030/225000 [435:05:07<7:25:32,  9.00s/it]
 99%|█████████▊| 222031/225000 [435:05:18<8:02:15,  9.75s/it]
 99%|█████████▊| 222032/225000 [435:05:30<8:35:14, 10.42s/it]
 99%|█████████▊| 222033/225000 [435:05:39<8:09:23,  9.90s/it]
 99%|█████████▊| 222034/225000 [435:05:47<7:49:33,  9.50s/it]
 99%|█████████▊| 222035/225000 [435:05:55<7:29:39,  9.10s/it]
 99%|█████████▊| 222036/225000 [435:06:02<6:47:40,  8.25s/it]
 99%|█████████▊| 222037/225000 [435:06:10<6:41:05,  8.12s/it]
 99%|█████████▊| 222038/225000 [435:06:15<6:07:25,  7.44s/it]
 99%|█████████▊| 222039/225000 [435:06:29<7:37:55,  9.28s/it]
 99%|█████████▊| 222040/225000 [435:06:34<6:42:24,  8.16s/it]
 99%|█████████▊| 222041/225000 [435:06:46<7:38:50,  9.30s/it]
 99%|█████████▊| 222042/225000 [435:06:53<6:58:41,  8.49s/it]
 99%|█████████▊| 222043/225000 [435:07:00<6:36:01,  8.04s/it]
 99%|█████████▊| 222044/225000 [435:07:05<5:44:43,  7.00s/it]
 99%|█████████▊| 222045/225000 [435:07:20<7:41:37,  9.37s/it]
 99%|█████████▊| 222046/225000 [435:07:28<7:25:58,  9.06s/it]
 99%|█████████▊| 222047/225000 [435:07:34<6:47:54,  8.29s/it]
 99%|█████████▊| 222048/225000 [435:07:43<6:52:18,  8.38s/it]
 99%|█████████▊| 222049/225000 [435:08:01<9:08:46, 11.16s/it]
 99%|█████████▊| 222050/225000 [435:08:10<8:45:47, 10.69s/it]
                                                             
{'loss': '1.325', 'grad_norm': '1.29', 'learning_rate': '2.489e-08', 'epoch': '0.9869', 'num_input_tokens_seen': 742280488, 'train_runtime': '1.566e+06', 'train_tokens_per_second': '473.8'}

 99%|█████████▊| 222050/225000 [435:08:10<8:45:47, 10.69s/it]
 99%|█████████▊| 222051/225000 [435:08:15<7:23:48,  9.03s/it]
 99%|█████████▊| 222052/225000 [435:08:24<7:16:24,  8.88s/it]
 99%|█████████▊| 222053/225000 [435:08:39<8:46:16, 10.71s/it]
 99%|█████████▊| 222054/225000 [435:08:44<7:26:20,  9.09s/it]
 99%|█████████▊| 222055/225000 [435:08:55<7:58:30,  9.75s/it]
 99%|█████████▊| 222056/225000 [435:09:03<7:22:22,  9.02s/it]
 99%|█████████▊| 222057/225000 [435:09:12<7:29:09,  9.16s/it]
 99%|█████████▊| 222058/225000 [435:09:21<7:19:54,  8.97s/it]
 99%|█████████▊| 222059/225000 [435:09:32<7:48:39,  9.56s/it]
 99%|█████████▊| 222060/225000 [435:09:37<6:44:58,  8.26s/it]
 99%|█████████▊| 222061/225000 [435:09:43<6:12:25,  7.60s/it]
 99%|█████████▊| 222062/225000 [435:09:51<6:18:16,  7.73s/it]
 99%|█████████▊| 222063/225000 [435:10:04<7:39:20,  9.38s/it]
 99%|█████████▊| 222064/225000 [435:10:11<7:04:54,  8.68s/it]
 99%|█████████▊| 222065/225000 [435:10:22<7:32:17,  9.25s/it]
 99%|█████████▊| 222066/225000 [435:10:31<7:24:00,  9.08s/it]
 99%|█████████▊| 222067/225000 [435:10:39<7:11:01,  8.82s/it]
 99%|█████████▊| 222068/225000 [435:10:44<6:21:49,  7.81s/it]
 99%|█████████▊| 222069/225000 [435:10:54<6:45:12,  8.30s/it]
 99%|█████████▊| 222070/225000 [435:11:06<7:37:17,  9.36s/it]
 99%|█████████▊| 222071/225000 [435:11:16<7:47:51,  9.58s/it]
 99%|█████████▊| 222072/225000 [435:11:21<6:49:00,  8.38s/it]
 99%|█████████▊| 222073/225000 [435:11:29<6:45:43,  8.32s/it]
 99%|█████████▊| 222074/225000 [435:11:35<6:07:09,  7.53s/it]
 99%|█████████▊| 222075/225000 [435:11:46<6:56:23,  8.54s/it]
 99%|█████████▊| 222076/225000 [435:11:53<6:30:44,  8.02s/it]
 99%|█████████▊| 222077/225000 [435:11:59<6:07:49,  7.55s/it]
 99%|█████████▊| 222078/225000 [435:12:09<6:43:59,  8.30s/it]
 99%|█████████▊| 222079/225000 [435:12:19<7:05:52,  8.75s/it]
 99%|█████████▊| 222080/225000 [435:12:35<8:51:21, 10.92s/it]
 99%|█████████▊| 222081/225000 [435:12:41<7:31:51,  9.29s/it]
 99%|█████████▊| 222082/225000 [435:12:51<7:51:43,  9.70s/it]
 99%|█████████▊| 222083/225000 [435:13:06<9:02:45, 11.16s/it]
 99%|█████████▊| 222084/225000 [435:13:15<8:28:55, 10.47s/it]
 99%|█████████▊| 222085/225000 [435:13:25<8:33:55, 10.58s/it]
 99%|█████████▊| 222086/225000 [435:13:33<7:49:52,  9.67s/it]
 99%|█████████▊| 222087/225000 [435:13:44<8:08:51, 10.07s/it]
 99%|█████████▊| 222088/225000 [435:13:51<7:19:49,  9.06s/it]
 99%|█████████▊| 222089/225000 [435:13:58<6:56:52,  8.59s/it]
 99%|█████████▊| 222090/225000 [435:14:07<7:00:44,  8.67s/it]
 99%|█████████▊| 222091/225000 [435:14:16<7:01:47,  8.70s/it]
 99%|█████████▊| 222092/225000 [435:14:24<6:55:38,  8.58s/it]
 99%|█████████▊| 222093/225000 [435:14:31<6:32:38,  8.10s/it]
 99%|█████████▊| 222094/225000 [435:14:44<7:37:31,  9.45s/it]
 99%|█████████▊| 222095/225000 [435:14:52<7:27:04,  9.23s/it]
 99%|█████████▊| 222096/225000 [435:15:02<7:37:38,  9.46s/it]
 99%|█████████▊| 222097/225000 [435:15:12<7:41:24,  9.54s/it]
 99%|█████████▊| 222098/225000 [435:15:19<7:04:47,  8.78s/it]
 99%|█████████▊| 222099/225000 [435:15:28<7:06:09,  8.81s/it]
 99%|█████████▊| 222100/225000 [435:15:34<6:31:27,  8.10s/it]
                                                             
{'loss': '1.325', 'grad_norm': '1.326', 'learning_rate': '2.408e-08', 'epoch': '0.9871', 'num_input_tokens_seen': 742523712, 'train_runtime': '1.567e+06', 'train_tokens_per_second': '473.9'}

 99%|█████████▊| 222100/225000 [435:15:34<6:31:27,  8.10s/it]
 99%|█████████▊| 222101/225000 [435:15:45<7:00:02,  8.69s/it]
 99%|█████████▊| 222102/225000 [435:15:53<6:59:53,  8.69s/it]
 99%|█████████▊| 222103/225000 [435:16:03<7:09:39,  8.90s/it]
 99%|█████████▊| 222104/225000 [435:16:12<7:09:29,  8.90s/it]
 99%|█████████▊| 222105/225000 [435:16:22<7:30:10,  9.33s/it]
 99%|█████████▊| 222106/225000 [435:16:29<6:57:09,  8.65s/it]
 99%|█████████▊| 222107/225000 [435:16:36<6:38:06,  8.26s/it]
 99%|█████████▊| 222108/225000 [435:16:45<6:49:44,  8.50s/it]
 99%|█████████▊| 222109/225000 [435:16:59<7:57:46,  9.92s/it]
 99%|█████████▊| 222110/225000 [435:17:05<7:02:34,  8.77s/it]
 99%|█████████▊| 222111/225000 [435:17:12<6:38:35,  8.28s/it]
 99%|█████████▊| 222112/225000 [435:17:23<7:20:46,  9.16s/it]
 99%|█████████▊| 222113/225000 [435:17:35<8:03:59, 10.06s/it]
 99%|█████████▊| 222114/225000 [435:17:40<6:41:52,  8.36s/it]
 99%|█████████▊| 222115/225000 [435:17:51<7:20:41,  9.17s/it]
 99%|█████████▊| 222116/225000 [435:17:58<7:00:18,  8.74s/it]
 99%|█████████▊| 222117/225000 [435:18:07<7:04:21,  8.83s/it]
 99%|█████████▊| 222118/225000 [435:18:19<7:48:02,  9.74s/it]
 99%|█████████▊| 222119/225000 [435:18:32<8:28:34, 10.59s/it]
 99%|█████████▊| 222120/225000 [435:18:42<8:16:50, 10.35s/it]
 99%|█████████▊| 222121/225000 [435:18:47<7:04:58,  8.86s/it]
 99%|█████████▊| 222122/225000 [435:18:58<7:30:43,  9.40s/it]
 99%|█████████▊| 222123/225000 [435:19:07<7:31:44,  9.42s/it]
 99%|█████████▊| 222124/225000 [435:19:14<6:53:59,  8.64s/it]
 99%|█████████▊| 222125/225000 [435:19:22<6:49:29,  8.55s/it]
 99%|█████████▊| 222126/225000 [435:19:30<6:36:55,  8.29s/it]
 99%|█████████▊| 222127/225000 [435:19:36<6:05:52,  7.64s/it]
 99%|█████████▊| 222128/225000 [435:19:45<6:22:20,  7.99s/it]
 99%|█████████▊| 222129/225000 [435:19:52<6:08:09,  7.69s/it]
 99%|█████████▊| 222130/225000 [435:20:06<7:36:33,  9.54s/it]
 99%|█████████▊| 222131/225000 [435:20:11<6:28:49,  8.13s/it]
 99%|█████████▊| 222132/225000 [435:20:17<5:58:32,  7.50s/it]
 99%|█████████▊| 222133/225000 [435:20:30<7:22:21,  9.26s/it]
 99%|█████████▊| 222134/225000 [435:20:42<8:02:39, 10.10s/it]
 99%|█████████▊| 222135/225000 [435:20:50<7:32:47,  9.48s/it]
 99%|█████████▊| 222136/225000 [435:21:00<7:38:20,  9.60s/it]
 99%|█████████▊| 222137/225000 [435:21:07<6:58:05,  8.76s/it]
 99%|█████████▊| 222138/225000 [435:21:15<6:51:31,  8.63s/it]
 99%|█████████▊| 222139/225000 [435:21:23<6:35:28,  8.29s/it]
 99%|█████████▊| 222140/225000 [435:21:30<6:26:33,  8.11s/it]
 99%|█████████▊| 222141/225000 [435:21:36<5:49:29,  7.33s/it]
 99%|█████████▊| 222142/225000 [435:21:49<7:12:26,  9.08s/it]
 99%|█████████▊| 222143/225000 [435:21:59<7:31:58,  9.49s/it]
 99%|█████████▊| 222144/225000 [435:22:05<6:34:10,  8.28s/it]
 99%|█████████▊| 222145/225000 [435:22:16<7:17:35,  9.20s/it]
 99%|█████████▊| 222146/225000 [435:22:26<7:21:48,  9.29s/it]
 99%|█████████▊| 222147/225000 [435:22:34<7:09:29,  9.03s/it]
 99%|█████████▊| 222148/225000 [435:22:43<7:00:49,  8.85s/it]
 99%|█████████▊| 222149/225000 [435:22:55<7:57:16, 10.04s/it]
 99%|█████████▊| 222150/225000 [435:23:03<7:22:10,  9.31s/it]
                                                             
{'loss': '1.302', 'grad_norm': '1.48', 'learning_rate': '2.329e-08', 'epoch': '0.9873', 'num_input_tokens_seen': 742764376, 'train_runtime': '1.567e+06', 'train_tokens_per_second': '473.9'}

 99%|█████████▊| 222150/225000 [435:23:03<7:22:10,  9.31s/it]
 99%|█████████▊| 222151/225000 [435:23:11<7:05:11,  8.95s/it]
 99%|█████████▊| 222152/225000 [435:23:27<8:39:28, 10.94s/it]
 99%|█████████▊| 222153/225000 [435:23:32<7:16:29,  9.20s/it]
 99%|█████████▊| 222154/225000 [435:23:41<7:17:10,  9.22s/it]
 99%|█████████▊| 222155/225000 [435:23:51<7:30:09,  9.49s/it]
 99%|█████████▊| 222156/225000 [435:24:02<7:54:23, 10.01s/it]
 99%|█████████▊| 222157/225000 [435:24:12<7:49:55,  9.92s/it]
 99%|█████████▊| 222158/225000 [435:24:20<7:26:57,  9.44s/it]
 99%|█████████▊| 222159/225000 [435:24:29<7:07:16,  9.02s/it]
 99%|█████████▊| 222160/225000 [435:24:35<6:25:56,  8.15s/it]
 99%|█████████▊| 222161/225000 [435:24:42<6:16:08,  7.95s/it]
 99%|█████████▊| 222162/225000 [435:24:56<7:41:49,  9.76s/it]
 99%|█████████▊| 222163/225000 [435:25:03<6:55:55,  8.80s/it]
 99%|█████████▊| 222164/225000 [435:25:16<7:58:28, 10.12s/it]
 99%|█████████▊| 222165/225000 [435:25:25<7:50:02,  9.95s/it]
 99%|█████████▊| 222166/225000 [435:25:34<7:24:03,  9.40s/it]
 99%|█████████▊| 222167/225000 [435:25:39<6:29:26,  8.25s/it]
 99%|█████████▊| 222168/225000 [435:25:49<6:59:24,  8.89s/it]
 99%|█████████▊| 222169/225000 [435:25:58<6:58:25,  8.87s/it]
 99%|█████████▊| 222170/225000 [435:26:08<7:04:51,  9.01s/it]
 99%|█████████▊| 222171/225000 [435:26:13<6:16:28,  7.98s/it]
 99%|█████████▊| 222172/225000 [435:26:21<6:06:22,  7.77s/it]
 99%|█████████▊| 222173/225000 [435:26:29<6:22:01,  8.11s/it]
 99%|█████████▊| 222174/225000 [435:26:39<6:42:38,  8.55s/it]
 99%|█████████▊| 222175/225000 [435:26:49<6:59:25,  8.91s/it]
 99%|█████████▊| 222176/225000 [435:26:59<7:16:25,  9.27s/it]
 99%|█████████▊| 222177/225000 [435:27:04<6:19:21,  8.06s/it]
 99%|█████████▊| 222178/225000 [435:27:12<6:15:06,  7.98s/it]
 99%|█████████▊| 222179/225000 [435:27:19<6:03:52,  7.74s/it]
 99%|█████████▊| 222180/225000 [435:27:39<8:49:45, 11.27s/it]
 99%|█████████▊| 222181/225000 [435:27:48<8:25:46, 10.76s/it]
 99%|█████████▊| 222182/225000 [435:27:59<8:23:29, 10.72s/it]
 99%|█████████▊| 222183/225000 [435:28:09<8:15:27, 10.55s/it]
 99%|█████████▊| 222184/225000 [435:28:15<7:07:58,  9.12s/it]
 99%|█████████▊| 222185/225000 [435:28:25<7:31:05,  9.61s/it]
 99%|█████████▊| 222186/225000 [435:28:30<6:19:06,  8.08s/it]
 99%|█████████▊| 222187/225000 [435:28:40<6:51:00,  8.77s/it]
 99%|█████████▉| 222188/225000 [435:28:47<6:18:01,  8.07s/it]
 99%|█████████▉| 222189/225000 [435:28:58<7:03:03,  9.03s/it]
 99%|█████████▉| 222190/225000 [435:29:05<6:30:16,  8.33s/it]
 99%|█████████▉| 222191/225000 [435:29:16<7:04:38,  9.07s/it]
 99%|█████████▉| 222192/225000 [435:29:23<6:36:45,  8.48s/it]
 99%|█████████▉| 222193/225000 [435:29:31<6:34:35,  8.43s/it]
 99%|█████████▉| 222194/225000 [435:29:43<7:25:54,  9.53s/it]
 99%|█████████▉| 222195/225000 [435:29:56<8:18:41, 10.67s/it]
 99%|█████████▉| 222196/225000 [435:30:04<7:36:14,  9.76s/it]
 99%|█████████▉| 222197/225000 [435:30:14<7:35:01,  9.74s/it]
 99%|█████████▉| 222198/225000 [435:30:24<7:44:15,  9.94s/it]
 99%|█████████▉| 222199/225000 [435:30:30<6:41:48,  8.61s/it]
 99%|█████████▉| 222200/225000 [435:30:36<6:14:33,  8.03s/it]
                                                             
{'loss': '1.316', 'grad_norm': '1.734', 'learning_rate': '2.25e-08', 'epoch': '0.9876', 'num_input_tokens_seen': 743005808, 'train_runtime': '1.568e+06', 'train_tokens_per_second': '473.9'}

 99%|█████████▉| 222200/225000 [435:30:36<6:14:33,  8.03s/it]
 99%|█████████▉| 222201/225000 [435:30:43<5:57:51,  7.67s/it]
 99%|█████████▉| 222202/225000 [435:30:57<7:30:07,  9.65s/it]
 99%|█████████▉| 222203/225000 [435:31:07<7:34:26,  9.75s/it]
 99%|█████████▉| 222204/225000 [435:31:12<6:28:20,  8.33s/it]
 99%|█████████▉| 222205/225000 [435:31:26<7:39:58,  9.87s/it]
 99%|█████████▉| 222206/225000 [435:31:32<6:42:24,  8.64s/it]
 99%|█████████▉| 222207/225000 [435:31:38<6:04:00,  7.82s/it]
 99%|█████████▉| 222208/225000 [435:31:47<6:19:43,  8.16s/it]
 99%|█████████▉| 222209/225000 [435:32:00<7:35:27,  9.79s/it]
 99%|█████████▉| 222210/225000 [435:32:06<6:37:34,  8.55s/it]
 99%|█████████▉| 222211/225000 [435:32:16<7:05:07,  9.15s/it]
 99%|█████████▉| 222212/225000 [435:32:24<6:43:46,  8.69s/it]
 99%|█████████▉| 222213/225000 [435:32:37<7:41:16,  9.93s/it]
 99%|█████████▉| 222214/225000 [435:32:51<8:43:45, 11.28s/it]
 99%|█████████▉| 222215/225000 [435:32:58<7:36:10,  9.83s/it]
 99%|█████████▉| 222216/225000 [435:33:10<8:11:59, 10.60s/it]
 99%|█████████▉| 222217/225000 [435:33:19<7:54:52, 10.24s/it]
 99%|█████████▉| 222218/225000 [435:33:28<7:28:25,  9.67s/it]
 99%|█████████▉| 222219/225000 [435:33:38<7:40:11,  9.93s/it]
 99%|█████████▉| 222220/225000 [435:33:45<6:59:38,  9.06s/it]
 99%|█████████▉| 222221/225000 [435:33:56<7:17:45,  9.45s/it]
 99%|█████████▉| 222222/225000 [435:34:05<7:15:47,  9.41s/it]
 99%|█████████▉| 222223/225000 [435:34:15<7:20:52,  9.53s/it]
 99%|█████████▉| 222224/225000 [435:34:20<6:26:33,  8.35s/it]
 99%|█████████▉| 222225/225000 [435:34:28<6:19:32,  8.21s/it]
 99%|█████████▉| 222226/225000 [435:34:36<6:07:56,  7.96s/it]
 99%|█████████▉| 222227/225000 [435:34:44<6:08:46,  7.98s/it]
 99%|█████████▉| 222228/225000 [435:34:52<6:08:36,  7.98s/it]
 99%|█████████▉| 222229/225000 [435:34:59<6:04:28,  7.89s/it]
 99%|█████████▉| 222230/225000 [435:35:14<7:31:18,  9.78s/it]
 99%|█████████▉| 222231/225000 [435:35:21<7:04:07,  9.19s/it]
 99%|█████████▉| 222232/225000 [435:35:35<8:09:00, 10.60s/it]
 99%|█████████▉| 222233/225000 [435:35:41<6:57:00,  9.04s/it]
 99%|█████████▉| 222234/225000 [435:35:51<7:18:59,  9.52s/it]
 99%|█████████▉| 222235/225000 [435:35:59<6:49:54,  8.90s/it]
 99%|█████████▉| 222236/225000 [435:36:07<6:41:27,  8.71s/it]
 99%|█████████▉| 222237/225000 [435:36:16<6:42:07,  8.73s/it]
 99%|█████████▉| 222238/225000 [435:36:23<6:14:06,  8.13s/it]
 99%|█████████▉| 222239/225000 [435:36:34<7:04:38,  9.23s/it]
 99%|█████████▉| 222240/225000 [435:36:42<6:40:36,  8.71s/it]
 99%|█████████▉| 222241/225000 [435:36:48<6:10:59,  8.07s/it]
 99%|█████████▉| 222242/225000 [435:37:01<7:20:33,  9.58s/it]
 99%|█████████▉| 222243/225000 [435:37:08<6:44:23,  8.80s/it]
 99%|█████████▉| 222244/225000 [435:37:17<6:34:04,  8.58s/it]
 99%|█████████▉| 222245/225000 [435:37:26<6:43:24,  8.79s/it]
 99%|█████████▉| 222246/225000 [435:37:33<6:14:41,  8.16s/it]
 99%|█████████▉| 222247/225000 [435:37:44<7:03:33,  9.23s/it]
 99%|█████████▉| 222248/225000 [435:37:53<6:58:17,  9.12s/it]
 99%|█████████▉| 222249/225000 [435:38:00<6:28:09,  8.47s/it]
 99%|█████████▉| 222250/225000 [435:38:09<6:31:02,  8.53s/it]
                                                             
{'loss': '1.315', 'grad_norm': '1.401', 'learning_rate': '2.174e-08', 'epoch': '0.9878', 'num_input_tokens_seen': 743248152, 'train_runtime': '1.568e+06', 'train_tokens_per_second': '473.9'}

 99%|█████████▉| 222250/225000 [435:38:09<6:31:02,  8.53s/it]
 99%|█████████▉| 222251/225000 [435:38:16<6:13:59,  8.16s/it]
 99%|█████████▉| 222252/225000 [435:38:23<6:04:03,  7.95s/it]
 99%|█████████▉| 222253/225000 [435:38:31<6:03:18,  7.94s/it]
 99%|█████████▉| 222254/225000 [435:38:46<7:29:45,  9.83s/it]
 99%|█████████▉| 222255/225000 [435:38:53<6:52:19,  9.01s/it]
 99%|█████████▉| 222256/225000 [435:38:59<6:14:20,  8.19s/it]
 99%|█████████▉| 222257/225000 [435:39:07<6:17:55,  8.27s/it]
 99%|█████████▉| 222258/225000 [435:39:14<6:00:59,  7.90s/it]
 99%|█████████▉| 222259/225000 [435:39:24<6:17:25,  8.26s/it]
 99%|█████████▉| 222260/225000 [435:39:41<8:16:06, 10.86s/it]
 99%|█████████▉| 222261/225000 [435:39:49<7:42:11, 10.12s/it]
 99%|█████████▉| 222262/225000 [435:39:55<6:41:47,  8.80s/it]
 99%|█████████▉| 222263/225000 [435:40:05<7:04:45,  9.31s/it]
 99%|█████████▉| 222264/225000 [435:40:16<7:29:15,  9.85s/it]
 99%|█████████▉| 222265/225000 [435:40:25<7:09:05,  9.41s/it]
 99%|█████████▉| 222266/225000 [435:40:32<6:39:15,  8.76s/it]
 99%|█████████▉| 222267/225000 [435:40:38<5:59:35,  7.89s/it]
 99%|█████████▉| 222268/225000 [435:40:49<6:51:47,  9.04s/it]
 99%|█████████▉| 222269/225000 [435:40:58<6:38:26,  8.75s/it]
 99%|█████████▉| 222270/225000 [435:41:05<6:24:05,  8.44s/it]
 99%|█████████▉| 222271/225000 [435:41:13<6:19:56,  8.35s/it]
 99%|█████████▉| 222272/225000 [435:41:25<7:03:13,  9.31s/it]
 99%|█████████▉| 222273/225000 [435:41:31<6:21:31,  8.39s/it]
 99%|█████████▉| 222274/225000 [435:41:38<6:03:54,  8.01s/it]
 99%|█████████▉| 222275/225000 [435:41:46<6:03:03,  7.99s/it]
 99%|█████████▉| 222276/225000 [435:41:59<7:04:13,  9.34s/it]
 99%|█████████▉| 222277/225000 [435:42:06<6:35:03,  8.70s/it]
 99%|█████████▉| 222278/225000 [435:42:13<6:18:32,  8.34s/it]
 99%|█████████▉| 222279/225000 [435:42:20<5:57:37,  7.89s/it]
 99%|█████████▉| 222280/225000 [435:42:29<6:04:47,  8.05s/it]
 99%|█████████▉| 222281/225000 [435:42:37<6:13:47,  8.25s/it]
 99%|█████████▉| 222282/225000 [435:42:53<7:55:46, 10.50s/it]
 99%|█████████▉| 222283/225000 [435:43:01<7:19:10,  9.70s/it]
 99%|█████████▉| 222284/225000 [435:43:10<7:06:36,  9.42s/it]
 99%|█████████▉| 222285/225000 [435:43:15<6:11:18,  8.21s/it]
 99%|█████████▉| 222286/225000 [435:43:23<6:00:16,  7.96s/it]
 99%|█████████▉| 222287/225000 [435:43:36<7:13:40,  9.59s/it]
 99%|█████████▉| 222288/225000 [435:43:43<6:44:09,  8.94s/it]
 99%|█████████▉| 222289/225000 [435:43:53<6:52:40,  9.13s/it]
 99%|█████████▉| 222290/225000 [435:44:01<6:43:26,  8.93s/it]
 99%|█████████▉| 222291/225000 [435:44:08<6:05:33,  8.10s/it]
 99%|█████████▉| 222292/225000 [435:44:14<5:43:04,  7.60s/it]
 99%|█████████▉| 222293/225000 [435:44:26<6:47:19,  9.03s/it]
 99%|█████████▉| 222294/225000 [435:44:35<6:35:38,  8.77s/it]
 99%|█████████▉| 222295/225000 [435:44:45<6:58:33,  9.28s/it]
 99%|█████████▉| 222296/225000 [435:44:51<6:16:15,  8.35s/it]
 99%|█████████▉| 222297/225000 [435:45:10<8:41:19, 11.57s/it]
 99%|█████████▉| 222298/225000 [435:45:18<7:44:43, 10.32s/it]
 99%|█████████▉| 222299/225000 [435:45:24<6:48:37,  9.08s/it]
 99%|█████████▉| 222300/225000 [435:45:40<8:17:34, 11.06s/it]
                                                             
{'loss': '1.282', 'grad_norm': '1.508', 'learning_rate': '2.1e-08', 'epoch': '0.988', 'num_input_tokens_seen': 743485520, 'train_runtime': '1.569e+06', 'train_tokens_per_second': '473.9'}

 99%|█████████▉| 222300/225000 [435:45:40<8:17:34, 11.06s/it]
 99%|█████████▉| 222301/225000 [435:45:49<8:02:19, 10.72s/it]
 99%|█████████▉| 222302/225000 [435:45:59<7:40:40, 10.24s/it]
 99%|█████████▉| 222303/225000 [435:46:06<7:05:56,  9.48s/it]
 99%|█████████▉| 222304/225000 [435:46:14<6:37:16,  8.84s/it]
 99%|█████████▉| 222305/225000 [435:46:26<7:25:10,  9.91s/it]
 99%|█████████▉| 222306/225000 [435:46:35<7:09:18,  9.56s/it]
 99%|█████████▉| 222307/225000 [435:46:44<7:09:40,  9.57s/it]
 99%|█████████▉| 222308/225000 [435:46:54<7:14:01,  9.67s/it]
 99%|█████████▉| 222309/225000 [435:47:01<6:36:54,  8.85s/it]
 99%|█████████▉| 222310/225000 [435:47:05<5:33:18,  7.43s/it]
 99%|█████████▉| 222311/225000 [435:47:11<5:02:11,  6.74s/it]
 99%|█████████▉| 222312/225000 [435:47:20<5:38:19,  7.55s/it]
 99%|█████████▉| 222313/225000 [435:47:31<6:30:35,  8.72s/it]
 99%|█████████▉| 222314/225000 [435:47:40<6:27:28,  8.66s/it]
 99%|█████████▉| 222315/225000 [435:47:47<6:00:10,  8.05s/it]
 99%|█████████▉| 222316/225000 [435:47:57<6:33:21,  8.79s/it]
 99%|█████████▉| 222317/225000 [435:48:11<7:35:47, 10.19s/it]
 99%|█████████▉| 222318/225000 [435:48:20<7:19:29,  9.83s/it]
 99%|█████████▉| 222319/225000 [435:48:26<6:36:55,  8.88s/it]
 99%|█████████▉| 222320/225000 [435:48:33<6:05:38,  8.19s/it]
 99%|█████████▉| 222321/225000 [435:48:49<7:52:29, 10.58s/it]
 99%|█████████▉| 222322/225000 [435:48:57<7:16:35,  9.78s/it]
 99%|█████████▉| 222323/225000 [435:49:04<6:34:56,  8.85s/it]
 99%|█████████▉| 222324/225000 [435:49:13<6:37:06,  8.90s/it]
 99%|█████████▉| 222325/225000 [435:49:25<7:29:53, 10.09s/it]
 99%|█████████▉| 222326/225000 [435:49:32<6:39:30,  8.96s/it]
 99%|█████████▉| 222327/225000 [435:49:36<5:33:41,  7.49s/it]
 99%|█████████▉| 222328/225000 [435:49:50<7:03:07,  9.50s/it]
 99%|█████████▉| 222329/225000 [435:49:55<6:08:23,  8.28s/it]
 99%|█████████▉| 222330/225000 [435:50:08<7:09:43,  9.66s/it]
 99%|█████████▉| 222331/225000 [435:50:18<7:06:03,  9.58s/it]
 99%|█████████▉| 222332/225000 [435:50:25<6:39:00,  8.97s/it]
 99%|█████████▉| 222333/225000 [435:50:33<6:28:30,  8.74s/it]
 99%|█████████▉| 222334/225000 [435:50:45<7:07:17,  9.62s/it]
 99%|█████████▉| 222335/225000 [435:50:53<6:44:40,  9.11s/it]
 99%|█████████▉| 222336/225000 [435:51:02<6:38:52,  8.98s/it]
 99%|█████████▉| 222337/225000 [435:51:11<6:45:08,  9.13s/it]
 99%|█████████▉| 222338/225000 [435:51:20<6:46:34,  9.16s/it]
 99%|█████████▉| 222339/225000 [435:51:29<6:42:26,  9.07s/it]
 99%|█████████▉| 222340/225000 [435:51:40<7:02:52,  9.54s/it]
 99%|█████████▉| 222341/225000 [435:51:47<6:29:06,  8.78s/it]
 99%|█████████▉| 222342/225000 [435:51:57<6:51:24,  9.29s/it]
 99%|█████████▉| 222343/225000 [435:52:07<7:01:21,  9.51s/it]
 99%|█████████▉| 222344/225000 [435:52:16<6:51:15,  9.29s/it]
 99%|█████████▉| 222345/225000 [435:52:27<7:09:05,  9.70s/it]
 99%|█████████▉| 222346/225000 [435:52:36<6:55:47,  9.40s/it]
 99%|█████████▉| 222347/225000 [435:52:46<7:09:25,  9.71s/it]
 99%|█████████▉| 222348/225000 [435:52:53<6:32:37,  8.88s/it]
 99%|█████████▉| 222349/225000 [435:53:01<6:27:59,  8.78s/it]
 99%|█████████▉| 222350/225000 [435:53:13<7:05:39,  9.64s/it]
                                                             
{'loss': '1.286', 'grad_norm': '1.569', 'learning_rate': '2.025e-08', 'epoch': '0.9882', 'num_input_tokens_seen': 743729840, 'train_runtime': '1.569e+06', 'train_tokens_per_second': '474'}

 99%|█████████▉| 222350/225000 [435:53:13<7:05:39,  9.64s/it]
 99%|█████████▉| 222351/225000 [435:53:20<6:33:50,  8.92s/it]
 99%|█████████▉| 222352/225000 [435:53:26<5:51:32,  7.97s/it]
 99%|█████████▉| 222353/225000 [435:53:37<6:34:03,  8.93s/it]
 99%|█████████▉| 222354/225000 [435:53:46<6:36:45,  9.00s/it]
 99%|█████████▉| 222355/225000 [435:53:54<6:16:24,  8.54s/it]
 99%|█████████▉| 222356/225000 [435:54:03<6:22:49,  8.69s/it]
 99%|█████████▉| 222357/225000 [435:54:10<5:54:26,  8.05s/it]
 99%|█████████▉| 222358/225000 [435:54:18<5:59:30,  8.16s/it]
 99%|█████████▉| 222359/225000 [435:54:24<5:31:42,  7.54s/it]
 99%|█████████▉| 222360/225000 [435:54:37<6:37:42,  9.04s/it]
 99%|█████████▉| 222361/225000 [435:54:47<6:52:20,  9.38s/it]
 99%|█████████▉| 222362/225000 [435:54:55<6:36:37,  9.02s/it]
 99%|█████████▉| 222363/225000 [435:55:06<7:08:01,  9.74s/it]
 99%|█████████▉| 222364/225000 [435:55:16<7:04:39,  9.67s/it]
 99%|█████████▉| 222365/225000 [435:55:26<7:07:15,  9.73s/it]
 99%|█████████▉| 222366/225000 [435:55:38<7:36:02, 10.39s/it]
 99%|█████████▉| 222367/225000 [435:55:47<7:17:10,  9.96s/it]
 99%|█████████▉| 222368/225000 [435:55:56<7:06:17,  9.72s/it]
 99%|█████████▉| 222369/225000 [435:56:02<6:26:48,  8.82s/it]
 99%|█████████▉| 222370/225000 [435:56:11<6:24:34,  8.77s/it]
 99%|█████████▉| 222371/225000 [435:56:24<7:23:25, 10.12s/it]
 99%|█████████▉| 222372/225000 [435:56:30<6:22:10,  8.73s/it]
 99%|█████████▉| 222373/225000 [435:56:38<6:17:38,  8.63s/it]
 99%|█████████▉| 222374/225000 [435:56:50<6:51:59,  9.41s/it]
 99%|█████████▉| 222375/225000 [435:56:57<6:22:00,  8.73s/it]
 99%|█████████▉| 222376/225000 [435:57:06<6:28:07,  8.87s/it]
 99%|█████████▉| 222377/225000 [435:57:17<6:54:25,  9.48s/it]
 99%|█████████▉| 222378/225000 [435:57:27<7:03:39,  9.69s/it]
 99%|█████████▉| 222379/225000 [435:57:37<7:06:59,  9.77s/it]
 99%|█████████▉| 222380/225000 [435:57:43<6:22:01,  8.75s/it]
 99%|█████████▉| 222381/225000 [435:57:55<7:01:22,  9.65s/it]
 99%|█████████▉| 222382/225000 [435:58:00<5:58:49,  8.22s/it]
 99%|█████████▉| 222383/225000 [435:58:11<6:39:31,  9.16s/it]
 99%|█████████▉| 222384/225000 [435:58:22<6:56:34,  9.55s/it]
 99%|█████████▉| 222385/225000 [435:58:29<6:30:18,  8.96s/it]
 99%|█████████▉| 222386/225000 [435:58:38<6:25:09,  8.84s/it]
 99%|█████████▉| 222387/225000 [435:58:45<5:56:55,  8.20s/it]
 99%|█████████▉| 222388/225000 [435:58:58<7:03:35,  9.73s/it]
 99%|█████████▉| 222389/225000 [435:59:06<6:37:37,  9.14s/it]
 99%|█████████▉| 222390/225000 [435:59:16<6:51:14,  9.45s/it]
 99%|█████████▉| 222391/225000 [435:59:24<6:36:58,  9.13s/it]
 99%|█████████▉| 222392/225000 [435:59:32<6:22:13,  8.79s/it]
 99%|█████████▉| 222393/225000 [435:59:41<6:23:26,  8.83s/it]
 99%|█████████▉| 222394/225000 [435:59:47<5:45:03,  7.94s/it]
 99%|█████████▉| 222395/225000 [436:00:00<6:47:21,  9.38s/it]
 99%|█████████▉| 222396/225000 [436:00:08<6:28:31,  8.95s/it]
 99%|█████████▉| 222397/225000 [436:00:19<7:03:32,  9.76s/it]
 99%|█████████▉| 222398/225000 [436:00:26<6:21:03,  8.79s/it]
 99%|█████████▉| 222399/225000 [436:00:32<5:50:51,  8.09s/it]
 99%|█████████▉| 222400/225000 [436:00:42<6:17:28,  8.71s/it]
                                                             
{'loss': '1.322', 'grad_norm': '1.418', 'learning_rate': '1.953e-08', 'epoch': '0.9884', 'num_input_tokens_seen': 743975864, 'train_runtime': '1.57e+06', 'train_tokens_per_second': '474'}

 99%|█████████▉| 222400/225000 [436:00:42<6:17:28,  8.71s/it]
 99%|█████████▉| 222401/225000 [436:00:50<6:02:43,  8.37s/it]
 99%|█████████▉| 222402/225000 [436:00:59<6:04:49,  8.43s/it]
 99%|█████████▉| 222403/225000 [436:01:12<7:04:41,  9.81s/it]
 99%|█████████▉| 222404/225000 [436:01:20<6:46:39,  9.40s/it]
 99%|█████████▉| 222405/225000 [436:01:27<6:12:10,  8.61s/it]
 99%|█████████▉| 222406/225000 [436:01:37<6:28:10,  8.98s/it]
 99%|█████████▉| 222407/225000 [436:01:45<6:25:32,  8.92s/it]
 99%|█████████▉| 222408/225000 [436:01:54<6:23:08,  8.87s/it]
 99%|█████████▉| 222409/225000 [436:02:03<6:26:55,  8.96s/it]
 99%|█████████▉| 222410/225000 [436:02:13<6:38:20,  9.23s/it]
 99%|█████████▉| 222411/225000 [436:02:19<5:57:09,  8.28s/it]
 99%|█████████▉| 222412/225000 [436:02:32<6:49:53,  9.50s/it]
 99%|█████████▉| 222413/225000 [436:02:41<6:44:09,  9.37s/it]
 99%|█████████▉| 222414/225000 [436:02:47<6:09:40,  8.58s/it]
 99%|█████████▉| 222415/225000 [436:02:59<6:53:04,  9.59s/it]
 99%|█████████▉| 222416/225000 [436:03:08<6:36:32,  9.21s/it]
 99%|█████████▉| 222417/225000 [436:03:15<6:05:53,  8.50s/it]
 99%|█████████▉| 222418/225000 [436:03:25<6:36:23,  9.21s/it]
 99%|█████████▉| 222419/225000 [436:03:32<6:03:40,  8.45s/it]
 99%|█████████▉| 222420/225000 [436:03:38<5:35:36,  7.80s/it]
 99%|█████████▉| 222421/225000 [436:03:49<6:06:56,  8.54s/it]
 99%|█████████▉| 222422/225000 [436:04:03<7:16:01, 10.15s/it]
 99%|█████████▉| 222423/225000 [436:04:09<6:23:10,  8.92s/it]
 99%|█████████▉| 222424/225000 [436:04:18<6:22:55,  8.92s/it]
 99%|█████████▉| 222425/225000 [436:04:27<6:27:55,  9.04s/it]
 99%|█████████▉| 222426/225000 [436:04:34<6:05:02,  8.51s/it]
 99%|█████████▉| 222427/225000 [436:04:45<6:33:43,  9.18s/it]
 99%|█████████▉| 222428/225000 [436:04:54<6:27:01,  9.03s/it]
 99%|█████████▉| 222429/225000 [436:05:01<6:02:56,  8.47s/it]
 99%|█████████▉| 222430/225000 [436:05:15<7:11:43, 10.08s/it]
 99%|█████████▉| 222431/225000 [436:05:23<6:48:52,  9.55s/it]
 99%|█████████▉| 222432/225000 [436:05:29<5:59:06,  8.39s/it]
 99%|█████████▉| 222433/225000 [436:05:39<6:19:27,  8.87s/it]
 99%|█████████▉| 222434/225000 [436:05:46<6:00:58,  8.44s/it]
 99%|█████████▉| 222435/225000 [436:05:58<6:52:35,  9.65s/it]
 99%|█████████▉| 222436/225000 [436:06:09<7:02:20,  9.88s/it]
 99%|█████████▉| 222437/225000 [436:06:16<6:20:40,  8.91s/it]
 99%|█████████▉| 222438/225000 [436:06:26<6:38:29,  9.33s/it]
 99%|█████████▉| 222439/225000 [436:06:34<6:18:44,  8.87s/it]
 99%|█████████▉| 222440/225000 [436:06:41<5:59:26,  8.42s/it]
 99%|█████████▉| 222441/225000 [436:06:54<6:53:11,  9.69s/it]
 99%|█████████▉| 222442/225000 [436:07:00<6:06:04,  8.59s/it]
 99%|█████████▉| 222443/225000 [436:07:07<5:54:56,  8.33s/it]
 99%|█████████▉| 222444/225000 [436:07:18<6:20:56,  8.94s/it]
 99%|█████████▉| 222445/225000 [436:07:23<5:28:00,  7.70s/it]
 99%|█████████▉| 222446/225000 [436:07:35<6:25:31,  9.06s/it]
 99%|█████████▉| 222447/225000 [436:07:43<6:17:55,  8.88s/it]
 99%|█████████▉| 222448/225000 [436:07:53<6:24:48,  9.05s/it]
 99%|█████████▉| 222449/225000 [436:07:59<5:52:19,  8.29s/it]
 99%|█████████▉| 222450/225000 [436:08:10<6:19:45,  8.94s/it]
                                                             
{'loss': '1.318', 'grad_norm': '1.337', 'learning_rate': '1.881e-08', 'epoch': '0.9887', 'num_input_tokens_seen': 744216280, 'train_runtime': '1.57e+06', 'train_tokens_per_second': '474'}

 99%|█████████▉| 222450/225000 [436:08:10<6:19:45,  8.94s/it]
 99%|█████████▉| 222451/225000 [436:08:16<5:51:49,  8.28s/it]
 99%|█████████▉| 222452/225000 [436:08:25<5:50:02,  8.24s/it]
 99%|█████████▉| 222453/225000 [436:08:38<7:00:32,  9.91s/it]
 99%|█████████▉| 222454/225000 [436:08:49<7:14:04, 10.23s/it]
 99%|█████████▉| 222455/225000 [436:09:01<7:27:39, 10.55s/it]
 99%|█████████▉| 222456/225000 [436:09:05<6:14:49,  8.84s/it]
 99%|█████████▉| 222457/225000 [436:09:13<5:57:01,  8.42s/it]
 99%|█████████▉| 222458/225000 [436:09:22<6:03:28,  8.58s/it]
 99%|█████████▉| 222459/225000 [436:09:34<6:44:05,  9.54s/it]
 99%|█████████▉| 222460/225000 [436:09:44<6:48:54,  9.66s/it]
 99%|█████████▉| 222461/225000 [436:09:52<6:34:06,  9.31s/it]
 99%|█████████▉| 222462/225000 [436:09:57<5:35:26,  7.93s/it]
 99%|█████████▉| 222463/225000 [436:10:04<5:20:41,  7.58s/it]
 99%|█████████▉| 222464/225000 [436:10:11<5:16:48,  7.50s/it]
 99%|█████████▉| 222465/225000 [436:10:21<5:52:11,  8.34s/it]
 99%|█████████▉| 222466/225000 [436:10:30<5:58:23,  8.49s/it]
 99%|█████████▉| 222467/225000 [436:10:38<5:57:21,  8.46s/it]
 99%|█████████▉| 222468/225000 [436:10:51<6:46:56,  9.64s/it]
 99%|█████████▉| 222469/225000 [436:10:58<6:09:56,  8.77s/it]
 99%|█████████▉| 222470/225000 [436:11:03<5:26:47,  7.75s/it]
 99%|█████████▉| 222471/225000 [436:11:11<5:35:41,  7.96s/it]
 99%|█████████▉| 222472/225000 [436:11:20<5:49:51,  8.30s/it]
 99%|█████████▉| 222473/225000 [436:11:26<5:20:16,  7.60s/it]
 99%|█████████▉| 222474/225000 [436:11:38<6:14:31,  8.90s/it]
 99%|█████████▉| 222475/225000 [436:11:46<6:04:04,  8.65s/it]
 99%|█████████▉| 222476/225000 [436:11:58<6:39:02,  9.49s/it]
 99%|█████████▉| 222477/225000 [436:12:04<5:59:39,  8.55s/it]
 99%|█████████▉| 222478/225000 [436:12:15<6:24:39,  9.15s/it]
 99%|█████████▉| 222479/225000 [436:12:25<6:41:23,  9.55s/it]
 99%|█████████▉| 222480/225000 [436:12:33<6:23:31,  9.13s/it]
 99%|█████████▉| 222481/225000 [436:12:41<6:00:57,  8.60s/it]
 99%|█████████▉| 222482/225000 [436:12:54<6:53:49,  9.86s/it]
 99%|█████████▉| 222483/225000 [436:12:59<5:55:46,  8.48s/it]
 99%|█████████▉| 222484/225000 [436:13:09<6:11:06,  8.85s/it]
 99%|█████████▉| 222485/225000 [436:13:16<5:53:40,  8.44s/it]
 99%|█████████▉| 222486/225000 [436:13:21<5:12:18,  7.45s/it]
 99%|█████████▉| 222487/225000 [436:13:28<4:59:32,  7.15s/it]
 99%|█████████▉| 222488/225000 [436:13:36<5:19:47,  7.64s/it]
 99%|█████████▉| 222489/225000 [436:13:49<6:25:50,  9.22s/it]
 99%|█████████▉| 222490/225000 [436:13:55<5:44:02,  8.22s/it]
 99%|█████████▉| 222491/225000 [436:14:04<5:47:34,  8.31s/it]
 99%|█████████▉| 222492/225000 [436:14:16<6:41:35,  9.61s/it]
 99%|█████████▉| 222493/225000 [436:14:25<6:23:39,  9.18s/it]
 99%|█████████▉| 222494/225000 [436:14:30<5:32:09,  7.95s/it]
 99%|█████████▉| 222495/225000 [436:14:41<6:15:43,  9.00s/it]
 99%|█████████▉| 222496/225000 [436:14:47<5:30:45,  7.93s/it]
 99%|█████████▉| 222497/225000 [436:14:53<5:14:50,  7.55s/it]
 99%|█████████▉| 222498/225000 [436:15:05<6:06:31,  8.79s/it]
 99%|█████████▉| 222499/225000 [436:15:18<6:58:20, 10.04s/it]
 99%|█████████▉| 222500/225000 [436:15:27<6:51:00,  9.86s/it]
                                                             
{'loss': '1.303', 'grad_norm': '1.297', 'learning_rate': '1.811e-08', 'epoch': '0.9889', 'num_input_tokens_seen': 744454440, 'train_runtime': '1.571e+06', 'train_tokens_per_second': '474'}

 99%|█████████▉| 222500/225000 [436:15:27<6:51:00,  9.86s/it]
 99%|█████████▉| 222501/225000 [436:15:39<7:12:14, 10.38s/it]
 99%|█████████▉| 222502/225000 [436:15:44<6:03:40,  8.74s/it]
 99%|█████████▉| 222503/225000 [436:15:50<5:30:06,  7.93s/it]
 99%|█████████▉| 222504/225000 [436:16:04<6:46:38,  9.78s/it]
 99%|█████████▉| 222505/225000 [436:16:11<6:13:08,  8.97s/it]
 99%|█████████▉| 222506/225000 [436:16:24<6:59:38, 10.10s/it]
 99%|█████████▉| 222507/225000 [436:16:30<6:10:46,  8.92s/it]
 99%|█████████▉| 222508/225000 [436:16:39<6:10:41,  8.93s/it]
 99%|█████████▉| 222509/225000 [436:16:44<5:18:48,  7.68s/it]
 99%|█████████▉| 222510/225000 [436:16:59<6:59:25, 10.11s/it]
 99%|█████████▉| 222511/225000 [436:17:05<6:05:09,  8.80s/it]
 99%|█████████▉| 222512/225000 [436:17:17<6:39:09,  9.63s/it]
 99%|█████████▉| 222513/225000 [436:17:23<5:55:40,  8.58s/it]
 99%|█████████▉| 222514/225000 [436:17:29<5:31:37,  8.00s/it]
 99%|█████████▉| 222515/225000 [436:17:42<6:28:43,  9.39s/it]
 99%|█████████▉| 222516/225000 [436:17:50<6:09:23,  8.92s/it]
 99%|█████████▉| 222517/225000 [436:17:57<5:45:47,  8.36s/it]
 99%|█████████▉| 222518/225000 [436:18:05<5:38:15,  8.18s/it]
 99%|█████████▉| 222519/225000 [436:18:22<7:30:17, 10.89s/it]
 99%|█████████▉| 222520/225000 [436:18:30<7:00:03, 10.16s/it]
 99%|█████████▉| 222521/225000 [436:18:40<6:53:49, 10.02s/it]
 99%|█████████▉| 222522/225000 [436:18:46<6:00:44,  8.73s/it]
 99%|█████████▉| 222523/225000 [436:18:58<6:45:03,  9.81s/it]
 99%|█████████▉| 222524/225000 [436:19:04<5:56:39,  8.64s/it]
 99%|█████████▉| 222525/225000 [436:19:15<6:21:28,  9.25s/it]
 99%|█████████▉| 222526/225000 [436:19:22<5:57:54,  8.68s/it]
 99%|█████████▉| 222527/225000 [436:19:29<5:40:40,  8.27s/it]
 99%|█████████▉| 222528/225000 [436:19:43<6:46:08,  9.86s/it]
 99%|█████████▉| 222529/225000 [436:19:49<6:00:15,  8.75s/it]
 99%|█████████▉| 222530/225000 [436:19:59<6:10:59,  9.01s/it]
 99%|█████████▉| 222531/225000 [436:20:05<5:35:53,  8.16s/it]
 99%|█████████▉| 222532/225000 [436:20:10<5:00:32,  7.31s/it]
 99%|█████████▉| 222533/225000 [436:20:21<5:42:50,  8.34s/it]
 99%|█████████▉| 222534/225000 [436:20:31<6:07:46,  8.95s/it]
 99%|█████████▉| 222535/225000 [436:20:40<5:58:08,  8.72s/it]
 99%|█████████▉| 222536/225000 [436:20:47<5:36:38,  8.20s/it]
 99%|█████████▉| 222537/225000 [436:20:55<5:40:16,  8.29s/it]
 99%|█████████▉| 222538/225000 [436:21:08<6:43:08,  9.82s/it]
 99%|█████████▉| 222539/225000 [436:21:14<5:56:45,  8.70s/it]
 99%|█████████▉| 222540/225000 [436:21:21<5:33:01,  8.12s/it]
 99%|█████████▉| 222541/225000 [436:21:33<6:11:09,  9.06s/it]
 99%|█████████▉| 222542/225000 [436:21:45<6:48:10,  9.96s/it]
 99%|█████████▉| 222543/225000 [436:21:51<6:09:51,  9.03s/it]
 99%|█████████▉| 222544/225000 [436:21:59<5:46:14,  8.46s/it]
 99%|█████████▉| 222545/225000 [436:22:08<5:57:28,  8.74s/it]
 99%|█████████▉| 222546/225000 [436:22:18<6:08:01,  9.00s/it]
 99%|█████████▉| 222547/225000 [436:22:29<6:32:47,  9.61s/it]
 99%|█████████▉| 222548/225000 [436:22:36<6:10:10,  9.06s/it]
 99%|█████████▉| 222549/225000 [436:22:42<5:33:33,  8.17s/it]
 99%|█████████▉| 222550/225000 [436:22:49<5:07:49,  7.54s/it]
                                                             
{'loss': '1.314', 'grad_norm': '1.962', 'learning_rate': '1.742e-08', 'epoch': '0.9891', 'num_input_tokens_seen': 744694880, 'train_runtime': '1.571e+06', 'train_tokens_per_second': '474'}

 99%|█████████▉| 222550/225000 [436:22:49<5:07:49,  7.54s/it]
 99%|█████████▉| 222551/225000 [436:22:56<5:06:44,  7.52s/it]
 99%|█████████▉| 222552/225000 [436:23:07<5:55:20,  8.71s/it]
 99%|█████████▉| 222553/225000 [436:23:14<5:23:22,  7.93s/it]
 99%|█████████▉| 222554/225000 [436:23:20<4:58:43,  7.33s/it]
 99%|█████████▉| 222555/225000 [436:23:33<6:11:27,  9.12s/it]
 99%|█████████▉| 222556/225000 [436:23:42<6:10:50,  9.10s/it]
 99%|█████████▉| 222557/225000 [436:23:47<5:23:05,  7.94s/it]
 99%|█████████▉| 222558/225000 [436:24:00<6:18:02,  9.29s/it]
 99%|█████████▉| 222559/225000 [436:24:08<6:08:54,  9.07s/it]
 99%|█████████▉| 222560/225000 [436:24:19<6:32:40,  9.66s/it]
 99%|█████████▉| 222561/225000 [436:24:25<5:48:20,  8.57s/it]
 99%|█████████▉| 222562/225000 [436:24:34<5:47:34,  8.55s/it]
 99%|█████████▉| 222563/225000 [436:24:43<6:02:13,  8.92s/it]
 99%|█████████▉| 222564/225000 [436:24:58<7:14:03, 10.69s/it]
 99%|█████████▉| 222565/225000 [436:25:09<7:10:59, 10.62s/it]
 99%|█████████▉| 222566/225000 [436:25:18<6:52:38, 10.17s/it]
 99%|█████████▉| 222567/225000 [436:25:27<6:38:02,  9.82s/it]
 99%|█████████▉| 222568/225000 [436:25:36<6:29:11,  9.60s/it]
 99%|█████████▉| 222569/225000 [436:25:47<6:43:16,  9.95s/it]
 99%|█████████▉| 222570/225000 [436:25:59<7:07:37, 10.56s/it]
 99%|█████████▉| 222571/225000 [436:26:03<5:55:30,  8.78s/it]
 99%|█████████▉| 222572/225000 [436:26:13<6:00:38,  8.91s/it]
 99%|█████████▉| 222573/225000 [436:26:24<6:35:04,  9.77s/it]
 99%|█████████▉| 222574/225000 [436:26:30<5:44:40,  8.52s/it]
 99%|█████████▉| 222575/225000 [436:26:40<5:57:23,  8.84s/it]
 99%|█████████▉| 222576/225000 [436:26:47<5:46:09,  8.57s/it]
 99%|█████████▉| 222577/225000 [436:26:55<5:36:37,  8.34s/it]
 99%|█████████▉| 222578/225000 [436:27:02<5:14:33,  7.79s/it]
 99%|█████████▉| 222579/225000 [436:27:09<5:09:05,  7.66s/it]
 99%|█████████▉| 222580/225000 [436:27:21<5:59:21,  8.91s/it]
 99%|█████████▉| 222581/225000 [436:27:30<6:03:52,  9.03s/it]
 99%|█████████▉| 222582/225000 [436:27:37<5:36:33,  8.35s/it]
 99%|█████████▉| 222583/225000 [436:27:50<6:38:36,  9.90s/it]
 99%|█████████▉| 222584/225000 [436:27:57<5:55:59,  8.84s/it]
 99%|█████████▉| 222585/225000 [436:28:07<6:17:10,  9.37s/it]
 99%|█████████▉| 222586/225000 [436:28:17<6:15:33,  9.33s/it]
 99%|█████████▉| 222587/225000 [436:28:22<5:29:56,  8.20s/it]
 99%|█████████▉| 222588/225000 [436:28:32<5:46:20,  8.62s/it]
 99%|█████████▉| 222589/225000 [436:28:41<5:48:18,  8.67s/it]
 99%|█████████▉| 222590/225000 [436:28:49<5:48:31,  8.68s/it]
 99%|█████████▉| 222591/225000 [436:28:58<5:52:03,  8.77s/it]
 99%|█████████▉| 222592/225000 [436:29:08<6:02:24,  9.03s/it]
 99%|█████████▉| 222593/225000 [436:29:15<5:39:37,  8.47s/it]
 99%|█████████▉| 222594/225000 [436:29:24<5:38:40,  8.45s/it]
 99%|█████████▉| 222595/225000 [436:29:29<5:01:05,  7.51s/it]
 99%|█████████▉| 222596/225000 [436:29:42<6:12:53,  9.31s/it]
 99%|█████████▉| 222597/225000 [436:29:52<6:10:47,  9.26s/it]
 99%|█████████▉| 222598/225000 [436:30:01<6:07:56,  9.19s/it]
 99%|█████████▉| 222599/225000 [436:30:06<5:21:47,  8.04s/it]
 99%|█████████▉| 222600/225000 [436:30:18<6:05:49,  9.15s/it]
                                                             
{'loss': '1.301', 'grad_norm': '1.693', 'learning_rate': '1.675e-08', 'epoch': '0.9893', 'num_input_tokens_seen': 744931840, 'train_runtime': '1.571e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222600/225000 [436:30:18<6:05:49,  9.15s/it]
 99%|█████████▉| 222601/225000 [436:30:25<5:44:28,  8.62s/it]
 99%|█████████▉| 222602/225000 [436:30:34<5:45:46,  8.65s/it]
 99%|█████████▉| 222603/225000 [436:30:42<5:35:17,  8.39s/it]
 99%|█████████▉| 222604/225000 [436:30:52<5:57:14,  8.95s/it]
 99%|█████████▉| 222605/225000 [436:31:01<6:03:15,  9.10s/it]
 99%|█████████▉| 222606/225000 [436:31:13<6:38:16,  9.98s/it]
 99%|█████████▉| 222607/225000 [436:31:19<5:50:07,  8.78s/it]
 99%|█████████▉| 222608/225000 [436:31:28<5:52:23,  8.84s/it]
 99%|█████████▉| 222609/225000 [436:31:35<5:29:52,  8.28s/it]
 99%|█████████▉| 222610/225000 [436:31:51<6:57:24, 10.48s/it]
 99%|█████████▉| 222611/225000 [436:32:00<6:47:49, 10.24s/it]
 99%|█████████▉| 222612/225000 [436:32:07<6:04:53,  9.17s/it]
 99%|█████████▉| 222613/225000 [436:32:13<5:26:26,  8.21s/it]
 99%|█████████▉| 222614/225000 [436:32:21<5:18:22,  8.01s/it]
 99%|█████████▉| 222615/225000 [436:32:33<6:07:46,  9.25s/it]
 99%|█████████▉| 222616/225000 [436:32:45<6:41:11, 10.10s/it]
 99%|█████████▉| 222617/225000 [436:32:51<5:53:07,  8.89s/it]
 99%|█████████▉| 222618/225000 [436:32:57<5:18:44,  8.03s/it]
 99%|█████████▉| 222619/225000 [436:33:07<5:46:06,  8.72s/it]
 99%|█████████▉| 222620/225000 [436:33:12<4:55:35,  7.45s/it]
 99%|█████████▉| 222621/225000 [436:33:23<5:37:13,  8.51s/it]
 99%|█████████▉| 222622/225000 [436:33:32<5:46:46,  8.75s/it]
 99%|█████████▉| 222623/225000 [436:33:41<5:52:24,  8.90s/it]
 99%|█████████▉| 222624/225000 [436:33:47<5:17:01,  8.01s/it]
 99%|█████████▉| 222625/225000 [436:33:58<5:50:58,  8.87s/it]
 99%|█████████▉| 222626/225000 [436:34:05<5:28:19,  8.30s/it]
 99%|█████████▉| 222627/225000 [436:34:18<6:19:52,  9.60s/it]
 99%|█████████▉| 222628/225000 [436:34:27<6:12:47,  9.43s/it]
 99%|█████████▉| 222629/225000 [436:34:37<6:18:10,  9.57s/it]
 99%|█████████▉| 222630/225000 [436:34:43<5:44:16,  8.72s/it]
 99%|█████████▉| 222631/225000 [436:34:49<5:07:45,  7.79s/it]
 99%|█████████▉| 222632/225000 [436:35:01<5:53:53,  8.97s/it]
 99%|█████████▉| 222633/225000 [436:35:08<5:39:08,  8.60s/it]
 99%|█████████▉| 222634/225000 [436:35:20<6:14:43,  9.50s/it]
 99%|█████████▉| 222635/225000 [436:35:31<6:30:43,  9.91s/it]
 99%|█████████▉| 222636/225000 [436:35:39<6:06:16,  9.30s/it]
 99%|█████████▉| 222637/225000 [436:35:51<6:38:15, 10.11s/it]
 99%|█████████▉| 222638/225000 [436:35:58<6:00:32,  9.16s/it]
 99%|█████████▉| 222639/225000 [436:36:05<5:33:26,  8.47s/it]
 99%|█████████▉| 222640/225000 [436:36:11<5:10:30,  7.89s/it]
 99%|█████████▉| 222641/225000 [436:36:23<5:52:58,  8.98s/it]
 99%|█████████▉| 222642/225000 [436:36:34<6:17:22,  9.60s/it]
 99%|█████████▉| 222643/225000 [436:36:44<6:22:42,  9.74s/it]
 99%|█████████▉| 222644/225000 [436:36:53<6:18:10,  9.63s/it]
 99%|█████████▉| 222645/225000 [436:37:04<6:37:14, 10.12s/it]
 99%|█████████▉| 222646/225000 [436:37:10<5:46:51,  8.84s/it]
 99%|█████████▉| 222647/225000 [436:37:23<6:30:18,  9.95s/it]
 99%|█████████▉| 222648/225000 [436:37:35<7:00:57, 10.74s/it]
 99%|█████████▉| 222649/225000 [436:37:45<6:44:01, 10.31s/it]
 99%|█████████▉| 222650/225000 [436:37:51<6:01:57,  9.24s/it]
                                                             
{'loss': '1.353', 'grad_norm': '1.656', 'learning_rate': '1.608e-08', 'epoch': '0.9896', 'num_input_tokens_seen': 745178808, 'train_runtime': '1.572e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222650/225000 [436:37:51<6:01:57,  9.24s/it]
 99%|█████████▉| 222651/225000 [436:37:59<5:38:22,  8.64s/it]
 99%|█████████▉| 222652/225000 [436:38:06<5:18:36,  8.14s/it]
 99%|█████████▉| 222653/225000 [436:38:19<6:19:39,  9.71s/it]
 99%|█████████▉| 222654/225000 [436:38:28<6:13:39,  9.56s/it]
 99%|█████████▉| 222655/225000 [436:38:40<6:40:49, 10.26s/it]
 99%|█████████▉| 222656/225000 [436:38:51<6:48:51, 10.47s/it]
 99%|█████████▉| 222657/225000 [436:39:02<6:48:02, 10.45s/it]
 99%|█████████▉| 222658/225000 [436:39:09<6:11:26,  9.52s/it]
 99%|█████████▉| 222659/225000 [436:39:15<5:32:52,  8.53s/it]
 99%|█████████▉| 222660/225000 [436:39:21<4:56:50,  7.61s/it]
 99%|█████████▉| 222661/225000 [436:39:28<4:52:05,  7.49s/it]
 99%|█████████▉| 222662/225000 [436:39:33<4:28:26,  6.89s/it]
 99%|█████████▉| 222663/225000 [436:39:47<5:44:51,  8.85s/it]
 99%|█████████▉| 222664/225000 [436:39:53<5:13:23,  8.05s/it]
 99%|█████████▉| 222665/225000 [436:40:00<5:05:56,  7.86s/it]
 99%|█████████▉| 222666/225000 [436:40:16<6:37:07, 10.21s/it]
 99%|█████████▉| 222667/225000 [436:40:22<5:43:54,  8.84s/it]
 99%|█████████▉| 222668/225000 [436:40:34<6:21:45,  9.82s/it]
 99%|█████████▉| 222669/225000 [436:40:41<5:53:53,  9.11s/it]
 99%|█████████▉| 222670/225000 [436:40:49<5:37:24,  8.69s/it]
 99%|█████████▉| 222671/225000 [436:40:58<5:42:07,  8.81s/it]
 99%|█████████▉| 222672/225000 [436:41:05<5:17:12,  8.18s/it]
 99%|█████████▉| 222673/225000 [436:41:20<6:45:52, 10.47s/it]
 99%|█████████▉| 222674/225000 [436:41:29<6:24:08,  9.91s/it]
 99%|█████████▉| 222675/225000 [436:41:37<5:55:32,  9.18s/it]
 99%|█████████▉| 222676/225000 [436:41:43<5:26:06,  8.42s/it]
 99%|█████████▉| 222677/225000 [436:41:49<4:53:35,  7.58s/it]
 99%|█████████▉| 222678/225000 [436:41:55<4:39:56,  7.23s/it]
 99%|█████████▉| 222679/225000 [436:42:03<4:42:39,  7.31s/it]
 99%|█████████▉| 222680/225000 [436:42:11<4:50:03,  7.50s/it]
 99%|█████████▉| 222681/225000 [436:42:27<6:33:31, 10.18s/it]
 99%|█████████▉| 222682/225000 [436:42:34<5:51:05,  9.09s/it]
 99%|█████████▉| 222683/225000 [436:42:43<5:58:27,  9.28s/it]
 99%|█████████▉| 222684/225000 [436:42:53<6:00:42,  9.34s/it]
 99%|█████████▉| 222685/225000 [436:43:05<6:28:54, 10.08s/it]
 99%|█████████▉| 222686/225000 [436:43:13<6:07:00,  9.52s/it]
 99%|█████████▉| 222687/225000 [436:43:28<7:11:33, 11.19s/it]
 99%|█████████▉| 222688/225000 [436:43:35<6:26:40, 10.03s/it]
 99%|█████████▉| 222689/225000 [436:43:45<6:23:30,  9.96s/it]
 99%|█████████▉| 222690/225000 [436:43:55<6:21:14,  9.90s/it]
 99%|█████████▉| 222691/225000 [436:44:03<6:01:40,  9.40s/it]
 99%|█████████▉| 222692/225000 [436:44:12<5:50:14,  9.11s/it]
 99%|█████████▉| 222693/225000 [436:44:24<6:28:19, 10.10s/it]
 99%|█████████▉| 222694/225000 [436:44:31<5:57:29,  9.30s/it]
 99%|█████████▉| 222695/225000 [436:44:42<6:10:50,  9.65s/it]
 99%|█████████▉| 222696/225000 [436:44:49<5:40:30,  8.87s/it]
 99%|█████████▉| 222697/225000 [436:44:58<5:47:57,  9.07s/it]
 99%|█████████▉| 222698/225000 [436:45:09<6:06:28,  9.55s/it]
 99%|█████████▉| 222699/225000 [436:45:17<5:43:59,  8.97s/it]
 99%|█████████▉| 222700/225000 [436:45:22<5:05:00,  7.96s/it]
                                                             
{'loss': '1.315', 'grad_norm': '1.352', 'learning_rate': '1.544e-08', 'epoch': '0.9898', 'num_input_tokens_seen': 745421952, 'train_runtime': '1.572e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222700/225000 [436:45:22<5:05:00,  7.96s/it]
 99%|█████████▉| 222701/225000 [436:45:29<4:49:55,  7.57s/it]
 99%|█████████▉| 222702/225000 [436:45:38<5:10:34,  8.11s/it]
 99%|█████████▉| 222703/225000 [436:45:50<5:47:56,  9.09s/it]
 99%|█████████▉| 222704/225000 [436:45:57<5:28:22,  8.58s/it]
 99%|█████████▉| 222705/225000 [436:46:06<5:33:59,  8.73s/it]
 99%|█████████▉| 222706/225000 [436:46:24<7:15:42, 11.40s/it]
 99%|█████████▉| 222707/225000 [436:46:35<7:08:23, 11.21s/it]
 99%|█████████▉| 222708/225000 [436:46:41<6:08:42,  9.65s/it]
 99%|█████████▉| 222709/225000 [436:46:46<5:15:24,  8.26s/it]
 99%|█████████▉| 222710/225000 [436:46:57<5:55:11,  9.31s/it]
 99%|█████████▉| 222711/225000 [436:47:10<6:27:49, 10.17s/it]
 99%|█████████▉| 222712/225000 [436:47:18<6:03:25,  9.53s/it]
 99%|█████████▉| 222713/225000 [436:47:25<5:33:50,  8.76s/it]
 99%|█████████▉| 222714/225000 [436:47:32<5:14:02,  8.24s/it]
 99%|█████████▉| 222715/225000 [436:47:44<6:02:53,  9.53s/it]
 99%|█████████▉| 222716/225000 [436:47:52<5:39:21,  8.91s/it]
 99%|█████████▉| 222717/225000 [436:47:58<5:14:13,  8.26s/it]
 99%|█████████▉| 222718/225000 [436:48:07<5:17:11,  8.34s/it]
 99%|█████████▉| 222719/225000 [436:48:17<5:41:22,  8.98s/it]
 99%|█████████▉| 222720/225000 [436:48:24<5:11:44,  8.20s/it]
 99%|█████████▉| 222721/225000 [436:48:32<5:15:20,  8.30s/it]
 99%|█████████▉| 222722/225000 [436:48:46<6:13:55,  9.85s/it]
 99%|█████████▉| 222723/225000 [436:48:51<5:27:06,  8.62s/it]
 99%|█████████▉| 222724/225000 [436:48:58<4:58:40,  7.87s/it]
 99%|█████████▉| 222725/225000 [436:49:04<4:43:21,  7.47s/it]
 99%|█████████▉| 222726/225000 [436:49:20<6:13:58,  9.87s/it]
 99%|█████████▉| 222727/225000 [436:49:31<6:26:00, 10.19s/it]
 99%|█████████▉| 222728/225000 [436:49:38<5:54:13,  9.35s/it]
 99%|█████████▉| 222729/225000 [436:49:48<6:06:27,  9.68s/it]
 99%|█████████▉| 222730/225000 [436:49:57<5:58:38,  9.48s/it]
 99%|█████████▉| 222731/225000 [436:50:04<5:25:38,  8.61s/it]
 99%|█████████▉| 222732/225000 [436:50:12<5:16:28,  8.37s/it]
 99%|█████████▉| 222733/225000 [436:50:25<6:14:35,  9.91s/it]
 99%|█████████▉| 222734/225000 [436:50:31<5:31:27,  8.78s/it]
 99%|█████████▉| 222735/225000 [436:50:43<5:58:11,  9.49s/it]
 99%|█████████▉| 222736/225000 [436:50:52<5:57:19,  9.47s/it]
 99%|█████████▉| 222737/225000 [436:50:58<5:18:34,  8.45s/it]
 99%|█████████▉| 222738/225000 [436:51:10<5:57:58,  9.50s/it]
 99%|█████████▉| 222739/225000 [436:51:20<6:02:59,  9.63s/it]
 99%|█████████▉| 222740/225000 [436:51:26<5:16:55,  8.41s/it]
 99%|█████████▉| 222741/225000 [436:51:42<6:42:53, 10.70s/it]
 99%|█████████▉| 222742/225000 [436:51:48<5:49:32,  9.29s/it]
 99%|█████████▉| 222743/225000 [436:51:57<5:46:02,  9.20s/it]
 99%|█████████▉| 222744/225000 [436:52:07<6:04:16,  9.69s/it]
 99%|█████████▉| 222745/225000 [436:52:16<5:52:35,  9.38s/it]
 99%|█████████▉| 222746/225000 [436:52:23<5:26:06,  8.68s/it]
 99%|█████████▉| 222747/225000 [436:52:34<5:49:17,  9.30s/it]
 99%|█████████▉| 222748/225000 [436:52:45<6:15:21, 10.00s/it]
 99%|█████████▉| 222749/225000 [436:52:53<5:42:25,  9.13s/it]
 99%|█████████▉| 222750/225000 [436:53:03<5:52:08,  9.39s/it]
                                                             
{'loss': '1.316', 'grad_norm': '1.112', 'learning_rate': '1.48e-08', 'epoch': '0.99', 'num_input_tokens_seen': 745671352, 'train_runtime': '1.573e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222750/225000 [436:53:03<5:52:08,  9.39s/it]
 99%|█████████▉| 222751/225000 [436:53:09<5:24:11,  8.65s/it]
 99%|█████████▉| 222752/225000 [436:53:22<6:09:13,  9.85s/it]
 99%|█████████▉| 222753/225000 [436:53:30<5:47:19,  9.27s/it]
 99%|█████████▉| 222754/225000 [436:53:38<5:27:55,  8.76s/it]
 99%|█████████▉| 222755/225000 [436:53:53<6:46:32, 10.87s/it]
 99%|█████████▉| 222756/225000 [436:54:00<5:55:33,  9.51s/it]
 99%|█████████▉| 222757/225000 [436:54:08<5:43:10,  9.18s/it]
 99%|█████████▉| 222758/225000 [436:54:13<4:52:20,  7.82s/it]
 99%|█████████▉| 222759/225000 [436:54:25<5:38:46,  9.07s/it]
 99%|█████████▉| 222760/225000 [436:54:30<4:56:26,  7.94s/it]
 99%|█████████▉| 222761/225000 [436:54:39<5:03:46,  8.14s/it]
 99%|█████████▉| 222762/225000 [436:54:49<5:23:31,  8.67s/it]
 99%|█████████▉| 222763/225000 [436:54:54<4:49:28,  7.76s/it]
 99%|█████████▉| 222764/225000 [436:55:05<5:23:58,  8.69s/it]
 99%|█████████▉| 222765/225000 [436:55:13<5:18:11,  8.54s/it]
 99%|█████████▉| 222766/225000 [436:55:18<4:34:53,  7.38s/it]
 99%|█████████▉| 222767/225000 [436:55:31<5:32:28,  8.93s/it]
 99%|█████████▉| 222768/225000 [436:55:42<5:55:20,  9.55s/it]
 99%|█████████▉| 222769/225000 [436:55:51<5:49:32,  9.40s/it]
 99%|█████████▉| 222770/225000 [436:55:58<5:26:35,  8.79s/it]
 99%|█████████▉| 222771/225000 [436:56:09<5:48:58,  9.39s/it]
 99%|█████████▉| 222772/225000 [436:56:21<6:18:32, 10.19s/it]
 99%|█████████▉| 222773/225000 [436:56:33<6:37:13, 10.70s/it]
 99%|█████████▉| 222774/225000 [436:56:39<5:52:11,  9.49s/it]
 99%|█████████▉| 222775/225000 [436:56:49<5:55:01,  9.57s/it]
 99%|█████████▉| 222776/225000 [436:57:00<6:07:35,  9.92s/it]
 99%|█████████▉| 222777/225000 [436:57:12<6:34:11, 10.64s/it]
 99%|█████████▉| 222778/225000 [436:57:25<7:03:32, 11.44s/it]
 99%|█████████▉| 222779/225000 [436:57:31<6:02:21,  9.79s/it]
 99%|█████████▉| 222780/225000 [436:57:44<6:36:40, 10.72s/it]
 99%|█████████▉| 222781/225000 [436:57:55<6:38:04, 10.76s/it]
 99%|█████████▉| 222782/225000 [436:58:00<5:34:10,  9.04s/it]
 99%|█████████▉| 222783/225000 [436:58:13<6:11:33, 10.06s/it]
 99%|█████████▉| 222784/225000 [436:58:21<5:53:04,  9.56s/it]
 99%|█████████▉| 222785/225000 [436:58:32<6:06:26,  9.93s/it]
 99%|█████████▉| 222786/225000 [436:58:39<5:37:38,  9.15s/it]
 99%|█████████▉| 222787/225000 [436:58:54<6:42:20, 10.91s/it]
 99%|█████████▉| 222788/225000 [436:59:01<5:52:45,  9.57s/it]
 99%|█████████▉| 222789/225000 [436:59:10<5:47:02,  9.42s/it]
 99%|█████████▉| 222790/225000 [436:59:16<5:14:33,  8.54s/it]
 99%|█████████▉| 222791/225000 [436:59:26<5:30:32,  8.98s/it]
 99%|█████████▉| 222792/225000 [436:59:37<5:46:28,  9.42s/it]
 99%|█████████▉| 222793/225000 [436:59:42<4:58:59,  8.13s/it]
 99%|█████████▉| 222794/225000 [436:59:51<5:16:18,  8.60s/it]
 99%|█████████▉| 222795/225000 [437:00:02<5:35:07,  9.12s/it]
 99%|█████████▉| 222796/225000 [437:00:09<5:16:51,  8.63s/it]
 99%|█████████▉| 222797/225000 [437:00:22<6:07:02, 10.00s/it]
 99%|█████████▉| 222798/225000 [437:00:30<5:39:17,  9.25s/it]
 99%|█████████▉| 222799/225000 [437:00:43<6:20:24, 10.37s/it]
 99%|█████████▉| 222800/225000 [437:00:56<6:45:50, 11.07s/it]
                                                             
{'loss': '1.315', 'grad_norm': '1.014', 'learning_rate': '1.418e-08', 'epoch': '0.9902', 'num_input_tokens_seen': 745923216, 'train_runtime': '1.573e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222800/225000 [437:00:56<6:45:50, 11.07s/it]
 99%|█████████▉| 222801/225000 [437:01:03<6:00:16,  9.83s/it]
 99%|█████████▉| 222802/225000 [437:01:09<5:25:24,  8.88s/it]
 99%|█████████▉| 222803/225000 [437:01:24<6:32:47, 10.73s/it]
 99%|█████████▉| 222804/225000 [437:01:32<6:03:19,  9.93s/it]
 99%|█████████▉| 222805/225000 [437:01:42<5:57:59,  9.79s/it]
 99%|█████████▉| 222806/225000 [437:01:48<5:22:31,  8.82s/it]
 99%|█████████▉| 222807/225000 [437:01:55<4:58:32,  8.17s/it]
 99%|█████████▉| 222808/225000 [437:02:00<4:29:21,  7.37s/it]
 99%|█████████▉| 222809/225000 [437:02:15<5:45:51,  9.47s/it]
 99%|█████████▉| 222810/225000 [437:02:24<5:41:00,  9.34s/it]
 99%|█████████▉| 222811/225000 [437:02:35<5:59:13,  9.85s/it]
 99%|█████████▉| 222812/225000 [437:02:44<5:48:52,  9.57s/it]
 99%|█████████▉| 222813/225000 [437:02:51<5:21:23,  8.82s/it]
 99%|█████████▉| 222814/225000 [437:02:58<5:01:21,  8.27s/it]
 99%|█████████▉| 222815/225000 [437:03:09<5:26:51,  8.98s/it]
 99%|█████████▉| 222816/225000 [437:03:17<5:25:54,  8.95s/it]
 99%|█████████▉| 222817/225000 [437:03:32<6:30:51, 10.74s/it]
 99%|█████████▉| 222818/225000 [437:03:38<5:39:37,  9.34s/it]
 99%|█████████▉| 222819/225000 [437:03:47<5:26:46,  8.99s/it]
 99%|█████████▉| 222820/225000 [437:03:57<5:39:26,  9.34s/it]
 99%|█████████▉| 222821/225000 [437:04:02<4:58:22,  8.22s/it]
 99%|█████████▉| 222822/225000 [437:04:12<5:16:14,  8.71s/it]
 99%|█████████▉| 222823/225000 [437:04:19<4:53:10,  8.08s/it]
 99%|█████████▉| 222824/225000 [437:04:26<4:41:31,  7.76s/it]
 99%|█████████▉| 222825/225000 [437:04:42<6:13:56, 10.32s/it]
 99%|█████████▉| 222826/225000 [437:04:47<5:18:15,  8.78s/it]
 99%|█████████▉| 222827/225000 [437:04:56<5:18:48,  8.80s/it]
 99%|█████████▉| 222828/225000 [437:05:05<5:22:33,  8.91s/it]
 99%|█████████▉| 222829/225000 [437:05:15<5:35:27,  9.27s/it]
 99%|█████████▉| 222830/225000 [437:05:25<5:36:56,  9.32s/it]
 99%|█████████▉| 222831/225000 [437:05:33<5:22:59,  8.93s/it]
 99%|█████████▉| 222832/225000 [437:05:39<4:47:43,  7.96s/it]
 99%|█████████▉| 222833/225000 [437:05:47<4:51:31,  8.07s/it]
 99%|█████████▉| 222834/225000 [437:05:56<5:07:30,  8.52s/it]
 99%|█████████▉| 222835/225000 [437:06:08<5:38:16,  9.37s/it]
 99%|█████████▉| 222836/225000 [437:06:14<5:00:24,  8.33s/it]
 99%|█████████▉| 222837/225000 [437:06:20<4:41:46,  7.82s/it]
 99%|█████████▉| 222838/225000 [437:06:33<5:31:31,  9.20s/it]
 99%|█████████▉| 222839/225000 [437:06:44<5:57:02,  9.91s/it]
 99%|█████████▉| 222840/225000 [437:06:57<6:29:23, 10.82s/it]
 99%|█████████▉| 222841/225000 [437:07:05<5:50:33,  9.74s/it]
 99%|█████████▉| 222842/225000 [437:07:11<5:15:10,  8.76s/it]
 99%|█████████▉| 222843/225000 [437:07:20<5:14:59,  8.76s/it]
 99%|█████████▉| 222844/225000 [437:07:32<5:48:53,  9.71s/it]
 99%|█████████▉| 222845/225000 [437:07:40<5:30:12,  9.19s/it]
 99%|█████████▉| 222846/225000 [437:07:48<5:25:48,  9.08s/it]
 99%|█████████▉| 222847/225000 [437:07:54<4:48:17,  8.03s/it]
 99%|█████████▉| 222848/225000 [437:08:03<4:58:01,  8.31s/it]
 99%|█████████▉| 222849/225000 [437:08:15<5:39:54,  9.48s/it]
 99%|█████████▉| 222850/225000 [437:08:25<5:47:05,  9.69s/it]
                                                             
{'loss': '1.316', 'grad_norm': '1.409', 'learning_rate': '1.357e-08', 'epoch': '0.9904', 'num_input_tokens_seen': 746165552, 'train_runtime': '1.574e+06', 'train_tokens_per_second': '474.1'}

 99%|█████████▉| 222850/225000 [437:08:25<5:47:05,  9.69s/it]
 99%|█████████▉| 222851/225000 [437:08:36<5:52:38,  9.85s/it]
 99%|█████████▉| 222852/225000 [437:08:45<5:43:17,  9.59s/it]
 99%|█████████▉| 222853/225000 [437:08:53<5:30:37,  9.24s/it]
 99%|█████████▉| 222854/225000 [437:08:59<4:57:08,  8.31s/it]
 99%|█████████▉| 222855/225000 [437:09:13<6:00:51, 10.09s/it]
 99%|█████████▉| 222856/225000 [437:09:21<5:35:34,  9.39s/it]
 99%|█████████▉| 222857/225000 [437:09:29<5:18:43,  8.92s/it]
 99%|█████████▉| 222858/225000 [437:09:39<5:30:04,  9.25s/it]
 99%|█████████▉| 222859/225000 [437:09:49<5:33:44,  9.35s/it]
 99%|█████████▉| 222860/225000 [437:09:58<5:30:45,  9.27s/it]
 99%|█████████▉| 222861/225000 [437:10:04<5:01:12,  8.45s/it]
 99%|█████████▉| 222862/225000 [437:10:16<5:34:50,  9.40s/it]
 99%|█████████▉| 222863/225000 [437:10:24<5:18:14,  8.94s/it]
 99%|█████████▉| 222864/225000 [437:10:33<5:17:14,  8.91s/it]
 99%|█████████▉| 222865/225000 [437:10:38<4:40:30,  7.88s/it]
 99%|█████████▉| 222866/225000 [437:10:51<5:37:46,  9.50s/it]
 99%|█████████▉| 222867/225000 [437:10:59<5:17:04,  8.92s/it]
 99%|█████████▉| 222868/225000 [437:11:08<5:17:50,  8.95s/it]
 99%|█████████▉| 222869/225000 [437:11:15<4:57:32,  8.38s/it]
 99%|█████████▉| 222870/225000 [437:11:25<5:16:53,  8.93s/it]
 99%|█████████▉| 222871/225000 [437:11:33<5:00:45,  8.48s/it]
 99%|█████████▉| 222872/225000 [437:11:42<5:06:11,  8.63s/it]
 99%|█████████▉| 222873/225000 [437:11:53<5:37:18,  9.52s/it]
 99%|█████████▉| 222874/225000 [437:12:04<5:47:09,  9.80s/it]
 99%|█████████▉| 222875/225000 [437:12:11<5:18:47,  9.00s/it]
 99%|█████████▉| 222876/225000 [437:12:17<4:44:28,  8.04s/it]
 99%|█████████▉| 222877/225000 [437:12:22<4:18:19,  7.30s/it]
 99%|█████████▉| 222878/225000 [437:12:38<5:53:39, 10.00s/it]
 99%|█████████▉| 222879/225000 [437:12:45<5:21:00,  9.08s/it]
 99%|█████████▉| 222880/225000 [437:12:55<5:26:14,  9.23s/it]
 99%|█████████▉| 222881/225000 [437:13:04<5:22:33,  9.13s/it]
 99%|█████████▉| 222882/225000 [437:13:13<5:17:32,  9.00s/it]
 99%|█████████▉| 222883/225000 [437:13:23<5:34:01,  9.47s/it]
 99%|█████████▉| 222884/225000 [437:13:30<5:09:25,  8.77s/it]
 99%|█████████▉| 222885/225000 [437:13:36<4:36:25,  7.84s/it]
 99%|█████████▉| 222886/225000 [437:13:49<5:30:24,  9.38s/it]
 99%|█████████▉| 222887/225000 [437:14:01<6:02:39, 10.30s/it]
 99%|█████████▉| 222888/225000 [437:14:10<5:44:00,  9.77s/it]
 99%|█████████▉| 222889/225000 [437:14:17<5:15:53,  8.98s/it]
 99%|█████████▉| 222890/225000 [437:14:27<5:29:32,  9.37s/it]
 99%|█████████▉| 222891/225000 [437:14:38<5:47:00,  9.87s/it]
 99%|█████████▉| 222892/225000 [437:14:49<5:58:52, 10.21s/it]
 99%|█████████▉| 222893/225000 [437:14:59<5:48:46,  9.93s/it]
 99%|█████████▉| 222894/225000 [437:15:06<5:17:22,  9.04s/it]
 99%|█████████▉| 222895/225000 [437:15:17<5:45:26,  9.85s/it]
 99%|█████████▉| 222896/225000 [437:15:24<5:14:01,  8.96s/it]
 99%|█████████▉| 222897/225000 [437:15:34<5:22:27,  9.20s/it]
 99%|█████████▉| 222898/225000 [437:15:42<5:11:23,  8.89s/it]
 99%|█████████▉| 222899/225000 [437:15:51<5:07:35,  8.78s/it]
 99%|█████████▉| 222900/225000 [437:15:55<4:25:34,  7.59s/it]
                                                             
{'loss': '1.322', 'grad_norm': '1.504', 'learning_rate': '1.298e-08', 'epoch': '0.9907', 'num_input_tokens_seen': 746408328, 'train_runtime': '1.574e+06', 'train_tokens_per_second': '474.2'}

 99%|█████████▉| 222900/225000 [437:15:55<4:25:34,  7.59s/it]
 99%|█████████▉| 222901/225000 [437:16:06<4:53:19,  8.38s/it]
 99%|█████████▉| 222902/225000 [437:16:17<5:27:27,  9.36s/it]
 99%|█████████▉| 222903/225000 [437:16:23<4:46:57,  8.21s/it]
 99%|█████████▉| 222904/225000 [437:16:33<5:08:42,  8.84s/it]
 99%|█████████▉| 222905/225000 [437:16:38<4:24:29,  7.57s/it]
 99%|█████████▉| 222906/225000 [437:16:51<5:22:01,  9.23s/it]
 99%|█████████▉| 222907/225000 [437:16:59<5:09:47,  8.88s/it]
 99%|█████████▉| 222908/225000 [437:17:07<4:58:11,  8.55s/it]
 99%|█████████▉| 222909/225000 [437:17:14<4:45:13,  8.18s/it]
 99%|█████████▉| 222910/225000 [437:17:24<5:07:18,  8.82s/it]
 99%|█████████▉| 222911/225000 [437:17:37<5:42:19,  9.83s/it]
 99%|█████████▉| 222912/225000 [437:17:44<5:22:25,  9.27s/it]
 99%|█████████▉| 222913/225000 [437:17:51<4:48:26,  8.29s/it]
 99%|█████████▉| 222914/225000 [437:17:58<4:44:37,  8.19s/it]
 99%|█████████▉| 222915/225000 [437:18:16<6:18:40, 10.90s/it]
 99%|█████████▉| 222916/225000 [437:18:21<5:16:55,  9.12s/it]
 99%|█████████▉| 222917/225000 [437:18:27<4:43:35,  8.17s/it]
 99%|█████████▉| 222918/225000 [437:18:39<5:27:37,  9.44s/it]
 99%|█████████▉| 222919/225000 [437:18:47<5:16:09,  9.12s/it]
 99%|█████████▉| 222920/225000 [437:18:55<5:01:13,  8.69s/it]
 99%|█████████▉| 222921/225000 [437:19:04<5:08:40,  8.91s/it]
 99%|█████████▉| 222922/225000 [437:19:12<4:52:23,  8.44s/it]
 99%|█████████▉| 222923/225000 [437:19:24<5:27:48,  9.47s/it]
 99%|█████████▉| 222924/225000 [437:19:35<5:42:24,  9.90s/it]
 99%|█████████▉| 222925/225000 [437:19:39<4:49:18,  8.37s/it]
 99%|█████████▉| 222926/225000 [437:19:45<4:16:56,  7.43s/it]
 99%|█████████▉| 222927/225000 [437:19:56<4:57:50,  8.62s/it]
 99%|█████████▉| 222928/225000 [437:20:03<4:35:34,  7.98s/it]
 99%|█████████▉| 222929/225000 [437:20:10<4:28:53,  7.79s/it]
 99%|█████████▉| 222930/225000 [437:20:22<5:16:03,  9.16s/it]
 99%|█████████▉| 222931/225000 [437:20:27<4:32:29,  7.90s/it]
 99%|█████████▉| 222932/225000 [437:20:40<5:18:32,  9.24s/it]
 99%|█████████▉| 222933/225000 [437:20:48<5:13:29,  9.10s/it]
 99%|█████████▉| 222934/225000 [437:20:57<5:11:54,  9.06s/it]
 99%|█████████▉| 222935/225000 [437:21:04<4:43:05,  8.23s/it]
 99%|█████████▉| 222936/225000 [437:21:14<5:09:44,  9.00s/it]
 99%|█████████▉| 222937/225000 [437:21:22<4:55:38,  8.60s/it]
 99%|█████████▉| 222938/225000 [437:21:31<5:00:44,  8.75s/it]
 99%|█████████▉| 222939/225000 [437:21:40<5:00:38,  8.75s/it]
 99%|█████████▉| 222940/225000 [437:21:55<6:03:57, 10.60s/it]
 99%|█████████▉| 222941/225000 [437:22:00<5:06:02,  8.92s/it]
 99%|█████████▉| 222942/225000 [437:22:10<5:19:07,  9.30s/it]
 99%|█████████▉| 222943/225000 [437:22:15<4:36:41,  8.07s/it]
 99%|█████████▉| 222944/225000 [437:22:28<5:23:43,  9.45s/it]
 99%|█████████▉| 222945/225000 [437:22:39<5:45:44, 10.09s/it]
 99%|█████████▉| 222946/225000 [437:22:46<5:10:55,  9.08s/it]
 99%|█████████▉| 222947/225000 [437:22:58<5:39:26,  9.92s/it]
 99%|█████████▉| 222948/225000 [437:23:09<5:48:15, 10.18s/it]
 99%|█████████▉| 222949/225000 [437:23:14<4:57:37,  8.71s/it]
 99%|█████████▉| 222950/225000 [437:23:25<5:15:18,  9.23s/it]
                                                             
{'loss': '1.336', 'grad_norm': '1.109', 'learning_rate': '1.239e-08', 'epoch': '0.9909', 'num_input_tokens_seen': 746650248, 'train_runtime': '1.575e+06', 'train_tokens_per_second': '474.2'}

 99%|█████████▉| 222950/225000 [437:23:25<5:15:18,  9.23s/it]
 99%|█████████▉| 222951/225000 [437:23:34<5:15:37,  9.24s/it]
 99%|█████████▉| 222952/225000 [437:23:39<4:38:38,  8.16s/it]
 99%|█████████▉| 222953/225000 [437:23:50<4:58:18,  8.74s/it]
 99%|█████████▉| 222954/225000 [437:23:56<4:31:16,  7.96s/it]
 99%|█████████▉| 222955/225000 [437:24:06<4:56:33,  8.70s/it]
 99%|█████████▉| 222956/225000 [437:24:13<4:42:10,  8.28s/it]
 99%|█████████▉| 222957/225000 [437:24:27<5:31:45,  9.74s/it]
 99%|█████████▉| 222958/225000 [437:24:37<5:37:01,  9.90s/it]
 99%|█████████▉| 222959/225000 [437:24:45<5:16:36,  9.31s/it]
 99%|█████████▉| 222960/225000 [437:24:55<5:25:00,  9.56s/it]
 99%|█████████▉| 222961/225000 [437:25:00<4:40:18,  8.25s/it]
 99%|█████████▉| 222962/225000 [437:25:11<5:06:19,  9.02s/it]
 99%|█████████▉| 222963/225000 [437:25:19<4:56:17,  8.73s/it]
 99%|█████████▉| 222964/225000 [437:25:25<4:30:31,  7.97s/it]
 99%|█████████▉| 222965/225000 [437:25:35<4:46:55,  8.46s/it]
 99%|█████████▉| 222966/225000 [437:25:41<4:20:28,  7.68s/it]
 99%|█████████▉| 222967/225000 [437:25:49<4:31:38,  8.02s/it]
 99%|█████████▉| 222968/225000 [437:25:59<4:46:56,  8.47s/it]
 99%|█████████▉| 222969/225000 [437:26:13<5:44:22, 10.17s/it]
 99%|█████████▉| 222970/225000 [437:26:22<5:33:23,  9.85s/it]
 99%|█████████▉| 222971/225000 [437:26:31<5:26:34,  9.66s/it]
 99%|█████████▉| 222972/225000 [437:26:39<5:07:00,  9.08s/it]
 99%|█████████▉| 222973/225000 [437:26:46<4:42:31,  8.36s/it]
 99%|█████████▉| 222974/225000 [437:26:53<4:25:02,  7.85s/it]
 99%|█████████▉| 222975/225000 [437:26:59<4:13:54,  7.52s/it]
 99%|█████████▉| 222976/225000 [437:27:14<5:23:14,  9.58s/it]
 99%|█████████▉| 222977/225000 [437:27:18<4:31:58,  8.07s/it]
 99%|█████████▉| 222978/225000 [437:27:25<4:20:00,  7.72s/it]
 99%|█████████▉| 222979/225000 [437:27:37<5:02:54,  8.99s/it]
 99%|█████████▉| 222980/225000 [437:27:47<5:15:30,  9.37s/it]
 99%|█████████▉| 222981/225000 [437:27:57<5:21:57,  9.57s/it]
 99%|█████████▉| 222982/225000 [437:28:09<5:40:10, 10.11s/it]
 99%|█████████▉| 222983/225000 [437:28:17<5:16:21,  9.41s/it]
 99%|█████████▉| 222984/225000 [437:28:24<4:56:29,  8.82s/it]
 99%|█████████▉| 222985/225000 [437:28:36<5:28:23,  9.78s/it]
 99%|█████████▉| 222986/225000 [437:28:46<5:31:40,  9.88s/it]
 99%|█████████▉| 222987/225000 [437:28:52<4:47:43,  8.58s/it]
 99%|█████████▉| 222988/225000 [437:28:59<4:36:14,  8.24s/it]
 99%|█████████▉| 222989/225000 [437:29:12<5:25:57,  9.73s/it]
 99%|█████████▉| 222990/225000 [437:29:20<5:01:47,  9.01s/it]
 99%|█████████▉| 222991/225000 [437:29:27<4:44:00,  8.48s/it]
 99%|█████████▉| 222992/225000 [437:29:37<4:59:44,  8.96s/it]
 99%|█████████▉| 222993/225000 [437:29:48<5:16:49,  9.47s/it]
 99%|█████████▉| 222994/225000 [437:29:54<4:49:04,  8.65s/it]
 99%|█████████▉| 222995/225000 [437:30:01<4:28:12,  8.03s/it]
 99%|█████████▉| 222996/225000 [437:30:16<5:36:52, 10.09s/it]
 99%|█████████▉| 222997/225000 [437:30:25<5:24:42,  9.73s/it]
 99%|█████████▉| 222998/225000 [437:30:31<4:52:05,  8.75s/it]
 99%|█████████▉| 222999/225000 [437:30:43<5:22:42,  9.68s/it]
 99%|█████████▉| 223000/225000 [437:30:54<5:32:26,  9.97s/it]
                                                             
{'loss': '1.321', 'grad_norm': '1.381', 'learning_rate': '1.183e-08', 'epoch': '0.9911', 'num_input_tokens_seen': 746891072, 'train_runtime': '1.575e+06', 'train_tokens_per_second': '474.2'}

 99%|█████████▉| 223000/225000 [437:30:54<5:32:26,  9.97s/it]
 99%|█████████▉| 223001/225000 [437:31:01<5:10:00,  9.31s/it]
 99%|█████████▉| 223002/225000 [437:31:08<4:43:49,  8.52s/it]
 99%|█████████▉| 223003/225000 [437:31:21<5:25:38,  9.78s/it]
 99%|█████████▉| 223004/225000 [437:31:28<4:57:44,  8.95s/it]
 99%|█████████▉| 223005/225000 [437:31:39<5:17:34,  9.55s/it]
 99%|█████████▉| 223006/225000 [437:31:47<5:02:04,  9.09s/it]
 99%|█████████▉| 223007/225000 [437:31:54<4:43:10,  8.53s/it]
 99%|█████████▉| 223008/225000 [437:32:07<5:26:30,  9.83s/it]
 99%|█████████▉| 223009/225000 [437:32:13<4:48:35,  8.70s/it]
 99%|█████████▉| 223010/225000 [437:32:21<4:38:27,  8.40s/it]
 99%|█████████▉| 223011/225000 [437:32:26<4:04:39,  7.38s/it]
 99%|█████████▉| 223012/225000 [437:32:36<4:33:22,  8.25s/it]
 99%|█████████▉| 223013/225000 [437:32:47<5:01:33,  9.11s/it]
 99%|█████████▉| 223014/225000 [437:32:58<5:19:53,  9.66s/it]
 99%|█████████▉| 223015/225000 [437:33:07<5:11:38,  9.42s/it]
 99%|█████████▉| 223016/225000 [437:33:11<4:22:59,  7.95s/it]
 99%|█████████▉| 223017/225000 [437:33:22<4:51:51,  8.83s/it]
 99%|█████████▉| 223018/225000 [437:33:26<4:06:07,  7.45s/it]
 99%|█████████▉| 223019/225000 [437:33:38<4:47:37,  8.71s/it]
 99%|█████████▉| 223020/225000 [437:33:47<4:51:45,  8.84s/it]
 99%|█████████▉| 223021/225000 [437:34:00<5:32:44, 10.09s/it]
 99%|█████████▉| 223022/225000 [437:34:07<4:56:45,  9.00s/it]
 99%|█████████▉| 223023/225000 [437:34:13<4:28:37,  8.15s/it]
 99%|█████████▉| 223024/225000 [437:34:20<4:15:32,  7.76s/it]
 99%|█████████▉| 223025/225000 [437:34:27<4:09:17,  7.57s/it]
 99%|█████████▉| 223026/225000 [437:34:40<5:02:32,  9.20s/it]
 99%|█████████▉| 223027/225000 [437:34:46<4:29:36,  8.20s/it]
 99%|█████████▉| 223028/225000 [437:34:55<4:40:11,  8.52s/it]
 99%|█████████▉| 223029/225000 [437:35:06<5:04:33,  9.27s/it]
 99%|█████████▉| 223030/225000 [437:35:13<4:45:22,  8.69s/it]
 99%|█████████▉| 223031/225000 [437:35:19<4:18:10,  7.87s/it]
 99%|█████████▉| 223032/225000 [437:35:35<5:39:19, 10.35s/it]
 99%|█████████▉| 223033/225000 [437:35:46<5:43:03, 10.46s/it]
 99%|█████████▉| 223034/225000 [437:35:53<5:11:02,  9.49s/it]
 99%|█████████▉| 223035/225000 [437:36:02<4:58:31,  9.12s/it]
 99%|█████████▉| 223036/225000 [437:36:10<4:54:35,  9.00s/it]
 99%|█████████▉| 223037/225000 [437:36:22<5:20:56,  9.81s/it]
 99%|█████████▉| 223038/225000 [437:36:27<4:30:05,  8.26s/it]
 99%|█████████▉| 223039/225000 [437:36:38<4:54:51,  9.02s/it]
 99%|█████████▉| 223040/225000 [437:36:45<4:41:29,  8.62s/it]
 99%|█████████▉| 223041/225000 [437:36:52<4:27:44,  8.20s/it]
 99%|█████████▉| 223042/225000 [437:37:00<4:16:56,  7.87s/it]
 99%|█████████▉| 223043/225000 [437:37:09<4:31:12,  8.32s/it]
 99%|█████████▉| 223044/225000 [437:37:21<5:06:47,  9.41s/it]
 99%|█████████▉| 223045/225000 [437:37:34<5:42:01, 10.50s/it]
 99%|█████████▉| 223046/225000 [437:37:39<4:45:42,  8.77s/it]
 99%|█████████▉| 223047/225000 [437:37:48<4:54:56,  9.06s/it]
 99%|█████████▉| 223048/225000 [437:37:57<4:50:03,  8.92s/it]
 99%|█████████▉| 223049/225000 [437:38:05<4:36:53,  8.52s/it]
 99%|█████████▉| 223050/225000 [437:38:12<4:27:47,  8.24s/it]
                                                             
{'loss': '1.306', 'grad_norm': '1.724', 'learning_rate': '1.127e-08', 'epoch': '0.9913', 'num_input_tokens_seen': 747133112, 'train_runtime': '1.575e+06', 'train_tokens_per_second': '474.2'}

 99%|█████████▉| 223050/225000 [437:38:12<4:27:47,  8.24s/it]
 99%|█████████▉| 223051/225000 [437:38:24<5:04:47,  9.38s/it]
 99%|█████████▉| 223052/225000 [437:38:34<5:09:19,  9.53s/it]
 99%|█████████▉| 223053/225000 [437:38:46<5:29:14, 10.15s/it]
 99%|█████████▉| 223054/225000 [437:38:55<5:22:26,  9.94s/it]
 99%|█████████▉| 223055/225000 [437:39:01<4:45:46,  8.82s/it]
 99%|█████████▉| 223056/225000 [437:39:10<4:42:21,  8.71s/it]
 99%|█████████▉| 223057/225000 [437:39:20<4:57:25,  9.18s/it]
 99%|█████████▉| 223058/225000 [437:39:26<4:29:12,  8.32s/it]
 99%|█████████▉| 223059/225000 [437:39:35<4:35:26,  8.51s/it]
 99%|█████████▉| 223060/225000 [437:39:46<4:52:54,  9.06s/it]
 99%|█████████▉| 223061/225000 [437:39:57<5:11:31,  9.64s/it]
 99%|█████████▉| 223062/225000 [437:40:04<4:52:28,  9.05s/it]
 99%|█████████▉| 223063/225000 [437:40:12<4:38:25,  8.62s/it]
 99%|█████████▉| 223064/225000 [437:40:22<4:50:24,  9.00s/it]
 99%|█████████▉| 223065/225000 [437:40:32<5:01:22,  9.35s/it]
 99%|█████████▉| 223066/225000 [437:40:38<4:31:45,  8.43s/it]
 99%|█████████▉| 223067/225000 [437:40:53<5:32:01, 10.31s/it]
 99%|█████████▉| 223068/225000 [437:40:58<4:42:52,  8.78s/it]
 99%|█████████▉| 223069/225000 [437:41:07<4:41:36,  8.75s/it]
 99%|█████████▉| 223070/225000 [437:41:14<4:27:29,  8.32s/it]
 99%|█████████▉| 223071/225000 [437:41:22<4:23:25,  8.19s/it]
 99%|█████████▉| 223072/225000 [437:41:29<4:13:17,  7.88s/it]
 99%|█████████▉| 223073/225000 [437:41:38<4:24:50,  8.25s/it]
 99%|█████████▉| 223074/225000 [437:41:50<4:58:39,  9.30s/it]
 99%|█████████▉| 223075/225000 [437:41:55<4:17:44,  8.03s/it]
 99%|█████████▉| 223076/225000 [437:42:03<4:14:05,  7.92s/it]
 99%|█████████▉| 223077/225000 [437:42:14<4:46:45,  8.95s/it]
 99%|█████████▉| 223078/225000 [437:42:22<4:37:18,  8.66s/it]
 99%|█████████▉| 223079/225000 [437:42:35<5:16:31,  9.89s/it]
 99%|█████████▉| 223080/225000 [437:42:46<5:24:24, 10.14s/it]
 99%|█████████▉| 223081/225000 [437:42:55<5:20:51, 10.03s/it]
 99%|█████████▉| 223082/225000 [437:43:03<5:01:48,  9.44s/it]
 99%|█████████▉| 223083/225000 [437:43:14<5:14:08,  9.83s/it]
 99%|█████████▉| 223084/225000 [437:43:25<5:25:49, 10.20s/it]
 99%|█████████▉| 223085/225000 [437:43:33<4:58:29,  9.35s/it]
 99%|█████████▉| 223086/225000 [437:43:39<4:29:50,  8.46s/it]
 99%|█████████▉| 223087/225000 [437:43:51<5:03:48,  9.53s/it]
 99%|█████████▉| 223088/225000 [437:44:01<5:10:43,  9.75s/it]
 99%|█████████▉| 223089/225000 [437:44:12<5:15:19,  9.90s/it]
 99%|█████████▉| 223090/225000 [437:44:24<5:39:43, 10.67s/it]
 99%|█████████▉| 223091/225000 [437:44:31<5:05:39,  9.61s/it]
 99%|█████████▉| 223092/225000 [437:44:41<5:06:12,  9.63s/it]
 99%|█████████▉| 223093/225000 [437:44:50<4:58:24,  9.39s/it]
 99%|█████████▉| 223094/225000 [437:44:58<4:43:35,  8.93s/it]
 99%|█████████▉| 223095/225000 [437:45:03<4:10:18,  7.88s/it]
 99%|█████████▉| 223096/225000 [437:45:13<4:29:40,  8.50s/it]
 99%|█████████▉| 223097/225000 [437:45:20<4:12:30,  7.96s/it]
 99%|█████████▉| 223098/225000 [437:45:28<4:19:17,  8.18s/it]
 99%|█████████▉| 223099/225000 [437:45:37<4:22:11,  8.28s/it]
 99%|█████████▉| 223100/225000 [437:45:45<4:20:51,  8.24s/it]
                                                             
{'loss': '1.321', 'grad_norm': '1.76', 'learning_rate': '1.073e-08', 'epoch': '0.9916', 'num_input_tokens_seen': 747373648, 'train_runtime': '1.576e+06', 'train_tokens_per_second': '474.2'}

 99%|█████████▉| 223100/225000 [437:45:45<4:20:51,  8.24s/it]
 99%|█████████▉| 223101/225000 [437:45:52<4:09:13,  7.87s/it]
 99%|█████████▉| 223102/225000 [437:45:59<3:59:47,  7.58s/it]
 99%|█████████▉| 223103/225000 [437:46:05<3:43:35,  7.07s/it]
 99%|█████████▉| 223104/225000 [437:46:23<5:31:57, 10.50s/it]
 99%|█████████▉| 223105/225000 [437:46:30<4:52:21,  9.26s/it]
 99%|█████████▉| 223106/225000 [437:46:41<5:09:26,  9.80s/it]
 99%|█████████▉| 223107/225000 [437:46:50<5:03:12,  9.61s/it]
 99%|█████████▉| 223108/225000 [437:46:57<4:41:33,  8.93s/it]
 99%|█████████▉| 223109/225000 [437:47:04<4:25:07,  8.41s/it]
 99%|█████████▉| 223110/225000 [437:47:16<4:55:25,  9.38s/it]
 99%|█████████▉| 223111/225000 [437:47:23<4:28:55,  8.54s/it]
 99%|█████████▉| 223112/225000 [437:47:32<4:41:08,  8.93s/it]
 99%|█████████▉| 223113/225000 [437:47:40<4:31:54,  8.65s/it]
 99%|█████████▉| 223114/225000 [437:47:47<4:14:57,  8.11s/it]
 99%|█████████▉| 223115/225000 [437:47:55<4:09:52,  7.95s/it]
 99%|█████████▉| 223116/225000 [437:48:07<4:45:46,  9.10s/it]
 99%|█████████▉| 223117/225000 [437:48:16<4:50:57,  9.27s/it]
 99%|█████████▉| 223118/225000 [437:48:22<4:21:26,  8.34s/it]
 99%|█████████▉| 223119/225000 [437:48:31<4:20:36,  8.31s/it]
 99%|█████████▉| 223120/225000 [437:48:42<4:47:17,  9.17s/it]
 99%|█████████▉| 223121/225000 [437:48:52<4:59:35,  9.57s/it]
 99%|█████████▉| 223122/225000 [437:48:59<4:35:53,  8.81s/it]
 99%|█████████▉| 223123/225000 [437:49:06<4:13:56,  8.12s/it]
 99%|█████████▉| 223124/225000 [437:49:17<4:39:08,  8.93s/it]
 99%|█████████▉| 223125/225000 [437:49:30<5:18:18, 10.19s/it]
 99%|█████████▉| 223126/225000 [437:49:36<4:36:24,  8.85s/it]
 99%|█████████▉| 223127/225000 [437:49:46<4:51:30,  9.34s/it]
 99%|█████████▉| 223128/225000 [437:49:52<4:17:10,  8.24s/it]
 99%|█████████▉| 223129/225000 [437:49:59<4:03:15,  7.80s/it]
 99%|█████████▉| 223130/225000 [437:50:05<3:51:36,  7.43s/it]
 99%|█████████▉| 223131/225000 [437:50:20<4:58:20,  9.58s/it]
 99%|█████████▉| 223132/225000 [437:50:24<4:12:28,  8.11s/it]
 99%|█████████▉| 223133/225000 [437:50:34<4:27:06,  8.58s/it]
 99%|█████████▉| 223134/225000 [437:50:40<4:05:28,  7.89s/it]
 99%|█████████▉| 223135/225000 [437:50:53<4:50:40,  9.35s/it]
 99%|█████████▉| 223136/225000 [437:51:04<5:05:24,  9.83s/it]
 99%|█████████▉| 223137/225000 [437:51:11<4:40:03,  9.02s/it]
 99%|█████████▉| 223138/225000 [437:51:20<4:36:54,  8.92s/it]
 99%|█████████▉| 223139/225000 [437:51:32<5:10:26, 10.01s/it]
 99%|█████████▉| 223140/225000 [437:51:43<5:18:10, 10.26s/it]
 99%|█████████▉| 223141/225000 [437:51:55<5:28:45, 10.61s/it]
 99%|█████████▉| 223142/225000 [437:52:03<5:07:37,  9.93s/it]
 99%|█████████▉| 223143/225000 [437:52:13<5:08:10,  9.96s/it]
 99%|█████████▉| 223144/225000 [437:52:25<5:29:00, 10.64s/it]
 99%|█████████▉| 223145/225000 [437:52:35<5:17:47, 10.28s/it]
 99%|█████████▉| 223146/225000 [437:52:43<4:55:28,  9.56s/it]
 99%|█████████▉| 223147/225000 [437:52:49<4:25:08,  8.59s/it]
 99%|█████████▉| 223148/225000 [437:52:55<3:59:12,  7.75s/it]
 99%|█████████▉| 223149/225000 [437:53:07<4:39:29,  9.06s/it]
 99%|█████████▉| 223150/225000 [437:53:15<4:26:01,  8.63s/it]
                                                             
{'loss': '1.315', 'grad_norm': '1.532', 'learning_rate': '1.02e-08', 'epoch': '0.9918', 'num_input_tokens_seen': 747611112, 'train_runtime': '1.576e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223150/225000 [437:53:15<4:26:01,  8.63s/it]
 99%|█████████▉| 223151/225000 [437:53:22<4:11:02,  8.15s/it]
 99%|█████████▉| 223152/225000 [437:53:35<5:00:16,  9.75s/it]
 99%|█████████▉| 223153/225000 [437:53:45<4:59:07,  9.72s/it]
 99%|█████████▉| 223154/225000 [437:53:51<4:24:57,  8.61s/it]
 99%|█████████▉| 223155/225000 [437:53:58<4:17:21,  8.37s/it]
 99%|█████████▉| 223156/225000 [437:54:14<5:20:22, 10.42s/it]
 99%|█████████▉| 223157/225000 [437:54:22<5:00:47,  9.79s/it]
 99%|█████████▉| 223158/225000 [437:54:33<5:15:24, 10.27s/it]
 99%|█████████▉| 223159/225000 [437:54:41<4:48:14,  9.39s/it]
 99%|█████████▉| 223160/225000 [437:54:47<4:15:12,  8.32s/it]
 99%|█████████▉| 223161/225000 [437:54:53<4:01:56,  7.89s/it]
 99%|█████████▉| 223162/225000 [437:55:05<4:38:32,  9.09s/it]
 99%|█████████▉| 223163/225000 [437:55:13<4:23:00,  8.59s/it]
 99%|█████████▉| 223164/225000 [437:55:18<3:48:36,  7.47s/it]
 99%|█████████▉| 223165/225000 [437:55:30<4:37:09,  9.06s/it]
 99%|█████████▉| 223166/225000 [437:55:39<4:28:31,  8.79s/it]
 99%|█████████▉| 223167/225000 [437:55:45<4:02:38,  7.94s/it]
 99%|█████████▉| 223168/225000 [437:56:00<5:12:46, 10.24s/it]
 99%|█████████▉| 223169/225000 [437:56:06<4:33:40,  8.97s/it]
 99%|█████████▉| 223170/225000 [437:56:15<4:27:53,  8.78s/it]
 99%|█████████▉| 223171/225000 [437:56:25<4:39:04,  9.16s/it]
 99%|█████████▉| 223172/225000 [437:56:35<4:48:39,  9.47s/it]
 99%|█████████▉| 223173/225000 [437:56:43<4:40:26,  9.21s/it]
 99%|█████████▉| 223174/225000 [437:56:55<5:00:47,  9.88s/it]
 99%|█████████▉| 223175/225000 [437:57:00<4:18:46,  8.51s/it]
 99%|█████████▉| 223176/225000 [437:57:08<4:15:46,  8.41s/it]
 99%|█████████▉| 223177/225000 [437:57:13<3:46:26,  7.45s/it]
 99%|█████████▉| 223178/225000 [437:57:26<4:33:02,  8.99s/it]
 99%|█████████▉| 223179/225000 [437:57:34<4:23:27,  8.68s/it]
 99%|█████████▉| 223180/225000 [437:57:51<5:36:31, 11.09s/it]
 99%|█████████▉| 223181/225000 [437:57:58<4:58:07,  9.83s/it]
 99%|█████████▉| 223182/225000 [437:58:04<4:28:05,  8.85s/it]
 99%|█████████▉| 223183/225000 [437:58:16<4:58:15,  9.85s/it]
 99%|█████████▉| 223184/225000 [437:58:21<4:08:51,  8.22s/it]
 99%|█████████▉| 223185/225000 [437:58:30<4:20:53,  8.62s/it]
 99%|█████████▉| 223186/225000 [437:58:38<4:13:55,  8.40s/it]
 99%|█████████▉| 223187/225000 [437:58:45<3:57:48,  7.87s/it]
 99%|█████████▉| 223188/225000 [437:58:54<4:09:26,  8.26s/it]
 99%|█████████▉| 223189/225000 [437:59:09<5:09:52, 10.27s/it]
 99%|█████████▉| 223190/225000 [437:59:15<4:35:26,  9.13s/it]
 99%|█████████▉| 223191/225000 [437:59:30<5:19:58, 10.61s/it]
 99%|█████████▉| 223192/225000 [437:59:37<4:52:11,  9.70s/it]
 99%|█████████▉| 223193/225000 [437:59:45<4:35:19,  9.14s/it]
 99%|█████████▉| 223194/225000 [437:59:54<4:33:35,  9.09s/it]
 99%|█████████▉| 223195/225000 [438:00:03<4:34:03,  9.11s/it]
 99%|█████████▉| 223196/225000 [438:00:10<4:10:12,  8.32s/it]
 99%|█████████▉| 223197/225000 [438:00:21<4:34:26,  9.13s/it]
 99%|█████████▉| 223198/225000 [438:00:26<4:04:40,  8.15s/it]
 99%|█████████▉| 223199/225000 [438:00:37<4:30:06,  9.00s/it]
 99%|█████████▉| 223200/225000 [438:00:49<4:49:07,  9.64s/it]
                                                             
{'loss': '1.349', 'grad_norm': '1.203', 'learning_rate': '9.685e-09', 'epoch': '0.992', 'num_input_tokens_seen': 747850400, 'train_runtime': '1.577e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223200/225000 [438:00:49<4:49:07,  9.64s/it]
 99%|█████████▉| 223201/225000 [438:00:55<4:20:04,  8.67s/it]
 99%|█████████▉| 223202/225000 [438:01:00<3:44:06,  7.48s/it]
 99%|█████████▉| 223203/225000 [438:01:09<4:04:17,  8.16s/it]
 99%|█████████▉| 223204/225000 [438:01:19<4:15:07,  8.52s/it]
 99%|█████████▉| 223205/225000 [438:01:27<4:14:46,  8.52s/it]
 99%|█████████▉| 223206/225000 [438:01:34<4:01:33,  8.08s/it]
 99%|█████████▉| 223207/225000 [438:01:50<5:12:24, 10.45s/it]
 99%|█████████▉| 223208/225000 [438:01:55<4:24:36,  8.86s/it]
 99%|█████████▉| 223209/225000 [438:02:03<4:12:54,  8.47s/it]
 99%|█████████▉| 223210/225000 [438:02:16<4:56:40,  9.94s/it]
 99%|█████████▉| 223211/225000 [438:02:23<4:28:26,  9.00s/it]
 99%|█████████▉| 223212/225000 [438:02:33<4:30:53,  9.09s/it]
 99%|█████████▉| 223213/225000 [438:02:44<4:48:46,  9.70s/it]
 99%|█████████▉| 223214/225000 [438:02:53<4:48:06,  9.68s/it]
 99%|█████████▉| 223215/225000 [438:03:00<4:26:03,  8.94s/it]
 99%|█████████▉| 223216/225000 [438:03:09<4:23:02,  8.85s/it]
 99%|█████████▉| 223217/225000 [438:03:20<4:44:37,  9.58s/it]
 99%|█████████▉| 223218/225000 [438:03:28<4:27:29,  9.01s/it]
 99%|█████████▉| 223219/225000 [438:03:35<4:10:57,  8.45s/it]
 99%|█████████▉| 223220/225000 [438:03:42<3:58:16,  8.03s/it]
 99%|█████████▉| 223221/225000 [438:03:53<4:19:25,  8.75s/it]
 99%|█████████▉| 223222/225000 [438:04:01<4:15:15,  8.61s/it]
 99%|█████████▉| 223223/225000 [438:04:07<3:50:47,  7.79s/it]
 99%|█████████▉| 223224/225000 [438:04:15<3:53:04,  7.87s/it]
 99%|█████████▉| 223225/225000 [438:04:29<4:46:47,  9.69s/it]
 99%|█████████▉| 223226/225000 [438:04:34<4:06:11,  8.33s/it]
 99%|█████████▉| 223227/225000 [438:04:41<3:52:15,  7.86s/it]
 99%|█████████▉| 223228/225000 [438:04:51<4:15:47,  8.66s/it]
 99%|█████████▉| 223229/225000 [438:05:01<4:27:49,  9.07s/it]
 99%|█████████▉| 223230/225000 [438:05:07<3:59:38,  8.12s/it]
 99%|█████████▉| 223231/225000 [438:05:14<3:50:52,  7.83s/it]
 99%|█████████▉| 223232/225000 [438:05:27<4:29:38,  9.15s/it]
 99%|█████████▉| 223233/225000 [438:05:37<4:36:35,  9.39s/it]
 99%|█████████▉| 223234/225000 [438:05:44<4:17:41,  8.76s/it]
 99%|█████████▉| 223235/225000 [438:05:54<4:33:53,  9.31s/it]
 99%|█████████▉| 223236/225000 [438:06:02<4:14:42,  8.66s/it]
 99%|█████████▉| 223237/225000 [438:06:09<4:00:06,  8.17s/it]
 99%|█████████▉| 223238/225000 [438:06:18<4:11:23,  8.56s/it]
 99%|█████████▉| 223239/225000 [438:06:23<3:41:37,  7.55s/it]
 99%|█████████▉| 223240/225000 [438:06:34<4:08:06,  8.46s/it]
 99%|█████████▉| 223241/225000 [438:06:41<3:52:45,  7.94s/it]
 99%|█████████▉| 223242/225000 [438:06:56<4:57:39, 10.16s/it]
 99%|█████████▉| 223243/225000 [438:07:03<4:30:46,  9.25s/it]
 99%|█████████▉| 223244/225000 [438:07:16<5:04:43, 10.41s/it]
 99%|█████████▉| 223245/225000 [438:07:23<4:36:55,  9.47s/it]
 99%|█████████▉| 223246/225000 [438:07:30<4:12:54,  8.65s/it]
 99%|█████████▉| 223247/225000 [438:07:38<4:07:57,  8.49s/it]
 99%|█████████▉| 223248/225000 [438:07:51<4:45:12,  9.77s/it]
 99%|█████████▉| 223249/225000 [438:08:03<5:05:33, 10.47s/it]
 99%|█████████▉| 223250/225000 [438:08:09<4:24:12,  9.06s/it]
                                                             
{'loss': '1.348', 'grad_norm': '1.633', 'learning_rate': '9.183e-09', 'epoch': '0.9922', 'num_input_tokens_seen': 748088888, 'train_runtime': '1.577e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223250/225000 [438:08:09<4:24:12,  9.06s/it]
 99%|█████████▉| 223251/225000 [438:08:20<4:44:54,  9.77s/it]
 99%|█████████▉| 223252/225000 [438:08:30<4:42:05,  9.68s/it]
 99%|█████████▉| 223253/225000 [438:08:37<4:15:28,  8.77s/it]
 99%|█████████▉| 223254/225000 [438:08:45<4:12:35,  8.68s/it]
 99%|█████████▉| 223255/225000 [438:08:54<4:15:12,  8.78s/it]
 99%|█████████▉| 223256/225000 [438:09:02<4:12:44,  8.69s/it]
 99%|█████████▉| 223257/225000 [438:09:13<4:32:02,  9.36s/it]
 99%|█████████▉| 223258/225000 [438:09:27<5:09:00, 10.64s/it]
 99%|█████████▉| 223259/225000 [438:09:33<4:30:37,  9.33s/it]
 99%|█████████▉| 223260/225000 [438:09:44<4:43:33,  9.78s/it]
 99%|█████████▉| 223261/225000 [438:09:49<4:03:30,  8.40s/it]
 99%|█████████▉| 223262/225000 [438:10:01<4:34:00,  9.46s/it]
 99%|█████████▉| 223263/225000 [438:10:12<4:44:20,  9.82s/it]
 99%|█████████▉| 223264/225000 [438:10:18<4:08:34,  8.59s/it]
 99%|█████████▉| 223265/225000 [438:10:29<4:29:25,  9.32s/it]
 99%|█████████▉| 223266/225000 [438:10:40<4:49:32, 10.02s/it]
 99%|█████████▉| 223267/225000 [438:10:45<4:06:08,  8.52s/it]
 99%|█████████▉| 223268/225000 [438:10:55<4:17:41,  8.93s/it]
 99%|█████████▉| 223269/225000 [438:11:01<3:51:56,  8.04s/it]
 99%|█████████▉| 223270/225000 [438:11:07<3:34:50,  7.45s/it]
 99%|█████████▉| 223271/225000 [438:11:15<3:37:12,  7.54s/it]
 99%|█████████▉| 223272/225000 [438:11:27<4:14:04,  8.82s/it]
 99%|█████████▉| 223273/225000 [438:11:35<4:05:47,  8.54s/it]
 99%|█████████▉| 223274/225000 [438:11:43<4:00:54,  8.37s/it]
 99%|█████████▉| 223275/225000 [438:11:52<4:05:25,  8.54s/it]
 99%|█████████▉| 223276/225000 [438:12:04<4:40:20,  9.76s/it]
 99%|█████████▉| 223277/225000 [438:12:10<4:02:56,  8.46s/it]
 99%|█████████▉| 223278/225000 [438:12:21<4:28:32,  9.36s/it]
 99%|█████████▉| 223279/225000 [438:12:31<4:30:29,  9.43s/it]
 99%|█████████▉| 223280/225000 [438:12:38<4:09:47,  8.71s/it]
 99%|█████████▉| 223281/225000 [438:12:45<3:55:52,  8.23s/it]
 99%|█████████▉| 223282/225000 [438:12:52<3:49:10,  8.00s/it]
 99%|█████████▉| 223283/225000 [438:13:04<4:21:02,  9.12s/it]
 99%|█████████▉| 223284/225000 [438:13:15<4:34:31,  9.60s/it]
 99%|█████████▉| 223285/225000 [438:13:24<4:34:08,  9.59s/it]
 99%|█████████▉| 223286/225000 [438:13:35<4:39:41,  9.79s/it]
 99%|█████████▉| 223287/225000 [438:13:42<4:23:15,  9.22s/it]
 99%|█████████▉| 223288/225000 [438:13:54<4:44:49,  9.98s/it]
 99%|█████████▉| 223289/225000 [438:14:00<4:04:31,  8.58s/it]
 99%|█████████▉| 223290/225000 [438:14:12<4:40:00,  9.82s/it]
 99%|█████████▉| 223291/225000 [438:14:22<4:40:39,  9.85s/it]
 99%|█████████▉| 223292/225000 [438:14:30<4:19:05,  9.10s/it]
 99%|█████████▉| 223293/225000 [438:14:36<3:54:15,  8.23s/it]
 99%|█████████▉| 223294/225000 [438:14:43<3:43:16,  7.85s/it]
 99%|█████████▉| 223295/225000 [438:14:54<4:14:12,  8.95s/it]
 99%|█████████▉| 223296/225000 [438:15:01<3:56:39,  8.33s/it]
 99%|█████████▉| 223297/225000 [438:15:09<3:50:59,  8.14s/it]
 99%|█████████▉| 223298/225000 [438:15:22<4:37:10,  9.77s/it]
 99%|█████████▉| 223299/225000 [438:15:28<4:05:55,  8.67s/it]
 99%|█████████▉| 223300/225000 [438:15:41<4:37:16,  9.79s/it]
                                                             
{'loss': '1.289', 'grad_norm': '2.225', 'learning_rate': '8.695e-09', 'epoch': '0.9924', 'num_input_tokens_seen': 748333272, 'train_runtime': '1.578e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223300/225000 [438:15:41<4:37:16,  9.79s/it]
 99%|█████████▉| 223301/225000 [438:15:48<4:17:06,  9.08s/it]
 99%|█████████▉| 223302/225000 [438:15:56<4:01:41,  8.54s/it]
 99%|█████████▉| 223303/225000 [438:16:03<3:53:58,  8.27s/it]
 99%|█████████▉| 223304/225000 [438:16:16<4:34:09,  9.70s/it]
 99%|█████████▉| 223305/225000 [438:16:23<4:10:37,  8.87s/it]
 99%|█████████▉| 223306/225000 [438:16:32<4:10:53,  8.89s/it]
 99%|█████████▉| 223307/225000 [438:16:38<3:42:19,  7.88s/it]
 99%|█████████▉| 223308/225000 [438:16:54<4:53:32, 10.41s/it]
 99%|█████████▉| 223309/225000 [438:17:00<4:18:17,  9.16s/it]
 99%|█████████▉| 223310/225000 [438:17:09<4:18:36,  9.18s/it]
 99%|█████████▉| 223311/225000 [438:17:14<3:41:32,  7.87s/it]
 99%|█████████▉| 223312/225000 [438:17:26<4:10:09,  8.89s/it]
 99%|█████████▉| 223313/225000 [438:17:34<4:02:28,  8.62s/it]
 99%|█████████▉| 223314/225000 [438:17:42<3:57:46,  8.46s/it]
 99%|█████████▉| 223315/225000 [438:17:48<3:43:27,  7.96s/it]
 99%|█████████▉| 223316/225000 [438:17:58<3:57:33,  8.46s/it]
 99%|█████████▉| 223317/225000 [438:18:05<3:42:47,  7.94s/it]
 99%|█████████▉| 223318/225000 [438:18:13<3:48:13,  8.14s/it]
 99%|█████████▉| 223319/225000 [438:18:24<4:10:11,  8.93s/it]
 99%|█████████▉| 223320/225000 [438:18:32<4:00:34,  8.59s/it]
 99%|█████████▉| 223321/225000 [438:18:40<3:56:48,  8.46s/it]
 99%|█████████▉| 223322/225000 [438:18:52<4:22:29,  9.39s/it]
 99%|█████████▉| 223323/225000 [438:19:00<4:13:23,  9.07s/it]
 99%|█████████▉| 223324/225000 [438:19:10<4:24:09,  9.46s/it]
 99%|█████████▉| 223325/225000 [438:19:18<4:13:02,  9.06s/it]
 99%|█████████▉| 223326/225000 [438:19:28<4:19:04,  9.29s/it]
 99%|█████████▉| 223327/225000 [438:19:40<4:41:06, 10.08s/it]
 99%|█████████▉| 223328/225000 [438:19:49<4:30:10,  9.70s/it]
 99%|█████████▉| 223329/225000 [438:19:54<3:48:16,  8.20s/it]
 99%|█████████▉| 223330/225000 [438:20:04<4:04:00,  8.77s/it]
 99%|█████████▉| 223331/225000 [438:20:14<4:12:29,  9.08s/it]
 99%|█████████▉| 223332/225000 [438:20:24<4:19:32,  9.34s/it]
 99%|█████████▉| 223333/225000 [438:20:34<4:32:50,  9.82s/it]
 99%|█████████▉| 223334/225000 [438:20:43<4:19:30,  9.35s/it]
 99%|█████████▉| 223335/225000 [438:20:50<3:58:53,  8.61s/it]
 99%|█████████▉| 223336/225000 [438:20:56<3:37:15,  7.83s/it]
 99%|█████████▉| 223337/225000 [438:21:09<4:20:36,  9.40s/it]
 99%|█████████▉| 223338/225000 [438:21:17<4:08:32,  8.97s/it]
 99%|█████████▉| 223339/225000 [438:21:25<3:59:17,  8.64s/it]
 99%|█████████▉| 223340/225000 [438:21:31<3:44:06,  8.10s/it]
 99%|█████████▉| 223341/225000 [438:21:37<3:27:15,  7.50s/it]
 99%|█████████▉| 223342/225000 [438:21:46<3:35:11,  7.79s/it]
 99%|█████████▉| 223343/225000 [438:21:56<3:51:08,  8.37s/it]
 99%|█████████▉| 223344/225000 [438:22:05<3:58:51,  8.65s/it]
 99%|█████████▉| 223345/225000 [438:22:11<3:34:05,  7.76s/it]
 99%|█████████▉| 223346/225000 [438:22:22<4:04:37,  8.87s/it]
 99%|█████████▉| 223347/225000 [438:22:30<3:58:12,  8.65s/it]
 99%|█████████▉| 223348/225000 [438:22:40<4:04:51,  8.89s/it]
 99%|█████████▉| 223349/225000 [438:22:54<4:46:52, 10.43s/it]
 99%|█████████▉| 223350/225000 [438:23:03<4:40:30, 10.20s/it]
                                                             
{'loss': '1.307', 'grad_norm': '1.324', 'learning_rate': '8.22e-09', 'epoch': '0.9927', 'num_input_tokens_seen': 748571552, 'train_runtime': '1.578e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223350/225000 [438:23:03<4:40:30, 10.20s/it]
 99%|█████████▉| 223351/225000 [438:23:13<4:32:03,  9.90s/it]
 99%|█████████▉| 223352/225000 [438:23:17<3:48:38,  8.32s/it]
 99%|█████████▉| 223353/225000 [438:23:23<3:24:35,  7.45s/it]
 99%|█████████▉| 223354/225000 [438:23:30<3:20:43,  7.32s/it]
 99%|█████████▉| 223355/225000 [438:23:45<4:22:45,  9.58s/it]
 99%|█████████▉| 223356/225000 [438:23:50<3:50:11,  8.40s/it]
 99%|█████████▉| 223357/225000 [438:23:57<3:37:54,  7.96s/it]
 99%|█████████▉| 223358/225000 [438:24:04<3:25:01,  7.49s/it]
 99%|█████████▉| 223359/225000 [438:24:15<3:57:50,  8.70s/it]
 99%|█████████▉| 223360/225000 [438:24:19<3:23:02,  7.43s/it]
 99%|█████████▉| 223361/225000 [438:24:35<4:33:14, 10.00s/it]
 99%|█████████▉| 223362/225000 [438:24:45<4:33:00, 10.00s/it]
 99%|█████████▉| 223363/225000 [438:24:53<4:11:32,  9.22s/it]
 99%|█████████▉| 223364/225000 [438:25:05<4:32:11,  9.98s/it]
 99%|█████████▉| 223365/225000 [438:25:12<4:09:58,  9.17s/it]
 99%|█████████▉| 223366/225000 [438:25:23<4:24:40,  9.72s/it]
 99%|█████████▉| 223367/225000 [438:25:31<4:13:41,  9.32s/it]
 99%|█████████▉| 223368/225000 [438:25:40<4:09:13,  9.16s/it]
 99%|█████████▉| 223369/225000 [438:25:52<4:28:56,  9.89s/it]
 99%|█████████▉| 223370/225000 [438:26:00<4:13:47,  9.34s/it]
 99%|█████████▉| 223371/225000 [438:26:07<3:54:28,  8.64s/it]
 99%|█████████▉| 223372/225000 [438:26:16<4:00:01,  8.85s/it]
 99%|█████████▉| 223373/225000 [438:26:23<3:45:44,  8.32s/it]
 99%|█████████▉| 223374/225000 [438:26:32<3:53:15,  8.61s/it]
 99%|█████████▉| 223375/225000 [438:26:41<3:51:42,  8.56s/it]
 99%|█████████▉| 223376/225000 [438:26:53<4:22:30,  9.70s/it]
 99%|█████████▉| 223377/225000 [438:27:00<3:58:35,  8.82s/it]
 99%|█████████▉| 223378/225000 [438:27:10<4:10:16,  9.26s/it]
 99%|█████████▉| 223379/225000 [438:27:19<4:07:30,  9.16s/it]
 99%|█████████▉| 223380/225000 [438:27:26<3:45:51,  8.37s/it]
 99%|█████████▉| 223381/225000 [438:27:33<3:38:22,  8.09s/it]
 99%|█████████▉| 223382/225000 [438:27:39<3:22:37,  7.51s/it]
 99%|█████████▉| 223383/225000 [438:27:49<3:36:12,  8.02s/it]
 99%|█████████▉| 223384/225000 [438:27:55<3:24:04,  7.58s/it]
 99%|█████████▉| 223385/225000 [438:28:04<3:36:28,  8.04s/it]
 99%|█████████▉| 223386/225000 [438:28:18<4:18:31,  9.61s/it]
 99%|█████████▉| 223387/225000 [438:28:26<4:10:22,  9.31s/it]
 99%|█████████▉| 223388/225000 [438:28:32<3:46:15,  8.42s/it]
 99%|█████████▉| 223389/225000 [438:28:43<3:59:33,  8.92s/it]
 99%|█████████▉| 223390/225000 [438:28:55<4:29:05, 10.03s/it]
 99%|█████████▉| 223391/225000 [438:29:05<4:25:17,  9.89s/it]
 99%|█████████▉| 223392/225000 [438:29:13<4:09:53,  9.32s/it]
 99%|█████████▉| 223393/225000 [438:29:20<3:52:33,  8.68s/it]
 99%|█████████▉| 223394/225000 [438:29:29<3:55:41,  8.81s/it]
 99%|█████████▉| 223395/225000 [438:29:40<4:15:38,  9.56s/it]
 99%|█████████▉| 223396/225000 [438:29:45<3:36:49,  8.11s/it]
 99%|█████████▉| 223397/225000 [438:29:55<3:51:23,  8.66s/it]
 99%|█████████▉| 223398/225000 [438:30:02<3:40:20,  8.25s/it]
 99%|█████████▉| 223399/225000 [438:30:13<3:59:19,  8.97s/it]
 99%|█████████▉| 223400/225000 [438:30:23<4:05:20,  9.20s/it]
                                                             
{'loss': '1.327', 'grad_norm': '1.574', 'learning_rate': '7.758e-09', 'epoch': '0.9929', 'num_input_tokens_seen': 748805704, 'train_runtime': '1.579e+06', 'train_tokens_per_second': '474.3'}

 99%|█████████▉| 223400/225000 [438:30:23<4:05:20,  9.20s/it]
 99%|█████████▉| 223401/225000 [438:30:30<3:49:58,  8.63s/it]
 99%|█████████▉| 223402/225000 [438:30:36<3:30:03,  7.89s/it]
 99%|█████████▉| 223403/225000 [438:30:46<3:44:41,  8.44s/it]
 99%|█████████▉| 223404/225000 [438:30:52<3:26:50,  7.78s/it]
 99%|█████████▉| 223405/225000 [438:30:58<3:13:59,  7.30s/it]
 99%|█████████▉| 223406/225000 [438:31:09<3:37:20,  8.18s/it]
 99%|█████████▉| 223407/225000 [438:31:18<3:43:38,  8.42s/it]
 99%|█████████▉| 223408/225000 [438:31:25<3:34:34,  8.09s/it]
 99%|█████████▉| 223409/225000 [438:31:37<4:10:16,  9.44s/it]
 99%|█████████▉| 223410/225000 [438:31:42<3:32:43,  8.03s/it]
 99%|█████████▉| 223411/225000 [438:31:51<3:42:47,  8.41s/it]
 99%|█████████▉| 223412/225000 [438:31:57<3:18:49,  7.51s/it]
 99%|█████████▉| 223413/225000 [438:32:09<3:57:19,  8.97s/it]
 99%|█████████▉| 223414/225000 [438:32:15<3:32:33,  8.04s/it]
 99%|█████████▉| 223415/225000 [438:32:31<4:34:52, 10.41s/it]
 99%|█████████▉| 223416/225000 [438:32:42<4:38:15, 10.54s/it]
 99%|█████████▉| 223417/225000 [438:32:49<4:07:16,  9.37s/it]
 99%|█████████▉| 223418/225000 [438:32:54<3:34:39,  8.14s/it]
 99%|█████████▉| 223419/225000 [438:33:06<4:09:32,  9.47s/it]
 99%|█████████▉| 223420/225000 [438:33:13<3:48:33,  8.68s/it]
 99%|█████████▉| 223421/225000 [438:33:25<4:12:42,  9.60s/it]
 99%|█████████▉| 223422/225000 [438:33:31<3:42:50,  8.47s/it]
 99%|█████████▉| 223423/225000 [438:33:38<3:33:34,  8.13s/it]
 99%|█████████▉| 223424/225000 [438:33:44<3:12:54,  7.34s/it]
 99%|█████████▉| 223425/225000 [438:33:56<3:55:23,  8.97s/it]
 99%|█████████▉| 223426/225000 [438:34:06<3:57:09,  9.04s/it]
 99%|█████████▉| 223427/225000 [438:34:11<3:28:25,  7.95s/it]
 99%|█████████▉| 223428/225000 [438:34:23<4:01:54,  9.23s/it]
 99%|█████████▉| 223429/225000 [438:34:29<3:33:13,  8.14s/it]
 99%|█████████▉| 223430/225000 [438:34:35<3:17:39,  7.55s/it]
 99%|█████████▉| 223431/225000 [438:34:43<3:18:38,  7.60s/it]
 99%|█████████▉| 223432/225000 [438:34:51<3:21:09,  7.70s/it]
 99%|█████████▉| 223433/225000 [438:35:03<4:00:04,  9.19s/it]
 99%|█████████▉| 223434/225000 [438:35:13<4:01:14,  9.24s/it]
 99%|█████████▉| 223435/225000 [438:35:23<4:10:50,  9.62s/it]
 99%|█████████▉| 223436/225000 [438:35:32<4:05:12,  9.41s/it]
 99%|█████████▉| 223437/225000 [438:35:41<4:02:15,  9.30s/it]
 99%|█████████▉| 223438/225000 [438:35:50<3:55:04,  9.03s/it]
 99%|█████████▉| 223439/225000 [438:36:00<4:05:23,  9.43s/it]
 99%|█████████▉| 223440/225000 [438:36:09<3:59:44,  9.22s/it]
 99%|█████████▉| 223441/225000 [438:36:22<4:35:04, 10.59s/it]
 99%|█████████▉| 223442/225000 [438:36:28<3:58:01,  9.17s/it]
 99%|█████████▉| 223443/225000 [438:36:40<4:17:08,  9.91s/it]
 99%|█████████▉| 223444/225000 [438:36:44<3:33:24,  8.23s/it]
 99%|█████████▉| 223445/225000 [438:36:55<3:50:06,  8.88s/it]
 99%|█████████▉| 223446/225000 [438:37:01<3:30:12,  8.12s/it]
 99%|█████████▉| 223447/225000 [438:37:13<3:57:03,  9.16s/it]
 99%|█████████▉| 223448/225000 [438:37:21<3:52:49,  9.00s/it]
 99%|█████████▉| 223449/225000 [438:37:27<3:25:14,  7.94s/it]
 99%|█████████▉| 223450/225000 [438:37:38<3:53:55,  9.06s/it]
                                                             
{'loss': '1.311', 'grad_norm': '1.415', 'learning_rate': '7.31e-09', 'epoch': '0.9931', 'num_input_tokens_seen': 749043384, 'train_runtime': '1.579e+06', 'train_tokens_per_second': '474.4'}

 99%|█████████▉| 223450/225000 [438:37:38<3:53:55,  9.06s/it]
 99%|█████████▉| 223451/225000 [438:37:43<3:17:23,  7.65s/it]
 99%|█████████▉| 223452/225000 [438:37:51<3:24:48,  7.94s/it]
 99%|█████████▉| 223453/225000 [438:38:02<3:49:44,  8.91s/it]
 99%|█████████▉| 223454/225000 [438:38:09<3:33:06,  8.27s/it]
 99%|█████████▉| 223455/225000 [438:38:20<3:52:03,  9.01s/it]
 99%|█████████▉| 223456/225000 [438:38:28<3:47:26,  8.84s/it]
 99%|█████████▉| 223457/225000 [438:38:37<3:42:01,  8.63s/it]
 99%|█████████▉| 223458/225000 [438:38:42<3:20:29,  7.80s/it]
 99%|█████████▉| 223459/225000 [438:38:55<3:54:08,  9.12s/it]
 99%|█████████▉| 223460/225000 [438:39:07<4:15:43,  9.96s/it]
 99%|█████████▉| 223461/225000 [438:39:19<4:33:40, 10.67s/it]
 99%|█████████▉| 223462/225000 [438:39:27<4:15:59,  9.99s/it]
 99%|█████████▉| 223463/225000 [438:39:34<3:50:33,  9.00s/it]
 99%|█████████▉| 223464/225000 [438:39:41<3:32:52,  8.32s/it]
 99%|█████████▉| 223465/225000 [438:39:53<4:05:07,  9.58s/it]
 99%|█████████▉| 223466/225000 [438:40:03<4:05:11,  9.59s/it]
 99%|█████████▉| 223467/225000 [438:40:07<3:26:51,  8.10s/it]
 99%|█████████▉| 223468/225000 [438:40:16<3:30:30,  8.24s/it]
 99%|█████████▉| 223469/225000 [438:40:24<3:25:23,  8.05s/it]
 99%|█████████▉| 223470/225000 [438:40:30<3:08:40,  7.40s/it]
 99%|█████████▉| 223471/225000 [438:40:45<4:07:39,  9.72s/it]
 99%|█████████▉| 223472/225000 [438:40:52<3:51:27,  9.09s/it]
 99%|█████████▉| 223473/225000 [438:41:04<4:08:06,  9.75s/it]
 99%|█████████▉| 223474/225000 [438:41:12<3:56:49,  9.31s/it]
 99%|█████████▉| 223475/225000 [438:41:23<4:13:34,  9.98s/it]
 99%|█████████▉| 223476/225000 [438:41:37<4:38:12, 10.95s/it]
 99%|█████████▉| 223477/225000 [438:41:45<4:20:48, 10.27s/it]
 99%|█████████▉| 223478/225000 [438:41:52<3:57:06,  9.35s/it]
 99%|█████████▉| 223479/225000 [438:42:00<3:39:34,  8.66s/it]
 99%|█████████▉| 223480/225000 [438:42:08<3:35:51,  8.52s/it]
 99%|█████████▉| 223481/225000 [438:42:16<3:33:22,  8.43s/it]
 99%|█████████▉| 223482/225000 [438:42:26<3:49:01,  9.05s/it]
 99%|█████████▉| 223483/225000 [438:42:39<4:18:48, 10.24s/it]
 99%|█████████▉| 223484/225000 [438:42:45<3:41:21,  8.76s/it]
 99%|█████████▉| 223485/225000 [438:42:58<4:17:08, 10.18s/it]
 99%|█████████▉| 223486/225000 [438:43:05<3:48:23,  9.05s/it]
 99%|█████████▉| 223487/225000 [438:43:18<4:21:54, 10.39s/it]
 99%|█████████▉| 223488/225000 [438:43:23<3:38:03,  8.65s/it]
 99%|█████████▉| 223489/225000 [438:43:34<3:54:52,  9.33s/it]
 99%|█████████▉| 223490/225000 [438:43:46<4:20:22, 10.35s/it]
 99%|█████████▉| 223491/225000 [438:43:52<3:40:55,  8.78s/it]
 99%|█████████▉| 223492/225000 [438:44:01<3:46:31,  9.01s/it]
 99%|█████████▉| 223493/225000 [438:44:08<3:32:14,  8.45s/it]
 99%|█████████▉| 223494/225000 [438:44:16<3:25:40,  8.19s/it]
 99%|█████████▉| 223495/225000 [438:44:30<4:07:23,  9.86s/it]
 99%|█████████▉| 223496/225000 [438:44:39<4:06:36,  9.84s/it]
 99%|█████████▉| 223497/225000 [438:44:44<3:27:32,  8.29s/it]
 99%|█████████▉| 223498/225000 [438:44:56<3:51:20,  9.24s/it]
 99%|█████████▉| 223499/225000 [438:45:03<3:36:25,  8.65s/it]
 99%|█████████▉| 223500/225000 [438:45:10<3:25:41,  8.23s/it]
                                                             
{'loss': '1.33', 'grad_norm': '1.865', 'learning_rate': '6.875e-09', 'epoch': '0.9933', 'num_input_tokens_seen': 749287296, 'train_runtime': '1.58e+06', 'train_tokens_per_second': '474.4'}

 99%|█████████▉| 223500/225000 [438:45:10<3:25:41,  8.23s/it]
 99%|█████████▉| 223501/225000 [438:45:23<4:01:57,  9.69s/it]
 99%|█████████▉| 223502/225000 [438:45:29<3:33:16,  8.54s/it]
 99%|█████████▉| 223503/225000 [438:45:47<4:42:37, 11.33s/it]
 99%|█████████▉| 223504/225000 [438:45:55<4:17:56, 10.35s/it]
 99%|█████████▉| 223505/225000 [438:46:00<3:40:42,  8.86s/it]
 99%|█████████▉| 223506/225000 [438:46:14<4:20:27, 10.46s/it]
 99%|█████████▉| 223507/225000 [438:46:21<3:47:33,  9.14s/it]
 99%|█████████▉| 223508/225000 [438:46:27<3:27:50,  8.36s/it]
 99%|█████████▉| 223509/225000 [438:46:35<3:26:30,  8.31s/it]
 99%|█████████▉| 223510/225000 [438:46:41<3:07:05,  7.53s/it]
 99%|█████████▉| 223511/225000 [438:46:56<3:59:11,  9.64s/it]
 99%|█████████▉| 223512/225000 [438:47:01<3:26:44,  8.34s/it]
 99%|█████████▉| 223513/225000 [438:47:14<4:04:45,  9.88s/it]
 99%|█████████▉| 223514/225000 [438:47:20<3:35:38,  8.71s/it]
 99%|█████████▉| 223515/225000 [438:47:32<3:54:49,  9.49s/it]
 99%|█████████▉| 223516/225000 [438:47:42<3:58:15,  9.63s/it]
 99%|█████████▉| 223517/225000 [438:47:47<3:29:03,  8.46s/it]
 99%|█████████▉| 223518/225000 [438:47:56<3:29:15,  8.47s/it]
 99%|█████████▉| 223519/225000 [438:48:03<3:22:03,  8.19s/it]
 99%|█████████▉| 223520/225000 [438:48:11<3:18:34,  8.05s/it]
 99%|█████████▉| 223521/225000 [438:48:26<4:10:06, 10.15s/it]
 99%|█████████▉| 223522/225000 [438:48:33<3:49:27,  9.32s/it]
 99%|█████████▉| 223523/225000 [438:48:47<4:19:54, 10.56s/it]
 99%|█████████▉| 223524/225000 [438:48:57<4:14:59, 10.37s/it]
 99%|█████████▉| 223525/225000 [438:49:07<4:16:56, 10.45s/it]
 99%|█████████▉| 223526/225000 [438:49:17<4:08:08, 10.10s/it]
 99%|█████████▉| 223527/225000 [438:49:24<3:50:02,  9.37s/it]
 99%|█████████▉| 223528/225000 [438:49:32<3:35:40,  8.79s/it]
 99%|█████████▉| 223529/225000 [438:49:38<3:15:32,  7.98s/it]
 99%|█████████▉| 223530/225000 [438:49:47<3:25:41,  8.40s/it]
 99%|█████████▉| 223531/225000 [438:50:02<4:09:38, 10.20s/it]
 99%|█████████▉| 223532/225000 [438:50:09<3:45:45,  9.23s/it]
 99%|█████████▉| 223533/225000 [438:50:19<3:52:46,  9.52s/it]
 99%|█████████▉| 223534/225000 [438:50:25<3:30:25,  8.61s/it]
 99%|█████████▉| 223535/225000 [438:50:34<3:29:58,  8.60s/it]
 99%|█████████▉| 223536/225000 [438:50:43<3:35:29,  8.83s/it]
 99%|█████████▉| 223537/225000 [438:50:51<3:24:28,  8.39s/it]
 99%|█████████▉| 223538/225000 [438:51:01<3:37:50,  8.94s/it]
 99%|█████████▉| 223539/225000 [438:51:12<3:50:45,  9.48s/it]
 99%|█████████▉| 223540/225000 [438:51:18<3:28:42,  8.58s/it]
 99%|█████████▉| 223541/225000 [438:51:26<3:20:29,  8.24s/it]
 99%|█████████▉| 223542/225000 [438:51:31<2:58:20,  7.34s/it]
 99%|█████████▉| 223543/225000 [438:51:39<3:02:17,  7.51s/it]
 99%|█████████▉| 223544/225000 [438:51:53<3:54:20,  9.66s/it]
 99%|█████████▉| 223545/225000 [438:52:06<4:16:47, 10.59s/it]
 99%|█████████▉| 223546/225000 [438:52:14<3:56:00,  9.74s/it]
 99%|█████████▉| 223547/225000 [438:52:19<3:24:41,  8.45s/it]
 99%|█████████▉| 223548/225000 [438:52:31<3:48:38,  9.45s/it]
 99%|█████████▉| 223549/225000 [438:52:39<3:34:03,  8.85s/it]
 99%|█████████▉| 223550/225000 [438:52:46<3:22:48,  8.39s/it]
                                                             
{'loss': '1.332', 'grad_norm': '1.425', 'learning_rate': '6.453e-09', 'epoch': '0.9936', 'num_input_tokens_seen': 749534768, 'train_runtime': '1.58e+06', 'train_tokens_per_second': '474.4'}

 99%|█████████▉| 223550/225000 [438:52:46<3:22:48,  8.39s/it]
 99%|█████████▉| 223551/225000 [438:53:02<4:21:17, 10.82s/it]
 99%|█████████▉| 223552/225000 [438:53:09<3:50:13,  9.54s/it]
 99%|█████████▉| 223553/225000 [438:53:19<3:50:46,  9.57s/it]
 99%|█████████▉| 223554/225000 [438:53:26<3:33:49,  8.87s/it]
 99%|█████████▉| 223555/225000 [438:53:34<3:25:32,  8.53s/it]
 99%|█████████▉| 223556/225000 [438:53:40<3:10:25,  7.91s/it]
 99%|█████████▉| 223557/225000 [438:53:48<3:12:38,  8.01s/it]
 99%|█████████▉| 223558/225000 [438:54:02<3:53:49,  9.73s/it]
 99%|█████████▉| 223559/225000 [438:54:06<3:15:10,  8.13s/it]
 99%|█████████▉| 223560/225000 [438:54:15<3:21:52,  8.41s/it]
 99%|█████████▉| 223561/225000 [438:54:24<3:21:01,  8.38s/it]
 99%|█████████▉| 223562/225000 [438:54:29<3:01:49,  7.59s/it]
 99%|█████████▉| 223563/225000 [438:54:40<3:26:12,  8.61s/it]
 99%|█████████▉| 223564/225000 [438:54:50<3:31:06,  8.82s/it]
 99%|█████████▉| 223565/225000 [438:55:02<3:56:20,  9.88s/it]
 99%|█████████▉| 223566/225000 [438:55:08<3:23:44,  8.52s/it]
 99%|█████████▉| 223567/225000 [438:55:22<4:05:41, 10.29s/it]
 99%|█████████▉| 223568/225000 [438:55:28<3:32:47,  8.92s/it]
 99%|█████████▉| 223569/225000 [438:55:38<3:45:10,  9.44s/it]
 99%|█████████▉| 223570/225000 [438:55:43<3:10:55,  8.01s/it]
 99%|█████████▉| 223571/225000 [438:55:53<3:23:51,  8.56s/it]
 99%|█████████▉| 223572/225000 [438:56:04<3:43:18,  9.38s/it]
 99%|█████████▉| 223573/225000 [438:56:14<3:43:22,  9.39s/it]
 99%|█████████▉| 223574/225000 [438:56:20<3:25:43,  8.66s/it]
 99%|█████████▉| 223575/225000 [438:56:31<3:37:05,  9.14s/it]
 99%|█████████▉| 223576/225000 [438:56:42<3:50:52,  9.73s/it]
 99%|█████████▉| 223577/225000 [438:56:50<3:41:09,  9.32s/it]
 99%|█████████▉| 223578/225000 [438:56:59<3:36:20,  9.13s/it]
 99%|█████████▉| 223579/225000 [438:57:10<3:48:03,  9.63s/it]
 99%|█████████▉| 223580/225000 [438:57:19<3:49:00,  9.68s/it]
 99%|█████████▉| 223581/225000 [438:57:28<3:43:43,  9.46s/it]
 99%|█████████▉| 223582/225000 [438:57:36<3:33:30,  9.03s/it]
 99%|█████████▉| 223583/225000 [438:57:42<3:06:35,  7.90s/it]
 99%|█████████▉| 223584/225000 [438:57:51<3:14:56,  8.26s/it]
 99%|█████████▉| 223585/225000 [438:58:07<4:11:52, 10.68s/it]
 99%|█████████▉| 223586/225000 [438:58:15<3:50:07,  9.77s/it]
 99%|█████████▉| 223587/225000 [438:58:22<3:34:01,  9.09s/it]
 99%|█████████▉| 223588/225000 [438:58:32<3:37:29,  9.24s/it]
 99%|█████████▉| 223589/225000 [438:58:37<3:09:47,  8.07s/it]
 99%|█████████▉| 223590/225000 [438:58:50<3:40:41,  9.39s/it]
 99%|█████████▉| 223591/225000 [438:58:57<3:23:09,  8.65s/it]
 99%|█████████▉| 223592/225000 [438:59:09<3:46:55,  9.67s/it]
 99%|█████████▉| 223593/225000 [438:59:15<3:24:16,  8.71s/it]
 99%|█████████▉| 223594/225000 [438:59:24<3:23:03,  8.67s/it]
 99%|█████████▉| 223595/225000 [438:59:29<3:01:28,  7.75s/it]
 99%|█████████▉| 223596/225000 [438:59:39<3:17:50,  8.45s/it]
 99%|█████████▉| 223597/225000 [438:59:48<3:15:17,  8.35s/it]
 99%|█████████▉| 223598/225000 [439:00:02<3:57:22, 10.16s/it]
 99%|█████████▉| 223599/225000 [439:00:10<3:43:10,  9.56s/it]
 99%|█████████▉| 223600/225000 [439:00:17<3:27:48,  8.91s/it]
                                                             
{'loss': '1.295', 'grad_norm': '1.114', 'learning_rate': '6.045e-09', 'epoch': '0.9938', 'num_input_tokens_seen': 749776176, 'train_runtime': '1.58e+06', 'train_tokens_per_second': '474.4'}

 99%|█████████▉| 223600/225000 [439:00:17<3:27:48,  8.91s/it]
 99%|█████████▉| 223601/225000 [439:00:25<3:19:56,  8.57s/it]
 99%|█████████▉| 223602/225000 [439:00:35<3:25:08,  8.80s/it]
 99%|█████████▉| 223603/225000 [439:00:44<3:30:27,  9.04s/it]
 99%|█████████▉| 223604/225000 [439:00:58<4:05:04, 10.53s/it]
 99%|█████████▉| 223605/225000 [439:01:03<3:26:55,  8.90s/it]
 99%|█████████▉| 223606/225000 [439:01:10<3:09:41,  8.16s/it]
 99%|█████████▉| 223607/225000 [439:01:16<2:56:10,  7.59s/it]
 99%|█████████▉| 223608/225000 [439:01:29<3:35:27,  9.29s/it]
 99%|█████████▉| 223609/225000 [439:01:35<3:14:01,  8.37s/it]
 99%|█████████▉| 223610/225000 [439:01:48<3:43:38,  9.65s/it]
 99%|█████████▉| 223611/225000 [439:01:55<3:23:35,  8.79s/it]
 99%|█████████▉| 223612/225000 [439:02:03<3:21:38,  8.72s/it]
 99%|█████████▉| 223613/225000 [439:02:13<3:27:07,  8.96s/it]
 99%|█████████▉| 223614/225000 [439:02:25<3:48:33,  9.89s/it]
 99%|█████████▉| 223615/225000 [439:02:30<3:16:15,  8.50s/it]
 99%|█████████▉| 223616/225000 [439:02:36<2:54:22,  7.56s/it]
 99%|█████████▉| 223617/225000 [439:02:44<3:02:53,  7.93s/it]
 99%|█████████▉| 223618/225000 [439:02:54<3:13:33,  8.40s/it]
 99%|█████████▉| 223619/225000 [439:03:02<3:09:52,  8.25s/it]
 99%|█████████▉| 223620/225000 [439:03:11<3:18:31,  8.63s/it]
 99%|█████████▉| 223621/225000 [439:03:18<3:05:22,  8.07s/it]
 99%|█████████▉| 223622/225000 [439:03:26<3:03:58,  8.01s/it]
 99%|█████████▉| 223623/225000 [439:03:39<3:41:26,  9.65s/it]
 99%|█████████▉| 223624/225000 [439:03:51<3:55:24, 10.26s/it]
 99%|█████████▉| 223625/225000 [439:03:59<3:35:50,  9.42s/it]
 99%|█████████▉| 223626/225000 [439:04:03<2:58:19,  7.79s/it]
 99%|█████████▉| 223627/225000 [439:04:11<3:03:13,  8.01s/it]
 99%|█████████▉| 223628/225000 [439:04:18<2:58:51,  7.82s/it]
 99%|█████████▉| 223629/225000 [439:04:24<2:43:10,  7.14s/it]
 99%|█████████▉| 223630/225000 [439:04:30<2:36:03,  6.83s/it]
 99%|█████████▉| 223631/225000 [439:04:41<3:00:42,  7.92s/it]
 99%|█████████▉| 223632/225000 [439:04:52<3:27:34,  9.10s/it]
 99%|█████████▉| 223633/225000 [439:05:03<3:36:31,  9.50s/it]
 99%|█████████▉| 223634/225000 [439:05:10<3:22:45,  8.91s/it]
 99%|█████████▉| 223635/225000 [439:05:15<2:55:12,  7.70s/it]
 99%|█████████▉| 223636/225000 [439:05:30<3:42:11,  9.77s/it]
 99%|█████████▉| 223637/225000 [439:05:35<3:11:21,  8.42s/it]
 99%|█████████▉| 223638/225000 [439:05:48<3:40:21,  9.71s/it]
 99%|█████████▉| 223639/225000 [439:05:54<3:17:16,  8.70s/it]
 99%|█████████▉| 223640/225000 [439:06:01<3:03:02,  8.08s/it]
 99%|█████████▉| 223641/225000 [439:06:13<3:32:16,  9.37s/it]
 99%|█████████▉| 223642/225000 [439:06:20<3:10:53,  8.43s/it]
 99%|█████████▉| 223643/225000 [439:06:31<3:33:52,  9.46s/it]
 99%|█████████▉| 223644/225000 [439:06:41<3:36:36,  9.58s/it]
 99%|█████████▉| 223645/225000 [439:06:50<3:27:34,  9.19s/it]
 99%|█████████▉| 223646/225000 [439:06:58<3:23:16,  9.01s/it]
 99%|█████████▉| 223647/225000 [439:07:07<3:21:13,  8.92s/it]
 99%|█████████▉| 223648/225000 [439:07:16<3:26:01,  9.14s/it]
 99%|█████████▉| 223649/225000 [439:07:28<3:42:41,  9.89s/it]
 99%|█████████▉| 223650/225000 [439:07:35<3:21:45,  8.97s/it]
                                                             
{'loss': '1.304', 'grad_norm': '1.781', 'learning_rate': '5.65e-09', 'epoch': '0.994', 'num_input_tokens_seen': 750013888, 'train_runtime': '1.581e+06', 'train_tokens_per_second': '474.4'}

 99%|█████████▉| 223650/225000 [439:07:35<3:21:45,  8.97s/it]
 99%|█████████▉| 223651/225000 [439:07:47<3:42:57,  9.92s/it]
 99%|█████████▉| 223652/225000 [439:07:59<3:59:31, 10.66s/it]
 99%|█████████▉| 223653/225000 [439:08:08<3:47:34, 10.14s/it]
 99%|█████████▉| 223654/225000 [439:08:15<3:25:30,  9.16s/it]
 99%|█████████▉| 223655/225000 [439:08:20<2:55:48,  7.84s/it]
 99%|█████████▉| 223656/225000 [439:08:28<2:58:55,  7.99s/it]
 99%|█████████▉| 223657/225000 [439:08:38<3:11:03,  8.54s/it]
 99%|█████████▉| 223658/225000 [439:08:46<3:06:05,  8.32s/it]
 99%|█████████▉| 223659/225000 [439:08:55<3:07:32,  8.39s/it]
 99%|█████████▉| 223660/225000 [439:09:00<2:50:51,  7.65s/it]
 99%|█████████▉| 223661/225000 [439:09:11<3:07:56,  8.42s/it]
 99%|█████████▉| 223662/225000 [439:09:21<3:20:08,  8.97s/it]
 99%|█████████▉| 223663/225000 [439:09:29<3:13:51,  8.70s/it]
 99%|█████████▉| 223664/225000 [439:09:36<2:59:45,  8.07s/it]
 99%|█████████▉| 223665/225000 [439:09:43<2:53:34,  7.80s/it]
 99%|█████████▉| 223666/225000 [439:09:56<3:32:25,  9.55s/it]
 99%|█████████▉| 223667/225000 [439:10:03<3:10:35,  8.58s/it]
 99%|█████████▉| 223668/225000 [439:10:14<3:30:35,  9.49s/it]
 99%|█████████▉| 223669/225000 [439:10:25<3:35:53,  9.73s/it]
 99%|█████████▉| 223670/225000 [439:10:30<3:06:33,  8.42s/it]
 99%|█████████▉| 223671/225000 [439:10:37<2:53:57,  7.85s/it]
 99%|█████████▉| 223672/225000 [439:10:49<3:26:03,  9.31s/it]
 99%|█████████▉| 223673/225000 [439:11:00<3:37:37,  9.84s/it]
 99%|█████████▉| 223674/225000 [439:11:10<3:39:02,  9.91s/it]
 99%|█████████▉| 223675/225000 [439:11:26<4:16:07, 11.60s/it]
 99%|█████████▉| 223676/225000 [439:11:36<4:03:04, 11.02s/it]
 99%|█████████▉| 223677/225000 [439:11:42<3:30:34,  9.55s/it]
 99%|█████████▉| 223678/225000 [439:11:53<3:42:01, 10.08s/it]
 99%|█████████▉| 223679/225000 [439:12:01<3:27:11,  9.41s/it]
 99%|█████████▉| 223680/225000 [439:12:08<3:09:28,  8.61s/it]
 99%|█████████▉| 223681/225000 [439:12:19<3:26:55,  9.41s/it]
 99%|█████████▉| 223682/225000 [439:12:25<3:07:21,  8.53s/it]
 99%|█████████▉| 223683/225000 [439:12:35<3:13:37,  8.82s/it]
 99%|█████████▉| 223684/225000 [439:12:46<3:26:26,  9.41s/it]
 99%|█████████▉| 223685/225000 [439:12:52<3:07:29,  8.56s/it]
 99%|█████████▉| 223686/225000 [439:13:01<3:09:50,  8.67s/it]
 99%|█████████▉| 223687/225000 [439:13:11<3:18:43,  9.08s/it]
 99%|█████████▉| 223688/225000 [439:13:18<3:03:22,  8.39s/it]
 99%|█████████▉| 223689/225000 [439:13:28<3:13:25,  8.85s/it]
 99%|█████████▉| 223690/225000 [439:13:39<3:25:38,  9.42s/it]
 99%|█████████▉| 223691/225000 [439:13:45<3:04:58,  8.48s/it]
 99%|█████████▉| 223692/225000 [439:13:54<3:06:33,  8.56s/it]
 99%|█████████▉| 223693/225000 [439:14:01<2:56:36,  8.11s/it]
 99%|█████████▉| 223694/225000 [439:14:14<3:32:39,  9.77s/it]
 99%|█████████▉| 223695/225000 [439:14:21<3:10:17,  8.75s/it]
 99%|█████████▉| 223696/225000 [439:14:30<3:11:33,  8.81s/it]
 99%|█████████▉| 223697/225000 [439:14:38<3:10:48,  8.79s/it]
 99%|█████████▉| 223698/225000 [439:14:48<3:15:03,  8.99s/it]
 99%|█████████▉| 223699/225000 [439:14:57<3:14:28,  8.97s/it]
 99%|█████████▉| 223700/225000 [439:15:03<2:59:14,  8.27s/it]
                                                             
{'loss': '1.335', 'grad_norm': '1.487', 'learning_rate': '5.268e-09', 'epoch': '0.9942', 'num_input_tokens_seen': 750257888, 'train_runtime': '1.581e+06', 'train_tokens_per_second': '474.5'}

 99%|█████████▉| 223700/225000 [439:15:03<2:59:14,  8.27s/it]
 99%|█████████▉| 223701/225000 [439:15:14<3:15:24,  9.03s/it]
 99%|█████████▉| 223702/225000 [439:15:21<3:02:38,  8.44s/it]
 99%|█████████▉| 223703/225000 [439:15:29<2:57:40,  8.22s/it]
 99%|█████████▉| 223704/225000 [439:15:45<3:46:17, 10.48s/it]
 99%|█████████▉| 223705/225000 [439:15:51<3:16:22,  9.10s/it]
 99%|█████████▉| 223706/225000 [439:15:57<2:55:34,  8.14s/it]
 99%|█████████▉| 223707/225000 [439:16:09<3:22:15,  9.39s/it]
 99%|█████████▉| 223708/225000 [439:16:17<3:16:40,  9.13s/it]
 99%|█████████▉| 223709/225000 [439:16:24<2:57:22,  8.24s/it]
 99%|█████████▉| 223710/225000 [439:16:35<3:17:44,  9.20s/it]
 99%|█████████▉| 223711/225000 [439:16:47<3:33:12,  9.92s/it]
 99%|█████████▉| 223712/225000 [439:16:53<3:09:54,  8.85s/it]
 99%|█████████▉| 223713/225000 [439:17:01<3:03:17,  8.55s/it]
 99%|█████████▉| 223714/225000 [439:17:10<3:05:29,  8.65s/it]
 99%|█████████▉| 223715/225000 [439:17:20<3:15:53,  9.15s/it]
 99%|█████████▉| 223716/225000 [439:17:25<2:46:05,  7.76s/it]
 99%|█████████▉| 223717/225000 [439:17:32<2:45:53,  7.76s/it]
 99%|█████████▉| 223718/225000 [439:17:43<3:03:51,  8.60s/it]
 99%|█████████▉| 223719/225000 [439:17:53<3:12:00,  8.99s/it]
 99%|█████████▉| 223720/225000 [439:17:59<2:55:13,  8.21s/it]
 99%|█████████▉| 223721/225000 [439:18:10<3:11:05,  8.96s/it]
 99%|█████████▉| 223722/225000 [439:18:19<3:13:13,  9.07s/it]
 99%|█████████▉| 223723/225000 [439:18:27<3:04:29,  8.67s/it]
 99%|█████████▉| 223724/225000 [439:18:34<2:51:47,  8.08s/it]
 99%|█████████▉| 223725/225000 [439:18:43<3:02:31,  8.59s/it]
 99%|█████████▉| 223726/225000 [439:18:50<2:50:36,  8.04s/it]
 99%|█████████▉| 223727/225000 [439:18:57<2:42:32,  7.66s/it]
 99%|█████████▉| 223728/225000 [439:19:04<2:36:42,  7.39s/it]
 99%|█████████▉| 223729/225000 [439:19:17<3:14:45,  9.19s/it]
 99%|█████████▉| 223730/225000 [439:19:28<3:25:14,  9.70s/it]
 99%|█████████▉| 223731/225000 [439:19:40<3:39:30, 10.38s/it]
 99%|█████████▉| 223732/225000 [439:19:46<3:11:25,  9.06s/it]
 99%|█████████▉| 223733/225000 [439:19:55<3:10:23,  9.02s/it]
 99%|█████████▉| 223734/225000 [439:20:01<2:50:06,  8.06s/it]
 99%|█████████▉| 223735/225000 [439:20:07<2:41:05,  7.64s/it]
 99%|█████████▉| 223736/225000 [439:20:19<3:04:35,  8.76s/it]
 99%|█████████▉| 223737/225000 [439:20:32<3:31:16, 10.04s/it]
 99%|█████████▉| 223738/225000 [439:20:40<3:19:24,  9.48s/it]
 99%|█████████▉| 223739/225000 [439:20:47<3:02:09,  8.67s/it]
 99%|█████████▉| 223740/225000 [439:20:58<3:18:39,  9.46s/it]
 99%|█████████▉| 223741/225000 [439:21:05<3:06:23,  8.88s/it]
 99%|█████████▉| 223742/225000 [439:21:18<3:28:01,  9.92s/it]
 99%|█████████▉| 223743/225000 [439:21:24<3:05:07,  8.84s/it]
 99%|█████████▉| 223744/225000 [439:21:30<2:44:07,  7.84s/it]
 99%|█████████▉| 223745/225000 [439:21:39<2:56:15,  8.43s/it]
 99%|█████████▉| 223746/225000 [439:21:47<2:51:42,  8.22s/it]
 99%|█████████▉| 223747/225000 [439:21:53<2:33:55,  7.37s/it]
 99%|█████████▉| 223748/225000 [439:22:07<3:17:30,  9.47s/it]
 99%|█████████▉| 223749/225000 [439:22:17<3:22:39,  9.72s/it]
 99%|█████████▉| 223750/225000 [439:22:29<3:34:23, 10.29s/it]
                                                             
{'loss': '1.299', 'grad_norm': '1.146', 'learning_rate': '4.9e-09', 'epoch': '0.9944', 'num_input_tokens_seen': 750494752, 'train_runtime': '1.582e+06', 'train_tokens_per_second': '474.5'}

 99%|█████████▉| 223750/225000 [439:22:29<3:34:23, 10.29s/it]
 99%|█████████▉| 223751/225000 [439:22:36<3:17:37,  9.49s/it]
 99%|█████████▉| 223752/225000 [439:22:43<2:59:04,  8.61s/it]
 99%|█████████▉| 223753/225000 [439:22:51<2:52:30,  8.30s/it]
 99%|█████████▉| 223754/225000 [439:22:59<2:50:34,  8.21s/it]
 99%|█████████▉| 223755/225000 [439:23:13<3:31:06, 10.17s/it]
 99%|█████████▉| 223756/225000 [439:23:21<3:12:39,  9.29s/it]
 99%|█████████▉| 223757/225000 [439:23:29<3:08:34,  9.10s/it]
 99%|█████████▉| 223758/225000 [439:23:39<3:11:46,  9.26s/it]
 99%|█████████▉| 223759/225000 [439:23:49<3:19:15,  9.63s/it]
 99%|█████████▉| 223760/225000 [439:23:57<3:09:09,  9.15s/it]
 99%|█████████▉| 223761/225000 [439:24:08<3:17:48,  9.58s/it]
 99%|█████████▉| 223762/225000 [439:24:20<3:31:52, 10.27s/it]
 99%|█████████▉| 223763/225000 [439:24:27<3:10:36,  9.25s/it]
 99%|█████████▉| 223764/225000 [439:24:38<3:20:27,  9.73s/it]
 99%|█████████▉| 223765/225000 [439:24:46<3:09:50,  9.22s/it]
 99%|█████████▉| 223766/225000 [439:24:52<2:54:55,  8.51s/it]
 99%|█████████▉| 223767/225000 [439:25:02<2:59:04,  8.71s/it]
 99%|█████████▉| 223768/225000 [439:25:12<3:05:53,  9.05s/it]
 99%|█████████▉| 223769/225000 [439:25:21<3:07:25,  9.13s/it]
 99%|█████████▉| 223770/225000 [439:25:27<2:45:54,  8.09s/it]
 99%|█████████▉| 223771/225000 [439:25:36<2:56:55,  8.64s/it]
 99%|█████████▉| 223772/225000 [439:25:45<2:57:39,  8.68s/it]
 99%|█████████▉| 223773/225000 [439:25:55<3:05:21,  9.06s/it]
 99%|█████████▉| 223774/225000 [439:26:01<2:44:37,  8.06s/it]
 99%|█████████▉| 223775/225000 [439:26:12<3:04:25,  9.03s/it]
 99%|█████████▉| 223776/225000 [439:26:21<3:01:20,  8.89s/it]
 99%|█████████▉| 223777/225000 [439:26:26<2:40:25,  7.87s/it]
 99%|█████████▉| 223778/225000 [439:26:35<2:48:13,  8.26s/it]
 99%|█████████▉| 223779/225000 [439:26:44<2:47:21,  8.22s/it]
 99%|█████████▉| 223780/225000 [439:26:49<2:32:31,  7.50s/it]
 99%|█████████▉| 223781/225000 [439:26:55<2:23:09,  7.05s/it]
 99%|█████████▉| 223782/225000 [439:27:12<3:22:53,  9.99s/it]
 99%|█████████▉| 223783/225000 [439:27:18<2:58:48,  8.82s/it]
 99%|█████████▉| 223784/225000 [439:27:30<3:15:03,  9.62s/it]
 99%|█████████▉| 223785/225000 [439:27:39<3:14:07,  9.59s/it]
 99%|█████████▉| 223786/225000 [439:27:52<3:33:17, 10.54s/it]
 99%|█████████▉| 223787/225000 [439:27:58<3:04:21,  9.12s/it]
 99%|█████████▉| 223788/225000 [439:28:05<2:49:25,  8.39s/it]
 99%|█████████▉| 223789/225000 [439:28:15<2:59:29,  8.89s/it]
 99%|█████████▉| 223790/225000 [439:28:24<2:59:26,  8.90s/it]
 99%|█████████▉| 223791/225000 [439:28:36<3:22:21, 10.04s/it]
 99%|█████████▉| 223792/225000 [439:28:46<3:21:40, 10.02s/it]
 99%|█████████▉| 223793/225000 [439:28:55<3:13:25,  9.61s/it]
 99%|█████████▉| 223794/225000 [439:29:01<2:52:41,  8.59s/it]
 99%|█████████▉| 223795/225000 [439:29:13<3:10:27,  9.48s/it]
 99%|█████████▉| 223796/225000 [439:29:20<2:56:13,  8.78s/it]
 99%|█████████▉| 223797/225000 [439:29:36<3:39:37, 10.95s/it]
 99%|█████████▉| 223798/225000 [439:29:44<3:20:23, 10.00s/it]
 99%|█████████▉| 223799/225000 [439:29:52<3:09:05,  9.45s/it]
 99%|█████████▉| 223800/225000 [439:30:02<3:15:16,  9.76s/it]
                                                             
{'loss': '1.275', 'grad_norm': '1.563', 'learning_rate': '4.545e-09', 'epoch': '0.9947', 'num_input_tokens_seen': 750734912, 'train_runtime': '1.582e+06', 'train_tokens_per_second': '474.5'}

 99%|█████████▉| 223800/225000 [439:30:02<3:15:16,  9.76s/it]
 99%|█████████▉| 223801/225000 [439:30:10<3:00:57,  9.06s/it]
 99%|█████████▉| 223802/225000 [439:30:20<3:09:39,  9.50s/it]
 99%|█████████▉| 223803/225000 [439:30:27<2:52:45,  8.66s/it]
 99%|█████████▉| 223804/225000 [439:30:32<2:34:19,  7.74s/it]
 99%|█████████▉| 223805/225000 [439:30:39<2:25:43,  7.32s/it]
 99%|█████████▉| 223806/225000 [439:30:50<2:48:35,  8.47s/it]
 99%|█████████▉| 223807/225000 [439:31:00<2:59:24,  9.02s/it]
 99%|█████████▉| 223808/225000 [439:31:10<3:01:50,  9.15s/it]
 99%|█████████▉| 223809/225000 [439:31:17<2:52:53,  8.71s/it]
 99%|█████████▉| 223810/225000 [439:31:28<3:06:49,  9.42s/it]
 99%|█████████▉| 223811/225000 [439:31:37<3:02:31,  9.21s/it]
 99%|█████████▉| 223812/225000 [439:31:45<2:53:13,  8.75s/it]
 99%|█████████▉| 223813/225000 [439:31:54<2:54:28,  8.82s/it]
 99%|█████████▉| 223814/225000 [439:32:05<3:05:47,  9.40s/it]
 99%|█████████▉| 223815/225000 [439:32:14<3:06:50,  9.46s/it]
 99%|█████████▉| 223816/225000 [439:32:22<2:54:29,  8.84s/it]
 99%|█████████▉| 223817/225000 [439:32:27<2:35:01,  7.86s/it]
 99%|█████████▉| 223818/225000 [439:32:37<2:46:58,  8.48s/it]
 99%|█████████▉| 223819/225000 [439:32:53<3:29:27, 10.64s/it]
 99%|█████████▉| 223820/225000 [439:33:05<3:36:28, 11.01s/it]
 99%|█████████▉| 223821/225000 [439:33:10<3:00:07,  9.17s/it]
 99%|█████████▉| 223822/225000 [439:33:22<3:20:36, 10.22s/it]
 99%|█████████▉| 223823/225000 [439:33:30<3:06:20,  9.50s/it]
 99%|█████████▉| 223824/225000 [439:33:37<2:50:11,  8.68s/it]
 99%|█████████▉| 223825/225000 [439:33:44<2:39:55,  8.17s/it]
 99%|█████████▉| 223826/225000 [439:33:50<2:29:56,  7.66s/it]
 99%|█████████▉| 223827/225000 [439:33:59<2:37:24,  8.05s/it]
 99%|█████████▉| 223828/225000 [439:34:05<2:25:42,  7.46s/it]
 99%|█████████▉| 223829/225000 [439:34:13<2:26:18,  7.50s/it]
 99%|█████████▉| 223830/225000 [439:34:20<2:24:06,  7.39s/it]
 99%|█████████▉| 223831/225000 [439:34:35<3:07:29,  9.62s/it]
 99%|█████████▉| 223832/225000 [439:34:44<3:04:23,  9.47s/it]
 99%|█████████▉| 223833/225000 [439:34:55<3:15:46, 10.07s/it]
 99%|█████████▉| 223834/225000 [439:35:08<3:28:41, 10.74s/it]
 99%|█████████▉| 223835/225000 [439:35:13<2:55:05,  9.02s/it]
 99%|█████████▉| 223836/225000 [439:35:20<2:45:35,  8.54s/it]
 99%|█████████▉| 223837/225000 [439:35:34<3:14:33, 10.04s/it]
 99%|█████████▉| 223838/225000 [439:35:43<3:11:03,  9.87s/it]
 99%|█████████▉| 223839/225000 [439:35:51<2:59:25,  9.27s/it]
 99%|█████████▉| 223840/225000 [439:36:00<2:57:00,  9.16s/it]
 99%|█████████▉| 223841/225000 [439:36:11<3:05:42,  9.61s/it]
 99%|█████████▉| 223842/225000 [439:36:17<2:47:16,  8.67s/it]
 99%|█████████▉| 223843/225000 [439:36:27<2:52:01,  8.92s/it]
 99%|█████████▉| 223844/225000 [439:36:33<2:39:15,  8.27s/it]
 99%|█████████▉| 223845/225000 [439:36:44<2:51:26,  8.91s/it]
 99%|█████████▉| 223846/225000 [439:36:52<2:49:10,  8.80s/it]
 99%|█████████▉| 223847/225000 [439:37:01<2:47:31,  8.72s/it]
 99%|█████████▉| 223848/225000 [439:37:10<2:48:37,  8.78s/it]
 99%|█████████▉| 223849/225000 [439:37:18<2:44:34,  8.58s/it]
 99%|█████████▉| 223850/225000 [439:37:26<2:43:55,  8.55s/it]
                                                             
{'loss': '1.307', 'grad_norm': '2.725', 'learning_rate': '4.203e-09', 'epoch': '0.9949', 'num_input_tokens_seen': 750971112, 'train_runtime': '1.583e+06', 'train_tokens_per_second': '474.5'}

 99%|█████████▉| 223850/225000 [439:37:26<2:43:55,  8.55s/it]
 99%|█████████▉| 223851/225000 [439:37:33<2:32:16,  7.95s/it]
 99%|█████████▉| 223852/225000 [439:37:44<2:53:04,  9.05s/it]
 99%|█████████▉| 223853/225000 [439:37:50<2:35:32,  8.14s/it]
 99%|█████████▉| 223854/225000 [439:38:00<2:43:24,  8.56s/it]
 99%|█████████▉| 223855/225000 [439:38:11<2:59:24,  9.40s/it]
 99%|█████████▉| 223856/225000 [439:38:21<3:01:25,  9.52s/it]
 99%|█████████▉| 223857/225000 [439:38:31<3:00:39,  9.48s/it]
 99%|█████████▉| 223858/225000 [439:38:39<2:53:59,  9.14s/it]
 99%|█████████▉| 223859/225000 [439:38:46<2:40:12,  8.42s/it]
 99%|█████████▉| 223860/225000 [439:38:57<2:54:46,  9.20s/it]
 99%|█████████▉| 223861/225000 [439:39:09<3:11:14, 10.07s/it]
 99%|█████████▉| 223862/225000 [439:39:19<3:13:29, 10.20s/it]
 99%|█████████▉| 223863/225000 [439:39:26<2:51:11,  9.03s/it]
 99%|█████████▉| 223864/225000 [439:39:37<3:06:36,  9.86s/it]
 99%|█████████▉| 223865/225000 [439:39:44<2:46:15,  8.79s/it]
 99%|█████████▉| 223866/225000 [439:39:51<2:39:06,  8.42s/it]
 99%|█████████▉| 223867/225000 [439:40:04<3:03:32,  9.72s/it]
 99%|█████████▉| 223868/225000 [439:40:11<2:48:09,  8.91s/it]
 99%|█████████▉| 223869/225000 [439:40:22<3:01:46,  9.64s/it]
 99%|█████████▉| 223870/225000 [439:40:33<3:08:00,  9.98s/it]
 99%|█████████▉| 223871/225000 [439:40:41<2:57:45,  9.45s/it]
 99%|█████████▉| 223872/225000 [439:40:50<2:55:58,  9.36s/it]
 99%|█████████▉| 223873/225000 [439:40:55<2:30:54,  8.03s/it]
 99%|█████████▉| 223874/225000 [439:41:01<2:14:41,  7.18s/it]
100%|█████████▉| 223875/225000 [439:41:09<2:21:30,  7.55s/it]
100%|█████████▉| 223876/225000 [439:41:15<2:11:57,  7.04s/it]
100%|█████████▉| 223877/225000 [439:41:25<2:27:46,  7.90s/it]
100%|█████████▉| 223878/225000 [439:41:39<3:02:56,  9.78s/it]
100%|█████████▉| 223879/225000 [439:41:50<3:07:12, 10.02s/it]
100%|█████████▉| 223880/225000 [439:41:58<2:56:23,  9.45s/it]
100%|█████████▉| 223881/225000 [439:42:01<2:24:46,  7.76s/it]
100%|█████████▉| 223882/225000 [439:42:12<2:39:53,  8.58s/it]
100%|█████████▉| 223883/225000 [439:42:17<2:17:58,  7.41s/it]
100%|█████████▉| 223884/225000 [439:42:26<2:29:34,  8.04s/it]
100%|█████████▉| 223885/225000 [439:42:36<2:38:43,  8.54s/it]
100%|█████████▉| 223886/225000 [439:42:47<2:55:39,  9.46s/it]
100%|█████████▉| 223887/225000 [439:42:54<2:38:00,  8.52s/it]
100%|█████████▉| 223888/225000 [439:43:04<2:49:54,  9.17s/it]
100%|█████████▉| 223889/225000 [439:43:10<2:28:04,  8.00s/it]
100%|█████████▉| 223890/225000 [439:43:20<2:41:09,  8.71s/it]
100%|█████████▉| 223891/225000 [439:43:27<2:30:43,  8.15s/it]
100%|█████████▉| 223892/225000 [439:43:37<2:39:40,  8.65s/it]
100%|█████████▉| 223893/225000 [439:43:43<2:28:23,  8.04s/it]
100%|█████████▉| 223894/225000 [439:43:51<2:25:36,  7.90s/it]
100%|█████████▉| 223895/225000 [439:44:07<3:08:12, 10.22s/it]
100%|█████████▉| 223896/225000 [439:44:16<3:03:45,  9.99s/it]
100%|█████████▉| 223897/225000 [439:44:23<2:47:44,  9.12s/it]
100%|█████████▉| 223898/225000 [439:44:32<2:46:28,  9.06s/it]
100%|█████████▉| 223899/225000 [439:44:37<2:21:28,  7.71s/it]
100%|█████████▉| 223900/225000 [439:44:45<2:23:34,  7.83s/it]
                                                             
{'loss': '1.318', 'grad_norm': '1.612', 'learning_rate': '3.875e-09', 'epoch': '0.9951', 'num_input_tokens_seen': 751206864, 'train_runtime': '1.583e+06', 'train_tokens_per_second': '474.5'}

100%|█████████▉| 223900/225000 [439:44:45<2:23:34,  7.83s/it]
100%|█████████▉| 223901/225000 [439:44:57<2:47:34,  9.15s/it]
100%|█████████▉| 223902/225000 [439:45:06<2:48:48,  9.22s/it]
100%|█████████▉| 223903/225000 [439:45:18<3:00:51,  9.89s/it]
100%|█████████▉| 223904/225000 [439:45:26<2:51:58,  9.41s/it]
100%|█████████▉| 223905/225000 [439:45:33<2:37:32,  8.63s/it]
100%|█████████▉| 223906/225000 [439:45:40<2:27:53,  8.11s/it]
100%|█████████▉| 223907/225000 [439:45:49<2:33:16,  8.41s/it]
100%|█████████▉| 223908/225000 [439:45:56<2:25:34,  8.00s/it]
100%|█████████▉| 223909/225000 [439:46:04<2:25:46,  8.02s/it]
100%|█████████▉| 223910/225000 [439:46:19<3:02:05, 10.02s/it]
100%|█████████▉| 223911/225000 [439:46:28<2:59:22,  9.88s/it]
100%|█████████▉| 223912/225000 [439:46:34<2:37:41,  8.70s/it]
100%|█████████▉| 223913/225000 [439:46:39<2:18:53,  7.67s/it]
100%|█████████▉| 223914/225000 [439:46:47<2:15:41,  7.50s/it]
100%|█████████▉| 223915/225000 [439:46:59<2:44:55,  9.12s/it]
100%|█████████▉| 223916/225000 [439:47:11<2:58:58,  9.91s/it]
100%|█████████▉| 223917/225000 [439:47:18<2:40:44,  8.91s/it]
100%|█████████▉| 223918/225000 [439:47:25<2:30:59,  8.37s/it]
100%|█████████▉| 223919/225000 [439:47:33<2:27:56,  8.21s/it]
100%|█████████▉| 223920/225000 [439:47:42<2:31:50,  8.44s/it]
100%|█████████▉| 223921/225000 [439:47:53<2:49:03,  9.40s/it]
100%|█████████▉| 223922/225000 [439:48:09<3:23:03, 11.30s/it]
100%|█████████▉| 223923/225000 [439:48:17<3:04:46, 10.29s/it]
100%|█████████▉| 223924/225000 [439:48:23<2:40:29,  8.95s/it]
100%|█████████▉| 223925/225000 [439:48:30<2:32:40,  8.52s/it]
100%|█████████▉| 223926/225000 [439:48:37<2:23:55,  8.04s/it]
100%|█████████▉| 223927/225000 [439:48:48<2:36:15,  8.74s/it]
100%|█████████▉| 223928/225000 [439:48:58<2:42:49,  9.11s/it]
100%|█████████▉| 223929/225000 [439:49:09<2:52:55,  9.69s/it]
100%|█████████▉| 223930/225000 [439:49:16<2:39:57,  8.97s/it]
100%|█████████▉| 223931/225000 [439:49:26<2:43:33,  9.18s/it]
100%|█████████▉| 223932/225000 [439:49:39<3:06:49, 10.50s/it]
100%|█████████▉| 223933/225000 [439:49:46<2:47:21,  9.41s/it]
100%|█████████▉| 223934/225000 [439:49:52<2:26:45,  8.26s/it]
100%|█████████▉| 223935/225000 [439:49:57<2:12:07,  7.44s/it]
100%|█████████▉| 223936/225000 [439:50:04<2:11:03,  7.39s/it]
100%|█████████▉| 223937/225000 [439:50:16<2:31:12,  8.53s/it]
100%|█████████▉| 223938/225000 [439:50:28<2:49:14,  9.56s/it]
100%|█████████▉| 223939/225000 [439:50:36<2:41:00,  9.10s/it]
100%|█████████▉| 223940/225000 [439:50:46<2:47:01,  9.45s/it]
100%|█████████▉| 223941/225000 [439:50:57<2:57:18, 10.05s/it]
100%|█████████▉| 223942/225000 [439:51:07<2:52:28,  9.78s/it]
100%|█████████▉| 223943/225000 [439:51:15<2:46:33,  9.45s/it]
100%|█████████▉| 223944/225000 [439:51:24<2:44:15,  9.33s/it]
100%|█████████▉| 223945/225000 [439:51:34<2:43:58,  9.33s/it]
100%|█████████▉| 223946/225000 [439:51:44<2:47:33,  9.54s/it]
100%|█████████▉| 223947/225000 [439:51:53<2:45:01,  9.40s/it]
100%|█████████▉| 223948/225000 [439:51:59<2:29:24,  8.52s/it]
100%|█████████▉| 223949/225000 [439:52:06<2:21:29,  8.08s/it]
100%|█████████▉| 223950/225000 [439:52:14<2:21:28,  8.08s/it]
                                                             
{'loss': '1.297', 'grad_norm': '2.199', 'learning_rate': '3.56e-09', 'epoch': '0.9953', 'num_input_tokens_seen': 751447840, 'train_runtime': '1.584e+06', 'train_tokens_per_second': '474.5'}

100%|█████████▉| 223950/225000 [439:52:14<2:21:28,  8.08s/it]
100%|█████████▉| 223951/225000 [439:52:29<2:53:47,  9.94s/it]
100%|█████████▉| 223952/225000 [439:52:35<2:33:32,  8.79s/it]
100%|█████████▉| 223953/225000 [439:52:42<2:26:34,  8.40s/it]
100%|█████████▉| 223954/225000 [439:52:49<2:16:53,  7.85s/it]
100%|█████████▉| 223955/225000 [439:52:54<2:03:25,  7.09s/it]
100%|█████████▉| 223956/225000 [439:53:06<2:28:30,  8.54s/it]
100%|█████████▉| 223957/225000 [439:53:12<2:13:40,  7.69s/it]
100%|█████████▉| 223958/225000 [439:53:17<2:03:01,  7.08s/it]
100%|█████████▉| 223959/225000 [439:53:27<2:15:20,  7.80s/it]
100%|█████████▉| 223960/225000 [439:53:37<2:28:31,  8.57s/it]
100%|█████████▉| 223961/225000 [439:53:45<2:25:30,  8.40s/it]
100%|█████████▉| 223962/225000 [439:53:55<2:31:19,  8.75s/it]
100%|█████████▉| 223963/225000 [439:54:09<3:00:28, 10.44s/it]
100%|█████████▉| 223964/225000 [439:54:14<2:30:47,  8.73s/it]
100%|█████████▉| 223965/225000 [439:54:19<2:10:46,  7.58s/it]
100%|█████████▉| 223966/225000 [439:54:31<2:36:42,  9.09s/it]
100%|█████████▉| 223967/225000 [439:54:39<2:27:54,  8.59s/it]
100%|█████████▉| 223968/225000 [439:54:46<2:20:07,  8.15s/it]
100%|█████████▉| 223969/225000 [439:54:53<2:12:53,  7.73s/it]
100%|█████████▉| 223970/225000 [439:55:07<2:44:20,  9.57s/it]
100%|█████████▉| 223971/225000 [439:55:14<2:32:18,  8.88s/it]
100%|█████████▉| 223972/225000 [439:55:19<2:11:28,  7.67s/it]
100%|█████████▉| 223973/225000 [439:55:31<2:34:02,  9.00s/it]
100%|█████████▉| 223974/225000 [439:55:39<2:29:07,  8.72s/it]
100%|█████████▉| 223975/225000 [439:55:50<2:41:38,  9.46s/it]
100%|█████████▉| 223976/225000 [439:55:55<2:19:30,  8.17s/it]
100%|█████████▉| 223977/225000 [439:56:03<2:15:54,  7.97s/it]
100%|█████████▉| 223978/225000 [439:56:12<2:24:24,  8.48s/it]
100%|█████████▉| 223979/225000 [439:56:21<2:27:26,  8.66s/it]
100%|█████████▉| 223980/225000 [439:56:36<2:54:53, 10.29s/it]
100%|█████████▉| 223981/225000 [439:56:45<2:48:02,  9.89s/it]
100%|█████████▉| 223982/225000 [439:56:52<2:36:41,  9.24s/it]
100%|█████████▉| 223983/225000 [439:57:03<2:44:00,  9.68s/it]
100%|█████████▉| 223984/225000 [439:57:11<2:33:28,  9.06s/it]
100%|█████████▉| 223985/225000 [439:57:18<2:24:13,  8.53s/it]
100%|█████████▉| 223986/225000 [439:57:25<2:16:04,  8.05s/it]
100%|█████████▉| 223987/225000 [439:57:31<2:07:49,  7.57s/it]
100%|█████████▉| 223988/225000 [439:57:40<2:11:46,  7.81s/it]
100%|█████████▉| 223989/225000 [439:57:54<2:44:47,  9.78s/it]
100%|█████████▉| 223990/225000 [439:58:02<2:33:45,  9.13s/it]
100%|█████████▉| 223991/225000 [439:58:15<2:55:09, 10.42s/it]
100%|█████████▉| 223992/225000 [439:58:24<2:49:43, 10.10s/it]
100%|█████████▉| 223993/225000 [439:58:31<2:30:17,  8.95s/it]
100%|█████████▉| 223994/225000 [439:58:35<2:09:20,  7.71s/it]
100%|█████████▉| 223995/225000 [439:58:49<2:37:20,  9.39s/it]
100%|█████████▉| 223996/225000 [439:58:55<2:18:48,  8.30s/it]
100%|█████████▉| 223997/225000 [439:59:05<2:27:05,  8.80s/it]
100%|█████████▉| 223998/225000 [439:59:12<2:19:22,  8.35s/it]
100%|█████████▉| 223999/225000 [439:59:23<2:35:28,  9.32s/it]
100%|█████████▉| 224000/225000 [439:59:32<2:33:43,  9.22s/it]
                                                             
{'loss': '1.306', 'grad_norm': '1.208', 'learning_rate': '3.259e-09', 'epoch': '0.9956', 'num_input_tokens_seen': 751687624, 'train_runtime': '1.584e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224000/225000 [439:59:32<2:33:43,  9.22s/it]
100%|█████████▉| 224001/225000 [439:59:42<2:33:59,  9.25s/it]
100%|█████████▉| 224002/225000 [439:59:49<2:26:31,  8.81s/it]
100%|█████████▉| 224003/225000 [439:59:57<2:17:46,  8.29s/it]
100%|█████████▉| 224004/225000 [440:00:11<2:46:30, 10.03s/it]
100%|█████████▉| 224005/225000 [440:00:17<2:26:13,  8.82s/it]
100%|█████████▉| 224006/225000 [440:00:22<2:09:47,  7.83s/it]
100%|█████████▉| 224007/225000 [440:00:35<2:35:36,  9.40s/it]
100%|█████████▉| 224008/225000 [440:00:44<2:34:30,  9.35s/it]
100%|█████████▉| 224009/225000 [440:00:57<2:50:55, 10.35s/it]
100%|█████████▉| 224010/225000 [440:01:05<2:37:21,  9.54s/it]
100%|█████████▉| 224011/225000 [440:01:13<2:32:33,  9.26s/it]
100%|█████████▉| 224012/225000 [440:01:23<2:33:23,  9.31s/it]
100%|█████████▉| 224013/225000 [440:01:30<2:20:45,  8.56s/it]
100%|█████████▉| 224014/225000 [440:01:36<2:11:51,  8.02s/it]
100%|█████████▉| 224015/225000 [440:01:50<2:41:27,  9.83s/it]
100%|█████████▉| 224016/225000 [440:02:00<2:41:31,  9.85s/it]
100%|█████████▉| 224017/225000 [440:02:08<2:30:19,  9.18s/it]
100%|█████████▉| 224018/225000 [440:02:18<2:34:32,  9.44s/it]
100%|█████████▉| 224019/225000 [440:02:25<2:20:39,  8.60s/it]
100%|█████████▉| 224020/225000 [440:02:35<2:28:39,  9.10s/it]
100%|█████████▉| 224021/225000 [440:02:43<2:24:07,  8.83s/it]
100%|█████████▉| 224022/225000 [440:02:50<2:14:15,  8.24s/it]
100%|█████████▉| 224023/225000 [440:02:59<2:20:09,  8.61s/it]
100%|█████████▉| 224024/225000 [440:03:13<2:45:36, 10.18s/it]
100%|█████████▉| 224025/225000 [440:03:24<2:49:03, 10.40s/it]
100%|█████████▉| 224026/225000 [440:03:34<2:47:48, 10.34s/it]
100%|█████████▉| 224027/225000 [440:03:39<2:20:08,  8.64s/it]
100%|█████████▉| 224028/225000 [440:03:51<2:37:11,  9.70s/it]
100%|█████████▉| 224029/225000 [440:04:05<2:55:21, 10.84s/it]
100%|█████████▉| 224030/225000 [440:04:10<2:26:54,  9.09s/it]
100%|█████████▉| 224031/225000 [440:04:18<2:20:32,  8.70s/it]
100%|█████████▉| 224032/225000 [440:04:27<2:22:33,  8.84s/it]
100%|█████████▉| 224033/225000 [440:04:34<2:14:05,  8.32s/it]
100%|█████████▉| 224034/225000 [440:04:42<2:11:40,  8.18s/it]
100%|█████████▉| 224035/225000 [440:04:50<2:12:12,  8.22s/it]
100%|█████████▉| 224036/225000 [440:04:57<2:03:49,  7.71s/it]
100%|█████████▉| 224037/225000 [440:05:03<1:59:24,  7.44s/it]
100%|█████████▉| 224038/225000 [440:05:15<2:17:17,  8.56s/it]
100%|█████████▉| 224039/225000 [440:05:31<2:57:34, 11.09s/it]
100%|█████████▉| 224040/225000 [440:05:40<2:43:14, 10.20s/it]
100%|█████████▉| 224041/225000 [440:05:44<2:17:15,  8.59s/it]
100%|█████████▉| 224042/225000 [440:05:56<2:30:36,  9.43s/it]
100%|█████████▉| 224043/225000 [440:06:03<2:19:37,  8.75s/it]
100%|█████████▉| 224044/225000 [440:06:15<2:32:36,  9.58s/it]
100%|█████████▉| 224045/225000 [440:06:27<2:43:57, 10.30s/it]
100%|█████████▉| 224046/225000 [440:06:31<2:17:10,  8.63s/it]
100%|█████████▉| 224047/225000 [440:06:41<2:24:49,  9.12s/it]
100%|█████████▉| 224048/225000 [440:06:48<2:10:00,  8.19s/it]
100%|█████████▉| 224049/225000 [440:06:59<2:26:57,  9.27s/it]
100%|█████████▉| 224050/225000 [440:07:06<2:16:00,  8.59s/it]
                                                             
{'loss': '1.288', 'grad_norm': '1.446', 'learning_rate': '2.97e-09', 'epoch': '0.9958', 'num_input_tokens_seen': 751929416, 'train_runtime': '1.584e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224050/225000 [440:07:06<2:16:00,  8.59s/it]
100%|█████████▉| 224051/225000 [440:07:13<2:05:02,  7.91s/it]
100%|█████████▉| 224052/225000 [440:07:19<1:58:46,  7.52s/it]
100%|█████████▉| 224053/225000 [440:07:34<2:31:13,  9.58s/it]
100%|█████████▉| 224054/225000 [440:07:42<2:23:02,  9.07s/it]
100%|█████████▉| 224055/225000 [440:07:47<2:06:26,  8.03s/it]
100%|█████████▉| 224056/225000 [440:07:54<2:02:54,  7.81s/it]
100%|█████████▉| 224057/225000 [440:08:07<2:26:30,  9.32s/it]
100%|█████████▉| 224058/225000 [440:08:12<2:04:33,  7.93s/it]
100%|█████████▉| 224059/225000 [440:08:22<2:12:45,  8.46s/it]
100%|█████████▉| 224060/225000 [440:08:33<2:26:05,  9.32s/it]
100%|█████████▉| 224061/225000 [440:08:38<2:05:20,  8.01s/it]
100%|█████████▉| 224062/225000 [440:08:43<1:53:25,  7.26s/it]
100%|█████████▉| 224063/225000 [440:08:50<1:50:11,  7.06s/it]
100%|█████████▉| 224064/225000 [440:08:58<1:54:20,  7.33s/it]
100%|█████████▉| 224065/225000 [440:09:09<2:13:41,  8.58s/it]
100%|█████████▉| 224066/225000 [440:09:20<2:23:31,  9.22s/it]
100%|█████████▉| 224067/225000 [440:09:28<2:15:51,  8.74s/it]
100%|█████████▉| 224068/225000 [440:09:39<2:26:32,  9.43s/it]
100%|█████████▉| 224069/225000 [440:09:44<2:08:05,  8.25s/it]
100%|█████████▉| 224070/225000 [440:09:52<2:04:19,  8.02s/it]
100%|█████████▉| 224071/225000 [440:09:59<1:58:30,  7.65s/it]
100%|█████████▉| 224072/225000 [440:10:06<1:57:12,  7.58s/it]
100%|█████████▉| 224073/225000 [440:10:13<1:54:30,  7.41s/it]
100%|█████████▉| 224074/225000 [440:10:26<2:20:07,  9.08s/it]
100%|█████████▉| 224075/225000 [440:10:33<2:08:06,  8.31s/it]
100%|█████████▉| 224076/225000 [440:10:44<2:20:24,  9.12s/it]
100%|█████████▉| 224077/225000 [440:10:53<2:21:14,  9.18s/it]
100%|█████████▉| 224078/225000 [440:11:00<2:09:49,  8.45s/it]
100%|█████████▉| 224079/225000 [440:11:08<2:11:06,  8.54s/it]
100%|█████████▉| 224080/225000 [440:11:18<2:17:07,  8.94s/it]
100%|█████████▉| 224081/225000 [440:11:34<2:49:17, 11.05s/it]
100%|█████████▉| 224082/225000 [440:11:43<2:39:20, 10.41s/it]
100%|█████████▉| 224083/225000 [440:11:50<2:22:18,  9.31s/it]
100%|█████████▉| 224084/225000 [440:11:58<2:14:52,  8.83s/it]
100%|█████████▉| 224085/225000 [440:12:10<2:28:59,  9.77s/it]
100%|█████████▉| 224086/225000 [440:12:18<2:20:27,  9.22s/it]
100%|█████████▉| 224087/225000 [440:12:28<2:24:17,  9.48s/it]
100%|█████████▉| 224088/225000 [440:12:35<2:13:26,  8.78s/it]
100%|█████████▉| 224089/225000 [440:12:45<2:21:01,  9.29s/it]
100%|█████████▉| 224090/225000 [440:12:57<2:31:11,  9.97s/it]
100%|█████████▉| 224091/225000 [440:13:02<2:10:32,  8.62s/it]
100%|█████████▉| 224092/225000 [440:13:11<2:08:50,  8.51s/it]
100%|█████████▉| 224093/225000 [440:13:17<2:01:13,  8.02s/it]
100%|█████████▉| 224094/225000 [440:13:26<2:02:31,  8.11s/it]
100%|█████████▉| 224095/225000 [440:13:37<2:15:32,  8.99s/it]
100%|█████████▉| 224096/225000 [440:13:43<2:04:48,  8.28s/it]
100%|█████████▉| 224097/225000 [440:13:51<2:03:26,  8.20s/it]
100%|█████████▉| 224098/225000 [440:14:04<2:23:11,  9.52s/it]
100%|█████████▉| 224099/225000 [440:14:13<2:19:10,  9.27s/it]
100%|█████████▉| 224100/225000 [440:14:22<2:21:26,  9.43s/it]
                                                             
{'loss': '1.296', 'grad_norm': '1.137', 'learning_rate': '2.696e-09', 'epoch': '0.996', 'num_input_tokens_seen': 752168208, 'train_runtime': '1.585e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224100/225000 [440:14:22<2:21:26,  9.43s/it]
100%|█████████▉| 224101/225000 [440:14:34<2:30:28, 10.04s/it]
100%|█████████▉| 224102/225000 [440:14:39<2:08:35,  8.59s/it]
100%|█████████▉| 224103/225000 [440:14:47<2:03:42,  8.27s/it]
100%|█████████▉| 224104/225000 [440:14:57<2:11:56,  8.84s/it]
100%|█████████▉| 224105/225000 [440:15:08<2:20:12,  9.40s/it]
100%|█████████▉| 224106/225000 [440:15:14<2:07:28,  8.56s/it]
100%|█████████▉| 224107/225000 [440:15:22<2:03:15,  8.28s/it]
100%|█████████▉| 224108/225000 [440:15:34<2:22:11,  9.56s/it]
100%|█████████▉| 224109/225000 [440:15:39<2:02:22,  8.24s/it]
100%|█████████▉| 224110/225000 [440:15:49<2:08:10,  8.64s/it]
100%|█████████▉| 224111/225000 [440:15:58<2:10:14,  8.79s/it]
100%|█████████▉| 224112/225000 [440:16:09<2:19:13,  9.41s/it]
100%|█████████▉| 224113/225000 [440:16:18<2:15:24,  9.16s/it]
100%|█████████▉| 224114/225000 [440:16:26<2:13:27,  9.04s/it]
100%|█████████▉| 224115/225000 [440:16:36<2:15:00,  9.15s/it]
100%|█████████▉| 224116/225000 [440:16:42<2:00:18,  8.17s/it]
100%|█████████▉| 224117/225000 [440:16:55<2:21:14,  9.60s/it]
100%|█████████▉| 224118/225000 [440:17:01<2:07:47,  8.69s/it]
100%|█████████▉| 224119/225000 [440:17:13<2:20:38,  9.58s/it]
100%|█████████▉| 224120/225000 [440:17:19<2:07:31,  8.70s/it]
100%|█████████▉| 224121/225000 [440:17:25<1:54:47,  7.84s/it]
100%|█████████▉| 224122/225000 [440:17:41<2:27:04, 10.05s/it]
100%|█████████▉| 224123/225000 [440:17:49<2:20:46,  9.63s/it]
100%|█████████▉| 224124/225000 [440:17:58<2:17:00,  9.38s/it]
100%|█████████▉| 224125/225000 [440:18:10<2:28:02, 10.15s/it]
100%|█████████▉| 224126/225000 [440:18:17<2:14:34,  9.24s/it]
100%|█████████▉| 224127/225000 [440:18:29<2:26:53, 10.10s/it]
100%|█████████▉| 224128/225000 [440:18:35<2:10:25,  8.97s/it]
100%|█████████▉| 224129/225000 [440:18:42<1:59:23,  8.22s/it]
100%|█████████▉| 224130/225000 [440:18:53<2:10:15,  8.98s/it]
100%|█████████▉| 224131/225000 [440:19:05<2:25:06, 10.02s/it]
100%|█████████▉| 224132/225000 [440:19:12<2:13:22,  9.22s/it]
100%|█████████▉| 224133/225000 [440:19:21<2:11:37,  9.11s/it]
100%|█████████▉| 224134/225000 [440:19:28<2:00:39,  8.36s/it]
100%|█████████▉| 224135/225000 [440:19:36<1:58:51,  8.24s/it]
100%|█████████▉| 224136/225000 [440:19:47<2:11:29,  9.13s/it]
100%|█████████▉| 224137/225000 [440:19:54<2:02:45,  8.53s/it]
100%|█████████▉| 224138/225000 [440:20:08<2:23:08,  9.96s/it]
100%|█████████▉| 224139/225000 [440:20:15<2:10:36,  9.10s/it]
100%|█████████▉| 224140/225000 [440:20:20<1:54:33,  7.99s/it]
100%|█████████▉| 224141/225000 [440:20:35<2:24:57, 10.12s/it]
100%|█████████▉| 224142/225000 [440:20:42<2:12:07,  9.24s/it]
100%|█████████▉| 224143/225000 [440:20:52<2:12:13,  9.26s/it]
100%|█████████▉| 224144/225000 [440:21:00<2:09:44,  9.09s/it]
100%|█████████▉| 224145/225000 [440:21:14<2:29:51, 10.52s/it]
100%|█████████▉| 224146/225000 [440:21:20<2:11:02,  9.21s/it]
100%|█████████▉| 224147/225000 [440:21:30<2:13:28,  9.39s/it]
100%|█████████▉| 224148/225000 [440:21:38<2:06:30,  8.91s/it]
100%|█████████▉| 224149/225000 [440:21:46<2:04:43,  8.79s/it]
100%|█████████▉| 224150/225000 [440:21:56<2:05:35,  8.87s/it]
                                                             
{'loss': '1.312', 'grad_norm': '1.163', 'learning_rate': '2.434e-09', 'epoch': '0.9962', 'num_input_tokens_seen': 752414808, 'train_runtime': '1.585e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224150/225000 [440:21:56<2:05:35,  8.87s/it]
100%|█████████▉| 224151/225000 [440:22:01<1:50:57,  7.84s/it]
100%|█████████▉| 224152/225000 [440:22:14<2:13:51,  9.47s/it]
100%|█████████▉| 224153/225000 [440:22:19<1:53:44,  8.06s/it]
100%|█████████▉| 224154/225000 [440:22:32<2:16:28,  9.68s/it]
100%|█████████▉| 224155/225000 [440:22:39<2:02:15,  8.68s/it]
100%|█████████▉| 224156/225000 [440:22:49<2:09:19,  9.19s/it]
100%|█████████▉| 224157/225000 [440:22:58<2:09:18,  9.20s/it]
100%|█████████▉| 224158/225000 [440:23:08<2:11:43,  9.39s/it]
100%|█████████▉| 224159/225000 [440:23:14<1:56:34,  8.32s/it]
100%|█████████▉| 224160/225000 [440:23:22<1:54:48,  8.20s/it]
100%|█████████▉| 224161/225000 [440:23:31<1:57:36,  8.41s/it]
100%|█████████▉| 224162/225000 [440:23:39<1:57:38,  8.42s/it]
100%|█████████▉| 224163/225000 [440:23:46<1:51:36,  8.00s/it]
100%|█████████▉| 224164/225000 [440:23:57<2:03:12,  8.84s/it]
100%|█████████▉| 224165/225000 [440:24:07<2:07:56,  9.19s/it]
100%|█████████▉| 224166/225000 [440:24:20<2:21:46, 10.20s/it]
100%|█████████▉| 224167/225000 [440:24:28<2:13:19,  9.60s/it]
100%|█████████▉| 224168/225000 [440:24:39<2:20:16, 10.12s/it]
100%|█████████▉| 224169/225000 [440:24:51<2:25:24, 10.50s/it]
100%|█████████▉| 224170/225000 [440:25:00<2:18:32, 10.01s/it]
100%|█████████▉| 224171/225000 [440:25:04<1:56:36,  8.44s/it]
100%|█████████▉| 224172/225000 [440:25:09<1:42:32,  7.43s/it]
100%|█████████▉| 224173/225000 [440:25:20<1:54:26,  8.30s/it]
100%|█████████▉| 224174/225000 [440:25:31<2:06:47,  9.21s/it]
100%|█████████▉| 224175/225000 [440:25:37<1:51:28,  8.11s/it]
100%|█████████▉| 224176/225000 [440:25:48<2:05:29,  9.14s/it]
100%|█████████▉| 224177/225000 [440:26:00<2:17:41, 10.04s/it]
100%|█████████▉| 224178/225000 [440:26:06<2:01:36,  8.88s/it]
100%|█████████▉| 224179/225000 [440:26:12<1:49:13,  7.98s/it]
100%|█████████▉| 224180/225000 [440:26:22<1:56:10,  8.50s/it]
100%|█████████▉| 224181/225000 [440:26:30<1:54:28,  8.39s/it]
100%|█████████▉| 224182/225000 [440:26:38<1:52:47,  8.27s/it]
100%|█████████▉| 224183/225000 [440:26:47<1:55:00,  8.45s/it]
100%|█████████▉| 224184/225000 [440:27:01<2:16:21, 10.03s/it]
100%|█████████▉| 224185/225000 [440:27:14<2:27:41, 10.87s/it]
100%|█████████▉| 224186/225000 [440:27:25<2:29:46, 11.04s/it]
100%|█████████▉| 224187/225000 [440:27:35<2:23:45, 10.61s/it]
100%|█████████▉| 224188/225000 [440:27:43<2:14:49,  9.96s/it]
100%|█████████▉| 224189/225000 [440:27:52<2:12:25,  9.80s/it]
100%|█████████▉| 224190/225000 [440:27:59<2:00:05,  8.90s/it]
100%|█████████▉| 224191/225000 [440:28:10<2:07:51,  9.48s/it]
100%|█████████▉| 224192/225000 [440:28:18<2:01:49,  9.05s/it]
100%|█████████▉| 224193/225000 [440:28:27<1:59:19,  8.87s/it]
100%|█████████▉| 224194/225000 [440:28:40<2:15:39, 10.10s/it]
100%|█████████▉| 224195/225000 [440:28:47<2:05:19,  9.34s/it]
100%|█████████▉| 224196/225000 [440:28:56<2:03:19,  9.20s/it]
100%|█████████▉| 224197/225000 [440:29:05<2:02:56,  9.19s/it]
100%|█████████▉| 224198/225000 [440:29:15<2:05:09,  9.36s/it]
100%|█████████▉| 224199/225000 [440:29:22<1:56:30,  8.73s/it]
100%|█████████▉| 224200/225000 [440:29:32<2:00:05,  9.01s/it]
                                                             
{'loss': '1.331', 'grad_norm': '1.762', 'learning_rate': '2.186e-09', 'epoch': '0.9964', 'num_input_tokens_seen': 752651952, 'train_runtime': '1.586e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224200/225000 [440:29:32<2:00:05,  9.01s/it]
100%|█████████▉| 224201/225000 [440:29:38<1:48:25,  8.14s/it]
100%|█████████▉| 224202/225000 [440:29:46<1:49:16,  8.22s/it]
100%|█████████▉| 224203/225000 [440:29:54<1:48:09,  8.14s/it]
100%|█████████▉| 224204/225000 [440:30:03<1:50:59,  8.37s/it]
100%|█████████▉| 224205/225000 [440:30:14<1:58:47,  8.97s/it]
100%|█████████▉| 224206/225000 [440:30:28<2:21:13, 10.67s/it]
100%|█████████▉| 224207/225000 [440:30:39<2:21:34, 10.71s/it]
100%|█████████▉| 224208/225000 [440:30:46<2:07:52,  9.69s/it]
100%|█████████▉| 224209/225000 [440:30:58<2:16:16, 10.34s/it]
100%|█████████▉| 224210/225000 [440:31:07<2:10:32,  9.91s/it]
100%|█████████▉| 224211/225000 [440:31:13<1:56:13,  8.84s/it]
100%|█████████▉| 224212/225000 [440:31:23<1:58:23,  9.01s/it]
100%|█████████▉| 224213/225000 [440:31:29<1:46:10,  8.09s/it]
100%|█████████▉| 224214/225000 [440:31:41<2:01:54,  9.31s/it]
100%|█████████▉| 224215/225000 [440:31:47<1:50:42,  8.46s/it]
100%|█████████▉| 224216/225000 [440:31:54<1:43:22,  7.91s/it]
100%|█████████▉| 224217/225000 [440:32:09<2:12:26, 10.15s/it]
100%|█████████▉| 224218/225000 [440:32:20<2:14:53, 10.35s/it]
100%|█████████▉| 224219/225000 [440:32:31<2:15:16, 10.39s/it]
100%|█████████▉| 224220/225000 [440:32:37<1:58:57,  9.15s/it]
100%|█████████▉| 224221/225000 [440:32:46<1:57:09,  9.02s/it]
100%|█████████▉| 224222/225000 [440:32:53<1:48:41,  8.38s/it]
100%|█████████▉| 224223/225000 [440:33:06<2:09:39, 10.01s/it]
100%|█████████▉| 224224/225000 [440:33:16<2:07:00,  9.82s/it]
100%|█████████▉| 224225/225000 [440:33:28<2:16:07, 10.54s/it]
100%|█████████▉| 224226/225000 [440:33:34<1:58:18,  9.17s/it]
100%|█████████▉| 224227/225000 [440:33:43<1:56:16,  9.02s/it]
100%|█████████▉| 224228/225000 [440:33:56<2:13:23, 10.37s/it]
100%|█████████▉| 224229/225000 [440:34:04<2:03:39,  9.62s/it]
100%|█████████▉| 224230/225000 [440:34:19<2:22:56, 11.14s/it]
100%|█████████▉| 224231/225000 [440:34:25<2:04:39,  9.73s/it]
100%|█████████▉| 224232/225000 [440:34:33<1:57:57,  9.21s/it]
100%|█████████▉| 224233/225000 [440:34:45<2:07:02,  9.94s/it]
100%|█████████▉| 224234/225000 [440:34:53<2:01:35,  9.52s/it]
100%|█████████▉| 224235/225000 [440:35:05<2:10:40, 10.25s/it]
100%|█████████▉| 224236/225000 [440:35:10<1:48:46,  8.54s/it]
100%|█████████▉| 224237/225000 [440:35:16<1:41:00,  7.94s/it]
100%|█████████▉| 224238/225000 [440:35:24<1:41:06,  7.96s/it]
100%|█████████▉| 224239/225000 [440:35:40<2:08:34, 10.14s/it]
100%|█████████▉| 224240/225000 [440:35:49<2:03:47,  9.77s/it]
100%|█████████▉| 224241/225000 [440:35:56<1:55:40,  9.14s/it]
100%|█████████▉| 224242/225000 [440:36:06<1:57:13,  9.28s/it]
100%|█████████▉| 224243/225000 [440:36:15<1:55:57,  9.19s/it]
100%|█████████▉| 224244/225000 [440:36:20<1:42:27,  8.13s/it]
100%|█████████▉| 224245/225000 [440:36:28<1:41:10,  8.04s/it]
100%|█████████▉| 224246/225000 [440:36:35<1:35:54,  7.63s/it]
100%|█████████▉| 224247/225000 [440:36:47<1:51:38,  8.90s/it]
100%|█████████▉| 224248/225000 [440:36:57<1:57:11,  9.35s/it]
100%|█████████▉| 224249/225000 [440:37:09<2:06:35, 10.11s/it]
100%|█████████▉| 224250/225000 [440:37:17<1:58:44,  9.50s/it]
                                                             
{'loss': '1.293', 'grad_norm': '2.025', 'learning_rate': '1.951e-09', 'epoch': '0.9967', 'num_input_tokens_seen': 752902536, 'train_runtime': '1.586e+06', 'train_tokens_per_second': '474.6'}

100%|█████████▉| 224250/225000 [440:37:17<1:58:44,  9.50s/it]
100%|█████████▉| 224251/225000 [440:37:30<2:12:34, 10.62s/it]
100%|█████████▉| 224252/225000 [440:37:38<2:00:45,  9.69s/it]
100%|█████████▉| 224253/225000 [440:37:44<1:45:39,  8.49s/it]
100%|█████████▉| 224254/225000 [440:37:54<1:52:13,  9.03s/it]
100%|█████████▉| 224255/225000 [440:38:04<1:57:17,  9.45s/it]
100%|█████████▉| 224256/225000 [440:38:15<2:02:06,  9.85s/it]
100%|█████████▉| 224257/225000 [440:38:21<1:47:39,  8.69s/it]
100%|█████████▉| 224258/225000 [440:38:33<2:00:15,  9.72s/it]
100%|█████████▉| 224259/225000 [440:38:37<1:38:07,  7.95s/it]
100%|█████████▉| 224260/225000 [440:38:47<1:45:01,  8.52s/it]
100%|█████████▉| 224261/225000 [440:38:58<1:53:02,  9.18s/it]
100%|█████████▉| 224262/225000 [440:39:09<2:00:44,  9.82s/it]
100%|█████████▉| 224263/225000 [440:39:16<1:49:50,  8.94s/it]
100%|█████████▉| 224264/225000 [440:39:22<1:39:08,  8.08s/it]
100%|█████████▉| 224265/225000 [440:39:29<1:37:09,  7.93s/it]
100%|█████████▉| 224266/225000 [440:39:40<1:46:07,  8.68s/it]
100%|█████████▉| 224267/225000 [440:39:48<1:44:23,  8.54s/it]
100%|█████████▉| 224268/225000 [440:40:03<2:08:32, 10.54s/it]
100%|█████████▉| 224269/225000 [440:40:12<2:02:34, 10.06s/it]
100%|█████████▉| 224270/225000 [440:40:19<1:51:34,  9.17s/it]
100%|█████████▉| 224271/225000 [440:40:31<1:58:56,  9.79s/it]
100%|█████████▉| 224272/225000 [440:40:37<1:45:37,  8.71s/it]
100%|█████████▉| 224273/225000 [440:40:42<1:32:34,  7.64s/it]
100%|█████████▉| 224274/225000 [440:40:59<2:05:19, 10.36s/it]
100%|█████████▉| 224275/225000 [440:41:10<2:10:14, 10.78s/it]
100%|█████████▉| 224276/225000 [440:41:18<2:00:16,  9.97s/it]
100%|█████████▉| 224277/225000 [440:41:25<1:48:38,  9.02s/it]
100%|█████████▉| 224278/225000 [440:41:33<1:42:52,  8.55s/it]
100%|█████████▉| 224279/225000 [440:41:39<1:33:43,  7.80s/it]
100%|█████████▉| 224280/225000 [440:41:51<1:48:25,  9.04s/it]
100%|█████████▉| 224281/225000 [440:42:00<1:48:47,  9.08s/it]
100%|█████████▉| 224282/225000 [440:42:16<2:12:51, 11.10s/it]
100%|█████████▉| 224283/225000 [440:42:22<1:54:46,  9.61s/it]
100%|█████████▉| 224284/225000 [440:42:26<1:36:49,  8.11s/it]
100%|█████████▉| 224285/225000 [440:42:35<1:37:43,  8.20s/it]
100%|█████████▉| 224286/225000 [440:42:45<1:44:09,  8.75s/it]
100%|█████████▉| 224287/225000 [440:42:52<1:37:29,  8.20s/it]
100%|█████████▉| 224288/225000 [440:43:06<1:58:51, 10.02s/it]
100%|█████████▉| 224289/225000 [440:43:14<1:52:10,  9.47s/it]
100%|█████████▉| 224290/225000 [440:43:23<1:50:36,  9.35s/it]
100%|█████████▉| 224291/225000 [440:43:32<1:49:49,  9.29s/it]
100%|█████████▉| 224292/225000 [440:43:41<1:47:34,  9.12s/it]
100%|█████████▉| 224293/225000 [440:43:50<1:46:53,  9.07s/it]
100%|█████████▉| 224294/225000 [440:43:57<1:40:39,  8.55s/it]
100%|█████████▉| 224295/225000 [440:44:10<1:53:29,  9.66s/it]
100%|█████████▉| 224296/225000 [440:44:20<1:55:05,  9.81s/it]
100%|█████████▉| 224297/225000 [440:44:28<1:47:41,  9.19s/it]
100%|█████████▉| 224298/225000 [440:44:39<1:55:05,  9.84s/it]
100%|█████████▉| 224299/225000 [440:44:49<1:54:18,  9.78s/it]
100%|█████████▉| 224300/225000 [440:44:55<1:41:52,  8.73s/it]
                                                             
{'loss': '1.335', 'grad_norm': '2.032', 'learning_rate': '1.734e-09', 'epoch': '0.9969', 'num_input_tokens_seen': 753145440, 'train_runtime': '1.587e+06', 'train_tokens_per_second': '474.7'}

100%|█████████▉| 224300/225000 [440:44:55<1:41:52,  8.73s/it]
100%|█████████▉| 224301/225000 [440:45:00<1:29:30,  7.68s/it]
100%|█████████▉| 224302/225000 [440:45:12<1:43:40,  8.91s/it]
100%|█████████▉| 224303/225000 [440:45:20<1:39:26,  8.56s/it]
100%|█████████▉| 224304/225000 [440:45:25<1:27:21,  7.53s/it]
100%|█████████▉| 224305/225000 [440:45:36<1:39:50,  8.62s/it]
100%|█████████▉| 224306/225000 [440:45:49<1:55:40, 10.00s/it]
100%|█████████▉| 224307/225000 [440:45:55<1:41:11,  8.76s/it]
100%|█████████▉| 224308/225000 [440:46:02<1:33:10,  8.08s/it]
100%|█████████▉| 224309/225000 [440:46:08<1:25:48,  7.45s/it]
100%|█████████▉| 224310/225000 [440:46:16<1:28:09,  7.67s/it]
100%|█████████▉| 224311/225000 [440:46:29<1:47:06,  9.33s/it]
100%|█████████▉| 224312/225000 [440:46:39<1:48:15,  9.44s/it]
100%|█████████▉| 224313/225000 [440:46:47<1:44:45,  9.15s/it]
100%|█████████▉| 224314/225000 [440:47:00<1:56:47, 10.22s/it]
100%|█████████▉| 224315/225000 [440:47:07<1:44:58,  9.20s/it]
100%|█████████▉| 224316/225000 [440:47:18<1:53:10,  9.93s/it]
100%|█████████▉| 224317/225000 [440:47:24<1:39:27,  8.74s/it]
100%|█████████▉| 224318/225000 [440:47:37<1:54:19, 10.06s/it]
100%|█████████▉| 224319/225000 [440:47:43<1:39:02,  8.73s/it]
100%|█████████▉| 224320/225000 [440:47:49<1:29:52,  7.93s/it]
100%|█████████▉| 224321/225000 [440:48:00<1:40:30,  8.88s/it]
100%|█████████▉| 224322/225000 [440:48:06<1:30:47,  8.03s/it]
100%|█████████▉| 224323/225000 [440:48:14<1:29:35,  7.94s/it]
100%|█████████▉| 224324/225000 [440:48:23<1:32:05,  8.17s/it]
100%|█████████▉| 224325/225000 [440:48:30<1:28:11,  7.84s/it]
100%|█████████▉| 224326/225000 [440:48:36<1:23:20,  7.42s/it]
100%|█████████▉| 224327/225000 [440:48:43<1:21:24,  7.26s/it]
100%|█████████▉| 224328/225000 [440:48:57<1:43:23,  9.23s/it]
100%|█████████▉| 224329/225000 [440:49:08<1:51:11,  9.94s/it]
100%|█████████▉| 224330/225000 [440:49:13<1:33:18,  8.36s/it]
100%|█████████▉| 224331/225000 [440:49:25<1:45:30,  9.46s/it]
100%|█████████▉| 224332/225000 [440:49:34<1:43:56,  9.34s/it]
100%|█████████▉| 224333/225000 [440:49:39<1:27:23,  7.86s/it]
100%|█████████▉| 224334/225000 [440:49:48<1:32:15,  8.31s/it]
100%|█████████▉| 224335/225000 [440:50:04<1:56:14, 10.49s/it]
100%|█████████▉| 224336/225000 [440:50:10<1:42:39,  9.28s/it]
100%|█████████▉| 224337/225000 [440:50:16<1:32:23,  8.36s/it]
100%|█████████▉| 224338/225000 [440:50:24<1:31:28,  8.29s/it]
100%|█████████▉| 224339/225000 [440:50:33<1:33:36,  8.50s/it]
100%|█████████▉| 224340/225000 [440:50:42<1:35:25,  8.68s/it]
100%|█████████▉| 224341/225000 [440:50:51<1:35:42,  8.71s/it]
100%|█████████▉| 224342/225000 [440:51:05<1:51:00, 10.12s/it]
100%|█████████▉| 224343/225000 [440:51:15<1:50:10, 10.06s/it]
100%|█████████▉| 224344/225000 [440:51:21<1:39:20,  9.09s/it]
100%|█████████▉| 224345/225000 [440:51:34<1:52:16, 10.28s/it]
100%|█████████▉| 224346/225000 [440:51:41<1:41:38,  9.33s/it]
100%|█████████▉| 224347/225000 [440:51:51<1:40:29,  9.23s/it]
100%|█████████▉| 224348/225000 [440:52:02<1:46:13,  9.77s/it]
100%|█████████▉| 224349/225000 [440:52:09<1:39:14,  9.15s/it]
100%|█████████▉| 224350/225000 [440:52:15<1:27:53,  8.11s/it]
                                                             
{'loss': '1.3', 'grad_norm': '1.478', 'learning_rate': '1.525e-09', 'epoch': '0.9971', 'num_input_tokens_seen': 753382856, 'train_runtime': '1.587e+06', 'train_tokens_per_second': '474.7'}

100%|█████████▉| 224350/225000 [440:52:15<1:27:53,  8.11s/it]
100%|█████████▉| 224351/225000 [440:52:25<1:33:16,  8.62s/it]
100%|█████████▉| 224352/225000 [440:52:31<1:24:30,  7.83s/it]
100%|█████████▉| 224353/225000 [440:52:35<1:13:04,  6.78s/it]
100%|█████████▉| 224354/225000 [440:52:44<1:20:00,  7.43s/it]
100%|█████████▉| 224355/225000 [440:52:53<1:23:48,  7.80s/it]
100%|█████████▉| 224356/225000 [440:53:04<1:35:34,  8.90s/it]
100%|█████████▉| 224357/225000 [440:53:19<1:53:45, 10.62s/it]
100%|█████████▉| 224358/225000 [440:53:25<1:39:46,  9.33s/it]
100%|█████████▉| 224359/225000 [440:53:37<1:48:09, 10.12s/it]
100%|█████████▉| 224360/225000 [440:53:43<1:33:19,  8.75s/it]
100%|█████████▉| 224361/225000 [440:53:51<1:30:53,  8.53s/it]
100%|█████████▉| 224362/225000 [440:53:58<1:28:26,  8.32s/it]
100%|█████████▉| 224363/225000 [440:54:04<1:17:58,  7.35s/it]
100%|█████████▉| 224364/225000 [440:54:14<1:29:18,  8.43s/it]
100%|█████████▉| 224365/225000 [440:54:27<1:41:11,  9.56s/it]
100%|█████████▉| 224366/225000 [440:54:33<1:30:16,  8.54s/it]
100%|█████████▉| 224367/225000 [440:54:41<1:28:54,  8.43s/it]
100%|█████████▉| 224368/225000 [440:54:53<1:40:21,  9.53s/it]
100%|█████████▉| 224369/225000 [440:55:02<1:37:33,  9.28s/it]
100%|█████████▉| 224370/225000 [440:55:15<1:49:47, 10.46s/it]
100%|█████████▉| 224371/225000 [440:55:22<1:37:26,  9.30s/it]
100%|█████████▉| 224372/225000 [440:55:34<1:46:49, 10.21s/it]
100%|█████████▉| 224373/225000 [440:55:44<1:45:08, 10.06s/it]
100%|█████████▉| 224374/225000 [440:55:53<1:43:44,  9.94s/it]
100%|█████████▉| 224375/225000 [440:56:02<1:39:31,  9.55s/it]
100%|█████████▉| 224376/225000 [440:56:12<1:40:03,  9.62s/it]
100%|█████████▉| 224377/225000 [440:56:20<1:35:10,  9.17s/it]
100%|█████████▉| 224378/225000 [440:56:29<1:35:29,  9.21s/it]
100%|█████████▉| 224379/225000 [440:56:40<1:39:16,  9.59s/it]
100%|█████████▉| 224380/225000 [440:56:48<1:34:40,  9.16s/it]
100%|█████████▉| 224381/225000 [440:56:54<1:25:59,  8.33s/it]
100%|█████████▉| 224382/225000 [440:57:03<1:27:24,  8.49s/it]
100%|█████████▉| 224383/225000 [440:57:15<1:39:22,  9.66s/it]
100%|█████████▉| 224384/225000 [440:57:28<1:46:48, 10.40s/it]
100%|█████████▉| 224385/225000 [440:57:36<1:41:18,  9.88s/it]
100%|█████████▉| 224386/225000 [440:57:43<1:30:41,  8.86s/it]
100%|█████████▉| 224387/225000 [440:57:53<1:33:53,  9.19s/it]
100%|█████████▉| 224388/225000 [440:58:03<1:36:46,  9.49s/it]
100%|█████████▉| 224389/225000 [440:58:08<1:23:15,  8.18s/it]
100%|█████████▉| 224390/225000 [440:58:18<1:28:29,  8.70s/it]
100%|█████████▉| 224391/225000 [440:58:26<1:26:01,  8.48s/it]
100%|█████████▉| 224392/225000 [440:58:37<1:35:20,  9.41s/it]
100%|█████████▉| 224393/225000 [440:58:47<1:36:21,  9.52s/it]
100%|█████████▉| 224394/225000 [440:58:57<1:36:59,  9.60s/it]
100%|█████████▉| 224395/225000 [440:59:04<1:30:00,  8.93s/it]
100%|█████████▉| 224396/225000 [440:59:14<1:33:12,  9.26s/it]
100%|█████████▉| 224397/225000 [440:59:25<1:36:14,  9.58s/it]
100%|█████████▉| 224398/225000 [440:59:33<1:30:48,  9.05s/it]
100%|█████████▉| 224399/225000 [440:59:41<1:28:30,  8.84s/it]
100%|█████████▉| 224400/225000 [440:59:46<1:18:44,  7.87s/it]
                                                             
{'loss': '1.336', 'grad_norm': '1.534', 'learning_rate': '1.33e-09', 'epoch': '0.9973', 'num_input_tokens_seen': 753629176, 'train_runtime': '1.588e+06', 'train_tokens_per_second': '474.7'}

100%|█████████▉| 224400/225000 [440:59:46<1:18:44,  7.87s/it]
100%|█████████▉| 224401/225000 [440:59:59<1:31:39,  9.18s/it]
100%|█████████▉| 224402/225000 [441:00:11<1:41:58, 10.23s/it]
100%|█████████▉| 224403/225000 [441:00:21<1:40:29, 10.10s/it]
100%|█████████▉| 224404/225000 [441:00:28<1:31:51,  9.25s/it]
100%|█████████▉| 224405/225000 [441:00:43<1:45:58, 10.69s/it]
100%|█████████▉| 224406/225000 [441:00:50<1:35:17,  9.63s/it]
100%|█████████▉| 224407/225000 [441:01:01<1:38:50, 10.00s/it]
100%|█████████▉| 224408/225000 [441:01:08<1:32:07,  9.34s/it]
100%|█████████▉| 224409/225000 [441:01:22<1:43:57, 10.55s/it]
100%|█████████▉| 224410/225000 [441:01:27<1:29:30,  9.10s/it]
100%|█████████▉| 224411/225000 [441:01:41<1:42:56, 10.49s/it]
100%|█████████▉| 224412/225000 [441:01:46<1:24:48,  8.65s/it]
100%|█████████▉| 224413/225000 [441:01:56<1:29:30,  9.15s/it]
100%|█████████▉| 224414/225000 [441:02:01<1:17:09,  7.90s/it]
100%|█████████▉| 224415/225000 [441:02:07<1:12:04,  7.39s/it]
100%|█████████▉| 224416/225000 [441:02:15<1:12:53,  7.49s/it]
100%|█████████▉| 224417/225000 [441:02:21<1:09:57,  7.20s/it]
100%|█████████▉| 224418/225000 [441:02:28<1:07:09,  6.92s/it]
100%|█████████▉| 224419/225000 [441:02:35<1:07:33,  6.98s/it]
100%|█████████▉| 224420/225000 [441:02:45<1:17:52,  8.06s/it]
100%|█████████▉| 224421/225000 [441:02:56<1:26:02,  8.92s/it]
100%|█████████▉| 224422/225000 [441:03:04<1:23:01,  8.62s/it]
100%|█████████▉| 224423/225000 [441:03:13<1:24:59,  8.84s/it]
100%|█████████▉| 224424/225000 [441:03:23<1:27:15,  9.09s/it]
100%|█████████▉| 224425/225000 [441:03:28<1:15:47,  7.91s/it]
100%|█████████▉| 224426/225000 [441:03:38<1:20:47,  8.44s/it]
100%|█████████▉| 224427/225000 [441:03:47<1:22:02,  8.59s/it]
100%|█████████▉| 224428/225000 [441:04:00<1:34:27,  9.91s/it]
100%|█████████▉| 224429/225000 [441:04:07<1:26:47,  9.12s/it]
100%|█████████▉| 224430/225000 [441:04:21<1:39:57, 10.52s/it]
100%|█████████▉| 224431/225000 [441:04:30<1:34:47, 10.00s/it]
100%|█████████▉| 224432/225000 [441:04:36<1:24:29,  8.92s/it]
100%|█████████▉| 224433/225000 [441:04:44<1:20:21,  8.50s/it]
100%|█████████▉| 224434/225000 [441:05:00<1:41:59, 10.81s/it]
100%|█████████▉| 224435/225000 [441:05:09<1:38:25, 10.45s/it]
100%|█████████▉| 224436/225000 [441:05:19<1:35:04, 10.11s/it]
100%|█████████▉| 224437/225000 [441:05:29<1:36:09, 10.25s/it]
100%|█████████▉| 224438/225000 [441:05:39<1:34:52, 10.13s/it]
100%|█████████▉| 224439/225000 [441:05:47<1:28:03,  9.42s/it]
100%|█████████▉| 224440/225000 [441:05:56<1:28:15,  9.46s/it]
100%|█████████▉| 224441/225000 [441:06:05<1:25:39,  9.19s/it]
100%|█████████▉| 224442/225000 [441:06:15<1:28:15,  9.49s/it]
100%|█████████▉| 224443/225000 [441:06:21<1:18:27,  8.45s/it]
100%|█████████▉| 224444/225000 [441:06:31<1:21:00,  8.74s/it]
100%|█████████▉| 224445/225000 [441:06:40<1:22:06,  8.88s/it]
100%|█████████▉| 224446/225000 [441:06:49<1:22:42,  8.96s/it]
100%|█████████▉| 224447/225000 [441:06:57<1:19:51,  8.66s/it]
100%|█████████▉| 224448/225000 [441:07:03<1:12:21,  7.86s/it]
100%|█████████▉| 224449/225000 [441:07:14<1:20:45,  8.79s/it]
100%|█████████▉| 224450/225000 [441:07:22<1:19:03,  8.62s/it]
                                                             
{'loss': '1.301', 'grad_norm': '1.579', 'learning_rate': '1.149e-09', 'epoch': '0.9976', 'num_input_tokens_seen': 753874096, 'train_runtime': '1.588e+06', 'train_tokens_per_second': '474.7'}

100%|█████████▉| 224450/225000 [441:07:22<1:19:03,  8.62s/it]
100%|█████████▉| 224451/225000 [441:07:30<1:17:29,  8.47s/it]
100%|█████████▉| 224452/225000 [441:07:38<1:14:57,  8.21s/it]
100%|█████████▉| 224453/225000 [441:07:47<1:17:43,  8.53s/it]
100%|█████████▉| 224454/225000 [441:07:59<1:25:13,  9.36s/it]
100%|█████████▉| 224455/225000 [441:08:04<1:14:54,  8.25s/it]
100%|█████████▉| 224456/225000 [441:08:10<1:08:32,  7.56s/it]
100%|█████████▉| 224457/225000 [441:08:18<1:08:09,  7.53s/it]
100%|█████████▉| 224458/225000 [441:08:30<1:22:34,  9.14s/it]
100%|█████████▉| 224459/225000 [441:08:39<1:21:17,  9.02s/it]
100%|█████████▉| 224460/225000 [441:08:45<1:12:07,  8.01s/it]
100%|█████████▉| 224461/225000 [441:08:57<1:23:51,  9.34s/it]
100%|█████████▉| 224462/225000 [441:09:03<1:13:58,  8.25s/it]
100%|█████████▉| 224463/225000 [441:09:11<1:12:28,  8.10s/it]
100%|█████████▉| 224464/225000 [441:09:23<1:24:16,  9.43s/it]
100%|█████████▉| 224465/225000 [441:09:29<1:13:22,  8.23s/it]
100%|█████████▉| 224466/225000 [441:09:39<1:17:48,  8.74s/it]
100%|█████████▉| 224467/225000 [441:09:48<1:19:41,  8.97s/it]
100%|█████████▉| 224468/225000 [441:09:55<1:14:58,  8.46s/it]
100%|█████████▉| 224469/225000 [441:10:10<1:30:13, 10.19s/it]
100%|█████████▉| 224470/225000 [441:10:23<1:37:05, 10.99s/it]
100%|█████████▉| 224471/225000 [441:10:28<1:22:48,  9.39s/it]
100%|█████████▉| 224472/225000 [441:10:38<1:23:11,  9.45s/it]
100%|█████████▉| 224473/225000 [441:10:46<1:18:53,  8.98s/it]
100%|█████████▉| 224474/225000 [441:10:57<1:24:25,  9.63s/it]
100%|█████████▉| 224475/225000 [441:11:04<1:17:49,  8.89s/it]
100%|█████████▉| 224476/225000 [441:11:13<1:19:01,  9.05s/it]
100%|█████████▉| 224477/225000 [441:11:19<1:09:02,  7.92s/it]
100%|█████████▉| 224478/225000 [441:11:29<1:15:36,  8.69s/it]
100%|█████████▉| 224479/225000 [441:11:37<1:14:28,  8.58s/it]
100%|█████████▉| 224480/225000 [441:11:45<1:12:30,  8.37s/it]
100%|█████████▉| 224481/225000 [441:11:53<1:10:33,  8.16s/it]
100%|█████████▉| 224482/225000 [441:11:58<1:01:48,  7.16s/it]
100%|█████████▉| 224483/225000 [441:12:12<1:20:23,  9.33s/it]
100%|█████████▉| 224484/225000 [441:12:20<1:15:47,  8.81s/it]
100%|█████████▉| 224485/225000 [441:12:27<1:11:17,  8.31s/it]
100%|█████████▉| 224486/225000 [441:12:41<1:25:51, 10.02s/it]
100%|█████████▉| 224487/225000 [441:12:44<1:08:38,  8.03s/it]
100%|█████████▉| 224488/225000 [441:12:52<1:07:10,  7.87s/it]
100%|█████████▉| 224489/225000 [441:13:02<1:12:50,  8.55s/it]
100%|█████████▉| 224490/225000 [441:13:14<1:20:30,  9.47s/it]
100%|█████████▉| 224491/225000 [441:13:25<1:24:45,  9.99s/it]
100%|█████████▉| 224492/225000 [441:13:35<1:23:56,  9.91s/it]
100%|█████████▉| 224493/225000 [441:13:43<1:20:56,  9.58s/it]
100%|█████████▉| 224494/225000 [441:13:51<1:16:49,  9.11s/it]
100%|█████████▉| 224495/225000 [441:13:59<1:12:35,  8.63s/it]
100%|█████████▉| 224496/225000 [441:14:09<1:15:36,  9.00s/it]
100%|█████████▉| 224497/225000 [441:14:18<1:15:20,  8.99s/it]
100%|█████████▉| 224498/225000 [441:14:26<1:12:20,  8.65s/it]
100%|█████████▉| 224499/225000 [441:14:34<1:11:00,  8.50s/it]
100%|█████████▉| 224500/225000 [441:14:41<1:07:42,  8.13s/it]
                                                             
{'loss': '1.309', 'grad_norm': '2.237', 'learning_rate': '9.801e-10', 'epoch': '0.9978', 'num_input_tokens_seen': 754115360, 'train_runtime': '1.588e+06', 'train_tokens_per_second': '474.7'}

100%|█████████▉| 224500/225000 [441:14:41<1:07:42,  8.13s/it]
100%|█████████▉| 224501/225000 [441:14:50<1:08:45,  8.27s/it]
100%|█████████▉| 224502/225000 [441:14:56<1:04:18,  7.75s/it]
100%|█████████▉| 224503/225000 [441:15:08<1:13:46,  8.91s/it]
100%|█████████▉| 224504/225000 [441:15:13<1:05:44,  7.95s/it]
100%|█████████▉| 224505/225000 [441:15:24<1:11:34,  8.68s/it]
100%|█████████▉| 224506/225000 [441:15:34<1:16:01,  9.23s/it]
100%|█████████▉| 224507/225000 [441:15:40<1:08:12,  8.30s/it]
100%|█████████▉| 224508/225000 [441:15:53<1:17:57,  9.51s/it]
100%|█████████▉| 224509/225000 [441:15:58<1:06:58,  8.18s/it]
100%|█████████▉| 224510/225000 [441:16:08<1:11:12,  8.72s/it]
100%|█████████▉| 224511/225000 [441:16:19<1:17:57,  9.57s/it]
100%|█████████▉| 224512/225000 [441:16:25<1:08:18,  8.40s/it]
100%|█████████▉| 224513/225000 [441:16:35<1:12:31,  8.94s/it]
100%|█████████▉| 224514/225000 [441:16:43<1:09:12,  8.54s/it]
100%|█████████▉| 224515/225000 [441:16:50<1:05:53,  8.15s/it]
100%|█████████▉| 224516/225000 [441:17:06<1:23:37, 10.37s/it]
100%|█████████▉| 224517/225000 [441:17:14<1:18:06,  9.70s/it]
100%|█████████▉| 224518/225000 [441:17:20<1:09:39,  8.67s/it]
100%|█████████▉| 224519/225000 [441:17:28<1:07:47,  8.46s/it]
100%|█████████▉| 224520/225000 [441:17:34<1:00:31,  7.57s/it]
100%|█████████▉| 224521/225000 [441:17:40<58:15,  7.30s/it]  
100%|█████████▉| 224522/225000 [441:17:51<1:05:59,  8.28s/it]
100%|█████████▉| 224523/225000 [441:18:01<1:09:17,  8.72s/it]
100%|█████████▉| 224524/225000 [441:18:09<1:07:52,  8.56s/it]
100%|█████████▉| 224525/225000 [441:18:17<1:07:04,  8.47s/it]
100%|█████████▉| 224526/225000 [441:18:22<57:57,  7.34s/it]  
100%|█████████▉| 224527/225000 [441:18:32<1:03:52,  8.10s/it]
100%|█████████▉| 224528/225000 [441:18:40<1:03:44,  8.10s/it]
100%|█████████▉| 224529/225000 [441:18:47<1:02:08,  7.92s/it]
100%|█████████▉| 224530/225000 [441:18:57<1:05:30,  8.36s/it]
100%|█████████▉| 224531/225000 [441:19:03<1:02:05,  7.94s/it]
100%|█████████▉| 224532/225000 [441:19:14<1:06:56,  8.58s/it]
100%|█████████▉| 224533/225000 [441:19:20<1:01:49,  7.94s/it]
100%|█████████▉| 224534/225000 [441:19:28<1:02:13,  8.01s/it]
100%|█████████▉| 224535/225000 [441:19:35<59:00,  7.61s/it]  
100%|█████████▉| 224536/225000 [441:19:47<1:10:08,  9.07s/it]
100%|█████████▉| 224537/225000 [441:19:54<1:04:10,  8.32s/it]
100%|█████████▉| 224538/225000 [441:20:09<1:18:46, 10.23s/it]
100%|█████████▉| 224539/225000 [441:20:17<1:15:04,  9.77s/it]
100%|█████████▉| 224540/225000 [441:20:23<1:05:30,  8.55s/it]
100%|█████████▉| 224541/225000 [441:20:36<1:14:35,  9.75s/it]
100%|█████████▉| 224542/225000 [441:20:43<1:09:21,  9.09s/it]
100%|█████████▉| 224543/225000 [441:20:48<1:00:33,  7.95s/it]
100%|█████████▉| 224544/225000 [441:20:54<54:46,  7.21s/it]  
100%|█████████▉| 224545/225000 [441:21:04<1:01:57,  8.17s/it]
100%|█████████▉| 224546/225000 [441:21:17<1:11:56,  9.51s/it]
100%|█████████▉| 224547/225000 [441:21:23<1:04:41,  8.57s/it]
100%|█████████▉| 224548/225000 [441:21:31<1:02:00,  8.23s/it]
100%|█████████▉| 224549/225000 [441:21:42<1:09:22,  9.23s/it]
100%|█████████▉| 224550/225000 [441:21:50<1:04:47,  8.64s/it]
                                                             
{'loss': '1.297', 'grad_norm': '1.025', 'learning_rate': '8.25e-10', 'epoch': '0.998', 'num_input_tokens_seen': 754353208, 'train_runtime': '1.589e+06', 'train_tokens_per_second': '474.8'}

100%|█████████▉| 224550/225000 [441:21:50<1:04:47,  8.64s/it]
100%|█████████▉| 224551/225000 [441:21:57<1:01:19,  8.19s/it]
100%|█████████▉| 224552/225000 [441:22:05<1:00:46,  8.14s/it]
100%|█████████▉| 224553/225000 [441:22:15<1:04:27,  8.65s/it]
100%|█████████▉| 224554/225000 [441:22:23<1:03:07,  8.49s/it]
100%|█████████▉| 224555/225000 [441:22:30<1:00:58,  8.22s/it]
100%|█████████▉| 224556/225000 [441:22:39<1:02:14,  8.41s/it]
100%|█████████▉| 224557/225000 [441:22:47<1:00:59,  8.26s/it]
100%|█████████▉| 224558/225000 [441:23:01<1:12:33,  9.85s/it]
100%|█████████▉| 224559/225000 [441:23:10<1:10:52,  9.64s/it]
100%|█████████▉| 224560/225000 [441:23:18<1:08:31,  9.34s/it]
100%|█████████▉| 224561/225000 [441:23:26<1:04:16,  8.78s/it]
100%|█████████▉| 224562/225000 [441:23:35<1:05:10,  8.93s/it]
100%|█████████▉| 224563/225000 [441:23:43<1:02:20,  8.56s/it]
100%|█████████▉| 224564/225000 [441:23:52<1:02:44,  8.63s/it]
100%|█████████▉| 224565/225000 [441:23:59<58:47,  8.11s/it]  
100%|█████████▉| 224566/225000 [441:24:10<1:05:02,  8.99s/it]
100%|█████████▉| 224567/225000 [441:24:18<1:02:42,  8.69s/it]
100%|█████████▉| 224568/225000 [441:24:27<1:03:51,  8.87s/it]
100%|█████████▉| 224569/225000 [441:24:34<1:00:02,  8.36s/it]
100%|█████████▉| 224570/225000 [441:24:40<55:17,  7.71s/it]  
100%|█████████▉| 224571/225000 [441:24:48<56:01,  7.83s/it]
100%|█████████▉| 224572/225000 [441:25:05<1:13:51, 10.35s/it]
100%|█████████▉| 224573/225000 [441:25:11<1:05:51,  9.25s/it]
100%|█████████▉| 224574/225000 [441:25:24<1:13:14, 10.32s/it]
100%|█████████▉| 224575/225000 [441:25:33<1:10:11,  9.91s/it]
100%|█████████▉| 224576/225000 [441:25:57<1:39:45, 14.12s/it]
100%|█████████▉| 224577/225000 [441:26:18<1:55:00, 16.31s/it]
100%|█████████▉| 224578/225000 [441:26:39<2:02:57, 17.48s/it]
100%|█████████▉| 224579/225000 [441:27:02<2:15:51, 19.36s/it]
100%|█████████▉| 224580/225000 [441:27:17<2:05:11, 17.88s/it]
100%|█████████▉| 224581/225000 [441:27:41<2:17:37, 19.71s/it]
100%|█████████▉| 224582/225000 [441:27:54<2:04:12, 17.83s/it]
100%|█████████▉| 224583/225000 [441:28:20<2:20:41, 20.24s/it]
100%|█████████▉| 224584/225000 [441:28:40<2:19:20, 20.10s/it]
100%|█████████▉| 224585/225000 [441:29:05<2:28:56, 21.53s/it]
100%|█████████▉| 224586/225000 [441:29:27<2:30:11, 21.77s/it]
100%|█████████▉| 224587/225000 [441:29:46<2:25:03, 21.07s/it]
100%|█████████▉| 224588/225000 [441:30:13<2:35:55, 22.71s/it]
100%|█████████▉| 224589/225000 [441:30:23<2:09:13, 18.87s/it]
100%|█████████▉| 224590/225000 [441:30:46<2:18:05, 20.21s/it]
100%|█████████▉| 224591/225000 [441:31:14<2:33:46, 22.56s/it]
100%|█████████▉| 224592/225000 [441:31:26<2:11:14, 19.30s/it]
100%|█████████▉| 224593/225000 [441:31:46<2:11:53, 19.44s/it]
100%|█████████▉| 224594/225000 [441:32:05<2:10:58, 19.36s/it]
100%|█████████▉| 224595/225000 [441:32:26<2:13:29, 19.78s/it]
100%|█████████▉| 224596/225000 [441:32:50<2:21:47, 21.06s/it]
100%|█████████▉| 224597/225000 [441:33:10<2:19:38, 20.79s/it]
100%|█████████▉| 224598/225000 [441:33:27<2:11:26, 19.62s/it]
100%|█████████▉| 224599/225000 [441:33:58<2:34:38, 23.14s/it]
100%|█████████▉| 224600/225000 [441:34:20<2:31:48, 22.77s/it]
                                                             
{'loss': '1.314', 'grad_norm': '1.015', 'learning_rate': '6.833e-10', 'epoch': '1', 'num_input_tokens_seen': 754754352, 'train_runtime': '1.59e+06', 'train_tokens_per_second': '474.8'}

100%|█████████▉| 224600/225000 [441:34:20<2:31:48, 22.77s/it]
100%|█████████▉| 224601/225000 [441:34:39<2:23:30, 21.58s/it]
100%|█████████▉| 224602/225000 [441:34:55<2:13:06, 20.07s/it]
100%|█████████▉| 224603/225000 [441:35:13<2:08:21, 19.40s/it]
100%|█████████▉| 224604/225000 [441:35:34<2:10:24, 19.76s/it]
100%|█████████▉| 224605/225000 [441:35:49<2:01:34, 18.47s/it]
100%|█████████▉| 224606/225000 [441:36:09<2:03:21, 18.79s/it]
100%|█████████▉| 224607/225000 [441:36:29<2:06:35, 19.33s/it]
100%|█████████▉| 224608/225000 [441:36:52<2:12:29, 20.28s/it]
100%|█████████▉| 224609/225000 [441:37:22<2:30:51, 23.15s/it]
100%|█████████▉| 224610/225000 [441:37:45<2:31:39, 23.33s/it]
100%|█████████▉| 224611/225000 [441:38:05<2:23:28, 22.13s/it]
100%|█████████▉| 224612/225000 [441:38:20<2:10:10, 20.13s/it]
100%|█████████▉| 224613/225000 [441:38:43<2:14:11, 20.80s/it]
100%|█████████▉| 224614/225000 [441:39:02<2:11:07, 20.38s/it]
100%|█████████▉| 224615/225000 [441:39:28<2:22:15, 22.17s/it]
100%|█████████▉| 224616/225000 [441:39:47<2:15:52, 21.23s/it]
100%|█████████▉| 224617/225000 [441:40:14<2:25:32, 22.80s/it]
100%|█████████▉| 224618/225000 [441:40:34<2:20:30, 22.07s/it]
100%|█████████▉| 224619/225000 [441:40:53<2:14:43, 21.22s/it]
100%|█████████▉| 224620/225000 [441:41:15<2:15:12, 21.35s/it]
100%|█████████▉| 224621/225000 [441:41:43<2:27:28, 23.35s/it]
100%|█████████▉| 224622/225000 [441:42:01<2:16:40, 21.69s/it]
100%|█████████▉| 224623/225000 [441:42:22<2:14:55, 21.47s/it]
100%|█████████▉| 224624/225000 [441:42:54<2:34:41, 24.69s/it]
100%|█████████▉| 224625/225000 [441:43:13<2:23:23, 22.94s/it]
100%|█████████▉| 224626/225000 [441:43:35<2:20:19, 22.51s/it]
100%|█████████▉| 224627/225000 [441:43:53<2:13:18, 21.44s/it]
100%|█████████▉| 224628/225000 [441:44:13<2:09:59, 20.97s/it]
100%|█████████▉| 224629/225000 [441:44:38<2:16:54, 22.14s/it]
100%|█████████▉| 224630/225000 [441:44:59<2:14:39, 21.84s/it]
100%|█████████▉| 224631/225000 [441:45:18<2:08:18, 20.86s/it]
100%|█████████▉| 224632/225000 [441:45:36<2:02:50, 20.03s/it]
100%|█████████▉| 224633/225000 [441:45:51<1:52:46, 18.44s/it]
100%|█████████▉| 224634/225000 [441:46:18<2:07:54, 20.97s/it]
100%|█████████▉| 224635/225000 [441:46:35<2:00:43, 19.84s/it]
100%|█████████▉| 224636/225000 [441:47:02<2:14:14, 22.13s/it]
100%|█████████▉| 224637/225000 [441:47:22<2:09:01, 21.33s/it]
100%|█████████▉| 224638/225000 [441:47:44<2:10:58, 21.71s/it]
100%|█████████▉| 224639/225000 [441:48:06<2:11:10, 21.80s/it]
100%|█████████▉| 224640/225000 [441:48:23<2:02:19, 20.39s/it]
100%|█████████▉| 224641/225000 [441:48:42<1:59:14, 19.93s/it]
100%|█████████▉| 224642/225000 [441:48:59<1:53:30, 19.02s/it]
100%|█████████▉| 224643/225000 [441:49:25<2:05:56, 21.17s/it]
100%|█████████▉| 224644/225000 [441:49:43<1:59:51, 20.20s/it]
100%|█████████▉| 224645/225000 [441:49:56<1:45:18, 17.80s/it]
100%|█████████▉| 224646/225000 [441:50:17<1:52:24, 19.05s/it]
100%|█████████▉| 224647/225000 [441:50:42<2:01:29, 20.65s/it]
100%|█████████▉| 224648/225000 [441:51:03<2:02:25, 20.87s/it]
100%|█████████▉| 224649/225000 [441:51:22<1:57:58, 20.17s/it]
100%|█████████▉| 224650/225000 [441:51:45<2:02:13, 20.95s/it]
                                                             
{'loss': '1.25', 'grad_norm': '0.9014', 'learning_rate': '5.55e-10', 'epoch': '1', 'num_input_tokens_seen': 755305464, 'train_runtime': '1.591e+06', 'train_tokens_per_second': '474.8'}

100%|█████████▉| 224650/225000 [441:51:45<2:02:13, 20.95s/it]
100%|█████████▉| 224651/225000 [441:52:12<2:13:43, 22.99s/it]
100%|█████████▉| 224652/225000 [441:52:35<2:13:15, 22.97s/it]
100%|█████████▉| 224653/225000 [441:52:48<1:55:00, 19.89s/it]
100%|█████████▉| 224654/225000 [441:53:07<1:54:04, 19.78s/it]
100%|█████████▉| 224655/225000 [441:53:28<1:55:40, 20.12s/it]
100%|█████████▉| 224656/225000 [441:53:42<1:44:15, 18.18s/it]
100%|█████████▉| 224657/225000 [441:53:59<1:42:08, 17.87s/it]
100%|█████████▉| 224658/225000 [441:54:21<1:48:50, 19.09s/it]
100%|█████████▉| 224659/225000 [441:54:40<1:47:48, 18.97s/it]
100%|█████████▉| 224660/225000 [441:55:10<2:05:58, 22.23s/it]
100%|█████████▉| 224661/225000 [441:55:30<2:02:02, 21.60s/it]
100%|█████████▉| 224662/225000 [441:55:54<2:05:22, 22.26s/it]
100%|█████████▉| 224663/225000 [441:56:17<2:07:33, 22.71s/it]
100%|█████████▉| 224664/225000 [441:56:33<1:54:49, 20.51s/it]
100%|█████████▉| 224665/225000 [441:56:52<1:53:02, 20.24s/it]
100%|█████████▉| 224666/225000 [441:57:15<1:56:11, 20.87s/it]
100%|█████████▉| 224667/225000 [441:57:36<1:57:17, 21.13s/it]
100%|█████████▉| 224668/225000 [441:57:57<1:55:42, 20.91s/it]
100%|█████████▉| 224669/225000 [441:58:14<1:49:33, 19.86s/it]
100%|█████████▉| 224670/225000 [441:58:35<1:50:09, 20.03s/it]
100%|█████████▉| 224671/225000 [441:58:59<1:56:38, 21.27s/it]
100%|█████████▉| 224672/225000 [441:59:14<1:46:29, 19.48s/it]
100%|█████████▉| 224673/225000 [441:59:37<1:51:44, 20.50s/it]
100%|█████████▉| 224674/225000 [442:00:01<1:56:21, 21.42s/it]
100%|█████████▉| 224675/225000 [442:00:24<1:58:47, 21.93s/it]
100%|█████████▉| 224676/225000 [442:00:42<1:53:11, 20.96s/it]
100%|█████████▉| 224677/225000 [442:01:04<1:54:09, 21.20s/it]
100%|█████████▉| 224678/225000 [442:01:20<1:45:53, 19.73s/it]
100%|█████████▉| 224679/225000 [442:01:32<1:32:28, 17.29s/it]
100%|█████████▉| 224680/225000 [442:01:59<1:48:22, 20.32s/it]
100%|█████████▉| 224681/225000 [442:02:20<1:48:02, 20.32s/it]
100%|█████████▉| 224682/225000 [442:02:41<1:48:37, 20.49s/it]
100%|█████████▉| 224683/225000 [442:02:59<1:44:49, 19.84s/it]
100%|█████████▉| 224684/225000 [442:03:22<1:49:23, 20.77s/it]
100%|█████████▉| 224685/225000 [442:03:37<1:40:16, 19.10s/it]
100%|█████████▉| 224686/225000 [442:04:03<1:50:52, 21.19s/it]
100%|█████████▉| 224687/225000 [442:04:19<1:41:48, 19.51s/it]
100%|█████████▉| 224688/225000 [442:04:32<1:32:03, 17.70s/it]
100%|█████████▉| 224689/225000 [442:04:57<1:42:12, 19.72s/it]
100%|█████████▉| 224690/225000 [442:05:17<1:43:32, 20.04s/it]
100%|█████████▉| 224691/225000 [442:05:37<1:42:34, 19.92s/it]
100%|█████████▉| 224692/225000 [442:06:00<1:46:27, 20.74s/it]
100%|█████████▉| 224693/225000 [442:06:25<1:52:29, 21.98s/it]
100%|█████████▉| 224694/225000 [442:06:37<1:37:48, 19.18s/it]
100%|█████████▉| 224695/225000 [442:06:53<1:32:01, 18.10s/it]
100%|█████████▉| 224696/225000 [442:07:07<1:26:14, 17.02s/it]
100%|█████████▉| 224697/225000 [442:07:26<1:29:05, 17.64s/it]
100%|█████████▉| 224698/225000 [442:07:47<1:33:48, 18.64s/it]
100%|█████████▉| 224699/225000 [442:08:13<1:44:17, 20.79s/it]
100%|█████████▉| 224700/225000 [442:08:30<1:38:39, 19.73s/it]
                                                             
{'loss': '1.246', 'grad_norm': '1.523', 'learning_rate': '4.399e-10', 'epoch': '1.001', 'num_input_tokens_seen': 755835944, 'train_runtime': '1.592e+06', 'train_tokens_per_second': '474.9'}

100%|█████████▉| 224700/225000 [442:08:30<1:38:39, 19.73s/it]
100%|█████████▉| 224701/225000 [442:08:48<1:34:57, 19.05s/it]
100%|█████████▉| 224702/225000 [442:09:05<1:31:14, 18.37s/it]
100%|█████████▉| 224703/225000 [442:09:28<1:38:25, 19.88s/it]
100%|█████████▉| 224704/225000 [442:09:43<1:31:00, 18.45s/it]
100%|█████████▉| 224705/225000 [442:10:09<1:42:07, 20.77s/it]
100%|█████████▉| 224706/225000 [442:10:27<1:36:33, 19.70s/it]
100%|█████████▉| 224707/225000 [442:10:51<1:43:35, 21.21s/it]
100%|█████████▉| 224708/225000 [442:11:12<1:41:45, 20.91s/it]
100%|█████████▉| 224709/225000 [442:11:35<1:44:50, 21.62s/it]
100%|█████████▉| 224710/225000 [442:12:01<1:51:08, 22.99s/it]
100%|█████████▉| 224711/225000 [442:12:28<1:56:46, 24.24s/it]
100%|█████████▉| 224712/225000 [442:12:53<1:56:44, 24.32s/it]
100%|█████████▉| 224713/225000 [442:13:04<1:37:48, 20.45s/it]
100%|█████████▉| 224714/225000 [442:13:23<1:35:09, 19.96s/it]
100%|█████████▉| 224715/225000 [442:13:42<1:33:23, 19.66s/it]
100%|█████████▉| 224716/225000 [442:13:56<1:24:43, 17.90s/it]
100%|█████████▉| 224717/225000 [442:14:21<1:35:20, 20.21s/it]
100%|█████████▉| 224718/225000 [442:14:38<1:30:13, 19.20s/it]
100%|█████████▉| 224719/225000 [442:15:02<1:36:04, 20.52s/it]
100%|█████████▉| 224720/225000 [442:15:18<1:29:45, 19.23s/it]
100%|█████████▉| 224721/225000 [442:15:43<1:37:47, 21.03s/it]
100%|█████████▉| 224722/225000 [442:15:56<1:26:29, 18.67s/it]
100%|█████████▉| 224723/225000 [442:16:18<1:29:57, 19.48s/it]
100%|█████████▉| 224724/225000 [442:16:38<1:30:22, 19.65s/it]
100%|█████████▉| 224725/225000 [442:16:59<1:32:10, 20.11s/it]
100%|█████████▉| 224726/225000 [442:17:17<1:28:28, 19.37s/it]
100%|█████████▉| 224727/225000 [442:17:38<1:31:35, 20.13s/it]
100%|█████████▉| 224728/225000 [442:17:50<1:20:06, 17.67s/it]
100%|█████████▉| 224729/225000 [442:18:07<1:18:14, 17.32s/it]
100%|█████████▉| 224730/225000 [442:18:22<1:15:18, 16.73s/it]
100%|█████████▉| 224731/225000 [442:18:41<1:17:17, 17.24s/it]
100%|█████████▉| 224732/225000 [442:18:56<1:14:14, 16.62s/it]
100%|█████████▉| 224733/225000 [442:19:25<1:30:59, 20.45s/it]
100%|█████████▉| 224734/225000 [442:19:47<1:32:23, 20.84s/it]
100%|█████████▉| 224735/225000 [442:20:13<1:38:37, 22.33s/it]
100%|█████████▉| 224736/225000 [442:20:27<1:28:04, 20.02s/it]
100%|█████████▉| 224737/225000 [442:20:53<1:35:12, 21.72s/it]
100%|█████████▉| 224738/225000 [442:21:07<1:24:44, 19.41s/it]
100%|█████████▉| 224739/225000 [442:21:24<1:21:30, 18.74s/it]
100%|█████████▉| 224740/225000 [442:21:50<1:30:32, 20.90s/it]
100%|█████████▉| 224741/225000 [442:22:13<1:32:45, 21.49s/it]
100%|█████████▉| 224742/225000 [442:22:26<1:21:07, 18.87s/it]
100%|█████████▉| 224743/225000 [442:22:42<1:16:49, 17.93s/it]
100%|█████████▉| 224744/225000 [442:23:10<1:29:51, 21.06s/it]
100%|█████████▉| 224745/225000 [442:23:30<1:28:09, 20.74s/it]
100%|█████████▉| 224746/225000 [442:23:51<1:27:40, 20.71s/it]
100%|█████████▉| 224747/225000 [442:24:11<1:26:59, 20.63s/it]
100%|█████████▉| 224748/225000 [442:24:33<1:28:51, 21.16s/it]
100%|█████████▉| 224749/225000 [442:24:55<1:28:41, 21.20s/it]
100%|█████████▉| 224750/225000 [442:25:05<1:15:02, 18.01s/it]
                                                             
{'loss': '1.235', 'grad_norm': '2.077', 'learning_rate': '3.382e-10', 'epoch': '1.001', 'num_input_tokens_seen': 756368032, 'train_runtime': '1.593e+06', 'train_tokens_per_second': '474.9'}

100%|█████████▉| 224750/225000 [442:25:05<1:15:02, 18.01s/it]
100%|█████████▉| 224751/225000 [442:25:19<1:09:21, 16.71s/it]
100%|█████████▉| 224752/225000 [442:25:48<1:24:50, 20.52s/it]
100%|█████████▉| 224753/225000 [442:26:10<1:25:16, 20.71s/it]
100%|█████████▉| 224754/225000 [442:26:33<1:27:42, 21.39s/it]
100%|█████████▉| 224755/225000 [442:26:52<1:24:40, 20.73s/it]
100%|█████████▉| 224756/225000 [442:27:11<1:23:01, 20.42s/it]
100%|█████████▉| 224757/225000 [442:27:31<1:21:32, 20.14s/it]
100%|█████████▉| 224758/225000 [442:27:58<1:28:59, 22.07s/it]
100%|█████████▉| 224759/225000 [442:28:12<1:19:11, 19.72s/it]
100%|█████████▉| 224760/225000 [442:28:31<1:18:13, 19.56s/it]
100%|█████████▉| 224761/225000 [442:28:52<1:19:28, 19.95s/it]
100%|█████████▉| 224762/225000 [442:29:17<1:24:51, 21.39s/it]
100%|█████████▉| 224763/225000 [442:29:34<1:19:48, 20.21s/it]
100%|█████████▉| 224764/225000 [442:29:48<1:12:09, 18.35s/it]
100%|█████████▉| 224765/225000 [442:30:14<1:20:26, 20.54s/it]
100%|█████████▉| 224766/225000 [442:30:33<1:18:39, 20.17s/it]
100%|█████████▉| 224767/225000 [442:30:48<1:11:58, 18.54s/it]
100%|█████████▉| 224768/225000 [442:31:11<1:17:38, 20.08s/it]
100%|█████████▉| 224769/225000 [442:31:30<1:15:49, 19.70s/it]
100%|█████████▉| 224770/225000 [442:31:56<1:22:06, 21.42s/it]
100%|█████████▉| 224771/225000 [442:32:14<1:18:12, 20.49s/it]
100%|█████████▉| 224772/225000 [442:32:35<1:18:17, 20.60s/it]
100%|█████████▉| 224773/225000 [442:32:48<1:09:11, 18.29s/it]
100%|█████████▉| 224774/225000 [442:33:05<1:08:01, 18.06s/it]
100%|█████████▉| 224775/225000 [442:33:28<1:12:36, 19.36s/it]
100%|█████████▉| 224776/225000 [442:33:47<1:12:00, 19.29s/it]
100%|█████████▉| 224777/225000 [442:34:11<1:16:49, 20.67s/it]
100%|█████████▉| 224778/225000 [442:34:27<1:11:46, 19.40s/it]
100%|█████████▉| 224779/225000 [442:34:48<1:13:16, 19.89s/it]
100%|█████████▉| 224780/225000 [442:35:10<1:15:13, 20.52s/it]
100%|█████████▉| 224781/225000 [442:35:31<1:15:43, 20.75s/it]
100%|█████████▉| 224782/225000 [442:35:50<1:12:54, 20.07s/it]
100%|█████████▉| 224783/225000 [442:36:16<1:19:35, 22.01s/it]
100%|█████████▉| 224784/225000 [442:36:37<1:17:17, 21.47s/it]
100%|█████████▉| 224785/225000 [442:37:00<1:19:00, 22.05s/it]
100%|█████████▉| 224786/225000 [442:37:17<1:13:41, 20.66s/it]
100%|█████████▉| 224787/225000 [442:37:38<1:13:38, 20.74s/it]
100%|█████████▉| 224788/225000 [442:37:51<1:04:23, 18.22s/it]
100%|█████████▉| 224789/225000 [442:38:11<1:06:17, 18.85s/it]
100%|█████████▉| 224790/225000 [442:38:35<1:11:15, 20.36s/it]
100%|█████████▉| 224791/225000 [442:38:55<1:10:38, 20.28s/it]
100%|█████████▉| 224792/225000 [442:39:17<1:11:40, 20.68s/it]
100%|█████████▉| 224793/225000 [442:39:31<1:04:33, 18.71s/it]
100%|█████████▉| 224794/225000 [442:39:55<1:09:31, 20.25s/it]
100%|█████████▉| 224795/225000 [442:40:10<1:03:50, 18.68s/it]
100%|█████████▉| 224796/225000 [442:40:36<1:11:11, 20.94s/it]
100%|█████████▉| 224797/225000 [442:40:53<1:06:56, 19.79s/it]
100%|█████████▉| 224798/225000 [442:41:11<1:04:36, 19.19s/it]
100%|█████████▉| 224799/225000 [442:41:26<1:00:02, 17.92s/it]
100%|█████████▉| 224800/225000 [442:41:43<58:49, 17.65s/it]  
                                                           
{'loss': '1.233', 'grad_norm': '0.7999', 'learning_rate': '2.499e-10', 'epoch': '1.001', 'num_input_tokens_seen': 756889152, 'train_runtime': '1.594e+06', 'train_tokens_per_second': '474.9'}

100%|█████████▉| 224800/225000 [442:41:43<58:49, 17.65s/it]
100%|█████████▉| 224801/225000 [442:41:59<57:09, 17.23s/it]
100%|█████████▉| 224802/225000 [442:42:16<57:10, 17.32s/it]
100%|█████████▉| 224803/225000 [442:42:39<1:02:11, 18.94s/it]
100%|█████████▉| 224804/225000 [442:43:01<1:04:25, 19.72s/it]
100%|█████████▉| 224805/225000 [442:43:18<1:01:37, 18.96s/it]
100%|█████████▉| 224806/225000 [442:43:43<1:06:56, 20.70s/it]
100%|█████████▉| 224807/225000 [442:44:06<1:09:02, 21.46s/it]
100%|█████████▉| 224808/225000 [442:44:33<1:13:58, 23.12s/it]
100%|█████████▉| 224809/225000 [442:44:55<1:13:01, 22.94s/it]
100%|█████████▉| 224810/225000 [442:45:12<1:06:45, 21.08s/it]
100%|█████████▉| 224811/225000 [442:45:36<1:09:02, 21.92s/it]
100%|█████████▉| 224812/225000 [442:45:58<1:08:30, 21.87s/it]
100%|█████████▉| 224813/225000 [442:46:15<1:04:12, 20.60s/it]
100%|█████████▉| 224814/225000 [442:46:39<1:06:47, 21.55s/it]
100%|█████████▉| 224815/225000 [442:47:03<1:08:34, 22.24s/it]
100%|█████████▉| 224816/225000 [442:47:21<1:04:08, 20.91s/it]
100%|█████████▉| 224817/225000 [442:47:43<1:04:44, 21.23s/it]
100%|█████████▉| 224818/225000 [442:47:59<1:00:04, 19.80s/it]
100%|█████████▉| 224819/225000 [442:48:17<57:36, 19.10s/it]  
100%|█████████▉| 224820/225000 [442:48:29<51:32, 17.18s/it]
100%|█████████▉| 224821/225000 [442:48:57<1:00:47, 20.38s/it]
100%|█████████▉| 224822/225000 [442:49:15<58:22, 19.68s/it]  
100%|█████████▉| 224823/225000 [442:49:38<1:00:18, 20.44s/it]
100%|█████████▉| 224824/225000 [442:49:53<55:50, 19.04s/it]  
100%|█████████▉| 224825/225000 [442:50:10<53:09, 18.23s/it]
100%|█████████▉| 224826/225000 [442:50:31<55:36, 19.18s/it]
100%|█████████▉| 224827/225000 [442:50:46<51:19, 17.80s/it]
100%|█████████▉| 224828/225000 [442:51:07<54:21, 18.97s/it]
100%|█████████▉| 224829/225000 [442:51:26<53:53, 18.91s/it]
100%|█████████▉| 224830/225000 [442:51:50<57:27, 20.28s/it]
100%|█████████▉| 224831/225000 [442:52:09<56:49, 20.18s/it]
100%|█████████▉| 224832/225000 [442:52:23<51:13, 18.30s/it]
100%|█████████▉| 224833/225000 [442:52:49<57:20, 20.60s/it]
100%|█████████▉| 224834/225000 [442:53:11<57:59, 20.96s/it]
100%|█████████▉| 224835/225000 [442:53:32<57:37, 20.95s/it]
100%|█████████▉| 224836/225000 [442:53:57<1:00:37, 22.18s/it]
100%|█████████▉| 224837/225000 [442:54:22<1:02:20, 22.95s/it]
100%|█████████▉| 224838/225000 [442:54:39<56:59, 21.11s/it]  
100%|█████████▉| 224839/225000 [442:55:00<56:36, 21.10s/it]
100%|█████████▉| 224840/225000 [442:55:18<54:02, 20.27s/it]
100%|█████████▉| 224841/225000 [442:55:35<50:47, 19.17s/it]
100%|█████████▉| 224842/225000 [442:55:58<54:04, 20.53s/it]
100%|█████████▉| 224843/225000 [442:56:16<51:07, 19.54s/it]
100%|█████████▉| 224844/225000 [442:56:32<48:36, 18.70s/it]
100%|█████████▉| 224845/225000 [442:57:03<57:13, 22.15s/it]
100%|█████████▉| 224846/225000 [442:57:19<52:19, 20.39s/it]
100%|█████████▉| 224847/225000 [442:57:42<54:03, 21.20s/it]
100%|█████████▉| 224848/225000 [442:58:03<53:47, 21.23s/it]
100%|█████████▉| 224849/225000 [442:58:25<53:35, 21.30s/it]
100%|█████████▉| 224850/225000 [442:58:41<49:23, 19.76s/it]
                                                           
{'loss': '1.231', 'grad_norm': '1.095', 'learning_rate': '1.749e-10', 'epoch': '1.001', 'num_input_tokens_seen': 757417080, 'train_runtime': '1.595e+06', 'train_tokens_per_second': '475'}

100%|█████████▉| 224850/225000 [442:58:41<49:23, 19.76s/it]
100%|█████████▉| 224851/225000 [442:59:03<50:29, 20.33s/it]
100%|█████████▉| 224852/225000 [442:59:24<50:51, 20.62s/it]
100%|█████████▉| 224853/225000 [442:59:43<49:07, 20.05s/it]
100%|█████████▉| 224854/225000 [443:00:00<47:04, 19.35s/it]
100%|█████████▉| 224855/225000 [443:00:23<49:14, 20.38s/it]
100%|█████████▉| 224856/225000 [443:00:47<51:15, 21.36s/it]
100%|█████████▉| 224857/225000 [443:00:58<43:55, 18.43s/it]
100%|█████████▉| 224858/225000 [443:01:14<41:55, 17.72s/it]
100%|█████████▉| 224859/225000 [443:01:41<48:08, 20.48s/it]
100%|█████████▉| 224860/225000 [443:02:00<46:30, 19.93s/it]
100%|█████████▉| 224861/225000 [443:02:19<45:15, 19.54s/it]
100%|█████████▉| 224862/225000 [443:02:46<50:08, 21.80s/it]
100%|█████████▉| 224863/225000 [443:03:03<47:03, 20.61s/it]
100%|█████████▉| 224864/225000 [443:03:26<48:05, 21.21s/it]
100%|█████████▉| 224865/225000 [443:03:48<48:00, 21.34s/it]
100%|█████████▉| 224866/225000 [443:04:04<43:56, 19.67s/it]
100%|█████████▉| 224867/225000 [443:04:29<47:28, 21.42s/it]
100%|█████████▉| 224868/225000 [443:04:51<47:42, 21.69s/it]
100%|█████████▉| 224869/225000 [443:05:20<52:07, 23.88s/it]
100%|█████████▉| 224870/225000 [443:05:38<47:42, 22.02s/it]
100%|█████████▉| 224871/225000 [443:05:56<44:31, 20.71s/it]
100%|█████████▉| 224872/225000 [443:06:11<40:56, 19.19s/it]
100%|█████████▉| 224873/225000 [443:06:33<42:06, 19.89s/it]
100%|█████████▉| 224874/225000 [443:06:49<39:37, 18.87s/it]
100%|█████████▉| 224875/225000 [443:07:08<39:12, 18.82s/it]
100%|█████████▉| 224876/225000 [443:07:26<38:07, 18.45s/it]
100%|█████████▉| 224877/225000 [443:07:41<35:45, 17.45s/it]
100%|█████████▉| 224878/225000 [443:08:07<40:49, 20.08s/it]
100%|█████████▉| 224879/225000 [443:08:19<35:25, 17.56s/it]
100%|█████████▉| 224880/225000 [443:08:36<34:47, 17.40s/it]
100%|█████████▉| 224881/225000 [443:09:00<38:55, 19.63s/it]
100%|█████████▉| 224882/225000 [443:09:25<41:32, 21.12s/it]
100%|█████████▉| 224883/225000 [443:09:49<42:40, 21.88s/it]
100%|█████████▉| 224884/225000 [443:10:10<42:02, 21.75s/it]
100%|█████████▉| 224885/225000 [443:10:35<43:25, 22.65s/it]
100%|█████████▉| 224886/225000 [443:10:49<38:19, 20.17s/it]
100%|█████████▉| 224887/225000 [443:11:01<33:02, 17.54s/it]
100%|█████████▉| 224888/225000 [443:11:20<33:45, 18.08s/it]
100%|█████████▉| 224889/225000 [443:11:37<32:43, 17.68s/it]
100%|█████████▉| 224890/225000 [443:11:50<29:52, 16.29s/it]
100%|█████████▉| 224891/225000 [443:12:05<28:58, 15.95s/it]
100%|█████████▉| 224892/225000 [443:12:18<27:04, 15.04s/it]
100%|█████████▉| 224893/225000 [443:12:39<30:02, 16.84s/it]
100%|█████████▉| 224894/225000 [443:12:55<29:13, 16.54s/it]
100%|█████████▉| 224895/225000 [443:13:10<28:07, 16.07s/it]
100%|█████████▉| 224896/225000 [443:13:27<28:22, 16.37s/it]
100%|█████████▉| 224897/225000 [443:13:52<32:28, 18.91s/it]
100%|█████████▉| 224898/225000 [443:14:10<31:49, 18.72s/it]
100%|█████████▉| 224899/225000 [443:14:28<30:55, 18.37s/it]
100%|█████████▉| 224900/225000 [443:14:43<29:14, 17.55s/it]
                                                           
{'loss': '1.222', 'grad_norm': '1.434', 'learning_rate': '1.133e-10', 'epoch': '1.001', 'num_input_tokens_seen': 757915072, 'train_runtime': '1.596e+06', 'train_tokens_per_second': '475'}

100%|█████████▉| 224900/225000 [443:14:43<29:14, 17.55s/it]
100%|█████████▉| 224901/225000 [443:14:57<27:19, 16.56s/it]
100%|█████████▉| 224902/225000 [443:15:17<28:33, 17.49s/it]
100%|█████████▉| 224903/225000 [443:15:28<25:03, 15.50s/it]
100%|█████████▉| 224904/225000 [443:15:47<26:27, 16.53s/it]
100%|█████████▉| 224905/225000 [443:16:15<31:37, 19.98s/it]
100%|█████████▉| 224906/225000 [443:16:34<31:00, 19.79s/it]
100%|█████████▉| 224907/225000 [443:16:49<28:28, 18.37s/it]
100%|█████████▉| 224908/225000 [443:17:02<25:36, 16.70s/it]
100%|█████████▉| 224909/225000 [443:17:21<26:30, 17.48s/it]
100%|█████████▉| 224910/225000 [443:17:42<27:24, 18.28s/it]
100%|█████████▉| 224911/225000 [443:18:04<28:57, 19.52s/it]
100%|█████████▉| 224912/225000 [443:18:27<30:12, 20.60s/it]
100%|█████████▉| 224913/225000 [443:18:46<29:03, 20.04s/it]
100%|█████████▉| 224914/225000 [443:19:04<27:54, 19.47s/it]
100%|█████████▉| 224915/225000 [443:19:28<29:18, 20.69s/it]
100%|█████████▉| 224916/225000 [443:19:43<26:54, 19.22s/it]
100%|█████████▉| 224917/225000 [443:20:01<26:07, 18.88s/it]
100%|█████████▉| 224918/225000 [443:20:29<29:11, 21.36s/it]
100%|█████████▉| 224919/225000 [443:20:43<25:51, 19.15s/it]
100%|█████████▉| 224920/225000 [443:20:59<24:29, 18.37s/it]
100%|█████████▉| 224921/225000 [443:21:20<25:12, 19.14s/it]
100%|█████████▉| 224922/225000 [443:21:43<26:11, 20.14s/it]
100%|█████████▉| 224923/225000 [443:22:01<25:15, 19.69s/it]
100%|█████████▉| 224924/225000 [443:22:29<27:51, 22.00s/it]
100%|█████████▉| 224925/225000 [443:22:48<26:27, 21.17s/it]
100%|█████████▉| 224926/225000 [443:23:02<23:41, 19.21s/it]
100%|█████████▉| 224927/225000 [443:23:27<25:24, 20.88s/it]
100%|█████████▉| 224928/225000 [443:23:55<27:32, 22.95s/it]
100%|█████████▉| 224929/225000 [443:24:09<24:09, 20.42s/it]
100%|█████████▉| 224930/225000 [443:24:32<24:34, 21.06s/it]
100%|█████████▉| 224931/225000 [443:24:54<24:38, 21.43s/it]
100%|█████████▉| 224932/225000 [443:25:14<23:48, 21.01s/it]
100%|█████████▉| 224933/225000 [443:25:34<23:09, 20.74s/it]
100%|█████████▉| 224934/225000 [443:25:47<20:09, 18.33s/it]
100%|█████████▉| 224935/225000 [443:26:03<19:03, 17.59s/it]
100%|█████████▉| 224936/225000 [443:26:25<20:12, 18.94s/it]
100%|█████████▉| 224937/225000 [443:26:43<19:37, 18.68s/it]
100%|█████████▉| 224938/225000 [443:27:05<20:07, 19.47s/it]
100%|█████████▉| 224939/225000 [443:27:16<17:20, 17.06s/it]
100%|█████████▉| 224940/225000 [443:27:46<20:53, 20.90s/it]
100%|█████████▉| 224941/225000 [443:28:00<18:32, 18.86s/it]
100%|█████████▉| 224942/225000 [443:28:25<19:59, 20.68s/it]
100%|█████████▉| 224943/225000 [443:28:43<18:49, 19.82s/it]
100%|█████████▉| 224944/225000 [443:29:02<18:15, 19.56s/it]
100%|█████████▉| 224945/225000 [443:29:23<18:32, 20.22s/it]
100%|█████████▉| 224946/225000 [443:29:42<17:39, 19.61s/it]
100%|█████████▉| 224947/225000 [443:30:04<18:10, 20.57s/it]
100%|█████████▉| 224948/225000 [443:30:22<17:06, 19.73s/it]
100%|█████████▉| 224949/225000 [443:30:39<16:07, 18.96s/it]
100%|█████████▉| 224950/225000 [443:31:00<16:11, 19.43s/it]
                                                           
{'loss': '1.227', 'grad_norm': '0.8929', 'learning_rate': '6.495e-11', 'epoch': '1.002', 'num_input_tokens_seen': 758426728, 'train_runtime': '1.597e+06', 'train_tokens_per_second': '475'}

100%|█████████▉| 224950/225000 [443:31:00<16:11, 19.43s/it]
100%|█████████▉| 224951/225000 [443:31:13<14:26, 17.68s/it]
100%|█████████▉| 224952/225000 [443:31:37<15:29, 19.37s/it]
100%|█████████▉| 224953/225000 [443:31:55<14:51, 18.96s/it]
100%|█████████▉| 224954/225000 [443:32:11<14:00, 18.27s/it]
100%|█████████▉| 224955/225000 [443:32:32<14:09, 18.87s/it]
100%|█████████▉| 224956/225000 [443:32:50<13:38, 18.61s/it]
100%|█████████▉| 224957/225000 [443:33:07<13:06, 18.30s/it]
100%|█████████▉| 224958/225000 [443:33:18<11:10, 15.97s/it]
100%|█████████▉| 224959/225000 [443:33:35<11:14, 16.44s/it]
100%|█████████▉| 224960/225000 [443:33:43<09:16, 13.90s/it]
100%|█████████▉| 224961/225000 [443:34:03<10:14, 15.77s/it]
100%|█████████▉| 224962/225000 [443:34:29<11:51, 18.72s/it]
100%|█████████▉| 224963/225000 [443:34:48<11:32, 18.72s/it]
100%|█████████▉| 224964/225000 [443:35:11<12:06, 20.17s/it]
100%|█████████▉| 224965/225000 [443:35:27<10:58, 18.81s/it]
100%|█████████▉| 224966/225000 [443:35:48<11:01, 19.46s/it]
100%|█████████▉| 224967/225000 [443:36:03<09:59, 18.17s/it]
100%|█████████▉| 224968/225000 [443:36:27<10:32, 19.77s/it]
100%|█████████▉| 224969/225000 [443:36:44<09:54, 19.19s/it]
100%|█████████▉| 224970/225000 [443:37:10<10:30, 21.01s/it]
100%|█████████▉| 224971/225000 [443:37:33<10:27, 21.65s/it]
100%|█████████▉| 224972/225000 [443:37:55<10:07, 21.71s/it]
100%|█████████▉| 224973/225000 [443:38:14<09:30, 21.13s/it]
100%|█████████▉| 224974/225000 [443:38:37<09:16, 21.41s/it]
100%|█████████▉| 224975/225000 [443:39:00<09:13, 22.16s/it]
100%|█████████▉| 224976/225000 [443:39:15<07:56, 19.83s/it]
100%|█████████▉| 224977/225000 [443:39:29<06:59, 18.22s/it]
100%|█████████▉| 224978/225000 [443:39:57<07:41, 20.99s/it]
100%|█████████▉| 224979/225000 [443:40:14<06:58, 19.93s/it]
100%|█████████▉| 224980/225000 [443:40:35<06:42, 20.11s/it]
100%|█████████▉| 224981/225000 [443:40:52<06:03, 19.15s/it]
100%|█████████▉| 224982/225000 [443:41:16<06:12, 20.70s/it]
100%|█████████▉| 224983/225000 [443:41:35<05:41, 20.08s/it]
100%|█████████▉| 224984/225000 [443:42:00<05:45, 21.59s/it]
100%|█████████▉| 224985/225000 [443:42:10<04:34, 18.28s/it]
100%|█████████▉| 224986/225000 [443:42:32<04:28, 19.21s/it]
100%|█████████▉| 224987/225000 [443:42:46<03:49, 17.62s/it]
100%|█████████▉| 224988/225000 [443:43:10<03:55, 19.65s/it]
100%|█████████▉| 224989/225000 [443:43:25<03:22, 18.37s/it]
100%|█████████▉| 224990/225000 [443:43:50<03:21, 20.12s/it]
100%|█████████▉| 224991/225000 [443:44:11<03:03, 20.42s/it]
100%|█████████▉| 224992/225000 [443:44:35<02:51, 21.46s/it]
100%|█████████▉| 224993/225000 [443:44:54<02:26, 20.91s/it]
100%|█████████▉| 224994/225000 [443:45:05<01:47, 17.92s/it]
100%|█████████▉| 224995/225000 [443:45:34<01:46, 21.30s/it]
100%|█████████▉| 224996/225000 [443:45:56<01:26, 21.50s/it]
100%|█████████▉| 224997/225000 [443:46:09<00:56, 18.99s/it]
100%|█████████▉| 224998/225000 [443:46:31<00:39, 19.70s/it]
100%|█████████▉| 224999/225000 [443:46:58<00:21, 21.92s/it]
100%|██████████| 225000/225000 [443:47:17<00:00, 21.14s/it]
                                                           
{'loss': '1.234', 'grad_norm': '1.158', 'learning_rate': '2.999e-11', 'epoch': '1.002', 'num_input_tokens_seen': 758936576, 'train_runtime': '1.598e+06', 'train_tokens_per_second': '475'}

100%|██████████| 225000/225000 [443:47:17<00:00, 21.14s/it]
***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)

                                                           
{'eval_loss': '1.221', 'eval_runtime': '430.3', 'eval_samples_per_second': '1.162', 'eval_steps_per_second': '0.581', 'epoch': '1.002', 'num_input_tokens_seen': 758936576}

100%|██████████| 225000/225000 [443:54:27<00:00, 21.14s/it]Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/checkpoint-225000
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}



Training completed. Do not forget to share your model on huggingface.co/models =)



                                                           
{'train_runtime': '1.598e+06', 'train_samples_per_second': '2.253', 'train_steps_per_second': '0.141', 'train_loss': '0.8487', 'epoch': '1.002', 'num_input_tokens_seen': 758936576}

100%|██████████| 225000/225000 [443:54:29<00:00, 21.14s/it]
100%|██████████| 225000/225000 [443:54:29<00:00,  7.10s/it]
Saving model checkpoint to /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k
loading configuration file config.json from cache at /home/schwarc/.cache/huggingface/hub/models--slovak-nlp--mistral-sk-7b/snapshots/089497ae0d72018e9591895c54f774bf0b4a83a4/config.json
Model config MistralConfig {
  "architectures": [
    "MistralForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "dtype": "bfloat16",
  "eos_token_id": 2,
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 32768,
  "model_type": "mistral",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pad_token_id": 2,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "sliding_window": 4096,
  "tie_word_embeddings": false,
  "transformers_version": "5.5.0",
  "use_cache": true,
  "vocab_size": 32000
}

***** train metrics *****
  epoch                    =               1.0019
  num_input_tokens_seen    =            758936576
  total_flos               =        30333393085GF
  train_loss               =               0.8487
  train_runtime            = 18 days, 11:54:29.31
  train_samples_per_second =                2.253
  train_steps_per_second   =                0.141
Figure saved at: /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/training_loss.png
Figure saved at: /home/schwarc/diplomovka/aya_streaming_lora_unsloth_2gpu_dispatchfalse_225k/training_eval_loss.png
[WARNING|2026-09-17 16:16:12] llamafactory.extras.ploting:149 >> No metric eval_accuracy to plot.

***** Running Evaluation *****
  Num examples: Unknown
  Batch size = 1
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:71: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:281: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
/home/schwarc/miniconda3/envs/llama_factory/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py:254: FutureWarning: The attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.
  warnings.warn(DEPRECATION_MESSAGE, FutureWarning)
***** eval metrics *****
  epoch                   =     1.0019
  eval_loss               =      1.221
  eval_runtime            = 0:07:06.92
  eval_samples_per_second =      1.171
  eval_steps_per_second   =      0.586
  num_input_tokens_seen   =  758936576
Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}